[2023-12-26 15:21:50,170][104569] Saving configuration to ./train_mujoco/mujoco_doublependulum_APPO/config.json... [2023-12-26 15:21:51,173][104569] Rollout worker 0 uses device cpu [2023-12-26 15:21:51,174][104569] Rollout worker 1 uses device cpu [2023-12-26 15:21:51,175][104569] Rollout worker 2 uses device cpu [2023-12-26 15:21:51,175][104569] Rollout worker 3 uses device cpu [2023-12-26 15:21:51,176][104569] Rollout worker 4 uses device cpu [2023-12-26 15:21:51,176][104569] Rollout worker 5 uses device cpu [2023-12-26 15:21:51,177][104569] Rollout worker 6 uses device cpu [2023-12-26 15:21:51,177][104569] Rollout worker 7 uses device cpu [2023-12-26 15:21:51,177][104569] Rollout worker 8 uses device cpu [2023-12-26 15:21:51,178][104569] Rollout worker 9 uses device cpu [2023-12-26 15:21:51,178][104569] Rollout worker 10 uses device cpu [2023-12-26 15:21:51,179][104569] Rollout worker 11 uses device cpu [2023-12-26 15:21:51,179][104569] Rollout worker 12 uses device cpu [2023-12-26 15:21:51,180][104569] Rollout worker 13 uses device cpu [2023-12-26 15:21:51,180][104569] Rollout worker 14 uses device cpu [2023-12-26 15:21:51,181][104569] Rollout worker 15 uses device cpu [2023-12-26 15:21:51,209][104569] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-12-26 15:21:51,210][104569] InferenceWorker_p0-w0: min num requests: 2 [2023-12-26 15:21:51,213][104569] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-12-26 15:21:51,213][104569] InferenceWorker_p1-w0: min num requests: 2 [2023-12-26 15:21:51,265][104569] Starting all processes... [2023-12-26 15:21:51,268][104569] Starting process learner_proc0 [2023-12-26 15:21:51,271][104569] Starting process learner_proc1 [2023-12-26 15:21:51,315][104569] Starting all processes... [2023-12-26 15:21:51,322][104569] Starting process inference_proc0-0 [2023-12-26 15:21:51,323][104569] Starting process inference_proc1-0 [2023-12-26 15:21:51,323][104569] Starting process rollout_proc0 [2023-12-26 15:21:51,323][104569] Starting process rollout_proc1 [2023-12-26 15:21:51,323][104569] Starting process rollout_proc2 [2023-12-26 15:21:51,324][104569] Starting process rollout_proc3 [2023-12-26 15:21:51,326][104569] Starting process rollout_proc4 [2023-12-26 15:21:51,327][104569] Starting process rollout_proc5 [2023-12-26 15:21:51,329][104569] Starting process rollout_proc6 [2023-12-26 15:21:51,331][104569] Starting process rollout_proc7 [2023-12-26 15:21:51,333][104569] Starting process rollout_proc8 [2023-12-26 15:21:51,333][104569] Starting process rollout_proc9 [2023-12-26 15:21:51,333][104569] Starting process rollout_proc10 [2023-12-26 15:21:51,334][104569] Starting process rollout_proc11 [2023-12-26 15:21:51,334][104569] Starting process rollout_proc12 [2023-12-26 15:21:51,334][104569] Starting process rollout_proc13 [2023-12-26 15:21:51,334][104569] Starting process rollout_proc14 [2023-12-26 15:21:51,349][104569] Starting process rollout_proc15 [2023-12-26 15:21:53,952][105702] Worker 5 uses CPU cores [10, 11] [2023-12-26 15:21:53,969][105698] Worker 2 uses CPU cores [4, 5] [2023-12-26 15:21:53,991][105585] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-12-26 15:21:53,991][105585] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2023-12-26 15:21:54,008][105701] Worker 4 uses CPU cores [8, 9] [2023-12-26 15:21:54,043][105585] Num visible devices: 1 [2023-12-26 15:21:54,083][105707] Worker 6 uses CPU cores [12, 13] [2023-12-26 15:21:54,085][105585] Setting fixed seed 1234 [2023-12-26 15:21:54,087][105585] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-12-26 15:21:54,088][105585] Initializing actor-critic model on device cuda:0 [2023-12-26 15:21:54,088][105585] RunningMeanStd input shape: (11,) [2023-12-26 15:21:54,089][105585] RunningMeanStd input shape: (1,) [2023-12-26 15:21:54,118][105699] Worker 3 uses CPU cores [6, 7] [2023-12-26 15:21:54,138][105585] Created Actor Critic model with architecture: [2023-12-26 15:21:54,138][105585] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): MlpEncoder( (mlp_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=Tanh) (2): RecursiveScriptModule(original_name=Linear) (3): RecursiveScriptModule(original_name=Tanh) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=64, out_features=1, bias=True) (action_parameterization): ActionParameterizationContinuousNonAdaptiveStddev( (distribution_linear): Linear(in_features=64, out_features=1, bias=True) ) ) [2023-12-26 15:21:54,141][105620] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-12-26 15:21:54,142][105620] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for inference process 1 [2023-12-26 15:21:54,200][105620] Num visible devices: 1 [2023-12-26 15:21:54,245][105765] Worker 13 uses CPU cores [26, 27] [2023-12-26 15:21:54,293][105726] Worker 14 uses CPU cores [28, 29] [2023-12-26 15:21:54,320][105727] Worker 15 uses CPU cores [30, 31] [2023-12-26 15:21:54,396][105692] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-12-26 15:21:54,396][105692] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2023-12-26 15:21:54,398][105688] Worker 0 uses CPU cores [0, 1] [2023-12-26 15:21:54,409][105718] Worker 8 uses CPU cores [16, 17] [2023-12-26 15:21:54,409][105724] Worker 11 uses CPU cores [22, 23] [2023-12-26 15:21:54,436][105692] Num visible devices: 1 [2023-12-26 15:21:54,481][105700] Worker 1 uses CPU cores [2, 3] [2023-12-26 15:21:54,543][105586] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-12-26 15:21:54,543][105586] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for learning process 1 [2023-12-26 15:21:54,589][105586] Num visible devices: 1 [2023-12-26 15:21:54,594][105715] Worker 7 uses CPU cores [14, 15] [2023-12-26 15:21:54,611][105586] Setting fixed seed 1234 [2023-12-26 15:21:54,612][105586] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-12-26 15:21:54,612][105586] Initializing actor-critic model on device cuda:0 [2023-12-26 15:21:54,613][105586] RunningMeanStd input shape: (11,) [2023-12-26 15:21:54,613][105586] RunningMeanStd input shape: (1,) [2023-12-26 15:21:54,647][105586] Created Actor Critic model with architecture: [2023-12-26 15:21:54,647][105586] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): MlpEncoder( (mlp_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=Tanh) (2): RecursiveScriptModule(original_name=Linear) (3): RecursiveScriptModule(original_name=Tanh) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=64, out_features=1, bias=True) (action_parameterization): ActionParameterizationContinuousNonAdaptiveStddev( (distribution_linear): Linear(in_features=64, out_features=1, bias=True) ) ) [2023-12-26 15:21:54,673][105723] Worker 9 uses CPU cores [18, 19] [2023-12-26 15:21:54,715][105728] Worker 12 uses CPU cores [24, 25] [2023-12-26 15:21:54,717][105725] Worker 10 uses CPU cores [20, 21] [2023-12-26 15:21:54,782][105585] Using optimizer [2023-12-26 15:21:54,783][105585] No checkpoints found [2023-12-26 15:21:54,783][105585] Did not load from checkpoint, starting from scratch! [2023-12-26 15:21:54,783][105585] Initialized policy 0 weights for model version 0 [2023-12-26 15:21:54,784][105585] LearnerWorker_p0 finished initialization! [2023-12-26 15:21:54,785][105585] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-12-26 15:21:55,278][105586] Using optimizer [2023-12-26 15:21:55,278][105586] No checkpoints found [2023-12-26 15:21:55,279][105586] Did not load from checkpoint, starting from scratch! [2023-12-26 15:21:55,279][105586] Initialized policy 1 weights for model version 0 [2023-12-26 15:21:55,280][105586] LearnerWorker_p1 finished initialization! [2023-12-26 15:21:55,281][105586] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-12-26 15:21:55,508][105692] RunningMeanStd input shape: (11,) [2023-12-26 15:21:55,509][105692] RunningMeanStd input shape: (1,) [2023-12-26 15:21:55,544][104569] Inference worker 0-0 is ready! [2023-12-26 15:21:55,871][105620] RunningMeanStd input shape: (11,) [2023-12-26 15:21:55,871][105620] RunningMeanStd input shape: (1,) [2023-12-26 15:21:55,905][104569] Inference worker 1-0 is ready! [2023-12-26 15:21:55,906][104569] All inference workers are ready! Signal rollout workers to start! [2023-12-26 15:21:55,907][105727] EnvRunner 15-0 uses policy 1 [2023-12-26 15:21:55,907][105715] EnvRunner 7-0 uses policy 1 [2023-12-26 15:21:55,907][105718] EnvRunner 8-0 uses policy 0 [2023-12-26 15:21:55,907][105701] EnvRunner 4-0 uses policy 0 [2023-12-26 15:21:55,907][105702] EnvRunner 5-0 uses policy 1 [2023-12-26 15:21:55,907][105724] EnvRunner 11-0 uses policy 1 [2023-12-26 15:21:55,907][105765] EnvRunner 13-0 uses policy 1 [2023-12-26 15:21:55,907][105688] EnvRunner 0-0 uses policy 0 [2023-12-26 15:21:55,907][105723] EnvRunner 9-0 uses policy 1 [2023-12-26 15:21:55,907][105707] EnvRunner 6-0 uses policy 0 [2023-12-26 15:21:55,907][105728] EnvRunner 12-0 uses policy 0 [2023-12-26 15:21:55,907][105726] EnvRunner 14-0 uses policy 0 [2023-12-26 15:21:55,907][105725] EnvRunner 10-0 uses policy 0 [2023-12-26 15:21:55,907][105698] EnvRunner 2-0 uses policy 0 [2023-12-26 15:21:55,907][105700] EnvRunner 1-0 uses policy 1 [2023-12-26 15:21:55,907][105699] EnvRunner 3-0 uses policy 1 [2023-12-26 15:21:55,998][105701] EnvRunner 4-1 uses policy 0 [2023-12-26 15:21:56,001][105718] EnvRunner 8-1 uses policy 0 [2023-12-26 15:21:56,002][105688] EnvRunner 0-1 uses policy 0 [2023-12-26 15:21:56,004][105728] EnvRunner 12-1 uses policy 0 [2023-12-26 15:21:56,004][105727] EnvRunner 15-1 uses policy 1 [2023-12-26 15:21:56,018][105715] EnvRunner 7-1 uses policy 1 [2023-12-26 15:21:56,029][105765] EnvRunner 13-1 uses policy 1 [2023-12-26 15:21:56,033][105724] EnvRunner 11-1 uses policy 1 [2023-12-26 15:21:56,042][105702] EnvRunner 5-1 uses policy 1 [2023-12-26 15:21:56,050][105707] EnvRunner 6-1 uses policy 0 [2023-12-26 15:21:56,050][105726] EnvRunner 14-1 uses policy 0 [2023-12-26 15:21:56,051][105699] EnvRunner 3-1 uses policy 1 [2023-12-26 15:21:56,056][105723] EnvRunner 9-1 uses policy 1 [2023-12-26 15:21:56,056][105700] EnvRunner 1-1 uses policy 1 [2023-12-26 15:21:56,057][105725] EnvRunner 10-1 uses policy 0 [2023-12-26 15:21:56,057][105698] EnvRunner 2-1 uses policy 0 [2023-12-26 15:21:56,062][104569] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan, 1: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-12-26 15:21:58,265][105585] Signal inference workers to stop experience collection... [2023-12-26 15:21:58,271][105692] InferenceWorker_p0-w0: stopping experience collection [2023-12-26 15:21:58,271][105620] InferenceWorker_p1-w0: stopping experience collection [2023-12-26 15:21:58,828][105585] Signal inference workers to resume experience collection... [2023-12-26 15:21:58,829][105692] InferenceWorker_p0-w0: resuming experience collection [2023-12-26 15:21:58,829][105620] InferenceWorker_p1-w0: resuming experience collection [2023-12-26 15:21:58,894][105620] Updated weights for policy 1, policy_version 23 (0.0007) [2023-12-26 15:21:59,904][105586] Signal inference workers to stop experience collection... [2023-12-26 15:22:00,079][105586] Signal inference workers to resume experience collection... [2023-12-26 15:22:00,082][105620] Updated weights for policy 1, policy_version 64 (0.0006) [2023-12-26 15:22:00,607][105620] Updated weights for policy 1, policy_version 74 (0.0009) [2023-12-26 15:22:00,669][105620] Updated weights for policy 1, policy_version 84 (0.0009) [2023-12-26 15:22:00,725][105620] Updated weights for policy 1, policy_version 94 (0.0008) [2023-12-26 15:22:00,850][105692] Updated weights for policy 0, policy_version 71 (0.0359) [2023-12-26 15:22:00,903][105692] Updated weights for policy 0, policy_version 81 (0.0009) [2023-12-26 15:22:00,958][105692] Updated weights for policy 0, policy_version 91 (0.0010) [2023-12-26 15:22:01,031][105692] Updated weights for policy 0, policy_version 104 (0.0010) [2023-12-26 15:22:01,062][104569] Fps is (10 sec: 9830.6, 60 sec: 9830.6, 300 sec: 9830.6). Total num frames: 49152. Throughput: 0: 3470.5, 1: 3204.9. Samples: 33376. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-26 15:22:01,062][104569] Avg episode reward: [(0, '63.359'), (1, '62.264')] [2023-12-26 15:22:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000000096_24576.pth... [2023-12-26 15:22:01,096][105692] Updated weights for policy 0, policy_version 114 (0.0009) [2023-12-26 15:22:01,148][105620] Updated weights for policy 1, policy_version 104 (0.0009) [2023-12-26 15:22:01,162][105692] Updated weights for policy 0, policy_version 124 (0.0008) [2023-12-26 15:22:01,183][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000000128_32768.pth... [2023-12-26 15:22:01,196][104569] Heartbeat connected on Batcher_0 [2023-12-26 15:22:01,198][105620] Updated weights for policy 1, policy_version 114 (0.0008) [2023-12-26 15:22:01,199][104569] Heartbeat connected on LearnerWorker_p0 [2023-12-26 15:22:01,202][104569] Heartbeat connected on Batcher_1 [2023-12-26 15:22:01,213][104569] Heartbeat connected on InferenceWorker_p0-w0 [2023-12-26 15:22:01,219][104569] Heartbeat connected on RolloutWorker_w0 [2023-12-26 15:22:01,220][104569] Heartbeat connected on RolloutWorker_w1 [2023-12-26 15:22:01,223][104569] Heartbeat connected on InferenceWorker_p1-w0 [2023-12-26 15:22:01,226][104569] Heartbeat connected on RolloutWorker_w2 [2023-12-26 15:22:01,226][104569] Heartbeat connected on RolloutWorker_w3 [2023-12-26 15:22:01,230][104569] Heartbeat connected on RolloutWorker_w4 [2023-12-26 15:22:01,234][104569] Heartbeat connected on RolloutWorker_w5 [2023-12-26 15:22:01,236][104569] Heartbeat connected on RolloutWorker_w6 [2023-12-26 15:22:01,240][104569] Heartbeat connected on RolloutWorker_w7 [2023-12-26 15:22:01,244][104569] Heartbeat connected on RolloutWorker_w8 [2023-12-26 15:22:01,245][104569] Heartbeat connected on RolloutWorker_w9 [2023-12-26 15:22:01,251][104569] Heartbeat connected on RolloutWorker_w10 [2023-12-26 15:22:01,253][104569] Heartbeat connected on RolloutWorker_w11 [2023-12-26 15:22:01,256][104569] Heartbeat connected on RolloutWorker_w12 [2023-12-26 15:22:01,257][105620] Updated weights for policy 1, policy_version 124 (0.0010) [2023-12-26 15:22:01,258][104569] Heartbeat connected on RolloutWorker_w13 [2023-12-26 15:22:01,264][104569] Heartbeat connected on RolloutWorker_w15 [2023-12-26 15:22:01,265][104569] Heartbeat connected on RolloutWorker_w14 [2023-12-26 15:22:01,285][104569] Heartbeat connected on LearnerWorker_p1 [2023-12-26 15:22:01,941][105692] Updated weights for policy 0, policy_version 134 (0.0007) [2023-12-26 15:22:02,011][105692] Updated weights for policy 0, policy_version 144 (0.0007) [2023-12-26 15:22:02,066][105692] Updated weights for policy 0, policy_version 154 (0.0008) [2023-12-26 15:22:02,080][105620] Updated weights for policy 1, policy_version 134 (0.0009) [2023-12-26 15:22:02,135][105620] Updated weights for policy 1, policy_version 144 (0.0010) [2023-12-26 15:22:02,195][105620] Updated weights for policy 1, policy_version 154 (0.0010) [2023-12-26 15:22:02,778][105692] Updated weights for policy 0, policy_version 164 (0.0005) [2023-12-26 15:22:02,836][105692] Updated weights for policy 0, policy_version 174 (0.0006) [2023-12-26 15:22:02,891][105692] Updated weights for policy 0, policy_version 184 (0.0008) [2023-12-26 15:22:03,016][105620] Updated weights for policy 1, policy_version 164 (0.0009) [2023-12-26 15:22:03,071][105620] Updated weights for policy 1, policy_version 174 (0.0008) [2023-12-26 15:22:03,133][105620] Updated weights for policy 1, policy_version 184 (0.0010) [2023-12-26 15:22:03,624][105692] Updated weights for policy 0, policy_version 194 (0.0009) [2023-12-26 15:22:03,684][105692] Updated weights for policy 0, policy_version 204 (0.0009) [2023-12-26 15:22:03,735][105692] Updated weights for policy 0, policy_version 214 (0.0009) [2023-12-26 15:22:03,787][105692] Updated weights for policy 0, policy_version 224 (0.0009) [2023-12-26 15:22:03,907][105620] Updated weights for policy 1, policy_version 194 (0.0009) [2023-12-26 15:22:03,961][105620] Updated weights for policy 1, policy_version 204 (0.0009) [2023-12-26 15:22:04,010][105620] Updated weights for policy 1, policy_version 214 (0.0009) [2023-12-26 15:22:04,077][105620] Updated weights for policy 1, policy_version 224 (0.0009) [2023-12-26 15:22:04,573][105692] Updated weights for policy 0, policy_version 234 (0.0010) [2023-12-26 15:22:04,626][105692] Updated weights for policy 0, policy_version 244 (0.0009) [2023-12-26 15:22:04,682][105692] Updated weights for policy 0, policy_version 254 (0.0009) [2023-12-26 15:22:04,872][105620] Updated weights for policy 1, policy_version 234 (0.0008) [2023-12-26 15:22:04,930][105620] Updated weights for policy 1, policy_version 244 (0.0009) [2023-12-26 15:22:04,977][105620] Updated weights for policy 1, policy_version 254 (0.0009) [2023-12-26 15:22:05,516][105692] Updated weights for policy 0, policy_version 264 (0.0008) [2023-12-26 15:22:05,573][105692] Updated weights for policy 0, policy_version 274 (0.0009) [2023-12-26 15:22:05,629][105692] Updated weights for policy 0, policy_version 284 (0.0009) [2023-12-26 15:22:05,754][105620] Updated weights for policy 1, policy_version 264 (0.0010) [2023-12-26 15:22:05,813][105620] Updated weights for policy 1, policy_version 274 (0.0009) [2023-12-26 15:22:05,861][105620] Updated weights for policy 1, policy_version 284 (0.0009) [2023-12-26 15:22:06,062][104569] Fps is (10 sec: 14745.8, 60 sec: 14745.8, 300 sec: 14745.8). Total num frames: 147456. Throughput: 0: 6785.3, 1: 6716.1. Samples: 135012. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-12-26 15:22:06,062][104569] Avg episode reward: [(0, '94.537'), (1, '89.239')] [2023-12-26 15:22:06,390][105692] Updated weights for policy 0, policy_version 294 (0.0009) [2023-12-26 15:22:06,443][105692] Updated weights for policy 0, policy_version 304 (0.0009) [2023-12-26 15:22:06,497][105692] Updated weights for policy 0, policy_version 314 (0.0009) [2023-12-26 15:22:06,653][105620] Updated weights for policy 1, policy_version 294 (0.0010) [2023-12-26 15:22:06,710][105620] Updated weights for policy 1, policy_version 304 (0.0009) [2023-12-26 15:22:06,766][105620] Updated weights for policy 1, policy_version 314 (0.0009) [2023-12-26 15:22:07,284][105692] Updated weights for policy 0, policy_version 324 (0.0009) [2023-12-26 15:22:07,345][105692] Updated weights for policy 0, policy_version 334 (0.0009) [2023-12-26 15:22:07,405][105692] Updated weights for policy 0, policy_version 344 (0.0010) [2023-12-26 15:22:07,569][105620] Updated weights for policy 1, policy_version 324 (0.0009) [2023-12-26 15:22:07,628][105620] Updated weights for policy 1, policy_version 334 (0.0009) [2023-12-26 15:22:07,685][105620] Updated weights for policy 1, policy_version 344 (0.0009) [2023-12-26 15:22:08,191][105692] Updated weights for policy 0, policy_version 354 (0.0009) [2023-12-26 15:22:08,250][105692] Updated weights for policy 0, policy_version 364 (0.0009) [2023-12-26 15:22:08,316][105692] Updated weights for policy 0, policy_version 374 (0.0010) [2023-12-26 15:22:08,381][105692] Updated weights for policy 0, policy_version 384 (0.0009) [2023-12-26 15:22:08,503][105620] Updated weights for policy 1, policy_version 354 (0.0009) [2023-12-26 15:22:08,570][105620] Updated weights for policy 1, policy_version 364 (0.0010) [2023-12-26 15:22:08,634][105620] Updated weights for policy 1, policy_version 374 (0.0009) [2023-12-26 15:22:08,697][105620] Updated weights for policy 1, policy_version 384 (0.0008) [2023-12-26 15:22:09,148][105692] Updated weights for policy 0, policy_version 394 (0.0010) [2023-12-26 15:22:09,224][105692] Updated weights for policy 0, policy_version 404 (0.0009) [2023-12-26 15:22:09,294][105692] Updated weights for policy 0, policy_version 414 (0.0007) [2023-12-26 15:22:09,436][105620] Updated weights for policy 1, policy_version 394 (0.0008) [2023-12-26 15:22:09,499][105620] Updated weights for policy 1, policy_version 404 (0.0010) [2023-12-26 15:22:09,569][105620] Updated weights for policy 1, policy_version 414 (0.0006) [2023-12-26 15:22:10,039][105692] Updated weights for policy 0, policy_version 424 (0.0007) [2023-12-26 15:22:10,104][105692] Updated weights for policy 0, policy_version 434 (0.0007) [2023-12-26 15:22:10,159][105692] Updated weights for policy 0, policy_version 444 (0.0009) [2023-12-26 15:22:10,244][105620] Updated weights for policy 1, policy_version 424 (0.0008) [2023-12-26 15:22:10,307][105620] Updated weights for policy 1, policy_version 434 (0.0009) [2023-12-26 15:22:10,374][105620] Updated weights for policy 1, policy_version 444 (0.0010) [2023-12-26 15:22:10,911][105692] Updated weights for policy 0, policy_version 454 (0.0009) [2023-12-26 15:22:10,964][105692] Updated weights for policy 0, policy_version 464 (0.0005) [2023-12-26 15:22:11,027][105692] Updated weights for policy 0, policy_version 474 (0.0006) [2023-12-26 15:22:11,062][104569] Fps is (10 sec: 18022.3, 60 sec: 15291.8, 300 sec: 15291.8). Total num frames: 229376. Throughput: 0: 8117.9, 1: 8169.9. Samples: 244316. Policy #0 lag: (min: 22.0, avg: 24.1, max: 50.0) [2023-12-26 15:22:11,063][104569] Avg episode reward: [(0, '112.233'), (1, '108.761')] [2023-12-26 15:22:11,063][105585] Saving new best policy, reward=112.233! [2023-12-26 15:22:11,064][105586] Saving new best policy, reward=108.761! [2023-12-26 15:22:11,136][105620] Updated weights for policy 1, policy_version 454 (0.0007) [2023-12-26 15:22:11,209][105620] Updated weights for policy 1, policy_version 464 (0.0007) [2023-12-26 15:22:11,269][105620] Updated weights for policy 1, policy_version 474 (0.0007) [2023-12-26 15:22:11,781][105692] Updated weights for policy 0, policy_version 484 (0.0007) [2023-12-26 15:22:11,846][105692] Updated weights for policy 0, policy_version 494 (0.0009) [2023-12-26 15:22:11,906][105692] Updated weights for policy 0, policy_version 504 (0.0008) [2023-12-26 15:22:12,022][105620] Updated weights for policy 1, policy_version 484 (0.0008) [2023-12-26 15:22:12,086][105620] Updated weights for policy 1, policy_version 494 (0.0011) [2023-12-26 15:22:12,150][105620] Updated weights for policy 1, policy_version 504 (0.0011) [2023-12-26 15:22:12,721][105692] Updated weights for policy 0, policy_version 514 (0.0008) [2023-12-26 15:22:12,788][105692] Updated weights for policy 0, policy_version 524 (0.0008) [2023-12-26 15:22:12,853][105692] Updated weights for policy 0, policy_version 534 (0.0009) [2023-12-26 15:22:12,897][105620] Updated weights for policy 1, policy_version 514 (0.0011) [2023-12-26 15:22:12,916][105692] Updated weights for policy 0, policy_version 544 (0.0007) [2023-12-26 15:22:12,958][105620] Updated weights for policy 1, policy_version 524 (0.0011) [2023-12-26 15:22:13,015][105620] Updated weights for policy 1, policy_version 534 (0.0011) [2023-12-26 15:22:13,068][105620] Updated weights for policy 1, policy_version 544 (0.0010) [2023-12-26 15:22:13,689][105692] Updated weights for policy 0, policy_version 554 (0.0008) [2023-12-26 15:22:13,752][105692] Updated weights for policy 0, policy_version 564 (0.0008) [2023-12-26 15:22:13,810][105692] Updated weights for policy 0, policy_version 574 (0.0009) [2023-12-26 15:22:13,834][105620] Updated weights for policy 1, policy_version 554 (0.0011) [2023-12-26 15:22:13,896][105620] Updated weights for policy 1, policy_version 564 (0.0011) [2023-12-26 15:22:13,963][105620] Updated weights for policy 1, policy_version 574 (0.0010) [2023-12-26 15:22:14,576][105692] Updated weights for policy 0, policy_version 584 (0.0009) [2023-12-26 15:22:14,636][105692] Updated weights for policy 0, policy_version 594 (0.0009) [2023-12-26 15:22:14,642][105620] Updated weights for policy 1, policy_version 584 (0.0008) [2023-12-26 15:22:14,699][105692] Updated weights for policy 0, policy_version 604 (0.0005) [2023-12-26 15:22:14,704][105620] Updated weights for policy 1, policy_version 594 (0.0011) [2023-12-26 15:22:14,768][105620] Updated weights for policy 1, policy_version 604 (0.0010) [2023-12-26 15:22:15,489][105692] Updated weights for policy 0, policy_version 614 (0.0009) [2023-12-26 15:22:15,521][105620] Updated weights for policy 1, policy_version 614 (0.0010) [2023-12-26 15:22:15,543][105692] Updated weights for policy 0, policy_version 624 (0.0007) [2023-12-26 15:22:15,577][105620] Updated weights for policy 1, policy_version 624 (0.0011) [2023-12-26 15:22:15,606][105692] Updated weights for policy 0, policy_version 634 (0.0008) [2023-12-26 15:22:15,631][105620] Updated weights for policy 1, policy_version 634 (0.0011) [2023-12-26 15:22:16,062][104569] Fps is (10 sec: 18021.7, 60 sec: 16383.8, 300 sec: 16383.8). Total num frames: 327680. Throughput: 0: 7451.3, 1: 7482.7. Samples: 298684. Policy #0 lag: (min: 17.0, avg: 39.8, max: 49.0) [2023-12-26 15:22:16,063][104569] Avg episode reward: [(0, '120.912'), (1, '144.698')] [2023-12-26 15:22:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000000640_163840.pth... [2023-12-26 15:22:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000000640_163840.pth... [2023-12-26 15:22:16,076][105586] Saving new best policy, reward=144.698! [2023-12-26 15:22:16,078][105585] Saving new best policy, reward=120.912! [2023-12-26 15:22:16,224][105692] Updated weights for policy 0, policy_version 644 (0.0007) [2023-12-26 15:22:16,272][105692] Updated weights for policy 0, policy_version 654 (0.0005) [2023-12-26 15:22:16,327][105692] Updated weights for policy 0, policy_version 664 (0.0005) [2023-12-26 15:22:16,397][105620] Updated weights for policy 1, policy_version 644 (0.0010) [2023-12-26 15:22:16,455][105620] Updated weights for policy 1, policy_version 654 (0.0010) [2023-12-26 15:22:16,508][105620] Updated weights for policy 1, policy_version 664 (0.0010) [2023-12-26 15:22:16,941][105692] Updated weights for policy 0, policy_version 674 (0.0006) [2023-12-26 15:22:16,989][105692] Updated weights for policy 0, policy_version 684 (0.0008) [2023-12-26 15:22:17,043][105692] Updated weights for policy 0, policy_version 694 (0.0005) [2023-12-26 15:22:17,091][105692] Updated weights for policy 0, policy_version 704 (0.0005) [2023-12-26 15:22:17,264][105620] Updated weights for policy 1, policy_version 674 (0.0011) [2023-12-26 15:22:17,327][105620] Updated weights for policy 1, policy_version 684 (0.0010) [2023-12-26 15:22:17,388][105620] Updated weights for policy 1, policy_version 694 (0.0010) [2023-12-26 15:22:17,438][105620] Updated weights for policy 1, policy_version 704 (0.0005) [2023-12-26 15:22:17,858][105692] Updated weights for policy 0, policy_version 714 (0.0009) [2023-12-26 15:22:17,916][105692] Updated weights for policy 0, policy_version 724 (0.0009) [2023-12-26 15:22:17,972][105692] Updated weights for policy 0, policy_version 735 (0.0008) [2023-12-26 15:22:18,000][105620] Updated weights for policy 1, policy_version 714 (0.0005) [2023-12-26 15:22:18,058][105620] Updated weights for policy 1, policy_version 724 (0.0008) [2023-12-26 15:22:18,114][105620] Updated weights for policy 1, policy_version 734 (0.0011) [2023-12-26 15:22:18,717][105692] Updated weights for policy 0, policy_version 745 (0.0010) [2023-12-26 15:22:18,783][105692] Updated weights for policy 0, policy_version 755 (0.0009) [2023-12-26 15:22:18,841][105692] Updated weights for policy 0, policy_version 765 (0.0008) [2023-12-26 15:22:18,890][105620] Updated weights for policy 1, policy_version 744 (0.0009) [2023-12-26 15:22:18,954][105620] Updated weights for policy 1, policy_version 754 (0.0009) [2023-12-26 15:22:19,019][105620] Updated weights for policy 1, policy_version 764 (0.0009) [2023-12-26 15:22:19,581][105692] Updated weights for policy 0, policy_version 775 (0.0009) [2023-12-26 15:22:19,652][105692] Updated weights for policy 0, policy_version 785 (0.0010) [2023-12-26 15:22:19,708][105620] Updated weights for policy 1, policy_version 774 (0.0010) [2023-12-26 15:22:19,714][105692] Updated weights for policy 0, policy_version 795 (0.0007) [2023-12-26 15:22:19,771][105620] Updated weights for policy 1, policy_version 784 (0.0007) [2023-12-26 15:22:19,835][105620] Updated weights for policy 1, policy_version 794 (0.0008) [2023-12-26 15:22:20,488][105692] Updated weights for policy 0, policy_version 805 (0.0008) [2023-12-26 15:22:20,553][105692] Updated weights for policy 0, policy_version 815 (0.0006) [2023-12-26 15:22:20,621][105692] Updated weights for policy 0, policy_version 825 (0.0007) [2023-12-26 15:22:20,632][105620] Updated weights for policy 1, policy_version 804 (0.0008) [2023-12-26 15:22:20,704][105620] Updated weights for policy 1, policy_version 814 (0.0006) [2023-12-26 15:22:20,773][105620] Updated weights for policy 1, policy_version 824 (0.0008) [2023-12-26 15:22:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 17039.4, 300 sec: 17039.4). Total num frames: 425984. Throughput: 0: 8273.9, 1: 8324.3. Samples: 414956. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-12-26 15:22:21,062][104569] Avg episode reward: [(0, '158.369'), (1, '156.981')] [2023-12-26 15:22:21,063][105585] Saving new best policy, reward=158.369! [2023-12-26 15:22:21,063][105586] Saving new best policy, reward=156.981! [2023-12-26 15:22:21,336][105692] Updated weights for policy 0, policy_version 835 (0.0008) [2023-12-26 15:22:21,406][105692] Updated weights for policy 0, policy_version 845 (0.0009) [2023-12-26 15:22:21,465][105692] Updated weights for policy 0, policy_version 855 (0.0008) [2023-12-26 15:22:21,577][105620] Updated weights for policy 1, policy_version 834 (0.0009) [2023-12-26 15:22:21,651][105620] Updated weights for policy 1, policy_version 844 (0.0008) [2023-12-26 15:22:21,699][105620] Updated weights for policy 1, policy_version 854 (0.0008) [2023-12-26 15:22:21,766][105620] Updated weights for policy 1, policy_version 864 (0.0009) [2023-12-26 15:22:22,220][105692] Updated weights for policy 0, policy_version 865 (0.0006) [2023-12-26 15:22:22,289][105692] Updated weights for policy 0, policy_version 875 (0.0011) [2023-12-26 15:22:22,361][105692] Updated weights for policy 0, policy_version 885 (0.0012) [2023-12-26 15:22:22,425][105692] Updated weights for policy 0, policy_version 895 (0.0007) [2023-12-26 15:22:22,596][105620] Updated weights for policy 1, policy_version 874 (0.0010) [2023-12-26 15:22:22,648][105620] Updated weights for policy 1, policy_version 884 (0.0011) [2023-12-26 15:22:22,705][105620] Updated weights for policy 1, policy_version 894 (0.0009) [2023-12-26 15:22:23,106][105692] Updated weights for policy 0, policy_version 905 (0.0009) [2023-12-26 15:22:23,161][105692] Updated weights for policy 0, policy_version 915 (0.0006) [2023-12-26 15:22:23,217][105692] Updated weights for policy 0, policy_version 925 (0.0008) [2023-12-26 15:22:23,514][105620] Updated weights for policy 1, policy_version 904 (0.0009) [2023-12-26 15:22:23,573][105620] Updated weights for policy 1, policy_version 914 (0.0009) [2023-12-26 15:22:23,627][105620] Updated weights for policy 1, policy_version 924 (0.0009) [2023-12-26 15:22:23,950][105692] Updated weights for policy 0, policy_version 935 (0.0007) [2023-12-26 15:22:23,997][105692] Updated weights for policy 0, policy_version 945 (0.0005) [2023-12-26 15:22:24,050][105692] Updated weights for policy 0, policy_version 955 (0.0006) [2023-12-26 15:22:24,441][105620] Updated weights for policy 1, policy_version 934 (0.0008) [2023-12-26 15:22:24,507][105620] Updated weights for policy 1, policy_version 944 (0.0008) [2023-12-26 15:22:24,575][105620] Updated weights for policy 1, policy_version 954 (0.0006) [2023-12-26 15:22:24,714][105692] Updated weights for policy 0, policy_version 965 (0.0009) [2023-12-26 15:22:24,772][105692] Updated weights for policy 0, policy_version 975 (0.0010) [2023-12-26 15:22:24,832][105692] Updated weights for policy 0, policy_version 985 (0.0007) [2023-12-26 15:22:25,227][105620] Updated weights for policy 1, policy_version 964 (0.0007) [2023-12-26 15:22:25,288][105620] Updated weights for policy 1, policy_version 974 (0.0008) [2023-12-26 15:22:25,356][105620] Updated weights for policy 1, policy_version 984 (0.0009) [2023-12-26 15:22:25,483][105692] Updated weights for policy 0, policy_version 995 (0.0009) [2023-12-26 15:22:25,545][105692] Updated weights for policy 0, policy_version 1005 (0.0005) [2023-12-26 15:22:25,607][105692] Updated weights for policy 0, policy_version 1015 (0.0005) [2023-12-26 15:22:26,062][104569] Fps is (10 sec: 18842.3, 60 sec: 17203.3, 300 sec: 17203.3). Total num frames: 516096. Throughput: 0: 8835.4, 1: 8701.8. Samples: 526112. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 15:22:26,062][104569] Avg episode reward: [(0, '184.982'), (1, '182.279')] [2023-12-26 15:22:26,063][105585] Saving new best policy, reward=184.982! [2023-12-26 15:22:26,063][105586] Saving new best policy, reward=182.279! [2023-12-26 15:22:26,189][105692] Updated weights for policy 0, policy_version 1025 (0.0005) [2023-12-26 15:22:26,210][105620] Updated weights for policy 1, policy_version 994 (0.0008) [2023-12-26 15:22:26,243][105692] Updated weights for policy 0, policy_version 1035 (0.0005) [2023-12-26 15:22:26,267][105620] Updated weights for policy 1, policy_version 1004 (0.0009) [2023-12-26 15:22:26,292][105692] Updated weights for policy 0, policy_version 1045 (0.0005) [2023-12-26 15:22:26,314][105620] Updated weights for policy 1, policy_version 1014 (0.0008) [2023-12-26 15:22:26,342][105692] Updated weights for policy 0, policy_version 1055 (0.0005) [2023-12-26 15:22:26,364][105620] Updated weights for policy 1, policy_version 1024 (0.0009) [2023-12-26 15:22:27,031][105692] Updated weights for policy 0, policy_version 1065 (0.0005) [2023-12-26 15:22:27,085][105692] Updated weights for policy 0, policy_version 1075 (0.0010) [2023-12-26 15:22:27,138][105692] Updated weights for policy 0, policy_version 1085 (0.0007) [2023-12-26 15:22:27,196][105620] Updated weights for policy 1, policy_version 1034 (0.0009) [2023-12-26 15:22:27,256][105620] Updated weights for policy 1, policy_version 1044 (0.0009) [2023-12-26 15:22:27,322][105620] Updated weights for policy 1, policy_version 1054 (0.0010) [2023-12-26 15:22:27,769][105692] Updated weights for policy 0, policy_version 1095 (0.0005) [2023-12-26 15:22:27,824][105692] Updated weights for policy 0, policy_version 1105 (0.0006) [2023-12-26 15:22:27,877][105692] Updated weights for policy 0, policy_version 1115 (0.0005) [2023-12-26 15:22:28,158][105620] Updated weights for policy 1, policy_version 1064 (0.0009) [2023-12-26 15:22:28,209][105620] Updated weights for policy 1, policy_version 1074 (0.0009) [2023-12-26 15:22:28,258][105620] Updated weights for policy 1, policy_version 1084 (0.0009) [2023-12-26 15:22:28,418][105692] Updated weights for policy 0, policy_version 1125 (0.0005) [2023-12-26 15:22:28,468][105692] Updated weights for policy 0, policy_version 1135 (0.0009) [2023-12-26 15:22:28,527][105692] Updated weights for policy 0, policy_version 1145 (0.0010) [2023-12-26 15:22:29,095][105620] Updated weights for policy 1, policy_version 1094 (0.0009) [2023-12-26 15:22:29,159][105620] Updated weights for policy 1, policy_version 1104 (0.0010) [2023-12-26 15:22:29,208][105620] Updated weights for policy 1, policy_version 1114 (0.0009) [2023-12-26 15:22:29,213][105692] Updated weights for policy 0, policy_version 1155 (0.0009) [2023-12-26 15:22:29,276][105692] Updated weights for policy 0, policy_version 1165 (0.0007) [2023-12-26 15:22:29,340][105692] Updated weights for policy 0, policy_version 1175 (0.0010) [2023-12-26 15:22:30,009][105692] Updated weights for policy 0, policy_version 1185 (0.0010) [2023-12-26 15:22:30,050][105620] Updated weights for policy 1, policy_version 1124 (0.0007) [2023-12-26 15:22:30,072][105692] Updated weights for policy 0, policy_version 1195 (0.0010) [2023-12-26 15:22:30,112][105620] Updated weights for policy 1, policy_version 1134 (0.0005) [2023-12-26 15:22:30,129][105692] Updated weights for policy 0, policy_version 1205 (0.0010) [2023-12-26 15:22:30,168][105620] Updated weights for policy 1, policy_version 1144 (0.0008) [2023-12-26 15:22:30,185][105692] Updated weights for policy 0, policy_version 1215 (0.0010) [2023-12-26 15:22:30,905][105620] Updated weights for policy 1, policy_version 1154 (0.0009) [2023-12-26 15:22:30,928][105692] Updated weights for policy 0, policy_version 1225 (0.0006) [2023-12-26 15:22:30,963][105620] Updated weights for policy 1, policy_version 1164 (0.0009) [2023-12-26 15:22:30,983][105692] Updated weights for policy 0, policy_version 1235 (0.0005) [2023-12-26 15:22:31,019][105620] Updated weights for policy 1, policy_version 1174 (0.0008) [2023-12-26 15:22:31,045][105692] Updated weights for policy 0, policy_version 1245 (0.0006) [2023-12-26 15:22:31,062][104569] Fps is (10 sec: 18022.4, 60 sec: 17320.3, 300 sec: 17320.3). Total num frames: 606208. Throughput: 0: 8529.8, 1: 8193.6. Samples: 585320. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-26 15:22:31,062][104569] Avg episode reward: [(0, '225.713'), (1, '205.714')] [2023-12-26 15:22:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000001248_319488.pth... [2023-12-26 15:22:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000000128_32768.pth [2023-12-26 15:22:31,073][105585] Saving new best policy, reward=225.713! [2023-12-26 15:22:31,088][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000001184_303104.pth... [2023-12-26 15:22:31,089][105620] Updated weights for policy 1, policy_version 1184 (0.0008) [2023-12-26 15:22:31,092][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000000096_24576.pth [2023-12-26 15:22:31,093][105586] Saving new best policy, reward=205.714! [2023-12-26 15:22:31,771][105692] Updated weights for policy 0, policy_version 1255 (0.0009) [2023-12-26 15:22:31,828][105692] Updated weights for policy 0, policy_version 1265 (0.0008) [2023-12-26 15:22:31,863][105620] Updated weights for policy 1, policy_version 1194 (0.0007) [2023-12-26 15:22:31,885][105692] Updated weights for policy 0, policy_version 1275 (0.0008) [2023-12-26 15:22:31,921][105620] Updated weights for policy 1, policy_version 1204 (0.0007) [2023-12-26 15:22:31,984][105620] Updated weights for policy 1, policy_version 1214 (0.0009) [2023-12-26 15:22:32,621][105692] Updated weights for policy 0, policy_version 1285 (0.0006) [2023-12-26 15:22:32,695][105692] Updated weights for policy 0, policy_version 1295 (0.0005) [2023-12-26 15:22:32,756][105692] Updated weights for policy 0, policy_version 1305 (0.0008) [2023-12-26 15:22:32,792][105620] Updated weights for policy 1, policy_version 1224 (0.0007) [2023-12-26 15:22:32,847][105620] Updated weights for policy 1, policy_version 1234 (0.0009) [2023-12-26 15:22:32,899][105620] Updated weights for policy 1, policy_version 1244 (0.0009) [2023-12-26 15:22:33,439][105692] Updated weights for policy 0, policy_version 1315 (0.0008) [2023-12-26 15:22:33,496][105692] Updated weights for policy 0, policy_version 1325 (0.0009) [2023-12-26 15:22:33,556][105692] Updated weights for policy 0, policy_version 1335 (0.0009) [2023-12-26 15:22:33,659][105620] Updated weights for policy 1, policy_version 1254 (0.0008) [2023-12-26 15:22:33,708][105620] Updated weights for policy 1, policy_version 1264 (0.0009) [2023-12-26 15:22:33,761][105620] Updated weights for policy 1, policy_version 1274 (0.0009) [2023-12-26 15:22:34,295][105692] Updated weights for policy 0, policy_version 1345 (0.0009) [2023-12-26 15:22:34,343][105692] Updated weights for policy 0, policy_version 1355 (0.0008) [2023-12-26 15:22:34,406][105692] Updated weights for policy 0, policy_version 1365 (0.0007) [2023-12-26 15:22:34,454][105692] Updated weights for policy 0, policy_version 1375 (0.0009) [2023-12-26 15:22:34,543][105620] Updated weights for policy 1, policy_version 1284 (0.0007) [2023-12-26 15:22:34,615][105620] Updated weights for policy 1, policy_version 1294 (0.0007) [2023-12-26 15:22:34,685][105620] Updated weights for policy 1, policy_version 1304 (0.0008) [2023-12-26 15:22:35,220][105692] Updated weights for policy 0, policy_version 1385 (0.0008) [2023-12-26 15:22:35,282][105692] Updated weights for policy 0, policy_version 1395 (0.0009) [2023-12-26 15:22:35,333][105692] Updated weights for policy 0, policy_version 1405 (0.0009) [2023-12-26 15:22:35,436][105620] Updated weights for policy 1, policy_version 1314 (0.0009) [2023-12-26 15:22:35,498][105620] Updated weights for policy 1, policy_version 1324 (0.0009) [2023-12-26 15:22:35,565][105620] Updated weights for policy 1, policy_version 1334 (0.0009) [2023-12-26 15:22:35,624][105620] Updated weights for policy 1, policy_version 1344 (0.0008) [2023-12-26 15:22:36,038][105692] Updated weights for policy 0, policy_version 1415 (0.0006) [2023-12-26 15:22:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 17612.9, 300 sec: 17612.9). Total num frames: 704512. Throughput: 0: 8919.7, 1: 8514.9. Samples: 697384. Policy #0 lag: (min: 26.0, avg: 36.7, max: 58.0) [2023-12-26 15:22:36,062][104569] Avg episode reward: [(0, '243.285'), (1, '229.112')] [2023-12-26 15:22:36,063][105586] Saving new best policy, reward=229.112! [2023-12-26 15:22:36,098][105692] Updated weights for policy 0, policy_version 1425 (0.0005) [2023-12-26 15:22:36,174][105692] Updated weights for policy 0, policy_version 1435 (0.0006) [2023-12-26 15:22:36,208][105585] Saving new best policy, reward=243.285! [2023-12-26 15:22:36,433][105620] Updated weights for policy 1, policy_version 1354 (0.0006) [2023-12-26 15:22:36,490][105620] Updated weights for policy 1, policy_version 1364 (0.0005) [2023-12-26 15:22:36,545][105620] Updated weights for policy 1, policy_version 1374 (0.0008) [2023-12-26 15:22:36,898][105692] Updated weights for policy 0, policy_version 1445 (0.0009) [2023-12-26 15:22:36,949][105692] Updated weights for policy 0, policy_version 1455 (0.0009) [2023-12-26 15:22:37,000][105692] Updated weights for policy 0, policy_version 1466 (0.0009) [2023-12-26 15:22:37,173][105620] Updated weights for policy 1, policy_version 1384 (0.0007) [2023-12-26 15:22:37,239][105620] Updated weights for policy 1, policy_version 1394 (0.0009) [2023-12-26 15:22:37,285][105620] Updated weights for policy 1, policy_version 1404 (0.0008) [2023-12-26 15:22:37,769][105692] Updated weights for policy 0, policy_version 1476 (0.0008) [2023-12-26 15:22:37,825][105692] Updated weights for policy 0, policy_version 1486 (0.0005) [2023-12-26 15:22:37,872][105692] Updated weights for policy 0, policy_version 1496 (0.0005) [2023-12-26 15:22:38,080][105620] Updated weights for policy 1, policy_version 1414 (0.0009) [2023-12-26 15:22:38,148][105620] Updated weights for policy 1, policy_version 1424 (0.0008) [2023-12-26 15:22:38,218][105620] Updated weights for policy 1, policy_version 1434 (0.0008) [2023-12-26 15:22:38,557][105692] Updated weights for policy 0, policy_version 1506 (0.0008) [2023-12-26 15:22:38,620][105692] Updated weights for policy 0, policy_version 1516 (0.0005) [2023-12-26 15:22:38,688][105692] Updated weights for policy 0, policy_version 1526 (0.0005) [2023-12-26 15:22:38,754][105692] Updated weights for policy 0, policy_version 1536 (0.0005) [2023-12-26 15:22:39,020][105620] Updated weights for policy 1, policy_version 1444 (0.0007) [2023-12-26 15:22:39,080][105620] Updated weights for policy 1, policy_version 1454 (0.0008) [2023-12-26 15:22:39,135][105620] Updated weights for policy 1, policy_version 1464 (0.0009) [2023-12-26 15:22:39,363][105692] Updated weights for policy 0, policy_version 1546 (0.0009) [2023-12-26 15:22:39,438][105692] Updated weights for policy 0, policy_version 1556 (0.0009) [2023-12-26 15:22:39,497][105692] Updated weights for policy 0, policy_version 1566 (0.0009) [2023-12-26 15:22:39,897][105620] Updated weights for policy 1, policy_version 1474 (0.0010) [2023-12-26 15:22:39,959][105620] Updated weights for policy 1, policy_version 1484 (0.0010) [2023-12-26 15:22:40,028][105620] Updated weights for policy 1, policy_version 1494 (0.0007) [2023-12-26 15:22:40,090][105620] Updated weights for policy 1, policy_version 1504 (0.0009) [2023-12-26 15:22:40,240][105692] Updated weights for policy 0, policy_version 1576 (0.0007) [2023-12-26 15:22:40,306][105692] Updated weights for policy 0, policy_version 1586 (0.0009) [2023-12-26 15:22:40,366][105692] Updated weights for policy 0, policy_version 1596 (0.0009) [2023-12-26 15:22:40,862][105620] Updated weights for policy 1, policy_version 1514 (0.0009) [2023-12-26 15:22:40,929][105620] Updated weights for policy 1, policy_version 1524 (0.0010) [2023-12-26 15:22:40,985][105620] Updated weights for policy 1, policy_version 1534 (0.0008) [2023-12-26 15:22:41,008][105692] Updated weights for policy 0, policy_version 1606 (0.0007) [2023-12-26 15:22:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 17840.4, 300 sec: 17840.4). Total num frames: 802816. Throughput: 0: 9216.9, 1: 8798.5. Samples: 810692. Policy #0 lag: (min: 23.0, avg: 44.5, max: 49.0) [2023-12-26 15:22:41,063][104569] Avg episode reward: [(0, '250.602'), (1, '266.215')] [2023-12-26 15:22:41,063][105586] Saving new best policy, reward=266.215! [2023-12-26 15:22:41,075][105692] Updated weights for policy 0, policy_version 1616 (0.0008) [2023-12-26 15:22:41,134][105692] Updated weights for policy 0, policy_version 1626 (0.0009) [2023-12-26 15:22:41,172][105585] Saving new best policy, reward=250.602! [2023-12-26 15:22:41,740][105620] Updated weights for policy 1, policy_version 1544 (0.0009) [2023-12-26 15:22:41,805][105620] Updated weights for policy 1, policy_version 1554 (0.0008) [2023-12-26 15:22:41,867][105620] Updated weights for policy 1, policy_version 1564 (0.0008) [2023-12-26 15:22:41,925][105692] Updated weights for policy 0, policy_version 1636 (0.0007) [2023-12-26 15:22:41,977][105692] Updated weights for policy 0, policy_version 1646 (0.0008) [2023-12-26 15:22:42,028][105692] Updated weights for policy 0, policy_version 1656 (0.0008) [2023-12-26 15:22:42,606][105620] Updated weights for policy 1, policy_version 1574 (0.0008) [2023-12-26 15:22:42,658][105620] Updated weights for policy 1, policy_version 1584 (0.0008) [2023-12-26 15:22:42,710][105620] Updated weights for policy 1, policy_version 1594 (0.0008) [2023-12-26 15:22:42,793][105692] Updated weights for policy 0, policy_version 1666 (0.0007) [2023-12-26 15:22:42,840][105692] Updated weights for policy 0, policy_version 1676 (0.0005) [2023-12-26 15:22:42,891][105692] Updated weights for policy 0, policy_version 1686 (0.0005) [2023-12-26 15:22:42,952][105692] Updated weights for policy 0, policy_version 1696 (0.0005) [2023-12-26 15:22:43,520][105620] Updated weights for policy 1, policy_version 1604 (0.0007) [2023-12-26 15:22:43,575][105620] Updated weights for policy 1, policy_version 1614 (0.0009) [2023-12-26 15:22:43,625][105620] Updated weights for policy 1, policy_version 1624 (0.0008) [2023-12-26 15:22:43,681][105692] Updated weights for policy 0, policy_version 1706 (0.0010) [2023-12-26 15:22:43,744][105692] Updated weights for policy 0, policy_version 1716 (0.0010) [2023-12-26 15:22:43,803][105692] Updated weights for policy 0, policy_version 1726 (0.0010) [2023-12-26 15:22:44,423][105620] Updated weights for policy 1, policy_version 1634 (0.0009) [2023-12-26 15:22:44,427][105692] Updated weights for policy 0, policy_version 1736 (0.0008) [2023-12-26 15:22:44,481][105620] Updated weights for policy 1, policy_version 1644 (0.0007) [2023-12-26 15:22:44,490][105692] Updated weights for policy 0, policy_version 1746 (0.0007) [2023-12-26 15:22:44,530][105620] Updated weights for policy 1, policy_version 1654 (0.0008) [2023-12-26 15:22:44,556][105692] Updated weights for policy 0, policy_version 1756 (0.0006) [2023-12-26 15:22:44,583][105620] Updated weights for policy 1, policy_version 1664 (0.0007) [2023-12-26 15:22:45,267][105692] Updated weights for policy 0, policy_version 1766 (0.0009) [2023-12-26 15:22:45,282][105620] Updated weights for policy 1, policy_version 1674 (0.0007) [2023-12-26 15:22:45,320][105692] Updated weights for policy 0, policy_version 1776 (0.0010) [2023-12-26 15:22:45,343][105620] Updated weights for policy 1, policy_version 1684 (0.0006) [2023-12-26 15:22:45,373][105692] Updated weights for policy 0, policy_version 1786 (0.0010) [2023-12-26 15:22:45,403][105620] Updated weights for policy 1, policy_version 1694 (0.0006) [2023-12-26 15:22:46,025][105692] Updated weights for policy 0, policy_version 1796 (0.0008) [2023-12-26 15:22:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 17858.5, 300 sec: 17858.5). Total num frames: 892928. Throughput: 0: 9486.4, 1: 9048.7. Samples: 867456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:22:46,063][104569] Avg episode reward: [(0, '295.759'), (1, '277.720')] [2023-12-26 15:22:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000001696_434176.pth... [2023-12-26 15:22:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000000640_163840.pth [2023-12-26 15:22:46,072][105586] Saving new best policy, reward=277.720! [2023-12-26 15:22:46,077][105692] Updated weights for policy 0, policy_version 1806 (0.0005) [2023-12-26 15:22:46,145][105692] Updated weights for policy 0, policy_version 1816 (0.0005) [2023-12-26 15:22:46,168][105620] Updated weights for policy 1, policy_version 1704 (0.0008) [2023-12-26 15:22:46,184][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000001824_466944.pth... [2023-12-26 15:22:46,187][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000000640_163840.pth [2023-12-26 15:22:46,187][105585] Saving new best policy, reward=295.759! [2023-12-26 15:22:46,220][105620] Updated weights for policy 1, policy_version 1714 (0.0011) [2023-12-26 15:22:46,279][105620] Updated weights for policy 1, policy_version 1724 (0.0009) [2023-12-26 15:22:46,801][105692] Updated weights for policy 0, policy_version 1826 (0.0005) [2023-12-26 15:22:46,844][105692] Updated weights for policy 0, policy_version 1836 (0.0005) [2023-12-26 15:22:46,903][105692] Updated weights for policy 0, policy_version 1846 (0.0007) [2023-12-26 15:22:46,951][105692] Updated weights for policy 0, policy_version 1856 (0.0009) [2023-12-26 15:22:47,100][105620] Updated weights for policy 1, policy_version 1734 (0.0009) [2023-12-26 15:22:47,148][105620] Updated weights for policy 1, policy_version 1744 (0.0009) [2023-12-26 15:22:47,201][105620] Updated weights for policy 1, policy_version 1754 (0.0010) [2023-12-26 15:22:47,665][105692] Updated weights for policy 0, policy_version 1866 (0.0009) [2023-12-26 15:22:47,718][105692] Updated weights for policy 0, policy_version 1876 (0.0008) [2023-12-26 15:22:47,782][105692] Updated weights for policy 0, policy_version 1886 (0.0009) [2023-12-26 15:22:47,962][105620] Updated weights for policy 1, policy_version 1764 (0.0008) [2023-12-26 15:22:48,022][105620] Updated weights for policy 1, policy_version 1774 (0.0008) [2023-12-26 15:22:48,083][105620] Updated weights for policy 1, policy_version 1784 (0.0008) [2023-12-26 15:22:48,557][105692] Updated weights for policy 0, policy_version 1896 (0.0010) [2023-12-26 15:22:48,623][105692] Updated weights for policy 0, policy_version 1906 (0.0010) [2023-12-26 15:22:48,682][105692] Updated weights for policy 0, policy_version 1916 (0.0010) [2023-12-26 15:22:48,852][105620] Updated weights for policy 1, policy_version 1794 (0.0008) [2023-12-26 15:22:48,905][105620] Updated weights for policy 1, policy_version 1804 (0.0008) [2023-12-26 15:22:48,969][105620] Updated weights for policy 1, policy_version 1814 (0.0008) [2023-12-26 15:22:49,026][105620] Updated weights for policy 1, policy_version 1824 (0.0008) [2023-12-26 15:22:49,379][105692] Updated weights for policy 0, policy_version 1926 (0.0009) [2023-12-26 15:22:49,442][105692] Updated weights for policy 0, policy_version 1936 (0.0009) [2023-12-26 15:22:49,507][105692] Updated weights for policy 0, policy_version 1946 (0.0008) [2023-12-26 15:22:49,887][105620] Updated weights for policy 1, policy_version 1834 (0.0007) [2023-12-26 15:22:49,953][105620] Updated weights for policy 1, policy_version 1844 (0.0008) [2023-12-26 15:22:50,019][105620] Updated weights for policy 1, policy_version 1854 (0.0005) [2023-12-26 15:22:50,121][105692] Updated weights for policy 0, policy_version 1956 (0.0007) [2023-12-26 15:22:50,181][105692] Updated weights for policy 0, policy_version 1966 (0.0009) [2023-12-26 15:22:50,235][105692] Updated weights for policy 0, policy_version 1976 (0.0009) [2023-12-26 15:22:50,662][105620] Updated weights for policy 1, policy_version 1864 (0.0008) [2023-12-26 15:22:50,720][105620] Updated weights for policy 1, policy_version 1874 (0.0009) [2023-12-26 15:22:50,769][105620] Updated weights for policy 1, policy_version 1884 (0.0009) [2023-12-26 15:22:51,043][105692] Updated weights for policy 0, policy_version 1986 (0.0009) [2023-12-26 15:22:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18022.4, 300 sec: 18022.4). Total num frames: 991232. Throughput: 0: 9676.6, 1: 9123.3. Samples: 981008. Policy #0 lag: (min: 31.0, avg: 31.4, max: 46.0) [2023-12-26 15:22:51,063][104569] Avg episode reward: [(0, '310.158'), (1, '297.605')] [2023-12-26 15:22:51,064][105586] Saving new best policy, reward=297.605! [2023-12-26 15:22:51,106][105692] Updated weights for policy 0, policy_version 1996 (0.0007) [2023-12-26 15:22:51,173][105692] Updated weights for policy 0, policy_version 2006 (0.0008) [2023-12-26 15:22:51,236][105585] Saving new best policy, reward=310.158! [2023-12-26 15:22:51,238][105692] Updated weights for policy 0, policy_version 2016 (0.0008) [2023-12-26 15:22:51,563][105620] Updated weights for policy 1, policy_version 1894 (0.0009) [2023-12-26 15:22:51,614][105620] Updated weights for policy 1, policy_version 1904 (0.0008) [2023-12-26 15:22:51,681][105620] Updated weights for policy 1, policy_version 1914 (0.0008) [2023-12-26 15:22:52,024][105692] Updated weights for policy 0, policy_version 2026 (0.0009) [2023-12-26 15:22:52,083][105692] Updated weights for policy 0, policy_version 2036 (0.0010) [2023-12-26 15:22:52,136][105692] Updated weights for policy 0, policy_version 2046 (0.0009) [2023-12-26 15:22:52,434][105620] Updated weights for policy 1, policy_version 1924 (0.0008) [2023-12-26 15:22:52,496][105620] Updated weights for policy 1, policy_version 1934 (0.0009) [2023-12-26 15:22:52,551][105620] Updated weights for policy 1, policy_version 1944 (0.0009) [2023-12-26 15:22:52,910][105692] Updated weights for policy 0, policy_version 2056 (0.0009) [2023-12-26 15:22:52,977][105692] Updated weights for policy 0, policy_version 2066 (0.0009) [2023-12-26 15:22:53,043][105692] Updated weights for policy 0, policy_version 2076 (0.0008) [2023-12-26 15:22:53,333][105620] Updated weights for policy 1, policy_version 1954 (0.0009) [2023-12-26 15:22:53,384][105620] Updated weights for policy 1, policy_version 1964 (0.0009) [2023-12-26 15:22:53,451][105620] Updated weights for policy 1, policy_version 1974 (0.0009) [2023-12-26 15:22:53,520][105620] Updated weights for policy 1, policy_version 1984 (0.0007) [2023-12-26 15:22:53,759][105692] Updated weights for policy 0, policy_version 2086 (0.0010) [2023-12-26 15:22:53,800][105692] Updated weights for policy 0, policy_version 2096 (0.0010) [2023-12-26 15:22:53,853][105692] Updated weights for policy 0, policy_version 2106 (0.0010) [2023-12-26 15:22:54,115][105620] Updated weights for policy 1, policy_version 1994 (0.0005) [2023-12-26 15:22:54,181][105620] Updated weights for policy 1, policy_version 2004 (0.0006) [2023-12-26 15:22:54,249][105620] Updated weights for policy 1, policy_version 2014 (0.0005) [2023-12-26 15:22:54,559][105692] Updated weights for policy 0, policy_version 2116 (0.0008) [2023-12-26 15:22:54,626][105692] Updated weights for policy 0, policy_version 2126 (0.0006) [2023-12-26 15:22:54,688][105692] Updated weights for policy 0, policy_version 2136 (0.0005) [2023-12-26 15:22:54,817][105620] Updated weights for policy 1, policy_version 2024 (0.0006) [2023-12-26 15:22:54,870][105620] Updated weights for policy 1, policy_version 2034 (0.0005) [2023-12-26 15:22:54,915][105620] Updated weights for policy 1, policy_version 2044 (0.0005) [2023-12-26 15:22:55,265][105692] Updated weights for policy 0, policy_version 2146 (0.0006) [2023-12-26 15:22:55,320][105692] Updated weights for policy 0, policy_version 2156 (0.0010) [2023-12-26 15:22:55,375][105692] Updated weights for policy 0, policy_version 2166 (0.0010) [2023-12-26 15:22:55,442][105692] Updated weights for policy 0, policy_version 2176 (0.0008) [2023-12-26 15:22:55,610][105620] Updated weights for policy 1, policy_version 2054 (0.0008) [2023-12-26 15:22:55,666][105620] Updated weights for policy 1, policy_version 2064 (0.0010) [2023-12-26 15:22:55,718][105620] Updated weights for policy 1, policy_version 2074 (0.0010) [2023-12-26 15:22:56,044][105692] Updated weights for policy 0, policy_version 2186 (0.0006) [2023-12-26 15:22:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 18158.9, 300 sec: 18158.9). Total num frames: 1089536. Throughput: 0: 9771.9, 1: 9215.2. Samples: 1098736. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) [2023-12-26 15:22:56,062][104569] Avg episode reward: [(0, '327.111'), (1, '310.318')] [2023-12-26 15:22:56,063][105586] Saving new best policy, reward=310.318! [2023-12-26 15:22:56,106][105692] Updated weights for policy 0, policy_version 2196 (0.0005) [2023-12-26 15:22:56,176][105692] Updated weights for policy 0, policy_version 2206 (0.0006) [2023-12-26 15:22:56,186][105585] Saving new best policy, reward=327.111! [2023-12-26 15:22:56,472][105620] Updated weights for policy 1, policy_version 2084 (0.0011) [2023-12-26 15:22:56,531][105620] Updated weights for policy 1, policy_version 2094 (0.0010) [2023-12-26 15:22:56,592][105620] Updated weights for policy 1, policy_version 2104 (0.0010) [2023-12-26 15:22:56,849][105692] Updated weights for policy 0, policy_version 2216 (0.0009) [2023-12-26 15:22:56,903][105692] Updated weights for policy 0, policy_version 2226 (0.0010) [2023-12-26 15:22:56,960][105692] Updated weights for policy 0, policy_version 2236 (0.0008) [2023-12-26 15:22:57,248][105620] Updated weights for policy 1, policy_version 2114 (0.0006) [2023-12-26 15:22:57,309][105620] Updated weights for policy 1, policy_version 2124 (0.0010) [2023-12-26 15:22:57,367][105620] Updated weights for policy 1, policy_version 2134 (0.0007) [2023-12-26 15:22:57,425][105620] Updated weights for policy 1, policy_version 2144 (0.0005) [2023-12-26 15:22:57,608][105692] Updated weights for policy 0, policy_version 2246 (0.0009) [2023-12-26 15:22:57,655][105692] Updated weights for policy 0, policy_version 2256 (0.0010) [2023-12-26 15:22:57,699][105692] Updated weights for policy 0, policy_version 2266 (0.0010) [2023-12-26 15:22:58,112][105620] Updated weights for policy 1, policy_version 2154 (0.0008) [2023-12-26 15:22:58,180][105620] Updated weights for policy 1, policy_version 2164 (0.0008) [2023-12-26 15:22:58,248][105620] Updated weights for policy 1, policy_version 2174 (0.0007) [2023-12-26 15:22:58,463][105692] Updated weights for policy 0, policy_version 2276 (0.0007) [2023-12-26 15:22:58,523][105692] Updated weights for policy 0, policy_version 2286 (0.0009) [2023-12-26 15:22:58,579][105692] Updated weights for policy 0, policy_version 2296 (0.0009) [2023-12-26 15:22:58,967][105620] Updated weights for policy 1, policy_version 2184 (0.0006) [2023-12-26 15:22:59,030][105620] Updated weights for policy 1, policy_version 2194 (0.0005) [2023-12-26 15:22:59,091][105620] Updated weights for policy 1, policy_version 2204 (0.0008) [2023-12-26 15:22:59,419][105692] Updated weights for policy 0, policy_version 2306 (0.0009) [2023-12-26 15:22:59,477][105692] Updated weights for policy 0, policy_version 2316 (0.0007) [2023-12-26 15:22:59,532][105692] Updated weights for policy 0, policy_version 2326 (0.0008) [2023-12-26 15:22:59,585][105692] Updated weights for policy 0, policy_version 2336 (0.0008) [2023-12-26 15:22:59,793][105620] Updated weights for policy 1, policy_version 2214 (0.0009) [2023-12-26 15:22:59,858][105620] Updated weights for policy 1, policy_version 2224 (0.0009) [2023-12-26 15:22:59,922][105620] Updated weights for policy 1, policy_version 2234 (0.0009) [2023-12-26 15:23:00,277][105692] Updated weights for policy 0, policy_version 2346 (0.0009) [2023-12-26 15:23:00,327][105692] Updated weights for policy 0, policy_version 2356 (0.0007) [2023-12-26 15:23:00,390][105692] Updated weights for policy 0, policy_version 2366 (0.0005) [2023-12-26 15:23:00,594][105620] Updated weights for policy 1, policy_version 2244 (0.0006) [2023-12-26 15:23:00,662][105620] Updated weights for policy 1, policy_version 2254 (0.0005) [2023-12-26 15:23:00,731][105620] Updated weights for policy 1, policy_version 2264 (0.0005) [2023-12-26 15:23:00,993][105692] Updated weights for policy 0, policy_version 2376 (0.0008) [2023-12-26 15:23:01,062][105692] Updated weights for policy 0, policy_version 2386 (0.0011) [2023-12-26 15:23:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 18978.1, 300 sec: 18274.5). Total num frames: 1187840. Throughput: 0: 9848.4, 1: 9273.9. Samples: 1159184. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-26 15:23:01,062][104569] Avg episode reward: [(0, '334.187'), (1, '314.943')] [2023-12-26 15:23:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000002272_581632.pth... [2023-12-26 15:23:01,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000001184_303104.pth [2023-12-26 15:23:01,070][105586] Saving new best policy, reward=314.943! [2023-12-26 15:23:01,108][105692] Updated weights for policy 0, policy_version 2396 (0.0010) [2023-12-26 15:23:01,133][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000002400_614400.pth... [2023-12-26 15:23:01,137][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000001248_319488.pth [2023-12-26 15:23:01,138][105585] Saving new best policy, reward=334.187! [2023-12-26 15:23:01,334][105620] Updated weights for policy 1, policy_version 2274 (0.0006) [2023-12-26 15:23:01,393][105620] Updated weights for policy 1, policy_version 2284 (0.0009) [2023-12-26 15:23:01,446][105620] Updated weights for policy 1, policy_version 2294 (0.0009) [2023-12-26 15:23:01,501][105620] Updated weights for policy 1, policy_version 2304 (0.0009) [2023-12-26 15:23:01,907][105692] Updated weights for policy 0, policy_version 2406 (0.0010) [2023-12-26 15:23:01,972][105692] Updated weights for policy 0, policy_version 2416 (0.0010) [2023-12-26 15:23:02,038][105692] Updated weights for policy 0, policy_version 2426 (0.0011) [2023-12-26 15:23:02,305][105620] Updated weights for policy 1, policy_version 2314 (0.0009) [2023-12-26 15:23:02,366][105620] Updated weights for policy 1, policy_version 2324 (0.0008) [2023-12-26 15:23:02,427][105620] Updated weights for policy 1, policy_version 2334 (0.0007) [2023-12-26 15:23:02,780][105692] Updated weights for policy 0, policy_version 2436 (0.0011) [2023-12-26 15:23:02,835][105692] Updated weights for policy 0, policy_version 2446 (0.0010) [2023-12-26 15:23:02,893][105692] Updated weights for policy 0, policy_version 2456 (0.0010) [2023-12-26 15:23:03,185][105620] Updated weights for policy 1, policy_version 2344 (0.0008) [2023-12-26 15:23:03,240][105620] Updated weights for policy 1, policy_version 2354 (0.0008) [2023-12-26 15:23:03,287][105620] Updated weights for policy 1, policy_version 2364 (0.0009) [2023-12-26 15:23:03,602][105692] Updated weights for policy 0, policy_version 2466 (0.0008) [2023-12-26 15:23:03,664][105692] Updated weights for policy 0, policy_version 2476 (0.0007) [2023-12-26 15:23:03,708][105692] Updated weights for policy 0, policy_version 2486 (0.0010) [2023-12-26 15:23:03,753][105692] Updated weights for policy 0, policy_version 2496 (0.0010) [2023-12-26 15:23:03,956][105620] Updated weights for policy 1, policy_version 2374 (0.0008) [2023-12-26 15:23:04,004][105620] Updated weights for policy 1, policy_version 2384 (0.0009) [2023-12-26 15:23:04,067][105620] Updated weights for policy 1, policy_version 2394 (0.0010) [2023-12-26 15:23:04,438][105692] Updated weights for policy 0, policy_version 2506 (0.0011) [2023-12-26 15:23:04,504][105692] Updated weights for policy 0, policy_version 2516 (0.0011) [2023-12-26 15:23:04,563][105692] Updated weights for policy 0, policy_version 2526 (0.0010) [2023-12-26 15:23:04,841][105620] Updated weights for policy 1, policy_version 2404 (0.0010) [2023-12-26 15:23:04,888][105620] Updated weights for policy 1, policy_version 2414 (0.0008) [2023-12-26 15:23:04,943][105620] Updated weights for policy 1, policy_version 2424 (0.0008) [2023-12-26 15:23:05,258][105692] Updated weights for policy 0, policy_version 2536 (0.0006) [2023-12-26 15:23:05,307][105692] Updated weights for policy 0, policy_version 2546 (0.0007) [2023-12-26 15:23:05,371][105692] Updated weights for policy 0, policy_version 2556 (0.0009) [2023-12-26 15:23:05,608][105620] Updated weights for policy 1, policy_version 2434 (0.0008) [2023-12-26 15:23:05,661][105620] Updated weights for policy 1, policy_version 2444 (0.0009) [2023-12-26 15:23:05,716][105620] Updated weights for policy 1, policy_version 2454 (0.0008) [2023-12-26 15:23:05,768][105620] Updated weights for policy 1, policy_version 2464 (0.0008) [2023-12-26 15:23:06,057][105692] Updated weights for policy 0, policy_version 2566 (0.0007) [2023-12-26 15:23:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 18978.1, 300 sec: 18373.5). Total num frames: 1286144. Throughput: 0: 9856.0, 1: 9261.0. Samples: 1275220. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-26 15:23:06,062][104569] Avg episode reward: [(0, '392.807'), (1, '331.451')] [2023-12-26 15:23:06,063][105586] Saving new best policy, reward=331.451! [2023-12-26 15:23:06,116][105692] Updated weights for policy 0, policy_version 2576 (0.0006) [2023-12-26 15:23:06,177][105692] Updated weights for policy 0, policy_version 2586 (0.0006) [2023-12-26 15:23:06,215][105585] Saving new best policy, reward=392.807! [2023-12-26 15:23:06,646][105620] Updated weights for policy 1, policy_version 2474 (0.0007) [2023-12-26 15:23:06,720][105620] Updated weights for policy 1, policy_version 2484 (0.0006) [2023-12-26 15:23:06,783][105620] Updated weights for policy 1, policy_version 2494 (0.0006) [2023-12-26 15:23:06,870][105692] Updated weights for policy 0, policy_version 2596 (0.0006) [2023-12-26 15:23:06,932][105692] Updated weights for policy 0, policy_version 2606 (0.0009) [2023-12-26 15:23:06,991][105692] Updated weights for policy 0, policy_version 2616 (0.0009) [2023-12-26 15:23:07,464][105620] Updated weights for policy 1, policy_version 2504 (0.0009) [2023-12-26 15:23:07,532][105620] Updated weights for policy 1, policy_version 2514 (0.0009) [2023-12-26 15:23:07,593][105620] Updated weights for policy 1, policy_version 2525 (0.0013) [2023-12-26 15:23:07,720][105692] Updated weights for policy 0, policy_version 2626 (0.0008) [2023-12-26 15:23:07,779][105692] Updated weights for policy 0, policy_version 2636 (0.0005) [2023-12-26 15:23:07,837][105692] Updated weights for policy 0, policy_version 2646 (0.0005) [2023-12-26 15:23:07,891][105692] Updated weights for policy 0, policy_version 2656 (0.0005) [2023-12-26 15:23:08,231][105620] Updated weights for policy 1, policy_version 2535 (0.0006) [2023-12-26 15:23:08,279][105620] Updated weights for policy 1, policy_version 2545 (0.0005) [2023-12-26 15:23:08,344][105620] Updated weights for policy 1, policy_version 2555 (0.0007) [2023-12-26 15:23:08,553][105692] Updated weights for policy 0, policy_version 2666 (0.0010) [2023-12-26 15:23:08,612][105692] Updated weights for policy 0, policy_version 2676 (0.0009) [2023-12-26 15:23:08,671][105692] Updated weights for policy 0, policy_version 2686 (0.0010) [2023-12-26 15:23:09,079][105620] Updated weights for policy 1, policy_version 2565 (0.0008) [2023-12-26 15:23:09,134][105620] Updated weights for policy 1, policy_version 2575 (0.0008) [2023-12-26 15:23:09,193][105620] Updated weights for policy 1, policy_version 2585 (0.0008) [2023-12-26 15:23:09,418][105692] Updated weights for policy 0, policy_version 2696 (0.0009) [2023-12-26 15:23:09,477][105692] Updated weights for policy 0, policy_version 2706 (0.0009) [2023-12-26 15:23:09,549][105692] Updated weights for policy 0, policy_version 2716 (0.0009) [2023-12-26 15:23:09,998][105620] Updated weights for policy 1, policy_version 2595 (0.0009) [2023-12-26 15:23:10,065][105620] Updated weights for policy 1, policy_version 2605 (0.0009) [2023-12-26 15:23:10,130][105620] Updated weights for policy 1, policy_version 2615 (0.0009) [2023-12-26 15:23:10,271][105692] Updated weights for policy 0, policy_version 2726 (0.0009) [2023-12-26 15:23:10,323][105692] Updated weights for policy 0, policy_version 2736 (0.0009) [2023-12-26 15:23:10,383][105692] Updated weights for policy 0, policy_version 2746 (0.0009) [2023-12-26 15:23:10,852][105620] Updated weights for policy 1, policy_version 2625 (0.0012) [2023-12-26 15:23:10,911][105620] Updated weights for policy 1, policy_version 2635 (0.0010) [2023-12-26 15:23:10,969][105620] Updated weights for policy 1, policy_version 2645 (0.0008) [2023-12-26 15:23:11,036][105620] Updated weights for policy 1, policy_version 2655 (0.0009) [2023-12-26 15:23:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 18459.3). Total num frames: 1384448. Throughput: 0: 9870.4, 1: 9359.5. Samples: 1391460. Policy #0 lag: (min: 14.0, avg: 22.2, max: 46.0) [2023-12-26 15:23:11,062][104569] Avg episode reward: [(0, '383.934'), (1, '363.484')] [2023-12-26 15:23:11,063][105586] Saving new best policy, reward=363.484! [2023-12-26 15:23:11,170][105692] Updated weights for policy 0, policy_version 2756 (0.0008) [2023-12-26 15:23:11,243][105692] Updated weights for policy 0, policy_version 2766 (0.0006) [2023-12-26 15:23:11,308][105692] Updated weights for policy 0, policy_version 2776 (0.0007) [2023-12-26 15:23:11,845][105620] Updated weights for policy 1, policy_version 2665 (0.0010) [2023-12-26 15:23:11,903][105620] Updated weights for policy 1, policy_version 2675 (0.0009) [2023-12-26 15:23:11,954][105620] Updated weights for policy 1, policy_version 2685 (0.0009) [2023-12-26 15:23:12,010][105692] Updated weights for policy 0, policy_version 2786 (0.0008) [2023-12-26 15:23:12,070][105692] Updated weights for policy 0, policy_version 2796 (0.0005) [2023-12-26 15:23:12,145][105692] Updated weights for policy 0, policy_version 2806 (0.0010) [2023-12-26 15:23:12,213][105692] Updated weights for policy 0, policy_version 2816 (0.0009) [2023-12-26 15:23:12,701][105620] Updated weights for policy 1, policy_version 2695 (0.0007) [2023-12-26 15:23:12,755][105620] Updated weights for policy 1, policy_version 2705 (0.0005) [2023-12-26 15:23:12,810][105620] Updated weights for policy 1, policy_version 2715 (0.0006) [2023-12-26 15:23:12,869][105692] Updated weights for policy 0, policy_version 2826 (0.0010) [2023-12-26 15:23:12,914][105692] Updated weights for policy 0, policy_version 2836 (0.0010) [2023-12-26 15:23:12,962][105692] Updated weights for policy 0, policy_version 2846 (0.0010) [2023-12-26 15:23:13,431][105620] Updated weights for policy 1, policy_version 2725 (0.0008) [2023-12-26 15:23:13,483][105620] Updated weights for policy 1, policy_version 2735 (0.0008) [2023-12-26 15:23:13,536][105620] Updated weights for policy 1, policy_version 2745 (0.0008) [2023-12-26 15:23:13,703][105692] Updated weights for policy 0, policy_version 2856 (0.0006) [2023-12-26 15:23:13,763][105692] Updated weights for policy 0, policy_version 2866 (0.0005) [2023-12-26 15:23:13,816][105692] Updated weights for policy 0, policy_version 2876 (0.0007) [2023-12-26 15:23:14,345][105692] Updated weights for policy 0, policy_version 2886 (0.0007) [2023-12-26 15:23:14,409][105620] Updated weights for policy 1, policy_version 2755 (0.0008) [2023-12-26 15:23:14,413][105692] Updated weights for policy 0, policy_version 2896 (0.0005) [2023-12-26 15:23:14,471][105620] Updated weights for policy 1, policy_version 2765 (0.0009) [2023-12-26 15:23:14,477][105692] Updated weights for policy 0, policy_version 2906 (0.0006) [2023-12-26 15:23:14,529][105620] Updated weights for policy 1, policy_version 2775 (0.0009) [2023-12-26 15:23:15,103][105692] Updated weights for policy 0, policy_version 2916 (0.0007) [2023-12-26 15:23:15,157][105692] Updated weights for policy 0, policy_version 2926 (0.0009) [2023-12-26 15:23:15,222][105692] Updated weights for policy 0, policy_version 2936 (0.0009) [2023-12-26 15:23:15,303][105620] Updated weights for policy 1, policy_version 2785 (0.0009) [2023-12-26 15:23:15,357][105620] Updated weights for policy 1, policy_version 2795 (0.0008) [2023-12-26 15:23:15,419][105620] Updated weights for policy 1, policy_version 2805 (0.0009) [2023-12-26 15:23:15,481][105620] Updated weights for policy 1, policy_version 2815 (0.0008) [2023-12-26 15:23:16,034][105692] Updated weights for policy 0, policy_version 2946 (0.0010) [2023-12-26 15:23:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.8, 300 sec: 18432.0). Total num frames: 1474560. Throughput: 0: 9767.6, 1: 9418.4. Samples: 1448688. Policy #0 lag: (min: 16.0, avg: 41.4, max: 48.0) [2023-12-26 15:23:16,062][104569] Avg episode reward: [(0, '431.988'), (1, '398.973')] [2023-12-26 15:23:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000002816_720896.pth... [2023-12-26 15:23:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000001696_434176.pth [2023-12-26 15:23:16,073][105586] Saving new best policy, reward=398.973! [2023-12-26 15:23:16,086][105692] Updated weights for policy 0, policy_version 2956 (0.0009) [2023-12-26 15:23:16,138][105692] Updated weights for policy 0, policy_version 2966 (0.0008) [2023-12-26 15:23:16,185][105620] Updated weights for policy 1, policy_version 2825 (0.0009) [2023-12-26 15:23:16,188][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000002976_761856.pth... [2023-12-26 15:23:16,191][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000001824_466944.pth [2023-12-26 15:23:16,191][105692] Updated weights for policy 0, policy_version 2976 (0.0008) [2023-12-26 15:23:16,191][105585] Saving new best policy, reward=431.988! [2023-12-26 15:23:16,241][105620] Updated weights for policy 1, policy_version 2835 (0.0008) [2023-12-26 15:23:16,296][105620] Updated weights for policy 1, policy_version 2845 (0.0009) [2023-12-26 15:23:16,952][105692] Updated weights for policy 0, policy_version 2986 (0.0008) [2023-12-26 15:23:17,013][105692] Updated weights for policy 0, policy_version 2996 (0.0009) [2023-12-26 15:23:17,039][105620] Updated weights for policy 1, policy_version 2855 (0.0007) [2023-12-26 15:23:17,067][105692] Updated weights for policy 0, policy_version 3006 (0.0009) [2023-12-26 15:23:17,103][105620] Updated weights for policy 1, policy_version 2865 (0.0007) [2023-12-26 15:23:17,162][105620] Updated weights for policy 1, policy_version 2875 (0.0005) [2023-12-26 15:23:17,808][105620] Updated weights for policy 1, policy_version 2885 (0.0006) [2023-12-26 15:23:17,872][105620] Updated weights for policy 1, policy_version 2895 (0.0007) [2023-12-26 15:23:17,891][105692] Updated weights for policy 0, policy_version 3016 (0.0008) [2023-12-26 15:23:17,933][105620] Updated weights for policy 1, policy_version 2905 (0.0006) [2023-12-26 15:23:17,949][105692] Updated weights for policy 0, policy_version 3026 (0.0008) [2023-12-26 15:23:18,001][105692] Updated weights for policy 0, policy_version 3036 (0.0009) [2023-12-26 15:23:18,536][105620] Updated weights for policy 1, policy_version 2915 (0.0005) [2023-12-26 15:23:18,606][105620] Updated weights for policy 1, policy_version 2925 (0.0007) [2023-12-26 15:23:18,671][105620] Updated weights for policy 1, policy_version 2935 (0.0007) [2023-12-26 15:23:18,790][105692] Updated weights for policy 0, policy_version 3046 (0.0009) [2023-12-26 15:23:18,851][105692] Updated weights for policy 0, policy_version 3056 (0.0008) [2023-12-26 15:23:18,911][105692] Updated weights for policy 0, policy_version 3066 (0.0009) [2023-12-26 15:23:19,361][105620] Updated weights for policy 1, policy_version 2945 (0.0009) [2023-12-26 15:23:19,414][105620] Updated weights for policy 1, policy_version 2955 (0.0007) [2023-12-26 15:23:19,477][105620] Updated weights for policy 1, policy_version 2965 (0.0006) [2023-12-26 15:23:19,544][105620] Updated weights for policy 1, policy_version 2975 (0.0007) [2023-12-26 15:23:19,722][105692] Updated weights for policy 0, policy_version 3076 (0.0009) [2023-12-26 15:23:19,781][105692] Updated weights for policy 0, policy_version 3086 (0.0009) [2023-12-26 15:23:19,845][105692] Updated weights for policy 0, policy_version 3096 (0.0009) [2023-12-26 15:23:20,288][105620] Updated weights for policy 1, policy_version 2985 (0.0009) [2023-12-26 15:23:20,353][105620] Updated weights for policy 1, policy_version 2995 (0.0008) [2023-12-26 15:23:20,407][105620] Updated weights for policy 1, policy_version 3005 (0.0005) [2023-12-26 15:23:20,623][105692] Updated weights for policy 0, policy_version 3106 (0.0007) [2023-12-26 15:23:20,691][105692] Updated weights for policy 0, policy_version 3116 (0.0009) [2023-12-26 15:23:20,761][105692] Updated weights for policy 0, policy_version 3126 (0.0010) [2023-12-26 15:23:20,820][105692] Updated weights for policy 0, policy_version 3136 (0.0010) [2023-12-26 15:23:21,038][105620] Updated weights for policy 1, policy_version 3015 (0.0008) [2023-12-26 15:23:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 18504.3). Total num frames: 1572864. Throughput: 0: 9738.3, 1: 9523.8. Samples: 1564180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:23:21,062][104569] Avg episode reward: [(0, '445.187'), (1, '388.312')] [2023-12-26 15:23:21,063][105585] Saving new best policy, reward=445.187! [2023-12-26 15:23:21,102][105620] Updated weights for policy 1, policy_version 3025 (0.0009) [2023-12-26 15:23:21,171][105620] Updated weights for policy 1, policy_version 3035 (0.0009) [2023-12-26 15:23:21,644][105692] Updated weights for policy 0, policy_version 3146 (0.0008) [2023-12-26 15:23:21,711][105692] Updated weights for policy 0, policy_version 3156 (0.0008) [2023-12-26 15:23:21,779][105692] Updated weights for policy 0, policy_version 3166 (0.0009) [2023-12-26 15:23:21,957][105620] Updated weights for policy 1, policy_version 3045 (0.0009) [2023-12-26 15:23:22,010][105620] Updated weights for policy 1, policy_version 3055 (0.0009) [2023-12-26 15:23:22,078][105620] Updated weights for policy 1, policy_version 3065 (0.0009) [2023-12-26 15:23:22,583][105692] Updated weights for policy 0, policy_version 3176 (0.0009) [2023-12-26 15:23:22,655][105692] Updated weights for policy 0, policy_version 3186 (0.0009) [2023-12-26 15:23:22,719][105692] Updated weights for policy 0, policy_version 3196 (0.0010) [2023-12-26 15:23:22,763][105620] Updated weights for policy 1, policy_version 3075 (0.0008) [2023-12-26 15:23:22,832][105620] Updated weights for policy 1, policy_version 3085 (0.0006) [2023-12-26 15:23:22,895][105620] Updated weights for policy 1, policy_version 3095 (0.0005) [2023-12-26 15:23:23,483][105620] Updated weights for policy 1, policy_version 3105 (0.0006) [2023-12-26 15:23:23,498][105692] Updated weights for policy 0, policy_version 3206 (0.0007) [2023-12-26 15:23:23,533][105620] Updated weights for policy 1, policy_version 3115 (0.0007) [2023-12-26 15:23:23,556][105692] Updated weights for policy 0, policy_version 3216 (0.0007) [2023-12-26 15:23:23,583][105620] Updated weights for policy 1, policy_version 3125 (0.0007) [2023-12-26 15:23:23,606][105692] Updated weights for policy 0, policy_version 3226 (0.0007) [2023-12-26 15:23:23,633][105620] Updated weights for policy 1, policy_version 3135 (0.0007) [2023-12-26 15:23:24,317][105620] Updated weights for policy 1, policy_version 3145 (0.0008) [2023-12-26 15:23:24,367][105620] Updated weights for policy 1, policy_version 3155 (0.0008) [2023-12-26 15:23:24,428][105692] Updated weights for policy 0, policy_version 3236 (0.0008) [2023-12-26 15:23:24,433][105620] Updated weights for policy 1, policy_version 3165 (0.0009) [2023-12-26 15:23:24,473][105692] Updated weights for policy 0, policy_version 3246 (0.0008) [2023-12-26 15:23:24,532][105692] Updated weights for policy 0, policy_version 3256 (0.0009) [2023-12-26 15:23:25,126][105620] Updated weights for policy 1, policy_version 3175 (0.0009) [2023-12-26 15:23:25,173][105620] Updated weights for policy 1, policy_version 3185 (0.0008) [2023-12-26 15:23:25,219][105620] Updated weights for policy 1, policy_version 3195 (0.0009) [2023-12-26 15:23:25,316][105692] Updated weights for policy 0, policy_version 3266 (0.0009) [2023-12-26 15:23:25,369][105692] Updated weights for policy 0, policy_version 3276 (0.0009) [2023-12-26 15:23:25,419][105692] Updated weights for policy 0, policy_version 3286 (0.0009) [2023-12-26 15:23:25,471][105692] Updated weights for policy 0, policy_version 3296 (0.0009) [2023-12-26 15:23:25,967][105620] Updated weights for policy 1, policy_version 3205 (0.0009) [2023-12-26 15:23:26,032][105620] Updated weights for policy 1, policy_version 3215 (0.0008) [2023-12-26 15:23:26,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19114.6, 300 sec: 18477.5). Total num frames: 1662976. Throughput: 0: 9614.9, 1: 9629.6. Samples: 1676696. Policy #0 lag: (min: 29.0, avg: 31.0, max: 61.0) [2023-12-26 15:23:26,063][104569] Avg episode reward: [(0, '445.491'), (1, '418.933')] [2023-12-26 15:23:26,063][105585] Saving new best policy, reward=445.491! [2023-12-26 15:23:26,100][105620] Updated weights for policy 1, policy_version 3225 (0.0008) [2023-12-26 15:23:26,143][105586] Saving new best policy, reward=418.933! [2023-12-26 15:23:26,265][105692] Updated weights for policy 0, policy_version 3306 (0.0009) [2023-12-26 15:23:26,322][105692] Updated weights for policy 0, policy_version 3316 (0.0008) [2023-12-26 15:23:26,369][105692] Updated weights for policy 0, policy_version 3326 (0.0008) [2023-12-26 15:23:26,746][105620] Updated weights for policy 1, policy_version 3235 (0.0009) [2023-12-26 15:23:26,810][105620] Updated weights for policy 1, policy_version 3245 (0.0008) [2023-12-26 15:23:26,867][105620] Updated weights for policy 1, policy_version 3255 (0.0006) [2023-12-26 15:23:27,171][105692] Updated weights for policy 0, policy_version 3336 (0.0006) [2023-12-26 15:23:27,234][105692] Updated weights for policy 0, policy_version 3346 (0.0005) [2023-12-26 15:23:27,287][105692] Updated weights for policy 0, policy_version 3356 (0.0005) [2023-12-26 15:23:27,565][105620] Updated weights for policy 1, policy_version 3265 (0.0006) [2023-12-26 15:23:27,627][105620] Updated weights for policy 1, policy_version 3275 (0.0005) [2023-12-26 15:23:27,700][105620] Updated weights for policy 1, policy_version 3285 (0.0008) [2023-12-26 15:23:27,760][105620] Updated weights for policy 1, policy_version 3295 (0.0011) [2023-12-26 15:23:27,953][105692] Updated weights for policy 0, policy_version 3366 (0.0005) [2023-12-26 15:23:28,011][105692] Updated weights for policy 0, policy_version 3376 (0.0006) [2023-12-26 15:23:28,054][105692] Updated weights for policy 0, policy_version 3386 (0.0007) [2023-12-26 15:23:28,366][105620] Updated weights for policy 1, policy_version 3305 (0.0008) [2023-12-26 15:23:28,428][105620] Updated weights for policy 1, policy_version 3315 (0.0010) [2023-12-26 15:23:28,486][105620] Updated weights for policy 1, policy_version 3325 (0.0010) [2023-12-26 15:23:28,840][105692] Updated weights for policy 0, policy_version 3396 (0.0009) [2023-12-26 15:23:28,888][105692] Updated weights for policy 0, policy_version 3406 (0.0009) [2023-12-26 15:23:28,943][105692] Updated weights for policy 0, policy_version 3416 (0.0009) [2023-12-26 15:23:29,109][105620] Updated weights for policy 1, policy_version 3335 (0.0007) [2023-12-26 15:23:29,157][105620] Updated weights for policy 1, policy_version 3345 (0.0005) [2023-12-26 15:23:29,204][105620] Updated weights for policy 1, policy_version 3355 (0.0005) [2023-12-26 15:23:29,775][105692] Updated weights for policy 0, policy_version 3426 (0.0009) [2023-12-26 15:23:29,839][105692] Updated weights for policy 0, policy_version 3436 (0.0008) [2023-12-26 15:23:29,896][105692] Updated weights for policy 0, policy_version 3446 (0.0008) [2023-12-26 15:23:29,906][105620] Updated weights for policy 1, policy_version 3365 (0.0009) [2023-12-26 15:23:29,957][105692] Updated weights for policy 0, policy_version 3456 (0.0007) [2023-12-26 15:23:29,976][105620] Updated weights for policy 1, policy_version 3375 (0.0009) [2023-12-26 15:23:30,047][105620] Updated weights for policy 1, policy_version 3385 (0.0009) [2023-12-26 15:23:30,631][105620] Updated weights for policy 1, policy_version 3395 (0.0007) [2023-12-26 15:23:30,687][105692] Updated weights for policy 0, policy_version 3466 (0.0005) [2023-12-26 15:23:30,693][105620] Updated weights for policy 1, policy_version 3405 (0.0007) [2023-12-26 15:23:30,741][105692] Updated weights for policy 0, policy_version 3476 (0.0005) [2023-12-26 15:23:30,750][105620] Updated weights for policy 1, policy_version 3415 (0.0010) [2023-12-26 15:23:30,787][105692] Updated weights for policy 0, policy_version 3486 (0.0005) [2023-12-26 15:23:31,062][104569] Fps is (10 sec: 19660.0, 60 sec: 19387.6, 300 sec: 18626.0). Total num frames: 1769472. Throughput: 0: 9589.1, 1: 9712.6. Samples: 1736036. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-26 15:23:31,063][104569] Avg episode reward: [(0, '524.360'), (1, '526.956')] [2023-12-26 15:23:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000003488_892928.pth... [2023-12-26 15:23:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000003424_876544.pth... [2023-12-26 15:23:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000002400_614400.pth [2023-12-26 15:23:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000002272_581632.pth [2023-12-26 15:23:31,072][105585] Saving new best policy, reward=524.360! [2023-12-26 15:23:31,072][105586] Saving new best policy, reward=526.956! [2023-12-26 15:23:31,406][105692] Updated weights for policy 0, policy_version 3496 (0.0008) [2023-12-26 15:23:31,463][105620] Updated weights for policy 1, policy_version 3425 (0.0010) [2023-12-26 15:23:31,466][105692] Updated weights for policy 0, policy_version 3506 (0.0006) [2023-12-26 15:23:31,522][105692] Updated weights for policy 0, policy_version 3516 (0.0007) [2023-12-26 15:23:31,526][105620] Updated weights for policy 1, policy_version 3435 (0.0011) [2023-12-26 15:23:31,592][105620] Updated weights for policy 1, policy_version 3445 (0.0011) [2023-12-26 15:23:31,655][105620] Updated weights for policy 1, policy_version 3455 (0.0011) [2023-12-26 15:23:32,207][105692] Updated weights for policy 0, policy_version 3526 (0.0008) [2023-12-26 15:23:32,262][105692] Updated weights for policy 0, policy_version 3536 (0.0010) [2023-12-26 15:23:32,318][105692] Updated weights for policy 0, policy_version 3546 (0.0008) [2023-12-26 15:23:32,401][105620] Updated weights for policy 1, policy_version 3465 (0.0010) [2023-12-26 15:23:32,460][105620] Updated weights for policy 1, policy_version 3475 (0.0010) [2023-12-26 15:23:32,515][105620] Updated weights for policy 1, policy_version 3485 (0.0010) [2023-12-26 15:23:33,050][105692] Updated weights for policy 0, policy_version 3556 (0.0008) [2023-12-26 15:23:33,104][105692] Updated weights for policy 0, policy_version 3566 (0.0008) [2023-12-26 15:23:33,161][105692] Updated weights for policy 0, policy_version 3576 (0.0007) [2023-12-26 15:23:33,260][105620] Updated weights for policy 1, policy_version 3495 (0.0010) [2023-12-26 15:23:33,308][105620] Updated weights for policy 1, policy_version 3505 (0.0010) [2023-12-26 15:23:33,365][105620] Updated weights for policy 1, policy_version 3515 (0.0010) [2023-12-26 15:23:33,826][105692] Updated weights for policy 0, policy_version 3586 (0.0006) [2023-12-26 15:23:33,877][105692] Updated weights for policy 0, policy_version 3596 (0.0007) [2023-12-26 15:23:33,930][105692] Updated weights for policy 0, policy_version 3606 (0.0009) [2023-12-26 15:23:33,986][105692] Updated weights for policy 0, policy_version 3616 (0.0006) [2023-12-26 15:23:34,126][105620] Updated weights for policy 1, policy_version 3525 (0.0010) [2023-12-26 15:23:34,188][105620] Updated weights for policy 1, policy_version 3535 (0.0011) [2023-12-26 15:23:34,248][105620] Updated weights for policy 1, policy_version 3545 (0.0010) [2023-12-26 15:23:34,736][105692] Updated weights for policy 0, policy_version 3626 (0.0008) [2023-12-26 15:23:34,793][105692] Updated weights for policy 0, policy_version 3636 (0.0008) [2023-12-26 15:23:34,849][105692] Updated weights for policy 0, policy_version 3646 (0.0008) [2023-12-26 15:23:34,994][105620] Updated weights for policy 1, policy_version 3555 (0.0010) [2023-12-26 15:23:35,049][105620] Updated weights for policy 1, policy_version 3565 (0.0010) [2023-12-26 15:23:35,111][105620] Updated weights for policy 1, policy_version 3575 (0.0010) [2023-12-26 15:23:35,603][105692] Updated weights for policy 0, policy_version 3656 (0.0005) [2023-12-26 15:23:35,649][105692] Updated weights for policy 0, policy_version 3666 (0.0005) [2023-12-26 15:23:35,705][105692] Updated weights for policy 0, policy_version 3676 (0.0005) [2023-12-26 15:23:35,860][105620] Updated weights for policy 1, policy_version 3585 (0.0010) [2023-12-26 15:23:35,915][105620] Updated weights for policy 1, policy_version 3595 (0.0010) [2023-12-26 15:23:35,967][105620] Updated weights for policy 1, policy_version 3605 (0.0010) [2023-12-26 15:23:36,018][105620] Updated weights for policy 1, policy_version 3615 (0.0010) [2023-12-26 15:23:36,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19387.7, 300 sec: 18677.8). Total num frames: 1867776. Throughput: 0: 9570.3, 1: 9805.9. Samples: 1852932. Policy #0 lag: (min: 31.0, avg: 32.6, max: 53.0) [2023-12-26 15:23:36,062][104569] Avg episode reward: [(0, '654.873'), (1, '513.197')] [2023-12-26 15:23:36,063][105585] Saving new best policy, reward=654.873! [2023-12-26 15:23:36,373][105692] Updated weights for policy 0, policy_version 3686 (0.0010) [2023-12-26 15:23:36,442][105692] Updated weights for policy 0, policy_version 3696 (0.0011) [2023-12-26 15:23:36,509][105692] Updated weights for policy 0, policy_version 3706 (0.0011) [2023-12-26 15:23:36,803][105620] Updated weights for policy 1, policy_version 3625 (0.0010) [2023-12-26 15:23:36,865][105620] Updated weights for policy 1, policy_version 3635 (0.0011) [2023-12-26 15:23:36,931][105620] Updated weights for policy 1, policy_version 3645 (0.0010) [2023-12-26 15:23:37,259][105692] Updated weights for policy 0, policy_version 3716 (0.0011) [2023-12-26 15:23:37,329][105692] Updated weights for policy 0, policy_version 3726 (0.0011) [2023-12-26 15:23:37,392][105692] Updated weights for policy 0, policy_version 3736 (0.0011) [2023-12-26 15:23:37,567][105620] Updated weights for policy 1, policy_version 3655 (0.0007) [2023-12-26 15:23:37,632][105620] Updated weights for policy 1, policy_version 3665 (0.0006) [2023-12-26 15:23:37,682][105620] Updated weights for policy 1, policy_version 3675 (0.0005) [2023-12-26 15:23:38,137][105692] Updated weights for policy 0, policy_version 3746 (0.0010) [2023-12-26 15:23:38,195][105692] Updated weights for policy 0, policy_version 3756 (0.0010) [2023-12-26 15:23:38,251][105692] Updated weights for policy 0, policy_version 3766 (0.0011) [2023-12-26 15:23:38,295][105692] Updated weights for policy 0, policy_version 3776 (0.0010) [2023-12-26 15:23:38,392][105620] Updated weights for policy 1, policy_version 3685 (0.0007) [2023-12-26 15:23:38,451][105620] Updated weights for policy 1, policy_version 3695 (0.0008) [2023-12-26 15:23:38,500][105620] Updated weights for policy 1, policy_version 3705 (0.0008) [2023-12-26 15:23:39,051][105692] Updated weights for policy 0, policy_version 3786 (0.0008) [2023-12-26 15:23:39,096][105692] Updated weights for policy 0, policy_version 3796 (0.0008) [2023-12-26 15:23:39,144][105692] Updated weights for policy 0, policy_version 3806 (0.0008) [2023-12-26 15:23:39,271][105620] Updated weights for policy 1, policy_version 3715 (0.0009) [2023-12-26 15:23:39,337][105620] Updated weights for policy 1, policy_version 3725 (0.0011) [2023-12-26 15:23:39,405][105620] Updated weights for policy 1, policy_version 3735 (0.0010) [2023-12-26 15:23:39,981][105692] Updated weights for policy 0, policy_version 3816 (0.0008) [2023-12-26 15:23:40,033][105692] Updated weights for policy 0, policy_version 3826 (0.0008) [2023-12-26 15:23:40,046][105620] Updated weights for policy 1, policy_version 3745 (0.0009) [2023-12-26 15:23:40,081][105692] Updated weights for policy 0, policy_version 3836 (0.0008) [2023-12-26 15:23:40,109][105620] Updated weights for policy 1, policy_version 3755 (0.0011) [2023-12-26 15:23:40,176][105620] Updated weights for policy 1, policy_version 3765 (0.0011) [2023-12-26 15:23:40,238][105620] Updated weights for policy 1, policy_version 3775 (0.0011) [2023-12-26 15:23:40,870][105692] Updated weights for policy 0, policy_version 3846 (0.0008) [2023-12-26 15:23:40,935][105692] Updated weights for policy 0, policy_version 3856 (0.0006) [2023-12-26 15:23:40,955][105620] Updated weights for policy 1, policy_version 3785 (0.0011) [2023-12-26 15:23:41,000][105692] Updated weights for policy 0, policy_version 3866 (0.0008) [2023-12-26 15:23:41,015][105620] Updated weights for policy 1, policy_version 3795 (0.0010) [2023-12-26 15:23:41,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19251.2, 300 sec: 18646.5). Total num frames: 1957888. Throughput: 0: 9528.0, 1: 9774.9. Samples: 1967368. Policy #0 lag: (min: 17.0, avg: 32.6, max: 49.0) [2023-12-26 15:23:41,063][104569] Avg episode reward: [(0, '768.902'), (1, '620.567')] [2023-12-26 15:23:41,064][105585] Saving new best policy, reward=768.902! [2023-12-26 15:23:41,082][105620] Updated weights for policy 1, policy_version 3805 (0.0008) [2023-12-26 15:23:41,101][105586] Saving new best policy, reward=620.567! [2023-12-26 15:23:41,714][105692] Updated weights for policy 0, policy_version 3876 (0.0010) [2023-12-26 15:23:41,785][105692] Updated weights for policy 0, policy_version 3886 (0.0011) [2023-12-26 15:23:41,848][105692] Updated weights for policy 0, policy_version 3896 (0.0010) [2023-12-26 15:23:41,863][105620] Updated weights for policy 1, policy_version 3815 (0.0006) [2023-12-26 15:23:41,911][105620] Updated weights for policy 1, policy_version 3825 (0.0008) [2023-12-26 15:23:41,965][105620] Updated weights for policy 1, policy_version 3836 (0.0009) [2023-12-26 15:23:42,527][105692] Updated weights for policy 0, policy_version 3906 (0.0010) [2023-12-26 15:23:42,583][105692] Updated weights for policy 0, policy_version 3916 (0.0010) [2023-12-26 15:23:42,642][105692] Updated weights for policy 0, policy_version 3926 (0.0010) [2023-12-26 15:23:42,700][105692] Updated weights for policy 0, policy_version 3936 (0.0010) [2023-12-26 15:23:42,811][105620] Updated weights for policy 1, policy_version 3846 (0.0010) [2023-12-26 15:23:42,863][105620] Updated weights for policy 1, policy_version 3856 (0.0009) [2023-12-26 15:23:42,930][105620] Updated weights for policy 1, policy_version 3866 (0.0006) [2023-12-26 15:23:43,319][105692] Updated weights for policy 0, policy_version 3946 (0.0006) [2023-12-26 15:23:43,387][105692] Updated weights for policy 0, policy_version 3956 (0.0005) [2023-12-26 15:23:43,451][105692] Updated weights for policy 0, policy_version 3966 (0.0005) [2023-12-26 15:23:43,575][105620] Updated weights for policy 1, policy_version 3876 (0.0008) [2023-12-26 15:23:43,643][105620] Updated weights for policy 1, policy_version 3886 (0.0010) [2023-12-26 15:23:43,688][105620] Updated weights for policy 1, policy_version 3896 (0.0010) [2023-12-26 15:23:43,994][105692] Updated weights for policy 0, policy_version 3976 (0.0007) [2023-12-26 15:23:44,055][105692] Updated weights for policy 0, policy_version 3986 (0.0008) [2023-12-26 15:23:44,120][105692] Updated weights for policy 0, policy_version 3996 (0.0008) [2023-12-26 15:23:44,354][105620] Updated weights for policy 1, policy_version 3906 (0.0010) [2023-12-26 15:23:44,403][105620] Updated weights for policy 1, policy_version 3916 (0.0010) [2023-12-26 15:23:44,461][105620] Updated weights for policy 1, policy_version 3926 (0.0010) [2023-12-26 15:23:44,531][105620] Updated weights for policy 1, policy_version 3936 (0.0005) [2023-12-26 15:23:44,936][105692] Updated weights for policy 0, policy_version 4006 (0.0006) [2023-12-26 15:23:44,993][105692] Updated weights for policy 0, policy_version 4016 (0.0007) [2023-12-26 15:23:45,052][105692] Updated weights for policy 0, policy_version 4026 (0.0011) [2023-12-26 15:23:45,245][105620] Updated weights for policy 1, policy_version 3946 (0.0008) [2023-12-26 15:23:45,307][105620] Updated weights for policy 1, policy_version 3956 (0.0008) [2023-12-26 15:23:45,369][105620] Updated weights for policy 1, policy_version 3966 (0.0008) [2023-12-26 15:23:45,725][105692] Updated weights for policy 0, policy_version 4036 (0.0010) [2023-12-26 15:23:45,785][105692] Updated weights for policy 0, policy_version 4046 (0.0010) [2023-12-26 15:23:45,843][105692] Updated weights for policy 0, policy_version 4056 (0.0010) [2023-12-26 15:23:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.8, 300 sec: 18692.7). Total num frames: 2056192. Throughput: 0: 9532.7, 1: 9742.4. Samples: 2026564. Policy #0 lag: (min: 9.0, avg: 36.5, max: 41.0) [2023-12-26 15:23:46,062][104569] Avg episode reward: [(0, '1010.429'), (1, '775.162')] [2023-12-26 15:23:46,065][105620] Updated weights for policy 1, policy_version 3976 (0.0008) [2023-12-26 15:23:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000004064_1040384.pth... [2023-12-26 15:23:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000002976_761856.pth [2023-12-26 15:23:46,073][105585] Saving new best policy, reward=1010.429! [2023-12-26 15:23:46,120][105620] Updated weights for policy 1, policy_version 3986 (0.0005) [2023-12-26 15:23:46,170][105620] Updated weights for policy 1, policy_version 3996 (0.0008) [2023-12-26 15:23:46,190][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000004000_1024000.pth... [2023-12-26 15:23:46,196][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000002816_720896.pth [2023-12-26 15:23:46,196][105586] Saving new best policy, reward=775.162! [2023-12-26 15:23:46,497][105692] Updated weights for policy 0, policy_version 4066 (0.0010) [2023-12-26 15:23:46,556][105692] Updated weights for policy 0, policy_version 4076 (0.0008) [2023-12-26 15:23:46,615][105692] Updated weights for policy 0, policy_version 4086 (0.0006) [2023-12-26 15:23:46,670][105692] Updated weights for policy 0, policy_version 4096 (0.0005) [2023-12-26 15:23:46,817][105620] Updated weights for policy 1, policy_version 4006 (0.0008) [2023-12-26 15:23:46,888][105620] Updated weights for policy 1, policy_version 4016 (0.0006) [2023-12-26 15:23:46,960][105620] Updated weights for policy 1, policy_version 4026 (0.0005) [2023-12-26 15:23:47,323][105692] Updated weights for policy 0, policy_version 4106 (0.0005) [2023-12-26 15:23:47,378][105692] Updated weights for policy 0, policy_version 4116 (0.0006) [2023-12-26 15:23:47,439][105692] Updated weights for policy 0, policy_version 4126 (0.0005) [2023-12-26 15:23:47,513][105620] Updated weights for policy 1, policy_version 4036 (0.0005) [2023-12-26 15:23:47,564][105620] Updated weights for policy 1, policy_version 4046 (0.0005) [2023-12-26 15:23:47,610][105620] Updated weights for policy 1, policy_version 4056 (0.0006) [2023-12-26 15:23:48,010][105692] Updated weights for policy 0, policy_version 4136 (0.0005) [2023-12-26 15:23:48,069][105692] Updated weights for policy 0, policy_version 4146 (0.0006) [2023-12-26 15:23:48,125][105692] Updated weights for policy 0, policy_version 4156 (0.0007) [2023-12-26 15:23:48,182][105620] Updated weights for policy 1, policy_version 4066 (0.0005) [2023-12-26 15:23:48,236][105620] Updated weights for policy 1, policy_version 4076 (0.0005) [2023-12-26 15:23:48,282][105620] Updated weights for policy 1, policy_version 4086 (0.0005) [2023-12-26 15:23:48,330][105620] Updated weights for policy 1, policy_version 4096 (0.0006) [2023-12-26 15:23:48,796][105692] Updated weights for policy 0, policy_version 4166 (0.0007) [2023-12-26 15:23:48,855][105692] Updated weights for policy 0, policy_version 4176 (0.0009) [2023-12-26 15:23:48,909][105692] Updated weights for policy 0, policy_version 4186 (0.0009) [2023-12-26 15:23:49,048][105620] Updated weights for policy 1, policy_version 4106 (0.0009) [2023-12-26 15:23:49,112][105620] Updated weights for policy 1, policy_version 4116 (0.0008) [2023-12-26 15:23:49,180][105620] Updated weights for policy 1, policy_version 4126 (0.0007) [2023-12-26 15:23:49,673][105692] Updated weights for policy 0, policy_version 4196 (0.0010) [2023-12-26 15:23:49,735][105692] Updated weights for policy 0, policy_version 4206 (0.0008) [2023-12-26 15:23:49,745][105620] Updated weights for policy 1, policy_version 4136 (0.0007) [2023-12-26 15:23:49,802][105692] Updated weights for policy 0, policy_version 4216 (0.0009) [2023-12-26 15:23:49,808][105620] Updated weights for policy 1, policy_version 4146 (0.0006) [2023-12-26 15:23:49,865][105620] Updated weights for policy 1, policy_version 4156 (0.0007) [2023-12-26 15:23:50,551][105692] Updated weights for policy 0, policy_version 4226 (0.0007) [2023-12-26 15:23:50,612][105692] Updated weights for policy 0, policy_version 4236 (0.0007) [2023-12-26 15:23:50,614][105620] Updated weights for policy 1, policy_version 4166 (0.0010) [2023-12-26 15:23:50,668][105620] Updated weights for policy 1, policy_version 4176 (0.0010) [2023-12-26 15:23:50,675][105692] Updated weights for policy 0, policy_version 4246 (0.0007) [2023-12-26 15:23:50,732][105620] Updated weights for policy 1, policy_version 4186 (0.0006) [2023-12-26 15:23:50,732][105692] Updated weights for policy 0, policy_version 4256 (0.0009) [2023-12-26 15:23:51,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 18806.0). Total num frames: 2162688. Throughput: 0: 9610.7, 1: 9865.3. Samples: 2151644. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:23:51,062][104569] Avg episode reward: [(0, '1139.866'), (1, '1124.102')] [2023-12-26 15:23:51,063][105585] Saving new best policy, reward=1139.866! [2023-12-26 15:23:51,063][105586] Saving new best policy, reward=1124.102! [2023-12-26 15:23:51,404][105620] Updated weights for policy 1, policy_version 4196 (0.0007) [2023-12-26 15:23:51,466][105620] Updated weights for policy 1, policy_version 4206 (0.0009) [2023-12-26 15:23:51,530][105620] Updated weights for policy 1, policy_version 4216 (0.0008) [2023-12-26 15:23:51,533][105692] Updated weights for policy 0, policy_version 4266 (0.0006) [2023-12-26 15:23:51,590][105692] Updated weights for policy 0, policy_version 4276 (0.0008) [2023-12-26 15:23:51,654][105692] Updated weights for policy 0, policy_version 4286 (0.0009) [2023-12-26 15:23:52,310][105620] Updated weights for policy 1, policy_version 4226 (0.0006) [2023-12-26 15:23:52,377][105620] Updated weights for policy 1, policy_version 4236 (0.0009) [2023-12-26 15:23:52,422][105692] Updated weights for policy 0, policy_version 4296 (0.0007) [2023-12-26 15:23:52,431][105620] Updated weights for policy 1, policy_version 4246 (0.0008) [2023-12-26 15:23:52,483][105692] Updated weights for policy 0, policy_version 4306 (0.0008) [2023-12-26 15:23:52,489][105620] Updated weights for policy 1, policy_version 4256 (0.0007) [2023-12-26 15:23:52,536][105692] Updated weights for policy 0, policy_version 4316 (0.0008) [2023-12-26 15:23:53,262][105620] Updated weights for policy 1, policy_version 4266 (0.0008) [2023-12-26 15:23:53,276][105692] Updated weights for policy 0, policy_version 4326 (0.0009) [2023-12-26 15:23:53,319][105620] Updated weights for policy 1, policy_version 4276 (0.0007) [2023-12-26 15:23:53,325][105692] Updated weights for policy 0, policy_version 4336 (0.0006) [2023-12-26 15:23:53,367][105692] Updated weights for policy 0, policy_version 4346 (0.0007) [2023-12-26 15:23:53,377][105620] Updated weights for policy 1, policy_version 4286 (0.0008) [2023-12-26 15:23:54,048][105620] Updated weights for policy 1, policy_version 4296 (0.0007) [2023-12-26 15:23:54,067][105692] Updated weights for policy 0, policy_version 4356 (0.0006) [2023-12-26 15:23:54,110][105620] Updated weights for policy 1, policy_version 4306 (0.0010) [2023-12-26 15:23:54,121][105692] Updated weights for policy 0, policy_version 4366 (0.0007) [2023-12-26 15:23:54,171][105620] Updated weights for policy 1, policy_version 4316 (0.0010) [2023-12-26 15:23:54,180][105692] Updated weights for policy 0, policy_version 4376 (0.0007) [2023-12-26 15:23:54,803][105692] Updated weights for policy 0, policy_version 4386 (0.0008) [2023-12-26 15:23:54,870][105692] Updated weights for policy 0, policy_version 4396 (0.0007) [2023-12-26 15:23:54,895][105620] Updated weights for policy 1, policy_version 4326 (0.0009) [2023-12-26 15:23:54,928][105692] Updated weights for policy 0, policy_version 4406 (0.0005) [2023-12-26 15:23:54,953][105620] Updated weights for policy 1, policy_version 4336 (0.0010) [2023-12-26 15:23:54,988][105692] Updated weights for policy 0, policy_version 4416 (0.0008) [2023-12-26 15:23:55,022][105620] Updated weights for policy 1, policy_version 4346 (0.0006) [2023-12-26 15:23:55,670][105692] Updated weights for policy 0, policy_version 4426 (0.0005) [2023-12-26 15:23:55,696][105620] Updated weights for policy 1, policy_version 4356 (0.0008) [2023-12-26 15:23:55,726][105692] Updated weights for policy 0, policy_version 4436 (0.0006) [2023-12-26 15:23:55,759][105620] Updated weights for policy 1, policy_version 4366 (0.0006) [2023-12-26 15:23:55,782][105692] Updated weights for policy 0, policy_version 4446 (0.0005) [2023-12-26 15:23:55,811][105620] Updated weights for policy 1, policy_version 4376 (0.0005) [2023-12-26 15:23:56,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 18841.6). Total num frames: 2260992. Throughput: 0: 9575.1, 1: 9879.6. Samples: 2266924. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 15:23:56,063][104569] Avg episode reward: [(0, '1603.800'), (1, '1280.897')] [2023-12-26 15:23:56,063][105585] Saving new best policy, reward=1603.800! [2023-12-26 15:23:56,064][105586] Saving new best policy, reward=1280.897! [2023-12-26 15:23:56,444][105620] Updated weights for policy 1, policy_version 4386 (0.0006) [2023-12-26 15:23:56,494][105692] Updated weights for policy 0, policy_version 4456 (0.0006) [2023-12-26 15:23:56,504][105620] Updated weights for policy 1, policy_version 4396 (0.0011) [2023-12-26 15:23:56,557][105692] Updated weights for policy 0, policy_version 4466 (0.0006) [2023-12-26 15:23:56,559][105620] Updated weights for policy 1, policy_version 4406 (0.0010) [2023-12-26 15:23:56,614][105620] Updated weights for policy 1, policy_version 4416 (0.0010) [2023-12-26 15:23:56,623][105692] Updated weights for policy 0, policy_version 4476 (0.0005) [2023-12-26 15:23:57,299][105692] Updated weights for policy 0, policy_version 4486 (0.0007) [2023-12-26 15:23:57,351][105692] Updated weights for policy 0, policy_version 4496 (0.0008) [2023-12-26 15:23:57,355][105620] Updated weights for policy 1, policy_version 4426 (0.0008) [2023-12-26 15:23:57,408][105692] Updated weights for policy 0, policy_version 4506 (0.0007) [2023-12-26 15:23:57,414][105620] Updated weights for policy 1, policy_version 4436 (0.0010) [2023-12-26 15:23:57,474][105620] Updated weights for policy 1, policy_version 4446 (0.0010) [2023-12-26 15:23:58,073][105692] Updated weights for policy 0, policy_version 4516 (0.0007) [2023-12-26 15:23:58,134][105692] Updated weights for policy 0, policy_version 4526 (0.0009) [2023-12-26 15:23:58,179][105620] Updated weights for policy 1, policy_version 4456 (0.0008) [2023-12-26 15:23:58,194][105692] Updated weights for policy 0, policy_version 4536 (0.0005) [2023-12-26 15:23:58,233][105620] Updated weights for policy 1, policy_version 4466 (0.0007) [2023-12-26 15:23:58,296][105620] Updated weights for policy 1, policy_version 4476 (0.0007) [2023-12-26 15:23:58,987][105692] Updated weights for policy 0, policy_version 4546 (0.0009) [2023-12-26 15:23:59,047][105692] Updated weights for policy 0, policy_version 4556 (0.0008) [2023-12-26 15:23:59,105][105692] Updated weights for policy 0, policy_version 4566 (0.0008) [2023-12-26 15:23:59,141][105620] Updated weights for policy 1, policy_version 4486 (0.0008) [2023-12-26 15:23:59,170][105692] Updated weights for policy 0, policy_version 4576 (0.0007) [2023-12-26 15:23:59,203][105620] Updated weights for policy 1, policy_version 4496 (0.0010) [2023-12-26 15:23:59,276][105620] Updated weights for policy 1, policy_version 4506 (0.0009) [2023-12-26 15:23:59,914][105692] Updated weights for policy 0, policy_version 4586 (0.0008) [2023-12-26 15:23:59,980][105692] Updated weights for policy 0, policy_version 4596 (0.0008) [2023-12-26 15:24:00,024][105620] Updated weights for policy 1, policy_version 4516 (0.0009) [2023-12-26 15:24:00,040][105692] Updated weights for policy 0, policy_version 4606 (0.0009) [2023-12-26 15:24:00,085][105620] Updated weights for policy 1, policy_version 4526 (0.0010) [2023-12-26 15:24:00,143][105620] Updated weights for policy 1, policy_version 4536 (0.0010) [2023-12-26 15:24:00,800][105692] Updated weights for policy 0, policy_version 4616 (0.0008) [2023-12-26 15:24:00,851][105620] Updated weights for policy 1, policy_version 4546 (0.0009) [2023-12-26 15:24:00,863][105692] Updated weights for policy 0, policy_version 4626 (0.0009) [2023-12-26 15:24:00,899][105620] Updated weights for policy 1, policy_version 4556 (0.0005) [2023-12-26 15:24:00,923][105692] Updated weights for policy 0, policy_version 4636 (0.0008) [2023-12-26 15:24:00,952][105620] Updated weights for policy 1, policy_version 4566 (0.0005) [2023-12-26 15:24:01,002][105620] Updated weights for policy 1, policy_version 4576 (0.0005) [2023-12-26 15:24:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 18874.4). Total num frames: 2359296. Throughput: 0: 9590.8, 1: 9882.9. Samples: 2325008. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-26 15:24:01,063][104569] Avg episode reward: [(0, '1901.051'), (1, '1417.470')] [2023-12-26 15:24:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000004640_1187840.pth... [2023-12-26 15:24:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000004576_1171456.pth... [2023-12-26 15:24:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000003488_892928.pth [2023-12-26 15:24:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000003424_876544.pth [2023-12-26 15:24:01,072][105585] Saving new best policy, reward=1901.051! [2023-12-26 15:24:01,072][105586] Saving new best policy, reward=1417.470! [2023-12-26 15:24:01,664][105692] Updated weights for policy 0, policy_version 4646 (0.0010) [2023-12-26 15:24:01,715][105620] Updated weights for policy 1, policy_version 4586 (0.0010) [2023-12-26 15:24:01,731][105692] Updated weights for policy 0, policy_version 4656 (0.0012) [2023-12-26 15:24:01,779][105620] Updated weights for policy 1, policy_version 4596 (0.0010) [2023-12-26 15:24:01,793][105692] Updated weights for policy 0, policy_version 4666 (0.0011) [2023-12-26 15:24:01,838][105620] Updated weights for policy 1, policy_version 4606 (0.0010) [2023-12-26 15:24:02,468][105692] Updated weights for policy 0, policy_version 4676 (0.0010) [2023-12-26 15:24:02,523][105692] Updated weights for policy 0, policy_version 4686 (0.0010) [2023-12-26 15:24:02,537][105620] Updated weights for policy 1, policy_version 4616 (0.0007) [2023-12-26 15:24:02,578][105692] Updated weights for policy 0, policy_version 4696 (0.0011) [2023-12-26 15:24:02,592][105620] Updated weights for policy 1, policy_version 4626 (0.0005) [2023-12-26 15:24:02,654][105620] Updated weights for policy 1, policy_version 4636 (0.0007) [2023-12-26 15:24:03,179][105692] Updated weights for policy 0, policy_version 4706 (0.0009) [2023-12-26 15:24:03,246][105692] Updated weights for policy 0, policy_version 4716 (0.0005) [2023-12-26 15:24:03,300][105692] Updated weights for policy 0, policy_version 4726 (0.0006) [2023-12-26 15:24:03,355][105692] Updated weights for policy 0, policy_version 4736 (0.0005) [2023-12-26 15:24:03,427][105620] Updated weights for policy 1, policy_version 4646 (0.0007) [2023-12-26 15:24:03,485][105620] Updated weights for policy 1, policy_version 4656 (0.0005) [2023-12-26 15:24:03,530][105620] Updated weights for policy 1, policy_version 4666 (0.0005) [2023-12-26 15:24:03,953][105692] Updated weights for policy 0, policy_version 4746 (0.0005) [2023-12-26 15:24:04,015][105692] Updated weights for policy 0, policy_version 4756 (0.0006) [2023-12-26 15:24:04,072][105692] Updated weights for policy 0, policy_version 4766 (0.0005) [2023-12-26 15:24:04,111][105620] Updated weights for policy 1, policy_version 4676 (0.0007) [2023-12-26 15:24:04,174][105620] Updated weights for policy 1, policy_version 4686 (0.0009) [2023-12-26 15:24:04,236][105620] Updated weights for policy 1, policy_version 4696 (0.0009) [2023-12-26 15:24:04,722][105692] Updated weights for policy 0, policy_version 4776 (0.0007) [2023-12-26 15:24:04,790][105692] Updated weights for policy 0, policy_version 4786 (0.0011) [2023-12-26 15:24:04,860][105692] Updated weights for policy 0, policy_version 4796 (0.0010) [2023-12-26 15:24:04,946][105620] Updated weights for policy 1, policy_version 4706 (0.0009) [2023-12-26 15:24:05,013][105620] Updated weights for policy 1, policy_version 4716 (0.0006) [2023-12-26 15:24:05,072][105620] Updated weights for policy 1, policy_version 4726 (0.0008) [2023-12-26 15:24:05,124][105620] Updated weights for policy 1, policy_version 4736 (0.0008) [2023-12-26 15:24:05,553][105692] Updated weights for policy 0, policy_version 4806 (0.0008) [2023-12-26 15:24:05,612][105692] Updated weights for policy 0, policy_version 4816 (0.0009) [2023-12-26 15:24:05,673][105692] Updated weights for policy 0, policy_version 4826 (0.0010) [2023-12-26 15:24:05,770][105620] Updated weights for policy 1, policy_version 4746 (0.0010) [2023-12-26 15:24:05,827][105620] Updated weights for policy 1, policy_version 4756 (0.0005) [2023-12-26 15:24:05,878][105620] Updated weights for policy 1, policy_version 4766 (0.0005) [2023-12-26 15:24:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 18904.6). Total num frames: 2457600. Throughput: 0: 9641.3, 1: 9887.1. Samples: 2442960. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-26 15:24:06,062][104569] Avg episode reward: [(0, '2878.337'), (1, '1939.562')] [2023-12-26 15:24:06,063][105585] Saving new best policy, reward=2878.337! [2023-12-26 15:24:06,064][105586] Saving new best policy, reward=1939.562! [2023-12-26 15:24:06,405][105692] Updated weights for policy 0, policy_version 4836 (0.0009) [2023-12-26 15:24:06,475][105692] Updated weights for policy 0, policy_version 4846 (0.0011) [2023-12-26 15:24:06,514][105620] Updated weights for policy 1, policy_version 4776 (0.0006) [2023-12-26 15:24:06,539][105692] Updated weights for policy 0, policy_version 4856 (0.0011) [2023-12-26 15:24:06,573][105620] Updated weights for policy 1, policy_version 4786 (0.0006) [2023-12-26 15:24:06,629][105620] Updated weights for policy 1, policy_version 4796 (0.0008) [2023-12-26 15:24:07,270][105692] Updated weights for policy 0, policy_version 4866 (0.0011) [2023-12-26 15:24:07,325][105692] Updated weights for policy 0, policy_version 4876 (0.0011) [2023-12-26 15:24:07,385][105620] Updated weights for policy 1, policy_version 4806 (0.0007) [2023-12-26 15:24:07,391][105692] Updated weights for policy 0, policy_version 4886 (0.0011) [2023-12-26 15:24:07,446][105620] Updated weights for policy 1, policy_version 4816 (0.0005) [2023-12-26 15:24:07,447][105692] Updated weights for policy 0, policy_version 4896 (0.0011) [2023-12-26 15:24:07,507][105620] Updated weights for policy 1, policy_version 4826 (0.0006) [2023-12-26 15:24:08,132][105692] Updated weights for policy 0, policy_version 4906 (0.0010) [2023-12-26 15:24:08,178][105620] Updated weights for policy 1, policy_version 4836 (0.0005) [2023-12-26 15:24:08,181][105692] Updated weights for policy 0, policy_version 4916 (0.0010) [2023-12-26 15:24:08,233][105692] Updated weights for policy 0, policy_version 4926 (0.0010) [2023-12-26 15:24:08,240][105620] Updated weights for policy 1, policy_version 4846 (0.0005) [2023-12-26 15:24:08,302][105620] Updated weights for policy 1, policy_version 4856 (0.0006) [2023-12-26 15:24:08,873][105620] Updated weights for policy 1, policy_version 4866 (0.0008) [2023-12-26 15:24:08,935][105620] Updated weights for policy 1, policy_version 4876 (0.0005) [2023-12-26 15:24:08,992][105620] Updated weights for policy 1, policy_version 4886 (0.0005) [2023-12-26 15:24:09,044][105620] Updated weights for policy 1, policy_version 4896 (0.0005) [2023-12-26 15:24:09,050][105692] Updated weights for policy 0, policy_version 4936 (0.0011) [2023-12-26 15:24:09,115][105692] Updated weights for policy 0, policy_version 4946 (0.0010) [2023-12-26 15:24:09,172][105692] Updated weights for policy 0, policy_version 4956 (0.0010) [2023-12-26 15:24:09,663][105620] Updated weights for policy 1, policy_version 4906 (0.0008) [2023-12-26 15:24:09,730][105620] Updated weights for policy 1, policy_version 4916 (0.0006) [2023-12-26 15:24:09,790][105620] Updated weights for policy 1, policy_version 4926 (0.0006) [2023-12-26 15:24:09,916][105692] Updated weights for policy 0, policy_version 4966 (0.0010) [2023-12-26 15:24:09,990][105692] Updated weights for policy 0, policy_version 4976 (0.0011) [2023-12-26 15:24:10,056][105692] Updated weights for policy 0, policy_version 4986 (0.0011) [2023-12-26 15:24:10,446][105620] Updated weights for policy 1, policy_version 4936 (0.0007) [2023-12-26 15:24:10,513][105620] Updated weights for policy 1, policy_version 4946 (0.0005) [2023-12-26 15:24:10,567][105620] Updated weights for policy 1, policy_version 4956 (0.0005) [2023-12-26 15:24:10,752][105692] Updated weights for policy 0, policy_version 4996 (0.0009) [2023-12-26 15:24:10,814][105692] Updated weights for policy 0, policy_version 5006 (0.0005) [2023-12-26 15:24:10,880][105692] Updated weights for policy 0, policy_version 5016 (0.0005) [2023-12-26 15:24:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 18932.6). Total num frames: 2555904. Throughput: 0: 9737.7, 1: 9937.5. Samples: 2562080. Policy #0 lag: (min: 12.0, avg: 15.8, max: 44.0) [2023-12-26 15:24:11,063][104569] Avg episode reward: [(0, '3339.510'), (1, '2855.558')] [2023-12-26 15:24:11,063][105585] Saving new best policy, reward=3339.510! [2023-12-26 15:24:11,064][105586] Saving new best policy, reward=2855.558! [2023-12-26 15:24:11,359][105620] Updated weights for policy 1, policy_version 4966 (0.0009) [2023-12-26 15:24:11,420][105620] Updated weights for policy 1, policy_version 4976 (0.0008) [2023-12-26 15:24:11,476][105620] Updated weights for policy 1, policy_version 4986 (0.0009) [2023-12-26 15:24:11,504][105692] Updated weights for policy 0, policy_version 5026 (0.0005) [2023-12-26 15:24:11,574][105692] Updated weights for policy 0, policy_version 5036 (0.0007) [2023-12-26 15:24:11,632][105692] Updated weights for policy 0, policy_version 5046 (0.0009) [2023-12-26 15:24:11,709][105692] Updated weights for policy 0, policy_version 5056 (0.0008) [2023-12-26 15:24:12,244][105620] Updated weights for policy 1, policy_version 4996 (0.0008) [2023-12-26 15:24:12,310][105620] Updated weights for policy 1, policy_version 5006 (0.0009) [2023-12-26 15:24:12,375][105620] Updated weights for policy 1, policy_version 5016 (0.0009) [2023-12-26 15:24:12,470][105692] Updated weights for policy 0, policy_version 5066 (0.0009) [2023-12-26 15:24:12,532][105692] Updated weights for policy 0, policy_version 5076 (0.0009) [2023-12-26 15:24:12,581][105692] Updated weights for policy 0, policy_version 5086 (0.0008) [2023-12-26 15:24:13,070][105620] Updated weights for policy 1, policy_version 5026 (0.0008) [2023-12-26 15:24:13,127][105620] Updated weights for policy 1, policy_version 5036 (0.0005) [2023-12-26 15:24:13,182][105620] Updated weights for policy 1, policy_version 5046 (0.0008) [2023-12-26 15:24:13,237][105620] Updated weights for policy 1, policy_version 5056 (0.0008) [2023-12-26 15:24:13,330][105692] Updated weights for policy 0, policy_version 5096 (0.0006) [2023-12-26 15:24:13,381][105692] Updated weights for policy 0, policy_version 5106 (0.0005) [2023-12-26 15:24:13,444][105692] Updated weights for policy 0, policy_version 5116 (0.0005) [2023-12-26 15:24:14,003][105692] Updated weights for policy 0, policy_version 5126 (0.0005) [2023-12-26 15:24:14,045][105620] Updated weights for policy 1, policy_version 5066 (0.0008) [2023-12-26 15:24:14,059][105692] Updated weights for policy 0, policy_version 5136 (0.0005) [2023-12-26 15:24:14,108][105620] Updated weights for policy 1, policy_version 5076 (0.0009) [2023-12-26 15:24:14,112][105692] Updated weights for policy 0, policy_version 5146 (0.0005) [2023-12-26 15:24:14,162][105620] Updated weights for policy 1, policy_version 5086 (0.0009) [2023-12-26 15:24:14,735][105620] Updated weights for policy 1, policy_version 5096 (0.0006) [2023-12-26 15:24:14,805][105620] Updated weights for policy 1, policy_version 5106 (0.0009) [2023-12-26 15:24:14,830][105692] Updated weights for policy 0, policy_version 5156 (0.0005) [2023-12-26 15:24:14,864][105620] Updated weights for policy 1, policy_version 5116 (0.0010) [2023-12-26 15:24:14,890][105692] Updated weights for policy 0, policy_version 5166 (0.0007) [2023-12-26 15:24:14,955][105692] Updated weights for policy 0, policy_version 5176 (0.0008) [2023-12-26 15:24:15,584][105620] Updated weights for policy 1, policy_version 5126 (0.0010) [2023-12-26 15:24:15,634][105620] Updated weights for policy 1, policy_version 5136 (0.0010) [2023-12-26 15:24:15,689][105620] Updated weights for policy 1, policy_version 5146 (0.0009) [2023-12-26 15:24:15,696][105692] Updated weights for policy 0, policy_version 5186 (0.0006) [2023-12-26 15:24:15,759][105692] Updated weights for policy 0, policy_version 5196 (0.0010) [2023-12-26 15:24:15,825][105692] Updated weights for policy 0, policy_version 5206 (0.0011) [2023-12-26 15:24:15,874][105692] Updated weights for policy 0, policy_version 5216 (0.0010) [2023-12-26 15:24:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 18958.6). Total num frames: 2654208. Throughput: 0: 9764.9, 1: 9853.1. Samples: 2618836. Policy #0 lag: (min: 27.0, avg: 30.2, max: 51.0) [2023-12-26 15:24:16,062][104569] Avg episode reward: [(0, '4433.792'), (1, '4190.338')] [2023-12-26 15:24:16,065][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000005152_1318912.pth... [2023-12-26 15:24:16,065][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000005216_1335296.pth... [2023-12-26 15:24:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000004000_1024000.pth [2023-12-26 15:24:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000004064_1040384.pth [2023-12-26 15:24:16,071][105586] Saving new best policy, reward=4190.338! [2023-12-26 15:24:16,071][105585] Saving new best policy, reward=4433.792! [2023-12-26 15:24:16,390][105620] Updated weights for policy 1, policy_version 5156 (0.0006) [2023-12-26 15:24:16,439][105620] Updated weights for policy 1, policy_version 5166 (0.0007) [2023-12-26 15:24:16,492][105620] Updated weights for policy 1, policy_version 5176 (0.0005) [2023-12-26 15:24:16,599][105692] Updated weights for policy 0, policy_version 5226 (0.0010) [2023-12-26 15:24:16,660][105692] Updated weights for policy 0, policy_version 5236 (0.0010) [2023-12-26 15:24:16,714][105692] Updated weights for policy 0, policy_version 5246 (0.0010) [2023-12-26 15:24:17,200][105620] Updated weights for policy 1, policy_version 5186 (0.0009) [2023-12-26 15:24:17,257][105620] Updated weights for policy 1, policy_version 5196 (0.0010) [2023-12-26 15:24:17,312][105620] Updated weights for policy 1, policy_version 5207 (0.0008) [2023-12-26 15:24:17,358][105692] Updated weights for policy 0, policy_version 5256 (0.0007) [2023-12-26 15:24:17,415][105692] Updated weights for policy 0, policy_version 5266 (0.0006) [2023-12-26 15:24:17,474][105692] Updated weights for policy 0, policy_version 5276 (0.0005) [2023-12-26 15:24:18,004][105692] Updated weights for policy 0, policy_version 5286 (0.0005) [2023-12-26 15:24:18,073][105620] Updated weights for policy 1, policy_version 5217 (0.0008) [2023-12-26 15:24:18,073][105692] Updated weights for policy 0, policy_version 5296 (0.0005) [2023-12-26 15:24:18,125][105692] Updated weights for policy 0, policy_version 5306 (0.0006) [2023-12-26 15:24:18,127][105620] Updated weights for policy 1, policy_version 5227 (0.0007) [2023-12-26 15:24:18,184][105620] Updated weights for policy 1, policy_version 5237 (0.0007) [2023-12-26 15:24:18,235][105620] Updated weights for policy 1, policy_version 5247 (0.0009) [2023-12-26 15:24:18,863][105692] Updated weights for policy 0, policy_version 5316 (0.0008) [2023-12-26 15:24:18,874][105620] Updated weights for policy 1, policy_version 5257 (0.0006) [2023-12-26 15:24:18,920][105692] Updated weights for policy 0, policy_version 5326 (0.0006) [2023-12-26 15:24:18,934][105620] Updated weights for policy 1, policy_version 5267 (0.0008) [2023-12-26 15:24:18,970][105692] Updated weights for policy 0, policy_version 5336 (0.0007) [2023-12-26 15:24:18,990][105620] Updated weights for policy 1, policy_version 5277 (0.0010) [2023-12-26 15:24:19,677][105692] Updated weights for policy 0, policy_version 5346 (0.0008) [2023-12-26 15:24:19,724][105620] Updated weights for policy 1, policy_version 5287 (0.0008) [2023-12-26 15:24:19,736][105692] Updated weights for policy 0, policy_version 5356 (0.0007) [2023-12-26 15:24:19,795][105620] Updated weights for policy 1, policy_version 5297 (0.0011) [2023-12-26 15:24:19,802][105692] Updated weights for policy 0, policy_version 5366 (0.0008) [2023-12-26 15:24:19,863][105620] Updated weights for policy 1, policy_version 5307 (0.0010) [2023-12-26 15:24:19,870][105692] Updated weights for policy 0, policy_version 5376 (0.0007) [2023-12-26 15:24:20,582][105692] Updated weights for policy 0, policy_version 5386 (0.0009) [2023-12-26 15:24:20,616][105620] Updated weights for policy 1, policy_version 5317 (0.0007) [2023-12-26 15:24:20,648][105692] Updated weights for policy 0, policy_version 5396 (0.0009) [2023-12-26 15:24:20,672][105620] Updated weights for policy 1, policy_version 5327 (0.0005) [2023-12-26 15:24:20,707][105692] Updated weights for policy 0, policy_version 5406 (0.0008) [2023-12-26 15:24:20,731][105620] Updated weights for policy 1, policy_version 5337 (0.0006) [2023-12-26 15:24:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 18982.8). Total num frames: 2752512. Throughput: 0: 9828.1, 1: 9878.4. Samples: 2739724. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-12-26 15:24:21,063][104569] Avg episode reward: [(0, '5874.228'), (1, '4448.901')] [2023-12-26 15:24:21,063][105585] Saving new best policy, reward=5874.228! [2023-12-26 15:24:21,063][105586] Saving new best policy, reward=4448.901! [2023-12-26 15:24:21,387][105620] Updated weights for policy 1, policy_version 5347 (0.0008) [2023-12-26 15:24:21,448][105620] Updated weights for policy 1, policy_version 5357 (0.0010) [2023-12-26 15:24:21,505][105692] Updated weights for policy 0, policy_version 5416 (0.0008) [2023-12-26 15:24:21,510][105620] Updated weights for policy 1, policy_version 5367 (0.0006) [2023-12-26 15:24:21,572][105692] Updated weights for policy 0, policy_version 5426 (0.0006) [2023-12-26 15:24:21,637][105692] Updated weights for policy 0, policy_version 5436 (0.0008) [2023-12-26 15:24:22,259][105620] Updated weights for policy 1, policy_version 5377 (0.0007) [2023-12-26 15:24:22,323][105620] Updated weights for policy 1, policy_version 5387 (0.0008) [2023-12-26 15:24:22,367][105692] Updated weights for policy 0, policy_version 5446 (0.0007) [2023-12-26 15:24:22,389][105620] Updated weights for policy 1, policy_version 5397 (0.0010) [2023-12-26 15:24:22,422][105692] Updated weights for policy 0, policy_version 5456 (0.0009) [2023-12-26 15:24:22,450][105620] Updated weights for policy 1, policy_version 5407 (0.0006) [2023-12-26 15:24:22,479][105692] Updated weights for policy 0, policy_version 5466 (0.0010) [2023-12-26 15:24:23,084][105620] Updated weights for policy 1, policy_version 5417 (0.0009) [2023-12-26 15:24:23,146][105620] Updated weights for policy 1, policy_version 5427 (0.0009) [2023-12-26 15:24:23,211][105620] Updated weights for policy 1, policy_version 5437 (0.0009) [2023-12-26 15:24:23,313][105692] Updated weights for policy 0, policy_version 5476 (0.0009) [2023-12-26 15:24:23,361][105692] Updated weights for policy 0, policy_version 5486 (0.0009) [2023-12-26 15:24:23,415][105692] Updated weights for policy 0, policy_version 5496 (0.0009) [2023-12-26 15:24:23,841][105620] Updated weights for policy 1, policy_version 5447 (0.0006) [2023-12-26 15:24:23,892][105620] Updated weights for policy 1, policy_version 5457 (0.0005) [2023-12-26 15:24:23,942][105620] Updated weights for policy 1, policy_version 5467 (0.0006) [2023-12-26 15:24:24,202][105692] Updated weights for policy 0, policy_version 5506 (0.0009) [2023-12-26 15:24:24,265][105692] Updated weights for policy 0, policy_version 5516 (0.0008) [2023-12-26 15:24:24,336][105692] Updated weights for policy 0, policy_version 5526 (0.0010) [2023-12-26 15:24:24,400][105692] Updated weights for policy 0, policy_version 5536 (0.0008) [2023-12-26 15:24:24,637][105620] Updated weights for policy 1, policy_version 5477 (0.0009) [2023-12-26 15:24:24,700][105620] Updated weights for policy 1, policy_version 5487 (0.0010) [2023-12-26 15:24:24,753][105620] Updated weights for policy 1, policy_version 5497 (0.0010) [2023-12-26 15:24:25,038][105692] Updated weights for policy 0, policy_version 5546 (0.0009) [2023-12-26 15:24:25,089][105692] Updated weights for policy 0, policy_version 5556 (0.0009) [2023-12-26 15:24:25,142][105692] Updated weights for policy 0, policy_version 5566 (0.0009) [2023-12-26 15:24:25,548][105620] Updated weights for policy 1, policy_version 5507 (0.0009) [2023-12-26 15:24:25,602][105620] Updated weights for policy 1, policy_version 5517 (0.0009) [2023-12-26 15:24:25,653][105620] Updated weights for policy 1, policy_version 5527 (0.0009) [2023-12-26 15:24:25,881][105692] Updated weights for policy 0, policy_version 5576 (0.0009) [2023-12-26 15:24:25,949][105692] Updated weights for policy 0, policy_version 5586 (0.0009) [2023-12-26 15:24:25,995][105692] Updated weights for policy 0, policy_version 5596 (0.0009) [2023-12-26 15:24:26,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19005.4). Total num frames: 2850816. Throughput: 0: 9815.1, 1: 9892.8. Samples: 2854224. Policy #0 lag: (min: 17.0, avg: 28.4, max: 49.0) [2023-12-26 15:24:26,063][104569] Avg episode reward: [(0, '6877.762'), (1, '5928.045')] [2023-12-26 15:24:26,063][105585] Saving new best policy, reward=6877.762! [2023-12-26 15:24:26,063][105586] Saving new best policy, reward=5928.045! [2023-12-26 15:24:26,394][105620] Updated weights for policy 1, policy_version 5537 (0.0009) [2023-12-26 15:24:26,460][105620] Updated weights for policy 1, policy_version 5547 (0.0007) [2023-12-26 15:24:26,524][105620] Updated weights for policy 1, policy_version 5557 (0.0008) [2023-12-26 15:24:26,584][105620] Updated weights for policy 1, policy_version 5567 (0.0008) [2023-12-26 15:24:26,763][105692] Updated weights for policy 0, policy_version 5606 (0.0010) [2023-12-26 15:24:26,824][105692] Updated weights for policy 0, policy_version 5616 (0.0008) [2023-12-26 15:24:26,873][105692] Updated weights for policy 0, policy_version 5626 (0.0005) [2023-12-26 15:24:27,400][105620] Updated weights for policy 1, policy_version 5577 (0.0008) [2023-12-26 15:24:27,444][105620] Updated weights for policy 1, policy_version 5587 (0.0008) [2023-12-26 15:24:27,464][105692] Updated weights for policy 0, policy_version 5636 (0.0007) [2023-12-26 15:24:27,498][105620] Updated weights for policy 1, policy_version 5597 (0.0006) [2023-12-26 15:24:27,516][105692] Updated weights for policy 0, policy_version 5646 (0.0010) [2023-12-26 15:24:27,577][105692] Updated weights for policy 0, policy_version 5656 (0.0010) [2023-12-26 15:24:28,246][105620] Updated weights for policy 1, policy_version 5607 (0.0006) [2023-12-26 15:24:28,298][105620] Updated weights for policy 1, policy_version 5617 (0.0008) [2023-12-26 15:24:28,312][105692] Updated weights for policy 0, policy_version 5666 (0.0009) [2023-12-26 15:24:28,358][105620] Updated weights for policy 1, policy_version 5628 (0.0008) [2023-12-26 15:24:28,376][105692] Updated weights for policy 0, policy_version 5676 (0.0008) [2023-12-26 15:24:28,432][105692] Updated weights for policy 0, policy_version 5686 (0.0010) [2023-12-26 15:24:28,492][105692] Updated weights for policy 0, policy_version 5696 (0.0010) [2023-12-26 15:24:29,017][105620] Updated weights for policy 1, policy_version 5638 (0.0009) [2023-12-26 15:24:29,066][105620] Updated weights for policy 1, policy_version 5648 (0.0008) [2023-12-26 15:24:29,124][105620] Updated weights for policy 1, policy_version 5658 (0.0010) [2023-12-26 15:24:29,163][105692] Updated weights for policy 0, policy_version 5706 (0.0005) [2023-12-26 15:24:29,217][105692] Updated weights for policy 0, policy_version 5716 (0.0006) [2023-12-26 15:24:29,286][105692] Updated weights for policy 0, policy_version 5726 (0.0006) [2023-12-26 15:24:29,854][105620] Updated weights for policy 1, policy_version 5669 (0.0009) [2023-12-26 15:24:29,916][105620] Updated weights for policy 1, policy_version 5679 (0.0009) [2023-12-26 15:24:29,920][105692] Updated weights for policy 0, policy_version 5736 (0.0008) [2023-12-26 15:24:29,978][105692] Updated weights for policy 0, policy_version 5746 (0.0008) [2023-12-26 15:24:29,981][105620] Updated weights for policy 1, policy_version 5689 (0.0006) [2023-12-26 15:24:30,029][105692] Updated weights for policy 0, policy_version 5756 (0.0008) [2023-12-26 15:24:30,577][105620] Updated weights for policy 1, policy_version 5699 (0.0007) [2023-12-26 15:24:30,635][105620] Updated weights for policy 1, policy_version 5709 (0.0010) [2023-12-26 15:24:30,677][105692] Updated weights for policy 0, policy_version 5766 (0.0006) [2023-12-26 15:24:30,701][105620] Updated weights for policy 1, policy_version 5719 (0.0010) [2023-12-26 15:24:30,736][105692] Updated weights for policy 0, policy_version 5776 (0.0006) [2023-12-26 15:24:30,805][105692] Updated weights for policy 0, policy_version 5786 (0.0005) [2023-12-26 15:24:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.9, 300 sec: 19026.6). Total num frames: 2949120. Throughput: 0: 9791.2, 1: 9900.5. Samples: 2912692. Policy #0 lag: (min: 1.0, avg: 14.6, max: 33.0) [2023-12-26 15:24:31,063][104569] Avg episode reward: [(0, '7306.584'), (1, '6949.721')] [2023-12-26 15:24:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000005792_1482752.pth... [2023-12-26 15:24:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000005728_1466368.pth... [2023-12-26 15:24:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000004640_1187840.pth [2023-12-26 15:24:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000004576_1171456.pth [2023-12-26 15:24:31,073][105585] Saving new best policy, reward=7306.584! [2023-12-26 15:24:31,074][105586] Saving new best policy, reward=6949.721! [2023-12-26 15:24:31,356][105692] Updated weights for policy 0, policy_version 5796 (0.0007) [2023-12-26 15:24:31,394][105620] Updated weights for policy 1, policy_version 5729 (0.0009) [2023-12-26 15:24:31,419][105692] Updated weights for policy 0, policy_version 5806 (0.0010) [2023-12-26 15:24:31,452][105620] Updated weights for policy 1, policy_version 5739 (0.0010) [2023-12-26 15:24:31,477][105692] Updated weights for policy 0, policy_version 5816 (0.0010) [2023-12-26 15:24:31,520][105620] Updated weights for policy 1, policy_version 5749 (0.0007) [2023-12-26 15:24:31,574][105620] Updated weights for policy 1, policy_version 5759 (0.0005) [2023-12-26 15:24:32,243][105692] Updated weights for policy 0, policy_version 5826 (0.0010) [2023-12-26 15:24:32,273][105620] Updated weights for policy 1, policy_version 5769 (0.0007) [2023-12-26 15:24:32,296][105692] Updated weights for policy 0, policy_version 5836 (0.0008) [2023-12-26 15:24:32,327][105620] Updated weights for policy 1, policy_version 5779 (0.0007) [2023-12-26 15:24:32,357][105692] Updated weights for policy 0, policy_version 5846 (0.0007) [2023-12-26 15:24:32,388][105620] Updated weights for policy 1, policy_version 5789 (0.0006) [2023-12-26 15:24:32,418][105692] Updated weights for policy 0, policy_version 5856 (0.0008) [2023-12-26 15:24:33,087][105692] Updated weights for policy 0, policy_version 5866 (0.0009) [2023-12-26 15:24:33,149][105692] Updated weights for policy 0, policy_version 5876 (0.0009) [2023-12-26 15:24:33,156][105620] Updated weights for policy 1, policy_version 5799 (0.0006) [2023-12-26 15:24:33,194][105692] Updated weights for policy 0, policy_version 5886 (0.0006) [2023-12-26 15:24:33,215][105620] Updated weights for policy 1, policy_version 5809 (0.0009) [2023-12-26 15:24:33,272][105620] Updated weights for policy 1, policy_version 5819 (0.0009) [2023-12-26 15:24:33,855][105692] Updated weights for policy 0, policy_version 5896 (0.0005) [2023-12-26 15:24:33,910][105692] Updated weights for policy 0, policy_version 5906 (0.0007) [2023-12-26 15:24:33,970][105692] Updated weights for policy 0, policy_version 5916 (0.0007) [2023-12-26 15:24:34,080][105620] Updated weights for policy 1, policy_version 5829 (0.0009) [2023-12-26 15:24:34,126][105620] Updated weights for policy 1, policy_version 5839 (0.0008) [2023-12-26 15:24:34,184][105620] Updated weights for policy 1, policy_version 5849 (0.0008) [2023-12-26 15:24:34,690][105692] Updated weights for policy 0, policy_version 5926 (0.0008) [2023-12-26 15:24:34,738][105692] Updated weights for policy 0, policy_version 5936 (0.0009) [2023-12-26 15:24:34,793][105692] Updated weights for policy 0, policy_version 5946 (0.0009) [2023-12-26 15:24:34,884][105620] Updated weights for policy 1, policy_version 5859 (0.0008) [2023-12-26 15:24:34,941][105620] Updated weights for policy 1, policy_version 5869 (0.0007) [2023-12-26 15:24:34,987][105620] Updated weights for policy 1, policy_version 5879 (0.0006) [2023-12-26 15:24:35,529][105620] Updated weights for policy 1, policy_version 5889 (0.0005) [2023-12-26 15:24:35,591][105620] Updated weights for policy 1, policy_version 5899 (0.0008) [2023-12-26 15:24:35,645][105620] Updated weights for policy 1, policy_version 5909 (0.0009) [2023-12-26 15:24:35,659][105692] Updated weights for policy 0, policy_version 5956 (0.0008) [2023-12-26 15:24:35,690][105620] Updated weights for policy 1, policy_version 5919 (0.0007) [2023-12-26 15:24:35,716][105692] Updated weights for policy 0, policy_version 5966 (0.0009) [2023-12-26 15:24:35,767][105692] Updated weights for policy 0, policy_version 5976 (0.0005) [2023-12-26 15:24:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.7, 300 sec: 19046.4). Total num frames: 3047424. Throughput: 0: 9806.7, 1: 9797.0. Samples: 3033812. Policy #0 lag: (min: 17.0, avg: 32.4, max: 49.0) [2023-12-26 15:24:36,063][104569] Avg episode reward: [(0, '7720.478'), (1, '7609.694')] [2023-12-26 15:24:36,063][105585] Saving new best policy, reward=7720.478! [2023-12-26 15:24:36,064][105586] Saving new best policy, reward=7609.694! [2023-12-26 15:24:36,291][105620] Updated weights for policy 1, policy_version 5929 (0.0006) [2023-12-26 15:24:36,343][105620] Updated weights for policy 1, policy_version 5939 (0.0007) [2023-12-26 15:24:36,391][105620] Updated weights for policy 1, policy_version 5949 (0.0009) [2023-12-26 15:24:36,458][105692] Updated weights for policy 0, policy_version 5986 (0.0006) [2023-12-26 15:24:36,515][105692] Updated weights for policy 0, policy_version 5996 (0.0009) [2023-12-26 15:24:36,573][105692] Updated weights for policy 0, policy_version 6006 (0.0008) [2023-12-26 15:24:36,644][105692] Updated weights for policy 0, policy_version 6016 (0.0008) [2023-12-26 15:24:37,107][105620] Updated weights for policy 1, policy_version 5959 (0.0007) [2023-12-26 15:24:37,166][105620] Updated weights for policy 1, policy_version 5969 (0.0007) [2023-12-26 15:24:37,230][105620] Updated weights for policy 1, policy_version 5979 (0.0008) [2023-12-26 15:24:37,305][105692] Updated weights for policy 0, policy_version 6026 (0.0006) [2023-12-26 15:24:37,365][105692] Updated weights for policy 0, policy_version 6036 (0.0009) [2023-12-26 15:24:37,436][105692] Updated weights for policy 0, policy_version 6046 (0.0009) [2023-12-26 15:24:37,758][105620] Updated weights for policy 1, policy_version 5989 (0.0006) [2023-12-26 15:24:37,817][105620] Updated weights for policy 1, policy_version 5999 (0.0005) [2023-12-26 15:24:37,876][105620] Updated weights for policy 1, policy_version 6009 (0.0005) [2023-12-26 15:24:38,303][105692] Updated weights for policy 0, policy_version 6056 (0.0009) [2023-12-26 15:24:38,373][105692] Updated weights for policy 0, policy_version 6066 (0.0009) [2023-12-26 15:24:38,443][105692] Updated weights for policy 0, policy_version 6076 (0.0006) [2023-12-26 15:24:38,445][105620] Updated weights for policy 1, policy_version 6019 (0.0006) [2023-12-26 15:24:38,502][105620] Updated weights for policy 1, policy_version 6029 (0.0009) [2023-12-26 15:24:38,560][105620] Updated weights for policy 1, policy_version 6039 (0.0009) [2023-12-26 15:24:39,187][105692] Updated weights for policy 0, policy_version 6086 (0.0008) [2023-12-26 15:24:39,252][105692] Updated weights for policy 0, policy_version 6096 (0.0008) [2023-12-26 15:24:39,281][105620] Updated weights for policy 1, policy_version 6049 (0.0009) [2023-12-26 15:24:39,314][105692] Updated weights for policy 0, policy_version 6106 (0.0007) [2023-12-26 15:24:39,346][105620] Updated weights for policy 1, policy_version 6059 (0.0006) [2023-12-26 15:24:39,411][105620] Updated weights for policy 1, policy_version 6069 (0.0007) [2023-12-26 15:24:39,479][105620] Updated weights for policy 1, policy_version 6079 (0.0006) [2023-12-26 15:24:40,153][105620] Updated weights for policy 1, policy_version 6089 (0.0009) [2023-12-26 15:24:40,185][105692] Updated weights for policy 0, policy_version 6116 (0.0007) [2023-12-26 15:24:40,215][105620] Updated weights for policy 1, policy_version 6099 (0.0010) [2023-12-26 15:24:40,243][105692] Updated weights for policy 0, policy_version 6126 (0.0010) [2023-12-26 15:24:40,274][105620] Updated weights for policy 1, policy_version 6109 (0.0009) [2023-12-26 15:24:40,295][105692] Updated weights for policy 0, policy_version 6136 (0.0006) [2023-12-26 15:24:41,025][105620] Updated weights for policy 1, policy_version 6119 (0.0009) [2023-12-26 15:24:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19015.4). Total num frames: 3137536. Throughput: 0: 9730.1, 1: 9933.6. Samples: 3151792. Policy #0 lag: (min: 17.0, avg: 32.4, max: 49.0) [2023-12-26 15:24:41,063][104569] Avg episode reward: [(0, '8374.447'), (1, '8225.709')] [2023-12-26 15:24:41,064][105692] Updated weights for policy 0, policy_version 6146 (0.0006) [2023-12-26 15:24:41,085][105620] Updated weights for policy 1, policy_version 6129 (0.0010) [2023-12-26 15:24:41,129][105692] Updated weights for policy 0, policy_version 6156 (0.0008) [2023-12-26 15:24:41,144][105620] Updated weights for policy 1, policy_version 6139 (0.0008) [2023-12-26 15:24:41,173][105586] Saving new best policy, reward=8225.709! [2023-12-26 15:24:41,194][105692] Updated weights for policy 0, policy_version 6166 (0.0010) [2023-12-26 15:24:41,250][105692] Updated weights for policy 0, policy_version 6176 (0.0009) [2023-12-26 15:24:41,253][105585] Saving new best policy, reward=8374.447! [2023-12-26 15:24:41,915][105620] Updated weights for policy 1, policy_version 6149 (0.0008) [2023-12-26 15:24:41,972][105620] Updated weights for policy 1, policy_version 6159 (0.0009) [2023-12-26 15:24:42,030][105620] Updated weights for policy 1, policy_version 6169 (0.0008) [2023-12-26 15:24:42,069][105692] Updated weights for policy 0, policy_version 6186 (0.0008) [2023-12-26 15:24:42,128][105692] Updated weights for policy 0, policy_version 6196 (0.0008) [2023-12-26 15:24:42,187][105692] Updated weights for policy 0, policy_version 6206 (0.0009) [2023-12-26 15:24:42,667][105620] Updated weights for policy 1, policy_version 6179 (0.0006) [2023-12-26 15:24:42,735][105620] Updated weights for policy 1, policy_version 6189 (0.0006) [2023-12-26 15:24:42,804][105620] Updated weights for policy 1, policy_version 6199 (0.0005) [2023-12-26 15:24:43,071][105692] Updated weights for policy 0, policy_version 6216 (0.0010) [2023-12-26 15:24:43,129][105692] Updated weights for policy 0, policy_version 6226 (0.0010) [2023-12-26 15:24:43,186][105692] Updated weights for policy 0, policy_version 6236 (0.0009) [2023-12-26 15:24:43,330][105620] Updated weights for policy 1, policy_version 6209 (0.0006) [2023-12-26 15:24:43,397][105620] Updated weights for policy 1, policy_version 6219 (0.0010) [2023-12-26 15:24:43,445][105620] Updated weights for policy 1, policy_version 6229 (0.0010) [2023-12-26 15:24:43,493][105620] Updated weights for policy 1, policy_version 6239 (0.0010) [2023-12-26 15:24:43,917][105692] Updated weights for policy 0, policy_version 6246 (0.0007) [2023-12-26 15:24:43,977][105692] Updated weights for policy 0, policy_version 6256 (0.0007) [2023-12-26 15:24:44,025][105692] Updated weights for policy 0, policy_version 6266 (0.0008) [2023-12-26 15:24:44,242][105620] Updated weights for policy 1, policy_version 6249 (0.0010) [2023-12-26 15:24:44,292][105620] Updated weights for policy 1, policy_version 6259 (0.0007) [2023-12-26 15:24:44,342][105620] Updated weights for policy 1, policy_version 6269 (0.0005) [2023-12-26 15:24:44,776][105692] Updated weights for policy 0, policy_version 6276 (0.0008) [2023-12-26 15:24:44,839][105692] Updated weights for policy 0, policy_version 6286 (0.0008) [2023-12-26 15:24:44,895][105692] Updated weights for policy 0, policy_version 6296 (0.0008) [2023-12-26 15:24:45,050][105620] Updated weights for policy 1, policy_version 6279 (0.0009) [2023-12-26 15:24:45,111][105620] Updated weights for policy 1, policy_version 6289 (0.0011) [2023-12-26 15:24:45,167][105620] Updated weights for policy 1, policy_version 6299 (0.0010) [2023-12-26 15:24:45,674][105692] Updated weights for policy 0, policy_version 6306 (0.0008) [2023-12-26 15:24:45,729][105692] Updated weights for policy 0, policy_version 6316 (0.0008) [2023-12-26 15:24:45,776][105692] Updated weights for policy 0, policy_version 6326 (0.0008) [2023-12-26 15:24:45,827][105692] Updated weights for policy 0, policy_version 6336 (0.0006) [2023-12-26 15:24:45,926][105620] Updated weights for policy 1, policy_version 6309 (0.0011) [2023-12-26 15:24:45,988][105620] Updated weights for policy 1, policy_version 6319 (0.0010) [2023-12-26 15:24:46,050][105620] Updated weights for policy 1, policy_version 6329 (0.0009) [2023-12-26 15:24:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19034.3). Total num frames: 3235840. Throughput: 0: 9637.5, 1: 9986.8. Samples: 3208100. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-26 15:24:46,063][104569] Avg episode reward: [(0, '8256.321'), (1, '8148.743')] [2023-12-26 15:24:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000006336_1622016.pth... [2023-12-26 15:24:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000005216_1335296.pth [2023-12-26 15:24:46,098][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000006336_1622016.pth... [2023-12-26 15:24:46,101][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000005152_1318912.pth [2023-12-26 15:24:46,588][105692] Updated weights for policy 0, policy_version 6346 (0.0009) [2023-12-26 15:24:46,646][105692] Updated weights for policy 0, policy_version 6356 (0.0007) [2023-12-26 15:24:46,695][105692] Updated weights for policy 0, policy_version 6366 (0.0008) [2023-12-26 15:24:46,733][105620] Updated weights for policy 1, policy_version 6339 (0.0010) [2023-12-26 15:24:46,799][105620] Updated weights for policy 1, policy_version 6349 (0.0009) [2023-12-26 15:24:46,847][105620] Updated weights for policy 1, policy_version 6359 (0.0009) [2023-12-26 15:24:47,446][105692] Updated weights for policy 0, policy_version 6376 (0.0009) [2023-12-26 15:24:47,499][105692] Updated weights for policy 0, policy_version 6386 (0.0009) [2023-12-26 15:24:47,550][105692] Updated weights for policy 0, policy_version 6396 (0.0009) [2023-12-26 15:24:47,575][105620] Updated weights for policy 1, policy_version 6369 (0.0008) [2023-12-26 15:24:47,643][105620] Updated weights for policy 1, policy_version 6379 (0.0005) [2023-12-26 15:24:47,711][105620] Updated weights for policy 1, policy_version 6389 (0.0006) [2023-12-26 15:24:47,775][105620] Updated weights for policy 1, policy_version 6399 (0.0008) [2023-12-26 15:24:48,363][105620] Updated weights for policy 1, policy_version 6409 (0.0009) [2023-12-26 15:24:48,381][105692] Updated weights for policy 0, policy_version 6406 (0.0008) [2023-12-26 15:24:48,421][105620] Updated weights for policy 1, policy_version 6419 (0.0008) [2023-12-26 15:24:48,436][105692] Updated weights for policy 0, policy_version 6416 (0.0007) [2023-12-26 15:24:48,473][105620] Updated weights for policy 1, policy_version 6429 (0.0008) [2023-12-26 15:24:48,494][105692] Updated weights for policy 0, policy_version 6426 (0.0007) [2023-12-26 15:24:49,106][105620] Updated weights for policy 1, policy_version 6439 (0.0010) [2023-12-26 15:24:49,166][105620] Updated weights for policy 1, policy_version 6449 (0.0011) [2023-12-26 15:24:49,220][105620] Updated weights for policy 1, policy_version 6459 (0.0007) [2023-12-26 15:24:49,338][105692] Updated weights for policy 0, policy_version 6436 (0.0009) [2023-12-26 15:24:49,396][105692] Updated weights for policy 0, policy_version 6446 (0.0009) [2023-12-26 15:24:49,449][105692] Updated weights for policy 0, policy_version 6456 (0.0008) [2023-12-26 15:24:49,953][105620] Updated weights for policy 1, policy_version 6469 (0.0009) [2023-12-26 15:24:50,015][105620] Updated weights for policy 1, policy_version 6479 (0.0010) [2023-12-26 15:24:50,071][105620] Updated weights for policy 1, policy_version 6489 (0.0010) [2023-12-26 15:24:50,193][105692] Updated weights for policy 0, policy_version 6466 (0.0008) [2023-12-26 15:24:50,244][105692] Updated weights for policy 0, policy_version 6476 (0.0006) [2023-12-26 15:24:50,305][105692] Updated weights for policy 0, policy_version 6486 (0.0008) [2023-12-26 15:24:50,365][105692] Updated weights for policy 0, policy_version 6496 (0.0008) [2023-12-26 15:24:50,807][105620] Updated weights for policy 1, policy_version 6499 (0.0009) [2023-12-26 15:24:50,861][105620] Updated weights for policy 1, policy_version 6509 (0.0005) [2023-12-26 15:24:50,918][105620] Updated weights for policy 1, policy_version 6519 (0.0006) [2023-12-26 15:24:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19052.3). Total num frames: 3334144. Throughput: 0: 9537.0, 1: 10015.7. Samples: 3322832. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 15:24:51,062][104569] Avg episode reward: [(0, '8490.891'), (1, '7911.935')] [2023-12-26 15:24:51,063][105585] Saving new best policy, reward=8490.891! [2023-12-26 15:24:51,157][105692] Updated weights for policy 0, policy_version 6506 (0.0009) [2023-12-26 15:24:51,217][105692] Updated weights for policy 0, policy_version 6516 (0.0008) [2023-12-26 15:24:51,279][105692] Updated weights for policy 0, policy_version 6526 (0.0009) [2023-12-26 15:24:51,644][105620] Updated weights for policy 1, policy_version 6529 (0.0010) [2023-12-26 15:24:51,696][105620] Updated weights for policy 1, policy_version 6539 (0.0007) [2023-12-26 15:24:51,757][105620] Updated weights for policy 1, policy_version 6549 (0.0010) [2023-12-26 15:24:51,813][105620] Updated weights for policy 1, policy_version 6559 (0.0010) [2023-12-26 15:24:52,040][105692] Updated weights for policy 0, policy_version 6536 (0.0006) [2023-12-26 15:24:52,105][105692] Updated weights for policy 0, policy_version 6546 (0.0006) [2023-12-26 15:24:52,159][105692] Updated weights for policy 0, policy_version 6556 (0.0008) [2023-12-26 15:24:52,565][105620] Updated weights for policy 1, policy_version 6569 (0.0006) [2023-12-26 15:24:52,624][105620] Updated weights for policy 1, policy_version 6579 (0.0005) [2023-12-26 15:24:52,689][105620] Updated weights for policy 1, policy_version 6589 (0.0005) [2023-12-26 15:24:52,883][105692] Updated weights for policy 0, policy_version 6566 (0.0010) [2023-12-26 15:24:52,943][105692] Updated weights for policy 0, policy_version 6576 (0.0007) [2023-12-26 15:24:53,001][105692] Updated weights for policy 0, policy_version 6586 (0.0005) [2023-12-26 15:24:53,376][105620] Updated weights for policy 1, policy_version 6599 (0.0008) [2023-12-26 15:24:53,439][105620] Updated weights for policy 1, policy_version 6609 (0.0008) [2023-12-26 15:24:53,500][105620] Updated weights for policy 1, policy_version 6619 (0.0009) [2023-12-26 15:24:53,617][105692] Updated weights for policy 0, policy_version 6596 (0.0005) [2023-12-26 15:24:53,677][105692] Updated weights for policy 0, policy_version 6606 (0.0005) [2023-12-26 15:24:53,740][105692] Updated weights for policy 0, policy_version 6616 (0.0009) [2023-12-26 15:24:54,154][105620] Updated weights for policy 1, policy_version 6629 (0.0009) [2023-12-26 15:24:54,209][105620] Updated weights for policy 1, policy_version 6639 (0.0009) [2023-12-26 15:24:54,261][105620] Updated weights for policy 1, policy_version 6649 (0.0009) [2023-12-26 15:24:54,485][105692] Updated weights for policy 0, policy_version 6626 (0.0009) [2023-12-26 15:24:54,539][105692] Updated weights for policy 0, policy_version 6637 (0.0010) [2023-12-26 15:24:54,592][105692] Updated weights for policy 0, policy_version 6648 (0.0010) [2023-12-26 15:24:54,871][105620] Updated weights for policy 1, policy_version 6659 (0.0009) [2023-12-26 15:24:54,918][105620] Updated weights for policy 1, policy_version 6669 (0.0009) [2023-12-26 15:24:54,977][105620] Updated weights for policy 1, policy_version 6679 (0.0009) [2023-12-26 15:24:55,457][105692] Updated weights for policy 0, policy_version 6658 (0.0011) [2023-12-26 15:24:55,518][105692] Updated weights for policy 0, policy_version 6668 (0.0009) [2023-12-26 15:24:55,566][105692] Updated weights for policy 0, policy_version 6678 (0.0009) [2023-12-26 15:24:55,615][105692] Updated weights for policy 0, policy_version 6688 (0.0010) [2023-12-26 15:24:55,660][105620] Updated weights for policy 1, policy_version 6689 (0.0008) [2023-12-26 15:24:55,719][105620] Updated weights for policy 1, policy_version 6699 (0.0005) [2023-12-26 15:24:55,772][105620] Updated weights for policy 1, policy_version 6709 (0.0005) [2023-12-26 15:24:55,826][105620] Updated weights for policy 1, policy_version 6719 (0.0009) [2023-12-26 15:24:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19069.2). Total num frames: 3432448. Throughput: 0: 9538.1, 1: 9967.8. Samples: 3439840. Policy #0 lag: (min: 15.0, avg: 19.7, max: 47.0) [2023-12-26 15:24:56,062][104569] Avg episode reward: [(0, '8736.539'), (1, '8119.280')] [2023-12-26 15:24:56,063][105585] Saving new best policy, reward=8736.539! [2023-12-26 15:24:56,435][105692] Updated weights for policy 0, policy_version 6698 (0.0009) [2023-12-26 15:24:56,492][105692] Updated weights for policy 0, policy_version 6708 (0.0008) [2023-12-26 15:24:56,512][105620] Updated weights for policy 1, policy_version 6729 (0.0009) [2023-12-26 15:24:56,551][105692] Updated weights for policy 0, policy_version 6718 (0.0006) [2023-12-26 15:24:56,574][105620] Updated weights for policy 1, policy_version 6739 (0.0008) [2023-12-26 15:24:56,638][105620] Updated weights for policy 1, policy_version 6749 (0.0008) [2023-12-26 15:24:57,291][105620] Updated weights for policy 1, policy_version 6759 (0.0008) [2023-12-26 15:24:57,350][105620] Updated weights for policy 1, policy_version 6769 (0.0007) [2023-12-26 15:24:57,352][105692] Updated weights for policy 0, policy_version 6728 (0.0007) [2023-12-26 15:24:57,409][105692] Updated weights for policy 0, policy_version 6738 (0.0009) [2023-12-26 15:24:57,413][105620] Updated weights for policy 1, policy_version 6779 (0.0005) [2023-12-26 15:24:57,459][105692] Updated weights for policy 0, policy_version 6748 (0.0006) [2023-12-26 15:24:58,007][105620] Updated weights for policy 1, policy_version 6789 (0.0005) [2023-12-26 15:24:58,060][105620] Updated weights for policy 1, policy_version 6799 (0.0006) [2023-12-26 15:24:58,129][105620] Updated weights for policy 1, policy_version 6809 (0.0009) [2023-12-26 15:24:58,245][105692] Updated weights for policy 0, policy_version 6758 (0.0008) [2023-12-26 15:24:58,304][105692] Updated weights for policy 0, policy_version 6768 (0.0010) [2023-12-26 15:24:58,374][105692] Updated weights for policy 0, policy_version 6778 (0.0008) [2023-12-26 15:24:58,958][105620] Updated weights for policy 1, policy_version 6819 (0.0009) [2023-12-26 15:24:59,024][105620] Updated weights for policy 1, policy_version 6829 (0.0011) [2023-12-26 15:24:59,096][105620] Updated weights for policy 1, policy_version 6839 (0.0009) [2023-12-26 15:24:59,231][105692] Updated weights for policy 0, policy_version 6788 (0.0008) [2023-12-26 15:24:59,299][105692] Updated weights for policy 0, policy_version 6798 (0.0006) [2023-12-26 15:24:59,367][105692] Updated weights for policy 0, policy_version 6808 (0.0011) [2023-12-26 15:24:59,751][105620] Updated weights for policy 1, policy_version 6849 (0.0007) [2023-12-26 15:24:59,816][105620] Updated weights for policy 1, policy_version 6859 (0.0010) [2023-12-26 15:24:59,878][105620] Updated weights for policy 1, policy_version 6869 (0.0009) [2023-12-26 15:24:59,944][105620] Updated weights for policy 1, policy_version 6879 (0.0011) [2023-12-26 15:25:00,075][105692] Updated weights for policy 0, policy_version 6818 (0.0010) [2023-12-26 15:25:00,134][105692] Updated weights for policy 0, policy_version 6828 (0.0006) [2023-12-26 15:25:00,183][105692] Updated weights for policy 0, policy_version 6838 (0.0005) [2023-12-26 15:25:00,241][105692] Updated weights for policy 0, policy_version 6848 (0.0005) [2023-12-26 15:25:00,661][105620] Updated weights for policy 1, policy_version 6889 (0.0008) [2023-12-26 15:25:00,725][105620] Updated weights for policy 1, policy_version 6899 (0.0010) [2023-12-26 15:25:00,782][105620] Updated weights for policy 1, policy_version 6909 (0.0010) [2023-12-26 15:25:00,823][105692] Updated weights for policy 0, policy_version 6858 (0.0005) [2023-12-26 15:25:00,874][105692] Updated weights for policy 0, policy_version 6868 (0.0005) [2023-12-26 15:25:00,920][105692] Updated weights for policy 0, policy_version 6878 (0.0005) [2023-12-26 15:25:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19085.1). Total num frames: 3530752. Throughput: 0: 9468.0, 1: 10023.0. Samples: 3495936. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-26 15:25:01,063][104569] Avg episode reward: [(0, '8577.656'), (1, '8124.167')] [2023-12-26 15:25:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000006880_1761280.pth... [2023-12-26 15:25:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000006912_1769472.pth... [2023-12-26 15:25:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000005792_1482752.pth [2023-12-26 15:25:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000005728_1466368.pth [2023-12-26 15:25:01,481][105620] Updated weights for policy 1, policy_version 6919 (0.0010) [2023-12-26 15:25:01,530][105620] Updated weights for policy 1, policy_version 6929 (0.0008) [2023-12-26 15:25:01,534][105692] Updated weights for policy 0, policy_version 6888 (0.0005) [2023-12-26 15:25:01,577][105620] Updated weights for policy 1, policy_version 6939 (0.0009) [2023-12-26 15:25:01,587][105692] Updated weights for policy 0, policy_version 6898 (0.0005) [2023-12-26 15:25:01,647][105692] Updated weights for policy 0, policy_version 6908 (0.0006) [2023-12-26 15:25:02,331][105620] Updated weights for policy 1, policy_version 6949 (0.0006) [2023-12-26 15:25:02,394][105692] Updated weights for policy 0, policy_version 6918 (0.0006) [2023-12-26 15:25:02,399][105620] Updated weights for policy 1, policy_version 6959 (0.0008) [2023-12-26 15:25:02,449][105620] Updated weights for policy 1, policy_version 6969 (0.0007) [2023-12-26 15:25:02,451][105692] Updated weights for policy 0, policy_version 6928 (0.0007) [2023-12-26 15:25:02,510][105692] Updated weights for policy 0, policy_version 6938 (0.0008) [2023-12-26 15:25:03,159][105692] Updated weights for policy 0, policy_version 6948 (0.0007) [2023-12-26 15:25:03,213][105692] Updated weights for policy 0, policy_version 6958 (0.0008) [2023-12-26 15:25:03,235][105620] Updated weights for policy 1, policy_version 6979 (0.0009) [2023-12-26 15:25:03,262][105692] Updated weights for policy 0, policy_version 6968 (0.0005) [2023-12-26 15:25:03,285][105620] Updated weights for policy 1, policy_version 6989 (0.0009) [2023-12-26 15:25:03,339][105620] Updated weights for policy 1, policy_version 6999 (0.0008) [2023-12-26 15:25:03,964][105692] Updated weights for policy 0, policy_version 6978 (0.0005) [2023-12-26 15:25:04,029][105692] Updated weights for policy 0, policy_version 6988 (0.0007) [2023-12-26 15:25:04,059][105620] Updated weights for policy 1, policy_version 7009 (0.0008) [2023-12-26 15:25:04,103][105692] Updated weights for policy 0, policy_version 6998 (0.0007) [2023-12-26 15:25:04,124][105620] Updated weights for policy 1, policy_version 7019 (0.0006) [2023-12-26 15:25:04,165][105692] Updated weights for policy 0, policy_version 7008 (0.0009) [2023-12-26 15:25:04,180][105620] Updated weights for policy 1, policy_version 7029 (0.0007) [2023-12-26 15:25:04,251][105620] Updated weights for policy 1, policy_version 7039 (0.0005) [2023-12-26 15:25:04,800][105692] Updated weights for policy 0, policy_version 7018 (0.0008) [2023-12-26 15:25:04,853][105692] Updated weights for policy 0, policy_version 7029 (0.0009) [2023-12-26 15:25:04,900][105692] Updated weights for policy 0, policy_version 7039 (0.0009) [2023-12-26 15:25:04,992][105620] Updated weights for policy 1, policy_version 7049 (0.0009) [2023-12-26 15:25:05,052][105620] Updated weights for policy 1, policy_version 7059 (0.0009) [2023-12-26 15:25:05,109][105620] Updated weights for policy 1, policy_version 7069 (0.0008) [2023-12-26 15:25:05,577][105692] Updated weights for policy 0, policy_version 7049 (0.0005) [2023-12-26 15:25:05,631][105692] Updated weights for policy 0, policy_version 7059 (0.0005) [2023-12-26 15:25:05,689][105692] Updated weights for policy 0, policy_version 7069 (0.0005) [2023-12-26 15:25:05,841][105620] Updated weights for policy 1, policy_version 7079 (0.0006) [2023-12-26 15:25:05,897][105620] Updated weights for policy 1, policy_version 7089 (0.0005) [2023-12-26 15:25:05,959][105620] Updated weights for policy 1, policy_version 7099 (0.0005) [2023-12-26 15:25:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19100.3). Total num frames: 3629056. Throughput: 0: 9462.7, 1: 9973.2. Samples: 3614344. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-26 15:25:06,063][104569] Avg episode reward: [(0, '8191.603'), (1, '8459.162')] [2023-12-26 15:25:06,063][105586] Saving new best policy, reward=8459.162! [2023-12-26 15:25:06,341][105692] Updated weights for policy 0, policy_version 7079 (0.0007) [2023-12-26 15:25:06,406][105692] Updated weights for policy 0, policy_version 7089 (0.0009) [2023-12-26 15:25:06,469][105692] Updated weights for policy 0, policy_version 7099 (0.0008) [2023-12-26 15:25:06,659][105620] Updated weights for policy 1, policy_version 7109 (0.0010) [2023-12-26 15:25:06,721][105620] Updated weights for policy 1, policy_version 7119 (0.0010) [2023-12-26 15:25:06,782][105620] Updated weights for policy 1, policy_version 7129 (0.0010) [2023-12-26 15:25:07,139][105692] Updated weights for policy 0, policy_version 7109 (0.0006) [2023-12-26 15:25:07,205][105692] Updated weights for policy 0, policy_version 7119 (0.0007) [2023-12-26 15:25:07,271][105692] Updated weights for policy 0, policy_version 7129 (0.0011) [2023-12-26 15:25:07,476][105620] Updated weights for policy 1, policy_version 7139 (0.0010) [2023-12-26 15:25:07,525][105620] Updated weights for policy 1, policy_version 7149 (0.0007) [2023-12-26 15:25:07,575][105620] Updated weights for policy 1, policy_version 7159 (0.0005) [2023-12-26 15:25:07,889][105692] Updated weights for policy 0, policy_version 7139 (0.0009) [2023-12-26 15:25:07,949][105692] Updated weights for policy 0, policy_version 7149 (0.0006) [2023-12-26 15:25:07,993][105692] Updated weights for policy 0, policy_version 7159 (0.0010) [2023-12-26 15:25:08,182][105620] Updated weights for policy 1, policy_version 7169 (0.0005) [2023-12-26 15:25:08,243][105620] Updated weights for policy 1, policy_version 7179 (0.0008) [2023-12-26 15:25:08,304][105620] Updated weights for policy 1, policy_version 7189 (0.0007) [2023-12-26 15:25:08,367][105620] Updated weights for policy 1, policy_version 7199 (0.0007) [2023-12-26 15:25:08,663][105692] Updated weights for policy 0, policy_version 7169 (0.0011) [2023-12-26 15:25:08,725][105692] Updated weights for policy 0, policy_version 7179 (0.0010) [2023-12-26 15:25:08,784][105692] Updated weights for policy 0, policy_version 7189 (0.0010) [2023-12-26 15:25:08,846][105692] Updated weights for policy 0, policy_version 7199 (0.0010) [2023-12-26 15:25:09,003][105620] Updated weights for policy 1, policy_version 7209 (0.0008) [2023-12-26 15:25:09,072][105620] Updated weights for policy 1, policy_version 7219 (0.0008) [2023-12-26 15:25:09,132][105620] Updated weights for policy 1, policy_version 7229 (0.0007) [2023-12-26 15:25:09,543][105692] Updated weights for policy 0, policy_version 7209 (0.0008) [2023-12-26 15:25:09,602][105692] Updated weights for policy 0, policy_version 7219 (0.0008) [2023-12-26 15:25:09,663][105692] Updated weights for policy 0, policy_version 7229 (0.0007) [2023-12-26 15:25:09,905][105620] Updated weights for policy 1, policy_version 7239 (0.0009) [2023-12-26 15:25:09,972][105620] Updated weights for policy 1, policy_version 7249 (0.0010) [2023-12-26 15:25:10,035][105620] Updated weights for policy 1, policy_version 7259 (0.0009) [2023-12-26 15:25:10,300][105692] Updated weights for policy 0, policy_version 7239 (0.0009) [2023-12-26 15:25:10,348][105692] Updated weights for policy 0, policy_version 7249 (0.0010) [2023-12-26 15:25:10,400][105692] Updated weights for policy 0, policy_version 7259 (0.0010) [2023-12-26 15:25:10,824][105620] Updated weights for policy 1, policy_version 7269 (0.0009) [2023-12-26 15:25:10,886][105620] Updated weights for policy 1, policy_version 7279 (0.0009) [2023-12-26 15:25:10,941][105620] Updated weights for policy 1, policy_version 7289 (0.0009) [2023-12-26 15:25:11,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19114.6). Total num frames: 3727360. Throughput: 0: 9598.8, 1: 9960.1. Samples: 3734380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:25:11,063][104569] Avg episode reward: [(0, '8596.037'), (1, '9012.290')] [2023-12-26 15:25:11,064][105586] Saving new best policy, reward=9012.290! [2023-12-26 15:25:11,191][105692] Updated weights for policy 0, policy_version 7269 (0.0010) [2023-12-26 15:25:11,256][105692] Updated weights for policy 0, policy_version 7279 (0.0009) [2023-12-26 15:25:11,312][105692] Updated weights for policy 0, policy_version 7289 (0.0009) [2023-12-26 15:25:11,737][105620] Updated weights for policy 1, policy_version 7299 (0.0009) [2023-12-26 15:25:11,786][105620] Updated weights for policy 1, policy_version 7309 (0.0010) [2023-12-26 15:25:11,851][105620] Updated weights for policy 1, policy_version 7319 (0.0008) [2023-12-26 15:25:12,044][105692] Updated weights for policy 0, policy_version 7299 (0.0007) [2023-12-26 15:25:12,103][105692] Updated weights for policy 0, policy_version 7309 (0.0007) [2023-12-26 15:25:12,170][105692] Updated weights for policy 0, policy_version 7319 (0.0009) [2023-12-26 15:25:12,613][105620] Updated weights for policy 1, policy_version 7329 (0.0011) [2023-12-26 15:25:12,668][105620] Updated weights for policy 1, policy_version 7339 (0.0010) [2023-12-26 15:25:12,726][105620] Updated weights for policy 1, policy_version 7349 (0.0010) [2023-12-26 15:25:12,781][105620] Updated weights for policy 1, policy_version 7359 (0.0010) [2023-12-26 15:25:12,846][105692] Updated weights for policy 0, policy_version 7329 (0.0007) [2023-12-26 15:25:12,895][105692] Updated weights for policy 0, policy_version 7339 (0.0008) [2023-12-26 15:25:12,948][105692] Updated weights for policy 0, policy_version 7349 (0.0008) [2023-12-26 15:25:13,005][105692] Updated weights for policy 0, policy_version 7359 (0.0008) [2023-12-26 15:25:13,531][105620] Updated weights for policy 1, policy_version 7369 (0.0006) [2023-12-26 15:25:13,584][105620] Updated weights for policy 1, policy_version 7379 (0.0005) [2023-12-26 15:25:13,637][105620] Updated weights for policy 1, policy_version 7389 (0.0005) [2023-12-26 15:25:13,701][105692] Updated weights for policy 0, policy_version 7369 (0.0010) [2023-12-26 15:25:13,760][105692] Updated weights for policy 0, policy_version 7380 (0.0010) [2023-12-26 15:25:13,819][105692] Updated weights for policy 0, policy_version 7390 (0.0009) [2023-12-26 15:25:14,216][105620] Updated weights for policy 1, policy_version 7399 (0.0008) [2023-12-26 15:25:14,280][105620] Updated weights for policy 1, policy_version 7409 (0.0010) [2023-12-26 15:25:14,336][105620] Updated weights for policy 1, policy_version 7419 (0.0009) [2023-12-26 15:25:14,509][105692] Updated weights for policy 0, policy_version 7400 (0.0008) [2023-12-26 15:25:14,567][105692] Updated weights for policy 0, policy_version 7410 (0.0005) [2023-12-26 15:25:14,615][105692] Updated weights for policy 0, policy_version 7420 (0.0005) [2023-12-26 15:25:15,197][105620] Updated weights for policy 1, policy_version 7429 (0.0008) [2023-12-26 15:25:15,259][105620] Updated weights for policy 1, policy_version 7439 (0.0007) [2023-12-26 15:25:15,269][105692] Updated weights for policy 0, policy_version 7430 (0.0008) [2023-12-26 15:25:15,320][105620] Updated weights for policy 1, policy_version 7449 (0.0009) [2023-12-26 15:25:15,332][105692] Updated weights for policy 0, policy_version 7440 (0.0010) [2023-12-26 15:25:15,390][105692] Updated weights for policy 0, policy_version 7450 (0.0010) [2023-12-26 15:25:16,036][105692] Updated weights for policy 0, policy_version 7460 (0.0010) [2023-12-26 15:25:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19087.4). Total num frames: 3817472. Throughput: 0: 9589.4, 1: 9968.5. Samples: 3792800. Policy #0 lag: (min: 13.0, avg: 13.2, max: 21.0) [2023-12-26 15:25:16,063][104569] Avg episode reward: [(0, '8666.674'), (1, '9044.951')] [2023-12-26 15:25:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000007456_1908736.pth... [2023-12-26 15:25:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000006336_1622016.pth [2023-12-26 15:25:16,072][105586] Saving new best policy, reward=9044.951! [2023-12-26 15:25:16,097][105692] Updated weights for policy 0, policy_version 7470 (0.0007) [2023-12-26 15:25:16,121][105620] Updated weights for policy 1, policy_version 7459 (0.0009) [2023-12-26 15:25:16,160][105692] Updated weights for policy 0, policy_version 7480 (0.0005) [2023-12-26 15:25:16,169][105620] Updated weights for policy 1, policy_version 7469 (0.0009) [2023-12-26 15:25:16,204][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000007488_1916928.pth... [2023-12-26 15:25:16,207][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000006336_1622016.pth [2023-12-26 15:25:16,217][105620] Updated weights for policy 1, policy_version 7479 (0.0006) [2023-12-26 15:25:16,796][105692] Updated weights for policy 0, policy_version 7490 (0.0009) [2023-12-26 15:25:16,849][105692] Updated weights for policy 0, policy_version 7500 (0.0006) [2023-12-26 15:25:16,908][105692] Updated weights for policy 0, policy_version 7510 (0.0006) [2023-12-26 15:25:16,975][105692] Updated weights for policy 0, policy_version 7520 (0.0010) [2023-12-26 15:25:16,979][105620] Updated weights for policy 1, policy_version 7489 (0.0008) [2023-12-26 15:25:17,045][105620] Updated weights for policy 1, policy_version 7499 (0.0011) [2023-12-26 15:25:17,112][105620] Updated weights for policy 1, policy_version 7509 (0.0011) [2023-12-26 15:25:17,174][105620] Updated weights for policy 1, policy_version 7519 (0.0010) [2023-12-26 15:25:17,578][105692] Updated weights for policy 0, policy_version 7530 (0.0005) [2023-12-26 15:25:17,638][105692] Updated weights for policy 0, policy_version 7540 (0.0006) [2023-12-26 15:25:17,693][105692] Updated weights for policy 0, policy_version 7550 (0.0009) [2023-12-26 15:25:17,917][105620] Updated weights for policy 1, policy_version 7529 (0.0010) [2023-12-26 15:25:17,976][105620] Updated weights for policy 1, policy_version 7539 (0.0009) [2023-12-26 15:25:18,049][105620] Updated weights for policy 1, policy_version 7549 (0.0009) [2023-12-26 15:25:18,374][105692] Updated weights for policy 0, policy_version 7560 (0.0008) [2023-12-26 15:25:18,438][105692] Updated weights for policy 0, policy_version 7570 (0.0007) [2023-12-26 15:25:18,501][105692] Updated weights for policy 0, policy_version 7580 (0.0006) [2023-12-26 15:25:18,699][105620] Updated weights for policy 1, policy_version 7559 (0.0010) [2023-12-26 15:25:18,769][105620] Updated weights for policy 1, policy_version 7569 (0.0010) [2023-12-26 15:25:18,838][105620] Updated weights for policy 1, policy_version 7579 (0.0011) [2023-12-26 15:25:19,211][105692] Updated weights for policy 0, policy_version 7590 (0.0008) [2023-12-26 15:25:19,273][105692] Updated weights for policy 0, policy_version 7600 (0.0007) [2023-12-26 15:25:19,332][105692] Updated weights for policy 0, policy_version 7610 (0.0006) [2023-12-26 15:25:19,554][105620] Updated weights for policy 1, policy_version 7589 (0.0010) [2023-12-26 15:25:19,610][105620] Updated weights for policy 1, policy_version 7599 (0.0009) [2023-12-26 15:25:19,666][105620] Updated weights for policy 1, policy_version 7609 (0.0011) [2023-12-26 15:25:20,033][105692] Updated weights for policy 0, policy_version 7620 (0.0008) [2023-12-26 15:25:20,095][105692] Updated weights for policy 0, policy_version 7630 (0.0008) [2023-12-26 15:25:20,154][105692] Updated weights for policy 0, policy_version 7640 (0.0008) [2023-12-26 15:25:20,407][105620] Updated weights for policy 1, policy_version 7619 (0.0011) [2023-12-26 15:25:20,479][105620] Updated weights for policy 1, policy_version 7629 (0.0010) [2023-12-26 15:25:20,545][105620] Updated weights for policy 1, policy_version 7639 (0.0010) [2023-12-26 15:25:20,789][105692] Updated weights for policy 0, policy_version 7650 (0.0008) [2023-12-26 15:25:20,857][105692] Updated weights for policy 0, policy_version 7660 (0.0009) [2023-12-26 15:25:20,919][105692] Updated weights for policy 0, policy_version 7670 (0.0011) [2023-12-26 15:25:20,982][105692] Updated weights for policy 0, policy_version 7680 (0.0011) [2023-12-26 15:25:21,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19524.3, 300 sec: 19141.3). Total num frames: 3923968. Throughput: 0: 9583.0, 1: 9898.7. Samples: 3910488. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 15:25:21,062][104569] Avg episode reward: [(0, '8511.941'), (1, '8891.624')] [2023-12-26 15:25:21,300][105620] Updated weights for policy 1, policy_version 7649 (0.0011) [2023-12-26 15:25:21,371][105620] Updated weights for policy 1, policy_version 7659 (0.0013) [2023-12-26 15:25:21,435][105620] Updated weights for policy 1, policy_version 7669 (0.0009) [2023-12-26 15:25:21,491][105620] Updated weights for policy 1, policy_version 7679 (0.0010) [2023-12-26 15:25:21,740][105692] Updated weights for policy 0, policy_version 7690 (0.0007) [2023-12-26 15:25:21,806][105692] Updated weights for policy 0, policy_version 7700 (0.0007) [2023-12-26 15:25:21,865][105692] Updated weights for policy 0, policy_version 7710 (0.0011) [2023-12-26 15:25:22,277][105620] Updated weights for policy 1, policy_version 7689 (0.0010) [2023-12-26 15:25:22,341][105620] Updated weights for policy 1, policy_version 7699 (0.0011) [2023-12-26 15:25:22,406][105620] Updated weights for policy 1, policy_version 7709 (0.0011) [2023-12-26 15:25:22,590][105692] Updated weights for policy 0, policy_version 7720 (0.0010) [2023-12-26 15:25:22,656][105692] Updated weights for policy 0, policy_version 7730 (0.0011) [2023-12-26 15:25:22,727][105692] Updated weights for policy 0, policy_version 7740 (0.0006) [2023-12-26 15:25:23,180][105620] Updated weights for policy 1, policy_version 7719 (0.0010) [2023-12-26 15:25:23,241][105620] Updated weights for policy 1, policy_version 7729 (0.0010) [2023-12-26 15:25:23,292][105620] Updated weights for policy 1, policy_version 7739 (0.0010) [2023-12-26 15:25:23,336][105692] Updated weights for policy 0, policy_version 7750 (0.0009) [2023-12-26 15:25:23,384][105692] Updated weights for policy 0, policy_version 7760 (0.0010) [2023-12-26 15:25:23,432][105692] Updated weights for policy 0, policy_version 7770 (0.0010) [2023-12-26 15:25:24,011][105620] Updated weights for policy 1, policy_version 7749 (0.0008) [2023-12-26 15:25:24,070][105620] Updated weights for policy 1, policy_version 7759 (0.0006) [2023-12-26 15:25:24,124][105620] Updated weights for policy 1, policy_version 7769 (0.0006) [2023-12-26 15:25:24,209][105692] Updated weights for policy 0, policy_version 7780 (0.0009) [2023-12-26 15:25:24,272][105692] Updated weights for policy 0, policy_version 7790 (0.0008) [2023-12-26 15:25:24,342][105692] Updated weights for policy 0, policy_version 7800 (0.0006) [2023-12-26 15:25:24,784][105620] Updated weights for policy 1, policy_version 7779 (0.0005) [2023-12-26 15:25:24,843][105620] Updated weights for policy 1, policy_version 7789 (0.0005) [2023-12-26 15:25:24,891][105620] Updated weights for policy 1, policy_version 7799 (0.0005) [2023-12-26 15:25:25,049][105692] Updated weights for policy 0, policy_version 7810 (0.0009) [2023-12-26 15:25:25,110][105692] Updated weights for policy 0, policy_version 7820 (0.0010) [2023-12-26 15:25:25,172][105692] Updated weights for policy 0, policy_version 7830 (0.0010) [2023-12-26 15:25:25,221][105692] Updated weights for policy 0, policy_version 7840 (0.0010) [2023-12-26 15:25:25,522][105620] Updated weights for policy 1, policy_version 7809 (0.0006) [2023-12-26 15:25:25,582][105620] Updated weights for policy 1, policy_version 7819 (0.0007) [2023-12-26 15:25:25,649][105620] Updated weights for policy 1, policy_version 7829 (0.0005) [2023-12-26 15:25:25,704][105620] Updated weights for policy 1, policy_version 7839 (0.0006) [2023-12-26 15:25:25,991][105692] Updated weights for policy 0, policy_version 7850 (0.0010) [2023-12-26 15:25:26,052][105692] Updated weights for policy 0, policy_version 7860 (0.0010) [2023-12-26 15:25:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19114.7). Total num frames: 4014080. Throughput: 0: 9692.4, 1: 9769.7. Samples: 4027584. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 15:25:26,062][104569] Avg episode reward: [(0, '8688.517'), (1, '8511.564')] [2023-12-26 15:25:26,117][105692] Updated weights for policy 0, policy_version 7870 (0.0010) [2023-12-26 15:25:26,317][105620] Updated weights for policy 1, policy_version 7849 (0.0008) [2023-12-26 15:25:26,376][105620] Updated weights for policy 1, policy_version 7859 (0.0008) [2023-12-26 15:25:26,429][105620] Updated weights for policy 1, policy_version 7869 (0.0008) [2023-12-26 15:25:26,820][105692] Updated weights for policy 0, policy_version 7880 (0.0005) [2023-12-26 15:25:26,887][105692] Updated weights for policy 0, policy_version 7890 (0.0005) [2023-12-26 15:25:26,939][105692] Updated weights for policy 0, policy_version 7900 (0.0005) [2023-12-26 15:25:27,238][105620] Updated weights for policy 1, policy_version 7879 (0.0008) [2023-12-26 15:25:27,289][105620] Updated weights for policy 1, policy_version 7889 (0.0007) [2023-12-26 15:25:27,348][105620] Updated weights for policy 1, policy_version 7899 (0.0008) [2023-12-26 15:25:27,585][105692] Updated weights for policy 0, policy_version 7910 (0.0009) [2023-12-26 15:25:27,643][105692] Updated weights for policy 0, policy_version 7920 (0.0010) [2023-12-26 15:25:27,707][105692] Updated weights for policy 0, policy_version 7930 (0.0010) [2023-12-26 15:25:28,130][105620] Updated weights for policy 1, policy_version 7909 (0.0009) [2023-12-26 15:25:28,182][105620] Updated weights for policy 1, policy_version 7919 (0.0009) [2023-12-26 15:25:28,238][105620] Updated weights for policy 1, policy_version 7929 (0.0009) [2023-12-26 15:25:28,316][105692] Updated weights for policy 0, policy_version 7940 (0.0009) [2023-12-26 15:25:28,382][105692] Updated weights for policy 0, policy_version 7950 (0.0010) [2023-12-26 15:25:28,433][105692] Updated weights for policy 0, policy_version 7960 (0.0011) [2023-12-26 15:25:29,038][105620] Updated weights for policy 1, policy_version 7939 (0.0009) [2023-12-26 15:25:29,093][105620] Updated weights for policy 1, policy_version 7949 (0.0008) [2023-12-26 15:25:29,147][105620] Updated weights for policy 1, policy_version 7959 (0.0008) [2023-12-26 15:25:29,162][105692] Updated weights for policy 0, policy_version 7970 (0.0011) [2023-12-26 15:25:29,210][105692] Updated weights for policy 0, policy_version 7980 (0.0010) [2023-12-26 15:25:29,277][105692] Updated weights for policy 0, policy_version 7990 (0.0008) [2023-12-26 15:25:29,347][105692] Updated weights for policy 0, policy_version 8000 (0.0009) [2023-12-26 15:25:29,851][105620] Updated weights for policy 1, policy_version 7969 (0.0006) [2023-12-26 15:25:29,908][105620] Updated weights for policy 1, policy_version 7979 (0.0009) [2023-12-26 15:25:29,974][105620] Updated weights for policy 1, policy_version 7989 (0.0009) [2023-12-26 15:25:30,035][105620] Updated weights for policy 1, policy_version 7999 (0.0008) [2023-12-26 15:25:30,107][105692] Updated weights for policy 0, policy_version 8010 (0.0009) [2023-12-26 15:25:30,166][105692] Updated weights for policy 0, policy_version 8020 (0.0009) [2023-12-26 15:25:30,228][105692] Updated weights for policy 0, policy_version 8030 (0.0009) [2023-12-26 15:25:30,701][105620] Updated weights for policy 1, policy_version 8009 (0.0006) [2023-12-26 15:25:30,755][105620] Updated weights for policy 1, policy_version 8019 (0.0009) [2023-12-26 15:25:30,813][105620] Updated weights for policy 1, policy_version 8031 (0.0010) [2023-12-26 15:25:30,966][105692] Updated weights for policy 0, policy_version 8040 (0.0009) [2023-12-26 15:25:31,017][105692] Updated weights for policy 0, policy_version 8050 (0.0008) [2023-12-26 15:25:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19127.4). Total num frames: 4112384. Throughput: 0: 9806.8, 1: 9691.0. Samples: 4085500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:25:31,063][104569] Avg episode reward: [(0, '8885.997'), (1, '8506.840')] [2023-12-26 15:25:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000008032_2056192.pth... [2023-12-26 15:25:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000006912_1769472.pth [2023-12-26 15:25:31,082][105692] Updated weights for policy 0, policy_version 8060 (0.0009) [2023-12-26 15:25:31,098][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000008064_2064384.pth... [2023-12-26 15:25:31,101][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000006880_1761280.pth [2023-12-26 15:25:31,102][105585] Saving new best policy, reward=8885.997! [2023-12-26 15:25:31,577][105620] Updated weights for policy 1, policy_version 8041 (0.0010) [2023-12-26 15:25:31,644][105620] Updated weights for policy 1, policy_version 8051 (0.0008) [2023-12-26 15:25:31,701][105620] Updated weights for policy 1, policy_version 8061 (0.0007) [2023-12-26 15:25:31,841][105692] Updated weights for policy 0, policy_version 8070 (0.0008) [2023-12-26 15:25:31,893][105692] Updated weights for policy 0, policy_version 8080 (0.0008) [2023-12-26 15:25:31,950][105692] Updated weights for policy 0, policy_version 8090 (0.0009) [2023-12-26 15:25:32,384][105620] Updated weights for policy 1, policy_version 8071 (0.0010) [2023-12-26 15:25:32,446][105620] Updated weights for policy 1, policy_version 8081 (0.0010) [2023-12-26 15:25:32,500][105620] Updated weights for policy 1, policy_version 8091 (0.0010) [2023-12-26 15:25:32,692][105692] Updated weights for policy 0, policy_version 8100 (0.0008) [2023-12-26 15:25:32,753][105692] Updated weights for policy 0, policy_version 8110 (0.0005) [2023-12-26 15:25:32,816][105692] Updated weights for policy 0, policy_version 8120 (0.0007) [2023-12-26 15:25:33,244][105620] Updated weights for policy 1, policy_version 8101 (0.0010) [2023-12-26 15:25:33,292][105620] Updated weights for policy 1, policy_version 8111 (0.0010) [2023-12-26 15:25:33,340][105620] Updated weights for policy 1, policy_version 8121 (0.0010) [2023-12-26 15:25:33,378][105692] Updated weights for policy 0, policy_version 8130 (0.0007) [2023-12-26 15:25:33,437][105692] Updated weights for policy 0, policy_version 8140 (0.0008) [2023-12-26 15:25:33,488][105692] Updated weights for policy 0, policy_version 8150 (0.0007) [2023-12-26 15:25:33,535][105692] Updated weights for policy 0, policy_version 8160 (0.0009) [2023-12-26 15:25:34,067][105620] Updated weights for policy 1, policy_version 8131 (0.0009) [2023-12-26 15:25:34,133][105620] Updated weights for policy 1, policy_version 8141 (0.0006) [2023-12-26 15:25:34,139][105692] Updated weights for policy 0, policy_version 8170 (0.0006) [2023-12-26 15:25:34,196][105620] Updated weights for policy 1, policy_version 8151 (0.0009) [2023-12-26 15:25:34,208][105692] Updated weights for policy 0, policy_version 8180 (0.0009) [2023-12-26 15:25:34,264][105692] Updated weights for policy 0, policy_version 8190 (0.0009) [2023-12-26 15:25:34,781][105620] Updated weights for policy 1, policy_version 8161 (0.0007) [2023-12-26 15:25:34,829][105620] Updated weights for policy 1, policy_version 8171 (0.0010) [2023-12-26 15:25:34,883][105620] Updated weights for policy 1, policy_version 8181 (0.0009) [2023-12-26 15:25:34,936][105620] Updated weights for policy 1, policy_version 8191 (0.0010) [2023-12-26 15:25:35,030][105692] Updated weights for policy 0, policy_version 8200 (0.0009) [2023-12-26 15:25:35,085][105692] Updated weights for policy 0, policy_version 8210 (0.0008) [2023-12-26 15:25:35,134][105692] Updated weights for policy 0, policy_version 8220 (0.0008) [2023-12-26 15:25:35,679][105620] Updated weights for policy 1, policy_version 8201 (0.0010) [2023-12-26 15:25:35,727][105620] Updated weights for policy 1, policy_version 8211 (0.0009) [2023-12-26 15:25:35,771][105620] Updated weights for policy 1, policy_version 8221 (0.0009) [2023-12-26 15:25:35,849][105692] Updated weights for policy 0, policy_version 8230 (0.0009) [2023-12-26 15:25:35,901][105692] Updated weights for policy 0, policy_version 8240 (0.0009) [2023-12-26 15:25:35,950][105692] Updated weights for policy 0, policy_version 8250 (0.0009) [2023-12-26 15:25:36,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19176.7). Total num frames: 4218880. Throughput: 0: 9906.9, 1: 9676.9. Samples: 4204100. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 15:25:36,062][104569] Avg episode reward: [(0, '8885.975'), (1, '8753.538')] [2023-12-26 15:25:36,516][105620] Updated weights for policy 1, policy_version 8231 (0.0009) [2023-12-26 15:25:36,581][105620] Updated weights for policy 1, policy_version 8241 (0.0009) [2023-12-26 15:25:36,647][105620] Updated weights for policy 1, policy_version 8251 (0.0009) [2023-12-26 15:25:36,730][105692] Updated weights for policy 0, policy_version 8260 (0.0010) [2023-12-26 15:25:36,790][105692] Updated weights for policy 0, policy_version 8270 (0.0009) [2023-12-26 15:25:36,850][105692] Updated weights for policy 0, policy_version 8280 (0.0009) [2023-12-26 15:25:37,398][105620] Updated weights for policy 1, policy_version 8261 (0.0009) [2023-12-26 15:25:37,446][105620] Updated weights for policy 1, policy_version 8271 (0.0009) [2023-12-26 15:25:37,497][105620] Updated weights for policy 1, policy_version 8281 (0.0009) [2023-12-26 15:25:37,607][105692] Updated weights for policy 0, policy_version 8290 (0.0008) [2023-12-26 15:25:37,660][105692] Updated weights for policy 0, policy_version 8300 (0.0008) [2023-12-26 15:25:37,719][105692] Updated weights for policy 0, policy_version 8310 (0.0010) [2023-12-26 15:25:38,214][105620] Updated weights for policy 1, policy_version 8291 (0.0008) [2023-12-26 15:25:38,269][105620] Updated weights for policy 1, policy_version 8301 (0.0009) [2023-12-26 15:25:38,330][105620] Updated weights for policy 1, policy_version 8311 (0.0008) [2023-12-26 15:25:38,518][105692] Updated weights for policy 0, policy_version 8321 (0.0010) [2023-12-26 15:25:38,580][105692] Updated weights for policy 0, policy_version 8331 (0.0010) [2023-12-26 15:25:38,630][105692] Updated weights for policy 0, policy_version 8341 (0.0008) [2023-12-26 15:25:38,678][105692] Updated weights for policy 0, policy_version 8351 (0.0010) [2023-12-26 15:25:39,021][105620] Updated weights for policy 1, policy_version 8321 (0.0006) [2023-12-26 15:25:39,071][105620] Updated weights for policy 1, policy_version 8331 (0.0006) [2023-12-26 15:25:39,126][105620] Updated weights for policy 1, policy_version 8341 (0.0006) [2023-12-26 15:25:39,181][105620] Updated weights for policy 1, policy_version 8351 (0.0005) [2023-12-26 15:25:39,429][105692] Updated weights for policy 0, policy_version 8361 (0.0011) [2023-12-26 15:25:39,481][105692] Updated weights for policy 0, policy_version 8371 (0.0010) [2023-12-26 15:25:39,544][105692] Updated weights for policy 0, policy_version 8381 (0.0009) [2023-12-26 15:25:39,868][105620] Updated weights for policy 1, policy_version 8361 (0.0009) [2023-12-26 15:25:39,930][105620] Updated weights for policy 1, policy_version 8371 (0.0006) [2023-12-26 15:25:39,988][105620] Updated weights for policy 1, policy_version 8381 (0.0006) [2023-12-26 15:25:40,340][105692] Updated weights for policy 0, policy_version 8391 (0.0008) [2023-12-26 15:25:40,402][105692] Updated weights for policy 0, policy_version 8401 (0.0010) [2023-12-26 15:25:40,466][105692] Updated weights for policy 0, policy_version 8411 (0.0009) [2023-12-26 15:25:40,658][105620] Updated weights for policy 1, policy_version 8391 (0.0009) [2023-12-26 15:25:40,719][105620] Updated weights for policy 1, policy_version 8401 (0.0008) [2023-12-26 15:25:40,786][105620] Updated weights for policy 1, policy_version 8411 (0.0009) [2023-12-26 15:25:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19151.1). Total num frames: 4308992. Throughput: 0: 9874.0, 1: 9640.8. Samples: 4318008. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 15:25:41,063][104569] Avg episode reward: [(0, '8702.020'), (1, '8576.241')] [2023-12-26 15:25:41,298][105692] Updated weights for policy 0, policy_version 8421 (0.0009) [2023-12-26 15:25:41,358][105692] Updated weights for policy 0, policy_version 8431 (0.0011) [2023-12-26 15:25:41,423][105692] Updated weights for policy 0, policy_version 8441 (0.0009) [2023-12-26 15:25:41,523][105620] Updated weights for policy 1, policy_version 8421 (0.0009) [2023-12-26 15:25:41,590][105620] Updated weights for policy 1, policy_version 8431 (0.0009) [2023-12-26 15:25:41,654][105620] Updated weights for policy 1, policy_version 8441 (0.0010) [2023-12-26 15:25:42,211][105692] Updated weights for policy 0, policy_version 8451 (0.0009) [2023-12-26 15:25:42,266][105692] Updated weights for policy 0, policy_version 8461 (0.0009) [2023-12-26 15:25:42,327][105692] Updated weights for policy 0, policy_version 8471 (0.0008) [2023-12-26 15:25:42,365][105620] Updated weights for policy 1, policy_version 8451 (0.0009) [2023-12-26 15:25:42,429][105620] Updated weights for policy 1, policy_version 8461 (0.0008) [2023-12-26 15:25:42,494][105620] Updated weights for policy 1, policy_version 8471 (0.0005) [2023-12-26 15:25:42,996][105692] Updated weights for policy 0, policy_version 8481 (0.0007) [2023-12-26 15:25:43,053][105692] Updated weights for policy 0, policy_version 8491 (0.0008) [2023-12-26 15:25:43,105][105692] Updated weights for policy 0, policy_version 8501 (0.0009) [2023-12-26 15:25:43,155][105692] Updated weights for policy 0, policy_version 8511 (0.0009) [2023-12-26 15:25:43,244][105620] Updated weights for policy 1, policy_version 8481 (0.0006) [2023-12-26 15:25:43,301][105620] Updated weights for policy 1, policy_version 8491 (0.0010) [2023-12-26 15:25:43,354][105620] Updated weights for policy 1, policy_version 8501 (0.0010) [2023-12-26 15:25:43,405][105620] Updated weights for policy 1, policy_version 8511 (0.0011) [2023-12-26 15:25:43,732][105692] Updated weights for policy 0, policy_version 8521 (0.0006) [2023-12-26 15:25:43,783][105692] Updated weights for policy 0, policy_version 8531 (0.0008) [2023-12-26 15:25:43,838][105692] Updated weights for policy 0, policy_version 8541 (0.0010) [2023-12-26 15:25:44,012][105620] Updated weights for policy 1, policy_version 8521 (0.0008) [2023-12-26 15:25:44,060][105620] Updated weights for policy 1, policy_version 8531 (0.0008) [2023-12-26 15:25:44,120][105620] Updated weights for policy 1, policy_version 8541 (0.0008) [2023-12-26 15:25:44,591][105692] Updated weights for policy 0, policy_version 8551 (0.0007) [2023-12-26 15:25:44,649][105692] Updated weights for policy 0, policy_version 8561 (0.0005) [2023-12-26 15:25:44,710][105692] Updated weights for policy 0, policy_version 8571 (0.0009) [2023-12-26 15:25:44,883][105620] Updated weights for policy 1, policy_version 8551 (0.0008) [2023-12-26 15:25:44,944][105620] Updated weights for policy 1, policy_version 8561 (0.0009) [2023-12-26 15:25:45,005][105620] Updated weights for policy 1, policy_version 8571 (0.0008) [2023-12-26 15:25:45,415][105692] Updated weights for policy 0, policy_version 8581 (0.0010) [2023-12-26 15:25:45,474][105692] Updated weights for policy 0, policy_version 8591 (0.0010) [2023-12-26 15:25:45,526][105692] Updated weights for policy 0, policy_version 8601 (0.0010) [2023-12-26 15:25:45,773][105620] Updated weights for policy 1, policy_version 8581 (0.0008) [2023-12-26 15:25:45,827][105620] Updated weights for policy 1, policy_version 8591 (0.0008) [2023-12-26 15:25:45,886][105620] Updated weights for policy 1, policy_version 8601 (0.0008) [2023-12-26 15:25:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.3, 300 sec: 19162.2). Total num frames: 4407296. Throughput: 0: 9931.2, 1: 9641.1. Samples: 4376688. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 15:25:46,063][104569] Avg episode reward: [(0, '8795.392'), (1, '8515.939')] [2023-12-26 15:25:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000008608_2203648.pth... [2023-12-26 15:25:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000008608_2203648.pth... [2023-12-26 15:25:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000007456_1908736.pth [2023-12-26 15:25:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000007488_1916928.pth [2023-12-26 15:25:46,284][105692] Updated weights for policy 0, policy_version 8611 (0.0011) [2023-12-26 15:25:46,333][105692] Updated weights for policy 0, policy_version 8621 (0.0010) [2023-12-26 15:25:46,377][105692] Updated weights for policy 0, policy_version 8631 (0.0010) [2023-12-26 15:25:46,659][105620] Updated weights for policy 1, policy_version 8611 (0.0008) [2023-12-26 15:25:46,713][105620] Updated weights for policy 1, policy_version 8621 (0.0008) [2023-12-26 15:25:46,768][105620] Updated weights for policy 1, policy_version 8631 (0.0008) [2023-12-26 15:25:47,140][105692] Updated weights for policy 0, policy_version 8641 (0.0010) [2023-12-26 15:25:47,192][105692] Updated weights for policy 0, policy_version 8651 (0.0010) [2023-12-26 15:25:47,243][105692] Updated weights for policy 0, policy_version 8661 (0.0010) [2023-12-26 15:25:47,298][105692] Updated weights for policy 0, policy_version 8671 (0.0010) [2023-12-26 15:25:47,519][105620] Updated weights for policy 1, policy_version 8641 (0.0008) [2023-12-26 15:25:47,586][105620] Updated weights for policy 1, policy_version 8651 (0.0008) [2023-12-26 15:25:47,649][105620] Updated weights for policy 1, policy_version 8661 (0.0008) [2023-12-26 15:25:47,705][105620] Updated weights for policy 1, policy_version 8671 (0.0008) [2023-12-26 15:25:48,069][105692] Updated weights for policy 0, policy_version 8681 (0.0010) [2023-12-26 15:25:48,130][105692] Updated weights for policy 0, policy_version 8691 (0.0010) [2023-12-26 15:25:48,195][105692] Updated weights for policy 0, policy_version 8701 (0.0010) [2023-12-26 15:25:48,493][105620] Updated weights for policy 1, policy_version 8681 (0.0008) [2023-12-26 15:25:48,553][105620] Updated weights for policy 1, policy_version 8691 (0.0008) [2023-12-26 15:25:48,609][105620] Updated weights for policy 1, policy_version 8701 (0.0008) [2023-12-26 15:25:48,871][105692] Updated weights for policy 0, policy_version 8711 (0.0010) [2023-12-26 15:25:48,930][105692] Updated weights for policy 0, policy_version 8721 (0.0010) [2023-12-26 15:25:48,989][105692] Updated weights for policy 0, policy_version 8731 (0.0010) [2023-12-26 15:25:49,470][105620] Updated weights for policy 1, policy_version 8711 (0.0009) [2023-12-26 15:25:49,533][105620] Updated weights for policy 1, policy_version 8721 (0.0008) [2023-12-26 15:25:49,592][105620] Updated weights for policy 1, policy_version 8731 (0.0008) [2023-12-26 15:25:49,667][105692] Updated weights for policy 0, policy_version 8741 (0.0007) [2023-12-26 15:25:49,721][105692] Updated weights for policy 0, policy_version 8751 (0.0010) [2023-12-26 15:25:49,776][105692] Updated weights for policy 0, policy_version 8761 (0.0010) [2023-12-26 15:25:50,364][105620] Updated weights for policy 1, policy_version 8741 (0.0008) [2023-12-26 15:25:50,434][105620] Updated weights for policy 1, policy_version 8751 (0.0008) [2023-12-26 15:25:50,494][105620] Updated weights for policy 1, policy_version 8761 (0.0008) [2023-12-26 15:25:50,536][105692] Updated weights for policy 0, policy_version 8771 (0.0010) [2023-12-26 15:25:50,601][105692] Updated weights for policy 0, policy_version 8781 (0.0010) [2023-12-26 15:25:50,653][105692] Updated weights for policy 0, policy_version 8791 (0.0010) [2023-12-26 15:25:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19137.9). Total num frames: 4497408. Throughput: 0: 9872.6, 1: 9565.9. Samples: 4489072. Policy #0 lag: (min: 17.0, avg: 38.8, max: 41.0) [2023-12-26 15:25:51,063][104569] Avg episode reward: [(0, '8331.673'), (1, '8422.880')] [2023-12-26 15:25:51,211][105620] Updated weights for policy 1, policy_version 8771 (0.0009) [2023-12-26 15:25:51,276][105620] Updated weights for policy 1, policy_version 8781 (0.0009) [2023-12-26 15:25:51,335][105620] Updated weights for policy 1, policy_version 8791 (0.0008) [2023-12-26 15:25:51,416][105692] Updated weights for policy 0, policy_version 8801 (0.0010) [2023-12-26 15:25:51,475][105692] Updated weights for policy 0, policy_version 8811 (0.0010) [2023-12-26 15:25:51,534][105692] Updated weights for policy 0, policy_version 8821 (0.0010) [2023-12-26 15:25:51,592][105692] Updated weights for policy 0, policy_version 8831 (0.0010) [2023-12-26 15:25:52,061][105620] Updated weights for policy 1, policy_version 8801 (0.0010) [2023-12-26 15:25:52,126][105620] Updated weights for policy 1, policy_version 8811 (0.0008) [2023-12-26 15:25:52,176][105620] Updated weights for policy 1, policy_version 8821 (0.0005) [2023-12-26 15:25:52,230][105620] Updated weights for policy 1, policy_version 8831 (0.0005) [2023-12-26 15:25:52,379][105692] Updated weights for policy 0, policy_version 8841 (0.0009) [2023-12-26 15:25:52,441][105692] Updated weights for policy 0, policy_version 8851 (0.0010) [2023-12-26 15:25:52,496][105692] Updated weights for policy 0, policy_version 8861 (0.0010) [2023-12-26 15:25:52,980][105620] Updated weights for policy 1, policy_version 8841 (0.0008) [2023-12-26 15:25:53,032][105620] Updated weights for policy 1, policy_version 8851 (0.0008) [2023-12-26 15:25:53,084][105620] Updated weights for policy 1, policy_version 8861 (0.0008) [2023-12-26 15:25:53,219][105692] Updated weights for policy 0, policy_version 8871 (0.0010) [2023-12-26 15:25:53,283][105692] Updated weights for policy 0, policy_version 8881 (0.0010) [2023-12-26 15:25:53,337][105692] Updated weights for policy 0, policy_version 8891 (0.0010) [2023-12-26 15:25:53,841][105620] Updated weights for policy 1, policy_version 8871 (0.0008) [2023-12-26 15:25:53,892][105620] Updated weights for policy 1, policy_version 8881 (0.0007) [2023-12-26 15:25:53,957][105620] Updated weights for policy 1, policy_version 8891 (0.0008) [2023-12-26 15:25:54,082][105692] Updated weights for policy 0, policy_version 8901 (0.0010) [2023-12-26 15:25:54,134][105692] Updated weights for policy 0, policy_version 8911 (0.0010) [2023-12-26 15:25:54,182][105692] Updated weights for policy 0, policy_version 8921 (0.0010) [2023-12-26 15:25:54,617][105620] Updated weights for policy 1, policy_version 8901 (0.0009) [2023-12-26 15:25:54,665][105620] Updated weights for policy 1, policy_version 8911 (0.0010) [2023-12-26 15:25:54,713][105620] Updated weights for policy 1, policy_version 8921 (0.0010) [2023-12-26 15:25:54,833][105692] Updated weights for policy 0, policy_version 8931 (0.0009) [2023-12-26 15:25:54,896][105692] Updated weights for policy 0, policy_version 8941 (0.0005) [2023-12-26 15:25:54,943][105692] Updated weights for policy 0, policy_version 8951 (0.0005) [2023-12-26 15:25:55,454][105692] Updated weights for policy 0, policy_version 8961 (0.0008) [2023-12-26 15:25:55,458][105620] Updated weights for policy 1, policy_version 8931 (0.0009) [2023-12-26 15:25:55,509][105692] Updated weights for policy 0, policy_version 8971 (0.0008) [2023-12-26 15:25:55,515][105620] Updated weights for policy 1, policy_version 8941 (0.0005) [2023-12-26 15:25:55,559][105692] Updated weights for policy 0, policy_version 8981 (0.0010) [2023-12-26 15:25:55,572][105620] Updated weights for policy 1, policy_version 8951 (0.0005) [2023-12-26 15:25:55,606][105692] Updated weights for policy 0, policy_version 8991 (0.0009) [2023-12-26 15:25:56,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19148.8). Total num frames: 4595712. Throughput: 0: 9813.9, 1: 9577.9. Samples: 4607008. Policy #0 lag: (min: 17.0, avg: 38.8, max: 41.0) [2023-12-26 15:25:56,063][104569] Avg episode reward: [(0, '8178.975'), (1, '7965.046')] [2023-12-26 15:25:56,152][105620] Updated weights for policy 1, policy_version 8961 (0.0006) [2023-12-26 15:25:56,195][105692] Updated weights for policy 0, policy_version 9001 (0.0010) [2023-12-26 15:25:56,203][105620] Updated weights for policy 1, policy_version 8971 (0.0010) [2023-12-26 15:25:56,243][105692] Updated weights for policy 0, policy_version 9011 (0.0010) [2023-12-26 15:25:56,251][105620] Updated weights for policy 1, policy_version 8981 (0.0010) [2023-12-26 15:25:56,292][105692] Updated weights for policy 0, policy_version 9021 (0.0010) [2023-12-26 15:25:56,299][105620] Updated weights for policy 1, policy_version 8991 (0.0010) [2023-12-26 15:25:56,910][105692] Updated weights for policy 0, policy_version 9031 (0.0007) [2023-12-26 15:25:56,974][105692] Updated weights for policy 0, policy_version 9041 (0.0006) [2023-12-26 15:25:57,037][105692] Updated weights for policy 0, policy_version 9051 (0.0011) [2023-12-26 15:25:57,053][105620] Updated weights for policy 1, policy_version 9001 (0.0011) [2023-12-26 15:25:57,117][105620] Updated weights for policy 1, policy_version 9011 (0.0011) [2023-12-26 15:25:57,176][105620] Updated weights for policy 1, policy_version 9021 (0.0010) [2023-12-26 15:25:57,709][105692] Updated weights for policy 0, policy_version 9061 (0.0009) [2023-12-26 15:25:57,753][105692] Updated weights for policy 0, policy_version 9071 (0.0010) [2023-12-26 15:25:57,801][105692] Updated weights for policy 0, policy_version 9081 (0.0010) [2023-12-26 15:25:57,900][105620] Updated weights for policy 1, policy_version 9031 (0.0007) [2023-12-26 15:25:57,965][105620] Updated weights for policy 1, policy_version 9041 (0.0005) [2023-12-26 15:25:58,021][105620] Updated weights for policy 1, policy_version 9051 (0.0005) [2023-12-26 15:25:58,564][105692] Updated weights for policy 0, policy_version 9091 (0.0010) [2023-12-26 15:25:58,605][105620] Updated weights for policy 1, policy_version 9061 (0.0009) [2023-12-26 15:25:58,624][105692] Updated weights for policy 0, policy_version 9101 (0.0010) [2023-12-26 15:25:58,672][105620] Updated weights for policy 1, policy_version 9071 (0.0009) [2023-12-26 15:25:58,689][105692] Updated weights for policy 0, policy_version 9111 (0.0011) [2023-12-26 15:25:58,730][105620] Updated weights for policy 1, policy_version 9081 (0.0008) [2023-12-26 15:25:59,532][105692] Updated weights for policy 0, policy_version 9122 (0.0008) [2023-12-26 15:25:59,532][105620] Updated weights for policy 1, policy_version 9091 (0.0008) [2023-12-26 15:25:59,583][105692] Updated weights for policy 0, policy_version 9132 (0.0010) [2023-12-26 15:25:59,586][105620] Updated weights for policy 1, policy_version 9101 (0.0006) [2023-12-26 15:25:59,635][105692] Updated weights for policy 0, policy_version 9142 (0.0008) [2023-12-26 15:25:59,645][105620] Updated weights for policy 1, policy_version 9111 (0.0006) [2023-12-26 15:25:59,691][105585] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000010 [2023-12-26 15:25:59,693][105692] Updated weights for policy 0, policy_version 9152 (0.0010) [2023-12-26 15:26:00,382][105620] Updated weights for policy 1, policy_version 9121 (0.0007) [2023-12-26 15:26:00,444][105620] Updated weights for policy 1, policy_version 9131 (0.0011) [2023-12-26 15:26:00,465][105692] Updated weights for policy 0, policy_version 9162 (0.0011) [2023-12-26 15:26:00,503][105620] Updated weights for policy 1, policy_version 9141 (0.0010) [2023-12-26 15:26:00,521][105692] Updated weights for policy 0, policy_version 9172 (0.0010) [2023-12-26 15:26:00,569][105620] Updated weights for policy 1, policy_version 9151 (0.0010) [2023-12-26 15:26:00,569][105692] Updated weights for policy 0, policy_version 9182 (0.0010) [2023-12-26 15:26:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19159.2). Total num frames: 4694016. Throughput: 0: 9867.5, 1: 9579.3. Samples: 4667904. Policy #0 lag: (min: 26.0, avg: 36.0, max: 58.0) [2023-12-26 15:26:01,063][104569] Avg episode reward: [(0, '8548.440'), (1, '8149.837')] [2023-12-26 15:26:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000009184_2351104.pth... [2023-12-26 15:26:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000009152_2342912.pth... [2023-12-26 15:26:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000008032_2056192.pth [2023-12-26 15:26:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000008064_2064384.pth [2023-12-26 15:26:01,310][105620] Updated weights for policy 1, policy_version 9161 (0.0010) [2023-12-26 15:26:01,324][105692] Updated weights for policy 0, policy_version 9192 (0.0010) [2023-12-26 15:26:01,375][105620] Updated weights for policy 1, policy_version 9171 (0.0010) [2023-12-26 15:26:01,387][105692] Updated weights for policy 0, policy_version 9202 (0.0010) [2023-12-26 15:26:01,435][105620] Updated weights for policy 1, policy_version 9181 (0.0010) [2023-12-26 15:26:01,440][105692] Updated weights for policy 0, policy_version 9212 (0.0009) [2023-12-26 15:26:02,136][105620] Updated weights for policy 1, policy_version 9191 (0.0007) [2023-12-26 15:26:02,191][105620] Updated weights for policy 1, policy_version 9201 (0.0005) [2023-12-26 15:26:02,225][105692] Updated weights for policy 0, policy_version 9222 (0.0011) [2023-12-26 15:26:02,240][105620] Updated weights for policy 1, policy_version 9211 (0.0005) [2023-12-26 15:26:02,285][105692] Updated weights for policy 0, policy_version 9232 (0.0011) [2023-12-26 15:26:02,344][105692] Updated weights for policy 0, policy_version 9242 (0.0011) [2023-12-26 15:26:02,818][105620] Updated weights for policy 1, policy_version 9221 (0.0009) [2023-12-26 15:26:02,886][105620] Updated weights for policy 1, policy_version 9231 (0.0010) [2023-12-26 15:26:02,954][105620] Updated weights for policy 1, policy_version 9241 (0.0010) [2023-12-26 15:26:03,099][105692] Updated weights for policy 0, policy_version 9252 (0.0011) [2023-12-26 15:26:03,154][105692] Updated weights for policy 0, policy_version 9262 (0.0010) [2023-12-26 15:26:03,218][105692] Updated weights for policy 0, policy_version 9272 (0.0010) [2023-12-26 15:26:03,507][105620] Updated weights for policy 1, policy_version 9251 (0.0009) [2023-12-26 15:26:03,561][105620] Updated weights for policy 1, policy_version 9261 (0.0008) [2023-12-26 15:26:03,613][105620] Updated weights for policy 1, policy_version 9271 (0.0008) [2023-12-26 15:26:03,962][105692] Updated weights for policy 0, policy_version 9282 (0.0010) [2023-12-26 15:26:04,011][105692] Updated weights for policy 0, policy_version 9292 (0.0010) [2023-12-26 15:26:04,073][105692] Updated weights for policy 0, policy_version 9302 (0.0011) [2023-12-26 15:26:04,133][105692] Updated weights for policy 0, policy_version 9312 (0.0011) [2023-12-26 15:26:04,396][105620] Updated weights for policy 1, policy_version 9281 (0.0008) [2023-12-26 15:26:04,449][105620] Updated weights for policy 1, policy_version 9291 (0.0008) [2023-12-26 15:26:04,502][105620] Updated weights for policy 1, policy_version 9301 (0.0009) [2023-12-26 15:26:04,561][105620] Updated weights for policy 1, policy_version 9311 (0.0008) [2023-12-26 15:26:04,886][105692] Updated weights for policy 0, policy_version 9322 (0.0010) [2023-12-26 15:26:04,930][105692] Updated weights for policy 0, policy_version 9332 (0.0010) [2023-12-26 15:26:04,978][105692] Updated weights for policy 0, policy_version 9342 (0.0010) [2023-12-26 15:26:05,251][105620] Updated weights for policy 1, policy_version 9321 (0.0008) [2023-12-26 15:26:05,300][105620] Updated weights for policy 1, policy_version 9331 (0.0005) [2023-12-26 15:26:05,347][105620] Updated weights for policy 1, policy_version 9341 (0.0006) [2023-12-26 15:26:05,755][105692] Updated weights for policy 0, policy_version 9352 (0.0010) [2023-12-26 15:26:05,803][105692] Updated weights for policy 0, policy_version 9362 (0.0010) [2023-12-26 15:26:05,861][105692] Updated weights for policy 0, policy_version 9372 (0.0010) [2023-12-26 15:26:05,983][105620] Updated weights for policy 1, policy_version 9351 (0.0009) [2023-12-26 15:26:06,040][105620] Updated weights for policy 1, policy_version 9361 (0.0010) [2023-12-26 15:26:06,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.8, 300 sec: 19169.3). Total num frames: 4792320. Throughput: 0: 9705.5, 1: 9677.2. Samples: 4782708. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-26 15:26:06,062][104569] Avg episode reward: [(0, '8794.150'), (1, '8053.303')] [2023-12-26 15:26:06,108][105620] Updated weights for policy 1, policy_version 9371 (0.0009) [2023-12-26 15:26:06,621][105692] Updated weights for policy 0, policy_version 9382 (0.0010) [2023-12-26 15:26:06,681][105692] Updated weights for policy 0, policy_version 9392 (0.0010) [2023-12-26 15:26:06,747][105692] Updated weights for policy 0, policy_version 9402 (0.0010) [2023-12-26 15:26:06,843][105620] Updated weights for policy 1, policy_version 9381 (0.0008) [2023-12-26 15:26:06,900][105620] Updated weights for policy 1, policy_version 9391 (0.0005) [2023-12-26 15:26:06,961][105620] Updated weights for policy 1, policy_version 9401 (0.0006) [2023-12-26 15:26:07,429][105692] Updated weights for policy 0, policy_version 9412 (0.0008) [2023-12-26 15:26:07,481][105692] Updated weights for policy 0, policy_version 9422 (0.0005) [2023-12-26 15:26:07,530][105692] Updated weights for policy 0, policy_version 9432 (0.0005) [2023-12-26 15:26:07,574][105620] Updated weights for policy 1, policy_version 9411 (0.0006) [2023-12-26 15:26:07,644][105620] Updated weights for policy 1, policy_version 9421 (0.0006) [2023-12-26 15:26:07,710][105620] Updated weights for policy 1, policy_version 9431 (0.0005) [2023-12-26 15:26:08,057][105692] Updated weights for policy 0, policy_version 9442 (0.0006) [2023-12-26 15:26:08,122][105692] Updated weights for policy 0, policy_version 9452 (0.0005) [2023-12-26 15:26:08,191][105692] Updated weights for policy 0, policy_version 9462 (0.0005) [2023-12-26 15:26:08,257][105692] Updated weights for policy 0, policy_version 9472 (0.0005) [2023-12-26 15:26:08,341][105620] Updated weights for policy 1, policy_version 9441 (0.0008) [2023-12-26 15:26:08,407][105620] Updated weights for policy 1, policy_version 9451 (0.0009) [2023-12-26 15:26:08,477][105620] Updated weights for policy 1, policy_version 9461 (0.0008) [2023-12-26 15:26:08,536][105620] Updated weights for policy 1, policy_version 9471 (0.0008) [2023-12-26 15:26:08,851][105692] Updated weights for policy 0, policy_version 9482 (0.0005) [2023-12-26 15:26:08,906][105692] Updated weights for policy 0, policy_version 9492 (0.0005) [2023-12-26 15:26:08,959][105692] Updated weights for policy 0, policy_version 9502 (0.0005) [2023-12-26 15:26:09,170][105620] Updated weights for policy 1, policy_version 9481 (0.0011) [2023-12-26 15:26:09,237][105620] Updated weights for policy 1, policy_version 9491 (0.0009) [2023-12-26 15:26:09,301][105620] Updated weights for policy 1, policy_version 9501 (0.0008) [2023-12-26 15:26:09,653][105692] Updated weights for policy 0, policy_version 9512 (0.0009) [2023-12-26 15:26:09,723][105692] Updated weights for policy 0, policy_version 9522 (0.0010) [2023-12-26 15:26:09,787][105692] Updated weights for policy 0, policy_version 9532 (0.0011) [2023-12-26 15:26:09,990][105620] Updated weights for policy 1, policy_version 9511 (0.0009) [2023-12-26 15:26:10,039][105620] Updated weights for policy 1, policy_version 9521 (0.0008) [2023-12-26 15:26:10,091][105620] Updated weights for policy 1, policy_version 9531 (0.0008) [2023-12-26 15:26:10,550][105692] Updated weights for policy 0, policy_version 9542 (0.0010) [2023-12-26 15:26:10,612][105692] Updated weights for policy 0, policy_version 9552 (0.0009) [2023-12-26 15:26:10,671][105692] Updated weights for policy 0, policy_version 9562 (0.0009) [2023-12-26 15:26:10,859][105620] Updated weights for policy 1, policy_version 9541 (0.0009) [2023-12-26 15:26:10,919][105620] Updated weights for policy 1, policy_version 9551 (0.0011) [2023-12-26 15:26:10,984][105620] Updated weights for policy 1, policy_version 9561 (0.0010) [2023-12-26 15:26:11,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.4, 300 sec: 19211.0). Total num frames: 4898816. Throughput: 0: 9741.8, 1: 9746.9. Samples: 4904576. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-26 15:26:11,062][104569] Avg episode reward: [(0, '8887.678'), (1, '8052.939')] [2023-12-26 15:26:11,063][105585] Saving new best policy, reward=8887.678! [2023-12-26 15:26:11,474][105692] Updated weights for policy 0, policy_version 9572 (0.0009) [2023-12-26 15:26:11,539][105692] Updated weights for policy 0, policy_version 9582 (0.0008) [2023-12-26 15:26:11,589][105692] Updated weights for policy 0, policy_version 9592 (0.0008) [2023-12-26 15:26:11,758][105620] Updated weights for policy 1, policy_version 9571 (0.0008) [2023-12-26 15:26:11,820][105620] Updated weights for policy 1, policy_version 9581 (0.0010) [2023-12-26 15:26:11,884][105620] Updated weights for policy 1, policy_version 9591 (0.0011) [2023-12-26 15:26:12,393][105692] Updated weights for policy 0, policy_version 9602 (0.0009) [2023-12-26 15:26:12,440][105692] Updated weights for policy 0, policy_version 9612 (0.0008) [2023-12-26 15:26:12,492][105692] Updated weights for policy 0, policy_version 9622 (0.0009) [2023-12-26 15:26:12,549][105692] Updated weights for policy 0, policy_version 9632 (0.0009) [2023-12-26 15:26:12,552][105620] Updated weights for policy 1, policy_version 9601 (0.0010) [2023-12-26 15:26:12,600][105620] Updated weights for policy 1, policy_version 9611 (0.0006) [2023-12-26 15:26:12,661][105620] Updated weights for policy 1, policy_version 9621 (0.0007) [2023-12-26 15:26:12,736][105620] Updated weights for policy 1, policy_version 9631 (0.0008) [2023-12-26 15:26:13,326][105692] Updated weights for policy 0, policy_version 9642 (0.0010) [2023-12-26 15:26:13,381][105692] Updated weights for policy 0, policy_version 9652 (0.0010) [2023-12-26 15:26:13,410][105620] Updated weights for policy 1, policy_version 9641 (0.0006) [2023-12-26 15:26:13,439][105692] Updated weights for policy 0, policy_version 9662 (0.0010) [2023-12-26 15:26:13,466][105620] Updated weights for policy 1, policy_version 9651 (0.0006) [2023-12-26 15:26:13,518][105620] Updated weights for policy 1, policy_version 9661 (0.0008) [2023-12-26 15:26:14,063][105692] Updated weights for policy 0, policy_version 9672 (0.0006) [2023-12-26 15:26:14,132][105692] Updated weights for policy 0, policy_version 9682 (0.0009) [2023-12-26 15:26:14,156][105620] Updated weights for policy 1, policy_version 9671 (0.0006) [2023-12-26 15:26:14,190][105692] Updated weights for policy 0, policy_version 9692 (0.0007) [2023-12-26 15:26:14,221][105620] Updated weights for policy 1, policy_version 9681 (0.0006) [2023-12-26 15:26:14,273][105620] Updated weights for policy 1, policy_version 9691 (0.0006) [2023-12-26 15:26:14,739][105692] Updated weights for policy 0, policy_version 9702 (0.0005) [2023-12-26 15:26:14,805][105692] Updated weights for policy 0, policy_version 9712 (0.0007) [2023-12-26 15:26:14,867][105692] Updated weights for policy 0, policy_version 9722 (0.0009) [2023-12-26 15:26:14,991][105620] Updated weights for policy 1, policy_version 9701 (0.0008) [2023-12-26 15:26:15,048][105620] Updated weights for policy 1, policy_version 9711 (0.0009) [2023-12-26 15:26:15,102][105620] Updated weights for policy 1, policy_version 9721 (0.0010) [2023-12-26 15:26:15,567][105692] Updated weights for policy 0, policy_version 9732 (0.0009) [2023-12-26 15:26:15,621][105692] Updated weights for policy 0, policy_version 9742 (0.0009) [2023-12-26 15:26:15,676][105692] Updated weights for policy 0, policy_version 9752 (0.0009) [2023-12-26 15:26:15,861][105620] Updated weights for policy 1, policy_version 9731 (0.0008) [2023-12-26 15:26:15,925][105620] Updated weights for policy 1, policy_version 9741 (0.0005) [2023-12-26 15:26:15,988][105620] Updated weights for policy 1, policy_version 9751 (0.0005) [2023-12-26 15:26:16,062][104569] Fps is (10 sec: 20479.3, 60 sec: 19660.7, 300 sec: 19219.7). Total num frames: 4997120. Throughput: 0: 9650.8, 1: 9790.7. Samples: 4960372. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-26 15:26:16,063][104569] Avg episode reward: [(0, '9073.362'), (1, '8239.119')] [2023-12-26 15:26:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000009760_2498560.pth... [2023-12-26 15:26:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000009760_2498560.pth... [2023-12-26 15:26:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000008608_2203648.pth [2023-12-26 15:26:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000008608_2203648.pth [2023-12-26 15:26:16,075][105585] Saving new best policy, reward=9073.362! [2023-12-26 15:26:16,518][105620] Updated weights for policy 1, policy_version 9761 (0.0005) [2023-12-26 15:26:16,520][105692] Updated weights for policy 0, policy_version 9762 (0.0009) [2023-12-26 15:26:16,570][105692] Updated weights for policy 0, policy_version 9772 (0.0008) [2023-12-26 15:26:16,570][105620] Updated weights for policy 1, policy_version 9771 (0.0005) [2023-12-26 15:26:16,629][105692] Updated weights for policy 0, policy_version 9782 (0.0007) [2023-12-26 15:26:16,636][105620] Updated weights for policy 1, policy_version 9781 (0.0010) [2023-12-26 15:26:16,687][105692] Updated weights for policy 0, policy_version 9792 (0.0006) [2023-12-26 15:26:16,693][105620] Updated weights for policy 1, policy_version 9791 (0.0011) [2023-12-26 15:26:17,392][105620] Updated weights for policy 1, policy_version 9801 (0.0009) [2023-12-26 15:26:17,417][105692] Updated weights for policy 0, policy_version 9802 (0.0005) [2023-12-26 15:26:17,441][105620] Updated weights for policy 1, policy_version 9811 (0.0009) [2023-12-26 15:26:17,471][105692] Updated weights for policy 0, policy_version 9812 (0.0006) [2023-12-26 15:26:17,504][105620] Updated weights for policy 1, policy_version 9821 (0.0009) [2023-12-26 15:26:17,529][105692] Updated weights for policy 0, policy_version 9822 (0.0007) [2023-12-26 15:26:18,076][105692] Updated weights for policy 0, policy_version 9832 (0.0006) [2023-12-26 15:26:18,141][105692] Updated weights for policy 0, policy_version 9842 (0.0006) [2023-12-26 15:26:18,177][105620] Updated weights for policy 1, policy_version 9831 (0.0006) [2023-12-26 15:26:18,200][105692] Updated weights for policy 0, policy_version 9852 (0.0006) [2023-12-26 15:26:18,237][105620] Updated weights for policy 1, policy_version 9841 (0.0007) [2023-12-26 15:26:18,300][105620] Updated weights for policy 1, policy_version 9851 (0.0010) [2023-12-26 15:26:18,803][105692] Updated weights for policy 0, policy_version 9862 (0.0006) [2023-12-26 15:26:18,863][105692] Updated weights for policy 0, policy_version 9872 (0.0005) [2023-12-26 15:26:18,923][105692] Updated weights for policy 0, policy_version 9882 (0.0007) [2023-12-26 15:26:19,023][105620] Updated weights for policy 1, policy_version 9861 (0.0008) [2023-12-26 15:26:19,091][105620] Updated weights for policy 1, policy_version 9871 (0.0005) [2023-12-26 15:26:19,145][105620] Updated weights for policy 1, policy_version 9881 (0.0006) [2023-12-26 15:26:19,695][105692] Updated weights for policy 0, policy_version 9892 (0.0008) [2023-12-26 15:26:19,759][105692] Updated weights for policy 0, policy_version 9902 (0.0008) [2023-12-26 15:26:19,783][105620] Updated weights for policy 1, policy_version 9891 (0.0007) [2023-12-26 15:26:19,826][105692] Updated weights for policy 0, policy_version 9912 (0.0007) [2023-12-26 15:26:19,847][105620] Updated weights for policy 1, policy_version 9901 (0.0010) [2023-12-26 15:26:19,904][105620] Updated weights for policy 1, policy_version 9911 (0.0011) [2023-12-26 15:26:20,548][105692] Updated weights for policy 0, policy_version 9922 (0.0007) [2023-12-26 15:26:20,608][105692] Updated weights for policy 0, policy_version 9932 (0.0007) [2023-12-26 15:26:20,664][105620] Updated weights for policy 1, policy_version 9921 (0.0010) [2023-12-26 15:26:20,666][105692] Updated weights for policy 0, policy_version 9942 (0.0008) [2023-12-26 15:26:20,716][105692] Updated weights for policy 0, policy_version 9952 (0.0007) [2023-12-26 15:26:20,735][105620] Updated weights for policy 1, policy_version 9931 (0.0011) [2023-12-26 15:26:20,791][105620] Updated weights for policy 1, policy_version 9941 (0.0011) [2023-12-26 15:26:20,844][105620] Updated weights for policy 1, policy_version 9951 (0.0011) [2023-12-26 15:26:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19228.0). Total num frames: 5095424. Throughput: 0: 9719.8, 1: 9830.1. Samples: 5083848. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-26 15:26:21,063][104569] Avg episode reward: [(0, '8611.489'), (1, '7499.770')] [2023-12-26 15:26:21,483][105692] Updated weights for policy 0, policy_version 9962 (0.0008) [2023-12-26 15:26:21,550][105692] Updated weights for policy 0, policy_version 9972 (0.0007) [2023-12-26 15:26:21,607][105620] Updated weights for policy 1, policy_version 9961 (0.0011) [2023-12-26 15:26:21,612][105692] Updated weights for policy 0, policy_version 9982 (0.0009) [2023-12-26 15:26:21,674][105620] Updated weights for policy 1, policy_version 9971 (0.0011) [2023-12-26 15:26:21,744][105620] Updated weights for policy 1, policy_version 9981 (0.0010) [2023-12-26 15:26:22,387][105692] Updated weights for policy 0, policy_version 9992 (0.0008) [2023-12-26 15:26:22,424][105620] Updated weights for policy 1, policy_version 9991 (0.0009) [2023-12-26 15:26:22,435][105692] Updated weights for policy 0, policy_version 10002 (0.0008) [2023-12-26 15:26:22,488][105620] Updated weights for policy 1, policy_version 10001 (0.0008) [2023-12-26 15:26:22,491][105692] Updated weights for policy 0, policy_version 10012 (0.0008) [2023-12-26 15:26:22,553][105620] Updated weights for policy 1, policy_version 10011 (0.0008) [2023-12-26 15:26:23,147][105620] Updated weights for policy 1, policy_version 10021 (0.0008) [2023-12-26 15:26:23,212][105620] Updated weights for policy 1, policy_version 10031 (0.0008) [2023-12-26 15:26:23,273][105620] Updated weights for policy 1, policy_version 10041 (0.0007) [2023-12-26 15:26:23,306][105692] Updated weights for policy 0, policy_version 10022 (0.0006) [2023-12-26 15:26:23,369][105692] Updated weights for policy 0, policy_version 10032 (0.0009) [2023-12-26 15:26:23,432][105692] Updated weights for policy 0, policy_version 10042 (0.0006) [2023-12-26 15:26:23,891][105620] Updated weights for policy 1, policy_version 10051 (0.0007) [2023-12-26 15:26:23,940][105620] Updated weights for policy 1, policy_version 10061 (0.0007) [2023-12-26 15:26:23,993][105620] Updated weights for policy 1, policy_version 10071 (0.0008) [2023-12-26 15:26:24,130][105692] Updated weights for policy 0, policy_version 10052 (0.0008) [2023-12-26 15:26:24,187][105692] Updated weights for policy 0, policy_version 10062 (0.0008) [2023-12-26 15:26:24,253][105692] Updated weights for policy 0, policy_version 10072 (0.0005) [2023-12-26 15:26:24,773][105620] Updated weights for policy 1, policy_version 10081 (0.0009) [2023-12-26 15:26:24,838][105620] Updated weights for policy 1, policy_version 10091 (0.0010) [2023-12-26 15:26:24,869][105692] Updated weights for policy 0, policy_version 10082 (0.0008) [2023-12-26 15:26:24,889][105620] Updated weights for policy 1, policy_version 10101 (0.0006) [2023-12-26 15:26:24,929][105692] Updated weights for policy 0, policy_version 10092 (0.0006) [2023-12-26 15:26:24,939][105620] Updated weights for policy 1, policy_version 10111 (0.0007) [2023-12-26 15:26:24,996][105692] Updated weights for policy 0, policy_version 10102 (0.0005) [2023-12-26 15:26:25,044][105692] Updated weights for policy 0, policy_version 10112 (0.0005) [2023-12-26 15:26:25,657][105620] Updated weights for policy 1, policy_version 10121 (0.0006) [2023-12-26 15:26:25,700][105692] Updated weights for policy 0, policy_version 10123 (0.0009) [2023-12-26 15:26:25,716][105620] Updated weights for policy 1, policy_version 10131 (0.0005) [2023-12-26 15:26:25,751][105692] Updated weights for policy 0, policy_version 10134 (0.0009) [2023-12-26 15:26:25,771][105620] Updated weights for policy 1, policy_version 10141 (0.0005) [2023-12-26 15:26:25,808][105692] Updated weights for policy 0, policy_version 10144 (0.0009) [2023-12-26 15:26:26,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.8, 300 sec: 19236.0). Total num frames: 5193728. Throughput: 0: 9769.9, 1: 9861.5. Samples: 5201420. Policy #0 lag: (min: 31.0, avg: 39.5, max: 63.0) [2023-12-26 15:26:26,063][104569] Avg episode reward: [(0, '8334.218'), (1, '8055.720')] [2023-12-26 15:26:26,297][105620] Updated weights for policy 1, policy_version 10151 (0.0009) [2023-12-26 15:26:26,348][105620] Updated weights for policy 1, policy_version 10161 (0.0009) [2023-12-26 15:26:26,390][105620] Updated weights for policy 1, policy_version 10171 (0.0006) [2023-12-26 15:26:26,610][105692] Updated weights for policy 0, policy_version 10154 (0.0010) [2023-12-26 15:26:26,668][105692] Updated weights for policy 0, policy_version 10164 (0.0009) [2023-12-26 15:26:26,718][105692] Updated weights for policy 0, policy_version 10174 (0.0009) [2023-12-26 15:26:27,110][105620] Updated weights for policy 1, policy_version 10181 (0.0008) [2023-12-26 15:26:27,161][105620] Updated weights for policy 1, policy_version 10191 (0.0008) [2023-12-26 15:26:27,225][105620] Updated weights for policy 1, policy_version 10201 (0.0007) [2023-12-26 15:26:27,344][105692] Updated weights for policy 0, policy_version 10184 (0.0008) [2023-12-26 15:26:27,396][105692] Updated weights for policy 0, policy_version 10194 (0.0010) [2023-12-26 15:26:27,456][105692] Updated weights for policy 0, policy_version 10204 (0.0010) [2023-12-26 15:26:27,908][105620] Updated weights for policy 1, policy_version 10211 (0.0010) [2023-12-26 15:26:27,956][105620] Updated weights for policy 1, policy_version 10221 (0.0008) [2023-12-26 15:26:28,008][105620] Updated weights for policy 1, policy_version 10231 (0.0009) [2023-12-26 15:26:28,171][105692] Updated weights for policy 0, policy_version 10214 (0.0007) [2023-12-26 15:26:28,222][105692] Updated weights for policy 0, policy_version 10224 (0.0006) [2023-12-26 15:26:28,273][105692] Updated weights for policy 0, policy_version 10234 (0.0005) [2023-12-26 15:26:28,747][105620] Updated weights for policy 1, policy_version 10241 (0.0008) [2023-12-26 15:26:28,803][105620] Updated weights for policy 1, policy_version 10251 (0.0005) [2023-12-26 15:26:28,854][105620] Updated weights for policy 1, policy_version 10261 (0.0005) [2023-12-26 15:26:28,913][105620] Updated weights for policy 1, policy_version 10271 (0.0005) [2023-12-26 15:26:29,063][105692] Updated weights for policy 0, policy_version 10244 (0.0007) [2023-12-26 15:26:29,112][105692] Updated weights for policy 0, policy_version 10254 (0.0009) [2023-12-26 15:26:29,162][105692] Updated weights for policy 0, policy_version 10264 (0.0009) [2023-12-26 15:26:29,505][105620] Updated weights for policy 1, policy_version 10281 (0.0008) [2023-12-26 15:26:29,555][105620] Updated weights for policy 1, policy_version 10291 (0.0007) [2023-12-26 15:26:29,616][105620] Updated weights for policy 1, policy_version 10301 (0.0005) [2023-12-26 15:26:29,990][105692] Updated weights for policy 0, policy_version 10274 (0.0007) [2023-12-26 15:26:30,046][105692] Updated weights for policy 0, policy_version 10284 (0.0008) [2023-12-26 15:26:30,097][105692] Updated weights for policy 0, policy_version 10294 (0.0009) [2023-12-26 15:26:30,147][105692] Updated weights for policy 0, policy_version 10304 (0.0008) [2023-12-26 15:26:30,240][105620] Updated weights for policy 1, policy_version 10311 (0.0007) [2023-12-26 15:26:30,300][105620] Updated weights for policy 1, policy_version 10321 (0.0009) [2023-12-26 15:26:30,366][105620] Updated weights for policy 1, policy_version 10331 (0.0009) [2023-12-26 15:26:30,892][105692] Updated weights for policy 0, policy_version 10314 (0.0005) [2023-12-26 15:26:30,950][105692] Updated weights for policy 0, policy_version 10324 (0.0005) [2023-12-26 15:26:31,006][105692] Updated weights for policy 0, policy_version 10334 (0.0005) [2023-12-26 15:26:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19243.8). Total num frames: 5292032. Throughput: 0: 9779.4, 1: 9891.3. Samples: 5261868. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 15:26:31,062][104569] Avg episode reward: [(0, '8704.274'), (1, '8519.466')] [2023-12-26 15:26:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000010336_2646016.pth... [2023-12-26 15:26:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000010336_2646016.pth... [2023-12-26 15:26:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000009184_2351104.pth [2023-12-26 15:26:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000009152_2342912.pth [2023-12-26 15:26:31,168][105620] Updated weights for policy 1, policy_version 10341 (0.0008) [2023-12-26 15:26:31,220][105620] Updated weights for policy 1, policy_version 10351 (0.0009) [2023-12-26 15:26:31,281][105620] Updated weights for policy 1, policy_version 10361 (0.0009) [2023-12-26 15:26:31,595][105692] Updated weights for policy 0, policy_version 10344 (0.0006) [2023-12-26 15:26:31,665][105692] Updated weights for policy 0, policy_version 10354 (0.0011) [2023-12-26 15:26:31,729][105692] Updated weights for policy 0, policy_version 10364 (0.0011) [2023-12-26 15:26:32,145][105620] Updated weights for policy 1, policy_version 10371 (0.0009) [2023-12-26 15:26:32,189][105620] Updated weights for policy 1, policy_version 10381 (0.0008) [2023-12-26 15:26:32,235][105620] Updated weights for policy 1, policy_version 10391 (0.0007) [2023-12-26 15:26:32,404][105692] Updated weights for policy 0, policy_version 10374 (0.0007) [2023-12-26 15:26:32,467][105692] Updated weights for policy 0, policy_version 10384 (0.0005) [2023-12-26 15:26:32,529][105692] Updated weights for policy 0, policy_version 10394 (0.0011) [2023-12-26 15:26:33,077][105620] Updated weights for policy 1, policy_version 10401 (0.0008) [2023-12-26 15:26:33,121][105692] Updated weights for policy 0, policy_version 10404 (0.0009) [2023-12-26 15:26:33,141][105620] Updated weights for policy 1, policy_version 10411 (0.0008) [2023-12-26 15:26:33,165][105692] Updated weights for policy 0, policy_version 10414 (0.0005) [2023-12-26 15:26:33,202][105620] Updated weights for policy 1, policy_version 10421 (0.0008) [2023-12-26 15:26:33,209][105692] Updated weights for policy 0, policy_version 10424 (0.0005) [2023-12-26 15:26:33,264][105620] Updated weights for policy 1, policy_version 10431 (0.0009) [2023-12-26 15:26:33,780][105692] Updated weights for policy 0, policy_version 10434 (0.0005) [2023-12-26 15:26:33,838][105692] Updated weights for policy 0, policy_version 10444 (0.0005) [2023-12-26 15:26:33,886][105692] Updated weights for policy 0, policy_version 10454 (0.0005) [2023-12-26 15:26:33,944][105692] Updated weights for policy 0, policy_version 10464 (0.0005) [2023-12-26 15:26:34,086][105620] Updated weights for policy 1, policy_version 10441 (0.0009) [2023-12-26 15:26:34,145][105620] Updated weights for policy 1, policy_version 10451 (0.0009) [2023-12-26 15:26:34,212][105620] Updated weights for policy 1, policy_version 10461 (0.0006) [2023-12-26 15:26:34,583][105692] Updated weights for policy 0, policy_version 10474 (0.0007) [2023-12-26 15:26:34,649][105692] Updated weights for policy 0, policy_version 10484 (0.0006) [2023-12-26 15:26:34,709][105692] Updated weights for policy 0, policy_version 10494 (0.0009) [2023-12-26 15:26:34,896][105620] Updated weights for policy 1, policy_version 10471 (0.0008) [2023-12-26 15:26:34,946][105620] Updated weights for policy 1, policy_version 10482 (0.0008) [2023-12-26 15:26:35,002][105620] Updated weights for policy 1, policy_version 10492 (0.0008) [2023-12-26 15:26:35,404][105692] Updated weights for policy 0, policy_version 10504 (0.0009) [2023-12-26 15:26:35,461][105692] Updated weights for policy 0, policy_version 10514 (0.0009) [2023-12-26 15:26:35,526][105692] Updated weights for policy 0, policy_version 10524 (0.0009) [2023-12-26 15:26:35,719][105620] Updated weights for policy 1, policy_version 10502 (0.0010) [2023-12-26 15:26:35,779][105620] Updated weights for policy 1, policy_version 10512 (0.0009) [2023-12-26 15:26:35,836][105620] Updated weights for policy 1, policy_version 10522 (0.0008) [2023-12-26 15:26:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19251.2). Total num frames: 5390336. Throughput: 0: 9851.8, 1: 9933.4. Samples: 5379408. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 15:26:36,062][104569] Avg episode reward: [(0, '8795.899'), (1, '8427.348')] [2023-12-26 15:26:36,213][105692] Updated weights for policy 0, policy_version 10534 (0.0009) [2023-12-26 15:26:36,282][105692] Updated weights for policy 0, policy_version 10544 (0.0009) [2023-12-26 15:26:36,348][105692] Updated weights for policy 0, policy_version 10554 (0.0009) [2023-12-26 15:26:36,615][105620] Updated weights for policy 1, policy_version 10532 (0.0009) [2023-12-26 15:26:36,680][105620] Updated weights for policy 1, policy_version 10542 (0.0009) [2023-12-26 15:26:36,741][105620] Updated weights for policy 1, policy_version 10552 (0.0009) [2023-12-26 15:26:37,090][105692] Updated weights for policy 0, policy_version 10564 (0.0009) [2023-12-26 15:26:37,153][105692] Updated weights for policy 0, policy_version 10574 (0.0008) [2023-12-26 15:26:37,215][105692] Updated weights for policy 0, policy_version 10584 (0.0008) [2023-12-26 15:26:37,496][105620] Updated weights for policy 1, policy_version 10562 (0.0009) [2023-12-26 15:26:37,554][105620] Updated weights for policy 1, policy_version 10572 (0.0010) [2023-12-26 15:26:37,612][105620] Updated weights for policy 1, policy_version 10582 (0.0010) [2023-12-26 15:26:37,665][105620] Updated weights for policy 1, policy_version 10592 (0.0010) [2023-12-26 15:26:37,978][105692] Updated weights for policy 0, policy_version 10594 (0.0008) [2023-12-26 15:26:38,031][105692] Updated weights for policy 0, policy_version 10604 (0.0010) [2023-12-26 15:26:38,093][105692] Updated weights for policy 0, policy_version 10614 (0.0010) [2023-12-26 15:26:38,144][105692] Updated weights for policy 0, policy_version 10624 (0.0009) [2023-12-26 15:26:38,326][105620] Updated weights for policy 1, policy_version 10602 (0.0008) [2023-12-26 15:26:38,386][105620] Updated weights for policy 1, policy_version 10612 (0.0007) [2023-12-26 15:26:38,448][105620] Updated weights for policy 1, policy_version 10622 (0.0011) [2023-12-26 15:26:38,985][105692] Updated weights for policy 0, policy_version 10634 (0.0009) [2023-12-26 15:26:39,042][105692] Updated weights for policy 0, policy_version 10644 (0.0008) [2023-12-26 15:26:39,101][105692] Updated weights for policy 0, policy_version 10654 (0.0009) [2023-12-26 15:26:39,114][105620] Updated weights for policy 1, policy_version 10632 (0.0007) [2023-12-26 15:26:39,175][105620] Updated weights for policy 1, policy_version 10642 (0.0007) [2023-12-26 15:26:39,233][105620] Updated weights for policy 1, policy_version 10652 (0.0007) [2023-12-26 15:26:39,882][105692] Updated weights for policy 0, policy_version 10664 (0.0008) [2023-12-26 15:26:39,950][105692] Updated weights for policy 0, policy_version 10674 (0.0008) [2023-12-26 15:26:39,967][105620] Updated weights for policy 1, policy_version 10662 (0.0008) [2023-12-26 15:26:40,015][105692] Updated weights for policy 0, policy_version 10684 (0.0007) [2023-12-26 15:26:40,033][105620] Updated weights for policy 1, policy_version 10672 (0.0008) [2023-12-26 15:26:40,089][105620] Updated weights for policy 1, policy_version 10682 (0.0008) [2023-12-26 15:26:40,770][105620] Updated weights for policy 1, policy_version 10692 (0.0007) [2023-12-26 15:26:40,804][105692] Updated weights for policy 0, policy_version 10694 (0.0008) [2023-12-26 15:26:40,834][105620] Updated weights for policy 1, policy_version 10702 (0.0005) [2023-12-26 15:26:40,853][105692] Updated weights for policy 0, policy_version 10704 (0.0009) [2023-12-26 15:26:40,901][105620] Updated weights for policy 1, policy_version 10712 (0.0005) [2023-12-26 15:26:40,906][105692] Updated weights for policy 0, policy_version 10714 (0.0010) [2023-12-26 15:26:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19258.4). Total num frames: 5488640. Throughput: 0: 9761.9, 1: 9931.5. Samples: 5493208. Policy #0 lag: (min: 27.0, avg: 37.4, max: 59.0) [2023-12-26 15:26:41,062][104569] Avg episode reward: [(0, '8334.138'), (1, '7591.317')] [2023-12-26 15:26:41,609][105620] Updated weights for policy 1, policy_version 10722 (0.0010) [2023-12-26 15:26:41,673][105620] Updated weights for policy 1, policy_version 10732 (0.0008) [2023-12-26 15:26:41,681][105692] Updated weights for policy 0, policy_version 10724 (0.0008) [2023-12-26 15:26:41,740][105620] Updated weights for policy 1, policy_version 10742 (0.0008) [2023-12-26 15:26:41,751][105692] Updated weights for policy 0, policy_version 10734 (0.0008) [2023-12-26 15:26:41,795][105620] Updated weights for policy 1, policy_version 10752 (0.0007) [2023-12-26 15:26:41,813][105692] Updated weights for policy 0, policy_version 10744 (0.0008) [2023-12-26 15:26:42,431][105692] Updated weights for policy 0, policy_version 10754 (0.0007) [2023-12-26 15:26:42,489][105692] Updated weights for policy 0, policy_version 10764 (0.0006) [2023-12-26 15:26:42,519][105620] Updated weights for policy 1, policy_version 10762 (0.0006) [2023-12-26 15:26:42,549][105692] Updated weights for policy 0, policy_version 10774 (0.0011) [2023-12-26 15:26:42,579][105620] Updated weights for policy 1, policy_version 10772 (0.0006) [2023-12-26 15:26:42,612][105692] Updated weights for policy 0, policy_version 10784 (0.0011) [2023-12-26 15:26:42,631][105620] Updated weights for policy 1, policy_version 10782 (0.0006) [2023-12-26 15:26:43,275][105692] Updated weights for policy 0, policy_version 10794 (0.0006) [2023-12-26 15:26:43,315][105620] Updated weights for policy 1, policy_version 10792 (0.0008) [2023-12-26 15:26:43,332][105692] Updated weights for policy 0, policy_version 10804 (0.0005) [2023-12-26 15:26:43,366][105620] Updated weights for policy 1, policy_version 10802 (0.0009) [2023-12-26 15:26:43,388][105692] Updated weights for policy 0, policy_version 10814 (0.0006) [2023-12-26 15:26:43,413][105620] Updated weights for policy 1, policy_version 10812 (0.0008) [2023-12-26 15:26:44,004][105692] Updated weights for policy 0, policy_version 10824 (0.0009) [2023-12-26 15:26:44,064][105692] Updated weights for policy 0, policy_version 10834 (0.0009) [2023-12-26 15:26:44,086][105620] Updated weights for policy 1, policy_version 10822 (0.0009) [2023-12-26 15:26:44,120][105692] Updated weights for policy 0, policy_version 10844 (0.0005) [2023-12-26 15:26:44,138][105620] Updated weights for policy 1, policy_version 10832 (0.0010) [2023-12-26 15:26:44,195][105620] Updated weights for policy 1, policy_version 10842 (0.0010) [2023-12-26 15:26:44,776][105620] Updated weights for policy 1, policy_version 10852 (0.0008) [2023-12-26 15:26:44,824][105692] Updated weights for policy 0, policy_version 10854 (0.0006) [2023-12-26 15:26:44,837][105620] Updated weights for policy 1, policy_version 10862 (0.0010) [2023-12-26 15:26:44,887][105692] Updated weights for policy 0, policy_version 10864 (0.0006) [2023-12-26 15:26:44,897][105620] Updated weights for policy 1, policy_version 10872 (0.0011) [2023-12-26 15:26:44,948][105692] Updated weights for policy 0, policy_version 10874 (0.0005) [2023-12-26 15:26:45,623][105692] Updated weights for policy 0, policy_version 10884 (0.0008) [2023-12-26 15:26:45,634][105620] Updated weights for policy 1, policy_version 10882 (0.0010) [2023-12-26 15:26:45,691][105692] Updated weights for policy 0, policy_version 10894 (0.0005) [2023-12-26 15:26:45,697][105620] Updated weights for policy 1, policy_version 10892 (0.0011) [2023-12-26 15:26:45,755][105620] Updated weights for policy 1, policy_version 10902 (0.0010) [2023-12-26 15:26:45,756][105692] Updated weights for policy 0, policy_version 10904 (0.0009) [2023-12-26 15:26:45,817][105620] Updated weights for policy 1, policy_version 10912 (0.0010) [2023-12-26 15:26:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19265.3). Total num frames: 5586944. Throughput: 0: 9719.6, 1: 9920.4. Samples: 5551700. Policy #0 lag: (min: 27.0, avg: 37.4, max: 59.0) [2023-12-26 15:26:46,063][104569] Avg episode reward: [(0, '8149.373'), (1, '7406.602')] [2023-12-26 15:26:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000010912_2793472.pth... [2023-12-26 15:26:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000010912_2793472.pth... [2023-12-26 15:26:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000009760_2498560.pth [2023-12-26 15:26:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000009760_2498560.pth [2023-12-26 15:26:46,279][105692] Updated weights for policy 0, policy_version 10914 (0.0007) [2023-12-26 15:26:46,330][105692] Updated weights for policy 0, policy_version 10924 (0.0010) [2023-12-26 15:26:46,388][105692] Updated weights for policy 0, policy_version 10934 (0.0010) [2023-12-26 15:26:46,435][105692] Updated weights for policy 0, policy_version 10944 (0.0010) [2023-12-26 15:26:46,549][105620] Updated weights for policy 1, policy_version 10922 (0.0010) [2023-12-26 15:26:46,597][105620] Updated weights for policy 1, policy_version 10932 (0.0010) [2023-12-26 15:26:46,648][105620] Updated weights for policy 1, policy_version 10942 (0.0010) [2023-12-26 15:26:47,038][105692] Updated weights for policy 0, policy_version 10954 (0.0010) [2023-12-26 15:26:47,096][105692] Updated weights for policy 0, policy_version 10964 (0.0010) [2023-12-26 15:26:47,158][105692] Updated weights for policy 0, policy_version 10974 (0.0007) [2023-12-26 15:26:47,359][105620] Updated weights for policy 1, policy_version 10952 (0.0010) [2023-12-26 15:26:47,417][105620] Updated weights for policy 1, policy_version 10962 (0.0010) [2023-12-26 15:26:47,481][105620] Updated weights for policy 1, policy_version 10972 (0.0010) [2023-12-26 15:26:47,799][105692] Updated weights for policy 0, policy_version 10984 (0.0009) [2023-12-26 15:26:47,854][105692] Updated weights for policy 0, policy_version 10994 (0.0010) [2023-12-26 15:26:47,909][105692] Updated weights for policy 0, policy_version 11004 (0.0010) [2023-12-26 15:26:48,216][105620] Updated weights for policy 1, policy_version 10982 (0.0010) [2023-12-26 15:26:48,278][105620] Updated weights for policy 1, policy_version 10992 (0.0010) [2023-12-26 15:26:48,340][105620] Updated weights for policy 1, policy_version 11002 (0.0010) [2023-12-26 15:26:48,601][105692] Updated weights for policy 0, policy_version 11014 (0.0010) [2023-12-26 15:26:48,670][105692] Updated weights for policy 0, policy_version 11024 (0.0011) [2023-12-26 15:26:48,737][105692] Updated weights for policy 0, policy_version 11034 (0.0010) [2023-12-26 15:26:49,044][105620] Updated weights for policy 1, policy_version 11012 (0.0011) [2023-12-26 15:26:49,101][105620] Updated weights for policy 1, policy_version 11022 (0.0011) [2023-12-26 15:26:49,160][105620] Updated weights for policy 1, policy_version 11032 (0.0010) [2023-12-26 15:26:49,493][105692] Updated weights for policy 0, policy_version 11044 (0.0010) [2023-12-26 15:26:49,561][105692] Updated weights for policy 0, policy_version 11054 (0.0008) [2023-12-26 15:26:49,625][105692] Updated weights for policy 0, policy_version 11064 (0.0008) [2023-12-26 15:26:49,860][105620] Updated weights for policy 1, policy_version 11042 (0.0010) [2023-12-26 15:26:49,928][105620] Updated weights for policy 1, policy_version 11052 (0.0008) [2023-12-26 15:26:49,984][105620] Updated weights for policy 1, policy_version 11062 (0.0010) [2023-12-26 15:26:50,039][105620] Updated weights for policy 1, policy_version 11072 (0.0010) [2023-12-26 15:26:50,300][105692] Updated weights for policy 0, policy_version 11074 (0.0010) [2023-12-26 15:26:50,363][105692] Updated weights for policy 0, policy_version 11084 (0.0011) [2023-12-26 15:26:50,422][105692] Updated weights for policy 0, policy_version 11094 (0.0011) [2023-12-26 15:26:50,487][105692] Updated weights for policy 0, policy_version 11104 (0.0010) [2023-12-26 15:26:50,746][105620] Updated weights for policy 1, policy_version 11082 (0.0010) [2023-12-26 15:26:50,812][105620] Updated weights for policy 1, policy_version 11092 (0.0011) [2023-12-26 15:26:50,874][105620] Updated weights for policy 1, policy_version 11102 (0.0010) [2023-12-26 15:26:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19272.0). Total num frames: 5685248. Throughput: 0: 9891.0, 1: 9915.1. Samples: 5673984. Policy #0 lag: (min: 26.0, avg: 37.4, max: 58.0) [2023-12-26 15:26:51,063][104569] Avg episode reward: [(0, '8612.148'), (1, '8241.409')] [2023-12-26 15:26:51,217][105692] Updated weights for policy 0, policy_version 11114 (0.0006) [2023-12-26 15:26:51,286][105692] Updated weights for policy 0, policy_version 11124 (0.0009) [2023-12-26 15:26:51,352][105692] Updated weights for policy 0, policy_version 11134 (0.0011) [2023-12-26 15:26:51,631][105620] Updated weights for policy 1, policy_version 11112 (0.0007) [2023-12-26 15:26:51,694][105620] Updated weights for policy 1, policy_version 11122 (0.0009) [2023-12-26 15:26:51,760][105620] Updated weights for policy 1, policy_version 11132 (0.0010) [2023-12-26 15:26:52,030][105692] Updated weights for policy 0, policy_version 11144 (0.0007) [2023-12-26 15:26:52,077][105692] Updated weights for policy 0, policy_version 11154 (0.0010) [2023-12-26 15:26:52,126][105692] Updated weights for policy 0, policy_version 11164 (0.0009) [2023-12-26 15:26:52,432][105620] Updated weights for policy 1, policy_version 11142 (0.0010) [2023-12-26 15:26:52,493][105620] Updated weights for policy 1, policy_version 11152 (0.0010) [2023-12-26 15:26:52,558][105620] Updated weights for policy 1, policy_version 11162 (0.0010) [2023-12-26 15:26:52,866][105692] Updated weights for policy 0, policy_version 11174 (0.0007) [2023-12-26 15:26:52,922][105692] Updated weights for policy 0, policy_version 11184 (0.0008) [2023-12-26 15:26:52,977][105692] Updated weights for policy 0, policy_version 11194 (0.0008) [2023-12-26 15:26:53,297][105620] Updated weights for policy 1, policy_version 11172 (0.0010) [2023-12-26 15:26:53,354][105620] Updated weights for policy 1, policy_version 11182 (0.0010) [2023-12-26 15:26:53,415][105620] Updated weights for policy 1, policy_version 11192 (0.0010) [2023-12-26 15:26:53,744][105692] Updated weights for policy 0, policy_version 11204 (0.0008) [2023-12-26 15:26:53,792][105692] Updated weights for policy 0, policy_version 11214 (0.0008) [2023-12-26 15:26:53,835][105692] Updated weights for policy 0, policy_version 11224 (0.0007) [2023-12-26 15:26:54,157][105620] Updated weights for policy 1, policy_version 11202 (0.0011) [2023-12-26 15:26:54,205][105620] Updated weights for policy 1, policy_version 11212 (0.0010) [2023-12-26 15:26:54,253][105620] Updated weights for policy 1, policy_version 11222 (0.0010) [2023-12-26 15:26:54,300][105620] Updated weights for policy 1, policy_version 11232 (0.0010) [2023-12-26 15:26:54,485][105692] Updated weights for policy 0, policy_version 11234 (0.0008) [2023-12-26 15:26:54,531][105692] Updated weights for policy 0, policy_version 11244 (0.0005) [2023-12-26 15:26:54,577][105692] Updated weights for policy 0, policy_version 11254 (0.0005) [2023-12-26 15:26:54,630][105692] Updated weights for policy 0, policy_version 11264 (0.0005) [2023-12-26 15:26:55,074][105620] Updated weights for policy 1, policy_version 11242 (0.0010) [2023-12-26 15:26:55,136][105620] Updated weights for policy 1, policy_version 11252 (0.0011) [2023-12-26 15:26:55,176][105692] Updated weights for policy 0, policy_version 11274 (0.0005) [2023-12-26 15:26:55,198][105620] Updated weights for policy 1, policy_version 11262 (0.0010) [2023-12-26 15:26:55,229][105692] Updated weights for policy 0, policy_version 11284 (0.0006) [2023-12-26 15:26:55,282][105692] Updated weights for policy 0, policy_version 11294 (0.0005) [2023-12-26 15:26:55,905][105692] Updated weights for policy 0, policy_version 11304 (0.0005) [2023-12-26 15:26:55,909][105620] Updated weights for policy 1, policy_version 11272 (0.0007) [2023-12-26 15:26:55,951][105692] Updated weights for policy 0, policy_version 11314 (0.0005) [2023-12-26 15:26:55,960][105620] Updated weights for policy 1, policy_version 11282 (0.0009) [2023-12-26 15:26:56,003][105692] Updated weights for policy 0, policy_version 11324 (0.0005) [2023-12-26 15:26:56,014][105620] Updated weights for policy 1, policy_version 11292 (0.0010) [2023-12-26 15:26:56,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19933.9, 300 sec: 19466.4). Total num frames: 5791744. Throughput: 0: 9930.3, 1: 9821.3. Samples: 5793400. Policy #0 lag: (min: 26.0, avg: 37.4, max: 58.0) [2023-12-26 15:26:56,062][104569] Avg episode reward: [(0, '8429.250'), (1, '8520.390')] [2023-12-26 15:26:56,610][105692] Updated weights for policy 0, policy_version 11334 (0.0005) [2023-12-26 15:26:56,665][105692] Updated weights for policy 0, policy_version 11344 (0.0005) [2023-12-26 15:26:56,670][105620] Updated weights for policy 1, policy_version 11302 (0.0007) [2023-12-26 15:26:56,722][105692] Updated weights for policy 0, policy_version 11354 (0.0007) [2023-12-26 15:26:56,732][105620] Updated weights for policy 1, policy_version 11312 (0.0005) [2023-12-26 15:26:56,788][105620] Updated weights for policy 1, policy_version 11322 (0.0005) [2023-12-26 15:26:57,356][105620] Updated weights for policy 1, policy_version 11332 (0.0007) [2023-12-26 15:26:57,411][105692] Updated weights for policy 0, policy_version 11364 (0.0010) [2023-12-26 15:26:57,416][105620] Updated weights for policy 1, policy_version 11342 (0.0006) [2023-12-26 15:26:57,473][105692] Updated weights for policy 0, policy_version 11374 (0.0011) [2023-12-26 15:26:57,475][105620] Updated weights for policy 1, policy_version 11352 (0.0008) [2023-12-26 15:26:57,528][105692] Updated weights for policy 0, policy_version 11384 (0.0010) [2023-12-26 15:26:58,221][105620] Updated weights for policy 1, policy_version 11362 (0.0006) [2023-12-26 15:26:58,270][105692] Updated weights for policy 0, policy_version 11394 (0.0007) [2023-12-26 15:26:58,277][105620] Updated weights for policy 1, policy_version 11372 (0.0007) [2023-12-26 15:26:58,322][105692] Updated weights for policy 0, policy_version 11404 (0.0010) [2023-12-26 15:26:58,328][105620] Updated weights for policy 1, policy_version 11382 (0.0006) [2023-12-26 15:26:58,385][105692] Updated weights for policy 0, policy_version 11414 (0.0009) [2023-12-26 15:26:58,445][105692] Updated weights for policy 0, policy_version 11424 (0.0010) [2023-12-26 15:26:59,163][105620] Updated weights for policy 1, policy_version 11393 (0.0008) [2023-12-26 15:26:59,234][105620] Updated weights for policy 1, policy_version 11403 (0.0008) [2023-12-26 15:26:59,280][105692] Updated weights for policy 0, policy_version 11434 (0.0009) [2023-12-26 15:26:59,299][105620] Updated weights for policy 1, policy_version 11413 (0.0007) [2023-12-26 15:26:59,342][105692] Updated weights for policy 0, policy_version 11444 (0.0010) [2023-12-26 15:26:59,366][105620] Updated weights for policy 1, policy_version 11423 (0.0008) [2023-12-26 15:26:59,405][105692] Updated weights for policy 0, policy_version 11454 (0.0011) [2023-12-26 15:26:59,983][105620] Updated weights for policy 1, policy_version 11433 (0.0008) [2023-12-26 15:27:00,041][105620] Updated weights for policy 1, policy_version 11443 (0.0009) [2023-12-26 15:27:00,096][105692] Updated weights for policy 0, policy_version 11464 (0.0010) [2023-12-26 15:27:00,098][105620] Updated weights for policy 1, policy_version 11453 (0.0007) [2023-12-26 15:27:00,144][105692] Updated weights for policy 0, policy_version 11474 (0.0010) [2023-12-26 15:27:00,195][105692] Updated weights for policy 0, policy_version 11484 (0.0010) [2023-12-26 15:27:00,842][105692] Updated weights for policy 0, policy_version 11494 (0.0007) [2023-12-26 15:27:00,894][105692] Updated weights for policy 0, policy_version 11504 (0.0005) [2023-12-26 15:27:00,913][105620] Updated weights for policy 1, policy_version 11463 (0.0007) [2023-12-26 15:27:00,949][105692] Updated weights for policy 0, policy_version 11514 (0.0005) [2023-12-26 15:27:00,967][105620] Updated weights for policy 1, policy_version 11473 (0.0006) [2023-12-26 15:27:01,020][105620] Updated weights for policy 1, policy_version 11484 (0.0010) [2023-12-26 15:27:01,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19933.8, 300 sec: 19466.4). Total num frames: 5890048. Throughput: 0: 10012.6, 1: 9830.6. Samples: 5853312. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 15:27:01,063][104569] Avg episode reward: [(0, '7963.520'), (1, '8613.598')] [2023-12-26 15:27:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000011520_2949120.pth... [2023-12-26 15:27:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000011488_2940928.pth... [2023-12-26 15:27:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000010336_2646016.pth [2023-12-26 15:27:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000010336_2646016.pth [2023-12-26 15:27:01,590][105692] Updated weights for policy 0, policy_version 11524 (0.0007) [2023-12-26 15:27:01,663][105692] Updated weights for policy 0, policy_version 11534 (0.0009) [2023-12-26 15:27:01,714][105692] Updated weights for policy 0, policy_version 11544 (0.0009) [2023-12-26 15:27:01,835][105620] Updated weights for policy 1, policy_version 11494 (0.0009) [2023-12-26 15:27:01,882][105620] Updated weights for policy 1, policy_version 11504 (0.0008) [2023-12-26 15:27:01,931][105620] Updated weights for policy 1, policy_version 11515 (0.0009) [2023-12-26 15:27:02,463][105692] Updated weights for policy 0, policy_version 11554 (0.0008) [2023-12-26 15:27:02,520][105692] Updated weights for policy 0, policy_version 11564 (0.0007) [2023-12-26 15:27:02,584][105692] Updated weights for policy 0, policy_version 11574 (0.0010) [2023-12-26 15:27:02,648][105692] Updated weights for policy 0, policy_version 11584 (0.0009) [2023-12-26 15:27:02,659][105620] Updated weights for policy 1, policy_version 11525 (0.0007) [2023-12-26 15:27:02,716][105620] Updated weights for policy 1, policy_version 11535 (0.0008) [2023-12-26 15:27:02,767][105620] Updated weights for policy 1, policy_version 11545 (0.0009) [2023-12-26 15:27:03,365][105692] Updated weights for policy 0, policy_version 11594 (0.0009) [2023-12-26 15:27:03,426][105692] Updated weights for policy 0, policy_version 11604 (0.0009) [2023-12-26 15:27:03,483][105692] Updated weights for policy 0, policy_version 11614 (0.0008) [2023-12-26 15:27:03,498][105620] Updated weights for policy 1, policy_version 11555 (0.0008) [2023-12-26 15:27:03,545][105620] Updated weights for policy 1, policy_version 11565 (0.0009) [2023-12-26 15:27:03,591][105620] Updated weights for policy 1, policy_version 11575 (0.0009) [2023-12-26 15:27:04,184][105692] Updated weights for policy 0, policy_version 11624 (0.0007) [2023-12-26 15:27:04,243][105692] Updated weights for policy 0, policy_version 11634 (0.0006) [2023-12-26 15:27:04,300][105692] Updated weights for policy 0, policy_version 11644 (0.0007) [2023-12-26 15:27:04,417][105620] Updated weights for policy 1, policy_version 11585 (0.0008) [2023-12-26 15:27:04,477][105620] Updated weights for policy 1, policy_version 11595 (0.0005) [2023-12-26 15:27:04,534][105620] Updated weights for policy 1, policy_version 11605 (0.0007) [2023-12-26 15:27:04,582][105620] Updated weights for policy 1, policy_version 11615 (0.0005) [2023-12-26 15:27:04,944][105692] Updated weights for policy 0, policy_version 11654 (0.0007) [2023-12-26 15:27:04,999][105692] Updated weights for policy 0, policy_version 11665 (0.0011) [2023-12-26 15:27:05,070][105692] Updated weights for policy 0, policy_version 11676 (0.0010) [2023-12-26 15:27:05,218][105620] Updated weights for policy 1, policy_version 11625 (0.0008) [2023-12-26 15:27:05,269][105620] Updated weights for policy 1, policy_version 11635 (0.0009) [2023-12-26 15:27:05,320][105620] Updated weights for policy 1, policy_version 11645 (0.0009) [2023-12-26 15:27:05,833][105692] Updated weights for policy 0, policy_version 11686 (0.0009) [2023-12-26 15:27:05,893][105692] Updated weights for policy 0, policy_version 11696 (0.0008) [2023-12-26 15:27:05,954][105692] Updated weights for policy 0, policy_version 11706 (0.0010) [2023-12-26 15:27:06,050][105620] Updated weights for policy 1, policy_version 11655 (0.0009) [2023-12-26 15:27:06,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19797.2, 300 sec: 19494.2). Total num frames: 5980160. Throughput: 0: 9951.5, 1: 9723.4. Samples: 5969224. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 15:27:06,063][104569] Avg episode reward: [(0, '7593.440'), (1, '8521.825')] [2023-12-26 15:27:06,118][105620] Updated weights for policy 1, policy_version 11665 (0.0009) [2023-12-26 15:27:06,180][105620] Updated weights for policy 1, policy_version 11675 (0.0008) [2023-12-26 15:27:06,772][105620] Updated weights for policy 1, policy_version 11685 (0.0007) [2023-12-26 15:27:06,831][105620] Updated weights for policy 1, policy_version 11695 (0.0007) [2023-12-26 15:27:06,833][105692] Updated weights for policy 0, policy_version 11716 (0.0008) [2023-12-26 15:27:06,892][105620] Updated weights for policy 1, policy_version 11705 (0.0007) [2023-12-26 15:27:06,898][105692] Updated weights for policy 0, policy_version 11726 (0.0006) [2023-12-26 15:27:06,953][105692] Updated weights for policy 0, policy_version 11736 (0.0007) [2023-12-26 15:27:07,643][105692] Updated weights for policy 0, policy_version 11746 (0.0010) [2023-12-26 15:27:07,650][105620] Updated weights for policy 1, policy_version 11715 (0.0008) [2023-12-26 15:27:07,695][105692] Updated weights for policy 0, policy_version 11756 (0.0010) [2023-12-26 15:27:07,708][105620] Updated weights for policy 1, policy_version 11725 (0.0005) [2023-12-26 15:27:07,739][105692] Updated weights for policy 0, policy_version 11766 (0.0010) [2023-12-26 15:27:07,758][105620] Updated weights for policy 1, policy_version 11735 (0.0005) [2023-12-26 15:27:07,792][105692] Updated weights for policy 0, policy_version 11776 (0.0010) [2023-12-26 15:27:08,409][105620] Updated weights for policy 1, policy_version 11745 (0.0006) [2023-12-26 15:27:08,457][105620] Updated weights for policy 1, policy_version 11755 (0.0005) [2023-12-26 15:27:08,520][105620] Updated weights for policy 1, policy_version 11765 (0.0006) [2023-12-26 15:27:08,560][105692] Updated weights for policy 0, policy_version 11786 (0.0010) [2023-12-26 15:27:08,581][105620] Updated weights for policy 1, policy_version 11775 (0.0007) [2023-12-26 15:27:08,619][105692] Updated weights for policy 0, policy_version 11796 (0.0005) [2023-12-26 15:27:08,683][105692] Updated weights for policy 0, policy_version 11806 (0.0005) [2023-12-26 15:27:09,213][105692] Updated weights for policy 0, policy_version 11816 (0.0006) [2023-12-26 15:27:09,276][105692] Updated weights for policy 0, policy_version 11826 (0.0008) [2023-12-26 15:27:09,337][105692] Updated weights for policy 0, policy_version 11836 (0.0010) [2023-12-26 15:27:09,412][105620] Updated weights for policy 1, policy_version 11785 (0.0008) [2023-12-26 15:27:09,486][105620] Updated weights for policy 1, policy_version 11795 (0.0008) [2023-12-26 15:27:09,554][105620] Updated weights for policy 1, policy_version 11805 (0.0008) [2023-12-26 15:27:10,085][105692] Updated weights for policy 0, policy_version 11846 (0.0009) [2023-12-26 15:27:10,153][105692] Updated weights for policy 0, policy_version 11856 (0.0006) [2023-12-26 15:27:10,209][105692] Updated weights for policy 0, policy_version 11866 (0.0009) [2023-12-26 15:27:10,290][105620] Updated weights for policy 1, policy_version 11815 (0.0009) [2023-12-26 15:27:10,350][105620] Updated weights for policy 1, policy_version 11825 (0.0009) [2023-12-26 15:27:10,415][105620] Updated weights for policy 1, policy_version 11835 (0.0009) [2023-12-26 15:27:10,894][105692] Updated weights for policy 0, policy_version 11877 (0.0008) [2023-12-26 15:27:10,946][105692] Updated weights for policy 0, policy_version 11887 (0.0009) [2023-12-26 15:27:10,997][105692] Updated weights for policy 0, policy_version 11897 (0.0009) [2023-12-26 15:27:11,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19660.7, 300 sec: 19494.2). Total num frames: 6078464. Throughput: 0: 9965.0, 1: 9676.2. Samples: 6085280. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 15:27:11,063][104569] Avg episode reward: [(0, '7779.221'), (1, '8430.608')] [2023-12-26 15:27:11,203][105620] Updated weights for policy 1, policy_version 11845 (0.0009) [2023-12-26 15:27:11,269][105620] Updated weights for policy 1, policy_version 11855 (0.0009) [2023-12-26 15:27:11,336][105620] Updated weights for policy 1, policy_version 11865 (0.0006) [2023-12-26 15:27:11,797][105692] Updated weights for policy 0, policy_version 11907 (0.0009) [2023-12-26 15:27:11,867][105692] Updated weights for policy 0, policy_version 11917 (0.0009) [2023-12-26 15:27:11,929][105692] Updated weights for policy 0, policy_version 11927 (0.0008) [2023-12-26 15:27:12,148][105620] Updated weights for policy 1, policy_version 11875 (0.0008) [2023-12-26 15:27:12,217][105620] Updated weights for policy 1, policy_version 11885 (0.0010) [2023-12-26 15:27:12,276][105620] Updated weights for policy 1, policy_version 11895 (0.0011) [2023-12-26 15:27:12,603][105692] Updated weights for policy 0, policy_version 11937 (0.0009) [2023-12-26 15:27:12,666][105692] Updated weights for policy 0, policy_version 11947 (0.0011) [2023-12-26 15:27:12,726][105692] Updated weights for policy 0, policy_version 11957 (0.0011) [2023-12-26 15:27:12,796][105692] Updated weights for policy 0, policy_version 11967 (0.0011) [2023-12-26 15:27:13,015][105620] Updated weights for policy 1, policy_version 11905 (0.0010) [2023-12-26 15:27:13,064][105620] Updated weights for policy 1, policy_version 11915 (0.0010) [2023-12-26 15:27:13,123][105620] Updated weights for policy 1, policy_version 11925 (0.0010) [2023-12-26 15:27:13,180][105620] Updated weights for policy 1, policy_version 11935 (0.0010) [2023-12-26 15:27:13,497][105692] Updated weights for policy 0, policy_version 11977 (0.0008) [2023-12-26 15:27:13,562][105692] Updated weights for policy 0, policy_version 11987 (0.0007) [2023-12-26 15:27:13,627][105692] Updated weights for policy 0, policy_version 11997 (0.0007) [2023-12-26 15:27:13,858][105620] Updated weights for policy 1, policy_version 11945 (0.0010) [2023-12-26 15:27:13,906][105620] Updated weights for policy 1, policy_version 11955 (0.0010) [2023-12-26 15:27:13,967][105620] Updated weights for policy 1, policy_version 11965 (0.0010) [2023-12-26 15:27:14,272][105692] Updated weights for policy 0, policy_version 12007 (0.0009) [2023-12-26 15:27:14,324][105692] Updated weights for policy 0, policy_version 12017 (0.0010) [2023-12-26 15:27:14,379][105692] Updated weights for policy 0, policy_version 12027 (0.0008) [2023-12-26 15:27:14,600][105620] Updated weights for policy 1, policy_version 11975 (0.0007) [2023-12-26 15:27:14,659][105620] Updated weights for policy 1, policy_version 11985 (0.0007) [2023-12-26 15:27:14,715][105620] Updated weights for policy 1, policy_version 11995 (0.0011) [2023-12-26 15:27:15,021][105692] Updated weights for policy 0, policy_version 12037 (0.0007) [2023-12-26 15:27:15,082][105692] Updated weights for policy 0, policy_version 12047 (0.0008) [2023-12-26 15:27:15,141][105692] Updated weights for policy 0, policy_version 12057 (0.0008) [2023-12-26 15:27:15,453][105620] Updated weights for policy 1, policy_version 12005 (0.0009) [2023-12-26 15:27:15,508][105620] Updated weights for policy 1, policy_version 12015 (0.0010) [2023-12-26 15:27:15,568][105620] Updated weights for policy 1, policy_version 12025 (0.0010) [2023-12-26 15:27:15,937][105692] Updated weights for policy 0, policy_version 12067 (0.0008) [2023-12-26 15:27:15,985][105692] Updated weights for policy 0, policy_version 12077 (0.0008) [2023-12-26 15:27:16,036][105692] Updated weights for policy 0, policy_version 12087 (0.0007) [2023-12-26 15:27:16,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 6168576. Throughput: 0: 9944.1, 1: 9626.0. Samples: 6142524. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 15:27:16,063][104569] Avg episode reward: [(0, '8613.688'), (1, '8801.617')] [2023-12-26 15:27:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000012032_3080192.pth... [2023-12-26 15:27:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000010912_2793472.pth [2023-12-26 15:27:16,087][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000012096_3096576.pth... [2023-12-26 15:27:16,090][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000010912_2793472.pth [2023-12-26 15:27:16,274][105620] Updated weights for policy 1, policy_version 12035 (0.0007) [2023-12-26 15:27:16,325][105620] Updated weights for policy 1, policy_version 12045 (0.0010) [2023-12-26 15:27:16,373][105620] Updated weights for policy 1, policy_version 12055 (0.0010) [2023-12-26 15:27:16,736][105692] Updated weights for policy 0, policy_version 12097 (0.0007) [2023-12-26 15:27:16,794][105692] Updated weights for policy 0, policy_version 12107 (0.0006) [2023-12-26 15:27:16,845][105692] Updated weights for policy 0, policy_version 12117 (0.0010) [2023-12-26 15:27:16,897][105692] Updated weights for policy 0, policy_version 12127 (0.0008) [2023-12-26 15:27:17,119][105620] Updated weights for policy 1, policy_version 12065 (0.0010) [2023-12-26 15:27:17,165][105620] Updated weights for policy 1, policy_version 12075 (0.0005) [2023-12-26 15:27:17,217][105620] Updated weights for policy 1, policy_version 12085 (0.0005) [2023-12-26 15:27:17,283][105620] Updated weights for policy 1, policy_version 12095 (0.0007) [2023-12-26 15:27:17,451][105692] Updated weights for policy 0, policy_version 12137 (0.0005) [2023-12-26 15:27:17,501][105692] Updated weights for policy 0, policy_version 12147 (0.0006) [2023-12-26 15:27:17,552][105692] Updated weights for policy 0, policy_version 12157 (0.0008) [2023-12-26 15:27:17,965][105620] Updated weights for policy 1, policy_version 12105 (0.0011) [2023-12-26 15:27:18,020][105620] Updated weights for policy 1, policy_version 12115 (0.0010) [2023-12-26 15:27:18,083][105620] Updated weights for policy 1, policy_version 12125 (0.0011) [2023-12-26 15:27:18,164][105692] Updated weights for policy 0, policy_version 12167 (0.0010) [2023-12-26 15:27:18,212][105692] Updated weights for policy 0, policy_version 12177 (0.0010) [2023-12-26 15:27:18,267][105692] Updated weights for policy 0, policy_version 12187 (0.0010) [2023-12-26 15:27:18,809][105620] Updated weights for policy 1, policy_version 12135 (0.0011) [2023-12-26 15:27:18,871][105620] Updated weights for policy 1, policy_version 12145 (0.0010) [2023-12-26 15:27:18,931][105620] Updated weights for policy 1, policy_version 12155 (0.0011) [2023-12-26 15:27:18,988][105692] Updated weights for policy 0, policy_version 12197 (0.0011) [2023-12-26 15:27:19,043][105692] Updated weights for policy 0, policy_version 12207 (0.0010) [2023-12-26 15:27:19,104][105692] Updated weights for policy 0, policy_version 12217 (0.0011) [2023-12-26 15:27:19,707][105620] Updated weights for policy 1, policy_version 12165 (0.0011) [2023-12-26 15:27:19,770][105620] Updated weights for policy 1, policy_version 12175 (0.0010) [2023-12-26 15:27:19,833][105692] Updated weights for policy 0, policy_version 12227 (0.0010) [2023-12-26 15:27:19,854][105620] Updated weights for policy 1, policy_version 12185 (0.0010) [2023-12-26 15:27:19,905][105692] Updated weights for policy 0, policy_version 12237 (0.0011) [2023-12-26 15:27:19,975][105692] Updated weights for policy 0, policy_version 12247 (0.0009) [2023-12-26 15:27:20,575][105620] Updated weights for policy 1, policy_version 12195 (0.0010) [2023-12-26 15:27:20,641][105620] Updated weights for policy 1, policy_version 12205 (0.0009) [2023-12-26 15:27:20,690][105692] Updated weights for policy 0, policy_version 12257 (0.0011) [2023-12-26 15:27:20,706][105620] Updated weights for policy 1, policy_version 12215 (0.0009) [2023-12-26 15:27:20,753][105692] Updated weights for policy 0, policy_version 12267 (0.0006) [2023-12-26 15:27:20,816][105692] Updated weights for policy 0, policy_version 12277 (0.0008) [2023-12-26 15:27:20,884][105692] Updated weights for policy 0, policy_version 12287 (0.0006) [2023-12-26 15:27:21,062][104569] Fps is (10 sec: 19661.7, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 6275072. Throughput: 0: 9950.2, 1: 9667.9. Samples: 6262220. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-12-26 15:27:21,062][104569] Avg episode reward: [(0, '8705.741'), (1, '8893.101')] [2023-12-26 15:27:21,488][105620] Updated weights for policy 1, policy_version 12225 (0.0009) [2023-12-26 15:27:21,505][105692] Updated weights for policy 0, policy_version 12297 (0.0008) [2023-12-26 15:27:21,538][105620] Updated weights for policy 1, policy_version 12235 (0.0010) [2023-12-26 15:27:21,561][105692] Updated weights for policy 0, policy_version 12307 (0.0008) [2023-12-26 15:27:21,600][105620] Updated weights for policy 1, policy_version 12245 (0.0010) [2023-12-26 15:27:21,621][105692] Updated weights for policy 0, policy_version 12317 (0.0008) [2023-12-26 15:27:21,669][105620] Updated weights for policy 1, policy_version 12255 (0.0011) [2023-12-26 15:27:22,352][105692] Updated weights for policy 0, policy_version 12327 (0.0010) [2023-12-26 15:27:22,418][105692] Updated weights for policy 0, policy_version 12337 (0.0009) [2023-12-26 15:27:22,469][105620] Updated weights for policy 1, policy_version 12265 (0.0007) [2023-12-26 15:27:22,480][105692] Updated weights for policy 0, policy_version 12347 (0.0006) [2023-12-26 15:27:22,531][105620] Updated weights for policy 1, policy_version 12275 (0.0007) [2023-12-26 15:27:22,589][105620] Updated weights for policy 1, policy_version 12285 (0.0009) [2023-12-26 15:27:23,274][105620] Updated weights for policy 1, policy_version 12295 (0.0009) [2023-12-26 15:27:23,280][105692] Updated weights for policy 0, policy_version 12357 (0.0006) [2023-12-26 15:27:23,326][105620] Updated weights for policy 1, policy_version 12305 (0.0006) [2023-12-26 15:27:23,329][105692] Updated weights for policy 0, policy_version 12367 (0.0006) [2023-12-26 15:27:23,375][105692] Updated weights for policy 0, policy_version 12377 (0.0006) [2023-12-26 15:27:23,375][105620] Updated weights for policy 1, policy_version 12315 (0.0007) [2023-12-26 15:27:24,035][105620] Updated weights for policy 1, policy_version 12325 (0.0008) [2023-12-26 15:27:24,082][105620] Updated weights for policy 1, policy_version 12335 (0.0006) [2023-12-26 15:27:24,136][105692] Updated weights for policy 0, policy_version 12387 (0.0007) [2023-12-26 15:27:24,151][105620] Updated weights for policy 1, policy_version 12345 (0.0006) [2023-12-26 15:27:24,197][105692] Updated weights for policy 0, policy_version 12397 (0.0007) [2023-12-26 15:27:24,248][105692] Updated weights for policy 0, policy_version 12407 (0.0008) [2023-12-26 15:27:24,823][105620] Updated weights for policy 1, policy_version 12355 (0.0008) [2023-12-26 15:27:24,893][105620] Updated weights for policy 1, policy_version 12365 (0.0005) [2023-12-26 15:27:24,952][105620] Updated weights for policy 1, policy_version 12375 (0.0005) [2023-12-26 15:27:25,060][105692] Updated weights for policy 0, policy_version 12417 (0.0009) [2023-12-26 15:27:25,109][105692] Updated weights for policy 0, policy_version 12427 (0.0009) [2023-12-26 15:27:25,164][105692] Updated weights for policy 0, policy_version 12437 (0.0010) [2023-12-26 15:27:25,227][105692] Updated weights for policy 0, policy_version 12447 (0.0009) [2023-12-26 15:27:25,542][105620] Updated weights for policy 1, policy_version 12385 (0.0005) [2023-12-26 15:27:25,600][105620] Updated weights for policy 1, policy_version 12395 (0.0005) [2023-12-26 15:27:25,662][105620] Updated weights for policy 1, policy_version 12405 (0.0005) [2023-12-26 15:27:25,719][105620] Updated weights for policy 1, policy_version 12415 (0.0006) [2023-12-26 15:27:25,890][105692] Updated weights for policy 0, policy_version 12457 (0.0006) [2023-12-26 15:27:25,947][105692] Updated weights for policy 0, policy_version 12467 (0.0008) [2023-12-26 15:27:25,995][105692] Updated weights for policy 0, policy_version 12477 (0.0010) [2023-12-26 15:27:26,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 6373376. Throughput: 0: 9982.9, 1: 9704.9. Samples: 6379164. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-12-26 15:27:26,063][104569] Avg episode reward: [(0, '8427.182'), (1, '8430.732')] [2023-12-26 15:27:26,311][105620] Updated weights for policy 1, policy_version 12425 (0.0008) [2023-12-26 15:27:26,367][105620] Updated weights for policy 1, policy_version 12435 (0.0007) [2023-12-26 15:27:26,411][105620] Updated weights for policy 1, policy_version 12445 (0.0008) [2023-12-26 15:27:26,703][105692] Updated weights for policy 0, policy_version 12487 (0.0007) [2023-12-26 15:27:26,748][105585] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-12-26 15:27:27,277][105692] Updated weights for policy 0, policy_version 12497 (0.0005) [2023-12-26 15:27:27,298][105620] Updated weights for policy 1, policy_version 12455 (0.0009) [2023-12-26 15:27:27,336][105692] Updated weights for policy 0, policy_version 12507 (0.0007) [2023-12-26 15:27:27,350][105620] Updated weights for policy 1, policy_version 12465 (0.0007) [2023-12-26 15:27:27,394][105692] Updated weights for policy 0, policy_version 12517 (0.0010) [2023-12-26 15:27:27,404][105620] Updated weights for policy 1, policy_version 12475 (0.0006) [2023-12-26 15:27:27,455][105692] Updated weights for policy 0, policy_version 12527 (0.0010) [2023-12-26 15:27:28,088][105620] Updated weights for policy 1, policy_version 12485 (0.0007) [2023-12-26 15:27:28,146][105620] Updated weights for policy 1, policy_version 12495 (0.0009) [2023-12-26 15:27:28,166][105692] Updated weights for policy 0, policy_version 12537 (0.0010) [2023-12-26 15:27:28,204][105620] Updated weights for policy 1, policy_version 12505 (0.0007) [2023-12-26 15:27:28,226][105692] Updated weights for policy 0, policy_version 12547 (0.0011) [2023-12-26 15:27:28,282][105692] Updated weights for policy 0, policy_version 12557 (0.0011) [2023-12-26 15:27:28,955][105620] Updated weights for policy 1, policy_version 12515 (0.0007) [2023-12-26 15:27:29,011][105620] Updated weights for policy 1, policy_version 12525 (0.0008) [2023-12-26 15:27:29,047][105692] Updated weights for policy 0, policy_version 12567 (0.0010) [2023-12-26 15:27:29,061][105620] Updated weights for policy 1, policy_version 12535 (0.0005) [2023-12-26 15:27:29,101][105692] Updated weights for policy 0, policy_version 12577 (0.0010) [2023-12-26 15:27:29,151][105692] Updated weights for policy 0, policy_version 12587 (0.0010) [2023-12-26 15:27:29,800][105620] Updated weights for policy 1, policy_version 12545 (0.0009) [2023-12-26 15:27:29,865][105620] Updated weights for policy 1, policy_version 12555 (0.0007) [2023-12-26 15:27:29,914][105692] Updated weights for policy 0, policy_version 12597 (0.0008) [2023-12-26 15:27:29,928][105620] Updated weights for policy 1, policy_version 12565 (0.0006) [2023-12-26 15:27:29,982][105692] Updated weights for policy 0, policy_version 12607 (0.0007) [2023-12-26 15:27:29,989][105620] Updated weights for policy 1, policy_version 12575 (0.0006) [2023-12-26 15:27:30,044][105692] Updated weights for policy 0, policy_version 12617 (0.0009) [2023-12-26 15:27:30,683][105620] Updated weights for policy 1, policy_version 12585 (0.0008) [2023-12-26 15:27:30,745][105620] Updated weights for policy 1, policy_version 12595 (0.0009) [2023-12-26 15:27:30,747][105692] Updated weights for policy 0, policy_version 12627 (0.0007) [2023-12-26 15:27:30,792][105692] Updated weights for policy 0, policy_version 12637 (0.0006) [2023-12-26 15:27:30,797][105620] Updated weights for policy 1, policy_version 12605 (0.0007) [2023-12-26 15:27:30,839][105692] Updated weights for policy 0, policy_version 12647 (0.0009) [2023-12-26 15:27:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 6471680. Throughput: 0: 10022.8, 1: 9689.1. Samples: 6438740. Policy #0 lag: (min: 17.0, avg: 31.2, max: 33.0) [2023-12-26 15:27:31,063][104569] Avg episode reward: [(0, '8520.551'), (1, '8709.301')] [2023-12-26 15:27:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000012656_3244032.pth... [2023-12-26 15:27:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000012608_3227648.pth... [2023-12-26 15:27:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000011520_2949120.pth [2023-12-26 15:27:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000011488_2940928.pth [2023-12-26 15:27:31,542][105620] Updated weights for policy 1, policy_version 12615 (0.0009) [2023-12-26 15:27:31,587][105692] Updated weights for policy 0, policy_version 12657 (0.0009) [2023-12-26 15:27:31,594][105620] Updated weights for policy 1, policy_version 12625 (0.0009) [2023-12-26 15:27:31,651][105692] Updated weights for policy 0, policy_version 12667 (0.0007) [2023-12-26 15:27:31,653][105620] Updated weights for policy 1, policy_version 12635 (0.0008) [2023-12-26 15:27:31,712][105692] Updated weights for policy 0, policy_version 12677 (0.0009) [2023-12-26 15:27:31,772][105692] Updated weights for policy 0, policy_version 12687 (0.0009) [2023-12-26 15:27:32,434][105620] Updated weights for policy 1, policy_version 12645 (0.0008) [2023-12-26 15:27:32,492][105620] Updated weights for policy 1, policy_version 12655 (0.0010) [2023-12-26 15:27:32,523][105692] Updated weights for policy 0, policy_version 12697 (0.0006) [2023-12-26 15:27:32,551][105620] Updated weights for policy 1, policy_version 12665 (0.0010) [2023-12-26 15:27:32,581][105692] Updated weights for policy 0, policy_version 12707 (0.0007) [2023-12-26 15:27:32,642][105692] Updated weights for policy 0, policy_version 12717 (0.0008) [2023-12-26 15:27:33,133][105620] Updated weights for policy 1, policy_version 12675 (0.0009) [2023-12-26 15:27:33,185][105620] Updated weights for policy 1, policy_version 12685 (0.0006) [2023-12-26 15:27:33,249][105620] Updated weights for policy 1, policy_version 12695 (0.0005) [2023-12-26 15:27:33,503][105692] Updated weights for policy 0, policy_version 12727 (0.0010) [2023-12-26 15:27:33,557][105692] Updated weights for policy 0, policy_version 12738 (0.0010) [2023-12-26 15:27:33,609][105692] Updated weights for policy 0, policy_version 12749 (0.0009) [2023-12-26 15:27:33,803][105620] Updated weights for policy 1, policy_version 12705 (0.0006) [2023-12-26 15:27:33,882][105620] Updated weights for policy 1, policy_version 12715 (0.0010) [2023-12-26 15:27:33,957][105620] Updated weights for policy 1, policy_version 12725 (0.0010) [2023-12-26 15:27:34,010][105620] Updated weights for policy 1, policy_version 12735 (0.0007) [2023-12-26 15:27:34,279][105692] Updated weights for policy 0, policy_version 12759 (0.0008) [2023-12-26 15:27:34,340][105692] Updated weights for policy 0, policy_version 12769 (0.0007) [2023-12-26 15:27:34,394][105692] Updated weights for policy 0, policy_version 12779 (0.0007) [2023-12-26 15:27:34,754][105620] Updated weights for policy 1, policy_version 12745 (0.0005) [2023-12-26 15:27:34,817][105620] Updated weights for policy 1, policy_version 12755 (0.0006) [2023-12-26 15:27:34,876][105620] Updated weights for policy 1, policy_version 12765 (0.0005) [2023-12-26 15:27:35,055][105692] Updated weights for policy 0, policy_version 12789 (0.0006) [2023-12-26 15:27:35,108][105692] Updated weights for policy 0, policy_version 12799 (0.0006) [2023-12-26 15:27:35,159][105692] Updated weights for policy 0, policy_version 12809 (0.0005) [2023-12-26 15:27:35,545][105620] Updated weights for policy 1, policy_version 12775 (0.0006) [2023-12-26 15:27:35,599][105620] Updated weights for policy 1, policy_version 12785 (0.0007) [2023-12-26 15:27:35,658][105620] Updated weights for policy 1, policy_version 12795 (0.0010) [2023-12-26 15:27:35,758][105692] Updated weights for policy 0, policy_version 12819 (0.0005) [2023-12-26 15:27:35,812][105692] Updated weights for policy 0, policy_version 12829 (0.0006) [2023-12-26 15:27:35,861][105692] Updated weights for policy 0, policy_version 12839 (0.0006) [2023-12-26 15:27:36,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 6569984. Throughput: 0: 9889.9, 1: 9691.2. Samples: 6555132. Policy #0 lag: (min: 17.0, avg: 31.2, max: 33.0) [2023-12-26 15:27:36,062][104569] Avg episode reward: [(0, '8892.111'), (1, '8802.106')] [2023-12-26 15:27:36,311][105620] Updated weights for policy 1, policy_version 12805 (0.0010) [2023-12-26 15:27:36,372][105620] Updated weights for policy 1, policy_version 12815 (0.0009) [2023-12-26 15:27:36,433][105620] Updated weights for policy 1, policy_version 12825 (0.0008) [2023-12-26 15:27:36,572][105692] Updated weights for policy 0, policy_version 12849 (0.0006) [2023-12-26 15:27:36,631][105692] Updated weights for policy 0, policy_version 12859 (0.0007) [2023-12-26 15:27:36,691][105692] Updated weights for policy 0, policy_version 12869 (0.0007) [2023-12-26 15:27:36,739][105692] Updated weights for policy 0, policy_version 12879 (0.0005) [2023-12-26 15:27:37,166][105620] Updated weights for policy 1, policy_version 12835 (0.0009) [2023-12-26 15:27:37,214][105620] Updated weights for policy 1, policy_version 12845 (0.0009) [2023-12-26 15:27:37,269][105620] Updated weights for policy 1, policy_version 12855 (0.0008) [2023-12-26 15:27:37,362][105692] Updated weights for policy 0, policy_version 12889 (0.0008) [2023-12-26 15:27:37,417][105692] Updated weights for policy 0, policy_version 12899 (0.0009) [2023-12-26 15:27:37,471][105692] Updated weights for policy 0, policy_version 12909 (0.0008) [2023-12-26 15:27:38,040][105620] Updated weights for policy 1, policy_version 12865 (0.0009) [2023-12-26 15:27:38,086][105620] Updated weights for policy 1, policy_version 12875 (0.0008) [2023-12-26 15:27:38,137][105620] Updated weights for policy 1, policy_version 12885 (0.0008) [2023-12-26 15:27:38,192][105620] Updated weights for policy 1, policy_version 12895 (0.0009) [2023-12-26 15:27:38,230][105692] Updated weights for policy 0, policy_version 12919 (0.0009) [2023-12-26 15:27:38,277][105692] Updated weights for policy 0, policy_version 12929 (0.0009) [2023-12-26 15:27:38,326][105692] Updated weights for policy 0, policy_version 12939 (0.0008) [2023-12-26 15:27:38,930][105620] Updated weights for policy 1, policy_version 12905 (0.0008) [2023-12-26 15:27:38,993][105620] Updated weights for policy 1, policy_version 12915 (0.0009) [2023-12-26 15:27:39,052][105620] Updated weights for policy 1, policy_version 12925 (0.0008) [2023-12-26 15:27:39,125][105692] Updated weights for policy 0, policy_version 12949 (0.0007) [2023-12-26 15:27:39,187][105692] Updated weights for policy 0, policy_version 12959 (0.0005) [2023-12-26 15:27:39,256][105692] Updated weights for policy 0, policy_version 12969 (0.0008) [2023-12-26 15:27:39,887][105620] Updated weights for policy 1, policy_version 12935 (0.0007) [2023-12-26 15:27:39,893][105692] Updated weights for policy 0, policy_version 12979 (0.0007) [2023-12-26 15:27:39,957][105620] Updated weights for policy 1, policy_version 12945 (0.0007) [2023-12-26 15:27:39,961][105692] Updated weights for policy 0, policy_version 12989 (0.0008) [2023-12-26 15:27:40,015][105620] Updated weights for policy 1, policy_version 12955 (0.0007) [2023-12-26 15:27:40,021][105692] Updated weights for policy 0, policy_version 12999 (0.0008) [2023-12-26 15:27:40,617][105692] Updated weights for policy 0, policy_version 13009 (0.0006) [2023-12-26 15:27:40,666][105692] Updated weights for policy 0, policy_version 13019 (0.0008) [2023-12-26 15:27:40,717][105692] Updated weights for policy 0, policy_version 13029 (0.0009) [2023-12-26 15:27:40,770][105692] Updated weights for policy 0, policy_version 13039 (0.0009) [2023-12-26 15:27:40,824][105620] Updated weights for policy 1, policy_version 12965 (0.0008) [2023-12-26 15:27:40,883][105620] Updated weights for policy 1, policy_version 12975 (0.0009) [2023-12-26 15:27:40,953][105620] Updated weights for policy 1, policy_version 12985 (0.0009) [2023-12-26 15:27:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 6668288. Throughput: 0: 9883.2, 1: 9654.4. Samples: 6672592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-12-26 15:27:41,062][104569] Avg episode reward: [(0, '8984.369'), (1, '8431.199')] [2023-12-26 15:27:41,581][105692] Updated weights for policy 0, policy_version 13049 (0.0009) [2023-12-26 15:27:41,645][105620] Updated weights for policy 1, policy_version 12995 (0.0008) [2023-12-26 15:27:41,646][105692] Updated weights for policy 0, policy_version 13059 (0.0009) [2023-12-26 15:27:41,705][105692] Updated weights for policy 0, policy_version 13069 (0.0008) [2023-12-26 15:27:41,711][105620] Updated weights for policy 1, policy_version 13005 (0.0007) [2023-12-26 15:27:41,782][105620] Updated weights for policy 1, policy_version 13015 (0.0008) [2023-12-26 15:27:42,392][105620] Updated weights for policy 1, policy_version 13025 (0.0008) [2023-12-26 15:27:42,448][105620] Updated weights for policy 1, policy_version 13035 (0.0009) [2023-12-26 15:27:42,496][105620] Updated weights for policy 1, policy_version 13045 (0.0008) [2023-12-26 15:27:42,557][105620] Updated weights for policy 1, policy_version 13055 (0.0008) [2023-12-26 15:27:42,590][105692] Updated weights for policy 0, policy_version 13079 (0.0008) [2023-12-26 15:27:42,656][105692] Updated weights for policy 0, policy_version 13089 (0.0009) [2023-12-26 15:27:42,718][105692] Updated weights for policy 0, policy_version 13099 (0.0009) [2023-12-26 15:27:43,268][105620] Updated weights for policy 1, policy_version 13065 (0.0010) [2023-12-26 15:27:43,327][105620] Updated weights for policy 1, policy_version 13075 (0.0009) [2023-12-26 15:27:43,381][105620] Updated weights for policy 1, policy_version 13085 (0.0009) [2023-12-26 15:27:43,490][105692] Updated weights for policy 0, policy_version 13109 (0.0008) [2023-12-26 15:27:43,538][105692] Updated weights for policy 0, policy_version 13119 (0.0008) [2023-12-26 15:27:43,591][105692] Updated weights for policy 0, policy_version 13129 (0.0007) [2023-12-26 15:27:44,126][105620] Updated weights for policy 1, policy_version 13095 (0.0010) [2023-12-26 15:27:44,180][105620] Updated weights for policy 1, policy_version 13105 (0.0006) [2023-12-26 15:27:44,247][105620] Updated weights for policy 1, policy_version 13115 (0.0006) [2023-12-26 15:27:44,403][105692] Updated weights for policy 0, policy_version 13139 (0.0009) [2023-12-26 15:27:44,457][105692] Updated weights for policy 0, policy_version 13150 (0.0010) [2023-12-26 15:27:44,511][105692] Updated weights for policy 0, policy_version 13160 (0.0009) [2023-12-26 15:27:44,798][105620] Updated weights for policy 1, policy_version 13125 (0.0010) [2023-12-26 15:27:44,861][105620] Updated weights for policy 1, policy_version 13135 (0.0006) [2023-12-26 15:27:44,924][105620] Updated weights for policy 1, policy_version 13145 (0.0006) [2023-12-26 15:27:45,319][105692] Updated weights for policy 0, policy_version 13170 (0.0008) [2023-12-26 15:27:45,381][105692] Updated weights for policy 0, policy_version 13180 (0.0009) [2023-12-26 15:27:45,443][105692] Updated weights for policy 0, policy_version 13190 (0.0009) [2023-12-26 15:27:45,505][105692] Updated weights for policy 0, policy_version 13200 (0.0009) [2023-12-26 15:27:45,628][105620] Updated weights for policy 1, policy_version 13155 (0.0009) [2023-12-26 15:27:45,676][105620] Updated weights for policy 1, policy_version 13165 (0.0009) [2023-12-26 15:27:45,735][105620] Updated weights for policy 1, policy_version 13175 (0.0009) [2023-12-26 15:27:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 6758400. Throughput: 0: 9785.3, 1: 9657.6. Samples: 6728240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-12-26 15:27:46,062][104569] Avg episode reward: [(0, '8890.785'), (1, '8523.940')] [2023-12-26 15:27:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000013200_3383296.pth... [2023-12-26 15:27:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000013184_3375104.pth... [2023-12-26 15:27:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000012096_3096576.pth [2023-12-26 15:27:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000012032_3080192.pth [2023-12-26 15:27:46,239][105692] Updated weights for policy 0, policy_version 13210 (0.0009) [2023-12-26 15:27:46,295][105692] Updated weights for policy 0, policy_version 13220 (0.0009) [2023-12-26 15:27:46,353][105692] Updated weights for policy 0, policy_version 13230 (0.0009) [2023-12-26 15:27:46,442][105620] Updated weights for policy 1, policy_version 13185 (0.0009) [2023-12-26 15:27:46,493][105620] Updated weights for policy 1, policy_version 13195 (0.0008) [2023-12-26 15:27:46,545][105620] Updated weights for policy 1, policy_version 13205 (0.0009) [2023-12-26 15:27:46,592][105620] Updated weights for policy 1, policy_version 13215 (0.0009) [2023-12-26 15:27:47,142][105692] Updated weights for policy 0, policy_version 13240 (0.0006) [2023-12-26 15:27:47,197][105692] Updated weights for policy 0, policy_version 13250 (0.0005) [2023-12-26 15:27:47,240][105692] Updated weights for policy 0, policy_version 13260 (0.0005) [2023-12-26 15:27:47,350][105620] Updated weights for policy 1, policy_version 13225 (0.0009) [2023-12-26 15:27:47,405][105620] Updated weights for policy 1, policy_version 13235 (0.0009) [2023-12-26 15:27:47,462][105620] Updated weights for policy 1, policy_version 13245 (0.0008) [2023-12-26 15:27:47,848][105692] Updated weights for policy 0, policy_version 13270 (0.0005) [2023-12-26 15:27:47,900][105692] Updated weights for policy 0, policy_version 13280 (0.0005) [2023-12-26 15:27:47,947][105692] Updated weights for policy 0, policy_version 13290 (0.0005) [2023-12-26 15:27:48,357][105620] Updated weights for policy 1, policy_version 13255 (0.0009) [2023-12-26 15:27:48,410][105620] Updated weights for policy 1, policy_version 13265 (0.0008) [2023-12-26 15:27:48,471][105620] Updated weights for policy 1, policy_version 13275 (0.0009) [2023-12-26 15:27:48,494][105692] Updated weights for policy 0, policy_version 13300 (0.0007) [2023-12-26 15:27:48,556][105692] Updated weights for policy 0, policy_version 13310 (0.0009) [2023-12-26 15:27:48,627][105692] Updated weights for policy 0, policy_version 13320 (0.0010) [2023-12-26 15:27:49,175][105620] Updated weights for policy 1, policy_version 13285 (0.0006) [2023-12-26 15:27:49,242][105620] Updated weights for policy 1, policy_version 13295 (0.0008) [2023-12-26 15:27:49,299][105620] Updated weights for policy 1, policy_version 13305 (0.0010) [2023-12-26 15:27:49,359][105692] Updated weights for policy 0, policy_version 13330 (0.0009) [2023-12-26 15:27:49,420][105692] Updated weights for policy 0, policy_version 13340 (0.0007) [2023-12-26 15:27:49,479][105692] Updated weights for policy 0, policy_version 13350 (0.0006) [2023-12-26 15:27:49,540][105692] Updated weights for policy 0, policy_version 13360 (0.0008) [2023-12-26 15:27:50,039][105620] Updated weights for policy 1, policy_version 13315 (0.0008) [2023-12-26 15:27:50,101][105620] Updated weights for policy 1, policy_version 13325 (0.0009) [2023-12-26 15:27:50,164][105620] Updated weights for policy 1, policy_version 13335 (0.0008) [2023-12-26 15:27:50,274][105692] Updated weights for policy 0, policy_version 13370 (0.0009) [2023-12-26 15:27:50,329][105692] Updated weights for policy 0, policy_version 13380 (0.0009) [2023-12-26 15:27:50,389][105692] Updated weights for policy 0, policy_version 13390 (0.0009) [2023-12-26 15:27:50,905][105620] Updated weights for policy 1, policy_version 13345 (0.0009) [2023-12-26 15:27:50,964][105620] Updated weights for policy 1, policy_version 13355 (0.0006) [2023-12-26 15:27:51,028][105620] Updated weights for policy 1, policy_version 13365 (0.0007) [2023-12-26 15:27:51,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 6848512. Throughput: 0: 9759.9, 1: 9693.3. Samples: 6844612. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-26 15:27:51,063][104569] Avg episode reward: [(0, '9169.396'), (1, '8524.436')] [2023-12-26 15:27:51,063][105585] Saving new best policy, reward=9169.396! [2023-12-26 15:27:51,093][105620] Updated weights for policy 1, policy_version 13375 (0.0010) [2023-12-26 15:27:51,164][105692] Updated weights for policy 0, policy_version 13400 (0.0008) [2023-12-26 15:27:51,216][105692] Updated weights for policy 0, policy_version 13410 (0.0008) [2023-12-26 15:27:51,273][105692] Updated weights for policy 0, policy_version 13420 (0.0007) [2023-12-26 15:27:51,842][105620] Updated weights for policy 1, policy_version 13385 (0.0008) [2023-12-26 15:27:51,906][105620] Updated weights for policy 1, policy_version 13395 (0.0008) [2023-12-26 15:27:51,965][105620] Updated weights for policy 1, policy_version 13405 (0.0008) [2023-12-26 15:27:51,988][105692] Updated weights for policy 0, policy_version 13430 (0.0006) [2023-12-26 15:27:52,042][105692] Updated weights for policy 0, policy_version 13440 (0.0005) [2023-12-26 15:27:52,092][105692] Updated weights for policy 0, policy_version 13450 (0.0008) [2023-12-26 15:27:52,719][105620] Updated weights for policy 1, policy_version 13415 (0.0010) [2023-12-26 15:27:52,760][105692] Updated weights for policy 0, policy_version 13460 (0.0009) [2023-12-26 15:27:52,784][105620] Updated weights for policy 1, policy_version 13425 (0.0011) [2023-12-26 15:27:52,819][105692] Updated weights for policy 0, policy_version 13470 (0.0009) [2023-12-26 15:27:52,843][105620] Updated weights for policy 1, policy_version 13435 (0.0010) [2023-12-26 15:27:52,880][105692] Updated weights for policy 0, policy_version 13480 (0.0008) [2023-12-26 15:27:53,547][105620] Updated weights for policy 1, policy_version 13445 (0.0008) [2023-12-26 15:27:53,596][105620] Updated weights for policy 1, policy_version 13455 (0.0007) [2023-12-26 15:27:53,610][105692] Updated weights for policy 0, policy_version 13490 (0.0008) [2023-12-26 15:27:53,657][105620] Updated weights for policy 1, policy_version 13465 (0.0007) [2023-12-26 15:27:53,658][105692] Updated weights for policy 0, policy_version 13500 (0.0008) [2023-12-26 15:27:53,712][105692] Updated weights for policy 0, policy_version 13510 (0.0008) [2023-12-26 15:27:53,763][105692] Updated weights for policy 0, policy_version 13520 (0.0009) [2023-12-26 15:27:54,349][105620] Updated weights for policy 1, policy_version 13475 (0.0007) [2023-12-26 15:27:54,396][105620] Updated weights for policy 1, policy_version 13485 (0.0008) [2023-12-26 15:27:54,414][105692] Updated weights for policy 0, policy_version 13530 (0.0005) [2023-12-26 15:27:54,448][105620] Updated weights for policy 1, policy_version 13495 (0.0008) [2023-12-26 15:27:54,457][105692] Updated weights for policy 0, policy_version 13540 (0.0005) [2023-12-26 15:27:54,504][105692] Updated weights for policy 0, policy_version 13550 (0.0005) [2023-12-26 15:27:55,218][105620] Updated weights for policy 1, policy_version 13505 (0.0009) [2023-12-26 15:27:55,237][105692] Updated weights for policy 0, policy_version 13560 (0.0007) [2023-12-26 15:27:55,272][105620] Updated weights for policy 1, policy_version 13515 (0.0007) [2023-12-26 15:27:55,279][105692] Updated weights for policy 0, policy_version 13570 (0.0007) [2023-12-26 15:27:55,330][105692] Updated weights for policy 0, policy_version 13580 (0.0005) [2023-12-26 15:27:55,332][105620] Updated weights for policy 1, policy_version 13525 (0.0008) [2023-12-26 15:27:55,390][105620] Updated weights for policy 1, policy_version 13535 (0.0008) [2023-12-26 15:27:56,048][105620] Updated weights for policy 1, policy_version 13545 (0.0006) [2023-12-26 15:27:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 6946816. Throughput: 0: 9775.0, 1: 9685.1. Samples: 6960980. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-26 15:27:56,062][104569] Avg episode reward: [(0, '9076.503'), (1, '8433.195')] [2023-12-26 15:27:56,098][105692] Updated weights for policy 0, policy_version 13590 (0.0006) [2023-12-26 15:27:56,107][105620] Updated weights for policy 1, policy_version 13555 (0.0009) [2023-12-26 15:27:56,150][105692] Updated weights for policy 0, policy_version 13600 (0.0006) [2023-12-26 15:27:56,156][105620] Updated weights for policy 1, policy_version 13565 (0.0010) [2023-12-26 15:27:56,211][105692] Updated weights for policy 0, policy_version 13610 (0.0007) [2023-12-26 15:27:56,782][105692] Updated weights for policy 0, policy_version 13620 (0.0008) [2023-12-26 15:27:56,832][105692] Updated weights for policy 0, policy_version 13630 (0.0008) [2023-12-26 15:27:56,855][105620] Updated weights for policy 1, policy_version 13575 (0.0010) [2023-12-26 15:27:56,878][105692] Updated weights for policy 0, policy_version 13640 (0.0009) [2023-12-26 15:27:56,910][105620] Updated weights for policy 1, policy_version 13585 (0.0010) [2023-12-26 15:27:56,972][105620] Updated weights for policy 1, policy_version 13595 (0.0010) [2023-12-26 15:27:57,533][105692] Updated weights for policy 0, policy_version 13650 (0.0005) [2023-12-26 15:27:57,582][105692] Updated weights for policy 0, policy_version 13660 (0.0005) [2023-12-26 15:27:57,640][105692] Updated weights for policy 0, policy_version 13670 (0.0005) [2023-12-26 15:27:57,643][105620] Updated weights for policy 1, policy_version 13605 (0.0010) [2023-12-26 15:27:57,696][105620] Updated weights for policy 1, policy_version 13615 (0.0005) [2023-12-26 15:27:57,700][105692] Updated weights for policy 0, policy_version 13680 (0.0005) [2023-12-26 15:27:57,743][105620] Updated weights for policy 1, policy_version 13625 (0.0006) [2023-12-26 15:27:58,352][105692] Updated weights for policy 0, policy_version 13690 (0.0008) [2023-12-26 15:27:58,388][105620] Updated weights for policy 1, policy_version 13635 (0.0006) [2023-12-26 15:27:58,415][105692] Updated weights for policy 0, policy_version 13700 (0.0007) [2023-12-26 15:27:58,452][105620] Updated weights for policy 1, policy_version 13645 (0.0007) [2023-12-26 15:27:58,477][105692] Updated weights for policy 0, policy_version 13710 (0.0008) [2023-12-26 15:27:58,518][105620] Updated weights for policy 1, policy_version 13655 (0.0009) [2023-12-26 15:27:59,122][105692] Updated weights for policy 0, policy_version 13720 (0.0008) [2023-12-26 15:27:59,172][105692] Updated weights for policy 0, policy_version 13730 (0.0008) [2023-12-26 15:27:59,231][105692] Updated weights for policy 0, policy_version 13740 (0.0009) [2023-12-26 15:27:59,269][105620] Updated weights for policy 1, policy_version 13665 (0.0011) [2023-12-26 15:27:59,324][105620] Updated weights for policy 1, policy_version 13675 (0.0009) [2023-12-26 15:27:59,392][105620] Updated weights for policy 1, policy_version 13685 (0.0008) [2023-12-26 15:27:59,452][105620] Updated weights for policy 1, policy_version 13695 (0.0008) [2023-12-26 15:27:59,987][105692] Updated weights for policy 0, policy_version 13750 (0.0007) [2023-12-26 15:28:00,041][105692] Updated weights for policy 0, policy_version 13760 (0.0009) [2023-12-26 15:28:00,094][105692] Updated weights for policy 0, policy_version 13770 (0.0008) [2023-12-26 15:28:00,186][105620] Updated weights for policy 1, policy_version 13705 (0.0009) [2023-12-26 15:28:00,250][105620] Updated weights for policy 1, policy_version 13715 (0.0009) [2023-12-26 15:28:00,302][105620] Updated weights for policy 1, policy_version 13725 (0.0008) [2023-12-26 15:28:00,878][105692] Updated weights for policy 0, policy_version 13780 (0.0009) [2023-12-26 15:28:00,924][105692] Updated weights for policy 0, policy_version 13790 (0.0009) [2023-12-26 15:28:00,968][105692] Updated weights for policy 0, policy_version 13800 (0.0007) [2023-12-26 15:28:00,989][105620] Updated weights for policy 1, policy_version 13735 (0.0009) [2023-12-26 15:28:01,046][105620] Updated weights for policy 1, policy_version 13745 (0.0008) [2023-12-26 15:28:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 7053312. Throughput: 0: 9839.6, 1: 9723.3. Samples: 7022856. Policy #0 lag: (min: 2.0, avg: 13.1, max: 34.0) [2023-12-26 15:28:01,063][104569] Avg episode reward: [(0, '8890.838'), (1, '8248.469')] [2023-12-26 15:28:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000013808_3538944.pth... [2023-12-26 15:28:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000012656_3244032.pth [2023-12-26 15:28:01,104][105620] Updated weights for policy 1, policy_version 13755 (0.0008) [2023-12-26 15:28:01,138][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000013760_3522560.pth... [2023-12-26 15:28:01,142][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000012608_3227648.pth [2023-12-26 15:28:01,806][105692] Updated weights for policy 0, policy_version 13810 (0.0009) [2023-12-26 15:28:01,870][105692] Updated weights for policy 0, policy_version 13820 (0.0009) [2023-12-26 15:28:01,909][105620] Updated weights for policy 1, policy_version 13765 (0.0009) [2023-12-26 15:28:01,921][105692] Updated weights for policy 0, policy_version 13830 (0.0006) [2023-12-26 15:28:01,971][105620] Updated weights for policy 1, policy_version 13775 (0.0009) [2023-12-26 15:28:01,976][105692] Updated weights for policy 0, policy_version 13840 (0.0006) [2023-12-26 15:28:02,024][105620] Updated weights for policy 1, policy_version 13785 (0.0009) [2023-12-26 15:28:02,749][105620] Updated weights for policy 1, policy_version 13795 (0.0008) [2023-12-26 15:28:02,752][105692] Updated weights for policy 0, policy_version 13850 (0.0009) [2023-12-26 15:28:02,800][105692] Updated weights for policy 0, policy_version 13860 (0.0008) [2023-12-26 15:28:02,807][105620] Updated weights for policy 1, policy_version 13805 (0.0005) [2023-12-26 15:28:02,863][105692] Updated weights for policy 0, policy_version 13870 (0.0008) [2023-12-26 15:28:02,863][105620] Updated weights for policy 1, policy_version 13815 (0.0006) [2023-12-26 15:28:03,443][105620] Updated weights for policy 1, policy_version 13825 (0.0006) [2023-12-26 15:28:03,506][105620] Updated weights for policy 1, policy_version 13835 (0.0005) [2023-12-26 15:28:03,541][105692] Updated weights for policy 0, policy_version 13880 (0.0006) [2023-12-26 15:28:03,571][105620] Updated weights for policy 1, policy_version 13845 (0.0005) [2023-12-26 15:28:03,606][105692] Updated weights for policy 0, policy_version 13890 (0.0005) [2023-12-26 15:28:03,638][105620] Updated weights for policy 1, policy_version 13855 (0.0005) [2023-12-26 15:28:03,672][105692] Updated weights for policy 0, policy_version 13900 (0.0005) [2023-12-26 15:28:04,157][105620] Updated weights for policy 1, policy_version 13865 (0.0007) [2023-12-26 15:28:04,193][105692] Updated weights for policy 0, policy_version 13910 (0.0006) [2023-12-26 15:28:04,212][105620] Updated weights for policy 1, policy_version 13875 (0.0008) [2023-12-26 15:28:04,247][105692] Updated weights for policy 0, policy_version 13920 (0.0007) [2023-12-26 15:28:04,269][105620] Updated weights for policy 1, policy_version 13885 (0.0007) [2023-12-26 15:28:04,304][105692] Updated weights for policy 0, policy_version 13930 (0.0007) [2023-12-26 15:28:04,992][105692] Updated weights for policy 0, policy_version 13940 (0.0009) [2023-12-26 15:28:05,031][105620] Updated weights for policy 1, policy_version 13895 (0.0006) [2023-12-26 15:28:05,047][105692] Updated weights for policy 0, policy_version 13950 (0.0008) [2023-12-26 15:28:05,095][105620] Updated weights for policy 1, policy_version 13905 (0.0005) [2023-12-26 15:28:05,103][105692] Updated weights for policy 0, policy_version 13960 (0.0009) [2023-12-26 15:28:05,151][105620] Updated weights for policy 1, policy_version 13915 (0.0005) [2023-12-26 15:28:05,781][105620] Updated weights for policy 1, policy_version 13925 (0.0007) [2023-12-26 15:28:05,828][105620] Updated weights for policy 1, policy_version 13935 (0.0009) [2023-12-26 15:28:05,878][105620] Updated weights for policy 1, policy_version 13945 (0.0008) [2023-12-26 15:28:05,887][105692] Updated weights for policy 0, policy_version 13970 (0.0008) [2023-12-26 15:28:05,944][105692] Updated weights for policy 0, policy_version 13980 (0.0005) [2023-12-26 15:28:05,993][105692] Updated weights for policy 0, policy_version 13990 (0.0006) [2023-12-26 15:28:06,047][105692] Updated weights for policy 0, policy_version 14000 (0.0008) [2023-12-26 15:28:06,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 7159808. Throughput: 0: 9771.7, 1: 9769.0. Samples: 7141552. Policy #0 lag: (min: 2.0, avg: 13.1, max: 34.0) [2023-12-26 15:28:06,062][104569] Avg episode reward: [(0, '8983.675'), (1, '8339.437')] [2023-12-26 15:28:06,693][105620] Updated weights for policy 1, policy_version 13955 (0.0010) [2023-12-26 15:28:06,745][105692] Updated weights for policy 0, policy_version 14010 (0.0006) [2023-12-26 15:28:06,755][105620] Updated weights for policy 1, policy_version 13965 (0.0008) [2023-12-26 15:28:06,807][105692] Updated weights for policy 0, policy_version 14020 (0.0006) [2023-12-26 15:28:06,821][105620] Updated weights for policy 1, policy_version 13975 (0.0009) [2023-12-26 15:28:06,875][105692] Updated weights for policy 0, policy_version 14030 (0.0006) [2023-12-26 15:28:07,514][105620] Updated weights for policy 1, policy_version 13985 (0.0008) [2023-12-26 15:28:07,576][105620] Updated weights for policy 1, policy_version 13995 (0.0009) [2023-12-26 15:28:07,616][105692] Updated weights for policy 0, policy_version 14040 (0.0006) [2023-12-26 15:28:07,634][105620] Updated weights for policy 1, policy_version 14005 (0.0007) [2023-12-26 15:28:07,679][105692] Updated weights for policy 0, policy_version 14050 (0.0006) [2023-12-26 15:28:07,683][105620] Updated weights for policy 1, policy_version 14015 (0.0008) [2023-12-26 15:28:07,747][105692] Updated weights for policy 0, policy_version 14060 (0.0005) [2023-12-26 15:28:08,444][105692] Updated weights for policy 0, policy_version 14070 (0.0007) [2023-12-26 15:28:08,458][105620] Updated weights for policy 1, policy_version 14025 (0.0008) [2023-12-26 15:28:08,508][105692] Updated weights for policy 0, policy_version 14080 (0.0006) [2023-12-26 15:28:08,518][105620] Updated weights for policy 1, policy_version 14035 (0.0007) [2023-12-26 15:28:08,565][105692] Updated weights for policy 0, policy_version 14090 (0.0006) [2023-12-26 15:28:08,574][105620] Updated weights for policy 1, policy_version 14045 (0.0007) [2023-12-26 15:28:09,333][105620] Updated weights for policy 1, policy_version 14055 (0.0008) [2023-12-26 15:28:09,357][105692] Updated weights for policy 0, policy_version 14100 (0.0008) [2023-12-26 15:28:09,402][105620] Updated weights for policy 1, policy_version 14065 (0.0007) [2023-12-26 15:28:09,420][105692] Updated weights for policy 0, policy_version 14110 (0.0008) [2023-12-26 15:28:09,463][105620] Updated weights for policy 1, policy_version 14075 (0.0007) [2023-12-26 15:28:09,474][105692] Updated weights for policy 0, policy_version 14120 (0.0007) [2023-12-26 15:28:10,101][105620] Updated weights for policy 1, policy_version 14085 (0.0008) [2023-12-26 15:28:10,161][105620] Updated weights for policy 1, policy_version 14095 (0.0008) [2023-12-26 15:28:10,224][105620] Updated weights for policy 1, policy_version 14105 (0.0009) [2023-12-26 15:28:10,315][105692] Updated weights for policy 0, policy_version 14130 (0.0009) [2023-12-26 15:28:10,374][105692] Updated weights for policy 0, policy_version 14140 (0.0008) [2023-12-26 15:28:10,431][105692] Updated weights for policy 0, policy_version 14150 (0.0006) [2023-12-26 15:28:10,485][105692] Updated weights for policy 0, policy_version 14160 (0.0009) [2023-12-26 15:28:11,027][105620] Updated weights for policy 1, policy_version 14115 (0.0009) [2023-12-26 15:28:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.9, 300 sec: 19549.7). Total num frames: 7241728. Throughput: 0: 9769.6, 1: 9701.4. Samples: 7255352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:28:11,062][104569] Avg episode reward: [(0, '9076.856'), (1, '8524.724')] [2023-12-26 15:28:11,088][105620] Updated weights for policy 1, policy_version 14125 (0.0006) [2023-12-26 15:28:11,135][105692] Updated weights for policy 0, policy_version 14170 (0.0009) [2023-12-26 15:28:11,157][105620] Updated weights for policy 1, policy_version 14135 (0.0006) [2023-12-26 15:28:11,189][105692] Updated weights for policy 0, policy_version 14180 (0.0009) [2023-12-26 15:28:11,245][105692] Updated weights for policy 0, policy_version 14190 (0.0009) [2023-12-26 15:28:11,803][105620] Updated weights for policy 1, policy_version 14145 (0.0007) [2023-12-26 15:28:11,862][105620] Updated weights for policy 1, policy_version 14155 (0.0006) [2023-12-26 15:28:11,923][105620] Updated weights for policy 1, policy_version 14165 (0.0005) [2023-12-26 15:28:11,984][105620] Updated weights for policy 1, policy_version 14175 (0.0006) [2023-12-26 15:28:12,021][105692] Updated weights for policy 0, policy_version 14200 (0.0009) [2023-12-26 15:28:12,092][105692] Updated weights for policy 0, policy_version 14210 (0.0010) [2023-12-26 15:28:12,143][105692] Updated weights for policy 0, policy_version 14220 (0.0009) [2023-12-26 15:28:12,652][105620] Updated weights for policy 1, policy_version 14185 (0.0008) [2023-12-26 15:28:12,716][105620] Updated weights for policy 1, policy_version 14195 (0.0009) [2023-12-26 15:28:12,776][105620] Updated weights for policy 1, policy_version 14205 (0.0009) [2023-12-26 15:28:12,922][105692] Updated weights for policy 0, policy_version 14230 (0.0010) [2023-12-26 15:28:12,977][105692] Updated weights for policy 0, policy_version 14240 (0.0008) [2023-12-26 15:28:13,040][105692] Updated weights for policy 0, policy_version 14250 (0.0009) [2023-12-26 15:28:13,522][105620] Updated weights for policy 1, policy_version 14215 (0.0010) [2023-12-26 15:28:13,591][105620] Updated weights for policy 1, policy_version 14225 (0.0010) [2023-12-26 15:28:13,660][105620] Updated weights for policy 1, policy_version 14235 (0.0010) [2023-12-26 15:28:13,699][105692] Updated weights for policy 0, policy_version 14260 (0.0008) [2023-12-26 15:28:13,760][105692] Updated weights for policy 0, policy_version 14270 (0.0009) [2023-12-26 15:28:13,822][105692] Updated weights for policy 0, policy_version 14280 (0.0009) [2023-12-26 15:28:14,285][105620] Updated weights for policy 1, policy_version 14245 (0.0010) [2023-12-26 15:28:14,343][105620] Updated weights for policy 1, policy_version 14255 (0.0010) [2023-12-26 15:28:14,402][105620] Updated weights for policy 1, policy_version 14265 (0.0010) [2023-12-26 15:28:14,695][105692] Updated weights for policy 0, policy_version 14290 (0.0009) [2023-12-26 15:28:14,752][105692] Updated weights for policy 0, policy_version 14300 (0.0010) [2023-12-26 15:28:14,815][105692] Updated weights for policy 0, policy_version 14310 (0.0009) [2023-12-26 15:28:14,872][105692] Updated weights for policy 0, policy_version 14320 (0.0010) [2023-12-26 15:28:14,952][105620] Updated weights for policy 1, policy_version 14275 (0.0008) [2023-12-26 15:28:15,001][105620] Updated weights for policy 1, policy_version 14285 (0.0006) [2023-12-26 15:28:15,056][105620] Updated weights for policy 1, policy_version 14295 (0.0007) [2023-12-26 15:28:15,729][105692] Updated weights for policy 0, policy_version 14330 (0.0009) [2023-12-26 15:28:15,744][105620] Updated weights for policy 1, policy_version 14305 (0.0007) [2023-12-26 15:28:15,790][105692] Updated weights for policy 0, policy_version 14340 (0.0006) [2023-12-26 15:28:15,796][105620] Updated weights for policy 1, policy_version 14315 (0.0010) [2023-12-26 15:28:15,852][105692] Updated weights for policy 0, policy_version 14350 (0.0005) [2023-12-26 15:28:15,858][105620] Updated weights for policy 1, policy_version 14325 (0.0010) [2023-12-26 15:28:15,916][105620] Updated weights for policy 1, policy_version 14335 (0.0010) [2023-12-26 15:28:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 7348224. Throughput: 0: 9706.9, 1: 9733.8. Samples: 7313572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:28:16,063][104569] Avg episode reward: [(0, '9077.839'), (1, '8894.614')] [2023-12-26 15:28:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000014352_3678208.pth... [2023-12-26 15:28:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000014336_3670016.pth... [2023-12-26 15:28:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000013200_3383296.pth [2023-12-26 15:28:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000013184_3375104.pth [2023-12-26 15:28:16,608][105620] Updated weights for policy 1, policy_version 14345 (0.0010) [2023-12-26 15:28:16,634][105692] Updated weights for policy 0, policy_version 14360 (0.0007) [2023-12-26 15:28:16,666][105620] Updated weights for policy 1, policy_version 14355 (0.0010) [2023-12-26 15:28:16,684][105692] Updated weights for policy 0, policy_version 14370 (0.0007) [2023-12-26 15:28:16,717][105620] Updated weights for policy 1, policy_version 14365 (0.0010) [2023-12-26 15:28:16,744][105692] Updated weights for policy 0, policy_version 14380 (0.0006) [2023-12-26 15:28:17,395][105692] Updated weights for policy 0, policy_version 14390 (0.0007) [2023-12-26 15:28:17,458][105692] Updated weights for policy 0, policy_version 14400 (0.0006) [2023-12-26 15:28:17,474][105620] Updated weights for policy 1, policy_version 14375 (0.0010) [2023-12-26 15:28:17,510][105692] Updated weights for policy 0, policy_version 14410 (0.0010) [2023-12-26 15:28:17,532][105620] Updated weights for policy 1, policy_version 14385 (0.0010) [2023-12-26 15:28:17,586][105620] Updated weights for policy 1, policy_version 14395 (0.0010) [2023-12-26 15:28:18,206][105692] Updated weights for policy 0, policy_version 14420 (0.0010) [2023-12-26 15:28:18,259][105692] Updated weights for policy 0, policy_version 14430 (0.0011) [2023-12-26 15:28:18,261][105620] Updated weights for policy 1, policy_version 14405 (0.0010) [2023-12-26 15:28:18,311][105692] Updated weights for policy 0, policy_version 14440 (0.0010) [2023-12-26 15:28:18,313][105620] Updated weights for policy 1, policy_version 14415 (0.0010) [2023-12-26 15:28:18,387][105620] Updated weights for policy 1, policy_version 14425 (0.0009) [2023-12-26 15:28:19,061][105692] Updated weights for policy 0, policy_version 14450 (0.0011) [2023-12-26 15:28:19,116][105692] Updated weights for policy 0, policy_version 14460 (0.0010) [2023-12-26 15:28:19,117][105620] Updated weights for policy 1, policy_version 14435 (0.0009) [2023-12-26 15:28:19,166][105692] Updated weights for policy 0, policy_version 14470 (0.0010) [2023-12-26 15:28:19,172][105620] Updated weights for policy 1, policy_version 14445 (0.0010) [2023-12-26 15:28:19,222][105692] Updated weights for policy 0, policy_version 14480 (0.0011) [2023-12-26 15:28:19,227][105620] Updated weights for policy 1, policy_version 14455 (0.0010) [2023-12-26 15:28:19,929][105620] Updated weights for policy 1, policy_version 14465 (0.0010) [2023-12-26 15:28:19,992][105620] Updated weights for policy 1, policy_version 14475 (0.0008) [2023-12-26 15:28:19,997][105692] Updated weights for policy 0, policy_version 14490 (0.0005) [2023-12-26 15:28:20,055][105620] Updated weights for policy 1, policy_version 14485 (0.0008) [2023-12-26 15:28:20,058][105692] Updated weights for policy 0, policy_version 14500 (0.0006) [2023-12-26 15:28:20,107][105620] Updated weights for policy 1, policy_version 14495 (0.0008) [2023-12-26 15:28:20,117][105692] Updated weights for policy 0, policy_version 14510 (0.0007) [2023-12-26 15:28:20,763][105692] Updated weights for policy 0, policy_version 14520 (0.0011) [2023-12-26 15:28:20,828][105692] Updated weights for policy 0, policy_version 14530 (0.0011) [2023-12-26 15:28:20,893][105692] Updated weights for policy 0, policy_version 14540 (0.0009) [2023-12-26 15:28:20,902][105620] Updated weights for policy 1, policy_version 14505 (0.0010) [2023-12-26 15:28:20,965][105620] Updated weights for policy 1, policy_version 14515 (0.0011) [2023-12-26 15:28:21,028][105620] Updated weights for policy 1, policy_version 14525 (0.0010) [2023-12-26 15:28:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 7446528. Throughput: 0: 9681.9, 1: 9775.2. Samples: 7430700. Policy #0 lag: (min: 27.0, avg: 54.7, max: 56.0) [2023-12-26 15:28:21,062][104569] Avg episode reward: [(0, '8799.426'), (1, '8525.407')] [2023-12-26 15:28:21,603][105692] Updated weights for policy 0, policy_version 14550 (0.0011) [2023-12-26 15:28:21,671][105692] Updated weights for policy 0, policy_version 14560 (0.0009) [2023-12-26 15:28:21,739][105692] Updated weights for policy 0, policy_version 14570 (0.0008) [2023-12-26 15:28:21,792][105620] Updated weights for policy 1, policy_version 14535 (0.0010) [2023-12-26 15:28:21,842][105620] Updated weights for policy 1, policy_version 14545 (0.0011) [2023-12-26 15:28:21,898][105620] Updated weights for policy 1, policy_version 14555 (0.0011) [2023-12-26 15:28:22,361][105692] Updated weights for policy 0, policy_version 14580 (0.0008) [2023-12-26 15:28:22,416][105692] Updated weights for policy 0, policy_version 14590 (0.0009) [2023-12-26 15:28:22,472][105692] Updated weights for policy 0, policy_version 14600 (0.0008) [2023-12-26 15:28:22,677][105620] Updated weights for policy 1, policy_version 14565 (0.0011) [2023-12-26 15:28:22,733][105620] Updated weights for policy 1, policy_version 14575 (0.0011) [2023-12-26 15:28:22,803][105620] Updated weights for policy 1, policy_version 14585 (0.0011) [2023-12-26 15:28:23,119][105692] Updated weights for policy 0, policy_version 14610 (0.0009) [2023-12-26 15:28:23,178][105692] Updated weights for policy 0, policy_version 14620 (0.0009) [2023-12-26 15:28:23,232][105692] Updated weights for policy 0, policy_version 14630 (0.0005) [2023-12-26 15:28:23,289][105692] Updated weights for policy 0, policy_version 14640 (0.0005) [2023-12-26 15:28:23,468][105620] Updated weights for policy 1, policy_version 14595 (0.0011) [2023-12-26 15:28:23,521][105620] Updated weights for policy 1, policy_version 14605 (0.0011) [2023-12-26 15:28:23,570][105620] Updated weights for policy 1, policy_version 14615 (0.0011) [2023-12-26 15:28:23,880][105692] Updated weights for policy 0, policy_version 14650 (0.0006) [2023-12-26 15:28:23,936][105692] Updated weights for policy 0, policy_version 14660 (0.0009) [2023-12-26 15:28:23,997][105692] Updated weights for policy 0, policy_version 14670 (0.0005) [2023-12-26 15:28:24,304][105620] Updated weights for policy 1, policy_version 14625 (0.0011) [2023-12-26 15:28:24,359][105620] Updated weights for policy 1, policy_version 14635 (0.0010) [2023-12-26 15:28:24,407][105620] Updated weights for policy 1, policy_version 14645 (0.0010) [2023-12-26 15:28:24,460][105620] Updated weights for policy 1, policy_version 14655 (0.0010) [2023-12-26 15:28:24,633][105692] Updated weights for policy 0, policy_version 14680 (0.0010) [2023-12-26 15:28:24,697][105692] Updated weights for policy 0, policy_version 14690 (0.0010) [2023-12-26 15:28:24,762][105692] Updated weights for policy 0, policy_version 14700 (0.0010) [2023-12-26 15:28:25,234][105620] Updated weights for policy 1, policy_version 14665 (0.0007) [2023-12-26 15:28:25,299][105620] Updated weights for policy 1, policy_version 14675 (0.0005) [2023-12-26 15:28:25,368][105620] Updated weights for policy 1, policy_version 14685 (0.0005) [2023-12-26 15:28:25,459][105692] Updated weights for policy 0, policy_version 14710 (0.0007) [2023-12-26 15:28:25,521][105692] Updated weights for policy 0, policy_version 14720 (0.0005) [2023-12-26 15:28:25,577][105692] Updated weights for policy 0, policy_version 14730 (0.0005) [2023-12-26 15:28:25,962][105620] Updated weights for policy 1, policy_version 14695 (0.0008) [2023-12-26 15:28:26,013][105620] Updated weights for policy 1, policy_version 14706 (0.0010) [2023-12-26 15:28:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 7536640. Throughput: 0: 9693.0, 1: 9810.5. Samples: 7550248. Policy #0 lag: (min: 27.0, avg: 54.7, max: 56.0) [2023-12-26 15:28:26,063][104569] Avg episode reward: [(0, '8707.041'), (1, '8063.402')] [2023-12-26 15:28:26,063][105620] Updated weights for policy 1, policy_version 14716 (0.0007) [2023-12-26 15:28:26,126][105692] Updated weights for policy 0, policy_version 14740 (0.0007) [2023-12-26 15:28:26,180][105692] Updated weights for policy 0, policy_version 14750 (0.0010) [2023-12-26 15:28:26,231][105692] Updated weights for policy 0, policy_version 14760 (0.0010) [2023-12-26 15:28:26,806][105620] Updated weights for policy 1, policy_version 14726 (0.0009) [2023-12-26 15:28:26,861][105620] Updated weights for policy 1, policy_version 14736 (0.0010) [2023-12-26 15:28:26,908][105620] Updated weights for policy 1, policy_version 14746 (0.0010) [2023-12-26 15:28:26,974][105692] Updated weights for policy 0, policy_version 14770 (0.0010) [2023-12-26 15:28:27,028][105692] Updated weights for policy 0, policy_version 14780 (0.0008) [2023-12-26 15:28:27,080][105692] Updated weights for policy 0, policy_version 14790 (0.0008) [2023-12-26 15:28:27,139][105692] Updated weights for policy 0, policy_version 14800 (0.0008) [2023-12-26 15:28:27,610][105620] Updated weights for policy 1, policy_version 14756 (0.0010) [2023-12-26 15:28:27,668][105620] Updated weights for policy 1, policy_version 14766 (0.0010) [2023-12-26 15:28:27,717][105620] Updated weights for policy 1, policy_version 14776 (0.0010) [2023-12-26 15:28:27,804][105692] Updated weights for policy 0, policy_version 14810 (0.0006) [2023-12-26 15:28:27,869][105692] Updated weights for policy 0, policy_version 14820 (0.0006) [2023-12-26 15:28:27,930][105692] Updated weights for policy 0, policy_version 14830 (0.0009) [2023-12-26 15:28:28,484][105692] Updated weights for policy 0, policy_version 14840 (0.0010) [2023-12-26 15:28:28,517][105620] Updated weights for policy 1, policy_version 14786 (0.0010) [2023-12-26 15:28:28,536][105692] Updated weights for policy 0, policy_version 14850 (0.0010) [2023-12-26 15:28:28,567][105620] Updated weights for policy 1, policy_version 14796 (0.0005) [2023-12-26 15:28:28,592][105692] Updated weights for policy 0, policy_version 14860 (0.0010) [2023-12-26 15:28:28,619][105620] Updated weights for policy 1, policy_version 14806 (0.0006) [2023-12-26 15:28:28,677][105620] Updated weights for policy 1, policy_version 14816 (0.0007) [2023-12-26 15:28:29,222][105692] Updated weights for policy 0, policy_version 14870 (0.0008) [2023-12-26 15:28:29,289][105692] Updated weights for policy 0, policy_version 14880 (0.0007) [2023-12-26 15:28:29,352][105692] Updated weights for policy 0, policy_version 14890 (0.0006) [2023-12-26 15:28:29,364][105620] Updated weights for policy 1, policy_version 14826 (0.0007) [2023-12-26 15:28:29,428][105620] Updated weights for policy 1, policy_version 14836 (0.0007) [2023-12-26 15:28:29,494][105620] Updated weights for policy 1, policy_version 14846 (0.0006) [2023-12-26 15:28:29,908][105692] Updated weights for policy 0, policy_version 14900 (0.0009) [2023-12-26 15:28:29,965][105692] Updated weights for policy 0, policy_version 14910 (0.0007) [2023-12-26 15:28:30,019][105692] Updated weights for policy 0, policy_version 14920 (0.0009) [2023-12-26 15:28:30,276][105620] Updated weights for policy 1, policy_version 14856 (0.0007) [2023-12-26 15:28:30,338][105620] Updated weights for policy 1, policy_version 14866 (0.0006) [2023-12-26 15:28:30,405][105620] Updated weights for policy 1, policy_version 14876 (0.0006) [2023-12-26 15:28:30,700][105692] Updated weights for policy 0, policy_version 14930 (0.0009) [2023-12-26 15:28:30,754][105692] Updated weights for policy 0, policy_version 14940 (0.0007) [2023-12-26 15:28:30,801][105692] Updated weights for policy 0, policy_version 14950 (0.0009) [2023-12-26 15:28:30,860][105692] Updated weights for policy 0, policy_version 14960 (0.0008) [2023-12-26 15:28:31,025][105620] Updated weights for policy 1, policy_version 14886 (0.0008) [2023-12-26 15:28:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 7643136. Throughput: 0: 9835.5, 1: 9795.9. Samples: 7611652. Policy #0 lag: (min: 27.0, avg: 54.7, max: 56.0) [2023-12-26 15:28:31,063][104569] Avg episode reward: [(0, '8984.957'), (1, '8432.006')] [2023-12-26 15:28:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000014960_3833856.pth... [2023-12-26 15:28:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000013808_3538944.pth [2023-12-26 15:28:31,092][105620] Updated weights for policy 1, policy_version 14896 (0.0011) [2023-12-26 15:28:31,161][105620] Updated weights for policy 1, policy_version 14906 (0.0008) [2023-12-26 15:28:31,197][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000014912_3817472.pth... [2023-12-26 15:28:31,201][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000013760_3522560.pth [2023-12-26 15:28:31,652][105692] Updated weights for policy 0, policy_version 14970 (0.0010) [2023-12-26 15:28:31,716][105692] Updated weights for policy 0, policy_version 14980 (0.0010) [2023-12-26 15:28:31,767][105692] Updated weights for policy 0, policy_version 14990 (0.0008) [2023-12-26 15:28:31,844][105620] Updated weights for policy 1, policy_version 14916 (0.0007) [2023-12-26 15:28:31,901][105620] Updated weights for policy 1, policy_version 14926 (0.0009) [2023-12-26 15:28:31,966][105620] Updated weights for policy 1, policy_version 14936 (0.0006) [2023-12-26 15:28:32,454][105692] Updated weights for policy 0, policy_version 15000 (0.0006) [2023-12-26 15:28:32,523][105692] Updated weights for policy 0, policy_version 15010 (0.0006) [2023-12-26 15:28:32,587][105692] Updated weights for policy 0, policy_version 15020 (0.0008) [2023-12-26 15:28:32,672][105620] Updated weights for policy 1, policy_version 14946 (0.0007) [2023-12-26 15:28:32,731][105620] Updated weights for policy 1, policy_version 14956 (0.0005) [2023-12-26 15:28:32,787][105620] Updated weights for policy 1, policy_version 14966 (0.0005) [2023-12-26 15:28:32,836][105620] Updated weights for policy 1, policy_version 14976 (0.0006) [2023-12-26 15:28:33,332][105692] Updated weights for policy 0, policy_version 15030 (0.0009) [2023-12-26 15:28:33,399][105692] Updated weights for policy 0, policy_version 15040 (0.0008) [2023-12-26 15:28:33,450][105620] Updated weights for policy 1, policy_version 14986 (0.0007) [2023-12-26 15:28:33,457][105692] Updated weights for policy 0, policy_version 15050 (0.0006) [2023-12-26 15:28:33,507][105620] Updated weights for policy 1, policy_version 14996 (0.0006) [2023-12-26 15:28:33,562][105620] Updated weights for policy 1, policy_version 15006 (0.0006) [2023-12-26 15:28:34,019][105692] Updated weights for policy 0, policy_version 15060 (0.0005) [2023-12-26 15:28:34,069][105692] Updated weights for policy 0, policy_version 15070 (0.0006) [2023-12-26 15:28:34,122][105692] Updated weights for policy 0, policy_version 15080 (0.0009) [2023-12-26 15:28:34,134][105620] Updated weights for policy 1, policy_version 15016 (0.0008) [2023-12-26 15:28:34,199][105620] Updated weights for policy 1, policy_version 15026 (0.0009) [2023-12-26 15:28:34,262][105620] Updated weights for policy 1, policy_version 15036 (0.0009) [2023-12-26 15:28:34,879][105692] Updated weights for policy 0, policy_version 15090 (0.0009) [2023-12-26 15:28:34,927][105692] Updated weights for policy 0, policy_version 15100 (0.0009) [2023-12-26 15:28:34,975][105692] Updated weights for policy 0, policy_version 15110 (0.0005) [2023-12-26 15:28:34,995][105620] Updated weights for policy 1, policy_version 15046 (0.0008) [2023-12-26 15:28:35,026][105692] Updated weights for policy 0, policy_version 15120 (0.0005) [2023-12-26 15:28:35,049][105620] Updated weights for policy 1, policy_version 15056 (0.0006) [2023-12-26 15:28:35,107][105620] Updated weights for policy 1, policy_version 15066 (0.0005) [2023-12-26 15:28:35,645][105620] Updated weights for policy 1, policy_version 15076 (0.0006) [2023-12-26 15:28:35,691][105620] Updated weights for policy 1, policy_version 15086 (0.0009) [2023-12-26 15:28:35,744][105620] Updated weights for policy 1, policy_version 15096 (0.0009) [2023-12-26 15:28:35,775][105692] Updated weights for policy 0, policy_version 15130 (0.0008) [2023-12-26 15:28:35,841][105692] Updated weights for policy 0, policy_version 15140 (0.0009) [2023-12-26 15:28:35,909][105692] Updated weights for policy 0, policy_version 15150 (0.0009) [2023-12-26 15:28:36,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 7749632. Throughput: 0: 9899.2, 1: 9857.6. Samples: 7733668. Policy #0 lag: (min: 3.0, avg: 4.1, max: 23.0) [2023-12-26 15:28:36,063][104569] Avg episode reward: [(0, '9263.154'), (1, '8708.369')] [2023-12-26 15:28:36,063][105585] Saving new best policy, reward=9263.154! [2023-12-26 15:28:36,500][105620] Updated weights for policy 1, policy_version 15106 (0.0007) [2023-12-26 15:28:36,567][105620] Updated weights for policy 1, policy_version 15116 (0.0009) [2023-12-26 15:28:36,636][105620] Updated weights for policy 1, policy_version 15126 (0.0008) [2023-12-26 15:28:36,641][105692] Updated weights for policy 0, policy_version 15160 (0.0006) [2023-12-26 15:28:36,700][105620] Updated weights for policy 1, policy_version 15136 (0.0008) [2023-12-26 15:28:36,706][105692] Updated weights for policy 0, policy_version 15170 (0.0008) [2023-12-26 15:28:36,756][105692] Updated weights for policy 0, policy_version 15180 (0.0009) [2023-12-26 15:28:37,372][105620] Updated weights for policy 1, policy_version 15146 (0.0010) [2023-12-26 15:28:37,427][105620] Updated weights for policy 1, policy_version 15156 (0.0010) [2023-12-26 15:28:37,478][105620] Updated weights for policy 1, policy_version 15166 (0.0010) [2023-12-26 15:28:37,517][105692] Updated weights for policy 0, policy_version 15190 (0.0009) [2023-12-26 15:28:37,584][105692] Updated weights for policy 0, policy_version 15200 (0.0008) [2023-12-26 15:28:37,639][105692] Updated weights for policy 0, policy_version 15210 (0.0008) [2023-12-26 15:28:38,237][105620] Updated weights for policy 1, policy_version 15176 (0.0010) [2023-12-26 15:28:38,299][105620] Updated weights for policy 1, policy_version 15186 (0.0010) [2023-12-26 15:28:38,361][105620] Updated weights for policy 1, policy_version 15196 (0.0010) [2023-12-26 15:28:38,382][105692] Updated weights for policy 0, policy_version 15220 (0.0008) [2023-12-26 15:28:38,431][105692] Updated weights for policy 0, policy_version 15230 (0.0009) [2023-12-26 15:28:38,490][105692] Updated weights for policy 0, policy_version 15240 (0.0008) [2023-12-26 15:28:39,134][105620] Updated weights for policy 1, policy_version 15206 (0.0010) [2023-12-26 15:28:39,189][105620] Updated weights for policy 1, policy_version 15216 (0.0009) [2023-12-26 15:28:39,238][105692] Updated weights for policy 0, policy_version 15250 (0.0008) [2023-12-26 15:28:39,255][105620] Updated weights for policy 1, policy_version 15226 (0.0008) [2023-12-26 15:28:39,303][105692] Updated weights for policy 0, policy_version 15260 (0.0008) [2023-12-26 15:28:39,370][105692] Updated weights for policy 0, policy_version 15270 (0.0009) [2023-12-26 15:28:39,444][105692] Updated weights for policy 0, policy_version 15280 (0.0008) [2023-12-26 15:28:40,067][105620] Updated weights for policy 1, policy_version 15236 (0.0007) [2023-12-26 15:28:40,122][105620] Updated weights for policy 1, policy_version 15246 (0.0008) [2023-12-26 15:28:40,176][105620] Updated weights for policy 1, policy_version 15256 (0.0007) [2023-12-26 15:28:40,191][105692] Updated weights for policy 0, policy_version 15290 (0.0008) [2023-12-26 15:28:40,239][105692] Updated weights for policy 0, policy_version 15300 (0.0008) [2023-12-26 15:28:40,287][105692] Updated weights for policy 0, policy_version 15310 (0.0008) [2023-12-26 15:28:40,946][105620] Updated weights for policy 1, policy_version 15266 (0.0008) [2023-12-26 15:28:40,998][105692] Updated weights for policy 0, policy_version 15320 (0.0006) [2023-12-26 15:28:40,998][105620] Updated weights for policy 1, policy_version 15276 (0.0010) [2023-12-26 15:28:41,060][105692] Updated weights for policy 0, policy_version 15330 (0.0008) [2023-12-26 15:28:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 7831552. Throughput: 0: 9832.5, 1: 9865.7. Samples: 7847396. Policy #0 lag: (min: 3.0, avg: 4.1, max: 23.0) [2023-12-26 15:28:41,062][104569] Avg episode reward: [(0, '9078.131'), (1, '8709.185')] [2023-12-26 15:28:41,063][105620] Updated weights for policy 1, policy_version 15286 (0.0009) [2023-12-26 15:28:41,125][105620] Updated weights for policy 1, policy_version 15296 (0.0010) [2023-12-26 15:28:41,127][105692] Updated weights for policy 0, policy_version 15340 (0.0006) [2023-12-26 15:28:41,849][105692] Updated weights for policy 0, policy_version 15350 (0.0006) [2023-12-26 15:28:41,880][105620] Updated weights for policy 1, policy_version 15306 (0.0008) [2023-12-26 15:28:41,900][105692] Updated weights for policy 0, policy_version 15360 (0.0006) [2023-12-26 15:28:41,939][105620] Updated weights for policy 1, policy_version 15316 (0.0009) [2023-12-26 15:28:41,958][105692] Updated weights for policy 0, policy_version 15370 (0.0006) [2023-12-26 15:28:41,997][105620] Updated weights for policy 1, policy_version 15326 (0.0006) [2023-12-26 15:28:42,686][105692] Updated weights for policy 0, policy_version 15380 (0.0008) [2023-12-26 15:28:42,747][105692] Updated weights for policy 0, policy_version 15390 (0.0008) [2023-12-26 15:28:42,761][105620] Updated weights for policy 1, policy_version 15336 (0.0009) [2023-12-26 15:28:42,808][105692] Updated weights for policy 0, policy_version 15400 (0.0007) [2023-12-26 15:28:42,819][105620] Updated weights for policy 1, policy_version 15346 (0.0006) [2023-12-26 15:28:42,881][105620] Updated weights for policy 1, policy_version 15356 (0.0009) [2023-12-26 15:28:43,544][105620] Updated weights for policy 1, policy_version 15366 (0.0007) [2023-12-26 15:28:43,586][105692] Updated weights for policy 0, policy_version 15410 (0.0007) [2023-12-26 15:28:43,601][105620] Updated weights for policy 1, policy_version 15376 (0.0007) [2023-12-26 15:28:43,643][105692] Updated weights for policy 0, policy_version 15420 (0.0009) [2023-12-26 15:28:43,665][105620] Updated weights for policy 1, policy_version 15386 (0.0007) [2023-12-26 15:28:43,712][105692] Updated weights for policy 0, policy_version 15430 (0.0008) [2023-12-26 15:28:43,776][105692] Updated weights for policy 0, policy_version 15440 (0.0009) [2023-12-26 15:28:44,273][105620] Updated weights for policy 1, policy_version 15396 (0.0006) [2023-12-26 15:28:44,334][105620] Updated weights for policy 1, policy_version 15406 (0.0009) [2023-12-26 15:28:44,385][105620] Updated weights for policy 1, policy_version 15416 (0.0009) [2023-12-26 15:28:44,549][105692] Updated weights for policy 0, policy_version 15450 (0.0009) [2023-12-26 15:28:44,608][105692] Updated weights for policy 0, policy_version 15460 (0.0009) [2023-12-26 15:28:44,675][105692] Updated weights for policy 0, policy_version 15470 (0.0010) [2023-12-26 15:28:45,100][105620] Updated weights for policy 1, policy_version 15426 (0.0009) [2023-12-26 15:28:45,157][105620] Updated weights for policy 1, policy_version 15436 (0.0008) [2023-12-26 15:28:45,218][105620] Updated weights for policy 1, policy_version 15446 (0.0006) [2023-12-26 15:28:45,277][105620] Updated weights for policy 1, policy_version 15456 (0.0005) [2023-12-26 15:28:45,534][105692] Updated weights for policy 0, policy_version 15480 (0.0010) [2023-12-26 15:28:45,591][105692] Updated weights for policy 0, policy_version 15491 (0.0010) [2023-12-26 15:28:45,653][105692] Updated weights for policy 0, policy_version 15501 (0.0010) [2023-12-26 15:28:45,866][105620] Updated weights for policy 1, policy_version 15466 (0.0009) [2023-12-26 15:28:45,930][105620] Updated weights for policy 1, policy_version 15476 (0.0009) [2023-12-26 15:28:45,991][105620] Updated weights for policy 1, policy_version 15486 (0.0008) [2023-12-26 15:28:46,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 7938048. Throughput: 0: 9778.6, 1: 9827.2. Samples: 7905124. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 15:28:46,063][104569] Avg episode reward: [(0, '9076.595'), (1, '8523.410')] [2023-12-26 15:28:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000015488_3964928.pth... [2023-12-26 15:28:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000015504_3973120.pth... [2023-12-26 15:28:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000014336_3670016.pth [2023-12-26 15:28:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000014352_3678208.pth [2023-12-26 15:28:46,433][105692] Updated weights for policy 0, policy_version 15511 (0.0009) [2023-12-26 15:28:46,480][105692] Updated weights for policy 0, policy_version 15521 (0.0009) [2023-12-26 15:28:46,530][105692] Updated weights for policy 0, policy_version 15531 (0.0008) [2023-12-26 15:28:46,735][105620] Updated weights for policy 1, policy_version 15496 (0.0006) [2023-12-26 15:28:46,785][105620] Updated weights for policy 1, policy_version 15506 (0.0005) [2023-12-26 15:28:46,836][105620] Updated weights for policy 1, policy_version 15516 (0.0006) [2023-12-26 15:28:47,322][105692] Updated weights for policy 0, policy_version 15541 (0.0009) [2023-12-26 15:28:47,370][105692] Updated weights for policy 0, policy_version 15551 (0.0009) [2023-12-26 15:28:47,421][105692] Updated weights for policy 0, policy_version 15561 (0.0009) [2023-12-26 15:28:47,516][105620] Updated weights for policy 1, policy_version 15526 (0.0009) [2023-12-26 15:28:47,562][105620] Updated weights for policy 1, policy_version 15536 (0.0008) [2023-12-26 15:28:47,608][105620] Updated weights for policy 1, policy_version 15546 (0.0009) [2023-12-26 15:28:48,203][105692] Updated weights for policy 0, policy_version 15571 (0.0009) [2023-12-26 15:28:48,256][105692] Updated weights for policy 0, policy_version 15581 (0.0009) [2023-12-26 15:28:48,311][105692] Updated weights for policy 0, policy_version 15591 (0.0009) [2023-12-26 15:28:48,349][105620] Updated weights for policy 1, policy_version 15556 (0.0009) [2023-12-26 15:28:48,408][105620] Updated weights for policy 1, policy_version 15566 (0.0008) [2023-12-26 15:28:48,467][105620] Updated weights for policy 1, policy_version 15576 (0.0009) [2023-12-26 15:28:49,070][105692] Updated weights for policy 0, policy_version 15601 (0.0009) [2023-12-26 15:28:49,124][105692] Updated weights for policy 0, policy_version 15611 (0.0009) [2023-12-26 15:28:49,175][105692] Updated weights for policy 0, policy_version 15621 (0.0009) [2023-12-26 15:28:49,234][105692] Updated weights for policy 0, policy_version 15631 (0.0008) [2023-12-26 15:28:49,241][105620] Updated weights for policy 1, policy_version 15586 (0.0009) [2023-12-26 15:28:49,292][105620] Updated weights for policy 1, policy_version 15596 (0.0008) [2023-12-26 15:28:49,341][105620] Updated weights for policy 1, policy_version 15606 (0.0008) [2023-12-26 15:28:49,402][105620] Updated weights for policy 1, policy_version 15616 (0.0007) [2023-12-26 15:28:50,061][105620] Updated weights for policy 1, policy_version 15626 (0.0009) [2023-12-26 15:28:50,112][105692] Updated weights for policy 0, policy_version 15641 (0.0008) [2023-12-26 15:28:50,126][105620] Updated weights for policy 1, policy_version 15636 (0.0006) [2023-12-26 15:28:50,169][105692] Updated weights for policy 0, policy_version 15651 (0.0007) [2023-12-26 15:28:50,188][105620] Updated weights for policy 1, policy_version 15646 (0.0006) [2023-12-26 15:28:50,232][105692] Updated weights for policy 0, policy_version 15661 (0.0009) [2023-12-26 15:28:50,864][105620] Updated weights for policy 1, policy_version 15656 (0.0008) [2023-12-26 15:28:50,926][105620] Updated weights for policy 1, policy_version 15666 (0.0009) [2023-12-26 15:28:50,974][105620] Updated weights for policy 1, policy_version 15676 (0.0009) [2023-12-26 15:28:51,041][105692] Updated weights for policy 0, policy_version 15671 (0.0009) [2023-12-26 15:28:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 8028160. Throughput: 0: 9659.4, 1: 9839.5. Samples: 8019000. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 15:28:51,063][104569] Avg episode reward: [(0, '9168.380'), (1, '8523.195')] [2023-12-26 15:28:51,103][105692] Updated weights for policy 0, policy_version 15681 (0.0009) [2023-12-26 15:28:51,166][105692] Updated weights for policy 0, policy_version 15691 (0.0009) [2023-12-26 15:28:51,773][105620] Updated weights for policy 1, policy_version 15686 (0.0009) [2023-12-26 15:28:51,835][105620] Updated weights for policy 1, policy_version 15696 (0.0011) [2023-12-26 15:28:51,898][105620] Updated weights for policy 1, policy_version 15706 (0.0008) [2023-12-26 15:28:51,967][105692] Updated weights for policy 0, policy_version 15701 (0.0009) [2023-12-26 15:28:52,027][105692] Updated weights for policy 0, policy_version 15711 (0.0008) [2023-12-26 15:28:52,088][105692] Updated weights for policy 0, policy_version 15721 (0.0008) [2023-12-26 15:28:52,644][105620] Updated weights for policy 1, policy_version 15716 (0.0010) [2023-12-26 15:28:52,707][105620] Updated weights for policy 1, policy_version 15726 (0.0008) [2023-12-26 15:28:52,768][105620] Updated weights for policy 1, policy_version 15736 (0.0010) [2023-12-26 15:28:52,865][105692] Updated weights for policy 0, policy_version 15731 (0.0009) [2023-12-26 15:28:52,919][105692] Updated weights for policy 0, policy_version 15742 (0.0010) [2023-12-26 15:28:52,964][105692] Updated weights for policy 0, policy_version 15752 (0.0006) [2023-12-26 15:28:53,461][105620] Updated weights for policy 1, policy_version 15746 (0.0010) [2023-12-26 15:28:53,513][105620] Updated weights for policy 1, policy_version 15756 (0.0010) [2023-12-26 15:28:53,550][105692] Updated weights for policy 0, policy_version 15762 (0.0007) [2023-12-26 15:28:53,567][105620] Updated weights for policy 1, policy_version 15766 (0.0010) [2023-12-26 15:28:53,612][105692] Updated weights for policy 0, policy_version 15772 (0.0010) [2023-12-26 15:28:53,615][105620] Updated weights for policy 1, policy_version 15776 (0.0010) [2023-12-26 15:28:53,664][105692] Updated weights for policy 0, policy_version 15782 (0.0010) [2023-12-26 15:28:53,709][105692] Updated weights for policy 0, policy_version 15792 (0.0010) [2023-12-26 15:28:54,375][105620] Updated weights for policy 1, policy_version 15786 (0.0010) [2023-12-26 15:28:54,430][105620] Updated weights for policy 1, policy_version 15796 (0.0010) [2023-12-26 15:28:54,439][105692] Updated weights for policy 0, policy_version 15802 (0.0006) [2023-12-26 15:28:54,493][105692] Updated weights for policy 0, policy_version 15812 (0.0007) [2023-12-26 15:28:54,499][105620] Updated weights for policy 1, policy_version 15806 (0.0010) [2023-12-26 15:28:54,545][105692] Updated weights for policy 0, policy_version 15822 (0.0010) [2023-12-26 15:28:55,158][105692] Updated weights for policy 0, policy_version 15832 (0.0011) [2023-12-26 15:28:55,181][105620] Updated weights for policy 1, policy_version 15816 (0.0006) [2023-12-26 15:28:55,220][105692] Updated weights for policy 0, policy_version 15842 (0.0009) [2023-12-26 15:28:55,234][105620] Updated weights for policy 1, policy_version 15826 (0.0010) [2023-12-26 15:28:55,273][105692] Updated weights for policy 0, policy_version 15852 (0.0010) [2023-12-26 15:28:55,284][105620] Updated weights for policy 1, policy_version 15836 (0.0010) [2023-12-26 15:28:55,922][105692] Updated weights for policy 0, policy_version 15862 (0.0006) [2023-12-26 15:28:55,968][105692] Updated weights for policy 0, policy_version 15872 (0.0005) [2023-12-26 15:28:56,021][105692] Updated weights for policy 0, policy_version 15882 (0.0005) [2023-12-26 15:28:56,022][105620] Updated weights for policy 1, policy_version 15846 (0.0010) [2023-12-26 15:28:56,062][104569] Fps is (10 sec: 18842.3, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 8126464. Throughput: 0: 9685.0, 1: 9843.0. Samples: 8134112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:28:56,062][104569] Avg episode reward: [(0, '9076.805'), (1, '8802.045')] [2023-12-26 15:28:56,081][105620] Updated weights for policy 1, policy_version 15856 (0.0010) [2023-12-26 15:28:56,135][105620] Updated weights for policy 1, policy_version 15866 (0.0010) [2023-12-26 15:28:56,658][105692] Updated weights for policy 0, policy_version 15892 (0.0006) [2023-12-26 15:28:56,712][105692] Updated weights for policy 0, policy_version 15902 (0.0006) [2023-12-26 15:28:56,767][105692] Updated weights for policy 0, policy_version 15912 (0.0006) [2023-12-26 15:28:56,866][105620] Updated weights for policy 1, policy_version 15876 (0.0010) [2023-12-26 15:28:56,910][105620] Updated weights for policy 1, policy_version 15886 (0.0010) [2023-12-26 15:28:56,958][105620] Updated weights for policy 1, policy_version 15896 (0.0010) [2023-12-26 15:28:57,427][105692] Updated weights for policy 0, policy_version 15922 (0.0006) [2023-12-26 15:28:57,473][105692] Updated weights for policy 0, policy_version 15932 (0.0008) [2023-12-26 15:28:57,528][105692] Updated weights for policy 0, policy_version 15942 (0.0009) [2023-12-26 15:28:57,592][105692] Updated weights for policy 0, policy_version 15952 (0.0006) [2023-12-26 15:28:57,645][105620] Updated weights for policy 1, policy_version 15906 (0.0006) [2023-12-26 15:28:57,709][105620] Updated weights for policy 1, policy_version 15916 (0.0005) [2023-12-26 15:28:57,766][105620] Updated weights for policy 1, policy_version 15926 (0.0007) [2023-12-26 15:28:57,817][105620] Updated weights for policy 1, policy_version 15936 (0.0010) [2023-12-26 15:28:58,272][105692] Updated weights for policy 0, policy_version 15962 (0.0011) [2023-12-26 15:28:58,334][105692] Updated weights for policy 0, policy_version 15972 (0.0010) [2023-12-26 15:28:58,401][105692] Updated weights for policy 0, policy_version 15982 (0.0010) [2023-12-26 15:28:58,476][105620] Updated weights for policy 1, policy_version 15947 (0.0007) [2023-12-26 15:28:58,545][105620] Updated weights for policy 1, policy_version 15957 (0.0008) [2023-12-26 15:28:58,609][105620] Updated weights for policy 1, policy_version 15967 (0.0008) [2023-12-26 15:28:59,119][105692] Updated weights for policy 0, policy_version 15992 (0.0008) [2023-12-26 15:28:59,176][105692] Updated weights for policy 0, policy_version 16002 (0.0005) [2023-12-26 15:28:59,232][105692] Updated weights for policy 0, policy_version 16012 (0.0008) [2023-12-26 15:28:59,249][105620] Updated weights for policy 1, policy_version 15977 (0.0007) [2023-12-26 15:28:59,311][105620] Updated weights for policy 1, policy_version 15987 (0.0006) [2023-12-26 15:28:59,374][105620] Updated weights for policy 1, policy_version 15997 (0.0009) [2023-12-26 15:28:59,981][105692] Updated weights for policy 0, policy_version 16022 (0.0010) [2023-12-26 15:29:00,044][105692] Updated weights for policy 0, policy_version 16032 (0.0010) [2023-12-26 15:29:00,058][105620] Updated weights for policy 1, policy_version 16007 (0.0008) [2023-12-26 15:29:00,106][105692] Updated weights for policy 0, policy_version 16042 (0.0007) [2023-12-26 15:29:00,112][105620] Updated weights for policy 1, policy_version 16017 (0.0009) [2023-12-26 15:29:00,173][105620] Updated weights for policy 1, policy_version 16027 (0.0007) [2023-12-26 15:29:00,764][105692] Updated weights for policy 0, policy_version 16052 (0.0008) [2023-12-26 15:29:00,812][105692] Updated weights for policy 0, policy_version 16062 (0.0006) [2023-12-26 15:29:00,867][105692] Updated weights for policy 0, policy_version 16072 (0.0010) [2023-12-26 15:29:00,926][105620] Updated weights for policy 1, policy_version 16037 (0.0008) [2023-12-26 15:29:00,978][105620] Updated weights for policy 1, policy_version 16048 (0.0010) [2023-12-26 15:29:01,035][105620] Updated weights for policy 1, policy_version 16058 (0.0010) [2023-12-26 15:29:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 8224768. Throughput: 0: 9734.7, 1: 9854.4. Samples: 8195080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:29:01,062][104569] Avg episode reward: [(0, '8984.624'), (1, '8895.058')] [2023-12-26 15:29:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000016080_4120576.pth... [2023-12-26 15:29:01,076][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000016064_4112384.pth... [2023-12-26 15:29:01,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000014912_3817472.pth [2023-12-26 15:29:01,084][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000014960_3833856.pth [2023-12-26 15:29:01,508][105692] Updated weights for policy 0, policy_version 16082 (0.0009) [2023-12-26 15:29:01,569][105692] Updated weights for policy 0, policy_version 16092 (0.0007) [2023-12-26 15:29:01,635][105692] Updated weights for policy 0, policy_version 16102 (0.0011) [2023-12-26 15:29:01,692][105692] Updated weights for policy 0, policy_version 16112 (0.0010) [2023-12-26 15:29:01,851][105620] Updated weights for policy 1, policy_version 16068 (0.0007) [2023-12-26 15:29:01,915][105620] Updated weights for policy 1, policy_version 16078 (0.0009) [2023-12-26 15:29:01,968][105620] Updated weights for policy 1, policy_version 16089 (0.0010) [2023-12-26 15:29:02,360][105692] Updated weights for policy 0, policy_version 16122 (0.0007) [2023-12-26 15:29:02,418][105692] Updated weights for policy 0, policy_version 16132 (0.0008) [2023-12-26 15:29:02,476][105692] Updated weights for policy 0, policy_version 16142 (0.0008) [2023-12-26 15:29:02,485][105585] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000008 [2023-12-26 15:29:02,693][105620] Updated weights for policy 1, policy_version 16099 (0.0007) [2023-12-26 15:29:02,758][105620] Updated weights for policy 1, policy_version 16109 (0.0005) [2023-12-26 15:29:02,828][105620] Updated weights for policy 1, policy_version 16119 (0.0007) [2023-12-26 15:29:03,148][105692] Updated weights for policy 0, policy_version 16152 (0.0009) [2023-12-26 15:29:03,196][105692] Updated weights for policy 0, policy_version 16162 (0.0009) [2023-12-26 15:29:03,247][105692] Updated weights for policy 0, policy_version 16172 (0.0009) [2023-12-26 15:29:03,521][105620] Updated weights for policy 1, policy_version 16129 (0.0009) [2023-12-26 15:29:03,588][105620] Updated weights for policy 1, policy_version 16139 (0.0005) [2023-12-26 15:29:03,641][105620] Updated weights for policy 1, policy_version 16149 (0.0005) [2023-12-26 15:29:03,691][105620] Updated weights for policy 1, policy_version 16159 (0.0005) [2023-12-26 15:29:04,061][105692] Updated weights for policy 0, policy_version 16182 (0.0010) [2023-12-26 15:29:04,124][105692] Updated weights for policy 0, policy_version 16192 (0.0009) [2023-12-26 15:29:04,184][105692] Updated weights for policy 0, policy_version 16202 (0.0009) [2023-12-26 15:29:04,380][105620] Updated weights for policy 1, policy_version 16169 (0.0008) [2023-12-26 15:29:04,445][105620] Updated weights for policy 1, policy_version 16179 (0.0007) [2023-12-26 15:29:04,510][105620] Updated weights for policy 1, policy_version 16189 (0.0008) [2023-12-26 15:29:04,838][105692] Updated weights for policy 0, policy_version 16212 (0.0006) [2023-12-26 15:29:04,896][105692] Updated weights for policy 0, policy_version 16222 (0.0005) [2023-12-26 15:29:04,958][105692] Updated weights for policy 0, policy_version 16232 (0.0007) [2023-12-26 15:29:05,264][105620] Updated weights for policy 1, policy_version 16199 (0.0008) [2023-12-26 15:29:05,317][105620] Updated weights for policy 1, policy_version 16209 (0.0009) [2023-12-26 15:29:05,375][105620] Updated weights for policy 1, policy_version 16219 (0.0009) [2023-12-26 15:29:05,642][105692] Updated weights for policy 0, policy_version 16242 (0.0005) [2023-12-26 15:29:05,698][105692] Updated weights for policy 0, policy_version 16252 (0.0006) [2023-12-26 15:29:05,744][105692] Updated weights for policy 0, policy_version 16262 (0.0008) [2023-12-26 15:29:05,799][105692] Updated weights for policy 0, policy_version 16272 (0.0009) [2023-12-26 15:29:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 8323072. Throughput: 0: 9847.8, 1: 9769.9. Samples: 8313496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:29:06,062][104569] Avg episode reward: [(0, '8892.461'), (1, '8987.559')] [2023-12-26 15:29:06,189][105620] Updated weights for policy 1, policy_version 16230 (0.0010) [2023-12-26 15:29:06,239][105620] Updated weights for policy 1, policy_version 16240 (0.0008) [2023-12-26 15:29:06,294][105620] Updated weights for policy 1, policy_version 16250 (0.0008) [2023-12-26 15:29:06,469][105692] Updated weights for policy 0, policy_version 16282 (0.0010) [2023-12-26 15:29:06,530][105692] Updated weights for policy 0, policy_version 16292 (0.0008) [2023-12-26 15:29:06,589][105692] Updated weights for policy 0, policy_version 16302 (0.0009) [2023-12-26 15:29:07,083][105620] Updated weights for policy 1, policy_version 16260 (0.0009) [2023-12-26 15:29:07,140][105620] Updated weights for policy 1, policy_version 16270 (0.0009) [2023-12-26 15:29:07,196][105620] Updated weights for policy 1, policy_version 16280 (0.0009) [2023-12-26 15:29:07,320][105692] Updated weights for policy 0, policy_version 16312 (0.0009) [2023-12-26 15:29:07,367][105692] Updated weights for policy 0, policy_version 16322 (0.0009) [2023-12-26 15:29:07,422][105692] Updated weights for policy 0, policy_version 16332 (0.0006) [2023-12-26 15:29:07,998][105620] Updated weights for policy 1, policy_version 16290 (0.0009) [2023-12-26 15:29:08,050][105620] Updated weights for policy 1, policy_version 16300 (0.0010) [2023-12-26 15:29:08,096][105692] Updated weights for policy 0, policy_version 16342 (0.0005) [2023-12-26 15:29:08,103][105620] Updated weights for policy 1, policy_version 16310 (0.0009) [2023-12-26 15:29:08,158][105692] Updated weights for policy 0, policy_version 16352 (0.0005) [2023-12-26 15:29:08,168][105620] Updated weights for policy 1, policy_version 16320 (0.0009) [2023-12-26 15:29:08,219][105692] Updated weights for policy 0, policy_version 16362 (0.0005) [2023-12-26 15:29:08,803][105692] Updated weights for policy 0, policy_version 16372 (0.0005) [2023-12-26 15:29:08,860][105692] Updated weights for policy 0, policy_version 16382 (0.0005) [2023-12-26 15:29:08,920][105692] Updated weights for policy 0, policy_version 16392 (0.0008) [2023-12-26 15:29:09,017][105620] Updated weights for policy 1, policy_version 16330 (0.0009) [2023-12-26 15:29:09,074][105620] Updated weights for policy 1, policy_version 16340 (0.0010) [2023-12-26 15:29:09,130][105620] Updated weights for policy 1, policy_version 16350 (0.0008) [2023-12-26 15:29:09,582][105692] Updated weights for policy 0, policy_version 16402 (0.0010) [2023-12-26 15:29:09,649][105692] Updated weights for policy 0, policy_version 16412 (0.0008) [2023-12-26 15:29:09,716][105692] Updated weights for policy 0, policy_version 16422 (0.0008) [2023-12-26 15:29:09,783][105692] Updated weights for policy 0, policy_version 16432 (0.0008) [2023-12-26 15:29:09,916][105620] Updated weights for policy 1, policy_version 16360 (0.0008) [2023-12-26 15:29:09,982][105620] Updated weights for policy 1, policy_version 16370 (0.0010) [2023-12-26 15:29:10,042][105620] Updated weights for policy 1, policy_version 16380 (0.0011) [2023-12-26 15:29:10,519][105692] Updated weights for policy 0, policy_version 16442 (0.0009) [2023-12-26 15:29:10,575][105692] Updated weights for policy 0, policy_version 16452 (0.0008) [2023-12-26 15:29:10,623][105692] Updated weights for policy 0, policy_version 16462 (0.0008) [2023-12-26 15:29:10,764][105620] Updated weights for policy 1, policy_version 16390 (0.0010) [2023-12-26 15:29:10,819][105620] Updated weights for policy 1, policy_version 16400 (0.0010) [2023-12-26 15:29:10,891][105620] Updated weights for policy 1, policy_version 16410 (0.0010) [2023-12-26 15:29:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 8421376. Throughput: 0: 9812.5, 1: 9699.7. Samples: 8428300. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-26 15:29:11,062][104569] Avg episode reward: [(0, '8893.607'), (1, '8708.881')] [2023-12-26 15:29:11,394][105692] Updated weights for policy 0, policy_version 16472 (0.0009) [2023-12-26 15:29:11,452][105692] Updated weights for policy 0, policy_version 16482 (0.0007) [2023-12-26 15:29:11,505][105692] Updated weights for policy 0, policy_version 16492 (0.0008) [2023-12-26 15:29:11,652][105620] Updated weights for policy 1, policy_version 16420 (0.0010) [2023-12-26 15:29:11,711][105620] Updated weights for policy 1, policy_version 16430 (0.0010) [2023-12-26 15:29:11,784][105620] Updated weights for policy 1, policy_version 16440 (0.0011) [2023-12-26 15:29:12,214][105692] Updated weights for policy 0, policy_version 16502 (0.0007) [2023-12-26 15:29:12,280][105692] Updated weights for policy 0, policy_version 16512 (0.0008) [2023-12-26 15:29:12,344][105692] Updated weights for policy 0, policy_version 16522 (0.0008) [2023-12-26 15:29:12,483][105620] Updated weights for policy 1, policy_version 16450 (0.0007) [2023-12-26 15:29:12,548][105620] Updated weights for policy 1, policy_version 16460 (0.0011) [2023-12-26 15:29:12,617][105620] Updated weights for policy 1, policy_version 16470 (0.0010) [2023-12-26 15:29:12,676][105620] Updated weights for policy 1, policy_version 16480 (0.0008) [2023-12-26 15:29:13,071][105692] Updated weights for policy 0, policy_version 16532 (0.0010) [2023-12-26 15:29:13,128][105692] Updated weights for policy 0, policy_version 16542 (0.0009) [2023-12-26 15:29:13,185][105692] Updated weights for policy 0, policy_version 16552 (0.0008) [2023-12-26 15:29:13,205][105620] Updated weights for policy 1, policy_version 16490 (0.0005) [2023-12-26 15:29:13,252][105620] Updated weights for policy 1, policy_version 16500 (0.0005) [2023-12-26 15:29:13,302][105620] Updated weights for policy 1, policy_version 16510 (0.0005) [2023-12-26 15:29:13,880][105620] Updated weights for policy 1, policy_version 16520 (0.0009) [2023-12-26 15:29:13,939][105620] Updated weights for policy 1, policy_version 16530 (0.0010) [2023-12-26 15:29:14,003][105620] Updated weights for policy 1, policy_version 16540 (0.0007) [2023-12-26 15:29:14,026][105692] Updated weights for policy 0, policy_version 16562 (0.0009) [2023-12-26 15:29:14,074][105692] Updated weights for policy 0, policy_version 16572 (0.0008) [2023-12-26 15:29:14,138][105692] Updated weights for policy 0, policy_version 16582 (0.0007) [2023-12-26 15:29:14,198][105692] Updated weights for policy 0, policy_version 16592 (0.0010) [2023-12-26 15:29:14,568][105620] Updated weights for policy 1, policy_version 16550 (0.0005) [2023-12-26 15:29:14,628][105620] Updated weights for policy 1, policy_version 16560 (0.0005) [2023-12-26 15:29:14,692][105620] Updated weights for policy 1, policy_version 16570 (0.0005) [2023-12-26 15:29:14,925][105692] Updated weights for policy 0, policy_version 16602 (0.0006) [2023-12-26 15:29:14,988][105692] Updated weights for policy 0, policy_version 16612 (0.0008) [2023-12-26 15:29:15,053][105692] Updated weights for policy 0, policy_version 16622 (0.0011) [2023-12-26 15:29:15,356][105620] Updated weights for policy 1, policy_version 16580 (0.0007) [2023-12-26 15:29:15,415][105620] Updated weights for policy 1, policy_version 16590 (0.0008) [2023-12-26 15:29:15,468][105620] Updated weights for policy 1, policy_version 16600 (0.0008) [2023-12-26 15:29:15,740][105692] Updated weights for policy 0, policy_version 16632 (0.0011) [2023-12-26 15:29:15,792][105692] Updated weights for policy 0, policy_version 16642 (0.0010) [2023-12-26 15:29:15,843][105692] Updated weights for policy 0, policy_version 16652 (0.0010) [2023-12-26 15:29:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 8519680. Throughput: 0: 9711.2, 1: 9752.5. Samples: 8487520. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-26 15:29:16,063][104569] Avg episode reward: [(0, '8892.919'), (1, '8893.374')] [2023-12-26 15:29:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000016656_4268032.pth... [2023-12-26 15:29:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000016608_4251648.pth... [2023-12-26 15:29:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000015504_3973120.pth [2023-12-26 15:29:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000015488_3964928.pth [2023-12-26 15:29:16,262][105620] Updated weights for policy 1, policy_version 16610 (0.0008) [2023-12-26 15:29:16,313][105620] Updated weights for policy 1, policy_version 16620 (0.0008) [2023-12-26 15:29:16,359][105620] Updated weights for policy 1, policy_version 16630 (0.0008) [2023-12-26 15:29:16,423][105620] Updated weights for policy 1, policy_version 16640 (0.0008) [2023-12-26 15:29:16,532][105692] Updated weights for policy 0, policy_version 16662 (0.0010) [2023-12-26 15:29:16,592][105692] Updated weights for policy 0, policy_version 16672 (0.0010) [2023-12-26 15:29:16,646][105692] Updated weights for policy 0, policy_version 16682 (0.0010) [2023-12-26 15:29:17,233][105620] Updated weights for policy 1, policy_version 16650 (0.0008) [2023-12-26 15:29:17,277][105692] Updated weights for policy 0, policy_version 16692 (0.0008) [2023-12-26 15:29:17,299][105620] Updated weights for policy 1, policy_version 16660 (0.0009) [2023-12-26 15:29:17,334][105692] Updated weights for policy 0, policy_version 16702 (0.0008) [2023-12-26 15:29:17,367][105620] Updated weights for policy 1, policy_version 16670 (0.0007) [2023-12-26 15:29:17,398][105692] Updated weights for policy 0, policy_version 16712 (0.0008) [2023-12-26 15:29:18,030][105620] Updated weights for policy 1, policy_version 16680 (0.0008) [2023-12-26 15:29:18,077][105620] Updated weights for policy 1, policy_version 16690 (0.0007) [2023-12-26 15:29:18,078][105692] Updated weights for policy 0, policy_version 16722 (0.0011) [2023-12-26 15:29:18,134][105620] Updated weights for policy 1, policy_version 16700 (0.0006) [2023-12-26 15:29:18,137][105692] Updated weights for policy 0, policy_version 16732 (0.0010) [2023-12-26 15:29:18,188][105692] Updated weights for policy 0, policy_version 16742 (0.0010) [2023-12-26 15:29:18,246][105692] Updated weights for policy 0, policy_version 16752 (0.0010) [2023-12-26 15:29:18,821][105620] Updated weights for policy 1, policy_version 16710 (0.0006) [2023-12-26 15:29:18,881][105620] Updated weights for policy 1, policy_version 16720 (0.0006) [2023-12-26 15:29:18,935][105620] Updated weights for policy 1, policy_version 16730 (0.0008) [2023-12-26 15:29:18,996][105692] Updated weights for policy 0, policy_version 16762 (0.0010) [2023-12-26 15:29:19,055][105692] Updated weights for policy 0, policy_version 16772 (0.0010) [2023-12-26 15:29:19,118][105692] Updated weights for policy 0, policy_version 16782 (0.0006) [2023-12-26 15:29:19,578][105620] Updated weights for policy 1, policy_version 16740 (0.0006) [2023-12-26 15:29:19,640][105620] Updated weights for policy 1, policy_version 16750 (0.0009) [2023-12-26 15:29:19,694][105620] Updated weights for policy 1, policy_version 16760 (0.0009) [2023-12-26 15:29:19,879][105692] Updated weights for policy 0, policy_version 16792 (0.0008) [2023-12-26 15:29:19,941][105692] Updated weights for policy 0, policy_version 16802 (0.0009) [2023-12-26 15:29:20,001][105692] Updated weights for policy 0, policy_version 16812 (0.0009) [2023-12-26 15:29:20,424][105620] Updated weights for policy 1, policy_version 16770 (0.0011) [2023-12-26 15:29:20,485][105620] Updated weights for policy 1, policy_version 16780 (0.0011) [2023-12-26 15:29:20,541][105620] Updated weights for policy 1, policy_version 16790 (0.0011) [2023-12-26 15:29:20,606][105620] Updated weights for policy 1, policy_version 16800 (0.0011) [2023-12-26 15:29:20,807][105692] Updated weights for policy 0, policy_version 16822 (0.0008) [2023-12-26 15:29:20,869][105692] Updated weights for policy 0, policy_version 16832 (0.0008) [2023-12-26 15:29:20,921][105692] Updated weights for policy 0, policy_version 16842 (0.0010) [2023-12-26 15:29:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 8617984. Throughput: 0: 9662.4, 1: 9731.6. Samples: 8606396. Policy #0 lag: (min: 29.0, avg: 36.3, max: 61.0) [2023-12-26 15:29:21,063][104569] Avg episode reward: [(0, '8984.273'), (1, '8986.798')] [2023-12-26 15:29:21,308][105620] Updated weights for policy 1, policy_version 16810 (0.0011) [2023-12-26 15:29:21,380][105620] Updated weights for policy 1, policy_version 16820 (0.0010) [2023-12-26 15:29:21,444][105620] Updated weights for policy 1, policy_version 16830 (0.0009) [2023-12-26 15:29:21,777][105692] Updated weights for policy 0, policy_version 16852 (0.0011) [2023-12-26 15:29:21,834][105692] Updated weights for policy 0, policy_version 16862 (0.0008) [2023-12-26 15:29:21,888][105692] Updated weights for policy 0, policy_version 16872 (0.0008) [2023-12-26 15:29:22,200][105620] Updated weights for policy 1, policy_version 16840 (0.0006) [2023-12-26 15:29:22,272][105620] Updated weights for policy 1, policy_version 16850 (0.0009) [2023-12-26 15:29:22,340][105620] Updated weights for policy 1, policy_version 16860 (0.0008) [2023-12-26 15:29:22,735][105692] Updated weights for policy 0, policy_version 16882 (0.0007) [2023-12-26 15:29:22,802][105692] Updated weights for policy 0, policy_version 16892 (0.0008) [2023-12-26 15:29:22,872][105692] Updated weights for policy 0, policy_version 16902 (0.0006) [2023-12-26 15:29:22,932][105692] Updated weights for policy 0, policy_version 16912 (0.0006) [2023-12-26 15:29:23,101][105620] Updated weights for policy 1, policy_version 16870 (0.0010) [2023-12-26 15:29:23,166][105620] Updated weights for policy 1, policy_version 16880 (0.0009) [2023-12-26 15:29:23,222][105620] Updated weights for policy 1, policy_version 16890 (0.0005) [2023-12-26 15:29:23,588][105692] Updated weights for policy 0, policy_version 16922 (0.0007) [2023-12-26 15:29:23,631][105692] Updated weights for policy 0, policy_version 16932 (0.0007) [2023-12-26 15:29:23,688][105692] Updated weights for policy 0, policy_version 16942 (0.0006) [2023-12-26 15:29:23,967][105620] Updated weights for policy 1, policy_version 16900 (0.0006) [2023-12-26 15:29:24,030][105620] Updated weights for policy 1, policy_version 16910 (0.0008) [2023-12-26 15:29:24,083][105620] Updated weights for policy 1, policy_version 16920 (0.0010) [2023-12-26 15:29:24,295][105692] Updated weights for policy 0, policy_version 16952 (0.0005) [2023-12-26 15:29:24,364][105692] Updated weights for policy 0, policy_version 16962 (0.0006) [2023-12-26 15:29:24,422][105692] Updated weights for policy 0, policy_version 16972 (0.0010) [2023-12-26 15:29:24,658][105620] Updated weights for policy 1, policy_version 16930 (0.0010) [2023-12-26 15:29:24,716][105620] Updated weights for policy 1, policy_version 16940 (0.0008) [2023-12-26 15:29:24,775][105620] Updated weights for policy 1, policy_version 16950 (0.0006) [2023-12-26 15:29:24,826][105620] Updated weights for policy 1, policy_version 16960 (0.0005) [2023-12-26 15:29:25,163][105692] Updated weights for policy 0, policy_version 16982 (0.0012) [2023-12-26 15:29:25,211][105692] Updated weights for policy 0, policy_version 16992 (0.0010) [2023-12-26 15:29:25,266][105692] Updated weights for policy 0, policy_version 17002 (0.0010) [2023-12-26 15:29:25,416][105620] Updated weights for policy 1, policy_version 16970 (0.0008) [2023-12-26 15:29:25,463][105620] Updated weights for policy 1, policy_version 16980 (0.0008) [2023-12-26 15:29:25,509][105620] Updated weights for policy 1, policy_version 16990 (0.0008) [2023-12-26 15:29:26,021][105692] Updated weights for policy 0, policy_version 17012 (0.0010) [2023-12-26 15:29:26,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 8708096. Throughput: 0: 9657.7, 1: 9770.1. Samples: 8721648. Policy #0 lag: (min: 29.0, avg: 36.3, max: 61.0) [2023-12-26 15:29:26,062][104569] Avg episode reward: [(0, '8893.384'), (1, '8985.824')] [2023-12-26 15:29:26,068][105692] Updated weights for policy 0, policy_version 17022 (0.0010) [2023-12-26 15:29:26,126][105692] Updated weights for policy 0, policy_version 17032 (0.0010) [2023-12-26 15:29:26,253][105620] Updated weights for policy 1, policy_version 17000 (0.0009) [2023-12-26 15:29:26,312][105620] Updated weights for policy 1, policy_version 17010 (0.0010) [2023-12-26 15:29:26,372][105620] Updated weights for policy 1, policy_version 17020 (0.0008) [2023-12-26 15:29:26,858][105692] Updated weights for policy 0, policy_version 17042 (0.0009) [2023-12-26 15:29:26,920][105692] Updated weights for policy 0, policy_version 17052 (0.0006) [2023-12-26 15:29:26,947][105620] Updated weights for policy 1, policy_version 17030 (0.0008) [2023-12-26 15:29:26,974][105692] Updated weights for policy 0, policy_version 17062 (0.0007) [2023-12-26 15:29:27,003][105620] Updated weights for policy 1, policy_version 17040 (0.0010) [2023-12-26 15:29:27,029][105692] Updated weights for policy 0, policy_version 17072 (0.0006) [2023-12-26 15:29:27,085][105620] Updated weights for policy 1, policy_version 17050 (0.0011) [2023-12-26 15:29:27,702][105692] Updated weights for policy 0, policy_version 17082 (0.0008) [2023-12-26 15:29:27,752][105692] Updated weights for policy 0, policy_version 17092 (0.0008) [2023-12-26 15:29:27,764][105620] Updated weights for policy 1, policy_version 17060 (0.0010) [2023-12-26 15:29:27,794][105692] Updated weights for policy 0, policy_version 17102 (0.0008) [2023-12-26 15:29:27,812][105620] Updated weights for policy 1, policy_version 17070 (0.0010) [2023-12-26 15:29:27,862][105620] Updated weights for policy 1, policy_version 17080 (0.0010) [2023-12-26 15:29:28,426][105692] Updated weights for policy 0, policy_version 17112 (0.0009) [2023-12-26 15:29:28,483][105692] Updated weights for policy 0, policy_version 17122 (0.0009) [2023-12-26 15:29:28,529][105620] Updated weights for policy 1, policy_version 17090 (0.0010) [2023-12-26 15:29:28,544][105692] Updated weights for policy 0, policy_version 17132 (0.0007) [2023-12-26 15:29:28,585][105620] Updated weights for policy 1, policy_version 17100 (0.0007) [2023-12-26 15:29:28,642][105620] Updated weights for policy 1, policy_version 17110 (0.0005) [2023-12-26 15:29:28,695][105620] Updated weights for policy 1, policy_version 17120 (0.0005) [2023-12-26 15:29:29,346][105692] Updated weights for policy 0, policy_version 17142 (0.0008) [2023-12-26 15:29:29,386][105620] Updated weights for policy 1, policy_version 17130 (0.0007) [2023-12-26 15:29:29,406][105692] Updated weights for policy 0, policy_version 17152 (0.0006) [2023-12-26 15:29:29,442][105620] Updated weights for policy 1, policy_version 17140 (0.0008) [2023-12-26 15:29:29,457][105692] Updated weights for policy 0, policy_version 17162 (0.0005) [2023-12-26 15:29:29,491][105620] Updated weights for policy 1, policy_version 17150 (0.0009) [2023-12-26 15:29:30,029][105692] Updated weights for policy 0, policy_version 17172 (0.0006) [2023-12-26 15:29:30,087][105692] Updated weights for policy 0, policy_version 17182 (0.0008) [2023-12-26 15:29:30,144][105692] Updated weights for policy 0, policy_version 17192 (0.0008) [2023-12-26 15:29:30,332][105620] Updated weights for policy 1, policy_version 17160 (0.0010) [2023-12-26 15:29:30,393][105620] Updated weights for policy 1, policy_version 17170 (0.0009) [2023-12-26 15:29:30,448][105620] Updated weights for policy 1, policy_version 17180 (0.0009) [2023-12-26 15:29:30,744][105692] Updated weights for policy 0, policy_version 17202 (0.0005) [2023-12-26 15:29:30,790][105692] Updated weights for policy 0, policy_version 17212 (0.0006) [2023-12-26 15:29:30,850][105692] Updated weights for policy 0, policy_version 17222 (0.0007) [2023-12-26 15:29:30,913][105692] Updated weights for policy 0, policy_version 17232 (0.0006) [2023-12-26 15:29:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 8814592. Throughput: 0: 9684.5, 1: 9837.0. Samples: 8783588. Policy #0 lag: (min: 29.0, avg: 36.3, max: 61.0) [2023-12-26 15:29:31,063][104569] Avg episode reward: [(0, '8987.315'), (1, '8707.769')] [2023-12-26 15:29:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000017232_4415488.pth... [2023-12-26 15:29:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000017184_4399104.pth... [2023-12-26 15:29:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000016080_4120576.pth [2023-12-26 15:29:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000016064_4112384.pth [2023-12-26 15:29:31,297][105620] Updated weights for policy 1, policy_version 17190 (0.0007) [2023-12-26 15:29:31,371][105620] Updated weights for policy 1, policy_version 17200 (0.0008) [2023-12-26 15:29:31,424][105620] Updated weights for policy 1, policy_version 17210 (0.0008) [2023-12-26 15:29:31,569][105692] Updated weights for policy 0, policy_version 17242 (0.0010) [2023-12-26 15:29:31,626][105692] Updated weights for policy 0, policy_version 17252 (0.0011) [2023-12-26 15:29:31,684][105692] Updated weights for policy 0, policy_version 17262 (0.0010) [2023-12-26 15:29:32,084][105620] Updated weights for policy 1, policy_version 17220 (0.0007) [2023-12-26 15:29:32,139][105620] Updated weights for policy 1, policy_version 17230 (0.0006) [2023-12-26 15:29:32,196][105620] Updated weights for policy 1, policy_version 17240 (0.0005) [2023-12-26 15:29:32,305][105692] Updated weights for policy 0, policy_version 17272 (0.0009) [2023-12-26 15:29:32,367][105692] Updated weights for policy 0, policy_version 17282 (0.0009) [2023-12-26 15:29:32,427][105692] Updated weights for policy 0, policy_version 17292 (0.0010) [2023-12-26 15:29:32,865][105620] Updated weights for policy 1, policy_version 17250 (0.0008) [2023-12-26 15:29:32,919][105620] Updated weights for policy 1, policy_version 17260 (0.0006) [2023-12-26 15:29:32,969][105620] Updated weights for policy 1, policy_version 17270 (0.0005) [2023-12-26 15:29:33,027][105620] Updated weights for policy 1, policy_version 17280 (0.0006) [2023-12-26 15:29:33,192][105692] Updated weights for policy 0, policy_version 17302 (0.0010) [2023-12-26 15:29:33,260][105692] Updated weights for policy 0, policy_version 17312 (0.0005) [2023-12-26 15:29:33,328][105692] Updated weights for policy 0, policy_version 17322 (0.0005) [2023-12-26 15:29:33,550][105620] Updated weights for policy 1, policy_version 17290 (0.0008) [2023-12-26 15:29:33,607][105620] Updated weights for policy 1, policy_version 17300 (0.0005) [2023-12-26 15:29:33,658][105620] Updated weights for policy 1, policy_version 17310 (0.0005) [2023-12-26 15:29:33,935][105692] Updated weights for policy 0, policy_version 17332 (0.0005) [2023-12-26 15:29:33,997][105692] Updated weights for policy 0, policy_version 17342 (0.0009) [2023-12-26 15:29:34,054][105692] Updated weights for policy 0, policy_version 17352 (0.0007) [2023-12-26 15:29:34,327][105620] Updated weights for policy 1, policy_version 17320 (0.0008) [2023-12-26 15:29:34,392][105620] Updated weights for policy 1, policy_version 17330 (0.0009) [2023-12-26 15:29:34,458][105620] Updated weights for policy 1, policy_version 17340 (0.0009) [2023-12-26 15:29:34,764][105692] Updated weights for policy 0, policy_version 17362 (0.0006) [2023-12-26 15:29:34,830][105692] Updated weights for policy 0, policy_version 17372 (0.0005) [2023-12-26 15:29:34,898][105692] Updated weights for policy 0, policy_version 17382 (0.0005) [2023-12-26 15:29:34,953][105692] Updated weights for policy 0, policy_version 17392 (0.0005) [2023-12-26 15:29:35,202][105620] Updated weights for policy 1, policy_version 17350 (0.0009) [2023-12-26 15:29:35,266][105620] Updated weights for policy 1, policy_version 17360 (0.0008) [2023-12-26 15:29:35,332][105620] Updated weights for policy 1, policy_version 17370 (0.0005) [2023-12-26 15:29:35,462][105692] Updated weights for policy 0, policy_version 17402 (0.0005) [2023-12-26 15:29:35,525][105692] Updated weights for policy 0, policy_version 17412 (0.0005) [2023-12-26 15:29:35,579][105692] Updated weights for policy 0, policy_version 17422 (0.0005) [2023-12-26 15:29:35,923][105620] Updated weights for policy 1, policy_version 17380 (0.0007) [2023-12-26 15:29:35,981][105620] Updated weights for policy 1, policy_version 17390 (0.0010) [2023-12-26 15:29:36,033][105620] Updated weights for policy 1, policy_version 17400 (0.0009) [2023-12-26 15:29:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 8912896. Throughput: 0: 9876.8, 1: 9784.2. Samples: 8903744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:29:36,062][104569] Avg episode reward: [(0, '8987.180'), (1, '8337.834')] [2023-12-26 15:29:36,143][105692] Updated weights for policy 0, policy_version 17432 (0.0009) [2023-12-26 15:29:36,206][105692] Updated weights for policy 0, policy_version 17442 (0.0010) [2023-12-26 15:29:36,265][105692] Updated weights for policy 0, policy_version 17452 (0.0011) [2023-12-26 15:29:36,684][105620] Updated weights for policy 1, policy_version 17410 (0.0005) [2023-12-26 15:29:36,735][105620] Updated weights for policy 1, policy_version 17420 (0.0008) [2023-12-26 15:29:36,783][105620] Updated weights for policy 1, policy_version 17430 (0.0009) [2023-12-26 15:29:36,829][105620] Updated weights for policy 1, policy_version 17440 (0.0008) [2023-12-26 15:29:36,961][105692] Updated weights for policy 0, policy_version 17462 (0.0007) [2023-12-26 15:29:37,025][105692] Updated weights for policy 0, policy_version 17472 (0.0009) [2023-12-26 15:29:37,076][105692] Updated weights for policy 0, policy_version 17482 (0.0010) [2023-12-26 15:29:37,472][105620] Updated weights for policy 1, policy_version 17450 (0.0010) [2023-12-26 15:29:37,532][105620] Updated weights for policy 1, policy_version 17460 (0.0011) [2023-12-26 15:29:37,596][105620] Updated weights for policy 1, policy_version 17470 (0.0011) [2023-12-26 15:29:37,751][105692] Updated weights for policy 0, policy_version 17492 (0.0010) [2023-12-26 15:29:37,815][105692] Updated weights for policy 0, policy_version 17502 (0.0010) [2023-12-26 15:29:37,866][105692] Updated weights for policy 0, policy_version 17512 (0.0010) [2023-12-26 15:29:38,343][105620] Updated weights for policy 1, policy_version 17480 (0.0010) [2023-12-26 15:29:38,405][105620] Updated weights for policy 1, policy_version 17490 (0.0008) [2023-12-26 15:29:38,466][105620] Updated weights for policy 1, policy_version 17500 (0.0008) [2023-12-26 15:29:38,602][105692] Updated weights for policy 0, policy_version 17522 (0.0010) [2023-12-26 15:29:38,661][105692] Updated weights for policy 0, policy_version 17532 (0.0010) [2023-12-26 15:29:38,715][105692] Updated weights for policy 0, policy_version 17542 (0.0010) [2023-12-26 15:29:38,763][105692] Updated weights for policy 0, policy_version 17552 (0.0010) [2023-12-26 15:29:39,174][105620] Updated weights for policy 1, policy_version 17510 (0.0010) [2023-12-26 15:29:39,244][105620] Updated weights for policy 1, policy_version 17520 (0.0009) [2023-12-26 15:29:39,309][105620] Updated weights for policy 1, policy_version 17530 (0.0007) [2023-12-26 15:29:39,572][105692] Updated weights for policy 0, policy_version 17562 (0.0008) [2023-12-26 15:29:39,623][105692] Updated weights for policy 0, policy_version 17572 (0.0009) [2023-12-26 15:29:39,677][105692] Updated weights for policy 0, policy_version 17582 (0.0010) [2023-12-26 15:29:39,958][105620] Updated weights for policy 1, policy_version 17540 (0.0009) [2023-12-26 15:29:40,020][105620] Updated weights for policy 1, policy_version 17550 (0.0009) [2023-12-26 15:29:40,075][105620] Updated weights for policy 1, policy_version 17560 (0.0011) [2023-12-26 15:29:40,396][105692] Updated weights for policy 0, policy_version 17592 (0.0009) [2023-12-26 15:29:40,457][105692] Updated weights for policy 0, policy_version 17602 (0.0008) [2023-12-26 15:29:40,522][105692] Updated weights for policy 0, policy_version 17612 (0.0009) [2023-12-26 15:29:40,846][105620] Updated weights for policy 1, policy_version 17570 (0.0009) [2023-12-26 15:29:40,908][105620] Updated weights for policy 1, policy_version 17580 (0.0010) [2023-12-26 15:29:40,973][105620] Updated weights for policy 1, policy_version 17590 (0.0010) [2023-12-26 15:29:41,042][105620] Updated weights for policy 1, policy_version 17600 (0.0011) [2023-12-26 15:29:41,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 9019392. Throughput: 0: 9969.2, 1: 9861.5. Samples: 9026492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:29:41,062][104569] Avg episode reward: [(0, '8801.153'), (1, '8708.441')] [2023-12-26 15:29:41,230][105692] Updated weights for policy 0, policy_version 17622 (0.0009) [2023-12-26 15:29:41,292][105692] Updated weights for policy 0, policy_version 17632 (0.0009) [2023-12-26 15:29:41,356][105692] Updated weights for policy 0, policy_version 17642 (0.0009) [2023-12-26 15:29:41,859][105620] Updated weights for policy 1, policy_version 17610 (0.0010) [2023-12-26 15:29:41,915][105620] Updated weights for policy 1, policy_version 17620 (0.0010) [2023-12-26 15:29:41,974][105620] Updated weights for policy 1, policy_version 17630 (0.0010) [2023-12-26 15:29:42,142][105692] Updated weights for policy 0, policy_version 17652 (0.0008) [2023-12-26 15:29:42,193][105692] Updated weights for policy 0, policy_version 17662 (0.0005) [2023-12-26 15:29:42,240][105692] Updated weights for policy 0, policy_version 17672 (0.0005) [2023-12-26 15:29:42,700][105620] Updated weights for policy 1, policy_version 17640 (0.0010) [2023-12-26 15:29:42,759][105620] Updated weights for policy 1, policy_version 17650 (0.0011) [2023-12-26 15:29:42,815][105620] Updated weights for policy 1, policy_version 17660 (0.0009) [2023-12-26 15:29:42,918][105692] Updated weights for policy 0, policy_version 17682 (0.0010) [2023-12-26 15:29:42,977][105692] Updated weights for policy 0, policy_version 17692 (0.0011) [2023-12-26 15:29:43,030][105692] Updated weights for policy 0, policy_version 17702 (0.0011) [2023-12-26 15:29:43,081][105692] Updated weights for policy 0, policy_version 17712 (0.0010) [2023-12-26 15:29:43,459][105620] Updated weights for policy 1, policy_version 17670 (0.0007) [2023-12-26 15:29:43,505][105620] Updated weights for policy 1, policy_version 17680 (0.0006) [2023-12-26 15:29:43,557][105620] Updated weights for policy 1, policy_version 17690 (0.0005) [2023-12-26 15:29:43,725][105692] Updated weights for policy 0, policy_version 17722 (0.0009) [2023-12-26 15:29:43,775][105692] Updated weights for policy 0, policy_version 17732 (0.0009) [2023-12-26 15:29:43,829][105692] Updated weights for policy 0, policy_version 17742 (0.0008) [2023-12-26 15:29:44,224][105620] Updated weights for policy 1, policy_version 17700 (0.0006) [2023-12-26 15:29:44,283][105620] Updated weights for policy 1, policy_version 17710 (0.0008) [2023-12-26 15:29:44,345][105620] Updated weights for policy 1, policy_version 17720 (0.0008) [2023-12-26 15:29:44,565][105692] Updated weights for policy 0, policy_version 17752 (0.0007) [2023-12-26 15:29:44,626][105692] Updated weights for policy 0, policy_version 17762 (0.0009) [2023-12-26 15:29:44,678][105692] Updated weights for policy 0, policy_version 17772 (0.0009) [2023-12-26 15:29:45,050][105620] Updated weights for policy 1, policy_version 17730 (0.0009) [2023-12-26 15:29:45,105][105620] Updated weights for policy 1, policy_version 17740 (0.0008) [2023-12-26 15:29:45,161][105620] Updated weights for policy 1, policy_version 17750 (0.0010) [2023-12-26 15:29:45,220][105620] Updated weights for policy 1, policy_version 17760 (0.0009) [2023-12-26 15:29:45,454][105692] Updated weights for policy 0, policy_version 17782 (0.0009) [2023-12-26 15:29:45,526][105692] Updated weights for policy 0, policy_version 17792 (0.0009) [2023-12-26 15:29:45,585][105692] Updated weights for policy 0, policy_version 17802 (0.0007) [2023-12-26 15:29:45,996][105620] Updated weights for policy 1, policy_version 17770 (0.0005) [2023-12-26 15:29:46,034][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000004 [2023-12-26 15:29:46,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 9117696. Throughput: 0: 9939.9, 1: 9844.7. Samples: 9085388. Policy #0 lag: (min: 10.0, avg: 26.5, max: 42.0) [2023-12-26 15:29:46,063][104569] Avg episode reward: [(0, '8984.794'), (1, '8801.530')] [2023-12-26 15:29:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000017808_4562944.pth... [2023-12-26 15:29:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000017776_4554752.pth... [2023-12-26 15:29:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000016656_4268032.pth [2023-12-26 15:29:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000016608_4251648.pth [2023-12-26 15:29:46,287][105692] Updated weights for policy 0, policy_version 17812 (0.0007) [2023-12-26 15:29:46,337][105692] Updated weights for policy 0, policy_version 17822 (0.0009) [2023-12-26 15:29:46,386][105692] Updated weights for policy 0, policy_version 17832 (0.0008) [2023-12-26 15:29:46,609][105620] Updated weights for policy 1, policy_version 17780 (0.0007) [2023-12-26 15:29:46,665][105620] Updated weights for policy 1, policy_version 17790 (0.0005) [2023-12-26 15:29:46,728][105620] Updated weights for policy 1, policy_version 17800 (0.0005) [2023-12-26 15:29:47,252][105692] Updated weights for policy 0, policy_version 17842 (0.0009) [2023-12-26 15:29:47,309][105692] Updated weights for policy 0, policy_version 17852 (0.0009) [2023-12-26 15:29:47,327][105620] Updated weights for policy 1, policy_version 17810 (0.0006) [2023-12-26 15:29:47,355][105692] Updated weights for policy 0, policy_version 17862 (0.0008) [2023-12-26 15:29:47,389][105620] Updated weights for policy 1, policy_version 17820 (0.0007) [2023-12-26 15:29:47,403][105692] Updated weights for policy 0, policy_version 17872 (0.0006) [2023-12-26 15:29:47,453][105620] Updated weights for policy 1, policy_version 17830 (0.0009) [2023-12-26 15:29:47,512][105620] Updated weights for policy 1, policy_version 17840 (0.0009) [2023-12-26 15:29:48,146][105692] Updated weights for policy 0, policy_version 17882 (0.0009) [2023-12-26 15:29:48,204][105692] Updated weights for policy 0, policy_version 17892 (0.0008) [2023-12-26 15:29:48,263][105692] Updated weights for policy 0, policy_version 17902 (0.0007) [2023-12-26 15:29:48,273][105620] Updated weights for policy 1, policy_version 17850 (0.0006) [2023-12-26 15:29:48,334][105620] Updated weights for policy 1, policy_version 17860 (0.0008) [2023-12-26 15:29:48,392][105620] Updated weights for policy 1, policy_version 17870 (0.0009) [2023-12-26 15:29:49,030][105692] Updated weights for policy 0, policy_version 17912 (0.0008) [2023-12-26 15:29:49,092][105692] Updated weights for policy 0, policy_version 17922 (0.0009) [2023-12-26 15:29:49,147][105692] Updated weights for policy 0, policy_version 17932 (0.0008) [2023-12-26 15:29:49,173][105620] Updated weights for policy 1, policy_version 17880 (0.0007) [2023-12-26 15:29:49,229][105620] Updated weights for policy 1, policy_version 17890 (0.0008) [2023-12-26 15:29:49,292][105620] Updated weights for policy 1, policy_version 17900 (0.0009) [2023-12-26 15:29:49,926][105692] Updated weights for policy 0, policy_version 17942 (0.0008) [2023-12-26 15:29:49,993][105692] Updated weights for policy 0, policy_version 17952 (0.0008) [2023-12-26 15:29:50,053][105692] Updated weights for policy 0, policy_version 17962 (0.0008) [2023-12-26 15:29:50,059][105620] Updated weights for policy 1, policy_version 17910 (0.0008) [2023-12-26 15:29:50,121][105620] Updated weights for policy 1, policy_version 17920 (0.0007) [2023-12-26 15:29:50,176][105620] Updated weights for policy 1, policy_version 17930 (0.0009) [2023-12-26 15:29:50,823][105692] Updated weights for policy 0, policy_version 17972 (0.0008) [2023-12-26 15:29:50,882][105692] Updated weights for policy 0, policy_version 17982 (0.0009) [2023-12-26 15:29:50,938][105692] Updated weights for policy 0, policy_version 17992 (0.0008) [2023-12-26 15:29:50,952][105620] Updated weights for policy 1, policy_version 17940 (0.0008) [2023-12-26 15:29:51,014][105620] Updated weights for policy 1, policy_version 17950 (0.0006) [2023-12-26 15:29:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 9207808. Throughput: 0: 9821.1, 1: 9874.7. Samples: 9199812. Policy #0 lag: (min: 10.0, avg: 26.5, max: 42.0) [2023-12-26 15:29:51,062][104569] Avg episode reward: [(0, '9169.445'), (1, '8152.475')] [2023-12-26 15:29:51,081][105620] Updated weights for policy 1, policy_version 17960 (0.0008) [2023-12-26 15:29:51,687][105692] Updated weights for policy 0, policy_version 18002 (0.0009) [2023-12-26 15:29:51,755][105692] Updated weights for policy 0, policy_version 18012 (0.0009) [2023-12-26 15:29:51,816][105620] Updated weights for policy 1, policy_version 17970 (0.0008) [2023-12-26 15:29:51,819][105692] Updated weights for policy 0, policy_version 18022 (0.0007) [2023-12-26 15:29:51,870][105620] Updated weights for policy 1, policy_version 17980 (0.0009) [2023-12-26 15:29:51,872][105692] Updated weights for policy 0, policy_version 18032 (0.0006) [2023-12-26 15:29:51,918][105620] Updated weights for policy 1, policy_version 17991 (0.0006) [2023-12-26 15:29:52,609][105620] Updated weights for policy 1, policy_version 18001 (0.0006) [2023-12-26 15:29:52,671][105620] Updated weights for policy 1, policy_version 18011 (0.0010) [2023-12-26 15:29:52,701][105692] Updated weights for policy 0, policy_version 18042 (0.0006) [2023-12-26 15:29:52,730][105620] Updated weights for policy 1, policy_version 18021 (0.0010) [2023-12-26 15:29:52,760][105692] Updated weights for policy 0, policy_version 18052 (0.0008) [2023-12-26 15:29:52,789][105620] Updated weights for policy 1, policy_version 18031 (0.0010) [2023-12-26 15:29:52,825][105692] Updated weights for policy 0, policy_version 18062 (0.0007) [2023-12-26 15:29:53,387][105620] Updated weights for policy 1, policy_version 18041 (0.0010) [2023-12-26 15:29:53,442][105620] Updated weights for policy 1, policy_version 18051 (0.0008) [2023-12-26 15:29:53,465][105692] Updated weights for policy 0, policy_version 18072 (0.0007) [2023-12-26 15:29:53,496][105620] Updated weights for policy 1, policy_version 18061 (0.0011) [2023-12-26 15:29:53,528][105692] Updated weights for policy 0, policy_version 18082 (0.0006) [2023-12-26 15:29:53,596][105692] Updated weights for policy 0, policy_version 18092 (0.0008) [2023-12-26 15:29:54,221][105620] Updated weights for policy 1, policy_version 18071 (0.0007) [2023-12-26 15:29:54,269][105620] Updated weights for policy 1, policy_version 18081 (0.0005) [2023-12-26 15:29:54,295][105692] Updated weights for policy 0, policy_version 18102 (0.0009) [2023-12-26 15:29:54,326][105620] Updated weights for policy 1, policy_version 18091 (0.0005) [2023-12-26 15:29:54,359][105692] Updated weights for policy 0, policy_version 18112 (0.0005) [2023-12-26 15:29:54,419][105692] Updated weights for policy 0, policy_version 18122 (0.0005) [2023-12-26 15:29:54,883][105620] Updated weights for policy 1, policy_version 18101 (0.0006) [2023-12-26 15:29:54,947][105620] Updated weights for policy 1, policy_version 18111 (0.0007) [2023-12-26 15:29:55,007][105620] Updated weights for policy 1, policy_version 18121 (0.0005) [2023-12-26 15:29:55,031][105692] Updated weights for policy 0, policy_version 18132 (0.0007) [2023-12-26 15:29:55,082][105692] Updated weights for policy 0, policy_version 18142 (0.0009) [2023-12-26 15:29:55,135][105692] Updated weights for policy 0, policy_version 18152 (0.0008) [2023-12-26 15:29:55,647][105620] Updated weights for policy 1, policy_version 18131 (0.0006) [2023-12-26 15:29:55,693][105620] Updated weights for policy 1, policy_version 18141 (0.0005) [2023-12-26 15:29:55,739][105620] Updated weights for policy 1, policy_version 18151 (0.0005) [2023-12-26 15:29:55,822][105692] Updated weights for policy 0, policy_version 18162 (0.0009) [2023-12-26 15:29:55,875][105692] Updated weights for policy 0, policy_version 18172 (0.0010) [2023-12-26 15:29:55,931][105692] Updated weights for policy 0, policy_version 18182 (0.0010) [2023-12-26 15:29:55,987][105692] Updated weights for policy 0, policy_version 18192 (0.0010) [2023-12-26 15:29:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 9314304. Throughput: 0: 9751.3, 1: 10042.6. Samples: 9319024. Policy #0 lag: (min: 10.0, avg: 26.5, max: 42.0) [2023-12-26 15:29:56,062][104569] Avg episode reward: [(0, '9260.622'), (1, '8335.095')] [2023-12-26 15:29:56,317][105620] Updated weights for policy 1, policy_version 18161 (0.0006) [2023-12-26 15:29:56,376][105620] Updated weights for policy 1, policy_version 18171 (0.0008) [2023-12-26 15:29:56,428][105620] Updated weights for policy 1, policy_version 18181 (0.0005) [2023-12-26 15:29:56,493][105620] Updated weights for policy 1, policy_version 18191 (0.0006) [2023-12-26 15:29:56,742][105692] Updated weights for policy 0, policy_version 18202 (0.0005) [2023-12-26 15:29:56,794][105692] Updated weights for policy 0, policy_version 18212 (0.0005) [2023-12-26 15:29:56,848][105692] Updated weights for policy 0, policy_version 18222 (0.0006) [2023-12-26 15:29:57,079][105620] Updated weights for policy 1, policy_version 18201 (0.0010) [2023-12-26 15:29:57,136][105620] Updated weights for policy 1, policy_version 18211 (0.0011) [2023-12-26 15:29:57,194][105620] Updated weights for policy 1, policy_version 18221 (0.0010) [2023-12-26 15:29:57,532][105692] Updated weights for policy 0, policy_version 18232 (0.0008) [2023-12-26 15:29:57,585][105692] Updated weights for policy 0, policy_version 18242 (0.0010) [2023-12-26 15:29:57,643][105692] Updated weights for policy 0, policy_version 18252 (0.0010) [2023-12-26 15:29:57,866][105620] Updated weights for policy 1, policy_version 18231 (0.0010) [2023-12-26 15:29:57,924][105620] Updated weights for policy 1, policy_version 18241 (0.0010) [2023-12-26 15:29:57,981][105620] Updated weights for policy 1, policy_version 18251 (0.0010) [2023-12-26 15:29:58,345][105692] Updated weights for policy 0, policy_version 18262 (0.0010) [2023-12-26 15:29:58,414][105692] Updated weights for policy 0, policy_version 18272 (0.0008) [2023-12-26 15:29:58,476][105692] Updated weights for policy 0, policy_version 18282 (0.0008) [2023-12-26 15:29:58,690][105620] Updated weights for policy 1, policy_version 18261 (0.0010) [2023-12-26 15:29:58,757][105620] Updated weights for policy 1, policy_version 18271 (0.0009) [2023-12-26 15:29:58,832][105620] Updated weights for policy 1, policy_version 18281 (0.0009) [2023-12-26 15:29:59,330][105692] Updated weights for policy 0, policy_version 18292 (0.0008) [2023-12-26 15:29:59,393][105692] Updated weights for policy 0, policy_version 18302 (0.0009) [2023-12-26 15:29:59,457][105692] Updated weights for policy 0, policy_version 18312 (0.0009) [2023-12-26 15:29:59,661][105620] Updated weights for policy 1, policy_version 18291 (0.0010) [2023-12-26 15:29:59,709][105620] Updated weights for policy 1, policy_version 18301 (0.0008) [2023-12-26 15:29:59,755][105620] Updated weights for policy 1, policy_version 18311 (0.0009) [2023-12-26 15:30:00,223][105692] Updated weights for policy 0, policy_version 18322 (0.0009) [2023-12-26 15:30:00,276][105692] Updated weights for policy 0, policy_version 18332 (0.0008) [2023-12-26 15:30:00,335][105692] Updated weights for policy 0, policy_version 18342 (0.0008) [2023-12-26 15:30:00,393][105692] Updated weights for policy 0, policy_version 18352 (0.0007) [2023-12-26 15:30:00,495][105620] Updated weights for policy 1, policy_version 18321 (0.0006) [2023-12-26 15:30:00,553][105620] Updated weights for policy 1, policy_version 18331 (0.0005) [2023-12-26 15:30:00,601][105620] Updated weights for policy 1, policy_version 18341 (0.0005) [2023-12-26 15:30:00,663][105620] Updated weights for policy 1, policy_version 18351 (0.0009) [2023-12-26 15:30:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 9404416. Throughput: 0: 9791.4, 1: 10023.2. Samples: 9379172. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-26 15:30:01,063][104569] Avg episode reward: [(0, '9074.476'), (1, '8704.883')] [2023-12-26 15:30:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000018352_4702208.pth... [2023-12-26 15:30:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000018352_4702208.pth... [2023-12-26 15:30:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000017232_4415488.pth [2023-12-26 15:30:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000017184_4399104.pth [2023-12-26 15:30:01,180][105692] Updated weights for policy 0, policy_version 18362 (0.0008) [2023-12-26 15:30:01,237][105692] Updated weights for policy 0, policy_version 18372 (0.0009) [2023-12-26 15:30:01,301][105692] Updated weights for policy 0, policy_version 18382 (0.0009) [2023-12-26 15:30:01,317][105620] Updated weights for policy 1, policy_version 18361 (0.0007) [2023-12-26 15:30:01,383][105620] Updated weights for policy 1, policy_version 18371 (0.0010) [2023-12-26 15:30:01,453][105620] Updated weights for policy 1, policy_version 18381 (0.0010) [2023-12-26 15:30:02,014][105692] Updated weights for policy 0, policy_version 18392 (0.0009) [2023-12-26 15:30:02,078][105692] Updated weights for policy 0, policy_version 18402 (0.0009) [2023-12-26 15:30:02,138][105692] Updated weights for policy 0, policy_version 18412 (0.0008) [2023-12-26 15:30:02,169][105620] Updated weights for policy 1, policy_version 18391 (0.0008) [2023-12-26 15:30:02,236][105620] Updated weights for policy 1, policy_version 18401 (0.0008) [2023-12-26 15:30:02,291][105620] Updated weights for policy 1, policy_version 18411 (0.0009) [2023-12-26 15:30:02,846][105692] Updated weights for policy 0, policy_version 18422 (0.0008) [2023-12-26 15:30:02,909][105692] Updated weights for policy 0, policy_version 18432 (0.0007) [2023-12-26 15:30:02,966][105692] Updated weights for policy 0, policy_version 18442 (0.0009) [2023-12-26 15:30:03,017][105620] Updated weights for policy 1, policy_version 18421 (0.0009) [2023-12-26 15:30:03,067][105620] Updated weights for policy 1, policy_version 18431 (0.0009) [2023-12-26 15:30:03,122][105620] Updated weights for policy 1, policy_version 18441 (0.0010) [2023-12-26 15:30:03,532][105692] Updated weights for policy 0, policy_version 18452 (0.0007) [2023-12-26 15:30:03,592][105692] Updated weights for policy 0, policy_version 18462 (0.0006) [2023-12-26 15:30:03,650][105692] Updated weights for policy 0, policy_version 18472 (0.0008) [2023-12-26 15:30:03,817][105620] Updated weights for policy 1, policy_version 18453 (0.0011) [2023-12-26 15:30:03,878][105620] Updated weights for policy 1, policy_version 18463 (0.0011) [2023-12-26 15:30:03,942][105620] Updated weights for policy 1, policy_version 18473 (0.0011) [2023-12-26 15:30:04,300][105692] Updated weights for policy 0, policy_version 18482 (0.0008) [2023-12-26 15:30:04,366][105692] Updated weights for policy 0, policy_version 18492 (0.0009) [2023-12-26 15:30:04,426][105692] Updated weights for policy 0, policy_version 18502 (0.0008) [2023-12-26 15:30:04,482][105692] Updated weights for policy 0, policy_version 18512 (0.0007) [2023-12-26 15:30:04,710][105620] Updated weights for policy 1, policy_version 18483 (0.0011) [2023-12-26 15:30:04,772][105620] Updated weights for policy 1, policy_version 18493 (0.0011) [2023-12-26 15:30:04,836][105620] Updated weights for policy 1, policy_version 18503 (0.0011) [2023-12-26 15:30:05,233][105692] Updated weights for policy 0, policy_version 18522 (0.0010) [2023-12-26 15:30:05,295][105692] Updated weights for policy 0, policy_version 18532 (0.0010) [2023-12-26 15:30:05,356][105692] Updated weights for policy 0, policy_version 18542 (0.0010) [2023-12-26 15:30:05,524][105620] Updated weights for policy 1, policy_version 18513 (0.0010) [2023-12-26 15:30:05,574][105620] Updated weights for policy 1, policy_version 18523 (0.0006) [2023-12-26 15:30:05,620][105620] Updated weights for policy 1, policy_version 18533 (0.0005) [2023-12-26 15:30:05,681][105620] Updated weights for policy 1, policy_version 18543 (0.0006) [2023-12-26 15:30:06,058][105692] Updated weights for policy 0, policy_version 18552 (0.0010) [2023-12-26 15:30:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 9502720. Throughput: 0: 9757.9, 1: 9981.0. Samples: 9494644. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-26 15:30:06,062][104569] Avg episode reward: [(0, '8980.403'), (1, '8798.415')] [2023-12-26 15:30:06,120][105692] Updated weights for policy 0, policy_version 18562 (0.0009) [2023-12-26 15:30:06,176][105692] Updated weights for policy 0, policy_version 18572 (0.0007) [2023-12-26 15:30:06,378][105620] Updated weights for policy 1, policy_version 18553 (0.0009) [2023-12-26 15:30:06,443][105620] Updated weights for policy 1, policy_version 18563 (0.0008) [2023-12-26 15:30:06,513][105620] Updated weights for policy 1, policy_version 18573 (0.0006) [2023-12-26 15:30:06,868][105692] Updated weights for policy 0, policy_version 18582 (0.0008) [2023-12-26 15:30:06,924][105692] Updated weights for policy 0, policy_version 18592 (0.0011) [2023-12-26 15:30:06,973][105692] Updated weights for policy 0, policy_version 18602 (0.0010) [2023-12-26 15:30:07,065][105620] Updated weights for policy 1, policy_version 18583 (0.0006) [2023-12-26 15:30:07,123][105620] Updated weights for policy 1, policy_version 18593 (0.0007) [2023-12-26 15:30:07,183][105620] Updated weights for policy 1, policy_version 18603 (0.0010) [2023-12-26 15:30:07,543][105692] Updated weights for policy 0, policy_version 18612 (0.0008) [2023-12-26 15:30:07,599][105692] Updated weights for policy 0, policy_version 18622 (0.0006) [2023-12-26 15:30:07,657][105692] Updated weights for policy 0, policy_version 18632 (0.0010) [2023-12-26 15:30:07,929][105620] Updated weights for policy 1, policy_version 18613 (0.0009) [2023-12-26 15:30:07,987][105620] Updated weights for policy 1, policy_version 18623 (0.0008) [2023-12-26 15:30:08,049][105620] Updated weights for policy 1, policy_version 18633 (0.0009) [2023-12-26 15:30:08,301][105692] Updated weights for policy 0, policy_version 18642 (0.0010) [2023-12-26 15:30:08,360][105692] Updated weights for policy 0, policy_version 18652 (0.0009) [2023-12-26 15:30:08,417][105692] Updated weights for policy 0, policy_version 18662 (0.0010) [2023-12-26 15:30:08,478][105692] Updated weights for policy 0, policy_version 18672 (0.0009) [2023-12-26 15:30:08,765][105620] Updated weights for policy 1, policy_version 18643 (0.0009) [2023-12-26 15:30:08,824][105620] Updated weights for policy 1, policy_version 18653 (0.0010) [2023-12-26 15:30:08,888][105620] Updated weights for policy 1, policy_version 18663 (0.0010) [2023-12-26 15:30:09,229][105692] Updated weights for policy 0, policy_version 18682 (0.0008) [2023-12-26 15:30:09,290][105692] Updated weights for policy 0, policy_version 18692 (0.0008) [2023-12-26 15:30:09,353][105692] Updated weights for policy 0, policy_version 18702 (0.0008) [2023-12-26 15:30:09,619][105620] Updated weights for policy 1, policy_version 18673 (0.0010) [2023-12-26 15:30:09,682][105620] Updated weights for policy 1, policy_version 18683 (0.0011) [2023-12-26 15:30:09,741][105620] Updated weights for policy 1, policy_version 18693 (0.0010) [2023-12-26 15:30:09,800][105620] Updated weights for policy 1, policy_version 18703 (0.0010) [2023-12-26 15:30:10,118][105692] Updated weights for policy 0, policy_version 18712 (0.0010) [2023-12-26 15:30:10,178][105692] Updated weights for policy 0, policy_version 18722 (0.0006) [2023-12-26 15:30:10,242][105692] Updated weights for policy 0, policy_version 18732 (0.0007) [2023-12-26 15:30:10,522][105620] Updated weights for policy 1, policy_version 18713 (0.0009) [2023-12-26 15:30:10,587][105620] Updated weights for policy 1, policy_version 18723 (0.0006) [2023-12-26 15:30:10,645][105620] Updated weights for policy 1, policy_version 18733 (0.0010) [2023-12-26 15:30:10,950][105692] Updated weights for policy 0, policy_version 18742 (0.0008) [2023-12-26 15:30:11,004][105692] Updated weights for policy 0, policy_version 18752 (0.0005) [2023-12-26 15:30:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 9601024. Throughput: 0: 9843.1, 1: 9987.9. Samples: 9614044. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-26 15:30:11,063][104569] Avg episode reward: [(0, '9164.169'), (1, '9076.894')] [2023-12-26 15:30:11,063][105586] Saving new best policy, reward=9076.894! [2023-12-26 15:30:11,074][105692] Updated weights for policy 0, policy_version 18762 (0.0009) [2023-12-26 15:30:11,347][105620] Updated weights for policy 1, policy_version 18743 (0.0012) [2023-12-26 15:30:11,416][105620] Updated weights for policy 1, policy_version 18753 (0.0010) [2023-12-26 15:30:11,478][105620] Updated weights for policy 1, policy_version 18763 (0.0011) [2023-12-26 15:30:11,844][105692] Updated weights for policy 0, policy_version 18772 (0.0009) [2023-12-26 15:30:11,903][105692] Updated weights for policy 0, policy_version 18782 (0.0010) [2023-12-26 15:30:11,961][105692] Updated weights for policy 0, policy_version 18792 (0.0010) [2023-12-26 15:30:12,262][105620] Updated weights for policy 1, policy_version 18773 (0.0011) [2023-12-26 15:30:12,326][105620] Updated weights for policy 1, policy_version 18783 (0.0011) [2023-12-26 15:30:12,396][105620] Updated weights for policy 1, policy_version 18793 (0.0011) [2023-12-26 15:30:12,673][105692] Updated weights for policy 0, policy_version 18802 (0.0010) [2023-12-26 15:30:12,739][105692] Updated weights for policy 0, policy_version 18812 (0.0007) [2023-12-26 15:30:12,801][105692] Updated weights for policy 0, policy_version 18822 (0.0006) [2023-12-26 15:30:12,863][105692] Updated weights for policy 0, policy_version 18832 (0.0007) [2023-12-26 15:30:13,086][105620] Updated weights for policy 1, policy_version 18803 (0.0010) [2023-12-26 15:30:13,147][105620] Updated weights for policy 1, policy_version 18813 (0.0008) [2023-12-26 15:30:13,194][105620] Updated weights for policy 1, policy_version 18823 (0.0008) [2023-12-26 15:30:13,497][105692] Updated weights for policy 0, policy_version 18842 (0.0010) [2023-12-26 15:30:13,541][105692] Updated weights for policy 0, policy_version 18852 (0.0010) [2023-12-26 15:30:13,597][105692] Updated weights for policy 0, policy_version 18862 (0.0010) [2023-12-26 15:30:13,943][105620] Updated weights for policy 1, policy_version 18833 (0.0008) [2023-12-26 15:30:13,994][105620] Updated weights for policy 1, policy_version 18843 (0.0008) [2023-12-26 15:30:14,053][105620] Updated weights for policy 1, policy_version 18853 (0.0008) [2023-12-26 15:30:14,112][105620] Updated weights for policy 1, policy_version 18863 (0.0008) [2023-12-26 15:30:14,361][105692] Updated weights for policy 0, policy_version 18872 (0.0010) [2023-12-26 15:30:14,416][105692] Updated weights for policy 0, policy_version 18882 (0.0010) [2023-12-26 15:30:14,469][105692] Updated weights for policy 0, policy_version 18892 (0.0010) [2023-12-26 15:30:14,891][105620] Updated weights for policy 1, policy_version 18873 (0.0008) [2023-12-26 15:30:14,942][105620] Updated weights for policy 1, policy_version 18883 (0.0008) [2023-12-26 15:30:15,002][105620] Updated weights for policy 1, policy_version 18893 (0.0008) [2023-12-26 15:30:15,262][105692] Updated weights for policy 0, policy_version 18902 (0.0011) [2023-12-26 15:30:15,322][105692] Updated weights for policy 0, policy_version 18912 (0.0011) [2023-12-26 15:30:15,374][105692] Updated weights for policy 0, policy_version 18922 (0.0010) [2023-12-26 15:30:15,745][105620] Updated weights for policy 1, policy_version 18903 (0.0007) [2023-12-26 15:30:15,807][105620] Updated weights for policy 1, policy_version 18913 (0.0006) [2023-12-26 15:30:15,874][105620] Updated weights for policy 1, policy_version 18923 (0.0005) [2023-12-26 15:30:15,977][105692] Updated weights for policy 0, policy_version 18932 (0.0008) [2023-12-26 15:30:16,024][105692] Updated weights for policy 0, policy_version 18942 (0.0005) [2023-12-26 15:30:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 9699328. Throughput: 0: 9824.6, 1: 9904.2. Samples: 9671380. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 15:30:16,062][104569] Avg episode reward: [(0, '8976.778'), (1, '8891.929')] [2023-12-26 15:30:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000018928_4849664.pth... [2023-12-26 15:30:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000017776_4554752.pth [2023-12-26 15:30:16,076][105692] Updated weights for policy 0, policy_version 18952 (0.0006) [2023-12-26 15:30:16,115][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000018960_4857856.pth... [2023-12-26 15:30:16,118][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000017808_4562944.pth [2023-12-26 15:30:16,575][105620] Updated weights for policy 1, policy_version 18933 (0.0007) [2023-12-26 15:30:16,625][105620] Updated weights for policy 1, policy_version 18943 (0.0009) [2023-12-26 15:30:16,671][105620] Updated weights for policy 1, policy_version 18953 (0.0007) [2023-12-26 15:30:16,718][105692] Updated weights for policy 0, policy_version 18962 (0.0010) [2023-12-26 15:30:16,780][105692] Updated weights for policy 0, policy_version 18972 (0.0010) [2023-12-26 15:30:16,842][105692] Updated weights for policy 0, policy_version 18982 (0.0010) [2023-12-26 15:30:16,902][105692] Updated weights for policy 0, policy_version 18992 (0.0009) [2023-12-26 15:30:17,247][105620] Updated weights for policy 1, policy_version 18963 (0.0006) [2023-12-26 15:30:17,311][105620] Updated weights for policy 1, policy_version 18973 (0.0007) [2023-12-26 15:30:17,361][105620] Updated weights for policy 1, policy_version 18983 (0.0007) [2023-12-26 15:30:17,664][105692] Updated weights for policy 0, policy_version 19002 (0.0009) [2023-12-26 15:30:17,718][105692] Updated weights for policy 0, policy_version 19012 (0.0008) [2023-12-26 15:30:17,768][105692] Updated weights for policy 0, policy_version 19022 (0.0005) [2023-12-26 15:30:18,131][105620] Updated weights for policy 1, policy_version 18993 (0.0008) [2023-12-26 15:30:18,194][105620] Updated weights for policy 1, policy_version 19003 (0.0009) [2023-12-26 15:30:18,252][105620] Updated weights for policy 1, policy_version 19013 (0.0009) [2023-12-26 15:30:18,309][105620] Updated weights for policy 1, policy_version 19023 (0.0007) [2023-12-26 15:30:18,441][105692] Updated weights for policy 0, policy_version 19032 (0.0009) [2023-12-26 15:30:18,490][105692] Updated weights for policy 0, policy_version 19042 (0.0009) [2023-12-26 15:30:18,545][105692] Updated weights for policy 0, policy_version 19052 (0.0009) [2023-12-26 15:30:18,948][105620] Updated weights for policy 1, policy_version 19033 (0.0007) [2023-12-26 15:30:18,999][105620] Updated weights for policy 1, policy_version 19043 (0.0009) [2023-12-26 15:30:19,052][105620] Updated weights for policy 1, policy_version 19054 (0.0010) [2023-12-26 15:30:19,257][105692] Updated weights for policy 0, policy_version 19062 (0.0010) [2023-12-26 15:30:19,325][105692] Updated weights for policy 0, policy_version 19072 (0.0009) [2023-12-26 15:30:19,391][105692] Updated weights for policy 0, policy_version 19082 (0.0008) [2023-12-26 15:30:19,851][105620] Updated weights for policy 1, policy_version 19064 (0.0010) [2023-12-26 15:30:19,917][105620] Updated weights for policy 1, policy_version 19074 (0.0008) [2023-12-26 15:30:19,981][105620] Updated weights for policy 1, policy_version 19084 (0.0007) [2023-12-26 15:30:20,142][105692] Updated weights for policy 0, policy_version 19092 (0.0008) [2023-12-26 15:30:20,198][105692] Updated weights for policy 0, policy_version 19102 (0.0008) [2023-12-26 15:30:20,255][105692] Updated weights for policy 0, policy_version 19112 (0.0008) [2023-12-26 15:30:20,667][105620] Updated weights for policy 1, policy_version 19094 (0.0011) [2023-12-26 15:30:20,733][105620] Updated weights for policy 1, policy_version 19104 (0.0009) [2023-12-26 15:30:20,796][105620] Updated weights for policy 1, policy_version 19114 (0.0011) [2023-12-26 15:30:21,031][105692] Updated weights for policy 0, policy_version 19122 (0.0008) [2023-12-26 15:30:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 9797632. Throughput: 0: 9767.8, 1: 9906.1. Samples: 9789072. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 15:30:21,063][104569] Avg episode reward: [(0, '8881.621'), (1, '8519.821')] [2023-12-26 15:30:21,101][105692] Updated weights for policy 0, policy_version 19132 (0.0011) [2023-12-26 15:30:21,169][105692] Updated weights for policy 0, policy_version 19142 (0.0011) [2023-12-26 15:30:21,246][105692] Updated weights for policy 0, policy_version 19152 (0.0010) [2023-12-26 15:30:21,551][105620] Updated weights for policy 1, policy_version 19124 (0.0010) [2023-12-26 15:30:21,608][105620] Updated weights for policy 1, policy_version 19134 (0.0009) [2023-12-26 15:30:21,677][105620] Updated weights for policy 1, policy_version 19144 (0.0009) [2023-12-26 15:30:21,962][105692] Updated weights for policy 0, policy_version 19162 (0.0011) [2023-12-26 15:30:22,015][105692] Updated weights for policy 0, policy_version 19172 (0.0011) [2023-12-26 15:30:22,078][105692] Updated weights for policy 0, policy_version 19182 (0.0007) [2023-12-26 15:30:22,394][105620] Updated weights for policy 1, policy_version 19154 (0.0007) [2023-12-26 15:30:22,454][105620] Updated weights for policy 1, policy_version 19164 (0.0008) [2023-12-26 15:30:22,510][105620] Updated weights for policy 1, policy_version 19174 (0.0009) [2023-12-26 15:30:22,569][105620] Updated weights for policy 1, policy_version 19184 (0.0007) [2023-12-26 15:30:22,818][105692] Updated weights for policy 0, policy_version 19192 (0.0010) [2023-12-26 15:30:22,872][105692] Updated weights for policy 0, policy_version 19202 (0.0007) [2023-12-26 15:30:22,939][105692] Updated weights for policy 0, policy_version 19212 (0.0011) [2023-12-26 15:30:23,339][105620] Updated weights for policy 1, policy_version 19195 (0.0009) [2023-12-26 15:30:23,395][105620] Updated weights for policy 1, policy_version 19205 (0.0008) [2023-12-26 15:30:23,445][105620] Updated weights for policy 1, policy_version 19215 (0.0008) [2023-12-26 15:30:23,630][105692] Updated weights for policy 0, policy_version 19222 (0.0011) [2023-12-26 15:30:23,678][105692] Updated weights for policy 0, policy_version 19232 (0.0010) [2023-12-26 15:30:23,723][105692] Updated weights for policy 0, policy_version 19242 (0.0010) [2023-12-26 15:30:24,182][105620] Updated weights for policy 1, policy_version 19225 (0.0008) [2023-12-26 15:30:24,237][105620] Updated weights for policy 1, policy_version 19235 (0.0008) [2023-12-26 15:30:24,288][105620] Updated weights for policy 1, policy_version 19245 (0.0008) [2023-12-26 15:30:24,370][105692] Updated weights for policy 0, policy_version 19252 (0.0009) [2023-12-26 15:30:24,432][105692] Updated weights for policy 0, policy_version 19262 (0.0008) [2023-12-26 15:30:24,503][105692] Updated weights for policy 0, policy_version 19272 (0.0007) [2023-12-26 15:30:24,930][105620] Updated weights for policy 1, policy_version 19255 (0.0006) [2023-12-26 15:30:24,984][105620] Updated weights for policy 1, policy_version 19265 (0.0005) [2023-12-26 15:30:25,048][105620] Updated weights for policy 1, policy_version 19275 (0.0008) [2023-12-26 15:30:25,230][105692] Updated weights for policy 0, policy_version 19282 (0.0009) [2023-12-26 15:30:25,294][105692] Updated weights for policy 0, policy_version 19292 (0.0008) [2023-12-26 15:30:25,352][105692] Updated weights for policy 0, policy_version 19302 (0.0010) [2023-12-26 15:30:25,412][105692] Updated weights for policy 0, policy_version 19312 (0.0009) [2023-12-26 15:30:25,627][105620] Updated weights for policy 1, policy_version 19285 (0.0007) [2023-12-26 15:30:25,673][105620] Updated weights for policy 1, policy_version 19295 (0.0005) [2023-12-26 15:30:25,728][105620] Updated weights for policy 1, policy_version 19305 (0.0005) [2023-12-26 15:30:26,054][105692] Updated weights for policy 0, policy_version 19322 (0.0005) [2023-12-26 15:30:26,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19797.2, 300 sec: 19605.3). Total num frames: 9895936. Throughput: 0: 9659.4, 1: 9898.0. Samples: 9906580. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-26 15:30:26,063][104569] Avg episode reward: [(0, '9161.577'), (1, '8519.461')] [2023-12-26 15:30:26,107][105692] Updated weights for policy 0, policy_version 19332 (0.0005) [2023-12-26 15:30:26,158][105692] Updated weights for policy 0, policy_version 19342 (0.0005) [2023-12-26 15:30:26,346][105620] Updated weights for policy 1, policy_version 19315 (0.0006) [2023-12-26 15:30:26,391][105620] Updated weights for policy 1, policy_version 19325 (0.0008) [2023-12-26 15:30:26,442][105620] Updated weights for policy 1, policy_version 19335 (0.0009) [2023-12-26 15:30:26,753][105692] Updated weights for policy 0, policy_version 19352 (0.0005) [2023-12-26 15:30:26,803][105692] Updated weights for policy 0, policy_version 19362 (0.0006) [2023-12-26 15:30:26,848][105692] Updated weights for policy 0, policy_version 19372 (0.0008) [2023-12-26 15:30:27,197][105620] Updated weights for policy 1, policy_version 19346 (0.0010) [2023-12-26 15:30:27,258][105620] Updated weights for policy 1, policy_version 19356 (0.0010) [2023-12-26 15:30:27,316][105620] Updated weights for policy 1, policy_version 19366 (0.0010) [2023-12-26 15:30:27,360][105620] Updated weights for policy 1, policy_version 19376 (0.0010) [2023-12-26 15:30:27,421][105692] Updated weights for policy 0, policy_version 19382 (0.0007) [2023-12-26 15:30:27,482][105692] Updated weights for policy 0, policy_version 19392 (0.0008) [2023-12-26 15:30:27,531][105692] Updated weights for policy 0, policy_version 19402 (0.0008) [2023-12-26 15:30:27,967][105620] Updated weights for policy 1, policy_version 19386 (0.0010) [2023-12-26 15:30:28,015][105620] Updated weights for policy 1, policy_version 19396 (0.0010) [2023-12-26 15:30:28,065][105620] Updated weights for policy 1, policy_version 19406 (0.0010) [2023-12-26 15:30:28,073][105692] Updated weights for policy 0, policy_version 19412 (0.0007) [2023-12-26 15:30:28,120][105692] Updated weights for policy 0, policy_version 19422 (0.0005) [2023-12-26 15:30:28,162][105692] Updated weights for policy 0, policy_version 19432 (0.0005) [2023-12-26 15:30:28,811][105620] Updated weights for policy 1, policy_version 19416 (0.0010) [2023-12-26 15:30:28,867][105620] Updated weights for policy 1, policy_version 19426 (0.0010) [2023-12-26 15:30:28,895][105692] Updated weights for policy 0, policy_version 19442 (0.0006) [2023-12-26 15:30:28,924][105620] Updated weights for policy 1, policy_version 19436 (0.0010) [2023-12-26 15:30:28,960][105692] Updated weights for policy 0, policy_version 19452 (0.0006) [2023-12-26 15:30:29,019][105692] Updated weights for policy 0, policy_version 19462 (0.0009) [2023-12-26 15:30:29,079][105692] Updated weights for policy 0, policy_version 19472 (0.0009) [2023-12-26 15:30:29,710][105620] Updated weights for policy 1, policy_version 19446 (0.0009) [2023-12-26 15:30:29,757][105620] Updated weights for policy 1, policy_version 19456 (0.0006) [2023-12-26 15:30:29,805][105620] Updated weights for policy 1, policy_version 19466 (0.0008) [2023-12-26 15:30:29,852][105692] Updated weights for policy 0, policy_version 19482 (0.0010) [2023-12-26 15:30:29,916][105692] Updated weights for policy 0, policy_version 19492 (0.0011) [2023-12-26 15:30:29,972][105692] Updated weights for policy 0, policy_version 19502 (0.0010) [2023-12-26 15:30:30,451][105620] Updated weights for policy 1, policy_version 19476 (0.0008) [2023-12-26 15:30:30,521][105620] Updated weights for policy 1, policy_version 19486 (0.0009) [2023-12-26 15:30:30,573][105620] Updated weights for policy 1, policy_version 19496 (0.0011) [2023-12-26 15:30:30,677][105692] Updated weights for policy 0, policy_version 19512 (0.0008) [2023-12-26 15:30:30,725][105692] Updated weights for policy 0, policy_version 19522 (0.0010) [2023-12-26 15:30:30,773][105692] Updated weights for policy 0, policy_version 19532 (0.0010) [2023-12-26 15:30:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 10002432. Throughput: 0: 9779.8, 1: 9915.4. Samples: 9971668. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-26 15:30:31,062][104569] Avg episode reward: [(0, '9162.909'), (1, '8799.336')] [2023-12-26 15:30:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000019536_5005312.pth... [2023-12-26 15:30:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000019504_4997120.pth... [2023-12-26 15:30:31,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000018352_4702208.pth [2023-12-26 15:30:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000018352_4702208.pth [2023-12-26 15:30:31,264][105620] Updated weights for policy 1, policy_version 19506 (0.0010) [2023-12-26 15:30:31,326][105620] Updated weights for policy 1, policy_version 19516 (0.0008) [2023-12-26 15:30:31,397][105620] Updated weights for policy 1, policy_version 19526 (0.0012) [2023-12-26 15:30:31,466][105620] Updated weights for policy 1, policy_version 19536 (0.0007) [2023-12-26 15:30:31,484][105692] Updated weights for policy 0, policy_version 19542 (0.0007) [2023-12-26 15:30:31,542][105692] Updated weights for policy 0, policy_version 19552 (0.0005) [2023-12-26 15:30:31,603][105692] Updated weights for policy 0, policy_version 19562 (0.0007) [2023-12-26 15:30:32,193][105620] Updated weights for policy 1, policy_version 19546 (0.0008) [2023-12-26 15:30:32,259][105620] Updated weights for policy 1, policy_version 19556 (0.0006) [2023-12-26 15:30:32,291][105692] Updated weights for policy 0, policy_version 19572 (0.0010) [2023-12-26 15:30:32,316][105620] Updated weights for policy 1, policy_version 19566 (0.0006) [2023-12-26 15:30:32,358][105692] Updated weights for policy 0, policy_version 19582 (0.0010) [2023-12-26 15:30:32,422][105692] Updated weights for policy 0, policy_version 19592 (0.0010) [2023-12-26 15:30:32,976][105620] Updated weights for policy 1, policy_version 19576 (0.0010) [2023-12-26 15:30:33,039][105620] Updated weights for policy 1, policy_version 19586 (0.0011) [2023-12-26 15:30:33,105][105620] Updated weights for policy 1, policy_version 19596 (0.0011) [2023-12-26 15:30:33,171][105692] Updated weights for policy 0, policy_version 19602 (0.0011) [2023-12-26 15:30:33,237][105692] Updated weights for policy 0, policy_version 19612 (0.0011) [2023-12-26 15:30:33,303][105692] Updated weights for policy 0, policy_version 19622 (0.0011) [2023-12-26 15:30:33,361][105692] Updated weights for policy 0, policy_version 19632 (0.0010) [2023-12-26 15:30:33,839][105620] Updated weights for policy 1, policy_version 19606 (0.0008) [2023-12-26 15:30:33,894][105620] Updated weights for policy 1, policy_version 19616 (0.0006) [2023-12-26 15:30:33,966][105620] Updated weights for policy 1, policy_version 19626 (0.0006) [2023-12-26 15:30:34,009][105692] Updated weights for policy 0, policy_version 19642 (0.0011) [2023-12-26 15:30:34,063][105692] Updated weights for policy 0, policy_version 19652 (0.0006) [2023-12-26 15:30:34,119][105692] Updated weights for policy 0, policy_version 19662 (0.0009) [2023-12-26 15:30:34,704][105620] Updated weights for policy 1, policy_version 19636 (0.0007) [2023-12-26 15:30:34,756][105620] Updated weights for policy 1, policy_version 19646 (0.0008) [2023-12-26 15:30:34,811][105620] Updated weights for policy 1, policy_version 19656 (0.0008) [2023-12-26 15:30:34,864][105692] Updated weights for policy 0, policy_version 19672 (0.0009) [2023-12-26 15:30:34,913][105692] Updated weights for policy 0, policy_version 19682 (0.0010) [2023-12-26 15:30:34,966][105692] Updated weights for policy 0, policy_version 19693 (0.0009) [2023-12-26 15:30:35,473][105620] Updated weights for policy 1, policy_version 19666 (0.0009) [2023-12-26 15:30:35,555][105620] Updated weights for policy 1, policy_version 19676 (0.0010) [2023-12-26 15:30:35,611][105620] Updated weights for policy 1, policy_version 19686 (0.0010) [2023-12-26 15:30:35,618][105692] Updated weights for policy 0, policy_version 19703 (0.0007) [2023-12-26 15:30:35,672][105620] Updated weights for policy 1, policy_version 19696 (0.0010) [2023-12-26 15:30:35,674][105692] Updated weights for policy 0, policy_version 19713 (0.0009) [2023-12-26 15:30:35,731][105692] Updated weights for policy 0, policy_version 19723 (0.0006) [2023-12-26 15:30:36,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.2, 300 sec: 19633.0). Total num frames: 10100736. Throughput: 0: 9837.8, 1: 9888.5. Samples: 10087504. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-26 15:30:36,063][104569] Avg episode reward: [(0, '9254.858'), (1, '8706.057')] [2023-12-26 15:30:36,281][105620] Updated weights for policy 1, policy_version 19706 (0.0010) [2023-12-26 15:30:36,335][105620] Updated weights for policy 1, policy_version 19716 (0.0011) [2023-12-26 15:30:36,399][105620] Updated weights for policy 1, policy_version 19726 (0.0011) [2023-12-26 15:30:36,563][105692] Updated weights for policy 0, policy_version 19733 (0.0011) [2023-12-26 15:30:36,626][105692] Updated weights for policy 0, policy_version 19743 (0.0010) [2023-12-26 15:30:36,695][105692] Updated weights for policy 0, policy_version 19753 (0.0008) [2023-12-26 15:30:37,057][105620] Updated weights for policy 1, policy_version 19736 (0.0007) [2023-12-26 15:30:37,126][105620] Updated weights for policy 1, policy_version 19746 (0.0007) [2023-12-26 15:30:37,187][105620] Updated weights for policy 1, policy_version 19756 (0.0010) [2023-12-26 15:30:37,419][105692] Updated weights for policy 0, policy_version 19763 (0.0009) [2023-12-26 15:30:37,476][105692] Updated weights for policy 0, policy_version 19774 (0.0010) [2023-12-26 15:30:37,530][105692] Updated weights for policy 0, policy_version 19784 (0.0008) [2023-12-26 15:30:37,864][105620] Updated weights for policy 1, policy_version 19766 (0.0009) [2023-12-26 15:30:37,928][105620] Updated weights for policy 1, policy_version 19776 (0.0009) [2023-12-26 15:30:37,986][105620] Updated weights for policy 1, policy_version 19786 (0.0009) [2023-12-26 15:30:38,324][105692] Updated weights for policy 0, policy_version 19794 (0.0009) [2023-12-26 15:30:38,390][105692] Updated weights for policy 0, policy_version 19804 (0.0009) [2023-12-26 15:30:38,449][105692] Updated weights for policy 0, policy_version 19814 (0.0009) [2023-12-26 15:30:38,512][105692] Updated weights for policy 0, policy_version 19824 (0.0009) [2023-12-26 15:30:38,761][105620] Updated weights for policy 1, policy_version 19796 (0.0010) [2023-12-26 15:30:38,827][105620] Updated weights for policy 1, policy_version 19806 (0.0008) [2023-12-26 15:30:38,887][105620] Updated weights for policy 1, policy_version 19816 (0.0005) [2023-12-26 15:30:39,251][105692] Updated weights for policy 0, policy_version 19834 (0.0008) [2023-12-26 15:30:39,308][105692] Updated weights for policy 0, policy_version 19844 (0.0008) [2023-12-26 15:30:39,368][105692] Updated weights for policy 0, policy_version 19854 (0.0010) [2023-12-26 15:30:39,604][105620] Updated weights for policy 1, policy_version 19826 (0.0007) [2023-12-26 15:30:39,665][105620] Updated weights for policy 1, policy_version 19836 (0.0009) [2023-12-26 15:30:39,731][105620] Updated weights for policy 1, policy_version 19846 (0.0010) [2023-12-26 15:30:39,795][105620] Updated weights for policy 1, policy_version 19856 (0.0010) [2023-12-26 15:30:40,027][105692] Updated weights for policy 0, policy_version 19864 (0.0010) [2023-12-26 15:30:40,094][105692] Updated weights for policy 0, policy_version 19874 (0.0010) [2023-12-26 15:30:40,160][105692] Updated weights for policy 0, policy_version 19884 (0.0010) [2023-12-26 15:30:40,603][105620] Updated weights for policy 1, policy_version 19866 (0.0009) [2023-12-26 15:30:40,656][105620] Updated weights for policy 1, policy_version 19876 (0.0009) [2023-12-26 15:30:40,710][105620] Updated weights for policy 1, policy_version 19886 (0.0009) [2023-12-26 15:30:40,927][105692] Updated weights for policy 0, policy_version 19894 (0.0009) [2023-12-26 15:30:40,990][105692] Updated weights for policy 0, policy_version 19904 (0.0009) [2023-12-26 15:30:41,058][105692] Updated weights for policy 0, policy_version 19914 (0.0009) [2023-12-26 15:30:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 10190848. Throughput: 0: 9816.9, 1: 9818.3. Samples: 10202608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:30:41,063][104569] Avg episode reward: [(0, '9347.756'), (1, '8335.552')] [2023-12-26 15:30:41,093][105585] Saving new best policy, reward=9347.756! [2023-12-26 15:30:41,525][105620] Updated weights for policy 1, policy_version 19896 (0.0007) [2023-12-26 15:30:41,581][105620] Updated weights for policy 1, policy_version 19906 (0.0007) [2023-12-26 15:30:41,644][105620] Updated weights for policy 1, policy_version 19916 (0.0007) [2023-12-26 15:30:41,855][105692] Updated weights for policy 0, policy_version 19924 (0.0008) [2023-12-26 15:30:41,911][105692] Updated weights for policy 0, policy_version 19934 (0.0009) [2023-12-26 15:30:41,968][105692] Updated weights for policy 0, policy_version 19944 (0.0008) [2023-12-26 15:30:42,333][105620] Updated weights for policy 1, policy_version 19926 (0.0007) [2023-12-26 15:30:42,396][105620] Updated weights for policy 1, policy_version 19936 (0.0009) [2023-12-26 15:30:42,464][105620] Updated weights for policy 1, policy_version 19946 (0.0008) [2023-12-26 15:30:42,755][105692] Updated weights for policy 0, policy_version 19954 (0.0008) [2023-12-26 15:30:42,822][105692] Updated weights for policy 0, policy_version 19964 (0.0006) [2023-12-26 15:30:42,891][105692] Updated weights for policy 0, policy_version 19974 (0.0006) [2023-12-26 15:30:42,953][105692] Updated weights for policy 0, policy_version 19984 (0.0006) [2023-12-26 15:30:43,201][105620] Updated weights for policy 1, policy_version 19956 (0.0009) [2023-12-26 15:30:43,253][105620] Updated weights for policy 1, policy_version 19966 (0.0010) [2023-12-26 15:30:43,318][105620] Updated weights for policy 1, policy_version 19976 (0.0010) [2023-12-26 15:30:43,453][105692] Updated weights for policy 0, policy_version 19994 (0.0011) [2023-12-26 15:30:43,510][105692] Updated weights for policy 0, policy_version 20004 (0.0006) [2023-12-26 15:30:43,574][105692] Updated weights for policy 0, policy_version 20014 (0.0006) [2023-12-26 15:30:44,066][105620] Updated weights for policy 1, policy_version 19986 (0.0010) [2023-12-26 15:30:44,113][105692] Updated weights for policy 0, policy_version 20024 (0.0010) [2023-12-26 15:30:44,122][105620] Updated weights for policy 1, policy_version 19996 (0.0010) [2023-12-26 15:30:44,170][105692] Updated weights for policy 0, policy_version 20034 (0.0010) [2023-12-26 15:30:44,180][105620] Updated weights for policy 1, policy_version 20006 (0.0010) [2023-12-26 15:30:44,228][105620] Updated weights for policy 1, policy_version 20016 (0.0010) [2023-12-26 15:30:44,231][105692] Updated weights for policy 0, policy_version 20044 (0.0005) [2023-12-26 15:30:44,815][105692] Updated weights for policy 0, policy_version 20054 (0.0008) [2023-12-26 15:30:44,879][105692] Updated weights for policy 0, policy_version 20064 (0.0010) [2023-12-26 15:30:44,921][105620] Updated weights for policy 1, policy_version 20026 (0.0008) [2023-12-26 15:30:44,936][105692] Updated weights for policy 0, policy_version 20074 (0.0009) [2023-12-26 15:30:44,978][105620] Updated weights for policy 1, policy_version 20036 (0.0009) [2023-12-26 15:30:45,039][105620] Updated weights for policy 1, policy_version 20046 (0.0009) [2023-12-26 15:30:45,769][105692] Updated weights for policy 0, policy_version 20084 (0.0009) [2023-12-26 15:30:45,817][105620] Updated weights for policy 1, policy_version 20056 (0.0010) [2023-12-26 15:30:45,821][105692] Updated weights for policy 0, policy_version 20094 (0.0010) [2023-12-26 15:30:45,865][105692] Updated weights for policy 0, policy_version 20104 (0.0010) [2023-12-26 15:30:45,878][105620] Updated weights for policy 1, policy_version 20066 (0.0010) [2023-12-26 15:30:45,943][105620] Updated weights for policy 1, policy_version 20076 (0.0010) [2023-12-26 15:30:46,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 10297344. Throughput: 0: 9815.4, 1: 9765.7. Samples: 10260324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:30:46,063][104569] Avg episode reward: [(0, '9163.355'), (1, '8244.201')] [2023-12-26 15:30:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000020080_5144576.pth... [2023-12-26 15:30:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000020112_5152768.pth... [2023-12-26 15:30:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000018928_4849664.pth [2023-12-26 15:30:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000018960_4857856.pth [2023-12-26 15:30:46,500][105692] Updated weights for policy 0, policy_version 20114 (0.0007) [2023-12-26 15:30:46,557][105692] Updated weights for policy 0, policy_version 20124 (0.0008) [2023-12-26 15:30:46,616][105692] Updated weights for policy 0, policy_version 20134 (0.0008) [2023-12-26 15:30:46,669][105620] Updated weights for policy 1, policy_version 20086 (0.0010) [2023-12-26 15:30:46,672][105692] Updated weights for policy 0, policy_version 20144 (0.0006) [2023-12-26 15:30:46,731][105620] Updated weights for policy 1, policy_version 20096 (0.0010) [2023-12-26 15:30:46,779][105620] Updated weights for policy 1, policy_version 20106 (0.0010) [2023-12-26 15:30:47,411][105692] Updated weights for policy 0, policy_version 20154 (0.0006) [2023-12-26 15:30:47,466][105692] Updated weights for policy 0, policy_version 20164 (0.0008) [2023-12-26 15:30:47,517][105692] Updated weights for policy 0, policy_version 20174 (0.0008) [2023-12-26 15:30:47,523][105620] Updated weights for policy 1, policy_version 20116 (0.0010) [2023-12-26 15:30:47,584][105620] Updated weights for policy 1, policy_version 20126 (0.0010) [2023-12-26 15:30:47,636][105620] Updated weights for policy 1, policy_version 20136 (0.0010) [2023-12-26 15:30:48,285][105692] Updated weights for policy 0, policy_version 20184 (0.0008) [2023-12-26 15:30:48,299][105620] Updated weights for policy 1, policy_version 20146 (0.0010) [2023-12-26 15:30:48,348][105692] Updated weights for policy 0, policy_version 20194 (0.0008) [2023-12-26 15:30:48,358][105620] Updated weights for policy 1, policy_version 20156 (0.0010) [2023-12-26 15:30:48,416][105692] Updated weights for policy 0, policy_version 20204 (0.0009) [2023-12-26 15:30:48,420][105620] Updated weights for policy 1, policy_version 20166 (0.0010) [2023-12-26 15:30:48,486][105620] Updated weights for policy 1, policy_version 20176 (0.0010) [2023-12-26 15:30:49,156][105692] Updated weights for policy 0, policy_version 20214 (0.0006) [2023-12-26 15:30:49,210][105692] Updated weights for policy 0, policy_version 20224 (0.0007) [2023-12-26 15:30:49,227][105620] Updated weights for policy 1, policy_version 20186 (0.0010) [2023-12-26 15:30:49,284][105692] Updated weights for policy 0, policy_version 20234 (0.0009) [2023-12-26 15:30:49,289][105620] Updated weights for policy 1, policy_version 20196 (0.0006) [2023-12-26 15:30:49,348][105620] Updated weights for policy 1, policy_version 20206 (0.0009) [2023-12-26 15:30:50,027][105620] Updated weights for policy 1, policy_version 20216 (0.0010) [2023-12-26 15:30:50,027][105692] Updated weights for policy 0, policy_version 20244 (0.0009) [2023-12-26 15:30:50,083][105692] Updated weights for policy 0, policy_version 20254 (0.0010) [2023-12-26 15:30:50,089][105620] Updated weights for policy 1, policy_version 20226 (0.0010) [2023-12-26 15:30:50,140][105692] Updated weights for policy 0, policy_version 20264 (0.0011) [2023-12-26 15:30:50,152][105620] Updated weights for policy 1, policy_version 20236 (0.0010) [2023-12-26 15:30:50,755][105620] Updated weights for policy 1, policy_version 20246 (0.0010) [2023-12-26 15:30:50,806][105692] Updated weights for policy 0, policy_version 20274 (0.0009) [2023-12-26 15:30:50,807][105620] Updated weights for policy 1, policy_version 20256 (0.0010) [2023-12-26 15:30:50,858][105620] Updated weights for policy 1, policy_version 20266 (0.0010) [2023-12-26 15:30:50,870][105692] Updated weights for policy 0, policy_version 20284 (0.0009) [2023-12-26 15:30:50,928][105692] Updated weights for policy 0, policy_version 20294 (0.0011) [2023-12-26 15:30:50,987][105692] Updated weights for policy 0, policy_version 20304 (0.0011) [2023-12-26 15:30:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 10395648. Throughput: 0: 9857.6, 1: 9767.5. Samples: 10377776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:30:51,063][104569] Avg episode reward: [(0, '9162.965'), (1, '8245.783')] [2023-12-26 15:30:51,641][105620] Updated weights for policy 1, policy_version 20276 (0.0009) [2023-12-26 15:30:51,701][105692] Updated weights for policy 0, policy_version 20314 (0.0008) [2023-12-26 15:30:51,716][105620] Updated weights for policy 1, policy_version 20286 (0.0009) [2023-12-26 15:30:51,760][105692] Updated weights for policy 0, policy_version 20324 (0.0008) [2023-12-26 15:30:51,778][105620] Updated weights for policy 1, policy_version 20296 (0.0008) [2023-12-26 15:30:51,817][105692] Updated weights for policy 0, policy_version 20334 (0.0007) [2023-12-26 15:30:52,538][105692] Updated weights for policy 0, policy_version 20344 (0.0008) [2023-12-26 15:30:52,540][105620] Updated weights for policy 1, policy_version 20306 (0.0011) [2023-12-26 15:30:52,586][105620] Updated weights for policy 1, policy_version 20316 (0.0010) [2023-12-26 15:30:52,603][105692] Updated weights for policy 0, policy_version 20354 (0.0006) [2023-12-26 15:30:52,642][105620] Updated weights for policy 1, policy_version 20326 (0.0011) [2023-12-26 15:30:52,674][105692] Updated weights for policy 0, policy_version 20364 (0.0007) [2023-12-26 15:30:52,691][105620] Updated weights for policy 1, policy_version 20336 (0.0010) [2023-12-26 15:30:53,303][105692] Updated weights for policy 0, policy_version 20374 (0.0007) [2023-12-26 15:30:53,322][105620] Updated weights for policy 1, policy_version 20346 (0.0010) [2023-12-26 15:30:53,358][105692] Updated weights for policy 0, policy_version 20384 (0.0010) [2023-12-26 15:30:53,380][105620] Updated weights for policy 1, policy_version 20356 (0.0010) [2023-12-26 15:30:53,416][105692] Updated weights for policy 0, policy_version 20394 (0.0010) [2023-12-26 15:30:53,434][105620] Updated weights for policy 1, policy_version 20366 (0.0010) [2023-12-26 15:30:54,132][105692] Updated weights for policy 0, policy_version 20404 (0.0010) [2023-12-26 15:30:54,150][105620] Updated weights for policy 1, policy_version 20376 (0.0010) [2023-12-26 15:30:54,194][105692] Updated weights for policy 0, policy_version 20414 (0.0008) [2023-12-26 15:30:54,211][105620] Updated weights for policy 1, policy_version 20386 (0.0010) [2023-12-26 15:30:54,260][105692] Updated weights for policy 0, policy_version 20424 (0.0008) [2023-12-26 15:30:54,273][105620] Updated weights for policy 1, policy_version 20396 (0.0010) [2023-12-26 15:30:54,915][105620] Updated weights for policy 1, policy_version 20406 (0.0008) [2023-12-26 15:30:54,966][105692] Updated weights for policy 0, policy_version 20434 (0.0007) [2023-12-26 15:30:54,975][105620] Updated weights for policy 1, policy_version 20416 (0.0010) [2023-12-26 15:30:55,029][105692] Updated weights for policy 0, policy_version 20444 (0.0008) [2023-12-26 15:30:55,035][105620] Updated weights for policy 1, policy_version 20426 (0.0010) [2023-12-26 15:30:55,088][105692] Updated weights for policy 0, policy_version 20454 (0.0010) [2023-12-26 15:30:55,150][105692] Updated weights for policy 0, policy_version 20464 (0.0010) [2023-12-26 15:30:55,617][105620] Updated weights for policy 1, policy_version 20436 (0.0008) [2023-12-26 15:30:55,685][105620] Updated weights for policy 1, policy_version 20446 (0.0005) [2023-12-26 15:30:55,747][105620] Updated weights for policy 1, policy_version 20456 (0.0006) [2023-12-26 15:30:55,803][105692] Updated weights for policy 0, policy_version 20474 (0.0009) [2023-12-26 15:30:55,851][105692] Updated weights for policy 0, policy_version 20484 (0.0010) [2023-12-26 15:30:55,895][105692] Updated weights for policy 0, policy_version 20494 (0.0006) [2023-12-26 15:30:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 10493952. Throughput: 0: 9852.6, 1: 9805.6. Samples: 10498660. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-12-26 15:30:56,062][104569] Avg episode reward: [(0, '9162.130'), (1, '8523.219')] [2023-12-26 15:30:56,264][105620] Updated weights for policy 1, policy_version 20466 (0.0007) [2023-12-26 15:30:56,312][105620] Updated weights for policy 1, policy_version 20476 (0.0008) [2023-12-26 15:30:56,363][105620] Updated weights for policy 1, policy_version 20486 (0.0008) [2023-12-26 15:30:56,411][105620] Updated weights for policy 1, policy_version 20496 (0.0009) [2023-12-26 15:30:56,579][105692] Updated weights for policy 0, policy_version 20504 (0.0005) [2023-12-26 15:30:56,632][105692] Updated weights for policy 0, policy_version 20514 (0.0005) [2023-12-26 15:30:56,678][105692] Updated weights for policy 0, policy_version 20524 (0.0005) [2023-12-26 15:30:57,224][105692] Updated weights for policy 0, policy_version 20534 (0.0008) [2023-12-26 15:30:57,264][105620] Updated weights for policy 1, policy_version 20506 (0.0005) [2023-12-26 15:30:57,281][105692] Updated weights for policy 0, policy_version 20544 (0.0009) [2023-12-26 15:30:57,327][105620] Updated weights for policy 1, policy_version 20516 (0.0007) [2023-12-26 15:30:57,346][105692] Updated weights for policy 0, policy_version 20554 (0.0007) [2023-12-26 15:30:57,390][105620] Updated weights for policy 1, policy_version 20526 (0.0010) [2023-12-26 15:30:58,004][105620] Updated weights for policy 1, policy_version 20536 (0.0009) [2023-12-26 15:30:58,066][105620] Updated weights for policy 1, policy_version 20546 (0.0009) [2023-12-26 15:30:58,118][105692] Updated weights for policy 0, policy_version 20564 (0.0006) [2023-12-26 15:30:58,133][105620] Updated weights for policy 1, policy_version 20556 (0.0007) [2023-12-26 15:30:58,181][105692] Updated weights for policy 0, policy_version 20574 (0.0008) [2023-12-26 15:30:58,237][105692] Updated weights for policy 0, policy_version 20584 (0.0009) [2023-12-26 15:30:58,971][105620] Updated weights for policy 1, policy_version 20566 (0.0008) [2023-12-26 15:30:59,039][105620] Updated weights for policy 1, policy_version 20576 (0.0008) [2023-12-26 15:30:59,098][105620] Updated weights for policy 1, policy_version 20586 (0.0009) [2023-12-26 15:30:59,130][105692] Updated weights for policy 0, policy_version 20594 (0.0009) [2023-12-26 15:30:59,194][105692] Updated weights for policy 0, policy_version 20604 (0.0010) [2023-12-26 15:30:59,259][105692] Updated weights for policy 0, policy_version 20614 (0.0008) [2023-12-26 15:30:59,308][105692] Updated weights for policy 0, policy_version 20624 (0.0005) [2023-12-26 15:30:59,799][105620] Updated weights for policy 1, policy_version 20596 (0.0009) [2023-12-26 15:30:59,867][105620] Updated weights for policy 1, policy_version 20606 (0.0008) [2023-12-26 15:30:59,935][105620] Updated weights for policy 1, policy_version 20616 (0.0008) [2023-12-26 15:31:00,114][105692] Updated weights for policy 0, policy_version 20634 (0.0008) [2023-12-26 15:31:00,177][105692] Updated weights for policy 0, policy_version 20644 (0.0008) [2023-12-26 15:31:00,233][105692] Updated weights for policy 0, policy_version 20654 (0.0008) [2023-12-26 15:31:00,556][105620] Updated weights for policy 1, policy_version 20626 (0.0009) [2023-12-26 15:31:00,608][105620] Updated weights for policy 1, policy_version 20636 (0.0005) [2023-12-26 15:31:00,658][105620] Updated weights for policy 1, policy_version 20646 (0.0005) [2023-12-26 15:31:00,712][105620] Updated weights for policy 1, policy_version 20656 (0.0005) [2023-12-26 15:31:00,921][105692] Updated weights for policy 0, policy_version 20664 (0.0008) [2023-12-26 15:31:00,976][105692] Updated weights for policy 0, policy_version 20675 (0.0012) [2023-12-26 15:31:01,036][105692] Updated weights for policy 0, policy_version 20685 (0.0010) [2023-12-26 15:31:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 10592256. Throughput: 0: 9881.1, 1: 9838.5. Samples: 10558760. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-12-26 15:31:01,062][104569] Avg episode reward: [(0, '9162.242'), (1, '8615.025')] [2023-12-26 15:31:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000020688_5300224.pth... [2023-12-26 15:31:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000020656_5292032.pth... [2023-12-26 15:31:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000019504_4997120.pth [2023-12-26 15:31:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000019536_5005312.pth [2023-12-26 15:31:01,263][105620] Updated weights for policy 1, policy_version 20666 (0.0010) [2023-12-26 15:31:01,319][105620] Updated weights for policy 1, policy_version 20676 (0.0011) [2023-12-26 15:31:01,385][105620] Updated weights for policy 1, policy_version 20686 (0.0011) [2023-12-26 15:31:01,749][105692] Updated weights for policy 0, policy_version 20695 (0.0007) [2023-12-26 15:31:01,805][105692] Updated weights for policy 0, policy_version 20705 (0.0005) [2023-12-26 15:31:01,859][105692] Updated weights for policy 0, policy_version 20715 (0.0008) [2023-12-26 15:31:02,062][105620] Updated weights for policy 1, policy_version 20696 (0.0010) [2023-12-26 15:31:02,118][105620] Updated weights for policy 1, policy_version 20706 (0.0009) [2023-12-26 15:31:02,179][105620] Updated weights for policy 1, policy_version 20716 (0.0010) [2023-12-26 15:31:02,573][105692] Updated weights for policy 0, policy_version 20725 (0.0008) [2023-12-26 15:31:02,620][105692] Updated weights for policy 0, policy_version 20735 (0.0008) [2023-12-26 15:31:02,679][105692] Updated weights for policy 0, policy_version 20745 (0.0009) [2023-12-26 15:31:02,883][105620] Updated weights for policy 1, policy_version 20726 (0.0008) [2023-12-26 15:31:02,936][105620] Updated weights for policy 1, policy_version 20737 (0.0009) [2023-12-26 15:31:02,988][105620] Updated weights for policy 1, policy_version 20747 (0.0010) [2023-12-26 15:31:03,363][105692] Updated weights for policy 0, policy_version 20755 (0.0009) [2023-12-26 15:31:03,418][105692] Updated weights for policy 0, policy_version 20765 (0.0010) [2023-12-26 15:31:03,474][105692] Updated weights for policy 0, policy_version 20775 (0.0011) [2023-12-26 15:31:03,769][105620] Updated weights for policy 1, policy_version 20758 (0.0011) [2023-12-26 15:31:03,854][105620] Updated weights for policy 1, policy_version 20768 (0.0010) [2023-12-26 15:31:03,920][105620] Updated weights for policy 1, policy_version 20778 (0.0010) [2023-12-26 15:31:04,101][105692] Updated weights for policy 0, policy_version 20785 (0.0010) [2023-12-26 15:31:04,164][105692] Updated weights for policy 0, policy_version 20795 (0.0006) [2023-12-26 15:31:04,227][105692] Updated weights for policy 0, policy_version 20805 (0.0008) [2023-12-26 15:31:04,291][105692] Updated weights for policy 0, policy_version 20815 (0.0011) [2023-12-26 15:31:04,612][105620] Updated weights for policy 1, policy_version 20788 (0.0010) [2023-12-26 15:31:04,670][105620] Updated weights for policy 1, policy_version 20798 (0.0010) [2023-12-26 15:31:04,722][105620] Updated weights for policy 1, policy_version 20808 (0.0010) [2023-12-26 15:31:05,013][105692] Updated weights for policy 0, policy_version 20825 (0.0006) [2023-12-26 15:31:05,070][105692] Updated weights for policy 0, policy_version 20835 (0.0010) [2023-12-26 15:31:05,132][105692] Updated weights for policy 0, policy_version 20845 (0.0011) [2023-12-26 15:31:05,379][105620] Updated weights for policy 1, policy_version 20818 (0.0010) [2023-12-26 15:31:05,430][105620] Updated weights for policy 1, policy_version 20828 (0.0010) [2023-12-26 15:31:05,478][105620] Updated weights for policy 1, policy_version 20838 (0.0010) [2023-12-26 15:31:05,529][105620] Updated weights for policy 1, policy_version 20848 (0.0010) [2023-12-26 15:31:05,800][105692] Updated weights for policy 0, policy_version 20855 (0.0009) [2023-12-26 15:31:05,861][105692] Updated weights for policy 0, policy_version 20865 (0.0007) [2023-12-26 15:31:05,919][105692] Updated weights for policy 0, policy_version 20875 (0.0008) [2023-12-26 15:31:06,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 10690560. Throughput: 0: 9854.5, 1: 9898.1. Samples: 10677940. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-12-26 15:31:06,063][104569] Avg episode reward: [(0, '9347.157'), (1, '8985.737')] [2023-12-26 15:31:06,289][105620] Updated weights for policy 1, policy_version 20858 (0.0011) [2023-12-26 15:31:06,352][105620] Updated weights for policy 1, policy_version 20868 (0.0007) [2023-12-26 15:31:06,410][105620] Updated weights for policy 1, policy_version 20878 (0.0005) [2023-12-26 15:31:06,675][105692] Updated weights for policy 0, policy_version 20885 (0.0010) [2023-12-26 15:31:06,733][105692] Updated weights for policy 0, policy_version 20895 (0.0009) [2023-12-26 15:31:06,798][105692] Updated weights for policy 0, policy_version 20905 (0.0005) [2023-12-26 15:31:07,102][105620] Updated weights for policy 1, policy_version 20888 (0.0010) [2023-12-26 15:31:07,170][105620] Updated weights for policy 1, policy_version 20898 (0.0010) [2023-12-26 15:31:07,236][105620] Updated weights for policy 1, policy_version 20908 (0.0010) [2023-12-26 15:31:07,497][105692] Updated weights for policy 0, policy_version 20915 (0.0008) [2023-12-26 15:31:07,545][105692] Updated weights for policy 0, policy_version 20925 (0.0010) [2023-12-26 15:31:07,598][105692] Updated weights for policy 0, policy_version 20935 (0.0011) [2023-12-26 15:31:07,961][105620] Updated weights for policy 1, policy_version 20918 (0.0009) [2023-12-26 15:31:08,016][105620] Updated weights for policy 1, policy_version 20928 (0.0008) [2023-12-26 15:31:08,074][105620] Updated weights for policy 1, policy_version 20938 (0.0008) [2023-12-26 15:31:08,367][105692] Updated weights for policy 0, policy_version 20945 (0.0010) [2023-12-26 15:31:08,420][105692] Updated weights for policy 0, policy_version 20955 (0.0008) [2023-12-26 15:31:08,481][105692] Updated weights for policy 0, policy_version 20965 (0.0009) [2023-12-26 15:31:08,550][105692] Updated weights for policy 0, policy_version 20975 (0.0010) [2023-12-26 15:31:08,722][105620] Updated weights for policy 1, policy_version 20948 (0.0007) [2023-12-26 15:31:08,780][105620] Updated weights for policy 1, policy_version 20958 (0.0005) [2023-12-26 15:31:08,841][105620] Updated weights for policy 1, policy_version 20968 (0.0005) [2023-12-26 15:31:09,317][105692] Updated weights for policy 0, policy_version 20985 (0.0011) [2023-12-26 15:31:09,382][105692] Updated weights for policy 0, policy_version 20995 (0.0011) [2023-12-26 15:31:09,418][105620] Updated weights for policy 1, policy_version 20978 (0.0007) [2023-12-26 15:31:09,448][105692] Updated weights for policy 0, policy_version 21005 (0.0011) [2023-12-26 15:31:09,481][105620] Updated weights for policy 1, policy_version 20988 (0.0009) [2023-12-26 15:31:09,534][105620] Updated weights for policy 1, policy_version 20998 (0.0010) [2023-12-26 15:31:09,589][105620] Updated weights for policy 1, policy_version 21008 (0.0011) [2023-12-26 15:31:10,140][105692] Updated weights for policy 0, policy_version 21015 (0.0010) [2023-12-26 15:31:10,192][105692] Updated weights for policy 0, policy_version 21025 (0.0010) [2023-12-26 15:31:10,253][105692] Updated weights for policy 0, policy_version 21035 (0.0008) [2023-12-26 15:31:10,358][105620] Updated weights for policy 1, policy_version 21018 (0.0008) [2023-12-26 15:31:10,415][105620] Updated weights for policy 1, policy_version 21028 (0.0005) [2023-12-26 15:31:10,473][105620] Updated weights for policy 1, policy_version 21038 (0.0005) [2023-12-26 15:31:10,923][105692] Updated weights for policy 0, policy_version 21045 (0.0009) [2023-12-26 15:31:10,983][105692] Updated weights for policy 0, policy_version 21055 (0.0008) [2023-12-26 15:31:11,040][105692] Updated weights for policy 0, policy_version 21065 (0.0008) [2023-12-26 15:31:11,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 10780672. Throughput: 0: 9855.3, 1: 9895.3. Samples: 10795356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:31:11,063][104569] Avg episode reward: [(0, '9253.010'), (1, '8985.537')] [2023-12-26 15:31:11,126][105620] Updated weights for policy 1, policy_version 21048 (0.0009) [2023-12-26 15:31:11,192][105620] Updated weights for policy 1, policy_version 21058 (0.0010) [2023-12-26 15:31:11,246][105620] Updated weights for policy 1, policy_version 21068 (0.0011) [2023-12-26 15:31:11,816][105692] Updated weights for policy 0, policy_version 21075 (0.0008) [2023-12-26 15:31:11,879][105692] Updated weights for policy 0, policy_version 21085 (0.0008) [2023-12-26 15:31:11,944][105692] Updated weights for policy 0, policy_version 21095 (0.0006) [2023-12-26 15:31:11,963][105620] Updated weights for policy 1, policy_version 21078 (0.0011) [2023-12-26 15:31:12,021][105620] Updated weights for policy 1, policy_version 21088 (0.0011) [2023-12-26 15:31:12,085][105620] Updated weights for policy 1, policy_version 21098 (0.0011) [2023-12-26 15:31:12,609][105692] Updated weights for policy 0, policy_version 21105 (0.0006) [2023-12-26 15:31:12,669][105692] Updated weights for policy 0, policy_version 21115 (0.0007) [2023-12-26 15:31:12,730][105692] Updated weights for policy 0, policy_version 21125 (0.0006) [2023-12-26 15:31:12,789][105692] Updated weights for policy 0, policy_version 21135 (0.0006) [2023-12-26 15:31:12,838][105620] Updated weights for policy 1, policy_version 21108 (0.0011) [2023-12-26 15:31:12,894][105620] Updated weights for policy 1, policy_version 21118 (0.0011) [2023-12-26 15:31:12,957][105620] Updated weights for policy 1, policy_version 21128 (0.0011) [2023-12-26 15:31:13,476][105692] Updated weights for policy 0, policy_version 21145 (0.0005) [2023-12-26 15:31:13,535][105692] Updated weights for policy 0, policy_version 21155 (0.0005) [2023-12-26 15:31:13,590][105692] Updated weights for policy 0, policy_version 21165 (0.0005) [2023-12-26 15:31:13,686][105620] Updated weights for policy 1, policy_version 21138 (0.0011) [2023-12-26 15:31:13,745][105620] Updated weights for policy 1, policy_version 21148 (0.0011) [2023-12-26 15:31:13,803][105620] Updated weights for policy 1, policy_version 21158 (0.0010) [2023-12-26 15:31:13,869][105620] Updated weights for policy 1, policy_version 21168 (0.0011) [2023-12-26 15:31:14,124][105692] Updated weights for policy 0, policy_version 21175 (0.0007) [2023-12-26 15:31:14,184][105692] Updated weights for policy 0, policy_version 21185 (0.0008) [2023-12-26 15:31:14,242][105692] Updated weights for policy 0, policy_version 21195 (0.0008) [2023-12-26 15:31:14,594][105620] Updated weights for policy 1, policy_version 21178 (0.0006) [2023-12-26 15:31:14,650][105620] Updated weights for policy 1, policy_version 21188 (0.0008) [2023-12-26 15:31:14,706][105620] Updated weights for policy 1, policy_version 21198 (0.0008) [2023-12-26 15:31:15,074][105692] Updated weights for policy 0, policy_version 21205 (0.0009) [2023-12-26 15:31:15,127][105692] Updated weights for policy 0, policy_version 21215 (0.0011) [2023-12-26 15:31:15,184][105692] Updated weights for policy 0, policy_version 21225 (0.0011) [2023-12-26 15:31:15,404][105620] Updated weights for policy 1, policy_version 21208 (0.0006) [2023-12-26 15:31:15,464][105620] Updated weights for policy 1, policy_version 21218 (0.0008) [2023-12-26 15:31:15,524][105620] Updated weights for policy 1, policy_version 21228 (0.0005) [2023-12-26 15:31:15,927][105692] Updated weights for policy 0, policy_version 21235 (0.0010) [2023-12-26 15:31:15,975][105692] Updated weights for policy 0, policy_version 21245 (0.0010) [2023-12-26 15:31:16,025][105692] Updated weights for policy 0, policy_version 21255 (0.0010) [2023-12-26 15:31:16,047][105620] Updated weights for policy 1, policy_version 21238 (0.0006) [2023-12-26 15:31:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 10878976. Throughput: 0: 9736.3, 1: 9852.7. Samples: 10853172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:31:16,063][104569] Avg episode reward: [(0, '9159.462'), (1, '8520.066')] [2023-12-26 15:31:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000021264_5447680.pth... [2023-12-26 15:31:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000020112_5152768.pth [2023-12-26 15:31:16,103][105620] Updated weights for policy 1, policy_version 21248 (0.0008) [2023-12-26 15:31:16,157][105620] Updated weights for policy 1, policy_version 21258 (0.0009) [2023-12-26 15:31:16,188][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000021264_5447680.pth... [2023-12-26 15:31:16,191][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000020080_5144576.pth [2023-12-26 15:31:16,695][105692] Updated weights for policy 0, policy_version 21265 (0.0008) [2023-12-26 15:31:16,760][105692] Updated weights for policy 0, policy_version 21275 (0.0009) [2023-12-26 15:31:16,821][105692] Updated weights for policy 0, policy_version 21285 (0.0009) [2023-12-26 15:31:16,880][105692] Updated weights for policy 0, policy_version 21295 (0.0006) [2023-12-26 15:31:16,968][105620] Updated weights for policy 1, policy_version 21268 (0.0010) [2023-12-26 15:31:17,018][105620] Updated weights for policy 1, policy_version 21278 (0.0008) [2023-12-26 15:31:17,069][105620] Updated weights for policy 1, policy_version 21288 (0.0009) [2023-12-26 15:31:17,529][105692] Updated weights for policy 0, policy_version 21305 (0.0008) [2023-12-26 15:31:17,591][105692] Updated weights for policy 0, policy_version 21315 (0.0009) [2023-12-26 15:31:17,645][105692] Updated weights for policy 0, policy_version 21325 (0.0010) [2023-12-26 15:31:17,711][105620] Updated weights for policy 1, policy_version 21298 (0.0009) [2023-12-26 15:31:17,776][105620] Updated weights for policy 1, policy_version 21308 (0.0005) [2023-12-26 15:31:17,840][105620] Updated weights for policy 1, policy_version 21318 (0.0009) [2023-12-26 15:31:17,899][105620] Updated weights for policy 1, policy_version 21328 (0.0009) [2023-12-26 15:31:18,411][105692] Updated weights for policy 0, policy_version 21335 (0.0009) [2023-12-26 15:31:18,478][105692] Updated weights for policy 0, policy_version 21345 (0.0010) [2023-12-26 15:31:18,532][105692] Updated weights for policy 0, policy_version 21355 (0.0010) [2023-12-26 15:31:18,601][105620] Updated weights for policy 1, policy_version 21338 (0.0005) [2023-12-26 15:31:18,661][105620] Updated weights for policy 1, policy_version 21348 (0.0005) [2023-12-26 15:31:18,723][105620] Updated weights for policy 1, policy_version 21358 (0.0006) [2023-12-26 15:31:19,318][105692] Updated weights for policy 0, policy_version 21365 (0.0008) [2023-12-26 15:31:19,383][105692] Updated weights for policy 0, policy_version 21375 (0.0007) [2023-12-26 15:31:19,405][105620] Updated weights for policy 1, policy_version 21368 (0.0010) [2023-12-26 15:31:19,437][105692] Updated weights for policy 0, policy_version 21385 (0.0005) [2023-12-26 15:31:19,472][105620] Updated weights for policy 1, policy_version 21378 (0.0011) [2023-12-26 15:31:19,541][105620] Updated weights for policy 1, policy_version 21388 (0.0008) [2023-12-26 15:31:20,129][105692] Updated weights for policy 0, policy_version 21395 (0.0005) [2023-12-26 15:31:20,188][105692] Updated weights for policy 0, policy_version 21405 (0.0005) [2023-12-26 15:31:20,258][105692] Updated weights for policy 0, policy_version 21415 (0.0006) [2023-12-26 15:31:20,326][105620] Updated weights for policy 1, policy_version 21398 (0.0007) [2023-12-26 15:31:20,386][105620] Updated weights for policy 1, policy_version 21408 (0.0008) [2023-12-26 15:31:20,453][105620] Updated weights for policy 1, policy_version 21418 (0.0008) [2023-12-26 15:31:20,948][105692] Updated weights for policy 0, policy_version 21425 (0.0008) [2023-12-26 15:31:21,007][105692] Updated weights for policy 0, policy_version 21435 (0.0010) [2023-12-26 15:31:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 10977280. Throughput: 0: 9751.9, 1: 9895.0. Samples: 10971608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:31:21,062][104569] Avg episode reward: [(0, '9252.327'), (1, '8517.292')] [2023-12-26 15:31:21,068][105692] Updated weights for policy 0, policy_version 21445 (0.0008) [2023-12-26 15:31:21,131][105692] Updated weights for policy 0, policy_version 21455 (0.0009) [2023-12-26 15:31:21,167][105620] Updated weights for policy 1, policy_version 21428 (0.0008) [2023-12-26 15:31:21,224][105620] Updated weights for policy 1, policy_version 21438 (0.0006) [2023-12-26 15:31:21,286][105620] Updated weights for policy 1, policy_version 21448 (0.0010) [2023-12-26 15:31:21,954][105692] Updated weights for policy 0, policy_version 21465 (0.0011) [2023-12-26 15:31:22,000][105620] Updated weights for policy 1, policy_version 21458 (0.0007) [2023-12-26 15:31:22,011][105692] Updated weights for policy 0, policy_version 21475 (0.0010) [2023-12-26 15:31:22,057][105620] Updated weights for policy 1, policy_version 21468 (0.0006) [2023-12-26 15:31:22,063][105692] Updated weights for policy 0, policy_version 21485 (0.0011) [2023-12-26 15:31:22,120][105620] Updated weights for policy 1, policy_version 21478 (0.0007) [2023-12-26 15:31:22,175][105620] Updated weights for policy 1, policy_version 21488 (0.0008) [2023-12-26 15:31:22,863][105692] Updated weights for policy 0, policy_version 21495 (0.0009) [2023-12-26 15:31:22,881][105620] Updated weights for policy 1, policy_version 21498 (0.0008) [2023-12-26 15:31:22,919][105692] Updated weights for policy 0, policy_version 21505 (0.0007) [2023-12-26 15:31:22,941][105620] Updated weights for policy 1, policy_version 21508 (0.0009) [2023-12-26 15:31:22,978][105692] Updated weights for policy 0, policy_version 21515 (0.0007) [2023-12-26 15:31:23,000][105620] Updated weights for policy 1, policy_version 21518 (0.0007) [2023-12-26 15:31:23,700][105620] Updated weights for policy 1, policy_version 21528 (0.0005) [2023-12-26 15:31:23,723][105692] Updated weights for policy 0, policy_version 21525 (0.0009) [2023-12-26 15:31:23,766][105620] Updated weights for policy 1, policy_version 21538 (0.0006) [2023-12-26 15:31:23,780][105692] Updated weights for policy 0, policy_version 21535 (0.0009) [2023-12-26 15:31:23,826][105620] Updated weights for policy 1, policy_version 21548 (0.0005) [2023-12-26 15:31:23,829][105692] Updated weights for policy 0, policy_version 21545 (0.0009) [2023-12-26 15:31:24,423][105620] Updated weights for policy 1, policy_version 21558 (0.0008) [2023-12-26 15:31:24,485][105620] Updated weights for policy 1, policy_version 21568 (0.0006) [2023-12-26 15:31:24,542][105620] Updated weights for policy 1, policy_version 21578 (0.0005) [2023-12-26 15:31:24,661][105692] Updated weights for policy 0, policy_version 21555 (0.0010) [2023-12-26 15:31:24,714][105692] Updated weights for policy 0, policy_version 21565 (0.0010) [2023-12-26 15:31:24,779][105692] Updated weights for policy 0, policy_version 21575 (0.0006) [2023-12-26 15:31:25,097][105620] Updated weights for policy 1, policy_version 21588 (0.0006) [2023-12-26 15:31:25,159][105620] Updated weights for policy 1, policy_version 21598 (0.0008) [2023-12-26 15:31:25,230][105620] Updated weights for policy 1, policy_version 21608 (0.0008) [2023-12-26 15:31:25,459][105692] Updated weights for policy 0, policy_version 21585 (0.0006) [2023-12-26 15:31:25,521][105692] Updated weights for policy 0, policy_version 21595 (0.0009) [2023-12-26 15:31:25,580][105692] Updated weights for policy 0, policy_version 21605 (0.0008) [2023-12-26 15:31:25,642][105692] Updated weights for policy 0, policy_version 21615 (0.0009) [2023-12-26 15:31:25,936][105620] Updated weights for policy 1, policy_version 21618 (0.0008) [2023-12-26 15:31:25,986][105620] Updated weights for policy 1, policy_version 21628 (0.0006) [2023-12-26 15:31:26,034][105620] Updated weights for policy 1, policy_version 21638 (0.0005) [2023-12-26 15:31:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 11075584. Throughput: 0: 9734.9, 1: 9937.0. Samples: 11087840. Policy #0 lag: (min: 10.0, avg: 21.5, max: 42.0) [2023-12-26 15:31:26,062][104569] Avg episode reward: [(0, '9252.949'), (1, '8793.597')] [2023-12-26 15:31:26,084][105620] Updated weights for policy 1, policy_version 21648 (0.0007) [2023-12-26 15:31:26,455][105692] Updated weights for policy 0, policy_version 21625 (0.0010) [2023-12-26 15:31:26,501][105692] Updated weights for policy 0, policy_version 21635 (0.0009) [2023-12-26 15:31:26,545][105692] Updated weights for policy 0, policy_version 21645 (0.0010) [2023-12-26 15:31:26,709][105620] Updated weights for policy 1, policy_version 21658 (0.0007) [2023-12-26 15:31:26,758][105620] Updated weights for policy 1, policy_version 21668 (0.0008) [2023-12-26 15:31:26,806][105620] Updated weights for policy 1, policy_version 21678 (0.0008) [2023-12-26 15:31:27,319][105692] Updated weights for policy 0, policy_version 21655 (0.0010) [2023-12-26 15:31:27,387][105692] Updated weights for policy 0, policy_version 21665 (0.0010) [2023-12-26 15:31:27,457][105692] Updated weights for policy 0, policy_version 21675 (0.0010) [2023-12-26 15:31:27,517][105620] Updated weights for policy 1, policy_version 21688 (0.0006) [2023-12-26 15:31:27,560][105620] Updated weights for policy 1, policy_version 21698 (0.0005) [2023-12-26 15:31:27,619][105620] Updated weights for policy 1, policy_version 21708 (0.0005) [2023-12-26 15:31:28,069][105692] Updated weights for policy 0, policy_version 21685 (0.0007) [2023-12-26 15:31:28,117][105692] Updated weights for policy 0, policy_version 21695 (0.0008) [2023-12-26 15:31:28,134][105620] Updated weights for policy 1, policy_version 21718 (0.0005) [2023-12-26 15:31:28,161][105692] Updated weights for policy 0, policy_version 21705 (0.0010) [2023-12-26 15:31:28,181][105620] Updated weights for policy 1, policy_version 21728 (0.0009) [2023-12-26 15:31:28,225][105620] Updated weights for policy 1, policy_version 21738 (0.0010) [2023-12-26 15:31:28,851][105692] Updated weights for policy 0, policy_version 21715 (0.0009) [2023-12-26 15:31:28,911][105692] Updated weights for policy 0, policy_version 21725 (0.0005) [2023-12-26 15:31:28,971][105692] Updated weights for policy 0, policy_version 21735 (0.0005) [2023-12-26 15:31:28,980][105620] Updated weights for policy 1, policy_version 21748 (0.0010) [2023-12-26 15:31:29,027][105620] Updated weights for policy 1, policy_version 21758 (0.0010) [2023-12-26 15:31:29,074][105620] Updated weights for policy 1, policy_version 21768 (0.0010) [2023-12-26 15:31:29,529][105692] Updated weights for policy 0, policy_version 21745 (0.0006) [2023-12-26 15:31:29,583][105692] Updated weights for policy 0, policy_version 21755 (0.0010) [2023-12-26 15:31:29,631][105692] Updated weights for policy 0, policy_version 21765 (0.0010) [2023-12-26 15:31:29,686][105692] Updated weights for policy 0, policy_version 21775 (0.0011) [2023-12-26 15:31:29,812][105620] Updated weights for policy 1, policy_version 21778 (0.0010) [2023-12-26 15:31:29,872][105620] Updated weights for policy 1, policy_version 21788 (0.0010) [2023-12-26 15:31:29,931][105620] Updated weights for policy 1, policy_version 21798 (0.0011) [2023-12-26 15:31:29,977][105620] Updated weights for policy 1, policy_version 21808 (0.0010) [2023-12-26 15:31:30,414][105692] Updated weights for policy 0, policy_version 21785 (0.0007) [2023-12-26 15:31:30,465][105692] Updated weights for policy 0, policy_version 21795 (0.0005) [2023-12-26 15:31:30,516][105692] Updated weights for policy 0, policy_version 21805 (0.0005) [2023-12-26 15:31:30,727][105620] Updated weights for policy 1, policy_version 21818 (0.0010) [2023-12-26 15:31:30,775][105620] Updated weights for policy 1, policy_version 21828 (0.0010) [2023-12-26 15:31:30,822][105620] Updated weights for policy 1, policy_version 21838 (0.0010) [2023-12-26 15:31:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 11182080. Throughput: 0: 9716.9, 1: 10026.3. Samples: 11148768. Policy #0 lag: (min: 10.0, avg: 21.5, max: 42.0) [2023-12-26 15:31:31,062][104569] Avg episode reward: [(0, '9346.184'), (1, '8886.438')] [2023-12-26 15:31:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000021840_5595136.pth... [2023-12-26 15:31:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000021808_5586944.pth... [2023-12-26 15:31:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000020656_5292032.pth [2023-12-26 15:31:31,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000020688_5300224.pth [2023-12-26 15:31:31,126][105692] Updated weights for policy 0, policy_version 21815 (0.0009) [2023-12-26 15:31:31,192][105692] Updated weights for policy 0, policy_version 21825 (0.0006) [2023-12-26 15:31:31,261][105692] Updated weights for policy 0, policy_version 21835 (0.0008) [2023-12-26 15:31:31,572][105620] Updated weights for policy 1, policy_version 21848 (0.0010) [2023-12-26 15:31:31,628][105620] Updated weights for policy 1, policy_version 21858 (0.0010) [2023-12-26 15:31:31,680][105620] Updated weights for policy 1, policy_version 21868 (0.0010) [2023-12-26 15:31:31,914][105692] Updated weights for policy 0, policy_version 21845 (0.0008) [2023-12-26 15:31:31,962][105692] Updated weights for policy 0, policy_version 21855 (0.0008) [2023-12-26 15:31:32,010][105692] Updated weights for policy 0, policy_version 21865 (0.0009) [2023-12-26 15:31:32,382][105620] Updated weights for policy 1, policy_version 21878 (0.0010) [2023-12-26 15:31:32,428][105620] Updated weights for policy 1, policy_version 21888 (0.0009) [2023-12-26 15:31:32,492][105620] Updated weights for policy 1, policy_version 21898 (0.0009) [2023-12-26 15:31:32,807][105692] Updated weights for policy 0, policy_version 21875 (0.0009) [2023-12-26 15:31:32,865][105692] Updated weights for policy 0, policy_version 21885 (0.0009) [2023-12-26 15:31:32,924][105692] Updated weights for policy 0, policy_version 21895 (0.0008) [2023-12-26 15:31:33,174][105620] Updated weights for policy 1, policy_version 21908 (0.0007) [2023-12-26 15:31:33,233][105620] Updated weights for policy 1, policy_version 21918 (0.0008) [2023-12-26 15:31:33,290][105620] Updated weights for policy 1, policy_version 21928 (0.0009) [2023-12-26 15:31:33,562][105692] Updated weights for policy 0, policy_version 21905 (0.0008) [2023-12-26 15:31:33,617][105692] Updated weights for policy 0, policy_version 21916 (0.0010) [2023-12-26 15:31:33,670][105692] Updated weights for policy 0, policy_version 21926 (0.0010) [2023-12-26 15:31:33,733][105692] Updated weights for policy 0, policy_version 21936 (0.0005) [2023-12-26 15:31:33,934][105620] Updated weights for policy 1, policy_version 21938 (0.0008) [2023-12-26 15:31:33,980][105620] Updated weights for policy 1, policy_version 21948 (0.0005) [2023-12-26 15:31:34,031][105620] Updated weights for policy 1, policy_version 21958 (0.0005) [2023-12-26 15:31:34,094][105620] Updated weights for policy 1, policy_version 21968 (0.0007) [2023-12-26 15:31:34,299][105692] Updated weights for policy 0, policy_version 21946 (0.0009) [2023-12-26 15:31:34,357][105692] Updated weights for policy 0, policy_version 21956 (0.0009) [2023-12-26 15:31:34,410][105692] Updated weights for policy 0, policy_version 21966 (0.0009) [2023-12-26 15:31:34,793][105620] Updated weights for policy 1, policy_version 21978 (0.0009) [2023-12-26 15:31:34,845][105620] Updated weights for policy 1, policy_version 21988 (0.0009) [2023-12-26 15:31:34,903][105620] Updated weights for policy 1, policy_version 21998 (0.0009) [2023-12-26 15:31:35,100][105692] Updated weights for policy 0, policy_version 21976 (0.0008) [2023-12-26 15:31:35,145][105692] Updated weights for policy 0, policy_version 21986 (0.0008) [2023-12-26 15:31:35,202][105692] Updated weights for policy 0, policy_version 21996 (0.0009) [2023-12-26 15:31:35,663][105620] Updated weights for policy 1, policy_version 22008 (0.0009) [2023-12-26 15:31:35,710][105620] Updated weights for policy 1, policy_version 22018 (0.0009) [2023-12-26 15:31:35,756][105620] Updated weights for policy 1, policy_version 22028 (0.0006) [2023-12-26 15:31:36,001][105692] Updated weights for policy 0, policy_version 22006 (0.0009) [2023-12-26 15:31:36,053][105692] Updated weights for policy 0, policy_version 22018 (0.0010) [2023-12-26 15:31:36,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.9, 300 sec: 19633.0). Total num frames: 11280384. Throughput: 0: 9793.6, 1: 10050.4. Samples: 11270756. Policy #0 lag: (min: 10.0, avg: 21.5, max: 42.0) [2023-12-26 15:31:36,063][104569] Avg episode reward: [(0, '9254.131'), (1, '8888.965')] [2023-12-26 15:31:36,107][105692] Updated weights for policy 0, policy_version 22029 (0.0009) [2023-12-26 15:31:36,340][105620] Updated weights for policy 1, policy_version 22038 (0.0008) [2023-12-26 15:31:36,393][105620] Updated weights for policy 1, policy_version 22048 (0.0010) [2023-12-26 15:31:36,447][105620] Updated weights for policy 1, policy_version 22058 (0.0010) [2023-12-26 15:31:36,808][105692] Updated weights for policy 0, policy_version 22039 (0.0008) [2023-12-26 15:31:36,855][105692] Updated weights for policy 0, policy_version 22049 (0.0009) [2023-12-26 15:31:36,915][105692] Updated weights for policy 0, policy_version 22059 (0.0009) [2023-12-26 15:31:37,179][105620] Updated weights for policy 1, policy_version 22068 (0.0008) [2023-12-26 15:31:37,231][105620] Updated weights for policy 1, policy_version 22078 (0.0005) [2023-12-26 15:31:37,278][105620] Updated weights for policy 1, policy_version 22088 (0.0005) [2023-12-26 15:31:37,589][105692] Updated weights for policy 0, policy_version 22069 (0.0007) [2023-12-26 15:31:37,648][105692] Updated weights for policy 0, policy_version 22079 (0.0005) [2023-12-26 15:31:37,700][105692] Updated weights for policy 0, policy_version 22089 (0.0005) [2023-12-26 15:31:37,883][105620] Updated weights for policy 1, policy_version 22098 (0.0006) [2023-12-26 15:31:37,939][105620] Updated weights for policy 1, policy_version 22108 (0.0010) [2023-12-26 15:31:37,993][105620] Updated weights for policy 1, policy_version 22118 (0.0010) [2023-12-26 15:31:38,047][105620] Updated weights for policy 1, policy_version 22128 (0.0007) [2023-12-26 15:31:38,355][105692] Updated weights for policy 0, policy_version 22099 (0.0007) [2023-12-26 15:31:38,416][105692] Updated weights for policy 0, policy_version 22109 (0.0008) [2023-12-26 15:31:38,476][105692] Updated weights for policy 0, policy_version 22119 (0.0009) [2023-12-26 15:31:38,725][105620] Updated weights for policy 1, policy_version 22138 (0.0007) [2023-12-26 15:31:38,790][105620] Updated weights for policy 1, policy_version 22148 (0.0008) [2023-12-26 15:31:38,854][105620] Updated weights for policy 1, policy_version 22158 (0.0008) [2023-12-26 15:31:39,218][105692] Updated weights for policy 0, policy_version 22129 (0.0008) [2023-12-26 15:31:39,279][105692] Updated weights for policy 0, policy_version 22139 (0.0008) [2023-12-26 15:31:39,341][105692] Updated weights for policy 0, policy_version 22149 (0.0008) [2023-12-26 15:31:39,419][105692] Updated weights for policy 0, policy_version 22159 (0.0008) [2023-12-26 15:31:39,556][105620] Updated weights for policy 1, policy_version 22168 (0.0010) [2023-12-26 15:31:39,621][105620] Updated weights for policy 1, policy_version 22178 (0.0005) [2023-12-26 15:31:39,693][105620] Updated weights for policy 1, policy_version 22188 (0.0007) [2023-12-26 15:31:40,107][105692] Updated weights for policy 0, policy_version 22169 (0.0009) [2023-12-26 15:31:40,179][105692] Updated weights for policy 0, policy_version 22179 (0.0006) [2023-12-26 15:31:40,243][105692] Updated weights for policy 0, policy_version 22189 (0.0005) [2023-12-26 15:31:40,332][105620] Updated weights for policy 1, policy_version 22198 (0.0008) [2023-12-26 15:31:40,397][105620] Updated weights for policy 1, policy_version 22208 (0.0007) [2023-12-26 15:31:40,462][105620] Updated weights for policy 1, policy_version 22218 (0.0005) [2023-12-26 15:31:40,929][105692] Updated weights for policy 0, policy_version 22199 (0.0010) [2023-12-26 15:31:40,986][105692] Updated weights for policy 0, policy_version 22209 (0.0010) [2023-12-26 15:31:41,040][105620] Updated weights for policy 1, policy_version 22228 (0.0007) [2023-12-26 15:31:41,050][105692] Updated weights for policy 0, policy_version 22219 (0.0009) [2023-12-26 15:31:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 11378688. Throughput: 0: 9821.4, 1: 10074.2. Samples: 11393964. Policy #0 lag: (min: 2.0, avg: 13.9, max: 34.0) [2023-12-26 15:31:41,063][104569] Avg episode reward: [(0, '9160.844'), (1, '8983.466')] [2023-12-26 15:31:41,103][105620] Updated weights for policy 1, policy_version 22238 (0.0007) [2023-12-26 15:31:41,168][105620] Updated weights for policy 1, policy_version 22248 (0.0010) [2023-12-26 15:31:41,812][105692] Updated weights for policy 0, policy_version 22229 (0.0009) [2023-12-26 15:31:41,868][105692] Updated weights for policy 0, policy_version 22239 (0.0008) [2023-12-26 15:31:41,933][105692] Updated weights for policy 0, policy_version 22249 (0.0008) [2023-12-26 15:31:41,946][105620] Updated weights for policy 1, policy_version 22258 (0.0011) [2023-12-26 15:31:42,005][105620] Updated weights for policy 1, policy_version 22268 (0.0011) [2023-12-26 15:31:42,061][105620] Updated weights for policy 1, policy_version 22278 (0.0011) [2023-12-26 15:31:42,120][105620] Updated weights for policy 1, policy_version 22288 (0.0010) [2023-12-26 15:31:42,723][105692] Updated weights for policy 0, policy_version 22259 (0.0007) [2023-12-26 15:31:42,772][105692] Updated weights for policy 0, policy_version 22269 (0.0008) [2023-12-26 15:31:42,829][105692] Updated weights for policy 0, policy_version 22279 (0.0008) [2023-12-26 15:31:42,911][105620] Updated weights for policy 1, policy_version 22298 (0.0010) [2023-12-26 15:31:42,959][105620] Updated weights for policy 1, policy_version 22308 (0.0010) [2023-12-26 15:31:43,007][105620] Updated weights for policy 1, policy_version 22318 (0.0010) [2023-12-26 15:31:43,549][105692] Updated weights for policy 0, policy_version 22289 (0.0007) [2023-12-26 15:31:43,616][105692] Updated weights for policy 0, policy_version 22299 (0.0006) [2023-12-26 15:31:43,671][105692] Updated weights for policy 0, policy_version 22309 (0.0005) [2023-12-26 15:31:43,731][105692] Updated weights for policy 0, policy_version 22319 (0.0006) [2023-12-26 15:31:43,773][105620] Updated weights for policy 1, policy_version 22328 (0.0010) [2023-12-26 15:31:43,824][105620] Updated weights for policy 1, policy_version 22338 (0.0010) [2023-12-26 15:31:43,876][105620] Updated weights for policy 1, policy_version 22348 (0.0010) [2023-12-26 15:31:44,332][105692] Updated weights for policy 0, policy_version 22329 (0.0010) [2023-12-26 15:31:44,390][105692] Updated weights for policy 0, policy_version 22339 (0.0010) [2023-12-26 15:31:44,456][105692] Updated weights for policy 0, policy_version 22349 (0.0010) [2023-12-26 15:31:44,631][105620] Updated weights for policy 1, policy_version 22358 (0.0010) [2023-12-26 15:31:44,681][105620] Updated weights for policy 1, policy_version 22368 (0.0010) [2023-12-26 15:31:44,732][105620] Updated weights for policy 1, policy_version 22378 (0.0010) [2023-12-26 15:31:45,147][105692] Updated weights for policy 0, policy_version 22359 (0.0005) [2023-12-26 15:31:45,205][105692] Updated weights for policy 0, policy_version 22369 (0.0006) [2023-12-26 15:31:45,257][105692] Updated weights for policy 0, policy_version 22379 (0.0005) [2023-12-26 15:31:45,493][105620] Updated weights for policy 1, policy_version 22388 (0.0008) [2023-12-26 15:31:45,556][105620] Updated weights for policy 1, policy_version 22398 (0.0005) [2023-12-26 15:31:45,625][105620] Updated weights for policy 1, policy_version 22408 (0.0005) [2023-12-26 15:31:45,935][105692] Updated weights for policy 0, policy_version 22389 (0.0009) [2023-12-26 15:31:45,983][105692] Updated weights for policy 0, policy_version 22399 (0.0010) [2023-12-26 15:31:46,046][105692] Updated weights for policy 0, policy_version 22409 (0.0010) [2023-12-26 15:31:46,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 11476992. Throughput: 0: 9758.0, 1: 10039.7. Samples: 11449660. Policy #0 lag: (min: 2.0, avg: 13.9, max: 34.0) [2023-12-26 15:31:46,062][104569] Avg episode reward: [(0, '9248.752'), (1, '8798.900')] [2023-12-26 15:31:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000022416_5742592.pth... [2023-12-26 15:31:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000021264_5447680.pth [2023-12-26 15:31:46,084][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000022416_5742592.pth... [2023-12-26 15:31:46,089][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000021264_5447680.pth [2023-12-26 15:31:46,214][105620] Updated weights for policy 1, policy_version 22418 (0.0005) [2023-12-26 15:31:46,270][105620] Updated weights for policy 1, policy_version 22428 (0.0005) [2023-12-26 15:31:46,322][105620] Updated weights for policy 1, policy_version 22438 (0.0005) [2023-12-26 15:31:46,380][105620] Updated weights for policy 1, policy_version 22448 (0.0005) [2023-12-26 15:31:46,724][105692] Updated weights for policy 0, policy_version 22419 (0.0009) [2023-12-26 15:31:46,780][105692] Updated weights for policy 0, policy_version 22429 (0.0005) [2023-12-26 15:31:46,829][105692] Updated weights for policy 0, policy_version 22439 (0.0005) [2023-12-26 15:31:46,893][105620] Updated weights for policy 1, policy_version 22458 (0.0010) [2023-12-26 15:31:46,956][105620] Updated weights for policy 1, policy_version 22468 (0.0010) [2023-12-26 15:31:47,029][105620] Updated weights for policy 1, policy_version 22478 (0.0009) [2023-12-26 15:31:47,377][105692] Updated weights for policy 0, policy_version 22449 (0.0009) [2023-12-26 15:31:47,439][105692] Updated weights for policy 0, policy_version 22459 (0.0010) [2023-12-26 15:31:47,499][105692] Updated weights for policy 0, policy_version 22469 (0.0010) [2023-12-26 15:31:47,562][105692] Updated weights for policy 0, policy_version 22479 (0.0005) [2023-12-26 15:31:47,609][105620] Updated weights for policy 1, policy_version 22488 (0.0006) [2023-12-26 15:31:47,667][105620] Updated weights for policy 1, policy_version 22498 (0.0005) [2023-12-26 15:31:47,733][105620] Updated weights for policy 1, policy_version 22508 (0.0007) [2023-12-26 15:31:48,082][105692] Updated weights for policy 0, policy_version 22489 (0.0010) [2023-12-26 15:31:48,130][105692] Updated weights for policy 0, policy_version 22499 (0.0010) [2023-12-26 15:31:48,175][105692] Updated weights for policy 0, policy_version 22509 (0.0010) [2023-12-26 15:31:48,357][105620] Updated weights for policy 1, policy_version 22518 (0.0011) [2023-12-26 15:31:48,419][105620] Updated weights for policy 1, policy_version 22528 (0.0011) [2023-12-26 15:31:48,483][105620] Updated weights for policy 1, policy_version 22538 (0.0011) [2023-12-26 15:31:48,941][105692] Updated weights for policy 0, policy_version 22519 (0.0010) [2023-12-26 15:31:48,999][105692] Updated weights for policy 0, policy_version 22529 (0.0010) [2023-12-26 15:31:49,048][105692] Updated weights for policy 0, policy_version 22539 (0.0010) [2023-12-26 15:31:49,215][105620] Updated weights for policy 1, policy_version 22548 (0.0011) [2023-12-26 15:31:49,277][105620] Updated weights for policy 1, policy_version 22558 (0.0009) [2023-12-26 15:31:49,335][105620] Updated weights for policy 1, policy_version 22568 (0.0008) [2023-12-26 15:31:49,843][105692] Updated weights for policy 0, policy_version 22549 (0.0009) [2023-12-26 15:31:49,898][105692] Updated weights for policy 0, policy_version 22559 (0.0008) [2023-12-26 15:31:49,956][105692] Updated weights for policy 0, policy_version 22569 (0.0008) [2023-12-26 15:31:50,096][105620] Updated weights for policy 1, policy_version 22578 (0.0008) [2023-12-26 15:31:50,159][105620] Updated weights for policy 1, policy_version 22588 (0.0007) [2023-12-26 15:31:50,215][105620] Updated weights for policy 1, policy_version 22598 (0.0010) [2023-12-26 15:31:50,284][105620] Updated weights for policy 1, policy_version 22608 (0.0010) [2023-12-26 15:31:50,654][105692] Updated weights for policy 0, policy_version 22579 (0.0009) [2023-12-26 15:31:50,714][105692] Updated weights for policy 0, policy_version 22589 (0.0011) [2023-12-26 15:31:50,779][105692] Updated weights for policy 0, policy_version 22599 (0.0011) [2023-12-26 15:31:50,906][105620] Updated weights for policy 1, policy_version 22618 (0.0010) [2023-12-26 15:31:50,955][105620] Updated weights for policy 1, policy_version 22628 (0.0010) [2023-12-26 15:31:51,010][105620] Updated weights for policy 1, policy_version 22638 (0.0010) [2023-12-26 15:31:51,062][104569] Fps is (10 sec: 21299.4, 60 sec: 19933.9, 300 sec: 19660.8). Total num frames: 11591680. Throughput: 0: 9862.0, 1: 10060.9. Samples: 11574468. Policy #0 lag: (min: 2.0, avg: 13.9, max: 34.0) [2023-12-26 15:31:51,062][104569] Avg episode reward: [(0, '9334.025'), (1, '8616.134')] [2023-12-26 15:31:51,479][105692] Updated weights for policy 0, policy_version 22609 (0.0011) [2023-12-26 15:31:51,537][105692] Updated weights for policy 0, policy_version 22619 (0.0008) [2023-12-26 15:31:51,589][105692] Updated weights for policy 0, policy_version 22629 (0.0005) [2023-12-26 15:31:51,653][105692] Updated weights for policy 0, policy_version 22639 (0.0007) [2023-12-26 15:31:51,825][105620] Updated weights for policy 1, policy_version 22648 (0.0009) [2023-12-26 15:31:51,889][105620] Updated weights for policy 1, policy_version 22658 (0.0009) [2023-12-26 15:31:51,947][105620] Updated weights for policy 1, policy_version 22668 (0.0008) [2023-12-26 15:31:52,351][105692] Updated weights for policy 0, policy_version 22649 (0.0009) [2023-12-26 15:31:52,409][105692] Updated weights for policy 0, policy_version 22659 (0.0009) [2023-12-26 15:31:52,461][105692] Updated weights for policy 0, policy_version 22669 (0.0008) [2023-12-26 15:31:52,672][105620] Updated weights for policy 1, policy_version 22678 (0.0007) [2023-12-26 15:31:52,738][105620] Updated weights for policy 1, policy_version 22688 (0.0008) [2023-12-26 15:31:52,801][105620] Updated weights for policy 1, policy_version 22698 (0.0010) [2023-12-26 15:31:53,275][105692] Updated weights for policy 0, policy_version 22679 (0.0009) [2023-12-26 15:31:53,325][105692] Updated weights for policy 0, policy_version 22689 (0.0009) [2023-12-26 15:31:53,368][105620] Updated weights for policy 1, policy_version 22708 (0.0008) [2023-12-26 15:31:53,369][105692] Updated weights for policy 0, policy_version 22699 (0.0006) [2023-12-26 15:31:53,429][105620] Updated weights for policy 1, policy_version 22718 (0.0009) [2023-12-26 15:31:53,478][105620] Updated weights for policy 1, policy_version 22728 (0.0009) [2023-12-26 15:31:54,073][105692] Updated weights for policy 0, policy_version 22709 (0.0005) [2023-12-26 15:31:54,135][105692] Updated weights for policy 0, policy_version 22719 (0.0005) [2023-12-26 15:31:54,193][105692] Updated weights for policy 0, policy_version 22729 (0.0006) [2023-12-26 15:31:54,195][105620] Updated weights for policy 1, policy_version 22738 (0.0007) [2023-12-26 15:31:54,261][105620] Updated weights for policy 1, policy_version 22748 (0.0011) [2023-12-26 15:31:54,334][105620] Updated weights for policy 1, policy_version 22758 (0.0011) [2023-12-26 15:31:54,400][105620] Updated weights for policy 1, policy_version 22768 (0.0011) [2023-12-26 15:31:54,777][105692] Updated weights for policy 0, policy_version 22739 (0.0006) [2023-12-26 15:31:54,840][105692] Updated weights for policy 0, policy_version 22749 (0.0007) [2023-12-26 15:31:54,885][105692] Updated weights for policy 0, policy_version 22759 (0.0005) [2023-12-26 15:31:55,088][105620] Updated weights for policy 1, policy_version 22778 (0.0009) [2023-12-26 15:31:55,144][105620] Updated weights for policy 1, policy_version 22788 (0.0010) [2023-12-26 15:31:55,203][105620] Updated weights for policy 1, policy_version 22798 (0.0010) [2023-12-26 15:31:55,620][105692] Updated weights for policy 0, policy_version 22769 (0.0010) [2023-12-26 15:31:55,683][105692] Updated weights for policy 0, policy_version 22779 (0.0011) [2023-12-26 15:31:55,744][105692] Updated weights for policy 0, policy_version 22789 (0.0010) [2023-12-26 15:31:55,774][105620] Updated weights for policy 1, policy_version 22808 (0.0007) [2023-12-26 15:31:55,805][105692] Updated weights for policy 0, policy_version 22799 (0.0010) [2023-12-26 15:31:55,826][105620] Updated weights for policy 1, policy_version 22818 (0.0006) [2023-12-26 15:31:55,882][105620] Updated weights for policy 1, policy_version 22828 (0.0005) [2023-12-26 15:31:56,062][104569] Fps is (10 sec: 21298.5, 60 sec: 19933.7, 300 sec: 19660.8). Total num frames: 11689984. Throughput: 0: 9907.3, 1: 10075.0. Samples: 11694564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:31:56,064][104569] Avg episode reward: [(0, '9238.796'), (1, '8803.105')] [2023-12-26 15:31:56,439][105620] Updated weights for policy 1, policy_version 22838 (0.0006) [2023-12-26 15:31:56,501][105620] Updated weights for policy 1, policy_version 22848 (0.0009) [2023-12-26 15:31:56,525][105692] Updated weights for policy 0, policy_version 22809 (0.0008) [2023-12-26 15:31:56,562][105620] Updated weights for policy 1, policy_version 22858 (0.0009) [2023-12-26 15:31:56,586][105692] Updated weights for policy 0, policy_version 22819 (0.0010) [2023-12-26 15:31:56,635][105692] Updated weights for policy 0, policy_version 22829 (0.0010) [2023-12-26 15:31:57,163][105620] Updated weights for policy 1, policy_version 22868 (0.0007) [2023-12-26 15:31:57,225][105620] Updated weights for policy 1, policy_version 22878 (0.0005) [2023-12-26 15:31:57,270][105620] Updated weights for policy 1, policy_version 22888 (0.0005) [2023-12-26 15:31:57,386][105692] Updated weights for policy 0, policy_version 22839 (0.0010) [2023-12-26 15:31:57,437][105692] Updated weights for policy 0, policy_version 22849 (0.0010) [2023-12-26 15:31:57,488][105692] Updated weights for policy 0, policy_version 22859 (0.0010) [2023-12-26 15:31:57,799][105620] Updated weights for policy 1, policy_version 22898 (0.0006) [2023-12-26 15:31:57,861][105620] Updated weights for policy 1, policy_version 22908 (0.0008) [2023-12-26 15:31:57,915][105620] Updated weights for policy 1, policy_version 22918 (0.0005) [2023-12-26 15:31:57,986][105620] Updated weights for policy 1, policy_version 22928 (0.0006) [2023-12-26 15:31:58,253][105692] Updated weights for policy 0, policy_version 22869 (0.0010) [2023-12-26 15:31:58,315][105692] Updated weights for policy 0, policy_version 22879 (0.0007) [2023-12-26 15:31:58,384][105692] Updated weights for policy 0, policy_version 22889 (0.0008) [2023-12-26 15:31:58,755][105620] Updated weights for policy 1, policy_version 22938 (0.0007) [2023-12-26 15:31:58,821][105620] Updated weights for policy 1, policy_version 22948 (0.0010) [2023-12-26 15:31:58,882][105620] Updated weights for policy 1, policy_version 22958 (0.0010) [2023-12-26 15:31:59,195][105692] Updated weights for policy 0, policy_version 22899 (0.0008) [2023-12-26 15:31:59,254][105692] Updated weights for policy 0, policy_version 22909 (0.0008) [2023-12-26 15:31:59,314][105692] Updated weights for policy 0, policy_version 22919 (0.0009) [2023-12-26 15:31:59,595][105620] Updated weights for policy 1, policy_version 22968 (0.0008) [2023-12-26 15:31:59,647][105620] Updated weights for policy 1, policy_version 22979 (0.0007) [2023-12-26 15:31:59,706][105620] Updated weights for policy 1, policy_version 22989 (0.0006) [2023-12-26 15:32:00,018][105692] Updated weights for policy 0, policy_version 22929 (0.0009) [2023-12-26 15:32:00,075][105692] Updated weights for policy 0, policy_version 22939 (0.0011) [2023-12-26 15:32:00,138][105692] Updated weights for policy 0, policy_version 22949 (0.0011) [2023-12-26 15:32:00,197][105692] Updated weights for policy 0, policy_version 22959 (0.0010) [2023-12-26 15:32:00,383][105620] Updated weights for policy 1, policy_version 22999 (0.0006) [2023-12-26 15:32:00,438][105620] Updated weights for policy 1, policy_version 23009 (0.0008) [2023-12-26 15:32:00,495][105620] Updated weights for policy 1, policy_version 23019 (0.0005) [2023-12-26 15:32:00,902][105692] Updated weights for policy 0, policy_version 22969 (0.0009) [2023-12-26 15:32:00,953][105692] Updated weights for policy 0, policy_version 22979 (0.0010) [2023-12-26 15:32:01,000][105692] Updated weights for policy 0, policy_version 22989 (0.0010) [2023-12-26 15:32:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19688.6). Total num frames: 11788288. Throughput: 0: 9865.0, 1: 10181.8. Samples: 11755276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:32:01,062][104569] Avg episode reward: [(0, '9150.218'), (1, '8801.816')] [2023-12-26 15:32:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000022992_5890048.pth... [2023-12-26 15:32:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000023024_5898240.pth... [2023-12-26 15:32:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000021808_5586944.pth [2023-12-26 15:32:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000021840_5595136.pth [2023-12-26 15:32:01,188][105620] Updated weights for policy 1, policy_version 23029 (0.0007) [2023-12-26 15:32:01,238][105620] Updated weights for policy 1, policy_version 23040 (0.0009) [2023-12-26 15:32:01,289][105620] Updated weights for policy 1, policy_version 23050 (0.0008) [2023-12-26 15:32:01,742][105692] Updated weights for policy 0, policy_version 22999 (0.0008) [2023-12-26 15:32:01,800][105692] Updated weights for policy 0, policy_version 23009 (0.0005) [2023-12-26 15:32:01,855][105692] Updated weights for policy 0, policy_version 23019 (0.0005) [2023-12-26 15:32:02,077][105620] Updated weights for policy 1, policy_version 23060 (0.0007) [2023-12-26 15:32:02,123][105620] Updated weights for policy 1, policy_version 23070 (0.0008) [2023-12-26 15:32:02,167][105620] Updated weights for policy 1, policy_version 23080 (0.0008) [2023-12-26 15:32:02,512][105692] Updated weights for policy 0, policy_version 23029 (0.0005) [2023-12-26 15:32:02,573][105692] Updated weights for policy 0, policy_version 23039 (0.0010) [2023-12-26 15:32:02,637][105692] Updated weights for policy 0, policy_version 23049 (0.0010) [2023-12-26 15:32:02,871][105620] Updated weights for policy 1, policy_version 23090 (0.0006) [2023-12-26 15:32:02,931][105620] Updated weights for policy 1, policy_version 23100 (0.0005) [2023-12-26 15:32:02,997][105620] Updated weights for policy 1, policy_version 23110 (0.0007) [2023-12-26 15:32:03,055][105620] Updated weights for policy 1, policy_version 23120 (0.0007) [2023-12-26 15:32:03,355][105692] Updated weights for policy 0, policy_version 23059 (0.0010) [2023-12-26 15:32:03,422][105692] Updated weights for policy 0, policy_version 23069 (0.0010) [2023-12-26 15:32:03,469][105692] Updated weights for policy 0, policy_version 23079 (0.0009) [2023-12-26 15:32:03,705][105620] Updated weights for policy 1, policy_version 23130 (0.0005) [2023-12-26 15:32:03,768][105620] Updated weights for policy 1, policy_version 23140 (0.0006) [2023-12-26 15:32:03,821][105620] Updated weights for policy 1, policy_version 23150 (0.0006) [2023-12-26 15:32:04,166][105692] Updated weights for policy 0, policy_version 23089 (0.0010) [2023-12-26 15:32:04,218][105692] Updated weights for policy 0, policy_version 23099 (0.0010) [2023-12-26 15:32:04,277][105692] Updated weights for policy 0, policy_version 23109 (0.0010) [2023-12-26 15:32:04,341][105692] Updated weights for policy 0, policy_version 23119 (0.0010) [2023-12-26 15:32:04,459][105620] Updated weights for policy 1, policy_version 23160 (0.0008) [2023-12-26 15:32:04,518][105620] Updated weights for policy 1, policy_version 23170 (0.0010) [2023-12-26 15:32:04,569][105620] Updated weights for policy 1, policy_version 23180 (0.0010) [2023-12-26 15:32:05,051][105692] Updated weights for policy 0, policy_version 23129 (0.0007) [2023-12-26 15:32:05,103][105692] Updated weights for policy 0, policy_version 23139 (0.0008) [2023-12-26 15:32:05,153][105692] Updated weights for policy 0, policy_version 23149 (0.0009) [2023-12-26 15:32:05,253][105620] Updated weights for policy 1, policy_version 23190 (0.0007) [2023-12-26 15:32:05,306][105620] Updated weights for policy 1, policy_version 23200 (0.0007) [2023-12-26 15:32:05,370][105620] Updated weights for policy 1, policy_version 23210 (0.0006) [2023-12-26 15:32:05,939][105620] Updated weights for policy 1, policy_version 23220 (0.0007) [2023-12-26 15:32:05,957][105692] Updated weights for policy 0, policy_version 23159 (0.0007) [2023-12-26 15:32:05,999][105620] Updated weights for policy 1, policy_version 23230 (0.0009) [2023-12-26 15:32:06,009][105692] Updated weights for policy 0, policy_version 23169 (0.0005) [2023-12-26 15:32:06,054][105620] Updated weights for policy 1, policy_version 23240 (0.0010) [2023-12-26 15:32:06,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 11878400. Throughput: 0: 9861.9, 1: 10203.0. Samples: 11874528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:32:06,062][104569] Avg episode reward: [(0, '9155.373'), (1, '8707.648')] [2023-12-26 15:32:06,064][105692] Updated weights for policy 0, policy_version 23179 (0.0006) [2023-12-26 15:32:06,717][105620] Updated weights for policy 1, policy_version 23250 (0.0009) [2023-12-26 15:32:06,772][105620] Updated weights for policy 1, policy_version 23260 (0.0005) [2023-12-26 15:32:06,817][105620] Updated weights for policy 1, policy_version 23270 (0.0005) [2023-12-26 15:32:06,872][105620] Updated weights for policy 1, policy_version 23280 (0.0006) [2023-12-26 15:32:06,893][105692] Updated weights for policy 0, policy_version 23189 (0.0009) [2023-12-26 15:32:06,956][105692] Updated weights for policy 0, policy_version 23199 (0.0008) [2023-12-26 15:32:07,016][105692] Updated weights for policy 0, policy_version 23209 (0.0008) [2023-12-26 15:32:07,566][105620] Updated weights for policy 1, policy_version 23290 (0.0007) [2023-12-26 15:32:07,617][105620] Updated weights for policy 1, policy_version 23300 (0.0007) [2023-12-26 15:32:07,670][105620] Updated weights for policy 1, policy_version 23310 (0.0005) [2023-12-26 15:32:07,766][105692] Updated weights for policy 0, policy_version 23219 (0.0008) [2023-12-26 15:32:07,819][105692] Updated weights for policy 0, policy_version 23229 (0.0009) [2023-12-26 15:32:07,873][105692] Updated weights for policy 0, policy_version 23239 (0.0010) [2023-12-26 15:32:08,250][105620] Updated weights for policy 1, policy_version 23320 (0.0005) [2023-12-26 15:32:08,304][105620] Updated weights for policy 1, policy_version 23330 (0.0006) [2023-12-26 15:32:08,365][105620] Updated weights for policy 1, policy_version 23340 (0.0009) [2023-12-26 15:32:08,622][105692] Updated weights for policy 0, policy_version 23249 (0.0009) [2023-12-26 15:32:08,676][105692] Updated weights for policy 0, policy_version 23259 (0.0009) [2023-12-26 15:32:08,724][105692] Updated weights for policy 0, policy_version 23269 (0.0009) [2023-12-26 15:32:08,780][105692] Updated weights for policy 0, policy_version 23279 (0.0009) [2023-12-26 15:32:09,081][105620] Updated weights for policy 1, policy_version 23350 (0.0009) [2023-12-26 15:32:09,131][105620] Updated weights for policy 1, policy_version 23360 (0.0008) [2023-12-26 15:32:09,181][105620] Updated weights for policy 1, policy_version 23370 (0.0007) [2023-12-26 15:32:09,613][105692] Updated weights for policy 0, policy_version 23289 (0.0008) [2023-12-26 15:32:09,674][105692] Updated weights for policy 0, policy_version 23299 (0.0008) [2023-12-26 15:32:09,734][105692] Updated weights for policy 0, policy_version 23309 (0.0008) [2023-12-26 15:32:09,976][105620] Updated weights for policy 1, policy_version 23380 (0.0009) [2023-12-26 15:32:10,039][105620] Updated weights for policy 1, policy_version 23390 (0.0008) [2023-12-26 15:32:10,098][105620] Updated weights for policy 1, policy_version 23400 (0.0009) [2023-12-26 15:32:10,525][105692] Updated weights for policy 0, policy_version 23319 (0.0009) [2023-12-26 15:32:10,584][105692] Updated weights for policy 0, policy_version 23329 (0.0008) [2023-12-26 15:32:10,647][105692] Updated weights for policy 0, policy_version 23339 (0.0009) [2023-12-26 15:32:10,878][105620] Updated weights for policy 1, policy_version 23410 (0.0009) [2023-12-26 15:32:10,932][105620] Updated weights for policy 1, policy_version 23420 (0.0009) [2023-12-26 15:32:10,983][105620] Updated weights for policy 1, policy_version 23430 (0.0008) [2023-12-26 15:32:11,033][105620] Updated weights for policy 1, policy_version 23440 (0.0008) [2023-12-26 15:32:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 20070.4, 300 sec: 19716.3). Total num frames: 11984896. Throughput: 0: 9832.2, 1: 10210.1. Samples: 11989744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:32:11,062][104569] Avg episode reward: [(0, '9065.268'), (1, '8712.230')] [2023-12-26 15:32:11,345][105692] Updated weights for policy 0, policy_version 23349 (0.0009) [2023-12-26 15:32:11,417][105692] Updated weights for policy 0, policy_version 23359 (0.0010) [2023-12-26 15:32:11,478][105692] Updated weights for policy 0, policy_version 23369 (0.0007) [2023-12-26 15:32:11,887][105620] Updated weights for policy 1, policy_version 23450 (0.0009) [2023-12-26 15:32:11,953][105620] Updated weights for policy 1, policy_version 23460 (0.0009) [2023-12-26 15:32:12,010][105620] Updated weights for policy 1, policy_version 23470 (0.0008) [2023-12-26 15:32:12,247][105692] Updated weights for policy 0, policy_version 23379 (0.0008) [2023-12-26 15:32:12,315][105692] Updated weights for policy 0, policy_version 23389 (0.0009) [2023-12-26 15:32:12,376][105692] Updated weights for policy 0, policy_version 23399 (0.0011) [2023-12-26 15:32:12,787][105620] Updated weights for policy 1, policy_version 23480 (0.0007) [2023-12-26 15:32:12,837][105620] Updated weights for policy 1, policy_version 23490 (0.0005) [2023-12-26 15:32:12,888][105620] Updated weights for policy 1, policy_version 23500 (0.0006) [2023-12-26 15:32:13,113][105692] Updated weights for policy 0, policy_version 23409 (0.0011) [2023-12-26 15:32:13,169][105692] Updated weights for policy 0, policy_version 23420 (0.0010) [2023-12-26 15:32:13,228][105692] Updated weights for policy 0, policy_version 23430 (0.0010) [2023-12-26 15:32:13,283][105692] Updated weights for policy 0, policy_version 23440 (0.0010) [2023-12-26 15:32:13,505][105620] Updated weights for policy 1, policy_version 23510 (0.0006) [2023-12-26 15:32:13,574][105620] Updated weights for policy 1, policy_version 23520 (0.0005) [2023-12-26 15:32:13,643][105620] Updated weights for policy 1, policy_version 23530 (0.0005) [2023-12-26 15:32:13,886][105692] Updated weights for policy 0, policy_version 23450 (0.0008) [2023-12-26 15:32:13,945][105692] Updated weights for policy 0, policy_version 23460 (0.0006) [2023-12-26 15:32:13,998][105692] Updated weights for policy 0, policy_version 23470 (0.0005) [2023-12-26 15:32:14,232][105620] Updated weights for policy 1, policy_version 23540 (0.0005) [2023-12-26 15:32:14,283][105620] Updated weights for policy 1, policy_version 23550 (0.0005) [2023-12-26 15:32:14,343][105620] Updated weights for policy 1, policy_version 23560 (0.0005) [2023-12-26 15:32:14,577][105692] Updated weights for policy 0, policy_version 23480 (0.0010) [2023-12-26 15:32:14,635][105692] Updated weights for policy 0, policy_version 23490 (0.0010) [2023-12-26 15:32:14,680][105692] Updated weights for policy 0, policy_version 23500 (0.0007) [2023-12-26 15:32:15,015][105620] Updated weights for policy 1, policy_version 23570 (0.0006) [2023-12-26 15:32:15,065][105620] Updated weights for policy 1, policy_version 23580 (0.0009) [2023-12-26 15:32:15,121][105620] Updated weights for policy 1, policy_version 23590 (0.0009) [2023-12-26 15:32:15,171][105620] Updated weights for policy 1, policy_version 23600 (0.0009) [2023-12-26 15:32:15,365][105692] Updated weights for policy 0, policy_version 23510 (0.0007) [2023-12-26 15:32:15,416][105692] Updated weights for policy 0, policy_version 23520 (0.0007) [2023-12-26 15:32:15,473][105692] Updated weights for policy 0, policy_version 23530 (0.0011) [2023-12-26 15:32:15,862][105620] Updated weights for policy 1, policy_version 23610 (0.0010) [2023-12-26 15:32:15,931][105620] Updated weights for policy 1, policy_version 23620 (0.0009) [2023-12-26 15:32:15,983][105620] Updated weights for policy 1, policy_version 23630 (0.0007) [2023-12-26 15:32:16,062][104569] Fps is (10 sec: 20479.5, 60 sec: 20070.4, 300 sec: 19688.6). Total num frames: 12083200. Throughput: 0: 9842.5, 1: 10154.6. Samples: 12048644. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:32:16,063][104569] Avg episode reward: [(0, '9065.093'), (1, '9083.087')] [2023-12-26 15:32:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000023536_6029312.pth... [2023-12-26 15:32:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000023632_6053888.pth... [2023-12-26 15:32:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000022416_5742592.pth [2023-12-26 15:32:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000022416_5742592.pth [2023-12-26 15:32:16,078][105586] Saving new best policy, reward=9083.087! [2023-12-26 15:32:16,225][105692] Updated weights for policy 0, policy_version 23540 (0.0009) [2023-12-26 15:32:16,280][105692] Updated weights for policy 0, policy_version 23550 (0.0008) [2023-12-26 15:32:16,338][105692] Updated weights for policy 0, policy_version 23560 (0.0009) [2023-12-26 15:32:16,656][105620] Updated weights for policy 1, policy_version 23640 (0.0009) [2023-12-26 15:32:16,702][105620] Updated weights for policy 1, policy_version 23650 (0.0008) [2023-12-26 15:32:16,755][105620] Updated weights for policy 1, policy_version 23660 (0.0008) [2023-12-26 15:32:16,913][105692] Updated weights for policy 0, policy_version 23570 (0.0008) [2023-12-26 15:32:16,965][105692] Updated weights for policy 0, policy_version 23581 (0.0007) [2023-12-26 15:32:17,021][105692] Updated weights for policy 0, policy_version 23591 (0.0009) [2023-12-26 15:32:17,342][105620] Updated weights for policy 1, policy_version 23670 (0.0007) [2023-12-26 15:32:17,406][105620] Updated weights for policy 1, policy_version 23680 (0.0005) [2023-12-26 15:32:17,467][105620] Updated weights for policy 1, policy_version 23690 (0.0005) [2023-12-26 15:32:17,834][105692] Updated weights for policy 0, policy_version 23601 (0.0009) [2023-12-26 15:32:17,896][105692] Updated weights for policy 0, policy_version 23611 (0.0006) [2023-12-26 15:32:17,945][105692] Updated weights for policy 0, policy_version 23621 (0.0006) [2023-12-26 15:32:17,991][105692] Updated weights for policy 0, policy_version 23631 (0.0005) [2023-12-26 15:32:18,009][105620] Updated weights for policy 1, policy_version 23700 (0.0008) [2023-12-26 15:32:18,064][105620] Updated weights for policy 1, policy_version 23710 (0.0010) [2023-12-26 15:32:18,108][105620] Updated weights for policy 1, policy_version 23720 (0.0009) [2023-12-26 15:32:18,695][105692] Updated weights for policy 0, policy_version 23641 (0.0008) [2023-12-26 15:32:18,742][105692] Updated weights for policy 0, policy_version 23651 (0.0009) [2023-12-26 15:32:18,796][105692] Updated weights for policy 0, policy_version 23661 (0.0010) [2023-12-26 15:32:18,814][105620] Updated weights for policy 1, policy_version 23730 (0.0008) [2023-12-26 15:32:18,861][105620] Updated weights for policy 1, policy_version 23740 (0.0005) [2023-12-26 15:32:18,916][105620] Updated weights for policy 1, policy_version 23750 (0.0005) [2023-12-26 15:32:18,971][105620] Updated weights for policy 1, policy_version 23760 (0.0006) [2023-12-26 15:32:19,573][105692] Updated weights for policy 0, policy_version 23671 (0.0010) [2023-12-26 15:32:19,646][105692] Updated weights for policy 0, policy_version 23681 (0.0011) [2023-12-26 15:32:19,710][105692] Updated weights for policy 0, policy_version 23691 (0.0011) [2023-12-26 15:32:19,717][105620] Updated weights for policy 1, policy_version 23770 (0.0006) [2023-12-26 15:32:19,773][105620] Updated weights for policy 1, policy_version 23780 (0.0009) [2023-12-26 15:32:19,835][105620] Updated weights for policy 1, policy_version 23790 (0.0008) [2023-12-26 15:32:20,418][105692] Updated weights for policy 0, policy_version 23701 (0.0010) [2023-12-26 15:32:20,470][105692] Updated weights for policy 0, policy_version 23711 (0.0009) [2023-12-26 15:32:20,517][105692] Updated weights for policy 0, policy_version 23721 (0.0009) [2023-12-26 15:32:20,600][105620] Updated weights for policy 1, policy_version 23800 (0.0009) [2023-12-26 15:32:20,668][105620] Updated weights for policy 1, policy_version 23810 (0.0007) [2023-12-26 15:32:20,734][105620] Updated weights for policy 1, policy_version 23820 (0.0007) [2023-12-26 15:32:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 20070.4, 300 sec: 19688.6). Total num frames: 12181504. Throughput: 0: 9791.9, 1: 10223.4. Samples: 12171440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:32:21,062][104569] Avg episode reward: [(0, '9248.810'), (1, '9171.211')] [2023-12-26 15:32:21,063][105586] Saving new best policy, reward=9171.211! [2023-12-26 15:32:21,303][105692] Updated weights for policy 0, policy_version 23731 (0.0008) [2023-12-26 15:32:21,361][105692] Updated weights for policy 0, policy_version 23741 (0.0009) [2023-12-26 15:32:21,428][105692] Updated weights for policy 0, policy_version 23751 (0.0007) [2023-12-26 15:32:21,447][105620] Updated weights for policy 1, policy_version 23830 (0.0009) [2023-12-26 15:32:21,502][105620] Updated weights for policy 1, policy_version 23840 (0.0009) [2023-12-26 15:32:21,564][105620] Updated weights for policy 1, policy_version 23850 (0.0009) [2023-12-26 15:32:22,199][105692] Updated weights for policy 0, policy_version 23761 (0.0007) [2023-12-26 15:32:22,249][105692] Updated weights for policy 0, policy_version 23771 (0.0008) [2023-12-26 15:32:22,305][105692] Updated weights for policy 0, policy_version 23781 (0.0009) [2023-12-26 15:32:22,347][105620] Updated weights for policy 1, policy_version 23860 (0.0008) [2023-12-26 15:32:22,370][105692] Updated weights for policy 0, policy_version 23791 (0.0008) [2023-12-26 15:32:22,407][105620] Updated weights for policy 1, policy_version 23870 (0.0009) [2023-12-26 15:32:22,461][105620] Updated weights for policy 1, policy_version 23880 (0.0009) [2023-12-26 15:32:23,146][105692] Updated weights for policy 0, policy_version 23801 (0.0007) [2023-12-26 15:32:23,198][105692] Updated weights for policy 0, policy_version 23811 (0.0006) [2023-12-26 15:32:23,244][105692] Updated weights for policy 0, policy_version 23821 (0.0005) [2023-12-26 15:32:23,252][105620] Updated weights for policy 1, policy_version 23890 (0.0009) [2023-12-26 15:32:23,303][105620] Updated weights for policy 1, policy_version 23900 (0.0009) [2023-12-26 15:32:23,356][105620] Updated weights for policy 1, policy_version 23911 (0.0010) [2023-12-26 15:32:23,806][105692] Updated weights for policy 0, policy_version 23831 (0.0007) [2023-12-26 15:32:23,853][105692] Updated weights for policy 0, policy_version 23841 (0.0009) [2023-12-26 15:32:23,899][105692] Updated weights for policy 0, policy_version 23851 (0.0009) [2023-12-26 15:32:24,203][105620] Updated weights for policy 1, policy_version 23921 (0.0010) [2023-12-26 15:32:24,269][105620] Updated weights for policy 1, policy_version 23931 (0.0010) [2023-12-26 15:32:24,330][105620] Updated weights for policy 1, policy_version 23941 (0.0009) [2023-12-26 15:32:24,392][105620] Updated weights for policy 1, policy_version 23951 (0.0009) [2023-12-26 15:32:24,634][105692] Updated weights for policy 0, policy_version 23861 (0.0008) [2023-12-26 15:32:24,693][105692] Updated weights for policy 0, policy_version 23871 (0.0005) [2023-12-26 15:32:24,746][105692] Updated weights for policy 0, policy_version 23881 (0.0005) [2023-12-26 15:32:25,244][105620] Updated weights for policy 1, policy_version 23961 (0.0008) [2023-12-26 15:32:25,297][105692] Updated weights for policy 0, policy_version 23891 (0.0009) [2023-12-26 15:32:25,298][105620] Updated weights for policy 1, policy_version 23971 (0.0009) [2023-12-26 15:32:25,346][105692] Updated weights for policy 0, policy_version 23901 (0.0007) [2023-12-26 15:32:25,363][105620] Updated weights for policy 1, policy_version 23981 (0.0007) [2023-12-26 15:32:25,404][105692] Updated weights for policy 0, policy_version 23911 (0.0007) [2023-12-26 15:32:25,991][105692] Updated weights for policy 0, policy_version 23921 (0.0006) [2023-12-26 15:32:26,052][105692] Updated weights for policy 0, policy_version 23931 (0.0005) [2023-12-26 15:32:26,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19933.9, 300 sec: 19660.8). Total num frames: 12271616. Throughput: 0: 9771.9, 1: 10029.0. Samples: 12285004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:32:26,063][104569] Avg episode reward: [(0, '9342.419'), (1, '8892.691')] [2023-12-26 15:32:26,084][105620] Updated weights for policy 1, policy_version 23991 (0.0006) [2023-12-26 15:32:26,113][105692] Updated weights for policy 0, policy_version 23941 (0.0006) [2023-12-26 15:32:26,133][105620] Updated weights for policy 1, policy_version 24001 (0.0006) [2023-12-26 15:32:26,166][105692] Updated weights for policy 0, policy_version 23951 (0.0006) [2023-12-26 15:32:26,178][105620] Updated weights for policy 1, policy_version 24011 (0.0005) [2023-12-26 15:32:26,807][105692] Updated weights for policy 0, policy_version 23961 (0.0006) [2023-12-26 15:32:26,835][105620] Updated weights for policy 1, policy_version 24021 (0.0005) [2023-12-26 15:32:26,869][105692] Updated weights for policy 0, policy_version 23971 (0.0008) [2023-12-26 15:32:26,892][105620] Updated weights for policy 1, policy_version 24031 (0.0009) [2023-12-26 15:32:26,929][105692] Updated weights for policy 0, policy_version 23981 (0.0009) [2023-12-26 15:32:26,945][105620] Updated weights for policy 1, policy_version 24041 (0.0007) [2023-12-26 15:32:27,587][105620] Updated weights for policy 1, policy_version 24051 (0.0008) [2023-12-26 15:32:27,631][105620] Updated weights for policy 1, policy_version 24061 (0.0010) [2023-12-26 15:32:27,678][105620] Updated weights for policy 1, policy_version 24071 (0.0009) [2023-12-26 15:32:27,700][105692] Updated weights for policy 0, policy_version 23991 (0.0008) [2023-12-26 15:32:27,754][105692] Updated weights for policy 0, policy_version 24001 (0.0007) [2023-12-26 15:32:27,806][105692] Updated weights for policy 0, policy_version 24012 (0.0010) [2023-12-26 15:32:28,294][105620] Updated weights for policy 1, policy_version 24081 (0.0008) [2023-12-26 15:32:28,349][105620] Updated weights for policy 1, policy_version 24091 (0.0008) [2023-12-26 15:32:28,401][105620] Updated weights for policy 1, policy_version 24101 (0.0010) [2023-12-26 15:32:28,454][105620] Updated weights for policy 1, policy_version 24111 (0.0008) [2023-12-26 15:32:28,587][105692] Updated weights for policy 0, policy_version 24022 (0.0007) [2023-12-26 15:32:28,633][105692] Updated weights for policy 0, policy_version 24032 (0.0007) [2023-12-26 15:32:28,695][105692] Updated weights for policy 0, policy_version 24042 (0.0008) [2023-12-26 15:32:29,028][105620] Updated weights for policy 1, policy_version 24121 (0.0010) [2023-12-26 15:32:29,080][105620] Updated weights for policy 1, policy_version 24131 (0.0009) [2023-12-26 15:32:29,140][105620] Updated weights for policy 1, policy_version 24141 (0.0007) [2023-12-26 15:32:29,479][105692] Updated weights for policy 0, policy_version 24052 (0.0010) [2023-12-26 15:32:29,537][105692] Updated weights for policy 0, policy_version 24062 (0.0010) [2023-12-26 15:32:29,588][105692] Updated weights for policy 0, policy_version 24072 (0.0010) [2023-12-26 15:32:29,825][105620] Updated weights for policy 1, policy_version 24151 (0.0006) [2023-12-26 15:32:29,889][105620] Updated weights for policy 1, policy_version 24161 (0.0007) [2023-12-26 15:32:29,954][105620] Updated weights for policy 1, policy_version 24171 (0.0010) [2023-12-26 15:32:30,322][105692] Updated weights for policy 0, policy_version 24082 (0.0009) [2023-12-26 15:32:30,382][105692] Updated weights for policy 0, policy_version 24092 (0.0008) [2023-12-26 15:32:30,444][105692] Updated weights for policy 0, policy_version 24102 (0.0008) [2023-12-26 15:32:30,506][105692] Updated weights for policy 0, policy_version 24112 (0.0008) [2023-12-26 15:32:30,621][105620] Updated weights for policy 1, policy_version 24181 (0.0008) [2023-12-26 15:32:30,667][105620] Updated weights for policy 1, policy_version 24191 (0.0005) [2023-12-26 15:32:30,722][105620] Updated weights for policy 1, policy_version 24201 (0.0005) [2023-12-26 15:32:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19933.8, 300 sec: 19688.6). Total num frames: 12378112. Throughput: 0: 9825.2, 1: 10152.0. Samples: 12348636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:32:31,062][104569] Avg episode reward: [(0, '9065.037'), (1, '8526.787')] [2023-12-26 15:32:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000024208_6201344.pth... [2023-12-26 15:32:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000023024_5898240.pth [2023-12-26 15:32:31,088][105692] Updated weights for policy 0, policy_version 24122 (0.0006) [2023-12-26 15:32:31,157][105692] Updated weights for policy 0, policy_version 24132 (0.0007) [2023-12-26 15:32:31,214][105692] Updated weights for policy 0, policy_version 24142 (0.0007) [2023-12-26 15:32:31,226][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000024144_6184960.pth... [2023-12-26 15:32:31,230][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000022992_5890048.pth [2023-12-26 15:32:31,348][105620] Updated weights for policy 1, policy_version 24211 (0.0006) [2023-12-26 15:32:31,420][105620] Updated weights for policy 1, policy_version 24221 (0.0011) [2023-12-26 15:32:31,468][105620] Updated weights for policy 1, policy_version 24231 (0.0010) [2023-12-26 15:32:31,898][105692] Updated weights for policy 0, policy_version 24152 (0.0008) [2023-12-26 15:32:31,962][105692] Updated weights for policy 0, policy_version 24162 (0.0008) [2023-12-26 15:32:32,024][105692] Updated weights for policy 0, policy_version 24172 (0.0008) [2023-12-26 15:32:32,190][105620] Updated weights for policy 1, policy_version 24241 (0.0010) [2023-12-26 15:32:32,249][105620] Updated weights for policy 1, policy_version 24251 (0.0009) [2023-12-26 15:32:32,310][105620] Updated weights for policy 1, policy_version 24261 (0.0007) [2023-12-26 15:32:32,374][105620] Updated weights for policy 1, policy_version 24271 (0.0007) [2023-12-26 15:32:32,706][105692] Updated weights for policy 0, policy_version 24182 (0.0010) [2023-12-26 15:32:32,761][105692] Updated weights for policy 0, policy_version 24194 (0.0011) [2023-12-26 15:32:32,814][105692] Updated weights for policy 0, policy_version 24204 (0.0009) [2023-12-26 15:32:33,002][105620] Updated weights for policy 1, policy_version 24281 (0.0005) [2023-12-26 15:32:33,062][105620] Updated weights for policy 1, policy_version 24291 (0.0005) [2023-12-26 15:32:33,131][105620] Updated weights for policy 1, policy_version 24301 (0.0005) [2023-12-26 15:32:33,591][105692] Updated weights for policy 0, policy_version 24214 (0.0008) [2023-12-26 15:32:33,635][105692] Updated weights for policy 0, policy_version 24224 (0.0008) [2023-12-26 15:32:33,689][105692] Updated weights for policy 0, policy_version 24234 (0.0008) [2023-12-26 15:32:33,715][105620] Updated weights for policy 1, policy_version 24311 (0.0009) [2023-12-26 15:32:33,778][105620] Updated weights for policy 1, policy_version 24321 (0.0010) [2023-12-26 15:32:33,847][105620] Updated weights for policy 1, policy_version 24331 (0.0010) [2023-12-26 15:32:34,381][105692] Updated weights for policy 0, policy_version 24244 (0.0007) [2023-12-26 15:32:34,429][105692] Updated weights for policy 0, policy_version 24254 (0.0007) [2023-12-26 15:32:34,473][105692] Updated weights for policy 0, policy_version 24264 (0.0005) [2023-12-26 15:32:34,581][105620] Updated weights for policy 1, policy_version 24341 (0.0009) [2023-12-26 15:32:34,642][105620] Updated weights for policy 1, policy_version 24351 (0.0009) [2023-12-26 15:32:34,706][105620] Updated weights for policy 1, policy_version 24361 (0.0008) [2023-12-26 15:32:35,176][105692] Updated weights for policy 0, policy_version 24274 (0.0006) [2023-12-26 15:32:35,237][105692] Updated weights for policy 0, policy_version 24284 (0.0005) [2023-12-26 15:32:35,301][105692] Updated weights for policy 0, policy_version 24294 (0.0005) [2023-12-26 15:32:35,353][105692] Updated weights for policy 0, policy_version 24304 (0.0010) [2023-12-26 15:32:35,419][105620] Updated weights for policy 1, policy_version 24371 (0.0008) [2023-12-26 15:32:35,468][105620] Updated weights for policy 1, policy_version 24381 (0.0005) [2023-12-26 15:32:35,512][105620] Updated weights for policy 1, policy_version 24391 (0.0005) [2023-12-26 15:32:35,911][105692] Updated weights for policy 0, policy_version 24314 (0.0009) [2023-12-26 15:32:35,955][105692] Updated weights for policy 0, policy_version 24324 (0.0007) [2023-12-26 15:32:36,012][105692] Updated weights for policy 0, policy_version 24334 (0.0007) [2023-12-26 15:32:36,062][104569] Fps is (10 sec: 21299.3, 60 sec: 20070.5, 300 sec: 19716.3). Total num frames: 12484608. Throughput: 0: 9747.6, 1: 10129.9. Samples: 12468960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:32:36,062][104569] Avg episode reward: [(0, '9061.009'), (1, '8526.084')] [2023-12-26 15:32:36,170][105620] Updated weights for policy 1, policy_version 24401 (0.0008) [2023-12-26 15:32:36,232][105620] Updated weights for policy 1, policy_version 24411 (0.0006) [2023-12-26 15:32:36,294][105620] Updated weights for policy 1, policy_version 24421 (0.0005) [2023-12-26 15:32:36,360][105620] Updated weights for policy 1, policy_version 24431 (0.0005) [2023-12-26 15:32:36,788][105692] Updated weights for policy 0, policy_version 24344 (0.0009) [2023-12-26 15:32:36,848][105692] Updated weights for policy 0, policy_version 24354 (0.0008) [2023-12-26 15:32:36,902][105692] Updated weights for policy 0, policy_version 24364 (0.0008) [2023-12-26 15:32:36,958][105620] Updated weights for policy 1, policy_version 24441 (0.0007) [2023-12-26 15:32:37,016][105620] Updated weights for policy 1, policy_version 24451 (0.0006) [2023-12-26 15:32:37,073][105620] Updated weights for policy 1, policy_version 24461 (0.0007) [2023-12-26 15:32:37,654][105692] Updated weights for policy 0, policy_version 24374 (0.0008) [2023-12-26 15:32:37,714][105692] Updated weights for policy 0, policy_version 24384 (0.0010) [2023-12-26 15:32:37,746][105620] Updated weights for policy 1, policy_version 24471 (0.0007) [2023-12-26 15:32:37,768][105692] Updated weights for policy 0, policy_version 24394 (0.0009) [2023-12-26 15:32:37,804][105620] Updated weights for policy 1, policy_version 24481 (0.0005) [2023-12-26 15:32:37,862][105620] Updated weights for policy 1, policy_version 24491 (0.0009) [2023-12-26 15:32:38,464][105620] Updated weights for policy 1, policy_version 24501 (0.0009) [2023-12-26 15:32:38,518][105620] Updated weights for policy 1, policy_version 24511 (0.0010) [2023-12-26 15:32:38,524][105692] Updated weights for policy 0, policy_version 24404 (0.0008) [2023-12-26 15:32:38,572][105620] Updated weights for policy 1, policy_version 24521 (0.0007) [2023-12-26 15:32:38,586][105692] Updated weights for policy 0, policy_version 24414 (0.0007) [2023-12-26 15:32:38,646][105692] Updated weights for policy 0, policy_version 24424 (0.0007) [2023-12-26 15:32:39,348][105620] Updated weights for policy 1, policy_version 24531 (0.0007) [2023-12-26 15:32:39,420][105620] Updated weights for policy 1, policy_version 24541 (0.0008) [2023-12-26 15:32:39,424][105692] Updated weights for policy 0, policy_version 24434 (0.0009) [2023-12-26 15:32:39,492][105620] Updated weights for policy 1, policy_version 24551 (0.0006) [2023-12-26 15:32:39,492][105692] Updated weights for policy 0, policy_version 24444 (0.0010) [2023-12-26 15:32:39,558][105692] Updated weights for policy 0, policy_version 24454 (0.0009) [2023-12-26 15:32:39,613][105692] Updated weights for policy 0, policy_version 24464 (0.0006) [2023-12-26 15:32:40,261][105620] Updated weights for policy 1, policy_version 24561 (0.0008) [2023-12-26 15:32:40,274][105692] Updated weights for policy 0, policy_version 24474 (0.0006) [2023-12-26 15:32:40,328][105620] Updated weights for policy 1, policy_version 24571 (0.0007) [2023-12-26 15:32:40,334][105692] Updated weights for policy 0, policy_version 24484 (0.0008) [2023-12-26 15:32:40,387][105620] Updated weights for policy 1, policy_version 24581 (0.0005) [2023-12-26 15:32:40,397][105692] Updated weights for policy 0, policy_version 24494 (0.0011) [2023-12-26 15:32:40,446][105620] Updated weights for policy 1, policy_version 24591 (0.0008) [2023-12-26 15:32:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19933.9, 300 sec: 19716.3). Total num frames: 12574720. Throughput: 0: 9737.4, 1: 10107.4. Samples: 12587572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:32:41,062][104569] Avg episode reward: [(0, '8873.108'), (1, '8892.786')] [2023-12-26 15:32:41,076][105692] Updated weights for policy 0, policy_version 24504 (0.0009) [2023-12-26 15:32:41,142][105692] Updated weights for policy 0, policy_version 24514 (0.0008) [2023-12-26 15:32:41,215][105692] Updated weights for policy 0, policy_version 24524 (0.0010) [2023-12-26 15:32:41,252][105620] Updated weights for policy 1, policy_version 24601 (0.0007) [2023-12-26 15:32:41,315][105620] Updated weights for policy 1, policy_version 24611 (0.0009) [2023-12-26 15:32:41,382][105620] Updated weights for policy 1, policy_version 24621 (0.0009) [2023-12-26 15:32:42,031][105692] Updated weights for policy 0, policy_version 24534 (0.0008) [2023-12-26 15:32:42,096][105620] Updated weights for policy 1, policy_version 24631 (0.0007) [2023-12-26 15:32:42,098][105692] Updated weights for policy 0, policy_version 24544 (0.0008) [2023-12-26 15:32:42,157][105692] Updated weights for policy 0, policy_version 24554 (0.0008) [2023-12-26 15:32:42,158][105620] Updated weights for policy 1, policy_version 24641 (0.0006) [2023-12-26 15:32:42,213][105620] Updated weights for policy 1, policy_version 24651 (0.0009) [2023-12-26 15:32:42,891][105692] Updated weights for policy 0, policy_version 24564 (0.0007) [2023-12-26 15:32:42,946][105692] Updated weights for policy 0, policy_version 24574 (0.0009) [2023-12-26 15:32:42,994][105692] Updated weights for policy 0, policy_version 24584 (0.0007) [2023-12-26 15:32:43,000][105620] Updated weights for policy 1, policy_version 24661 (0.0009) [2023-12-26 15:32:43,064][105620] Updated weights for policy 1, policy_version 24671 (0.0008) [2023-12-26 15:32:43,116][105620] Updated weights for policy 1, policy_version 24681 (0.0009) [2023-12-26 15:32:43,656][105692] Updated weights for policy 0, policy_version 24594 (0.0006) [2023-12-26 15:32:43,710][105692] Updated weights for policy 0, policy_version 24604 (0.0008) [2023-12-26 15:32:43,771][105692] Updated weights for policy 0, policy_version 24614 (0.0009) [2023-12-26 15:32:43,837][105692] Updated weights for policy 0, policy_version 24624 (0.0009) [2023-12-26 15:32:43,908][105620] Updated weights for policy 1, policy_version 24691 (0.0009) [2023-12-26 15:32:43,960][105620] Updated weights for policy 1, policy_version 24701 (0.0009) [2023-12-26 15:32:44,021][105620] Updated weights for policy 1, policy_version 24711 (0.0009) [2023-12-26 15:32:44,562][105692] Updated weights for policy 0, policy_version 24634 (0.0008) [2023-12-26 15:32:44,625][105692] Updated weights for policy 0, policy_version 24644 (0.0009) [2023-12-26 15:32:44,687][105692] Updated weights for policy 0, policy_version 24654 (0.0009) [2023-12-26 15:32:44,777][105620] Updated weights for policy 1, policy_version 24721 (0.0009) [2023-12-26 15:32:44,844][105620] Updated weights for policy 1, policy_version 24731 (0.0009) [2023-12-26 15:32:44,899][105620] Updated weights for policy 1, policy_version 24741 (0.0009) [2023-12-26 15:32:44,951][105620] Updated weights for policy 1, policy_version 24751 (0.0009) [2023-12-26 15:32:45,461][105692] Updated weights for policy 0, policy_version 24664 (0.0009) [2023-12-26 15:32:45,517][105692] Updated weights for policy 0, policy_version 24674 (0.0009) [2023-12-26 15:32:45,570][105692] Updated weights for policy 0, policy_version 24684 (0.0007) [2023-12-26 15:32:45,698][105620] Updated weights for policy 1, policy_version 24761 (0.0009) [2023-12-26 15:32:45,746][105620] Updated weights for policy 1, policy_version 24771 (0.0009) [2023-12-26 15:32:45,794][105620] Updated weights for policy 1, policy_version 24781 (0.0009) [2023-12-26 15:32:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19933.9, 300 sec: 19744.1). Total num frames: 12673024. Throughput: 0: 9746.0, 1: 9985.0. Samples: 12643172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:32:46,062][104569] Avg episode reward: [(0, '8007.158'), (1, '8986.764')] [2023-12-26 15:32:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000024688_6324224.pth... [2023-12-26 15:32:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000024784_6348800.pth... [2023-12-26 15:32:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000023536_6029312.pth [2023-12-26 15:32:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000023632_6053888.pth [2023-12-26 15:32:46,215][105692] Updated weights for policy 0, policy_version 24694 (0.0009) [2023-12-26 15:32:46,269][105692] Updated weights for policy 0, policy_version 24704 (0.0009) [2023-12-26 15:32:46,328][105692] Updated weights for policy 0, policy_version 24714 (0.0009) [2023-12-26 15:32:46,619][105620] Updated weights for policy 1, policy_version 24791 (0.0007) [2023-12-26 15:32:46,690][105620] Updated weights for policy 1, policy_version 24801 (0.0006) [2023-12-26 15:32:46,738][105620] Updated weights for policy 1, policy_version 24811 (0.0009) [2023-12-26 15:32:47,012][105692] Updated weights for policy 0, policy_version 24724 (0.0007) [2023-12-26 15:32:47,065][105692] Updated weights for policy 0, policy_version 24734 (0.0005) [2023-12-26 15:32:47,133][105692] Updated weights for policy 0, policy_version 24744 (0.0007) [2023-12-26 15:32:47,310][105620] Updated weights for policy 1, policy_version 24821 (0.0007) [2023-12-26 15:32:47,366][105620] Updated weights for policy 1, policy_version 24831 (0.0009) [2023-12-26 15:32:47,421][105620] Updated weights for policy 1, policy_version 24841 (0.0009) [2023-12-26 15:32:47,828][105692] Updated weights for policy 0, policy_version 24754 (0.0007) [2023-12-26 15:32:47,892][105692] Updated weights for policy 0, policy_version 24764 (0.0009) [2023-12-26 15:32:47,950][105692] Updated weights for policy 0, policy_version 24774 (0.0010) [2023-12-26 15:32:48,047][105620] Updated weights for policy 1, policy_version 24851 (0.0006) [2023-12-26 15:32:48,116][105620] Updated weights for policy 1, policy_version 24861 (0.0008) [2023-12-26 15:32:48,179][105620] Updated weights for policy 1, policy_version 24871 (0.0008) [2023-12-26 15:32:48,784][105692] Updated weights for policy 0, policy_version 24785 (0.0010) [2023-12-26 15:32:48,806][105620] Updated weights for policy 1, policy_version 24881 (0.0008) [2023-12-26 15:32:48,842][105692] Updated weights for policy 0, policy_version 24795 (0.0007) [2023-12-26 15:32:48,859][105620] Updated weights for policy 1, policy_version 24891 (0.0007) [2023-12-26 15:32:48,890][105692] Updated weights for policy 0, policy_version 24805 (0.0009) [2023-12-26 15:32:48,923][105620] Updated weights for policy 1, policy_version 24901 (0.0008) [2023-12-26 15:32:48,946][105692] Updated weights for policy 0, policy_version 24815 (0.0009) [2023-12-26 15:32:48,977][105620] Updated weights for policy 1, policy_version 24911 (0.0010) [2023-12-26 15:32:49,732][105620] Updated weights for policy 1, policy_version 24921 (0.0008) [2023-12-26 15:32:49,746][105692] Updated weights for policy 0, policy_version 24825 (0.0006) [2023-12-26 15:32:49,793][105620] Updated weights for policy 1, policy_version 24931 (0.0008) [2023-12-26 15:32:49,795][105692] Updated weights for policy 0, policy_version 24835 (0.0007) [2023-12-26 15:32:49,847][105620] Updated weights for policy 1, policy_version 24941 (0.0008) [2023-12-26 15:32:49,854][105692] Updated weights for policy 0, policy_version 24845 (0.0007) [2023-12-26 15:32:50,623][105620] Updated weights for policy 1, policy_version 24951 (0.0008) [2023-12-26 15:32:50,634][105692] Updated weights for policy 0, policy_version 24855 (0.0009) [2023-12-26 15:32:50,684][105620] Updated weights for policy 1, policy_version 24961 (0.0009) [2023-12-26 15:32:50,691][105692] Updated weights for policy 0, policy_version 24865 (0.0006) [2023-12-26 15:32:50,741][105692] Updated weights for policy 0, policy_version 24875 (0.0007) [2023-12-26 15:32:50,744][105620] Updated weights for policy 1, policy_version 24971 (0.0008) [2023-12-26 15:32:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19744.1). Total num frames: 12771328. Throughput: 0: 9725.1, 1: 9936.4. Samples: 12759300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:32:51,063][104569] Avg episode reward: [(0, '8056.430'), (1, '9264.171')] [2023-12-26 15:32:51,063][105586] Saving new best policy, reward=9264.171! [2023-12-26 15:32:51,499][105620] Updated weights for policy 1, policy_version 24981 (0.0008) [2023-12-26 15:32:51,518][105692] Updated weights for policy 0, policy_version 24885 (0.0006) [2023-12-26 15:32:51,561][105620] Updated weights for policy 1, policy_version 24991 (0.0008) [2023-12-26 15:32:51,579][105692] Updated weights for policy 0, policy_version 24895 (0.0006) [2023-12-26 15:32:51,619][105620] Updated weights for policy 1, policy_version 25001 (0.0007) [2023-12-26 15:32:51,635][105692] Updated weights for policy 0, policy_version 24905 (0.0008) [2023-12-26 15:32:52,362][105692] Updated weights for policy 0, policy_version 24916 (0.0007) [2023-12-26 15:32:52,427][105620] Updated weights for policy 1, policy_version 25011 (0.0007) [2023-12-26 15:32:52,427][105692] Updated weights for policy 0, policy_version 24926 (0.0010) [2023-12-26 15:32:52,479][105692] Updated weights for policy 0, policy_version 24936 (0.0009) [2023-12-26 15:32:52,489][105620] Updated weights for policy 1, policy_version 25021 (0.0007) [2023-12-26 15:32:52,545][105620] Updated weights for policy 1, policy_version 25031 (0.0007) [2023-12-26 15:32:53,186][105692] Updated weights for policy 0, policy_version 24946 (0.0010) [2023-12-26 15:32:53,234][105692] Updated weights for policy 0, policy_version 24956 (0.0005) [2023-12-26 15:32:53,282][105692] Updated weights for policy 0, policy_version 24966 (0.0005) [2023-12-26 15:32:53,339][105620] Updated weights for policy 1, policy_version 25041 (0.0008) [2023-12-26 15:32:53,344][105692] Updated weights for policy 0, policy_version 24976 (0.0006) [2023-12-26 15:32:53,399][105620] Updated weights for policy 1, policy_version 25051 (0.0010) [2023-12-26 15:32:53,452][105620] Updated weights for policy 1, policy_version 25062 (0.0010) [2023-12-26 15:32:53,505][105620] Updated weights for policy 1, policy_version 25072 (0.0008) [2023-12-26 15:32:53,959][105692] Updated weights for policy 0, policy_version 24986 (0.0009) [2023-12-26 15:32:54,007][105692] Updated weights for policy 0, policy_version 24996 (0.0009) [2023-12-26 15:32:54,058][105692] Updated weights for policy 0, policy_version 25006 (0.0009) [2023-12-26 15:32:54,353][105620] Updated weights for policy 1, policy_version 25082 (0.0009) [2023-12-26 15:32:54,427][105620] Updated weights for policy 1, policy_version 25092 (0.0009) [2023-12-26 15:32:54,489][105620] Updated weights for policy 1, policy_version 25102 (0.0009) [2023-12-26 15:32:54,725][105692] Updated weights for policy 0, policy_version 25016 (0.0006) [2023-12-26 15:32:54,790][105692] Updated weights for policy 0, policy_version 25026 (0.0005) [2023-12-26 15:32:54,849][105692] Updated weights for policy 0, policy_version 25036 (0.0005) [2023-12-26 15:32:55,308][105620] Updated weights for policy 1, policy_version 25112 (0.0009) [2023-12-26 15:32:55,371][105620] Updated weights for policy 1, policy_version 25122 (0.0008) [2023-12-26 15:32:55,430][105620] Updated weights for policy 1, policy_version 25132 (0.0009) [2023-12-26 15:32:55,444][105692] Updated weights for policy 0, policy_version 25046 (0.0008) [2023-12-26 15:32:55,501][105692] Updated weights for policy 0, policy_version 25056 (0.0005) [2023-12-26 15:32:55,563][105692] Updated weights for policy 0, policy_version 25066 (0.0008) [2023-12-26 15:32:56,006][105620] Updated weights for policy 1, policy_version 25142 (0.0007) [2023-12-26 15:32:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.4, 300 sec: 19688.6). Total num frames: 12861440. Throughput: 0: 9848.6, 1: 9809.8. Samples: 12874372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:32:56,062][104569] Avg episode reward: [(0, '8596.238'), (1, '9078.247')] [2023-12-26 15:32:56,071][105620] Updated weights for policy 1, policy_version 25152 (0.0006) [2023-12-26 15:32:56,095][105692] Updated weights for policy 0, policy_version 25076 (0.0010) [2023-12-26 15:32:56,129][105620] Updated weights for policy 1, policy_version 25162 (0.0005) [2023-12-26 15:32:56,144][105692] Updated weights for policy 0, policy_version 25086 (0.0010) [2023-12-26 15:32:56,196][105692] Updated weights for policy 0, policy_version 25096 (0.0007) [2023-12-26 15:32:56,651][105620] Updated weights for policy 1, policy_version 25172 (0.0006) [2023-12-26 15:32:56,711][105620] Updated weights for policy 1, policy_version 25182 (0.0007) [2023-12-26 15:32:56,762][105620] Updated weights for policy 1, policy_version 25192 (0.0008) [2023-12-26 15:32:56,897][105692] Updated weights for policy 0, policy_version 25106 (0.0007) [2023-12-26 15:32:56,954][105692] Updated weights for policy 0, policy_version 25116 (0.0005) [2023-12-26 15:32:57,018][105692] Updated weights for policy 0, policy_version 25126 (0.0005) [2023-12-26 15:32:57,082][105692] Updated weights for policy 0, policy_version 25136 (0.0010) [2023-12-26 15:32:57,384][105620] Updated weights for policy 1, policy_version 25202 (0.0007) [2023-12-26 15:32:57,460][105620] Updated weights for policy 1, policy_version 25212 (0.0005) [2023-12-26 15:32:57,533][105620] Updated weights for policy 1, policy_version 25222 (0.0007) [2023-12-26 15:32:57,589][105620] Updated weights for policy 1, policy_version 25232 (0.0009) [2023-12-26 15:32:57,771][105692] Updated weights for policy 0, policy_version 25146 (0.0005) [2023-12-26 15:32:57,834][105692] Updated weights for policy 0, policy_version 25156 (0.0007) [2023-12-26 15:32:57,886][105692] Updated weights for policy 0, policy_version 25166 (0.0005) [2023-12-26 15:32:58,359][105620] Updated weights for policy 1, policy_version 25242 (0.0009) [2023-12-26 15:32:58,430][105620] Updated weights for policy 1, policy_version 25252 (0.0007) [2023-12-26 15:32:58,498][105620] Updated weights for policy 1, policy_version 25262 (0.0010) [2023-12-26 15:32:58,555][105692] Updated weights for policy 0, policy_version 25176 (0.0007) [2023-12-26 15:32:58,623][105692] Updated weights for policy 0, policy_version 25186 (0.0008) [2023-12-26 15:32:58,693][105692] Updated weights for policy 0, policy_version 25196 (0.0008) [2023-12-26 15:32:59,320][105620] Updated weights for policy 1, policy_version 25272 (0.0010) [2023-12-26 15:32:59,374][105692] Updated weights for policy 0, policy_version 25207 (0.0010) [2023-12-26 15:32:59,387][105620] Updated weights for policy 1, policy_version 25282 (0.0008) [2023-12-26 15:32:59,435][105620] Updated weights for policy 1, policy_version 25292 (0.0005) [2023-12-26 15:32:59,436][105692] Updated weights for policy 0, policy_version 25217 (0.0008) [2023-12-26 15:32:59,498][105692] Updated weights for policy 0, policy_version 25227 (0.0007) [2023-12-26 15:33:00,108][105620] Updated weights for policy 1, policy_version 25302 (0.0007) [2023-12-26 15:33:00,160][105620] Updated weights for policy 1, policy_version 25312 (0.0008) [2023-12-26 15:33:00,214][105692] Updated weights for policy 0, policy_version 25237 (0.0008) [2023-12-26 15:33:00,220][105620] Updated weights for policy 1, policy_version 25322 (0.0007) [2023-12-26 15:33:00,269][105692] Updated weights for policy 0, policy_version 25247 (0.0008) [2023-12-26 15:33:00,327][105692] Updated weights for policy 0, policy_version 25257 (0.0008) [2023-12-26 15:33:00,990][105620] Updated weights for policy 1, policy_version 25332 (0.0007) [2023-12-26 15:33:01,045][105620] Updated weights for policy 1, policy_version 25342 (0.0009) [2023-12-26 15:33:01,055][105692] Updated weights for policy 0, policy_version 25267 (0.0008) [2023-12-26 15:33:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19660.8). Total num frames: 12959744. Throughput: 0: 9901.3, 1: 9821.6. Samples: 12936172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:33:01,063][104569] Avg episode reward: [(0, '8879.854'), (1, '8984.855')] [2023-12-26 15:33:01,104][105620] Updated weights for policy 1, policy_version 25352 (0.0006) [2023-12-26 15:33:01,114][105692] Updated weights for policy 0, policy_version 25277 (0.0011) [2023-12-26 15:33:01,152][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000025360_6496256.pth... [2023-12-26 15:33:01,157][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000024208_6201344.pth [2023-12-26 15:33:01,167][105692] Updated weights for policy 0, policy_version 25287 (0.0010) [2023-12-26 15:33:01,219][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000025296_6479872.pth... [2023-12-26 15:33:01,224][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000024144_6184960.pth [2023-12-26 15:33:01,834][105692] Updated weights for policy 0, policy_version 25297 (0.0007) [2023-12-26 15:33:01,888][105692] Updated weights for policy 0, policy_version 25307 (0.0010) [2023-12-26 15:33:01,939][105692] Updated weights for policy 0, policy_version 25317 (0.0010) [2023-12-26 15:33:01,962][105620] Updated weights for policy 1, policy_version 25362 (0.0007) [2023-12-26 15:33:01,998][105692] Updated weights for policy 0, policy_version 25327 (0.0010) [2023-12-26 15:33:02,017][105620] Updated weights for policy 1, policy_version 25372 (0.0007) [2023-12-26 15:33:02,073][105620] Updated weights for policy 1, policy_version 25382 (0.0008) [2023-12-26 15:33:02,132][105620] Updated weights for policy 1, policy_version 25392 (0.0008) [2023-12-26 15:33:02,742][105692] Updated weights for policy 0, policy_version 25337 (0.0010) [2023-12-26 15:33:02,799][105692] Updated weights for policy 0, policy_version 25347 (0.0010) [2023-12-26 15:33:02,860][105692] Updated weights for policy 0, policy_version 25357 (0.0010) [2023-12-26 15:33:02,891][105620] Updated weights for policy 1, policy_version 25402 (0.0008) [2023-12-26 15:33:02,941][105620] Updated weights for policy 1, policy_version 25413 (0.0009) [2023-12-26 15:33:02,988][105620] Updated weights for policy 1, policy_version 25423 (0.0009) [2023-12-26 15:33:03,572][105692] Updated weights for policy 0, policy_version 25367 (0.0006) [2023-12-26 15:33:03,627][105692] Updated weights for policy 0, policy_version 25377 (0.0005) [2023-12-26 15:33:03,675][105692] Updated weights for policy 0, policy_version 25387 (0.0005) [2023-12-26 15:33:03,802][105620] Updated weights for policy 1, policy_version 25433 (0.0009) [2023-12-26 15:33:03,873][105620] Updated weights for policy 1, policy_version 25443 (0.0009) [2023-12-26 15:33:03,927][105620] Updated weights for policy 1, policy_version 25453 (0.0009) [2023-12-26 15:33:04,371][105692] Updated weights for policy 0, policy_version 25397 (0.0008) [2023-12-26 15:33:04,423][105692] Updated weights for policy 0, policy_version 25407 (0.0009) [2023-12-26 15:33:04,469][105692] Updated weights for policy 0, policy_version 25417 (0.0008) [2023-12-26 15:33:04,643][105620] Updated weights for policy 1, policy_version 25463 (0.0009) [2023-12-26 15:33:04,694][105620] Updated weights for policy 1, policy_version 25473 (0.0009) [2023-12-26 15:33:04,744][105620] Updated weights for policy 1, policy_version 25483 (0.0009) [2023-12-26 15:33:05,174][105692] Updated weights for policy 0, policy_version 25427 (0.0008) [2023-12-26 15:33:05,221][105692] Updated weights for policy 0, policy_version 25437 (0.0009) [2023-12-26 15:33:05,272][105692] Updated weights for policy 0, policy_version 25447 (0.0009) [2023-12-26 15:33:05,482][105620] Updated weights for policy 1, policy_version 25493 (0.0007) [2023-12-26 15:33:05,540][105620] Updated weights for policy 1, policy_version 25503 (0.0009) [2023-12-26 15:33:05,595][105620] Updated weights for policy 1, policy_version 25513 (0.0009) [2023-12-26 15:33:06,055][105692] Updated weights for policy 0, policy_version 25457 (0.0010) [2023-12-26 15:33:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19716.3). Total num frames: 13058048. Throughput: 0: 9856.6, 1: 9660.5. Samples: 13049712. Policy #0 lag: (min: 3.0, avg: 10.9, max: 35.0) [2023-12-26 15:33:06,062][104569] Avg episode reward: [(0, '8518.486'), (1, '8984.308')] [2023-12-26 15:33:06,115][105692] Updated weights for policy 0, policy_version 25467 (0.0009) [2023-12-26 15:33:06,177][105692] Updated weights for policy 0, policy_version 25477 (0.0006) [2023-12-26 15:33:06,242][105692] Updated weights for policy 0, policy_version 25487 (0.0009) [2023-12-26 15:33:06,340][105620] Updated weights for policy 1, policy_version 25523 (0.0008) [2023-12-26 15:33:06,400][105620] Updated weights for policy 1, policy_version 25533 (0.0009) [2023-12-26 15:33:06,458][105620] Updated weights for policy 1, policy_version 25543 (0.0008) [2023-12-26 15:33:06,994][105692] Updated weights for policy 0, policy_version 25497 (0.0009) [2023-12-26 15:33:07,056][105692] Updated weights for policy 0, policy_version 25507 (0.0009) [2023-12-26 15:33:07,117][105692] Updated weights for policy 0, policy_version 25517 (0.0008) [2023-12-26 15:33:07,180][105620] Updated weights for policy 1, policy_version 25553 (0.0006) [2023-12-26 15:33:07,238][105620] Updated weights for policy 1, policy_version 25563 (0.0009) [2023-12-26 15:33:07,287][105620] Updated weights for policy 1, policy_version 25573 (0.0009) [2023-12-26 15:33:07,339][105620] Updated weights for policy 1, policy_version 25583 (0.0009) [2023-12-26 15:33:07,759][105692] Updated weights for policy 0, policy_version 25527 (0.0008) [2023-12-26 15:33:07,811][105692] Updated weights for policy 0, policy_version 25537 (0.0005) [2023-12-26 15:33:07,862][105692] Updated weights for policy 0, policy_version 25547 (0.0005) [2023-12-26 15:33:08,119][105620] Updated weights for policy 1, policy_version 25593 (0.0009) [2023-12-26 15:33:08,166][105620] Updated weights for policy 1, policy_version 25603 (0.0009) [2023-12-26 15:33:08,224][105620] Updated weights for policy 1, policy_version 25613 (0.0009) [2023-12-26 15:33:08,591][105692] Updated weights for policy 0, policy_version 25557 (0.0008) [2023-12-26 15:33:08,653][105692] Updated weights for policy 0, policy_version 25567 (0.0009) [2023-12-26 15:33:08,720][105692] Updated weights for policy 0, policy_version 25577 (0.0009) [2023-12-26 15:33:08,887][105620] Updated weights for policy 1, policy_version 25623 (0.0009) [2023-12-26 15:33:08,939][105620] Updated weights for policy 1, policy_version 25633 (0.0009) [2023-12-26 15:33:08,992][105620] Updated weights for policy 1, policy_version 25643 (0.0009) [2023-12-26 15:33:09,458][105692] Updated weights for policy 0, policy_version 25587 (0.0009) [2023-12-26 15:33:09,521][105692] Updated weights for policy 0, policy_version 25597 (0.0009) [2023-12-26 15:33:09,588][105692] Updated weights for policy 0, policy_version 25607 (0.0010) [2023-12-26 15:33:09,718][105620] Updated weights for policy 1, policy_version 25653 (0.0008) [2023-12-26 15:33:09,786][105620] Updated weights for policy 1, policy_version 25663 (0.0008) [2023-12-26 15:33:09,849][105620] Updated weights for policy 1, policy_version 25673 (0.0007) [2023-12-26 15:33:10,392][105692] Updated weights for policy 0, policy_version 25617 (0.0010) [2023-12-26 15:33:10,458][105692] Updated weights for policy 0, policy_version 25627 (0.0009) [2023-12-26 15:33:10,509][105620] Updated weights for policy 1, policy_version 25683 (0.0006) [2023-12-26 15:33:10,522][105692] Updated weights for policy 0, policy_version 25637 (0.0011) [2023-12-26 15:33:10,560][105620] Updated weights for policy 1, policy_version 25693 (0.0009) [2023-12-26 15:33:10,581][105692] Updated weights for policy 0, policy_version 25647 (0.0010) [2023-12-26 15:33:10,613][105620] Updated weights for policy 1, policy_version 25703 (0.0007) [2023-12-26 15:33:11,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 13156352. Throughput: 0: 9805.8, 1: 9766.1. Samples: 13165740. Policy #0 lag: (min: 3.0, avg: 10.9, max: 35.0) [2023-12-26 15:33:11,062][104569] Avg episode reward: [(0, '8428.473'), (1, '8891.558')] [2023-12-26 15:33:11,322][105692] Updated weights for policy 0, policy_version 25657 (0.0010) [2023-12-26 15:33:11,355][105620] Updated weights for policy 1, policy_version 25713 (0.0006) [2023-12-26 15:33:11,398][105692] Updated weights for policy 0, policy_version 25667 (0.0009) [2023-12-26 15:33:11,424][105620] Updated weights for policy 1, policy_version 25723 (0.0010) [2023-12-26 15:33:11,464][105692] Updated weights for policy 0, policy_version 25677 (0.0010) [2023-12-26 15:33:11,488][105620] Updated weights for policy 1, policy_version 25733 (0.0008) [2023-12-26 15:33:11,552][105620] Updated weights for policy 1, policy_version 25743 (0.0008) [2023-12-26 15:33:12,157][105692] Updated weights for policy 0, policy_version 25687 (0.0011) [2023-12-26 15:33:12,224][105692] Updated weights for policy 0, policy_version 25697 (0.0011) [2023-12-26 15:33:12,261][105620] Updated weights for policy 1, policy_version 25753 (0.0006) [2023-12-26 15:33:12,283][105692] Updated weights for policy 0, policy_version 25707 (0.0010) [2023-12-26 15:33:12,328][105620] Updated weights for policy 1, policy_version 25763 (0.0007) [2023-12-26 15:33:12,400][105620] Updated weights for policy 1, policy_version 25773 (0.0009) [2023-12-26 15:33:12,998][105620] Updated weights for policy 1, policy_version 25783 (0.0006) [2023-12-26 15:33:13,039][105692] Updated weights for policy 0, policy_version 25717 (0.0010) [2023-12-26 15:33:13,041][105620] Updated weights for policy 1, policy_version 25793 (0.0006) [2023-12-26 15:33:13,090][105692] Updated weights for policy 0, policy_version 25727 (0.0010) [2023-12-26 15:33:13,093][105620] Updated weights for policy 1, policy_version 25803 (0.0007) [2023-12-26 15:33:13,140][105692] Updated weights for policy 0, policy_version 25737 (0.0010) [2023-12-26 15:33:13,849][105620] Updated weights for policy 1, policy_version 25813 (0.0008) [2023-12-26 15:33:13,886][105692] Updated weights for policy 0, policy_version 25747 (0.0010) [2023-12-26 15:33:13,900][105620] Updated weights for policy 1, policy_version 25823 (0.0007) [2023-12-26 15:33:13,935][105692] Updated weights for policy 0, policy_version 25757 (0.0010) [2023-12-26 15:33:13,953][105620] Updated weights for policy 1, policy_version 25833 (0.0006) [2023-12-26 15:33:13,994][105692] Updated weights for policy 0, policy_version 25767 (0.0010) [2023-12-26 15:33:14,667][105692] Updated weights for policy 0, policy_version 25777 (0.0006) [2023-12-26 15:33:14,721][105692] Updated weights for policy 0, policy_version 25787 (0.0005) [2023-12-26 15:33:14,761][105620] Updated weights for policy 1, policy_version 25843 (0.0006) [2023-12-26 15:33:14,777][105692] Updated weights for policy 0, policy_version 25797 (0.0007) [2023-12-26 15:33:14,820][105620] Updated weights for policy 1, policy_version 25853 (0.0007) [2023-12-26 15:33:14,838][105692] Updated weights for policy 0, policy_version 25807 (0.0010) [2023-12-26 15:33:14,869][105620] Updated weights for policy 1, policy_version 25863 (0.0008) [2023-12-26 15:33:15,583][105692] Updated weights for policy 0, policy_version 25817 (0.0006) [2023-12-26 15:33:15,617][105620] Updated weights for policy 1, policy_version 25873 (0.0009) [2023-12-26 15:33:15,634][105692] Updated weights for policy 0, policy_version 25827 (0.0005) [2023-12-26 15:33:15,676][105620] Updated weights for policy 1, policy_version 25883 (0.0009) [2023-12-26 15:33:15,688][105692] Updated weights for policy 0, policy_version 25837 (0.0006) [2023-12-26 15:33:15,737][105620] Updated weights for policy 1, policy_version 25893 (0.0009) [2023-12-26 15:33:15,802][105620] Updated weights for policy 1, policy_version 25903 (0.0009) [2023-12-26 15:33:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 13254656. Throughput: 0: 9759.7, 1: 9677.6. Samples: 13223312. Policy #0 lag: (min: 3.0, avg: 10.9, max: 35.0) [2023-12-26 15:33:16,062][104569] Avg episode reward: [(0, '8702.480'), (1, '8613.926')] [2023-12-26 15:33:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000025904_6635520.pth... [2023-12-26 15:33:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000025840_6619136.pth... [2023-12-26 15:33:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000024688_6324224.pth [2023-12-26 15:33:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000024784_6348800.pth [2023-12-26 15:33:16,433][105692] Updated weights for policy 0, policy_version 25847 (0.0007) [2023-12-26 15:33:16,459][105620] Updated weights for policy 1, policy_version 25913 (0.0006) [2023-12-26 15:33:16,497][105692] Updated weights for policy 0, policy_version 25857 (0.0005) [2023-12-26 15:33:16,509][105620] Updated weights for policy 1, policy_version 25923 (0.0005) [2023-12-26 15:33:16,546][105692] Updated weights for policy 0, policy_version 25867 (0.0005) [2023-12-26 15:33:16,555][105620] Updated weights for policy 1, policy_version 25933 (0.0005) [2023-12-26 15:33:17,092][105692] Updated weights for policy 0, policy_version 25877 (0.0005) [2023-12-26 15:33:17,095][105620] Updated weights for policy 1, policy_version 25943 (0.0005) [2023-12-26 15:33:17,147][105692] Updated weights for policy 0, policy_version 25887 (0.0005) [2023-12-26 15:33:17,151][105620] Updated weights for policy 1, policy_version 25953 (0.0005) [2023-12-26 15:33:17,206][105692] Updated weights for policy 0, policy_version 25897 (0.0005) [2023-12-26 15:33:17,210][105620] Updated weights for policy 1, policy_version 25963 (0.0006) [2023-12-26 15:33:17,784][105692] Updated weights for policy 0, policy_version 25907 (0.0007) [2023-12-26 15:33:17,835][105692] Updated weights for policy 0, policy_version 25917 (0.0010) [2023-12-26 15:33:17,850][105620] Updated weights for policy 1, policy_version 25973 (0.0006) [2023-12-26 15:33:17,883][105692] Updated weights for policy 0, policy_version 25927 (0.0010) [2023-12-26 15:33:17,897][105620] Updated weights for policy 1, policy_version 25983 (0.0006) [2023-12-26 15:33:17,958][105620] Updated weights for policy 1, policy_version 25993 (0.0007) [2023-12-26 15:33:18,634][105692] Updated weights for policy 0, policy_version 25937 (0.0010) [2023-12-26 15:33:18,692][105692] Updated weights for policy 0, policy_version 25947 (0.0010) [2023-12-26 15:33:18,734][105620] Updated weights for policy 1, policy_version 26003 (0.0007) [2023-12-26 15:33:18,755][105692] Updated weights for policy 0, policy_version 25957 (0.0011) [2023-12-26 15:33:18,781][105620] Updated weights for policy 1, policy_version 26013 (0.0009) [2023-12-26 15:33:18,807][105692] Updated weights for policy 0, policy_version 25967 (0.0009) [2023-12-26 15:33:18,838][105620] Updated weights for policy 1, policy_version 26023 (0.0007) [2023-12-26 15:33:19,526][105692] Updated weights for policy 0, policy_version 25977 (0.0010) [2023-12-26 15:33:19,596][105692] Updated weights for policy 0, policy_version 25987 (0.0010) [2023-12-26 15:33:19,615][105620] Updated weights for policy 1, policy_version 26033 (0.0009) [2023-12-26 15:33:19,665][105692] Updated weights for policy 0, policy_version 25997 (0.0010) [2023-12-26 15:33:19,676][105620] Updated weights for policy 1, policy_version 26043 (0.0006) [2023-12-26 15:33:19,740][105620] Updated weights for policy 1, policy_version 26053 (0.0007) [2023-12-26 15:33:19,801][105620] Updated weights for policy 1, policy_version 26063 (0.0008) [2023-12-26 15:33:20,390][105692] Updated weights for policy 0, policy_version 26007 (0.0010) [2023-12-26 15:33:20,442][105692] Updated weights for policy 0, policy_version 26017 (0.0010) [2023-12-26 15:33:20,445][105620] Updated weights for policy 1, policy_version 26073 (0.0006) [2023-12-26 15:33:20,505][105620] Updated weights for policy 1, policy_version 26083 (0.0010) [2023-12-26 15:33:20,508][105692] Updated weights for policy 0, policy_version 26027 (0.0009) [2023-12-26 15:33:20,568][105620] Updated weights for policy 1, policy_version 26093 (0.0010) [2023-12-26 15:33:21,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19716.3). Total num frames: 13352960. Throughput: 0: 9794.3, 1: 9642.2. Samples: 13343604. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-26 15:33:21,063][104569] Avg episode reward: [(0, '8520.684'), (1, '8800.516')] [2023-12-26 15:33:21,281][105692] Updated weights for policy 0, policy_version 26037 (0.0011) [2023-12-26 15:33:21,311][105620] Updated weights for policy 1, policy_version 26103 (0.0007) [2023-12-26 15:33:21,349][105692] Updated weights for policy 0, policy_version 26047 (0.0011) [2023-12-26 15:33:21,382][105620] Updated weights for policy 1, policy_version 26113 (0.0008) [2023-12-26 15:33:21,410][105692] Updated weights for policy 0, policy_version 26057 (0.0011) [2023-12-26 15:33:21,445][105620] Updated weights for policy 1, policy_version 26123 (0.0006) [2023-12-26 15:33:22,148][105692] Updated weights for policy 0, policy_version 26067 (0.0010) [2023-12-26 15:33:22,200][105692] Updated weights for policy 0, policy_version 26077 (0.0010) [2023-12-26 15:33:22,250][105620] Updated weights for policy 1, policy_version 26133 (0.0007) [2023-12-26 15:33:22,261][105692] Updated weights for policy 0, policy_version 26087 (0.0010) [2023-12-26 15:33:22,311][105620] Updated weights for policy 1, policy_version 26143 (0.0008) [2023-12-26 15:33:22,379][105620] Updated weights for policy 1, policy_version 26153 (0.0009) [2023-12-26 15:33:22,909][105692] Updated weights for policy 0, policy_version 26097 (0.0006) [2023-12-26 15:33:22,975][105692] Updated weights for policy 0, policy_version 26107 (0.0010) [2023-12-26 15:33:23,040][105692] Updated weights for policy 0, policy_version 26117 (0.0008) [2023-12-26 15:33:23,112][105692] Updated weights for policy 0, policy_version 26127 (0.0007) [2023-12-26 15:33:23,148][105620] Updated weights for policy 1, policy_version 26163 (0.0008) [2023-12-26 15:33:23,204][105620] Updated weights for policy 1, policy_version 26173 (0.0008) [2023-12-26 15:33:23,266][105620] Updated weights for policy 1, policy_version 26183 (0.0008) [2023-12-26 15:33:23,779][105692] Updated weights for policy 0, policy_version 26137 (0.0010) [2023-12-26 15:33:23,822][105692] Updated weights for policy 0, policy_version 26147 (0.0010) [2023-12-26 15:33:23,877][105692] Updated weights for policy 0, policy_version 26157 (0.0010) [2023-12-26 15:33:23,892][105620] Updated weights for policy 1, policy_version 26193 (0.0008) [2023-12-26 15:33:23,941][105620] Updated weights for policy 1, policy_version 26203 (0.0005) [2023-12-26 15:33:23,987][105620] Updated weights for policy 1, policy_version 26213 (0.0005) [2023-12-26 15:33:24,044][105620] Updated weights for policy 1, policy_version 26223 (0.0005) [2023-12-26 15:33:24,561][105692] Updated weights for policy 0, policy_version 26167 (0.0010) [2023-12-26 15:33:24,609][105692] Updated weights for policy 0, policy_version 26177 (0.0010) [2023-12-26 15:33:24,662][105692] Updated weights for policy 0, policy_version 26187 (0.0010) [2023-12-26 15:33:24,692][105620] Updated weights for policy 1, policy_version 26233 (0.0005) [2023-12-26 15:33:24,755][105620] Updated weights for policy 1, policy_version 26243 (0.0008) [2023-12-26 15:33:24,810][105620] Updated weights for policy 1, policy_version 26253 (0.0008) [2023-12-26 15:33:25,398][105692] Updated weights for policy 0, policy_version 26197 (0.0009) [2023-12-26 15:33:25,456][105692] Updated weights for policy 0, policy_version 26207 (0.0010) [2023-12-26 15:33:25,485][105620] Updated weights for policy 1, policy_version 26263 (0.0006) [2023-12-26 15:33:25,508][105692] Updated weights for policy 0, policy_version 26217 (0.0010) [2023-12-26 15:33:25,538][105620] Updated weights for policy 1, policy_version 26273 (0.0006) [2023-12-26 15:33:25,596][105620] Updated weights for policy 1, policy_version 26283 (0.0005) [2023-12-26 15:33:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 13451264. Throughput: 0: 9788.2, 1: 9647.3. Samples: 13462168. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-26 15:33:26,063][104569] Avg episode reward: [(0, '8797.830'), (1, '8710.729')] [2023-12-26 15:33:26,125][105620] Updated weights for policy 1, policy_version 26293 (0.0005) [2023-12-26 15:33:26,184][105620] Updated weights for policy 1, policy_version 26303 (0.0005) [2023-12-26 15:33:26,242][105692] Updated weights for policy 0, policy_version 26227 (0.0010) [2023-12-26 15:33:26,244][105620] Updated weights for policy 1, policy_version 26313 (0.0006) [2023-12-26 15:33:26,303][105692] Updated weights for policy 0, policy_version 26237 (0.0010) [2023-12-26 15:33:26,361][105692] Updated weights for policy 0, policy_version 26247 (0.0010) [2023-12-26 15:33:26,887][105620] Updated weights for policy 1, policy_version 26323 (0.0007) [2023-12-26 15:33:26,942][105620] Updated weights for policy 1, policy_version 26333 (0.0009) [2023-12-26 15:33:26,990][105620] Updated weights for policy 1, policy_version 26343 (0.0010) [2023-12-26 15:33:27,091][105692] Updated weights for policy 0, policy_version 26257 (0.0010) [2023-12-26 15:33:27,146][105692] Updated weights for policy 0, policy_version 26267 (0.0009) [2023-12-26 15:33:27,198][105692] Updated weights for policy 0, policy_version 26277 (0.0007) [2023-12-26 15:33:27,245][105692] Updated weights for policy 0, policy_version 26287 (0.0010) [2023-12-26 15:33:27,729][105620] Updated weights for policy 1, policy_version 26353 (0.0010) [2023-12-26 15:33:27,777][105620] Updated weights for policy 1, policy_version 26363 (0.0008) [2023-12-26 15:33:27,824][105620] Updated weights for policy 1, policy_version 26373 (0.0008) [2023-12-26 15:33:27,875][105620] Updated weights for policy 1, policy_version 26383 (0.0008) [2023-12-26 15:33:27,982][105692] Updated weights for policy 0, policy_version 26297 (0.0010) [2023-12-26 15:33:28,030][105692] Updated weights for policy 0, policy_version 26307 (0.0010) [2023-12-26 15:33:28,079][105692] Updated weights for policy 0, policy_version 26317 (0.0010) [2023-12-26 15:33:28,637][105620] Updated weights for policy 1, policy_version 26393 (0.0008) [2023-12-26 15:33:28,685][105620] Updated weights for policy 1, policy_version 26403 (0.0008) [2023-12-26 15:33:28,733][105620] Updated weights for policy 1, policy_version 26413 (0.0010) [2023-12-26 15:33:28,820][105692] Updated weights for policy 0, policy_version 26327 (0.0009) [2023-12-26 15:33:28,878][105692] Updated weights for policy 0, policy_version 26337 (0.0007) [2023-12-26 15:33:28,927][105692] Updated weights for policy 0, policy_version 26347 (0.0008) [2023-12-26 15:33:29,462][105620] Updated weights for policy 1, policy_version 26423 (0.0011) [2023-12-26 15:33:29,511][105620] Updated weights for policy 1, policy_version 26433 (0.0010) [2023-12-26 15:33:29,563][105620] Updated weights for policy 1, policy_version 26443 (0.0010) [2023-12-26 15:33:29,624][105692] Updated weights for policy 0, policy_version 26357 (0.0006) [2023-12-26 15:33:29,686][105692] Updated weights for policy 0, policy_version 26367 (0.0005) [2023-12-26 15:33:29,753][105692] Updated weights for policy 0, policy_version 26377 (0.0005) [2023-12-26 15:33:30,295][105620] Updated weights for policy 1, policy_version 26453 (0.0010) [2023-12-26 15:33:30,343][105692] Updated weights for policy 0, policy_version 26387 (0.0006) [2023-12-26 15:33:30,358][105620] Updated weights for policy 1, policy_version 26463 (0.0011) [2023-12-26 15:33:30,401][105692] Updated weights for policy 0, policy_version 26397 (0.0006) [2023-12-26 15:33:30,407][105620] Updated weights for policy 1, policy_version 26473 (0.0010) [2023-12-26 15:33:30,455][105692] Updated weights for policy 0, policy_version 26407 (0.0006) [2023-12-26 15:33:31,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 13549568. Throughput: 0: 9802.3, 1: 9713.4. Samples: 13521380. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-26 15:33:31,062][104569] Avg episode reward: [(0, '8982.278'), (1, '8802.907')] [2023-12-26 15:33:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000026416_6766592.pth... [2023-12-26 15:33:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000026480_6782976.pth... [2023-12-26 15:33:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000025360_6496256.pth [2023-12-26 15:33:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000025296_6479872.pth [2023-12-26 15:33:31,126][105620] Updated weights for policy 1, policy_version 26483 (0.0010) [2023-12-26 15:33:31,185][105620] Updated weights for policy 1, policy_version 26493 (0.0010) [2023-12-26 15:33:31,238][105620] Updated weights for policy 1, policy_version 26503 (0.0006) [2023-12-26 15:33:31,249][105692] Updated weights for policy 0, policy_version 26417 (0.0008) [2023-12-26 15:33:31,310][105692] Updated weights for policy 0, policy_version 26427 (0.0007) [2023-12-26 15:33:31,381][105692] Updated weights for policy 0, policy_version 26437 (0.0008) [2023-12-26 15:33:31,443][105692] Updated weights for policy 0, policy_version 26447 (0.0006) [2023-12-26 15:33:31,918][105620] Updated weights for policy 1, policy_version 26513 (0.0010) [2023-12-26 15:33:31,979][105620] Updated weights for policy 1, policy_version 26523 (0.0006) [2023-12-26 15:33:32,039][105620] Updated weights for policy 1, policy_version 26533 (0.0005) [2023-12-26 15:33:32,098][105620] Updated weights for policy 1, policy_version 26543 (0.0005) [2023-12-26 15:33:32,219][105692] Updated weights for policy 0, policy_version 26457 (0.0009) [2023-12-26 15:33:32,289][105692] Updated weights for policy 0, policy_version 26467 (0.0008) [2023-12-26 15:33:32,341][105692] Updated weights for policy 0, policy_version 26477 (0.0009) [2023-12-26 15:33:32,675][105620] Updated weights for policy 1, policy_version 26553 (0.0009) [2023-12-26 15:33:32,724][105620] Updated weights for policy 1, policy_version 26563 (0.0010) [2023-12-26 15:33:32,772][105620] Updated weights for policy 1, policy_version 26573 (0.0010) [2023-12-26 15:33:33,114][105692] Updated weights for policy 0, policy_version 26487 (0.0007) [2023-12-26 15:33:33,165][105692] Updated weights for policy 0, policy_version 26497 (0.0006) [2023-12-26 15:33:33,218][105692] Updated weights for policy 0, policy_version 26509 (0.0010) [2023-12-26 15:33:33,436][105620] Updated weights for policy 1, policy_version 26583 (0.0010) [2023-12-26 15:33:33,496][105620] Updated weights for policy 1, policy_version 26593 (0.0009) [2023-12-26 15:33:33,547][105620] Updated weights for policy 1, policy_version 26603 (0.0007) [2023-12-26 15:33:33,952][105692] Updated weights for policy 0, policy_version 26520 (0.0007) [2023-12-26 15:33:34,000][105692] Updated weights for policy 0, policy_version 26530 (0.0009) [2023-12-26 15:33:34,045][105692] Updated weights for policy 0, policy_version 26540 (0.0005) [2023-12-26 15:33:34,105][105620] Updated weights for policy 1, policy_version 26613 (0.0005) [2023-12-26 15:33:34,169][105620] Updated weights for policy 1, policy_version 26623 (0.0009) [2023-12-26 15:33:34,236][105620] Updated weights for policy 1, policy_version 26633 (0.0010) [2023-12-26 15:33:34,797][105692] Updated weights for policy 0, policy_version 26550 (0.0008) [2023-12-26 15:33:34,852][105692] Updated weights for policy 0, policy_version 26560 (0.0009) [2023-12-26 15:33:34,868][105620] Updated weights for policy 1, policy_version 26643 (0.0008) [2023-12-26 15:33:34,908][105692] Updated weights for policy 0, policy_version 26570 (0.0008) [2023-12-26 15:33:34,919][105620] Updated weights for policy 1, policy_version 26653 (0.0005) [2023-12-26 15:33:34,977][105620] Updated weights for policy 1, policy_version 26663 (0.0009) [2023-12-26 15:33:35,616][105692] Updated weights for policy 0, policy_version 26580 (0.0007) [2023-12-26 15:33:35,632][105620] Updated weights for policy 1, policy_version 26673 (0.0010) [2023-12-26 15:33:35,671][105692] Updated weights for policy 0, policy_version 26590 (0.0007) [2023-12-26 15:33:35,687][105620] Updated weights for policy 1, policy_version 26683 (0.0006) [2023-12-26 15:33:35,721][105692] Updated weights for policy 0, policy_version 26600 (0.0007) [2023-12-26 15:33:35,745][105620] Updated weights for policy 1, policy_version 26693 (0.0010) [2023-12-26 15:33:35,807][105620] Updated weights for policy 1, policy_version 26703 (0.0010) [2023-12-26 15:33:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.2, 300 sec: 19744.1). Total num frames: 13656064. Throughput: 0: 9808.6, 1: 9814.8. Samples: 13642352. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-26 15:33:36,062][104569] Avg episode reward: [(0, '8612.500'), (1, '8801.303')] [2023-12-26 15:33:36,366][105692] Updated weights for policy 0, policy_version 26610 (0.0006) [2023-12-26 15:33:36,432][105692] Updated weights for policy 0, policy_version 26620 (0.0007) [2023-12-26 15:33:36,502][105692] Updated weights for policy 0, policy_version 26630 (0.0009) [2023-12-26 15:33:36,564][105620] Updated weights for policy 1, policy_version 26713 (0.0008) [2023-12-26 15:33:36,570][105692] Updated weights for policy 0, policy_version 26640 (0.0009) [2023-12-26 15:33:36,625][105620] Updated weights for policy 1, policy_version 26723 (0.0009) [2023-12-26 15:33:36,682][105620] Updated weights for policy 1, policy_version 26733 (0.0009) [2023-12-26 15:33:37,256][105692] Updated weights for policy 0, policy_version 26650 (0.0009) [2023-12-26 15:33:37,303][105692] Updated weights for policy 0, policy_version 26660 (0.0009) [2023-12-26 15:33:37,326][105620] Updated weights for policy 1, policy_version 26743 (0.0007) [2023-12-26 15:33:37,368][105692] Updated weights for policy 0, policy_version 26670 (0.0008) [2023-12-26 15:33:37,379][105620] Updated weights for policy 1, policy_version 26753 (0.0006) [2023-12-26 15:33:37,439][105620] Updated weights for policy 1, policy_version 26763 (0.0007) [2023-12-26 15:33:38,029][105692] Updated weights for policy 0, policy_version 26680 (0.0006) [2023-12-26 15:33:38,083][105692] Updated weights for policy 0, policy_version 26690 (0.0006) [2023-12-26 15:33:38,137][105692] Updated weights for policy 0, policy_version 26700 (0.0005) [2023-12-26 15:33:38,182][105620] Updated weights for policy 1, policy_version 26773 (0.0008) [2023-12-26 15:33:38,233][105620] Updated weights for policy 1, policy_version 26783 (0.0006) [2023-12-26 15:33:38,289][105620] Updated weights for policy 1, policy_version 26793 (0.0006) [2023-12-26 15:33:38,902][105620] Updated weights for policy 1, policy_version 26803 (0.0010) [2023-12-26 15:33:38,941][105692] Updated weights for policy 0, policy_version 26710 (0.0008) [2023-12-26 15:33:38,961][105620] Updated weights for policy 1, policy_version 26813 (0.0007) [2023-12-26 15:33:38,994][105692] Updated weights for policy 0, policy_version 26720 (0.0005) [2023-12-26 15:33:39,020][105620] Updated weights for policy 1, policy_version 26823 (0.0008) [2023-12-26 15:33:39,044][105692] Updated weights for policy 0, policy_version 26730 (0.0005) [2023-12-26 15:33:39,739][105620] Updated weights for policy 1, policy_version 26833 (0.0009) [2023-12-26 15:33:39,762][105692] Updated weights for policy 0, policy_version 26740 (0.0006) [2023-12-26 15:33:39,790][105620] Updated weights for policy 1, policy_version 26843 (0.0006) [2023-12-26 15:33:39,820][105692] Updated weights for policy 0, policy_version 26750 (0.0007) [2023-12-26 15:33:39,851][105620] Updated weights for policy 1, policy_version 26853 (0.0008) [2023-12-26 15:33:39,881][105692] Updated weights for policy 0, policy_version 26760 (0.0006) [2023-12-26 15:33:39,919][105620] Updated weights for policy 1, policy_version 26863 (0.0008) [2023-12-26 15:33:40,622][105620] Updated weights for policy 1, policy_version 26873 (0.0007) [2023-12-26 15:33:40,677][105620] Updated weights for policy 1, policy_version 26883 (0.0008) [2023-12-26 15:33:40,686][105692] Updated weights for policy 0, policy_version 26770 (0.0008) [2023-12-26 15:33:40,739][105620] Updated weights for policy 1, policy_version 26893 (0.0008) [2023-12-26 15:33:40,749][105692] Updated weights for policy 0, policy_version 26780 (0.0006) [2023-12-26 15:33:40,802][105692] Updated weights for policy 0, policy_version 26790 (0.0009) [2023-12-26 15:33:40,854][105692] Updated weights for policy 0, policy_version 26800 (0.0009) [2023-12-26 15:33:41,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19716.4). Total num frames: 13754368. Throughput: 0: 9769.9, 1: 9928.6. Samples: 13760808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:33:41,063][104569] Avg episode reward: [(0, '8704.520'), (1, '8893.826')] [2023-12-26 15:33:41,419][105620] Updated weights for policy 1, policy_version 26903 (0.0009) [2023-12-26 15:33:41,483][105620] Updated weights for policy 1, policy_version 26913 (0.0007) [2023-12-26 15:33:41,550][105620] Updated weights for policy 1, policy_version 26923 (0.0006) [2023-12-26 15:33:41,683][105692] Updated weights for policy 0, policy_version 26810 (0.0009) [2023-12-26 15:33:41,755][105692] Updated weights for policy 0, policy_version 26820 (0.0008) [2023-12-26 15:33:41,810][105692] Updated weights for policy 0, policy_version 26830 (0.0008) [2023-12-26 15:33:42,254][105620] Updated weights for policy 1, policy_version 26933 (0.0008) [2023-12-26 15:33:42,311][105620] Updated weights for policy 1, policy_version 26943 (0.0009) [2023-12-26 15:33:42,378][105620] Updated weights for policy 1, policy_version 26953 (0.0009) [2023-12-26 15:33:42,550][105692] Updated weights for policy 0, policy_version 26840 (0.0009) [2023-12-26 15:33:42,613][105692] Updated weights for policy 0, policy_version 26850 (0.0009) [2023-12-26 15:33:42,674][105692] Updated weights for policy 0, policy_version 26860 (0.0009) [2023-12-26 15:33:43,116][105620] Updated weights for policy 1, policy_version 26963 (0.0008) [2023-12-26 15:33:43,168][105620] Updated weights for policy 1, policy_version 26973 (0.0005) [2023-12-26 15:33:43,220][105620] Updated weights for policy 1, policy_version 26983 (0.0005) [2023-12-26 15:33:43,484][105692] Updated weights for policy 0, policy_version 26870 (0.0009) [2023-12-26 15:33:43,535][105692] Updated weights for policy 0, policy_version 26880 (0.0009) [2023-12-26 15:33:43,601][105692] Updated weights for policy 0, policy_version 26890 (0.0009) [2023-12-26 15:33:43,880][105620] Updated weights for policy 1, policy_version 26993 (0.0006) [2023-12-26 15:33:43,940][105620] Updated weights for policy 1, policy_version 27003 (0.0009) [2023-12-26 15:33:43,997][105620] Updated weights for policy 1, policy_version 27013 (0.0010) [2023-12-26 15:33:44,050][105620] Updated weights for policy 1, policy_version 27023 (0.0010) [2023-12-26 15:33:44,302][105692] Updated weights for policy 0, policy_version 26900 (0.0009) [2023-12-26 15:33:44,353][105692] Updated weights for policy 0, policy_version 26910 (0.0010) [2023-12-26 15:33:44,404][105692] Updated weights for policy 0, policy_version 26920 (0.0010) [2023-12-26 15:33:44,839][105620] Updated weights for policy 1, policy_version 27033 (0.0009) [2023-12-26 15:33:44,899][105620] Updated weights for policy 1, policy_version 27043 (0.0009) [2023-12-26 15:33:44,959][105620] Updated weights for policy 1, policy_version 27053 (0.0009) [2023-12-26 15:33:45,145][105692] Updated weights for policy 0, policy_version 26930 (0.0010) [2023-12-26 15:33:45,199][105692] Updated weights for policy 0, policy_version 26940 (0.0007) [2023-12-26 15:33:45,257][105692] Updated weights for policy 0, policy_version 26950 (0.0009) [2023-12-26 15:33:45,315][105692] Updated weights for policy 0, policy_version 26960 (0.0009) [2023-12-26 15:33:45,734][105620] Updated weights for policy 1, policy_version 27063 (0.0010) [2023-12-26 15:33:45,790][105620] Updated weights for policy 1, policy_version 27073 (0.0010) [2023-12-26 15:33:45,835][105620] Updated weights for policy 1, policy_version 27083 (0.0010) [2023-12-26 15:33:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19716.3). Total num frames: 13844480. Throughput: 0: 9658.7, 1: 9897.7. Samples: 13816208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:33:46,062][104569] Avg episode reward: [(0, '9164.519'), (1, '8986.966')] [2023-12-26 15:33:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000027088_6938624.pth... [2023-12-26 15:33:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000025904_6635520.pth [2023-12-26 15:33:46,085][105692] Updated weights for policy 0, policy_version 26970 (0.0008) [2023-12-26 15:33:46,146][105692] Updated weights for policy 0, policy_version 26980 (0.0008) [2023-12-26 15:33:46,206][105692] Updated weights for policy 0, policy_version 26990 (0.0008) [2023-12-26 15:33:46,213][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000026992_6914048.pth... [2023-12-26 15:33:46,217][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000025840_6619136.pth [2023-12-26 15:33:46,596][105620] Updated weights for policy 1, policy_version 27093 (0.0010) [2023-12-26 15:33:46,647][105620] Updated weights for policy 1, policy_version 27103 (0.0011) [2023-12-26 15:33:46,696][105620] Updated weights for policy 1, policy_version 27113 (0.0010) [2023-12-26 15:33:46,980][105692] Updated weights for policy 0, policy_version 27000 (0.0008) [2023-12-26 15:33:47,041][105692] Updated weights for policy 0, policy_version 27010 (0.0008) [2023-12-26 15:33:47,096][105692] Updated weights for policy 0, policy_version 27020 (0.0008) [2023-12-26 15:33:47,463][105620] Updated weights for policy 1, policy_version 27123 (0.0010) [2023-12-26 15:33:47,521][105620] Updated weights for policy 1, policy_version 27133 (0.0010) [2023-12-26 15:33:47,579][105620] Updated weights for policy 1, policy_version 27143 (0.0010) [2023-12-26 15:33:47,836][105692] Updated weights for policy 0, policy_version 27030 (0.0008) [2023-12-26 15:33:47,887][105692] Updated weights for policy 0, policy_version 27040 (0.0008) [2023-12-26 15:33:47,939][105692] Updated weights for policy 0, policy_version 27050 (0.0008) [2023-12-26 15:33:48,316][105620] Updated weights for policy 1, policy_version 27153 (0.0010) [2023-12-26 15:33:48,372][105620] Updated weights for policy 1, policy_version 27163 (0.0010) [2023-12-26 15:33:48,434][105620] Updated weights for policy 1, policy_version 27173 (0.0010) [2023-12-26 15:33:48,496][105620] Updated weights for policy 1, policy_version 27183 (0.0010) [2023-12-26 15:33:48,738][105692] Updated weights for policy 0, policy_version 27060 (0.0008) [2023-12-26 15:33:48,798][105692] Updated weights for policy 0, policy_version 27070 (0.0008) [2023-12-26 15:33:48,846][105692] Updated weights for policy 0, policy_version 27080 (0.0008) [2023-12-26 15:33:49,233][105620] Updated weights for policy 1, policy_version 27193 (0.0010) [2023-12-26 15:33:49,295][105620] Updated weights for policy 1, policy_version 27203 (0.0010) [2023-12-26 15:33:49,357][105620] Updated weights for policy 1, policy_version 27213 (0.0010) [2023-12-26 15:33:49,632][105692] Updated weights for policy 0, policy_version 27090 (0.0008) [2023-12-26 15:33:49,696][105692] Updated weights for policy 0, policy_version 27100 (0.0009) [2023-12-26 15:33:49,755][105692] Updated weights for policy 0, policy_version 27110 (0.0008) [2023-12-26 15:33:49,815][105692] Updated weights for policy 0, policy_version 27120 (0.0008) [2023-12-26 15:33:50,122][105620] Updated weights for policy 1, policy_version 27223 (0.0009) [2023-12-26 15:33:50,186][105620] Updated weights for policy 1, policy_version 27233 (0.0009) [2023-12-26 15:33:50,245][105620] Updated weights for policy 1, policy_version 27243 (0.0009) [2023-12-26 15:33:50,567][105692] Updated weights for policy 0, policy_version 27130 (0.0010) [2023-12-26 15:33:50,632][105692] Updated weights for policy 0, policy_version 27140 (0.0010) [2023-12-26 15:33:50,693][105692] Updated weights for policy 0, policy_version 27150 (0.0009) [2023-12-26 15:33:50,958][105620] Updated weights for policy 1, policy_version 27253 (0.0009) [2023-12-26 15:33:51,006][105620] Updated weights for policy 1, policy_version 27263 (0.0009) [2023-12-26 15:33:51,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19387.7, 300 sec: 19688.6). Total num frames: 13934592. Throughput: 0: 9594.5, 1: 9909.5. Samples: 13927392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:33:51,063][104569] Avg episode reward: [(0, '9166.539'), (1, '8894.411')] [2023-12-26 15:33:51,067][105620] Updated weights for policy 1, policy_version 27273 (0.0008) [2023-12-26 15:33:51,467][105692] Updated weights for policy 0, policy_version 27160 (0.0010) [2023-12-26 15:33:51,527][105692] Updated weights for policy 0, policy_version 27170 (0.0009) [2023-12-26 15:33:51,581][105692] Updated weights for policy 0, policy_version 27180 (0.0009) [2023-12-26 15:33:51,825][105620] Updated weights for policy 1, policy_version 27283 (0.0009) [2023-12-26 15:33:51,888][105620] Updated weights for policy 1, policy_version 27293 (0.0008) [2023-12-26 15:33:51,949][105620] Updated weights for policy 1, policy_version 27303 (0.0009) [2023-12-26 15:33:52,422][105692] Updated weights for policy 0, policy_version 27190 (0.0009) [2023-12-26 15:33:52,482][105692] Updated weights for policy 0, policy_version 27200 (0.0008) [2023-12-26 15:33:52,529][105692] Updated weights for policy 0, policy_version 27210 (0.0008) [2023-12-26 15:33:52,615][105620] Updated weights for policy 1, policy_version 27313 (0.0006) [2023-12-26 15:33:52,670][105620] Updated weights for policy 1, policy_version 27323 (0.0009) [2023-12-26 15:33:52,722][105620] Updated weights for policy 1, policy_version 27333 (0.0009) [2023-12-26 15:33:52,769][105620] Updated weights for policy 1, policy_version 27343 (0.0009) [2023-12-26 15:33:53,283][105692] Updated weights for policy 0, policy_version 27220 (0.0009) [2023-12-26 15:33:53,330][105692] Updated weights for policy 0, policy_version 27230 (0.0009) [2023-12-26 15:33:53,377][105692] Updated weights for policy 0, policy_version 27240 (0.0009) [2023-12-26 15:33:53,542][105620] Updated weights for policy 1, policy_version 27353 (0.0007) [2023-12-26 15:33:53,589][105620] Updated weights for policy 1, policy_version 27363 (0.0005) [2023-12-26 15:33:53,638][105620] Updated weights for policy 1, policy_version 27373 (0.0005) [2023-12-26 15:33:54,195][105620] Updated weights for policy 1, policy_version 27383 (0.0009) [2023-12-26 15:33:54,254][105620] Updated weights for policy 1, policy_version 27393 (0.0007) [2023-12-26 15:33:54,267][105692] Updated weights for policy 0, policy_version 27250 (0.0008) [2023-12-26 15:33:54,306][105620] Updated weights for policy 1, policy_version 27403 (0.0006) [2023-12-26 15:33:54,321][105692] Updated weights for policy 0, policy_version 27260 (0.0009) [2023-12-26 15:33:54,377][105692] Updated weights for policy 0, policy_version 27270 (0.0009) [2023-12-26 15:33:54,430][105692] Updated weights for policy 0, policy_version 27280 (0.0010) [2023-12-26 15:33:54,918][105620] Updated weights for policy 1, policy_version 27413 (0.0008) [2023-12-26 15:33:54,981][105620] Updated weights for policy 1, policy_version 27423 (0.0010) [2023-12-26 15:33:55,042][105620] Updated weights for policy 1, policy_version 27433 (0.0006) [2023-12-26 15:33:55,228][105692] Updated weights for policy 0, policy_version 27290 (0.0008) [2023-12-26 15:33:55,273][105692] Updated weights for policy 0, policy_version 27300 (0.0008) [2023-12-26 15:33:55,335][105692] Updated weights for policy 0, policy_version 27310 (0.0008) [2023-12-26 15:33:55,726][105620] Updated weights for policy 1, policy_version 27443 (0.0010) [2023-12-26 15:33:55,788][105620] Updated weights for policy 1, policy_version 27453 (0.0010) [2023-12-26 15:33:55,843][105620] Updated weights for policy 1, policy_version 27463 (0.0010) [2023-12-26 15:33:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19688.6). Total num frames: 14032896. Throughput: 0: 9498.8, 1: 9955.1. Samples: 14041168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:33:56,063][104569] Avg episode reward: [(0, '9075.293'), (1, '9171.807')] [2023-12-26 15:33:56,181][105692] Updated weights for policy 0, policy_version 27320 (0.0009) [2023-12-26 15:33:56,249][105692] Updated weights for policy 0, policy_version 27330 (0.0010) [2023-12-26 15:33:56,307][105692] Updated weights for policy 0, policy_version 27340 (0.0009) [2023-12-26 15:33:56,416][105620] Updated weights for policy 1, policy_version 27473 (0.0010) [2023-12-26 15:33:56,475][105620] Updated weights for policy 1, policy_version 27483 (0.0008) [2023-12-26 15:33:56,524][105620] Updated weights for policy 1, policy_version 27493 (0.0005) [2023-12-26 15:33:56,583][105620] Updated weights for policy 1, policy_version 27503 (0.0006) [2023-12-26 15:33:57,165][105692] Updated weights for policy 0, policy_version 27350 (0.0008) [2023-12-26 15:33:57,178][105620] Updated weights for policy 1, policy_version 27513 (0.0008) [2023-12-26 15:33:57,216][105692] Updated weights for policy 0, policy_version 27360 (0.0006) [2023-12-26 15:33:57,229][105620] Updated weights for policy 1, policy_version 27523 (0.0007) [2023-12-26 15:33:57,270][105692] Updated weights for policy 0, policy_version 27370 (0.0006) [2023-12-26 15:33:57,288][105620] Updated weights for policy 1, policy_version 27533 (0.0007) [2023-12-26 15:33:57,905][105620] Updated weights for policy 1, policy_version 27543 (0.0005) [2023-12-26 15:33:57,972][105620] Updated weights for policy 1, policy_version 27553 (0.0005) [2023-12-26 15:33:58,038][105620] Updated weights for policy 1, policy_version 27563 (0.0007) [2023-12-26 15:33:58,101][105692] Updated weights for policy 0, policy_version 27380 (0.0007) [2023-12-26 15:33:58,160][105692] Updated weights for policy 0, policy_version 27390 (0.0008) [2023-12-26 15:33:58,218][105692] Updated weights for policy 0, policy_version 27400 (0.0010) [2023-12-26 15:33:58,777][105620] Updated weights for policy 1, policy_version 27573 (0.0008) [2023-12-26 15:33:58,838][105620] Updated weights for policy 1, policy_version 27583 (0.0008) [2023-12-26 15:33:58,899][105620] Updated weights for policy 1, policy_version 27593 (0.0008) [2023-12-26 15:33:58,974][105692] Updated weights for policy 0, policy_version 27410 (0.0009) [2023-12-26 15:33:59,021][105692] Updated weights for policy 0, policy_version 27420 (0.0008) [2023-12-26 15:33:59,068][105692] Updated weights for policy 0, policy_version 27430 (0.0007) [2023-12-26 15:33:59,126][105692] Updated weights for policy 0, policy_version 27440 (0.0006) [2023-12-26 15:33:59,591][105620] Updated weights for policy 1, policy_version 27603 (0.0008) [2023-12-26 15:33:59,649][105620] Updated weights for policy 1, policy_version 27613 (0.0005) [2023-12-26 15:33:59,710][105620] Updated weights for policy 1, policy_version 27623 (0.0007) [2023-12-26 15:33:59,940][105692] Updated weights for policy 0, policy_version 27450 (0.0008) [2023-12-26 15:33:59,991][105692] Updated weights for policy 0, policy_version 27460 (0.0008) [2023-12-26 15:34:00,047][105692] Updated weights for policy 0, policy_version 27470 (0.0008) [2023-12-26 15:34:00,398][105620] Updated weights for policy 1, policy_version 27633 (0.0010) [2023-12-26 15:34:00,460][105620] Updated weights for policy 1, policy_version 27643 (0.0007) [2023-12-26 15:34:00,521][105620] Updated weights for policy 1, policy_version 27653 (0.0009) [2023-12-26 15:34:00,583][105620] Updated weights for policy 1, policy_version 27663 (0.0009) [2023-12-26 15:34:00,822][105692] Updated weights for policy 0, policy_version 27480 (0.0009) [2023-12-26 15:34:00,877][105692] Updated weights for policy 0, policy_version 27490 (0.0006) [2023-12-26 15:34:00,940][105692] Updated weights for policy 0, policy_version 27500 (0.0005) [2023-12-26 15:34:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 14131200. Throughput: 0: 9446.3, 1: 10019.5. Samples: 14099272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:34:01,062][104569] Avg episode reward: [(0, '9166.932'), (1, '9173.830')] [2023-12-26 15:34:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000027504_7045120.pth... [2023-12-26 15:34:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000027664_7086080.pth... [2023-12-26 15:34:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000026416_6766592.pth [2023-12-26 15:34:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000026480_6782976.pth [2023-12-26 15:34:01,328][105620] Updated weights for policy 1, policy_version 27673 (0.0010) [2023-12-26 15:34:01,397][105620] Updated weights for policy 1, policy_version 27683 (0.0008) [2023-12-26 15:34:01,460][105620] Updated weights for policy 1, policy_version 27693 (0.0006) [2023-12-26 15:34:01,623][105692] Updated weights for policy 0, policy_version 27510 (0.0007) [2023-12-26 15:34:01,684][105692] Updated weights for policy 0, policy_version 27520 (0.0008) [2023-12-26 15:34:01,741][105692] Updated weights for policy 0, policy_version 27530 (0.0008) [2023-12-26 15:34:02,226][105620] Updated weights for policy 1, policy_version 27703 (0.0008) [2023-12-26 15:34:02,290][105620] Updated weights for policy 1, policy_version 27713 (0.0009) [2023-12-26 15:34:02,347][105620] Updated weights for policy 1, policy_version 27723 (0.0008) [2023-12-26 15:34:02,489][105692] Updated weights for policy 0, policy_version 27540 (0.0009) [2023-12-26 15:34:02,548][105692] Updated weights for policy 0, policy_version 27550 (0.0009) [2023-12-26 15:34:02,605][105692] Updated weights for policy 0, policy_version 27560 (0.0008) [2023-12-26 15:34:03,089][105620] Updated weights for policy 1, policy_version 27733 (0.0009) [2023-12-26 15:34:03,143][105620] Updated weights for policy 1, policy_version 27743 (0.0009) [2023-12-26 15:34:03,194][105620] Updated weights for policy 1, policy_version 27753 (0.0009) [2023-12-26 15:34:03,302][105692] Updated weights for policy 0, policy_version 27570 (0.0009) [2023-12-26 15:34:03,353][105692] Updated weights for policy 0, policy_version 27580 (0.0009) [2023-12-26 15:34:03,413][105692] Updated weights for policy 0, policy_version 27590 (0.0009) [2023-12-26 15:34:03,473][105692] Updated weights for policy 0, policy_version 27600 (0.0009) [2023-12-26 15:34:03,889][105620] Updated weights for policy 1, policy_version 27763 (0.0008) [2023-12-26 15:34:03,955][105620] Updated weights for policy 1, policy_version 27773 (0.0009) [2023-12-26 15:34:04,018][105620] Updated weights for policy 1, policy_version 27783 (0.0010) [2023-12-26 15:34:04,147][105692] Updated weights for policy 0, policy_version 27610 (0.0009) [2023-12-26 15:34:04,200][105692] Updated weights for policy 0, policy_version 27620 (0.0010) [2023-12-26 15:34:04,259][105692] Updated weights for policy 0, policy_version 27630 (0.0008) [2023-12-26 15:34:04,759][105620] Updated weights for policy 1, policy_version 27793 (0.0009) [2023-12-26 15:34:04,807][105620] Updated weights for policy 1, policy_version 27803 (0.0008) [2023-12-26 15:34:04,852][105620] Updated weights for policy 1, policy_version 27813 (0.0007) [2023-12-26 15:34:04,901][105620] Updated weights for policy 1, policy_version 27823 (0.0006) [2023-12-26 15:34:05,002][105692] Updated weights for policy 0, policy_version 27640 (0.0006) [2023-12-26 15:34:05,060][105692] Updated weights for policy 0, policy_version 27650 (0.0006) [2023-12-26 15:34:05,122][105692] Updated weights for policy 0, policy_version 27660 (0.0006) [2023-12-26 15:34:05,593][105620] Updated weights for policy 1, policy_version 27833 (0.0007) [2023-12-26 15:34:05,641][105620] Updated weights for policy 1, policy_version 27843 (0.0005) [2023-12-26 15:34:05,698][105620] Updated weights for policy 1, policy_version 27853 (0.0005) [2023-12-26 15:34:05,723][105692] Updated weights for policy 0, policy_version 27670 (0.0010) [2023-12-26 15:34:05,791][105692] Updated weights for policy 0, policy_version 27680 (0.0011) [2023-12-26 15:34:05,852][105692] Updated weights for policy 0, policy_version 27690 (0.0010) [2023-12-26 15:34:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19688.6). Total num frames: 14229504. Throughput: 0: 9368.1, 1: 9977.5. Samples: 14214156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:34:06,063][104569] Avg episode reward: [(0, '9351.763'), (1, '9083.401')] [2023-12-26 15:34:06,063][105585] Saving new best policy, reward=9351.763! [2023-12-26 15:34:06,381][105620] Updated weights for policy 1, policy_version 27863 (0.0008) [2023-12-26 15:34:06,446][105620] Updated weights for policy 1, policy_version 27873 (0.0010) [2023-12-26 15:34:06,503][105620] Updated weights for policy 1, policy_version 27883 (0.0011) [2023-12-26 15:34:06,546][105692] Updated weights for policy 0, policy_version 27700 (0.0008) [2023-12-26 15:34:06,605][105692] Updated weights for policy 0, policy_version 27710 (0.0006) [2023-12-26 15:34:06,670][105692] Updated weights for policy 0, policy_version 27720 (0.0010) [2023-12-26 15:34:07,174][105620] Updated weights for policy 1, policy_version 27893 (0.0008) [2023-12-26 15:34:07,225][105620] Updated weights for policy 1, policy_version 27903 (0.0005) [2023-12-26 15:34:07,283][105692] Updated weights for policy 0, policy_version 27730 (0.0010) [2023-12-26 15:34:07,285][105620] Updated weights for policy 1, policy_version 27913 (0.0006) [2023-12-26 15:34:07,343][105692] Updated weights for policy 0, policy_version 27740 (0.0007) [2023-12-26 15:34:07,400][105692] Updated weights for policy 0, policy_version 27750 (0.0008) [2023-12-26 15:34:07,445][105692] Updated weights for policy 0, policy_version 27760 (0.0008) [2023-12-26 15:34:07,878][105620] Updated weights for policy 1, policy_version 27923 (0.0007) [2023-12-26 15:34:07,937][105620] Updated weights for policy 1, policy_version 27933 (0.0006) [2023-12-26 15:34:07,996][105692] Updated weights for policy 0, policy_version 27770 (0.0005) [2023-12-26 15:34:07,997][105620] Updated weights for policy 1, policy_version 27943 (0.0010) [2023-12-26 15:34:08,055][105692] Updated weights for policy 0, policy_version 27780 (0.0006) [2023-12-26 15:34:08,119][105692] Updated weights for policy 0, policy_version 27790 (0.0006) [2023-12-26 15:34:08,571][105620] Updated weights for policy 1, policy_version 27953 (0.0010) [2023-12-26 15:34:08,637][105620] Updated weights for policy 1, policy_version 27963 (0.0006) [2023-12-26 15:34:08,692][105620] Updated weights for policy 1, policy_version 27973 (0.0005) [2023-12-26 15:34:08,742][105692] Updated weights for policy 0, policy_version 27800 (0.0010) [2023-12-26 15:34:08,747][105620] Updated weights for policy 1, policy_version 27983 (0.0010) [2023-12-26 15:34:08,801][105692] Updated weights for policy 0, policy_version 27810 (0.0011) [2023-12-26 15:34:08,868][105692] Updated weights for policy 0, policy_version 27820 (0.0010) [2023-12-26 15:34:09,369][105620] Updated weights for policy 1, policy_version 27993 (0.0009) [2023-12-26 15:34:09,432][105620] Updated weights for policy 1, policy_version 28003 (0.0008) [2023-12-26 15:34:09,498][105620] Updated weights for policy 1, policy_version 28013 (0.0010) [2023-12-26 15:34:09,609][105692] Updated weights for policy 0, policy_version 27830 (0.0011) [2023-12-26 15:34:09,668][105692] Updated weights for policy 0, policy_version 27840 (0.0011) [2023-12-26 15:34:09,732][105692] Updated weights for policy 0, policy_version 27850 (0.0011) [2023-12-26 15:34:10,239][105620] Updated weights for policy 1, policy_version 28023 (0.0010) [2023-12-26 15:34:10,291][105620] Updated weights for policy 1, policy_version 28033 (0.0010) [2023-12-26 15:34:10,347][105620] Updated weights for policy 1, policy_version 28043 (0.0010) [2023-12-26 15:34:10,499][105692] Updated weights for policy 0, policy_version 27860 (0.0010) [2023-12-26 15:34:10,564][105692] Updated weights for policy 0, policy_version 27870 (0.0010) [2023-12-26 15:34:10,623][105692] Updated weights for policy 0, policy_version 27880 (0.0010) [2023-12-26 15:34:10,997][105620] Updated weights for policy 1, policy_version 28053 (0.0010) [2023-12-26 15:34:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19688.6). Total num frames: 14327808. Throughput: 0: 9453.3, 1: 10055.4. Samples: 14340060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:34:11,063][104569] Avg episode reward: [(0, '9352.334'), (1, '8988.787')] [2023-12-26 15:34:11,063][105585] Saving new best policy, reward=9352.334! [2023-12-26 15:34:11,065][105620] Updated weights for policy 1, policy_version 28063 (0.0010) [2023-12-26 15:34:11,122][105620] Updated weights for policy 1, policy_version 28073 (0.0009) [2023-12-26 15:34:11,302][105692] Updated weights for policy 0, policy_version 27890 (0.0009) [2023-12-26 15:34:11,373][105692] Updated weights for policy 0, policy_version 27900 (0.0009) [2023-12-26 15:34:11,438][105692] Updated weights for policy 0, policy_version 27910 (0.0007) [2023-12-26 15:34:11,501][105692] Updated weights for policy 0, policy_version 27920 (0.0008) [2023-12-26 15:34:12,008][105620] Updated weights for policy 1, policy_version 28083 (0.0011) [2023-12-26 15:34:12,072][105620] Updated weights for policy 1, policy_version 28093 (0.0008) [2023-12-26 15:34:12,140][105620] Updated weights for policy 1, policy_version 28103 (0.0008) [2023-12-26 15:34:12,143][105692] Updated weights for policy 0, policy_version 27930 (0.0007) [2023-12-26 15:34:12,203][105692] Updated weights for policy 0, policy_version 27940 (0.0008) [2023-12-26 15:34:12,267][105692] Updated weights for policy 0, policy_version 27950 (0.0006) [2023-12-26 15:34:12,849][105692] Updated weights for policy 0, policy_version 27960 (0.0010) [2023-12-26 15:34:12,908][105692] Updated weights for policy 0, policy_version 27970 (0.0010) [2023-12-26 15:34:12,976][105620] Updated weights for policy 1, policy_version 28113 (0.0006) [2023-12-26 15:34:12,993][105692] Updated weights for policy 0, policy_version 27980 (0.0010) [2023-12-26 15:34:13,039][105620] Updated weights for policy 1, policy_version 28123 (0.0006) [2023-12-26 15:34:13,101][105620] Updated weights for policy 1, policy_version 28133 (0.0008) [2023-12-26 15:34:13,158][105620] Updated weights for policy 1, policy_version 28143 (0.0009) [2023-12-26 15:34:13,675][105692] Updated weights for policy 0, policy_version 27990 (0.0010) [2023-12-26 15:34:13,726][105692] Updated weights for policy 0, policy_version 28000 (0.0010) [2023-12-26 15:34:13,776][105692] Updated weights for policy 0, policy_version 28010 (0.0006) [2023-12-26 15:34:13,937][105620] Updated weights for policy 1, policy_version 28153 (0.0009) [2023-12-26 15:34:13,989][105620] Updated weights for policy 1, policy_version 28163 (0.0009) [2023-12-26 15:34:14,044][105620] Updated weights for policy 1, policy_version 28173 (0.0008) [2023-12-26 15:34:14,450][105692] Updated weights for policy 0, policy_version 28020 (0.0007) [2023-12-26 15:34:14,514][105692] Updated weights for policy 0, policy_version 28030 (0.0010) [2023-12-26 15:34:14,573][105692] Updated weights for policy 0, policy_version 28040 (0.0010) [2023-12-26 15:34:14,888][105620] Updated weights for policy 1, policy_version 28183 (0.0009) [2023-12-26 15:34:14,954][105620] Updated weights for policy 1, policy_version 28193 (0.0010) [2023-12-26 15:34:15,024][105620] Updated weights for policy 1, policy_version 28203 (0.0009) [2023-12-26 15:34:15,115][105692] Updated weights for policy 0, policy_version 28050 (0.0006) [2023-12-26 15:34:15,176][105692] Updated weights for policy 0, policy_version 28060 (0.0010) [2023-12-26 15:34:15,239][105692] Updated weights for policy 0, policy_version 28070 (0.0011) [2023-12-26 15:34:15,305][105692] Updated weights for policy 0, policy_version 28080 (0.0011) [2023-12-26 15:34:15,832][105620] Updated weights for policy 1, policy_version 28213 (0.0009) [2023-12-26 15:34:15,892][105620] Updated weights for policy 1, policy_version 28223 (0.0008) [2023-12-26 15:34:15,940][105620] Updated weights for policy 1, policy_version 28233 (0.0008) [2023-12-26 15:34:16,028][105692] Updated weights for policy 0, policy_version 28090 (0.0010) [2023-12-26 15:34:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 14426112. Throughput: 0: 9501.6, 1: 9943.5. Samples: 14396408. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 15:34:16,062][104569] Avg episode reward: [(0, '9258.713'), (1, '9078.923')] [2023-12-26 15:34:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000028240_7233536.pth... [2023-12-26 15:34:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000027088_6938624.pth [2023-12-26 15:34:16,086][105692] Updated weights for policy 0, policy_version 28100 (0.0010) [2023-12-26 15:34:16,141][105692] Updated weights for policy 0, policy_version 28110 (0.0010) [2023-12-26 15:34:16,148][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000028112_7200768.pth... [2023-12-26 15:34:16,151][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000026992_6914048.pth [2023-12-26 15:34:16,746][105620] Updated weights for policy 1, policy_version 28243 (0.0007) [2023-12-26 15:34:16,751][105692] Updated weights for policy 0, policy_version 28120 (0.0007) [2023-12-26 15:34:16,802][105692] Updated weights for policy 0, policy_version 28130 (0.0007) [2023-12-26 15:34:16,811][105620] Updated weights for policy 1, policy_version 28253 (0.0006) [2023-12-26 15:34:16,854][105692] Updated weights for policy 0, policy_version 28140 (0.0008) [2023-12-26 15:34:16,879][105620] Updated weights for policy 1, policy_version 28263 (0.0005) [2023-12-26 15:34:17,571][105692] Updated weights for policy 0, policy_version 28150 (0.0008) [2023-12-26 15:34:17,589][105620] Updated weights for policy 1, policy_version 28273 (0.0009) [2023-12-26 15:34:17,624][105692] Updated weights for policy 0, policy_version 28160 (0.0008) [2023-12-26 15:34:17,636][105620] Updated weights for policy 1, policy_version 28284 (0.0008) [2023-12-26 15:34:17,675][105692] Updated weights for policy 0, policy_version 28170 (0.0006) [2023-12-26 15:34:17,685][105620] Updated weights for policy 1, policy_version 28294 (0.0007) [2023-12-26 15:34:17,740][105620] Updated weights for policy 1, policy_version 28304 (0.0006) [2023-12-26 15:34:18,372][105620] Updated weights for policy 1, policy_version 28314 (0.0009) [2023-12-26 15:34:18,441][105620] Updated weights for policy 1, policy_version 28324 (0.0008) [2023-12-26 15:34:18,446][105692] Updated weights for policy 0, policy_version 28180 (0.0006) [2023-12-26 15:34:18,496][105620] Updated weights for policy 1, policy_version 28334 (0.0008) [2023-12-26 15:34:18,507][105692] Updated weights for policy 0, policy_version 28190 (0.0009) [2023-12-26 15:34:18,569][105692] Updated weights for policy 0, policy_version 28200 (0.0009) [2023-12-26 15:34:19,233][105692] Updated weights for policy 0, policy_version 28210 (0.0009) [2023-12-26 15:34:19,246][105620] Updated weights for policy 1, policy_version 28344 (0.0009) [2023-12-26 15:34:19,294][105692] Updated weights for policy 0, policy_version 28220 (0.0007) [2023-12-26 15:34:19,305][105620] Updated weights for policy 1, policy_version 28354 (0.0007) [2023-12-26 15:34:19,358][105692] Updated weights for policy 0, policy_version 28230 (0.0006) [2023-12-26 15:34:19,377][105620] Updated weights for policy 1, policy_version 28364 (0.0009) [2023-12-26 15:34:19,417][105692] Updated weights for policy 0, policy_version 28240 (0.0008) [2023-12-26 15:34:20,077][105692] Updated weights for policy 0, policy_version 28250 (0.0005) [2023-12-26 15:34:20,147][105692] Updated weights for policy 0, policy_version 28260 (0.0005) [2023-12-26 15:34:20,204][105692] Updated weights for policy 0, policy_version 28270 (0.0006) [2023-12-26 15:34:20,211][105620] Updated weights for policy 1, policy_version 28374 (0.0007) [2023-12-26 15:34:20,275][105620] Updated weights for policy 1, policy_version 28384 (0.0009) [2023-12-26 15:34:20,333][105620] Updated weights for policy 1, policy_version 28394 (0.0009) [2023-12-26 15:34:20,908][105692] Updated weights for policy 0, policy_version 28280 (0.0009) [2023-12-26 15:34:20,980][105692] Updated weights for policy 0, policy_version 28290 (0.0010) [2023-12-26 15:34:21,047][105692] Updated weights for policy 0, policy_version 28300 (0.0009) [2023-12-26 15:34:21,050][105620] Updated weights for policy 1, policy_version 28405 (0.0008) [2023-12-26 15:34:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19688.6). Total num frames: 14516224. Throughput: 0: 9581.0, 1: 9751.5. Samples: 14512312. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 15:34:21,062][104569] Avg episode reward: [(0, '9256.327'), (1, '8985.833')] [2023-12-26 15:34:21,117][105620] Updated weights for policy 1, policy_version 28415 (0.0007) [2023-12-26 15:34:21,176][105620] Updated weights for policy 1, policy_version 28425 (0.0009) [2023-12-26 15:34:21,694][105692] Updated weights for policy 0, policy_version 28310 (0.0009) [2023-12-26 15:34:21,756][105692] Updated weights for policy 0, policy_version 28320 (0.0011) [2023-12-26 15:34:21,807][105692] Updated weights for policy 0, policy_version 28330 (0.0008) [2023-12-26 15:34:21,904][105620] Updated weights for policy 1, policy_version 28435 (0.0010) [2023-12-26 15:34:21,965][105620] Updated weights for policy 1, policy_version 28445 (0.0008) [2023-12-26 15:34:22,033][105620] Updated weights for policy 1, policy_version 28455 (0.0008) [2023-12-26 15:34:22,602][105692] Updated weights for policy 0, policy_version 28340 (0.0010) [2023-12-26 15:34:22,660][105692] Updated weights for policy 0, policy_version 28350 (0.0009) [2023-12-26 15:34:22,717][105692] Updated weights for policy 0, policy_version 28360 (0.0010) [2023-12-26 15:34:22,772][105620] Updated weights for policy 1, policy_version 28465 (0.0008) [2023-12-26 15:34:22,829][105620] Updated weights for policy 1, policy_version 28475 (0.0008) [2023-12-26 15:34:22,887][105620] Updated weights for policy 1, policy_version 28485 (0.0010) [2023-12-26 15:34:22,939][105620] Updated weights for policy 1, policy_version 28495 (0.0009) [2023-12-26 15:34:23,460][105692] Updated weights for policy 0, policy_version 28370 (0.0007) [2023-12-26 15:34:23,517][105692] Updated weights for policy 0, policy_version 28380 (0.0006) [2023-12-26 15:34:23,585][105692] Updated weights for policy 0, policy_version 28390 (0.0007) [2023-12-26 15:34:23,646][105692] Updated weights for policy 0, policy_version 28400 (0.0009) [2023-12-26 15:34:23,719][105620] Updated weights for policy 1, policy_version 28505 (0.0010) [2023-12-26 15:34:23,780][105620] Updated weights for policy 1, policy_version 28515 (0.0009) [2023-12-26 15:34:23,840][105620] Updated weights for policy 1, policy_version 28525 (0.0009) [2023-12-26 15:34:24,287][105692] Updated weights for policy 0, policy_version 28410 (0.0006) [2023-12-26 15:34:24,335][105692] Updated weights for policy 0, policy_version 28420 (0.0007) [2023-12-26 15:34:24,392][105692] Updated weights for policy 0, policy_version 28430 (0.0009) [2023-12-26 15:34:24,629][105620] Updated weights for policy 1, policy_version 28535 (0.0009) [2023-12-26 15:34:24,682][105620] Updated weights for policy 1, policy_version 28545 (0.0010) [2023-12-26 15:34:24,739][105620] Updated weights for policy 1, policy_version 28555 (0.0009) [2023-12-26 15:34:25,013][105692] Updated weights for policy 0, policy_version 28440 (0.0009) [2023-12-26 15:34:25,074][105692] Updated weights for policy 0, policy_version 28450 (0.0009) [2023-12-26 15:34:25,136][105692] Updated weights for policy 0, policy_version 28460 (0.0009) [2023-12-26 15:34:25,485][105620] Updated weights for policy 1, policy_version 28565 (0.0008) [2023-12-26 15:34:25,541][105620] Updated weights for policy 1, policy_version 28575 (0.0005) [2023-12-26 15:34:25,595][105620] Updated weights for policy 1, policy_version 28585 (0.0005) [2023-12-26 15:34:25,862][105692] Updated weights for policy 0, policy_version 28470 (0.0010) [2023-12-26 15:34:25,909][105692] Updated weights for policy 0, policy_version 28480 (0.0007) [2023-12-26 15:34:25,960][105692] Updated weights for policy 0, policy_version 28490 (0.0005) [2023-12-26 15:34:26,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19688.6). Total num frames: 14622720. Throughput: 0: 9608.3, 1: 9676.9. Samples: 14628644. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 15:34:26,063][104569] Avg episode reward: [(0, '9347.610'), (1, '8985.660')] [2023-12-26 15:34:26,208][105620] Updated weights for policy 1, policy_version 28595 (0.0007) [2023-12-26 15:34:26,265][105620] Updated weights for policy 1, policy_version 28605 (0.0009) [2023-12-26 15:34:26,318][105620] Updated weights for policy 1, policy_version 28615 (0.0009) [2023-12-26 15:34:26,662][105692] Updated weights for policy 0, policy_version 28500 (0.0007) [2023-12-26 15:34:26,716][105692] Updated weights for policy 0, policy_version 28510 (0.0009) [2023-12-26 15:34:26,763][105692] Updated weights for policy 0, policy_version 28520 (0.0009) [2023-12-26 15:34:27,072][105620] Updated weights for policy 1, policy_version 28625 (0.0009) [2023-12-26 15:34:27,135][105620] Updated weights for policy 1, policy_version 28635 (0.0009) [2023-12-26 15:34:27,197][105620] Updated weights for policy 1, policy_version 28645 (0.0009) [2023-12-26 15:34:27,254][105620] Updated weights for policy 1, policy_version 28655 (0.0009) [2023-12-26 15:34:27,526][105692] Updated weights for policy 0, policy_version 28530 (0.0009) [2023-12-26 15:34:27,580][105692] Updated weights for policy 0, policy_version 28541 (0.0010) [2023-12-26 15:34:27,639][105692] Updated weights for policy 0, policy_version 28551 (0.0009) [2023-12-26 15:34:27,840][105620] Updated weights for policy 1, policy_version 28665 (0.0009) [2023-12-26 15:34:27,891][105620] Updated weights for policy 1, policy_version 28675 (0.0009) [2023-12-26 15:34:27,949][105620] Updated weights for policy 1, policy_version 28685 (0.0009) [2023-12-26 15:34:28,394][105692] Updated weights for policy 0, policy_version 28561 (0.0010) [2023-12-26 15:34:28,442][105692] Updated weights for policy 0, policy_version 28571 (0.0009) [2023-12-26 15:34:28,496][105692] Updated weights for policy 0, policy_version 28581 (0.0009) [2023-12-26 15:34:28,547][105692] Updated weights for policy 0, policy_version 28591 (0.0009) [2023-12-26 15:34:28,709][105620] Updated weights for policy 1, policy_version 28695 (0.0009) [2023-12-26 15:34:28,771][105620] Updated weights for policy 1, policy_version 28705 (0.0008) [2023-12-26 15:34:28,839][105620] Updated weights for policy 1, policy_version 28715 (0.0005) [2023-12-26 15:34:29,404][105692] Updated weights for policy 0, policy_version 28601 (0.0008) [2023-12-26 15:34:29,418][105620] Updated weights for policy 1, policy_version 28725 (0.0006) [2023-12-26 15:34:29,458][105692] Updated weights for policy 0, policy_version 28611 (0.0008) [2023-12-26 15:34:29,481][105620] Updated weights for policy 1, policy_version 28735 (0.0006) [2023-12-26 15:34:29,508][105692] Updated weights for policy 0, policy_version 28621 (0.0008) [2023-12-26 15:34:29,543][105620] Updated weights for policy 1, policy_version 28745 (0.0006) [2023-12-26 15:34:30,143][105620] Updated weights for policy 1, policy_version 28755 (0.0006) [2023-12-26 15:34:30,208][105620] Updated weights for policy 1, policy_version 28765 (0.0007) [2023-12-26 15:34:30,272][105620] Updated weights for policy 1, policy_version 28775 (0.0005) [2023-12-26 15:34:30,302][105692] Updated weights for policy 0, policy_version 28631 (0.0008) [2023-12-26 15:34:30,364][105692] Updated weights for policy 0, policy_version 28641 (0.0009) [2023-12-26 15:34:30,432][105692] Updated weights for policy 0, policy_version 28651 (0.0007) [2023-12-26 15:34:30,831][105620] Updated weights for policy 1, policy_version 28785 (0.0006) [2023-12-26 15:34:30,882][105620] Updated weights for policy 1, policy_version 28795 (0.0010) [2023-12-26 15:34:30,939][105620] Updated weights for policy 1, policy_version 28805 (0.0010) [2023-12-26 15:34:30,996][105620] Updated weights for policy 1, policy_version 28815 (0.0010) [2023-12-26 15:34:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.2, 300 sec: 19688.6). Total num frames: 14721024. Throughput: 0: 9649.5, 1: 9687.9. Samples: 14686392. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 15:34:31,063][104569] Avg episode reward: [(0, '8546.898'), (1, '9262.718')] [2023-12-26 15:34:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000028656_7340032.pth... [2023-12-26 15:34:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000028816_7380992.pth... [2023-12-26 15:34:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000027504_7045120.pth [2023-12-26 15:34:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000027664_7086080.pth [2023-12-26 15:34:31,191][105692] Updated weights for policy 0, policy_version 28661 (0.0006) [2023-12-26 15:34:31,251][105692] Updated weights for policy 0, policy_version 28671 (0.0009) [2023-12-26 15:34:31,311][105692] Updated weights for policy 0, policy_version 28681 (0.0009) [2023-12-26 15:34:31,680][105620] Updated weights for policy 1, policy_version 28825 (0.0010) [2023-12-26 15:34:31,747][105620] Updated weights for policy 1, policy_version 28835 (0.0009) [2023-12-26 15:34:31,808][105620] Updated weights for policy 1, policy_version 28845 (0.0010) [2023-12-26 15:34:32,069][105692] Updated weights for policy 0, policy_version 28691 (0.0009) [2023-12-26 15:34:32,131][105692] Updated weights for policy 0, policy_version 28701 (0.0010) [2023-12-26 15:34:32,193][105692] Updated weights for policy 0, policy_version 28711 (0.0009) [2023-12-26 15:34:32,498][105620] Updated weights for policy 1, policy_version 28855 (0.0010) [2023-12-26 15:34:32,557][105620] Updated weights for policy 1, policy_version 28865 (0.0011) [2023-12-26 15:34:32,616][105620] Updated weights for policy 1, policy_version 28875 (0.0011) [2023-12-26 15:34:32,922][105692] Updated weights for policy 0, policy_version 28721 (0.0006) [2023-12-26 15:34:32,988][105692] Updated weights for policy 0, policy_version 28731 (0.0009) [2023-12-26 15:34:33,061][105692] Updated weights for policy 0, policy_version 28741 (0.0010) [2023-12-26 15:34:33,123][105692] Updated weights for policy 0, policy_version 28751 (0.0010) [2023-12-26 15:34:33,242][105620] Updated weights for policy 1, policy_version 28885 (0.0010) [2023-12-26 15:34:33,300][105620] Updated weights for policy 1, policy_version 28895 (0.0010) [2023-12-26 15:34:33,352][105620] Updated weights for policy 1, policy_version 28905 (0.0010) [2023-12-26 15:34:33,812][105692] Updated weights for policy 0, policy_version 28761 (0.0010) [2023-12-26 15:34:33,872][105692] Updated weights for policy 0, policy_version 28771 (0.0010) [2023-12-26 15:34:33,935][105692] Updated weights for policy 0, policy_version 28781 (0.0010) [2023-12-26 15:34:34,091][105620] Updated weights for policy 1, policy_version 28915 (0.0010) [2023-12-26 15:34:34,151][105620] Updated weights for policy 1, policy_version 28925 (0.0010) [2023-12-26 15:34:34,215][105620] Updated weights for policy 1, policy_version 28935 (0.0007) [2023-12-26 15:34:34,710][105692] Updated weights for policy 0, policy_version 28791 (0.0009) [2023-12-26 15:34:34,765][105692] Updated weights for policy 0, policy_version 28801 (0.0006) [2023-12-26 15:34:34,825][105692] Updated weights for policy 0, policy_version 28811 (0.0005) [2023-12-26 15:34:34,899][105620] Updated weights for policy 1, policy_version 28945 (0.0005) [2023-12-26 15:34:34,956][105620] Updated weights for policy 1, policy_version 28955 (0.0007) [2023-12-26 15:34:35,026][105620] Updated weights for policy 1, policy_version 28965 (0.0007) [2023-12-26 15:34:35,078][105620] Updated weights for policy 1, policy_version 28975 (0.0005) [2023-12-26 15:34:35,429][105692] Updated weights for policy 0, policy_version 28821 (0.0008) [2023-12-26 15:34:35,491][105692] Updated weights for policy 0, policy_version 28831 (0.0010) [2023-12-26 15:34:35,559][105692] Updated weights for policy 0, policy_version 28841 (0.0010) [2023-12-26 15:34:35,630][105620] Updated weights for policy 1, policy_version 28985 (0.0005) [2023-12-26 15:34:35,680][105620] Updated weights for policy 1, policy_version 28995 (0.0005) [2023-12-26 15:34:35,730][105620] Updated weights for policy 1, policy_version 29005 (0.0005) [2023-12-26 15:34:36,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.8, 300 sec: 19660.8). Total num frames: 14819328. Throughput: 0: 9645.3, 1: 9865.3. Samples: 14805364. Policy #0 lag: (min: 3.0, avg: 10.0, max: 35.0) [2023-12-26 15:34:36,062][104569] Avg episode reward: [(0, '8455.769'), (1, '9261.836')] [2023-12-26 15:34:36,299][105692] Updated weights for policy 0, policy_version 28851 (0.0010) [2023-12-26 15:34:36,366][105692] Updated weights for policy 0, policy_version 28861 (0.0011) [2023-12-26 15:34:36,379][105620] Updated weights for policy 1, policy_version 29015 (0.0009) [2023-12-26 15:34:36,432][105692] Updated weights for policy 0, policy_version 28871 (0.0010) [2023-12-26 15:34:36,442][105620] Updated weights for policy 1, policy_version 29025 (0.0011) [2023-12-26 15:34:36,507][105620] Updated weights for policy 1, policy_version 29035 (0.0011) [2023-12-26 15:34:37,129][105692] Updated weights for policy 0, policy_version 28881 (0.0010) [2023-12-26 15:34:37,180][105692] Updated weights for policy 0, policy_version 28891 (0.0010) [2023-12-26 15:34:37,230][105692] Updated weights for policy 0, policy_version 28901 (0.0008) [2023-12-26 15:34:37,271][105620] Updated weights for policy 1, policy_version 29045 (0.0010) [2023-12-26 15:34:37,280][105692] Updated weights for policy 0, policy_version 28911 (0.0008) [2023-12-26 15:34:37,321][105620] Updated weights for policy 1, policy_version 29055 (0.0008) [2023-12-26 15:34:37,376][105620] Updated weights for policy 1, policy_version 29065 (0.0008) [2023-12-26 15:34:38,028][105692] Updated weights for policy 0, policy_version 28921 (0.0010) [2023-12-26 15:34:38,082][105692] Updated weights for policy 0, policy_version 28931 (0.0010) [2023-12-26 15:34:38,091][105620] Updated weights for policy 1, policy_version 29075 (0.0008) [2023-12-26 15:34:38,135][105692] Updated weights for policy 0, policy_version 28941 (0.0008) [2023-12-26 15:34:38,141][105620] Updated weights for policy 1, policy_version 29085 (0.0006) [2023-12-26 15:34:38,201][105620] Updated weights for policy 1, policy_version 29095 (0.0009) [2023-12-26 15:34:38,899][105620] Updated weights for policy 1, policy_version 29105 (0.0010) [2023-12-26 15:34:38,953][105620] Updated weights for policy 1, policy_version 29115 (0.0005) [2023-12-26 15:34:39,018][105692] Updated weights for policy 0, policy_version 28951 (0.0006) [2023-12-26 15:34:39,020][105620] Updated weights for policy 1, policy_version 29125 (0.0008) [2023-12-26 15:34:39,071][105692] Updated weights for policy 0, policy_version 28961 (0.0006) [2023-12-26 15:34:39,072][105620] Updated weights for policy 1, policy_version 29135 (0.0008) [2023-12-26 15:34:39,123][105692] Updated weights for policy 0, policy_version 28971 (0.0009) [2023-12-26 15:34:39,734][105620] Updated weights for policy 1, policy_version 29145 (0.0008) [2023-12-26 15:34:39,785][105620] Updated weights for policy 1, policy_version 29155 (0.0005) [2023-12-26 15:34:39,868][105620] Updated weights for policy 1, policy_version 29165 (0.0006) [2023-12-26 15:34:39,976][105692] Updated weights for policy 0, policy_version 28982 (0.0009) [2023-12-26 15:34:40,039][105692] Updated weights for policy 0, policy_version 28992 (0.0009) [2023-12-26 15:34:40,099][105692] Updated weights for policy 0, policy_version 29002 (0.0009) [2023-12-26 15:34:40,581][105620] Updated weights for policy 1, policy_version 29175 (0.0008) [2023-12-26 15:34:40,631][105620] Updated weights for policy 1, policy_version 29185 (0.0009) [2023-12-26 15:34:40,692][105620] Updated weights for policy 1, policy_version 29195 (0.0008) [2023-12-26 15:34:40,880][105692] Updated weights for policy 0, policy_version 29012 (0.0008) [2023-12-26 15:34:40,945][105692] Updated weights for policy 0, policy_version 29022 (0.0008) [2023-12-26 15:34:41,002][105692] Updated weights for policy 0, policy_version 29032 (0.0005) [2023-12-26 15:34:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.8, 300 sec: 19660.8). Total num frames: 14917632. Throughput: 0: 9711.7, 1: 9856.4. Samples: 14921728. Policy #0 lag: (min: 3.0, avg: 10.0, max: 35.0) [2023-12-26 15:34:41,062][104569] Avg episode reward: [(0, '9077.105'), (1, '9079.765')] [2023-12-26 15:34:41,490][105620] Updated weights for policy 1, policy_version 29205 (0.0009) [2023-12-26 15:34:41,549][105620] Updated weights for policy 1, policy_version 29215 (0.0009) [2023-12-26 15:34:41,614][105620] Updated weights for policy 1, policy_version 29225 (0.0009) [2023-12-26 15:34:41,707][105692] Updated weights for policy 0, policy_version 29042 (0.0008) [2023-12-26 15:34:41,776][105692] Updated weights for policy 0, policy_version 29052 (0.0010) [2023-12-26 15:34:41,840][105692] Updated weights for policy 0, policy_version 29062 (0.0009) [2023-12-26 15:34:41,900][105692] Updated weights for policy 0, policy_version 29072 (0.0009) [2023-12-26 15:34:42,392][105620] Updated weights for policy 1, policy_version 29235 (0.0008) [2023-12-26 15:34:42,440][105620] Updated weights for policy 1, policy_version 29245 (0.0009) [2023-12-26 15:34:42,487][105620] Updated weights for policy 1, policy_version 29255 (0.0008) [2023-12-26 15:34:42,627][105692] Updated weights for policy 0, policy_version 29082 (0.0009) [2023-12-26 15:34:42,677][105692] Updated weights for policy 0, policy_version 29092 (0.0009) [2023-12-26 15:34:42,728][105692] Updated weights for policy 0, policy_version 29102 (0.0010) [2023-12-26 15:34:43,339][105620] Updated weights for policy 1, policy_version 29265 (0.0008) [2023-12-26 15:34:43,345][105692] Updated weights for policy 0, policy_version 29112 (0.0009) [2023-12-26 15:34:43,398][105620] Updated weights for policy 1, policy_version 29275 (0.0007) [2023-12-26 15:34:43,404][105692] Updated weights for policy 0, policy_version 29122 (0.0006) [2023-12-26 15:34:43,460][105692] Updated weights for policy 0, policy_version 29132 (0.0006) [2023-12-26 15:34:43,462][105620] Updated weights for policy 1, policy_version 29285 (0.0009) [2023-12-26 15:34:43,523][105620] Updated weights for policy 1, policy_version 29295 (0.0007) [2023-12-26 15:34:44,216][105692] Updated weights for policy 0, policy_version 29142 (0.0008) [2023-12-26 15:34:44,237][105620] Updated weights for policy 1, policy_version 29305 (0.0005) [2023-12-26 15:34:44,269][105692] Updated weights for policy 0, policy_version 29152 (0.0008) [2023-12-26 15:34:44,303][105620] Updated weights for policy 1, policy_version 29315 (0.0007) [2023-12-26 15:34:44,313][105692] Updated weights for policy 0, policy_version 29162 (0.0008) [2023-12-26 15:34:44,371][105620] Updated weights for policy 1, policy_version 29325 (0.0008) [2023-12-26 15:34:45,057][105620] Updated weights for policy 1, policy_version 29335 (0.0008) [2023-12-26 15:34:45,113][105620] Updated weights for policy 1, policy_version 29345 (0.0008) [2023-12-26 15:34:45,119][105692] Updated weights for policy 0, policy_version 29172 (0.0008) [2023-12-26 15:34:45,171][105620] Updated weights for policy 1, policy_version 29355 (0.0006) [2023-12-26 15:34:45,179][105692] Updated weights for policy 0, policy_version 29182 (0.0007) [2023-12-26 15:34:45,247][105692] Updated weights for policy 0, policy_version 29192 (0.0006) [2023-12-26 15:34:45,881][105692] Updated weights for policy 0, policy_version 29202 (0.0008) [2023-12-26 15:34:45,934][105692] Updated weights for policy 0, policy_version 29212 (0.0006) [2023-12-26 15:34:45,957][105620] Updated weights for policy 1, policy_version 29365 (0.0007) [2023-12-26 15:34:45,983][105692] Updated weights for policy 0, policy_version 29222 (0.0009) [2023-12-26 15:34:46,011][105620] Updated weights for policy 1, policy_version 29375 (0.0005) [2023-12-26 15:34:46,036][105692] Updated weights for policy 0, policy_version 29232 (0.0009) [2023-12-26 15:34:46,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19387.6, 300 sec: 19660.8). Total num frames: 15007744. Throughput: 0: 9793.7, 1: 9743.8. Samples: 14978468. Policy #0 lag: (min: 3.0, avg: 10.0, max: 35.0) [2023-12-26 15:34:46,063][104569] Avg episode reward: [(0, '9077.664'), (1, '9171.848')] [2023-12-26 15:34:46,065][105620] Updated weights for policy 1, policy_version 29385 (0.0005) [2023-12-26 15:34:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000029232_7487488.pth... [2023-12-26 15:34:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000028112_7200768.pth [2023-12-26 15:34:46,098][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000029392_7528448.pth... [2023-12-26 15:34:46,101][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000028240_7233536.pth [2023-12-26 15:34:46,660][105620] Updated weights for policy 1, policy_version 29395 (0.0007) [2023-12-26 15:34:46,722][105620] Updated weights for policy 1, policy_version 29405 (0.0010) [2023-12-26 15:34:46,785][105620] Updated weights for policy 1, policy_version 29415 (0.0011) [2023-12-26 15:34:46,806][105692] Updated weights for policy 0, policy_version 29242 (0.0006) [2023-12-26 15:34:46,868][105692] Updated weights for policy 0, policy_version 29252 (0.0006) [2023-12-26 15:34:46,933][105692] Updated weights for policy 0, policy_version 29262 (0.0005) [2023-12-26 15:34:47,459][105620] Updated weights for policy 1, policy_version 29425 (0.0010) [2023-12-26 15:34:47,461][105692] Updated weights for policy 0, policy_version 29272 (0.0007) [2023-12-26 15:34:47,512][105692] Updated weights for policy 0, policy_version 29282 (0.0006) [2023-12-26 15:34:47,515][105620] Updated weights for policy 1, policy_version 29435 (0.0008) [2023-12-26 15:34:47,569][105692] Updated weights for policy 0, policy_version 29292 (0.0009) [2023-12-26 15:34:47,582][105620] Updated weights for policy 1, policy_version 29445 (0.0005) [2023-12-26 15:34:47,643][105620] Updated weights for policy 1, policy_version 29455 (0.0005) [2023-12-26 15:34:48,265][105692] Updated weights for policy 0, policy_version 29302 (0.0009) [2023-12-26 15:34:48,322][105692] Updated weights for policy 0, policy_version 29312 (0.0007) [2023-12-26 15:34:48,327][105620] Updated weights for policy 1, policy_version 29465 (0.0008) [2023-12-26 15:34:48,379][105692] Updated weights for policy 0, policy_version 29322 (0.0009) [2023-12-26 15:34:48,385][105620] Updated weights for policy 1, policy_version 29475 (0.0008) [2023-12-26 15:34:48,429][105620] Updated weights for policy 1, policy_version 29485 (0.0008) [2023-12-26 15:34:49,088][105692] Updated weights for policy 0, policy_version 29332 (0.0007) [2023-12-26 15:34:49,143][105692] Updated weights for policy 0, policy_version 29342 (0.0006) [2023-12-26 15:34:49,199][105692] Updated weights for policy 0, policy_version 29352 (0.0006) [2023-12-26 15:34:49,265][105620] Updated weights for policy 1, policy_version 29495 (0.0007) [2023-12-26 15:34:49,328][105620] Updated weights for policy 1, policy_version 29505 (0.0008) [2023-12-26 15:34:49,393][105620] Updated weights for policy 1, policy_version 29515 (0.0008) [2023-12-26 15:34:49,889][105692] Updated weights for policy 0, policy_version 29362 (0.0008) [2023-12-26 15:34:49,953][105692] Updated weights for policy 0, policy_version 29372 (0.0007) [2023-12-26 15:34:50,016][105692] Updated weights for policy 0, policy_version 29382 (0.0008) [2023-12-26 15:34:50,063][105692] Updated weights for policy 0, policy_version 29392 (0.0008) [2023-12-26 15:34:50,143][105620] Updated weights for policy 1, policy_version 29525 (0.0007) [2023-12-26 15:34:50,196][105620] Updated weights for policy 1, policy_version 29535 (0.0007) [2023-12-26 15:34:50,249][105620] Updated weights for policy 1, policy_version 29545 (0.0010) [2023-12-26 15:34:50,720][105692] Updated weights for policy 0, policy_version 29402 (0.0009) [2023-12-26 15:34:50,781][105692] Updated weights for policy 0, policy_version 29412 (0.0008) [2023-12-26 15:34:50,849][105692] Updated weights for policy 0, policy_version 29422 (0.0006) [2023-12-26 15:34:51,030][105620] Updated weights for policy 1, policy_version 29555 (0.0009) [2023-12-26 15:34:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 15106048. Throughput: 0: 9861.2, 1: 9763.4. Samples: 15097260. Policy #0 lag: (min: 3.0, avg: 10.0, max: 35.0) [2023-12-26 15:34:51,063][104569] Avg episode reward: [(0, '9260.676'), (1, '9172.457')] [2023-12-26 15:34:51,092][105620] Updated weights for policy 1, policy_version 29565 (0.0008) [2023-12-26 15:34:51,154][105620] Updated weights for policy 1, policy_version 29575 (0.0009) [2023-12-26 15:34:51,579][105692] Updated weights for policy 0, policy_version 29432 (0.0009) [2023-12-26 15:34:51,636][105692] Updated weights for policy 0, policy_version 29442 (0.0008) [2023-12-26 15:34:51,694][105692] Updated weights for policy 0, policy_version 29452 (0.0008) [2023-12-26 15:34:51,931][105620] Updated weights for policy 1, policy_version 29585 (0.0008) [2023-12-26 15:34:51,986][105620] Updated weights for policy 1, policy_version 29595 (0.0008) [2023-12-26 15:34:52,043][105620] Updated weights for policy 1, policy_version 29605 (0.0008) [2023-12-26 15:34:52,101][105620] Updated weights for policy 1, policy_version 29615 (0.0009) [2023-12-26 15:34:52,454][105692] Updated weights for policy 0, policy_version 29462 (0.0010) [2023-12-26 15:34:52,517][105692] Updated weights for policy 0, policy_version 29472 (0.0011) [2023-12-26 15:34:52,573][105692] Updated weights for policy 0, policy_version 29482 (0.0010) [2023-12-26 15:34:52,880][105620] Updated weights for policy 1, policy_version 29625 (0.0009) [2023-12-26 15:34:52,943][105620] Updated weights for policy 1, policy_version 29635 (0.0008) [2023-12-26 15:34:53,010][105620] Updated weights for policy 1, policy_version 29645 (0.0008) [2023-12-26 15:34:53,339][105692] Updated weights for policy 0, policy_version 29492 (0.0010) [2023-12-26 15:34:53,394][105692] Updated weights for policy 0, policy_version 29502 (0.0010) [2023-12-26 15:34:53,450][105692] Updated weights for policy 0, policy_version 29512 (0.0010) [2023-12-26 15:34:53,731][105620] Updated weights for policy 1, policy_version 29655 (0.0009) [2023-12-26 15:34:53,793][105620] Updated weights for policy 1, policy_version 29665 (0.0008) [2023-12-26 15:34:53,856][105620] Updated weights for policy 1, policy_version 29675 (0.0006) [2023-12-26 15:34:54,180][105692] Updated weights for policy 0, policy_version 29522 (0.0010) [2023-12-26 15:34:54,242][105692] Updated weights for policy 0, policy_version 29532 (0.0009) [2023-12-26 15:34:54,304][105692] Updated weights for policy 0, policy_version 29542 (0.0009) [2023-12-26 15:34:54,366][105692] Updated weights for policy 0, policy_version 29552 (0.0009) [2023-12-26 15:34:54,575][105620] Updated weights for policy 1, policy_version 29685 (0.0008) [2023-12-26 15:34:54,621][105620] Updated weights for policy 1, policy_version 29695 (0.0008) [2023-12-26 15:34:54,678][105620] Updated weights for policy 1, policy_version 29705 (0.0009) [2023-12-26 15:34:55,077][105692] Updated weights for policy 0, policy_version 29562 (0.0005) [2023-12-26 15:34:55,139][105692] Updated weights for policy 0, policy_version 29572 (0.0005) [2023-12-26 15:34:55,199][105692] Updated weights for policy 0, policy_version 29582 (0.0006) [2023-12-26 15:34:55,394][105620] Updated weights for policy 1, policy_version 29715 (0.0008) [2023-12-26 15:34:55,459][105620] Updated weights for policy 1, policy_version 29725 (0.0007) [2023-12-26 15:34:55,523][105620] Updated weights for policy 1, policy_version 29735 (0.0008) [2023-12-26 15:34:55,840][105692] Updated weights for policy 0, policy_version 29592 (0.0007) [2023-12-26 15:34:55,894][105692] Updated weights for policy 0, policy_version 29602 (0.0005) [2023-12-26 15:34:55,945][105692] Updated weights for policy 0, policy_version 29612 (0.0006) [2023-12-26 15:34:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.2, 300 sec: 19660.8). Total num frames: 15204352. Throughput: 0: 9753.8, 1: 9601.5. Samples: 15211048. Policy #0 lag: (min: 31.0, avg: 43.3, max: 63.0) [2023-12-26 15:34:56,063][104569] Avg episode reward: [(0, '9349.785'), (1, '9171.472')] [2023-12-26 15:34:56,139][105620] Updated weights for policy 1, policy_version 29745 (0.0006) [2023-12-26 15:34:56,186][105620] Updated weights for policy 1, policy_version 29755 (0.0005) [2023-12-26 15:34:56,232][105620] Updated weights for policy 1, policy_version 29765 (0.0005) [2023-12-26 15:34:56,283][105620] Updated weights for policy 1, policy_version 29775 (0.0005) [2023-12-26 15:34:56,740][105692] Updated weights for policy 0, policy_version 29622 (0.0007) [2023-12-26 15:34:56,797][105692] Updated weights for policy 0, policy_version 29632 (0.0006) [2023-12-26 15:34:56,861][105692] Updated weights for policy 0, policy_version 29642 (0.0009) [2023-12-26 15:34:56,953][105620] Updated weights for policy 1, policy_version 29785 (0.0008) [2023-12-26 15:34:56,999][105620] Updated weights for policy 1, policy_version 29795 (0.0008) [2023-12-26 15:34:57,051][105620] Updated weights for policy 1, policy_version 29805 (0.0010) [2023-12-26 15:34:57,525][105692] Updated weights for policy 0, policy_version 29652 (0.0009) [2023-12-26 15:34:57,580][105692] Updated weights for policy 0, policy_version 29662 (0.0010) [2023-12-26 15:34:57,637][105692] Updated weights for policy 0, policy_version 29672 (0.0010) [2023-12-26 15:34:57,763][105620] Updated weights for policy 1, policy_version 29815 (0.0007) [2023-12-26 15:34:57,823][105620] Updated weights for policy 1, policy_version 29825 (0.0009) [2023-12-26 15:34:57,876][105620] Updated weights for policy 1, policy_version 29835 (0.0010) [2023-12-26 15:34:58,338][105692] Updated weights for policy 0, policy_version 29682 (0.0009) [2023-12-26 15:34:58,405][105692] Updated weights for policy 0, policy_version 29692 (0.0009) [2023-12-26 15:34:58,462][105692] Updated weights for policy 0, policy_version 29702 (0.0008) [2023-12-26 15:34:58,527][105692] Updated weights for policy 0, policy_version 29712 (0.0008) [2023-12-26 15:34:58,581][105620] Updated weights for policy 1, policy_version 29845 (0.0010) [2023-12-26 15:34:58,646][105620] Updated weights for policy 1, policy_version 29855 (0.0010) [2023-12-26 15:34:58,714][105620] Updated weights for policy 1, policy_version 29865 (0.0009) [2023-12-26 15:34:59,355][105692] Updated weights for policy 0, policy_version 29722 (0.0007) [2023-12-26 15:34:59,413][105692] Updated weights for policy 0, policy_version 29732 (0.0006) [2023-12-26 15:34:59,476][105692] Updated weights for policy 0, policy_version 29742 (0.0006) [2023-12-26 15:34:59,508][105620] Updated weights for policy 1, policy_version 29875 (0.0009) [2023-12-26 15:34:59,572][105620] Updated weights for policy 1, policy_version 29885 (0.0009) [2023-12-26 15:34:59,642][105620] Updated weights for policy 1, policy_version 29895 (0.0009) [2023-12-26 15:35:00,105][105692] Updated weights for policy 0, policy_version 29752 (0.0008) [2023-12-26 15:35:00,169][105692] Updated weights for policy 0, policy_version 29762 (0.0008) [2023-12-26 15:35:00,227][105692] Updated weights for policy 0, policy_version 29772 (0.0005) [2023-12-26 15:35:00,450][105620] Updated weights for policy 1, policy_version 29905 (0.0009) [2023-12-26 15:35:00,508][105620] Updated weights for policy 1, policy_version 29915 (0.0009) [2023-12-26 15:35:00,566][105620] Updated weights for policy 1, policy_version 29925 (0.0010) [2023-12-26 15:35:00,620][105620] Updated weights for policy 1, policy_version 29935 (0.0009) [2023-12-26 15:35:00,820][105692] Updated weights for policy 0, policy_version 29782 (0.0006) [2023-12-26 15:35:00,870][105692] Updated weights for policy 0, policy_version 29792 (0.0005) [2023-12-26 15:35:00,928][105692] Updated weights for policy 0, policy_version 29802 (0.0006) [2023-12-26 15:35:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 15302656. Throughput: 0: 9704.0, 1: 9709.9. Samples: 15270032. Policy #0 lag: (min: 31.0, avg: 43.3, max: 63.0) [2023-12-26 15:35:01,062][104569] Avg episode reward: [(0, '9348.799'), (1, '9166.446')] [2023-12-26 15:35:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000029808_7634944.pth... [2023-12-26 15:35:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000029936_7667712.pth... [2023-12-26 15:35:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000028656_7340032.pth [2023-12-26 15:35:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000028816_7380992.pth [2023-12-26 15:35:01,465][105620] Updated weights for policy 1, policy_version 29945 (0.0009) [2023-12-26 15:35:01,530][105620] Updated weights for policy 1, policy_version 29955 (0.0009) [2023-12-26 15:35:01,566][105692] Updated weights for policy 0, policy_version 29812 (0.0006) [2023-12-26 15:35:01,596][105620] Updated weights for policy 1, policy_version 29965 (0.0007) [2023-12-26 15:35:01,621][105692] Updated weights for policy 0, policy_version 29822 (0.0007) [2023-12-26 15:35:01,690][105692] Updated weights for policy 0, policy_version 29832 (0.0009) [2023-12-26 15:35:02,316][105620] Updated weights for policy 1, policy_version 29975 (0.0006) [2023-12-26 15:35:02,385][105620] Updated weights for policy 1, policy_version 29985 (0.0009) [2023-12-26 15:35:02,434][105692] Updated weights for policy 0, policy_version 29842 (0.0009) [2023-12-26 15:35:02,447][105620] Updated weights for policy 1, policy_version 29995 (0.0009) [2023-12-26 15:35:02,490][105692] Updated weights for policy 0, policy_version 29852 (0.0007) [2023-12-26 15:35:02,544][105692] Updated weights for policy 0, policy_version 29862 (0.0010) [2023-12-26 15:35:02,598][105692] Updated weights for policy 0, policy_version 29872 (0.0010) [2023-12-26 15:35:03,020][105620] Updated weights for policy 1, policy_version 30005 (0.0008) [2023-12-26 15:35:03,072][105620] Updated weights for policy 1, policy_version 30015 (0.0005) [2023-12-26 15:35:03,117][105620] Updated weights for policy 1, policy_version 30025 (0.0005) [2023-12-26 15:35:03,421][105692] Updated weights for policy 0, policy_version 29882 (0.0006) [2023-12-26 15:35:03,478][105692] Updated weights for policy 0, policy_version 29892 (0.0010) [2023-12-26 15:35:03,525][105692] Updated weights for policy 0, policy_version 29902 (0.0010) [2023-12-26 15:35:03,793][105620] Updated weights for policy 1, policy_version 30035 (0.0005) [2023-12-26 15:35:03,856][105620] Updated weights for policy 1, policy_version 30045 (0.0006) [2023-12-26 15:35:03,922][105620] Updated weights for policy 1, policy_version 30055 (0.0008) [2023-12-26 15:35:04,260][105692] Updated weights for policy 0, policy_version 29912 (0.0009) [2023-12-26 15:35:04,320][105692] Updated weights for policy 0, policy_version 29922 (0.0009) [2023-12-26 15:35:04,375][105692] Updated weights for policy 0, policy_version 29932 (0.0010) [2023-12-26 15:35:04,511][105620] Updated weights for policy 1, policy_version 30065 (0.0006) [2023-12-26 15:35:04,567][105620] Updated weights for policy 1, policy_version 30075 (0.0005) [2023-12-26 15:35:04,644][105620] Updated weights for policy 1, policy_version 30085 (0.0009) [2023-12-26 15:35:04,700][105620] Updated weights for policy 1, policy_version 30095 (0.0010) [2023-12-26 15:35:05,177][105692] Updated weights for policy 0, policy_version 29942 (0.0007) [2023-12-26 15:35:05,237][105692] Updated weights for policy 0, policy_version 29952 (0.0005) [2023-12-26 15:35:05,286][105692] Updated weights for policy 0, policy_version 29962 (0.0005) [2023-12-26 15:35:05,475][105620] Updated weights for policy 1, policy_version 30105 (0.0008) [2023-12-26 15:35:05,537][105620] Updated weights for policy 1, policy_version 30115 (0.0006) [2023-12-26 15:35:05,600][105620] Updated weights for policy 1, policy_version 30125 (0.0005) [2023-12-26 15:35:05,988][105692] Updated weights for policy 0, policy_version 29972 (0.0007) [2023-12-26 15:35:06,050][105692] Updated weights for policy 0, policy_version 29982 (0.0010) [2023-12-26 15:35:06,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 15392768. Throughput: 0: 9641.2, 1: 9785.9. Samples: 15386528. Policy #0 lag: (min: 31.0, avg: 43.3, max: 63.0) [2023-12-26 15:35:06,062][104569] Avg episode reward: [(0, '9348.377'), (1, '9259.131')] [2023-12-26 15:35:06,118][105692] Updated weights for policy 0, policy_version 29992 (0.0008) [2023-12-26 15:35:06,128][105620] Updated weights for policy 1, policy_version 30135 (0.0008) [2023-12-26 15:35:06,188][105620] Updated weights for policy 1, policy_version 30145 (0.0008) [2023-12-26 15:35:06,256][105620] Updated weights for policy 1, policy_version 30155 (0.0005) [2023-12-26 15:35:06,833][105620] Updated weights for policy 1, policy_version 30165 (0.0005) [2023-12-26 15:35:06,898][105620] Updated weights for policy 1, policy_version 30175 (0.0007) [2023-12-26 15:35:06,965][105620] Updated weights for policy 1, policy_version 30185 (0.0005) [2023-12-26 15:35:06,973][105692] Updated weights for policy 0, policy_version 30002 (0.0007) [2023-12-26 15:35:07,026][105692] Updated weights for policy 0, policy_version 30012 (0.0009) [2023-12-26 15:35:07,092][105692] Updated weights for policy 0, policy_version 30022 (0.0009) [2023-12-26 15:35:07,147][105692] Updated weights for policy 0, policy_version 30032 (0.0008) [2023-12-26 15:35:07,494][105620] Updated weights for policy 1, policy_version 30195 (0.0006) [2023-12-26 15:35:07,556][105620] Updated weights for policy 1, policy_version 30205 (0.0010) [2023-12-26 15:35:07,613][105620] Updated weights for policy 1, policy_version 30215 (0.0010) [2023-12-26 15:35:07,975][105692] Updated weights for policy 0, policy_version 30042 (0.0010) [2023-12-26 15:35:08,031][105692] Updated weights for policy 0, policy_version 30053 (0.0007) [2023-12-26 15:35:08,086][105692] Updated weights for policy 0, policy_version 30063 (0.0008) [2023-12-26 15:35:08,281][105620] Updated weights for policy 1, policy_version 30225 (0.0010) [2023-12-26 15:35:08,343][105620] Updated weights for policy 1, policy_version 30235 (0.0007) [2023-12-26 15:35:08,405][105620] Updated weights for policy 1, policy_version 30245 (0.0010) [2023-12-26 15:35:08,465][105620] Updated weights for policy 1, policy_version 30255 (0.0010) [2023-12-26 15:35:08,874][105692] Updated weights for policy 0, policy_version 30073 (0.0008) [2023-12-26 15:35:08,934][105692] Updated weights for policy 0, policy_version 30083 (0.0008) [2023-12-26 15:35:08,996][105692] Updated weights for policy 0, policy_version 30093 (0.0008) [2023-12-26 15:35:09,184][105620] Updated weights for policy 1, policy_version 30265 (0.0006) [2023-12-26 15:35:09,250][105620] Updated weights for policy 1, policy_version 30275 (0.0007) [2023-12-26 15:35:09,320][105620] Updated weights for policy 1, policy_version 30285 (0.0006) [2023-12-26 15:35:09,849][105692] Updated weights for policy 0, policy_version 30103 (0.0009) [2023-12-26 15:35:09,917][105692] Updated weights for policy 0, policy_version 30113 (0.0007) [2023-12-26 15:35:09,951][105620] Updated weights for policy 1, policy_version 30295 (0.0008) [2023-12-26 15:35:09,984][105692] Updated weights for policy 0, policy_version 30123 (0.0008) [2023-12-26 15:35:10,013][105620] Updated weights for policy 1, policy_version 30305 (0.0007) [2023-12-26 15:35:10,076][105620] Updated weights for policy 1, policy_version 30315 (0.0006) [2023-12-26 15:35:10,663][105620] Updated weights for policy 1, policy_version 30325 (0.0005) [2023-12-26 15:35:10,714][105620] Updated weights for policy 1, policy_version 30335 (0.0005) [2023-12-26 15:35:10,778][105620] Updated weights for policy 1, policy_version 30345 (0.0005) [2023-12-26 15:35:10,831][105692] Updated weights for policy 0, policy_version 30133 (0.0009) [2023-12-26 15:35:10,891][105692] Updated weights for policy 0, policy_version 30143 (0.0008) [2023-12-26 15:35:10,947][105692] Updated weights for policy 0, policy_version 30153 (0.0009) [2023-12-26 15:35:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 15499264. Throughput: 0: 9483.3, 1: 9968.5. Samples: 15503972. Policy #0 lag: (min: 31.0, avg: 43.3, max: 63.0) [2023-12-26 15:35:11,062][104569] Avg episode reward: [(0, '9348.152'), (1, '9259.952')] [2023-12-26 15:35:11,421][105620] Updated weights for policy 1, policy_version 30355 (0.0006) [2023-12-26 15:35:11,484][105620] Updated weights for policy 1, policy_version 30365 (0.0008) [2023-12-26 15:35:11,543][105620] Updated weights for policy 1, policy_version 30375 (0.0008) [2023-12-26 15:35:11,758][105692] Updated weights for policy 0, policy_version 30163 (0.0009) [2023-12-26 15:35:11,813][105692] Updated weights for policy 0, policy_version 30173 (0.0008) [2023-12-26 15:35:11,876][105692] Updated weights for policy 0, policy_version 30183 (0.0008) [2023-12-26 15:35:12,294][105620] Updated weights for policy 1, policy_version 30385 (0.0008) [2023-12-26 15:35:12,353][105620] Updated weights for policy 1, policy_version 30395 (0.0008) [2023-12-26 15:35:12,410][105620] Updated weights for policy 1, policy_version 30405 (0.0008) [2023-12-26 15:35:12,475][105620] Updated weights for policy 1, policy_version 30415 (0.0008) [2023-12-26 15:35:12,588][105692] Updated weights for policy 0, policy_version 30193 (0.0008) [2023-12-26 15:35:12,652][105692] Updated weights for policy 0, policy_version 30203 (0.0006) [2023-12-26 15:35:12,713][105692] Updated weights for policy 0, policy_version 30213 (0.0005) [2023-12-26 15:35:12,772][105692] Updated weights for policy 0, policy_version 30223 (0.0005) [2023-12-26 15:35:13,281][105620] Updated weights for policy 1, policy_version 30425 (0.0008) [2023-12-26 15:35:13,337][105620] Updated weights for policy 1, policy_version 30435 (0.0010) [2023-12-26 15:35:13,360][105692] Updated weights for policy 0, policy_version 30233 (0.0007) [2023-12-26 15:35:13,392][105620] Updated weights for policy 1, policy_version 30445 (0.0010) [2023-12-26 15:35:13,418][105692] Updated weights for policy 0, policy_version 30243 (0.0007) [2023-12-26 15:35:13,476][105692] Updated weights for policy 0, policy_version 30253 (0.0008) [2023-12-26 15:35:14,087][105620] Updated weights for policy 1, policy_version 30455 (0.0009) [2023-12-26 15:35:14,135][105692] Updated weights for policy 0, policy_version 30263 (0.0006) [2023-12-26 15:35:14,148][105620] Updated weights for policy 1, policy_version 30465 (0.0010) [2023-12-26 15:35:14,186][105692] Updated weights for policy 0, policy_version 30273 (0.0007) [2023-12-26 15:35:14,214][105620] Updated weights for policy 1, policy_version 30475 (0.0010) [2023-12-26 15:35:14,243][105692] Updated weights for policy 0, policy_version 30283 (0.0007) [2023-12-26 15:35:14,800][105692] Updated weights for policy 0, policy_version 30293 (0.0007) [2023-12-26 15:35:14,864][105692] Updated weights for policy 0, policy_version 30303 (0.0007) [2023-12-26 15:35:14,922][105692] Updated weights for policy 0, policy_version 30313 (0.0008) [2023-12-26 15:35:14,923][105620] Updated weights for policy 1, policy_version 30485 (0.0009) [2023-12-26 15:35:14,972][105620] Updated weights for policy 1, policy_version 30495 (0.0008) [2023-12-26 15:35:15,018][105620] Updated weights for policy 1, policy_version 30505 (0.0008) [2023-12-26 15:35:15,614][105692] Updated weights for policy 0, policy_version 30323 (0.0009) [2023-12-26 15:35:15,678][105692] Updated weights for policy 0, policy_version 30333 (0.0005) [2023-12-26 15:35:15,730][105692] Updated weights for policy 0, policy_version 30343 (0.0009) [2023-12-26 15:35:15,844][105620] Updated weights for policy 1, policy_version 30515 (0.0008) [2023-12-26 15:35:15,907][105620] Updated weights for policy 1, policy_version 30525 (0.0008) [2023-12-26 15:35:15,955][105620] Updated weights for policy 1, policy_version 30535 (0.0008) [2023-12-26 15:35:16,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19524.2, 300 sec: 19660.8). Total num frames: 15597568. Throughput: 0: 9488.8, 1: 9936.4. Samples: 15560524. Policy #0 lag: (min: 10.0, avg: 19.4, max: 42.0) [2023-12-26 15:35:16,063][104569] Avg episode reward: [(0, '9348.542'), (1, '9259.508')] [2023-12-26 15:35:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000030352_7774208.pth... [2023-12-26 15:35:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000030544_7823360.pth... [2023-12-26 15:35:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000029392_7528448.pth [2023-12-26 15:35:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000029232_7487488.pth [2023-12-26 15:35:16,410][105692] Updated weights for policy 0, policy_version 30353 (0.0010) [2023-12-26 15:35:16,476][105692] Updated weights for policy 0, policy_version 30363 (0.0006) [2023-12-26 15:35:16,538][105692] Updated weights for policy 0, policy_version 30373 (0.0005) [2023-12-26 15:35:16,604][105692] Updated weights for policy 0, policy_version 30383 (0.0005) [2023-12-26 15:35:16,744][105620] Updated weights for policy 1, policy_version 30545 (0.0007) [2023-12-26 15:35:16,800][105620] Updated weights for policy 1, policy_version 30555 (0.0008) [2023-12-26 15:35:16,860][105620] Updated weights for policy 1, policy_version 30565 (0.0007) [2023-12-26 15:35:16,909][105620] Updated weights for policy 1, policy_version 30575 (0.0008) [2023-12-26 15:35:17,255][105692] Updated weights for policy 0, policy_version 30393 (0.0009) [2023-12-26 15:35:17,326][105692] Updated weights for policy 0, policy_version 30403 (0.0009) [2023-12-26 15:35:17,395][105692] Updated weights for policy 0, policy_version 30413 (0.0008) [2023-12-26 15:35:17,589][105620] Updated weights for policy 1, policy_version 30585 (0.0010) [2023-12-26 15:35:17,648][105620] Updated weights for policy 1, policy_version 30595 (0.0010) [2023-12-26 15:35:17,693][105620] Updated weights for policy 1, policy_version 30605 (0.0010) [2023-12-26 15:35:17,963][105692] Updated weights for policy 0, policy_version 30423 (0.0007) [2023-12-26 15:35:18,014][105692] Updated weights for policy 0, policy_version 30433 (0.0007) [2023-12-26 15:35:18,081][105692] Updated weights for policy 0, policy_version 30444 (0.0006) [2023-12-26 15:35:18,370][105620] Updated weights for policy 1, policy_version 30615 (0.0009) [2023-12-26 15:35:18,425][105620] Updated weights for policy 1, policy_version 30625 (0.0007) [2023-12-26 15:35:18,481][105620] Updated weights for policy 1, policy_version 30635 (0.0010) [2023-12-26 15:35:18,658][105692] Updated weights for policy 0, policy_version 30454 (0.0005) [2023-12-26 15:35:18,715][105692] Updated weights for policy 0, policy_version 30464 (0.0005) [2023-12-26 15:35:18,785][105692] Updated weights for policy 0, policy_version 30474 (0.0008) [2023-12-26 15:35:19,164][105620] Updated weights for policy 1, policy_version 30645 (0.0010) [2023-12-26 15:35:19,229][105620] Updated weights for policy 1, policy_version 30655 (0.0010) [2023-12-26 15:35:19,294][105620] Updated weights for policy 1, policy_version 30665 (0.0010) [2023-12-26 15:35:19,505][105692] Updated weights for policy 0, policy_version 30484 (0.0009) [2023-12-26 15:35:19,561][105692] Updated weights for policy 0, policy_version 30494 (0.0006) [2023-12-26 15:35:19,626][105692] Updated weights for policy 0, policy_version 30504 (0.0006) [2023-12-26 15:35:20,071][105620] Updated weights for policy 1, policy_version 30675 (0.0011) [2023-12-26 15:35:20,137][105620] Updated weights for policy 1, policy_version 30685 (0.0010) [2023-12-26 15:35:20,195][105620] Updated weights for policy 1, policy_version 30695 (0.0010) [2023-12-26 15:35:20,280][105692] Updated weights for policy 0, policy_version 30514 (0.0007) [2023-12-26 15:35:20,340][105692] Updated weights for policy 0, policy_version 30524 (0.0010) [2023-12-26 15:35:20,406][105692] Updated weights for policy 0, policy_version 30534 (0.0008) [2023-12-26 15:35:20,465][105692] Updated weights for policy 0, policy_version 30544 (0.0010) [2023-12-26 15:35:20,967][105620] Updated weights for policy 1, policy_version 30705 (0.0010) [2023-12-26 15:35:21,019][105620] Updated weights for policy 1, policy_version 30715 (0.0010) [2023-12-26 15:35:21,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 15687680. Throughput: 0: 9687.6, 1: 9814.9. Samples: 15682980. Policy #0 lag: (min: 10.0, avg: 19.4, max: 42.0) [2023-12-26 15:35:21,063][104569] Avg episode reward: [(0, '9348.275'), (1, '9350.794')] [2023-12-26 15:35:21,097][105620] Updated weights for policy 1, policy_version 30725 (0.0012) [2023-12-26 15:35:21,187][105620] Updated weights for policy 1, policy_version 30735 (0.0011) [2023-12-26 15:35:21,192][105586] Saving new best policy, reward=9350.794! [2023-12-26 15:35:21,211][105692] Updated weights for policy 0, policy_version 30554 (0.0011) [2023-12-26 15:35:21,273][105692] Updated weights for policy 0, policy_version 30564 (0.0011) [2023-12-26 15:35:21,327][105692] Updated weights for policy 0, policy_version 30574 (0.0010) [2023-12-26 15:35:21,920][105620] Updated weights for policy 1, policy_version 30745 (0.0011) [2023-12-26 15:35:21,982][105620] Updated weights for policy 1, policy_version 30755 (0.0011) [2023-12-26 15:35:22,034][105620] Updated weights for policy 1, policy_version 30765 (0.0011) [2023-12-26 15:35:22,102][105692] Updated weights for policy 0, policy_version 30584 (0.0011) [2023-12-26 15:35:22,155][105692] Updated weights for policy 0, policy_version 30594 (0.0011) [2023-12-26 15:35:22,200][105692] Updated weights for policy 0, policy_version 30604 (0.0010) [2023-12-26 15:35:22,773][105620] Updated weights for policy 1, policy_version 30775 (0.0010) [2023-12-26 15:35:22,835][105620] Updated weights for policy 1, policy_version 30785 (0.0011) [2023-12-26 15:35:22,894][105620] Updated weights for policy 1, policy_version 30795 (0.0011) [2023-12-26 15:35:22,986][105692] Updated weights for policy 0, policy_version 30614 (0.0007) [2023-12-26 15:35:23,044][105692] Updated weights for policy 0, policy_version 30624 (0.0011) [2023-12-26 15:35:23,104][105692] Updated weights for policy 0, policy_version 30634 (0.0011) [2023-12-26 15:35:23,646][105620] Updated weights for policy 1, policy_version 30805 (0.0010) [2023-12-26 15:35:23,673][105692] Updated weights for policy 0, policy_version 30644 (0.0007) [2023-12-26 15:35:23,715][105620] Updated weights for policy 1, policy_version 30815 (0.0011) [2023-12-26 15:35:23,742][105692] Updated weights for policy 0, policy_version 30654 (0.0005) [2023-12-26 15:35:23,783][105620] Updated weights for policy 1, policy_version 30825 (0.0010) [2023-12-26 15:35:23,800][105692] Updated weights for policy 0, policy_version 30664 (0.0010) [2023-12-26 15:35:24,423][105692] Updated weights for policy 0, policy_version 30674 (0.0008) [2023-12-26 15:35:24,492][105692] Updated weights for policy 0, policy_version 30684 (0.0011) [2023-12-26 15:35:24,498][105620] Updated weights for policy 1, policy_version 30835 (0.0011) [2023-12-26 15:35:24,547][105692] Updated weights for policy 0, policy_version 30694 (0.0011) [2023-12-26 15:35:24,556][105620] Updated weights for policy 1, policy_version 30845 (0.0010) [2023-12-26 15:35:24,606][105692] Updated weights for policy 0, policy_version 30704 (0.0011) [2023-12-26 15:35:24,614][105620] Updated weights for policy 1, policy_version 30855 (0.0010) [2023-12-26 15:35:25,280][105620] Updated weights for policy 1, policy_version 30865 (0.0010) [2023-12-26 15:35:25,282][105692] Updated weights for policy 0, policy_version 30714 (0.0005) [2023-12-26 15:35:25,329][105692] Updated weights for policy 0, policy_version 30724 (0.0005) [2023-12-26 15:35:25,347][105620] Updated weights for policy 1, policy_version 30875 (0.0006) [2023-12-26 15:35:25,383][105692] Updated weights for policy 0, policy_version 30734 (0.0005) [2023-12-26 15:35:25,405][105620] Updated weights for policy 1, policy_version 30885 (0.0007) [2023-12-26 15:35:25,456][105620] Updated weights for policy 1, policy_version 30895 (0.0006) [2023-12-26 15:35:26,052][105692] Updated weights for policy 0, policy_version 30744 (0.0010) [2023-12-26 15:35:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 15785984. Throughput: 0: 9790.8, 1: 9746.6. Samples: 15800908. Policy #0 lag: (min: 10.0, avg: 19.4, max: 42.0) [2023-12-26 15:35:26,062][104569] Avg episode reward: [(0, '9348.248'), (1, '9164.758')] [2023-12-26 15:35:26,104][105692] Updated weights for policy 0, policy_version 30754 (0.0010) [2023-12-26 15:35:26,107][105620] Updated weights for policy 1, policy_version 30905 (0.0010) [2023-12-26 15:35:26,149][105692] Updated weights for policy 0, policy_version 30764 (0.0010) [2023-12-26 15:35:26,162][105620] Updated weights for policy 1, policy_version 30915 (0.0010) [2023-12-26 15:35:26,210][105620] Updated weights for policy 1, policy_version 30925 (0.0010) [2023-12-26 15:35:26,797][105692] Updated weights for policy 0, policy_version 30774 (0.0007) [2023-12-26 15:35:26,854][105692] Updated weights for policy 0, policy_version 30784 (0.0005) [2023-12-26 15:35:26,904][105692] Updated weights for policy 0, policy_version 30794 (0.0005) [2023-12-26 15:35:26,957][105620] Updated weights for policy 1, policy_version 30935 (0.0010) [2023-12-26 15:35:27,005][105620] Updated weights for policy 1, policy_version 30945 (0.0010) [2023-12-26 15:35:27,056][105620] Updated weights for policy 1, policy_version 30955 (0.0010) [2023-12-26 15:35:27,489][105692] Updated weights for policy 0, policy_version 30804 (0.0005) [2023-12-26 15:35:27,544][105692] Updated weights for policy 0, policy_version 30814 (0.0005) [2023-12-26 15:35:27,594][105692] Updated weights for policy 0, policy_version 30824 (0.0007) [2023-12-26 15:35:27,691][105620] Updated weights for policy 1, policy_version 30965 (0.0008) [2023-12-26 15:35:27,739][105620] Updated weights for policy 1, policy_version 30975 (0.0010) [2023-12-26 15:35:27,796][105620] Updated weights for policy 1, policy_version 30985 (0.0010) [2023-12-26 15:35:28,278][105692] Updated weights for policy 0, policy_version 30834 (0.0008) [2023-12-26 15:35:28,321][105692] Updated weights for policy 0, policy_version 30844 (0.0008) [2023-12-26 15:35:28,383][105692] Updated weights for policy 0, policy_version 30854 (0.0008) [2023-12-26 15:35:28,431][105692] Updated weights for policy 0, policy_version 30864 (0.0008) [2023-12-26 15:35:28,534][105620] Updated weights for policy 1, policy_version 30995 (0.0010) [2023-12-26 15:35:28,585][105620] Updated weights for policy 1, policy_version 31005 (0.0010) [2023-12-26 15:35:28,640][105620] Updated weights for policy 1, policy_version 31015 (0.0010) [2023-12-26 15:35:29,192][105692] Updated weights for policy 0, policy_version 30874 (0.0008) [2023-12-26 15:35:29,249][105692] Updated weights for policy 0, policy_version 30884 (0.0008) [2023-12-26 15:35:29,300][105692] Updated weights for policy 0, policy_version 30894 (0.0008) [2023-12-26 15:35:29,396][105620] Updated weights for policy 1, policy_version 31025 (0.0010) [2023-12-26 15:35:29,460][105620] Updated weights for policy 1, policy_version 31035 (0.0009) [2023-12-26 15:35:29,511][105620] Updated weights for policy 1, policy_version 31045 (0.0009) [2023-12-26 15:35:29,570][105620] Updated weights for policy 1, policy_version 31055 (0.0009) [2023-12-26 15:35:30,055][105692] Updated weights for policy 0, policy_version 30904 (0.0008) [2023-12-26 15:35:30,113][105692] Updated weights for policy 0, policy_version 30914 (0.0008) [2023-12-26 15:35:30,177][105692] Updated weights for policy 0, policy_version 30924 (0.0008) [2023-12-26 15:35:30,250][105620] Updated weights for policy 1, policy_version 31065 (0.0006) [2023-12-26 15:35:30,306][105620] Updated weights for policy 1, policy_version 31075 (0.0005) [2023-12-26 15:35:30,372][105620] Updated weights for policy 1, policy_version 31085 (0.0006) [2023-12-26 15:35:30,813][105692] Updated weights for policy 0, policy_version 30934 (0.0009) [2023-12-26 15:35:30,872][105692] Updated weights for policy 0, policy_version 30944 (0.0010) [2023-12-26 15:35:30,935][105692] Updated weights for policy 0, policy_version 30954 (0.0009) [2023-12-26 15:35:30,950][105620] Updated weights for policy 1, policy_version 31095 (0.0008) [2023-12-26 15:35:31,006][105620] Updated weights for policy 1, policy_version 31105 (0.0008) [2023-12-26 15:35:31,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 15892480. Throughput: 0: 9833.1, 1: 9795.0. Samples: 15861732. Policy #0 lag: (min: 10.0, avg: 19.4, max: 42.0) [2023-12-26 15:35:31,063][104569] Avg episode reward: [(0, '9182.088'), (1, '9071.979')] [2023-12-26 15:35:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000030960_7929856.pth... [2023-12-26 15:35:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000029808_7634944.pth [2023-12-26 15:35:31,075][105620] Updated weights for policy 1, policy_version 31115 (0.0009) [2023-12-26 15:35:31,108][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000031120_7970816.pth... [2023-12-26 15:35:31,113][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000029936_7667712.pth [2023-12-26 15:35:31,734][105692] Updated weights for policy 0, policy_version 30964 (0.0007) [2023-12-26 15:35:31,746][105620] Updated weights for policy 1, policy_version 31125 (0.0009) [2023-12-26 15:35:31,790][105692] Updated weights for policy 0, policy_version 30974 (0.0008) [2023-12-26 15:35:31,796][105620] Updated weights for policy 1, policy_version 31135 (0.0007) [2023-12-26 15:35:31,849][105692] Updated weights for policy 0, policy_version 30984 (0.0006) [2023-12-26 15:35:31,851][105620] Updated weights for policy 1, policy_version 31145 (0.0008) [2023-12-26 15:35:32,608][105692] Updated weights for policy 0, policy_version 30994 (0.0007) [2023-12-26 15:35:32,623][105620] Updated weights for policy 1, policy_version 31155 (0.0005) [2023-12-26 15:35:32,663][105692] Updated weights for policy 0, policy_version 31004 (0.0008) [2023-12-26 15:35:32,672][105620] Updated weights for policy 1, policy_version 31165 (0.0008) [2023-12-26 15:35:32,721][105692] Updated weights for policy 0, policy_version 31014 (0.0006) [2023-12-26 15:35:32,727][105620] Updated weights for policy 1, policy_version 31175 (0.0009) [2023-12-26 15:35:32,784][105692] Updated weights for policy 0, policy_version 31024 (0.0006) [2023-12-26 15:35:33,368][105620] Updated weights for policy 1, policy_version 31185 (0.0008) [2023-12-26 15:35:33,419][105620] Updated weights for policy 1, policy_version 31195 (0.0005) [2023-12-26 15:35:33,446][105692] Updated weights for policy 0, policy_version 31034 (0.0005) [2023-12-26 15:35:33,465][105620] Updated weights for policy 1, policy_version 31205 (0.0006) [2023-12-26 15:35:33,504][105692] Updated weights for policy 0, policy_version 31044 (0.0007) [2023-12-26 15:35:33,511][105620] Updated weights for policy 1, policy_version 31215 (0.0005) [2023-12-26 15:35:33,561][105692] Updated weights for policy 0, policy_version 31054 (0.0008) [2023-12-26 15:35:34,087][105620] Updated weights for policy 1, policy_version 31225 (0.0010) [2023-12-26 15:35:34,148][105620] Updated weights for policy 1, policy_version 31235 (0.0011) [2023-12-26 15:35:34,212][105620] Updated weights for policy 1, policy_version 31245 (0.0011) [2023-12-26 15:35:34,329][105692] Updated weights for policy 0, policy_version 31064 (0.0008) [2023-12-26 15:35:34,382][105692] Updated weights for policy 0, policy_version 31074 (0.0009) [2023-12-26 15:35:34,437][105692] Updated weights for policy 0, policy_version 31084 (0.0010) [2023-12-26 15:35:34,981][105620] Updated weights for policy 1, policy_version 31255 (0.0010) [2023-12-26 15:35:35,046][105620] Updated weights for policy 1, policy_version 31265 (0.0006) [2023-12-26 15:35:35,105][105620] Updated weights for policy 1, policy_version 31275 (0.0006) [2023-12-26 15:35:35,131][105692] Updated weights for policy 0, policy_version 31094 (0.0009) [2023-12-26 15:35:35,184][105692] Updated weights for policy 0, policy_version 31105 (0.0010) [2023-12-26 15:35:35,237][105692] Updated weights for policy 0, policy_version 31117 (0.0011) [2023-12-26 15:35:35,761][105620] Updated weights for policy 1, policy_version 31285 (0.0007) [2023-12-26 15:35:35,810][105620] Updated weights for policy 1, policy_version 31295 (0.0009) [2023-12-26 15:35:35,869][105620] Updated weights for policy 1, policy_version 31305 (0.0008) [2023-12-26 15:35:35,937][105692] Updated weights for policy 0, policy_version 31127 (0.0009) [2023-12-26 15:35:35,994][105692] Updated weights for policy 0, policy_version 31137 (0.0010) [2023-12-26 15:35:36,052][105692] Updated weights for policy 0, policy_version 31147 (0.0010) [2023-12-26 15:35:36,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19524.2, 300 sec: 19660.8). Total num frames: 15990784. Throughput: 0: 9754.9, 1: 9870.7. Samples: 15980416. Policy #0 lag: (min: 0.0, avg: 25.1, max: 32.0) [2023-12-26 15:35:36,063][104569] Avg episode reward: [(0, '9102.511'), (1, '9256.918')] [2023-12-26 15:35:36,626][105620] Updated weights for policy 1, policy_version 31315 (0.0007) [2023-12-26 15:35:36,679][105620] Updated weights for policy 1, policy_version 31325 (0.0008) [2023-12-26 15:35:36,746][105620] Updated weights for policy 1, policy_version 31335 (0.0008) [2023-12-26 15:35:36,834][105692] Updated weights for policy 0, policy_version 31157 (0.0007) [2023-12-26 15:35:36,896][105692] Updated weights for policy 0, policy_version 31167 (0.0007) [2023-12-26 15:35:36,956][105692] Updated weights for policy 0, policy_version 31177 (0.0010) [2023-12-26 15:35:37,434][105620] Updated weights for policy 1, policy_version 31345 (0.0009) [2023-12-26 15:35:37,491][105620] Updated weights for policy 1, policy_version 31355 (0.0005) [2023-12-26 15:35:37,552][105620] Updated weights for policy 1, policy_version 31365 (0.0008) [2023-12-26 15:35:37,611][105620] Updated weights for policy 1, policy_version 31375 (0.0009) [2023-12-26 15:35:37,659][105692] Updated weights for policy 0, policy_version 31187 (0.0009) [2023-12-26 15:35:37,722][105692] Updated weights for policy 0, policy_version 31197 (0.0008) [2023-12-26 15:35:37,795][105692] Updated weights for policy 0, policy_version 31207 (0.0008) [2023-12-26 15:35:38,230][105620] Updated weights for policy 1, policy_version 31385 (0.0006) [2023-12-26 15:35:38,288][105620] Updated weights for policy 1, policy_version 31395 (0.0010) [2023-12-26 15:35:38,351][105620] Updated weights for policy 1, policy_version 31405 (0.0010) [2023-12-26 15:35:38,419][105692] Updated weights for policy 0, policy_version 31217 (0.0007) [2023-12-26 15:35:38,476][105692] Updated weights for policy 0, policy_version 31227 (0.0008) [2023-12-26 15:35:38,538][105692] Updated weights for policy 0, policy_version 31237 (0.0008) [2023-12-26 15:35:38,599][105692] Updated weights for policy 0, policy_version 31247 (0.0008) [2023-12-26 15:35:39,075][105620] Updated weights for policy 1, policy_version 31415 (0.0010) [2023-12-26 15:35:39,140][105620] Updated weights for policy 1, policy_version 31425 (0.0010) [2023-12-26 15:35:39,199][105620] Updated weights for policy 1, policy_version 31435 (0.0011) [2023-12-26 15:35:39,342][105692] Updated weights for policy 0, policy_version 31257 (0.0008) [2023-12-26 15:35:39,412][105692] Updated weights for policy 0, policy_version 31267 (0.0008) [2023-12-26 15:35:39,478][105692] Updated weights for policy 0, policy_version 31277 (0.0008) [2023-12-26 15:35:39,884][105620] Updated weights for policy 1, policy_version 31445 (0.0011) [2023-12-26 15:35:39,944][105620] Updated weights for policy 1, policy_version 31455 (0.0010) [2023-12-26 15:35:40,005][105620] Updated weights for policy 1, policy_version 31465 (0.0008) [2023-12-26 15:35:40,229][105692] Updated weights for policy 0, policy_version 31287 (0.0006) [2023-12-26 15:35:40,295][105692] Updated weights for policy 0, policy_version 31297 (0.0010) [2023-12-26 15:35:40,357][105692] Updated weights for policy 0, policy_version 31307 (0.0011) [2023-12-26 15:35:40,685][105620] Updated weights for policy 1, policy_version 31475 (0.0008) [2023-12-26 15:35:40,752][105620] Updated weights for policy 1, policy_version 31485 (0.0009) [2023-12-26 15:35:40,809][105620] Updated weights for policy 1, policy_version 31495 (0.0009) [2023-12-26 15:35:40,971][105692] Updated weights for policy 0, policy_version 31317 (0.0011) [2023-12-26 15:35:41,032][105692] Updated weights for policy 0, policy_version 31327 (0.0011) [2023-12-26 15:35:41,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 16089088. Throughput: 0: 9792.2, 1: 9937.3. Samples: 16098872. Policy #0 lag: (min: 0.0, avg: 25.1, max: 32.0) [2023-12-26 15:35:41,062][104569] Avg episode reward: [(0, '9271.019'), (1, '9257.425')] [2023-12-26 15:35:41,098][105692] Updated weights for policy 0, policy_version 31337 (0.0011) [2023-12-26 15:35:41,535][105620] Updated weights for policy 1, policy_version 31505 (0.0009) [2023-12-26 15:35:41,597][105620] Updated weights for policy 1, policy_version 31515 (0.0006) [2023-12-26 15:35:41,660][105620] Updated weights for policy 1, policy_version 31525 (0.0008) [2023-12-26 15:35:41,723][105620] Updated weights for policy 1, policy_version 31535 (0.0010) [2023-12-26 15:35:41,893][105692] Updated weights for policy 0, policy_version 31347 (0.0008) [2023-12-26 15:35:41,956][105692] Updated weights for policy 0, policy_version 31357 (0.0008) [2023-12-26 15:35:42,017][105692] Updated weights for policy 0, policy_version 31367 (0.0008) [2023-12-26 15:35:42,492][105620] Updated weights for policy 1, policy_version 31545 (0.0010) [2023-12-26 15:35:42,547][105620] Updated weights for policy 1, policy_version 31555 (0.0009) [2023-12-26 15:35:42,605][105620] Updated weights for policy 1, policy_version 31565 (0.0006) [2023-12-26 15:35:42,802][105692] Updated weights for policy 0, policy_version 31377 (0.0008) [2023-12-26 15:35:42,861][105692] Updated weights for policy 0, policy_version 31387 (0.0010) [2023-12-26 15:35:42,919][105692] Updated weights for policy 0, policy_version 31397 (0.0009) [2023-12-26 15:35:42,972][105692] Updated weights for policy 0, policy_version 31407 (0.0008) [2023-12-26 15:35:43,208][105620] Updated weights for policy 1, policy_version 31575 (0.0008) [2023-12-26 15:35:43,259][105620] Updated weights for policy 1, policy_version 31585 (0.0005) [2023-12-26 15:35:43,319][105620] Updated weights for policy 1, policy_version 31595 (0.0005) [2023-12-26 15:35:43,819][105692] Updated weights for policy 0, policy_version 31418 (0.0009) [2023-12-26 15:35:43,872][105692] Updated weights for policy 0, policy_version 31430 (0.0010) [2023-12-26 15:35:43,903][105620] Updated weights for policy 1, policy_version 31605 (0.0005) [2023-12-26 15:35:43,964][105620] Updated weights for policy 1, policy_version 31615 (0.0007) [2023-12-26 15:35:44,030][105620] Updated weights for policy 1, policy_version 31625 (0.0009) [2023-12-26 15:35:44,612][105692] Updated weights for policy 0, policy_version 31441 (0.0009) [2023-12-26 15:35:44,662][105692] Updated weights for policy 0, policy_version 31451 (0.0009) [2023-12-26 15:35:44,720][105692] Updated weights for policy 0, policy_version 31461 (0.0008) [2023-12-26 15:35:44,751][105620] Updated weights for policy 1, policy_version 31635 (0.0008) [2023-12-26 15:35:44,783][105692] Updated weights for policy 0, policy_version 31471 (0.0008) [2023-12-26 15:35:44,811][105620] Updated weights for policy 1, policy_version 31645 (0.0007) [2023-12-26 15:35:44,865][105620] Updated weights for policy 1, policy_version 31655 (0.0005) [2023-12-26 15:35:45,545][105620] Updated weights for policy 1, policy_version 31665 (0.0005) [2023-12-26 15:35:45,609][105620] Updated weights for policy 1, policy_version 31675 (0.0005) [2023-12-26 15:35:45,621][105692] Updated weights for policy 0, policy_version 31481 (0.0008) [2023-12-26 15:35:45,674][105620] Updated weights for policy 1, policy_version 31685 (0.0005) [2023-12-26 15:35:45,686][105692] Updated weights for policy 0, policy_version 31491 (0.0008) [2023-12-26 15:35:45,742][105620] Updated weights for policy 1, policy_version 31695 (0.0005) [2023-12-26 15:35:45,751][105692] Updated weights for policy 0, policy_version 31501 (0.0010) [2023-12-26 15:35:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 16187392. Throughput: 0: 9752.8, 1: 9968.7. Samples: 16157508. Policy #0 lag: (min: 0.0, avg: 25.1, max: 32.0) [2023-12-26 15:35:46,063][104569] Avg episode reward: [(0, '9352.226'), (1, '8980.233')] [2023-12-26 15:35:46,073][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000031504_8069120.pth... [2023-12-26 15:35:46,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000031696_8118272.pth... [2023-12-26 15:35:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000030544_7823360.pth [2023-12-26 15:35:46,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000030352_7774208.pth [2023-12-26 15:35:46,389][105620] Updated weights for policy 1, policy_version 31705 (0.0008) [2023-12-26 15:35:46,452][105620] Updated weights for policy 1, policy_version 31715 (0.0009) [2023-12-26 15:35:46,466][105692] Updated weights for policy 0, policy_version 31511 (0.0008) [2023-12-26 15:35:46,508][105620] Updated weights for policy 1, policy_version 31725 (0.0006) [2023-12-26 15:35:46,522][105692] Updated weights for policy 0, policy_version 31521 (0.0007) [2023-12-26 15:35:46,573][105692] Updated weights for policy 0, policy_version 31531 (0.0008) [2023-12-26 15:35:47,262][105620] Updated weights for policy 1, policy_version 31735 (0.0009) [2023-12-26 15:35:47,324][105620] Updated weights for policy 1, policy_version 31745 (0.0010) [2023-12-26 15:35:47,350][105692] Updated weights for policy 0, policy_version 31541 (0.0007) [2023-12-26 15:35:47,383][105620] Updated weights for policy 1, policy_version 31755 (0.0010) [2023-12-26 15:35:47,405][105692] Updated weights for policy 0, policy_version 31551 (0.0007) [2023-12-26 15:35:47,471][105692] Updated weights for policy 0, policy_version 31561 (0.0009) [2023-12-26 15:35:47,964][105620] Updated weights for policy 1, policy_version 31765 (0.0008) [2023-12-26 15:35:48,026][105620] Updated weights for policy 1, policy_version 31775 (0.0009) [2023-12-26 15:35:48,077][105620] Updated weights for policy 1, policy_version 31785 (0.0006) [2023-12-26 15:35:48,325][105692] Updated weights for policy 0, policy_version 31571 (0.0009) [2023-12-26 15:35:48,387][105692] Updated weights for policy 0, policy_version 31581 (0.0009) [2023-12-26 15:35:48,439][105692] Updated weights for policy 0, policy_version 31591 (0.0009) [2023-12-26 15:35:48,726][105620] Updated weights for policy 1, policy_version 31795 (0.0006) [2023-12-26 15:35:48,784][105620] Updated weights for policy 1, policy_version 31805 (0.0009) [2023-12-26 15:35:48,838][105620] Updated weights for policy 1, policy_version 31815 (0.0007) [2023-12-26 15:35:49,206][105692] Updated weights for policy 0, policy_version 31601 (0.0009) [2023-12-26 15:35:49,272][105692] Updated weights for policy 0, policy_version 31611 (0.0009) [2023-12-26 15:35:49,341][105692] Updated weights for policy 0, policy_version 31621 (0.0009) [2023-12-26 15:35:49,406][105692] Updated weights for policy 0, policy_version 31631 (0.0009) [2023-12-26 15:35:49,580][105620] Updated weights for policy 1, policy_version 31825 (0.0008) [2023-12-26 15:35:49,639][105620] Updated weights for policy 1, policy_version 31835 (0.0005) [2023-12-26 15:35:49,692][105620] Updated weights for policy 1, policy_version 31845 (0.0005) [2023-12-26 15:35:49,749][105620] Updated weights for policy 1, policy_version 31855 (0.0006) [2023-12-26 15:35:50,271][105692] Updated weights for policy 0, policy_version 31641 (0.0009) [2023-12-26 15:35:50,330][105692] Updated weights for policy 0, policy_version 31651 (0.0010) [2023-12-26 15:35:50,388][105692] Updated weights for policy 0, policy_version 31661 (0.0007) [2023-12-26 15:35:50,398][105620] Updated weights for policy 1, policy_version 31865 (0.0010) [2023-12-26 15:35:50,446][105620] Updated weights for policy 1, policy_version 31875 (0.0010) [2023-12-26 15:35:50,498][105620] Updated weights for policy 1, policy_version 31885 (0.0010) [2023-12-26 15:35:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 16277504. Throughput: 0: 9643.4, 1: 10021.3. Samples: 16271444. Policy #0 lag: (min: 0.0, avg: 25.1, max: 32.0) [2023-12-26 15:35:51,062][104569] Avg episode reward: [(0, '9351.907'), (1, '8794.085')] [2023-12-26 15:35:51,197][105692] Updated weights for policy 0, policy_version 31671 (0.0008) [2023-12-26 15:35:51,233][105620] Updated weights for policy 1, policy_version 31895 (0.0008) [2023-12-26 15:35:51,268][105692] Updated weights for policy 0, policy_version 31681 (0.0010) [2023-12-26 15:35:51,296][105620] Updated weights for policy 1, policy_version 31905 (0.0008) [2023-12-26 15:35:51,328][105692] Updated weights for policy 0, policy_version 31691 (0.0008) [2023-12-26 15:35:51,361][105620] Updated weights for policy 1, policy_version 31915 (0.0010) [2023-12-26 15:35:52,029][105620] Updated weights for policy 1, policy_version 31925 (0.0010) [2023-12-26 15:35:52,094][105620] Updated weights for policy 1, policy_version 31935 (0.0008) [2023-12-26 15:35:52,112][105692] Updated weights for policy 0, policy_version 31701 (0.0008) [2023-12-26 15:35:52,149][105620] Updated weights for policy 1, policy_version 31945 (0.0009) [2023-12-26 15:35:52,172][105692] Updated weights for policy 0, policy_version 31711 (0.0007) [2023-12-26 15:35:52,231][105692] Updated weights for policy 0, policy_version 31721 (0.0009) [2023-12-26 15:35:52,874][105620] Updated weights for policy 1, policy_version 31955 (0.0007) [2023-12-26 15:35:52,928][105620] Updated weights for policy 1, policy_version 31965 (0.0010) [2023-12-26 15:35:52,978][105620] Updated weights for policy 1, policy_version 31975 (0.0009) [2023-12-26 15:35:52,981][105692] Updated weights for policy 0, policy_version 31731 (0.0009) [2023-12-26 15:35:53,035][105692] Updated weights for policy 0, policy_version 31741 (0.0005) [2023-12-26 15:35:53,094][105692] Updated weights for policy 0, policy_version 31751 (0.0006) [2023-12-26 15:35:53,753][105620] Updated weights for policy 1, policy_version 31985 (0.0008) [2023-12-26 15:35:53,796][105692] Updated weights for policy 0, policy_version 31761 (0.0009) [2023-12-26 15:35:53,810][105620] Updated weights for policy 1, policy_version 31995 (0.0011) [2023-12-26 15:35:53,844][105692] Updated weights for policy 0, policy_version 31771 (0.0005) [2023-12-26 15:35:53,866][105620] Updated weights for policy 1, policy_version 32005 (0.0009) [2023-12-26 15:35:53,896][105692] Updated weights for policy 0, policy_version 31781 (0.0006) [2023-12-26 15:35:53,922][105620] Updated weights for policy 1, policy_version 32015 (0.0008) [2023-12-26 15:35:53,943][105692] Updated weights for policy 0, policy_version 31791 (0.0006) [2023-12-26 15:35:54,652][105620] Updated weights for policy 1, policy_version 32025 (0.0010) [2023-12-26 15:35:54,706][105620] Updated weights for policy 1, policy_version 32035 (0.0010) [2023-12-26 15:35:54,711][105692] Updated weights for policy 0, policy_version 31801 (0.0009) [2023-12-26 15:35:54,765][105620] Updated weights for policy 1, policy_version 32045 (0.0010) [2023-12-26 15:35:54,773][105692] Updated weights for policy 0, policy_version 31811 (0.0011) [2023-12-26 15:35:54,835][105692] Updated weights for policy 0, policy_version 31821 (0.0010) [2023-12-26 15:35:55,403][105620] Updated weights for policy 1, policy_version 32055 (0.0007) [2023-12-26 15:35:55,457][105620] Updated weights for policy 1, policy_version 32065 (0.0006) [2023-12-26 15:35:55,517][105692] Updated weights for policy 0, policy_version 31831 (0.0010) [2023-12-26 15:35:55,519][105620] Updated weights for policy 1, policy_version 32075 (0.0006) [2023-12-26 15:35:55,569][105692] Updated weights for policy 0, policy_version 31841 (0.0010) [2023-12-26 15:35:55,621][105692] Updated weights for policy 0, policy_version 31851 (0.0010) [2023-12-26 15:35:56,029][105620] Updated weights for policy 1, policy_version 32085 (0.0008) [2023-12-26 15:35:56,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19605.2). Total num frames: 16375808. Throughput: 0: 9713.0, 1: 9921.7. Samples: 16387540. Policy #0 lag: (min: 24.0, avg: 53.6, max: 56.0) [2023-12-26 15:35:56,063][104569] Avg episode reward: [(0, '9261.242'), (1, '8424.738')] [2023-12-26 15:35:56,096][105620] Updated weights for policy 1, policy_version 32095 (0.0009) [2023-12-26 15:35:56,153][105620] Updated weights for policy 1, policy_version 32105 (0.0005) [2023-12-26 15:35:56,406][105692] Updated weights for policy 0, policy_version 31861 (0.0010) [2023-12-26 15:35:56,457][105692] Updated weights for policy 0, policy_version 31871 (0.0009) [2023-12-26 15:35:56,510][105692] Updated weights for policy 0, policy_version 31881 (0.0010) [2023-12-26 15:35:56,665][105620] Updated weights for policy 1, policy_version 32115 (0.0005) [2023-12-26 15:35:56,722][105620] Updated weights for policy 1, policy_version 32125 (0.0007) [2023-12-26 15:35:56,792][105620] Updated weights for policy 1, policy_version 32135 (0.0006) [2023-12-26 15:35:57,330][105692] Updated weights for policy 0, policy_version 31891 (0.0009) [2023-12-26 15:35:57,386][105692] Updated weights for policy 0, policy_version 31901 (0.0008) [2023-12-26 15:35:57,430][105692] Updated weights for policy 0, policy_version 31911 (0.0007) [2023-12-26 15:35:57,495][105620] Updated weights for policy 1, policy_version 32145 (0.0010) [2023-12-26 15:35:57,549][105620] Updated weights for policy 1, policy_version 32155 (0.0010) [2023-12-26 15:35:57,609][105620] Updated weights for policy 1, policy_version 32165 (0.0010) [2023-12-26 15:35:57,663][105620] Updated weights for policy 1, policy_version 32175 (0.0011) [2023-12-26 15:35:58,236][105692] Updated weights for policy 0, policy_version 31921 (0.0008) [2023-12-26 15:35:58,295][105692] Updated weights for policy 0, policy_version 31931 (0.0008) [2023-12-26 15:35:58,345][105620] Updated weights for policy 1, policy_version 32185 (0.0011) [2023-12-26 15:35:58,357][105692] Updated weights for policy 0, policy_version 31941 (0.0009) [2023-12-26 15:35:58,407][105620] Updated weights for policy 1, policy_version 32195 (0.0011) [2023-12-26 15:35:58,421][105692] Updated weights for policy 0, policy_version 31951 (0.0006) [2023-12-26 15:35:58,469][105620] Updated weights for policy 1, policy_version 32205 (0.0008) [2023-12-26 15:35:59,237][105620] Updated weights for policy 1, policy_version 32215 (0.0008) [2023-12-26 15:35:59,278][105692] Updated weights for policy 0, policy_version 31961 (0.0006) [2023-12-26 15:35:59,300][105620] Updated weights for policy 1, policy_version 32225 (0.0010) [2023-12-26 15:35:59,344][105692] Updated weights for policy 0, policy_version 31971 (0.0007) [2023-12-26 15:35:59,357][105620] Updated weights for policy 1, policy_version 32235 (0.0009) [2023-12-26 15:35:59,412][105692] Updated weights for policy 0, policy_version 31981 (0.0008) [2023-12-26 15:36:00,121][105620] Updated weights for policy 1, policy_version 32245 (0.0014) [2023-12-26 15:36:00,149][105692] Updated weights for policy 0, policy_version 31991 (0.0009) [2023-12-26 15:36:00,169][105620] Updated weights for policy 1, policy_version 32255 (0.0010) [2023-12-26 15:36:00,199][105692] Updated weights for policy 0, policy_version 32001 (0.0009) [2023-12-26 15:36:00,224][105620] Updated weights for policy 1, policy_version 32265 (0.0010) [2023-12-26 15:36:00,257][105692] Updated weights for policy 0, policy_version 32011 (0.0005) [2023-12-26 15:36:00,975][105620] Updated weights for policy 1, policy_version 32275 (0.0009) [2023-12-26 15:36:00,982][105692] Updated weights for policy 0, policy_version 32021 (0.0007) [2023-12-26 15:36:01,028][105620] Updated weights for policy 1, policy_version 32285 (0.0006) [2023-12-26 15:36:01,045][105692] Updated weights for policy 0, policy_version 32031 (0.0007) [2023-12-26 15:36:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 16465920. Throughput: 0: 9666.4, 1: 9991.4. Samples: 16445124. Policy #0 lag: (min: 24.0, avg: 53.6, max: 56.0) [2023-12-26 15:36:01,062][104569] Avg episode reward: [(0, '9259.563'), (1, '8331.187')] [2023-12-26 15:36:01,090][105620] Updated weights for policy 1, policy_version 32295 (0.0009) [2023-12-26 15:36:01,101][105692] Updated weights for policy 0, policy_version 32041 (0.0007) [2023-12-26 15:36:01,148][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000032304_8273920.pth... [2023-12-26 15:36:01,151][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000032048_8208384.pth... [2023-12-26 15:36:01,152][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000031120_7970816.pth [2023-12-26 15:36:01,157][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000030960_7929856.pth [2023-12-26 15:36:01,814][105620] Updated weights for policy 1, policy_version 32305 (0.0009) [2023-12-26 15:36:01,819][105692] Updated weights for policy 0, policy_version 32051 (0.0008) [2023-12-26 15:36:01,874][105620] Updated weights for policy 1, policy_version 32315 (0.0009) [2023-12-26 15:36:01,884][105692] Updated weights for policy 0, policy_version 32061 (0.0006) [2023-12-26 15:36:01,937][105620] Updated weights for policy 1, policy_version 32325 (0.0010) [2023-12-26 15:36:01,942][105692] Updated weights for policy 0, policy_version 32071 (0.0005) [2023-12-26 15:36:02,000][105620] Updated weights for policy 1, policy_version 32335 (0.0011) [2023-12-26 15:36:02,687][105620] Updated weights for policy 1, policy_version 32345 (0.0009) [2023-12-26 15:36:02,710][105692] Updated weights for policy 0, policy_version 32081 (0.0006) [2023-12-26 15:36:02,745][105620] Updated weights for policy 1, policy_version 32355 (0.0007) [2023-12-26 15:36:02,766][105692] Updated weights for policy 0, policy_version 32091 (0.0007) [2023-12-26 15:36:02,805][105620] Updated weights for policy 1, policy_version 32365 (0.0008) [2023-12-26 15:36:02,814][105692] Updated weights for policy 0, policy_version 32101 (0.0006) [2023-12-26 15:36:02,869][105692] Updated weights for policy 0, policy_version 32111 (0.0009) [2023-12-26 15:36:03,530][105692] Updated weights for policy 0, policy_version 32121 (0.0008) [2023-12-26 15:36:03,555][105620] Updated weights for policy 1, policy_version 32375 (0.0007) [2023-12-26 15:36:03,574][105692] Updated weights for policy 0, policy_version 32131 (0.0006) [2023-12-26 15:36:03,612][105620] Updated weights for policy 1, policy_version 32385 (0.0007) [2023-12-26 15:36:03,618][105692] Updated weights for policy 0, policy_version 32141 (0.0009) [2023-12-26 15:36:03,679][105620] Updated weights for policy 1, policy_version 32395 (0.0006) [2023-12-26 15:36:04,341][105620] Updated weights for policy 1, policy_version 32405 (0.0008) [2023-12-26 15:36:04,374][105692] Updated weights for policy 0, policy_version 32152 (0.0009) [2023-12-26 15:36:04,405][105620] Updated weights for policy 1, policy_version 32415 (0.0006) [2023-12-26 15:36:04,428][105692] Updated weights for policy 0, policy_version 32162 (0.0007) [2023-12-26 15:36:04,471][105620] Updated weights for policy 1, policy_version 32425 (0.0009) [2023-12-26 15:36:04,478][105692] Updated weights for policy 0, policy_version 32172 (0.0006) [2023-12-26 15:36:05,218][105620] Updated weights for policy 1, policy_version 32435 (0.0009) [2023-12-26 15:36:05,242][105692] Updated weights for policy 0, policy_version 32182 (0.0008) [2023-12-26 15:36:05,276][105620] Updated weights for policy 1, policy_version 32445 (0.0007) [2023-12-26 15:36:05,291][105692] Updated weights for policy 0, policy_version 32192 (0.0007) [2023-12-26 15:36:05,335][105620] Updated weights for policy 1, policy_version 32455 (0.0008) [2023-12-26 15:36:05,339][105692] Updated weights for policy 0, policy_version 32202 (0.0008) [2023-12-26 15:36:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 16564224. Throughput: 0: 9493.6, 1: 9973.1. Samples: 16558984. Policy #0 lag: (min: 24.0, avg: 53.6, max: 56.0) [2023-12-26 15:36:06,063][104569] Avg episode reward: [(0, '9346.828'), (1, '8885.773')] [2023-12-26 15:36:06,085][105620] Updated weights for policy 1, policy_version 32465 (0.0006) [2023-12-26 15:36:06,110][105692] Updated weights for policy 0, policy_version 32212 (0.0007) [2023-12-26 15:36:06,146][105620] Updated weights for policy 1, policy_version 32475 (0.0007) [2023-12-26 15:36:06,171][105692] Updated weights for policy 0, policy_version 32222 (0.0008) [2023-12-26 15:36:06,199][105620] Updated weights for policy 1, policy_version 32485 (0.0009) [2023-12-26 15:36:06,228][105692] Updated weights for policy 0, policy_version 32232 (0.0009) [2023-12-26 15:36:06,257][105620] Updated weights for policy 1, policy_version 32495 (0.0009) [2023-12-26 15:36:06,958][105620] Updated weights for policy 1, policy_version 32505 (0.0009) [2023-12-26 15:36:07,026][105620] Updated weights for policy 1, policy_version 32515 (0.0006) [2023-12-26 15:36:07,029][105692] Updated weights for policy 0, policy_version 32242 (0.0007) [2023-12-26 15:36:07,087][105620] Updated weights for policy 1, policy_version 32525 (0.0006) [2023-12-26 15:36:07,088][105692] Updated weights for policy 0, policy_version 32252 (0.0008) [2023-12-26 15:36:07,153][105692] Updated weights for policy 0, policy_version 32262 (0.0010) [2023-12-26 15:36:07,207][105692] Updated weights for policy 0, policy_version 32272 (0.0010) [2023-12-26 15:36:07,770][105620] Updated weights for policy 1, policy_version 32535 (0.0007) [2023-12-26 15:36:07,819][105620] Updated weights for policy 1, policy_version 32545 (0.0008) [2023-12-26 15:36:07,869][105620] Updated weights for policy 1, policy_version 32555 (0.0009) [2023-12-26 15:36:07,962][105692] Updated weights for policy 0, policy_version 32282 (0.0006) [2023-12-26 15:36:08,023][105692] Updated weights for policy 0, policy_version 32292 (0.0007) [2023-12-26 15:36:08,078][105692] Updated weights for policy 0, policy_version 32302 (0.0011) [2023-12-26 15:36:08,642][105620] Updated weights for policy 1, policy_version 32565 (0.0009) [2023-12-26 15:36:08,699][105620] Updated weights for policy 1, policy_version 32575 (0.0009) [2023-12-26 15:36:08,751][105692] Updated weights for policy 0, policy_version 32312 (0.0006) [2023-12-26 15:36:08,756][105620] Updated weights for policy 1, policy_version 32585 (0.0009) [2023-12-26 15:36:08,811][105692] Updated weights for policy 0, policy_version 32322 (0.0005) [2023-12-26 15:36:08,869][105692] Updated weights for policy 0, policy_version 32332 (0.0008) [2023-12-26 15:36:09,442][105692] Updated weights for policy 0, policy_version 32342 (0.0006) [2023-12-26 15:36:09,503][105692] Updated weights for policy 0, policy_version 32352 (0.0006) [2023-12-26 15:36:09,560][105692] Updated weights for policy 0, policy_version 32362 (0.0006) [2023-12-26 15:36:09,664][105620] Updated weights for policy 1, policy_version 32595 (0.0008) [2023-12-26 15:36:09,724][105620] Updated weights for policy 1, policy_version 32605 (0.0009) [2023-12-26 15:36:09,783][105620] Updated weights for policy 1, policy_version 32616 (0.0010) [2023-12-26 15:36:10,228][105692] Updated weights for policy 0, policy_version 32372 (0.0008) [2023-12-26 15:36:10,282][105692] Updated weights for policy 0, policy_version 32383 (0.0010) [2023-12-26 15:36:10,336][105692] Updated weights for policy 0, policy_version 32393 (0.0009) [2023-12-26 15:36:10,550][105620] Updated weights for policy 1, policy_version 32626 (0.0010) [2023-12-26 15:36:10,598][105620] Updated weights for policy 1, policy_version 32636 (0.0009) [2023-12-26 15:36:10,650][105620] Updated weights for policy 1, policy_version 32646 (0.0009) [2023-12-26 15:36:10,701][105620] Updated weights for policy 1, policy_version 32656 (0.0009) [2023-12-26 15:36:11,037][105692] Updated weights for policy 0, policy_version 32403 (0.0009) [2023-12-26 15:36:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 16662528. Throughput: 0: 9446.6, 1: 9918.6. Samples: 16672340. Policy #0 lag: (min: 24.0, avg: 53.6, max: 56.0) [2023-12-26 15:36:11,063][104569] Avg episode reward: [(0, '9256.949'), (1, '9165.003')] [2023-12-26 15:36:11,100][105692] Updated weights for policy 0, policy_version 32413 (0.0009) [2023-12-26 15:36:11,173][105692] Updated weights for policy 0, policy_version 32423 (0.0008) [2023-12-26 15:36:11,522][105620] Updated weights for policy 1, policy_version 32666 (0.0006) [2023-12-26 15:36:11,594][105620] Updated weights for policy 1, policy_version 32676 (0.0009) [2023-12-26 15:36:11,668][105620] Updated weights for policy 1, policy_version 32686 (0.0010) [2023-12-26 15:36:11,907][105692] Updated weights for policy 0, policy_version 32433 (0.0007) [2023-12-26 15:36:11,969][105692] Updated weights for policy 0, policy_version 32443 (0.0011) [2023-12-26 15:36:12,036][105692] Updated weights for policy 0, policy_version 32453 (0.0006) [2023-12-26 15:36:12,105][105692] Updated weights for policy 0, policy_version 32463 (0.0011) [2023-12-26 15:36:12,338][105620] Updated weights for policy 1, policy_version 32696 (0.0011) [2023-12-26 15:36:12,403][105620] Updated weights for policy 1, policy_version 32706 (0.0012) [2023-12-26 15:36:12,465][105620] Updated weights for policy 1, policy_version 32716 (0.0011) [2023-12-26 15:36:12,817][105692] Updated weights for policy 0, policy_version 32473 (0.0011) [2023-12-26 15:36:12,869][105692] Updated weights for policy 0, policy_version 32483 (0.0010) [2023-12-26 15:36:12,931][105692] Updated weights for policy 0, policy_version 32493 (0.0011) [2023-12-26 15:36:13,195][105620] Updated weights for policy 1, policy_version 32726 (0.0008) [2023-12-26 15:36:13,247][105620] Updated weights for policy 1, policy_version 32736 (0.0005) [2023-12-26 15:36:13,304][105620] Updated weights for policy 1, policy_version 32746 (0.0005) [2023-12-26 15:36:13,597][105692] Updated weights for policy 0, policy_version 32503 (0.0011) [2023-12-26 15:36:13,653][105692] Updated weights for policy 0, policy_version 32513 (0.0010) [2023-12-26 15:36:13,719][105692] Updated weights for policy 0, policy_version 32523 (0.0010) [2023-12-26 15:36:13,998][105620] Updated weights for policy 1, policy_version 32756 (0.0007) [2023-12-26 15:36:14,053][105620] Updated weights for policy 1, policy_version 32766 (0.0006) [2023-12-26 15:36:14,104][105620] Updated weights for policy 1, policy_version 32776 (0.0006) [2023-12-26 15:36:14,419][105692] Updated weights for policy 0, policy_version 32533 (0.0011) [2023-12-26 15:36:14,474][105692] Updated weights for policy 0, policy_version 32543 (0.0010) [2023-12-26 15:36:14,537][105692] Updated weights for policy 0, policy_version 32553 (0.0010) [2023-12-26 15:36:14,826][105620] Updated weights for policy 1, policy_version 32786 (0.0008) [2023-12-26 15:36:14,889][105620] Updated weights for policy 1, policy_version 32796 (0.0005) [2023-12-26 15:36:14,952][105620] Updated weights for policy 1, policy_version 32806 (0.0009) [2023-12-26 15:36:15,011][105620] Updated weights for policy 1, policy_version 32816 (0.0010) [2023-12-26 15:36:15,207][105692] Updated weights for policy 0, policy_version 32563 (0.0011) [2023-12-26 15:36:15,256][105692] Updated weights for policy 0, policy_version 32573 (0.0011) [2023-12-26 15:36:15,321][105692] Updated weights for policy 0, policy_version 32583 (0.0009) [2023-12-26 15:36:15,680][105620] Updated weights for policy 1, policy_version 32826 (0.0011) [2023-12-26 15:36:15,743][105620] Updated weights for policy 1, policy_version 32836 (0.0011) [2023-12-26 15:36:15,791][105620] Updated weights for policy 1, policy_version 32846 (0.0010) [2023-12-26 15:36:16,043][105692] Updated weights for policy 0, policy_version 32593 (0.0006) [2023-12-26 15:36:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 16760832. Throughput: 0: 9423.8, 1: 9910.7. Samples: 16731780. Policy #0 lag: (min: 24.0, avg: 53.6, max: 56.0) [2023-12-26 15:36:16,062][104569] Avg episode reward: [(0, '9256.434'), (1, '9257.642')] [2023-12-26 15:36:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000032848_8413184.pth... [2023-12-26 15:36:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000031696_8118272.pth [2023-12-26 15:36:16,093][105692] Updated weights for policy 0, policy_version 32603 (0.0010) [2023-12-26 15:36:16,142][105692] Updated weights for policy 0, policy_version 32613 (0.0005) [2023-12-26 15:36:16,195][105692] Updated weights for policy 0, policy_version 32623 (0.0005) [2023-12-26 15:36:16,203][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000032624_8355840.pth... [2023-12-26 15:36:16,206][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000031504_8069120.pth [2023-12-26 15:36:16,523][105620] Updated weights for policy 1, policy_version 32856 (0.0009) [2023-12-26 15:36:16,587][105620] Updated weights for policy 1, policy_version 32866 (0.0005) [2023-12-26 15:36:16,642][105620] Updated weights for policy 1, policy_version 32876 (0.0005) [2023-12-26 15:36:16,913][105692] Updated weights for policy 0, policy_version 32633 (0.0010) [2023-12-26 15:36:16,968][105692] Updated weights for policy 0, policy_version 32643 (0.0010) [2023-12-26 15:36:17,023][105692] Updated weights for policy 0, policy_version 32653 (0.0011) [2023-12-26 15:36:17,166][105620] Updated weights for policy 1, policy_version 32886 (0.0008) [2023-12-26 15:36:17,230][105620] Updated weights for policy 1, policy_version 32896 (0.0010) [2023-12-26 15:36:17,292][105620] Updated weights for policy 1, policy_version 32906 (0.0010) [2023-12-26 15:36:17,748][105692] Updated weights for policy 0, policy_version 32663 (0.0011) [2023-12-26 15:36:17,807][105692] Updated weights for policy 0, policy_version 32673 (0.0007) [2023-12-26 15:36:17,872][105692] Updated weights for policy 0, policy_version 32683 (0.0008) [2023-12-26 15:36:18,019][105620] Updated weights for policy 1, policy_version 32916 (0.0009) [2023-12-26 15:36:18,072][105620] Updated weights for policy 1, policy_version 32926 (0.0005) [2023-12-26 15:36:18,118][105620] Updated weights for policy 1, policy_version 32936 (0.0006) [2023-12-26 15:36:18,545][105692] Updated weights for policy 0, policy_version 32693 (0.0010) [2023-12-26 15:36:18,613][105692] Updated weights for policy 0, policy_version 32703 (0.0007) [2023-12-26 15:36:18,676][105692] Updated weights for policy 0, policy_version 32713 (0.0010) [2023-12-26 15:36:18,804][105620] Updated weights for policy 1, policy_version 32946 (0.0006) [2023-12-26 15:36:18,868][105620] Updated weights for policy 1, policy_version 32956 (0.0009) [2023-12-26 15:36:18,933][105620] Updated weights for policy 1, policy_version 32966 (0.0008) [2023-12-26 15:36:18,989][105620] Updated weights for policy 1, policy_version 32976 (0.0008) [2023-12-26 15:36:19,340][105692] Updated weights for policy 0, policy_version 32723 (0.0008) [2023-12-26 15:36:19,408][105692] Updated weights for policy 0, policy_version 32733 (0.0007) [2023-12-26 15:36:19,467][105692] Updated weights for policy 0, policy_version 32743 (0.0005) [2023-12-26 15:36:19,795][105620] Updated weights for policy 1, policy_version 32986 (0.0009) [2023-12-26 15:36:19,862][105620] Updated weights for policy 1, policy_version 32996 (0.0009) [2023-12-26 15:36:19,930][105620] Updated weights for policy 1, policy_version 33006 (0.0008) [2023-12-26 15:36:20,125][105692] Updated weights for policy 0, policy_version 32753 (0.0010) [2023-12-26 15:36:20,178][105692] Updated weights for policy 0, policy_version 32763 (0.0011) [2023-12-26 15:36:20,230][105692] Updated weights for policy 0, policy_version 32773 (0.0010) [2023-12-26 15:36:20,282][105692] Updated weights for policy 0, policy_version 32783 (0.0011) [2023-12-26 15:36:20,577][105620] Updated weights for policy 1, policy_version 33016 (0.0008) [2023-12-26 15:36:20,640][105620] Updated weights for policy 1, policy_version 33026 (0.0009) [2023-12-26 15:36:20,708][105620] Updated weights for policy 1, policy_version 33036 (0.0010) [2023-12-26 15:36:20,933][105692] Updated weights for policy 0, policy_version 32793 (0.0006) [2023-12-26 15:36:20,989][105692] Updated weights for policy 0, policy_version 32803 (0.0005) [2023-12-26 15:36:21,043][105692] Updated weights for policy 0, policy_version 32813 (0.0010) [2023-12-26 15:36:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 16859136. Throughput: 0: 9474.5, 1: 9853.2. Samples: 16850160. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-26 15:36:21,062][104569] Avg episode reward: [(0, '9253.579'), (1, '9255.257')] [2023-12-26 15:36:21,515][105620] Updated weights for policy 1, policy_version 33046 (0.0009) [2023-12-26 15:36:21,575][105620] Updated weights for policy 1, policy_version 33056 (0.0006) [2023-12-26 15:36:21,641][105620] Updated weights for policy 1, policy_version 33066 (0.0008) [2023-12-26 15:36:21,760][105692] Updated weights for policy 0, policy_version 32823 (0.0010) [2023-12-26 15:36:21,823][105692] Updated weights for policy 0, policy_version 32833 (0.0011) [2023-12-26 15:36:21,889][105692] Updated weights for policy 0, policy_version 32843 (0.0007) [2023-12-26 15:36:22,451][105620] Updated weights for policy 1, policy_version 33076 (0.0008) [2023-12-26 15:36:22,515][105620] Updated weights for policy 1, policy_version 33086 (0.0006) [2023-12-26 15:36:22,564][105692] Updated weights for policy 0, policy_version 32853 (0.0006) [2023-12-26 15:36:22,568][105620] Updated weights for policy 1, policy_version 33096 (0.0008) [2023-12-26 15:36:22,613][105692] Updated weights for policy 0, policy_version 32863 (0.0006) [2023-12-26 15:36:22,669][105692] Updated weights for policy 0, policy_version 32873 (0.0009) [2023-12-26 15:36:23,229][105620] Updated weights for policy 1, policy_version 33106 (0.0007) [2023-12-26 15:36:23,288][105620] Updated weights for policy 1, policy_version 33116 (0.0007) [2023-12-26 15:36:23,350][105620] Updated weights for policy 1, policy_version 33126 (0.0005) [2023-12-26 15:36:23,409][105620] Updated weights for policy 1, policy_version 33136 (0.0005) [2023-12-26 15:36:23,516][105692] Updated weights for policy 0, policy_version 32883 (0.0009) [2023-12-26 15:36:23,568][105692] Updated weights for policy 0, policy_version 32893 (0.0010) [2023-12-26 15:36:23,628][105692] Updated weights for policy 0, policy_version 32903 (0.0008) [2023-12-26 15:36:23,935][105620] Updated weights for policy 1, policy_version 33146 (0.0009) [2023-12-26 15:36:23,987][105620] Updated weights for policy 1, policy_version 33156 (0.0011) [2023-12-26 15:36:24,046][105620] Updated weights for policy 1, policy_version 33166 (0.0010) [2023-12-26 15:36:24,482][105692] Updated weights for policy 0, policy_version 32913 (0.0010) [2023-12-26 15:36:24,538][105692] Updated weights for policy 0, policy_version 32923 (0.0009) [2023-12-26 15:36:24,604][105692] Updated weights for policy 0, policy_version 32933 (0.0010) [2023-12-26 15:36:24,660][105692] Updated weights for policy 0, policy_version 32943 (0.0010) [2023-12-26 15:36:24,691][105620] Updated weights for policy 1, policy_version 33176 (0.0006) [2023-12-26 15:36:24,749][105620] Updated weights for policy 1, policy_version 33186 (0.0005) [2023-12-26 15:36:24,805][105620] Updated weights for policy 1, policy_version 33196 (0.0005) [2023-12-26 15:36:25,395][105620] Updated weights for policy 1, policy_version 33206 (0.0005) [2023-12-26 15:36:25,428][105692] Updated weights for policy 0, policy_version 32953 (0.0008) [2023-12-26 15:36:25,444][105620] Updated weights for policy 1, policy_version 33216 (0.0009) [2023-12-26 15:36:25,473][105692] Updated weights for policy 0, policy_version 32963 (0.0010) [2023-12-26 15:36:25,489][105620] Updated weights for policy 1, policy_version 33226 (0.0009) [2023-12-26 15:36:25,524][105692] Updated weights for policy 0, policy_version 32973 (0.0009) [2023-12-26 15:36:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 16957440. Throughput: 0: 9418.0, 1: 9917.0. Samples: 16968948. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-26 15:36:26,063][104569] Avg episode reward: [(0, '9254.924'), (1, '8977.427')] [2023-12-26 15:36:26,134][105620] Updated weights for policy 1, policy_version 33236 (0.0007) [2023-12-26 15:36:26,178][105620] Updated weights for policy 1, policy_version 33246 (0.0010) [2023-12-26 15:36:26,226][105620] Updated weights for policy 1, policy_version 33256 (0.0010) [2023-12-26 15:36:26,285][105692] Updated weights for policy 0, policy_version 32983 (0.0010) [2023-12-26 15:36:26,329][105692] Updated weights for policy 0, policy_version 32993 (0.0010) [2023-12-26 15:36:26,379][105692] Updated weights for policy 0, policy_version 33003 (0.0010) [2023-12-26 15:36:26,990][105620] Updated weights for policy 1, policy_version 33266 (0.0010) [2023-12-26 15:36:27,034][105620] Updated weights for policy 1, policy_version 33276 (0.0010) [2023-12-26 15:36:27,082][105620] Updated weights for policy 1, policy_version 33286 (0.0010) [2023-12-26 15:36:27,126][105620] Updated weights for policy 1, policy_version 33296 (0.0010) [2023-12-26 15:36:27,144][105692] Updated weights for policy 0, policy_version 33013 (0.0010) [2023-12-26 15:36:27,188][105692] Updated weights for policy 0, policy_version 33023 (0.0010) [2023-12-26 15:36:27,238][105692] Updated weights for policy 0, policy_version 33033 (0.0010) [2023-12-26 15:36:27,793][105620] Updated weights for policy 1, policy_version 33306 (0.0005) [2023-12-26 15:36:27,808][105692] Updated weights for policy 0, policy_version 33043 (0.0009) [2023-12-26 15:36:27,842][105620] Updated weights for policy 1, policy_version 33316 (0.0005) [2023-12-26 15:36:27,859][105692] Updated weights for policy 0, policy_version 33053 (0.0005) [2023-12-26 15:36:27,893][105620] Updated weights for policy 1, policy_version 33326 (0.0005) [2023-12-26 15:36:27,908][105692] Updated weights for policy 0, policy_version 33063 (0.0005) [2023-12-26 15:36:28,461][105620] Updated weights for policy 1, policy_version 33336 (0.0010) [2023-12-26 15:36:28,474][105692] Updated weights for policy 0, policy_version 33073 (0.0005) [2023-12-26 15:36:28,516][105620] Updated weights for policy 1, policy_version 33346 (0.0007) [2023-12-26 15:36:28,533][105692] Updated weights for policy 0, policy_version 33083 (0.0008) [2023-12-26 15:36:28,584][105620] Updated weights for policy 1, policy_version 33356 (0.0006) [2023-12-26 15:36:28,597][105692] Updated weights for policy 0, policy_version 33093 (0.0006) [2023-12-26 15:36:28,651][105692] Updated weights for policy 0, policy_version 33103 (0.0007) [2023-12-26 15:36:29,210][105620] Updated weights for policy 1, policy_version 33366 (0.0007) [2023-12-26 15:36:29,281][105620] Updated weights for policy 1, policy_version 33376 (0.0010) [2023-12-26 15:36:29,297][105692] Updated weights for policy 0, policy_version 33113 (0.0007) [2023-12-26 15:36:29,345][105620] Updated weights for policy 1, policy_version 33386 (0.0007) [2023-12-26 15:36:29,360][105692] Updated weights for policy 0, policy_version 33123 (0.0009) [2023-12-26 15:36:29,423][105692] Updated weights for policy 0, policy_version 33133 (0.0008) [2023-12-26 15:36:30,047][105692] Updated weights for policy 0, policy_version 33143 (0.0009) [2023-12-26 15:36:30,095][105620] Updated weights for policy 1, policy_version 33396 (0.0008) [2023-12-26 15:36:30,098][105692] Updated weights for policy 0, policy_version 33153 (0.0007) [2023-12-26 15:36:30,152][105692] Updated weights for policy 0, policy_version 33163 (0.0005) [2023-12-26 15:36:30,158][105620] Updated weights for policy 1, policy_version 33406 (0.0008) [2023-12-26 15:36:30,205][105620] Updated weights for policy 1, policy_version 33416 (0.0007) [2023-12-26 15:36:30,715][105692] Updated weights for policy 0, policy_version 33173 (0.0010) [2023-12-26 15:36:30,769][105692] Updated weights for policy 0, policy_version 33183 (0.0010) [2023-12-26 15:36:30,820][105692] Updated weights for policy 0, policy_version 33193 (0.0010) [2023-12-26 15:36:30,875][105620] Updated weights for policy 1, policy_version 33426 (0.0009) [2023-12-26 15:36:30,937][105620] Updated weights for policy 1, policy_version 33436 (0.0006) [2023-12-26 15:36:30,996][105620] Updated weights for policy 1, policy_version 33446 (0.0005) [2023-12-26 15:36:31,059][105620] Updated weights for policy 1, policy_version 33456 (0.0010) [2023-12-26 15:36:31,062][104569] Fps is (10 sec: 21298.6, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 17072128. Throughput: 0: 9526.9, 1: 9925.1. Samples: 17032844. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-26 15:36:31,063][104569] Avg episode reward: [(0, '9347.675'), (1, '9163.062')] [2023-12-26 15:36:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000033200_8503296.pth... [2023-12-26 15:36:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000033456_8568832.pth... [2023-12-26 15:36:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000032048_8208384.pth [2023-12-26 15:36:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000032304_8273920.pth [2023-12-26 15:36:31,505][105692] Updated weights for policy 0, policy_version 33203 (0.0009) [2023-12-26 15:36:31,569][105692] Updated weights for policy 0, policy_version 33213 (0.0005) [2023-12-26 15:36:31,639][105692] Updated weights for policy 0, policy_version 33223 (0.0007) [2023-12-26 15:36:31,693][105620] Updated weights for policy 1, policy_version 33466 (0.0007) [2023-12-26 15:36:31,766][105620] Updated weights for policy 1, policy_version 33476 (0.0008) [2023-12-26 15:36:31,825][105620] Updated weights for policy 1, policy_version 33486 (0.0010) [2023-12-26 15:36:32,338][105692] Updated weights for policy 0, policy_version 33233 (0.0010) [2023-12-26 15:36:32,406][105692] Updated weights for policy 0, policy_version 33243 (0.0011) [2023-12-26 15:36:32,468][105692] Updated weights for policy 0, policy_version 33253 (0.0011) [2023-12-26 15:36:32,518][105692] Updated weights for policy 0, policy_version 33263 (0.0011) [2023-12-26 15:36:32,559][105620] Updated weights for policy 1, policy_version 33496 (0.0011) [2023-12-26 15:36:32,623][105620] Updated weights for policy 1, policy_version 33506 (0.0011) [2023-12-26 15:36:32,681][105620] Updated weights for policy 1, policy_version 33516 (0.0011) [2023-12-26 15:36:33,233][105692] Updated weights for policy 0, policy_version 33273 (0.0006) [2023-12-26 15:36:33,272][105620] Updated weights for policy 1, policy_version 33526 (0.0007) [2023-12-26 15:36:33,288][105692] Updated weights for policy 0, policy_version 33283 (0.0007) [2023-12-26 15:36:33,332][105620] Updated weights for policy 1, policy_version 33536 (0.0005) [2023-12-26 15:36:33,342][105692] Updated weights for policy 0, policy_version 33293 (0.0006) [2023-12-26 15:36:33,400][105620] Updated weights for policy 1, policy_version 33546 (0.0009) [2023-12-26 15:36:33,982][105620] Updated weights for policy 1, policy_version 33556 (0.0010) [2023-12-26 15:36:34,013][105692] Updated weights for policy 0, policy_version 33303 (0.0006) [2023-12-26 15:36:34,039][105620] Updated weights for policy 1, policy_version 33566 (0.0006) [2023-12-26 15:36:34,075][105692] Updated weights for policy 0, policy_version 33313 (0.0009) [2023-12-26 15:36:34,093][105620] Updated weights for policy 1, policy_version 33576 (0.0008) [2023-12-26 15:36:34,141][105692] Updated weights for policy 0, policy_version 33323 (0.0008) [2023-12-26 15:36:34,825][105692] Updated weights for policy 0, policy_version 33333 (0.0008) [2023-12-26 15:36:34,827][105620] Updated weights for policy 1, policy_version 33586 (0.0008) [2023-12-26 15:36:34,879][105692] Updated weights for policy 0, policy_version 33343 (0.0005) [2023-12-26 15:36:34,880][105620] Updated weights for policy 1, policy_version 33596 (0.0008) [2023-12-26 15:36:34,932][105692] Updated weights for policy 0, policy_version 33353 (0.0005) [2023-12-26 15:36:34,942][105620] Updated weights for policy 1, policy_version 33606 (0.0007) [2023-12-26 15:36:34,996][105620] Updated weights for policy 1, policy_version 33616 (0.0007) [2023-12-26 15:36:35,473][105692] Updated weights for policy 0, policy_version 33363 (0.0006) [2023-12-26 15:36:35,534][105692] Updated weights for policy 0, policy_version 33373 (0.0009) [2023-12-26 15:36:35,591][105692] Updated weights for policy 0, policy_version 33383 (0.0008) [2023-12-26 15:36:35,708][105620] Updated weights for policy 1, policy_version 33626 (0.0009) [2023-12-26 15:36:35,765][105620] Updated weights for policy 1, policy_version 33636 (0.0008) [2023-12-26 15:36:35,821][105620] Updated weights for policy 1, policy_version 33646 (0.0005) [2023-12-26 15:36:36,062][104569] Fps is (10 sec: 21298.9, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 17170432. Throughput: 0: 9728.3, 1: 9937.2. Samples: 17156396. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-26 15:36:36,063][104569] Avg episode reward: [(0, '9347.986'), (1, '9348.625')] [2023-12-26 15:36:36,351][105692] Updated weights for policy 0, policy_version 33393 (0.0008) [2023-12-26 15:36:36,413][105692] Updated weights for policy 0, policy_version 33403 (0.0007) [2023-12-26 15:36:36,470][105620] Updated weights for policy 1, policy_version 33656 (0.0008) [2023-12-26 15:36:36,472][105692] Updated weights for policy 0, policy_version 33413 (0.0007) [2023-12-26 15:36:36,520][105620] Updated weights for policy 1, policy_version 33666 (0.0006) [2023-12-26 15:36:36,530][105692] Updated weights for policy 0, policy_version 33423 (0.0007) [2023-12-26 15:36:36,570][105620] Updated weights for policy 1, policy_version 33676 (0.0005) [2023-12-26 15:36:37,147][105692] Updated weights for policy 0, policy_version 33433 (0.0006) [2023-12-26 15:36:37,210][105692] Updated weights for policy 0, policy_version 33443 (0.0009) [2023-12-26 15:36:37,267][105692] Updated weights for policy 0, policy_version 33453 (0.0008) [2023-12-26 15:36:37,279][105620] Updated weights for policy 1, policy_version 33686 (0.0006) [2023-12-26 15:36:37,328][105620] Updated weights for policy 1, policy_version 33696 (0.0009) [2023-12-26 15:36:37,379][105620] Updated weights for policy 1, policy_version 33706 (0.0006) [2023-12-26 15:36:37,907][105692] Updated weights for policy 0, policy_version 33463 (0.0009) [2023-12-26 15:36:37,965][105692] Updated weights for policy 0, policy_version 33473 (0.0009) [2023-12-26 15:36:38,028][105692] Updated weights for policy 0, policy_version 33483 (0.0007) [2023-12-26 15:36:38,063][105620] Updated weights for policy 1, policy_version 33716 (0.0005) [2023-12-26 15:36:38,129][105620] Updated weights for policy 1, policy_version 33726 (0.0005) [2023-12-26 15:36:38,195][105620] Updated weights for policy 1, policy_version 33736 (0.0005) [2023-12-26 15:36:38,819][105692] Updated weights for policy 0, policy_version 33493 (0.0008) [2023-12-26 15:36:38,836][105620] Updated weights for policy 1, policy_version 33746 (0.0005) [2023-12-26 15:36:38,880][105692] Updated weights for policy 0, policy_version 33503 (0.0008) [2023-12-26 15:36:38,891][105620] Updated weights for policy 1, policy_version 33756 (0.0008) [2023-12-26 15:36:38,938][105692] Updated weights for policy 0, policy_version 33513 (0.0007) [2023-12-26 15:36:38,951][105620] Updated weights for policy 1, policy_version 33766 (0.0007) [2023-12-26 15:36:39,006][105620] Updated weights for policy 1, policy_version 33776 (0.0007) [2023-12-26 15:36:39,736][105692] Updated weights for policy 0, policy_version 33523 (0.0007) [2023-12-26 15:36:39,785][105692] Updated weights for policy 0, policy_version 33533 (0.0008) [2023-12-26 15:36:39,820][105620] Updated weights for policy 1, policy_version 33786 (0.0009) [2023-12-26 15:36:39,847][105692] Updated weights for policy 0, policy_version 33543 (0.0006) [2023-12-26 15:36:39,882][105620] Updated weights for policy 1, policy_version 33796 (0.0007) [2023-12-26 15:36:39,934][105620] Updated weights for policy 1, policy_version 33806 (0.0007) [2023-12-26 15:36:40,658][105692] Updated weights for policy 0, policy_version 33553 (0.0008) [2023-12-26 15:36:40,710][105620] Updated weights for policy 1, policy_version 33816 (0.0010) [2023-12-26 15:36:40,717][105692] Updated weights for policy 0, policy_version 33563 (0.0008) [2023-12-26 15:36:40,768][105620] Updated weights for policy 1, policy_version 33826 (0.0007) [2023-12-26 15:36:40,778][105692] Updated weights for policy 0, policy_version 33573 (0.0006) [2023-12-26 15:36:40,829][105620] Updated weights for policy 1, policy_version 33836 (0.0007) [2023-12-26 15:36:40,835][105692] Updated weights for policy 0, policy_version 33583 (0.0007) [2023-12-26 15:36:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 17268736. Throughput: 0: 9805.3, 1: 9900.5. Samples: 17274296. Policy #0 lag: (min: 33.0, avg: 48.0, max: 49.0) [2023-12-26 15:36:41,063][104569] Avg episode reward: [(0, '9348.070'), (1, '9257.004')] [2023-12-26 15:36:41,614][105692] Updated weights for policy 0, policy_version 33593 (0.0007) [2023-12-26 15:36:41,619][105620] Updated weights for policy 1, policy_version 33846 (0.0009) [2023-12-26 15:36:41,680][105692] Updated weights for policy 0, policy_version 33603 (0.0007) [2023-12-26 15:36:41,684][105620] Updated weights for policy 1, policy_version 33856 (0.0008) [2023-12-26 15:36:41,746][105692] Updated weights for policy 0, policy_version 33613 (0.0010) [2023-12-26 15:36:41,756][105620] Updated weights for policy 1, policy_version 33866 (0.0009) [2023-12-26 15:36:42,443][105692] Updated weights for policy 0, policy_version 33623 (0.0009) [2023-12-26 15:36:42,491][105620] Updated weights for policy 1, policy_version 33876 (0.0010) [2023-12-26 15:36:42,505][105692] Updated weights for policy 0, policy_version 33633 (0.0007) [2023-12-26 15:36:42,547][105620] Updated weights for policy 1, policy_version 33886 (0.0011) [2023-12-26 15:36:42,562][105692] Updated weights for policy 0, policy_version 33643 (0.0005) [2023-12-26 15:36:42,610][105620] Updated weights for policy 1, policy_version 33896 (0.0011) [2023-12-26 15:36:43,267][105692] Updated weights for policy 0, policy_version 33653 (0.0007) [2023-12-26 15:36:43,329][105692] Updated weights for policy 0, policy_version 33663 (0.0009) [2023-12-26 15:36:43,333][105620] Updated weights for policy 1, policy_version 33906 (0.0009) [2023-12-26 15:36:43,377][105692] Updated weights for policy 0, policy_version 33673 (0.0009) [2023-12-26 15:36:43,379][105620] Updated weights for policy 1, policy_version 33916 (0.0005) [2023-12-26 15:36:43,425][105620] Updated weights for policy 1, policy_version 33926 (0.0005) [2023-12-26 15:36:43,485][105620] Updated weights for policy 1, policy_version 33936 (0.0005) [2023-12-26 15:36:44,055][105620] Updated weights for policy 1, policy_version 33946 (0.0005) [2023-12-26 15:36:44,115][105620] Updated weights for policy 1, policy_version 33956 (0.0006) [2023-12-26 15:36:44,178][105620] Updated weights for policy 1, policy_version 33966 (0.0011) [2023-12-26 15:36:44,229][105692] Updated weights for policy 0, policy_version 33683 (0.0008) [2023-12-26 15:36:44,290][105692] Updated weights for policy 0, policy_version 33693 (0.0010) [2023-12-26 15:36:44,355][105692] Updated weights for policy 0, policy_version 33703 (0.0011) [2023-12-26 15:36:44,791][105620] Updated weights for policy 1, policy_version 33976 (0.0007) [2023-12-26 15:36:44,858][105620] Updated weights for policy 1, policy_version 33986 (0.0006) [2023-12-26 15:36:44,920][105620] Updated weights for policy 1, policy_version 33996 (0.0006) [2023-12-26 15:36:45,069][105692] Updated weights for policy 0, policy_version 33713 (0.0010) [2023-12-26 15:36:45,142][105692] Updated weights for policy 0, policy_version 33723 (0.0011) [2023-12-26 15:36:45,209][105692] Updated weights for policy 0, policy_version 33733 (0.0011) [2023-12-26 15:36:45,273][105692] Updated weights for policy 0, policy_version 33743 (0.0011) [2023-12-26 15:36:45,620][105620] Updated weights for policy 1, policy_version 34006 (0.0009) [2023-12-26 15:36:45,668][105620] Updated weights for policy 1, policy_version 34016 (0.0010) [2023-12-26 15:36:45,719][105620] Updated weights for policy 1, policy_version 34026 (0.0010) [2023-12-26 15:36:45,984][105692] Updated weights for policy 0, policy_version 33753 (0.0006) [2023-12-26 15:36:46,048][105692] Updated weights for policy 0, policy_version 33763 (0.0005) [2023-12-26 15:36:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 17358848. Throughput: 0: 9837.0, 1: 9863.5. Samples: 17331652. Policy #0 lag: (min: 33.0, avg: 48.0, max: 49.0) [2023-12-26 15:36:46,063][104569] Avg episode reward: [(0, '9348.938'), (1, '8980.871')] [2023-12-26 15:36:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000034032_8716288.pth... [2023-12-26 15:36:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000032848_8413184.pth [2023-12-26 15:36:46,100][105692] Updated weights for policy 0, policy_version 33773 (0.0007) [2023-12-26 15:36:46,122][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000033776_8650752.pth... [2023-12-26 15:36:46,126][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000032624_8355840.pth [2023-12-26 15:36:46,370][105620] Updated weights for policy 1, policy_version 34036 (0.0006) [2023-12-26 15:36:46,429][105620] Updated weights for policy 1, policy_version 34046 (0.0009) [2023-12-26 15:36:46,492][105620] Updated weights for policy 1, policy_version 34056 (0.0009) [2023-12-26 15:36:46,672][105692] Updated weights for policy 0, policy_version 33783 (0.0008) [2023-12-26 15:36:46,727][105692] Updated weights for policy 0, policy_version 33793 (0.0009) [2023-12-26 15:36:46,771][105692] Updated weights for policy 0, policy_version 33803 (0.0008) [2023-12-26 15:36:47,133][105620] Updated weights for policy 1, policy_version 34066 (0.0010) [2023-12-26 15:36:47,194][105620] Updated weights for policy 1, policy_version 34076 (0.0010) [2023-12-26 15:36:47,260][105620] Updated weights for policy 1, policy_version 34086 (0.0010) [2023-12-26 15:36:47,324][105620] Updated weights for policy 1, policy_version 34096 (0.0010) [2023-12-26 15:36:47,576][105692] Updated weights for policy 0, policy_version 33813 (0.0009) [2023-12-26 15:36:47,630][105692] Updated weights for policy 0, policy_version 33823 (0.0010) [2023-12-26 15:36:47,695][105692] Updated weights for policy 0, policy_version 33833 (0.0010) [2023-12-26 15:36:48,016][105620] Updated weights for policy 1, policy_version 34106 (0.0006) [2023-12-26 15:36:48,082][105620] Updated weights for policy 1, policy_version 34116 (0.0007) [2023-12-26 15:36:48,133][105620] Updated weights for policy 1, policy_version 34126 (0.0010) [2023-12-26 15:36:48,432][105692] Updated weights for policy 0, policy_version 33843 (0.0011) [2023-12-26 15:36:48,495][105692] Updated weights for policy 0, policy_version 33853 (0.0011) [2023-12-26 15:36:48,557][105692] Updated weights for policy 0, policy_version 33863 (0.0011) [2023-12-26 15:36:48,840][105620] Updated weights for policy 1, policy_version 34136 (0.0009) [2023-12-26 15:36:48,889][105620] Updated weights for policy 1, policy_version 34146 (0.0008) [2023-12-26 15:36:48,938][105620] Updated weights for policy 1, policy_version 34156 (0.0008) [2023-12-26 15:36:49,297][105692] Updated weights for policy 0, policy_version 33873 (0.0010) [2023-12-26 15:36:49,361][105692] Updated weights for policy 0, policy_version 33883 (0.0010) [2023-12-26 15:36:49,419][105692] Updated weights for policy 0, policy_version 33893 (0.0010) [2023-12-26 15:36:49,478][105692] Updated weights for policy 0, policy_version 33903 (0.0011) [2023-12-26 15:36:49,684][105620] Updated weights for policy 1, policy_version 34166 (0.0009) [2023-12-26 15:36:49,743][105620] Updated weights for policy 1, policy_version 34176 (0.0010) [2023-12-26 15:36:49,796][105620] Updated weights for policy 1, policy_version 34186 (0.0009) [2023-12-26 15:36:50,170][105692] Updated weights for policy 0, policy_version 33913 (0.0011) [2023-12-26 15:36:50,229][105692] Updated weights for policy 0, policy_version 33923 (0.0008) [2023-12-26 15:36:50,321][105692] Updated weights for policy 0, policy_version 33933 (0.0006) [2023-12-26 15:36:50,586][105620] Updated weights for policy 1, policy_version 34196 (0.0009) [2023-12-26 15:36:50,654][105620] Updated weights for policy 1, policy_version 34206 (0.0008) [2023-12-26 15:36:50,713][105620] Updated weights for policy 1, policy_version 34216 (0.0008) [2023-12-26 15:36:50,868][105692] Updated weights for policy 0, policy_version 33943 (0.0009) [2023-12-26 15:36:50,931][105692] Updated weights for policy 0, policy_version 33953 (0.0011) [2023-12-26 15:36:50,986][105692] Updated weights for policy 0, policy_version 33963 (0.0011) [2023-12-26 15:36:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 17465344. Throughput: 0: 9849.9, 1: 9928.2. Samples: 17448996. Policy #0 lag: (min: 33.0, avg: 48.0, max: 49.0) [2023-12-26 15:36:51,063][104569] Avg episode reward: [(0, '9349.966'), (1, '8745.543')] [2023-12-26 15:36:51,462][105620] Updated weights for policy 1, policy_version 34226 (0.0008) [2023-12-26 15:36:51,521][105620] Updated weights for policy 1, policy_version 34236 (0.0010) [2023-12-26 15:36:51,570][105620] Updated weights for policy 1, policy_version 34246 (0.0010) [2023-12-26 15:36:51,628][105620] Updated weights for policy 1, policy_version 34256 (0.0011) [2023-12-26 15:36:51,641][105692] Updated weights for policy 0, policy_version 33973 (0.0011) [2023-12-26 15:36:51,693][105692] Updated weights for policy 0, policy_version 33983 (0.0010) [2023-12-26 15:36:51,758][105692] Updated weights for policy 0, policy_version 33993 (0.0011) [2023-12-26 15:36:52,413][105620] Updated weights for policy 1, policy_version 34266 (0.0008) [2023-12-26 15:36:52,472][105620] Updated weights for policy 1, policy_version 34276 (0.0008) [2023-12-26 15:36:52,491][105692] Updated weights for policy 0, policy_version 34003 (0.0010) [2023-12-26 15:36:52,521][105620] Updated weights for policy 1, policy_version 34286 (0.0005) [2023-12-26 15:36:52,535][105692] Updated weights for policy 0, policy_version 34013 (0.0010) [2023-12-26 15:36:52,591][105692] Updated weights for policy 0, policy_version 34023 (0.0010) [2023-12-26 15:36:53,248][105620] Updated weights for policy 1, policy_version 34296 (0.0008) [2023-12-26 15:36:53,299][105692] Updated weights for policy 0, policy_version 34033 (0.0010) [2023-12-26 15:36:53,301][105620] Updated weights for policy 1, policy_version 34306 (0.0008) [2023-12-26 15:36:53,348][105692] Updated weights for policy 0, policy_version 34043 (0.0007) [2023-12-26 15:36:53,355][105620] Updated weights for policy 1, policy_version 34316 (0.0006) [2023-12-26 15:36:53,396][105692] Updated weights for policy 0, policy_version 34053 (0.0007) [2023-12-26 15:36:53,441][105692] Updated weights for policy 0, policy_version 34063 (0.0008) [2023-12-26 15:36:54,111][105692] Updated weights for policy 0, policy_version 34073 (0.0009) [2023-12-26 15:36:54,168][105692] Updated weights for policy 0, policy_version 34083 (0.0009) [2023-12-26 15:36:54,203][105620] Updated weights for policy 1, policy_version 34326 (0.0006) [2023-12-26 15:36:54,230][105692] Updated weights for policy 0, policy_version 34093 (0.0007) [2023-12-26 15:36:54,260][105620] Updated weights for policy 1, policy_version 34336 (0.0007) [2023-12-26 15:36:54,320][105620] Updated weights for policy 1, policy_version 34346 (0.0009) [2023-12-26 15:36:54,951][105692] Updated weights for policy 0, policy_version 34103 (0.0008) [2023-12-26 15:36:55,006][105692] Updated weights for policy 0, policy_version 34113 (0.0006) [2023-12-26 15:36:55,062][105692] Updated weights for policy 0, policy_version 34123 (0.0009) [2023-12-26 15:36:55,081][105620] Updated weights for policy 1, policy_version 34356 (0.0008) [2023-12-26 15:36:55,142][105620] Updated weights for policy 1, policy_version 34366 (0.0007) [2023-12-26 15:36:55,205][105620] Updated weights for policy 1, policy_version 34376 (0.0007) [2023-12-26 15:36:55,659][105692] Updated weights for policy 0, policy_version 34133 (0.0009) [2023-12-26 15:36:55,714][105692] Updated weights for policy 0, policy_version 34143 (0.0009) [2023-12-26 15:36:55,765][105692] Updated weights for policy 0, policy_version 34153 (0.0009) [2023-12-26 15:36:55,973][105620] Updated weights for policy 1, policy_version 34386 (0.0009) [2023-12-26 15:36:56,030][105620] Updated weights for policy 1, policy_version 34396 (0.0009) [2023-12-26 15:36:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 17555456. Throughput: 0: 9934.9, 1: 9938.8. Samples: 17566652. Policy #0 lag: (min: 33.0, avg: 48.0, max: 49.0) [2023-12-26 15:36:56,062][104569] Avg episode reward: [(0, '9349.945'), (1, '8847.585')] [2023-12-26 15:36:56,089][105620] Updated weights for policy 1, policy_version 34406 (0.0008) [2023-12-26 15:36:56,156][105620] Updated weights for policy 1, policy_version 34416 (0.0009) [2023-12-26 15:36:56,451][105692] Updated weights for policy 0, policy_version 34163 (0.0008) [2023-12-26 15:36:56,508][105692] Updated weights for policy 0, policy_version 34173 (0.0005) [2023-12-26 15:36:56,569][105692] Updated weights for policy 0, policy_version 34183 (0.0005) [2023-12-26 15:36:56,981][105620] Updated weights for policy 1, policy_version 34426 (0.0009) [2023-12-26 15:36:57,027][105620] Updated weights for policy 1, policy_version 34436 (0.0009) [2023-12-26 15:36:57,076][105620] Updated weights for policy 1, policy_version 34446 (0.0008) [2023-12-26 15:36:57,203][105692] Updated weights for policy 0, policy_version 34193 (0.0006) [2023-12-26 15:36:57,264][105692] Updated weights for policy 0, policy_version 34203 (0.0009) [2023-12-26 15:36:57,316][105692] Updated weights for policy 0, policy_version 34213 (0.0008) [2023-12-26 15:36:57,376][105692] Updated weights for policy 0, policy_version 34223 (0.0010) [2023-12-26 15:36:57,836][105620] Updated weights for policy 1, policy_version 34456 (0.0009) [2023-12-26 15:36:57,893][105620] Updated weights for policy 1, policy_version 34466 (0.0009) [2023-12-26 15:36:57,952][105620] Updated weights for policy 1, policy_version 34476 (0.0008) [2023-12-26 15:36:58,039][105692] Updated weights for policy 0, policy_version 34233 (0.0009) [2023-12-26 15:36:58,095][105692] Updated weights for policy 0, policy_version 34243 (0.0007) [2023-12-26 15:36:58,154][105692] Updated weights for policy 0, policy_version 34253 (0.0008) [2023-12-26 15:36:58,751][105620] Updated weights for policy 1, policy_version 34486 (0.0010) [2023-12-26 15:36:58,826][105620] Updated weights for policy 1, policy_version 34496 (0.0009) [2023-12-26 15:36:58,895][105620] Updated weights for policy 1, policy_version 34506 (0.0009) [2023-12-26 15:36:58,918][105692] Updated weights for policy 0, policy_version 34263 (0.0007) [2023-12-26 15:36:58,984][105692] Updated weights for policy 0, policy_version 34273 (0.0010) [2023-12-26 15:36:59,047][105692] Updated weights for policy 0, policy_version 34283 (0.0011) [2023-12-26 15:36:59,687][105620] Updated weights for policy 1, policy_version 34516 (0.0009) [2023-12-26 15:36:59,752][105620] Updated weights for policy 1, policy_version 34526 (0.0006) [2023-12-26 15:36:59,762][105692] Updated weights for policy 0, policy_version 34293 (0.0010) [2023-12-26 15:36:59,819][105620] Updated weights for policy 1, policy_version 34536 (0.0008) [2023-12-26 15:36:59,820][105692] Updated weights for policy 0, policy_version 34303 (0.0011) [2023-12-26 15:36:59,880][105692] Updated weights for policy 0, policy_version 34313 (0.0011) [2023-12-26 15:37:00,401][105620] Updated weights for policy 1, policy_version 34546 (0.0007) [2023-12-26 15:37:00,448][105620] Updated weights for policy 1, policy_version 34556 (0.0009) [2023-12-26 15:37:00,482][105692] Updated weights for policy 0, policy_version 34323 (0.0010) [2023-12-26 15:37:00,496][105620] Updated weights for policy 1, policy_version 34566 (0.0010) [2023-12-26 15:37:00,539][105620] Updated weights for policy 1, policy_version 34576 (0.0010) [2023-12-26 15:37:00,546][105692] Updated weights for policy 0, policy_version 34333 (0.0007) [2023-12-26 15:37:00,612][105692] Updated weights for policy 0, policy_version 34343 (0.0007) [2023-12-26 15:37:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 17653760. Throughput: 0: 9935.0, 1: 9869.3. Samples: 17622976. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 15:37:01,062][104569] Avg episode reward: [(0, '9349.070'), (1, '9185.080')] [2023-12-26 15:37:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000034352_8798208.pth... [2023-12-26 15:37:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000034576_8855552.pth... [2023-12-26 15:37:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000033200_8503296.pth [2023-12-26 15:37:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000033456_8568832.pth [2023-12-26 15:37:01,171][105620] Updated weights for policy 1, policy_version 34586 (0.0011) [2023-12-26 15:37:01,224][105620] Updated weights for policy 1, policy_version 34596 (0.0010) [2023-12-26 15:37:01,242][105692] Updated weights for policy 0, policy_version 34353 (0.0006) [2023-12-26 15:37:01,285][105620] Updated weights for policy 1, policy_version 34606 (0.0011) [2023-12-26 15:37:01,300][105692] Updated weights for policy 0, policy_version 34363 (0.0006) [2023-12-26 15:37:01,364][105692] Updated weights for policy 0, policy_version 34373 (0.0008) [2023-12-26 15:37:01,425][105692] Updated weights for policy 0, policy_version 34383 (0.0008) [2023-12-26 15:37:01,909][105620] Updated weights for policy 1, policy_version 34616 (0.0006) [2023-12-26 15:37:01,974][105620] Updated weights for policy 1, policy_version 34626 (0.0005) [2023-12-26 15:37:02,042][105620] Updated weights for policy 1, policy_version 34636 (0.0006) [2023-12-26 15:37:02,222][105692] Updated weights for policy 0, policy_version 34394 (0.0011) [2023-12-26 15:37:02,280][105692] Updated weights for policy 0, policy_version 34404 (0.0008) [2023-12-26 15:37:02,345][105692] Updated weights for policy 0, policy_version 34414 (0.0007) [2023-12-26 15:37:02,587][105620] Updated weights for policy 1, policy_version 34646 (0.0009) [2023-12-26 15:37:02,643][105620] Updated weights for policy 1, policy_version 34656 (0.0010) [2023-12-26 15:37:02,710][105620] Updated weights for policy 1, policy_version 34666 (0.0007) [2023-12-26 15:37:03,150][105692] Updated weights for policy 0, policy_version 34424 (0.0008) [2023-12-26 15:37:03,218][105692] Updated weights for policy 0, policy_version 34434 (0.0009) [2023-12-26 15:37:03,274][105692] Updated weights for policy 0, policy_version 34444 (0.0007) [2023-12-26 15:37:03,282][105620] Updated weights for policy 1, policy_version 34676 (0.0006) [2023-12-26 15:37:03,348][105620] Updated weights for policy 1, policy_version 34686 (0.0007) [2023-12-26 15:37:03,422][105620] Updated weights for policy 1, policy_version 34696 (0.0006) [2023-12-26 15:37:04,050][105620] Updated weights for policy 1, policy_version 34706 (0.0008) [2023-12-26 15:37:04,063][105692] Updated weights for policy 0, policy_version 34454 (0.0006) [2023-12-26 15:37:04,106][105620] Updated weights for policy 1, policy_version 34716 (0.0007) [2023-12-26 15:37:04,127][105692] Updated weights for policy 0, policy_version 34464 (0.0009) [2023-12-26 15:37:04,171][105620] Updated weights for policy 1, policy_version 34726 (0.0006) [2023-12-26 15:37:04,195][105692] Updated weights for policy 0, policy_version 34474 (0.0010) [2023-12-26 15:37:04,235][105620] Updated weights for policy 1, policy_version 34736 (0.0006) [2023-12-26 15:37:04,886][105620] Updated weights for policy 1, policy_version 34746 (0.0010) [2023-12-26 15:37:04,941][105692] Updated weights for policy 0, policy_version 34484 (0.0009) [2023-12-26 15:37:04,949][105620] Updated weights for policy 1, policy_version 34756 (0.0009) [2023-12-26 15:37:04,991][105692] Updated weights for policy 0, policy_version 34494 (0.0009) [2023-12-26 15:37:05,004][105620] Updated weights for policy 1, policy_version 34766 (0.0007) [2023-12-26 15:37:05,046][105692] Updated weights for policy 0, policy_version 34504 (0.0009) [2023-12-26 15:37:05,709][105620] Updated weights for policy 1, policy_version 34776 (0.0009) [2023-12-26 15:37:05,773][105620] Updated weights for policy 1, policy_version 34786 (0.0010) [2023-12-26 15:37:05,783][105692] Updated weights for policy 0, policy_version 34514 (0.0009) [2023-12-26 15:37:05,821][105620] Updated weights for policy 1, policy_version 34796 (0.0010) [2023-12-26 15:37:05,842][105692] Updated weights for policy 0, policy_version 34524 (0.0006) [2023-12-26 15:37:05,894][105692] Updated weights for policy 0, policy_version 34534 (0.0007) [2023-12-26 15:37:05,942][105692] Updated weights for policy 0, policy_version 34544 (0.0007) [2023-12-26 15:37:06,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 17760256. Throughput: 0: 9877.5, 1: 10003.3. Samples: 17744796. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 15:37:06,063][104569] Avg episode reward: [(0, '9255.746'), (1, '8998.851')] [2023-12-26 15:37:06,540][105620] Updated weights for policy 1, policy_version 34806 (0.0009) [2023-12-26 15:37:06,599][105620] Updated weights for policy 1, policy_version 34816 (0.0009) [2023-12-26 15:37:06,661][105620] Updated weights for policy 1, policy_version 34826 (0.0009) [2023-12-26 15:37:06,733][105692] Updated weights for policy 0, policy_version 34554 (0.0009) [2023-12-26 15:37:06,786][105692] Updated weights for policy 0, policy_version 34564 (0.0009) [2023-12-26 15:37:06,848][105692] Updated weights for policy 0, policy_version 34574 (0.0007) [2023-12-26 15:37:07,311][105620] Updated weights for policy 1, policy_version 34836 (0.0005) [2023-12-26 15:37:07,370][105620] Updated weights for policy 1, policy_version 34846 (0.0005) [2023-12-26 15:37:07,432][105620] Updated weights for policy 1, policy_version 34856 (0.0005) [2023-12-26 15:37:07,672][105692] Updated weights for policy 0, policy_version 34584 (0.0008) [2023-12-26 15:37:07,718][105692] Updated weights for policy 0, policy_version 34594 (0.0008) [2023-12-26 15:37:07,761][105692] Updated weights for policy 0, policy_version 34604 (0.0007) [2023-12-26 15:37:08,037][105620] Updated weights for policy 1, policy_version 34866 (0.0005) [2023-12-26 15:37:08,096][105620] Updated weights for policy 1, policy_version 34876 (0.0005) [2023-12-26 15:37:08,142][105620] Updated weights for policy 1, policy_version 34886 (0.0005) [2023-12-26 15:37:08,193][105620] Updated weights for policy 1, policy_version 34896 (0.0005) [2023-12-26 15:37:08,492][105692] Updated weights for policy 0, policy_version 34614 (0.0007) [2023-12-26 15:37:08,564][105692] Updated weights for policy 0, policy_version 34624 (0.0005) [2023-12-26 15:37:08,629][105692] Updated weights for policy 0, policy_version 34634 (0.0006) [2023-12-26 15:37:08,780][105620] Updated weights for policy 1, policy_version 34906 (0.0009) [2023-12-26 15:37:08,842][105620] Updated weights for policy 1, policy_version 34916 (0.0009) [2023-12-26 15:37:08,912][105620] Updated weights for policy 1, policy_version 34926 (0.0009) [2023-12-26 15:37:09,150][105692] Updated weights for policy 0, policy_version 34644 (0.0006) [2023-12-26 15:37:09,204][105692] Updated weights for policy 0, policy_version 34654 (0.0008) [2023-12-26 15:37:09,268][105692] Updated weights for policy 0, policy_version 34664 (0.0008) [2023-12-26 15:37:09,727][105620] Updated weights for policy 1, policy_version 34936 (0.0009) [2023-12-26 15:37:09,790][105620] Updated weights for policy 1, policy_version 34946 (0.0009) [2023-12-26 15:37:09,864][105620] Updated weights for policy 1, policy_version 34956 (0.0010) [2023-12-26 15:37:09,971][105692] Updated weights for policy 0, policy_version 34674 (0.0006) [2023-12-26 15:37:10,039][105692] Updated weights for policy 0, policy_version 34684 (0.0008) [2023-12-26 15:37:10,097][105692] Updated weights for policy 0, policy_version 34694 (0.0009) [2023-12-26 15:37:10,163][105692] Updated weights for policy 0, policy_version 34704 (0.0009) [2023-12-26 15:37:10,597][105620] Updated weights for policy 1, policy_version 34966 (0.0009) [2023-12-26 15:37:10,659][105620] Updated weights for policy 1, policy_version 34976 (0.0010) [2023-12-26 15:37:10,729][105620] Updated weights for policy 1, policy_version 34986 (0.0009) [2023-12-26 15:37:10,878][105692] Updated weights for policy 0, policy_version 34714 (0.0007) [2023-12-26 15:37:10,937][105692] Updated weights for policy 0, policy_version 34724 (0.0010) [2023-12-26 15:37:11,003][105692] Updated weights for policy 0, policy_version 34734 (0.0010) [2023-12-26 15:37:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 17858560. Throughput: 0: 9905.7, 1: 9940.7. Samples: 17862036. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 15:37:11,063][104569] Avg episode reward: [(0, '9162.739'), (1, '8829.664')] [2023-12-26 15:37:11,465][105620] Updated weights for policy 1, policy_version 34996 (0.0008) [2023-12-26 15:37:11,527][105620] Updated weights for policy 1, policy_version 35006 (0.0008) [2023-12-26 15:37:11,574][105620] Updated weights for policy 1, policy_version 35016 (0.0008) [2023-12-26 15:37:11,794][105692] Updated weights for policy 0, policy_version 34744 (0.0011) [2023-12-26 15:37:11,856][105692] Updated weights for policy 0, policy_version 34754 (0.0010) [2023-12-26 15:37:11,922][105692] Updated weights for policy 0, policy_version 34764 (0.0007) [2023-12-26 15:37:12,379][105620] Updated weights for policy 1, policy_version 35026 (0.0009) [2023-12-26 15:37:12,442][105620] Updated weights for policy 1, policy_version 35036 (0.0008) [2023-12-26 15:37:12,507][105620] Updated weights for policy 1, policy_version 35046 (0.0009) [2023-12-26 15:37:12,569][105620] Updated weights for policy 1, policy_version 35056 (0.0009) [2023-12-26 15:37:12,649][105692] Updated weights for policy 0, policy_version 34774 (0.0010) [2023-12-26 15:37:12,697][105692] Updated weights for policy 0, policy_version 34784 (0.0009) [2023-12-26 15:37:12,743][105692] Updated weights for policy 0, policy_version 34794 (0.0008) [2023-12-26 15:37:13,297][105620] Updated weights for policy 1, policy_version 35066 (0.0009) [2023-12-26 15:37:13,351][105620] Updated weights for policy 1, policy_version 35076 (0.0009) [2023-12-26 15:37:13,401][105620] Updated weights for policy 1, policy_version 35086 (0.0009) [2023-12-26 15:37:13,417][105692] Updated weights for policy 0, policy_version 34804 (0.0006) [2023-12-26 15:37:13,469][105692] Updated weights for policy 0, policy_version 34814 (0.0009) [2023-12-26 15:37:13,514][105692] Updated weights for policy 0, policy_version 34824 (0.0010) [2023-12-26 15:37:14,063][105620] Updated weights for policy 1, policy_version 35096 (0.0008) [2023-12-26 15:37:14,121][105620] Updated weights for policy 1, policy_version 35106 (0.0009) [2023-12-26 15:37:14,174][105692] Updated weights for policy 0, policy_version 34834 (0.0008) [2023-12-26 15:37:14,178][105620] Updated weights for policy 1, policy_version 35116 (0.0010) [2023-12-26 15:37:14,221][105692] Updated weights for policy 0, policy_version 34844 (0.0006) [2023-12-26 15:37:14,270][105692] Updated weights for policy 0, policy_version 34854 (0.0005) [2023-12-26 15:37:14,313][105692] Updated weights for policy 0, policy_version 34864 (0.0005) [2023-12-26 15:37:14,925][105692] Updated weights for policy 0, policy_version 34874 (0.0010) [2023-12-26 15:37:14,984][105692] Updated weights for policy 0, policy_version 34884 (0.0010) [2023-12-26 15:37:15,021][105620] Updated weights for policy 1, policy_version 35126 (0.0010) [2023-12-26 15:37:15,053][105692] Updated weights for policy 0, policy_version 34894 (0.0011) [2023-12-26 15:37:15,073][105620] Updated weights for policy 1, policy_version 35136 (0.0010) [2023-12-26 15:37:15,130][105620] Updated weights for policy 1, policy_version 35146 (0.0011) [2023-12-26 15:37:15,771][105620] Updated weights for policy 1, policy_version 35156 (0.0008) [2023-12-26 15:37:15,801][105692] Updated weights for policy 0, policy_version 34904 (0.0010) [2023-12-26 15:37:15,822][105620] Updated weights for policy 1, policy_version 35166 (0.0005) [2023-12-26 15:37:15,856][105692] Updated weights for policy 0, policy_version 34914 (0.0010) [2023-12-26 15:37:15,878][105620] Updated weights for policy 1, policy_version 35176 (0.0006) [2023-12-26 15:37:15,911][105692] Updated weights for policy 0, policy_version 34924 (0.0010) [2023-12-26 15:37:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 17956864. Throughput: 0: 9839.6, 1: 9878.3. Samples: 17920144. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 15:37:16,062][104569] Avg episode reward: [(0, '9253.432'), (1, '8997.747')] [2023-12-26 15:37:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000034928_8945664.pth... [2023-12-26 15:37:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000035184_9011200.pth... [2023-12-26 15:37:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000033776_8650752.pth [2023-12-26 15:37:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000034032_8716288.pth [2023-12-26 15:37:16,538][105620] Updated weights for policy 1, policy_version 35186 (0.0006) [2023-12-26 15:37:16,589][105620] Updated weights for policy 1, policy_version 35196 (0.0008) [2023-12-26 15:37:16,636][105620] Updated weights for policy 1, policy_version 35206 (0.0007) [2023-12-26 15:37:16,650][105692] Updated weights for policy 0, policy_version 34934 (0.0010) [2023-12-26 15:37:16,691][105620] Updated weights for policy 1, policy_version 35216 (0.0005) [2023-12-26 15:37:16,704][105692] Updated weights for policy 0, policy_version 34944 (0.0010) [2023-12-26 15:37:16,765][105692] Updated weights for policy 0, policy_version 34954 (0.0010) [2023-12-26 15:37:17,447][105620] Updated weights for policy 1, policy_version 35226 (0.0008) [2023-12-26 15:37:17,495][105620] Updated weights for policy 1, policy_version 35236 (0.0008) [2023-12-26 15:37:17,512][105692] Updated weights for policy 0, policy_version 34964 (0.0010) [2023-12-26 15:37:17,543][105620] Updated weights for policy 1, policy_version 35246 (0.0010) [2023-12-26 15:37:17,563][105692] Updated weights for policy 0, policy_version 34974 (0.0010) [2023-12-26 15:37:17,621][105692] Updated weights for policy 0, policy_version 34984 (0.0010) [2023-12-26 15:37:18,303][105692] Updated weights for policy 0, policy_version 34994 (0.0009) [2023-12-26 15:37:18,367][105692] Updated weights for policy 0, policy_version 35004 (0.0009) [2023-12-26 15:37:18,371][105620] Updated weights for policy 1, policy_version 35256 (0.0008) [2023-12-26 15:37:18,417][105620] Updated weights for policy 1, policy_version 35266 (0.0007) [2023-12-26 15:37:18,419][105692] Updated weights for policy 0, policy_version 35014 (0.0010) [2023-12-26 15:37:18,469][105620] Updated weights for policy 1, policy_version 35276 (0.0007) [2023-12-26 15:37:18,474][105692] Updated weights for policy 0, policy_version 35024 (0.0007) [2023-12-26 15:37:19,003][105692] Updated weights for policy 0, policy_version 35034 (0.0010) [2023-12-26 15:37:19,058][105692] Updated weights for policy 0, policy_version 35044 (0.0010) [2023-12-26 15:37:19,102][105692] Updated weights for policy 0, policy_version 35054 (0.0010) [2023-12-26 15:37:19,356][105620] Updated weights for policy 1, policy_version 35286 (0.0009) [2023-12-26 15:37:19,419][105620] Updated weights for policy 1, policy_version 35296 (0.0008) [2023-12-26 15:37:19,481][105620] Updated weights for policy 1, policy_version 35306 (0.0008) [2023-12-26 15:37:19,869][105692] Updated weights for policy 0, policy_version 35064 (0.0008) [2023-12-26 15:37:19,937][105692] Updated weights for policy 0, policy_version 35074 (0.0011) [2023-12-26 15:37:19,997][105692] Updated weights for policy 0, policy_version 35084 (0.0011) [2023-12-26 15:37:20,288][105620] Updated weights for policy 1, policy_version 35316 (0.0009) [2023-12-26 15:37:20,337][105620] Updated weights for policy 1, policy_version 35326 (0.0008) [2023-12-26 15:37:20,386][105620] Updated weights for policy 1, policy_version 35336 (0.0006) [2023-12-26 15:37:20,655][105692] Updated weights for policy 0, policy_version 35094 (0.0011) [2023-12-26 15:37:20,719][105692] Updated weights for policy 0, policy_version 35104 (0.0011) [2023-12-26 15:37:20,780][105692] Updated weights for policy 0, policy_version 35114 (0.0011) [2023-12-26 15:37:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 18046976. Throughput: 0: 9812.4, 1: 9748.9. Samples: 18036652. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 15:37:21,063][104569] Avg episode reward: [(0, '9346.367'), (1, '9090.237')] [2023-12-26 15:37:21,129][105620] Updated weights for policy 1, policy_version 35346 (0.0006) [2023-12-26 15:37:21,182][105620] Updated weights for policy 1, policy_version 35356 (0.0008) [2023-12-26 15:37:21,239][105620] Updated weights for policy 1, policy_version 35366 (0.0008) [2023-12-26 15:37:21,306][105620] Updated weights for policy 1, policy_version 35376 (0.0009) [2023-12-26 15:37:21,555][105692] Updated weights for policy 0, policy_version 35124 (0.0011) [2023-12-26 15:37:21,619][105692] Updated weights for policy 0, policy_version 35134 (0.0010) [2023-12-26 15:37:21,683][105692] Updated weights for policy 0, policy_version 35144 (0.0010) [2023-12-26 15:37:22,063][105620] Updated weights for policy 1, policy_version 35386 (0.0008) [2023-12-26 15:37:22,125][105620] Updated weights for policy 1, policy_version 35396 (0.0008) [2023-12-26 15:37:22,182][105620] Updated weights for policy 1, policy_version 35406 (0.0006) [2023-12-26 15:37:22,326][105692] Updated weights for policy 0, policy_version 35154 (0.0009) [2023-12-26 15:37:22,390][105692] Updated weights for policy 0, policy_version 35164 (0.0009) [2023-12-26 15:37:22,451][105692] Updated weights for policy 0, policy_version 35174 (0.0008) [2023-12-26 15:37:22,515][105692] Updated weights for policy 0, policy_version 35184 (0.0008) [2023-12-26 15:37:22,867][105620] Updated weights for policy 1, policy_version 35416 (0.0007) [2023-12-26 15:37:22,922][105620] Updated weights for policy 1, policy_version 35426 (0.0008) [2023-12-26 15:37:22,977][105620] Updated weights for policy 1, policy_version 35436 (0.0009) [2023-12-26 15:37:23,320][105692] Updated weights for policy 0, policy_version 35194 (0.0007) [2023-12-26 15:37:23,382][105692] Updated weights for policy 0, policy_version 35204 (0.0005) [2023-12-26 15:37:23,441][105692] Updated weights for policy 0, policy_version 35214 (0.0006) [2023-12-26 15:37:23,688][105620] Updated weights for policy 1, policy_version 35446 (0.0007) [2023-12-26 15:37:23,742][105620] Updated weights for policy 1, policy_version 35456 (0.0007) [2023-12-26 15:37:23,793][105620] Updated weights for policy 1, policy_version 35466 (0.0010) [2023-12-26 15:37:24,174][105692] Updated weights for policy 0, policy_version 35225 (0.0011) [2023-12-26 15:37:24,229][105692] Updated weights for policy 0, policy_version 35236 (0.0010) [2023-12-26 15:37:24,282][105692] Updated weights for policy 0, policy_version 35247 (0.0010) [2023-12-26 15:37:24,370][105620] Updated weights for policy 1, policy_version 35476 (0.0007) [2023-12-26 15:37:24,431][105620] Updated weights for policy 1, policy_version 35486 (0.0005) [2023-12-26 15:37:24,488][105620] Updated weights for policy 1, policy_version 35496 (0.0010) [2023-12-26 15:37:25,089][105692] Updated weights for policy 0, policy_version 35257 (0.0010) [2023-12-26 15:37:25,115][105620] Updated weights for policy 1, policy_version 35506 (0.0008) [2023-12-26 15:37:25,149][105692] Updated weights for policy 0, policy_version 35267 (0.0011) [2023-12-26 15:37:25,177][105620] Updated weights for policy 1, policy_version 35516 (0.0007) [2023-12-26 15:37:25,204][105692] Updated weights for policy 0, policy_version 35277 (0.0011) [2023-12-26 15:37:25,235][105620] Updated weights for policy 1, policy_version 35526 (0.0010) [2023-12-26 15:37:25,303][105620] Updated weights for policy 1, policy_version 35536 (0.0010) [2023-12-26 15:37:25,921][105692] Updated weights for policy 0, policy_version 35287 (0.0010) [2023-12-26 15:37:25,969][105692] Updated weights for policy 0, policy_version 35297 (0.0010) [2023-12-26 15:37:25,996][105620] Updated weights for policy 1, policy_version 35546 (0.0010) [2023-12-26 15:37:26,028][105692] Updated weights for policy 0, policy_version 35307 (0.0010) [2023-12-26 15:37:26,051][105620] Updated weights for policy 1, policy_version 35556 (0.0010) [2023-12-26 15:37:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 18145280. Throughput: 0: 9769.4, 1: 9770.3. Samples: 18153584. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-26 15:37:26,062][104569] Avg episode reward: [(0, '9348.254'), (1, '9351.734')] [2023-12-26 15:37:26,108][105620] Updated weights for policy 1, policy_version 35566 (0.0011) [2023-12-26 15:37:26,116][105586] Saving new best policy, reward=9351.734! [2023-12-26 15:37:26,726][105620] Updated weights for policy 1, policy_version 35576 (0.0010) [2023-12-26 15:37:26,781][105692] Updated weights for policy 0, policy_version 35317 (0.0011) [2023-12-26 15:37:26,784][105620] Updated weights for policy 1, policy_version 35586 (0.0008) [2023-12-26 15:37:26,825][105692] Updated weights for policy 0, policy_version 35327 (0.0010) [2023-12-26 15:37:26,834][105620] Updated weights for policy 1, policy_version 35596 (0.0009) [2023-12-26 15:37:26,870][105692] Updated weights for policy 0, policy_version 35337 (0.0010) [2023-12-26 15:37:27,543][105620] Updated weights for policy 1, policy_version 35606 (0.0007) [2023-12-26 15:37:27,593][105620] Updated weights for policy 1, policy_version 35616 (0.0005) [2023-12-26 15:37:27,625][105692] Updated weights for policy 0, policy_version 35347 (0.0010) [2023-12-26 15:37:27,648][105620] Updated weights for policy 1, policy_version 35626 (0.0006) [2023-12-26 15:37:27,681][105692] Updated weights for policy 0, policy_version 35357 (0.0011) [2023-12-26 15:37:27,733][105692] Updated weights for policy 0, policy_version 35367 (0.0010) [2023-12-26 15:37:28,202][105620] Updated weights for policy 1, policy_version 35636 (0.0008) [2023-12-26 15:37:28,246][105620] Updated weights for policy 1, policy_version 35646 (0.0005) [2023-12-26 15:37:28,296][105620] Updated weights for policy 1, policy_version 35656 (0.0005) [2023-12-26 15:37:28,431][105692] Updated weights for policy 0, policy_version 35377 (0.0010) [2023-12-26 15:37:28,480][105692] Updated weights for policy 0, policy_version 35387 (0.0005) [2023-12-26 15:37:28,524][105692] Updated weights for policy 0, policy_version 35397 (0.0005) [2023-12-26 15:37:28,582][105692] Updated weights for policy 0, policy_version 35407 (0.0006) [2023-12-26 15:37:29,058][105620] Updated weights for policy 1, policy_version 35666 (0.0006) [2023-12-26 15:37:29,118][105620] Updated weights for policy 1, policy_version 35676 (0.0005) [2023-12-26 15:37:29,169][105620] Updated weights for policy 1, policy_version 35686 (0.0005) [2023-12-26 15:37:29,215][105620] Updated weights for policy 1, policy_version 35696 (0.0005) [2023-12-26 15:37:29,283][105692] Updated weights for policy 0, policy_version 35417 (0.0007) [2023-12-26 15:37:29,346][105692] Updated weights for policy 0, policy_version 35427 (0.0007) [2023-12-26 15:37:29,416][105692] Updated weights for policy 0, policy_version 35437 (0.0011) [2023-12-26 15:37:29,842][105620] Updated weights for policy 1, policy_version 35706 (0.0008) [2023-12-26 15:37:29,906][105620] Updated weights for policy 1, policy_version 35716 (0.0008) [2023-12-26 15:37:29,972][105620] Updated weights for policy 1, policy_version 35726 (0.0007) [2023-12-26 15:37:30,078][105692] Updated weights for policy 0, policy_version 35447 (0.0011) [2023-12-26 15:37:30,130][105692] Updated weights for policy 0, policy_version 35457 (0.0010) [2023-12-26 15:37:30,179][105692] Updated weights for policy 0, policy_version 35467 (0.0010) [2023-12-26 15:37:30,632][105620] Updated weights for policy 1, policy_version 35736 (0.0008) [2023-12-26 15:37:30,681][105620] Updated weights for policy 1, policy_version 35746 (0.0007) [2023-12-26 15:37:30,732][105620] Updated weights for policy 1, policy_version 35756 (0.0005) [2023-12-26 15:37:30,906][105692] Updated weights for policy 0, policy_version 35477 (0.0010) [2023-12-26 15:37:30,971][105692] Updated weights for policy 0, policy_version 35487 (0.0010) [2023-12-26 15:37:31,028][105692] Updated weights for policy 0, policy_version 35497 (0.0010) [2023-12-26 15:37:31,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 18243584. Throughput: 0: 9798.6, 1: 9818.7. Samples: 18214436. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-26 15:37:31,063][104569] Avg episode reward: [(0, '9257.553'), (1, '8734.684')] [2023-12-26 15:37:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000035760_9158656.pth... [2023-12-26 15:37:31,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000035504_9093120.pth... [2023-12-26 15:37:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000034576_8855552.pth [2023-12-26 15:37:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000034352_8798208.pth [2023-12-26 15:37:31,396][105620] Updated weights for policy 1, policy_version 35766 (0.0008) [2023-12-26 15:37:31,457][105620] Updated weights for policy 1, policy_version 35776 (0.0010) [2023-12-26 15:37:31,516][105620] Updated weights for policy 1, policy_version 35786 (0.0008) [2023-12-26 15:37:31,700][105692] Updated weights for policy 0, policy_version 35507 (0.0010) [2023-12-26 15:37:31,759][105692] Updated weights for policy 0, policy_version 35517 (0.0010) [2023-12-26 15:37:31,820][105692] Updated weights for policy 0, policy_version 35527 (0.0007) [2023-12-26 15:37:32,109][105620] Updated weights for policy 1, policy_version 35796 (0.0006) [2023-12-26 15:37:32,166][105620] Updated weights for policy 1, policy_version 35806 (0.0008) [2023-12-26 15:37:32,222][105620] Updated weights for policy 1, policy_version 35816 (0.0008) [2023-12-26 15:37:32,386][105692] Updated weights for policy 0, policy_version 35537 (0.0006) [2023-12-26 15:37:32,448][105692] Updated weights for policy 0, policy_version 35547 (0.0005) [2023-12-26 15:37:32,512][105692] Updated weights for policy 0, policy_version 35557 (0.0005) [2023-12-26 15:37:32,560][105692] Updated weights for policy 0, policy_version 35567 (0.0005) [2023-12-26 15:37:33,092][105692] Updated weights for policy 0, policy_version 35577 (0.0010) [2023-12-26 15:37:33,094][105620] Updated weights for policy 1, policy_version 35826 (0.0009) [2023-12-26 15:37:33,150][105620] Updated weights for policy 1, policy_version 35836 (0.0005) [2023-12-26 15:37:33,152][105692] Updated weights for policy 0, policy_version 35587 (0.0010) [2023-12-26 15:37:33,206][105620] Updated weights for policy 1, policy_version 35846 (0.0009) [2023-12-26 15:37:33,217][105692] Updated weights for policy 0, policy_version 35597 (0.0010) [2023-12-26 15:37:33,271][105620] Updated weights for policy 1, policy_version 35856 (0.0008) [2023-12-26 15:37:33,948][105692] Updated weights for policy 0, policy_version 35607 (0.0010) [2023-12-26 15:37:33,954][105620] Updated weights for policy 1, policy_version 35866 (0.0007) [2023-12-26 15:37:33,999][105692] Updated weights for policy 0, policy_version 35617 (0.0010) [2023-12-26 15:37:34,002][105620] Updated weights for policy 1, policy_version 35876 (0.0005) [2023-12-26 15:37:34,043][105692] Updated weights for policy 0, policy_version 35627 (0.0010) [2023-12-26 15:37:34,046][105620] Updated weights for policy 1, policy_version 35886 (0.0005) [2023-12-26 15:37:34,759][105620] Updated weights for policy 1, policy_version 35896 (0.0006) [2023-12-26 15:37:34,788][105692] Updated weights for policy 0, policy_version 35637 (0.0010) [2023-12-26 15:37:34,810][105620] Updated weights for policy 1, policy_version 35906 (0.0005) [2023-12-26 15:37:34,851][105692] Updated weights for policy 0, policy_version 35647 (0.0010) [2023-12-26 15:37:34,866][105620] Updated weights for policy 1, policy_version 35916 (0.0005) [2023-12-26 15:37:34,920][105692] Updated weights for policy 0, policy_version 35657 (0.0011) [2023-12-26 15:37:35,587][105692] Updated weights for policy 0, policy_version 35667 (0.0009) [2023-12-26 15:37:35,611][105620] Updated weights for policy 1, policy_version 35926 (0.0005) [2023-12-26 15:37:35,632][105692] Updated weights for policy 0, policy_version 35677 (0.0005) [2023-12-26 15:37:35,658][105620] Updated weights for policy 1, policy_version 35936 (0.0005) [2023-12-26 15:37:35,682][105692] Updated weights for policy 0, policy_version 35687 (0.0005) [2023-12-26 15:37:35,711][105620] Updated weights for policy 1, policy_version 35946 (0.0005) [2023-12-26 15:37:36,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 18350080. Throughput: 0: 9898.9, 1: 9836.4. Samples: 18337088. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-26 15:37:36,063][104569] Avg episode reward: [(0, '9257.505'), (1, '8203.079')] [2023-12-26 15:37:36,345][105620] Updated weights for policy 1, policy_version 35956 (0.0005) [2023-12-26 15:37:36,408][105620] Updated weights for policy 1, policy_version 35966 (0.0006) [2023-12-26 15:37:36,471][105620] Updated weights for policy 1, policy_version 35976 (0.0006) [2023-12-26 15:37:36,492][105692] Updated weights for policy 0, policy_version 35697 (0.0007) [2023-12-26 15:37:36,559][105692] Updated weights for policy 0, policy_version 35707 (0.0008) [2023-12-26 15:37:36,622][105692] Updated weights for policy 0, policy_version 35717 (0.0009) [2023-12-26 15:37:36,682][105692] Updated weights for policy 0, policy_version 35727 (0.0009) [2023-12-26 15:37:37,077][105620] Updated weights for policy 1, policy_version 35986 (0.0007) [2023-12-26 15:37:37,126][105620] Updated weights for policy 1, policy_version 35996 (0.0005) [2023-12-26 15:37:37,173][105620] Updated weights for policy 1, policy_version 36006 (0.0008) [2023-12-26 15:37:37,221][105620] Updated weights for policy 1, policy_version 36016 (0.0009) [2023-12-26 15:37:37,515][105692] Updated weights for policy 0, policy_version 35737 (0.0010) [2023-12-26 15:37:37,569][105692] Updated weights for policy 0, policy_version 35747 (0.0009) [2023-12-26 15:37:37,621][105692] Updated weights for policy 0, policy_version 35757 (0.0008) [2023-12-26 15:37:37,899][105620] Updated weights for policy 1, policy_version 36026 (0.0009) [2023-12-26 15:37:37,961][105620] Updated weights for policy 1, policy_version 36036 (0.0009) [2023-12-26 15:37:38,021][105620] Updated weights for policy 1, policy_version 36046 (0.0008) [2023-12-26 15:37:38,362][105692] Updated weights for policy 0, policy_version 35767 (0.0009) [2023-12-26 15:37:38,413][105692] Updated weights for policy 0, policy_version 35777 (0.0008) [2023-12-26 15:37:38,465][105692] Updated weights for policy 0, policy_version 35787 (0.0008) [2023-12-26 15:37:38,788][105620] Updated weights for policy 1, policy_version 36056 (0.0006) [2023-12-26 15:37:38,851][105620] Updated weights for policy 1, policy_version 36066 (0.0009) [2023-12-26 15:37:38,907][105620] Updated weights for policy 1, policy_version 36076 (0.0010) [2023-12-26 15:37:39,148][105692] Updated weights for policy 0, policy_version 35797 (0.0007) [2023-12-26 15:37:39,203][105692] Updated weights for policy 0, policy_version 35807 (0.0006) [2023-12-26 15:37:39,273][105692] Updated weights for policy 0, policy_version 35817 (0.0008) [2023-12-26 15:37:39,699][105620] Updated weights for policy 1, policy_version 36086 (0.0007) [2023-12-26 15:37:39,766][105620] Updated weights for policy 1, policy_version 36096 (0.0006) [2023-12-26 15:37:39,828][105620] Updated weights for policy 1, policy_version 36106 (0.0006) [2023-12-26 15:37:39,981][105692] Updated weights for policy 0, policy_version 35827 (0.0007) [2023-12-26 15:37:40,042][105692] Updated weights for policy 0, policy_version 35837 (0.0011) [2023-12-26 15:37:40,109][105692] Updated weights for policy 0, policy_version 35847 (0.0011) [2023-12-26 15:37:40,532][105620] Updated weights for policy 1, policy_version 36116 (0.0009) [2023-12-26 15:37:40,590][105620] Updated weights for policy 1, policy_version 36126 (0.0007) [2023-12-26 15:37:40,644][105620] Updated weights for policy 1, policy_version 36136 (0.0006) [2023-12-26 15:37:40,886][105692] Updated weights for policy 0, policy_version 35857 (0.0011) [2023-12-26 15:37:40,948][105692] Updated weights for policy 0, policy_version 35867 (0.0011) [2023-12-26 15:37:41,011][105692] Updated weights for policy 0, policy_version 35877 (0.0007) [2023-12-26 15:37:41,062][104569] Fps is (10 sec: 19661.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 18440192. Throughput: 0: 9771.6, 1: 9944.2. Samples: 18453860. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-26 15:37:41,063][104569] Avg episode reward: [(0, '9345.704'), (1, '8298.553')] [2023-12-26 15:37:41,074][105692] Updated weights for policy 0, policy_version 35887 (0.0008) [2023-12-26 15:37:41,313][105620] Updated weights for policy 1, policy_version 36146 (0.0006) [2023-12-26 15:37:41,384][105620] Updated weights for policy 1, policy_version 36156 (0.0009) [2023-12-26 15:37:41,449][105620] Updated weights for policy 1, policy_version 36166 (0.0008) [2023-12-26 15:37:41,510][105620] Updated weights for policy 1, policy_version 36176 (0.0008) [2023-12-26 15:37:41,801][105692] Updated weights for policy 0, policy_version 35897 (0.0010) [2023-12-26 15:37:41,861][105692] Updated weights for policy 0, policy_version 35907 (0.0011) [2023-12-26 15:37:41,926][105692] Updated weights for policy 0, policy_version 35917 (0.0011) [2023-12-26 15:37:42,309][105620] Updated weights for policy 1, policy_version 36186 (0.0009) [2023-12-26 15:37:42,377][105620] Updated weights for policy 1, policy_version 36196 (0.0008) [2023-12-26 15:37:42,436][105620] Updated weights for policy 1, policy_version 36206 (0.0008) [2023-12-26 15:37:42,696][105692] Updated weights for policy 0, policy_version 35927 (0.0007) [2023-12-26 15:37:42,751][105692] Updated weights for policy 0, policy_version 35937 (0.0005) [2023-12-26 15:37:42,800][105692] Updated weights for policy 0, policy_version 35947 (0.0005) [2023-12-26 15:37:43,110][105620] Updated weights for policy 1, policy_version 36216 (0.0010) [2023-12-26 15:37:43,163][105620] Updated weights for policy 1, policy_version 36226 (0.0010) [2023-12-26 15:37:43,211][105620] Updated weights for policy 1, policy_version 36236 (0.0007) [2023-12-26 15:37:43,411][105692] Updated weights for policy 0, policy_version 35957 (0.0008) [2023-12-26 15:37:43,469][105692] Updated weights for policy 0, policy_version 35967 (0.0011) [2023-12-26 15:37:43,532][105692] Updated weights for policy 0, policy_version 35977 (0.0011) [2023-12-26 15:37:43,831][105620] Updated weights for policy 1, policy_version 36246 (0.0005) [2023-12-26 15:37:43,890][105620] Updated weights for policy 1, policy_version 36256 (0.0005) [2023-12-26 15:37:43,938][105620] Updated weights for policy 1, policy_version 36266 (0.0005) [2023-12-26 15:37:44,270][105692] Updated weights for policy 0, policy_version 35987 (0.0008) [2023-12-26 15:37:44,326][105692] Updated weights for policy 0, policy_version 35997 (0.0006) [2023-12-26 15:37:44,373][105692] Updated weights for policy 0, policy_version 36007 (0.0005) [2023-12-26 15:37:44,496][105620] Updated weights for policy 1, policy_version 36276 (0.0010) [2023-12-26 15:37:44,544][105620] Updated weights for policy 1, policy_version 36286 (0.0010) [2023-12-26 15:37:44,592][105620] Updated weights for policy 1, policy_version 36296 (0.0010) [2023-12-26 15:37:45,033][105692] Updated weights for policy 0, policy_version 36017 (0.0005) [2023-12-26 15:37:45,090][105692] Updated weights for policy 0, policy_version 36027 (0.0008) [2023-12-26 15:37:45,154][105692] Updated weights for policy 0, policy_version 36037 (0.0009) [2023-12-26 15:37:45,222][105692] Updated weights for policy 0, policy_version 36047 (0.0009) [2023-12-26 15:37:45,361][105620] Updated weights for policy 1, policy_version 36306 (0.0009) [2023-12-26 15:37:45,424][105620] Updated weights for policy 1, policy_version 36316 (0.0011) [2023-12-26 15:37:45,483][105620] Updated weights for policy 1, policy_version 36326 (0.0010) [2023-12-26 15:37:45,546][105620] Updated weights for policy 1, policy_version 36336 (0.0010) [2023-12-26 15:37:45,874][105692] Updated weights for policy 0, policy_version 36057 (0.0008) [2023-12-26 15:37:45,935][105692] Updated weights for policy 0, policy_version 36067 (0.0007) [2023-12-26 15:37:45,990][105692] Updated weights for policy 0, policy_version 36077 (0.0008) [2023-12-26 15:37:46,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 18546688. Throughput: 0: 9738.7, 1: 10020.0. Samples: 18512116. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-26 15:37:46,062][104569] Avg episode reward: [(0, '9344.767'), (1, '2256.706')] [2023-12-26 15:37:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000036080_9240576.pth... [2023-12-26 15:37:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000036336_9306112.pth... [2023-12-26 15:37:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000035184_9011200.pth [2023-12-26 15:37:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000034928_8945664.pth [2023-12-26 15:37:46,291][105620] Updated weights for policy 1, policy_version 36346 (0.0005) [2023-12-26 15:37:46,361][105620] Updated weights for policy 1, policy_version 36356 (0.0009) [2023-12-26 15:37:46,428][105620] Updated weights for policy 1, policy_version 36366 (0.0010) [2023-12-26 15:37:46,559][105692] Updated weights for policy 0, policy_version 36087 (0.0005) [2023-12-26 15:37:46,607][105692] Updated weights for policy 0, policy_version 36097 (0.0005) [2023-12-26 15:37:46,657][105692] Updated weights for policy 0, policy_version 36107 (0.0005) [2023-12-26 15:37:47,043][105620] Updated weights for policy 1, policy_version 36376 (0.0010) [2023-12-26 15:37:47,097][105620] Updated weights for policy 1, policy_version 36386 (0.0010) [2023-12-26 15:37:47,152][105620] Updated weights for policy 1, policy_version 36396 (0.0010) [2023-12-26 15:37:47,400][105692] Updated weights for policy 0, policy_version 36117 (0.0007) [2023-12-26 15:37:47,445][105692] Updated weights for policy 0, policy_version 36127 (0.0008) [2023-12-26 15:37:47,489][105692] Updated weights for policy 0, policy_version 36137 (0.0008) [2023-12-26 15:37:47,841][105620] Updated weights for policy 1, policy_version 36406 (0.0009) [2023-12-26 15:37:47,891][105620] Updated weights for policy 1, policy_version 36416 (0.0006) [2023-12-26 15:37:47,944][105620] Updated weights for policy 1, policy_version 36426 (0.0009) [2023-12-26 15:37:48,289][105692] Updated weights for policy 0, policy_version 36147 (0.0007) [2023-12-26 15:37:48,351][105692] Updated weights for policy 0, policy_version 36157 (0.0006) [2023-12-26 15:37:48,411][105692] Updated weights for policy 0, policy_version 36167 (0.0007) [2023-12-26 15:37:48,671][105620] Updated weights for policy 1, policy_version 36436 (0.0008) [2023-12-26 15:37:48,725][105620] Updated weights for policy 1, policy_version 36448 (0.0010) [2023-12-26 15:37:48,788][105620] Updated weights for policy 1, policy_version 36459 (0.0011) [2023-12-26 15:37:49,008][105692] Updated weights for policy 0, policy_version 36177 (0.0008) [2023-12-26 15:37:49,071][105692] Updated weights for policy 0, policy_version 36187 (0.0010) [2023-12-26 15:37:49,126][105692] Updated weights for policy 0, policy_version 36197 (0.0008) [2023-12-26 15:37:49,177][105692] Updated weights for policy 0, policy_version 36207 (0.0010) [2023-12-26 15:37:49,632][105620] Updated weights for policy 1, policy_version 36469 (0.0009) [2023-12-26 15:37:49,691][105620] Updated weights for policy 1, policy_version 36479 (0.0009) [2023-12-26 15:37:49,759][105620] Updated weights for policy 1, policy_version 36489 (0.0007) [2023-12-26 15:37:49,819][105692] Updated weights for policy 0, policy_version 36217 (0.0008) [2023-12-26 15:37:49,887][105692] Updated weights for policy 0, policy_version 36227 (0.0008) [2023-12-26 15:37:49,947][105692] Updated weights for policy 0, policy_version 36237 (0.0009) [2023-12-26 15:37:50,501][105620] Updated weights for policy 1, policy_version 36499 (0.0008) [2023-12-26 15:37:50,553][105620] Updated weights for policy 1, policy_version 36509 (0.0009) [2023-12-26 15:37:50,612][105620] Updated weights for policy 1, policy_version 36519 (0.0008) [2023-12-26 15:37:50,636][105692] Updated weights for policy 0, policy_version 36247 (0.0008) [2023-12-26 15:37:50,688][105692] Updated weights for policy 0, policy_version 36257 (0.0008) [2023-12-26 15:37:50,744][105692] Updated weights for policy 0, policy_version 36267 (0.0008) [2023-12-26 15:37:51,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 18644992. Throughput: 0: 9852.9, 1: 9877.6. Samples: 18632668. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-12-26 15:37:51,062][104569] Avg episode reward: [(0, '9346.750'), (1, '1164.773')] [2023-12-26 15:37:51,302][105620] Updated weights for policy 1, policy_version 36529 (0.0006) [2023-12-26 15:37:51,365][105620] Updated weights for policy 1, policy_version 36539 (0.0008) [2023-12-26 15:37:51,427][105620] Updated weights for policy 1, policy_version 36549 (0.0008) [2023-12-26 15:37:51,490][105620] Updated weights for policy 1, policy_version 36559 (0.0008) [2023-12-26 15:37:51,518][105692] Updated weights for policy 0, policy_version 36277 (0.0008) [2023-12-26 15:37:51,571][105692] Updated weights for policy 0, policy_version 36287 (0.0008) [2023-12-26 15:37:51,623][105692] Updated weights for policy 0, policy_version 36297 (0.0008) [2023-12-26 15:37:52,209][105620] Updated weights for policy 1, policy_version 36569 (0.0010) [2023-12-26 15:37:52,275][105620] Updated weights for policy 1, policy_version 36579 (0.0010) [2023-12-26 15:37:52,334][105620] Updated weights for policy 1, policy_version 36589 (0.0011) [2023-12-26 15:37:52,404][105692] Updated weights for policy 0, policy_version 36307 (0.0007) [2023-12-26 15:37:52,465][105692] Updated weights for policy 0, policy_version 36317 (0.0007) [2023-12-26 15:37:52,536][105692] Updated weights for policy 0, policy_version 36327 (0.0009) [2023-12-26 15:37:53,036][105620] Updated weights for policy 1, policy_version 36599 (0.0010) [2023-12-26 15:37:53,087][105620] Updated weights for policy 1, policy_version 36609 (0.0010) [2023-12-26 15:37:53,138][105620] Updated weights for policy 1, policy_version 36619 (0.0010) [2023-12-26 15:37:53,231][105692] Updated weights for policy 0, policy_version 36337 (0.0010) [2023-12-26 15:37:53,291][105692] Updated weights for policy 0, policy_version 36347 (0.0008) [2023-12-26 15:37:53,349][105692] Updated weights for policy 0, policy_version 36357 (0.0010) [2023-12-26 15:37:53,407][105692] Updated weights for policy 0, policy_version 36367 (0.0010) [2023-12-26 15:37:53,801][105620] Updated weights for policy 1, policy_version 36629 (0.0008) [2023-12-26 15:37:53,848][105620] Updated weights for policy 1, policy_version 36639 (0.0005) [2023-12-26 15:37:53,895][105620] Updated weights for policy 1, policy_version 36649 (0.0005) [2023-12-26 15:37:54,038][105692] Updated weights for policy 0, policy_version 36377 (0.0008) [2023-12-26 15:37:54,102][105692] Updated weights for policy 0, policy_version 36387 (0.0010) [2023-12-26 15:37:54,154][105692] Updated weights for policy 0, policy_version 36397 (0.0010) [2023-12-26 15:37:54,458][105620] Updated weights for policy 1, policy_version 36659 (0.0005) [2023-12-26 15:37:54,511][105620] Updated weights for policy 1, policy_version 36669 (0.0008) [2023-12-26 15:37:54,582][105620] Updated weights for policy 1, policy_version 36679 (0.0009) [2023-12-26 15:37:54,955][105692] Updated weights for policy 0, policy_version 36407 (0.0009) [2023-12-26 15:37:55,019][105692] Updated weights for policy 0, policy_version 36417 (0.0009) [2023-12-26 15:37:55,073][105692] Updated weights for policy 0, policy_version 36427 (0.0009) [2023-12-26 15:37:55,264][105620] Updated weights for policy 1, policy_version 36689 (0.0009) [2023-12-26 15:37:55,329][105620] Updated weights for policy 1, policy_version 36699 (0.0008) [2023-12-26 15:37:55,396][105620] Updated weights for policy 1, policy_version 36709 (0.0007) [2023-12-26 15:37:55,454][105620] Updated weights for policy 1, policy_version 36719 (0.0009) [2023-12-26 15:37:55,846][105692] Updated weights for policy 0, policy_version 36437 (0.0009) [2023-12-26 15:37:55,900][105692] Updated weights for policy 0, policy_version 36447 (0.0010) [2023-12-26 15:37:55,957][105692] Updated weights for policy 0, policy_version 36458 (0.0009) [2023-12-26 15:37:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 18743296. Throughput: 0: 9840.6, 1: 9910.7. Samples: 18750840. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-12-26 15:37:56,062][104569] Avg episode reward: [(0, '9346.997'), (1, '1729.732')] [2023-12-26 15:37:56,105][105620] Updated weights for policy 1, policy_version 36729 (0.0009) [2023-12-26 15:37:56,166][105620] Updated weights for policy 1, policy_version 36739 (0.0009) [2023-12-26 15:37:56,228][105620] Updated weights for policy 1, policy_version 36749 (0.0009) [2023-12-26 15:37:56,735][105692] Updated weights for policy 0, policy_version 36468 (0.0009) [2023-12-26 15:37:56,795][105692] Updated weights for policy 0, policy_version 36478 (0.0009) [2023-12-26 15:37:56,844][105692] Updated weights for policy 0, policy_version 36488 (0.0008) [2023-12-26 15:37:56,963][105620] Updated weights for policy 1, policy_version 36759 (0.0009) [2023-12-26 15:37:57,024][105620] Updated weights for policy 1, policy_version 36769 (0.0009) [2023-12-26 15:37:57,083][105620] Updated weights for policy 1, policy_version 36779 (0.0008) [2023-12-26 15:37:57,614][105692] Updated weights for policy 0, policy_version 36498 (0.0009) [2023-12-26 15:37:57,676][105692] Updated weights for policy 0, policy_version 36508 (0.0009) [2023-12-26 15:37:57,730][105692] Updated weights for policy 0, policy_version 36519 (0.0010) [2023-12-26 15:37:57,767][105620] Updated weights for policy 1, policy_version 36789 (0.0007) [2023-12-26 15:37:57,817][105620] Updated weights for policy 1, policy_version 36799 (0.0005) [2023-12-26 15:37:57,876][105620] Updated weights for policy 1, policy_version 36809 (0.0005) [2023-12-26 15:37:58,506][105692] Updated weights for policy 0, policy_version 36529 (0.0009) [2023-12-26 15:37:58,569][105692] Updated weights for policy 0, policy_version 36539 (0.0008) [2023-12-26 15:37:58,614][105620] Updated weights for policy 1, policy_version 36819 (0.0008) [2023-12-26 15:37:58,637][105692] Updated weights for policy 0, policy_version 36549 (0.0007) [2023-12-26 15:37:58,703][105620] Updated weights for policy 1, policy_version 36829 (0.0007) [2023-12-26 15:37:58,705][105692] Updated weights for policy 0, policy_version 36559 (0.0008) [2023-12-26 15:37:58,768][105620] Updated weights for policy 1, policy_version 36839 (0.0007) [2023-12-26 15:37:59,464][105620] Updated weights for policy 1, policy_version 36849 (0.0008) [2023-12-26 15:37:59,468][105692] Updated weights for policy 0, policy_version 36569 (0.0009) [2023-12-26 15:37:59,513][105620] Updated weights for policy 1, policy_version 36859 (0.0011) [2023-12-26 15:37:59,519][105692] Updated weights for policy 0, policy_version 36579 (0.0006) [2023-12-26 15:37:59,572][105620] Updated weights for policy 1, policy_version 36869 (0.0010) [2023-12-26 15:37:59,576][105692] Updated weights for policy 0, policy_version 36589 (0.0005) [2023-12-26 15:37:59,624][105620] Updated weights for policy 1, policy_version 36879 (0.0010) [2023-12-26 15:38:00,227][105692] Updated weights for policy 0, policy_version 36599 (0.0009) [2023-12-26 15:38:00,288][105692] Updated weights for policy 0, policy_version 36609 (0.0008) [2023-12-26 15:38:00,300][105620] Updated weights for policy 1, policy_version 36889 (0.0007) [2023-12-26 15:38:00,346][105692] Updated weights for policy 0, policy_version 36619 (0.0008) [2023-12-26 15:38:00,364][105620] Updated weights for policy 1, policy_version 36899 (0.0008) [2023-12-26 15:38:00,420][105620] Updated weights for policy 1, policy_version 36909 (0.0010) [2023-12-26 15:38:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 18833408. Throughput: 0: 9802.7, 1: 9897.7. Samples: 18806664. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-12-26 15:38:01,062][104569] Avg episode reward: [(0, '9344.251'), (1, '3016.297')] [2023-12-26 15:38:01,096][105692] Updated weights for policy 0, policy_version 36629 (0.0008) [2023-12-26 15:38:01,114][105620] Updated weights for policy 1, policy_version 36919 (0.0010) [2023-12-26 15:38:01,164][105692] Updated weights for policy 0, policy_version 36639 (0.0008) [2023-12-26 15:38:01,178][105620] Updated weights for policy 1, policy_version 36929 (0.0009) [2023-12-26 15:38:01,221][105692] Updated weights for policy 0, policy_version 36649 (0.0007) [2023-12-26 15:38:01,237][105620] Updated weights for policy 1, policy_version 36939 (0.0006) [2023-12-26 15:38:01,261][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000036656_9388032.pth... [2023-12-26 15:38:01,264][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000036944_9461760.pth... [2023-12-26 15:38:01,265][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000035504_9093120.pth [2023-12-26 15:38:01,268][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000035760_9158656.pth [2023-12-26 15:38:01,945][105620] Updated weights for policy 1, policy_version 36949 (0.0008) [2023-12-26 15:38:01,952][105692] Updated weights for policy 0, policy_version 36659 (0.0007) [2023-12-26 15:38:01,999][105620] Updated weights for policy 1, policy_version 36959 (0.0006) [2023-12-26 15:38:02,009][105692] Updated weights for policy 0, policy_version 36669 (0.0007) [2023-12-26 15:38:02,053][105620] Updated weights for policy 1, policy_version 36969 (0.0007) [2023-12-26 15:38:02,060][105692] Updated weights for policy 0, policy_version 36679 (0.0007) [2023-12-26 15:38:02,752][105620] Updated weights for policy 1, policy_version 36979 (0.0009) [2023-12-26 15:38:02,803][105620] Updated weights for policy 1, policy_version 36989 (0.0009) [2023-12-26 15:38:02,856][105620] Updated weights for policy 1, policy_version 36999 (0.0008) [2023-12-26 15:38:02,863][105692] Updated weights for policy 0, policy_version 36689 (0.0006) [2023-12-26 15:38:02,929][105692] Updated weights for policy 0, policy_version 36699 (0.0008) [2023-12-26 15:38:02,989][105692] Updated weights for policy 0, policy_version 36709 (0.0010) [2023-12-26 15:38:03,047][105692] Updated weights for policy 0, policy_version 36719 (0.0009) [2023-12-26 15:38:03,540][105620] Updated weights for policy 1, policy_version 37009 (0.0009) [2023-12-26 15:38:03,598][105620] Updated weights for policy 1, policy_version 37019 (0.0005) [2023-12-26 15:38:03,647][105620] Updated weights for policy 1, policy_version 37029 (0.0005) [2023-12-26 15:38:03,692][105620] Updated weights for policy 1, policy_version 37039 (0.0006) [2023-12-26 15:38:03,866][105692] Updated weights for policy 0, policy_version 36729 (0.0009) [2023-12-26 15:38:03,922][105692] Updated weights for policy 0, policy_version 36739 (0.0009) [2023-12-26 15:38:03,982][105692] Updated weights for policy 0, policy_version 36749 (0.0010) [2023-12-26 15:38:04,407][105620] Updated weights for policy 1, policy_version 37049 (0.0010) [2023-12-26 15:38:04,466][105620] Updated weights for policy 1, policy_version 37059 (0.0008) [2023-12-26 15:38:04,528][105620] Updated weights for policy 1, policy_version 37069 (0.0009) [2023-12-26 15:38:04,747][105692] Updated weights for policy 0, policy_version 36759 (0.0007) [2023-12-26 15:38:04,813][105692] Updated weights for policy 0, policy_version 36769 (0.0006) [2023-12-26 15:38:04,869][105692] Updated weights for policy 0, policy_version 36779 (0.0009) [2023-12-26 15:38:05,289][105620] Updated weights for policy 1, policy_version 37079 (0.0007) [2023-12-26 15:38:05,349][105620] Updated weights for policy 1, policy_version 37089 (0.0005) [2023-12-26 15:38:05,404][105620] Updated weights for policy 1, policy_version 37099 (0.0005) [2023-12-26 15:38:05,639][105692] Updated weights for policy 0, policy_version 36789 (0.0009) [2023-12-26 15:38:05,705][105692] Updated weights for policy 0, policy_version 36799 (0.0009) [2023-12-26 15:38:05,763][105692] Updated weights for policy 0, policy_version 36809 (0.0008) [2023-12-26 15:38:05,968][105620] Updated weights for policy 1, policy_version 37109 (0.0005) [2023-12-26 15:38:06,022][105620] Updated weights for policy 1, policy_version 37119 (0.0005) [2023-12-26 15:38:06,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 18931712. Throughput: 0: 9672.7, 1: 9996.3. Samples: 18921756. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-12-26 15:38:06,063][104569] Avg episode reward: [(0, '9342.586'), (1, '4159.814')] [2023-12-26 15:38:06,071][105620] Updated weights for policy 1, policy_version 37129 (0.0006) [2023-12-26 15:38:06,583][105692] Updated weights for policy 0, policy_version 36819 (0.0009) [2023-12-26 15:38:06,641][105692] Updated weights for policy 0, policy_version 36829 (0.0010) [2023-12-26 15:38:06,702][105692] Updated weights for policy 0, policy_version 36839 (0.0010) [2023-12-26 15:38:06,727][105620] Updated weights for policy 1, policy_version 37139 (0.0009) [2023-12-26 15:38:06,794][105620] Updated weights for policy 1, policy_version 37149 (0.0005) [2023-12-26 15:38:06,864][105620] Updated weights for policy 1, policy_version 37159 (0.0006) [2023-12-26 15:38:07,482][105620] Updated weights for policy 1, policy_version 37169 (0.0007) [2023-12-26 15:38:07,532][105692] Updated weights for policy 0, policy_version 36849 (0.0009) [2023-12-26 15:38:07,535][105620] Updated weights for policy 1, policy_version 37179 (0.0005) [2023-12-26 15:38:07,594][105692] Updated weights for policy 0, policy_version 36859 (0.0009) [2023-12-26 15:38:07,598][105620] Updated weights for policy 1, policy_version 37189 (0.0005) [2023-12-26 15:38:07,649][105692] Updated weights for policy 0, policy_version 36869 (0.0008) [2023-12-26 15:38:07,659][105620] Updated weights for policy 1, policy_version 37199 (0.0005) [2023-12-26 15:38:07,714][105692] Updated weights for policy 0, policy_version 36879 (0.0006) [2023-12-26 15:38:08,210][105620] Updated weights for policy 1, policy_version 37209 (0.0005) [2023-12-26 15:38:08,270][105620] Updated weights for policy 1, policy_version 37219 (0.0005) [2023-12-26 15:38:08,337][105620] Updated weights for policy 1, policy_version 37229 (0.0006) [2023-12-26 15:38:08,392][105692] Updated weights for policy 0, policy_version 36889 (0.0010) [2023-12-26 15:38:08,444][105692] Updated weights for policy 0, policy_version 36899 (0.0011) [2023-12-26 15:38:08,503][105692] Updated weights for policy 0, policy_version 36909 (0.0010) [2023-12-26 15:38:09,000][105620] Updated weights for policy 1, policy_version 37239 (0.0010) [2023-12-26 15:38:09,072][105620] Updated weights for policy 1, policy_version 37249 (0.0011) [2023-12-26 15:38:09,137][105620] Updated weights for policy 1, policy_version 37259 (0.0010) [2023-12-26 15:38:09,193][105692] Updated weights for policy 0, policy_version 36919 (0.0007) [2023-12-26 15:38:09,262][105692] Updated weights for policy 0, policy_version 36929 (0.0009) [2023-12-26 15:38:09,324][105692] Updated weights for policy 0, policy_version 36939 (0.0010) [2023-12-26 15:38:09,859][105620] Updated weights for policy 1, policy_version 37269 (0.0008) [2023-12-26 15:38:09,916][105620] Updated weights for policy 1, policy_version 37279 (0.0010) [2023-12-26 15:38:09,980][105620] Updated weights for policy 1, policy_version 37289 (0.0011) [2023-12-26 15:38:10,083][105692] Updated weights for policy 0, policy_version 36949 (0.0009) [2023-12-26 15:38:10,150][105692] Updated weights for policy 0, policy_version 36959 (0.0008) [2023-12-26 15:38:10,217][105692] Updated weights for policy 0, policy_version 36969 (0.0008) [2023-12-26 15:38:10,724][105620] Updated weights for policy 1, policy_version 37299 (0.0011) [2023-12-26 15:38:10,782][105620] Updated weights for policy 1, policy_version 37309 (0.0010) [2023-12-26 15:38:10,837][105620] Updated weights for policy 1, policy_version 37319 (0.0010) [2023-12-26 15:38:10,867][105692] Updated weights for policy 0, policy_version 36979 (0.0008) [2023-12-26 15:38:10,925][105692] Updated weights for policy 0, policy_version 36989 (0.0005) [2023-12-26 15:38:10,977][105692] Updated weights for policy 0, policy_version 36999 (0.0007) [2023-12-26 15:38:11,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 19038208. Throughput: 0: 9631.0, 1: 10052.9. Samples: 19039364. Policy #0 lag: (min: 2.0, avg: 13.7, max: 34.0) [2023-12-26 15:38:11,063][104569] Avg episode reward: [(0, '9343.801'), (1, '7207.547')] [2023-12-26 15:38:11,629][105620] Updated weights for policy 1, policy_version 37329 (0.0010) [2023-12-26 15:38:11,693][105620] Updated weights for policy 1, policy_version 37339 (0.0010) [2023-12-26 15:38:11,703][105692] Updated weights for policy 0, policy_version 37009 (0.0009) [2023-12-26 15:38:11,762][105620] Updated weights for policy 1, policy_version 37349 (0.0010) [2023-12-26 15:38:11,768][105692] Updated weights for policy 0, policy_version 37019 (0.0007) [2023-12-26 15:38:11,828][105620] Updated weights for policy 1, policy_version 37359 (0.0010) [2023-12-26 15:38:11,829][105692] Updated weights for policy 0, policy_version 37029 (0.0007) [2023-12-26 15:38:11,892][105692] Updated weights for policy 0, policy_version 37039 (0.0008) [2023-12-26 15:38:12,569][105620] Updated weights for policy 1, policy_version 37369 (0.0007) [2023-12-26 15:38:12,606][105692] Updated weights for policy 0, policy_version 37049 (0.0006) [2023-12-26 15:38:12,631][105620] Updated weights for policy 1, policy_version 37379 (0.0010) [2023-12-26 15:38:12,665][105692] Updated weights for policy 0, policy_version 37059 (0.0005) [2023-12-26 15:38:12,686][105620] Updated weights for policy 1, policy_version 37389 (0.0010) [2023-12-26 15:38:12,739][105692] Updated weights for policy 0, policy_version 37072 (0.0006) [2023-12-26 15:38:13,345][105620] Updated weights for policy 1, policy_version 37399 (0.0007) [2023-12-26 15:38:13,358][105692] Updated weights for policy 0, policy_version 37082 (0.0008) [2023-12-26 15:38:13,412][105620] Updated weights for policy 1, policy_version 37409 (0.0009) [2023-12-26 15:38:13,419][105692] Updated weights for policy 0, policy_version 37092 (0.0006) [2023-12-26 15:38:13,475][105692] Updated weights for policy 0, policy_version 37102 (0.0008) [2023-12-26 15:38:13,476][105620] Updated weights for policy 1, policy_version 37419 (0.0011) [2023-12-26 15:38:14,171][105620] Updated weights for policy 1, policy_version 37429 (0.0010) [2023-12-26 15:38:14,201][105692] Updated weights for policy 0, policy_version 37112 (0.0006) [2023-12-26 15:38:14,237][105620] Updated weights for policy 1, policy_version 37439 (0.0010) [2023-12-26 15:38:14,261][105692] Updated weights for policy 0, policy_version 37122 (0.0007) [2023-12-26 15:38:14,296][105620] Updated weights for policy 1, policy_version 37449 (0.0010) [2023-12-26 15:38:14,319][105692] Updated weights for policy 0, policy_version 37132 (0.0007) [2023-12-26 15:38:15,006][105692] Updated weights for policy 0, policy_version 37142 (0.0007) [2023-12-26 15:38:15,006][105620] Updated weights for policy 1, policy_version 37459 (0.0009) [2023-12-26 15:38:15,069][105692] Updated weights for policy 0, policy_version 37152 (0.0006) [2023-12-26 15:38:15,074][105620] Updated weights for policy 1, policy_version 37469 (0.0006) [2023-12-26 15:38:15,130][105692] Updated weights for policy 0, policy_version 37162 (0.0008) [2023-12-26 15:38:15,140][105620] Updated weights for policy 1, policy_version 37479 (0.0009) [2023-12-26 15:38:15,777][105620] Updated weights for policy 1, policy_version 37489 (0.0010) [2023-12-26 15:38:15,834][105692] Updated weights for policy 0, policy_version 37172 (0.0007) [2023-12-26 15:38:15,838][105620] Updated weights for policy 1, policy_version 37499 (0.0005) [2023-12-26 15:38:15,893][105692] Updated weights for policy 0, policy_version 37182 (0.0009) [2023-12-26 15:38:15,894][105620] Updated weights for policy 1, policy_version 37509 (0.0005) [2023-12-26 15:38:15,953][105692] Updated weights for policy 0, policy_version 37192 (0.0007) [2023-12-26 15:38:15,954][105620] Updated weights for policy 1, policy_version 37519 (0.0009) [2023-12-26 15:38:16,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 19136512. Throughput: 0: 9660.6, 1: 9989.6. Samples: 19098688. Policy #0 lag: (min: 2.0, avg: 13.7, max: 34.0) [2023-12-26 15:38:16,062][104569] Avg episode reward: [(0, '9346.454'), (1, '8895.096')] [2023-12-26 15:38:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000037200_9527296.pth... [2023-12-26 15:38:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000037520_9609216.pth... [2023-12-26 15:38:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000036080_9240576.pth [2023-12-26 15:38:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000036336_9306112.pth [2023-12-26 15:38:16,623][105620] Updated weights for policy 1, policy_version 37529 (0.0010) [2023-12-26 15:38:16,636][105692] Updated weights for policy 0, policy_version 37202 (0.0007) [2023-12-26 15:38:16,671][105620] Updated weights for policy 1, policy_version 37539 (0.0010) [2023-12-26 15:38:16,689][105692] Updated weights for policy 0, policy_version 37212 (0.0005) [2023-12-26 15:38:16,729][105620] Updated weights for policy 1, policy_version 37549 (0.0010) [2023-12-26 15:38:16,739][105692] Updated weights for policy 0, policy_version 37222 (0.0005) [2023-12-26 15:38:16,795][105692] Updated weights for policy 0, policy_version 37232 (0.0005) [2023-12-26 15:38:17,300][105692] Updated weights for policy 0, policy_version 37242 (0.0010) [2023-12-26 15:38:17,346][105692] Updated weights for policy 0, policy_version 37252 (0.0008) [2023-12-26 15:38:17,400][105692] Updated weights for policy 0, policy_version 37262 (0.0005) [2023-12-26 15:38:17,444][105620] Updated weights for policy 1, policy_version 37559 (0.0010) [2023-12-26 15:38:17,506][105620] Updated weights for policy 1, policy_version 37569 (0.0011) [2023-12-26 15:38:17,564][105620] Updated weights for policy 1, policy_version 37579 (0.0010) [2023-12-26 15:38:17,958][105692] Updated weights for policy 0, policy_version 37272 (0.0005) [2023-12-26 15:38:18,006][105692] Updated weights for policy 0, policy_version 37282 (0.0006) [2023-12-26 15:38:18,065][105692] Updated weights for policy 0, policy_version 37292 (0.0008) [2023-12-26 15:38:18,296][105620] Updated weights for policy 1, policy_version 37589 (0.0010) [2023-12-26 15:38:18,354][105620] Updated weights for policy 1, policy_version 37599 (0.0010) [2023-12-26 15:38:18,428][105620] Updated weights for policy 1, policy_version 37609 (0.0011) [2023-12-26 15:38:18,772][105692] Updated weights for policy 0, policy_version 37302 (0.0008) [2023-12-26 15:38:18,831][105692] Updated weights for policy 0, policy_version 37312 (0.0008) [2023-12-26 15:38:18,887][105692] Updated weights for policy 0, policy_version 37322 (0.0008) [2023-12-26 15:38:19,134][105620] Updated weights for policy 1, policy_version 37619 (0.0009) [2023-12-26 15:38:19,196][105620] Updated weights for policy 1, policy_version 37629 (0.0007) [2023-12-26 15:38:19,267][105620] Updated weights for policy 1, policy_version 37639 (0.0008) [2023-12-26 15:38:19,575][105692] Updated weights for policy 0, policy_version 37332 (0.0008) [2023-12-26 15:38:19,632][105692] Updated weights for policy 0, policy_version 37342 (0.0006) [2023-12-26 15:38:19,689][105692] Updated weights for policy 0, policy_version 37352 (0.0006) [2023-12-26 15:38:19,956][105620] Updated weights for policy 1, policy_version 37649 (0.0008) [2023-12-26 15:38:20,023][105620] Updated weights for policy 1, policy_version 37659 (0.0010) [2023-12-26 15:38:20,090][105620] Updated weights for policy 1, policy_version 37669 (0.0011) [2023-12-26 15:38:20,145][105620] Updated weights for policy 1, policy_version 37679 (0.0011) [2023-12-26 15:38:20,396][105692] Updated weights for policy 0, policy_version 37362 (0.0007) [2023-12-26 15:38:20,460][105692] Updated weights for policy 0, policy_version 37372 (0.0008) [2023-12-26 15:38:20,516][105692] Updated weights for policy 0, policy_version 37382 (0.0008) [2023-12-26 15:38:20,588][105692] Updated weights for policy 0, policy_version 37392 (0.0008) [2023-12-26 15:38:20,913][105620] Updated weights for policy 1, policy_version 37689 (0.0011) [2023-12-26 15:38:20,973][105620] Updated weights for policy 1, policy_version 37699 (0.0011) [2023-12-26 15:38:21,026][105620] Updated weights for policy 1, policy_version 37709 (0.0010) [2023-12-26 15:38:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 19234816. Throughput: 0: 9696.1, 1: 9953.5. Samples: 19221320. Policy #0 lag: (min: 2.0, avg: 13.7, max: 34.0) [2023-12-26 15:38:21,062][104569] Avg episode reward: [(0, '9186.862'), (1, '9070.786')] [2023-12-26 15:38:21,242][105692] Updated weights for policy 0, policy_version 37402 (0.0010) [2023-12-26 15:38:21,306][105692] Updated weights for policy 0, policy_version 37412 (0.0009) [2023-12-26 15:38:21,372][105692] Updated weights for policy 0, policy_version 37422 (0.0010) [2023-12-26 15:38:21,754][105620] Updated weights for policy 1, policy_version 37719 (0.0010) [2023-12-26 15:38:21,827][105620] Updated weights for policy 1, policy_version 37729 (0.0011) [2023-12-26 15:38:21,886][105620] Updated weights for policy 1, policy_version 37739 (0.0011) [2023-12-26 15:38:22,058][105692] Updated weights for policy 0, policy_version 37432 (0.0008) [2023-12-26 15:38:22,118][105692] Updated weights for policy 0, policy_version 37442 (0.0009) [2023-12-26 15:38:22,178][105692] Updated weights for policy 0, policy_version 37452 (0.0009) [2023-12-26 15:38:22,548][105620] Updated weights for policy 1, policy_version 37749 (0.0010) [2023-12-26 15:38:22,601][105620] Updated weights for policy 1, policy_version 37759 (0.0010) [2023-12-26 15:38:22,654][105620] Updated weights for policy 1, policy_version 37769 (0.0011) [2023-12-26 15:38:22,978][105692] Updated weights for policy 0, policy_version 37462 (0.0010) [2023-12-26 15:38:23,034][105692] Updated weights for policy 0, policy_version 37472 (0.0007) [2023-12-26 15:38:23,089][105692] Updated weights for policy 0, policy_version 37482 (0.0005) [2023-12-26 15:38:23,423][105620] Updated weights for policy 1, policy_version 37779 (0.0009) [2023-12-26 15:38:23,472][105620] Updated weights for policy 1, policy_version 37789 (0.0005) [2023-12-26 15:38:23,515][105620] Updated weights for policy 1, policy_version 37799 (0.0005) [2023-12-26 15:38:23,646][105692] Updated weights for policy 0, policy_version 37492 (0.0005) [2023-12-26 15:38:23,700][105692] Updated weights for policy 0, policy_version 37502 (0.0005) [2023-12-26 15:38:23,747][105692] Updated weights for policy 0, policy_version 37512 (0.0007) [2023-12-26 15:38:24,090][105620] Updated weights for policy 1, policy_version 37809 (0.0005) [2023-12-26 15:38:24,139][105620] Updated weights for policy 1, policy_version 37819 (0.0005) [2023-12-26 15:38:24,189][105620] Updated weights for policy 1, policy_version 37829 (0.0005) [2023-12-26 15:38:24,239][105620] Updated weights for policy 1, policy_version 37839 (0.0005) [2023-12-26 15:38:24,368][105692] Updated weights for policy 0, policy_version 37522 (0.0006) [2023-12-26 15:38:24,426][105692] Updated weights for policy 0, policy_version 37532 (0.0011) [2023-12-26 15:38:24,481][105692] Updated weights for policy 0, policy_version 37542 (0.0010) [2023-12-26 15:38:24,535][105692] Updated weights for policy 0, policy_version 37552 (0.0010) [2023-12-26 15:38:24,872][105620] Updated weights for policy 1, policy_version 37849 (0.0010) [2023-12-26 15:38:24,930][105620] Updated weights for policy 1, policy_version 37859 (0.0010) [2023-12-26 15:38:24,992][105620] Updated weights for policy 1, policy_version 37869 (0.0010) [2023-12-26 15:38:25,126][105692] Updated weights for policy 0, policy_version 37562 (0.0010) [2023-12-26 15:38:25,183][105692] Updated weights for policy 0, policy_version 37572 (0.0010) [2023-12-26 15:38:25,237][105692] Updated weights for policy 0, policy_version 37582 (0.0010) [2023-12-26 15:38:25,736][105620] Updated weights for policy 1, policy_version 37879 (0.0010) [2023-12-26 15:38:25,791][105620] Updated weights for policy 1, policy_version 37889 (0.0010) [2023-12-26 15:38:25,853][105620] Updated weights for policy 1, policy_version 37899 (0.0010) [2023-12-26 15:38:25,970][105692] Updated weights for policy 0, policy_version 37592 (0.0010) [2023-12-26 15:38:26,034][105692] Updated weights for policy 0, policy_version 37602 (0.0009) [2023-12-26 15:38:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19797.2, 300 sec: 19605.2). Total num frames: 19333120. Throughput: 0: 9814.8, 1: 9946.4. Samples: 19343120. Policy #0 lag: (min: 2.0, avg: 13.7, max: 34.0) [2023-12-26 15:38:26,063][104569] Avg episode reward: [(0, '2101.282'), (1, '9159.307')] [2023-12-26 15:38:26,088][105692] Updated weights for policy 0, policy_version 37612 (0.0005) [2023-12-26 15:38:26,638][105620] Updated weights for policy 1, policy_version 37909 (0.0009) [2023-12-26 15:38:26,666][105692] Updated weights for policy 0, policy_version 37622 (0.0008) [2023-12-26 15:38:26,684][105620] Updated weights for policy 1, policy_version 37919 (0.0009) [2023-12-26 15:38:26,711][105692] Updated weights for policy 0, policy_version 37632 (0.0010) [2023-12-26 15:38:26,729][105620] Updated weights for policy 1, policy_version 37929 (0.0005) [2023-12-26 15:38:26,758][105692] Updated weights for policy 0, policy_version 37642 (0.0010) [2023-12-26 15:38:27,340][105620] Updated weights for policy 1, policy_version 37939 (0.0006) [2023-12-26 15:38:27,399][105620] Updated weights for policy 1, policy_version 37949 (0.0008) [2023-12-26 15:38:27,453][105620] Updated weights for policy 1, policy_version 37959 (0.0007) [2023-12-26 15:38:27,529][105692] Updated weights for policy 0, policy_version 37652 (0.0010) [2023-12-26 15:38:27,573][105692] Updated weights for policy 0, policy_version 37662 (0.0010) [2023-12-26 15:38:27,637][105692] Updated weights for policy 0, policy_version 37672 (0.0010) [2023-12-26 15:38:28,190][105620] Updated weights for policy 1, policy_version 37969 (0.0008) [2023-12-26 15:38:28,238][105620] Updated weights for policy 1, policy_version 37979 (0.0008) [2023-12-26 15:38:28,285][105620] Updated weights for policy 1, policy_version 37990 (0.0008) [2023-12-26 15:38:28,344][105620] Updated weights for policy 1, policy_version 38000 (0.0008) [2023-12-26 15:38:28,374][105692] Updated weights for policy 0, policy_version 37682 (0.0010) [2023-12-26 15:38:28,421][105692] Updated weights for policy 0, policy_version 37692 (0.0010) [2023-12-26 15:38:28,473][105692] Updated weights for policy 0, policy_version 37702 (0.0008) [2023-12-26 15:38:28,527][105692] Updated weights for policy 0, policy_version 37712 (0.0007) [2023-12-26 15:38:29,121][105620] Updated weights for policy 1, policy_version 38010 (0.0006) [2023-12-26 15:38:29,140][105692] Updated weights for policy 0, policy_version 37722 (0.0006) [2023-12-26 15:38:29,192][105620] Updated weights for policy 1, policy_version 38020 (0.0005) [2023-12-26 15:38:29,192][105692] Updated weights for policy 0, policy_version 37732 (0.0010) [2023-12-26 15:38:29,255][105692] Updated weights for policy 0, policy_version 37742 (0.0011) [2023-12-26 15:38:29,256][105620] Updated weights for policy 1, policy_version 38030 (0.0008) [2023-12-26 15:38:29,948][105620] Updated weights for policy 1, policy_version 38040 (0.0010) [2023-12-26 15:38:29,998][105692] Updated weights for policy 0, policy_version 37752 (0.0009) [2023-12-26 15:38:30,003][105620] Updated weights for policy 1, policy_version 38050 (0.0010) [2023-12-26 15:38:30,052][105692] Updated weights for policy 0, policy_version 37762 (0.0010) [2023-12-26 15:38:30,055][105620] Updated weights for policy 1, policy_version 38060 (0.0010) [2023-12-26 15:38:30,109][105692] Updated weights for policy 0, policy_version 37772 (0.0010) [2023-12-26 15:38:30,730][105692] Updated weights for policy 0, policy_version 37782 (0.0007) [2023-12-26 15:38:30,785][105692] Updated weights for policy 0, policy_version 37792 (0.0005) [2023-12-26 15:38:30,802][105620] Updated weights for policy 1, policy_version 38070 (0.0010) [2023-12-26 15:38:30,842][105692] Updated weights for policy 0, policy_version 37802 (0.0005) [2023-12-26 15:38:30,846][105620] Updated weights for policy 1, policy_version 38080 (0.0010) [2023-12-26 15:38:30,894][105620] Updated weights for policy 1, policy_version 38090 (0.0010) [2023-12-26 15:38:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19934.0, 300 sec: 19605.3). Total num frames: 19439616. Throughput: 0: 9838.4, 1: 9947.8. Samples: 19402496. Policy #0 lag: (min: 2.0, avg: 13.7, max: 34.0) [2023-12-26 15:38:31,062][104569] Avg episode reward: [(0, '2403.786'), (1, '9164.305')] [2023-12-26 15:38:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000037808_9682944.pth... [2023-12-26 15:38:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000038096_9756672.pth... [2023-12-26 15:38:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000036656_9388032.pth [2023-12-26 15:38:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000036944_9461760.pth [2023-12-26 15:38:31,464][105692] Updated weights for policy 0, policy_version 37812 (0.0006) [2023-12-26 15:38:31,525][105692] Updated weights for policy 0, policy_version 37822 (0.0008) [2023-12-26 15:38:31,589][105692] Updated weights for policy 0, policy_version 37832 (0.0010) [2023-12-26 15:38:31,636][105620] Updated weights for policy 1, policy_version 38100 (0.0009) [2023-12-26 15:38:31,697][105620] Updated weights for policy 1, policy_version 38110 (0.0009) [2023-12-26 15:38:31,760][105620] Updated weights for policy 1, policy_version 38120 (0.0008) [2023-12-26 15:38:32,273][105692] Updated weights for policy 0, policy_version 37842 (0.0010) [2023-12-26 15:38:32,333][105692] Updated weights for policy 0, policy_version 37852 (0.0006) [2023-12-26 15:38:32,404][105692] Updated weights for policy 0, policy_version 37862 (0.0009) [2023-12-26 15:38:32,456][105692] Updated weights for policy 0, policy_version 37872 (0.0006) [2023-12-26 15:38:32,564][105620] Updated weights for policy 1, policy_version 38130 (0.0008) [2023-12-26 15:38:32,611][105620] Updated weights for policy 1, policy_version 38140 (0.0005) [2023-12-26 15:38:32,664][105620] Updated weights for policy 1, policy_version 38150 (0.0009) [2023-12-26 15:38:32,717][105620] Updated weights for policy 1, policy_version 38160 (0.0010) [2023-12-26 15:38:33,065][105692] Updated weights for policy 0, policy_version 37882 (0.0005) [2023-12-26 15:38:33,129][105692] Updated weights for policy 0, policy_version 37892 (0.0008) [2023-12-26 15:38:33,190][105692] Updated weights for policy 0, policy_version 37902 (0.0008) [2023-12-26 15:38:33,479][105620] Updated weights for policy 1, policy_version 38170 (0.0010) [2023-12-26 15:38:33,541][105620] Updated weights for policy 1, policy_version 38180 (0.0010) [2023-12-26 15:38:33,599][105620] Updated weights for policy 1, policy_version 38190 (0.0010) [2023-12-26 15:38:33,922][105692] Updated weights for policy 0, policy_version 37912 (0.0009) [2023-12-26 15:38:33,981][105692] Updated weights for policy 0, policy_version 37922 (0.0009) [2023-12-26 15:38:34,038][105692] Updated weights for policy 0, policy_version 37932 (0.0009) [2023-12-26 15:38:34,345][105620] Updated weights for policy 1, policy_version 38200 (0.0009) [2023-12-26 15:38:34,416][105620] Updated weights for policy 1, policy_version 38210 (0.0009) [2023-12-26 15:38:34,480][105620] Updated weights for policy 1, policy_version 38220 (0.0009) [2023-12-26 15:38:34,697][105692] Updated weights for policy 0, policy_version 37942 (0.0009) [2023-12-26 15:38:34,747][105692] Updated weights for policy 0, policy_version 37952 (0.0009) [2023-12-26 15:38:34,808][105692] Updated weights for policy 0, policy_version 37962 (0.0009) [2023-12-26 15:38:35,251][105620] Updated weights for policy 1, policy_version 38230 (0.0009) [2023-12-26 15:38:35,315][105620] Updated weights for policy 1, policy_version 38240 (0.0009) [2023-12-26 15:38:35,377][105620] Updated weights for policy 1, policy_version 38250 (0.0010) [2023-12-26 15:38:35,498][105692] Updated weights for policy 0, policy_version 37972 (0.0009) [2023-12-26 15:38:35,560][105692] Updated weights for policy 0, policy_version 37982 (0.0010) [2023-12-26 15:38:35,612][105692] Updated weights for policy 0, policy_version 37993 (0.0009) [2023-12-26 15:38:36,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 19529728. Throughput: 0: 9849.1, 1: 9891.1. Samples: 19520976. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 15:38:36,062][104569] Avg episode reward: [(0, '4990.689'), (1, '8906.879')] [2023-12-26 15:38:36,074][105620] Updated weights for policy 1, policy_version 38260 (0.0008) [2023-12-26 15:38:36,130][105620] Updated weights for policy 1, policy_version 38270 (0.0008) [2023-12-26 15:38:36,191][105620] Updated weights for policy 1, policy_version 38280 (0.0009) [2023-12-26 15:38:36,387][105692] Updated weights for policy 0, policy_version 38003 (0.0009) [2023-12-26 15:38:36,450][105692] Updated weights for policy 0, policy_version 38013 (0.0009) [2023-12-26 15:38:36,514][105692] Updated weights for policy 0, policy_version 38023 (0.0009) [2023-12-26 15:38:36,979][105620] Updated weights for policy 1, policy_version 38290 (0.0009) [2023-12-26 15:38:37,033][105620] Updated weights for policy 1, policy_version 38300 (0.0009) [2023-12-26 15:38:37,095][105620] Updated weights for policy 1, policy_version 38310 (0.0008) [2023-12-26 15:38:37,146][105620] Updated weights for policy 1, policy_version 38320 (0.0005) [2023-12-26 15:38:37,225][105692] Updated weights for policy 0, policy_version 38033 (0.0009) [2023-12-26 15:38:37,287][105692] Updated weights for policy 0, policy_version 38043 (0.0010) [2023-12-26 15:38:37,351][105692] Updated weights for policy 0, policy_version 38053 (0.0008) [2023-12-26 15:38:37,409][105692] Updated weights for policy 0, policy_version 38063 (0.0010) [2023-12-26 15:38:37,731][105620] Updated weights for policy 1, policy_version 38330 (0.0009) [2023-12-26 15:38:37,783][105620] Updated weights for policy 1, policy_version 38340 (0.0009) [2023-12-26 15:38:37,830][105620] Updated weights for policy 1, policy_version 38350 (0.0008) [2023-12-26 15:38:38,190][105692] Updated weights for policy 0, policy_version 38073 (0.0011) [2023-12-26 15:38:38,242][105692] Updated weights for policy 0, policy_version 38083 (0.0010) [2023-12-26 15:38:38,310][105692] Updated weights for policy 0, policy_version 38093 (0.0010) [2023-12-26 15:38:38,640][105620] Updated weights for policy 1, policy_version 38360 (0.0008) [2023-12-26 15:38:38,684][105620] Updated weights for policy 1, policy_version 38370 (0.0008) [2023-12-26 15:38:38,732][105620] Updated weights for policy 1, policy_version 38380 (0.0008) [2023-12-26 15:38:39,077][105692] Updated weights for policy 0, policy_version 38103 (0.0011) [2023-12-26 15:38:39,138][105692] Updated weights for policy 0, policy_version 38113 (0.0010) [2023-12-26 15:38:39,189][105692] Updated weights for policy 0, policy_version 38123 (0.0010) [2023-12-26 15:38:39,571][105620] Updated weights for policy 1, policy_version 38390 (0.0008) [2023-12-26 15:38:39,635][105620] Updated weights for policy 1, policy_version 38400 (0.0008) [2023-12-26 15:38:39,702][105620] Updated weights for policy 1, policy_version 38410 (0.0009) [2023-12-26 15:38:39,964][105692] Updated weights for policy 0, policy_version 38133 (0.0009) [2023-12-26 15:38:40,029][105692] Updated weights for policy 0, policy_version 38143 (0.0008) [2023-12-26 15:38:40,091][105692] Updated weights for policy 0, policy_version 38153 (0.0011) [2023-12-26 15:38:40,441][105620] Updated weights for policy 1, policy_version 38420 (0.0008) [2023-12-26 15:38:40,485][105620] Updated weights for policy 1, policy_version 38430 (0.0008) [2023-12-26 15:38:40,535][105620] Updated weights for policy 1, policy_version 38440 (0.0009) [2023-12-26 15:38:40,767][105692] Updated weights for policy 0, policy_version 38163 (0.0009) [2023-12-26 15:38:40,829][105692] Updated weights for policy 0, policy_version 38173 (0.0005) [2023-12-26 15:38:40,892][105692] Updated weights for policy 0, policy_version 38183 (0.0005) [2023-12-26 15:38:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 19628032. Throughput: 0: 9825.9, 1: 9817.6. Samples: 19634800. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 15:38:41,063][104569] Avg episode reward: [(0, '3910.698'), (1, '9088.231')] [2023-12-26 15:38:41,167][105620] Updated weights for policy 1, policy_version 38450 (0.0009) [2023-12-26 15:38:41,238][105620] Updated weights for policy 1, policy_version 38460 (0.0008) [2023-12-26 15:38:41,312][105620] Updated weights for policy 1, policy_version 38470 (0.0008) [2023-12-26 15:38:41,387][105620] Updated weights for policy 1, policy_version 38480 (0.0009) [2023-12-26 15:38:41,537][105692] Updated weights for policy 0, policy_version 38193 (0.0006) [2023-12-26 15:38:41,603][105692] Updated weights for policy 0, policy_version 38203 (0.0010) [2023-12-26 15:38:41,668][105692] Updated weights for policy 0, policy_version 38213 (0.0010) [2023-12-26 15:38:41,729][105692] Updated weights for policy 0, policy_version 38223 (0.0008) [2023-12-26 15:38:42,131][105620] Updated weights for policy 1, policy_version 38490 (0.0008) [2023-12-26 15:38:42,188][105620] Updated weights for policy 1, policy_version 38500 (0.0006) [2023-12-26 15:38:42,246][105620] Updated weights for policy 1, policy_version 38510 (0.0006) [2023-12-26 15:38:42,476][105692] Updated weights for policy 0, policy_version 38233 (0.0007) [2023-12-26 15:38:42,540][105692] Updated weights for policy 0, policy_version 38243 (0.0005) [2023-12-26 15:38:42,604][105692] Updated weights for policy 0, policy_version 38253 (0.0009) [2023-12-26 15:38:42,841][105620] Updated weights for policy 1, policy_version 38520 (0.0009) [2023-12-26 15:38:42,899][105620] Updated weights for policy 1, policy_version 38530 (0.0010) [2023-12-26 15:38:42,953][105620] Updated weights for policy 1, policy_version 38540 (0.0010) [2023-12-26 15:38:43,201][105692] Updated weights for policy 0, policy_version 38263 (0.0009) [2023-12-26 15:38:43,258][105692] Updated weights for policy 0, policy_version 38273 (0.0010) [2023-12-26 15:38:43,313][105692] Updated weights for policy 0, policy_version 38283 (0.0009) [2023-12-26 15:38:43,655][105620] Updated weights for policy 1, policy_version 38550 (0.0007) [2023-12-26 15:38:43,707][105620] Updated weights for policy 1, policy_version 38560 (0.0005) [2023-12-26 15:38:43,770][105620] Updated weights for policy 1, policy_version 38570 (0.0008) [2023-12-26 15:38:44,002][105692] Updated weights for policy 0, policy_version 38293 (0.0006) [2023-12-26 15:38:44,072][105692] Updated weights for policy 0, policy_version 38303 (0.0006) [2023-12-26 15:38:44,138][105692] Updated weights for policy 0, policy_version 38313 (0.0006) [2023-12-26 15:38:44,378][105620] Updated weights for policy 1, policy_version 38580 (0.0010) [2023-12-26 15:38:44,437][105620] Updated weights for policy 1, policy_version 38590 (0.0008) [2023-12-26 15:38:44,494][105620] Updated weights for policy 1, policy_version 38600 (0.0009) [2023-12-26 15:38:44,703][105692] Updated weights for policy 0, policy_version 38323 (0.0006) [2023-12-26 15:38:44,765][105692] Updated weights for policy 0, policy_version 38333 (0.0007) [2023-12-26 15:38:44,832][105692] Updated weights for policy 0, policy_version 38343 (0.0007) [2023-12-26 15:38:45,279][105620] Updated weights for policy 1, policy_version 38610 (0.0008) [2023-12-26 15:38:45,341][105620] Updated weights for policy 1, policy_version 38620 (0.0009) [2023-12-26 15:38:45,400][105620] Updated weights for policy 1, policy_version 38630 (0.0009) [2023-12-26 15:38:45,467][105620] Updated weights for policy 1, policy_version 38640 (0.0009) [2023-12-26 15:38:45,528][105692] Updated weights for policy 0, policy_version 38353 (0.0006) [2023-12-26 15:38:45,591][105692] Updated weights for policy 0, policy_version 38363 (0.0009) [2023-12-26 15:38:45,654][105692] Updated weights for policy 0, policy_version 38373 (0.0010) [2023-12-26 15:38:45,715][105692] Updated weights for policy 0, policy_version 38383 (0.0008) [2023-12-26 15:38:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 19726336. Throughput: 0: 9902.1, 1: 9846.7. Samples: 19695364. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 15:38:46,062][104569] Avg episode reward: [(0, '7248.960'), (1, '9178.419')] [2023-12-26 15:38:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000038384_9830400.pth... [2023-12-26 15:38:46,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000037200_9527296.pth [2023-12-26 15:38:46,118][105620] Updated weights for policy 1, policy_version 38650 (0.0009) [2023-12-26 15:38:46,176][105620] Updated weights for policy 1, policy_version 38660 (0.0009) [2023-12-26 15:38:46,244][105620] Updated weights for policy 1, policy_version 38670 (0.0009) [2023-12-26 15:38:46,253][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000038672_9904128.pth... [2023-12-26 15:38:46,258][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000037520_9609216.pth [2023-12-26 15:38:46,490][105692] Updated weights for policy 0, policy_version 38393 (0.0008) [2023-12-26 15:38:46,540][105692] Updated weights for policy 0, policy_version 38403 (0.0009) [2023-12-26 15:38:46,595][105692] Updated weights for policy 0, policy_version 38413 (0.0008) [2023-12-26 15:38:46,963][105620] Updated weights for policy 1, policy_version 38680 (0.0006) [2023-12-26 15:38:47,020][105620] Updated weights for policy 1, policy_version 38690 (0.0005) [2023-12-26 15:38:47,075][105620] Updated weights for policy 1, policy_version 38700 (0.0006) [2023-12-26 15:38:47,396][105692] Updated weights for policy 0, policy_version 38423 (0.0007) [2023-12-26 15:38:47,441][105692] Updated weights for policy 0, policy_version 38433 (0.0005) [2023-12-26 15:38:47,487][105692] Updated weights for policy 0, policy_version 38443 (0.0005) [2023-12-26 15:38:47,692][105620] Updated weights for policy 1, policy_version 38710 (0.0008) [2023-12-26 15:38:47,743][105620] Updated weights for policy 1, policy_version 38720 (0.0008) [2023-12-26 15:38:47,795][105620] Updated weights for policy 1, policy_version 38731 (0.0010) [2023-12-26 15:38:48,084][105692] Updated weights for policy 0, policy_version 38453 (0.0008) [2023-12-26 15:38:48,134][105692] Updated weights for policy 0, policy_version 38463 (0.0006) [2023-12-26 15:38:48,180][105692] Updated weights for policy 0, policy_version 38473 (0.0005) [2023-12-26 15:38:48,453][105620] Updated weights for policy 1, policy_version 38741 (0.0009) [2023-12-26 15:38:48,509][105620] Updated weights for policy 1, policy_version 38751 (0.0009) [2023-12-26 15:38:48,579][105620] Updated weights for policy 1, policy_version 38761 (0.0011) [2023-12-26 15:38:48,853][105692] Updated weights for policy 0, policy_version 38483 (0.0005) [2023-12-26 15:38:48,910][105692] Updated weights for policy 0, policy_version 38493 (0.0005) [2023-12-26 15:38:48,970][105692] Updated weights for policy 0, policy_version 38503 (0.0005) [2023-12-26 15:38:49,319][105620] Updated weights for policy 1, policy_version 38771 (0.0010) [2023-12-26 15:38:49,388][105620] Updated weights for policy 1, policy_version 38781 (0.0009) [2023-12-26 15:38:49,449][105620] Updated weights for policy 1, policy_version 38791 (0.0009) [2023-12-26 15:38:49,639][105692] Updated weights for policy 0, policy_version 38513 (0.0007) [2023-12-26 15:38:49,703][105692] Updated weights for policy 0, policy_version 38523 (0.0007) [2023-12-26 15:38:49,766][105692] Updated weights for policy 0, policy_version 38533 (0.0005) [2023-12-26 15:38:49,827][105692] Updated weights for policy 0, policy_version 38543 (0.0006) [2023-12-26 15:38:50,112][105620] Updated weights for policy 1, policy_version 38801 (0.0005) [2023-12-26 15:38:50,170][105620] Updated weights for policy 1, policy_version 38811 (0.0011) [2023-12-26 15:38:50,225][105620] Updated weights for policy 1, policy_version 38821 (0.0009) [2023-12-26 15:38:50,287][105620] Updated weights for policy 1, policy_version 38831 (0.0011) [2023-12-26 15:38:50,495][105692] Updated weights for policy 0, policy_version 38553 (0.0006) [2023-12-26 15:38:50,561][105692] Updated weights for policy 0, policy_version 38563 (0.0006) [2023-12-26 15:38:50,631][105692] Updated weights for policy 0, policy_version 38573 (0.0008) [2023-12-26 15:38:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 19824640. Throughput: 0: 10031.8, 1: 9865.7. Samples: 19817140. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 15:38:51,062][104569] Avg episode reward: [(0, '9343.325'), (1, '9089.060')] [2023-12-26 15:38:51,095][105620] Updated weights for policy 1, policy_version 38841 (0.0008) [2023-12-26 15:38:51,160][105620] Updated weights for policy 1, policy_version 38851 (0.0009) [2023-12-26 15:38:51,222][105620] Updated weights for policy 1, policy_version 38861 (0.0008) [2023-12-26 15:38:51,238][105692] Updated weights for policy 0, policy_version 38583 (0.0009) [2023-12-26 15:38:51,302][105692] Updated weights for policy 0, policy_version 38593 (0.0008) [2023-12-26 15:38:51,367][105692] Updated weights for policy 0, policy_version 38603 (0.0009) [2023-12-26 15:38:51,977][105620] Updated weights for policy 1, policy_version 38871 (0.0006) [2023-12-26 15:38:52,046][105620] Updated weights for policy 1, policy_version 38881 (0.0006) [2023-12-26 15:38:52,113][105620] Updated weights for policy 1, policy_version 38891 (0.0005) [2023-12-26 15:38:52,167][105692] Updated weights for policy 0, policy_version 38613 (0.0009) [2023-12-26 15:38:52,230][105692] Updated weights for policy 0, policy_version 38623 (0.0007) [2023-12-26 15:38:52,302][105692] Updated weights for policy 0, policy_version 38633 (0.0009) [2023-12-26 15:38:52,796][105620] Updated weights for policy 1, policy_version 38901 (0.0008) [2023-12-26 15:38:52,850][105620] Updated weights for policy 1, policy_version 38911 (0.0009) [2023-12-26 15:38:52,908][105620] Updated weights for policy 1, policy_version 38921 (0.0009) [2023-12-26 15:38:52,955][105692] Updated weights for policy 0, policy_version 38643 (0.0009) [2023-12-26 15:38:53,002][105692] Updated weights for policy 0, policy_version 38653 (0.0009) [2023-12-26 15:38:53,056][105692] Updated weights for policy 0, policy_version 38663 (0.0009) [2023-12-26 15:38:53,540][105620] Updated weights for policy 1, policy_version 38931 (0.0008) [2023-12-26 15:38:53,601][105620] Updated weights for policy 1, policy_version 38941 (0.0008) [2023-12-26 15:38:53,661][105620] Updated weights for policy 1, policy_version 38951 (0.0009) [2023-12-26 15:38:53,814][105692] Updated weights for policy 0, policy_version 38673 (0.0009) [2023-12-26 15:38:53,875][105692] Updated weights for policy 0, policy_version 38683 (0.0009) [2023-12-26 15:38:53,936][105692] Updated weights for policy 0, policy_version 38693 (0.0009) [2023-12-26 15:38:53,994][105692] Updated weights for policy 0, policy_version 38703 (0.0009) [2023-12-26 15:38:54,317][105620] Updated weights for policy 1, policy_version 38962 (0.0010) [2023-12-26 15:38:54,375][105620] Updated weights for policy 1, policy_version 38972 (0.0009) [2023-12-26 15:38:54,440][105620] Updated weights for policy 1, policy_version 38982 (0.0009) [2023-12-26 15:38:54,501][105620] Updated weights for policy 1, policy_version 38992 (0.0009) [2023-12-26 15:38:54,749][105692] Updated weights for policy 0, policy_version 38713 (0.0009) [2023-12-26 15:38:54,810][105692] Updated weights for policy 0, policy_version 38723 (0.0009) [2023-12-26 15:38:54,875][105692] Updated weights for policy 0, policy_version 38733 (0.0009) [2023-12-26 15:38:55,236][105620] Updated weights for policy 1, policy_version 39002 (0.0005) [2023-12-26 15:38:55,291][105620] Updated weights for policy 1, policy_version 39012 (0.0005) [2023-12-26 15:38:55,346][105620] Updated weights for policy 1, policy_version 39022 (0.0005) [2023-12-26 15:38:55,606][105692] Updated weights for policy 0, policy_version 38743 (0.0009) [2023-12-26 15:38:55,663][105692] Updated weights for policy 0, policy_version 38753 (0.0008) [2023-12-26 15:38:55,713][105692] Updated weights for policy 0, policy_version 38763 (0.0008) [2023-12-26 15:38:55,961][105620] Updated weights for policy 1, policy_version 39032 (0.0009) [2023-12-26 15:38:56,022][105620] Updated weights for policy 1, policy_version 39042 (0.0009) [2023-12-26 15:38:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 19922944. Throughput: 0: 10087.8, 1: 9805.9. Samples: 19934576. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 15:38:56,062][104569] Avg episode reward: [(0, '9345.852'), (1, '9088.416')] [2023-12-26 15:38:56,070][105620] Updated weights for policy 1, policy_version 39052 (0.0008) [2023-12-26 15:38:56,487][105692] Updated weights for policy 0, policy_version 38773 (0.0008) [2023-12-26 15:38:56,553][105692] Updated weights for policy 0, policy_version 38783 (0.0008) [2023-12-26 15:38:56,619][105692] Updated weights for policy 0, policy_version 38793 (0.0008) [2023-12-26 15:38:56,837][105620] Updated weights for policy 1, policy_version 39062 (0.0010) [2023-12-26 15:38:56,896][105620] Updated weights for policy 1, policy_version 39072 (0.0011) [2023-12-26 15:38:56,942][105620] Updated weights for policy 1, policy_version 39082 (0.0009) [2023-12-26 15:38:57,451][105692] Updated weights for policy 0, policy_version 38803 (0.0009) [2023-12-26 15:38:57,488][105620] Updated weights for policy 1, policy_version 39092 (0.0006) [2023-12-26 15:38:57,510][105692] Updated weights for policy 0, policy_version 38813 (0.0008) [2023-12-26 15:38:57,539][105620] Updated weights for policy 1, policy_version 39102 (0.0009) [2023-12-26 15:38:57,569][105692] Updated weights for policy 0, policy_version 38823 (0.0006) [2023-12-26 15:38:57,594][105620] Updated weights for policy 1, policy_version 39112 (0.0010) [2023-12-26 15:38:58,296][105620] Updated weights for policy 1, policy_version 39122 (0.0010) [2023-12-26 15:38:58,330][105692] Updated weights for policy 0, policy_version 38833 (0.0006) [2023-12-26 15:38:58,371][105620] Updated weights for policy 1, policy_version 39132 (0.0009) [2023-12-26 15:38:58,394][105692] Updated weights for policy 0, policy_version 38843 (0.0007) [2023-12-26 15:38:58,432][105620] Updated weights for policy 1, policy_version 39142 (0.0008) [2023-12-26 15:38:58,459][105692] Updated weights for policy 0, policy_version 38853 (0.0007) [2023-12-26 15:38:58,496][105620] Updated weights for policy 1, policy_version 39152 (0.0008) [2023-12-26 15:38:58,519][105692] Updated weights for policy 0, policy_version 38863 (0.0008) [2023-12-26 15:38:59,295][105692] Updated weights for policy 0, policy_version 38873 (0.0008) [2023-12-26 15:38:59,351][105620] Updated weights for policy 1, policy_version 39162 (0.0007) [2023-12-26 15:38:59,354][105692] Updated weights for policy 0, policy_version 38883 (0.0008) [2023-12-26 15:38:59,408][105620] Updated weights for policy 1, policy_version 39172 (0.0006) [2023-12-26 15:38:59,422][105692] Updated weights for policy 0, policy_version 38893 (0.0008) [2023-12-26 15:38:59,458][105620] Updated weights for policy 1, policy_version 39182 (0.0006) [2023-12-26 15:39:00,138][105620] Updated weights for policy 1, policy_version 39192 (0.0008) [2023-12-26 15:39:00,193][105620] Updated weights for policy 1, policy_version 39202 (0.0008) [2023-12-26 15:39:00,216][105692] Updated weights for policy 0, policy_version 38903 (0.0008) [2023-12-26 15:39:00,250][105620] Updated weights for policy 1, policy_version 39212 (0.0007) [2023-12-26 15:39:00,266][105692] Updated weights for policy 0, policy_version 38913 (0.0008) [2023-12-26 15:39:00,319][105692] Updated weights for policy 0, policy_version 38923 (0.0009) [2023-12-26 15:39:00,813][105620] Updated weights for policy 1, policy_version 39222 (0.0005) [2023-12-26 15:39:00,869][105620] Updated weights for policy 1, policy_version 39232 (0.0010) [2023-12-26 15:39:00,926][105620] Updated weights for policy 1, policy_version 39242 (0.0010) [2023-12-26 15:39:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 20021248. Throughput: 0: 9998.1, 1: 9817.9. Samples: 19990408. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 15:39:01,063][104569] Avg episode reward: [(0, '9347.208'), (1, '9087.084')] [2023-12-26 15:39:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000038928_9969664.pth... [2023-12-26 15:39:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000039248_10051584.pth... [2023-12-26 15:39:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000037808_9682944.pth [2023-12-26 15:39:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000038096_9756672.pth [2023-12-26 15:39:01,192][105692] Updated weights for policy 0, policy_version 38934 (0.0008) [2023-12-26 15:39:01,237][105692] Updated weights for policy 0, policy_version 38944 (0.0008) [2023-12-26 15:39:01,293][105692] Updated weights for policy 0, policy_version 38954 (0.0009) [2023-12-26 15:39:01,643][105620] Updated weights for policy 1, policy_version 39252 (0.0009) [2023-12-26 15:39:01,697][105620] Updated weights for policy 1, policy_version 39262 (0.0008) [2023-12-26 15:39:01,767][105620] Updated weights for policy 1, policy_version 39272 (0.0009) [2023-12-26 15:39:02,048][105692] Updated weights for policy 0, policy_version 38964 (0.0010) [2023-12-26 15:39:02,097][105692] Updated weights for policy 0, policy_version 38974 (0.0010) [2023-12-26 15:39:02,148][105692] Updated weights for policy 0, policy_version 38984 (0.0010) [2023-12-26 15:39:02,377][105620] Updated weights for policy 1, policy_version 39282 (0.0008) [2023-12-26 15:39:02,427][105620] Updated weights for policy 1, policy_version 39292 (0.0005) [2023-12-26 15:39:02,479][105620] Updated weights for policy 1, policy_version 39302 (0.0005) [2023-12-26 15:39:02,542][105620] Updated weights for policy 1, policy_version 39312 (0.0005) [2023-12-26 15:39:02,750][105692] Updated weights for policy 0, policy_version 38994 (0.0009) [2023-12-26 15:39:02,810][105692] Updated weights for policy 0, policy_version 39004 (0.0010) [2023-12-26 15:39:02,870][105692] Updated weights for policy 0, policy_version 39014 (0.0006) [2023-12-26 15:39:02,917][105692] Updated weights for policy 0, policy_version 39024 (0.0005) [2023-12-26 15:39:03,118][105620] Updated weights for policy 1, policy_version 39322 (0.0010) [2023-12-26 15:39:03,174][105620] Updated weights for policy 1, policy_version 39332 (0.0009) [2023-12-26 15:39:03,232][105620] Updated weights for policy 1, policy_version 39342 (0.0010) [2023-12-26 15:39:03,459][105692] Updated weights for policy 0, policy_version 39034 (0.0005) [2023-12-26 15:39:03,516][105692] Updated weights for policy 0, policy_version 39044 (0.0009) [2023-12-26 15:39:03,564][105692] Updated weights for policy 0, policy_version 39054 (0.0008) [2023-12-26 15:39:04,068][105620] Updated weights for policy 1, policy_version 39352 (0.0007) [2023-12-26 15:39:04,126][105620] Updated weights for policy 1, policy_version 39362 (0.0008) [2023-12-26 15:39:04,181][105620] Updated weights for policy 1, policy_version 39372 (0.0008) [2023-12-26 15:39:04,197][105692] Updated weights for policy 0, policy_version 39064 (0.0010) [2023-12-26 15:39:04,256][105692] Updated weights for policy 0, policy_version 39074 (0.0010) [2023-12-26 15:39:04,317][105692] Updated weights for policy 0, policy_version 39084 (0.0010) [2023-12-26 15:39:04,799][105620] Updated weights for policy 1, policy_version 39382 (0.0009) [2023-12-26 15:39:04,858][105620] Updated weights for policy 1, policy_version 39392 (0.0005) [2023-12-26 15:39:04,920][105620] Updated weights for policy 1, policy_version 39402 (0.0005) [2023-12-26 15:39:05,056][105692] Updated weights for policy 0, policy_version 39094 (0.0010) [2023-12-26 15:39:05,123][105692] Updated weights for policy 0, policy_version 39104 (0.0006) [2023-12-26 15:39:05,189][105692] Updated weights for policy 0, policy_version 39114 (0.0005) [2023-12-26 15:39:05,518][105620] Updated weights for policy 1, policy_version 39412 (0.0008) [2023-12-26 15:39:05,573][105620] Updated weights for policy 1, policy_version 39422 (0.0010) [2023-12-26 15:39:05,624][105620] Updated weights for policy 1, policy_version 39432 (0.0010) [2023-12-26 15:39:05,734][105692] Updated weights for policy 0, policy_version 39124 (0.0005) [2023-12-26 15:39:05,794][105692] Updated weights for policy 0, policy_version 39134 (0.0005) [2023-12-26 15:39:05,850][105692] Updated weights for policy 0, policy_version 39144 (0.0005) [2023-12-26 15:39:06,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19933.8, 300 sec: 19660.8). Total num frames: 20127744. Throughput: 0: 9908.1, 1: 9901.9. Samples: 20112776. Policy #0 lag: (min: 16.0, avg: 45.3, max: 48.0) [2023-12-26 15:39:06,063][104569] Avg episode reward: [(0, '9345.159'), (1, '9177.285')] [2023-12-26 15:39:06,307][105620] Updated weights for policy 1, policy_version 39442 (0.0011) [2023-12-26 15:39:06,383][105620] Updated weights for policy 1, policy_version 39452 (0.0011) [2023-12-26 15:39:06,453][105620] Updated weights for policy 1, policy_version 39462 (0.0011) [2023-12-26 15:39:06,509][105620] Updated weights for policy 1, policy_version 39472 (0.0010) [2023-12-26 15:39:06,531][105692] Updated weights for policy 0, policy_version 39154 (0.0007) [2023-12-26 15:39:06,579][105692] Updated weights for policy 0, policy_version 39164 (0.0008) [2023-12-26 15:39:06,642][105692] Updated weights for policy 0, policy_version 39174 (0.0008) [2023-12-26 15:39:06,706][105692] Updated weights for policy 0, policy_version 39184 (0.0008) [2023-12-26 15:39:07,227][105620] Updated weights for policy 1, policy_version 39482 (0.0011) [2023-12-26 15:39:07,293][105620] Updated weights for policy 1, policy_version 39492 (0.0011) [2023-12-26 15:39:07,342][105620] Updated weights for policy 1, policy_version 39502 (0.0010) [2023-12-26 15:39:07,407][105692] Updated weights for policy 0, policy_version 39194 (0.0008) [2023-12-26 15:39:07,467][105692] Updated weights for policy 0, policy_version 39204 (0.0008) [2023-12-26 15:39:07,525][105692] Updated weights for policy 0, policy_version 39214 (0.0005) [2023-12-26 15:39:07,972][105620] Updated weights for policy 1, policy_version 39512 (0.0009) [2023-12-26 15:39:08,025][105620] Updated weights for policy 1, policy_version 39522 (0.0010) [2023-12-26 15:39:08,089][105620] Updated weights for policy 1, policy_version 39532 (0.0009) [2023-12-26 15:39:08,258][105692] Updated weights for policy 0, policy_version 39224 (0.0008) [2023-12-26 15:39:08,316][105692] Updated weights for policy 0, policy_version 39234 (0.0009) [2023-12-26 15:39:08,385][105692] Updated weights for policy 0, policy_version 39244 (0.0008) [2023-12-26 15:39:08,851][105620] Updated weights for policy 1, policy_version 39542 (0.0008) [2023-12-26 15:39:08,906][105620] Updated weights for policy 1, policy_version 39552 (0.0009) [2023-12-26 15:39:08,958][105620] Updated weights for policy 1, policy_version 39562 (0.0009) [2023-12-26 15:39:09,042][105692] Updated weights for policy 0, policy_version 39254 (0.0009) [2023-12-26 15:39:09,101][105692] Updated weights for policy 0, policy_version 39264 (0.0009) [2023-12-26 15:39:09,156][105692] Updated weights for policy 0, policy_version 39274 (0.0008) [2023-12-26 15:39:09,739][105620] Updated weights for policy 1, policy_version 39572 (0.0010) [2023-12-26 15:39:09,812][105620] Updated weights for policy 1, policy_version 39582 (0.0009) [2023-12-26 15:39:09,881][105620] Updated weights for policy 1, policy_version 39592 (0.0009) [2023-12-26 15:39:09,964][105692] Updated weights for policy 0, policy_version 39284 (0.0006) [2023-12-26 15:39:10,019][105692] Updated weights for policy 0, policy_version 39294 (0.0006) [2023-12-26 15:39:10,074][105692] Updated weights for policy 0, policy_version 39304 (0.0006) [2023-12-26 15:39:10,538][105620] Updated weights for policy 1, policy_version 39602 (0.0010) [2023-12-26 15:39:10,594][105620] Updated weights for policy 1, policy_version 39612 (0.0011) [2023-12-26 15:39:10,660][105620] Updated weights for policy 1, policy_version 39622 (0.0011) [2023-12-26 15:39:10,667][105692] Updated weights for policy 0, policy_version 39314 (0.0006) [2023-12-26 15:39:10,718][105692] Updated weights for policy 0, policy_version 39324 (0.0006) [2023-12-26 15:39:10,723][105620] Updated weights for policy 1, policy_version 39632 (0.0011) [2023-12-26 15:39:10,769][105692] Updated weights for policy 0, policy_version 39334 (0.0006) [2023-12-26 15:39:10,830][105692] Updated weights for policy 0, policy_version 39344 (0.0006) [2023-12-26 15:39:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 20226048. Throughput: 0: 9883.6, 1: 9883.4. Samples: 20232632. Policy #0 lag: (min: 16.0, avg: 45.3, max: 48.0) [2023-12-26 15:39:11,062][104569] Avg episode reward: [(0, '9250.589'), (1, '9093.546')] [2023-12-26 15:39:11,467][105692] Updated weights for policy 0, policy_version 39354 (0.0008) [2023-12-26 15:39:11,491][105620] Updated weights for policy 1, policy_version 39642 (0.0008) [2023-12-26 15:39:11,524][105692] Updated weights for policy 0, policy_version 39364 (0.0009) [2023-12-26 15:39:11,555][105620] Updated weights for policy 1, policy_version 39652 (0.0006) [2023-12-26 15:39:11,582][105692] Updated weights for policy 0, policy_version 39374 (0.0008) [2023-12-26 15:39:11,617][105620] Updated weights for policy 1, policy_version 39662 (0.0006) [2023-12-26 15:39:12,339][105692] Updated weights for policy 0, policy_version 39384 (0.0009) [2023-12-26 15:39:12,389][105620] Updated weights for policy 1, policy_version 39672 (0.0009) [2023-12-26 15:39:12,404][105692] Updated weights for policy 0, policy_version 39394 (0.0008) [2023-12-26 15:39:12,446][105620] Updated weights for policy 1, policy_version 39682 (0.0007) [2023-12-26 15:39:12,460][105692] Updated weights for policy 0, policy_version 39404 (0.0009) [2023-12-26 15:39:12,510][105620] Updated weights for policy 1, policy_version 39692 (0.0005) [2023-12-26 15:39:13,110][105620] Updated weights for policy 1, policy_version 39702 (0.0009) [2023-12-26 15:39:13,162][105620] Updated weights for policy 1, policy_version 39712 (0.0011) [2023-12-26 15:39:13,222][105620] Updated weights for policy 1, policy_version 39722 (0.0011) [2023-12-26 15:39:13,297][105692] Updated weights for policy 0, policy_version 39414 (0.0009) [2023-12-26 15:39:13,357][105692] Updated weights for policy 0, policy_version 39424 (0.0008) [2023-12-26 15:39:13,417][105692] Updated weights for policy 0, policy_version 39434 (0.0009) [2023-12-26 15:39:13,966][105620] Updated weights for policy 1, policy_version 39732 (0.0010) [2023-12-26 15:39:14,027][105620] Updated weights for policy 1, policy_version 39742 (0.0007) [2023-12-26 15:39:14,065][105692] Updated weights for policy 0, policy_version 39444 (0.0008) [2023-12-26 15:39:14,089][105620] Updated weights for policy 1, policy_version 39752 (0.0010) [2023-12-26 15:39:14,127][105692] Updated weights for policy 0, policy_version 39454 (0.0011) [2023-12-26 15:39:14,186][105692] Updated weights for policy 0, policy_version 39464 (0.0011) [2023-12-26 15:39:14,755][105620] Updated weights for policy 1, policy_version 39762 (0.0010) [2023-12-26 15:39:14,823][105620] Updated weights for policy 1, policy_version 39772 (0.0009) [2023-12-26 15:39:14,852][105692] Updated weights for policy 0, policy_version 39474 (0.0010) [2023-12-26 15:39:14,888][105620] Updated weights for policy 1, policy_version 39782 (0.0008) [2023-12-26 15:39:14,910][105692] Updated weights for policy 0, policy_version 39484 (0.0011) [2023-12-26 15:39:14,951][105620] Updated weights for policy 1, policy_version 39792 (0.0006) [2023-12-26 15:39:14,965][105692] Updated weights for policy 0, policy_version 39494 (0.0011) [2023-12-26 15:39:15,021][105692] Updated weights for policy 0, policy_version 39504 (0.0011) [2023-12-26 15:39:15,740][105620] Updated weights for policy 1, policy_version 39802 (0.0009) [2023-12-26 15:39:15,775][105692] Updated weights for policy 0, policy_version 39514 (0.0006) [2023-12-26 15:39:15,799][105620] Updated weights for policy 1, policy_version 39812 (0.0010) [2023-12-26 15:39:15,835][105692] Updated weights for policy 0, policy_version 39524 (0.0007) [2023-12-26 15:39:15,851][105620] Updated weights for policy 1, policy_version 39822 (0.0008) [2023-12-26 15:39:15,892][105692] Updated weights for policy 0, policy_version 39534 (0.0008) [2023-12-26 15:39:16,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 20324352. Throughput: 0: 9847.6, 1: 9878.8. Samples: 20290188. Policy #0 lag: (min: 16.0, avg: 45.3, max: 48.0) [2023-12-26 15:39:16,062][104569] Avg episode reward: [(0, '9248.515'), (1, '9010.129')] [2023-12-26 15:39:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000039824_10199040.pth... [2023-12-26 15:39:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000039536_10125312.pth... [2023-12-26 15:39:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000038672_9904128.pth [2023-12-26 15:39:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000038384_9830400.pth [2023-12-26 15:39:16,572][105620] Updated weights for policy 1, policy_version 39832 (0.0008) [2023-12-26 15:39:16,626][105620] Updated weights for policy 1, policy_version 39842 (0.0009) [2023-12-26 15:39:16,641][105692] Updated weights for policy 0, policy_version 39544 (0.0009) [2023-12-26 15:39:16,676][105620] Updated weights for policy 1, policy_version 39852 (0.0006) [2023-12-26 15:39:16,699][105692] Updated weights for policy 0, policy_version 39554 (0.0008) [2023-12-26 15:39:16,747][105692] Updated weights for policy 0, policy_version 39564 (0.0009) [2023-12-26 15:39:17,431][105620] Updated weights for policy 1, policy_version 39862 (0.0008) [2023-12-26 15:39:17,488][105620] Updated weights for policy 1, policy_version 39872 (0.0008) [2023-12-26 15:39:17,511][105692] Updated weights for policy 0, policy_version 39574 (0.0008) [2023-12-26 15:39:17,542][105620] Updated weights for policy 1, policy_version 39882 (0.0007) [2023-12-26 15:39:17,568][105692] Updated weights for policy 0, policy_version 39584 (0.0008) [2023-12-26 15:39:17,614][105692] Updated weights for policy 0, policy_version 39594 (0.0008) [2023-12-26 15:39:18,214][105620] Updated weights for policy 1, policy_version 39892 (0.0006) [2023-12-26 15:39:18,277][105620] Updated weights for policy 1, policy_version 39902 (0.0005) [2023-12-26 15:39:18,331][105620] Updated weights for policy 1, policy_version 39912 (0.0008) [2023-12-26 15:39:18,413][105692] Updated weights for policy 0, policy_version 39604 (0.0009) [2023-12-26 15:39:18,476][105692] Updated weights for policy 0, policy_version 39614 (0.0008) [2023-12-26 15:39:18,538][105692] Updated weights for policy 0, policy_version 39624 (0.0005) [2023-12-26 15:39:19,004][105620] Updated weights for policy 1, policy_version 39922 (0.0008) [2023-12-26 15:39:19,063][105620] Updated weights for policy 1, policy_version 39932 (0.0008) [2023-12-26 15:39:19,126][105620] Updated weights for policy 1, policy_version 39942 (0.0010) [2023-12-26 15:39:19,184][105620] Updated weights for policy 1, policy_version 39952 (0.0010) [2023-12-26 15:39:19,246][105692] Updated weights for policy 0, policy_version 39634 (0.0009) [2023-12-26 15:39:19,298][105692] Updated weights for policy 0, policy_version 39644 (0.0011) [2023-12-26 15:39:19,359][105692] Updated weights for policy 0, policy_version 39654 (0.0011) [2023-12-26 15:39:19,427][105692] Updated weights for policy 0, policy_version 39664 (0.0008) [2023-12-26 15:39:19,915][105620] Updated weights for policy 1, policy_version 39962 (0.0009) [2023-12-26 15:39:19,982][105620] Updated weights for policy 1, policy_version 39972 (0.0008) [2023-12-26 15:39:20,048][105620] Updated weights for policy 1, policy_version 39982 (0.0007) [2023-12-26 15:39:20,108][105692] Updated weights for policy 0, policy_version 39674 (0.0007) [2023-12-26 15:39:20,177][105692] Updated weights for policy 0, policy_version 39684 (0.0008) [2023-12-26 15:39:20,246][105692] Updated weights for policy 0, policy_version 39694 (0.0005) [2023-12-26 15:39:20,722][105620] Updated weights for policy 1, policy_version 39992 (0.0008) [2023-12-26 15:39:20,778][105620] Updated weights for policy 1, policy_version 40002 (0.0008) [2023-12-26 15:39:20,840][105620] Updated weights for policy 1, policy_version 40012 (0.0007) [2023-12-26 15:39:20,857][105692] Updated weights for policy 0, policy_version 39704 (0.0010) [2023-12-26 15:39:20,925][105692] Updated weights for policy 0, policy_version 39714 (0.0011) [2023-12-26 15:39:20,981][105692] Updated weights for policy 0, policy_version 39724 (0.0011) [2023-12-26 15:39:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 20422656. Throughput: 0: 9754.8, 1: 9932.1. Samples: 20406888. Policy #0 lag: (min: 16.0, avg: 45.3, max: 48.0) [2023-12-26 15:39:21,062][104569] Avg episode reward: [(0, '9338.779'), (1, '9007.803')] [2023-12-26 15:39:21,557][105620] Updated weights for policy 1, policy_version 40022 (0.0007) [2023-12-26 15:39:21,610][105620] Updated weights for policy 1, policy_version 40032 (0.0008) [2023-12-26 15:39:21,675][105620] Updated weights for policy 1, policy_version 40042 (0.0008) [2023-12-26 15:39:21,737][105692] Updated weights for policy 0, policy_version 39734 (0.0010) [2023-12-26 15:39:21,801][105692] Updated weights for policy 0, policy_version 39744 (0.0011) [2023-12-26 15:39:21,865][105692] Updated weights for policy 0, policy_version 39754 (0.0011) [2023-12-26 15:39:22,493][105620] Updated weights for policy 1, policy_version 40052 (0.0008) [2023-12-26 15:39:22,547][105620] Updated weights for policy 1, policy_version 40062 (0.0008) [2023-12-26 15:39:22,588][105692] Updated weights for policy 0, policy_version 39764 (0.0009) [2023-12-26 15:39:22,605][105620] Updated weights for policy 1, policy_version 40072 (0.0009) [2023-12-26 15:39:22,651][105692] Updated weights for policy 0, policy_version 39774 (0.0010) [2023-12-26 15:39:22,714][105692] Updated weights for policy 0, policy_version 39784 (0.0011) [2023-12-26 15:39:23,289][105620] Updated weights for policy 1, policy_version 40082 (0.0006) [2023-12-26 15:39:23,351][105620] Updated weights for policy 1, policy_version 40092 (0.0008) [2023-12-26 15:39:23,410][105620] Updated weights for policy 1, policy_version 40102 (0.0008) [2023-12-26 15:39:23,423][105692] Updated weights for policy 0, policy_version 39794 (0.0010) [2023-12-26 15:39:23,471][105620] Updated weights for policy 1, policy_version 40112 (0.0007) [2023-12-26 15:39:23,475][105692] Updated weights for policy 0, policy_version 39804 (0.0006) [2023-12-26 15:39:23,524][105692] Updated weights for policy 0, policy_version 39814 (0.0008) [2023-12-26 15:39:23,580][105692] Updated weights for policy 0, policy_version 39824 (0.0008) [2023-12-26 15:39:24,193][105620] Updated weights for policy 1, policy_version 40122 (0.0010) [2023-12-26 15:39:24,246][105620] Updated weights for policy 1, policy_version 40132 (0.0009) [2023-12-26 15:39:24,296][105620] Updated weights for policy 1, policy_version 40143 (0.0008) [2023-12-26 15:39:24,299][105692] Updated weights for policy 0, policy_version 39834 (0.0005) [2023-12-26 15:39:24,358][105692] Updated weights for policy 0, policy_version 39844 (0.0006) [2023-12-26 15:39:24,418][105692] Updated weights for policy 0, policy_version 39854 (0.0005) [2023-12-26 15:39:25,080][105620] Updated weights for policy 1, policy_version 40153 (0.0007) [2023-12-26 15:39:25,109][105692] Updated weights for policy 0, policy_version 39864 (0.0008) [2023-12-26 15:39:25,138][105620] Updated weights for policy 1, policy_version 40163 (0.0008) [2023-12-26 15:39:25,187][105692] Updated weights for policy 0, policy_version 39874 (0.0006) [2023-12-26 15:39:25,196][105620] Updated weights for policy 1, policy_version 40173 (0.0008) [2023-12-26 15:39:25,245][105692] Updated weights for policy 0, policy_version 39884 (0.0008) [2023-12-26 15:39:25,863][105692] Updated weights for policy 0, policy_version 39894 (0.0007) [2023-12-26 15:39:25,874][105620] Updated weights for policy 1, policy_version 40183 (0.0007) [2023-12-26 15:39:25,908][105692] Updated weights for policy 0, policy_version 39904 (0.0005) [2023-12-26 15:39:25,938][105620] Updated weights for policy 1, policy_version 40193 (0.0006) [2023-12-26 15:39:25,963][105692] Updated weights for policy 0, policy_version 39914 (0.0005) [2023-12-26 15:39:25,991][105620] Updated weights for policy 1, policy_version 40203 (0.0006) [2023-12-26 15:39:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 20520960. Throughput: 0: 9816.2, 1: 9933.9. Samples: 20523552. Policy #0 lag: (min: 16.0, avg: 45.3, max: 48.0) [2023-12-26 15:39:26,062][104569] Avg episode reward: [(0, '9246.358'), (1, '9268.257')] [2023-12-26 15:39:26,662][105620] Updated weights for policy 1, policy_version 40213 (0.0007) [2023-12-26 15:39:26,680][105692] Updated weights for policy 0, policy_version 39924 (0.0006) [2023-12-26 15:39:26,717][105620] Updated weights for policy 1, policy_version 40223 (0.0009) [2023-12-26 15:39:26,728][105692] Updated weights for policy 0, policy_version 39934 (0.0005) [2023-12-26 15:39:26,763][105620] Updated weights for policy 1, policy_version 40233 (0.0008) [2023-12-26 15:39:26,788][105692] Updated weights for policy 0, policy_version 39945 (0.0007) [2023-12-26 15:39:27,363][105692] Updated weights for policy 0, policy_version 39955 (0.0006) [2023-12-26 15:39:27,411][105692] Updated weights for policy 0, policy_version 39965 (0.0005) [2023-12-26 15:39:27,459][105692] Updated weights for policy 0, policy_version 39975 (0.0005) [2023-12-26 15:39:27,647][105620] Updated weights for policy 1, policy_version 40243 (0.0008) [2023-12-26 15:39:27,693][105620] Updated weights for policy 1, policy_version 40253 (0.0009) [2023-12-26 15:39:27,739][105620] Updated weights for policy 1, policy_version 40263 (0.0009) [2023-12-26 15:39:28,040][105692] Updated weights for policy 0, policy_version 39985 (0.0005) [2023-12-26 15:39:28,100][105692] Updated weights for policy 0, policy_version 39995 (0.0008) [2023-12-26 15:39:28,155][105692] Updated weights for policy 0, policy_version 40005 (0.0008) [2023-12-26 15:39:28,206][105692] Updated weights for policy 0, policy_version 40015 (0.0008) [2023-12-26 15:39:28,498][105620] Updated weights for policy 1, policy_version 40273 (0.0009) [2023-12-26 15:39:28,546][105620] Updated weights for policy 1, policy_version 40283 (0.0011) [2023-12-26 15:39:28,598][105620] Updated weights for policy 1, policy_version 40293 (0.0011) [2023-12-26 15:39:28,657][105620] Updated weights for policy 1, policy_version 40303 (0.0011) [2023-12-26 15:39:28,987][105692] Updated weights for policy 0, policy_version 40025 (0.0008) [2023-12-26 15:39:29,038][105692] Updated weights for policy 0, policy_version 40035 (0.0009) [2023-12-26 15:39:29,092][105692] Updated weights for policy 0, policy_version 40045 (0.0008) [2023-12-26 15:39:29,437][105620] Updated weights for policy 1, policy_version 40313 (0.0010) [2023-12-26 15:39:29,495][105620] Updated weights for policy 1, policy_version 40323 (0.0010) [2023-12-26 15:39:29,553][105620] Updated weights for policy 1, policy_version 40333 (0.0010) [2023-12-26 15:39:29,895][105692] Updated weights for policy 0, policy_version 40055 (0.0008) [2023-12-26 15:39:29,955][105692] Updated weights for policy 0, policy_version 40065 (0.0009) [2023-12-26 15:39:30,004][105692] Updated weights for policy 0, policy_version 40075 (0.0008) [2023-12-26 15:39:30,310][105620] Updated weights for policy 1, policy_version 40343 (0.0010) [2023-12-26 15:39:30,365][105620] Updated weights for policy 1, policy_version 40353 (0.0010) [2023-12-26 15:39:30,419][105620] Updated weights for policy 1, policy_version 40363 (0.0010) [2023-12-26 15:39:30,790][105692] Updated weights for policy 0, policy_version 40085 (0.0008) [2023-12-26 15:39:30,837][105692] Updated weights for policy 0, policy_version 40095 (0.0008) [2023-12-26 15:39:30,896][105692] Updated weights for policy 0, policy_version 40105 (0.0008) [2023-12-26 15:39:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 20611072. Throughput: 0: 9861.9, 1: 9881.8. Samples: 20583828. Policy #0 lag: (min: 3.0, avg: 11.4, max: 35.0) [2023-12-26 15:39:31,062][104569] Avg episode reward: [(0, '9243.802'), (1, '9353.239')] [2023-12-26 15:39:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000040368_10338304.pth... [2023-12-26 15:39:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000040112_10272768.pth... [2023-12-26 15:39:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000039248_10051584.pth [2023-12-26 15:39:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000038928_9969664.pth [2023-12-26 15:39:31,073][105586] Saving new best policy, reward=9353.239! [2023-12-26 15:39:31,158][105620] Updated weights for policy 1, policy_version 40373 (0.0010) [2023-12-26 15:39:31,207][105620] Updated weights for policy 1, policy_version 40383 (0.0010) [2023-12-26 15:39:31,269][105620] Updated weights for policy 1, policy_version 40393 (0.0011) [2023-12-26 15:39:31,691][105692] Updated weights for policy 0, policy_version 40115 (0.0008) [2023-12-26 15:39:31,752][105692] Updated weights for policy 0, policy_version 40125 (0.0008) [2023-12-26 15:39:31,812][105692] Updated weights for policy 0, policy_version 40135 (0.0008) [2023-12-26 15:39:32,025][105620] Updated weights for policy 1, policy_version 40403 (0.0010) [2023-12-26 15:39:32,073][105620] Updated weights for policy 1, policy_version 40413 (0.0010) [2023-12-26 15:39:32,121][105620] Updated weights for policy 1, policy_version 40423 (0.0010) [2023-12-26 15:39:32,589][105692] Updated weights for policy 0, policy_version 40145 (0.0008) [2023-12-26 15:39:32,640][105692] Updated weights for policy 0, policy_version 40155 (0.0008) [2023-12-26 15:39:32,685][105692] Updated weights for policy 0, policy_version 40165 (0.0008) [2023-12-26 15:39:32,730][105692] Updated weights for policy 0, policy_version 40175 (0.0008) [2023-12-26 15:39:32,885][105620] Updated weights for policy 1, policy_version 40433 (0.0010) [2023-12-26 15:39:32,941][105620] Updated weights for policy 1, policy_version 40443 (0.0010) [2023-12-26 15:39:32,995][105620] Updated weights for policy 1, policy_version 40453 (0.0010) [2023-12-26 15:39:33,042][105620] Updated weights for policy 1, policy_version 40463 (0.0010) [2023-12-26 15:39:33,524][105692] Updated weights for policy 0, policy_version 40185 (0.0008) [2023-12-26 15:39:33,583][105692] Updated weights for policy 0, policy_version 40195 (0.0008) [2023-12-26 15:39:33,630][105692] Updated weights for policy 0, policy_version 40205 (0.0008) [2023-12-26 15:39:33,774][105620] Updated weights for policy 1, policy_version 40473 (0.0010) [2023-12-26 15:39:33,818][105620] Updated weights for policy 1, policy_version 40483 (0.0010) [2023-12-26 15:39:33,865][105620] Updated weights for policy 1, policy_version 40493 (0.0010) [2023-12-26 15:39:34,406][105692] Updated weights for policy 0, policy_version 40215 (0.0009) [2023-12-26 15:39:34,463][105692] Updated weights for policy 0, policy_version 40225 (0.0008) [2023-12-26 15:39:34,524][105692] Updated weights for policy 0, policy_version 40235 (0.0009) [2023-12-26 15:39:34,650][105620] Updated weights for policy 1, policy_version 40503 (0.0010) [2023-12-26 15:39:34,702][105620] Updated weights for policy 1, policy_version 40513 (0.0010) [2023-12-26 15:39:34,759][105620] Updated weights for policy 1, policy_version 40523 (0.0010) [2023-12-26 15:39:35,300][105692] Updated weights for policy 0, policy_version 40245 (0.0008) [2023-12-26 15:39:35,356][105692] Updated weights for policy 0, policy_version 40255 (0.0008) [2023-12-26 15:39:35,408][105692] Updated weights for policy 0, policy_version 40265 (0.0008) [2023-12-26 15:39:35,503][105620] Updated weights for policy 1, policy_version 40533 (0.0010) [2023-12-26 15:39:35,569][105620] Updated weights for policy 1, policy_version 40543 (0.0010) [2023-12-26 15:39:35,629][105620] Updated weights for policy 1, policy_version 40553 (0.0010) [2023-12-26 15:39:36,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 20701184. Throughput: 0: 9699.4, 1: 9782.8. Samples: 20693836. Policy #0 lag: (min: 3.0, avg: 11.4, max: 35.0) [2023-12-26 15:39:36,062][104569] Avg episode reward: [(0, '9241.266'), (1, '9352.953')] [2023-12-26 15:39:36,181][105692] Updated weights for policy 0, policy_version 40275 (0.0008) [2023-12-26 15:39:36,239][105692] Updated weights for policy 0, policy_version 40285 (0.0008) [2023-12-26 15:39:36,297][105692] Updated weights for policy 0, policy_version 40295 (0.0009) [2023-12-26 15:39:36,366][105620] Updated weights for policy 1, policy_version 40563 (0.0010) [2023-12-26 15:39:36,429][105620] Updated weights for policy 1, policy_version 40573 (0.0010) [2023-12-26 15:39:36,483][105620] Updated weights for policy 1, policy_version 40583 (0.0011) [2023-12-26 15:39:37,089][105692] Updated weights for policy 0, policy_version 40305 (0.0008) [2023-12-26 15:39:37,150][105692] Updated weights for policy 0, policy_version 40315 (0.0009) [2023-12-26 15:39:37,206][105692] Updated weights for policy 0, policy_version 40325 (0.0008) [2023-12-26 15:39:37,237][105620] Updated weights for policy 1, policy_version 40593 (0.0011) [2023-12-26 15:39:37,264][105692] Updated weights for policy 0, policy_version 40335 (0.0006) [2023-12-26 15:39:37,285][105620] Updated weights for policy 1, policy_version 40603 (0.0010) [2023-12-26 15:39:37,334][105620] Updated weights for policy 1, policy_version 40613 (0.0010) [2023-12-26 15:39:37,382][105620] Updated weights for policy 1, policy_version 40623 (0.0010) [2023-12-26 15:39:38,032][105692] Updated weights for policy 0, policy_version 40345 (0.0008) [2023-12-26 15:39:38,081][105692] Updated weights for policy 0, policy_version 40355 (0.0008) [2023-12-26 15:39:38,134][105692] Updated weights for policy 0, policy_version 40365 (0.0008) [2023-12-26 15:39:38,170][105620] Updated weights for policy 1, policy_version 40633 (0.0010) [2023-12-26 15:39:38,225][105620] Updated weights for policy 1, policy_version 40643 (0.0010) [2023-12-26 15:39:38,276][105620] Updated weights for policy 1, policy_version 40653 (0.0010) [2023-12-26 15:39:38,935][105692] Updated weights for policy 0, policy_version 40375 (0.0007) [2023-12-26 15:39:38,987][105692] Updated weights for policy 0, policy_version 40385 (0.0008) [2023-12-26 15:39:39,017][105620] Updated weights for policy 1, policy_version 40663 (0.0010) [2023-12-26 15:39:39,039][105692] Updated weights for policy 0, policy_version 40395 (0.0005) [2023-12-26 15:39:39,076][105620] Updated weights for policy 1, policy_version 40673 (0.0010) [2023-12-26 15:39:39,127][105620] Updated weights for policy 1, policy_version 40683 (0.0010) [2023-12-26 15:39:39,860][105692] Updated weights for policy 0, policy_version 40405 (0.0008) [2023-12-26 15:39:39,907][105620] Updated weights for policy 1, policy_version 40693 (0.0010) [2023-12-26 15:39:39,926][105692] Updated weights for policy 0, policy_version 40415 (0.0006) [2023-12-26 15:39:39,975][105620] Updated weights for policy 1, policy_version 40703 (0.0011) [2023-12-26 15:39:39,987][105692] Updated weights for policy 0, policy_version 40425 (0.0008) [2023-12-26 15:39:40,036][105620] Updated weights for policy 1, policy_version 40713 (0.0010) [2023-12-26 15:39:40,798][105692] Updated weights for policy 0, policy_version 40435 (0.0007) [2023-12-26 15:39:40,843][105620] Updated weights for policy 1, policy_version 40723 (0.0009) [2023-12-26 15:39:40,858][105692] Updated weights for policy 0, policy_version 40445 (0.0007) [2023-12-26 15:39:40,892][105620] Updated weights for policy 1, policy_version 40733 (0.0010) [2023-12-26 15:39:40,915][105692] Updated weights for policy 0, policy_version 40455 (0.0006) [2023-12-26 15:39:40,948][105620] Updated weights for policy 1, policy_version 40743 (0.0006) [2023-12-26 15:39:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 20799488. Throughput: 0: 9609.2, 1: 9681.2. Samples: 20802648. Policy #0 lag: (min: 3.0, avg: 11.4, max: 35.0) [2023-12-26 15:39:41,063][104569] Avg episode reward: [(0, '9330.610'), (1, '9261.446')] [2023-12-26 15:39:41,712][105620] Updated weights for policy 1, policy_version 40753 (0.0006) [2023-12-26 15:39:41,746][105692] Updated weights for policy 0, policy_version 40465 (0.0008) [2023-12-26 15:39:41,777][105620] Updated weights for policy 1, policy_version 40763 (0.0008) [2023-12-26 15:39:41,808][105692] Updated weights for policy 0, policy_version 40475 (0.0007) [2023-12-26 15:39:41,839][105620] Updated weights for policy 1, policy_version 40773 (0.0005) [2023-12-26 15:39:41,866][105692] Updated weights for policy 0, policy_version 40485 (0.0009) [2023-12-26 15:39:41,902][105620] Updated weights for policy 1, policy_version 40783 (0.0005) [2023-12-26 15:39:41,926][105692] Updated weights for policy 0, policy_version 40495 (0.0009) [2023-12-26 15:39:42,611][105620] Updated weights for policy 1, policy_version 40793 (0.0008) [2023-12-26 15:39:42,669][105620] Updated weights for policy 1, policy_version 40803 (0.0007) [2023-12-26 15:39:42,723][105620] Updated weights for policy 1, policy_version 40813 (0.0007) [2023-12-26 15:39:42,734][105692] Updated weights for policy 0, policy_version 40505 (0.0007) [2023-12-26 15:39:42,796][105692] Updated weights for policy 0, policy_version 40515 (0.0006) [2023-12-26 15:39:42,860][105692] Updated weights for policy 0, policy_version 40525 (0.0009) [2023-12-26 15:39:43,269][105620] Updated weights for policy 1, policy_version 40823 (0.0006) [2023-12-26 15:39:43,315][105620] Updated weights for policy 1, policy_version 40833 (0.0006) [2023-12-26 15:39:43,359][105620] Updated weights for policy 1, policy_version 40843 (0.0005) [2023-12-26 15:39:43,750][105692] Updated weights for policy 0, policy_version 40535 (0.0010) [2023-12-26 15:39:43,813][105692] Updated weights for policy 0, policy_version 40545 (0.0010) [2023-12-26 15:39:43,875][105692] Updated weights for policy 0, policy_version 40555 (0.0009) [2023-12-26 15:39:43,911][105620] Updated weights for policy 1, policy_version 40853 (0.0006) [2023-12-26 15:39:43,970][105620] Updated weights for policy 1, policy_version 40863 (0.0010) [2023-12-26 15:39:44,024][105620] Updated weights for policy 1, policy_version 40873 (0.0010) [2023-12-26 15:39:44,491][105692] Updated weights for policy 0, policy_version 40565 (0.0005) [2023-12-26 15:39:44,549][105692] Updated weights for policy 0, policy_version 40575 (0.0006) [2023-12-26 15:39:44,600][105692] Updated weights for policy 0, policy_version 40585 (0.0009) [2023-12-26 15:39:44,832][105620] Updated weights for policy 1, policy_version 40883 (0.0009) [2023-12-26 15:39:44,884][105620] Updated weights for policy 1, policy_version 40893 (0.0009) [2023-12-26 15:39:44,947][105620] Updated weights for policy 1, policy_version 40903 (0.0009) [2023-12-26 15:39:45,324][105692] Updated weights for policy 0, policy_version 40595 (0.0009) [2023-12-26 15:39:45,387][105692] Updated weights for policy 0, policy_version 40605 (0.0010) [2023-12-26 15:39:45,446][105692] Updated weights for policy 0, policy_version 40615 (0.0009) [2023-12-26 15:39:45,738][105620] Updated weights for policy 1, policy_version 40913 (0.0009) [2023-12-26 15:39:45,786][105620] Updated weights for policy 1, policy_version 40923 (0.0009) [2023-12-26 15:39:45,835][105620] Updated weights for policy 1, policy_version 40934 (0.0009) [2023-12-26 15:39:45,890][105620] Updated weights for policy 1, policy_version 40944 (0.0009) [2023-12-26 15:39:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 20889600. Throughput: 0: 9594.7, 1: 9758.9. Samples: 20861316. Policy #0 lag: (min: 3.0, avg: 11.4, max: 35.0) [2023-12-26 15:39:46,062][104569] Avg episode reward: [(0, '9328.275'), (1, '9168.857')] [2023-12-26 15:39:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000040624_10403840.pth... [2023-12-26 15:39:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000040944_10485760.pth... [2023-12-26 15:39:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000039536_10125312.pth [2023-12-26 15:39:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000039824_10199040.pth [2023-12-26 15:39:46,192][105692] Updated weights for policy 0, policy_version 40625 (0.0009) [2023-12-26 15:39:46,244][105692] Updated weights for policy 0, policy_version 40635 (0.0006) [2023-12-26 15:39:46,305][105692] Updated weights for policy 0, policy_version 40645 (0.0006) [2023-12-26 15:39:46,352][105692] Updated weights for policy 0, policy_version 40655 (0.0006) [2023-12-26 15:39:46,494][105620] Updated weights for policy 1, policy_version 40954 (0.0009) [2023-12-26 15:39:46,545][105620] Updated weights for policy 1, policy_version 40964 (0.0009) [2023-12-26 15:39:46,600][105620] Updated weights for policy 1, policy_version 40974 (0.0009) [2023-12-26 15:39:47,062][105692] Updated weights for policy 0, policy_version 40665 (0.0007) [2023-12-26 15:39:47,123][105692] Updated weights for policy 0, policy_version 40675 (0.0006) [2023-12-26 15:39:47,182][105692] Updated weights for policy 0, policy_version 40685 (0.0006) [2023-12-26 15:39:47,438][105620] Updated weights for policy 1, policy_version 40984 (0.0009) [2023-12-26 15:39:47,502][105620] Updated weights for policy 1, policy_version 40994 (0.0009) [2023-12-26 15:39:47,559][105620] Updated weights for policy 1, policy_version 41004 (0.0009) [2023-12-26 15:39:47,770][105692] Updated weights for policy 0, policy_version 40695 (0.0008) [2023-12-26 15:39:47,829][105692] Updated weights for policy 0, policy_version 40705 (0.0009) [2023-12-26 15:39:47,891][105692] Updated weights for policy 0, policy_version 40715 (0.0009) [2023-12-26 15:39:48,211][105620] Updated weights for policy 1, policy_version 41014 (0.0009) [2023-12-26 15:39:48,273][105620] Updated weights for policy 1, policy_version 41024 (0.0009) [2023-12-26 15:39:48,334][105620] Updated weights for policy 1, policy_version 41034 (0.0009) [2023-12-26 15:39:48,766][105692] Updated weights for policy 0, policy_version 40725 (0.0009) [2023-12-26 15:39:48,829][105692] Updated weights for policy 0, policy_version 40735 (0.0009) [2023-12-26 15:39:48,877][105692] Updated weights for policy 0, policy_version 40745 (0.0009) [2023-12-26 15:39:49,003][105620] Updated weights for policy 1, policy_version 41044 (0.0008) [2023-12-26 15:39:49,050][105620] Updated weights for policy 1, policy_version 41054 (0.0009) [2023-12-26 15:39:49,113][105620] Updated weights for policy 1, policy_version 41064 (0.0009) [2023-12-26 15:39:49,596][105692] Updated weights for policy 0, policy_version 40755 (0.0008) [2023-12-26 15:39:49,661][105692] Updated weights for policy 0, policy_version 40765 (0.0005) [2023-12-26 15:39:49,728][105692] Updated weights for policy 0, policy_version 40775 (0.0005) [2023-12-26 15:39:49,914][105620] Updated weights for policy 1, policy_version 41074 (0.0009) [2023-12-26 15:39:49,974][105620] Updated weights for policy 1, policy_version 41084 (0.0009) [2023-12-26 15:39:50,028][105620] Updated weights for policy 1, policy_version 41094 (0.0009) [2023-12-26 15:39:50,082][105620] Updated weights for policy 1, policy_version 41104 (0.0008) [2023-12-26 15:39:50,372][105692] Updated weights for policy 0, policy_version 40785 (0.0006) [2023-12-26 15:39:50,427][105692] Updated weights for policy 0, policy_version 40795 (0.0009) [2023-12-26 15:39:50,481][105692] Updated weights for policy 0, policy_version 40805 (0.0009) [2023-12-26 15:39:50,532][105692] Updated weights for policy 0, policy_version 40815 (0.0009) [2023-12-26 15:39:50,835][105620] Updated weights for policy 1, policy_version 41114 (0.0009) [2023-12-26 15:39:50,897][105620] Updated weights for policy 1, policy_version 41124 (0.0009) [2023-12-26 15:39:50,955][105620] Updated weights for policy 1, policy_version 41134 (0.0009) [2023-12-26 15:39:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 20987904. Throughput: 0: 9570.8, 1: 9628.5. Samples: 20976740. Policy #0 lag: (min: 3.0, avg: 11.4, max: 35.0) [2023-12-26 15:39:51,062][104569] Avg episode reward: [(0, '9332.225'), (1, '9172.190')] [2023-12-26 15:39:51,324][105692] Updated weights for policy 0, policy_version 40825 (0.0008) [2023-12-26 15:39:51,390][105692] Updated weights for policy 0, policy_version 40835 (0.0007) [2023-12-26 15:39:51,448][105692] Updated weights for policy 0, policy_version 40845 (0.0005) [2023-12-26 15:39:51,787][105620] Updated weights for policy 1, policy_version 41144 (0.0009) [2023-12-26 15:39:51,841][105620] Updated weights for policy 1, policy_version 41154 (0.0009) [2023-12-26 15:39:51,894][105620] Updated weights for policy 1, policy_version 41164 (0.0009) [2023-12-26 15:39:52,085][105692] Updated weights for policy 0, policy_version 40855 (0.0005) [2023-12-26 15:39:52,142][105692] Updated weights for policy 0, policy_version 40865 (0.0005) [2023-12-26 15:39:52,194][105692] Updated weights for policy 0, policy_version 40875 (0.0005) [2023-12-26 15:39:52,724][105620] Updated weights for policy 1, policy_version 41174 (0.0008) [2023-12-26 15:39:52,774][105620] Updated weights for policy 1, policy_version 41184 (0.0007) [2023-12-26 15:39:52,820][105620] Updated weights for policy 1, policy_version 41194 (0.0008) [2023-12-26 15:39:52,881][105692] Updated weights for policy 0, policy_version 40885 (0.0007) [2023-12-26 15:39:52,950][105692] Updated weights for policy 0, policy_version 40895 (0.0005) [2023-12-26 15:39:53,012][105692] Updated weights for policy 0, policy_version 40905 (0.0005) [2023-12-26 15:39:53,424][105620] Updated weights for policy 1, policy_version 41204 (0.0009) [2023-12-26 15:39:53,485][105620] Updated weights for policy 1, policy_version 41214 (0.0010) [2023-12-26 15:39:53,517][105692] Updated weights for policy 0, policy_version 40915 (0.0005) [2023-12-26 15:39:53,537][105620] Updated weights for policy 1, policy_version 41224 (0.0010) [2023-12-26 15:39:53,569][105692] Updated weights for policy 0, policy_version 40925 (0.0006) [2023-12-26 15:39:53,624][105692] Updated weights for policy 0, policy_version 40935 (0.0010) [2023-12-26 15:39:54,146][105620] Updated weights for policy 1, policy_version 41234 (0.0009) [2023-12-26 15:39:54,182][105692] Updated weights for policy 0, policy_version 40945 (0.0010) [2023-12-26 15:39:54,204][105620] Updated weights for policy 1, policy_version 41244 (0.0005) [2023-12-26 15:39:54,234][105692] Updated weights for policy 0, policy_version 40955 (0.0010) [2023-12-26 15:39:54,260][105620] Updated weights for policy 1, policy_version 41254 (0.0005) [2023-12-26 15:39:54,285][105692] Updated weights for policy 0, policy_version 40965 (0.0010) [2023-12-26 15:39:54,313][105620] Updated weights for policy 1, policy_version 41264 (0.0008) [2023-12-26 15:39:54,332][105692] Updated weights for policy 0, policy_version 40975 (0.0010) [2023-12-26 15:39:54,849][105620] Updated weights for policy 1, policy_version 41274 (0.0010) [2023-12-26 15:39:54,910][105620] Updated weights for policy 1, policy_version 41284 (0.0010) [2023-12-26 15:39:54,969][105620] Updated weights for policy 1, policy_version 41294 (0.0008) [2023-12-26 15:39:55,088][105692] Updated weights for policy 0, policy_version 40985 (0.0009) [2023-12-26 15:39:55,151][105692] Updated weights for policy 0, policy_version 40995 (0.0009) [2023-12-26 15:39:55,213][105692] Updated weights for policy 0, policy_version 41005 (0.0009) [2023-12-26 15:39:55,617][105620] Updated weights for policy 1, policy_version 41304 (0.0005) [2023-12-26 15:39:55,687][105620] Updated weights for policy 1, policy_version 41314 (0.0006) [2023-12-26 15:39:55,737][105620] Updated weights for policy 1, policy_version 41324 (0.0009) [2023-12-26 15:39:55,987][105692] Updated weights for policy 0, policy_version 41015 (0.0010) [2023-12-26 15:39:56,034][105692] Updated weights for policy 0, policy_version 41025 (0.0007) [2023-12-26 15:39:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 21086208. Throughput: 0: 9594.9, 1: 9665.1. Samples: 21099332. Policy #0 lag: (min: 11.0, avg: 19.4, max: 43.0) [2023-12-26 15:39:56,062][104569] Avg episode reward: [(0, '8347.979'), (1, '9090.406')] [2023-12-26 15:39:56,087][105692] Updated weights for policy 0, policy_version 41035 (0.0005) [2023-12-26 15:39:56,334][105620] Updated weights for policy 1, policy_version 41334 (0.0006) [2023-12-26 15:39:56,398][105620] Updated weights for policy 1, policy_version 41344 (0.0005) [2023-12-26 15:39:56,451][105620] Updated weights for policy 1, policy_version 41354 (0.0005) [2023-12-26 15:39:56,696][105692] Updated weights for policy 0, policy_version 41045 (0.0008) [2023-12-26 15:39:56,753][105692] Updated weights for policy 0, policy_version 41055 (0.0010) [2023-12-26 15:39:56,804][105692] Updated weights for policy 0, policy_version 41065 (0.0010) [2023-12-26 15:39:57,111][105620] Updated weights for policy 1, policy_version 41364 (0.0009) [2023-12-26 15:39:57,166][105620] Updated weights for policy 1, policy_version 41374 (0.0008) [2023-12-26 15:39:57,231][105620] Updated weights for policy 1, policy_version 41384 (0.0008) [2023-12-26 15:39:57,498][105692] Updated weights for policy 0, policy_version 41075 (0.0009) [2023-12-26 15:39:57,555][105692] Updated weights for policy 0, policy_version 41085 (0.0005) [2023-12-26 15:39:57,616][105692] Updated weights for policy 0, policy_version 41095 (0.0005) [2023-12-26 15:39:57,874][105620] Updated weights for policy 1, policy_version 41394 (0.0005) [2023-12-26 15:39:57,927][105620] Updated weights for policy 1, policy_version 41404 (0.0005) [2023-12-26 15:39:57,990][105620] Updated weights for policy 1, policy_version 41414 (0.0005) [2023-12-26 15:39:58,042][105620] Updated weights for policy 1, policy_version 41424 (0.0005) [2023-12-26 15:39:58,230][105692] Updated weights for policy 0, policy_version 41105 (0.0006) [2023-12-26 15:39:58,287][105692] Updated weights for policy 0, policy_version 41115 (0.0010) [2023-12-26 15:39:58,354][105692] Updated weights for policy 0, policy_version 41125 (0.0009) [2023-12-26 15:39:58,412][105692] Updated weights for policy 0, policy_version 41135 (0.0008) [2023-12-26 15:39:58,787][105620] Updated weights for policy 1, policy_version 41434 (0.0008) [2023-12-26 15:39:58,843][105620] Updated weights for policy 1, policy_version 41444 (0.0008) [2023-12-26 15:39:58,905][105620] Updated weights for policy 1, policy_version 41454 (0.0008) [2023-12-26 15:39:59,219][105692] Updated weights for policy 0, policy_version 41145 (0.0008) [2023-12-26 15:39:59,284][105692] Updated weights for policy 0, policy_version 41155 (0.0008) [2023-12-26 15:39:59,352][105692] Updated weights for policy 0, policy_version 41165 (0.0008) [2023-12-26 15:39:59,606][105620] Updated weights for policy 1, policy_version 41464 (0.0006) [2023-12-26 15:39:59,659][105620] Updated weights for policy 1, policy_version 41474 (0.0006) [2023-12-26 15:39:59,708][105620] Updated weights for policy 1, policy_version 41484 (0.0005) [2023-12-26 15:40:00,189][105692] Updated weights for policy 0, policy_version 41175 (0.0007) [2023-12-26 15:40:00,236][105692] Updated weights for policy 0, policy_version 41185 (0.0008) [2023-12-26 15:40:00,282][105692] Updated weights for policy 0, policy_version 41195 (0.0008) [2023-12-26 15:40:00,391][105620] Updated weights for policy 1, policy_version 41494 (0.0007) [2023-12-26 15:40:00,449][105620] Updated weights for policy 1, policy_version 41504 (0.0009) [2023-12-26 15:40:00,503][105620] Updated weights for policy 1, policy_version 41514 (0.0009) [2023-12-26 15:40:00,982][105692] Updated weights for policy 0, policy_version 41205 (0.0008) [2023-12-26 15:40:01,030][105692] Updated weights for policy 0, policy_version 41215 (0.0008) [2023-12-26 15:40:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 21184512. Throughput: 0: 9663.6, 1: 9688.8. Samples: 21161044. Policy #0 lag: (min: 11.0, avg: 19.4, max: 43.0) [2023-12-26 15:40:01,062][104569] Avg episode reward: [(0, '2371.429'), (1, '9004.601')] [2023-12-26 15:40:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000041520_10633216.pth... [2023-12-26 15:40:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000040368_10338304.pth [2023-12-26 15:40:01,095][105692] Updated weights for policy 0, policy_version 41225 (0.0008) [2023-12-26 15:40:01,140][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000041232_10559488.pth... [2023-12-26 15:40:01,143][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000040112_10272768.pth [2023-12-26 15:40:01,274][105620] Updated weights for policy 1, policy_version 41524 (0.0010) [2023-12-26 15:40:01,332][105620] Updated weights for policy 1, policy_version 41534 (0.0009) [2023-12-26 15:40:01,392][105620] Updated weights for policy 1, policy_version 41544 (0.0010) [2023-12-26 15:40:01,863][105692] Updated weights for policy 0, policy_version 41235 (0.0008) [2023-12-26 15:40:01,911][105692] Updated weights for policy 0, policy_version 41245 (0.0008) [2023-12-26 15:40:01,955][105692] Updated weights for policy 0, policy_version 41255 (0.0008) [2023-12-26 15:40:02,150][105620] Updated weights for policy 1, policy_version 41554 (0.0010) [2023-12-26 15:40:02,211][105620] Updated weights for policy 1, policy_version 41564 (0.0010) [2023-12-26 15:40:02,273][105620] Updated weights for policy 1, policy_version 41574 (0.0011) [2023-12-26 15:40:02,322][105620] Updated weights for policy 1, policy_version 41584 (0.0010) [2023-12-26 15:40:02,734][105692] Updated weights for policy 0, policy_version 41265 (0.0007) [2023-12-26 15:40:02,798][105692] Updated weights for policy 0, policy_version 41275 (0.0005) [2023-12-26 15:40:02,866][105692] Updated weights for policy 0, policy_version 41285 (0.0005) [2023-12-26 15:40:02,927][105692] Updated weights for policy 0, policy_version 41295 (0.0006) [2023-12-26 15:40:02,963][105620] Updated weights for policy 1, policy_version 41594 (0.0005) [2023-12-26 15:40:03,020][105620] Updated weights for policy 1, policy_version 41604 (0.0005) [2023-12-26 15:40:03,084][105620] Updated weights for policy 1, policy_version 41614 (0.0005) [2023-12-26 15:40:03,431][105692] Updated weights for policy 0, policy_version 41305 (0.0006) [2023-12-26 15:40:03,490][105692] Updated weights for policy 0, policy_version 41315 (0.0005) [2023-12-26 15:40:03,559][105692] Updated weights for policy 0, policy_version 41325 (0.0005) [2023-12-26 15:40:03,734][105620] Updated weights for policy 1, policy_version 41624 (0.0008) [2023-12-26 15:40:03,795][105620] Updated weights for policy 1, policy_version 41634 (0.0009) [2023-12-26 15:40:03,857][105620] Updated weights for policy 1, policy_version 41644 (0.0009) [2023-12-26 15:40:04,211][105692] Updated weights for policy 0, policy_version 41335 (0.0008) [2023-12-26 15:40:04,264][105692] Updated weights for policy 0, policy_version 41345 (0.0010) [2023-12-26 15:40:04,313][105692] Updated weights for policy 0, policy_version 41355 (0.0008) [2023-12-26 15:40:04,669][105620] Updated weights for policy 1, policy_version 41654 (0.0009) [2023-12-26 15:40:04,723][105620] Updated weights for policy 1, policy_version 41664 (0.0010) [2023-12-26 15:40:04,781][105620] Updated weights for policy 1, policy_version 41675 (0.0010) [2023-12-26 15:40:04,959][105692] Updated weights for policy 0, policy_version 41365 (0.0009) [2023-12-26 15:40:05,013][105692] Updated weights for policy 0, policy_version 41375 (0.0009) [2023-12-26 15:40:05,074][105692] Updated weights for policy 0, policy_version 41385 (0.0008) [2023-12-26 15:40:05,578][105620] Updated weights for policy 1, policy_version 41686 (0.0009) [2023-12-26 15:40:05,643][105620] Updated weights for policy 1, policy_version 41696 (0.0009) [2023-12-26 15:40:05,701][105620] Updated weights for policy 1, policy_version 41706 (0.0009) [2023-12-26 15:40:05,798][105692] Updated weights for policy 0, policy_version 41395 (0.0010) [2023-12-26 15:40:05,853][105692] Updated weights for policy 0, policy_version 41405 (0.0010) [2023-12-26 15:40:05,915][105692] Updated weights for policy 0, policy_version 41415 (0.0010) [2023-12-26 15:40:06,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 21291008. Throughput: 0: 9661.6, 1: 9670.0. Samples: 21276808. Policy #0 lag: (min: 11.0, avg: 19.4, max: 43.0) [2023-12-26 15:40:06,062][104569] Avg episode reward: [(0, '3503.236'), (1, '9174.406')] [2023-12-26 15:40:06,455][105620] Updated weights for policy 1, policy_version 41716 (0.0007) [2023-12-26 15:40:06,528][105620] Updated weights for policy 1, policy_version 41726 (0.0005) [2023-12-26 15:40:06,599][105620] Updated weights for policy 1, policy_version 41736 (0.0007) [2023-12-26 15:40:06,609][105692] Updated weights for policy 0, policy_version 41425 (0.0010) [2023-12-26 15:40:06,674][105692] Updated weights for policy 0, policy_version 41435 (0.0009) [2023-12-26 15:40:06,736][105692] Updated weights for policy 0, policy_version 41445 (0.0007) [2023-12-26 15:40:06,801][105692] Updated weights for policy 0, policy_version 41455 (0.0009) [2023-12-26 15:40:07,217][105620] Updated weights for policy 1, policy_version 41746 (0.0006) [2023-12-26 15:40:07,272][105620] Updated weights for policy 1, policy_version 41756 (0.0009) [2023-12-26 15:40:07,329][105620] Updated weights for policy 1, policy_version 41766 (0.0008) [2023-12-26 15:40:07,381][105620] Updated weights for policy 1, policy_version 41776 (0.0009) [2023-12-26 15:40:07,543][105692] Updated weights for policy 0, policy_version 41465 (0.0009) [2023-12-26 15:40:07,598][105692] Updated weights for policy 0, policy_version 41475 (0.0008) [2023-12-26 15:40:07,654][105692] Updated weights for policy 0, policy_version 41485 (0.0009) [2023-12-26 15:40:08,141][105620] Updated weights for policy 1, policy_version 41786 (0.0008) [2023-12-26 15:40:08,195][105620] Updated weights for policy 1, policy_version 41796 (0.0008) [2023-12-26 15:40:08,244][105620] Updated weights for policy 1, policy_version 41806 (0.0008) [2023-12-26 15:40:08,436][105692] Updated weights for policy 0, policy_version 41495 (0.0008) [2023-12-26 15:40:08,490][105692] Updated weights for policy 0, policy_version 41505 (0.0009) [2023-12-26 15:40:08,539][105692] Updated weights for policy 0, policy_version 41515 (0.0008) [2023-12-26 15:40:08,891][105620] Updated weights for policy 1, policy_version 41816 (0.0008) [2023-12-26 15:40:08,940][105620] Updated weights for policy 1, policy_version 41826 (0.0010) [2023-12-26 15:40:08,991][105620] Updated weights for policy 1, policy_version 41836 (0.0007) [2023-12-26 15:40:09,416][105692] Updated weights for policy 0, policy_version 41525 (0.0009) [2023-12-26 15:40:09,479][105692] Updated weights for policy 0, policy_version 41535 (0.0008) [2023-12-26 15:40:09,542][105692] Updated weights for policy 0, policy_version 41545 (0.0008) [2023-12-26 15:40:09,633][105620] Updated weights for policy 1, policy_version 41846 (0.0008) [2023-12-26 15:40:09,699][105620] Updated weights for policy 1, policy_version 41856 (0.0010) [2023-12-26 15:40:09,762][105620] Updated weights for policy 1, policy_version 41866 (0.0010) [2023-12-26 15:40:10,321][105692] Updated weights for policy 0, policy_version 41555 (0.0008) [2023-12-26 15:40:10,382][105692] Updated weights for policy 0, policy_version 41565 (0.0008) [2023-12-26 15:40:10,445][105692] Updated weights for policy 0, policy_version 41575 (0.0008) [2023-12-26 15:40:10,497][105620] Updated weights for policy 1, policy_version 41876 (0.0009) [2023-12-26 15:40:10,549][105620] Updated weights for policy 1, policy_version 41886 (0.0005) [2023-12-26 15:40:10,601][105620] Updated weights for policy 1, policy_version 41896 (0.0005) [2023-12-26 15:40:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 21381120. Throughput: 0: 9592.8, 1: 9719.4. Samples: 21392600. Policy #0 lag: (min: 11.0, avg: 19.4, max: 43.0) [2023-12-26 15:40:11,062][104569] Avg episode reward: [(0, '6321.522'), (1, '9177.606')] [2023-12-26 15:40:11,219][105620] Updated weights for policy 1, policy_version 41906 (0.0007) [2023-12-26 15:40:11,236][105692] Updated weights for policy 0, policy_version 41585 (0.0008) [2023-12-26 15:40:11,285][105620] Updated weights for policy 1, policy_version 41916 (0.0010) [2023-12-26 15:40:11,305][105692] Updated weights for policy 0, policy_version 41595 (0.0007) [2023-12-26 15:40:11,346][105620] Updated weights for policy 1, policy_version 41926 (0.0009) [2023-12-26 15:40:11,372][105692] Updated weights for policy 0, policy_version 41605 (0.0007) [2023-12-26 15:40:11,416][105620] Updated weights for policy 1, policy_version 41936 (0.0008) [2023-12-26 15:40:11,445][105692] Updated weights for policy 0, policy_version 41615 (0.0009) [2023-12-26 15:40:12,083][105692] Updated weights for policy 0, policy_version 41625 (0.0008) [2023-12-26 15:40:12,141][105692] Updated weights for policy 0, policy_version 41635 (0.0008) [2023-12-26 15:40:12,199][105692] Updated weights for policy 0, policy_version 41645 (0.0009) [2023-12-26 15:40:12,233][105620] Updated weights for policy 1, policy_version 41946 (0.0007) [2023-12-26 15:40:12,291][105620] Updated weights for policy 1, policy_version 41956 (0.0010) [2023-12-26 15:40:12,351][105620] Updated weights for policy 1, policy_version 41966 (0.0008) [2023-12-26 15:40:12,892][105692] Updated weights for policy 0, policy_version 41655 (0.0007) [2023-12-26 15:40:12,947][105692] Updated weights for policy 0, policy_version 41665 (0.0005) [2023-12-26 15:40:13,002][105692] Updated weights for policy 0, policy_version 41675 (0.0005) [2023-12-26 15:40:13,151][105620] Updated weights for policy 1, policy_version 41976 (0.0008) [2023-12-26 15:40:13,219][105620] Updated weights for policy 1, policy_version 41986 (0.0010) [2023-12-26 15:40:13,272][105620] Updated weights for policy 1, policy_version 41996 (0.0010) [2023-12-26 15:40:13,582][105692] Updated weights for policy 0, policy_version 41685 (0.0008) [2023-12-26 15:40:13,631][105692] Updated weights for policy 0, policy_version 41695 (0.0010) [2023-12-26 15:40:13,686][105692] Updated weights for policy 0, policy_version 41705 (0.0010) [2023-12-26 15:40:13,906][105620] Updated weights for policy 1, policy_version 42006 (0.0010) [2023-12-26 15:40:13,954][105620] Updated weights for policy 1, policy_version 42016 (0.0009) [2023-12-26 15:40:14,020][105620] Updated weights for policy 1, policy_version 42026 (0.0005) [2023-12-26 15:40:14,331][105692] Updated weights for policy 0, policy_version 41715 (0.0011) [2023-12-26 15:40:14,392][105692] Updated weights for policy 0, policy_version 41725 (0.0010) [2023-12-26 15:40:14,449][105692] Updated weights for policy 0, policy_version 41735 (0.0010) [2023-12-26 15:40:14,678][105620] Updated weights for policy 1, policy_version 42036 (0.0007) [2023-12-26 15:40:14,732][105620] Updated weights for policy 1, policy_version 42046 (0.0010) [2023-12-26 15:40:14,797][105620] Updated weights for policy 1, policy_version 42056 (0.0011) [2023-12-26 15:40:15,154][105692] Updated weights for policy 0, policy_version 41745 (0.0010) [2023-12-26 15:40:15,218][105692] Updated weights for policy 0, policy_version 41755 (0.0008) [2023-12-26 15:40:15,287][105692] Updated weights for policy 0, policy_version 41765 (0.0009) [2023-12-26 15:40:15,356][105692] Updated weights for policy 0, policy_version 41775 (0.0010) [2023-12-26 15:40:15,529][105620] Updated weights for policy 1, policy_version 42066 (0.0011) [2023-12-26 15:40:15,589][105620] Updated weights for policy 1, policy_version 42076 (0.0011) [2023-12-26 15:40:15,648][105620] Updated weights for policy 1, policy_version 42086 (0.0011) [2023-12-26 15:40:15,704][105620] Updated weights for policy 1, policy_version 42096 (0.0010) [2023-12-26 15:40:16,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19251.1, 300 sec: 19633.0). Total num frames: 21479424. Throughput: 0: 9545.2, 1: 9732.4. Samples: 21451324. Policy #0 lag: (min: 11.0, avg: 19.4, max: 43.0) [2023-12-26 15:40:16,063][104569] Avg episode reward: [(0, '8656.531'), (1, '9091.202')] [2023-12-26 15:40:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000042096_10780672.pth... [2023-12-26 15:40:16,075][105692] Updated weights for policy 0, policy_version 41785 (0.0006) [2023-12-26 15:40:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000040944_10485760.pth [2023-12-26 15:40:16,139][105692] Updated weights for policy 0, policy_version 41795 (0.0005) [2023-12-26 15:40:16,201][105692] Updated weights for policy 0, policy_version 41805 (0.0005) [2023-12-26 15:40:16,221][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000041808_10706944.pth... [2023-12-26 15:40:16,226][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000040624_10403840.pth [2023-12-26 15:40:16,368][105620] Updated weights for policy 1, policy_version 42106 (0.0005) [2023-12-26 15:40:16,420][105620] Updated weights for policy 1, policy_version 42116 (0.0005) [2023-12-26 15:40:16,480][105620] Updated weights for policy 1, policy_version 42126 (0.0005) [2023-12-26 15:40:16,866][105692] Updated weights for policy 0, policy_version 41815 (0.0009) [2023-12-26 15:40:16,928][105692] Updated weights for policy 0, policy_version 41825 (0.0010) [2023-12-26 15:40:16,990][105692] Updated weights for policy 0, policy_version 41835 (0.0010) [2023-12-26 15:40:17,076][105620] Updated weights for policy 1, policy_version 42136 (0.0009) [2023-12-26 15:40:17,124][105620] Updated weights for policy 1, policy_version 42146 (0.0010) [2023-12-26 15:40:17,172][105620] Updated weights for policy 1, policy_version 42156 (0.0010) [2023-12-26 15:40:17,679][105692] Updated weights for policy 0, policy_version 41845 (0.0010) [2023-12-26 15:40:17,742][105692] Updated weights for policy 0, policy_version 41855 (0.0011) [2023-12-26 15:40:17,758][105620] Updated weights for policy 1, policy_version 42166 (0.0008) [2023-12-26 15:40:17,800][105692] Updated weights for policy 0, policy_version 41865 (0.0010) [2023-12-26 15:40:17,810][105620] Updated weights for policy 1, policy_version 42176 (0.0010) [2023-12-26 15:40:17,868][105620] Updated weights for policy 1, policy_version 42186 (0.0010) [2023-12-26 15:40:18,542][105692] Updated weights for policy 0, policy_version 41875 (0.0010) [2023-12-26 15:40:18,598][105692] Updated weights for policy 0, policy_version 41885 (0.0010) [2023-12-26 15:40:18,613][105620] Updated weights for policy 1, policy_version 42196 (0.0010) [2023-12-26 15:40:18,652][105692] Updated weights for policy 0, policy_version 41895 (0.0010) [2023-12-26 15:40:18,665][105620] Updated weights for policy 1, policy_version 42206 (0.0010) [2023-12-26 15:40:18,727][105620] Updated weights for policy 1, policy_version 42216 (0.0010) [2023-12-26 15:40:19,402][105692] Updated weights for policy 0, policy_version 41905 (0.0010) [2023-12-26 15:40:19,453][105692] Updated weights for policy 0, policy_version 41915 (0.0010) [2023-12-26 15:40:19,516][105620] Updated weights for policy 1, policy_version 42226 (0.0009) [2023-12-26 15:40:19,517][105692] Updated weights for policy 0, policy_version 41925 (0.0011) [2023-12-26 15:40:19,575][105620] Updated weights for policy 1, policy_version 42236 (0.0010) [2023-12-26 15:40:19,582][105692] Updated weights for policy 0, policy_version 41935 (0.0011) [2023-12-26 15:40:19,631][105620] Updated weights for policy 1, policy_version 42246 (0.0010) [2023-12-26 15:40:19,691][105620] Updated weights for policy 1, policy_version 42256 (0.0011) [2023-12-26 15:40:20,268][105692] Updated weights for policy 0, policy_version 41945 (0.0006) [2023-12-26 15:40:20,330][105692] Updated weights for policy 0, policy_version 41955 (0.0009) [2023-12-26 15:40:20,387][105692] Updated weights for policy 0, policy_version 41965 (0.0009) [2023-12-26 15:40:20,463][105620] Updated weights for policy 1, policy_version 42266 (0.0011) [2023-12-26 15:40:20,512][105620] Updated weights for policy 1, policy_version 42276 (0.0010) [2023-12-26 15:40:20,565][105620] Updated weights for policy 1, policy_version 42286 (0.0011) [2023-12-26 15:40:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19633.0). Total num frames: 21577728. Throughput: 0: 9641.5, 1: 9830.1. Samples: 21570060. Policy #0 lag: (min: 9.0, avg: 28.9, max: 41.0) [2023-12-26 15:40:21,063][104569] Avg episode reward: [(0, '8853.815'), (1, '9174.986')] [2023-12-26 15:40:21,141][105692] Updated weights for policy 0, policy_version 41975 (0.0010) [2023-12-26 15:40:21,197][105692] Updated weights for policy 0, policy_version 41985 (0.0009) [2023-12-26 15:40:21,258][105692] Updated weights for policy 0, policy_version 41995 (0.0009) [2023-12-26 15:40:21,312][105620] Updated weights for policy 1, policy_version 42296 (0.0007) [2023-12-26 15:40:21,392][105620] Updated weights for policy 1, policy_version 42306 (0.0009) [2023-12-26 15:40:21,459][105620] Updated weights for policy 1, policy_version 42316 (0.0009) [2023-12-26 15:40:22,021][105692] Updated weights for policy 0, policy_version 42005 (0.0009) [2023-12-26 15:40:22,076][105620] Updated weights for policy 1, policy_version 42326 (0.0007) [2023-12-26 15:40:22,086][105692] Updated weights for policy 0, policy_version 42015 (0.0008) [2023-12-26 15:40:22,144][105620] Updated weights for policy 1, policy_version 42336 (0.0009) [2023-12-26 15:40:22,147][105692] Updated weights for policy 0, policy_version 42025 (0.0006) [2023-12-26 15:40:22,203][105620] Updated weights for policy 1, policy_version 42346 (0.0009) [2023-12-26 15:40:22,795][105692] Updated weights for policy 0, policy_version 42035 (0.0006) [2023-12-26 15:40:22,844][105692] Updated weights for policy 0, policy_version 42045 (0.0006) [2023-12-26 15:40:22,900][105692] Updated weights for policy 0, policy_version 42055 (0.0006) [2023-12-26 15:40:22,992][105620] Updated weights for policy 1, policy_version 42356 (0.0008) [2023-12-26 15:40:23,043][105620] Updated weights for policy 1, policy_version 42366 (0.0008) [2023-12-26 15:40:23,096][105620] Updated weights for policy 1, policy_version 42376 (0.0009) [2023-12-26 15:40:23,460][105692] Updated weights for policy 0, policy_version 42065 (0.0006) [2023-12-26 15:40:23,506][105692] Updated weights for policy 0, policy_version 42075 (0.0008) [2023-12-26 15:40:23,562][105692] Updated weights for policy 0, policy_version 42085 (0.0009) [2023-12-26 15:40:23,622][105692] Updated weights for policy 0, policy_version 42095 (0.0009) [2023-12-26 15:40:23,897][105620] Updated weights for policy 1, policy_version 42386 (0.0009) [2023-12-26 15:40:23,950][105620] Updated weights for policy 1, policy_version 42396 (0.0005) [2023-12-26 15:40:24,011][105620] Updated weights for policy 1, policy_version 42406 (0.0005) [2023-12-26 15:40:24,080][105620] Updated weights for policy 1, policy_version 42416 (0.0005) [2023-12-26 15:40:24,378][105692] Updated weights for policy 0, policy_version 42105 (0.0008) [2023-12-26 15:40:24,439][105692] Updated weights for policy 0, policy_version 42115 (0.0009) [2023-12-26 15:40:24,493][105692] Updated weights for policy 0, policy_version 42125 (0.0006) [2023-12-26 15:40:24,686][105620] Updated weights for policy 1, policy_version 42426 (0.0009) [2023-12-26 15:40:24,748][105620] Updated weights for policy 1, policy_version 42436 (0.0009) [2023-12-26 15:40:24,801][105620] Updated weights for policy 1, policy_version 42446 (0.0008) [2023-12-26 15:40:25,176][105692] Updated weights for policy 0, policy_version 42135 (0.0008) [2023-12-26 15:40:25,236][105692] Updated weights for policy 0, policy_version 42145 (0.0009) [2023-12-26 15:40:25,289][105692] Updated weights for policy 0, policy_version 42155 (0.0009) [2023-12-26 15:40:25,558][105620] Updated weights for policy 1, policy_version 42456 (0.0009) [2023-12-26 15:40:25,603][105620] Updated weights for policy 1, policy_version 42466 (0.0008) [2023-12-26 15:40:25,656][105620] Updated weights for policy 1, policy_version 42476 (0.0009) [2023-12-26 15:40:25,970][105692] Updated weights for policy 0, policy_version 42166 (0.0010) [2023-12-26 15:40:26,024][105692] Updated weights for policy 0, policy_version 42176 (0.0010) [2023-12-26 15:40:26,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 21676032. Throughput: 0: 9766.8, 1: 9867.3. Samples: 21686180. Policy #0 lag: (min: 9.0, avg: 28.9, max: 41.0) [2023-12-26 15:40:26,062][104569] Avg episode reward: [(0, '9016.771'), (1, '9259.600')] [2023-12-26 15:40:26,075][105692] Updated weights for policy 0, policy_version 42186 (0.0010) [2023-12-26 15:40:26,475][105620] Updated weights for policy 1, policy_version 42486 (0.0009) [2023-12-26 15:40:26,531][105620] Updated weights for policy 1, policy_version 42496 (0.0008) [2023-12-26 15:40:26,594][105620] Updated weights for policy 1, policy_version 42506 (0.0008) [2023-12-26 15:40:26,791][105692] Updated weights for policy 0, policy_version 42196 (0.0010) [2023-12-26 15:40:26,858][105692] Updated weights for policy 0, policy_version 42206 (0.0010) [2023-12-26 15:40:26,919][105692] Updated weights for policy 0, policy_version 42216 (0.0010) [2023-12-26 15:40:27,346][105620] Updated weights for policy 1, policy_version 42516 (0.0007) [2023-12-26 15:40:27,412][105620] Updated weights for policy 1, policy_version 42526 (0.0007) [2023-12-26 15:40:27,464][105620] Updated weights for policy 1, policy_version 42536 (0.0008) [2023-12-26 15:40:27,646][105692] Updated weights for policy 0, policy_version 42226 (0.0010) [2023-12-26 15:40:27,693][105692] Updated weights for policy 0, policy_version 42236 (0.0010) [2023-12-26 15:40:27,737][105692] Updated weights for policy 0, policy_version 42246 (0.0010) [2023-12-26 15:40:27,784][105692] Updated weights for policy 0, policy_version 42256 (0.0010) [2023-12-26 15:40:28,194][105620] Updated weights for policy 1, policy_version 42546 (0.0008) [2023-12-26 15:40:28,242][105620] Updated weights for policy 1, policy_version 42556 (0.0007) [2023-12-26 15:40:28,293][105620] Updated weights for policy 1, policy_version 42566 (0.0008) [2023-12-26 15:40:28,349][105620] Updated weights for policy 1, policy_version 42576 (0.0009) [2023-12-26 15:40:28,500][105692] Updated weights for policy 0, policy_version 42266 (0.0010) [2023-12-26 15:40:28,552][105692] Updated weights for policy 0, policy_version 42276 (0.0010) [2023-12-26 15:40:28,599][105692] Updated weights for policy 0, policy_version 42286 (0.0010) [2023-12-26 15:40:29,108][105620] Updated weights for policy 1, policy_version 42586 (0.0005) [2023-12-26 15:40:29,152][105620] Updated weights for policy 1, policy_version 42596 (0.0006) [2023-12-26 15:40:29,199][105620] Updated weights for policy 1, policy_version 42606 (0.0006) [2023-12-26 15:40:29,323][105692] Updated weights for policy 0, policy_version 42296 (0.0007) [2023-12-26 15:40:29,391][105692] Updated weights for policy 0, policy_version 42306 (0.0010) [2023-12-26 15:40:29,447][105692] Updated weights for policy 0, policy_version 42316 (0.0010) [2023-12-26 15:40:29,859][105620] Updated weights for policy 1, policy_version 42616 (0.0007) [2023-12-26 15:40:29,923][105620] Updated weights for policy 1, policy_version 42626 (0.0006) [2023-12-26 15:40:30,008][105620] Updated weights for policy 1, policy_version 42636 (0.0007) [2023-12-26 15:40:30,124][105692] Updated weights for policy 0, policy_version 42326 (0.0008) [2023-12-26 15:40:30,187][105692] Updated weights for policy 0, policy_version 42336 (0.0010) [2023-12-26 15:40:30,236][105692] Updated weights for policy 0, policy_version 42346 (0.0010) [2023-12-26 15:40:30,609][105620] Updated weights for policy 1, policy_version 42646 (0.0006) [2023-12-26 15:40:30,670][105620] Updated weights for policy 1, policy_version 42656 (0.0006) [2023-12-26 15:40:30,727][105620] Updated weights for policy 1, policy_version 42666 (0.0011) [2023-12-26 15:40:30,966][105692] Updated weights for policy 0, policy_version 42356 (0.0010) [2023-12-26 15:40:31,015][105692] Updated weights for policy 0, policy_version 42366 (0.0010) [2023-12-26 15:40:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 21774336. Throughput: 0: 9861.1, 1: 9758.1. Samples: 21744184. Policy #0 lag: (min: 9.0, avg: 28.9, max: 41.0) [2023-12-26 15:40:31,062][104569] Avg episode reward: [(0, '8479.475'), (1, '9347.161')] [2023-12-26 15:40:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000042672_10928128.pth... [2023-12-26 15:40:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000041520_10633216.pth [2023-12-26 15:40:31,081][105692] Updated weights for policy 0, policy_version 42376 (0.0010) [2023-12-26 15:40:31,131][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000042384_10854400.pth... [2023-12-26 15:40:31,150][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000041232_10559488.pth [2023-12-26 15:40:31,391][105620] Updated weights for policy 1, policy_version 42676 (0.0009) [2023-12-26 15:40:31,450][105620] Updated weights for policy 1, policy_version 42686 (0.0006) [2023-12-26 15:40:31,516][105620] Updated weights for policy 1, policy_version 42696 (0.0006) [2023-12-26 15:40:31,819][105692] Updated weights for policy 0, policy_version 42386 (0.0010) [2023-12-26 15:40:31,879][105692] Updated weights for policy 0, policy_version 42396 (0.0008) [2023-12-26 15:40:31,940][105692] Updated weights for policy 0, policy_version 42406 (0.0008) [2023-12-26 15:40:32,005][105692] Updated weights for policy 0, policy_version 42416 (0.0008) [2023-12-26 15:40:32,141][105620] Updated weights for policy 1, policy_version 42706 (0.0007) [2023-12-26 15:40:32,195][105620] Updated weights for policy 1, policy_version 42716 (0.0008) [2023-12-26 15:40:32,260][105620] Updated weights for policy 1, policy_version 42726 (0.0006) [2023-12-26 15:40:32,329][105620] Updated weights for policy 1, policy_version 42736 (0.0006) [2023-12-26 15:40:32,704][105692] Updated weights for policy 0, policy_version 42426 (0.0008) [2023-12-26 15:40:32,769][105692] Updated weights for policy 0, policy_version 42436 (0.0007) [2023-12-26 15:40:32,830][105692] Updated weights for policy 0, policy_version 42446 (0.0008) [2023-12-26 15:40:33,001][105620] Updated weights for policy 1, policy_version 42746 (0.0009) [2023-12-26 15:40:33,055][105620] Updated weights for policy 1, policy_version 42756 (0.0009) [2023-12-26 15:40:33,112][105620] Updated weights for policy 1, policy_version 42766 (0.0008) [2023-12-26 15:40:33,583][105692] Updated weights for policy 0, policy_version 42456 (0.0006) [2023-12-26 15:40:33,645][105692] Updated weights for policy 0, policy_version 42466 (0.0005) [2023-12-26 15:40:33,709][105692] Updated weights for policy 0, policy_version 42476 (0.0005) [2023-12-26 15:40:33,920][105620] Updated weights for policy 1, policy_version 42776 (0.0009) [2023-12-26 15:40:33,972][105620] Updated weights for policy 1, policy_version 42786 (0.0008) [2023-12-26 15:40:34,025][105620] Updated weights for policy 1, policy_version 42796 (0.0009) [2023-12-26 15:40:34,270][105692] Updated weights for policy 0, policy_version 42486 (0.0005) [2023-12-26 15:40:34,338][105692] Updated weights for policy 0, policy_version 42496 (0.0006) [2023-12-26 15:40:34,395][105692] Updated weights for policy 0, policy_version 42506 (0.0007) [2023-12-26 15:40:34,815][105620] Updated weights for policy 1, policy_version 42806 (0.0009) [2023-12-26 15:40:34,869][105620] Updated weights for policy 1, policy_version 42816 (0.0010) [2023-12-26 15:40:34,933][105620] Updated weights for policy 1, policy_version 42826 (0.0009) [2023-12-26 15:40:34,973][105692] Updated weights for policy 0, policy_version 42516 (0.0008) [2023-12-26 15:40:35,031][105692] Updated weights for policy 0, policy_version 42526 (0.0009) [2023-12-26 15:40:35,093][105692] Updated weights for policy 0, policy_version 42536 (0.0009) [2023-12-26 15:40:35,686][105620] Updated weights for policy 1, policy_version 42836 (0.0008) [2023-12-26 15:40:35,747][105620] Updated weights for policy 1, policy_version 42846 (0.0009) [2023-12-26 15:40:35,793][105620] Updated weights for policy 1, policy_version 42856 (0.0009) [2023-12-26 15:40:35,840][105692] Updated weights for policy 0, policy_version 42546 (0.0009) [2023-12-26 15:40:35,896][105692] Updated weights for policy 0, policy_version 42556 (0.0005) [2023-12-26 15:40:35,949][105692] Updated weights for policy 0, policy_version 42566 (0.0008) [2023-12-26 15:40:36,003][105692] Updated weights for policy 0, policy_version 42576 (0.0009) [2023-12-26 15:40:36,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.7, 300 sec: 19633.0). Total num frames: 21880832. Throughput: 0: 9899.0, 1: 9809.6. Samples: 21863628. Policy #0 lag: (min: 9.0, avg: 28.9, max: 41.0) [2023-12-26 15:40:36,063][104569] Avg episode reward: [(0, '3773.437'), (1, '9346.324')] [2023-12-26 15:40:36,474][105620] Updated weights for policy 1, policy_version 42866 (0.0008) [2023-12-26 15:40:36,540][105620] Updated weights for policy 1, policy_version 42876 (0.0009) [2023-12-26 15:40:36,594][105620] Updated weights for policy 1, policy_version 42886 (0.0008) [2023-12-26 15:40:36,657][105620] Updated weights for policy 1, policy_version 42896 (0.0009) [2023-12-26 15:40:36,795][105692] Updated weights for policy 0, policy_version 42586 (0.0006) [2023-12-26 15:40:36,846][105692] Updated weights for policy 0, policy_version 42596 (0.0005) [2023-12-26 15:40:36,911][105692] Updated weights for policy 0, policy_version 42606 (0.0007) [2023-12-26 15:40:37,443][105620] Updated weights for policy 1, policy_version 42906 (0.0008) [2023-12-26 15:40:37,502][105620] Updated weights for policy 1, policy_version 42916 (0.0008) [2023-12-26 15:40:37,560][105620] Updated weights for policy 1, policy_version 42926 (0.0008) [2023-12-26 15:40:37,563][105692] Updated weights for policy 0, policy_version 42616 (0.0006) [2023-12-26 15:40:37,629][105692] Updated weights for policy 0, policy_version 42626 (0.0007) [2023-12-26 15:40:37,692][105692] Updated weights for policy 0, policy_version 42636 (0.0007) [2023-12-26 15:40:38,355][105692] Updated weights for policy 0, policy_version 42646 (0.0009) [2023-12-26 15:40:38,358][105620] Updated weights for policy 1, policy_version 42936 (0.0010) [2023-12-26 15:40:38,412][105692] Updated weights for policy 0, policy_version 42656 (0.0008) [2023-12-26 15:40:38,421][105620] Updated weights for policy 1, policy_version 42946 (0.0006) [2023-12-26 15:40:38,474][105692] Updated weights for policy 0, policy_version 42666 (0.0011) [2023-12-26 15:40:38,481][105620] Updated weights for policy 1, policy_version 42956 (0.0009) [2023-12-26 15:40:39,135][105620] Updated weights for policy 1, policy_version 42966 (0.0010) [2023-12-26 15:40:39,182][105692] Updated weights for policy 0, policy_version 42676 (0.0009) [2023-12-26 15:40:39,191][105620] Updated weights for policy 1, policy_version 42976 (0.0007) [2023-12-26 15:40:39,243][105692] Updated weights for policy 0, policy_version 42686 (0.0012) [2023-12-26 15:40:39,254][105620] Updated weights for policy 1, policy_version 42986 (0.0008) [2023-12-26 15:40:39,301][105692] Updated weights for policy 0, policy_version 42696 (0.0010) [2023-12-26 15:40:40,009][105620] Updated weights for policy 1, policy_version 42996 (0.0008) [2023-12-26 15:40:40,039][105692] Updated weights for policy 0, policy_version 42706 (0.0010) [2023-12-26 15:40:40,068][105620] Updated weights for policy 1, policy_version 43006 (0.0008) [2023-12-26 15:40:40,103][105692] Updated weights for policy 0, policy_version 42716 (0.0009) [2023-12-26 15:40:40,134][105620] Updated weights for policy 1, policy_version 43016 (0.0006) [2023-12-26 15:40:40,164][105692] Updated weights for policy 0, policy_version 42726 (0.0008) [2023-12-26 15:40:40,225][105692] Updated weights for policy 0, policy_version 42736 (0.0008) [2023-12-26 15:40:40,863][105620] Updated weights for policy 1, policy_version 43026 (0.0007) [2023-12-26 15:40:40,928][105620] Updated weights for policy 1, policy_version 43036 (0.0008) [2023-12-26 15:40:40,977][105692] Updated weights for policy 0, policy_version 42746 (0.0010) [2023-12-26 15:40:40,989][105620] Updated weights for policy 1, policy_version 43046 (0.0007) [2023-12-26 15:40:41,041][105692] Updated weights for policy 0, policy_version 42756 (0.0009) [2023-12-26 15:40:41,055][105620] Updated weights for policy 1, policy_version 43056 (0.0007) [2023-12-26 15:40:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 21970944. Throughput: 0: 9828.5, 1: 9721.3. Samples: 21979076. Policy #0 lag: (min: 9.0, avg: 28.9, max: 41.0) [2023-12-26 15:40:41,063][104569] Avg episode reward: [(0, '4727.372'), (1, '9175.020')] [2023-12-26 15:40:41,105][105692] Updated weights for policy 0, policy_version 42766 (0.0011) [2023-12-26 15:40:41,777][105620] Updated weights for policy 1, policy_version 43066 (0.0009) [2023-12-26 15:40:41,833][105620] Updated weights for policy 1, policy_version 43076 (0.0008) [2023-12-26 15:40:41,882][105620] Updated weights for policy 1, policy_version 43086 (0.0006) [2023-12-26 15:40:41,892][105692] Updated weights for policy 0, policy_version 42776 (0.0009) [2023-12-26 15:40:41,956][105692] Updated weights for policy 0, policy_version 42786 (0.0007) [2023-12-26 15:40:42,015][105692] Updated weights for policy 0, policy_version 42796 (0.0006) [2023-12-26 15:40:42,593][105620] Updated weights for policy 1, policy_version 43096 (0.0010) [2023-12-26 15:40:42,665][105620] Updated weights for policy 1, policy_version 43106 (0.0010) [2023-12-26 15:40:42,731][105692] Updated weights for policy 0, policy_version 42806 (0.0007) [2023-12-26 15:40:42,733][105620] Updated weights for policy 1, policy_version 43116 (0.0011) [2023-12-26 15:40:42,797][105692] Updated weights for policy 0, policy_version 42816 (0.0008) [2023-12-26 15:40:42,856][105692] Updated weights for policy 0, policy_version 42826 (0.0011) [2023-12-26 15:40:43,429][105620] Updated weights for policy 1, policy_version 43126 (0.0007) [2023-12-26 15:40:43,475][105620] Updated weights for policy 1, policy_version 43136 (0.0005) [2023-12-26 15:40:43,528][105620] Updated weights for policy 1, policy_version 43146 (0.0005) [2023-12-26 15:40:43,528][105692] Updated weights for policy 0, policy_version 42836 (0.0011) [2023-12-26 15:40:43,591][105692] Updated weights for policy 0, policy_version 42846 (0.0010) [2023-12-26 15:40:43,643][105692] Updated weights for policy 0, policy_version 42856 (0.0008) [2023-12-26 15:40:44,068][105620] Updated weights for policy 1, policy_version 43156 (0.0005) [2023-12-26 15:40:44,123][105620] Updated weights for policy 1, policy_version 43166 (0.0005) [2023-12-26 15:40:44,182][105620] Updated weights for policy 1, policy_version 43176 (0.0005) [2023-12-26 15:40:44,223][105692] Updated weights for policy 0, policy_version 42866 (0.0009) [2023-12-26 15:40:44,290][105692] Updated weights for policy 0, policy_version 42876 (0.0009) [2023-12-26 15:40:44,356][105692] Updated weights for policy 0, policy_version 42886 (0.0010) [2023-12-26 15:40:44,421][105692] Updated weights for policy 0, policy_version 42896 (0.0010) [2023-12-26 15:40:44,812][105620] Updated weights for policy 1, policy_version 43186 (0.0006) [2023-12-26 15:40:44,872][105620] Updated weights for policy 1, policy_version 43196 (0.0011) [2023-12-26 15:40:44,929][105620] Updated weights for policy 1, policy_version 43206 (0.0011) [2023-12-26 15:40:44,985][105620] Updated weights for policy 1, policy_version 43216 (0.0011) [2023-12-26 15:40:45,161][105692] Updated weights for policy 0, policy_version 42906 (0.0010) [2023-12-26 15:40:45,214][105692] Updated weights for policy 0, policy_version 42916 (0.0010) [2023-12-26 15:40:45,263][105692] Updated weights for policy 0, policy_version 42926 (0.0010) [2023-12-26 15:40:45,754][105620] Updated weights for policy 1, policy_version 43226 (0.0011) [2023-12-26 15:40:45,811][105620] Updated weights for policy 1, policy_version 43236 (0.0011) [2023-12-26 15:40:45,869][105620] Updated weights for policy 1, policy_version 43246 (0.0010) [2023-12-26 15:40:45,999][105692] Updated weights for policy 0, policy_version 42936 (0.0006) [2023-12-26 15:40:46,057][105692] Updated weights for policy 0, policy_version 42946 (0.0005) [2023-12-26 15:40:46,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 22069248. Throughput: 0: 9774.9, 1: 9734.6. Samples: 22038972. Policy #0 lag: (min: 9.0, avg: 28.9, max: 41.0) [2023-12-26 15:40:46,063][104569] Avg episode reward: [(0, '1903.381'), (1, '9093.527')] [2023-12-26 15:40:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000043248_11075584.pth... [2023-12-26 15:40:46,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000042096_10780672.pth [2023-12-26 15:40:46,119][105692] Updated weights for policy 0, policy_version 42956 (0.0006) [2023-12-26 15:40:46,137][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000042960_11001856.pth... [2023-12-26 15:40:46,140][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000041808_10706944.pth [2023-12-26 15:40:46,470][105620] Updated weights for policy 1, policy_version 43256 (0.0008) [2023-12-26 15:40:46,529][105620] Updated weights for policy 1, policy_version 43266 (0.0010) [2023-12-26 15:40:46,602][105620] Updated weights for policy 1, policy_version 43276 (0.0008) [2023-12-26 15:40:46,740][105692] Updated weights for policy 0, policy_version 42966 (0.0010) [2023-12-26 15:40:46,796][105692] Updated weights for policy 0, policy_version 42976 (0.0009) [2023-12-26 15:40:46,845][105692] Updated weights for policy 0, policy_version 42986 (0.0009) [2023-12-26 15:40:47,188][105620] Updated weights for policy 1, policy_version 43286 (0.0006) [2023-12-26 15:40:47,245][105620] Updated weights for policy 1, policy_version 43296 (0.0006) [2023-12-26 15:40:47,303][105620] Updated weights for policy 1, policy_version 43306 (0.0009) [2023-12-26 15:40:47,661][105692] Updated weights for policy 0, policy_version 42996 (0.0009) [2023-12-26 15:40:47,718][105692] Updated weights for policy 0, policy_version 43006 (0.0008) [2023-12-26 15:40:47,767][105692] Updated weights for policy 0, policy_version 43016 (0.0008) [2023-12-26 15:40:48,010][105620] Updated weights for policy 1, policy_version 43316 (0.0009) [2023-12-26 15:40:48,068][105620] Updated weights for policy 1, policy_version 43326 (0.0010) [2023-12-26 15:40:48,122][105620] Updated weights for policy 1, policy_version 43336 (0.0010) [2023-12-26 15:40:48,506][105692] Updated weights for policy 0, policy_version 43026 (0.0008) [2023-12-26 15:40:48,558][105692] Updated weights for policy 0, policy_version 43036 (0.0008) [2023-12-26 15:40:48,611][105692] Updated weights for policy 0, policy_version 43046 (0.0008) [2023-12-26 15:40:48,660][105692] Updated weights for policy 0, policy_version 43056 (0.0008) [2023-12-26 15:40:48,879][105620] Updated weights for policy 1, policy_version 43346 (0.0010) [2023-12-26 15:40:48,937][105620] Updated weights for policy 1, policy_version 43356 (0.0010) [2023-12-26 15:40:48,999][105620] Updated weights for policy 1, policy_version 43366 (0.0010) [2023-12-26 15:40:49,068][105620] Updated weights for policy 1, policy_version 43376 (0.0011) [2023-12-26 15:40:49,414][105692] Updated weights for policy 0, policy_version 43066 (0.0010) [2023-12-26 15:40:49,475][105692] Updated weights for policy 0, policy_version 43076 (0.0009) [2023-12-26 15:40:49,535][105692] Updated weights for policy 0, policy_version 43086 (0.0009) [2023-12-26 15:40:49,750][105620] Updated weights for policy 1, policy_version 43386 (0.0009) [2023-12-26 15:40:49,804][105620] Updated weights for policy 1, policy_version 43396 (0.0009) [2023-12-26 15:40:49,862][105620] Updated weights for policy 1, policy_version 43406 (0.0007) [2023-12-26 15:40:50,316][105692] Updated weights for policy 0, policy_version 43096 (0.0008) [2023-12-26 15:40:50,364][105692] Updated weights for policy 0, policy_version 43106 (0.0008) [2023-12-26 15:40:50,416][105692] Updated weights for policy 0, policy_version 43116 (0.0008) [2023-12-26 15:40:50,618][105620] Updated weights for policy 1, policy_version 43416 (0.0010) [2023-12-26 15:40:50,671][105620] Updated weights for policy 1, policy_version 43426 (0.0011) [2023-12-26 15:40:50,727][105620] Updated weights for policy 1, policy_version 43436 (0.0010) [2023-12-26 15:40:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 22167552. Throughput: 0: 9777.6, 1: 9799.2. Samples: 22157764. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-26 15:40:51,062][104569] Avg episode reward: [(0, '2868.552'), (1, '9180.156')] [2023-12-26 15:40:51,216][105692] Updated weights for policy 0, policy_version 43126 (0.0008) [2023-12-26 15:40:51,278][105692] Updated weights for policy 0, policy_version 43136 (0.0008) [2023-12-26 15:40:51,332][105692] Updated weights for policy 0, policy_version 43146 (0.0009) [2023-12-26 15:40:51,532][105620] Updated weights for policy 1, policy_version 43446 (0.0009) [2023-12-26 15:40:51,595][105620] Updated weights for policy 1, policy_version 43456 (0.0008) [2023-12-26 15:40:51,662][105620] Updated weights for policy 1, policy_version 43466 (0.0009) [2023-12-26 15:40:52,089][105692] Updated weights for policy 0, policy_version 43156 (0.0008) [2023-12-26 15:40:52,149][105692] Updated weights for policy 0, policy_version 43166 (0.0008) [2023-12-26 15:40:52,209][105692] Updated weights for policy 0, policy_version 43176 (0.0008) [2023-12-26 15:40:52,399][105620] Updated weights for policy 1, policy_version 43476 (0.0010) [2023-12-26 15:40:52,459][105620] Updated weights for policy 1, policy_version 43486 (0.0011) [2023-12-26 15:40:52,525][105620] Updated weights for policy 1, policy_version 43496 (0.0011) [2023-12-26 15:40:52,956][105692] Updated weights for policy 0, policy_version 43186 (0.0008) [2023-12-26 15:40:53,014][105692] Updated weights for policy 0, policy_version 43196 (0.0008) [2023-12-26 15:40:53,069][105692] Updated weights for policy 0, policy_version 43206 (0.0008) [2023-12-26 15:40:53,121][105692] Updated weights for policy 0, policy_version 43216 (0.0008) [2023-12-26 15:40:53,271][105620] Updated weights for policy 1, policy_version 43506 (0.0011) [2023-12-26 15:40:53,332][105620] Updated weights for policy 1, policy_version 43516 (0.0010) [2023-12-26 15:40:53,376][105620] Updated weights for policy 1, policy_version 43526 (0.0010) [2023-12-26 15:40:53,419][105620] Updated weights for policy 1, policy_version 43536 (0.0010) [2023-12-26 15:40:53,873][105692] Updated weights for policy 0, policy_version 43226 (0.0008) [2023-12-26 15:40:53,924][105692] Updated weights for policy 0, policy_version 43236 (0.0008) [2023-12-26 15:40:53,978][105692] Updated weights for policy 0, policy_version 43247 (0.0010) [2023-12-26 15:40:54,135][105620] Updated weights for policy 1, policy_version 43546 (0.0009) [2023-12-26 15:40:54,183][105620] Updated weights for policy 1, policy_version 43556 (0.0010) [2023-12-26 15:40:54,241][105620] Updated weights for policy 1, policy_version 43566 (0.0010) [2023-12-26 15:40:54,782][105692] Updated weights for policy 0, policy_version 43257 (0.0008) [2023-12-26 15:40:54,834][105692] Updated weights for policy 0, policy_version 43267 (0.0008) [2023-12-26 15:40:54,893][105692] Updated weights for policy 0, policy_version 43277 (0.0008) [2023-12-26 15:40:54,970][105620] Updated weights for policy 1, policy_version 43576 (0.0011) [2023-12-26 15:40:55,029][105620] Updated weights for policy 1, policy_version 43586 (0.0011) [2023-12-26 15:40:55,085][105620] Updated weights for policy 1, policy_version 43596 (0.0010) [2023-12-26 15:40:55,637][105692] Updated weights for policy 0, policy_version 43287 (0.0007) [2023-12-26 15:40:55,696][105692] Updated weights for policy 0, policy_version 43297 (0.0005) [2023-12-26 15:40:55,758][105692] Updated weights for policy 0, policy_version 43307 (0.0005) [2023-12-26 15:40:55,828][105620] Updated weights for policy 1, policy_version 43606 (0.0010) [2023-12-26 15:40:55,880][105620] Updated weights for policy 1, policy_version 43616 (0.0010) [2023-12-26 15:40:55,929][105620] Updated weights for policy 1, policy_version 43626 (0.0010) [2023-12-26 15:40:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 22265856. Throughput: 0: 9777.1, 1: 9724.8. Samples: 22270184. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-26 15:40:56,062][104569] Avg episode reward: [(0, '4335.684'), (1, '9179.866')] [2023-12-26 15:40:56,366][105692] Updated weights for policy 0, policy_version 43317 (0.0006) [2023-12-26 15:40:56,413][105692] Updated weights for policy 0, policy_version 43327 (0.0008) [2023-12-26 15:40:56,456][105692] Updated weights for policy 0, policy_version 43337 (0.0005) [2023-12-26 15:40:56,693][105620] Updated weights for policy 1, policy_version 43636 (0.0009) [2023-12-26 15:40:56,748][105620] Updated weights for policy 1, policy_version 43646 (0.0008) [2023-12-26 15:40:56,810][105620] Updated weights for policy 1, policy_version 43656 (0.0010) [2023-12-26 15:40:57,130][105692] Updated weights for policy 0, policy_version 43347 (0.0007) [2023-12-26 15:40:57,184][105692] Updated weights for policy 0, policy_version 43357 (0.0010) [2023-12-26 15:40:57,227][105692] Updated weights for policy 0, policy_version 43367 (0.0010) [2023-12-26 15:40:57,503][105620] Updated weights for policy 1, policy_version 43666 (0.0009) [2023-12-26 15:40:57,560][105620] Updated weights for policy 1, policy_version 43676 (0.0010) [2023-12-26 15:40:57,617][105620] Updated weights for policy 1, policy_version 43686 (0.0009) [2023-12-26 15:40:57,668][105620] Updated weights for policy 1, policy_version 43696 (0.0005) [2023-12-26 15:40:57,860][105692] Updated weights for policy 0, policy_version 43377 (0.0009) [2023-12-26 15:40:57,911][105692] Updated weights for policy 0, policy_version 43387 (0.0009) [2023-12-26 15:40:57,958][105692] Updated weights for policy 0, policy_version 43397 (0.0009) [2023-12-26 15:40:58,006][105692] Updated weights for policy 0, policy_version 43407 (0.0009) [2023-12-26 15:40:58,379][105620] Updated weights for policy 1, policy_version 43706 (0.0009) [2023-12-26 15:40:58,445][105620] Updated weights for policy 1, policy_version 43716 (0.0008) [2023-12-26 15:40:58,510][105620] Updated weights for policy 1, policy_version 43726 (0.0007) [2023-12-26 15:40:58,859][105692] Updated weights for policy 0, policy_version 43417 (0.0008) [2023-12-26 15:40:58,916][105692] Updated weights for policy 0, policy_version 43427 (0.0009) [2023-12-26 15:40:58,984][105692] Updated weights for policy 0, policy_version 43437 (0.0008) [2023-12-26 15:40:59,366][105620] Updated weights for policy 1, policy_version 43736 (0.0008) [2023-12-26 15:40:59,426][105620] Updated weights for policy 1, policy_version 43746 (0.0007) [2023-12-26 15:40:59,491][105620] Updated weights for policy 1, policy_version 43756 (0.0009) [2023-12-26 15:40:59,670][105692] Updated weights for policy 0, policy_version 43447 (0.0008) [2023-12-26 15:40:59,730][105692] Updated weights for policy 0, policy_version 43457 (0.0007) [2023-12-26 15:40:59,788][105692] Updated weights for policy 0, policy_version 43467 (0.0005) [2023-12-26 15:41:00,096][105620] Updated weights for policy 1, policy_version 43766 (0.0006) [2023-12-26 15:41:00,147][105620] Updated weights for policy 1, policy_version 43776 (0.0005) [2023-12-26 15:41:00,203][105620] Updated weights for policy 1, policy_version 43786 (0.0009) [2023-12-26 15:41:00,484][105692] Updated weights for policy 0, policy_version 43477 (0.0007) [2023-12-26 15:41:00,540][105692] Updated weights for policy 0, policy_version 43487 (0.0008) [2023-12-26 15:41:00,610][105692] Updated weights for policy 0, policy_version 43497 (0.0008) [2023-12-26 15:41:00,869][105620] Updated weights for policy 1, policy_version 43797 (0.0010) [2023-12-26 15:41:00,922][105620] Updated weights for policy 1, policy_version 43807 (0.0009) [2023-12-26 15:41:00,975][105620] Updated weights for policy 1, policy_version 43817 (0.0010) [2023-12-26 15:41:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 22364160. Throughput: 0: 9781.6, 1: 9734.1. Samples: 22329528. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-26 15:41:01,063][104569] Avg episode reward: [(0, '7146.373'), (1, '9350.140')] [2023-12-26 15:41:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000043504_11141120.pth... [2023-12-26 15:41:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000043824_11223040.pth... [2023-12-26 15:41:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000042672_10928128.pth [2023-12-26 15:41:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000042384_10854400.pth [2023-12-26 15:41:01,292][105692] Updated weights for policy 0, policy_version 43507 (0.0010) [2023-12-26 15:41:01,359][105692] Updated weights for policy 0, policy_version 43517 (0.0009) [2023-12-26 15:41:01,424][105692] Updated weights for policy 0, policy_version 43527 (0.0009) [2023-12-26 15:41:01,713][105620] Updated weights for policy 1, policy_version 43827 (0.0009) [2023-12-26 15:41:01,779][105620] Updated weights for policy 1, policy_version 43837 (0.0009) [2023-12-26 15:41:01,832][105620] Updated weights for policy 1, policy_version 43847 (0.0007) [2023-12-26 15:41:02,247][105692] Updated weights for policy 0, policy_version 43538 (0.0009) [2023-12-26 15:41:02,303][105692] Updated weights for policy 0, policy_version 43548 (0.0009) [2023-12-26 15:41:02,359][105692] Updated weights for policy 0, policy_version 43558 (0.0008) [2023-12-26 15:41:02,423][105692] Updated weights for policy 0, policy_version 43568 (0.0009) [2023-12-26 15:41:02,566][105620] Updated weights for policy 1, policy_version 43857 (0.0009) [2023-12-26 15:41:02,623][105620] Updated weights for policy 1, policy_version 43867 (0.0009) [2023-12-26 15:41:02,674][105620] Updated weights for policy 1, policy_version 43877 (0.0009) [2023-12-26 15:41:02,734][105620] Updated weights for policy 1, policy_version 43887 (0.0008) [2023-12-26 15:41:03,226][105692] Updated weights for policy 0, policy_version 43578 (0.0007) [2023-12-26 15:41:03,270][105692] Updated weights for policy 0, policy_version 43588 (0.0006) [2023-12-26 15:41:03,319][105692] Updated weights for policy 0, policy_version 43598 (0.0007) [2023-12-26 15:41:03,415][105620] Updated weights for policy 1, policy_version 43897 (0.0008) [2023-12-26 15:41:03,470][105620] Updated weights for policy 1, policy_version 43907 (0.0008) [2023-12-26 15:41:03,526][105620] Updated weights for policy 1, policy_version 43917 (0.0008) [2023-12-26 15:41:04,063][105692] Updated weights for policy 0, policy_version 43608 (0.0009) [2023-12-26 15:41:04,120][105692] Updated weights for policy 0, policy_version 43618 (0.0008) [2023-12-26 15:41:04,180][105692] Updated weights for policy 0, policy_version 43628 (0.0008) [2023-12-26 15:41:04,292][105620] Updated weights for policy 1, policy_version 43927 (0.0010) [2023-12-26 15:41:04,360][105620] Updated weights for policy 1, policy_version 43937 (0.0010) [2023-12-26 15:41:04,422][105620] Updated weights for policy 1, policy_version 43947 (0.0011) [2023-12-26 15:41:04,964][105692] Updated weights for policy 0, policy_version 43638 (0.0008) [2023-12-26 15:41:05,020][105692] Updated weights for policy 0, policy_version 43648 (0.0008) [2023-12-26 15:41:05,075][105692] Updated weights for policy 0, policy_version 43658 (0.0008) [2023-12-26 15:41:05,168][105620] Updated weights for policy 1, policy_version 43957 (0.0011) [2023-12-26 15:41:05,227][105620] Updated weights for policy 1, policy_version 43967 (0.0010) [2023-12-26 15:41:05,286][105620] Updated weights for policy 1, policy_version 43977 (0.0011) [2023-12-26 15:41:05,829][105692] Updated weights for policy 0, policy_version 43668 (0.0007) [2023-12-26 15:41:05,877][105692] Updated weights for policy 0, policy_version 43678 (0.0008) [2023-12-26 15:41:05,929][105692] Updated weights for policy 0, policy_version 43688 (0.0008) [2023-12-26 15:41:06,021][105620] Updated weights for policy 1, policy_version 43987 (0.0010) [2023-12-26 15:41:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 22454272. Throughput: 0: 9729.7, 1: 9703.3. Samples: 22444544. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-26 15:41:06,062][104569] Avg episode reward: [(0, '7919.593'), (1, '9260.777')] [2023-12-26 15:41:06,076][105620] Updated weights for policy 1, policy_version 43997 (0.0010) [2023-12-26 15:41:06,139][105620] Updated weights for policy 1, policy_version 44007 (0.0011) [2023-12-26 15:41:06,705][105692] Updated weights for policy 0, policy_version 43698 (0.0008) [2023-12-26 15:41:06,769][105692] Updated weights for policy 0, policy_version 43708 (0.0008) [2023-12-26 15:41:06,829][105692] Updated weights for policy 0, policy_version 43718 (0.0008) [2023-12-26 15:41:06,885][105620] Updated weights for policy 1, policy_version 44017 (0.0010) [2023-12-26 15:41:06,892][105692] Updated weights for policy 0, policy_version 43728 (0.0008) [2023-12-26 15:41:06,945][105620] Updated weights for policy 1, policy_version 44027 (0.0011) [2023-12-26 15:41:07,005][105620] Updated weights for policy 1, policy_version 44037 (0.0011) [2023-12-26 15:41:07,059][105620] Updated weights for policy 1, policy_version 44047 (0.0011) [2023-12-26 15:41:07,651][105692] Updated weights for policy 0, policy_version 43738 (0.0008) [2023-12-26 15:41:07,702][105692] Updated weights for policy 0, policy_version 43748 (0.0008) [2023-12-26 15:41:07,757][105692] Updated weights for policy 0, policy_version 43758 (0.0008) [2023-12-26 15:41:07,820][105620] Updated weights for policy 1, policy_version 44057 (0.0010) [2023-12-26 15:41:07,876][105620] Updated weights for policy 1, policy_version 44067 (0.0010) [2023-12-26 15:41:07,929][105620] Updated weights for policy 1, policy_version 44077 (0.0010) [2023-12-26 15:41:08,525][105692] Updated weights for policy 0, policy_version 43768 (0.0008) [2023-12-26 15:41:08,581][105692] Updated weights for policy 0, policy_version 43778 (0.0008) [2023-12-26 15:41:08,629][105692] Updated weights for policy 0, policy_version 43788 (0.0008) [2023-12-26 15:41:08,678][105620] Updated weights for policy 1, policy_version 44087 (0.0011) [2023-12-26 15:41:08,733][105620] Updated weights for policy 1, policy_version 44097 (0.0010) [2023-12-26 15:41:08,794][105620] Updated weights for policy 1, policy_version 44107 (0.0010) [2023-12-26 15:41:09,395][105692] Updated weights for policy 0, policy_version 43798 (0.0008) [2023-12-26 15:41:09,454][105692] Updated weights for policy 0, policy_version 43808 (0.0008) [2023-12-26 15:41:09,506][105692] Updated weights for policy 0, policy_version 43818 (0.0008) [2023-12-26 15:41:09,569][105620] Updated weights for policy 1, policy_version 44117 (0.0011) [2023-12-26 15:41:09,637][105620] Updated weights for policy 1, policy_version 44127 (0.0009) [2023-12-26 15:41:09,700][105620] Updated weights for policy 1, policy_version 44137 (0.0010) [2023-12-26 15:41:10,285][105692] Updated weights for policy 0, policy_version 43828 (0.0008) [2023-12-26 15:41:10,345][105692] Updated weights for policy 0, policy_version 43838 (0.0008) [2023-12-26 15:41:10,404][105692] Updated weights for policy 0, policy_version 43848 (0.0008) [2023-12-26 15:41:10,437][105620] Updated weights for policy 1, policy_version 44147 (0.0010) [2023-12-26 15:41:10,486][105620] Updated weights for policy 1, policy_version 44157 (0.0010) [2023-12-26 15:41:10,534][105620] Updated weights for policy 1, policy_version 44167 (0.0010) [2023-12-26 15:41:11,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 22544384. Throughput: 0: 9627.4, 1: 9680.1. Samples: 22555016. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-26 15:41:11,063][104569] Avg episode reward: [(0, '8850.020'), (1, '9261.138')] [2023-12-26 15:41:11,176][105692] Updated weights for policy 0, policy_version 43858 (0.0008) [2023-12-26 15:41:11,239][105692] Updated weights for policy 0, policy_version 43868 (0.0008) [2023-12-26 15:41:11,305][105692] Updated weights for policy 0, policy_version 43878 (0.0008) [2023-12-26 15:41:11,342][105620] Updated weights for policy 1, policy_version 44177 (0.0010) [2023-12-26 15:41:11,381][105692] Updated weights for policy 0, policy_version 43888 (0.0009) [2023-12-26 15:41:11,413][105620] Updated weights for policy 1, policy_version 44187 (0.0011) [2023-12-26 15:41:11,468][105620] Updated weights for policy 1, policy_version 44197 (0.0010) [2023-12-26 15:41:11,523][105620] Updated weights for policy 1, policy_version 44207 (0.0010) [2023-12-26 15:41:12,136][105692] Updated weights for policy 0, policy_version 43898 (0.0008) [2023-12-26 15:41:12,191][105692] Updated weights for policy 0, policy_version 43908 (0.0008) [2023-12-26 15:41:12,240][105692] Updated weights for policy 0, policy_version 43918 (0.0008) [2023-12-26 15:41:12,303][105620] Updated weights for policy 1, policy_version 44217 (0.0010) [2023-12-26 15:41:12,373][105620] Updated weights for policy 1, policy_version 44227 (0.0011) [2023-12-26 15:41:12,432][105620] Updated weights for policy 1, policy_version 44237 (0.0011) [2023-12-26 15:41:12,928][105692] Updated weights for policy 0, policy_version 43928 (0.0006) [2023-12-26 15:41:12,989][105692] Updated weights for policy 0, policy_version 43938 (0.0006) [2023-12-26 15:41:13,039][105692] Updated weights for policy 0, policy_version 43948 (0.0005) [2023-12-26 15:41:13,189][105620] Updated weights for policy 1, policy_version 44247 (0.0010) [2023-12-26 15:41:13,251][105620] Updated weights for policy 1, policy_version 44257 (0.0010) [2023-12-26 15:41:13,312][105620] Updated weights for policy 1, policy_version 44267 (0.0010) [2023-12-26 15:41:13,652][105692] Updated weights for policy 0, policy_version 43958 (0.0008) [2023-12-26 15:41:13,701][105692] Updated weights for policy 0, policy_version 43968 (0.0008) [2023-12-26 15:41:13,756][105692] Updated weights for policy 0, policy_version 43978 (0.0009) [2023-12-26 15:41:14,007][105620] Updated weights for policy 1, policy_version 44277 (0.0008) [2023-12-26 15:41:14,073][105620] Updated weights for policy 1, policy_version 44287 (0.0005) [2023-12-26 15:41:14,127][105620] Updated weights for policy 1, policy_version 44297 (0.0008) [2023-12-26 15:41:14,490][105692] Updated weights for policy 0, policy_version 43988 (0.0009) [2023-12-26 15:41:14,542][105692] Updated weights for policy 0, policy_version 43998 (0.0009) [2023-12-26 15:41:14,595][105692] Updated weights for policy 0, policy_version 44008 (0.0010) [2023-12-26 15:41:14,780][105620] Updated weights for policy 1, policy_version 44307 (0.0009) [2023-12-26 15:41:14,842][105620] Updated weights for policy 1, policy_version 44317 (0.0010) [2023-12-26 15:41:14,904][105620] Updated weights for policy 1, policy_version 44327 (0.0009) [2023-12-26 15:41:15,410][105692] Updated weights for policy 0, policy_version 44018 (0.0009) [2023-12-26 15:41:15,464][105692] Updated weights for policy 0, policy_version 44028 (0.0008) [2023-12-26 15:41:15,514][105692] Updated weights for policy 0, policy_version 44038 (0.0009) [2023-12-26 15:41:15,571][105692] Updated weights for policy 0, policy_version 44048 (0.0009) [2023-12-26 15:41:15,607][105620] Updated weights for policy 1, policy_version 44337 (0.0008) [2023-12-26 15:41:15,669][105620] Updated weights for policy 1, policy_version 44347 (0.0008) [2023-12-26 15:41:15,723][105620] Updated weights for policy 1, policy_version 44357 (0.0009) [2023-12-26 15:41:15,781][105620] Updated weights for policy 1, policy_version 44367 (0.0009) [2023-12-26 15:41:16,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19387.8, 300 sec: 19605.2). Total num frames: 22642688. Throughput: 0: 9622.4, 1: 9677.7. Samples: 22612696. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 15:41:16,063][104569] Avg episode reward: [(0, '8861.786'), (1, '9169.470')] [2023-12-26 15:41:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000044368_11362304.pth... [2023-12-26 15:41:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000044048_11280384.pth... [2023-12-26 15:41:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000043248_11075584.pth [2023-12-26 15:41:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000042960_11001856.pth [2023-12-26 15:41:16,207][105692] Updated weights for policy 0, policy_version 44058 (0.0007) [2023-12-26 15:41:16,255][105692] Updated weights for policy 0, policy_version 44068 (0.0005) [2023-12-26 15:41:16,306][105692] Updated weights for policy 0, policy_version 44078 (0.0005) [2023-12-26 15:41:16,477][105620] Updated weights for policy 1, policy_version 44377 (0.0010) [2023-12-26 15:41:16,525][105620] Updated weights for policy 1, policy_version 44387 (0.0010) [2023-12-26 15:41:16,584][105620] Updated weights for policy 1, policy_version 44397 (0.0010) [2023-12-26 15:41:16,903][105692] Updated weights for policy 0, policy_version 44088 (0.0005) [2023-12-26 15:41:16,948][105692] Updated weights for policy 0, policy_version 44098 (0.0006) [2023-12-26 15:41:17,003][105692] Updated weights for policy 0, policy_version 44108 (0.0010) [2023-12-26 15:41:17,186][105620] Updated weights for policy 1, policy_version 44407 (0.0007) [2023-12-26 15:41:17,241][105620] Updated weights for policy 1, policy_version 44417 (0.0005) [2023-12-26 15:41:17,288][105620] Updated weights for policy 1, policy_version 44427 (0.0009) [2023-12-26 15:41:17,596][105692] Updated weights for policy 0, policy_version 44118 (0.0007) [2023-12-26 15:41:17,666][105692] Updated weights for policy 0, policy_version 44128 (0.0005) [2023-12-26 15:41:17,720][105692] Updated weights for policy 0, policy_version 44138 (0.0006) [2023-12-26 15:41:18,022][105620] Updated weights for policy 1, policy_version 44437 (0.0011) [2023-12-26 15:41:18,080][105620] Updated weights for policy 1, policy_version 44447 (0.0010) [2023-12-26 15:41:18,138][105620] Updated weights for policy 1, policy_version 44457 (0.0010) [2023-12-26 15:41:18,318][105692] Updated weights for policy 0, policy_version 44148 (0.0011) [2023-12-26 15:41:18,393][105692] Updated weights for policy 0, policy_version 44158 (0.0011) [2023-12-26 15:41:18,453][105692] Updated weights for policy 0, policy_version 44168 (0.0011) [2023-12-26 15:41:18,735][105620] Updated weights for policy 1, policy_version 44467 (0.0009) [2023-12-26 15:41:18,790][105620] Updated weights for policy 1, policy_version 44477 (0.0005) [2023-12-26 15:41:18,854][105620] Updated weights for policy 1, policy_version 44487 (0.0008) [2023-12-26 15:41:19,166][105692] Updated weights for policy 0, policy_version 44178 (0.0009) [2023-12-26 15:41:19,230][105692] Updated weights for policy 0, policy_version 44188 (0.0006) [2023-12-26 15:41:19,289][105692] Updated weights for policy 0, policy_version 44198 (0.0009) [2023-12-26 15:41:19,345][105692] Updated weights for policy 0, policy_version 44208 (0.0008) [2023-12-26 15:41:19,518][105620] Updated weights for policy 1, policy_version 44497 (0.0010) [2023-12-26 15:41:19,569][105620] Updated weights for policy 1, policy_version 44507 (0.0009) [2023-12-26 15:41:19,637][105620] Updated weights for policy 1, policy_version 44517 (0.0010) [2023-12-26 15:41:19,707][105620] Updated weights for policy 1, policy_version 44527 (0.0011) [2023-12-26 15:41:20,036][105692] Updated weights for policy 0, policy_version 44218 (0.0008) [2023-12-26 15:41:20,102][105692] Updated weights for policy 0, policy_version 44228 (0.0007) [2023-12-26 15:41:20,163][105692] Updated weights for policy 0, policy_version 44238 (0.0007) [2023-12-26 15:41:20,464][105620] Updated weights for policy 1, policy_version 44537 (0.0010) [2023-12-26 15:41:20,523][105620] Updated weights for policy 1, policy_version 44547 (0.0008) [2023-12-26 15:41:20,586][105620] Updated weights for policy 1, policy_version 44557 (0.0011) [2023-12-26 15:41:20,862][105692] Updated weights for policy 0, policy_version 44248 (0.0007) [2023-12-26 15:41:20,907][105692] Updated weights for policy 0, policy_version 44258 (0.0008) [2023-12-26 15:41:20,965][105692] Updated weights for policy 0, policy_version 44268 (0.0005) [2023-12-26 15:41:21,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 22749184. Throughput: 0: 9657.9, 1: 9718.9. Samples: 22735580. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 15:41:21,062][104569] Avg episode reward: [(0, '9031.303'), (1, '8995.657')] [2023-12-26 15:41:21,335][105620] Updated weights for policy 1, policy_version 44567 (0.0010) [2023-12-26 15:41:21,408][105620] Updated weights for policy 1, policy_version 44577 (0.0011) [2023-12-26 15:41:21,467][105620] Updated weights for policy 1, policy_version 44587 (0.0011) [2023-12-26 15:41:21,704][105692] Updated weights for policy 0, policy_version 44278 (0.0007) [2023-12-26 15:41:21,775][105692] Updated weights for policy 0, policy_version 44288 (0.0008) [2023-12-26 15:41:21,837][105692] Updated weights for policy 0, policy_version 44298 (0.0008) [2023-12-26 15:41:22,221][105620] Updated weights for policy 1, policy_version 44597 (0.0008) [2023-12-26 15:41:22,285][105620] Updated weights for policy 1, policy_version 44607 (0.0007) [2023-12-26 15:41:22,349][105620] Updated weights for policy 1, policy_version 44617 (0.0007) [2023-12-26 15:41:22,642][105692] Updated weights for policy 0, policy_version 44308 (0.0009) [2023-12-26 15:41:22,690][105692] Updated weights for policy 0, policy_version 44318 (0.0009) [2023-12-26 15:41:22,744][105692] Updated weights for policy 0, policy_version 44328 (0.0009) [2023-12-26 15:41:23,103][105620] Updated weights for policy 1, policy_version 44627 (0.0008) [2023-12-26 15:41:23,161][105620] Updated weights for policy 1, policy_version 44637 (0.0009) [2023-12-26 15:41:23,214][105620] Updated weights for policy 1, policy_version 44647 (0.0008) [2023-12-26 15:41:23,453][105692] Updated weights for policy 0, policy_version 44338 (0.0009) [2023-12-26 15:41:23,503][105692] Updated weights for policy 0, policy_version 44348 (0.0008) [2023-12-26 15:41:23,554][105692] Updated weights for policy 0, policy_version 44358 (0.0005) [2023-12-26 15:41:23,609][105692] Updated weights for policy 0, policy_version 44368 (0.0006) [2023-12-26 15:41:23,979][105620] Updated weights for policy 1, policy_version 44657 (0.0009) [2023-12-26 15:41:24,033][105620] Updated weights for policy 1, policy_version 44667 (0.0008) [2023-12-26 15:41:24,100][105620] Updated weights for policy 1, policy_version 44677 (0.0009) [2023-12-26 15:41:24,158][105620] Updated weights for policy 1, policy_version 44687 (0.0009) [2023-12-26 15:41:24,305][105692] Updated weights for policy 0, policy_version 44378 (0.0009) [2023-12-26 15:41:24,360][105692] Updated weights for policy 0, policy_version 44388 (0.0010) [2023-12-26 15:41:24,408][105692] Updated weights for policy 0, policy_version 44398 (0.0010) [2023-12-26 15:41:24,964][105620] Updated weights for policy 1, policy_version 44698 (0.0009) [2023-12-26 15:41:25,018][105620] Updated weights for policy 1, policy_version 44708 (0.0006) [2023-12-26 15:41:25,019][105692] Updated weights for policy 0, policy_version 44408 (0.0010) [2023-12-26 15:41:25,079][105620] Updated weights for policy 1, policy_version 44718 (0.0006) [2023-12-26 15:41:25,084][105692] Updated weights for policy 0, policy_version 44418 (0.0011) [2023-12-26 15:41:25,139][105692] Updated weights for policy 0, policy_version 44428 (0.0010) [2023-12-26 15:41:25,762][105692] Updated weights for policy 0, policy_version 44438 (0.0007) [2023-12-26 15:41:25,772][105620] Updated weights for policy 1, policy_version 44728 (0.0008) [2023-12-26 15:41:25,826][105692] Updated weights for policy 0, policy_version 44448 (0.0007) [2023-12-26 15:41:25,828][105620] Updated weights for policy 1, policy_version 44738 (0.0007) [2023-12-26 15:41:25,877][105692] Updated weights for policy 0, policy_version 44458 (0.0009) [2023-12-26 15:41:25,880][105620] Updated weights for policy 1, policy_version 44748 (0.0006) [2023-12-26 15:41:26,062][104569] Fps is (10 sec: 20480.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 22847488. Throughput: 0: 9686.9, 1: 9685.3. Samples: 22850824. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 15:41:26,062][104569] Avg episode reward: [(0, '9280.423'), (1, '9176.878')] [2023-12-26 15:41:26,469][105692] Updated weights for policy 0, policy_version 44468 (0.0005) [2023-12-26 15:41:26,520][105692] Updated weights for policy 0, policy_version 44478 (0.0005) [2023-12-26 15:41:26,575][105692] Updated weights for policy 0, policy_version 44488 (0.0007) [2023-12-26 15:41:26,671][105620] Updated weights for policy 1, policy_version 44758 (0.0007) [2023-12-26 15:41:26,733][105620] Updated weights for policy 1, policy_version 44768 (0.0005) [2023-12-26 15:41:26,791][105620] Updated weights for policy 1, policy_version 44778 (0.0005) [2023-12-26 15:41:27,299][105692] Updated weights for policy 0, policy_version 44498 (0.0010) [2023-12-26 15:41:27,360][105692] Updated weights for policy 0, policy_version 44508 (0.0010) [2023-12-26 15:41:27,403][105620] Updated weights for policy 1, policy_version 44788 (0.0007) [2023-12-26 15:41:27,415][105692] Updated weights for policy 0, policy_version 44518 (0.0010) [2023-12-26 15:41:27,452][105620] Updated weights for policy 1, policy_version 44798 (0.0007) [2023-12-26 15:41:27,479][105692] Updated weights for policy 0, policy_version 44528 (0.0010) [2023-12-26 15:41:27,504][105620] Updated weights for policy 1, policy_version 44808 (0.0007) [2023-12-26 15:41:28,149][105692] Updated weights for policy 0, policy_version 44538 (0.0010) [2023-12-26 15:41:28,209][105692] Updated weights for policy 0, policy_version 44548 (0.0010) [2023-12-26 15:41:28,258][105620] Updated weights for policy 1, policy_version 44818 (0.0007) [2023-12-26 15:41:28,270][105692] Updated weights for policy 0, policy_version 44558 (0.0010) [2023-12-26 15:41:28,318][105620] Updated weights for policy 1, policy_version 44828 (0.0007) [2023-12-26 15:41:28,381][105620] Updated weights for policy 1, policy_version 44838 (0.0008) [2023-12-26 15:41:28,446][105620] Updated weights for policy 1, policy_version 44848 (0.0008) [2023-12-26 15:41:28,860][105692] Updated weights for policy 0, policy_version 44568 (0.0010) [2023-12-26 15:41:28,918][105692] Updated weights for policy 0, policy_version 44578 (0.0010) [2023-12-26 15:41:28,969][105692] Updated weights for policy 0, policy_version 44588 (0.0009) [2023-12-26 15:41:29,155][105620] Updated weights for policy 1, policy_version 44858 (0.0008) [2023-12-26 15:41:29,226][105620] Updated weights for policy 1, policy_version 44868 (0.0010) [2023-12-26 15:41:29,292][105620] Updated weights for policy 1, policy_version 44878 (0.0008) [2023-12-26 15:41:29,700][105692] Updated weights for policy 0, policy_version 44598 (0.0008) [2023-12-26 15:41:29,759][105692] Updated weights for policy 0, policy_version 44608 (0.0011) [2023-12-26 15:41:29,810][105692] Updated weights for policy 0, policy_version 44618 (0.0008) [2023-12-26 15:41:29,962][105620] Updated weights for policy 1, policy_version 44888 (0.0006) [2023-12-26 15:41:30,012][105620] Updated weights for policy 1, policy_version 44898 (0.0006) [2023-12-26 15:41:30,057][105620] Updated weights for policy 1, policy_version 44908 (0.0010) [2023-12-26 15:41:30,507][105692] Updated weights for policy 0, policy_version 44628 (0.0009) [2023-12-26 15:41:30,555][105692] Updated weights for policy 0, policy_version 44638 (0.0010) [2023-12-26 15:41:30,608][105692] Updated weights for policy 0, policy_version 44648 (0.0011) [2023-12-26 15:41:30,693][105620] Updated weights for policy 1, policy_version 44918 (0.0008) [2023-12-26 15:41:30,764][105620] Updated weights for policy 1, policy_version 44928 (0.0005) [2023-12-26 15:41:30,811][105620] Updated weights for policy 1, policy_version 44938 (0.0005) [2023-12-26 15:41:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 22945792. Throughput: 0: 9737.9, 1: 9657.8. Samples: 22911776. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 15:41:31,063][104569] Avg episode reward: [(0, '9280.487'), (1, '9081.148')] [2023-12-26 15:41:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000044944_11509760.pth... [2023-12-26 15:41:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000044656_11436032.pth... [2023-12-26 15:41:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000043504_11141120.pth [2023-12-26 15:41:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000043824_11223040.pth [2023-12-26 15:41:31,384][105692] Updated weights for policy 0, policy_version 44658 (0.0011) [2023-12-26 15:41:31,430][105620] Updated weights for policy 1, policy_version 44948 (0.0008) [2023-12-26 15:41:31,444][105692] Updated weights for policy 0, policy_version 44668 (0.0011) [2023-12-26 15:41:31,491][105620] Updated weights for policy 1, policy_version 44958 (0.0011) [2023-12-26 15:41:31,504][105692] Updated weights for policy 0, policy_version 44678 (0.0009) [2023-12-26 15:41:31,560][105620] Updated weights for policy 1, policy_version 44968 (0.0011) [2023-12-26 15:41:31,573][105692] Updated weights for policy 0, policy_version 44688 (0.0008) [2023-12-26 15:41:32,213][105692] Updated weights for policy 0, policy_version 44698 (0.0008) [2023-12-26 15:41:32,283][105692] Updated weights for policy 0, policy_version 44708 (0.0010) [2023-12-26 15:41:32,309][105620] Updated weights for policy 1, policy_version 44978 (0.0010) [2023-12-26 15:41:32,340][105692] Updated weights for policy 0, policy_version 44718 (0.0007) [2023-12-26 15:41:32,377][105620] Updated weights for policy 1, policy_version 44988 (0.0011) [2023-12-26 15:41:32,443][105620] Updated weights for policy 1, policy_version 44998 (0.0011) [2023-12-26 15:41:32,499][105620] Updated weights for policy 1, policy_version 45008 (0.0011) [2023-12-26 15:41:32,903][105692] Updated weights for policy 0, policy_version 44728 (0.0006) [2023-12-26 15:41:32,961][105692] Updated weights for policy 0, policy_version 44738 (0.0005) [2023-12-26 15:41:33,024][105692] Updated weights for policy 0, policy_version 44748 (0.0005) [2023-12-26 15:41:33,226][105620] Updated weights for policy 1, policy_version 45018 (0.0005) [2023-12-26 15:41:33,285][105620] Updated weights for policy 1, policy_version 45028 (0.0007) [2023-12-26 15:41:33,339][105620] Updated weights for policy 1, policy_version 45038 (0.0008) [2023-12-26 15:41:33,619][105692] Updated weights for policy 0, policy_version 44758 (0.0005) [2023-12-26 15:41:33,678][105692] Updated weights for policy 0, policy_version 44768 (0.0005) [2023-12-26 15:41:33,736][105692] Updated weights for policy 0, policy_version 44778 (0.0006) [2023-12-26 15:41:34,078][105620] Updated weights for policy 1, policy_version 45048 (0.0008) [2023-12-26 15:41:34,125][105620] Updated weights for policy 1, policy_version 45058 (0.0007) [2023-12-26 15:41:34,185][105620] Updated weights for policy 1, policy_version 45068 (0.0007) [2023-12-26 15:41:34,378][105692] Updated weights for policy 0, policy_version 44788 (0.0008) [2023-12-26 15:41:34,443][105692] Updated weights for policy 0, policy_version 44798 (0.0009) [2023-12-26 15:41:34,501][105692] Updated weights for policy 0, policy_version 44808 (0.0008) [2023-12-26 15:41:34,966][105620] Updated weights for policy 1, policy_version 45078 (0.0008) [2023-12-26 15:41:35,029][105620] Updated weights for policy 1, policy_version 45088 (0.0010) [2023-12-26 15:41:35,091][105620] Updated weights for policy 1, policy_version 45098 (0.0009) [2023-12-26 15:41:35,194][105692] Updated weights for policy 0, policy_version 44818 (0.0005) [2023-12-26 15:41:35,259][105692] Updated weights for policy 0, policy_version 44828 (0.0007) [2023-12-26 15:41:35,321][105692] Updated weights for policy 0, policy_version 44838 (0.0009) [2023-12-26 15:41:35,378][105692] Updated weights for policy 0, policy_version 44848 (0.0009) [2023-12-26 15:41:35,874][105620] Updated weights for policy 1, policy_version 45108 (0.0008) [2023-12-26 15:41:35,935][105620] Updated weights for policy 1, policy_version 45118 (0.0005) [2023-12-26 15:41:35,988][105620] Updated weights for policy 1, policy_version 45128 (0.0008) [2023-12-26 15:41:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 23044096. Throughput: 0: 9833.4, 1: 9615.5. Samples: 23032964. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 15:41:36,062][104569] Avg episode reward: [(0, '9103.897'), (1, '8902.107')] [2023-12-26 15:41:36,108][105692] Updated weights for policy 0, policy_version 44858 (0.0009) [2023-12-26 15:41:36,171][105692] Updated weights for policy 0, policy_version 44868 (0.0007) [2023-12-26 15:41:36,231][105692] Updated weights for policy 0, policy_version 44878 (0.0005) [2023-12-26 15:41:36,703][105620] Updated weights for policy 1, policy_version 45138 (0.0009) [2023-12-26 15:41:36,758][105620] Updated weights for policy 1, policy_version 45148 (0.0010) [2023-12-26 15:41:36,821][105620] Updated weights for policy 1, policy_version 45158 (0.0011) [2023-12-26 15:41:36,876][105620] Updated weights for policy 1, policy_version 45168 (0.0010) [2023-12-26 15:41:36,919][105692] Updated weights for policy 0, policy_version 44888 (0.0007) [2023-12-26 15:41:36,980][105692] Updated weights for policy 0, policy_version 44898 (0.0008) [2023-12-26 15:41:37,048][105692] Updated weights for policy 0, policy_version 44908 (0.0008) [2023-12-26 15:41:37,557][105620] Updated weights for policy 1, policy_version 45178 (0.0006) [2023-12-26 15:41:37,617][105620] Updated weights for policy 1, policy_version 45188 (0.0005) [2023-12-26 15:41:37,689][105620] Updated weights for policy 1, policy_version 45198 (0.0005) [2023-12-26 15:41:37,765][105692] Updated weights for policy 0, policy_version 44918 (0.0007) [2023-12-26 15:41:37,814][105692] Updated weights for policy 0, policy_version 44928 (0.0005) [2023-12-26 15:41:37,866][105692] Updated weights for policy 0, policy_version 44938 (0.0005) [2023-12-26 15:41:38,224][105620] Updated weights for policy 1, policy_version 45208 (0.0008) [2023-12-26 15:41:38,275][105620] Updated weights for policy 1, policy_version 45218 (0.0008) [2023-12-26 15:41:38,334][105620] Updated weights for policy 1, policy_version 45228 (0.0008) [2023-12-26 15:41:38,516][105692] Updated weights for policy 0, policy_version 44948 (0.0007) [2023-12-26 15:41:38,568][105692] Updated weights for policy 0, policy_version 44958 (0.0010) [2023-12-26 15:41:38,623][105692] Updated weights for policy 0, policy_version 44968 (0.0010) [2023-12-26 15:41:39,056][105620] Updated weights for policy 1, policy_version 45238 (0.0008) [2023-12-26 15:41:39,107][105620] Updated weights for policy 1, policy_version 45248 (0.0009) [2023-12-26 15:41:39,160][105620] Updated weights for policy 1, policy_version 45258 (0.0007) [2023-12-26 15:41:39,345][105692] Updated weights for policy 0, policy_version 44978 (0.0010) [2023-12-26 15:41:39,414][105692] Updated weights for policy 0, policy_version 44988 (0.0007) [2023-12-26 15:41:39,480][105692] Updated weights for policy 0, policy_version 44998 (0.0009) [2023-12-26 15:41:39,539][105692] Updated weights for policy 0, policy_version 45008 (0.0009) [2023-12-26 15:41:39,951][105620] Updated weights for policy 1, policy_version 45268 (0.0009) [2023-12-26 15:41:40,007][105620] Updated weights for policy 1, policy_version 45278 (0.0008) [2023-12-26 15:41:40,058][105620] Updated weights for policy 1, policy_version 45288 (0.0008) [2023-12-26 15:41:40,295][105692] Updated weights for policy 0, policy_version 45018 (0.0008) [2023-12-26 15:41:40,356][105692] Updated weights for policy 0, policy_version 45028 (0.0008) [2023-12-26 15:41:40,410][105692] Updated weights for policy 0, policy_version 45038 (0.0008) [2023-12-26 15:41:40,849][105620] Updated weights for policy 1, policy_version 45298 (0.0009) [2023-12-26 15:41:40,905][105620] Updated weights for policy 1, policy_version 45308 (0.0008) [2023-12-26 15:41:40,954][105620] Updated weights for policy 1, policy_version 45319 (0.0009) [2023-12-26 15:41:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 23142400. Throughput: 0: 9884.6, 1: 9642.1. Samples: 23148888. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 15:41:41,062][104569] Avg episode reward: [(0, '8934.053'), (1, '8731.774')] [2023-12-26 15:41:41,101][105692] Updated weights for policy 0, policy_version 45048 (0.0009) [2023-12-26 15:41:41,173][105692] Updated weights for policy 0, policy_version 45058 (0.0009) [2023-12-26 15:41:41,240][105692] Updated weights for policy 0, policy_version 45068 (0.0009) [2023-12-26 15:41:41,747][105620] Updated weights for policy 1, policy_version 45329 (0.0008) [2023-12-26 15:41:41,811][105620] Updated weights for policy 1, policy_version 45339 (0.0006) [2023-12-26 15:41:41,873][105620] Updated weights for policy 1, policy_version 45349 (0.0006) [2023-12-26 15:41:41,941][105620] Updated weights for policy 1, policy_version 45359 (0.0008) [2023-12-26 15:41:42,008][105692] Updated weights for policy 0, policy_version 45078 (0.0009) [2023-12-26 15:41:42,059][105692] Updated weights for policy 0, policy_version 45088 (0.0009) [2023-12-26 15:41:42,119][105692] Updated weights for policy 0, policy_version 45098 (0.0009) [2023-12-26 15:41:42,594][105620] Updated weights for policy 1, policy_version 45369 (0.0008) [2023-12-26 15:41:42,651][105620] Updated weights for policy 1, policy_version 45379 (0.0009) [2023-12-26 15:41:42,703][105620] Updated weights for policy 1, policy_version 45389 (0.0010) [2023-12-26 15:41:42,889][105692] Updated weights for policy 0, policy_version 45108 (0.0010) [2023-12-26 15:41:42,939][105692] Updated weights for policy 0, policy_version 45118 (0.0009) [2023-12-26 15:41:42,991][105692] Updated weights for policy 0, policy_version 45128 (0.0009) [2023-12-26 15:41:43,495][105620] Updated weights for policy 1, policy_version 45399 (0.0010) [2023-12-26 15:41:43,547][105620] Updated weights for policy 1, policy_version 45410 (0.0006) [2023-12-26 15:41:43,593][105620] Updated weights for policy 1, policy_version 45420 (0.0005) [2023-12-26 15:41:43,688][105692] Updated weights for policy 0, policy_version 45138 (0.0008) [2023-12-26 15:41:43,754][105692] Updated weights for policy 0, policy_version 45148 (0.0005) [2023-12-26 15:41:43,809][105692] Updated weights for policy 0, policy_version 45158 (0.0005) [2023-12-26 15:41:43,875][105692] Updated weights for policy 0, policy_version 45168 (0.0009) [2023-12-26 15:41:44,220][105620] Updated weights for policy 1, policy_version 45430 (0.0007) [2023-12-26 15:41:44,282][105620] Updated weights for policy 1, policy_version 45440 (0.0006) [2023-12-26 15:41:44,352][105620] Updated weights for policy 1, policy_version 45450 (0.0006) [2023-12-26 15:41:44,494][105692] Updated weights for policy 0, policy_version 45178 (0.0005) [2023-12-26 15:41:44,547][105692] Updated weights for policy 0, policy_version 45188 (0.0006) [2023-12-26 15:41:44,598][105692] Updated weights for policy 0, policy_version 45198 (0.0006) [2023-12-26 15:41:44,986][105620] Updated weights for policy 1, policy_version 45460 (0.0009) [2023-12-26 15:41:45,039][105620] Updated weights for policy 1, policy_version 45470 (0.0009) [2023-12-26 15:41:45,097][105620] Updated weights for policy 1, policy_version 45480 (0.0008) [2023-12-26 15:41:45,302][105692] Updated weights for policy 0, policy_version 45208 (0.0009) [2023-12-26 15:41:45,363][105692] Updated weights for policy 0, policy_version 45218 (0.0010) [2023-12-26 15:41:45,414][105692] Updated weights for policy 0, policy_version 45228 (0.0008) [2023-12-26 15:41:45,809][105620] Updated weights for policy 1, policy_version 45490 (0.0008) [2023-12-26 15:41:45,882][105620] Updated weights for policy 1, policy_version 45500 (0.0006) [2023-12-26 15:41:45,939][105620] Updated weights for policy 1, policy_version 45510 (0.0009) [2023-12-26 15:41:45,999][105620] Updated weights for policy 1, policy_version 45520 (0.0008) [2023-12-26 15:41:46,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 23240704. Throughput: 0: 9843.3, 1: 9664.0. Samples: 23207364. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 15:41:46,063][104569] Avg episode reward: [(0, '9110.776'), (1, '8102.913')] [2023-12-26 15:41:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000045520_11657216.pth... [2023-12-26 15:41:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000044368_11362304.pth [2023-12-26 15:41:46,076][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_000045520_11657216.pth [2023-12-26 15:41:46,115][105692] Updated weights for policy 0, policy_version 45238 (0.0007) [2023-12-26 15:41:46,175][105692] Updated weights for policy 0, policy_version 45248 (0.0008) [2023-12-26 15:41:46,242][105692] Updated weights for policy 0, policy_version 45258 (0.0008) [2023-12-26 15:41:46,273][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000045264_11591680.pth... [2023-12-26 15:41:46,276][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000044048_11280384.pth [2023-12-26 15:41:46,276][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_000045264_11591680.pth [2023-12-26 15:41:46,592][105620] Updated weights for policy 1, policy_version 45530 (0.0010) [2023-12-26 15:41:46,650][105620] Updated weights for policy 1, policy_version 45540 (0.0007) [2023-12-26 15:41:46,709][105620] Updated weights for policy 1, policy_version 45550 (0.0006) [2023-12-26 15:41:46,975][105692] Updated weights for policy 0, policy_version 45268 (0.0010) [2023-12-26 15:41:47,025][105692] Updated weights for policy 0, policy_version 45278 (0.0010) [2023-12-26 15:41:47,083][105692] Updated weights for policy 0, policy_version 45288 (0.0010) [2023-12-26 15:41:47,303][105620] Updated weights for policy 1, policy_version 45560 (0.0006) [2023-12-26 15:41:47,350][105620] Updated weights for policy 1, policy_version 45570 (0.0010) [2023-12-26 15:41:47,399][105620] Updated weights for policy 1, policy_version 45580 (0.0010) [2023-12-26 15:41:47,789][105692] Updated weights for policy 0, policy_version 45298 (0.0009) [2023-12-26 15:41:47,848][105692] Updated weights for policy 0, policy_version 45308 (0.0005) [2023-12-26 15:41:47,908][105692] Updated weights for policy 0, policy_version 45318 (0.0005) [2023-12-26 15:41:47,970][105692] Updated weights for policy 0, policy_version 45328 (0.0005) [2023-12-26 15:41:48,015][105620] Updated weights for policy 1, policy_version 45590 (0.0007) [2023-12-26 15:41:48,069][105620] Updated weights for policy 1, policy_version 45600 (0.0005) [2023-12-26 15:41:48,139][105620] Updated weights for policy 1, policy_version 45610 (0.0008) [2023-12-26 15:41:48,608][105692] Updated weights for policy 0, policy_version 45338 (0.0011) [2023-12-26 15:41:48,667][105692] Updated weights for policy 0, policy_version 45348 (0.0007) [2023-12-26 15:41:48,718][105692] Updated weights for policy 0, policy_version 45358 (0.0010) [2023-12-26 15:41:48,868][105620] Updated weights for policy 1, policy_version 45620 (0.0010) [2023-12-26 15:41:48,930][105620] Updated weights for policy 1, policy_version 45630 (0.0011) [2023-12-26 15:41:48,985][105620] Updated weights for policy 1, policy_version 45640 (0.0010) [2023-12-26 15:41:49,450][105692] Updated weights for policy 0, policy_version 45368 (0.0010) [2023-12-26 15:41:49,509][105692] Updated weights for policy 0, policy_version 45378 (0.0010) [2023-12-26 15:41:49,560][105692] Updated weights for policy 0, policy_version 45388 (0.0009) [2023-12-26 15:41:49,683][105620] Updated weights for policy 1, policy_version 45650 (0.0010) [2023-12-26 15:41:49,751][105620] Updated weights for policy 1, policy_version 45660 (0.0007) [2023-12-26 15:41:49,806][105620] Updated weights for policy 1, policy_version 45670 (0.0007) [2023-12-26 15:41:49,873][105620] Updated weights for policy 1, policy_version 45680 (0.0007) [2023-12-26 15:41:50,273][105692] Updated weights for policy 0, policy_version 45398 (0.0010) [2023-12-26 15:41:50,339][105692] Updated weights for policy 0, policy_version 45408 (0.0007) [2023-12-26 15:41:50,398][105692] Updated weights for policy 0, policy_version 45418 (0.0009) [2023-12-26 15:41:50,511][105620] Updated weights for policy 1, policy_version 45690 (0.0008) [2023-12-26 15:41:50,567][105620] Updated weights for policy 1, policy_version 45700 (0.0008) [2023-12-26 15:41:50,634][105620] Updated weights for policy 1, policy_version 45710 (0.0009) [2023-12-26 15:41:51,020][105692] Updated weights for policy 0, policy_version 45428 (0.0009) [2023-12-26 15:41:51,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 23339008. Throughput: 0: 9917.9, 1: 9743.4. Samples: 23329308. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 15:41:51,063][104569] Avg episode reward: [(0, '9359.316'), (1, '8455.747')] [2023-12-26 15:41:51,090][105692] Updated weights for policy 0, policy_version 45438 (0.0011) [2023-12-26 15:41:51,156][105692] Updated weights for policy 0, policy_version 45448 (0.0011) [2023-12-26 15:41:51,207][105585] Saving new best policy, reward=9359.316! [2023-12-26 15:41:51,334][105620] Updated weights for policy 1, policy_version 45720 (0.0009) [2023-12-26 15:41:51,401][105620] Updated weights for policy 1, policy_version 45730 (0.0008) [2023-12-26 15:41:51,459][105620] Updated weights for policy 1, policy_version 45740 (0.0010) [2023-12-26 15:41:51,882][105692] Updated weights for policy 0, policy_version 45458 (0.0011) [2023-12-26 15:41:51,938][105692] Updated weights for policy 0, policy_version 45468 (0.0011) [2023-12-26 15:41:51,987][105692] Updated weights for policy 0, policy_version 45478 (0.0010) [2023-12-26 15:41:52,053][105692] Updated weights for policy 0, policy_version 45488 (0.0011) [2023-12-26 15:41:52,255][105620] Updated weights for policy 1, policy_version 45750 (0.0008) [2023-12-26 15:41:52,323][105620] Updated weights for policy 1, policy_version 45760 (0.0009) [2023-12-26 15:41:52,397][105620] Updated weights for policy 1, policy_version 45770 (0.0007) [2023-12-26 15:41:52,755][105692] Updated weights for policy 0, policy_version 45498 (0.0008) [2023-12-26 15:41:52,815][105692] Updated weights for policy 0, policy_version 45508 (0.0006) [2023-12-26 15:41:52,882][105692] Updated weights for policy 0, policy_version 45518 (0.0007) [2023-12-26 15:41:53,149][105620] Updated weights for policy 1, policy_version 45780 (0.0009) [2023-12-26 15:41:53,210][105620] Updated weights for policy 1, policy_version 45790 (0.0008) [2023-12-26 15:41:53,262][105620] Updated weights for policy 1, policy_version 45800 (0.0006) [2023-12-26 15:41:53,562][105692] Updated weights for policy 0, policy_version 45528 (0.0009) [2023-12-26 15:41:53,616][105692] Updated weights for policy 0, policy_version 45538 (0.0010) [2023-12-26 15:41:53,681][105692] Updated weights for policy 0, policy_version 45548 (0.0010) [2023-12-26 15:41:53,808][105620] Updated weights for policy 1, policy_version 45810 (0.0006) [2023-12-26 15:41:53,859][105620] Updated weights for policy 1, policy_version 45820 (0.0009) [2023-12-26 15:41:53,916][105620] Updated weights for policy 1, policy_version 45830 (0.0008) [2023-12-26 15:41:53,977][105620] Updated weights for policy 1, policy_version 45840 (0.0009) [2023-12-26 15:41:54,479][105692] Updated weights for policy 0, policy_version 45558 (0.0009) [2023-12-26 15:41:54,531][105692] Updated weights for policy 0, policy_version 45568 (0.0007) [2023-12-26 15:41:54,600][105692] Updated weights for policy 0, policy_version 45578 (0.0005) [2023-12-26 15:41:54,715][105620] Updated weights for policy 1, policy_version 45850 (0.0009) [2023-12-26 15:41:54,779][105620] Updated weights for policy 1, policy_version 45860 (0.0009) [2023-12-26 15:41:54,845][105620] Updated weights for policy 1, policy_version 45870 (0.0006) [2023-12-26 15:41:55,258][105692] Updated weights for policy 0, policy_version 45588 (0.0007) [2023-12-26 15:41:55,313][105692] Updated weights for policy 0, policy_version 45598 (0.0010) [2023-12-26 15:41:55,364][105692] Updated weights for policy 0, policy_version 45608 (0.0009) [2023-12-26 15:41:55,512][105620] Updated weights for policy 1, policy_version 45880 (0.0008) [2023-12-26 15:41:55,576][105620] Updated weights for policy 1, policy_version 45890 (0.0009) [2023-12-26 15:41:55,635][105620] Updated weights for policy 1, policy_version 45900 (0.0008) [2023-12-26 15:41:56,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 23437312. Throughput: 0: 10018.2, 1: 9836.5. Samples: 23448472. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 15:41:56,062][104569] Avg episode reward: [(0, '9359.296'), (1, '8817.643')] [2023-12-26 15:41:56,126][105692] Updated weights for policy 0, policy_version 45618 (0.0009) [2023-12-26 15:41:56,171][105692] Updated weights for policy 0, policy_version 45628 (0.0005) [2023-12-26 15:41:56,223][105692] Updated weights for policy 0, policy_version 45638 (0.0007) [2023-12-26 15:41:56,232][105620] Updated weights for policy 1, policy_version 45910 (0.0006) [2023-12-26 15:41:56,267][105692] Updated weights for policy 0, policy_version 45648 (0.0005) [2023-12-26 15:41:56,289][105620] Updated weights for policy 1, policy_version 45920 (0.0006) [2023-12-26 15:41:56,333][105620] Updated weights for policy 1, policy_version 45930 (0.0005) [2023-12-26 15:41:56,903][105692] Updated weights for policy 0, policy_version 45658 (0.0008) [2023-12-26 15:41:56,945][105620] Updated weights for policy 1, policy_version 45940 (0.0007) [2023-12-26 15:41:56,958][105692] Updated weights for policy 0, policy_version 45668 (0.0010) [2023-12-26 15:41:57,002][105620] Updated weights for policy 1, policy_version 45950 (0.0006) [2023-12-26 15:41:57,003][105692] Updated weights for policy 0, policy_version 45678 (0.0010) [2023-12-26 15:41:57,056][105620] Updated weights for policy 1, policy_version 45960 (0.0005) [2023-12-26 15:41:57,584][105620] Updated weights for policy 1, policy_version 45970 (0.0006) [2023-12-26 15:41:57,613][105692] Updated weights for policy 0, policy_version 45688 (0.0010) [2023-12-26 15:41:57,634][105620] Updated weights for policy 1, policy_version 45980 (0.0005) [2023-12-26 15:41:57,665][105692] Updated weights for policy 0, policy_version 45698 (0.0010) [2023-12-26 15:41:57,683][105620] Updated weights for policy 1, policy_version 45990 (0.0005) [2023-12-26 15:41:57,717][105692] Updated weights for policy 0, policy_version 45708 (0.0010) [2023-12-26 15:41:57,739][105620] Updated weights for policy 1, policy_version 46000 (0.0005) [2023-12-26 15:41:58,392][105620] Updated weights for policy 1, policy_version 46010 (0.0008) [2023-12-26 15:41:58,399][105692] Updated weights for policy 0, policy_version 45718 (0.0009) [2023-12-26 15:41:58,457][105620] Updated weights for policy 1, policy_version 46020 (0.0008) [2023-12-26 15:41:58,465][105692] Updated weights for policy 0, policy_version 45729 (0.0009) [2023-12-26 15:41:58,521][105620] Updated weights for policy 1, policy_version 46030 (0.0009) [2023-12-26 15:41:58,531][105692] Updated weights for policy 0, policy_version 45739 (0.0008) [2023-12-26 15:41:59,252][105620] Updated weights for policy 1, policy_version 46040 (0.0009) [2023-12-26 15:41:59,295][105692] Updated weights for policy 0, policy_version 45749 (0.0008) [2023-12-26 15:41:59,311][105620] Updated weights for policy 1, policy_version 46050 (0.0009) [2023-12-26 15:41:59,355][105692] Updated weights for policy 0, policy_version 45759 (0.0010) [2023-12-26 15:41:59,374][105620] Updated weights for policy 1, policy_version 46060 (0.0010) [2023-12-26 15:41:59,419][105692] Updated weights for policy 0, policy_version 45769 (0.0008) [2023-12-26 15:42:00,076][105620] Updated weights for policy 1, policy_version 46070 (0.0010) [2023-12-26 15:42:00,131][105620] Updated weights for policy 1, policy_version 46080 (0.0010) [2023-12-26 15:42:00,183][105620] Updated weights for policy 1, policy_version 46090 (0.0010) [2023-12-26 15:42:00,190][105692] Updated weights for policy 0, policy_version 45779 (0.0007) [2023-12-26 15:42:00,242][105692] Updated weights for policy 0, policy_version 45789 (0.0007) [2023-12-26 15:42:00,291][105692] Updated weights for policy 0, policy_version 45799 (0.0008) [2023-12-26 15:42:00,934][105620] Updated weights for policy 1, policy_version 46100 (0.0010) [2023-12-26 15:42:00,978][105692] Updated weights for policy 0, policy_version 45809 (0.0008) [2023-12-26 15:42:00,990][105620] Updated weights for policy 1, policy_version 46110 (0.0007) [2023-12-26 15:42:01,039][105692] Updated weights for policy 0, policy_version 45819 (0.0008) [2023-12-26 15:42:01,051][105620] Updated weights for policy 1, policy_version 46120 (0.0006) [2023-12-26 15:42:01,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 23535616. Throughput: 0: 10061.6, 1: 9960.7. Samples: 23513692. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 15:42:01,062][104569] Avg episode reward: [(0, '9359.200'), (1, '8557.749')] [2023-12-26 15:42:01,098][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000046128_11812864.pth... [2023-12-26 15:42:01,101][105692] Updated weights for policy 0, policy_version 45829 (0.0009) [2023-12-26 15:42:01,103][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000044944_11509760.pth [2023-12-26 15:42:01,162][105692] Updated weights for policy 0, policy_version 45839 (0.0008) [2023-12-26 15:42:01,165][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000045840_11739136.pth... [2023-12-26 15:42:01,168][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000044656_11436032.pth [2023-12-26 15:42:01,743][105620] Updated weights for policy 1, policy_version 46130 (0.0006) [2023-12-26 15:42:01,801][105620] Updated weights for policy 1, policy_version 46140 (0.0005) [2023-12-26 15:42:01,867][105620] Updated weights for policy 1, policy_version 46150 (0.0005) [2023-12-26 15:42:01,939][105620] Updated weights for policy 1, policy_version 46160 (0.0009) [2023-12-26 15:42:01,961][105692] Updated weights for policy 0, policy_version 45849 (0.0009) [2023-12-26 15:42:02,024][105692] Updated weights for policy 0, policy_version 45859 (0.0008) [2023-12-26 15:42:02,090][105692] Updated weights for policy 0, policy_version 45869 (0.0008) [2023-12-26 15:42:02,581][105620] Updated weights for policy 1, policy_version 46170 (0.0010) [2023-12-26 15:42:02,630][105620] Updated weights for policy 1, policy_version 46180 (0.0009) [2023-12-26 15:42:02,686][105620] Updated weights for policy 1, policy_version 46190 (0.0005) [2023-12-26 15:42:02,863][105692] Updated weights for policy 0, policy_version 45879 (0.0009) [2023-12-26 15:42:02,923][105692] Updated weights for policy 0, policy_version 45890 (0.0012) [2023-12-26 15:42:02,977][105692] Updated weights for policy 0, policy_version 45900 (0.0010) [2023-12-26 15:42:03,307][105620] Updated weights for policy 1, policy_version 46200 (0.0009) [2023-12-26 15:42:03,361][105620] Updated weights for policy 1, policy_version 46210 (0.0010) [2023-12-26 15:42:03,422][105620] Updated weights for policy 1, policy_version 46220 (0.0010) [2023-12-26 15:42:03,728][105692] Updated weights for policy 0, policy_version 45910 (0.0007) [2023-12-26 15:42:03,784][105692] Updated weights for policy 0, policy_version 45920 (0.0008) [2023-12-26 15:42:03,846][105692] Updated weights for policy 0, policy_version 45930 (0.0008) [2023-12-26 15:42:04,083][105620] Updated weights for policy 1, policy_version 46230 (0.0008) [2023-12-26 15:42:04,134][105620] Updated weights for policy 1, policy_version 46240 (0.0006) [2023-12-26 15:42:04,184][105620] Updated weights for policy 1, policy_version 46250 (0.0005) [2023-12-26 15:42:04,663][105692] Updated weights for policy 0, policy_version 45940 (0.0010) [2023-12-26 15:42:04,718][105692] Updated weights for policy 0, policy_version 45950 (0.0010) [2023-12-26 15:42:04,771][105692] Updated weights for policy 0, policy_version 45960 (0.0010) [2023-12-26 15:42:04,888][105620] Updated weights for policy 1, policy_version 46260 (0.0007) [2023-12-26 15:42:04,943][105620] Updated weights for policy 1, policy_version 46270 (0.0010) [2023-12-26 15:42:04,997][105620] Updated weights for policy 1, policy_version 46280 (0.0010) [2023-12-26 15:42:05,526][105692] Updated weights for policy 0, policy_version 45970 (0.0010) [2023-12-26 15:42:05,592][105692] Updated weights for policy 0, policy_version 45980 (0.0010) [2023-12-26 15:42:05,657][105692] Updated weights for policy 0, policy_version 45990 (0.0010) [2023-12-26 15:42:05,722][105692] Updated weights for policy 0, policy_version 46000 (0.0010) [2023-12-26 15:42:05,768][105620] Updated weights for policy 1, policy_version 46290 (0.0010) [2023-12-26 15:42:05,822][105620] Updated weights for policy 1, policy_version 46300 (0.0009) [2023-12-26 15:42:05,871][105620] Updated weights for policy 1, policy_version 46310 (0.0009) [2023-12-26 15:42:05,921][105620] Updated weights for policy 1, policy_version 46320 (0.0009) [2023-12-26 15:42:06,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 23642112. Throughput: 0: 9913.3, 1: 9932.2. Samples: 23628632. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 15:42:06,063][104569] Avg episode reward: [(0, '9359.084'), (1, '8468.816')] [2023-12-26 15:42:06,389][105692] Updated weights for policy 0, policy_version 46010 (0.0008) [2023-12-26 15:42:06,453][105692] Updated weights for policy 0, policy_version 46020 (0.0010) [2023-12-26 15:42:06,511][105692] Updated weights for policy 0, policy_version 46030 (0.0009) [2023-12-26 15:42:06,688][105620] Updated weights for policy 1, policy_version 46330 (0.0006) [2023-12-26 15:42:06,749][105620] Updated weights for policy 1, policy_version 46340 (0.0005) [2023-12-26 15:42:06,814][105620] Updated weights for policy 1, policy_version 46350 (0.0006) [2023-12-26 15:42:07,233][105692] Updated weights for policy 0, policy_version 46040 (0.0009) [2023-12-26 15:42:07,290][105692] Updated weights for policy 0, policy_version 46051 (0.0010) [2023-12-26 15:42:07,347][105692] Updated weights for policy 0, policy_version 46061 (0.0010) [2023-12-26 15:42:07,376][105620] Updated weights for policy 1, policy_version 46360 (0.0006) [2023-12-26 15:42:07,428][105620] Updated weights for policy 1, policy_version 46370 (0.0005) [2023-12-26 15:42:07,496][105620] Updated weights for policy 1, policy_version 46380 (0.0005) [2023-12-26 15:42:08,064][105620] Updated weights for policy 1, policy_version 46390 (0.0005) [2023-12-26 15:42:08,116][105620] Updated weights for policy 1, policy_version 46400 (0.0005) [2023-12-26 15:42:08,165][105620] Updated weights for policy 1, policy_version 46410 (0.0005) [2023-12-26 15:42:08,190][105692] Updated weights for policy 0, policy_version 46071 (0.0006) [2023-12-26 15:42:08,241][105692] Updated weights for policy 0, policy_version 46081 (0.0005) [2023-12-26 15:42:08,288][105692] Updated weights for policy 0, policy_version 46091 (0.0005) [2023-12-26 15:42:08,919][105620] Updated weights for policy 1, policy_version 46420 (0.0006) [2023-12-26 15:42:08,950][105692] Updated weights for policy 0, policy_version 46101 (0.0007) [2023-12-26 15:42:08,973][105620] Updated weights for policy 1, policy_version 46430 (0.0007) [2023-12-26 15:42:09,008][105692] Updated weights for policy 0, policy_version 46111 (0.0006) [2023-12-26 15:42:09,025][105620] Updated weights for policy 1, policy_version 46440 (0.0007) [2023-12-26 15:42:09,068][105692] Updated weights for policy 0, policy_version 46121 (0.0006) [2023-12-26 15:42:09,818][105620] Updated weights for policy 1, policy_version 46450 (0.0009) [2023-12-26 15:42:09,845][105692] Updated weights for policy 0, policy_version 46131 (0.0008) [2023-12-26 15:42:09,882][105620] Updated weights for policy 1, policy_version 46460 (0.0008) [2023-12-26 15:42:09,903][105692] Updated weights for policy 0, policy_version 46141 (0.0008) [2023-12-26 15:42:09,947][105620] Updated weights for policy 1, policy_version 46470 (0.0008) [2023-12-26 15:42:09,969][105692] Updated weights for policy 0, policy_version 46151 (0.0008) [2023-12-26 15:42:10,005][105620] Updated weights for policy 1, policy_version 46480 (0.0007) [2023-12-26 15:42:10,576][105692] Updated weights for policy 0, policy_version 46161 (0.0007) [2023-12-26 15:42:10,631][105692] Updated weights for policy 0, policy_version 46171 (0.0009) [2023-12-26 15:42:10,688][105692] Updated weights for policy 0, policy_version 46181 (0.0009) [2023-12-26 15:42:10,750][105692] Updated weights for policy 0, policy_version 46191 (0.0008) [2023-12-26 15:42:10,812][105620] Updated weights for policy 1, policy_version 46490 (0.0009) [2023-12-26 15:42:10,876][105620] Updated weights for policy 1, policy_version 46500 (0.0010) [2023-12-26 15:42:10,928][105620] Updated weights for policy 1, policy_version 46510 (0.0007) [2023-12-26 15:42:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 23740416. Throughput: 0: 9861.5, 1: 9994.1. Samples: 23744328. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 15:42:11,062][104569] Avg episode reward: [(0, '9358.820'), (1, '7736.131')] [2023-12-26 15:42:11,508][105692] Updated weights for policy 0, policy_version 46201 (0.0008) [2023-12-26 15:42:11,564][105692] Updated weights for policy 0, policy_version 46211 (0.0008) [2023-12-26 15:42:11,625][105692] Updated weights for policy 0, policy_version 46221 (0.0008) [2023-12-26 15:42:11,701][105620] Updated weights for policy 1, policy_version 46520 (0.0008) [2023-12-26 15:42:11,773][105620] Updated weights for policy 1, policy_version 46530 (0.0007) [2023-12-26 15:42:11,846][105620] Updated weights for policy 1, policy_version 46540 (0.0005) [2023-12-26 15:42:12,287][105692] Updated weights for policy 0, policy_version 46231 (0.0007) [2023-12-26 15:42:12,360][105692] Updated weights for policy 0, policy_version 46241 (0.0006) [2023-12-26 15:42:12,421][105692] Updated weights for policy 0, policy_version 46251 (0.0008) [2023-12-26 15:42:12,554][105620] Updated weights for policy 1, policy_version 46550 (0.0009) [2023-12-26 15:42:12,622][105620] Updated weights for policy 1, policy_version 46560 (0.0005) [2023-12-26 15:42:12,691][105620] Updated weights for policy 1, policy_version 46570 (0.0005) [2023-12-26 15:42:13,141][105692] Updated weights for policy 0, policy_version 46261 (0.0008) [2023-12-26 15:42:13,206][105692] Updated weights for policy 0, policy_version 46271 (0.0009) [2023-12-26 15:42:13,269][105692] Updated weights for policy 0, policy_version 46281 (0.0008) [2023-12-26 15:42:13,334][105620] Updated weights for policy 1, policy_version 46580 (0.0006) [2023-12-26 15:42:13,396][105620] Updated weights for policy 1, policy_version 46590 (0.0005) [2023-12-26 15:42:13,454][105620] Updated weights for policy 1, policy_version 46600 (0.0007) [2023-12-26 15:42:13,875][105692] Updated weights for policy 0, policy_version 46291 (0.0007) [2023-12-26 15:42:13,932][105692] Updated weights for policy 0, policy_version 46301 (0.0005) [2023-12-26 15:42:13,981][105692] Updated weights for policy 0, policy_version 46311 (0.0005) [2023-12-26 15:42:14,129][105620] Updated weights for policy 1, policy_version 46610 (0.0010) [2023-12-26 15:42:14,190][105620] Updated weights for policy 1, policy_version 46620 (0.0010) [2023-12-26 15:42:14,245][105620] Updated weights for policy 1, policy_version 46630 (0.0008) [2023-12-26 15:42:14,290][105620] Updated weights for policy 1, policy_version 46640 (0.0005) [2023-12-26 15:42:14,581][105692] Updated weights for policy 0, policy_version 46321 (0.0008) [2023-12-26 15:42:14,633][105692] Updated weights for policy 0, policy_version 46331 (0.0005) [2023-12-26 15:42:14,687][105692] Updated weights for policy 0, policy_version 46341 (0.0007) [2023-12-26 15:42:14,755][105692] Updated weights for policy 0, policy_version 46351 (0.0010) [2023-12-26 15:42:14,931][105620] Updated weights for policy 1, policy_version 46650 (0.0006) [2023-12-26 15:42:15,002][105620] Updated weights for policy 1, policy_version 46660 (0.0006) [2023-12-26 15:42:15,060][105620] Updated weights for policy 1, policy_version 46670 (0.0008) [2023-12-26 15:42:15,397][105692] Updated weights for policy 0, policy_version 46361 (0.0011) [2023-12-26 15:42:15,464][105692] Updated weights for policy 0, policy_version 46371 (0.0011) [2023-12-26 15:42:15,519][105692] Updated weights for policy 0, policy_version 46381 (0.0010) [2023-12-26 15:42:15,782][105620] Updated weights for policy 1, policy_version 46680 (0.0010) [2023-12-26 15:42:15,840][105620] Updated weights for policy 1, policy_version 46690 (0.0008) [2023-12-26 15:42:15,893][105620] Updated weights for policy 1, policy_version 46700 (0.0009) [2023-12-26 15:42:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19934.0, 300 sec: 19633.0). Total num frames: 23838720. Throughput: 0: 9815.6, 1: 9991.7. Samples: 23803100. Policy #0 lag: (min: 14.0, avg: 21.9, max: 46.0) [2023-12-26 15:42:16,062][104569] Avg episode reward: [(0, '9358.250'), (1, '6812.410')] [2023-12-26 15:42:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000046384_11878400.pth... [2023-12-26 15:42:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000046704_11960320.pth... [2023-12-26 15:42:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000045520_11657216.pth [2023-12-26 15:42:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000045264_11591680.pth [2023-12-26 15:42:16,146][105692] Updated weights for policy 0, policy_version 46391 (0.0008) [2023-12-26 15:42:16,201][105692] Updated weights for policy 0, policy_version 46401 (0.0007) [2023-12-26 15:42:16,263][105692] Updated weights for policy 0, policy_version 46411 (0.0007) [2023-12-26 15:42:16,691][105620] Updated weights for policy 1, policy_version 46710 (0.0007) [2023-12-26 15:42:16,753][105620] Updated weights for policy 1, policy_version 46720 (0.0010) [2023-12-26 15:42:16,814][105620] Updated weights for policy 1, policy_version 46730 (0.0009) [2023-12-26 15:42:16,818][105692] Updated weights for policy 0, policy_version 46421 (0.0006) [2023-12-26 15:42:16,866][105692] Updated weights for policy 0, policy_version 46431 (0.0005) [2023-12-26 15:42:16,914][105692] Updated weights for policy 0, policy_version 46441 (0.0005) [2023-12-26 15:42:17,526][105620] Updated weights for policy 1, policy_version 46740 (0.0007) [2023-12-26 15:42:17,575][105620] Updated weights for policy 1, policy_version 46750 (0.0007) [2023-12-26 15:42:17,580][105692] Updated weights for policy 0, policy_version 46451 (0.0007) [2023-12-26 15:42:17,636][105620] Updated weights for policy 1, policy_version 46760 (0.0007) [2023-12-26 15:42:17,641][105692] Updated weights for policy 0, policy_version 46461 (0.0010) [2023-12-26 15:42:17,703][105692] Updated weights for policy 0, policy_version 46471 (0.0010) [2023-12-26 15:42:18,379][105620] Updated weights for policy 1, policy_version 46770 (0.0005) [2023-12-26 15:42:18,429][105620] Updated weights for policy 1, policy_version 46780 (0.0005) [2023-12-26 15:42:18,472][105692] Updated weights for policy 0, policy_version 46481 (0.0010) [2023-12-26 15:42:18,484][105620] Updated weights for policy 1, policy_version 46790 (0.0007) [2023-12-26 15:42:18,530][105692] Updated weights for policy 0, policy_version 46491 (0.0008) [2023-12-26 15:42:18,537][105620] Updated weights for policy 1, policy_version 46800 (0.0010) [2023-12-26 15:42:18,590][105692] Updated weights for policy 0, policy_version 46501 (0.0009) [2023-12-26 15:42:18,655][105692] Updated weights for policy 0, policy_version 46511 (0.0010) [2023-12-26 15:42:19,281][105620] Updated weights for policy 1, policy_version 46810 (0.0009) [2023-12-26 15:42:19,353][105620] Updated weights for policy 1, policy_version 46820 (0.0008) [2023-12-26 15:42:19,386][105692] Updated weights for policy 0, policy_version 46521 (0.0010) [2023-12-26 15:42:19,416][105620] Updated weights for policy 1, policy_version 46830 (0.0007) [2023-12-26 15:42:19,452][105692] Updated weights for policy 0, policy_version 46531 (0.0011) [2023-12-26 15:42:19,519][105692] Updated weights for policy 0, policy_version 46541 (0.0011) [2023-12-26 15:42:20,179][105620] Updated weights for policy 1, policy_version 46840 (0.0006) [2023-12-26 15:42:20,232][105692] Updated weights for policy 0, policy_version 46551 (0.0007) [2023-12-26 15:42:20,245][105620] Updated weights for policy 1, policy_version 46850 (0.0006) [2023-12-26 15:42:20,296][105692] Updated weights for policy 0, policy_version 46561 (0.0007) [2023-12-26 15:42:20,309][105620] Updated weights for policy 1, policy_version 46860 (0.0007) [2023-12-26 15:42:20,360][105692] Updated weights for policy 0, policy_version 46571 (0.0007) [2023-12-26 15:42:20,901][105620] Updated weights for policy 1, policy_version 46870 (0.0010) [2023-12-26 15:42:20,955][105620] Updated weights for policy 1, policy_version 46880 (0.0007) [2023-12-26 15:42:21,002][105620] Updated weights for policy 1, policy_version 46890 (0.0010) [2023-12-26 15:42:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 23937024. Throughput: 0: 9828.9, 1: 9966.0. Samples: 23923732. Policy #0 lag: (min: 14.0, avg: 21.9, max: 46.0) [2023-12-26 15:42:21,063][104569] Avg episode reward: [(0, '9357.831'), (1, '7782.632')] [2023-12-26 15:42:21,118][105692] Updated weights for policy 0, policy_version 46581 (0.0007) [2023-12-26 15:42:21,172][105692] Updated weights for policy 0, policy_version 46591 (0.0008) [2023-12-26 15:42:21,235][105692] Updated weights for policy 0, policy_version 46601 (0.0007) [2023-12-26 15:42:21,740][105620] Updated weights for policy 1, policy_version 46900 (0.0008) [2023-12-26 15:42:21,806][105620] Updated weights for policy 1, policy_version 46910 (0.0008) [2023-12-26 15:42:21,875][105620] Updated weights for policy 1, policy_version 46920 (0.0008) [2023-12-26 15:42:22,003][105692] Updated weights for policy 0, policy_version 46611 (0.0007) [2023-12-26 15:42:22,058][105692] Updated weights for policy 0, policy_version 46621 (0.0008) [2023-12-26 15:42:22,119][105692] Updated weights for policy 0, policy_version 46631 (0.0008) [2023-12-26 15:42:22,630][105620] Updated weights for policy 1, policy_version 46930 (0.0009) [2023-12-26 15:42:22,689][105620] Updated weights for policy 1, policy_version 46940 (0.0009) [2023-12-26 15:42:22,752][105620] Updated weights for policy 1, policy_version 46950 (0.0009) [2023-12-26 15:42:22,814][105620] Updated weights for policy 1, policy_version 46960 (0.0009) [2023-12-26 15:42:22,838][105692] Updated weights for policy 0, policy_version 46641 (0.0007) [2023-12-26 15:42:22,885][105692] Updated weights for policy 0, policy_version 46651 (0.0009) [2023-12-26 15:42:22,939][105692] Updated weights for policy 0, policy_version 46661 (0.0009) [2023-12-26 15:42:22,998][105692] Updated weights for policy 0, policy_version 46671 (0.0009) [2023-12-26 15:42:23,555][105620] Updated weights for policy 1, policy_version 46970 (0.0009) [2023-12-26 15:42:23,606][105620] Updated weights for policy 1, policy_version 46980 (0.0009) [2023-12-26 15:42:23,658][105620] Updated weights for policy 1, policy_version 46990 (0.0009) [2023-12-26 15:42:23,782][105692] Updated weights for policy 0, policy_version 46681 (0.0008) [2023-12-26 15:42:23,843][105692] Updated weights for policy 0, policy_version 46691 (0.0008) [2023-12-26 15:42:23,906][105692] Updated weights for policy 0, policy_version 46701 (0.0007) [2023-12-26 15:42:24,462][105692] Updated weights for policy 0, policy_version 46711 (0.0008) [2023-12-26 15:42:24,516][105692] Updated weights for policy 0, policy_version 46721 (0.0009) [2023-12-26 15:42:24,527][105620] Updated weights for policy 1, policy_version 47000 (0.0009) [2023-12-26 15:42:24,569][105692] Updated weights for policy 0, policy_version 46731 (0.0009) [2023-12-26 15:42:24,575][105620] Updated weights for policy 1, policy_version 47010 (0.0006) [2023-12-26 15:42:24,624][105620] Updated weights for policy 1, policy_version 47020 (0.0008) [2023-12-26 15:42:25,271][105692] Updated weights for policy 0, policy_version 46741 (0.0008) [2023-12-26 15:42:25,334][105692] Updated weights for policy 0, policy_version 46751 (0.0006) [2023-12-26 15:42:25,395][105692] Updated weights for policy 0, policy_version 46761 (0.0005) [2023-12-26 15:42:25,409][105620] Updated weights for policy 1, policy_version 47030 (0.0009) [2023-12-26 15:42:25,463][105620] Updated weights for policy 1, policy_version 47041 (0.0011) [2023-12-26 15:42:25,505][105620] Updated weights for policy 1, policy_version 47051 (0.0006) [2023-12-26 15:42:25,933][105692] Updated weights for policy 0, policy_version 46771 (0.0006) [2023-12-26 15:42:25,999][105692] Updated weights for policy 0, policy_version 46781 (0.0009) [2023-12-26 15:42:26,057][105692] Updated weights for policy 0, policy_version 46791 (0.0009) [2023-12-26 15:42:26,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19660.7, 300 sec: 19605.3). Total num frames: 24027136. Throughput: 0: 9854.1, 1: 9948.7. Samples: 24040020. Policy #0 lag: (min: 14.0, avg: 21.9, max: 46.0) [2023-12-26 15:42:26,063][104569] Avg episode reward: [(0, '9181.118'), (1, '8988.365')] [2023-12-26 15:42:26,154][105620] Updated weights for policy 1, policy_version 47061 (0.0007) [2023-12-26 15:42:26,209][105620] Updated weights for policy 1, policy_version 47071 (0.0009) [2023-12-26 15:42:26,265][105620] Updated weights for policy 1, policy_version 47081 (0.0008) [2023-12-26 15:42:26,688][105692] Updated weights for policy 0, policy_version 46801 (0.0007) [2023-12-26 15:42:26,740][105692] Updated weights for policy 0, policy_version 46811 (0.0007) [2023-12-26 15:42:26,799][105692] Updated weights for policy 0, policy_version 46821 (0.0007) [2023-12-26 15:42:26,853][105692] Updated weights for policy 0, policy_version 46831 (0.0008) [2023-12-26 15:42:27,028][105620] Updated weights for policy 1, policy_version 47091 (0.0006) [2023-12-26 15:42:27,080][105620] Updated weights for policy 1, policy_version 47101 (0.0007) [2023-12-26 15:42:27,127][105620] Updated weights for policy 1, policy_version 47111 (0.0008) [2023-12-26 15:42:27,501][105692] Updated weights for policy 0, policy_version 46841 (0.0007) [2023-12-26 15:42:27,556][105692] Updated weights for policy 0, policy_version 46851 (0.0006) [2023-12-26 15:42:27,613][105692] Updated weights for policy 0, policy_version 46861 (0.0010) [2023-12-26 15:42:27,709][105620] Updated weights for policy 1, policy_version 47121 (0.0008) [2023-12-26 15:42:27,766][105620] Updated weights for policy 1, policy_version 47131 (0.0005) [2023-12-26 15:42:27,819][105620] Updated weights for policy 1, policy_version 47141 (0.0005) [2023-12-26 15:42:27,887][105620] Updated weights for policy 1, policy_version 47151 (0.0005) [2023-12-26 15:42:28,317][105692] Updated weights for policy 0, policy_version 46871 (0.0010) [2023-12-26 15:42:28,382][105692] Updated weights for policy 0, policy_version 46881 (0.0011) [2023-12-26 15:42:28,389][105620] Updated weights for policy 1, policy_version 47161 (0.0007) [2023-12-26 15:42:28,434][105692] Updated weights for policy 0, policy_version 46891 (0.0010) [2023-12-26 15:42:28,444][105620] Updated weights for policy 1, policy_version 47171 (0.0007) [2023-12-26 15:42:28,501][105620] Updated weights for policy 1, policy_version 47181 (0.0007) [2023-12-26 15:42:29,183][105692] Updated weights for policy 0, policy_version 46901 (0.0010) [2023-12-26 15:42:29,193][105620] Updated weights for policy 1, policy_version 47191 (0.0006) [2023-12-26 15:42:29,243][105692] Updated weights for policy 0, policy_version 46911 (0.0010) [2023-12-26 15:42:29,249][105620] Updated weights for policy 1, policy_version 47201 (0.0007) [2023-12-26 15:42:29,302][105692] Updated weights for policy 0, policy_version 46921 (0.0010) [2023-12-26 15:42:29,310][105620] Updated weights for policy 1, policy_version 47211 (0.0010) [2023-12-26 15:42:30,007][105692] Updated weights for policy 0, policy_version 46931 (0.0010) [2023-12-26 15:42:30,062][105692] Updated weights for policy 0, policy_version 46941 (0.0010) [2023-12-26 15:42:30,095][105620] Updated weights for policy 1, policy_version 47221 (0.0008) [2023-12-26 15:42:30,113][105692] Updated weights for policy 0, policy_version 46951 (0.0010) [2023-12-26 15:42:30,151][105620] Updated weights for policy 1, policy_version 47231 (0.0005) [2023-12-26 15:42:30,202][105620] Updated weights for policy 1, policy_version 47241 (0.0008) [2023-12-26 15:42:30,864][105692] Updated weights for policy 0, policy_version 46961 (0.0010) [2023-12-26 15:42:30,901][105620] Updated weights for policy 1, policy_version 47251 (0.0008) [2023-12-26 15:42:30,922][105692] Updated weights for policy 0, policy_version 46971 (0.0011) [2023-12-26 15:42:30,953][105620] Updated weights for policy 1, policy_version 47261 (0.0007) [2023-12-26 15:42:30,973][105692] Updated weights for policy 0, policy_version 46981 (0.0010) [2023-12-26 15:42:31,003][105620] Updated weights for policy 1, policy_version 47271 (0.0010) [2023-12-26 15:42:31,023][105692] Updated weights for policy 0, policy_version 46991 (0.0010) [2023-12-26 15:42:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 24141824. Throughput: 0: 9904.9, 1: 10017.7. Samples: 24103872. Policy #0 lag: (min: 14.0, avg: 21.9, max: 46.0) [2023-12-26 15:42:31,062][104569] Avg episode reward: [(0, '9094.211'), (1, '9171.696')] [2023-12-26 15:42:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000046992_12034048.pth... [2023-12-26 15:42:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000047280_12107776.pth... [2023-12-26 15:42:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000045840_11739136.pth [2023-12-26 15:42:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000046128_11812864.pth [2023-12-26 15:42:31,777][105692] Updated weights for policy 0, policy_version 47001 (0.0010) [2023-12-26 15:42:31,805][105620] Updated weights for policy 1, policy_version 47281 (0.0009) [2023-12-26 15:42:31,825][105692] Updated weights for policy 0, policy_version 47011 (0.0010) [2023-12-26 15:42:31,856][105620] Updated weights for policy 1, policy_version 47291 (0.0006) [2023-12-26 15:42:31,884][105692] Updated weights for policy 0, policy_version 47021 (0.0010) [2023-12-26 15:42:31,912][105620] Updated weights for policy 1, policy_version 47301 (0.0007) [2023-12-26 15:42:31,961][105620] Updated weights for policy 1, policy_version 47311 (0.0008) [2023-12-26 15:42:32,603][105692] Updated weights for policy 0, policy_version 47031 (0.0009) [2023-12-26 15:42:32,659][105692] Updated weights for policy 0, policy_version 47041 (0.0007) [2023-12-26 15:42:32,706][105692] Updated weights for policy 0, policy_version 47051 (0.0005) [2023-12-26 15:42:32,763][105620] Updated weights for policy 1, policy_version 47321 (0.0009) [2023-12-26 15:42:32,829][105620] Updated weights for policy 1, policy_version 47331 (0.0008) [2023-12-26 15:42:32,887][105620] Updated weights for policy 1, policy_version 47341 (0.0006) [2023-12-26 15:42:33,374][105692] Updated weights for policy 0, policy_version 47061 (0.0006) [2023-12-26 15:42:33,429][105692] Updated weights for policy 0, policy_version 47071 (0.0009) [2023-12-26 15:42:33,482][105692] Updated weights for policy 0, policy_version 47081 (0.0008) [2023-12-26 15:42:33,578][105620] Updated weights for policy 1, policy_version 47351 (0.0009) [2023-12-26 15:42:33,638][105620] Updated weights for policy 1, policy_version 47361 (0.0010) [2023-12-26 15:42:33,696][105620] Updated weights for policy 1, policy_version 47371 (0.0010) [2023-12-26 15:42:34,170][105692] Updated weights for policy 0, policy_version 47091 (0.0007) [2023-12-26 15:42:34,227][105692] Updated weights for policy 0, policy_version 47101 (0.0008) [2023-12-26 15:42:34,276][105692] Updated weights for policy 0, policy_version 47111 (0.0008) [2023-12-26 15:42:34,445][105620] Updated weights for policy 1, policy_version 47381 (0.0008) [2023-12-26 15:42:34,512][105620] Updated weights for policy 1, policy_version 47391 (0.0008) [2023-12-26 15:42:34,575][105620] Updated weights for policy 1, policy_version 47401 (0.0010) [2023-12-26 15:42:35,023][105692] Updated weights for policy 0, policy_version 47121 (0.0008) [2023-12-26 15:42:35,075][105692] Updated weights for policy 0, policy_version 47131 (0.0010) [2023-12-26 15:42:35,128][105692] Updated weights for policy 0, policy_version 47141 (0.0011) [2023-12-26 15:42:35,186][105692] Updated weights for policy 0, policy_version 47151 (0.0010) [2023-12-26 15:42:35,287][105620] Updated weights for policy 1, policy_version 47411 (0.0010) [2023-12-26 15:42:35,345][105620] Updated weights for policy 1, policy_version 47421 (0.0009) [2023-12-26 15:42:35,398][105620] Updated weights for policy 1, policy_version 47432 (0.0010) [2023-12-26 15:42:35,908][105692] Updated weights for policy 0, policy_version 47161 (0.0008) [2023-12-26 15:42:35,953][105692] Updated weights for policy 0, policy_version 47171 (0.0008) [2023-12-26 15:42:36,002][105692] Updated weights for policy 0, policy_version 47181 (0.0008) [2023-12-26 15:42:36,062][104569] Fps is (10 sec: 20480.7, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 24231936. Throughput: 0: 9889.9, 1: 9875.5. Samples: 24218748. Policy #0 lag: (min: 14.0, avg: 21.9, max: 46.0) [2023-12-26 15:42:36,062][104569] Avg episode reward: [(0, '9183.261'), (1, '9261.993')] [2023-12-26 15:42:36,187][105620] Updated weights for policy 1, policy_version 47443 (0.0010) [2023-12-26 15:42:36,247][105620] Updated weights for policy 1, policy_version 47453 (0.0011) [2023-12-26 15:42:36,302][105620] Updated weights for policy 1, policy_version 47463 (0.0011) [2023-12-26 15:42:36,846][105692] Updated weights for policy 0, policy_version 47191 (0.0009) [2023-12-26 15:42:36,902][105692] Updated weights for policy 0, policy_version 47201 (0.0010) [2023-12-26 15:42:36,959][105692] Updated weights for policy 0, policy_version 47211 (0.0012) [2023-12-26 15:42:37,004][105620] Updated weights for policy 1, policy_version 47473 (0.0010) [2023-12-26 15:42:37,053][105620] Updated weights for policy 1, policy_version 47483 (0.0005) [2023-12-26 15:42:37,104][105620] Updated weights for policy 1, policy_version 47493 (0.0005) [2023-12-26 15:42:37,152][105620] Updated weights for policy 1, policy_version 47503 (0.0010) [2023-12-26 15:42:37,729][105692] Updated weights for policy 0, policy_version 47222 (0.0009) [2023-12-26 15:42:37,794][105692] Updated weights for policy 0, policy_version 47232 (0.0009) [2023-12-26 15:42:37,853][105692] Updated weights for policy 0, policy_version 47242 (0.0009) [2023-12-26 15:42:37,891][105620] Updated weights for policy 1, policy_version 47513 (0.0008) [2023-12-26 15:42:37,955][105620] Updated weights for policy 1, policy_version 47523 (0.0006) [2023-12-26 15:42:38,013][105620] Updated weights for policy 1, policy_version 47533 (0.0005) [2023-12-26 15:42:38,653][105620] Updated weights for policy 1, policy_version 47543 (0.0009) [2023-12-26 15:42:38,671][105692] Updated weights for policy 0, policy_version 47252 (0.0007) [2023-12-26 15:42:38,712][105620] Updated weights for policy 1, policy_version 47553 (0.0010) [2023-12-26 15:42:38,714][105692] Updated weights for policy 0, policy_version 47262 (0.0007) [2023-12-26 15:42:38,762][105692] Updated weights for policy 0, policy_version 47272 (0.0007) [2023-12-26 15:42:38,771][105620] Updated weights for policy 1, policy_version 47563 (0.0010) [2023-12-26 15:42:39,534][105692] Updated weights for policy 0, policy_version 47282 (0.0007) [2023-12-26 15:42:39,580][105620] Updated weights for policy 1, policy_version 47573 (0.0009) [2023-12-26 15:42:39,593][105692] Updated weights for policy 0, policy_version 47292 (0.0007) [2023-12-26 15:42:39,644][105620] Updated weights for policy 1, policy_version 47583 (0.0008) [2023-12-26 15:42:39,648][105692] Updated weights for policy 0, policy_version 47302 (0.0007) [2023-12-26 15:42:39,706][105692] Updated weights for policy 0, policy_version 47312 (0.0007) [2023-12-26 15:42:39,707][105620] Updated weights for policy 1, policy_version 47593 (0.0009) [2023-12-26 15:42:40,449][105692] Updated weights for policy 0, policy_version 47322 (0.0008) [2023-12-26 15:42:40,479][105620] Updated weights for policy 1, policy_version 47603 (0.0009) [2023-12-26 15:42:40,508][105692] Updated weights for policy 0, policy_version 47332 (0.0008) [2023-12-26 15:42:40,538][105620] Updated weights for policy 1, policy_version 47613 (0.0008) [2023-12-26 15:42:40,570][105692] Updated weights for policy 0, policy_version 47342 (0.0007) [2023-12-26 15:42:40,602][105620] Updated weights for policy 1, policy_version 47623 (0.0008) [2023-12-26 15:42:41,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 24322048. Throughput: 0: 9803.9, 1: 9803.7. Samples: 24330816. Policy #0 lag: (min: 14.0, avg: 21.9, max: 46.0) [2023-12-26 15:42:41,063][104569] Avg episode reward: [(0, '9009.819'), (1, '9084.626')] [2023-12-26 15:42:41,331][105692] Updated weights for policy 0, policy_version 47352 (0.0009) [2023-12-26 15:42:41,375][105620] Updated weights for policy 1, policy_version 47633 (0.0009) [2023-12-26 15:42:41,402][105692] Updated weights for policy 0, policy_version 47362 (0.0009) [2023-12-26 15:42:41,436][105620] Updated weights for policy 1, policy_version 47643 (0.0007) [2023-12-26 15:42:41,454][105692] Updated weights for policy 0, policy_version 47372 (0.0009) [2023-12-26 15:42:41,506][105620] Updated weights for policy 1, policy_version 47653 (0.0005) [2023-12-26 15:42:41,568][105620] Updated weights for policy 1, policy_version 47663 (0.0009) [2023-12-26 15:42:42,245][105620] Updated weights for policy 1, policy_version 47673 (0.0008) [2023-12-26 15:42:42,290][105692] Updated weights for policy 0, policy_version 47382 (0.0008) [2023-12-26 15:42:42,309][105620] Updated weights for policy 1, policy_version 47683 (0.0008) [2023-12-26 15:42:42,356][105692] Updated weights for policy 0, policy_version 47392 (0.0008) [2023-12-26 15:42:42,372][105620] Updated weights for policy 1, policy_version 47693 (0.0008) [2023-12-26 15:42:42,417][105692] Updated weights for policy 0, policy_version 47402 (0.0010) [2023-12-26 15:42:43,009][105620] Updated weights for policy 1, policy_version 47703 (0.0005) [2023-12-26 15:42:43,057][105620] Updated weights for policy 1, policy_version 47713 (0.0005) [2023-12-26 15:42:43,112][105620] Updated weights for policy 1, policy_version 47723 (0.0005) [2023-12-26 15:42:43,258][105692] Updated weights for policy 0, policy_version 47412 (0.0010) [2023-12-26 15:42:43,310][105692] Updated weights for policy 0, policy_version 47422 (0.0009) [2023-12-26 15:42:43,363][105692] Updated weights for policy 0, policy_version 47432 (0.0009) [2023-12-26 15:42:43,812][105620] Updated weights for policy 1, policy_version 47733 (0.0007) [2023-12-26 15:42:43,865][105620] Updated weights for policy 1, policy_version 47743 (0.0009) [2023-12-26 15:42:43,912][105620] Updated weights for policy 1, policy_version 47753 (0.0009) [2023-12-26 15:42:44,099][105692] Updated weights for policy 0, policy_version 47442 (0.0008) [2023-12-26 15:42:44,146][105692] Updated weights for policy 0, policy_version 47452 (0.0005) [2023-12-26 15:42:44,199][105692] Updated weights for policy 0, policy_version 47462 (0.0005) [2023-12-26 15:42:44,250][105692] Updated weights for policy 0, policy_version 47472 (0.0008) [2023-12-26 15:42:44,750][105620] Updated weights for policy 1, policy_version 47763 (0.0008) [2023-12-26 15:42:44,808][105620] Updated weights for policy 1, policy_version 47773 (0.0008) [2023-12-26 15:42:44,866][105620] Updated weights for policy 1, policy_version 47783 (0.0007) [2023-12-26 15:42:44,887][105692] Updated weights for policy 0, policy_version 47482 (0.0011) [2023-12-26 15:42:44,946][105692] Updated weights for policy 0, policy_version 47492 (0.0010) [2023-12-26 15:42:45,009][105692] Updated weights for policy 0, policy_version 47502 (0.0010) [2023-12-26 15:42:45,672][105620] Updated weights for policy 1, policy_version 47793 (0.0006) [2023-12-26 15:42:45,729][105620] Updated weights for policy 1, policy_version 47803 (0.0010) [2023-12-26 15:42:45,753][105692] Updated weights for policy 0, policy_version 47512 (0.0011) [2023-12-26 15:42:45,774][105620] Updated weights for policy 1, policy_version 47813 (0.0010) [2023-12-26 15:42:45,815][105692] Updated weights for policy 0, policy_version 47522 (0.0010) [2023-12-26 15:42:45,828][105620] Updated weights for policy 1, policy_version 47823 (0.0010) [2023-12-26 15:42:45,881][105692] Updated weights for policy 0, policy_version 47532 (0.0011) [2023-12-26 15:42:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 24420352. Throughput: 0: 9679.3, 1: 9719.7. Samples: 24386648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:42:46,062][104569] Avg episode reward: [(0, '8921.798'), (1, '9086.576')] [2023-12-26 15:42:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000047536_12173312.pth... [2023-12-26 15:42:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000047824_12247040.pth... [2023-12-26 15:42:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000046384_11878400.pth [2023-12-26 15:42:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000046704_11960320.pth [2023-12-26 15:42:46,533][105620] Updated weights for policy 1, policy_version 47833 (0.0010) [2023-12-26 15:42:46,577][105620] Updated weights for policy 1, policy_version 47843 (0.0005) [2023-12-26 15:42:46,607][105692] Updated weights for policy 0, policy_version 47542 (0.0011) [2023-12-26 15:42:46,625][105620] Updated weights for policy 1, policy_version 47853 (0.0005) [2023-12-26 15:42:46,655][105692] Updated weights for policy 0, policy_version 47552 (0.0010) [2023-12-26 15:42:46,703][105692] Updated weights for policy 0, policy_version 47562 (0.0010) [2023-12-26 15:42:47,373][105620] Updated weights for policy 1, policy_version 47863 (0.0008) [2023-12-26 15:42:47,425][105620] Updated weights for policy 1, policy_version 47873 (0.0006) [2023-12-26 15:42:47,469][105692] Updated weights for policy 0, policy_version 47572 (0.0010) [2023-12-26 15:42:47,476][105620] Updated weights for policy 1, policy_version 47883 (0.0005) [2023-12-26 15:42:47,537][105692] Updated weights for policy 0, policy_version 47582 (0.0010) [2023-12-26 15:42:47,609][105692] Updated weights for policy 0, policy_version 47592 (0.0010) [2023-12-26 15:42:48,103][105620] Updated weights for policy 1, policy_version 47893 (0.0005) [2023-12-26 15:42:48,154][105620] Updated weights for policy 1, policy_version 47903 (0.0005) [2023-12-26 15:42:48,210][105620] Updated weights for policy 1, policy_version 47913 (0.0005) [2023-12-26 15:42:48,348][105692] Updated weights for policy 0, policy_version 47602 (0.0010) [2023-12-26 15:42:48,406][105692] Updated weights for policy 0, policy_version 47612 (0.0011) [2023-12-26 15:42:48,476][105692] Updated weights for policy 0, policy_version 47622 (0.0011) [2023-12-26 15:42:48,541][105692] Updated weights for policy 0, policy_version 47632 (0.0011) [2023-12-26 15:42:48,855][105620] Updated weights for policy 1, policy_version 47923 (0.0007) [2023-12-26 15:42:48,921][105620] Updated weights for policy 1, policy_version 47933 (0.0010) [2023-12-26 15:42:48,987][105620] Updated weights for policy 1, policy_version 47943 (0.0010) [2023-12-26 15:42:49,209][105692] Updated weights for policy 0, policy_version 47642 (0.0005) [2023-12-26 15:42:49,274][105692] Updated weights for policy 0, policy_version 47652 (0.0008) [2023-12-26 15:42:49,327][105692] Updated weights for policy 0, policy_version 47662 (0.0010) [2023-12-26 15:42:49,664][105620] Updated weights for policy 1, policy_version 47953 (0.0007) [2023-12-26 15:42:49,723][105620] Updated weights for policy 1, policy_version 47963 (0.0010) [2023-12-26 15:42:49,791][105620] Updated weights for policy 1, policy_version 47973 (0.0006) [2023-12-26 15:42:49,862][105620] Updated weights for policy 1, policy_version 47983 (0.0009) [2023-12-26 15:42:50,060][105692] Updated weights for policy 0, policy_version 47672 (0.0010) [2023-12-26 15:42:50,127][105692] Updated weights for policy 0, policy_version 47682 (0.0011) [2023-12-26 15:42:50,194][105692] Updated weights for policy 0, policy_version 47692 (0.0011) [2023-12-26 15:42:50,526][105620] Updated weights for policy 1, policy_version 47993 (0.0007) [2023-12-26 15:42:50,592][105620] Updated weights for policy 1, policy_version 48003 (0.0012) [2023-12-26 15:42:50,661][105620] Updated weights for policy 1, policy_version 48013 (0.0007) [2023-12-26 15:42:50,981][105692] Updated weights for policy 0, policy_version 47702 (0.0010) [2023-12-26 15:42:51,037][105692] Updated weights for policy 0, policy_version 47712 (0.0009) [2023-12-26 15:42:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 24510464. Throughput: 0: 9752.9, 1: 9696.7. Samples: 24503864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:42:51,062][104569] Avg episode reward: [(0, '9098.402'), (1, '9172.810')] [2023-12-26 15:42:51,088][105692] Updated weights for policy 0, policy_version 47722 (0.0009) [2023-12-26 15:42:51,306][105620] Updated weights for policy 1, policy_version 48023 (0.0010) [2023-12-26 15:42:51,369][105620] Updated weights for policy 1, policy_version 48033 (0.0011) [2023-12-26 15:42:51,422][105620] Updated weights for policy 1, policy_version 48043 (0.0009) [2023-12-26 15:42:51,887][105692] Updated weights for policy 0, policy_version 47732 (0.0007) [2023-12-26 15:42:51,944][105692] Updated weights for policy 0, policy_version 47742 (0.0005) [2023-12-26 15:42:51,997][105692] Updated weights for policy 0, policy_version 47752 (0.0005) [2023-12-26 15:42:52,094][105620] Updated weights for policy 1, policy_version 48053 (0.0007) [2023-12-26 15:42:52,152][105620] Updated weights for policy 1, policy_version 48063 (0.0007) [2023-12-26 15:42:52,213][105620] Updated weights for policy 1, policy_version 48073 (0.0005) [2023-12-26 15:42:52,649][105692] Updated weights for policy 0, policy_version 47762 (0.0007) [2023-12-26 15:42:52,702][105692] Updated weights for policy 0, policy_version 47772 (0.0006) [2023-12-26 15:42:52,764][105692] Updated weights for policy 0, policy_version 47782 (0.0009) [2023-12-26 15:42:52,828][105692] Updated weights for policy 0, policy_version 47792 (0.0009) [2023-12-26 15:42:52,850][105620] Updated weights for policy 1, policy_version 48083 (0.0008) [2023-12-26 15:42:52,905][105620] Updated weights for policy 1, policy_version 48093 (0.0010) [2023-12-26 15:42:52,971][105620] Updated weights for policy 1, policy_version 48103 (0.0010) [2023-12-26 15:42:53,420][105692] Updated weights for policy 0, policy_version 47802 (0.0009) [2023-12-26 15:42:53,464][105692] Updated weights for policy 0, policy_version 47812 (0.0009) [2023-12-26 15:42:53,523][105692] Updated weights for policy 0, policy_version 47822 (0.0010) [2023-12-26 15:42:53,669][105620] Updated weights for policy 1, policy_version 48113 (0.0010) [2023-12-26 15:42:53,717][105620] Updated weights for policy 1, policy_version 48123 (0.0010) [2023-12-26 15:42:53,761][105620] Updated weights for policy 1, policy_version 48133 (0.0010) [2023-12-26 15:42:53,822][105620] Updated weights for policy 1, policy_version 48143 (0.0010) [2023-12-26 15:42:54,168][105692] Updated weights for policy 0, policy_version 47832 (0.0007) [2023-12-26 15:42:54,235][105692] Updated weights for policy 0, policy_version 47842 (0.0005) [2023-12-26 15:42:54,286][105692] Updated weights for policy 0, policy_version 47852 (0.0005) [2023-12-26 15:42:54,488][105620] Updated weights for policy 1, policy_version 48153 (0.0009) [2023-12-26 15:42:54,540][105620] Updated weights for policy 1, policy_version 48163 (0.0008) [2023-12-26 15:42:54,599][105620] Updated weights for policy 1, policy_version 48173 (0.0008) [2023-12-26 15:42:54,936][105692] Updated weights for policy 0, policy_version 47862 (0.0008) [2023-12-26 15:42:55,000][105692] Updated weights for policy 0, policy_version 47872 (0.0011) [2023-12-26 15:42:55,048][105692] Updated weights for policy 0, policy_version 47882 (0.0010) [2023-12-26 15:42:55,355][105620] Updated weights for policy 1, policy_version 48183 (0.0006) [2023-12-26 15:42:55,416][105620] Updated weights for policy 1, policy_version 48193 (0.0005) [2023-12-26 15:42:55,473][105620] Updated weights for policy 1, policy_version 48203 (0.0009) [2023-12-26 15:42:55,824][105692] Updated weights for policy 0, policy_version 47892 (0.0008) [2023-12-26 15:42:55,874][105692] Updated weights for policy 0, policy_version 47902 (0.0005) [2023-12-26 15:42:55,936][105692] Updated weights for policy 0, policy_version 47912 (0.0008) [2023-12-26 15:42:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.7, 300 sec: 19605.2). Total num frames: 24616960. Throughput: 0: 9788.3, 1: 9759.9. Samples: 24624000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:42:56,063][104569] Avg episode reward: [(0, '8672.170'), (1, '9261.015')] [2023-12-26 15:42:56,162][105620] Updated weights for policy 1, policy_version 48213 (0.0009) [2023-12-26 15:42:56,217][105620] Updated weights for policy 1, policy_version 48223 (0.0008) [2023-12-26 15:42:56,270][105620] Updated weights for policy 1, policy_version 48233 (0.0006) [2023-12-26 15:42:56,519][105692] Updated weights for policy 0, policy_version 47922 (0.0010) [2023-12-26 15:42:56,571][105692] Updated weights for policy 0, policy_version 47932 (0.0009) [2023-12-26 15:42:56,630][105692] Updated weights for policy 0, policy_version 47943 (0.0012) [2023-12-26 15:42:56,879][105620] Updated weights for policy 1, policy_version 48243 (0.0006) [2023-12-26 15:42:56,936][105620] Updated weights for policy 1, policy_version 48253 (0.0006) [2023-12-26 15:42:56,982][105620] Updated weights for policy 1, policy_version 48263 (0.0005) [2023-12-26 15:42:57,296][105692] Updated weights for policy 0, policy_version 47954 (0.0009) [2023-12-26 15:42:57,349][105692] Updated weights for policy 0, policy_version 47964 (0.0005) [2023-12-26 15:42:57,404][105692] Updated weights for policy 0, policy_version 47974 (0.0010) [2023-12-26 15:42:57,549][105620] Updated weights for policy 1, policy_version 48273 (0.0005) [2023-12-26 15:42:57,605][105620] Updated weights for policy 1, policy_version 48283 (0.0005) [2023-12-26 15:42:57,660][105620] Updated weights for policy 1, policy_version 48293 (0.0005) [2023-12-26 15:42:57,715][105620] Updated weights for policy 1, policy_version 48303 (0.0006) [2023-12-26 15:42:58,074][105692] Updated weights for policy 0, policy_version 47985 (0.0010) [2023-12-26 15:42:58,134][105692] Updated weights for policy 0, policy_version 47995 (0.0009) [2023-12-26 15:42:58,198][105692] Updated weights for policy 0, policy_version 48005 (0.0010) [2023-12-26 15:42:58,254][105692] Updated weights for policy 0, policy_version 48015 (0.0008) [2023-12-26 15:42:58,478][105620] Updated weights for policy 1, policy_version 48313 (0.0008) [2023-12-26 15:42:58,540][105620] Updated weights for policy 1, policy_version 48323 (0.0008) [2023-12-26 15:42:58,605][105620] Updated weights for policy 1, policy_version 48333 (0.0008) [2023-12-26 15:42:59,084][105692] Updated weights for policy 0, policy_version 48025 (0.0008) [2023-12-26 15:42:59,142][105692] Updated weights for policy 0, policy_version 48035 (0.0010) [2023-12-26 15:42:59,203][105692] Updated weights for policy 0, policy_version 48045 (0.0010) [2023-12-26 15:42:59,363][105620] Updated weights for policy 1, policy_version 48343 (0.0008) [2023-12-26 15:42:59,417][105620] Updated weights for policy 1, policy_version 48353 (0.0009) [2023-12-26 15:42:59,466][105620] Updated weights for policy 1, policy_version 48363 (0.0008) [2023-12-26 15:42:59,984][105692] Updated weights for policy 0, policy_version 48055 (0.0009) [2023-12-26 15:43:00,045][105692] Updated weights for policy 0, policy_version 48065 (0.0009) [2023-12-26 15:43:00,104][105692] Updated weights for policy 0, policy_version 48075 (0.0009) [2023-12-26 15:43:00,233][105620] Updated weights for policy 1, policy_version 48373 (0.0009) [2023-12-26 15:43:00,286][105620] Updated weights for policy 1, policy_version 48383 (0.0009) [2023-12-26 15:43:00,332][105620] Updated weights for policy 1, policy_version 48393 (0.0009) [2023-12-26 15:43:00,829][105692] Updated weights for policy 0, policy_version 48085 (0.0009) [2023-12-26 15:43:00,883][105692] Updated weights for policy 0, policy_version 48095 (0.0009) [2023-12-26 15:43:00,941][105692] Updated weights for policy 0, policy_version 48105 (0.0009) [2023-12-26 15:43:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 24715264. Throughput: 0: 9826.7, 1: 9783.7. Samples: 24685568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:43:01,063][104569] Avg episode reward: [(0, '8358.230'), (1, '9354.014')] [2023-12-26 15:43:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000048112_12320768.pth... [2023-12-26 15:43:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000046992_12034048.pth [2023-12-26 15:43:01,089][105620] Updated weights for policy 1, policy_version 48403 (0.0008) [2023-12-26 15:43:01,150][105620] Updated weights for policy 1, policy_version 48413 (0.0008) [2023-12-26 15:43:01,208][105620] Updated weights for policy 1, policy_version 48423 (0.0009) [2023-12-26 15:43:01,258][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000048432_12402688.pth... [2023-12-26 15:43:01,262][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000047280_12107776.pth [2023-12-26 15:43:01,263][105586] Saving new best policy, reward=9354.014! [2023-12-26 15:43:01,666][105692] Updated weights for policy 0, policy_version 48115 (0.0009) [2023-12-26 15:43:01,714][105692] Updated weights for policy 0, policy_version 48125 (0.0010) [2023-12-26 15:43:01,778][105692] Updated weights for policy 0, policy_version 48135 (0.0011) [2023-12-26 15:43:02,036][105620] Updated weights for policy 1, policy_version 48433 (0.0008) [2023-12-26 15:43:02,085][105620] Updated weights for policy 1, policy_version 48443 (0.0005) [2023-12-26 15:43:02,138][105620] Updated weights for policy 1, policy_version 48453 (0.0005) [2023-12-26 15:43:02,199][105620] Updated weights for policy 1, policy_version 48463 (0.0005) [2023-12-26 15:43:02,545][105692] Updated weights for policy 0, policy_version 48145 (0.0011) [2023-12-26 15:43:02,603][105692] Updated weights for policy 0, policy_version 48155 (0.0006) [2023-12-26 15:43:02,665][105692] Updated weights for policy 0, policy_version 48165 (0.0007) [2023-12-26 15:43:02,721][105692] Updated weights for policy 0, policy_version 48175 (0.0009) [2023-12-26 15:43:02,899][105620] Updated weights for policy 1, policy_version 48473 (0.0009) [2023-12-26 15:43:02,955][105620] Updated weights for policy 1, policy_version 48483 (0.0008) [2023-12-26 15:43:03,006][105620] Updated weights for policy 1, policy_version 48493 (0.0009) [2023-12-26 15:43:03,452][105692] Updated weights for policy 0, policy_version 48185 (0.0009) [2023-12-26 15:43:03,498][105692] Updated weights for policy 0, policy_version 48195 (0.0008) [2023-12-26 15:43:03,544][105692] Updated weights for policy 0, policy_version 48205 (0.0008) [2023-12-26 15:43:03,692][105620] Updated weights for policy 1, policy_version 48503 (0.0008) [2023-12-26 15:43:03,749][105620] Updated weights for policy 1, policy_version 48513 (0.0007) [2023-12-26 15:43:03,798][105620] Updated weights for policy 1, policy_version 48523 (0.0009) [2023-12-26 15:43:04,207][105692] Updated weights for policy 0, policy_version 48215 (0.0008) [2023-12-26 15:43:04,265][105692] Updated weights for policy 0, policy_version 48225 (0.0006) [2023-12-26 15:43:04,320][105692] Updated weights for policy 0, policy_version 48235 (0.0005) [2023-12-26 15:43:04,695][105620] Updated weights for policy 1, policy_version 48533 (0.0009) [2023-12-26 15:43:04,748][105620] Updated weights for policy 1, policy_version 48543 (0.0010) [2023-12-26 15:43:04,806][105620] Updated weights for policy 1, policy_version 48553 (0.0008) [2023-12-26 15:43:04,893][105692] Updated weights for policy 0, policy_version 48245 (0.0008) [2023-12-26 15:43:04,954][105692] Updated weights for policy 0, policy_version 48255 (0.0010) [2023-12-26 15:43:05,012][105692] Updated weights for policy 0, policy_version 48265 (0.0010) [2023-12-26 15:43:05,514][105620] Updated weights for policy 1, policy_version 48563 (0.0008) [2023-12-26 15:43:05,574][105620] Updated weights for policy 1, policy_version 48573 (0.0007) [2023-12-26 15:43:05,665][105620] Updated weights for policy 1, policy_version 48583 (0.0009) [2023-12-26 15:43:05,686][105692] Updated weights for policy 0, policy_version 48275 (0.0009) [2023-12-26 15:43:05,749][105692] Updated weights for policy 0, policy_version 48285 (0.0006) [2023-12-26 15:43:05,811][105692] Updated weights for policy 0, policy_version 48295 (0.0006) [2023-12-26 15:43:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 24813568. Throughput: 0: 9713.9, 1: 9738.9. Samples: 24799112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:43:06,063][104569] Avg episode reward: [(0, '8364.544'), (1, '9266.605')] [2023-12-26 15:43:06,378][105620] Updated weights for policy 1, policy_version 48593 (0.0009) [2023-12-26 15:43:06,432][105620] Updated weights for policy 1, policy_version 48603 (0.0009) [2023-12-26 15:43:06,480][105620] Updated weights for policy 1, policy_version 48613 (0.0008) [2023-12-26 15:43:06,535][105620] Updated weights for policy 1, policy_version 48623 (0.0009) [2023-12-26 15:43:06,547][105692] Updated weights for policy 0, policy_version 48305 (0.0007) [2023-12-26 15:43:06,600][105692] Updated weights for policy 0, policy_version 48315 (0.0009) [2023-12-26 15:43:06,657][105692] Updated weights for policy 0, policy_version 48326 (0.0010) [2023-12-26 15:43:06,751][105692] Updated weights for policy 0, policy_version 48336 (0.0010) [2023-12-26 15:43:07,244][105620] Updated weights for policy 1, policy_version 48633 (0.0006) [2023-12-26 15:43:07,296][105620] Updated weights for policy 1, policy_version 48643 (0.0009) [2023-12-26 15:43:07,350][105620] Updated weights for policy 1, policy_version 48653 (0.0009) [2023-12-26 15:43:07,432][105692] Updated weights for policy 0, policy_version 48346 (0.0007) [2023-12-26 15:43:07,491][105692] Updated weights for policy 0, policy_version 48356 (0.0005) [2023-12-26 15:43:07,550][105692] Updated weights for policy 0, policy_version 48366 (0.0005) [2023-12-26 15:43:08,096][105620] Updated weights for policy 1, policy_version 48663 (0.0006) [2023-12-26 15:43:08,146][105620] Updated weights for policy 1, policy_version 48673 (0.0005) [2023-12-26 15:43:08,189][105692] Updated weights for policy 0, policy_version 48376 (0.0009) [2023-12-26 15:43:08,191][105620] Updated weights for policy 1, policy_version 48683 (0.0005) [2023-12-26 15:43:08,244][105692] Updated weights for policy 0, policy_version 48386 (0.0010) [2023-12-26 15:43:08,299][105692] Updated weights for policy 0, policy_version 48396 (0.0010) [2023-12-26 15:43:08,871][105620] Updated weights for policy 1, policy_version 48693 (0.0005) [2023-12-26 15:43:08,941][105620] Updated weights for policy 1, policy_version 48703 (0.0007) [2023-12-26 15:43:09,007][105620] Updated weights for policy 1, policy_version 48713 (0.0007) [2023-12-26 15:43:09,040][105692] Updated weights for policy 0, policy_version 48406 (0.0008) [2023-12-26 15:43:09,100][105692] Updated weights for policy 0, policy_version 48416 (0.0006) [2023-12-26 15:43:09,154][105692] Updated weights for policy 0, policy_version 48426 (0.0010) [2023-12-26 15:43:09,610][105620] Updated weights for policy 1, policy_version 48723 (0.0007) [2023-12-26 15:43:09,668][105620] Updated weights for policy 1, policy_version 48733 (0.0008) [2023-12-26 15:43:09,742][105620] Updated weights for policy 1, policy_version 48743 (0.0006) [2023-12-26 15:43:09,922][105692] Updated weights for policy 0, policy_version 48436 (0.0008) [2023-12-26 15:43:09,995][105692] Updated weights for policy 0, policy_version 48446 (0.0010) [2023-12-26 15:43:10,058][105692] Updated weights for policy 0, policy_version 48456 (0.0008) [2023-12-26 15:43:10,427][105620] Updated weights for policy 1, policy_version 48753 (0.0006) [2023-12-26 15:43:10,490][105620] Updated weights for policy 1, policy_version 48763 (0.0011) [2023-12-26 15:43:10,559][105620] Updated weights for policy 1, policy_version 48773 (0.0011) [2023-12-26 15:43:10,622][105620] Updated weights for policy 1, policy_version 48783 (0.0010) [2023-12-26 15:43:10,754][105692] Updated weights for policy 0, policy_version 48466 (0.0009) [2023-12-26 15:43:10,807][105692] Updated weights for policy 0, policy_version 48476 (0.0009) [2023-12-26 15:43:10,864][105692] Updated weights for policy 0, policy_version 48486 (0.0008) [2023-12-26 15:43:10,917][105692] Updated weights for policy 0, policy_version 48496 (0.0008) [2023-12-26 15:43:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 24911872. Throughput: 0: 9705.5, 1: 9798.5. Samples: 24917696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:43:11,063][104569] Avg episode reward: [(0, '2248.866'), (1, '9173.866')] [2023-12-26 15:43:11,327][105620] Updated weights for policy 1, policy_version 48793 (0.0010) [2023-12-26 15:43:11,396][105620] Updated weights for policy 1, policy_version 48803 (0.0010) [2023-12-26 15:43:11,464][105620] Updated weights for policy 1, policy_version 48813 (0.0011) [2023-12-26 15:43:11,726][105692] Updated weights for policy 0, policy_version 48506 (0.0008) [2023-12-26 15:43:11,788][105692] Updated weights for policy 0, policy_version 48516 (0.0008) [2023-12-26 15:43:11,848][105692] Updated weights for policy 0, policy_version 48526 (0.0008) [2023-12-26 15:43:12,216][105620] Updated weights for policy 1, policy_version 48823 (0.0010) [2023-12-26 15:43:12,279][105620] Updated weights for policy 1, policy_version 48833 (0.0011) [2023-12-26 15:43:12,347][105620] Updated weights for policy 1, policy_version 48843 (0.0010) [2023-12-26 15:43:12,526][105692] Updated weights for policy 0, policy_version 48536 (0.0008) [2023-12-26 15:43:12,574][105692] Updated weights for policy 0, policy_version 48546 (0.0007) [2023-12-26 15:43:12,626][105692] Updated weights for policy 0, policy_version 48556 (0.0008) [2023-12-26 15:43:13,038][105620] Updated weights for policy 1, policy_version 48853 (0.0011) [2023-12-26 15:43:13,086][105620] Updated weights for policy 1, policy_version 48863 (0.0010) [2023-12-26 15:43:13,134][105620] Updated weights for policy 1, policy_version 48873 (0.0010) [2023-12-26 15:43:13,327][105692] Updated weights for policy 0, policy_version 48566 (0.0009) [2023-12-26 15:43:13,390][105692] Updated weights for policy 0, policy_version 48576 (0.0005) [2023-12-26 15:43:13,444][105692] Updated weights for policy 0, policy_version 48586 (0.0005) [2023-12-26 15:43:13,727][105620] Updated weights for policy 1, policy_version 48883 (0.0009) [2023-12-26 15:43:13,787][105620] Updated weights for policy 1, policy_version 48893 (0.0006) [2023-12-26 15:43:13,850][105620] Updated weights for policy 1, policy_version 48903 (0.0007) [2023-12-26 15:43:14,116][105692] Updated weights for policy 0, policy_version 48596 (0.0007) [2023-12-26 15:43:14,188][105692] Updated weights for policy 0, policy_version 48606 (0.0010) [2023-12-26 15:43:14,250][105692] Updated weights for policy 0, policy_version 48616 (0.0008) [2023-12-26 15:43:14,393][105620] Updated weights for policy 1, policy_version 48913 (0.0006) [2023-12-26 15:43:14,448][105620] Updated weights for policy 1, policy_version 48923 (0.0010) [2023-12-26 15:43:14,499][105620] Updated weights for policy 1, policy_version 48933 (0.0010) [2023-12-26 15:43:14,554][105620] Updated weights for policy 1, policy_version 48943 (0.0010) [2023-12-26 15:43:14,856][105692] Updated weights for policy 0, policy_version 48626 (0.0007) [2023-12-26 15:43:14,923][105692] Updated weights for policy 0, policy_version 48636 (0.0010) [2023-12-26 15:43:14,985][105692] Updated weights for policy 0, policy_version 48646 (0.0008) [2023-12-26 15:43:15,050][105692] Updated weights for policy 0, policy_version 48656 (0.0009) [2023-12-26 15:43:15,252][105620] Updated weights for policy 1, policy_version 48953 (0.0008) [2023-12-26 15:43:15,312][105620] Updated weights for policy 1, policy_version 48963 (0.0006) [2023-12-26 15:43:15,360][105620] Updated weights for policy 1, policy_version 48973 (0.0005) [2023-12-26 15:43:15,895][105692] Updated weights for policy 0, policy_version 48666 (0.0009) [2023-12-26 15:43:15,899][105620] Updated weights for policy 1, policy_version 48983 (0.0005) [2023-12-26 15:43:15,947][105692] Updated weights for policy 0, policy_version 48676 (0.0009) [2023-12-26 15:43:15,954][105620] Updated weights for policy 1, policy_version 48993 (0.0007) [2023-12-26 15:43:16,004][105692] Updated weights for policy 0, policy_version 48686 (0.0006) [2023-12-26 15:43:16,010][105620] Updated weights for policy 1, policy_version 49003 (0.0010) [2023-12-26 15:43:16,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 25018368. Throughput: 0: 9666.5, 1: 9757.4. Samples: 24977944. Policy #0 lag: (min: 26.0, avg: 29.1, max: 58.0) [2023-12-26 15:43:16,062][104569] Avg episode reward: [(0, '1286.025'), (1, '9260.343')] [2023-12-26 15:43:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000048688_12468224.pth... [2023-12-26 15:43:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000049008_12550144.pth... [2023-12-26 15:43:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000047536_12173312.pth [2023-12-26 15:43:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000047824_12247040.pth [2023-12-26 15:43:16,643][105620] Updated weights for policy 1, policy_version 49013 (0.0008) [2023-12-26 15:43:16,695][105620] Updated weights for policy 1, policy_version 49023 (0.0006) [2023-12-26 15:43:16,764][105620] Updated weights for policy 1, policy_version 49033 (0.0005) [2023-12-26 15:43:16,790][105692] Updated weights for policy 0, policy_version 48696 (0.0008) [2023-12-26 15:43:16,854][105692] Updated weights for policy 0, policy_version 48706 (0.0009) [2023-12-26 15:43:16,906][105692] Updated weights for policy 0, policy_version 48716 (0.0008) [2023-12-26 15:43:17,412][105620] Updated weights for policy 1, policy_version 49043 (0.0007) [2023-12-26 15:43:17,469][105620] Updated weights for policy 1, policy_version 49053 (0.0010) [2023-12-26 15:43:17,524][105620] Updated weights for policy 1, policy_version 49063 (0.0010) [2023-12-26 15:43:17,674][105692] Updated weights for policy 0, policy_version 48726 (0.0009) [2023-12-26 15:43:17,733][105692] Updated weights for policy 0, policy_version 48736 (0.0008) [2023-12-26 15:43:17,789][105692] Updated weights for policy 0, policy_version 48746 (0.0008) [2023-12-26 15:43:18,269][105620] Updated weights for policy 1, policy_version 49073 (0.0010) [2023-12-26 15:43:18,324][105620] Updated weights for policy 1, policy_version 49083 (0.0010) [2023-12-26 15:43:18,396][105620] Updated weights for policy 1, policy_version 49093 (0.0011) [2023-12-26 15:43:18,455][105620] Updated weights for policy 1, policy_version 49103 (0.0011) [2023-12-26 15:43:18,564][105692] Updated weights for policy 0, policy_version 48756 (0.0009) [2023-12-26 15:43:18,620][105692] Updated weights for policy 0, policy_version 48766 (0.0008) [2023-12-26 15:43:18,676][105692] Updated weights for policy 0, policy_version 48776 (0.0008) [2023-12-26 15:43:19,193][105620] Updated weights for policy 1, policy_version 49113 (0.0011) [2023-12-26 15:43:19,260][105620] Updated weights for policy 1, policy_version 49123 (0.0010) [2023-12-26 15:43:19,331][105620] Updated weights for policy 1, policy_version 49133 (0.0006) [2023-12-26 15:43:19,418][105692] Updated weights for policy 0, policy_version 48786 (0.0006) [2023-12-26 15:43:19,481][105692] Updated weights for policy 0, policy_version 48796 (0.0009) [2023-12-26 15:43:19,543][105692] Updated weights for policy 0, policy_version 48806 (0.0010) [2023-12-26 15:43:19,607][105692] Updated weights for policy 0, policy_version 48816 (0.0010) [2023-12-26 15:43:20,080][105620] Updated weights for policy 1, policy_version 49143 (0.0011) [2023-12-26 15:43:20,138][105620] Updated weights for policy 1, policy_version 49153 (0.0011) [2023-12-26 15:43:20,198][105620] Updated weights for policy 1, policy_version 49163 (0.0011) [2023-12-26 15:43:20,305][105692] Updated weights for policy 0, policy_version 48826 (0.0006) [2023-12-26 15:43:20,375][105692] Updated weights for policy 0, policy_version 48836 (0.0008) [2023-12-26 15:43:20,441][105692] Updated weights for policy 0, policy_version 48846 (0.0011) [2023-12-26 15:43:20,959][105620] Updated weights for policy 1, policy_version 49173 (0.0011) [2023-12-26 15:43:21,018][105620] Updated weights for policy 1, policy_version 49183 (0.0010) [2023-12-26 15:43:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 25100288. Throughput: 0: 9596.7, 1: 9888.3. Samples: 25095572. Policy #0 lag: (min: 26.0, avg: 29.1, max: 58.0) [2023-12-26 15:43:21,063][104569] Avg episode reward: [(0, '1557.996'), (1, '9165.608')] [2023-12-26 15:43:21,082][105620] Updated weights for policy 1, policy_version 49193 (0.0011) [2023-12-26 15:43:21,141][105692] Updated weights for policy 0, policy_version 48856 (0.0009) [2023-12-26 15:43:21,207][105692] Updated weights for policy 0, policy_version 48866 (0.0008) [2023-12-26 15:43:21,276][105692] Updated weights for policy 0, policy_version 48876 (0.0008) [2023-12-26 15:43:21,826][105620] Updated weights for policy 1, policy_version 49203 (0.0010) [2023-12-26 15:43:21,884][105620] Updated weights for policy 1, policy_version 49213 (0.0011) [2023-12-26 15:43:21,942][105620] Updated weights for policy 1, policy_version 49223 (0.0011) [2023-12-26 15:43:21,997][105692] Updated weights for policy 0, policy_version 48886 (0.0007) [2023-12-26 15:43:22,061][105692] Updated weights for policy 0, policy_version 48896 (0.0006) [2023-12-26 15:43:22,118][105692] Updated weights for policy 0, policy_version 48906 (0.0007) [2023-12-26 15:43:22,624][105620] Updated weights for policy 1, policy_version 49233 (0.0011) [2023-12-26 15:43:22,679][105620] Updated weights for policy 1, policy_version 49243 (0.0007) [2023-12-26 15:43:22,731][105620] Updated weights for policy 1, policy_version 49253 (0.0008) [2023-12-26 15:43:22,782][105620] Updated weights for policy 1, policy_version 49263 (0.0007) [2023-12-26 15:43:22,904][105692] Updated weights for policy 0, policy_version 48916 (0.0009) [2023-12-26 15:43:22,958][105692] Updated weights for policy 0, policy_version 48926 (0.0008) [2023-12-26 15:43:23,019][105692] Updated weights for policy 0, policy_version 48936 (0.0005) [2023-12-26 15:43:23,396][105620] Updated weights for policy 1, policy_version 49273 (0.0009) [2023-12-26 15:43:23,448][105620] Updated weights for policy 1, policy_version 49283 (0.0010) [2023-12-26 15:43:23,504][105620] Updated weights for policy 1, policy_version 49293 (0.0010) [2023-12-26 15:43:23,733][105692] Updated weights for policy 0, policy_version 48946 (0.0007) [2023-12-26 15:43:23,794][105692] Updated weights for policy 0, policy_version 48956 (0.0010) [2023-12-26 15:43:23,849][105692] Updated weights for policy 0, policy_version 48966 (0.0010) [2023-12-26 15:43:23,908][105692] Updated weights for policy 0, policy_version 48976 (0.0006) [2023-12-26 15:43:24,153][105620] Updated weights for policy 1, policy_version 49303 (0.0010) [2023-12-26 15:43:24,212][105620] Updated weights for policy 1, policy_version 49313 (0.0010) [2023-12-26 15:43:24,260][105620] Updated weights for policy 1, policy_version 49323 (0.0010) [2023-12-26 15:43:24,557][105692] Updated weights for policy 0, policy_version 48986 (0.0010) [2023-12-26 15:43:24,621][105692] Updated weights for policy 0, policy_version 48996 (0.0010) [2023-12-26 15:43:24,679][105692] Updated weights for policy 0, policy_version 49006 (0.0010) [2023-12-26 15:43:24,979][105620] Updated weights for policy 1, policy_version 49333 (0.0008) [2023-12-26 15:43:25,034][105620] Updated weights for policy 1, policy_version 49343 (0.0005) [2023-12-26 15:43:25,091][105620] Updated weights for policy 1, policy_version 49353 (0.0005) [2023-12-26 15:43:25,385][105692] Updated weights for policy 0, policy_version 49016 (0.0010) [2023-12-26 15:43:25,432][105692] Updated weights for policy 0, policy_version 49026 (0.0010) [2023-12-26 15:43:25,500][105692] Updated weights for policy 0, policy_version 49036 (0.0010) [2023-12-26 15:43:25,608][105620] Updated weights for policy 1, policy_version 49363 (0.0007) [2023-12-26 15:43:25,655][105620] Updated weights for policy 1, policy_version 49373 (0.0010) [2023-12-26 15:43:25,698][105620] Updated weights for policy 1, policy_version 49383 (0.0006) [2023-12-26 15:43:26,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 25206784. Throughput: 0: 9651.3, 1: 10019.0. Samples: 25215980. Policy #0 lag: (min: 26.0, avg: 29.1, max: 58.0) [2023-12-26 15:43:26,062][104569] Avg episode reward: [(0, '1225.420'), (1, '9162.852')] [2023-12-26 15:43:26,213][105692] Updated weights for policy 0, policy_version 49046 (0.0010) [2023-12-26 15:43:26,272][105692] Updated weights for policy 0, policy_version 49056 (0.0008) [2023-12-26 15:43:26,332][105620] Updated weights for policy 1, policy_version 49393 (0.0005) [2023-12-26 15:43:26,340][105692] Updated weights for policy 0, policy_version 49066 (0.0005) [2023-12-26 15:43:26,393][105620] Updated weights for policy 1, policy_version 49403 (0.0005) [2023-12-26 15:43:26,448][105620] Updated weights for policy 1, policy_version 49413 (0.0005) [2023-12-26 15:43:26,508][105620] Updated weights for policy 1, policy_version 49423 (0.0006) [2023-12-26 15:43:26,888][105692] Updated weights for policy 0, policy_version 49076 (0.0007) [2023-12-26 15:43:26,943][105692] Updated weights for policy 0, policy_version 49086 (0.0007) [2023-12-26 15:43:26,996][105692] Updated weights for policy 0, policy_version 49096 (0.0005) [2023-12-26 15:43:27,003][105620] Updated weights for policy 1, policy_version 49433 (0.0005) [2023-12-26 15:43:27,051][105620] Updated weights for policy 1, policy_version 49443 (0.0010) [2023-12-26 15:43:27,110][105620] Updated weights for policy 1, policy_version 49453 (0.0011) [2023-12-26 15:43:27,615][105692] Updated weights for policy 0, policy_version 49106 (0.0005) [2023-12-26 15:43:27,667][105692] Updated weights for policy 0, policy_version 49116 (0.0005) [2023-12-26 15:43:27,724][105692] Updated weights for policy 0, policy_version 49126 (0.0007) [2023-12-26 15:43:27,781][105692] Updated weights for policy 0, policy_version 49136 (0.0010) [2023-12-26 15:43:27,840][105620] Updated weights for policy 1, policy_version 49463 (0.0010) [2023-12-26 15:43:27,900][105620] Updated weights for policy 1, policy_version 49473 (0.0010) [2023-12-26 15:43:27,960][105620] Updated weights for policy 1, policy_version 49483 (0.0010) [2023-12-26 15:43:28,399][105692] Updated weights for policy 0, policy_version 49146 (0.0008) [2023-12-26 15:43:28,450][105692] Updated weights for policy 0, policy_version 49156 (0.0008) [2023-12-26 15:43:28,510][105692] Updated weights for policy 0, policy_version 49166 (0.0008) [2023-12-26 15:43:28,563][105620] Updated weights for policy 1, policy_version 49493 (0.0008) [2023-12-26 15:43:28,622][105620] Updated weights for policy 1, policy_version 49503 (0.0005) [2023-12-26 15:43:28,685][105620] Updated weights for policy 1, policy_version 49513 (0.0008) [2023-12-26 15:43:29,190][105692] Updated weights for policy 0, policy_version 49176 (0.0007) [2023-12-26 15:43:29,253][105692] Updated weights for policy 0, policy_version 49186 (0.0008) [2023-12-26 15:43:29,319][105692] Updated weights for policy 0, policy_version 49196 (0.0009) [2023-12-26 15:43:29,362][105620] Updated weights for policy 1, policy_version 49523 (0.0010) [2023-12-26 15:43:29,422][105620] Updated weights for policy 1, policy_version 49533 (0.0009) [2023-12-26 15:43:29,491][105620] Updated weights for policy 1, policy_version 49543 (0.0011) [2023-12-26 15:43:29,954][105692] Updated weights for policy 0, policy_version 49206 (0.0008) [2023-12-26 15:43:30,014][105692] Updated weights for policy 0, policy_version 49216 (0.0006) [2023-12-26 15:43:30,071][105692] Updated weights for policy 0, policy_version 49226 (0.0005) [2023-12-26 15:43:30,185][105620] Updated weights for policy 1, policy_version 49553 (0.0010) [2023-12-26 15:43:30,248][105620] Updated weights for policy 1, policy_version 49563 (0.0006) [2023-12-26 15:43:30,310][105620] Updated weights for policy 1, policy_version 49573 (0.0009) [2023-12-26 15:43:30,360][105620] Updated weights for policy 1, policy_version 49583 (0.0009) [2023-12-26 15:43:30,795][105692] Updated weights for policy 0, policy_version 49236 (0.0006) [2023-12-26 15:43:30,851][105692] Updated weights for policy 0, policy_version 49246 (0.0005) [2023-12-26 15:43:30,890][105620] Updated weights for policy 1, policy_version 49593 (0.0008) [2023-12-26 15:43:30,907][105692] Updated weights for policy 0, policy_version 49256 (0.0005) [2023-12-26 15:43:30,946][105620] Updated weights for policy 1, policy_version 49603 (0.0009) [2023-12-26 15:43:31,004][105620] Updated weights for policy 1, policy_version 49613 (0.0009) [2023-12-26 15:43:31,062][104569] Fps is (10 sec: 22118.3, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 25321472. Throughput: 0: 9799.1, 1: 10088.5. Samples: 25281584. Policy #0 lag: (min: 26.0, avg: 29.1, max: 58.0) [2023-12-26 15:43:31,062][104569] Avg episode reward: [(0, '1572.271'), (1, '9255.246')] [2023-12-26 15:43:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000049264_12615680.pth... [2023-12-26 15:43:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000049616_12705792.pth... [2023-12-26 15:43:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000048112_12320768.pth [2023-12-26 15:43:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000048432_12402688.pth [2023-12-26 15:43:31,512][105692] Updated weights for policy 0, policy_version 49266 (0.0006) [2023-12-26 15:43:31,570][105692] Updated weights for policy 0, policy_version 49276 (0.0007) [2023-12-26 15:43:31,642][105692] Updated weights for policy 0, policy_version 49286 (0.0006) [2023-12-26 15:43:31,704][105692] Updated weights for policy 0, policy_version 49296 (0.0006) [2023-12-26 15:43:31,760][105620] Updated weights for policy 1, policy_version 49623 (0.0008) [2023-12-26 15:43:31,825][105620] Updated weights for policy 1, policy_version 49633 (0.0009) [2023-12-26 15:43:31,882][105620] Updated weights for policy 1, policy_version 49643 (0.0010) [2023-12-26 15:43:32,238][105692] Updated weights for policy 0, policy_version 49306 (0.0005) [2023-12-26 15:43:32,292][105692] Updated weights for policy 0, policy_version 49316 (0.0006) [2023-12-26 15:43:32,358][105692] Updated weights for policy 0, policy_version 49326 (0.0008) [2023-12-26 15:43:32,558][105620] Updated weights for policy 1, policy_version 49653 (0.0008) [2023-12-26 15:43:32,612][105620] Updated weights for policy 1, policy_version 49663 (0.0009) [2023-12-26 15:43:32,666][105620] Updated weights for policy 1, policy_version 49673 (0.0009) [2023-12-26 15:43:33,035][105692] Updated weights for policy 0, policy_version 49336 (0.0008) [2023-12-26 15:43:33,082][105692] Updated weights for policy 0, policy_version 49346 (0.0009) [2023-12-26 15:43:33,139][105692] Updated weights for policy 0, policy_version 49356 (0.0009) [2023-12-26 15:43:33,374][105620] Updated weights for policy 1, policy_version 49683 (0.0010) [2023-12-26 15:43:33,428][105620] Updated weights for policy 1, policy_version 49693 (0.0007) [2023-12-26 15:43:33,472][105620] Updated weights for policy 1, policy_version 49703 (0.0010) [2023-12-26 15:43:33,782][105692] Updated weights for policy 0, policy_version 49366 (0.0008) [2023-12-26 15:43:33,847][105692] Updated weights for policy 0, policy_version 49376 (0.0009) [2023-12-26 15:43:33,903][105692] Updated weights for policy 0, policy_version 49386 (0.0009) [2023-12-26 15:43:34,221][105620] Updated weights for policy 1, policy_version 49713 (0.0010) [2023-12-26 15:43:34,286][105620] Updated weights for policy 1, policy_version 49723 (0.0010) [2023-12-26 15:43:34,349][105620] Updated weights for policy 1, policy_version 49733 (0.0010) [2023-12-26 15:43:34,412][105620] Updated weights for policy 1, policy_version 49743 (0.0007) [2023-12-26 15:43:34,570][105692] Updated weights for policy 0, policy_version 49396 (0.0008) [2023-12-26 15:43:34,627][105692] Updated weights for policy 0, policy_version 49406 (0.0011) [2023-12-26 15:43:34,684][105692] Updated weights for policy 0, policy_version 49416 (0.0006) [2023-12-26 15:43:35,097][105620] Updated weights for policy 1, policy_version 49753 (0.0010) [2023-12-26 15:43:35,152][105620] Updated weights for policy 1, policy_version 49763 (0.0010) [2023-12-26 15:43:35,205][105620] Updated weights for policy 1, policy_version 49773 (0.0010) [2023-12-26 15:43:35,416][105692] Updated weights for policy 0, policy_version 49426 (0.0011) [2023-12-26 15:43:35,464][105692] Updated weights for policy 0, policy_version 49436 (0.0010) [2023-12-26 15:43:35,512][105692] Updated weights for policy 0, policy_version 49446 (0.0010) [2023-12-26 15:43:35,560][105692] Updated weights for policy 0, policy_version 49456 (0.0010) [2023-12-26 15:43:35,840][105620] Updated weights for policy 1, policy_version 49783 (0.0007) [2023-12-26 15:43:35,884][105620] Updated weights for policy 1, policy_version 49793 (0.0005) [2023-12-26 15:43:35,950][105620] Updated weights for policy 1, policy_version 49803 (0.0005) [2023-12-26 15:43:36,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 25419776. Throughput: 0: 9919.2, 1: 10124.3. Samples: 25405820. Policy #0 lag: (min: 26.0, avg: 29.1, max: 58.0) [2023-12-26 15:43:36,062][104569] Avg episode reward: [(0, '2217.633'), (1, '9165.735')] [2023-12-26 15:43:36,306][105692] Updated weights for policy 0, policy_version 49466 (0.0009) [2023-12-26 15:43:36,367][105692] Updated weights for policy 0, policy_version 49476 (0.0009) [2023-12-26 15:43:36,430][105692] Updated weights for policy 0, policy_version 49486 (0.0008) [2023-12-26 15:43:36,672][105620] Updated weights for policy 1, policy_version 49813 (0.0008) [2023-12-26 15:43:36,726][105620] Updated weights for policy 1, policy_version 49823 (0.0010) [2023-12-26 15:43:36,778][105620] Updated weights for policy 1, policy_version 49833 (0.0010) [2023-12-26 15:43:37,132][105692] Updated weights for policy 0, policy_version 49496 (0.0009) [2023-12-26 15:43:37,189][105692] Updated weights for policy 0, policy_version 49506 (0.0008) [2023-12-26 15:43:37,242][105692] Updated weights for policy 0, policy_version 49516 (0.0008) [2023-12-26 15:43:37,535][105620] Updated weights for policy 1, policy_version 49843 (0.0010) [2023-12-26 15:43:37,590][105620] Updated weights for policy 1, policy_version 49853 (0.0010) [2023-12-26 15:43:37,649][105620] Updated weights for policy 1, policy_version 49863 (0.0010) [2023-12-26 15:43:38,015][105692] Updated weights for policy 0, policy_version 49526 (0.0006) [2023-12-26 15:43:38,068][105692] Updated weights for policy 0, policy_version 49536 (0.0006) [2023-12-26 15:43:38,114][105692] Updated weights for policy 0, policy_version 49546 (0.0008) [2023-12-26 15:43:38,394][105620] Updated weights for policy 1, policy_version 49873 (0.0010) [2023-12-26 15:43:38,462][105620] Updated weights for policy 1, policy_version 49883 (0.0009) [2023-12-26 15:43:38,521][105620] Updated weights for policy 1, policy_version 49893 (0.0005) [2023-12-26 15:43:38,587][105620] Updated weights for policy 1, policy_version 49903 (0.0005) [2023-12-26 15:43:38,814][105692] Updated weights for policy 0, policy_version 49556 (0.0008) [2023-12-26 15:43:38,875][105692] Updated weights for policy 0, policy_version 49566 (0.0005) [2023-12-26 15:43:38,935][105692] Updated weights for policy 0, policy_version 49576 (0.0006) [2023-12-26 15:43:39,161][105620] Updated weights for policy 1, policy_version 49913 (0.0010) [2023-12-26 15:43:39,219][105620] Updated weights for policy 1, policy_version 49923 (0.0008) [2023-12-26 15:43:39,286][105620] Updated weights for policy 1, policy_version 49933 (0.0010) [2023-12-26 15:43:39,575][105692] Updated weights for policy 0, policy_version 49586 (0.0008) [2023-12-26 15:43:39,624][105692] Updated weights for policy 0, policy_version 49596 (0.0010) [2023-12-26 15:43:39,674][105692] Updated weights for policy 0, policy_version 49606 (0.0007) [2023-12-26 15:43:39,721][105692] Updated weights for policy 0, policy_version 49616 (0.0006) [2023-12-26 15:43:40,112][105620] Updated weights for policy 1, policy_version 49943 (0.0009) [2023-12-26 15:43:40,171][105620] Updated weights for policy 1, policy_version 49953 (0.0010) [2023-12-26 15:43:40,230][105620] Updated weights for policy 1, policy_version 49963 (0.0010) [2023-12-26 15:43:40,369][105692] Updated weights for policy 0, policy_version 49626 (0.0010) [2023-12-26 15:43:40,432][105692] Updated weights for policy 0, policy_version 49636 (0.0010) [2023-12-26 15:43:40,490][105692] Updated weights for policy 0, policy_version 49646 (0.0011) [2023-12-26 15:43:40,967][105620] Updated weights for policy 1, policy_version 49973 (0.0009) [2023-12-26 15:43:41,031][105620] Updated weights for policy 1, policy_version 49983 (0.0009) [2023-12-26 15:43:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 25509888. Throughput: 0: 9920.6, 1: 10078.4. Samples: 25523952. Policy #0 lag: (min: 26.0, avg: 29.1, max: 58.0) [2023-12-26 15:43:41,063][104569] Avg episode reward: [(0, '6371.449'), (1, '9078.730')] [2023-12-26 15:43:41,092][105620] Updated weights for policy 1, policy_version 49993 (0.0008) [2023-12-26 15:43:41,266][105692] Updated weights for policy 0, policy_version 49656 (0.0009) [2023-12-26 15:43:41,334][105692] Updated weights for policy 0, policy_version 49666 (0.0008) [2023-12-26 15:43:41,415][105692] Updated weights for policy 0, policy_version 49676 (0.0008) [2023-12-26 15:43:41,827][105620] Updated weights for policy 1, policy_version 50003 (0.0007) [2023-12-26 15:43:41,893][105620] Updated weights for policy 1, policy_version 50013 (0.0007) [2023-12-26 15:43:41,953][105620] Updated weights for policy 1, policy_version 50023 (0.0008) [2023-12-26 15:43:42,126][105692] Updated weights for policy 0, policy_version 49686 (0.0007) [2023-12-26 15:43:42,191][105692] Updated weights for policy 0, policy_version 49696 (0.0010) [2023-12-26 15:43:42,256][105692] Updated weights for policy 0, policy_version 49706 (0.0007) [2023-12-26 15:43:42,679][105620] Updated weights for policy 1, policy_version 50033 (0.0009) [2023-12-26 15:43:42,735][105620] Updated weights for policy 1, policy_version 50043 (0.0010) [2023-12-26 15:43:42,783][105620] Updated weights for policy 1, policy_version 50053 (0.0010) [2023-12-26 15:43:42,839][105620] Updated weights for policy 1, policy_version 50063 (0.0010) [2023-12-26 15:43:42,948][105692] Updated weights for policy 0, policy_version 49716 (0.0009) [2023-12-26 15:43:43,015][105692] Updated weights for policy 0, policy_version 49726 (0.0011) [2023-12-26 15:43:43,074][105692] Updated weights for policy 0, policy_version 49736 (0.0010) [2023-12-26 15:43:43,488][105620] Updated weights for policy 1, policy_version 50073 (0.0006) [2023-12-26 15:43:43,536][105620] Updated weights for policy 1, policy_version 50083 (0.0005) [2023-12-26 15:43:43,604][105620] Updated weights for policy 1, policy_version 50093 (0.0005) [2023-12-26 15:43:43,772][105692] Updated weights for policy 0, policy_version 49746 (0.0009) [2023-12-26 15:43:43,818][105692] Updated weights for policy 0, policy_version 49756 (0.0005) [2023-12-26 15:43:43,885][105692] Updated weights for policy 0, policy_version 49766 (0.0006) [2023-12-26 15:43:43,952][105692] Updated weights for policy 0, policy_version 49776 (0.0005) [2023-12-26 15:43:44,209][105620] Updated weights for policy 1, policy_version 50103 (0.0008) [2023-12-26 15:43:44,270][105620] Updated weights for policy 1, policy_version 50113 (0.0010) [2023-12-26 15:43:44,328][105620] Updated weights for policy 1, policy_version 50123 (0.0008) [2023-12-26 15:43:44,548][105692] Updated weights for policy 0, policy_version 49786 (0.0010) [2023-12-26 15:43:44,606][105692] Updated weights for policy 0, policy_version 49797 (0.0009) [2023-12-26 15:43:44,658][105692] Updated weights for policy 0, policy_version 49807 (0.0010) [2023-12-26 15:43:44,951][105620] Updated weights for policy 1, policy_version 50133 (0.0008) [2023-12-26 15:43:45,004][105620] Updated weights for policy 1, policy_version 50143 (0.0010) [2023-12-26 15:43:45,063][105620] Updated weights for policy 1, policy_version 50153 (0.0008) [2023-12-26 15:43:45,396][105692] Updated weights for policy 0, policy_version 49817 (0.0011) [2023-12-26 15:43:45,455][105692] Updated weights for policy 0, policy_version 49827 (0.0011) [2023-12-26 15:43:45,511][105692] Updated weights for policy 0, policy_version 49837 (0.0011) [2023-12-26 15:43:45,727][105620] Updated weights for policy 1, policy_version 50163 (0.0008) [2023-12-26 15:43:45,782][105620] Updated weights for policy 1, policy_version 50173 (0.0010) [2023-12-26 15:43:45,843][105620] Updated weights for policy 1, policy_version 50183 (0.0010) [2023-12-26 15:43:46,062][104569] Fps is (10 sec: 19659.8, 60 sec: 19933.7, 300 sec: 19633.0). Total num frames: 25616384. Throughput: 0: 9868.4, 1: 10061.3. Samples: 25582416. Policy #0 lag: (min: 23.0, avg: 46.4, max: 48.0) [2023-12-26 15:43:46,064][104569] Avg episode reward: [(0, '8727.171'), (1, '9171.989')] [2023-12-26 15:43:46,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000050192_12853248.pth... [2023-12-26 15:43:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000049008_12550144.pth [2023-12-26 15:43:46,098][105692] Updated weights for policy 0, policy_version 49847 (0.0007) [2023-12-26 15:43:46,154][105692] Updated weights for policy 0, policy_version 49857 (0.0005) [2023-12-26 15:43:46,202][105692] Updated weights for policy 0, policy_version 49867 (0.0005) [2023-12-26 15:43:46,224][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000049872_12771328.pth... [2023-12-26 15:43:46,227][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000048688_12468224.pth [2023-12-26 15:43:46,590][105620] Updated weights for policy 1, policy_version 50193 (0.0010) [2023-12-26 15:43:46,637][105620] Updated weights for policy 1, policy_version 50203 (0.0010) [2023-12-26 15:43:46,681][105620] Updated weights for policy 1, policy_version 50213 (0.0010) [2023-12-26 15:43:46,729][105620] Updated weights for policy 1, policy_version 50223 (0.0010) [2023-12-26 15:43:46,818][105692] Updated weights for policy 0, policy_version 49877 (0.0007) [2023-12-26 15:43:46,871][105692] Updated weights for policy 0, policy_version 49887 (0.0009) [2023-12-26 15:43:46,923][105692] Updated weights for policy 0, policy_version 49897 (0.0009) [2023-12-26 15:43:47,461][105620] Updated weights for policy 1, policy_version 50234 (0.0010) [2023-12-26 15:43:47,515][105620] Updated weights for policy 1, policy_version 50245 (0.0010) [2023-12-26 15:43:47,562][105692] Updated weights for policy 0, policy_version 49907 (0.0005) [2023-12-26 15:43:47,570][105620] Updated weights for policy 1, policy_version 50255 (0.0008) [2023-12-26 15:43:47,632][105692] Updated weights for policy 0, policy_version 49917 (0.0006) [2023-12-26 15:43:47,692][105692] Updated weights for policy 0, policy_version 49927 (0.0010) [2023-12-26 15:43:48,250][105620] Updated weights for policy 1, policy_version 50265 (0.0010) [2023-12-26 15:43:48,304][105620] Updated weights for policy 1, policy_version 50275 (0.0010) [2023-12-26 15:43:48,324][105692] Updated weights for policy 0, policy_version 49937 (0.0006) [2023-12-26 15:43:48,366][105620] Updated weights for policy 1, policy_version 50285 (0.0008) [2023-12-26 15:43:48,387][105692] Updated weights for policy 0, policy_version 49947 (0.0008) [2023-12-26 15:43:48,442][105692] Updated weights for policy 0, policy_version 49957 (0.0008) [2023-12-26 15:43:48,508][105692] Updated weights for policy 0, policy_version 49967 (0.0008) [2023-12-26 15:43:49,117][105620] Updated weights for policy 1, policy_version 50295 (0.0009) [2023-12-26 15:43:49,179][105620] Updated weights for policy 1, policy_version 50305 (0.0010) [2023-12-26 15:43:49,244][105620] Updated weights for policy 1, policy_version 50315 (0.0011) [2023-12-26 15:43:49,277][105692] Updated weights for policy 0, policy_version 49977 (0.0007) [2023-12-26 15:43:49,343][105692] Updated weights for policy 0, policy_version 49987 (0.0008) [2023-12-26 15:43:49,405][105692] Updated weights for policy 0, policy_version 49997 (0.0008) [2023-12-26 15:43:49,994][105620] Updated weights for policy 1, policy_version 50325 (0.0008) [2023-12-26 15:43:50,061][105620] Updated weights for policy 1, policy_version 50335 (0.0005) [2023-12-26 15:43:50,130][105620] Updated weights for policy 1, policy_version 50345 (0.0005) [2023-12-26 15:43:50,170][105692] Updated weights for policy 0, policy_version 50007 (0.0007) [2023-12-26 15:43:50,225][105692] Updated weights for policy 0, policy_version 50017 (0.0009) [2023-12-26 15:43:50,276][105692] Updated weights for policy 0, policy_version 50027 (0.0009) [2023-12-26 15:43:50,743][105620] Updated weights for policy 1, policy_version 50355 (0.0006) [2023-12-26 15:43:50,807][105620] Updated weights for policy 1, policy_version 50365 (0.0007) [2023-12-26 15:43:50,861][105620] Updated weights for policy 1, policy_version 50375 (0.0010) [2023-12-26 15:43:50,934][105692] Updated weights for policy 0, policy_version 50037 (0.0008) [2023-12-26 15:43:51,000][105692] Updated weights for policy 0, policy_version 50047 (0.0005) [2023-12-26 15:43:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 20070.4, 300 sec: 19633.0). Total num frames: 25714688. Throughput: 0: 9975.6, 1: 10153.9. Samples: 25704936. Policy #0 lag: (min: 23.0, avg: 46.4, max: 48.0) [2023-12-26 15:43:51,063][104569] Avg episode reward: [(0, '8785.559'), (1, '9352.898')] [2023-12-26 15:43:51,063][105692] Updated weights for policy 0, policy_version 50057 (0.0007) [2023-12-26 15:43:51,675][105692] Updated weights for policy 0, policy_version 50067 (0.0007) [2023-12-26 15:43:51,714][105620] Updated weights for policy 1, policy_version 50385 (0.0009) [2023-12-26 15:43:51,741][105692] Updated weights for policy 0, policy_version 50077 (0.0009) [2023-12-26 15:43:51,779][105620] Updated weights for policy 1, policy_version 50395 (0.0007) [2023-12-26 15:43:51,810][105692] Updated weights for policy 0, policy_version 50087 (0.0007) [2023-12-26 15:43:51,831][105620] Updated weights for policy 1, policy_version 50405 (0.0007) [2023-12-26 15:43:51,893][105620] Updated weights for policy 1, policy_version 50415 (0.0006) [2023-12-26 15:43:52,501][105692] Updated weights for policy 0, policy_version 50097 (0.0006) [2023-12-26 15:43:52,565][105692] Updated weights for policy 0, policy_version 50107 (0.0008) [2023-12-26 15:43:52,587][105620] Updated weights for policy 1, policy_version 50425 (0.0008) [2023-12-26 15:43:52,621][105692] Updated weights for policy 0, policy_version 50117 (0.0006) [2023-12-26 15:43:52,643][105620] Updated weights for policy 1, policy_version 50435 (0.0008) [2023-12-26 15:43:52,684][105692] Updated weights for policy 0, policy_version 50127 (0.0006) [2023-12-26 15:43:52,707][105620] Updated weights for policy 1, policy_version 50445 (0.0008) [2023-12-26 15:43:53,296][105692] Updated weights for policy 0, policy_version 50137 (0.0007) [2023-12-26 15:43:53,349][105692] Updated weights for policy 0, policy_version 50147 (0.0009) [2023-12-26 15:43:53,402][105692] Updated weights for policy 0, policy_version 50157 (0.0008) [2023-12-26 15:43:53,522][105620] Updated weights for policy 1, policy_version 50455 (0.0007) [2023-12-26 15:43:53,578][105620] Updated weights for policy 1, policy_version 50465 (0.0006) [2023-12-26 15:43:53,647][105620] Updated weights for policy 1, policy_version 50475 (0.0005) [2023-12-26 15:43:54,156][105692] Updated weights for policy 0, policy_version 50167 (0.0007) [2023-12-26 15:43:54,215][105692] Updated weights for policy 0, policy_version 50177 (0.0007) [2023-12-26 15:43:54,217][105620] Updated weights for policy 1, policy_version 50485 (0.0005) [2023-12-26 15:43:54,271][105620] Updated weights for policy 1, policy_version 50495 (0.0007) [2023-12-26 15:43:54,273][105692] Updated weights for policy 0, policy_version 50187 (0.0007) [2023-12-26 15:43:54,330][105620] Updated weights for policy 1, policy_version 50505 (0.0006) [2023-12-26 15:43:54,968][105692] Updated weights for policy 0, policy_version 50197 (0.0009) [2023-12-26 15:43:55,023][105692] Updated weights for policy 0, policy_version 50207 (0.0009) [2023-12-26 15:43:55,040][105620] Updated weights for policy 1, policy_version 50515 (0.0007) [2023-12-26 15:43:55,084][105692] Updated weights for policy 0, policy_version 50217 (0.0006) [2023-12-26 15:43:55,095][105620] Updated weights for policy 1, policy_version 50525 (0.0008) [2023-12-26 15:43:55,149][105620] Updated weights for policy 1, policy_version 50535 (0.0009) [2023-12-26 15:43:55,652][105692] Updated weights for policy 0, policy_version 50227 (0.0008) [2023-12-26 15:43:55,705][105692] Updated weights for policy 0, policy_version 50237 (0.0008) [2023-12-26 15:43:55,753][105692] Updated weights for policy 0, policy_version 50247 (0.0007) [2023-12-26 15:43:55,989][105620] Updated weights for policy 1, policy_version 50545 (0.0009) [2023-12-26 15:43:56,047][105620] Updated weights for policy 1, policy_version 50555 (0.0008) [2023-12-26 15:43:56,062][104569] Fps is (10 sec: 19661.7, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 25812992. Throughput: 0: 10037.7, 1: 10094.8. Samples: 25823660. Policy #0 lag: (min: 23.0, avg: 46.4, max: 48.0) [2023-12-26 15:43:56,063][104569] Avg episode reward: [(0, '7722.788'), (1, '9352.204')] [2023-12-26 15:43:56,107][105620] Updated weights for policy 1, policy_version 50565 (0.0008) [2023-12-26 15:43:56,157][105620] Updated weights for policy 1, policy_version 50575 (0.0008) [2023-12-26 15:43:56,392][105692] Updated weights for policy 0, policy_version 50257 (0.0007) [2023-12-26 15:43:56,449][105692] Updated weights for policy 0, policy_version 50267 (0.0007) [2023-12-26 15:43:56,509][105692] Updated weights for policy 0, policy_version 50277 (0.0010) [2023-12-26 15:43:56,567][105692] Updated weights for policy 0, policy_version 50287 (0.0007) [2023-12-26 15:43:56,993][105620] Updated weights for policy 1, policy_version 50585 (0.0007) [2023-12-26 15:43:57,044][105620] Updated weights for policy 1, policy_version 50595 (0.0008) [2023-12-26 15:43:57,088][105620] Updated weights for policy 1, policy_version 50605 (0.0008) [2023-12-26 15:43:57,178][105692] Updated weights for policy 0, policy_version 50297 (0.0010) [2023-12-26 15:43:57,235][105692] Updated weights for policy 0, policy_version 50307 (0.0010) [2023-12-26 15:43:57,288][105692] Updated weights for policy 0, policy_version 50317 (0.0010) [2023-12-26 15:43:57,903][105620] Updated weights for policy 1, policy_version 50615 (0.0007) [2023-12-26 15:43:57,935][105692] Updated weights for policy 0, policy_version 50327 (0.0009) [2023-12-26 15:43:57,960][105620] Updated weights for policy 1, policy_version 50625 (0.0006) [2023-12-26 15:43:57,979][105692] Updated weights for policy 0, policy_version 50337 (0.0010) [2023-12-26 15:43:58,016][105620] Updated weights for policy 1, policy_version 50635 (0.0006) [2023-12-26 15:43:58,026][105692] Updated weights for policy 0, policy_version 50347 (0.0010) [2023-12-26 15:43:58,847][105692] Updated weights for policy 0, policy_version 50357 (0.0009) [2023-12-26 15:43:58,850][105620] Updated weights for policy 1, policy_version 50645 (0.0006) [2023-12-26 15:43:58,908][105692] Updated weights for policy 0, policy_version 50367 (0.0008) [2023-12-26 15:43:58,914][105620] Updated weights for policy 1, policy_version 50655 (0.0010) [2023-12-26 15:43:58,967][105692] Updated weights for policy 0, policy_version 50377 (0.0009) [2023-12-26 15:43:58,973][105620] Updated weights for policy 1, policy_version 50665 (0.0008) [2023-12-26 15:43:59,670][105692] Updated weights for policy 0, policy_version 50387 (0.0006) [2023-12-26 15:43:59,738][105692] Updated weights for policy 0, policy_version 50397 (0.0005) [2023-12-26 15:43:59,788][105692] Updated weights for policy 0, policy_version 50407 (0.0005) [2023-12-26 15:43:59,791][105620] Updated weights for policy 1, policy_version 50675 (0.0007) [2023-12-26 15:43:59,855][105620] Updated weights for policy 1, policy_version 50685 (0.0009) [2023-12-26 15:43:59,917][105620] Updated weights for policy 1, policy_version 50695 (0.0009) [2023-12-26 15:44:00,495][105692] Updated weights for policy 0, policy_version 50417 (0.0007) [2023-12-26 15:44:00,554][105692] Updated weights for policy 0, policy_version 50427 (0.0009) [2023-12-26 15:44:00,571][105620] Updated weights for policy 1, policy_version 50705 (0.0009) [2023-12-26 15:44:00,612][105692] Updated weights for policy 0, policy_version 50437 (0.0008) [2023-12-26 15:44:00,622][105620] Updated weights for policy 1, policy_version 50715 (0.0008) [2023-12-26 15:44:00,663][105692] Updated weights for policy 0, policy_version 50447 (0.0005) [2023-12-26 15:44:00,673][105620] Updated weights for policy 1, policy_version 50725 (0.0010) [2023-12-26 15:44:00,720][105620] Updated weights for policy 1, policy_version 50735 (0.0010) [2023-12-26 15:44:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 25911296. Throughput: 0: 10083.8, 1: 9984.1. Samples: 25881000. Policy #0 lag: (min: 23.0, avg: 46.4, max: 48.0) [2023-12-26 15:44:01,063][104569] Avg episode reward: [(0, '8409.737'), (1, '9353.809')] [2023-12-26 15:44:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000050448_12918784.pth... [2023-12-26 15:44:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000050736_12992512.pth... [2023-12-26 15:44:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000049616_12705792.pth [2023-12-26 15:44:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000049264_12615680.pth [2023-12-26 15:44:01,421][105620] Updated weights for policy 1, policy_version 50747 (0.0009) [2023-12-26 15:44:01,484][105620] Updated weights for policy 1, policy_version 50757 (0.0010) [2023-12-26 15:44:01,499][105692] Updated weights for policy 0, policy_version 50457 (0.0005) [2023-12-26 15:44:01,541][105620] Updated weights for policy 1, policy_version 50767 (0.0011) [2023-12-26 15:44:01,555][105692] Updated weights for policy 0, policy_version 50467 (0.0006) [2023-12-26 15:44:01,607][105692] Updated weights for policy 0, policy_version 50477 (0.0007) [2023-12-26 15:44:02,301][105620] Updated weights for policy 1, policy_version 50777 (0.0009) [2023-12-26 15:44:02,370][105620] Updated weights for policy 1, policy_version 50787 (0.0007) [2023-12-26 15:44:02,376][105692] Updated weights for policy 0, policy_version 50487 (0.0008) [2023-12-26 15:44:02,422][105620] Updated weights for policy 1, policy_version 50797 (0.0007) [2023-12-26 15:44:02,432][105692] Updated weights for policy 0, policy_version 50497 (0.0007) [2023-12-26 15:44:02,482][105692] Updated weights for policy 0, policy_version 50507 (0.0009) [2023-12-26 15:44:03,088][105620] Updated weights for policy 1, policy_version 50807 (0.0007) [2023-12-26 15:44:03,131][105620] Updated weights for policy 1, policy_version 50817 (0.0005) [2023-12-26 15:44:03,176][105620] Updated weights for policy 1, policy_version 50827 (0.0005) [2023-12-26 15:44:03,311][105692] Updated weights for policy 0, policy_version 50517 (0.0009) [2023-12-26 15:44:03,369][105692] Updated weights for policy 0, policy_version 50527 (0.0009) [2023-12-26 15:44:03,429][105692] Updated weights for policy 0, policy_version 50537 (0.0009) [2023-12-26 15:44:03,745][105620] Updated weights for policy 1, policy_version 50837 (0.0007) [2023-12-26 15:44:03,798][105620] Updated weights for policy 1, policy_version 50847 (0.0010) [2023-12-26 15:44:03,852][105620] Updated weights for policy 1, policy_version 50857 (0.0010) [2023-12-26 15:44:04,103][105692] Updated weights for policy 0, policy_version 50548 (0.0009) [2023-12-26 15:44:04,168][105692] Updated weights for policy 0, policy_version 50558 (0.0009) [2023-12-26 15:44:04,232][105692] Updated weights for policy 0, policy_version 50568 (0.0008) [2023-12-26 15:44:04,607][105620] Updated weights for policy 1, policy_version 50867 (0.0010) [2023-12-26 15:44:04,667][105620] Updated weights for policy 1, policy_version 50877 (0.0009) [2023-12-26 15:44:04,729][105620] Updated weights for policy 1, policy_version 50887 (0.0009) [2023-12-26 15:44:04,990][105692] Updated weights for policy 0, policy_version 50578 (0.0007) [2023-12-26 15:44:05,046][105692] Updated weights for policy 0, policy_version 50588 (0.0005) [2023-12-26 15:44:05,097][105692] Updated weights for policy 0, policy_version 50598 (0.0005) [2023-12-26 15:44:05,142][105692] Updated weights for policy 0, policy_version 50608 (0.0005) [2023-12-26 15:44:05,470][105620] Updated weights for policy 1, policy_version 50897 (0.0008) [2023-12-26 15:44:05,525][105620] Updated weights for policy 1, policy_version 50907 (0.0005) [2023-12-26 15:44:05,587][105620] Updated weights for policy 1, policy_version 50917 (0.0005) [2023-12-26 15:44:05,656][105620] Updated weights for policy 1, policy_version 50927 (0.0005) [2023-12-26 15:44:05,775][105692] Updated weights for policy 0, policy_version 50618 (0.0010) [2023-12-26 15:44:05,830][105692] Updated weights for policy 0, policy_version 50628 (0.0010) [2023-12-26 15:44:05,888][105692] Updated weights for policy 0, policy_version 50638 (0.0011) [2023-12-26 15:44:06,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 26009600. Throughput: 0: 10074.7, 1: 9923.9. Samples: 25995512. Policy #0 lag: (min: 23.0, avg: 46.4, max: 48.0) [2023-12-26 15:44:06,063][104569] Avg episode reward: [(0, '9098.918'), (1, '9099.724')] [2023-12-26 15:44:06,164][105620] Updated weights for policy 1, policy_version 50937 (0.0010) [2023-12-26 15:44:06,226][105620] Updated weights for policy 1, policy_version 50947 (0.0011) [2023-12-26 15:44:06,297][105620] Updated weights for policy 1, policy_version 50957 (0.0011) [2023-12-26 15:44:06,572][105692] Updated weights for policy 0, policy_version 50648 (0.0007) [2023-12-26 15:44:06,620][105692] Updated weights for policy 0, policy_version 50658 (0.0008) [2023-12-26 15:44:06,665][105692] Updated weights for policy 0, policy_version 50668 (0.0008) [2023-12-26 15:44:06,919][105620] Updated weights for policy 1, policy_version 50967 (0.0010) [2023-12-26 15:44:06,977][105620] Updated weights for policy 1, policy_version 50977 (0.0010) [2023-12-26 15:44:07,028][105620] Updated weights for policy 1, policy_version 50987 (0.0010) [2023-12-26 15:44:07,386][105692] Updated weights for policy 0, policy_version 50678 (0.0007) [2023-12-26 15:44:07,446][105692] Updated weights for policy 0, policy_version 50688 (0.0006) [2023-12-26 15:44:07,497][105692] Updated weights for policy 0, policy_version 50698 (0.0005) [2023-12-26 15:44:07,696][105620] Updated weights for policy 1, policy_version 50997 (0.0008) [2023-12-26 15:44:07,747][105620] Updated weights for policy 1, policy_version 51007 (0.0006) [2023-12-26 15:44:07,804][105620] Updated weights for policy 1, policy_version 51017 (0.0009) [2023-12-26 15:44:08,211][105692] Updated weights for policy 0, policy_version 50708 (0.0009) [2023-12-26 15:44:08,269][105692] Updated weights for policy 0, policy_version 50718 (0.0010) [2023-12-26 15:44:08,317][105692] Updated weights for policy 0, policy_version 50728 (0.0009) [2023-12-26 15:44:08,452][105620] Updated weights for policy 1, policy_version 51027 (0.0007) [2023-12-26 15:44:08,513][105620] Updated weights for policy 1, policy_version 51037 (0.0007) [2023-12-26 15:44:08,575][105620] Updated weights for policy 1, policy_version 51047 (0.0009) [2023-12-26 15:44:09,043][105692] Updated weights for policy 0, policy_version 50738 (0.0012) [2023-12-26 15:44:09,099][105692] Updated weights for policy 0, policy_version 50748 (0.0006) [2023-12-26 15:44:09,161][105692] Updated weights for policy 0, policy_version 50758 (0.0008) [2023-12-26 15:44:09,214][105692] Updated weights for policy 0, policy_version 50768 (0.0008) [2023-12-26 15:44:09,343][105620] Updated weights for policy 1, policy_version 51057 (0.0009) [2023-12-26 15:44:09,414][105620] Updated weights for policy 1, policy_version 51067 (0.0009) [2023-12-26 15:44:09,475][105620] Updated weights for policy 1, policy_version 51077 (0.0009) [2023-12-26 15:44:09,538][105620] Updated weights for policy 1, policy_version 51087 (0.0009) [2023-12-26 15:44:09,995][105692] Updated weights for policy 0, policy_version 50778 (0.0008) [2023-12-26 15:44:10,054][105692] Updated weights for policy 0, policy_version 50788 (0.0009) [2023-12-26 15:44:10,116][105692] Updated weights for policy 0, policy_version 50798 (0.0009) [2023-12-26 15:44:10,315][105620] Updated weights for policy 1, policy_version 51097 (0.0009) [2023-12-26 15:44:10,378][105620] Updated weights for policy 1, policy_version 51107 (0.0008) [2023-12-26 15:44:10,447][105620] Updated weights for policy 1, policy_version 51117 (0.0006) [2023-12-26 15:44:10,886][105692] Updated weights for policy 0, policy_version 50808 (0.0009) [2023-12-26 15:44:10,941][105692] Updated weights for policy 0, policy_version 50818 (0.0010) [2023-12-26 15:44:10,995][105692] Updated weights for policy 0, policy_version 50829 (0.0010) [2023-12-26 15:44:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 26107904. Throughput: 0: 10097.5, 1: 9893.9. Samples: 26115592. Policy #0 lag: (min: 23.0, avg: 46.4, max: 48.0) [2023-12-26 15:44:11,062][104569] Avg episode reward: [(0, '8931.423'), (1, '8753.084')] [2023-12-26 15:44:11,082][105620] Updated weights for policy 1, policy_version 51127 (0.0008) [2023-12-26 15:44:11,146][105620] Updated weights for policy 1, policy_version 51137 (0.0007) [2023-12-26 15:44:11,195][105620] Updated weights for policy 1, policy_version 51147 (0.0006) [2023-12-26 15:44:11,827][105692] Updated weights for policy 0, policy_version 50839 (0.0007) [2023-12-26 15:44:11,892][105692] Updated weights for policy 0, policy_version 50849 (0.0006) [2023-12-26 15:44:11,938][105620] Updated weights for policy 1, policy_version 51157 (0.0007) [2023-12-26 15:44:11,958][105692] Updated weights for policy 0, policy_version 50859 (0.0007) [2023-12-26 15:44:12,001][105620] Updated weights for policy 1, policy_version 51167 (0.0009) [2023-12-26 15:44:12,061][105620] Updated weights for policy 1, policy_version 51177 (0.0008) [2023-12-26 15:44:12,604][105692] Updated weights for policy 0, policy_version 50869 (0.0008) [2023-12-26 15:44:12,664][105692] Updated weights for policy 0, policy_version 50879 (0.0008) [2023-12-26 15:44:12,731][105692] Updated weights for policy 0, policy_version 50889 (0.0005) [2023-12-26 15:44:12,778][105620] Updated weights for policy 1, policy_version 51187 (0.0010) [2023-12-26 15:44:12,838][105620] Updated weights for policy 1, policy_version 51197 (0.0011) [2023-12-26 15:44:12,902][105620] Updated weights for policy 1, policy_version 51207 (0.0011) [2023-12-26 15:44:13,482][105620] Updated weights for policy 1, policy_version 51217 (0.0010) [2023-12-26 15:44:13,523][105692] Updated weights for policy 0, policy_version 50899 (0.0005) [2023-12-26 15:44:13,546][105620] Updated weights for policy 1, policy_version 51227 (0.0007) [2023-12-26 15:44:13,582][105692] Updated weights for policy 0, policy_version 50909 (0.0005) [2023-12-26 15:44:13,601][105620] Updated weights for policy 1, policy_version 51237 (0.0006) [2023-12-26 15:44:13,641][105692] Updated weights for policy 0, policy_version 50919 (0.0009) [2023-12-26 15:44:13,660][105620] Updated weights for policy 1, policy_version 51247 (0.0005) [2023-12-26 15:44:14,199][105620] Updated weights for policy 1, policy_version 51257 (0.0005) [2023-12-26 15:44:14,248][105620] Updated weights for policy 1, policy_version 51267 (0.0006) [2023-12-26 15:44:14,301][105620] Updated weights for policy 1, policy_version 51277 (0.0005) [2023-12-26 15:44:14,320][105692] Updated weights for policy 0, policy_version 50929 (0.0010) [2023-12-26 15:44:14,379][105692] Updated weights for policy 0, policy_version 50939 (0.0010) [2023-12-26 15:44:14,438][105692] Updated weights for policy 0, policy_version 50949 (0.0005) [2023-12-26 15:44:14,497][105692] Updated weights for policy 0, policy_version 50959 (0.0005) [2023-12-26 15:44:14,933][105620] Updated weights for policy 1, policy_version 51287 (0.0008) [2023-12-26 15:44:14,993][105620] Updated weights for policy 1, policy_version 51297 (0.0009) [2023-12-26 15:44:15,055][105620] Updated weights for policy 1, policy_version 51307 (0.0010) [2023-12-26 15:44:15,170][105692] Updated weights for policy 0, policy_version 50969 (0.0009) [2023-12-26 15:44:15,233][105692] Updated weights for policy 0, policy_version 50979 (0.0008) [2023-12-26 15:44:15,291][105692] Updated weights for policy 0, policy_version 50989 (0.0006) [2023-12-26 15:44:15,806][105620] Updated weights for policy 1, policy_version 51317 (0.0010) [2023-12-26 15:44:15,858][105620] Updated weights for policy 1, policy_version 51327 (0.0010) [2023-12-26 15:44:15,909][105620] Updated weights for policy 1, policy_version 51337 (0.0010) [2023-12-26 15:44:15,999][105692] Updated weights for policy 0, policy_version 50999 (0.0005) [2023-12-26 15:44:16,059][105692] Updated weights for policy 0, policy_version 51009 (0.0005) [2023-12-26 15:44:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 26206208. Throughput: 0: 9984.4, 1: 9864.4. Samples: 26174776. Policy #0 lag: (min: 23.0, avg: 46.4, max: 48.0) [2023-12-26 15:44:16,062][104569] Avg episode reward: [(0, '9105.276'), (1, '8921.798')] [2023-12-26 15:44:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000051344_13148160.pth... [2023-12-26 15:44:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000050192_12853248.pth [2023-12-26 15:44:16,116][105692] Updated weights for policy 0, policy_version 51019 (0.0006) [2023-12-26 15:44:16,148][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000051024_13066240.pth... [2023-12-26 15:44:16,153][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000049872_12771328.pth [2023-12-26 15:44:16,540][105620] Updated weights for policy 1, policy_version 51347 (0.0009) [2023-12-26 15:44:16,592][105620] Updated weights for policy 1, policy_version 51357 (0.0005) [2023-12-26 15:44:16,646][105620] Updated weights for policy 1, policy_version 51367 (0.0005) [2023-12-26 15:44:16,717][105692] Updated weights for policy 0, policy_version 51029 (0.0006) [2023-12-26 15:44:16,773][105692] Updated weights for policy 0, policy_version 51039 (0.0006) [2023-12-26 15:44:16,835][105692] Updated weights for policy 0, policy_version 51049 (0.0006) [2023-12-26 15:44:17,274][105620] Updated weights for policy 1, policy_version 51377 (0.0007) [2023-12-26 15:44:17,339][105620] Updated weights for policy 1, policy_version 51387 (0.0007) [2023-12-26 15:44:17,397][105620] Updated weights for policy 1, policy_version 51397 (0.0005) [2023-12-26 15:44:17,444][105620] Updated weights for policy 1, policy_version 51407 (0.0007) [2023-12-26 15:44:17,511][105692] Updated weights for policy 0, policy_version 51059 (0.0007) [2023-12-26 15:44:17,575][105692] Updated weights for policy 0, policy_version 51069 (0.0009) [2023-12-26 15:44:17,641][105692] Updated weights for policy 0, policy_version 51079 (0.0010) [2023-12-26 15:44:18,006][105620] Updated weights for policy 1, policy_version 51417 (0.0006) [2023-12-26 15:44:18,058][105620] Updated weights for policy 1, policy_version 51427 (0.0005) [2023-12-26 15:44:18,104][105620] Updated weights for policy 1, policy_version 51437 (0.0005) [2023-12-26 15:44:18,358][105692] Updated weights for policy 0, policy_version 51089 (0.0010) [2023-12-26 15:44:18,421][105692] Updated weights for policy 0, policy_version 51099 (0.0007) [2023-12-26 15:44:18,487][105692] Updated weights for policy 0, policy_version 51109 (0.0010) [2023-12-26 15:44:18,552][105692] Updated weights for policy 0, policy_version 51119 (0.0010) [2023-12-26 15:44:18,676][105620] Updated weights for policy 1, policy_version 51447 (0.0006) [2023-12-26 15:44:18,743][105620] Updated weights for policy 1, policy_version 51457 (0.0005) [2023-12-26 15:44:18,815][105620] Updated weights for policy 1, policy_version 51467 (0.0006) [2023-12-26 15:44:19,278][105692] Updated weights for policy 0, policy_version 51129 (0.0008) [2023-12-26 15:44:19,339][105692] Updated weights for policy 0, policy_version 51139 (0.0006) [2023-12-26 15:44:19,404][105692] Updated weights for policy 0, policy_version 51149 (0.0007) [2023-12-26 15:44:19,446][105620] Updated weights for policy 1, policy_version 51477 (0.0007) [2023-12-26 15:44:19,514][105620] Updated weights for policy 1, policy_version 51487 (0.0008) [2023-12-26 15:44:19,569][105620] Updated weights for policy 1, policy_version 51497 (0.0008) [2023-12-26 15:44:20,140][105692] Updated weights for policy 0, policy_version 51159 (0.0009) [2023-12-26 15:44:20,203][105692] Updated weights for policy 0, policy_version 51169 (0.0008) [2023-12-26 15:44:20,266][105692] Updated weights for policy 0, policy_version 51179 (0.0009) [2023-12-26 15:44:20,301][105620] Updated weights for policy 1, policy_version 51507 (0.0007) [2023-12-26 15:44:20,363][105620] Updated weights for policy 1, policy_version 51517 (0.0009) [2023-12-26 15:44:20,433][105620] Updated weights for policy 1, policy_version 51527 (0.0005) [2023-12-26 15:44:21,055][105692] Updated weights for policy 0, policy_version 51189 (0.0008) [2023-12-26 15:44:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 20070.4, 300 sec: 19605.3). Total num frames: 26304512. Throughput: 0: 9906.4, 1: 9958.9. Samples: 26299756. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-26 15:44:21,062][104569] Avg episode reward: [(0, '1731.298'), (1, '9175.905')] [2023-12-26 15:44:21,088][105620] Updated weights for policy 1, policy_version 51537 (0.0005) [2023-12-26 15:44:21,118][105692] Updated weights for policy 0, policy_version 51199 (0.0007) [2023-12-26 15:44:21,158][105620] Updated weights for policy 1, policy_version 51547 (0.0009) [2023-12-26 15:44:21,183][105692] Updated weights for policy 0, policy_version 51209 (0.0007) [2023-12-26 15:44:21,222][105620] Updated weights for policy 1, policy_version 51557 (0.0011) [2023-12-26 15:44:21,291][105620] Updated weights for policy 1, policy_version 51567 (0.0011) [2023-12-26 15:44:21,954][105692] Updated weights for policy 0, policy_version 51219 (0.0007) [2023-12-26 15:44:21,983][105620] Updated weights for policy 1, policy_version 51577 (0.0006) [2023-12-26 15:44:22,013][105692] Updated weights for policy 0, policy_version 51229 (0.0008) [2023-12-26 15:44:22,043][105620] Updated weights for policy 1, policy_version 51587 (0.0007) [2023-12-26 15:44:22,073][105692] Updated weights for policy 0, policy_version 51239 (0.0008) [2023-12-26 15:44:22,098][105620] Updated weights for policy 1, policy_version 51597 (0.0007) [2023-12-26 15:44:22,690][105620] Updated weights for policy 1, policy_version 51607 (0.0008) [2023-12-26 15:44:22,748][105620] Updated weights for policy 1, policy_version 51617 (0.0009) [2023-12-26 15:44:22,803][105620] Updated weights for policy 1, policy_version 51627 (0.0009) [2023-12-26 15:44:22,906][105692] Updated weights for policy 0, policy_version 51249 (0.0010) [2023-12-26 15:44:22,973][105692] Updated weights for policy 0, policy_version 51259 (0.0009) [2023-12-26 15:44:23,038][105692] Updated weights for policy 0, policy_version 51269 (0.0009) [2023-12-26 15:44:23,093][105692] Updated weights for policy 0, policy_version 51279 (0.0010) [2023-12-26 15:44:23,588][105620] Updated weights for policy 1, policy_version 51637 (0.0009) [2023-12-26 15:44:23,641][105620] Updated weights for policy 1, policy_version 51647 (0.0009) [2023-12-26 15:44:23,685][105620] Updated weights for policy 1, policy_version 51657 (0.0008) [2023-12-26 15:44:23,795][105692] Updated weights for policy 0, policy_version 51289 (0.0006) [2023-12-26 15:44:23,856][105692] Updated weights for policy 0, policy_version 51299 (0.0006) [2023-12-26 15:44:23,904][105692] Updated weights for policy 0, policy_version 51309 (0.0009) [2023-12-26 15:44:24,495][105620] Updated weights for policy 1, policy_version 51667 (0.0008) [2023-12-26 15:44:24,550][105620] Updated weights for policy 1, policy_version 51677 (0.0008) [2023-12-26 15:44:24,558][105692] Updated weights for policy 0, policy_version 51319 (0.0007) [2023-12-26 15:44:24,603][105620] Updated weights for policy 1, policy_version 51687 (0.0007) [2023-12-26 15:44:24,614][105692] Updated weights for policy 0, policy_version 51329 (0.0006) [2023-12-26 15:44:24,674][105692] Updated weights for policy 0, policy_version 51339 (0.0006) [2023-12-26 15:44:25,232][105692] Updated weights for policy 0, policy_version 51349 (0.0006) [2023-12-26 15:44:25,291][105692] Updated weights for policy 0, policy_version 51359 (0.0010) [2023-12-26 15:44:25,309][105620] Updated weights for policy 1, policy_version 51697 (0.0009) [2023-12-26 15:44:25,352][105692] Updated weights for policy 0, policy_version 51369 (0.0008) [2023-12-26 15:44:25,377][105620] Updated weights for policy 1, policy_version 51707 (0.0010) [2023-12-26 15:44:25,435][105620] Updated weights for policy 1, policy_version 51717 (0.0010) [2023-12-26 15:44:25,493][105620] Updated weights for policy 1, policy_version 51727 (0.0010) [2023-12-26 15:44:26,011][105692] Updated weights for policy 0, policy_version 51379 (0.0006) [2023-12-26 15:44:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19933.8, 300 sec: 19633.0). Total num frames: 26402816. Throughput: 0: 9870.3, 1: 9963.5. Samples: 26416476. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-26 15:44:26,063][104569] Avg episode reward: [(0, '2389.539'), (1, '9172.493')] [2023-12-26 15:44:26,072][105692] Updated weights for policy 0, policy_version 51389 (0.0006) [2023-12-26 15:44:26,123][105692] Updated weights for policy 0, policy_version 51399 (0.0010) [2023-12-26 15:44:26,124][105620] Updated weights for policy 1, policy_version 51737 (0.0006) [2023-12-26 15:44:26,178][105620] Updated weights for policy 1, policy_version 51747 (0.0010) [2023-12-26 15:44:26,235][105620] Updated weights for policy 1, policy_version 51757 (0.0010) [2023-12-26 15:44:26,800][105692] Updated weights for policy 0, policy_version 51409 (0.0010) [2023-12-26 15:44:26,869][105692] Updated weights for policy 0, policy_version 51419 (0.0005) [2023-12-26 15:44:26,873][105620] Updated weights for policy 1, policy_version 51767 (0.0007) [2023-12-26 15:44:26,919][105620] Updated weights for policy 1, policy_version 51777 (0.0005) [2023-12-26 15:44:26,934][105692] Updated weights for policy 0, policy_version 51429 (0.0005) [2023-12-26 15:44:26,968][105620] Updated weights for policy 1, policy_version 51787 (0.0005) [2023-12-26 15:44:26,993][105692] Updated weights for policy 0, policy_version 51439 (0.0005) [2023-12-26 15:44:27,550][105692] Updated weights for policy 0, policy_version 51449 (0.0007) [2023-12-26 15:44:27,593][105692] Updated weights for policy 0, policy_version 51459 (0.0005) [2023-12-26 15:44:27,641][105692] Updated weights for policy 0, policy_version 51469 (0.0005) [2023-12-26 15:44:27,694][105620] Updated weights for policy 1, policy_version 51797 (0.0009) [2023-12-26 15:44:27,754][105620] Updated weights for policy 1, policy_version 51807 (0.0010) [2023-12-26 15:44:27,818][105620] Updated weights for policy 1, policy_version 51817 (0.0010) [2023-12-26 15:44:28,201][105692] Updated weights for policy 0, policy_version 51479 (0.0005) [2023-12-26 15:44:28,258][105692] Updated weights for policy 0, policy_version 51490 (0.0011) [2023-12-26 15:44:28,320][105692] Updated weights for policy 0, policy_version 51501 (0.0010) [2023-12-26 15:44:28,353][105620] Updated weights for policy 1, policy_version 51827 (0.0008) [2023-12-26 15:44:28,410][105620] Updated weights for policy 1, policy_version 51837 (0.0008) [2023-12-26 15:44:28,471][105620] Updated weights for policy 1, policy_version 51847 (0.0008) [2023-12-26 15:44:28,967][105692] Updated weights for policy 0, policy_version 51511 (0.0007) [2023-12-26 15:44:29,024][105692] Updated weights for policy 0, policy_version 51521 (0.0008) [2023-12-26 15:44:29,077][105692] Updated weights for policy 0, policy_version 51531 (0.0008) [2023-12-26 15:44:29,220][105620] Updated weights for policy 1, policy_version 51857 (0.0008) [2023-12-26 15:44:29,281][105620] Updated weights for policy 1, policy_version 51867 (0.0010) [2023-12-26 15:44:29,340][105620] Updated weights for policy 1, policy_version 51877 (0.0009) [2023-12-26 15:44:29,406][105620] Updated weights for policy 1, policy_version 51887 (0.0008) [2023-12-26 15:44:29,848][105692] Updated weights for policy 0, policy_version 51541 (0.0008) [2023-12-26 15:44:29,903][105692] Updated weights for policy 0, policy_version 51551 (0.0005) [2023-12-26 15:44:29,969][105692] Updated weights for policy 0, policy_version 51561 (0.0007) [2023-12-26 15:44:30,117][105620] Updated weights for policy 1, policy_version 51897 (0.0010) [2023-12-26 15:44:30,171][105620] Updated weights for policy 1, policy_version 51907 (0.0010) [2023-12-26 15:44:30,226][105620] Updated weights for policy 1, policy_version 51917 (0.0010) [2023-12-26 15:44:30,586][105692] Updated weights for policy 0, policy_version 51571 (0.0008) [2023-12-26 15:44:30,635][105692] Updated weights for policy 0, policy_version 51581 (0.0008) [2023-12-26 15:44:30,684][105692] Updated weights for policy 0, policy_version 51591 (0.0008) [2023-12-26 15:44:30,950][105620] Updated weights for policy 1, policy_version 51927 (0.0010) [2023-12-26 15:44:31,007][105620] Updated weights for policy 1, policy_version 51937 (0.0010) [2023-12-26 15:44:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 26509312. Throughput: 0: 9966.0, 1: 10006.3. Samples: 26481156. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-26 15:44:31,062][104569] Avg episode reward: [(0, '6525.467'), (1, '8480.372')] [2023-12-26 15:44:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000051600_13213696.pth... [2023-12-26 15:44:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000050448_12918784.pth [2023-12-26 15:44:31,075][105620] Updated weights for policy 1, policy_version 51947 (0.0009) [2023-12-26 15:44:31,102][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000051952_13303808.pth... [2023-12-26 15:44:31,106][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000050736_12992512.pth [2023-12-26 15:44:31,464][105692] Updated weights for policy 0, policy_version 51601 (0.0010) [2023-12-26 15:44:31,523][105692] Updated weights for policy 0, policy_version 51611 (0.0009) [2023-12-26 15:44:31,591][105692] Updated weights for policy 0, policy_version 51621 (0.0005) [2023-12-26 15:44:31,655][105692] Updated weights for policy 0, policy_version 51631 (0.0009) [2023-12-26 15:44:31,792][105620] Updated weights for policy 1, policy_version 51957 (0.0010) [2023-12-26 15:44:31,844][105620] Updated weights for policy 1, policy_version 51967 (0.0010) [2023-12-26 15:44:31,899][105620] Updated weights for policy 1, policy_version 51977 (0.0010) [2023-12-26 15:44:32,331][105692] Updated weights for policy 0, policy_version 51641 (0.0011) [2023-12-26 15:44:32,394][105692] Updated weights for policy 0, policy_version 51651 (0.0011) [2023-12-26 15:44:32,447][105692] Updated weights for policy 0, policy_version 51661 (0.0010) [2023-12-26 15:44:32,562][105620] Updated weights for policy 1, policy_version 51987 (0.0006) [2023-12-26 15:44:32,623][105620] Updated weights for policy 1, policy_version 51997 (0.0006) [2023-12-26 15:44:32,689][105620] Updated weights for policy 1, policy_version 52007 (0.0006) [2023-12-26 15:44:33,166][105692] Updated weights for policy 0, policy_version 51671 (0.0007) [2023-12-26 15:44:33,220][105620] Updated weights for policy 1, policy_version 52017 (0.0006) [2023-12-26 15:44:33,228][105692] Updated weights for policy 0, policy_version 51681 (0.0005) [2023-12-26 15:44:33,276][105620] Updated weights for policy 1, policy_version 52027 (0.0007) [2023-12-26 15:44:33,291][105692] Updated weights for policy 0, policy_version 51691 (0.0005) [2023-12-26 15:44:33,328][105620] Updated weights for policy 1, policy_version 52037 (0.0007) [2023-12-26 15:44:33,386][105620] Updated weights for policy 1, policy_version 52047 (0.0005) [2023-12-26 15:44:33,796][105692] Updated weights for policy 0, policy_version 51701 (0.0006) [2023-12-26 15:44:33,856][105692] Updated weights for policy 0, policy_version 51711 (0.0007) [2023-12-26 15:44:33,916][105692] Updated weights for policy 0, policy_version 51721 (0.0007) [2023-12-26 15:44:33,986][105620] Updated weights for policy 1, policy_version 52057 (0.0007) [2023-12-26 15:44:34,035][105620] Updated weights for policy 1, policy_version 52068 (0.0009) [2023-12-26 15:44:34,081][105620] Updated weights for policy 1, policy_version 52078 (0.0008) [2023-12-26 15:44:34,631][105692] Updated weights for policy 0, policy_version 51731 (0.0008) [2023-12-26 15:44:34,690][105692] Updated weights for policy 0, policy_version 51741 (0.0011) [2023-12-26 15:44:34,756][105692] Updated weights for policy 0, policy_version 51751 (0.0008) [2023-12-26 15:44:34,811][105620] Updated weights for policy 1, policy_version 52088 (0.0007) [2023-12-26 15:44:34,870][105620] Updated weights for policy 1, policy_version 52098 (0.0006) [2023-12-26 15:44:34,920][105620] Updated weights for policy 1, policy_version 52108 (0.0009) [2023-12-26 15:44:35,500][105692] Updated weights for policy 0, policy_version 51761 (0.0007) [2023-12-26 15:44:35,560][105692] Updated weights for policy 0, policy_version 51771 (0.0009) [2023-12-26 15:44:35,579][105620] Updated weights for policy 1, policy_version 52118 (0.0008) [2023-12-26 15:44:35,613][105692] Updated weights for policy 0, policy_version 51781 (0.0008) [2023-12-26 15:44:35,623][105620] Updated weights for policy 1, policy_version 52128 (0.0005) [2023-12-26 15:44:35,668][105692] Updated weights for policy 0, policy_version 51791 (0.0009) [2023-12-26 15:44:35,670][105620] Updated weights for policy 1, policy_version 52138 (0.0006) [2023-12-26 15:44:36,062][104569] Fps is (10 sec: 21299.7, 60 sec: 19933.9, 300 sec: 19716.3). Total num frames: 26615808. Throughput: 0: 9942.2, 1: 10043.9. Samples: 26604308. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-26 15:44:36,062][104569] Avg episode reward: [(0, '8499.225'), (1, '8477.725')] [2023-12-26 15:44:36,369][105620] Updated weights for policy 1, policy_version 52148 (0.0008) [2023-12-26 15:44:36,435][105620] Updated weights for policy 1, policy_version 52158 (0.0008) [2023-12-26 15:44:36,473][105692] Updated weights for policy 0, policy_version 51801 (0.0009) [2023-12-26 15:44:36,495][105620] Updated weights for policy 1, policy_version 52168 (0.0009) [2023-12-26 15:44:36,531][105585] KL-divergence is very high: 113.2670 [2023-12-26 15:44:36,539][105692] Updated weights for policy 0, policy_version 51811 (0.0011) [2023-12-26 15:44:36,587][105585] KL-divergence is very high: 158.8330 [2023-12-26 15:44:36,606][105692] Updated weights for policy 0, policy_version 51821 (0.0011) [2023-12-26 15:44:37,081][105620] Updated weights for policy 1, policy_version 52178 (0.0008) [2023-12-26 15:44:37,129][105620] Updated weights for policy 1, policy_version 52188 (0.0010) [2023-12-26 15:44:37,181][105620] Updated weights for policy 1, policy_version 52198 (0.0010) [2023-12-26 15:44:37,236][105620] Updated weights for policy 1, policy_version 52208 (0.0010) [2023-12-26 15:44:37,274][105692] Updated weights for policy 0, policy_version 51831 (0.0010) [2023-12-26 15:44:37,326][105692] Updated weights for policy 0, policy_version 51841 (0.0011) [2023-12-26 15:44:37,378][105692] Updated weights for policy 0, policy_version 51851 (0.0010) [2023-12-26 15:44:37,919][105620] Updated weights for policy 1, policy_version 52218 (0.0010) [2023-12-26 15:44:37,974][105620] Updated weights for policy 1, policy_version 52228 (0.0010) [2023-12-26 15:44:38,026][105620] Updated weights for policy 1, policy_version 52238 (0.0010) [2023-12-26 15:44:38,130][105692] Updated weights for policy 0, policy_version 51861 (0.0011) [2023-12-26 15:44:38,178][105692] Updated weights for policy 0, policy_version 51871 (0.0009) [2023-12-26 15:44:38,235][105692] Updated weights for policy 0, policy_version 51881 (0.0005) [2023-12-26 15:44:38,799][105620] Updated weights for policy 1, policy_version 52248 (0.0010) [2023-12-26 15:44:38,862][105620] Updated weights for policy 1, policy_version 52258 (0.0011) [2023-12-26 15:44:38,918][105620] Updated weights for policy 1, policy_version 52268 (0.0009) [2023-12-26 15:44:38,949][105692] Updated weights for policy 0, policy_version 51891 (0.0007) [2023-12-26 15:44:39,014][105692] Updated weights for policy 0, policy_version 51901 (0.0009) [2023-12-26 15:44:39,078][105692] Updated weights for policy 0, policy_version 51911 (0.0010) [2023-12-26 15:44:39,558][105620] Updated weights for policy 1, policy_version 52278 (0.0006) [2023-12-26 15:44:39,626][105620] Updated weights for policy 1, policy_version 52288 (0.0006) [2023-12-26 15:44:39,694][105620] Updated weights for policy 1, policy_version 52298 (0.0006) [2023-12-26 15:44:39,768][105692] Updated weights for policy 0, policy_version 51921 (0.0010) [2023-12-26 15:44:39,841][105692] Updated weights for policy 0, policy_version 51931 (0.0006) [2023-12-26 15:44:39,905][105692] Updated weights for policy 0, policy_version 51941 (0.0008) [2023-12-26 15:44:39,975][105692] Updated weights for policy 0, policy_version 51951 (0.0008) [2023-12-26 15:44:40,366][105620] Updated weights for policy 1, policy_version 52308 (0.0008) [2023-12-26 15:44:40,432][105620] Updated weights for policy 1, policy_version 52318 (0.0009) [2023-12-26 15:44:40,492][105620] Updated weights for policy 1, policy_version 52328 (0.0009) [2023-12-26 15:44:40,672][105692] Updated weights for policy 0, policy_version 51961 (0.0008) [2023-12-26 15:44:40,732][105692] Updated weights for policy 0, policy_version 51971 (0.0009) [2023-12-26 15:44:40,799][105692] Updated weights for policy 0, policy_version 51981 (0.0009) [2023-12-26 15:44:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 20070.4, 300 sec: 19744.1). Total num frames: 26714112. Throughput: 0: 9843.0, 1: 10142.5. Samples: 26723000. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-26 15:44:41,062][104569] Avg episode reward: [(0, '8590.363'), (1, '9169.186')] [2023-12-26 15:44:41,153][105620] Updated weights for policy 1, policy_version 52338 (0.0009) [2023-12-26 15:44:41,223][105620] Updated weights for policy 1, policy_version 52348 (0.0008) [2023-12-26 15:44:41,288][105620] Updated weights for policy 1, policy_version 52358 (0.0009) [2023-12-26 15:44:41,350][105620] Updated weights for policy 1, policy_version 52368 (0.0010) [2023-12-26 15:44:41,633][105692] Updated weights for policy 0, policy_version 51991 (0.0010) [2023-12-26 15:44:41,698][105692] Updated weights for policy 0, policy_version 52001 (0.0010) [2023-12-26 15:44:41,766][105692] Updated weights for policy 0, policy_version 52011 (0.0010) [2023-12-26 15:44:42,078][105620] Updated weights for policy 1, policy_version 52378 (0.0008) [2023-12-26 15:44:42,141][105620] Updated weights for policy 1, policy_version 52388 (0.0007) [2023-12-26 15:44:42,203][105620] Updated weights for policy 1, policy_version 52398 (0.0007) [2023-12-26 15:44:42,499][105692] Updated weights for policy 0, policy_version 52021 (0.0010) [2023-12-26 15:44:42,550][105692] Updated weights for policy 0, policy_version 52031 (0.0008) [2023-12-26 15:44:42,614][105692] Updated weights for policy 0, policy_version 52041 (0.0010) [2023-12-26 15:44:42,968][105620] Updated weights for policy 1, policy_version 52408 (0.0010) [2023-12-26 15:44:43,021][105620] Updated weights for policy 1, policy_version 52418 (0.0010) [2023-12-26 15:44:43,079][105620] Updated weights for policy 1, policy_version 52428 (0.0010) [2023-12-26 15:44:43,301][105692] Updated weights for policy 0, policy_version 52051 (0.0010) [2023-12-26 15:44:43,353][105692] Updated weights for policy 0, policy_version 52061 (0.0009) [2023-12-26 15:44:43,399][105692] Updated weights for policy 0, policy_version 52071 (0.0005) [2023-12-26 15:44:43,822][105620] Updated weights for policy 1, policy_version 52438 (0.0010) [2023-12-26 15:44:43,879][105620] Updated weights for policy 1, policy_version 52448 (0.0010) [2023-12-26 15:44:43,941][105620] Updated weights for policy 1, policy_version 52458 (0.0010) [2023-12-26 15:44:44,017][105692] Updated weights for policy 0, policy_version 52081 (0.0005) [2023-12-26 15:44:44,084][105692] Updated weights for policy 0, policy_version 52091 (0.0005) [2023-12-26 15:44:44,150][105692] Updated weights for policy 0, policy_version 52101 (0.0008) [2023-12-26 15:44:44,208][105692] Updated weights for policy 0, policy_version 52111 (0.0008) [2023-12-26 15:44:44,674][105620] Updated weights for policy 1, policy_version 52468 (0.0010) [2023-12-26 15:44:44,722][105620] Updated weights for policy 1, policy_version 52478 (0.0010) [2023-12-26 15:44:44,772][105620] Updated weights for policy 1, policy_version 52488 (0.0010) [2023-12-26 15:44:44,907][105692] Updated weights for policy 0, policy_version 52121 (0.0008) [2023-12-26 15:44:44,975][105692] Updated weights for policy 0, policy_version 52131 (0.0008) [2023-12-26 15:44:45,036][105692] Updated weights for policy 0, policy_version 52141 (0.0008) [2023-12-26 15:44:45,545][105620] Updated weights for policy 1, policy_version 52498 (0.0010) [2023-12-26 15:44:45,605][105620] Updated weights for policy 1, policy_version 52508 (0.0010) [2023-12-26 15:44:45,659][105620] Updated weights for policy 1, policy_version 52518 (0.0010) [2023-12-26 15:44:45,723][105620] Updated weights for policy 1, policy_version 52528 (0.0010) [2023-12-26 15:44:45,797][105692] Updated weights for policy 0, policy_version 52151 (0.0009) [2023-12-26 15:44:45,841][105692] Updated weights for policy 0, policy_version 52161 (0.0008) [2023-12-26 15:44:45,889][105692] Updated weights for policy 0, policy_version 52171 (0.0008) [2023-12-26 15:44:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19934.1, 300 sec: 19744.1). Total num frames: 26812416. Throughput: 0: 9759.1, 1: 10197.4. Samples: 26779040. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-26 15:44:46,063][104569] Avg episode reward: [(0, '8933.989'), (1, '9174.207')] [2023-12-26 15:44:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000052176_13361152.pth... [2023-12-26 15:44:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000052528_13451264.pth... [2023-12-26 15:44:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000051024_13066240.pth [2023-12-26 15:44:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000051344_13148160.pth [2023-12-26 15:44:46,431][105620] Updated weights for policy 1, policy_version 52538 (0.0011) [2023-12-26 15:44:46,494][105620] Updated weights for policy 1, policy_version 52548 (0.0007) [2023-12-26 15:44:46,546][105620] Updated weights for policy 1, policy_version 52558 (0.0005) [2023-12-26 15:44:46,650][105692] Updated weights for policy 0, policy_version 52181 (0.0007) [2023-12-26 15:44:46,718][105692] Updated weights for policy 0, policy_version 52191 (0.0005) [2023-12-26 15:44:46,781][105692] Updated weights for policy 0, policy_version 52201 (0.0006) [2023-12-26 15:44:47,089][105620] Updated weights for policy 1, policy_version 52568 (0.0005) [2023-12-26 15:44:47,140][105620] Updated weights for policy 1, policy_version 52578 (0.0006) [2023-12-26 15:44:47,200][105620] Updated weights for policy 1, policy_version 52588 (0.0008) [2023-12-26 15:44:47,317][105692] Updated weights for policy 0, policy_version 52211 (0.0006) [2023-12-26 15:44:47,364][105692] Updated weights for policy 0, policy_version 52221 (0.0009) [2023-12-26 15:44:47,411][105692] Updated weights for policy 0, policy_version 52231 (0.0007) [2023-12-26 15:44:47,801][105620] Updated weights for policy 1, policy_version 52598 (0.0007) [2023-12-26 15:44:47,858][105620] Updated weights for policy 1, policy_version 52608 (0.0005) [2023-12-26 15:44:47,915][105620] Updated weights for policy 1, policy_version 52618 (0.0006) [2023-12-26 15:44:48,306][105692] Updated weights for policy 0, policy_version 52241 (0.0009) [2023-12-26 15:44:48,370][105692] Updated weights for policy 0, policy_version 52251 (0.0010) [2023-12-26 15:44:48,426][105692] Updated weights for policy 0, policy_version 52261 (0.0007) [2023-12-26 15:44:48,452][105620] Updated weights for policy 1, policy_version 52628 (0.0007) [2023-12-26 15:44:48,479][105692] Updated weights for policy 0, policy_version 52271 (0.0007) [2023-12-26 15:44:48,511][105620] Updated weights for policy 1, policy_version 52638 (0.0006) [2023-12-26 15:44:48,576][105620] Updated weights for policy 1, policy_version 52648 (0.0005) [2023-12-26 15:44:49,235][105692] Updated weights for policy 0, policy_version 52281 (0.0007) [2023-12-26 15:44:49,256][105620] Updated weights for policy 1, policy_version 52658 (0.0006) [2023-12-26 15:44:49,298][105692] Updated weights for policy 0, policy_version 52291 (0.0008) [2023-12-26 15:44:49,314][105620] Updated weights for policy 1, policy_version 52668 (0.0006) [2023-12-26 15:44:49,362][105692] Updated weights for policy 0, policy_version 52301 (0.0008) [2023-12-26 15:44:49,379][105620] Updated weights for policy 1, policy_version 52678 (0.0008) [2023-12-26 15:44:49,431][105620] Updated weights for policy 1, policy_version 52688 (0.0009) [2023-12-26 15:44:50,037][105692] Updated weights for policy 0, policy_version 52311 (0.0008) [2023-12-26 15:44:50,091][105620] Updated weights for policy 1, policy_version 52698 (0.0007) [2023-12-26 15:44:50,093][105692] Updated weights for policy 0, policy_version 52321 (0.0008) [2023-12-26 15:44:50,138][105620] Updated weights for policy 1, policy_version 52708 (0.0008) [2023-12-26 15:44:50,153][105692] Updated weights for policy 0, policy_version 52331 (0.0007) [2023-12-26 15:44:50,193][105620] Updated weights for policy 1, policy_version 52718 (0.0008) [2023-12-26 15:44:50,905][105620] Updated weights for policy 1, policy_version 52728 (0.0008) [2023-12-26 15:44:50,911][105692] Updated weights for policy 0, policy_version 52341 (0.0008) [2023-12-26 15:44:50,964][105620] Updated weights for policy 1, policy_version 52738 (0.0007) [2023-12-26 15:44:50,971][105692] Updated weights for policy 0, policy_version 52351 (0.0006) [2023-12-26 15:44:51,027][105620] Updated weights for policy 1, policy_version 52748 (0.0007) [2023-12-26 15:44:51,036][105692] Updated weights for policy 0, policy_version 52361 (0.0008) [2023-12-26 15:44:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19933.9, 300 sec: 19744.1). Total num frames: 26910720. Throughput: 0: 9817.1, 1: 10282.5. Samples: 26899992. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 15:44:51,062][104569] Avg episode reward: [(0, '7898.493'), (1, '9094.090')] [2023-12-26 15:44:51,779][105692] Updated weights for policy 0, policy_version 52371 (0.0008) [2023-12-26 15:44:51,783][105620] Updated weights for policy 1, policy_version 52758 (0.0009) [2023-12-26 15:44:51,839][105620] Updated weights for policy 1, policy_version 52768 (0.0008) [2023-12-26 15:44:51,839][105692] Updated weights for policy 0, policy_version 52381 (0.0008) [2023-12-26 15:44:51,894][105692] Updated weights for policy 0, policy_version 52391 (0.0009) [2023-12-26 15:44:51,901][105620] Updated weights for policy 1, policy_version 52778 (0.0008) [2023-12-26 15:44:52,597][105692] Updated weights for policy 0, policy_version 52401 (0.0009) [2023-12-26 15:44:52,666][105692] Updated weights for policy 0, policy_version 52411 (0.0005) [2023-12-26 15:44:52,710][105620] Updated weights for policy 1, policy_version 52788 (0.0007) [2023-12-26 15:44:52,733][105692] Updated weights for policy 0, policy_version 52421 (0.0007) [2023-12-26 15:44:52,762][105620] Updated weights for policy 1, policy_version 52798 (0.0005) [2023-12-26 15:44:52,789][105692] Updated weights for policy 0, policy_version 52431 (0.0008) [2023-12-26 15:44:52,827][105620] Updated weights for policy 1, policy_version 52808 (0.0008) [2023-12-26 15:44:53,356][105692] Updated weights for policy 0, policy_version 52441 (0.0006) [2023-12-26 15:44:53,416][105692] Updated weights for policy 0, policy_version 52451 (0.0006) [2023-12-26 15:44:53,465][105692] Updated weights for policy 0, policy_version 52461 (0.0006) [2023-12-26 15:44:53,664][105620] Updated weights for policy 1, policy_version 52818 (0.0009) [2023-12-26 15:44:53,711][105620] Updated weights for policy 1, policy_version 52828 (0.0009) [2023-12-26 15:44:53,757][105620] Updated weights for policy 1, policy_version 52838 (0.0008) [2023-12-26 15:44:53,814][105620] Updated weights for policy 1, policy_version 52848 (0.0008) [2023-12-26 15:44:54,080][105692] Updated weights for policy 0, policy_version 52471 (0.0006) [2023-12-26 15:44:54,137][105692] Updated weights for policy 0, policy_version 52483 (0.0010) [2023-12-26 15:44:54,200][105692] Updated weights for policy 0, policy_version 52493 (0.0009) [2023-12-26 15:44:54,552][105620] Updated weights for policy 1, policy_version 52858 (0.0009) [2023-12-26 15:44:54,615][105620] Updated weights for policy 1, policy_version 52868 (0.0009) [2023-12-26 15:44:54,677][105620] Updated weights for policy 1, policy_version 52878 (0.0009) [2023-12-26 15:44:54,891][105692] Updated weights for policy 0, policy_version 52503 (0.0006) [2023-12-26 15:44:54,951][105692] Updated weights for policy 0, policy_version 52513 (0.0009) [2023-12-26 15:44:55,006][105692] Updated weights for policy 0, policy_version 52523 (0.0009) [2023-12-26 15:44:55,508][105620] Updated weights for policy 1, policy_version 52888 (0.0010) [2023-12-26 15:44:55,555][105620] Updated weights for policy 1, policy_version 52898 (0.0009) [2023-12-26 15:44:55,604][105620] Updated weights for policy 1, policy_version 52908 (0.0007) [2023-12-26 15:44:55,609][105692] Updated weights for policy 0, policy_version 52533 (0.0008) [2023-12-26 15:44:55,665][105692] Updated weights for policy 0, policy_version 52543 (0.0009) [2023-12-26 15:44:55,720][105692] Updated weights for policy 0, policy_version 52553 (0.0007) [2023-12-26 15:44:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19744.1). Total num frames: 27009024. Throughput: 0: 9890.0, 1: 10135.0. Samples: 27016716. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 15:44:56,062][104569] Avg episode reward: [(0, '1932.869'), (1, '9011.078')] [2023-12-26 15:44:56,372][105692] Updated weights for policy 0, policy_version 52563 (0.0006) [2023-12-26 15:44:56,429][105692] Updated weights for policy 0, policy_version 52573 (0.0007) [2023-12-26 15:44:56,448][105620] Updated weights for policy 1, policy_version 52918 (0.0007) [2023-12-26 15:44:56,488][105692] Updated weights for policy 0, policy_version 52583 (0.0008) [2023-12-26 15:44:56,510][105620] Updated weights for policy 1, policy_version 52928 (0.0010) [2023-12-26 15:44:56,566][105620] Updated weights for policy 1, policy_version 52938 (0.0009) [2023-12-26 15:44:57,125][105692] Updated weights for policy 0, policy_version 52593 (0.0007) [2023-12-26 15:44:57,178][105692] Updated weights for policy 0, policy_version 52603 (0.0008) [2023-12-26 15:44:57,241][105692] Updated weights for policy 0, policy_version 52613 (0.0005) [2023-12-26 15:44:57,294][105692] Updated weights for policy 0, policy_version 52623 (0.0005) [2023-12-26 15:44:57,346][105620] Updated weights for policy 1, policy_version 52948 (0.0010) [2023-12-26 15:44:57,398][105620] Updated weights for policy 1, policy_version 52958 (0.0009) [2023-12-26 15:44:57,451][105620] Updated weights for policy 1, policy_version 52968 (0.0009) [2023-12-26 15:44:57,923][105692] Updated weights for policy 0, policy_version 52633 (0.0010) [2023-12-26 15:44:57,976][105692] Updated weights for policy 0, policy_version 52643 (0.0010) [2023-12-26 15:44:58,024][105692] Updated weights for policy 0, policy_version 52653 (0.0010) [2023-12-26 15:44:58,227][105620] Updated weights for policy 1, policy_version 52978 (0.0009) [2023-12-26 15:44:58,289][105620] Updated weights for policy 1, policy_version 52988 (0.0008) [2023-12-26 15:44:58,357][105620] Updated weights for policy 1, policy_version 52998 (0.0009) [2023-12-26 15:44:58,419][105620] Updated weights for policy 1, policy_version 53008 (0.0007) [2023-12-26 15:44:58,848][105692] Updated weights for policy 0, policy_version 52663 (0.0010) [2023-12-26 15:44:58,911][105692] Updated weights for policy 0, policy_version 52673 (0.0009) [2023-12-26 15:44:58,974][105692] Updated weights for policy 0, policy_version 52683 (0.0008) [2023-12-26 15:44:59,091][105620] Updated weights for policy 1, policy_version 53018 (0.0009) [2023-12-26 15:44:59,160][105620] Updated weights for policy 1, policy_version 53028 (0.0009) [2023-12-26 15:44:59,228][105620] Updated weights for policy 1, policy_version 53038 (0.0009) [2023-12-26 15:44:59,695][105692] Updated weights for policy 0, policy_version 52693 (0.0009) [2023-12-26 15:44:59,753][105692] Updated weights for policy 0, policy_version 52703 (0.0009) [2023-12-26 15:44:59,811][105692] Updated weights for policy 0, policy_version 52713 (0.0008) [2023-12-26 15:44:59,936][105620] Updated weights for policy 1, policy_version 53048 (0.0008) [2023-12-26 15:44:59,997][105620] Updated weights for policy 1, policy_version 53058 (0.0009) [2023-12-26 15:45:00,053][105620] Updated weights for policy 1, policy_version 53068 (0.0008) [2023-12-26 15:45:00,566][105692] Updated weights for policy 0, policy_version 52723 (0.0009) [2023-12-26 15:45:00,623][105692] Updated weights for policy 0, policy_version 52733 (0.0010) [2023-12-26 15:45:00,686][105692] Updated weights for policy 0, policy_version 52743 (0.0009) [2023-12-26 15:45:00,778][105620] Updated weights for policy 1, policy_version 53078 (0.0008) [2023-12-26 15:45:00,823][105620] Updated weights for policy 1, policy_version 53088 (0.0010) [2023-12-26 15:45:00,870][105620] Updated weights for policy 1, policy_version 53098 (0.0009) [2023-12-26 15:45:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19933.9, 300 sec: 19716.3). Total num frames: 27107328. Throughput: 0: 9966.2, 1: 10048.4. Samples: 27075436. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 15:45:01,062][104569] Avg episode reward: [(0, '3973.322'), (1, '9184.968')] [2023-12-26 15:45:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000052752_13508608.pth... [2023-12-26 15:45:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000053104_13598720.pth... [2023-12-26 15:45:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000051952_13303808.pth [2023-12-26 15:45:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000051600_13213696.pth [2023-12-26 15:45:01,443][105692] Updated weights for policy 0, policy_version 52753 (0.0009) [2023-12-26 15:45:01,507][105692] Updated weights for policy 0, policy_version 52763 (0.0010) [2023-12-26 15:45:01,561][105620] Updated weights for policy 1, policy_version 53108 (0.0007) [2023-12-26 15:45:01,568][105692] Updated weights for policy 0, policy_version 52773 (0.0008) [2023-12-26 15:45:01,623][105620] Updated weights for policy 1, policy_version 53118 (0.0008) [2023-12-26 15:45:01,627][105692] Updated weights for policy 0, policy_version 52783 (0.0007) [2023-12-26 15:45:01,680][105620] Updated weights for policy 1, policy_version 53128 (0.0009) [2023-12-26 15:45:02,344][105692] Updated weights for policy 0, policy_version 52793 (0.0009) [2023-12-26 15:45:02,350][105620] Updated weights for policy 1, policy_version 53138 (0.0008) [2023-12-26 15:45:02,406][105692] Updated weights for policy 0, policy_version 52803 (0.0006) [2023-12-26 15:45:02,408][105620] Updated weights for policy 1, policy_version 53148 (0.0008) [2023-12-26 15:45:02,460][105692] Updated weights for policy 0, policy_version 52813 (0.0005) [2023-12-26 15:45:02,466][105620] Updated weights for policy 1, policy_version 53158 (0.0008) [2023-12-26 15:45:02,526][105620] Updated weights for policy 1, policy_version 53168 (0.0008) [2023-12-26 15:45:03,187][105620] Updated weights for policy 1, policy_version 53178 (0.0005) [2023-12-26 15:45:03,215][105692] Updated weights for policy 0, policy_version 52823 (0.0008) [2023-12-26 15:45:03,238][105620] Updated weights for policy 1, policy_version 53188 (0.0006) [2023-12-26 15:45:03,265][105692] Updated weights for policy 0, policy_version 52833 (0.0006) [2023-12-26 15:45:03,283][105620] Updated weights for policy 1, policy_version 53198 (0.0007) [2023-12-26 15:45:03,322][105692] Updated weights for policy 0, policy_version 52843 (0.0008) [2023-12-26 15:45:03,973][105620] Updated weights for policy 1, policy_version 53208 (0.0008) [2023-12-26 15:45:04,028][105620] Updated weights for policy 1, policy_version 53218 (0.0007) [2023-12-26 15:45:04,088][105620] Updated weights for policy 1, policy_version 53228 (0.0007) [2023-12-26 15:45:04,088][105692] Updated weights for policy 0, policy_version 52853 (0.0009) [2023-12-26 15:45:04,150][105692] Updated weights for policy 0, policy_version 52863 (0.0009) [2023-12-26 15:45:04,212][105692] Updated weights for policy 0, policy_version 52873 (0.0009) [2023-12-26 15:45:04,844][105620] Updated weights for policy 1, policy_version 53238 (0.0009) [2023-12-26 15:45:04,890][105620] Updated weights for policy 1, policy_version 53248 (0.0008) [2023-12-26 15:45:04,926][105692] Updated weights for policy 0, policy_version 52883 (0.0009) [2023-12-26 15:45:04,944][105620] Updated weights for policy 1, policy_version 53258 (0.0005) [2023-12-26 15:45:04,978][105692] Updated weights for policy 0, policy_version 52893 (0.0008) [2023-12-26 15:45:05,031][105692] Updated weights for policy 0, policy_version 52903 (0.0008) [2023-12-26 15:45:05,679][105692] Updated weights for policy 0, policy_version 52913 (0.0009) [2023-12-26 15:45:05,734][105620] Updated weights for policy 1, policy_version 53268 (0.0007) [2023-12-26 15:45:05,740][105692] Updated weights for policy 0, policy_version 52923 (0.0005) [2023-12-26 15:45:05,783][105620] Updated weights for policy 1, policy_version 53278 (0.0008) [2023-12-26 15:45:05,803][105692] Updated weights for policy 0, policy_version 52933 (0.0005) [2023-12-26 15:45:05,836][105620] Updated weights for policy 1, policy_version 53288 (0.0007) [2023-12-26 15:45:05,868][105692] Updated weights for policy 0, policy_version 52943 (0.0008) [2023-12-26 15:45:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19933.9, 300 sec: 19744.1). Total num frames: 27205632. Throughput: 0: 9864.2, 1: 9926.1. Samples: 27190316. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 15:45:06,062][104569] Avg episode reward: [(0, '7114.500'), (1, '9267.973')] [2023-12-26 15:45:06,526][105692] Updated weights for policy 0, policy_version 52953 (0.0009) [2023-12-26 15:45:06,586][105692] Updated weights for policy 0, policy_version 52963 (0.0011) [2023-12-26 15:45:06,648][105620] Updated weights for policy 1, policy_version 53298 (0.0006) [2023-12-26 15:45:06,660][105692] Updated weights for policy 0, policy_version 52973 (0.0010) [2023-12-26 15:45:06,717][105620] Updated weights for policy 1, policy_version 53308 (0.0007) [2023-12-26 15:45:06,777][105620] Updated weights for policy 1, policy_version 53318 (0.0006) [2023-12-26 15:45:06,847][105620] Updated weights for policy 1, policy_version 53328 (0.0006) [2023-12-26 15:45:07,319][105692] Updated weights for policy 0, policy_version 52983 (0.0008) [2023-12-26 15:45:07,373][105692] Updated weights for policy 0, policy_version 52994 (0.0010) [2023-12-26 15:45:07,427][105692] Updated weights for policy 0, policy_version 53005 (0.0010) [2023-12-26 15:45:07,532][105620] Updated weights for policy 1, policy_version 53338 (0.0006) [2023-12-26 15:45:07,594][105620] Updated weights for policy 1, policy_version 53348 (0.0010) [2023-12-26 15:45:07,658][105620] Updated weights for policy 1, policy_version 53358 (0.0011) [2023-12-26 15:45:08,177][105692] Updated weights for policy 0, policy_version 53016 (0.0010) [2023-12-26 15:45:08,238][105692] Updated weights for policy 0, policy_version 53026 (0.0010) [2023-12-26 15:45:08,286][105692] Updated weights for policy 0, policy_version 53036 (0.0010) [2023-12-26 15:45:08,375][105620] Updated weights for policy 1, policy_version 53368 (0.0009) [2023-12-26 15:45:08,439][105620] Updated weights for policy 1, policy_version 53378 (0.0010) [2023-12-26 15:45:08,492][105620] Updated weights for policy 1, policy_version 53388 (0.0007) [2023-12-26 15:45:08,870][105692] Updated weights for policy 0, policy_version 53046 (0.0007) [2023-12-26 15:45:08,924][105692] Updated weights for policy 0, policy_version 53056 (0.0010) [2023-12-26 15:45:08,973][105692] Updated weights for policy 0, policy_version 53066 (0.0005) [2023-12-26 15:45:09,307][105620] Updated weights for policy 1, policy_version 53398 (0.0007) [2023-12-26 15:45:09,375][105620] Updated weights for policy 1, policy_version 53408 (0.0008) [2023-12-26 15:45:09,442][105620] Updated weights for policy 1, policy_version 53418 (0.0010) [2023-12-26 15:45:09,580][105692] Updated weights for policy 0, policy_version 53076 (0.0008) [2023-12-26 15:45:09,639][105692] Updated weights for policy 0, policy_version 53086 (0.0010) [2023-12-26 15:45:09,705][105692] Updated weights for policy 0, policy_version 53096 (0.0011) [2023-12-26 15:45:10,263][105620] Updated weights for policy 1, policy_version 53428 (0.0010) [2023-12-26 15:45:10,325][105620] Updated weights for policy 1, policy_version 53438 (0.0008) [2023-12-26 15:45:10,387][105620] Updated weights for policy 1, policy_version 53448 (0.0008) [2023-12-26 15:45:10,446][105692] Updated weights for policy 0, policy_version 53106 (0.0010) [2023-12-26 15:45:10,512][105692] Updated weights for policy 0, policy_version 53116 (0.0010) [2023-12-26 15:45:10,576][105692] Updated weights for policy 0, policy_version 53126 (0.0010) [2023-12-26 15:45:10,624][105692] Updated weights for policy 0, policy_version 53136 (0.0010) [2023-12-26 15:45:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19797.3, 300 sec: 19716.4). Total num frames: 27295744. Throughput: 0: 9953.9, 1: 9827.1. Samples: 27306616. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 15:45:11,062][104569] Avg episode reward: [(0, '8771.079'), (1, '9354.474')] [2023-12-26 15:45:11,063][105586] Saving new best policy, reward=9354.474! [2023-12-26 15:45:11,143][105620] Updated weights for policy 1, policy_version 53458 (0.0009) [2023-12-26 15:45:11,216][105620] Updated weights for policy 1, policy_version 53468 (0.0008) [2023-12-26 15:45:11,284][105620] Updated weights for policy 1, policy_version 53478 (0.0008) [2023-12-26 15:45:11,347][105620] Updated weights for policy 1, policy_version 53488 (0.0008) [2023-12-26 15:45:11,376][105692] Updated weights for policy 0, policy_version 53147 (0.0009) [2023-12-26 15:45:11,428][105692] Updated weights for policy 0, policy_version 53157 (0.0010) [2023-12-26 15:45:11,484][105692] Updated weights for policy 0, policy_version 53167 (0.0011) [2023-12-26 15:45:12,113][105620] Updated weights for policy 1, policy_version 53498 (0.0008) [2023-12-26 15:45:12,172][105620] Updated weights for policy 1, policy_version 53508 (0.0008) [2023-12-26 15:45:12,207][105692] Updated weights for policy 0, policy_version 53177 (0.0011) [2023-12-26 15:45:12,225][105620] Updated weights for policy 1, policy_version 53518 (0.0006) [2023-12-26 15:45:12,259][105692] Updated weights for policy 0, policy_version 53187 (0.0011) [2023-12-26 15:45:12,323][105692] Updated weights for policy 0, policy_version 53197 (0.0011) [2023-12-26 15:45:13,026][105620] Updated weights for policy 1, policy_version 53528 (0.0009) [2023-12-26 15:45:13,059][105692] Updated weights for policy 0, policy_version 53207 (0.0011) [2023-12-26 15:45:13,076][105620] Updated weights for policy 1, policy_version 53538 (0.0011) [2023-12-26 15:45:13,123][105692] Updated weights for policy 0, policy_version 53217 (0.0011) [2023-12-26 15:45:13,130][105620] Updated weights for policy 1, policy_version 53548 (0.0011) [2023-12-26 15:45:13,171][105692] Updated weights for policy 0, policy_version 53227 (0.0010) [2023-12-26 15:45:13,889][105692] Updated weights for policy 0, policy_version 53237 (0.0007) [2023-12-26 15:45:13,898][105620] Updated weights for policy 1, policy_version 53558 (0.0007) [2023-12-26 15:45:13,943][105620] Updated weights for policy 1, policy_version 53568 (0.0008) [2023-12-26 15:45:13,944][105692] Updated weights for policy 0, policy_version 53247 (0.0005) [2023-12-26 15:45:13,986][105620] Updated weights for policy 1, policy_version 53578 (0.0007) [2023-12-26 15:45:13,992][105692] Updated weights for policy 0, policy_version 53257 (0.0006) [2023-12-26 15:45:14,648][105620] Updated weights for policy 1, policy_version 53588 (0.0008) [2023-12-26 15:45:14,654][105692] Updated weights for policy 0, policy_version 53267 (0.0006) [2023-12-26 15:45:14,712][105620] Updated weights for policy 1, policy_version 53598 (0.0010) [2023-12-26 15:45:14,714][105692] Updated weights for policy 0, policy_version 53277 (0.0007) [2023-12-26 15:45:14,755][105620] Updated weights for policy 1, policy_version 53608 (0.0011) [2023-12-26 15:45:14,774][105692] Updated weights for policy 0, policy_version 53287 (0.0008) [2023-12-26 15:45:15,418][105620] Updated weights for policy 1, policy_version 53618 (0.0009) [2023-12-26 15:45:15,473][105692] Updated weights for policy 0, policy_version 53297 (0.0008) [2023-12-26 15:45:15,478][105620] Updated weights for policy 1, policy_version 53628 (0.0011) [2023-12-26 15:45:15,529][105692] Updated weights for policy 0, policy_version 53307 (0.0011) [2023-12-26 15:45:15,537][105620] Updated weights for policy 1, policy_version 53638 (0.0011) [2023-12-26 15:45:15,581][105692] Updated weights for policy 0, policy_version 53317 (0.0010) [2023-12-26 15:45:15,592][105620] Updated weights for policy 1, policy_version 53648 (0.0010) [2023-12-26 15:45:15,633][105692] Updated weights for policy 0, policy_version 53327 (0.0010) [2023-12-26 15:45:16,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19797.3, 300 sec: 19716.3). Total num frames: 27394048. Throughput: 0: 9852.9, 1: 9725.2. Samples: 27362176. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 15:45:16,063][104569] Avg episode reward: [(0, '8930.211'), (1, '8991.787')] [2023-12-26 15:45:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000053328_13656064.pth... [2023-12-26 15:45:16,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000053648_13737984.pth... [2023-12-26 15:45:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000052528_13451264.pth [2023-12-26 15:45:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000052176_13361152.pth [2023-12-26 15:45:16,284][105620] Updated weights for policy 1, policy_version 53658 (0.0008) [2023-12-26 15:45:16,346][105620] Updated weights for policy 1, policy_version 53668 (0.0010) [2023-12-26 15:45:16,388][105692] Updated weights for policy 0, policy_version 53337 (0.0009) [2023-12-26 15:45:16,402][105620] Updated weights for policy 1, policy_version 53678 (0.0010) [2023-12-26 15:45:16,436][105692] Updated weights for policy 0, policy_version 53347 (0.0009) [2023-12-26 15:45:16,484][105692] Updated weights for policy 0, policy_version 53357 (0.0009) [2023-12-26 15:45:17,074][105620] Updated weights for policy 1, policy_version 53688 (0.0007) [2023-12-26 15:45:17,141][105620] Updated weights for policy 1, policy_version 53698 (0.0006) [2023-12-26 15:45:17,191][105620] Updated weights for policy 1, policy_version 53708 (0.0006) [2023-12-26 15:45:17,296][105692] Updated weights for policy 0, policy_version 53367 (0.0008) [2023-12-26 15:45:17,349][105692] Updated weights for policy 0, policy_version 53378 (0.0009) [2023-12-26 15:45:17,406][105692] Updated weights for policy 0, policy_version 53388 (0.0010) [2023-12-26 15:45:17,707][105620] Updated weights for policy 1, policy_version 53718 (0.0007) [2023-12-26 15:45:17,777][105620] Updated weights for policy 1, policy_version 53728 (0.0006) [2023-12-26 15:45:17,833][105620] Updated weights for policy 1, policy_version 53738 (0.0006) [2023-12-26 15:45:18,274][105692] Updated weights for policy 0, policy_version 53399 (0.0009) [2023-12-26 15:45:18,330][105692] Updated weights for policy 0, policy_version 53409 (0.0008) [2023-12-26 15:45:18,391][105692] Updated weights for policy 0, policy_version 53419 (0.0008) [2023-12-26 15:45:18,464][105620] Updated weights for policy 1, policy_version 53748 (0.0007) [2023-12-26 15:45:18,526][105620] Updated weights for policy 1, policy_version 53758 (0.0010) [2023-12-26 15:45:18,584][105620] Updated weights for policy 1, policy_version 53768 (0.0010) [2023-12-26 15:45:19,110][105692] Updated weights for policy 0, policy_version 53429 (0.0007) [2023-12-26 15:45:19,170][105692] Updated weights for policy 0, policy_version 53439 (0.0005) [2023-12-26 15:45:19,229][105692] Updated weights for policy 0, policy_version 53449 (0.0007) [2023-12-26 15:45:19,332][105620] Updated weights for policy 1, policy_version 53778 (0.0010) [2023-12-26 15:45:19,388][105620] Updated weights for policy 1, policy_version 53788 (0.0010) [2023-12-26 15:45:19,449][105620] Updated weights for policy 1, policy_version 53798 (0.0010) [2023-12-26 15:45:19,508][105620] Updated weights for policy 1, policy_version 53808 (0.0010) [2023-12-26 15:45:19,925][105692] Updated weights for policy 0, policy_version 53459 (0.0009) [2023-12-26 15:45:19,976][105692] Updated weights for policy 0, policy_version 53469 (0.0010) [2023-12-26 15:45:20,039][105692] Updated weights for policy 0, policy_version 53479 (0.0011) [2023-12-26 15:45:20,293][105620] Updated weights for policy 1, policy_version 53818 (0.0010) [2023-12-26 15:45:20,359][105620] Updated weights for policy 1, policy_version 53828 (0.0010) [2023-12-26 15:45:20,418][105620] Updated weights for policy 1, policy_version 53838 (0.0010) [2023-12-26 15:45:20,845][105692] Updated weights for policy 0, policy_version 53489 (0.0010) [2023-12-26 15:45:20,902][105692] Updated weights for policy 0, policy_version 53499 (0.0008) [2023-12-26 15:45:20,951][105692] Updated weights for policy 0, policy_version 53509 (0.0008) [2023-12-26 15:45:21,004][105692] Updated weights for policy 0, policy_version 53519 (0.0008) [2023-12-26 15:45:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19716.3). Total num frames: 27492352. Throughput: 0: 9750.9, 1: 9728.7. Samples: 27480892. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 15:45:21,063][104569] Avg episode reward: [(0, '8925.914'), (1, '8991.862')] [2023-12-26 15:45:21,167][105620] Updated weights for policy 1, policy_version 53848 (0.0009) [2023-12-26 15:45:21,229][105620] Updated weights for policy 1, policy_version 53858 (0.0009) [2023-12-26 15:45:21,289][105620] Updated weights for policy 1, policy_version 53868 (0.0009) [2023-12-26 15:45:21,843][105692] Updated weights for policy 0, policy_version 53529 (0.0009) [2023-12-26 15:45:21,903][105692] Updated weights for policy 0, policy_version 53539 (0.0009) [2023-12-26 15:45:21,962][105692] Updated weights for policy 0, policy_version 53549 (0.0010) [2023-12-26 15:45:22,009][105620] Updated weights for policy 1, policy_version 53878 (0.0008) [2023-12-26 15:45:22,062][105620] Updated weights for policy 1, policy_version 53888 (0.0005) [2023-12-26 15:45:22,129][105620] Updated weights for policy 1, policy_version 53898 (0.0007) [2023-12-26 15:45:22,814][105692] Updated weights for policy 0, policy_version 53559 (0.0009) [2023-12-26 15:45:22,864][105620] Updated weights for policy 1, policy_version 53908 (0.0008) [2023-12-26 15:45:22,870][105692] Updated weights for policy 0, policy_version 53569 (0.0011) [2023-12-26 15:45:22,931][105692] Updated weights for policy 0, policy_version 53579 (0.0009) [2023-12-26 15:45:22,933][105620] Updated weights for policy 1, policy_version 53918 (0.0007) [2023-12-26 15:45:22,989][105620] Updated weights for policy 1, policy_version 53928 (0.0007) [2023-12-26 15:45:23,574][105692] Updated weights for policy 0, policy_version 53589 (0.0007) [2023-12-26 15:45:23,635][105692] Updated weights for policy 0, policy_version 53599 (0.0005) [2023-12-26 15:45:23,698][105692] Updated weights for policy 0, policy_version 53609 (0.0007) [2023-12-26 15:45:23,709][105620] Updated weights for policy 1, policy_version 53938 (0.0006) [2023-12-26 15:45:23,765][105620] Updated weights for policy 1, policy_version 53948 (0.0007) [2023-12-26 15:45:23,820][105620] Updated weights for policy 1, policy_version 53958 (0.0007) [2023-12-26 15:45:23,868][105620] Updated weights for policy 1, policy_version 53968 (0.0008) [2023-12-26 15:45:24,357][105692] Updated weights for policy 0, policy_version 53619 (0.0009) [2023-12-26 15:45:24,412][105692] Updated weights for policy 0, policy_version 53629 (0.0005) [2023-12-26 15:45:24,466][105692] Updated weights for policy 0, policy_version 53639 (0.0005) [2023-12-26 15:45:24,538][105620] Updated weights for policy 1, policy_version 53978 (0.0010) [2023-12-26 15:45:24,592][105620] Updated weights for policy 1, policy_version 53988 (0.0008) [2023-12-26 15:45:24,648][105620] Updated weights for policy 1, policy_version 53998 (0.0005) [2023-12-26 15:45:25,088][105692] Updated weights for policy 0, policy_version 53649 (0.0006) [2023-12-26 15:45:25,136][105692] Updated weights for policy 0, policy_version 53659 (0.0010) [2023-12-26 15:45:25,194][105692] Updated weights for policy 0, policy_version 53669 (0.0010) [2023-12-26 15:45:25,258][105692] Updated weights for policy 0, policy_version 53679 (0.0010) [2023-12-26 15:45:25,316][105620] Updated weights for policy 1, policy_version 54008 (0.0005) [2023-12-26 15:45:25,374][105620] Updated weights for policy 1, policy_version 54018 (0.0007) [2023-12-26 15:45:25,442][105620] Updated weights for policy 1, policy_version 54028 (0.0011) [2023-12-26 15:45:26,015][105620] Updated weights for policy 1, policy_version 54038 (0.0008) [2023-12-26 15:45:26,018][105692] Updated weights for policy 0, policy_version 53689 (0.0010) [2023-12-26 15:45:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 27582464. Throughput: 0: 9750.5, 1: 9676.2. Samples: 27597208. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 15:45:26,063][104569] Avg episode reward: [(0, '8583.943'), (1, '9353.919')] [2023-12-26 15:45:26,076][105620] Updated weights for policy 1, policy_version 54048 (0.0006) [2023-12-26 15:45:26,077][105692] Updated weights for policy 0, policy_version 53699 (0.0011) [2023-12-26 15:45:26,136][105620] Updated weights for policy 1, policy_version 54058 (0.0005) [2023-12-26 15:45:26,143][105692] Updated weights for policy 0, policy_version 53709 (0.0010) [2023-12-26 15:45:26,690][105620] Updated weights for policy 1, policy_version 54068 (0.0007) [2023-12-26 15:45:26,754][105620] Updated weights for policy 1, policy_version 54078 (0.0005) [2023-12-26 15:45:26,767][105692] Updated weights for policy 0, policy_version 53719 (0.0007) [2023-12-26 15:45:26,808][105620] Updated weights for policy 1, policy_version 54088 (0.0005) [2023-12-26 15:45:26,819][105692] Updated weights for policy 0, policy_version 53729 (0.0010) [2023-12-26 15:45:26,867][105692] Updated weights for policy 0, policy_version 53739 (0.0010) [2023-12-26 15:45:27,389][105620] Updated weights for policy 1, policy_version 54098 (0.0006) [2023-12-26 15:45:27,455][105620] Updated weights for policy 1, policy_version 54108 (0.0008) [2023-12-26 15:45:27,518][105620] Updated weights for policy 1, policy_version 54118 (0.0009) [2023-12-26 15:45:27,562][105692] Updated weights for policy 0, policy_version 53749 (0.0008) [2023-12-26 15:45:27,579][105620] Updated weights for policy 1, policy_version 54128 (0.0009) [2023-12-26 15:45:27,614][105692] Updated weights for policy 0, policy_version 53759 (0.0008) [2023-12-26 15:45:27,665][105692] Updated weights for policy 0, policy_version 53769 (0.0010) [2023-12-26 15:45:28,315][105620] Updated weights for policy 1, policy_version 54138 (0.0008) [2023-12-26 15:45:28,380][105620] Updated weights for policy 1, policy_version 54148 (0.0006) [2023-12-26 15:45:28,388][105692] Updated weights for policy 0, policy_version 53779 (0.0010) [2023-12-26 15:45:28,440][105620] Updated weights for policy 1, policy_version 54158 (0.0009) [2023-12-26 15:45:28,443][105692] Updated weights for policy 0, policy_version 53789 (0.0006) [2023-12-26 15:45:28,494][105692] Updated weights for policy 0, policy_version 53799 (0.0009) [2023-12-26 15:45:29,129][105692] Updated weights for policy 0, policy_version 53809 (0.0010) [2023-12-26 15:45:29,190][105692] Updated weights for policy 0, policy_version 53819 (0.0006) [2023-12-26 15:45:29,255][105692] Updated weights for policy 0, policy_version 53829 (0.0008) [2023-12-26 15:45:29,262][105620] Updated weights for policy 1, policy_version 54168 (0.0007) [2023-12-26 15:45:29,315][105692] Updated weights for policy 0, policy_version 53839 (0.0010) [2023-12-26 15:45:29,323][105620] Updated weights for policy 1, policy_version 54178 (0.0006) [2023-12-26 15:45:29,391][105620] Updated weights for policy 1, policy_version 54188 (0.0007) [2023-12-26 15:45:29,979][105692] Updated weights for policy 0, policy_version 53849 (0.0006) [2023-12-26 15:45:30,028][105692] Updated weights for policy 0, policy_version 53859 (0.0006) [2023-12-26 15:45:30,086][105692] Updated weights for policy 0, policy_version 53869 (0.0006) [2023-12-26 15:45:30,169][105620] Updated weights for policy 1, policy_version 54198 (0.0009) [2023-12-26 15:45:30,229][105620] Updated weights for policy 1, policy_version 54208 (0.0008) [2023-12-26 15:45:30,281][105620] Updated weights for policy 1, policy_version 54218 (0.0009) [2023-12-26 15:45:30,753][105692] Updated weights for policy 0, policy_version 53879 (0.0008) [2023-12-26 15:45:30,804][105692] Updated weights for policy 0, policy_version 53889 (0.0010) [2023-12-26 15:45:30,849][105692] Updated weights for policy 0, policy_version 53900 (0.0007) [2023-12-26 15:45:30,977][105620] Updated weights for policy 1, policy_version 54228 (0.0009) [2023-12-26 15:45:31,031][105620] Updated weights for policy 1, policy_version 54238 (0.0009) [2023-12-26 15:45:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 27688960. Throughput: 0: 9798.1, 1: 9736.6. Samples: 27658104. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 15:45:31,063][104569] Avg episode reward: [(0, '8845.260'), (1, '9269.709')] [2023-12-26 15:45:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000053904_13803520.pth... [2023-12-26 15:45:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000052752_13508608.pth [2023-12-26 15:45:31,096][105620] Updated weights for policy 1, policy_version 54248 (0.0008) [2023-12-26 15:45:31,147][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000054256_13893632.pth... [2023-12-26 15:45:31,151][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000053104_13598720.pth [2023-12-26 15:45:31,623][105692] Updated weights for policy 0, policy_version 53910 (0.0007) [2023-12-26 15:45:31,684][105692] Updated weights for policy 0, policy_version 53920 (0.0006) [2023-12-26 15:45:31,747][105692] Updated weights for policy 0, policy_version 53930 (0.0009) [2023-12-26 15:45:31,861][105620] Updated weights for policy 1, policy_version 54258 (0.0009) [2023-12-26 15:45:31,923][105620] Updated weights for policy 1, policy_version 54268 (0.0009) [2023-12-26 15:45:31,987][105620] Updated weights for policy 1, policy_version 54278 (0.0009) [2023-12-26 15:45:32,038][105620] Updated weights for policy 1, policy_version 54288 (0.0008) [2023-12-26 15:45:32,425][105692] Updated weights for policy 0, policy_version 53940 (0.0008) [2023-12-26 15:45:32,477][105692] Updated weights for policy 0, policy_version 53950 (0.0010) [2023-12-26 15:45:32,537][105692] Updated weights for policy 0, policy_version 53960 (0.0010) [2023-12-26 15:45:32,790][105620] Updated weights for policy 1, policy_version 54298 (0.0009) [2023-12-26 15:45:32,846][105620] Updated weights for policy 1, policy_version 54308 (0.0009) [2023-12-26 15:45:32,899][105620] Updated weights for policy 1, policy_version 54318 (0.0009) [2023-12-26 15:45:33,166][105692] Updated weights for policy 0, policy_version 53970 (0.0008) [2023-12-26 15:45:33,229][105692] Updated weights for policy 0, policy_version 53980 (0.0009) [2023-12-26 15:45:33,293][105692] Updated weights for policy 0, policy_version 53990 (0.0009) [2023-12-26 15:45:33,353][105692] Updated weights for policy 0, policy_version 54000 (0.0009) [2023-12-26 15:45:33,631][105620] Updated weights for policy 1, policy_version 54328 (0.0009) [2023-12-26 15:45:33,687][105620] Updated weights for policy 1, policy_version 54338 (0.0010) [2023-12-26 15:45:33,749][105620] Updated weights for policy 1, policy_version 54348 (0.0010) [2023-12-26 15:45:34,071][105692] Updated weights for policy 0, policy_version 54010 (0.0007) [2023-12-26 15:45:34,133][105692] Updated weights for policy 0, policy_version 54020 (0.0008) [2023-12-26 15:45:34,199][105692] Updated weights for policy 0, policy_version 54030 (0.0008) [2023-12-26 15:45:34,511][105620] Updated weights for policy 1, policy_version 54358 (0.0010) [2023-12-26 15:45:34,577][105620] Updated weights for policy 1, policy_version 54368 (0.0010) [2023-12-26 15:45:34,634][105620] Updated weights for policy 1, policy_version 54378 (0.0010) [2023-12-26 15:45:34,956][105692] Updated weights for policy 0, policy_version 54040 (0.0008) [2023-12-26 15:45:35,022][105692] Updated weights for policy 0, policy_version 54050 (0.0010) [2023-12-26 15:45:35,077][105692] Updated weights for policy 0, policy_version 54060 (0.0009) [2023-12-26 15:45:35,280][105620] Updated weights for policy 1, policy_version 54388 (0.0010) [2023-12-26 15:45:35,342][105620] Updated weights for policy 1, policy_version 54398 (0.0009) [2023-12-26 15:45:35,398][105620] Updated weights for policy 1, policy_version 54408 (0.0009) [2023-12-26 15:45:35,858][105692] Updated weights for policy 0, policy_version 54070 (0.0010) [2023-12-26 15:45:35,928][105692] Updated weights for policy 0, policy_version 54080 (0.0009) [2023-12-26 15:45:35,986][105692] Updated weights for policy 0, policy_version 54090 (0.0010) [2023-12-26 15:45:36,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.3, 300 sec: 19716.3). Total num frames: 27787264. Throughput: 0: 9855.5, 1: 9570.2. Samples: 27774148. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 15:45:36,062][104569] Avg episode reward: [(0, '9359.563'), (1, '9269.762')] [2023-12-26 15:45:36,063][105585] Saving new best policy, reward=9359.563! [2023-12-26 15:45:36,070][105620] Updated weights for policy 1, policy_version 54418 (0.0009) [2023-12-26 15:45:36,137][105620] Updated weights for policy 1, policy_version 54428 (0.0009) [2023-12-26 15:45:36,193][105620] Updated weights for policy 1, policy_version 54438 (0.0009) [2023-12-26 15:45:36,253][105620] Updated weights for policy 1, policy_version 54448 (0.0009) [2023-12-26 15:45:36,806][105692] Updated weights for policy 0, policy_version 54100 (0.0009) [2023-12-26 15:45:36,855][105692] Updated weights for policy 0, policy_version 54110 (0.0006) [2023-12-26 15:45:36,907][105692] Updated weights for policy 0, policy_version 54120 (0.0008) [2023-12-26 15:45:37,003][105620] Updated weights for policy 1, policy_version 54458 (0.0011) [2023-12-26 15:45:37,064][105620] Updated weights for policy 1, policy_version 54468 (0.0010) [2023-12-26 15:45:37,126][105620] Updated weights for policy 1, policy_version 54478 (0.0010) [2023-12-26 15:45:37,662][105692] Updated weights for policy 0, policy_version 54130 (0.0008) [2023-12-26 15:45:37,727][105692] Updated weights for policy 0, policy_version 54140 (0.0009) [2023-12-26 15:45:37,782][105692] Updated weights for policy 0, policy_version 54150 (0.0009) [2023-12-26 15:45:37,842][105692] Updated weights for policy 0, policy_version 54160 (0.0009) [2023-12-26 15:45:37,904][105620] Updated weights for policy 1, policy_version 54488 (0.0011) [2023-12-26 15:45:37,971][105620] Updated weights for policy 1, policy_version 54498 (0.0010) [2023-12-26 15:45:38,036][105620] Updated weights for policy 1, policy_version 54508 (0.0011) [2023-12-26 15:45:38,628][105692] Updated weights for policy 0, policy_version 54170 (0.0008) [2023-12-26 15:45:38,683][105692] Updated weights for policy 0, policy_version 54180 (0.0008) [2023-12-26 15:45:38,732][105692] Updated weights for policy 0, policy_version 54190 (0.0008) [2023-12-26 15:45:38,790][105620] Updated weights for policy 1, policy_version 54518 (0.0011) [2023-12-26 15:45:38,845][105620] Updated weights for policy 1, policy_version 54528 (0.0010) [2023-12-26 15:45:38,924][105620] Updated weights for policy 1, policy_version 54538 (0.0010) [2023-12-26 15:45:39,500][105692] Updated weights for policy 0, policy_version 54200 (0.0008) [2023-12-26 15:45:39,557][105692] Updated weights for policy 0, policy_version 54210 (0.0009) [2023-12-26 15:45:39,613][105692] Updated weights for policy 0, policy_version 54220 (0.0008) [2023-12-26 15:45:39,701][105620] Updated weights for policy 1, policy_version 54548 (0.0010) [2023-12-26 15:45:39,756][105620] Updated weights for policy 1, policy_version 54558 (0.0010) [2023-12-26 15:45:39,819][105620] Updated weights for policy 1, policy_version 54568 (0.0011) [2023-12-26 15:45:40,387][105692] Updated weights for policy 0, policy_version 54230 (0.0008) [2023-12-26 15:45:40,439][105692] Updated weights for policy 0, policy_version 54240 (0.0007) [2023-12-26 15:45:40,498][105692] Updated weights for policy 0, policy_version 54250 (0.0008) [2023-12-26 15:45:40,568][105620] Updated weights for policy 1, policy_version 54578 (0.0011) [2023-12-26 15:45:40,630][105620] Updated weights for policy 1, policy_version 54588 (0.0011) [2023-12-26 15:45:40,675][105620] Updated weights for policy 1, policy_version 54598 (0.0010) [2023-12-26 15:45:40,740][105620] Updated weights for policy 1, policy_version 54608 (0.0010) [2023-12-26 15:45:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19688.6). Total num frames: 27877376. Throughput: 0: 9677.4, 1: 9616.1. Samples: 27884924. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 15:45:41,062][104569] Avg episode reward: [(0, '8607.715'), (1, '9267.900')] [2023-12-26 15:45:41,279][105692] Updated weights for policy 0, policy_version 54260 (0.0009) [2023-12-26 15:45:41,336][105692] Updated weights for policy 0, policy_version 54270 (0.0010) [2023-12-26 15:45:41,414][105692] Updated weights for policy 0, policy_version 54280 (0.0009) [2023-12-26 15:45:41,470][105620] Updated weights for policy 1, policy_version 54618 (0.0011) [2023-12-26 15:45:41,526][105620] Updated weights for policy 1, policy_version 54628 (0.0011) [2023-12-26 15:45:41,584][105620] Updated weights for policy 1, policy_version 54638 (0.0011) [2023-12-26 15:45:42,210][105692] Updated weights for policy 0, policy_version 54290 (0.0008) [2023-12-26 15:45:42,282][105692] Updated weights for policy 0, policy_version 54300 (0.0007) [2023-12-26 15:45:42,348][105692] Updated weights for policy 0, policy_version 54310 (0.0008) [2023-12-26 15:45:42,376][105620] Updated weights for policy 1, policy_version 54648 (0.0010) [2023-12-26 15:45:42,409][105692] Updated weights for policy 0, policy_version 54320 (0.0007) [2023-12-26 15:45:42,441][105620] Updated weights for policy 1, policy_version 54658 (0.0009) [2023-12-26 15:45:42,506][105620] Updated weights for policy 1, policy_version 54668 (0.0011) [2023-12-26 15:45:42,986][105692] Updated weights for policy 0, policy_version 54330 (0.0006) [2023-12-26 15:45:43,032][105692] Updated weights for policy 0, policy_version 54340 (0.0005) [2023-12-26 15:45:43,080][105692] Updated weights for policy 0, policy_version 54350 (0.0005) [2023-12-26 15:45:43,298][105620] Updated weights for policy 1, policy_version 54678 (0.0010) [2023-12-26 15:45:43,355][105620] Updated weights for policy 1, policy_version 54688 (0.0009) [2023-12-26 15:45:43,415][105620] Updated weights for policy 1, policy_version 54698 (0.0009) [2023-12-26 15:45:43,741][105692] Updated weights for policy 0, policy_version 54360 (0.0008) [2023-12-26 15:45:43,802][105692] Updated weights for policy 0, policy_version 54370 (0.0009) [2023-12-26 15:45:43,867][105692] Updated weights for policy 0, policy_version 54380 (0.0009) [2023-12-26 15:45:44,156][105620] Updated weights for policy 1, policy_version 54708 (0.0009) [2023-12-26 15:45:44,217][105620] Updated weights for policy 1, policy_version 54718 (0.0008) [2023-12-26 15:45:44,275][105620] Updated weights for policy 1, policy_version 54728 (0.0010) [2023-12-26 15:45:44,602][105692] Updated weights for policy 0, policy_version 54390 (0.0008) [2023-12-26 15:45:44,653][105692] Updated weights for policy 0, policy_version 54400 (0.0009) [2023-12-26 15:45:44,700][105692] Updated weights for policy 0, policy_version 54410 (0.0009) [2023-12-26 15:45:45,064][105620] Updated weights for policy 1, policy_version 54738 (0.0009) [2023-12-26 15:45:45,130][105620] Updated weights for policy 1, policy_version 54748 (0.0009) [2023-12-26 15:45:45,183][105620] Updated weights for policy 1, policy_version 54758 (0.0008) [2023-12-26 15:45:45,245][105620] Updated weights for policy 1, policy_version 54768 (0.0008) [2023-12-26 15:45:45,473][105692] Updated weights for policy 0, policy_version 54420 (0.0009) [2023-12-26 15:45:45,520][105692] Updated weights for policy 0, policy_version 54430 (0.0009) [2023-12-26 15:45:45,568][105692] Updated weights for policy 0, policy_version 54440 (0.0009) [2023-12-26 15:45:45,979][105620] Updated weights for policy 1, policy_version 54778 (0.0009) [2023-12-26 15:45:46,034][105620] Updated weights for policy 1, policy_version 54788 (0.0009) [2023-12-26 15:45:46,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19251.2, 300 sec: 19660.8). Total num frames: 27967488. Throughput: 0: 9646.7, 1: 9611.6. Samples: 27942064. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 15:45:46,063][104569] Avg episode reward: [(0, '5497.389'), (1, '9180.468')] [2023-12-26 15:45:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000054448_13942784.pth... [2023-12-26 15:45:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000053328_13656064.pth [2023-12-26 15:45:46,086][105620] Updated weights for policy 1, policy_version 54798 (0.0007) [2023-12-26 15:45:46,095][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000054800_14032896.pth... [2023-12-26 15:45:46,098][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000053648_13737984.pth [2023-12-26 15:45:46,343][105692] Updated weights for policy 0, policy_version 54450 (0.0008) [2023-12-26 15:45:46,417][105692] Updated weights for policy 0, policy_version 54460 (0.0009) [2023-12-26 15:45:46,470][105692] Updated weights for policy 0, policy_version 54470 (0.0008) [2023-12-26 15:45:46,524][105692] Updated weights for policy 0, policy_version 54480 (0.0009) [2023-12-26 15:45:46,857][105620] Updated weights for policy 1, policy_version 54808 (0.0011) [2023-12-26 15:45:46,906][105620] Updated weights for policy 1, policy_version 54818 (0.0010) [2023-12-26 15:45:46,953][105620] Updated weights for policy 1, policy_version 54828 (0.0010) [2023-12-26 15:45:47,208][105692] Updated weights for policy 0, policy_version 54490 (0.0009) [2023-12-26 15:45:47,275][105692] Updated weights for policy 0, policy_version 54500 (0.0010) [2023-12-26 15:45:47,337][105692] Updated weights for policy 0, policy_version 54510 (0.0010) [2023-12-26 15:45:47,570][105620] Updated weights for policy 1, policy_version 54838 (0.0006) [2023-12-26 15:45:47,617][105620] Updated weights for policy 1, policy_version 54848 (0.0007) [2023-12-26 15:45:47,669][105620] Updated weights for policy 1, policy_version 54858 (0.0010) [2023-12-26 15:45:48,037][105692] Updated weights for policy 0, policy_version 54520 (0.0006) [2023-12-26 15:45:48,084][105692] Updated weights for policy 0, policy_version 54530 (0.0005) [2023-12-26 15:45:48,130][105692] Updated weights for policy 0, policy_version 54540 (0.0005) [2023-12-26 15:45:48,300][105620] Updated weights for policy 1, policy_version 54868 (0.0011) [2023-12-26 15:45:48,356][105620] Updated weights for policy 1, policy_version 54878 (0.0011) [2023-12-26 15:45:48,418][105620] Updated weights for policy 1, policy_version 54888 (0.0009) [2023-12-26 15:45:48,707][105692] Updated weights for policy 0, policy_version 54550 (0.0006) [2023-12-26 15:45:48,769][105692] Updated weights for policy 0, policy_version 54560 (0.0005) [2023-12-26 15:45:48,830][105692] Updated weights for policy 0, policy_version 54570 (0.0005) [2023-12-26 15:45:49,051][105620] Updated weights for policy 1, policy_version 54898 (0.0008) [2023-12-26 15:45:49,122][105620] Updated weights for policy 1, policy_version 54908 (0.0010) [2023-12-26 15:45:49,184][105620] Updated weights for policy 1, policy_version 54918 (0.0011) [2023-12-26 15:45:49,244][105620] Updated weights for policy 1, policy_version 54928 (0.0010) [2023-12-26 15:45:49,518][105692] Updated weights for policy 0, policy_version 54580 (0.0009) [2023-12-26 15:45:49,570][105692] Updated weights for policy 0, policy_version 54590 (0.0009) [2023-12-26 15:45:49,624][105692] Updated weights for policy 0, policy_version 54600 (0.0010) [2023-12-26 15:45:49,891][105620] Updated weights for policy 1, policy_version 54938 (0.0008) [2023-12-26 15:45:49,968][105620] Updated weights for policy 1, policy_version 54948 (0.0008) [2023-12-26 15:45:50,032][105620] Updated weights for policy 1, policy_version 54958 (0.0006) [2023-12-26 15:45:50,441][105692] Updated weights for policy 0, policy_version 54610 (0.0009) [2023-12-26 15:45:50,503][105692] Updated weights for policy 0, policy_version 54620 (0.0009) [2023-12-26 15:45:50,559][105692] Updated weights for policy 0, policy_version 54630 (0.0009) [2023-12-26 15:45:50,621][105692] Updated weights for policy 0, policy_version 54640 (0.0008) [2023-12-26 15:45:50,746][105620] Updated weights for policy 1, policy_version 54968 (0.0009) [2023-12-26 15:45:50,802][105620] Updated weights for policy 1, policy_version 54978 (0.0009) [2023-12-26 15:45:50,864][105620] Updated weights for policy 1, policy_version 54988 (0.0009) [2023-12-26 15:45:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19688.6). Total num frames: 28073984. Throughput: 0: 9730.0, 1: 9634.9. Samples: 28061736. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 15:45:51,062][104569] Avg episode reward: [(0, '2759.366'), (1, '9266.794')] [2023-12-26 15:45:51,412][105692] Updated weights for policy 0, policy_version 54650 (0.0009) [2023-12-26 15:45:51,475][105692] Updated weights for policy 0, policy_version 54660 (0.0009) [2023-12-26 15:45:51,534][105692] Updated weights for policy 0, policy_version 54670 (0.0009) [2023-12-26 15:45:51,657][105620] Updated weights for policy 1, policy_version 54998 (0.0009) [2023-12-26 15:45:51,720][105620] Updated weights for policy 1, policy_version 55008 (0.0008) [2023-12-26 15:45:51,790][105620] Updated weights for policy 1, policy_version 55018 (0.0010) [2023-12-26 15:45:52,295][105692] Updated weights for policy 0, policy_version 54680 (0.0008) [2023-12-26 15:45:52,350][105692] Updated weights for policy 0, policy_version 54690 (0.0009) [2023-12-26 15:45:52,409][105692] Updated weights for policy 0, policy_version 54700 (0.0009) [2023-12-26 15:45:52,518][105620] Updated weights for policy 1, policy_version 55028 (0.0009) [2023-12-26 15:45:52,578][105620] Updated weights for policy 1, policy_version 55038 (0.0008) [2023-12-26 15:45:52,629][105620] Updated weights for policy 1, policy_version 55048 (0.0009) [2023-12-26 15:45:53,196][105692] Updated weights for policy 0, policy_version 54710 (0.0009) [2023-12-26 15:45:53,258][105692] Updated weights for policy 0, policy_version 54720 (0.0009) [2023-12-26 15:45:53,311][105692] Updated weights for policy 0, policy_version 54730 (0.0005) [2023-12-26 15:45:53,371][105620] Updated weights for policy 1, policy_version 55058 (0.0008) [2023-12-26 15:45:53,432][105620] Updated weights for policy 1, policy_version 55068 (0.0005) [2023-12-26 15:45:53,498][105620] Updated weights for policy 1, policy_version 55078 (0.0006) [2023-12-26 15:45:53,566][105620] Updated weights for policy 1, policy_version 55088 (0.0006) [2023-12-26 15:45:53,990][105692] Updated weights for policy 0, policy_version 54740 (0.0006) [2023-12-26 15:45:54,044][105692] Updated weights for policy 0, policy_version 54750 (0.0009) [2023-12-26 15:45:54,108][105692] Updated weights for policy 0, policy_version 54760 (0.0008) [2023-12-26 15:45:54,157][105620] Updated weights for policy 1, policy_version 55098 (0.0006) [2023-12-26 15:45:54,204][105620] Updated weights for policy 1, policy_version 55108 (0.0009) [2023-12-26 15:45:54,252][105620] Updated weights for policy 1, policy_version 55118 (0.0010) [2023-12-26 15:45:54,864][105692] Updated weights for policy 0, policy_version 54770 (0.0008) [2023-12-26 15:45:54,877][105620] Updated weights for policy 1, policy_version 55128 (0.0006) [2023-12-26 15:45:54,916][105692] Updated weights for policy 0, policy_version 54780 (0.0009) [2023-12-26 15:45:54,922][105620] Updated weights for policy 1, policy_version 55138 (0.0005) [2023-12-26 15:45:54,976][105692] Updated weights for policy 0, policy_version 54790 (0.0008) [2023-12-26 15:45:54,985][105620] Updated weights for policy 1, policy_version 55148 (0.0008) [2023-12-26 15:45:55,038][105692] Updated weights for policy 0, policy_version 54800 (0.0011) [2023-12-26 15:45:55,720][105620] Updated weights for policy 1, policy_version 55158 (0.0011) [2023-12-26 15:45:55,723][105692] Updated weights for policy 0, policy_version 54810 (0.0009) [2023-12-26 15:45:55,770][105692] Updated weights for policy 0, policy_version 54820 (0.0007) [2023-12-26 15:45:55,772][105620] Updated weights for policy 1, policy_version 55168 (0.0009) [2023-12-26 15:45:55,826][105692] Updated weights for policy 0, policy_version 54830 (0.0007) [2023-12-26 15:45:55,828][105620] Updated weights for policy 1, policy_version 55178 (0.0006) [2023-12-26 15:45:56,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19387.7, 300 sec: 19688.6). Total num frames: 28172288. Throughput: 0: 9599.6, 1: 9725.4. Samples: 28176240. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 15:45:56,062][104569] Avg episode reward: [(0, '5892.509'), (1, '9354.102')] [2023-12-26 15:45:56,534][105692] Updated weights for policy 0, policy_version 54840 (0.0007) [2023-12-26 15:45:56,552][105620] Updated weights for policy 1, policy_version 55188 (0.0008) [2023-12-26 15:45:56,582][105692] Updated weights for policy 0, policy_version 54850 (0.0007) [2023-12-26 15:45:56,604][105620] Updated weights for policy 1, policy_version 55198 (0.0008) [2023-12-26 15:45:56,627][105692] Updated weights for policy 0, policy_version 54860 (0.0010) [2023-12-26 15:45:56,660][105620] Updated weights for policy 1, policy_version 55208 (0.0007) [2023-12-26 15:45:57,362][105692] Updated weights for policy 0, policy_version 54870 (0.0010) [2023-12-26 15:45:57,394][105620] Updated weights for policy 1, policy_version 55218 (0.0007) [2023-12-26 15:45:57,420][105692] Updated weights for policy 0, policy_version 54880 (0.0010) [2023-12-26 15:45:57,446][105620] Updated weights for policy 1, policy_version 55228 (0.0006) [2023-12-26 15:45:57,474][105692] Updated weights for policy 0, policy_version 54890 (0.0010) [2023-12-26 15:45:57,499][105620] Updated weights for policy 1, policy_version 55238 (0.0005) [2023-12-26 15:45:57,553][105620] Updated weights for policy 1, policy_version 55248 (0.0005) [2023-12-26 15:45:58,077][105620] Updated weights for policy 1, policy_version 55258 (0.0006) [2023-12-26 15:45:58,139][105620] Updated weights for policy 1, policy_version 55268 (0.0008) [2023-12-26 15:45:58,150][105692] Updated weights for policy 0, policy_version 54900 (0.0010) [2023-12-26 15:45:58,197][105620] Updated weights for policy 1, policy_version 55278 (0.0007) [2023-12-26 15:45:58,203][105692] Updated weights for policy 0, policy_version 54910 (0.0010) [2023-12-26 15:45:58,263][105692] Updated weights for policy 0, policy_version 54920 (0.0010) [2023-12-26 15:45:58,939][105620] Updated weights for policy 1, policy_version 55288 (0.0008) [2023-12-26 15:45:58,995][105620] Updated weights for policy 1, policy_version 55298 (0.0008) [2023-12-26 15:45:59,026][105692] Updated weights for policy 0, policy_version 54930 (0.0007) [2023-12-26 15:45:59,048][105620] Updated weights for policy 1, policy_version 55308 (0.0006) [2023-12-26 15:45:59,070][105692] Updated weights for policy 0, policy_version 54940 (0.0010) [2023-12-26 15:45:59,114][105692] Updated weights for policy 0, policy_version 54950 (0.0010) [2023-12-26 15:45:59,158][105692] Updated weights for policy 0, policy_version 54960 (0.0010) [2023-12-26 15:45:59,835][105620] Updated weights for policy 1, policy_version 55318 (0.0006) [2023-12-26 15:45:59,861][105692] Updated weights for policy 0, policy_version 54970 (0.0011) [2023-12-26 15:45:59,891][105620] Updated weights for policy 1, policy_version 55328 (0.0006) [2023-12-26 15:45:59,917][105692] Updated weights for policy 0, policy_version 54980 (0.0010) [2023-12-26 15:45:59,954][105620] Updated weights for policy 1, policy_version 55338 (0.0007) [2023-12-26 15:45:59,977][105692] Updated weights for policy 0, policy_version 54990 (0.0008) [2023-12-26 15:46:00,688][105692] Updated weights for policy 0, policy_version 55000 (0.0006) [2023-12-26 15:46:00,698][105620] Updated weights for policy 1, policy_version 55348 (0.0006) [2023-12-26 15:46:00,741][105620] Updated weights for policy 1, policy_version 55358 (0.0005) [2023-12-26 15:46:00,756][105692] Updated weights for policy 0, policy_version 55010 (0.0005) [2023-12-26 15:46:00,790][105620] Updated weights for policy 1, policy_version 55368 (0.0007) [2023-12-26 15:46:00,818][105692] Updated weights for policy 0, policy_version 55020 (0.0008) [2023-12-26 15:46:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19716.3). Total num frames: 28270592. Throughput: 0: 9618.1, 1: 9810.0. Samples: 28236440. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-12-26 15:46:01,063][104569] Avg episode reward: [(0, '8055.311'), (1, '8077.619')] [2023-12-26 15:46:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000055024_14090240.pth... [2023-12-26 15:46:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000055376_14180352.pth... [2023-12-26 15:46:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000053904_13803520.pth [2023-12-26 15:46:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000054256_13893632.pth [2023-12-26 15:46:01,454][105692] Updated weights for policy 0, policy_version 55030 (0.0009) [2023-12-26 15:46:01,503][105620] Updated weights for policy 1, policy_version 55378 (0.0008) [2023-12-26 15:46:01,517][105692] Updated weights for policy 0, policy_version 55040 (0.0008) [2023-12-26 15:46:01,548][105620] Updated weights for policy 1, policy_version 55388 (0.0005) [2023-12-26 15:46:01,575][105692] Updated weights for policy 0, policy_version 55050 (0.0007) [2023-12-26 15:46:01,602][105620] Updated weights for policy 1, policy_version 55398 (0.0008) [2023-12-26 15:46:01,663][105620] Updated weights for policy 1, policy_version 55408 (0.0009) [2023-12-26 15:46:02,307][105620] Updated weights for policy 1, policy_version 55418 (0.0007) [2023-12-26 15:46:02,368][105620] Updated weights for policy 1, policy_version 55428 (0.0009) [2023-12-26 15:46:02,413][105692] Updated weights for policy 0, policy_version 55060 (0.0006) [2023-12-26 15:46:02,433][105620] Updated weights for policy 1, policy_version 55438 (0.0008) [2023-12-26 15:46:02,465][105692] Updated weights for policy 0, policy_version 55070 (0.0008) [2023-12-26 15:46:02,518][105692] Updated weights for policy 0, policy_version 55080 (0.0010) [2023-12-26 15:46:03,037][105620] Updated weights for policy 1, policy_version 55448 (0.0009) [2023-12-26 15:46:03,088][105620] Updated weights for policy 1, policy_version 55458 (0.0010) [2023-12-26 15:46:03,140][105620] Updated weights for policy 1, policy_version 55468 (0.0008) [2023-12-26 15:46:03,270][105692] Updated weights for policy 0, policy_version 55090 (0.0008) [2023-12-26 15:46:03,338][105692] Updated weights for policy 0, policy_version 55100 (0.0005) [2023-12-26 15:46:03,386][105692] Updated weights for policy 0, policy_version 55110 (0.0007) [2023-12-26 15:46:03,432][105692] Updated weights for policy 0, policy_version 55120 (0.0005) [2023-12-26 15:46:03,899][105620] Updated weights for policy 1, policy_version 55478 (0.0010) [2023-12-26 15:46:03,954][105620] Updated weights for policy 1, policy_version 55488 (0.0011) [2023-12-26 15:46:03,986][105692] Updated weights for policy 0, policy_version 55130 (0.0008) [2023-12-26 15:46:04,018][105620] Updated weights for policy 1, policy_version 55498 (0.0011) [2023-12-26 15:46:04,046][105692] Updated weights for policy 0, policy_version 55140 (0.0011) [2023-12-26 15:46:04,105][105692] Updated weights for policy 0, policy_version 55150 (0.0011) [2023-12-26 15:46:04,781][105620] Updated weights for policy 1, policy_version 55508 (0.0011) [2023-12-26 15:46:04,833][105620] Updated weights for policy 1, policy_version 55518 (0.0010) [2023-12-26 15:46:04,839][105692] Updated weights for policy 0, policy_version 55160 (0.0007) [2023-12-26 15:46:04,878][105620] Updated weights for policy 1, policy_version 55528 (0.0010) [2023-12-26 15:46:04,888][105692] Updated weights for policy 0, policy_version 55170 (0.0005) [2023-12-26 15:46:04,934][105692] Updated weights for policy 0, policy_version 55180 (0.0006) [2023-12-26 15:46:05,517][105620] Updated weights for policy 1, policy_version 55538 (0.0009) [2023-12-26 15:46:05,575][105620] Updated weights for policy 1, policy_version 55548 (0.0006) [2023-12-26 15:46:05,628][105620] Updated weights for policy 1, policy_version 55558 (0.0010) [2023-12-26 15:46:05,637][105692] Updated weights for policy 0, policy_version 55190 (0.0008) [2023-12-26 15:46:05,677][105620] Updated weights for policy 1, policy_version 55568 (0.0010) [2023-12-26 15:46:05,700][105692] Updated weights for policy 0, policy_version 55200 (0.0008) [2023-12-26 15:46:05,769][105692] Updated weights for policy 0, policy_version 55210 (0.0010) [2023-12-26 15:46:06,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19387.7, 300 sec: 19744.1). Total num frames: 28368896. Throughput: 0: 9666.1, 1: 9739.7. Samples: 28354156. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-12-26 15:46:06,063][104569] Avg episode reward: [(0, '8417.320'), (1, '5276.710')] [2023-12-26 15:46:06,331][105620] Updated weights for policy 1, policy_version 55578 (0.0011) [2023-12-26 15:46:06,393][105620] Updated weights for policy 1, policy_version 55588 (0.0007) [2023-12-26 15:46:06,462][105620] Updated weights for policy 1, policy_version 55598 (0.0010) [2023-12-26 15:46:06,519][105692] Updated weights for policy 0, policy_version 55220 (0.0009) [2023-12-26 15:46:06,578][105692] Updated weights for policy 0, policy_version 55230 (0.0008) [2023-12-26 15:46:06,641][105692] Updated weights for policy 0, policy_version 55240 (0.0009) [2023-12-26 15:46:07,160][105620] Updated weights for policy 1, policy_version 55608 (0.0010) [2023-12-26 15:46:07,228][105620] Updated weights for policy 1, policy_version 55618 (0.0009) [2023-12-26 15:46:07,287][105620] Updated weights for policy 1, policy_version 55628 (0.0008) [2023-12-26 15:46:07,427][105692] Updated weights for policy 0, policy_version 55250 (0.0008) [2023-12-26 15:46:07,489][105692] Updated weights for policy 0, policy_version 55260 (0.0008) [2023-12-26 15:46:07,555][105692] Updated weights for policy 0, policy_version 55270 (0.0009) [2023-12-26 15:46:07,619][105692] Updated weights for policy 0, policy_version 55280 (0.0008) [2023-12-26 15:46:07,966][105620] Updated weights for policy 1, policy_version 55638 (0.0008) [2023-12-26 15:46:08,027][105620] Updated weights for policy 1, policy_version 55648 (0.0005) [2023-12-26 15:46:08,090][105620] Updated weights for policy 1, policy_version 55658 (0.0008) [2023-12-26 15:46:08,247][105692] Updated weights for policy 0, policy_version 55290 (0.0008) [2023-12-26 15:46:08,301][105692] Updated weights for policy 0, policy_version 55300 (0.0005) [2023-12-26 15:46:08,361][105692] Updated weights for policy 0, policy_version 55310 (0.0007) [2023-12-26 15:46:08,865][105620] Updated weights for policy 1, policy_version 55668 (0.0009) [2023-12-26 15:46:08,917][105620] Updated weights for policy 1, policy_version 55678 (0.0010) [2023-12-26 15:46:08,978][105620] Updated weights for policy 1, policy_version 55688 (0.0009) [2023-12-26 15:46:08,997][105692] Updated weights for policy 0, policy_version 55320 (0.0006) [2023-12-26 15:46:09,057][105692] Updated weights for policy 0, policy_version 55330 (0.0007) [2023-12-26 15:46:09,112][105692] Updated weights for policy 0, policy_version 55340 (0.0009) [2023-12-26 15:46:09,723][105620] Updated weights for policy 1, policy_version 55698 (0.0007) [2023-12-26 15:46:09,783][105620] Updated weights for policy 1, policy_version 55708 (0.0006) [2023-12-26 15:46:09,856][105620] Updated weights for policy 1, policy_version 55718 (0.0008) [2023-12-26 15:46:09,917][105620] Updated weights for policy 1, policy_version 55728 (0.0008) [2023-12-26 15:46:09,934][105692] Updated weights for policy 0, policy_version 55350 (0.0009) [2023-12-26 15:46:10,000][105692] Updated weights for policy 0, policy_version 55360 (0.0009) [2023-12-26 15:46:10,055][105692] Updated weights for policy 0, policy_version 55370 (0.0009) [2023-12-26 15:46:10,551][105620] Updated weights for policy 1, policy_version 55738 (0.0009) [2023-12-26 15:46:10,596][105620] Updated weights for policy 1, policy_version 55748 (0.0009) [2023-12-26 15:46:10,649][105620] Updated weights for policy 1, policy_version 55758 (0.0010) [2023-12-26 15:46:10,840][105692] Updated weights for policy 0, policy_version 55380 (0.0009) [2023-12-26 15:46:10,895][105692] Updated weights for policy 0, policy_version 55390 (0.0008) [2023-12-26 15:46:10,952][105692] Updated weights for policy 0, policy_version 55400 (0.0009) [2023-12-26 15:46:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19744.1). Total num frames: 28467200. Throughput: 0: 9674.2, 1: 9756.1. Samples: 28471572. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-12-26 15:46:11,062][104569] Avg episode reward: [(0, '7433.424'), (1, '7139.702')] [2023-12-26 15:46:11,411][105620] Updated weights for policy 1, policy_version 55768 (0.0009) [2023-12-26 15:46:11,476][105620] Updated weights for policy 1, policy_version 55778 (0.0009) [2023-12-26 15:46:11,542][105620] Updated weights for policy 1, policy_version 55788 (0.0009) [2023-12-26 15:46:11,647][105692] Updated weights for policy 0, policy_version 55410 (0.0009) [2023-12-26 15:46:11,706][105692] Updated weights for policy 0, policy_version 55420 (0.0008) [2023-12-26 15:46:11,772][105692] Updated weights for policy 0, policy_version 55430 (0.0008) [2023-12-26 15:46:11,827][105692] Updated weights for policy 0, policy_version 55440 (0.0008) [2023-12-26 15:46:12,281][105620] Updated weights for policy 1, policy_version 55798 (0.0009) [2023-12-26 15:46:12,339][105620] Updated weights for policy 1, policy_version 55808 (0.0008) [2023-12-26 15:46:12,403][105620] Updated weights for policy 1, policy_version 55818 (0.0007) [2023-12-26 15:46:12,578][105692] Updated weights for policy 0, policy_version 55450 (0.0008) [2023-12-26 15:46:12,640][105692] Updated weights for policy 0, policy_version 55460 (0.0007) [2023-12-26 15:46:12,707][105692] Updated weights for policy 0, policy_version 55470 (0.0008) [2023-12-26 15:46:13,078][105620] Updated weights for policy 1, policy_version 55828 (0.0006) [2023-12-26 15:46:13,142][105620] Updated weights for policy 1, policy_version 55838 (0.0006) [2023-12-26 15:46:13,210][105620] Updated weights for policy 1, policy_version 55848 (0.0006) [2023-12-26 15:46:13,468][105692] Updated weights for policy 0, policy_version 55480 (0.0009) [2023-12-26 15:46:13,521][105692] Updated weights for policy 0, policy_version 55491 (0.0010) [2023-12-26 15:46:13,588][105692] Updated weights for policy 0, policy_version 55501 (0.0010) [2023-12-26 15:46:13,713][105620] Updated weights for policy 1, policy_version 55858 (0.0005) [2023-12-26 15:46:13,784][105620] Updated weights for policy 1, policy_version 55868 (0.0005) [2023-12-26 15:46:13,835][105620] Updated weights for policy 1, policy_version 55878 (0.0005) [2023-12-26 15:46:13,894][105620] Updated weights for policy 1, policy_version 55888 (0.0006) [2023-12-26 15:46:14,417][105692] Updated weights for policy 0, policy_version 55511 (0.0008) [2023-12-26 15:46:14,477][105692] Updated weights for policy 0, policy_version 55521 (0.0008) [2023-12-26 15:46:14,479][105620] Updated weights for policy 1, policy_version 55898 (0.0006) [2023-12-26 15:46:14,530][105692] Updated weights for policy 0, policy_version 55531 (0.0007) [2023-12-26 15:46:14,537][105620] Updated weights for policy 1, policy_version 55908 (0.0009) [2023-12-26 15:46:14,587][105620] Updated weights for policy 1, policy_version 55918 (0.0009) [2023-12-26 15:46:15,247][105620] Updated weights for policy 1, policy_version 55928 (0.0006) [2023-12-26 15:46:15,264][105692] Updated weights for policy 0, policy_version 55541 (0.0009) [2023-12-26 15:46:15,313][105620] Updated weights for policy 1, policy_version 55938 (0.0006) [2023-12-26 15:46:15,333][105692] Updated weights for policy 0, policy_version 55551 (0.0006) [2023-12-26 15:46:15,382][105620] Updated weights for policy 1, policy_version 55948 (0.0006) [2023-12-26 15:46:15,398][105692] Updated weights for policy 0, policy_version 55561 (0.0005) [2023-12-26 15:46:16,005][105692] Updated weights for policy 0, policy_version 55571 (0.0005) [2023-12-26 15:46:16,010][105620] Updated weights for policy 1, policy_version 55958 (0.0007) [2023-12-26 15:46:16,057][105692] Updated weights for policy 0, policy_version 55581 (0.0005) [2023-12-26 15:46:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19688.6). Total num frames: 28557312. Throughput: 0: 9624.8, 1: 9771.3. Samples: 28530928. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-12-26 15:46:16,063][104569] Avg episode reward: [(0, '7691.429'), (1, '9003.233')] [2023-12-26 15:46:16,063][105620] Updated weights for policy 1, policy_version 55968 (0.0006) [2023-12-26 15:46:16,116][105692] Updated weights for policy 0, policy_version 55591 (0.0008) [2023-12-26 15:46:16,122][105620] Updated weights for policy 1, policy_version 55978 (0.0006) [2023-12-26 15:46:16,159][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000055984_14336000.pth... [2023-12-26 15:46:16,163][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000054800_14032896.pth [2023-12-26 15:46:16,178][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000055600_14237696.pth... [2023-12-26 15:46:16,181][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000054448_13942784.pth [2023-12-26 15:46:16,658][105620] Updated weights for policy 1, policy_version 55988 (0.0007) [2023-12-26 15:46:16,720][105620] Updated weights for policy 1, policy_version 55999 (0.0007) [2023-12-26 15:46:16,766][105692] Updated weights for policy 0, policy_version 55601 (0.0010) [2023-12-26 15:46:16,777][105620] Updated weights for policy 1, policy_version 56009 (0.0006) [2023-12-26 15:46:16,826][105692] Updated weights for policy 0, policy_version 55611 (0.0006) [2023-12-26 15:46:16,878][105692] Updated weights for policy 0, policy_version 55621 (0.0007) [2023-12-26 15:46:16,931][105692] Updated weights for policy 0, policy_version 55631 (0.0005) [2023-12-26 15:46:17,505][105620] Updated weights for policy 1, policy_version 56019 (0.0007) [2023-12-26 15:46:17,560][105620] Updated weights for policy 1, policy_version 56029 (0.0008) [2023-12-26 15:46:17,610][105620] Updated weights for policy 1, policy_version 56039 (0.0008) [2023-12-26 15:46:17,628][105692] Updated weights for policy 0, policy_version 55641 (0.0010) [2023-12-26 15:46:17,683][105692] Updated weights for policy 0, policy_version 55651 (0.0010) [2023-12-26 15:46:17,737][105692] Updated weights for policy 0, policy_version 55661 (0.0010) [2023-12-26 15:46:18,364][105620] Updated weights for policy 1, policy_version 56049 (0.0006) [2023-12-26 15:46:18,424][105620] Updated weights for policy 1, policy_version 56059 (0.0008) [2023-12-26 15:46:18,489][105620] Updated weights for policy 1, policy_version 56069 (0.0008) [2023-12-26 15:46:18,534][105692] Updated weights for policy 0, policy_version 55671 (0.0009) [2023-12-26 15:46:18,541][105620] Updated weights for policy 1, policy_version 56079 (0.0007) [2023-12-26 15:46:18,586][105692] Updated weights for policy 0, policy_version 55681 (0.0010) [2023-12-26 15:46:18,651][105692] Updated weights for policy 0, policy_version 55691 (0.0010) [2023-12-26 15:46:19,175][105620] Updated weights for policy 1, policy_version 56089 (0.0005) [2023-12-26 15:46:19,242][105620] Updated weights for policy 1, policy_version 56099 (0.0006) [2023-12-26 15:46:19,311][105620] Updated weights for policy 1, policy_version 56109 (0.0008) [2023-12-26 15:46:19,392][105692] Updated weights for policy 0, policy_version 55701 (0.0010) [2023-12-26 15:46:19,458][105692] Updated weights for policy 0, policy_version 55711 (0.0005) [2023-12-26 15:46:19,524][105692] Updated weights for policy 0, policy_version 55721 (0.0007) [2023-12-26 15:46:20,022][105620] Updated weights for policy 1, policy_version 56119 (0.0010) [2023-12-26 15:46:20,088][105620] Updated weights for policy 1, policy_version 56129 (0.0011) [2023-12-26 15:46:20,148][105620] Updated weights for policy 1, policy_version 56139 (0.0011) [2023-12-26 15:46:20,220][105692] Updated weights for policy 0, policy_version 55731 (0.0008) [2023-12-26 15:46:20,280][105692] Updated weights for policy 0, policy_version 55741 (0.0008) [2023-12-26 15:46:20,342][105692] Updated weights for policy 0, policy_version 55751 (0.0006) [2023-12-26 15:46:20,905][105620] Updated weights for policy 1, policy_version 56149 (0.0011) [2023-12-26 15:46:20,972][105620] Updated weights for policy 1, policy_version 56159 (0.0011) [2023-12-26 15:46:21,037][105620] Updated weights for policy 1, policy_version 56169 (0.0011) [2023-12-26 15:46:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19688.6). Total num frames: 28655616. Throughput: 0: 9585.5, 1: 9920.4. Samples: 28651916. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-12-26 15:46:21,062][104569] Avg episode reward: [(0, '7970.160'), (1, '9094.988')] [2023-12-26 15:46:21,071][105692] Updated weights for policy 0, policy_version 55761 (0.0009) [2023-12-26 15:46:21,143][105692] Updated weights for policy 0, policy_version 55771 (0.0010) [2023-12-26 15:46:21,211][105692] Updated weights for policy 0, policy_version 55781 (0.0008) [2023-12-26 15:46:21,284][105692] Updated weights for policy 0, policy_version 55791 (0.0009) [2023-12-26 15:46:21,801][105620] Updated weights for policy 1, policy_version 56179 (0.0010) [2023-12-26 15:46:21,865][105620] Updated weights for policy 1, policy_version 56189 (0.0011) [2023-12-26 15:46:21,931][105620] Updated weights for policy 1, policy_version 56199 (0.0011) [2023-12-26 15:46:22,035][105692] Updated weights for policy 0, policy_version 55801 (0.0007) [2023-12-26 15:46:22,091][105692] Updated weights for policy 0, policy_version 55811 (0.0008) [2023-12-26 15:46:22,155][105692] Updated weights for policy 0, policy_version 55821 (0.0008) [2023-12-26 15:46:22,642][105620] Updated weights for policy 1, policy_version 56209 (0.0011) [2023-12-26 15:46:22,703][105620] Updated weights for policy 1, policy_version 56219 (0.0011) [2023-12-26 15:46:22,757][105620] Updated weights for policy 1, policy_version 56229 (0.0011) [2023-12-26 15:46:22,818][105620] Updated weights for policy 1, policy_version 56239 (0.0011) [2023-12-26 15:46:22,940][105692] Updated weights for policy 0, policy_version 55831 (0.0008) [2023-12-26 15:46:22,987][105692] Updated weights for policy 0, policy_version 55841 (0.0008) [2023-12-26 15:46:23,048][105692] Updated weights for policy 0, policy_version 55851 (0.0009) [2023-12-26 15:46:23,543][105620] Updated weights for policy 1, policy_version 56249 (0.0010) [2023-12-26 15:46:23,597][105620] Updated weights for policy 1, policy_version 56259 (0.0009) [2023-12-26 15:46:23,648][105692] Updated weights for policy 0, policy_version 55861 (0.0008) [2023-12-26 15:46:23,651][105620] Updated weights for policy 1, policy_version 56269 (0.0009) [2023-12-26 15:46:23,697][105692] Updated weights for policy 0, policy_version 55871 (0.0006) [2023-12-26 15:46:23,741][105692] Updated weights for policy 0, policy_version 55881 (0.0005) [2023-12-26 15:46:24,285][105692] Updated weights for policy 0, policy_version 55891 (0.0007) [2023-12-26 15:46:24,346][105692] Updated weights for policy 0, policy_version 55901 (0.0010) [2023-12-26 15:46:24,415][105692] Updated weights for policy 0, policy_version 55911 (0.0010) [2023-12-26 15:46:24,435][105620] Updated weights for policy 1, policy_version 56279 (0.0006) [2023-12-26 15:46:24,503][105620] Updated weights for policy 1, policy_version 56289 (0.0005) [2023-12-26 15:46:24,561][105620] Updated weights for policy 1, policy_version 56299 (0.0010) [2023-12-26 15:46:25,028][105692] Updated weights for policy 0, policy_version 55921 (0.0010) [2023-12-26 15:46:25,093][105692] Updated weights for policy 0, policy_version 55931 (0.0005) [2023-12-26 15:46:25,144][105692] Updated weights for policy 0, policy_version 55941 (0.0007) [2023-12-26 15:46:25,194][105692] Updated weights for policy 0, policy_version 55951 (0.0006) [2023-12-26 15:46:25,231][105620] Updated weights for policy 1, policy_version 56309 (0.0011) [2023-12-26 15:46:25,295][105620] Updated weights for policy 1, policy_version 56319 (0.0010) [2023-12-26 15:46:25,352][105620] Updated weights for policy 1, policy_version 56329 (0.0010) [2023-12-26 15:46:25,739][105692] Updated weights for policy 0, policy_version 55961 (0.0005) [2023-12-26 15:46:25,787][105692] Updated weights for policy 0, policy_version 55971 (0.0006) [2023-12-26 15:46:25,834][105692] Updated weights for policy 0, policy_version 55981 (0.0009) [2023-12-26 15:46:25,959][105620] Updated weights for policy 1, policy_version 56339 (0.0010) [2023-12-26 15:46:26,010][105620] Updated weights for policy 1, policy_version 56349 (0.0005) [2023-12-26 15:46:26,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19716.3). Total num frames: 28762112. Throughput: 0: 9771.4, 1: 9949.0. Samples: 28772340. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-12-26 15:46:26,062][104569] Avg episode reward: [(0, '2373.076'), (1, '9095.595')] [2023-12-26 15:46:26,065][105620] Updated weights for policy 1, policy_version 56359 (0.0009) [2023-12-26 15:46:26,461][105692] Updated weights for policy 0, policy_version 55991 (0.0009) [2023-12-26 15:46:26,516][105692] Updated weights for policy 0, policy_version 56001 (0.0005) [2023-12-26 15:46:26,585][105692] Updated weights for policy 0, policy_version 56011 (0.0005) [2023-12-26 15:46:26,782][105620] Updated weights for policy 1, policy_version 56369 (0.0010) [2023-12-26 15:46:26,835][105620] Updated weights for policy 1, policy_version 56379 (0.0010) [2023-12-26 15:46:26,895][105620] Updated weights for policy 1, policy_version 56389 (0.0008) [2023-12-26 15:46:26,953][105620] Updated weights for policy 1, policy_version 56399 (0.0007) [2023-12-26 15:46:27,256][105692] Updated weights for policy 0, policy_version 56021 (0.0007) [2023-12-26 15:46:27,313][105692] Updated weights for policy 0, policy_version 56031 (0.0009) [2023-12-26 15:46:27,369][105692] Updated weights for policy 0, policy_version 56041 (0.0009) [2023-12-26 15:46:27,523][105620] Updated weights for policy 1, policy_version 56409 (0.0008) [2023-12-26 15:46:27,570][105620] Updated weights for policy 1, policy_version 56419 (0.0009) [2023-12-26 15:46:27,615][105620] Updated weights for policy 1, policy_version 56429 (0.0008) [2023-12-26 15:46:28,042][105692] Updated weights for policy 0, policy_version 56051 (0.0008) [2023-12-26 15:46:28,099][105692] Updated weights for policy 0, policy_version 56061 (0.0005) [2023-12-26 15:46:28,151][105692] Updated weights for policy 0, policy_version 56071 (0.0009) [2023-12-26 15:46:28,469][105620] Updated weights for policy 1, policy_version 56439 (0.0008) [2023-12-26 15:46:28,524][105620] Updated weights for policy 1, policy_version 56449 (0.0008) [2023-12-26 15:46:28,572][105620] Updated weights for policy 1, policy_version 56459 (0.0008) [2023-12-26 15:46:28,813][105692] Updated weights for policy 0, policy_version 56081 (0.0009) [2023-12-26 15:46:28,871][105692] Updated weights for policy 0, policy_version 56091 (0.0010) [2023-12-26 15:46:28,930][105692] Updated weights for policy 0, policy_version 56101 (0.0010) [2023-12-26 15:46:28,992][105692] Updated weights for policy 0, policy_version 56111 (0.0010) [2023-12-26 15:46:29,374][105620] Updated weights for policy 1, policy_version 56469 (0.0008) [2023-12-26 15:46:29,425][105620] Updated weights for policy 1, policy_version 56479 (0.0008) [2023-12-26 15:46:29,470][105620] Updated weights for policy 1, policy_version 56489 (0.0008) [2023-12-26 15:46:29,750][105692] Updated weights for policy 0, policy_version 56121 (0.0010) [2023-12-26 15:46:29,817][105692] Updated weights for policy 0, policy_version 56131 (0.0010) [2023-12-26 15:46:29,878][105692] Updated weights for policy 0, policy_version 56141 (0.0007) [2023-12-26 15:46:30,141][105620] Updated weights for policy 1, policy_version 56499 (0.0007) [2023-12-26 15:46:30,196][105620] Updated weights for policy 1, policy_version 56509 (0.0006) [2023-12-26 15:46:30,261][105620] Updated weights for policy 1, policy_version 56519 (0.0009) [2023-12-26 15:46:30,531][105692] Updated weights for policy 0, policy_version 56151 (0.0009) [2023-12-26 15:46:30,582][105692] Updated weights for policy 0, policy_version 56161 (0.0010) [2023-12-26 15:46:30,633][105692] Updated weights for policy 0, policy_version 56171 (0.0010) [2023-12-26 15:46:30,961][105620] Updated weights for policy 1, policy_version 56529 (0.0007) [2023-12-26 15:46:31,019][105620] Updated weights for policy 1, policy_version 56539 (0.0008) [2023-12-26 15:46:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19716.3). Total num frames: 28860416. Throughput: 0: 9804.4, 1: 9986.5. Samples: 28832652. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-12-26 15:46:31,062][104569] Avg episode reward: [(0, '1479.930'), (1, '9358.430')] [2023-12-26 15:46:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000056176_14385152.pth... [2023-12-26 15:46:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000055024_14090240.pth [2023-12-26 15:46:31,089][105620] Updated weights for policy 1, policy_version 56549 (0.0008) [2023-12-26 15:46:31,155][105620] Updated weights for policy 1, policy_version 56559 (0.0009) [2023-12-26 15:46:31,158][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000056560_14483456.pth... [2023-12-26 15:46:31,161][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000055376_14180352.pth [2023-12-26 15:46:31,161][105586] Saving new best policy, reward=9358.430! [2023-12-26 15:46:31,323][105692] Updated weights for policy 0, policy_version 56181 (0.0008) [2023-12-26 15:46:31,373][105692] Updated weights for policy 0, policy_version 56191 (0.0006) [2023-12-26 15:46:31,445][105692] Updated weights for policy 0, policy_version 56201 (0.0009) [2023-12-26 15:46:31,981][105620] Updated weights for policy 1, policy_version 56569 (0.0009) [2023-12-26 15:46:32,038][105692] Updated weights for policy 0, policy_version 56211 (0.0009) [2023-12-26 15:46:32,043][105620] Updated weights for policy 1, policy_version 56579 (0.0009) [2023-12-26 15:46:32,097][105692] Updated weights for policy 0, policy_version 56221 (0.0011) [2023-12-26 15:46:32,103][105620] Updated weights for policy 1, policy_version 56589 (0.0007) [2023-12-26 15:46:32,159][105692] Updated weights for policy 0, policy_version 56231 (0.0010) [2023-12-26 15:46:32,815][105692] Updated weights for policy 0, policy_version 56241 (0.0010) [2023-12-26 15:46:32,863][105692] Updated weights for policy 0, policy_version 56251 (0.0010) [2023-12-26 15:46:32,867][105620] Updated weights for policy 1, policy_version 56599 (0.0005) [2023-12-26 15:46:32,907][105692] Updated weights for policy 0, policy_version 56261 (0.0010) [2023-12-26 15:46:32,914][105620] Updated weights for policy 1, policy_version 56609 (0.0005) [2023-12-26 15:46:32,951][105692] Updated weights for policy 0, policy_version 56271 (0.0010) [2023-12-26 15:46:32,963][105620] Updated weights for policy 1, policy_version 56619 (0.0005) [2023-12-26 15:46:33,580][105620] Updated weights for policy 1, policy_version 56629 (0.0007) [2023-12-26 15:46:33,631][105620] Updated weights for policy 1, policy_version 56639 (0.0007) [2023-12-26 15:46:33,681][105620] Updated weights for policy 1, policy_version 56649 (0.0008) [2023-12-26 15:46:33,714][105692] Updated weights for policy 0, policy_version 56281 (0.0010) [2023-12-26 15:46:33,773][105692] Updated weights for policy 0, policy_version 56291 (0.0010) [2023-12-26 15:46:33,828][105692] Updated weights for policy 0, policy_version 56301 (0.0010) [2023-12-26 15:46:34,449][105620] Updated weights for policy 1, policy_version 56659 (0.0006) [2023-12-26 15:46:34,511][105620] Updated weights for policy 1, policy_version 56669 (0.0008) [2023-12-26 15:46:34,565][105620] Updated weights for policy 1, policy_version 56679 (0.0007) [2023-12-26 15:46:34,570][105692] Updated weights for policy 0, policy_version 56311 (0.0010) [2023-12-26 15:46:34,629][105692] Updated weights for policy 0, policy_version 56321 (0.0010) [2023-12-26 15:46:34,695][105692] Updated weights for policy 0, policy_version 56331 (0.0010) [2023-12-26 15:46:35,306][105620] Updated weights for policy 1, policy_version 56689 (0.0006) [2023-12-26 15:46:35,370][105620] Updated weights for policy 1, policy_version 56699 (0.0005) [2023-12-26 15:46:35,426][105692] Updated weights for policy 0, policy_version 56341 (0.0010) [2023-12-26 15:46:35,428][105620] Updated weights for policy 1, policy_version 56709 (0.0006) [2023-12-26 15:46:35,486][105692] Updated weights for policy 0, policy_version 56351 (0.0005) [2023-12-26 15:46:35,486][105620] Updated weights for policy 1, policy_version 56719 (0.0009) [2023-12-26 15:46:35,537][105692] Updated weights for policy 0, policy_version 56361 (0.0007) [2023-12-26 15:46:36,061][105620] Updated weights for policy 1, policy_version 56729 (0.0009) [2023-12-26 15:46:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19716.3). Total num frames: 28958720. Throughput: 0: 9832.2, 1: 9910.9. Samples: 28950176. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 15:46:36,062][104569] Avg episode reward: [(0, '6544.418'), (1, '9357.049')] [2023-12-26 15:46:36,133][105620] Updated weights for policy 1, policy_version 56739 (0.0006) [2023-12-26 15:46:36,202][105620] Updated weights for policy 1, policy_version 56749 (0.0008) [2023-12-26 15:46:36,277][105692] Updated weights for policy 0, policy_version 56371 (0.0006) [2023-12-26 15:46:36,343][105692] Updated weights for policy 0, policy_version 56381 (0.0009) [2023-12-26 15:46:36,401][105692] Updated weights for policy 0, policy_version 56391 (0.0009) [2023-12-26 15:46:36,915][105620] Updated weights for policy 1, policy_version 56759 (0.0009) [2023-12-26 15:46:36,975][105620] Updated weights for policy 1, policy_version 56769 (0.0008) [2023-12-26 15:46:37,047][105620] Updated weights for policy 1, policy_version 56779 (0.0006) [2023-12-26 15:46:37,144][105692] Updated weights for policy 0, policy_version 56401 (0.0009) [2023-12-26 15:46:37,202][105692] Updated weights for policy 0, policy_version 56411 (0.0009) [2023-12-26 15:46:37,263][105692] Updated weights for policy 0, policy_version 56421 (0.0008) [2023-12-26 15:46:37,337][105692] Updated weights for policy 0, policy_version 56431 (0.0010) [2023-12-26 15:46:37,743][105620] Updated weights for policy 1, policy_version 56789 (0.0008) [2023-12-26 15:46:37,790][105620] Updated weights for policy 1, policy_version 56799 (0.0010) [2023-12-26 15:46:37,850][105620] Updated weights for policy 1, policy_version 56809 (0.0009) [2023-12-26 15:46:38,109][105692] Updated weights for policy 0, policy_version 56441 (0.0006) [2023-12-26 15:46:38,180][105692] Updated weights for policy 0, policy_version 56451 (0.0005) [2023-12-26 15:46:38,238][105692] Updated weights for policy 0, policy_version 56461 (0.0006) [2023-12-26 15:46:38,536][105620] Updated weights for policy 1, policy_version 56819 (0.0009) [2023-12-26 15:46:38,591][105620] Updated weights for policy 1, policy_version 56829 (0.0005) [2023-12-26 15:46:38,648][105620] Updated weights for policy 1, policy_version 56839 (0.0005) [2023-12-26 15:46:38,916][105692] Updated weights for policy 0, policy_version 56471 (0.0008) [2023-12-26 15:46:38,974][105692] Updated weights for policy 0, policy_version 56481 (0.0009) [2023-12-26 15:46:39,034][105692] Updated weights for policy 0, policy_version 56491 (0.0009) [2023-12-26 15:46:39,358][105620] Updated weights for policy 1, policy_version 56849 (0.0008) [2023-12-26 15:46:39,433][105620] Updated weights for policy 1, policy_version 56859 (0.0009) [2023-12-26 15:46:39,500][105620] Updated weights for policy 1, policy_version 56869 (0.0005) [2023-12-26 15:46:39,563][105620] Updated weights for policy 1, policy_version 56879 (0.0006) [2023-12-26 15:46:39,845][105692] Updated weights for policy 0, policy_version 56501 (0.0008) [2023-12-26 15:46:39,906][105692] Updated weights for policy 0, policy_version 56511 (0.0008) [2023-12-26 15:46:39,970][105692] Updated weights for policy 0, policy_version 56521 (0.0009) [2023-12-26 15:46:40,135][105620] Updated weights for policy 1, policy_version 56889 (0.0010) [2023-12-26 15:46:40,184][105620] Updated weights for policy 1, policy_version 56899 (0.0010) [2023-12-26 15:46:40,240][105620] Updated weights for policy 1, policy_version 56909 (0.0010) [2023-12-26 15:46:40,708][105692] Updated weights for policy 0, policy_version 56531 (0.0009) [2023-12-26 15:46:40,757][105692] Updated weights for policy 0, policy_version 56541 (0.0008) [2023-12-26 15:46:40,806][105692] Updated weights for policy 0, policy_version 56551 (0.0008) [2023-12-26 15:46:41,006][105620] Updated weights for policy 1, policy_version 56919 (0.0010) [2023-12-26 15:46:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19716.4). Total num frames: 29057024. Throughput: 0: 9831.2, 1: 9955.1. Samples: 29066624. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 15:46:41,062][104569] Avg episode reward: [(0, '7823.966'), (1, '9263.720')] [2023-12-26 15:46:41,068][105620] Updated weights for policy 1, policy_version 56929 (0.0010) [2023-12-26 15:46:41,127][105620] Updated weights for policy 1, policy_version 56939 (0.0009) [2023-12-26 15:46:41,610][105692] Updated weights for policy 0, policy_version 56561 (0.0008) [2023-12-26 15:46:41,677][105692] Updated weights for policy 0, policy_version 56571 (0.0009) [2023-12-26 15:46:41,739][105692] Updated weights for policy 0, policy_version 56581 (0.0010) [2023-12-26 15:46:41,804][105692] Updated weights for policy 0, policy_version 56591 (0.0009) [2023-12-26 15:46:41,871][105620] Updated weights for policy 1, policy_version 56949 (0.0009) [2023-12-26 15:46:41,925][105620] Updated weights for policy 1, policy_version 56959 (0.0009) [2023-12-26 15:46:41,981][105620] Updated weights for policy 1, policy_version 56969 (0.0010) [2023-12-26 15:46:42,531][105692] Updated weights for policy 0, policy_version 56601 (0.0009) [2023-12-26 15:46:42,589][105692] Updated weights for policy 0, policy_version 56611 (0.0007) [2023-12-26 15:46:42,643][105692] Updated weights for policy 0, policy_version 56621 (0.0006) [2023-12-26 15:46:42,750][105620] Updated weights for policy 1, policy_version 56979 (0.0009) [2023-12-26 15:46:42,797][105620] Updated weights for policy 1, policy_version 56989 (0.0008) [2023-12-26 15:46:42,848][105620] Updated weights for policy 1, policy_version 56999 (0.0009) [2023-12-26 15:46:43,382][105692] Updated weights for policy 0, policy_version 56631 (0.0007) [2023-12-26 15:46:43,441][105692] Updated weights for policy 0, policy_version 56641 (0.0008) [2023-12-26 15:46:43,497][105692] Updated weights for policy 0, policy_version 56651 (0.0008) [2023-12-26 15:46:43,619][105620] Updated weights for policy 1, policy_version 57009 (0.0009) [2023-12-26 15:46:43,678][105620] Updated weights for policy 1, policy_version 57019 (0.0006) [2023-12-26 15:46:43,736][105620] Updated weights for policy 1, policy_version 57029 (0.0010) [2023-12-26 15:46:43,794][105620] Updated weights for policy 1, policy_version 57039 (0.0010) [2023-12-26 15:46:44,146][105692] Updated weights for policy 0, policy_version 56661 (0.0005) [2023-12-26 15:46:44,194][105692] Updated weights for policy 0, policy_version 56671 (0.0005) [2023-12-26 15:46:44,242][105692] Updated weights for policy 0, policy_version 56681 (0.0007) [2023-12-26 15:46:44,509][105620] Updated weights for policy 1, policy_version 57049 (0.0010) [2023-12-26 15:46:44,564][105620] Updated weights for policy 1, policy_version 57059 (0.0009) [2023-12-26 15:46:44,620][105620] Updated weights for policy 1, policy_version 57069 (0.0009) [2023-12-26 15:46:44,944][105692] Updated weights for policy 0, policy_version 56691 (0.0008) [2023-12-26 15:46:45,001][105692] Updated weights for policy 0, policy_version 56701 (0.0009) [2023-12-26 15:46:45,055][105692] Updated weights for policy 0, policy_version 56711 (0.0011) [2023-12-26 15:46:45,334][105620] Updated weights for policy 1, policy_version 57079 (0.0008) [2023-12-26 15:46:45,384][105620] Updated weights for policy 1, policy_version 57089 (0.0008) [2023-12-26 15:46:45,439][105620] Updated weights for policy 1, policy_version 57099 (0.0008) [2023-12-26 15:46:45,776][105692] Updated weights for policy 0, policy_version 56721 (0.0007) [2023-12-26 15:46:45,822][105692] Updated weights for policy 0, policy_version 56731 (0.0008) [2023-12-26 15:46:45,885][105692] Updated weights for policy 0, policy_version 56741 (0.0009) [2023-12-26 15:46:45,947][105692] Updated weights for policy 0, policy_version 56751 (0.0010) [2023-12-26 15:46:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19797.3, 300 sec: 19716.3). Total num frames: 29155328. Throughput: 0: 9797.0, 1: 9887.7. Samples: 29122256. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 15:46:46,063][104569] Avg episode reward: [(0, '7386.987'), (1, '9263.230')] [2023-12-26 15:46:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000057104_14622720.pth... [2023-12-26 15:46:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000056752_14532608.pth... [2023-12-26 15:46:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000055600_14237696.pth [2023-12-26 15:46:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000055984_14336000.pth [2023-12-26 15:46:46,149][105620] Updated weights for policy 1, policy_version 57109 (0.0010) [2023-12-26 15:46:46,204][105620] Updated weights for policy 1, policy_version 57119 (0.0010) [2023-12-26 15:46:46,259][105620] Updated weights for policy 1, policy_version 57129 (0.0010) [2023-12-26 15:46:46,800][105692] Updated weights for policy 0, policy_version 56761 (0.0010) [2023-12-26 15:46:46,825][105620] Updated weights for policy 1, policy_version 57139 (0.0009) [2023-12-26 15:46:46,863][105692] Updated weights for policy 0, policy_version 56771 (0.0007) [2023-12-26 15:46:46,888][105620] Updated weights for policy 1, policy_version 57149 (0.0005) [2023-12-26 15:46:46,912][105692] Updated weights for policy 0, policy_version 56781 (0.0005) [2023-12-26 15:46:46,948][105620] Updated weights for policy 1, policy_version 57159 (0.0006) [2023-12-26 15:46:47,519][105692] Updated weights for policy 0, policy_version 56791 (0.0009) [2023-12-26 15:46:47,551][105620] Updated weights for policy 1, policy_version 57169 (0.0009) [2023-12-26 15:46:47,574][105692] Updated weights for policy 0, policy_version 56801 (0.0010) [2023-12-26 15:46:47,603][105620] Updated weights for policy 1, policy_version 57179 (0.0006) [2023-12-26 15:46:47,633][105692] Updated weights for policy 0, policy_version 56811 (0.0010) [2023-12-26 15:46:47,657][105620] Updated weights for policy 1, policy_version 57189 (0.0005) [2023-12-26 15:46:47,705][105620] Updated weights for policy 1, policy_version 57199 (0.0005) [2023-12-26 15:46:48,306][105620] Updated weights for policy 1, policy_version 57209 (0.0010) [2023-12-26 15:46:48,333][105692] Updated weights for policy 0, policy_version 56821 (0.0008) [2023-12-26 15:46:48,368][105620] Updated weights for policy 1, policy_version 57219 (0.0009) [2023-12-26 15:46:48,398][105692] Updated weights for policy 0, policy_version 56831 (0.0007) [2023-12-26 15:46:48,424][105620] Updated weights for policy 1, policy_version 57229 (0.0008) [2023-12-26 15:46:48,464][105692] Updated weights for policy 0, policy_version 56841 (0.0007) [2023-12-26 15:46:49,123][105620] Updated weights for policy 1, policy_version 57239 (0.0010) [2023-12-26 15:46:49,160][105692] Updated weights for policy 0, policy_version 56851 (0.0007) [2023-12-26 15:46:49,185][105620] Updated weights for policy 1, policy_version 57249 (0.0010) [2023-12-26 15:46:49,207][105692] Updated weights for policy 0, policy_version 56861 (0.0009) [2023-12-26 15:46:49,241][105620] Updated weights for policy 1, policy_version 57259 (0.0011) [2023-12-26 15:46:49,272][105692] Updated weights for policy 0, policy_version 56871 (0.0008) [2023-12-26 15:46:50,038][105692] Updated weights for policy 0, policy_version 56881 (0.0008) [2023-12-26 15:46:50,076][105620] Updated weights for policy 1, policy_version 57269 (0.0008) [2023-12-26 15:46:50,091][105692] Updated weights for policy 0, policy_version 56891 (0.0011) [2023-12-26 15:46:50,138][105620] Updated weights for policy 1, policy_version 57279 (0.0006) [2023-12-26 15:46:50,140][105692] Updated weights for policy 0, policy_version 56901 (0.0011) [2023-12-26 15:46:50,190][105692] Updated weights for policy 0, policy_version 56911 (0.0007) [2023-12-26 15:46:50,199][105620] Updated weights for policy 1, policy_version 57289 (0.0008) [2023-12-26 15:46:50,870][105620] Updated weights for policy 1, policy_version 57299 (0.0010) [2023-12-26 15:46:50,889][105692] Updated weights for policy 0, policy_version 56921 (0.0006) [2023-12-26 15:46:50,924][105620] Updated weights for policy 1, policy_version 57309 (0.0008) [2023-12-26 15:46:50,957][105692] Updated weights for policy 0, policy_version 56931 (0.0006) [2023-12-26 15:46:50,977][105620] Updated weights for policy 1, policy_version 57319 (0.0008) [2023-12-26 15:46:51,024][105692] Updated weights for policy 0, policy_version 56941 (0.0007) [2023-12-26 15:46:51,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19744.1). Total num frames: 29261824. Throughput: 0: 9789.0, 1: 9961.1. Samples: 29242908. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 15:46:51,063][104569] Avg episode reward: [(0, '8060.653'), (1, '9354.935')] [2023-12-26 15:46:51,655][105692] Updated weights for policy 0, policy_version 56951 (0.0009) [2023-12-26 15:46:51,712][105692] Updated weights for policy 0, policy_version 56961 (0.0008) [2023-12-26 15:46:51,767][105692] Updated weights for policy 0, policy_version 56971 (0.0006) [2023-12-26 15:46:51,785][105620] Updated weights for policy 1, policy_version 57329 (0.0008) [2023-12-26 15:46:51,851][105620] Updated weights for policy 1, policy_version 57339 (0.0008) [2023-12-26 15:46:51,915][105620] Updated weights for policy 1, policy_version 57349 (0.0008) [2023-12-26 15:46:51,976][105620] Updated weights for policy 1, policy_version 57359 (0.0008) [2023-12-26 15:46:52,487][105692] Updated weights for policy 0, policy_version 56981 (0.0006) [2023-12-26 15:46:52,542][105692] Updated weights for policy 0, policy_version 56991 (0.0006) [2023-12-26 15:46:52,613][105692] Updated weights for policy 0, policy_version 57001 (0.0010) [2023-12-26 15:46:52,733][105620] Updated weights for policy 1, policy_version 57369 (0.0006) [2023-12-26 15:46:52,789][105620] Updated weights for policy 1, policy_version 57379 (0.0008) [2023-12-26 15:46:52,850][105620] Updated weights for policy 1, policy_version 57389 (0.0008) [2023-12-26 15:46:53,231][105692] Updated weights for policy 0, policy_version 57011 (0.0009) [2023-12-26 15:46:53,289][105692] Updated weights for policy 0, policy_version 57021 (0.0006) [2023-12-26 15:46:53,335][105692] Updated weights for policy 0, policy_version 57031 (0.0005) [2023-12-26 15:46:53,748][105620] Updated weights for policy 1, policy_version 57399 (0.0008) [2023-12-26 15:46:53,810][105620] Updated weights for policy 1, policy_version 57409 (0.0005) [2023-12-26 15:46:53,869][105620] Updated weights for policy 1, policy_version 57419 (0.0006) [2023-12-26 15:46:53,978][105692] Updated weights for policy 0, policy_version 57041 (0.0006) [2023-12-26 15:46:54,033][105692] Updated weights for policy 0, policy_version 57051 (0.0010) [2023-12-26 15:46:54,086][105692] Updated weights for policy 0, policy_version 57061 (0.0007) [2023-12-26 15:46:54,138][105692] Updated weights for policy 0, policy_version 57071 (0.0006) [2023-12-26 15:46:54,457][105620] Updated weights for policy 1, policy_version 57429 (0.0006) [2023-12-26 15:46:54,511][105620] Updated weights for policy 1, policy_version 57439 (0.0005) [2023-12-26 15:46:54,570][105620] Updated weights for policy 1, policy_version 57449 (0.0005) [2023-12-26 15:46:54,816][105692] Updated weights for policy 0, policy_version 57081 (0.0005) [2023-12-26 15:46:54,870][105692] Updated weights for policy 0, policy_version 57091 (0.0005) [2023-12-26 15:46:54,915][105692] Updated weights for policy 0, policy_version 57101 (0.0005) [2023-12-26 15:46:55,182][105620] Updated weights for policy 1, policy_version 57459 (0.0006) [2023-12-26 15:46:55,231][105620] Updated weights for policy 1, policy_version 57469 (0.0005) [2023-12-26 15:46:55,286][105620] Updated weights for policy 1, policy_version 57479 (0.0011) [2023-12-26 15:46:55,506][105692] Updated weights for policy 0, policy_version 57111 (0.0009) [2023-12-26 15:46:55,553][105692] Updated weights for policy 0, policy_version 57121 (0.0010) [2023-12-26 15:46:55,598][105692] Updated weights for policy 0, policy_version 57131 (0.0010) [2023-12-26 15:46:56,009][105620] Updated weights for policy 1, policy_version 57489 (0.0010) [2023-12-26 15:46:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19716.3). Total num frames: 29351936. Throughput: 0: 9907.4, 1: 9910.2. Samples: 29363364. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 15:46:56,062][104569] Avg episode reward: [(0, '8588.757'), (1, '9354.903')] [2023-12-26 15:46:56,074][105620] Updated weights for policy 1, policy_version 57499 (0.0005) [2023-12-26 15:46:56,129][105620] Updated weights for policy 1, policy_version 57509 (0.0010) [2023-12-26 15:46:56,177][105620] Updated weights for policy 1, policy_version 57519 (0.0010) [2023-12-26 15:46:56,367][105692] Updated weights for policy 0, policy_version 57141 (0.0008) [2023-12-26 15:46:56,431][105692] Updated weights for policy 0, policy_version 57151 (0.0011) [2023-12-26 15:46:56,494][105692] Updated weights for policy 0, policy_version 57161 (0.0011) [2023-12-26 15:46:56,731][105620] Updated weights for policy 1, policy_version 57529 (0.0006) [2023-12-26 15:46:56,787][105620] Updated weights for policy 1, policy_version 57539 (0.0009) [2023-12-26 15:46:56,846][105620] Updated weights for policy 1, policy_version 57549 (0.0007) [2023-12-26 15:46:57,126][105692] Updated weights for policy 0, policy_version 57171 (0.0009) [2023-12-26 15:46:57,179][105692] Updated weights for policy 0, policy_version 57181 (0.0005) [2023-12-26 15:46:57,227][105692] Updated weights for policy 0, policy_version 57191 (0.0005) [2023-12-26 15:46:57,377][105620] Updated weights for policy 1, policy_version 57559 (0.0010) [2023-12-26 15:46:57,434][105620] Updated weights for policy 1, policy_version 57569 (0.0010) [2023-12-26 15:46:57,494][105620] Updated weights for policy 1, policy_version 57579 (0.0008) [2023-12-26 15:46:57,909][105692] Updated weights for policy 0, policy_version 57201 (0.0006) [2023-12-26 15:46:57,966][105692] Updated weights for policy 0, policy_version 57211 (0.0009) [2023-12-26 15:46:58,033][105692] Updated weights for policy 0, policy_version 57221 (0.0007) [2023-12-26 15:46:58,058][105620] Updated weights for policy 1, policy_version 57589 (0.0006) [2023-12-26 15:46:58,090][105692] Updated weights for policy 0, policy_version 57231 (0.0007) [2023-12-26 15:46:58,124][105620] Updated weights for policy 1, policy_version 57599 (0.0008) [2023-12-26 15:46:58,188][105620] Updated weights for policy 1, policy_version 57609 (0.0010) [2023-12-26 15:46:58,867][105692] Updated weights for policy 0, policy_version 57241 (0.0009) [2023-12-26 15:46:58,925][105692] Updated weights for policy 0, policy_version 57251 (0.0007) [2023-12-26 15:46:58,991][105692] Updated weights for policy 0, policy_version 57261 (0.0008) [2023-12-26 15:46:58,996][105620] Updated weights for policy 1, policy_version 57619 (0.0009) [2023-12-26 15:46:59,056][105620] Updated weights for policy 1, policy_version 57629 (0.0009) [2023-12-26 15:46:59,121][105620] Updated weights for policy 1, policy_version 57639 (0.0008) [2023-12-26 15:46:59,795][105692] Updated weights for policy 0, policy_version 57271 (0.0009) [2023-12-26 15:46:59,856][105692] Updated weights for policy 0, policy_version 57281 (0.0008) [2023-12-26 15:46:59,872][105620] Updated weights for policy 1, policy_version 57649 (0.0006) [2023-12-26 15:46:59,919][105692] Updated weights for policy 0, policy_version 57291 (0.0006) [2023-12-26 15:46:59,942][105620] Updated weights for policy 1, policy_version 57659 (0.0008) [2023-12-26 15:47:00,009][105620] Updated weights for policy 1, policy_version 57669 (0.0008) [2023-12-26 15:47:00,063][105620] Updated weights for policy 1, policy_version 57679 (0.0005) [2023-12-26 15:47:00,634][105620] Updated weights for policy 1, policy_version 57689 (0.0005) [2023-12-26 15:47:00,673][105692] Updated weights for policy 0, policy_version 57301 (0.0009) [2023-12-26 15:47:00,690][105620] Updated weights for policy 1, policy_version 57699 (0.0006) [2023-12-26 15:47:00,734][105692] Updated weights for policy 0, policy_version 57311 (0.0010) [2023-12-26 15:47:00,738][105620] Updated weights for policy 1, policy_version 57709 (0.0007) [2023-12-26 15:47:00,806][105692] Updated weights for policy 0, policy_version 57321 (0.0010) [2023-12-26 15:47:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19716.3). Total num frames: 29458432. Throughput: 0: 9954.1, 1: 9944.6. Samples: 29426364. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 15:47:01,062][104569] Avg episode reward: [(0, '9020.863'), (1, '9355.075')] [2023-12-26 15:47:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000057328_14680064.pth... [2023-12-26 15:47:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000057712_14778368.pth... [2023-12-26 15:47:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000056176_14385152.pth [2023-12-26 15:47:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000056560_14483456.pth [2023-12-26 15:47:01,300][105620] Updated weights for policy 1, policy_version 57719 (0.0009) [2023-12-26 15:47:01,362][105620] Updated weights for policy 1, policy_version 57729 (0.0009) [2023-12-26 15:47:01,426][105620] Updated weights for policy 1, policy_version 57739 (0.0006) [2023-12-26 15:47:01,445][105692] Updated weights for policy 0, policy_version 57331 (0.0009) [2023-12-26 15:47:01,494][105692] Updated weights for policy 0, policy_version 57341 (0.0006) [2023-12-26 15:47:01,551][105692] Updated weights for policy 0, policy_version 57351 (0.0007) [2023-12-26 15:47:02,103][105620] Updated weights for policy 1, policy_version 57749 (0.0005) [2023-12-26 15:47:02,160][105620] Updated weights for policy 1, policy_version 57759 (0.0006) [2023-12-26 15:47:02,220][105620] Updated weights for policy 1, policy_version 57769 (0.0005) [2023-12-26 15:47:02,232][105692] Updated weights for policy 0, policy_version 57361 (0.0006) [2023-12-26 15:47:02,295][105692] Updated weights for policy 0, policy_version 57371 (0.0007) [2023-12-26 15:47:02,358][105692] Updated weights for policy 0, policy_version 57381 (0.0008) [2023-12-26 15:47:02,411][105692] Updated weights for policy 0, policy_version 57391 (0.0009) [2023-12-26 15:47:02,813][105620] Updated weights for policy 1, policy_version 57779 (0.0009) [2023-12-26 15:47:02,872][105620] Updated weights for policy 1, policy_version 57789 (0.0011) [2023-12-26 15:47:02,930][105620] Updated weights for policy 1, policy_version 57799 (0.0010) [2023-12-26 15:47:03,041][105692] Updated weights for policy 0, policy_version 57401 (0.0008) [2023-12-26 15:47:03,097][105692] Updated weights for policy 0, policy_version 57411 (0.0006) [2023-12-26 15:47:03,144][105692] Updated weights for policy 0, policy_version 57421 (0.0005) [2023-12-26 15:47:03,656][105620] Updated weights for policy 1, policy_version 57809 (0.0010) [2023-12-26 15:47:03,672][105692] Updated weights for policy 0, policy_version 57431 (0.0007) [2023-12-26 15:47:03,704][105620] Updated weights for policy 1, policy_version 57819 (0.0010) [2023-12-26 15:47:03,727][105692] Updated weights for policy 0, policy_version 57441 (0.0007) [2023-12-26 15:47:03,752][105620] Updated weights for policy 1, policy_version 57829 (0.0010) [2023-12-26 15:47:03,782][105692] Updated weights for policy 0, policy_version 57451 (0.0005) [2023-12-26 15:47:03,800][105620] Updated weights for policy 1, policy_version 57839 (0.0010) [2023-12-26 15:47:04,441][105620] Updated weights for policy 1, policy_version 57849 (0.0007) [2023-12-26 15:47:04,493][105620] Updated weights for policy 1, policy_version 57859 (0.0009) [2023-12-26 15:47:04,559][105620] Updated weights for policy 1, policy_version 57869 (0.0011) [2023-12-26 15:47:04,648][105692] Updated weights for policy 0, policy_version 57461 (0.0005) [2023-12-26 15:47:04,702][105692] Updated weights for policy 0, policy_version 57471 (0.0005) [2023-12-26 15:47:04,752][105692] Updated weights for policy 0, policy_version 57481 (0.0008) [2023-12-26 15:47:05,211][105620] Updated weights for policy 1, policy_version 57879 (0.0010) [2023-12-26 15:47:05,259][105620] Updated weights for policy 1, policy_version 57889 (0.0010) [2023-12-26 15:47:05,310][105620] Updated weights for policy 1, policy_version 57899 (0.0010) [2023-12-26 15:47:05,470][105692] Updated weights for policy 0, policy_version 57491 (0.0007) [2023-12-26 15:47:05,531][105692] Updated weights for policy 0, policy_version 57501 (0.0008) [2023-12-26 15:47:05,589][105692] Updated weights for policy 0, policy_version 57511 (0.0008) [2023-12-26 15:47:06,022][105620] Updated weights for policy 1, policy_version 57909 (0.0010) [2023-12-26 15:47:06,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.4, 300 sec: 19716.3). Total num frames: 29556736. Throughput: 0: 9963.5, 1: 9958.2. Samples: 29548396. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 15:47:06,063][104569] Avg episode reward: [(0, '9116.158'), (1, '9355.945')] [2023-12-26 15:47:06,076][105620] Updated weights for policy 1, policy_version 57919 (0.0010) [2023-12-26 15:47:06,139][105620] Updated weights for policy 1, policy_version 57929 (0.0011) [2023-12-26 15:47:06,258][105692] Updated weights for policy 0, policy_version 57521 (0.0006) [2023-12-26 15:47:06,318][105692] Updated weights for policy 0, policy_version 57531 (0.0008) [2023-12-26 15:47:06,373][105692] Updated weights for policy 0, policy_version 57541 (0.0007) [2023-12-26 15:47:06,429][105692] Updated weights for policy 0, policy_version 57551 (0.0008) [2023-12-26 15:47:06,826][105620] Updated weights for policy 1, policy_version 57939 (0.0009) [2023-12-26 15:47:06,883][105620] Updated weights for policy 1, policy_version 57949 (0.0005) [2023-12-26 15:47:06,932][105620] Updated weights for policy 1, policy_version 57959 (0.0005) [2023-12-26 15:47:07,304][105692] Updated weights for policy 0, policy_version 57561 (0.0009) [2023-12-26 15:47:07,356][105692] Updated weights for policy 0, policy_version 57571 (0.0010) [2023-12-26 15:47:07,411][105692] Updated weights for policy 0, policy_version 57581 (0.0008) [2023-12-26 15:47:07,464][105620] Updated weights for policy 1, policy_version 57969 (0.0005) [2023-12-26 15:47:07,522][105620] Updated weights for policy 1, policy_version 57979 (0.0008) [2023-12-26 15:47:07,575][105620] Updated weights for policy 1, policy_version 57989 (0.0009) [2023-12-26 15:47:07,633][105620] Updated weights for policy 1, policy_version 57999 (0.0009) [2023-12-26 15:47:08,190][105692] Updated weights for policy 0, policy_version 57591 (0.0009) [2023-12-26 15:47:08,248][105692] Updated weights for policy 0, policy_version 57601 (0.0010) [2023-12-26 15:47:08,308][105692] Updated weights for policy 0, policy_version 57611 (0.0010) [2023-12-26 15:47:08,334][105620] Updated weights for policy 1, policy_version 58010 (0.0007) [2023-12-26 15:47:08,400][105620] Updated weights for policy 1, policy_version 58020 (0.0007) [2023-12-26 15:47:08,460][105620] Updated weights for policy 1, policy_version 58030 (0.0005) [2023-12-26 15:47:09,042][105692] Updated weights for policy 0, policy_version 57621 (0.0009) [2023-12-26 15:47:09,060][105620] Updated weights for policy 1, policy_version 58040 (0.0006) [2023-12-26 15:47:09,104][105692] Updated weights for policy 0, policy_version 57631 (0.0008) [2023-12-26 15:47:09,110][105620] Updated weights for policy 1, policy_version 58050 (0.0006) [2023-12-26 15:47:09,158][105692] Updated weights for policy 0, policy_version 57641 (0.0007) [2023-12-26 15:47:09,164][105620] Updated weights for policy 1, policy_version 58060 (0.0008) [2023-12-26 15:47:09,867][105620] Updated weights for policy 1, policy_version 58070 (0.0008) [2023-12-26 15:47:09,920][105692] Updated weights for policy 0, policy_version 57651 (0.0009) [2023-12-26 15:47:09,937][105620] Updated weights for policy 1, policy_version 58080 (0.0009) [2023-12-26 15:47:09,979][105692] Updated weights for policy 0, policy_version 57661 (0.0008) [2023-12-26 15:47:09,995][105620] Updated weights for policy 1, policy_version 58090 (0.0007) [2023-12-26 15:47:10,037][105692] Updated weights for policy 0, policy_version 57671 (0.0009) [2023-12-26 15:47:10,723][105620] Updated weights for policy 1, policy_version 58100 (0.0007) [2023-12-26 15:47:10,771][105620] Updated weights for policy 1, policy_version 58110 (0.0007) [2023-12-26 15:47:10,817][105692] Updated weights for policy 0, policy_version 57681 (0.0008) [2023-12-26 15:47:10,831][105620] Updated weights for policy 1, policy_version 58120 (0.0006) [2023-12-26 15:47:10,880][105692] Updated weights for policy 0, policy_version 57691 (0.0008) [2023-12-26 15:47:10,942][105692] Updated weights for policy 0, policy_version 57701 (0.0010) [2023-12-26 15:47:11,000][105692] Updated weights for policy 0, policy_version 57711 (0.0010) [2023-12-26 15:47:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19933.8, 300 sec: 19744.1). Total num frames: 29663232. Throughput: 0: 9806.1, 1: 10057.6. Samples: 29666208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:47:11,063][104569] Avg episode reward: [(0, '7014.678'), (1, '9356.734')] [2023-12-26 15:47:11,527][105620] Updated weights for policy 1, policy_version 58130 (0.0007) [2023-12-26 15:47:11,582][105620] Updated weights for policy 1, policy_version 58140 (0.0008) [2023-12-26 15:47:11,637][105620] Updated weights for policy 1, policy_version 58150 (0.0009) [2023-12-26 15:47:11,691][105620] Updated weights for policy 1, policy_version 58160 (0.0008) [2023-12-26 15:47:11,890][105692] Updated weights for policy 0, policy_version 57721 (0.0009) [2023-12-26 15:47:11,946][105692] Updated weights for policy 0, policy_version 57731 (0.0009) [2023-12-26 15:47:12,002][105692] Updated weights for policy 0, policy_version 57741 (0.0008) [2023-12-26 15:47:12,450][105620] Updated weights for policy 1, policy_version 58170 (0.0011) [2023-12-26 15:47:12,505][105620] Updated weights for policy 1, policy_version 58180 (0.0010) [2023-12-26 15:47:12,572][105620] Updated weights for policy 1, policy_version 58190 (0.0010) [2023-12-26 15:47:12,795][105692] Updated weights for policy 0, policy_version 57751 (0.0008) [2023-12-26 15:47:12,854][105692] Updated weights for policy 0, policy_version 57761 (0.0008) [2023-12-26 15:47:12,919][105692] Updated weights for policy 0, policy_version 57771 (0.0009) [2023-12-26 15:47:13,314][105620] Updated weights for policy 1, policy_version 58200 (0.0009) [2023-12-26 15:47:13,375][105620] Updated weights for policy 1, policy_version 58210 (0.0009) [2023-12-26 15:47:13,431][105620] Updated weights for policy 1, policy_version 58220 (0.0007) [2023-12-26 15:47:13,617][105692] Updated weights for policy 0, policy_version 57781 (0.0009) [2023-12-26 15:47:13,667][105692] Updated weights for policy 0, policy_version 57791 (0.0009) [2023-12-26 15:47:13,718][105692] Updated weights for policy 0, policy_version 57801 (0.0009) [2023-12-26 15:47:14,158][105620] Updated weights for policy 1, policy_version 58230 (0.0008) [2023-12-26 15:47:14,215][105620] Updated weights for policy 1, policy_version 58240 (0.0007) [2023-12-26 15:47:14,279][105620] Updated weights for policy 1, policy_version 58250 (0.0009) [2023-12-26 15:47:14,460][105692] Updated weights for policy 0, policy_version 57811 (0.0009) [2023-12-26 15:47:14,522][105692] Updated weights for policy 0, policy_version 57821 (0.0008) [2023-12-26 15:47:14,585][105692] Updated weights for policy 0, policy_version 57831 (0.0008) [2023-12-26 15:47:15,000][105620] Updated weights for policy 1, policy_version 58260 (0.0011) [2023-12-26 15:47:15,067][105620] Updated weights for policy 1, policy_version 58270 (0.0011) [2023-12-26 15:47:15,134][105620] Updated weights for policy 1, policy_version 58280 (0.0010) [2023-12-26 15:47:15,360][105692] Updated weights for policy 0, policy_version 57841 (0.0008) [2023-12-26 15:47:15,425][105692] Updated weights for policy 0, policy_version 57851 (0.0007) [2023-12-26 15:47:15,475][105692] Updated weights for policy 0, policy_version 57861 (0.0008) [2023-12-26 15:47:15,536][105692] Updated weights for policy 0, policy_version 57871 (0.0008) [2023-12-26 15:47:15,860][105620] Updated weights for policy 1, policy_version 58290 (0.0010) [2023-12-26 15:47:15,922][105620] Updated weights for policy 1, policy_version 58300 (0.0009) [2023-12-26 15:47:15,986][105620] Updated weights for policy 1, policy_version 58310 (0.0005) [2023-12-26 15:47:16,050][105620] Updated weights for policy 1, policy_version 58320 (0.0005) [2023-12-26 15:47:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19933.8, 300 sec: 19716.3). Total num frames: 29753344. Throughput: 0: 9713.1, 1: 10037.6. Samples: 29721440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:47:16,063][104569] Avg episode reward: [(0, '6656.348'), (1, '9273.269')] [2023-12-26 15:47:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000057872_14819328.pth... [2023-12-26 15:47:16,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000058320_14934016.pth... [2023-12-26 15:47:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000056752_14532608.pth [2023-12-26 15:47:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000057104_14622720.pth [2023-12-26 15:47:16,222][105692] Updated weights for policy 0, policy_version 57881 (0.0010) [2023-12-26 15:47:16,284][105692] Updated weights for policy 0, policy_version 57891 (0.0005) [2023-12-26 15:47:16,350][105692] Updated weights for policy 0, policy_version 57901 (0.0005) [2023-12-26 15:47:16,593][105620] Updated weights for policy 1, policy_version 58330 (0.0010) [2023-12-26 15:47:16,641][105620] Updated weights for policy 1, policy_version 58340 (0.0010) [2023-12-26 15:47:16,686][105620] Updated weights for policy 1, policy_version 58350 (0.0010) [2023-12-26 15:47:16,901][105692] Updated weights for policy 0, policy_version 57911 (0.0005) [2023-12-26 15:47:16,957][105692] Updated weights for policy 0, policy_version 57921 (0.0005) [2023-12-26 15:47:17,023][105692] Updated weights for policy 0, policy_version 57931 (0.0005) [2023-12-26 15:47:17,281][105620] Updated weights for policy 1, policy_version 58360 (0.0006) [2023-12-26 15:47:17,332][105620] Updated weights for policy 1, policy_version 58370 (0.0006) [2023-12-26 15:47:17,386][105620] Updated weights for policy 1, policy_version 58380 (0.0010) [2023-12-26 15:47:17,593][105692] Updated weights for policy 0, policy_version 57941 (0.0005) [2023-12-26 15:47:17,647][105692] Updated weights for policy 0, policy_version 57951 (0.0005) [2023-12-26 15:47:17,715][105692] Updated weights for policy 0, policy_version 57961 (0.0008) [2023-12-26 15:47:18,109][105620] Updated weights for policy 1, policy_version 58390 (0.0010) [2023-12-26 15:47:18,157][105620] Updated weights for policy 1, policy_version 58400 (0.0010) [2023-12-26 15:47:18,216][105620] Updated weights for policy 1, policy_version 58410 (0.0010) [2023-12-26 15:47:18,395][105692] Updated weights for policy 0, policy_version 57971 (0.0010) [2023-12-26 15:47:18,457][105692] Updated weights for policy 0, policy_version 57981 (0.0010) [2023-12-26 15:47:18,520][105692] Updated weights for policy 0, policy_version 57991 (0.0009) [2023-12-26 15:47:18,872][105620] Updated weights for policy 1, policy_version 58420 (0.0008) [2023-12-26 15:47:18,933][105620] Updated weights for policy 1, policy_version 58430 (0.0006) [2023-12-26 15:47:18,984][105620] Updated weights for policy 1, policy_version 58440 (0.0005) [2023-12-26 15:47:19,092][105692] Updated weights for policy 0, policy_version 58001 (0.0011) [2023-12-26 15:47:19,147][105692] Updated weights for policy 0, policy_version 58011 (0.0010) [2023-12-26 15:47:19,214][105692] Updated weights for policy 0, policy_version 58021 (0.0010) [2023-12-26 15:47:19,278][105692] Updated weights for policy 0, policy_version 58031 (0.0011) [2023-12-26 15:47:19,690][105620] Updated weights for policy 1, policy_version 58450 (0.0007) [2023-12-26 15:47:19,754][105620] Updated weights for policy 1, policy_version 58460 (0.0009) [2023-12-26 15:47:19,819][105620] Updated weights for policy 1, policy_version 58470 (0.0009) [2023-12-26 15:47:19,882][105620] Updated weights for policy 1, policy_version 58480 (0.0008) [2023-12-26 15:47:19,944][105692] Updated weights for policy 0, policy_version 58041 (0.0011) [2023-12-26 15:47:20,008][105692] Updated weights for policy 0, policy_version 58051 (0.0006) [2023-12-26 15:47:20,064][105692] Updated weights for policy 0, policy_version 58061 (0.0006) [2023-12-26 15:47:20,688][105692] Updated weights for policy 0, policy_version 58071 (0.0008) [2023-12-26 15:47:20,719][105620] Updated weights for policy 1, policy_version 58490 (0.0008) [2023-12-26 15:47:20,756][105692] Updated weights for policy 0, policy_version 58081 (0.0006) [2023-12-26 15:47:20,784][105620] Updated weights for policy 1, policy_version 58500 (0.0009) [2023-12-26 15:47:20,818][105692] Updated weights for policy 0, policy_version 58091 (0.0006) [2023-12-26 15:47:20,846][105620] Updated weights for policy 1, policy_version 58510 (0.0007) [2023-12-26 15:47:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 20070.4, 300 sec: 19771.9). Total num frames: 29859840. Throughput: 0: 9751.9, 1: 10137.6. Samples: 29845204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:47:21,063][104569] Avg episode reward: [(0, '6903.425'), (1, '9016.612')] [2023-12-26 15:47:21,498][105692] Updated weights for policy 0, policy_version 58101 (0.0010) [2023-12-26 15:47:21,559][105692] Updated weights for policy 0, policy_version 58111 (0.0009) [2023-12-26 15:47:21,629][105692] Updated weights for policy 0, policy_version 58121 (0.0009) [2023-12-26 15:47:21,657][105620] Updated weights for policy 1, policy_version 58520 (0.0008) [2023-12-26 15:47:21,723][105620] Updated weights for policy 1, policy_version 58530 (0.0009) [2023-12-26 15:47:21,780][105620] Updated weights for policy 1, policy_version 58540 (0.0009) [2023-12-26 15:47:22,338][105692] Updated weights for policy 0, policy_version 58131 (0.0008) [2023-12-26 15:47:22,400][105692] Updated weights for policy 0, policy_version 58141 (0.0008) [2023-12-26 15:47:22,459][105692] Updated weights for policy 0, policy_version 58151 (0.0008) [2023-12-26 15:47:22,576][105620] Updated weights for policy 1, policy_version 58550 (0.0010) [2023-12-26 15:47:22,639][105620] Updated weights for policy 1, policy_version 58560 (0.0011) [2023-12-26 15:47:22,685][105620] Updated weights for policy 1, policy_version 58570 (0.0010) [2023-12-26 15:47:23,192][105692] Updated weights for policy 0, policy_version 58161 (0.0008) [2023-12-26 15:47:23,256][105692] Updated weights for policy 0, policy_version 58171 (0.0005) [2023-12-26 15:47:23,329][105692] Updated weights for policy 0, policy_version 58181 (0.0009) [2023-12-26 15:47:23,387][105692] Updated weights for policy 0, policy_version 58191 (0.0005) [2023-12-26 15:47:23,450][105620] Updated weights for policy 1, policy_version 58580 (0.0009) [2023-12-26 15:47:23,504][105620] Updated weights for policy 1, policy_version 58590 (0.0005) [2023-12-26 15:47:23,563][105620] Updated weights for policy 1, policy_version 58600 (0.0005) [2023-12-26 15:47:24,052][105692] Updated weights for policy 0, policy_version 58201 (0.0005) [2023-12-26 15:47:24,101][105692] Updated weights for policy 0, policy_version 58211 (0.0005) [2023-12-26 15:47:24,158][105692] Updated weights for policy 0, policy_version 58221 (0.0005) [2023-12-26 15:47:24,184][105620] Updated weights for policy 1, policy_version 58610 (0.0006) [2023-12-26 15:47:24,238][105620] Updated weights for policy 1, policy_version 58620 (0.0010) [2023-12-26 15:47:24,293][105620] Updated weights for policy 1, policy_version 58630 (0.0008) [2023-12-26 15:47:24,351][105620] Updated weights for policy 1, policy_version 58640 (0.0005) [2023-12-26 15:47:24,733][105692] Updated weights for policy 0, policy_version 58231 (0.0006) [2023-12-26 15:47:24,794][105692] Updated weights for policy 0, policy_version 58241 (0.0006) [2023-12-26 15:47:24,858][105692] Updated weights for policy 0, policy_version 58251 (0.0006) [2023-12-26 15:47:25,042][105620] Updated weights for policy 1, policy_version 58650 (0.0010) [2023-12-26 15:47:25,099][105620] Updated weights for policy 1, policy_version 58660 (0.0010) [2023-12-26 15:47:25,152][105620] Updated weights for policy 1, policy_version 58670 (0.0009) [2023-12-26 15:47:25,430][105692] Updated weights for policy 0, policy_version 58261 (0.0008) [2023-12-26 15:47:25,483][105692] Updated weights for policy 0, policy_version 58272 (0.0011) [2023-12-26 15:47:25,528][105692] Updated weights for policy 0, policy_version 58282 (0.0006) [2023-12-26 15:47:25,890][105620] Updated weights for policy 1, policy_version 58680 (0.0006) [2023-12-26 15:47:25,949][105620] Updated weights for policy 1, policy_version 58690 (0.0006) [2023-12-26 15:47:26,000][105620] Updated weights for policy 1, policy_version 58700 (0.0006) [2023-12-26 15:47:26,062][104569] Fps is (10 sec: 20480.6, 60 sec: 19933.9, 300 sec: 19716.3). Total num frames: 29958144. Throughput: 0: 9897.5, 1: 10047.0. Samples: 29964128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:47:26,063][104569] Avg episode reward: [(0, '7432.295'), (1, '8845.086')] [2023-12-26 15:47:26,264][105692] Updated weights for policy 0, policy_version 58292 (0.0010) [2023-12-26 15:47:26,322][105692] Updated weights for policy 0, policy_version 58302 (0.0010) [2023-12-26 15:47:26,381][105692] Updated weights for policy 0, policy_version 58312 (0.0010) [2023-12-26 15:47:26,550][105620] Updated weights for policy 1, policy_version 58710 (0.0005) [2023-12-26 15:47:26,602][105620] Updated weights for policy 1, policy_version 58720 (0.0005) [2023-12-26 15:47:26,664][105620] Updated weights for policy 1, policy_version 58730 (0.0005) [2023-12-26 15:47:27,133][105692] Updated weights for policy 0, policy_version 58322 (0.0010) [2023-12-26 15:47:27,181][105692] Updated weights for policy 0, policy_version 58332 (0.0008) [2023-12-26 15:47:27,231][105620] Updated weights for policy 1, policy_version 58740 (0.0007) [2023-12-26 15:47:27,234][105692] Updated weights for policy 0, policy_version 58342 (0.0006) [2023-12-26 15:47:27,280][105620] Updated weights for policy 1, policy_version 58750 (0.0008) [2023-12-26 15:47:27,290][105692] Updated weights for policy 0, policy_version 58352 (0.0006) [2023-12-26 15:47:27,341][105620] Updated weights for policy 1, policy_version 58760 (0.0007) [2023-12-26 15:47:27,952][105620] Updated weights for policy 1, policy_version 58770 (0.0007) [2023-12-26 15:47:27,995][105692] Updated weights for policy 0, policy_version 58362 (0.0005) [2023-12-26 15:47:28,008][105620] Updated weights for policy 1, policy_version 58780 (0.0007) [2023-12-26 15:47:28,045][105692] Updated weights for policy 0, policy_version 58372 (0.0009) [2023-12-26 15:47:28,059][105620] Updated weights for policy 1, policy_version 58790 (0.0010) [2023-12-26 15:47:28,092][105692] Updated weights for policy 0, policy_version 58382 (0.0010) [2023-12-26 15:47:28,110][105620] Updated weights for policy 1, policy_version 58800 (0.0010) [2023-12-26 15:47:28,786][105692] Updated weights for policy 0, policy_version 58392 (0.0007) [2023-12-26 15:47:28,803][105620] Updated weights for policy 1, policy_version 58810 (0.0006) [2023-12-26 15:47:28,842][105692] Updated weights for policy 0, policy_version 58402 (0.0006) [2023-12-26 15:47:28,867][105620] Updated weights for policy 1, policy_version 58820 (0.0005) [2023-12-26 15:47:28,896][105692] Updated weights for policy 0, policy_version 58412 (0.0010) [2023-12-26 15:47:28,932][105620] Updated weights for policy 1, policy_version 58830 (0.0008) [2023-12-26 15:47:29,577][105692] Updated weights for policy 0, policy_version 58422 (0.0010) [2023-12-26 15:47:29,611][105620] Updated weights for policy 1, policy_version 58840 (0.0006) [2023-12-26 15:47:29,636][105692] Updated weights for policy 0, policy_version 58432 (0.0010) [2023-12-26 15:47:29,665][105620] Updated weights for policy 1, policy_version 58850 (0.0007) [2023-12-26 15:47:29,690][105692] Updated weights for policy 0, policy_version 58442 (0.0010) [2023-12-26 15:47:29,720][105620] Updated weights for policy 1, policy_version 58860 (0.0005) [2023-12-26 15:47:30,311][105692] Updated weights for policy 0, policy_version 58452 (0.0011) [2023-12-26 15:47:30,374][105692] Updated weights for policy 0, policy_version 58462 (0.0011) [2023-12-26 15:47:30,439][105692] Updated weights for policy 0, policy_version 58472 (0.0010) [2023-12-26 15:47:30,559][105620] Updated weights for policy 1, policy_version 58870 (0.0008) [2023-12-26 15:47:30,617][105620] Updated weights for policy 1, policy_version 58880 (0.0007) [2023-12-26 15:47:30,677][105620] Updated weights for policy 1, policy_version 58890 (0.0008) [2023-12-26 15:47:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19933.9, 300 sec: 19744.1). Total num frames: 30056448. Throughput: 0: 9938.3, 1: 10174.1. Samples: 30027308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:47:31,063][105692] Updated weights for policy 0, policy_version 58482 (0.0010) [2023-12-26 15:47:31,063][104569] Avg episode reward: [(0, '7966.459'), (1, '9010.628')] [2023-12-26 15:47:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000058896_15081472.pth... [2023-12-26 15:47:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000057712_14778368.pth [2023-12-26 15:47:31,125][105692] Updated weights for policy 0, policy_version 58492 (0.0009) [2023-12-26 15:47:31,188][105692] Updated weights for policy 0, policy_version 58502 (0.0006) [2023-12-26 15:47:31,246][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000058512_14983168.pth... [2023-12-26 15:47:31,248][105692] Updated weights for policy 0, policy_version 58512 (0.0008) [2023-12-26 15:47:31,250][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000057328_14680064.pth [2023-12-26 15:47:31,470][105620] Updated weights for policy 1, policy_version 58900 (0.0009) [2023-12-26 15:47:31,531][105620] Updated weights for policy 1, policy_version 58910 (0.0007) [2023-12-26 15:47:31,585][105620] Updated weights for policy 1, policy_version 58920 (0.0008) [2023-12-26 15:47:31,982][105692] Updated weights for policy 0, policy_version 58522 (0.0010) [2023-12-26 15:47:32,030][105692] Updated weights for policy 0, policy_version 58532 (0.0010) [2023-12-26 15:47:32,092][105692] Updated weights for policy 0, policy_version 58542 (0.0009) [2023-12-26 15:47:32,351][105620] Updated weights for policy 1, policy_version 58930 (0.0007) [2023-12-26 15:47:32,420][105620] Updated weights for policy 1, policy_version 58940 (0.0008) [2023-12-26 15:47:32,488][105620] Updated weights for policy 1, policy_version 58950 (0.0008) [2023-12-26 15:47:32,549][105620] Updated weights for policy 1, policy_version 58960 (0.0008) [2023-12-26 15:47:32,739][105692] Updated weights for policy 0, policy_version 58552 (0.0007) [2023-12-26 15:47:32,791][105692] Updated weights for policy 0, policy_version 58562 (0.0009) [2023-12-26 15:47:32,845][105692] Updated weights for policy 0, policy_version 58572 (0.0010) [2023-12-26 15:47:33,105][105620] Updated weights for policy 1, policy_version 58970 (0.0005) [2023-12-26 15:47:33,158][105620] Updated weights for policy 1, policy_version 58980 (0.0005) [2023-12-26 15:47:33,215][105620] Updated weights for policy 1, policy_version 58990 (0.0005) [2023-12-26 15:47:33,686][105692] Updated weights for policy 0, policy_version 58582 (0.0010) [2023-12-26 15:47:33,722][105620] Updated weights for policy 1, policy_version 59000 (0.0005) [2023-12-26 15:47:33,743][105692] Updated weights for policy 0, policy_version 58592 (0.0010) [2023-12-26 15:47:33,780][105620] Updated weights for policy 1, policy_version 59010 (0.0006) [2023-12-26 15:47:33,804][105692] Updated weights for policy 0, policy_version 58602 (0.0010) [2023-12-26 15:47:33,826][105620] Updated weights for policy 1, policy_version 59020 (0.0007) [2023-12-26 15:47:34,470][105620] Updated weights for policy 1, policy_version 59030 (0.0010) [2023-12-26 15:47:34,529][105620] Updated weights for policy 1, policy_version 59040 (0.0010) [2023-12-26 15:47:34,548][105692] Updated weights for policy 0, policy_version 58612 (0.0010) [2023-12-26 15:47:34,588][105620] Updated weights for policy 1, policy_version 59050 (0.0010) [2023-12-26 15:47:34,597][105692] Updated weights for policy 0, policy_version 58622 (0.0010) [2023-12-26 15:47:34,655][105692] Updated weights for policy 0, policy_version 58632 (0.0009) [2023-12-26 15:47:35,286][105620] Updated weights for policy 1, policy_version 59060 (0.0009) [2023-12-26 15:47:35,343][105620] Updated weights for policy 1, policy_version 59070 (0.0006) [2023-12-26 15:47:35,392][105620] Updated weights for policy 1, policy_version 59080 (0.0010) [2023-12-26 15:47:35,404][105692] Updated weights for policy 0, policy_version 58642 (0.0010) [2023-12-26 15:47:35,457][105692] Updated weights for policy 0, policy_version 58652 (0.0010) [2023-12-26 15:47:35,505][105692] Updated weights for policy 0, policy_version 58662 (0.0010) [2023-12-26 15:47:35,555][105692] Updated weights for policy 0, policy_version 58672 (0.0010) [2023-12-26 15:47:36,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19933.9, 300 sec: 19771.9). Total num frames: 30154752. Throughput: 0: 9956.3, 1: 10141.4. Samples: 30147300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:47:36,062][104569] Avg episode reward: [(0, '7438.960'), (1, '9094.699')] [2023-12-26 15:47:36,070][105620] Updated weights for policy 1, policy_version 59090 (0.0010) [2023-12-26 15:47:36,127][105620] Updated weights for policy 1, policy_version 59100 (0.0009) [2023-12-26 15:47:36,191][105620] Updated weights for policy 1, policy_version 59110 (0.0010) [2023-12-26 15:47:36,247][105620] Updated weights for policy 1, policy_version 59120 (0.0010) [2023-12-26 15:47:36,262][105692] Updated weights for policy 0, policy_version 58682 (0.0008) [2023-12-26 15:47:36,329][105692] Updated weights for policy 0, policy_version 58692 (0.0006) [2023-12-26 15:47:36,396][105692] Updated weights for policy 0, policy_version 58702 (0.0009) [2023-12-26 15:47:36,961][105620] Updated weights for policy 1, policy_version 59130 (0.0006) [2023-12-26 15:47:37,009][105620] Updated weights for policy 1, policy_version 59140 (0.0005) [2023-12-26 15:47:37,065][105620] Updated weights for policy 1, policy_version 59150 (0.0006) [2023-12-26 15:47:37,153][105692] Updated weights for policy 0, policy_version 58712 (0.0010) [2023-12-26 15:47:37,211][105692] Updated weights for policy 0, policy_version 58722 (0.0008) [2023-12-26 15:47:37,277][105692] Updated weights for policy 0, policy_version 58732 (0.0005) [2023-12-26 15:47:37,684][105620] Updated weights for policy 1, policy_version 59160 (0.0006) [2023-12-26 15:47:37,751][105620] Updated weights for policy 1, policy_version 59170 (0.0008) [2023-12-26 15:47:37,812][105620] Updated weights for policy 1, policy_version 59180 (0.0009) [2023-12-26 15:47:37,938][105692] Updated weights for policy 0, policy_version 58742 (0.0008) [2023-12-26 15:47:37,987][105692] Updated weights for policy 0, policy_version 58752 (0.0009) [2023-12-26 15:47:38,048][105692] Updated weights for policy 0, policy_version 58762 (0.0009) [2023-12-26 15:47:38,493][105620] Updated weights for policy 1, policy_version 59190 (0.0009) [2023-12-26 15:47:38,543][105620] Updated weights for policy 1, policy_version 59200 (0.0008) [2023-12-26 15:47:38,605][105620] Updated weights for policy 1, policy_version 59210 (0.0008) [2023-12-26 15:47:38,845][105692] Updated weights for policy 0, policy_version 58772 (0.0010) [2023-12-26 15:47:38,896][105692] Updated weights for policy 0, policy_version 58782 (0.0008) [2023-12-26 15:47:38,948][105692] Updated weights for policy 0, policy_version 58792 (0.0009) [2023-12-26 15:47:39,414][105620] Updated weights for policy 1, policy_version 59220 (0.0009) [2023-12-26 15:47:39,467][105620] Updated weights for policy 1, policy_version 59230 (0.0005) [2023-12-26 15:47:39,518][105620] Updated weights for policy 1, policy_version 59240 (0.0005) [2023-12-26 15:47:39,741][105692] Updated weights for policy 0, policy_version 58802 (0.0010) [2023-12-26 15:47:39,808][105692] Updated weights for policy 0, policy_version 58812 (0.0010) [2023-12-26 15:47:39,877][105692] Updated weights for policy 0, policy_version 58822 (0.0009) [2023-12-26 15:47:39,942][105692] Updated weights for policy 0, policy_version 58832 (0.0009) [2023-12-26 15:47:40,167][105620] Updated weights for policy 1, policy_version 59250 (0.0006) [2023-12-26 15:47:40,234][105620] Updated weights for policy 1, policy_version 59260 (0.0008) [2023-12-26 15:47:40,284][105620] Updated weights for policy 1, policy_version 59270 (0.0008) [2023-12-26 15:47:40,336][105620] Updated weights for policy 1, policy_version 59280 (0.0009) [2023-12-26 15:47:40,705][105692] Updated weights for policy 0, policy_version 58842 (0.0009) [2023-12-26 15:47:40,755][105692] Updated weights for policy 0, policy_version 58852 (0.0009) [2023-12-26 15:47:40,816][105692] Updated weights for policy 0, policy_version 58862 (0.0009) [2023-12-26 15:47:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19933.8, 300 sec: 19771.9). Total num frames: 30253056. Throughput: 0: 9803.0, 1: 10197.1. Samples: 30263372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:47:41,063][105620] Updated weights for policy 1, policy_version 59290 (0.0008) [2023-12-26 15:47:41,063][104569] Avg episode reward: [(0, '8042.394'), (1, '9353.604')] [2023-12-26 15:47:41,127][105620] Updated weights for policy 1, policy_version 59300 (0.0007) [2023-12-26 15:47:41,202][105620] Updated weights for policy 1, policy_version 59310 (0.0006) [2023-12-26 15:47:41,629][105692] Updated weights for policy 0, policy_version 58872 (0.0008) [2023-12-26 15:47:41,692][105692] Updated weights for policy 0, policy_version 58882 (0.0010) [2023-12-26 15:47:41,749][105692] Updated weights for policy 0, policy_version 58892 (0.0009) [2023-12-26 15:47:41,898][105620] Updated weights for policy 1, policy_version 59320 (0.0007) [2023-12-26 15:47:41,952][105620] Updated weights for policy 1, policy_version 59330 (0.0007) [2023-12-26 15:47:42,017][105620] Updated weights for policy 1, policy_version 59340 (0.0008) [2023-12-26 15:47:42,495][105692] Updated weights for policy 0, policy_version 58902 (0.0009) [2023-12-26 15:47:42,550][105692] Updated weights for policy 0, policy_version 58912 (0.0009) [2023-12-26 15:47:42,599][105692] Updated weights for policy 0, policy_version 58922 (0.0009) [2023-12-26 15:47:42,830][105620] Updated weights for policy 1, policy_version 59350 (0.0007) [2023-12-26 15:47:42,892][105620] Updated weights for policy 1, policy_version 59360 (0.0007) [2023-12-26 15:47:42,950][105620] Updated weights for policy 1, policy_version 59370 (0.0009) [2023-12-26 15:47:43,273][105692] Updated weights for policy 0, policy_version 58932 (0.0009) [2023-12-26 15:47:43,341][105692] Updated weights for policy 0, policy_version 58942 (0.0008) [2023-12-26 15:47:43,395][105692] Updated weights for policy 0, policy_version 58952 (0.0010) [2023-12-26 15:47:43,624][105620] Updated weights for policy 1, policy_version 59380 (0.0007) [2023-12-26 15:47:43,676][105620] Updated weights for policy 1, policy_version 59390 (0.0005) [2023-12-26 15:47:43,732][105620] Updated weights for policy 1, policy_version 59400 (0.0005) [2023-12-26 15:47:44,071][105692] Updated weights for policy 0, policy_version 58962 (0.0008) [2023-12-26 15:47:44,134][105692] Updated weights for policy 0, policy_version 58972 (0.0008) [2023-12-26 15:47:44,185][105692] Updated weights for policy 0, policy_version 58982 (0.0008) [2023-12-26 15:47:44,229][105692] Updated weights for policy 0, policy_version 58992 (0.0008) [2023-12-26 15:47:44,320][105620] Updated weights for policy 1, policy_version 59410 (0.0007) [2023-12-26 15:47:44,370][105620] Updated weights for policy 1, policy_version 59420 (0.0010) [2023-12-26 15:47:44,418][105620] Updated weights for policy 1, policy_version 59430 (0.0010) [2023-12-26 15:47:44,462][105620] Updated weights for policy 1, policy_version 59440 (0.0010) [2023-12-26 15:47:44,943][105692] Updated weights for policy 0, policy_version 59002 (0.0011) [2023-12-26 15:47:44,996][105692] Updated weights for policy 0, policy_version 59012 (0.0010) [2023-12-26 15:47:45,060][105692] Updated weights for policy 0, policy_version 59022 (0.0011) [2023-12-26 15:47:45,121][105620] Updated weights for policy 1, policy_version 59450 (0.0011) [2023-12-26 15:47:45,184][105620] Updated weights for policy 1, policy_version 59460 (0.0008) [2023-12-26 15:47:45,243][105620] Updated weights for policy 1, policy_version 59470 (0.0005) [2023-12-26 15:47:45,822][105692] Updated weights for policy 0, policy_version 59032 (0.0010) [2023-12-26 15:47:45,882][105692] Updated weights for policy 0, policy_version 59042 (0.0011) [2023-12-26 15:47:45,932][105620] Updated weights for policy 1, policy_version 59480 (0.0010) [2023-12-26 15:47:45,938][105692] Updated weights for policy 0, policy_version 59052 (0.0010) [2023-12-26 15:47:45,990][105620] Updated weights for policy 1, policy_version 59490 (0.0007) [2023-12-26 15:47:46,047][105620] Updated weights for policy 1, policy_version 59500 (0.0006) [2023-12-26 15:47:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19933.9, 300 sec: 19799.6). Total num frames: 30351360. Throughput: 0: 9758.8, 1: 10123.5. Samples: 30321072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:47:46,063][104569] Avg episode reward: [(0, '8474.957'), (1, '9355.121')] [2023-12-26 15:47:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000059056_15122432.pth... [2023-12-26 15:47:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000059504_15237120.pth... [2023-12-26 15:47:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000057872_14819328.pth [2023-12-26 15:47:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000058320_14934016.pth [2023-12-26 15:47:46,574][105692] Updated weights for policy 0, policy_version 59062 (0.0008) [2023-12-26 15:47:46,628][105692] Updated weights for policy 0, policy_version 59072 (0.0005) [2023-12-26 15:47:46,682][105692] Updated weights for policy 0, policy_version 59082 (0.0005) [2023-12-26 15:47:46,689][105620] Updated weights for policy 1, policy_version 59510 (0.0007) [2023-12-26 15:47:46,739][105620] Updated weights for policy 1, policy_version 59520 (0.0009) [2023-12-26 15:47:46,794][105620] Updated weights for policy 1, policy_version 59531 (0.0008) [2023-12-26 15:47:47,371][105692] Updated weights for policy 0, policy_version 59092 (0.0005) [2023-12-26 15:47:47,422][105692] Updated weights for policy 0, policy_version 59102 (0.0005) [2023-12-26 15:47:47,471][105692] Updated weights for policy 0, policy_version 59112 (0.0008) [2023-12-26 15:47:47,534][105620] Updated weights for policy 1, policy_version 59541 (0.0007) [2023-12-26 15:47:47,590][105620] Updated weights for policy 1, policy_version 59551 (0.0008) [2023-12-26 15:47:47,641][105620] Updated weights for policy 1, policy_version 59561 (0.0009) [2023-12-26 15:47:48,197][105692] Updated weights for policy 0, policy_version 59122 (0.0009) [2023-12-26 15:47:48,256][105692] Updated weights for policy 0, policy_version 59132 (0.0009) [2023-12-26 15:47:48,317][105692] Updated weights for policy 0, policy_version 59142 (0.0009) [2023-12-26 15:47:48,326][105620] Updated weights for policy 1, policy_version 59571 (0.0009) [2023-12-26 15:47:48,384][105692] Updated weights for policy 0, policy_version 59152 (0.0009) [2023-12-26 15:47:48,391][105620] Updated weights for policy 1, policy_version 59581 (0.0006) [2023-12-26 15:47:48,454][105620] Updated weights for policy 1, policy_version 59591 (0.0007) [2023-12-26 15:47:49,099][105620] Updated weights for policy 1, policy_version 59601 (0.0009) [2023-12-26 15:47:49,156][105620] Updated weights for policy 1, policy_version 59611 (0.0005) [2023-12-26 15:47:49,204][105692] Updated weights for policy 0, policy_version 59162 (0.0008) [2023-12-26 15:47:49,206][105620] Updated weights for policy 1, policy_version 59621 (0.0005) [2023-12-26 15:47:49,268][105620] Updated weights for policy 1, policy_version 59631 (0.0008) [2023-12-26 15:47:49,269][105692] Updated weights for policy 0, policy_version 59172 (0.0008) [2023-12-26 15:47:49,328][105692] Updated weights for policy 0, policy_version 59182 (0.0008) [2023-12-26 15:47:49,925][105620] Updated weights for policy 1, policy_version 59641 (0.0008) [2023-12-26 15:47:49,991][105620] Updated weights for policy 1, policy_version 59651 (0.0009) [2023-12-26 15:47:50,054][105620] Updated weights for policy 1, policy_version 59661 (0.0007) [2023-12-26 15:47:50,152][105692] Updated weights for policy 0, policy_version 59192 (0.0007) [2023-12-26 15:47:50,211][105692] Updated weights for policy 0, policy_version 59202 (0.0009) [2023-12-26 15:47:50,269][105692] Updated weights for policy 0, policy_version 59212 (0.0009) [2023-12-26 15:47:50,732][105620] Updated weights for policy 1, policy_version 59671 (0.0006) [2023-12-26 15:47:50,800][105620] Updated weights for policy 1, policy_version 59681 (0.0006) [2023-12-26 15:47:50,855][105620] Updated weights for policy 1, policy_version 59691 (0.0009) [2023-12-26 15:47:51,059][105692] Updated weights for policy 0, policy_version 59222 (0.0008) [2023-12-26 15:47:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19771.9). Total num frames: 30449664. Throughput: 0: 9735.4, 1: 10093.7. Samples: 30440704. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-12-26 15:47:51,062][104569] Avg episode reward: [(0, '8463.773'), (1, '9356.384')] [2023-12-26 15:47:51,119][105692] Updated weights for policy 0, policy_version 59232 (0.0009) [2023-12-26 15:47:51,193][105692] Updated weights for policy 0, policy_version 59242 (0.0008) [2023-12-26 15:47:51,563][105620] Updated weights for policy 1, policy_version 59701 (0.0009) [2023-12-26 15:47:51,611][105620] Updated weights for policy 1, policy_version 59711 (0.0008) [2023-12-26 15:47:51,677][105620] Updated weights for policy 1, policy_version 59721 (0.0009) [2023-12-26 15:47:51,952][105692] Updated weights for policy 0, policy_version 59252 (0.0009) [2023-12-26 15:47:52,016][105692] Updated weights for policy 0, policy_version 59262 (0.0009) [2023-12-26 15:47:52,073][105692] Updated weights for policy 0, policy_version 59272 (0.0008) [2023-12-26 15:47:52,472][105620] Updated weights for policy 1, policy_version 59731 (0.0007) [2023-12-26 15:47:52,535][105620] Updated weights for policy 1, policy_version 59741 (0.0005) [2023-12-26 15:47:52,589][105620] Updated weights for policy 1, policy_version 59751 (0.0005) [2023-12-26 15:47:52,810][105692] Updated weights for policy 0, policy_version 59282 (0.0007) [2023-12-26 15:47:52,872][105692] Updated weights for policy 0, policy_version 59292 (0.0009) [2023-12-26 15:47:52,923][105692] Updated weights for policy 0, policy_version 59302 (0.0009) [2023-12-26 15:47:52,985][105692] Updated weights for policy 0, policy_version 59312 (0.0009) [2023-12-26 15:47:53,233][105620] Updated weights for policy 1, policy_version 59761 (0.0006) [2023-12-26 15:47:53,283][105620] Updated weights for policy 1, policy_version 59771 (0.0008) [2023-12-26 15:47:53,338][105620] Updated weights for policy 1, policy_version 59781 (0.0009) [2023-12-26 15:47:53,392][105620] Updated weights for policy 1, policy_version 59791 (0.0008) [2023-12-26 15:47:53,744][105692] Updated weights for policy 0, policy_version 59322 (0.0008) [2023-12-26 15:47:53,801][105692] Updated weights for policy 0, policy_version 59332 (0.0005) [2023-12-26 15:47:53,854][105692] Updated weights for policy 0, policy_version 59342 (0.0006) [2023-12-26 15:47:54,194][105620] Updated weights for policy 1, policy_version 59801 (0.0007) [2023-12-26 15:47:54,251][105620] Updated weights for policy 1, policy_version 59811 (0.0008) [2023-12-26 15:47:54,310][105620] Updated weights for policy 1, policy_version 59821 (0.0009) [2023-12-26 15:47:54,510][105692] Updated weights for policy 0, policy_version 59352 (0.0005) [2023-12-26 15:47:54,562][105692] Updated weights for policy 0, policy_version 59362 (0.0006) [2023-12-26 15:47:54,612][105692] Updated weights for policy 0, policy_version 59372 (0.0009) [2023-12-26 15:47:55,083][105620] Updated weights for policy 1, policy_version 59831 (0.0009) [2023-12-26 15:47:55,131][105620] Updated weights for policy 1, policy_version 59841 (0.0007) [2023-12-26 15:47:55,187][105620] Updated weights for policy 1, policy_version 59851 (0.0005) [2023-12-26 15:47:55,379][105692] Updated weights for policy 0, policy_version 59382 (0.0009) [2023-12-26 15:47:55,444][105692] Updated weights for policy 0, policy_version 59392 (0.0008) [2023-12-26 15:47:55,504][105692] Updated weights for policy 0, policy_version 59402 (0.0010) [2023-12-26 15:47:55,795][105620] Updated weights for policy 1, policy_version 59861 (0.0008) [2023-12-26 15:47:55,848][105620] Updated weights for policy 1, policy_version 59871 (0.0008) [2023-12-26 15:47:55,907][105620] Updated weights for policy 1, policy_version 59881 (0.0008) [2023-12-26 15:47:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19933.8, 300 sec: 19771.9). Total num frames: 30547968. Throughput: 0: 9733.1, 1: 10002.6. Samples: 30554312. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-12-26 15:47:56,063][104569] Avg episode reward: [(0, '8218.453'), (1, '9356.676')] [2023-12-26 15:47:56,318][105692] Updated weights for policy 0, policy_version 59412 (0.0009) [2023-12-26 15:47:56,378][105692] Updated weights for policy 0, policy_version 59422 (0.0009) [2023-12-26 15:47:56,441][105692] Updated weights for policy 0, policy_version 59432 (0.0008) [2023-12-26 15:47:56,572][105620] Updated weights for policy 1, policy_version 59891 (0.0010) [2023-12-26 15:47:56,620][105620] Updated weights for policy 1, policy_version 59901 (0.0009) [2023-12-26 15:47:56,675][105620] Updated weights for policy 1, policy_version 59911 (0.0009) [2023-12-26 15:47:57,198][105692] Updated weights for policy 0, policy_version 59442 (0.0010) [2023-12-26 15:47:57,249][105692] Updated weights for policy 0, policy_version 59452 (0.0009) [2023-12-26 15:47:57,311][105692] Updated weights for policy 0, policy_version 59462 (0.0009) [2023-12-26 15:47:57,373][105692] Updated weights for policy 0, policy_version 59472 (0.0009) [2023-12-26 15:47:57,383][105620] Updated weights for policy 1, policy_version 59921 (0.0009) [2023-12-26 15:47:57,434][105620] Updated weights for policy 1, policy_version 59931 (0.0010) [2023-12-26 15:47:57,482][105620] Updated weights for policy 1, policy_version 59941 (0.0010) [2023-12-26 15:47:57,534][105620] Updated weights for policy 1, policy_version 59951 (0.0011) [2023-12-26 15:47:58,188][105620] Updated weights for policy 1, policy_version 59961 (0.0007) [2023-12-26 15:47:58,204][105692] Updated weights for policy 0, policy_version 59482 (0.0008) [2023-12-26 15:47:58,245][105620] Updated weights for policy 1, policy_version 59971 (0.0008) [2023-12-26 15:47:58,261][105692] Updated weights for policy 0, policy_version 59492 (0.0007) [2023-12-26 15:47:58,303][105620] Updated weights for policy 1, policy_version 59981 (0.0006) [2023-12-26 15:47:58,320][105692] Updated weights for policy 0, policy_version 59502 (0.0008) [2023-12-26 15:47:59,068][105620] Updated weights for policy 1, policy_version 59991 (0.0007) [2023-12-26 15:47:59,136][105620] Updated weights for policy 1, policy_version 60001 (0.0008) [2023-12-26 15:47:59,146][105692] Updated weights for policy 0, policy_version 59512 (0.0007) [2023-12-26 15:47:59,199][105620] Updated weights for policy 1, policy_version 60011 (0.0008) [2023-12-26 15:47:59,206][105692] Updated weights for policy 0, policy_version 59522 (0.0008) [2023-12-26 15:47:59,274][105692] Updated weights for policy 0, policy_version 59532 (0.0008) [2023-12-26 15:47:59,933][105620] Updated weights for policy 1, policy_version 60021 (0.0009) [2023-12-26 15:47:59,985][105620] Updated weights for policy 1, policy_version 60031 (0.0009) [2023-12-26 15:48:00,009][105692] Updated weights for policy 0, policy_version 59542 (0.0006) [2023-12-26 15:48:00,043][105620] Updated weights for policy 1, policy_version 60041 (0.0008) [2023-12-26 15:48:00,053][105692] Updated weights for policy 0, policy_version 59552 (0.0005) [2023-12-26 15:48:00,110][105692] Updated weights for policy 0, policy_version 59562 (0.0005) [2023-12-26 15:48:00,754][105692] Updated weights for policy 0, policy_version 59572 (0.0007) [2023-12-26 15:48:00,799][105692] Updated weights for policy 0, policy_version 59582 (0.0008) [2023-12-26 15:48:00,801][105620] Updated weights for policy 1, policy_version 60051 (0.0008) [2023-12-26 15:48:00,844][105692] Updated weights for policy 0, policy_version 59592 (0.0006) [2023-12-26 15:48:00,846][105620] Updated weights for policy 1, policy_version 60061 (0.0006) [2023-12-26 15:48:00,893][105620] Updated weights for policy 1, policy_version 60071 (0.0008) [2023-12-26 15:48:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19771.9). Total num frames: 30646272. Throughput: 0: 9731.5, 1: 10055.4. Samples: 30611848. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-12-26 15:48:01,063][104569] Avg episode reward: [(0, '8759.809'), (1, '9356.451')] [2023-12-26 15:48:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000059600_15261696.pth... [2023-12-26 15:48:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000060080_15384576.pth... [2023-12-26 15:48:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000058512_14983168.pth [2023-12-26 15:48:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000058896_15081472.pth [2023-12-26 15:48:01,539][105692] Updated weights for policy 0, policy_version 59602 (0.0005) [2023-12-26 15:48:01,598][105692] Updated weights for policy 0, policy_version 59612 (0.0006) [2023-12-26 15:48:01,628][105620] Updated weights for policy 1, policy_version 60081 (0.0008) [2023-12-26 15:48:01,664][105692] Updated weights for policy 0, policy_version 59622 (0.0007) [2023-12-26 15:48:01,693][105620] Updated weights for policy 1, policy_version 60091 (0.0007) [2023-12-26 15:48:01,728][105692] Updated weights for policy 0, policy_version 59632 (0.0008) [2023-12-26 15:48:01,765][105620] Updated weights for policy 1, policy_version 60101 (0.0008) [2023-12-26 15:48:01,828][105620] Updated weights for policy 1, policy_version 60111 (0.0009) [2023-12-26 15:48:02,394][105620] Updated weights for policy 1, policy_version 60121 (0.0008) [2023-12-26 15:48:02,447][105620] Updated weights for policy 1, policy_version 60131 (0.0006) [2023-12-26 15:48:02,500][105620] Updated weights for policy 1, policy_version 60141 (0.0007) [2023-12-26 15:48:02,585][105692] Updated weights for policy 0, policy_version 59642 (0.0009) [2023-12-26 15:48:02,638][105692] Updated weights for policy 0, policy_version 59652 (0.0008) [2023-12-26 15:48:02,695][105692] Updated weights for policy 0, policy_version 59662 (0.0008) [2023-12-26 15:48:03,211][105620] Updated weights for policy 1, policy_version 60151 (0.0008) [2023-12-26 15:48:03,257][105620] Updated weights for policy 1, policy_version 60161 (0.0008) [2023-12-26 15:48:03,304][105620] Updated weights for policy 1, policy_version 60171 (0.0008) [2023-12-26 15:48:03,388][105692] Updated weights for policy 0, policy_version 59672 (0.0009) [2023-12-26 15:48:03,433][105692] Updated weights for policy 0, policy_version 59682 (0.0009) [2023-12-26 15:48:03,482][105692] Updated weights for policy 0, policy_version 59692 (0.0008) [2023-12-26 15:48:03,991][105620] Updated weights for policy 1, policy_version 60181 (0.0009) [2023-12-26 15:48:04,046][105620] Updated weights for policy 1, policy_version 60191 (0.0009) [2023-12-26 15:48:04,098][105620] Updated weights for policy 1, policy_version 60201 (0.0009) [2023-12-26 15:48:04,290][105692] Updated weights for policy 0, policy_version 59702 (0.0009) [2023-12-26 15:48:04,346][105692] Updated weights for policy 0, policy_version 59712 (0.0008) [2023-12-26 15:48:04,399][105692] Updated weights for policy 0, policy_version 59722 (0.0008) [2023-12-26 15:48:04,801][105620] Updated weights for policy 1, policy_version 60211 (0.0009) [2023-12-26 15:48:04,852][105620] Updated weights for policy 1, policy_version 60221 (0.0009) [2023-12-26 15:48:04,918][105620] Updated weights for policy 1, policy_version 60232 (0.0010) [2023-12-26 15:48:05,138][105692] Updated weights for policy 0, policy_version 59732 (0.0008) [2023-12-26 15:48:05,192][105692] Updated weights for policy 0, policy_version 59742 (0.0009) [2023-12-26 15:48:05,238][105692] Updated weights for policy 0, policy_version 59752 (0.0008) [2023-12-26 15:48:05,715][105620] Updated weights for policy 1, policy_version 60243 (0.0009) [2023-12-26 15:48:05,763][105620] Updated weights for policy 1, policy_version 60253 (0.0009) [2023-12-26 15:48:05,814][105620] Updated weights for policy 1, policy_version 60264 (0.0010) [2023-12-26 15:48:05,942][105692] Updated weights for policy 0, policy_version 59762 (0.0009) [2023-12-26 15:48:05,988][105692] Updated weights for policy 0, policy_version 59772 (0.0009) [2023-12-26 15:48:06,035][105692] Updated weights for policy 0, policy_version 59782 (0.0008) [2023-12-26 15:48:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19744.1). Total num frames: 30736384. Throughput: 0: 9603.6, 1: 10001.2. Samples: 30727420. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-12-26 15:48:06,062][104569] Avg episode reward: [(0, '8843.728'), (1, '9354.785')] [2023-12-26 15:48:06,099][105692] Updated weights for policy 0, policy_version 59792 (0.0009) [2023-12-26 15:48:06,610][105620] Updated weights for policy 1, policy_version 60275 (0.0009) [2023-12-26 15:48:06,669][105620] Updated weights for policy 1, policy_version 60285 (0.0009) [2023-12-26 15:48:06,728][105620] Updated weights for policy 1, policy_version 60295 (0.0009) [2023-12-26 15:48:06,890][105692] Updated weights for policy 0, policy_version 59802 (0.0008) [2023-12-26 15:48:06,948][105692] Updated weights for policy 0, policy_version 59812 (0.0009) [2023-12-26 15:48:07,006][105692] Updated weights for policy 0, policy_version 59822 (0.0009) [2023-12-26 15:48:07,501][105620] Updated weights for policy 1, policy_version 60305 (0.0009) [2023-12-26 15:48:07,559][105620] Updated weights for policy 1, policy_version 60316 (0.0009) [2023-12-26 15:48:07,620][105620] Updated weights for policy 1, policy_version 60326 (0.0009) [2023-12-26 15:48:07,683][105620] Updated weights for policy 1, policy_version 60336 (0.0008) [2023-12-26 15:48:07,706][105692] Updated weights for policy 0, policy_version 59832 (0.0008) [2023-12-26 15:48:07,768][105692] Updated weights for policy 0, policy_version 59842 (0.0009) [2023-12-26 15:48:07,815][105692] Updated weights for policy 0, policy_version 59852 (0.0008) [2023-12-26 15:48:08,390][105620] Updated weights for policy 1, policy_version 60346 (0.0008) [2023-12-26 15:48:08,454][105620] Updated weights for policy 1, policy_version 60356 (0.0006) [2023-12-26 15:48:08,512][105620] Updated weights for policy 1, policy_version 60366 (0.0005) [2023-12-26 15:48:08,574][105692] Updated weights for policy 0, policy_version 59862 (0.0009) [2023-12-26 15:48:08,630][105692] Updated weights for policy 0, policy_version 59872 (0.0008) [2023-12-26 15:48:08,692][105692] Updated weights for policy 0, policy_version 59882 (0.0009) [2023-12-26 15:48:09,155][105620] Updated weights for policy 1, policy_version 60376 (0.0008) [2023-12-26 15:48:09,214][105620] Updated weights for policy 1, policy_version 60386 (0.0008) [2023-12-26 15:48:09,282][105620] Updated weights for policy 1, policy_version 60396 (0.0007) [2023-12-26 15:48:09,412][105692] Updated weights for policy 0, policy_version 59892 (0.0008) [2023-12-26 15:48:09,468][105692] Updated weights for policy 0, policy_version 59902 (0.0008) [2023-12-26 15:48:09,528][105692] Updated weights for policy 0, policy_version 59912 (0.0009) [2023-12-26 15:48:10,062][105620] Updated weights for policy 1, policy_version 60406 (0.0009) [2023-12-26 15:48:10,123][105620] Updated weights for policy 1, policy_version 60416 (0.0008) [2023-12-26 15:48:10,183][105620] Updated weights for policy 1, policy_version 60426 (0.0007) [2023-12-26 15:48:10,270][105692] Updated weights for policy 0, policy_version 59922 (0.0009) [2023-12-26 15:48:10,326][105692] Updated weights for policy 0, policy_version 59932 (0.0008) [2023-12-26 15:48:10,395][105692] Updated weights for policy 0, policy_version 59942 (0.0008) [2023-12-26 15:48:10,455][105692] Updated weights for policy 0, policy_version 59952 (0.0008) [2023-12-26 15:48:10,999][105620] Updated weights for policy 1, policy_version 60436 (0.0009) [2023-12-26 15:48:11,049][105692] Updated weights for policy 0, policy_version 59962 (0.0007) [2023-12-26 15:48:11,059][105620] Updated weights for policy 1, policy_version 60446 (0.0008) [2023-12-26 15:48:11,062][104569] Fps is (10 sec: 18022.7, 60 sec: 19387.8, 300 sec: 19688.6). Total num frames: 30826496. Throughput: 0: 9503.0, 1: 9990.9. Samples: 30841348. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-12-26 15:48:11,062][104569] Avg episode reward: [(0, '8408.407'), (1, '9352.613')] [2023-12-26 15:48:11,112][105692] Updated weights for policy 0, policy_version 59972 (0.0007) [2023-12-26 15:48:11,125][105620] Updated weights for policy 1, policy_version 60456 (0.0008) [2023-12-26 15:48:11,177][105692] Updated weights for policy 0, policy_version 59982 (0.0011) [2023-12-26 15:48:11,855][105692] Updated weights for policy 0, policy_version 59992 (0.0007) [2023-12-26 15:48:11,919][105692] Updated weights for policy 0, policy_version 60002 (0.0006) [2023-12-26 15:48:11,979][105692] Updated weights for policy 0, policy_version 60012 (0.0006) [2023-12-26 15:48:11,994][105620] Updated weights for policy 1, policy_version 60466 (0.0007) [2023-12-26 15:48:12,096][105620] Updated weights for policy 1, policy_version 60476 (0.0008) [2023-12-26 15:48:12,151][105620] Updated weights for policy 1, policy_version 60486 (0.0009) [2023-12-26 15:48:12,201][105620] Updated weights for policy 1, policy_version 60496 (0.0009) [2023-12-26 15:48:12,718][105692] Updated weights for policy 0, policy_version 60022 (0.0008) [2023-12-26 15:48:12,777][105692] Updated weights for policy 0, policy_version 60032 (0.0009) [2023-12-26 15:48:12,834][105692] Updated weights for policy 0, policy_version 60042 (0.0007) [2023-12-26 15:48:12,870][105620] Updated weights for policy 1, policy_version 60506 (0.0008) [2023-12-26 15:48:12,924][105620] Updated weights for policy 1, policy_version 60516 (0.0009) [2023-12-26 15:48:12,981][105620] Updated weights for policy 1, policy_version 60526 (0.0010) [2023-12-26 15:48:13,522][105692] Updated weights for policy 0, policy_version 60052 (0.0007) [2023-12-26 15:48:13,569][105692] Updated weights for policy 0, policy_version 60062 (0.0005) [2023-12-26 15:48:13,618][105692] Updated weights for policy 0, policy_version 60072 (0.0005) [2023-12-26 15:48:13,845][105620] Updated weights for policy 1, policy_version 60536 (0.0010) [2023-12-26 15:48:13,903][105620] Updated weights for policy 1, policy_version 60546 (0.0009) [2023-12-26 15:48:13,959][105620] Updated weights for policy 1, policy_version 60556 (0.0008) [2023-12-26 15:48:14,240][105692] Updated weights for policy 0, policy_version 60082 (0.0005) [2023-12-26 15:48:14,294][105692] Updated weights for policy 0, policy_version 60092 (0.0006) [2023-12-26 15:48:14,348][105692] Updated weights for policy 0, policy_version 60102 (0.0009) [2023-12-26 15:48:14,409][105692] Updated weights for policy 0, policy_version 60112 (0.0009) [2023-12-26 15:48:14,740][105620] Updated weights for policy 1, policy_version 60566 (0.0009) [2023-12-26 15:48:14,795][105620] Updated weights for policy 1, policy_version 60576 (0.0009) [2023-12-26 15:48:14,853][105620] Updated weights for policy 1, policy_version 60586 (0.0008) [2023-12-26 15:48:15,086][105692] Updated weights for policy 0, policy_version 60122 (0.0010) [2023-12-26 15:48:15,149][105692] Updated weights for policy 0, policy_version 60132 (0.0008) [2023-12-26 15:48:15,219][105692] Updated weights for policy 0, policy_version 60142 (0.0006) [2023-12-26 15:48:15,505][105620] Updated weights for policy 1, policy_version 60596 (0.0007) [2023-12-26 15:48:15,560][105620] Updated weights for policy 1, policy_version 60606 (0.0010) [2023-12-26 15:48:15,614][105620] Updated weights for policy 1, policy_version 60617 (0.0010) [2023-12-26 15:48:15,856][105692] Updated weights for policy 0, policy_version 60152 (0.0005) [2023-12-26 15:48:15,910][105692] Updated weights for policy 0, policy_version 60162 (0.0005) [2023-12-26 15:48:15,957][105692] Updated weights for policy 0, policy_version 60172 (0.0006) [2023-12-26 15:48:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.9, 300 sec: 19771.9). Total num frames: 30932992. Throughput: 0: 9531.9, 1: 9818.0. Samples: 30898052. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-12-26 15:48:16,062][104569] Avg episode reward: [(0, '8830.579'), (1, '9351.705')] [2023-12-26 15:48:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000060176_15409152.pth... [2023-12-26 15:48:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000060624_15523840.pth... [2023-12-26 15:48:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000059056_15122432.pth [2023-12-26 15:48:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000059504_15237120.pth [2023-12-26 15:48:16,473][105620] Updated weights for policy 1, policy_version 60627 (0.0009) [2023-12-26 15:48:16,531][105620] Updated weights for policy 1, policy_version 60637 (0.0008) [2023-12-26 15:48:16,578][105620] Updated weights for policy 1, policy_version 60647 (0.0007) [2023-12-26 15:48:16,609][105692] Updated weights for policy 0, policy_version 60182 (0.0009) [2023-12-26 15:48:16,654][105692] Updated weights for policy 0, policy_version 60192 (0.0010) [2023-12-26 15:48:16,698][105692] Updated weights for policy 0, policy_version 60202 (0.0010) [2023-12-26 15:48:17,369][105692] Updated weights for policy 0, policy_version 60212 (0.0008) [2023-12-26 15:48:17,405][105620] Updated weights for policy 1, policy_version 60657 (0.0006) [2023-12-26 15:48:17,424][105692] Updated weights for policy 0, policy_version 60222 (0.0008) [2023-12-26 15:48:17,470][105620] Updated weights for policy 1, policy_version 60667 (0.0006) [2023-12-26 15:48:17,500][105692] Updated weights for policy 0, policy_version 60232 (0.0011) [2023-12-26 15:48:17,527][105620] Updated weights for policy 1, policy_version 60677 (0.0006) [2023-12-26 15:48:17,591][105620] Updated weights for policy 1, policy_version 60687 (0.0007) [2023-12-26 15:48:18,126][105692] Updated weights for policy 0, policy_version 60242 (0.0010) [2023-12-26 15:48:18,190][105692] Updated weights for policy 0, policy_version 60252 (0.0009) [2023-12-26 15:48:18,246][105692] Updated weights for policy 0, policy_version 60262 (0.0008) [2023-12-26 15:48:18,282][105620] Updated weights for policy 1, policy_version 60697 (0.0007) [2023-12-26 15:48:18,292][105692] Updated weights for policy 0, policy_version 60272 (0.0008) [2023-12-26 15:48:18,332][105620] Updated weights for policy 1, policy_version 60707 (0.0007) [2023-12-26 15:48:18,388][105620] Updated weights for policy 1, policy_version 60717 (0.0009) [2023-12-26 15:48:19,001][105692] Updated weights for policy 0, policy_version 60282 (0.0009) [2023-12-26 15:48:19,047][105692] Updated weights for policy 0, policy_version 60292 (0.0009) [2023-12-26 15:48:19,098][105692] Updated weights for policy 0, policy_version 60302 (0.0009) [2023-12-26 15:48:19,185][105620] Updated weights for policy 1, policy_version 60727 (0.0008) [2023-12-26 15:48:19,258][105620] Updated weights for policy 1, policy_version 60737 (0.0009) [2023-12-26 15:48:19,308][105620] Updated weights for policy 1, policy_version 60747 (0.0009) [2023-12-26 15:48:19,863][105692] Updated weights for policy 0, policy_version 60312 (0.0009) [2023-12-26 15:48:19,928][105692] Updated weights for policy 0, policy_version 60322 (0.0009) [2023-12-26 15:48:19,990][105692] Updated weights for policy 0, policy_version 60332 (0.0006) [2023-12-26 15:48:20,094][105620] Updated weights for policy 1, policy_version 60757 (0.0009) [2023-12-26 15:48:20,153][105620] Updated weights for policy 1, policy_version 60767 (0.0009) [2023-12-26 15:48:20,205][105620] Updated weights for policy 1, policy_version 60777 (0.0009) [2023-12-26 15:48:20,651][105692] Updated weights for policy 0, policy_version 60342 (0.0006) [2023-12-26 15:48:20,714][105692] Updated weights for policy 0, policy_version 60352 (0.0007) [2023-12-26 15:48:20,781][105692] Updated weights for policy 0, policy_version 60362 (0.0009) [2023-12-26 15:48:20,997][105620] Updated weights for policy 1, policy_version 60787 (0.0009) [2023-12-26 15:48:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19716.3). Total num frames: 31023104. Throughput: 0: 9584.6, 1: 9703.8. Samples: 31015280. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-12-26 15:48:21,062][104569] Avg episode reward: [(0, '8904.250'), (1, '9350.375')] [2023-12-26 15:48:21,064][105620] Updated weights for policy 1, policy_version 60797 (0.0008) [2023-12-26 15:48:21,129][105620] Updated weights for policy 1, policy_version 60807 (0.0008) [2023-12-26 15:48:21,513][105692] Updated weights for policy 0, policy_version 60372 (0.0008) [2023-12-26 15:48:21,572][105692] Updated weights for policy 0, policy_version 60382 (0.0005) [2023-12-26 15:48:21,645][105692] Updated weights for policy 0, policy_version 60392 (0.0008) [2023-12-26 15:48:21,902][105620] Updated weights for policy 1, policy_version 60817 (0.0008) [2023-12-26 15:48:21,968][105620] Updated weights for policy 1, policy_version 60827 (0.0006) [2023-12-26 15:48:22,030][105620] Updated weights for policy 1, policy_version 60837 (0.0007) [2023-12-26 15:48:22,089][105620] Updated weights for policy 1, policy_version 60847 (0.0006) [2023-12-26 15:48:22,434][105692] Updated weights for policy 0, policy_version 60403 (0.0010) [2023-12-26 15:48:22,497][105692] Updated weights for policy 0, policy_version 60413 (0.0009) [2023-12-26 15:48:22,563][105692] Updated weights for policy 0, policy_version 60423 (0.0009) [2023-12-26 15:48:22,725][105620] Updated weights for policy 1, policy_version 60857 (0.0009) [2023-12-26 15:48:22,787][105620] Updated weights for policy 1, policy_version 60867 (0.0009) [2023-12-26 15:48:22,843][105620] Updated weights for policy 1, policy_version 60877 (0.0009) [2023-12-26 15:48:23,294][105692] Updated weights for policy 0, policy_version 60433 (0.0009) [2023-12-26 15:48:23,344][105692] Updated weights for policy 0, policy_version 60443 (0.0010) [2023-12-26 15:48:23,396][105692] Updated weights for policy 0, policy_version 60453 (0.0010) [2023-12-26 15:48:23,457][105692] Updated weights for policy 0, policy_version 60463 (0.0010) [2023-12-26 15:48:23,613][105620] Updated weights for policy 1, policy_version 60887 (0.0008) [2023-12-26 15:48:23,665][105620] Updated weights for policy 1, policy_version 60897 (0.0008) [2023-12-26 15:48:23,713][105620] Updated weights for policy 1, policy_version 60907 (0.0007) [2023-12-26 15:48:24,191][105692] Updated weights for policy 0, policy_version 60473 (0.0009) [2023-12-26 15:48:24,246][105692] Updated weights for policy 0, policy_version 60483 (0.0009) [2023-12-26 15:48:24,301][105692] Updated weights for policy 0, policy_version 60493 (0.0009) [2023-12-26 15:48:24,489][105620] Updated weights for policy 1, policy_version 60917 (0.0007) [2023-12-26 15:48:24,547][105620] Updated weights for policy 1, policy_version 60927 (0.0006) [2023-12-26 15:48:24,603][105620] Updated weights for policy 1, policy_version 60937 (0.0008) [2023-12-26 15:48:25,016][105692] Updated weights for policy 0, policy_version 60503 (0.0006) [2023-12-26 15:48:25,059][105692] Updated weights for policy 0, policy_version 60513 (0.0005) [2023-12-26 15:48:25,108][105692] Updated weights for policy 0, policy_version 60523 (0.0005) [2023-12-26 15:48:25,415][105620] Updated weights for policy 1, policy_version 60947 (0.0009) [2023-12-26 15:48:25,476][105620] Updated weights for policy 1, policy_version 60957 (0.0009) [2023-12-26 15:48:25,536][105620] Updated weights for policy 1, policy_version 60967 (0.0009) [2023-12-26 15:48:25,669][105692] Updated weights for policy 0, policy_version 60533 (0.0005) [2023-12-26 15:48:25,728][105692] Updated weights for policy 0, policy_version 60543 (0.0005) [2023-12-26 15:48:25,792][105692] Updated weights for policy 0, policy_version 60553 (0.0009) [2023-12-26 15:48:26,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 31121408. Throughput: 0: 9657.5, 1: 9589.6. Samples: 31129492. Policy #0 lag: (min: 34.0, avg: 47.7, max: 48.0) [2023-12-26 15:48:26,063][104569] Avg episode reward: [(0, '8540.860'), (1, '9258.532')] [2023-12-26 15:48:26,344][105692] Updated weights for policy 0, policy_version 60563 (0.0006) [2023-12-26 15:48:26,399][105692] Updated weights for policy 0, policy_version 60573 (0.0009) [2023-12-26 15:48:26,401][105620] Updated weights for policy 1, policy_version 60977 (0.0010) [2023-12-26 15:48:26,452][105692] Updated weights for policy 0, policy_version 60583 (0.0005) [2023-12-26 15:48:26,453][105620] Updated weights for policy 1, policy_version 60987 (0.0008) [2023-12-26 15:48:26,503][105620] Updated weights for policy 1, policy_version 60997 (0.0007) [2023-12-26 15:48:26,557][105620] Updated weights for policy 1, policy_version 61007 (0.0009) [2023-12-26 15:48:27,127][105692] Updated weights for policy 0, policy_version 60593 (0.0006) [2023-12-26 15:48:27,202][105692] Updated weights for policy 0, policy_version 60603 (0.0005) [2023-12-26 15:48:27,266][105692] Updated weights for policy 0, policy_version 60613 (0.0005) [2023-12-26 15:48:27,305][105620] Updated weights for policy 1, policy_version 61017 (0.0007) [2023-12-26 15:48:27,322][105692] Updated weights for policy 0, policy_version 60623 (0.0007) [2023-12-26 15:48:27,354][105620] Updated weights for policy 1, policy_version 61027 (0.0009) [2023-12-26 15:48:27,405][105620] Updated weights for policy 1, policy_version 61037 (0.0010) [2023-12-26 15:48:28,006][105620] Updated weights for policy 1, policy_version 61047 (0.0010) [2023-12-26 15:48:28,025][105692] Updated weights for policy 0, policy_version 60633 (0.0005) [2023-12-26 15:48:28,058][105620] Updated weights for policy 1, policy_version 61057 (0.0011) [2023-12-26 15:48:28,089][105692] Updated weights for policy 0, policy_version 60643 (0.0006) [2023-12-26 15:48:28,111][105620] Updated weights for policy 1, policy_version 61067 (0.0010) [2023-12-26 15:48:28,146][105692] Updated weights for policy 0, policy_version 60653 (0.0007) [2023-12-26 15:48:28,781][105620] Updated weights for policy 1, policy_version 61077 (0.0010) [2023-12-26 15:48:28,830][105620] Updated weights for policy 1, policy_version 61087 (0.0010) [2023-12-26 15:48:28,878][105620] Updated weights for policy 1, policy_version 61097 (0.0010) [2023-12-26 15:48:28,927][105692] Updated weights for policy 0, policy_version 60663 (0.0006) [2023-12-26 15:48:28,983][105692] Updated weights for policy 0, policy_version 60673 (0.0008) [2023-12-26 15:48:29,041][105692] Updated weights for policy 0, policy_version 60683 (0.0008) [2023-12-26 15:48:29,615][105620] Updated weights for policy 1, policy_version 61108 (0.0011) [2023-12-26 15:48:29,668][105620] Updated weights for policy 1, policy_version 61118 (0.0009) [2023-12-26 15:48:29,713][105692] Updated weights for policy 0, policy_version 60693 (0.0007) [2023-12-26 15:48:29,730][105620] Updated weights for policy 1, policy_version 61128 (0.0007) [2023-12-26 15:48:29,765][105692] Updated weights for policy 0, policy_version 60703 (0.0005) [2023-12-26 15:48:29,819][105692] Updated weights for policy 0, policy_version 60713 (0.0006) [2023-12-26 15:48:30,465][105692] Updated weights for policy 0, policy_version 60723 (0.0009) [2023-12-26 15:48:30,517][105692] Updated weights for policy 0, policy_version 60733 (0.0008) [2023-12-26 15:48:30,520][105620] Updated weights for policy 1, policy_version 61138 (0.0007) [2023-12-26 15:48:30,571][105692] Updated weights for policy 0, policy_version 60743 (0.0006) [2023-12-26 15:48:30,572][105620] Updated weights for policy 1, policy_version 61148 (0.0008) [2023-12-26 15:48:30,623][105620] Updated weights for policy 1, policy_version 61158 (0.0010) [2023-12-26 15:48:30,676][105620] Updated weights for policy 1, policy_version 61168 (0.0010) [2023-12-26 15:48:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 31219712. Throughput: 0: 9712.2, 1: 9590.2. Samples: 31189680. Policy #0 lag: (min: 34.0, avg: 47.7, max: 48.0) [2023-12-26 15:48:31,063][104569] Avg episode reward: [(0, '8542.780'), (1, '9168.084')] [2023-12-26 15:48:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000060752_15556608.pth... [2023-12-26 15:48:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000061168_15663104.pth... [2023-12-26 15:48:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000059600_15261696.pth [2023-12-26 15:48:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000060080_15384576.pth [2023-12-26 15:48:31,240][105692] Updated weights for policy 0, policy_version 60753 (0.0006) [2023-12-26 15:48:31,295][105692] Updated weights for policy 0, policy_version 60763 (0.0006) [2023-12-26 15:48:31,351][105692] Updated weights for policy 0, policy_version 60773 (0.0007) [2023-12-26 15:48:31,421][105692] Updated weights for policy 0, policy_version 60783 (0.0007) [2023-12-26 15:48:31,515][105620] Updated weights for policy 1, policy_version 61178 (0.0009) [2023-12-26 15:48:31,564][105620] Updated weights for policy 1, policy_version 61188 (0.0008) [2023-12-26 15:48:31,624][105620] Updated weights for policy 1, policy_version 61198 (0.0008) [2023-12-26 15:48:32,005][105692] Updated weights for policy 0, policy_version 60793 (0.0007) [2023-12-26 15:48:32,071][105692] Updated weights for policy 0, policy_version 60803 (0.0009) [2023-12-26 15:48:32,125][105692] Updated weights for policy 0, policy_version 60813 (0.0009) [2023-12-26 15:48:32,484][105620] Updated weights for policy 1, policy_version 61208 (0.0009) [2023-12-26 15:48:32,532][105620] Updated weights for policy 1, policy_version 61218 (0.0008) [2023-12-26 15:48:32,580][105620] Updated weights for policy 1, policy_version 61228 (0.0009) [2023-12-26 15:48:32,811][105692] Updated weights for policy 0, policy_version 60823 (0.0009) [2023-12-26 15:48:32,859][105692] Updated weights for policy 0, policy_version 60833 (0.0009) [2023-12-26 15:48:32,908][105692] Updated weights for policy 0, policy_version 60843 (0.0006) [2023-12-26 15:48:33,367][105620] Updated weights for policy 1, policy_version 61238 (0.0007) [2023-12-26 15:48:33,426][105620] Updated weights for policy 1, policy_version 61248 (0.0005) [2023-12-26 15:48:33,478][105620] Updated weights for policy 1, policy_version 61258 (0.0005) [2023-12-26 15:48:33,645][105692] Updated weights for policy 0, policy_version 60853 (0.0007) [2023-12-26 15:48:33,697][105692] Updated weights for policy 0, policy_version 60864 (0.0010) [2023-12-26 15:48:33,750][105692] Updated weights for policy 0, policy_version 60876 (0.0010) [2023-12-26 15:48:34,027][105620] Updated weights for policy 1, policy_version 61268 (0.0005) [2023-12-26 15:48:34,083][105620] Updated weights for policy 1, policy_version 61278 (0.0005) [2023-12-26 15:48:34,143][105620] Updated weights for policy 1, policy_version 61288 (0.0006) [2023-12-26 15:48:34,486][105692] Updated weights for policy 0, policy_version 60886 (0.0007) [2023-12-26 15:48:34,550][105692] Updated weights for policy 0, policy_version 60896 (0.0006) [2023-12-26 15:48:34,608][105692] Updated weights for policy 0, policy_version 60906 (0.0007) [2023-12-26 15:48:34,790][105620] Updated weights for policy 1, policy_version 61298 (0.0007) [2023-12-26 15:48:34,853][105620] Updated weights for policy 1, policy_version 61308 (0.0007) [2023-12-26 15:48:34,918][105620] Updated weights for policy 1, policy_version 61318 (0.0007) [2023-12-26 15:48:34,985][105620] Updated weights for policy 1, policy_version 61328 (0.0006) [2023-12-26 15:48:35,313][105692] Updated weights for policy 0, policy_version 60916 (0.0007) [2023-12-26 15:48:35,370][105692] Updated weights for policy 0, policy_version 60926 (0.0005) [2023-12-26 15:48:35,425][105692] Updated weights for policy 0, policy_version 60936 (0.0005) [2023-12-26 15:48:35,664][105620] Updated weights for policy 1, policy_version 61338 (0.0011) [2023-12-26 15:48:35,725][105620] Updated weights for policy 1, policy_version 61349 (0.0010) [2023-12-26 15:48:35,779][105620] Updated weights for policy 1, policy_version 61360 (0.0010) [2023-12-26 15:48:35,935][105692] Updated weights for policy 0, policy_version 60946 (0.0005) [2023-12-26 15:48:35,988][105692] Updated weights for policy 0, policy_version 60956 (0.0005) [2023-12-26 15:48:36,046][105692] Updated weights for policy 0, policy_version 60966 (0.0005) [2023-12-26 15:48:36,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19688.6). Total num frames: 31318016. Throughput: 0: 9797.2, 1: 9490.0. Samples: 31308628. Policy #0 lag: (min: 34.0, avg: 47.7, max: 48.0) [2023-12-26 15:48:36,062][104569] Avg episode reward: [(0, '8558.288'), (1, '9260.816')] [2023-12-26 15:48:36,108][105692] Updated weights for policy 0, policy_version 60976 (0.0006) [2023-12-26 15:48:36,561][105620] Updated weights for policy 1, policy_version 61370 (0.0009) [2023-12-26 15:48:36,620][105620] Updated weights for policy 1, policy_version 61380 (0.0007) [2023-12-26 15:48:36,688][105620] Updated weights for policy 1, policy_version 61390 (0.0005) [2023-12-26 15:48:36,770][105692] Updated weights for policy 0, policy_version 60986 (0.0010) [2023-12-26 15:48:36,832][105692] Updated weights for policy 0, policy_version 60996 (0.0010) [2023-12-26 15:48:36,896][105692] Updated weights for policy 0, policy_version 61006 (0.0008) [2023-12-26 15:48:37,294][105620] Updated weights for policy 1, policy_version 61400 (0.0006) [2023-12-26 15:48:37,361][105620] Updated weights for policy 1, policy_version 61410 (0.0007) [2023-12-26 15:48:37,415][105620] Updated weights for policy 1, policy_version 61420 (0.0008) [2023-12-26 15:48:37,635][105692] Updated weights for policy 0, policy_version 61016 (0.0008) [2023-12-26 15:48:37,699][105692] Updated weights for policy 0, policy_version 61026 (0.0009) [2023-12-26 15:48:37,762][105692] Updated weights for policy 0, policy_version 61036 (0.0009) [2023-12-26 15:48:38,061][105620] Updated weights for policy 1, policy_version 61430 (0.0007) [2023-12-26 15:48:38,133][105620] Updated weights for policy 1, policy_version 61440 (0.0006) [2023-12-26 15:48:38,191][105620] Updated weights for policy 1, policy_version 61450 (0.0005) [2023-12-26 15:48:38,488][105692] Updated weights for policy 0, policy_version 61046 (0.0009) [2023-12-26 15:48:38,557][105692] Updated weights for policy 0, policy_version 61056 (0.0009) [2023-12-26 15:48:38,607][105692] Updated weights for policy 0, policy_version 61066 (0.0008) [2023-12-26 15:48:38,774][105620] Updated weights for policy 1, policy_version 61460 (0.0007) [2023-12-26 15:48:38,836][105620] Updated weights for policy 1, policy_version 61470 (0.0009) [2023-12-26 15:48:38,902][105620] Updated weights for policy 1, policy_version 61480 (0.0008) [2023-12-26 15:48:39,462][105692] Updated weights for policy 0, policy_version 61076 (0.0008) [2023-12-26 15:48:39,525][105692] Updated weights for policy 0, policy_version 61086 (0.0009) [2023-12-26 15:48:39,552][105620] Updated weights for policy 1, policy_version 61490 (0.0007) [2023-12-26 15:48:39,579][105692] Updated weights for policy 0, policy_version 61096 (0.0008) [2023-12-26 15:48:39,602][105620] Updated weights for policy 1, policy_version 61500 (0.0006) [2023-12-26 15:48:39,654][105620] Updated weights for policy 1, policy_version 61510 (0.0008) [2023-12-26 15:48:39,708][105620] Updated weights for policy 1, policy_version 61520 (0.0009) [2023-12-26 15:48:40,403][105692] Updated weights for policy 0, policy_version 61106 (0.0008) [2023-12-26 15:48:40,445][105620] Updated weights for policy 1, policy_version 61530 (0.0011) [2023-12-26 15:48:40,464][105692] Updated weights for policy 0, policy_version 61116 (0.0006) [2023-12-26 15:48:40,507][105620] Updated weights for policy 1, policy_version 61540 (0.0011) [2023-12-26 15:48:40,521][105692] Updated weights for policy 0, policy_version 61126 (0.0006) [2023-12-26 15:48:40,566][105620] Updated weights for policy 1, policy_version 61550 (0.0011) [2023-12-26 15:48:40,569][105692] Updated weights for policy 0, policy_version 61136 (0.0007) [2023-12-26 15:48:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 31416320. Throughput: 0: 9846.9, 1: 9577.5. Samples: 31428412. Policy #0 lag: (min: 34.0, avg: 47.7, max: 48.0) [2023-12-26 15:48:41,063][104569] Avg episode reward: [(0, '8838.921'), (1, '9353.003')] [2023-12-26 15:48:41,145][105620] Updated weights for policy 1, policy_version 61560 (0.0009) [2023-12-26 15:48:41,204][105620] Updated weights for policy 1, policy_version 61570 (0.0009) [2023-12-26 15:48:41,272][105620] Updated weights for policy 1, policy_version 61580 (0.0009) [2023-12-26 15:48:41,431][105692] Updated weights for policy 0, policy_version 61146 (0.0009) [2023-12-26 15:48:41,485][105692] Updated weights for policy 0, policy_version 61156 (0.0008) [2023-12-26 15:48:41,549][105692] Updated weights for policy 0, policy_version 61166 (0.0008) [2023-12-26 15:48:42,060][105620] Updated weights for policy 1, policy_version 61590 (0.0009) [2023-12-26 15:48:42,123][105620] Updated weights for policy 1, policy_version 61600 (0.0009) [2023-12-26 15:48:42,189][105620] Updated weights for policy 1, policy_version 61610 (0.0009) [2023-12-26 15:48:42,303][105692] Updated weights for policy 0, policy_version 61176 (0.0009) [2023-12-26 15:48:42,360][105692] Updated weights for policy 0, policy_version 61186 (0.0009) [2023-12-26 15:48:42,423][105692] Updated weights for policy 0, policy_version 61196 (0.0009) [2023-12-26 15:48:42,910][105620] Updated weights for policy 1, policy_version 61620 (0.0008) [2023-12-26 15:48:42,969][105620] Updated weights for policy 1, policy_version 61630 (0.0006) [2023-12-26 15:48:43,025][105620] Updated weights for policy 1, policy_version 61640 (0.0005) [2023-12-26 15:48:43,180][105692] Updated weights for policy 0, policy_version 61206 (0.0009) [2023-12-26 15:48:43,243][105692] Updated weights for policy 0, policy_version 61216 (0.0009) [2023-12-26 15:48:43,305][105692] Updated weights for policy 0, policy_version 61226 (0.0009) [2023-12-26 15:48:43,587][105620] Updated weights for policy 1, policy_version 61650 (0.0006) [2023-12-26 15:48:43,652][105620] Updated weights for policy 1, policy_version 61660 (0.0010) [2023-12-26 15:48:43,717][105620] Updated weights for policy 1, policy_version 61670 (0.0009) [2023-12-26 15:48:43,771][105620] Updated weights for policy 1, policy_version 61680 (0.0009) [2023-12-26 15:48:43,982][105692] Updated weights for policy 0, policy_version 61236 (0.0008) [2023-12-26 15:48:44,043][105692] Updated weights for policy 0, policy_version 61246 (0.0006) [2023-12-26 15:48:44,105][105692] Updated weights for policy 0, policy_version 61256 (0.0007) [2023-12-26 15:48:44,531][105620] Updated weights for policy 1, policy_version 61690 (0.0009) [2023-12-26 15:48:44,593][105620] Updated weights for policy 1, policy_version 61700 (0.0010) [2023-12-26 15:48:44,655][105620] Updated weights for policy 1, policy_version 61710 (0.0010) [2023-12-26 15:48:44,759][105692] Updated weights for policy 0, policy_version 61266 (0.0008) [2023-12-26 15:48:44,824][105692] Updated weights for policy 0, policy_version 61276 (0.0008) [2023-12-26 15:48:44,885][105692] Updated weights for policy 0, policy_version 61286 (0.0008) [2023-12-26 15:48:44,948][105692] Updated weights for policy 0, policy_version 61296 (0.0008) [2023-12-26 15:48:45,384][105620] Updated weights for policy 1, policy_version 61720 (0.0009) [2023-12-26 15:48:45,438][105620] Updated weights for policy 1, policy_version 61730 (0.0010) [2023-12-26 15:48:45,486][105620] Updated weights for policy 1, policy_version 61740 (0.0010) [2023-12-26 15:48:45,664][105692] Updated weights for policy 0, policy_version 61306 (0.0008) [2023-12-26 15:48:45,715][105692] Updated weights for policy 0, policy_version 61316 (0.0008) [2023-12-26 15:48:45,768][105692] Updated weights for policy 0, policy_version 61326 (0.0006) [2023-12-26 15:48:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 31514624. Throughput: 0: 9845.6, 1: 9560.0. Samples: 31485100. Policy #0 lag: (min: 34.0, avg: 47.7, max: 48.0) [2023-12-26 15:48:46,063][104569] Avg episode reward: [(0, '9021.938'), (1, '9353.773')] [2023-12-26 15:48:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000061328_15704064.pth... [2023-12-26 15:48:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000061744_15810560.pth... [2023-12-26 15:48:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000060624_15523840.pth [2023-12-26 15:48:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000060176_15409152.pth [2023-12-26 15:48:46,214][105620] Updated weights for policy 1, policy_version 61750 (0.0007) [2023-12-26 15:48:46,279][105620] Updated weights for policy 1, policy_version 61760 (0.0010) [2023-12-26 15:48:46,346][105620] Updated weights for policy 1, policy_version 61770 (0.0007) [2023-12-26 15:48:46,460][105692] Updated weights for policy 0, policy_version 61336 (0.0008) [2023-12-26 15:48:46,524][105692] Updated weights for policy 0, policy_version 61346 (0.0010) [2023-12-26 15:48:46,572][105692] Updated weights for policy 0, policy_version 61356 (0.0010) [2023-12-26 15:48:46,984][105620] Updated weights for policy 1, policy_version 61780 (0.0007) [2023-12-26 15:48:47,040][105620] Updated weights for policy 1, policy_version 61790 (0.0008) [2023-12-26 15:48:47,104][105620] Updated weights for policy 1, policy_version 61800 (0.0008) [2023-12-26 15:48:47,301][105692] Updated weights for policy 0, policy_version 61366 (0.0010) [2023-12-26 15:48:47,358][105692] Updated weights for policy 0, policy_version 61376 (0.0010) [2023-12-26 15:48:47,415][105692] Updated weights for policy 0, policy_version 61386 (0.0010) [2023-12-26 15:48:47,873][105620] Updated weights for policy 1, policy_version 61810 (0.0008) [2023-12-26 15:48:47,922][105620] Updated weights for policy 1, policy_version 61820 (0.0008) [2023-12-26 15:48:47,971][105620] Updated weights for policy 1, policy_version 61830 (0.0008) [2023-12-26 15:48:48,028][105620] Updated weights for policy 1, policy_version 61840 (0.0008) [2023-12-26 15:48:48,161][105692] Updated weights for policy 0, policy_version 61396 (0.0010) [2023-12-26 15:48:48,221][105692] Updated weights for policy 0, policy_version 61406 (0.0011) [2023-12-26 15:48:48,272][105692] Updated weights for policy 0, policy_version 61416 (0.0010) [2023-12-26 15:48:48,820][105620] Updated weights for policy 1, policy_version 61850 (0.0010) [2023-12-26 15:48:48,879][105620] Updated weights for policy 1, policy_version 61860 (0.0010) [2023-12-26 15:48:48,935][105620] Updated weights for policy 1, policy_version 61870 (0.0010) [2023-12-26 15:48:48,955][105692] Updated weights for policy 0, policy_version 61426 (0.0010) [2023-12-26 15:48:49,004][105692] Updated weights for policy 0, policy_version 61436 (0.0010) [2023-12-26 15:48:49,054][105692] Updated weights for policy 0, policy_version 61446 (0.0006) [2023-12-26 15:48:49,116][105692] Updated weights for policy 0, policy_version 61456 (0.0005) [2023-12-26 15:48:49,653][105620] Updated weights for policy 1, policy_version 61880 (0.0006) [2023-12-26 15:48:49,713][105620] Updated weights for policy 1, policy_version 61890 (0.0005) [2023-12-26 15:48:49,769][105620] Updated weights for policy 1, policy_version 61900 (0.0005) [2023-12-26 15:48:49,860][105692] Updated weights for policy 0, policy_version 61466 (0.0008) [2023-12-26 15:48:49,928][105692] Updated weights for policy 0, policy_version 61476 (0.0007) [2023-12-26 15:48:49,990][105692] Updated weights for policy 0, policy_version 61486 (0.0006) [2023-12-26 15:48:50,510][105620] Updated weights for policy 1, policy_version 61910 (0.0007) [2023-12-26 15:48:50,566][105620] Updated weights for policy 1, policy_version 61920 (0.0009) [2023-12-26 15:48:50,625][105692] Updated weights for policy 0, policy_version 61496 (0.0010) [2023-12-26 15:48:50,632][105620] Updated weights for policy 1, policy_version 61930 (0.0010) [2023-12-26 15:48:50,688][105692] Updated weights for policy 0, policy_version 61506 (0.0011) [2023-12-26 15:48:50,751][105692] Updated weights for policy 0, policy_version 61516 (0.0011) [2023-12-26 15:48:51,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 31612928. Throughput: 0: 9914.5, 1: 9530.8. Samples: 31602460. Policy #0 lag: (min: 34.0, avg: 47.7, max: 48.0) [2023-12-26 15:48:51,062][104569] Avg episode reward: [(0, '9266.330'), (1, '9260.472')] [2023-12-26 15:48:51,347][105692] Updated weights for policy 0, policy_version 61526 (0.0011) [2023-12-26 15:48:51,415][105692] Updated weights for policy 0, policy_version 61536 (0.0008) [2023-12-26 15:48:51,427][105620] Updated weights for policy 1, policy_version 61940 (0.0007) [2023-12-26 15:48:51,473][105692] Updated weights for policy 0, policy_version 61546 (0.0005) [2023-12-26 15:48:51,484][105620] Updated weights for policy 1, policy_version 61950 (0.0009) [2023-12-26 15:48:51,539][105620] Updated weights for policy 1, policy_version 61960 (0.0008) [2023-12-26 15:48:52,143][105692] Updated weights for policy 0, policy_version 61556 (0.0006) [2023-12-26 15:48:52,195][105692] Updated weights for policy 0, policy_version 61566 (0.0009) [2023-12-26 15:48:52,257][105692] Updated weights for policy 0, policy_version 61576 (0.0008) [2023-12-26 15:48:52,362][105620] Updated weights for policy 1, policy_version 61970 (0.0008) [2023-12-26 15:48:52,424][105620] Updated weights for policy 1, policy_version 61980 (0.0007) [2023-12-26 15:48:52,486][105620] Updated weights for policy 1, policy_version 61990 (0.0006) [2023-12-26 15:48:52,537][105620] Updated weights for policy 1, policy_version 62000 (0.0005) [2023-12-26 15:48:52,923][105692] Updated weights for policy 0, policy_version 61586 (0.0008) [2023-12-26 15:48:52,976][105692] Updated weights for policy 0, policy_version 61596 (0.0010) [2023-12-26 15:48:53,035][105692] Updated weights for policy 0, policy_version 61607 (0.0012) [2023-12-26 15:48:53,216][105620] Updated weights for policy 1, policy_version 62010 (0.0009) [2023-12-26 15:48:53,282][105620] Updated weights for policy 1, policy_version 62020 (0.0009) [2023-12-26 15:48:53,336][105620] Updated weights for policy 1, policy_version 62031 (0.0010) [2023-12-26 15:48:53,625][105692] Updated weights for policy 0, policy_version 61617 (0.0006) [2023-12-26 15:48:53,686][105692] Updated weights for policy 0, policy_version 61627 (0.0008) [2023-12-26 15:48:53,737][105692] Updated weights for policy 0, policy_version 61637 (0.0009) [2023-12-26 15:48:53,791][105692] Updated weights for policy 0, policy_version 61647 (0.0009) [2023-12-26 15:48:53,980][105620] Updated weights for policy 1, policy_version 62041 (0.0009) [2023-12-26 15:48:54,043][105620] Updated weights for policy 1, policy_version 62051 (0.0009) [2023-12-26 15:48:54,107][105620] Updated weights for policy 1, policy_version 62061 (0.0008) [2023-12-26 15:48:54,548][105692] Updated weights for policy 0, policy_version 61657 (0.0011) [2023-12-26 15:48:54,603][105692] Updated weights for policy 0, policy_version 61667 (0.0011) [2023-12-26 15:48:54,651][105692] Updated weights for policy 0, policy_version 61677 (0.0010) [2023-12-26 15:48:54,765][105620] Updated weights for policy 1, policy_version 62071 (0.0008) [2023-12-26 15:48:54,833][105620] Updated weights for policy 1, policy_version 62081 (0.0005) [2023-12-26 15:48:54,897][105620] Updated weights for policy 1, policy_version 62091 (0.0005) [2023-12-26 15:48:55,435][105692] Updated weights for policy 0, policy_version 61687 (0.0009) [2023-12-26 15:48:55,444][105620] Updated weights for policy 1, policy_version 62101 (0.0007) [2023-12-26 15:48:55,494][105692] Updated weights for policy 0, policy_version 61697 (0.0007) [2023-12-26 15:48:55,496][105620] Updated weights for policy 1, policy_version 62111 (0.0006) [2023-12-26 15:48:55,543][105620] Updated weights for policy 1, policy_version 62121 (0.0005) [2023-12-26 15:48:55,548][105692] Updated weights for policy 0, policy_version 61707 (0.0008) [2023-12-26 15:48:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.8, 300 sec: 19660.8). Total num frames: 31711232. Throughput: 0: 9974.8, 1: 9620.2. Samples: 31723120. Policy #0 lag: (min: 34.0, avg: 47.7, max: 48.0) [2023-12-26 15:48:56,062][104569] Avg episode reward: [(0, '9083.004'), (1, '9085.790')] [2023-12-26 15:48:56,147][105620] Updated weights for policy 1, policy_version 62131 (0.0006) [2023-12-26 15:48:56,193][105620] Updated weights for policy 1, policy_version 62141 (0.0005) [2023-12-26 15:48:56,259][105620] Updated weights for policy 1, policy_version 62151 (0.0005) [2023-12-26 15:48:56,369][105692] Updated weights for policy 0, policy_version 61718 (0.0010) [2023-12-26 15:48:56,427][105692] Updated weights for policy 0, policy_version 61728 (0.0011) [2023-12-26 15:48:56,488][105692] Updated weights for policy 0, policy_version 61738 (0.0006) [2023-12-26 15:48:56,783][105620] Updated weights for policy 1, policy_version 62161 (0.0005) [2023-12-26 15:48:56,833][105620] Updated weights for policy 1, policy_version 62171 (0.0005) [2023-12-26 15:48:56,884][105620] Updated weights for policy 1, policy_version 62181 (0.0005) [2023-12-26 15:48:56,931][105620] Updated weights for policy 1, policy_version 62191 (0.0005) [2023-12-26 15:48:57,060][105692] Updated weights for policy 0, policy_version 61748 (0.0007) [2023-12-26 15:48:57,104][105692] Updated weights for policy 0, policy_version 61758 (0.0010) [2023-12-26 15:48:57,159][105692] Updated weights for policy 0, policy_version 61768 (0.0010) [2023-12-26 15:48:57,476][105620] Updated weights for policy 1, policy_version 62201 (0.0005) [2023-12-26 15:48:57,538][105620] Updated weights for policy 1, policy_version 62211 (0.0006) [2023-12-26 15:48:57,597][105620] Updated weights for policy 1, policy_version 62221 (0.0005) [2023-12-26 15:48:57,920][105692] Updated weights for policy 0, policy_version 61778 (0.0011) [2023-12-26 15:48:57,975][105692] Updated weights for policy 0, policy_version 61788 (0.0011) [2023-12-26 15:48:58,044][105692] Updated weights for policy 0, policy_version 61798 (0.0010) [2023-12-26 15:48:58,110][105692] Updated weights for policy 0, policy_version 61808 (0.0011) [2023-12-26 15:48:58,189][105620] Updated weights for policy 1, policy_version 62231 (0.0007) [2023-12-26 15:48:58,254][105620] Updated weights for policy 1, policy_version 62241 (0.0007) [2023-12-26 15:48:58,316][105620] Updated weights for policy 1, policy_version 62251 (0.0008) [2023-12-26 15:48:58,914][105692] Updated weights for policy 0, policy_version 61818 (0.0010) [2023-12-26 15:48:58,978][105692] Updated weights for policy 0, policy_version 61828 (0.0008) [2023-12-26 15:48:59,043][105692] Updated weights for policy 0, policy_version 61838 (0.0008) [2023-12-26 15:48:59,065][105620] Updated weights for policy 1, policy_version 62261 (0.0008) [2023-12-26 15:48:59,123][105620] Updated weights for policy 1, policy_version 62271 (0.0007) [2023-12-26 15:48:59,180][105620] Updated weights for policy 1, policy_version 62281 (0.0008) [2023-12-26 15:48:59,731][105692] Updated weights for policy 0, policy_version 61848 (0.0007) [2023-12-26 15:48:59,787][105692] Updated weights for policy 0, policy_version 61858 (0.0008) [2023-12-26 15:48:59,860][105692] Updated weights for policy 0, policy_version 61868 (0.0007) [2023-12-26 15:48:59,937][105620] Updated weights for policy 1, policy_version 62291 (0.0008) [2023-12-26 15:48:59,998][105620] Updated weights for policy 1, policy_version 62301 (0.0010) [2023-12-26 15:49:00,060][105620] Updated weights for policy 1, policy_version 62311 (0.0009) [2023-12-26 15:49:00,472][105692] Updated weights for policy 0, policy_version 61878 (0.0005) [2023-12-26 15:49:00,524][105692] Updated weights for policy 0, policy_version 61888 (0.0005) [2023-12-26 15:49:00,591][105692] Updated weights for policy 0, policy_version 61898 (0.0006) [2023-12-26 15:49:00,728][105620] Updated weights for policy 1, policy_version 62321 (0.0008) [2023-12-26 15:49:00,788][105620] Updated weights for policy 1, policy_version 62331 (0.0009) [2023-12-26 15:49:00,835][105620] Updated weights for policy 1, policy_version 62341 (0.0009) [2023-12-26 15:49:00,887][105620] Updated weights for policy 1, policy_version 62352 (0.0010) [2023-12-26 15:49:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 31817728. Throughput: 0: 9941.7, 1: 9793.6. Samples: 31786140. Policy #0 lag: (min: 34.0, avg: 47.7, max: 48.0) [2023-12-26 15:49:01,063][104569] Avg episode reward: [(0, '9356.327'), (1, '9005.148')] [2023-12-26 15:49:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000061904_15851520.pth... [2023-12-26 15:49:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000062352_15966208.pth... [2023-12-26 15:49:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000060752_15556608.pth [2023-12-26 15:49:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000061168_15663104.pth [2023-12-26 15:49:01,172][105692] Updated weights for policy 0, policy_version 61908 (0.0005) [2023-12-26 15:49:01,236][105692] Updated weights for policy 0, policy_version 61918 (0.0006) [2023-12-26 15:49:01,298][105692] Updated weights for policy 0, policy_version 61928 (0.0009) [2023-12-26 15:49:01,716][105620] Updated weights for policy 1, policy_version 62362 (0.0011) [2023-12-26 15:49:01,774][105620] Updated weights for policy 1, policy_version 62372 (0.0007) [2023-12-26 15:49:01,833][105620] Updated weights for policy 1, policy_version 62382 (0.0005) [2023-12-26 15:49:01,961][105692] Updated weights for policy 0, policy_version 61938 (0.0009) [2023-12-26 15:49:02,014][105692] Updated weights for policy 0, policy_version 61948 (0.0009) [2023-12-26 15:49:02,066][105692] Updated weights for policy 0, policy_version 61958 (0.0009) [2023-12-26 15:49:02,118][105692] Updated weights for policy 0, policy_version 61968 (0.0008) [2023-12-26 15:49:02,461][105620] Updated weights for policy 1, policy_version 62392 (0.0005) [2023-12-26 15:49:02,522][105620] Updated weights for policy 1, policy_version 62402 (0.0005) [2023-12-26 15:49:02,579][105620] Updated weights for policy 1, policy_version 62412 (0.0007) [2023-12-26 15:49:02,955][105692] Updated weights for policy 0, policy_version 61978 (0.0008) [2023-12-26 15:49:03,012][105692] Updated weights for policy 0, policy_version 61988 (0.0008) [2023-12-26 15:49:03,063][105692] Updated weights for policy 0, policy_version 61998 (0.0008) [2023-12-26 15:49:03,282][105620] Updated weights for policy 1, policy_version 62422 (0.0010) [2023-12-26 15:49:03,340][105620] Updated weights for policy 1, policy_version 62432 (0.0010) [2023-12-26 15:49:03,391][105620] Updated weights for policy 1, policy_version 62442 (0.0010) [2023-12-26 15:49:03,821][105692] Updated weights for policy 0, policy_version 62008 (0.0008) [2023-12-26 15:49:03,889][105692] Updated weights for policy 0, policy_version 62018 (0.0008) [2023-12-26 15:49:03,948][105692] Updated weights for policy 0, policy_version 62028 (0.0008) [2023-12-26 15:49:04,138][105620] Updated weights for policy 1, policy_version 62452 (0.0010) [2023-12-26 15:49:04,194][105620] Updated weights for policy 1, policy_version 62462 (0.0010) [2023-12-26 15:49:04,258][105620] Updated weights for policy 1, policy_version 62472 (0.0009) [2023-12-26 15:49:04,721][105692] Updated weights for policy 0, policy_version 62038 (0.0009) [2023-12-26 15:49:04,774][105692] Updated weights for policy 0, policy_version 62048 (0.0010) [2023-12-26 15:49:04,824][105692] Updated weights for policy 0, policy_version 62058 (0.0009) [2023-12-26 15:49:04,897][105620] Updated weights for policy 1, policy_version 62482 (0.0006) [2023-12-26 15:49:04,954][105620] Updated weights for policy 1, policy_version 62492 (0.0006) [2023-12-26 15:49:05,002][105620] Updated weights for policy 1, policy_version 62502 (0.0006) [2023-12-26 15:49:05,058][105620] Updated weights for policy 1, policy_version 62512 (0.0006) [2023-12-26 15:49:05,476][105692] Updated weights for policy 0, policy_version 62068 (0.0008) [2023-12-26 15:49:05,529][105692] Updated weights for policy 0, policy_version 62078 (0.0010) [2023-12-26 15:49:05,578][105692] Updated weights for policy 0, policy_version 62089 (0.0009) [2023-12-26 15:49:05,597][105620] Updated weights for policy 1, policy_version 62522 (0.0007) [2023-12-26 15:49:05,646][105620] Updated weights for policy 1, policy_version 62532 (0.0008) [2023-12-26 15:49:05,694][105620] Updated weights for policy 1, policy_version 62542 (0.0006) [2023-12-26 15:49:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 31916032. Throughput: 0: 9869.2, 1: 9879.3. Samples: 31903960. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 15:49:06,062][104569] Avg episode reward: [(0, '9264.642'), (1, '8745.428')] [2023-12-26 15:49:06,311][105620] Updated weights for policy 1, policy_version 62552 (0.0007) [2023-12-26 15:49:06,368][105620] Updated weights for policy 1, policy_version 62562 (0.0006) [2023-12-26 15:49:06,424][105620] Updated weights for policy 1, policy_version 62572 (0.0006) [2023-12-26 15:49:06,426][105692] Updated weights for policy 0, policy_version 62099 (0.0009) [2023-12-26 15:49:06,493][105692] Updated weights for policy 0, policy_version 62109 (0.0011) [2023-12-26 15:49:06,555][105692] Updated weights for policy 0, policy_version 62119 (0.0010) [2023-12-26 15:49:07,126][105620] Updated weights for policy 1, policy_version 62582 (0.0007) [2023-12-26 15:49:07,185][105620] Updated weights for policy 1, policy_version 62592 (0.0008) [2023-12-26 15:49:07,246][105620] Updated weights for policy 1, policy_version 62602 (0.0008) [2023-12-26 15:49:07,300][105692] Updated weights for policy 0, policy_version 62129 (0.0010) [2023-12-26 15:49:07,365][105692] Updated weights for policy 0, policy_version 62139 (0.0006) [2023-12-26 15:49:07,431][105692] Updated weights for policy 0, policy_version 62149 (0.0006) [2023-12-26 15:49:07,494][105692] Updated weights for policy 0, policy_version 62159 (0.0008) [2023-12-26 15:49:08,012][105620] Updated weights for policy 1, policy_version 62612 (0.0007) [2023-12-26 15:49:08,069][105620] Updated weights for policy 1, policy_version 62622 (0.0007) [2023-12-26 15:49:08,119][105620] Updated weights for policy 1, policy_version 62632 (0.0008) [2023-12-26 15:49:08,140][105692] Updated weights for policy 0, policy_version 62169 (0.0009) [2023-12-26 15:49:08,191][105692] Updated weights for policy 0, policy_version 62179 (0.0008) [2023-12-26 15:49:08,240][105692] Updated weights for policy 0, policy_version 62189 (0.0009) [2023-12-26 15:49:08,808][105620] Updated weights for policy 1, policy_version 62642 (0.0006) [2023-12-26 15:49:08,871][105620] Updated weights for policy 1, policy_version 62652 (0.0009) [2023-12-26 15:49:08,935][105620] Updated weights for policy 1, policy_version 62662 (0.0008) [2023-12-26 15:49:08,986][105620] Updated weights for policy 1, policy_version 62672 (0.0007) [2023-12-26 15:49:08,988][105692] Updated weights for policy 0, policy_version 62199 (0.0006) [2023-12-26 15:49:09,053][105692] Updated weights for policy 0, policy_version 62209 (0.0008) [2023-12-26 15:49:09,108][105692] Updated weights for policy 0, policy_version 62219 (0.0005) [2023-12-26 15:49:09,716][105692] Updated weights for policy 0, policy_version 62229 (0.0006) [2023-12-26 15:49:09,781][105692] Updated weights for policy 0, policy_version 62239 (0.0011) [2023-12-26 15:49:09,811][105620] Updated weights for policy 1, policy_version 62682 (0.0008) [2023-12-26 15:49:09,849][105692] Updated weights for policy 0, policy_version 62249 (0.0009) [2023-12-26 15:49:09,877][105620] Updated weights for policy 1, policy_version 62692 (0.0007) [2023-12-26 15:49:09,954][105620] Updated weights for policy 1, policy_version 62702 (0.0010) [2023-12-26 15:49:10,566][105692] Updated weights for policy 0, policy_version 62259 (0.0009) [2023-12-26 15:49:10,620][105692] Updated weights for policy 0, policy_version 62270 (0.0010) [2023-12-26 15:49:10,670][105692] Updated weights for policy 0, policy_version 62280 (0.0010) [2023-12-26 15:49:10,674][105620] Updated weights for policy 1, policy_version 62712 (0.0006) [2023-12-26 15:49:10,728][105620] Updated weights for policy 1, policy_version 62722 (0.0008) [2023-12-26 15:49:10,782][105620] Updated weights for policy 1, policy_version 62732 (0.0009) [2023-12-26 15:49:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 32014336. Throughput: 0: 9853.4, 1: 9982.1. Samples: 32022088. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 15:49:11,062][104569] Avg episode reward: [(0, '9177.912'), (1, '8569.048')] [2023-12-26 15:49:11,464][105692] Updated weights for policy 0, policy_version 62290 (0.0009) [2023-12-26 15:49:11,525][105692] Updated weights for policy 0, policy_version 62300 (0.0008) [2023-12-26 15:49:11,527][105620] Updated weights for policy 1, policy_version 62742 (0.0007) [2023-12-26 15:49:11,587][105692] Updated weights for policy 0, policy_version 62310 (0.0008) [2023-12-26 15:49:11,589][105620] Updated weights for policy 1, policy_version 62752 (0.0006) [2023-12-26 15:49:11,654][105692] Updated weights for policy 0, policy_version 62320 (0.0008) [2023-12-26 15:49:11,655][105620] Updated weights for policy 1, policy_version 62762 (0.0007) [2023-12-26 15:49:12,274][105620] Updated weights for policy 1, policy_version 62772 (0.0008) [2023-12-26 15:49:12,340][105620] Updated weights for policy 1, policy_version 62782 (0.0009) [2023-12-26 15:49:12,400][105692] Updated weights for policy 0, policy_version 62330 (0.0010) [2023-12-26 15:49:12,404][105620] Updated weights for policy 1, policy_version 62792 (0.0008) [2023-12-26 15:49:12,455][105692] Updated weights for policy 0, policy_version 62340 (0.0006) [2023-12-26 15:49:12,506][105692] Updated weights for policy 0, policy_version 62350 (0.0009) [2023-12-26 15:49:13,071][105620] Updated weights for policy 1, policy_version 62802 (0.0008) [2023-12-26 15:49:13,115][105620] Updated weights for policy 1, policy_version 62812 (0.0010) [2023-12-26 15:49:13,177][105620] Updated weights for policy 1, policy_version 62822 (0.0011) [2023-12-26 15:49:13,214][105692] Updated weights for policy 0, policy_version 62360 (0.0008) [2023-12-26 15:49:13,225][105620] Updated weights for policy 1, policy_version 62832 (0.0010) [2023-12-26 15:49:13,263][105692] Updated weights for policy 0, policy_version 62370 (0.0007) [2023-12-26 15:49:13,314][105692] Updated weights for policy 0, policy_version 62380 (0.0008) [2023-12-26 15:49:13,974][105620] Updated weights for policy 1, policy_version 62842 (0.0010) [2023-12-26 15:49:14,025][105620] Updated weights for policy 1, policy_version 62852 (0.0010) [2023-12-26 15:49:14,063][105692] Updated weights for policy 0, policy_version 62390 (0.0007) [2023-12-26 15:49:14,084][105620] Updated weights for policy 1, policy_version 62862 (0.0008) [2023-12-26 15:49:14,114][105692] Updated weights for policy 0, policy_version 62400 (0.0009) [2023-12-26 15:49:14,176][105692] Updated weights for policy 0, policy_version 62410 (0.0009) [2023-12-26 15:49:14,813][105620] Updated weights for policy 1, policy_version 62872 (0.0008) [2023-12-26 15:49:14,850][105692] Updated weights for policy 0, policy_version 62420 (0.0007) [2023-12-26 15:49:14,861][105620] Updated weights for policy 1, policy_version 62882 (0.0008) [2023-12-26 15:49:14,907][105692] Updated weights for policy 0, policy_version 62430 (0.0007) [2023-12-26 15:49:14,917][105620] Updated weights for policy 1, policy_version 62892 (0.0007) [2023-12-26 15:49:14,964][105692] Updated weights for policy 0, policy_version 62440 (0.0007) [2023-12-26 15:49:15,659][105620] Updated weights for policy 1, policy_version 62902 (0.0010) [2023-12-26 15:49:15,690][105692] Updated weights for policy 0, policy_version 62450 (0.0008) [2023-12-26 15:49:15,717][105620] Updated weights for policy 1, policy_version 62912 (0.0008) [2023-12-26 15:49:15,740][105692] Updated weights for policy 0, policy_version 62460 (0.0006) [2023-12-26 15:49:15,782][105620] Updated weights for policy 1, policy_version 62922 (0.0008) [2023-12-26 15:49:15,799][105692] Updated weights for policy 0, policy_version 62470 (0.0008) [2023-12-26 15:49:15,851][105692] Updated weights for policy 0, policy_version 62480 (0.0006) [2023-12-26 15:49:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 32112640. Throughput: 0: 9792.4, 1: 9988.9. Samples: 32079836. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 15:49:16,062][104569] Avg episode reward: [(0, '9176.697'), (1, '4634.106')] [2023-12-26 15:49:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000062480_15998976.pth... [2023-12-26 15:49:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000062928_16113664.pth... [2023-12-26 15:49:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000061328_15704064.pth [2023-12-26 15:49:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000061744_15810560.pth [2023-12-26 15:49:16,417][105620] Updated weights for policy 1, policy_version 62932 (0.0005) [2023-12-26 15:49:16,472][105620] Updated weights for policy 1, policy_version 62942 (0.0005) [2023-12-26 15:49:16,519][105620] Updated weights for policy 1, policy_version 62952 (0.0005) [2023-12-26 15:49:16,607][105692] Updated weights for policy 0, policy_version 62490 (0.0005) [2023-12-26 15:49:16,671][105692] Updated weights for policy 0, policy_version 62500 (0.0005) [2023-12-26 15:49:16,727][105692] Updated weights for policy 0, policy_version 62510 (0.0005) [2023-12-26 15:49:17,195][105620] Updated weights for policy 1, policy_version 62962 (0.0006) [2023-12-26 15:49:17,257][105620] Updated weights for policy 1, policy_version 62972 (0.0006) [2023-12-26 15:49:17,313][105620] Updated weights for policy 1, policy_version 62982 (0.0005) [2023-12-26 15:49:17,330][105692] Updated weights for policy 0, policy_version 62520 (0.0008) [2023-12-26 15:49:17,376][105620] Updated weights for policy 1, policy_version 62992 (0.0005) [2023-12-26 15:49:17,388][105692] Updated weights for policy 0, policy_version 62530 (0.0009) [2023-12-26 15:49:17,450][105692] Updated weights for policy 0, policy_version 62540 (0.0010) [2023-12-26 15:49:17,961][105620] Updated weights for policy 1, policy_version 63002 (0.0009) [2023-12-26 15:49:18,023][105620] Updated weights for policy 1, policy_version 63012 (0.0009) [2023-12-26 15:49:18,079][105620] Updated weights for policy 1, policy_version 63022 (0.0008) [2023-12-26 15:49:18,243][105692] Updated weights for policy 0, policy_version 62550 (0.0007) [2023-12-26 15:49:18,304][105692] Updated weights for policy 0, policy_version 62560 (0.0005) [2023-12-26 15:49:18,365][105692] Updated weights for policy 0, policy_version 62570 (0.0007) [2023-12-26 15:49:18,916][105620] Updated weights for policy 1, policy_version 63032 (0.0009) [2023-12-26 15:49:18,979][105620] Updated weights for policy 1, policy_version 63042 (0.0007) [2023-12-26 15:49:18,985][105692] Updated weights for policy 0, policy_version 62580 (0.0007) [2023-12-26 15:49:19,031][105692] Updated weights for policy 0, policy_version 62590 (0.0007) [2023-12-26 15:49:19,035][105620] Updated weights for policy 1, policy_version 63052 (0.0007) [2023-12-26 15:49:19,096][105692] Updated weights for policy 0, policy_version 62600 (0.0007) [2023-12-26 15:49:19,724][105620] Updated weights for policy 1, policy_version 63062 (0.0006) [2023-12-26 15:49:19,796][105620] Updated weights for policy 1, policy_version 63072 (0.0006) [2023-12-26 15:49:19,865][105620] Updated weights for policy 1, policy_version 63082 (0.0009) [2023-12-26 15:49:19,875][105692] Updated weights for policy 0, policy_version 62610 (0.0008) [2023-12-26 15:49:19,931][105692] Updated weights for policy 0, policy_version 62620 (0.0007) [2023-12-26 15:49:19,990][105692] Updated weights for policy 0, policy_version 62630 (0.0008) [2023-12-26 15:49:20,054][105692] Updated weights for policy 0, policy_version 62640 (0.0008) [2023-12-26 15:49:20,506][105620] Updated weights for policy 1, policy_version 63092 (0.0008) [2023-12-26 15:49:20,569][105620] Updated weights for policy 1, policy_version 63102 (0.0009) [2023-12-26 15:49:20,628][105620] Updated weights for policy 1, policy_version 63112 (0.0009) [2023-12-26 15:49:20,842][105692] Updated weights for policy 0, policy_version 62650 (0.0008) [2023-12-26 15:49:20,908][105692] Updated weights for policy 0, policy_version 62660 (0.0008) [2023-12-26 15:49:20,964][105692] Updated weights for policy 0, policy_version 62670 (0.0007) [2023-12-26 15:49:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 32210944. Throughput: 0: 9731.1, 1: 10040.8. Samples: 32198364. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 15:49:21,063][104569] Avg episode reward: [(0, '9078.274'), (1, '5317.379')] [2023-12-26 15:49:21,356][105620] Updated weights for policy 1, policy_version 63122 (0.0006) [2023-12-26 15:49:21,415][105620] Updated weights for policy 1, policy_version 63132 (0.0008) [2023-12-26 15:49:21,469][105620] Updated weights for policy 1, policy_version 63142 (0.0008) [2023-12-26 15:49:21,538][105620] Updated weights for policy 1, policy_version 63152 (0.0008) [2023-12-26 15:49:21,700][105692] Updated weights for policy 0, policy_version 62680 (0.0010) [2023-12-26 15:49:21,777][105692] Updated weights for policy 0, policy_version 62690 (0.0010) [2023-12-26 15:49:21,826][105692] Updated weights for policy 0, policy_version 62700 (0.0011) [2023-12-26 15:49:22,182][105620] Updated weights for policy 1, policy_version 63162 (0.0010) [2023-12-26 15:49:22,240][105620] Updated weights for policy 1, policy_version 63172 (0.0007) [2023-12-26 15:49:22,303][105620] Updated weights for policy 1, policy_version 63182 (0.0008) [2023-12-26 15:49:22,601][105692] Updated weights for policy 0, policy_version 62710 (0.0011) [2023-12-26 15:49:22,650][105692] Updated weights for policy 0, policy_version 62720 (0.0011) [2023-12-26 15:49:22,699][105692] Updated weights for policy 0, policy_version 62730 (0.0010) [2023-12-26 15:49:23,007][105620] Updated weights for policy 1, policy_version 63192 (0.0009) [2023-12-26 15:49:23,056][105620] Updated weights for policy 1, policy_version 63202 (0.0010) [2023-12-26 15:49:23,115][105620] Updated weights for policy 1, policy_version 63212 (0.0010) [2023-12-26 15:49:23,473][105692] Updated weights for policy 0, policy_version 62740 (0.0010) [2023-12-26 15:49:23,521][105692] Updated weights for policy 0, policy_version 62750 (0.0010) [2023-12-26 15:49:23,565][105692] Updated weights for policy 0, policy_version 62760 (0.0010) [2023-12-26 15:49:23,805][105620] Updated weights for policy 1, policy_version 63222 (0.0010) [2023-12-26 15:49:23,850][105620] Updated weights for policy 1, policy_version 63232 (0.0008) [2023-12-26 15:49:23,896][105620] Updated weights for policy 1, policy_version 63242 (0.0005) [2023-12-26 15:49:24,173][105692] Updated weights for policy 0, policy_version 62770 (0.0010) [2023-12-26 15:49:24,235][105692] Updated weights for policy 0, policy_version 62780 (0.0010) [2023-12-26 15:49:24,294][105692] Updated weights for policy 0, policy_version 62790 (0.0010) [2023-12-26 15:49:24,346][105692] Updated weights for policy 0, policy_version 62800 (0.0010) [2023-12-26 15:49:24,629][105620] Updated weights for policy 1, policy_version 63252 (0.0007) [2023-12-26 15:49:24,682][105620] Updated weights for policy 1, policy_version 63262 (0.0009) [2023-12-26 15:49:24,730][105620] Updated weights for policy 1, policy_version 63272 (0.0008) [2023-12-26 15:49:25,075][105692] Updated weights for policy 0, policy_version 62810 (0.0006) [2023-12-26 15:49:25,127][105692] Updated weights for policy 0, policy_version 62820 (0.0007) [2023-12-26 15:49:25,184][105692] Updated weights for policy 0, policy_version 62830 (0.0010) [2023-12-26 15:49:25,527][105620] Updated weights for policy 1, policy_version 63282 (0.0008) [2023-12-26 15:49:25,574][105620] Updated weights for policy 1, policy_version 63292 (0.0007) [2023-12-26 15:49:25,629][105620] Updated weights for policy 1, policy_version 63302 (0.0008) [2023-12-26 15:49:25,673][105620] Updated weights for policy 1, policy_version 63312 (0.0007) [2023-12-26 15:49:25,914][105692] Updated weights for policy 0, policy_version 62840 (0.0010) [2023-12-26 15:49:25,972][105692] Updated weights for policy 0, policy_version 62850 (0.0010) [2023-12-26 15:49:26,039][105692] Updated weights for policy 0, policy_version 62860 (0.0010) [2023-12-26 15:49:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 32309248. Throughput: 0: 9722.3, 1: 9956.5. Samples: 32313956. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 15:49:26,063][104569] Avg episode reward: [(0, '9078.504'), (1, '6662.858')] [2023-12-26 15:49:26,427][105620] Updated weights for policy 1, policy_version 63322 (0.0006) [2023-12-26 15:49:26,475][105620] Updated weights for policy 1, policy_version 63332 (0.0005) [2023-12-26 15:49:26,529][105620] Updated weights for policy 1, policy_version 63342 (0.0005) [2023-12-26 15:49:26,769][105692] Updated weights for policy 0, policy_version 62870 (0.0010) [2023-12-26 15:49:26,833][105692] Updated weights for policy 0, policy_version 62880 (0.0010) [2023-12-26 15:49:26,891][105692] Updated weights for policy 0, policy_version 62890 (0.0010) [2023-12-26 15:49:27,053][105620] Updated weights for policy 1, policy_version 63352 (0.0005) [2023-12-26 15:49:27,104][105620] Updated weights for policy 1, policy_version 63362 (0.0005) [2023-12-26 15:49:27,158][105620] Updated weights for policy 1, policy_version 63372 (0.0006) [2023-12-26 15:49:27,558][105692] Updated weights for policy 0, policy_version 62900 (0.0008) [2023-12-26 15:49:27,625][105692] Updated weights for policy 0, policy_version 62910 (0.0005) [2023-12-26 15:49:27,681][105692] Updated weights for policy 0, policy_version 62920 (0.0010) [2023-12-26 15:49:27,833][105620] Updated weights for policy 1, policy_version 63382 (0.0006) [2023-12-26 15:49:27,895][105620] Updated weights for policy 1, policy_version 63392 (0.0008) [2023-12-26 15:49:27,955][105620] Updated weights for policy 1, policy_version 63402 (0.0006) [2023-12-26 15:49:28,315][105692] Updated weights for policy 0, policy_version 62930 (0.0009) [2023-12-26 15:49:28,380][105692] Updated weights for policy 0, policy_version 62940 (0.0008) [2023-12-26 15:49:28,442][105692] Updated weights for policy 0, policy_version 62950 (0.0007) [2023-12-26 15:49:28,501][105692] Updated weights for policy 0, policy_version 62960 (0.0011) [2023-12-26 15:49:28,616][105620] Updated weights for policy 1, policy_version 63412 (0.0007) [2023-12-26 15:49:28,676][105620] Updated weights for policy 1, policy_version 63422 (0.0011) [2023-12-26 15:49:28,735][105620] Updated weights for policy 1, policy_version 63432 (0.0007) [2023-12-26 15:49:29,116][105692] Updated weights for policy 0, policy_version 62970 (0.0010) [2023-12-26 15:49:29,174][105692] Updated weights for policy 0, policy_version 62980 (0.0010) [2023-12-26 15:49:29,234][105692] Updated weights for policy 0, policy_version 62990 (0.0010) [2023-12-26 15:49:29,416][105620] Updated weights for policy 1, policy_version 63442 (0.0006) [2023-12-26 15:49:29,481][105620] Updated weights for policy 1, policy_version 63452 (0.0008) [2023-12-26 15:49:29,539][105620] Updated weights for policy 1, policy_version 63462 (0.0006) [2023-12-26 15:49:29,595][105620] Updated weights for policy 1, policy_version 63472 (0.0005) [2023-12-26 15:49:29,954][105692] Updated weights for policy 0, policy_version 63000 (0.0008) [2023-12-26 15:49:30,004][105692] Updated weights for policy 0, policy_version 63010 (0.0008) [2023-12-26 15:49:30,068][105692] Updated weights for policy 0, policy_version 63020 (0.0005) [2023-12-26 15:49:30,286][105620] Updated weights for policy 1, policy_version 63482 (0.0010) [2023-12-26 15:49:30,337][105620] Updated weights for policy 1, policy_version 63492 (0.0010) [2023-12-26 15:49:30,382][105620] Updated weights for policy 1, policy_version 63502 (0.0007) [2023-12-26 15:49:30,615][105692] Updated weights for policy 0, policy_version 63030 (0.0007) [2023-12-26 15:49:30,675][105692] Updated weights for policy 0, policy_version 63040 (0.0006) [2023-12-26 15:49:30,730][105692] Updated weights for policy 0, policy_version 63050 (0.0008) [2023-12-26 15:49:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 32407552. Throughput: 0: 9794.7, 1: 10016.4. Samples: 32376596. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 15:49:31,063][104569] Avg episode reward: [(0, '9272.273'), (1, '9009.342')] [2023-12-26 15:49:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000063056_16146432.pth... [2023-12-26 15:49:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000061904_15851520.pth [2023-12-26 15:49:31,098][105620] Updated weights for policy 1, policy_version 63512 (0.0010) [2023-12-26 15:49:31,158][105620] Updated weights for policy 1, policy_version 63522 (0.0009) [2023-12-26 15:49:31,215][105620] Updated weights for policy 1, policy_version 63532 (0.0008) [2023-12-26 15:49:31,236][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000063536_16269312.pth... [2023-12-26 15:49:31,241][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000062352_15966208.pth [2023-12-26 15:49:31,374][105692] Updated weights for policy 0, policy_version 63060 (0.0008) [2023-12-26 15:49:31,433][105692] Updated weights for policy 0, policy_version 63070 (0.0009) [2023-12-26 15:49:31,500][105692] Updated weights for policy 0, policy_version 63080 (0.0011) [2023-12-26 15:49:31,984][105620] Updated weights for policy 1, policy_version 63542 (0.0011) [2023-12-26 15:49:32,049][105620] Updated weights for policy 1, policy_version 63552 (0.0011) [2023-12-26 15:49:32,108][105620] Updated weights for policy 1, policy_version 63562 (0.0010) [2023-12-26 15:49:32,246][105692] Updated weights for policy 0, policy_version 63090 (0.0010) [2023-12-26 15:49:32,301][105692] Updated weights for policy 0, policy_version 63100 (0.0008) [2023-12-26 15:49:32,345][105692] Updated weights for policy 0, policy_version 63110 (0.0008) [2023-12-26 15:49:32,403][105692] Updated weights for policy 0, policy_version 63120 (0.0008) [2023-12-26 15:49:32,792][105620] Updated weights for policy 1, policy_version 63572 (0.0008) [2023-12-26 15:49:32,845][105620] Updated weights for policy 1, policy_version 63582 (0.0008) [2023-12-26 15:49:32,907][105620] Updated weights for policy 1, policy_version 63592 (0.0010) [2023-12-26 15:49:33,107][105692] Updated weights for policy 0, policy_version 63130 (0.0005) [2023-12-26 15:49:33,155][105692] Updated weights for policy 0, policy_version 63140 (0.0005) [2023-12-26 15:49:33,200][105692] Updated weights for policy 0, policy_version 63150 (0.0005) [2023-12-26 15:49:33,623][105620] Updated weights for policy 1, policy_version 63602 (0.0010) [2023-12-26 15:49:33,676][105620] Updated weights for policy 1, policy_version 63613 (0.0010) [2023-12-26 15:49:33,728][105620] Updated weights for policy 1, policy_version 63623 (0.0009) [2023-12-26 15:49:33,786][105692] Updated weights for policy 0, policy_version 63160 (0.0005) [2023-12-26 15:49:33,841][105692] Updated weights for policy 0, policy_version 63170 (0.0005) [2023-12-26 15:49:33,901][105692] Updated weights for policy 0, policy_version 63180 (0.0005) [2023-12-26 15:49:34,486][105692] Updated weights for policy 0, policy_version 63190 (0.0008) [2023-12-26 15:49:34,545][105692] Updated weights for policy 0, policy_version 63200 (0.0011) [2023-12-26 15:49:34,587][105620] Updated weights for policy 1, policy_version 63633 (0.0009) [2023-12-26 15:49:34,609][105692] Updated weights for policy 0, policy_version 63210 (0.0011) [2023-12-26 15:49:34,652][105620] Updated weights for policy 1, policy_version 63643 (0.0006) [2023-12-26 15:49:34,708][105620] Updated weights for policy 1, policy_version 63653 (0.0008) [2023-12-26 15:49:34,757][105620] Updated weights for policy 1, policy_version 63663 (0.0008) [2023-12-26 15:49:35,353][105692] Updated weights for policy 0, policy_version 63220 (0.0010) [2023-12-26 15:49:35,407][105692] Updated weights for policy 0, policy_version 63230 (0.0009) [2023-12-26 15:49:35,454][105692] Updated weights for policy 0, policy_version 63240 (0.0008) [2023-12-26 15:49:35,513][105620] Updated weights for policy 1, policy_version 63673 (0.0008) [2023-12-26 15:49:35,558][105620] Updated weights for policy 1, policy_version 63683 (0.0006) [2023-12-26 15:49:35,605][105620] Updated weights for policy 1, policy_version 63693 (0.0006) [2023-12-26 15:49:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 32505856. Throughput: 0: 9892.6, 1: 9997.8. Samples: 32497528. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 15:49:36,063][104569] Avg episode reward: [(0, '9272.291'), (1, '9356.117')] [2023-12-26 15:49:36,169][105692] Updated weights for policy 0, policy_version 63250 (0.0009) [2023-12-26 15:49:36,221][105692] Updated weights for policy 0, policy_version 63260 (0.0008) [2023-12-26 15:49:36,273][105692] Updated weights for policy 0, policy_version 63270 (0.0008) [2023-12-26 15:49:36,315][105620] Updated weights for policy 1, policy_version 63703 (0.0007) [2023-12-26 15:49:36,341][105692] Updated weights for policy 0, policy_version 63280 (0.0010) [2023-12-26 15:49:36,369][105620] Updated weights for policy 1, policy_version 63713 (0.0005) [2023-12-26 15:49:36,438][105620] Updated weights for policy 1, policy_version 63723 (0.0010) [2023-12-26 15:49:37,137][105692] Updated weights for policy 0, policy_version 63290 (0.0009) [2023-12-26 15:49:37,148][105620] Updated weights for policy 1, policy_version 63733 (0.0008) [2023-12-26 15:49:37,188][105692] Updated weights for policy 0, policy_version 63300 (0.0007) [2023-12-26 15:49:37,197][105620] Updated weights for policy 1, policy_version 63743 (0.0008) [2023-12-26 15:49:37,252][105692] Updated weights for policy 0, policy_version 63310 (0.0008) [2023-12-26 15:49:37,258][105620] Updated weights for policy 1, policy_version 63753 (0.0006) [2023-12-26 15:49:37,978][105620] Updated weights for policy 1, policy_version 63763 (0.0007) [2023-12-26 15:49:38,032][105692] Updated weights for policy 0, policy_version 63320 (0.0006) [2023-12-26 15:49:38,038][105620] Updated weights for policy 1, policy_version 63773 (0.0008) [2023-12-26 15:49:38,089][105692] Updated weights for policy 0, policy_version 63330 (0.0005) [2023-12-26 15:49:38,102][105620] Updated weights for policy 1, policy_version 63783 (0.0008) [2023-12-26 15:49:38,151][105692] Updated weights for policy 0, policy_version 63340 (0.0005) [2023-12-26 15:49:38,811][105620] Updated weights for policy 1, policy_version 63793 (0.0008) [2023-12-26 15:49:38,876][105620] Updated weights for policy 1, policy_version 63803 (0.0008) [2023-12-26 15:49:38,885][105692] Updated weights for policy 0, policy_version 63350 (0.0007) [2023-12-26 15:49:38,931][105620] Updated weights for policy 1, policy_version 63813 (0.0006) [2023-12-26 15:49:38,943][105692] Updated weights for policy 0, policy_version 63360 (0.0008) [2023-12-26 15:49:38,983][105620] Updated weights for policy 1, policy_version 63823 (0.0008) [2023-12-26 15:49:39,006][105692] Updated weights for policy 0, policy_version 63370 (0.0008) [2023-12-26 15:49:39,669][105620] Updated weights for policy 1, policy_version 63833 (0.0005) [2023-12-26 15:49:39,729][105620] Updated weights for policy 1, policy_version 63843 (0.0009) [2023-12-26 15:49:39,772][105692] Updated weights for policy 0, policy_version 63380 (0.0008) [2023-12-26 15:49:39,791][105620] Updated weights for policy 1, policy_version 63853 (0.0007) [2023-12-26 15:49:39,836][105692] Updated weights for policy 0, policy_version 63390 (0.0008) [2023-12-26 15:49:39,895][105692] Updated weights for policy 0, policy_version 63400 (0.0009) [2023-12-26 15:49:40,472][105620] Updated weights for policy 1, policy_version 63863 (0.0009) [2023-12-26 15:49:40,525][105620] Updated weights for policy 1, policy_version 63873 (0.0009) [2023-12-26 15:49:40,577][105620] Updated weights for policy 1, policy_version 63883 (0.0009) [2023-12-26 15:49:40,627][105692] Updated weights for policy 0, policy_version 63410 (0.0008) [2023-12-26 15:49:40,681][105692] Updated weights for policy 0, policy_version 63420 (0.0005) [2023-12-26 15:49:40,732][105692] Updated weights for policy 0, policy_version 63430 (0.0005) [2023-12-26 15:49:40,778][105692] Updated weights for policy 0, policy_version 63440 (0.0009) [2023-12-26 15:49:41,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 32604160. Throughput: 0: 9781.1, 1: 9967.1. Samples: 32611792. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 15:49:41,062][104569] Avg episode reward: [(0, '9262.971'), (1, '9356.564')] [2023-12-26 15:49:41,423][105620] Updated weights for policy 1, policy_version 63894 (0.0008) [2023-12-26 15:49:41,484][105620] Updated weights for policy 1, policy_version 63904 (0.0008) [2023-12-26 15:49:41,514][105692] Updated weights for policy 0, policy_version 63450 (0.0007) [2023-12-26 15:49:41,549][105620] Updated weights for policy 1, policy_version 63914 (0.0008) [2023-12-26 15:49:41,574][105692] Updated weights for policy 0, policy_version 63460 (0.0005) [2023-12-26 15:49:41,631][105692] Updated weights for policy 0, policy_version 63470 (0.0008) [2023-12-26 15:49:42,273][105620] Updated weights for policy 1, policy_version 63924 (0.0008) [2023-12-26 15:49:42,331][105620] Updated weights for policy 1, policy_version 63934 (0.0008) [2023-12-26 15:49:42,398][105620] Updated weights for policy 1, policy_version 63944 (0.0008) [2023-12-26 15:49:42,426][105692] Updated weights for policy 0, policy_version 63480 (0.0010) [2023-12-26 15:49:42,482][105692] Updated weights for policy 0, policy_version 63490 (0.0010) [2023-12-26 15:49:42,537][105692] Updated weights for policy 0, policy_version 63500 (0.0010) [2023-12-26 15:49:43,167][105620] Updated weights for policy 1, policy_version 63954 (0.0007) [2023-12-26 15:49:43,223][105620] Updated weights for policy 1, policy_version 63964 (0.0009) [2023-12-26 15:49:43,282][105620] Updated weights for policy 1, policy_version 63974 (0.0010) [2023-12-26 15:49:43,289][105692] Updated weights for policy 0, policy_version 63510 (0.0010) [2023-12-26 15:49:43,344][105692] Updated weights for policy 0, policy_version 63520 (0.0010) [2023-12-26 15:49:43,345][105620] Updated weights for policy 1, policy_version 63984 (0.0006) [2023-12-26 15:49:43,395][105692] Updated weights for policy 0, policy_version 63530 (0.0010) [2023-12-26 15:49:43,904][105620] Updated weights for policy 1, policy_version 63994 (0.0005) [2023-12-26 15:49:43,971][105620] Updated weights for policy 1, policy_version 64004 (0.0006) [2023-12-26 15:49:44,032][105620] Updated weights for policy 1, policy_version 64014 (0.0005) [2023-12-26 15:49:44,142][105692] Updated weights for policy 0, policy_version 63540 (0.0010) [2023-12-26 15:49:44,199][105692] Updated weights for policy 0, policy_version 63550 (0.0010) [2023-12-26 15:49:44,254][105692] Updated weights for policy 0, policy_version 63560 (0.0010) [2023-12-26 15:49:44,675][105620] Updated weights for policy 1, policy_version 64024 (0.0009) [2023-12-26 15:49:44,720][105620] Updated weights for policy 1, policy_version 64034 (0.0010) [2023-12-26 15:49:44,778][105620] Updated weights for policy 1, policy_version 64044 (0.0008) [2023-12-26 15:49:44,908][105692] Updated weights for policy 0, policy_version 63570 (0.0009) [2023-12-26 15:49:44,968][105692] Updated weights for policy 0, policy_version 63580 (0.0006) [2023-12-26 15:49:45,026][105692] Updated weights for policy 0, policy_version 63590 (0.0008) [2023-12-26 15:49:45,096][105692] Updated weights for policy 0, policy_version 63600 (0.0010) [2023-12-26 15:49:45,422][105620] Updated weights for policy 1, policy_version 64054 (0.0006) [2023-12-26 15:49:45,488][105620] Updated weights for policy 1, policy_version 64064 (0.0005) [2023-12-26 15:49:45,554][105620] Updated weights for policy 1, policy_version 64074 (0.0010) [2023-12-26 15:49:45,804][105692] Updated weights for policy 0, policy_version 63610 (0.0010) [2023-12-26 15:49:45,848][105692] Updated weights for policy 0, policy_version 63620 (0.0005) [2023-12-26 15:49:45,896][105692] Updated weights for policy 0, policy_version 63630 (0.0005) [2023-12-26 15:49:46,062][104569] Fps is (10 sec: 19659.7, 60 sec: 19797.2, 300 sec: 19633.0). Total num frames: 32702464. Throughput: 0: 9754.7, 1: 9865.0. Samples: 32669036. Policy #0 lag: (min: 0.0, avg: 25.4, max: 32.0) [2023-12-26 15:49:46,064][104569] Avg episode reward: [(0, '9261.907'), (1, '9268.816')] [2023-12-26 15:49:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000063632_16293888.pth... [2023-12-26 15:49:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000064080_16408576.pth... [2023-12-26 15:49:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000062480_15998976.pth [2023-12-26 15:49:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000062928_16113664.pth [2023-12-26 15:49:46,178][105620] Updated weights for policy 1, policy_version 64084 (0.0007) [2023-12-26 15:49:46,229][105620] Updated weights for policy 1, policy_version 64094 (0.0005) [2023-12-26 15:49:46,291][105620] Updated weights for policy 1, policy_version 64104 (0.0005) [2023-12-26 15:49:46,615][105692] Updated weights for policy 0, policy_version 63640 (0.0009) [2023-12-26 15:49:46,679][105692] Updated weights for policy 0, policy_version 63650 (0.0010) [2023-12-26 15:49:46,740][105692] Updated weights for policy 0, policy_version 63660 (0.0010) [2023-12-26 15:49:46,887][105620] Updated weights for policy 1, policy_version 64114 (0.0007) [2023-12-26 15:49:46,932][105620] Updated weights for policy 1, policy_version 64124 (0.0010) [2023-12-26 15:49:46,980][105620] Updated weights for policy 1, policy_version 64134 (0.0010) [2023-12-26 15:49:47,024][105620] Updated weights for policy 1, policy_version 64144 (0.0010) [2023-12-26 15:49:47,480][105692] Updated weights for policy 0, policy_version 63670 (0.0009) [2023-12-26 15:49:47,548][105692] Updated weights for policy 0, policy_version 63680 (0.0009) [2023-12-26 15:49:47,609][105692] Updated weights for policy 0, policy_version 63690 (0.0008) [2023-12-26 15:49:47,803][105620] Updated weights for policy 1, policy_version 64154 (0.0010) [2023-12-26 15:49:47,861][105620] Updated weights for policy 1, policy_version 64164 (0.0010) [2023-12-26 15:49:47,926][105620] Updated weights for policy 1, policy_version 64174 (0.0010) [2023-12-26 15:49:48,443][105692] Updated weights for policy 0, policy_version 63700 (0.0009) [2023-12-26 15:49:48,499][105692] Updated weights for policy 0, policy_version 63710 (0.0008) [2023-12-26 15:49:48,551][105692] Updated weights for policy 0, policy_version 63720 (0.0009) [2023-12-26 15:49:48,574][105620] Updated weights for policy 1, policy_version 64184 (0.0011) [2023-12-26 15:49:48,634][105620] Updated weights for policy 1, policy_version 64194 (0.0011) [2023-12-26 15:49:48,698][105620] Updated weights for policy 1, policy_version 64204 (0.0011) [2023-12-26 15:49:49,259][105692] Updated weights for policy 0, policy_version 63730 (0.0006) [2023-12-26 15:49:49,325][105692] Updated weights for policy 0, policy_version 63740 (0.0007) [2023-12-26 15:49:49,346][105620] Updated weights for policy 1, policy_version 64214 (0.0009) [2023-12-26 15:49:49,394][105692] Updated weights for policy 0, policy_version 63750 (0.0008) [2023-12-26 15:49:49,412][105620] Updated weights for policy 1, policy_version 64224 (0.0009) [2023-12-26 15:49:49,459][105692] Updated weights for policy 0, policy_version 63760 (0.0006) [2023-12-26 15:49:49,467][105620] Updated weights for policy 1, policy_version 64234 (0.0009) [2023-12-26 15:49:50,160][105620] Updated weights for policy 1, policy_version 64244 (0.0007) [2023-12-26 15:49:50,218][105620] Updated weights for policy 1, policy_version 64254 (0.0006) [2023-12-26 15:49:50,261][105692] Updated weights for policy 0, policy_version 63770 (0.0009) [2023-12-26 15:49:50,268][105620] Updated weights for policy 1, policy_version 64264 (0.0006) [2023-12-26 15:49:50,322][105692] Updated weights for policy 0, policy_version 63780 (0.0008) [2023-12-26 15:49:50,378][105692] Updated weights for policy 0, policy_version 63790 (0.0009) [2023-12-26 15:49:50,950][105620] Updated weights for policy 1, policy_version 64274 (0.0006) [2023-12-26 15:49:51,004][105620] Updated weights for policy 1, policy_version 64284 (0.0010) [2023-12-26 15:49:51,061][105620] Updated weights for policy 1, policy_version 64294 (0.0009) [2023-12-26 15:49:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 32792576. Throughput: 0: 9717.5, 1: 9936.2. Samples: 32788376. Policy #0 lag: (min: 0.0, avg: 25.4, max: 32.0) [2023-12-26 15:49:51,062][104569] Avg episode reward: [(0, '9171.602'), (1, '9268.986')] [2023-12-26 15:49:51,126][105620] Updated weights for policy 1, policy_version 64304 (0.0009) [2023-12-26 15:49:51,161][105692] Updated weights for policy 0, policy_version 63800 (0.0008) [2023-12-26 15:49:51,218][105692] Updated weights for policy 0, policy_version 63810 (0.0010) [2023-12-26 15:49:51,280][105692] Updated weights for policy 0, policy_version 63820 (0.0007) [2023-12-26 15:49:51,964][105620] Updated weights for policy 1, policy_version 64314 (0.0008) [2023-12-26 15:49:51,974][105692] Updated weights for policy 0, policy_version 63830 (0.0007) [2023-12-26 15:49:52,021][105620] Updated weights for policy 1, policy_version 64324 (0.0007) [2023-12-26 15:49:52,027][105692] Updated weights for policy 0, policy_version 63840 (0.0006) [2023-12-26 15:49:52,082][105620] Updated weights for policy 1, policy_version 64334 (0.0007) [2023-12-26 15:49:52,088][105692] Updated weights for policy 0, policy_version 63850 (0.0007) [2023-12-26 15:49:52,839][105620] Updated weights for policy 1, policy_version 64344 (0.0008) [2023-12-26 15:49:52,853][105692] Updated weights for policy 0, policy_version 63860 (0.0007) [2023-12-26 15:49:52,892][105620] Updated weights for policy 1, policy_version 64354 (0.0007) [2023-12-26 15:49:52,910][105692] Updated weights for policy 0, policy_version 63870 (0.0007) [2023-12-26 15:49:52,953][105620] Updated weights for policy 1, policy_version 64364 (0.0006) [2023-12-26 15:49:52,970][105692] Updated weights for policy 0, policy_version 63880 (0.0008) [2023-12-26 15:49:53,571][105620] Updated weights for policy 1, policy_version 64374 (0.0007) [2023-12-26 15:49:53,618][105620] Updated weights for policy 1, policy_version 64384 (0.0009) [2023-12-26 15:49:53,642][105692] Updated weights for policy 0, policy_version 63890 (0.0009) [2023-12-26 15:49:53,669][105620] Updated weights for policy 1, policy_version 64394 (0.0007) [2023-12-26 15:49:53,698][105692] Updated weights for policy 0, policy_version 63900 (0.0009) [2023-12-26 15:49:53,754][105692] Updated weights for policy 0, policy_version 63910 (0.0008) [2023-12-26 15:49:53,801][105692] Updated weights for policy 0, policy_version 63920 (0.0009) [2023-12-26 15:49:54,446][105620] Updated weights for policy 1, policy_version 64404 (0.0007) [2023-12-26 15:49:54,509][105620] Updated weights for policy 1, policy_version 64414 (0.0009) [2023-12-26 15:49:54,563][105620] Updated weights for policy 1, policy_version 64424 (0.0008) [2023-12-26 15:49:54,568][105692] Updated weights for policy 0, policy_version 63930 (0.0008) [2023-12-26 15:49:54,629][105692] Updated weights for policy 0, policy_version 63940 (0.0009) [2023-12-26 15:49:54,686][105692] Updated weights for policy 0, policy_version 63950 (0.0009) [2023-12-26 15:49:55,318][105620] Updated weights for policy 1, policy_version 64434 (0.0009) [2023-12-26 15:49:55,380][105620] Updated weights for policy 1, policy_version 64444 (0.0009) [2023-12-26 15:49:55,435][105620] Updated weights for policy 1, policy_version 64454 (0.0007) [2023-12-26 15:49:55,445][105692] Updated weights for policy 0, policy_version 63960 (0.0008) [2023-12-26 15:49:55,491][105620] Updated weights for policy 1, policy_version 64464 (0.0006) [2023-12-26 15:49:55,505][105692] Updated weights for policy 0, policy_version 63970 (0.0007) [2023-12-26 15:49:55,551][105692] Updated weights for policy 0, policy_version 63980 (0.0008) [2023-12-26 15:49:56,062][104569] Fps is (10 sec: 18842.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 32890880. Throughput: 0: 9666.8, 1: 9884.1. Samples: 32901880. Policy #0 lag: (min: 0.0, avg: 25.4, max: 32.0) [2023-12-26 15:49:56,063][104569] Avg episode reward: [(0, '2029.219'), (1, '7969.189')] [2023-12-26 15:49:56,156][105692] Updated weights for policy 0, policy_version 63990 (0.0007) [2023-12-26 15:49:56,163][105620] Updated weights for policy 1, policy_version 64474 (0.0006) [2023-12-26 15:49:56,206][105692] Updated weights for policy 0, policy_version 64000 (0.0005) [2023-12-26 15:49:56,230][105620] Updated weights for policy 1, policy_version 64484 (0.0005) [2023-12-26 15:49:56,268][105692] Updated weights for policy 0, policy_version 64010 (0.0006) [2023-12-26 15:49:56,292][105620] Updated weights for policy 1, policy_version 64494 (0.0008) [2023-12-26 15:49:56,912][105692] Updated weights for policy 0, policy_version 64020 (0.0005) [2023-12-26 15:49:56,979][105692] Updated weights for policy 0, policy_version 64030 (0.0008) [2023-12-26 15:49:56,983][105620] Updated weights for policy 1, policy_version 64504 (0.0005) [2023-12-26 15:49:57,034][105620] Updated weights for policy 1, policy_version 64514 (0.0006) [2023-12-26 15:49:57,039][105692] Updated weights for policy 0, policy_version 64040 (0.0009) [2023-12-26 15:49:57,087][105620] Updated weights for policy 1, policy_version 64524 (0.0006) [2023-12-26 15:49:57,682][105692] Updated weights for policy 0, policy_version 64050 (0.0008) [2023-12-26 15:49:57,737][105692] Updated weights for policy 0, policy_version 64060 (0.0008) [2023-12-26 15:49:57,796][105692] Updated weights for policy 0, policy_version 64070 (0.0007) [2023-12-26 15:49:57,799][105620] Updated weights for policy 1, policy_version 64534 (0.0007) [2023-12-26 15:49:57,851][105692] Updated weights for policy 0, policy_version 64080 (0.0005) [2023-12-26 15:49:57,852][105620] Updated weights for policy 1, policy_version 64544 (0.0005) [2023-12-26 15:49:57,899][105620] Updated weights for policy 1, policy_version 64554 (0.0005) [2023-12-26 15:49:58,533][105692] Updated weights for policy 0, policy_version 64090 (0.0011) [2023-12-26 15:49:58,592][105620] Updated weights for policy 1, policy_version 64564 (0.0006) [2023-12-26 15:49:58,598][105692] Updated weights for policy 0, policy_version 64100 (0.0011) [2023-12-26 15:49:58,654][105620] Updated weights for policy 1, policy_version 64574 (0.0007) [2023-12-26 15:49:58,662][105692] Updated weights for policy 0, policy_version 64110 (0.0011) [2023-12-26 15:49:58,717][105620] Updated weights for policy 1, policy_version 64584 (0.0008) [2023-12-26 15:49:59,410][105620] Updated weights for policy 1, policy_version 64594 (0.0008) [2023-12-26 15:49:59,446][105692] Updated weights for policy 0, policy_version 64120 (0.0010) [2023-12-26 15:49:59,469][105620] Updated weights for policy 1, policy_version 64604 (0.0008) [2023-12-26 15:49:59,498][105692] Updated weights for policy 0, policy_version 64130 (0.0011) [2023-12-26 15:49:59,525][105620] Updated weights for policy 1, policy_version 64614 (0.0006) [2023-12-26 15:49:59,554][105692] Updated weights for policy 0, policy_version 64140 (0.0011) [2023-12-26 15:49:59,585][105620] Updated weights for policy 1, policy_version 64624 (0.0006) [2023-12-26 15:50:00,309][105692] Updated weights for policy 0, policy_version 64150 (0.0010) [2023-12-26 15:50:00,324][105620] Updated weights for policy 1, policy_version 64634 (0.0010) [2023-12-26 15:50:00,353][105692] Updated weights for policy 0, policy_version 64160 (0.0010) [2023-12-26 15:50:00,375][105620] Updated weights for policy 1, policy_version 64644 (0.0010) [2023-12-26 15:50:00,408][105692] Updated weights for policy 0, policy_version 64170 (0.0010) [2023-12-26 15:50:00,426][105620] Updated weights for policy 1, policy_version 64654 (0.0010) [2023-12-26 15:50:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 32989184. Throughput: 0: 9767.8, 1: 9893.0. Samples: 32964576. Policy #0 lag: (min: 0.0, avg: 25.4, max: 32.0) [2023-12-26 15:50:01,062][104569] Avg episode reward: [(0, '4483.916'), (1, '7609.284')] [2023-12-26 15:50:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000064176_16433152.pth... [2023-12-26 15:50:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000064656_16556032.pth... [2023-12-26 15:50:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000063536_16269312.pth [2023-12-26 15:50:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000063056_16146432.pth [2023-12-26 15:50:01,164][105692] Updated weights for policy 0, policy_version 64180 (0.0010) [2023-12-26 15:50:01,193][105620] Updated weights for policy 1, policy_version 64664 (0.0008) [2023-12-26 15:50:01,218][105692] Updated weights for policy 0, policy_version 64190 (0.0010) [2023-12-26 15:50:01,259][105620] Updated weights for policy 1, policy_version 64674 (0.0007) [2023-12-26 15:50:01,274][105692] Updated weights for policy 0, policy_version 64200 (0.0010) [2023-12-26 15:50:01,314][105620] Updated weights for policy 1, policy_version 64684 (0.0008) [2023-12-26 15:50:01,981][105692] Updated weights for policy 0, policy_version 64210 (0.0009) [2023-12-26 15:50:02,028][105692] Updated weights for policy 0, policy_version 64220 (0.0008) [2023-12-26 15:50:02,077][105692] Updated weights for policy 0, policy_version 64230 (0.0009) [2023-12-26 15:50:02,088][105620] Updated weights for policy 1, policy_version 64694 (0.0009) [2023-12-26 15:50:02,127][105692] Updated weights for policy 0, policy_version 64240 (0.0006) [2023-12-26 15:50:02,145][105620] Updated weights for policy 1, policy_version 64704 (0.0007) [2023-12-26 15:50:02,207][105620] Updated weights for policy 1, policy_version 64714 (0.0009) [2023-12-26 15:50:02,874][105692] Updated weights for policy 0, policy_version 64250 (0.0009) [2023-12-26 15:50:02,929][105692] Updated weights for policy 0, policy_version 64260 (0.0009) [2023-12-26 15:50:02,962][105620] Updated weights for policy 1, policy_version 64724 (0.0008) [2023-12-26 15:50:02,991][105692] Updated weights for policy 0, policy_version 64270 (0.0009) [2023-12-26 15:50:03,019][105620] Updated weights for policy 1, policy_version 64734 (0.0006) [2023-12-26 15:50:03,079][105620] Updated weights for policy 1, policy_version 64744 (0.0008) [2023-12-26 15:50:03,749][105692] Updated weights for policy 0, policy_version 64280 (0.0009) [2023-12-26 15:50:03,783][105620] Updated weights for policy 1, policy_version 64754 (0.0009) [2023-12-26 15:50:03,793][105692] Updated weights for policy 0, policy_version 64290 (0.0009) [2023-12-26 15:50:03,842][105692] Updated weights for policy 0, policy_version 64300 (0.0007) [2023-12-26 15:50:03,843][105620] Updated weights for policy 1, policy_version 64764 (0.0008) [2023-12-26 15:50:03,907][105620] Updated weights for policy 1, policy_version 64774 (0.0009) [2023-12-26 15:50:03,969][105620] Updated weights for policy 1, policy_version 64784 (0.0010) [2023-12-26 15:50:04,636][105620] Updated weights for policy 1, policy_version 64794 (0.0005) [2023-12-26 15:50:04,667][105692] Updated weights for policy 0, policy_version 64310 (0.0007) [2023-12-26 15:50:04,687][105620] Updated weights for policy 1, policy_version 64804 (0.0005) [2023-12-26 15:50:04,716][105692] Updated weights for policy 0, policy_version 64320 (0.0009) [2023-12-26 15:50:04,734][105620] Updated weights for policy 1, policy_version 64814 (0.0005) [2023-12-26 15:50:04,767][105692] Updated weights for policy 0, policy_version 64332 (0.0009) [2023-12-26 15:50:05,266][105620] Updated weights for policy 1, policy_version 64824 (0.0008) [2023-12-26 15:50:05,323][105620] Updated weights for policy 1, policy_version 64834 (0.0009) [2023-12-26 15:50:05,377][105620] Updated weights for policy 1, policy_version 64844 (0.0009) [2023-12-26 15:50:05,618][105692] Updated weights for policy 0, policy_version 64342 (0.0009) [2023-12-26 15:50:05,666][105692] Updated weights for policy 0, policy_version 64352 (0.0009) [2023-12-26 15:50:05,716][105692] Updated weights for policy 0, policy_version 64362 (0.0008) [2023-12-26 15:50:06,060][105620] Updated weights for policy 1, policy_version 64854 (0.0008) [2023-12-26 15:50:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 33087488. Throughput: 0: 9698.0, 1: 9861.8. Samples: 33078552. Policy #0 lag: (min: 0.0, avg: 25.4, max: 32.0) [2023-12-26 15:50:06,063][104569] Avg episode reward: [(0, '6836.286'), (1, '8379.215')] [2023-12-26 15:50:06,118][105620] Updated weights for policy 1, policy_version 64864 (0.0007) [2023-12-26 15:50:06,177][105620] Updated weights for policy 1, policy_version 64874 (0.0009) [2023-12-26 15:50:06,561][105692] Updated weights for policy 0, policy_version 64372 (0.0009) [2023-12-26 15:50:06,621][105692] Updated weights for policy 0, policy_version 64382 (0.0010) [2023-12-26 15:50:06,680][105692] Updated weights for policy 0, policy_version 64392 (0.0009) [2023-12-26 15:50:06,928][105620] Updated weights for policy 1, policy_version 64884 (0.0007) [2023-12-26 15:50:06,989][105620] Updated weights for policy 1, policy_version 64894 (0.0006) [2023-12-26 15:50:07,047][105620] Updated weights for policy 1, policy_version 64904 (0.0007) [2023-12-26 15:50:07,461][105692] Updated weights for policy 0, policy_version 64402 (0.0009) [2023-12-26 15:50:07,520][105692] Updated weights for policy 0, policy_version 64412 (0.0008) [2023-12-26 15:50:07,580][105692] Updated weights for policy 0, policy_version 64422 (0.0008) [2023-12-26 15:50:07,646][105692] Updated weights for policy 0, policy_version 64432 (0.0008) [2023-12-26 15:50:07,762][105620] Updated weights for policy 1, policy_version 64914 (0.0010) [2023-12-26 15:50:07,809][105620] Updated weights for policy 1, policy_version 64924 (0.0010) [2023-12-26 15:50:07,866][105620] Updated weights for policy 1, policy_version 64934 (0.0010) [2023-12-26 15:50:07,925][105620] Updated weights for policy 1, policy_version 64944 (0.0010) [2023-12-26 15:50:08,402][105692] Updated weights for policy 0, policy_version 64442 (0.0006) [2023-12-26 15:50:08,462][105692] Updated weights for policy 0, policy_version 64452 (0.0008) [2023-12-26 15:50:08,527][105692] Updated weights for policy 0, policy_version 64462 (0.0009) [2023-12-26 15:50:08,721][105620] Updated weights for policy 1, policy_version 64954 (0.0010) [2023-12-26 15:50:08,777][105620] Updated weights for policy 1, policy_version 64964 (0.0009) [2023-12-26 15:50:08,828][105620] Updated weights for policy 1, policy_version 64974 (0.0008) [2023-12-26 15:50:09,245][105692] Updated weights for policy 0, policy_version 64472 (0.0008) [2023-12-26 15:50:09,309][105692] Updated weights for policy 0, policy_version 64482 (0.0010) [2023-12-26 15:50:09,379][105692] Updated weights for policy 0, policy_version 64492 (0.0009) [2023-12-26 15:50:09,574][105620] Updated weights for policy 1, policy_version 64984 (0.0007) [2023-12-26 15:50:09,623][105620] Updated weights for policy 1, policy_version 64994 (0.0008) [2023-12-26 15:50:09,685][105620] Updated weights for policy 1, policy_version 65004 (0.0008) [2023-12-26 15:50:10,188][105692] Updated weights for policy 0, policy_version 64502 (0.0009) [2023-12-26 15:50:10,244][105692] Updated weights for policy 0, policy_version 64512 (0.0009) [2023-12-26 15:50:10,307][105692] Updated weights for policy 0, policy_version 64522 (0.0010) [2023-12-26 15:50:10,351][105620] Updated weights for policy 1, policy_version 65014 (0.0008) [2023-12-26 15:50:10,403][105620] Updated weights for policy 1, policy_version 65024 (0.0009) [2023-12-26 15:50:10,458][105620] Updated weights for policy 1, policy_version 65034 (0.0006) [2023-12-26 15:50:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 33177600. Throughput: 0: 9608.3, 1: 9872.1. Samples: 33190572. Policy #0 lag: (min: 0.0, avg: 25.4, max: 32.0) [2023-12-26 15:50:11,062][105692] Updated weights for policy 0, policy_version 64532 (0.0008) [2023-12-26 15:50:11,063][104569] Avg episode reward: [(0, '8906.409'), (1, '8115.394')] [2023-12-26 15:50:11,134][105692] Updated weights for policy 0, policy_version 64542 (0.0009) [2023-12-26 15:50:11,165][105620] Updated weights for policy 1, policy_version 65044 (0.0008) [2023-12-26 15:50:11,196][105692] Updated weights for policy 0, policy_version 64552 (0.0008) [2023-12-26 15:50:11,225][105620] Updated weights for policy 1, policy_version 65054 (0.0007) [2023-12-26 15:50:11,290][105620] Updated weights for policy 1, policy_version 65064 (0.0008) [2023-12-26 15:50:12,019][105692] Updated weights for policy 0, policy_version 64562 (0.0007) [2023-12-26 15:50:12,020][105620] Updated weights for policy 1, policy_version 65074 (0.0009) [2023-12-26 15:50:12,078][105620] Updated weights for policy 1, policy_version 65084 (0.0008) [2023-12-26 15:50:12,080][105692] Updated weights for policy 0, policy_version 64572 (0.0006) [2023-12-26 15:50:12,140][105620] Updated weights for policy 1, policy_version 65094 (0.0008) [2023-12-26 15:50:12,141][105692] Updated weights for policy 0, policy_version 64582 (0.0007) [2023-12-26 15:50:12,197][105620] Updated weights for policy 1, policy_version 65104 (0.0008) [2023-12-26 15:50:12,200][105692] Updated weights for policy 0, policy_version 64592 (0.0009) [2023-12-26 15:50:12,944][105692] Updated weights for policy 0, policy_version 64602 (0.0011) [2023-12-26 15:50:12,975][105620] Updated weights for policy 1, policy_version 65114 (0.0006) [2023-12-26 15:50:12,993][105692] Updated weights for policy 0, policy_version 64612 (0.0010) [2023-12-26 15:50:13,027][105620] Updated weights for policy 1, policy_version 65124 (0.0005) [2023-12-26 15:50:13,048][105692] Updated weights for policy 0, policy_version 64622 (0.0010) [2023-12-26 15:50:13,082][105620] Updated weights for policy 1, policy_version 65134 (0.0007) [2023-12-26 15:50:13,776][105692] Updated weights for policy 0, policy_version 64632 (0.0009) [2023-12-26 15:50:13,824][105692] Updated weights for policy 0, policy_version 64642 (0.0009) [2023-12-26 15:50:13,871][105620] Updated weights for policy 1, policy_version 65144 (0.0008) [2023-12-26 15:50:13,877][105692] Updated weights for policy 0, policy_version 64652 (0.0006) [2023-12-26 15:50:13,927][105620] Updated weights for policy 1, policy_version 65154 (0.0007) [2023-12-26 15:50:13,985][105620] Updated weights for policy 1, policy_version 65164 (0.0009) [2023-12-26 15:50:14,595][105692] Updated weights for policy 0, policy_version 64662 (0.0008) [2023-12-26 15:50:14,645][105692] Updated weights for policy 0, policy_version 64672 (0.0009) [2023-12-26 15:50:14,704][105692] Updated weights for policy 0, policy_version 64682 (0.0009) [2023-12-26 15:50:14,736][105620] Updated weights for policy 1, policy_version 65174 (0.0008) [2023-12-26 15:50:14,801][105620] Updated weights for policy 1, policy_version 65184 (0.0007) [2023-12-26 15:50:14,864][105620] Updated weights for policy 1, policy_version 65194 (0.0009) [2023-12-26 15:50:15,406][105692] Updated weights for policy 0, policy_version 64692 (0.0008) [2023-12-26 15:50:15,468][105692] Updated weights for policy 0, policy_version 64702 (0.0009) [2023-12-26 15:50:15,519][105692] Updated weights for policy 0, policy_version 64712 (0.0009) [2023-12-26 15:50:15,642][105620] Updated weights for policy 1, policy_version 65204 (0.0008) [2023-12-26 15:50:15,700][105620] Updated weights for policy 1, policy_version 65214 (0.0007) [2023-12-26 15:50:15,760][105620] Updated weights for policy 1, policy_version 65224 (0.0009) [2023-12-26 15:50:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 33275904. Throughput: 0: 9560.1, 1: 9760.4. Samples: 33246016. Policy #0 lag: (min: 0.0, avg: 25.4, max: 32.0) [2023-12-26 15:50:16,062][104569] Avg episode reward: [(0, '8814.171'), (1, '8650.261')] [2023-12-26 15:50:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000064720_16572416.pth... [2023-12-26 15:50:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000065232_16703488.pth... [2023-12-26 15:50:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000063632_16293888.pth [2023-12-26 15:50:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000064080_16408576.pth [2023-12-26 15:50:16,158][105692] Updated weights for policy 0, policy_version 64722 (0.0008) [2023-12-26 15:50:16,219][105692] Updated weights for policy 0, policy_version 64732 (0.0005) [2023-12-26 15:50:16,271][105692] Updated weights for policy 0, policy_version 64742 (0.0005) [2023-12-26 15:50:16,319][105692] Updated weights for policy 0, policy_version 64752 (0.0008) [2023-12-26 15:50:16,441][105620] Updated weights for policy 1, policy_version 65234 (0.0009) [2023-12-26 15:50:16,492][105620] Updated weights for policy 1, policy_version 65244 (0.0008) [2023-12-26 15:50:16,544][105620] Updated weights for policy 1, policy_version 65254 (0.0009) [2023-12-26 15:50:16,599][105620] Updated weights for policy 1, policy_version 65264 (0.0008) [2023-12-26 15:50:17,025][105692] Updated weights for policy 0, policy_version 64762 (0.0010) [2023-12-26 15:50:17,078][105692] Updated weights for policy 0, policy_version 64772 (0.0008) [2023-12-26 15:50:17,136][105692] Updated weights for policy 0, policy_version 64782 (0.0010) [2023-12-26 15:50:17,385][105620] Updated weights for policy 1, policy_version 65274 (0.0009) [2023-12-26 15:50:17,439][105620] Updated weights for policy 1, policy_version 65284 (0.0009) [2023-12-26 15:50:17,490][105620] Updated weights for policy 1, policy_version 65294 (0.0009) [2023-12-26 15:50:17,730][105692] Updated weights for policy 0, policy_version 64792 (0.0006) [2023-12-26 15:50:17,782][105692] Updated weights for policy 0, policy_version 64802 (0.0005) [2023-12-26 15:50:17,834][105692] Updated weights for policy 0, policy_version 64812 (0.0005) [2023-12-26 15:50:18,286][105620] Updated weights for policy 1, policy_version 65304 (0.0010) [2023-12-26 15:50:18,351][105620] Updated weights for policy 1, policy_version 65314 (0.0009) [2023-12-26 15:50:18,415][105620] Updated weights for policy 1, policy_version 65324 (0.0008) [2023-12-26 15:50:18,496][105692] Updated weights for policy 0, policy_version 64822 (0.0008) [2023-12-26 15:50:18,562][105692] Updated weights for policy 0, policy_version 64832 (0.0010) [2023-12-26 15:50:18,625][105692] Updated weights for policy 0, policy_version 64842 (0.0009) [2023-12-26 15:50:19,124][105620] Updated weights for policy 1, policy_version 65334 (0.0009) [2023-12-26 15:50:19,192][105620] Updated weights for policy 1, policy_version 65344 (0.0011) [2023-12-26 15:50:19,255][105620] Updated weights for policy 1, policy_version 65354 (0.0011) [2023-12-26 15:50:19,449][105692] Updated weights for policy 0, policy_version 64852 (0.0008) [2023-12-26 15:50:19,516][105692] Updated weights for policy 0, policy_version 64862 (0.0009) [2023-12-26 15:50:19,578][105692] Updated weights for policy 0, policy_version 64872 (0.0008) [2023-12-26 15:50:20,051][105620] Updated weights for policy 1, policy_version 65364 (0.0011) [2023-12-26 15:50:20,108][105620] Updated weights for policy 1, policy_version 65374 (0.0011) [2023-12-26 15:50:20,171][105620] Updated weights for policy 1, policy_version 65384 (0.0011) [2023-12-26 15:50:20,391][105692] Updated weights for policy 0, policy_version 64882 (0.0008) [2023-12-26 15:50:20,457][105692] Updated weights for policy 0, policy_version 64892 (0.0008) [2023-12-26 15:50:20,510][105692] Updated weights for policy 0, policy_version 64902 (0.0008) [2023-12-26 15:50:20,570][105692] Updated weights for policy 0, policy_version 64912 (0.0007) [2023-12-26 15:50:20,939][105620] Updated weights for policy 1, policy_version 65394 (0.0011) [2023-12-26 15:50:20,988][105620] Updated weights for policy 1, policy_version 65404 (0.0010) [2023-12-26 15:50:21,054][105620] Updated weights for policy 1, policy_version 65414 (0.0011) [2023-12-26 15:50:21,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19251.1, 300 sec: 19605.3). Total num frames: 33366016. Throughput: 0: 9479.0, 1: 9734.3. Samples: 33362128. Policy #0 lag: (min: 0.0, avg: 25.4, max: 32.0) [2023-12-26 15:50:21,063][104569] Avg episode reward: [(0, '8906.781'), (1, '9007.771')] [2023-12-26 15:50:21,111][105620] Updated weights for policy 1, policy_version 65424 (0.0011) [2023-12-26 15:50:21,254][105692] Updated weights for policy 0, policy_version 64922 (0.0006) [2023-12-26 15:50:21,315][105692] Updated weights for policy 0, policy_version 64932 (0.0006) [2023-12-26 15:50:21,384][105692] Updated weights for policy 0, policy_version 64942 (0.0009) [2023-12-26 15:50:21,915][105620] Updated weights for policy 1, policy_version 65434 (0.0011) [2023-12-26 15:50:21,960][105620] Updated weights for policy 1, policy_version 65444 (0.0011) [2023-12-26 15:50:22,011][105620] Updated weights for policy 1, policy_version 65454 (0.0010) [2023-12-26 15:50:22,119][105692] Updated weights for policy 0, policy_version 64952 (0.0008) [2023-12-26 15:50:22,191][105692] Updated weights for policy 0, policy_version 64962 (0.0009) [2023-12-26 15:50:22,251][105692] Updated weights for policy 0, policy_version 64972 (0.0009) [2023-12-26 15:50:22,805][105620] Updated weights for policy 1, policy_version 65464 (0.0010) [2023-12-26 15:50:22,870][105620] Updated weights for policy 1, policy_version 65474 (0.0010) [2023-12-26 15:50:22,927][105620] Updated weights for policy 1, policy_version 65484 (0.0010) [2023-12-26 15:50:23,013][105692] Updated weights for policy 0, policy_version 64982 (0.0010) [2023-12-26 15:50:23,076][105692] Updated weights for policy 0, policy_version 64992 (0.0011) [2023-12-26 15:50:23,138][105692] Updated weights for policy 0, policy_version 65002 (0.0010) [2023-12-26 15:50:23,666][105620] Updated weights for policy 1, policy_version 65494 (0.0010) [2023-12-26 15:50:23,717][105620] Updated weights for policy 1, policy_version 65504 (0.0010) [2023-12-26 15:50:23,765][105620] Updated weights for policy 1, policy_version 65514 (0.0010) [2023-12-26 15:50:23,794][105692] Updated weights for policy 0, policy_version 65012 (0.0008) [2023-12-26 15:50:23,852][105692] Updated weights for policy 0, policy_version 65022 (0.0005) [2023-12-26 15:50:23,909][105692] Updated weights for policy 0, policy_version 65032 (0.0005) [2023-12-26 15:50:24,508][105620] Updated weights for policy 1, policy_version 65524 (0.0010) [2023-12-26 15:50:24,555][105692] Updated weights for policy 0, policy_version 65042 (0.0008) [2023-12-26 15:50:24,573][105620] Updated weights for policy 1, policy_version 65534 (0.0010) [2023-12-26 15:50:24,614][105692] Updated weights for policy 0, policy_version 65052 (0.0011) [2023-12-26 15:50:24,632][105620] Updated weights for policy 1, policy_version 65544 (0.0010) [2023-12-26 15:50:24,678][105692] Updated weights for policy 0, policy_version 65062 (0.0011) [2023-12-26 15:50:24,737][105692] Updated weights for policy 0, policy_version 65072 (0.0011) [2023-12-26 15:50:25,361][105620] Updated weights for policy 1, policy_version 65554 (0.0010) [2023-12-26 15:50:25,415][105620] Updated weights for policy 1, policy_version 65564 (0.0010) [2023-12-26 15:50:25,470][105692] Updated weights for policy 0, policy_version 65082 (0.0011) [2023-12-26 15:50:25,473][105620] Updated weights for policy 1, policy_version 65574 (0.0010) [2023-12-26 15:50:25,524][105692] Updated weights for policy 0, policy_version 65092 (0.0010) [2023-12-26 15:50:25,533][105620] Updated weights for policy 1, policy_version 65584 (0.0010) [2023-12-26 15:50:25,576][105692] Updated weights for policy 0, policy_version 65102 (0.0010) [2023-12-26 15:50:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 33464320. Throughput: 0: 9504.0, 1: 9670.0. Samples: 33474620. Policy #0 lag: (min: 9.0, avg: 27.2, max: 41.0) [2023-12-26 15:50:26,062][104569] Avg episode reward: [(0, '9081.020'), (1, '9096.188')] [2023-12-26 15:50:26,237][105620] Updated weights for policy 1, policy_version 65594 (0.0010) [2023-12-26 15:50:26,294][105620] Updated weights for policy 1, policy_version 65604 (0.0010) [2023-12-26 15:50:26,342][105692] Updated weights for policy 0, policy_version 65112 (0.0011) [2023-12-26 15:50:26,355][105620] Updated weights for policy 1, policy_version 65614 (0.0010) [2023-12-26 15:50:26,394][105692] Updated weights for policy 0, policy_version 65122 (0.0011) [2023-12-26 15:50:26,442][105692] Updated weights for policy 0, policy_version 65132 (0.0010) [2023-12-26 15:50:27,003][105692] Updated weights for policy 0, policy_version 65142 (0.0007) [2023-12-26 15:50:27,066][105692] Updated weights for policy 0, policy_version 65152 (0.0005) [2023-12-26 15:50:27,091][105620] Updated weights for policy 1, policy_version 65624 (0.0010) [2023-12-26 15:50:27,122][105692] Updated weights for policy 0, policy_version 65162 (0.0005) [2023-12-26 15:50:27,155][105620] Updated weights for policy 1, policy_version 65634 (0.0010) [2023-12-26 15:50:27,232][105620] Updated weights for policy 1, policy_version 65644 (0.0010) [2023-12-26 15:50:27,696][105692] Updated weights for policy 0, policy_version 65172 (0.0005) [2023-12-26 15:50:27,750][105692] Updated weights for policy 0, policy_version 65182 (0.0005) [2023-12-26 15:50:27,807][105692] Updated weights for policy 0, policy_version 65192 (0.0006) [2023-12-26 15:50:27,824][105620] Updated weights for policy 1, policy_version 65654 (0.0010) [2023-12-26 15:50:27,871][105620] Updated weights for policy 1, policy_version 65664 (0.0010) [2023-12-26 15:50:27,918][105620] Updated weights for policy 1, policy_version 65674 (0.0010) [2023-12-26 15:50:28,371][105692] Updated weights for policy 0, policy_version 65202 (0.0009) [2023-12-26 15:50:28,422][105692] Updated weights for policy 0, policy_version 65212 (0.0005) [2023-12-26 15:50:28,481][105692] Updated weights for policy 0, policy_version 65222 (0.0005) [2023-12-26 15:50:28,527][105692] Updated weights for policy 0, policy_version 65232 (0.0005) [2023-12-26 15:50:28,690][105620] Updated weights for policy 1, policy_version 65684 (0.0010) [2023-12-26 15:50:28,741][105620] Updated weights for policy 1, policy_version 65694 (0.0010) [2023-12-26 15:50:28,793][105620] Updated weights for policy 1, policy_version 65704 (0.0010) [2023-12-26 15:50:29,157][105692] Updated weights for policy 0, policy_version 65242 (0.0009) [2023-12-26 15:50:29,211][105692] Updated weights for policy 0, policy_version 65252 (0.0009) [2023-12-26 15:50:29,268][105692] Updated weights for policy 0, policy_version 65262 (0.0006) [2023-12-26 15:50:29,513][105620] Updated weights for policy 1, policy_version 65714 (0.0009) [2023-12-26 15:50:29,568][105620] Updated weights for policy 1, policy_version 65724 (0.0006) [2023-12-26 15:50:29,632][105620] Updated weights for policy 1, policy_version 65734 (0.0008) [2023-12-26 15:50:29,703][105620] Updated weights for policy 1, policy_version 65744 (0.0011) [2023-12-26 15:50:30,023][105692] Updated weights for policy 0, policy_version 65272 (0.0007) [2023-12-26 15:50:30,071][105692] Updated weights for policy 0, policy_version 65282 (0.0008) [2023-12-26 15:50:30,121][105692] Updated weights for policy 0, policy_version 65292 (0.0008) [2023-12-26 15:50:30,468][105620] Updated weights for policy 1, policy_version 65754 (0.0010) [2023-12-26 15:50:30,516][105620] Updated weights for policy 1, policy_version 65764 (0.0010) [2023-12-26 15:50:30,560][105620] Updated weights for policy 1, policy_version 65774 (0.0010) [2023-12-26 15:50:30,844][105692] Updated weights for policy 0, policy_version 65302 (0.0007) [2023-12-26 15:50:30,895][105692] Updated weights for policy 0, policy_version 65312 (0.0008) [2023-12-26 15:50:30,942][105692] Updated weights for policy 0, policy_version 65322 (0.0008) [2023-12-26 15:50:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 33570816. Throughput: 0: 9630.3, 1: 9674.3. Samples: 33537736. Policy #0 lag: (min: 9.0, avg: 27.2, max: 41.0) [2023-12-26 15:50:31,062][104569] Avg episode reward: [(0, '9261.046'), (1, '9092.971')] [2023-12-26 15:50:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000065328_16728064.pth... [2023-12-26 15:50:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000065776_16842752.pth... [2023-12-26 15:50:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000064176_16433152.pth [2023-12-26 15:50:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000064656_16556032.pth [2023-12-26 15:50:31,302][105620] Updated weights for policy 1, policy_version 65784 (0.0011) [2023-12-26 15:50:31,355][105620] Updated weights for policy 1, policy_version 65794 (0.0010) [2023-12-26 15:50:31,424][105620] Updated weights for policy 1, policy_version 65804 (0.0010) [2023-12-26 15:50:31,754][105692] Updated weights for policy 0, policy_version 65332 (0.0007) [2023-12-26 15:50:31,820][105692] Updated weights for policy 0, policy_version 65342 (0.0006) [2023-12-26 15:50:31,875][105692] Updated weights for policy 0, policy_version 65352 (0.0006) [2023-12-26 15:50:32,166][105620] Updated weights for policy 1, policy_version 65814 (0.0009) [2023-12-26 15:50:32,223][105620] Updated weights for policy 1, policy_version 65824 (0.0009) [2023-12-26 15:50:32,299][105620] Updated weights for policy 1, policy_version 65834 (0.0009) [2023-12-26 15:50:32,499][105692] Updated weights for policy 0, policy_version 65362 (0.0005) [2023-12-26 15:50:32,557][105692] Updated weights for policy 0, policy_version 65372 (0.0005) [2023-12-26 15:50:32,622][105692] Updated weights for policy 0, policy_version 65382 (0.0006) [2023-12-26 15:50:32,683][105692] Updated weights for policy 0, policy_version 65392 (0.0007) [2023-12-26 15:50:32,941][105620] Updated weights for policy 1, policy_version 65844 (0.0008) [2023-12-26 15:50:32,993][105620] Updated weights for policy 1, policy_version 65854 (0.0006) [2023-12-26 15:50:33,039][105620] Updated weights for policy 1, policy_version 65864 (0.0005) [2023-12-26 15:50:33,200][105692] Updated weights for policy 0, policy_version 65402 (0.0005) [2023-12-26 15:50:33,250][105692] Updated weights for policy 0, policy_version 65412 (0.0005) [2023-12-26 15:50:33,310][105692] Updated weights for policy 0, policy_version 65422 (0.0005) [2023-12-26 15:50:33,722][105620] Updated weights for policy 1, policy_version 65874 (0.0006) [2023-12-26 15:50:33,773][105620] Updated weights for policy 1, policy_version 65884 (0.0005) [2023-12-26 15:50:33,801][105586] KL-divergence is very high: 126.9442 [2023-12-26 15:50:33,814][105586] KL-divergence is very high: 108.7636 [2023-12-26 15:50:33,818][105620] Updated weights for policy 1, policy_version 65894 (0.0005) [2023-12-26 15:50:33,838][105586] KL-divergence is very high: 121.2547 [2023-12-26 15:50:33,869][105620] Updated weights for policy 1, policy_version 65904 (0.0005) [2023-12-26 15:50:33,992][105692] Updated weights for policy 0, policy_version 65432 (0.0010) [2023-12-26 15:50:34,043][105692] Updated weights for policy 0, policy_version 65442 (0.0010) [2023-12-26 15:50:34,098][105692] Updated weights for policy 0, policy_version 65452 (0.0009) [2023-12-26 15:50:34,621][105620] Updated weights for policy 1, policy_version 65914 (0.0008) [2023-12-26 15:50:34,685][105620] Updated weights for policy 1, policy_version 65924 (0.0008) [2023-12-26 15:50:34,737][105692] Updated weights for policy 0, policy_version 65462 (0.0006) [2023-12-26 15:50:34,744][105620] Updated weights for policy 1, policy_version 65934 (0.0010) [2023-12-26 15:50:34,789][105692] Updated weights for policy 0, policy_version 65472 (0.0006) [2023-12-26 15:50:34,846][105692] Updated weights for policy 0, policy_version 65482 (0.0009) [2023-12-26 15:50:35,402][105692] Updated weights for policy 0, policy_version 65492 (0.0009) [2023-12-26 15:50:35,409][105620] Updated weights for policy 1, policy_version 65944 (0.0006) [2023-12-26 15:50:35,466][105692] Updated weights for policy 0, policy_version 65502 (0.0007) [2023-12-26 15:50:35,472][105620] Updated weights for policy 1, policy_version 65954 (0.0005) [2023-12-26 15:50:35,512][105692] Updated weights for policy 0, policy_version 65512 (0.0005) [2023-12-26 15:50:35,537][105620] Updated weights for policy 1, policy_version 65964 (0.0005) [2023-12-26 15:50:36,044][105620] Updated weights for policy 1, policy_version 65974 (0.0006) [2023-12-26 15:50:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 33669120. Throughput: 0: 9737.0, 1: 9588.9. Samples: 33658040. Policy #0 lag: (min: 9.0, avg: 27.2, max: 41.0) [2023-12-26 15:50:36,062][104569] Avg episode reward: [(0, '8492.902'), (1, '8667.452')] [2023-12-26 15:50:36,104][105620] Updated weights for policy 1, policy_version 65984 (0.0008) [2023-12-26 15:50:36,167][105620] Updated weights for policy 1, policy_version 65994 (0.0007) [2023-12-26 15:50:36,227][105692] Updated weights for policy 0, policy_version 65522 (0.0009) [2023-12-26 15:50:36,285][105692] Updated weights for policy 0, policy_version 65532 (0.0011) [2023-12-26 15:50:36,343][105692] Updated weights for policy 0, policy_version 65542 (0.0010) [2023-12-26 15:50:36,401][105692] Updated weights for policy 0, policy_version 65552 (0.0010) [2023-12-26 15:50:36,940][105620] Updated weights for policy 1, policy_version 66004 (0.0008) [2023-12-26 15:50:36,995][105620] Updated weights for policy 1, policy_version 66014 (0.0006) [2023-12-26 15:50:37,047][105620] Updated weights for policy 1, policy_version 66024 (0.0005) [2023-12-26 15:50:37,105][105692] Updated weights for policy 0, policy_version 65562 (0.0007) [2023-12-26 15:50:37,168][105692] Updated weights for policy 0, policy_version 65572 (0.0009) [2023-12-26 15:50:37,227][105692] Updated weights for policy 0, policy_version 65582 (0.0009) [2023-12-26 15:50:37,819][105620] Updated weights for policy 1, policy_version 66034 (0.0007) [2023-12-26 15:50:37,869][105620] Updated weights for policy 1, policy_version 66044 (0.0007) [2023-12-26 15:50:37,871][105692] Updated weights for policy 0, policy_version 65592 (0.0008) [2023-12-26 15:50:37,923][105620] Updated weights for policy 1, policy_version 66054 (0.0007) [2023-12-26 15:50:37,927][105692] Updated weights for policy 0, policy_version 65602 (0.0007) [2023-12-26 15:50:37,980][105620] Updated weights for policy 1, policy_version 66064 (0.0009) [2023-12-26 15:50:37,987][105692] Updated weights for policy 0, policy_version 65612 (0.0006) [2023-12-26 15:50:38,694][105692] Updated weights for policy 0, policy_version 65622 (0.0007) [2023-12-26 15:50:38,753][105692] Updated weights for policy 0, policy_version 65632 (0.0005) [2023-12-26 15:50:38,799][105620] Updated weights for policy 1, policy_version 66074 (0.0008) [2023-12-26 15:50:38,809][105692] Updated weights for policy 0, policy_version 65642 (0.0006) [2023-12-26 15:50:38,848][105620] Updated weights for policy 1, policy_version 66084 (0.0007) [2023-12-26 15:50:38,897][105620] Updated weights for policy 1, policy_version 66094 (0.0008) [2023-12-26 15:50:39,551][105692] Updated weights for policy 0, policy_version 65652 (0.0007) [2023-12-26 15:50:39,620][105692] Updated weights for policy 0, policy_version 65662 (0.0008) [2023-12-26 15:50:39,667][105620] Updated weights for policy 1, policy_version 66104 (0.0009) [2023-12-26 15:50:39,683][105692] Updated weights for policy 0, policy_version 65672 (0.0011) [2023-12-26 15:50:39,716][105620] Updated weights for policy 1, policy_version 66114 (0.0009) [2023-12-26 15:50:39,767][105620] Updated weights for policy 1, policy_version 66124 (0.0008) [2023-12-26 15:50:40,454][105692] Updated weights for policy 0, policy_version 65682 (0.0010) [2023-12-26 15:50:40,511][105692] Updated weights for policy 0, policy_version 65692 (0.0008) [2023-12-26 15:50:40,572][105692] Updated weights for policy 0, policy_version 65702 (0.0008) [2023-12-26 15:50:40,609][105620] Updated weights for policy 1, policy_version 66134 (0.0010) [2023-12-26 15:50:40,634][105692] Updated weights for policy 0, policy_version 65712 (0.0011) [2023-12-26 15:50:40,667][105620] Updated weights for policy 1, policy_version 66144 (0.0010) [2023-12-26 15:50:40,726][105620] Updated weights for policy 1, policy_version 66154 (0.0010) [2023-12-26 15:50:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 33767424. Throughput: 0: 9814.3, 1: 9593.0. Samples: 33775208. Policy #0 lag: (min: 9.0, avg: 27.2, max: 41.0) [2023-12-26 15:50:41,062][104569] Avg episode reward: [(0, '8576.872'), (1, '8399.125')] [2023-12-26 15:50:41,399][105692] Updated weights for policy 0, policy_version 65722 (0.0008) [2023-12-26 15:50:41,466][105692] Updated weights for policy 0, policy_version 65732 (0.0009) [2023-12-26 15:50:41,483][105620] Updated weights for policy 1, policy_version 66164 (0.0009) [2023-12-26 15:50:41,528][105692] Updated weights for policy 0, policy_version 65742 (0.0009) [2023-12-26 15:50:41,550][105620] Updated weights for policy 1, policy_version 66174 (0.0006) [2023-12-26 15:50:41,624][105620] Updated weights for policy 1, policy_version 66184 (0.0008) [2023-12-26 15:50:42,292][105692] Updated weights for policy 0, policy_version 65752 (0.0010) [2023-12-26 15:50:42,349][105692] Updated weights for policy 0, policy_version 65762 (0.0010) [2023-12-26 15:50:42,350][105620] Updated weights for policy 1, policy_version 66194 (0.0010) [2023-12-26 15:50:42,414][105692] Updated weights for policy 0, policy_version 65772 (0.0011) [2023-12-26 15:50:42,414][105620] Updated weights for policy 1, policy_version 66204 (0.0011) [2023-12-26 15:50:42,474][105620] Updated weights for policy 1, policy_version 66214 (0.0011) [2023-12-26 15:50:42,529][105620] Updated weights for policy 1, policy_version 66224 (0.0010) [2023-12-26 15:50:43,148][105692] Updated weights for policy 0, policy_version 65782 (0.0008) [2023-12-26 15:50:43,206][105692] Updated weights for policy 0, policy_version 65792 (0.0005) [2023-12-26 15:50:43,275][105692] Updated weights for policy 0, policy_version 65802 (0.0005) [2023-12-26 15:50:43,277][105620] Updated weights for policy 1, policy_version 66234 (0.0010) [2023-12-26 15:50:43,336][105620] Updated weights for policy 1, policy_version 66244 (0.0010) [2023-12-26 15:50:43,394][105620] Updated weights for policy 1, policy_version 66254 (0.0010) [2023-12-26 15:50:43,856][105692] Updated weights for policy 0, policy_version 65812 (0.0006) [2023-12-26 15:50:43,906][105692] Updated weights for policy 0, policy_version 65822 (0.0005) [2023-12-26 15:50:43,962][105692] Updated weights for policy 0, policy_version 65832 (0.0005) [2023-12-26 15:50:44,148][105620] Updated weights for policy 1, policy_version 66264 (0.0010) [2023-12-26 15:50:44,199][105620] Updated weights for policy 1, policy_version 66274 (0.0010) [2023-12-26 15:50:44,261][105620] Updated weights for policy 1, policy_version 66284 (0.0010) [2023-12-26 15:50:44,568][105692] Updated weights for policy 0, policy_version 65842 (0.0006) [2023-12-26 15:50:44,623][105692] Updated weights for policy 0, policy_version 65852 (0.0010) [2023-12-26 15:50:44,667][105692] Updated weights for policy 0, policy_version 65862 (0.0008) [2023-12-26 15:50:44,718][105692] Updated weights for policy 0, policy_version 65872 (0.0005) [2023-12-26 15:50:45,005][105620] Updated weights for policy 1, policy_version 66294 (0.0010) [2023-12-26 15:50:45,055][105620] Updated weights for policy 1, policy_version 66304 (0.0008) [2023-12-26 15:50:45,106][105620] Updated weights for policy 1, policy_version 66314 (0.0009) [2023-12-26 15:50:45,371][105692] Updated weights for policy 0, policy_version 65882 (0.0008) [2023-12-26 15:50:45,428][105692] Updated weights for policy 0, policy_version 65892 (0.0008) [2023-12-26 15:50:45,483][105692] Updated weights for policy 0, policy_version 65902 (0.0009) [2023-12-26 15:50:45,818][105620] Updated weights for policy 1, policy_version 66324 (0.0008) [2023-12-26 15:50:45,864][105620] Updated weights for policy 1, policy_version 66334 (0.0005) [2023-12-26 15:50:45,913][105620] Updated weights for policy 1, policy_version 66344 (0.0005) [2023-12-26 15:50:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.9, 300 sec: 19633.0). Total num frames: 33865728. Throughput: 0: 9728.5, 1: 9530.6. Samples: 33831236. Policy #0 lag: (min: 9.0, avg: 27.2, max: 41.0) [2023-12-26 15:50:46,062][104569] Avg episode reward: [(0, '8576.187'), (1, '8138.596')] [2023-12-26 15:50:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000065904_16875520.pth... [2023-12-26 15:50:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000066352_16990208.pth... [2023-12-26 15:50:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000065232_16703488.pth [2023-12-26 15:50:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000064720_16572416.pth [2023-12-26 15:50:46,195][105692] Updated weights for policy 0, policy_version 65912 (0.0010) [2023-12-26 15:50:46,247][105692] Updated weights for policy 0, policy_version 65922 (0.0010) [2023-12-26 15:50:46,301][105692] Updated weights for policy 0, policy_version 65932 (0.0010) [2023-12-26 15:50:46,637][105620] Updated weights for policy 1, policy_version 66354 (0.0007) [2023-12-26 15:50:46,699][105620] Updated weights for policy 1, policy_version 66364 (0.0007) [2023-12-26 15:50:46,762][105620] Updated weights for policy 1, policy_version 66374 (0.0008) [2023-12-26 15:50:46,830][105620] Updated weights for policy 1, policy_version 66384 (0.0007) [2023-12-26 15:50:47,050][105692] Updated weights for policy 0, policy_version 65942 (0.0010) [2023-12-26 15:50:47,098][105692] Updated weights for policy 0, policy_version 65952 (0.0010) [2023-12-26 15:50:47,146][105692] Updated weights for policy 0, policy_version 65962 (0.0010) [2023-12-26 15:50:47,395][105620] Updated weights for policy 1, policy_version 66394 (0.0005) [2023-12-26 15:50:47,451][105620] Updated weights for policy 1, policy_version 66404 (0.0005) [2023-12-26 15:50:47,535][105620] Updated weights for policy 1, policy_version 66414 (0.0005) [2023-12-26 15:50:47,883][105692] Updated weights for policy 0, policy_version 65972 (0.0010) [2023-12-26 15:50:47,935][105692] Updated weights for policy 0, policy_version 65982 (0.0010) [2023-12-26 15:50:47,990][105692] Updated weights for policy 0, policy_version 65992 (0.0010) [2023-12-26 15:50:48,033][105620] Updated weights for policy 1, policy_version 66424 (0.0005) [2023-12-26 15:50:48,091][105620] Updated weights for policy 1, policy_version 66434 (0.0006) [2023-12-26 15:50:48,153][105620] Updated weights for policy 1, policy_version 66444 (0.0005) [2023-12-26 15:50:48,591][105692] Updated weights for policy 0, policy_version 66002 (0.0010) [2023-12-26 15:50:48,639][105692] Updated weights for policy 0, policy_version 66012 (0.0008) [2023-12-26 15:50:48,687][105692] Updated weights for policy 0, policy_version 66022 (0.0009) [2023-12-26 15:50:48,741][105692] Updated weights for policy 0, policy_version 66032 (0.0008) [2023-12-26 15:50:48,771][105620] Updated weights for policy 1, policy_version 66454 (0.0007) [2023-12-26 15:50:48,818][105620] Updated weights for policy 1, policy_version 66464 (0.0008) [2023-12-26 15:50:48,865][105620] Updated weights for policy 1, policy_version 66474 (0.0008) [2023-12-26 15:50:49,465][105692] Updated weights for policy 0, policy_version 66042 (0.0009) [2023-12-26 15:50:49,523][105692] Updated weights for policy 0, policy_version 66052 (0.0008) [2023-12-26 15:50:49,592][105692] Updated weights for policy 0, policy_version 66062 (0.0008) [2023-12-26 15:50:49,666][105620] Updated weights for policy 1, policy_version 66484 (0.0008) [2023-12-26 15:50:49,725][105620] Updated weights for policy 1, policy_version 66494 (0.0009) [2023-12-26 15:50:49,780][105620] Updated weights for policy 1, policy_version 66504 (0.0009) [2023-12-26 15:50:50,401][105692] Updated weights for policy 0, policy_version 66072 (0.0009) [2023-12-26 15:50:50,456][105692] Updated weights for policy 0, policy_version 66082 (0.0006) [2023-12-26 15:50:50,458][105620] Updated weights for policy 1, policy_version 66514 (0.0009) [2023-12-26 15:50:50,510][105692] Updated weights for policy 0, policy_version 66092 (0.0006) [2023-12-26 15:50:50,517][105620] Updated weights for policy 1, policy_version 66524 (0.0008) [2023-12-26 15:50:50,582][105620] Updated weights for policy 1, policy_version 66534 (0.0008) [2023-12-26 15:50:50,646][105620] Updated weights for policy 1, policy_version 66544 (0.0009) [2023-12-26 15:50:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 33964032. Throughput: 0: 9864.9, 1: 9607.3. Samples: 33954800. Policy #0 lag: (min: 9.0, avg: 27.2, max: 41.0) [2023-12-26 15:50:51,063][104569] Avg episode reward: [(0, '9266.293'), (1, '8313.465')] [2023-12-26 15:50:51,266][105692] Updated weights for policy 0, policy_version 66102 (0.0007) [2023-12-26 15:50:51,315][105692] Updated weights for policy 0, policy_version 66112 (0.0008) [2023-12-26 15:50:51,385][105692] Updated weights for policy 0, policy_version 66122 (0.0008) [2023-12-26 15:50:51,468][105620] Updated weights for policy 1, policy_version 66554 (0.0008) [2023-12-26 15:50:51,535][105620] Updated weights for policy 1, policy_version 66564 (0.0008) [2023-12-26 15:50:51,593][105620] Updated weights for policy 1, policy_version 66574 (0.0008) [2023-12-26 15:50:52,155][105692] Updated weights for policy 0, policy_version 66132 (0.0009) [2023-12-26 15:50:52,214][105692] Updated weights for policy 0, policy_version 66142 (0.0009) [2023-12-26 15:50:52,277][105692] Updated weights for policy 0, policy_version 66152 (0.0009) [2023-12-26 15:50:52,348][105620] Updated weights for policy 1, policy_version 66584 (0.0009) [2023-12-26 15:50:52,416][105620] Updated weights for policy 1, policy_version 66594 (0.0009) [2023-12-26 15:50:52,469][105620] Updated weights for policy 1, policy_version 66604 (0.0010) [2023-12-26 15:50:52,962][105692] Updated weights for policy 0, policy_version 66162 (0.0008) [2023-12-26 15:50:53,044][105692] Updated weights for policy 0, policy_version 66172 (0.0009) [2023-12-26 15:50:53,111][105692] Updated weights for policy 0, policy_version 66182 (0.0009) [2023-12-26 15:50:53,170][105692] Updated weights for policy 0, policy_version 66192 (0.0007) [2023-12-26 15:50:53,259][105620] Updated weights for policy 1, policy_version 66614 (0.0009) [2023-12-26 15:50:53,316][105620] Updated weights for policy 1, policy_version 66624 (0.0009) [2023-12-26 15:50:53,379][105620] Updated weights for policy 1, policy_version 66634 (0.0008) [2023-12-26 15:50:53,814][105692] Updated weights for policy 0, policy_version 66202 (0.0009) [2023-12-26 15:50:53,872][105692] Updated weights for policy 0, policy_version 66212 (0.0009) [2023-12-26 15:50:53,929][105692] Updated weights for policy 0, policy_version 66222 (0.0009) [2023-12-26 15:50:54,100][105620] Updated weights for policy 1, policy_version 66644 (0.0007) [2023-12-26 15:50:54,150][105620] Updated weights for policy 1, policy_version 66654 (0.0006) [2023-12-26 15:50:54,207][105620] Updated weights for policy 1, policy_version 66664 (0.0008) [2023-12-26 15:50:54,769][105692] Updated weights for policy 0, policy_version 66232 (0.0009) [2023-12-26 15:50:54,821][105620] Updated weights for policy 1, policy_version 66674 (0.0008) [2023-12-26 15:50:54,826][105692] Updated weights for policy 0, policy_version 66243 (0.0009) [2023-12-26 15:50:54,875][105692] Updated weights for policy 0, policy_version 66253 (0.0006) [2023-12-26 15:50:54,887][105620] Updated weights for policy 1, policy_version 66684 (0.0008) [2023-12-26 15:50:54,953][105620] Updated weights for policy 1, policy_version 66694 (0.0007) [2023-12-26 15:50:55,024][105620] Updated weights for policy 1, policy_version 66704 (0.0009) [2023-12-26 15:50:55,508][105692] Updated weights for policy 0, policy_version 66263 (0.0008) [2023-12-26 15:50:55,553][105692] Updated weights for policy 0, policy_version 66273 (0.0008) [2023-12-26 15:50:55,614][105692] Updated weights for policy 0, policy_version 66283 (0.0010) [2023-12-26 15:50:55,708][105620] Updated weights for policy 1, policy_version 66714 (0.0010) [2023-12-26 15:50:55,755][105620] Updated weights for policy 1, policy_version 66724 (0.0010) [2023-12-26 15:50:55,814][105620] Updated weights for policy 1, policy_version 66734 (0.0005) [2023-12-26 15:50:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 34062336. Throughput: 0: 9954.8, 1: 9580.5. Samples: 34069664. Policy #0 lag: (min: 9.0, avg: 27.2, max: 41.0) [2023-12-26 15:50:56,062][104569] Avg episode reward: [(0, '9261.151'), (1, '8307.180')] [2023-12-26 15:50:56,419][105692] Updated weights for policy 0, policy_version 66293 (0.0009) [2023-12-26 15:50:56,471][105692] Updated weights for policy 0, policy_version 66303 (0.0008) [2023-12-26 15:50:56,512][105620] Updated weights for policy 1, policy_version 66744 (0.0009) [2023-12-26 15:50:56,526][105692] Updated weights for policy 0, policy_version 66313 (0.0005) [2023-12-26 15:50:56,570][105620] Updated weights for policy 1, policy_version 66754 (0.0010) [2023-12-26 15:50:56,630][105620] Updated weights for policy 1, policy_version 66764 (0.0009) [2023-12-26 15:50:57,171][105692] Updated weights for policy 0, policy_version 66323 (0.0005) [2023-12-26 15:50:57,229][105692] Updated weights for policy 0, policy_version 66333 (0.0005) [2023-12-26 15:50:57,282][105692] Updated weights for policy 0, policy_version 66343 (0.0005) [2023-12-26 15:50:57,324][105620] Updated weights for policy 1, policy_version 66774 (0.0007) [2023-12-26 15:50:57,376][105620] Updated weights for policy 1, policy_version 66785 (0.0008) [2023-12-26 15:50:57,429][105620] Updated weights for policy 1, policy_version 66796 (0.0010) [2023-12-26 15:50:57,879][105692] Updated weights for policy 0, policy_version 66353 (0.0006) [2023-12-26 15:50:57,929][105692] Updated weights for policy 0, policy_version 66363 (0.0008) [2023-12-26 15:50:57,976][105692] Updated weights for policy 0, policy_version 66373 (0.0009) [2023-12-26 15:50:58,029][105692] Updated weights for policy 0, policy_version 66383 (0.0008) [2023-12-26 15:50:58,225][105620] Updated weights for policy 1, policy_version 66806 (0.0009) [2023-12-26 15:50:58,290][105620] Updated weights for policy 1, policy_version 66816 (0.0007) [2023-12-26 15:50:58,352][105620] Updated weights for policy 1, policy_version 66826 (0.0008) [2023-12-26 15:50:58,936][105692] Updated weights for policy 0, policy_version 66393 (0.0008) [2023-12-26 15:50:58,999][105692] Updated weights for policy 0, policy_version 66403 (0.0007) [2023-12-26 15:50:59,063][105692] Updated weights for policy 0, policy_version 66413 (0.0008) [2023-12-26 15:50:59,101][105620] Updated weights for policy 1, policy_version 66836 (0.0008) [2023-12-26 15:50:59,164][105620] Updated weights for policy 1, policy_version 66846 (0.0008) [2023-12-26 15:50:59,239][105620] Updated weights for policy 1, policy_version 66856 (0.0009) [2023-12-26 15:50:59,714][105692] Updated weights for policy 0, policy_version 66423 (0.0008) [2023-12-26 15:50:59,781][105692] Updated weights for policy 0, policy_version 66433 (0.0011) [2023-12-26 15:50:59,845][105692] Updated weights for policy 0, policy_version 66443 (0.0010) [2023-12-26 15:51:00,004][105620] Updated weights for policy 1, policy_version 66866 (0.0007) [2023-12-26 15:51:00,058][105620] Updated weights for policy 1, policy_version 66876 (0.0009) [2023-12-26 15:51:00,111][105620] Updated weights for policy 1, policy_version 66886 (0.0009) [2023-12-26 15:51:00,172][105620] Updated weights for policy 1, policy_version 66896 (0.0009) [2023-12-26 15:51:00,607][105692] Updated weights for policy 0, policy_version 66453 (0.0010) [2023-12-26 15:51:00,656][105692] Updated weights for policy 0, policy_version 66463 (0.0010) [2023-12-26 15:51:00,715][105692] Updated weights for policy 0, policy_version 66473 (0.0008) [2023-12-26 15:51:00,911][105620] Updated weights for policy 1, policy_version 66906 (0.0008) [2023-12-26 15:51:00,958][105620] Updated weights for policy 1, policy_version 66916 (0.0008) [2023-12-26 15:51:01,002][105620] Updated weights for policy 1, policy_version 66926 (0.0008) [2023-12-26 15:51:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 34160640. Throughput: 0: 9995.1, 1: 9607.8. Samples: 34128148. Policy #0 lag: (min: 9.0, avg: 27.2, max: 41.0) [2023-12-26 15:51:01,062][104569] Avg episode reward: [(0, '9091.647'), (1, '8655.335')] [2023-12-26 15:51:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000066480_17022976.pth... [2023-12-26 15:51:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000066928_17137664.pth... [2023-12-26 15:51:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000065776_16842752.pth [2023-12-26 15:51:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000065328_16728064.pth [2023-12-26 15:51:01,440][105692] Updated weights for policy 0, policy_version 66483 (0.0006) [2023-12-26 15:51:01,496][105692] Updated weights for policy 0, policy_version 66493 (0.0006) [2023-12-26 15:51:01,561][105692] Updated weights for policy 0, policy_version 66503 (0.0005) [2023-12-26 15:51:01,831][105620] Updated weights for policy 1, policy_version 66936 (0.0006) [2023-12-26 15:51:01,889][105620] Updated weights for policy 1, policy_version 66946 (0.0009) [2023-12-26 15:51:01,944][105620] Updated weights for policy 1, policy_version 66956 (0.0009) [2023-12-26 15:51:02,201][105692] Updated weights for policy 0, policy_version 66513 (0.0006) [2023-12-26 15:51:02,259][105692] Updated weights for policy 0, policy_version 66523 (0.0010) [2023-12-26 15:51:02,316][105692] Updated weights for policy 0, policy_version 66533 (0.0010) [2023-12-26 15:51:02,380][105692] Updated weights for policy 0, policy_version 66543 (0.0010) [2023-12-26 15:51:02,642][105620] Updated weights for policy 1, policy_version 66966 (0.0009) [2023-12-26 15:51:02,707][105620] Updated weights for policy 1, policy_version 66976 (0.0007) [2023-12-26 15:51:02,773][105620] Updated weights for policy 1, policy_version 66986 (0.0006) [2023-12-26 15:51:03,142][105692] Updated weights for policy 0, policy_version 66553 (0.0011) [2023-12-26 15:51:03,203][105692] Updated weights for policy 0, policy_version 66563 (0.0005) [2023-12-26 15:51:03,252][105692] Updated weights for policy 0, policy_version 66573 (0.0005) [2023-12-26 15:51:03,524][105620] Updated weights for policy 1, policy_version 66996 (0.0007) [2023-12-26 15:51:03,586][105620] Updated weights for policy 1, policy_version 67007 (0.0009) [2023-12-26 15:51:03,641][105620] Updated weights for policy 1, policy_version 67017 (0.0010) [2023-12-26 15:51:03,815][105692] Updated weights for policy 0, policy_version 66583 (0.0009) [2023-12-26 15:51:03,879][105692] Updated weights for policy 0, policy_version 66593 (0.0011) [2023-12-26 15:51:03,930][105692] Updated weights for policy 0, policy_version 66603 (0.0009) [2023-12-26 15:51:04,425][105620] Updated weights for policy 1, policy_version 67027 (0.0007) [2023-12-26 15:51:04,473][105620] Updated weights for policy 1, policy_version 67037 (0.0008) [2023-12-26 15:51:04,526][105620] Updated weights for policy 1, policy_version 67047 (0.0008) [2023-12-26 15:51:04,653][105692] Updated weights for policy 0, policy_version 66613 (0.0008) [2023-12-26 15:51:04,710][105692] Updated weights for policy 0, policy_version 66623 (0.0010) [2023-12-26 15:51:04,772][105692] Updated weights for policy 0, policy_version 66633 (0.0009) [2023-12-26 15:51:05,245][105620] Updated weights for policy 1, policy_version 67057 (0.0008) [2023-12-26 15:51:05,312][105620] Updated weights for policy 1, policy_version 67067 (0.0010) [2023-12-26 15:51:05,383][105620] Updated weights for policy 1, policy_version 67077 (0.0008) [2023-12-26 15:51:05,424][105692] Updated weights for policy 0, policy_version 66643 (0.0008) [2023-12-26 15:51:05,448][105620] Updated weights for policy 1, policy_version 67087 (0.0010) [2023-12-26 15:51:05,468][105692] Updated weights for policy 0, policy_version 66653 (0.0005) [2023-12-26 15:51:05,518][105692] Updated weights for policy 0, policy_version 66663 (0.0005) [2023-12-26 15:51:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 34250752. Throughput: 0: 9971.1, 1: 9604.3. Samples: 34243016. Policy #0 lag: (min: 9.0, avg: 33.1, max: 41.0) [2023-12-26 15:51:06,062][104569] Avg episode reward: [(0, '9096.781'), (1, '8920.405')] [2023-12-26 15:51:06,094][105692] Updated weights for policy 0, policy_version 66673 (0.0005) [2023-12-26 15:51:06,158][105692] Updated weights for policy 0, policy_version 66683 (0.0008) [2023-12-26 15:51:06,161][105620] Updated weights for policy 1, policy_version 67097 (0.0007) [2023-12-26 15:51:06,219][105620] Updated weights for policy 1, policy_version 67107 (0.0005) [2023-12-26 15:51:06,221][105692] Updated weights for policy 0, policy_version 66693 (0.0008) [2023-12-26 15:51:06,266][105620] Updated weights for policy 1, policy_version 67117 (0.0007) [2023-12-26 15:51:06,288][105692] Updated weights for policy 0, policy_version 66703 (0.0008) [2023-12-26 15:51:06,890][105620] Updated weights for policy 1, policy_version 67127 (0.0007) [2023-12-26 15:51:06,962][105620] Updated weights for policy 1, policy_version 67137 (0.0009) [2023-12-26 15:51:07,013][105692] Updated weights for policy 0, policy_version 66713 (0.0008) [2023-12-26 15:51:07,023][105620] Updated weights for policy 1, policy_version 67147 (0.0006) [2023-12-26 15:51:07,068][105692] Updated weights for policy 0, policy_version 66723 (0.0009) [2023-12-26 15:51:07,125][105692] Updated weights for policy 0, policy_version 66734 (0.0010) [2023-12-26 15:51:07,707][105620] Updated weights for policy 1, policy_version 67157 (0.0008) [2023-12-26 15:51:07,762][105692] Updated weights for policy 0, policy_version 66744 (0.0006) [2023-12-26 15:51:07,763][105620] Updated weights for policy 1, policy_version 67167 (0.0010) [2023-12-26 15:51:07,816][105620] Updated weights for policy 1, policy_version 67177 (0.0011) [2023-12-26 15:51:07,820][105692] Updated weights for policy 0, policy_version 66754 (0.0006) [2023-12-26 15:51:07,880][105692] Updated weights for policy 0, policy_version 66764 (0.0005) [2023-12-26 15:51:08,558][105692] Updated weights for policy 0, policy_version 66774 (0.0009) [2023-12-26 15:51:08,576][105620] Updated weights for policy 1, policy_version 67187 (0.0009) [2023-12-26 15:51:08,617][105692] Updated weights for policy 0, policy_version 66784 (0.0011) [2023-12-26 15:51:08,624][105620] Updated weights for policy 1, policy_version 67197 (0.0007) [2023-12-26 15:51:08,670][105620] Updated weights for policy 1, policy_version 67207 (0.0009) [2023-12-26 15:51:08,678][105692] Updated weights for policy 0, policy_version 66794 (0.0010) [2023-12-26 15:51:09,405][105620] Updated weights for policy 1, policy_version 67217 (0.0008) [2023-12-26 15:51:09,422][105692] Updated weights for policy 0, policy_version 66804 (0.0011) [2023-12-26 15:51:09,470][105620] Updated weights for policy 1, policy_version 67227 (0.0010) [2023-12-26 15:51:09,478][105692] Updated weights for policy 0, policy_version 66814 (0.0011) [2023-12-26 15:51:09,532][105620] Updated weights for policy 1, policy_version 67237 (0.0010) [2023-12-26 15:51:09,537][105692] Updated weights for policy 0, policy_version 66824 (0.0011) [2023-12-26 15:51:09,598][105620] Updated weights for policy 1, policy_version 67247 (0.0010) [2023-12-26 15:51:10,277][105692] Updated weights for policy 0, policy_version 66834 (0.0009) [2023-12-26 15:51:10,331][105620] Updated weights for policy 1, policy_version 67257 (0.0010) [2023-12-26 15:51:10,332][105692] Updated weights for policy 0, policy_version 66844 (0.0006) [2023-12-26 15:51:10,382][105692] Updated weights for policy 0, policy_version 66854 (0.0011) [2023-12-26 15:51:10,394][105620] Updated weights for policy 1, policy_version 67267 (0.0010) [2023-12-26 15:51:10,431][105692] Updated weights for policy 0, policy_version 66864 (0.0010) [2023-12-26 15:51:10,456][105620] Updated weights for policy 1, policy_version 67277 (0.0010) [2023-12-26 15:51:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 34349056. Throughput: 0: 10060.8, 1: 9667.3. Samples: 34362384. Policy #0 lag: (min: 9.0, avg: 33.1, max: 41.0) [2023-12-26 15:51:11,062][104569] Avg episode reward: [(0, '9097.229'), (1, '9004.013')] [2023-12-26 15:51:11,139][105620] Updated weights for policy 1, policy_version 67287 (0.0011) [2023-12-26 15:51:11,154][105692] Updated weights for policy 0, policy_version 66874 (0.0010) [2023-12-26 15:51:11,197][105620] Updated weights for policy 1, policy_version 67297 (0.0009) [2023-12-26 15:51:11,213][105692] Updated weights for policy 0, policy_version 66884 (0.0006) [2023-12-26 15:51:11,252][105620] Updated weights for policy 1, policy_version 67307 (0.0009) [2023-12-26 15:51:11,278][105692] Updated weights for policy 0, policy_version 66894 (0.0008) [2023-12-26 15:51:12,007][105692] Updated weights for policy 0, policy_version 66904 (0.0007) [2023-12-26 15:51:12,046][105620] Updated weights for policy 1, policy_version 67317 (0.0007) [2023-12-26 15:51:12,074][105692] Updated weights for policy 0, policy_version 66914 (0.0009) [2023-12-26 15:51:12,105][105620] Updated weights for policy 1, policy_version 67327 (0.0007) [2023-12-26 15:51:12,140][105692] Updated weights for policy 0, policy_version 66924 (0.0007) [2023-12-26 15:51:12,163][105620] Updated weights for policy 1, policy_version 67337 (0.0009) [2023-12-26 15:51:12,796][105620] Updated weights for policy 1, policy_version 67347 (0.0007) [2023-12-26 15:51:12,854][105620] Updated weights for policy 1, policy_version 67357 (0.0008) [2023-12-26 15:51:12,864][105692] Updated weights for policy 0, policy_version 66934 (0.0009) [2023-12-26 15:51:12,915][105620] Updated weights for policy 1, policy_version 67367 (0.0005) [2023-12-26 15:51:12,916][105692] Updated weights for policy 0, policy_version 66944 (0.0010) [2023-12-26 15:51:12,976][105692] Updated weights for policy 0, policy_version 66954 (0.0005) [2023-12-26 15:51:13,553][105692] Updated weights for policy 0, policy_version 66964 (0.0006) [2023-12-26 15:51:13,615][105620] Updated weights for policy 1, policy_version 67377 (0.0009) [2023-12-26 15:51:13,619][105692] Updated weights for policy 0, policy_version 66974 (0.0008) [2023-12-26 15:51:13,676][105620] Updated weights for policy 1, policy_version 67387 (0.0009) [2023-12-26 15:51:13,677][105692] Updated weights for policy 0, policy_version 66984 (0.0005) [2023-12-26 15:51:13,732][105620] Updated weights for policy 1, policy_version 67397 (0.0011) [2023-12-26 15:51:13,792][105620] Updated weights for policy 1, policy_version 67407 (0.0011) [2023-12-26 15:51:14,235][105692] Updated weights for policy 0, policy_version 66994 (0.0007) [2023-12-26 15:51:14,279][105692] Updated weights for policy 0, policy_version 67004 (0.0010) [2023-12-26 15:51:14,324][105692] Updated weights for policy 0, policy_version 67014 (0.0005) [2023-12-26 15:51:14,368][105692] Updated weights for policy 0, policy_version 67024 (0.0005) [2023-12-26 15:51:14,538][105620] Updated weights for policy 1, policy_version 67417 (0.0010) [2023-12-26 15:51:14,599][105620] Updated weights for policy 1, policy_version 67427 (0.0010) [2023-12-26 15:51:14,643][105620] Updated weights for policy 1, policy_version 67437 (0.0009) [2023-12-26 15:51:14,960][105692] Updated weights for policy 0, policy_version 67034 (0.0006) [2023-12-26 15:51:15,019][105692] Updated weights for policy 0, policy_version 67044 (0.0006) [2023-12-26 15:51:15,075][105692] Updated weights for policy 0, policy_version 67054 (0.0005) [2023-12-26 15:51:15,414][105620] Updated weights for policy 1, policy_version 67447 (0.0010) [2023-12-26 15:51:15,479][105620] Updated weights for policy 1, policy_version 67457 (0.0010) [2023-12-26 15:51:15,537][105620] Updated weights for policy 1, policy_version 67467 (0.0010) [2023-12-26 15:51:15,752][105692] Updated weights for policy 0, policy_version 67064 (0.0009) [2023-12-26 15:51:15,797][105692] Updated weights for policy 0, policy_version 67074 (0.0010) [2023-12-26 15:51:15,845][105692] Updated weights for policy 0, policy_version 67084 (0.0010) [2023-12-26 15:51:16,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 34455552. Throughput: 0: 9989.5, 1: 9668.7. Samples: 34422348. Policy #0 lag: (min: 9.0, avg: 33.1, max: 41.0) [2023-12-26 15:51:16,062][104569] Avg episode reward: [(0, '9181.115'), (1, '8053.763')] [2023-12-26 15:51:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000067088_17178624.pth... [2023-12-26 15:51:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000067472_17276928.pth... [2023-12-26 15:51:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000066352_16990208.pth [2023-12-26 15:51:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000065904_16875520.pth [2023-12-26 15:51:16,251][105620] Updated weights for policy 1, policy_version 67477 (0.0010) [2023-12-26 15:51:16,298][105620] Updated weights for policy 1, policy_version 67487 (0.0010) [2023-12-26 15:51:16,345][105620] Updated weights for policy 1, policy_version 67497 (0.0008) [2023-12-26 15:51:16,557][105692] Updated weights for policy 0, policy_version 67094 (0.0010) [2023-12-26 15:51:16,623][105692] Updated weights for policy 0, policy_version 67104 (0.0010) [2023-12-26 15:51:16,681][105692] Updated weights for policy 0, policy_version 67114 (0.0010) [2023-12-26 15:51:17,099][105620] Updated weights for policy 1, policy_version 67507 (0.0008) [2023-12-26 15:51:17,151][105620] Updated weights for policy 1, policy_version 67517 (0.0007) [2023-12-26 15:51:17,206][105620] Updated weights for policy 1, policy_version 67527 (0.0008) [2023-12-26 15:51:17,422][105692] Updated weights for policy 0, policy_version 67124 (0.0010) [2023-12-26 15:51:17,474][105692] Updated weights for policy 0, policy_version 67134 (0.0011) [2023-12-26 15:51:17,522][105692] Updated weights for policy 0, policy_version 67144 (0.0010) [2023-12-26 15:51:17,970][105620] Updated weights for policy 1, policy_version 67537 (0.0010) [2023-12-26 15:51:18,025][105620] Updated weights for policy 1, policy_version 67547 (0.0010) [2023-12-26 15:51:18,084][105620] Updated weights for policy 1, policy_version 67558 (0.0010) [2023-12-26 15:51:18,145][105620] Updated weights for policy 1, policy_version 67568 (0.0010) [2023-12-26 15:51:18,258][105692] Updated weights for policy 0, policy_version 67154 (0.0010) [2023-12-26 15:51:18,321][105692] Updated weights for policy 0, policy_version 67164 (0.0010) [2023-12-26 15:51:18,393][105692] Updated weights for policy 0, policy_version 67174 (0.0009) [2023-12-26 15:51:18,459][105692] Updated weights for policy 0, policy_version 67184 (0.0011) [2023-12-26 15:51:18,908][105620] Updated weights for policy 1, policy_version 67578 (0.0011) [2023-12-26 15:51:18,967][105620] Updated weights for policy 1, policy_version 67588 (0.0010) [2023-12-26 15:51:19,023][105620] Updated weights for policy 1, policy_version 67598 (0.0011) [2023-12-26 15:51:19,154][105692] Updated weights for policy 0, policy_version 67194 (0.0011) [2023-12-26 15:51:19,224][105692] Updated weights for policy 0, policy_version 67204 (0.0010) [2023-12-26 15:51:19,292][105692] Updated weights for policy 0, policy_version 67214 (0.0008) [2023-12-26 15:51:19,756][105620] Updated weights for policy 1, policy_version 67608 (0.0011) [2023-12-26 15:51:19,823][105620] Updated weights for policy 1, policy_version 67618 (0.0011) [2023-12-26 15:51:19,886][105620] Updated weights for policy 1, policy_version 67628 (0.0009) [2023-12-26 15:51:20,054][105692] Updated weights for policy 0, policy_version 67224 (0.0008) [2023-12-26 15:51:20,120][105692] Updated weights for policy 0, policy_version 67234 (0.0008) [2023-12-26 15:51:20,179][105692] Updated weights for policy 0, policy_version 67244 (0.0008) [2023-12-26 15:51:20,576][105620] Updated weights for policy 1, policy_version 67638 (0.0007) [2023-12-26 15:51:20,644][105620] Updated weights for policy 1, policy_version 67648 (0.0008) [2023-12-26 15:51:20,706][105620] Updated weights for policy 1, policy_version 67658 (0.0006) [2023-12-26 15:51:20,878][105692] Updated weights for policy 0, policy_version 67254 (0.0007) [2023-12-26 15:51:20,932][105692] Updated weights for policy 0, policy_version 67264 (0.0011) [2023-12-26 15:51:20,981][105692] Updated weights for policy 0, policy_version 67274 (0.0010) [2023-12-26 15:51:21,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 34553856. Throughput: 0: 9965.1, 1: 9620.9. Samples: 34539408. Policy #0 lag: (min: 9.0, avg: 33.1, max: 41.0) [2023-12-26 15:51:21,062][104569] Avg episode reward: [(0, '9181.832'), (1, '7800.607')] [2023-12-26 15:51:21,435][105620] Updated weights for policy 1, policy_version 67668 (0.0010) [2023-12-26 15:51:21,498][105620] Updated weights for policy 1, policy_version 67678 (0.0009) [2023-12-26 15:51:21,559][105620] Updated weights for policy 1, policy_version 67688 (0.0009) [2023-12-26 15:51:21,670][105692] Updated weights for policy 0, policy_version 67284 (0.0010) [2023-12-26 15:51:21,753][105692] Updated weights for policy 0, policy_version 67294 (0.0012) [2023-12-26 15:51:21,816][105692] Updated weights for policy 0, policy_version 67304 (0.0010) [2023-12-26 15:51:22,426][105620] Updated weights for policy 1, policy_version 67698 (0.0008) [2023-12-26 15:51:22,496][105620] Updated weights for policy 1, policy_version 67708 (0.0006) [2023-12-26 15:51:22,544][105692] Updated weights for policy 0, policy_version 67314 (0.0010) [2023-12-26 15:51:22,559][105620] Updated weights for policy 1, policy_version 67718 (0.0008) [2023-12-26 15:51:22,606][105692] Updated weights for policy 0, policy_version 67324 (0.0008) [2023-12-26 15:51:22,629][105620] Updated weights for policy 1, policy_version 67728 (0.0006) [2023-12-26 15:51:22,672][105692] Updated weights for policy 0, policy_version 67334 (0.0009) [2023-12-26 15:51:22,739][105692] Updated weights for policy 0, policy_version 67344 (0.0009) [2023-12-26 15:51:23,308][105620] Updated weights for policy 1, policy_version 67738 (0.0007) [2023-12-26 15:51:23,379][105620] Updated weights for policy 1, policy_version 67748 (0.0008) [2023-12-26 15:51:23,386][105692] Updated weights for policy 0, policy_version 67354 (0.0006) [2023-12-26 15:51:23,435][105692] Updated weights for policy 0, policy_version 67364 (0.0006) [2023-12-26 15:51:23,446][105620] Updated weights for policy 1, policy_version 67758 (0.0008) [2023-12-26 15:51:23,488][105692] Updated weights for policy 0, policy_version 67374 (0.0007) [2023-12-26 15:51:24,138][105620] Updated weights for policy 1, policy_version 67768 (0.0008) [2023-12-26 15:51:24,184][105620] Updated weights for policy 1, policy_version 67778 (0.0007) [2023-12-26 15:51:24,220][105692] Updated weights for policy 0, policy_version 67384 (0.0010) [2023-12-26 15:51:24,237][105620] Updated weights for policy 1, policy_version 67788 (0.0005) [2023-12-26 15:51:24,276][105692] Updated weights for policy 0, policy_version 67394 (0.0010) [2023-12-26 15:51:24,326][105692] Updated weights for policy 0, policy_version 67404 (0.0007) [2023-12-26 15:51:24,864][105620] Updated weights for policy 1, policy_version 67798 (0.0006) [2023-12-26 15:51:24,911][105620] Updated weights for policy 1, policy_version 67808 (0.0010) [2023-12-26 15:51:24,979][105620] Updated weights for policy 1, policy_version 67818 (0.0011) [2023-12-26 15:51:25,153][105692] Updated weights for policy 0, policy_version 67414 (0.0007) [2023-12-26 15:51:25,210][105692] Updated weights for policy 0, policy_version 67424 (0.0007) [2023-12-26 15:51:25,259][105692] Updated weights for policy 0, policy_version 67434 (0.0006) [2023-12-26 15:51:25,691][105620] Updated weights for policy 1, policy_version 67828 (0.0010) [2023-12-26 15:51:25,742][105620] Updated weights for policy 1, policy_version 67838 (0.0010) [2023-12-26 15:51:25,793][105620] Updated weights for policy 1, policy_version 67848 (0.0010) [2023-12-26 15:51:25,954][105692] Updated weights for policy 0, policy_version 67444 (0.0008) [2023-12-26 15:51:26,008][105692] Updated weights for policy 0, policy_version 67454 (0.0010) [2023-12-26 15:51:26,060][105692] Updated weights for policy 0, policy_version 67464 (0.0010) [2023-12-26 15:51:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 34643968. Throughput: 0: 9923.6, 1: 9637.7. Samples: 34655464. Policy #0 lag: (min: 9.0, avg: 33.1, max: 41.0) [2023-12-26 15:51:26,062][104569] Avg episode reward: [(0, '9267.369'), (1, '8230.162')] [2023-12-26 15:51:26,363][105620] Updated weights for policy 1, policy_version 67858 (0.0007) [2023-12-26 15:51:26,411][105620] Updated weights for policy 1, policy_version 67868 (0.0005) [2023-12-26 15:51:26,465][105620] Updated weights for policy 1, policy_version 67878 (0.0005) [2023-12-26 15:51:26,526][105620] Updated weights for policy 1, policy_version 67888 (0.0005) [2023-12-26 15:51:26,802][105692] Updated weights for policy 0, policy_version 67474 (0.0010) [2023-12-26 15:51:26,863][105692] Updated weights for policy 0, policy_version 67484 (0.0005) [2023-12-26 15:51:26,919][105692] Updated weights for policy 0, policy_version 67494 (0.0005) [2023-12-26 15:51:26,973][105692] Updated weights for policy 0, policy_version 67504 (0.0005) [2023-12-26 15:51:27,029][105620] Updated weights for policy 1, policy_version 67898 (0.0005) [2023-12-26 15:51:27,087][105620] Updated weights for policy 1, policy_version 67908 (0.0005) [2023-12-26 15:51:27,151][105620] Updated weights for policy 1, policy_version 67918 (0.0007) [2023-12-26 15:51:27,494][105692] Updated weights for policy 0, policy_version 67514 (0.0005) [2023-12-26 15:51:27,549][105692] Updated weights for policy 0, policy_version 67524 (0.0005) [2023-12-26 15:51:27,602][105692] Updated weights for policy 0, policy_version 67534 (0.0005) [2023-12-26 15:51:27,817][105620] Updated weights for policy 1, policy_version 67928 (0.0010) [2023-12-26 15:51:27,869][105620] Updated weights for policy 1, policy_version 67938 (0.0010) [2023-12-26 15:51:27,927][105620] Updated weights for policy 1, policy_version 67948 (0.0010) [2023-12-26 15:51:28,232][105692] Updated weights for policy 0, policy_version 67544 (0.0008) [2023-12-26 15:51:28,296][105692] Updated weights for policy 0, policy_version 67554 (0.0010) [2023-12-26 15:51:28,350][105692] Updated weights for policy 0, policy_version 67564 (0.0008) [2023-12-26 15:51:28,555][105620] Updated weights for policy 1, policy_version 67958 (0.0008) [2023-12-26 15:51:28,602][105620] Updated weights for policy 1, policy_version 67968 (0.0010) [2023-12-26 15:51:28,646][105620] Updated weights for policy 1, policy_version 67978 (0.0005) [2023-12-26 15:51:29,157][105692] Updated weights for policy 0, policy_version 67574 (0.0008) [2023-12-26 15:51:29,208][105692] Updated weights for policy 0, policy_version 67584 (0.0008) [2023-12-26 15:51:29,243][105620] Updated weights for policy 1, policy_version 67988 (0.0007) [2023-12-26 15:51:29,272][105692] Updated weights for policy 0, policy_version 67594 (0.0008) [2023-12-26 15:51:29,305][105620] Updated weights for policy 1, policy_version 67998 (0.0010) [2023-12-26 15:51:29,370][105620] Updated weights for policy 1, policy_version 68008 (0.0011) [2023-12-26 15:51:29,936][105692] Updated weights for policy 0, policy_version 67604 (0.0006) [2023-12-26 15:51:29,995][105692] Updated weights for policy 0, policy_version 67614 (0.0010) [2023-12-26 15:51:30,050][105692] Updated weights for policy 0, policy_version 67624 (0.0010) [2023-12-26 15:51:30,125][105620] Updated weights for policy 1, policy_version 68018 (0.0010) [2023-12-26 15:51:30,186][105620] Updated weights for policy 1, policy_version 68028 (0.0011) [2023-12-26 15:51:30,251][105620] Updated weights for policy 1, policy_version 68038 (0.0011) [2023-12-26 15:51:30,315][105620] Updated weights for policy 1, policy_version 68048 (0.0011) [2023-12-26 15:51:30,685][105692] Updated weights for policy 0, policy_version 67634 (0.0007) [2023-12-26 15:51:30,738][105692] Updated weights for policy 0, policy_version 67644 (0.0005) [2023-12-26 15:51:30,786][105692] Updated weights for policy 0, policy_version 67654 (0.0005) [2023-12-26 15:51:30,844][105692] Updated weights for policy 0, policy_version 67664 (0.0005) [2023-12-26 15:51:31,049][105620] Updated weights for policy 1, policy_version 68058 (0.0009) [2023-12-26 15:51:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 34750464. Throughput: 0: 10006.4, 1: 9802.5. Samples: 34722636. Policy #0 lag: (min: 9.0, avg: 33.1, max: 41.0) [2023-12-26 15:51:31,063][104569] Avg episode reward: [(0, '9101.386'), (1, '8662.218')] [2023-12-26 15:51:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000067664_17326080.pth... [2023-12-26 15:51:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000066480_17022976.pth [2023-12-26 15:51:31,101][105620] Updated weights for policy 1, policy_version 68068 (0.0007) [2023-12-26 15:51:31,162][105620] Updated weights for policy 1, policy_version 68078 (0.0009) [2023-12-26 15:51:31,172][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000068080_17432576.pth... [2023-12-26 15:51:31,177][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000066928_17137664.pth [2023-12-26 15:51:31,507][105692] Updated weights for policy 0, policy_version 67674 (0.0008) [2023-12-26 15:51:31,564][105692] Updated weights for policy 0, policy_version 67684 (0.0008) [2023-12-26 15:51:31,627][105692] Updated weights for policy 0, policy_version 67694 (0.0009) [2023-12-26 15:51:31,909][105620] Updated weights for policy 1, policy_version 68088 (0.0008) [2023-12-26 15:51:31,961][105620] Updated weights for policy 1, policy_version 68098 (0.0008) [2023-12-26 15:51:32,020][105620] Updated weights for policy 1, policy_version 68108 (0.0008) [2023-12-26 15:51:32,386][105692] Updated weights for policy 0, policy_version 67704 (0.0011) [2023-12-26 15:51:32,443][105692] Updated weights for policy 0, policy_version 67714 (0.0006) [2023-12-26 15:51:32,494][105692] Updated weights for policy 0, policy_version 67724 (0.0006) [2023-12-26 15:51:32,801][105620] Updated weights for policy 1, policy_version 68118 (0.0007) [2023-12-26 15:51:32,856][105620] Updated weights for policy 1, policy_version 68128 (0.0008) [2023-12-26 15:51:32,917][105620] Updated weights for policy 1, policy_version 68138 (0.0009) [2023-12-26 15:51:33,146][105692] Updated weights for policy 0, policy_version 67734 (0.0008) [2023-12-26 15:51:33,203][105692] Updated weights for policy 0, policy_version 67744 (0.0007) [2023-12-26 15:51:33,268][105692] Updated weights for policy 0, policy_version 67754 (0.0006) [2023-12-26 15:51:33,581][105620] Updated weights for policy 1, policy_version 68148 (0.0008) [2023-12-26 15:51:33,638][105620] Updated weights for policy 1, policy_version 68158 (0.0005) [2023-12-26 15:51:33,694][105620] Updated weights for policy 1, policy_version 68168 (0.0005) [2023-12-26 15:51:33,901][105692] Updated weights for policy 0, policy_version 67764 (0.0005) [2023-12-26 15:51:33,944][105692] Updated weights for policy 0, policy_version 67774 (0.0005) [2023-12-26 15:51:33,987][105692] Updated weights for policy 0, policy_version 67784 (0.0005) [2023-12-26 15:51:34,251][105620] Updated weights for policy 1, policy_version 68178 (0.0006) [2023-12-26 15:51:34,307][105620] Updated weights for policy 1, policy_version 68188 (0.0009) [2023-12-26 15:51:34,361][105620] Updated weights for policy 1, policy_version 68198 (0.0008) [2023-12-26 15:51:34,409][105620] Updated weights for policy 1, policy_version 68208 (0.0009) [2023-12-26 15:51:34,698][105692] Updated weights for policy 0, policy_version 67794 (0.0006) [2023-12-26 15:51:34,765][105692] Updated weights for policy 0, policy_version 67804 (0.0010) [2023-12-26 15:51:34,832][105692] Updated weights for policy 0, policy_version 67814 (0.0010) [2023-12-26 15:51:34,898][105692] Updated weights for policy 0, policy_version 67824 (0.0009) [2023-12-26 15:51:35,051][105620] Updated weights for policy 1, policy_version 68218 (0.0006) [2023-12-26 15:51:35,102][105620] Updated weights for policy 1, policy_version 68228 (0.0007) [2023-12-26 15:51:35,161][105620] Updated weights for policy 1, policy_version 68238 (0.0009) [2023-12-26 15:51:35,685][105692] Updated weights for policy 0, policy_version 67834 (0.0009) [2023-12-26 15:51:35,733][105692] Updated weights for policy 0, policy_version 67844 (0.0009) [2023-12-26 15:51:35,786][105692] Updated weights for policy 0, policy_version 67854 (0.0010) [2023-12-26 15:51:35,856][105620] Updated weights for policy 1, policy_version 68248 (0.0007) [2023-12-26 15:51:35,907][105620] Updated weights for policy 1, policy_version 68258 (0.0005) [2023-12-26 15:51:35,958][105620] Updated weights for policy 1, policy_version 68268 (0.0005) [2023-12-26 15:51:36,062][104569] Fps is (10 sec: 21298.7, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 34856960. Throughput: 0: 9980.9, 1: 9756.0. Samples: 34842964. Policy #0 lag: (min: 9.0, avg: 33.1, max: 41.0) [2023-12-26 15:51:36,063][104569] Avg episode reward: [(0, '9014.660'), (1, '8919.978')] [2023-12-26 15:51:36,472][105692] Updated weights for policy 0, policy_version 67864 (0.0010) [2023-12-26 15:51:36,520][105692] Updated weights for policy 0, policy_version 67874 (0.0009) [2023-12-26 15:51:36,574][105692] Updated weights for policy 0, policy_version 67884 (0.0007) [2023-12-26 15:51:36,697][105620] Updated weights for policy 1, policy_version 68278 (0.0006) [2023-12-26 15:51:36,758][105620] Updated weights for policy 1, policy_version 68288 (0.0006) [2023-12-26 15:51:36,822][105620] Updated weights for policy 1, policy_version 68298 (0.0006) [2023-12-26 15:51:37,356][105692] Updated weights for policy 0, policy_version 67894 (0.0006) [2023-12-26 15:51:37,415][105692] Updated weights for policy 0, policy_version 67904 (0.0005) [2023-12-26 15:51:37,426][105620] Updated weights for policy 1, policy_version 68308 (0.0007) [2023-12-26 15:51:37,476][105692] Updated weights for policy 0, policy_version 67914 (0.0006) [2023-12-26 15:51:37,494][105620] Updated weights for policy 1, policy_version 68318 (0.0009) [2023-12-26 15:51:37,546][105620] Updated weights for policy 1, policy_version 68328 (0.0009) [2023-12-26 15:51:38,072][105692] Updated weights for policy 0, policy_version 67924 (0.0007) [2023-12-26 15:51:38,127][105692] Updated weights for policy 0, policy_version 67934 (0.0005) [2023-12-26 15:51:38,188][105692] Updated weights for policy 0, policy_version 67944 (0.0005) [2023-12-26 15:51:38,438][105620] Updated weights for policy 1, policy_version 68338 (0.0008) [2023-12-26 15:51:38,498][105620] Updated weights for policy 1, policy_version 68348 (0.0007) [2023-12-26 15:51:38,543][105620] Updated weights for policy 1, policy_version 68358 (0.0008) [2023-12-26 15:51:38,603][105620] Updated weights for policy 1, policy_version 68368 (0.0008) [2023-12-26 15:51:38,849][105692] Updated weights for policy 0, policy_version 67954 (0.0006) [2023-12-26 15:51:38,897][105692] Updated weights for policy 0, policy_version 67964 (0.0010) [2023-12-26 15:51:38,957][105692] Updated weights for policy 0, policy_version 67974 (0.0010) [2023-12-26 15:51:39,026][105692] Updated weights for policy 0, policy_version 67984 (0.0011) [2023-12-26 15:51:39,210][105620] Updated weights for policy 1, policy_version 68378 (0.0008) [2023-12-26 15:51:39,282][105620] Updated weights for policy 1, policy_version 68388 (0.0007) [2023-12-26 15:51:39,352][105620] Updated weights for policy 1, policy_version 68398 (0.0006) [2023-12-26 15:51:39,737][105692] Updated weights for policy 0, policy_version 67994 (0.0010) [2023-12-26 15:51:39,792][105692] Updated weights for policy 0, policy_version 68004 (0.0010) [2023-12-26 15:51:39,866][105692] Updated weights for policy 0, policy_version 68014 (0.0011) [2023-12-26 15:51:40,098][105620] Updated weights for policy 1, policy_version 68408 (0.0008) [2023-12-26 15:51:40,165][105620] Updated weights for policy 1, policy_version 68418 (0.0008) [2023-12-26 15:51:40,223][105620] Updated weights for policy 1, policy_version 68428 (0.0007) [2023-12-26 15:51:40,617][105692] Updated weights for policy 0, policy_version 68024 (0.0011) [2023-12-26 15:51:40,672][105692] Updated weights for policy 0, policy_version 68034 (0.0010) [2023-12-26 15:51:40,724][105692] Updated weights for policy 0, policy_version 68044 (0.0010) [2023-12-26 15:51:40,832][105620] Updated weights for policy 1, policy_version 68438 (0.0007) [2023-12-26 15:51:40,898][105620] Updated weights for policy 1, policy_version 68448 (0.0005) [2023-12-26 15:51:40,959][105620] Updated weights for policy 1, policy_version 68458 (0.0005) [2023-12-26 15:51:41,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 34955264. Throughput: 0: 10001.3, 1: 9805.8. Samples: 34960984. Policy #0 lag: (min: 9.0, avg: 33.1, max: 41.0) [2023-12-26 15:51:41,063][104569] Avg episode reward: [(0, '9353.693'), (1, '9093.327')] [2023-12-26 15:51:41,474][105692] Updated weights for policy 0, policy_version 68054 (0.0007) [2023-12-26 15:51:41,537][105692] Updated weights for policy 0, policy_version 68064 (0.0007) [2023-12-26 15:51:41,593][105692] Updated weights for policy 0, policy_version 68074 (0.0006) [2023-12-26 15:51:41,641][105620] Updated weights for policy 1, policy_version 68468 (0.0009) [2023-12-26 15:51:41,711][105620] Updated weights for policy 1, policy_version 68478 (0.0008) [2023-12-26 15:51:41,771][105620] Updated weights for policy 1, policy_version 68488 (0.0010) [2023-12-26 15:51:42,162][105692] Updated weights for policy 0, policy_version 68084 (0.0008) [2023-12-26 15:51:42,226][105692] Updated weights for policy 0, policy_version 68094 (0.0009) [2023-12-26 15:51:42,293][105692] Updated weights for policy 0, policy_version 68104 (0.0009) [2023-12-26 15:51:42,604][105620] Updated weights for policy 1, policy_version 68499 (0.0009) [2023-12-26 15:51:42,664][105620] Updated weights for policy 1, policy_version 68509 (0.0008) [2023-12-26 15:51:42,721][105620] Updated weights for policy 1, policy_version 68519 (0.0009) [2023-12-26 15:51:43,072][105692] Updated weights for policy 0, policy_version 68114 (0.0010) [2023-12-26 15:51:43,133][105692] Updated weights for policy 0, policy_version 68124 (0.0009) [2023-12-26 15:51:43,191][105692] Updated weights for policy 0, policy_version 68134 (0.0009) [2023-12-26 15:51:43,240][105692] Updated weights for policy 0, policy_version 68144 (0.0009) [2023-12-26 15:51:43,465][105620] Updated weights for policy 1, policy_version 68529 (0.0009) [2023-12-26 15:51:43,524][105620] Updated weights for policy 1, policy_version 68539 (0.0011) [2023-12-26 15:51:43,570][105620] Updated weights for policy 1, policy_version 68549 (0.0010) [2023-12-26 15:51:43,639][105620] Updated weights for policy 1, policy_version 68559 (0.0011) [2023-12-26 15:51:43,833][105692] Updated weights for policy 0, policy_version 68154 (0.0006) [2023-12-26 15:51:43,903][105692] Updated weights for policy 0, policy_version 68164 (0.0006) [2023-12-26 15:51:43,966][105692] Updated weights for policy 0, policy_version 68174 (0.0011) [2023-12-26 15:51:44,269][105620] Updated weights for policy 1, policy_version 68569 (0.0010) [2023-12-26 15:51:44,326][105620] Updated weights for policy 1, policy_version 68579 (0.0010) [2023-12-26 15:51:44,378][105620] Updated weights for policy 1, policy_version 68589 (0.0010) [2023-12-26 15:51:44,597][105692] Updated weights for policy 0, policy_version 68184 (0.0010) [2023-12-26 15:51:44,653][105692] Updated weights for policy 0, policy_version 68194 (0.0010) [2023-12-26 15:51:44,707][105692] Updated weights for policy 0, policy_version 68204 (0.0010) [2023-12-26 15:51:45,125][105620] Updated weights for policy 1, policy_version 68599 (0.0009) [2023-12-26 15:51:45,184][105620] Updated weights for policy 1, policy_version 68609 (0.0011) [2023-12-26 15:51:45,233][105620] Updated weights for policy 1, policy_version 68619 (0.0010) [2023-12-26 15:51:45,443][105692] Updated weights for policy 0, policy_version 68214 (0.0010) [2023-12-26 15:51:45,495][105692] Updated weights for policy 0, policy_version 68224 (0.0009) [2023-12-26 15:51:45,549][105692] Updated weights for policy 0, policy_version 68234 (0.0005) [2023-12-26 15:51:45,828][105620] Updated weights for policy 1, policy_version 68629 (0.0008) [2023-12-26 15:51:45,883][105620] Updated weights for policy 1, policy_version 68639 (0.0005) [2023-12-26 15:51:45,943][105620] Updated weights for policy 1, policy_version 68649 (0.0005) [2023-12-26 15:51:46,062][104569] Fps is (10 sec: 19660.0, 60 sec: 19797.2, 300 sec: 19633.0). Total num frames: 35053568. Throughput: 0: 10006.8, 1: 9778.5. Samples: 35018500. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 15:51:46,064][104569] Avg episode reward: [(0, '9267.586'), (1, '8748.830')] [2023-12-26 15:51:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000068240_17473536.pth... [2023-12-26 15:51:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000068656_17580032.pth... [2023-12-26 15:51:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000067088_17178624.pth [2023-12-26 15:51:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000067472_17276928.pth [2023-12-26 15:51:46,152][105692] Updated weights for policy 0, policy_version 68244 (0.0007) [2023-12-26 15:51:46,210][105692] Updated weights for policy 0, policy_version 68255 (0.0010) [2023-12-26 15:51:46,267][105692] Updated weights for policy 0, policy_version 68266 (0.0007) [2023-12-26 15:51:46,475][105620] Updated weights for policy 1, policy_version 68659 (0.0006) [2023-12-26 15:51:46,545][105620] Updated weights for policy 1, policy_version 68669 (0.0005) [2023-12-26 15:51:46,614][105620] Updated weights for policy 1, policy_version 68679 (0.0005) [2023-12-26 15:51:47,090][105692] Updated weights for policy 0, policy_version 68276 (0.0007) [2023-12-26 15:51:47,145][105692] Updated weights for policy 0, policy_version 68286 (0.0007) [2023-12-26 15:51:47,192][105620] Updated weights for policy 1, policy_version 68689 (0.0009) [2023-12-26 15:51:47,198][105692] Updated weights for policy 0, policy_version 68296 (0.0005) [2023-12-26 15:51:47,241][105620] Updated weights for policy 1, policy_version 68699 (0.0008) [2023-12-26 15:51:47,295][105620] Updated weights for policy 1, policy_version 68711 (0.0009) [2023-12-26 15:51:47,820][105692] Updated weights for policy 0, policy_version 68306 (0.0006) [2023-12-26 15:51:47,874][105692] Updated weights for policy 0, policy_version 68316 (0.0009) [2023-12-26 15:51:47,936][105692] Updated weights for policy 0, policy_version 68326 (0.0010) [2023-12-26 15:51:47,990][105692] Updated weights for policy 0, policy_version 68336 (0.0013) [2023-12-26 15:51:48,055][105620] Updated weights for policy 1, policy_version 68721 (0.0006) [2023-12-26 15:51:48,114][105620] Updated weights for policy 1, policy_version 68731 (0.0008) [2023-12-26 15:51:48,172][105620] Updated weights for policy 1, policy_version 68741 (0.0008) [2023-12-26 15:51:48,228][105620] Updated weights for policy 1, policy_version 68751 (0.0009) [2023-12-26 15:51:48,750][105692] Updated weights for policy 0, policy_version 68346 (0.0009) [2023-12-26 15:51:48,810][105692] Updated weights for policy 0, policy_version 68356 (0.0009) [2023-12-26 15:51:48,870][105692] Updated weights for policy 0, policy_version 68366 (0.0009) [2023-12-26 15:51:49,015][105620] Updated weights for policy 1, policy_version 68761 (0.0008) [2023-12-26 15:51:49,072][105620] Updated weights for policy 1, policy_version 68771 (0.0009) [2023-12-26 15:51:49,128][105620] Updated weights for policy 1, policy_version 68781 (0.0006) [2023-12-26 15:51:49,655][105692] Updated weights for policy 0, policy_version 68376 (0.0008) [2023-12-26 15:51:49,706][105692] Updated weights for policy 0, policy_version 68386 (0.0008) [2023-12-26 15:51:49,762][105692] Updated weights for policy 0, policy_version 68396 (0.0008) [2023-12-26 15:51:49,853][105620] Updated weights for policy 1, policy_version 68791 (0.0008) [2023-12-26 15:51:49,924][105620] Updated weights for policy 1, policy_version 68801 (0.0009) [2023-12-26 15:51:49,989][105620] Updated weights for policy 1, policy_version 68811 (0.0010) [2023-12-26 15:51:50,556][105692] Updated weights for policy 0, policy_version 68406 (0.0008) [2023-12-26 15:51:50,601][105620] Updated weights for policy 1, policy_version 68821 (0.0009) [2023-12-26 15:51:50,626][105692] Updated weights for policy 0, policy_version 68416 (0.0008) [2023-12-26 15:51:50,668][105620] Updated weights for policy 1, policy_version 68831 (0.0008) [2023-12-26 15:51:50,692][105692] Updated weights for policy 0, policy_version 68426 (0.0009) [2023-12-26 15:51:50,724][105620] Updated weights for policy 1, policy_version 68841 (0.0008) [2023-12-26 15:51:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 35151872. Throughput: 0: 10010.1, 1: 9929.7. Samples: 35140308. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 15:51:51,063][104569] Avg episode reward: [(0, '9267.671'), (1, '8051.339')] [2023-12-26 15:51:51,420][105692] Updated weights for policy 0, policy_version 68436 (0.0008) [2023-12-26 15:51:51,476][105692] Updated weights for policy 0, policy_version 68446 (0.0007) [2023-12-26 15:51:51,478][105620] Updated weights for policy 1, policy_version 68851 (0.0009) [2023-12-26 15:51:51,529][105692] Updated weights for policy 0, policy_version 68456 (0.0006) [2023-12-26 15:51:51,537][105620] Updated weights for policy 1, policy_version 68861 (0.0011) [2023-12-26 15:51:51,593][105620] Updated weights for policy 1, policy_version 68871 (0.0010) [2023-12-26 15:51:52,342][105692] Updated weights for policy 0, policy_version 68466 (0.0008) [2023-12-26 15:51:52,364][105620] Updated weights for policy 1, policy_version 68881 (0.0010) [2023-12-26 15:51:52,414][105692] Updated weights for policy 0, policy_version 68476 (0.0006) [2023-12-26 15:51:52,431][105620] Updated weights for policy 1, policy_version 68891 (0.0008) [2023-12-26 15:51:52,469][105692] Updated weights for policy 0, policy_version 68486 (0.0008) [2023-12-26 15:51:52,495][105620] Updated weights for policy 1, policy_version 68901 (0.0008) [2023-12-26 15:51:52,530][105692] Updated weights for policy 0, policy_version 68496 (0.0006) [2023-12-26 15:51:52,556][105620] Updated weights for policy 1, policy_version 68911 (0.0007) [2023-12-26 15:51:53,148][105692] Updated weights for policy 0, policy_version 68506 (0.0005) [2023-12-26 15:51:53,191][105692] Updated weights for policy 0, policy_version 68516 (0.0005) [2023-12-26 15:51:53,237][105692] Updated weights for policy 0, policy_version 68526 (0.0005) [2023-12-26 15:51:53,352][105620] Updated weights for policy 1, policy_version 68921 (0.0006) [2023-12-26 15:51:53,404][105620] Updated weights for policy 1, policy_version 68931 (0.0009) [2023-12-26 15:51:53,462][105620] Updated weights for policy 1, policy_version 68941 (0.0009) [2023-12-26 15:51:53,836][105692] Updated weights for policy 0, policy_version 68536 (0.0005) [2023-12-26 15:51:53,897][105692] Updated weights for policy 0, policy_version 68546 (0.0005) [2023-12-26 15:51:53,954][105692] Updated weights for policy 0, policy_version 68556 (0.0005) [2023-12-26 15:51:54,268][105620] Updated weights for policy 1, policy_version 68951 (0.0009) [2023-12-26 15:51:54,315][105620] Updated weights for policy 1, policy_version 68961 (0.0008) [2023-12-26 15:51:54,369][105620] Updated weights for policy 1, policy_version 68971 (0.0008) [2023-12-26 15:51:54,564][105692] Updated weights for policy 0, policy_version 68566 (0.0006) [2023-12-26 15:51:54,621][105692] Updated weights for policy 0, policy_version 68576 (0.0008) [2023-12-26 15:51:54,676][105692] Updated weights for policy 0, policy_version 68586 (0.0009) [2023-12-26 15:51:55,065][105620] Updated weights for policy 1, policy_version 68981 (0.0007) [2023-12-26 15:51:55,122][105620] Updated weights for policy 1, policy_version 68991 (0.0005) [2023-12-26 15:51:55,180][105620] Updated weights for policy 1, policy_version 69001 (0.0006) [2023-12-26 15:51:55,419][105692] Updated weights for policy 0, policy_version 68596 (0.0010) [2023-12-26 15:51:55,489][105692] Updated weights for policy 0, policy_version 68606 (0.0009) [2023-12-26 15:51:55,552][105692] Updated weights for policy 0, policy_version 68616 (0.0010) [2023-12-26 15:51:55,735][105620] Updated weights for policy 1, policy_version 69011 (0.0007) [2023-12-26 15:51:55,789][105620] Updated weights for policy 1, policy_version 69021 (0.0009) [2023-12-26 15:51:55,847][105620] Updated weights for policy 1, policy_version 69031 (0.0007) [2023-12-26 15:51:56,062][104569] Fps is (10 sec: 19661.7, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 35250176. Throughput: 0: 9970.8, 1: 9910.5. Samples: 35257044. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 15:51:56,063][104569] Avg episode reward: [(0, '9267.838'), (1, '7864.288')] [2023-12-26 15:51:56,269][105692] Updated weights for policy 0, policy_version 68626 (0.0009) [2023-12-26 15:51:56,320][105692] Updated weights for policy 0, policy_version 68636 (0.0005) [2023-12-26 15:51:56,375][105692] Updated weights for policy 0, policy_version 68646 (0.0006) [2023-12-26 15:51:56,424][105692] Updated weights for policy 0, policy_version 68656 (0.0006) [2023-12-26 15:51:56,689][105620] Updated weights for policy 1, policy_version 69041 (0.0007) [2023-12-26 15:51:56,734][105620] Updated weights for policy 1, policy_version 69051 (0.0008) [2023-12-26 15:51:56,782][105620] Updated weights for policy 1, policy_version 69061 (0.0008) [2023-12-26 15:51:56,831][105620] Updated weights for policy 1, policy_version 69071 (0.0008) [2023-12-26 15:51:57,001][105692] Updated weights for policy 0, policy_version 68666 (0.0005) [2023-12-26 15:51:57,053][105692] Updated weights for policy 0, policy_version 68676 (0.0005) [2023-12-26 15:51:57,098][105692] Updated weights for policy 0, policy_version 68686 (0.0005) [2023-12-26 15:51:57,691][105620] Updated weights for policy 1, policy_version 69081 (0.0009) [2023-12-26 15:51:57,718][105692] Updated weights for policy 0, policy_version 68696 (0.0006) [2023-12-26 15:51:57,744][105620] Updated weights for policy 1, policy_version 69091 (0.0006) [2023-12-26 15:51:57,770][105692] Updated weights for policy 0, policy_version 68706 (0.0007) [2023-12-26 15:51:57,799][105620] Updated weights for policy 1, policy_version 69101 (0.0008) [2023-12-26 15:51:57,828][105692] Updated weights for policy 0, policy_version 68716 (0.0005) [2023-12-26 15:51:58,590][105692] Updated weights for policy 0, policy_version 68726 (0.0009) [2023-12-26 15:51:58,591][105620] Updated weights for policy 1, policy_version 69111 (0.0008) [2023-12-26 15:51:58,656][105620] Updated weights for policy 1, policy_version 69121 (0.0009) [2023-12-26 15:51:58,663][105692] Updated weights for policy 0, policy_version 68736 (0.0009) [2023-12-26 15:51:58,718][105620] Updated weights for policy 1, policy_version 69131 (0.0007) [2023-12-26 15:51:58,729][105692] Updated weights for policy 0, policy_version 68746 (0.0008) [2023-12-26 15:51:59,438][105620] Updated weights for policy 1, policy_version 69141 (0.0008) [2023-12-26 15:51:59,474][105692] Updated weights for policy 0, policy_version 68756 (0.0009) [2023-12-26 15:51:59,489][105620] Updated weights for policy 1, policy_version 69151 (0.0007) [2023-12-26 15:51:59,531][105692] Updated weights for policy 0, policy_version 68766 (0.0009) [2023-12-26 15:51:59,545][105620] Updated weights for policy 1, policy_version 69161 (0.0006) [2023-12-26 15:51:59,592][105692] Updated weights for policy 0, policy_version 68776 (0.0009) [2023-12-26 15:52:00,215][105620] Updated weights for policy 1, policy_version 69171 (0.0007) [2023-12-26 15:52:00,276][105620] Updated weights for policy 1, policy_version 69181 (0.0009) [2023-12-26 15:52:00,333][105620] Updated weights for policy 1, policy_version 69191 (0.0009) [2023-12-26 15:52:00,400][105692] Updated weights for policy 0, policy_version 68786 (0.0009) [2023-12-26 15:52:00,454][105692] Updated weights for policy 0, policy_version 68796 (0.0008) [2023-12-26 15:52:00,522][105692] Updated weights for policy 0, policy_version 68806 (0.0009) [2023-12-26 15:52:00,582][105692] Updated weights for policy 0, policy_version 68816 (0.0009) [2023-12-26 15:52:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 35340288. Throughput: 0: 10001.1, 1: 9838.6. Samples: 35315140. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 15:52:01,063][104569] Avg episode reward: [(0, '9185.385'), (1, '8563.557')] [2023-12-26 15:52:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000068816_17620992.pth... [2023-12-26 15:52:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000067664_17326080.pth [2023-12-26 15:52:01,084][105620] Updated weights for policy 1, policy_version 69201 (0.0009) [2023-12-26 15:52:01,149][105620] Updated weights for policy 1, policy_version 69211 (0.0009) [2023-12-26 15:52:01,211][105620] Updated weights for policy 1, policy_version 69221 (0.0007) [2023-12-26 15:52:01,282][105620] Updated weights for policy 1, policy_version 69231 (0.0008) [2023-12-26 15:52:01,285][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000069232_17727488.pth... [2023-12-26 15:52:01,290][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000068080_17432576.pth [2023-12-26 15:52:01,338][105692] Updated weights for policy 0, policy_version 68826 (0.0006) [2023-12-26 15:52:01,399][105692] Updated weights for policy 0, policy_version 68836 (0.0008) [2023-12-26 15:52:01,446][105692] Updated weights for policy 0, policy_version 68846 (0.0007) [2023-12-26 15:52:01,966][105620] Updated weights for policy 1, policy_version 69241 (0.0006) [2023-12-26 15:52:02,017][105620] Updated weights for policy 1, policy_version 69251 (0.0005) [2023-12-26 15:52:02,086][105620] Updated weights for policy 1, policy_version 69261 (0.0006) [2023-12-26 15:52:02,137][105692] Updated weights for policy 0, policy_version 68856 (0.0008) [2023-12-26 15:52:02,194][105692] Updated weights for policy 0, policy_version 68866 (0.0010) [2023-12-26 15:52:02,249][105692] Updated weights for policy 0, policy_version 68876 (0.0009) [2023-12-26 15:52:02,825][105692] Updated weights for policy 0, policy_version 68886 (0.0007) [2023-12-26 15:52:02,840][105620] Updated weights for policy 1, policy_version 69271 (0.0007) [2023-12-26 15:52:02,875][105692] Updated weights for policy 0, policy_version 68896 (0.0006) [2023-12-26 15:52:02,893][105620] Updated weights for policy 1, policy_version 69281 (0.0007) [2023-12-26 15:52:02,921][105692] Updated weights for policy 0, policy_version 68906 (0.0006) [2023-12-26 15:52:02,944][105620] Updated weights for policy 1, policy_version 69291 (0.0008) [2023-12-26 15:52:03,569][105692] Updated weights for policy 0, policy_version 68916 (0.0005) [2023-12-26 15:52:03,571][105620] Updated weights for policy 1, policy_version 69301 (0.0007) [2023-12-26 15:52:03,613][105620] Updated weights for policy 1, policy_version 69311 (0.0005) [2023-12-26 15:52:03,620][105692] Updated weights for policy 0, policy_version 68926 (0.0005) [2023-12-26 15:52:03,663][105620] Updated weights for policy 1, policy_version 69321 (0.0006) [2023-12-26 15:52:03,666][105692] Updated weights for policy 0, policy_version 68936 (0.0005) [2023-12-26 15:52:04,334][105692] Updated weights for policy 0, policy_version 68946 (0.0006) [2023-12-26 15:52:04,353][105620] Updated weights for policy 1, policy_version 69331 (0.0006) [2023-12-26 15:52:04,383][105692] Updated weights for policy 0, policy_version 68956 (0.0009) [2023-12-26 15:52:04,419][105620] Updated weights for policy 1, policy_version 69341 (0.0005) [2023-12-26 15:52:04,431][105692] Updated weights for policy 0, policy_version 68966 (0.0008) [2023-12-26 15:52:04,481][105620] Updated weights for policy 1, policy_version 69351 (0.0006) [2023-12-26 15:52:04,487][105692] Updated weights for policy 0, policy_version 68976 (0.0006) [2023-12-26 15:52:05,167][105620] Updated weights for policy 1, policy_version 69361 (0.0006) [2023-12-26 15:52:05,222][105620] Updated weights for policy 1, policy_version 69371 (0.0010) [2023-12-26 15:52:05,239][105692] Updated weights for policy 0, policy_version 68986 (0.0005) [2023-12-26 15:52:05,276][105620] Updated weights for policy 1, policy_version 69381 (0.0010) [2023-12-26 15:52:05,299][105692] Updated weights for policy 0, policy_version 68996 (0.0006) [2023-12-26 15:52:05,322][105620] Updated weights for policy 1, policy_version 69391 (0.0010) [2023-12-26 15:52:05,345][105692] Updated weights for policy 0, policy_version 69006 (0.0006) [2023-12-26 15:52:06,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 35438592. Throughput: 0: 9973.0, 1: 9918.6. Samples: 35434528. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 15:52:06,062][104569] Avg episode reward: [(0, '8402.846'), (1, '9023.295')] [2023-12-26 15:52:06,080][105620] Updated weights for policy 1, policy_version 69401 (0.0010) [2023-12-26 15:52:06,094][105692] Updated weights for policy 0, policy_version 69016 (0.0008) [2023-12-26 15:52:06,145][105620] Updated weights for policy 1, policy_version 69411 (0.0010) [2023-12-26 15:52:06,160][105692] Updated weights for policy 0, policy_version 69026 (0.0008) [2023-12-26 15:52:06,211][105620] Updated weights for policy 1, policy_version 69421 (0.0008) [2023-12-26 15:52:06,221][105692] Updated weights for policy 0, policy_version 69036 (0.0008) [2023-12-26 15:52:06,895][105692] Updated weights for policy 0, policy_version 69046 (0.0006) [2023-12-26 15:52:06,924][105620] Updated weights for policy 1, policy_version 69431 (0.0007) [2023-12-26 15:52:06,960][105692] Updated weights for policy 0, policy_version 69056 (0.0007) [2023-12-26 15:52:06,978][105620] Updated weights for policy 1, policy_version 69441 (0.0008) [2023-12-26 15:52:07,018][105692] Updated weights for policy 0, policy_version 69066 (0.0006) [2023-12-26 15:52:07,033][105620] Updated weights for policy 1, policy_version 69451 (0.0007) [2023-12-26 15:52:07,785][105692] Updated weights for policy 0, policy_version 69076 (0.0009) [2023-12-26 15:52:07,832][105692] Updated weights for policy 0, policy_version 69086 (0.0010) [2023-12-26 15:52:07,838][105620] Updated weights for policy 1, policy_version 69461 (0.0009) [2023-12-26 15:52:07,882][105620] Updated weights for policy 1, policy_version 69471 (0.0005) [2023-12-26 15:52:07,884][105692] Updated weights for policy 0, policy_version 69096 (0.0010) [2023-12-26 15:52:07,929][105620] Updated weights for policy 1, policy_version 69481 (0.0007) [2023-12-26 15:52:08,649][105692] Updated weights for policy 0, policy_version 69106 (0.0010) [2023-12-26 15:52:08,697][105692] Updated weights for policy 0, policy_version 69116 (0.0010) [2023-12-26 15:52:08,705][105620] Updated weights for policy 1, policy_version 69491 (0.0009) [2023-12-26 15:52:08,750][105692] Updated weights for policy 0, policy_version 69126 (0.0010) [2023-12-26 15:52:08,761][105620] Updated weights for policy 1, policy_version 69501 (0.0011) [2023-12-26 15:52:08,802][105692] Updated weights for policy 0, policy_version 69136 (0.0011) [2023-12-26 15:52:08,811][105620] Updated weights for policy 1, policy_version 69511 (0.0011) [2023-12-26 15:52:09,556][105620] Updated weights for policy 1, policy_version 69521 (0.0011) [2023-12-26 15:52:09,558][105692] Updated weights for policy 0, policy_version 69146 (0.0011) [2023-12-26 15:52:09,613][105620] Updated weights for policy 1, policy_version 69531 (0.0011) [2023-12-26 15:52:09,617][105692] Updated weights for policy 0, policy_version 69156 (0.0011) [2023-12-26 15:52:09,673][105620] Updated weights for policy 1, policy_version 69541 (0.0011) [2023-12-26 15:52:09,677][105692] Updated weights for policy 0, policy_version 69166 (0.0011) [2023-12-26 15:52:09,736][105620] Updated weights for policy 1, policy_version 69551 (0.0011) [2023-12-26 15:52:10,445][105620] Updated weights for policy 1, policy_version 69561 (0.0009) [2023-12-26 15:52:10,511][105620] Updated weights for policy 1, policy_version 69571 (0.0009) [2023-12-26 15:52:10,516][105692] Updated weights for policy 0, policy_version 69176 (0.0007) [2023-12-26 15:52:10,570][105620] Updated weights for policy 1, policy_version 69581 (0.0009) [2023-12-26 15:52:10,576][105692] Updated weights for policy 0, policy_version 69186 (0.0007) [2023-12-26 15:52:10,631][105692] Updated weights for policy 0, policy_version 69196 (0.0008) [2023-12-26 15:52:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 35536896. Throughput: 0: 9935.4, 1: 9870.5. Samples: 35546736. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 15:52:11,063][104569] Avg episode reward: [(0, '8058.713'), (1, '9199.718')] [2023-12-26 15:52:11,281][105620] Updated weights for policy 1, policy_version 69591 (0.0008) [2023-12-26 15:52:11,350][105620] Updated weights for policy 1, policy_version 69601 (0.0008) [2023-12-26 15:52:11,377][105692] Updated weights for policy 0, policy_version 69206 (0.0007) [2023-12-26 15:52:11,420][105620] Updated weights for policy 1, policy_version 69611 (0.0007) [2023-12-26 15:52:11,440][105692] Updated weights for policy 0, policy_version 69216 (0.0008) [2023-12-26 15:52:11,498][105692] Updated weights for policy 0, policy_version 69226 (0.0005) [2023-12-26 15:52:12,155][105692] Updated weights for policy 0, policy_version 69236 (0.0008) [2023-12-26 15:52:12,213][105692] Updated weights for policy 0, policy_version 69246 (0.0009) [2023-12-26 15:52:12,243][105620] Updated weights for policy 1, policy_version 69621 (0.0007) [2023-12-26 15:52:12,273][105692] Updated weights for policy 0, policy_version 69256 (0.0007) [2023-12-26 15:52:12,310][105620] Updated weights for policy 1, policy_version 69631 (0.0009) [2023-12-26 15:52:12,374][105620] Updated weights for policy 1, policy_version 69641 (0.0008) [2023-12-26 15:52:13,058][105620] Updated weights for policy 1, policy_version 69651 (0.0007) [2023-12-26 15:52:13,097][105692] Updated weights for policy 0, policy_version 69266 (0.0008) [2023-12-26 15:52:13,117][105620] Updated weights for policy 1, policy_version 69661 (0.0008) [2023-12-26 15:52:13,156][105692] Updated weights for policy 0, policy_version 69276 (0.0009) [2023-12-26 15:52:13,175][105620] Updated weights for policy 1, policy_version 69671 (0.0005) [2023-12-26 15:52:13,206][105692] Updated weights for policy 0, policy_version 69286 (0.0009) [2023-12-26 15:52:13,256][105692] Updated weights for policy 0, policy_version 69296 (0.0009) [2023-12-26 15:52:13,745][105620] Updated weights for policy 1, policy_version 69681 (0.0006) [2023-12-26 15:52:13,807][105620] Updated weights for policy 1, policy_version 69691 (0.0008) [2023-12-26 15:52:13,866][105620] Updated weights for policy 1, policy_version 69701 (0.0009) [2023-12-26 15:52:13,925][105620] Updated weights for policy 1, policy_version 69711 (0.0008) [2023-12-26 15:52:14,003][105692] Updated weights for policy 0, policy_version 69306 (0.0010) [2023-12-26 15:52:14,054][105692] Updated weights for policy 0, policy_version 69316 (0.0010) [2023-12-26 15:52:14,110][105692] Updated weights for policy 0, policy_version 69326 (0.0010) [2023-12-26 15:52:14,696][105620] Updated weights for policy 1, policy_version 69721 (0.0010) [2023-12-26 15:52:14,755][105620] Updated weights for policy 1, policy_version 69731 (0.0010) [2023-12-26 15:52:14,816][105692] Updated weights for policy 0, policy_version 69336 (0.0009) [2023-12-26 15:52:14,816][105620] Updated weights for policy 1, policy_version 69741 (0.0011) [2023-12-26 15:52:14,874][105692] Updated weights for policy 0, policy_version 69346 (0.0008) [2023-12-26 15:52:14,919][105692] Updated weights for policy 0, policy_version 69356 (0.0008) [2023-12-26 15:52:15,584][105620] Updated weights for policy 1, policy_version 69751 (0.0011) [2023-12-26 15:52:15,632][105692] Updated weights for policy 0, policy_version 69366 (0.0006) [2023-12-26 15:52:15,647][105620] Updated weights for policy 1, policy_version 69761 (0.0010) [2023-12-26 15:52:15,698][105692] Updated weights for policy 0, policy_version 69376 (0.0009) [2023-12-26 15:52:15,707][105620] Updated weights for policy 1, policy_version 69771 (0.0011) [2023-12-26 15:52:15,757][105692] Updated weights for policy 0, policy_version 69386 (0.0011) [2023-12-26 15:52:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 35635200. Throughput: 0: 9837.4, 1: 9749.9. Samples: 35604068. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 15:52:16,063][104569] Avg episode reward: [(0, '6241.442'), (1, '9184.539')] [2023-12-26 15:52:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000069392_17768448.pth... [2023-12-26 15:52:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000069776_17866752.pth... [2023-12-26 15:52:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000068240_17473536.pth [2023-12-26 15:52:16,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000068656_17580032.pth [2023-12-26 15:52:16,403][105620] Updated weights for policy 1, policy_version 69781 (0.0008) [2023-12-26 15:52:16,404][105692] Updated weights for policy 0, policy_version 69396 (0.0010) [2023-12-26 15:52:16,463][105692] Updated weights for policy 0, policy_version 69406 (0.0007) [2023-12-26 15:52:16,465][105620] Updated weights for policy 1, policy_version 69791 (0.0009) [2023-12-26 15:52:16,520][105620] Updated weights for policy 1, policy_version 69801 (0.0009) [2023-12-26 15:52:16,521][105692] Updated weights for policy 0, policy_version 69416 (0.0007) [2023-12-26 15:52:17,077][105620] Updated weights for policy 1, policy_version 69811 (0.0009) [2023-12-26 15:52:17,103][105692] Updated weights for policy 0, policy_version 69426 (0.0007) [2023-12-26 15:52:17,137][105620] Updated weights for policy 1, policy_version 69821 (0.0005) [2023-12-26 15:52:17,163][105692] Updated weights for policy 0, policy_version 69436 (0.0005) [2023-12-26 15:52:17,194][105620] Updated weights for policy 1, policy_version 69831 (0.0006) [2023-12-26 15:52:17,219][105692] Updated weights for policy 0, policy_version 69447 (0.0008) [2023-12-26 15:52:17,807][105620] Updated weights for policy 1, policy_version 69841 (0.0006) [2023-12-26 15:52:17,855][105620] Updated weights for policy 1, policy_version 69851 (0.0010) [2023-12-26 15:52:17,883][105692] Updated weights for policy 0, policy_version 69457 (0.0009) [2023-12-26 15:52:17,910][105620] Updated weights for policy 1, policy_version 69861 (0.0010) [2023-12-26 15:52:17,938][105692] Updated weights for policy 0, policy_version 69467 (0.0010) [2023-12-26 15:52:17,955][105620] Updated weights for policy 1, policy_version 69871 (0.0010) [2023-12-26 15:52:17,997][105692] Updated weights for policy 0, policy_version 69477 (0.0010) [2023-12-26 15:52:18,055][105692] Updated weights for policy 0, policy_version 69487 (0.0010) [2023-12-26 15:52:18,644][105692] Updated weights for policy 0, policy_version 69497 (0.0009) [2023-12-26 15:52:18,707][105692] Updated weights for policy 0, policy_version 69507 (0.0011) [2023-12-26 15:52:18,710][105620] Updated weights for policy 1, policy_version 69881 (0.0011) [2023-12-26 15:52:18,765][105620] Updated weights for policy 1, policy_version 69891 (0.0011) [2023-12-26 15:52:18,767][105692] Updated weights for policy 0, policy_version 69517 (0.0009) [2023-12-26 15:52:18,835][105620] Updated weights for policy 1, policy_version 69901 (0.0011) [2023-12-26 15:52:19,397][105692] Updated weights for policy 0, policy_version 69527 (0.0009) [2023-12-26 15:52:19,452][105692] Updated weights for policy 0, policy_version 69537 (0.0007) [2023-12-26 15:52:19,521][105692] Updated weights for policy 0, policy_version 69547 (0.0007) [2023-12-26 15:52:19,599][105620] Updated weights for policy 1, policy_version 69911 (0.0009) [2023-12-26 15:52:19,646][105620] Updated weights for policy 1, policy_version 69921 (0.0008) [2023-12-26 15:52:19,705][105620] Updated weights for policy 1, policy_version 69931 (0.0008) [2023-12-26 15:52:20,242][105692] Updated weights for policy 0, policy_version 69557 (0.0009) [2023-12-26 15:52:20,307][105692] Updated weights for policy 0, policy_version 69567 (0.0010) [2023-12-26 15:52:20,370][105692] Updated weights for policy 0, policy_version 69577 (0.0011) [2023-12-26 15:52:20,394][105620] Updated weights for policy 1, policy_version 69941 (0.0006) [2023-12-26 15:52:20,462][105620] Updated weights for policy 1, policy_version 69951 (0.0006) [2023-12-26 15:52:20,516][105620] Updated weights for policy 1, policy_version 69961 (0.0011) [2023-12-26 15:52:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 35733504. Throughput: 0: 9895.6, 1: 9750.3. Samples: 35727028. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 15:52:21,063][104569] Avg episode reward: [(0, '6652.505'), (1, '8843.378')] [2023-12-26 15:52:21,124][105692] Updated weights for policy 0, policy_version 69587 (0.0011) [2023-12-26 15:52:21,189][105692] Updated weights for policy 0, policy_version 69597 (0.0011) [2023-12-26 15:52:21,247][105692] Updated weights for policy 0, policy_version 69607 (0.0010) [2023-12-26 15:52:21,293][105620] Updated weights for policy 1, policy_version 69971 (0.0011) [2023-12-26 15:52:21,363][105620] Updated weights for policy 1, policy_version 69981 (0.0011) [2023-12-26 15:52:21,432][105620] Updated weights for policy 1, policy_version 69991 (0.0011) [2023-12-26 15:52:22,084][105692] Updated weights for policy 0, policy_version 69617 (0.0010) [2023-12-26 15:52:22,137][105692] Updated weights for policy 0, policy_version 69627 (0.0008) [2023-12-26 15:52:22,187][105692] Updated weights for policy 0, policy_version 69637 (0.0008) [2023-12-26 15:52:22,206][105620] Updated weights for policy 1, policy_version 70001 (0.0009) [2023-12-26 15:52:22,233][105692] Updated weights for policy 0, policy_version 69647 (0.0007) [2023-12-26 15:52:22,261][105620] Updated weights for policy 1, policy_version 70011 (0.0010) [2023-12-26 15:52:22,317][105620] Updated weights for policy 1, policy_version 70021 (0.0009) [2023-12-26 15:52:22,379][105620] Updated weights for policy 1, policy_version 70031 (0.0010) [2023-12-26 15:52:22,980][105692] Updated weights for policy 0, policy_version 69657 (0.0010) [2023-12-26 15:52:23,043][105692] Updated weights for policy 0, policy_version 69667 (0.0010) [2023-12-26 15:52:23,056][105620] Updated weights for policy 1, policy_version 70041 (0.0006) [2023-12-26 15:52:23,110][105692] Updated weights for policy 0, policy_version 69677 (0.0008) [2023-12-26 15:52:23,120][105620] Updated weights for policy 1, policy_version 70051 (0.0006) [2023-12-26 15:52:23,186][105620] Updated weights for policy 1, policy_version 70061 (0.0006) [2023-12-26 15:52:23,658][105692] Updated weights for policy 0, policy_version 69687 (0.0005) [2023-12-26 15:52:23,718][105692] Updated weights for policy 0, policy_version 69697 (0.0005) [2023-12-26 15:52:23,773][105692] Updated weights for policy 0, policy_version 69707 (0.0005) [2023-12-26 15:52:23,852][105620] Updated weights for policy 1, policy_version 70071 (0.0009) [2023-12-26 15:52:23,907][105620] Updated weights for policy 1, policy_version 70081 (0.0008) [2023-12-26 15:52:23,962][105620] Updated weights for policy 1, policy_version 70091 (0.0008) [2023-12-26 15:52:24,415][105692] Updated weights for policy 0, policy_version 69717 (0.0008) [2023-12-26 15:52:24,485][105692] Updated weights for policy 0, policy_version 69727 (0.0009) [2023-12-26 15:52:24,553][105692] Updated weights for policy 0, policy_version 69737 (0.0008) [2023-12-26 15:52:24,686][105620] Updated weights for policy 1, policy_version 70101 (0.0009) [2023-12-26 15:52:24,743][105620] Updated weights for policy 1, policy_version 70111 (0.0009) [2023-12-26 15:52:24,797][105620] Updated weights for policy 1, policy_version 70122 (0.0010) [2023-12-26 15:52:25,099][105692] Updated weights for policy 0, policy_version 69747 (0.0010) [2023-12-26 15:52:25,154][105692] Updated weights for policy 0, policy_version 69757 (0.0010) [2023-12-26 15:52:25,216][105692] Updated weights for policy 0, policy_version 69767 (0.0010) [2023-12-26 15:52:25,647][105620] Updated weights for policy 1, policy_version 70133 (0.0010) [2023-12-26 15:52:25,714][105620] Updated weights for policy 1, policy_version 70143 (0.0011) [2023-12-26 15:52:25,782][105620] Updated weights for policy 1, policy_version 70153 (0.0011) [2023-12-26 15:52:25,903][105692] Updated weights for policy 0, policy_version 69777 (0.0010) [2023-12-26 15:52:25,963][105692] Updated weights for policy 0, policy_version 69787 (0.0008) [2023-12-26 15:52:26,014][105692] Updated weights for policy 0, policy_version 69797 (0.0008) [2023-12-26 15:52:26,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 35831808. Throughput: 0: 9953.9, 1: 9672.1. Samples: 35844152. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 15:52:26,062][104569] Avg episode reward: [(0, '8036.905'), (1, '8932.073')] [2023-12-26 15:52:26,073][105692] Updated weights for policy 0, policy_version 69807 (0.0008) [2023-12-26 15:52:26,520][105620] Updated weights for policy 1, policy_version 70163 (0.0011) [2023-12-26 15:52:26,576][105620] Updated weights for policy 1, policy_version 70173 (0.0010) [2023-12-26 15:52:26,633][105620] Updated weights for policy 1, policy_version 70183 (0.0010) [2023-12-26 15:52:26,774][105692] Updated weights for policy 0, policy_version 69817 (0.0006) [2023-12-26 15:52:26,834][105692] Updated weights for policy 0, policy_version 69827 (0.0005) [2023-12-26 15:52:26,886][105692] Updated weights for policy 0, policy_version 69837 (0.0005) [2023-12-26 15:52:27,439][105620] Updated weights for policy 1, policy_version 70193 (0.0010) [2023-12-26 15:52:27,441][105692] Updated weights for policy 0, policy_version 69847 (0.0005) [2023-12-26 15:52:27,487][105692] Updated weights for policy 0, policy_version 69857 (0.0005) [2023-12-26 15:52:27,493][105620] Updated weights for policy 1, policy_version 70203 (0.0008) [2023-12-26 15:52:27,534][105692] Updated weights for policy 0, policy_version 69867 (0.0005) [2023-12-26 15:52:27,541][105620] Updated weights for policy 1, policy_version 70213 (0.0005) [2023-12-26 15:52:27,595][105620] Updated weights for policy 1, policy_version 70223 (0.0008) [2023-12-26 15:52:28,054][105692] Updated weights for policy 0, policy_version 69877 (0.0005) [2023-12-26 15:52:28,105][105692] Updated weights for policy 0, policy_version 69887 (0.0005) [2023-12-26 15:52:28,161][105692] Updated weights for policy 0, policy_version 69897 (0.0006) [2023-12-26 15:52:28,198][105620] Updated weights for policy 1, policy_version 70233 (0.0006) [2023-12-26 15:52:28,265][105620] Updated weights for policy 1, policy_version 70243 (0.0008) [2023-12-26 15:52:28,329][105620] Updated weights for policy 1, policy_version 70253 (0.0009) [2023-12-26 15:52:28,717][105692] Updated weights for policy 0, policy_version 69907 (0.0006) [2023-12-26 15:52:28,771][105692] Updated weights for policy 0, policy_version 69917 (0.0009) [2023-12-26 15:52:28,818][105692] Updated weights for policy 0, policy_version 69927 (0.0009) [2023-12-26 15:52:29,035][105620] Updated weights for policy 1, policy_version 70263 (0.0006) [2023-12-26 15:52:29,085][105620] Updated weights for policy 1, policy_version 70273 (0.0005) [2023-12-26 15:52:29,142][105620] Updated weights for policy 1, policy_version 70283 (0.0006) [2023-12-26 15:52:29,635][105692] Updated weights for policy 0, policy_version 69937 (0.0009) [2023-12-26 15:52:29,697][105692] Updated weights for policy 0, policy_version 69947 (0.0007) [2023-12-26 15:52:29,765][105692] Updated weights for policy 0, policy_version 69957 (0.0007) [2023-12-26 15:52:29,835][105692] Updated weights for policy 0, policy_version 69967 (0.0006) [2023-12-26 15:52:29,863][105620] Updated weights for policy 1, policy_version 70293 (0.0008) [2023-12-26 15:52:29,919][105620] Updated weights for policy 1, policy_version 70303 (0.0010) [2023-12-26 15:52:29,982][105620] Updated weights for policy 1, policy_version 70313 (0.0009) [2023-12-26 15:52:30,401][105692] Updated weights for policy 0, policy_version 69977 (0.0006) [2023-12-26 15:52:30,445][105692] Updated weights for policy 0, policy_version 69987 (0.0005) [2023-12-26 15:52:30,491][105692] Updated weights for policy 0, policy_version 69997 (0.0005) [2023-12-26 15:52:30,819][105620] Updated weights for policy 1, policy_version 70323 (0.0011) [2023-12-26 15:52:30,865][105620] Updated weights for policy 1, policy_version 70333 (0.0008) [2023-12-26 15:52:30,915][105620] Updated weights for policy 1, policy_version 70344 (0.0010) [2023-12-26 15:52:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19605.2). Total num frames: 35938304. Throughput: 0: 10044.5, 1: 9708.1. Samples: 35907356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:52:31,063][104569] Avg episode reward: [(0, '8567.859'), (1, '9020.832')] [2023-12-26 15:52:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000070352_18014208.pth... [2023-12-26 15:52:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000069232_17727488.pth [2023-12-26 15:52:31,088][105692] Updated weights for policy 0, policy_version 70007 (0.0007) [2023-12-26 15:52:31,160][105692] Updated weights for policy 0, policy_version 70017 (0.0008) [2023-12-26 15:52:31,224][105692] Updated weights for policy 0, policy_version 70027 (0.0005) [2023-12-26 15:52:31,254][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000070032_17932288.pth... [2023-12-26 15:52:31,258][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000068816_17620992.pth [2023-12-26 15:52:31,721][105620] Updated weights for policy 1, policy_version 70355 (0.0010) [2023-12-26 15:52:31,787][105620] Updated weights for policy 1, policy_version 70365 (0.0010) [2023-12-26 15:52:31,844][105620] Updated weights for policy 1, policy_version 70375 (0.0007) [2023-12-26 15:52:31,858][105692] Updated weights for policy 0, policy_version 70037 (0.0007) [2023-12-26 15:52:31,914][105692] Updated weights for policy 0, policy_version 70047 (0.0009) [2023-12-26 15:52:31,978][105692] Updated weights for policy 0, policy_version 70057 (0.0009) [2023-12-26 15:52:32,612][105692] Updated weights for policy 0, policy_version 70067 (0.0007) [2023-12-26 15:52:32,661][105620] Updated weights for policy 1, policy_version 70385 (0.0011) [2023-12-26 15:52:32,673][105692] Updated weights for policy 0, policy_version 70077 (0.0006) [2023-12-26 15:52:32,725][105620] Updated weights for policy 1, policy_version 70395 (0.0008) [2023-12-26 15:52:32,735][105692] Updated weights for policy 0, policy_version 70087 (0.0006) [2023-12-26 15:52:32,774][105620] Updated weights for policy 1, policy_version 70405 (0.0009) [2023-12-26 15:52:32,834][105620] Updated weights for policy 1, policy_version 70415 (0.0008) [2023-12-26 15:52:33,354][105692] Updated weights for policy 0, policy_version 70097 (0.0008) [2023-12-26 15:52:33,414][105692] Updated weights for policy 0, policy_version 70107 (0.0008) [2023-12-26 15:52:33,468][105692] Updated weights for policy 0, policy_version 70117 (0.0009) [2023-12-26 15:52:33,517][105692] Updated weights for policy 0, policy_version 70127 (0.0008) [2023-12-26 15:52:33,626][105620] Updated weights for policy 1, policy_version 70425 (0.0006) [2023-12-26 15:52:33,690][105620] Updated weights for policy 1, policy_version 70435 (0.0005) [2023-12-26 15:52:33,760][105620] Updated weights for policy 1, policy_version 70445 (0.0005) [2023-12-26 15:52:34,301][105620] Updated weights for policy 1, policy_version 70455 (0.0006) [2023-12-26 15:52:34,307][105692] Updated weights for policy 0, policy_version 70137 (0.0009) [2023-12-26 15:52:34,359][105620] Updated weights for policy 1, policy_version 70465 (0.0007) [2023-12-26 15:52:34,366][105692] Updated weights for policy 0, policy_version 70147 (0.0007) [2023-12-26 15:52:34,420][105620] Updated weights for policy 1, policy_version 70475 (0.0008) [2023-12-26 15:52:34,426][105692] Updated weights for policy 0, policy_version 70157 (0.0006) [2023-12-26 15:52:35,103][105692] Updated weights for policy 0, policy_version 70167 (0.0008) [2023-12-26 15:52:35,156][105620] Updated weights for policy 1, policy_version 70485 (0.0007) [2023-12-26 15:52:35,157][105692] Updated weights for policy 0, policy_version 70177 (0.0009) [2023-12-26 15:52:35,205][105620] Updated weights for policy 1, policy_version 70495 (0.0005) [2023-12-26 15:52:35,210][105692] Updated weights for policy 0, policy_version 70187 (0.0009) [2023-12-26 15:52:35,258][105620] Updated weights for policy 1, policy_version 70505 (0.0008) [2023-12-26 15:52:35,781][105692] Updated weights for policy 0, policy_version 70197 (0.0006) [2023-12-26 15:52:35,832][105692] Updated weights for policy 0, policy_version 70207 (0.0005) [2023-12-26 15:52:35,887][105692] Updated weights for policy 0, policy_version 70217 (0.0007) [2023-12-26 15:52:35,914][105620] Updated weights for policy 1, policy_version 70515 (0.0008) [2023-12-26 15:52:35,970][105620] Updated weights for policy 1, policy_version 70525 (0.0006) [2023-12-26 15:52:36,025][105620] Updated weights for policy 1, policy_version 70535 (0.0009) [2023-12-26 15:52:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 36036608. Throughput: 0: 10090.7, 1: 9566.1. Samples: 36024860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:52:36,063][104569] Avg episode reward: [(0, '8657.040'), (1, '9359.421')] [2023-12-26 15:52:36,074][105586] Saving new best policy, reward=9359.421! [2023-12-26 15:52:36,583][105692] Updated weights for policy 0, policy_version 70227 (0.0008) [2023-12-26 15:52:36,636][105692] Updated weights for policy 0, policy_version 70237 (0.0008) [2023-12-26 15:52:36,701][105692] Updated weights for policy 0, policy_version 70247 (0.0008) [2023-12-26 15:52:36,712][105620] Updated weights for policy 1, policy_version 70545 (0.0010) [2023-12-26 15:52:36,773][105620] Updated weights for policy 1, policy_version 70555 (0.0006) [2023-12-26 15:52:36,838][105620] Updated weights for policy 1, policy_version 70565 (0.0009) [2023-12-26 15:52:36,886][105620] Updated weights for policy 1, policy_version 70575 (0.0010) [2023-12-26 15:52:37,350][105692] Updated weights for policy 0, policy_version 70257 (0.0008) [2023-12-26 15:52:37,400][105692] Updated weights for policy 0, policy_version 70267 (0.0009) [2023-12-26 15:52:37,459][105692] Updated weights for policy 0, policy_version 70277 (0.0010) [2023-12-26 15:52:37,512][105692] Updated weights for policy 0, policy_version 70287 (0.0010) [2023-12-26 15:52:37,616][105620] Updated weights for policy 1, policy_version 70585 (0.0008) [2023-12-26 15:52:37,671][105620] Updated weights for policy 1, policy_version 70595 (0.0007) [2023-12-26 15:52:37,727][105620] Updated weights for policy 1, policy_version 70605 (0.0009) [2023-12-26 15:52:38,235][105692] Updated weights for policy 0, policy_version 70297 (0.0011) [2023-12-26 15:52:38,290][105692] Updated weights for policy 0, policy_version 70307 (0.0010) [2023-12-26 15:52:38,357][105692] Updated weights for policy 0, policy_version 70317 (0.0011) [2023-12-26 15:52:38,371][105620] Updated weights for policy 1, policy_version 70615 (0.0008) [2023-12-26 15:52:38,434][105620] Updated weights for policy 1, policy_version 70625 (0.0008) [2023-12-26 15:52:38,487][105620] Updated weights for policy 1, policy_version 70635 (0.0007) [2023-12-26 15:52:39,062][105692] Updated weights for policy 0, policy_version 70327 (0.0008) [2023-12-26 15:52:39,109][105692] Updated weights for policy 0, policy_version 70337 (0.0006) [2023-12-26 15:52:39,154][105692] Updated weights for policy 0, policy_version 70347 (0.0007) [2023-12-26 15:52:39,269][105620] Updated weights for policy 1, policy_version 70645 (0.0009) [2023-12-26 15:52:39,330][105620] Updated weights for policy 1, policy_version 70655 (0.0008) [2023-12-26 15:52:39,397][105620] Updated weights for policy 1, policy_version 70665 (0.0009) [2023-12-26 15:52:39,874][105692] Updated weights for policy 0, policy_version 70357 (0.0009) [2023-12-26 15:52:39,935][105692] Updated weights for policy 0, policy_version 70367 (0.0010) [2023-12-26 15:52:39,994][105692] Updated weights for policy 0, policy_version 70377 (0.0009) [2023-12-26 15:52:40,174][105620] Updated weights for policy 1, policy_version 70675 (0.0008) [2023-12-26 15:52:40,240][105620] Updated weights for policy 1, policy_version 70685 (0.0010) [2023-12-26 15:52:40,296][105620] Updated weights for policy 1, policy_version 70695 (0.0009) [2023-12-26 15:52:40,620][105692] Updated weights for policy 0, policy_version 70387 (0.0007) [2023-12-26 15:52:40,680][105692] Updated weights for policy 0, policy_version 70397 (0.0010) [2023-12-26 15:52:40,745][105692] Updated weights for policy 0, policy_version 70407 (0.0009) [2023-12-26 15:52:41,023][105620] Updated weights for policy 1, policy_version 70705 (0.0009) [2023-12-26 15:52:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 36134912. Throughput: 0: 10162.7, 1: 9585.8. Samples: 36145728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:52:41,063][104569] Avg episode reward: [(0, '8739.574'), (1, '9359.436')] [2023-12-26 15:52:41,090][105620] Updated weights for policy 1, policy_version 70715 (0.0007) [2023-12-26 15:52:41,155][105620] Updated weights for policy 1, policy_version 70725 (0.0006) [2023-12-26 15:52:41,224][105620] Updated weights for policy 1, policy_version 70735 (0.0006) [2023-12-26 15:52:41,231][105586] Saving new best policy, reward=9359.436! [2023-12-26 15:52:41,550][105692] Updated weights for policy 0, policy_version 70417 (0.0007) [2023-12-26 15:52:41,610][105692] Updated weights for policy 0, policy_version 70427 (0.0010) [2023-12-26 15:52:41,673][105692] Updated weights for policy 0, policy_version 70437 (0.0008) [2023-12-26 15:52:41,741][105692] Updated weights for policy 0, policy_version 70447 (0.0008) [2023-12-26 15:52:41,860][105620] Updated weights for policy 1, policy_version 70745 (0.0010) [2023-12-26 15:52:41,931][105620] Updated weights for policy 1, policy_version 70755 (0.0010) [2023-12-26 15:52:41,991][105620] Updated weights for policy 1, policy_version 70765 (0.0007) [2023-12-26 15:52:42,527][105692] Updated weights for policy 0, policy_version 70457 (0.0009) [2023-12-26 15:52:42,586][105692] Updated weights for policy 0, policy_version 70467 (0.0008) [2023-12-26 15:52:42,638][105692] Updated weights for policy 0, policy_version 70478 (0.0010) [2023-12-26 15:52:42,690][105620] Updated weights for policy 1, policy_version 70775 (0.0008) [2023-12-26 15:52:42,744][105620] Updated weights for policy 1, policy_version 70785 (0.0009) [2023-12-26 15:52:42,801][105620] Updated weights for policy 1, policy_version 70795 (0.0009) [2023-12-26 15:52:43,268][105692] Updated weights for policy 0, policy_version 70488 (0.0009) [2023-12-26 15:52:43,316][105692] Updated weights for policy 0, policy_version 70498 (0.0009) [2023-12-26 15:52:43,364][105692] Updated weights for policy 0, policy_version 70508 (0.0009) [2023-12-26 15:52:43,593][105620] Updated weights for policy 1, policy_version 70805 (0.0010) [2023-12-26 15:52:43,639][105620] Updated weights for policy 1, policy_version 70815 (0.0008) [2023-12-26 15:52:43,685][105620] Updated weights for policy 1, policy_version 70825 (0.0008) [2023-12-26 15:52:44,159][105692] Updated weights for policy 0, policy_version 70518 (0.0009) [2023-12-26 15:52:44,208][105692] Updated weights for policy 0, policy_version 70528 (0.0009) [2023-12-26 15:52:44,260][105692] Updated weights for policy 0, policy_version 70538 (0.0009) [2023-12-26 15:52:44,453][105620] Updated weights for policy 1, policy_version 70835 (0.0009) [2023-12-26 15:52:44,506][105620] Updated weights for policy 1, policy_version 70845 (0.0009) [2023-12-26 15:52:44,574][105620] Updated weights for policy 1, policy_version 70855 (0.0005) [2023-12-26 15:52:45,030][105692] Updated weights for policy 0, policy_version 70548 (0.0009) [2023-12-26 15:52:45,093][105692] Updated weights for policy 0, policy_version 70558 (0.0009) [2023-12-26 15:52:45,157][105692] Updated weights for policy 0, policy_version 70568 (0.0009) [2023-12-26 15:52:45,303][105620] Updated weights for policy 1, policy_version 70865 (0.0006) [2023-12-26 15:52:45,375][105620] Updated weights for policy 1, policy_version 70875 (0.0005) [2023-12-26 15:52:45,440][105620] Updated weights for policy 1, policy_version 70885 (0.0007) [2023-12-26 15:52:45,498][105620] Updated weights for policy 1, policy_version 70895 (0.0009) [2023-12-26 15:52:45,939][105692] Updated weights for policy 0, policy_version 70578 (0.0008) [2023-12-26 15:52:45,982][105692] Updated weights for policy 0, policy_version 70588 (0.0005) [2023-12-26 15:52:46,026][105692] Updated weights for policy 0, policy_version 70598 (0.0005) [2023-12-26 15:52:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.4, 300 sec: 19577.5). Total num frames: 36225024. Throughput: 0: 10071.9, 1: 9638.4. Samples: 36202104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:52:46,063][104569] Avg episode reward: [(0, '9179.678'), (1, '9359.423')] [2023-12-26 15:52:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000070896_18153472.pth... [2023-12-26 15:52:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000069776_17866752.pth [2023-12-26 15:52:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000070608_18079744.pth... [2023-12-26 15:52:46,074][105692] Updated weights for policy 0, policy_version 70608 (0.0005) [2023-12-26 15:52:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000069392_17768448.pth [2023-12-26 15:52:46,209][105620] Updated weights for policy 1, policy_version 70905 (0.0009) [2023-12-26 15:52:46,263][105620] Updated weights for policy 1, policy_version 70915 (0.0009) [2023-12-26 15:52:46,326][105620] Updated weights for policy 1, policy_version 70925 (0.0008) [2023-12-26 15:52:46,741][105692] Updated weights for policy 0, policy_version 70618 (0.0006) [2023-12-26 15:52:46,803][105692] Updated weights for policy 0, policy_version 70628 (0.0007) [2023-12-26 15:52:46,861][105692] Updated weights for policy 0, policy_version 70638 (0.0009) [2023-12-26 15:52:47,153][105620] Updated weights for policy 1, policy_version 70935 (0.0009) [2023-12-26 15:52:47,201][105620] Updated weights for policy 1, policy_version 70945 (0.0009) [2023-12-26 15:52:47,254][105620] Updated weights for policy 1, policy_version 70955 (0.0009) [2023-12-26 15:52:47,458][105692] Updated weights for policy 0, policy_version 70648 (0.0009) [2023-12-26 15:52:47,506][105692] Updated weights for policy 0, policy_version 70658 (0.0006) [2023-12-26 15:52:47,556][105692] Updated weights for policy 0, policy_version 70668 (0.0005) [2023-12-26 15:52:48,080][105620] Updated weights for policy 1, policy_version 70965 (0.0008) [2023-12-26 15:52:48,129][105620] Updated weights for policy 1, policy_version 70975 (0.0008) [2023-12-26 15:52:48,178][105620] Updated weights for policy 1, policy_version 70985 (0.0008) [2023-12-26 15:52:48,229][105692] Updated weights for policy 0, policy_version 70678 (0.0006) [2023-12-26 15:52:48,292][105692] Updated weights for policy 0, policy_version 70688 (0.0010) [2023-12-26 15:52:48,355][105692] Updated weights for policy 0, policy_version 70698 (0.0011) [2023-12-26 15:52:49,001][105692] Updated weights for policy 0, policy_version 70708 (0.0009) [2023-12-26 15:52:49,003][105620] Updated weights for policy 1, policy_version 70995 (0.0007) [2023-12-26 15:52:49,056][105692] Updated weights for policy 0, policy_version 70718 (0.0005) [2023-12-26 15:52:49,062][105620] Updated weights for policy 1, policy_version 71005 (0.0008) [2023-12-26 15:52:49,112][105692] Updated weights for policy 0, policy_version 70728 (0.0008) [2023-12-26 15:52:49,118][105620] Updated weights for policy 1, policy_version 71015 (0.0006) [2023-12-26 15:52:49,747][105692] Updated weights for policy 0, policy_version 70738 (0.0007) [2023-12-26 15:52:49,808][105692] Updated weights for policy 0, policy_version 70748 (0.0009) [2023-12-26 15:52:49,871][105692] Updated weights for policy 0, policy_version 70758 (0.0008) [2023-12-26 15:52:49,939][105692] Updated weights for policy 0, policy_version 70768 (0.0007) [2023-12-26 15:52:49,944][105620] Updated weights for policy 1, policy_version 71025 (0.0007) [2023-12-26 15:52:50,014][105620] Updated weights for policy 1, policy_version 71035 (0.0009) [2023-12-26 15:52:50,074][105620] Updated weights for policy 1, policy_version 71045 (0.0009) [2023-12-26 15:52:50,135][105620] Updated weights for policy 1, policy_version 71055 (0.0009) [2023-12-26 15:52:50,595][105692] Updated weights for policy 0, policy_version 70778 (0.0007) [2023-12-26 15:52:50,659][105692] Updated weights for policy 0, policy_version 70788 (0.0006) [2023-12-26 15:52:50,721][105692] Updated weights for policy 0, policy_version 70798 (0.0006) [2023-12-26 15:52:50,921][105620] Updated weights for policy 1, policy_version 71065 (0.0010) [2023-12-26 15:52:50,974][105620] Updated weights for policy 1, policy_version 71075 (0.0011) [2023-12-26 15:52:51,034][105620] Updated weights for policy 1, policy_version 71085 (0.0011) [2023-12-26 15:52:51,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 36331520. Throughput: 0: 10089.8, 1: 9510.1. Samples: 36316524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:52:51,063][104569] Avg episode reward: [(0, '8838.635'), (1, '9275.982')] [2023-12-26 15:52:51,380][105692] Updated weights for policy 0, policy_version 70808 (0.0007) [2023-12-26 15:52:51,450][105692] Updated weights for policy 0, policy_version 70818 (0.0009) [2023-12-26 15:52:51,511][105692] Updated weights for policy 0, policy_version 70828 (0.0009) [2023-12-26 15:52:51,837][105620] Updated weights for policy 1, policy_version 71095 (0.0009) [2023-12-26 15:52:51,891][105620] Updated weights for policy 1, policy_version 71105 (0.0009) [2023-12-26 15:52:51,951][105620] Updated weights for policy 1, policy_version 71115 (0.0009) [2023-12-26 15:52:52,228][105692] Updated weights for policy 0, policy_version 70838 (0.0009) [2023-12-26 15:52:52,297][105692] Updated weights for policy 0, policy_version 70848 (0.0010) [2023-12-26 15:52:52,355][105692] Updated weights for policy 0, policy_version 70858 (0.0009) [2023-12-26 15:52:52,759][105620] Updated weights for policy 1, policy_version 71125 (0.0007) [2023-12-26 15:52:52,830][105620] Updated weights for policy 1, policy_version 71135 (0.0006) [2023-12-26 15:52:52,892][105620] Updated weights for policy 1, policy_version 71145 (0.0007) [2023-12-26 15:52:53,066][105692] Updated weights for policy 0, policy_version 70868 (0.0007) [2023-12-26 15:52:53,136][105692] Updated weights for policy 0, policy_version 70878 (0.0007) [2023-12-26 15:52:53,204][105692] Updated weights for policy 0, policy_version 70888 (0.0006) [2023-12-26 15:52:53,548][105620] Updated weights for policy 1, policy_version 71155 (0.0008) [2023-12-26 15:52:53,610][105620] Updated weights for policy 1, policy_version 71165 (0.0009) [2023-12-26 15:52:53,672][105620] Updated weights for policy 1, policy_version 71175 (0.0009) [2023-12-26 15:52:53,866][105692] Updated weights for policy 0, policy_version 70898 (0.0008) [2023-12-26 15:52:53,922][105692] Updated weights for policy 0, policy_version 70908 (0.0005) [2023-12-26 15:52:53,977][105692] Updated weights for policy 0, policy_version 70918 (0.0005) [2023-12-26 15:52:54,025][105692] Updated weights for policy 0, policy_version 70928 (0.0005) [2023-12-26 15:52:54,481][105620] Updated weights for policy 1, policy_version 71185 (0.0009) [2023-12-26 15:52:54,546][105620] Updated weights for policy 1, policy_version 71195 (0.0011) [2023-12-26 15:52:54,601][105620] Updated weights for policy 1, policy_version 71205 (0.0010) [2023-12-26 15:52:54,617][105692] Updated weights for policy 0, policy_version 70938 (0.0005) [2023-12-26 15:52:54,649][105620] Updated weights for policy 1, policy_version 71215 (0.0008) [2023-12-26 15:52:54,682][105692] Updated weights for policy 0, policy_version 70948 (0.0009) [2023-12-26 15:52:54,730][105692] Updated weights for policy 0, policy_version 70958 (0.0010) [2023-12-26 15:52:55,340][105692] Updated weights for policy 0, policy_version 70968 (0.0006) [2023-12-26 15:52:55,370][105620] Updated weights for policy 1, policy_version 71225 (0.0010) [2023-12-26 15:52:55,386][105692] Updated weights for policy 0, policy_version 70978 (0.0005) [2023-12-26 15:52:55,425][105620] Updated weights for policy 1, policy_version 71235 (0.0010) [2023-12-26 15:52:55,434][105692] Updated weights for policy 0, policy_version 70988 (0.0005) [2023-12-26 15:52:55,476][105620] Updated weights for policy 1, policy_version 71245 (0.0010) [2023-12-26 15:52:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 36421632. Throughput: 0: 10239.5, 1: 9477.3. Samples: 36433988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:52:56,062][104569] Avg episode reward: [(0, '8659.997'), (1, '9188.231')] [2023-12-26 15:52:56,150][105692] Updated weights for policy 0, policy_version 70998 (0.0007) [2023-12-26 15:52:56,212][105692] Updated weights for policy 0, policy_version 71008 (0.0008) [2023-12-26 15:52:56,238][105620] Updated weights for policy 1, policy_version 71255 (0.0010) [2023-12-26 15:52:56,275][105692] Updated weights for policy 0, policy_version 71018 (0.0005) [2023-12-26 15:52:56,292][105620] Updated weights for policy 1, policy_version 71265 (0.0010) [2023-12-26 15:52:56,358][105620] Updated weights for policy 1, policy_version 71275 (0.0010) [2023-12-26 15:52:56,992][105620] Updated weights for policy 1, policy_version 71285 (0.0008) [2023-12-26 15:52:57,059][105620] Updated weights for policy 1, policy_version 71295 (0.0010) [2023-12-26 15:52:57,084][105692] Updated weights for policy 0, policy_version 71028 (0.0006) [2023-12-26 15:52:57,123][105620] Updated weights for policy 1, policy_version 71305 (0.0011) [2023-12-26 15:52:57,136][105692] Updated weights for policy 0, policy_version 71038 (0.0005) [2023-12-26 15:52:57,192][105692] Updated weights for policy 0, policy_version 71048 (0.0007) [2023-12-26 15:52:57,735][105620] Updated weights for policy 1, policy_version 71315 (0.0007) [2023-12-26 15:52:57,787][105620] Updated weights for policy 1, policy_version 71325 (0.0007) [2023-12-26 15:52:57,851][105620] Updated weights for policy 1, policy_version 71335 (0.0006) [2023-12-26 15:52:57,937][105692] Updated weights for policy 0, policy_version 71058 (0.0010) [2023-12-26 15:52:57,989][105692] Updated weights for policy 0, policy_version 71068 (0.0009) [2023-12-26 15:52:58,039][105692] Updated weights for policy 0, policy_version 71078 (0.0009) [2023-12-26 15:52:58,107][105692] Updated weights for policy 0, policy_version 71088 (0.0007) [2023-12-26 15:52:58,518][105620] Updated weights for policy 1, policy_version 71345 (0.0006) [2023-12-26 15:52:58,584][105620] Updated weights for policy 1, policy_version 71355 (0.0008) [2023-12-26 15:52:58,661][105620] Updated weights for policy 1, policy_version 71365 (0.0007) [2023-12-26 15:52:58,738][105620] Updated weights for policy 1, policy_version 71375 (0.0007) [2023-12-26 15:52:58,966][105692] Updated weights for policy 0, policy_version 71098 (0.0009) [2023-12-26 15:52:59,028][105692] Updated weights for policy 0, policy_version 71108 (0.0009) [2023-12-26 15:52:59,088][105692] Updated weights for policy 0, policy_version 71118 (0.0007) [2023-12-26 15:52:59,555][105620] Updated weights for policy 1, policy_version 71385 (0.0009) [2023-12-26 15:52:59,610][105620] Updated weights for policy 1, policy_version 71395 (0.0008) [2023-12-26 15:52:59,670][105620] Updated weights for policy 1, policy_version 71405 (0.0009) [2023-12-26 15:52:59,795][105692] Updated weights for policy 0, policy_version 71128 (0.0009) [2023-12-26 15:52:59,860][105692] Updated weights for policy 0, policy_version 71138 (0.0010) [2023-12-26 15:52:59,915][105692] Updated weights for policy 0, policy_version 71148 (0.0009) [2023-12-26 15:53:00,382][105620] Updated weights for policy 1, policy_version 71415 (0.0009) [2023-12-26 15:53:00,440][105620] Updated weights for policy 1, policy_version 71425 (0.0009) [2023-12-26 15:53:00,494][105620] Updated weights for policy 1, policy_version 71435 (0.0008) [2023-12-26 15:53:00,661][105692] Updated weights for policy 0, policy_version 71158 (0.0009) [2023-12-26 15:53:00,724][105692] Updated weights for policy 0, policy_version 71168 (0.0009) [2023-12-26 15:53:00,775][105692] Updated weights for policy 0, policy_version 71178 (0.0009) [2023-12-26 15:53:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 36519936. Throughput: 0: 10222.8, 1: 9502.2. Samples: 36491688. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:53:01,063][104569] Avg episode reward: [(0, '8846.549'), (1, '9012.551')] [2023-12-26 15:53:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000071184_18227200.pth... [2023-12-26 15:53:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000071440_18292736.pth... [2023-12-26 15:53:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000070032_17932288.pth [2023-12-26 15:53:01,085][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000070352_18014208.pth [2023-12-26 15:53:01,246][105620] Updated weights for policy 1, policy_version 71445 (0.0008) [2023-12-26 15:53:01,306][105620] Updated weights for policy 1, policy_version 71455 (0.0010) [2023-12-26 15:53:01,364][105620] Updated weights for policy 1, policy_version 71465 (0.0009) [2023-12-26 15:53:01,559][105692] Updated weights for policy 0, policy_version 71188 (0.0009) [2023-12-26 15:53:01,610][105692] Updated weights for policy 0, policy_version 71198 (0.0010) [2023-12-26 15:53:01,665][105692] Updated weights for policy 0, policy_version 71208 (0.0011) [2023-12-26 15:53:02,127][105620] Updated weights for policy 1, policy_version 71475 (0.0008) [2023-12-26 15:53:02,174][105620] Updated weights for policy 1, policy_version 71485 (0.0008) [2023-12-26 15:53:02,225][105620] Updated weights for policy 1, policy_version 71495 (0.0008) [2023-12-26 15:53:02,431][105692] Updated weights for policy 0, policy_version 71218 (0.0009) [2023-12-26 15:53:02,492][105692] Updated weights for policy 0, policy_version 71228 (0.0009) [2023-12-26 15:53:02,551][105692] Updated weights for policy 0, policy_version 71238 (0.0007) [2023-12-26 15:53:02,606][105692] Updated weights for policy 0, policy_version 71248 (0.0006) [2023-12-26 15:53:03,055][105620] Updated weights for policy 1, policy_version 71505 (0.0008) [2023-12-26 15:53:03,105][105620] Updated weights for policy 1, policy_version 71515 (0.0009) [2023-12-26 15:53:03,151][105620] Updated weights for policy 1, policy_version 71525 (0.0008) [2023-12-26 15:53:03,204][105620] Updated weights for policy 1, policy_version 71535 (0.0007) [2023-12-26 15:53:03,207][105692] Updated weights for policy 0, policy_version 71258 (0.0007) [2023-12-26 15:53:03,264][105692] Updated weights for policy 0, policy_version 71268 (0.0008) [2023-12-26 15:53:03,315][105692] Updated weights for policy 0, policy_version 71278 (0.0005) [2023-12-26 15:53:03,873][105620] Updated weights for policy 1, policy_version 71545 (0.0007) [2023-12-26 15:53:03,881][105692] Updated weights for policy 0, policy_version 71288 (0.0007) [2023-12-26 15:53:03,930][105620] Updated weights for policy 1, policy_version 71555 (0.0008) [2023-12-26 15:53:03,939][105692] Updated weights for policy 0, policy_version 71298 (0.0008) [2023-12-26 15:53:03,993][105620] Updated weights for policy 1, policy_version 71565 (0.0007) [2023-12-26 15:53:03,996][105692] Updated weights for policy 0, policy_version 71308 (0.0007) [2023-12-26 15:53:04,616][105692] Updated weights for policy 0, policy_version 71318 (0.0005) [2023-12-26 15:53:04,662][105692] Updated weights for policy 0, policy_version 71328 (0.0005) [2023-12-26 15:53:04,713][105692] Updated weights for policy 0, policy_version 71338 (0.0005) [2023-12-26 15:53:04,762][105620] Updated weights for policy 1, policy_version 71575 (0.0009) [2023-12-26 15:53:04,827][105620] Updated weights for policy 1, policy_version 71585 (0.0010) [2023-12-26 15:53:04,892][105620] Updated weights for policy 1, policy_version 71595 (0.0010) [2023-12-26 15:53:05,243][105692] Updated weights for policy 0, policy_version 71348 (0.0005) [2023-12-26 15:53:05,294][105692] Updated weights for policy 0, policy_version 71358 (0.0005) [2023-12-26 15:53:05,353][105692] Updated weights for policy 0, policy_version 71368 (0.0005) [2023-12-26 15:53:05,598][105620] Updated weights for policy 1, policy_version 71605 (0.0007) [2023-12-26 15:53:05,668][105620] Updated weights for policy 1, policy_version 71615 (0.0007) [2023-12-26 15:53:05,733][105620] Updated weights for policy 1, policy_version 71625 (0.0010) [2023-12-26 15:53:05,860][105692] Updated weights for policy 0, policy_version 71378 (0.0006) [2023-12-26 15:53:05,923][105692] Updated weights for policy 0, policy_version 71388 (0.0009) [2023-12-26 15:53:05,976][105692] Updated weights for policy 0, policy_version 71398 (0.0009) [2023-12-26 15:53:06,033][105692] Updated weights for policy 0, policy_version 71408 (0.0009) [2023-12-26 15:53:06,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 36626432. Throughput: 0: 10145.2, 1: 9426.6. Samples: 36607760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:53:06,063][104569] Avg episode reward: [(0, '8505.900'), (1, '8844.914')] [2023-12-26 15:53:06,281][105620] Updated weights for policy 1, policy_version 71635 (0.0008) [2023-12-26 15:53:06,343][105620] Updated weights for policy 1, policy_version 71645 (0.0008) [2023-12-26 15:53:06,409][105620] Updated weights for policy 1, policy_version 71655 (0.0010) [2023-12-26 15:53:06,735][105692] Updated weights for policy 0, policy_version 71418 (0.0006) [2023-12-26 15:53:06,802][105692] Updated weights for policy 0, policy_version 71428 (0.0006) [2023-12-26 15:53:06,853][105692] Updated weights for policy 0, policy_version 71438 (0.0006) [2023-12-26 15:53:07,077][105620] Updated weights for policy 1, policy_version 71665 (0.0005) [2023-12-26 15:53:07,126][105620] Updated weights for policy 1, policy_version 71675 (0.0005) [2023-12-26 15:53:07,171][105620] Updated weights for policy 1, policy_version 71685 (0.0005) [2023-12-26 15:53:07,230][105620] Updated weights for policy 1, policy_version 71695 (0.0005) [2023-12-26 15:53:07,531][105692] Updated weights for policy 0, policy_version 71448 (0.0007) [2023-12-26 15:53:07,593][105692] Updated weights for policy 0, policy_version 71458 (0.0009) [2023-12-26 15:53:07,646][105692] Updated weights for policy 0, policy_version 71470 (0.0010) [2023-12-26 15:53:07,828][105620] Updated weights for policy 1, policy_version 71705 (0.0010) [2023-12-26 15:53:07,875][105620] Updated weights for policy 1, policy_version 71715 (0.0010) [2023-12-26 15:53:07,937][105620] Updated weights for policy 1, policy_version 71725 (0.0010) [2023-12-26 15:53:08,392][105692] Updated weights for policy 0, policy_version 71480 (0.0009) [2023-12-26 15:53:08,450][105692] Updated weights for policy 0, policy_version 71491 (0.0008) [2023-12-26 15:53:08,503][105692] Updated weights for policy 0, policy_version 71501 (0.0007) [2023-12-26 15:53:08,704][105620] Updated weights for policy 1, policy_version 71735 (0.0010) [2023-12-26 15:53:08,716][105586] KL-divergence is very high: 140.0969 [2023-12-26 15:53:08,721][105586] KL-divergence is very high: 184.1958 [2023-12-26 15:53:08,748][105620] Updated weights for policy 1, policy_version 71745 (0.0010) [2023-12-26 15:53:08,754][105586] KL-divergence is very high: 183.7720 [2023-12-26 15:53:08,760][105586] KL-divergence is very high: 215.8897 [2023-12-26 15:53:08,805][105586] KL-divergence is very high: 126.1025 [2023-12-26 15:53:08,809][105620] Updated weights for policy 1, policy_version 71755 (0.0010) [2023-12-26 15:53:08,810][105586] KL-divergence is very high: 153.3549 [2023-12-26 15:53:09,191][105692] Updated weights for policy 0, policy_version 71511 (0.0007) [2023-12-26 15:53:09,262][105692] Updated weights for policy 0, policy_version 71521 (0.0008) [2023-12-26 15:53:09,319][105692] Updated weights for policy 0, policy_version 71531 (0.0008) [2023-12-26 15:53:09,576][105620] Updated weights for policy 1, policy_version 71765 (0.0010) [2023-12-26 15:53:09,635][105620] Updated weights for policy 1, policy_version 71775 (0.0010) [2023-12-26 15:53:09,700][105620] Updated weights for policy 1, policy_version 71785 (0.0010) [2023-12-26 15:53:10,100][105692] Updated weights for policy 0, policy_version 71541 (0.0008) [2023-12-26 15:53:10,152][105692] Updated weights for policy 0, policy_version 71551 (0.0008) [2023-12-26 15:53:10,205][105692] Updated weights for policy 0, policy_version 71561 (0.0009) [2023-12-26 15:53:10,435][105620] Updated weights for policy 1, policy_version 71795 (0.0010) [2023-12-26 15:53:10,490][105620] Updated weights for policy 1, policy_version 71805 (0.0010) [2023-12-26 15:53:10,545][105620] Updated weights for policy 1, policy_version 71815 (0.0010) [2023-12-26 15:53:11,002][105692] Updated weights for policy 0, policy_version 71571 (0.0008) [2023-12-26 15:53:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 36716544. Throughput: 0: 10169.9, 1: 9500.8. Samples: 36729336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 15:53:11,063][104569] Avg episode reward: [(0, '7881.031'), (1, '8287.631')] [2023-12-26 15:53:11,072][105692] Updated weights for policy 0, policy_version 71581 (0.0008) [2023-12-26 15:53:11,140][105692] Updated weights for policy 0, policy_version 71591 (0.0007) [2023-12-26 15:53:11,339][105620] Updated weights for policy 1, policy_version 71825 (0.0010) [2023-12-26 15:53:11,398][105620] Updated weights for policy 1, policy_version 71835 (0.0009) [2023-12-26 15:53:11,460][105620] Updated weights for policy 1, policy_version 71845 (0.0009) [2023-12-26 15:53:11,522][105620] Updated weights for policy 1, policy_version 71855 (0.0007) [2023-12-26 15:53:11,898][105692] Updated weights for policy 0, policy_version 71601 (0.0006) [2023-12-26 15:53:11,958][105692] Updated weights for policy 0, policy_version 71611 (0.0008) [2023-12-26 15:53:12,013][105692] Updated weights for policy 0, policy_version 71621 (0.0008) [2023-12-26 15:53:12,062][105692] Updated weights for policy 0, policy_version 71631 (0.0008) [2023-12-26 15:53:12,215][105620] Updated weights for policy 1, policy_version 71865 (0.0009) [2023-12-26 15:53:12,278][105620] Updated weights for policy 1, policy_version 71875 (0.0009) [2023-12-26 15:53:12,352][105620] Updated weights for policy 1, policy_version 71885 (0.0008) [2023-12-26 15:53:12,851][105692] Updated weights for policy 0, policy_version 71641 (0.0008) [2023-12-26 15:53:12,917][105692] Updated weights for policy 0, policy_version 71651 (0.0008) [2023-12-26 15:53:12,969][105692] Updated weights for policy 0, policy_version 71661 (0.0009) [2023-12-26 15:53:13,062][105620] Updated weights for policy 1, policy_version 71895 (0.0006) [2023-12-26 15:53:13,129][105620] Updated weights for policy 1, policy_version 71905 (0.0005) [2023-12-26 15:53:13,189][105620] Updated weights for policy 1, policy_version 71915 (0.0005) [2023-12-26 15:53:13,686][105620] Updated weights for policy 1, policy_version 71925 (0.0005) [2023-12-26 15:53:13,744][105620] Updated weights for policy 1, policy_version 71935 (0.0009) [2023-12-26 15:53:13,754][105692] Updated weights for policy 0, policy_version 71671 (0.0007) [2023-12-26 15:53:13,802][105620] Updated weights for policy 1, policy_version 71945 (0.0010) [2023-12-26 15:53:13,813][105692] Updated weights for policy 0, policy_version 71681 (0.0006) [2023-12-26 15:53:13,869][105692] Updated weights for policy 0, policy_version 71691 (0.0008) [2023-12-26 15:53:14,440][105620] Updated weights for policy 1, policy_version 71955 (0.0009) [2023-12-26 15:53:14,486][105620] Updated weights for policy 1, policy_version 71965 (0.0005) [2023-12-26 15:53:14,532][105620] Updated weights for policy 1, policy_version 71975 (0.0005) [2023-12-26 15:53:14,655][105692] Updated weights for policy 0, policy_version 71701 (0.0009) [2023-12-26 15:53:14,708][105692] Updated weights for policy 0, policy_version 71711 (0.0009) [2023-12-26 15:53:14,773][105692] Updated weights for policy 0, policy_version 71721 (0.0009) [2023-12-26 15:53:15,256][105620] Updated weights for policy 1, policy_version 71985 (0.0008) [2023-12-26 15:53:15,319][105620] Updated weights for policy 1, policy_version 71995 (0.0009) [2023-12-26 15:53:15,378][105620] Updated weights for policy 1, policy_version 72005 (0.0010) [2023-12-26 15:53:15,421][105692] Updated weights for policy 0, policy_version 71731 (0.0009) [2023-12-26 15:53:15,427][105620] Updated weights for policy 1, policy_version 72015 (0.0009) [2023-12-26 15:53:15,470][105692] Updated weights for policy 0, policy_version 71741 (0.0005) [2023-12-26 15:53:15,523][105692] Updated weights for policy 0, policy_version 71751 (0.0005) [2023-12-26 15:53:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 36814848. Throughput: 0: 10015.3, 1: 9535.6. Samples: 36787148. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 15:53:16,063][104569] Avg episode reward: [(0, '7957.362'), (1, '8283.784')] [2023-12-26 15:53:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000071760_18374656.pth... [2023-12-26 15:53:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000072016_18440192.pth... [2023-12-26 15:53:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000070896_18153472.pth [2023-12-26 15:53:16,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000070608_18079744.pth [2023-12-26 15:53:16,158][105620] Updated weights for policy 1, policy_version 72025 (0.0009) [2023-12-26 15:53:16,211][105692] Updated weights for policy 0, policy_version 71761 (0.0005) [2023-12-26 15:53:16,213][105620] Updated weights for policy 1, policy_version 72035 (0.0009) [2023-12-26 15:53:16,267][105620] Updated weights for policy 1, policy_version 72045 (0.0007) [2023-12-26 15:53:16,271][105692] Updated weights for policy 0, policy_version 71771 (0.0007) [2023-12-26 15:53:16,317][105692] Updated weights for policy 0, policy_version 71781 (0.0005) [2023-12-26 15:53:16,363][105692] Updated weights for policy 0, policy_version 71791 (0.0008) [2023-12-26 15:53:17,065][105620] Updated weights for policy 1, policy_version 72055 (0.0008) [2023-12-26 15:53:17,080][105692] Updated weights for policy 0, policy_version 71801 (0.0006) [2023-12-26 15:53:17,119][105620] Updated weights for policy 1, policy_version 72065 (0.0009) [2023-12-26 15:53:17,134][105692] Updated weights for policy 0, policy_version 71811 (0.0008) [2023-12-26 15:53:17,171][105620] Updated weights for policy 1, policy_version 72075 (0.0006) [2023-12-26 15:53:17,190][105692] Updated weights for policy 0, policy_version 71821 (0.0006) [2023-12-26 15:53:17,767][105692] Updated weights for policy 0, policy_version 71831 (0.0005) [2023-12-26 15:53:17,839][105692] Updated weights for policy 0, policy_version 71841 (0.0005) [2023-12-26 15:53:17,905][105692] Updated weights for policy 0, policy_version 71851 (0.0007) [2023-12-26 15:53:18,032][105620] Updated weights for policy 1, policy_version 72085 (0.0008) [2023-12-26 15:53:18,085][105620] Updated weights for policy 1, policy_version 72095 (0.0008) [2023-12-26 15:53:18,132][105620] Updated weights for policy 1, policy_version 72105 (0.0009) [2023-12-26 15:53:18,550][105692] Updated weights for policy 0, policy_version 71861 (0.0009) [2023-12-26 15:53:18,609][105692] Updated weights for policy 0, policy_version 71871 (0.0010) [2023-12-26 15:53:18,672][105692] Updated weights for policy 0, policy_version 71881 (0.0009) [2023-12-26 15:53:18,895][105620] Updated weights for policy 1, policy_version 72115 (0.0008) [2023-12-26 15:53:18,945][105620] Updated weights for policy 1, policy_version 72125 (0.0008) [2023-12-26 15:53:19,001][105620] Updated weights for policy 1, policy_version 72135 (0.0006) [2023-12-26 15:53:19,454][105692] Updated weights for policy 0, policy_version 71891 (0.0009) [2023-12-26 15:53:19,515][105692] Updated weights for policy 0, policy_version 71901 (0.0008) [2023-12-26 15:53:19,579][105692] Updated weights for policy 0, policy_version 71911 (0.0009) [2023-12-26 15:53:19,726][105620] Updated weights for policy 1, policy_version 72145 (0.0009) [2023-12-26 15:53:19,795][105620] Updated weights for policy 1, policy_version 72155 (0.0006) [2023-12-26 15:53:19,854][105620] Updated weights for policy 1, policy_version 72165 (0.0009) [2023-12-26 15:53:19,913][105620] Updated weights for policy 1, policy_version 72175 (0.0009) [2023-12-26 15:53:20,401][105692] Updated weights for policy 0, policy_version 71921 (0.0010) [2023-12-26 15:53:20,463][105692] Updated weights for policy 0, policy_version 71931 (0.0009) [2023-12-26 15:53:20,514][105692] Updated weights for policy 0, policy_version 71941 (0.0009) [2023-12-26 15:53:20,568][105692] Updated weights for policy 0, policy_version 71951 (0.0009) [2023-12-26 15:53:20,640][105620] Updated weights for policy 1, policy_version 72185 (0.0009) [2023-12-26 15:53:20,695][105620] Updated weights for policy 1, policy_version 72195 (0.0009) [2023-12-26 15:53:20,749][105620] Updated weights for policy 1, policy_version 72205 (0.0009) [2023-12-26 15:53:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 36913152. Throughput: 0: 9970.8, 1: 9548.9. Samples: 36903248. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 15:53:21,062][104569] Avg episode reward: [(0, '7718.059'), (1, '9018.267')] [2023-12-26 15:53:21,355][105692] Updated weights for policy 0, policy_version 71961 (0.0009) [2023-12-26 15:53:21,421][105692] Updated weights for policy 0, policy_version 71971 (0.0009) [2023-12-26 15:53:21,480][105692] Updated weights for policy 0, policy_version 71981 (0.0009) [2023-12-26 15:53:21,546][105620] Updated weights for policy 1, policy_version 72215 (0.0009) [2023-12-26 15:53:21,601][105620] Updated weights for policy 1, policy_version 72225 (0.0009) [2023-12-26 15:53:21,669][105620] Updated weights for policy 1, policy_version 72235 (0.0008) [2023-12-26 15:53:22,218][105692] Updated weights for policy 0, policy_version 71991 (0.0009) [2023-12-26 15:53:22,280][105692] Updated weights for policy 0, policy_version 72001 (0.0009) [2023-12-26 15:53:22,342][105692] Updated weights for policy 0, policy_version 72011 (0.0007) [2023-12-26 15:53:22,441][105620] Updated weights for policy 1, policy_version 72245 (0.0008) [2023-12-26 15:53:22,500][105620] Updated weights for policy 1, policy_version 72255 (0.0009) [2023-12-26 15:53:22,564][105620] Updated weights for policy 1, policy_version 72265 (0.0008) [2023-12-26 15:53:23,018][105692] Updated weights for policy 0, policy_version 72021 (0.0008) [2023-12-26 15:53:23,085][105692] Updated weights for policy 0, policy_version 72031 (0.0009) [2023-12-26 15:53:23,141][105692] Updated weights for policy 0, policy_version 72041 (0.0009) [2023-12-26 15:53:23,361][105620] Updated weights for policy 1, policy_version 72275 (0.0009) [2023-12-26 15:53:23,419][105620] Updated weights for policy 1, policy_version 72285 (0.0010) [2023-12-26 15:53:23,473][105620] Updated weights for policy 1, policy_version 72295 (0.0010) [2023-12-26 15:53:23,772][105692] Updated weights for policy 0, policy_version 72051 (0.0009) [2023-12-26 15:53:23,819][105692] Updated weights for policy 0, policy_version 72061 (0.0009) [2023-12-26 15:53:23,874][105692] Updated weights for policy 0, policy_version 72071 (0.0009) [2023-12-26 15:53:24,279][105620] Updated weights for policy 1, policy_version 72305 (0.0010) [2023-12-26 15:53:24,333][105620] Updated weights for policy 1, policy_version 72315 (0.0009) [2023-12-26 15:53:24,387][105620] Updated weights for policy 1, policy_version 72325 (0.0009) [2023-12-26 15:53:24,442][105620] Updated weights for policy 1, policy_version 72335 (0.0009) [2023-12-26 15:53:24,655][105692] Updated weights for policy 0, policy_version 72081 (0.0009) [2023-12-26 15:53:24,709][105692] Updated weights for policy 0, policy_version 72091 (0.0009) [2023-12-26 15:53:24,759][105692] Updated weights for policy 0, policy_version 72101 (0.0009) [2023-12-26 15:53:24,806][105692] Updated weights for policy 0, policy_version 72111 (0.0009) [2023-12-26 15:53:25,121][105620] Updated weights for policy 1, policy_version 72345 (0.0006) [2023-12-26 15:53:25,178][105620] Updated weights for policy 1, policy_version 72355 (0.0010) [2023-12-26 15:53:25,229][105620] Updated weights for policy 1, policy_version 72365 (0.0009) [2023-12-26 15:53:25,582][105692] Updated weights for policy 0, policy_version 72122 (0.0009) [2023-12-26 15:53:25,629][105692] Updated weights for policy 0, policy_version 72132 (0.0009) [2023-12-26 15:53:25,685][105692] Updated weights for policy 0, policy_version 72142 (0.0009) [2023-12-26 15:53:25,967][105620] Updated weights for policy 1, policy_version 72375 (0.0009) [2023-12-26 15:53:26,028][105620] Updated weights for policy 1, policy_version 72385 (0.0009) [2023-12-26 15:53:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 37003264. Throughput: 0: 9843.1, 1: 9476.2. Samples: 37015096. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 15:53:26,063][104569] Avg episode reward: [(0, '8071.622'), (1, '9098.327')] [2023-12-26 15:53:26,090][105620] Updated weights for policy 1, policy_version 72395 (0.0008) [2023-12-26 15:53:26,479][105692] Updated weights for policy 0, policy_version 72152 (0.0010) [2023-12-26 15:53:26,529][105692] Updated weights for policy 0, policy_version 72162 (0.0009) [2023-12-26 15:53:26,579][105692] Updated weights for policy 0, policy_version 72172 (0.0009) [2023-12-26 15:53:26,768][105620] Updated weights for policy 1, policy_version 72405 (0.0006) [2023-12-26 15:53:26,820][105620] Updated weights for policy 1, policy_version 72415 (0.0005) [2023-12-26 15:53:26,874][105620] Updated weights for policy 1, policy_version 72425 (0.0005) [2023-12-26 15:53:27,221][105692] Updated weights for policy 0, policy_version 72182 (0.0008) [2023-12-26 15:53:27,278][105692] Updated weights for policy 0, policy_version 72192 (0.0006) [2023-12-26 15:53:27,332][105692] Updated weights for policy 0, policy_version 72202 (0.0010) [2023-12-26 15:53:27,518][105620] Updated weights for policy 1, policy_version 72435 (0.0005) [2023-12-26 15:53:27,567][105620] Updated weights for policy 1, policy_version 72445 (0.0005) [2023-12-26 15:53:27,616][105620] Updated weights for policy 1, policy_version 72455 (0.0008) [2023-12-26 15:53:28,057][105692] Updated weights for policy 0, policy_version 72212 (0.0010) [2023-12-26 15:53:28,110][105692] Updated weights for policy 0, policy_version 72222 (0.0010) [2023-12-26 15:53:28,167][105692] Updated weights for policy 0, policy_version 72232 (0.0009) [2023-12-26 15:53:28,196][105620] Updated weights for policy 1, policy_version 72465 (0.0007) [2023-12-26 15:53:28,257][105620] Updated weights for policy 1, policy_version 72475 (0.0008) [2023-12-26 15:53:28,323][105620] Updated weights for policy 1, policy_version 72485 (0.0009) [2023-12-26 15:53:28,390][105620] Updated weights for policy 1, policy_version 72495 (0.0010) [2023-12-26 15:53:28,791][105692] Updated weights for policy 0, policy_version 72242 (0.0008) [2023-12-26 15:53:28,854][105692] Updated weights for policy 0, policy_version 72252 (0.0006) [2023-12-26 15:53:28,901][105692] Updated weights for policy 0, policy_version 72262 (0.0009) [2023-12-26 15:53:28,952][105692] Updated weights for policy 0, policy_version 72272 (0.0009) [2023-12-26 15:53:29,169][105620] Updated weights for policy 1, policy_version 72505 (0.0008) [2023-12-26 15:53:29,225][105620] Updated weights for policy 1, policy_version 72515 (0.0009) [2023-12-26 15:53:29,282][105620] Updated weights for policy 1, policy_version 72525 (0.0008) [2023-12-26 15:53:29,736][105692] Updated weights for policy 0, policy_version 72282 (0.0009) [2023-12-26 15:53:29,789][105692] Updated weights for policy 0, policy_version 72292 (0.0010) [2023-12-26 15:53:29,852][105692] Updated weights for policy 0, policy_version 72302 (0.0010) [2023-12-26 15:53:29,952][105620] Updated weights for policy 1, policy_version 72535 (0.0009) [2023-12-26 15:53:30,003][105620] Updated weights for policy 1, policy_version 72545 (0.0009) [2023-12-26 15:53:30,054][105620] Updated weights for policy 1, policy_version 72555 (0.0006) [2023-12-26 15:53:30,586][105692] Updated weights for policy 0, policy_version 72312 (0.0010) [2023-12-26 15:53:30,640][105692] Updated weights for policy 0, policy_version 72322 (0.0010) [2023-12-26 15:53:30,695][105692] Updated weights for policy 0, policy_version 72332 (0.0010) [2023-12-26 15:53:30,759][105620] Updated weights for policy 1, policy_version 72565 (0.0007) [2023-12-26 15:53:30,817][105620] Updated weights for policy 1, policy_version 72575 (0.0007) [2023-12-26 15:53:30,864][105620] Updated weights for policy 1, policy_version 72585 (0.0008) [2023-12-26 15:53:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 37109760. Throughput: 0: 9883.6, 1: 9520.8. Samples: 37075300. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 15:53:31,062][104569] Avg episode reward: [(0, '8832.884'), (1, '8664.200')] [2023-12-26 15:53:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000072336_18522112.pth... [2023-12-26 15:53:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000072592_18587648.pth... [2023-12-26 15:53:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000071184_18227200.pth [2023-12-26 15:53:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000071440_18292736.pth [2023-12-26 15:53:31,358][105692] Updated weights for policy 0, policy_version 72342 (0.0008) [2023-12-26 15:53:31,422][105692] Updated weights for policy 0, policy_version 72352 (0.0007) [2023-12-26 15:53:31,477][105692] Updated weights for policy 0, policy_version 72362 (0.0007) [2023-12-26 15:53:31,701][105620] Updated weights for policy 1, policy_version 72595 (0.0008) [2023-12-26 15:53:31,763][105620] Updated weights for policy 1, policy_version 72605 (0.0009) [2023-12-26 15:53:31,812][105620] Updated weights for policy 1, policy_version 72615 (0.0009) [2023-12-26 15:53:32,116][105692] Updated weights for policy 0, policy_version 72372 (0.0009) [2023-12-26 15:53:32,175][105692] Updated weights for policy 0, policy_version 72382 (0.0008) [2023-12-26 15:53:32,227][105692] Updated weights for policy 0, policy_version 72392 (0.0008) [2023-12-26 15:53:32,594][105620] Updated weights for policy 1, policy_version 72625 (0.0010) [2023-12-26 15:53:32,653][105620] Updated weights for policy 1, policy_version 72635 (0.0009) [2023-12-26 15:53:32,707][105620] Updated weights for policy 1, policy_version 72645 (0.0009) [2023-12-26 15:53:32,768][105620] Updated weights for policy 1, policy_version 72655 (0.0009) [2023-12-26 15:53:32,980][105692] Updated weights for policy 0, policy_version 72403 (0.0010) [2023-12-26 15:53:33,033][105692] Updated weights for policy 0, policy_version 72413 (0.0009) [2023-12-26 15:53:33,083][105692] Updated weights for policy 0, policy_version 72423 (0.0009) [2023-12-26 15:53:33,384][105620] Updated weights for policy 1, policy_version 72665 (0.0010) [2023-12-26 15:53:33,433][105620] Updated weights for policy 1, policy_version 72675 (0.0010) [2023-12-26 15:53:33,488][105620] Updated weights for policy 1, policy_version 72685 (0.0010) [2023-12-26 15:53:33,933][105692] Updated weights for policy 0, policy_version 72433 (0.0009) [2023-12-26 15:53:33,986][105692] Updated weights for policy 0, policy_version 72443 (0.0009) [2023-12-26 15:53:34,043][105692] Updated weights for policy 0, policy_version 72454 (0.0010) [2023-12-26 15:53:34,105][105692] Updated weights for policy 0, policy_version 72464 (0.0009) [2023-12-26 15:53:34,121][105620] Updated weights for policy 1, policy_version 72695 (0.0007) [2023-12-26 15:53:34,187][105620] Updated weights for policy 1, policy_version 72705 (0.0009) [2023-12-26 15:53:34,247][105620] Updated weights for policy 1, policy_version 72715 (0.0010) [2023-12-26 15:53:34,905][105620] Updated weights for policy 1, policy_version 72725 (0.0008) [2023-12-26 15:53:34,948][105692] Updated weights for policy 0, policy_version 72474 (0.0009) [2023-12-26 15:53:34,955][105620] Updated weights for policy 1, policy_version 72735 (0.0006) [2023-12-26 15:53:35,000][105692] Updated weights for policy 0, policy_version 72484 (0.0006) [2023-12-26 15:53:35,023][105620] Updated weights for policy 1, policy_version 72745 (0.0010) [2023-12-26 15:53:35,056][105692] Updated weights for policy 0, policy_version 72494 (0.0006) [2023-12-26 15:53:35,664][105620] Updated weights for policy 1, policy_version 72755 (0.0011) [2023-12-26 15:53:35,670][105692] Updated weights for policy 0, policy_version 72504 (0.0006) [2023-12-26 15:53:35,722][105620] Updated weights for policy 1, policy_version 72765 (0.0010) [2023-12-26 15:53:35,724][105692] Updated weights for policy 0, policy_version 72514 (0.0005) [2023-12-26 15:53:35,778][105692] Updated weights for policy 0, policy_version 72524 (0.0006) [2023-12-26 15:53:35,780][105620] Updated weights for policy 1, policy_version 72775 (0.0010) [2023-12-26 15:53:36,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 37208064. Throughput: 0: 9811.8, 1: 9658.4. Samples: 37192680. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 15:53:36,062][104569] Avg episode reward: [(0, '9272.892'), (1, '8837.089')] [2023-12-26 15:53:36,451][105620] Updated weights for policy 1, policy_version 72785 (0.0007) [2023-12-26 15:53:36,470][105692] Updated weights for policy 0, policy_version 72534 (0.0006) [2023-12-26 15:53:36,511][105620] Updated weights for policy 1, policy_version 72795 (0.0011) [2023-12-26 15:53:36,529][105692] Updated weights for policy 0, policy_version 72544 (0.0006) [2023-12-26 15:53:36,570][105620] Updated weights for policy 1, policy_version 72805 (0.0011) [2023-12-26 15:53:36,592][105692] Updated weights for policy 0, policy_version 72554 (0.0005) [2023-12-26 15:53:36,627][105620] Updated weights for policy 1, policy_version 72815 (0.0011) [2023-12-26 15:53:37,212][105692] Updated weights for policy 0, policy_version 72564 (0.0007) [2023-12-26 15:53:37,259][105692] Updated weights for policy 0, policy_version 72574 (0.0009) [2023-12-26 15:53:37,314][105692] Updated weights for policy 0, policy_version 72584 (0.0009) [2023-12-26 15:53:37,394][105620] Updated weights for policy 1, policy_version 72825 (0.0008) [2023-12-26 15:53:37,448][105620] Updated weights for policy 1, policy_version 72835 (0.0009) [2023-12-26 15:53:37,497][105620] Updated weights for policy 1, policy_version 72845 (0.0008) [2023-12-26 15:53:38,100][105692] Updated weights for policy 0, policy_version 72594 (0.0009) [2023-12-26 15:53:38,162][105692] Updated weights for policy 0, policy_version 72604 (0.0010) [2023-12-26 15:53:38,165][105620] Updated weights for policy 1, policy_version 72855 (0.0006) [2023-12-26 15:53:38,211][105692] Updated weights for policy 0, policy_version 72614 (0.0010) [2023-12-26 15:53:38,229][105620] Updated weights for policy 1, policy_version 72865 (0.0007) [2023-12-26 15:53:38,262][105692] Updated weights for policy 0, policy_version 72624 (0.0010) [2023-12-26 15:53:38,282][105620] Updated weights for policy 1, policy_version 72875 (0.0007) [2023-12-26 15:53:38,988][105620] Updated weights for policy 1, policy_version 72885 (0.0007) [2023-12-26 15:53:39,025][105692] Updated weights for policy 0, policy_version 72634 (0.0011) [2023-12-26 15:53:39,040][105620] Updated weights for policy 1, policy_version 72895 (0.0006) [2023-12-26 15:53:39,077][105692] Updated weights for policy 0, policy_version 72644 (0.0011) [2023-12-26 15:53:39,095][105620] Updated weights for policy 1, policy_version 72905 (0.0007) [2023-12-26 15:53:39,136][105692] Updated weights for policy 0, policy_version 72654 (0.0010) [2023-12-26 15:53:39,885][105620] Updated weights for policy 1, policy_version 72915 (0.0008) [2023-12-26 15:53:39,958][105620] Updated weights for policy 1, policy_version 72925 (0.0008) [2023-12-26 15:53:39,962][105692] Updated weights for policy 0, policy_version 72664 (0.0009) [2023-12-26 15:53:40,015][105620] Updated weights for policy 1, policy_version 72935 (0.0008) [2023-12-26 15:53:40,025][105692] Updated weights for policy 0, policy_version 72674 (0.0006) [2023-12-26 15:53:40,084][105692] Updated weights for policy 0, policy_version 72684 (0.0007) [2023-12-26 15:53:40,757][105620] Updated weights for policy 1, policy_version 72945 (0.0009) [2023-12-26 15:53:40,821][105692] Updated weights for policy 0, policy_version 72694 (0.0007) [2023-12-26 15:53:40,823][105620] Updated weights for policy 1, policy_version 72955 (0.0009) [2023-12-26 15:53:40,874][105620] Updated weights for policy 1, policy_version 72965 (0.0009) [2023-12-26 15:53:40,880][105692] Updated weights for policy 0, policy_version 72704 (0.0005) [2023-12-26 15:53:40,922][105620] Updated weights for policy 1, policy_version 72975 (0.0009) [2023-12-26 15:53:40,928][105692] Updated weights for policy 0, policy_version 72714 (0.0005) [2023-12-26 15:53:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 37306368. Throughput: 0: 9711.7, 1: 9735.4. Samples: 37309104. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 15:53:41,063][104569] Avg episode reward: [(0, '9271.900'), (1, '8920.873')] [2023-12-26 15:53:41,693][105620] Updated weights for policy 1, policy_version 72985 (0.0008) [2023-12-26 15:53:41,703][105692] Updated weights for policy 0, policy_version 72724 (0.0007) [2023-12-26 15:53:41,758][105620] Updated weights for policy 1, policy_version 72995 (0.0008) [2023-12-26 15:53:41,769][105692] Updated weights for policy 0, policy_version 72734 (0.0008) [2023-12-26 15:53:41,819][105620] Updated weights for policy 1, policy_version 73005 (0.0007) [2023-12-26 15:53:41,821][105692] Updated weights for policy 0, policy_version 72744 (0.0006) [2023-12-26 15:53:42,444][105692] Updated weights for policy 0, policy_version 72754 (0.0008) [2023-12-26 15:53:42,496][105692] Updated weights for policy 0, policy_version 72764 (0.0006) [2023-12-26 15:53:42,548][105692] Updated weights for policy 0, policy_version 72774 (0.0008) [2023-12-26 15:53:42,594][105620] Updated weights for policy 1, policy_version 73015 (0.0006) [2023-12-26 15:53:42,604][105692] Updated weights for policy 0, policy_version 72784 (0.0007) [2023-12-26 15:53:42,642][105620] Updated weights for policy 1, policy_version 73025 (0.0008) [2023-12-26 15:53:42,697][105620] Updated weights for policy 1, policy_version 73035 (0.0009) [2023-12-26 15:53:43,171][105692] Updated weights for policy 0, policy_version 72794 (0.0005) [2023-12-26 15:53:43,226][105692] Updated weights for policy 0, policy_version 72804 (0.0006) [2023-12-26 15:53:43,277][105692] Updated weights for policy 0, policy_version 72814 (0.0007) [2023-12-26 15:53:43,545][105620] Updated weights for policy 1, policy_version 73045 (0.0010) [2023-12-26 15:53:43,594][105620] Updated weights for policy 1, policy_version 73055 (0.0010) [2023-12-26 15:53:43,650][105620] Updated weights for policy 1, policy_version 73065 (0.0011) [2023-12-26 15:53:44,035][105692] Updated weights for policy 0, policy_version 72824 (0.0009) [2023-12-26 15:53:44,088][105692] Updated weights for policy 0, policy_version 72834 (0.0010) [2023-12-26 15:53:44,144][105692] Updated weights for policy 0, policy_version 72845 (0.0009) [2023-12-26 15:53:44,316][105620] Updated weights for policy 1, policy_version 73075 (0.0010) [2023-12-26 15:53:44,369][105620] Updated weights for policy 1, policy_version 73085 (0.0009) [2023-12-26 15:53:44,429][105620] Updated weights for policy 1, policy_version 73095 (0.0006) [2023-12-26 15:53:44,932][105692] Updated weights for policy 0, policy_version 72855 (0.0007) [2023-12-26 15:53:44,994][105692] Updated weights for policy 0, policy_version 72865 (0.0008) [2023-12-26 15:53:45,020][105620] Updated weights for policy 1, policy_version 73105 (0.0005) [2023-12-26 15:53:45,051][105692] Updated weights for policy 0, policy_version 72875 (0.0009) [2023-12-26 15:53:45,079][105620] Updated weights for policy 1, policy_version 73115 (0.0006) [2023-12-26 15:53:45,141][105620] Updated weights for policy 1, policy_version 73125 (0.0008) [2023-12-26 15:53:45,201][105620] Updated weights for policy 1, policy_version 73135 (0.0009) [2023-12-26 15:53:45,823][105692] Updated weights for policy 0, policy_version 72885 (0.0009) [2023-12-26 15:53:45,871][105620] Updated weights for policy 1, policy_version 73145 (0.0005) [2023-12-26 15:53:45,873][105692] Updated weights for policy 0, policy_version 72895 (0.0008) [2023-12-26 15:53:45,919][105620] Updated weights for policy 1, policy_version 73155 (0.0005) [2023-12-26 15:53:45,925][105692] Updated weights for policy 0, policy_version 72905 (0.0009) [2023-12-26 15:53:45,967][105620] Updated weights for policy 1, policy_version 73165 (0.0010) [2023-12-26 15:53:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 37404672. Throughput: 0: 9804.7, 1: 9651.7. Samples: 37367220. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 15:53:46,062][104569] Avg episode reward: [(0, '8908.104'), (1, '8755.413')] [2023-12-26 15:53:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000072912_18669568.pth... [2023-12-26 15:53:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000073168_18735104.pth... [2023-12-26 15:53:46,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000071760_18374656.pth [2023-12-26 15:53:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000072016_18440192.pth [2023-12-26 15:53:46,673][105620] Updated weights for policy 1, policy_version 73175 (0.0008) [2023-12-26 15:53:46,714][105692] Updated weights for policy 0, policy_version 72915 (0.0008) [2023-12-26 15:53:46,736][105620] Updated weights for policy 1, policy_version 73185 (0.0008) [2023-12-26 15:53:46,778][105692] Updated weights for policy 0, policy_version 72925 (0.0006) [2023-12-26 15:53:46,791][105620] Updated weights for policy 1, policy_version 73195 (0.0007) [2023-12-26 15:53:46,835][105692] Updated weights for policy 0, policy_version 72935 (0.0007) [2023-12-26 15:53:47,436][105620] Updated weights for policy 1, policy_version 73205 (0.0008) [2023-12-26 15:53:47,487][105620] Updated weights for policy 1, policy_version 73215 (0.0007) [2023-12-26 15:53:47,543][105620] Updated weights for policy 1, policy_version 73225 (0.0005) [2023-12-26 15:53:47,677][105692] Updated weights for policy 0, policy_version 72945 (0.0009) [2023-12-26 15:53:47,730][105692] Updated weights for policy 0, policy_version 72955 (0.0009) [2023-12-26 15:53:47,778][105692] Updated weights for policy 0, policy_version 72965 (0.0009) [2023-12-26 15:53:47,835][105692] Updated weights for policy 0, policy_version 72975 (0.0009) [2023-12-26 15:53:48,153][105620] Updated weights for policy 1, policy_version 73235 (0.0007) [2023-12-26 15:53:48,200][105620] Updated weights for policy 1, policy_version 73245 (0.0008) [2023-12-26 15:53:48,246][105620] Updated weights for policy 1, policy_version 73255 (0.0008) [2023-12-26 15:53:48,611][105692] Updated weights for policy 0, policy_version 72985 (0.0005) [2023-12-26 15:53:48,668][105692] Updated weights for policy 0, policy_version 72995 (0.0006) [2023-12-26 15:53:48,724][105692] Updated weights for policy 0, policy_version 73005 (0.0009) [2023-12-26 15:53:49,038][105620] Updated weights for policy 1, policy_version 73265 (0.0009) [2023-12-26 15:53:49,088][105620] Updated weights for policy 1, policy_version 73275 (0.0009) [2023-12-26 15:53:49,147][105620] Updated weights for policy 1, policy_version 73285 (0.0009) [2023-12-26 15:53:49,208][105620] Updated weights for policy 1, policy_version 73295 (0.0009) [2023-12-26 15:53:49,470][105692] Updated weights for policy 0, policy_version 73015 (0.0009) [2023-12-26 15:53:49,535][105692] Updated weights for policy 0, policy_version 73025 (0.0010) [2023-12-26 15:53:49,602][105692] Updated weights for policy 0, policy_version 73035 (0.0010) [2023-12-26 15:53:49,929][105620] Updated weights for policy 1, policy_version 73305 (0.0010) [2023-12-26 15:53:49,989][105620] Updated weights for policy 1, policy_version 73315 (0.0008) [2023-12-26 15:53:50,052][105620] Updated weights for policy 1, policy_version 73326 (0.0009) [2023-12-26 15:53:50,304][105692] Updated weights for policy 0, policy_version 73045 (0.0010) [2023-12-26 15:53:50,370][105692] Updated weights for policy 0, policy_version 73055 (0.0010) [2023-12-26 15:53:50,434][105692] Updated weights for policy 0, policy_version 73065 (0.0009) [2023-12-26 15:53:50,769][105620] Updated weights for policy 1, policy_version 73336 (0.0009) [2023-12-26 15:53:50,832][105620] Updated weights for policy 1, policy_version 73346 (0.0007) [2023-12-26 15:53:50,894][105620] Updated weights for policy 1, policy_version 73356 (0.0010) [2023-12-26 15:53:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 37494784. Throughput: 0: 9671.1, 1: 9783.1. Samples: 37483200. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 15:53:51,063][104569] Avg episode reward: [(0, '8451.181'), (1, '8671.521')] [2023-12-26 15:53:51,192][105692] Updated weights for policy 0, policy_version 73075 (0.0010) [2023-12-26 15:53:51,245][105692] Updated weights for policy 0, policy_version 73085 (0.0011) [2023-12-26 15:53:51,308][105692] Updated weights for policy 0, policy_version 73095 (0.0011) [2023-12-26 15:53:51,579][105620] Updated weights for policy 1, policy_version 73366 (0.0008) [2023-12-26 15:53:51,643][105620] Updated weights for policy 1, policy_version 73376 (0.0007) [2023-12-26 15:53:51,707][105620] Updated weights for policy 1, policy_version 73386 (0.0006) [2023-12-26 15:53:52,022][105692] Updated weights for policy 0, policy_version 73105 (0.0009) [2023-12-26 15:53:52,080][105692] Updated weights for policy 0, policy_version 73115 (0.0006) [2023-12-26 15:53:52,138][105692] Updated weights for policy 0, policy_version 73125 (0.0007) [2023-12-26 15:53:52,196][105692] Updated weights for policy 0, policy_version 73135 (0.0009) [2023-12-26 15:53:52,449][105620] Updated weights for policy 1, policy_version 73396 (0.0007) [2023-12-26 15:53:52,511][105620] Updated weights for policy 1, policy_version 73406 (0.0006) [2023-12-26 15:53:52,574][105620] Updated weights for policy 1, policy_version 73416 (0.0009) [2023-12-26 15:53:52,952][105692] Updated weights for policy 0, policy_version 73145 (0.0008) [2023-12-26 15:53:53,000][105692] Updated weights for policy 0, policy_version 73155 (0.0009) [2023-12-26 15:53:53,055][105692] Updated weights for policy 0, policy_version 73165 (0.0010) [2023-12-26 15:53:53,274][105620] Updated weights for policy 1, policy_version 73426 (0.0009) [2023-12-26 15:53:53,335][105620] Updated weights for policy 1, policy_version 73436 (0.0005) [2023-12-26 15:53:53,399][105620] Updated weights for policy 1, policy_version 73446 (0.0007) [2023-12-26 15:53:53,444][105620] Updated weights for policy 1, policy_version 73456 (0.0008) [2023-12-26 15:53:53,901][105692] Updated weights for policy 0, policy_version 73175 (0.0010) [2023-12-26 15:53:53,958][105692] Updated weights for policy 0, policy_version 73185 (0.0009) [2023-12-26 15:53:53,999][105620] Updated weights for policy 1, policy_version 73466 (0.0006) [2023-12-26 15:53:54,010][105692] Updated weights for policy 0, policy_version 73195 (0.0009) [2023-12-26 15:53:54,054][105620] Updated weights for policy 1, policy_version 73476 (0.0006) [2023-12-26 15:53:54,112][105620] Updated weights for policy 1, policy_version 73486 (0.0010) [2023-12-26 15:53:54,695][105620] Updated weights for policy 1, policy_version 73496 (0.0011) [2023-12-26 15:53:54,746][105620] Updated weights for policy 1, policy_version 73506 (0.0005) [2023-12-26 15:53:54,807][105620] Updated weights for policy 1, policy_version 73516 (0.0006) [2023-12-26 15:53:54,873][105692] Updated weights for policy 0, policy_version 73205 (0.0009) [2023-12-26 15:53:54,934][105692] Updated weights for policy 0, policy_version 73215 (0.0010) [2023-12-26 15:53:54,993][105692] Updated weights for policy 0, policy_version 73225 (0.0008) [2023-12-26 15:53:55,401][105620] Updated weights for policy 1, policy_version 73526 (0.0008) [2023-12-26 15:53:55,445][105620] Updated weights for policy 1, policy_version 73536 (0.0008) [2023-12-26 15:53:55,489][105620] Updated weights for policy 1, policy_version 73546 (0.0005) [2023-12-26 15:53:55,843][105692] Updated weights for policy 0, policy_version 73235 (0.0010) [2023-12-26 15:53:55,898][105692] Updated weights for policy 0, policy_version 73245 (0.0010) [2023-12-26 15:53:55,960][105692] Updated weights for policy 0, policy_version 73255 (0.0011) [2023-12-26 15:53:56,059][105620] Updated weights for policy 1, policy_version 73556 (0.0007) [2023-12-26 15:53:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 37593088. Throughput: 0: 9498.5, 1: 9890.1. Samples: 37601824. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 15:53:56,062][104569] Avg episode reward: [(0, '8985.727'), (1, '9101.200')] [2023-12-26 15:53:56,108][105620] Updated weights for policy 1, policy_version 73566 (0.0010) [2023-12-26 15:53:56,163][105620] Updated weights for policy 1, policy_version 73576 (0.0010) [2023-12-26 15:53:56,559][105692] Updated weights for policy 0, policy_version 73265 (0.0010) [2023-12-26 15:53:56,631][105692] Updated weights for policy 0, policy_version 73275 (0.0009) [2023-12-26 15:53:56,694][105692] Updated weights for policy 0, policy_version 73285 (0.0011) [2023-12-26 15:53:56,756][105692] Updated weights for policy 0, policy_version 73295 (0.0010) [2023-12-26 15:53:56,848][105620] Updated weights for policy 1, policy_version 73586 (0.0010) [2023-12-26 15:53:56,904][105620] Updated weights for policy 1, policy_version 73596 (0.0011) [2023-12-26 15:53:56,956][105620] Updated weights for policy 1, policy_version 73606 (0.0010) [2023-12-26 15:53:57,002][105620] Updated weights for policy 1, policy_version 73616 (0.0011) [2023-12-26 15:53:57,371][105692] Updated weights for policy 0, policy_version 73305 (0.0010) [2023-12-26 15:53:57,415][105692] Updated weights for policy 0, policy_version 73315 (0.0010) [2023-12-26 15:53:57,465][105692] Updated weights for policy 0, policy_version 73325 (0.0010) [2023-12-26 15:53:57,765][105620] Updated weights for policy 1, policy_version 73626 (0.0010) [2023-12-26 15:53:57,830][105620] Updated weights for policy 1, policy_version 73636 (0.0010) [2023-12-26 15:53:57,879][105620] Updated weights for policy 1, policy_version 73646 (0.0010) [2023-12-26 15:53:58,195][105692] Updated weights for policy 0, policy_version 73335 (0.0009) [2023-12-26 15:53:58,250][105692] Updated weights for policy 0, policy_version 73345 (0.0009) [2023-12-26 15:53:58,311][105692] Updated weights for policy 0, policy_version 73355 (0.0009) [2023-12-26 15:53:58,609][105620] Updated weights for policy 1, policy_version 73656 (0.0008) [2023-12-26 15:53:58,675][105620] Updated weights for policy 1, policy_version 73666 (0.0009) [2023-12-26 15:53:58,746][105620] Updated weights for policy 1, policy_version 73676 (0.0010) [2023-12-26 15:53:59,134][105692] Updated weights for policy 0, policy_version 73365 (0.0008) [2023-12-26 15:53:59,201][105692] Updated weights for policy 0, policy_version 73375 (0.0009) [2023-12-26 15:53:59,267][105692] Updated weights for policy 0, policy_version 73385 (0.0008) [2023-12-26 15:53:59,479][105620] Updated weights for policy 1, policy_version 73686 (0.0008) [2023-12-26 15:53:59,540][105620] Updated weights for policy 1, policy_version 73696 (0.0005) [2023-12-26 15:53:59,601][105620] Updated weights for policy 1, policy_version 73706 (0.0007) [2023-12-26 15:54:00,054][105692] Updated weights for policy 0, policy_version 73395 (0.0008) [2023-12-26 15:54:00,109][105692] Updated weights for policy 0, policy_version 73405 (0.0011) [2023-12-26 15:54:00,165][105692] Updated weights for policy 0, policy_version 73415 (0.0010) [2023-12-26 15:54:00,221][105620] Updated weights for policy 1, policy_version 73716 (0.0008) [2023-12-26 15:54:00,268][105620] Updated weights for policy 1, policy_version 73726 (0.0007) [2023-12-26 15:54:00,323][105620] Updated weights for policy 1, policy_version 73736 (0.0006) [2023-12-26 15:54:00,763][105692] Updated weights for policy 0, policy_version 73425 (0.0009) [2023-12-26 15:54:00,810][105692] Updated weights for policy 0, policy_version 73435 (0.0005) [2023-12-26 15:54:00,863][105692] Updated weights for policy 0, policy_version 73445 (0.0005) [2023-12-26 15:54:00,910][105692] Updated weights for policy 0, policy_version 73455 (0.0005) [2023-12-26 15:54:00,961][105620] Updated weights for policy 1, policy_version 73746 (0.0007) [2023-12-26 15:54:01,016][105620] Updated weights for policy 1, policy_version 73756 (0.0008) [2023-12-26 15:54:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 37691392. Throughput: 0: 9560.8, 1: 9843.6. Samples: 37660344. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 15:54:01,063][104569] Avg episode reward: [(0, '9167.004'), (1, '9185.782')] [2023-12-26 15:54:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000073456_18808832.pth... [2023-12-26 15:54:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000072336_18522112.pth [2023-12-26 15:54:01,081][105620] Updated weights for policy 1, policy_version 73766 (0.0009) [2023-12-26 15:54:01,147][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000073776_18890752.pth... [2023-12-26 15:54:01,149][105620] Updated weights for policy 1, policy_version 73776 (0.0007) [2023-12-26 15:54:01,152][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000072592_18587648.pth [2023-12-26 15:54:01,579][105692] Updated weights for policy 0, policy_version 73465 (0.0009) [2023-12-26 15:54:01,645][105692] Updated weights for policy 0, policy_version 73475 (0.0012) [2023-12-26 15:54:01,711][105692] Updated weights for policy 0, policy_version 73485 (0.0008) [2023-12-26 15:54:01,872][105620] Updated weights for policy 1, policy_version 73786 (0.0009) [2023-12-26 15:54:01,924][105620] Updated weights for policy 1, policy_version 73796 (0.0008) [2023-12-26 15:54:01,973][105620] Updated weights for policy 1, policy_version 73806 (0.0008) [2023-12-26 15:54:02,426][105692] Updated weights for policy 0, policy_version 73495 (0.0007) [2023-12-26 15:54:02,487][105692] Updated weights for policy 0, policy_version 73505 (0.0009) [2023-12-26 15:54:02,548][105692] Updated weights for policy 0, policy_version 73515 (0.0009) [2023-12-26 15:54:02,693][105620] Updated weights for policy 1, policy_version 73816 (0.0005) [2023-12-26 15:54:02,760][105620] Updated weights for policy 1, policy_version 73826 (0.0005) [2023-12-26 15:54:02,820][105620] Updated weights for policy 1, policy_version 73836 (0.0005) [2023-12-26 15:54:03,224][105692] Updated weights for policy 0, policy_version 73525 (0.0009) [2023-12-26 15:54:03,278][105692] Updated weights for policy 0, policy_version 73535 (0.0008) [2023-12-26 15:54:03,330][105692] Updated weights for policy 0, policy_version 73545 (0.0005) [2023-12-26 15:54:03,510][105620] Updated weights for policy 1, policy_version 73846 (0.0006) [2023-12-26 15:54:03,566][105620] Updated weights for policy 1, policy_version 73856 (0.0005) [2023-12-26 15:54:03,636][105620] Updated weights for policy 1, policy_version 73866 (0.0005) [2023-12-26 15:54:03,947][105692] Updated weights for policy 0, policy_version 73555 (0.0005) [2023-12-26 15:54:04,015][105692] Updated weights for policy 0, policy_version 73565 (0.0006) [2023-12-26 15:54:04,085][105692] Updated weights for policy 0, policy_version 73575 (0.0006) [2023-12-26 15:54:04,202][105620] Updated weights for policy 1, policy_version 73876 (0.0006) [2023-12-26 15:54:04,267][105620] Updated weights for policy 1, policy_version 73886 (0.0006) [2023-12-26 15:54:04,323][105620] Updated weights for policy 1, policy_version 73896 (0.0006) [2023-12-26 15:54:04,876][105692] Updated weights for policy 0, policy_version 73585 (0.0008) [2023-12-26 15:54:04,895][105620] Updated weights for policy 1, policy_version 73906 (0.0009) [2023-12-26 15:54:04,929][105692] Updated weights for policy 0, policy_version 73595 (0.0009) [2023-12-26 15:54:04,950][105620] Updated weights for policy 1, policy_version 73916 (0.0005) [2023-12-26 15:54:04,977][105692] Updated weights for policy 0, policy_version 73605 (0.0005) [2023-12-26 15:54:05,012][105620] Updated weights for policy 1, policy_version 73926 (0.0006) [2023-12-26 15:54:05,034][105692] Updated weights for policy 0, policy_version 73615 (0.0007) [2023-12-26 15:54:05,074][105620] Updated weights for policy 1, policy_version 73936 (0.0006) [2023-12-26 15:54:05,592][105620] Updated weights for policy 1, policy_version 73946 (0.0006) [2023-12-26 15:54:05,643][105620] Updated weights for policy 1, policy_version 73956 (0.0005) [2023-12-26 15:54:05,660][105692] Updated weights for policy 0, policy_version 73625 (0.0006) [2023-12-26 15:54:05,693][105620] Updated weights for policy 1, policy_version 73966 (0.0005) [2023-12-26 15:54:05,729][105692] Updated weights for policy 0, policy_version 73635 (0.0005) [2023-12-26 15:54:05,787][105692] Updated weights for policy 0, policy_version 73645 (0.0005) [2023-12-26 15:54:06,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19524.2, 300 sec: 19605.2). Total num frames: 37797888. Throughput: 0: 9573.5, 1: 9990.0. Samples: 37783608. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 15:54:06,063][104569] Avg episode reward: [(0, '9088.334'), (1, '9187.658')] [2023-12-26 15:54:06,252][105620] Updated weights for policy 1, policy_version 73976 (0.0009) [2023-12-26 15:54:06,319][105620] Updated weights for policy 1, policy_version 73986 (0.0007) [2023-12-26 15:54:06,331][105692] Updated weights for policy 0, policy_version 73655 (0.0008) [2023-12-26 15:54:06,383][105620] Updated weights for policy 1, policy_version 73996 (0.0006) [2023-12-26 15:54:06,395][105692] Updated weights for policy 0, policy_version 73665 (0.0009) [2023-12-26 15:54:06,453][105692] Updated weights for policy 0, policy_version 73675 (0.0008) [2023-12-26 15:54:07,110][105620] Updated weights for policy 1, policy_version 74006 (0.0010) [2023-12-26 15:54:07,176][105620] Updated weights for policy 1, policy_version 74016 (0.0011) [2023-12-26 15:54:07,211][105692] Updated weights for policy 0, policy_version 73685 (0.0007) [2023-12-26 15:54:07,236][105620] Updated weights for policy 1, policy_version 74026 (0.0011) [2023-12-26 15:54:07,271][105692] Updated weights for policy 0, policy_version 73695 (0.0005) [2023-12-26 15:54:07,326][105692] Updated weights for policy 0, policy_version 73705 (0.0008) [2023-12-26 15:54:07,829][105620] Updated weights for policy 1, policy_version 74036 (0.0008) [2023-12-26 15:54:07,879][105620] Updated weights for policy 1, policy_version 74046 (0.0005) [2023-12-26 15:54:07,932][105620] Updated weights for policy 1, policy_version 74056 (0.0005) [2023-12-26 15:54:08,141][105692] Updated weights for policy 0, policy_version 73715 (0.0007) [2023-12-26 15:54:08,193][105692] Updated weights for policy 0, policy_version 73725 (0.0009) [2023-12-26 15:54:08,255][105692] Updated weights for policy 0, policy_version 73735 (0.0008) [2023-12-26 15:54:08,549][105620] Updated weights for policy 1, policy_version 74066 (0.0005) [2023-12-26 15:54:08,609][105620] Updated weights for policy 1, policy_version 74076 (0.0006) [2023-12-26 15:54:08,664][105620] Updated weights for policy 1, policy_version 74086 (0.0006) [2023-12-26 15:54:08,716][105620] Updated weights for policy 1, policy_version 74096 (0.0005) [2023-12-26 15:54:09,094][105692] Updated weights for policy 0, policy_version 73745 (0.0008) [2023-12-26 15:54:09,156][105692] Updated weights for policy 0, policy_version 73755 (0.0010) [2023-12-26 15:54:09,217][105692] Updated weights for policy 0, policy_version 73765 (0.0010) [2023-12-26 15:54:09,282][105692] Updated weights for policy 0, policy_version 73775 (0.0008) [2023-12-26 15:54:09,323][105620] Updated weights for policy 1, policy_version 74106 (0.0007) [2023-12-26 15:54:09,394][105620] Updated weights for policy 1, policy_version 74116 (0.0008) [2023-12-26 15:54:09,459][105620] Updated weights for policy 1, policy_version 74126 (0.0009) [2023-12-26 15:54:10,001][105692] Updated weights for policy 0, policy_version 73785 (0.0007) [2023-12-26 15:54:10,057][105692] Updated weights for policy 0, policy_version 73795 (0.0007) [2023-12-26 15:54:10,111][105692] Updated weights for policy 0, policy_version 73805 (0.0008) [2023-12-26 15:54:10,182][105620] Updated weights for policy 1, policy_version 74136 (0.0010) [2023-12-26 15:54:10,247][105620] Updated weights for policy 1, policy_version 74146 (0.0011) [2023-12-26 15:54:10,315][105620] Updated weights for policy 1, policy_version 74156 (0.0008) [2023-12-26 15:54:10,772][105692] Updated weights for policy 0, policy_version 73815 (0.0006) [2023-12-26 15:54:10,823][105692] Updated weights for policy 0, policy_version 73825 (0.0005) [2023-12-26 15:54:10,882][105692] Updated weights for policy 0, policy_version 73835 (0.0009) [2023-12-26 15:54:11,004][105620] Updated weights for policy 1, policy_version 74166 (0.0008) [2023-12-26 15:54:11,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 37896192. Throughput: 0: 9606.3, 1: 10189.3. Samples: 37905892. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 15:54:11,063][104569] Avg episode reward: [(0, '9178.644'), (1, '9099.914')] [2023-12-26 15:54:11,073][105620] Updated weights for policy 1, policy_version 74176 (0.0008) [2023-12-26 15:54:11,135][105620] Updated weights for policy 1, policy_version 74186 (0.0008) [2023-12-26 15:54:11,581][105692] Updated weights for policy 0, policy_version 73845 (0.0009) [2023-12-26 15:54:11,657][105692] Updated weights for policy 0, policy_version 73855 (0.0009) [2023-12-26 15:54:11,725][105692] Updated weights for policy 0, policy_version 73865 (0.0009) [2023-12-26 15:54:11,927][105620] Updated weights for policy 1, policy_version 74196 (0.0008) [2023-12-26 15:54:11,985][105620] Updated weights for policy 1, policy_version 74206 (0.0009) [2023-12-26 15:54:12,043][105620] Updated weights for policy 1, policy_version 74216 (0.0009) [2023-12-26 15:54:12,415][105692] Updated weights for policy 0, policy_version 73875 (0.0009) [2023-12-26 15:54:12,483][105692] Updated weights for policy 0, policy_version 73885 (0.0005) [2023-12-26 15:54:12,540][105692] Updated weights for policy 0, policy_version 73895 (0.0005) [2023-12-26 15:54:12,797][105620] Updated weights for policy 1, policy_version 74226 (0.0009) [2023-12-26 15:54:12,845][105620] Updated weights for policy 1, policy_version 74236 (0.0005) [2023-12-26 15:54:12,891][105620] Updated weights for policy 1, policy_version 74246 (0.0005) [2023-12-26 15:54:12,942][105620] Updated weights for policy 1, policy_version 74256 (0.0007) [2023-12-26 15:54:13,264][105692] Updated weights for policy 0, policy_version 73905 (0.0006) [2023-12-26 15:54:13,312][105692] Updated weights for policy 0, policy_version 73915 (0.0010) [2023-12-26 15:54:13,360][105692] Updated weights for policy 0, policy_version 73925 (0.0010) [2023-12-26 15:54:13,414][105692] Updated weights for policy 0, policy_version 73935 (0.0010) [2023-12-26 15:54:13,676][105620] Updated weights for policy 1, policy_version 74266 (0.0008) [2023-12-26 15:54:13,736][105620] Updated weights for policy 1, policy_version 74276 (0.0007) [2023-12-26 15:54:13,798][105620] Updated weights for policy 1, policy_version 74286 (0.0007) [2023-12-26 15:54:14,119][105692] Updated weights for policy 0, policy_version 73945 (0.0006) [2023-12-26 15:54:14,166][105692] Updated weights for policy 0, policy_version 73955 (0.0007) [2023-12-26 15:54:14,211][105692] Updated weights for policy 0, policy_version 73965 (0.0010) [2023-12-26 15:54:14,574][105620] Updated weights for policy 1, policy_version 74296 (0.0008) [2023-12-26 15:54:14,634][105620] Updated weights for policy 1, policy_version 74306 (0.0009) [2023-12-26 15:54:14,686][105620] Updated weights for policy 1, policy_version 74316 (0.0008) [2023-12-26 15:54:14,878][105692] Updated weights for policy 0, policy_version 73975 (0.0011) [2023-12-26 15:54:14,927][105692] Updated weights for policy 0, policy_version 73985 (0.0011) [2023-12-26 15:54:14,979][105692] Updated weights for policy 0, policy_version 73995 (0.0011) [2023-12-26 15:54:15,434][105620] Updated weights for policy 1, policy_version 74326 (0.0009) [2023-12-26 15:54:15,480][105620] Updated weights for policy 1, policy_version 74336 (0.0008) [2023-12-26 15:54:15,526][105620] Updated weights for policy 1, policy_version 74346 (0.0007) [2023-12-26 15:54:15,685][105692] Updated weights for policy 0, policy_version 74005 (0.0009) [2023-12-26 15:54:15,740][105692] Updated weights for policy 0, policy_version 74015 (0.0008) [2023-12-26 15:54:15,788][105692] Updated weights for policy 0, policy_version 74025 (0.0008) [2023-12-26 15:54:16,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 37994496. Throughput: 0: 9603.1, 1: 10129.5. Samples: 37963268. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 15:54:16,062][104569] Avg episode reward: [(0, '9175.890'), (1, '9187.959')] [2023-12-26 15:54:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000074032_18956288.pth... [2023-12-26 15:54:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000074352_19038208.pth... [2023-12-26 15:54:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000072912_18669568.pth [2023-12-26 15:54:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000073168_18735104.pth [2023-12-26 15:54:16,321][105620] Updated weights for policy 1, policy_version 74356 (0.0007) [2023-12-26 15:54:16,340][105692] Updated weights for policy 0, policy_version 74035 (0.0005) [2023-12-26 15:54:16,377][105620] Updated weights for policy 1, policy_version 74366 (0.0009) [2023-12-26 15:54:16,397][105692] Updated weights for policy 0, policy_version 74045 (0.0005) [2023-12-26 15:54:16,435][105620] Updated weights for policy 1, policy_version 74376 (0.0008) [2023-12-26 15:54:16,449][105692] Updated weights for policy 0, policy_version 74055 (0.0006) [2023-12-26 15:54:17,181][105692] Updated weights for policy 0, policy_version 74065 (0.0008) [2023-12-26 15:54:17,200][105620] Updated weights for policy 1, policy_version 74386 (0.0008) [2023-12-26 15:54:17,243][105692] Updated weights for policy 0, policy_version 74075 (0.0010) [2023-12-26 15:54:17,258][105620] Updated weights for policy 1, policy_version 74396 (0.0009) [2023-12-26 15:54:17,293][105692] Updated weights for policy 0, policy_version 74085 (0.0008) [2023-12-26 15:54:17,316][105620] Updated weights for policy 1, policy_version 74406 (0.0006) [2023-12-26 15:54:17,346][105692] Updated weights for policy 0, policy_version 74095 (0.0008) [2023-12-26 15:54:17,380][105620] Updated weights for policy 1, policy_version 74416 (0.0007) [2023-12-26 15:54:18,022][105692] Updated weights for policy 0, policy_version 74105 (0.0008) [2023-12-26 15:54:18,083][105692] Updated weights for policy 0, policy_version 74115 (0.0009) [2023-12-26 15:54:18,132][105620] Updated weights for policy 1, policy_version 74426 (0.0005) [2023-12-26 15:54:18,142][105692] Updated weights for policy 0, policy_version 74125 (0.0009) [2023-12-26 15:54:18,185][105620] Updated weights for policy 1, policy_version 74436 (0.0005) [2023-12-26 15:54:18,246][105620] Updated weights for policy 1, policy_version 74446 (0.0005) [2023-12-26 15:54:18,863][105692] Updated weights for policy 0, policy_version 74135 (0.0007) [2023-12-26 15:54:18,897][105620] Updated weights for policy 1, policy_version 74456 (0.0007) [2023-12-26 15:54:18,930][105692] Updated weights for policy 0, policy_version 74145 (0.0011) [2023-12-26 15:54:18,956][105620] Updated weights for policy 1, policy_version 74466 (0.0006) [2023-12-26 15:54:18,994][105692] Updated weights for policy 0, policy_version 74155 (0.0010) [2023-12-26 15:54:19,007][105620] Updated weights for policy 1, policy_version 74476 (0.0007) [2023-12-26 15:54:19,666][105620] Updated weights for policy 1, policy_version 74486 (0.0007) [2023-12-26 15:54:19,720][105620] Updated weights for policy 1, policy_version 74496 (0.0005) [2023-12-26 15:54:19,779][105620] Updated weights for policy 1, policy_version 74506 (0.0006) [2023-12-26 15:54:19,785][105692] Updated weights for policy 0, policy_version 74165 (0.0011) [2023-12-26 15:54:19,846][105692] Updated weights for policy 0, policy_version 74175 (0.0010) [2023-12-26 15:54:19,910][105692] Updated weights for policy 0, policy_version 74185 (0.0011) [2023-12-26 15:54:20,486][105620] Updated weights for policy 1, policy_version 74516 (0.0007) [2023-12-26 15:54:20,553][105620] Updated weights for policy 1, policy_version 74526 (0.0008) [2023-12-26 15:54:20,620][105620] Updated weights for policy 1, policy_version 74536 (0.0008) [2023-12-26 15:54:20,664][105692] Updated weights for policy 0, policy_version 74195 (0.0011) [2023-12-26 15:54:20,739][105692] Updated weights for policy 0, policy_version 74205 (0.0010) [2023-12-26 15:54:20,807][105692] Updated weights for policy 0, policy_version 74215 (0.0010) [2023-12-26 15:54:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 38092800. Throughput: 0: 9693.5, 1: 10060.7. Samples: 38081620. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 15:54:21,063][104569] Avg episode reward: [(0, '8911.245'), (1, '9176.613')] [2023-12-26 15:54:21,340][105620] Updated weights for policy 1, policy_version 74546 (0.0007) [2023-12-26 15:54:21,409][105620] Updated weights for policy 1, policy_version 74556 (0.0008) [2023-12-26 15:54:21,468][105620] Updated weights for policy 1, policy_version 74566 (0.0008) [2023-12-26 15:54:21,530][105620] Updated weights for policy 1, policy_version 74576 (0.0008) [2023-12-26 15:54:21,552][105692] Updated weights for policy 0, policy_version 74225 (0.0011) [2023-12-26 15:54:21,613][105692] Updated weights for policy 0, policy_version 74235 (0.0008) [2023-12-26 15:54:21,673][105692] Updated weights for policy 0, policy_version 74245 (0.0008) [2023-12-26 15:54:21,759][105692] Updated weights for policy 0, policy_version 74255 (0.0008) [2023-12-26 15:54:22,318][105620] Updated weights for policy 1, policy_version 74586 (0.0008) [2023-12-26 15:54:22,391][105620] Updated weights for policy 1, policy_version 74596 (0.0008) [2023-12-26 15:54:22,453][105620] Updated weights for policy 1, policy_version 74606 (0.0008) [2023-12-26 15:54:22,520][105692] Updated weights for policy 0, policy_version 74265 (0.0009) [2023-12-26 15:54:22,573][105692] Updated weights for policy 0, policy_version 74275 (0.0009) [2023-12-26 15:54:22,621][105692] Updated weights for policy 0, policy_version 74285 (0.0008) [2023-12-26 15:54:23,176][105620] Updated weights for policy 1, policy_version 74616 (0.0009) [2023-12-26 15:54:23,232][105620] Updated weights for policy 1, policy_version 74626 (0.0009) [2023-12-26 15:54:23,282][105620] Updated weights for policy 1, policy_version 74636 (0.0009) [2023-12-26 15:54:23,402][105692] Updated weights for policy 0, policy_version 74295 (0.0009) [2023-12-26 15:54:23,449][105692] Updated weights for policy 0, policy_version 74305 (0.0007) [2023-12-26 15:54:23,505][105692] Updated weights for policy 0, policy_version 74315 (0.0005) [2023-12-26 15:54:24,046][105620] Updated weights for policy 1, policy_version 74646 (0.0009) [2023-12-26 15:54:24,109][105620] Updated weights for policy 1, policy_version 74656 (0.0009) [2023-12-26 15:54:24,131][105692] Updated weights for policy 0, policy_version 74325 (0.0005) [2023-12-26 15:54:24,169][105620] Updated weights for policy 1, policy_version 74666 (0.0007) [2023-12-26 15:54:24,187][105692] Updated weights for policy 0, policy_version 74335 (0.0007) [2023-12-26 15:54:24,250][105692] Updated weights for policy 0, policy_version 74345 (0.0006) [2023-12-26 15:54:24,797][105692] Updated weights for policy 0, policy_version 74355 (0.0006) [2023-12-26 15:54:24,867][105692] Updated weights for policy 0, policy_version 74365 (0.0005) [2023-12-26 15:54:24,922][105692] Updated weights for policy 0, policy_version 74375 (0.0005) [2023-12-26 15:54:25,054][105620] Updated weights for policy 1, policy_version 74676 (0.0007) [2023-12-26 15:54:25,115][105620] Updated weights for policy 1, policy_version 74686 (0.0008) [2023-12-26 15:54:25,175][105620] Updated weights for policy 1, policy_version 74696 (0.0009) [2023-12-26 15:54:25,470][105692] Updated weights for policy 0, policy_version 74385 (0.0006) [2023-12-26 15:54:25,534][105692] Updated weights for policy 0, policy_version 74395 (0.0005) [2023-12-26 15:54:25,598][105692] Updated weights for policy 0, policy_version 74405 (0.0005) [2023-12-26 15:54:25,659][105692] Updated weights for policy 0, policy_version 74415 (0.0005) [2023-12-26 15:54:26,042][105620] Updated weights for policy 1, policy_version 74706 (0.0009) [2023-12-26 15:54:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 38182912. Throughput: 0: 9723.6, 1: 9957.9. Samples: 38194772. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 15:54:26,062][104569] Avg episode reward: [(0, '8912.004'), (1, '9021.868')] [2023-12-26 15:54:26,111][105620] Updated weights for policy 1, policy_version 74716 (0.0010) [2023-12-26 15:54:26,175][105620] Updated weights for policy 1, policy_version 74726 (0.0009) [2023-12-26 15:54:26,221][105692] Updated weights for policy 0, policy_version 74425 (0.0008) [2023-12-26 15:54:26,224][105620] Updated weights for policy 1, policy_version 74736 (0.0005) [2023-12-26 15:54:26,279][105692] Updated weights for policy 0, policy_version 74435 (0.0009) [2023-12-26 15:54:26,331][105692] Updated weights for policy 0, policy_version 74445 (0.0009) [2023-12-26 15:54:27,001][105620] Updated weights for policy 1, policy_version 74746 (0.0009) [2023-12-26 15:54:27,049][105692] Updated weights for policy 0, policy_version 74455 (0.0007) [2023-12-26 15:54:27,059][105620] Updated weights for policy 1, policy_version 74756 (0.0009) [2023-12-26 15:54:27,098][105692] Updated weights for policy 0, policy_version 74465 (0.0005) [2023-12-26 15:54:27,122][105620] Updated weights for policy 1, policy_version 74766 (0.0009) [2023-12-26 15:54:27,146][105692] Updated weights for policy 0, policy_version 74475 (0.0005) [2023-12-26 15:54:27,838][105692] Updated weights for policy 0, policy_version 74485 (0.0005) [2023-12-26 15:54:27,840][105620] Updated weights for policy 1, policy_version 74776 (0.0008) [2023-12-26 15:54:27,886][105692] Updated weights for policy 0, policy_version 74495 (0.0006) [2023-12-26 15:54:27,901][105620] Updated weights for policy 1, policy_version 74786 (0.0008) [2023-12-26 15:54:27,942][105692] Updated weights for policy 0, policy_version 74505 (0.0007) [2023-12-26 15:54:27,960][105620] Updated weights for policy 1, policy_version 74796 (0.0007) [2023-12-26 15:54:28,685][105692] Updated weights for policy 0, policy_version 74515 (0.0007) [2023-12-26 15:54:28,690][105620] Updated weights for policy 1, policy_version 74806 (0.0008) [2023-12-26 15:54:28,741][105692] Updated weights for policy 0, policy_version 74525 (0.0006) [2023-12-26 15:54:28,743][105620] Updated weights for policy 1, policy_version 74816 (0.0007) [2023-12-26 15:54:28,796][105692] Updated weights for policy 0, policy_version 74535 (0.0006) [2023-12-26 15:54:28,799][105620] Updated weights for policy 1, policy_version 74826 (0.0007) [2023-12-26 15:54:29,510][105620] Updated weights for policy 1, policy_version 74836 (0.0008) [2023-12-26 15:54:29,550][105692] Updated weights for policy 0, policy_version 74545 (0.0008) [2023-12-26 15:54:29,561][105620] Updated weights for policy 1, policy_version 74846 (0.0008) [2023-12-26 15:54:29,606][105620] Updated weights for policy 1, policy_version 74856 (0.0007) [2023-12-26 15:54:29,612][105692] Updated weights for policy 0, policy_version 74555 (0.0010) [2023-12-26 15:54:29,666][105692] Updated weights for policy 0, policy_version 74565 (0.0010) [2023-12-26 15:54:30,304][105620] Updated weights for policy 1, policy_version 74866 (0.0006) [2023-12-26 15:54:30,363][105620] Updated weights for policy 1, policy_version 74876 (0.0011) [2023-12-26 15:54:30,418][105620] Updated weights for policy 1, policy_version 74886 (0.0010) [2023-12-26 15:54:30,470][105692] Updated weights for policy 0, policy_version 74577 (0.0009) [2023-12-26 15:54:30,476][105620] Updated weights for policy 1, policy_version 74896 (0.0010) [2023-12-26 15:54:30,535][105692] Updated weights for policy 0, policy_version 74587 (0.0008) [2023-12-26 15:54:30,589][105692] Updated weights for policy 0, policy_version 74597 (0.0009) [2023-12-26 15:54:30,640][105692] Updated weights for policy 0, policy_version 74607 (0.0009) [2023-12-26 15:54:31,062][105620] Updated weights for policy 1, policy_version 74906 (0.0009) [2023-12-26 15:54:31,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 38281216. Throughput: 0: 9711.3, 1: 9979.5. Samples: 38253312. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 15:54:31,063][104569] Avg episode reward: [(0, '9092.644'), (1, '8938.069')] [2023-12-26 15:54:31,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000074608_19103744.pth... [2023-12-26 15:54:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000073456_18808832.pth [2023-12-26 15:54:31,123][105620] Updated weights for policy 1, policy_version 74916 (0.0010) [2023-12-26 15:54:31,187][105620] Updated weights for policy 1, policy_version 74926 (0.0011) [2023-12-26 15:54:31,197][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000074928_19185664.pth... [2023-12-26 15:54:31,202][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000073776_18890752.pth [2023-12-26 15:54:31,472][105692] Updated weights for policy 0, policy_version 74617 (0.0006) [2023-12-26 15:54:31,526][105692] Updated weights for policy 0, policy_version 74627 (0.0006) [2023-12-26 15:54:31,588][105692] Updated weights for policy 0, policy_version 74637 (0.0008) [2023-12-26 15:54:31,967][105620] Updated weights for policy 1, policy_version 74937 (0.0010) [2023-12-26 15:54:32,026][105620] Updated weights for policy 1, policy_version 74947 (0.0009) [2023-12-26 15:54:32,080][105620] Updated weights for policy 1, policy_version 74957 (0.0009) [2023-12-26 15:54:32,232][105692] Updated weights for policy 0, policy_version 74647 (0.0008) [2023-12-26 15:54:32,293][105692] Updated weights for policy 0, policy_version 74657 (0.0008) [2023-12-26 15:54:32,347][105692] Updated weights for policy 0, policy_version 74667 (0.0008) [2023-12-26 15:54:32,936][105620] Updated weights for policy 1, policy_version 74967 (0.0010) [2023-12-26 15:54:32,952][105692] Updated weights for policy 0, policy_version 74677 (0.0007) [2023-12-26 15:54:32,999][105620] Updated weights for policy 1, policy_version 74977 (0.0011) [2023-12-26 15:54:33,015][105692] Updated weights for policy 0, policy_version 74687 (0.0009) [2023-12-26 15:54:33,058][105620] Updated weights for policy 1, policy_version 74987 (0.0011) [2023-12-26 15:54:33,072][105692] Updated weights for policy 0, policy_version 74697 (0.0006) [2023-12-26 15:54:33,792][105620] Updated weights for policy 1, policy_version 74997 (0.0011) [2023-12-26 15:54:33,813][105692] Updated weights for policy 0, policy_version 74707 (0.0009) [2023-12-26 15:54:33,839][105620] Updated weights for policy 1, policy_version 75007 (0.0010) [2023-12-26 15:54:33,861][105692] Updated weights for policy 0, policy_version 74717 (0.0005) [2023-12-26 15:54:33,886][105620] Updated weights for policy 1, policy_version 75017 (0.0010) [2023-12-26 15:54:33,920][105692] Updated weights for policy 0, policy_version 74727 (0.0005) [2023-12-26 15:54:34,617][105620] Updated weights for policy 1, policy_version 75027 (0.0010) [2023-12-26 15:54:34,676][105620] Updated weights for policy 1, policy_version 75037 (0.0011) [2023-12-26 15:54:34,720][105692] Updated weights for policy 0, policy_version 74737 (0.0008) [2023-12-26 15:54:34,740][105620] Updated weights for policy 1, policy_version 75047 (0.0011) [2023-12-26 15:54:34,780][105692] Updated weights for policy 0, policy_version 74747 (0.0010) [2023-12-26 15:54:34,833][105692] Updated weights for policy 0, policy_version 74757 (0.0010) [2023-12-26 15:54:34,885][105692] Updated weights for policy 0, policy_version 74767 (0.0010) [2023-12-26 15:54:35,494][105620] Updated weights for policy 1, policy_version 75057 (0.0010) [2023-12-26 15:54:35,542][105692] Updated weights for policy 0, policy_version 74777 (0.0008) [2023-12-26 15:54:35,553][105620] Updated weights for policy 1, policy_version 75067 (0.0010) [2023-12-26 15:54:35,588][105692] Updated weights for policy 0, policy_version 74787 (0.0006) [2023-12-26 15:54:35,615][105620] Updated weights for policy 1, policy_version 75077 (0.0010) [2023-12-26 15:54:35,637][105692] Updated weights for policy 0, policy_version 74797 (0.0006) [2023-12-26 15:54:35,675][105620] Updated weights for policy 1, policy_version 75087 (0.0010) [2023-12-26 15:54:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 38379520. Throughput: 0: 9776.5, 1: 9886.1. Samples: 38368016. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 15:54:36,062][104569] Avg episode reward: [(0, '9183.013'), (1, '9094.097')] [2023-12-26 15:54:36,348][105620] Updated weights for policy 1, policy_version 75097 (0.0006) [2023-12-26 15:54:36,407][105620] Updated weights for policy 1, policy_version 75107 (0.0006) [2023-12-26 15:54:36,458][105692] Updated weights for policy 0, policy_version 74807 (0.0008) [2023-12-26 15:54:36,459][105620] Updated weights for policy 1, policy_version 75117 (0.0006) [2023-12-26 15:54:36,525][105692] Updated weights for policy 0, policy_version 74817 (0.0008) [2023-12-26 15:54:36,589][105692] Updated weights for policy 0, policy_version 74827 (0.0008) [2023-12-26 15:54:37,155][105620] Updated weights for policy 1, policy_version 75127 (0.0009) [2023-12-26 15:54:37,213][105620] Updated weights for policy 1, policy_version 75137 (0.0008) [2023-12-26 15:54:37,272][105620] Updated weights for policy 1, policy_version 75147 (0.0009) [2023-12-26 15:54:37,323][105692] Updated weights for policy 0, policy_version 74837 (0.0008) [2023-12-26 15:54:37,383][105692] Updated weights for policy 0, policy_version 74847 (0.0009) [2023-12-26 15:54:37,443][105692] Updated weights for policy 0, policy_version 74857 (0.0009) [2023-12-26 15:54:37,944][105620] Updated weights for policy 1, policy_version 75157 (0.0007) [2023-12-26 15:54:37,993][105620] Updated weights for policy 1, policy_version 75167 (0.0005) [2023-12-26 15:54:38,057][105620] Updated weights for policy 1, policy_version 75177 (0.0008) [2023-12-26 15:54:38,250][105692] Updated weights for policy 0, policy_version 74867 (0.0008) [2023-12-26 15:54:38,298][105692] Updated weights for policy 0, policy_version 74877 (0.0008) [2023-12-26 15:54:38,357][105692] Updated weights for policy 0, policy_version 74887 (0.0007) [2023-12-26 15:54:38,743][105620] Updated weights for policy 1, policy_version 75187 (0.0010) [2023-12-26 15:54:38,802][105620] Updated weights for policy 1, policy_version 75197 (0.0006) [2023-12-26 15:54:38,861][105620] Updated weights for policy 1, policy_version 75207 (0.0005) [2023-12-26 15:54:39,071][105692] Updated weights for policy 0, policy_version 74897 (0.0009) [2023-12-26 15:54:39,123][105692] Updated weights for policy 0, policy_version 74907 (0.0009) [2023-12-26 15:54:39,181][105692] Updated weights for policy 0, policy_version 74917 (0.0009) [2023-12-26 15:54:39,255][105692] Updated weights for policy 0, policy_version 74927 (0.0009) [2023-12-26 15:54:39,495][105620] Updated weights for policy 1, policy_version 75217 (0.0006) [2023-12-26 15:54:39,556][105620] Updated weights for policy 1, policy_version 75227 (0.0006) [2023-12-26 15:54:39,611][105620] Updated weights for policy 1, policy_version 75237 (0.0005) [2023-12-26 15:54:39,674][105620] Updated weights for policy 1, policy_version 75247 (0.0006) [2023-12-26 15:54:40,020][105692] Updated weights for policy 0, policy_version 74937 (0.0009) [2023-12-26 15:54:40,078][105692] Updated weights for policy 0, policy_version 74947 (0.0008) [2023-12-26 15:54:40,145][105692] Updated weights for policy 0, policy_version 74957 (0.0008) [2023-12-26 15:54:40,385][105620] Updated weights for policy 1, policy_version 75257 (0.0008) [2023-12-26 15:54:40,448][105620] Updated weights for policy 1, policy_version 75267 (0.0008) [2023-12-26 15:54:40,506][105620] Updated weights for policy 1, policy_version 75277 (0.0008) [2023-12-26 15:54:40,867][105692] Updated weights for policy 0, policy_version 74967 (0.0009) [2023-12-26 15:54:40,933][105692] Updated weights for policy 0, policy_version 74977 (0.0010) [2023-12-26 15:54:40,985][105692] Updated weights for policy 0, policy_version 74987 (0.0010) [2023-12-26 15:54:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 38477824. Throughput: 0: 9825.7, 1: 9809.0. Samples: 38485392. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 15:54:41,063][104569] Avg episode reward: [(0, '9178.688'), (1, '9089.801')] [2023-12-26 15:54:41,119][105620] Updated weights for policy 1, policy_version 75287 (0.0008) [2023-12-26 15:54:41,185][105620] Updated weights for policy 1, policy_version 75297 (0.0008) [2023-12-26 15:54:41,252][105620] Updated weights for policy 1, policy_version 75307 (0.0008) [2023-12-26 15:54:41,784][105692] Updated weights for policy 0, policy_version 74997 (0.0008) [2023-12-26 15:54:41,847][105692] Updated weights for policy 0, policy_version 75007 (0.0009) [2023-12-26 15:54:41,912][105692] Updated weights for policy 0, policy_version 75017 (0.0009) [2023-12-26 15:54:42,054][105620] Updated weights for policy 1, policy_version 75317 (0.0008) [2023-12-26 15:54:42,116][105620] Updated weights for policy 1, policy_version 75327 (0.0009) [2023-12-26 15:54:42,171][105620] Updated weights for policy 1, policy_version 75337 (0.0009) [2023-12-26 15:54:42,699][105692] Updated weights for policy 0, policy_version 75027 (0.0009) [2023-12-26 15:54:42,758][105692] Updated weights for policy 0, policy_version 75037 (0.0009) [2023-12-26 15:54:42,809][105692] Updated weights for policy 0, policy_version 75047 (0.0009) [2023-12-26 15:54:42,925][105620] Updated weights for policy 1, policy_version 75347 (0.0009) [2023-12-26 15:54:42,987][105620] Updated weights for policy 1, policy_version 75357 (0.0009) [2023-12-26 15:54:43,046][105620] Updated weights for policy 1, policy_version 75367 (0.0009) [2023-12-26 15:54:43,594][105692] Updated weights for policy 0, policy_version 75057 (0.0009) [2023-12-26 15:54:43,653][105692] Updated weights for policy 0, policy_version 75067 (0.0010) [2023-12-26 15:54:43,707][105620] Updated weights for policy 1, policy_version 75377 (0.0009) [2023-12-26 15:54:43,712][105692] Updated weights for policy 0, policy_version 75078 (0.0012) [2023-12-26 15:54:43,765][105620] Updated weights for policy 1, policy_version 75387 (0.0005) [2023-12-26 15:54:43,768][105692] Updated weights for policy 0, policy_version 75088 (0.0009) [2023-12-26 15:54:43,819][105620] Updated weights for policy 1, policy_version 75397 (0.0005) [2023-12-26 15:54:43,872][105620] Updated weights for policy 1, policy_version 75407 (0.0005) [2023-12-26 15:54:44,384][105620] Updated weights for policy 1, policy_version 75417 (0.0008) [2023-12-26 15:54:44,434][105620] Updated weights for policy 1, policy_version 75427 (0.0009) [2023-12-26 15:54:44,489][105620] Updated weights for policy 1, policy_version 75437 (0.0009) [2023-12-26 15:54:44,649][105692] Updated weights for policy 0, policy_version 75098 (0.0009) [2023-12-26 15:54:44,702][105692] Updated weights for policy 0, policy_version 75108 (0.0009) [2023-12-26 15:54:44,772][105692] Updated weights for policy 0, policy_version 75118 (0.0009) [2023-12-26 15:54:45,189][105620] Updated weights for policy 1, policy_version 75447 (0.0009) [2023-12-26 15:54:45,254][105620] Updated weights for policy 1, policy_version 75457 (0.0008) [2023-12-26 15:54:45,315][105620] Updated weights for policy 1, policy_version 75467 (0.0009) [2023-12-26 15:54:45,564][105692] Updated weights for policy 0, policy_version 75128 (0.0009) [2023-12-26 15:54:45,618][105692] Updated weights for policy 0, policy_version 75138 (0.0009) [2023-12-26 15:54:45,675][105692] Updated weights for policy 0, policy_version 75148 (0.0008) [2023-12-26 15:54:45,997][105620] Updated weights for policy 1, policy_version 75477 (0.0006) [2023-12-26 15:54:46,059][105620] Updated weights for policy 1, policy_version 75487 (0.0006) [2023-12-26 15:54:46,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19387.6, 300 sec: 19577.5). Total num frames: 38567936. Throughput: 0: 9750.8, 1: 9824.3. Samples: 38541228. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-12-26 15:54:46,063][104569] Avg episode reward: [(0, '9091.902'), (1, '9094.667')] [2023-12-26 15:54:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000075152_19243008.pth... [2023-12-26 15:54:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000074032_18956288.pth [2023-12-26 15:54:46,114][105620] Updated weights for policy 1, policy_version 75497 (0.0010) [2023-12-26 15:54:46,152][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000075504_19333120.pth... [2023-12-26 15:54:46,155][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000074352_19038208.pth [2023-12-26 15:54:46,455][105692] Updated weights for policy 0, policy_version 75158 (0.0009) [2023-12-26 15:54:46,514][105692] Updated weights for policy 0, policy_version 75168 (0.0008) [2023-12-26 15:54:46,559][105692] Updated weights for policy 0, policy_version 75178 (0.0008) [2023-12-26 15:54:46,812][105620] Updated weights for policy 1, policy_version 75507 (0.0010) [2023-12-26 15:54:46,867][105620] Updated weights for policy 1, policy_version 75517 (0.0010) [2023-12-26 15:54:46,925][105620] Updated weights for policy 1, policy_version 75527 (0.0010) [2023-12-26 15:54:47,366][105692] Updated weights for policy 0, policy_version 75188 (0.0008) [2023-12-26 15:54:47,419][105692] Updated weights for policy 0, policy_version 75199 (0.0010) [2023-12-26 15:54:47,475][105692] Updated weights for policy 0, policy_version 75209 (0.0010) [2023-12-26 15:54:47,508][105620] Updated weights for policy 1, policy_version 75537 (0.0010) [2023-12-26 15:54:47,575][105620] Updated weights for policy 1, policy_version 75547 (0.0005) [2023-12-26 15:54:47,641][105620] Updated weights for policy 1, policy_version 75557 (0.0005) [2023-12-26 15:54:47,709][105620] Updated weights for policy 1, policy_version 75567 (0.0008) [2023-12-26 15:54:48,206][105620] Updated weights for policy 1, policy_version 75577 (0.0010) [2023-12-26 15:54:48,268][105620] Updated weights for policy 1, policy_version 75587 (0.0011) [2023-12-26 15:54:48,331][105692] Updated weights for policy 0, policy_version 75219 (0.0009) [2023-12-26 15:54:48,342][105620] Updated weights for policy 1, policy_version 75597 (0.0011) [2023-12-26 15:54:48,389][105692] Updated weights for policy 0, policy_version 75229 (0.0007) [2023-12-26 15:54:48,445][105692] Updated weights for policy 0, policy_version 75239 (0.0008) [2023-12-26 15:54:49,045][105620] Updated weights for policy 1, policy_version 75607 (0.0011) [2023-12-26 15:54:49,101][105620] Updated weights for policy 1, policy_version 75617 (0.0010) [2023-12-26 15:54:49,146][105620] Updated weights for policy 1, policy_version 75627 (0.0010) [2023-12-26 15:54:49,220][105692] Updated weights for policy 0, policy_version 75249 (0.0008) [2023-12-26 15:54:49,287][105692] Updated weights for policy 0, policy_version 75259 (0.0010) [2023-12-26 15:54:49,345][105692] Updated weights for policy 0, policy_version 75269 (0.0010) [2023-12-26 15:54:49,405][105692] Updated weights for policy 0, policy_version 75279 (0.0008) [2023-12-26 15:54:49,902][105620] Updated weights for policy 1, policy_version 75637 (0.0011) [2023-12-26 15:54:49,966][105620] Updated weights for policy 1, policy_version 75647 (0.0008) [2023-12-26 15:54:50,030][105620] Updated weights for policy 1, policy_version 75657 (0.0006) [2023-12-26 15:54:50,077][105692] Updated weights for policy 0, policy_version 75289 (0.0009) [2023-12-26 15:54:50,126][105692] Updated weights for policy 0, policy_version 75299 (0.0007) [2023-12-26 15:54:50,177][105692] Updated weights for policy 0, policy_version 75309 (0.0009) [2023-12-26 15:54:50,656][105620] Updated weights for policy 1, policy_version 75667 (0.0007) [2023-12-26 15:54:50,714][105620] Updated weights for policy 1, policy_version 75677 (0.0009) [2023-12-26 15:54:50,772][105620] Updated weights for policy 1, policy_version 75687 (0.0008) [2023-12-26 15:54:50,993][105692] Updated weights for policy 0, policy_version 75320 (0.0010) [2023-12-26 15:54:51,059][105692] Updated weights for policy 0, policy_version 75330 (0.0009) [2023-12-26 15:54:51,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 38666240. Throughput: 0: 9579.6, 1: 9822.3. Samples: 38656688. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-12-26 15:54:51,062][104569] Avg episode reward: [(0, '9095.654'), (1, '9000.449')] [2023-12-26 15:54:51,125][105692] Updated weights for policy 0, policy_version 75340 (0.0009) [2023-12-26 15:54:51,539][105620] Updated weights for policy 1, policy_version 75697 (0.0009) [2023-12-26 15:54:51,603][105620] Updated weights for policy 1, policy_version 75707 (0.0009) [2023-12-26 15:54:51,669][105620] Updated weights for policy 1, policy_version 75717 (0.0008) [2023-12-26 15:54:51,741][105620] Updated weights for policy 1, policy_version 75727 (0.0009) [2023-12-26 15:54:51,862][105692] Updated weights for policy 0, policy_version 75350 (0.0009) [2023-12-26 15:54:51,912][105692] Updated weights for policy 0, policy_version 75360 (0.0009) [2023-12-26 15:54:51,974][105692] Updated weights for policy 0, policy_version 75370 (0.0009) [2023-12-26 15:54:52,521][105620] Updated weights for policy 1, policy_version 75737 (0.0009) [2023-12-26 15:54:52,584][105620] Updated weights for policy 1, policy_version 75747 (0.0008) [2023-12-26 15:54:52,646][105620] Updated weights for policy 1, policy_version 75757 (0.0009) [2023-12-26 15:54:52,710][105692] Updated weights for policy 0, policy_version 75380 (0.0009) [2023-12-26 15:54:52,782][105692] Updated weights for policy 0, policy_version 75390 (0.0009) [2023-12-26 15:54:52,844][105692] Updated weights for policy 0, policy_version 75400 (0.0009) [2023-12-26 15:54:53,359][105620] Updated weights for policy 1, policy_version 75767 (0.0006) [2023-12-26 15:54:53,405][105620] Updated weights for policy 1, policy_version 75777 (0.0005) [2023-12-26 15:54:53,459][105620] Updated weights for policy 1, policy_version 75787 (0.0009) [2023-12-26 15:54:53,605][105692] Updated weights for policy 0, policy_version 75410 (0.0009) [2023-12-26 15:54:53,651][105692] Updated weights for policy 0, policy_version 75420 (0.0009) [2023-12-26 15:54:53,698][105692] Updated weights for policy 0, policy_version 75430 (0.0008) [2023-12-26 15:54:53,749][105692] Updated weights for policy 0, policy_version 75440 (0.0009) [2023-12-26 15:54:54,157][105620] Updated weights for policy 1, policy_version 75797 (0.0009) [2023-12-26 15:54:54,204][105620] Updated weights for policy 1, policy_version 75807 (0.0009) [2023-12-26 15:54:54,256][105620] Updated weights for policy 1, policy_version 75817 (0.0008) [2023-12-26 15:54:54,489][105692] Updated weights for policy 0, policy_version 75450 (0.0005) [2023-12-26 15:54:54,534][105692] Updated weights for policy 0, policy_version 75460 (0.0005) [2023-12-26 15:54:54,582][105692] Updated weights for policy 0, policy_version 75470 (0.0005) [2023-12-26 15:54:55,062][105620] Updated weights for policy 1, policy_version 75827 (0.0009) [2023-12-26 15:54:55,127][105620] Updated weights for policy 1, policy_version 75837 (0.0010) [2023-12-26 15:54:55,186][105620] Updated weights for policy 1, policy_version 75847 (0.0010) [2023-12-26 15:54:55,250][105692] Updated weights for policy 0, policy_version 75480 (0.0005) [2023-12-26 15:54:55,298][105692] Updated weights for policy 0, policy_version 75490 (0.0005) [2023-12-26 15:54:55,349][105692] Updated weights for policy 0, policy_version 75500 (0.0006) [2023-12-26 15:54:55,947][105692] Updated weights for policy 0, policy_version 75510 (0.0008) [2023-12-26 15:54:55,993][105692] Updated weights for policy 0, policy_version 75520 (0.0008) [2023-12-26 15:54:56,018][105620] Updated weights for policy 1, policy_version 75857 (0.0009) [2023-12-26 15:54:56,041][105692] Updated weights for policy 0, policy_version 75530 (0.0007) [2023-12-26 15:54:56,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 38756352. Throughput: 0: 9595.1, 1: 9636.4. Samples: 38771308. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-12-26 15:54:56,062][104569] Avg episode reward: [(0, '9006.836'), (1, '8813.270')] [2023-12-26 15:54:56,075][105620] Updated weights for policy 1, policy_version 75867 (0.0008) [2023-12-26 15:54:56,135][105620] Updated weights for policy 1, policy_version 75877 (0.0010) [2023-12-26 15:54:56,189][105620] Updated weights for policy 1, policy_version 75887 (0.0010) [2023-12-26 15:54:56,752][105692] Updated weights for policy 0, policy_version 75540 (0.0007) [2023-12-26 15:54:56,796][105692] Updated weights for policy 0, policy_version 75550 (0.0010) [2023-12-26 15:54:56,833][105620] Updated weights for policy 1, policy_version 75897 (0.0008) [2023-12-26 15:54:56,845][105692] Updated weights for policy 0, policy_version 75560 (0.0010) [2023-12-26 15:54:56,898][105620] Updated weights for policy 1, policy_version 75907 (0.0009) [2023-12-26 15:54:56,957][105620] Updated weights for policy 1, policy_version 75917 (0.0008) [2023-12-26 15:54:57,620][105692] Updated weights for policy 0, policy_version 75570 (0.0010) [2023-12-26 15:54:57,677][105692] Updated weights for policy 0, policy_version 75580 (0.0010) [2023-12-26 15:54:57,733][105620] Updated weights for policy 1, policy_version 75927 (0.0008) [2023-12-26 15:54:57,735][105692] Updated weights for policy 0, policy_version 75590 (0.0010) [2023-12-26 15:54:57,777][105620] Updated weights for policy 1, policy_version 75937 (0.0005) [2023-12-26 15:54:57,783][105692] Updated weights for policy 0, policy_version 75600 (0.0010) [2023-12-26 15:54:57,824][105620] Updated weights for policy 1, policy_version 75947 (0.0008) [2023-12-26 15:54:58,605][105692] Updated weights for policy 0, policy_version 75610 (0.0009) [2023-12-26 15:54:58,630][105620] Updated weights for policy 1, policy_version 75957 (0.0008) [2023-12-26 15:54:58,670][105692] Updated weights for policy 0, policy_version 75620 (0.0008) [2023-12-26 15:54:58,697][105620] Updated weights for policy 1, policy_version 75967 (0.0009) [2023-12-26 15:54:58,734][105692] Updated weights for policy 0, policy_version 75630 (0.0007) [2023-12-26 15:54:58,767][105620] Updated weights for policy 1, policy_version 75977 (0.0008) [2023-12-26 15:54:59,472][105692] Updated weights for policy 0, policy_version 75640 (0.0009) [2023-12-26 15:54:59,534][105692] Updated weights for policy 0, policy_version 75650 (0.0009) [2023-12-26 15:54:59,579][105620] Updated weights for policy 1, policy_version 75987 (0.0008) [2023-12-26 15:54:59,588][105692] Updated weights for policy 0, policy_version 75660 (0.0007) [2023-12-26 15:54:59,648][105620] Updated weights for policy 1, policy_version 75997 (0.0007) [2023-12-26 15:54:59,711][105620] Updated weights for policy 1, policy_version 76007 (0.0007) [2023-12-26 15:55:00,256][105692] Updated weights for policy 0, policy_version 75670 (0.0007) [2023-12-26 15:55:00,315][105692] Updated weights for policy 0, policy_version 75680 (0.0005) [2023-12-26 15:55:00,382][105692] Updated weights for policy 0, policy_version 75690 (0.0007) [2023-12-26 15:55:00,517][105620] Updated weights for policy 1, policy_version 76017 (0.0007) [2023-12-26 15:55:00,573][105620] Updated weights for policy 1, policy_version 76027 (0.0008) [2023-12-26 15:55:00,633][105620] Updated weights for policy 1, policy_version 76037 (0.0009) [2023-12-26 15:55:00,693][105620] Updated weights for policy 1, policy_version 76047 (0.0009) [2023-12-26 15:55:01,028][105692] Updated weights for policy 0, policy_version 75700 (0.0009) [2023-12-26 15:55:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 38854656. Throughput: 0: 9587.2, 1: 9641.2. Samples: 38828548. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-12-26 15:55:01,063][104569] Avg episode reward: [(0, '9177.542'), (1, '8908.584')] [2023-12-26 15:55:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000076048_19472384.pth... [2023-12-26 15:55:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000074928_19185664.pth [2023-12-26 15:55:01,091][105692] Updated weights for policy 0, policy_version 75710 (0.0009) [2023-12-26 15:55:01,155][105692] Updated weights for policy 0, policy_version 75720 (0.0010) [2023-12-26 15:55:01,207][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000075728_19390464.pth... [2023-12-26 15:55:01,211][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000074608_19103744.pth [2023-12-26 15:55:01,411][105620] Updated weights for policy 1, policy_version 76057 (0.0007) [2023-12-26 15:55:01,478][105620] Updated weights for policy 1, policy_version 76067 (0.0008) [2023-12-26 15:55:01,539][105620] Updated weights for policy 1, policy_version 76077 (0.0009) [2023-12-26 15:55:01,891][105692] Updated weights for policy 0, policy_version 75730 (0.0010) [2023-12-26 15:55:01,952][105692] Updated weights for policy 0, policy_version 75740 (0.0010) [2023-12-26 15:55:02,000][105692] Updated weights for policy 0, policy_version 75750 (0.0007) [2023-12-26 15:55:02,054][105692] Updated weights for policy 0, policy_version 75760 (0.0006) [2023-12-26 15:55:02,141][105620] Updated weights for policy 1, policy_version 76087 (0.0009) [2023-12-26 15:55:02,200][105620] Updated weights for policy 1, policy_version 76097 (0.0010) [2023-12-26 15:55:02,262][105620] Updated weights for policy 1, policy_version 76107 (0.0011) [2023-12-26 15:55:02,715][105692] Updated weights for policy 0, policy_version 75770 (0.0005) [2023-12-26 15:55:02,775][105692] Updated weights for policy 0, policy_version 75780 (0.0005) [2023-12-26 15:55:02,833][105692] Updated weights for policy 0, policy_version 75790 (0.0005) [2023-12-26 15:55:02,937][105620] Updated weights for policy 1, policy_version 76117 (0.0006) [2023-12-26 15:55:02,991][105620] Updated weights for policy 1, policy_version 76127 (0.0007) [2023-12-26 15:55:03,049][105620] Updated weights for policy 1, policy_version 76137 (0.0010) [2023-12-26 15:55:03,408][105692] Updated weights for policy 0, policy_version 75800 (0.0005) [2023-12-26 15:55:03,455][105692] Updated weights for policy 0, policy_version 75810 (0.0006) [2023-12-26 15:55:03,505][105692] Updated weights for policy 0, policy_version 75820 (0.0006) [2023-12-26 15:55:03,704][105620] Updated weights for policy 1, policy_version 76147 (0.0009) [2023-12-26 15:55:03,775][105620] Updated weights for policy 1, policy_version 76157 (0.0006) [2023-12-26 15:55:03,835][105620] Updated weights for policy 1, policy_version 76167 (0.0007) [2023-12-26 15:55:04,069][105692] Updated weights for policy 0, policy_version 75830 (0.0007) [2023-12-26 15:55:04,135][105692] Updated weights for policy 0, policy_version 75840 (0.0008) [2023-12-26 15:55:04,196][105692] Updated weights for policy 0, policy_version 75850 (0.0007) [2023-12-26 15:55:04,541][105620] Updated weights for policy 1, policy_version 76177 (0.0009) [2023-12-26 15:55:04,600][105620] Updated weights for policy 1, policy_version 76187 (0.0011) [2023-12-26 15:55:04,659][105620] Updated weights for policy 1, policy_version 76197 (0.0011) [2023-12-26 15:55:04,718][105620] Updated weights for policy 1, policy_version 76207 (0.0011) [2023-12-26 15:55:04,859][105692] Updated weights for policy 0, policy_version 75860 (0.0006) [2023-12-26 15:55:04,920][105692] Updated weights for policy 0, policy_version 75870 (0.0006) [2023-12-26 15:55:04,985][105692] Updated weights for policy 0, policy_version 75880 (0.0005) [2023-12-26 15:55:05,469][105620] Updated weights for policy 1, policy_version 76217 (0.0010) [2023-12-26 15:55:05,522][105620] Updated weights for policy 1, policy_version 76227 (0.0011) [2023-12-26 15:55:05,571][105620] Updated weights for policy 1, policy_version 76237 (0.0010) [2023-12-26 15:55:05,599][105692] Updated weights for policy 0, policy_version 75890 (0.0006) [2023-12-26 15:55:05,666][105692] Updated weights for policy 0, policy_version 75900 (0.0008) [2023-12-26 15:55:05,729][105692] Updated weights for policy 0, policy_version 75910 (0.0008) [2023-12-26 15:55:05,794][105692] Updated weights for policy 0, policy_version 75920 (0.0008) [2023-12-26 15:55:06,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 38961152. Throughput: 0: 9614.6, 1: 9670.7. Samples: 38949456. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-12-26 15:55:06,063][104569] Avg episode reward: [(0, '8921.155'), (1, '8909.999')] [2023-12-26 15:55:06,344][105620] Updated weights for policy 1, policy_version 76247 (0.0011) [2023-12-26 15:55:06,407][105620] Updated weights for policy 1, policy_version 76257 (0.0011) [2023-12-26 15:55:06,473][105620] Updated weights for policy 1, policy_version 76267 (0.0011) [2023-12-26 15:55:06,567][105692] Updated weights for policy 0, policy_version 75930 (0.0008) [2023-12-26 15:55:06,633][105692] Updated weights for policy 0, policy_version 75940 (0.0007) [2023-12-26 15:55:06,701][105692] Updated weights for policy 0, policy_version 75950 (0.0008) [2023-12-26 15:55:07,162][105620] Updated weights for policy 1, policy_version 76277 (0.0008) [2023-12-26 15:55:07,218][105620] Updated weights for policy 1, policy_version 76287 (0.0009) [2023-12-26 15:55:07,281][105620] Updated weights for policy 1, policy_version 76297 (0.0010) [2023-12-26 15:55:07,481][105692] Updated weights for policy 0, policy_version 75960 (0.0008) [2023-12-26 15:55:07,530][105692] Updated weights for policy 0, policy_version 75970 (0.0008) [2023-12-26 15:55:07,577][105692] Updated weights for policy 0, policy_version 75980 (0.0008) [2023-12-26 15:55:08,011][105620] Updated weights for policy 1, policy_version 76307 (0.0010) [2023-12-26 15:55:08,079][105620] Updated weights for policy 1, policy_version 76317 (0.0010) [2023-12-26 15:55:08,147][105620] Updated weights for policy 1, policy_version 76327 (0.0010) [2023-12-26 15:55:08,317][105692] Updated weights for policy 0, policy_version 75990 (0.0007) [2023-12-26 15:55:08,378][105692] Updated weights for policy 0, policy_version 76000 (0.0008) [2023-12-26 15:55:08,445][105692] Updated weights for policy 0, policy_version 76010 (0.0006) [2023-12-26 15:55:08,842][105620] Updated weights for policy 1, policy_version 76337 (0.0010) [2023-12-26 15:55:08,903][105620] Updated weights for policy 1, policy_version 76347 (0.0009) [2023-12-26 15:55:08,970][105620] Updated weights for policy 1, policy_version 76357 (0.0009) [2023-12-26 15:55:09,022][105620] Updated weights for policy 1, policy_version 76367 (0.0006) [2023-12-26 15:55:09,043][105692] Updated weights for policy 0, policy_version 76020 (0.0007) [2023-12-26 15:55:09,102][105692] Updated weights for policy 0, policy_version 76030 (0.0010) [2023-12-26 15:55:09,150][105692] Updated weights for policy 0, policy_version 76040 (0.0010) [2023-12-26 15:55:09,799][105620] Updated weights for policy 1, policy_version 76377 (0.0010) [2023-12-26 15:55:09,814][105692] Updated weights for policy 0, policy_version 76050 (0.0010) [2023-12-26 15:55:09,879][105620] Updated weights for policy 1, policy_version 76387 (0.0008) [2023-12-26 15:55:09,882][105692] Updated weights for policy 0, policy_version 76060 (0.0008) [2023-12-26 15:55:09,946][105620] Updated weights for policy 1, policy_version 76397 (0.0009) [2023-12-26 15:55:09,950][105692] Updated weights for policy 0, policy_version 76070 (0.0009) [2023-12-26 15:55:10,015][105692] Updated weights for policy 0, policy_version 76080 (0.0011) [2023-12-26 15:55:10,592][105620] Updated weights for policy 1, policy_version 76407 (0.0011) [2023-12-26 15:55:10,642][105620] Updated weights for policy 1, policy_version 76417 (0.0010) [2023-12-26 15:55:10,705][105620] Updated weights for policy 1, policy_version 76427 (0.0011) [2023-12-26 15:55:10,733][105692] Updated weights for policy 0, policy_version 76090 (0.0011) [2023-12-26 15:55:10,784][105692] Updated weights for policy 0, policy_version 76100 (0.0010) [2023-12-26 15:55:10,839][105692] Updated weights for policy 0, policy_version 76110 (0.0010) [2023-12-26 15:55:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 39059456. Throughput: 0: 9602.4, 1: 9752.8. Samples: 39065756. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-12-26 15:55:11,063][104569] Avg episode reward: [(0, '8397.217'), (1, '8905.185')] [2023-12-26 15:55:11,467][105620] Updated weights for policy 1, policy_version 76437 (0.0011) [2023-12-26 15:55:11,528][105620] Updated weights for policy 1, policy_version 76447 (0.0009) [2023-12-26 15:55:11,592][105620] Updated weights for policy 1, policy_version 76457 (0.0006) [2023-12-26 15:55:11,621][105692] Updated weights for policy 0, policy_version 76120 (0.0009) [2023-12-26 15:55:11,686][105692] Updated weights for policy 0, policy_version 76130 (0.0010) [2023-12-26 15:55:11,753][105692] Updated weights for policy 0, policy_version 76140 (0.0008) [2023-12-26 15:55:12,376][105620] Updated weights for policy 1, policy_version 76467 (0.0008) [2023-12-26 15:55:12,438][105620] Updated weights for policy 1, policy_version 76477 (0.0009) [2023-12-26 15:55:12,473][105692] Updated weights for policy 0, policy_version 76150 (0.0007) [2023-12-26 15:55:12,500][105620] Updated weights for policy 1, policy_version 76487 (0.0007) [2023-12-26 15:55:12,537][105692] Updated weights for policy 0, policy_version 76160 (0.0008) [2023-12-26 15:55:12,589][105692] Updated weights for policy 0, policy_version 76170 (0.0009) [2023-12-26 15:55:13,174][105620] Updated weights for policy 1, policy_version 76497 (0.0006) [2023-12-26 15:55:13,236][105620] Updated weights for policy 1, policy_version 76507 (0.0008) [2023-12-26 15:55:13,293][105692] Updated weights for policy 0, policy_version 76180 (0.0007) [2023-12-26 15:55:13,294][105620] Updated weights for policy 1, policy_version 76517 (0.0009) [2023-12-26 15:55:13,339][105692] Updated weights for policy 0, policy_version 76190 (0.0005) [2023-12-26 15:55:13,349][105620] Updated weights for policy 1, policy_version 76527 (0.0009) [2023-12-26 15:55:13,393][105692] Updated weights for policy 0, policy_version 76200 (0.0007) [2023-12-26 15:55:14,037][105692] Updated weights for policy 0, policy_version 76210 (0.0008) [2023-12-26 15:55:14,093][105692] Updated weights for policy 0, policy_version 76220 (0.0005) [2023-12-26 15:55:14,129][105620] Updated weights for policy 1, policy_version 76537 (0.0009) [2023-12-26 15:55:14,152][105692] Updated weights for policy 0, policy_version 76230 (0.0005) [2023-12-26 15:55:14,191][105620] Updated weights for policy 1, policy_version 76547 (0.0008) [2023-12-26 15:55:14,209][105692] Updated weights for policy 0, policy_version 76240 (0.0008) [2023-12-26 15:55:14,244][105620] Updated weights for policy 1, policy_version 76557 (0.0008) [2023-12-26 15:55:14,813][105692] Updated weights for policy 0, policy_version 76250 (0.0009) [2023-12-26 15:55:14,879][105692] Updated weights for policy 0, policy_version 76260 (0.0010) [2023-12-26 15:55:14,944][105692] Updated weights for policy 0, policy_version 76270 (0.0005) [2023-12-26 15:55:15,043][105620] Updated weights for policy 1, policy_version 76567 (0.0008) [2023-12-26 15:55:15,098][105620] Updated weights for policy 1, policy_version 76577 (0.0009) [2023-12-26 15:55:15,147][105620] Updated weights for policy 1, policy_version 76587 (0.0008) [2023-12-26 15:55:15,639][105692] Updated weights for policy 0, policy_version 76280 (0.0010) [2023-12-26 15:55:15,692][105692] Updated weights for policy 0, policy_version 76290 (0.0011) [2023-12-26 15:55:15,755][105692] Updated weights for policy 0, policy_version 76300 (0.0011) [2023-12-26 15:55:15,937][105620] Updated weights for policy 1, policy_version 76597 (0.0007) [2023-12-26 15:55:15,996][105620] Updated weights for policy 1, policy_version 76607 (0.0005) [2023-12-26 15:55:16,054][105620] Updated weights for policy 1, policy_version 76617 (0.0005) [2023-12-26 15:55:16,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 39149568. Throughput: 0: 9552.8, 1: 9757.0. Samples: 39122248. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-12-26 15:55:16,062][104569] Avg episode reward: [(0, '8475.049'), (1, '8816.310')] [2023-12-26 15:55:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000076304_19537920.pth... [2023-12-26 15:55:16,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000075152_19243008.pth [2023-12-26 15:55:16,092][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000076624_19619840.pth... [2023-12-26 15:55:16,097][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000075504_19333120.pth [2023-12-26 15:55:16,429][105692] Updated weights for policy 0, policy_version 76310 (0.0011) [2023-12-26 15:55:16,491][105692] Updated weights for policy 0, policy_version 76320 (0.0011) [2023-12-26 15:55:16,563][105692] Updated weights for policy 0, policy_version 76330 (0.0009) [2023-12-26 15:55:16,565][105620] Updated weights for policy 1, policy_version 76627 (0.0006) [2023-12-26 15:55:16,634][105620] Updated weights for policy 1, policy_version 76637 (0.0006) [2023-12-26 15:55:16,711][105620] Updated weights for policy 1, policy_version 76647 (0.0008) [2023-12-26 15:55:17,169][105692] Updated weights for policy 0, policy_version 76340 (0.0005) [2023-12-26 15:55:17,234][105620] Updated weights for policy 1, policy_version 76657 (0.0007) [2023-12-26 15:55:17,238][105692] Updated weights for policy 0, policy_version 76350 (0.0006) [2023-12-26 15:55:17,289][105620] Updated weights for policy 1, policy_version 76667 (0.0006) [2023-12-26 15:55:17,299][105692] Updated weights for policy 0, policy_version 76360 (0.0009) [2023-12-26 15:55:17,345][105620] Updated weights for policy 1, policy_version 76677 (0.0006) [2023-12-26 15:55:17,394][105620] Updated weights for policy 1, policy_version 76687 (0.0007) [2023-12-26 15:55:17,934][105692] Updated weights for policy 0, policy_version 76370 (0.0009) [2023-12-26 15:55:17,956][105620] Updated weights for policy 1, policy_version 76697 (0.0005) [2023-12-26 15:55:17,993][105692] Updated weights for policy 0, policy_version 76380 (0.0006) [2023-12-26 15:55:18,002][105620] Updated weights for policy 1, policy_version 76707 (0.0006) [2023-12-26 15:55:18,045][105692] Updated weights for policy 0, policy_version 76390 (0.0007) [2023-12-26 15:55:18,056][105620] Updated weights for policy 1, policy_version 76717 (0.0008) [2023-12-26 15:55:18,098][105692] Updated weights for policy 0, policy_version 76400 (0.0005) [2023-12-26 15:55:18,668][105692] Updated weights for policy 0, policy_version 76410 (0.0005) [2023-12-26 15:55:18,717][105692] Updated weights for policy 0, policy_version 76420 (0.0005) [2023-12-26 15:55:18,781][105692] Updated weights for policy 0, policy_version 76430 (0.0006) [2023-12-26 15:55:18,785][105620] Updated weights for policy 1, policy_version 76727 (0.0007) [2023-12-26 15:55:18,849][105620] Updated weights for policy 1, policy_version 76737 (0.0007) [2023-12-26 15:55:18,914][105620] Updated weights for policy 1, policy_version 76747 (0.0008) [2023-12-26 15:55:19,468][105692] Updated weights for policy 0, policy_version 76440 (0.0006) [2023-12-26 15:55:19,523][105620] Updated weights for policy 1, policy_version 76757 (0.0007) [2023-12-26 15:55:19,535][105692] Updated weights for policy 0, policy_version 76450 (0.0007) [2023-12-26 15:55:19,588][105620] Updated weights for policy 1, policy_version 76767 (0.0006) [2023-12-26 15:55:19,595][105692] Updated weights for policy 0, policy_version 76460 (0.0007) [2023-12-26 15:55:19,650][105620] Updated weights for policy 1, policy_version 76777 (0.0007) [2023-12-26 15:55:20,334][105692] Updated weights for policy 0, policy_version 76470 (0.0009) [2023-12-26 15:55:20,358][105620] Updated weights for policy 1, policy_version 76787 (0.0007) [2023-12-26 15:55:20,400][105692] Updated weights for policy 0, policy_version 76480 (0.0010) [2023-12-26 15:55:20,414][105620] Updated weights for policy 1, policy_version 76797 (0.0007) [2023-12-26 15:55:20,462][105692] Updated weights for policy 0, policy_version 76490 (0.0008) [2023-12-26 15:55:20,477][105620] Updated weights for policy 1, policy_version 76807 (0.0006) [2023-12-26 15:55:21,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 39256064. Throughput: 0: 9716.6, 1: 9876.1. Samples: 39249688. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-12-26 15:55:21,062][104569] Avg episode reward: [(0, '8823.513'), (1, '9267.523')] [2023-12-26 15:55:21,228][105692] Updated weights for policy 0, policy_version 76500 (0.0009) [2023-12-26 15:55:21,266][105620] Updated weights for policy 1, policy_version 76817 (0.0006) [2023-12-26 15:55:21,291][105692] Updated weights for policy 0, policy_version 76510 (0.0008) [2023-12-26 15:55:21,328][105620] Updated weights for policy 1, policy_version 76827 (0.0008) [2023-12-26 15:55:21,363][105692] Updated weights for policy 0, policy_version 76520 (0.0010) [2023-12-26 15:55:21,394][105620] Updated weights for policy 1, policy_version 76837 (0.0007) [2023-12-26 15:55:21,456][105620] Updated weights for policy 1, policy_version 76847 (0.0008) [2023-12-26 15:55:22,053][105692] Updated weights for policy 0, policy_version 76530 (0.0009) [2023-12-26 15:55:22,122][105692] Updated weights for policy 0, policy_version 76540 (0.0009) [2023-12-26 15:55:22,188][105692] Updated weights for policy 0, policy_version 76550 (0.0006) [2023-12-26 15:55:22,209][105620] Updated weights for policy 1, policy_version 76857 (0.0006) [2023-12-26 15:55:22,250][105692] Updated weights for policy 0, policy_version 76560 (0.0008) [2023-12-26 15:55:22,266][105620] Updated weights for policy 1, policy_version 76867 (0.0006) [2023-12-26 15:55:22,337][105620] Updated weights for policy 1, policy_version 76877 (0.0006) [2023-12-26 15:55:22,921][105692] Updated weights for policy 0, policy_version 76570 (0.0006) [2023-12-26 15:55:22,976][105620] Updated weights for policy 1, policy_version 76887 (0.0009) [2023-12-26 15:55:22,978][105692] Updated weights for policy 0, policy_version 76580 (0.0008) [2023-12-26 15:55:23,036][105692] Updated weights for policy 0, policy_version 76590 (0.0006) [2023-12-26 15:55:23,042][105620] Updated weights for policy 1, policy_version 76897 (0.0009) [2023-12-26 15:55:23,106][105620] Updated weights for policy 1, policy_version 76907 (0.0009) [2023-12-26 15:55:23,646][105692] Updated weights for policy 0, policy_version 76600 (0.0007) [2023-12-26 15:55:23,694][105692] Updated weights for policy 0, policy_version 76610 (0.0005) [2023-12-26 15:55:23,745][105692] Updated weights for policy 0, policy_version 76620 (0.0005) [2023-12-26 15:55:23,969][105620] Updated weights for policy 1, policy_version 76917 (0.0009) [2023-12-26 15:55:24,024][105620] Updated weights for policy 1, policy_version 76927 (0.0008) [2023-12-26 15:55:24,083][105620] Updated weights for policy 1, policy_version 76937 (0.0008) [2023-12-26 15:55:24,378][105692] Updated weights for policy 0, policy_version 76630 (0.0009) [2023-12-26 15:55:24,430][105692] Updated weights for policy 0, policy_version 76640 (0.0010) [2023-12-26 15:55:24,478][105692] Updated weights for policy 0, policy_version 76650 (0.0010) [2023-12-26 15:55:24,802][105620] Updated weights for policy 1, policy_version 76947 (0.0009) [2023-12-26 15:55:24,871][105620] Updated weights for policy 1, policy_version 76957 (0.0009) [2023-12-26 15:55:24,930][105620] Updated weights for policy 1, policy_version 76967 (0.0006) [2023-12-26 15:55:25,175][105692] Updated weights for policy 0, policy_version 76660 (0.0008) [2023-12-26 15:55:25,232][105692] Updated weights for policy 0, policy_version 76670 (0.0005) [2023-12-26 15:55:25,279][105692] Updated weights for policy 0, policy_version 76680 (0.0005) [2023-12-26 15:55:25,480][105620] Updated weights for policy 1, policy_version 76977 (0.0009) [2023-12-26 15:55:25,533][105620] Updated weights for policy 1, policy_version 76987 (0.0006) [2023-12-26 15:55:25,580][105620] Updated weights for policy 1, policy_version 76997 (0.0005) [2023-12-26 15:55:25,632][105620] Updated weights for policy 1, policy_version 77007 (0.0005) [2023-12-26 15:55:25,907][105692] Updated weights for policy 0, policy_version 76690 (0.0009) [2023-12-26 15:55:25,969][105692] Updated weights for policy 0, policy_version 76700 (0.0010) [2023-12-26 15:55:26,023][105692] Updated weights for policy 0, policy_version 76710 (0.0010) [2023-12-26 15:55:26,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 39354368. Throughput: 0: 9806.6, 1: 9822.2. Samples: 39368688. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-12-26 15:55:26,063][104569] Avg episode reward: [(0, '8823.008'), (1, '9174.758')] [2023-12-26 15:55:26,084][105692] Updated weights for policy 0, policy_version 76720 (0.0010) [2023-12-26 15:55:26,255][105620] Updated weights for policy 1, policy_version 77017 (0.0005) [2023-12-26 15:55:26,301][105620] Updated weights for policy 1, policy_version 77027 (0.0005) [2023-12-26 15:55:26,364][105620] Updated weights for policy 1, policy_version 77037 (0.0005) [2023-12-26 15:55:26,817][105692] Updated weights for policy 0, policy_version 76730 (0.0010) [2023-12-26 15:55:26,882][105692] Updated weights for policy 0, policy_version 76740 (0.0010) [2023-12-26 15:55:26,943][105692] Updated weights for policy 0, policy_version 76750 (0.0010) [2023-12-26 15:55:27,017][105620] Updated weights for policy 1, policy_version 77047 (0.0007) [2023-12-26 15:55:27,072][105620] Updated weights for policy 1, policy_version 77057 (0.0008) [2023-12-26 15:55:27,120][105620] Updated weights for policy 1, policy_version 77067 (0.0008) [2023-12-26 15:55:27,652][105692] Updated weights for policy 0, policy_version 76760 (0.0010) [2023-12-26 15:55:27,710][105692] Updated weights for policy 0, policy_version 76770 (0.0010) [2023-12-26 15:55:27,768][105692] Updated weights for policy 0, policy_version 76780 (0.0010) [2023-12-26 15:55:27,850][105620] Updated weights for policy 1, policy_version 77077 (0.0007) [2023-12-26 15:55:27,912][105620] Updated weights for policy 1, policy_version 77087 (0.0005) [2023-12-26 15:55:27,977][105620] Updated weights for policy 1, policy_version 77097 (0.0011) [2023-12-26 15:55:28,483][105692] Updated weights for policy 0, policy_version 76790 (0.0010) [2023-12-26 15:55:28,537][105692] Updated weights for policy 0, policy_version 76800 (0.0010) [2023-12-26 15:55:28,552][105620] Updated weights for policy 1, policy_version 77107 (0.0009) [2023-12-26 15:55:28,590][105692] Updated weights for policy 0, policy_version 76810 (0.0010) [2023-12-26 15:55:28,611][105620] Updated weights for policy 1, policy_version 77117 (0.0010) [2023-12-26 15:55:28,670][105620] Updated weights for policy 1, policy_version 77127 (0.0010) [2023-12-26 15:55:29,265][105620] Updated weights for policy 1, policy_version 77137 (0.0010) [2023-12-26 15:55:29,323][105620] Updated weights for policy 1, policy_version 77147 (0.0007) [2023-12-26 15:55:29,356][105692] Updated weights for policy 0, policy_version 76820 (0.0010) [2023-12-26 15:55:29,389][105620] Updated weights for policy 1, policy_version 77157 (0.0008) [2023-12-26 15:55:29,420][105692] Updated weights for policy 0, policy_version 76830 (0.0011) [2023-12-26 15:55:29,445][105620] Updated weights for policy 1, policy_version 77167 (0.0006) [2023-12-26 15:55:29,475][105692] Updated weights for policy 0, policy_version 76840 (0.0010) [2023-12-26 15:55:30,181][105620] Updated weights for policy 1, policy_version 77177 (0.0010) [2023-12-26 15:55:30,207][105692] Updated weights for policy 0, policy_version 76850 (0.0010) [2023-12-26 15:55:30,240][105620] Updated weights for policy 1, policy_version 77187 (0.0010) [2023-12-26 15:55:30,261][105692] Updated weights for policy 0, policy_version 76860 (0.0011) [2023-12-26 15:55:30,299][105620] Updated weights for policy 1, policy_version 77197 (0.0010) [2023-12-26 15:55:30,324][105692] Updated weights for policy 0, policy_version 76870 (0.0010) [2023-12-26 15:55:30,375][105692] Updated weights for policy 0, policy_version 76880 (0.0010) [2023-12-26 15:55:30,866][105620] Updated weights for policy 1, policy_version 77207 (0.0007) [2023-12-26 15:55:30,913][105620] Updated weights for policy 1, policy_version 77217 (0.0010) [2023-12-26 15:55:30,958][105620] Updated weights for policy 1, policy_version 77227 (0.0007) [2023-12-26 15:55:31,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.9, 300 sec: 19633.0). Total num frames: 39460864. Throughput: 0: 9856.9, 1: 9871.9. Samples: 39429016. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-12-26 15:55:31,063][104569] Avg episode reward: [(0, '8821.050'), (1, '8825.129')] [2023-12-26 15:55:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000077232_19775488.pth... [2023-12-26 15:55:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000076048_19472384.pth [2023-12-26 15:55:31,098][105692] Updated weights for policy 0, policy_version 76890 (0.0006) [2023-12-26 15:55:31,167][105692] Updated weights for policy 0, policy_version 76900 (0.0009) [2023-12-26 15:55:31,234][105692] Updated weights for policy 0, policy_version 76910 (0.0008) [2023-12-26 15:55:31,248][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000076912_19693568.pth... [2023-12-26 15:55:31,253][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000075728_19390464.pth [2023-12-26 15:55:31,630][105620] Updated weights for policy 1, policy_version 77237 (0.0008) [2023-12-26 15:55:31,701][105620] Updated weights for policy 1, policy_version 77247 (0.0007) [2023-12-26 15:55:31,767][105620] Updated weights for policy 1, policy_version 77257 (0.0006) [2023-12-26 15:55:31,931][105692] Updated weights for policy 0, policy_version 76920 (0.0009) [2023-12-26 15:55:31,982][105692] Updated weights for policy 0, policy_version 76930 (0.0009) [2023-12-26 15:55:32,041][105692] Updated weights for policy 0, policy_version 76940 (0.0009) [2023-12-26 15:55:32,405][105620] Updated weights for policy 1, policy_version 77267 (0.0008) [2023-12-26 15:55:32,463][105620] Updated weights for policy 1, policy_version 77277 (0.0008) [2023-12-26 15:55:32,513][105620] Updated weights for policy 1, policy_version 77287 (0.0005) [2023-12-26 15:55:32,870][105692] Updated weights for policy 0, policy_version 76950 (0.0009) [2023-12-26 15:55:32,917][105692] Updated weights for policy 0, policy_version 76960 (0.0009) [2023-12-26 15:55:32,975][105692] Updated weights for policy 0, policy_version 76970 (0.0009) [2023-12-26 15:55:33,176][105620] Updated weights for policy 1, policy_version 77297 (0.0006) [2023-12-26 15:55:33,240][105620] Updated weights for policy 1, policy_version 77307 (0.0007) [2023-12-26 15:55:33,308][105620] Updated weights for policy 1, policy_version 77317 (0.0007) [2023-12-26 15:55:33,373][105620] Updated weights for policy 1, policy_version 77327 (0.0008) [2023-12-26 15:55:33,565][105692] Updated weights for policy 0, policy_version 76980 (0.0007) [2023-12-26 15:55:33,611][105692] Updated weights for policy 0, policy_version 76990 (0.0005) [2023-12-26 15:55:33,670][105692] Updated weights for policy 0, policy_version 77000 (0.0008) [2023-12-26 15:55:33,960][105620] Updated weights for policy 1, policy_version 77337 (0.0007) [2023-12-26 15:55:34,020][105620] Updated weights for policy 1, policy_version 77347 (0.0009) [2023-12-26 15:55:34,073][105620] Updated weights for policy 1, policy_version 77357 (0.0009) [2023-12-26 15:55:34,319][105692] Updated weights for policy 0, policy_version 77010 (0.0009) [2023-12-26 15:55:34,379][105692] Updated weights for policy 0, policy_version 77020 (0.0006) [2023-12-26 15:55:34,441][105692] Updated weights for policy 0, policy_version 77030 (0.0007) [2023-12-26 15:55:34,504][105692] Updated weights for policy 0, policy_version 77040 (0.0009) [2023-12-26 15:55:34,892][105620] Updated weights for policy 1, policy_version 77367 (0.0010) [2023-12-26 15:55:34,941][105620] Updated weights for policy 1, policy_version 77377 (0.0010) [2023-12-26 15:55:35,020][105620] Updated weights for policy 1, policy_version 77387 (0.0011) [2023-12-26 15:55:35,205][105692] Updated weights for policy 0, policy_version 77050 (0.0008) [2023-12-26 15:55:35,252][105692] Updated weights for policy 0, policy_version 77060 (0.0007) [2023-12-26 15:55:35,307][105692] Updated weights for policy 0, policy_version 77070 (0.0006) [2023-12-26 15:55:35,634][105620] Updated weights for policy 1, policy_version 77397 (0.0008) [2023-12-26 15:55:35,682][105620] Updated weights for policy 1, policy_version 77407 (0.0006) [2023-12-26 15:55:35,739][105620] Updated weights for policy 1, policy_version 77417 (0.0007) [2023-12-26 15:55:36,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 39559168. Throughput: 0: 9997.0, 1: 9853.7. Samples: 39549968. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 15:55:36,062][104569] Avg episode reward: [(0, '8553.850'), (1, '8651.327')] [2023-12-26 15:55:36,170][105692] Updated weights for policy 0, policy_version 77080 (0.0009) [2023-12-26 15:55:36,232][105692] Updated weights for policy 0, policy_version 77090 (0.0009) [2023-12-26 15:55:36,298][105692] Updated weights for policy 0, policy_version 77100 (0.0009) [2023-12-26 15:55:36,391][105620] Updated weights for policy 1, policy_version 77427 (0.0009) [2023-12-26 15:55:36,454][105620] Updated weights for policy 1, policy_version 77437 (0.0008) [2023-12-26 15:55:36,518][105620] Updated weights for policy 1, policy_version 77447 (0.0009) [2023-12-26 15:55:37,112][105692] Updated weights for policy 0, policy_version 77110 (0.0009) [2023-12-26 15:55:37,168][105620] Updated weights for policy 1, policy_version 77457 (0.0009) [2023-12-26 15:55:37,169][105692] Updated weights for policy 0, policy_version 77120 (0.0010) [2023-12-26 15:55:37,225][105620] Updated weights for policy 1, policy_version 77467 (0.0005) [2023-12-26 15:55:37,226][105692] Updated weights for policy 0, policy_version 77130 (0.0010) [2023-12-26 15:55:37,277][105620] Updated weights for policy 1, policy_version 77477 (0.0006) [2023-12-26 15:55:37,345][105620] Updated weights for policy 1, policy_version 77487 (0.0006) [2023-12-26 15:55:37,974][105692] Updated weights for policy 0, policy_version 77140 (0.0009) [2023-12-26 15:55:38,032][105692] Updated weights for policy 0, policy_version 77150 (0.0006) [2023-12-26 15:55:38,042][105620] Updated weights for policy 1, policy_version 77497 (0.0009) [2023-12-26 15:55:38,081][105692] Updated weights for policy 0, policy_version 77160 (0.0005) [2023-12-26 15:55:38,095][105620] Updated weights for policy 1, policy_version 77507 (0.0008) [2023-12-26 15:55:38,143][105620] Updated weights for policy 1, policy_version 77517 (0.0008) [2023-12-26 15:55:38,804][105692] Updated weights for policy 0, policy_version 77170 (0.0005) [2023-12-26 15:55:38,861][105620] Updated weights for policy 1, policy_version 77527 (0.0009) [2023-12-26 15:55:38,863][105692] Updated weights for policy 0, policy_version 77180 (0.0005) [2023-12-26 15:55:38,915][105692] Updated weights for policy 0, policy_version 77190 (0.0006) [2023-12-26 15:55:38,921][105620] Updated weights for policy 1, policy_version 77537 (0.0010) [2023-12-26 15:55:38,967][105692] Updated weights for policy 0, policy_version 77200 (0.0006) [2023-12-26 15:55:38,982][105620] Updated weights for policy 1, policy_version 77547 (0.0008) [2023-12-26 15:55:39,719][105620] Updated weights for policy 1, policy_version 77557 (0.0007) [2023-12-26 15:55:39,744][105692] Updated weights for policy 0, policy_version 77210 (0.0007) [2023-12-26 15:55:39,782][105620] Updated weights for policy 1, policy_version 77567 (0.0006) [2023-12-26 15:55:39,796][105692] Updated weights for policy 0, policy_version 77220 (0.0008) [2023-12-26 15:55:39,852][105620] Updated weights for policy 1, policy_version 77577 (0.0007) [2023-12-26 15:55:39,860][105692] Updated weights for policy 0, policy_version 77230 (0.0007) [2023-12-26 15:55:40,504][105692] Updated weights for policy 0, policy_version 77240 (0.0008) [2023-12-26 15:55:40,559][105692] Updated weights for policy 0, policy_version 77250 (0.0009) [2023-12-26 15:55:40,621][105692] Updated weights for policy 0, policy_version 77260 (0.0009) [2023-12-26 15:55:40,670][105620] Updated weights for policy 1, policy_version 77587 (0.0009) [2023-12-26 15:55:40,728][105620] Updated weights for policy 1, policy_version 77597 (0.0010) [2023-12-26 15:55:40,787][105620] Updated weights for policy 1, policy_version 77607 (0.0010) [2023-12-26 15:55:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.9, 300 sec: 19633.0). Total num frames: 39657472. Throughput: 0: 9928.0, 1: 9920.0. Samples: 39664468. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 15:55:41,063][104569] Avg episode reward: [(0, '8727.652'), (1, '8556.621')] [2023-12-26 15:55:41,358][105692] Updated weights for policy 0, policy_version 77270 (0.0008) [2023-12-26 15:55:41,424][105692] Updated weights for policy 0, policy_version 77280 (0.0009) [2023-12-26 15:55:41,484][105692] Updated weights for policy 0, policy_version 77290 (0.0008) [2023-12-26 15:55:41,553][105620] Updated weights for policy 1, policy_version 77617 (0.0010) [2023-12-26 15:55:41,617][105620] Updated weights for policy 1, policy_version 77627 (0.0007) [2023-12-26 15:55:41,684][105620] Updated weights for policy 1, policy_version 77637 (0.0006) [2023-12-26 15:55:41,756][105620] Updated weights for policy 1, policy_version 77647 (0.0010) [2023-12-26 15:55:42,299][105692] Updated weights for policy 0, policy_version 77300 (0.0009) [2023-12-26 15:55:42,360][105692] Updated weights for policy 0, policy_version 77310 (0.0009) [2023-12-26 15:55:42,415][105620] Updated weights for policy 1, policy_version 77657 (0.0011) [2023-12-26 15:55:42,421][105692] Updated weights for policy 0, policy_version 77320 (0.0006) [2023-12-26 15:55:42,478][105620] Updated weights for policy 1, policy_version 77667 (0.0011) [2023-12-26 15:55:42,540][105620] Updated weights for policy 1, policy_version 77677 (0.0011) [2023-12-26 15:55:43,144][105692] Updated weights for policy 0, policy_version 77330 (0.0006) [2023-12-26 15:55:43,192][105692] Updated weights for policy 0, policy_version 77340 (0.0008) [2023-12-26 15:55:43,247][105692] Updated weights for policy 0, policy_version 77350 (0.0009) [2023-12-26 15:55:43,285][105620] Updated weights for policy 1, policy_version 77687 (0.0011) [2023-12-26 15:55:43,299][105692] Updated weights for policy 0, policy_version 77360 (0.0005) [2023-12-26 15:55:43,339][105620] Updated weights for policy 1, policy_version 77697 (0.0010) [2023-12-26 15:55:43,385][105620] Updated weights for policy 1, policy_version 77707 (0.0009) [2023-12-26 15:55:44,063][105620] Updated weights for policy 1, policy_version 77717 (0.0006) [2023-12-26 15:55:44,110][105620] Updated weights for policy 1, policy_version 77727 (0.0007) [2023-12-26 15:55:44,118][105692] Updated weights for policy 0, policy_version 77370 (0.0008) [2023-12-26 15:55:44,167][105620] Updated weights for policy 1, policy_version 77737 (0.0008) [2023-12-26 15:55:44,178][105692] Updated weights for policy 0, policy_version 77380 (0.0006) [2023-12-26 15:55:44,240][105692] Updated weights for policy 0, policy_version 77390 (0.0009) [2023-12-26 15:55:44,783][105620] Updated weights for policy 1, policy_version 77747 (0.0006) [2023-12-26 15:55:44,846][105620] Updated weights for policy 1, policy_version 77757 (0.0006) [2023-12-26 15:55:44,903][105620] Updated weights for policy 1, policy_version 77767 (0.0006) [2023-12-26 15:55:44,957][105692] Updated weights for policy 0, policy_version 77400 (0.0007) [2023-12-26 15:55:45,020][105692] Updated weights for policy 0, policy_version 77410 (0.0008) [2023-12-26 15:55:45,081][105692] Updated weights for policy 0, policy_version 77420 (0.0005) [2023-12-26 15:55:45,539][105620] Updated weights for policy 1, policy_version 77777 (0.0006) [2023-12-26 15:55:45,587][105620] Updated weights for policy 1, policy_version 77787 (0.0005) [2023-12-26 15:55:45,654][105692] Updated weights for policy 0, policy_version 77430 (0.0006) [2023-12-26 15:55:45,658][105620] Updated weights for policy 1, policy_version 77797 (0.0006) [2023-12-26 15:55:45,705][105692] Updated weights for policy 0, policy_version 77440 (0.0008) [2023-12-26 15:55:45,715][105620] Updated weights for policy 1, policy_version 77807 (0.0006) [2023-12-26 15:55:45,757][105692] Updated weights for policy 0, policy_version 77450 (0.0010) [2023-12-26 15:55:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 39755776. Throughput: 0: 9889.3, 1: 9940.0. Samples: 39720868. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 15:55:46,063][104569] Avg episode reward: [(0, '8645.692'), (1, '8908.101')] [2023-12-26 15:55:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000077456_19832832.pth... [2023-12-26 15:55:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000077808_19922944.pth... [2023-12-26 15:55:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000076304_19537920.pth [2023-12-26 15:55:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000076624_19619840.pth [2023-12-26 15:55:46,251][105620] Updated weights for policy 1, policy_version 77817 (0.0010) [2023-12-26 15:55:46,306][105620] Updated weights for policy 1, policy_version 77827 (0.0010) [2023-12-26 15:55:46,358][105620] Updated weights for policy 1, policy_version 77837 (0.0010) [2023-12-26 15:55:46,405][105692] Updated weights for policy 0, policy_version 77461 (0.0008) [2023-12-26 15:55:46,456][105692] Updated weights for policy 0, policy_version 77471 (0.0005) [2023-12-26 15:55:46,512][105692] Updated weights for policy 0, policy_version 77481 (0.0005) [2023-12-26 15:55:47,026][105620] Updated weights for policy 1, policy_version 77847 (0.0007) [2023-12-26 15:55:47,080][105620] Updated weights for policy 1, policy_version 77857 (0.0005) [2023-12-26 15:55:47,142][105620] Updated weights for policy 1, policy_version 77867 (0.0005) [2023-12-26 15:55:47,195][105692] Updated weights for policy 0, policy_version 77491 (0.0005) [2023-12-26 15:55:47,249][105692] Updated weights for policy 0, policy_version 77501 (0.0005) [2023-12-26 15:55:47,295][105692] Updated weights for policy 0, policy_version 77511 (0.0005) [2023-12-26 15:55:47,754][105620] Updated weights for policy 1, policy_version 77877 (0.0006) [2023-12-26 15:55:47,804][105620] Updated weights for policy 1, policy_version 77887 (0.0005) [2023-12-26 15:55:47,831][105692] Updated weights for policy 0, policy_version 77521 (0.0005) [2023-12-26 15:55:47,864][105620] Updated weights for policy 1, policy_version 77897 (0.0008) [2023-12-26 15:55:47,895][105692] Updated weights for policy 0, policy_version 77531 (0.0008) [2023-12-26 15:55:47,946][105692] Updated weights for policy 0, policy_version 77541 (0.0007) [2023-12-26 15:55:47,998][105692] Updated weights for policy 0, policy_version 77551 (0.0009) [2023-12-26 15:55:48,492][105620] Updated weights for policy 1, policy_version 77907 (0.0007) [2023-12-26 15:55:48,545][105620] Updated weights for policy 1, policy_version 77917 (0.0008) [2023-12-26 15:55:48,599][105620] Updated weights for policy 1, policy_version 77927 (0.0009) [2023-12-26 15:55:48,812][105692] Updated weights for policy 0, policy_version 77561 (0.0008) [2023-12-26 15:55:48,873][105692] Updated weights for policy 0, policy_version 77571 (0.0005) [2023-12-26 15:55:48,940][105692] Updated weights for policy 0, policy_version 77581 (0.0007) [2023-12-26 15:55:49,460][105620] Updated weights for policy 1, policy_version 77937 (0.0008) [2023-12-26 15:55:49,516][105620] Updated weights for policy 1, policy_version 77947 (0.0009) [2023-12-26 15:55:49,546][105692] Updated weights for policy 0, policy_version 77591 (0.0009) [2023-12-26 15:55:49,575][105620] Updated weights for policy 1, policy_version 77957 (0.0007) [2023-12-26 15:55:49,601][105692] Updated weights for policy 0, policy_version 77601 (0.0008) [2023-12-26 15:55:49,625][105620] Updated weights for policy 1, policy_version 77967 (0.0006) [2023-12-26 15:55:49,672][105692] Updated weights for policy 0, policy_version 77611 (0.0009) [2023-12-26 15:55:50,372][105620] Updated weights for policy 1, policy_version 77977 (0.0009) [2023-12-26 15:55:50,437][105620] Updated weights for policy 1, policy_version 77987 (0.0009) [2023-12-26 15:55:50,476][105692] Updated weights for policy 0, policy_version 77621 (0.0007) [2023-12-26 15:55:50,499][105620] Updated weights for policy 1, policy_version 77997 (0.0007) [2023-12-26 15:55:50,527][105692] Updated weights for policy 0, policy_version 77631 (0.0008) [2023-12-26 15:55:50,580][105692] Updated weights for policy 0, policy_version 77641 (0.0008) [2023-12-26 15:55:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 39854080. Throughput: 0: 9885.5, 1: 10042.4. Samples: 39846212. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 15:55:51,062][104569] Avg episode reward: [(0, '8565.809'), (1, '8740.213')] [2023-12-26 15:55:51,241][105620] Updated weights for policy 1, policy_version 78007 (0.0008) [2023-12-26 15:55:51,298][105620] Updated weights for policy 1, policy_version 78017 (0.0008) [2023-12-26 15:55:51,349][105692] Updated weights for policy 0, policy_version 77651 (0.0008) [2023-12-26 15:55:51,360][105620] Updated weights for policy 1, policy_version 78027 (0.0009) [2023-12-26 15:55:51,418][105692] Updated weights for policy 0, policy_version 77661 (0.0008) [2023-12-26 15:55:51,483][105692] Updated weights for policy 0, policy_version 77671 (0.0009) [2023-12-26 15:55:52,147][105620] Updated weights for policy 1, policy_version 78037 (0.0008) [2023-12-26 15:55:52,204][105620] Updated weights for policy 1, policy_version 78047 (0.0009) [2023-12-26 15:55:52,206][105692] Updated weights for policy 0, policy_version 77681 (0.0009) [2023-12-26 15:55:52,263][105620] Updated weights for policy 1, policy_version 78057 (0.0006) [2023-12-26 15:55:52,268][105692] Updated weights for policy 0, policy_version 77691 (0.0007) [2023-12-26 15:55:52,331][105692] Updated weights for policy 0, policy_version 77701 (0.0010) [2023-12-26 15:55:52,406][105692] Updated weights for policy 0, policy_version 77712 (0.0008) [2023-12-26 15:55:53,026][105620] Updated weights for policy 1, policy_version 78067 (0.0008) [2023-12-26 15:55:53,088][105620] Updated weights for policy 1, policy_version 78077 (0.0008) [2023-12-26 15:55:53,146][105620] Updated weights for policy 1, policy_version 78087 (0.0007) [2023-12-26 15:55:53,168][105692] Updated weights for policy 0, policy_version 77722 (0.0011) [2023-12-26 15:55:53,216][105692] Updated weights for policy 0, policy_version 77732 (0.0010) [2023-12-26 15:55:53,264][105692] Updated weights for policy 0, policy_version 77742 (0.0010) [2023-12-26 15:55:53,891][105620] Updated weights for policy 1, policy_version 78097 (0.0006) [2023-12-26 15:55:53,934][105620] Updated weights for policy 1, policy_version 78107 (0.0008) [2023-12-26 15:55:53,982][105620] Updated weights for policy 1, policy_version 78117 (0.0008) [2023-12-26 15:55:54,028][105692] Updated weights for policy 0, policy_version 77752 (0.0010) [2023-12-26 15:55:54,034][105620] Updated weights for policy 1, policy_version 78127 (0.0006) [2023-12-26 15:55:54,079][105692] Updated weights for policy 0, policy_version 77762 (0.0010) [2023-12-26 15:55:54,135][105692] Updated weights for policy 0, policy_version 77772 (0.0010) [2023-12-26 15:55:54,747][105620] Updated weights for policy 1, policy_version 78137 (0.0010) [2023-12-26 15:55:54,798][105692] Updated weights for policy 0, policy_version 77782 (0.0011) [2023-12-26 15:55:54,806][105620] Updated weights for policy 1, policy_version 78147 (0.0011) [2023-12-26 15:55:54,857][105692] Updated weights for policy 0, policy_version 77792 (0.0011) [2023-12-26 15:55:54,865][105620] Updated weights for policy 1, policy_version 78157 (0.0011) [2023-12-26 15:55:54,916][105692] Updated weights for policy 0, policy_version 77802 (0.0010) [2023-12-26 15:55:55,606][105620] Updated weights for policy 1, policy_version 78167 (0.0011) [2023-12-26 15:55:55,657][105692] Updated weights for policy 0, policy_version 77812 (0.0011) [2023-12-26 15:55:55,671][105620] Updated weights for policy 1, policy_version 78177 (0.0010) [2023-12-26 15:55:55,709][105692] Updated weights for policy 0, policy_version 77822 (0.0010) [2023-12-26 15:55:55,730][105620] Updated weights for policy 1, policy_version 78187 (0.0010) [2023-12-26 15:55:55,757][105692] Updated weights for policy 0, policy_version 77832 (0.0010) [2023-12-26 15:55:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 39952384. Throughput: 0: 9823.3, 1: 10028.5. Samples: 39959084. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 15:55:56,062][104569] Avg episode reward: [(0, '1068.537'), (1, '8829.296')] [2023-12-26 15:55:56,452][105620] Updated weights for policy 1, policy_version 78197 (0.0009) [2023-12-26 15:55:56,504][105692] Updated weights for policy 0, policy_version 77842 (0.0010) [2023-12-26 15:55:56,510][105620] Updated weights for policy 1, policy_version 78207 (0.0007) [2023-12-26 15:55:56,565][105620] Updated weights for policy 1, policy_version 78217 (0.0007) [2023-12-26 15:55:56,567][105692] Updated weights for policy 0, policy_version 77852 (0.0010) [2023-12-26 15:55:56,627][105692] Updated weights for policy 0, policy_version 77862 (0.0010) [2023-12-26 15:55:56,681][105692] Updated weights for policy 0, policy_version 77872 (0.0010) [2023-12-26 15:55:57,212][105620] Updated weights for policy 1, policy_version 78227 (0.0006) [2023-12-26 15:55:57,275][105620] Updated weights for policy 1, policy_version 78237 (0.0007) [2023-12-26 15:55:57,336][105620] Updated weights for policy 1, policy_version 78247 (0.0006) [2023-12-26 15:55:57,338][105692] Updated weights for policy 0, policy_version 77882 (0.0009) [2023-12-26 15:55:57,393][105692] Updated weights for policy 0, policy_version 77892 (0.0011) [2023-12-26 15:55:57,453][105692] Updated weights for policy 0, policy_version 77902 (0.0008) [2023-12-26 15:55:58,000][105692] Updated weights for policy 0, policy_version 77912 (0.0006) [2023-12-26 15:55:58,058][105692] Updated weights for policy 0, policy_version 77922 (0.0010) [2023-12-26 15:55:58,128][105692] Updated weights for policy 0, policy_version 77932 (0.0010) [2023-12-26 15:55:58,164][105620] Updated weights for policy 1, policy_version 78257 (0.0006) [2023-12-26 15:55:58,229][105620] Updated weights for policy 1, policy_version 78267 (0.0008) [2023-12-26 15:55:58,294][105620] Updated weights for policy 1, policy_version 78277 (0.0008) [2023-12-26 15:55:58,362][105620] Updated weights for policy 1, policy_version 78287 (0.0008) [2023-12-26 15:55:58,909][105692] Updated weights for policy 0, policy_version 77942 (0.0009) [2023-12-26 15:55:58,969][105692] Updated weights for policy 0, policy_version 77952 (0.0008) [2023-12-26 15:55:59,030][105692] Updated weights for policy 0, policy_version 77962 (0.0008) [2023-12-26 15:55:59,136][105620] Updated weights for policy 1, policy_version 78297 (0.0010) [2023-12-26 15:55:59,192][105620] Updated weights for policy 1, policy_version 78307 (0.0011) [2023-12-26 15:55:59,259][105620] Updated weights for policy 1, policy_version 78317 (0.0010) [2023-12-26 15:55:59,795][105692] Updated weights for policy 0, policy_version 77972 (0.0008) [2023-12-26 15:55:59,858][105692] Updated weights for policy 0, policy_version 77982 (0.0008) [2023-12-26 15:55:59,910][105692] Updated weights for policy 0, policy_version 77992 (0.0008) [2023-12-26 15:56:00,033][105620] Updated weights for policy 1, policy_version 78327 (0.0008) [2023-12-26 15:56:00,091][105620] Updated weights for policy 1, policy_version 78337 (0.0008) [2023-12-26 15:56:00,152][105620] Updated weights for policy 1, policy_version 78347 (0.0008) [2023-12-26 15:56:00,682][105692] Updated weights for policy 0, policy_version 78002 (0.0007) [2023-12-26 15:56:00,739][105692] Updated weights for policy 0, policy_version 78012 (0.0005) [2023-12-26 15:56:00,774][105620] Updated weights for policy 1, policy_version 78357 (0.0006) [2023-12-26 15:56:00,807][105692] Updated weights for policy 0, policy_version 78022 (0.0005) [2023-12-26 15:56:00,841][105620] Updated weights for policy 1, policy_version 78367 (0.0006) [2023-12-26 15:56:00,861][105692] Updated weights for policy 0, policy_version 78032 (0.0005) [2023-12-26 15:56:00,903][105620] Updated weights for policy 1, policy_version 78377 (0.0006) [2023-12-26 15:56:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19660.8). Total num frames: 40050688. Throughput: 0: 9874.3, 1: 10022.6. Samples: 40017608. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 15:56:01,063][104569] Avg episode reward: [(0, '2494.684'), (1, '8822.255')] [2023-12-26 15:56:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000078032_19980288.pth... [2023-12-26 15:56:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000078384_20070400.pth... [2023-12-26 15:56:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000076912_19693568.pth [2023-12-26 15:56:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000077232_19775488.pth [2023-12-26 15:56:01,539][105620] Updated weights for policy 1, policy_version 78387 (0.0005) [2023-12-26 15:56:01,545][105692] Updated weights for policy 0, policy_version 78042 (0.0010) [2023-12-26 15:56:01,594][105620] Updated weights for policy 1, policy_version 78397 (0.0006) [2023-12-26 15:56:01,596][105692] Updated weights for policy 0, policy_version 78052 (0.0010) [2023-12-26 15:56:01,657][105620] Updated weights for policy 1, policy_version 78407 (0.0007) [2023-12-26 15:56:01,662][105692] Updated weights for policy 0, policy_version 78062 (0.0007) [2023-12-26 15:56:02,311][105692] Updated weights for policy 0, policy_version 78072 (0.0009) [2023-12-26 15:56:02,378][105692] Updated weights for policy 0, policy_version 78082 (0.0009) [2023-12-26 15:56:02,412][105620] Updated weights for policy 1, policy_version 78417 (0.0007) [2023-12-26 15:56:02,438][105692] Updated weights for policy 0, policy_version 78092 (0.0009) [2023-12-26 15:56:02,467][105620] Updated weights for policy 1, policy_version 78427 (0.0006) [2023-12-26 15:56:02,520][105620] Updated weights for policy 1, policy_version 78437 (0.0007) [2023-12-26 15:56:02,572][105620] Updated weights for policy 1, policy_version 78447 (0.0008) [2023-12-26 15:56:03,104][105692] Updated weights for policy 0, policy_version 78102 (0.0010) [2023-12-26 15:56:03,165][105692] Updated weights for policy 0, policy_version 78112 (0.0006) [2023-12-26 15:56:03,224][105692] Updated weights for policy 0, policy_version 78122 (0.0005) [2023-12-26 15:56:03,340][105620] Updated weights for policy 1, policy_version 78457 (0.0009) [2023-12-26 15:56:03,392][105620] Updated weights for policy 1, policy_version 78467 (0.0009) [2023-12-26 15:56:03,447][105620] Updated weights for policy 1, policy_version 78477 (0.0008) [2023-12-26 15:56:03,867][105692] Updated weights for policy 0, policy_version 78132 (0.0006) [2023-12-26 15:56:03,931][105692] Updated weights for policy 0, policy_version 78142 (0.0008) [2023-12-26 15:56:03,994][105692] Updated weights for policy 0, policy_version 78152 (0.0011) [2023-12-26 15:56:04,187][105620] Updated weights for policy 1, policy_version 78487 (0.0008) [2023-12-26 15:56:04,244][105620] Updated weights for policy 1, policy_version 78497 (0.0008) [2023-12-26 15:56:04,303][105620] Updated weights for policy 1, policy_version 78507 (0.0008) [2023-12-26 15:56:04,682][105692] Updated weights for policy 0, policy_version 78162 (0.0011) [2023-12-26 15:56:04,737][105692] Updated weights for policy 0, policy_version 78172 (0.0010) [2023-12-26 15:56:04,791][105692] Updated weights for policy 0, policy_version 78182 (0.0010) [2023-12-26 15:56:04,840][105692] Updated weights for policy 0, policy_version 78192 (0.0010) [2023-12-26 15:56:05,084][105620] Updated weights for policy 1, policy_version 78517 (0.0008) [2023-12-26 15:56:05,139][105620] Updated weights for policy 1, policy_version 78527 (0.0008) [2023-12-26 15:56:05,187][105620] Updated weights for policy 1, policy_version 78537 (0.0008) [2023-12-26 15:56:05,592][105692] Updated weights for policy 0, policy_version 78202 (0.0010) [2023-12-26 15:56:05,642][105692] Updated weights for policy 0, policy_version 78212 (0.0010) [2023-12-26 15:56:05,693][105692] Updated weights for policy 0, policy_version 78222 (0.0005) [2023-12-26 15:56:05,961][105620] Updated weights for policy 1, policy_version 78547 (0.0008) [2023-12-26 15:56:06,013][105620] Updated weights for policy 1, policy_version 78557 (0.0007) [2023-12-26 15:56:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.9, 300 sec: 19633.0). Total num frames: 40140800. Throughput: 0: 9746.3, 1: 9898.8. Samples: 40133720. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 15:56:06,063][104569] Avg episode reward: [(0, '4715.324'), (1, '9095.464')] [2023-12-26 15:56:06,071][105620] Updated weights for policy 1, policy_version 78567 (0.0008) [2023-12-26 15:56:06,123][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000010 [2023-12-26 15:56:06,442][105692] Updated weights for policy 0, policy_version 78232 (0.0010) [2023-12-26 15:56:06,506][105692] Updated weights for policy 0, policy_version 78242 (0.0011) [2023-12-26 15:56:06,567][105692] Updated weights for policy 0, policy_version 78252 (0.0011) [2023-12-26 15:56:06,897][105620] Updated weights for policy 1, policy_version 78577 (0.0008) [2023-12-26 15:56:06,952][105620] Updated weights for policy 1, policy_version 78587 (0.0008) [2023-12-26 15:56:07,010][105620] Updated weights for policy 1, policy_version 78597 (0.0009) [2023-12-26 15:56:07,063][105620] Updated weights for policy 1, policy_version 78607 (0.0008) [2023-12-26 15:56:07,321][105692] Updated weights for policy 0, policy_version 78262 (0.0010) [2023-12-26 15:56:07,383][105692] Updated weights for policy 0, policy_version 78272 (0.0010) [2023-12-26 15:56:07,447][105692] Updated weights for policy 0, policy_version 78282 (0.0010) [2023-12-26 15:56:07,872][105620] Updated weights for policy 1, policy_version 78617 (0.0009) [2023-12-26 15:56:07,928][105620] Updated weights for policy 1, policy_version 78629 (0.0010) [2023-12-26 15:56:07,982][105620] Updated weights for policy 1, policy_version 78640 (0.0010) [2023-12-26 15:56:08,030][105692] Updated weights for policy 0, policy_version 78292 (0.0007) [2023-12-26 15:56:08,082][105692] Updated weights for policy 0, policy_version 78302 (0.0006) [2023-12-26 15:56:08,138][105692] Updated weights for policy 0, policy_version 78312 (0.0006) [2023-12-26 15:56:08,722][105692] Updated weights for policy 0, policy_version 78322 (0.0008) [2023-12-26 15:56:08,785][105692] Updated weights for policy 0, policy_version 78332 (0.0006) [2023-12-26 15:56:08,848][105692] Updated weights for policy 0, policy_version 78342 (0.0008) [2023-12-26 15:56:08,885][105620] Updated weights for policy 1, policy_version 78650 (0.0008) [2023-12-26 15:56:08,906][105692] Updated weights for policy 0, policy_version 78352 (0.0010) [2023-12-26 15:56:08,943][105620] Updated weights for policy 1, policy_version 78660 (0.0007) [2023-12-26 15:56:08,991][105620] Updated weights for policy 1, policy_version 78670 (0.0008) [2023-12-26 15:56:09,625][105692] Updated weights for policy 0, policy_version 78362 (0.0011) [2023-12-26 15:56:09,687][105692] Updated weights for policy 0, policy_version 78372 (0.0010) [2023-12-26 15:56:09,749][105692] Updated weights for policy 0, policy_version 78382 (0.0011) [2023-12-26 15:56:09,801][105620] Updated weights for policy 1, policy_version 78680 (0.0007) [2023-12-26 15:56:09,869][105620] Updated weights for policy 1, policy_version 78690 (0.0008) [2023-12-26 15:56:09,923][105620] Updated weights for policy 1, policy_version 78700 (0.0008) [2023-12-26 15:56:10,479][105692] Updated weights for policy 0, policy_version 78392 (0.0011) [2023-12-26 15:56:10,545][105692] Updated weights for policy 0, policy_version 78402 (0.0011) [2023-12-26 15:56:10,595][105692] Updated weights for policy 0, policy_version 78412 (0.0009) [2023-12-26 15:56:10,655][105620] Updated weights for policy 1, policy_version 78710 (0.0008) [2023-12-26 15:56:10,713][105620] Updated weights for policy 1, policy_version 78720 (0.0006) [2023-12-26 15:56:10,771][105620] Updated weights for policy 1, policy_version 78730 (0.0006) [2023-12-26 15:56:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 40239104. Throughput: 0: 9730.2, 1: 9785.6. Samples: 40246896. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 15:56:11,062][104569] Avg episode reward: [(0, '6730.483'), (1, '9096.472')] [2023-12-26 15:56:11,269][105692] Updated weights for policy 0, policy_version 78422 (0.0009) [2023-12-26 15:56:11,338][105692] Updated weights for policy 0, policy_version 78432 (0.0006) [2023-12-26 15:56:11,403][105692] Updated weights for policy 0, policy_version 78442 (0.0009) [2023-12-26 15:56:11,450][105620] Updated weights for policy 1, policy_version 78740 (0.0009) [2023-12-26 15:56:11,520][105620] Updated weights for policy 1, policy_version 78750 (0.0011) [2023-12-26 15:56:11,591][105620] Updated weights for policy 1, policy_version 78760 (0.0011) [2023-12-26 15:56:12,139][105692] Updated weights for policy 0, policy_version 78452 (0.0009) [2023-12-26 15:56:12,201][105692] Updated weights for policy 0, policy_version 78462 (0.0009) [2023-12-26 15:56:12,237][105620] Updated weights for policy 1, policy_version 78770 (0.0008) [2023-12-26 15:56:12,267][105692] Updated weights for policy 0, policy_version 78472 (0.0009) [2023-12-26 15:56:12,299][105620] Updated weights for policy 1, policy_version 78780 (0.0007) [2023-12-26 15:56:12,356][105620] Updated weights for policy 1, policy_version 78790 (0.0008) [2023-12-26 15:56:12,421][105620] Updated weights for policy 1, policy_version 78800 (0.0009) [2023-12-26 15:56:13,028][105692] Updated weights for policy 0, policy_version 78482 (0.0009) [2023-12-26 15:56:13,081][105692] Updated weights for policy 0, policy_version 78492 (0.0010) [2023-12-26 15:56:13,140][105692] Updated weights for policy 0, policy_version 78502 (0.0010) [2023-12-26 15:56:13,202][105692] Updated weights for policy 0, policy_version 78512 (0.0010) [2023-12-26 15:56:13,208][105620] Updated weights for policy 1, policy_version 78810 (0.0006) [2023-12-26 15:56:13,270][105620] Updated weights for policy 1, policy_version 78820 (0.0008) [2023-12-26 15:56:13,329][105620] Updated weights for policy 1, policy_version 78830 (0.0008) [2023-12-26 15:56:13,927][105692] Updated weights for policy 0, policy_version 78522 (0.0005) [2023-12-26 15:56:13,986][105692] Updated weights for policy 0, policy_version 78532 (0.0009) [2023-12-26 15:56:14,043][105692] Updated weights for policy 0, policy_version 78542 (0.0010) [2023-12-26 15:56:14,114][105620] Updated weights for policy 1, policy_version 78840 (0.0010) [2023-12-26 15:56:14,168][105620] Updated weights for policy 1, policy_version 78850 (0.0010) [2023-12-26 15:56:14,226][105620] Updated weights for policy 1, policy_version 78860 (0.0010) [2023-12-26 15:56:14,618][105692] Updated weights for policy 0, policy_version 78552 (0.0009) [2023-12-26 15:56:14,678][105692] Updated weights for policy 0, policy_version 78562 (0.0006) [2023-12-26 15:56:14,746][105692] Updated weights for policy 0, policy_version 78572 (0.0010) [2023-12-26 15:56:15,004][105620] Updated weights for policy 1, policy_version 78870 (0.0010) [2023-12-26 15:56:15,063][105620] Updated weights for policy 1, policy_version 78880 (0.0011) [2023-12-26 15:56:15,130][105620] Updated weights for policy 1, policy_version 78890 (0.0011) [2023-12-26 15:56:15,435][105692] Updated weights for policy 0, policy_version 78582 (0.0011) [2023-12-26 15:56:15,494][105692] Updated weights for policy 0, policy_version 78592 (0.0007) [2023-12-26 15:56:15,554][105692] Updated weights for policy 0, policy_version 78602 (0.0011) [2023-12-26 15:56:15,743][105620] Updated weights for policy 1, policy_version 78900 (0.0010) [2023-12-26 15:56:15,801][105620] Updated weights for policy 1, policy_version 78910 (0.0007) [2023-12-26 15:56:15,855][105620] Updated weights for policy 1, policy_version 78920 (0.0005) [2023-12-26 15:56:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 40337408. Throughput: 0: 9717.9, 1: 9718.4. Samples: 40303648. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 15:56:16,062][104569] Avg episode reward: [(0, '7410.354'), (1, '9092.729')] [2023-12-26 15:56:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000078928_20209664.pth... [2023-12-26 15:56:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000078608_20127744.pth... [2023-12-26 15:56:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000077456_19832832.pth [2023-12-26 15:56:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000077808_19922944.pth [2023-12-26 15:56:16,250][105692] Updated weights for policy 0, policy_version 78612 (0.0009) [2023-12-26 15:56:16,304][105692] Updated weights for policy 0, policy_version 78622 (0.0008) [2023-12-26 15:56:16,356][105692] Updated weights for policy 0, policy_version 78632 (0.0008) [2023-12-26 15:56:16,410][105620] Updated weights for policy 1, policy_version 78930 (0.0005) [2023-12-26 15:56:16,471][105620] Updated weights for policy 1, policy_version 78940 (0.0005) [2023-12-26 15:56:16,531][105620] Updated weights for policy 1, policy_version 78950 (0.0005) [2023-12-26 15:56:16,585][105620] Updated weights for policy 1, policy_version 78960 (0.0006) [2023-12-26 15:56:16,934][105692] Updated weights for policy 0, policy_version 78642 (0.0007) [2023-12-26 15:56:16,986][105692] Updated weights for policy 0, policy_version 78652 (0.0006) [2023-12-26 15:56:17,045][105692] Updated weights for policy 0, policy_version 78662 (0.0006) [2023-12-26 15:56:17,104][105692] Updated weights for policy 0, policy_version 78672 (0.0010) [2023-12-26 15:56:17,263][105620] Updated weights for policy 1, policy_version 78970 (0.0010) [2023-12-26 15:56:17,317][105620] Updated weights for policy 1, policy_version 78980 (0.0010) [2023-12-26 15:56:17,374][105620] Updated weights for policy 1, policy_version 78990 (0.0007) [2023-12-26 15:56:17,800][105692] Updated weights for policy 0, policy_version 78682 (0.0010) [2023-12-26 15:56:17,855][105692] Updated weights for policy 0, policy_version 78692 (0.0010) [2023-12-26 15:56:17,913][105692] Updated weights for policy 0, policy_version 78702 (0.0010) [2023-12-26 15:56:17,922][105620] Updated weights for policy 1, policy_version 79000 (0.0005) [2023-12-26 15:56:17,987][105620] Updated weights for policy 1, policy_version 79010 (0.0008) [2023-12-26 15:56:18,036][105620] Updated weights for policy 1, policy_version 79020 (0.0010) [2023-12-26 15:56:18,655][105692] Updated weights for policy 0, policy_version 78712 (0.0010) [2023-12-26 15:56:18,718][105692] Updated weights for policy 0, policy_version 78722 (0.0009) [2023-12-26 15:56:18,773][105620] Updated weights for policy 1, policy_version 79030 (0.0008) [2023-12-26 15:56:18,779][105692] Updated weights for policy 0, policy_version 78732 (0.0009) [2023-12-26 15:56:18,825][105620] Updated weights for policy 1, policy_version 79040 (0.0007) [2023-12-26 15:56:18,877][105620] Updated weights for policy 1, policy_version 79050 (0.0008) [2023-12-26 15:56:19,523][105692] Updated weights for policy 0, policy_version 78742 (0.0008) [2023-12-26 15:56:19,582][105692] Updated weights for policy 0, policy_version 78752 (0.0009) [2023-12-26 15:56:19,644][105692] Updated weights for policy 0, policy_version 78762 (0.0010) [2023-12-26 15:56:19,675][105620] Updated weights for policy 1, policy_version 79060 (0.0008) [2023-12-26 15:56:19,728][105620] Updated weights for policy 1, policy_version 79070 (0.0006) [2023-12-26 15:56:19,775][105620] Updated weights for policy 1, policy_version 79080 (0.0005) [2023-12-26 15:56:20,421][105692] Updated weights for policy 0, policy_version 78772 (0.0008) [2023-12-26 15:56:20,479][105692] Updated weights for policy 0, policy_version 78782 (0.0006) [2023-12-26 15:56:20,527][105692] Updated weights for policy 0, policy_version 78792 (0.0006) [2023-12-26 15:56:20,570][105620] Updated weights for policy 1, policy_version 79090 (0.0007) [2023-12-26 15:56:20,633][105620] Updated weights for policy 1, policy_version 79100 (0.0009) [2023-12-26 15:56:20,699][105620] Updated weights for policy 1, policy_version 79110 (0.0009) [2023-12-26 15:56:20,771][105620] Updated weights for policy 1, policy_version 79120 (0.0009) [2023-12-26 15:56:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 40435712. Throughput: 0: 9773.4, 1: 9697.8. Samples: 40426172. Policy #0 lag: (min: 27.0, avg: 32.5, max: 59.0) [2023-12-26 15:56:21,062][104569] Avg episode reward: [(0, '8468.598'), (1, '9004.822')] [2023-12-26 15:56:21,223][105692] Updated weights for policy 0, policy_version 78802 (0.0007) [2023-12-26 15:56:21,285][105692] Updated weights for policy 0, policy_version 78812 (0.0009) [2023-12-26 15:56:21,352][105692] Updated weights for policy 0, policy_version 78822 (0.0009) [2023-12-26 15:56:21,411][105692] Updated weights for policy 0, policy_version 78832 (0.0007) [2023-12-26 15:56:21,541][105620] Updated weights for policy 1, policy_version 79130 (0.0009) [2023-12-26 15:56:21,588][105620] Updated weights for policy 1, policy_version 79140 (0.0008) [2023-12-26 15:56:21,649][105620] Updated weights for policy 1, policy_version 79150 (0.0009) [2023-12-26 15:56:22,187][105692] Updated weights for policy 0, policy_version 78842 (0.0009) [2023-12-26 15:56:22,251][105692] Updated weights for policy 0, policy_version 78852 (0.0009) [2023-12-26 15:56:22,310][105692] Updated weights for policy 0, policy_version 78862 (0.0009) [2023-12-26 15:56:22,433][105620] Updated weights for policy 1, policy_version 79160 (0.0009) [2023-12-26 15:56:22,494][105620] Updated weights for policy 1, policy_version 79170 (0.0008) [2023-12-26 15:56:22,555][105620] Updated weights for policy 1, policy_version 79180 (0.0009) [2023-12-26 15:56:22,990][105692] Updated weights for policy 0, policy_version 78872 (0.0009) [2023-12-26 15:56:23,049][105692] Updated weights for policy 0, policy_version 78882 (0.0009) [2023-12-26 15:56:23,105][105692] Updated weights for policy 0, policy_version 78892 (0.0009) [2023-12-26 15:56:23,407][105620] Updated weights for policy 1, policy_version 79190 (0.0007) [2023-12-26 15:56:23,463][105620] Updated weights for policy 1, policy_version 79200 (0.0009) [2023-12-26 15:56:23,523][105620] Updated weights for policy 1, policy_version 79210 (0.0009) [2023-12-26 15:56:23,680][105692] Updated weights for policy 0, policy_version 78902 (0.0007) [2023-12-26 15:56:23,735][105692] Updated weights for policy 0, policy_version 78912 (0.0007) [2023-12-26 15:56:23,793][105692] Updated weights for policy 0, policy_version 78922 (0.0010) [2023-12-26 15:56:24,293][105620] Updated weights for policy 1, policy_version 79220 (0.0010) [2023-12-26 15:56:24,360][105620] Updated weights for policy 1, policy_version 79230 (0.0010) [2023-12-26 15:56:24,394][105692] Updated weights for policy 0, policy_version 78932 (0.0010) [2023-12-26 15:56:24,412][105620] Updated weights for policy 1, policy_version 79240 (0.0010) [2023-12-26 15:56:24,449][105692] Updated weights for policy 0, policy_version 78942 (0.0010) [2023-12-26 15:56:24,497][105692] Updated weights for policy 0, policy_version 78952 (0.0010) [2023-12-26 15:56:25,051][105620] Updated weights for policy 1, policy_version 79250 (0.0007) [2023-12-26 15:56:25,074][105692] Updated weights for policy 0, policy_version 78962 (0.0010) [2023-12-26 15:56:25,110][105620] Updated weights for policy 1, policy_version 79260 (0.0011) [2023-12-26 15:56:25,122][105692] Updated weights for policy 0, policy_version 78972 (0.0005) [2023-12-26 15:56:25,168][105620] Updated weights for policy 1, policy_version 79270 (0.0010) [2023-12-26 15:56:25,174][105692] Updated weights for policy 0, policy_version 78982 (0.0005) [2023-12-26 15:56:25,227][105620] Updated weights for policy 1, policy_version 79280 (0.0010) [2023-12-26 15:56:25,231][105692] Updated weights for policy 0, policy_version 78992 (0.0005) [2023-12-26 15:56:25,820][105692] Updated weights for policy 0, policy_version 79002 (0.0010) [2023-12-26 15:56:25,886][105692] Updated weights for policy 0, policy_version 79012 (0.0011) [2023-12-26 15:56:25,920][105620] Updated weights for policy 1, policy_version 79290 (0.0010) [2023-12-26 15:56:25,936][105692] Updated weights for policy 0, policy_version 79022 (0.0009) [2023-12-26 15:56:25,980][105620] Updated weights for policy 1, policy_version 79300 (0.0010) [2023-12-26 15:56:26,040][105620] Updated weights for policy 1, policy_version 79310 (0.0010) [2023-12-26 15:56:26,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 40542208. Throughput: 0: 9943.8, 1: 9613.3. Samples: 40544540. Policy #0 lag: (min: 27.0, avg: 32.5, max: 59.0) [2023-12-26 15:56:26,063][104569] Avg episode reward: [(0, '8914.020'), (1, '8830.853')] [2023-12-26 15:56:26,679][105692] Updated weights for policy 0, policy_version 79032 (0.0010) [2023-12-26 15:56:26,723][105692] Updated weights for policy 0, policy_version 79042 (0.0010) [2023-12-26 15:56:26,767][105692] Updated weights for policy 0, policy_version 79052 (0.0010) [2023-12-26 15:56:26,782][105620] Updated weights for policy 1, policy_version 79320 (0.0008) [2023-12-26 15:56:26,830][105620] Updated weights for policy 1, policy_version 79330 (0.0008) [2023-12-26 15:56:26,877][105620] Updated weights for policy 1, policy_version 79340 (0.0008) [2023-12-26 15:56:27,486][105692] Updated weights for policy 0, policy_version 79062 (0.0005) [2023-12-26 15:56:27,538][105692] Updated weights for policy 0, policy_version 79072 (0.0009) [2023-12-26 15:56:27,582][105692] Updated weights for policy 0, policy_version 79082 (0.0010) [2023-12-26 15:56:27,648][105620] Updated weights for policy 1, policy_version 79350 (0.0006) [2023-12-26 15:56:27,736][105620] Updated weights for policy 1, policy_version 79363 (0.0010) [2023-12-26 15:56:27,804][105620] Updated weights for policy 1, policy_version 79373 (0.0010) [2023-12-26 15:56:28,366][105692] Updated weights for policy 0, policy_version 79092 (0.0009) [2023-12-26 15:56:28,423][105692] Updated weights for policy 0, policy_version 79102 (0.0007) [2023-12-26 15:56:28,478][105692] Updated weights for policy 0, policy_version 79112 (0.0008) [2023-12-26 15:56:28,480][105620] Updated weights for policy 1, policy_version 79383 (0.0008) [2023-12-26 15:56:28,542][105620] Updated weights for policy 1, policy_version 79393 (0.0008) [2023-12-26 15:56:28,609][105620] Updated weights for policy 1, policy_version 79403 (0.0009) [2023-12-26 15:56:29,057][105692] Updated weights for policy 0, policy_version 79122 (0.0007) [2023-12-26 15:56:29,104][105692] Updated weights for policy 0, policy_version 79132 (0.0005) [2023-12-26 15:56:29,155][105692] Updated weights for policy 0, policy_version 79142 (0.0005) [2023-12-26 15:56:29,214][105692] Updated weights for policy 0, policy_version 79152 (0.0006) [2023-12-26 15:56:29,287][105620] Updated weights for policy 1, policy_version 79413 (0.0009) [2023-12-26 15:56:29,344][105620] Updated weights for policy 1, policy_version 79423 (0.0008) [2023-12-26 15:56:29,409][105620] Updated weights for policy 1, policy_version 79433 (0.0008) [2023-12-26 15:56:29,918][105692] Updated weights for policy 0, policy_version 79162 (0.0010) [2023-12-26 15:56:29,980][105692] Updated weights for policy 0, policy_version 79172 (0.0006) [2023-12-26 15:56:30,034][105692] Updated weights for policy 0, policy_version 79182 (0.0008) [2023-12-26 15:56:30,141][105620] Updated weights for policy 1, policy_version 79443 (0.0007) [2023-12-26 15:56:30,206][105620] Updated weights for policy 1, policy_version 79453 (0.0007) [2023-12-26 15:56:30,271][105620] Updated weights for policy 1, policy_version 79463 (0.0008) [2023-12-26 15:56:30,752][105692] Updated weights for policy 0, policy_version 79192 (0.0010) [2023-12-26 15:56:30,803][105692] Updated weights for policy 0, policy_version 79202 (0.0010) [2023-12-26 15:56:30,854][105692] Updated weights for policy 0, policy_version 79212 (0.0010) [2023-12-26 15:56:31,001][105620] Updated weights for policy 1, policy_version 79473 (0.0008) [2023-12-26 15:56:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 40632320. Throughput: 0: 9968.9, 1: 9593.9. Samples: 40601192. Policy #0 lag: (min: 27.0, avg: 32.5, max: 59.0) [2023-12-26 15:56:31,062][104569] Avg episode reward: [(0, '8385.447'), (1, '9008.187')] [2023-12-26 15:56:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000079216_20283392.pth... [2023-12-26 15:56:31,067][105620] Updated weights for policy 1, policy_version 79483 (0.0010) [2023-12-26 15:56:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000078032_19980288.pth [2023-12-26 15:56:31,133][105620] Updated weights for policy 1, policy_version 79493 (0.0007) [2023-12-26 15:56:31,183][105620] Updated weights for policy 1, policy_version 79503 (0.0010) [2023-12-26 15:56:31,187][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000079504_20357120.pth... [2023-12-26 15:56:31,190][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000078384_20070400.pth [2023-12-26 15:56:31,591][105692] Updated weights for policy 0, policy_version 79222 (0.0010) [2023-12-26 15:56:31,659][105692] Updated weights for policy 0, policy_version 79232 (0.0011) [2023-12-26 15:56:31,719][105692] Updated weights for policy 0, policy_version 79242 (0.0009) [2023-12-26 15:56:31,925][105620] Updated weights for policy 1, policy_version 79513 (0.0009) [2023-12-26 15:56:31,981][105620] Updated weights for policy 1, policy_version 79523 (0.0008) [2023-12-26 15:56:32,034][105620] Updated weights for policy 1, policy_version 79533 (0.0008) [2023-12-26 15:56:32,406][105692] Updated weights for policy 0, policy_version 79252 (0.0006) [2023-12-26 15:56:32,457][105692] Updated weights for policy 0, policy_version 79262 (0.0008) [2023-12-26 15:56:32,516][105692] Updated weights for policy 0, policy_version 79272 (0.0009) [2023-12-26 15:56:32,726][105620] Updated weights for policy 1, policy_version 79543 (0.0009) [2023-12-26 15:56:32,773][105620] Updated weights for policy 1, policy_version 79553 (0.0009) [2023-12-26 15:56:32,828][105620] Updated weights for policy 1, policy_version 79563 (0.0009) [2023-12-26 15:56:33,179][105692] Updated weights for policy 0, policy_version 79282 (0.0009) [2023-12-26 15:56:33,234][105692] Updated weights for policy 0, policy_version 79292 (0.0009) [2023-12-26 15:56:33,291][105692] Updated weights for policy 0, policy_version 79302 (0.0009) [2023-12-26 15:56:33,348][105692] Updated weights for policy 0, policy_version 79312 (0.0009) [2023-12-26 15:56:33,435][105620] Updated weights for policy 1, policy_version 79573 (0.0007) [2023-12-26 15:56:33,493][105620] Updated weights for policy 1, policy_version 79583 (0.0009) [2023-12-26 15:56:33,550][105620] Updated weights for policy 1, policy_version 79593 (0.0009) [2023-12-26 15:56:34,100][105692] Updated weights for policy 0, policy_version 79322 (0.0009) [2023-12-26 15:56:34,159][105692] Updated weights for policy 0, policy_version 79332 (0.0008) [2023-12-26 15:56:34,219][105692] Updated weights for policy 0, policy_version 79342 (0.0008) [2023-12-26 15:56:34,278][105620] Updated weights for policy 1, policy_version 79603 (0.0009) [2023-12-26 15:56:34,340][105620] Updated weights for policy 1, policy_version 79613 (0.0011) [2023-12-26 15:56:34,403][105620] Updated weights for policy 1, policy_version 79623 (0.0010) [2023-12-26 15:56:35,052][105692] Updated weights for policy 0, policy_version 79352 (0.0009) [2023-12-26 15:56:35,058][105620] Updated weights for policy 1, policy_version 79633 (0.0010) [2023-12-26 15:56:35,111][105692] Updated weights for policy 0, policy_version 79362 (0.0009) [2023-12-26 15:56:35,115][105620] Updated weights for policy 1, policy_version 79643 (0.0007) [2023-12-26 15:56:35,165][105692] Updated weights for policy 0, policy_version 79372 (0.0007) [2023-12-26 15:56:35,166][105620] Updated weights for policy 1, policy_version 79653 (0.0010) [2023-12-26 15:56:35,221][105620] Updated weights for policy 1, policy_version 79663 (0.0010) [2023-12-26 15:56:35,894][105620] Updated weights for policy 1, policy_version 79673 (0.0006) [2023-12-26 15:56:35,950][105620] Updated weights for policy 1, policy_version 79683 (0.0009) [2023-12-26 15:56:35,957][105692] Updated weights for policy 0, policy_version 79382 (0.0007) [2023-12-26 15:56:36,007][105620] Updated weights for policy 1, policy_version 79693 (0.0006) [2023-12-26 15:56:36,019][105692] Updated weights for policy 0, policy_version 79392 (0.0009) [2023-12-26 15:56:36,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 40730624. Throughput: 0: 9906.1, 1: 9534.7. Samples: 40721048. Policy #0 lag: (min: 27.0, avg: 32.5, max: 59.0) [2023-12-26 15:56:36,062][104569] Avg episode reward: [(0, '8296.812'), (1, '9270.803')] [2023-12-26 15:56:36,069][105692] Updated weights for policy 0, policy_version 79402 (0.0009) [2023-12-26 15:56:36,579][105620] Updated weights for policy 1, policy_version 79703 (0.0009) [2023-12-26 15:56:36,643][105620] Updated weights for policy 1, policy_version 79713 (0.0011) [2023-12-26 15:56:36,706][105620] Updated weights for policy 1, policy_version 79723 (0.0011) [2023-12-26 15:56:36,929][105692] Updated weights for policy 0, policy_version 79412 (0.0008) [2023-12-26 15:56:36,994][105692] Updated weights for policy 0, policy_version 79422 (0.0008) [2023-12-26 15:56:37,055][105692] Updated weights for policy 0, policy_version 79432 (0.0008) [2023-12-26 15:56:37,415][105620] Updated weights for policy 1, policy_version 79733 (0.0008) [2023-12-26 15:56:37,464][105620] Updated weights for policy 1, policy_version 79743 (0.0006) [2023-12-26 15:56:37,518][105620] Updated weights for policy 1, policy_version 79753 (0.0005) [2023-12-26 15:56:37,877][105692] Updated weights for policy 0, policy_version 79442 (0.0008) [2023-12-26 15:56:37,931][105692] Updated weights for policy 0, policy_version 79453 (0.0010) [2023-12-26 15:56:37,992][105692] Updated weights for policy 0, policy_version 79463 (0.0005) [2023-12-26 15:56:38,051][105620] Updated weights for policy 1, policy_version 79763 (0.0007) [2023-12-26 15:56:38,105][105620] Updated weights for policy 1, policy_version 79773 (0.0006) [2023-12-26 15:56:38,156][105620] Updated weights for policy 1, policy_version 79783 (0.0005) [2023-12-26 15:56:38,737][105692] Updated weights for policy 0, policy_version 79473 (0.0006) [2023-12-26 15:56:38,805][105692] Updated weights for policy 0, policy_version 79483 (0.0008) [2023-12-26 15:56:38,862][105692] Updated weights for policy 0, policy_version 79493 (0.0008) [2023-12-26 15:56:38,878][105620] Updated weights for policy 1, policy_version 79793 (0.0007) [2023-12-26 15:56:38,920][105692] Updated weights for policy 0, policy_version 79503 (0.0007) [2023-12-26 15:56:38,940][105620] Updated weights for policy 1, policy_version 79803 (0.0011) [2023-12-26 15:56:39,001][105620] Updated weights for policy 1, policy_version 79813 (0.0010) [2023-12-26 15:56:39,049][105620] Updated weights for policy 1, policy_version 79823 (0.0010) [2023-12-26 15:56:39,664][105692] Updated weights for policy 0, policy_version 79513 (0.0009) [2023-12-26 15:56:39,725][105692] Updated weights for policy 0, policy_version 79523 (0.0008) [2023-12-26 15:56:39,783][105692] Updated weights for policy 0, policy_version 79533 (0.0009) [2023-12-26 15:56:39,809][105620] Updated weights for policy 1, policy_version 79833 (0.0009) [2023-12-26 15:56:39,880][105620] Updated weights for policy 1, policy_version 79843 (0.0011) [2023-12-26 15:56:39,949][105620] Updated weights for policy 1, policy_version 79853 (0.0011) [2023-12-26 15:56:40,506][105692] Updated weights for policy 0, policy_version 79543 (0.0007) [2023-12-26 15:56:40,566][105692] Updated weights for policy 0, policy_version 79553 (0.0006) [2023-12-26 15:56:40,628][105692] Updated weights for policy 0, policy_version 79563 (0.0011) [2023-12-26 15:56:40,675][105620] Updated weights for policy 1, policy_version 79863 (0.0010) [2023-12-26 15:56:40,724][105620] Updated weights for policy 1, policy_version 79873 (0.0010) [2023-12-26 15:56:40,783][105620] Updated weights for policy 1, policy_version 79883 (0.0010) [2023-12-26 15:56:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 40828928. Throughput: 0: 9851.5, 1: 9635.2. Samples: 40835984. Policy #0 lag: (min: 27.0, avg: 32.5, max: 59.0) [2023-12-26 15:56:41,062][104569] Avg episode reward: [(0, '9003.833'), (1, '9179.666')] [2023-12-26 15:56:41,342][105692] Updated weights for policy 0, policy_version 79573 (0.0009) [2023-12-26 15:56:41,405][105692] Updated weights for policy 0, policy_version 79583 (0.0009) [2023-12-26 15:56:41,458][105692] Updated weights for policy 0, policy_version 79593 (0.0010) [2023-12-26 15:56:41,517][105620] Updated weights for policy 1, policy_version 79893 (0.0008) [2023-12-26 15:56:41,574][105620] Updated weights for policy 1, policy_version 79903 (0.0008) [2023-12-26 15:56:41,638][105620] Updated weights for policy 1, policy_version 79913 (0.0007) [2023-12-26 15:56:42,282][105692] Updated weights for policy 0, policy_version 79603 (0.0009) [2023-12-26 15:56:42,344][105692] Updated weights for policy 0, policy_version 79613 (0.0007) [2023-12-26 15:56:42,399][105620] Updated weights for policy 1, policy_version 79923 (0.0008) [2023-12-26 15:56:42,409][105692] Updated weights for policy 0, policy_version 79623 (0.0008) [2023-12-26 15:56:42,460][105620] Updated weights for policy 1, policy_version 79933 (0.0009) [2023-12-26 15:56:42,522][105620] Updated weights for policy 1, policy_version 79943 (0.0009) [2023-12-26 15:56:43,129][105692] Updated weights for policy 0, policy_version 79633 (0.0006) [2023-12-26 15:56:43,176][105692] Updated weights for policy 0, policy_version 79643 (0.0005) [2023-12-26 15:56:43,228][105692] Updated weights for policy 0, policy_version 79653 (0.0006) [2023-12-26 15:56:43,283][105692] Updated weights for policy 0, policy_version 79663 (0.0005) [2023-12-26 15:56:43,338][105620] Updated weights for policy 1, policy_version 79953 (0.0009) [2023-12-26 15:56:43,392][105620] Updated weights for policy 1, policy_version 79963 (0.0007) [2023-12-26 15:56:43,440][105620] Updated weights for policy 1, policy_version 79973 (0.0005) [2023-12-26 15:56:43,498][105620] Updated weights for policy 1, policy_version 79983 (0.0005) [2023-12-26 15:56:43,965][105692] Updated weights for policy 0, policy_version 79673 (0.0007) [2023-12-26 15:56:44,018][105692] Updated weights for policy 0, policy_version 79683 (0.0007) [2023-12-26 15:56:44,068][105692] Updated weights for policy 0, policy_version 79693 (0.0007) [2023-12-26 15:56:44,093][105620] Updated weights for policy 1, policy_version 79993 (0.0005) [2023-12-26 15:56:44,156][105620] Updated weights for policy 1, policy_version 80003 (0.0005) [2023-12-26 15:56:44,213][105620] Updated weights for policy 1, policy_version 80013 (0.0007) [2023-12-26 15:56:44,795][105692] Updated weights for policy 0, policy_version 79703 (0.0009) [2023-12-26 15:56:44,846][105620] Updated weights for policy 1, policy_version 80023 (0.0006) [2023-12-26 15:56:44,855][105692] Updated weights for policy 0, policy_version 79713 (0.0008) [2023-12-26 15:56:44,904][105692] Updated weights for policy 0, policy_version 79723 (0.0006) [2023-12-26 15:56:44,910][105620] Updated weights for policy 1, policy_version 80033 (0.0007) [2023-12-26 15:56:44,974][105620] Updated weights for policy 1, policy_version 80043 (0.0010) [2023-12-26 15:56:45,567][105692] Updated weights for policy 0, policy_version 79733 (0.0006) [2023-12-26 15:56:45,618][105692] Updated weights for policy 0, policy_version 79743 (0.0008) [2023-12-26 15:56:45,673][105692] Updated weights for policy 0, policy_version 79753 (0.0009) [2023-12-26 15:56:45,737][105620] Updated weights for policy 1, policy_version 80053 (0.0007) [2023-12-26 15:56:45,798][105620] Updated weights for policy 1, policy_version 80063 (0.0005) [2023-12-26 15:56:45,859][105620] Updated weights for policy 1, policy_version 80073 (0.0005) [2023-12-26 15:56:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 40927232. Throughput: 0: 9803.2, 1: 9651.1. Samples: 40893052. Policy #0 lag: (min: 27.0, avg: 32.5, max: 59.0) [2023-12-26 15:56:46,063][104569] Avg episode reward: [(0, '9089.395'), (1, '9091.495')] [2023-12-26 15:56:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000079760_20422656.pth... [2023-12-26 15:56:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000080080_20504576.pth... [2023-12-26 15:56:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000078608_20127744.pth [2023-12-26 15:56:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000078928_20209664.pth [2023-12-26 15:56:46,478][105620] Updated weights for policy 1, policy_version 80083 (0.0007) [2023-12-26 15:56:46,485][105692] Updated weights for policy 0, policy_version 79763 (0.0010) [2023-12-26 15:56:46,535][105620] Updated weights for policy 1, policy_version 80093 (0.0005) [2023-12-26 15:56:46,537][105692] Updated weights for policy 0, policy_version 79773 (0.0008) [2023-12-26 15:56:46,590][105692] Updated weights for policy 0, policy_version 79783 (0.0009) [2023-12-26 15:56:46,594][105620] Updated weights for policy 1, policy_version 80103 (0.0005) [2023-12-26 15:56:47,193][105620] Updated weights for policy 1, policy_version 80113 (0.0007) [2023-12-26 15:56:47,245][105620] Updated weights for policy 1, policy_version 80123 (0.0006) [2023-12-26 15:56:47,295][105620] Updated weights for policy 1, policy_version 80133 (0.0005) [2023-12-26 15:56:47,349][105620] Updated weights for policy 1, policy_version 80143 (0.0005) [2023-12-26 15:56:47,433][105692] Updated weights for policy 0, policy_version 79793 (0.0007) [2023-12-26 15:56:47,489][105692] Updated weights for policy 0, policy_version 79803 (0.0009) [2023-12-26 15:56:47,551][105692] Updated weights for policy 0, policy_version 79814 (0.0010) [2023-12-26 15:56:47,608][105692] Updated weights for policy 0, policy_version 79824 (0.0010) [2023-12-26 15:56:47,918][105620] Updated weights for policy 1, policy_version 80153 (0.0010) [2023-12-26 15:56:47,983][105620] Updated weights for policy 1, policy_version 80163 (0.0010) [2023-12-26 15:56:48,043][105620] Updated weights for policy 1, policy_version 80173 (0.0007) [2023-12-26 15:56:48,359][105692] Updated weights for policy 0, policy_version 79834 (0.0007) [2023-12-26 15:56:48,422][105692] Updated weights for policy 0, policy_version 79844 (0.0008) [2023-12-26 15:56:48,478][105692] Updated weights for policy 0, policy_version 79854 (0.0008) [2023-12-26 15:56:48,670][105620] Updated weights for policy 1, policy_version 80183 (0.0006) [2023-12-26 15:56:48,738][105620] Updated weights for policy 1, policy_version 80193 (0.0007) [2023-12-26 15:56:48,798][105620] Updated weights for policy 1, policy_version 80203 (0.0011) [2023-12-26 15:56:49,145][105692] Updated weights for policy 0, policy_version 79864 (0.0008) [2023-12-26 15:56:49,199][105692] Updated weights for policy 0, policy_version 79874 (0.0008) [2023-12-26 15:56:49,259][105692] Updated weights for policy 0, policy_version 79884 (0.0007) [2023-12-26 15:56:49,503][105620] Updated weights for policy 1, policy_version 80213 (0.0008) [2023-12-26 15:56:49,551][105620] Updated weights for policy 1, policy_version 80223 (0.0005) [2023-12-26 15:56:49,599][105620] Updated weights for policy 1, policy_version 80233 (0.0006) [2023-12-26 15:56:50,084][105692] Updated weights for policy 0, policy_version 79894 (0.0008) [2023-12-26 15:56:50,146][105692] Updated weights for policy 0, policy_version 79904 (0.0006) [2023-12-26 15:56:50,213][105692] Updated weights for policy 0, policy_version 79914 (0.0006) [2023-12-26 15:56:50,233][105620] Updated weights for policy 1, policy_version 80243 (0.0007) [2023-12-26 15:56:50,296][105620] Updated weights for policy 1, policy_version 80253 (0.0011) [2023-12-26 15:56:50,361][105620] Updated weights for policy 1, policy_version 80263 (0.0008) [2023-12-26 15:56:50,828][105692] Updated weights for policy 0, policy_version 79924 (0.0009) [2023-12-26 15:56:50,894][105692] Updated weights for policy 0, policy_version 79934 (0.0009) [2023-12-26 15:56:50,952][105692] Updated weights for policy 0, policy_version 79944 (0.0009) [2023-12-26 15:56:51,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 41025536. Throughput: 0: 9761.2, 1: 9787.9. Samples: 41013432. Policy #0 lag: (min: 27.0, avg: 32.5, max: 59.0) [2023-12-26 15:56:51,063][104569] Avg episode reward: [(0, '8999.353'), (1, '9089.816')] [2023-12-26 15:56:51,194][105620] Updated weights for policy 1, policy_version 80273 (0.0009) [2023-12-26 15:56:51,250][105620] Updated weights for policy 1, policy_version 80283 (0.0009) [2023-12-26 15:56:51,308][105620] Updated weights for policy 1, policy_version 80293 (0.0009) [2023-12-26 15:56:51,373][105620] Updated weights for policy 1, policy_version 80303 (0.0009) [2023-12-26 15:56:51,618][105692] Updated weights for policy 0, policy_version 79954 (0.0008) [2023-12-26 15:56:51,679][105692] Updated weights for policy 0, policy_version 79964 (0.0006) [2023-12-26 15:56:51,750][105692] Updated weights for policy 0, policy_version 79974 (0.0009) [2023-12-26 15:56:51,812][105692] Updated weights for policy 0, policy_version 79984 (0.0009) [2023-12-26 15:56:52,181][105620] Updated weights for policy 1, policy_version 80313 (0.0007) [2023-12-26 15:56:52,228][105620] Updated weights for policy 1, policy_version 80323 (0.0005) [2023-12-26 15:56:52,292][105620] Updated weights for policy 1, policy_version 80333 (0.0006) [2023-12-26 15:56:52,490][105692] Updated weights for policy 0, policy_version 79994 (0.0007) [2023-12-26 15:56:52,556][105692] Updated weights for policy 0, policy_version 80004 (0.0005) [2023-12-26 15:56:52,616][105692] Updated weights for policy 0, policy_version 80014 (0.0005) [2023-12-26 15:56:52,912][105620] Updated weights for policy 1, policy_version 80343 (0.0007) [2023-12-26 15:56:52,971][105620] Updated weights for policy 1, policy_version 80353 (0.0008) [2023-12-26 15:56:53,042][105620] Updated weights for policy 1, policy_version 80363 (0.0005) [2023-12-26 15:56:53,319][105692] Updated weights for policy 0, policy_version 80024 (0.0009) [2023-12-26 15:56:53,372][105692] Updated weights for policy 0, policy_version 80035 (0.0011) [2023-12-26 15:56:53,431][105692] Updated weights for policy 0, policy_version 80046 (0.0010) [2023-12-26 15:56:53,612][105620] Updated weights for policy 1, policy_version 80373 (0.0007) [2023-12-26 15:56:53,665][105620] Updated weights for policy 1, policy_version 80383 (0.0010) [2023-12-26 15:56:53,718][105620] Updated weights for policy 1, policy_version 80393 (0.0009) [2023-12-26 15:56:54,147][105692] Updated weights for policy 0, policy_version 80056 (0.0009) [2023-12-26 15:56:54,209][105692] Updated weights for policy 0, policy_version 80066 (0.0009) [2023-12-26 15:56:54,265][105692] Updated weights for policy 0, policy_version 80076 (0.0009) [2023-12-26 15:56:54,529][105620] Updated weights for policy 1, policy_version 80403 (0.0010) [2023-12-26 15:56:54,582][105620] Updated weights for policy 1, policy_version 80414 (0.0009) [2023-12-26 15:56:54,630][105620] Updated weights for policy 1, policy_version 80424 (0.0008) [2023-12-26 15:56:54,910][105692] Updated weights for policy 0, policy_version 80086 (0.0009) [2023-12-26 15:56:54,968][105692] Updated weights for policy 0, policy_version 80096 (0.0009) [2023-12-26 15:56:55,023][105692] Updated weights for policy 0, policy_version 80106 (0.0009) [2023-12-26 15:56:55,360][105620] Updated weights for policy 1, policy_version 80434 (0.0009) [2023-12-26 15:56:55,421][105620] Updated weights for policy 1, policy_version 80444 (0.0009) [2023-12-26 15:56:55,482][105620] Updated weights for policy 1, policy_version 80454 (0.0009) [2023-12-26 15:56:55,540][105620] Updated weights for policy 1, policy_version 80464 (0.0009) [2023-12-26 15:56:55,803][105692] Updated weights for policy 0, policy_version 80116 (0.0009) [2023-12-26 15:56:55,857][105692] Updated weights for policy 0, policy_version 80126 (0.0009) [2023-12-26 15:56:55,903][105692] Updated weights for policy 0, policy_version 80136 (0.0008) [2023-12-26 15:56:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 41123840. Throughput: 0: 9760.3, 1: 9875.8. Samples: 41130524. Policy #0 lag: (min: 27.0, avg: 32.5, max: 59.0) [2023-12-26 15:56:56,063][104569] Avg episode reward: [(0, '8472.340'), (1, '9091.299')] [2023-12-26 15:56:56,290][105620] Updated weights for policy 1, policy_version 80474 (0.0009) [2023-12-26 15:56:56,338][105620] Updated weights for policy 1, policy_version 80484 (0.0009) [2023-12-26 15:56:56,384][105620] Updated weights for policy 1, policy_version 80494 (0.0009) [2023-12-26 15:56:56,647][105692] Updated weights for policy 0, policy_version 80146 (0.0008) [2023-12-26 15:56:56,700][105692] Updated weights for policy 0, policy_version 80156 (0.0009) [2023-12-26 15:56:56,760][105692] Updated weights for policy 0, policy_version 80166 (0.0009) [2023-12-26 15:56:56,822][105692] Updated weights for policy 0, policy_version 80176 (0.0009) [2023-12-26 15:56:57,102][105620] Updated weights for policy 1, policy_version 80504 (0.0006) [2023-12-26 15:56:57,165][105620] Updated weights for policy 1, policy_version 80514 (0.0010) [2023-12-26 15:56:57,231][105620] Updated weights for policy 1, policy_version 80524 (0.0010) [2023-12-26 15:56:57,569][105692] Updated weights for policy 0, policy_version 80186 (0.0005) [2023-12-26 15:56:57,623][105692] Updated weights for policy 0, policy_version 80196 (0.0007) [2023-12-26 15:56:57,702][105692] Updated weights for policy 0, policy_version 80206 (0.0008) [2023-12-26 15:56:57,820][105620] Updated weights for policy 1, policy_version 80534 (0.0007) [2023-12-26 15:56:57,871][105620] Updated weights for policy 1, policy_version 80544 (0.0008) [2023-12-26 15:56:57,919][105620] Updated weights for policy 1, policy_version 80554 (0.0007) [2023-12-26 15:56:58,455][105692] Updated weights for policy 0, policy_version 80216 (0.0008) [2023-12-26 15:56:58,526][105692] Updated weights for policy 0, policy_version 80227 (0.0008) [2023-12-26 15:56:58,588][105692] Updated weights for policy 0, policy_version 80237 (0.0008) [2023-12-26 15:56:58,688][105620] Updated weights for policy 1, policy_version 80564 (0.0008) [2023-12-26 15:56:58,758][105620] Updated weights for policy 1, policy_version 80574 (0.0008) [2023-12-26 15:56:58,816][105620] Updated weights for policy 1, policy_version 80584 (0.0008) [2023-12-26 15:56:59,237][105692] Updated weights for policy 0, policy_version 80247 (0.0009) [2023-12-26 15:56:59,300][105692] Updated weights for policy 0, policy_version 80257 (0.0009) [2023-12-26 15:56:59,366][105692] Updated weights for policy 0, policy_version 80267 (0.0009) [2023-12-26 15:56:59,637][105620] Updated weights for policy 1, policy_version 80594 (0.0008) [2023-12-26 15:56:59,694][105620] Updated weights for policy 1, policy_version 80605 (0.0010) [2023-12-26 15:56:59,748][105620] Updated weights for policy 1, policy_version 80616 (0.0010) [2023-12-26 15:57:00,001][105692] Updated weights for policy 0, policy_version 80277 (0.0009) [2023-12-26 15:57:00,053][105692] Updated weights for policy 0, policy_version 80287 (0.0009) [2023-12-26 15:57:00,109][105692] Updated weights for policy 0, policy_version 80297 (0.0009) [2023-12-26 15:57:00,489][105620] Updated weights for policy 1, policy_version 80626 (0.0009) [2023-12-26 15:57:00,559][105620] Updated weights for policy 1, policy_version 80636 (0.0010) [2023-12-26 15:57:00,619][105620] Updated weights for policy 1, policy_version 80646 (0.0010) [2023-12-26 15:57:00,677][105620] Updated weights for policy 1, policy_version 80656 (0.0010) [2023-12-26 15:57:00,821][105692] Updated weights for policy 0, policy_version 80307 (0.0009) [2023-12-26 15:57:00,880][105692] Updated weights for policy 0, policy_version 80317 (0.0006) [2023-12-26 15:57:00,926][105692] Updated weights for policy 0, policy_version 80327 (0.0005) [2023-12-26 15:57:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19605.2). Total num frames: 41222144. Throughput: 0: 9749.3, 1: 9895.3. Samples: 41187656. Policy #0 lag: (min: 27.0, avg: 32.5, max: 59.0) [2023-12-26 15:57:01,062][104569] Avg episode reward: [(0, '8032.178'), (1, '9013.181')] [2023-12-26 15:57:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000080336_20570112.pth... [2023-12-26 15:57:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000080656_20652032.pth... [2023-12-26 15:57:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000079504_20357120.pth [2023-12-26 15:57:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000079216_20283392.pth [2023-12-26 15:57:01,519][105620] Updated weights for policy 1, policy_version 80666 (0.0008) [2023-12-26 15:57:01,583][105620] Updated weights for policy 1, policy_version 80676 (0.0008) [2023-12-26 15:57:01,622][105692] Updated weights for policy 0, policy_version 80337 (0.0006) [2023-12-26 15:57:01,645][105620] Updated weights for policy 1, policy_version 80686 (0.0009) [2023-12-26 15:57:01,683][105692] Updated weights for policy 0, policy_version 80347 (0.0008) [2023-12-26 15:57:01,744][105692] Updated weights for policy 0, policy_version 80357 (0.0009) [2023-12-26 15:57:01,795][105692] Updated weights for policy 0, policy_version 80367 (0.0009) [2023-12-26 15:57:02,295][105620] Updated weights for policy 1, policy_version 80696 (0.0008) [2023-12-26 15:57:02,347][105620] Updated weights for policy 1, policy_version 80706 (0.0008) [2023-12-26 15:57:02,407][105620] Updated weights for policy 1, policy_version 80716 (0.0007) [2023-12-26 15:57:02,460][105692] Updated weights for policy 0, policy_version 80377 (0.0009) [2023-12-26 15:57:02,519][105692] Updated weights for policy 0, policy_version 80387 (0.0011) [2023-12-26 15:57:02,574][105692] Updated weights for policy 0, policy_version 80397 (0.0011) [2023-12-26 15:57:02,977][105620] Updated weights for policy 1, policy_version 80726 (0.0009) [2023-12-26 15:57:03,022][105620] Updated weights for policy 1, policy_version 80736 (0.0010) [2023-12-26 15:57:03,071][105620] Updated weights for policy 1, policy_version 80746 (0.0007) [2023-12-26 15:57:03,178][105692] Updated weights for policy 0, policy_version 80407 (0.0007) [2023-12-26 15:57:03,237][105692] Updated weights for policy 0, policy_version 80417 (0.0007) [2023-12-26 15:57:03,287][105692] Updated weights for policy 0, policy_version 80427 (0.0007) [2023-12-26 15:57:03,632][105620] Updated weights for policy 1, policy_version 80756 (0.0005) [2023-12-26 15:57:03,680][105620] Updated weights for policy 1, policy_version 80766 (0.0005) [2023-12-26 15:57:03,724][105620] Updated weights for policy 1, policy_version 80776 (0.0008) [2023-12-26 15:57:03,936][105692] Updated weights for policy 0, policy_version 80437 (0.0008) [2023-12-26 15:57:03,995][105692] Updated weights for policy 0, policy_version 80447 (0.0011) [2023-12-26 15:57:04,054][105692] Updated weights for policy 0, policy_version 80457 (0.0011) [2023-12-26 15:57:04,446][105620] Updated weights for policy 1, policy_version 80786 (0.0010) [2023-12-26 15:57:04,514][105620] Updated weights for policy 1, policy_version 80796 (0.0008) [2023-12-26 15:57:04,580][105620] Updated weights for policy 1, policy_version 80806 (0.0005) [2023-12-26 15:57:04,632][105620] Updated weights for policy 1, policy_version 80816 (0.0008) [2023-12-26 15:57:04,661][105692] Updated weights for policy 0, policy_version 80467 (0.0009) [2023-12-26 15:57:04,718][105692] Updated weights for policy 0, policy_version 80477 (0.0006) [2023-12-26 15:57:04,769][105692] Updated weights for policy 0, policy_version 80487 (0.0010) [2023-12-26 15:57:05,164][105620] Updated weights for policy 1, policy_version 80826 (0.0010) [2023-12-26 15:57:05,213][105620] Updated weights for policy 1, policy_version 80836 (0.0010) [2023-12-26 15:57:05,266][105620] Updated weights for policy 1, policy_version 80846 (0.0010) [2023-12-26 15:57:05,347][105692] Updated weights for policy 0, policy_version 80497 (0.0009) [2023-12-26 15:57:05,410][105692] Updated weights for policy 0, policy_version 80507 (0.0006) [2023-12-26 15:57:05,475][105692] Updated weights for policy 0, policy_version 80517 (0.0006) [2023-12-26 15:57:05,541][105692] Updated weights for policy 0, policy_version 80527 (0.0006) [2023-12-26 15:57:05,872][105620] Updated weights for policy 1, policy_version 80856 (0.0010) [2023-12-26 15:57:05,930][105620] Updated weights for policy 1, policy_version 80866 (0.0010) [2023-12-26 15:57:06,001][105620] Updated weights for policy 1, policy_version 80876 (0.0010) [2023-12-26 15:57:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 41328640. Throughput: 0: 9796.0, 1: 9881.3. Samples: 41311652. Policy #0 lag: (min: 27.0, avg: 32.5, max: 59.0) [2023-12-26 15:57:06,062][104569] Avg episode reward: [(0, '8373.257'), (1, '9019.113')] [2023-12-26 15:57:06,094][105692] Updated weights for policy 0, policy_version 80537 (0.0006) [2023-12-26 15:57:06,155][105692] Updated weights for policy 0, policy_version 80547 (0.0010) [2023-12-26 15:57:06,218][105692] Updated weights for policy 0, policy_version 80557 (0.0011) [2023-12-26 15:57:06,707][105620] Updated weights for policy 1, policy_version 80886 (0.0011) [2023-12-26 15:57:06,763][105620] Updated weights for policy 1, policy_version 80896 (0.0011) [2023-12-26 15:57:06,825][105620] Updated weights for policy 1, policy_version 80906 (0.0010) [2023-12-26 15:57:06,888][105692] Updated weights for policy 0, policy_version 80567 (0.0007) [2023-12-26 15:57:06,942][105692] Updated weights for policy 0, policy_version 80577 (0.0006) [2023-12-26 15:57:07,004][105692] Updated weights for policy 0, policy_version 80587 (0.0008) [2023-12-26 15:57:07,582][105620] Updated weights for policy 1, policy_version 80916 (0.0010) [2023-12-26 15:57:07,642][105620] Updated weights for policy 1, policy_version 80926 (0.0008) [2023-12-26 15:57:07,664][105692] Updated weights for policy 0, policy_version 80597 (0.0009) [2023-12-26 15:57:07,694][105620] Updated weights for policy 1, policy_version 80936 (0.0005) [2023-12-26 15:57:07,716][105692] Updated weights for policy 0, policy_version 80607 (0.0010) [2023-12-26 15:57:07,777][105692] Updated weights for policy 0, policy_version 80617 (0.0011) [2023-12-26 15:57:08,376][105692] Updated weights for policy 0, policy_version 80627 (0.0011) [2023-12-26 15:57:08,428][105692] Updated weights for policy 0, policy_version 80637 (0.0009) [2023-12-26 15:57:08,487][105620] Updated weights for policy 1, policy_version 80946 (0.0006) [2023-12-26 15:57:08,489][105692] Updated weights for policy 0, policy_version 80647 (0.0007) [2023-12-26 15:57:08,550][105620] Updated weights for policy 1, policy_version 80956 (0.0007) [2023-12-26 15:57:08,619][105620] Updated weights for policy 1, policy_version 80966 (0.0009) [2023-12-26 15:57:08,686][105620] Updated weights for policy 1, policy_version 80976 (0.0010) [2023-12-26 15:57:09,172][105692] Updated weights for policy 0, policy_version 80657 (0.0007) [2023-12-26 15:57:09,241][105692] Updated weights for policy 0, policy_version 80667 (0.0009) [2023-12-26 15:57:09,304][105692] Updated weights for policy 0, policy_version 80677 (0.0008) [2023-12-26 15:57:09,370][105692] Updated weights for policy 0, policy_version 80687 (0.0009) [2023-12-26 15:57:09,507][105620] Updated weights for policy 1, policy_version 80986 (0.0008) [2023-12-26 15:57:09,567][105620] Updated weights for policy 1, policy_version 80996 (0.0007) [2023-12-26 15:57:09,636][105620] Updated weights for policy 1, policy_version 81006 (0.0008) [2023-12-26 15:57:10,114][105692] Updated weights for policy 0, policy_version 80697 (0.0008) [2023-12-26 15:57:10,177][105692] Updated weights for policy 0, policy_version 80707 (0.0009) [2023-12-26 15:57:10,240][105692] Updated weights for policy 0, policy_version 80717 (0.0009) [2023-12-26 15:57:10,329][105620] Updated weights for policy 1, policy_version 81016 (0.0009) [2023-12-26 15:57:10,385][105620] Updated weights for policy 1, policy_version 81026 (0.0008) [2023-12-26 15:57:10,446][105620] Updated weights for policy 1, policy_version 81036 (0.0006) [2023-12-26 15:57:11,049][105620] Updated weights for policy 1, policy_version 81046 (0.0011) [2023-12-26 15:57:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 41418752. Throughput: 0: 9787.0, 1: 9959.7. Samples: 41433144. Policy #0 lag: (min: 1.0, avg: 15.6, max: 33.0) [2023-12-26 15:57:11,062][104569] Avg episode reward: [(0, '8284.188'), (1, '8916.760')] [2023-12-26 15:57:11,099][105692] Updated weights for policy 0, policy_version 80727 (0.0007) [2023-12-26 15:57:11,109][105620] Updated weights for policy 1, policy_version 81056 (0.0011) [2023-12-26 15:57:11,167][105692] Updated weights for policy 0, policy_version 80737 (0.0007) [2023-12-26 15:57:11,181][105620] Updated weights for policy 1, policy_version 81066 (0.0011) [2023-12-26 15:57:11,228][105692] Updated weights for policy 0, policy_version 80747 (0.0008) [2023-12-26 15:57:11,935][105620] Updated weights for policy 1, policy_version 81076 (0.0010) [2023-12-26 15:57:11,993][105620] Updated weights for policy 1, policy_version 81086 (0.0009) [2023-12-26 15:57:12,025][105692] Updated weights for policy 0, policy_version 80757 (0.0007) [2023-12-26 15:57:12,051][105620] Updated weights for policy 1, policy_version 81096 (0.0008) [2023-12-26 15:57:12,086][105692] Updated weights for policy 0, policy_version 80767 (0.0008) [2023-12-26 15:57:12,145][105692] Updated weights for policy 0, policy_version 80777 (0.0007) [2023-12-26 15:57:12,867][105620] Updated weights for policy 1, policy_version 81106 (0.0006) [2023-12-26 15:57:12,875][105692] Updated weights for policy 0, policy_version 80787 (0.0009) [2023-12-26 15:57:12,924][105620] Updated weights for policy 1, policy_version 81116 (0.0007) [2023-12-26 15:57:12,926][105692] Updated weights for policy 0, policy_version 80797 (0.0006) [2023-12-26 15:57:12,971][105692] Updated weights for policy 0, policy_version 80807 (0.0006) [2023-12-26 15:57:12,977][105620] Updated weights for policy 1, policy_version 81126 (0.0008) [2023-12-26 15:57:13,038][105620] Updated weights for policy 1, policy_version 81136 (0.0008) [2023-12-26 15:57:13,721][105692] Updated weights for policy 0, policy_version 80817 (0.0006) [2023-12-26 15:57:13,778][105692] Updated weights for policy 0, policy_version 80827 (0.0008) [2023-12-26 15:57:13,824][105620] Updated weights for policy 1, policy_version 81146 (0.0009) [2023-12-26 15:57:13,833][105692] Updated weights for policy 0, policy_version 80837 (0.0005) [2023-12-26 15:57:13,885][105620] Updated weights for policy 1, policy_version 81156 (0.0008) [2023-12-26 15:57:13,889][105692] Updated weights for policy 0, policy_version 80847 (0.0005) [2023-12-26 15:57:13,959][105620] Updated weights for policy 1, policy_version 81166 (0.0010) [2023-12-26 15:57:14,541][105692] Updated weights for policy 0, policy_version 80857 (0.0009) [2023-12-26 15:57:14,590][105620] Updated weights for policy 1, policy_version 81176 (0.0006) [2023-12-26 15:57:14,601][105692] Updated weights for policy 0, policy_version 80867 (0.0009) [2023-12-26 15:57:14,643][105620] Updated weights for policy 1, policy_version 81186 (0.0005) [2023-12-26 15:57:14,655][105692] Updated weights for policy 0, policy_version 80877 (0.0008) [2023-12-26 15:57:14,694][105620] Updated weights for policy 1, policy_version 81196 (0.0005) [2023-12-26 15:57:15,309][105692] Updated weights for policy 0, policy_version 80887 (0.0006) [2023-12-26 15:57:15,312][105620] Updated weights for policy 1, policy_version 81206 (0.0008) [2023-12-26 15:57:15,367][105692] Updated weights for policy 0, policy_version 80897 (0.0006) [2023-12-26 15:57:15,372][105620] Updated weights for policy 1, policy_version 81216 (0.0010) [2023-12-26 15:57:15,423][105692] Updated weights for policy 0, policy_version 80907 (0.0006) [2023-12-26 15:57:15,432][105620] Updated weights for policy 1, policy_version 81226 (0.0011) [2023-12-26 15:57:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19660.7, 300 sec: 19605.3). Total num frames: 41517056. Throughput: 0: 9759.2, 1: 9926.4. Samples: 41487052. Policy #0 lag: (min: 1.0, avg: 15.6, max: 33.0) [2023-12-26 15:57:16,063][104569] Avg episode reward: [(0, '8465.147'), (1, '9267.141')] [2023-12-26 15:57:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000080912_20717568.pth... [2023-12-26 15:57:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000081232_20799488.pth... [2023-12-26 15:57:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000080080_20504576.pth [2023-12-26 15:57:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000079760_20422656.pth [2023-12-26 15:57:16,127][105620] Updated weights for policy 1, policy_version 81236 (0.0011) [2023-12-26 15:57:16,174][105692] Updated weights for policy 0, policy_version 80917 (0.0009) [2023-12-26 15:57:16,178][105620] Updated weights for policy 1, policy_version 81246 (0.0010) [2023-12-26 15:57:16,224][105692] Updated weights for policy 0, policy_version 80927 (0.0010) [2023-12-26 15:57:16,236][105620] Updated weights for policy 1, policy_version 81256 (0.0010) [2023-12-26 15:57:16,273][105692] Updated weights for policy 0, policy_version 80937 (0.0010) [2023-12-26 15:57:16,937][105620] Updated weights for policy 1, policy_version 81266 (0.0009) [2023-12-26 15:57:17,005][105620] Updated weights for policy 1, policy_version 81276 (0.0006) [2023-12-26 15:57:17,025][105692] Updated weights for policy 0, policy_version 80947 (0.0009) [2023-12-26 15:57:17,074][105620] Updated weights for policy 1, policy_version 81286 (0.0009) [2023-12-26 15:57:17,080][105692] Updated weights for policy 0, policy_version 80957 (0.0010) [2023-12-26 15:57:17,129][105620] Updated weights for policy 1, policy_version 81296 (0.0010) [2023-12-26 15:57:17,132][105692] Updated weights for policy 0, policy_version 80967 (0.0010) [2023-12-26 15:57:17,697][105620] Updated weights for policy 1, policy_version 81306 (0.0006) [2023-12-26 15:57:17,766][105620] Updated weights for policy 1, policy_version 81316 (0.0005) [2023-12-26 15:57:17,839][105620] Updated weights for policy 1, policy_version 81326 (0.0005) [2023-12-26 15:57:17,881][105692] Updated weights for policy 0, policy_version 80977 (0.0010) [2023-12-26 15:57:17,929][105692] Updated weights for policy 0, policy_version 80987 (0.0010) [2023-12-26 15:57:17,986][105692] Updated weights for policy 0, policy_version 80997 (0.0010) [2023-12-26 15:57:18,041][105692] Updated weights for policy 0, policy_version 81007 (0.0010) [2023-12-26 15:57:18,431][105620] Updated weights for policy 1, policy_version 81336 (0.0010) [2023-12-26 15:57:18,493][105620] Updated weights for policy 1, policy_version 81346 (0.0011) [2023-12-26 15:57:18,551][105620] Updated weights for policy 1, policy_version 81356 (0.0010) [2023-12-26 15:57:18,670][105692] Updated weights for policy 0, policy_version 81017 (0.0006) [2023-12-26 15:57:18,740][105692] Updated weights for policy 0, policy_version 81027 (0.0005) [2023-12-26 15:57:18,799][105692] Updated weights for policy 0, policy_version 81037 (0.0005) [2023-12-26 15:57:19,279][105620] Updated weights for policy 1, policy_version 81366 (0.0011) [2023-12-26 15:57:19,344][105620] Updated weights for policy 1, policy_version 81376 (0.0009) [2023-12-26 15:57:19,406][105620] Updated weights for policy 1, policy_version 81386 (0.0007) [2023-12-26 15:57:19,505][105692] Updated weights for policy 0, policy_version 81047 (0.0008) [2023-12-26 15:57:19,578][105692] Updated weights for policy 0, policy_version 81057 (0.0010) [2023-12-26 15:57:19,643][105692] Updated weights for policy 0, policy_version 81067 (0.0010) [2023-12-26 15:57:20,089][105620] Updated weights for policy 1, policy_version 81396 (0.0006) [2023-12-26 15:57:20,146][105620] Updated weights for policy 1, policy_version 81406 (0.0009) [2023-12-26 15:57:20,198][105620] Updated weights for policy 1, policy_version 81416 (0.0008) [2023-12-26 15:57:20,430][105692] Updated weights for policy 0, policy_version 81077 (0.0010) [2023-12-26 15:57:20,489][105692] Updated weights for policy 0, policy_version 81087 (0.0009) [2023-12-26 15:57:20,543][105692] Updated weights for policy 0, policy_version 81098 (0.0010) [2023-12-26 15:57:20,875][105620] Updated weights for policy 1, policy_version 81426 (0.0008) [2023-12-26 15:57:20,941][105620] Updated weights for policy 1, policy_version 81436 (0.0007) [2023-12-26 15:57:21,001][105620] Updated weights for policy 1, policy_version 81446 (0.0007) [2023-12-26 15:57:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 41615360. Throughput: 0: 9770.5, 1: 9986.6. Samples: 41610120. Policy #0 lag: (min: 1.0, avg: 15.6, max: 33.0) [2023-12-26 15:57:21,063][104569] Avg episode reward: [(0, '8729.817'), (1, '9268.374')] [2023-12-26 15:57:21,066][105620] Updated weights for policy 1, policy_version 81456 (0.0008) [2023-12-26 15:57:21,329][105692] Updated weights for policy 0, policy_version 81108 (0.0009) [2023-12-26 15:57:21,400][105692] Updated weights for policy 0, policy_version 81118 (0.0011) [2023-12-26 15:57:21,464][105692] Updated weights for policy 0, policy_version 81128 (0.0009) [2023-12-26 15:57:21,815][105620] Updated weights for policy 1, policy_version 81466 (0.0008) [2023-12-26 15:57:21,876][105620] Updated weights for policy 1, policy_version 81476 (0.0008) [2023-12-26 15:57:21,934][105620] Updated weights for policy 1, policy_version 81486 (0.0009) [2023-12-26 15:57:22,207][105692] Updated weights for policy 0, policy_version 81138 (0.0010) [2023-12-26 15:57:22,269][105692] Updated weights for policy 0, policy_version 81148 (0.0011) [2023-12-26 15:57:22,331][105692] Updated weights for policy 0, policy_version 81158 (0.0009) [2023-12-26 15:57:22,401][105692] Updated weights for policy 0, policy_version 81168 (0.0007) [2023-12-26 15:57:22,747][105620] Updated weights for policy 1, policy_version 81496 (0.0009) [2023-12-26 15:57:22,802][105620] Updated weights for policy 1, policy_version 81506 (0.0009) [2023-12-26 15:57:22,856][105620] Updated weights for policy 1, policy_version 81516 (0.0009) [2023-12-26 15:57:23,162][105692] Updated weights for policy 0, policy_version 81178 (0.0007) [2023-12-26 15:57:23,218][105692] Updated weights for policy 0, policy_version 81188 (0.0006) [2023-12-26 15:57:23,280][105692] Updated weights for policy 0, policy_version 81198 (0.0005) [2023-12-26 15:57:23,685][105620] Updated weights for policy 1, policy_version 81526 (0.0008) [2023-12-26 15:57:23,743][105620] Updated weights for policy 1, policy_version 81536 (0.0009) [2023-12-26 15:57:23,800][105620] Updated weights for policy 1, policy_version 81546 (0.0009) [2023-12-26 15:57:23,862][105692] Updated weights for policy 0, policy_version 81208 (0.0008) [2023-12-26 15:57:23,925][105692] Updated weights for policy 0, policy_version 81218 (0.0009) [2023-12-26 15:57:23,996][105692] Updated weights for policy 0, policy_version 81228 (0.0010) [2023-12-26 15:57:24,523][105620] Updated weights for policy 1, policy_version 81556 (0.0008) [2023-12-26 15:57:24,579][105620] Updated weights for policy 1, policy_version 81566 (0.0009) [2023-12-26 15:57:24,634][105620] Updated weights for policy 1, policy_version 81576 (0.0010) [2023-12-26 15:57:24,691][105692] Updated weights for policy 0, policy_version 81239 (0.0008) [2023-12-26 15:57:24,753][105692] Updated weights for policy 0, policy_version 81249 (0.0008) [2023-12-26 15:57:24,819][105692] Updated weights for policy 0, policy_version 81259 (0.0008) [2023-12-26 15:57:25,286][105620] Updated weights for policy 1, policy_version 81586 (0.0010) [2023-12-26 15:57:25,333][105620] Updated weights for policy 1, policy_version 81596 (0.0009) [2023-12-26 15:57:25,378][105620] Updated weights for policy 1, policy_version 81606 (0.0008) [2023-12-26 15:57:25,430][105620] Updated weights for policy 1, policy_version 81616 (0.0009) [2023-12-26 15:57:25,564][105692] Updated weights for policy 0, policy_version 81269 (0.0008) [2023-12-26 15:57:25,617][105692] Updated weights for policy 0, policy_version 81279 (0.0009) [2023-12-26 15:57:25,676][105692] Updated weights for policy 0, policy_version 81289 (0.0009) [2023-12-26 15:57:26,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 41713664. Throughput: 0: 9842.5, 1: 9896.6. Samples: 41724244. Policy #0 lag: (min: 1.0, avg: 15.6, max: 33.0) [2023-12-26 15:57:26,063][104569] Avg episode reward: [(0, '7674.163'), (1, '9268.096')] [2023-12-26 15:57:26,131][105620] Updated weights for policy 1, policy_version 81626 (0.0009) [2023-12-26 15:57:26,202][105620] Updated weights for policy 1, policy_version 81636 (0.0010) [2023-12-26 15:57:26,268][105620] Updated weights for policy 1, policy_version 81646 (0.0009) [2023-12-26 15:57:26,394][105692] Updated weights for policy 0, policy_version 81299 (0.0008) [2023-12-26 15:57:26,451][105692] Updated weights for policy 0, policy_version 81309 (0.0005) [2023-12-26 15:57:26,499][105692] Updated weights for policy 0, policy_version 81319 (0.0005) [2023-12-26 15:57:27,009][105620] Updated weights for policy 1, policy_version 81656 (0.0006) [2023-12-26 15:57:27,058][105620] Updated weights for policy 1, policy_version 81666 (0.0008) [2023-12-26 15:57:27,108][105620] Updated weights for policy 1, policy_version 81676 (0.0009) [2023-12-26 15:57:27,192][105692] Updated weights for policy 0, policy_version 81329 (0.0006) [2023-12-26 15:57:27,247][105692] Updated weights for policy 0, policy_version 81339 (0.0009) [2023-12-26 15:57:27,310][105692] Updated weights for policy 0, policy_version 81349 (0.0009) [2023-12-26 15:57:27,369][105692] Updated weights for policy 0, policy_version 81359 (0.0009) [2023-12-26 15:57:27,843][105620] Updated weights for policy 1, policy_version 81686 (0.0010) [2023-12-26 15:57:27,901][105620] Updated weights for policy 1, policy_version 81696 (0.0010) [2023-12-26 15:57:27,959][105620] Updated weights for policy 1, policy_version 81706 (0.0010) [2023-12-26 15:57:28,144][105692] Updated weights for policy 0, policy_version 81369 (0.0007) [2023-12-26 15:57:28,200][105692] Updated weights for policy 0, policy_version 81379 (0.0005) [2023-12-26 15:57:28,256][105692] Updated weights for policy 0, policy_version 81389 (0.0005) [2023-12-26 15:57:28,657][105620] Updated weights for policy 1, policy_version 81716 (0.0010) [2023-12-26 15:57:28,710][105620] Updated weights for policy 1, policy_version 81726 (0.0010) [2023-12-26 15:57:28,768][105620] Updated weights for policy 1, policy_version 81736 (0.0009) [2023-12-26 15:57:28,973][105692] Updated weights for policy 0, policy_version 81399 (0.0008) [2023-12-26 15:57:29,035][105692] Updated weights for policy 0, policy_version 81409 (0.0009) [2023-12-26 15:57:29,096][105692] Updated weights for policy 0, policy_version 81419 (0.0008) [2023-12-26 15:57:29,522][105620] Updated weights for policy 1, policy_version 81746 (0.0011) [2023-12-26 15:57:29,574][105620] Updated weights for policy 1, policy_version 81756 (0.0009) [2023-12-26 15:57:29,639][105620] Updated weights for policy 1, policy_version 81766 (0.0010) [2023-12-26 15:57:29,703][105620] Updated weights for policy 1, policy_version 81776 (0.0010) [2023-12-26 15:57:29,848][105692] Updated weights for policy 0, policy_version 81429 (0.0008) [2023-12-26 15:57:29,910][105692] Updated weights for policy 0, policy_version 81439 (0.0008) [2023-12-26 15:57:29,976][105692] Updated weights for policy 0, policy_version 81449 (0.0008) [2023-12-26 15:57:30,386][105620] Updated weights for policy 1, policy_version 81786 (0.0008) [2023-12-26 15:57:30,432][105620] Updated weights for policy 1, policy_version 81796 (0.0008) [2023-12-26 15:57:30,477][105620] Updated weights for policy 1, policy_version 81806 (0.0008) [2023-12-26 15:57:30,716][105692] Updated weights for policy 0, policy_version 81459 (0.0009) [2023-12-26 15:57:30,771][105692] Updated weights for policy 0, policy_version 81470 (0.0010) [2023-12-26 15:57:30,829][105692] Updated weights for policy 0, policy_version 81481 (0.0010) [2023-12-26 15:57:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 41811968. Throughput: 0: 9852.0, 1: 9898.5. Samples: 41781824. Policy #0 lag: (min: 1.0, avg: 15.6, max: 33.0) [2023-12-26 15:57:31,062][104569] Avg episode reward: [(0, '7591.685'), (1, '8729.876')] [2023-12-26 15:57:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000081808_20946944.pth... [2023-12-26 15:57:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000081488_20865024.pth... [2023-12-26 15:57:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000080656_20652032.pth [2023-12-26 15:57:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000080336_20570112.pth [2023-12-26 15:57:31,168][105620] Updated weights for policy 1, policy_version 81816 (0.0008) [2023-12-26 15:57:31,216][105620] Updated weights for policy 1, policy_version 81826 (0.0010) [2023-12-26 15:57:31,271][105620] Updated weights for policy 1, policy_version 81836 (0.0010) [2023-12-26 15:57:31,620][105692] Updated weights for policy 0, policy_version 81492 (0.0009) [2023-12-26 15:57:31,678][105692] Updated weights for policy 0, policy_version 81502 (0.0008) [2023-12-26 15:57:31,738][105692] Updated weights for policy 0, policy_version 81512 (0.0007) [2023-12-26 15:57:32,038][105620] Updated weights for policy 1, policy_version 81846 (0.0010) [2023-12-26 15:57:32,107][105620] Updated weights for policy 1, policy_version 81856 (0.0010) [2023-12-26 15:57:32,170][105620] Updated weights for policy 1, policy_version 81866 (0.0010) [2023-12-26 15:57:32,337][105692] Updated weights for policy 0, policy_version 81522 (0.0006) [2023-12-26 15:57:32,399][105692] Updated weights for policy 0, policy_version 81532 (0.0008) [2023-12-26 15:57:32,455][105692] Updated weights for policy 0, policy_version 81542 (0.0008) [2023-12-26 15:57:32,510][105692] Updated weights for policy 0, policy_version 81552 (0.0008) [2023-12-26 15:57:32,848][105620] Updated weights for policy 1, policy_version 81876 (0.0009) [2023-12-26 15:57:32,905][105620] Updated weights for policy 1, policy_version 81886 (0.0009) [2023-12-26 15:57:32,966][105620] Updated weights for policy 1, policy_version 81896 (0.0009) [2023-12-26 15:57:33,162][105692] Updated weights for policy 0, policy_version 81562 (0.0005) [2023-12-26 15:57:33,219][105692] Updated weights for policy 0, policy_version 81572 (0.0005) [2023-12-26 15:57:33,273][105692] Updated weights for policy 0, policy_version 81582 (0.0005) [2023-12-26 15:57:33,774][105692] Updated weights for policy 0, policy_version 81592 (0.0005) [2023-12-26 15:57:33,829][105692] Updated weights for policy 0, policy_version 81602 (0.0006) [2023-12-26 15:57:33,872][105620] Updated weights for policy 1, policy_version 81906 (0.0008) [2023-12-26 15:57:33,884][105692] Updated weights for policy 0, policy_version 81612 (0.0005) [2023-12-26 15:57:33,920][105620] Updated weights for policy 1, policy_version 81916 (0.0009) [2023-12-26 15:57:33,972][105620] Updated weights for policy 1, policy_version 81928 (0.0009) [2023-12-26 15:57:34,503][105692] Updated weights for policy 0, policy_version 81622 (0.0007) [2023-12-26 15:57:34,563][105692] Updated weights for policy 0, policy_version 81632 (0.0007) [2023-12-26 15:57:34,625][105692] Updated weights for policy 0, policy_version 81642 (0.0007) [2023-12-26 15:57:34,810][105620] Updated weights for policy 1, policy_version 81939 (0.0009) [2023-12-26 15:57:34,869][105620] Updated weights for policy 1, policy_version 81949 (0.0008) [2023-12-26 15:57:34,926][105620] Updated weights for policy 1, policy_version 81959 (0.0008) [2023-12-26 15:57:35,258][105692] Updated weights for policy 0, policy_version 81652 (0.0005) [2023-12-26 15:57:35,317][105692] Updated weights for policy 0, policy_version 81662 (0.0006) [2023-12-26 15:57:35,371][105692] Updated weights for policy 0, policy_version 81672 (0.0006) [2023-12-26 15:57:35,495][105620] Updated weights for policy 1, policy_version 81969 (0.0006) [2023-12-26 15:57:35,542][105620] Updated weights for policy 1, policy_version 81979 (0.0005) [2023-12-26 15:57:35,588][105620] Updated weights for policy 1, policy_version 81989 (0.0008) [2023-12-26 15:57:35,633][105620] Updated weights for policy 1, policy_version 81999 (0.0010) [2023-12-26 15:57:35,960][105692] Updated weights for policy 0, policy_version 81682 (0.0007) [2023-12-26 15:57:36,017][105692] Updated weights for policy 0, policy_version 81692 (0.0008) [2023-12-26 15:57:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 41910272. Throughput: 0: 9952.4, 1: 9724.1. Samples: 41898872. Policy #0 lag: (min: 1.0, avg: 15.6, max: 33.0) [2023-12-26 15:57:36,062][104569] Avg episode reward: [(0, '8023.452'), (1, '1618.144')] [2023-12-26 15:57:36,081][105692] Updated weights for policy 0, policy_version 81702 (0.0008) [2023-12-26 15:57:36,153][105692] Updated weights for policy 0, policy_version 81712 (0.0008) [2023-12-26 15:57:36,339][105620] Updated weights for policy 1, policy_version 82010 (0.0006) [2023-12-26 15:57:36,395][105620] Updated weights for policy 1, policy_version 82020 (0.0005) [2023-12-26 15:57:36,456][105620] Updated weights for policy 1, policy_version 82030 (0.0010) [2023-12-26 15:57:36,874][105692] Updated weights for policy 0, policy_version 81722 (0.0005) [2023-12-26 15:57:36,928][105692] Updated weights for policy 0, policy_version 81732 (0.0008) [2023-12-26 15:57:36,979][105692] Updated weights for policy 0, policy_version 81742 (0.0008) [2023-12-26 15:57:37,152][105620] Updated weights for policy 1, policy_version 82040 (0.0011) [2023-12-26 15:57:37,204][105620] Updated weights for policy 1, policy_version 82050 (0.0010) [2023-12-26 15:57:37,252][105620] Updated weights for policy 1, policy_version 82060 (0.0010) [2023-12-26 15:57:37,731][105692] Updated weights for policy 0, policy_version 81752 (0.0009) [2023-12-26 15:57:37,784][105692] Updated weights for policy 0, policy_version 81762 (0.0008) [2023-12-26 15:57:37,838][105692] Updated weights for policy 0, policy_version 81772 (0.0008) [2023-12-26 15:57:38,024][105620] Updated weights for policy 1, policy_version 82070 (0.0008) [2023-12-26 15:57:38,085][105620] Updated weights for policy 1, policy_version 82080 (0.0011) [2023-12-26 15:57:38,137][105620] Updated weights for policy 1, policy_version 82090 (0.0010) [2023-12-26 15:57:38,586][105692] Updated weights for policy 0, policy_version 81782 (0.0008) [2023-12-26 15:57:38,650][105692] Updated weights for policy 0, policy_version 81792 (0.0008) [2023-12-26 15:57:38,718][105692] Updated weights for policy 0, policy_version 81802 (0.0006) [2023-12-26 15:57:38,917][105620] Updated weights for policy 1, policy_version 82100 (0.0011) [2023-12-26 15:57:38,980][105620] Updated weights for policy 1, policy_version 82110 (0.0010) [2023-12-26 15:57:39,038][105620] Updated weights for policy 1, policy_version 82120 (0.0010) [2023-12-26 15:57:39,282][105692] Updated weights for policy 0, policy_version 81812 (0.0008) [2023-12-26 15:57:39,348][105692] Updated weights for policy 0, policy_version 81822 (0.0011) [2023-12-26 15:57:39,411][105692] Updated weights for policy 0, policy_version 81832 (0.0011) [2023-12-26 15:57:39,792][105620] Updated weights for policy 1, policy_version 82130 (0.0010) [2023-12-26 15:57:39,858][105620] Updated weights for policy 1, policy_version 82140 (0.0009) [2023-12-26 15:57:39,920][105620] Updated weights for policy 1, policy_version 82150 (0.0008) [2023-12-26 15:57:39,980][105620] Updated weights for policy 1, policy_version 82160 (0.0008) [2023-12-26 15:57:40,194][105692] Updated weights for policy 0, policy_version 81842 (0.0010) [2023-12-26 15:57:40,250][105692] Updated weights for policy 0, policy_version 81852 (0.0010) [2023-12-26 15:57:40,302][105692] Updated weights for policy 0, policy_version 81862 (0.0010) [2023-12-26 15:57:40,354][105692] Updated weights for policy 0, policy_version 81872 (0.0010) [2023-12-26 15:57:40,756][105620] Updated weights for policy 1, policy_version 82170 (0.0008) [2023-12-26 15:57:40,817][105620] Updated weights for policy 1, policy_version 82180 (0.0008) [2023-12-26 15:57:40,884][105620] Updated weights for policy 1, policy_version 82190 (0.0008) [2023-12-26 15:57:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 42008576. Throughput: 0: 9958.9, 1: 9736.5. Samples: 42016812. Policy #0 lag: (min: 1.0, avg: 15.6, max: 33.0) [2023-12-26 15:57:41,063][104569] Avg episode reward: [(0, '7579.980'), (1, '1575.146')] [2023-12-26 15:57:41,151][105692] Updated weights for policy 0, policy_version 81882 (0.0011) [2023-12-26 15:57:41,217][105692] Updated weights for policy 0, policy_version 81892 (0.0010) [2023-12-26 15:57:41,284][105692] Updated weights for policy 0, policy_version 81902 (0.0011) [2023-12-26 15:57:41,708][105620] Updated weights for policy 1, policy_version 82200 (0.0010) [2023-12-26 15:57:41,782][105620] Updated weights for policy 1, policy_version 82210 (0.0011) [2023-12-26 15:57:41,850][105620] Updated weights for policy 1, policy_version 82220 (0.0011) [2023-12-26 15:57:41,984][105692] Updated weights for policy 0, policy_version 81912 (0.0008) [2023-12-26 15:57:42,041][105692] Updated weights for policy 0, policy_version 81922 (0.0005) [2023-12-26 15:57:42,106][105692] Updated weights for policy 0, policy_version 81932 (0.0006) [2023-12-26 15:57:42,570][105620] Updated weights for policy 1, policy_version 82230 (0.0008) [2023-12-26 15:57:42,636][105620] Updated weights for policy 1, policy_version 82240 (0.0006) [2023-12-26 15:57:42,703][105620] Updated weights for policy 1, policy_version 82250 (0.0007) [2023-12-26 15:57:42,751][105692] Updated weights for policy 0, policy_version 81942 (0.0008) [2023-12-26 15:57:42,818][105692] Updated weights for policy 0, policy_version 81952 (0.0008) [2023-12-26 15:57:42,888][105692] Updated weights for policy 0, policy_version 81962 (0.0010) [2023-12-26 15:57:43,366][105620] Updated weights for policy 1, policy_version 82260 (0.0008) [2023-12-26 15:57:43,438][105620] Updated weights for policy 1, policy_version 82270 (0.0009) [2023-12-26 15:57:43,493][105620] Updated weights for policy 1, policy_version 82280 (0.0009) [2023-12-26 15:57:43,548][105692] Updated weights for policy 0, policy_version 81972 (0.0009) [2023-12-26 15:57:43,602][105692] Updated weights for policy 0, policy_version 81982 (0.0010) [2023-12-26 15:57:43,659][105692] Updated weights for policy 0, policy_version 81993 (0.0009) [2023-12-26 15:57:44,069][105620] Updated weights for policy 1, policy_version 82290 (0.0006) [2023-12-26 15:57:44,139][105620] Updated weights for policy 1, policy_version 82300 (0.0005) [2023-12-26 15:57:44,208][105620] Updated weights for policy 1, policy_version 82310 (0.0005) [2023-12-26 15:57:44,268][105620] Updated weights for policy 1, policy_version 82320 (0.0007) [2023-12-26 15:57:44,568][105692] Updated weights for policy 0, policy_version 82003 (0.0009) [2023-12-26 15:57:44,622][105692] Updated weights for policy 0, policy_version 82013 (0.0009) [2023-12-26 15:57:44,671][105692] Updated weights for policy 0, policy_version 82023 (0.0008) [2023-12-26 15:57:44,885][105620] Updated weights for policy 1, policy_version 82330 (0.0009) [2023-12-26 15:57:44,943][105620] Updated weights for policy 1, policy_version 82340 (0.0009) [2023-12-26 15:57:45,005][105620] Updated weights for policy 1, policy_version 82350 (0.0009) [2023-12-26 15:57:45,356][105692] Updated weights for policy 0, policy_version 82033 (0.0009) [2023-12-26 15:57:45,406][105692] Updated weights for policy 0, policy_version 82043 (0.0005) [2023-12-26 15:57:45,468][105692] Updated weights for policy 0, policy_version 82053 (0.0005) [2023-12-26 15:57:45,531][105692] Updated weights for policy 0, policy_version 82063 (0.0006) [2023-12-26 15:57:45,770][105620] Updated weights for policy 1, policy_version 82360 (0.0010) [2023-12-26 15:57:45,827][105620] Updated weights for policy 1, policy_version 82370 (0.0010) [2023-12-26 15:57:45,877][105620] Updated weights for policy 1, policy_version 82380 (0.0009) [2023-12-26 15:57:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 42106880. Throughput: 0: 10005.1, 1: 9734.7. Samples: 42075944. Policy #0 lag: (min: 1.0, avg: 15.6, max: 33.0) [2023-12-26 15:57:46,063][104569] Avg episode reward: [(0, '7579.306'), (1, '6470.925')] [2023-12-26 15:57:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000082384_21094400.pth... [2023-12-26 15:57:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000081232_20799488.pth [2023-12-26 15:57:46,087][105692] Updated weights for policy 0, policy_version 82073 (0.0005) [2023-12-26 15:57:46,147][105692] Updated weights for policy 0, policy_version 82083 (0.0005) [2023-12-26 15:57:46,211][105692] Updated weights for policy 0, policy_version 82093 (0.0005) [2023-12-26 15:57:46,230][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000082096_21020672.pth... [2023-12-26 15:57:46,235][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000080912_20717568.pth [2023-12-26 15:57:46,704][105620] Updated weights for policy 1, policy_version 82390 (0.0009) [2023-12-26 15:57:46,755][105620] Updated weights for policy 1, policy_version 82400 (0.0006) [2023-12-26 15:57:46,802][105692] Updated weights for policy 0, policy_version 82103 (0.0007) [2023-12-26 15:57:46,806][105620] Updated weights for policy 1, policy_version 82410 (0.0006) [2023-12-26 15:57:46,851][105692] Updated weights for policy 0, policy_version 82113 (0.0009) [2023-12-26 15:57:46,916][105692] Updated weights for policy 0, policy_version 82123 (0.0009) [2023-12-26 15:57:47,382][105620] Updated weights for policy 1, policy_version 82420 (0.0005) [2023-12-26 15:57:47,442][105620] Updated weights for policy 1, policy_version 82430 (0.0007) [2023-12-26 15:57:47,507][105620] Updated weights for policy 1, policy_version 82440 (0.0009) [2023-12-26 15:57:47,743][105692] Updated weights for policy 0, policy_version 82133 (0.0010) [2023-12-26 15:57:47,801][105692] Updated weights for policy 0, policy_version 82143 (0.0010) [2023-12-26 15:57:47,855][105692] Updated weights for policy 0, policy_version 82153 (0.0010) [2023-12-26 15:57:48,071][105620] Updated weights for policy 1, policy_version 82450 (0.0005) [2023-12-26 15:57:48,127][105620] Updated weights for policy 1, policy_version 82460 (0.0005) [2023-12-26 15:57:48,189][105620] Updated weights for policy 1, policy_version 82470 (0.0010) [2023-12-26 15:57:48,248][105620] Updated weights for policy 1, policy_version 82480 (0.0011) [2023-12-26 15:57:48,681][105692] Updated weights for policy 0, policy_version 82163 (0.0009) [2023-12-26 15:57:48,744][105692] Updated weights for policy 0, policy_version 82173 (0.0008) [2023-12-26 15:57:48,806][105692] Updated weights for policy 0, policy_version 82183 (0.0010) [2023-12-26 15:57:48,967][105620] Updated weights for policy 1, policy_version 82490 (0.0009) [2023-12-26 15:57:49,025][105620] Updated weights for policy 1, policy_version 82500 (0.0009) [2023-12-26 15:57:49,084][105620] Updated weights for policy 1, policy_version 82510 (0.0009) [2023-12-26 15:57:49,557][105692] Updated weights for policy 0, policy_version 82193 (0.0009) [2023-12-26 15:57:49,606][105692] Updated weights for policy 0, policy_version 82203 (0.0008) [2023-12-26 15:57:49,659][105692] Updated weights for policy 0, policy_version 82213 (0.0010) [2023-12-26 15:57:49,715][105692] Updated weights for policy 0, policy_version 82223 (0.0009) [2023-12-26 15:57:49,835][105620] Updated weights for policy 1, policy_version 82520 (0.0007) [2023-12-26 15:57:49,902][105620] Updated weights for policy 1, policy_version 82530 (0.0006) [2023-12-26 15:57:49,972][105620] Updated weights for policy 1, policy_version 82540 (0.0006) [2023-12-26 15:57:50,569][105620] Updated weights for policy 1, policy_version 82550 (0.0006) [2023-12-26 15:57:50,580][105692] Updated weights for policy 0, policy_version 82233 (0.0008) [2023-12-26 15:57:50,630][105620] Updated weights for policy 1, policy_version 82560 (0.0006) [2023-12-26 15:57:50,642][105692] Updated weights for policy 0, policy_version 82243 (0.0011) [2023-12-26 15:57:50,692][105620] Updated weights for policy 1, policy_version 82570 (0.0006) [2023-12-26 15:57:50,702][105692] Updated weights for policy 0, policy_version 82253 (0.0011) [2023-12-26 15:57:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 42205184. Throughput: 0: 9864.4, 1: 9731.7. Samples: 42193472. Policy #0 lag: (min: 1.0, avg: 15.6, max: 33.0) [2023-12-26 15:57:51,062][104569] Avg episode reward: [(0, '8030.340'), (1, '7740.521')] [2023-12-26 15:57:51,329][105692] Updated weights for policy 0, policy_version 82263 (0.0007) [2023-12-26 15:57:51,395][105692] Updated weights for policy 0, policy_version 82273 (0.0009) [2023-12-26 15:57:51,457][105692] Updated weights for policy 0, policy_version 82283 (0.0009) [2023-12-26 15:57:51,490][105620] Updated weights for policy 1, policy_version 82580 (0.0008) [2023-12-26 15:57:51,551][105620] Updated weights for policy 1, policy_version 82590 (0.0009) [2023-12-26 15:57:51,617][105620] Updated weights for policy 1, policy_version 82600 (0.0009) [2023-12-26 15:57:52,123][105692] Updated weights for policy 0, policy_version 82293 (0.0009) [2023-12-26 15:57:52,184][105692] Updated weights for policy 0, policy_version 82303 (0.0008) [2023-12-26 15:57:52,240][105692] Updated weights for policy 0, policy_version 82313 (0.0009) [2023-12-26 15:57:52,412][105620] Updated weights for policy 1, policy_version 82610 (0.0009) [2023-12-26 15:57:52,482][105620] Updated weights for policy 1, policy_version 82620 (0.0007) [2023-12-26 15:57:52,532][105620] Updated weights for policy 1, policy_version 82630 (0.0005) [2023-12-26 15:57:52,587][105620] Updated weights for policy 1, policy_version 82640 (0.0006) [2023-12-26 15:57:52,958][105692] Updated weights for policy 0, policy_version 82323 (0.0006) [2023-12-26 15:57:53,012][105692] Updated weights for policy 0, policy_version 82333 (0.0006) [2023-12-26 15:57:53,071][105692] Updated weights for policy 0, policy_version 82343 (0.0005) [2023-12-26 15:57:53,366][105620] Updated weights for policy 1, policy_version 82650 (0.0009) [2023-12-26 15:57:53,428][105620] Updated weights for policy 1, policy_version 82660 (0.0007) [2023-12-26 15:57:53,492][105620] Updated weights for policy 1, policy_version 82670 (0.0007) [2023-12-26 15:57:53,720][105692] Updated weights for policy 0, policy_version 82353 (0.0006) [2023-12-26 15:57:53,778][105692] Updated weights for policy 0, policy_version 82363 (0.0010) [2023-12-26 15:57:53,831][105692] Updated weights for policy 0, policy_version 82373 (0.0010) [2023-12-26 15:57:53,884][105692] Updated weights for policy 0, policy_version 82383 (0.0010) [2023-12-26 15:57:54,123][105620] Updated weights for policy 1, policy_version 82680 (0.0006) [2023-12-26 15:57:54,173][105620] Updated weights for policy 1, policy_version 82690 (0.0009) [2023-12-26 15:57:54,221][105620] Updated weights for policy 1, policy_version 82700 (0.0006) [2023-12-26 15:57:54,742][105692] Updated weights for policy 0, policy_version 82393 (0.0009) [2023-12-26 15:57:54,773][105620] Updated weights for policy 1, policy_version 82710 (0.0007) [2023-12-26 15:57:54,784][105692] Updated weights for policy 0, policy_version 82403 (0.0008) [2023-12-26 15:57:54,825][105692] Updated weights for policy 0, policy_version 82413 (0.0008) [2023-12-26 15:57:54,831][105620] Updated weights for policy 1, policy_version 82720 (0.0010) [2023-12-26 15:57:54,887][105620] Updated weights for policy 1, policy_version 82730 (0.0008) [2023-12-26 15:57:55,587][105692] Updated weights for policy 0, policy_version 82423 (0.0006) [2023-12-26 15:57:55,587][105620] Updated weights for policy 1, policy_version 82740 (0.0008) [2023-12-26 15:57:55,632][105692] Updated weights for policy 0, policy_version 82433 (0.0005) [2023-12-26 15:57:55,649][105620] Updated weights for policy 1, policy_version 82750 (0.0005) [2023-12-26 15:57:55,680][105692] Updated weights for policy 0, policy_version 82443 (0.0010) [2023-12-26 15:57:55,698][105620] Updated weights for policy 1, policy_version 82760 (0.0006) [2023-12-26 15:57:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 42303488. Throughput: 0: 9741.7, 1: 9759.5. Samples: 42310696. Policy #0 lag: (min: 1.0, avg: 15.6, max: 33.0) [2023-12-26 15:57:56,062][104569] Avg episode reward: [(0, '8017.648'), (1, '6546.170')] [2023-12-26 15:57:56,311][105692] Updated weights for policy 0, policy_version 82453 (0.0008) [2023-12-26 15:57:56,360][105692] Updated weights for policy 0, policy_version 82463 (0.0005) [2023-12-26 15:57:56,413][105692] Updated weights for policy 0, policy_version 82473 (0.0006) [2023-12-26 15:57:56,419][105620] Updated weights for policy 1, policy_version 82770 (0.0005) [2023-12-26 15:57:56,476][105620] Updated weights for policy 1, policy_version 82780 (0.0007) [2023-12-26 15:57:56,542][105620] Updated weights for policy 1, policy_version 82790 (0.0008) [2023-12-26 15:57:56,599][105620] Updated weights for policy 1, policy_version 82800 (0.0008) [2023-12-26 15:57:56,986][105692] Updated weights for policy 0, policy_version 82483 (0.0009) [2023-12-26 15:57:57,040][105692] Updated weights for policy 0, policy_version 82493 (0.0008) [2023-12-26 15:57:57,097][105692] Updated weights for policy 0, policy_version 82503 (0.0010) [2023-12-26 15:57:57,389][105620] Updated weights for policy 1, policy_version 82810 (0.0009) [2023-12-26 15:57:57,440][105620] Updated weights for policy 1, policy_version 82821 (0.0009) [2023-12-26 15:57:57,495][105620] Updated weights for policy 1, policy_version 82831 (0.0009) [2023-12-26 15:57:57,748][105692] Updated weights for policy 0, policy_version 82513 (0.0010) [2023-12-26 15:57:57,798][105692] Updated weights for policy 0, policy_version 82523 (0.0009) [2023-12-26 15:57:57,848][105692] Updated weights for policy 0, policy_version 82533 (0.0009) [2023-12-26 15:57:57,894][105692] Updated weights for policy 0, policy_version 82543 (0.0009) [2023-12-26 15:57:58,242][105620] Updated weights for policy 1, policy_version 82841 (0.0008) [2023-12-26 15:57:58,304][105620] Updated weights for policy 1, policy_version 82851 (0.0008) [2023-12-26 15:57:58,380][105620] Updated weights for policy 1, policy_version 82861 (0.0008) [2023-12-26 15:57:58,748][105692] Updated weights for policy 0, policy_version 82553 (0.0007) [2023-12-26 15:57:58,811][105692] Updated weights for policy 0, policy_version 82563 (0.0008) [2023-12-26 15:57:58,877][105692] Updated weights for policy 0, policy_version 82573 (0.0008) [2023-12-26 15:57:59,152][105620] Updated weights for policy 1, policy_version 82871 (0.0006) [2023-12-26 15:57:59,205][105620] Updated weights for policy 1, policy_version 82881 (0.0005) [2023-12-26 15:57:59,272][105620] Updated weights for policy 1, policy_version 82891 (0.0008) [2023-12-26 15:57:59,646][105692] Updated weights for policy 0, policy_version 82583 (0.0008) [2023-12-26 15:57:59,711][105692] Updated weights for policy 0, policy_version 82593 (0.0007) [2023-12-26 15:57:59,776][105692] Updated weights for policy 0, policy_version 82603 (0.0010) [2023-12-26 15:57:59,921][105620] Updated weights for policy 1, policy_version 82901 (0.0008) [2023-12-26 15:57:59,988][105620] Updated weights for policy 1, policy_version 82911 (0.0006) [2023-12-26 15:58:00,048][105620] Updated weights for policy 1, policy_version 82921 (0.0007) [2023-12-26 15:58:00,617][105620] Updated weights for policy 1, policy_version 82931 (0.0008) [2023-12-26 15:58:00,621][105692] Updated weights for policy 0, policy_version 82613 (0.0008) [2023-12-26 15:58:00,670][105692] Updated weights for policy 0, policy_version 82623 (0.0007) [2023-12-26 15:58:00,682][105620] Updated weights for policy 1, policy_version 82941 (0.0008) [2023-12-26 15:58:00,724][105692] Updated weights for policy 0, policy_version 82633 (0.0008) [2023-12-26 15:58:00,749][105620] Updated weights for policy 1, policy_version 82951 (0.0008) [2023-12-26 15:58:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 42401792. Throughput: 0: 9835.1, 1: 9756.9. Samples: 42368688. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 15:58:01,062][104569] Avg episode reward: [(0, '8209.320'), (1, '7442.528')] [2023-12-26 15:58:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000082640_21159936.pth... [2023-12-26 15:58:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000082960_21241856.pth... [2023-12-26 15:58:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000081488_20865024.pth [2023-12-26 15:58:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000081808_20946944.pth [2023-12-26 15:58:01,387][105692] Updated weights for policy 0, policy_version 82643 (0.0007) [2023-12-26 15:58:01,397][105620] Updated weights for policy 1, policy_version 82961 (0.0007) [2023-12-26 15:58:01,452][105692] Updated weights for policy 0, policy_version 82653 (0.0006) [2023-12-26 15:58:01,460][105620] Updated weights for policy 1, policy_version 82971 (0.0006) [2023-12-26 15:58:01,520][105692] Updated weights for policy 0, policy_version 82663 (0.0005) [2023-12-26 15:58:01,528][105620] Updated weights for policy 1, policy_version 82981 (0.0005) [2023-12-26 15:58:01,584][105620] Updated weights for policy 1, policy_version 82991 (0.0005) [2023-12-26 15:58:02,186][105620] Updated weights for policy 1, policy_version 83001 (0.0009) [2023-12-26 15:58:02,236][105692] Updated weights for policy 0, policy_version 82673 (0.0006) [2023-12-26 15:58:02,239][105620] Updated weights for policy 1, policy_version 83011 (0.0008) [2023-12-26 15:58:02,294][105692] Updated weights for policy 0, policy_version 82683 (0.0009) [2023-12-26 15:58:02,300][105620] Updated weights for policy 1, policy_version 83021 (0.0008) [2023-12-26 15:58:02,358][105692] Updated weights for policy 0, policy_version 82693 (0.0007) [2023-12-26 15:58:02,425][105692] Updated weights for policy 0, policy_version 82703 (0.0009) [2023-12-26 15:58:02,932][105620] Updated weights for policy 1, policy_version 83031 (0.0005) [2023-12-26 15:58:02,992][105620] Updated weights for policy 1, policy_version 83041 (0.0006) [2023-12-26 15:58:03,046][105620] Updated weights for policy 1, policy_version 83051 (0.0006) [2023-12-26 15:58:03,216][105692] Updated weights for policy 0, policy_version 82713 (0.0009) [2023-12-26 15:58:03,274][105692] Updated weights for policy 0, policy_version 82723 (0.0009) [2023-12-26 15:58:03,328][105692] Updated weights for policy 0, policy_version 82733 (0.0010) [2023-12-26 15:58:03,672][105620] Updated weights for policy 1, policy_version 83061 (0.0007) [2023-12-26 15:58:03,716][105620] Updated weights for policy 1, policy_version 83071 (0.0006) [2023-12-26 15:58:03,770][105620] Updated weights for policy 1, policy_version 83081 (0.0005) [2023-12-26 15:58:04,096][105692] Updated weights for policy 0, policy_version 82743 (0.0009) [2023-12-26 15:58:04,159][105692] Updated weights for policy 0, policy_version 82753 (0.0007) [2023-12-26 15:58:04,223][105692] Updated weights for policy 0, policy_version 82763 (0.0008) [2023-12-26 15:58:04,496][105620] Updated weights for policy 1, policy_version 83091 (0.0007) [2023-12-26 15:58:04,547][105620] Updated weights for policy 1, policy_version 83101 (0.0009) [2023-12-26 15:58:04,611][105620] Updated weights for policy 1, policy_version 83111 (0.0008) [2023-12-26 15:58:05,004][105692] Updated weights for policy 0, policy_version 82773 (0.0008) [2023-12-26 15:58:05,055][105692] Updated weights for policy 0, policy_version 82783 (0.0008) [2023-12-26 15:58:05,109][105692] Updated weights for policy 0, policy_version 82793 (0.0008) [2023-12-26 15:58:05,172][105620] Updated weights for policy 1, policy_version 83121 (0.0007) [2023-12-26 15:58:05,229][105620] Updated weights for policy 1, policy_version 83131 (0.0005) [2023-12-26 15:58:05,288][105620] Updated weights for policy 1, policy_version 83141 (0.0005) [2023-12-26 15:58:05,349][105620] Updated weights for policy 1, policy_version 83151 (0.0009) [2023-12-26 15:58:05,917][105692] Updated weights for policy 0, policy_version 82804 (0.0009) [2023-12-26 15:58:05,970][105692] Updated weights for policy 0, policy_version 82814 (0.0009) [2023-12-26 15:58:05,992][105620] Updated weights for policy 1, policy_version 83161 (0.0006) [2023-12-26 15:58:06,022][105692] Updated weights for policy 0, policy_version 82824 (0.0009) [2023-12-26 15:58:06,053][105620] Updated weights for policy 1, policy_version 83171 (0.0009) [2023-12-26 15:58:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 42491904. Throughput: 0: 9719.2, 1: 9779.7. Samples: 42487568. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 15:58:06,063][104569] Avg episode reward: [(0, '8560.529'), (1, '9061.637')] [2023-12-26 15:58:06,109][105620] Updated weights for policy 1, policy_version 83181 (0.0010) [2023-12-26 15:58:06,698][105620] Updated weights for policy 1, policy_version 83191 (0.0010) [2023-12-26 15:58:06,752][105620] Updated weights for policy 1, policy_version 83201 (0.0010) [2023-12-26 15:58:06,804][105620] Updated weights for policy 1, policy_version 83211 (0.0005) [2023-12-26 15:58:06,882][105692] Updated weights for policy 0, policy_version 82834 (0.0006) [2023-12-26 15:58:06,930][105692] Updated weights for policy 0, policy_version 82844 (0.0006) [2023-12-26 15:58:06,975][105692] Updated weights for policy 0, policy_version 82854 (0.0005) [2023-12-26 15:58:07,028][105692] Updated weights for policy 0, policy_version 82864 (0.0008) [2023-12-26 15:58:07,504][105620] Updated weights for policy 1, policy_version 83221 (0.0008) [2023-12-26 15:58:07,567][105620] Updated weights for policy 1, policy_version 83231 (0.0009) [2023-12-26 15:58:07,614][105620] Updated weights for policy 1, policy_version 83241 (0.0009) [2023-12-26 15:58:07,810][105692] Updated weights for policy 0, policy_version 82874 (0.0009) [2023-12-26 15:58:07,864][105692] Updated weights for policy 0, policy_version 82884 (0.0009) [2023-12-26 15:58:07,917][105692] Updated weights for policy 0, policy_version 82894 (0.0009) [2023-12-26 15:58:08,373][105620] Updated weights for policy 1, policy_version 83251 (0.0009) [2023-12-26 15:58:08,432][105620] Updated weights for policy 1, policy_version 83261 (0.0010) [2023-12-26 15:58:08,495][105620] Updated weights for policy 1, policy_version 83271 (0.0009) [2023-12-26 15:58:08,626][105692] Updated weights for policy 0, policy_version 82904 (0.0008) [2023-12-26 15:58:08,689][105692] Updated weights for policy 0, policy_version 82914 (0.0010) [2023-12-26 15:58:08,749][105692] Updated weights for policy 0, policy_version 82924 (0.0010) [2023-12-26 15:58:09,152][105620] Updated weights for policy 1, policy_version 83281 (0.0009) [2023-12-26 15:58:09,203][105620] Updated weights for policy 1, policy_version 83291 (0.0006) [2023-12-26 15:58:09,269][105620] Updated weights for policy 1, policy_version 83301 (0.0009) [2023-12-26 15:58:09,326][105620] Updated weights for policy 1, policy_version 83311 (0.0009) [2023-12-26 15:58:09,585][105692] Updated weights for policy 0, policy_version 82935 (0.0010) [2023-12-26 15:58:09,642][105692] Updated weights for policy 0, policy_version 82945 (0.0008) [2023-12-26 15:58:09,709][105692] Updated weights for policy 0, policy_version 82955 (0.0010) [2023-12-26 15:58:10,126][105620] Updated weights for policy 1, policy_version 83321 (0.0009) [2023-12-26 15:58:10,218][105620] Updated weights for policy 1, policy_version 83331 (0.0009) [2023-12-26 15:58:10,280][105620] Updated weights for policy 1, policy_version 83341 (0.0009) [2023-12-26 15:58:10,489][105692] Updated weights for policy 0, policy_version 82965 (0.0009) [2023-12-26 15:58:10,547][105692] Updated weights for policy 0, policy_version 82975 (0.0009) [2023-12-26 15:58:10,602][105692] Updated weights for policy 0, policy_version 82985 (0.0008) [2023-12-26 15:58:11,007][105620] Updated weights for policy 1, policy_version 83351 (0.0009) [2023-12-26 15:58:11,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 42590208. Throughput: 0: 9632.7, 1: 9861.8. Samples: 42601500. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 15:58:11,063][104569] Avg episode reward: [(0, '7665.777'), (1, '8937.595')] [2023-12-26 15:58:11,075][105620] Updated weights for policy 1, policy_version 83361 (0.0010) [2023-12-26 15:58:11,139][105620] Updated weights for policy 1, policy_version 83371 (0.0009) [2023-12-26 15:58:11,362][105692] Updated weights for policy 0, policy_version 82995 (0.0009) [2023-12-26 15:58:11,427][105692] Updated weights for policy 0, policy_version 83005 (0.0009) [2023-12-26 15:58:11,480][105692] Updated weights for policy 0, policy_version 83015 (0.0010) [2023-12-26 15:58:11,900][105620] Updated weights for policy 1, policy_version 83381 (0.0009) [2023-12-26 15:58:11,957][105620] Updated weights for policy 1, policy_version 83391 (0.0009) [2023-12-26 15:58:12,015][105620] Updated weights for policy 1, policy_version 83401 (0.0009) [2023-12-26 15:58:12,203][105692] Updated weights for policy 0, policy_version 83025 (0.0009) [2023-12-26 15:58:12,262][105692] Updated weights for policy 0, policy_version 83035 (0.0006) [2023-12-26 15:58:12,323][105692] Updated weights for policy 0, policy_version 83045 (0.0008) [2023-12-26 15:58:12,396][105692] Updated weights for policy 0, policy_version 83055 (0.0007) [2023-12-26 15:58:12,750][105620] Updated weights for policy 1, policy_version 83411 (0.0008) [2023-12-26 15:58:12,815][105620] Updated weights for policy 1, policy_version 83421 (0.0007) [2023-12-26 15:58:12,866][105620] Updated weights for policy 1, policy_version 83431 (0.0008) [2023-12-26 15:58:13,182][105692] Updated weights for policy 0, policy_version 83065 (0.0009) [2023-12-26 15:58:13,247][105692] Updated weights for policy 0, policy_version 83075 (0.0009) [2023-12-26 15:58:13,297][105692] Updated weights for policy 0, policy_version 83085 (0.0008) [2023-12-26 15:58:13,549][105620] Updated weights for policy 1, policy_version 83441 (0.0009) [2023-12-26 15:58:13,605][105620] Updated weights for policy 1, policy_version 83451 (0.0011) [2023-12-26 15:58:13,662][105620] Updated weights for policy 1, policy_version 83461 (0.0010) [2023-12-26 15:58:13,730][105620] Updated weights for policy 1, policy_version 83471 (0.0009) [2023-12-26 15:58:14,072][105692] Updated weights for policy 0, policy_version 83095 (0.0009) [2023-12-26 15:58:14,119][105692] Updated weights for policy 0, policy_version 83105 (0.0009) [2023-12-26 15:58:14,171][105692] Updated weights for policy 0, policy_version 83115 (0.0009) [2023-12-26 15:58:14,477][105620] Updated weights for policy 1, policy_version 83481 (0.0006) [2023-12-26 15:58:14,525][105620] Updated weights for policy 1, policy_version 83491 (0.0005) [2023-12-26 15:58:14,574][105620] Updated weights for policy 1, policy_version 83501 (0.0005) [2023-12-26 15:58:14,955][105692] Updated weights for policy 0, policy_version 83125 (0.0009) [2023-12-26 15:58:15,017][105692] Updated weights for policy 0, policy_version 83135 (0.0009) [2023-12-26 15:58:15,080][105692] Updated weights for policy 0, policy_version 83145 (0.0009) [2023-12-26 15:58:15,319][105620] Updated weights for policy 1, policy_version 83511 (0.0009) [2023-12-26 15:58:15,386][105620] Updated weights for policy 1, policy_version 83521 (0.0009) [2023-12-26 15:58:15,448][105620] Updated weights for policy 1, policy_version 83531 (0.0009) [2023-12-26 15:58:15,840][105692] Updated weights for policy 0, policy_version 83155 (0.0009) [2023-12-26 15:58:15,887][105692] Updated weights for policy 0, policy_version 83165 (0.0009) [2023-12-26 15:58:15,934][105692] Updated weights for policy 0, policy_version 83175 (0.0008) [2023-12-26 15:58:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 42688512. Throughput: 0: 9601.6, 1: 9849.5. Samples: 42657124. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 15:58:16,062][104569] Avg episode reward: [(0, '6794.706'), (1, '8937.524')] [2023-12-26 15:58:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000083536_21389312.pth... [2023-12-26 15:58:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000083184_21299200.pth... [2023-12-26 15:58:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000082384_21094400.pth [2023-12-26 15:58:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000082096_21020672.pth [2023-12-26 15:58:16,152][105620] Updated weights for policy 1, policy_version 83541 (0.0007) [2023-12-26 15:58:16,210][105620] Updated weights for policy 1, policy_version 83551 (0.0005) [2023-12-26 15:58:16,270][105620] Updated weights for policy 1, policy_version 83561 (0.0008) [2023-12-26 15:58:16,771][105692] Updated weights for policy 0, policy_version 83185 (0.0009) [2023-12-26 15:58:16,828][105692] Updated weights for policy 0, policy_version 83195 (0.0009) [2023-12-26 15:58:16,884][105692] Updated weights for policy 0, policy_version 83205 (0.0009) [2023-12-26 15:58:16,912][105620] Updated weights for policy 1, policy_version 83571 (0.0008) [2023-12-26 15:58:16,945][105692] Updated weights for policy 0, policy_version 83215 (0.0008) [2023-12-26 15:58:16,975][105620] Updated weights for policy 1, policy_version 83581 (0.0008) [2023-12-26 15:58:17,037][105620] Updated weights for policy 1, policy_version 83591 (0.0008) [2023-12-26 15:58:17,700][105692] Updated weights for policy 0, policy_version 83225 (0.0008) [2023-12-26 15:58:17,740][105620] Updated weights for policy 1, policy_version 83601 (0.0009) [2023-12-26 15:58:17,750][105692] Updated weights for policy 0, policy_version 83235 (0.0010) [2023-12-26 15:58:17,791][105620] Updated weights for policy 1, policy_version 83611 (0.0005) [2023-12-26 15:58:17,806][105692] Updated weights for policy 0, policy_version 83245 (0.0009) [2023-12-26 15:58:17,837][105620] Updated weights for policy 1, policy_version 83621 (0.0005) [2023-12-26 15:58:17,894][105620] Updated weights for policy 1, policy_version 83631 (0.0005) [2023-12-26 15:58:18,534][105620] Updated weights for policy 1, policy_version 83641 (0.0007) [2023-12-26 15:58:18,540][105692] Updated weights for policy 0, policy_version 83255 (0.0007) [2023-12-26 15:58:18,590][105692] Updated weights for policy 0, policy_version 83265 (0.0006) [2023-12-26 15:58:18,596][105620] Updated weights for policy 1, policy_version 83651 (0.0008) [2023-12-26 15:58:18,636][105692] Updated weights for policy 0, policy_version 83275 (0.0006) [2023-12-26 15:58:18,656][105620] Updated weights for policy 1, policy_version 83661 (0.0008) [2023-12-26 15:58:19,419][105620] Updated weights for policy 1, policy_version 83671 (0.0008) [2023-12-26 15:58:19,425][105692] Updated weights for policy 0, policy_version 83285 (0.0009) [2023-12-26 15:58:19,487][105620] Updated weights for policy 1, policy_version 83681 (0.0007) [2023-12-26 15:58:19,494][105692] Updated weights for policy 0, policy_version 83295 (0.0008) [2023-12-26 15:58:19,547][105620] Updated weights for policy 1, policy_version 83691 (0.0007) [2023-12-26 15:58:19,556][105692] Updated weights for policy 0, policy_version 83305 (0.0008) [2023-12-26 15:58:20,252][105620] Updated weights for policy 1, policy_version 83701 (0.0007) [2023-12-26 15:58:20,312][105620] Updated weights for policy 1, policy_version 83711 (0.0009) [2023-12-26 15:58:20,357][105692] Updated weights for policy 0, policy_version 83315 (0.0008) [2023-12-26 15:58:20,376][105620] Updated weights for policy 1, policy_version 83721 (0.0008) [2023-12-26 15:58:20,418][105692] Updated weights for policy 0, policy_version 83325 (0.0006) [2023-12-26 15:58:20,481][105692] Updated weights for policy 0, policy_version 83335 (0.0006) [2023-12-26 15:58:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 42778624. Throughput: 0: 9462.4, 1: 9928.6. Samples: 42771468. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 15:58:21,063][104569] Avg episode reward: [(0, '7054.572'), (1, '8757.312')] [2023-12-26 15:58:21,132][105620] Updated weights for policy 1, policy_version 83731 (0.0009) [2023-12-26 15:58:21,190][105620] Updated weights for policy 1, policy_version 83741 (0.0009) [2023-12-26 15:58:21,218][105692] Updated weights for policy 0, policy_version 83345 (0.0009) [2023-12-26 15:58:21,249][105620] Updated weights for policy 1, policy_version 83751 (0.0008) [2023-12-26 15:58:21,277][105692] Updated weights for policy 0, policy_version 83355 (0.0009) [2023-12-26 15:58:21,339][105692] Updated weights for policy 0, policy_version 83365 (0.0008) [2023-12-26 15:58:21,407][105692] Updated weights for policy 0, policy_version 83375 (0.0009) [2023-12-26 15:58:22,046][105620] Updated weights for policy 1, policy_version 83761 (0.0009) [2023-12-26 15:58:22,100][105620] Updated weights for policy 1, policy_version 83771 (0.0009) [2023-12-26 15:58:22,139][105692] Updated weights for policy 0, policy_version 83385 (0.0006) [2023-12-26 15:58:22,157][105620] Updated weights for policy 1, policy_version 83781 (0.0008) [2023-12-26 15:58:22,199][105692] Updated weights for policy 0, policy_version 83395 (0.0006) [2023-12-26 15:58:22,205][105620] Updated weights for policy 1, policy_version 83791 (0.0007) [2023-12-26 15:58:22,258][105692] Updated weights for policy 0, policy_version 83405 (0.0008) [2023-12-26 15:58:22,981][105620] Updated weights for policy 1, policy_version 83801 (0.0009) [2023-12-26 15:58:23,030][105692] Updated weights for policy 0, policy_version 83415 (0.0007) [2023-12-26 15:58:23,037][105620] Updated weights for policy 1, policy_version 83811 (0.0007) [2023-12-26 15:58:23,088][105692] Updated weights for policy 0, policy_version 83425 (0.0009) [2023-12-26 15:58:23,099][105620] Updated weights for policy 1, policy_version 83821 (0.0006) [2023-12-26 15:58:23,138][105692] Updated weights for policy 0, policy_version 83435 (0.0007) [2023-12-26 15:58:23,827][105692] Updated weights for policy 0, policy_version 83445 (0.0009) [2023-12-26 15:58:23,872][105692] Updated weights for policy 0, policy_version 83455 (0.0008) [2023-12-26 15:58:23,886][105620] Updated weights for policy 1, policy_version 83831 (0.0007) [2023-12-26 15:58:23,913][105692] Updated weights for policy 0, policy_version 83465 (0.0006) [2023-12-26 15:58:23,944][105620] Updated weights for policy 1, policy_version 83841 (0.0007) [2023-12-26 15:58:23,995][105620] Updated weights for policy 1, policy_version 83851 (0.0009) [2023-12-26 15:58:24,652][105692] Updated weights for policy 0, policy_version 83475 (0.0007) [2023-12-26 15:58:24,698][105692] Updated weights for policy 0, policy_version 83485 (0.0008) [2023-12-26 15:58:24,745][105692] Updated weights for policy 0, policy_version 83495 (0.0008) [2023-12-26 15:58:24,772][105620] Updated weights for policy 1, policy_version 83861 (0.0008) [2023-12-26 15:58:24,831][105620] Updated weights for policy 1, policy_version 83871 (0.0009) [2023-12-26 15:58:24,897][105620] Updated weights for policy 1, policy_version 83881 (0.0010) [2023-12-26 15:58:25,369][105692] Updated weights for policy 0, policy_version 83505 (0.0006) [2023-12-26 15:58:25,420][105692] Updated weights for policy 0, policy_version 83515 (0.0009) [2023-12-26 15:58:25,467][105692] Updated weights for policy 0, policy_version 83525 (0.0009) [2023-12-26 15:58:25,514][105692] Updated weights for policy 0, policy_version 83535 (0.0008) [2023-12-26 15:58:25,705][105620] Updated weights for policy 1, policy_version 83891 (0.0010) [2023-12-26 15:58:25,759][105620] Updated weights for policy 1, policy_version 83901 (0.0009) [2023-12-26 15:58:25,817][105620] Updated weights for policy 1, policy_version 83911 (0.0009) [2023-12-26 15:58:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 42876928. Throughput: 0: 9411.4, 1: 9835.3. Samples: 42882912. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 15:58:26,062][104569] Avg episode reward: [(0, '7667.424'), (1, '8835.890')] [2023-12-26 15:58:26,258][105692] Updated weights for policy 0, policy_version 83545 (0.0006) [2023-12-26 15:58:26,305][105692] Updated weights for policy 0, policy_version 83555 (0.0005) [2023-12-26 15:58:26,355][105692] Updated weights for policy 0, policy_version 83565 (0.0006) [2023-12-26 15:58:26,607][105620] Updated weights for policy 1, policy_version 83921 (0.0009) [2023-12-26 15:58:26,660][105620] Updated weights for policy 1, policy_version 83931 (0.0009) [2023-12-26 15:58:26,717][105620] Updated weights for policy 1, policy_version 83941 (0.0009) [2023-12-26 15:58:26,769][105620] Updated weights for policy 1, policy_version 83951 (0.0009) [2023-12-26 15:58:26,892][105692] Updated weights for policy 0, policy_version 83575 (0.0005) [2023-12-26 15:58:26,943][105692] Updated weights for policy 0, policy_version 83585 (0.0005) [2023-12-26 15:58:26,989][105692] Updated weights for policy 0, policy_version 83595 (0.0005) [2023-12-26 15:58:27,537][105692] Updated weights for policy 0, policy_version 83605 (0.0005) [2023-12-26 15:58:27,580][105692] Updated weights for policy 0, policy_version 83615 (0.0005) [2023-12-26 15:58:27,632][105692] Updated weights for policy 0, policy_version 83625 (0.0005) [2023-12-26 15:58:27,679][105620] Updated weights for policy 1, policy_version 83961 (0.0009) [2023-12-26 15:58:27,736][105620] Updated weights for policy 1, policy_version 83972 (0.0010) [2023-12-26 15:58:27,802][105620] Updated weights for policy 1, policy_version 83982 (0.0010) [2023-12-26 15:58:28,205][105692] Updated weights for policy 0, policy_version 83635 (0.0005) [2023-12-26 15:58:28,256][105692] Updated weights for policy 0, policy_version 83645 (0.0005) [2023-12-26 15:58:28,313][105692] Updated weights for policy 0, policy_version 83655 (0.0006) [2023-12-26 15:58:28,635][105620] Updated weights for policy 1, policy_version 83992 (0.0010) [2023-12-26 15:58:28,698][105620] Updated weights for policy 1, policy_version 84002 (0.0009) [2023-12-26 15:58:28,760][105620] Updated weights for policy 1, policy_version 84012 (0.0010) [2023-12-26 15:58:28,969][105692] Updated weights for policy 0, policy_version 83665 (0.0009) [2023-12-26 15:58:29,037][105692] Updated weights for policy 0, policy_version 83675 (0.0009) [2023-12-26 15:58:29,092][105692] Updated weights for policy 0, policy_version 83685 (0.0009) [2023-12-26 15:58:29,140][105692] Updated weights for policy 0, policy_version 83695 (0.0009) [2023-12-26 15:58:29,512][105620] Updated weights for policy 1, policy_version 84022 (0.0007) [2023-12-26 15:58:29,573][105620] Updated weights for policy 1, policy_version 84032 (0.0007) [2023-12-26 15:58:29,619][105620] Updated weights for policy 1, policy_version 84042 (0.0009) [2023-12-26 15:58:29,965][105692] Updated weights for policy 0, policy_version 83705 (0.0009) [2023-12-26 15:58:30,016][105692] Updated weights for policy 0, policy_version 83715 (0.0009) [2023-12-26 15:58:30,070][105692] Updated weights for policy 0, policy_version 83725 (0.0008) [2023-12-26 15:58:30,305][105620] Updated weights for policy 1, policy_version 84052 (0.0009) [2023-12-26 15:58:30,359][105620] Updated weights for policy 1, policy_version 84062 (0.0009) [2023-12-26 15:58:30,420][105620] Updated weights for policy 1, policy_version 84072 (0.0009) [2023-12-26 15:58:30,813][105692] Updated weights for policy 0, policy_version 83735 (0.0008) [2023-12-26 15:58:30,860][105692] Updated weights for policy 0, policy_version 83745 (0.0008) [2023-12-26 15:58:30,906][105692] Updated weights for policy 0, policy_version 83755 (0.0008) [2023-12-26 15:58:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 42975232. Throughput: 0: 9529.5, 1: 9745.4. Samples: 42943316. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 15:58:31,063][104569] Avg episode reward: [(0, '7849.883'), (1, '9182.271')] [2023-12-26 15:58:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000083760_21446656.pth... [2023-12-26 15:58:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000084080_21528576.pth... [2023-12-26 15:58:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000082640_21159936.pth [2023-12-26 15:58:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000082960_21241856.pth [2023-12-26 15:58:31,186][105620] Updated weights for policy 1, policy_version 84082 (0.0009) [2023-12-26 15:58:31,245][105620] Updated weights for policy 1, policy_version 84092 (0.0009) [2023-12-26 15:58:31,305][105620] Updated weights for policy 1, policy_version 84102 (0.0009) [2023-12-26 15:58:31,369][105620] Updated weights for policy 1, policy_version 84112 (0.0008) [2023-12-26 15:58:31,642][105692] Updated weights for policy 0, policy_version 83765 (0.0009) [2023-12-26 15:58:31,708][105692] Updated weights for policy 0, policy_version 83775 (0.0007) [2023-12-26 15:58:31,766][105692] Updated weights for policy 0, policy_version 83785 (0.0008) [2023-12-26 15:58:32,115][105620] Updated weights for policy 1, policy_version 84122 (0.0009) [2023-12-26 15:58:32,172][105620] Updated weights for policy 1, policy_version 84132 (0.0008) [2023-12-26 15:58:32,232][105620] Updated weights for policy 1, policy_version 84142 (0.0010) [2023-12-26 15:58:32,490][105692] Updated weights for policy 0, policy_version 83795 (0.0009) [2023-12-26 15:58:32,541][105692] Updated weights for policy 0, policy_version 83805 (0.0008) [2023-12-26 15:58:32,605][105692] Updated weights for policy 0, policy_version 83815 (0.0005) [2023-12-26 15:58:33,058][105620] Updated weights for policy 1, policy_version 84152 (0.0009) [2023-12-26 15:58:33,115][105620] Updated weights for policy 1, policy_version 84162 (0.0010) [2023-12-26 15:58:33,170][105620] Updated weights for policy 1, policy_version 84172 (0.0009) [2023-12-26 15:58:33,209][105692] Updated weights for policy 0, policy_version 83825 (0.0008) [2023-12-26 15:58:33,269][105692] Updated weights for policy 0, policy_version 83835 (0.0009) [2023-12-26 15:58:33,329][105692] Updated weights for policy 0, policy_version 83845 (0.0009) [2023-12-26 15:58:33,390][105692] Updated weights for policy 0, policy_version 83855 (0.0009) [2023-12-26 15:58:33,901][105620] Updated weights for policy 1, policy_version 84182 (0.0006) [2023-12-26 15:58:33,944][105620] Updated weights for policy 1, policy_version 84192 (0.0005) [2023-12-26 15:58:33,998][105620] Updated weights for policy 1, policy_version 84202 (0.0008) [2023-12-26 15:58:34,155][105692] Updated weights for policy 0, policy_version 83865 (0.0009) [2023-12-26 15:58:34,214][105692] Updated weights for policy 0, policy_version 83875 (0.0008) [2023-12-26 15:58:34,272][105692] Updated weights for policy 0, policy_version 83885 (0.0009) [2023-12-26 15:58:34,724][105620] Updated weights for policy 1, policy_version 84212 (0.0009) [2023-12-26 15:58:34,778][105620] Updated weights for policy 1, policy_version 84222 (0.0009) [2023-12-26 15:58:34,837][105620] Updated weights for policy 1, policy_version 84232 (0.0008) [2023-12-26 15:58:34,940][105692] Updated weights for policy 0, policy_version 83895 (0.0010) [2023-12-26 15:58:34,987][105692] Updated weights for policy 0, policy_version 83905 (0.0009) [2023-12-26 15:58:35,035][105692] Updated weights for policy 0, policy_version 83916 (0.0007) [2023-12-26 15:58:35,619][105692] Updated weights for policy 0, policy_version 83926 (0.0008) [2023-12-26 15:58:35,643][105620] Updated weights for policy 1, policy_version 84242 (0.0008) [2023-12-26 15:58:35,677][105692] Updated weights for policy 0, policy_version 83936 (0.0010) [2023-12-26 15:58:35,693][105620] Updated weights for policy 1, policy_version 84252 (0.0010) [2023-12-26 15:58:35,744][105692] Updated weights for policy 0, policy_version 83946 (0.0010) [2023-12-26 15:58:35,750][105620] Updated weights for policy 1, policy_version 84262 (0.0009) [2023-12-26 15:58:35,810][105620] Updated weights for policy 1, policy_version 84272 (0.0010) [2023-12-26 15:58:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 43073536. Throughput: 0: 9550.0, 1: 9646.2. Samples: 43057300. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 15:58:36,062][104569] Avg episode reward: [(0, '6963.459'), (1, '9093.194')] [2023-12-26 15:58:36,480][105692] Updated weights for policy 0, policy_version 83956 (0.0011) [2023-12-26 15:58:36,539][105692] Updated weights for policy 0, policy_version 83966 (0.0011) [2023-12-26 15:58:36,581][105620] Updated weights for policy 1, policy_version 84282 (0.0006) [2023-12-26 15:58:36,598][105692] Updated weights for policy 0, policy_version 83976 (0.0011) [2023-12-26 15:58:36,636][105620] Updated weights for policy 1, policy_version 84292 (0.0006) [2023-12-26 15:58:36,697][105620] Updated weights for policy 1, policy_version 84302 (0.0007) [2023-12-26 15:58:37,351][105692] Updated weights for policy 0, policy_version 83986 (0.0010) [2023-12-26 15:58:37,393][105620] Updated weights for policy 1, policy_version 84312 (0.0006) [2023-12-26 15:58:37,417][105692] Updated weights for policy 0, policy_version 83996 (0.0009) [2023-12-26 15:58:37,450][105620] Updated weights for policy 1, policy_version 84322 (0.0005) [2023-12-26 15:58:37,472][105692] Updated weights for policy 0, policy_version 84006 (0.0010) [2023-12-26 15:58:37,506][105620] Updated weights for policy 1, policy_version 84332 (0.0005) [2023-12-26 15:58:37,529][105692] Updated weights for policy 0, policy_version 84016 (0.0011) [2023-12-26 15:58:38,105][105620] Updated weights for policy 1, policy_version 84342 (0.0008) [2023-12-26 15:58:38,118][105692] Updated weights for policy 0, policy_version 84026 (0.0007) [2023-12-26 15:58:38,154][105620] Updated weights for policy 1, policy_version 84352 (0.0010) [2023-12-26 15:58:38,167][105692] Updated weights for policy 0, policy_version 84036 (0.0005) [2023-12-26 15:58:38,212][105620] Updated weights for policy 1, policy_version 84362 (0.0011) [2023-12-26 15:58:38,229][105692] Updated weights for policy 0, policy_version 84046 (0.0005) [2023-12-26 15:58:38,873][105692] Updated weights for policy 0, policy_version 84056 (0.0010) [2023-12-26 15:58:38,933][105692] Updated weights for policy 0, policy_version 84066 (0.0010) [2023-12-26 15:58:38,994][105620] Updated weights for policy 1, policy_version 84372 (0.0008) [2023-12-26 15:58:38,996][105692] Updated weights for policy 0, policy_version 84076 (0.0011) [2023-12-26 15:58:39,055][105620] Updated weights for policy 1, policy_version 84382 (0.0007) [2023-12-26 15:58:39,110][105620] Updated weights for policy 1, policy_version 84392 (0.0010) [2023-12-26 15:58:39,753][105692] Updated weights for policy 0, policy_version 84086 (0.0011) [2023-12-26 15:58:39,781][105620] Updated weights for policy 1, policy_version 84402 (0.0007) [2023-12-26 15:58:39,812][105692] Updated weights for policy 0, policy_version 84096 (0.0010) [2023-12-26 15:58:39,846][105620] Updated weights for policy 1, policy_version 84412 (0.0009) [2023-12-26 15:58:39,880][105692] Updated weights for policy 0, policy_version 84106 (0.0009) [2023-12-26 15:58:39,912][105620] Updated weights for policy 1, policy_version 84422 (0.0007) [2023-12-26 15:58:39,977][105620] Updated weights for policy 1, policy_version 84432 (0.0008) [2023-12-26 15:58:40,594][105692] Updated weights for policy 0, policy_version 84116 (0.0011) [2023-12-26 15:58:40,610][105620] Updated weights for policy 1, policy_version 84442 (0.0011) [2023-12-26 15:58:40,646][105692] Updated weights for policy 0, policy_version 84126 (0.0011) [2023-12-26 15:58:40,665][105620] Updated weights for policy 1, policy_version 84452 (0.0010) [2023-12-26 15:58:40,701][105692] Updated weights for policy 0, policy_version 84136 (0.0010) [2023-12-26 15:58:40,722][105620] Updated weights for policy 1, policy_version 84462 (0.0007) [2023-12-26 15:58:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 43171840. Throughput: 0: 9644.0, 1: 9650.0. Samples: 43178924. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 15:58:41,063][104569] Avg episode reward: [(0, '6879.708'), (1, '9090.654')] [2023-12-26 15:58:41,408][105692] Updated weights for policy 0, policy_version 84146 (0.0010) [2023-12-26 15:58:41,470][105692] Updated weights for policy 0, policy_version 84156 (0.0008) [2023-12-26 15:58:41,473][105620] Updated weights for policy 1, policy_version 84472 (0.0008) [2023-12-26 15:58:41,530][105692] Updated weights for policy 0, policy_version 84166 (0.0008) [2023-12-26 15:58:41,531][105620] Updated weights for policy 1, policy_version 84482 (0.0008) [2023-12-26 15:58:41,587][105692] Updated weights for policy 0, policy_version 84176 (0.0007) [2023-12-26 15:58:41,589][105620] Updated weights for policy 1, policy_version 84492 (0.0007) [2023-12-26 15:58:42,265][105692] Updated weights for policy 0, policy_version 84186 (0.0008) [2023-12-26 15:58:42,328][105692] Updated weights for policy 0, policy_version 84196 (0.0008) [2023-12-26 15:58:42,378][105620] Updated weights for policy 1, policy_version 84502 (0.0010) [2023-12-26 15:58:42,384][105692] Updated weights for policy 0, policy_version 84206 (0.0010) [2023-12-26 15:58:42,442][105620] Updated weights for policy 1, policy_version 84512 (0.0010) [2023-12-26 15:58:42,505][105620] Updated weights for policy 1, policy_version 84522 (0.0009) [2023-12-26 15:58:43,113][105692] Updated weights for policy 0, policy_version 84216 (0.0011) [2023-12-26 15:58:43,156][105620] Updated weights for policy 1, policy_version 84532 (0.0009) [2023-12-26 15:58:43,161][105692] Updated weights for policy 0, policy_version 84226 (0.0010) [2023-12-26 15:58:43,210][105692] Updated weights for policy 0, policy_version 84236 (0.0010) [2023-12-26 15:58:43,210][105620] Updated weights for policy 1, policy_version 84542 (0.0006) [2023-12-26 15:58:43,261][105620] Updated weights for policy 1, policy_version 84552 (0.0005) [2023-12-26 15:58:43,811][105620] Updated weights for policy 1, policy_version 84562 (0.0007) [2023-12-26 15:58:43,821][105692] Updated weights for policy 0, policy_version 84246 (0.0007) [2023-12-26 15:58:43,877][105620] Updated weights for policy 1, policy_version 84572 (0.0011) [2023-12-26 15:58:43,879][105692] Updated weights for policy 0, policy_version 84256 (0.0007) [2023-12-26 15:58:43,933][105620] Updated weights for policy 1, policy_version 84582 (0.0010) [2023-12-26 15:58:43,936][105692] Updated weights for policy 0, policy_version 84266 (0.0011) [2023-12-26 15:58:43,994][105620] Updated weights for policy 1, policy_version 84592 (0.0009) [2023-12-26 15:58:44,668][105692] Updated weights for policy 0, policy_version 84276 (0.0011) [2023-12-26 15:58:44,679][105620] Updated weights for policy 1, policy_version 84602 (0.0008) [2023-12-26 15:58:44,723][105692] Updated weights for policy 0, policy_version 84286 (0.0010) [2023-12-26 15:58:44,746][105620] Updated weights for policy 1, policy_version 84612 (0.0008) [2023-12-26 15:58:44,783][105692] Updated weights for policy 0, policy_version 84296 (0.0010) [2023-12-26 15:58:44,813][105620] Updated weights for policy 1, policy_version 84622 (0.0010) [2023-12-26 15:58:45,415][105620] Updated weights for policy 1, policy_version 84632 (0.0011) [2023-12-26 15:58:45,445][105692] Updated weights for policy 0, policy_version 84306 (0.0009) [2023-12-26 15:58:45,471][105620] Updated weights for policy 1, policy_version 84642 (0.0011) [2023-12-26 15:58:45,509][105692] Updated weights for policy 0, policy_version 84316 (0.0006) [2023-12-26 15:58:45,534][105620] Updated weights for policy 1, policy_version 84652 (0.0011) [2023-12-26 15:58:45,567][105692] Updated weights for policy 0, policy_version 84326 (0.0007) [2023-12-26 15:58:45,620][105692] Updated weights for policy 0, policy_version 84336 (0.0007) [2023-12-26 15:58:46,062][104569] Fps is (10 sec: 19659.9, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 43270144. Throughput: 0: 9605.5, 1: 9729.4. Samples: 43238768. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 15:58:46,063][104569] Avg episode reward: [(0, '7316.296'), (1, '8197.498')] [2023-12-26 15:58:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000084336_21594112.pth... [2023-12-26 15:58:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000084656_21676032.pth... [2023-12-26 15:58:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000083184_21299200.pth [2023-12-26 15:58:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000083536_21389312.pth [2023-12-26 15:58:46,175][105620] Updated weights for policy 1, policy_version 84662 (0.0007) [2023-12-26 15:58:46,226][105620] Updated weights for policy 1, policy_version 84672 (0.0005) [2023-12-26 15:58:46,290][105620] Updated weights for policy 1, policy_version 84682 (0.0005) [2023-12-26 15:58:46,325][105692] Updated weights for policy 0, policy_version 84346 (0.0007) [2023-12-26 15:58:46,379][105692] Updated weights for policy 0, policy_version 84356 (0.0010) [2023-12-26 15:58:46,433][105692] Updated weights for policy 0, policy_version 84366 (0.0010) [2023-12-26 15:58:46,827][105620] Updated weights for policy 1, policy_version 84692 (0.0005) [2023-12-26 15:58:46,885][105620] Updated weights for policy 1, policy_version 84702 (0.0010) [2023-12-26 15:58:46,934][105620] Updated weights for policy 1, policy_version 84712 (0.0010) [2023-12-26 15:58:47,217][105692] Updated weights for policy 0, policy_version 84376 (0.0010) [2023-12-26 15:58:47,270][105692] Updated weights for policy 0, policy_version 84386 (0.0010) [2023-12-26 15:58:47,318][105692] Updated weights for policy 0, policy_version 84396 (0.0010) [2023-12-26 15:58:47,648][105620] Updated weights for policy 1, policy_version 84722 (0.0010) [2023-12-26 15:58:47,710][105620] Updated weights for policy 1, policy_version 84732 (0.0010) [2023-12-26 15:58:47,768][105620] Updated weights for policy 1, policy_version 84742 (0.0010) [2023-12-26 15:58:47,830][105620] Updated weights for policy 1, policy_version 84752 (0.0010) [2023-12-26 15:58:47,963][105692] Updated weights for policy 0, policy_version 84406 (0.0010) [2023-12-26 15:58:48,026][105692] Updated weights for policy 0, policy_version 84416 (0.0009) [2023-12-26 15:58:48,080][105692] Updated weights for policy 0, policy_version 84426 (0.0005) [2023-12-26 15:58:48,563][105620] Updated weights for policy 1, policy_version 84762 (0.0008) [2023-12-26 15:58:48,623][105620] Updated weights for policy 1, policy_version 84772 (0.0008) [2023-12-26 15:58:48,668][105620] Updated weights for policy 1, policy_version 84782 (0.0008) [2023-12-26 15:58:48,723][105692] Updated weights for policy 0, policy_version 84436 (0.0007) [2023-12-26 15:58:48,788][105692] Updated weights for policy 0, policy_version 84446 (0.0010) [2023-12-26 15:58:48,839][105692] Updated weights for policy 0, policy_version 84456 (0.0010) [2023-12-26 15:58:49,365][105620] Updated weights for policy 1, policy_version 84792 (0.0008) [2023-12-26 15:58:49,414][105620] Updated weights for policy 1, policy_version 84802 (0.0008) [2023-12-26 15:58:49,466][105620] Updated weights for policy 1, policy_version 84812 (0.0008) [2023-12-26 15:58:49,576][105692] Updated weights for policy 0, policy_version 84466 (0.0010) [2023-12-26 15:58:49,625][105692] Updated weights for policy 0, policy_version 84476 (0.0009) [2023-12-26 15:58:49,680][105692] Updated weights for policy 0, policy_version 84486 (0.0009) [2023-12-26 15:58:49,731][105692] Updated weights for policy 0, policy_version 84496 (0.0009) [2023-12-26 15:58:50,254][105620] Updated weights for policy 1, policy_version 84822 (0.0007) [2023-12-26 15:58:50,318][105620] Updated weights for policy 1, policy_version 84832 (0.0007) [2023-12-26 15:58:50,379][105620] Updated weights for policy 1, policy_version 84842 (0.0005) [2023-12-26 15:58:50,556][105692] Updated weights for policy 0, policy_version 84506 (0.0007) [2023-12-26 15:58:50,619][105692] Updated weights for policy 0, policy_version 84516 (0.0009) [2023-12-26 15:58:50,671][105692] Updated weights for policy 0, policy_version 84526 (0.0009) [2023-12-26 15:58:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 43368448. Throughput: 0: 9721.6, 1: 9680.8. Samples: 43360676. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 15:58:51,062][104569] Avg episode reward: [(0, '6874.653'), (1, '8214.519')] [2023-12-26 15:58:51,082][105620] Updated weights for policy 1, policy_version 84852 (0.0009) [2023-12-26 15:58:51,144][105620] Updated weights for policy 1, policy_version 84862 (0.0008) [2023-12-26 15:58:51,200][105620] Updated weights for policy 1, policy_version 84872 (0.0008) [2023-12-26 15:58:51,405][105692] Updated weights for policy 0, policy_version 84536 (0.0007) [2023-12-26 15:58:51,463][105692] Updated weights for policy 0, policy_version 84546 (0.0006) [2023-12-26 15:58:51,522][105692] Updated weights for policy 0, policy_version 84556 (0.0006) [2023-12-26 15:58:52,002][105620] Updated weights for policy 1, policy_version 84882 (0.0009) [2023-12-26 15:58:52,057][105620] Updated weights for policy 1, policy_version 84892 (0.0010) [2023-12-26 15:58:52,123][105620] Updated weights for policy 1, policy_version 84902 (0.0009) [2023-12-26 15:58:52,177][105692] Updated weights for policy 0, policy_version 84566 (0.0005) [2023-12-26 15:58:52,193][105620] Updated weights for policy 1, policy_version 84912 (0.0010) [2023-12-26 15:58:52,227][105692] Updated weights for policy 0, policy_version 84576 (0.0005) [2023-12-26 15:58:52,288][105692] Updated weights for policy 0, policy_version 84586 (0.0008) [2023-12-26 15:58:52,880][105692] Updated weights for policy 0, policy_version 84596 (0.0006) [2023-12-26 15:58:52,943][105692] Updated weights for policy 0, policy_version 84606 (0.0007) [2023-12-26 15:58:52,995][105692] Updated weights for policy 0, policy_version 84616 (0.0010) [2023-12-26 15:58:53,049][105620] Updated weights for policy 1, policy_version 84922 (0.0006) [2023-12-26 15:58:53,095][105620] Updated weights for policy 1, policy_version 84932 (0.0006) [2023-12-26 15:58:53,142][105620] Updated weights for policy 1, policy_version 84942 (0.0006) [2023-12-26 15:58:53,612][105692] Updated weights for policy 0, policy_version 84626 (0.0010) [2023-12-26 15:58:53,666][105692] Updated weights for policy 0, policy_version 84636 (0.0009) [2023-12-26 15:58:53,710][105692] Updated weights for policy 0, policy_version 84646 (0.0006) [2023-12-26 15:58:53,752][105692] Updated weights for policy 0, policy_version 84656 (0.0005) [2023-12-26 15:58:53,918][105620] Updated weights for policy 1, policy_version 84952 (0.0009) [2023-12-26 15:58:53,987][105620] Updated weights for policy 1, policy_version 84962 (0.0008) [2023-12-26 15:58:54,053][105620] Updated weights for policy 1, policy_version 84972 (0.0009) [2023-12-26 15:58:54,376][105692] Updated weights for policy 0, policy_version 84666 (0.0005) [2023-12-26 15:58:54,440][105692] Updated weights for policy 0, policy_version 84676 (0.0005) [2023-12-26 15:58:54,503][105692] Updated weights for policy 0, policy_version 84686 (0.0005) [2023-12-26 15:58:54,938][105620] Updated weights for policy 1, policy_version 84982 (0.0010) [2023-12-26 15:58:54,985][105692] Updated weights for policy 0, policy_version 84696 (0.0005) [2023-12-26 15:58:54,999][105620] Updated weights for policy 1, policy_version 84992 (0.0008) [2023-12-26 15:58:55,043][105692] Updated weights for policy 0, policy_version 84706 (0.0005) [2023-12-26 15:58:55,055][105620] Updated weights for policy 1, policy_version 85002 (0.0009) [2023-12-26 15:58:55,101][105692] Updated weights for policy 0, policy_version 84716 (0.0005) [2023-12-26 15:58:55,796][105620] Updated weights for policy 1, policy_version 85012 (0.0007) [2023-12-26 15:58:55,797][105692] Updated weights for policy 0, policy_version 84726 (0.0007) [2023-12-26 15:58:55,844][105620] Updated weights for policy 1, policy_version 85022 (0.0005) [2023-12-26 15:58:55,853][105692] Updated weights for policy 0, policy_version 84736 (0.0008) [2023-12-26 15:58:55,907][105620] Updated weights for policy 1, policy_version 85032 (0.0005) [2023-12-26 15:58:55,914][105692] Updated weights for policy 0, policy_version 84746 (0.0009) [2023-12-26 15:58:56,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 43474944. Throughput: 0: 9956.6, 1: 9521.4. Samples: 43478012. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 15:58:56,063][104569] Avg episode reward: [(0, '7398.354'), (1, '8391.315')] [2023-12-26 15:58:56,602][105620] Updated weights for policy 1, policy_version 85042 (0.0006) [2023-12-26 15:58:56,620][105692] Updated weights for policy 0, policy_version 84756 (0.0009) [2023-12-26 15:58:56,660][105620] Updated weights for policy 1, policy_version 85052 (0.0008) [2023-12-26 15:58:56,671][105692] Updated weights for policy 0, policy_version 84766 (0.0008) [2023-12-26 15:58:56,713][105620] Updated weights for policy 1, policy_version 85062 (0.0005) [2023-12-26 15:58:56,723][105692] Updated weights for policy 0, policy_version 84776 (0.0009) [2023-12-26 15:58:56,774][105620] Updated weights for policy 1, policy_version 85072 (0.0007) [2023-12-26 15:58:57,508][105620] Updated weights for policy 1, policy_version 85082 (0.0009) [2023-12-26 15:58:57,519][105692] Updated weights for policy 0, policy_version 84786 (0.0007) [2023-12-26 15:58:57,558][105620] Updated weights for policy 1, policy_version 85092 (0.0006) [2023-12-26 15:58:57,576][105692] Updated weights for policy 0, policy_version 84796 (0.0007) [2023-12-26 15:58:57,613][105620] Updated weights for policy 1, policy_version 85102 (0.0008) [2023-12-26 15:58:57,623][105692] Updated weights for policy 0, policy_version 84806 (0.0006) [2023-12-26 15:58:57,674][105692] Updated weights for policy 0, policy_version 84816 (0.0008) [2023-12-26 15:58:58,344][105620] Updated weights for policy 1, policy_version 85112 (0.0008) [2023-12-26 15:58:58,408][105620] Updated weights for policy 1, policy_version 85122 (0.0008) [2023-12-26 15:58:58,471][105620] Updated weights for policy 1, policy_version 85132 (0.0008) [2023-12-26 15:58:58,473][105692] Updated weights for policy 0, policy_version 84826 (0.0007) [2023-12-26 15:58:58,531][105692] Updated weights for policy 0, policy_version 84836 (0.0008) [2023-12-26 15:58:58,592][105692] Updated weights for policy 0, policy_version 84846 (0.0009) [2023-12-26 15:58:59,206][105620] Updated weights for policy 1, policy_version 85142 (0.0008) [2023-12-26 15:58:59,276][105620] Updated weights for policy 1, policy_version 85152 (0.0009) [2023-12-26 15:58:59,343][105620] Updated weights for policy 1, policy_version 85162 (0.0008) [2023-12-26 15:58:59,450][105692] Updated weights for policy 0, policy_version 84856 (0.0007) [2023-12-26 15:58:59,505][105692] Updated weights for policy 0, policy_version 84866 (0.0008) [2023-12-26 15:58:59,568][105692] Updated weights for policy 0, policy_version 84876 (0.0010) [2023-12-26 15:58:59,980][105620] Updated weights for policy 1, policy_version 85172 (0.0007) [2023-12-26 15:59:00,046][105620] Updated weights for policy 1, policy_version 85182 (0.0009) [2023-12-26 15:59:00,110][105620] Updated weights for policy 1, policy_version 85192 (0.0011) [2023-12-26 15:59:00,243][105692] Updated weights for policy 0, policy_version 84886 (0.0010) [2023-12-26 15:59:00,296][105692] Updated weights for policy 0, policy_version 84896 (0.0011) [2023-12-26 15:59:00,363][105692] Updated weights for policy 0, policy_version 84906 (0.0011) [2023-12-26 15:59:00,825][105620] Updated weights for policy 1, policy_version 85202 (0.0010) [2023-12-26 15:59:00,877][105620] Updated weights for policy 1, policy_version 85212 (0.0005) [2023-12-26 15:59:00,931][105620] Updated weights for policy 1, policy_version 85222 (0.0007) [2023-12-26 15:59:00,976][105620] Updated weights for policy 1, policy_version 85232 (0.0010) [2023-12-26 15:59:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 43565056. Throughput: 0: 9972.4, 1: 9536.8. Samples: 43535040. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 15:59:01,063][104569] Avg episode reward: [(0, '7837.781'), (1, '7352.081')] [2023-12-26 15:59:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000085232_21823488.pth... [2023-12-26 15:59:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000084912_21741568.pth... [2023-12-26 15:59:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000084080_21528576.pth [2023-12-26 15:59:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000083760_21446656.pth [2023-12-26 15:59:01,116][105692] Updated weights for policy 0, policy_version 84916 (0.0011) [2023-12-26 15:59:01,182][105692] Updated weights for policy 0, policy_version 84926 (0.0008) [2023-12-26 15:59:01,247][105692] Updated weights for policy 0, policy_version 84936 (0.0008) [2023-12-26 15:59:01,641][105620] Updated weights for policy 1, policy_version 85242 (0.0008) [2023-12-26 15:59:01,705][105620] Updated weights for policy 1, policy_version 85252 (0.0010) [2023-12-26 15:59:01,771][105620] Updated weights for policy 1, policy_version 85262 (0.0008) [2023-12-26 15:59:02,078][105692] Updated weights for policy 0, policy_version 84946 (0.0008) [2023-12-26 15:59:02,131][105692] Updated weights for policy 0, policy_version 84956 (0.0008) [2023-12-26 15:59:02,179][105692] Updated weights for policy 0, policy_version 84966 (0.0008) [2023-12-26 15:59:02,238][105692] Updated weights for policy 0, policy_version 84976 (0.0008) [2023-12-26 15:59:02,364][105620] Updated weights for policy 1, policy_version 85272 (0.0008) [2023-12-26 15:59:02,421][105620] Updated weights for policy 1, policy_version 85282 (0.0008) [2023-12-26 15:59:02,468][105620] Updated weights for policy 1, policy_version 85292 (0.0007) [2023-12-26 15:59:03,030][105692] Updated weights for policy 0, policy_version 84986 (0.0010) [2023-12-26 15:59:03,081][105692] Updated weights for policy 0, policy_version 84996 (0.0008) [2023-12-26 15:59:03,100][105620] Updated weights for policy 1, policy_version 85302 (0.0010) [2023-12-26 15:59:03,137][105692] Updated weights for policy 0, policy_version 85006 (0.0006) [2023-12-26 15:59:03,158][105620] Updated weights for policy 1, policy_version 85312 (0.0010) [2023-12-26 15:59:03,218][105620] Updated weights for policy 1, policy_version 85322 (0.0010) [2023-12-26 15:59:03,911][105692] Updated weights for policy 0, policy_version 85016 (0.0008) [2023-12-26 15:59:03,951][105620] Updated weights for policy 1, policy_version 85332 (0.0010) [2023-12-26 15:59:03,972][105692] Updated weights for policy 0, policy_version 85026 (0.0006) [2023-12-26 15:59:04,001][105620] Updated weights for policy 1, policy_version 85342 (0.0008) [2023-12-26 15:59:04,023][105692] Updated weights for policy 0, policy_version 85036 (0.0006) [2023-12-26 15:59:04,051][105620] Updated weights for policy 1, policy_version 85352 (0.0008) [2023-12-26 15:59:04,734][105692] Updated weights for policy 0, policy_version 85046 (0.0008) [2023-12-26 15:59:04,785][105692] Updated weights for policy 0, policy_version 85056 (0.0009) [2023-12-26 15:59:04,817][105620] Updated weights for policy 1, policy_version 85362 (0.0006) [2023-12-26 15:59:04,847][105692] Updated weights for policy 0, policy_version 85066 (0.0008) [2023-12-26 15:59:04,881][105620] Updated weights for policy 1, policy_version 85372 (0.0009) [2023-12-26 15:59:04,935][105620] Updated weights for policy 1, policy_version 85382 (0.0009) [2023-12-26 15:59:04,994][105620] Updated weights for policy 1, policy_version 85392 (0.0009) [2023-12-26 15:59:05,635][105692] Updated weights for policy 0, policy_version 85076 (0.0008) [2023-12-26 15:59:05,681][105620] Updated weights for policy 1, policy_version 85402 (0.0007) [2023-12-26 15:59:05,684][105692] Updated weights for policy 0, policy_version 85086 (0.0008) [2023-12-26 15:59:05,728][105692] Updated weights for policy 0, policy_version 85096 (0.0006) [2023-12-26 15:59:05,734][105620] Updated weights for policy 1, policy_version 85412 (0.0008) [2023-12-26 15:59:05,791][105620] Updated weights for policy 1, policy_version 85422 (0.0007) [2023-12-26 15:59:06,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 43663360. Throughput: 0: 9961.0, 1: 9567.1. Samples: 43650232. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 15:59:06,063][104569] Avg episode reward: [(0, '8376.015'), (1, '6717.851')] [2023-12-26 15:59:06,477][105692] Updated weights for policy 0, policy_version 85106 (0.0006) [2023-12-26 15:59:06,538][105692] Updated weights for policy 0, policy_version 85116 (0.0006) [2023-12-26 15:59:06,569][105620] Updated weights for policy 1, policy_version 85432 (0.0007) [2023-12-26 15:59:06,588][105692] Updated weights for policy 0, policy_version 85126 (0.0006) [2023-12-26 15:59:06,619][105620] Updated weights for policy 1, policy_version 85442 (0.0006) [2023-12-26 15:59:06,647][105692] Updated weights for policy 0, policy_version 85136 (0.0007) [2023-12-26 15:59:06,674][105620] Updated weights for policy 1, policy_version 85452 (0.0008) [2023-12-26 15:59:07,346][105692] Updated weights for policy 0, policy_version 85146 (0.0005) [2023-12-26 15:59:07,394][105692] Updated weights for policy 0, policy_version 85156 (0.0005) [2023-12-26 15:59:07,446][105692] Updated weights for policy 0, policy_version 85166 (0.0007) [2023-12-26 15:59:07,468][105620] Updated weights for policy 1, policy_version 85462 (0.0007) [2023-12-26 15:59:07,519][105620] Updated weights for policy 1, policy_version 85472 (0.0005) [2023-12-26 15:59:07,579][105620] Updated weights for policy 1, policy_version 85482 (0.0008) [2023-12-26 15:59:08,167][105692] Updated weights for policy 0, policy_version 85176 (0.0007) [2023-12-26 15:59:08,222][105692] Updated weights for policy 0, policy_version 85186 (0.0009) [2023-12-26 15:59:08,247][105620] Updated weights for policy 1, policy_version 85492 (0.0008) [2023-12-26 15:59:08,281][105692] Updated weights for policy 0, policy_version 85196 (0.0009) [2023-12-26 15:59:08,310][105620] Updated weights for policy 1, policy_version 85502 (0.0005) [2023-12-26 15:59:08,374][105620] Updated weights for policy 1, policy_version 85512 (0.0007) [2023-12-26 15:59:09,004][105620] Updated weights for policy 1, policy_version 85522 (0.0006) [2023-12-26 15:59:09,037][105692] Updated weights for policy 0, policy_version 85206 (0.0010) [2023-12-26 15:59:09,053][105620] Updated weights for policy 1, policy_version 85532 (0.0005) [2023-12-26 15:59:09,086][105692] Updated weights for policy 0, policy_version 85216 (0.0010) [2023-12-26 15:59:09,103][105620] Updated weights for policy 1, policy_version 85542 (0.0010) [2023-12-26 15:59:09,137][105692] Updated weights for policy 0, policy_version 85226 (0.0010) [2023-12-26 15:59:09,155][105620] Updated weights for policy 1, policy_version 85552 (0.0010) [2023-12-26 15:59:09,898][105620] Updated weights for policy 1, policy_version 85562 (0.0011) [2023-12-26 15:59:09,910][105692] Updated weights for policy 0, policy_version 85236 (0.0011) [2023-12-26 15:59:09,970][105620] Updated weights for policy 1, policy_version 85572 (0.0011) [2023-12-26 15:59:09,978][105692] Updated weights for policy 0, policy_version 85246 (0.0011) [2023-12-26 15:59:10,035][105620] Updated weights for policy 1, policy_version 85582 (0.0010) [2023-12-26 15:59:10,042][105692] Updated weights for policy 0, policy_version 85256 (0.0011) [2023-12-26 15:59:10,792][105692] Updated weights for policy 0, policy_version 85266 (0.0011) [2023-12-26 15:59:10,799][105620] Updated weights for policy 1, policy_version 85592 (0.0008) [2023-12-26 15:59:10,854][105692] Updated weights for policy 0, policy_version 85276 (0.0010) [2023-12-26 15:59:10,861][105620] Updated weights for policy 1, policy_version 85602 (0.0010) [2023-12-26 15:59:10,914][105692] Updated weights for policy 0, policy_version 85286 (0.0011) [2023-12-26 15:59:10,923][105620] Updated weights for policy 1, policy_version 85612 (0.0009) [2023-12-26 15:59:10,980][105692] Updated weights for policy 0, policy_version 85296 (0.0007) [2023-12-26 15:59:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 43761664. Throughput: 0: 9945.3, 1: 9656.1. Samples: 43764976. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 15:59:11,063][104569] Avg episode reward: [(0, '8113.764'), (1, '7772.039')] [2023-12-26 15:59:11,560][105620] Updated weights for policy 1, policy_version 85622 (0.0006) [2023-12-26 15:59:11,629][105620] Updated weights for policy 1, policy_version 85632 (0.0007) [2023-12-26 15:59:11,694][105620] Updated weights for policy 1, policy_version 85642 (0.0008) [2023-12-26 15:59:11,731][105692] Updated weights for policy 0, policy_version 85306 (0.0011) [2023-12-26 15:59:11,789][105692] Updated weights for policy 0, policy_version 85317 (0.0010) [2023-12-26 15:59:11,846][105692] Updated weights for policy 0, policy_version 85327 (0.0008) [2023-12-26 15:59:12,392][105620] Updated weights for policy 1, policy_version 85652 (0.0008) [2023-12-26 15:59:12,440][105620] Updated weights for policy 1, policy_version 85662 (0.0009) [2023-12-26 15:59:12,499][105620] Updated weights for policy 1, policy_version 85672 (0.0008) [2023-12-26 15:59:12,576][105692] Updated weights for policy 0, policy_version 85337 (0.0007) [2023-12-26 15:59:12,644][105692] Updated weights for policy 0, policy_version 85347 (0.0009) [2023-12-26 15:59:12,704][105692] Updated weights for policy 0, policy_version 85357 (0.0008) [2023-12-26 15:59:13,252][105620] Updated weights for policy 1, policy_version 85682 (0.0008) [2023-12-26 15:59:13,308][105620] Updated weights for policy 1, policy_version 85692 (0.0010) [2023-12-26 15:59:13,367][105620] Updated weights for policy 1, policy_version 85702 (0.0010) [2023-12-26 15:59:13,407][105692] Updated weights for policy 0, policy_version 85367 (0.0006) [2023-12-26 15:59:13,422][105620] Updated weights for policy 1, policy_version 85712 (0.0010) [2023-12-26 15:59:13,457][105692] Updated weights for policy 0, policy_version 85377 (0.0005) [2023-12-26 15:59:13,506][105692] Updated weights for policy 0, policy_version 85387 (0.0005) [2023-12-26 15:59:14,045][105692] Updated weights for policy 0, policy_version 85397 (0.0005) [2023-12-26 15:59:14,099][105692] Updated weights for policy 0, policy_version 85407 (0.0005) [2023-12-26 15:59:14,128][105620] Updated weights for policy 1, policy_version 85722 (0.0005) [2023-12-26 15:59:14,154][105692] Updated weights for policy 0, policy_version 85417 (0.0005) [2023-12-26 15:59:14,189][105620] Updated weights for policy 1, policy_version 85732 (0.0006) [2023-12-26 15:59:14,249][105620] Updated weights for policy 1, policy_version 85742 (0.0008) [2023-12-26 15:59:14,759][105692] Updated weights for policy 0, policy_version 85427 (0.0006) [2023-12-26 15:59:14,821][105692] Updated weights for policy 0, policy_version 85437 (0.0008) [2023-12-26 15:59:14,889][105692] Updated weights for policy 0, policy_version 85447 (0.0009) [2023-12-26 15:59:14,901][105620] Updated weights for policy 1, policy_version 85752 (0.0007) [2023-12-26 15:59:14,960][105620] Updated weights for policy 1, policy_version 85762 (0.0007) [2023-12-26 15:59:15,019][105620] Updated weights for policy 1, policy_version 85772 (0.0009) [2023-12-26 15:59:15,573][105692] Updated weights for policy 0, policy_version 85457 (0.0008) [2023-12-26 15:59:15,620][105692] Updated weights for policy 0, policy_version 85467 (0.0008) [2023-12-26 15:59:15,682][105692] Updated weights for policy 0, policy_version 85477 (0.0009) [2023-12-26 15:59:15,736][105692] Updated weights for policy 0, policy_version 85487 (0.0009) [2023-12-26 15:59:15,785][105620] Updated weights for policy 1, policy_version 85782 (0.0009) [2023-12-26 15:59:15,837][105620] Updated weights for policy 1, policy_version 85793 (0.0010) [2023-12-26 15:59:15,887][105620] Updated weights for policy 1, policy_version 85803 (0.0008) [2023-12-26 15:59:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 43859968. Throughput: 0: 9806.4, 1: 9738.2. Samples: 43822820. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 15:59:16,062][104569] Avg episode reward: [(0, '8208.965'), (1, '7848.884')] [2023-12-26 15:59:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000085488_21889024.pth... [2023-12-26 15:59:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000085808_21970944.pth... [2023-12-26 15:59:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000084336_21594112.pth [2023-12-26 15:59:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000084656_21676032.pth [2023-12-26 15:59:16,491][105692] Updated weights for policy 0, policy_version 85497 (0.0009) [2023-12-26 15:59:16,553][105692] Updated weights for policy 0, policy_version 85507 (0.0009) [2023-12-26 15:59:16,611][105692] Updated weights for policy 0, policy_version 85517 (0.0009) [2023-12-26 15:59:16,658][105620] Updated weights for policy 1, policy_version 85813 (0.0007) [2023-12-26 15:59:16,712][105620] Updated weights for policy 1, policy_version 85823 (0.0005) [2023-12-26 15:59:16,773][105620] Updated weights for policy 1, policy_version 85833 (0.0005) [2023-12-26 15:59:17,292][105620] Updated weights for policy 1, policy_version 85843 (0.0006) [2023-12-26 15:59:17,346][105620] Updated weights for policy 1, policy_version 85853 (0.0005) [2023-12-26 15:59:17,404][105620] Updated weights for policy 1, policy_version 85863 (0.0005) [2023-12-26 15:59:17,465][105692] Updated weights for policy 0, policy_version 85527 (0.0009) [2023-12-26 15:59:17,531][105692] Updated weights for policy 0, policy_version 85537 (0.0009) [2023-12-26 15:59:17,579][105692] Updated weights for policy 0, policy_version 85547 (0.0008) [2023-12-26 15:59:17,960][105620] Updated weights for policy 1, policy_version 85873 (0.0006) [2023-12-26 15:59:18,026][105620] Updated weights for policy 1, policy_version 85883 (0.0009) [2023-12-26 15:59:18,084][105620] Updated weights for policy 1, policy_version 85893 (0.0009) [2023-12-26 15:59:18,144][105620] Updated weights for policy 1, policy_version 85903 (0.0009) [2023-12-26 15:59:18,432][105692] Updated weights for policy 0, policy_version 85557 (0.0007) [2023-12-26 15:59:18,483][105692] Updated weights for policy 0, policy_version 85567 (0.0008) [2023-12-26 15:59:18,532][105692] Updated weights for policy 0, policy_version 85577 (0.0006) [2023-12-26 15:59:18,871][105620] Updated weights for policy 1, policy_version 85913 (0.0009) [2023-12-26 15:59:18,925][105620] Updated weights for policy 1, policy_version 85924 (0.0010) [2023-12-26 15:59:18,982][105620] Updated weights for policy 1, policy_version 85934 (0.0009) [2023-12-26 15:59:19,145][105692] Updated weights for policy 0, policy_version 85587 (0.0006) [2023-12-26 15:59:19,200][105692] Updated weights for policy 0, policy_version 85597 (0.0005) [2023-12-26 15:59:19,260][105692] Updated weights for policy 0, policy_version 85607 (0.0007) [2023-12-26 15:59:19,770][105620] Updated weights for policy 1, policy_version 85944 (0.0009) [2023-12-26 15:59:19,837][105620] Updated weights for policy 1, policy_version 85954 (0.0009) [2023-12-26 15:59:19,899][105620] Updated weights for policy 1, policy_version 85964 (0.0008) [2023-12-26 15:59:19,989][105692] Updated weights for policy 0, policy_version 85617 (0.0006) [2023-12-26 15:59:20,048][105692] Updated weights for policy 0, policy_version 85627 (0.0009) [2023-12-26 15:59:20,114][105692] Updated weights for policy 0, policy_version 85637 (0.0008) [2023-12-26 15:59:20,178][105692] Updated weights for policy 0, policy_version 85647 (0.0006) [2023-12-26 15:59:20,693][105620] Updated weights for policy 1, policy_version 85974 (0.0010) [2023-12-26 15:59:20,754][105620] Updated weights for policy 1, policy_version 85984 (0.0011) [2023-12-26 15:59:20,818][105620] Updated weights for policy 1, policy_version 85994 (0.0011) [2023-12-26 15:59:20,849][105692] Updated weights for policy 0, policy_version 85657 (0.0006) [2023-12-26 15:59:20,918][105692] Updated weights for policy 0, policy_version 85667 (0.0007) [2023-12-26 15:59:20,944][105585] KL-divergence is very high: 111.6087 [2023-12-26 15:59:20,978][105692] Updated weights for policy 0, policy_version 85677 (0.0009) [2023-12-26 15:59:20,992][105585] KL-divergence is very high: 115.6206 [2023-12-26 15:59:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 43958272. Throughput: 0: 9834.2, 1: 9832.2. Samples: 43942292. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 15:59:21,063][104569] Avg episode reward: [(0, '8488.762'), (1, '8558.619')] [2023-12-26 15:59:21,600][105620] Updated weights for policy 1, policy_version 86004 (0.0011) [2023-12-26 15:59:21,666][105620] Updated weights for policy 1, policy_version 86014 (0.0008) [2023-12-26 15:59:21,685][105692] Updated weights for policy 0, policy_version 85687 (0.0008) [2023-12-26 15:59:21,727][105620] Updated weights for policy 1, policy_version 86024 (0.0006) [2023-12-26 15:59:21,750][105692] Updated weights for policy 0, policy_version 85697 (0.0009) [2023-12-26 15:59:21,804][105692] Updated weights for policy 0, policy_version 85707 (0.0008) [2023-12-26 15:59:22,315][105620] Updated weights for policy 1, policy_version 86034 (0.0008) [2023-12-26 15:59:22,379][105620] Updated weights for policy 1, policy_version 86044 (0.0007) [2023-12-26 15:59:22,443][105620] Updated weights for policy 1, policy_version 86054 (0.0008) [2023-12-26 15:59:22,499][105620] Updated weights for policy 1, policy_version 86064 (0.0010) [2023-12-26 15:59:22,668][105692] Updated weights for policy 0, policy_version 85717 (0.0009) [2023-12-26 15:59:22,717][105692] Updated weights for policy 0, policy_version 85727 (0.0008) [2023-12-26 15:59:22,767][105692] Updated weights for policy 0, policy_version 85737 (0.0008) [2023-12-26 15:59:23,228][105620] Updated weights for policy 1, policy_version 86074 (0.0009) [2023-12-26 15:59:23,276][105620] Updated weights for policy 1, policy_version 86084 (0.0009) [2023-12-26 15:59:23,333][105620] Updated weights for policy 1, policy_version 86094 (0.0009) [2023-12-26 15:59:23,449][105692] Updated weights for policy 0, policy_version 85747 (0.0007) [2023-12-26 15:59:23,519][105692] Updated weights for policy 0, policy_version 85757 (0.0006) [2023-12-26 15:59:23,578][105692] Updated weights for policy 0, policy_version 85767 (0.0009) [2023-12-26 15:59:24,081][105620] Updated weights for policy 1, policy_version 86104 (0.0006) [2023-12-26 15:59:24,147][105620] Updated weights for policy 1, policy_version 86114 (0.0009) [2023-12-26 15:59:24,211][105620] Updated weights for policy 1, policy_version 86124 (0.0009) [2023-12-26 15:59:24,319][105692] Updated weights for policy 0, policy_version 85777 (0.0009) [2023-12-26 15:59:24,370][105692] Updated weights for policy 0, policy_version 85787 (0.0009) [2023-12-26 15:59:24,417][105692] Updated weights for policy 0, policy_version 85797 (0.0009) [2023-12-26 15:59:24,469][105692] Updated weights for policy 0, policy_version 85807 (0.0009) [2023-12-26 15:59:24,937][105620] Updated weights for policy 1, policy_version 86134 (0.0008) [2023-12-26 15:59:24,987][105620] Updated weights for policy 1, policy_version 86144 (0.0009) [2023-12-26 15:59:25,037][105620] Updated weights for policy 1, policy_version 86154 (0.0008) [2023-12-26 15:59:25,196][105692] Updated weights for policy 0, policy_version 85817 (0.0009) [2023-12-26 15:59:25,257][105692] Updated weights for policy 0, policy_version 85827 (0.0008) [2023-12-26 15:59:25,318][105692] Updated weights for policy 0, policy_version 85837 (0.0009) [2023-12-26 15:59:25,803][105620] Updated weights for policy 1, policy_version 86164 (0.0010) [2023-12-26 15:59:25,864][105620] Updated weights for policy 1, policy_version 86174 (0.0010) [2023-12-26 15:59:25,917][105620] Updated weights for policy 1, policy_version 86184 (0.0009) [2023-12-26 15:59:25,985][105692] Updated weights for policy 0, policy_version 85847 (0.0006) [2023-12-26 15:59:26,037][105692] Updated weights for policy 0, policy_version 85857 (0.0005) [2023-12-26 15:59:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 44048384. Throughput: 0: 9720.2, 1: 9756.6. Samples: 44055380. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 15:59:26,063][104569] Avg episode reward: [(0, '7679.372'), (1, '8209.411')] [2023-12-26 15:59:26,094][105692] Updated weights for policy 0, policy_version 85867 (0.0005) [2023-12-26 15:59:26,715][105620] Updated weights for policy 1, policy_version 86194 (0.0010) [2023-12-26 15:59:26,775][105692] Updated weights for policy 0, policy_version 85877 (0.0005) [2023-12-26 15:59:26,785][105620] Updated weights for policy 1, policy_version 86204 (0.0010) [2023-12-26 15:59:26,829][105692] Updated weights for policy 0, policy_version 85887 (0.0005) [2023-12-26 15:59:26,843][105620] Updated weights for policy 1, policy_version 86214 (0.0009) [2023-12-26 15:59:26,879][105692] Updated weights for policy 0, policy_version 85897 (0.0005) [2023-12-26 15:59:26,898][105620] Updated weights for policy 1, policy_version 86224 (0.0007) [2023-12-26 15:59:27,423][105692] Updated weights for policy 0, policy_version 85907 (0.0008) [2023-12-26 15:59:27,471][105692] Updated weights for policy 0, policy_version 85917 (0.0005) [2023-12-26 15:59:27,522][105692] Updated weights for policy 0, policy_version 85927 (0.0005) [2023-12-26 15:59:27,781][105620] Updated weights for policy 1, policy_version 86234 (0.0008) [2023-12-26 15:59:27,829][105620] Updated weights for policy 1, policy_version 86244 (0.0008) [2023-12-26 15:59:27,876][105620] Updated weights for policy 1, policy_version 86254 (0.0007) [2023-12-26 15:59:28,150][105692] Updated weights for policy 0, policy_version 85937 (0.0007) [2023-12-26 15:59:28,197][105692] Updated weights for policy 0, policy_version 85947 (0.0008) [2023-12-26 15:59:28,244][105692] Updated weights for policy 0, policy_version 85957 (0.0009) [2023-12-26 15:59:28,291][105692] Updated weights for policy 0, policy_version 85967 (0.0010) [2023-12-26 15:59:28,644][105620] Updated weights for policy 1, policy_version 86264 (0.0007) [2023-12-26 15:59:28,700][105620] Updated weights for policy 1, policy_version 86274 (0.0007) [2023-12-26 15:59:28,750][105620] Updated weights for policy 1, policy_version 86284 (0.0008) [2023-12-26 15:59:29,048][105692] Updated weights for policy 0, policy_version 85977 (0.0006) [2023-12-26 15:59:29,101][105692] Updated weights for policy 0, policy_version 85987 (0.0005) [2023-12-26 15:59:29,157][105692] Updated weights for policy 0, policy_version 85997 (0.0005) [2023-12-26 15:59:29,548][105620] Updated weights for policy 1, policy_version 86294 (0.0008) [2023-12-26 15:59:29,607][105620] Updated weights for policy 1, policy_version 86304 (0.0008) [2023-12-26 15:59:29,670][105620] Updated weights for policy 1, policy_version 86314 (0.0009) [2023-12-26 15:59:29,817][105692] Updated weights for policy 0, policy_version 86007 (0.0008) [2023-12-26 15:59:29,874][105692] Updated weights for policy 0, policy_version 86017 (0.0008) [2023-12-26 15:59:29,932][105692] Updated weights for policy 0, policy_version 86027 (0.0009) [2023-12-26 15:59:30,389][105620] Updated weights for policy 1, policy_version 86324 (0.0009) [2023-12-26 15:59:30,444][105620] Updated weights for policy 1, policy_version 86334 (0.0009) [2023-12-26 15:59:30,502][105620] Updated weights for policy 1, policy_version 86344 (0.0008) [2023-12-26 15:59:30,701][105692] Updated weights for policy 0, policy_version 86037 (0.0010) [2023-12-26 15:59:30,751][105692] Updated weights for policy 0, policy_version 86047 (0.0008) [2023-12-26 15:59:30,797][105692] Updated weights for policy 0, policy_version 86057 (0.0009) [2023-12-26 15:59:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 44146688. Throughput: 0: 9797.3, 1: 9670.2. Samples: 44114800. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 15:59:31,063][104569] Avg episode reward: [(0, '6252.204'), (1, '8118.185')] [2023-12-26 15:59:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000086064_22036480.pth... [2023-12-26 15:59:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000086352_22110208.pth... [2023-12-26 15:59:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000084912_21741568.pth [2023-12-26 15:59:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000085232_21823488.pth [2023-12-26 15:59:31,322][105620] Updated weights for policy 1, policy_version 86354 (0.0009) [2023-12-26 15:59:31,391][105620] Updated weights for policy 1, policy_version 86364 (0.0008) [2023-12-26 15:59:31,451][105620] Updated weights for policy 1, policy_version 86374 (0.0008) [2023-12-26 15:59:31,513][105620] Updated weights for policy 1, policy_version 86384 (0.0008) [2023-12-26 15:59:31,534][105692] Updated weights for policy 0, policy_version 86067 (0.0008) [2023-12-26 15:59:31,591][105692] Updated weights for policy 0, policy_version 86077 (0.0009) [2023-12-26 15:59:31,653][105692] Updated weights for policy 0, policy_version 86087 (0.0009) [2023-12-26 15:59:32,223][105620] Updated weights for policy 1, policy_version 86394 (0.0010) [2023-12-26 15:59:32,279][105620] Updated weights for policy 1, policy_version 86405 (0.0009) [2023-12-26 15:59:32,334][105620] Updated weights for policy 1, policy_version 86415 (0.0009) [2023-12-26 15:59:32,377][105692] Updated weights for policy 0, policy_version 86097 (0.0010) [2023-12-26 15:59:32,442][105692] Updated weights for policy 0, policy_version 86107 (0.0009) [2023-12-26 15:59:32,504][105692] Updated weights for policy 0, policy_version 86117 (0.0009) [2023-12-26 15:59:32,552][105692] Updated weights for policy 0, policy_version 86127 (0.0009) [2023-12-26 15:59:33,123][105620] Updated weights for policy 1, policy_version 86425 (0.0008) [2023-12-26 15:59:33,180][105620] Updated weights for policy 1, policy_version 86435 (0.0008) [2023-12-26 15:59:33,241][105620] Updated weights for policy 1, policy_version 86445 (0.0009) [2023-12-26 15:59:33,305][105692] Updated weights for policy 0, policy_version 86137 (0.0008) [2023-12-26 15:59:33,374][105692] Updated weights for policy 0, policy_version 86147 (0.0009) [2023-12-26 15:59:33,423][105692] Updated weights for policy 0, policy_version 86157 (0.0008) [2023-12-26 15:59:33,985][105620] Updated weights for policy 1, policy_version 86455 (0.0009) [2023-12-26 15:59:34,040][105620] Updated weights for policy 1, policy_version 86465 (0.0009) [2023-12-26 15:59:34,086][105620] Updated weights for policy 1, policy_version 86475 (0.0008) [2023-12-26 15:59:34,166][105692] Updated weights for policy 0, policy_version 86167 (0.0008) [2023-12-26 15:59:34,228][105692] Updated weights for policy 0, policy_version 86177 (0.0009) [2023-12-26 15:59:34,285][105692] Updated weights for policy 0, policy_version 86187 (0.0009) [2023-12-26 15:59:34,862][105620] Updated weights for policy 1, policy_version 86485 (0.0009) [2023-12-26 15:59:34,915][105620] Updated weights for policy 1, policy_version 86496 (0.0009) [2023-12-26 15:59:34,976][105620] Updated weights for policy 1, policy_version 86506 (0.0008) [2023-12-26 15:59:35,043][105692] Updated weights for policy 0, policy_version 86197 (0.0009) [2023-12-26 15:59:35,089][105692] Updated weights for policy 0, policy_version 86207 (0.0008) [2023-12-26 15:59:35,136][105692] Updated weights for policy 0, policy_version 86217 (0.0007) [2023-12-26 15:59:35,684][105692] Updated weights for policy 0, policy_version 86227 (0.0005) [2023-12-26 15:59:35,747][105692] Updated weights for policy 0, policy_version 86237 (0.0005) [2023-12-26 15:59:35,799][105692] Updated weights for policy 0, policy_version 86247 (0.0005) [2023-12-26 15:59:35,847][105620] Updated weights for policy 1, policy_version 86516 (0.0008) [2023-12-26 15:59:35,908][105620] Updated weights for policy 1, policy_version 86526 (0.0008) [2023-12-26 15:59:35,969][105620] Updated weights for policy 1, policy_version 86536 (0.0009) [2023-12-26 15:59:36,062][104569] Fps is (10 sec: 19660.0, 60 sec: 19524.1, 300 sec: 19549.7). Total num frames: 44244992. Throughput: 0: 9744.0, 1: 9534.7. Samples: 44228224. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 15:59:36,064][104569] Avg episode reward: [(0, '6952.241'), (1, '8209.117')] [2023-12-26 15:59:36,440][105692] Updated weights for policy 0, policy_version 86257 (0.0009) [2023-12-26 15:59:36,511][105692] Updated weights for policy 0, policy_version 86267 (0.0009) [2023-12-26 15:59:36,569][105692] Updated weights for policy 0, policy_version 86277 (0.0006) [2023-12-26 15:59:36,637][105692] Updated weights for policy 0, policy_version 86287 (0.0007) [2023-12-26 15:59:36,761][105620] Updated weights for policy 1, policy_version 86546 (0.0009) [2023-12-26 15:59:36,814][105620] Updated weights for policy 1, policy_version 86556 (0.0010) [2023-12-26 15:59:36,878][105620] Updated weights for policy 1, policy_version 86566 (0.0011) [2023-12-26 15:59:36,934][105620] Updated weights for policy 1, policy_version 86576 (0.0010) [2023-12-26 15:59:37,271][105692] Updated weights for policy 0, policy_version 86297 (0.0008) [2023-12-26 15:59:37,335][105692] Updated weights for policy 0, policy_version 86307 (0.0008) [2023-12-26 15:59:37,398][105692] Updated weights for policy 0, policy_version 86317 (0.0010) [2023-12-26 15:59:37,615][105620] Updated weights for policy 1, policy_version 86586 (0.0010) [2023-12-26 15:59:37,678][105620] Updated weights for policy 1, policy_version 86596 (0.0010) [2023-12-26 15:59:37,739][105620] Updated weights for policy 1, policy_version 86606 (0.0011) [2023-12-26 15:59:38,119][105692] Updated weights for policy 0, policy_version 86327 (0.0010) [2023-12-26 15:59:38,173][105692] Updated weights for policy 0, policy_version 86337 (0.0010) [2023-12-26 15:59:38,221][105692] Updated weights for policy 0, policy_version 86347 (0.0010) [2023-12-26 15:59:38,385][105620] Updated weights for policy 1, policy_version 86616 (0.0007) [2023-12-26 15:59:38,447][105620] Updated weights for policy 1, policy_version 86626 (0.0006) [2023-12-26 15:59:38,508][105620] Updated weights for policy 1, policy_version 86636 (0.0005) [2023-12-26 15:59:38,990][105692] Updated weights for policy 0, policy_version 86357 (0.0010) [2023-12-26 15:59:39,057][105692] Updated weights for policy 0, policy_version 86367 (0.0006) [2023-12-26 15:59:39,117][105692] Updated weights for policy 0, policy_version 86377 (0.0005) [2023-12-26 15:59:39,167][105620] Updated weights for policy 1, policy_version 86646 (0.0010) [2023-12-26 15:59:39,233][105620] Updated weights for policy 1, policy_version 86656 (0.0014) [2023-12-26 15:59:39,301][105620] Updated weights for policy 1, policy_version 86666 (0.0010) [2023-12-26 15:59:39,733][105692] Updated weights for policy 0, policy_version 86387 (0.0005) [2023-12-26 15:59:39,793][105692] Updated weights for policy 0, policy_version 86397 (0.0009) [2023-12-26 15:59:39,856][105692] Updated weights for policy 0, policy_version 86407 (0.0009) [2023-12-26 15:59:40,102][105620] Updated weights for policy 1, policy_version 86676 (0.0009) [2023-12-26 15:59:40,168][105620] Updated weights for policy 1, policy_version 86686 (0.0009) [2023-12-26 15:59:40,231][105620] Updated weights for policy 1, policy_version 86696 (0.0009) [2023-12-26 15:59:40,600][105692] Updated weights for policy 0, policy_version 86417 (0.0009) [2023-12-26 15:59:40,668][105692] Updated weights for policy 0, policy_version 86427 (0.0009) [2023-12-26 15:59:40,719][105585] KL-divergence is very high: 146.4744 [2023-12-26 15:59:40,732][105692] Updated weights for policy 0, policy_version 86437 (0.0010) [2023-12-26 15:59:40,745][105585] KL-divergence is very high: 129.0750 [2023-12-26 15:59:40,751][105585] KL-divergence is very high: 122.8825 [2023-12-26 15:59:40,761][105585] KL-divergence is very high: 156.1170 [2023-12-26 15:59:40,782][105692] Updated weights for policy 0, policy_version 86447 (0.0008) [2023-12-26 15:59:40,992][105620] Updated weights for policy 1, policy_version 86706 (0.0009) [2023-12-26 15:59:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 44335104. Throughput: 0: 9681.4, 1: 9606.7. Samples: 44345972. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 15:59:41,063][104569] Avg episode reward: [(0, '7150.051'), (1, '8122.414')] [2023-12-26 15:59:41,063][105620] Updated weights for policy 1, policy_version 86716 (0.0009) [2023-12-26 15:59:41,135][105620] Updated weights for policy 1, policy_version 86726 (0.0009) [2023-12-26 15:59:41,198][105620] Updated weights for policy 1, policy_version 86736 (0.0008) [2023-12-26 15:59:41,495][105585] KL-divergence is very high: 146.1430 [2023-12-26 15:59:41,550][105585] KL-divergence is very high: 139.6161 [2023-12-26 15:59:41,555][105692] Updated weights for policy 0, policy_version 86457 (0.0008) [2023-12-26 15:59:41,603][105585] KL-divergence is very high: 127.3961 [2023-12-26 15:59:41,622][105692] Updated weights for policy 0, policy_version 86467 (0.0010) [2023-12-26 15:59:41,656][105585] KL-divergence is very high: 117.5051 [2023-12-26 15:59:41,691][105692] Updated weights for policy 0, policy_version 86477 (0.0007) [2023-12-26 15:59:41,988][105620] Updated weights for policy 1, policy_version 86746 (0.0009) [2023-12-26 15:59:42,058][105620] Updated weights for policy 1, policy_version 86756 (0.0006) [2023-12-26 15:59:42,123][105620] Updated weights for policy 1, policy_version 86766 (0.0008) [2023-12-26 15:59:42,367][105692] Updated weights for policy 0, policy_version 86487 (0.0010) [2023-12-26 15:59:42,435][105692] Updated weights for policy 0, policy_version 86497 (0.0009) [2023-12-26 15:59:42,500][105692] Updated weights for policy 0, policy_version 86507 (0.0009) [2023-12-26 15:59:42,784][105620] Updated weights for policy 1, policy_version 86776 (0.0006) [2023-12-26 15:59:42,840][105620] Updated weights for policy 1, policy_version 86786 (0.0006) [2023-12-26 15:59:42,897][105620] Updated weights for policy 1, policy_version 86796 (0.0006) [2023-12-26 15:59:43,076][105692] Updated weights for policy 0, policy_version 86517 (0.0008) [2023-12-26 15:59:43,123][105692] Updated weights for policy 0, policy_version 86527 (0.0006) [2023-12-26 15:59:43,172][105692] Updated weights for policy 0, policy_version 86537 (0.0005) [2023-12-26 15:59:43,628][105620] Updated weights for policy 1, policy_version 86806 (0.0010) [2023-12-26 15:59:43,695][105620] Updated weights for policy 1, policy_version 86816 (0.0010) [2023-12-26 15:59:43,733][105692] Updated weights for policy 0, policy_version 86547 (0.0006) [2023-12-26 15:59:43,761][105620] Updated weights for policy 1, policy_version 86826 (0.0008) [2023-12-26 15:59:43,782][105692] Updated weights for policy 0, policy_version 86557 (0.0005) [2023-12-26 15:59:43,829][105692] Updated weights for policy 0, policy_version 86567 (0.0007) [2023-12-26 15:59:44,322][105620] Updated weights for policy 1, policy_version 86836 (0.0008) [2023-12-26 15:59:44,384][105620] Updated weights for policy 1, policy_version 86846 (0.0010) [2023-12-26 15:59:44,442][105620] Updated weights for policy 1, policy_version 86856 (0.0010) [2023-12-26 15:59:44,477][105692] Updated weights for policy 0, policy_version 86577 (0.0009) [2023-12-26 15:59:44,534][105692] Updated weights for policy 0, policy_version 86587 (0.0008) [2023-12-26 15:59:44,596][105692] Updated weights for policy 0, policy_version 86597 (0.0008) [2023-12-26 15:59:44,662][105692] Updated weights for policy 0, policy_version 86607 (0.0008) [2023-12-26 15:59:45,080][105620] Updated weights for policy 1, policy_version 86866 (0.0009) [2023-12-26 15:59:45,133][105620] Updated weights for policy 1, policy_version 86876 (0.0005) [2023-12-26 15:59:45,195][105620] Updated weights for policy 1, policy_version 86886 (0.0007) [2023-12-26 15:59:45,265][105620] Updated weights for policy 1, policy_version 86896 (0.0008) [2023-12-26 15:59:45,403][105692] Updated weights for policy 0, policy_version 86617 (0.0010) [2023-12-26 15:59:45,461][105692] Updated weights for policy 0, policy_version 86627 (0.0010) [2023-12-26 15:59:45,519][105692] Updated weights for policy 0, policy_version 86637 (0.0010) [2023-12-26 15:59:45,951][105620] Updated weights for policy 1, policy_version 86906 (0.0007) [2023-12-26 15:59:46,008][105620] Updated weights for policy 1, policy_version 86916 (0.0006) [2023-12-26 15:59:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 44433408. Throughput: 0: 9752.5, 1: 9585.3. Samples: 44405252. Policy #0 lag: (min: 31.0, avg: 32.4, max: 62.0) [2023-12-26 15:59:46,063][104569] Avg episode reward: [(0, '6273.360'), (1, '8119.904')] [2023-12-26 15:59:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000086640_22183936.pth... [2023-12-26 15:59:46,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000085488_21889024.pth [2023-12-26 15:59:46,081][105620] Updated weights for policy 1, policy_version 86926 (0.0006) [2023-12-26 15:59:46,092][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000086928_22257664.pth... [2023-12-26 15:59:46,095][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000085808_21970944.pth [2023-12-26 15:59:46,215][105692] Updated weights for policy 0, policy_version 86647 (0.0009) [2023-12-26 15:59:46,269][105692] Updated weights for policy 0, policy_version 86657 (0.0009) [2023-12-26 15:59:46,322][105692] Updated weights for policy 0, policy_version 86668 (0.0010) [2023-12-26 15:59:46,674][105620] Updated weights for policy 1, policy_version 86936 (0.0006) [2023-12-26 15:59:46,733][105620] Updated weights for policy 1, policy_version 86946 (0.0005) [2023-12-26 15:59:46,790][105620] Updated weights for policy 1, policy_version 86956 (0.0005) [2023-12-26 15:59:47,136][105692] Updated weights for policy 0, policy_version 86678 (0.0010) [2023-12-26 15:59:47,197][105692] Updated weights for policy 0, policy_version 86688 (0.0009) [2023-12-26 15:59:47,261][105692] Updated weights for policy 0, policy_version 86698 (0.0009) [2023-12-26 15:59:47,442][105620] Updated weights for policy 1, policy_version 86966 (0.0007) [2023-12-26 15:59:47,494][105620] Updated weights for policy 1, policy_version 86976 (0.0009) [2023-12-26 15:59:47,546][105620] Updated weights for policy 1, policy_version 86986 (0.0009) [2023-12-26 15:59:48,028][105692] Updated weights for policy 0, policy_version 86708 (0.0009) [2023-12-26 15:59:48,086][105692] Updated weights for policy 0, policy_version 86718 (0.0010) [2023-12-26 15:59:48,140][105692] Updated weights for policy 0, policy_version 86728 (0.0008) [2023-12-26 15:59:48,183][105620] Updated weights for policy 1, policy_version 86996 (0.0010) [2023-12-26 15:59:48,235][105620] Updated weights for policy 1, policy_version 87006 (0.0008) [2023-12-26 15:59:48,279][105620] Updated weights for policy 1, policy_version 87016 (0.0006) [2023-12-26 15:59:48,947][105620] Updated weights for policy 1, policy_version 87026 (0.0010) [2023-12-26 15:59:48,967][105692] Updated weights for policy 0, policy_version 86738 (0.0007) [2023-12-26 15:59:49,010][105620] Updated weights for policy 1, policy_version 87036 (0.0011) [2023-12-26 15:59:49,020][105692] Updated weights for policy 0, policy_version 86748 (0.0006) [2023-12-26 15:59:49,071][105692] Updated weights for policy 0, policy_version 86758 (0.0006) [2023-12-26 15:59:49,071][105620] Updated weights for policy 1, policy_version 87046 (0.0010) [2023-12-26 15:59:49,120][105692] Updated weights for policy 0, policy_version 86768 (0.0008) [2023-12-26 15:59:49,129][105620] Updated weights for policy 1, policy_version 87056 (0.0010) [2023-12-26 15:59:49,847][105620] Updated weights for policy 1, policy_version 87066 (0.0011) [2023-12-26 15:59:49,911][105620] Updated weights for policy 1, policy_version 87076 (0.0009) [2023-12-26 15:59:49,928][105692] Updated weights for policy 0, policy_version 86778 (0.0007) [2023-12-26 15:59:49,973][105620] Updated weights for policy 1, policy_version 87086 (0.0010) [2023-12-26 15:59:49,995][105692] Updated weights for policy 0, policy_version 86788 (0.0008) [2023-12-26 15:59:50,057][105692] Updated weights for policy 0, policy_version 86798 (0.0009) [2023-12-26 15:59:50,620][105620] Updated weights for policy 1, policy_version 87096 (0.0010) [2023-12-26 15:59:50,680][105620] Updated weights for policy 1, policy_version 87106 (0.0011) [2023-12-26 15:59:50,740][105620] Updated weights for policy 1, policy_version 87116 (0.0011) [2023-12-26 15:59:50,850][105692] Updated weights for policy 0, policy_version 86808 (0.0008) [2023-12-26 15:59:50,912][105692] Updated weights for policy 0, policy_version 86818 (0.0009) [2023-12-26 15:59:50,974][105692] Updated weights for policy 0, policy_version 86828 (0.0008) [2023-12-26 15:59:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 44539904. Throughput: 0: 9793.1, 1: 9643.0. Samples: 44524856. Policy #0 lag: (min: 31.0, avg: 32.4, max: 62.0) [2023-12-26 15:59:51,062][104569] Avg episode reward: [(0, '7236.985'), (1, '8378.428')] [2023-12-26 15:59:51,440][105620] Updated weights for policy 1, policy_version 87126 (0.0010) [2023-12-26 15:59:51,488][105620] Updated weights for policy 1, policy_version 87136 (0.0009) [2023-12-26 15:59:51,547][105620] Updated weights for policy 1, policy_version 87146 (0.0009) [2023-12-26 15:59:51,810][105692] Updated weights for policy 0, policy_version 86838 (0.0007) [2023-12-26 15:59:51,877][105692] Updated weights for policy 0, policy_version 86848 (0.0007) [2023-12-26 15:59:51,926][105692] Updated weights for policy 0, policy_version 86858 (0.0008) [2023-12-26 15:59:52,322][105620] Updated weights for policy 1, policy_version 87156 (0.0009) [2023-12-26 15:59:52,388][105620] Updated weights for policy 1, policy_version 87166 (0.0008) [2023-12-26 15:59:52,457][105620] Updated weights for policy 1, policy_version 87176 (0.0008) [2023-12-26 15:59:52,649][105692] Updated weights for policy 0, policy_version 86868 (0.0009) [2023-12-26 15:59:52,711][105692] Updated weights for policy 0, policy_version 86878 (0.0009) [2023-12-26 15:59:52,771][105692] Updated weights for policy 0, policy_version 86888 (0.0009) [2023-12-26 15:59:53,095][105620] Updated weights for policy 1, policy_version 87186 (0.0006) [2023-12-26 15:59:53,141][105620] Updated weights for policy 1, policy_version 87196 (0.0005) [2023-12-26 15:59:53,187][105620] Updated weights for policy 1, policy_version 87206 (0.0005) [2023-12-26 15:59:53,255][105620] Updated weights for policy 1, policy_version 87216 (0.0006) [2023-12-26 15:59:53,613][105692] Updated weights for policy 0, policy_version 86898 (0.0009) [2023-12-26 15:59:53,670][105692] Updated weights for policy 0, policy_version 86908 (0.0009) [2023-12-26 15:59:53,725][105692] Updated weights for policy 0, policy_version 86918 (0.0007) [2023-12-26 15:59:53,779][105692] Updated weights for policy 0, policy_version 86928 (0.0009) [2023-12-26 15:59:53,891][105620] Updated weights for policy 1, policy_version 87226 (0.0008) [2023-12-26 15:59:53,938][105620] Updated weights for policy 1, policy_version 87236 (0.0009) [2023-12-26 15:59:53,992][105620] Updated weights for policy 1, policy_version 87246 (0.0009) [2023-12-26 15:59:54,463][105692] Updated weights for policy 0, policy_version 86938 (0.0009) [2023-12-26 15:59:54,529][105692] Updated weights for policy 0, policy_version 86948 (0.0008) [2023-12-26 15:59:54,589][105692] Updated weights for policy 0, policy_version 86958 (0.0008) [2023-12-26 15:59:54,798][105620] Updated weights for policy 1, policy_version 87256 (0.0006) [2023-12-26 15:59:54,871][105620] Updated weights for policy 1, policy_version 87266 (0.0005) [2023-12-26 15:59:54,921][105620] Updated weights for policy 1, policy_version 87276 (0.0006) [2023-12-26 15:59:55,250][105692] Updated weights for policy 0, policy_version 86968 (0.0009) [2023-12-26 15:59:55,297][105692] Updated weights for policy 0, policy_version 86978 (0.0008) [2023-12-26 15:59:55,360][105692] Updated weights for policy 0, policy_version 86988 (0.0009) [2023-12-26 15:59:55,540][105620] Updated weights for policy 1, policy_version 87286 (0.0005) [2023-12-26 15:59:55,585][105620] Updated weights for policy 1, policy_version 87296 (0.0005) [2023-12-26 15:59:55,640][105620] Updated weights for policy 1, policy_version 87306 (0.0005) [2023-12-26 15:59:56,062][104569] Fps is (10 sec: 19661.7, 60 sec: 19251.3, 300 sec: 19577.5). Total num frames: 44630016. Throughput: 0: 9753.3, 1: 9697.8. Samples: 44640272. Policy #0 lag: (min: 31.0, avg: 32.4, max: 62.0) [2023-12-26 15:59:56,062][104569] Avg episode reward: [(0, '7129.967'), (1, '8476.312')] [2023-12-26 15:59:56,171][105692] Updated weights for policy 0, policy_version 86998 (0.0009) [2023-12-26 15:59:56,232][105692] Updated weights for policy 0, policy_version 87008 (0.0008) [2023-12-26 15:59:56,259][105620] Updated weights for policy 1, policy_version 87316 (0.0007) [2023-12-26 15:59:56,293][105692] Updated weights for policy 0, policy_version 87018 (0.0009) [2023-12-26 15:59:56,316][105620] Updated weights for policy 1, policy_version 87326 (0.0007) [2023-12-26 15:59:56,368][105620] Updated weights for policy 1, policy_version 87336 (0.0008) [2023-12-26 15:59:57,053][105692] Updated weights for policy 0, policy_version 87028 (0.0008) [2023-12-26 15:59:57,111][105692] Updated weights for policy 0, policy_version 87038 (0.0007) [2023-12-26 15:59:57,123][105620] Updated weights for policy 1, policy_version 87346 (0.0009) [2023-12-26 15:59:57,159][105692] Updated weights for policy 0, policy_version 87048 (0.0006) [2023-12-26 15:59:57,176][105620] Updated weights for policy 1, policy_version 87356 (0.0007) [2023-12-26 15:59:57,234][105620] Updated weights for policy 1, policy_version 87366 (0.0009) [2023-12-26 15:59:57,280][105620] Updated weights for policy 1, policy_version 87376 (0.0008) [2023-12-26 15:59:57,905][105692] Updated weights for policy 0, policy_version 87058 (0.0006) [2023-12-26 15:59:57,959][105692] Updated weights for policy 0, policy_version 87068 (0.0007) [2023-12-26 15:59:57,970][105620] Updated weights for policy 1, policy_version 87386 (0.0007) [2023-12-26 15:59:58,007][105692] Updated weights for policy 0, policy_version 87078 (0.0009) [2023-12-26 15:59:58,018][105620] Updated weights for policy 1, policy_version 87396 (0.0007) [2023-12-26 15:59:58,059][105692] Updated weights for policy 0, policy_version 87088 (0.0006) [2023-12-26 15:59:58,071][105620] Updated weights for policy 1, policy_version 87406 (0.0008) [2023-12-26 15:59:58,827][105620] Updated weights for policy 1, policy_version 87416 (0.0008) [2023-12-26 15:59:58,892][105620] Updated weights for policy 1, policy_version 87426 (0.0008) [2023-12-26 15:59:58,928][105692] Updated weights for policy 0, policy_version 87098 (0.0007) [2023-12-26 15:59:58,964][105620] Updated weights for policy 1, policy_version 87436 (0.0008) [2023-12-26 15:59:58,994][105692] Updated weights for policy 0, policy_version 87108 (0.0009) [2023-12-26 15:59:59,054][105692] Updated weights for policy 0, policy_version 87118 (0.0008) [2023-12-26 15:59:59,693][105620] Updated weights for policy 1, policy_version 87446 (0.0008) [2023-12-26 15:59:59,739][105620] Updated weights for policy 1, policy_version 87456 (0.0008) [2023-12-26 15:59:59,791][105620] Updated weights for policy 1, policy_version 87466 (0.0008) [2023-12-26 15:59:59,821][105692] Updated weights for policy 0, policy_version 87128 (0.0009) [2023-12-26 15:59:59,881][105692] Updated weights for policy 0, policy_version 87138 (0.0009) [2023-12-26 15:59:59,942][105692] Updated weights for policy 0, policy_version 87148 (0.0009) [2023-12-26 16:00:00,499][105620] Updated weights for policy 1, policy_version 87476 (0.0007) [2023-12-26 16:00:00,559][105620] Updated weights for policy 1, policy_version 87486 (0.0007) [2023-12-26 16:00:00,629][105620] Updated weights for policy 1, policy_version 87496 (0.0006) [2023-12-26 16:00:00,762][105692] Updated weights for policy 0, policy_version 87158 (0.0009) [2023-12-26 16:00:00,808][105692] Updated weights for policy 0, policy_version 87168 (0.0009) [2023-12-26 16:00:00,858][105692] Updated weights for policy 0, policy_version 87178 (0.0008) [2023-12-26 16:00:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 44728320. Throughput: 0: 9720.0, 1: 9704.6. Samples: 44696928. Policy #0 lag: (min: 31.0, avg: 32.4, max: 62.0) [2023-12-26 16:00:01,062][104569] Avg episode reward: [(0, '6309.059'), (1, '8560.038')] [2023-12-26 16:00:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000087184_22323200.pth... [2023-12-26 16:00:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000087504_22405120.pth... [2023-12-26 16:00:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000086064_22036480.pth [2023-12-26 16:00:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000086352_22110208.pth [2023-12-26 16:00:01,290][105620] Updated weights for policy 1, policy_version 87506 (0.0009) [2023-12-26 16:00:01,356][105620] Updated weights for policy 1, policy_version 87516 (0.0009) [2023-12-26 16:00:01,418][105620] Updated weights for policy 1, policy_version 87526 (0.0009) [2023-12-26 16:00:01,476][105620] Updated weights for policy 1, policy_version 87536 (0.0009) [2023-12-26 16:00:01,620][105692] Updated weights for policy 0, policy_version 87188 (0.0009) [2023-12-26 16:00:01,690][105692] Updated weights for policy 0, policy_version 87198 (0.0010) [2023-12-26 16:00:01,751][105692] Updated weights for policy 0, policy_version 87208 (0.0010) [2023-12-26 16:00:02,234][105620] Updated weights for policy 1, policy_version 87546 (0.0009) [2023-12-26 16:00:02,293][105620] Updated weights for policy 1, policy_version 87556 (0.0009) [2023-12-26 16:00:02,351][105620] Updated weights for policy 1, policy_version 87566 (0.0010) [2023-12-26 16:00:02,455][105692] Updated weights for policy 0, policy_version 87218 (0.0008) [2023-12-26 16:00:02,518][105692] Updated weights for policy 0, policy_version 87228 (0.0005) [2023-12-26 16:00:02,570][105692] Updated weights for policy 0, policy_version 87238 (0.0005) [2023-12-26 16:00:02,620][105692] Updated weights for policy 0, policy_version 87248 (0.0005) [2023-12-26 16:00:03,059][105620] Updated weights for policy 1, policy_version 87576 (0.0006) [2023-12-26 16:00:03,122][105620] Updated weights for policy 1, policy_version 87586 (0.0006) [2023-12-26 16:00:03,176][105620] Updated weights for policy 1, policy_version 87596 (0.0009) [2023-12-26 16:00:03,190][105692] Updated weights for policy 0, policy_version 87258 (0.0006) [2023-12-26 16:00:03,253][105692] Updated weights for policy 0, policy_version 87268 (0.0009) [2023-12-26 16:00:03,299][105692] Updated weights for policy 0, policy_version 87278 (0.0006) [2023-12-26 16:00:03,757][105620] Updated weights for policy 1, policy_version 87606 (0.0007) [2023-12-26 16:00:03,826][105620] Updated weights for policy 1, policy_version 87616 (0.0006) [2023-12-26 16:00:03,894][105620] Updated weights for policy 1, policy_version 87626 (0.0009) [2023-12-26 16:00:03,923][105692] Updated weights for policy 0, policy_version 87288 (0.0008) [2023-12-26 16:00:03,986][105692] Updated weights for policy 0, policy_version 87298 (0.0009) [2023-12-26 16:00:04,054][105692] Updated weights for policy 0, policy_version 87308 (0.0010) [2023-12-26 16:00:04,436][105620] Updated weights for policy 1, policy_version 87636 (0.0007) [2023-12-26 16:00:04,500][105620] Updated weights for policy 1, policy_version 87646 (0.0008) [2023-12-26 16:00:04,565][105620] Updated weights for policy 1, policy_version 87656 (0.0007) [2023-12-26 16:00:04,866][105692] Updated weights for policy 0, policy_version 87318 (0.0010) [2023-12-26 16:00:04,926][105692] Updated weights for policy 0, policy_version 87328 (0.0008) [2023-12-26 16:00:04,988][105692] Updated weights for policy 0, policy_version 87338 (0.0006) [2023-12-26 16:00:05,114][105620] Updated weights for policy 1, policy_version 87666 (0.0006) [2023-12-26 16:00:05,162][105620] Updated weights for policy 1, policy_version 87676 (0.0010) [2023-12-26 16:00:05,210][105620] Updated weights for policy 1, policy_version 87686 (0.0010) [2023-12-26 16:00:05,260][105620] Updated weights for policy 1, policy_version 87696 (0.0010) [2023-12-26 16:00:05,652][105692] Updated weights for policy 0, policy_version 87348 (0.0006) [2023-12-26 16:00:05,708][105692] Updated weights for policy 0, policy_version 87358 (0.0005) [2023-12-26 16:00:05,771][105692] Updated weights for policy 0, policy_version 87368 (0.0006) [2023-12-26 16:00:05,864][105620] Updated weights for policy 1, policy_version 87706 (0.0008) [2023-12-26 16:00:05,909][105620] Updated weights for policy 1, policy_version 87716 (0.0010) [2023-12-26 16:00:05,958][105620] Updated weights for policy 1, policy_version 87726 (0.0010) [2023-12-26 16:00:06,062][104569] Fps is (10 sec: 20479.4, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 44834816. Throughput: 0: 9680.5, 1: 9748.2. Samples: 44816588. Policy #0 lag: (min: 31.0, avg: 32.4, max: 62.0) [2023-12-26 16:00:06,063][104569] Avg episode reward: [(0, '6685.973'), (1, '8824.931')] [2023-12-26 16:00:06,370][105692] Updated weights for policy 0, policy_version 87378 (0.0008) [2023-12-26 16:00:06,435][105692] Updated weights for policy 0, policy_version 87388 (0.0010) [2023-12-26 16:00:06,498][105692] Updated weights for policy 0, policy_version 87398 (0.0011) [2023-12-26 16:00:06,559][105692] Updated weights for policy 0, policy_version 87408 (0.0009) [2023-12-26 16:00:06,723][105620] Updated weights for policy 1, policy_version 87736 (0.0011) [2023-12-26 16:00:06,782][105620] Updated weights for policy 1, policy_version 87746 (0.0009) [2023-12-26 16:00:06,848][105620] Updated weights for policy 1, policy_version 87756 (0.0011) [2023-12-26 16:00:07,254][105692] Updated weights for policy 0, policy_version 87418 (0.0006) [2023-12-26 16:00:07,322][105692] Updated weights for policy 0, policy_version 87428 (0.0010) [2023-12-26 16:00:07,392][105692] Updated weights for policy 0, policy_version 87438 (0.0007) [2023-12-26 16:00:07,591][105620] Updated weights for policy 1, policy_version 87766 (0.0010) [2023-12-26 16:00:07,655][105620] Updated weights for policy 1, policy_version 87776 (0.0008) [2023-12-26 16:00:07,724][105620] Updated weights for policy 1, policy_version 87786 (0.0005) [2023-12-26 16:00:08,026][105692] Updated weights for policy 0, policy_version 87448 (0.0006) [2023-12-26 16:00:08,076][105692] Updated weights for policy 0, policy_version 87458 (0.0008) [2023-12-26 16:00:08,126][105692] Updated weights for policy 0, policy_version 87468 (0.0008) [2023-12-26 16:00:08,420][105620] Updated weights for policy 1, policy_version 87796 (0.0007) [2023-12-26 16:00:08,468][105620] Updated weights for policy 1, policy_version 87806 (0.0008) [2023-12-26 16:00:08,523][105620] Updated weights for policy 1, policy_version 87816 (0.0006) [2023-12-26 16:00:08,922][105692] Updated weights for policy 0, policy_version 87478 (0.0009) [2023-12-26 16:00:08,983][105692] Updated weights for policy 0, policy_version 87488 (0.0010) [2023-12-26 16:00:09,048][105692] Updated weights for policy 0, policy_version 87498 (0.0010) [2023-12-26 16:00:09,139][105620] Updated weights for policy 1, policy_version 87826 (0.0006) [2023-12-26 16:00:09,194][105620] Updated weights for policy 1, policy_version 87836 (0.0008) [2023-12-26 16:00:09,257][105620] Updated weights for policy 1, policy_version 87846 (0.0008) [2023-12-26 16:00:09,313][105620] Updated weights for policy 1, policy_version 87856 (0.0008) [2023-12-26 16:00:09,793][105692] Updated weights for policy 0, policy_version 87508 (0.0010) [2023-12-26 16:00:09,863][105692] Updated weights for policy 0, policy_version 87518 (0.0011) [2023-12-26 16:00:09,919][105692] Updated weights for policy 0, policy_version 87528 (0.0011) [2023-12-26 16:00:10,161][105620] Updated weights for policy 1, policy_version 87866 (0.0008) [2023-12-26 16:00:10,224][105620] Updated weights for policy 1, policy_version 87876 (0.0005) [2023-12-26 16:00:10,282][105620] Updated weights for policy 1, policy_version 87886 (0.0005) [2023-12-26 16:00:10,678][105692] Updated weights for policy 0, policy_version 87538 (0.0010) [2023-12-26 16:00:10,742][105692] Updated weights for policy 0, policy_version 87548 (0.0009) [2023-12-26 16:00:10,805][105692] Updated weights for policy 0, policy_version 87558 (0.0009) [2023-12-26 16:00:10,856][105692] Updated weights for policy 0, policy_version 87568 (0.0006) [2023-12-26 16:00:10,909][105620] Updated weights for policy 1, policy_version 87896 (0.0008) [2023-12-26 16:00:10,963][105620] Updated weights for policy 1, policy_version 87906 (0.0009) [2023-12-26 16:00:11,018][105620] Updated weights for policy 1, policy_version 87916 (0.0010) [2023-12-26 16:00:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 44933120. Throughput: 0: 9731.4, 1: 9819.3. Samples: 44935164. Policy #0 lag: (min: 31.0, avg: 32.4, max: 62.0) [2023-12-26 16:00:11,063][104569] Avg episode reward: [(0, '6381.795'), (1, '8646.887')] [2023-12-26 16:00:11,570][105692] Updated weights for policy 0, policy_version 87578 (0.0006) [2023-12-26 16:00:11,635][105692] Updated weights for policy 0, policy_version 87588 (0.0007) [2023-12-26 16:00:11,710][105692] Updated weights for policy 0, policy_version 87598 (0.0008) [2023-12-26 16:00:11,919][105620] Updated weights for policy 1, policy_version 87926 (0.0008) [2023-12-26 16:00:11,988][105620] Updated weights for policy 1, policy_version 87936 (0.0008) [2023-12-26 16:00:12,052][105620] Updated weights for policy 1, policy_version 87946 (0.0008) [2023-12-26 16:00:12,474][105692] Updated weights for policy 0, policy_version 87608 (0.0010) [2023-12-26 16:00:12,540][105692] Updated weights for policy 0, policy_version 87618 (0.0010) [2023-12-26 16:00:12,598][105692] Updated weights for policy 0, policy_version 87628 (0.0008) [2023-12-26 16:00:12,837][105620] Updated weights for policy 1, policy_version 87956 (0.0009) [2023-12-26 16:00:12,895][105620] Updated weights for policy 1, policy_version 87966 (0.0009) [2023-12-26 16:00:12,953][105620] Updated weights for policy 1, policy_version 87976 (0.0009) [2023-12-26 16:00:13,237][105692] Updated weights for policy 0, policy_version 87638 (0.0008) [2023-12-26 16:00:13,291][105692] Updated weights for policy 0, policy_version 87648 (0.0010) [2023-12-26 16:00:13,344][105692] Updated weights for policy 0, policy_version 87658 (0.0009) [2023-12-26 16:00:13,553][105620] Updated weights for policy 1, policy_version 87986 (0.0008) [2023-12-26 16:00:13,601][105620] Updated weights for policy 1, policy_version 87996 (0.0008) [2023-12-26 16:00:13,643][105620] Updated weights for policy 1, policy_version 88006 (0.0005) [2023-12-26 16:00:13,696][105620] Updated weights for policy 1, policy_version 88016 (0.0005) [2023-12-26 16:00:14,226][105692] Updated weights for policy 0, policy_version 87668 (0.0009) [2023-12-26 16:00:14,272][105692] Updated weights for policy 0, policy_version 87678 (0.0009) [2023-12-26 16:00:14,321][105692] Updated weights for policy 0, policy_version 87688 (0.0008) [2023-12-26 16:00:14,339][105620] Updated weights for policy 1, policy_version 88026 (0.0010) [2023-12-26 16:00:14,394][105620] Updated weights for policy 1, policy_version 88036 (0.0008) [2023-12-26 16:00:14,446][105620] Updated weights for policy 1, policy_version 88046 (0.0010) [2023-12-26 16:00:15,035][105692] Updated weights for policy 0, policy_version 87698 (0.0006) [2023-12-26 16:00:15,106][105692] Updated weights for policy 0, policy_version 87708 (0.0006) [2023-12-26 16:00:15,172][105620] Updated weights for policy 1, policy_version 88056 (0.0009) [2023-12-26 16:00:15,172][105692] Updated weights for policy 0, policy_version 87718 (0.0006) [2023-12-26 16:00:15,236][105620] Updated weights for policy 1, policy_version 88066 (0.0008) [2023-12-26 16:00:15,239][105692] Updated weights for policy 0, policy_version 87728 (0.0006) [2023-12-26 16:00:15,302][105620] Updated weights for policy 1, policy_version 88076 (0.0009) [2023-12-26 16:00:15,902][105620] Updated weights for policy 1, policy_version 88086 (0.0007) [2023-12-26 16:00:15,940][105692] Updated weights for policy 0, policy_version 87738 (0.0007) [2023-12-26 16:00:15,954][105620] Updated weights for policy 1, policy_version 88096 (0.0007) [2023-12-26 16:00:15,988][105692] Updated weights for policy 0, policy_version 87748 (0.0006) [2023-12-26 16:00:16,003][105620] Updated weights for policy 1, policy_version 88106 (0.0006) [2023-12-26 16:00:16,034][105692] Updated weights for policy 0, policy_version 87758 (0.0006) [2023-12-26 16:00:16,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 45031424. Throughput: 0: 9630.5, 1: 9889.9. Samples: 44993216. Policy #0 lag: (min: 31.0, avg: 32.4, max: 62.0) [2023-12-26 16:00:16,063][104569] Avg episode reward: [(0, '6284.707'), (1, '8738.159')] [2023-12-26 16:00:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000087760_22470656.pth... [2023-12-26 16:00:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000088112_22560768.pth... [2023-12-26 16:00:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000086928_22257664.pth [2023-12-26 16:00:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000086640_22183936.pth [2023-12-26 16:00:16,773][105620] Updated weights for policy 1, policy_version 88116 (0.0007) [2023-12-26 16:00:16,807][105692] Updated weights for policy 0, policy_version 87768 (0.0007) [2023-12-26 16:00:16,828][105620] Updated weights for policy 1, policy_version 88126 (0.0007) [2023-12-26 16:00:16,857][105692] Updated weights for policy 0, policy_version 87778 (0.0007) [2023-12-26 16:00:16,878][105620] Updated weights for policy 1, policy_version 88136 (0.0006) [2023-12-26 16:00:16,914][105692] Updated weights for policy 0, policy_version 87788 (0.0008) [2023-12-26 16:00:17,563][105620] Updated weights for policy 1, policy_version 88146 (0.0005) [2023-12-26 16:00:17,614][105620] Updated weights for policy 1, policy_version 88156 (0.0010) [2023-12-26 16:00:17,658][105620] Updated weights for policy 1, policy_version 88166 (0.0010) [2023-12-26 16:00:17,692][105692] Updated weights for policy 0, policy_version 87798 (0.0008) [2023-12-26 16:00:17,706][105620] Updated weights for policy 1, policy_version 88176 (0.0010) [2023-12-26 16:00:17,752][105692] Updated weights for policy 0, policy_version 87808 (0.0007) [2023-12-26 16:00:17,808][105692] Updated weights for policy 0, policy_version 87818 (0.0007) [2023-12-26 16:00:18,473][105620] Updated weights for policy 1, policy_version 88186 (0.0010) [2023-12-26 16:00:18,523][105692] Updated weights for policy 0, policy_version 87828 (0.0008) [2023-12-26 16:00:18,526][105620] Updated weights for policy 1, policy_version 88196 (0.0010) [2023-12-26 16:00:18,584][105692] Updated weights for policy 0, policy_version 87838 (0.0006) [2023-12-26 16:00:18,588][105620] Updated weights for policy 1, policy_version 88206 (0.0009) [2023-12-26 16:00:18,647][105692] Updated weights for policy 0, policy_version 87848 (0.0010) [2023-12-26 16:00:19,288][105620] Updated weights for policy 1, policy_version 88216 (0.0009) [2023-12-26 16:00:19,358][105620] Updated weights for policy 1, policy_version 88226 (0.0010) [2023-12-26 16:00:19,422][105692] Updated weights for policy 0, policy_version 87858 (0.0007) [2023-12-26 16:00:19,427][105620] Updated weights for policy 1, policy_version 88236 (0.0006) [2023-12-26 16:00:19,490][105692] Updated weights for policy 0, policy_version 87868 (0.0007) [2023-12-26 16:00:19,555][105692] Updated weights for policy 0, policy_version 87878 (0.0009) [2023-12-26 16:00:19,620][105692] Updated weights for policy 0, policy_version 87888 (0.0009) [2023-12-26 16:00:20,132][105620] Updated weights for policy 1, policy_version 88246 (0.0006) [2023-12-26 16:00:20,202][105620] Updated weights for policy 1, policy_version 88256 (0.0005) [2023-12-26 16:00:20,263][105620] Updated weights for policy 1, policy_version 88266 (0.0008) [2023-12-26 16:00:20,285][105692] Updated weights for policy 0, policy_version 87898 (0.0010) [2023-12-26 16:00:20,348][105692] Updated weights for policy 0, policy_version 87908 (0.0009) [2023-12-26 16:00:20,406][105692] Updated weights for policy 0, policy_version 87918 (0.0009) [2023-12-26 16:00:20,888][105620] Updated weights for policy 1, policy_version 88276 (0.0008) [2023-12-26 16:00:20,949][105620] Updated weights for policy 1, policy_version 88286 (0.0005) [2023-12-26 16:00:21,011][105620] Updated weights for policy 1, policy_version 88296 (0.0009) [2023-12-26 16:00:21,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 45121536. Throughput: 0: 9592.4, 1: 9962.5. Samples: 45108184. Policy #0 lag: (min: 31.0, avg: 32.4, max: 62.0) [2023-12-26 16:00:21,062][104569] Avg episode reward: [(0, '6028.804'), (1, '8651.064')] [2023-12-26 16:00:21,237][105692] Updated weights for policy 0, policy_version 87928 (0.0010) [2023-12-26 16:00:21,307][105692] Updated weights for policy 0, policy_version 87938 (0.0009) [2023-12-26 16:00:21,377][105692] Updated weights for policy 0, policy_version 87948 (0.0009) [2023-12-26 16:00:21,746][105620] Updated weights for policy 1, policy_version 88306 (0.0009) [2023-12-26 16:00:21,811][105620] Updated weights for policy 1, policy_version 88316 (0.0008) [2023-12-26 16:00:21,877][105620] Updated weights for policy 1, policy_version 88326 (0.0008) [2023-12-26 16:00:21,937][105620] Updated weights for policy 1, policy_version 88336 (0.0007) [2023-12-26 16:00:22,168][105692] Updated weights for policy 0, policy_version 87958 (0.0009) [2023-12-26 16:00:22,227][105692] Updated weights for policy 0, policy_version 87968 (0.0009) [2023-12-26 16:00:22,289][105692] Updated weights for policy 0, policy_version 87978 (0.0009) [2023-12-26 16:00:22,676][105620] Updated weights for policy 1, policy_version 88346 (0.0005) [2023-12-26 16:00:22,736][105620] Updated weights for policy 1, policy_version 88356 (0.0005) [2023-12-26 16:00:22,799][105620] Updated weights for policy 1, policy_version 88366 (0.0008) [2023-12-26 16:00:23,105][105692] Updated weights for policy 0, policy_version 87988 (0.0009) [2023-12-26 16:00:23,153][105692] Updated weights for policy 0, policy_version 87998 (0.0009) [2023-12-26 16:00:23,209][105692] Updated weights for policy 0, policy_version 88008 (0.0009) [2023-12-26 16:00:23,470][105620] Updated weights for policy 1, policy_version 88376 (0.0006) [2023-12-26 16:00:23,520][105620] Updated weights for policy 1, policy_version 88386 (0.0009) [2023-12-26 16:00:23,585][105620] Updated weights for policy 1, policy_version 88397 (0.0010) [2023-12-26 16:00:23,847][105692] Updated weights for policy 0, policy_version 88018 (0.0008) [2023-12-26 16:00:23,896][105692] Updated weights for policy 0, policy_version 88028 (0.0005) [2023-12-26 16:00:23,949][105692] Updated weights for policy 0, policy_version 88038 (0.0005) [2023-12-26 16:00:24,013][105692] Updated weights for policy 0, policy_version 88048 (0.0005) [2023-12-26 16:00:24,478][105620] Updated weights for policy 1, policy_version 88407 (0.0009) [2023-12-26 16:00:24,520][105692] Updated weights for policy 0, policy_version 88058 (0.0005) [2023-12-26 16:00:24,533][105620] Updated weights for policy 1, policy_version 88417 (0.0008) [2023-12-26 16:00:24,572][105692] Updated weights for policy 0, policy_version 88068 (0.0007) [2023-12-26 16:00:24,583][105620] Updated weights for policy 1, policy_version 88427 (0.0006) [2023-12-26 16:00:24,622][105692] Updated weights for policy 0, policy_version 88078 (0.0007) [2023-12-26 16:00:25,310][105692] Updated weights for policy 0, policy_version 88088 (0.0006) [2023-12-26 16:00:25,368][105692] Updated weights for policy 0, policy_version 88098 (0.0005) [2023-12-26 16:00:25,383][105620] Updated weights for policy 1, policy_version 88437 (0.0007) [2023-12-26 16:00:25,417][105692] Updated weights for policy 0, policy_version 88108 (0.0005) [2023-12-26 16:00:25,446][105620] Updated weights for policy 1, policy_version 88447 (0.0009) [2023-12-26 16:00:25,507][105620] Updated weights for policy 1, policy_version 88457 (0.0010) [2023-12-26 16:00:26,029][105692] Updated weights for policy 0, policy_version 88118 (0.0009) [2023-12-26 16:00:26,062][104569] Fps is (10 sec: 18022.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 45211648. Throughput: 0: 9569.7, 1: 9938.7. Samples: 45223848. Policy #0 lag: (min: 31.0, avg: 32.4, max: 62.0) [2023-12-26 16:00:26,063][104569] Avg episode reward: [(0, '6081.914'), (1, '8295.684')] [2023-12-26 16:00:26,092][105692] Updated weights for policy 0, policy_version 88128 (0.0008) [2023-12-26 16:00:26,148][105692] Updated weights for policy 0, policy_version 88138 (0.0009) [2023-12-26 16:00:26,295][105620] Updated weights for policy 1, policy_version 88467 (0.0009) [2023-12-26 16:00:26,347][105620] Updated weights for policy 1, policy_version 88477 (0.0009) [2023-12-26 16:00:26,406][105620] Updated weights for policy 1, policy_version 88487 (0.0010) [2023-12-26 16:00:26,879][105692] Updated weights for policy 0, policy_version 88148 (0.0009) [2023-12-26 16:00:26,937][105692] Updated weights for policy 0, policy_version 88158 (0.0009) [2023-12-26 16:00:26,998][105692] Updated weights for policy 0, policy_version 88168 (0.0009) [2023-12-26 16:00:27,150][105620] Updated weights for policy 1, policy_version 88497 (0.0010) [2023-12-26 16:00:27,208][105620] Updated weights for policy 1, policy_version 88507 (0.0009) [2023-12-26 16:00:27,269][105620] Updated weights for policy 1, policy_version 88517 (0.0009) [2023-12-26 16:00:27,323][105620] Updated weights for policy 1, policy_version 88527 (0.0008) [2023-12-26 16:00:27,735][105692] Updated weights for policy 0, policy_version 88178 (0.0008) [2023-12-26 16:00:27,791][105692] Updated weights for policy 0, policy_version 88188 (0.0008) [2023-12-26 16:00:27,852][105692] Updated weights for policy 0, policy_version 88198 (0.0009) [2023-12-26 16:00:27,915][105692] Updated weights for policy 0, policy_version 88208 (0.0009) [2023-12-26 16:00:28,082][105620] Updated weights for policy 1, policy_version 88537 (0.0009) [2023-12-26 16:00:28,131][105620] Updated weights for policy 1, policy_version 88547 (0.0008) [2023-12-26 16:00:28,178][105620] Updated weights for policy 1, policy_version 88557 (0.0008) [2023-12-26 16:00:28,636][105692] Updated weights for policy 0, policy_version 88218 (0.0010) [2023-12-26 16:00:28,702][105692] Updated weights for policy 0, policy_version 88228 (0.0010) [2023-12-26 16:00:28,759][105692] Updated weights for policy 0, policy_version 88238 (0.0009) [2023-12-26 16:00:28,861][105620] Updated weights for policy 1, policy_version 88567 (0.0008) [2023-12-26 16:00:28,929][105620] Updated weights for policy 1, policy_version 88577 (0.0008) [2023-12-26 16:00:28,987][105620] Updated weights for policy 1, policy_version 88587 (0.0008) [2023-12-26 16:00:29,611][105620] Updated weights for policy 1, policy_version 88597 (0.0008) [2023-12-26 16:00:29,628][105692] Updated weights for policy 0, policy_version 88248 (0.0008) [2023-12-26 16:00:29,672][105620] Updated weights for policy 1, policy_version 88607 (0.0010) [2023-12-26 16:00:29,690][105692] Updated weights for policy 0, policy_version 88258 (0.0006) [2023-12-26 16:00:29,725][105620] Updated weights for policy 1, policy_version 88617 (0.0010) [2023-12-26 16:00:29,749][105692] Updated weights for policy 0, policy_version 88268 (0.0006) [2023-12-26 16:00:30,422][105620] Updated weights for policy 1, policy_version 88627 (0.0011) [2023-12-26 16:00:30,477][105692] Updated weights for policy 0, policy_version 88278 (0.0008) [2023-12-26 16:00:30,486][105620] Updated weights for policy 1, policy_version 88637 (0.0009) [2023-12-26 16:00:30,534][105692] Updated weights for policy 0, policy_version 88288 (0.0009) [2023-12-26 16:00:30,543][105620] Updated weights for policy 1, policy_version 88647 (0.0009) [2023-12-26 16:00:30,582][105692] Updated weights for policy 0, policy_version 88298 (0.0008) [2023-12-26 16:00:31,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 45309952. Throughput: 0: 9523.4, 1: 9966.3. Samples: 45282284. Policy #0 lag: (min: 31.0, avg: 32.4, max: 62.0) [2023-12-26 16:00:31,063][104569] Avg episode reward: [(0, '6452.241'), (1, '8470.894')] [2023-12-26 16:00:31,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000088304_22609920.pth... [2023-12-26 16:00:31,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000088656_22700032.pth... [2023-12-26 16:00:31,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000087184_22323200.pth [2023-12-26 16:00:31,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000087504_22405120.pth [2023-12-26 16:00:31,298][105692] Updated weights for policy 0, policy_version 88308 (0.0008) [2023-12-26 16:00:31,330][105620] Updated weights for policy 1, policy_version 88657 (0.0009) [2023-12-26 16:00:31,350][105692] Updated weights for policy 0, policy_version 88318 (0.0008) [2023-12-26 16:00:31,388][105620] Updated weights for policy 1, policy_version 88667 (0.0007) [2023-12-26 16:00:31,417][105692] Updated weights for policy 0, policy_version 88328 (0.0008) [2023-12-26 16:00:31,458][105620] Updated weights for policy 1, policy_version 88677 (0.0007) [2023-12-26 16:00:31,510][105620] Updated weights for policy 1, policy_version 88687 (0.0008) [2023-12-26 16:00:32,205][105692] Updated weights for policy 0, policy_version 88338 (0.0009) [2023-12-26 16:00:32,251][105620] Updated weights for policy 1, policy_version 88697 (0.0011) [2023-12-26 16:00:32,264][105692] Updated weights for policy 0, policy_version 88348 (0.0011) [2023-12-26 16:00:32,312][105620] Updated weights for policy 1, policy_version 88707 (0.0010) [2023-12-26 16:00:32,324][105692] Updated weights for policy 0, policy_version 88358 (0.0011) [2023-12-26 16:00:32,378][105620] Updated weights for policy 1, policy_version 88717 (0.0010) [2023-12-26 16:00:32,390][105692] Updated weights for policy 0, policy_version 88368 (0.0011) [2023-12-26 16:00:33,090][105692] Updated weights for policy 0, policy_version 88378 (0.0005) [2023-12-26 16:00:33,155][105692] Updated weights for policy 0, policy_version 88388 (0.0007) [2023-12-26 16:00:33,184][105620] Updated weights for policy 1, policy_version 88727 (0.0006) [2023-12-26 16:00:33,206][105692] Updated weights for policy 0, policy_version 88398 (0.0010) [2023-12-26 16:00:33,243][105620] Updated weights for policy 1, policy_version 88737 (0.0006) [2023-12-26 16:00:33,310][105620] Updated weights for policy 1, policy_version 88747 (0.0006) [2023-12-26 16:00:33,912][105692] Updated weights for policy 0, policy_version 88408 (0.0010) [2023-12-26 16:00:33,949][105620] Updated weights for policy 1, policy_version 88757 (0.0007) [2023-12-26 16:00:33,966][105692] Updated weights for policy 0, policy_version 88418 (0.0009) [2023-12-26 16:00:34,004][105620] Updated weights for policy 1, policy_version 88767 (0.0006) [2023-12-26 16:00:34,030][105692] Updated weights for policy 0, policy_version 88428 (0.0009) [2023-12-26 16:00:34,060][105620] Updated weights for policy 1, policy_version 88777 (0.0008) [2023-12-26 16:00:34,777][105620] Updated weights for policy 1, policy_version 88787 (0.0010) [2023-12-26 16:00:34,807][105692] Updated weights for policy 0, policy_version 88438 (0.0006) [2023-12-26 16:00:34,829][105620] Updated weights for policy 1, policy_version 88797 (0.0010) [2023-12-26 16:00:34,872][105692] Updated weights for policy 0, policy_version 88448 (0.0006) [2023-12-26 16:00:34,882][105620] Updated weights for policy 1, policy_version 88807 (0.0011) [2023-12-26 16:00:34,938][105692] Updated weights for policy 0, policy_version 88458 (0.0006) [2023-12-26 16:00:35,570][105692] Updated weights for policy 0, policy_version 88468 (0.0007) [2023-12-26 16:00:35,620][105692] Updated weights for policy 0, policy_version 88478 (0.0008) [2023-12-26 16:00:35,648][105620] Updated weights for policy 1, policy_version 88817 (0.0010) [2023-12-26 16:00:35,667][105692] Updated weights for policy 0, policy_version 88488 (0.0008) [2023-12-26 16:00:35,706][105620] Updated weights for policy 1, policy_version 88827 (0.0010) [2023-12-26 16:00:35,763][105620] Updated weights for policy 1, policy_version 88837 (0.0010) [2023-12-26 16:00:35,821][105620] Updated weights for policy 1, policy_version 88847 (0.0010) [2023-12-26 16:00:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 45408256. Throughput: 0: 9503.6, 1: 9823.2. Samples: 45394564. Policy #0 lag: (min: 31.0, avg: 32.4, max: 62.0) [2023-12-26 16:00:36,062][104569] Avg episode reward: [(0, '7004.868'), (1, '8819.979')] [2023-12-26 16:00:36,441][105692] Updated weights for policy 0, policy_version 88498 (0.0007) [2023-12-26 16:00:36,497][105692] Updated weights for policy 0, policy_version 88508 (0.0008) [2023-12-26 16:00:36,560][105692] Updated weights for policy 0, policy_version 88518 (0.0007) [2023-12-26 16:00:36,564][105620] Updated weights for policy 1, policy_version 88857 (0.0011) [2023-12-26 16:00:36,620][105692] Updated weights for policy 0, policy_version 88528 (0.0009) [2023-12-26 16:00:36,624][105620] Updated weights for policy 1, policy_version 88867 (0.0010) [2023-12-26 16:00:36,672][105620] Updated weights for policy 1, policy_version 88877 (0.0010) [2023-12-26 16:00:37,408][105692] Updated weights for policy 0, policy_version 88538 (0.0009) [2023-12-26 16:00:37,431][105620] Updated weights for policy 1, policy_version 88887 (0.0009) [2023-12-26 16:00:37,464][105692] Updated weights for policy 0, policy_version 88548 (0.0009) [2023-12-26 16:00:37,490][105620] Updated weights for policy 1, policy_version 88897 (0.0010) [2023-12-26 16:00:37,522][105692] Updated weights for policy 0, policy_version 88558 (0.0006) [2023-12-26 16:00:37,558][105620] Updated weights for policy 1, policy_version 88907 (0.0011) [2023-12-26 16:00:38,247][105692] Updated weights for policy 0, policy_version 88568 (0.0008) [2023-12-26 16:00:38,254][105620] Updated weights for policy 1, policy_version 88917 (0.0011) [2023-12-26 16:00:38,310][105692] Updated weights for policy 0, policy_version 88578 (0.0008) [2023-12-26 16:00:38,318][105620] Updated weights for policy 1, policy_version 88927 (0.0008) [2023-12-26 16:00:38,376][105692] Updated weights for policy 0, policy_version 88588 (0.0008) [2023-12-26 16:00:38,386][105620] Updated weights for policy 1, policy_version 88937 (0.0011) [2023-12-26 16:00:39,159][105692] Updated weights for policy 0, policy_version 88598 (0.0008) [2023-12-26 16:00:39,209][105620] Updated weights for policy 1, policy_version 88947 (0.0007) [2023-12-26 16:00:39,228][105692] Updated weights for policy 0, policy_version 88608 (0.0008) [2023-12-26 16:00:39,275][105620] Updated weights for policy 1, policy_version 88957 (0.0009) [2023-12-26 16:00:39,283][105692] Updated weights for policy 0, policy_version 88618 (0.0007) [2023-12-26 16:00:39,336][105620] Updated weights for policy 1, policy_version 88967 (0.0010) [2023-12-26 16:00:40,054][105620] Updated weights for policy 1, policy_version 88977 (0.0008) [2023-12-26 16:00:40,071][105692] Updated weights for policy 0, policy_version 88628 (0.0008) [2023-12-26 16:00:40,121][105620] Updated weights for policy 1, policy_version 88987 (0.0007) [2023-12-26 16:00:40,133][105692] Updated weights for policy 0, policy_version 88638 (0.0008) [2023-12-26 16:00:40,182][105620] Updated weights for policy 1, policy_version 88997 (0.0011) [2023-12-26 16:00:40,194][105692] Updated weights for policy 0, policy_version 88648 (0.0011) [2023-12-26 16:00:40,236][105620] Updated weights for policy 1, policy_version 89007 (0.0009) [2023-12-26 16:00:40,863][105692] Updated weights for policy 0, policy_version 88658 (0.0008) [2023-12-26 16:00:40,927][105692] Updated weights for policy 0, policy_version 88668 (0.0008) [2023-12-26 16:00:40,984][105692] Updated weights for policy 0, policy_version 88678 (0.0011) [2023-12-26 16:00:41,049][105620] Updated weights for policy 1, policy_version 89017 (0.0008) [2023-12-26 16:00:41,055][105692] Updated weights for policy 0, policy_version 88688 (0.0010) [2023-12-26 16:00:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 45498368. Throughput: 0: 9536.0, 1: 9729.5. Samples: 45507220. Policy #0 lag: (min: 17.0, avg: 39.9, max: 49.0) [2023-12-26 16:00:41,063][104569] Avg episode reward: [(0, '7393.931'), (1, '8910.530')] [2023-12-26 16:00:41,115][105620] Updated weights for policy 1, policy_version 89027 (0.0009) [2023-12-26 16:00:41,184][105620] Updated weights for policy 1, policy_version 89037 (0.0008) [2023-12-26 16:00:41,852][105692] Updated weights for policy 0, policy_version 88698 (0.0008) [2023-12-26 16:00:41,898][105620] Updated weights for policy 1, policy_version 89047 (0.0008) [2023-12-26 16:00:41,905][105692] Updated weights for policy 0, policy_version 88708 (0.0006) [2023-12-26 16:00:41,954][105620] Updated weights for policy 1, policy_version 89057 (0.0008) [2023-12-26 16:00:41,960][105692] Updated weights for policy 0, policy_version 88718 (0.0007) [2023-12-26 16:00:42,010][105620] Updated weights for policy 1, policy_version 89067 (0.0008) [2023-12-26 16:00:42,687][105620] Updated weights for policy 1, policy_version 89077 (0.0007) [2023-12-26 16:00:42,745][105620] Updated weights for policy 1, policy_version 89087 (0.0009) [2023-12-26 16:00:42,792][105692] Updated weights for policy 0, policy_version 88728 (0.0007) [2023-12-26 16:00:42,810][105620] Updated weights for policy 1, policy_version 89097 (0.0011) [2023-12-26 16:00:42,856][105692] Updated weights for policy 0, policy_version 88738 (0.0007) [2023-12-26 16:00:42,927][105692] Updated weights for policy 0, policy_version 88748 (0.0008) [2023-12-26 16:00:43,446][105620] Updated weights for policy 1, policy_version 89107 (0.0008) [2023-12-26 16:00:43,496][105620] Updated weights for policy 1, policy_version 89117 (0.0005) [2023-12-26 16:00:43,547][105620] Updated weights for policy 1, policy_version 89127 (0.0005) [2023-12-26 16:00:43,700][105692] Updated weights for policy 0, policy_version 88758 (0.0008) [2023-12-26 16:00:43,754][105692] Updated weights for policy 0, policy_version 88768 (0.0010) [2023-12-26 16:00:43,808][105692] Updated weights for policy 0, policy_version 88778 (0.0010) [2023-12-26 16:00:44,170][105620] Updated weights for policy 1, policy_version 89137 (0.0010) [2023-12-26 16:00:44,232][105620] Updated weights for policy 1, policy_version 89147 (0.0005) [2023-12-26 16:00:44,300][105620] Updated weights for policy 1, policy_version 89157 (0.0006) [2023-12-26 16:00:44,363][105620] Updated weights for policy 1, policy_version 89167 (0.0005) [2023-12-26 16:00:44,472][105692] Updated weights for policy 0, policy_version 88788 (0.0009) [2023-12-26 16:00:44,530][105692] Updated weights for policy 0, policy_version 88798 (0.0009) [2023-12-26 16:00:44,588][105692] Updated weights for policy 0, policy_version 88808 (0.0010) [2023-12-26 16:00:45,008][105620] Updated weights for policy 1, policy_version 89177 (0.0009) [2023-12-26 16:00:45,074][105620] Updated weights for policy 1, policy_version 89187 (0.0008) [2023-12-26 16:00:45,136][105620] Updated weights for policy 1, policy_version 89197 (0.0008) [2023-12-26 16:00:45,305][105692] Updated weights for policy 0, policy_version 88818 (0.0009) [2023-12-26 16:00:45,363][105692] Updated weights for policy 0, policy_version 88828 (0.0010) [2023-12-26 16:00:45,418][105692] Updated weights for policy 0, policy_version 88838 (0.0010) [2023-12-26 16:00:45,481][105692] Updated weights for policy 0, policy_version 88848 (0.0010) [2023-12-26 16:00:45,979][105620] Updated weights for policy 1, policy_version 89207 (0.0010) [2023-12-26 16:00:46,037][105620] Updated weights for policy 1, policy_version 89217 (0.0010) [2023-12-26 16:00:46,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19251.3, 300 sec: 19438.6). Total num frames: 45588480. Throughput: 0: 9514.1, 1: 9753.9. Samples: 45563988. Policy #0 lag: (min: 17.0, avg: 39.9, max: 49.0) [2023-12-26 16:00:46,062][104569] Avg episode reward: [(0, '6566.366'), (1, '9180.612')] [2023-12-26 16:00:46,090][105692] Updated weights for policy 0, policy_version 88858 (0.0009) [2023-12-26 16:00:46,092][105620] Updated weights for policy 1, policy_version 89227 (0.0007) [2023-12-26 16:00:46,119][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000089232_22847488.pth... [2023-12-26 16:00:46,124][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000088112_22560768.pth [2023-12-26 16:00:46,138][105692] Updated weights for policy 0, policy_version 88868 (0.0010) [2023-12-26 16:00:46,189][105692] Updated weights for policy 0, policy_version 88878 (0.0010) [2023-12-26 16:00:46,196][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000088880_22757376.pth... [2023-12-26 16:00:46,201][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000087760_22470656.pth [2023-12-26 16:00:46,771][105692] Updated weights for policy 0, policy_version 88888 (0.0010) [2023-12-26 16:00:46,818][105692] Updated weights for policy 0, policy_version 88898 (0.0010) [2023-12-26 16:00:46,842][105620] Updated weights for policy 1, policy_version 89237 (0.0005) [2023-12-26 16:00:46,866][105692] Updated weights for policy 0, policy_version 88908 (0.0010) [2023-12-26 16:00:46,891][105620] Updated weights for policy 1, policy_version 89247 (0.0007) [2023-12-26 16:00:46,943][105620] Updated weights for policy 1, policy_version 89257 (0.0006) [2023-12-26 16:00:47,572][105620] Updated weights for policy 1, policy_version 89267 (0.0008) [2023-12-26 16:00:47,574][105692] Updated weights for policy 0, policy_version 88918 (0.0010) [2023-12-26 16:00:47,623][105620] Updated weights for policy 1, policy_version 89277 (0.0010) [2023-12-26 16:00:47,628][105692] Updated weights for policy 0, policy_version 88928 (0.0010) [2023-12-26 16:00:47,674][105620] Updated weights for policy 1, policy_version 89287 (0.0010) [2023-12-26 16:00:47,679][105692] Updated weights for policy 0, policy_version 88938 (0.0010) [2023-12-26 16:00:48,251][105692] Updated weights for policy 0, policy_version 88948 (0.0010) [2023-12-26 16:00:48,312][105692] Updated weights for policy 0, policy_version 88958 (0.0010) [2023-12-26 16:00:48,323][105620] Updated weights for policy 1, policy_version 89297 (0.0010) [2023-12-26 16:00:48,375][105692] Updated weights for policy 0, policy_version 88968 (0.0008) [2023-12-26 16:00:48,385][105620] Updated weights for policy 1, policy_version 89307 (0.0008) [2023-12-26 16:00:48,444][105620] Updated weights for policy 1, policy_version 89317 (0.0007) [2023-12-26 16:00:48,500][105620] Updated weights for policy 1, policy_version 89327 (0.0009) [2023-12-26 16:00:49,134][105620] Updated weights for policy 1, policy_version 89337 (0.0009) [2023-12-26 16:00:49,172][105692] Updated weights for policy 0, policy_version 88978 (0.0009) [2023-12-26 16:00:49,190][105620] Updated weights for policy 1, policy_version 89347 (0.0008) [2023-12-26 16:00:49,235][105692] Updated weights for policy 0, policy_version 88988 (0.0008) [2023-12-26 16:00:49,248][105620] Updated weights for policy 1, policy_version 89357 (0.0007) [2023-12-26 16:00:49,299][105692] Updated weights for policy 0, policy_version 88999 (0.0010) [2023-12-26 16:00:49,987][105620] Updated weights for policy 1, policy_version 89367 (0.0008) [2023-12-26 16:00:50,051][105620] Updated weights for policy 1, policy_version 89377 (0.0009) [2023-12-26 16:00:50,093][105692] Updated weights for policy 0, policy_version 89009 (0.0010) [2023-12-26 16:00:50,115][105620] Updated weights for policy 1, policy_version 89387 (0.0008) [2023-12-26 16:00:50,142][105692] Updated weights for policy 0, policy_version 89019 (0.0006) [2023-12-26 16:00:50,207][105692] Updated weights for policy 0, policy_version 89029 (0.0009) [2023-12-26 16:00:50,268][105692] Updated weights for policy 0, policy_version 89039 (0.0009) [2023-12-26 16:00:50,864][105620] Updated weights for policy 1, policy_version 89397 (0.0008) [2023-12-26 16:00:50,914][105620] Updated weights for policy 1, policy_version 89407 (0.0006) [2023-12-26 16:00:50,982][105620] Updated weights for policy 1, policy_version 89417 (0.0005) [2023-12-26 16:00:51,046][105692] Updated weights for policy 0, policy_version 89049 (0.0011) [2023-12-26 16:00:51,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 45694976. Throughput: 0: 9610.6, 1: 9705.2. Samples: 45685792. Policy #0 lag: (min: 17.0, avg: 39.9, max: 49.0) [2023-12-26 16:00:51,062][104569] Avg episode reward: [(0, '6171.446'), (1, '8908.352')] [2023-12-26 16:00:51,099][105692] Updated weights for policy 0, policy_version 89059 (0.0011) [2023-12-26 16:00:51,164][105692] Updated weights for policy 0, policy_version 89069 (0.0009) [2023-12-26 16:00:51,691][105620] Updated weights for policy 1, policy_version 89427 (0.0007) [2023-12-26 16:00:51,755][105620] Updated weights for policy 1, policy_version 89437 (0.0011) [2023-12-26 16:00:51,807][105620] Updated weights for policy 1, policy_version 89447 (0.0010) [2023-12-26 16:00:51,980][105692] Updated weights for policy 0, policy_version 89079 (0.0008) [2023-12-26 16:00:52,036][105692] Updated weights for policy 0, policy_version 89089 (0.0009) [2023-12-26 16:00:52,095][105692] Updated weights for policy 0, policy_version 89099 (0.0008) [2023-12-26 16:00:52,566][105620] Updated weights for policy 1, policy_version 89457 (0.0010) [2023-12-26 16:00:52,635][105620] Updated weights for policy 1, policy_version 89467 (0.0010) [2023-12-26 16:00:52,697][105620] Updated weights for policy 1, policy_version 89477 (0.0010) [2023-12-26 16:00:52,753][105620] Updated weights for policy 1, policy_version 89487 (0.0010) [2023-12-26 16:00:52,836][105692] Updated weights for policy 0, policy_version 89109 (0.0009) [2023-12-26 16:00:52,903][105692] Updated weights for policy 0, policy_version 89119 (0.0010) [2023-12-26 16:00:52,968][105692] Updated weights for policy 0, policy_version 89129 (0.0009) [2023-12-26 16:00:53,483][105620] Updated weights for policy 1, policy_version 89497 (0.0010) [2023-12-26 16:00:53,537][105620] Updated weights for policy 1, policy_version 89507 (0.0009) [2023-12-26 16:00:53,543][105692] Updated weights for policy 0, policy_version 89139 (0.0005) [2023-12-26 16:00:53,592][105692] Updated weights for policy 0, policy_version 89149 (0.0005) [2023-12-26 16:00:53,593][105620] Updated weights for policy 1, policy_version 89517 (0.0010) [2023-12-26 16:00:53,642][105692] Updated weights for policy 0, policy_version 89159 (0.0005) [2023-12-26 16:00:54,227][105692] Updated weights for policy 0, policy_version 89169 (0.0007) [2023-12-26 16:00:54,246][105620] Updated weights for policy 1, policy_version 89527 (0.0007) [2023-12-26 16:00:54,278][105692] Updated weights for policy 0, policy_version 89179 (0.0008) [2023-12-26 16:00:54,315][105620] Updated weights for policy 1, policy_version 89537 (0.0006) [2023-12-26 16:00:54,324][105692] Updated weights for policy 0, policy_version 89189 (0.0008) [2023-12-26 16:00:54,378][105692] Updated weights for policy 0, policy_version 89199 (0.0008) [2023-12-26 16:00:54,384][105620] Updated weights for policy 1, policy_version 89547 (0.0005) [2023-12-26 16:00:54,909][105620] Updated weights for policy 1, policy_version 89557 (0.0009) [2023-12-26 16:00:54,963][105620] Updated weights for policy 1, policy_version 89567 (0.0011) [2023-12-26 16:00:55,022][105620] Updated weights for policy 1, policy_version 89577 (0.0010) [2023-12-26 16:00:55,052][105692] Updated weights for policy 0, policy_version 89209 (0.0007) [2023-12-26 16:00:55,104][105692] Updated weights for policy 0, policy_version 89219 (0.0005) [2023-12-26 16:00:55,150][105692] Updated weights for policy 0, policy_version 89229 (0.0005) [2023-12-26 16:00:55,707][105620] Updated weights for policy 1, policy_version 89587 (0.0010) [2023-12-26 16:00:55,758][105620] Updated weights for policy 1, policy_version 89597 (0.0010) [2023-12-26 16:00:55,820][105620] Updated weights for policy 1, policy_version 89607 (0.0010) [2023-12-26 16:00:55,890][105692] Updated weights for policy 0, policy_version 89239 (0.0006) [2023-12-26 16:00:55,943][105692] Updated weights for policy 0, policy_version 89249 (0.0006) [2023-12-26 16:00:55,995][105692] Updated weights for policy 0, policy_version 89259 (0.0006) [2023-12-26 16:00:56,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 45801472. Throughput: 0: 9627.8, 1: 9700.9. Samples: 45804956. Policy #0 lag: (min: 17.0, avg: 39.9, max: 49.0) [2023-12-26 16:00:56,062][104569] Avg episode reward: [(0, '6120.319'), (1, '8281.536')] [2023-12-26 16:00:56,534][105620] Updated weights for policy 1, policy_version 89617 (0.0008) [2023-12-26 16:00:56,582][105620] Updated weights for policy 1, policy_version 89627 (0.0006) [2023-12-26 16:00:56,594][105692] Updated weights for policy 0, policy_version 89269 (0.0008) [2023-12-26 16:00:56,628][105620] Updated weights for policy 1, policy_version 89637 (0.0005) [2023-12-26 16:00:56,652][105692] Updated weights for policy 0, policy_version 89279 (0.0005) [2023-12-26 16:00:56,680][105620] Updated weights for policy 1, policy_version 89647 (0.0005) [2023-12-26 16:00:56,715][105692] Updated weights for policy 0, policy_version 89289 (0.0005) [2023-12-26 16:00:57,222][105692] Updated weights for policy 0, policy_version 89299 (0.0006) [2023-12-26 16:00:57,236][105620] Updated weights for policy 1, policy_version 89657 (0.0005) [2023-12-26 16:00:57,279][105692] Updated weights for policy 0, policy_version 89309 (0.0009) [2023-12-26 16:00:57,293][105620] Updated weights for policy 1, policy_version 89667 (0.0005) [2023-12-26 16:00:57,333][105692] Updated weights for policy 0, policy_version 89319 (0.0006) [2023-12-26 16:00:57,353][105620] Updated weights for policy 1, policy_version 89677 (0.0007) [2023-12-26 16:00:57,959][105692] Updated weights for policy 0, policy_version 89329 (0.0008) [2023-12-26 16:00:58,010][105620] Updated weights for policy 1, policy_version 89687 (0.0008) [2023-12-26 16:00:58,011][105692] Updated weights for policy 0, policy_version 89339 (0.0006) [2023-12-26 16:00:58,062][105620] Updated weights for policy 1, policy_version 89697 (0.0010) [2023-12-26 16:00:58,066][105692] Updated weights for policy 0, policy_version 89349 (0.0007) [2023-12-26 16:00:58,120][105620] Updated weights for policy 1, policy_version 89707 (0.0010) [2023-12-26 16:00:58,125][105692] Updated weights for policy 0, policy_version 89359 (0.0009) [2023-12-26 16:00:58,910][105692] Updated weights for policy 0, policy_version 89369 (0.0008) [2023-12-26 16:00:58,963][105692] Updated weights for policy 0, policy_version 89379 (0.0007) [2023-12-26 16:00:58,968][105620] Updated weights for policy 1, policy_version 89717 (0.0010) [2023-12-26 16:00:59,023][105692] Updated weights for policy 0, policy_version 89389 (0.0007) [2023-12-26 16:00:59,027][105620] Updated weights for policy 1, policy_version 89727 (0.0007) [2023-12-26 16:00:59,084][105620] Updated weights for policy 1, policy_version 89737 (0.0010) [2023-12-26 16:00:59,728][105692] Updated weights for policy 0, policy_version 89399 (0.0007) [2023-12-26 16:00:59,788][105692] Updated weights for policy 0, policy_version 89409 (0.0009) [2023-12-26 16:00:59,800][105620] Updated weights for policy 1, policy_version 89747 (0.0010) [2023-12-26 16:00:59,846][105692] Updated weights for policy 0, policy_version 89419 (0.0006) [2023-12-26 16:00:59,864][105620] Updated weights for policy 1, policy_version 89757 (0.0007) [2023-12-26 16:00:59,923][105620] Updated weights for policy 1, policy_version 89767 (0.0008) [2023-12-26 16:01:00,580][105692] Updated weights for policy 0, policy_version 89429 (0.0009) [2023-12-26 16:01:00,624][105692] Updated weights for policy 0, policy_version 89439 (0.0010) [2023-12-26 16:01:00,672][105692] Updated weights for policy 0, policy_version 89449 (0.0006) [2023-12-26 16:01:00,708][105620] Updated weights for policy 1, policy_version 89777 (0.0008) [2023-12-26 16:01:00,755][105620] Updated weights for policy 1, policy_version 89787 (0.0008) [2023-12-26 16:01:00,800][105620] Updated weights for policy 1, policy_version 89797 (0.0006) [2023-12-26 16:01:00,856][105620] Updated weights for policy 1, policy_version 89807 (0.0006) [2023-12-26 16:01:01,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 45899776. Throughput: 0: 9722.4, 1: 9729.1. Samples: 45868536. Policy #0 lag: (min: 17.0, avg: 39.9, max: 49.0) [2023-12-26 16:01:01,063][104569] Avg episode reward: [(0, '6738.649'), (1, '8190.647')] [2023-12-26 16:01:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000089808_22994944.pth... [2023-12-26 16:01:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000089456_22904832.pth... [2023-12-26 16:01:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000088656_22700032.pth [2023-12-26 16:01:01,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000088304_22609920.pth [2023-12-26 16:01:01,436][105692] Updated weights for policy 0, policy_version 89459 (0.0010) [2023-12-26 16:01:01,498][105692] Updated weights for policy 0, policy_version 89469 (0.0010) [2023-12-26 16:01:01,546][105692] Updated weights for policy 0, policy_version 89479 (0.0010) [2023-12-26 16:01:01,601][105620] Updated weights for policy 1, policy_version 89817 (0.0006) [2023-12-26 16:01:01,667][105620] Updated weights for policy 1, policy_version 89827 (0.0010) [2023-12-26 16:01:01,731][105620] Updated weights for policy 1, policy_version 89837 (0.0010) [2023-12-26 16:01:02,227][105692] Updated weights for policy 0, policy_version 89489 (0.0010) [2023-12-26 16:01:02,292][105692] Updated weights for policy 0, policy_version 89499 (0.0008) [2023-12-26 16:01:02,355][105692] Updated weights for policy 0, policy_version 89509 (0.0008) [2023-12-26 16:01:02,409][105692] Updated weights for policy 0, policy_version 89519 (0.0007) [2023-12-26 16:01:02,426][105620] Updated weights for policy 1, policy_version 89847 (0.0010) [2023-12-26 16:01:02,486][105620] Updated weights for policy 1, policy_version 89857 (0.0010) [2023-12-26 16:01:02,533][105620] Updated weights for policy 1, policy_version 89867 (0.0009) [2023-12-26 16:01:03,077][105692] Updated weights for policy 0, policy_version 89529 (0.0008) [2023-12-26 16:01:03,137][105692] Updated weights for policy 0, policy_version 89539 (0.0008) [2023-12-26 16:01:03,190][105692] Updated weights for policy 0, policy_version 89549 (0.0008) [2023-12-26 16:01:03,347][105620] Updated weights for policy 1, policy_version 89877 (0.0009) [2023-12-26 16:01:03,400][105620] Updated weights for policy 1, policy_version 89887 (0.0009) [2023-12-26 16:01:03,445][105620] Updated weights for policy 1, policy_version 89897 (0.0008) [2023-12-26 16:01:03,813][105692] Updated weights for policy 0, policy_version 89559 (0.0006) [2023-12-26 16:01:03,879][105692] Updated weights for policy 0, policy_version 89569 (0.0007) [2023-12-26 16:01:03,939][105692] Updated weights for policy 0, policy_version 89579 (0.0007) [2023-12-26 16:01:04,224][105620] Updated weights for policy 1, policy_version 89907 (0.0009) [2023-12-26 16:01:04,288][105620] Updated weights for policy 1, policy_version 89917 (0.0008) [2023-12-26 16:01:04,348][105620] Updated weights for policy 1, policy_version 89927 (0.0008) [2023-12-26 16:01:04,609][105692] Updated weights for policy 0, policy_version 89589 (0.0008) [2023-12-26 16:01:04,670][105692] Updated weights for policy 0, policy_version 89599 (0.0006) [2023-12-26 16:01:04,732][105692] Updated weights for policy 0, policy_version 89609 (0.0007) [2023-12-26 16:01:05,061][105620] Updated weights for policy 1, policy_version 89937 (0.0008) [2023-12-26 16:01:05,130][105620] Updated weights for policy 1, policy_version 89947 (0.0008) [2023-12-26 16:01:05,192][105620] Updated weights for policy 1, policy_version 89957 (0.0008) [2023-12-26 16:01:05,250][105620] Updated weights for policy 1, policy_version 89967 (0.0008) [2023-12-26 16:01:05,448][105692] Updated weights for policy 0, policy_version 89619 (0.0009) [2023-12-26 16:01:05,499][105692] Updated weights for policy 0, policy_version 89629 (0.0010) [2023-12-26 16:01:05,546][105692] Updated weights for policy 0, policy_version 89639 (0.0010) [2023-12-26 16:01:05,984][105620] Updated weights for policy 1, policy_version 89977 (0.0009) [2023-12-26 16:01:06,036][105620] Updated weights for policy 1, policy_version 89987 (0.0008) [2023-12-26 16:01:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.3, 300 sec: 19494.2). Total num frames: 45989888. Throughput: 0: 9806.2, 1: 9668.4. Samples: 45984544. Policy #0 lag: (min: 17.0, avg: 39.9, max: 49.0) [2023-12-26 16:01:06,062][104569] Avg episode reward: [(0, '7095.456'), (1, '8729.105')] [2023-12-26 16:01:06,102][105620] Updated weights for policy 1, policy_version 89997 (0.0008) [2023-12-26 16:01:06,157][105692] Updated weights for policy 0, policy_version 89649 (0.0010) [2023-12-26 16:01:06,213][105692] Updated weights for policy 0, policy_version 89659 (0.0008) [2023-12-26 16:01:06,279][105692] Updated weights for policy 0, policy_version 89669 (0.0010) [2023-12-26 16:01:06,337][105692] Updated weights for policy 0, policy_version 89679 (0.0010) [2023-12-26 16:01:06,913][105620] Updated weights for policy 1, policy_version 90007 (0.0008) [2023-12-26 16:01:06,961][105692] Updated weights for policy 0, policy_version 89689 (0.0010) [2023-12-26 16:01:06,967][105620] Updated weights for policy 1, policy_version 90017 (0.0008) [2023-12-26 16:01:07,018][105620] Updated weights for policy 1, policy_version 90027 (0.0006) [2023-12-26 16:01:07,020][105692] Updated weights for policy 0, policy_version 89699 (0.0010) [2023-12-26 16:01:07,072][105692] Updated weights for policy 0, policy_version 89709 (0.0010) [2023-12-26 16:01:07,719][105620] Updated weights for policy 1, policy_version 90037 (0.0008) [2023-12-26 16:01:07,774][105620] Updated weights for policy 1, policy_version 90047 (0.0010) [2023-12-26 16:01:07,821][105692] Updated weights for policy 0, policy_version 89719 (0.0010) [2023-12-26 16:01:07,831][105620] Updated weights for policy 1, policy_version 90057 (0.0008) [2023-12-26 16:01:07,882][105692] Updated weights for policy 0, policy_version 89729 (0.0010) [2023-12-26 16:01:07,933][105692] Updated weights for policy 0, policy_version 89739 (0.0010) [2023-12-26 16:01:08,520][105620] Updated weights for policy 1, policy_version 90067 (0.0007) [2023-12-26 16:01:08,578][105620] Updated weights for policy 1, policy_version 90077 (0.0010) [2023-12-26 16:01:08,644][105620] Updated weights for policy 1, policy_version 90087 (0.0009) [2023-12-26 16:01:08,698][105692] Updated weights for policy 0, policy_version 89749 (0.0008) [2023-12-26 16:01:08,758][105692] Updated weights for policy 0, policy_version 89759 (0.0005) [2023-12-26 16:01:08,816][105692] Updated weights for policy 0, policy_version 89769 (0.0005) [2023-12-26 16:01:09,353][105620] Updated weights for policy 1, policy_version 90097 (0.0009) [2023-12-26 16:01:09,422][105620] Updated weights for policy 1, policy_version 90107 (0.0010) [2023-12-26 16:01:09,446][105692] Updated weights for policy 0, policy_version 89779 (0.0006) [2023-12-26 16:01:09,487][105620] Updated weights for policy 1, policy_version 90117 (0.0011) [2023-12-26 16:01:09,502][105692] Updated weights for policy 0, policy_version 89789 (0.0006) [2023-12-26 16:01:09,544][105620] Updated weights for policy 1, policy_version 90127 (0.0007) [2023-12-26 16:01:09,559][105692] Updated weights for policy 0, policy_version 89799 (0.0009) [2023-12-26 16:01:10,223][105620] Updated weights for policy 1, policy_version 90137 (0.0010) [2023-12-26 16:01:10,267][105620] Updated weights for policy 1, policy_version 90147 (0.0010) [2023-12-26 16:01:10,285][105692] Updated weights for policy 0, policy_version 89809 (0.0010) [2023-12-26 16:01:10,318][105620] Updated weights for policy 1, policy_version 90157 (0.0011) [2023-12-26 16:01:10,344][105692] Updated weights for policy 0, policy_version 89819 (0.0009) [2023-12-26 16:01:10,407][105692] Updated weights for policy 0, policy_version 89829 (0.0008) [2023-12-26 16:01:10,475][105692] Updated weights for policy 0, policy_version 89839 (0.0009) [2023-12-26 16:01:10,924][105620] Updated weights for policy 1, policy_version 90167 (0.0007) [2023-12-26 16:01:10,972][105620] Updated weights for policy 1, policy_version 90177 (0.0005) [2023-12-26 16:01:11,047][105620] Updated weights for policy 1, policy_version 90187 (0.0006) [2023-12-26 16:01:11,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 46088192. Throughput: 0: 9807.1, 1: 9738.8. Samples: 46103412. Policy #0 lag: (min: 17.0, avg: 39.9, max: 49.0) [2023-12-26 16:01:11,062][104569] Avg episode reward: [(0, '7107.699'), (1, '9182.268')] [2023-12-26 16:01:11,197][105692] Updated weights for policy 0, policy_version 89849 (0.0010) [2023-12-26 16:01:11,251][105692] Updated weights for policy 0, policy_version 89859 (0.0010) [2023-12-26 16:01:11,321][105692] Updated weights for policy 0, policy_version 89869 (0.0007) [2023-12-26 16:01:11,749][105620] Updated weights for policy 1, policy_version 90197 (0.0010) [2023-12-26 16:01:11,812][105620] Updated weights for policy 1, policy_version 90207 (0.0011) [2023-12-26 16:01:11,872][105620] Updated weights for policy 1, policy_version 90217 (0.0010) [2023-12-26 16:01:11,985][105692] Updated weights for policy 0, policy_version 89879 (0.0009) [2023-12-26 16:01:12,042][105692] Updated weights for policy 0, policy_version 89889 (0.0009) [2023-12-26 16:01:12,102][105692] Updated weights for policy 0, policy_version 89899 (0.0009) [2023-12-26 16:01:12,602][105620] Updated weights for policy 1, policy_version 90227 (0.0009) [2023-12-26 16:01:12,647][105620] Updated weights for policy 1, policy_version 90237 (0.0007) [2023-12-26 16:01:12,694][105620] Updated weights for policy 1, policy_version 90247 (0.0005) [2023-12-26 16:01:12,888][105692] Updated weights for policy 0, policy_version 89909 (0.0007) [2023-12-26 16:01:12,941][105692] Updated weights for policy 0, policy_version 89919 (0.0005) [2023-12-26 16:01:12,990][105692] Updated weights for policy 0, policy_version 89929 (0.0005) [2023-12-26 16:01:13,386][105620] Updated weights for policy 1, policy_version 90257 (0.0006) [2023-12-26 16:01:13,440][105620] Updated weights for policy 1, policy_version 90267 (0.0009) [2023-12-26 16:01:13,487][105620] Updated weights for policy 1, policy_version 90277 (0.0009) [2023-12-26 16:01:13,533][105620] Updated weights for policy 1, policy_version 90287 (0.0008) [2023-12-26 16:01:13,615][105692] Updated weights for policy 0, policy_version 89939 (0.0007) [2023-12-26 16:01:13,668][105692] Updated weights for policy 0, policy_version 89950 (0.0010) [2023-12-26 16:01:13,723][105692] Updated weights for policy 0, policy_version 89962 (0.0011) [2023-12-26 16:01:14,204][105620] Updated weights for policy 1, policy_version 90297 (0.0009) [2023-12-26 16:01:14,264][105620] Updated weights for policy 1, policy_version 90307 (0.0008) [2023-12-26 16:01:14,329][105620] Updated weights for policy 1, policy_version 90317 (0.0009) [2023-12-26 16:01:14,512][105692] Updated weights for policy 0, policy_version 89972 (0.0010) [2023-12-26 16:01:14,566][105692] Updated weights for policy 0, policy_version 89982 (0.0009) [2023-12-26 16:01:14,622][105692] Updated weights for policy 0, policy_version 89992 (0.0009) [2023-12-26 16:01:14,999][105620] Updated weights for policy 1, policy_version 90327 (0.0006) [2023-12-26 16:01:15,051][105620] Updated weights for policy 1, policy_version 90337 (0.0008) [2023-12-26 16:01:15,100][105620] Updated weights for policy 1, policy_version 90347 (0.0009) [2023-12-26 16:01:15,421][105692] Updated weights for policy 0, policy_version 90002 (0.0008) [2023-12-26 16:01:15,481][105692] Updated weights for policy 0, policy_version 90012 (0.0005) [2023-12-26 16:01:15,558][105692] Updated weights for policy 0, policy_version 90022 (0.0010) [2023-12-26 16:01:15,619][105692] Updated weights for policy 0, policy_version 90032 (0.0010) [2023-12-26 16:01:15,778][105620] Updated weights for policy 1, policy_version 90357 (0.0008) [2023-12-26 16:01:15,825][105620] Updated weights for policy 1, policy_version 90367 (0.0009) [2023-12-26 16:01:15,872][105620] Updated weights for policy 1, policy_version 90377 (0.0008) [2023-12-26 16:01:16,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 46194688. Throughput: 0: 9831.5, 1: 9764.0. Samples: 46164080. Policy #0 lag: (min: 17.0, avg: 39.9, max: 49.0) [2023-12-26 16:01:16,063][104569] Avg episode reward: [(0, '7564.956'), (1, '9093.829')] [2023-12-26 16:01:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000090032_23052288.pth... [2023-12-26 16:01:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000090384_23142400.pth... [2023-12-26 16:01:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000088880_22757376.pth [2023-12-26 16:01:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000089232_22847488.pth [2023-12-26 16:01:16,278][105692] Updated weights for policy 0, policy_version 90042 (0.0008) [2023-12-26 16:01:16,336][105692] Updated weights for policy 0, policy_version 90053 (0.0010) [2023-12-26 16:01:16,397][105692] Updated weights for policy 0, policy_version 90063 (0.0009) [2023-12-26 16:01:16,613][105620] Updated weights for policy 1, policy_version 90387 (0.0009) [2023-12-26 16:01:16,667][105620] Updated weights for policy 1, policy_version 90397 (0.0009) [2023-12-26 16:01:16,728][105620] Updated weights for policy 1, policy_version 90407 (0.0005) [2023-12-26 16:01:17,174][105692] Updated weights for policy 0, policy_version 90073 (0.0009) [2023-12-26 16:01:17,226][105692] Updated weights for policy 0, policy_version 90083 (0.0010) [2023-12-26 16:01:17,273][105692] Updated weights for policy 0, policy_version 90093 (0.0010) [2023-12-26 16:01:17,382][105620] Updated weights for policy 1, policy_version 90417 (0.0006) [2023-12-26 16:01:17,436][105620] Updated weights for policy 1, policy_version 90427 (0.0009) [2023-12-26 16:01:17,482][105620] Updated weights for policy 1, policy_version 90437 (0.0008) [2023-12-26 16:01:17,539][105620] Updated weights for policy 1, policy_version 90447 (0.0009) [2023-12-26 16:01:18,008][105692] Updated weights for policy 0, policy_version 90103 (0.0009) [2023-12-26 16:01:18,060][105692] Updated weights for policy 0, policy_version 90113 (0.0006) [2023-12-26 16:01:18,120][105692] Updated weights for policy 0, policy_version 90123 (0.0006) [2023-12-26 16:01:18,329][105620] Updated weights for policy 1, policy_version 90457 (0.0009) [2023-12-26 16:01:18,393][105620] Updated weights for policy 1, policy_version 90467 (0.0008) [2023-12-26 16:01:18,449][105620] Updated weights for policy 1, policy_version 90477 (0.0009) [2023-12-26 16:01:18,800][105692] Updated weights for policy 0, policy_version 90133 (0.0007) [2023-12-26 16:01:18,868][105692] Updated weights for policy 0, policy_version 90143 (0.0008) [2023-12-26 16:01:18,936][105692] Updated weights for policy 0, policy_version 90153 (0.0008) [2023-12-26 16:01:19,161][105620] Updated weights for policy 1, policy_version 90487 (0.0009) [2023-12-26 16:01:19,232][105620] Updated weights for policy 1, policy_version 90497 (0.0009) [2023-12-26 16:01:19,296][105620] Updated weights for policy 1, policy_version 90507 (0.0008) [2023-12-26 16:01:19,568][105692] Updated weights for policy 0, policy_version 90163 (0.0009) [2023-12-26 16:01:19,634][105692] Updated weights for policy 0, policy_version 90173 (0.0006) [2023-12-26 16:01:19,690][105692] Updated weights for policy 0, policy_version 90183 (0.0006) [2023-12-26 16:01:20,054][105620] Updated weights for policy 1, policy_version 90517 (0.0009) [2023-12-26 16:01:20,115][105620] Updated weights for policy 1, policy_version 90527 (0.0007) [2023-12-26 16:01:20,173][105620] Updated weights for policy 1, policy_version 90537 (0.0008) [2023-12-26 16:01:20,357][105692] Updated weights for policy 0, policy_version 90193 (0.0005) [2023-12-26 16:01:20,404][105692] Updated weights for policy 0, policy_version 90203 (0.0007) [2023-12-26 16:01:20,453][105692] Updated weights for policy 0, policy_version 90213 (0.0010) [2023-12-26 16:01:20,499][105692] Updated weights for policy 0, policy_version 90223 (0.0010) [2023-12-26 16:01:20,965][105620] Updated weights for policy 1, policy_version 90547 (0.0008) [2023-12-26 16:01:21,031][105620] Updated weights for policy 1, policy_version 90557 (0.0009) [2023-12-26 16:01:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 46284800. Throughput: 0: 9886.0, 1: 9793.7. Samples: 46280152. Policy #0 lag: (min: 17.0, avg: 39.9, max: 49.0) [2023-12-26 16:01:21,063][104569] Avg episode reward: [(0, '7376.500'), (1, '8824.025')] [2023-12-26 16:01:21,104][105620] Updated weights for policy 1, policy_version 90567 (0.0008) [2023-12-26 16:01:21,270][105692] Updated weights for policy 0, policy_version 90233 (0.0009) [2023-12-26 16:01:21,337][105692] Updated weights for policy 0, policy_version 90243 (0.0009) [2023-12-26 16:01:21,397][105692] Updated weights for policy 0, policy_version 90253 (0.0009) [2023-12-26 16:01:21,825][105620] Updated weights for policy 1, policy_version 90577 (0.0008) [2023-12-26 16:01:21,878][105620] Updated weights for policy 1, policy_version 90587 (0.0006) [2023-12-26 16:01:21,939][105620] Updated weights for policy 1, policy_version 90597 (0.0007) [2023-12-26 16:01:21,987][105620] Updated weights for policy 1, policy_version 90607 (0.0006) [2023-12-26 16:01:22,284][105692] Updated weights for policy 0, policy_version 90263 (0.0008) [2023-12-26 16:01:22,347][105692] Updated weights for policy 0, policy_version 90273 (0.0009) [2023-12-26 16:01:22,415][105692] Updated weights for policy 0, policy_version 90283 (0.0009) [2023-12-26 16:01:22,702][105620] Updated weights for policy 1, policy_version 90617 (0.0009) [2023-12-26 16:01:22,763][105620] Updated weights for policy 1, policy_version 90627 (0.0009) [2023-12-26 16:01:22,830][105620] Updated weights for policy 1, policy_version 90637 (0.0010) [2023-12-26 16:01:23,119][105692] Updated weights for policy 0, policy_version 90293 (0.0007) [2023-12-26 16:01:23,182][105692] Updated weights for policy 0, policy_version 90303 (0.0009) [2023-12-26 16:01:23,233][105692] Updated weights for policy 0, policy_version 90313 (0.0009) [2023-12-26 16:01:23,603][105620] Updated weights for policy 1, policy_version 90647 (0.0009) [2023-12-26 16:01:23,664][105620] Updated weights for policy 1, policy_version 90657 (0.0009) [2023-12-26 16:01:23,726][105620] Updated weights for policy 1, policy_version 90667 (0.0009) [2023-12-26 16:01:23,874][105692] Updated weights for policy 0, policy_version 90323 (0.0008) [2023-12-26 16:01:23,924][105692] Updated weights for policy 0, policy_version 90333 (0.0009) [2023-12-26 16:01:23,982][105692] Updated weights for policy 0, policy_version 90343 (0.0008) [2023-12-26 16:01:24,522][105620] Updated weights for policy 1, policy_version 90677 (0.0008) [2023-12-26 16:01:24,578][105620] Updated weights for policy 1, policy_version 90687 (0.0005) [2023-12-26 16:01:24,626][105620] Updated weights for policy 1, policy_version 90697 (0.0005) [2023-12-26 16:01:24,713][105692] Updated weights for policy 0, policy_version 90353 (0.0007) [2023-12-26 16:01:24,774][105692] Updated weights for policy 0, policy_version 90363 (0.0010) [2023-12-26 16:01:24,828][105692] Updated weights for policy 0, policy_version 90373 (0.0010) [2023-12-26 16:01:24,876][105692] Updated weights for policy 0, policy_version 90383 (0.0010) [2023-12-26 16:01:25,274][105620] Updated weights for policy 1, policy_version 90707 (0.0007) [2023-12-26 16:01:25,335][105620] Updated weights for policy 1, policy_version 90717 (0.0009) [2023-12-26 16:01:25,402][105620] Updated weights for policy 1, policy_version 90727 (0.0008) [2023-12-26 16:01:25,613][105692] Updated weights for policy 0, policy_version 90393 (0.0010) [2023-12-26 16:01:25,660][105692] Updated weights for policy 0, policy_version 90403 (0.0010) [2023-12-26 16:01:25,704][105692] Updated weights for policy 0, policy_version 90413 (0.0010) [2023-12-26 16:01:26,031][105620] Updated weights for policy 1, policy_version 90737 (0.0008) [2023-12-26 16:01:26,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 46383104. Throughput: 0: 9903.2, 1: 9826.3. Samples: 46395048. Policy #0 lag: (min: 17.0, avg: 39.9, max: 49.0) [2023-12-26 16:01:26,063][104569] Avg episode reward: [(0, '6921.843'), (1, '8726.354')] [2023-12-26 16:01:26,087][105620] Updated weights for policy 1, policy_version 90747 (0.0008) [2023-12-26 16:01:26,147][105620] Updated weights for policy 1, policy_version 90757 (0.0006) [2023-12-26 16:01:26,216][105620] Updated weights for policy 1, policy_version 90767 (0.0005) [2023-12-26 16:01:26,469][105692] Updated weights for policy 0, policy_version 90423 (0.0010) [2023-12-26 16:01:26,518][105692] Updated weights for policy 0, policy_version 90433 (0.0010) [2023-12-26 16:01:26,575][105692] Updated weights for policy 0, policy_version 90443 (0.0010) [2023-12-26 16:01:26,786][105620] Updated weights for policy 1, policy_version 90777 (0.0008) [2023-12-26 16:01:26,834][105620] Updated weights for policy 1, policy_version 90787 (0.0007) [2023-12-26 16:01:26,899][105620] Updated weights for policy 1, policy_version 90797 (0.0006) [2023-12-26 16:01:27,324][105692] Updated weights for policy 0, policy_version 90453 (0.0010) [2023-12-26 16:01:27,389][105692] Updated weights for policy 0, policy_version 90463 (0.0010) [2023-12-26 16:01:27,447][105692] Updated weights for policy 0, policy_version 90473 (0.0010) [2023-12-26 16:01:27,624][105620] Updated weights for policy 1, policy_version 90807 (0.0008) [2023-12-26 16:01:27,677][105620] Updated weights for policy 1, policy_version 90817 (0.0010) [2023-12-26 16:01:27,729][105620] Updated weights for policy 1, policy_version 90827 (0.0009) [2023-12-26 16:01:28,094][105692] Updated weights for policy 0, policy_version 90483 (0.0009) [2023-12-26 16:01:28,145][105692] Updated weights for policy 0, policy_version 90493 (0.0005) [2023-12-26 16:01:28,215][105692] Updated weights for policy 0, policy_version 90503 (0.0005) [2023-12-26 16:01:28,496][105620] Updated weights for policy 1, policy_version 90837 (0.0007) [2023-12-26 16:01:28,554][105620] Updated weights for policy 1, policy_version 90847 (0.0009) [2023-12-26 16:01:28,607][105620] Updated weights for policy 1, policy_version 90857 (0.0009) [2023-12-26 16:01:28,893][105692] Updated weights for policy 0, policy_version 90513 (0.0009) [2023-12-26 16:01:28,938][105692] Updated weights for policy 0, policy_version 90523 (0.0009) [2023-12-26 16:01:28,983][105692] Updated weights for policy 0, policy_version 90533 (0.0007) [2023-12-26 16:01:29,038][105692] Updated weights for policy 0, policy_version 90543 (0.0005) [2023-12-26 16:01:29,352][105620] Updated weights for policy 1, policy_version 90867 (0.0009) [2023-12-26 16:01:29,415][105620] Updated weights for policy 1, policy_version 90877 (0.0009) [2023-12-26 16:01:29,474][105620] Updated weights for policy 1, policy_version 90887 (0.0008) [2023-12-26 16:01:29,714][105692] Updated weights for policy 0, policy_version 90553 (0.0010) [2023-12-26 16:01:29,767][105692] Updated weights for policy 0, policy_version 90563 (0.0008) [2023-12-26 16:01:29,824][105692] Updated weights for policy 0, policy_version 90573 (0.0008) [2023-12-26 16:01:30,210][105620] Updated weights for policy 1, policy_version 90897 (0.0008) [2023-12-26 16:01:30,266][105620] Updated weights for policy 1, policy_version 90907 (0.0005) [2023-12-26 16:01:30,327][105620] Updated weights for policy 1, policy_version 90917 (0.0006) [2023-12-26 16:01:30,387][105620] Updated weights for policy 1, policy_version 90927 (0.0008) [2023-12-26 16:01:30,563][105692] Updated weights for policy 0, policy_version 90583 (0.0010) [2023-12-26 16:01:30,622][105692] Updated weights for policy 0, policy_version 90593 (0.0010) [2023-12-26 16:01:30,686][105692] Updated weights for policy 0, policy_version 90603 (0.0010) [2023-12-26 16:01:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 46481408. Throughput: 0: 9971.2, 1: 9819.3. Samples: 46454560. Policy #0 lag: (min: 17.0, avg: 39.9, max: 49.0) [2023-12-26 16:01:31,062][104569] Avg episode reward: [(0, '7387.344'), (1, '8907.078')] [2023-12-26 16:01:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000090608_23199744.pth... [2023-12-26 16:01:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000089456_22904832.pth [2023-12-26 16:01:31,106][105620] Updated weights for policy 1, policy_version 90937 (0.0007) [2023-12-26 16:01:31,170][105620] Updated weights for policy 1, policy_version 90947 (0.0006) [2023-12-26 16:01:31,236][105620] Updated weights for policy 1, policy_version 90957 (0.0006) [2023-12-26 16:01:31,255][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000090960_23289856.pth... [2023-12-26 16:01:31,260][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000089808_22994944.pth [2023-12-26 16:01:31,435][105692] Updated weights for policy 0, policy_version 90613 (0.0010) [2023-12-26 16:01:31,499][105692] Updated weights for policy 0, policy_version 90623 (0.0010) [2023-12-26 16:01:31,544][105692] Updated weights for policy 0, policy_version 90633 (0.0010) [2023-12-26 16:01:31,938][105620] Updated weights for policy 1, policy_version 90967 (0.0008) [2023-12-26 16:01:31,993][105620] Updated weights for policy 1, policy_version 90977 (0.0009) [2023-12-26 16:01:32,045][105620] Updated weights for policy 1, policy_version 90987 (0.0009) [2023-12-26 16:01:32,223][105692] Updated weights for policy 0, policy_version 90644 (0.0010) [2023-12-26 16:01:32,283][105692] Updated weights for policy 0, policy_version 90654 (0.0009) [2023-12-26 16:01:32,341][105692] Updated weights for policy 0, policy_version 90664 (0.0009) [2023-12-26 16:01:32,764][105620] Updated weights for policy 1, policy_version 90997 (0.0009) [2023-12-26 16:01:32,819][105620] Updated weights for policy 1, policy_version 91007 (0.0010) [2023-12-26 16:01:32,873][105620] Updated weights for policy 1, policy_version 91017 (0.0010) [2023-12-26 16:01:32,968][105692] Updated weights for policy 0, policy_version 90674 (0.0008) [2023-12-26 16:01:33,025][105692] Updated weights for policy 0, policy_version 90684 (0.0009) [2023-12-26 16:01:33,079][105692] Updated weights for policy 0, policy_version 90694 (0.0007) [2023-12-26 16:01:33,135][105692] Updated weights for policy 0, policy_version 90704 (0.0007) [2023-12-26 16:01:33,676][105620] Updated weights for policy 1, policy_version 91027 (0.0009) [2023-12-26 16:01:33,735][105620] Updated weights for policy 1, policy_version 91037 (0.0010) [2023-12-26 16:01:33,787][105620] Updated weights for policy 1, policy_version 91047 (0.0010) [2023-12-26 16:01:33,876][105692] Updated weights for policy 0, policy_version 90714 (0.0010) [2023-12-26 16:01:33,934][105692] Updated weights for policy 0, policy_version 90724 (0.0010) [2023-12-26 16:01:33,996][105692] Updated weights for policy 0, policy_version 90734 (0.0010) [2023-12-26 16:01:34,571][105692] Updated weights for policy 0, policy_version 90744 (0.0006) [2023-12-26 16:01:34,631][105692] Updated weights for policy 0, policy_version 90754 (0.0009) [2023-12-26 16:01:34,650][105620] Updated weights for policy 1, policy_version 91057 (0.0010) [2023-12-26 16:01:34,695][105692] Updated weights for policy 0, policy_version 90764 (0.0007) [2023-12-26 16:01:34,712][105620] Updated weights for policy 1, policy_version 91067 (0.0009) [2023-12-26 16:01:34,771][105620] Updated weights for policy 1, policy_version 91077 (0.0009) [2023-12-26 16:01:34,825][105620] Updated weights for policy 1, policy_version 91087 (0.0009) [2023-12-26 16:01:35,231][105692] Updated weights for policy 0, policy_version 90774 (0.0008) [2023-12-26 16:01:35,278][105692] Updated weights for policy 0, policy_version 90784 (0.0010) [2023-12-26 16:01:35,331][105692] Updated weights for policy 0, policy_version 90794 (0.0010) [2023-12-26 16:01:35,626][105620] Updated weights for policy 1, policy_version 91097 (0.0010) [2023-12-26 16:01:35,682][105620] Updated weights for policy 1, policy_version 91107 (0.0011) [2023-12-26 16:01:35,737][105620] Updated weights for policy 1, policy_version 91117 (0.0010) [2023-12-26 16:01:36,053][105692] Updated weights for policy 0, policy_version 90804 (0.0009) [2023-12-26 16:01:36,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 46579712. Throughput: 0: 9958.0, 1: 9706.8. Samples: 46570712. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:01:36,063][104569] Avg episode reward: [(0, '7824.064'), (1, '8730.434')] [2023-12-26 16:01:36,122][105692] Updated weights for policy 0, policy_version 90814 (0.0007) [2023-12-26 16:01:36,181][105692] Updated weights for policy 0, policy_version 90824 (0.0010) [2023-12-26 16:01:36,488][105620] Updated weights for policy 1, policy_version 91127 (0.0010) [2023-12-26 16:01:36,544][105620] Updated weights for policy 1, policy_version 91137 (0.0010) [2023-12-26 16:01:36,596][105620] Updated weights for policy 1, policy_version 91147 (0.0008) [2023-12-26 16:01:36,902][105692] Updated weights for policy 0, policy_version 90834 (0.0007) [2023-12-26 16:01:36,948][105692] Updated weights for policy 0, policy_version 90844 (0.0008) [2023-12-26 16:01:37,001][105692] Updated weights for policy 0, policy_version 90854 (0.0010) [2023-12-26 16:01:37,056][105692] Updated weights for policy 0, policy_version 90864 (0.0010) [2023-12-26 16:01:37,186][105620] Updated weights for policy 1, policy_version 91157 (0.0007) [2023-12-26 16:01:37,241][105620] Updated weights for policy 1, policy_version 91167 (0.0008) [2023-12-26 16:01:37,289][105620] Updated weights for policy 1, policy_version 91177 (0.0010) [2023-12-26 16:01:37,828][105692] Updated weights for policy 0, policy_version 90874 (0.0008) [2023-12-26 16:01:37,885][105692] Updated weights for policy 0, policy_version 90884 (0.0008) [2023-12-26 16:01:37,951][105692] Updated weights for policy 0, policy_version 90894 (0.0008) [2023-12-26 16:01:38,026][105620] Updated weights for policy 1, policy_version 91187 (0.0010) [2023-12-26 16:01:38,078][105620] Updated weights for policy 1, policy_version 91197 (0.0010) [2023-12-26 16:01:38,140][105620] Updated weights for policy 1, policy_version 91207 (0.0010) [2023-12-26 16:01:38,599][105692] Updated weights for policy 0, policy_version 90904 (0.0009) [2023-12-26 16:01:38,661][105692] Updated weights for policy 0, policy_version 90914 (0.0008) [2023-12-26 16:01:38,728][105692] Updated weights for policy 0, policy_version 90924 (0.0007) [2023-12-26 16:01:38,908][105620] Updated weights for policy 1, policy_version 91217 (0.0010) [2023-12-26 16:01:38,971][105620] Updated weights for policy 1, policy_version 91227 (0.0009) [2023-12-26 16:01:39,035][105620] Updated weights for policy 1, policy_version 91237 (0.0009) [2023-12-26 16:01:39,099][105620] Updated weights for policy 1, policy_version 91247 (0.0009) [2023-12-26 16:01:39,342][105692] Updated weights for policy 0, policy_version 90934 (0.0008) [2023-12-26 16:01:39,410][105692] Updated weights for policy 0, policy_version 90944 (0.0007) [2023-12-26 16:01:39,476][105692] Updated weights for policy 0, policy_version 90954 (0.0008) [2023-12-26 16:01:39,917][105620] Updated weights for policy 1, policy_version 91257 (0.0010) [2023-12-26 16:01:39,979][105620] Updated weights for policy 1, policy_version 91267 (0.0009) [2023-12-26 16:01:40,039][105620] Updated weights for policy 1, policy_version 91277 (0.0008) [2023-12-26 16:01:40,192][105692] Updated weights for policy 0, policy_version 90964 (0.0008) [2023-12-26 16:01:40,243][105692] Updated weights for policy 0, policy_version 90974 (0.0008) [2023-12-26 16:01:40,312][105692] Updated weights for policy 0, policy_version 90984 (0.0009) [2023-12-26 16:01:40,788][105620] Updated weights for policy 1, policy_version 91287 (0.0009) [2023-12-26 16:01:40,842][105620] Updated weights for policy 1, policy_version 91297 (0.0009) [2023-12-26 16:01:40,896][105620] Updated weights for policy 1, policy_version 91307 (0.0009) [2023-12-26 16:01:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 46678016. Throughput: 0: 9973.5, 1: 9639.3. Samples: 46687532. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:01:41,063][104569] Avg episode reward: [(0, '7817.580'), (1, '8651.068')] [2023-12-26 16:01:41,065][105692] Updated weights for policy 0, policy_version 90994 (0.0009) [2023-12-26 16:01:41,121][105692] Updated weights for policy 0, policy_version 91004 (0.0008) [2023-12-26 16:01:41,184][105692] Updated weights for policy 0, policy_version 91014 (0.0008) [2023-12-26 16:01:41,243][105692] Updated weights for policy 0, policy_version 91024 (0.0008) [2023-12-26 16:01:41,726][105620] Updated weights for policy 1, policy_version 91317 (0.0010) [2023-12-26 16:01:41,792][105620] Updated weights for policy 1, policy_version 91327 (0.0008) [2023-12-26 16:01:41,853][105620] Updated weights for policy 1, policy_version 91337 (0.0008) [2023-12-26 16:01:41,927][105692] Updated weights for policy 0, policy_version 91034 (0.0007) [2023-12-26 16:01:41,993][105692] Updated weights for policy 0, policy_version 91044 (0.0009) [2023-12-26 16:01:42,061][105692] Updated weights for policy 0, policy_version 91054 (0.0007) [2023-12-26 16:01:42,562][105620] Updated weights for policy 1, policy_version 91347 (0.0008) [2023-12-26 16:01:42,606][105620] Updated weights for policy 1, policy_version 91357 (0.0008) [2023-12-26 16:01:42,656][105620] Updated weights for policy 1, policy_version 91367 (0.0005) [2023-12-26 16:01:42,765][105692] Updated weights for policy 0, policy_version 91064 (0.0006) [2023-12-26 16:01:42,824][105692] Updated weights for policy 0, policy_version 91074 (0.0006) [2023-12-26 16:01:42,877][105692] Updated weights for policy 0, policy_version 91084 (0.0010) [2023-12-26 16:01:43,290][105620] Updated weights for policy 1, policy_version 91377 (0.0008) [2023-12-26 16:01:43,343][105620] Updated weights for policy 1, policy_version 91388 (0.0010) [2023-12-26 16:01:43,402][105620] Updated weights for policy 1, policy_version 91400 (0.0011) [2023-12-26 16:01:43,486][105692] Updated weights for policy 0, policy_version 91094 (0.0007) [2023-12-26 16:01:43,550][105692] Updated weights for policy 0, policy_version 91104 (0.0005) [2023-12-26 16:01:43,607][105692] Updated weights for policy 0, policy_version 91114 (0.0005) [2023-12-26 16:01:44,217][105620] Updated weights for policy 1, policy_version 91410 (0.0010) [2023-12-26 16:01:44,263][105692] Updated weights for policy 0, policy_version 91124 (0.0007) [2023-12-26 16:01:44,279][105620] Updated weights for policy 1, policy_version 91420 (0.0009) [2023-12-26 16:01:44,317][105692] Updated weights for policy 0, policy_version 91134 (0.0009) [2023-12-26 16:01:44,328][105620] Updated weights for policy 1, policy_version 91430 (0.0007) [2023-12-26 16:01:44,370][105692] Updated weights for policy 0, policy_version 91144 (0.0007) [2023-12-26 16:01:44,379][105620] Updated weights for policy 1, policy_version 91440 (0.0006) [2023-12-26 16:01:45,095][105620] Updated weights for policy 1, policy_version 91450 (0.0009) [2023-12-26 16:01:45,160][105620] Updated weights for policy 1, policy_version 91460 (0.0008) [2023-12-26 16:01:45,165][105692] Updated weights for policy 0, policy_version 91154 (0.0009) [2023-12-26 16:01:45,222][105620] Updated weights for policy 1, policy_version 91470 (0.0007) [2023-12-26 16:01:45,228][105692] Updated weights for policy 0, policy_version 91164 (0.0008) [2023-12-26 16:01:45,289][105692] Updated weights for policy 0, policy_version 91174 (0.0006) [2023-12-26 16:01:45,352][105692] Updated weights for policy 0, policy_version 91184 (0.0008) [2023-12-26 16:01:46,020][105692] Updated weights for policy 0, policy_version 91194 (0.0007) [2023-12-26 16:01:46,020][105620] Updated weights for policy 1, policy_version 91480 (0.0009) [2023-12-26 16:01:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 46768128. Throughput: 0: 9918.6, 1: 9602.2. Samples: 46746968. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:01:46,062][104569] Avg episode reward: [(0, '7907.916'), (1, '8386.039')] [2023-12-26 16:01:46,074][105620] Updated weights for policy 1, policy_version 91490 (0.0008) [2023-12-26 16:01:46,076][105692] Updated weights for policy 0, policy_version 91204 (0.0007) [2023-12-26 16:01:46,131][105620] Updated weights for policy 1, policy_version 91500 (0.0007) [2023-12-26 16:01:46,136][105692] Updated weights for policy 0, policy_version 91214 (0.0009) [2023-12-26 16:01:46,147][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000091216_23355392.pth... [2023-12-26 16:01:46,151][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000090032_23052288.pth [2023-12-26 16:01:46,152][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_000091216_23355392.pth [2023-12-26 16:01:46,154][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000091504_23429120.pth... [2023-12-26 16:01:46,158][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000090384_23142400.pth [2023-12-26 16:01:46,159][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_000091504_23429120.pth [2023-12-26 16:01:46,848][105692] Updated weights for policy 0, policy_version 91224 (0.0009) [2023-12-26 16:01:46,896][105692] Updated weights for policy 0, policy_version 91234 (0.0008) [2023-12-26 16:01:46,906][105620] Updated weights for policy 1, policy_version 91510 (0.0007) [2023-12-26 16:01:46,948][105692] Updated weights for policy 0, policy_version 91244 (0.0007) [2023-12-26 16:01:46,962][105620] Updated weights for policy 1, policy_version 91520 (0.0006) [2023-12-26 16:01:47,012][105620] Updated weights for policy 1, policy_version 91530 (0.0009) [2023-12-26 16:01:47,577][105620] Updated weights for policy 1, policy_version 91540 (0.0008) [2023-12-26 16:01:47,635][105620] Updated weights for policy 1, policy_version 91550 (0.0009) [2023-12-26 16:01:47,700][105620] Updated weights for policy 1, policy_version 91560 (0.0008) [2023-12-26 16:01:47,789][105692] Updated weights for policy 0, policy_version 91254 (0.0008) [2023-12-26 16:01:47,851][105692] Updated weights for policy 0, policy_version 91264 (0.0009) [2023-12-26 16:01:47,901][105692] Updated weights for policy 0, policy_version 91274 (0.0009) [2023-12-26 16:01:48,468][105620] Updated weights for policy 1, policy_version 91570 (0.0009) [2023-12-26 16:01:48,535][105620] Updated weights for policy 1, policy_version 91580 (0.0008) [2023-12-26 16:01:48,594][105620] Updated weights for policy 1, policy_version 91590 (0.0008) [2023-12-26 16:01:48,615][105692] Updated weights for policy 0, policy_version 91284 (0.0010) [2023-12-26 16:01:48,656][105620] Updated weights for policy 1, policy_version 91600 (0.0007) [2023-12-26 16:01:48,672][105692] Updated weights for policy 0, policy_version 91294 (0.0006) [2023-12-26 16:01:48,723][105692] Updated weights for policy 0, policy_version 91304 (0.0009) [2023-12-26 16:01:49,279][105620] Updated weights for policy 1, policy_version 91610 (0.0008) [2023-12-26 16:01:49,341][105620] Updated weights for policy 1, policy_version 91620 (0.0009) [2023-12-26 16:01:49,414][105620] Updated weights for policy 1, policy_version 91630 (0.0008) [2023-12-26 16:01:49,497][105692] Updated weights for policy 0, policy_version 91314 (0.0009) [2023-12-26 16:01:49,554][105692] Updated weights for policy 0, policy_version 91324 (0.0005) [2023-12-26 16:01:49,613][105692] Updated weights for policy 0, policy_version 91334 (0.0005) [2023-12-26 16:01:49,671][105692] Updated weights for policy 0, policy_version 91344 (0.0006) [2023-12-26 16:01:50,269][105692] Updated weights for policy 0, policy_version 91354 (0.0009) [2023-12-26 16:01:50,269][105620] Updated weights for policy 1, policy_version 91640 (0.0009) [2023-12-26 16:01:50,320][105620] Updated weights for policy 1, policy_version 91650 (0.0009) [2023-12-26 16:01:50,334][105692] Updated weights for policy 0, policy_version 91364 (0.0008) [2023-12-26 16:01:50,379][105620] Updated weights for policy 1, policy_version 91660 (0.0006) [2023-12-26 16:01:50,396][105692] Updated weights for policy 0, policy_version 91374 (0.0008) [2023-12-26 16:01:51,057][105692] Updated weights for policy 0, policy_version 91384 (0.0010) [2023-12-26 16:01:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 46866432. Throughput: 0: 9870.8, 1: 9618.0. Samples: 46861544. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:01:51,062][104569] Avg episode reward: [(0, '7454.502'), (1, '8639.414')] [2023-12-26 16:01:51,094][105620] Updated weights for policy 1, policy_version 91670 (0.0007) [2023-12-26 16:01:51,118][105692] Updated weights for policy 0, policy_version 91394 (0.0007) [2023-12-26 16:01:51,160][105620] Updated weights for policy 1, policy_version 91680 (0.0009) [2023-12-26 16:01:51,183][105692] Updated weights for policy 0, policy_version 91404 (0.0006) [2023-12-26 16:01:51,220][105620] Updated weights for policy 1, policy_version 91690 (0.0010) [2023-12-26 16:01:51,901][105692] Updated weights for policy 0, policy_version 91414 (0.0006) [2023-12-26 16:01:51,951][105692] Updated weights for policy 0, policy_version 91424 (0.0008) [2023-12-26 16:01:52,006][105692] Updated weights for policy 0, policy_version 91434 (0.0011) [2023-12-26 16:01:52,045][105620] Updated weights for policy 1, policy_version 91700 (0.0009) [2023-12-26 16:01:52,110][105620] Updated weights for policy 1, policy_version 91710 (0.0006) [2023-12-26 16:01:52,174][105620] Updated weights for policy 1, policy_version 91720 (0.0008) [2023-12-26 16:01:52,629][105692] Updated weights for policy 0, policy_version 91444 (0.0008) [2023-12-26 16:01:52,690][105692] Updated weights for policy 0, policy_version 91454 (0.0006) [2023-12-26 16:01:52,759][105692] Updated weights for policy 0, policy_version 91464 (0.0006) [2023-12-26 16:01:52,900][105620] Updated weights for policy 1, policy_version 91730 (0.0008) [2023-12-26 16:01:52,947][105620] Updated weights for policy 1, policy_version 91740 (0.0009) [2023-12-26 16:01:53,001][105620] Updated weights for policy 1, policy_version 91750 (0.0009) [2023-12-26 16:01:53,059][105620] Updated weights for policy 1, policy_version 91760 (0.0008) [2023-12-26 16:01:53,395][105692] Updated weights for policy 0, policy_version 91474 (0.0006) [2023-12-26 16:01:53,444][105692] Updated weights for policy 0, policy_version 91484 (0.0009) [2023-12-26 16:01:53,501][105692] Updated weights for policy 0, policy_version 91495 (0.0010) [2023-12-26 16:01:53,742][105620] Updated weights for policy 1, policy_version 91770 (0.0006) [2023-12-26 16:01:53,797][105620] Updated weights for policy 1, policy_version 91780 (0.0005) [2023-12-26 16:01:53,851][105620] Updated weights for policy 1, policy_version 91790 (0.0005) [2023-12-26 16:01:54,091][105692] Updated weights for policy 0, policy_version 91505 (0.0007) [2023-12-26 16:01:54,152][105692] Updated weights for policy 0, policy_version 91515 (0.0009) [2023-12-26 16:01:54,207][105692] Updated weights for policy 0, policy_version 91525 (0.0009) [2023-12-26 16:01:54,271][105692] Updated weights for policy 0, policy_version 91535 (0.0010) [2023-12-26 16:01:54,426][105620] Updated weights for policy 1, policy_version 91800 (0.0010) [2023-12-26 16:01:54,491][105620] Updated weights for policy 1, policy_version 91810 (0.0011) [2023-12-26 16:01:54,552][105620] Updated weights for policy 1, policy_version 91820 (0.0006) [2023-12-26 16:01:55,003][105692] Updated weights for policy 0, policy_version 91545 (0.0009) [2023-12-26 16:01:55,055][105692] Updated weights for policy 0, policy_version 91555 (0.0007) [2023-12-26 16:01:55,118][105692] Updated weights for policy 0, policy_version 91565 (0.0008) [2023-12-26 16:01:55,154][105620] Updated weights for policy 1, policy_version 91830 (0.0009) [2023-12-26 16:01:55,212][105620] Updated weights for policy 1, policy_version 91840 (0.0010) [2023-12-26 16:01:55,273][105620] Updated weights for policy 1, policy_version 91850 (0.0010) [2023-12-26 16:01:55,784][105692] Updated weights for policy 0, policy_version 91575 (0.0006) [2023-12-26 16:01:55,835][105692] Updated weights for policy 0, policy_version 91585 (0.0005) [2023-12-26 16:01:55,891][105692] Updated weights for policy 0, policy_version 91595 (0.0005) [2023-12-26 16:01:55,931][105620] Updated weights for policy 1, policy_version 91860 (0.0009) [2023-12-26 16:01:55,980][105620] Updated weights for policy 1, policy_version 91870 (0.0005) [2023-12-26 16:01:56,027][105620] Updated weights for policy 1, policy_version 91880 (0.0006) [2023-12-26 16:01:56,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 46972928. Throughput: 0: 9914.5, 1: 9648.5. Samples: 46983748. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:01:56,062][104569] Avg episode reward: [(0, '7735.677'), (1, '9175.294')] [2023-12-26 16:01:56,530][105692] Updated weights for policy 0, policy_version 91605 (0.0008) [2023-12-26 16:01:56,559][105620] Updated weights for policy 1, policy_version 91890 (0.0005) [2023-12-26 16:01:56,578][105692] Updated weights for policy 0, policy_version 91615 (0.0010) [2023-12-26 16:01:56,605][105620] Updated weights for policy 1, policy_version 91900 (0.0005) [2023-12-26 16:01:56,640][105692] Updated weights for policy 0, policy_version 91625 (0.0010) [2023-12-26 16:01:56,654][105620] Updated weights for policy 1, policy_version 91910 (0.0005) [2023-12-26 16:01:56,703][105620] Updated weights for policy 1, policy_version 91920 (0.0007) [2023-12-26 16:01:57,327][105692] Updated weights for policy 0, policy_version 91635 (0.0010) [2023-12-26 16:01:57,378][105692] Updated weights for policy 0, policy_version 91645 (0.0010) [2023-12-26 16:01:57,430][105692] Updated weights for policy 0, policy_version 91655 (0.0011) [2023-12-26 16:01:57,452][105620] Updated weights for policy 1, policy_version 91930 (0.0006) [2023-12-26 16:01:57,505][105620] Updated weights for policy 1, policy_version 91940 (0.0007) [2023-12-26 16:01:57,559][105620] Updated weights for policy 1, policy_version 91950 (0.0008) [2023-12-26 16:01:58,184][105692] Updated weights for policy 0, policy_version 91665 (0.0010) [2023-12-26 16:01:58,242][105692] Updated weights for policy 0, policy_version 91675 (0.0008) [2023-12-26 16:01:58,299][105692] Updated weights for policy 0, policy_version 91685 (0.0007) [2023-12-26 16:01:58,327][105620] Updated weights for policy 1, policy_version 91960 (0.0008) [2023-12-26 16:01:58,360][105692] Updated weights for policy 0, policy_version 91695 (0.0008) [2023-12-26 16:01:58,399][105620] Updated weights for policy 1, policy_version 91970 (0.0008) [2023-12-26 16:01:58,460][105620] Updated weights for policy 1, policy_version 91980 (0.0008) [2023-12-26 16:01:59,128][105692] Updated weights for policy 0, policy_version 91705 (0.0010) [2023-12-26 16:01:59,191][105692] Updated weights for policy 0, policy_version 91715 (0.0010) [2023-12-26 16:01:59,256][105692] Updated weights for policy 0, policy_version 91725 (0.0010) [2023-12-26 16:01:59,288][105620] Updated weights for policy 1, policy_version 91990 (0.0008) [2023-12-26 16:01:59,341][105620] Updated weights for policy 1, policy_version 92000 (0.0008) [2023-12-26 16:01:59,406][105620] Updated weights for policy 1, policy_version 92010 (0.0007) [2023-12-26 16:01:59,998][105692] Updated weights for policy 0, policy_version 91735 (0.0009) [2023-12-26 16:02:00,056][105692] Updated weights for policy 0, policy_version 91745 (0.0008) [2023-12-26 16:02:00,110][105620] Updated weights for policy 1, policy_version 92020 (0.0007) [2023-12-26 16:02:00,121][105692] Updated weights for policy 0, policy_version 91755 (0.0008) [2023-12-26 16:02:00,158][105620] Updated weights for policy 1, policy_version 92030 (0.0005) [2023-12-26 16:02:00,217][105620] Updated weights for policy 1, policy_version 92040 (0.0006) [2023-12-26 16:02:00,827][105620] Updated weights for policy 1, policy_version 92050 (0.0007) [2023-12-26 16:02:00,870][105692] Updated weights for policy 0, policy_version 91765 (0.0009) [2023-12-26 16:02:00,882][105620] Updated weights for policy 1, policy_version 92060 (0.0011) [2023-12-26 16:02:00,929][105692] Updated weights for policy 0, policy_version 91775 (0.0010) [2023-12-26 16:02:00,938][105620] Updated weights for policy 1, policy_version 92070 (0.0011) [2023-12-26 16:02:00,988][105692] Updated weights for policy 0, policy_version 91785 (0.0006) [2023-12-26 16:02:01,001][105620] Updated weights for policy 1, policy_version 92080 (0.0010) [2023-12-26 16:02:01,062][104569] Fps is (10 sec: 21299.3, 60 sec: 19660.9, 300 sec: 19494.2). Total num frames: 47079424. Throughput: 0: 9917.5, 1: 9626.4. Samples: 47043556. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:02:01,062][104569] Avg episode reward: [(0, '7285.875'), (1, '9264.493')] [2023-12-26 16:02:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000091792_23502848.pth... [2023-12-26 16:02:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000092080_23576576.pth... [2023-12-26 16:02:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000090960_23289856.pth [2023-12-26 16:02:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000090608_23199744.pth [2023-12-26 16:02:01,688][105692] Updated weights for policy 0, policy_version 91795 (0.0007) [2023-12-26 16:02:01,754][105692] Updated weights for policy 0, policy_version 91805 (0.0011) [2023-12-26 16:02:01,768][105620] Updated weights for policy 1, policy_version 92090 (0.0006) [2023-12-26 16:02:01,816][105692] Updated weights for policy 0, policy_version 91815 (0.0010) [2023-12-26 16:02:01,827][105620] Updated weights for policy 1, policy_version 92100 (0.0009) [2023-12-26 16:02:01,882][105620] Updated weights for policy 1, policy_version 92110 (0.0008) [2023-12-26 16:02:02,501][105692] Updated weights for policy 0, policy_version 91825 (0.0010) [2023-12-26 16:02:02,569][105692] Updated weights for policy 0, policy_version 91835 (0.0008) [2023-12-26 16:02:02,624][105692] Updated weights for policy 0, policy_version 91845 (0.0009) [2023-12-26 16:02:02,679][105620] Updated weights for policy 1, policy_version 92120 (0.0009) [2023-12-26 16:02:02,687][105692] Updated weights for policy 0, policy_version 91855 (0.0011) [2023-12-26 16:02:02,744][105620] Updated weights for policy 1, policy_version 92130 (0.0011) [2023-12-26 16:02:02,792][105620] Updated weights for policy 1, policy_version 92140 (0.0008) [2023-12-26 16:02:03,365][105620] Updated weights for policy 1, policy_version 92150 (0.0005) [2023-12-26 16:02:03,411][105620] Updated weights for policy 1, policy_version 92160 (0.0005) [2023-12-26 16:02:03,456][105692] Updated weights for policy 0, policy_version 91865 (0.0010) [2023-12-26 16:02:03,457][105620] Updated weights for policy 1, policy_version 92170 (0.0005) [2023-12-26 16:02:03,507][105692] Updated weights for policy 0, policy_version 91875 (0.0010) [2023-12-26 16:02:03,559][105692] Updated weights for policy 0, policy_version 91885 (0.0010) [2023-12-26 16:02:04,004][105620] Updated weights for policy 1, policy_version 92180 (0.0007) [2023-12-26 16:02:04,058][105620] Updated weights for policy 1, policy_version 92190 (0.0005) [2023-12-26 16:02:04,117][105620] Updated weights for policy 1, policy_version 92200 (0.0007) [2023-12-26 16:02:04,253][105692] Updated weights for policy 0, policy_version 91895 (0.0011) [2023-12-26 16:02:04,318][105692] Updated weights for policy 0, policy_version 91905 (0.0009) [2023-12-26 16:02:04,386][105692] Updated weights for policy 0, policy_version 91915 (0.0009) [2023-12-26 16:02:04,798][105620] Updated weights for policy 1, policy_version 92210 (0.0010) [2023-12-26 16:02:04,846][105620] Updated weights for policy 1, policy_version 92220 (0.0005) [2023-12-26 16:02:04,897][105620] Updated weights for policy 1, policy_version 92230 (0.0006) [2023-12-26 16:02:04,942][105620] Updated weights for policy 1, policy_version 92240 (0.0008) [2023-12-26 16:02:05,093][105692] Updated weights for policy 0, policy_version 91925 (0.0008) [2023-12-26 16:02:05,150][105692] Updated weights for policy 0, policy_version 91935 (0.0005) [2023-12-26 16:02:05,198][105692] Updated weights for policy 0, policy_version 91945 (0.0005) [2023-12-26 16:02:05,714][105692] Updated weights for policy 0, policy_version 91955 (0.0006) [2023-12-26 16:02:05,736][105620] Updated weights for policy 1, policy_version 92250 (0.0008) [2023-12-26 16:02:05,770][105692] Updated weights for policy 0, policy_version 91965 (0.0005) [2023-12-26 16:02:05,798][105620] Updated weights for policy 1, policy_version 92260 (0.0008) [2023-12-26 16:02:05,823][105692] Updated weights for policy 0, policy_version 91975 (0.0005) [2023-12-26 16:02:05,857][105620] Updated weights for policy 1, policy_version 92270 (0.0008) [2023-12-26 16:02:06,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 47177728. Throughput: 0: 9897.0, 1: 9703.5. Samples: 47162172. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:02:06,063][104569] Avg episode reward: [(0, '7008.174'), (1, '8640.968')] [2023-12-26 16:02:06,427][105692] Updated weights for policy 0, policy_version 91985 (0.0006) [2023-12-26 16:02:06,490][105692] Updated weights for policy 0, policy_version 91995 (0.0011) [2023-12-26 16:02:06,549][105692] Updated weights for policy 0, policy_version 92005 (0.0011) [2023-12-26 16:02:06,598][105692] Updated weights for policy 0, policy_version 92015 (0.0011) [2023-12-26 16:02:06,685][105620] Updated weights for policy 1, policy_version 92280 (0.0008) [2023-12-26 16:02:06,745][105620] Updated weights for policy 1, policy_version 92290 (0.0008) [2023-12-26 16:02:06,804][105620] Updated weights for policy 1, policy_version 92300 (0.0008) [2023-12-26 16:02:07,264][105692] Updated weights for policy 0, policy_version 92025 (0.0007) [2023-12-26 16:02:07,329][105692] Updated weights for policy 0, policy_version 92035 (0.0008) [2023-12-26 16:02:07,391][105692] Updated weights for policy 0, policy_version 92045 (0.0009) [2023-12-26 16:02:07,601][105620] Updated weights for policy 1, policy_version 92310 (0.0009) [2023-12-26 16:02:07,653][105620] Updated weights for policy 1, policy_version 92320 (0.0009) [2023-12-26 16:02:07,699][105620] Updated weights for policy 1, policy_version 92330 (0.0009) [2023-12-26 16:02:08,077][105692] Updated weights for policy 0, policy_version 92055 (0.0006) [2023-12-26 16:02:08,141][105692] Updated weights for policy 0, policy_version 92065 (0.0009) [2023-12-26 16:02:08,202][105692] Updated weights for policy 0, policy_version 92075 (0.0009) [2023-12-26 16:02:08,440][105620] Updated weights for policy 1, policy_version 92340 (0.0009) [2023-12-26 16:02:08,492][105620] Updated weights for policy 1, policy_version 92351 (0.0009) [2023-12-26 16:02:08,547][105620] Updated weights for policy 1, policy_version 92363 (0.0011) [2023-12-26 16:02:08,856][105692] Updated weights for policy 0, policy_version 92085 (0.0009) [2023-12-26 16:02:08,940][105692] Updated weights for policy 0, policy_version 92095 (0.0008) [2023-12-26 16:02:08,991][105692] Updated weights for policy 0, policy_version 92105 (0.0006) [2023-12-26 16:02:09,423][105620] Updated weights for policy 1, policy_version 92373 (0.0010) [2023-12-26 16:02:09,481][105620] Updated weights for policy 1, policy_version 92383 (0.0010) [2023-12-26 16:02:09,547][105620] Updated weights for policy 1, policy_version 92393 (0.0009) [2023-12-26 16:02:09,611][105692] Updated weights for policy 0, policy_version 92115 (0.0007) [2023-12-26 16:02:09,675][105692] Updated weights for policy 0, policy_version 92125 (0.0009) [2023-12-26 16:02:09,742][105692] Updated weights for policy 0, policy_version 92135 (0.0009) [2023-12-26 16:02:10,222][105620] Updated weights for policy 1, policy_version 92403 (0.0009) [2023-12-26 16:02:10,270][105620] Updated weights for policy 1, policy_version 92413 (0.0009) [2023-12-26 16:02:10,322][105620] Updated weights for policy 1, policy_version 92423 (0.0009) [2023-12-26 16:02:10,529][105692] Updated weights for policy 0, policy_version 92145 (0.0010) [2023-12-26 16:02:10,587][105692] Updated weights for policy 0, policy_version 92155 (0.0009) [2023-12-26 16:02:10,645][105692] Updated weights for policy 0, policy_version 92165 (0.0009) [2023-12-26 16:02:10,706][105692] Updated weights for policy 0, policy_version 92175 (0.0009) [2023-12-26 16:02:11,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 47267840. Throughput: 0: 10014.1, 1: 9633.2. Samples: 47279172. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:02:11,063][104569] Avg episode reward: [(0, '7824.071'), (1, '8109.446')] [2023-12-26 16:02:11,091][105620] Updated weights for policy 1, policy_version 92433 (0.0008) [2023-12-26 16:02:11,159][105620] Updated weights for policy 1, policy_version 92443 (0.0009) [2023-12-26 16:02:11,218][105620] Updated weights for policy 1, policy_version 92453 (0.0009) [2023-12-26 16:02:11,283][105620] Updated weights for policy 1, policy_version 92463 (0.0009) [2023-12-26 16:02:11,512][105692] Updated weights for policy 0, policy_version 92185 (0.0009) [2023-12-26 16:02:11,574][105692] Updated weights for policy 0, policy_version 92195 (0.0010) [2023-12-26 16:02:11,629][105692] Updated weights for policy 0, policy_version 92205 (0.0010) [2023-12-26 16:02:12,045][105620] Updated weights for policy 1, policy_version 92473 (0.0009) [2023-12-26 16:02:12,108][105620] Updated weights for policy 1, policy_version 92483 (0.0008) [2023-12-26 16:02:12,171][105620] Updated weights for policy 1, policy_version 92493 (0.0009) [2023-12-26 16:02:12,395][105692] Updated weights for policy 0, policy_version 92215 (0.0009) [2023-12-26 16:02:12,449][105692] Updated weights for policy 0, policy_version 92226 (0.0010) [2023-12-26 16:02:12,503][105692] Updated weights for policy 0, policy_version 92236 (0.0010) [2023-12-26 16:02:12,824][105620] Updated weights for policy 1, policy_version 92503 (0.0008) [2023-12-26 16:02:12,880][105620] Updated weights for policy 1, policy_version 92513 (0.0009) [2023-12-26 16:02:12,933][105620] Updated weights for policy 1, policy_version 92523 (0.0007) [2023-12-26 16:02:13,343][105692] Updated weights for policy 0, policy_version 92246 (0.0009) [2023-12-26 16:02:13,398][105692] Updated weights for policy 0, policy_version 92256 (0.0009) [2023-12-26 16:02:13,452][105692] Updated weights for policy 0, policy_version 92266 (0.0009) [2023-12-26 16:02:13,684][105620] Updated weights for policy 1, policy_version 92533 (0.0009) [2023-12-26 16:02:13,742][105620] Updated weights for policy 1, policy_version 92543 (0.0009) [2023-12-26 16:02:13,792][105620] Updated weights for policy 1, policy_version 92553 (0.0009) [2023-12-26 16:02:14,289][105692] Updated weights for policy 0, policy_version 92276 (0.0009) [2023-12-26 16:02:14,341][105692] Updated weights for policy 0, policy_version 92286 (0.0009) [2023-12-26 16:02:14,402][105692] Updated weights for policy 0, policy_version 92296 (0.0008) [2023-12-26 16:02:14,408][105620] Updated weights for policy 1, policy_version 92563 (0.0009) [2023-12-26 16:02:14,466][105620] Updated weights for policy 1, policy_version 92573 (0.0007) [2023-12-26 16:02:14,528][105620] Updated weights for policy 1, policy_version 92583 (0.0007) [2023-12-26 16:02:15,164][105620] Updated weights for policy 1, policy_version 92593 (0.0006) [2023-12-26 16:02:15,225][105620] Updated weights for policy 1, policy_version 92603 (0.0009) [2023-12-26 16:02:15,228][105692] Updated weights for policy 0, policy_version 92306 (0.0008) [2023-12-26 16:02:15,276][105620] Updated weights for policy 1, policy_version 92613 (0.0006) [2023-12-26 16:02:15,278][105692] Updated weights for policy 0, policy_version 92316 (0.0006) [2023-12-26 16:02:15,326][105620] Updated weights for policy 1, policy_version 92623 (0.0006) [2023-12-26 16:02:15,341][105692] Updated weights for policy 0, policy_version 92326 (0.0008) [2023-12-26 16:02:15,395][105692] Updated weights for policy 0, policy_version 92336 (0.0009) [2023-12-26 16:02:15,931][105620] Updated weights for policy 1, policy_version 92633 (0.0008) [2023-12-26 16:02:15,999][105620] Updated weights for policy 1, policy_version 92643 (0.0008) [2023-12-26 16:02:16,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 47357952. Throughput: 0: 9952.4, 1: 9597.4. Samples: 47334300. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:02:16,062][104569] Avg episode reward: [(0, '7285.058'), (1, '8465.394')] [2023-12-26 16:02:16,065][105620] Updated weights for policy 1, policy_version 92653 (0.0009) [2023-12-26 16:02:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000092336_23642112.pth... [2023-12-26 16:02:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000091216_23355392.pth [2023-12-26 16:02:16,081][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000092656_23724032.pth... [2023-12-26 16:02:16,085][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000091504_23429120.pth [2023-12-26 16:02:16,232][105692] Updated weights for policy 0, policy_version 92346 (0.0009) [2023-12-26 16:02:16,283][105692] Updated weights for policy 0, policy_version 92356 (0.0009) [2023-12-26 16:02:16,330][105692] Updated weights for policy 0, policy_version 92366 (0.0009) [2023-12-26 16:02:16,803][105620] Updated weights for policy 1, policy_version 92663 (0.0009) [2023-12-26 16:02:16,862][105620] Updated weights for policy 1, policy_version 92673 (0.0009) [2023-12-26 16:02:16,910][105620] Updated weights for policy 1, policy_version 92683 (0.0009) [2023-12-26 16:02:17,106][105692] Updated weights for policy 0, policy_version 92376 (0.0009) [2023-12-26 16:02:17,161][105692] Updated weights for policy 0, policy_version 92386 (0.0009) [2023-12-26 16:02:17,214][105692] Updated weights for policy 0, policy_version 92396 (0.0008) [2023-12-26 16:02:17,693][105620] Updated weights for policy 1, policy_version 92693 (0.0007) [2023-12-26 16:02:17,741][105620] Updated weights for policy 1, policy_version 92703 (0.0005) [2023-12-26 16:02:17,809][105620] Updated weights for policy 1, policy_version 92713 (0.0006) [2023-12-26 16:02:17,843][105692] Updated weights for policy 0, policy_version 92406 (0.0007) [2023-12-26 16:02:17,889][105692] Updated weights for policy 0, policy_version 92416 (0.0005) [2023-12-26 16:02:17,952][105692] Updated weights for policy 0, policy_version 92426 (0.0005) [2023-12-26 16:02:18,420][105620] Updated weights for policy 1, policy_version 92723 (0.0009) [2023-12-26 16:02:18,477][105620] Updated weights for policy 1, policy_version 92733 (0.0007) [2023-12-26 16:02:18,529][105620] Updated weights for policy 1, policy_version 92743 (0.0005) [2023-12-26 16:02:18,657][105692] Updated weights for policy 0, policy_version 92436 (0.0008) [2023-12-26 16:02:18,711][105692] Updated weights for policy 0, policy_version 92446 (0.0009) [2023-12-26 16:02:18,770][105692] Updated weights for policy 0, policy_version 92456 (0.0008) [2023-12-26 16:02:19,143][105620] Updated weights for policy 1, policy_version 92753 (0.0006) [2023-12-26 16:02:19,191][105620] Updated weights for policy 1, policy_version 92763 (0.0010) [2023-12-26 16:02:19,250][105620] Updated weights for policy 1, policy_version 92773 (0.0011) [2023-12-26 16:02:19,310][105620] Updated weights for policy 1, policy_version 92783 (0.0011) [2023-12-26 16:02:19,586][105692] Updated weights for policy 0, policy_version 92466 (0.0009) [2023-12-26 16:02:19,638][105692] Updated weights for policy 0, policy_version 92476 (0.0008) [2023-12-26 16:02:19,691][105692] Updated weights for policy 0, policy_version 92486 (0.0008) [2023-12-26 16:02:19,755][105692] Updated weights for policy 0, policy_version 92496 (0.0009) [2023-12-26 16:02:20,091][105620] Updated weights for policy 1, policy_version 92793 (0.0008) [2023-12-26 16:02:20,159][105620] Updated weights for policy 1, policy_version 92803 (0.0009) [2023-12-26 16:02:20,215][105620] Updated weights for policy 1, policy_version 92813 (0.0009) [2023-12-26 16:02:20,523][105692] Updated weights for policy 0, policy_version 92506 (0.0008) [2023-12-26 16:02:20,586][105692] Updated weights for policy 0, policy_version 92516 (0.0008) [2023-12-26 16:02:20,643][105692] Updated weights for policy 0, policy_version 92526 (0.0009) [2023-12-26 16:02:20,969][105620] Updated weights for policy 1, policy_version 92823 (0.0010) [2023-12-26 16:02:21,025][105620] Updated weights for policy 1, policy_version 92833 (0.0011) [2023-12-26 16:02:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 47456256. Throughput: 0: 9822.4, 1: 9754.7. Samples: 47451680. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:02:21,062][104569] Avg episode reward: [(0, '7741.780'), (1, '8817.873')] [2023-12-26 16:02:21,089][105620] Updated weights for policy 1, policy_version 92843 (0.0010) [2023-12-26 16:02:21,377][105692] Updated weights for policy 0, policy_version 92536 (0.0007) [2023-12-26 16:02:21,448][105692] Updated weights for policy 0, policy_version 92546 (0.0010) [2023-12-26 16:02:21,510][105692] Updated weights for policy 0, policy_version 92556 (0.0010) [2023-12-26 16:02:21,871][105620] Updated weights for policy 1, policy_version 92853 (0.0009) [2023-12-26 16:02:21,928][105620] Updated weights for policy 1, policy_version 92863 (0.0006) [2023-12-26 16:02:21,992][105620] Updated weights for policy 1, policy_version 92873 (0.0009) [2023-12-26 16:02:22,245][105692] Updated weights for policy 0, policy_version 92566 (0.0011) [2023-12-26 16:02:22,299][105692] Updated weights for policy 0, policy_version 92576 (0.0010) [2023-12-26 16:02:22,361][105692] Updated weights for policy 0, policy_version 92586 (0.0007) [2023-12-26 16:02:22,725][105620] Updated weights for policy 1, policy_version 92883 (0.0009) [2023-12-26 16:02:22,795][105620] Updated weights for policy 1, policy_version 92893 (0.0011) [2023-12-26 16:02:22,862][105620] Updated weights for policy 1, policy_version 92903 (0.0011) [2023-12-26 16:02:22,973][105692] Updated weights for policy 0, policy_version 92596 (0.0008) [2023-12-26 16:02:23,032][105692] Updated weights for policy 0, policy_version 92606 (0.0011) [2023-12-26 16:02:23,081][105692] Updated weights for policy 0, policy_version 92616 (0.0010) [2023-12-26 16:02:23,576][105620] Updated weights for policy 1, policy_version 92913 (0.0011) [2023-12-26 16:02:23,624][105620] Updated weights for policy 1, policy_version 92923 (0.0010) [2023-12-26 16:02:23,671][105620] Updated weights for policy 1, policy_version 92933 (0.0010) [2023-12-26 16:02:23,726][105692] Updated weights for policy 0, policy_version 92626 (0.0009) [2023-12-26 16:02:23,730][105620] Updated weights for policy 1, policy_version 92943 (0.0010) [2023-12-26 16:02:23,771][105692] Updated weights for policy 0, policy_version 92636 (0.0005) [2023-12-26 16:02:23,821][105692] Updated weights for policy 0, policy_version 92647 (0.0009) [2023-12-26 16:02:24,449][105620] Updated weights for policy 1, policy_version 92953 (0.0006) [2023-12-26 16:02:24,505][105620] Updated weights for policy 1, policy_version 92963 (0.0005) [2023-12-26 16:02:24,559][105620] Updated weights for policy 1, policy_version 92973 (0.0006) [2023-12-26 16:02:24,628][105692] Updated weights for policy 0, policy_version 92658 (0.0008) [2023-12-26 16:02:24,691][105692] Updated weights for policy 0, policy_version 92668 (0.0008) [2023-12-26 16:02:24,753][105692] Updated weights for policy 0, policy_version 92678 (0.0010) [2023-12-26 16:02:25,200][105620] Updated weights for policy 1, policy_version 92983 (0.0008) [2023-12-26 16:02:25,252][105620] Updated weights for policy 1, policy_version 92993 (0.0008) [2023-12-26 16:02:25,308][105620] Updated weights for policy 1, policy_version 93003 (0.0009) [2023-12-26 16:02:25,442][105692] Updated weights for policy 0, policy_version 92689 (0.0010) [2023-12-26 16:02:25,501][105692] Updated weights for policy 0, policy_version 92699 (0.0009) [2023-12-26 16:02:25,552][105692] Updated weights for policy 0, policy_version 92709 (0.0009) [2023-12-26 16:02:25,611][105692] Updated weights for policy 0, policy_version 92719 (0.0009) [2023-12-26 16:02:26,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 47554560. Throughput: 0: 9780.3, 1: 9774.6. Samples: 47567508. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:02:26,063][104569] Avg episode reward: [(0, '8103.811'), (1, '8557.762')] [2023-12-26 16:02:26,095][105620] Updated weights for policy 1, policy_version 93013 (0.0009) [2023-12-26 16:02:26,143][105620] Updated weights for policy 1, policy_version 93023 (0.0009) [2023-12-26 16:02:26,199][105620] Updated weights for policy 1, policy_version 93033 (0.0010) [2023-12-26 16:02:26,289][105692] Updated weights for policy 0, policy_version 92729 (0.0009) [2023-12-26 16:02:26,350][105692] Updated weights for policy 0, policy_version 92739 (0.0009) [2023-12-26 16:02:26,411][105692] Updated weights for policy 0, policy_version 92749 (0.0009) [2023-12-26 16:02:26,902][105620] Updated weights for policy 1, policy_version 93043 (0.0010) [2023-12-26 16:02:26,953][105620] Updated weights for policy 1, policy_version 93053 (0.0009) [2023-12-26 16:02:27,006][105620] Updated weights for policy 1, policy_version 93063 (0.0009) [2023-12-26 16:02:27,181][105692] Updated weights for policy 0, policy_version 92759 (0.0009) [2023-12-26 16:02:27,239][105692] Updated weights for policy 0, policy_version 92769 (0.0009) [2023-12-26 16:02:27,309][105692] Updated weights for policy 0, policy_version 92779 (0.0009) [2023-12-26 16:02:27,722][105620] Updated weights for policy 1, policy_version 93073 (0.0008) [2023-12-26 16:02:27,770][105620] Updated weights for policy 1, policy_version 93083 (0.0005) [2023-12-26 16:02:27,822][105620] Updated weights for policy 1, policy_version 93093 (0.0005) [2023-12-26 16:02:27,873][105620] Updated weights for policy 1, policy_version 93103 (0.0005) [2023-12-26 16:02:28,014][105692] Updated weights for policy 0, policy_version 92789 (0.0008) [2023-12-26 16:02:28,070][105692] Updated weights for policy 0, policy_version 92800 (0.0009) [2023-12-26 16:02:28,129][105692] Updated weights for policy 0, policy_version 92810 (0.0010) [2023-12-26 16:02:28,454][105620] Updated weights for policy 1, policy_version 93113 (0.0006) [2023-12-26 16:02:28,514][105620] Updated weights for policy 1, policy_version 93123 (0.0007) [2023-12-26 16:02:28,562][105620] Updated weights for policy 1, policy_version 93133 (0.0008) [2023-12-26 16:02:28,921][105692] Updated weights for policy 0, policy_version 92820 (0.0009) [2023-12-26 16:02:28,980][105692] Updated weights for policy 0, policy_version 92831 (0.0010) [2023-12-26 16:02:29,040][105692] Updated weights for policy 0, policy_version 92841 (0.0009) [2023-12-26 16:02:29,120][105620] Updated weights for policy 1, policy_version 93143 (0.0006) [2023-12-26 16:02:29,175][105620] Updated weights for policy 1, policy_version 93153 (0.0005) [2023-12-26 16:02:29,232][105620] Updated weights for policy 1, policy_version 93163 (0.0007) [2023-12-26 16:02:29,879][105692] Updated weights for policy 0, policy_version 92851 (0.0008) [2023-12-26 16:02:29,885][105620] Updated weights for policy 1, policy_version 93173 (0.0010) [2023-12-26 16:02:29,943][105692] Updated weights for policy 0, policy_version 92861 (0.0008) [2023-12-26 16:02:29,945][105620] Updated weights for policy 1, policy_version 93183 (0.0008) [2023-12-26 16:02:29,992][105692] Updated weights for policy 0, policy_version 92871 (0.0006) [2023-12-26 16:02:30,006][105620] Updated weights for policy 1, policy_version 93193 (0.0010) [2023-12-26 16:02:30,588][105692] Updated weights for policy 0, policy_version 92881 (0.0007) [2023-12-26 16:02:30,634][105692] Updated weights for policy 0, policy_version 92891 (0.0005) [2023-12-26 16:02:30,691][105692] Updated weights for policy 0, policy_version 92901 (0.0005) [2023-12-26 16:02:30,730][105620] Updated weights for policy 1, policy_version 93203 (0.0008) [2023-12-26 16:02:30,754][105692] Updated weights for policy 0, policy_version 92911 (0.0006) [2023-12-26 16:02:30,796][105620] Updated weights for policy 1, policy_version 93213 (0.0009) [2023-12-26 16:02:30,850][105620] Updated weights for policy 1, policy_version 93223 (0.0009) [2023-12-26 16:02:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 47661056. Throughput: 0: 9745.2, 1: 9836.7. Samples: 47628156. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:02:31,062][104569] Avg episode reward: [(0, '8023.518'), (1, '8192.061')] [2023-12-26 16:02:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000092912_23789568.pth... [2023-12-26 16:02:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000093232_23871488.pth... [2023-12-26 16:02:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000092080_23576576.pth [2023-12-26 16:02:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000091792_23502848.pth [2023-12-26 16:02:31,447][105692] Updated weights for policy 0, policy_version 92921 (0.0008) [2023-12-26 16:02:31,506][105692] Updated weights for policy 0, policy_version 92931 (0.0009) [2023-12-26 16:02:31,554][105620] Updated weights for policy 1, policy_version 93233 (0.0008) [2023-12-26 16:02:31,561][105692] Updated weights for policy 0, policy_version 92941 (0.0009) [2023-12-26 16:02:31,620][105620] Updated weights for policy 1, policy_version 93243 (0.0009) [2023-12-26 16:02:31,682][105620] Updated weights for policy 1, policy_version 93253 (0.0009) [2023-12-26 16:02:31,742][105620] Updated weights for policy 1, policy_version 93263 (0.0008) [2023-12-26 16:02:32,339][105692] Updated weights for policy 0, policy_version 92951 (0.0008) [2023-12-26 16:02:32,398][105692] Updated weights for policy 0, policy_version 92961 (0.0009) [2023-12-26 16:02:32,454][105692] Updated weights for policy 0, policy_version 92971 (0.0009) [2023-12-26 16:02:32,487][105620] Updated weights for policy 1, policy_version 93273 (0.0006) [2023-12-26 16:02:32,546][105620] Updated weights for policy 1, policy_version 93283 (0.0005) [2023-12-26 16:02:32,610][105620] Updated weights for policy 1, policy_version 93293 (0.0005) [2023-12-26 16:02:33,126][105620] Updated weights for policy 1, policy_version 93303 (0.0005) [2023-12-26 16:02:33,172][105620] Updated weights for policy 1, policy_version 93313 (0.0005) [2023-12-26 16:02:33,217][105620] Updated weights for policy 1, policy_version 93323 (0.0007) [2023-12-26 16:02:33,230][105692] Updated weights for policy 0, policy_version 92981 (0.0010) [2023-12-26 16:02:33,288][105692] Updated weights for policy 0, policy_version 92991 (0.0010) [2023-12-26 16:02:33,353][105692] Updated weights for policy 0, policy_version 93001 (0.0008) [2023-12-26 16:02:33,976][105620] Updated weights for policy 1, policy_version 93333 (0.0007) [2023-12-26 16:02:33,978][105692] Updated weights for policy 0, policy_version 93011 (0.0008) [2023-12-26 16:02:34,024][105620] Updated weights for policy 1, policy_version 93343 (0.0006) [2023-12-26 16:02:34,026][105692] Updated weights for policy 0, policy_version 93021 (0.0007) [2023-12-26 16:02:34,066][105620] Updated weights for policy 1, policy_version 93353 (0.0007) [2023-12-26 16:02:34,091][105692] Updated weights for policy 0, policy_version 93031 (0.0009) [2023-12-26 16:02:34,708][105692] Updated weights for policy 0, policy_version 93041 (0.0008) [2023-12-26 16:02:34,777][105692] Updated weights for policy 0, policy_version 93051 (0.0010) [2023-12-26 16:02:34,840][105692] Updated weights for policy 0, policy_version 93061 (0.0008) [2023-12-26 16:02:34,853][105620] Updated weights for policy 1, policy_version 93363 (0.0006) [2023-12-26 16:02:34,907][105692] Updated weights for policy 0, policy_version 93071 (0.0008) [2023-12-26 16:02:34,911][105620] Updated weights for policy 1, policy_version 93373 (0.0005) [2023-12-26 16:02:34,962][105620] Updated weights for policy 1, policy_version 93383 (0.0005) [2023-12-26 16:02:35,637][105620] Updated weights for policy 1, policy_version 93393 (0.0006) [2023-12-26 16:02:35,680][105692] Updated weights for policy 0, policy_version 93081 (0.0010) [2023-12-26 16:02:35,706][105620] Updated weights for policy 1, policy_version 93403 (0.0007) [2023-12-26 16:02:35,728][105692] Updated weights for policy 0, policy_version 93091 (0.0010) [2023-12-26 16:02:35,768][105620] Updated weights for policy 1, policy_version 93413 (0.0005) [2023-12-26 16:02:35,782][105692] Updated weights for policy 0, policy_version 93101 (0.0010) [2023-12-26 16:02:35,817][105620] Updated weights for policy 1, policy_version 93423 (0.0006) [2023-12-26 16:02:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 47759360. Throughput: 0: 9767.8, 1: 9910.4. Samples: 47747064. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-12-26 16:02:36,063][104569] Avg episode reward: [(0, '8017.834'), (1, '8458.151')] [2023-12-26 16:02:36,531][105692] Updated weights for policy 0, policy_version 93111 (0.0010) [2023-12-26 16:02:36,571][105620] Updated weights for policy 1, policy_version 93433 (0.0010) [2023-12-26 16:02:36,590][105692] Updated weights for policy 0, policy_version 93121 (0.0011) [2023-12-26 16:02:36,632][105620] Updated weights for policy 1, policy_version 93443 (0.0008) [2023-12-26 16:02:36,646][105692] Updated weights for policy 0, policy_version 93131 (0.0010) [2023-12-26 16:02:36,694][105620] Updated weights for policy 1, policy_version 93453 (0.0006) [2023-12-26 16:02:37,399][105692] Updated weights for policy 0, policy_version 93141 (0.0010) [2023-12-26 16:02:37,433][105620] Updated weights for policy 1, policy_version 93463 (0.0006) [2023-12-26 16:02:37,458][105692] Updated weights for policy 0, policy_version 93151 (0.0010) [2023-12-26 16:02:37,483][105620] Updated weights for policy 1, policy_version 93473 (0.0008) [2023-12-26 16:02:37,516][105692] Updated weights for policy 0, policy_version 93161 (0.0010) [2023-12-26 16:02:37,534][105620] Updated weights for policy 1, policy_version 93483 (0.0006) [2023-12-26 16:02:38,251][105692] Updated weights for policy 0, policy_version 93171 (0.0009) [2023-12-26 16:02:38,304][105692] Updated weights for policy 0, policy_version 93181 (0.0005) [2023-12-26 16:02:38,323][105620] Updated weights for policy 1, policy_version 93493 (0.0007) [2023-12-26 16:02:38,367][105692] Updated weights for policy 0, policy_version 93191 (0.0008) [2023-12-26 16:02:38,385][105620] Updated weights for policy 1, policy_version 93503 (0.0009) [2023-12-26 16:02:38,443][105620] Updated weights for policy 1, policy_version 93513 (0.0006) [2023-12-26 16:02:39,093][105692] Updated weights for policy 0, policy_version 93201 (0.0008) [2023-12-26 16:02:39,156][105692] Updated weights for policy 0, policy_version 93211 (0.0009) [2023-12-26 16:02:39,159][105620] Updated weights for policy 1, policy_version 93523 (0.0008) [2023-12-26 16:02:39,222][105692] Updated weights for policy 0, policy_version 93221 (0.0009) [2023-12-26 16:02:39,226][105620] Updated weights for policy 1, policy_version 93533 (0.0006) [2023-12-26 16:02:39,284][105620] Updated weights for policy 1, policy_version 93543 (0.0006) [2023-12-26 16:02:39,286][105692] Updated weights for policy 0, policy_version 93231 (0.0008) [2023-12-26 16:02:40,011][105692] Updated weights for policy 0, policy_version 93241 (0.0009) [2023-12-26 16:02:40,056][105620] Updated weights for policy 1, policy_version 93553 (0.0007) [2023-12-26 16:02:40,067][105692] Updated weights for policy 0, policy_version 93251 (0.0010) [2023-12-26 16:02:40,123][105620] Updated weights for policy 1, policy_version 93563 (0.0007) [2023-12-26 16:02:40,127][105692] Updated weights for policy 0, policy_version 93261 (0.0008) [2023-12-26 16:02:40,191][105620] Updated weights for policy 1, policy_version 93573 (0.0009) [2023-12-26 16:02:40,258][105620] Updated weights for policy 1, policy_version 93583 (0.0009) [2023-12-26 16:02:40,782][105692] Updated weights for policy 0, policy_version 93271 (0.0008) [2023-12-26 16:02:40,842][105692] Updated weights for policy 0, policy_version 93281 (0.0005) [2023-12-26 16:02:40,897][105692] Updated weights for policy 0, policy_version 93291 (0.0005) [2023-12-26 16:02:40,947][105620] Updated weights for policy 1, policy_version 93593 (0.0006) [2023-12-26 16:02:41,006][105620] Updated weights for policy 1, policy_version 93603 (0.0007) [2023-12-26 16:02:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 47849472. Throughput: 0: 9663.5, 1: 9825.2. Samples: 47860740. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-12-26 16:02:41,062][104569] Avg episode reward: [(0, '8187.134'), (1, '8996.303')] [2023-12-26 16:02:41,069][105620] Updated weights for policy 1, policy_version 93613 (0.0010) [2023-12-26 16:02:41,506][105692] Updated weights for policy 0, policy_version 93301 (0.0008) [2023-12-26 16:02:41,560][105692] Updated weights for policy 0, policy_version 93311 (0.0010) [2023-12-26 16:02:41,619][105692] Updated weights for policy 0, policy_version 93321 (0.0006) [2023-12-26 16:02:41,850][105620] Updated weights for policy 1, policy_version 93623 (0.0009) [2023-12-26 16:02:41,919][105620] Updated weights for policy 1, policy_version 93633 (0.0008) [2023-12-26 16:02:41,977][105620] Updated weights for policy 1, policy_version 93643 (0.0009) [2023-12-26 16:02:42,270][105692] Updated weights for policy 0, policy_version 93331 (0.0009) [2023-12-26 16:02:42,322][105692] Updated weights for policy 0, policy_version 93341 (0.0009) [2023-12-26 16:02:42,381][105692] Updated weights for policy 0, policy_version 93351 (0.0009) [2023-12-26 16:02:42,785][105620] Updated weights for policy 1, policy_version 93653 (0.0008) [2023-12-26 16:02:42,857][105620] Updated weights for policy 1, policy_version 93663 (0.0005) [2023-12-26 16:02:42,925][105620] Updated weights for policy 1, policy_version 93673 (0.0006) [2023-12-26 16:02:43,067][105692] Updated weights for policy 0, policy_version 93361 (0.0009) [2023-12-26 16:02:43,126][105692] Updated weights for policy 0, policy_version 93371 (0.0010) [2023-12-26 16:02:43,164][105585] KL-divergence is very high: 390.1270 [2023-12-26 16:02:43,189][105692] Updated weights for policy 0, policy_version 93381 (0.0009) [2023-12-26 16:02:43,190][105585] KL-divergence is very high: 182.0756 [2023-12-26 16:02:43,197][105585] KL-divergence is very high: 148.4590 [2023-12-26 16:02:43,216][105585] KL-divergence is very high: 582.9178 [2023-12-26 16:02:43,241][105585] KL-divergence is very high: 212.1614 [2023-12-26 16:02:43,248][105585] KL-divergence is very high: 153.3250 [2023-12-26 16:02:43,254][105692] Updated weights for policy 0, policy_version 93391 (0.0010) [2023-12-26 16:02:43,523][105620] Updated weights for policy 1, policy_version 93683 (0.0007) [2023-12-26 16:02:43,577][105620] Updated weights for policy 1, policy_version 93693 (0.0005) [2023-12-26 16:02:43,647][105620] Updated weights for policy 1, policy_version 93703 (0.0005) [2023-12-26 16:02:43,971][105692] Updated weights for policy 0, policy_version 93401 (0.0006) [2023-12-26 16:02:44,019][105692] Updated weights for policy 0, policy_version 93411 (0.0005) [2023-12-26 16:02:44,062][105692] Updated weights for policy 0, policy_version 93421 (0.0005) [2023-12-26 16:02:44,148][105620] Updated weights for policy 1, policy_version 93713 (0.0005) [2023-12-26 16:02:44,214][105620] Updated weights for policy 1, policy_version 93723 (0.0006) [2023-12-26 16:02:44,270][105620] Updated weights for policy 1, policy_version 93733 (0.0006) [2023-12-26 16:02:44,329][105620] Updated weights for policy 1, policy_version 93743 (0.0006) [2023-12-26 16:02:44,692][105692] Updated weights for policy 0, policy_version 93431 (0.0007) [2023-12-26 16:02:44,759][105692] Updated weights for policy 0, policy_version 93441 (0.0008) [2023-12-26 16:02:44,816][105692] Updated weights for policy 0, policy_version 93451 (0.0008) [2023-12-26 16:02:45,013][105620] Updated weights for policy 1, policy_version 93753 (0.0010) [2023-12-26 16:02:45,062][105620] Updated weights for policy 1, policy_version 93763 (0.0010) [2023-12-26 16:02:45,111][105620] Updated weights for policy 1, policy_version 93773 (0.0010) [2023-12-26 16:02:45,558][105692] Updated weights for policy 0, policy_version 93461 (0.0009) [2023-12-26 16:02:45,603][105692] Updated weights for policy 0, policy_version 93471 (0.0005) [2023-12-26 16:02:45,662][105692] Updated weights for policy 0, policy_version 93481 (0.0005) [2023-12-26 16:02:45,829][105620] Updated weights for policy 1, policy_version 93783 (0.0007) [2023-12-26 16:02:45,882][105620] Updated weights for policy 1, policy_version 93793 (0.0007) [2023-12-26 16:02:45,949][105620] Updated weights for policy 1, policy_version 93803 (0.0010) [2023-12-26 16:02:46,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 47955968. Throughput: 0: 9680.0, 1: 9827.6. Samples: 47921396. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-12-26 16:02:46,062][104569] Avg episode reward: [(0, '8179.507'), (1, '9175.538')] [2023-12-26 16:02:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000093488_23937024.pth... [2023-12-26 16:02:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000093808_24018944.pth... [2023-12-26 16:02:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000092336_23642112.pth [2023-12-26 16:02:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000092656_23724032.pth [2023-12-26 16:02:46,264][105692] Updated weights for policy 0, policy_version 93491 (0.0009) [2023-12-26 16:02:46,320][105692] Updated weights for policy 0, policy_version 93501 (0.0006) [2023-12-26 16:02:46,378][105692] Updated weights for policy 0, policy_version 93511 (0.0006) [2023-12-26 16:02:46,656][105620] Updated weights for policy 1, policy_version 93813 (0.0010) [2023-12-26 16:02:46,718][105620] Updated weights for policy 1, policy_version 93823 (0.0010) [2023-12-26 16:02:46,779][105620] Updated weights for policy 1, policy_version 93833 (0.0010) [2023-12-26 16:02:46,967][105692] Updated weights for policy 0, policy_version 93521 (0.0006) [2023-12-26 16:02:47,029][105692] Updated weights for policy 0, policy_version 93531 (0.0010) [2023-12-26 16:02:47,090][105692] Updated weights for policy 0, policy_version 93541 (0.0011) [2023-12-26 16:02:47,142][105692] Updated weights for policy 0, policy_version 93551 (0.0010) [2023-12-26 16:02:47,508][105620] Updated weights for policy 1, policy_version 93843 (0.0010) [2023-12-26 16:02:47,555][105620] Updated weights for policy 1, policy_version 93853 (0.0008) [2023-12-26 16:02:47,605][105620] Updated weights for policy 1, policy_version 93863 (0.0008) [2023-12-26 16:02:47,830][105692] Updated weights for policy 0, policy_version 93561 (0.0010) [2023-12-26 16:02:47,894][105692] Updated weights for policy 0, policy_version 93571 (0.0010) [2023-12-26 16:02:47,956][105692] Updated weights for policy 0, policy_version 93581 (0.0010) [2023-12-26 16:02:48,363][105620] Updated weights for policy 1, policy_version 93873 (0.0008) [2023-12-26 16:02:48,431][105620] Updated weights for policy 1, policy_version 93883 (0.0009) [2023-12-26 16:02:48,496][105620] Updated weights for policy 1, policy_version 93893 (0.0009) [2023-12-26 16:02:48,566][105620] Updated weights for policy 1, policy_version 93903 (0.0008) [2023-12-26 16:02:48,676][105692] Updated weights for policy 0, policy_version 93591 (0.0010) [2023-12-26 16:02:48,722][105692] Updated weights for policy 0, policy_version 93601 (0.0009) [2023-12-26 16:02:48,771][105692] Updated weights for policy 0, policy_version 93611 (0.0008) [2023-12-26 16:02:49,206][105620] Updated weights for policy 1, policy_version 93913 (0.0005) [2023-12-26 16:02:49,268][105620] Updated weights for policy 1, policy_version 93923 (0.0008) [2023-12-26 16:02:49,340][105620] Updated weights for policy 1, policy_version 93933 (0.0008) [2023-12-26 16:02:49,592][105692] Updated weights for policy 0, policy_version 93621 (0.0009) [2023-12-26 16:02:49,668][105692] Updated weights for policy 0, policy_version 93631 (0.0009) [2023-12-26 16:02:49,732][105692] Updated weights for policy 0, policy_version 93641 (0.0008) [2023-12-26 16:02:49,930][105620] Updated weights for policy 1, policy_version 93943 (0.0009) [2023-12-26 16:02:49,985][105620] Updated weights for policy 1, policy_version 93953 (0.0006) [2023-12-26 16:02:50,041][105620] Updated weights for policy 1, policy_version 93963 (0.0006) [2023-12-26 16:02:50,457][105692] Updated weights for policy 0, policy_version 93651 (0.0009) [2023-12-26 16:02:50,506][105692] Updated weights for policy 0, policy_version 93661 (0.0011) [2023-12-26 16:02:50,568][105692] Updated weights for policy 0, policy_version 93671 (0.0010) [2023-12-26 16:02:50,746][105620] Updated weights for policy 1, policy_version 93973 (0.0009) [2023-12-26 16:02:50,814][105620] Updated weights for policy 1, policy_version 93983 (0.0009) [2023-12-26 16:02:50,876][105620] Updated weights for policy 1, policy_version 93993 (0.0009) [2023-12-26 16:02:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 48054272. Throughput: 0: 9763.2, 1: 9801.5. Samples: 48042584. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-12-26 16:02:51,062][104569] Avg episode reward: [(0, '7997.433'), (1, '9262.097')] [2023-12-26 16:02:51,259][105692] Updated weights for policy 0, policy_version 93681 (0.0008) [2023-12-26 16:02:51,310][105692] Updated weights for policy 0, policy_version 93691 (0.0009) [2023-12-26 16:02:51,372][105692] Updated weights for policy 0, policy_version 93701 (0.0009) [2023-12-26 16:02:51,434][105692] Updated weights for policy 0, policy_version 93711 (0.0010) [2023-12-26 16:02:51,601][105620] Updated weights for policy 1, policy_version 94003 (0.0008) [2023-12-26 16:02:51,670][105620] Updated weights for policy 1, policy_version 94013 (0.0007) [2023-12-26 16:02:51,728][105620] Updated weights for policy 1, policy_version 94023 (0.0006) [2023-12-26 16:02:52,142][105692] Updated weights for policy 0, policy_version 93721 (0.0007) [2023-12-26 16:02:52,205][105692] Updated weights for policy 0, policy_version 93731 (0.0009) [2023-12-26 16:02:52,266][105692] Updated weights for policy 0, policy_version 93741 (0.0009) [2023-12-26 16:02:52,459][105620] Updated weights for policy 1, policy_version 94033 (0.0009) [2023-12-26 16:02:52,519][105620] Updated weights for policy 1, policy_version 94043 (0.0006) [2023-12-26 16:02:52,577][105620] Updated weights for policy 1, policy_version 94053 (0.0009) [2023-12-26 16:02:52,639][105620] Updated weights for policy 1, policy_version 94063 (0.0006) [2023-12-26 16:02:53,050][105692] Updated weights for policy 0, policy_version 93751 (0.0009) [2023-12-26 16:02:53,107][105692] Updated weights for policy 0, policy_version 93761 (0.0007) [2023-12-26 16:02:53,162][105692] Updated weights for policy 0, policy_version 93771 (0.0009) [2023-12-26 16:02:53,231][105620] Updated weights for policy 1, policy_version 94073 (0.0006) [2023-12-26 16:02:53,278][105620] Updated weights for policy 1, policy_version 94083 (0.0009) [2023-12-26 16:02:53,329][105620] Updated weights for policy 1, policy_version 94094 (0.0009) [2023-12-26 16:02:53,822][105692] Updated weights for policy 0, policy_version 93781 (0.0007) [2023-12-26 16:02:53,892][105692] Updated weights for policy 0, policy_version 93791 (0.0005) [2023-12-26 16:02:53,951][105692] Updated weights for policy 0, policy_version 93801 (0.0009) [2023-12-26 16:02:53,999][105620] Updated weights for policy 1, policy_version 94104 (0.0007) [2023-12-26 16:02:54,046][105620] Updated weights for policy 1, policy_version 94114 (0.0008) [2023-12-26 16:02:54,094][105620] Updated weights for policy 1, policy_version 94124 (0.0009) [2023-12-26 16:02:54,547][105692] Updated weights for policy 0, policy_version 93811 (0.0008) [2023-12-26 16:02:54,604][105692] Updated weights for policy 0, policy_version 93821 (0.0005) [2023-12-26 16:02:54,663][105692] Updated weights for policy 0, policy_version 93831 (0.0005) [2023-12-26 16:02:54,911][105620] Updated weights for policy 1, policy_version 94134 (0.0009) [2023-12-26 16:02:54,968][105620] Updated weights for policy 1, policy_version 94144 (0.0009) [2023-12-26 16:02:55,016][105620] Updated weights for policy 1, policy_version 94154 (0.0009) [2023-12-26 16:02:55,306][105692] Updated weights for policy 0, policy_version 93841 (0.0006) [2023-12-26 16:02:55,360][105692] Updated weights for policy 0, policy_version 93851 (0.0009) [2023-12-26 16:02:55,406][105692] Updated weights for policy 0, policy_version 93861 (0.0008) [2023-12-26 16:02:55,473][105692] Updated weights for policy 0, policy_version 93871 (0.0008) [2023-12-26 16:02:55,792][105620] Updated weights for policy 1, policy_version 94164 (0.0009) [2023-12-26 16:02:55,856][105620] Updated weights for policy 1, policy_version 94174 (0.0006) [2023-12-26 16:02:55,910][105620] Updated weights for policy 1, policy_version 94184 (0.0005) [2023-12-26 16:02:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 48152576. Throughput: 0: 9688.4, 1: 9893.5. Samples: 48160352. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-12-26 16:02:56,062][104569] Avg episode reward: [(0, '8187.404'), (1, '9352.633')] [2023-12-26 16:02:56,286][105692] Updated weights for policy 0, policy_version 93881 (0.0010) [2023-12-26 16:02:56,339][105692] Updated weights for policy 0, policy_version 93891 (0.0009) [2023-12-26 16:02:56,398][105692] Updated weights for policy 0, policy_version 93901 (0.0010) [2023-12-26 16:02:56,477][105620] Updated weights for policy 1, policy_version 94194 (0.0005) [2023-12-26 16:02:56,526][105620] Updated weights for policy 1, policy_version 94204 (0.0005) [2023-12-26 16:02:56,580][105620] Updated weights for policy 1, policy_version 94214 (0.0005) [2023-12-26 16:02:56,631][105620] Updated weights for policy 1, policy_version 94224 (0.0010) [2023-12-26 16:02:57,122][105692] Updated weights for policy 0, policy_version 93911 (0.0007) [2023-12-26 16:02:57,181][105692] Updated weights for policy 0, policy_version 93921 (0.0005) [2023-12-26 16:02:57,235][105692] Updated weights for policy 0, policy_version 93931 (0.0005) [2023-12-26 16:02:57,350][105620] Updated weights for policy 1, policy_version 94234 (0.0006) [2023-12-26 16:02:57,397][105620] Updated weights for policy 1, policy_version 94244 (0.0006) [2023-12-26 16:02:57,444][105620] Updated weights for policy 1, policy_version 94254 (0.0005) [2023-12-26 16:02:57,772][105692] Updated weights for policy 0, policy_version 93941 (0.0006) [2023-12-26 16:02:57,819][105692] Updated weights for policy 0, policy_version 93951 (0.0008) [2023-12-26 16:02:57,874][105692] Updated weights for policy 0, policy_version 93961 (0.0008) [2023-12-26 16:02:58,070][105620] Updated weights for policy 1, policy_version 94264 (0.0009) [2023-12-26 16:02:58,131][105620] Updated weights for policy 1, policy_version 94274 (0.0009) [2023-12-26 16:02:58,191][105620] Updated weights for policy 1, policy_version 94284 (0.0010) [2023-12-26 16:02:58,642][105692] Updated weights for policy 0, policy_version 93971 (0.0008) [2023-12-26 16:02:58,708][105692] Updated weights for policy 0, policy_version 93981 (0.0008) [2023-12-26 16:02:58,777][105692] Updated weights for policy 0, policy_version 93991 (0.0007) [2023-12-26 16:02:59,028][105620] Updated weights for policy 1, policy_version 94294 (0.0010) [2023-12-26 16:02:59,082][105620] Updated weights for policy 1, policy_version 94304 (0.0008) [2023-12-26 16:02:59,143][105620] Updated weights for policy 1, policy_version 94314 (0.0007) [2023-12-26 16:02:59,633][105692] Updated weights for policy 0, policy_version 94001 (0.0007) [2023-12-26 16:02:59,687][105692] Updated weights for policy 0, policy_version 94011 (0.0010) [2023-12-26 16:02:59,738][105692] Updated weights for policy 0, policy_version 94021 (0.0009) [2023-12-26 16:02:59,793][105692] Updated weights for policy 0, policy_version 94031 (0.0009) [2023-12-26 16:02:59,803][105620] Updated weights for policy 1, policy_version 94324 (0.0007) [2023-12-26 16:02:59,875][105620] Updated weights for policy 1, policy_version 94334 (0.0008) [2023-12-26 16:02:59,948][105620] Updated weights for policy 1, policy_version 94344 (0.0009) [2023-12-26 16:03:00,547][105692] Updated weights for policy 0, policy_version 94041 (0.0007) [2023-12-26 16:03:00,594][105692] Updated weights for policy 0, policy_version 94051 (0.0007) [2023-12-26 16:03:00,649][105692] Updated weights for policy 0, policy_version 94061 (0.0008) [2023-12-26 16:03:00,663][105620] Updated weights for policy 1, policy_version 94354 (0.0010) [2023-12-26 16:03:00,712][105620] Updated weights for policy 1, policy_version 94364 (0.0007) [2023-12-26 16:03:00,760][105620] Updated weights for policy 1, policy_version 94374 (0.0005) [2023-12-26 16:03:00,810][105620] Updated weights for policy 1, policy_version 94384 (0.0005) [2023-12-26 16:03:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 48250880. Throughput: 0: 9759.9, 1: 9938.7. Samples: 48220740. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-12-26 16:03:01,062][104569] Avg episode reward: [(0, '8090.225'), (1, '9181.628')] [2023-12-26 16:03:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000094384_24166400.pth... [2023-12-26 16:03:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000094064_24084480.pth... [2023-12-26 16:03:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000092912_23789568.pth [2023-12-26 16:03:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000093232_23871488.pth [2023-12-26 16:03:01,365][105692] Updated weights for policy 0, policy_version 94071 (0.0010) [2023-12-26 16:03:01,416][105620] Updated weights for policy 1, policy_version 94394 (0.0005) [2023-12-26 16:03:01,424][105692] Updated weights for policy 0, policy_version 94081 (0.0008) [2023-12-26 16:03:01,478][105620] Updated weights for policy 1, policy_version 94404 (0.0006) [2023-12-26 16:03:01,484][105692] Updated weights for policy 0, policy_version 94091 (0.0006) [2023-12-26 16:03:01,531][105620] Updated weights for policy 1, policy_version 94414 (0.0010) [2023-12-26 16:03:02,128][105620] Updated weights for policy 1, policy_version 94424 (0.0006) [2023-12-26 16:03:02,182][105620] Updated weights for policy 1, policy_version 94434 (0.0008) [2023-12-26 16:03:02,221][105692] Updated weights for policy 0, policy_version 94101 (0.0008) [2023-12-26 16:03:02,245][105620] Updated weights for policy 1, policy_version 94444 (0.0006) [2023-12-26 16:03:02,280][105692] Updated weights for policy 0, policy_version 94111 (0.0009) [2023-12-26 16:03:02,334][105692] Updated weights for policy 0, policy_version 94121 (0.0007) [2023-12-26 16:03:02,841][105620] Updated weights for policy 1, policy_version 94454 (0.0007) [2023-12-26 16:03:02,894][105620] Updated weights for policy 1, policy_version 94464 (0.0005) [2023-12-26 16:03:02,944][105620] Updated weights for policy 1, policy_version 94474 (0.0005) [2023-12-26 16:03:03,123][105692] Updated weights for policy 0, policy_version 94131 (0.0009) [2023-12-26 16:03:03,176][105692] Updated weights for policy 0, policy_version 94141 (0.0010) [2023-12-26 16:03:03,229][105692] Updated weights for policy 0, policy_version 94151 (0.0010) [2023-12-26 16:03:03,565][105620] Updated weights for policy 1, policy_version 94484 (0.0007) [2023-12-26 16:03:03,615][105620] Updated weights for policy 1, policy_version 94494 (0.0005) [2023-12-26 16:03:03,675][105620] Updated weights for policy 1, policy_version 94504 (0.0005) [2023-12-26 16:03:03,921][105692] Updated weights for policy 0, policy_version 94161 (0.0010) [2023-12-26 16:03:03,989][105692] Updated weights for policy 0, policy_version 94171 (0.0007) [2023-12-26 16:03:04,048][105692] Updated weights for policy 0, policy_version 94181 (0.0010) [2023-12-26 16:03:04,116][105692] Updated weights for policy 0, policy_version 94191 (0.0011) [2023-12-26 16:03:04,404][105620] Updated weights for policy 1, policy_version 94514 (0.0009) [2023-12-26 16:03:04,456][105620] Updated weights for policy 1, policy_version 94524 (0.0010) [2023-12-26 16:03:04,508][105620] Updated weights for policy 1, policy_version 94534 (0.0010) [2023-12-26 16:03:04,563][105620] Updated weights for policy 1, policy_version 94544 (0.0008) [2023-12-26 16:03:04,781][105692] Updated weights for policy 0, policy_version 94201 (0.0006) [2023-12-26 16:03:04,838][105692] Updated weights for policy 0, policy_version 94211 (0.0007) [2023-12-26 16:03:04,894][105692] Updated weights for policy 0, policy_version 94221 (0.0011) [2023-12-26 16:03:05,173][105620] Updated weights for policy 1, policy_version 94554 (0.0007) [2023-12-26 16:03:05,228][105620] Updated weights for policy 1, policy_version 94564 (0.0005) [2023-12-26 16:03:05,291][105620] Updated weights for policy 1, policy_version 94574 (0.0005) [2023-12-26 16:03:05,439][105692] Updated weights for policy 0, policy_version 94231 (0.0007) [2023-12-26 16:03:05,490][105692] Updated weights for policy 0, policy_version 94241 (0.0005) [2023-12-26 16:03:05,557][105692] Updated weights for policy 0, policy_version 94251 (0.0005) [2023-12-26 16:03:06,020][105620] Updated weights for policy 1, policy_version 94584 (0.0008) [2023-12-26 16:03:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 48349184. Throughput: 0: 9789.0, 1: 9984.8. Samples: 48341500. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-12-26 16:03:06,063][104569] Avg episode reward: [(0, '8268.431'), (1, '9181.565')] [2023-12-26 16:03:06,084][105620] Updated weights for policy 1, policy_version 94594 (0.0008) [2023-12-26 16:03:06,146][105620] Updated weights for policy 1, policy_version 94604 (0.0008) [2023-12-26 16:03:06,205][105692] Updated weights for policy 0, policy_version 94261 (0.0008) [2023-12-26 16:03:06,260][105692] Updated weights for policy 0, policy_version 94271 (0.0010) [2023-12-26 16:03:06,309][105692] Updated weights for policy 0, policy_version 94281 (0.0010) [2023-12-26 16:03:06,929][105620] Updated weights for policy 1, policy_version 94614 (0.0008) [2023-12-26 16:03:06,972][105692] Updated weights for policy 0, policy_version 94291 (0.0007) [2023-12-26 16:03:06,998][105620] Updated weights for policy 1, policy_version 94624 (0.0007) [2023-12-26 16:03:07,031][105692] Updated weights for policy 0, policy_version 94301 (0.0010) [2023-12-26 16:03:07,059][105620] Updated weights for policy 1, policy_version 94634 (0.0008) [2023-12-26 16:03:07,101][105692] Updated weights for policy 0, policy_version 94311 (0.0011) [2023-12-26 16:03:07,644][105620] Updated weights for policy 1, policy_version 94644 (0.0007) [2023-12-26 16:03:07,705][105620] Updated weights for policy 1, policy_version 94654 (0.0005) [2023-12-26 16:03:07,755][105620] Updated weights for policy 1, policy_version 94664 (0.0005) [2023-12-26 16:03:07,836][105692] Updated weights for policy 0, policy_version 94321 (0.0010) [2023-12-26 16:03:07,890][105692] Updated weights for policy 0, policy_version 94331 (0.0010) [2023-12-26 16:03:07,945][105692] Updated weights for policy 0, policy_version 94341 (0.0010) [2023-12-26 16:03:08,004][105692] Updated weights for policy 0, policy_version 94351 (0.0010) [2023-12-26 16:03:08,499][105620] Updated weights for policy 1, policy_version 94674 (0.0006) [2023-12-26 16:03:08,564][105620] Updated weights for policy 1, policy_version 94684 (0.0008) [2023-12-26 16:03:08,586][105692] Updated weights for policy 0, policy_version 94361 (0.0006) [2023-12-26 16:03:08,624][105620] Updated weights for policy 1, policy_version 94694 (0.0008) [2023-12-26 16:03:08,649][105692] Updated weights for policy 0, policy_version 94371 (0.0011) [2023-12-26 16:03:08,675][105620] Updated weights for policy 1, policy_version 94704 (0.0006) [2023-12-26 16:03:08,711][105692] Updated weights for policy 0, policy_version 94381 (0.0010) [2023-12-26 16:03:09,429][105620] Updated weights for policy 1, policy_version 94714 (0.0008) [2023-12-26 16:03:09,435][105692] Updated weights for policy 0, policy_version 94391 (0.0010) [2023-12-26 16:03:09,479][105620] Updated weights for policy 1, policy_version 94724 (0.0006) [2023-12-26 16:03:09,485][105692] Updated weights for policy 0, policy_version 94401 (0.0007) [2023-12-26 16:03:09,536][105620] Updated weights for policy 1, policy_version 94734 (0.0007) [2023-12-26 16:03:09,543][105692] Updated weights for policy 0, policy_version 94411 (0.0007) [2023-12-26 16:03:10,229][105692] Updated weights for policy 0, policy_version 94421 (0.0008) [2023-12-26 16:03:10,287][105692] Updated weights for policy 0, policy_version 94431 (0.0009) [2023-12-26 16:03:10,343][105692] Updated weights for policy 0, policy_version 94441 (0.0008) [2023-12-26 16:03:10,362][105620] Updated weights for policy 1, policy_version 94744 (0.0007) [2023-12-26 16:03:10,414][105620] Updated weights for policy 1, policy_version 94754 (0.0008) [2023-12-26 16:03:10,470][105620] Updated weights for policy 1, policy_version 94764 (0.0009) [2023-12-26 16:03:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 48447488. Throughput: 0: 9875.5, 1: 9960.1. Samples: 48460104. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-12-26 16:03:11,062][104569] Avg episode reward: [(0, '7896.601'), (1, '9269.615')] [2023-12-26 16:03:11,106][105692] Updated weights for policy 0, policy_version 94451 (0.0008) [2023-12-26 16:03:11,168][105692] Updated weights for policy 0, policy_version 94461 (0.0009) [2023-12-26 16:03:11,212][105620] Updated weights for policy 1, policy_version 94774 (0.0007) [2023-12-26 16:03:11,224][105692] Updated weights for policy 0, policy_version 94471 (0.0008) [2023-12-26 16:03:11,272][105620] Updated weights for policy 1, policy_version 94784 (0.0007) [2023-12-26 16:03:11,332][105620] Updated weights for policy 1, policy_version 94794 (0.0009) [2023-12-26 16:03:11,978][105692] Updated weights for policy 0, policy_version 94481 (0.0008) [2023-12-26 16:03:12,039][105692] Updated weights for policy 0, policy_version 94491 (0.0008) [2023-12-26 16:03:12,068][105620] Updated weights for policy 1, policy_version 94804 (0.0007) [2023-12-26 16:03:12,100][105692] Updated weights for policy 0, policy_version 94501 (0.0008) [2023-12-26 16:03:12,128][105620] Updated weights for policy 1, policy_version 94814 (0.0007) [2023-12-26 16:03:12,161][105692] Updated weights for policy 0, policy_version 94511 (0.0008) [2023-12-26 16:03:12,180][105620] Updated weights for policy 1, policy_version 94824 (0.0009) [2023-12-26 16:03:12,871][105692] Updated weights for policy 0, policy_version 94521 (0.0007) [2023-12-26 16:03:12,931][105692] Updated weights for policy 0, policy_version 94531 (0.0008) [2023-12-26 16:03:12,950][105620] Updated weights for policy 1, policy_version 94834 (0.0008) [2023-12-26 16:03:12,991][105692] Updated weights for policy 0, policy_version 94541 (0.0008) [2023-12-26 16:03:13,014][105620] Updated weights for policy 1, policy_version 94844 (0.0007) [2023-12-26 16:03:13,076][105620] Updated weights for policy 1, policy_version 94854 (0.0006) [2023-12-26 16:03:13,144][105620] Updated weights for policy 1, policy_version 94864 (0.0007) [2023-12-26 16:03:13,559][105692] Updated weights for policy 0, policy_version 94551 (0.0006) [2023-12-26 16:03:13,607][105692] Updated weights for policy 0, policy_version 94561 (0.0005) [2023-12-26 16:03:13,658][105692] Updated weights for policy 0, policy_version 94571 (0.0009) [2023-12-26 16:03:13,855][105620] Updated weights for policy 1, policy_version 94874 (0.0009) [2023-12-26 16:03:13,902][105620] Updated weights for policy 1, policy_version 94884 (0.0010) [2023-12-26 16:03:13,956][105620] Updated weights for policy 1, policy_version 94894 (0.0010) [2023-12-26 16:03:14,362][105692] Updated weights for policy 0, policy_version 94581 (0.0010) [2023-12-26 16:03:14,415][105692] Updated weights for policy 0, policy_version 94591 (0.0010) [2023-12-26 16:03:14,477][105692] Updated weights for policy 0, policy_version 94601 (0.0010) [2023-12-26 16:03:14,634][105620] Updated weights for policy 1, policy_version 94904 (0.0006) [2023-12-26 16:03:14,692][105620] Updated weights for policy 1, policy_version 94914 (0.0007) [2023-12-26 16:03:14,737][105620] Updated weights for policy 1, policy_version 94924 (0.0007) [2023-12-26 16:03:15,253][105692] Updated weights for policy 0, policy_version 94611 (0.0010) [2023-12-26 16:03:15,309][105692] Updated weights for policy 0, policy_version 94621 (0.0010) [2023-12-26 16:03:15,371][105692] Updated weights for policy 0, policy_version 94631 (0.0010) [2023-12-26 16:03:15,381][105620] Updated weights for policy 1, policy_version 94934 (0.0009) [2023-12-26 16:03:15,446][105620] Updated weights for policy 1, policy_version 94944 (0.0011) [2023-12-26 16:03:15,511][105620] Updated weights for policy 1, policy_version 94954 (0.0011) [2023-12-26 16:03:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 48545792. Throughput: 0: 9897.0, 1: 9889.6. Samples: 48518556. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-12-26 16:03:16,063][104569] Avg episode reward: [(0, '7979.690'), (1, '9352.361')] [2023-12-26 16:03:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000094640_24231936.pth... [2023-12-26 16:03:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000094960_24313856.pth... [2023-12-26 16:03:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000093808_24018944.pth [2023-12-26 16:03:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000093488_23937024.pth [2023-12-26 16:03:16,129][105692] Updated weights for policy 0, policy_version 94641 (0.0010) [2023-12-26 16:03:16,190][105620] Updated weights for policy 1, policy_version 94964 (0.0009) [2023-12-26 16:03:16,194][105692] Updated weights for policy 0, policy_version 94651 (0.0005) [2023-12-26 16:03:16,236][105620] Updated weights for policy 1, policy_version 94974 (0.0005) [2023-12-26 16:03:16,249][105692] Updated weights for policy 0, policy_version 94661 (0.0009) [2023-12-26 16:03:16,289][105620] Updated weights for policy 1, policy_version 94984 (0.0007) [2023-12-26 16:03:16,297][105692] Updated weights for policy 0, policy_version 94671 (0.0010) [2023-12-26 16:03:16,925][105692] Updated weights for policy 0, policy_version 94681 (0.0006) [2023-12-26 16:03:16,978][105692] Updated weights for policy 0, policy_version 94691 (0.0006) [2023-12-26 16:03:17,011][105620] Updated weights for policy 1, policy_version 94994 (0.0007) [2023-12-26 16:03:17,030][105692] Updated weights for policy 0, policy_version 94701 (0.0011) [2023-12-26 16:03:17,065][105620] Updated weights for policy 1, policy_version 95004 (0.0006) [2023-12-26 16:03:17,124][105620] Updated weights for policy 1, policy_version 95014 (0.0008) [2023-12-26 16:03:17,180][105620] Updated weights for policy 1, policy_version 95024 (0.0009) [2023-12-26 16:03:17,641][105692] Updated weights for policy 0, policy_version 94711 (0.0009) [2023-12-26 16:03:17,688][105692] Updated weights for policy 0, policy_version 94721 (0.0008) [2023-12-26 16:03:17,736][105692] Updated weights for policy 0, policy_version 94731 (0.0009) [2023-12-26 16:03:17,967][105620] Updated weights for policy 1, policy_version 95034 (0.0009) [2023-12-26 16:03:18,032][105620] Updated weights for policy 1, policy_version 95044 (0.0009) [2023-12-26 16:03:18,097][105620] Updated weights for policy 1, policy_version 95054 (0.0009) [2023-12-26 16:03:18,506][105692] Updated weights for policy 0, policy_version 94741 (0.0007) [2023-12-26 16:03:18,559][105692] Updated weights for policy 0, policy_version 94751 (0.0008) [2023-12-26 16:03:18,607][105692] Updated weights for policy 0, policy_version 94761 (0.0005) [2023-12-26 16:03:18,755][105620] Updated weights for policy 1, policy_version 95064 (0.0009) [2023-12-26 16:03:18,820][105620] Updated weights for policy 1, policy_version 95074 (0.0009) [2023-12-26 16:03:18,882][105620] Updated weights for policy 1, policy_version 95084 (0.0009) [2023-12-26 16:03:19,272][105692] Updated weights for policy 0, policy_version 94771 (0.0005) [2023-12-26 16:03:19,330][105692] Updated weights for policy 0, policy_version 94781 (0.0006) [2023-12-26 16:03:19,399][105692] Updated weights for policy 0, policy_version 94791 (0.0009) [2023-12-26 16:03:19,664][105620] Updated weights for policy 1, policy_version 95094 (0.0010) [2023-12-26 16:03:19,729][105620] Updated weights for policy 1, policy_version 95104 (0.0010) [2023-12-26 16:03:19,796][105620] Updated weights for policy 1, policy_version 95114 (0.0010) [2023-12-26 16:03:20,084][105692] Updated weights for policy 0, policy_version 94801 (0.0010) [2023-12-26 16:03:20,137][105692] Updated weights for policy 0, policy_version 94811 (0.0010) [2023-12-26 16:03:20,189][105692] Updated weights for policy 0, policy_version 94821 (0.0010) [2023-12-26 16:03:20,245][105692] Updated weights for policy 0, policy_version 94831 (0.0011) [2023-12-26 16:03:20,525][105620] Updated weights for policy 1, policy_version 95124 (0.0009) [2023-12-26 16:03:20,595][105620] Updated weights for policy 1, policy_version 95134 (0.0011) [2023-12-26 16:03:20,658][105620] Updated weights for policy 1, policy_version 95144 (0.0011) [2023-12-26 16:03:21,043][105692] Updated weights for policy 0, policy_version 94841 (0.0009) [2023-12-26 16:03:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 48644096. Throughput: 0: 9919.6, 1: 9855.0. Samples: 48636916. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-12-26 16:03:21,063][104569] Avg episode reward: [(0, '8530.706'), (1, '9260.119')] [2023-12-26 16:03:21,107][105692] Updated weights for policy 0, policy_version 94851 (0.0009) [2023-12-26 16:03:21,176][105692] Updated weights for policy 0, policy_version 94861 (0.0009) [2023-12-26 16:03:21,324][105620] Updated weights for policy 1, policy_version 95154 (0.0010) [2023-12-26 16:03:21,397][105620] Updated weights for policy 1, policy_version 95164 (0.0008) [2023-12-26 16:03:21,455][105620] Updated weights for policy 1, policy_version 95174 (0.0010) [2023-12-26 16:03:21,516][105620] Updated weights for policy 1, policy_version 95184 (0.0010) [2023-12-26 16:03:21,873][105692] Updated weights for policy 0, policy_version 94871 (0.0009) [2023-12-26 16:03:21,941][105692] Updated weights for policy 0, policy_version 94881 (0.0009) [2023-12-26 16:03:22,004][105692] Updated weights for policy 0, policy_version 94891 (0.0008) [2023-12-26 16:03:22,293][105620] Updated weights for policy 1, policy_version 95194 (0.0009) [2023-12-26 16:03:22,355][105620] Updated weights for policy 1, policy_version 95204 (0.0009) [2023-12-26 16:03:22,419][105620] Updated weights for policy 1, policy_version 95214 (0.0009) [2023-12-26 16:03:22,736][105692] Updated weights for policy 0, policy_version 94901 (0.0007) [2023-12-26 16:03:22,799][105692] Updated weights for policy 0, policy_version 94911 (0.0006) [2023-12-26 16:03:22,859][105692] Updated weights for policy 0, policy_version 94921 (0.0005) [2023-12-26 16:03:23,220][105620] Updated weights for policy 1, policy_version 95224 (0.0009) [2023-12-26 16:03:23,281][105620] Updated weights for policy 1, policy_version 95234 (0.0010) [2023-12-26 16:03:23,339][105620] Updated weights for policy 1, policy_version 95245 (0.0010) [2023-12-26 16:03:23,413][105692] Updated weights for policy 0, policy_version 94931 (0.0007) [2023-12-26 16:03:23,462][105692] Updated weights for policy 0, policy_version 94941 (0.0005) [2023-12-26 16:03:23,510][105692] Updated weights for policy 0, policy_version 94951 (0.0005) [2023-12-26 16:03:24,065][105692] Updated weights for policy 0, policy_version 94961 (0.0006) [2023-12-26 16:03:24,122][105692] Updated weights for policy 0, policy_version 94971 (0.0008) [2023-12-26 16:03:24,187][105692] Updated weights for policy 0, policy_version 94981 (0.0009) [2023-12-26 16:03:24,213][105620] Updated weights for policy 1, policy_version 95255 (0.0007) [2023-12-26 16:03:24,245][105692] Updated weights for policy 0, policy_version 94991 (0.0007) [2023-12-26 16:03:24,260][105620] Updated weights for policy 1, policy_version 95265 (0.0008) [2023-12-26 16:03:24,304][105620] Updated weights for policy 1, policy_version 95275 (0.0009) [2023-12-26 16:03:24,910][105620] Updated weights for policy 1, policy_version 95285 (0.0007) [2023-12-26 16:03:24,962][105620] Updated weights for policy 1, policy_version 95295 (0.0005) [2023-12-26 16:03:25,021][105620] Updated weights for policy 1, policy_version 95305 (0.0005) [2023-12-26 16:03:25,092][105692] Updated weights for policy 0, policy_version 95001 (0.0009) [2023-12-26 16:03:25,145][105692] Updated weights for policy 0, policy_version 95011 (0.0009) [2023-12-26 16:03:25,204][105692] Updated weights for policy 0, policy_version 95021 (0.0010) [2023-12-26 16:03:25,530][105620] Updated weights for policy 1, policy_version 95315 (0.0005) [2023-12-26 16:03:25,587][105620] Updated weights for policy 1, policy_version 95325 (0.0008) [2023-12-26 16:03:25,653][105620] Updated weights for policy 1, policy_version 95335 (0.0009) [2023-12-26 16:03:25,920][105692] Updated weights for policy 0, policy_version 95031 (0.0009) [2023-12-26 16:03:25,979][105692] Updated weights for policy 0, policy_version 95041 (0.0008) [2023-12-26 16:03:26,039][105692] Updated weights for policy 0, policy_version 95051 (0.0009) [2023-12-26 16:03:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 48742400. Throughput: 0: 9964.2, 1: 9901.7. Samples: 48754708. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-12-26 16:03:26,063][104569] Avg episode reward: [(0, '8996.613'), (1, '9260.026')] [2023-12-26 16:03:26,373][105620] Updated weights for policy 1, policy_version 95345 (0.0009) [2023-12-26 16:03:26,424][105620] Updated weights for policy 1, policy_version 95355 (0.0009) [2023-12-26 16:03:26,476][105620] Updated weights for policy 1, policy_version 95365 (0.0010) [2023-12-26 16:03:26,531][105620] Updated weights for policy 1, policy_version 95375 (0.0010) [2023-12-26 16:03:26,735][105692] Updated weights for policy 0, policy_version 95061 (0.0009) [2023-12-26 16:03:26,784][105692] Updated weights for policy 0, policy_version 95071 (0.0008) [2023-12-26 16:03:26,837][105692] Updated weights for policy 0, policy_version 95081 (0.0006) [2023-12-26 16:03:27,282][105620] Updated weights for policy 1, policy_version 95385 (0.0009) [2023-12-26 16:03:27,341][105620] Updated weights for policy 1, policy_version 95395 (0.0009) [2023-12-26 16:03:27,404][105620] Updated weights for policy 1, policy_version 95406 (0.0007) [2023-12-26 16:03:27,604][105692] Updated weights for policy 0, policy_version 95091 (0.0008) [2023-12-26 16:03:27,665][105692] Updated weights for policy 0, policy_version 95101 (0.0010) [2023-12-26 16:03:27,718][105692] Updated weights for policy 0, policy_version 95111 (0.0010) [2023-12-26 16:03:27,997][105620] Updated weights for policy 1, policy_version 95416 (0.0008) [2023-12-26 16:03:28,046][105620] Updated weights for policy 1, policy_version 95426 (0.0009) [2023-12-26 16:03:28,092][105620] Updated weights for policy 1, policy_version 95436 (0.0008) [2023-12-26 16:03:28,509][105692] Updated weights for policy 0, policy_version 95121 (0.0009) [2023-12-26 16:03:28,560][105692] Updated weights for policy 0, policy_version 95131 (0.0005) [2023-12-26 16:03:28,618][105692] Updated weights for policy 0, policy_version 95141 (0.0009) [2023-12-26 16:03:28,675][105692] Updated weights for policy 0, policy_version 95151 (0.0009) [2023-12-26 16:03:28,870][105620] Updated weights for policy 1, policy_version 95446 (0.0009) [2023-12-26 16:03:28,920][105620] Updated weights for policy 1, policy_version 95456 (0.0010) [2023-12-26 16:03:28,976][105620] Updated weights for policy 1, policy_version 95466 (0.0010) [2023-12-26 16:03:29,425][105692] Updated weights for policy 0, policy_version 95161 (0.0006) [2023-12-26 16:03:29,484][105692] Updated weights for policy 0, policy_version 95171 (0.0006) [2023-12-26 16:03:29,541][105692] Updated weights for policy 0, policy_version 95181 (0.0008) [2023-12-26 16:03:29,778][105620] Updated weights for policy 1, policy_version 95476 (0.0010) [2023-12-26 16:03:29,840][105620] Updated weights for policy 1, policy_version 95486 (0.0011) [2023-12-26 16:03:29,905][105620] Updated weights for policy 1, policy_version 95496 (0.0010) [2023-12-26 16:03:30,211][105692] Updated weights for policy 0, policy_version 95191 (0.0008) [2023-12-26 16:03:30,270][105692] Updated weights for policy 0, policy_version 95201 (0.0009) [2023-12-26 16:03:30,320][105692] Updated weights for policy 0, policy_version 95211 (0.0009) [2023-12-26 16:03:30,586][105620] Updated weights for policy 1, policy_version 95506 (0.0009) [2023-12-26 16:03:30,641][105620] Updated weights for policy 1, policy_version 95516 (0.0006) [2023-12-26 16:03:30,700][105620] Updated weights for policy 1, policy_version 95526 (0.0005) [2023-12-26 16:03:30,746][105620] Updated weights for policy 1, policy_version 95536 (0.0005) [2023-12-26 16:03:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 48840704. Throughput: 0: 9911.2, 1: 9909.0. Samples: 48813308. Policy #0 lag: (min: 26.0, avg: 39.4, max: 40.0) [2023-12-26 16:03:31,063][104569] Avg episode reward: [(0, '9088.106'), (1, '9169.749')] [2023-12-26 16:03:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000095216_24379392.pth... [2023-12-26 16:03:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000095536_24461312.pth... [2023-12-26 16:03:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000094064_24084480.pth [2023-12-26 16:03:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000094384_24166400.pth [2023-12-26 16:03:31,189][105692] Updated weights for policy 0, policy_version 95221 (0.0009) [2023-12-26 16:03:31,240][105692] Updated weights for policy 0, policy_version 95231 (0.0008) [2023-12-26 16:03:31,296][105692] Updated weights for policy 0, policy_version 95241 (0.0006) [2023-12-26 16:03:31,306][105620] Updated weights for policy 1, policy_version 95546 (0.0011) [2023-12-26 16:03:31,368][105620] Updated weights for policy 1, policy_version 95556 (0.0010) [2023-12-26 16:03:31,430][105620] Updated weights for policy 1, policy_version 95566 (0.0009) [2023-12-26 16:03:32,094][105692] Updated weights for policy 0, policy_version 95251 (0.0006) [2023-12-26 16:03:32,156][105692] Updated weights for policy 0, policy_version 95261 (0.0007) [2023-12-26 16:03:32,166][105620] Updated weights for policy 1, policy_version 95576 (0.0006) [2023-12-26 16:03:32,217][105692] Updated weights for policy 0, policy_version 95271 (0.0007) [2023-12-26 16:03:32,227][105620] Updated weights for policy 1, policy_version 95586 (0.0007) [2023-12-26 16:03:32,289][105620] Updated weights for policy 1, policy_version 95596 (0.0008) [2023-12-26 16:03:32,884][105692] Updated weights for policy 0, policy_version 95281 (0.0007) [2023-12-26 16:03:32,942][105692] Updated weights for policy 0, policy_version 95291 (0.0010) [2023-12-26 16:03:32,993][105692] Updated weights for policy 0, policy_version 95301 (0.0009) [2023-12-26 16:03:33,040][105620] Updated weights for policy 1, policy_version 95606 (0.0007) [2023-12-26 16:03:33,043][105692] Updated weights for policy 0, policy_version 95311 (0.0010) [2023-12-26 16:03:33,100][105620] Updated weights for policy 1, policy_version 95616 (0.0005) [2023-12-26 16:03:33,160][105620] Updated weights for policy 1, policy_version 95626 (0.0007) [2023-12-26 16:03:33,748][105620] Updated weights for policy 1, policy_version 95636 (0.0006) [2023-12-26 16:03:33,798][105620] Updated weights for policy 1, policy_version 95646 (0.0005) [2023-12-26 16:03:33,841][105620] Updated weights for policy 1, policy_version 95656 (0.0005) [2023-12-26 16:03:33,887][105692] Updated weights for policy 0, policy_version 95321 (0.0008) [2023-12-26 16:03:33,938][105692] Updated weights for policy 0, policy_version 95331 (0.0009) [2023-12-26 16:03:33,987][105692] Updated weights for policy 0, policy_version 95341 (0.0009) [2023-12-26 16:03:34,437][105620] Updated weights for policy 1, policy_version 95666 (0.0006) [2023-12-26 16:03:34,493][105620] Updated weights for policy 1, policy_version 95676 (0.0009) [2023-12-26 16:03:34,558][105620] Updated weights for policy 1, policy_version 95686 (0.0009) [2023-12-26 16:03:34,620][105620] Updated weights for policy 1, policy_version 95696 (0.0009) [2023-12-26 16:03:34,844][105692] Updated weights for policy 0, policy_version 95351 (0.0009) [2023-12-26 16:03:34,900][105692] Updated weights for policy 0, policy_version 95361 (0.0008) [2023-12-26 16:03:34,954][105692] Updated weights for policy 0, policy_version 95371 (0.0009) [2023-12-26 16:03:35,328][105620] Updated weights for policy 1, policy_version 95706 (0.0009) [2023-12-26 16:03:35,378][105620] Updated weights for policy 1, policy_version 95716 (0.0008) [2023-12-26 16:03:35,435][105620] Updated weights for policy 1, policy_version 95726 (0.0009) [2023-12-26 16:03:35,692][105692] Updated weights for policy 0, policy_version 95381 (0.0009) [2023-12-26 16:03:35,749][105692] Updated weights for policy 0, policy_version 95391 (0.0009) [2023-12-26 16:03:35,800][105692] Updated weights for policy 0, policy_version 95401 (0.0009) [2023-12-26 16:03:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 48939008. Throughput: 0: 9774.0, 1: 9931.8. Samples: 48929348. Policy #0 lag: (min: 26.0, avg: 39.4, max: 40.0) [2023-12-26 16:03:36,063][104569] Avg episode reward: [(0, '9088.094'), (1, '9186.067')] [2023-12-26 16:03:36,186][105620] Updated weights for policy 1, policy_version 95736 (0.0009) [2023-12-26 16:03:36,248][105620] Updated weights for policy 1, policy_version 95746 (0.0008) [2023-12-26 16:03:36,306][105620] Updated weights for policy 1, policy_version 95756 (0.0009) [2023-12-26 16:03:36,585][105692] Updated weights for policy 0, policy_version 95411 (0.0009) [2023-12-26 16:03:36,645][105692] Updated weights for policy 0, policy_version 95421 (0.0009) [2023-12-26 16:03:36,697][105692] Updated weights for policy 0, policy_version 95431 (0.0009) [2023-12-26 16:03:37,027][105620] Updated weights for policy 1, policy_version 95766 (0.0008) [2023-12-26 16:03:37,086][105620] Updated weights for policy 1, policy_version 95776 (0.0009) [2023-12-26 16:03:37,137][105620] Updated weights for policy 1, policy_version 95786 (0.0009) [2023-12-26 16:03:37,468][105692] Updated weights for policy 0, policy_version 95441 (0.0009) [2023-12-26 16:03:37,530][105692] Updated weights for policy 0, policy_version 95451 (0.0008) [2023-12-26 16:03:37,589][105692] Updated weights for policy 0, policy_version 95461 (0.0008) [2023-12-26 16:03:37,644][105692] Updated weights for policy 0, policy_version 95471 (0.0009) [2023-12-26 16:03:37,895][105620] Updated weights for policy 1, policy_version 95796 (0.0010) [2023-12-26 16:03:37,959][105620] Updated weights for policy 1, policy_version 95806 (0.0009) [2023-12-26 16:03:38,021][105620] Updated weights for policy 1, policy_version 95816 (0.0009) [2023-12-26 16:03:38,393][105692] Updated weights for policy 0, policy_version 95481 (0.0009) [2023-12-26 16:03:38,452][105692] Updated weights for policy 0, policy_version 95491 (0.0009) [2023-12-26 16:03:38,510][105692] Updated weights for policy 0, policy_version 95501 (0.0009) [2023-12-26 16:03:38,784][105620] Updated weights for policy 1, policy_version 95826 (0.0009) [2023-12-26 16:03:38,832][105620] Updated weights for policy 1, policy_version 95836 (0.0008) [2023-12-26 16:03:38,889][105620] Updated weights for policy 1, policy_version 95846 (0.0007) [2023-12-26 16:03:38,954][105620] Updated weights for policy 1, policy_version 95856 (0.0009) [2023-12-26 16:03:39,268][105692] Updated weights for policy 0, policy_version 95511 (0.0007) [2023-12-26 16:03:39,323][105692] Updated weights for policy 0, policy_version 95521 (0.0009) [2023-12-26 16:03:39,400][105692] Updated weights for policy 0, policy_version 95531 (0.0009) [2023-12-26 16:03:39,600][105620] Updated weights for policy 1, policy_version 95866 (0.0009) [2023-12-26 16:03:39,662][105620] Updated weights for policy 1, policy_version 95876 (0.0009) [2023-12-26 16:03:39,725][105620] Updated weights for policy 1, policy_version 95886 (0.0009) [2023-12-26 16:03:40,202][105692] Updated weights for policy 0, policy_version 95541 (0.0009) [2023-12-26 16:03:40,260][105692] Updated weights for policy 0, policy_version 95551 (0.0009) [2023-12-26 16:03:40,321][105692] Updated weights for policy 0, policy_version 95561 (0.0008) [2023-12-26 16:03:40,450][105620] Updated weights for policy 1, policy_version 95896 (0.0006) [2023-12-26 16:03:40,524][105620] Updated weights for policy 1, policy_version 95906 (0.0006) [2023-12-26 16:03:40,590][105620] Updated weights for policy 1, policy_version 95916 (0.0006) [2023-12-26 16:03:41,022][105692] Updated weights for policy 0, policy_version 95571 (0.0008) [2023-12-26 16:03:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 49029120. Throughput: 0: 9666.9, 1: 9918.9. Samples: 49041716. Policy #0 lag: (min: 26.0, avg: 39.4, max: 40.0) [2023-12-26 16:03:41,063][104569] Avg episode reward: [(0, '9267.048'), (1, '9353.126')] [2023-12-26 16:03:41,085][105692] Updated weights for policy 0, policy_version 95581 (0.0008) [2023-12-26 16:03:41,150][105692] Updated weights for policy 0, policy_version 95591 (0.0007) [2023-12-26 16:03:41,252][105620] Updated weights for policy 1, policy_version 95926 (0.0006) [2023-12-26 16:03:41,323][105620] Updated weights for policy 1, policy_version 95936 (0.0006) [2023-12-26 16:03:41,395][105620] Updated weights for policy 1, policy_version 95946 (0.0007) [2023-12-26 16:03:41,928][105692] Updated weights for policy 0, policy_version 95601 (0.0006) [2023-12-26 16:03:41,995][105692] Updated weights for policy 0, policy_version 95611 (0.0009) [2023-12-26 16:03:42,004][105620] Updated weights for policy 1, policy_version 95956 (0.0007) [2023-12-26 16:03:42,052][105692] Updated weights for policy 0, policy_version 95621 (0.0006) [2023-12-26 16:03:42,058][105620] Updated weights for policy 1, policy_version 95966 (0.0009) [2023-12-26 16:03:42,112][105692] Updated weights for policy 0, policy_version 95631 (0.0007) [2023-12-26 16:03:42,127][105620] Updated weights for policy 1, policy_version 95976 (0.0006) [2023-12-26 16:03:42,742][105620] Updated weights for policy 1, policy_version 95986 (0.0005) [2023-12-26 16:03:42,793][105620] Updated weights for policy 1, policy_version 95996 (0.0006) [2023-12-26 16:03:42,854][105620] Updated weights for policy 1, policy_version 96006 (0.0009) [2023-12-26 16:03:42,914][105620] Updated weights for policy 1, policy_version 96016 (0.0008) [2023-12-26 16:03:42,948][105692] Updated weights for policy 0, policy_version 95641 (0.0008) [2023-12-26 16:03:42,997][105692] Updated weights for policy 0, policy_version 95651 (0.0009) [2023-12-26 16:03:43,055][105692] Updated weights for policy 0, policy_version 95661 (0.0009) [2023-12-26 16:03:43,574][105620] Updated weights for policy 1, policy_version 96026 (0.0005) [2023-12-26 16:03:43,623][105620] Updated weights for policy 1, policy_version 96036 (0.0005) [2023-12-26 16:03:43,679][105620] Updated weights for policy 1, policy_version 96046 (0.0005) [2023-12-26 16:03:43,768][105692] Updated weights for policy 0, policy_version 95671 (0.0007) [2023-12-26 16:03:43,829][105692] Updated weights for policy 0, policy_version 95681 (0.0005) [2023-12-26 16:03:43,877][105692] Updated weights for policy 0, policy_version 95691 (0.0005) [2023-12-26 16:03:44,291][105620] Updated weights for policy 1, policy_version 96056 (0.0008) [2023-12-26 16:03:44,356][105620] Updated weights for policy 1, policy_version 96066 (0.0009) [2023-12-26 16:03:44,423][105620] Updated weights for policy 1, policy_version 96076 (0.0005) [2023-12-26 16:03:44,480][105692] Updated weights for policy 0, policy_version 95701 (0.0007) [2023-12-26 16:03:44,532][105692] Updated weights for policy 0, policy_version 95711 (0.0010) [2023-12-26 16:03:44,587][105692] Updated weights for policy 0, policy_version 95721 (0.0008) [2023-12-26 16:03:45,169][105620] Updated weights for policy 1, policy_version 96086 (0.0008) [2023-12-26 16:03:45,228][105620] Updated weights for policy 1, policy_version 96096 (0.0010) [2023-12-26 16:03:45,245][105692] Updated weights for policy 0, policy_version 95731 (0.0006) [2023-12-26 16:03:45,279][105620] Updated weights for policy 1, policy_version 96106 (0.0008) [2023-12-26 16:03:45,310][105692] Updated weights for policy 0, policy_version 95741 (0.0007) [2023-12-26 16:03:45,372][105692] Updated weights for policy 0, policy_version 95751 (0.0008) [2023-12-26 16:03:46,045][105620] Updated weights for policy 1, policy_version 96116 (0.0008) [2023-12-26 16:03:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 49127424. Throughput: 0: 9604.4, 1: 9960.5. Samples: 49101160. Policy #0 lag: (min: 26.0, avg: 39.4, max: 40.0) [2023-12-26 16:03:46,062][104569] Avg episode reward: [(0, '9359.331'), (1, '9352.262')] [2023-12-26 16:03:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000095760_24518656.pth... [2023-12-26 16:03:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000094640_24231936.pth [2023-12-26 16:03:46,105][105620] Updated weights for policy 1, policy_version 96126 (0.0010) [2023-12-26 16:03:46,118][105692] Updated weights for policy 0, policy_version 95761 (0.0008) [2023-12-26 16:03:46,157][105620] Updated weights for policy 1, policy_version 96136 (0.0010) [2023-12-26 16:03:46,179][105692] Updated weights for policy 0, policy_version 95771 (0.0007) [2023-12-26 16:03:46,199][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000096144_24616960.pth... [2023-12-26 16:03:46,202][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000094960_24313856.pth [2023-12-26 16:03:46,235][105692] Updated weights for policy 0, policy_version 95781 (0.0008) [2023-12-26 16:03:46,295][105692] Updated weights for policy 0, policy_version 95791 (0.0010) [2023-12-26 16:03:46,864][105620] Updated weights for policy 1, policy_version 96146 (0.0011) [2023-12-26 16:03:46,928][105620] Updated weights for policy 1, policy_version 96156 (0.0011) [2023-12-26 16:03:46,985][105620] Updated weights for policy 1, policy_version 96166 (0.0011) [2023-12-26 16:03:46,995][105692] Updated weights for policy 0, policy_version 95801 (0.0006) [2023-12-26 16:03:47,037][105620] Updated weights for policy 1, policy_version 96176 (0.0010) [2023-12-26 16:03:47,051][105692] Updated weights for policy 0, policy_version 95811 (0.0006) [2023-12-26 16:03:47,103][105692] Updated weights for policy 0, policy_version 95821 (0.0008) [2023-12-26 16:03:47,779][105620] Updated weights for policy 1, policy_version 96186 (0.0011) [2023-12-26 16:03:47,820][105692] Updated weights for policy 0, policy_version 95831 (0.0006) [2023-12-26 16:03:47,838][105620] Updated weights for policy 1, policy_version 96196 (0.0011) [2023-12-26 16:03:47,868][105692] Updated weights for policy 0, policy_version 95841 (0.0005) [2023-12-26 16:03:47,893][105620] Updated weights for policy 1, policy_version 96206 (0.0010) [2023-12-26 16:03:47,924][105692] Updated weights for policy 0, policy_version 95851 (0.0006) [2023-12-26 16:03:48,551][105620] Updated weights for policy 1, policy_version 96216 (0.0006) [2023-12-26 16:03:48,607][105692] Updated weights for policy 0, policy_version 95861 (0.0008) [2023-12-26 16:03:48,609][105620] Updated weights for policy 1, policy_version 96226 (0.0005) [2023-12-26 16:03:48,666][105692] Updated weights for policy 0, policy_version 95871 (0.0009) [2023-12-26 16:03:48,666][105620] Updated weights for policy 1, policy_version 96236 (0.0006) [2023-12-26 16:03:48,726][105692] Updated weights for policy 0, policy_version 95881 (0.0008) [2023-12-26 16:03:49,257][105620] Updated weights for policy 1, policy_version 96246 (0.0009) [2023-12-26 16:03:49,312][105620] Updated weights for policy 1, policy_version 96256 (0.0006) [2023-12-26 16:03:49,372][105620] Updated weights for policy 1, policy_version 96266 (0.0008) [2023-12-26 16:03:49,521][105692] Updated weights for policy 0, policy_version 95891 (0.0007) [2023-12-26 16:03:49,579][105692] Updated weights for policy 0, policy_version 95901 (0.0008) [2023-12-26 16:03:49,636][105692] Updated weights for policy 0, policy_version 95911 (0.0008) [2023-12-26 16:03:50,066][105620] Updated weights for policy 1, policy_version 96276 (0.0007) [2023-12-26 16:03:50,125][105620] Updated weights for policy 1, policy_version 96286 (0.0005) [2023-12-26 16:03:50,187][105620] Updated weights for policy 1, policy_version 96296 (0.0005) [2023-12-26 16:03:50,368][105692] Updated weights for policy 0, policy_version 95921 (0.0009) [2023-12-26 16:03:50,434][105692] Updated weights for policy 0, policy_version 95931 (0.0010) [2023-12-26 16:03:50,493][105692] Updated weights for policy 0, policy_version 95941 (0.0010) [2023-12-26 16:03:50,546][105692] Updated weights for policy 0, policy_version 95952 (0.0009) [2023-12-26 16:03:50,774][105620] Updated weights for policy 1, policy_version 96306 (0.0006) [2023-12-26 16:03:50,834][105620] Updated weights for policy 1, policy_version 96316 (0.0006) [2023-12-26 16:03:50,904][105620] Updated weights for policy 1, policy_version 96326 (0.0005) [2023-12-26 16:03:50,977][105620] Updated weights for policy 1, policy_version 96336 (0.0005) [2023-12-26 16:03:51,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 49233920. Throughput: 0: 9683.6, 1: 9839.3. Samples: 49220028. Policy #0 lag: (min: 26.0, avg: 39.4, max: 40.0) [2023-12-26 16:03:51,062][104569] Avg episode reward: [(0, '9087.600'), (1, '8766.492')] [2023-12-26 16:03:51,301][105692] Updated weights for policy 0, policy_version 95962 (0.0010) [2023-12-26 16:03:51,370][105692] Updated weights for policy 0, policy_version 95972 (0.0009) [2023-12-26 16:03:51,438][105692] Updated weights for policy 0, policy_version 95982 (0.0009) [2023-12-26 16:03:51,561][105620] Updated weights for policy 1, policy_version 96346 (0.0009) [2023-12-26 16:03:51,624][105620] Updated weights for policy 1, policy_version 96356 (0.0008) [2023-12-26 16:03:51,688][105620] Updated weights for policy 1, policy_version 96366 (0.0006) [2023-12-26 16:03:52,088][105692] Updated weights for policy 0, policy_version 95992 (0.0008) [2023-12-26 16:03:52,143][105692] Updated weights for policy 0, policy_version 96002 (0.0009) [2023-12-26 16:03:52,204][105692] Updated weights for policy 0, policy_version 96012 (0.0008) [2023-12-26 16:03:52,502][105620] Updated weights for policy 1, policy_version 96376 (0.0009) [2023-12-26 16:03:52,555][105620] Updated weights for policy 1, policy_version 96386 (0.0011) [2023-12-26 16:03:52,608][105620] Updated weights for policy 1, policy_version 96396 (0.0010) [2023-12-26 16:03:52,934][105692] Updated weights for policy 0, policy_version 96022 (0.0009) [2023-12-26 16:03:52,986][105692] Updated weights for policy 0, policy_version 96032 (0.0008) [2023-12-26 16:03:53,045][105692] Updated weights for policy 0, policy_version 96042 (0.0008) [2023-12-26 16:03:53,379][105620] Updated weights for policy 1, policy_version 96406 (0.0009) [2023-12-26 16:03:53,436][105620] Updated weights for policy 1, policy_version 96416 (0.0009) [2023-12-26 16:03:53,486][105620] Updated weights for policy 1, policy_version 96426 (0.0009) [2023-12-26 16:03:53,803][105692] Updated weights for policy 0, policy_version 96052 (0.0009) [2023-12-26 16:03:53,857][105692] Updated weights for policy 0, policy_version 96062 (0.0010) [2023-12-26 16:03:53,908][105692] Updated weights for policy 0, policy_version 96072 (0.0010) [2023-12-26 16:03:54,227][105620] Updated weights for policy 1, policy_version 96436 (0.0008) [2023-12-26 16:03:54,290][105620] Updated weights for policy 1, policy_version 96446 (0.0011) [2023-12-26 16:03:54,348][105620] Updated weights for policy 1, policy_version 96456 (0.0010) [2023-12-26 16:03:54,494][105692] Updated weights for policy 0, policy_version 96082 (0.0007) [2023-12-26 16:03:54,558][105692] Updated weights for policy 0, policy_version 96092 (0.0010) [2023-12-26 16:03:54,611][105692] Updated weights for policy 0, policy_version 96102 (0.0009) [2023-12-26 16:03:54,934][105620] Updated weights for policy 1, policy_version 96466 (0.0010) [2023-12-26 16:03:54,990][105620] Updated weights for policy 1, policy_version 96476 (0.0008) [2023-12-26 16:03:55,037][105620] Updated weights for policy 1, policy_version 96486 (0.0008) [2023-12-26 16:03:55,095][105620] Updated weights for policy 1, policy_version 96496 (0.0009) [2023-12-26 16:03:55,453][105692] Updated weights for policy 0, policy_version 96113 (0.0010) [2023-12-26 16:03:55,507][105692] Updated weights for policy 0, policy_version 96124 (0.0010) [2023-12-26 16:03:55,562][105692] Updated weights for policy 0, policy_version 96135 (0.0010) [2023-12-26 16:03:55,674][105620] Updated weights for policy 1, policy_version 96506 (0.0005) [2023-12-26 16:03:55,730][105620] Updated weights for policy 1, policy_version 96516 (0.0005) [2023-12-26 16:03:55,783][105620] Updated weights for policy 1, policy_version 96526 (0.0005) [2023-12-26 16:03:56,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 49332224. Throughput: 0: 9570.5, 1: 9970.5. Samples: 49339448. Policy #0 lag: (min: 26.0, avg: 39.4, max: 40.0) [2023-12-26 16:03:56,063][104569] Avg episode reward: [(0, '8816.644'), (1, '8765.853')] [2023-12-26 16:03:56,351][105620] Updated weights for policy 1, policy_version 96536 (0.0007) [2023-12-26 16:03:56,401][105620] Updated weights for policy 1, policy_version 96546 (0.0005) [2023-12-26 16:03:56,412][105692] Updated weights for policy 0, policy_version 96147 (0.0009) [2023-12-26 16:03:56,457][105620] Updated weights for policy 1, policy_version 96556 (0.0005) [2023-12-26 16:03:56,468][105692] Updated weights for policy 0, policy_version 96157 (0.0006) [2023-12-26 16:03:56,534][105692] Updated weights for policy 0, policy_version 96167 (0.0005) [2023-12-26 16:03:56,998][105620] Updated weights for policy 1, policy_version 96566 (0.0008) [2023-12-26 16:03:57,049][105620] Updated weights for policy 1, policy_version 96576 (0.0007) [2023-12-26 16:03:57,059][105692] Updated weights for policy 0, policy_version 96177 (0.0005) [2023-12-26 16:03:57,108][105620] Updated weights for policy 1, policy_version 96586 (0.0005) [2023-12-26 16:03:57,118][105692] Updated weights for policy 0, policy_version 96187 (0.0005) [2023-12-26 16:03:57,182][105692] Updated weights for policy 0, policy_version 96197 (0.0008) [2023-12-26 16:03:57,235][105692] Updated weights for policy 0, policy_version 96207 (0.0009) [2023-12-26 16:03:57,678][105620] Updated weights for policy 1, policy_version 96596 (0.0006) [2023-12-26 16:03:57,734][105620] Updated weights for policy 1, policy_version 96606 (0.0006) [2023-12-26 16:03:57,782][105620] Updated weights for policy 1, policy_version 96616 (0.0005) [2023-12-26 16:03:57,897][105692] Updated weights for policy 0, policy_version 96217 (0.0010) [2023-12-26 16:03:57,965][105692] Updated weights for policy 0, policy_version 96227 (0.0010) [2023-12-26 16:03:58,025][105692] Updated weights for policy 0, policy_version 96237 (0.0007) [2023-12-26 16:03:58,414][105620] Updated weights for policy 1, policy_version 96626 (0.0006) [2023-12-26 16:03:58,480][105620] Updated weights for policy 1, policy_version 96636 (0.0009) [2023-12-26 16:03:58,540][105620] Updated weights for policy 1, policy_version 96646 (0.0009) [2023-12-26 16:03:58,603][105620] Updated weights for policy 1, policy_version 96656 (0.0008) [2023-12-26 16:03:58,770][105692] Updated weights for policy 0, policy_version 96247 (0.0006) [2023-12-26 16:03:58,836][105692] Updated weights for policy 0, policy_version 96257 (0.0007) [2023-12-26 16:03:58,900][105692] Updated weights for policy 0, policy_version 96267 (0.0007) [2023-12-26 16:03:59,381][105620] Updated weights for policy 1, policy_version 96666 (0.0008) [2023-12-26 16:03:59,438][105620] Updated weights for policy 1, policy_version 96676 (0.0010) [2023-12-26 16:03:59,490][105620] Updated weights for policy 1, policy_version 96686 (0.0009) [2023-12-26 16:03:59,532][105692] Updated weights for policy 0, policy_version 96277 (0.0008) [2023-12-26 16:03:59,597][105692] Updated weights for policy 0, policy_version 96287 (0.0009) [2023-12-26 16:03:59,667][105692] Updated weights for policy 0, policy_version 96297 (0.0008) [2023-12-26 16:04:00,323][105620] Updated weights for policy 1, policy_version 96696 (0.0009) [2023-12-26 16:04:00,373][105620] Updated weights for policy 1, policy_version 96706 (0.0006) [2023-12-26 16:04:00,375][105692] Updated weights for policy 0, policy_version 96307 (0.0008) [2023-12-26 16:04:00,429][105692] Updated weights for policy 0, policy_version 96317 (0.0008) [2023-12-26 16:04:00,435][105620] Updated weights for policy 1, policy_version 96716 (0.0006) [2023-12-26 16:04:00,478][105692] Updated weights for policy 0, policy_version 96327 (0.0008) [2023-12-26 16:04:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 49430528. Throughput: 0: 9585.6, 1: 10072.1. Samples: 49403152. Policy #0 lag: (min: 26.0, avg: 39.4, max: 40.0) [2023-12-26 16:04:01,062][104569] Avg episode reward: [(0, '8819.081'), (1, '9262.912')] [2023-12-26 16:04:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000096336_24666112.pth... [2023-12-26 16:04:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000095216_24379392.pth [2023-12-26 16:04:01,081][105620] Updated weights for policy 1, policy_version 96726 (0.0006) [2023-12-26 16:04:01,156][105620] Updated weights for policy 1, policy_version 96736 (0.0011) [2023-12-26 16:04:01,195][105692] Updated weights for policy 0, policy_version 96338 (0.0009) [2023-12-26 16:04:01,223][105620] Updated weights for policy 1, policy_version 96746 (0.0008) [2023-12-26 16:04:01,261][105692] Updated weights for policy 0, policy_version 96348 (0.0006) [2023-12-26 16:04:01,263][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000096752_24772608.pth... [2023-12-26 16:04:01,266][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000095536_24461312.pth [2023-12-26 16:04:01,319][105692] Updated weights for policy 0, policy_version 96358 (0.0007) [2023-12-26 16:04:01,381][105692] Updated weights for policy 0, policy_version 96368 (0.0008) [2023-12-26 16:04:01,945][105620] Updated weights for policy 1, policy_version 96756 (0.0008) [2023-12-26 16:04:01,991][105692] Updated weights for policy 0, policy_version 96378 (0.0009) [2023-12-26 16:04:02,003][105620] Updated weights for policy 1, policy_version 96766 (0.0007) [2023-12-26 16:04:02,046][105692] Updated weights for policy 0, policy_version 96388 (0.0009) [2023-12-26 16:04:02,058][105620] Updated weights for policy 1, policy_version 96776 (0.0008) [2023-12-26 16:04:02,092][105692] Updated weights for policy 0, policy_version 96398 (0.0007) [2023-12-26 16:04:02,737][105620] Updated weights for policy 1, policy_version 96786 (0.0009) [2023-12-26 16:04:02,794][105620] Updated weights for policy 1, policy_version 96796 (0.0009) [2023-12-26 16:04:02,855][105620] Updated weights for policy 1, policy_version 96808 (0.0010) [2023-12-26 16:04:02,866][105692] Updated weights for policy 0, policy_version 96408 (0.0006) [2023-12-26 16:04:02,924][105692] Updated weights for policy 0, policy_version 96418 (0.0009) [2023-12-26 16:04:02,977][105692] Updated weights for policy 0, policy_version 96428 (0.0010) [2023-12-26 16:04:03,491][105620] Updated weights for policy 1, policy_version 96818 (0.0005) [2023-12-26 16:04:03,544][105620] Updated weights for policy 1, policy_version 96828 (0.0008) [2023-12-26 16:04:03,560][105692] Updated weights for policy 0, policy_version 96438 (0.0007) [2023-12-26 16:04:03,605][105620] Updated weights for policy 1, policy_version 96838 (0.0010) [2023-12-26 16:04:03,612][105692] Updated weights for policy 0, policy_version 96448 (0.0006) [2023-12-26 16:04:03,652][105620] Updated weights for policy 1, policy_version 96848 (0.0007) [2023-12-26 16:04:03,672][105692] Updated weights for policy 0, policy_version 96458 (0.0009) [2023-12-26 16:04:04,319][105620] Updated weights for policy 1, policy_version 96858 (0.0011) [2023-12-26 16:04:04,374][105620] Updated weights for policy 1, policy_version 96868 (0.0010) [2023-12-26 16:04:04,437][105620] Updated weights for policy 1, policy_version 96878 (0.0010) [2023-12-26 16:04:04,472][105692] Updated weights for policy 0, policy_version 96468 (0.0008) [2023-12-26 16:04:04,532][105692] Updated weights for policy 0, policy_version 96478 (0.0008) [2023-12-26 16:04:04,581][105692] Updated weights for policy 0, policy_version 96488 (0.0008) [2023-12-26 16:04:05,187][105620] Updated weights for policy 1, policy_version 96888 (0.0010) [2023-12-26 16:04:05,251][105620] Updated weights for policy 1, policy_version 96898 (0.0010) [2023-12-26 16:04:05,305][105620] Updated weights for policy 1, policy_version 96908 (0.0010) [2023-12-26 16:04:05,338][105692] Updated weights for policy 0, policy_version 96498 (0.0008) [2023-12-26 16:04:05,397][105692] Updated weights for policy 0, policy_version 96508 (0.0006) [2023-12-26 16:04:05,452][105692] Updated weights for policy 0, policy_version 96518 (0.0008) [2023-12-26 16:04:05,515][105692] Updated weights for policy 0, policy_version 96528 (0.0008) [2023-12-26 16:04:05,976][105620] Updated weights for policy 1, policy_version 96918 (0.0007) [2023-12-26 16:04:06,029][105620] Updated weights for policy 1, policy_version 96928 (0.0005) [2023-12-26 16:04:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 49528832. Throughput: 0: 9597.7, 1: 10075.7. Samples: 49522220. Policy #0 lag: (min: 26.0, avg: 39.4, max: 40.0) [2023-12-26 16:04:06,063][104569] Avg episode reward: [(0, '8813.490'), (1, '9091.871')] [2023-12-26 16:04:06,082][105620] Updated weights for policy 1, policy_version 96938 (0.0005) [2023-12-26 16:04:06,313][105692] Updated weights for policy 0, policy_version 96538 (0.0008) [2023-12-26 16:04:06,377][105692] Updated weights for policy 0, policy_version 96548 (0.0008) [2023-12-26 16:04:06,437][105692] Updated weights for policy 0, policy_version 96558 (0.0009) [2023-12-26 16:04:06,751][105620] Updated weights for policy 1, policy_version 96948 (0.0009) [2023-12-26 16:04:06,795][105620] Updated weights for policy 1, policy_version 96958 (0.0010) [2023-12-26 16:04:06,847][105620] Updated weights for policy 1, policy_version 96968 (0.0010) [2023-12-26 16:04:07,112][105692] Updated weights for policy 0, policy_version 96568 (0.0009) [2023-12-26 16:04:07,172][105692] Updated weights for policy 0, policy_version 96578 (0.0008) [2023-12-26 16:04:07,225][105692] Updated weights for policy 0, policy_version 96588 (0.0008) [2023-12-26 16:04:07,515][105620] Updated weights for policy 1, policy_version 96978 (0.0009) [2023-12-26 16:04:07,565][105620] Updated weights for policy 1, policy_version 96988 (0.0005) [2023-12-26 16:04:07,611][105620] Updated weights for policy 1, policy_version 96998 (0.0005) [2023-12-26 16:04:07,657][105620] Updated weights for policy 1, policy_version 97008 (0.0005) [2023-12-26 16:04:07,979][105692] Updated weights for policy 0, policy_version 96598 (0.0008) [2023-12-26 16:04:08,034][105692] Updated weights for policy 0, policy_version 96608 (0.0008) [2023-12-26 16:04:08,086][105692] Updated weights for policy 0, policy_version 96618 (0.0008) [2023-12-26 16:04:08,261][105620] Updated weights for policy 1, policy_version 97018 (0.0011) [2023-12-26 16:04:08,322][105620] Updated weights for policy 1, policy_version 97028 (0.0010) [2023-12-26 16:04:08,384][105620] Updated weights for policy 1, policy_version 97038 (0.0010) [2023-12-26 16:04:08,856][105692] Updated weights for policy 0, policy_version 96628 (0.0008) [2023-12-26 16:04:08,908][105692] Updated weights for policy 0, policy_version 96638 (0.0008) [2023-12-26 16:04:08,960][105692] Updated weights for policy 0, policy_version 96648 (0.0008) [2023-12-26 16:04:09,148][105620] Updated weights for policy 1, policy_version 97048 (0.0010) [2023-12-26 16:04:09,197][105620] Updated weights for policy 1, policy_version 97058 (0.0010) [2023-12-26 16:04:09,261][105620] Updated weights for policy 1, policy_version 97068 (0.0008) [2023-12-26 16:04:09,733][105692] Updated weights for policy 0, policy_version 96658 (0.0008) [2023-12-26 16:04:09,800][105692] Updated weights for policy 0, policy_version 96668 (0.0006) [2023-12-26 16:04:09,871][105692] Updated weights for policy 0, policy_version 96678 (0.0008) [2023-12-26 16:04:09,935][105692] Updated weights for policy 0, policy_version 96688 (0.0007) [2023-12-26 16:04:10,077][105620] Updated weights for policy 1, policy_version 97078 (0.0008) [2023-12-26 16:04:10,126][105620] Updated weights for policy 1, policy_version 97088 (0.0008) [2023-12-26 16:04:10,177][105620] Updated weights for policy 1, policy_version 97098 (0.0008) [2023-12-26 16:04:10,565][105692] Updated weights for policy 0, policy_version 96698 (0.0006) [2023-12-26 16:04:10,623][105692] Updated weights for policy 0, policy_version 96708 (0.0007) [2023-12-26 16:04:10,679][105692] Updated weights for policy 0, policy_version 96718 (0.0005) [2023-12-26 16:04:10,938][105620] Updated weights for policy 1, policy_version 97108 (0.0008) [2023-12-26 16:04:10,996][105620] Updated weights for policy 1, policy_version 97118 (0.0008) [2023-12-26 16:04:11,058][105620] Updated weights for policy 1, policy_version 97128 (0.0008) [2023-12-26 16:04:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 49627136. Throughput: 0: 9548.5, 1: 10113.0. Samples: 49639472. Policy #0 lag: (min: 26.0, avg: 39.4, max: 40.0) [2023-12-26 16:04:11,062][104569] Avg episode reward: [(0, '8642.711'), (1, '8915.446')] [2023-12-26 16:04:11,394][105692] Updated weights for policy 0, policy_version 96728 (0.0009) [2023-12-26 16:04:11,454][105692] Updated weights for policy 0, policy_version 96738 (0.0009) [2023-12-26 16:04:11,510][105692] Updated weights for policy 0, policy_version 96748 (0.0010) [2023-12-26 16:04:11,859][105620] Updated weights for policy 1, policy_version 97138 (0.0009) [2023-12-26 16:04:11,918][105620] Updated weights for policy 1, policy_version 97148 (0.0011) [2023-12-26 16:04:11,978][105620] Updated weights for policy 1, policy_version 97158 (0.0010) [2023-12-26 16:04:12,046][105620] Updated weights for policy 1, policy_version 97168 (0.0005) [2023-12-26 16:04:12,262][105692] Updated weights for policy 0, policy_version 96758 (0.0007) [2023-12-26 16:04:12,322][105692] Updated weights for policy 0, policy_version 96768 (0.0008) [2023-12-26 16:04:12,384][105692] Updated weights for policy 0, policy_version 96778 (0.0008) [2023-12-26 16:04:12,690][105620] Updated weights for policy 1, policy_version 97178 (0.0010) [2023-12-26 16:04:12,756][105620] Updated weights for policy 1, policy_version 97188 (0.0010) [2023-12-26 16:04:12,816][105620] Updated weights for policy 1, policy_version 97198 (0.0007) [2023-12-26 16:04:12,964][105692] Updated weights for policy 0, policy_version 96788 (0.0006) [2023-12-26 16:04:13,030][105692] Updated weights for policy 0, policy_version 96798 (0.0006) [2023-12-26 16:04:13,095][105692] Updated weights for policy 0, policy_version 96808 (0.0007) [2023-12-26 16:04:13,453][105620] Updated weights for policy 1, policy_version 97208 (0.0009) [2023-12-26 16:04:13,501][105620] Updated weights for policy 1, policy_version 97218 (0.0010) [2023-12-26 16:04:13,548][105620] Updated weights for policy 1, policy_version 97228 (0.0007) [2023-12-26 16:04:13,850][105692] Updated weights for policy 0, policy_version 96818 (0.0008) [2023-12-26 16:04:13,904][105692] Updated weights for policy 0, policy_version 96828 (0.0010) [2023-12-26 16:04:13,958][105692] Updated weights for policy 0, policy_version 96838 (0.0010) [2023-12-26 16:04:14,021][105692] Updated weights for policy 0, policy_version 96848 (0.0010) [2023-12-26 16:04:14,095][105620] Updated weights for policy 1, policy_version 97238 (0.0005) [2023-12-26 16:04:14,156][105620] Updated weights for policy 1, policy_version 97248 (0.0006) [2023-12-26 16:04:14,224][105620] Updated weights for policy 1, policy_version 97258 (0.0005) [2023-12-26 16:04:14,787][105692] Updated weights for policy 0, policy_version 96858 (0.0008) [2023-12-26 16:04:14,840][105692] Updated weights for policy 0, policy_version 96868 (0.0009) [2023-12-26 16:04:14,846][105620] Updated weights for policy 1, policy_version 97268 (0.0007) [2023-12-26 16:04:14,899][105620] Updated weights for policy 1, policy_version 97278 (0.0010) [2023-12-26 16:04:14,901][105692] Updated weights for policy 0, policy_version 96878 (0.0006) [2023-12-26 16:04:14,954][105620] Updated weights for policy 1, policy_version 97288 (0.0010) [2023-12-26 16:04:15,638][105692] Updated weights for policy 0, policy_version 96888 (0.0007) [2023-12-26 16:04:15,697][105692] Updated weights for policy 0, policy_version 96898 (0.0008) [2023-12-26 16:04:15,719][105620] Updated weights for policy 1, policy_version 97298 (0.0010) [2023-12-26 16:04:15,757][105692] Updated weights for policy 0, policy_version 96908 (0.0007) [2023-12-26 16:04:15,781][105620] Updated weights for policy 1, policy_version 97308 (0.0010) [2023-12-26 16:04:15,845][105620] Updated weights for policy 1, policy_version 97318 (0.0010) [2023-12-26 16:04:15,903][105620] Updated weights for policy 1, policy_version 97328 (0.0010) [2023-12-26 16:04:16,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 49733632. Throughput: 0: 9569.2, 1: 10145.7. Samples: 49700480. Policy #0 lag: (min: 26.0, avg: 39.4, max: 40.0) [2023-12-26 16:04:16,063][104569] Avg episode reward: [(0, '7937.893'), (1, '9091.125')] [2023-12-26 16:04:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000096912_24813568.pth... [2023-12-26 16:04:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000097328_24920064.pth... [2023-12-26 16:04:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000095760_24518656.pth [2023-12-26 16:04:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000096144_24616960.pth [2023-12-26 16:04:16,524][105692] Updated weights for policy 0, policy_version 96918 (0.0007) [2023-12-26 16:04:16,586][105692] Updated weights for policy 0, policy_version 96928 (0.0008) [2023-12-26 16:04:16,629][105620] Updated weights for policy 1, policy_version 97338 (0.0010) [2023-12-26 16:04:16,647][105692] Updated weights for policy 0, policy_version 96938 (0.0006) [2023-12-26 16:04:16,683][105620] Updated weights for policy 1, policy_version 97348 (0.0010) [2023-12-26 16:04:16,741][105620] Updated weights for policy 1, policy_version 97358 (0.0010) [2023-12-26 16:04:17,401][105692] Updated weights for policy 0, policy_version 96948 (0.0007) [2023-12-26 16:04:17,445][105692] Updated weights for policy 0, policy_version 96958 (0.0007) [2023-12-26 16:04:17,498][105692] Updated weights for policy 0, policy_version 96968 (0.0008) [2023-12-26 16:04:17,505][105620] Updated weights for policy 1, policy_version 97368 (0.0010) [2023-12-26 16:04:17,556][105620] Updated weights for policy 1, policy_version 97378 (0.0010) [2023-12-26 16:04:17,607][105620] Updated weights for policy 1, policy_version 97388 (0.0010) [2023-12-26 16:04:18,175][105620] Updated weights for policy 1, policy_version 97398 (0.0007) [2023-12-26 16:04:18,228][105620] Updated weights for policy 1, policy_version 97408 (0.0005) [2023-12-26 16:04:18,280][105620] Updated weights for policy 1, policy_version 97418 (0.0007) [2023-12-26 16:04:18,366][105692] Updated weights for policy 0, policy_version 96978 (0.0007) [2023-12-26 16:04:18,422][105692] Updated weights for policy 0, policy_version 96988 (0.0008) [2023-12-26 16:04:18,474][105692] Updated weights for policy 0, policy_version 96998 (0.0007) [2023-12-26 16:04:18,519][105692] Updated weights for policy 0, policy_version 97008 (0.0008) [2023-12-26 16:04:18,988][105620] Updated weights for policy 1, policy_version 97428 (0.0010) [2023-12-26 16:04:19,040][105620] Updated weights for policy 1, policy_version 97438 (0.0010) [2023-12-26 16:04:19,095][105620] Updated weights for policy 1, policy_version 97448 (0.0010) [2023-12-26 16:04:19,305][105692] Updated weights for policy 0, policy_version 97018 (0.0008) [2023-12-26 16:04:19,373][105692] Updated weights for policy 0, policy_version 97028 (0.0009) [2023-12-26 16:04:19,429][105692] Updated weights for policy 0, policy_version 97038 (0.0008) [2023-12-26 16:04:19,873][105620] Updated weights for policy 1, policy_version 97458 (0.0010) [2023-12-26 16:04:19,941][105620] Updated weights for policy 1, policy_version 97468 (0.0011) [2023-12-26 16:04:19,999][105620] Updated weights for policy 1, policy_version 97478 (0.0011) [2023-12-26 16:04:20,064][105620] Updated weights for policy 1, policy_version 97488 (0.0011) [2023-12-26 16:04:20,195][105692] Updated weights for policy 0, policy_version 97048 (0.0008) [2023-12-26 16:04:20,252][105692] Updated weights for policy 0, policy_version 97058 (0.0008) [2023-12-26 16:04:20,317][105692] Updated weights for policy 0, policy_version 97068 (0.0008) [2023-12-26 16:04:20,736][105620] Updated weights for policy 1, policy_version 97498 (0.0011) [2023-12-26 16:04:20,799][105620] Updated weights for policy 1, policy_version 97508 (0.0009) [2023-12-26 16:04:20,853][105620] Updated weights for policy 1, policy_version 97518 (0.0008) [2023-12-26 16:04:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 49823744. Throughput: 0: 9554.5, 1: 10100.0. Samples: 49813792. Policy #0 lag: (min: 26.0, avg: 39.4, max: 40.0) [2023-12-26 16:04:21,063][104569] Avg episode reward: [(0, '2865.989'), (1, '8946.414')] [2023-12-26 16:04:21,168][105692] Updated weights for policy 0, policy_version 97078 (0.0010) [2023-12-26 16:04:21,236][105692] Updated weights for policy 0, policy_version 97088 (0.0011) [2023-12-26 16:04:21,300][105692] Updated weights for policy 0, policy_version 97098 (0.0009) [2023-12-26 16:04:21,521][105620] Updated weights for policy 1, policy_version 97528 (0.0007) [2023-12-26 16:04:21,591][105620] Updated weights for policy 1, policy_version 97538 (0.0006) [2023-12-26 16:04:21,665][105620] Updated weights for policy 1, policy_version 97548 (0.0009) [2023-12-26 16:04:22,065][105692] Updated weights for policy 0, policy_version 97108 (0.0011) [2023-12-26 16:04:22,125][105692] Updated weights for policy 0, policy_version 97118 (0.0011) [2023-12-26 16:04:22,185][105692] Updated weights for policy 0, policy_version 97128 (0.0010) [2023-12-26 16:04:22,366][105620] Updated weights for policy 1, policy_version 97558 (0.0009) [2023-12-26 16:04:22,425][105620] Updated weights for policy 1, policy_version 97568 (0.0008) [2023-12-26 16:04:22,488][105620] Updated weights for policy 1, policy_version 97578 (0.0010) [2023-12-26 16:04:22,908][105692] Updated weights for policy 0, policy_version 97138 (0.0011) [2023-12-26 16:04:22,963][105692] Updated weights for policy 0, policy_version 97148 (0.0010) [2023-12-26 16:04:23,014][105692] Updated weights for policy 0, policy_version 97158 (0.0009) [2023-12-26 16:04:23,080][105692] Updated weights for policy 0, policy_version 97168 (0.0006) [2023-12-26 16:04:23,224][105620] Updated weights for policy 1, policy_version 97588 (0.0008) [2023-12-26 16:04:23,301][105620] Updated weights for policy 1, policy_version 97598 (0.0006) [2023-12-26 16:04:23,356][105620] Updated weights for policy 1, policy_version 97608 (0.0008) [2023-12-26 16:04:23,734][105692] Updated weights for policy 0, policy_version 97178 (0.0010) [2023-12-26 16:04:23,788][105692] Updated weights for policy 0, policy_version 97189 (0.0010) [2023-12-26 16:04:23,841][105692] Updated weights for policy 0, policy_version 97199 (0.0010) [2023-12-26 16:04:23,924][105620] Updated weights for policy 1, policy_version 97618 (0.0006) [2023-12-26 16:04:23,981][105620] Updated weights for policy 1, policy_version 97628 (0.0005) [2023-12-26 16:04:24,033][105620] Updated weights for policy 1, policy_version 97638 (0.0005) [2023-12-26 16:04:24,093][105620] Updated weights for policy 1, policy_version 97648 (0.0006) [2023-12-26 16:04:24,575][105692] Updated weights for policy 0, policy_version 97209 (0.0006) [2023-12-26 16:04:24,631][105692] Updated weights for policy 0, policy_version 97219 (0.0010) [2023-12-26 16:04:24,688][105692] Updated weights for policy 0, policy_version 97229 (0.0005) [2023-12-26 16:04:24,696][105620] Updated weights for policy 1, policy_version 97658 (0.0008) [2023-12-26 16:04:24,746][105620] Updated weights for policy 1, policy_version 97668 (0.0006) [2023-12-26 16:04:24,793][105620] Updated weights for policy 1, policy_version 97678 (0.0005) [2023-12-26 16:04:25,339][105692] Updated weights for policy 0, policy_version 97239 (0.0005) [2023-12-26 16:04:25,398][105692] Updated weights for policy 0, policy_version 97249 (0.0007) [2023-12-26 16:04:25,458][105692] Updated weights for policy 0, policy_version 97259 (0.0006) [2023-12-26 16:04:25,512][105620] Updated weights for policy 1, policy_version 97688 (0.0009) [2023-12-26 16:04:25,570][105620] Updated weights for policy 1, policy_version 97698 (0.0010) [2023-12-26 16:04:25,630][105620] Updated weights for policy 1, policy_version 97708 (0.0008) [2023-12-26 16:04:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 49922048. Throughput: 0: 9618.9, 1: 10173.9. Samples: 49932388. Policy #0 lag: (min: 26.0, avg: 39.4, max: 40.0) [2023-12-26 16:04:26,062][104569] Avg episode reward: [(0, '2390.228'), (1, '8416.946')] [2023-12-26 16:04:26,165][105692] Updated weights for policy 0, policy_version 97269 (0.0007) [2023-12-26 16:04:26,221][105692] Updated weights for policy 0, policy_version 97279 (0.0009) [2023-12-26 16:04:26,273][105692] Updated weights for policy 0, policy_version 97289 (0.0009) [2023-12-26 16:04:26,342][105620] Updated weights for policy 1, policy_version 97718 (0.0008) [2023-12-26 16:04:26,406][105620] Updated weights for policy 1, policy_version 97728 (0.0009) [2023-12-26 16:04:26,459][105620] Updated weights for policy 1, policy_version 97738 (0.0007) [2023-12-26 16:04:27,067][105620] Updated weights for policy 1, policy_version 97748 (0.0005) [2023-12-26 16:04:27,111][105692] Updated weights for policy 0, policy_version 97299 (0.0009) [2023-12-26 16:04:27,126][105620] Updated weights for policy 1, policy_version 97758 (0.0006) [2023-12-26 16:04:27,162][105692] Updated weights for policy 0, policy_version 97309 (0.0009) [2023-12-26 16:04:27,185][105620] Updated weights for policy 1, policy_version 97768 (0.0005) [2023-12-26 16:04:27,215][105692] Updated weights for policy 0, policy_version 97319 (0.0010) [2023-12-26 16:04:27,733][105620] Updated weights for policy 1, policy_version 97778 (0.0006) [2023-12-26 16:04:27,791][105620] Updated weights for policy 1, policy_version 97788 (0.0005) [2023-12-26 16:04:27,852][105620] Updated weights for policy 1, policy_version 97798 (0.0005) [2023-12-26 16:04:27,914][105620] Updated weights for policy 1, policy_version 97808 (0.0005) [2023-12-26 16:04:28,064][105692] Updated weights for policy 0, policy_version 97329 (0.0009) [2023-12-26 16:04:28,127][105692] Updated weights for policy 0, policy_version 97339 (0.0010) [2023-12-26 16:04:28,198][105692] Updated weights for policy 0, policy_version 97349 (0.0010) [2023-12-26 16:04:28,263][105692] Updated weights for policy 0, policy_version 97359 (0.0010) [2023-12-26 16:04:28,402][105620] Updated weights for policy 1, policy_version 97818 (0.0008) [2023-12-26 16:04:28,457][105620] Updated weights for policy 1, policy_version 97828 (0.0005) [2023-12-26 16:04:28,506][105620] Updated weights for policy 1, policy_version 97838 (0.0005) [2023-12-26 16:04:29,123][105620] Updated weights for policy 1, policy_version 97848 (0.0009) [2023-12-26 16:04:29,137][105692] Updated weights for policy 0, policy_version 97369 (0.0006) [2023-12-26 16:04:29,181][105620] Updated weights for policy 1, policy_version 97858 (0.0010) [2023-12-26 16:04:29,195][105692] Updated weights for policy 0, policy_version 97379 (0.0006) [2023-12-26 16:04:29,243][105620] Updated weights for policy 1, policy_version 97868 (0.0008) [2023-12-26 16:04:29,256][105692] Updated weights for policy 0, policy_version 97389 (0.0009) [2023-12-26 16:04:29,887][105692] Updated weights for policy 0, policy_version 97399 (0.0008) [2023-12-26 16:04:29,952][105692] Updated weights for policy 0, policy_version 97409 (0.0008) [2023-12-26 16:04:29,971][105620] Updated weights for policy 1, policy_version 97878 (0.0009) [2023-12-26 16:04:30,014][105692] Updated weights for policy 0, policy_version 97419 (0.0008) [2023-12-26 16:04:30,034][105620] Updated weights for policy 1, policy_version 97888 (0.0011) [2023-12-26 16:04:30,087][105620] Updated weights for policy 1, policy_version 97898 (0.0010) [2023-12-26 16:04:30,715][105620] Updated weights for policy 1, policy_version 97908 (0.0010) [2023-12-26 16:04:30,776][105620] Updated weights for policy 1, policy_version 97918 (0.0008) [2023-12-26 16:04:30,832][105620] Updated weights for policy 1, policy_version 97928 (0.0006) [2023-12-26 16:04:30,885][105692] Updated weights for policy 0, policy_version 97429 (0.0008) [2023-12-26 16:04:30,940][105692] Updated weights for policy 0, policy_version 97439 (0.0010) [2023-12-26 16:04:30,988][105692] Updated weights for policy 0, policy_version 97449 (0.0010) [2023-12-26 16:04:31,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 50028544. Throughput: 0: 9581.6, 1: 10235.3. Samples: 49992920. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-26 16:04:31,063][104569] Avg episode reward: [(0, '6650.772'), (1, '8405.225')] [2023-12-26 16:04:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000097456_24952832.pth... [2023-12-26 16:04:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000097936_25075712.pth... [2023-12-26 16:04:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000096752_24772608.pth [2023-12-26 16:04:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000096336_24666112.pth [2023-12-26 16:04:31,507][105620] Updated weights for policy 1, policy_version 97938 (0.0006) [2023-12-26 16:04:31,567][105620] Updated weights for policy 1, policy_version 97948 (0.0010) [2023-12-26 16:04:31,632][105620] Updated weights for policy 1, policy_version 97958 (0.0009) [2023-12-26 16:04:31,695][105620] Updated weights for policy 1, policy_version 97968 (0.0009) [2023-12-26 16:04:31,760][105692] Updated weights for policy 0, policy_version 97459 (0.0009) [2023-12-26 16:04:31,809][105692] Updated weights for policy 0, policy_version 97469 (0.0008) [2023-12-26 16:04:31,858][105692] Updated weights for policy 0, policy_version 97479 (0.0011) [2023-12-26 16:04:32,532][105620] Updated weights for policy 1, policy_version 97978 (0.0010) [2023-12-26 16:04:32,545][105692] Updated weights for policy 0, policy_version 97489 (0.0010) [2023-12-26 16:04:32,591][105620] Updated weights for policy 1, policy_version 97988 (0.0010) [2023-12-26 16:04:32,603][105692] Updated weights for policy 0, policy_version 97499 (0.0010) [2023-12-26 16:04:32,650][105620] Updated weights for policy 1, policy_version 97998 (0.0010) [2023-12-26 16:04:32,662][105692] Updated weights for policy 0, policy_version 97509 (0.0010) [2023-12-26 16:04:32,720][105692] Updated weights for policy 0, policy_version 97519 (0.0010) [2023-12-26 16:04:33,391][105692] Updated weights for policy 0, policy_version 97529 (0.0006) [2023-12-26 16:04:33,397][105620] Updated weights for policy 1, policy_version 98008 (0.0008) [2023-12-26 16:04:33,441][105692] Updated weights for policy 0, policy_version 97539 (0.0005) [2023-12-26 16:04:33,445][105620] Updated weights for policy 1, policy_version 98018 (0.0009) [2023-12-26 16:04:33,491][105620] Updated weights for policy 1, policy_version 98028 (0.0007) [2023-12-26 16:04:33,500][105692] Updated weights for policy 0, policy_version 97549 (0.0008) [2023-12-26 16:04:34,200][105692] Updated weights for policy 0, policy_version 97559 (0.0009) [2023-12-26 16:04:34,266][105692] Updated weights for policy 0, policy_version 97569 (0.0008) [2023-12-26 16:04:34,273][105620] Updated weights for policy 1, policy_version 98038 (0.0007) [2023-12-26 16:04:34,324][105692] Updated weights for policy 0, policy_version 97579 (0.0008) [2023-12-26 16:04:34,326][105620] Updated weights for policy 1, policy_version 98048 (0.0006) [2023-12-26 16:04:34,384][105620] Updated weights for policy 1, policy_version 98058 (0.0008) [2023-12-26 16:04:35,035][105692] Updated weights for policy 0, policy_version 97589 (0.0008) [2023-12-26 16:04:35,094][105692] Updated weights for policy 0, policy_version 97599 (0.0007) [2023-12-26 16:04:35,145][105692] Updated weights for policy 0, policy_version 97609 (0.0006) [2023-12-26 16:04:35,172][105620] Updated weights for policy 1, policy_version 98068 (0.0010) [2023-12-26 16:04:35,224][105620] Updated weights for policy 1, policy_version 98078 (0.0008) [2023-12-26 16:04:35,287][105620] Updated weights for policy 1, policy_version 98088 (0.0009) [2023-12-26 16:04:35,756][105692] Updated weights for policy 0, policy_version 97619 (0.0005) [2023-12-26 16:04:35,805][105692] Updated weights for policy 0, policy_version 97629 (0.0005) [2023-12-26 16:04:35,859][105692] Updated weights for policy 0, policy_version 97639 (0.0006) [2023-12-26 16:04:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 50118656. Throughput: 0: 9528.1, 1: 10185.6. Samples: 50107144. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-26 16:04:36,062][104569] Avg episode reward: [(0, '8825.537'), (1, '8753.825')] [2023-12-26 16:04:36,152][105620] Updated weights for policy 1, policy_version 98098 (0.0009) [2023-12-26 16:04:36,215][105620] Updated weights for policy 1, policy_version 98108 (0.0008) [2023-12-26 16:04:36,277][105620] Updated weights for policy 1, policy_version 98118 (0.0009) [2023-12-26 16:04:36,337][105620] Updated weights for policy 1, policy_version 98128 (0.0009) [2023-12-26 16:04:36,521][105692] Updated weights for policy 0, policy_version 97649 (0.0006) [2023-12-26 16:04:36,582][105692] Updated weights for policy 0, policy_version 97659 (0.0009) [2023-12-26 16:04:36,645][105692] Updated weights for policy 0, policy_version 97669 (0.0009) [2023-12-26 16:04:36,716][105692] Updated weights for policy 0, policy_version 97679 (0.0010) [2023-12-26 16:04:37,071][105620] Updated weights for policy 1, policy_version 98138 (0.0009) [2023-12-26 16:04:37,133][105620] Updated weights for policy 1, policy_version 98148 (0.0009) [2023-12-26 16:04:37,195][105620] Updated weights for policy 1, policy_version 98158 (0.0009) [2023-12-26 16:04:37,466][105692] Updated weights for policy 0, policy_version 97689 (0.0009) [2023-12-26 16:04:37,526][105692] Updated weights for policy 0, policy_version 97699 (0.0009) [2023-12-26 16:04:37,579][105692] Updated weights for policy 0, policy_version 97709 (0.0009) [2023-12-26 16:04:37,948][105620] Updated weights for policy 1, policy_version 98168 (0.0009) [2023-12-26 16:04:38,013][105620] Updated weights for policy 1, policy_version 98178 (0.0009) [2023-12-26 16:04:38,064][105620] Updated weights for policy 1, policy_version 98188 (0.0009) [2023-12-26 16:04:38,343][105692] Updated weights for policy 0, policy_version 97719 (0.0010) [2023-12-26 16:04:38,409][105692] Updated weights for policy 0, policy_version 97729 (0.0011) [2023-12-26 16:04:38,477][105692] Updated weights for policy 0, policy_version 97739 (0.0011) [2023-12-26 16:04:38,722][105620] Updated weights for policy 1, policy_version 98198 (0.0012) [2023-12-26 16:04:38,776][105620] Updated weights for policy 1, policy_version 98208 (0.0009) [2023-12-26 16:04:38,830][105620] Updated weights for policy 1, policy_version 98218 (0.0007) [2023-12-26 16:04:39,136][105692] Updated weights for policy 0, policy_version 97749 (0.0011) [2023-12-26 16:04:39,199][105692] Updated weights for policy 0, policy_version 97759 (0.0009) [2023-12-26 16:04:39,270][105692] Updated weights for policy 0, policy_version 97769 (0.0010) [2023-12-26 16:04:39,584][105620] Updated weights for policy 1, policy_version 98228 (0.0007) [2023-12-26 16:04:39,647][105620] Updated weights for policy 1, policy_version 98238 (0.0010) [2023-12-26 16:04:39,706][105620] Updated weights for policy 1, policy_version 98248 (0.0010) [2023-12-26 16:04:39,915][105692] Updated weights for policy 0, policy_version 97779 (0.0010) [2023-12-26 16:04:39,982][105692] Updated weights for policy 0, policy_version 97789 (0.0009) [2023-12-26 16:04:40,045][105692] Updated weights for policy 0, policy_version 97799 (0.0009) [2023-12-26 16:04:40,540][105620] Updated weights for policy 1, policy_version 98258 (0.0010) [2023-12-26 16:04:40,602][105620] Updated weights for policy 1, policy_version 98268 (0.0009) [2023-12-26 16:04:40,649][105620] Updated weights for policy 1, policy_version 98278 (0.0009) [2023-12-26 16:04:40,696][105620] Updated weights for policy 1, policy_version 98288 (0.0009) [2023-12-26 16:04:40,767][105692] Updated weights for policy 0, policy_version 97809 (0.0010) [2023-12-26 16:04:40,827][105692] Updated weights for policy 0, policy_version 97819 (0.0010) [2023-12-26 16:04:40,878][105692] Updated weights for policy 0, policy_version 97829 (0.0010) [2023-12-26 16:04:40,925][105692] Updated weights for policy 0, policy_version 97839 (0.0009) [2023-12-26 16:04:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 50216960. Throughput: 0: 9592.2, 1: 10017.0. Samples: 50221860. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-26 16:04:41,062][104569] Avg episode reward: [(0, '9007.668'), (1, '8754.905')] [2023-12-26 16:04:41,513][105620] Updated weights for policy 1, policy_version 98298 (0.0009) [2023-12-26 16:04:41,574][105620] Updated weights for policy 1, policy_version 98308 (0.0009) [2023-12-26 16:04:41,632][105620] Updated weights for policy 1, policy_version 98318 (0.0008) [2023-12-26 16:04:41,687][105692] Updated weights for policy 0, policy_version 97849 (0.0008) [2023-12-26 16:04:41,751][105692] Updated weights for policy 0, policy_version 97859 (0.0008) [2023-12-26 16:04:41,813][105692] Updated weights for policy 0, policy_version 97869 (0.0009) [2023-12-26 16:04:42,442][105620] Updated weights for policy 1, policy_version 98328 (0.0010) [2023-12-26 16:04:42,498][105620] Updated weights for policy 1, policy_version 98338 (0.0011) [2023-12-26 16:04:42,505][105692] Updated weights for policy 0, policy_version 97879 (0.0007) [2023-12-26 16:04:42,558][105620] Updated weights for policy 1, policy_version 98348 (0.0011) [2023-12-26 16:04:42,561][105692] Updated weights for policy 0, policy_version 97889 (0.0006) [2023-12-26 16:04:42,619][105692] Updated weights for policy 0, policy_version 97899 (0.0009) [2023-12-26 16:04:43,254][105692] Updated weights for policy 0, policy_version 97909 (0.0007) [2023-12-26 16:04:43,301][105692] Updated weights for policy 0, policy_version 97919 (0.0005) [2023-12-26 16:04:43,352][105692] Updated weights for policy 0, policy_version 97929 (0.0005) [2023-12-26 16:04:43,367][105620] Updated weights for policy 1, policy_version 98358 (0.0008) [2023-12-26 16:04:43,422][105620] Updated weights for policy 1, policy_version 98368 (0.0006) [2023-12-26 16:04:43,468][105620] Updated weights for policy 1, policy_version 98378 (0.0005) [2023-12-26 16:04:43,882][105692] Updated weights for policy 0, policy_version 97939 (0.0006) [2023-12-26 16:04:43,938][105692] Updated weights for policy 0, policy_version 97949 (0.0005) [2023-12-26 16:04:44,003][105692] Updated weights for policy 0, policy_version 97959 (0.0006) [2023-12-26 16:04:44,054][105620] Updated weights for policy 1, policy_version 98388 (0.0005) [2023-12-26 16:04:44,111][105620] Updated weights for policy 1, policy_version 98398 (0.0005) [2023-12-26 16:04:44,171][105620] Updated weights for policy 1, policy_version 98408 (0.0006) [2023-12-26 16:04:44,685][105692] Updated weights for policy 0, policy_version 97969 (0.0006) [2023-12-26 16:04:44,700][105620] Updated weights for policy 1, policy_version 98418 (0.0005) [2023-12-26 16:04:44,734][105692] Updated weights for policy 0, policy_version 97980 (0.0009) [2023-12-26 16:04:44,754][105620] Updated weights for policy 1, policy_version 98428 (0.0006) [2023-12-26 16:04:44,796][105692] Updated weights for policy 0, policy_version 97990 (0.0009) [2023-12-26 16:04:44,819][105620] Updated weights for policy 1, policy_version 98438 (0.0008) [2023-12-26 16:04:44,850][105692] Updated weights for policy 0, policy_version 98000 (0.0007) [2023-12-26 16:04:44,879][105620] Updated weights for policy 1, policy_version 98448 (0.0008) [2023-12-26 16:04:45,612][105620] Updated weights for policy 1, policy_version 98458 (0.0009) [2023-12-26 16:04:45,642][105692] Updated weights for policy 0, policy_version 98010 (0.0007) [2023-12-26 16:04:45,675][105620] Updated weights for policy 1, policy_version 98468 (0.0008) [2023-12-26 16:04:45,702][105692] Updated weights for policy 0, policy_version 98020 (0.0006) [2023-12-26 16:04:45,733][105620] Updated weights for policy 1, policy_version 98478 (0.0009) [2023-12-26 16:04:45,761][105692] Updated weights for policy 0, policy_version 98030 (0.0007) [2023-12-26 16:04:46,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 50315264. Throughput: 0: 9590.3, 1: 9899.4. Samples: 50280196. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-26 16:04:46,063][104569] Avg episode reward: [(0, '9007.133'), (1, '8659.998')] [2023-12-26 16:04:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000098032_25100288.pth... [2023-12-26 16:04:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000098480_25214976.pth... [2023-12-26 16:04:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000097328_24920064.pth [2023-12-26 16:04:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000096912_24813568.pth [2023-12-26 16:04:46,493][105692] Updated weights for policy 0, policy_version 98040 (0.0008) [2023-12-26 16:04:46,496][105620] Updated weights for policy 1, policy_version 98488 (0.0006) [2023-12-26 16:04:46,545][105620] Updated weights for policy 1, policy_version 98498 (0.0006) [2023-12-26 16:04:46,551][105692] Updated weights for policy 0, policy_version 98050 (0.0007) [2023-12-26 16:04:46,590][105620] Updated weights for policy 1, policy_version 98508 (0.0007) [2023-12-26 16:04:46,606][105692] Updated weights for policy 0, policy_version 98060 (0.0008) [2023-12-26 16:04:47,365][105692] Updated weights for policy 0, policy_version 98070 (0.0009) [2023-12-26 16:04:47,395][105620] Updated weights for policy 1, policy_version 98518 (0.0008) [2023-12-26 16:04:47,426][105692] Updated weights for policy 0, policy_version 98080 (0.0007) [2023-12-26 16:04:47,446][105620] Updated weights for policy 1, policy_version 98528 (0.0006) [2023-12-26 16:04:47,488][105692] Updated weights for policy 0, policy_version 98090 (0.0007) [2023-12-26 16:04:47,510][105620] Updated weights for policy 1, policy_version 98538 (0.0008) [2023-12-26 16:04:48,174][105620] Updated weights for policy 1, policy_version 98548 (0.0009) [2023-12-26 16:04:48,231][105620] Updated weights for policy 1, policy_version 98558 (0.0010) [2023-12-26 16:04:48,247][105692] Updated weights for policy 0, policy_version 98100 (0.0007) [2023-12-26 16:04:48,290][105620] Updated weights for policy 1, policy_version 98568 (0.0010) [2023-12-26 16:04:48,300][105692] Updated weights for policy 0, policy_version 98110 (0.0009) [2023-12-26 16:04:48,356][105692] Updated weights for policy 0, policy_version 98120 (0.0006) [2023-12-26 16:04:49,011][105692] Updated weights for policy 0, policy_version 98130 (0.0006) [2023-12-26 16:04:49,028][105620] Updated weights for policy 1, policy_version 98578 (0.0011) [2023-12-26 16:04:49,072][105692] Updated weights for policy 0, policy_version 98140 (0.0008) [2023-12-26 16:04:49,084][105620] Updated weights for policy 1, policy_version 98588 (0.0010) [2023-12-26 16:04:49,130][105692] Updated weights for policy 0, policy_version 98150 (0.0008) [2023-12-26 16:04:49,136][105620] Updated weights for policy 1, policy_version 98598 (0.0009) [2023-12-26 16:04:49,183][105620] Updated weights for policy 1, policy_version 98608 (0.0010) [2023-12-26 16:04:49,186][105692] Updated weights for policy 0, policy_version 98160 (0.0007) [2023-12-26 16:04:49,927][105692] Updated weights for policy 0, policy_version 98170 (0.0009) [2023-12-26 16:04:49,979][105620] Updated weights for policy 1, policy_version 98618 (0.0010) [2023-12-26 16:04:49,987][105692] Updated weights for policy 0, policy_version 98180 (0.0009) [2023-12-26 16:04:50,036][105620] Updated weights for policy 1, policy_version 98628 (0.0007) [2023-12-26 16:04:50,049][105692] Updated weights for policy 0, policy_version 98190 (0.0008) [2023-12-26 16:04:50,090][105620] Updated weights for policy 1, policy_version 98638 (0.0008) [2023-12-26 16:04:50,776][105692] Updated weights for policy 0, policy_version 98200 (0.0008) [2023-12-26 16:04:50,840][105692] Updated weights for policy 0, policy_version 98210 (0.0009) [2023-12-26 16:04:50,869][105620] Updated weights for policy 1, policy_version 98648 (0.0007) [2023-12-26 16:04:50,902][105692] Updated weights for policy 0, policy_version 98220 (0.0006) [2023-12-26 16:04:50,938][105620] Updated weights for policy 1, policy_version 98658 (0.0008) [2023-12-26 16:04:51,001][105620] Updated weights for policy 1, policy_version 98668 (0.0009) [2023-12-26 16:04:51,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.7, 300 sec: 19605.3). Total num frames: 50413568. Throughput: 0: 9559.0, 1: 9893.5. Samples: 50397584. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-26 16:04:51,063][104569] Avg episode reward: [(0, '9269.695'), (1, '9176.100')] [2023-12-26 16:04:51,620][105692] Updated weights for policy 0, policy_version 98230 (0.0009) [2023-12-26 16:04:51,689][105692] Updated weights for policy 0, policy_version 98240 (0.0010) [2023-12-26 16:04:51,758][105692] Updated weights for policy 0, policy_version 98250 (0.0011) [2023-12-26 16:04:51,771][105620] Updated weights for policy 1, policy_version 98678 (0.0008) [2023-12-26 16:04:51,833][105620] Updated weights for policy 1, policy_version 98688 (0.0009) [2023-12-26 16:04:51,883][105620] Updated weights for policy 1, policy_version 98698 (0.0009) [2023-12-26 16:04:52,510][105692] Updated weights for policy 0, policy_version 98260 (0.0010) [2023-12-26 16:04:52,564][105692] Updated weights for policy 0, policy_version 98271 (0.0009) [2023-12-26 16:04:52,619][105692] Updated weights for policy 0, policy_version 98281 (0.0005) [2023-12-26 16:04:52,655][105620] Updated weights for policy 1, policy_version 98708 (0.0008) [2023-12-26 16:04:52,720][105620] Updated weights for policy 1, policy_version 98718 (0.0009) [2023-12-26 16:04:52,787][105620] Updated weights for policy 1, policy_version 98728 (0.0010) [2023-12-26 16:04:53,216][105692] Updated weights for policy 0, policy_version 98291 (0.0005) [2023-12-26 16:04:53,276][105692] Updated weights for policy 0, policy_version 98301 (0.0005) [2023-12-26 16:04:53,322][105692] Updated weights for policy 0, policy_version 98311 (0.0005) [2023-12-26 16:04:53,464][105620] Updated weights for policy 1, policy_version 98738 (0.0009) [2023-12-26 16:04:53,523][105620] Updated weights for policy 1, policy_version 98748 (0.0005) [2023-12-26 16:04:53,586][105620] Updated weights for policy 1, policy_version 98758 (0.0005) [2023-12-26 16:04:53,645][105620] Updated weights for policy 1, policy_version 98768 (0.0005) [2023-12-26 16:04:53,930][105692] Updated weights for policy 0, policy_version 98321 (0.0005) [2023-12-26 16:04:53,980][105692] Updated weights for policy 0, policy_version 98331 (0.0005) [2023-12-26 16:04:54,034][105692] Updated weights for policy 0, policy_version 98341 (0.0005) [2023-12-26 16:04:54,097][105692] Updated weights for policy 0, policy_version 98351 (0.0005) [2023-12-26 16:04:54,313][105620] Updated weights for policy 1, policy_version 98778 (0.0010) [2023-12-26 16:04:54,369][105620] Updated weights for policy 1, policy_version 98788 (0.0010) [2023-12-26 16:04:54,429][105620] Updated weights for policy 1, policy_version 98798 (0.0011) [2023-12-26 16:04:54,669][105692] Updated weights for policy 0, policy_version 98361 (0.0005) [2023-12-26 16:04:54,724][105692] Updated weights for policy 0, policy_version 98371 (0.0010) [2023-12-26 16:04:54,774][105692] Updated weights for policy 0, policy_version 98381 (0.0007) [2023-12-26 16:04:55,185][105620] Updated weights for policy 1, policy_version 98808 (0.0010) [2023-12-26 16:04:55,242][105620] Updated weights for policy 1, policy_version 98818 (0.0010) [2023-12-26 16:04:55,310][105620] Updated weights for policy 1, policy_version 98828 (0.0010) [2023-12-26 16:04:55,327][105692] Updated weights for policy 0, policy_version 98391 (0.0006) [2023-12-26 16:04:55,377][105692] Updated weights for policy 0, policy_version 98401 (0.0005) [2023-12-26 16:04:55,429][105692] Updated weights for policy 0, policy_version 98411 (0.0008) [2023-12-26 16:04:55,963][105620] Updated weights for policy 1, policy_version 98838 (0.0007) [2023-12-26 16:04:56,013][105620] Updated weights for policy 1, policy_version 98848 (0.0006) [2023-12-26 16:04:56,062][104569] Fps is (10 sec: 18842.3, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 50503680. Throughput: 0: 9688.0, 1: 9827.1. Samples: 50517652. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-26 16:04:56,062][104569] Avg episode reward: [(0, '9175.454'), (1, '9175.858')] [2023-12-26 16:04:56,065][105620] Updated weights for policy 1, policy_version 98858 (0.0007) [2023-12-26 16:04:56,153][105692] Updated weights for policy 0, policy_version 98421 (0.0010) [2023-12-26 16:04:56,209][105692] Updated weights for policy 0, policy_version 98431 (0.0010) [2023-12-26 16:04:56,261][105692] Updated weights for policy 0, policy_version 98441 (0.0010) [2023-12-26 16:04:56,793][105620] Updated weights for policy 1, policy_version 98868 (0.0007) [2023-12-26 16:04:56,852][105620] Updated weights for policy 1, policy_version 98878 (0.0005) [2023-12-26 16:04:56,877][105692] Updated weights for policy 0, policy_version 98451 (0.0007) [2023-12-26 16:04:56,912][105620] Updated weights for policy 1, policy_version 98888 (0.0006) [2023-12-26 16:04:56,931][105692] Updated weights for policy 0, policy_version 98461 (0.0010) [2023-12-26 16:04:56,975][105692] Updated weights for policy 0, policy_version 98471 (0.0010) [2023-12-26 16:04:57,466][105620] Updated weights for policy 1, policy_version 98898 (0.0009) [2023-12-26 16:04:57,522][105620] Updated weights for policy 1, policy_version 98908 (0.0006) [2023-12-26 16:04:57,585][105620] Updated weights for policy 1, policy_version 98918 (0.0005) [2023-12-26 16:04:57,651][105620] Updated weights for policy 1, policy_version 98928 (0.0005) [2023-12-26 16:04:57,733][105692] Updated weights for policy 0, policy_version 98481 (0.0010) [2023-12-26 16:04:57,784][105692] Updated weights for policy 0, policy_version 98491 (0.0010) [2023-12-26 16:04:57,834][105692] Updated weights for policy 0, policy_version 98501 (0.0010) [2023-12-26 16:04:57,889][105692] Updated weights for policy 0, policy_version 98511 (0.0010) [2023-12-26 16:04:58,181][105620] Updated weights for policy 1, policy_version 98938 (0.0007) [2023-12-26 16:04:58,244][105620] Updated weights for policy 1, policy_version 98948 (0.0008) [2023-12-26 16:04:58,310][105620] Updated weights for policy 1, policy_version 98958 (0.0007) [2023-12-26 16:04:58,662][105692] Updated weights for policy 0, policy_version 98521 (0.0008) [2023-12-26 16:04:58,722][105692] Updated weights for policy 0, policy_version 98531 (0.0007) [2023-12-26 16:04:58,792][105692] Updated weights for policy 0, policy_version 98541 (0.0009) [2023-12-26 16:04:59,054][105620] Updated weights for policy 1, policy_version 98968 (0.0010) [2023-12-26 16:04:59,119][105620] Updated weights for policy 1, policy_version 98978 (0.0008) [2023-12-26 16:04:59,189][105620] Updated weights for policy 1, policy_version 98988 (0.0009) [2023-12-26 16:04:59,375][105692] Updated weights for policy 0, policy_version 98551 (0.0007) [2023-12-26 16:04:59,447][105692] Updated weights for policy 0, policy_version 98561 (0.0005) [2023-12-26 16:04:59,512][105692] Updated weights for policy 0, policy_version 98571 (0.0006) [2023-12-26 16:04:59,949][105620] Updated weights for policy 1, policy_version 98998 (0.0008) [2023-12-26 16:05:00,012][105620] Updated weights for policy 1, policy_version 99008 (0.0008) [2023-12-26 16:05:00,076][105620] Updated weights for policy 1, policy_version 99018 (0.0005) [2023-12-26 16:05:00,077][105692] Updated weights for policy 0, policy_version 98581 (0.0008) [2023-12-26 16:05:00,142][105692] Updated weights for policy 0, policy_version 98591 (0.0010) [2023-12-26 16:05:00,213][105692] Updated weights for policy 0, policy_version 98601 (0.0007) [2023-12-26 16:05:00,635][105620] Updated weights for policy 1, policy_version 99028 (0.0007) [2023-12-26 16:05:00,700][105620] Updated weights for policy 1, policy_version 99038 (0.0006) [2023-12-26 16:05:00,763][105620] Updated weights for policy 1, policy_version 99048 (0.0005) [2023-12-26 16:05:00,902][105692] Updated weights for policy 0, policy_version 98611 (0.0006) [2023-12-26 16:05:00,961][105692] Updated weights for policy 0, policy_version 98622 (0.0010) [2023-12-26 16:05:01,015][105692] Updated weights for policy 0, policy_version 98632 (0.0006) [2023-12-26 16:05:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 50610176. Throughput: 0: 9682.1, 1: 9843.5. Samples: 50579132. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-26 16:05:01,063][104569] Avg episode reward: [(0, '9177.028'), (1, '9263.159')] [2023-12-26 16:05:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000098640_25255936.pth... [2023-12-26 16:05:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000099056_25362432.pth... [2023-12-26 16:05:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000097936_25075712.pth [2023-12-26 16:05:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000097456_24952832.pth [2023-12-26 16:05:01,348][105620] Updated weights for policy 1, policy_version 99058 (0.0006) [2023-12-26 16:05:01,413][105620] Updated weights for policy 1, policy_version 99068 (0.0010) [2023-12-26 16:05:01,466][105620] Updated weights for policy 1, policy_version 99078 (0.0011) [2023-12-26 16:05:01,518][105620] Updated weights for policy 1, policy_version 99088 (0.0009) [2023-12-26 16:05:01,790][105692] Updated weights for policy 0, policy_version 98642 (0.0009) [2023-12-26 16:05:01,852][105692] Updated weights for policy 0, policy_version 98652 (0.0009) [2023-12-26 16:05:01,914][105692] Updated weights for policy 0, policy_version 98662 (0.0005) [2023-12-26 16:05:01,979][105692] Updated weights for policy 0, policy_version 98672 (0.0005) [2023-12-26 16:05:02,318][105620] Updated weights for policy 1, policy_version 99098 (0.0008) [2023-12-26 16:05:02,367][105620] Updated weights for policy 1, policy_version 99108 (0.0008) [2023-12-26 16:05:02,418][105620] Updated weights for policy 1, policy_version 99118 (0.0008) [2023-12-26 16:05:02,648][105692] Updated weights for policy 0, policy_version 98682 (0.0009) [2023-12-26 16:05:02,699][105692] Updated weights for policy 0, policy_version 98692 (0.0009) [2023-12-26 16:05:02,749][105692] Updated weights for policy 0, policy_version 98702 (0.0009) [2023-12-26 16:05:03,195][105620] Updated weights for policy 1, policy_version 99128 (0.0009) [2023-12-26 16:05:03,248][105620] Updated weights for policy 1, policy_version 99138 (0.0009) [2023-12-26 16:05:03,298][105620] Updated weights for policy 1, policy_version 99148 (0.0009) [2023-12-26 16:05:03,506][105692] Updated weights for policy 0, policy_version 98712 (0.0009) [2023-12-26 16:05:03,554][105692] Updated weights for policy 0, policy_version 98722 (0.0009) [2023-12-26 16:05:03,604][105692] Updated weights for policy 0, policy_version 98732 (0.0009) [2023-12-26 16:05:04,028][105620] Updated weights for policy 1, policy_version 99158 (0.0007) [2023-12-26 16:05:04,082][105620] Updated weights for policy 1, policy_version 99168 (0.0005) [2023-12-26 16:05:04,141][105620] Updated weights for policy 1, policy_version 99178 (0.0006) [2023-12-26 16:05:04,451][105692] Updated weights for policy 0, policy_version 98742 (0.0007) [2023-12-26 16:05:04,511][105692] Updated weights for policy 0, policy_version 98752 (0.0008) [2023-12-26 16:05:04,571][105692] Updated weights for policy 0, policy_version 98762 (0.0010) [2023-12-26 16:05:04,727][105620] Updated weights for policy 1, policy_version 99188 (0.0006) [2023-12-26 16:05:04,781][105620] Updated weights for policy 1, policy_version 99198 (0.0005) [2023-12-26 16:05:04,840][105620] Updated weights for policy 1, policy_version 99208 (0.0007) [2023-12-26 16:05:05,330][105692] Updated weights for policy 0, policy_version 98772 (0.0009) [2023-12-26 16:05:05,373][105692] Updated weights for policy 0, policy_version 98782 (0.0005) [2023-12-26 16:05:05,437][105692] Updated weights for policy 0, policy_version 98792 (0.0005) [2023-12-26 16:05:05,570][105620] Updated weights for policy 1, policy_version 99218 (0.0008) [2023-12-26 16:05:05,623][105620] Updated weights for policy 1, policy_version 99228 (0.0005) [2023-12-26 16:05:05,681][105620] Updated weights for policy 1, policy_version 99238 (0.0005) [2023-12-26 16:05:05,746][105620] Updated weights for policy 1, policy_version 99248 (0.0008) [2023-12-26 16:05:05,983][105692] Updated weights for policy 0, policy_version 98802 (0.0005) [2023-12-26 16:05:06,041][105692] Updated weights for policy 0, policy_version 98812 (0.0008) [2023-12-26 16:05:06,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 50708480. Throughput: 0: 9804.2, 1: 9864.7. Samples: 50698892. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-26 16:05:06,062][104569] Avg episode reward: [(0, '9091.825'), (1, '9263.725')] [2023-12-26 16:05:06,102][105692] Updated weights for policy 0, policy_version 98822 (0.0010) [2023-12-26 16:05:06,168][105692] Updated weights for policy 0, policy_version 98832 (0.0007) [2023-12-26 16:05:06,548][105620] Updated weights for policy 1, policy_version 99258 (0.0008) [2023-12-26 16:05:06,615][105620] Updated weights for policy 1, policy_version 99268 (0.0008) [2023-12-26 16:05:06,674][105620] Updated weights for policy 1, policy_version 99278 (0.0008) [2023-12-26 16:05:06,819][105692] Updated weights for policy 0, policy_version 98842 (0.0009) [2023-12-26 16:05:06,873][105692] Updated weights for policy 0, policy_version 98852 (0.0011) [2023-12-26 16:05:06,927][105692] Updated weights for policy 0, policy_version 98862 (0.0011) [2023-12-26 16:05:07,453][105620] Updated weights for policy 1, policy_version 99288 (0.0008) [2023-12-26 16:05:07,508][105620] Updated weights for policy 1, policy_version 99298 (0.0008) [2023-12-26 16:05:07,577][105620] Updated weights for policy 1, policy_version 99308 (0.0007) [2023-12-26 16:05:07,686][105692] Updated weights for policy 0, policy_version 98872 (0.0009) [2023-12-26 16:05:07,745][105692] Updated weights for policy 0, policy_version 98882 (0.0006) [2023-12-26 16:05:07,790][105692] Updated weights for policy 0, policy_version 98892 (0.0005) [2023-12-26 16:05:08,349][105620] Updated weights for policy 1, policy_version 99318 (0.0009) [2023-12-26 16:05:08,418][105620] Updated weights for policy 1, policy_version 99328 (0.0008) [2023-12-26 16:05:08,443][105692] Updated weights for policy 0, policy_version 98902 (0.0007) [2023-12-26 16:05:08,477][105620] Updated weights for policy 1, policy_version 99338 (0.0006) [2023-12-26 16:05:08,508][105692] Updated weights for policy 0, policy_version 98912 (0.0008) [2023-12-26 16:05:08,573][105692] Updated weights for policy 0, policy_version 98922 (0.0009) [2023-12-26 16:05:09,159][105692] Updated weights for policy 0, policy_version 98932 (0.0007) [2023-12-26 16:05:09,203][105692] Updated weights for policy 0, policy_version 98942 (0.0005) [2023-12-26 16:05:09,262][105692] Updated weights for policy 0, policy_version 98952 (0.0008) [2023-12-26 16:05:09,286][105620] Updated weights for policy 1, policy_version 99348 (0.0007) [2023-12-26 16:05:09,352][105620] Updated weights for policy 1, policy_version 99358 (0.0007) [2023-12-26 16:05:09,418][105620] Updated weights for policy 1, policy_version 99368 (0.0009) [2023-12-26 16:05:10,015][105692] Updated weights for policy 0, policy_version 98962 (0.0008) [2023-12-26 16:05:10,079][105692] Updated weights for policy 0, policy_version 98972 (0.0005) [2023-12-26 16:05:10,142][105692] Updated weights for policy 0, policy_version 98982 (0.0009) [2023-12-26 16:05:10,206][105692] Updated weights for policy 0, policy_version 98992 (0.0006) [2023-12-26 16:05:10,239][105620] Updated weights for policy 1, policy_version 99378 (0.0009) [2023-12-26 16:05:10,301][105620] Updated weights for policy 1, policy_version 99388 (0.0009) [2023-12-26 16:05:10,366][105620] Updated weights for policy 1, policy_version 99398 (0.0008) [2023-12-26 16:05:10,428][105620] Updated weights for policy 1, policy_version 99408 (0.0005) [2023-12-26 16:05:10,872][105692] Updated weights for policy 0, policy_version 99002 (0.0009) [2023-12-26 16:05:10,934][105692] Updated weights for policy 0, policy_version 99012 (0.0010) [2023-12-26 16:05:10,988][105692] Updated weights for policy 0, policy_version 99022 (0.0010) [2023-12-26 16:05:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 50806784. Throughput: 0: 9892.6, 1: 9702.9. Samples: 50814188. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-26 16:05:11,063][104569] Avg episode reward: [(0, '9270.553'), (1, '9352.376')] [2023-12-26 16:05:11,082][105620] Updated weights for policy 1, policy_version 99418 (0.0008) [2023-12-26 16:05:11,150][105620] Updated weights for policy 1, policy_version 99428 (0.0009) [2023-12-26 16:05:11,205][105620] Updated weights for policy 1, policy_version 99438 (0.0008) [2023-12-26 16:05:11,767][105692] Updated weights for policy 0, policy_version 99032 (0.0009) [2023-12-26 16:05:11,816][105692] Updated weights for policy 0, policy_version 99042 (0.0008) [2023-12-26 16:05:11,868][105692] Updated weights for policy 0, policy_version 99052 (0.0006) [2023-12-26 16:05:11,976][105620] Updated weights for policy 1, policy_version 99448 (0.0008) [2023-12-26 16:05:12,044][105620] Updated weights for policy 1, policy_version 99458 (0.0009) [2023-12-26 16:05:12,110][105620] Updated weights for policy 1, policy_version 99468 (0.0010) [2023-12-26 16:05:12,627][105692] Updated weights for policy 0, policy_version 99062 (0.0005) [2023-12-26 16:05:12,690][105692] Updated weights for policy 0, policy_version 99072 (0.0008) [2023-12-26 16:05:12,754][105692] Updated weights for policy 0, policy_version 99082 (0.0007) [2023-12-26 16:05:12,831][105620] Updated weights for policy 1, policy_version 99478 (0.0008) [2023-12-26 16:05:12,887][105620] Updated weights for policy 1, policy_version 99488 (0.0005) [2023-12-26 16:05:12,942][105620] Updated weights for policy 1, policy_version 99498 (0.0006) [2023-12-26 16:05:13,536][105692] Updated weights for policy 0, policy_version 99092 (0.0009) [2023-12-26 16:05:13,538][105620] Updated weights for policy 1, policy_version 99508 (0.0006) [2023-12-26 16:05:13,592][105692] Updated weights for policy 0, policy_version 99102 (0.0007) [2023-12-26 16:05:13,594][105620] Updated weights for policy 1, policy_version 99518 (0.0007) [2023-12-26 16:05:13,642][105620] Updated weights for policy 1, policy_version 99528 (0.0006) [2023-12-26 16:05:13,651][105692] Updated weights for policy 0, policy_version 99112 (0.0008) [2023-12-26 16:05:14,317][105692] Updated weights for policy 0, policy_version 99122 (0.0009) [2023-12-26 16:05:14,381][105692] Updated weights for policy 0, policy_version 99132 (0.0006) [2023-12-26 16:05:14,415][105620] Updated weights for policy 1, policy_version 99538 (0.0009) [2023-12-26 16:05:14,445][105692] Updated weights for policy 0, policy_version 99142 (0.0005) [2023-12-26 16:05:14,469][105620] Updated weights for policy 1, policy_version 99548 (0.0009) [2023-12-26 16:05:14,507][105692] Updated weights for policy 0, policy_version 99152 (0.0007) [2023-12-26 16:05:14,518][105620] Updated weights for policy 1, policy_version 99558 (0.0007) [2023-12-26 16:05:14,563][105620] Updated weights for policy 1, policy_version 99568 (0.0007) [2023-12-26 16:05:15,217][105692] Updated weights for policy 0, policy_version 99162 (0.0007) [2023-12-26 16:05:15,263][105692] Updated weights for policy 0, policy_version 99172 (0.0008) [2023-12-26 16:05:15,313][105692] Updated weights for policy 0, policy_version 99182 (0.0009) [2023-12-26 16:05:15,352][105620] Updated weights for policy 1, policy_version 99578 (0.0008) [2023-12-26 16:05:15,407][105620] Updated weights for policy 1, policy_version 99588 (0.0009) [2023-12-26 16:05:15,465][105620] Updated weights for policy 1, policy_version 99598 (0.0010) [2023-12-26 16:05:16,050][105692] Updated weights for policy 0, policy_version 99192 (0.0005) [2023-12-26 16:05:16,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 50896896. Throughput: 0: 9948.6, 1: 9571.1. Samples: 50871312. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-26 16:05:16,063][104569] Avg episode reward: [(0, '9354.485'), (1, '9352.990')] [2023-12-26 16:05:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000099600_25501696.pth... [2023-12-26 16:05:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000098480_25214976.pth [2023-12-26 16:05:16,109][105692] Updated weights for policy 0, policy_version 99202 (0.0006) [2023-12-26 16:05:16,138][105620] Updated weights for policy 1, policy_version 99608 (0.0010) [2023-12-26 16:05:16,154][105692] Updated weights for policy 0, policy_version 99212 (0.0005) [2023-12-26 16:05:16,174][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000099216_25403392.pth... [2023-12-26 16:05:16,177][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000098032_25100288.pth [2023-12-26 16:05:16,190][105620] Updated weights for policy 1, policy_version 99618 (0.0007) [2023-12-26 16:05:16,252][105620] Updated weights for policy 1, policy_version 99628 (0.0005) [2023-12-26 16:05:16,683][105692] Updated weights for policy 0, policy_version 99222 (0.0005) [2023-12-26 16:05:16,745][105692] Updated weights for policy 0, policy_version 99232 (0.0005) [2023-12-26 16:05:16,802][105692] Updated weights for policy 0, policy_version 99242 (0.0005) [2023-12-26 16:05:16,888][105620] Updated weights for policy 1, policy_version 99638 (0.0005) [2023-12-26 16:05:16,939][105620] Updated weights for policy 1, policy_version 99648 (0.0005) [2023-12-26 16:05:16,993][105620] Updated weights for policy 1, policy_version 99658 (0.0008) [2023-12-26 16:05:17,336][105692] Updated weights for policy 0, policy_version 99252 (0.0006) [2023-12-26 16:05:17,397][105692] Updated weights for policy 0, policy_version 99262 (0.0009) [2023-12-26 16:05:17,463][105692] Updated weights for policy 0, policy_version 99272 (0.0008) [2023-12-26 16:05:17,645][105620] Updated weights for policy 1, policy_version 99668 (0.0009) [2023-12-26 16:05:17,711][105620] Updated weights for policy 1, policy_version 99678 (0.0010) [2023-12-26 16:05:17,770][105620] Updated weights for policy 1, policy_version 99688 (0.0008) [2023-12-26 16:05:18,305][105692] Updated weights for policy 0, policy_version 99282 (0.0009) [2023-12-26 16:05:18,371][105692] Updated weights for policy 0, policy_version 99292 (0.0009) [2023-12-26 16:05:18,437][105692] Updated weights for policy 0, policy_version 99302 (0.0008) [2023-12-26 16:05:18,439][105620] Updated weights for policy 1, policy_version 99698 (0.0006) [2023-12-26 16:05:18,496][105620] Updated weights for policy 1, policy_version 99708 (0.0006) [2023-12-26 16:05:18,498][105692] Updated weights for policy 0, policy_version 99312 (0.0008) [2023-12-26 16:05:18,554][105620] Updated weights for policy 1, policy_version 99718 (0.0008) [2023-12-26 16:05:18,609][105620] Updated weights for policy 1, policy_version 99728 (0.0009) [2023-12-26 16:05:19,266][105692] Updated weights for policy 0, policy_version 99322 (0.0008) [2023-12-26 16:05:19,323][105692] Updated weights for policy 0, policy_version 99332 (0.0009) [2023-12-26 16:05:19,379][105620] Updated weights for policy 1, policy_version 99738 (0.0008) [2023-12-26 16:05:19,393][105692] Updated weights for policy 0, policy_version 99342 (0.0009) [2023-12-26 16:05:19,434][105620] Updated weights for policy 1, policy_version 99748 (0.0009) [2023-12-26 16:05:19,491][105620] Updated weights for policy 1, policy_version 99758 (0.0009) [2023-12-26 16:05:20,158][105692] Updated weights for policy 0, policy_version 99352 (0.0009) [2023-12-26 16:05:20,220][105692] Updated weights for policy 0, policy_version 99362 (0.0009) [2023-12-26 16:05:20,264][105620] Updated weights for policy 1, policy_version 99768 (0.0008) [2023-12-26 16:05:20,270][105692] Updated weights for policy 0, policy_version 99372 (0.0007) [2023-12-26 16:05:20,319][105620] Updated weights for policy 1, policy_version 99778 (0.0007) [2023-12-26 16:05:20,376][105620] Updated weights for policy 1, policy_version 99788 (0.0008) [2023-12-26 16:05:20,987][105692] Updated weights for policy 0, policy_version 99382 (0.0008) [2023-12-26 16:05:21,056][105692] Updated weights for policy 0, policy_version 99392 (0.0009) [2023-12-26 16:05:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 50995200. Throughput: 0: 10005.3, 1: 9627.0. Samples: 50990600. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-26 16:05:21,062][104569] Avg episode reward: [(0, '9086.500'), (1, '9352.668')] [2023-12-26 16:05:21,122][105692] Updated weights for policy 0, policy_version 99402 (0.0009) [2023-12-26 16:05:21,175][105620] Updated weights for policy 1, policy_version 99798 (0.0008) [2023-12-26 16:05:21,238][105620] Updated weights for policy 1, policy_version 99808 (0.0008) [2023-12-26 16:05:21,298][105620] Updated weights for policy 1, policy_version 99818 (0.0008) [2023-12-26 16:05:21,878][105692] Updated weights for policy 0, policy_version 99412 (0.0008) [2023-12-26 16:05:21,939][105692] Updated weights for policy 0, policy_version 99422 (0.0006) [2023-12-26 16:05:21,995][105692] Updated weights for policy 0, policy_version 99432 (0.0006) [2023-12-26 16:05:22,109][105620] Updated weights for policy 1, policy_version 99828 (0.0007) [2023-12-26 16:05:22,177][105620] Updated weights for policy 1, policy_version 99838 (0.0005) [2023-12-26 16:05:22,245][105620] Updated weights for policy 1, policy_version 99848 (0.0006) [2023-12-26 16:05:22,665][105692] Updated weights for policy 0, policy_version 99442 (0.0006) [2023-12-26 16:05:22,721][105692] Updated weights for policy 0, policy_version 99452 (0.0008) [2023-12-26 16:05:22,785][105692] Updated weights for policy 0, policy_version 99462 (0.0009) [2023-12-26 16:05:22,849][105692] Updated weights for policy 0, policy_version 99472 (0.0009) [2023-12-26 16:05:22,957][105620] Updated weights for policy 1, policy_version 99858 (0.0008) [2023-12-26 16:05:23,008][105620] Updated weights for policy 1, policy_version 99868 (0.0009) [2023-12-26 16:05:23,067][105620] Updated weights for policy 1, policy_version 99878 (0.0009) [2023-12-26 16:05:23,130][105620] Updated weights for policy 1, policy_version 99888 (0.0009) [2023-12-26 16:05:23,513][105692] Updated weights for policy 0, policy_version 99482 (0.0010) [2023-12-26 16:05:23,562][105692] Updated weights for policy 0, policy_version 99492 (0.0010) [2023-12-26 16:05:23,613][105692] Updated weights for policy 0, policy_version 99502 (0.0010) [2023-12-26 16:05:23,900][105620] Updated weights for policy 1, policy_version 99898 (0.0005) [2023-12-26 16:05:23,952][105620] Updated weights for policy 1, policy_version 99908 (0.0005) [2023-12-26 16:05:24,000][105620] Updated weights for policy 1, policy_version 99918 (0.0005) [2023-12-26 16:05:24,364][105692] Updated weights for policy 0, policy_version 99512 (0.0009) [2023-12-26 16:05:24,421][105692] Updated weights for policy 0, policy_version 99522 (0.0009) [2023-12-26 16:05:24,478][105692] Updated weights for policy 0, policy_version 99532 (0.0008) [2023-12-26 16:05:24,695][105620] Updated weights for policy 1, policy_version 99928 (0.0009) [2023-12-26 16:05:24,742][105620] Updated weights for policy 1, policy_version 99938 (0.0009) [2023-12-26 16:05:24,788][105620] Updated weights for policy 1, policy_version 99948 (0.0009) [2023-12-26 16:05:25,190][105692] Updated weights for policy 0, policy_version 99542 (0.0008) [2023-12-26 16:05:25,243][105692] Updated weights for policy 0, policy_version 99552 (0.0008) [2023-12-26 16:05:25,304][105692] Updated weights for policy 0, policy_version 99562 (0.0009) [2023-12-26 16:05:25,557][105620] Updated weights for policy 1, policy_version 99958 (0.0010) [2023-12-26 16:05:25,609][105620] Updated weights for policy 1, policy_version 99968 (0.0009) [2023-12-26 16:05:25,660][105620] Updated weights for policy 1, policy_version 99978 (0.0009) [2023-12-26 16:05:25,911][105692] Updated weights for policy 0, policy_version 99572 (0.0006) [2023-12-26 16:05:25,966][105692] Updated weights for policy 0, policy_version 99582 (0.0005) [2023-12-26 16:05:26,017][105692] Updated weights for policy 0, policy_version 99592 (0.0005) [2023-12-26 16:05:26,062][104569] Fps is (10 sec: 20480.7, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 51101696. Throughput: 0: 9976.5, 1: 9638.6. Samples: 51104540. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-26 16:05:26,062][104569] Avg episode reward: [(0, '9086.549'), (1, '9351.785')] [2023-12-26 16:05:26,441][105620] Updated weights for policy 1, policy_version 99988 (0.0010) [2023-12-26 16:05:26,489][105620] Updated weights for policy 1, policy_version 99998 (0.0010) [2023-12-26 16:05:26,551][105620] Updated weights for policy 1, policy_version 100008 (0.0010) [2023-12-26 16:05:26,588][105692] Updated weights for policy 0, policy_version 99602 (0.0006) [2023-12-26 16:05:26,653][105692] Updated weights for policy 0, policy_version 99612 (0.0005) [2023-12-26 16:05:26,702][105692] Updated weights for policy 0, policy_version 99622 (0.0005) [2023-12-26 16:05:26,754][105692] Updated weights for policy 0, policy_version 99632 (0.0005) [2023-12-26 16:05:27,141][105620] Updated weights for policy 1, policy_version 100018 (0.0009) [2023-12-26 16:05:27,205][105620] Updated weights for policy 1, policy_version 100028 (0.0007) [2023-12-26 16:05:27,262][105620] Updated weights for policy 1, policy_version 100038 (0.0010) [2023-12-26 16:05:27,323][105620] Updated weights for policy 1, policy_version 100048 (0.0010) [2023-12-26 16:05:27,343][105692] Updated weights for policy 0, policy_version 99642 (0.0007) [2023-12-26 16:05:27,400][105692] Updated weights for policy 0, policy_version 99652 (0.0006) [2023-12-26 16:05:27,456][105692] Updated weights for policy 0, policy_version 99662 (0.0010) [2023-12-26 16:05:27,987][105620] Updated weights for policy 1, policy_version 100058 (0.0008) [2023-12-26 16:05:28,050][105620] Updated weights for policy 1, policy_version 100068 (0.0008) [2023-12-26 16:05:28,103][105620] Updated weights for policy 1, policy_version 100078 (0.0008) [2023-12-26 16:05:28,144][105692] Updated weights for policy 0, policy_version 99672 (0.0010) [2023-12-26 16:05:28,195][105692] Updated weights for policy 0, policy_version 99682 (0.0010) [2023-12-26 16:05:28,251][105692] Updated weights for policy 0, policy_version 99692 (0.0010) [2023-12-26 16:05:28,867][105620] Updated weights for policy 1, policy_version 100088 (0.0008) [2023-12-26 16:05:28,928][105620] Updated weights for policy 1, policy_version 100098 (0.0008) [2023-12-26 16:05:28,983][105620] Updated weights for policy 1, policy_version 100108 (0.0008) [2023-12-26 16:05:29,003][105692] Updated weights for policy 0, policy_version 99702 (0.0010) [2023-12-26 16:05:29,060][105692] Updated weights for policy 0, policy_version 99712 (0.0010) [2023-12-26 16:05:29,108][105692] Updated weights for policy 0, policy_version 99722 (0.0009) [2023-12-26 16:05:29,747][105620] Updated weights for policy 1, policy_version 100118 (0.0008) [2023-12-26 16:05:29,798][105620] Updated weights for policy 1, policy_version 100128 (0.0009) [2023-12-26 16:05:29,858][105620] Updated weights for policy 1, policy_version 100138 (0.0008) [2023-12-26 16:05:29,863][105692] Updated weights for policy 0, policy_version 99732 (0.0008) [2023-12-26 16:05:29,921][105692] Updated weights for policy 0, policy_version 99742 (0.0009) [2023-12-26 16:05:29,983][105692] Updated weights for policy 0, policy_version 99752 (0.0008) [2023-12-26 16:05:30,640][105692] Updated weights for policy 0, policy_version 99762 (0.0008) [2023-12-26 16:05:30,659][105620] Updated weights for policy 1, policy_version 100148 (0.0008) [2023-12-26 16:05:30,685][105692] Updated weights for policy 0, policy_version 99772 (0.0005) [2023-12-26 16:05:30,711][105620] Updated weights for policy 1, policy_version 100158 (0.0009) [2023-12-26 16:05:30,733][105692] Updated weights for policy 0, policy_version 99782 (0.0005) [2023-12-26 16:05:30,756][105586] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000008 [2023-12-26 16:05:30,759][105620] Updated weights for policy 1, policy_version 100168 (0.0008) [2023-12-26 16:05:30,781][105692] Updated weights for policy 0, policy_version 99792 (0.0005) [2023-12-26 16:05:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 51200000. Throughput: 0: 10025.4, 1: 9691.5. Samples: 51167452. Policy #0 lag: (min: 10.0, avg: 18.4, max: 42.0) [2023-12-26 16:05:31,062][104569] Avg episode reward: [(0, '9080.843'), (1, '9351.996')] [2023-12-26 16:05:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000099792_25550848.pth... [2023-12-26 16:05:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000100168_25649152.pth... [2023-12-26 16:05:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000098640_25255936.pth [2023-12-26 16:05:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000099056_25362432.pth [2023-12-26 16:05:31,438][105692] Updated weights for policy 0, policy_version 99802 (0.0009) [2023-12-26 16:05:31,494][105692] Updated weights for policy 0, policy_version 99812 (0.0010) [2023-12-26 16:05:31,544][105692] Updated weights for policy 0, policy_version 99822 (0.0008) [2023-12-26 16:05:31,598][105620] Updated weights for policy 1, policy_version 100178 (0.0009) [2023-12-26 16:05:31,658][105620] Updated weights for policy 1, policy_version 100188 (0.0009) [2023-12-26 16:05:31,720][105620] Updated weights for policy 1, policy_version 100198 (0.0008) [2023-12-26 16:05:32,326][105620] Updated weights for policy 1, policy_version 100208 (0.0008) [2023-12-26 16:05:32,393][105620] Updated weights for policy 1, policy_version 100218 (0.0008) [2023-12-26 16:05:32,436][105692] Updated weights for policy 0, policy_version 99832 (0.0007) [2023-12-26 16:05:32,453][105620] Updated weights for policy 1, policy_version 100228 (0.0008) [2023-12-26 16:05:32,495][105692] Updated weights for policy 0, policy_version 99842 (0.0007) [2023-12-26 16:05:32,547][105692] Updated weights for policy 0, policy_version 99852 (0.0009) [2023-12-26 16:05:33,228][105692] Updated weights for policy 0, policy_version 99862 (0.0008) [2023-12-26 16:05:33,244][105620] Updated weights for policy 1, policy_version 100238 (0.0007) [2023-12-26 16:05:33,285][105692] Updated weights for policy 0, policy_version 99872 (0.0008) [2023-12-26 16:05:33,302][105620] Updated weights for policy 1, policy_version 100248 (0.0005) [2023-12-26 16:05:33,343][105692] Updated weights for policy 0, policy_version 99882 (0.0009) [2023-12-26 16:05:33,358][105620] Updated weights for policy 1, policy_version 100258 (0.0005) [2023-12-26 16:05:33,861][105620] Updated weights for policy 1, policy_version 100268 (0.0005) [2023-12-26 16:05:33,917][105620] Updated weights for policy 1, policy_version 100278 (0.0005) [2023-12-26 16:05:33,973][105620] Updated weights for policy 1, policy_version 100288 (0.0005) [2023-12-26 16:05:33,979][105692] Updated weights for policy 0, policy_version 99892 (0.0009) [2023-12-26 16:05:34,029][105692] Updated weights for policy 0, policy_version 99902 (0.0009) [2023-12-26 16:05:34,084][105692] Updated weights for policy 0, policy_version 99912 (0.0008) [2023-12-26 16:05:34,645][105620] Updated weights for policy 1, policy_version 100298 (0.0005) [2023-12-26 16:05:34,709][105620] Updated weights for policy 1, policy_version 100308 (0.0009) [2023-12-26 16:05:34,769][105620] Updated weights for policy 1, policy_version 100318 (0.0007) [2023-12-26 16:05:34,831][105620] Updated weights for policy 1, policy_version 100328 (0.0009) [2023-12-26 16:05:34,942][105692] Updated weights for policy 0, policy_version 99922 (0.0009) [2023-12-26 16:05:35,009][105692] Updated weights for policy 0, policy_version 99932 (0.0010) [2023-12-26 16:05:35,068][105692] Updated weights for policy 0, policy_version 99942 (0.0009) [2023-12-26 16:05:35,124][105692] Updated weights for policy 0, policy_version 99952 (0.0009) [2023-12-26 16:05:35,494][105620] Updated weights for policy 1, policy_version 100338 (0.0007) [2023-12-26 16:05:35,547][105620] Updated weights for policy 1, policy_version 100348 (0.0009) [2023-12-26 16:05:35,593][105620] Updated weights for policy 1, policy_version 100358 (0.0010) [2023-12-26 16:05:35,930][105692] Updated weights for policy 0, policy_version 99962 (0.0009) [2023-12-26 16:05:35,984][105692] Updated weights for policy 0, policy_version 99972 (0.0009) [2023-12-26 16:05:36,039][105692] Updated weights for policy 0, policy_version 99982 (0.0009) [2023-12-26 16:05:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 51298304. Throughput: 0: 10015.7, 1: 9699.9. Samples: 51284784. Policy #0 lag: (min: 10.0, avg: 18.4, max: 42.0) [2023-12-26 16:05:36,062][104569] Avg episode reward: [(0, '8990.691'), (1, '9260.475')] [2023-12-26 16:05:36,272][105620] Updated weights for policy 1, policy_version 100368 (0.0010) [2023-12-26 16:05:36,326][105620] Updated weights for policy 1, policy_version 100378 (0.0009) [2023-12-26 16:05:36,388][105620] Updated weights for policy 1, policy_version 100388 (0.0007) [2023-12-26 16:05:36,699][105692] Updated weights for policy 0, policy_version 99992 (0.0006) [2023-12-26 16:05:36,773][105692] Updated weights for policy 0, policy_version 100002 (0.0006) [2023-12-26 16:05:36,835][105692] Updated weights for policy 0, policy_version 100012 (0.0006) [2023-12-26 16:05:37,047][105620] Updated weights for policy 1, policy_version 100398 (0.0007) [2023-12-26 16:05:37,109][105620] Updated weights for policy 1, policy_version 100408 (0.0007) [2023-12-26 16:05:37,170][105620] Updated weights for policy 1, policy_version 100418 (0.0007) [2023-12-26 16:05:37,557][105692] Updated weights for policy 0, policy_version 100022 (0.0009) [2023-12-26 16:05:37,619][105692] Updated weights for policy 0, policy_version 100032 (0.0010) [2023-12-26 16:05:37,674][105692] Updated weights for policy 0, policy_version 100042 (0.0010) [2023-12-26 16:05:37,763][105620] Updated weights for policy 1, policy_version 100428 (0.0007) [2023-12-26 16:05:37,812][105620] Updated weights for policy 1, policy_version 100438 (0.0005) [2023-12-26 16:05:37,866][105620] Updated weights for policy 1, policy_version 100448 (0.0006) [2023-12-26 16:05:38,520][105692] Updated weights for policy 0, policy_version 100053 (0.0009) [2023-12-26 16:05:38,562][105620] Updated weights for policy 1, policy_version 100458 (0.0007) [2023-12-26 16:05:38,573][105692] Updated weights for policy 0, policy_version 100063 (0.0008) [2023-12-26 16:05:38,624][105620] Updated weights for policy 1, policy_version 100468 (0.0010) [2023-12-26 16:05:38,635][105692] Updated weights for policy 0, policy_version 100073 (0.0008) [2023-12-26 16:05:38,683][105620] Updated weights for policy 1, policy_version 100478 (0.0008) [2023-12-26 16:05:38,747][105620] Updated weights for policy 1, policy_version 100488 (0.0009) [2023-12-26 16:05:39,431][105692] Updated weights for policy 0, policy_version 100083 (0.0006) [2023-12-26 16:05:39,490][105692] Updated weights for policy 0, policy_version 100093 (0.0009) [2023-12-26 16:05:39,518][105620] Updated weights for policy 1, policy_version 100498 (0.0008) [2023-12-26 16:05:39,545][105692] Updated weights for policy 0, policy_version 100103 (0.0009) [2023-12-26 16:05:39,579][105620] Updated weights for policy 1, policy_version 100508 (0.0007) [2023-12-26 16:05:39,642][105620] Updated weights for policy 1, policy_version 100518 (0.0008) [2023-12-26 16:05:40,309][105692] Updated weights for policy 0, policy_version 100113 (0.0007) [2023-12-26 16:05:40,370][105692] Updated weights for policy 0, policy_version 100123 (0.0010) [2023-12-26 16:05:40,381][105620] Updated weights for policy 1, policy_version 100528 (0.0007) [2023-12-26 16:05:40,434][105620] Updated weights for policy 1, policy_version 100538 (0.0008) [2023-12-26 16:05:40,434][105692] Updated weights for policy 0, policy_version 100133 (0.0008) [2023-12-26 16:05:40,491][105620] Updated weights for policy 1, policy_version 100548 (0.0007) [2023-12-26 16:05:40,505][105692] Updated weights for policy 0, policy_version 100143 (0.0010) [2023-12-26 16:05:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 51388416. Throughput: 0: 9830.6, 1: 9758.4. Samples: 51399156. Policy #0 lag: (min: 10.0, avg: 18.4, max: 42.0) [2023-12-26 16:05:41,063][104569] Avg episode reward: [(0, '8989.650'), (1, '9260.053')] [2023-12-26 16:05:41,152][105692] Updated weights for policy 0, policy_version 100153 (0.0010) [2023-12-26 16:05:41,213][105692] Updated weights for policy 0, policy_version 100163 (0.0009) [2023-12-26 16:05:41,274][105620] Updated weights for policy 1, policy_version 100558 (0.0007) [2023-12-26 16:05:41,277][105692] Updated weights for policy 0, policy_version 100173 (0.0008) [2023-12-26 16:05:41,338][105620] Updated weights for policy 1, policy_version 100568 (0.0008) [2023-12-26 16:05:41,402][105620] Updated weights for policy 1, policy_version 100578 (0.0008) [2023-12-26 16:05:42,007][105692] Updated weights for policy 0, policy_version 100183 (0.0007) [2023-12-26 16:05:42,059][105692] Updated weights for policy 0, policy_version 100193 (0.0006) [2023-12-26 16:05:42,126][105692] Updated weights for policy 0, policy_version 100203 (0.0009) [2023-12-26 16:05:42,177][105620] Updated weights for policy 1, policy_version 100588 (0.0006) [2023-12-26 16:05:42,245][105620] Updated weights for policy 1, policy_version 100598 (0.0009) [2023-12-26 16:05:42,315][105620] Updated weights for policy 1, policy_version 100608 (0.0009) [2023-12-26 16:05:42,851][105692] Updated weights for policy 0, policy_version 100213 (0.0007) [2023-12-26 16:05:42,901][105692] Updated weights for policy 0, policy_version 100223 (0.0005) [2023-12-26 16:05:42,952][105692] Updated weights for policy 0, policy_version 100233 (0.0006) [2023-12-26 16:05:43,016][105620] Updated weights for policy 1, policy_version 100618 (0.0010) [2023-12-26 16:05:43,070][105620] Updated weights for policy 1, policy_version 100628 (0.0010) [2023-12-26 16:05:43,122][105620] Updated weights for policy 1, policy_version 100638 (0.0010) [2023-12-26 16:05:43,181][105620] Updated weights for policy 1, policy_version 100648 (0.0010) [2023-12-26 16:05:43,699][105692] Updated weights for policy 0, policy_version 100243 (0.0007) [2023-12-26 16:05:43,749][105692] Updated weights for policy 0, policy_version 100253 (0.0010) [2023-12-26 16:05:43,780][105620] Updated weights for policy 1, policy_version 100658 (0.0006) [2023-12-26 16:05:43,809][105692] Updated weights for policy 0, policy_version 100263 (0.0008) [2023-12-26 16:05:43,832][105620] Updated weights for policy 1, policy_version 100668 (0.0005) [2023-12-26 16:05:43,890][105620] Updated weights for policy 1, policy_version 100678 (0.0010) [2023-12-26 16:05:44,541][105620] Updated weights for policy 1, policy_version 100688 (0.0010) [2023-12-26 16:05:44,605][105620] Updated weights for policy 1, policy_version 100698 (0.0010) [2023-12-26 16:05:44,625][105692] Updated weights for policy 0, policy_version 100273 (0.0007) [2023-12-26 16:05:44,669][105620] Updated weights for policy 1, policy_version 100708 (0.0010) [2023-12-26 16:05:44,672][105692] Updated weights for policy 0, policy_version 100283 (0.0006) [2023-12-26 16:05:44,735][105692] Updated weights for policy 0, policy_version 100293 (0.0007) [2023-12-26 16:05:44,794][105692] Updated weights for policy 0, policy_version 100303 (0.0009) [2023-12-26 16:05:45,441][105620] Updated weights for policy 1, policy_version 100718 (0.0010) [2023-12-26 16:05:45,497][105692] Updated weights for policy 0, policy_version 100313 (0.0006) [2023-12-26 16:05:45,501][105620] Updated weights for policy 1, policy_version 100728 (0.0011) [2023-12-26 16:05:45,549][105692] Updated weights for policy 0, policy_version 100323 (0.0005) [2023-12-26 16:05:45,553][105620] Updated weights for policy 1, policy_version 100738 (0.0010) [2023-12-26 16:05:45,610][105692] Updated weights for policy 0, policy_version 100333 (0.0005) [2023-12-26 16:05:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.4, 300 sec: 19633.0). Total num frames: 51486720. Throughput: 0: 9827.8, 1: 9700.0. Samples: 51457884. Policy #0 lag: (min: 10.0, avg: 18.4, max: 42.0) [2023-12-26 16:05:46,063][104569] Avg episode reward: [(0, '9079.301'), (1, '9350.529')] [2023-12-26 16:05:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000100336_25690112.pth... [2023-12-26 16:05:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000100744_25796608.pth... [2023-12-26 16:05:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000099216_25403392.pth [2023-12-26 16:05:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000099600_25501696.pth [2023-12-26 16:05:46,174][105692] Updated weights for policy 0, policy_version 100343 (0.0008) [2023-12-26 16:05:46,226][105692] Updated weights for policy 0, policy_version 100353 (0.0008) [2023-12-26 16:05:46,270][105692] Updated weights for policy 0, policy_version 100363 (0.0008) [2023-12-26 16:05:46,300][105620] Updated weights for policy 1, policy_version 100748 (0.0010) [2023-12-26 16:05:46,350][105620] Updated weights for policy 1, policy_version 100758 (0.0010) [2023-12-26 16:05:46,408][105620] Updated weights for policy 1, policy_version 100768 (0.0010) [2023-12-26 16:05:47,002][105692] Updated weights for policy 0, policy_version 100373 (0.0008) [2023-12-26 16:05:47,050][105692] Updated weights for policy 0, policy_version 100383 (0.0008) [2023-12-26 16:05:47,094][105692] Updated weights for policy 0, policy_version 100393 (0.0007) [2023-12-26 16:05:47,150][105620] Updated weights for policy 1, policy_version 100778 (0.0010) [2023-12-26 16:05:47,218][105620] Updated weights for policy 1, policy_version 100788 (0.0010) [2023-12-26 16:05:47,280][105620] Updated weights for policy 1, policy_version 100798 (0.0010) [2023-12-26 16:05:47,337][105620] Updated weights for policy 1, policy_version 100808 (0.0010) [2023-12-26 16:05:47,702][105692] Updated weights for policy 0, policy_version 100403 (0.0008) [2023-12-26 16:05:47,766][105692] Updated weights for policy 0, policy_version 100413 (0.0008) [2023-12-26 16:05:47,825][105692] Updated weights for policy 0, policy_version 100423 (0.0008) [2023-12-26 16:05:48,062][105620] Updated weights for policy 1, policy_version 100818 (0.0010) [2023-12-26 16:05:48,108][105620] Updated weights for policy 1, policy_version 100828 (0.0010) [2023-12-26 16:05:48,171][105620] Updated weights for policy 1, policy_version 100838 (0.0011) [2023-12-26 16:05:48,572][105692] Updated weights for policy 0, policy_version 100433 (0.0009) [2023-12-26 16:05:48,627][105692] Updated weights for policy 0, policy_version 100443 (0.0008) [2023-12-26 16:05:48,685][105692] Updated weights for policy 0, policy_version 100453 (0.0009) [2023-12-26 16:05:48,752][105692] Updated weights for policy 0, policy_version 100463 (0.0009) [2023-12-26 16:05:48,917][105620] Updated weights for policy 1, policy_version 100848 (0.0009) [2023-12-26 16:05:48,974][105620] Updated weights for policy 1, policy_version 100858 (0.0008) [2023-12-26 16:05:49,023][105620] Updated weights for policy 1, policy_version 100868 (0.0008) [2023-12-26 16:05:49,542][105692] Updated weights for policy 0, policy_version 100473 (0.0009) [2023-12-26 16:05:49,597][105692] Updated weights for policy 0, policy_version 100483 (0.0008) [2023-12-26 16:05:49,644][105692] Updated weights for policy 0, policy_version 100493 (0.0008) [2023-12-26 16:05:49,783][105620] Updated weights for policy 1, policy_version 100878 (0.0009) [2023-12-26 16:05:49,845][105620] Updated weights for policy 1, policy_version 100888 (0.0009) [2023-12-26 16:05:49,908][105620] Updated weights for policy 1, policy_version 100898 (0.0006) [2023-12-26 16:05:50,399][105692] Updated weights for policy 0, policy_version 100503 (0.0007) [2023-12-26 16:05:50,470][105692] Updated weights for policy 0, policy_version 100513 (0.0005) [2023-12-26 16:05:50,539][105692] Updated weights for policy 0, policy_version 100523 (0.0005) [2023-12-26 16:05:50,699][105620] Updated weights for policy 1, policy_version 100908 (0.0007) [2023-12-26 16:05:50,762][105620] Updated weights for policy 1, policy_version 100918 (0.0006) [2023-12-26 16:05:50,819][105620] Updated weights for policy 1, policy_version 100928 (0.0007) [2023-12-26 16:05:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 51585024. Throughput: 0: 9826.1, 1: 9618.3. Samples: 51573888. Policy #0 lag: (min: 10.0, avg: 18.4, max: 42.0) [2023-12-26 16:05:51,062][104569] Avg episode reward: [(0, '9175.216'), (1, '9349.787')] [2023-12-26 16:05:51,176][105692] Updated weights for policy 0, policy_version 100533 (0.0008) [2023-12-26 16:05:51,247][105692] Updated weights for policy 0, policy_version 100543 (0.0009) [2023-12-26 16:05:51,310][105692] Updated weights for policy 0, policy_version 100553 (0.0009) [2023-12-26 16:05:51,537][105620] Updated weights for policy 1, policy_version 100938 (0.0007) [2023-12-26 16:05:51,594][105620] Updated weights for policy 1, policy_version 100948 (0.0009) [2023-12-26 16:05:51,650][105620] Updated weights for policy 1, policy_version 100958 (0.0009) [2023-12-26 16:05:51,701][105620] Updated weights for policy 1, policy_version 100968 (0.0009) [2023-12-26 16:05:52,122][105692] Updated weights for policy 0, policy_version 100563 (0.0010) [2023-12-26 16:05:52,176][105692] Updated weights for policy 0, policy_version 100573 (0.0010) [2023-12-26 16:05:52,235][105692] Updated weights for policy 0, policy_version 100583 (0.0009) [2023-12-26 16:05:52,327][105620] Updated weights for policy 1, policy_version 100978 (0.0009) [2023-12-26 16:05:52,385][105620] Updated weights for policy 1, policy_version 100988 (0.0009) [2023-12-26 16:05:52,441][105620] Updated weights for policy 1, policy_version 100998 (0.0009) [2023-12-26 16:05:53,011][105692] Updated weights for policy 0, policy_version 100593 (0.0009) [2023-12-26 16:05:53,066][105692] Updated weights for policy 0, policy_version 100603 (0.0009) [2023-12-26 16:05:53,114][105692] Updated weights for policy 0, policy_version 100613 (0.0007) [2023-12-26 16:05:53,128][105620] Updated weights for policy 1, policy_version 101008 (0.0008) [2023-12-26 16:05:53,159][105692] Updated weights for policy 0, policy_version 100623 (0.0006) [2023-12-26 16:05:53,182][105620] Updated weights for policy 1, policy_version 101018 (0.0008) [2023-12-26 16:05:53,236][105620] Updated weights for policy 1, policy_version 101028 (0.0009) [2023-12-26 16:05:53,931][105692] Updated weights for policy 0, policy_version 100633 (0.0008) [2023-12-26 16:05:53,980][105692] Updated weights for policy 0, policy_version 100643 (0.0008) [2023-12-26 16:05:54,000][105620] Updated weights for policy 1, policy_version 101038 (0.0009) [2023-12-26 16:05:54,031][105692] Updated weights for policy 0, policy_version 100653 (0.0008) [2023-12-26 16:05:54,049][105620] Updated weights for policy 1, policy_version 101048 (0.0010) [2023-12-26 16:05:54,110][105620] Updated weights for policy 1, policy_version 101058 (0.0009) [2023-12-26 16:05:54,817][105692] Updated weights for policy 0, policy_version 100663 (0.0009) [2023-12-26 16:05:54,840][105620] Updated weights for policy 1, policy_version 101068 (0.0008) [2023-12-26 16:05:54,878][105692] Updated weights for policy 0, policy_version 100673 (0.0008) [2023-12-26 16:05:54,900][105620] Updated weights for policy 1, policy_version 101078 (0.0007) [2023-12-26 16:05:54,923][105692] Updated weights for policy 0, policy_version 100683 (0.0005) [2023-12-26 16:05:54,949][105620] Updated weights for policy 1, policy_version 101088 (0.0006) [2023-12-26 16:05:55,651][105692] Updated weights for policy 0, policy_version 100693 (0.0007) [2023-12-26 16:05:55,698][105620] Updated weights for policy 1, policy_version 101098 (0.0009) [2023-12-26 16:05:55,704][105692] Updated weights for policy 0, policy_version 100703 (0.0007) [2023-12-26 16:05:55,753][105620] Updated weights for policy 1, policy_version 101108 (0.0010) [2023-12-26 16:05:55,760][105692] Updated weights for policy 0, policy_version 100713 (0.0006) [2023-12-26 16:05:55,818][105620] Updated weights for policy 1, policy_version 101118 (0.0010) [2023-12-26 16:05:55,880][105620] Updated weights for policy 1, policy_version 101128 (0.0010) [2023-12-26 16:05:56,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.7, 300 sec: 19605.3). Total num frames: 51683328. Throughput: 0: 9704.1, 1: 9699.7. Samples: 51687360. Policy #0 lag: (min: 10.0, avg: 18.4, max: 42.0) [2023-12-26 16:05:56,063][104569] Avg episode reward: [(0, '8563.727'), (1, '9349.795')] [2023-12-26 16:05:56,428][105692] Updated weights for policy 0, policy_version 100723 (0.0007) [2023-12-26 16:05:56,484][105692] Updated weights for policy 0, policy_version 100733 (0.0010) [2023-12-26 16:05:56,552][105692] Updated weights for policy 0, policy_version 100743 (0.0010) [2023-12-26 16:05:56,607][105620] Updated weights for policy 1, policy_version 101138 (0.0009) [2023-12-26 16:05:56,662][105620] Updated weights for policy 1, policy_version 101148 (0.0008) [2023-12-26 16:05:56,718][105620] Updated weights for policy 1, policy_version 101158 (0.0009) [2023-12-26 16:05:57,278][105692] Updated weights for policy 0, policy_version 100753 (0.0010) [2023-12-26 16:05:57,336][105692] Updated weights for policy 0, policy_version 100763 (0.0010) [2023-12-26 16:05:57,384][105692] Updated weights for policy 0, policy_version 100773 (0.0010) [2023-12-26 16:05:57,428][105692] Updated weights for policy 0, policy_version 100783 (0.0010) [2023-12-26 16:05:57,469][105620] Updated weights for policy 1, policy_version 101168 (0.0009) [2023-12-26 16:05:57,513][105620] Updated weights for policy 1, policy_version 101178 (0.0008) [2023-12-26 16:05:57,556][105620] Updated weights for policy 1, policy_version 101188 (0.0008) [2023-12-26 16:05:58,196][105692] Updated weights for policy 0, policy_version 100793 (0.0010) [2023-12-26 16:05:58,256][105692] Updated weights for policy 0, policy_version 100803 (0.0010) [2023-12-26 16:05:58,321][105692] Updated weights for policy 0, policy_version 100813 (0.0010) [2023-12-26 16:05:58,344][105620] Updated weights for policy 1, policy_version 101198 (0.0007) [2023-12-26 16:05:58,411][105620] Updated weights for policy 1, policy_version 101208 (0.0008) [2023-12-26 16:05:58,475][105620] Updated weights for policy 1, policy_version 101218 (0.0009) [2023-12-26 16:05:59,195][105692] Updated weights for policy 0, policy_version 100823 (0.0008) [2023-12-26 16:05:59,260][105692] Updated weights for policy 0, policy_version 100833 (0.0009) [2023-12-26 16:05:59,318][105692] Updated weights for policy 0, policy_version 100843 (0.0007) [2023-12-26 16:05:59,368][105620] Updated weights for policy 1, policy_version 101228 (0.0008) [2023-12-26 16:05:59,416][105620] Updated weights for policy 1, policy_version 101238 (0.0005) [2023-12-26 16:05:59,471][105620] Updated weights for policy 1, policy_version 101248 (0.0007) [2023-12-26 16:06:00,065][105692] Updated weights for policy 0, policy_version 100853 (0.0008) [2023-12-26 16:06:00,121][105692] Updated weights for policy 0, policy_version 100863 (0.0008) [2023-12-26 16:06:00,170][105692] Updated weights for policy 0, policy_version 100873 (0.0008) [2023-12-26 16:06:00,266][105620] Updated weights for policy 1, policy_version 101258 (0.0009) [2023-12-26 16:06:00,326][105620] Updated weights for policy 1, policy_version 101268 (0.0009) [2023-12-26 16:06:00,384][105620] Updated weights for policy 1, policy_version 101278 (0.0009) [2023-12-26 16:06:00,442][105620] Updated weights for policy 1, policy_version 101288 (0.0009) [2023-12-26 16:06:00,984][105692] Updated weights for policy 0, policy_version 100883 (0.0008) [2023-12-26 16:06:01,037][105692] Updated weights for policy 0, policy_version 100893 (0.0008) [2023-12-26 16:06:01,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 51765248. Throughput: 0: 9721.8, 1: 9658.8. Samples: 51743436. Policy #0 lag: (min: 10.0, avg: 18.4, max: 42.0) [2023-12-26 16:06:01,063][104569] Avg episode reward: [(0, '8742.519'), (1, '9349.815')] [2023-12-26 16:06:01,079][105620] Updated weights for policy 1, policy_version 101298 (0.0011) [2023-12-26 16:06:01,097][105692] Updated weights for policy 0, policy_version 100903 (0.0005) [2023-12-26 16:06:01,138][105620] Updated weights for policy 1, policy_version 101308 (0.0009) [2023-12-26 16:06:01,153][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000100912_25837568.pth... [2023-12-26 16:06:01,157][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000099792_25550848.pth [2023-12-26 16:06:01,188][105620] Updated weights for policy 1, policy_version 101318 (0.0008) [2023-12-26 16:06:01,196][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000101320_25944064.pth... [2023-12-26 16:06:01,200][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000100168_25649152.pth [2023-12-26 16:06:01,798][105620] Updated weights for policy 1, policy_version 101328 (0.0010) [2023-12-26 16:06:01,853][105620] Updated weights for policy 1, policy_version 101338 (0.0010) [2023-12-26 16:06:01,912][105620] Updated weights for policy 1, policy_version 101348 (0.0009) [2023-12-26 16:06:01,923][105692] Updated weights for policy 0, policy_version 100913 (0.0008) [2023-12-26 16:06:01,988][105692] Updated weights for policy 0, policy_version 100923 (0.0009) [2023-12-26 16:06:02,053][105692] Updated weights for policy 0, policy_version 100933 (0.0008) [2023-12-26 16:06:02,108][105692] Updated weights for policy 0, policy_version 100943 (0.0008) [2023-12-26 16:06:02,577][105620] Updated weights for policy 1, policy_version 101358 (0.0007) [2023-12-26 16:06:02,631][105620] Updated weights for policy 1, policy_version 101368 (0.0005) [2023-12-26 16:06:02,686][105620] Updated weights for policy 1, policy_version 101378 (0.0005) [2023-12-26 16:06:02,867][105692] Updated weights for policy 0, policy_version 100953 (0.0008) [2023-12-26 16:06:02,911][105692] Updated weights for policy 0, policy_version 100963 (0.0006) [2023-12-26 16:06:02,959][105692] Updated weights for policy 0, policy_version 100973 (0.0005) [2023-12-26 16:06:03,332][105620] Updated weights for policy 1, policy_version 101388 (0.0007) [2023-12-26 16:06:03,384][105620] Updated weights for policy 1, policy_version 101398 (0.0010) [2023-12-26 16:06:03,435][105620] Updated weights for policy 1, policy_version 101408 (0.0008) [2023-12-26 16:06:03,599][105692] Updated weights for policy 0, policy_version 100983 (0.0006) [2023-12-26 16:06:03,650][105692] Updated weights for policy 0, policy_version 100993 (0.0007) [2023-12-26 16:06:03,700][105692] Updated weights for policy 0, policy_version 101003 (0.0008) [2023-12-26 16:06:04,198][105620] Updated weights for policy 1, policy_version 101418 (0.0010) [2023-12-26 16:06:04,261][105620] Updated weights for policy 1, policy_version 101428 (0.0010) [2023-12-26 16:06:04,321][105620] Updated weights for policy 1, policy_version 101438 (0.0011) [2023-12-26 16:06:04,387][105620] Updated weights for policy 1, policy_version 101448 (0.0011) [2023-12-26 16:06:04,415][105692] Updated weights for policy 0, policy_version 101013 (0.0008) [2023-12-26 16:06:04,473][105692] Updated weights for policy 0, policy_version 101023 (0.0008) [2023-12-26 16:06:04,530][105692] Updated weights for policy 0, policy_version 101033 (0.0008) [2023-12-26 16:06:05,114][105620] Updated weights for policy 1, policy_version 101458 (0.0007) [2023-12-26 16:06:05,176][105620] Updated weights for policy 1, policy_version 101468 (0.0009) [2023-12-26 16:06:05,237][105620] Updated weights for policy 1, policy_version 101478 (0.0009) [2023-12-26 16:06:05,268][105692] Updated weights for policy 0, policy_version 101043 (0.0007) [2023-12-26 16:06:05,331][105692] Updated weights for policy 0, policy_version 101053 (0.0010) [2023-12-26 16:06:05,383][105692] Updated weights for policy 0, policy_version 101063 (0.0009) [2023-12-26 16:06:05,934][105620] Updated weights for policy 1, policy_version 101488 (0.0009) [2023-12-26 16:06:05,987][105620] Updated weights for policy 1, policy_version 101498 (0.0009) [2023-12-26 16:06:06,034][105620] Updated weights for policy 1, policy_version 101508 (0.0009) [2023-12-26 16:06:06,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 51871744. Throughput: 0: 9618.7, 1: 9671.2. Samples: 51858644. Policy #0 lag: (min: 10.0, avg: 18.4, max: 42.0) [2023-12-26 16:06:06,062][104569] Avg episode reward: [(0, '8742.068'), (1, '9349.522')] [2023-12-26 16:06:06,165][105692] Updated weights for policy 0, policy_version 101073 (0.0009) [2023-12-26 16:06:06,213][105692] Updated weights for policy 0, policy_version 101083 (0.0009) [2023-12-26 16:06:06,265][105692] Updated weights for policy 0, policy_version 101093 (0.0009) [2023-12-26 16:06:06,320][105692] Updated weights for policy 0, policy_version 101103 (0.0009) [2023-12-26 16:06:06,800][105620] Updated weights for policy 1, policy_version 101518 (0.0008) [2023-12-26 16:06:06,869][105620] Updated weights for policy 1, policy_version 101528 (0.0008) [2023-12-26 16:06:06,937][105620] Updated weights for policy 1, policy_version 101538 (0.0008) [2023-12-26 16:06:07,127][105692] Updated weights for policy 0, policy_version 101113 (0.0010) [2023-12-26 16:06:07,182][105692] Updated weights for policy 0, policy_version 101123 (0.0008) [2023-12-26 16:06:07,233][105692] Updated weights for policy 0, policy_version 101133 (0.0008) [2023-12-26 16:06:07,595][105620] Updated weights for policy 1, policy_version 101548 (0.0007) [2023-12-26 16:06:07,641][105620] Updated weights for policy 1, policy_version 101558 (0.0005) [2023-12-26 16:06:07,689][105620] Updated weights for policy 1, policy_version 101568 (0.0009) [2023-12-26 16:06:07,898][105692] Updated weights for policy 0, policy_version 101144 (0.0009) [2023-12-26 16:06:07,943][105692] Updated weights for policy 0, policy_version 101154 (0.0008) [2023-12-26 16:06:07,988][105692] Updated weights for policy 0, policy_version 101164 (0.0008) [2023-12-26 16:06:08,421][105620] Updated weights for policy 1, policy_version 101578 (0.0010) [2023-12-26 16:06:08,481][105620] Updated weights for policy 1, policy_version 101588 (0.0010) [2023-12-26 16:06:08,547][105620] Updated weights for policy 1, policy_version 101598 (0.0010) [2023-12-26 16:06:08,613][105620] Updated weights for policy 1, policy_version 101608 (0.0010) [2023-12-26 16:06:08,693][105692] Updated weights for policy 0, policy_version 101174 (0.0008) [2023-12-26 16:06:08,742][105692] Updated weights for policy 0, policy_version 101184 (0.0008) [2023-12-26 16:06:08,794][105692] Updated weights for policy 0, policy_version 101194 (0.0008) [2023-12-26 16:06:09,308][105620] Updated weights for policy 1, policy_version 101618 (0.0010) [2023-12-26 16:06:09,375][105620] Updated weights for policy 1, policy_version 101628 (0.0009) [2023-12-26 16:06:09,443][105620] Updated weights for policy 1, policy_version 101638 (0.0009) [2023-12-26 16:06:09,640][105692] Updated weights for policy 0, policy_version 101204 (0.0008) [2023-12-26 16:06:09,703][105692] Updated weights for policy 0, policy_version 101214 (0.0009) [2023-12-26 16:06:09,770][105692] Updated weights for policy 0, policy_version 101224 (0.0009) [2023-12-26 16:06:10,273][105620] Updated weights for policy 1, policy_version 101648 (0.0009) [2023-12-26 16:06:10,336][105620] Updated weights for policy 1, policy_version 101658 (0.0009) [2023-12-26 16:06:10,394][105620] Updated weights for policy 1, policy_version 101668 (0.0009) [2023-12-26 16:06:10,506][105692] Updated weights for policy 0, policy_version 101234 (0.0009) [2023-12-26 16:06:10,563][105692] Updated weights for policy 0, policy_version 101244 (0.0009) [2023-12-26 16:06:10,617][105692] Updated weights for policy 0, policy_version 101254 (0.0005) [2023-12-26 16:06:10,676][105692] Updated weights for policy 0, policy_version 101264 (0.0009) [2023-12-26 16:06:11,014][105620] Updated weights for policy 1, policy_version 101678 (0.0010) [2023-12-26 16:06:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 51961856. Throughput: 0: 9571.1, 1: 9736.1. Samples: 51973364. Policy #0 lag: (min: 10.0, avg: 18.4, max: 42.0) [2023-12-26 16:06:11,062][104569] Avg episode reward: [(0, '8451.770'), (1, '9349.801')] [2023-12-26 16:06:11,087][105620] Updated weights for policy 1, policy_version 101688 (0.0011) [2023-12-26 16:06:11,158][105620] Updated weights for policy 1, policy_version 101698 (0.0010) [2023-12-26 16:06:11,482][105692] Updated weights for policy 0, policy_version 101274 (0.0009) [2023-12-26 16:06:11,551][105692] Updated weights for policy 0, policy_version 101284 (0.0009) [2023-12-26 16:06:11,617][105692] Updated weights for policy 0, policy_version 101294 (0.0008) [2023-12-26 16:06:11,876][105620] Updated weights for policy 1, policy_version 101708 (0.0010) [2023-12-26 16:06:11,946][105620] Updated weights for policy 1, policy_version 101718 (0.0010) [2023-12-26 16:06:12,004][105620] Updated weights for policy 1, policy_version 101728 (0.0008) [2023-12-26 16:06:12,360][105692] Updated weights for policy 0, policy_version 101304 (0.0008) [2023-12-26 16:06:12,434][105692] Updated weights for policy 0, policy_version 101314 (0.0008) [2023-12-26 16:06:12,492][105692] Updated weights for policy 0, policy_version 101324 (0.0008) [2023-12-26 16:06:12,701][105620] Updated weights for policy 1, policy_version 101738 (0.0009) [2023-12-26 16:06:12,756][105620] Updated weights for policy 1, policy_version 101748 (0.0008) [2023-12-26 16:06:12,819][105620] Updated weights for policy 1, policy_version 101758 (0.0008) [2023-12-26 16:06:12,884][105620] Updated weights for policy 1, policy_version 101768 (0.0008) [2023-12-26 16:06:13,166][105692] Updated weights for policy 0, policy_version 101334 (0.0009) [2023-12-26 16:06:13,223][105692] Updated weights for policy 0, policy_version 101344 (0.0010) [2023-12-26 16:06:13,274][105692] Updated weights for policy 0, policy_version 101354 (0.0010) [2023-12-26 16:06:13,459][105620] Updated weights for policy 1, policy_version 101778 (0.0010) [2023-12-26 16:06:13,510][105620] Updated weights for policy 1, policy_version 101788 (0.0011) [2023-12-26 16:06:13,557][105620] Updated weights for policy 1, policy_version 101798 (0.0009) [2023-12-26 16:06:13,990][105692] Updated weights for policy 0, policy_version 101364 (0.0008) [2023-12-26 16:06:14,039][105692] Updated weights for policy 0, policy_version 101374 (0.0005) [2023-12-26 16:06:14,085][105692] Updated weights for policy 0, policy_version 101384 (0.0005) [2023-12-26 16:06:14,202][105620] Updated weights for policy 1, policy_version 101808 (0.0005) [2023-12-26 16:06:14,263][105620] Updated weights for policy 1, policy_version 101818 (0.0005) [2023-12-26 16:06:14,309][105620] Updated weights for policy 1, policy_version 101828 (0.0005) [2023-12-26 16:06:14,661][105692] Updated weights for policy 0, policy_version 101394 (0.0007) [2023-12-26 16:06:14,731][105692] Updated weights for policy 0, policy_version 101404 (0.0005) [2023-12-26 16:06:14,800][105692] Updated weights for policy 0, policy_version 101414 (0.0010) [2023-12-26 16:06:14,849][105692] Updated weights for policy 0, policy_version 101424 (0.0009) [2023-12-26 16:06:14,968][105620] Updated weights for policy 1, policy_version 101838 (0.0007) [2023-12-26 16:06:15,028][105620] Updated weights for policy 1, policy_version 101848 (0.0010) [2023-12-26 16:06:15,084][105620] Updated weights for policy 1, policy_version 101858 (0.0010) [2023-12-26 16:06:15,442][105692] Updated weights for policy 0, policy_version 101434 (0.0006) [2023-12-26 16:06:15,503][105692] Updated weights for policy 0, policy_version 101444 (0.0011) [2023-12-26 16:06:15,555][105692] Updated weights for policy 0, policy_version 101454 (0.0011) [2023-12-26 16:06:15,838][105620] Updated weights for policy 1, policy_version 101868 (0.0009) [2023-12-26 16:06:15,890][105620] Updated weights for policy 1, policy_version 101878 (0.0008) [2023-12-26 16:06:15,942][105620] Updated weights for policy 1, policy_version 101888 (0.0008) [2023-12-26 16:06:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 52068352. Throughput: 0: 9461.4, 1: 9729.7. Samples: 52031052. Policy #0 lag: (min: 10.0, avg: 18.4, max: 42.0) [2023-12-26 16:06:16,063][104569] Avg episode reward: [(0, '8633.976'), (1, '9260.312')] [2023-12-26 16:06:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000101456_25976832.pth... [2023-12-26 16:06:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000101896_26091520.pth... [2023-12-26 16:06:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000100336_25690112.pth [2023-12-26 16:06:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000100744_25796608.pth [2023-12-26 16:06:16,272][105692] Updated weights for policy 0, policy_version 101464 (0.0010) [2023-12-26 16:06:16,324][105692] Updated weights for policy 0, policy_version 101474 (0.0010) [2023-12-26 16:06:16,381][105692] Updated weights for policy 0, policy_version 101484 (0.0010) [2023-12-26 16:06:16,631][105620] Updated weights for policy 1, policy_version 101898 (0.0007) [2023-12-26 16:06:16,685][105620] Updated weights for policy 1, policy_version 101908 (0.0008) [2023-12-26 16:06:16,738][105620] Updated weights for policy 1, policy_version 101918 (0.0009) [2023-12-26 16:06:16,970][105692] Updated weights for policy 0, policy_version 101494 (0.0010) [2023-12-26 16:06:17,028][105692] Updated weights for policy 0, policy_version 101504 (0.0010) [2023-12-26 16:06:17,086][105692] Updated weights for policy 0, policy_version 101514 (0.0006) [2023-12-26 16:06:17,491][105620] Updated weights for policy 1, policy_version 101929 (0.0010) [2023-12-26 16:06:17,540][105620] Updated weights for policy 1, policy_version 101939 (0.0010) [2023-12-26 16:06:17,589][105620] Updated weights for policy 1, policy_version 101949 (0.0010) [2023-12-26 16:06:17,630][105692] Updated weights for policy 0, policy_version 101524 (0.0005) [2023-12-26 16:06:17,638][105620] Updated weights for policy 1, policy_version 101959 (0.0010) [2023-12-26 16:06:17,689][105692] Updated weights for policy 0, policy_version 101534 (0.0005) [2023-12-26 16:06:17,746][105692] Updated weights for policy 0, policy_version 101544 (0.0005) [2023-12-26 16:06:18,397][105620] Updated weights for policy 1, policy_version 101969 (0.0009) [2023-12-26 16:06:18,422][105692] Updated weights for policy 0, policy_version 101554 (0.0006) [2023-12-26 16:06:18,461][105620] Updated weights for policy 1, policy_version 101979 (0.0010) [2023-12-26 16:06:18,483][105692] Updated weights for policy 0, policy_version 101564 (0.0007) [2023-12-26 16:06:18,523][105620] Updated weights for policy 1, policy_version 101989 (0.0008) [2023-12-26 16:06:18,546][105692] Updated weights for policy 0, policy_version 101574 (0.0008) [2023-12-26 16:06:18,599][105692] Updated weights for policy 0, policy_version 101584 (0.0008) [2023-12-26 16:06:19,186][105620] Updated weights for policy 1, policy_version 101999 (0.0009) [2023-12-26 16:06:19,248][105620] Updated weights for policy 1, policy_version 102009 (0.0009) [2023-12-26 16:06:19,268][105692] Updated weights for policy 0, policy_version 101594 (0.0008) [2023-12-26 16:06:19,300][105620] Updated weights for policy 1, policy_version 102019 (0.0010) [2023-12-26 16:06:19,327][105692] Updated weights for policy 0, policy_version 101604 (0.0007) [2023-12-26 16:06:19,386][105692] Updated weights for policy 0, policy_version 101614 (0.0008) [2023-12-26 16:06:20,076][105620] Updated weights for policy 1, policy_version 102029 (0.0010) [2023-12-26 16:06:20,139][105620] Updated weights for policy 1, policy_version 102039 (0.0008) [2023-12-26 16:06:20,154][105692] Updated weights for policy 0, policy_version 101624 (0.0007) [2023-12-26 16:06:20,202][105620] Updated weights for policy 1, policy_version 102049 (0.0008) [2023-12-26 16:06:20,215][105692] Updated weights for policy 0, policy_version 101634 (0.0009) [2023-12-26 16:06:20,265][105692] Updated weights for policy 0, policy_version 101645 (0.0010) [2023-12-26 16:06:20,964][105620] Updated weights for policy 1, policy_version 102059 (0.0007) [2023-12-26 16:06:21,034][105620] Updated weights for policy 1, policy_version 102069 (0.0009) [2023-12-26 16:06:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 52158464. Throughput: 0: 9614.1, 1: 9735.4. Samples: 52155512. Policy #0 lag: (min: 10.0, avg: 18.4, max: 42.0) [2023-12-26 16:06:21,062][104569] Avg episode reward: [(0, '8918.590'), (1, '9258.481')] [2023-12-26 16:06:21,095][105620] Updated weights for policy 1, policy_version 102079 (0.0007) [2023-12-26 16:06:21,097][105692] Updated weights for policy 0, policy_version 101655 (0.0008) [2023-12-26 16:06:21,159][105692] Updated weights for policy 0, policy_version 101665 (0.0008) [2023-12-26 16:06:21,207][105692] Updated weights for policy 0, policy_version 101675 (0.0009) [2023-12-26 16:06:21,812][105620] Updated weights for policy 1, policy_version 102089 (0.0007) [2023-12-26 16:06:21,883][105620] Updated weights for policy 1, policy_version 102099 (0.0009) [2023-12-26 16:06:21,942][105620] Updated weights for policy 1, policy_version 102109 (0.0010) [2023-12-26 16:06:22,003][105620] Updated weights for policy 1, policy_version 102119 (0.0009) [2023-12-26 16:06:22,017][105692] Updated weights for policy 0, policy_version 101685 (0.0008) [2023-12-26 16:06:22,079][105692] Updated weights for policy 0, policy_version 101695 (0.0009) [2023-12-26 16:06:22,142][105692] Updated weights for policy 0, policy_version 101705 (0.0009) [2023-12-26 16:06:22,809][105692] Updated weights for policy 0, policy_version 101715 (0.0008) [2023-12-26 16:06:22,832][105620] Updated weights for policy 1, policy_version 102129 (0.0007) [2023-12-26 16:06:22,872][105692] Updated weights for policy 0, policy_version 101725 (0.0009) [2023-12-26 16:06:22,894][105620] Updated weights for policy 1, policy_version 102139 (0.0007) [2023-12-26 16:06:22,936][105692] Updated weights for policy 0, policy_version 101735 (0.0006) [2023-12-26 16:06:22,953][105620] Updated weights for policy 1, policy_version 102149 (0.0007) [2023-12-26 16:06:23,599][105620] Updated weights for policy 1, policy_version 102159 (0.0007) [2023-12-26 16:06:23,668][105620] Updated weights for policy 1, policy_version 102169 (0.0006) [2023-12-26 16:06:23,726][105692] Updated weights for policy 0, policy_version 101745 (0.0007) [2023-12-26 16:06:23,740][105620] Updated weights for policy 1, policy_version 102179 (0.0006) [2023-12-26 16:06:23,775][105692] Updated weights for policy 0, policy_version 101755 (0.0009) [2023-12-26 16:06:23,836][105692] Updated weights for policy 0, policy_version 101765 (0.0010) [2023-12-26 16:06:23,897][105692] Updated weights for policy 0, policy_version 101775 (0.0010) [2023-12-26 16:06:24,328][105620] Updated weights for policy 1, policy_version 102189 (0.0007) [2023-12-26 16:06:24,384][105620] Updated weights for policy 1, policy_version 102199 (0.0008) [2023-12-26 16:06:24,443][105620] Updated weights for policy 1, policy_version 102209 (0.0005) [2023-12-26 16:06:24,703][105692] Updated weights for policy 0, policy_version 101785 (0.0006) [2023-12-26 16:06:24,762][105692] Updated weights for policy 0, policy_version 101795 (0.0006) [2023-12-26 16:06:24,821][105692] Updated weights for policy 0, policy_version 101805 (0.0005) [2023-12-26 16:06:24,989][105620] Updated weights for policy 1, policy_version 102219 (0.0005) [2023-12-26 16:06:25,037][105620] Updated weights for policy 1, policy_version 102229 (0.0005) [2023-12-26 16:06:25,088][105620] Updated weights for policy 1, policy_version 102239 (0.0005) [2023-12-26 16:06:25,465][105692] Updated weights for policy 0, policy_version 101815 (0.0009) [2023-12-26 16:06:25,516][105692] Updated weights for policy 0, policy_version 101826 (0.0010) [2023-12-26 16:06:25,560][105692] Updated weights for policy 0, policy_version 101836 (0.0006) [2023-12-26 16:06:25,614][105620] Updated weights for policy 1, policy_version 102249 (0.0005) [2023-12-26 16:06:25,666][105620] Updated weights for policy 1, policy_version 102259 (0.0005) [2023-12-26 16:06:25,712][105620] Updated weights for policy 1, policy_version 102269 (0.0005) [2023-12-26 16:06:25,759][105620] Updated weights for policy 1, policy_version 102279 (0.0005) [2023-12-26 16:06:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 52264960. Throughput: 0: 9632.2, 1: 9792.3. Samples: 52273256. Policy #0 lag: (min: 10.0, avg: 18.4, max: 42.0) [2023-12-26 16:06:26,062][104569] Avg episode reward: [(0, '9097.610'), (1, '9258.119')] [2023-12-26 16:06:26,301][105692] Updated weights for policy 0, policy_version 101846 (0.0006) [2023-12-26 16:06:26,311][105620] Updated weights for policy 1, policy_version 102289 (0.0007) [2023-12-26 16:06:26,353][105692] Updated weights for policy 0, policy_version 101856 (0.0007) [2023-12-26 16:06:26,370][105620] Updated weights for policy 1, policy_version 102299 (0.0006) [2023-12-26 16:06:26,404][105692] Updated weights for policy 0, policy_version 101866 (0.0006) [2023-12-26 16:06:26,431][105620] Updated weights for policy 1, policy_version 102309 (0.0005) [2023-12-26 16:06:27,096][105692] Updated weights for policy 0, policy_version 101876 (0.0006) [2023-12-26 16:06:27,131][105620] Updated weights for policy 1, policy_version 102319 (0.0006) [2023-12-26 16:06:27,151][105692] Updated weights for policy 0, policy_version 101886 (0.0005) [2023-12-26 16:06:27,179][105620] Updated weights for policy 1, policy_version 102329 (0.0009) [2023-12-26 16:06:27,215][105692] Updated weights for policy 0, policy_version 101896 (0.0005) [2023-12-26 16:06:27,227][105620] Updated weights for policy 1, policy_version 102340 (0.0009) [2023-12-26 16:06:27,798][105692] Updated weights for policy 0, policy_version 101906 (0.0005) [2023-12-26 16:06:27,846][105692] Updated weights for policy 0, policy_version 101916 (0.0005) [2023-12-26 16:06:27,898][105692] Updated weights for policy 0, policy_version 101926 (0.0005) [2023-12-26 16:06:27,955][105692] Updated weights for policy 0, policy_version 101936 (0.0005) [2023-12-26 16:06:28,069][105620] Updated weights for policy 1, policy_version 102350 (0.0009) [2023-12-26 16:06:28,116][105620] Updated weights for policy 1, policy_version 102360 (0.0009) [2023-12-26 16:06:28,162][105620] Updated weights for policy 1, policy_version 102370 (0.0008) [2023-12-26 16:06:28,590][105692] Updated weights for policy 0, policy_version 101946 (0.0009) [2023-12-26 16:06:28,648][105692] Updated weights for policy 0, policy_version 101956 (0.0009) [2023-12-26 16:06:28,706][105692] Updated weights for policy 0, policy_version 101966 (0.0009) [2023-12-26 16:06:28,933][105620] Updated weights for policy 1, policy_version 102380 (0.0008) [2023-12-26 16:06:28,986][105620] Updated weights for policy 1, policy_version 102390 (0.0009) [2023-12-26 16:06:29,035][105620] Updated weights for policy 1, policy_version 102400 (0.0008) [2023-12-26 16:06:29,470][105692] Updated weights for policy 0, policy_version 101976 (0.0008) [2023-12-26 16:06:29,524][105692] Updated weights for policy 0, policy_version 101986 (0.0008) [2023-12-26 16:06:29,575][105692] Updated weights for policy 0, policy_version 101996 (0.0009) [2023-12-26 16:06:29,814][105620] Updated weights for policy 1, policy_version 102410 (0.0009) [2023-12-26 16:06:29,883][105620] Updated weights for policy 1, policy_version 102420 (0.0007) [2023-12-26 16:06:29,954][105620] Updated weights for policy 1, policy_version 102430 (0.0007) [2023-12-26 16:06:30,018][105620] Updated weights for policy 1, policy_version 102440 (0.0006) [2023-12-26 16:06:30,313][105692] Updated weights for policy 0, policy_version 102006 (0.0007) [2023-12-26 16:06:30,369][105692] Updated weights for policy 0, policy_version 102016 (0.0005) [2023-12-26 16:06:30,432][105692] Updated weights for policy 0, policy_version 102026 (0.0005) [2023-12-26 16:06:30,774][105620] Updated weights for policy 1, policy_version 102450 (0.0009) [2023-12-26 16:06:30,831][105620] Updated weights for policy 1, policy_version 102462 (0.0010) [2023-12-26 16:06:30,883][105620] Updated weights for policy 1, policy_version 102472 (0.0009) [2023-12-26 16:06:30,925][105692] Updated weights for policy 0, policy_version 102036 (0.0005) [2023-12-26 16:06:30,980][105692] Updated weights for policy 0, policy_version 102046 (0.0010) [2023-12-26 16:06:31,032][105692] Updated weights for policy 0, policy_version 102057 (0.0010) [2023-12-26 16:06:31,063][104569] Fps is (10 sec: 20478.2, 60 sec: 19387.4, 300 sec: 19605.2). Total num frames: 52363264. Throughput: 0: 9692.8, 1: 9781.3. Samples: 52334236. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:06:31,063][104569] Avg episode reward: [(0, '9355.119'), (1, '9258.475')] [2023-12-26 16:06:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000102472_26238976.pth... [2023-12-26 16:06:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000101320_25944064.pth [2023-12-26 16:06:31,076][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000102064_26132480.pth... [2023-12-26 16:06:31,080][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000100912_25837568.pth [2023-12-26 16:06:31,698][105620] Updated weights for policy 1, policy_version 102482 (0.0008) [2023-12-26 16:06:31,757][105692] Updated weights for policy 0, policy_version 102067 (0.0009) [2023-12-26 16:06:31,759][105620] Updated weights for policy 1, policy_version 102492 (0.0007) [2023-12-26 16:06:31,822][105620] Updated weights for policy 1, policy_version 102502 (0.0005) [2023-12-26 16:06:31,824][105692] Updated weights for policy 0, policy_version 102077 (0.0009) [2023-12-26 16:06:31,886][105692] Updated weights for policy 0, policy_version 102087 (0.0009) [2023-12-26 16:06:32,531][105692] Updated weights for policy 0, policy_version 102097 (0.0006) [2023-12-26 16:06:32,547][105620] Updated weights for policy 1, policy_version 102512 (0.0010) [2023-12-26 16:06:32,584][105692] Updated weights for policy 0, policy_version 102107 (0.0007) [2023-12-26 16:06:32,606][105620] Updated weights for policy 1, policy_version 102522 (0.0007) [2023-12-26 16:06:32,632][105692] Updated weights for policy 0, policy_version 102117 (0.0008) [2023-12-26 16:06:32,666][105620] Updated weights for policy 1, policy_version 102532 (0.0007) [2023-12-26 16:06:32,681][105692] Updated weights for policy 0, policy_version 102127 (0.0007) [2023-12-26 16:06:33,365][105620] Updated weights for policy 1, policy_version 102542 (0.0008) [2023-12-26 16:06:33,433][105620] Updated weights for policy 1, policy_version 102552 (0.0005) [2023-12-26 16:06:33,499][105620] Updated weights for policy 1, policy_version 102562 (0.0008) [2023-12-26 16:06:33,517][105692] Updated weights for policy 0, policy_version 102137 (0.0007) [2023-12-26 16:06:33,565][105692] Updated weights for policy 0, policy_version 102147 (0.0008) [2023-12-26 16:06:33,615][105692] Updated weights for policy 0, policy_version 102157 (0.0008) [2023-12-26 16:06:34,150][105620] Updated weights for policy 1, policy_version 102572 (0.0008) [2023-12-26 16:06:34,214][105620] Updated weights for policy 1, policy_version 102582 (0.0009) [2023-12-26 16:06:34,262][105620] Updated weights for policy 1, policy_version 102592 (0.0007) [2023-12-26 16:06:34,414][105692] Updated weights for policy 0, policy_version 102167 (0.0010) [2023-12-26 16:06:34,477][105692] Updated weights for policy 0, policy_version 102177 (0.0011) [2023-12-26 16:06:34,536][105692] Updated weights for policy 0, policy_version 102187 (0.0010) [2023-12-26 16:06:34,943][105620] Updated weights for policy 1, policy_version 102602 (0.0006) [2023-12-26 16:06:35,012][105620] Updated weights for policy 1, policy_version 102612 (0.0011) [2023-12-26 16:06:35,075][105620] Updated weights for policy 1, policy_version 102622 (0.0010) [2023-12-26 16:06:35,141][105620] Updated weights for policy 1, policy_version 102632 (0.0010) [2023-12-26 16:06:35,148][105692] Updated weights for policy 0, policy_version 102197 (0.0008) [2023-12-26 16:06:35,203][105692] Updated weights for policy 0, policy_version 102207 (0.0006) [2023-12-26 16:06:35,258][105692] Updated weights for policy 0, policy_version 102217 (0.0006) [2023-12-26 16:06:35,783][105692] Updated weights for policy 0, policy_version 102227 (0.0006) [2023-12-26 16:06:35,816][105620] Updated weights for policy 1, policy_version 102642 (0.0005) [2023-12-26 16:06:35,844][105692] Updated weights for policy 0, policy_version 102237 (0.0005) [2023-12-26 16:06:35,877][105620] Updated weights for policy 1, policy_version 102652 (0.0005) [2023-12-26 16:06:35,911][105692] Updated weights for policy 0, policy_version 102247 (0.0006) [2023-12-26 16:06:35,942][105620] Updated weights for policy 1, policy_version 102662 (0.0007) [2023-12-26 16:06:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 52469760. Throughput: 0: 9682.5, 1: 9787.8. Samples: 52450052. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:06:36,062][104569] Avg episode reward: [(0, '9263.291'), (1, '9172.256')] [2023-12-26 16:06:36,616][105620] Updated weights for policy 1, policy_version 102672 (0.0011) [2023-12-26 16:06:36,619][105692] Updated weights for policy 0, policy_version 102257 (0.0008) [2023-12-26 16:06:36,663][105620] Updated weights for policy 1, policy_version 102682 (0.0011) [2023-12-26 16:06:36,676][105692] Updated weights for policy 0, policy_version 102267 (0.0011) [2023-12-26 16:06:36,709][105620] Updated weights for policy 1, policy_version 102692 (0.0011) [2023-12-26 16:06:36,732][105692] Updated weights for policy 0, policy_version 102277 (0.0011) [2023-12-26 16:06:36,784][105692] Updated weights for policy 0, policy_version 102287 (0.0010) [2023-12-26 16:06:37,362][105620] Updated weights for policy 1, policy_version 102702 (0.0007) [2023-12-26 16:06:37,413][105620] Updated weights for policy 1, policy_version 102712 (0.0005) [2023-12-26 16:06:37,429][105692] Updated weights for policy 0, policy_version 102297 (0.0011) [2023-12-26 16:06:37,484][105620] Updated weights for policy 1, policy_version 102722 (0.0005) [2023-12-26 16:06:37,493][105692] Updated weights for policy 0, policy_version 102307 (0.0011) [2023-12-26 16:06:37,556][105692] Updated weights for policy 0, policy_version 102317 (0.0010) [2023-12-26 16:06:38,162][105620] Updated weights for policy 1, policy_version 102732 (0.0008) [2023-12-26 16:06:38,214][105620] Updated weights for policy 1, policy_version 102742 (0.0010) [2023-12-26 16:06:38,259][105620] Updated weights for policy 1, policy_version 102752 (0.0010) [2023-12-26 16:06:38,278][105692] Updated weights for policy 0, policy_version 102327 (0.0010) [2023-12-26 16:06:38,330][105692] Updated weights for policy 0, policy_version 102337 (0.0010) [2023-12-26 16:06:38,399][105692] Updated weights for policy 0, policy_version 102347 (0.0009) [2023-12-26 16:06:38,989][105620] Updated weights for policy 1, policy_version 102762 (0.0010) [2023-12-26 16:06:39,050][105620] Updated weights for policy 1, policy_version 102772 (0.0008) [2023-12-26 16:06:39,094][105692] Updated weights for policy 0, policy_version 102357 (0.0011) [2023-12-26 16:06:39,102][105620] Updated weights for policy 1, policy_version 102782 (0.0010) [2023-12-26 16:06:39,146][105692] Updated weights for policy 0, policy_version 102367 (0.0010) [2023-12-26 16:06:39,154][105620] Updated weights for policy 1, policy_version 102792 (0.0010) [2023-12-26 16:06:39,198][105692] Updated weights for policy 0, policy_version 102377 (0.0010) [2023-12-26 16:06:39,865][105620] Updated weights for policy 1, policy_version 102802 (0.0009) [2023-12-26 16:06:39,885][105692] Updated weights for policy 0, policy_version 102387 (0.0008) [2023-12-26 16:06:39,934][105620] Updated weights for policy 1, policy_version 102812 (0.0011) [2023-12-26 16:06:39,951][105692] Updated weights for policy 0, policy_version 102397 (0.0008) [2023-12-26 16:06:39,999][105620] Updated weights for policy 1, policy_version 102822 (0.0009) [2023-12-26 16:06:40,013][105692] Updated weights for policy 0, policy_version 102407 (0.0008) [2023-12-26 16:06:40,752][105620] Updated weights for policy 1, policy_version 102832 (0.0010) [2023-12-26 16:06:40,764][105692] Updated weights for policy 0, policy_version 102417 (0.0011) [2023-12-26 16:06:40,808][105620] Updated weights for policy 1, policy_version 102842 (0.0010) [2023-12-26 16:06:40,822][105692] Updated weights for policy 0, policy_version 102427 (0.0010) [2023-12-26 16:06:40,857][105620] Updated weights for policy 1, policy_version 102852 (0.0010) [2023-12-26 16:06:40,874][105692] Updated weights for policy 0, policy_version 102437 (0.0010) [2023-12-26 16:06:40,925][105692] Updated weights for policy 0, policy_version 102447 (0.0010) [2023-12-26 16:06:41,062][104569] Fps is (10 sec: 20481.8, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 52568064. Throughput: 0: 9813.4, 1: 9821.2. Samples: 52570912. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:06:41,062][104569] Avg episode reward: [(0, '9262.643'), (1, '9021.546')] [2023-12-26 16:06:41,604][105620] Updated weights for policy 1, policy_version 102862 (0.0011) [2023-12-26 16:06:41,668][105692] Updated weights for policy 0, policy_version 102457 (0.0009) [2023-12-26 16:06:41,673][105620] Updated weights for policy 1, policy_version 102872 (0.0011) [2023-12-26 16:06:41,736][105692] Updated weights for policy 0, policy_version 102467 (0.0009) [2023-12-26 16:06:41,737][105620] Updated weights for policy 1, policy_version 102882 (0.0010) [2023-12-26 16:06:41,803][105692] Updated weights for policy 0, policy_version 102477 (0.0008) [2023-12-26 16:06:42,524][105620] Updated weights for policy 1, policy_version 102892 (0.0009) [2023-12-26 16:06:42,561][105692] Updated weights for policy 0, policy_version 102487 (0.0008) [2023-12-26 16:06:42,588][105620] Updated weights for policy 1, policy_version 102902 (0.0006) [2023-12-26 16:06:42,621][105692] Updated weights for policy 0, policy_version 102497 (0.0008) [2023-12-26 16:06:42,649][105620] Updated weights for policy 1, policy_version 102912 (0.0006) [2023-12-26 16:06:42,680][105692] Updated weights for policy 0, policy_version 102507 (0.0008) [2023-12-26 16:06:43,246][105620] Updated weights for policy 1, policy_version 102922 (0.0008) [2023-12-26 16:06:43,304][105620] Updated weights for policy 1, policy_version 102932 (0.0007) [2023-12-26 16:06:43,351][105620] Updated weights for policy 1, policy_version 102942 (0.0007) [2023-12-26 16:06:43,399][105620] Updated weights for policy 1, policy_version 102952 (0.0008) [2023-12-26 16:06:43,478][105692] Updated weights for policy 0, policy_version 102517 (0.0010) [2023-12-26 16:06:43,532][105692] Updated weights for policy 0, policy_version 102527 (0.0010) [2023-12-26 16:06:43,581][105692] Updated weights for policy 0, policy_version 102537 (0.0010) [2023-12-26 16:06:44,220][105620] Updated weights for policy 1, policy_version 102962 (0.0007) [2023-12-26 16:06:44,222][105692] Updated weights for policy 0, policy_version 102547 (0.0010) [2023-12-26 16:06:44,274][105620] Updated weights for policy 1, policy_version 102972 (0.0005) [2023-12-26 16:06:44,280][105692] Updated weights for policy 0, policy_version 102557 (0.0010) [2023-12-26 16:06:44,337][105692] Updated weights for policy 0, policy_version 102567 (0.0010) [2023-12-26 16:06:44,339][105620] Updated weights for policy 1, policy_version 102982 (0.0006) [2023-12-26 16:06:44,963][105692] Updated weights for policy 0, policy_version 102577 (0.0010) [2023-12-26 16:06:45,002][105620] Updated weights for policy 1, policy_version 102992 (0.0008) [2023-12-26 16:06:45,030][105692] Updated weights for policy 0, policy_version 102587 (0.0009) [2023-12-26 16:06:45,064][105620] Updated weights for policy 1, policy_version 103002 (0.0008) [2023-12-26 16:06:45,097][105692] Updated weights for policy 0, policy_version 102597 (0.0011) [2023-12-26 16:06:45,120][105620] Updated weights for policy 1, policy_version 103012 (0.0006) [2023-12-26 16:06:45,164][105692] Updated weights for policy 0, policy_version 102607 (0.0011) [2023-12-26 16:06:45,828][105620] Updated weights for policy 1, policy_version 103022 (0.0007) [2023-12-26 16:06:45,847][105692] Updated weights for policy 0, policy_version 102617 (0.0009) [2023-12-26 16:06:45,888][105620] Updated weights for policy 1, policy_version 103032 (0.0005) [2023-12-26 16:06:45,904][105692] Updated weights for policy 0, policy_version 102627 (0.0008) [2023-12-26 16:06:45,952][105692] Updated weights for policy 0, policy_version 102637 (0.0007) [2023-12-26 16:06:45,953][105620] Updated weights for policy 1, policy_version 103042 (0.0007) [2023-12-26 16:06:46,062][104569] Fps is (10 sec: 19660.0, 60 sec: 19660.7, 300 sec: 19660.8). Total num frames: 52666368. Throughput: 0: 9783.0, 1: 9853.8. Samples: 52627096. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:06:46,063][104569] Avg episode reward: [(0, '9353.701'), (1, '8848.293')] [2023-12-26 16:06:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000102640_26279936.pth... [2023-12-26 16:06:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000103048_26386432.pth... [2023-12-26 16:06:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000101896_26091520.pth [2023-12-26 16:06:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000101456_25976832.pth [2023-12-26 16:06:46,587][105620] Updated weights for policy 1, policy_version 103052 (0.0009) [2023-12-26 16:06:46,646][105620] Updated weights for policy 1, policy_version 103062 (0.0009) [2023-12-26 16:06:46,673][105692] Updated weights for policy 0, policy_version 102647 (0.0007) [2023-12-26 16:06:46,702][105620] Updated weights for policy 1, policy_version 103072 (0.0008) [2023-12-26 16:06:46,725][105692] Updated weights for policy 0, policy_version 102657 (0.0007) [2023-12-26 16:06:46,777][105692] Updated weights for policy 0, policy_version 102667 (0.0008) [2023-12-26 16:06:47,397][105620] Updated weights for policy 1, policy_version 103082 (0.0008) [2023-12-26 16:06:47,441][105692] Updated weights for policy 0, policy_version 102677 (0.0007) [2023-12-26 16:06:47,450][105620] Updated weights for policy 1, policy_version 103092 (0.0007) [2023-12-26 16:06:47,504][105692] Updated weights for policy 0, policy_version 102687 (0.0005) [2023-12-26 16:06:47,509][105620] Updated weights for policy 1, policy_version 103102 (0.0009) [2023-12-26 16:06:47,569][105692] Updated weights for policy 0, policy_version 102697 (0.0008) [2023-12-26 16:06:47,571][105620] Updated weights for policy 1, policy_version 103112 (0.0010) [2023-12-26 16:06:48,237][105620] Updated weights for policy 1, policy_version 103122 (0.0009) [2023-12-26 16:06:48,288][105620] Updated weights for policy 1, policy_version 103132 (0.0009) [2023-12-26 16:06:48,298][105692] Updated weights for policy 0, policy_version 102707 (0.0008) [2023-12-26 16:06:48,343][105620] Updated weights for policy 1, policy_version 103142 (0.0007) [2023-12-26 16:06:48,365][105692] Updated weights for policy 0, policy_version 102717 (0.0008) [2023-12-26 16:06:48,425][105692] Updated weights for policy 0, policy_version 102727 (0.0006) [2023-12-26 16:06:49,066][105620] Updated weights for policy 1, policy_version 103152 (0.0006) [2023-12-26 16:06:49,121][105620] Updated weights for policy 1, policy_version 103162 (0.0006) [2023-12-26 16:06:49,168][105620] Updated weights for policy 1, policy_version 103172 (0.0007) [2023-12-26 16:06:49,206][105692] Updated weights for policy 0, policy_version 102737 (0.0009) [2023-12-26 16:06:49,269][105692] Updated weights for policy 0, policy_version 102747 (0.0008) [2023-12-26 16:06:49,325][105692] Updated weights for policy 0, policy_version 102757 (0.0008) [2023-12-26 16:06:49,392][105692] Updated weights for policy 0, policy_version 102767 (0.0008) [2023-12-26 16:06:49,906][105620] Updated weights for policy 1, policy_version 103182 (0.0011) [2023-12-26 16:06:49,968][105620] Updated weights for policy 1, policy_version 103192 (0.0011) [2023-12-26 16:06:50,032][105620] Updated weights for policy 1, policy_version 103202 (0.0011) [2023-12-26 16:06:50,147][105692] Updated weights for policy 0, policy_version 102777 (0.0007) [2023-12-26 16:06:50,219][105692] Updated weights for policy 0, policy_version 102787 (0.0008) [2023-12-26 16:06:50,279][105692] Updated weights for policy 0, policy_version 102797 (0.0007) [2023-12-26 16:06:50,788][105620] Updated weights for policy 1, policy_version 103212 (0.0010) [2023-12-26 16:06:50,839][105692] Updated weights for policy 0, policy_version 102807 (0.0008) [2023-12-26 16:06:50,853][105620] Updated weights for policy 1, policy_version 103222 (0.0007) [2023-12-26 16:06:50,900][105692] Updated weights for policy 0, policy_version 102817 (0.0008) [2023-12-26 16:06:50,910][105620] Updated weights for policy 1, policy_version 103232 (0.0005) [2023-12-26 16:06:50,960][105692] Updated weights for policy 0, policy_version 102827 (0.0009) [2023-12-26 16:06:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 52764672. Throughput: 0: 9875.2, 1: 9865.7. Samples: 52746984. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:06:51,063][104569] Avg episode reward: [(0, '9353.383'), (1, '8335.942')] [2023-12-26 16:06:51,596][105620] Updated weights for policy 1, policy_version 103242 (0.0006) [2023-12-26 16:06:51,658][105620] Updated weights for policy 1, policy_version 103252 (0.0008) [2023-12-26 16:06:51,709][105692] Updated weights for policy 0, policy_version 102837 (0.0009) [2023-12-26 16:06:51,734][105620] Updated weights for policy 1, policy_version 103262 (0.0009) [2023-12-26 16:06:51,765][105692] Updated weights for policy 0, policy_version 102847 (0.0008) [2023-12-26 16:06:51,800][105620] Updated weights for policy 1, policy_version 103272 (0.0010) [2023-12-26 16:06:51,825][105692] Updated weights for policy 0, policy_version 102857 (0.0008) [2023-12-26 16:06:52,532][105692] Updated weights for policy 0, policy_version 102867 (0.0009) [2023-12-26 16:06:52,547][105620] Updated weights for policy 1, policy_version 103282 (0.0007) [2023-12-26 16:06:52,589][105692] Updated weights for policy 0, policy_version 102877 (0.0011) [2023-12-26 16:06:52,599][105620] Updated weights for policy 1, policy_version 103292 (0.0006) [2023-12-26 16:06:52,649][105692] Updated weights for policy 0, policy_version 102887 (0.0010) [2023-12-26 16:06:52,659][105620] Updated weights for policy 1, policy_version 103302 (0.0005) [2023-12-26 16:06:53,392][105692] Updated weights for policy 0, policy_version 102897 (0.0010) [2023-12-26 16:06:53,411][105620] Updated weights for policy 1, policy_version 103312 (0.0008) [2023-12-26 16:06:53,443][105692] Updated weights for policy 0, policy_version 102907 (0.0010) [2023-12-26 16:06:53,475][105620] Updated weights for policy 1, policy_version 103322 (0.0006) [2023-12-26 16:06:53,505][105692] Updated weights for policy 0, policy_version 102917 (0.0010) [2023-12-26 16:06:53,534][105620] Updated weights for policy 1, policy_version 103332 (0.0005) [2023-12-26 16:06:53,563][105692] Updated weights for policy 0, policy_version 102927 (0.0010) [2023-12-26 16:06:54,091][105620] Updated weights for policy 1, policy_version 103342 (0.0006) [2023-12-26 16:06:54,149][105620] Updated weights for policy 1, policy_version 103352 (0.0007) [2023-12-26 16:06:54,204][105620] Updated weights for policy 1, policy_version 103362 (0.0007) [2023-12-26 16:06:54,308][105692] Updated weights for policy 0, policy_version 102937 (0.0010) [2023-12-26 16:06:54,366][105692] Updated weights for policy 0, policy_version 102947 (0.0010) [2023-12-26 16:06:54,430][105692] Updated weights for policy 0, policy_version 102957 (0.0010) [2023-12-26 16:06:54,912][105620] Updated weights for policy 1, policy_version 103372 (0.0008) [2023-12-26 16:06:54,977][105620] Updated weights for policy 1, policy_version 103382 (0.0006) [2023-12-26 16:06:55,039][105620] Updated weights for policy 1, policy_version 103392 (0.0006) [2023-12-26 16:06:55,162][105692] Updated weights for policy 0, policy_version 102967 (0.0010) [2023-12-26 16:06:55,217][105692] Updated weights for policy 0, policy_version 102977 (0.0010) [2023-12-26 16:06:55,275][105692] Updated weights for policy 0, policy_version 102987 (0.0010) [2023-12-26 16:06:55,724][105620] Updated weights for policy 1, policy_version 103402 (0.0008) [2023-12-26 16:06:55,768][105620] Updated weights for policy 1, policy_version 103412 (0.0008) [2023-12-26 16:06:55,817][105620] Updated weights for policy 1, policy_version 103422 (0.0008) [2023-12-26 16:06:55,865][105620] Updated weights for policy 1, policy_version 103432 (0.0008) [2023-12-26 16:06:56,020][105692] Updated weights for policy 0, policy_version 102997 (0.0010) [2023-12-26 16:06:56,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 52854784. Throughput: 0: 9907.4, 1: 9881.8. Samples: 52863876. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:06:56,063][104569] Avg episode reward: [(0, '9264.559'), (1, '2452.599')] [2023-12-26 16:06:56,085][105692] Updated weights for policy 0, policy_version 103007 (0.0009) [2023-12-26 16:06:56,140][105692] Updated weights for policy 0, policy_version 103017 (0.0009) [2023-12-26 16:06:56,632][105620] Updated weights for policy 1, policy_version 103442 (0.0009) [2023-12-26 16:06:56,681][105620] Updated weights for policy 1, policy_version 103452 (0.0009) [2023-12-26 16:06:56,732][105620] Updated weights for policy 1, policy_version 103462 (0.0009) [2023-12-26 16:06:56,880][105692] Updated weights for policy 0, policy_version 103027 (0.0009) [2023-12-26 16:06:56,937][105692] Updated weights for policy 0, policy_version 103037 (0.0010) [2023-12-26 16:06:56,994][105692] Updated weights for policy 0, policy_version 103047 (0.0010) [2023-12-26 16:06:57,521][105620] Updated weights for policy 1, policy_version 103472 (0.0009) [2023-12-26 16:06:57,577][105620] Updated weights for policy 1, policy_version 103482 (0.0007) [2023-12-26 16:06:57,627][105620] Updated weights for policy 1, policy_version 103492 (0.0005) [2023-12-26 16:06:57,705][105692] Updated weights for policy 0, policy_version 103057 (0.0010) [2023-12-26 16:06:57,760][105692] Updated weights for policy 0, policy_version 103067 (0.0010) [2023-12-26 16:06:57,818][105692] Updated weights for policy 0, policy_version 103077 (0.0010) [2023-12-26 16:06:57,882][105692] Updated weights for policy 0, policy_version 103087 (0.0010) [2023-12-26 16:06:58,304][105620] Updated weights for policy 1, policy_version 103502 (0.0007) [2023-12-26 16:06:58,369][105620] Updated weights for policy 1, policy_version 103512 (0.0009) [2023-12-26 16:06:58,433][105620] Updated weights for policy 1, policy_version 103522 (0.0008) [2023-12-26 16:06:58,649][105692] Updated weights for policy 0, policy_version 103097 (0.0008) [2023-12-26 16:06:58,722][105692] Updated weights for policy 0, policy_version 103107 (0.0008) [2023-12-26 16:06:58,796][105692] Updated weights for policy 0, policy_version 103118 (0.0009) [2023-12-26 16:06:59,278][105620] Updated weights for policy 1, policy_version 103532 (0.0009) [2023-12-26 16:06:59,349][105620] Updated weights for policy 1, policy_version 103542 (0.0007) [2023-12-26 16:06:59,411][105620] Updated weights for policy 1, policy_version 103552 (0.0006) [2023-12-26 16:06:59,500][105692] Updated weights for policy 0, policy_version 103128 (0.0010) [2023-12-26 16:06:59,561][105692] Updated weights for policy 0, policy_version 103138 (0.0010) [2023-12-26 16:06:59,625][105692] Updated weights for policy 0, policy_version 103148 (0.0010) [2023-12-26 16:07:00,119][105620] Updated weights for policy 1, policy_version 103562 (0.0007) [2023-12-26 16:07:00,182][105620] Updated weights for policy 1, policy_version 103572 (0.0010) [2023-12-26 16:07:00,245][105620] Updated weights for policy 1, policy_version 103582 (0.0009) [2023-12-26 16:07:00,259][105692] Updated weights for policy 0, policy_version 103158 (0.0008) [2023-12-26 16:07:00,305][105620] Updated weights for policy 1, policy_version 103592 (0.0005) [2023-12-26 16:07:00,321][105692] Updated weights for policy 0, policy_version 103168 (0.0010) [2023-12-26 16:07:00,381][105692] Updated weights for policy 0, policy_version 103178 (0.0006) [2023-12-26 16:07:00,898][105692] Updated weights for policy 0, policy_version 103188 (0.0005) [2023-12-26 16:07:00,948][105692] Updated weights for policy 0, policy_version 103198 (0.0005) [2023-12-26 16:07:01,003][105692] Updated weights for policy 0, policy_version 103208 (0.0005) [2023-12-26 16:07:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 52953088. Throughput: 0: 9923.8, 1: 9835.2. Samples: 52920204. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:07:01,062][104569] Avg episode reward: [(0, '8986.269'), (1, '2446.276')] [2023-12-26 16:07:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000103216_26427392.pth... [2023-12-26 16:07:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000102064_26132480.pth [2023-12-26 16:07:01,127][105620] Updated weights for policy 1, policy_version 103602 (0.0010) [2023-12-26 16:07:01,183][105620] Updated weights for policy 1, policy_version 103612 (0.0006) [2023-12-26 16:07:01,239][105620] Updated weights for policy 1, policy_version 103622 (0.0006) [2023-12-26 16:07:01,249][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000103624_26533888.pth... [2023-12-26 16:07:01,253][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000102472_26238976.pth [2023-12-26 16:07:01,572][105692] Updated weights for policy 0, policy_version 103218 (0.0008) [2023-12-26 16:07:01,630][105692] Updated weights for policy 0, policy_version 103228 (0.0008) [2023-12-26 16:07:01,683][105692] Updated weights for policy 0, policy_version 103238 (0.0005) [2023-12-26 16:07:01,745][105692] Updated weights for policy 0, policy_version 103248 (0.0009) [2023-12-26 16:07:01,961][105620] Updated weights for policy 1, policy_version 103632 (0.0008) [2023-12-26 16:07:02,023][105620] Updated weights for policy 1, policy_version 103642 (0.0009) [2023-12-26 16:07:02,077][105620] Updated weights for policy 1, policy_version 103652 (0.0010) [2023-12-26 16:07:02,429][105692] Updated weights for policy 0, policy_version 103258 (0.0009) [2023-12-26 16:07:02,491][105692] Updated weights for policy 0, policy_version 103268 (0.0010) [2023-12-26 16:07:02,555][105692] Updated weights for policy 0, policy_version 103278 (0.0009) [2023-12-26 16:07:02,773][105620] Updated weights for policy 1, policy_version 103662 (0.0007) [2023-12-26 16:07:02,821][105620] Updated weights for policy 1, policy_version 103672 (0.0005) [2023-12-26 16:07:02,875][105620] Updated weights for policy 1, policy_version 103682 (0.0008) [2023-12-26 16:07:03,271][105692] Updated weights for policy 0, policy_version 103288 (0.0006) [2023-12-26 16:07:03,334][105692] Updated weights for policy 0, policy_version 103298 (0.0006) [2023-12-26 16:07:03,397][105692] Updated weights for policy 0, policy_version 103308 (0.0009) [2023-12-26 16:07:03,630][105620] Updated weights for policy 1, policy_version 103692 (0.0009) [2023-12-26 16:07:03,684][105620] Updated weights for policy 1, policy_version 103702 (0.0009) [2023-12-26 16:07:03,742][105620] Updated weights for policy 1, policy_version 103712 (0.0009) [2023-12-26 16:07:04,095][105692] Updated weights for policy 0, policy_version 103318 (0.0008) [2023-12-26 16:07:04,163][105692] Updated weights for policy 0, policy_version 103328 (0.0009) [2023-12-26 16:07:04,229][105692] Updated weights for policy 0, policy_version 103338 (0.0009) [2023-12-26 16:07:04,506][105620] Updated weights for policy 1, policy_version 103722 (0.0009) [2023-12-26 16:07:04,555][105620] Updated weights for policy 1, policy_version 103732 (0.0010) [2023-12-26 16:07:04,604][105620] Updated weights for policy 1, policy_version 103742 (0.0010) [2023-12-26 16:07:04,662][105620] Updated weights for policy 1, policy_version 103752 (0.0010) [2023-12-26 16:07:04,837][105692] Updated weights for policy 0, policy_version 103348 (0.0010) [2023-12-26 16:07:04,894][105692] Updated weights for policy 0, policy_version 103358 (0.0005) [2023-12-26 16:07:04,946][105692] Updated weights for policy 0, policy_version 103368 (0.0005) [2023-12-26 16:07:05,249][105620] Updated weights for policy 1, policy_version 103762 (0.0005) [2023-12-26 16:07:05,293][105620] Updated weights for policy 1, policy_version 103772 (0.0005) [2023-12-26 16:07:05,352][105620] Updated weights for policy 1, policy_version 103782 (0.0005) [2023-12-26 16:07:05,580][105692] Updated weights for policy 0, policy_version 103378 (0.0006) [2023-12-26 16:07:05,648][105692] Updated weights for policy 0, policy_version 103388 (0.0010) [2023-12-26 16:07:05,702][105692] Updated weights for policy 0, policy_version 103398 (0.0010) [2023-12-26 16:07:05,749][105692] Updated weights for policy 0, policy_version 103408 (0.0010) [2023-12-26 16:07:05,892][105620] Updated weights for policy 1, policy_version 103792 (0.0005) [2023-12-26 16:07:05,950][105620] Updated weights for policy 1, policy_version 103802 (0.0005) [2023-12-26 16:07:06,018][105620] Updated weights for policy 1, policy_version 103812 (0.0005) [2023-12-26 16:07:06,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 53059584. Throughput: 0: 9858.7, 1: 9769.6. Samples: 53038784. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:07:06,062][104569] Avg episode reward: [(0, '9166.803'), (1, '6320.627')] [2023-12-26 16:07:06,379][105692] Updated weights for policy 0, policy_version 103418 (0.0009) [2023-12-26 16:07:06,438][105692] Updated weights for policy 0, policy_version 103428 (0.0006) [2023-12-26 16:07:06,505][105692] Updated weights for policy 0, policy_version 103438 (0.0006) [2023-12-26 16:07:06,592][105620] Updated weights for policy 1, policy_version 103822 (0.0006) [2023-12-26 16:07:06,657][105620] Updated weights for policy 1, policy_version 103832 (0.0006) [2023-12-26 16:07:06,721][105620] Updated weights for policy 1, policy_version 103842 (0.0009) [2023-12-26 16:07:07,169][105692] Updated weights for policy 0, policy_version 103448 (0.0007) [2023-12-26 16:07:07,217][105692] Updated weights for policy 0, policy_version 103458 (0.0007) [2023-12-26 16:07:07,277][105692] Updated weights for policy 0, policy_version 103468 (0.0008) [2023-12-26 16:07:07,401][105620] Updated weights for policy 1, policy_version 103852 (0.0010) [2023-12-26 16:07:07,462][105620] Updated weights for policy 1, policy_version 103862 (0.0010) [2023-12-26 16:07:07,521][105620] Updated weights for policy 1, policy_version 103872 (0.0010) [2023-12-26 16:07:08,058][105692] Updated weights for policy 0, policy_version 103478 (0.0008) [2023-12-26 16:07:08,123][105692] Updated weights for policy 0, policy_version 103488 (0.0009) [2023-12-26 16:07:08,175][105692] Updated weights for policy 0, policy_version 103498 (0.0008) [2023-12-26 16:07:08,265][105620] Updated weights for policy 1, policy_version 103882 (0.0010) [2023-12-26 16:07:08,328][105620] Updated weights for policy 1, policy_version 103892 (0.0010) [2023-12-26 16:07:08,385][105620] Updated weights for policy 1, policy_version 103902 (0.0010) [2023-12-26 16:07:08,444][105620] Updated weights for policy 1, policy_version 103912 (0.0010) [2023-12-26 16:07:08,944][105692] Updated weights for policy 0, policy_version 103508 (0.0007) [2023-12-26 16:07:09,002][105692] Updated weights for policy 0, policy_version 103518 (0.0005) [2023-12-26 16:07:09,047][105692] Updated weights for policy 0, policy_version 103528 (0.0007) [2023-12-26 16:07:09,157][105620] Updated weights for policy 1, policy_version 103922 (0.0008) [2023-12-26 16:07:09,219][105620] Updated weights for policy 1, policy_version 103932 (0.0008) [2023-12-26 16:07:09,286][105620] Updated weights for policy 1, policy_version 103942 (0.0010) [2023-12-26 16:07:09,844][105692] Updated weights for policy 0, policy_version 103538 (0.0007) [2023-12-26 16:07:09,911][105692] Updated weights for policy 0, policy_version 103548 (0.0008) [2023-12-26 16:07:09,979][105692] Updated weights for policy 0, policy_version 103558 (0.0008) [2023-12-26 16:07:09,982][105620] Updated weights for policy 1, policy_version 103952 (0.0009) [2023-12-26 16:07:10,039][105692] Updated weights for policy 0, policy_version 103568 (0.0008) [2023-12-26 16:07:10,039][105620] Updated weights for policy 1, policy_version 103962 (0.0006) [2023-12-26 16:07:10,098][105620] Updated weights for policy 1, policy_version 103972 (0.0006) [2023-12-26 16:07:10,732][105692] Updated weights for policy 0, policy_version 103578 (0.0009) [2023-12-26 16:07:10,789][105692] Updated weights for policy 0, policy_version 103588 (0.0008) [2023-12-26 16:07:10,815][105620] Updated weights for policy 1, policy_version 103982 (0.0007) [2023-12-26 16:07:10,850][105692] Updated weights for policy 0, policy_version 103598 (0.0009) [2023-12-26 16:07:10,864][105620] Updated weights for policy 1, policy_version 103992 (0.0007) [2023-12-26 16:07:10,929][105620] Updated weights for policy 1, policy_version 104002 (0.0006) [2023-12-26 16:07:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19933.9, 300 sec: 19660.8). Total num frames: 53157888. Throughput: 0: 9957.4, 1: 9766.3. Samples: 53160824. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:07:11,062][104569] Avg episode reward: [(0, '9166.698'), (1, '7592.031')] [2023-12-26 16:07:11,623][105620] Updated weights for policy 1, policy_version 104012 (0.0007) [2023-12-26 16:07:11,687][105620] Updated weights for policy 1, policy_version 104022 (0.0008) [2023-12-26 16:07:11,744][105692] Updated weights for policy 0, policy_version 103608 (0.0009) [2023-12-26 16:07:11,752][105620] Updated weights for policy 1, policy_version 104032 (0.0007) [2023-12-26 16:07:11,809][105692] Updated weights for policy 0, policy_version 103618 (0.0008) [2023-12-26 16:07:11,871][105692] Updated weights for policy 0, policy_version 103628 (0.0009) [2023-12-26 16:07:12,410][105620] Updated weights for policy 1, policy_version 104042 (0.0008) [2023-12-26 16:07:12,472][105620] Updated weights for policy 1, policy_version 104052 (0.0007) [2023-12-26 16:07:12,533][105620] Updated weights for policy 1, policy_version 104062 (0.0009) [2023-12-26 16:07:12,599][105620] Updated weights for policy 1, policy_version 104072 (0.0011) [2023-12-26 16:07:12,648][105692] Updated weights for policy 0, policy_version 103638 (0.0009) [2023-12-26 16:07:12,697][105692] Updated weights for policy 0, policy_version 103648 (0.0008) [2023-12-26 16:07:12,747][105692] Updated weights for policy 0, policy_version 103658 (0.0008) [2023-12-26 16:07:13,300][105620] Updated weights for policy 1, policy_version 104082 (0.0010) [2023-12-26 16:07:13,354][105620] Updated weights for policy 1, policy_version 104092 (0.0010) [2023-12-26 16:07:13,413][105620] Updated weights for policy 1, policy_version 104102 (0.0008) [2023-12-26 16:07:13,426][105692] Updated weights for policy 0, policy_version 103668 (0.0008) [2023-12-26 16:07:13,488][105692] Updated weights for policy 0, policy_version 103678 (0.0006) [2023-12-26 16:07:13,544][105692] Updated weights for policy 0, policy_version 103688 (0.0008) [2023-12-26 16:07:14,047][105620] Updated weights for policy 1, policy_version 104112 (0.0006) [2023-12-26 16:07:14,091][105620] Updated weights for policy 1, policy_version 104122 (0.0005) [2023-12-26 16:07:14,144][105620] Updated weights for policy 1, policy_version 104132 (0.0008) [2023-12-26 16:07:14,281][105692] Updated weights for policy 0, policy_version 103698 (0.0008) [2023-12-26 16:07:14,334][105692] Updated weights for policy 0, policy_version 103708 (0.0010) [2023-12-26 16:07:14,387][105692] Updated weights for policy 0, policy_version 103718 (0.0009) [2023-12-26 16:07:14,703][105620] Updated weights for policy 1, policy_version 104142 (0.0010) [2023-12-26 16:07:14,757][105620] Updated weights for policy 1, policy_version 104152 (0.0010) [2023-12-26 16:07:14,820][105620] Updated weights for policy 1, policy_version 104162 (0.0008) [2023-12-26 16:07:15,244][105692] Updated weights for policy 0, policy_version 103729 (0.0010) [2023-12-26 16:07:15,296][105692] Updated weights for policy 0, policy_version 103739 (0.0010) [2023-12-26 16:07:15,345][105692] Updated weights for policy 0, policy_version 103749 (0.0010) [2023-12-26 16:07:15,405][105692] Updated weights for policy 0, policy_version 103759 (0.0010) [2023-12-26 16:07:15,625][105620] Updated weights for policy 1, policy_version 104172 (0.0008) [2023-12-26 16:07:15,669][105620] Updated weights for policy 1, policy_version 104182 (0.0008) [2023-12-26 16:07:15,717][105620] Updated weights for policy 1, policy_version 104192 (0.0008) [2023-12-26 16:07:16,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19660.7, 300 sec: 19633.0). Total num frames: 53248000. Throughput: 0: 9850.2, 1: 9810.4. Samples: 53218952. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:07:16,063][104569] Avg episode reward: [(0, '9166.728'), (1, '7862.347')] [2023-12-26 16:07:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000103760_26566656.pth... [2023-12-26 16:07:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000104200_26681344.pth... [2023-12-26 16:07:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000102640_26279936.pth [2023-12-26 16:07:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000103048_26386432.pth [2023-12-26 16:07:16,154][105692] Updated weights for policy 0, policy_version 103769 (0.0006) [2023-12-26 16:07:16,203][105692] Updated weights for policy 0, policy_version 103779 (0.0005) [2023-12-26 16:07:16,253][105692] Updated weights for policy 0, policy_version 103789 (0.0009) [2023-12-26 16:07:16,542][105620] Updated weights for policy 1, policy_version 104202 (0.0008) [2023-12-26 16:07:16,607][105620] Updated weights for policy 1, policy_version 104212 (0.0009) [2023-12-26 16:07:16,673][105620] Updated weights for policy 1, policy_version 104222 (0.0010) [2023-12-26 16:07:16,735][105620] Updated weights for policy 1, policy_version 104232 (0.0009) [2023-12-26 16:07:16,817][105692] Updated weights for policy 0, policy_version 103799 (0.0007) [2023-12-26 16:07:16,877][105692] Updated weights for policy 0, policy_version 103809 (0.0010) [2023-12-26 16:07:16,939][105692] Updated weights for policy 0, policy_version 103819 (0.0010) [2023-12-26 16:07:17,516][105620] Updated weights for policy 1, policy_version 104242 (0.0008) [2023-12-26 16:07:17,576][105620] Updated weights for policy 1, policy_version 104252 (0.0008) [2023-12-26 16:07:17,641][105620] Updated weights for policy 1, policy_version 104262 (0.0009) [2023-12-26 16:07:17,649][105692] Updated weights for policy 0, policy_version 103829 (0.0010) [2023-12-26 16:07:17,702][105692] Updated weights for policy 0, policy_version 103839 (0.0010) [2023-12-26 16:07:17,768][105692] Updated weights for policy 0, policy_version 103849 (0.0006) [2023-12-26 16:07:18,410][105692] Updated weights for policy 0, policy_version 103859 (0.0006) [2023-12-26 16:07:18,433][105620] Updated weights for policy 1, policy_version 104272 (0.0008) [2023-12-26 16:07:18,472][105692] Updated weights for policy 0, policy_version 103869 (0.0008) [2023-12-26 16:07:18,503][105620] Updated weights for policy 1, policy_version 104282 (0.0007) [2023-12-26 16:07:18,534][105692] Updated weights for policy 0, policy_version 103879 (0.0006) [2023-12-26 16:07:18,562][105620] Updated weights for policy 1, policy_version 104292 (0.0008) [2023-12-26 16:07:19,255][105692] Updated weights for policy 0, policy_version 103889 (0.0006) [2023-12-26 16:07:19,310][105692] Updated weights for policy 0, policy_version 103899 (0.0009) [2023-12-26 16:07:19,366][105620] Updated weights for policy 1, policy_version 104302 (0.0008) [2023-12-26 16:07:19,377][105692] Updated weights for policy 0, policy_version 103909 (0.0008) [2023-12-26 16:07:19,424][105620] Updated weights for policy 1, policy_version 104312 (0.0008) [2023-12-26 16:07:19,430][105692] Updated weights for policy 0, policy_version 103919 (0.0006) [2023-12-26 16:07:19,480][105620] Updated weights for policy 1, policy_version 104322 (0.0009) [2023-12-26 16:07:20,171][105692] Updated weights for policy 0, policy_version 103929 (0.0008) [2023-12-26 16:07:20,230][105620] Updated weights for policy 1, policy_version 104332 (0.0010) [2023-12-26 16:07:20,231][105692] Updated weights for policy 0, policy_version 103939 (0.0008) [2023-12-26 16:07:20,284][105620] Updated weights for policy 1, policy_version 104342 (0.0010) [2023-12-26 16:07:20,288][105692] Updated weights for policy 0, policy_version 103949 (0.0009) [2023-12-26 16:07:20,351][105620] Updated weights for policy 1, policy_version 104352 (0.0011) [2023-12-26 16:07:21,045][105620] Updated weights for policy 1, policy_version 104362 (0.0011) [2023-12-26 16:07:21,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 53338112. Throughput: 0: 9850.2, 1: 9777.8. Samples: 53333312. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:07:21,063][104569] Avg episode reward: [(0, '9351.340'), (1, '8310.107')] [2023-12-26 16:07:21,095][105620] Updated weights for policy 1, policy_version 104372 (0.0011) [2023-12-26 16:07:21,102][105692] Updated weights for policy 0, policy_version 103959 (0.0007) [2023-12-26 16:07:21,155][105620] Updated weights for policy 1, policy_version 104382 (0.0010) [2023-12-26 16:07:21,169][105692] Updated weights for policy 0, policy_version 103969 (0.0007) [2023-12-26 16:07:21,216][105620] Updated weights for policy 1, policy_version 104392 (0.0009) [2023-12-26 16:07:21,226][105692] Updated weights for policy 0, policy_version 103979 (0.0009) [2023-12-26 16:07:22,022][105620] Updated weights for policy 1, policy_version 104402 (0.0010) [2023-12-26 16:07:22,055][105692] Updated weights for policy 0, policy_version 103989 (0.0009) [2023-12-26 16:07:22,072][105620] Updated weights for policy 1, policy_version 104412 (0.0010) [2023-12-26 16:07:22,118][105692] Updated weights for policy 0, policy_version 103999 (0.0007) [2023-12-26 16:07:22,129][105620] Updated weights for policy 1, policy_version 104422 (0.0010) [2023-12-26 16:07:22,182][105692] Updated weights for policy 0, policy_version 104009 (0.0011) [2023-12-26 16:07:22,874][105620] Updated weights for policy 1, policy_version 104432 (0.0011) [2023-12-26 16:07:22,927][105692] Updated weights for policy 0, policy_version 104019 (0.0010) [2023-12-26 16:07:22,934][105620] Updated weights for policy 1, policy_version 104442 (0.0010) [2023-12-26 16:07:22,993][105692] Updated weights for policy 0, policy_version 104029 (0.0010) [2023-12-26 16:07:22,993][105620] Updated weights for policy 1, policy_version 104452 (0.0011) [2023-12-26 16:07:23,054][105692] Updated weights for policy 0, policy_version 104039 (0.0008) [2023-12-26 16:07:23,738][105692] Updated weights for policy 0, policy_version 104049 (0.0008) [2023-12-26 16:07:23,739][105620] Updated weights for policy 1, policy_version 104462 (0.0010) [2023-12-26 16:07:23,784][105692] Updated weights for policy 0, policy_version 104059 (0.0008) [2023-12-26 16:07:23,786][105620] Updated weights for policy 1, policy_version 104472 (0.0010) [2023-12-26 16:07:23,831][105620] Updated weights for policy 1, policy_version 104482 (0.0010) [2023-12-26 16:07:23,836][105692] Updated weights for policy 0, policy_version 104069 (0.0005) [2023-12-26 16:07:23,890][105692] Updated weights for policy 0, policy_version 104079 (0.0007) [2023-12-26 16:07:24,537][105620] Updated weights for policy 1, policy_version 104492 (0.0009) [2023-12-26 16:07:24,589][105620] Updated weights for policy 1, policy_version 104502 (0.0005) [2023-12-26 16:07:24,640][105620] Updated weights for policy 1, policy_version 104512 (0.0005) [2023-12-26 16:07:24,683][105692] Updated weights for policy 0, policy_version 104089 (0.0009) [2023-12-26 16:07:24,737][105692] Updated weights for policy 0, policy_version 104099 (0.0010) [2023-12-26 16:07:24,786][105692] Updated weights for policy 0, policy_version 104109 (0.0009) [2023-12-26 16:07:25,348][105620] Updated weights for policy 1, policy_version 104522 (0.0005) [2023-12-26 16:07:25,401][105620] Updated weights for policy 1, policy_version 104532 (0.0005) [2023-12-26 16:07:25,449][105692] Updated weights for policy 0, policy_version 104119 (0.0007) [2023-12-26 16:07:25,452][105620] Updated weights for policy 1, policy_version 104542 (0.0005) [2023-12-26 16:07:25,500][105620] Updated weights for policy 1, policy_version 104552 (0.0007) [2023-12-26 16:07:25,504][105692] Updated weights for policy 0, policy_version 104129 (0.0006) [2023-12-26 16:07:25,563][105692] Updated weights for policy 0, policy_version 104139 (0.0006) [2023-12-26 16:07:26,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 53436416. Throughput: 0: 9730.8, 1: 9738.7. Samples: 53447036. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:07:26,062][104569] Avg episode reward: [(0, '8996.093'), (1, '8238.572')] [2023-12-26 16:07:26,150][105692] Updated weights for policy 0, policy_version 104149 (0.0007) [2023-12-26 16:07:26,210][105692] Updated weights for policy 0, policy_version 104159 (0.0008) [2023-12-26 16:07:26,245][105620] Updated weights for policy 1, policy_version 104562 (0.0006) [2023-12-26 16:07:26,263][105692] Updated weights for policy 0, policy_version 104169 (0.0007) [2023-12-26 16:07:26,293][105620] Updated weights for policy 1, policy_version 104572 (0.0006) [2023-12-26 16:07:26,339][105620] Updated weights for policy 1, policy_version 104582 (0.0008) [2023-12-26 16:07:27,015][105692] Updated weights for policy 0, policy_version 104179 (0.0008) [2023-12-26 16:07:27,062][105692] Updated weights for policy 0, policy_version 104189 (0.0009) [2023-12-26 16:07:27,115][105692] Updated weights for policy 0, policy_version 104199 (0.0007) [2023-12-26 16:07:27,121][105620] Updated weights for policy 1, policy_version 104592 (0.0009) [2023-12-26 16:07:27,178][105620] Updated weights for policy 1, policy_version 104602 (0.0008) [2023-12-26 16:07:27,232][105620] Updated weights for policy 1, policy_version 104612 (0.0008) [2023-12-26 16:07:27,864][105692] Updated weights for policy 0, policy_version 104209 (0.0006) [2023-12-26 16:07:27,913][105692] Updated weights for policy 0, policy_version 104219 (0.0009) [2023-12-26 16:07:27,964][105692] Updated weights for policy 0, policy_version 104229 (0.0008) [2023-12-26 16:07:27,974][105620] Updated weights for policy 1, policy_version 104622 (0.0007) [2023-12-26 16:07:28,024][105692] Updated weights for policy 0, policy_version 104239 (0.0007) [2023-12-26 16:07:28,030][105620] Updated weights for policy 1, policy_version 104632 (0.0007) [2023-12-26 16:07:28,094][105620] Updated weights for policy 1, policy_version 104642 (0.0006) [2023-12-26 16:07:28,786][105620] Updated weights for policy 1, policy_version 104652 (0.0006) [2023-12-26 16:07:28,793][105692] Updated weights for policy 0, policy_version 104249 (0.0007) [2023-12-26 16:07:28,842][105620] Updated weights for policy 1, policy_version 104662 (0.0008) [2023-12-26 16:07:28,845][105692] Updated weights for policy 0, policy_version 104259 (0.0005) [2023-12-26 16:07:28,902][105620] Updated weights for policy 1, policy_version 104672 (0.0007) [2023-12-26 16:07:28,904][105692] Updated weights for policy 0, policy_version 104269 (0.0006) [2023-12-26 16:07:29,642][105692] Updated weights for policy 0, policy_version 104279 (0.0008) [2023-12-26 16:07:29,663][105620] Updated weights for policy 1, policy_version 104682 (0.0008) [2023-12-26 16:07:29,692][105692] Updated weights for policy 0, policy_version 104289 (0.0010) [2023-12-26 16:07:29,712][105620] Updated weights for policy 1, policy_version 104692 (0.0009) [2023-12-26 16:07:29,743][105692] Updated weights for policy 0, policy_version 104299 (0.0008) [2023-12-26 16:07:29,765][105620] Updated weights for policy 1, policy_version 104702 (0.0008) [2023-12-26 16:07:29,814][105620] Updated weights for policy 1, policy_version 104712 (0.0009) [2023-12-26 16:07:30,517][105620] Updated weights for policy 1, policy_version 104722 (0.0008) [2023-12-26 16:07:30,529][105692] Updated weights for policy 0, policy_version 104309 (0.0007) [2023-12-26 16:07:30,572][105620] Updated weights for policy 1, policy_version 104732 (0.0005) [2023-12-26 16:07:30,582][105692] Updated weights for policy 0, policy_version 104319 (0.0009) [2023-12-26 16:07:30,620][105620] Updated weights for policy 1, policy_version 104742 (0.0005) [2023-12-26 16:07:30,639][105692] Updated weights for policy 0, policy_version 104329 (0.0009) [2023-12-26 16:07:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.5, 300 sec: 19577.5). Total num frames: 53534720. Throughput: 0: 9774.3, 1: 9735.6. Samples: 53505036. Policy #0 lag: (min: 30.0, avg: 36.5, max: 62.0) [2023-12-26 16:07:31,062][104569] Avg episode reward: [(0, '8996.282'), (1, '8757.864')] [2023-12-26 16:07:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000104336_26714112.pth... [2023-12-26 16:07:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000104744_26820608.pth... [2023-12-26 16:07:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000103216_26427392.pth [2023-12-26 16:07:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000103624_26533888.pth [2023-12-26 16:07:31,309][105620] Updated weights for policy 1, policy_version 104752 (0.0007) [2023-12-26 16:07:31,364][105620] Updated weights for policy 1, policy_version 104762 (0.0008) [2023-12-26 16:07:31,396][105692] Updated weights for policy 0, policy_version 104339 (0.0009) [2023-12-26 16:07:31,425][105620] Updated weights for policy 1, policy_version 104772 (0.0007) [2023-12-26 16:07:31,453][105692] Updated weights for policy 0, policy_version 104349 (0.0009) [2023-12-26 16:07:31,500][105692] Updated weights for policy 0, policy_version 104359 (0.0009) [2023-12-26 16:07:32,124][105620] Updated weights for policy 1, policy_version 104782 (0.0010) [2023-12-26 16:07:32,182][105620] Updated weights for policy 1, policy_version 104792 (0.0010) [2023-12-26 16:07:32,243][105620] Updated weights for policy 1, policy_version 104802 (0.0010) [2023-12-26 16:07:32,282][105692] Updated weights for policy 0, policy_version 104369 (0.0009) [2023-12-26 16:07:32,342][105692] Updated weights for policy 0, policy_version 104379 (0.0010) [2023-12-26 16:07:32,404][105692] Updated weights for policy 0, policy_version 104389 (0.0010) [2023-12-26 16:07:32,462][105692] Updated weights for policy 0, policy_version 104399 (0.0010) [2023-12-26 16:07:32,888][105620] Updated weights for policy 1, policy_version 104812 (0.0009) [2023-12-26 16:07:32,948][105620] Updated weights for policy 1, policy_version 104822 (0.0010) [2023-12-26 16:07:33,012][105620] Updated weights for policy 1, policy_version 104832 (0.0010) [2023-12-26 16:07:33,141][105692] Updated weights for policy 0, policy_version 104409 (0.0010) [2023-12-26 16:07:33,191][105692] Updated weights for policy 0, policy_version 104419 (0.0010) [2023-12-26 16:07:33,242][105692] Updated weights for policy 0, policy_version 104429 (0.0010) [2023-12-26 16:07:33,670][105620] Updated weights for policy 1, policy_version 104842 (0.0008) [2023-12-26 16:07:33,716][105620] Updated weights for policy 1, policy_version 104852 (0.0005) [2023-12-26 16:07:33,768][105620] Updated weights for policy 1, policy_version 104862 (0.0006) [2023-12-26 16:07:33,824][105620] Updated weights for policy 1, policy_version 104872 (0.0005) [2023-12-26 16:07:34,005][105692] Updated weights for policy 0, policy_version 104439 (0.0010) [2023-12-26 16:07:34,067][105692] Updated weights for policy 0, policy_version 104449 (0.0010) [2023-12-26 16:07:34,119][105692] Updated weights for policy 0, policy_version 104459 (0.0010) [2023-12-26 16:07:34,524][105620] Updated weights for policy 1, policy_version 104882 (0.0011) [2023-12-26 16:07:34,573][105620] Updated weights for policy 1, policy_version 104892 (0.0010) [2023-12-26 16:07:34,640][105620] Updated weights for policy 1, policy_version 104902 (0.0011) [2023-12-26 16:07:34,799][105692] Updated weights for policy 0, policy_version 104469 (0.0007) [2023-12-26 16:07:34,855][105692] Updated weights for policy 0, policy_version 104479 (0.0005) [2023-12-26 16:07:34,901][105692] Updated weights for policy 0, policy_version 104489 (0.0005) [2023-12-26 16:07:35,385][105620] Updated weights for policy 1, policy_version 104912 (0.0006) [2023-12-26 16:07:35,434][105620] Updated weights for policy 1, policy_version 104922 (0.0005) [2023-12-26 16:07:35,487][105620] Updated weights for policy 1, policy_version 104932 (0.0005) [2023-12-26 16:07:35,576][105692] Updated weights for policy 0, policy_version 104499 (0.0005) [2023-12-26 16:07:35,659][105692] Updated weights for policy 0, policy_version 104509 (0.0005) [2023-12-26 16:07:35,728][105692] Updated weights for policy 0, policy_version 104519 (0.0009) [2023-12-26 16:07:36,018][105620] Updated weights for policy 1, policy_version 104942 (0.0005) [2023-12-26 16:07:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 53633024. Throughput: 0: 9704.4, 1: 9735.1. Samples: 53621760. Policy #0 lag: (min: 30.0, avg: 36.5, max: 62.0) [2023-12-26 16:07:36,062][104569] Avg episode reward: [(0, '8991.939'), (1, '8931.376')] [2023-12-26 16:07:36,072][105620] Updated weights for policy 1, policy_version 104952 (0.0006) [2023-12-26 16:07:36,127][105620] Updated weights for policy 1, policy_version 104962 (0.0008) [2023-12-26 16:07:36,347][105692] Updated weights for policy 0, policy_version 104529 (0.0010) [2023-12-26 16:07:36,415][105692] Updated weights for policy 0, policy_version 104539 (0.0009) [2023-12-26 16:07:36,477][105692] Updated weights for policy 0, policy_version 104549 (0.0010) [2023-12-26 16:07:36,540][105692] Updated weights for policy 0, policy_version 104559 (0.0010) [2023-12-26 16:07:36,898][105620] Updated weights for policy 1, policy_version 104972 (0.0008) [2023-12-26 16:07:36,950][105620] Updated weights for policy 1, policy_version 104982 (0.0008) [2023-12-26 16:07:37,014][105620] Updated weights for policy 1, policy_version 104992 (0.0008) [2023-12-26 16:07:37,275][105692] Updated weights for policy 0, policy_version 104569 (0.0010) [2023-12-26 16:07:37,337][105692] Updated weights for policy 0, policy_version 104579 (0.0010) [2023-12-26 16:07:37,391][105692] Updated weights for policy 0, policy_version 104589 (0.0010) [2023-12-26 16:07:37,792][105620] Updated weights for policy 1, policy_version 105002 (0.0008) [2023-12-26 16:07:37,847][105620] Updated weights for policy 1, policy_version 105012 (0.0008) [2023-12-26 16:07:37,911][105620] Updated weights for policy 1, policy_version 105022 (0.0008) [2023-12-26 16:07:37,973][105620] Updated weights for policy 1, policy_version 105032 (0.0008) [2023-12-26 16:07:38,036][105692] Updated weights for policy 0, policy_version 104599 (0.0010) [2023-12-26 16:07:38,080][105692] Updated weights for policy 0, policy_version 104609 (0.0010) [2023-12-26 16:07:38,136][105692] Updated weights for policy 0, policy_version 104619 (0.0010) [2023-12-26 16:07:38,702][105620] Updated weights for policy 1, policy_version 105042 (0.0008) [2023-12-26 16:07:38,765][105620] Updated weights for policy 1, policy_version 105052 (0.0008) [2023-12-26 16:07:38,820][105620] Updated weights for policy 1, policy_version 105062 (0.0008) [2023-12-26 16:07:38,941][105692] Updated weights for policy 0, policy_version 104629 (0.0011) [2023-12-26 16:07:39,002][105692] Updated weights for policy 0, policy_version 104639 (0.0010) [2023-12-26 16:07:39,066][105692] Updated weights for policy 0, policy_version 104649 (0.0010) [2023-12-26 16:07:39,516][105620] Updated weights for policy 1, policy_version 105072 (0.0006) [2023-12-26 16:07:39,580][105620] Updated weights for policy 1, policy_version 105082 (0.0007) [2023-12-26 16:07:39,631][105620] Updated weights for policy 1, policy_version 105092 (0.0010) [2023-12-26 16:07:39,790][105692] Updated weights for policy 0, policy_version 104659 (0.0011) [2023-12-26 16:07:39,853][105692] Updated weights for policy 0, policy_version 104669 (0.0008) [2023-12-26 16:07:39,901][105692] Updated weights for policy 0, policy_version 104679 (0.0008) [2023-12-26 16:07:40,328][105620] Updated weights for policy 1, policy_version 105102 (0.0008) [2023-12-26 16:07:40,386][105620] Updated weights for policy 1, policy_version 105112 (0.0006) [2023-12-26 16:07:40,450][105620] Updated weights for policy 1, policy_version 105122 (0.0009) [2023-12-26 16:07:40,687][105692] Updated weights for policy 0, policy_version 104689 (0.0008) [2023-12-26 16:07:40,752][105692] Updated weights for policy 0, policy_version 104699 (0.0007) [2023-12-26 16:07:40,809][105692] Updated weights for policy 0, policy_version 104709 (0.0008) [2023-12-26 16:07:40,868][105692] Updated weights for policy 0, policy_version 104719 (0.0009) [2023-12-26 16:07:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 53731328. Throughput: 0: 9717.3, 1: 9753.8. Samples: 53740072. Policy #0 lag: (min: 30.0, avg: 36.5, max: 62.0) [2023-12-26 16:07:41,062][104569] Avg episode reward: [(0, '8622.209'), (1, '9184.445')] [2023-12-26 16:07:41,076][105620] Updated weights for policy 1, policy_version 105132 (0.0007) [2023-12-26 16:07:41,144][105620] Updated weights for policy 1, policy_version 105142 (0.0008) [2023-12-26 16:07:41,213][105620] Updated weights for policy 1, policy_version 105152 (0.0009) [2023-12-26 16:07:41,674][105692] Updated weights for policy 0, policy_version 104729 (0.0009) [2023-12-26 16:07:41,739][105692] Updated weights for policy 0, policy_version 104739 (0.0008) [2023-12-26 16:07:41,791][105692] Updated weights for policy 0, policy_version 104749 (0.0006) [2023-12-26 16:07:42,067][105620] Updated weights for policy 1, policy_version 105162 (0.0009) [2023-12-26 16:07:42,123][105620] Updated weights for policy 1, policy_version 105172 (0.0010) [2023-12-26 16:07:42,182][105620] Updated weights for policy 1, policy_version 105182 (0.0008) [2023-12-26 16:07:42,242][105620] Updated weights for policy 1, policy_version 105192 (0.0010) [2023-12-26 16:07:42,406][105692] Updated weights for policy 0, policy_version 104759 (0.0007) [2023-12-26 16:07:42,463][105692] Updated weights for policy 0, policy_version 104769 (0.0008) [2023-12-26 16:07:42,519][105692] Updated weights for policy 0, policy_version 104779 (0.0008) [2023-12-26 16:07:42,993][105620] Updated weights for policy 1, policy_version 105202 (0.0010) [2023-12-26 16:07:43,058][105620] Updated weights for policy 1, policy_version 105212 (0.0010) [2023-12-26 16:07:43,119][105620] Updated weights for policy 1, policy_version 105222 (0.0010) [2023-12-26 16:07:43,280][105692] Updated weights for policy 0, policy_version 104789 (0.0009) [2023-12-26 16:07:43,334][105692] Updated weights for policy 0, policy_version 104799 (0.0010) [2023-12-26 16:07:43,402][105692] Updated weights for policy 0, policy_version 104809 (0.0010) [2023-12-26 16:07:43,730][105620] Updated weights for policy 1, policy_version 105232 (0.0006) [2023-12-26 16:07:43,793][105620] Updated weights for policy 1, policy_version 105242 (0.0005) [2023-12-26 16:07:43,855][105620] Updated weights for policy 1, policy_version 105252 (0.0005) [2023-12-26 16:07:44,127][105692] Updated weights for policy 0, policy_version 104819 (0.0009) [2023-12-26 16:07:44,192][105692] Updated weights for policy 0, policy_version 104829 (0.0005) [2023-12-26 16:07:44,243][105692] Updated weights for policy 0, policy_version 104839 (0.0005) [2023-12-26 16:07:44,401][105620] Updated weights for policy 1, policy_version 105262 (0.0005) [2023-12-26 16:07:44,447][105620] Updated weights for policy 1, policy_version 105272 (0.0008) [2023-12-26 16:07:44,492][105620] Updated weights for policy 1, policy_version 105282 (0.0010) [2023-12-26 16:07:44,880][105692] Updated weights for policy 0, policy_version 104849 (0.0006) [2023-12-26 16:07:44,940][105692] Updated weights for policy 0, policy_version 104859 (0.0008) [2023-12-26 16:07:45,001][105692] Updated weights for policy 0, policy_version 104869 (0.0009) [2023-12-26 16:07:45,060][105692] Updated weights for policy 0, policy_version 104879 (0.0008) [2023-12-26 16:07:45,252][105620] Updated weights for policy 1, policy_version 105292 (0.0010) [2023-12-26 16:07:45,315][105620] Updated weights for policy 1, policy_version 105302 (0.0009) [2023-12-26 16:07:45,369][105620] Updated weights for policy 1, policy_version 105312 (0.0009) [2023-12-26 16:07:45,725][105692] Updated weights for policy 0, policy_version 104889 (0.0008) [2023-12-26 16:07:45,772][105692] Updated weights for policy 0, policy_version 104899 (0.0008) [2023-12-26 16:07:45,820][105692] Updated weights for policy 0, policy_version 104909 (0.0009) [2023-12-26 16:07:46,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 53829632. Throughput: 0: 9710.3, 1: 9785.6. Samples: 53797528. Policy #0 lag: (min: 30.0, avg: 36.5, max: 62.0) [2023-12-26 16:07:46,063][104569] Avg episode reward: [(0, '8634.277'), (1, '9179.228')] [2023-12-26 16:07:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000104912_26861568.pth... [2023-12-26 16:07:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000105320_26968064.pth... [2023-12-26 16:07:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000103760_26566656.pth [2023-12-26 16:07:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000104200_26681344.pth [2023-12-26 16:07:46,120][105620] Updated weights for policy 1, policy_version 105322 (0.0009) [2023-12-26 16:07:46,174][105620] Updated weights for policy 1, policy_version 105332 (0.0009) [2023-12-26 16:07:46,235][105620] Updated weights for policy 1, policy_version 105342 (0.0008) [2023-12-26 16:07:46,301][105620] Updated weights for policy 1, policy_version 105352 (0.0009) [2023-12-26 16:07:46,536][105692] Updated weights for policy 0, policy_version 104919 (0.0006) [2023-12-26 16:07:46,587][105692] Updated weights for policy 0, policy_version 104929 (0.0009) [2023-12-26 16:07:46,635][105692] Updated weights for policy 0, policy_version 104939 (0.0010) [2023-12-26 16:07:46,982][105620] Updated weights for policy 1, policy_version 105362 (0.0010) [2023-12-26 16:07:47,043][105620] Updated weights for policy 1, policy_version 105372 (0.0010) [2023-12-26 16:07:47,100][105620] Updated weights for policy 1, policy_version 105382 (0.0010) [2023-12-26 16:07:47,306][105692] Updated weights for policy 0, policy_version 104949 (0.0010) [2023-12-26 16:07:47,360][105692] Updated weights for policy 0, policy_version 104959 (0.0010) [2023-12-26 16:07:47,404][105692] Updated weights for policy 0, policy_version 104969 (0.0010) [2023-12-26 16:07:47,672][105620] Updated weights for policy 1, policy_version 105392 (0.0006) [2023-12-26 16:07:47,731][105620] Updated weights for policy 1, policy_version 105402 (0.0005) [2023-12-26 16:07:47,782][105620] Updated weights for policy 1, policy_version 105412 (0.0007) [2023-12-26 16:07:48,079][105692] Updated weights for policy 0, policy_version 104979 (0.0010) [2023-12-26 16:07:48,145][105692] Updated weights for policy 0, policy_version 104989 (0.0011) [2023-12-26 16:07:48,213][105692] Updated weights for policy 0, policy_version 104999 (0.0010) [2023-12-26 16:07:48,375][105620] Updated weights for policy 1, policy_version 105422 (0.0009) [2023-12-26 16:07:48,431][105620] Updated weights for policy 1, policy_version 105432 (0.0009) [2023-12-26 16:07:48,482][105620] Updated weights for policy 1, policy_version 105442 (0.0008) [2023-12-26 16:07:48,892][105692] Updated weights for policy 0, policy_version 105009 (0.0010) [2023-12-26 16:07:48,961][105692] Updated weights for policy 0, policy_version 105019 (0.0010) [2023-12-26 16:07:49,017][105692] Updated weights for policy 0, policy_version 105029 (0.0011) [2023-12-26 16:07:49,071][105692] Updated weights for policy 0, policy_version 105039 (0.0010) [2023-12-26 16:07:49,277][105620] Updated weights for policy 1, policy_version 105452 (0.0008) [2023-12-26 16:07:49,348][105620] Updated weights for policy 1, policy_version 105462 (0.0009) [2023-12-26 16:07:49,401][105620] Updated weights for policy 1, policy_version 105472 (0.0005) [2023-12-26 16:07:49,839][105692] Updated weights for policy 0, policy_version 105049 (0.0010) [2023-12-26 16:07:49,908][105692] Updated weights for policy 0, policy_version 105059 (0.0008) [2023-12-26 16:07:49,979][105692] Updated weights for policy 0, policy_version 105069 (0.0008) [2023-12-26 16:07:50,116][105620] Updated weights for policy 1, policy_version 105482 (0.0006) [2023-12-26 16:07:50,163][105620] Updated weights for policy 1, policy_version 105492 (0.0008) [2023-12-26 16:07:50,214][105620] Updated weights for policy 1, policy_version 105502 (0.0008) [2023-12-26 16:07:50,262][105620] Updated weights for policy 1, policy_version 105512 (0.0008) [2023-12-26 16:07:50,728][105692] Updated weights for policy 0, policy_version 105079 (0.0010) [2023-12-26 16:07:50,787][105692] Updated weights for policy 0, policy_version 105089 (0.0011) [2023-12-26 16:07:50,845][105692] Updated weights for policy 0, policy_version 105099 (0.0011) [2023-12-26 16:07:50,954][105620] Updated weights for policy 1, policy_version 105522 (0.0008) [2023-12-26 16:07:51,020][105620] Updated weights for policy 1, policy_version 105532 (0.0008) [2023-12-26 16:07:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 53927936. Throughput: 0: 9675.3, 1: 9883.9. Samples: 53918948. Policy #0 lag: (min: 30.0, avg: 36.5, max: 62.0) [2023-12-26 16:07:51,062][104569] Avg episode reward: [(0, '8543.097'), (1, '9266.617')] [2023-12-26 16:07:51,083][105620] Updated weights for policy 1, policy_version 105542 (0.0008) [2023-12-26 16:07:51,631][105692] Updated weights for policy 0, policy_version 105109 (0.0011) [2023-12-26 16:07:51,687][105692] Updated weights for policy 0, policy_version 105119 (0.0010) [2023-12-26 16:07:51,753][105692] Updated weights for policy 0, policy_version 105129 (0.0010) [2023-12-26 16:07:51,818][105620] Updated weights for policy 1, policy_version 105552 (0.0009) [2023-12-26 16:07:51,866][105620] Updated weights for policy 1, policy_version 105562 (0.0008) [2023-12-26 16:07:51,925][105620] Updated weights for policy 1, policy_version 105572 (0.0008) [2023-12-26 16:07:52,420][105692] Updated weights for policy 0, policy_version 105139 (0.0010) [2023-12-26 16:07:52,470][105692] Updated weights for policy 0, policy_version 105149 (0.0008) [2023-12-26 16:07:52,528][105692] Updated weights for policy 0, policy_version 105159 (0.0006) [2023-12-26 16:07:52,789][105620] Updated weights for policy 1, policy_version 105582 (0.0008) [2023-12-26 16:07:52,850][105620] Updated weights for policy 1, policy_version 105592 (0.0009) [2023-12-26 16:07:52,912][105620] Updated weights for policy 1, policy_version 105602 (0.0008) [2023-12-26 16:07:53,236][105692] Updated weights for policy 0, policy_version 105169 (0.0007) [2023-12-26 16:07:53,298][105692] Updated weights for policy 0, policy_version 105179 (0.0010) [2023-12-26 16:07:53,361][105692] Updated weights for policy 0, policy_version 105189 (0.0010) [2023-12-26 16:07:53,427][105692] Updated weights for policy 0, policy_version 105199 (0.0010) [2023-12-26 16:07:53,507][105620] Updated weights for policy 1, policy_version 105612 (0.0009) [2023-12-26 16:07:53,556][105620] Updated weights for policy 1, policy_version 105622 (0.0008) [2023-12-26 16:07:53,604][105620] Updated weights for policy 1, policy_version 105632 (0.0009) [2023-12-26 16:07:54,223][105692] Updated weights for policy 0, policy_version 105209 (0.0009) [2023-12-26 16:07:54,273][105692] Updated weights for policy 0, policy_version 105219 (0.0009) [2023-12-26 16:07:54,312][105620] Updated weights for policy 1, policy_version 105642 (0.0009) [2023-12-26 16:07:54,335][105692] Updated weights for policy 0, policy_version 105229 (0.0008) [2023-12-26 16:07:54,363][105620] Updated weights for policy 1, policy_version 105652 (0.0007) [2023-12-26 16:07:54,423][105620] Updated weights for policy 1, policy_version 105662 (0.0008) [2023-12-26 16:07:54,490][105620] Updated weights for policy 1, policy_version 105672 (0.0009) [2023-12-26 16:07:55,083][105692] Updated weights for policy 0, policy_version 105239 (0.0007) [2023-12-26 16:07:55,145][105692] Updated weights for policy 0, policy_version 105249 (0.0009) [2023-12-26 16:07:55,196][105692] Updated weights for policy 0, policy_version 105259 (0.0009) [2023-12-26 16:07:55,247][105620] Updated weights for policy 1, policy_version 105682 (0.0008) [2023-12-26 16:07:55,305][105620] Updated weights for policy 1, policy_version 105692 (0.0009) [2023-12-26 16:07:55,357][105620] Updated weights for policy 1, policy_version 105702 (0.0008) [2023-12-26 16:07:55,869][105692] Updated weights for policy 0, policy_version 105269 (0.0007) [2023-12-26 16:07:55,934][105692] Updated weights for policy 0, policy_version 105279 (0.0005) [2023-12-26 16:07:55,966][105620] Updated weights for policy 1, policy_version 105712 (0.0006) [2023-12-26 16:07:55,995][105692] Updated weights for policy 0, policy_version 105289 (0.0005) [2023-12-26 16:07:56,023][105620] Updated weights for policy 1, policy_version 105722 (0.0006) [2023-12-26 16:07:56,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 54026240. Throughput: 0: 9601.8, 1: 9812.9. Samples: 54034484. Policy #0 lag: (min: 30.0, avg: 36.5, max: 62.0) [2023-12-26 16:07:56,062][104569] Avg episode reward: [(0, '8535.423'), (1, '9356.348')] [2023-12-26 16:07:56,077][105620] Updated weights for policy 1, policy_version 105732 (0.0005) [2023-12-26 16:07:56,499][105692] Updated weights for policy 0, policy_version 105299 (0.0005) [2023-12-26 16:07:56,554][105692] Updated weights for policy 0, policy_version 105309 (0.0006) [2023-12-26 16:07:56,603][105620] Updated weights for policy 1, policy_version 105742 (0.0005) [2023-12-26 16:07:56,605][105692] Updated weights for policy 0, policy_version 105319 (0.0008) [2023-12-26 16:07:56,661][105620] Updated weights for policy 1, policy_version 105752 (0.0005) [2023-12-26 16:07:56,711][105620] Updated weights for policy 1, policy_version 105762 (0.0005) [2023-12-26 16:07:57,225][105692] Updated weights for policy 0, policy_version 105329 (0.0009) [2023-12-26 16:07:57,296][105692] Updated weights for policy 0, policy_version 105339 (0.0008) [2023-12-26 16:07:57,351][105692] Updated weights for policy 0, policy_version 105349 (0.0008) [2023-12-26 16:07:57,382][105620] Updated weights for policy 1, policy_version 105772 (0.0008) [2023-12-26 16:07:57,398][105692] Updated weights for policy 0, policy_version 105359 (0.0010) [2023-12-26 16:07:57,426][105620] Updated weights for policy 1, policy_version 105782 (0.0010) [2023-12-26 16:07:57,472][105620] Updated weights for policy 1, policy_version 105792 (0.0006) [2023-12-26 16:07:58,007][105692] Updated weights for policy 0, policy_version 105369 (0.0006) [2023-12-26 16:07:58,053][105620] Updated weights for policy 1, policy_version 105802 (0.0006) [2023-12-26 16:07:58,067][105692] Updated weights for policy 0, policy_version 105379 (0.0005) [2023-12-26 16:07:58,117][105620] Updated weights for policy 1, policy_version 105812 (0.0007) [2023-12-26 16:07:58,132][105692] Updated weights for policy 0, policy_version 105389 (0.0010) [2023-12-26 16:07:58,181][105620] Updated weights for policy 1, policy_version 105822 (0.0008) [2023-12-26 16:07:58,245][105620] Updated weights for policy 1, policy_version 105832 (0.0008) [2023-12-26 16:07:58,846][105692] Updated weights for policy 0, policy_version 105399 (0.0009) [2023-12-26 16:07:58,900][105692] Updated weights for policy 0, policy_version 105409 (0.0009) [2023-12-26 16:07:58,949][105692] Updated weights for policy 0, policy_version 105419 (0.0009) [2023-12-26 16:07:58,999][105620] Updated weights for policy 1, policy_version 105842 (0.0008) [2023-12-26 16:07:59,050][105620] Updated weights for policy 1, policy_version 105852 (0.0008) [2023-12-26 16:07:59,100][105620] Updated weights for policy 1, policy_version 105862 (0.0008) [2023-12-26 16:07:59,657][105692] Updated weights for policy 0, policy_version 105429 (0.0009) [2023-12-26 16:07:59,711][105692] Updated weights for policy 0, policy_version 105439 (0.0009) [2023-12-26 16:07:59,780][105692] Updated weights for policy 0, policy_version 105449 (0.0006) [2023-12-26 16:07:59,887][105620] Updated weights for policy 1, policy_version 105872 (0.0009) [2023-12-26 16:07:59,943][105620] Updated weights for policy 1, policy_version 105882 (0.0007) [2023-12-26 16:07:59,995][105620] Updated weights for policy 1, policy_version 105892 (0.0009) [2023-12-26 16:08:00,019][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000006 [2023-12-26 16:08:00,524][105692] Updated weights for policy 0, policy_version 105459 (0.0008) [2023-12-26 16:08:00,583][105692] Updated weights for policy 0, policy_version 105469 (0.0007) [2023-12-26 16:08:00,643][105692] Updated weights for policy 0, policy_version 105479 (0.0006) [2023-12-26 16:08:00,713][105620] Updated weights for policy 1, policy_version 105902 (0.0008) [2023-12-26 16:08:00,759][105620] Updated weights for policy 1, policy_version 105912 (0.0008) [2023-12-26 16:08:00,812][105620] Updated weights for policy 1, policy_version 105922 (0.0009) [2023-12-26 16:08:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 54132736. Throughput: 0: 9740.7, 1: 9855.5. Samples: 54100776. Policy #0 lag: (min: 30.0, avg: 36.5, max: 62.0) [2023-12-26 16:08:01,062][104569] Avg episode reward: [(0, '8900.091'), (1, '9356.204')] [2023-12-26 16:08:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000105488_27009024.pth... [2023-12-26 16:08:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000105928_27123712.pth... [2023-12-26 16:08:01,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000104336_26714112.pth [2023-12-26 16:08:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000104744_26820608.pth [2023-12-26 16:08:01,318][105692] Updated weights for policy 0, policy_version 105489 (0.0006) [2023-12-26 16:08:01,386][105692] Updated weights for policy 0, policy_version 105499 (0.0010) [2023-12-26 16:08:01,446][105692] Updated weights for policy 0, policy_version 105509 (0.0009) [2023-12-26 16:08:01,505][105692] Updated weights for policy 0, policy_version 105519 (0.0010) [2023-12-26 16:08:01,615][105620] Updated weights for policy 1, policy_version 105932 (0.0010) [2023-12-26 16:08:01,686][105620] Updated weights for policy 1, policy_version 105942 (0.0009) [2023-12-26 16:08:01,756][105620] Updated weights for policy 1, policy_version 105952 (0.0008) [2023-12-26 16:08:02,172][105692] Updated weights for policy 0, policy_version 105529 (0.0006) [2023-12-26 16:08:02,235][105692] Updated weights for policy 0, policy_version 105539 (0.0006) [2023-12-26 16:08:02,296][105692] Updated weights for policy 0, policy_version 105549 (0.0006) [2023-12-26 16:08:02,568][105620] Updated weights for policy 1, policy_version 105962 (0.0008) [2023-12-26 16:08:02,621][105620] Updated weights for policy 1, policy_version 105972 (0.0010) [2023-12-26 16:08:02,679][105620] Updated weights for policy 1, policy_version 105982 (0.0010) [2023-12-26 16:08:02,731][105620] Updated weights for policy 1, policy_version 105992 (0.0010) [2023-12-26 16:08:02,806][105692] Updated weights for policy 0, policy_version 105559 (0.0007) [2023-12-26 16:08:02,852][105692] Updated weights for policy 0, policy_version 105569 (0.0005) [2023-12-26 16:08:02,911][105692] Updated weights for policy 0, policy_version 105579 (0.0007) [2023-12-26 16:08:03,481][105620] Updated weights for policy 1, policy_version 106002 (0.0010) [2023-12-26 16:08:03,533][105620] Updated weights for policy 1, policy_version 106012 (0.0008) [2023-12-26 16:08:03,580][105620] Updated weights for policy 1, policy_version 106022 (0.0007) [2023-12-26 16:08:03,583][105692] Updated weights for policy 0, policy_version 105589 (0.0007) [2023-12-26 16:08:03,635][105692] Updated weights for policy 0, policy_version 105599 (0.0008) [2023-12-26 16:08:03,689][105692] Updated weights for policy 0, policy_version 105609 (0.0007) [2023-12-26 16:08:04,317][105620] Updated weights for policy 1, policy_version 106032 (0.0008) [2023-12-26 16:08:04,356][105692] Updated weights for policy 0, policy_version 105619 (0.0005) [2023-12-26 16:08:04,384][105620] Updated weights for policy 1, policy_version 106042 (0.0009) [2023-12-26 16:08:04,417][105692] Updated weights for policy 0, policy_version 105629 (0.0009) [2023-12-26 16:08:04,444][105620] Updated weights for policy 1, policy_version 106052 (0.0009) [2023-12-26 16:08:04,463][105692] Updated weights for policy 0, policy_version 105639 (0.0006) [2023-12-26 16:08:05,202][105620] Updated weights for policy 1, policy_version 106062 (0.0009) [2023-12-26 16:08:05,219][105692] Updated weights for policy 0, policy_version 105649 (0.0008) [2023-12-26 16:08:05,249][105620] Updated weights for policy 1, policy_version 106072 (0.0008) [2023-12-26 16:08:05,268][105692] Updated weights for policy 0, policy_version 105659 (0.0006) [2023-12-26 16:08:05,295][105620] Updated weights for policy 1, policy_version 106082 (0.0006) [2023-12-26 16:08:05,318][105692] Updated weights for policy 0, policy_version 105669 (0.0008) [2023-12-26 16:08:05,363][105692] Updated weights for policy 0, policy_version 105679 (0.0008) [2023-12-26 16:08:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 54222848. Throughput: 0: 9806.8, 1: 9842.7. Samples: 54217536. Policy #0 lag: (min: 30.0, avg: 36.5, max: 62.0) [2023-12-26 16:08:06,062][104569] Avg episode reward: [(0, '9171.210'), (1, '9355.589')] [2023-12-26 16:08:06,075][105620] Updated weights for policy 1, policy_version 106092 (0.0008) [2023-12-26 16:08:06,136][105692] Updated weights for policy 0, policy_version 105689 (0.0007) [2023-12-26 16:08:06,140][105620] Updated weights for policy 1, policy_version 106102 (0.0008) [2023-12-26 16:08:06,198][105692] Updated weights for policy 0, policy_version 105699 (0.0007) [2023-12-26 16:08:06,200][105620] Updated weights for policy 1, policy_version 106112 (0.0008) [2023-12-26 16:08:06,259][105692] Updated weights for policy 0, policy_version 105709 (0.0006) [2023-12-26 16:08:06,940][105692] Updated weights for policy 0, policy_version 105719 (0.0008) [2023-12-26 16:08:06,972][105620] Updated weights for policy 1, policy_version 106122 (0.0008) [2023-12-26 16:08:06,994][105692] Updated weights for policy 0, policy_version 105729 (0.0007) [2023-12-26 16:08:07,031][105620] Updated weights for policy 1, policy_version 106132 (0.0010) [2023-12-26 16:08:07,046][105692] Updated weights for policy 0, policy_version 105739 (0.0006) [2023-12-26 16:08:07,087][105620] Updated weights for policy 1, policy_version 106142 (0.0010) [2023-12-26 16:08:07,153][105620] Updated weights for policy 1, policy_version 106152 (0.0011) [2023-12-26 16:08:07,722][105692] Updated weights for policy 0, policy_version 105749 (0.0006) [2023-12-26 16:08:07,746][105620] Updated weights for policy 1, policy_version 106162 (0.0005) [2023-12-26 16:08:07,770][105692] Updated weights for policy 0, policy_version 105759 (0.0005) [2023-12-26 16:08:07,808][105620] Updated weights for policy 1, policy_version 106172 (0.0007) [2023-12-26 16:08:07,823][105692] Updated weights for policy 0, policy_version 105769 (0.0007) [2023-12-26 16:08:07,862][105620] Updated weights for policy 1, policy_version 106182 (0.0005) [2023-12-26 16:08:08,441][105620] Updated weights for policy 1, policy_version 106192 (0.0008) [2023-12-26 16:08:08,508][105620] Updated weights for policy 1, policy_version 106202 (0.0005) [2023-12-26 16:08:08,518][105692] Updated weights for policy 0, policy_version 105779 (0.0010) [2023-12-26 16:08:08,575][105620] Updated weights for policy 1, policy_version 106212 (0.0006) [2023-12-26 16:08:08,581][105692] Updated weights for policy 0, policy_version 105789 (0.0005) [2023-12-26 16:08:08,648][105692] Updated weights for policy 0, policy_version 105799 (0.0007) [2023-12-26 16:08:09,154][105620] Updated weights for policy 1, policy_version 106222 (0.0009) [2023-12-26 16:08:09,212][105620] Updated weights for policy 1, policy_version 106232 (0.0010) [2023-12-26 16:08:09,284][105620] Updated weights for policy 1, policy_version 106243 (0.0008) [2023-12-26 16:08:09,311][105692] Updated weights for policy 0, policy_version 105809 (0.0010) [2023-12-26 16:08:09,371][105692] Updated weights for policy 0, policy_version 105819 (0.0007) [2023-12-26 16:08:09,441][105692] Updated weights for policy 0, policy_version 105829 (0.0008) [2023-12-26 16:08:09,511][105692] Updated weights for policy 0, policy_version 105839 (0.0007) [2023-12-26 16:08:10,068][105620] Updated weights for policy 1, policy_version 106253 (0.0009) [2023-12-26 16:08:10,126][105620] Updated weights for policy 1, policy_version 106263 (0.0009) [2023-12-26 16:08:10,133][105692] Updated weights for policy 0, policy_version 105849 (0.0008) [2023-12-26 16:08:10,183][105620] Updated weights for policy 1, policy_version 106273 (0.0009) [2023-12-26 16:08:10,190][105692] Updated weights for policy 0, policy_version 105859 (0.0007) [2023-12-26 16:08:10,245][105692] Updated weights for policy 0, policy_version 105869 (0.0007) [2023-12-26 16:08:10,918][105620] Updated weights for policy 1, policy_version 106283 (0.0008) [2023-12-26 16:08:10,959][105692] Updated weights for policy 0, policy_version 105879 (0.0008) [2023-12-26 16:08:10,966][105620] Updated weights for policy 1, policy_version 106293 (0.0007) [2023-12-26 16:08:11,013][105692] Updated weights for policy 0, policy_version 105889 (0.0007) [2023-12-26 16:08:11,025][105620] Updated weights for policy 1, policy_version 106303 (0.0007) [2023-12-26 16:08:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 54321152. Throughput: 0: 9876.3, 1: 9901.6. Samples: 54337044. Policy #0 lag: (min: 30.0, avg: 36.5, max: 62.0) [2023-12-26 16:08:11,063][104569] Avg episode reward: [(0, '9263.319'), (1, '9354.669')] [2023-12-26 16:08:11,076][105692] Updated weights for policy 0, policy_version 105899 (0.0009) [2023-12-26 16:08:11,882][105692] Updated weights for policy 0, policy_version 105909 (0.0008) [2023-12-26 16:08:11,911][105620] Updated weights for policy 1, policy_version 106313 (0.0008) [2023-12-26 16:08:11,942][105692] Updated weights for policy 0, policy_version 105919 (0.0008) [2023-12-26 16:08:11,979][105620] Updated weights for policy 1, policy_version 106323 (0.0010) [2023-12-26 16:08:12,002][105692] Updated weights for policy 0, policy_version 105929 (0.0008) [2023-12-26 16:08:12,043][105620] Updated weights for policy 1, policy_version 106333 (0.0011) [2023-12-26 16:08:12,104][105620] Updated weights for policy 1, policy_version 106343 (0.0010) [2023-12-26 16:08:12,836][105692] Updated weights for policy 0, policy_version 105939 (0.0008) [2023-12-26 16:08:12,873][105620] Updated weights for policy 1, policy_version 106353 (0.0010) [2023-12-26 16:08:12,885][105692] Updated weights for policy 0, policy_version 105949 (0.0008) [2023-12-26 16:08:12,927][105620] Updated weights for policy 1, policy_version 106363 (0.0010) [2023-12-26 16:08:12,931][105692] Updated weights for policy 0, policy_version 105959 (0.0006) [2023-12-26 16:08:12,982][105620] Updated weights for policy 1, policy_version 106373 (0.0010) [2023-12-26 16:08:13,656][105620] Updated weights for policy 1, policy_version 106383 (0.0007) [2023-12-26 16:08:13,721][105620] Updated weights for policy 1, policy_version 106393 (0.0005) [2023-12-26 16:08:13,788][105620] Updated weights for policy 1, policy_version 106403 (0.0005) [2023-12-26 16:08:13,790][105692] Updated weights for policy 0, policy_version 105969 (0.0008) [2023-12-26 16:08:13,848][105692] Updated weights for policy 0, policy_version 105979 (0.0009) [2023-12-26 16:08:13,903][105692] Updated weights for policy 0, policy_version 105990 (0.0008) [2023-12-26 16:08:13,947][105692] Updated weights for policy 0, policy_version 106000 (0.0007) [2023-12-26 16:08:14,342][105620] Updated weights for policy 1, policy_version 106413 (0.0008) [2023-12-26 16:08:14,386][105620] Updated weights for policy 1, policy_version 106423 (0.0010) [2023-12-26 16:08:14,438][105620] Updated weights for policy 1, policy_version 106433 (0.0010) [2023-12-26 16:08:14,680][105692] Updated weights for policy 0, policy_version 106010 (0.0008) [2023-12-26 16:08:14,727][105692] Updated weights for policy 0, policy_version 106020 (0.0007) [2023-12-26 16:08:14,780][105692] Updated weights for policy 0, policy_version 106030 (0.0008) [2023-12-26 16:08:15,203][105620] Updated weights for policy 1, policy_version 106443 (0.0010) [2023-12-26 16:08:15,262][105620] Updated weights for policy 1, policy_version 106453 (0.0010) [2023-12-26 16:08:15,322][105620] Updated weights for policy 1, policy_version 106463 (0.0010) [2023-12-26 16:08:15,583][105692] Updated weights for policy 0, policy_version 106040 (0.0008) [2023-12-26 16:08:15,641][105692] Updated weights for policy 0, policy_version 106050 (0.0008) [2023-12-26 16:08:15,685][105692] Updated weights for policy 0, policy_version 106060 (0.0008) [2023-12-26 16:08:16,061][105620] Updated weights for policy 1, policy_version 106473 (0.0010) [2023-12-26 16:08:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.4, 300 sec: 19577.5). Total num frames: 54419456. Throughput: 0: 9828.4, 1: 9905.8. Samples: 54393072. Policy #0 lag: (min: 30.0, avg: 36.5, max: 62.0) [2023-12-26 16:08:16,062][104569] Avg episode reward: [(0, '9263.321'), (1, '9354.035')] [2023-12-26 16:08:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000106064_27156480.pth... [2023-12-26 16:08:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000104912_26861568.pth [2023-12-26 16:08:16,116][105620] Updated weights for policy 1, policy_version 106483 (0.0010) [2023-12-26 16:08:16,168][105620] Updated weights for policy 1, policy_version 106493 (0.0010) [2023-12-26 16:08:16,221][105620] Updated weights for policy 1, policy_version 106503 (0.0010) [2023-12-26 16:08:16,226][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000106504_27271168.pth... [2023-12-26 16:08:16,231][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000105320_26968064.pth [2023-12-26 16:08:16,468][105692] Updated weights for policy 0, policy_version 106070 (0.0008) [2023-12-26 16:08:16,522][105692] Updated weights for policy 0, policy_version 106080 (0.0008) [2023-12-26 16:08:16,572][105692] Updated weights for policy 0, policy_version 106090 (0.0008) [2023-12-26 16:08:16,945][105620] Updated weights for policy 1, policy_version 106513 (0.0006) [2023-12-26 16:08:17,004][105620] Updated weights for policy 1, policy_version 106523 (0.0009) [2023-12-26 16:08:17,060][105620] Updated weights for policy 1, policy_version 106533 (0.0010) [2023-12-26 16:08:17,401][105692] Updated weights for policy 0, policy_version 106100 (0.0008) [2023-12-26 16:08:17,457][105692] Updated weights for policy 0, policy_version 106110 (0.0010) [2023-12-26 16:08:17,510][105692] Updated weights for policy 0, policy_version 106120 (0.0010) [2023-12-26 16:08:17,652][105620] Updated weights for policy 1, policy_version 106543 (0.0009) [2023-12-26 16:08:17,703][105620] Updated weights for policy 1, policy_version 106553 (0.0010) [2023-12-26 16:08:17,757][105620] Updated weights for policy 1, policy_version 106563 (0.0010) [2023-12-26 16:08:18,309][105692] Updated weights for policy 0, policy_version 106131 (0.0010) [2023-12-26 16:08:18,371][105692] Updated weights for policy 0, policy_version 106141 (0.0008) [2023-12-26 16:08:18,442][105692] Updated weights for policy 0, policy_version 106151 (0.0008) [2023-12-26 16:08:18,472][105620] Updated weights for policy 1, policy_version 106573 (0.0010) [2023-12-26 16:08:18,530][105620] Updated weights for policy 1, policy_version 106583 (0.0010) [2023-12-26 16:08:18,581][105620] Updated weights for policy 1, policy_version 106593 (0.0010) [2023-12-26 16:08:19,205][105692] Updated weights for policy 0, policy_version 106161 (0.0006) [2023-12-26 16:08:19,272][105692] Updated weights for policy 0, policy_version 106171 (0.0008) [2023-12-26 16:08:19,334][105620] Updated weights for policy 1, policy_version 106603 (0.0010) [2023-12-26 16:08:19,336][105692] Updated weights for policy 0, policy_version 106181 (0.0008) [2023-12-26 16:08:19,400][105620] Updated weights for policy 1, policy_version 106613 (0.0010) [2023-12-26 16:08:19,402][105692] Updated weights for policy 0, policy_version 106191 (0.0006) [2023-12-26 16:08:19,463][105620] Updated weights for policy 1, policy_version 106623 (0.0010) [2023-12-26 16:08:20,151][105620] Updated weights for policy 1, policy_version 106633 (0.0010) [2023-12-26 16:08:20,183][105692] Updated weights for policy 0, policy_version 106201 (0.0008) [2023-12-26 16:08:20,221][105620] Updated weights for policy 1, policy_version 106643 (0.0006) [2023-12-26 16:08:20,246][105692] Updated weights for policy 0, policy_version 106211 (0.0008) [2023-12-26 16:08:20,282][105620] Updated weights for policy 1, policy_version 106653 (0.0006) [2023-12-26 16:08:20,307][105692] Updated weights for policy 0, policy_version 106221 (0.0008) [2023-12-26 16:08:20,338][105620] Updated weights for policy 1, policy_version 106663 (0.0008) [2023-12-26 16:08:20,902][105620] Updated weights for policy 1, policy_version 106673 (0.0010) [2023-12-26 16:08:20,965][105620] Updated weights for policy 1, policy_version 106683 (0.0010) [2023-12-26 16:08:21,028][105620] Updated weights for policy 1, policy_version 106693 (0.0011) [2023-12-26 16:08:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 54517760. Throughput: 0: 9771.5, 1: 9879.7. Samples: 54506064. Policy #0 lag: (min: 30.0, avg: 36.5, max: 62.0) [2023-12-26 16:08:21,062][104569] Avg episode reward: [(0, '9355.322'), (1, '9353.735')] [2023-12-26 16:08:21,083][105692] Updated weights for policy 0, policy_version 106231 (0.0009) [2023-12-26 16:08:21,149][105692] Updated weights for policy 0, policy_version 106241 (0.0008) [2023-12-26 16:08:21,209][105692] Updated weights for policy 0, policy_version 106251 (0.0008) [2023-12-26 16:08:21,817][105620] Updated weights for policy 1, policy_version 106703 (0.0008) [2023-12-26 16:08:21,885][105620] Updated weights for policy 1, policy_version 106713 (0.0007) [2023-12-26 16:08:21,951][105620] Updated weights for policy 1, policy_version 106723 (0.0007) [2023-12-26 16:08:21,974][105692] Updated weights for policy 0, policy_version 106261 (0.0008) [2023-12-26 16:08:22,040][105692] Updated weights for policy 0, policy_version 106271 (0.0008) [2023-12-26 16:08:22,103][105692] Updated weights for policy 0, policy_version 106281 (0.0008) [2023-12-26 16:08:22,681][105620] Updated weights for policy 1, policy_version 106733 (0.0008) [2023-12-26 16:08:22,743][105620] Updated weights for policy 1, policy_version 106743 (0.0009) [2023-12-26 16:08:22,799][105692] Updated weights for policy 0, policy_version 106291 (0.0008) [2023-12-26 16:08:22,800][105620] Updated weights for policy 1, policy_version 106753 (0.0008) [2023-12-26 16:08:22,857][105692] Updated weights for policy 0, policy_version 106301 (0.0006) [2023-12-26 16:08:22,923][105692] Updated weights for policy 0, policy_version 106311 (0.0006) [2023-12-26 16:08:23,580][105692] Updated weights for policy 0, policy_version 106321 (0.0006) [2023-12-26 16:08:23,590][105620] Updated weights for policy 1, policy_version 106763 (0.0007) [2023-12-26 16:08:23,628][105692] Updated weights for policy 0, policy_version 106331 (0.0009) [2023-12-26 16:08:23,656][105620] Updated weights for policy 1, policy_version 106773 (0.0007) [2023-12-26 16:08:23,682][105692] Updated weights for policy 0, policy_version 106341 (0.0009) [2023-12-26 16:08:23,718][105620] Updated weights for policy 1, policy_version 106783 (0.0007) [2023-12-26 16:08:23,728][105692] Updated weights for policy 0, policy_version 106351 (0.0007) [2023-12-26 16:08:24,306][105620] Updated weights for policy 1, policy_version 106793 (0.0008) [2023-12-26 16:08:24,374][105620] Updated weights for policy 1, policy_version 106803 (0.0009) [2023-12-26 16:08:24,428][105620] Updated weights for policy 1, policy_version 106813 (0.0007) [2023-12-26 16:08:24,481][105692] Updated weights for policy 0, policy_version 106361 (0.0008) [2023-12-26 16:08:24,482][105620] Updated weights for policy 1, policy_version 106823 (0.0005) [2023-12-26 16:08:24,545][105692] Updated weights for policy 0, policy_version 106371 (0.0009) [2023-12-26 16:08:24,602][105692] Updated weights for policy 0, policy_version 106381 (0.0009) [2023-12-26 16:08:25,121][105620] Updated weights for policy 1, policy_version 106833 (0.0010) [2023-12-26 16:08:25,169][105620] Updated weights for policy 1, policy_version 106843 (0.0010) [2023-12-26 16:08:25,214][105620] Updated weights for policy 1, policy_version 106853 (0.0010) [2023-12-26 16:08:25,398][105692] Updated weights for policy 0, policy_version 106391 (0.0009) [2023-12-26 16:08:25,447][105692] Updated weights for policy 0, policy_version 106401 (0.0008) [2023-12-26 16:08:25,498][105692] Updated weights for policy 0, policy_version 106411 (0.0006) [2023-12-26 16:08:25,877][105620] Updated weights for policy 1, policy_version 106863 (0.0007) [2023-12-26 16:08:25,938][105620] Updated weights for policy 1, policy_version 106873 (0.0006) [2023-12-26 16:08:25,997][105620] Updated weights for policy 1, policy_version 106883 (0.0005) [2023-12-26 16:08:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 54616064. Throughput: 0: 9709.8, 1: 9904.3. Samples: 54622712. Policy #0 lag: (min: 30.0, avg: 36.5, max: 62.0) [2023-12-26 16:08:26,063][104569] Avg episode reward: [(0, '6840.920'), (1, '9353.271')] [2023-12-26 16:08:26,118][105692] Updated weights for policy 0, policy_version 106421 (0.0008) [2023-12-26 16:08:26,170][105692] Updated weights for policy 0, policy_version 106431 (0.0010) [2023-12-26 16:08:26,237][105692] Updated weights for policy 0, policy_version 106441 (0.0005) [2023-12-26 16:08:26,620][105620] Updated weights for policy 1, policy_version 106893 (0.0007) [2023-12-26 16:08:26,686][105620] Updated weights for policy 1, policy_version 106903 (0.0008) [2023-12-26 16:08:26,747][105620] Updated weights for policy 1, policy_version 106913 (0.0008) [2023-12-26 16:08:26,934][105692] Updated weights for policy 0, policy_version 106451 (0.0007) [2023-12-26 16:08:26,982][105692] Updated weights for policy 0, policy_version 106461 (0.0010) [2023-12-26 16:08:27,030][105692] Updated weights for policy 0, policy_version 106471 (0.0010) [2023-12-26 16:08:27,353][105620] Updated weights for policy 1, policy_version 106923 (0.0006) [2023-12-26 16:08:27,410][105620] Updated weights for policy 1, policy_version 106933 (0.0005) [2023-12-26 16:08:27,461][105620] Updated weights for policy 1, policy_version 106943 (0.0007) [2023-12-26 16:08:27,705][105692] Updated weights for policy 0, policy_version 106481 (0.0010) [2023-12-26 16:08:27,764][105692] Updated weights for policy 0, policy_version 106491 (0.0011) [2023-12-26 16:08:27,815][105692] Updated weights for policy 0, policy_version 106501 (0.0010) [2023-12-26 16:08:27,873][105692] Updated weights for policy 0, policy_version 106511 (0.0006) [2023-12-26 16:08:28,078][105620] Updated weights for policy 1, policy_version 106954 (0.0009) [2023-12-26 16:08:28,136][105620] Updated weights for policy 1, policy_version 106964 (0.0008) [2023-12-26 16:08:28,189][105620] Updated weights for policy 1, policy_version 106975 (0.0010) [2023-12-26 16:08:28,475][105692] Updated weights for policy 0, policy_version 106521 (0.0010) [2023-12-26 16:08:28,538][105692] Updated weights for policy 0, policy_version 106531 (0.0011) [2023-12-26 16:08:28,607][105692] Updated weights for policy 0, policy_version 106541 (0.0010) [2023-12-26 16:08:28,800][105620] Updated weights for policy 1, policy_version 106985 (0.0009) [2023-12-26 16:08:28,850][105620] Updated weights for policy 1, policy_version 106995 (0.0006) [2023-12-26 16:08:28,912][105620] Updated weights for policy 1, policy_version 107005 (0.0008) [2023-12-26 16:08:28,971][105620] Updated weights for policy 1, policy_version 107015 (0.0008) [2023-12-26 16:08:29,368][105692] Updated weights for policy 0, policy_version 106551 (0.0011) [2023-12-26 16:08:29,430][105692] Updated weights for policy 0, policy_version 106561 (0.0011) [2023-12-26 16:08:29,478][105692] Updated weights for policy 0, policy_version 106571 (0.0010) [2023-12-26 16:08:29,659][105620] Updated weights for policy 1, policy_version 107025 (0.0008) [2023-12-26 16:08:29,706][105620] Updated weights for policy 1, policy_version 107035 (0.0008) [2023-12-26 16:08:29,761][105620] Updated weights for policy 1, policy_version 107045 (0.0008) [2023-12-26 16:08:30,235][105692] Updated weights for policy 0, policy_version 106581 (0.0011) [2023-12-26 16:08:30,294][105692] Updated weights for policy 0, policy_version 106591 (0.0011) [2023-12-26 16:08:30,363][105692] Updated weights for policy 0, policy_version 106601 (0.0011) [2023-12-26 16:08:30,439][105620] Updated weights for policy 1, policy_version 107055 (0.0006) [2023-12-26 16:08:30,497][105620] Updated weights for policy 1, policy_version 107065 (0.0005) [2023-12-26 16:08:30,568][105620] Updated weights for policy 1, policy_version 107075 (0.0005) [2023-12-26 16:08:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 54714368. Throughput: 0: 9799.5, 1: 9987.9. Samples: 54687952. Policy #0 lag: (min: 31.0, avg: 53.6, max: 63.0) [2023-12-26 16:08:31,062][104569] Avg episode reward: [(0, '7081.233'), (1, '9352.990')] [2023-12-26 16:08:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000106608_27295744.pth... [2023-12-26 16:08:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000107080_27418624.pth... [2023-12-26 16:08:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000105488_27009024.pth [2023-12-26 16:08:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000105928_27123712.pth [2023-12-26 16:08:31,116][105692] Updated weights for policy 0, policy_version 106611 (0.0011) [2023-12-26 16:08:31,187][105692] Updated weights for policy 0, policy_version 106621 (0.0011) [2023-12-26 16:08:31,190][105620] Updated weights for policy 1, policy_version 107085 (0.0006) [2023-12-26 16:08:31,243][105692] Updated weights for policy 0, policy_version 106631 (0.0010) [2023-12-26 16:08:31,261][105620] Updated weights for policy 1, policy_version 107095 (0.0006) [2023-12-26 16:08:31,318][105620] Updated weights for policy 1, policy_version 107105 (0.0008) [2023-12-26 16:08:31,987][105620] Updated weights for policy 1, policy_version 107115 (0.0008) [2023-12-26 16:08:32,033][105692] Updated weights for policy 0, policy_version 106641 (0.0011) [2023-12-26 16:08:32,045][105620] Updated weights for policy 1, policy_version 107125 (0.0005) [2023-12-26 16:08:32,095][105692] Updated weights for policy 0, policy_version 106651 (0.0011) [2023-12-26 16:08:32,104][105620] Updated weights for policy 1, policy_version 107135 (0.0005) [2023-12-26 16:08:32,165][105692] Updated weights for policy 0, policy_version 106661 (0.0011) [2023-12-26 16:08:32,231][105692] Updated weights for policy 0, policy_version 106671 (0.0011) [2023-12-26 16:08:32,722][105620] Updated weights for policy 1, policy_version 107145 (0.0006) [2023-12-26 16:08:32,783][105620] Updated weights for policy 1, policy_version 107155 (0.0008) [2023-12-26 16:08:32,842][105620] Updated weights for policy 1, policy_version 107165 (0.0011) [2023-12-26 16:08:32,890][105620] Updated weights for policy 1, policy_version 107175 (0.0010) [2023-12-26 16:08:32,977][105692] Updated weights for policy 0, policy_version 106681 (0.0011) [2023-12-26 16:08:33,025][105692] Updated weights for policy 0, policy_version 106691 (0.0010) [2023-12-26 16:08:33,076][105692] Updated weights for policy 0, policy_version 106701 (0.0010) [2023-12-26 16:08:33,620][105620] Updated weights for policy 1, policy_version 107185 (0.0010) [2023-12-26 16:08:33,675][105620] Updated weights for policy 1, policy_version 107195 (0.0010) [2023-12-26 16:08:33,722][105620] Updated weights for policy 1, policy_version 107205 (0.0010) [2023-12-26 16:08:33,825][105692] Updated weights for policy 0, policy_version 106711 (0.0010) [2023-12-26 16:08:33,885][105692] Updated weights for policy 0, policy_version 106721 (0.0010) [2023-12-26 16:08:33,945][105692] Updated weights for policy 0, policy_version 106731 (0.0010) [2023-12-26 16:08:34,466][105620] Updated weights for policy 1, policy_version 107215 (0.0010) [2023-12-26 16:08:34,528][105620] Updated weights for policy 1, policy_version 107225 (0.0010) [2023-12-26 16:08:34,587][105620] Updated weights for policy 1, policy_version 107235 (0.0010) [2023-12-26 16:08:34,683][105692] Updated weights for policy 0, policy_version 106741 (0.0010) [2023-12-26 16:08:34,742][105692] Updated weights for policy 0, policy_version 106751 (0.0011) [2023-12-26 16:08:34,800][105692] Updated weights for policy 0, policy_version 106761 (0.0010) [2023-12-26 16:08:35,227][105620] Updated weights for policy 1, policy_version 107245 (0.0010) [2023-12-26 16:08:35,275][105620] Updated weights for policy 1, policy_version 107255 (0.0010) [2023-12-26 16:08:35,322][105620] Updated weights for policy 1, policy_version 107265 (0.0010) [2023-12-26 16:08:35,541][105692] Updated weights for policy 0, policy_version 106771 (0.0011) [2023-12-26 16:08:35,592][105692] Updated weights for policy 0, policy_version 106781 (0.0010) [2023-12-26 16:08:35,649][105692] Updated weights for policy 0, policy_version 106791 (0.0008) [2023-12-26 16:08:35,980][105620] Updated weights for policy 1, policy_version 107275 (0.0011) [2023-12-26 16:08:36,043][105620] Updated weights for policy 1, policy_version 107285 (0.0009) [2023-12-26 16:08:36,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 54812672. Throughput: 0: 9694.4, 1: 9983.4. Samples: 54804448. Policy #0 lag: (min: 31.0, avg: 53.6, max: 63.0) [2023-12-26 16:08:36,062][104569] Avg episode reward: [(0, '7608.018'), (1, '9352.374')] [2023-12-26 16:08:36,091][105620] Updated weights for policy 1, policy_version 107295 (0.0005) [2023-12-26 16:08:36,364][105692] Updated weights for policy 0, policy_version 106801 (0.0007) [2023-12-26 16:08:36,417][105692] Updated weights for policy 0, policy_version 106811 (0.0011) [2023-12-26 16:08:36,480][105692] Updated weights for policy 0, policy_version 106821 (0.0011) [2023-12-26 16:08:36,546][105692] Updated weights for policy 0, policy_version 106831 (0.0011) [2023-12-26 16:08:36,692][105620] Updated weights for policy 1, policy_version 107305 (0.0008) [2023-12-26 16:08:36,756][105620] Updated weights for policy 1, policy_version 107315 (0.0011) [2023-12-26 16:08:36,816][105620] Updated weights for policy 1, policy_version 107325 (0.0011) [2023-12-26 16:08:36,878][105620] Updated weights for policy 1, policy_version 107335 (0.0011) [2023-12-26 16:08:37,288][105692] Updated weights for policy 0, policy_version 106841 (0.0011) [2023-12-26 16:08:37,350][105692] Updated weights for policy 0, policy_version 106851 (0.0010) [2023-12-26 16:08:37,405][105692] Updated weights for policy 0, policy_version 106861 (0.0010) [2023-12-26 16:08:37,542][105620] Updated weights for policy 1, policy_version 107345 (0.0006) [2023-12-26 16:08:37,588][105620] Updated weights for policy 1, policy_version 107355 (0.0005) [2023-12-26 16:08:37,638][105620] Updated weights for policy 1, policy_version 107365 (0.0009) [2023-12-26 16:08:37,980][105692] Updated weights for policy 0, policy_version 106871 (0.0007) [2023-12-26 16:08:38,049][105692] Updated weights for policy 0, policy_version 106881 (0.0010) [2023-12-26 16:08:38,109][105692] Updated weights for policy 0, policy_version 106891 (0.0010) [2023-12-26 16:08:38,376][105620] Updated weights for policy 1, policy_version 107375 (0.0009) [2023-12-26 16:08:38,438][105620] Updated weights for policy 1, policy_version 107385 (0.0008) [2023-12-26 16:08:38,506][105620] Updated weights for policy 1, policy_version 107395 (0.0006) [2023-12-26 16:08:38,812][105692] Updated weights for policy 0, policy_version 106901 (0.0010) [2023-12-26 16:08:38,867][105692] Updated weights for policy 0, policy_version 106911 (0.0010) [2023-12-26 16:08:38,922][105692] Updated weights for policy 0, policy_version 106921 (0.0010) [2023-12-26 16:08:39,151][105620] Updated weights for policy 1, policy_version 107405 (0.0009) [2023-12-26 16:08:39,218][105620] Updated weights for policy 1, policy_version 107415 (0.0011) [2023-12-26 16:08:39,286][105620] Updated weights for policy 1, policy_version 107425 (0.0010) [2023-12-26 16:08:39,719][105692] Updated weights for policy 0, policy_version 106931 (0.0009) [2023-12-26 16:08:39,778][105692] Updated weights for policy 0, policy_version 106941 (0.0006) [2023-12-26 16:08:39,841][105692] Updated weights for policy 0, policy_version 106951 (0.0007) [2023-12-26 16:08:40,079][105620] Updated weights for policy 1, policy_version 107435 (0.0011) [2023-12-26 16:08:40,143][105620] Updated weights for policy 1, policy_version 107445 (0.0011) [2023-12-26 16:08:40,210][105620] Updated weights for policy 1, policy_version 107455 (0.0011) [2023-12-26 16:08:40,522][105692] Updated weights for policy 0, policy_version 106961 (0.0007) [2023-12-26 16:08:40,590][105692] Updated weights for policy 0, policy_version 106971 (0.0005) [2023-12-26 16:08:40,646][105692] Updated weights for policy 0, policy_version 106981 (0.0005) [2023-12-26 16:08:40,704][105692] Updated weights for policy 0, policy_version 106991 (0.0005) [2023-12-26 16:08:40,920][105620] Updated weights for policy 1, policy_version 107465 (0.0010) [2023-12-26 16:08:40,986][105620] Updated weights for policy 1, policy_version 107475 (0.0006) [2023-12-26 16:08:41,052][105620] Updated weights for policy 1, policy_version 107485 (0.0007) [2023-12-26 16:08:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 54910976. Throughput: 0: 9746.3, 1: 10035.4. Samples: 54924660. Policy #0 lag: (min: 31.0, avg: 53.6, max: 63.0) [2023-12-26 16:08:41,063][104569] Avg episode reward: [(0, '8649.261'), (1, '9351.823')] [2023-12-26 16:08:41,118][105620] Updated weights for policy 1, policy_version 107495 (0.0006) [2023-12-26 16:08:41,390][105692] Updated weights for policy 0, policy_version 107001 (0.0011) [2023-12-26 16:08:41,454][105692] Updated weights for policy 0, policy_version 107011 (0.0009) [2023-12-26 16:08:41,514][105692] Updated weights for policy 0, policy_version 107021 (0.0008) [2023-12-26 16:08:41,805][105620] Updated weights for policy 1, policy_version 107505 (0.0008) [2023-12-26 16:08:41,863][105620] Updated weights for policy 1, policy_version 107515 (0.0005) [2023-12-26 16:08:41,919][105620] Updated weights for policy 1, policy_version 107525 (0.0006) [2023-12-26 16:08:42,318][105692] Updated weights for policy 0, policy_version 107031 (0.0008) [2023-12-26 16:08:42,387][105692] Updated weights for policy 0, policy_version 107041 (0.0008) [2023-12-26 16:08:42,454][105692] Updated weights for policy 0, policy_version 107051 (0.0008) [2023-12-26 16:08:42,627][105620] Updated weights for policy 1, policy_version 107535 (0.0009) [2023-12-26 16:08:42,686][105620] Updated weights for policy 1, policy_version 107545 (0.0011) [2023-12-26 16:08:42,746][105620] Updated weights for policy 1, policy_version 107555 (0.0009) [2023-12-26 16:08:43,259][105692] Updated weights for policy 0, policy_version 107061 (0.0008) [2023-12-26 16:08:43,311][105620] Updated weights for policy 1, policy_version 107565 (0.0005) [2023-12-26 16:08:43,326][105692] Updated weights for policy 0, policy_version 107071 (0.0009) [2023-12-26 16:08:43,371][105620] Updated weights for policy 1, policy_version 107575 (0.0008) [2023-12-26 16:08:43,377][105692] Updated weights for policy 0, policy_version 107081 (0.0006) [2023-12-26 16:08:43,434][105620] Updated weights for policy 1, policy_version 107585 (0.0006) [2023-12-26 16:08:43,985][105620] Updated weights for policy 1, policy_version 107595 (0.0005) [2023-12-26 16:08:44,043][105620] Updated weights for policy 1, policy_version 107605 (0.0005) [2023-12-26 16:08:44,112][105620] Updated weights for policy 1, policy_version 107615 (0.0006) [2023-12-26 16:08:44,182][105692] Updated weights for policy 0, policy_version 107091 (0.0009) [2023-12-26 16:08:44,244][105692] Updated weights for policy 0, policy_version 107101 (0.0009) [2023-12-26 16:08:44,302][105692] Updated weights for policy 0, policy_version 107111 (0.0009) [2023-12-26 16:08:44,799][105620] Updated weights for policy 1, policy_version 107625 (0.0009) [2023-12-26 16:08:44,851][105620] Updated weights for policy 1, policy_version 107635 (0.0009) [2023-12-26 16:08:44,901][105620] Updated weights for policy 1, policy_version 107645 (0.0010) [2023-12-26 16:08:44,947][105620] Updated weights for policy 1, policy_version 107655 (0.0011) [2023-12-26 16:08:45,093][105692] Updated weights for policy 0, policy_version 107121 (0.0009) [2023-12-26 16:08:45,157][105692] Updated weights for policy 0, policy_version 107131 (0.0008) [2023-12-26 16:08:45,220][105692] Updated weights for policy 0, policy_version 107141 (0.0008) [2023-12-26 16:08:45,281][105692] Updated weights for policy 0, policy_version 107151 (0.0008) [2023-12-26 16:08:45,746][105620] Updated weights for policy 1, policy_version 107665 (0.0010) [2023-12-26 16:08:45,798][105620] Updated weights for policy 1, policy_version 107675 (0.0010) [2023-12-26 16:08:45,863][105620] Updated weights for policy 1, policy_version 107685 (0.0010) [2023-12-26 16:08:46,003][105692] Updated weights for policy 0, policy_version 107161 (0.0009) [2023-12-26 16:08:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 55009280. Throughput: 0: 9591.5, 1: 10017.4. Samples: 54983180. Policy #0 lag: (min: 31.0, avg: 53.6, max: 63.0) [2023-12-26 16:08:46,063][104569] Avg episode reward: [(0, '8395.682'), (1, '9267.154')] [2023-12-26 16:08:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000107688_27574272.pth... [2023-12-26 16:08:46,071][105692] Updated weights for policy 0, policy_version 107171 (0.0009) [2023-12-26 16:08:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000106504_27271168.pth [2023-12-26 16:08:46,092][105585] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000000 [2023-12-26 16:08:46,094][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000107176_27443200.pth... [2023-12-26 16:08:46,098][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000106064_27156480.pth [2023-12-26 16:08:46,588][105620] Updated weights for policy 1, policy_version 107695 (0.0010) [2023-12-26 16:08:46,636][105620] Updated weights for policy 1, policy_version 107705 (0.0009) [2023-12-26 16:08:46,697][105620] Updated weights for policy 1, policy_version 107715 (0.0009) [2023-12-26 16:08:46,849][105692] Updated weights for policy 0, policy_version 107181 (0.0009) [2023-12-26 16:08:46,900][105692] Updated weights for policy 0, policy_version 107191 (0.0009) [2023-12-26 16:08:46,953][105692] Updated weights for policy 0, policy_version 107201 (0.0008) [2023-12-26 16:08:47,437][105620] Updated weights for policy 1, policy_version 107725 (0.0008) [2023-12-26 16:08:47,494][105620] Updated weights for policy 1, policy_version 107735 (0.0009) [2023-12-26 16:08:47,553][105620] Updated weights for policy 1, policy_version 107745 (0.0009) [2023-12-26 16:08:47,684][105692] Updated weights for policy 0, policy_version 107211 (0.0007) [2023-12-26 16:08:47,747][105692] Updated weights for policy 0, policy_version 107221 (0.0009) [2023-12-26 16:08:47,794][105692] Updated weights for policy 0, policy_version 107231 (0.0009) [2023-12-26 16:08:48,284][105620] Updated weights for policy 1, policy_version 107755 (0.0008) [2023-12-26 16:08:48,350][105620] Updated weights for policy 1, policy_version 107765 (0.0009) [2023-12-26 16:08:48,400][105620] Updated weights for policy 1, policy_version 107775 (0.0008) [2023-12-26 16:08:48,553][105692] Updated weights for policy 0, policy_version 107241 (0.0009) [2023-12-26 16:08:48,616][105692] Updated weights for policy 0, policy_version 107251 (0.0009) [2023-12-26 16:08:48,679][105692] Updated weights for policy 0, policy_version 107261 (0.0009) [2023-12-26 16:08:48,746][105692] Updated weights for policy 0, policy_version 107271 (0.0009) [2023-12-26 16:08:49,167][105620] Updated weights for policy 1, policy_version 107785 (0.0009) [2023-12-26 16:08:49,235][105620] Updated weights for policy 1, policy_version 107795 (0.0008) [2023-12-26 16:08:49,299][105620] Updated weights for policy 1, policy_version 107805 (0.0009) [2023-12-26 16:08:49,388][105620] Updated weights for policy 1, policy_version 107815 (0.0008) [2023-12-26 16:08:49,518][105692] Updated weights for policy 0, policy_version 107281 (0.0009) [2023-12-26 16:08:49,582][105692] Updated weights for policy 0, policy_version 107291 (0.0010) [2023-12-26 16:08:49,647][105692] Updated weights for policy 0, policy_version 107301 (0.0008) [2023-12-26 16:08:50,141][105620] Updated weights for policy 1, policy_version 107825 (0.0010) [2023-12-26 16:08:50,197][105620] Updated weights for policy 1, policy_version 107835 (0.0010) [2023-12-26 16:08:50,245][105620] Updated weights for policy 1, policy_version 107845 (0.0009) [2023-12-26 16:08:50,393][105692] Updated weights for policy 0, policy_version 107311 (0.0008) [2023-12-26 16:08:50,450][105692] Updated weights for policy 0, policy_version 107321 (0.0005) [2023-12-26 16:08:50,506][105692] Updated weights for policy 0, policy_version 107331 (0.0009) [2023-12-26 16:08:51,003][105620] Updated weights for policy 1, policy_version 107855 (0.0009) [2023-12-26 16:08:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 55099392. Throughput: 0: 9457.0, 1: 10053.2. Samples: 55095492. Policy #0 lag: (min: 31.0, avg: 53.6, max: 63.0) [2023-12-26 16:08:51,063][104569] Avg episode reward: [(0, '8650.822'), (1, '9093.658')] [2023-12-26 16:08:51,075][105620] Updated weights for policy 1, policy_version 107865 (0.0008) [2023-12-26 16:08:51,148][105620] Updated weights for policy 1, policy_version 107875 (0.0008) [2023-12-26 16:08:51,279][105692] Updated weights for policy 0, policy_version 107341 (0.0009) [2023-12-26 16:08:51,343][105692] Updated weights for policy 0, policy_version 107351 (0.0008) [2023-12-26 16:08:51,409][105692] Updated weights for policy 0, policy_version 107361 (0.0008) [2023-12-26 16:08:51,876][105620] Updated weights for policy 1, policy_version 107885 (0.0008) [2023-12-26 16:08:51,939][105620] Updated weights for policy 1, policy_version 107895 (0.0009) [2023-12-26 16:08:52,003][105620] Updated weights for policy 1, policy_version 107905 (0.0009) [2023-12-26 16:08:52,162][105692] Updated weights for policy 0, policy_version 107371 (0.0008) [2023-12-26 16:08:52,213][105692] Updated weights for policy 0, policy_version 107381 (0.0009) [2023-12-26 16:08:52,268][105692] Updated weights for policy 0, policy_version 107391 (0.0009) [2023-12-26 16:08:52,722][105620] Updated weights for policy 1, policy_version 107915 (0.0009) [2023-12-26 16:08:52,773][105620] Updated weights for policy 1, policy_version 107925 (0.0009) [2023-12-26 16:08:52,830][105620] Updated weights for policy 1, policy_version 107935 (0.0008) [2023-12-26 16:08:53,088][105692] Updated weights for policy 0, policy_version 107401 (0.0009) [2023-12-26 16:08:53,141][105692] Updated weights for policy 0, policy_version 107411 (0.0008) [2023-12-26 16:08:53,188][105692] Updated weights for policy 0, policy_version 107421 (0.0008) [2023-12-26 16:08:53,233][105692] Updated weights for policy 0, policy_version 107431 (0.0008) [2023-12-26 16:08:53,517][105620] Updated weights for policy 1, policy_version 107945 (0.0010) [2023-12-26 16:08:53,587][105620] Updated weights for policy 1, policy_version 107955 (0.0005) [2023-12-26 16:08:53,654][105620] Updated weights for policy 1, policy_version 107965 (0.0006) [2023-12-26 16:08:53,720][105620] Updated weights for policy 1, policy_version 107975 (0.0010) [2023-12-26 16:08:54,068][105692] Updated weights for policy 0, policy_version 107441 (0.0008) [2023-12-26 16:08:54,130][105692] Updated weights for policy 0, policy_version 107451 (0.0008) [2023-12-26 16:08:54,182][105692] Updated weights for policy 0, policy_version 107461 (0.0008) [2023-12-26 16:08:54,342][105620] Updated weights for policy 1, policy_version 107985 (0.0006) [2023-12-26 16:08:54,398][105620] Updated weights for policy 1, policy_version 107995 (0.0005) [2023-12-26 16:08:54,467][105620] Updated weights for policy 1, policy_version 108005 (0.0005) [2023-12-26 16:08:54,983][105692] Updated weights for policy 0, policy_version 107471 (0.0008) [2023-12-26 16:08:55,041][105692] Updated weights for policy 0, policy_version 107481 (0.0008) [2023-12-26 16:08:55,093][105692] Updated weights for policy 0, policy_version 107491 (0.0008) [2023-12-26 16:08:55,140][105620] Updated weights for policy 1, policy_version 108015 (0.0009) [2023-12-26 16:08:55,185][105620] Updated weights for policy 1, policy_version 108025 (0.0010) [2023-12-26 16:08:55,233][105620] Updated weights for policy 1, policy_version 108035 (0.0010) [2023-12-26 16:08:55,846][105692] Updated weights for policy 0, policy_version 107501 (0.0008) [2023-12-26 16:08:55,896][105692] Updated weights for policy 0, policy_version 107511 (0.0009) [2023-12-26 16:08:55,943][105692] Updated weights for policy 0, policy_version 107521 (0.0009) [2023-12-26 16:08:55,992][105620] Updated weights for policy 1, policy_version 108045 (0.0009) [2023-12-26 16:08:56,043][105620] Updated weights for policy 1, policy_version 108055 (0.0008) [2023-12-26 16:08:56,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 55197696. Throughput: 0: 9332.1, 1: 10007.7. Samples: 55207336. Policy #0 lag: (min: 31.0, avg: 53.6, max: 63.0) [2023-12-26 16:08:56,062][104569] Avg episode reward: [(0, '8553.775'), (1, '8917.698')] [2023-12-26 16:08:56,090][105620] Updated weights for policy 1, policy_version 108065 (0.0009) [2023-12-26 16:08:56,724][105692] Updated weights for policy 0, policy_version 107531 (0.0009) [2023-12-26 16:08:56,769][105692] Updated weights for policy 0, policy_version 107541 (0.0008) [2023-12-26 16:08:56,815][105692] Updated weights for policy 0, policy_version 107551 (0.0008) [2023-12-26 16:08:56,856][105620] Updated weights for policy 1, policy_version 108075 (0.0008) [2023-12-26 16:08:56,902][105620] Updated weights for policy 1, policy_version 108085 (0.0008) [2023-12-26 16:08:56,952][105620] Updated weights for policy 1, policy_version 108095 (0.0009) [2023-12-26 16:08:57,542][105692] Updated weights for policy 0, policy_version 107561 (0.0008) [2023-12-26 16:08:57,597][105692] Updated weights for policy 0, policy_version 107571 (0.0009) [2023-12-26 16:08:57,646][105692] Updated weights for policy 0, policy_version 107581 (0.0009) [2023-12-26 16:08:57,695][105620] Updated weights for policy 1, policy_version 108105 (0.0008) [2023-12-26 16:08:57,696][105692] Updated weights for policy 0, policy_version 107591 (0.0009) [2023-12-26 16:08:57,747][105620] Updated weights for policy 1, policy_version 108115 (0.0009) [2023-12-26 16:08:57,810][105620] Updated weights for policy 1, policy_version 108125 (0.0010) [2023-12-26 16:08:57,868][105620] Updated weights for policy 1, policy_version 108135 (0.0009) [2023-12-26 16:08:58,496][105692] Updated weights for policy 0, policy_version 107601 (0.0008) [2023-12-26 16:08:58,546][105692] Updated weights for policy 0, policy_version 107611 (0.0008) [2023-12-26 16:08:58,605][105692] Updated weights for policy 0, policy_version 107621 (0.0007) [2023-12-26 16:08:58,739][105620] Updated weights for policy 1, policy_version 108145 (0.0008) [2023-12-26 16:08:58,804][105620] Updated weights for policy 1, policy_version 108155 (0.0009) [2023-12-26 16:08:58,866][105620] Updated weights for policy 1, policy_version 108165 (0.0009) [2023-12-26 16:08:59,327][105692] Updated weights for policy 0, policy_version 107631 (0.0008) [2023-12-26 16:08:59,383][105692] Updated weights for policy 0, policy_version 107641 (0.0009) [2023-12-26 16:08:59,446][105692] Updated weights for policy 0, policy_version 107651 (0.0007) [2023-12-26 16:08:59,634][105620] Updated weights for policy 1, policy_version 108175 (0.0010) [2023-12-26 16:08:59,693][105620] Updated weights for policy 1, policy_version 108185 (0.0009) [2023-12-26 16:08:59,755][105620] Updated weights for policy 1, policy_version 108195 (0.0005) [2023-12-26 16:09:00,145][105692] Updated weights for policy 0, policy_version 107661 (0.0008) [2023-12-26 16:09:00,204][105692] Updated weights for policy 0, policy_version 107671 (0.0007) [2023-12-26 16:09:00,263][105692] Updated weights for policy 0, policy_version 107681 (0.0008) [2023-12-26 16:09:00,363][105620] Updated weights for policy 1, policy_version 108205 (0.0007) [2023-12-26 16:09:00,424][105620] Updated weights for policy 1, policy_version 108215 (0.0009) [2023-12-26 16:09:00,486][105620] Updated weights for policy 1, policy_version 108225 (0.0006) [2023-12-26 16:09:00,935][105692] Updated weights for policy 0, policy_version 107691 (0.0006) [2023-12-26 16:09:00,992][105692] Updated weights for policy 0, policy_version 107701 (0.0006) [2023-12-26 16:09:01,053][105692] Updated weights for policy 0, policy_version 107711 (0.0006) [2023-12-26 16:09:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 55287808. Throughput: 0: 9348.5, 1: 9965.6. Samples: 55262208. Policy #0 lag: (min: 31.0, avg: 53.6, max: 63.0) [2023-12-26 16:09:01,062][104569] Avg episode reward: [(0, '8540.890'), (1, '9090.827')] [2023-12-26 16:09:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000108232_27713536.pth... [2023-12-26 16:09:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000107080_27418624.pth [2023-12-26 16:09:01,111][105620] Updated weights for policy 1, policy_version 108235 (0.0010) [2023-12-26 16:09:01,111][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000107720_27582464.pth... [2023-12-26 16:09:01,116][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000106608_27295744.pth [2023-12-26 16:09:01,180][105620] Updated weights for policy 1, policy_version 108245 (0.0009) [2023-12-26 16:09:01,242][105620] Updated weights for policy 1, policy_version 108255 (0.0010) [2023-12-26 16:09:01,645][105692] Updated weights for policy 0, policy_version 107721 (0.0010) [2023-12-26 16:09:01,712][105692] Updated weights for policy 0, policy_version 107731 (0.0009) [2023-12-26 16:09:01,777][105692] Updated weights for policy 0, policy_version 107741 (0.0009) [2023-12-26 16:09:01,838][105692] Updated weights for policy 0, policy_version 107751 (0.0010) [2023-12-26 16:09:01,945][105620] Updated weights for policy 1, policy_version 108265 (0.0011) [2023-12-26 16:09:02,004][105620] Updated weights for policy 1, policy_version 108275 (0.0008) [2023-12-26 16:09:02,060][105620] Updated weights for policy 1, policy_version 108285 (0.0006) [2023-12-26 16:09:02,115][105620] Updated weights for policy 1, policy_version 108295 (0.0010) [2023-12-26 16:09:02,577][105692] Updated weights for policy 0, policy_version 107761 (0.0010) [2023-12-26 16:09:02,634][105692] Updated weights for policy 0, policy_version 107771 (0.0010) [2023-12-26 16:09:02,682][105692] Updated weights for policy 0, policy_version 107781 (0.0010) [2023-12-26 16:09:02,808][105620] Updated weights for policy 1, policy_version 108305 (0.0010) [2023-12-26 16:09:02,866][105620] Updated weights for policy 1, policy_version 108315 (0.0010) [2023-12-26 16:09:02,924][105620] Updated weights for policy 1, policy_version 108325 (0.0010) [2023-12-26 16:09:03,330][105692] Updated weights for policy 0, policy_version 107791 (0.0009) [2023-12-26 16:09:03,377][105692] Updated weights for policy 0, policy_version 107801 (0.0008) [2023-12-26 16:09:03,424][105692] Updated weights for policy 0, policy_version 107811 (0.0008) [2023-12-26 16:09:03,633][105620] Updated weights for policy 1, policy_version 108335 (0.0007) [2023-12-26 16:09:03,704][105620] Updated weights for policy 1, policy_version 108345 (0.0010) [2023-12-26 16:09:03,772][105620] Updated weights for policy 1, policy_version 108355 (0.0010) [2023-12-26 16:09:04,069][105692] Updated weights for policy 0, policy_version 107821 (0.0006) [2023-12-26 16:09:04,124][105692] Updated weights for policy 0, policy_version 107831 (0.0007) [2023-12-26 16:09:04,188][105692] Updated weights for policy 0, policy_version 107841 (0.0008) [2023-12-26 16:09:04,506][105620] Updated weights for policy 1, policy_version 108365 (0.0011) [2023-12-26 16:09:04,566][105620] Updated weights for policy 1, policy_version 108375 (0.0011) [2023-12-26 16:09:04,619][105620] Updated weights for policy 1, policy_version 108385 (0.0010) [2023-12-26 16:09:04,904][105692] Updated weights for policy 0, policy_version 107851 (0.0008) [2023-12-26 16:09:04,959][105692] Updated weights for policy 0, policy_version 107861 (0.0007) [2023-12-26 16:09:05,009][105692] Updated weights for policy 0, policy_version 107871 (0.0007) [2023-12-26 16:09:05,375][105620] Updated weights for policy 1, policy_version 108395 (0.0011) [2023-12-26 16:09:05,429][105620] Updated weights for policy 1, policy_version 108405 (0.0010) [2023-12-26 16:09:05,477][105620] Updated weights for policy 1, policy_version 108415 (0.0010) [2023-12-26 16:09:05,734][105692] Updated weights for policy 0, policy_version 107881 (0.0007) [2023-12-26 16:09:05,796][105692] Updated weights for policy 0, policy_version 107891 (0.0008) [2023-12-26 16:09:05,858][105692] Updated weights for policy 0, policy_version 107901 (0.0008) [2023-12-26 16:09:05,906][105692] Updated weights for policy 0, policy_version 107911 (0.0008) [2023-12-26 16:09:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 55394304. Throughput: 0: 9510.0, 1: 9976.4. Samples: 55382952. Policy #0 lag: (min: 31.0, avg: 53.6, max: 63.0) [2023-12-26 16:09:06,062][104569] Avg episode reward: [(0, '8906.838'), (1, '9262.379')] [2023-12-26 16:09:06,235][105620] Updated weights for policy 1, policy_version 108425 (0.0010) [2023-12-26 16:09:06,294][105620] Updated weights for policy 1, policy_version 108435 (0.0009) [2023-12-26 16:09:06,346][105620] Updated weights for policy 1, policy_version 108445 (0.0008) [2023-12-26 16:09:06,394][105620] Updated weights for policy 1, policy_version 108455 (0.0009) [2023-12-26 16:09:06,614][105692] Updated weights for policy 0, policy_version 107921 (0.0009) [2023-12-26 16:09:06,675][105692] Updated weights for policy 0, policy_version 107931 (0.0008) [2023-12-26 16:09:06,732][105692] Updated weights for policy 0, policy_version 107942 (0.0009) [2023-12-26 16:09:07,039][105620] Updated weights for policy 1, policy_version 108465 (0.0006) [2023-12-26 16:09:07,105][105620] Updated weights for policy 1, policy_version 108475 (0.0008) [2023-12-26 16:09:07,168][105620] Updated weights for policy 1, policy_version 108485 (0.0009) [2023-12-26 16:09:07,581][105692] Updated weights for policy 0, policy_version 107952 (0.0009) [2023-12-26 16:09:07,634][105692] Updated weights for policy 0, policy_version 107963 (0.0010) [2023-12-26 16:09:07,695][105692] Updated weights for policy 0, policy_version 107973 (0.0009) [2023-12-26 16:09:07,779][105620] Updated weights for policy 1, policy_version 108495 (0.0009) [2023-12-26 16:09:07,826][105620] Updated weights for policy 1, policy_version 108505 (0.0009) [2023-12-26 16:09:07,881][105620] Updated weights for policy 1, policy_version 108515 (0.0009) [2023-12-26 16:09:08,492][105692] Updated weights for policy 0, policy_version 107983 (0.0010) [2023-12-26 16:09:08,549][105692] Updated weights for policy 0, policy_version 107993 (0.0010) [2023-12-26 16:09:08,596][105620] Updated weights for policy 1, policy_version 108525 (0.0008) [2023-12-26 16:09:08,606][105692] Updated weights for policy 0, policy_version 108003 (0.0011) [2023-12-26 16:09:08,643][105620] Updated weights for policy 1, policy_version 108535 (0.0006) [2023-12-26 16:09:08,706][105620] Updated weights for policy 1, policy_version 108545 (0.0010) [2023-12-26 16:09:09,227][105692] Updated weights for policy 0, policy_version 108013 (0.0010) [2023-12-26 16:09:09,283][105692] Updated weights for policy 0, policy_version 108023 (0.0011) [2023-12-26 16:09:09,363][105692] Updated weights for policy 0, policy_version 108033 (0.0010) [2023-12-26 16:09:09,545][105620] Updated weights for policy 1, policy_version 108555 (0.0009) [2023-12-26 16:09:09,603][105620] Updated weights for policy 1, policy_version 108565 (0.0010) [2023-12-26 16:09:09,663][105620] Updated weights for policy 1, policy_version 108575 (0.0009) [2023-12-26 16:09:10,108][105692] Updated weights for policy 0, policy_version 108043 (0.0009) [2023-12-26 16:09:10,160][105692] Updated weights for policy 0, policy_version 108053 (0.0009) [2023-12-26 16:09:10,223][105692] Updated weights for policy 0, policy_version 108063 (0.0009) [2023-12-26 16:09:10,469][105620] Updated weights for policy 1, policy_version 108585 (0.0009) [2023-12-26 16:09:10,534][105620] Updated weights for policy 1, policy_version 108595 (0.0010) [2023-12-26 16:09:10,598][105620] Updated weights for policy 1, policy_version 108605 (0.0009) [2023-12-26 16:09:10,660][105620] Updated weights for policy 1, policy_version 108615 (0.0009) [2023-12-26 16:09:10,966][105692] Updated weights for policy 0, policy_version 108073 (0.0010) [2023-12-26 16:09:11,011][105692] Updated weights for policy 0, policy_version 108083 (0.0008) [2023-12-26 16:09:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 55484416. Throughput: 0: 9525.2, 1: 9897.3. Samples: 55496720. Policy #0 lag: (min: 31.0, avg: 53.6, max: 63.0) [2023-12-26 16:09:11,063][104569] Avg episode reward: [(0, '8655.678'), (1, '8651.500')] [2023-12-26 16:09:11,079][105692] Updated weights for policy 0, policy_version 108093 (0.0010) [2023-12-26 16:09:11,131][105692] Updated weights for policy 0, policy_version 108103 (0.0009) [2023-12-26 16:09:11,399][105620] Updated weights for policy 1, policy_version 108625 (0.0008) [2023-12-26 16:09:11,456][105620] Updated weights for policy 1, policy_version 108635 (0.0006) [2023-12-26 16:09:11,511][105620] Updated weights for policy 1, policy_version 108645 (0.0006) [2023-12-26 16:09:12,003][105692] Updated weights for policy 0, policy_version 108113 (0.0007) [2023-12-26 16:09:12,062][105692] Updated weights for policy 0, policy_version 108123 (0.0009) [2023-12-26 16:09:12,113][105692] Updated weights for policy 0, policy_version 108133 (0.0009) [2023-12-26 16:09:12,221][105620] Updated weights for policy 1, policy_version 108655 (0.0008) [2023-12-26 16:09:12,274][105620] Updated weights for policy 1, policy_version 108665 (0.0009) [2023-12-26 16:09:12,331][105620] Updated weights for policy 1, policy_version 108675 (0.0010) [2023-12-26 16:09:12,825][105692] Updated weights for policy 0, policy_version 108143 (0.0009) [2023-12-26 16:09:12,885][105692] Updated weights for policy 0, policy_version 108153 (0.0008) [2023-12-26 16:09:12,941][105692] Updated weights for policy 0, policy_version 108163 (0.0008) [2023-12-26 16:09:13,038][105620] Updated weights for policy 1, policy_version 108685 (0.0008) [2023-12-26 16:09:13,106][105620] Updated weights for policy 1, policy_version 108695 (0.0006) [2023-12-26 16:09:13,160][105620] Updated weights for policy 1, policy_version 108705 (0.0006) [2023-12-26 16:09:13,732][105620] Updated weights for policy 1, policy_version 108715 (0.0009) [2023-12-26 16:09:13,788][105620] Updated weights for policy 1, policy_version 108725 (0.0005) [2023-12-26 16:09:13,802][105692] Updated weights for policy 0, policy_version 108173 (0.0009) [2023-12-26 16:09:13,838][105620] Updated weights for policy 1, policy_version 108735 (0.0005) [2023-12-26 16:09:13,866][105692] Updated weights for policy 0, policy_version 108183 (0.0009) [2023-12-26 16:09:13,925][105692] Updated weights for policy 0, policy_version 108193 (0.0010) [2023-12-26 16:09:14,413][105620] Updated weights for policy 1, policy_version 108745 (0.0006) [2023-12-26 16:09:14,474][105620] Updated weights for policy 1, policy_version 108755 (0.0009) [2023-12-26 16:09:14,536][105620] Updated weights for policy 1, policy_version 108765 (0.0009) [2023-12-26 16:09:14,590][105620] Updated weights for policy 1, policy_version 108775 (0.0009) [2023-12-26 16:09:14,750][105692] Updated weights for policy 0, policy_version 108203 (0.0010) [2023-12-26 16:09:14,822][105692] Updated weights for policy 0, policy_version 108213 (0.0010) [2023-12-26 16:09:14,896][105692] Updated weights for policy 0, policy_version 108223 (0.0009) [2023-12-26 16:09:15,335][105620] Updated weights for policy 1, policy_version 108785 (0.0009) [2023-12-26 16:09:15,394][105620] Updated weights for policy 1, policy_version 108795 (0.0009) [2023-12-26 16:09:15,449][105620] Updated weights for policy 1, policy_version 108805 (0.0010) [2023-12-26 16:09:15,681][105692] Updated weights for policy 0, policy_version 108233 (0.0009) [2023-12-26 16:09:15,733][105692] Updated weights for policy 0, policy_version 108243 (0.0009) [2023-12-26 16:09:15,789][105692] Updated weights for policy 0, policy_version 108253 (0.0009) [2023-12-26 16:09:15,843][105692] Updated weights for policy 0, policy_version 108263 (0.0008) [2023-12-26 16:09:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 55582720. Throughput: 0: 9407.2, 1: 9853.9. Samples: 55554700. Policy #0 lag: (min: 31.0, avg: 53.6, max: 63.0) [2023-12-26 16:09:16,062][104569] Avg episode reward: [(0, '7967.205'), (1, '8472.602')] [2023-12-26 16:09:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000108264_27721728.pth... [2023-12-26 16:09:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000108808_27860992.pth... [2023-12-26 16:09:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000107176_27443200.pth [2023-12-26 16:09:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000107688_27574272.pth [2023-12-26 16:09:16,258][105620] Updated weights for policy 1, policy_version 108815 (0.0009) [2023-12-26 16:09:16,309][105620] Updated weights for policy 1, policy_version 108825 (0.0009) [2023-12-26 16:09:16,364][105620] Updated weights for policy 1, policy_version 108835 (0.0008) [2023-12-26 16:09:16,544][105692] Updated weights for policy 0, policy_version 108273 (0.0010) [2023-12-26 16:09:16,619][105692] Updated weights for policy 0, policy_version 108283 (0.0010) [2023-12-26 16:09:16,686][105692] Updated weights for policy 0, policy_version 108293 (0.0009) [2023-12-26 16:09:17,088][105620] Updated weights for policy 1, policy_version 108845 (0.0008) [2023-12-26 16:09:17,147][105620] Updated weights for policy 1, policy_version 108855 (0.0009) [2023-12-26 16:09:17,207][105620] Updated weights for policy 1, policy_version 108865 (0.0009) [2023-12-26 16:09:17,433][105692] Updated weights for policy 0, policy_version 108303 (0.0008) [2023-12-26 16:09:17,490][105692] Updated weights for policy 0, policy_version 108313 (0.0010) [2023-12-26 16:09:17,544][105692] Updated weights for policy 0, policy_version 108323 (0.0009) [2023-12-26 16:09:17,922][105620] Updated weights for policy 1, policy_version 108875 (0.0009) [2023-12-26 16:09:17,972][105620] Updated weights for policy 1, policy_version 108885 (0.0008) [2023-12-26 16:09:18,024][105620] Updated weights for policy 1, policy_version 108895 (0.0009) [2023-12-26 16:09:18,300][105692] Updated weights for policy 0, policy_version 108333 (0.0009) [2023-12-26 16:09:18,361][105692] Updated weights for policy 0, policy_version 108343 (0.0009) [2023-12-26 16:09:18,423][105692] Updated weights for policy 0, policy_version 108353 (0.0009) [2023-12-26 16:09:18,799][105620] Updated weights for policy 1, policy_version 108905 (0.0008) [2023-12-26 16:09:18,853][105620] Updated weights for policy 1, policy_version 108915 (0.0009) [2023-12-26 16:09:18,908][105620] Updated weights for policy 1, policy_version 108925 (0.0009) [2023-12-26 16:09:18,969][105620] Updated weights for policy 1, policy_version 108935 (0.0005) [2023-12-26 16:09:19,235][105692] Updated weights for policy 0, policy_version 108363 (0.0009) [2023-12-26 16:09:19,302][105692] Updated weights for policy 0, policy_version 108373 (0.0006) [2023-12-26 16:09:19,368][105692] Updated weights for policy 0, policy_version 108383 (0.0008) [2023-12-26 16:09:19,582][105620] Updated weights for policy 1, policy_version 108945 (0.0005) [2023-12-26 16:09:19,643][105620] Updated weights for policy 1, policy_version 108955 (0.0009) [2023-12-26 16:09:19,701][105620] Updated weights for policy 1, policy_version 108965 (0.0007) [2023-12-26 16:09:20,131][105692] Updated weights for policy 0, policy_version 108393 (0.0010) [2023-12-26 16:09:20,194][105692] Updated weights for policy 0, policy_version 108403 (0.0010) [2023-12-26 16:09:20,256][105692] Updated weights for policy 0, policy_version 108413 (0.0006) [2023-12-26 16:09:20,320][105692] Updated weights for policy 0, policy_version 108423 (0.0007) [2023-12-26 16:09:20,369][105620] Updated weights for policy 1, policy_version 108975 (0.0008) [2023-12-26 16:09:20,430][105620] Updated weights for policy 1, policy_version 108985 (0.0009) [2023-12-26 16:09:20,482][105620] Updated weights for policy 1, policy_version 108995 (0.0009) [2023-12-26 16:09:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 55672832. Throughput: 0: 9361.0, 1: 9810.6. Samples: 55667172. Policy #0 lag: (min: 31.0, avg: 53.6, max: 63.0) [2023-12-26 16:09:21,063][104569] Avg episode reward: [(0, '8221.378'), (1, '8557.856')] [2023-12-26 16:09:21,096][105692] Updated weights for policy 0, policy_version 108433 (0.0008) [2023-12-26 16:09:21,127][105620] Updated weights for policy 1, policy_version 109005 (0.0009) [2023-12-26 16:09:21,161][105692] Updated weights for policy 0, policy_version 108443 (0.0007) [2023-12-26 16:09:21,190][105620] Updated weights for policy 1, policy_version 109015 (0.0011) [2023-12-26 16:09:21,217][105692] Updated weights for policy 0, policy_version 108453 (0.0005) [2023-12-26 16:09:21,251][105620] Updated weights for policy 1, policy_version 109025 (0.0011) [2023-12-26 16:09:21,999][105620] Updated weights for policy 1, policy_version 109035 (0.0010) [2023-12-26 16:09:22,052][105620] Updated weights for policy 1, policy_version 109045 (0.0009) [2023-12-26 16:09:22,099][105692] Updated weights for policy 0, policy_version 108463 (0.0008) [2023-12-26 16:09:22,109][105620] Updated weights for policy 1, policy_version 109055 (0.0009) [2023-12-26 16:09:22,155][105692] Updated weights for policy 0, policy_version 108473 (0.0009) [2023-12-26 16:09:22,214][105692] Updated weights for policy 0, policy_version 108483 (0.0008) [2023-12-26 16:09:22,945][105620] Updated weights for policy 1, policy_version 109065 (0.0010) [2023-12-26 16:09:22,952][105692] Updated weights for policy 0, policy_version 108493 (0.0008) [2023-12-26 16:09:23,004][105620] Updated weights for policy 1, policy_version 109075 (0.0007) [2023-12-26 16:09:23,014][105692] Updated weights for policy 0, policy_version 108503 (0.0006) [2023-12-26 16:09:23,065][105620] Updated weights for policy 1, policy_version 109085 (0.0008) [2023-12-26 16:09:23,072][105692] Updated weights for policy 0, policy_version 108513 (0.0007) [2023-12-26 16:09:23,132][105620] Updated weights for policy 1, policy_version 109095 (0.0008) [2023-12-26 16:09:23,770][105620] Updated weights for policy 1, policy_version 109105 (0.0006) [2023-12-26 16:09:23,826][105620] Updated weights for policy 1, policy_version 109115 (0.0005) [2023-12-26 16:09:23,880][105620] Updated weights for policy 1, policy_version 109125 (0.0006) [2023-12-26 16:09:23,896][105692] Updated weights for policy 0, policy_version 108523 (0.0007) [2023-12-26 16:09:23,945][105692] Updated weights for policy 0, policy_version 108533 (0.0008) [2023-12-26 16:09:23,992][105692] Updated weights for policy 0, policy_version 108543 (0.0005) [2023-12-26 16:09:24,457][105620] Updated weights for policy 1, policy_version 109135 (0.0007) [2023-12-26 16:09:24,523][105620] Updated weights for policy 1, policy_version 109145 (0.0009) [2023-12-26 16:09:24,575][105620] Updated weights for policy 1, policy_version 109155 (0.0005) [2023-12-26 16:09:24,658][105692] Updated weights for policy 0, policy_version 108553 (0.0006) [2023-12-26 16:09:24,710][105692] Updated weights for policy 0, policy_version 108563 (0.0005) [2023-12-26 16:09:24,764][105692] Updated weights for policy 0, policy_version 108573 (0.0006) [2023-12-26 16:09:24,817][105692] Updated weights for policy 0, policy_version 108583 (0.0005) [2023-12-26 16:09:25,103][105620] Updated weights for policy 1, policy_version 109165 (0.0005) [2023-12-26 16:09:25,166][105620] Updated weights for policy 1, policy_version 109175 (0.0007) [2023-12-26 16:09:25,217][105620] Updated weights for policy 1, policy_version 109185 (0.0009) [2023-12-26 16:09:25,490][105692] Updated weights for policy 0, policy_version 108593 (0.0006) [2023-12-26 16:09:25,538][105692] Updated weights for policy 0, policy_version 108603 (0.0006) [2023-12-26 16:09:25,586][105692] Updated weights for policy 0, policy_version 108613 (0.0005) [2023-12-26 16:09:26,014][105620] Updated weights for policy 1, policy_version 109195 (0.0009) [2023-12-26 16:09:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.3, 300 sec: 19466.4). Total num frames: 55771136. Throughput: 0: 9296.8, 1: 9811.9. Samples: 55784552. Policy #0 lag: (min: 31.0, avg: 53.6, max: 63.0) [2023-12-26 16:09:26,062][104569] Avg episode reward: [(0, '8826.778'), (1, '8387.012')] [2023-12-26 16:09:26,068][105620] Updated weights for policy 1, policy_version 109205 (0.0010) [2023-12-26 16:09:26,120][105620] Updated weights for policy 1, policy_version 109215 (0.0010) [2023-12-26 16:09:26,149][105692] Updated weights for policy 0, policy_version 108623 (0.0005) [2023-12-26 16:09:26,203][105692] Updated weights for policy 0, policy_version 108633 (0.0005) [2023-12-26 16:09:26,263][105692] Updated weights for policy 0, policy_version 108643 (0.0005) [2023-12-26 16:09:26,814][105620] Updated weights for policy 1, policy_version 109225 (0.0010) [2023-12-26 16:09:26,883][105620] Updated weights for policy 1, policy_version 109235 (0.0005) [2023-12-26 16:09:26,939][105620] Updated weights for policy 1, policy_version 109245 (0.0005) [2023-12-26 16:09:26,966][105692] Updated weights for policy 0, policy_version 108653 (0.0009) [2023-12-26 16:09:26,997][105620] Updated weights for policy 1, policy_version 109255 (0.0005) [2023-12-26 16:09:27,017][105692] Updated weights for policy 0, policy_version 108663 (0.0006) [2023-12-26 16:09:27,081][105692] Updated weights for policy 0, policy_version 108673 (0.0010) [2023-12-26 16:09:27,491][105620] Updated weights for policy 1, policy_version 109265 (0.0005) [2023-12-26 16:09:27,538][105620] Updated weights for policy 1, policy_version 109275 (0.0005) [2023-12-26 16:09:27,596][105620] Updated weights for policy 1, policy_version 109285 (0.0008) [2023-12-26 16:09:27,801][105692] Updated weights for policy 0, policy_version 108683 (0.0010) [2023-12-26 16:09:27,861][105692] Updated weights for policy 0, policy_version 108693 (0.0010) [2023-12-26 16:09:27,921][105692] Updated weights for policy 0, policy_version 108703 (0.0010) [2023-12-26 16:09:28,290][105620] Updated weights for policy 1, policy_version 109295 (0.0010) [2023-12-26 16:09:28,345][105620] Updated weights for policy 1, policy_version 109305 (0.0010) [2023-12-26 16:09:28,400][105620] Updated weights for policy 1, policy_version 109315 (0.0010) [2023-12-26 16:09:28,650][105692] Updated weights for policy 0, policy_version 108713 (0.0009) [2023-12-26 16:09:28,712][105692] Updated weights for policy 0, policy_version 108723 (0.0007) [2023-12-26 16:09:28,763][105692] Updated weights for policy 0, policy_version 108733 (0.0006) [2023-12-26 16:09:28,811][105692] Updated weights for policy 0, policy_version 108743 (0.0007) [2023-12-26 16:09:29,149][105620] Updated weights for policy 1, policy_version 109325 (0.0011) [2023-12-26 16:09:29,206][105620] Updated weights for policy 1, policy_version 109335 (0.0010) [2023-12-26 16:09:29,277][105620] Updated weights for policy 1, policy_version 109345 (0.0011) [2023-12-26 16:09:29,493][105692] Updated weights for policy 0, policy_version 108753 (0.0010) [2023-12-26 16:09:29,548][105692] Updated weights for policy 0, policy_version 108763 (0.0010) [2023-12-26 16:09:29,606][105692] Updated weights for policy 0, policy_version 108773 (0.0008) [2023-12-26 16:09:30,003][105620] Updated weights for policy 1, policy_version 109355 (0.0009) [2023-12-26 16:09:30,059][105620] Updated weights for policy 1, policy_version 109365 (0.0008) [2023-12-26 16:09:30,121][105620] Updated weights for policy 1, policy_version 109375 (0.0008) [2023-12-26 16:09:30,275][105692] Updated weights for policy 0, policy_version 108783 (0.0009) [2023-12-26 16:09:30,343][105692] Updated weights for policy 0, policy_version 108793 (0.0010) [2023-12-26 16:09:30,412][105692] Updated weights for policy 0, policy_version 108803 (0.0010) [2023-12-26 16:09:30,761][105620] Updated weights for policy 1, policy_version 109385 (0.0008) [2023-12-26 16:09:30,828][105620] Updated weights for policy 1, policy_version 109395 (0.0006) [2023-12-26 16:09:30,881][105620] Updated weights for policy 1, policy_version 109405 (0.0006) [2023-12-26 16:09:30,937][105620] Updated weights for policy 1, policy_version 109415 (0.0008) [2023-12-26 16:09:31,001][105692] Updated weights for policy 0, policy_version 108813 (0.0010) [2023-12-26 16:09:31,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 55877632. Throughput: 0: 9375.2, 1: 9805.3. Samples: 55846300. Policy #0 lag: (min: 31.0, avg: 53.6, max: 63.0) [2023-12-26 16:09:31,063][104569] Avg episode reward: [(0, '8918.409'), (1, '7936.962')] [2023-12-26 16:09:31,063][105692] Updated weights for policy 0, policy_version 108823 (0.0011) [2023-12-26 16:09:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000109416_28016640.pth... [2023-12-26 16:09:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000108232_27713536.pth [2023-12-26 16:09:31,127][105692] Updated weights for policy 0, policy_version 108833 (0.0010) [2023-12-26 16:09:31,166][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000108840_27869184.pth... [2023-12-26 16:09:31,171][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000107720_27582464.pth [2023-12-26 16:09:31,683][105620] Updated weights for policy 1, policy_version 109425 (0.0007) [2023-12-26 16:09:31,750][105620] Updated weights for policy 1, policy_version 109435 (0.0008) [2023-12-26 16:09:31,816][105620] Updated weights for policy 1, policy_version 109445 (0.0008) [2023-12-26 16:09:31,865][105692] Updated weights for policy 0, policy_version 108843 (0.0010) [2023-12-26 16:09:31,925][105692] Updated weights for policy 0, policy_version 108853 (0.0008) [2023-12-26 16:09:31,985][105692] Updated weights for policy 0, policy_version 108863 (0.0008) [2023-12-26 16:09:32,573][105620] Updated weights for policy 1, policy_version 109455 (0.0009) [2023-12-26 16:09:32,622][105692] Updated weights for policy 0, policy_version 108873 (0.0008) [2023-12-26 16:09:32,629][105620] Updated weights for policy 1, policy_version 109465 (0.0010) [2023-12-26 16:09:32,679][105620] Updated weights for policy 1, policy_version 109475 (0.0007) [2023-12-26 16:09:32,688][105692] Updated weights for policy 0, policy_version 108883 (0.0007) [2023-12-26 16:09:32,743][105692] Updated weights for policy 0, policy_version 108893 (0.0008) [2023-12-26 16:09:32,796][105692] Updated weights for policy 0, policy_version 108903 (0.0009) [2023-12-26 16:09:33,425][105692] Updated weights for policy 0, policy_version 108913 (0.0009) [2023-12-26 16:09:33,471][105692] Updated weights for policy 0, policy_version 108923 (0.0008) [2023-12-26 16:09:33,492][105620] Updated weights for policy 1, policy_version 109485 (0.0008) [2023-12-26 16:09:33,521][105692] Updated weights for policy 0, policy_version 108933 (0.0008) [2023-12-26 16:09:33,551][105620] Updated weights for policy 1, policy_version 109495 (0.0008) [2023-12-26 16:09:33,603][105620] Updated weights for policy 1, policy_version 109505 (0.0009) [2023-12-26 16:09:34,272][105692] Updated weights for policy 0, policy_version 108943 (0.0008) [2023-12-26 16:09:34,326][105692] Updated weights for policy 0, policy_version 108953 (0.0009) [2023-12-26 16:09:34,350][105620] Updated weights for policy 1, policy_version 109515 (0.0009) [2023-12-26 16:09:34,375][105692] Updated weights for policy 0, policy_version 108963 (0.0008) [2023-12-26 16:09:34,412][105620] Updated weights for policy 1, policy_version 109525 (0.0006) [2023-12-26 16:09:34,466][105620] Updated weights for policy 1, policy_version 109535 (0.0009) [2023-12-26 16:09:35,180][105692] Updated weights for policy 0, policy_version 108973 (0.0009) [2023-12-26 16:09:35,218][105620] Updated weights for policy 1, policy_version 109545 (0.0009) [2023-12-26 16:09:35,229][105692] Updated weights for policy 0, policy_version 108983 (0.0007) [2023-12-26 16:09:35,270][105620] Updated weights for policy 1, policy_version 109555 (0.0009) [2023-12-26 16:09:35,281][105692] Updated weights for policy 0, policy_version 108993 (0.0007) [2023-12-26 16:09:35,320][105620] Updated weights for policy 1, policy_version 109565 (0.0007) [2023-12-26 16:09:35,384][105620] Updated weights for policy 1, policy_version 109575 (0.0008) [2023-12-26 16:09:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.1, 300 sec: 19494.2). Total num frames: 55967744. Throughput: 0: 9488.7, 1: 9800.0. Samples: 55963484. Policy #0 lag: (min: 10.0, avg: 18.9, max: 42.0) [2023-12-26 16:09:36,062][104569] Avg episode reward: [(0, '8896.131'), (1, '8464.592')] [2023-12-26 16:09:36,071][105620] Updated weights for policy 1, policy_version 109585 (0.0009) [2023-12-26 16:09:36,078][105692] Updated weights for policy 0, policy_version 109003 (0.0006) [2023-12-26 16:09:36,134][105620] Updated weights for policy 1, policy_version 109595 (0.0008) [2023-12-26 16:09:36,148][105692] Updated weights for policy 0, policy_version 109013 (0.0007) [2023-12-26 16:09:36,201][105620] Updated weights for policy 1, policy_version 109605 (0.0007) [2023-12-26 16:09:36,203][105692] Updated weights for policy 0, policy_version 109023 (0.0007) [2023-12-26 16:09:36,867][105620] Updated weights for policy 1, policy_version 109615 (0.0007) [2023-12-26 16:09:36,930][105620] Updated weights for policy 1, policy_version 109625 (0.0011) [2023-12-26 16:09:36,993][105620] Updated weights for policy 1, policy_version 109635 (0.0011) [2023-12-26 16:09:37,009][105692] Updated weights for policy 0, policy_version 109033 (0.0008) [2023-12-26 16:09:37,071][105692] Updated weights for policy 0, policy_version 109043 (0.0008) [2023-12-26 16:09:37,138][105692] Updated weights for policy 0, policy_version 109053 (0.0008) [2023-12-26 16:09:37,204][105692] Updated weights for policy 0, policy_version 109063 (0.0008) [2023-12-26 16:09:37,729][105620] Updated weights for policy 1, policy_version 109645 (0.0011) [2023-12-26 16:09:37,783][105692] Updated weights for policy 0, policy_version 109073 (0.0006) [2023-12-26 16:09:37,785][105620] Updated weights for policy 1, policy_version 109655 (0.0010) [2023-12-26 16:09:37,836][105692] Updated weights for policy 0, policy_version 109083 (0.0006) [2023-12-26 16:09:37,845][105620] Updated weights for policy 1, policy_version 109665 (0.0011) [2023-12-26 16:09:37,884][105692] Updated weights for policy 0, policy_version 109093 (0.0007) [2023-12-26 16:09:38,530][105692] Updated weights for policy 0, policy_version 109103 (0.0008) [2023-12-26 16:09:38,587][105620] Updated weights for policy 1, policy_version 109675 (0.0010) [2023-12-26 16:09:38,593][105692] Updated weights for policy 0, policy_version 109113 (0.0008) [2023-12-26 16:09:38,636][105620] Updated weights for policy 1, policy_version 109685 (0.0010) [2023-12-26 16:09:38,649][105692] Updated weights for policy 0, policy_version 109123 (0.0006) [2023-12-26 16:09:38,688][105620] Updated weights for policy 1, policy_version 109695 (0.0010) [2023-12-26 16:09:39,413][105692] Updated weights for policy 0, policy_version 109133 (0.0007) [2023-12-26 16:09:39,472][105620] Updated weights for policy 1, policy_version 109705 (0.0010) [2023-12-26 16:09:39,474][105692] Updated weights for policy 0, policy_version 109143 (0.0008) [2023-12-26 16:09:39,525][105620] Updated weights for policy 1, policy_version 109715 (0.0010) [2023-12-26 16:09:39,532][105692] Updated weights for policy 0, policy_version 109153 (0.0006) [2023-12-26 16:09:39,589][105620] Updated weights for policy 1, policy_version 109725 (0.0010) [2023-12-26 16:09:39,654][105620] Updated weights for policy 1, policy_version 109735 (0.0010) [2023-12-26 16:09:40,319][105692] Updated weights for policy 0, policy_version 109163 (0.0009) [2023-12-26 16:09:40,376][105692] Updated weights for policy 0, policy_version 109173 (0.0008) [2023-12-26 16:09:40,430][105692] Updated weights for policy 0, policy_version 109183 (0.0006) [2023-12-26 16:09:40,435][105620] Updated weights for policy 1, policy_version 109745 (0.0010) [2023-12-26 16:09:40,494][105620] Updated weights for policy 1, policy_version 109755 (0.0010) [2023-12-26 16:09:40,556][105620] Updated weights for policy 1, policy_version 109765 (0.0010) [2023-12-26 16:09:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 56066048. Throughput: 0: 9550.1, 1: 9777.8. Samples: 56077092. Policy #0 lag: (min: 10.0, avg: 18.9, max: 42.0) [2023-12-26 16:09:41,063][104569] Avg episode reward: [(0, '8895.551'), (1, '8383.104')] [2023-12-26 16:09:41,237][105692] Updated weights for policy 0, policy_version 109193 (0.0007) [2023-12-26 16:09:41,300][105692] Updated weights for policy 0, policy_version 109203 (0.0008) [2023-12-26 16:09:41,303][105620] Updated weights for policy 1, policy_version 109775 (0.0011) [2023-12-26 16:09:41,372][105692] Updated weights for policy 0, policy_version 109213 (0.0008) [2023-12-26 16:09:41,369][105620] Updated weights for policy 1, policy_version 109785 (0.0010) [2023-12-26 16:09:41,436][105620] Updated weights for policy 1, policy_version 109795 (0.0008) [2023-12-26 16:09:41,439][105692] Updated weights for policy 0, policy_version 109223 (0.0008) [2023-12-26 16:09:42,166][105692] Updated weights for policy 0, policy_version 109233 (0.0009) [2023-12-26 16:09:42,203][105620] Updated weights for policy 1, policy_version 109805 (0.0007) [2023-12-26 16:09:42,225][105692] Updated weights for policy 0, policy_version 109243 (0.0009) [2023-12-26 16:09:42,266][105620] Updated weights for policy 1, policy_version 109815 (0.0006) [2023-12-26 16:09:42,289][105692] Updated weights for policy 0, policy_version 109253 (0.0010) [2023-12-26 16:09:42,325][105620] Updated weights for policy 1, policy_version 109825 (0.0007) [2023-12-26 16:09:42,976][105692] Updated weights for policy 0, policy_version 109263 (0.0008) [2023-12-26 16:09:43,022][105692] Updated weights for policy 0, policy_version 109273 (0.0008) [2023-12-26 16:09:43,077][105692] Updated weights for policy 0, policy_version 109283 (0.0009) [2023-12-26 16:09:43,096][105620] Updated weights for policy 1, policy_version 109835 (0.0008) [2023-12-26 16:09:43,145][105620] Updated weights for policy 1, policy_version 109845 (0.0008) [2023-12-26 16:09:43,208][105620] Updated weights for policy 1, policy_version 109855 (0.0009) [2023-12-26 16:09:43,767][105692] Updated weights for policy 0, policy_version 109293 (0.0007) [2023-12-26 16:09:43,831][105692] Updated weights for policy 0, policy_version 109303 (0.0007) [2023-12-26 16:09:43,892][105692] Updated weights for policy 0, policy_version 109313 (0.0008) [2023-12-26 16:09:44,032][105620] Updated weights for policy 1, policy_version 109865 (0.0008) [2023-12-26 16:09:44,096][105620] Updated weights for policy 1, policy_version 109875 (0.0009) [2023-12-26 16:09:44,143][105620] Updated weights for policy 1, policy_version 109885 (0.0009) [2023-12-26 16:09:44,189][105620] Updated weights for policy 1, policy_version 109895 (0.0008) [2023-12-26 16:09:44,576][105692] Updated weights for policy 0, policy_version 109323 (0.0007) [2023-12-26 16:09:44,631][105692] Updated weights for policy 0, policy_version 109333 (0.0010) [2023-12-26 16:09:44,685][105692] Updated weights for policy 0, policy_version 109343 (0.0007) [2023-12-26 16:09:44,820][105620] Updated weights for policy 1, policy_version 109905 (0.0006) [2023-12-26 16:09:44,877][105620] Updated weights for policy 1, policy_version 109915 (0.0006) [2023-12-26 16:09:44,938][105620] Updated weights for policy 1, policy_version 109925 (0.0005) [2023-12-26 16:09:45,378][105692] Updated weights for policy 0, policy_version 109353 (0.0005) [2023-12-26 16:09:45,438][105692] Updated weights for policy 0, policy_version 109363 (0.0009) [2023-12-26 16:09:45,499][105692] Updated weights for policy 0, policy_version 109373 (0.0009) [2023-12-26 16:09:45,558][105692] Updated weights for policy 0, policy_version 109383 (0.0009) [2023-12-26 16:09:45,671][105620] Updated weights for policy 1, policy_version 109935 (0.0008) [2023-12-26 16:09:45,733][105620] Updated weights for policy 1, policy_version 109945 (0.0007) [2023-12-26 16:09:45,792][105620] Updated weights for policy 1, policy_version 109955 (0.0008) [2023-12-26 16:09:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19251.1, 300 sec: 19494.2). Total num frames: 56164352. Throughput: 0: 9570.8, 1: 9775.5. Samples: 56132796. Policy #0 lag: (min: 10.0, avg: 18.9, max: 42.0) [2023-12-26 16:09:46,063][104569] Avg episode reward: [(0, '9081.798'), (1, '8384.717')] [2023-12-26 16:09:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000109960_28155904.pth... [2023-12-26 16:09:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000109384_28008448.pth... [2023-12-26 16:09:46,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000108808_27860992.pth [2023-12-26 16:09:46,082][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000108264_27721728.pth [2023-12-26 16:09:46,253][105692] Updated weights for policy 0, policy_version 109393 (0.0007) [2023-12-26 16:09:46,310][105692] Updated weights for policy 0, policy_version 109403 (0.0006) [2023-12-26 16:09:46,333][105620] Updated weights for policy 1, policy_version 109965 (0.0005) [2023-12-26 16:09:46,364][105692] Updated weights for policy 0, policy_version 109413 (0.0009) [2023-12-26 16:09:46,398][105620] Updated weights for policy 1, policy_version 109975 (0.0005) [2023-12-26 16:09:46,454][105620] Updated weights for policy 1, policy_version 109985 (0.0005) [2023-12-26 16:09:47,074][105692] Updated weights for policy 0, policy_version 109423 (0.0007) [2023-12-26 16:09:47,120][105620] Updated weights for policy 1, policy_version 109995 (0.0005) [2023-12-26 16:09:47,135][105692] Updated weights for policy 0, policy_version 109433 (0.0007) [2023-12-26 16:09:47,180][105620] Updated weights for policy 1, policy_version 110005 (0.0008) [2023-12-26 16:09:47,195][105692] Updated weights for policy 0, policy_version 109443 (0.0006) [2023-12-26 16:09:47,241][105620] Updated weights for policy 1, policy_version 110015 (0.0009) [2023-12-26 16:09:47,829][105692] Updated weights for policy 0, policy_version 109453 (0.0007) [2023-12-26 16:09:47,873][105692] Updated weights for policy 0, policy_version 109463 (0.0008) [2023-12-26 16:09:47,926][105692] Updated weights for policy 0, policy_version 109473 (0.0005) [2023-12-26 16:09:47,997][105620] Updated weights for policy 1, policy_version 110025 (0.0009) [2023-12-26 16:09:48,054][105620] Updated weights for policy 1, policy_version 110035 (0.0010) [2023-12-26 16:09:48,109][105620] Updated weights for policy 1, policy_version 110045 (0.0009) [2023-12-26 16:09:48,160][105620] Updated weights for policy 1, policy_version 110055 (0.0009) [2023-12-26 16:09:48,618][105692] Updated weights for policy 0, policy_version 109483 (0.0007) [2023-12-26 16:09:48,666][105692] Updated weights for policy 0, policy_version 109493 (0.0009) [2023-12-26 16:09:48,728][105692] Updated weights for policy 0, policy_version 109503 (0.0008) [2023-12-26 16:09:48,923][105620] Updated weights for policy 1, policy_version 110065 (0.0009) [2023-12-26 16:09:48,985][105620] Updated weights for policy 1, policy_version 110075 (0.0009) [2023-12-26 16:09:49,039][105620] Updated weights for policy 1, policy_version 110085 (0.0010) [2023-12-26 16:09:49,460][105692] Updated weights for policy 0, policy_version 109513 (0.0009) [2023-12-26 16:09:49,522][105692] Updated weights for policy 0, policy_version 109523 (0.0009) [2023-12-26 16:09:49,577][105692] Updated weights for policy 0, policy_version 109533 (0.0006) [2023-12-26 16:09:49,628][105692] Updated weights for policy 0, policy_version 109543 (0.0006) [2023-12-26 16:09:49,850][105620] Updated weights for policy 1, policy_version 110095 (0.0009) [2023-12-26 16:09:49,902][105620] Updated weights for policy 1, policy_version 110105 (0.0006) [2023-12-26 16:09:49,968][105620] Updated weights for policy 1, policy_version 110115 (0.0006) [2023-12-26 16:09:50,412][105692] Updated weights for policy 0, policy_version 109553 (0.0010) [2023-12-26 16:09:50,466][105692] Updated weights for policy 0, policy_version 109563 (0.0010) [2023-12-26 16:09:50,519][105692] Updated weights for policy 0, policy_version 109574 (0.0010) [2023-12-26 16:09:50,554][105620] Updated weights for policy 1, policy_version 110125 (0.0006) [2023-12-26 16:09:50,620][105620] Updated weights for policy 1, policy_version 110135 (0.0008) [2023-12-26 16:09:50,680][105620] Updated weights for policy 1, policy_version 110145 (0.0009) [2023-12-26 16:09:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 56262656. Throughput: 0: 9542.6, 1: 9768.4. Samples: 56251952. Policy #0 lag: (min: 10.0, avg: 18.9, max: 42.0) [2023-12-26 16:09:51,063][104569] Avg episode reward: [(0, '9081.869'), (1, '8302.947')] [2023-12-26 16:09:51,341][105692] Updated weights for policy 0, policy_version 109584 (0.0009) [2023-12-26 16:09:51,411][105692] Updated weights for policy 0, policy_version 109594 (0.0008) [2023-12-26 16:09:51,464][105620] Updated weights for policy 1, policy_version 110155 (0.0009) [2023-12-26 16:09:51,464][105692] Updated weights for policy 0, policy_version 109604 (0.0009) [2023-12-26 16:09:51,520][105620] Updated weights for policy 1, policy_version 110165 (0.0007) [2023-12-26 16:09:51,571][105620] Updated weights for policy 1, policy_version 110175 (0.0008) [2023-12-26 16:09:52,234][105620] Updated weights for policy 1, policy_version 110185 (0.0009) [2023-12-26 16:09:52,270][105692] Updated weights for policy 0, policy_version 109614 (0.0009) [2023-12-26 16:09:52,297][105620] Updated weights for policy 1, policy_version 110195 (0.0008) [2023-12-26 16:09:52,330][105692] Updated weights for policy 0, policy_version 109624 (0.0007) [2023-12-26 16:09:52,369][105620] Updated weights for policy 1, policy_version 110205 (0.0008) [2023-12-26 16:09:52,399][105692] Updated weights for policy 0, policy_version 109634 (0.0010) [2023-12-26 16:09:52,422][105620] Updated weights for policy 1, policy_version 110215 (0.0006) [2023-12-26 16:09:53,077][105692] Updated weights for policy 0, policy_version 109644 (0.0010) [2023-12-26 16:09:53,128][105692] Updated weights for policy 0, policy_version 109654 (0.0008) [2023-12-26 16:09:53,154][105620] Updated weights for policy 1, policy_version 110225 (0.0006) [2023-12-26 16:09:53,178][105692] Updated weights for policy 0, policy_version 109664 (0.0008) [2023-12-26 16:09:53,203][105620] Updated weights for policy 1, policy_version 110235 (0.0006) [2023-12-26 16:09:53,248][105620] Updated weights for policy 1, policy_version 110245 (0.0008) [2023-12-26 16:09:53,834][105692] Updated weights for policy 0, policy_version 109674 (0.0007) [2023-12-26 16:09:53,835][105620] Updated weights for policy 1, policy_version 110255 (0.0007) [2023-12-26 16:09:53,883][105620] Updated weights for policy 1, policy_version 110265 (0.0005) [2023-12-26 16:09:53,884][105692] Updated weights for policy 0, policy_version 109684 (0.0010) [2023-12-26 16:09:53,937][105692] Updated weights for policy 0, policy_version 109694 (0.0008) [2023-12-26 16:09:53,939][105620] Updated weights for policy 1, policy_version 110275 (0.0006) [2023-12-26 16:09:54,006][105692] Updated weights for policy 0, policy_version 109704 (0.0008) [2023-12-26 16:09:54,575][105620] Updated weights for policy 1, policy_version 110285 (0.0005) [2023-12-26 16:09:54,640][105620] Updated weights for policy 1, policy_version 110295 (0.0009) [2023-12-26 16:09:54,689][105692] Updated weights for policy 0, policy_version 109714 (0.0007) [2023-12-26 16:09:54,696][105620] Updated weights for policy 1, policy_version 110305 (0.0006) [2023-12-26 16:09:54,750][105692] Updated weights for policy 0, policy_version 109724 (0.0006) [2023-12-26 16:09:54,810][105692] Updated weights for policy 0, policy_version 109734 (0.0005) [2023-12-26 16:09:55,311][105620] Updated weights for policy 1, policy_version 110315 (0.0007) [2023-12-26 16:09:55,352][105692] Updated weights for policy 0, policy_version 109744 (0.0009) [2023-12-26 16:09:55,365][105620] Updated weights for policy 1, policy_version 110325 (0.0010) [2023-12-26 16:09:55,403][105692] Updated weights for policy 0, policy_version 109754 (0.0010) [2023-12-26 16:09:55,416][105620] Updated weights for policy 1, policy_version 110335 (0.0010) [2023-12-26 16:09:55,452][105692] Updated weights for policy 0, policy_version 109764 (0.0010) [2023-12-26 16:09:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 56360960. Throughput: 0: 9594.9, 1: 9883.3. Samples: 56373240. Policy #0 lag: (min: 10.0, avg: 18.9, max: 42.0) [2023-12-26 16:09:56,063][104569] Avg episode reward: [(0, '8900.515'), (1, '8470.431')] [2023-12-26 16:09:56,094][105620] Updated weights for policy 1, policy_version 110345 (0.0010) [2023-12-26 16:09:56,141][105620] Updated weights for policy 1, policy_version 110355 (0.0005) [2023-12-26 16:09:56,187][105620] Updated weights for policy 1, policy_version 110365 (0.0005) [2023-12-26 16:09:56,198][105692] Updated weights for policy 0, policy_version 109774 (0.0010) [2023-12-26 16:09:56,235][105620] Updated weights for policy 1, policy_version 110375 (0.0005) [2023-12-26 16:09:56,248][105692] Updated weights for policy 0, policy_version 109784 (0.0010) [2023-12-26 16:09:56,302][105692] Updated weights for policy 0, policy_version 109794 (0.0010) [2023-12-26 16:09:56,764][105620] Updated weights for policy 1, policy_version 110385 (0.0007) [2023-12-26 16:09:56,822][105620] Updated weights for policy 1, policy_version 110395 (0.0010) [2023-12-26 16:09:56,870][105620] Updated weights for policy 1, policy_version 110405 (0.0010) [2023-12-26 16:09:56,996][105692] Updated weights for policy 0, policy_version 109804 (0.0008) [2023-12-26 16:09:57,048][105692] Updated weights for policy 0, policy_version 109814 (0.0005) [2023-12-26 16:09:57,108][105692] Updated weights for policy 0, policy_version 109824 (0.0005) [2023-12-26 16:09:57,574][105620] Updated weights for policy 1, policy_version 110415 (0.0010) [2023-12-26 16:09:57,628][105620] Updated weights for policy 1, policy_version 110425 (0.0010) [2023-12-26 16:09:57,671][105620] Updated weights for policy 1, policy_version 110435 (0.0010) [2023-12-26 16:09:57,678][105692] Updated weights for policy 0, policy_version 109834 (0.0006) [2023-12-26 16:09:57,735][105692] Updated weights for policy 0, policy_version 109844 (0.0010) [2023-12-26 16:09:57,789][105692] Updated weights for policy 0, policy_version 109854 (0.0010) [2023-12-26 16:09:57,840][105692] Updated weights for policy 0, policy_version 109864 (0.0010) [2023-12-26 16:09:58,414][105620] Updated weights for policy 1, policy_version 110445 (0.0009) [2023-12-26 16:09:58,484][105620] Updated weights for policy 1, policy_version 110455 (0.0009) [2023-12-26 16:09:58,552][105620] Updated weights for policy 1, policy_version 110465 (0.0008) [2023-12-26 16:09:58,593][105692] Updated weights for policy 0, policy_version 109874 (0.0007) [2023-12-26 16:09:58,657][105692] Updated weights for policy 0, policy_version 109884 (0.0007) [2023-12-26 16:09:58,724][105692] Updated weights for policy 0, policy_version 109894 (0.0008) [2023-12-26 16:09:59,364][105620] Updated weights for policy 1, policy_version 110475 (0.0008) [2023-12-26 16:09:59,431][105620] Updated weights for policy 1, policy_version 110485 (0.0008) [2023-12-26 16:09:59,464][105692] Updated weights for policy 0, policy_version 109904 (0.0010) [2023-12-26 16:09:59,488][105620] Updated weights for policy 1, policy_version 110495 (0.0006) [2023-12-26 16:09:59,515][105692] Updated weights for policy 0, policy_version 109914 (0.0007) [2023-12-26 16:09:59,563][105692] Updated weights for policy 0, policy_version 109924 (0.0009) [2023-12-26 16:10:00,269][105692] Updated weights for policy 0, policy_version 109934 (0.0009) [2023-12-26 16:10:00,274][105620] Updated weights for policy 1, policy_version 110505 (0.0007) [2023-12-26 16:10:00,319][105692] Updated weights for policy 0, policy_version 109944 (0.0008) [2023-12-26 16:10:00,335][105620] Updated weights for policy 1, policy_version 110515 (0.0008) [2023-12-26 16:10:00,372][105692] Updated weights for policy 0, policy_version 109954 (0.0007) [2023-12-26 16:10:00,395][105620] Updated weights for policy 1, policy_version 110525 (0.0008) [2023-12-26 16:10:00,453][105620] Updated weights for policy 1, policy_version 110535 (0.0008) [2023-12-26 16:10:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 56459264. Throughput: 0: 9684.3, 1: 9873.1. Samples: 56434784. Policy #0 lag: (min: 10.0, avg: 18.9, max: 42.0) [2023-12-26 16:10:01,062][104569] Avg episode reward: [(0, '8910.770'), (1, '8643.057')] [2023-12-26 16:10:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000109960_28155904.pth... [2023-12-26 16:10:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000110536_28303360.pth... [2023-12-26 16:10:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000108840_27869184.pth [2023-12-26 16:10:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000109416_28016640.pth [2023-12-26 16:10:01,132][105692] Updated weights for policy 0, policy_version 109964 (0.0008) [2023-12-26 16:10:01,190][105692] Updated weights for policy 0, policy_version 109974 (0.0009) [2023-12-26 16:10:01,218][105620] Updated weights for policy 1, policy_version 110545 (0.0010) [2023-12-26 16:10:01,248][105692] Updated weights for policy 0, policy_version 109984 (0.0009) [2023-12-26 16:10:01,276][105620] Updated weights for policy 1, policy_version 110555 (0.0007) [2023-12-26 16:10:01,340][105620] Updated weights for policy 1, policy_version 110565 (0.0008) [2023-12-26 16:10:02,005][105692] Updated weights for policy 0, policy_version 109994 (0.0008) [2023-12-26 16:10:02,068][105692] Updated weights for policy 0, policy_version 110004 (0.0007) [2023-12-26 16:10:02,104][105620] Updated weights for policy 1, policy_version 110575 (0.0008) [2023-12-26 16:10:02,118][105692] Updated weights for policy 0, policy_version 110014 (0.0007) [2023-12-26 16:10:02,159][105620] Updated weights for policy 1, policy_version 110585 (0.0007) [2023-12-26 16:10:02,165][105692] Updated weights for policy 0, policy_version 110024 (0.0008) [2023-12-26 16:10:02,221][105620] Updated weights for policy 1, policy_version 110595 (0.0008) [2023-12-26 16:10:02,860][105692] Updated weights for policy 0, policy_version 110034 (0.0006) [2023-12-26 16:10:02,915][105692] Updated weights for policy 0, policy_version 110044 (0.0010) [2023-12-26 16:10:02,934][105620] Updated weights for policy 1, policy_version 110605 (0.0007) [2023-12-26 16:10:02,963][105692] Updated weights for policy 0, policy_version 110054 (0.0010) [2023-12-26 16:10:02,993][105620] Updated weights for policy 1, policy_version 110615 (0.0005) [2023-12-26 16:10:03,066][105620] Updated weights for policy 1, policy_version 110625 (0.0005) [2023-12-26 16:10:03,594][105620] Updated weights for policy 1, policy_version 110635 (0.0005) [2023-12-26 16:10:03,600][105692] Updated weights for policy 0, policy_version 110064 (0.0007) [2023-12-26 16:10:03,647][105620] Updated weights for policy 1, policy_version 110645 (0.0005) [2023-12-26 16:10:03,650][105692] Updated weights for policy 0, policy_version 110074 (0.0006) [2023-12-26 16:10:03,697][105620] Updated weights for policy 1, policy_version 110655 (0.0005) [2023-12-26 16:10:03,698][105692] Updated weights for policy 0, policy_version 110084 (0.0010) [2023-12-26 16:10:04,347][105620] Updated weights for policy 1, policy_version 110665 (0.0006) [2023-12-26 16:10:04,399][105620] Updated weights for policy 1, policy_version 110675 (0.0009) [2023-12-26 16:10:04,446][105692] Updated weights for policy 0, policy_version 110094 (0.0008) [2023-12-26 16:10:04,464][105620] Updated weights for policy 1, policy_version 110685 (0.0007) [2023-12-26 16:10:04,503][105692] Updated weights for policy 0, policy_version 110104 (0.0006) [2023-12-26 16:10:04,522][105620] Updated weights for policy 1, policy_version 110695 (0.0007) [2023-12-26 16:10:04,535][105585] KL-divergence is very high: 148.2479 [2023-12-26 16:10:04,569][105692] Updated weights for policy 0, policy_version 110114 (0.0008) [2023-12-26 16:10:04,585][105585] KL-divergence is very high: 157.3062 [2023-12-26 16:10:05,219][105620] Updated weights for policy 1, policy_version 110705 (0.0008) [2023-12-26 16:10:05,267][105620] Updated weights for policy 1, policy_version 110715 (0.0005) [2023-12-26 16:10:05,275][105692] Updated weights for policy 0, policy_version 110124 (0.0008) [2023-12-26 16:10:05,319][105620] Updated weights for policy 1, policy_version 110725 (0.0005) [2023-12-26 16:10:05,337][105692] Updated weights for policy 0, policy_version 110134 (0.0007) [2023-12-26 16:10:05,401][105692] Updated weights for policy 0, policy_version 110144 (0.0009) [2023-12-26 16:10:05,960][105620] Updated weights for policy 1, policy_version 110735 (0.0006) [2023-12-26 16:10:06,010][105620] Updated weights for policy 1, policy_version 110745 (0.0006) [2023-12-26 16:10:06,055][105620] Updated weights for policy 1, policy_version 110755 (0.0005) [2023-12-26 16:10:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 56557568. Throughput: 0: 9781.7, 1: 9873.7. Samples: 56551664. Policy #0 lag: (min: 10.0, avg: 18.9, max: 42.0) [2023-12-26 16:10:06,063][104569] Avg episode reward: [(0, '9000.544'), (1, '8471.582')] [2023-12-26 16:10:06,240][105692] Updated weights for policy 0, policy_version 110154 (0.0009) [2023-12-26 16:10:06,287][105692] Updated weights for policy 0, policy_version 110164 (0.0008) [2023-12-26 16:10:06,336][105692] Updated weights for policy 0, policy_version 110174 (0.0009) [2023-12-26 16:10:06,393][105692] Updated weights for policy 0, policy_version 110184 (0.0010) [2023-12-26 16:10:06,725][105620] Updated weights for policy 1, policy_version 110765 (0.0007) [2023-12-26 16:10:06,780][105620] Updated weights for policy 1, policy_version 110775 (0.0009) [2023-12-26 16:10:06,828][105620] Updated weights for policy 1, policy_version 110785 (0.0008) [2023-12-26 16:10:07,148][105692] Updated weights for policy 0, policy_version 110194 (0.0007) [2023-12-26 16:10:07,215][105692] Updated weights for policy 0, policy_version 110204 (0.0010) [2023-12-26 16:10:07,277][105692] Updated weights for policy 0, policy_version 110214 (0.0010) [2023-12-26 16:10:07,579][105620] Updated weights for policy 1, policy_version 110795 (0.0009) [2023-12-26 16:10:07,641][105620] Updated weights for policy 1, policy_version 110805 (0.0009) [2023-12-26 16:10:07,699][105620] Updated weights for policy 1, policy_version 110815 (0.0009) [2023-12-26 16:10:08,036][105692] Updated weights for policy 0, policy_version 110224 (0.0009) [2023-12-26 16:10:08,090][105692] Updated weights for policy 0, policy_version 110234 (0.0008) [2023-12-26 16:10:08,145][105692] Updated weights for policy 0, policy_version 110244 (0.0009) [2023-12-26 16:10:08,424][105620] Updated weights for policy 1, policy_version 110825 (0.0009) [2023-12-26 16:10:08,479][105620] Updated weights for policy 1, policy_version 110835 (0.0008) [2023-12-26 16:10:08,535][105620] Updated weights for policy 1, policy_version 110845 (0.0008) [2023-12-26 16:10:08,599][105620] Updated weights for policy 1, policy_version 110855 (0.0009) [2023-12-26 16:10:08,901][105692] Updated weights for policy 0, policy_version 110254 (0.0009) [2023-12-26 16:10:08,952][105692] Updated weights for policy 0, policy_version 110264 (0.0009) [2023-12-26 16:10:09,007][105692] Updated weights for policy 0, policy_version 110274 (0.0009) [2023-12-26 16:10:09,391][105620] Updated weights for policy 1, policy_version 110865 (0.0009) [2023-12-26 16:10:09,460][105620] Updated weights for policy 1, policy_version 110875 (0.0009) [2023-12-26 16:10:09,518][105620] Updated weights for policy 1, policy_version 110885 (0.0007) [2023-12-26 16:10:09,829][105692] Updated weights for policy 0, policy_version 110284 (0.0009) [2023-12-26 16:10:09,893][105692] Updated weights for policy 0, policy_version 110294 (0.0009) [2023-12-26 16:10:09,952][105692] Updated weights for policy 0, policy_version 110304 (0.0009) [2023-12-26 16:10:10,284][105620] Updated weights for policy 1, policy_version 110895 (0.0009) [2023-12-26 16:10:10,339][105620] Updated weights for policy 1, policy_version 110905 (0.0009) [2023-12-26 16:10:10,395][105620] Updated weights for policy 1, policy_version 110915 (0.0010) [2023-12-26 16:10:10,703][105692] Updated weights for policy 0, policy_version 110314 (0.0009) [2023-12-26 16:10:10,758][105692] Updated weights for policy 0, policy_version 110324 (0.0009) [2023-12-26 16:10:10,806][105692] Updated weights for policy 0, policy_version 110334 (0.0009) [2023-12-26 16:10:10,856][105692] Updated weights for policy 0, policy_version 110344 (0.0009) [2023-12-26 16:10:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 56655872. Throughput: 0: 9766.5, 1: 9798.1. Samples: 56664960. Policy #0 lag: (min: 10.0, avg: 18.9, max: 42.0) [2023-12-26 16:10:11,062][104569] Avg episode reward: [(0, '8816.525'), (1, '8229.741')] [2023-12-26 16:10:11,181][105620] Updated weights for policy 1, policy_version 110925 (0.0009) [2023-12-26 16:10:11,240][105620] Updated weights for policy 1, policy_version 110935 (0.0009) [2023-12-26 16:10:11,304][105620] Updated weights for policy 1, policy_version 110945 (0.0008) [2023-12-26 16:10:11,699][105692] Updated weights for policy 0, policy_version 110354 (0.0010) [2023-12-26 16:10:11,766][105692] Updated weights for policy 0, policy_version 110364 (0.0009) [2023-12-26 16:10:11,813][105692] Updated weights for policy 0, policy_version 110374 (0.0009) [2023-12-26 16:10:12,052][105620] Updated weights for policy 1, policy_version 110955 (0.0008) [2023-12-26 16:10:12,113][105620] Updated weights for policy 1, policy_version 110965 (0.0009) [2023-12-26 16:10:12,172][105620] Updated weights for policy 1, policy_version 110975 (0.0010) [2023-12-26 16:10:12,540][105692] Updated weights for policy 0, policy_version 110384 (0.0007) [2023-12-26 16:10:12,602][105692] Updated weights for policy 0, policy_version 110394 (0.0007) [2023-12-26 16:10:12,660][105692] Updated weights for policy 0, policy_version 110404 (0.0007) [2023-12-26 16:10:12,987][105620] Updated weights for policy 1, policy_version 110985 (0.0011) [2023-12-26 16:10:13,045][105620] Updated weights for policy 1, policy_version 110995 (0.0010) [2023-12-26 16:10:13,107][105620] Updated weights for policy 1, policy_version 111005 (0.0009) [2023-12-26 16:10:13,166][105620] Updated weights for policy 1, policy_version 111015 (0.0009) [2023-12-26 16:10:13,360][105692] Updated weights for policy 0, policy_version 110414 (0.0008) [2023-12-26 16:10:13,424][105692] Updated weights for policy 0, policy_version 110424 (0.0006) [2023-12-26 16:10:13,471][105692] Updated weights for policy 0, policy_version 110434 (0.0008) [2023-12-26 16:10:13,934][105620] Updated weights for policy 1, policy_version 111025 (0.0009) [2023-12-26 16:10:13,989][105620] Updated weights for policy 1, policy_version 111035 (0.0009) [2023-12-26 16:10:14,047][105620] Updated weights for policy 1, policy_version 111045 (0.0009) [2023-12-26 16:10:14,201][105692] Updated weights for policy 0, policy_version 110444 (0.0007) [2023-12-26 16:10:14,254][105692] Updated weights for policy 0, policy_version 110454 (0.0006) [2023-12-26 16:10:14,317][105692] Updated weights for policy 0, policy_version 110464 (0.0008) [2023-12-26 16:10:14,796][105620] Updated weights for policy 1, policy_version 111055 (0.0007) [2023-12-26 16:10:14,868][105620] Updated weights for policy 1, policy_version 111065 (0.0010) [2023-12-26 16:10:14,932][105620] Updated weights for policy 1, policy_version 111075 (0.0010) [2023-12-26 16:10:15,009][105692] Updated weights for policy 0, policy_version 110474 (0.0009) [2023-12-26 16:10:15,074][105692] Updated weights for policy 0, policy_version 110484 (0.0009) [2023-12-26 16:10:15,139][105692] Updated weights for policy 0, policy_version 110494 (0.0009) [2023-12-26 16:10:15,200][105692] Updated weights for policy 0, policy_version 110504 (0.0009) [2023-12-26 16:10:15,690][105620] Updated weights for policy 1, policy_version 111085 (0.0008) [2023-12-26 16:10:15,745][105620] Updated weights for policy 1, policy_version 111095 (0.0009) [2023-12-26 16:10:15,791][105620] Updated weights for policy 1, policy_version 111105 (0.0009) [2023-12-26 16:10:15,921][105692] Updated weights for policy 0, policy_version 110514 (0.0008) [2023-12-26 16:10:15,976][105692] Updated weights for policy 0, policy_version 110524 (0.0009) [2023-12-26 16:10:16,030][105692] Updated weights for policy 0, policy_version 110535 (0.0010) [2023-12-26 16:10:16,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 56754176. Throughput: 0: 9714.3, 1: 9695.9. Samples: 56719760. Policy #0 lag: (min: 10.0, avg: 18.9, max: 42.0) [2023-12-26 16:10:16,062][104569] Avg episode reward: [(0, '8816.128'), (1, '6872.711')] [2023-12-26 16:10:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000110536_28303360.pth... [2023-12-26 16:10:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000111112_28450816.pth... [2023-12-26 16:10:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000109384_28008448.pth [2023-12-26 16:10:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000109960_28155904.pth [2023-12-26 16:10:16,524][105620] Updated weights for policy 1, policy_version 111115 (0.0009) [2023-12-26 16:10:16,585][105620] Updated weights for policy 1, policy_version 111125 (0.0009) [2023-12-26 16:10:16,646][105620] Updated weights for policy 1, policy_version 111135 (0.0008) [2023-12-26 16:10:16,792][105692] Updated weights for policy 0, policy_version 110545 (0.0009) [2023-12-26 16:10:16,839][105692] Updated weights for policy 0, policy_version 110555 (0.0009) [2023-12-26 16:10:16,885][105692] Updated weights for policy 0, policy_version 110565 (0.0008) [2023-12-26 16:10:17,394][105620] Updated weights for policy 1, policy_version 111145 (0.0009) [2023-12-26 16:10:17,444][105620] Updated weights for policy 1, policy_version 111155 (0.0008) [2023-12-26 16:10:17,500][105620] Updated weights for policy 1, policy_version 111165 (0.0009) [2023-12-26 16:10:17,554][105620] Updated weights for policy 1, policy_version 111175 (0.0010) [2023-12-26 16:10:17,625][105692] Updated weights for policy 0, policy_version 110575 (0.0007) [2023-12-26 16:10:17,679][105692] Updated weights for policy 0, policy_version 110585 (0.0007) [2023-12-26 16:10:17,744][105692] Updated weights for policy 0, policy_version 110595 (0.0006) [2023-12-26 16:10:18,375][105692] Updated weights for policy 0, policy_version 110605 (0.0006) [2023-12-26 16:10:18,424][105620] Updated weights for policy 1, policy_version 111185 (0.0007) [2023-12-26 16:10:18,435][105692] Updated weights for policy 0, policy_version 110615 (0.0007) [2023-12-26 16:10:18,486][105620] Updated weights for policy 1, policy_version 111195 (0.0008) [2023-12-26 16:10:18,491][105692] Updated weights for policy 0, policy_version 110625 (0.0006) [2023-12-26 16:10:18,551][105620] Updated weights for policy 1, policy_version 111205 (0.0006) [2023-12-26 16:10:19,233][105692] Updated weights for policy 0, policy_version 110635 (0.0008) [2023-12-26 16:10:19,291][105692] Updated weights for policy 0, policy_version 110645 (0.0009) [2023-12-26 16:10:19,326][105620] Updated weights for policy 1, policy_version 111215 (0.0007) [2023-12-26 16:10:19,360][105692] Updated weights for policy 0, policy_version 110655 (0.0008) [2023-12-26 16:10:19,397][105620] Updated weights for policy 1, policy_version 111225 (0.0008) [2023-12-26 16:10:19,455][105620] Updated weights for policy 1, policy_version 111235 (0.0009) [2023-12-26 16:10:20,100][105692] Updated weights for policy 0, policy_version 110665 (0.0007) [2023-12-26 16:10:20,128][105620] Updated weights for policy 1, policy_version 111245 (0.0008) [2023-12-26 16:10:20,159][105692] Updated weights for policy 0, policy_version 110675 (0.0008) [2023-12-26 16:10:20,186][105620] Updated weights for policy 1, policy_version 111255 (0.0006) [2023-12-26 16:10:20,225][105692] Updated weights for policy 0, policy_version 110685 (0.0008) [2023-12-26 16:10:20,236][105620] Updated weights for policy 1, policy_version 111265 (0.0007) [2023-12-26 16:10:20,288][105692] Updated weights for policy 0, policy_version 110695 (0.0008) [2023-12-26 16:10:20,951][105692] Updated weights for policy 0, policy_version 110705 (0.0008) [2023-12-26 16:10:21,000][105620] Updated weights for policy 1, policy_version 111275 (0.0008) [2023-12-26 16:10:21,009][105692] Updated weights for policy 0, policy_version 110715 (0.0007) [2023-12-26 16:10:21,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 56836096. Throughput: 0: 9658.0, 1: 9661.2. Samples: 56832844. Policy #0 lag: (min: 10.0, avg: 18.9, max: 42.0) [2023-12-26 16:10:21,063][104569] Avg episode reward: [(0, '8809.011'), (1, '6609.230')] [2023-12-26 16:10:21,068][105620] Updated weights for policy 1, policy_version 111285 (0.0011) [2023-12-26 16:10:21,073][105692] Updated weights for policy 0, policy_version 110725 (0.0006) [2023-12-26 16:10:21,127][105620] Updated weights for policy 1, policy_version 111295 (0.0011) [2023-12-26 16:10:21,853][105692] Updated weights for policy 0, policy_version 110735 (0.0009) [2023-12-26 16:10:21,903][105692] Updated weights for policy 0, policy_version 110745 (0.0008) [2023-12-26 16:10:21,915][105620] Updated weights for policy 1, policy_version 111305 (0.0010) [2023-12-26 16:10:21,962][105692] Updated weights for policy 0, policy_version 110755 (0.0007) [2023-12-26 16:10:21,976][105620] Updated weights for policy 1, policy_version 111315 (0.0007) [2023-12-26 16:10:22,034][105620] Updated weights for policy 1, policy_version 111325 (0.0009) [2023-12-26 16:10:22,096][105620] Updated weights for policy 1, policy_version 111335 (0.0010) [2023-12-26 16:10:22,774][105692] Updated weights for policy 0, policy_version 110765 (0.0008) [2023-12-26 16:10:22,780][105620] Updated weights for policy 1, policy_version 111345 (0.0006) [2023-12-26 16:10:22,843][105692] Updated weights for policy 0, policy_version 110775 (0.0008) [2023-12-26 16:10:22,844][105620] Updated weights for policy 1, policy_version 111355 (0.0005) [2023-12-26 16:10:22,900][105692] Updated weights for policy 0, policy_version 110785 (0.0007) [2023-12-26 16:10:22,912][105620] Updated weights for policy 1, policy_version 111365 (0.0006) [2023-12-26 16:10:23,531][105620] Updated weights for policy 1, policy_version 111375 (0.0008) [2023-12-26 16:10:23,593][105620] Updated weights for policy 1, policy_version 111385 (0.0008) [2023-12-26 16:10:23,659][105620] Updated weights for policy 1, policy_version 111395 (0.0009) [2023-12-26 16:10:23,675][105692] Updated weights for policy 0, policy_version 110795 (0.0009) [2023-12-26 16:10:23,722][105692] Updated weights for policy 0, policy_version 110805 (0.0008) [2023-12-26 16:10:23,784][105692] Updated weights for policy 0, policy_version 110815 (0.0008) [2023-12-26 16:10:24,283][105620] Updated weights for policy 1, policy_version 111405 (0.0009) [2023-12-26 16:10:24,347][105620] Updated weights for policy 1, policy_version 111415 (0.0009) [2023-12-26 16:10:24,410][105620] Updated weights for policy 1, policy_version 111425 (0.0008) [2023-12-26 16:10:24,477][105692] Updated weights for policy 0, policy_version 110825 (0.0007) [2023-12-26 16:10:24,537][105692] Updated weights for policy 0, policy_version 110835 (0.0008) [2023-12-26 16:10:24,593][105692] Updated weights for policy 0, policy_version 110845 (0.0008) [2023-12-26 16:10:24,643][105692] Updated weights for policy 0, policy_version 110855 (0.0009) [2023-12-26 16:10:25,133][105620] Updated weights for policy 1, policy_version 111435 (0.0007) [2023-12-26 16:10:25,183][105620] Updated weights for policy 1, policy_version 111445 (0.0006) [2023-12-26 16:10:25,231][105620] Updated weights for policy 1, policy_version 111455 (0.0005) [2023-12-26 16:10:25,446][105692] Updated weights for policy 0, policy_version 110865 (0.0009) [2023-12-26 16:10:25,498][105692] Updated weights for policy 0, policy_version 110875 (0.0009) [2023-12-26 16:10:25,556][105692] Updated weights for policy 0, policy_version 110885 (0.0008) [2023-12-26 16:10:25,781][105620] Updated weights for policy 1, policy_version 111465 (0.0006) [2023-12-26 16:10:25,852][105620] Updated weights for policy 1, policy_version 111475 (0.0005) [2023-12-26 16:10:25,912][105620] Updated weights for policy 1, policy_version 111485 (0.0005) [2023-12-26 16:10:25,976][105620] Updated weights for policy 1, policy_version 111495 (0.0005) [2023-12-26 16:10:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 56942592. Throughput: 0: 9629.9, 1: 9751.8. Samples: 56949268. Policy #0 lag: (min: 10.0, avg: 18.9, max: 42.0) [2023-12-26 16:10:26,063][104569] Avg episode reward: [(0, '8819.338'), (1, '7222.676')] [2023-12-26 16:10:26,331][105692] Updated weights for policy 0, policy_version 110895 (0.0007) [2023-12-26 16:10:26,391][105692] Updated weights for policy 0, policy_version 110905 (0.0005) [2023-12-26 16:10:26,450][105692] Updated weights for policy 0, policy_version 110915 (0.0005) [2023-12-26 16:10:26,597][105620] Updated weights for policy 1, policy_version 111505 (0.0009) [2023-12-26 16:10:26,646][105620] Updated weights for policy 1, policy_version 111515 (0.0009) [2023-12-26 16:10:26,709][105620] Updated weights for policy 1, policy_version 111525 (0.0009) [2023-12-26 16:10:27,054][105692] Updated weights for policy 0, policy_version 110925 (0.0007) [2023-12-26 16:10:27,100][105692] Updated weights for policy 0, policy_version 110935 (0.0008) [2023-12-26 16:10:27,154][105692] Updated weights for policy 0, policy_version 110945 (0.0009) [2023-12-26 16:10:27,372][105620] Updated weights for policy 1, policy_version 111535 (0.0010) [2023-12-26 16:10:27,416][105620] Updated weights for policy 1, policy_version 111545 (0.0010) [2023-12-26 16:10:27,463][105620] Updated weights for policy 1, policy_version 111555 (0.0010) [2023-12-26 16:10:27,917][105692] Updated weights for policy 0, policy_version 110955 (0.0008) [2023-12-26 16:10:27,960][105692] Updated weights for policy 0, policy_version 110965 (0.0005) [2023-12-26 16:10:28,016][105692] Updated weights for policy 0, policy_version 110975 (0.0005) [2023-12-26 16:10:28,182][105620] Updated weights for policy 1, policy_version 111565 (0.0010) [2023-12-26 16:10:28,230][105620] Updated weights for policy 1, policy_version 111575 (0.0010) [2023-12-26 16:10:28,287][105620] Updated weights for policy 1, policy_version 111585 (0.0010) [2023-12-26 16:10:28,553][105692] Updated weights for policy 0, policy_version 110985 (0.0005) [2023-12-26 16:10:28,612][105692] Updated weights for policy 0, policy_version 110995 (0.0006) [2023-12-26 16:10:28,657][105692] Updated weights for policy 0, policy_version 111005 (0.0008) [2023-12-26 16:10:28,718][105692] Updated weights for policy 0, policy_version 111015 (0.0008) [2023-12-26 16:10:28,978][105620] Updated weights for policy 1, policy_version 111595 (0.0009) [2023-12-26 16:10:29,026][105620] Updated weights for policy 1, policy_version 111605 (0.0005) [2023-12-26 16:10:29,083][105620] Updated weights for policy 1, policy_version 111615 (0.0005) [2023-12-26 16:10:29,487][105692] Updated weights for policy 0, policy_version 111025 (0.0008) [2023-12-26 16:10:29,546][105692] Updated weights for policy 0, policy_version 111035 (0.0008) [2023-12-26 16:10:29,603][105692] Updated weights for policy 0, policy_version 111045 (0.0009) [2023-12-26 16:10:29,709][105620] Updated weights for policy 1, policy_version 111625 (0.0005) [2023-12-26 16:10:29,760][105620] Updated weights for policy 1, policy_version 111635 (0.0005) [2023-12-26 16:10:29,806][105620] Updated weights for policy 1, policy_version 111645 (0.0005) [2023-12-26 16:10:29,869][105620] Updated weights for policy 1, policy_version 111655 (0.0008) [2023-12-26 16:10:30,414][105692] Updated weights for policy 0, policy_version 111055 (0.0009) [2023-12-26 16:10:30,467][105692] Updated weights for policy 0, policy_version 111065 (0.0008) [2023-12-26 16:10:30,517][105620] Updated weights for policy 1, policy_version 111665 (0.0007) [2023-12-26 16:10:30,530][105692] Updated weights for policy 0, policy_version 111075 (0.0008) [2023-12-26 16:10:30,572][105620] Updated weights for policy 1, policy_version 111675 (0.0008) [2023-12-26 16:10:30,618][105620] Updated weights for policy 1, policy_version 111685 (0.0008) [2023-12-26 16:10:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 57040896. Throughput: 0: 9692.4, 1: 9834.0. Samples: 57011480. Policy #0 lag: (min: 10.0, avg: 18.9, max: 42.0) [2023-12-26 16:10:31,063][104569] Avg episode reward: [(0, '8555.157'), (1, '7586.607')] [2023-12-26 16:10:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000111080_28442624.pth... [2023-12-26 16:10:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000111688_28598272.pth... [2023-12-26 16:10:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000110536_28303360.pth [2023-12-26 16:10:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000109960_28155904.pth [2023-12-26 16:10:31,334][105692] Updated weights for policy 0, policy_version 111085 (0.0006) [2023-12-26 16:10:31,341][105620] Updated weights for policy 1, policy_version 111695 (0.0006) [2023-12-26 16:10:31,403][105692] Updated weights for policy 0, policy_version 111095 (0.0007) [2023-12-26 16:10:31,404][105620] Updated weights for policy 1, policy_version 111705 (0.0008) [2023-12-26 16:10:31,457][105620] Updated weights for policy 1, policy_version 111715 (0.0007) [2023-12-26 16:10:31,467][105692] Updated weights for policy 0, policy_version 111105 (0.0007) [2023-12-26 16:10:32,132][105692] Updated weights for policy 0, policy_version 111115 (0.0008) [2023-12-26 16:10:32,190][105692] Updated weights for policy 0, policy_version 111125 (0.0009) [2023-12-26 16:10:32,240][105620] Updated weights for policy 1, policy_version 111725 (0.0007) [2023-12-26 16:10:32,247][105692] Updated weights for policy 0, policy_version 111135 (0.0008) [2023-12-26 16:10:32,305][105620] Updated weights for policy 1, policy_version 111735 (0.0008) [2023-12-26 16:10:32,368][105620] Updated weights for policy 1, policy_version 111745 (0.0008) [2023-12-26 16:10:32,962][105692] Updated weights for policy 0, policy_version 111145 (0.0009) [2023-12-26 16:10:33,011][105692] Updated weights for policy 0, policy_version 111155 (0.0008) [2023-12-26 16:10:33,057][105692] Updated weights for policy 0, policy_version 111165 (0.0009) [2023-12-26 16:10:33,104][105692] Updated weights for policy 0, policy_version 111175 (0.0008) [2023-12-26 16:10:33,111][105620] Updated weights for policy 1, policy_version 111755 (0.0008) [2023-12-26 16:10:33,159][105620] Updated weights for policy 1, policy_version 111765 (0.0008) [2023-12-26 16:10:33,213][105620] Updated weights for policy 1, policy_version 111775 (0.0009) [2023-12-26 16:10:33,750][105692] Updated weights for policy 0, policy_version 111185 (0.0005) [2023-12-26 16:10:33,806][105692] Updated weights for policy 0, policy_version 111195 (0.0005) [2023-12-26 16:10:33,859][105692] Updated weights for policy 0, policy_version 111205 (0.0005) [2023-12-26 16:10:33,878][105620] Updated weights for policy 1, policy_version 111785 (0.0009) [2023-12-26 16:10:33,930][105620] Updated weights for policy 1, policy_version 111795 (0.0009) [2023-12-26 16:10:33,984][105620] Updated weights for policy 1, policy_version 111806 (0.0010) [2023-12-26 16:10:34,035][105620] Updated weights for policy 1, policy_version 111816 (0.0009) [2023-12-26 16:10:34,380][105692] Updated weights for policy 0, policy_version 111215 (0.0006) [2023-12-26 16:10:34,447][105692] Updated weights for policy 0, policy_version 111225 (0.0009) [2023-12-26 16:10:34,516][105692] Updated weights for policy 0, policy_version 111235 (0.0009) [2023-12-26 16:10:34,897][105620] Updated weights for policy 1, policy_version 111826 (0.0009) [2023-12-26 16:10:34,954][105620] Updated weights for policy 1, policy_version 111836 (0.0009) [2023-12-26 16:10:35,008][105620] Updated weights for policy 1, policy_version 111846 (0.0009) [2023-12-26 16:10:35,224][105692] Updated weights for policy 0, policy_version 111245 (0.0009) [2023-12-26 16:10:35,282][105692] Updated weights for policy 0, policy_version 111255 (0.0009) [2023-12-26 16:10:35,340][105692] Updated weights for policy 0, policy_version 111265 (0.0009) [2023-12-26 16:10:35,771][105620] Updated weights for policy 1, policy_version 111856 (0.0009) [2023-12-26 16:10:35,817][105620] Updated weights for policy 1, policy_version 111866 (0.0008) [2023-12-26 16:10:35,864][105620] Updated weights for policy 1, policy_version 111876 (0.0009) [2023-12-26 16:10:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 57139200. Throughput: 0: 9685.1, 1: 9817.9. Samples: 57129584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:10:36,062][104569] Avg episode reward: [(0, '7927.725'), (1, '7269.652')] [2023-12-26 16:10:36,064][105692] Updated weights for policy 0, policy_version 111275 (0.0008) [2023-12-26 16:10:36,130][105692] Updated weights for policy 0, policy_version 111285 (0.0007) [2023-12-26 16:10:36,195][105692] Updated weights for policy 0, policy_version 111295 (0.0008) [2023-12-26 16:10:36,661][105620] Updated weights for policy 1, policy_version 111886 (0.0009) [2023-12-26 16:10:36,722][105620] Updated weights for policy 1, policy_version 111896 (0.0009) [2023-12-26 16:10:36,789][105620] Updated weights for policy 1, policy_version 111906 (0.0006) [2023-12-26 16:10:36,928][105692] Updated weights for policy 0, policy_version 111305 (0.0009) [2023-12-26 16:10:36,995][105692] Updated weights for policy 0, policy_version 111315 (0.0008) [2023-12-26 16:10:37,055][105692] Updated weights for policy 0, policy_version 111325 (0.0008) [2023-12-26 16:10:37,113][105692] Updated weights for policy 0, policy_version 111335 (0.0008) [2023-12-26 16:10:37,520][105620] Updated weights for policy 1, policy_version 111916 (0.0010) [2023-12-26 16:10:37,572][105620] Updated weights for policy 1, policy_version 111926 (0.0010) [2023-12-26 16:10:37,625][105620] Updated weights for policy 1, policy_version 111936 (0.0011) [2023-12-26 16:10:37,884][105692] Updated weights for policy 0, policy_version 111345 (0.0007) [2023-12-26 16:10:37,946][105692] Updated weights for policy 0, policy_version 111355 (0.0005) [2023-12-26 16:10:38,006][105692] Updated weights for policy 0, policy_version 111365 (0.0005) [2023-12-26 16:10:38,284][105620] Updated weights for policy 1, policy_version 111946 (0.0010) [2023-12-26 16:10:38,341][105620] Updated weights for policy 1, policy_version 111956 (0.0008) [2023-12-26 16:10:38,403][105620] Updated weights for policy 1, policy_version 111966 (0.0009) [2023-12-26 16:10:38,460][105620] Updated weights for policy 1, policy_version 111976 (0.0008) [2023-12-26 16:10:38,678][105692] Updated weights for policy 0, policy_version 111375 (0.0008) [2023-12-26 16:10:38,740][105692] Updated weights for policy 0, policy_version 111385 (0.0009) [2023-12-26 16:10:38,805][105692] Updated weights for policy 0, policy_version 111395 (0.0009) [2023-12-26 16:10:39,201][105620] Updated weights for policy 1, policy_version 111986 (0.0009) [2023-12-26 16:10:39,267][105620] Updated weights for policy 1, policy_version 111996 (0.0009) [2023-12-26 16:10:39,324][105620] Updated weights for policy 1, policy_version 112006 (0.0007) [2023-12-26 16:10:39,560][105692] Updated weights for policy 0, policy_version 111405 (0.0009) [2023-12-26 16:10:39,624][105692] Updated weights for policy 0, policy_version 111415 (0.0009) [2023-12-26 16:10:39,690][105692] Updated weights for policy 0, policy_version 111425 (0.0009) [2023-12-26 16:10:40,088][105620] Updated weights for policy 1, policy_version 112016 (0.0009) [2023-12-26 16:10:40,149][105620] Updated weights for policy 1, policy_version 112026 (0.0009) [2023-12-26 16:10:40,215][105620] Updated weights for policy 1, policy_version 112036 (0.0007) [2023-12-26 16:10:40,402][105692] Updated weights for policy 0, policy_version 111435 (0.0008) [2023-12-26 16:10:40,462][105692] Updated weights for policy 0, policy_version 111445 (0.0005) [2023-12-26 16:10:40,527][105692] Updated weights for policy 0, policy_version 111455 (0.0005) [2023-12-26 16:10:41,058][105692] Updated weights for policy 0, policy_version 111465 (0.0006) [2023-12-26 16:10:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 57229312. Throughput: 0: 9645.9, 1: 9666.5. Samples: 57242296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:10:41,063][104569] Avg episode reward: [(0, '6890.955'), (1, '7431.143')] [2023-12-26 16:10:41,097][105620] Updated weights for policy 1, policy_version 112046 (0.0010) [2023-12-26 16:10:41,117][105692] Updated weights for policy 0, policy_version 111475 (0.0007) [2023-12-26 16:10:41,161][105620] Updated weights for policy 1, policy_version 112056 (0.0008) [2023-12-26 16:10:41,184][105692] Updated weights for policy 0, policy_version 111485 (0.0009) [2023-12-26 16:10:41,220][105620] Updated weights for policy 1, policy_version 112066 (0.0007) [2023-12-26 16:10:41,248][105692] Updated weights for policy 0, policy_version 111495 (0.0007) [2023-12-26 16:10:42,005][105620] Updated weights for policy 1, policy_version 112076 (0.0009) [2023-12-26 16:10:42,012][105692] Updated weights for policy 0, policy_version 111505 (0.0007) [2023-12-26 16:10:42,062][105620] Updated weights for policy 1, policy_version 112086 (0.0011) [2023-12-26 16:10:42,069][105692] Updated weights for policy 0, policy_version 111515 (0.0007) [2023-12-26 16:10:42,125][105620] Updated weights for policy 1, policy_version 112096 (0.0009) [2023-12-26 16:10:42,134][105692] Updated weights for policy 0, policy_version 111525 (0.0008) [2023-12-26 16:10:42,847][105620] Updated weights for policy 1, policy_version 112106 (0.0010) [2023-12-26 16:10:42,909][105620] Updated weights for policy 1, policy_version 112116 (0.0010) [2023-12-26 16:10:42,919][105692] Updated weights for policy 0, policy_version 111535 (0.0006) [2023-12-26 16:10:42,965][105692] Updated weights for policy 0, policy_version 111545 (0.0008) [2023-12-26 16:10:42,967][105620] Updated weights for policy 1, policy_version 112126 (0.0010) [2023-12-26 16:10:43,014][105692] Updated weights for policy 0, policy_version 111555 (0.0008) [2023-12-26 16:10:43,025][105620] Updated weights for policy 1, policy_version 112136 (0.0010) [2023-12-26 16:10:43,709][105620] Updated weights for policy 1, policy_version 112146 (0.0011) [2023-12-26 16:10:43,764][105620] Updated weights for policy 1, policy_version 112156 (0.0010) [2023-12-26 16:10:43,820][105692] Updated weights for policy 0, policy_version 111565 (0.0007) [2023-12-26 16:10:43,821][105620] Updated weights for policy 1, policy_version 112166 (0.0010) [2023-12-26 16:10:43,873][105692] Updated weights for policy 0, policy_version 111575 (0.0007) [2023-12-26 16:10:43,939][105692] Updated weights for policy 0, policy_version 111585 (0.0007) [2023-12-26 16:10:44,574][105620] Updated weights for policy 1, policy_version 112176 (0.0011) [2023-12-26 16:10:44,630][105620] Updated weights for policy 1, policy_version 112186 (0.0011) [2023-12-26 16:10:44,631][105692] Updated weights for policy 0, policy_version 111595 (0.0006) [2023-12-26 16:10:44,690][105620] Updated weights for policy 1, policy_version 112196 (0.0011) [2023-12-26 16:10:44,696][105692] Updated weights for policy 0, policy_version 111605 (0.0008) [2023-12-26 16:10:44,755][105692] Updated weights for policy 0, policy_version 111615 (0.0007) [2023-12-26 16:10:45,453][105620] Updated weights for policy 1, policy_version 112206 (0.0011) [2023-12-26 16:10:45,467][105692] Updated weights for policy 0, policy_version 111625 (0.0008) [2023-12-26 16:10:45,508][105620] Updated weights for policy 1, policy_version 112216 (0.0007) [2023-12-26 16:10:45,517][105692] Updated weights for policy 0, policy_version 111635 (0.0008) [2023-12-26 16:10:45,554][105620] Updated weights for policy 1, policy_version 112226 (0.0006) [2023-12-26 16:10:45,568][105692] Updated weights for policy 0, policy_version 111645 (0.0006) [2023-12-26 16:10:45,621][105692] Updated weights for policy 0, policy_version 111655 (0.0008) [2023-12-26 16:10:46,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 57327616. Throughput: 0: 9607.4, 1: 9603.5. Samples: 57299280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:10:46,063][104569] Avg episode reward: [(0, '6824.324'), (1, '8021.814')] [2023-12-26 16:10:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000111656_28590080.pth... [2023-12-26 16:10:46,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000112232_28737536.pth... [2023-12-26 16:10:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000111112_28450816.pth [2023-12-26 16:10:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000110536_28303360.pth [2023-12-26 16:10:46,259][105620] Updated weights for policy 1, policy_version 112236 (0.0008) [2023-12-26 16:10:46,312][105620] Updated weights for policy 1, policy_version 112246 (0.0009) [2023-12-26 16:10:46,367][105620] Updated weights for policy 1, policy_version 112256 (0.0009) [2023-12-26 16:10:46,389][105692] Updated weights for policy 0, policy_version 111665 (0.0009) [2023-12-26 16:10:46,442][105692] Updated weights for policy 0, policy_version 111675 (0.0007) [2023-12-26 16:10:46,501][105692] Updated weights for policy 0, policy_version 111685 (0.0008) [2023-12-26 16:10:46,998][105620] Updated weights for policy 1, policy_version 112266 (0.0010) [2023-12-26 16:10:47,060][105620] Updated weights for policy 1, policy_version 112276 (0.0009) [2023-12-26 16:10:47,122][105620] Updated weights for policy 1, policy_version 112286 (0.0006) [2023-12-26 16:10:47,178][105620] Updated weights for policy 1, policy_version 112296 (0.0006) [2023-12-26 16:10:47,344][105692] Updated weights for policy 0, policy_version 111695 (0.0009) [2023-12-26 16:10:47,402][105692] Updated weights for policy 0, policy_version 111705 (0.0009) [2023-12-26 16:10:47,452][105692] Updated weights for policy 0, policy_version 111715 (0.0008) [2023-12-26 16:10:47,826][105620] Updated weights for policy 1, policy_version 112306 (0.0007) [2023-12-26 16:10:47,873][105620] Updated weights for policy 1, policy_version 112316 (0.0009) [2023-12-26 16:10:47,919][105620] Updated weights for policy 1, policy_version 112326 (0.0008) [2023-12-26 16:10:48,236][105692] Updated weights for policy 0, policy_version 111725 (0.0009) [2023-12-26 16:10:48,298][105692] Updated weights for policy 0, policy_version 111735 (0.0008) [2023-12-26 16:10:48,354][105692] Updated weights for policy 0, policy_version 111745 (0.0009) [2023-12-26 16:10:48,664][105620] Updated weights for policy 1, policy_version 112336 (0.0008) [2023-12-26 16:10:48,728][105620] Updated weights for policy 1, policy_version 112346 (0.0008) [2023-12-26 16:10:48,797][105620] Updated weights for policy 1, policy_version 112356 (0.0008) [2023-12-26 16:10:49,150][105692] Updated weights for policy 0, policy_version 111755 (0.0009) [2023-12-26 16:10:49,206][105692] Updated weights for policy 0, policy_version 111765 (0.0008) [2023-12-26 16:10:49,274][105692] Updated weights for policy 0, policy_version 111775 (0.0008) [2023-12-26 16:10:49,512][105620] Updated weights for policy 1, policy_version 112366 (0.0009) [2023-12-26 16:10:49,570][105620] Updated weights for policy 1, policy_version 112376 (0.0010) [2023-12-26 16:10:49,629][105620] Updated weights for policy 1, policy_version 112386 (0.0010) [2023-12-26 16:10:50,041][105692] Updated weights for policy 0, policy_version 111785 (0.0009) [2023-12-26 16:10:50,099][105692] Updated weights for policy 0, policy_version 111796 (0.0010) [2023-12-26 16:10:50,158][105692] Updated weights for policy 0, policy_version 111806 (0.0009) [2023-12-26 16:10:50,208][105692] Updated weights for policy 0, policy_version 111816 (0.0005) [2023-12-26 16:10:50,432][105620] Updated weights for policy 1, policy_version 112396 (0.0010) [2023-12-26 16:10:50,499][105620] Updated weights for policy 1, policy_version 112406 (0.0009) [2023-12-26 16:10:50,565][105620] Updated weights for policy 1, policy_version 112416 (0.0010) [2023-12-26 16:10:50,885][105692] Updated weights for policy 0, policy_version 111826 (0.0010) [2023-12-26 16:10:50,930][105692] Updated weights for policy 0, policy_version 111836 (0.0010) [2023-12-26 16:10:50,979][105692] Updated weights for policy 0, policy_version 111846 (0.0010) [2023-12-26 16:10:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 57425920. Throughput: 0: 9530.9, 1: 9608.5. Samples: 57412932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:10:51,063][104569] Avg episode reward: [(0, '7468.091'), (1, '8205.749')] [2023-12-26 16:10:51,271][105620] Updated weights for policy 1, policy_version 112426 (0.0012) [2023-12-26 16:10:51,324][105620] Updated weights for policy 1, policy_version 112436 (0.0010) [2023-12-26 16:10:51,385][105620] Updated weights for policy 1, policy_version 112446 (0.0011) [2023-12-26 16:10:51,446][105620] Updated weights for policy 1, policy_version 112456 (0.0010) [2023-12-26 16:10:51,696][105692] Updated weights for policy 0, policy_version 111856 (0.0010) [2023-12-26 16:10:51,762][105692] Updated weights for policy 0, policy_version 111866 (0.0011) [2023-12-26 16:10:51,820][105692] Updated weights for policy 0, policy_version 111876 (0.0007) [2023-12-26 16:10:52,236][105620] Updated weights for policy 1, policy_version 112466 (0.0011) [2023-12-26 16:10:52,303][105620] Updated weights for policy 1, policy_version 112476 (0.0010) [2023-12-26 16:10:52,366][105620] Updated weights for policy 1, policy_version 112486 (0.0010) [2023-12-26 16:10:52,512][105692] Updated weights for policy 0, policy_version 111886 (0.0008) [2023-12-26 16:10:52,578][105692] Updated weights for policy 0, policy_version 111896 (0.0009) [2023-12-26 16:10:52,640][105692] Updated weights for policy 0, policy_version 111906 (0.0009) [2023-12-26 16:10:53,110][105620] Updated weights for policy 1, policy_version 112496 (0.0009) [2023-12-26 16:10:53,161][105620] Updated weights for policy 1, policy_version 112506 (0.0009) [2023-12-26 16:10:53,219][105620] Updated weights for policy 1, policy_version 112516 (0.0009) [2023-12-26 16:10:53,337][105692] Updated weights for policy 0, policy_version 111916 (0.0006) [2023-12-26 16:10:53,389][105692] Updated weights for policy 0, policy_version 111926 (0.0009) [2023-12-26 16:10:53,449][105692] Updated weights for policy 0, policy_version 111936 (0.0005) [2023-12-26 16:10:54,003][105620] Updated weights for policy 1, policy_version 112526 (0.0007) [2023-12-26 16:10:54,014][105692] Updated weights for policy 0, policy_version 111946 (0.0006) [2023-12-26 16:10:54,056][105620] Updated weights for policy 1, policy_version 112537 (0.0008) [2023-12-26 16:10:54,060][105692] Updated weights for policy 0, policy_version 111956 (0.0005) [2023-12-26 16:10:54,108][105620] Updated weights for policy 1, policy_version 112547 (0.0008) [2023-12-26 16:10:54,110][105692] Updated weights for policy 0, policy_version 111966 (0.0005) [2023-12-26 16:10:54,162][105692] Updated weights for policy 0, policy_version 111976 (0.0006) [2023-12-26 16:10:54,827][105692] Updated weights for policy 0, policy_version 111986 (0.0005) [2023-12-26 16:10:54,889][105692] Updated weights for policy 0, policy_version 111996 (0.0009) [2023-12-26 16:10:54,895][105620] Updated weights for policy 1, policy_version 112557 (0.0007) [2023-12-26 16:10:54,956][105692] Updated weights for policy 0, policy_version 112006 (0.0009) [2023-12-26 16:10:54,957][105620] Updated weights for policy 1, policy_version 112567 (0.0009) [2023-12-26 16:10:55,017][105620] Updated weights for policy 1, policy_version 112577 (0.0011) [2023-12-26 16:10:55,631][105620] Updated weights for policy 1, policy_version 112587 (0.0010) [2023-12-26 16:10:55,679][105692] Updated weights for policy 0, policy_version 112016 (0.0007) [2023-12-26 16:10:55,696][105620] Updated weights for policy 1, policy_version 112597 (0.0010) [2023-12-26 16:10:55,738][105692] Updated weights for policy 0, policy_version 112026 (0.0007) [2023-12-26 16:10:55,759][105620] Updated weights for policy 1, policy_version 112607 (0.0007) [2023-12-26 16:10:55,794][105692] Updated weights for policy 0, policy_version 112036 (0.0007) [2023-12-26 16:10:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 57524224. Throughput: 0: 9666.0, 1: 9573.2. Samples: 57530724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:10:56,062][104569] Avg episode reward: [(0, '8033.131'), (1, '8039.087')] [2023-12-26 16:10:56,430][105692] Updated weights for policy 0, policy_version 112046 (0.0005) [2023-12-26 16:10:56,462][105620] Updated weights for policy 1, policy_version 112617 (0.0006) [2023-12-26 16:10:56,483][105692] Updated weights for policy 0, policy_version 112056 (0.0005) [2023-12-26 16:10:56,522][105620] Updated weights for policy 1, policy_version 112627 (0.0009) [2023-12-26 16:10:56,530][105692] Updated weights for policy 0, policy_version 112066 (0.0005) [2023-12-26 16:10:56,587][105620] Updated weights for policy 1, policy_version 112637 (0.0009) [2023-12-26 16:10:56,652][105620] Updated weights for policy 1, policy_version 112647 (0.0008) [2023-12-26 16:10:57,218][105692] Updated weights for policy 0, policy_version 112076 (0.0007) [2023-12-26 16:10:57,269][105692] Updated weights for policy 0, policy_version 112086 (0.0008) [2023-12-26 16:10:57,295][105620] Updated weights for policy 1, policy_version 112657 (0.0006) [2023-12-26 16:10:57,328][105692] Updated weights for policy 0, policy_version 112096 (0.0008) [2023-12-26 16:10:57,353][105620] Updated weights for policy 1, policy_version 112667 (0.0007) [2023-12-26 16:10:57,417][105620] Updated weights for policy 1, policy_version 112677 (0.0010) [2023-12-26 16:10:57,930][105692] Updated weights for policy 0, policy_version 112106 (0.0007) [2023-12-26 16:10:57,993][105692] Updated weights for policy 0, policy_version 112116 (0.0008) [2023-12-26 16:10:58,040][105692] Updated weights for policy 0, policy_version 112126 (0.0009) [2023-12-26 16:10:58,094][105692] Updated weights for policy 0, policy_version 112136 (0.0009) [2023-12-26 16:10:58,204][105620] Updated weights for policy 1, policy_version 112687 (0.0009) [2023-12-26 16:10:58,257][105620] Updated weights for policy 1, policy_version 112697 (0.0009) [2023-12-26 16:10:58,311][105620] Updated weights for policy 1, policy_version 112707 (0.0009) [2023-12-26 16:10:58,865][105692] Updated weights for policy 0, policy_version 112146 (0.0013) [2023-12-26 16:10:58,930][105692] Updated weights for policy 0, policy_version 112156 (0.0008) [2023-12-26 16:10:58,998][105692] Updated weights for policy 0, policy_version 112166 (0.0007) [2023-12-26 16:10:59,225][105620] Updated weights for policy 1, policy_version 112717 (0.0009) [2023-12-26 16:10:59,295][105620] Updated weights for policy 1, policy_version 112727 (0.0008) [2023-12-26 16:10:59,352][105620] Updated weights for policy 1, policy_version 112737 (0.0006) [2023-12-26 16:10:59,671][105692] Updated weights for policy 0, policy_version 112176 (0.0006) [2023-12-26 16:10:59,724][105692] Updated weights for policy 0, policy_version 112186 (0.0008) [2023-12-26 16:10:59,778][105692] Updated weights for policy 0, policy_version 112196 (0.0008) [2023-12-26 16:11:00,005][105620] Updated weights for policy 1, policy_version 112747 (0.0007) [2023-12-26 16:11:00,064][105620] Updated weights for policy 1, policy_version 112757 (0.0010) [2023-12-26 16:11:00,112][105620] Updated weights for policy 1, policy_version 112767 (0.0010) [2023-12-26 16:11:00,547][105692] Updated weights for policy 0, policy_version 112206 (0.0008) [2023-12-26 16:11:00,610][105692] Updated weights for policy 0, policy_version 112216 (0.0008) [2023-12-26 16:11:00,658][105692] Updated weights for policy 0, policy_version 112226 (0.0008) [2023-12-26 16:11:00,807][105620] Updated weights for policy 1, policy_version 112777 (0.0010) [2023-12-26 16:11:00,863][105620] Updated weights for policy 1, policy_version 112787 (0.0010) [2023-12-26 16:11:00,913][105620] Updated weights for policy 1, policy_version 112797 (0.0010) [2023-12-26 16:11:00,965][105620] Updated weights for policy 1, policy_version 112807 (0.0010) [2023-12-26 16:11:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 57622528. Throughput: 0: 9746.1, 1: 9575.8. Samples: 57589244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:11:01,062][104569] Avg episode reward: [(0, '8561.187'), (1, '8129.623')] [2023-12-26 16:11:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000112232_28737536.pth... [2023-12-26 16:11:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000112808_28884992.pth... [2023-12-26 16:11:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000111688_28598272.pth [2023-12-26 16:11:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000111080_28442624.pth [2023-12-26 16:11:01,439][105692] Updated weights for policy 0, policy_version 112236 (0.0007) [2023-12-26 16:11:01,507][105692] Updated weights for policy 0, policy_version 112246 (0.0005) [2023-12-26 16:11:01,569][105692] Updated weights for policy 0, policy_version 112256 (0.0005) [2023-12-26 16:11:01,663][105620] Updated weights for policy 1, policy_version 112817 (0.0012) [2023-12-26 16:11:01,734][105620] Updated weights for policy 1, policy_version 112827 (0.0011) [2023-12-26 16:11:01,785][105620] Updated weights for policy 1, policy_version 112837 (0.0010) [2023-12-26 16:11:02,232][105692] Updated weights for policy 0, policy_version 112266 (0.0007) [2023-12-26 16:11:02,284][105692] Updated weights for policy 0, policy_version 112276 (0.0008) [2023-12-26 16:11:02,336][105692] Updated weights for policy 0, policy_version 112286 (0.0008) [2023-12-26 16:11:02,401][105692] Updated weights for policy 0, policy_version 112296 (0.0008) [2023-12-26 16:11:02,525][105620] Updated weights for policy 1, policy_version 112847 (0.0010) [2023-12-26 16:11:02,580][105620] Updated weights for policy 1, policy_version 112857 (0.0010) [2023-12-26 16:11:02,642][105620] Updated weights for policy 1, policy_version 112867 (0.0010) [2023-12-26 16:11:03,157][105692] Updated weights for policy 0, policy_version 112306 (0.0008) [2023-12-26 16:11:03,201][105692] Updated weights for policy 0, policy_version 112316 (0.0008) [2023-12-26 16:11:03,255][105692] Updated weights for policy 0, policy_version 112326 (0.0008) [2023-12-26 16:11:03,391][105620] Updated weights for policy 1, policy_version 112877 (0.0010) [2023-12-26 16:11:03,448][105620] Updated weights for policy 1, policy_version 112887 (0.0010) [2023-12-26 16:11:03,509][105620] Updated weights for policy 1, policy_version 112897 (0.0010) [2023-12-26 16:11:03,949][105692] Updated weights for policy 0, policy_version 112336 (0.0006) [2023-12-26 16:11:04,008][105692] Updated weights for policy 0, policy_version 112346 (0.0007) [2023-12-26 16:11:04,057][105692] Updated weights for policy 0, policy_version 112356 (0.0010) [2023-12-26 16:11:04,252][105620] Updated weights for policy 1, policy_version 112907 (0.0010) [2023-12-26 16:11:04,308][105620] Updated weights for policy 1, policy_version 112917 (0.0011) [2023-12-26 16:11:04,372][105620] Updated weights for policy 1, policy_version 112927 (0.0011) [2023-12-26 16:11:04,772][105692] Updated weights for policy 0, policy_version 112366 (0.0010) [2023-12-26 16:11:04,824][105692] Updated weights for policy 0, policy_version 112376 (0.0010) [2023-12-26 16:11:04,886][105692] Updated weights for policy 0, policy_version 112386 (0.0010) [2023-12-26 16:11:05,105][105620] Updated weights for policy 1, policy_version 112937 (0.0010) [2023-12-26 16:11:05,163][105620] Updated weights for policy 1, policy_version 112947 (0.0005) [2023-12-26 16:11:05,209][105620] Updated weights for policy 1, policy_version 112957 (0.0005) [2023-12-26 16:11:05,260][105620] Updated weights for policy 1, policy_version 112967 (0.0005) [2023-12-26 16:11:05,597][105692] Updated weights for policy 0, policy_version 112396 (0.0011) [2023-12-26 16:11:05,664][105692] Updated weights for policy 0, policy_version 112406 (0.0009) [2023-12-26 16:11:05,726][105692] Updated weights for policy 0, policy_version 112416 (0.0011) [2023-12-26 16:11:05,917][105620] Updated weights for policy 1, policy_version 112977 (0.0010) [2023-12-26 16:11:05,984][105620] Updated weights for policy 1, policy_version 112987 (0.0011) [2023-12-26 16:11:06,048][105620] Updated weights for policy 1, policy_version 112997 (0.0011) [2023-12-26 16:11:06,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 57712640. Throughput: 0: 9731.0, 1: 9643.3. Samples: 57704692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:11:06,063][104569] Avg episode reward: [(0, '8914.201'), (1, '7855.750')] [2023-12-26 16:11:06,387][105692] Updated weights for policy 0, policy_version 112426 (0.0010) [2023-12-26 16:11:06,452][105692] Updated weights for policy 0, policy_version 112436 (0.0009) [2023-12-26 16:11:06,520][105692] Updated weights for policy 0, policy_version 112446 (0.0008) [2023-12-26 16:11:06,592][105692] Updated weights for policy 0, policy_version 112456 (0.0005) [2023-12-26 16:11:06,825][105620] Updated weights for policy 1, policy_version 113007 (0.0010) [2023-12-26 16:11:06,889][105620] Updated weights for policy 1, policy_version 113017 (0.0009) [2023-12-26 16:11:06,954][105620] Updated weights for policy 1, policy_version 113027 (0.0005) [2023-12-26 16:11:07,157][105692] Updated weights for policy 0, policy_version 112466 (0.0005) [2023-12-26 16:11:07,210][105692] Updated weights for policy 0, policy_version 112476 (0.0005) [2023-12-26 16:11:07,264][105692] Updated weights for policy 0, policy_version 112486 (0.0005) [2023-12-26 16:11:07,659][105620] Updated weights for policy 1, policy_version 113037 (0.0005) [2023-12-26 16:11:07,713][105620] Updated weights for policy 1, policy_version 113047 (0.0006) [2023-12-26 16:11:07,766][105620] Updated weights for policy 1, policy_version 113057 (0.0005) [2023-12-26 16:11:07,855][105692] Updated weights for policy 0, policy_version 112496 (0.0009) [2023-12-26 16:11:07,915][105692] Updated weights for policy 0, policy_version 112506 (0.0005) [2023-12-26 16:11:07,975][105692] Updated weights for policy 0, policy_version 112516 (0.0005) [2023-12-26 16:11:08,301][105620] Updated weights for policy 1, policy_version 113067 (0.0006) [2023-12-26 16:11:08,363][105620] Updated weights for policy 1, policy_version 113077 (0.0008) [2023-12-26 16:11:08,437][105620] Updated weights for policy 1, policy_version 113087 (0.0009) [2023-12-26 16:11:08,608][105692] Updated weights for policy 0, policy_version 112526 (0.0006) [2023-12-26 16:11:08,660][105692] Updated weights for policy 0, policy_version 112536 (0.0008) [2023-12-26 16:11:08,712][105692] Updated weights for policy 0, policy_version 112546 (0.0009) [2023-12-26 16:11:09,287][105620] Updated weights for policy 1, policy_version 113097 (0.0009) [2023-12-26 16:11:09,342][105692] Updated weights for policy 0, policy_version 112556 (0.0007) [2023-12-26 16:11:09,353][105620] Updated weights for policy 1, policy_version 113107 (0.0008) [2023-12-26 16:11:09,405][105692] Updated weights for policy 0, policy_version 112566 (0.0007) [2023-12-26 16:11:09,419][105620] Updated weights for policy 1, policy_version 113117 (0.0008) [2023-12-26 16:11:09,468][105692] Updated weights for policy 0, policy_version 112576 (0.0008) [2023-12-26 16:11:09,485][105620] Updated weights for policy 1, policy_version 113127 (0.0008) [2023-12-26 16:11:10,244][105692] Updated weights for policy 0, policy_version 112586 (0.0007) [2023-12-26 16:11:10,249][105620] Updated weights for policy 1, policy_version 113137 (0.0007) [2023-12-26 16:11:10,293][105692] Updated weights for policy 0, policy_version 112596 (0.0009) [2023-12-26 16:11:10,306][105620] Updated weights for policy 1, policy_version 113147 (0.0007) [2023-12-26 16:11:10,355][105692] Updated weights for policy 0, policy_version 112606 (0.0008) [2023-12-26 16:11:10,366][105620] Updated weights for policy 1, policy_version 113157 (0.0007) [2023-12-26 16:11:10,412][105692] Updated weights for policy 0, policy_version 112616 (0.0007) [2023-12-26 16:11:11,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 57810944. Throughput: 0: 9869.6, 1: 9582.3. Samples: 57824604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:11:11,063][104569] Avg episode reward: [(0, '8724.733'), (1, '7669.833')] [2023-12-26 16:11:11,103][105620] Updated weights for policy 1, policy_version 113167 (0.0008) [2023-12-26 16:11:11,161][105692] Updated weights for policy 0, policy_version 112626 (0.0007) [2023-12-26 16:11:11,176][105620] Updated weights for policy 1, policy_version 113177 (0.0008) [2023-12-26 16:11:11,227][105692] Updated weights for policy 0, policy_version 112636 (0.0007) [2023-12-26 16:11:11,246][105620] Updated weights for policy 1, policy_version 113187 (0.0007) [2023-12-26 16:11:11,295][105692] Updated weights for policy 0, policy_version 112646 (0.0007) [2023-12-26 16:11:11,885][105620] Updated weights for policy 1, policy_version 113197 (0.0009) [2023-12-26 16:11:11,947][105620] Updated weights for policy 1, policy_version 113207 (0.0009) [2023-12-26 16:11:12,009][105620] Updated weights for policy 1, policy_version 113217 (0.0008) [2023-12-26 16:11:12,108][105692] Updated weights for policy 0, policy_version 112656 (0.0009) [2023-12-26 16:11:12,177][105692] Updated weights for policy 0, policy_version 112666 (0.0008) [2023-12-26 16:11:12,243][105692] Updated weights for policy 0, policy_version 112676 (0.0009) [2023-12-26 16:11:12,816][105620] Updated weights for policy 1, policy_version 113227 (0.0009) [2023-12-26 16:11:12,874][105620] Updated weights for policy 1, policy_version 113237 (0.0009) [2023-12-26 16:11:12,919][105692] Updated weights for policy 0, policy_version 112686 (0.0009) [2023-12-26 16:11:12,923][105620] Updated weights for policy 1, policy_version 113247 (0.0006) [2023-12-26 16:11:12,983][105692] Updated weights for policy 0, policy_version 112696 (0.0008) [2023-12-26 16:11:13,044][105692] Updated weights for policy 0, policy_version 112706 (0.0008) [2023-12-26 16:11:13,609][105692] Updated weights for policy 0, policy_version 112716 (0.0007) [2023-12-26 16:11:13,660][105692] Updated weights for policy 0, policy_version 112726 (0.0009) [2023-12-26 16:11:13,712][105692] Updated weights for policy 0, policy_version 112736 (0.0008) [2023-12-26 16:11:13,746][105620] Updated weights for policy 1, policy_version 113257 (0.0008) [2023-12-26 16:11:13,805][105620] Updated weights for policy 1, policy_version 113267 (0.0009) [2023-12-26 16:11:13,858][105620] Updated weights for policy 1, policy_version 113277 (0.0010) [2023-12-26 16:11:13,916][105620] Updated weights for policy 1, policy_version 113287 (0.0009) [2023-12-26 16:11:14,286][105692] Updated weights for policy 0, policy_version 112746 (0.0005) [2023-12-26 16:11:14,338][105692] Updated weights for policy 0, policy_version 112756 (0.0006) [2023-12-26 16:11:14,388][105692] Updated weights for policy 0, policy_version 112767 (0.0010) [2023-12-26 16:11:14,737][105620] Updated weights for policy 1, policy_version 113297 (0.0009) [2023-12-26 16:11:14,796][105620] Updated weights for policy 1, policy_version 113307 (0.0009) [2023-12-26 16:11:14,848][105620] Updated weights for policy 1, policy_version 113317 (0.0009) [2023-12-26 16:11:15,094][105692] Updated weights for policy 0, policy_version 112777 (0.0009) [2023-12-26 16:11:15,157][105692] Updated weights for policy 0, policy_version 112787 (0.0009) [2023-12-26 16:11:15,219][105692] Updated weights for policy 0, policy_version 112797 (0.0006) [2023-12-26 16:11:15,281][105692] Updated weights for policy 0, policy_version 112807 (0.0007) [2023-12-26 16:11:15,654][105620] Updated weights for policy 1, policy_version 113327 (0.0009) [2023-12-26 16:11:15,713][105620] Updated weights for policy 1, policy_version 113337 (0.0010) [2023-12-26 16:11:15,779][105620] Updated weights for policy 1, policy_version 113347 (0.0009) [2023-12-26 16:11:15,935][105692] Updated weights for policy 0, policy_version 112817 (0.0009) [2023-12-26 16:11:15,998][105692] Updated weights for policy 0, policy_version 112827 (0.0009) [2023-12-26 16:11:16,056][105692] Updated weights for policy 0, policy_version 112837 (0.0009) [2023-12-26 16:11:16,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 57909248. Throughput: 0: 9829.7, 1: 9496.3. Samples: 57881152. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:11:16,063][104569] Avg episode reward: [(0, '8016.041'), (1, '7939.566')] [2023-12-26 16:11:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000113352_29024256.pth... [2023-12-26 16:11:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000112840_28893184.pth... [2023-12-26 16:11:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000112232_28737536.pth [2023-12-26 16:11:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000111656_28590080.pth [2023-12-26 16:11:16,484][105620] Updated weights for policy 1, policy_version 113357 (0.0009) [2023-12-26 16:11:16,545][105620] Updated weights for policy 1, policy_version 113367 (0.0007) [2023-12-26 16:11:16,604][105620] Updated weights for policy 1, policy_version 113377 (0.0005) [2023-12-26 16:11:16,825][105692] Updated weights for policy 0, policy_version 112847 (0.0010) [2023-12-26 16:11:16,880][105692] Updated weights for policy 0, policy_version 112859 (0.0010) [2023-12-26 16:11:16,933][105692] Updated weights for policy 0, policy_version 112870 (0.0009) [2023-12-26 16:11:17,157][105620] Updated weights for policy 1, policy_version 113387 (0.0006) [2023-12-26 16:11:17,220][105620] Updated weights for policy 1, policy_version 113397 (0.0008) [2023-12-26 16:11:17,277][105620] Updated weights for policy 1, policy_version 113407 (0.0010) [2023-12-26 16:11:17,752][105692] Updated weights for policy 0, policy_version 112880 (0.0010) [2023-12-26 16:11:17,799][105692] Updated weights for policy 0, policy_version 112890 (0.0009) [2023-12-26 16:11:17,855][105692] Updated weights for policy 0, policy_version 112900 (0.0009) [2023-12-26 16:11:17,957][105620] Updated weights for policy 1, policy_version 113417 (0.0009) [2023-12-26 16:11:18,010][105620] Updated weights for policy 1, policy_version 113427 (0.0006) [2023-12-26 16:11:18,062][105620] Updated weights for policy 1, policy_version 113437 (0.0006) [2023-12-26 16:11:18,107][105620] Updated weights for policy 1, policy_version 113447 (0.0010) [2023-12-26 16:11:18,676][105692] Updated weights for policy 0, policy_version 112910 (0.0009) [2023-12-26 16:11:18,739][105692] Updated weights for policy 0, policy_version 112920 (0.0008) [2023-12-26 16:11:18,796][105692] Updated weights for policy 0, policy_version 112930 (0.0006) [2023-12-26 16:11:18,813][105620] Updated weights for policy 1, policy_version 113457 (0.0011) [2023-12-26 16:11:18,862][105620] Updated weights for policy 1, policy_version 113467 (0.0011) [2023-12-26 16:11:18,919][105620] Updated weights for policy 1, policy_version 113477 (0.0006) [2023-12-26 16:11:19,587][105620] Updated weights for policy 1, policy_version 113487 (0.0009) [2023-12-26 16:11:19,588][105692] Updated weights for policy 0, policy_version 112940 (0.0006) [2023-12-26 16:11:19,645][105692] Updated weights for policy 0, policy_version 112950 (0.0005) [2023-12-26 16:11:19,655][105620] Updated weights for policy 1, policy_version 113497 (0.0011) [2023-12-26 16:11:19,712][105692] Updated weights for policy 0, policy_version 112960 (0.0008) [2023-12-26 16:11:19,712][105620] Updated weights for policy 1, policy_version 113507 (0.0008) [2023-12-26 16:11:20,354][105692] Updated weights for policy 0, policy_version 112970 (0.0008) [2023-12-26 16:11:20,421][105692] Updated weights for policy 0, policy_version 112980 (0.0011) [2023-12-26 16:11:20,448][105620] Updated weights for policy 1, policy_version 113517 (0.0006) [2023-12-26 16:11:20,479][105692] Updated weights for policy 0, policy_version 112990 (0.0011) [2023-12-26 16:11:20,505][105620] Updated weights for policy 1, policy_version 113527 (0.0006) [2023-12-26 16:11:20,528][105692] Updated weights for policy 0, policy_version 113000 (0.0010) [2023-12-26 16:11:20,573][105620] Updated weights for policy 1, policy_version 113537 (0.0006) [2023-12-26 16:11:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 58007552. Throughput: 0: 9783.9, 1: 9516.1. Samples: 57998084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:11:21,063][104569] Avg episode reward: [(0, '8196.815'), (1, '8207.108')] [2023-12-26 16:11:21,295][105620] Updated weights for policy 1, policy_version 113547 (0.0008) [2023-12-26 16:11:21,322][105692] Updated weights for policy 0, policy_version 113010 (0.0011) [2023-12-26 16:11:21,370][105620] Updated weights for policy 1, policy_version 113557 (0.0007) [2023-12-26 16:11:21,388][105692] Updated weights for policy 0, policy_version 113020 (0.0010) [2023-12-26 16:11:21,423][105620] Updated weights for policy 1, policy_version 113567 (0.0006) [2023-12-26 16:11:21,452][105692] Updated weights for policy 0, policy_version 113030 (0.0010) [2023-12-26 16:11:22,225][105692] Updated weights for policy 0, policy_version 113040 (0.0006) [2023-12-26 16:11:22,244][105620] Updated weights for policy 1, policy_version 113577 (0.0009) [2023-12-26 16:11:22,292][105692] Updated weights for policy 0, policy_version 113050 (0.0008) [2023-12-26 16:11:22,304][105620] Updated weights for policy 1, policy_version 113587 (0.0008) [2023-12-26 16:11:22,352][105692] Updated weights for policy 0, policy_version 113060 (0.0009) [2023-12-26 16:11:22,375][105620] Updated weights for policy 1, policy_version 113597 (0.0007) [2023-12-26 16:11:22,433][105620] Updated weights for policy 1, policy_version 113607 (0.0009) [2023-12-26 16:11:22,987][105692] Updated weights for policy 0, policy_version 113070 (0.0007) [2023-12-26 16:11:23,051][105692] Updated weights for policy 0, policy_version 113080 (0.0007) [2023-12-26 16:11:23,114][105692] Updated weights for policy 0, policy_version 113090 (0.0007) [2023-12-26 16:11:23,224][105620] Updated weights for policy 1, policy_version 113617 (0.0009) [2023-12-26 16:11:23,297][105620] Updated weights for policy 1, policy_version 113627 (0.0010) [2023-12-26 16:11:23,361][105620] Updated weights for policy 1, policy_version 113637 (0.0009) [2023-12-26 16:11:23,736][105692] Updated weights for policy 0, policy_version 113100 (0.0006) [2023-12-26 16:11:23,784][105692] Updated weights for policy 0, policy_version 113110 (0.0008) [2023-12-26 16:11:23,831][105692] Updated weights for policy 0, policy_version 113120 (0.0008) [2023-12-26 16:11:24,104][105620] Updated weights for policy 1, policy_version 113647 (0.0010) [2023-12-26 16:11:24,166][105620] Updated weights for policy 1, policy_version 113657 (0.0011) [2023-12-26 16:11:24,235][105620] Updated weights for policy 1, policy_version 113667 (0.0011) [2023-12-26 16:11:24,542][105692] Updated weights for policy 0, policy_version 113130 (0.0008) [2023-12-26 16:11:24,595][105692] Updated weights for policy 0, policy_version 113140 (0.0009) [2023-12-26 16:11:24,654][105692] Updated weights for policy 0, policy_version 113150 (0.0009) [2023-12-26 16:11:24,710][105692] Updated weights for policy 0, policy_version 113160 (0.0008) [2023-12-26 16:11:24,871][105620] Updated weights for policy 1, policy_version 113677 (0.0008) [2023-12-26 16:11:24,926][105620] Updated weights for policy 1, policy_version 113687 (0.0005) [2023-12-26 16:11:24,972][105620] Updated weights for policy 1, policy_version 113697 (0.0005) [2023-12-26 16:11:25,325][105692] Updated weights for policy 0, policy_version 113170 (0.0006) [2023-12-26 16:11:25,391][105692] Updated weights for policy 0, policy_version 113180 (0.0009) [2023-12-26 16:11:25,458][105692] Updated weights for policy 0, policy_version 113190 (0.0011) [2023-12-26 16:11:25,625][105620] Updated weights for policy 1, policy_version 113707 (0.0005) [2023-12-26 16:11:25,670][105620] Updated weights for policy 1, policy_version 113717 (0.0010) [2023-12-26 16:11:25,717][105620] Updated weights for policy 1, policy_version 113727 (0.0010) [2023-12-26 16:11:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19466.5). Total num frames: 58105856. Throughput: 0: 9851.2, 1: 9561.3. Samples: 58115860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:11:26,063][104569] Avg episode reward: [(0, '8633.131'), (1, '8106.402')] [2023-12-26 16:11:26,126][105692] Updated weights for policy 0, policy_version 113200 (0.0010) [2023-12-26 16:11:26,181][105692] Updated weights for policy 0, policy_version 113210 (0.0010) [2023-12-26 16:11:26,239][105692] Updated weights for policy 0, policy_version 113220 (0.0010) [2023-12-26 16:11:26,305][105620] Updated weights for policy 1, policy_version 113737 (0.0010) [2023-12-26 16:11:26,367][105620] Updated weights for policy 1, policy_version 113747 (0.0009) [2023-12-26 16:11:26,452][105620] Updated weights for policy 1, policy_version 113757 (0.0010) [2023-12-26 16:11:26,499][105620] Updated weights for policy 1, policy_version 113767 (0.0010) [2023-12-26 16:11:26,890][105692] Updated weights for policy 0, policy_version 113230 (0.0008) [2023-12-26 16:11:26,938][105692] Updated weights for policy 0, policy_version 113240 (0.0010) [2023-12-26 16:11:26,996][105692] Updated weights for policy 0, policy_version 113250 (0.0010) [2023-12-26 16:11:27,144][105620] Updated weights for policy 1, policy_version 113777 (0.0006) [2023-12-26 16:11:27,202][105620] Updated weights for policy 1, policy_version 113787 (0.0006) [2023-12-26 16:11:27,250][105620] Updated weights for policy 1, policy_version 113797 (0.0010) [2023-12-26 16:11:27,698][105692] Updated weights for policy 0, policy_version 113260 (0.0007) [2023-12-26 16:11:27,750][105692] Updated weights for policy 0, policy_version 113270 (0.0006) [2023-12-26 16:11:27,793][105692] Updated weights for policy 0, policy_version 113280 (0.0006) [2023-12-26 16:11:27,907][105620] Updated weights for policy 1, policy_version 113807 (0.0010) [2023-12-26 16:11:27,954][105620] Updated weights for policy 1, policy_version 113817 (0.0006) [2023-12-26 16:11:28,008][105620] Updated weights for policy 1, policy_version 113827 (0.0006) [2023-12-26 16:11:28,475][105692] Updated weights for policy 0, policy_version 113290 (0.0010) [2023-12-26 16:11:28,543][105692] Updated weights for policy 0, policy_version 113300 (0.0008) [2023-12-26 16:11:28,605][105692] Updated weights for policy 0, policy_version 113310 (0.0008) [2023-12-26 16:11:28,660][105692] Updated weights for policy 0, policy_version 113320 (0.0008) [2023-12-26 16:11:28,691][105620] Updated weights for policy 1, policy_version 113837 (0.0010) [2023-12-26 16:11:28,747][105620] Updated weights for policy 1, policy_version 113847 (0.0010) [2023-12-26 16:11:28,805][105620] Updated weights for policy 1, policy_version 113857 (0.0010) [2023-12-26 16:11:29,286][105692] Updated weights for policy 0, policy_version 113330 (0.0006) [2023-12-26 16:11:29,348][105692] Updated weights for policy 0, policy_version 113340 (0.0008) [2023-12-26 16:11:29,413][105692] Updated weights for policy 0, policy_version 113350 (0.0011) [2023-12-26 16:11:29,490][105620] Updated weights for policy 1, policy_version 113867 (0.0009) [2023-12-26 16:11:29,549][105620] Updated weights for policy 1, policy_version 113877 (0.0005) [2023-12-26 16:11:29,601][105620] Updated weights for policy 1, policy_version 113887 (0.0005) [2023-12-26 16:11:29,991][105692] Updated weights for policy 0, policy_version 113360 (0.0009) [2023-12-26 16:11:30,043][105692] Updated weights for policy 0, policy_version 113370 (0.0009) [2023-12-26 16:11:30,091][105692] Updated weights for policy 0, policy_version 113380 (0.0009) [2023-12-26 16:11:30,250][105620] Updated weights for policy 1, policy_version 113897 (0.0006) [2023-12-26 16:11:30,301][105620] Updated weights for policy 1, policy_version 113907 (0.0009) [2023-12-26 16:11:30,362][105620] Updated weights for policy 1, policy_version 113917 (0.0009) [2023-12-26 16:11:30,417][105620] Updated weights for policy 1, policy_version 113927 (0.0009) [2023-12-26 16:11:30,738][105692] Updated weights for policy 0, policy_version 113390 (0.0007) [2023-12-26 16:11:30,788][105692] Updated weights for policy 0, policy_version 113400 (0.0009) [2023-12-26 16:11:30,838][105692] Updated weights for policy 0, policy_version 113410 (0.0009) [2023-12-26 16:11:31,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 58212352. Throughput: 0: 9885.3, 1: 9647.8. Samples: 58178264. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:11:31,062][104569] Avg episode reward: [(0, '8722.652'), (1, '7663.382')] [2023-12-26 16:11:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000113416_29040640.pth... [2023-12-26 16:11:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000113928_29171712.pth... [2023-12-26 16:11:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000112808_28884992.pth [2023-12-26 16:11:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000112232_28737536.pth [2023-12-26 16:11:31,234][105620] Updated weights for policy 1, policy_version 113937 (0.0009) [2023-12-26 16:11:31,294][105620] Updated weights for policy 1, policy_version 113947 (0.0009) [2023-12-26 16:11:31,357][105620] Updated weights for policy 1, policy_version 113957 (0.0009) [2023-12-26 16:11:31,575][105692] Updated weights for policy 0, policy_version 113420 (0.0009) [2023-12-26 16:11:31,644][105692] Updated weights for policy 0, policy_version 113430 (0.0009) [2023-12-26 16:11:31,715][105692] Updated weights for policy 0, policy_version 113440 (0.0011) [2023-12-26 16:11:32,126][105620] Updated weights for policy 1, policy_version 113967 (0.0008) [2023-12-26 16:11:32,177][105620] Updated weights for policy 1, policy_version 113977 (0.0008) [2023-12-26 16:11:32,226][105620] Updated weights for policy 1, policy_version 113987 (0.0009) [2023-12-26 16:11:32,460][105692] Updated weights for policy 0, policy_version 113450 (0.0011) [2023-12-26 16:11:32,523][105692] Updated weights for policy 0, policy_version 113460 (0.0010) [2023-12-26 16:11:32,585][105692] Updated weights for policy 0, policy_version 113470 (0.0009) [2023-12-26 16:11:32,647][105692] Updated weights for policy 0, policy_version 113480 (0.0009) [2023-12-26 16:11:33,002][105620] Updated weights for policy 1, policy_version 113997 (0.0008) [2023-12-26 16:11:33,060][105620] Updated weights for policy 1, policy_version 114007 (0.0009) [2023-12-26 16:11:33,114][105620] Updated weights for policy 1, policy_version 114017 (0.0008) [2023-12-26 16:11:33,282][105692] Updated weights for policy 0, policy_version 113490 (0.0009) [2023-12-26 16:11:33,334][105692] Updated weights for policy 0, policy_version 113500 (0.0008) [2023-12-26 16:11:33,394][105692] Updated weights for policy 0, policy_version 113510 (0.0005) [2023-12-26 16:11:33,879][105620] Updated weights for policy 1, policy_version 114027 (0.0009) [2023-12-26 16:11:33,935][105620] Updated weights for policy 1, policy_version 114037 (0.0008) [2023-12-26 16:11:33,985][105620] Updated weights for policy 1, policy_version 114047 (0.0008) [2023-12-26 16:11:34,087][105692] Updated weights for policy 0, policy_version 113520 (0.0008) [2023-12-26 16:11:34,140][105692] Updated weights for policy 0, policy_version 113531 (0.0011) [2023-12-26 16:11:34,199][105692] Updated weights for policy 0, policy_version 113541 (0.0009) [2023-12-26 16:11:34,667][105620] Updated weights for policy 1, policy_version 114057 (0.0009) [2023-12-26 16:11:34,722][105620] Updated weights for policy 1, policy_version 114067 (0.0009) [2023-12-26 16:11:34,783][105620] Updated weights for policy 1, policy_version 114077 (0.0009) [2023-12-26 16:11:34,849][105620] Updated weights for policy 1, policy_version 114087 (0.0009) [2023-12-26 16:11:35,009][105692] Updated weights for policy 0, policy_version 113551 (0.0009) [2023-12-26 16:11:35,068][105692] Updated weights for policy 0, policy_version 113561 (0.0009) [2023-12-26 16:11:35,132][105692] Updated weights for policy 0, policy_version 113571 (0.0007) [2023-12-26 16:11:35,660][105620] Updated weights for policy 1, policy_version 114097 (0.0009) [2023-12-26 16:11:35,721][105620] Updated weights for policy 1, policy_version 114107 (0.0008) [2023-12-26 16:11:35,771][105692] Updated weights for policy 0, policy_version 113581 (0.0008) [2023-12-26 16:11:35,785][105620] Updated weights for policy 1, policy_version 114117 (0.0007) [2023-12-26 16:11:35,824][105692] Updated weights for policy 0, policy_version 113591 (0.0011) [2023-12-26 16:11:35,877][105692] Updated weights for policy 0, policy_version 113601 (0.0011) [2023-12-26 16:11:36,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 58310656. Throughput: 0: 10020.3, 1: 9606.9. Samples: 58296152. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 16:11:36,062][104569] Avg episode reward: [(0, '7681.009'), (1, '7935.867')] [2023-12-26 16:11:36,425][105620] Updated weights for policy 1, policy_version 114127 (0.0008) [2023-12-26 16:11:36,487][105620] Updated weights for policy 1, policy_version 114137 (0.0008) [2023-12-26 16:11:36,555][105620] Updated weights for policy 1, policy_version 114147 (0.0008) [2023-12-26 16:11:36,646][105692] Updated weights for policy 0, policy_version 113611 (0.0011) [2023-12-26 16:11:36,706][105692] Updated weights for policy 0, policy_version 113621 (0.0011) [2023-12-26 16:11:36,775][105692] Updated weights for policy 0, policy_version 113631 (0.0011) [2023-12-26 16:11:37,119][105620] Updated weights for policy 1, policy_version 114157 (0.0008) [2023-12-26 16:11:37,177][105620] Updated weights for policy 1, policy_version 114167 (0.0007) [2023-12-26 16:11:37,246][105620] Updated weights for policy 1, policy_version 114177 (0.0005) [2023-12-26 16:11:37,517][105692] Updated weights for policy 0, policy_version 113641 (0.0011) [2023-12-26 16:11:37,581][105692] Updated weights for policy 0, policy_version 113652 (0.0011) [2023-12-26 16:11:37,631][105692] Updated weights for policy 0, policy_version 113662 (0.0008) [2023-12-26 16:11:37,694][105692] Updated weights for policy 0, policy_version 113672 (0.0006) [2023-12-26 16:11:37,772][105620] Updated weights for policy 1, policy_version 114187 (0.0005) [2023-12-26 16:11:37,825][105620] Updated weights for policy 1, policy_version 114197 (0.0006) [2023-12-26 16:11:37,880][105620] Updated weights for policy 1, policy_version 114207 (0.0009) [2023-12-26 16:11:38,346][105692] Updated weights for policy 0, policy_version 113682 (0.0007) [2023-12-26 16:11:38,405][105692] Updated weights for policy 0, policy_version 113692 (0.0008) [2023-12-26 16:11:38,462][105692] Updated weights for policy 0, policy_version 113702 (0.0008) [2023-12-26 16:11:38,554][105620] Updated weights for policy 1, policy_version 114217 (0.0009) [2023-12-26 16:11:38,618][105620] Updated weights for policy 1, policy_version 114227 (0.0007) [2023-12-26 16:11:38,675][105620] Updated weights for policy 1, policy_version 114237 (0.0009) [2023-12-26 16:11:38,733][105620] Updated weights for policy 1, policy_version 114247 (0.0011) [2023-12-26 16:11:39,242][105692] Updated weights for policy 0, policy_version 113712 (0.0008) [2023-12-26 16:11:39,307][105692] Updated weights for policy 0, policy_version 113722 (0.0008) [2023-12-26 16:11:39,367][105692] Updated weights for policy 0, policy_version 113732 (0.0008) [2023-12-26 16:11:39,482][105620] Updated weights for policy 1, policy_version 114257 (0.0008) [2023-12-26 16:11:39,542][105620] Updated weights for policy 1, policy_version 114267 (0.0006) [2023-12-26 16:11:39,609][105620] Updated weights for policy 1, policy_version 114277 (0.0006) [2023-12-26 16:11:40,129][105692] Updated weights for policy 0, policy_version 113742 (0.0007) [2023-12-26 16:11:40,184][105692] Updated weights for policy 0, policy_version 113752 (0.0006) [2023-12-26 16:11:40,242][105620] Updated weights for policy 1, policy_version 114287 (0.0009) [2023-12-26 16:11:40,246][105692] Updated weights for policy 0, policy_version 113762 (0.0006) [2023-12-26 16:11:40,309][105620] Updated weights for policy 1, policy_version 114297 (0.0011) [2023-12-26 16:11:40,367][105620] Updated weights for policy 1, policy_version 114307 (0.0011) [2023-12-26 16:11:40,824][105692] Updated weights for policy 0, policy_version 113772 (0.0005) [2023-12-26 16:11:40,872][105692] Updated weights for policy 0, policy_version 113782 (0.0005) [2023-12-26 16:11:40,923][105692] Updated weights for policy 0, policy_version 113792 (0.0009) [2023-12-26 16:11:41,057][105620] Updated weights for policy 1, policy_version 114317 (0.0011) [2023-12-26 16:11:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 58408960. Throughput: 0: 9950.2, 1: 9742.1. Samples: 58416880. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 16:11:41,063][104569] Avg episode reward: [(0, '7777.409'), (1, '8922.670')] [2023-12-26 16:11:41,121][105620] Updated weights for policy 1, policy_version 114327 (0.0011) [2023-12-26 16:11:41,186][105620] Updated weights for policy 1, policy_version 114337 (0.0009) [2023-12-26 16:11:41,715][105692] Updated weights for policy 0, policy_version 113802 (0.0008) [2023-12-26 16:11:41,779][105692] Updated weights for policy 0, policy_version 113812 (0.0008) [2023-12-26 16:11:41,836][105692] Updated weights for policy 0, policy_version 113822 (0.0009) [2023-12-26 16:11:41,889][105692] Updated weights for policy 0, policy_version 113832 (0.0009) [2023-12-26 16:11:41,907][105620] Updated weights for policy 1, policy_version 114347 (0.0007) [2023-12-26 16:11:41,968][105620] Updated weights for policy 1, policy_version 114357 (0.0009) [2023-12-26 16:11:42,031][105620] Updated weights for policy 1, policy_version 114367 (0.0009) [2023-12-26 16:11:42,646][105692] Updated weights for policy 0, policy_version 113842 (0.0009) [2023-12-26 16:11:42,680][105620] Updated weights for policy 1, policy_version 114377 (0.0007) [2023-12-26 16:11:42,702][105692] Updated weights for policy 0, policy_version 113852 (0.0009) [2023-12-26 16:11:42,735][105620] Updated weights for policy 1, policy_version 114387 (0.0006) [2023-12-26 16:11:42,758][105692] Updated weights for policy 0, policy_version 113862 (0.0009) [2023-12-26 16:11:42,796][105620] Updated weights for policy 1, policy_version 114397 (0.0007) [2023-12-26 16:11:42,855][105620] Updated weights for policy 1, policy_version 114407 (0.0009) [2023-12-26 16:11:43,386][105692] Updated weights for policy 0, policy_version 113872 (0.0006) [2023-12-26 16:11:43,442][105692] Updated weights for policy 0, policy_version 113882 (0.0006) [2023-12-26 16:11:43,499][105692] Updated weights for policy 0, policy_version 113892 (0.0010) [2023-12-26 16:11:43,542][105620] Updated weights for policy 1, policy_version 114417 (0.0006) [2023-12-26 16:11:43,589][105620] Updated weights for policy 1, policy_version 114427 (0.0005) [2023-12-26 16:11:43,638][105620] Updated weights for policy 1, policy_version 114437 (0.0005) [2023-12-26 16:11:44,048][105692] Updated weights for policy 0, policy_version 113902 (0.0007) [2023-12-26 16:11:44,114][105692] Updated weights for policy 0, policy_version 113912 (0.0006) [2023-12-26 16:11:44,178][105692] Updated weights for policy 0, policy_version 113922 (0.0007) [2023-12-26 16:11:44,268][105620] Updated weights for policy 1, policy_version 114447 (0.0008) [2023-12-26 16:11:44,325][105620] Updated weights for policy 1, policy_version 114457 (0.0008) [2023-12-26 16:11:44,383][105620] Updated weights for policy 1, policy_version 114467 (0.0008) [2023-12-26 16:11:44,773][105692] Updated weights for policy 0, policy_version 113932 (0.0010) [2023-12-26 16:11:44,844][105692] Updated weights for policy 0, policy_version 113942 (0.0011) [2023-12-26 16:11:44,910][105692] Updated weights for policy 0, policy_version 113952 (0.0010) [2023-12-26 16:11:45,065][105620] Updated weights for policy 1, policy_version 114477 (0.0008) [2023-12-26 16:11:45,127][105620] Updated weights for policy 1, policy_version 114487 (0.0008) [2023-12-26 16:11:45,187][105620] Updated weights for policy 1, policy_version 114497 (0.0008) [2023-12-26 16:11:45,544][105692] Updated weights for policy 0, policy_version 113962 (0.0010) [2023-12-26 16:11:45,603][105692] Updated weights for policy 0, policy_version 113972 (0.0005) [2023-12-26 16:11:45,657][105692] Updated weights for policy 0, policy_version 113982 (0.0006) [2023-12-26 16:11:45,716][105692] Updated weights for policy 0, policy_version 113992 (0.0006) [2023-12-26 16:11:45,880][105620] Updated weights for policy 1, policy_version 114507 (0.0005) [2023-12-26 16:11:45,944][105620] Updated weights for policy 1, policy_version 114517 (0.0005) [2023-12-26 16:11:45,996][105620] Updated weights for policy 1, policy_version 114527 (0.0005) [2023-12-26 16:11:46,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 58515456. Throughput: 0: 9916.2, 1: 9826.1. Samples: 58477652. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 16:11:46,063][104569] Avg episode reward: [(0, '8460.181'), (1, '9006.263')] [2023-12-26 16:11:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000114536_29327360.pth... [2023-12-26 16:11:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000113992_29188096.pth... [2023-12-26 16:11:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000112840_28893184.pth [2023-12-26 16:11:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000113352_29024256.pth [2023-12-26 16:11:46,308][105692] Updated weights for policy 0, policy_version 114002 (0.0010) [2023-12-26 16:11:46,381][105692] Updated weights for policy 0, policy_version 114012 (0.0011) [2023-12-26 16:11:46,447][105692] Updated weights for policy 0, policy_version 114022 (0.0010) [2023-12-26 16:11:46,506][105620] Updated weights for policy 1, policy_version 114537 (0.0005) [2023-12-26 16:11:46,571][105620] Updated weights for policy 1, policy_version 114547 (0.0005) [2023-12-26 16:11:46,620][105620] Updated weights for policy 1, policy_version 114557 (0.0005) [2023-12-26 16:11:46,669][105620] Updated weights for policy 1, policy_version 114567 (0.0005) [2023-12-26 16:11:47,172][105692] Updated weights for policy 0, policy_version 114032 (0.0010) [2023-12-26 16:11:47,195][105620] Updated weights for policy 1, policy_version 114577 (0.0010) [2023-12-26 16:11:47,231][105692] Updated weights for policy 0, policy_version 114042 (0.0010) [2023-12-26 16:11:47,250][105620] Updated weights for policy 1, policy_version 114587 (0.0010) [2023-12-26 16:11:47,290][105692] Updated weights for policy 0, policy_version 114052 (0.0010) [2023-12-26 16:11:47,301][105620] Updated weights for policy 1, policy_version 114597 (0.0010) [2023-12-26 16:11:47,937][105620] Updated weights for policy 1, policy_version 114607 (0.0010) [2023-12-26 16:11:47,990][105620] Updated weights for policy 1, policy_version 114617 (0.0008) [2023-12-26 16:11:48,035][105692] Updated weights for policy 0, policy_version 114062 (0.0010) [2023-12-26 16:11:48,041][105620] Updated weights for policy 1, policy_version 114627 (0.0006) [2023-12-26 16:11:48,087][105692] Updated weights for policy 0, policy_version 114072 (0.0010) [2023-12-26 16:11:48,147][105692] Updated weights for policy 0, policy_version 114082 (0.0011) [2023-12-26 16:11:48,778][105620] Updated weights for policy 1, policy_version 114637 (0.0008) [2023-12-26 16:11:48,837][105620] Updated weights for policy 1, policy_version 114647 (0.0008) [2023-12-26 16:11:48,888][105620] Updated weights for policy 1, policy_version 114657 (0.0008) [2023-12-26 16:11:48,911][105692] Updated weights for policy 0, policy_version 114092 (0.0011) [2023-12-26 16:11:48,973][105692] Updated weights for policy 0, policy_version 114102 (0.0011) [2023-12-26 16:11:49,034][105692] Updated weights for policy 0, policy_version 114112 (0.0010) [2023-12-26 16:11:49,655][105620] Updated weights for policy 1, policy_version 114667 (0.0006) [2023-12-26 16:11:49,714][105620] Updated weights for policy 1, policy_version 114677 (0.0008) [2023-12-26 16:11:49,777][105692] Updated weights for policy 0, policy_version 114122 (0.0010) [2023-12-26 16:11:49,778][105620] Updated weights for policy 1, policy_version 114687 (0.0008) [2023-12-26 16:11:49,830][105692] Updated weights for policy 0, policy_version 114132 (0.0011) [2023-12-26 16:11:49,894][105692] Updated weights for policy 0, policy_version 114142 (0.0008) [2023-12-26 16:11:49,961][105692] Updated weights for policy 0, policy_version 114152 (0.0009) [2023-12-26 16:11:50,571][105692] Updated weights for policy 0, policy_version 114162 (0.0008) [2023-12-26 16:11:50,612][105620] Updated weights for policy 1, policy_version 114697 (0.0007) [2023-12-26 16:11:50,630][105692] Updated weights for policy 0, policy_version 114172 (0.0007) [2023-12-26 16:11:50,675][105620] Updated weights for policy 1, policy_version 114707 (0.0007) [2023-12-26 16:11:50,689][105692] Updated weights for policy 0, policy_version 114182 (0.0009) [2023-12-26 16:11:50,739][105620] Updated weights for policy 1, policy_version 114717 (0.0008) [2023-12-26 16:11:50,809][105620] Updated weights for policy 1, policy_version 114727 (0.0009) [2023-12-26 16:11:51,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19797.4, 300 sec: 19522.0). Total num frames: 58613760. Throughput: 0: 9996.3, 1: 9931.9. Samples: 58601456. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 16:11:51,062][104569] Avg episode reward: [(0, '8547.262'), (1, '8388.289')] [2023-12-26 16:11:51,474][105620] Updated weights for policy 1, policy_version 114737 (0.0008) [2023-12-26 16:11:51,526][105692] Updated weights for policy 0, policy_version 114192 (0.0006) [2023-12-26 16:11:51,532][105620] Updated weights for policy 1, policy_version 114747 (0.0008) [2023-12-26 16:11:51,571][105692] Updated weights for policy 0, policy_version 114202 (0.0006) [2023-12-26 16:11:51,589][105620] Updated weights for policy 1, policy_version 114757 (0.0008) [2023-12-26 16:11:51,629][105692] Updated weights for policy 0, policy_version 114212 (0.0008) [2023-12-26 16:11:52,290][105620] Updated weights for policy 1, policy_version 114767 (0.0009) [2023-12-26 16:11:52,357][105620] Updated weights for policy 1, policy_version 114777 (0.0008) [2023-12-26 16:11:52,413][105692] Updated weights for policy 0, policy_version 114222 (0.0010) [2023-12-26 16:11:52,415][105620] Updated weights for policy 1, policy_version 114787 (0.0006) [2023-12-26 16:11:52,472][105692] Updated weights for policy 0, policy_version 114232 (0.0011) [2023-12-26 16:11:52,539][105692] Updated weights for policy 0, policy_version 114242 (0.0011) [2023-12-26 16:11:53,161][105620] Updated weights for policy 1, policy_version 114797 (0.0007) [2023-12-26 16:11:53,224][105620] Updated weights for policy 1, policy_version 114807 (0.0008) [2023-12-26 16:11:53,274][105692] Updated weights for policy 0, policy_version 114253 (0.0010) [2023-12-26 16:11:53,280][105620] Updated weights for policy 1, policy_version 114817 (0.0006) [2023-12-26 16:11:53,333][105692] Updated weights for policy 0, policy_version 114263 (0.0010) [2023-12-26 16:11:53,393][105692] Updated weights for policy 0, policy_version 114273 (0.0010) [2023-12-26 16:11:54,045][105620] Updated weights for policy 1, policy_version 114827 (0.0005) [2023-12-26 16:11:54,092][105620] Updated weights for policy 1, policy_version 114837 (0.0005) [2023-12-26 16:11:54,135][105692] Updated weights for policy 0, policy_version 114283 (0.0010) [2023-12-26 16:11:54,149][105620] Updated weights for policy 1, policy_version 114847 (0.0006) [2023-12-26 16:11:54,193][105692] Updated weights for policy 0, policy_version 114293 (0.0010) [2023-12-26 16:11:54,240][105692] Updated weights for policy 0, policy_version 114303 (0.0010) [2023-12-26 16:11:54,873][105620] Updated weights for policy 1, policy_version 114857 (0.0008) [2023-12-26 16:11:54,921][105620] Updated weights for policy 1, policy_version 114867 (0.0008) [2023-12-26 16:11:54,976][105692] Updated weights for policy 0, policy_version 114313 (0.0010) [2023-12-26 16:11:54,984][105620] Updated weights for policy 1, policy_version 114877 (0.0008) [2023-12-26 16:11:55,038][105692] Updated weights for policy 0, policy_version 114323 (0.0006) [2023-12-26 16:11:55,044][105620] Updated weights for policy 1, policy_version 114887 (0.0007) [2023-12-26 16:11:55,094][105692] Updated weights for policy 0, policy_version 114333 (0.0007) [2023-12-26 16:11:55,156][105692] Updated weights for policy 0, policy_version 114343 (0.0008) [2023-12-26 16:11:55,783][105692] Updated weights for policy 0, policy_version 114353 (0.0006) [2023-12-26 16:11:55,840][105692] Updated weights for policy 0, policy_version 114363 (0.0007) [2023-12-26 16:11:55,873][105620] Updated weights for policy 1, policy_version 114897 (0.0009) [2023-12-26 16:11:55,897][105692] Updated weights for policy 0, policy_version 114373 (0.0006) [2023-12-26 16:11:55,925][105620] Updated weights for policy 1, policy_version 114907 (0.0006) [2023-12-26 16:11:55,977][105620] Updated weights for policy 1, policy_version 114917 (0.0009) [2023-12-26 16:11:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 58712064. Throughput: 0: 9904.0, 1: 9874.1. Samples: 58714616. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 16:11:56,062][104569] Avg episode reward: [(0, '8280.877'), (1, '8117.806')] [2023-12-26 16:11:56,631][105692] Updated weights for policy 0, policy_version 114383 (0.0008) [2023-12-26 16:11:56,684][105692] Updated weights for policy 0, policy_version 114393 (0.0008) [2023-12-26 16:11:56,714][105620] Updated weights for policy 1, policy_version 114927 (0.0007) [2023-12-26 16:11:56,736][105692] Updated weights for policy 0, policy_version 114403 (0.0007) [2023-12-26 16:11:56,772][105620] Updated weights for policy 1, policy_version 114937 (0.0008) [2023-12-26 16:11:56,826][105620] Updated weights for policy 1, policy_version 114947 (0.0009) [2023-12-26 16:11:57,489][105692] Updated weights for policy 0, policy_version 114413 (0.0007) [2023-12-26 16:11:57,548][105692] Updated weights for policy 0, policy_version 114423 (0.0008) [2023-12-26 16:11:57,577][105620] Updated weights for policy 1, policy_version 114957 (0.0007) [2023-12-26 16:11:57,602][105692] Updated weights for policy 0, policy_version 114433 (0.0010) [2023-12-26 16:11:57,624][105620] Updated weights for policy 1, policy_version 114967 (0.0005) [2023-12-26 16:11:57,679][105620] Updated weights for policy 1, policy_version 114977 (0.0007) [2023-12-26 16:11:58,191][105692] Updated weights for policy 0, policy_version 114443 (0.0009) [2023-12-26 16:11:58,263][105692] Updated weights for policy 0, policy_version 114453 (0.0009) [2023-12-26 16:11:58,332][105692] Updated weights for policy 0, policy_version 114463 (0.0010) [2023-12-26 16:11:58,415][105620] Updated weights for policy 1, policy_version 114987 (0.0009) [2023-12-26 16:11:58,477][105620] Updated weights for policy 1, policy_version 114997 (0.0008) [2023-12-26 16:11:58,543][105620] Updated weights for policy 1, policy_version 115007 (0.0008) [2023-12-26 16:11:59,118][105692] Updated weights for policy 0, policy_version 114473 (0.0012) [2023-12-26 16:11:59,178][105692] Updated weights for policy 0, policy_version 114483 (0.0010) [2023-12-26 16:11:59,240][105692] Updated weights for policy 0, policy_version 114493 (0.0009) [2023-12-26 16:11:59,302][105692] Updated weights for policy 0, policy_version 114503 (0.0007) [2023-12-26 16:11:59,372][105620] Updated weights for policy 1, policy_version 115017 (0.0008) [2023-12-26 16:11:59,431][105620] Updated weights for policy 1, policy_version 115027 (0.0007) [2023-12-26 16:11:59,492][105620] Updated weights for policy 1, policy_version 115037 (0.0007) [2023-12-26 16:11:59,551][105620] Updated weights for policy 1, policy_version 115047 (0.0009) [2023-12-26 16:12:00,132][105692] Updated weights for policy 0, policy_version 114513 (0.0008) [2023-12-26 16:12:00,182][105620] Updated weights for policy 1, policy_version 115057 (0.0008) [2023-12-26 16:12:00,188][105692] Updated weights for policy 0, policy_version 114523 (0.0008) [2023-12-26 16:12:00,241][105692] Updated weights for policy 0, policy_version 114533 (0.0007) [2023-12-26 16:12:00,246][105620] Updated weights for policy 1, policy_version 115067 (0.0009) [2023-12-26 16:12:00,311][105620] Updated weights for policy 1, policy_version 115077 (0.0009) [2023-12-26 16:12:00,973][105620] Updated weights for policy 1, policy_version 115087 (0.0007) [2023-12-26 16:12:01,042][105620] Updated weights for policy 1, policy_version 115097 (0.0006) [2023-12-26 16:12:01,050][105692] Updated weights for policy 0, policy_version 114543 (0.0008) [2023-12-26 16:12:01,062][104569] Fps is (10 sec: 18022.1, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 58793984. Throughput: 0: 9907.1, 1: 9896.7. Samples: 58772324. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 16:12:01,062][104569] Avg episode reward: [(0, '8366.063'), (1, '8035.709')] [2023-12-26 16:12:01,105][105620] Updated weights for policy 1, policy_version 115107 (0.0007) [2023-12-26 16:12:01,107][105692] Updated weights for policy 0, policy_version 114553 (0.0008) [2023-12-26 16:12:01,140][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000115112_29474816.pth... [2023-12-26 16:12:01,145][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000113928_29171712.pth [2023-12-26 16:12:01,167][105692] Updated weights for policy 0, policy_version 114563 (0.0009) [2023-12-26 16:12:01,191][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000114568_29335552.pth... [2023-12-26 16:12:01,194][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000113416_29040640.pth [2023-12-26 16:12:01,834][105620] Updated weights for policy 1, policy_version 115117 (0.0009) [2023-12-26 16:12:01,893][105692] Updated weights for policy 0, policy_version 114573 (0.0008) [2023-12-26 16:12:01,896][105620] Updated weights for policy 1, policy_version 115127 (0.0007) [2023-12-26 16:12:01,954][105692] Updated weights for policy 0, policy_version 114583 (0.0007) [2023-12-26 16:12:01,958][105620] Updated weights for policy 1, policy_version 115137 (0.0006) [2023-12-26 16:12:02,013][105692] Updated weights for policy 0, policy_version 114593 (0.0007) [2023-12-26 16:12:02,665][105692] Updated weights for policy 0, policy_version 114603 (0.0009) [2023-12-26 16:12:02,720][105692] Updated weights for policy 0, policy_version 114613 (0.0009) [2023-12-26 16:12:02,731][105620] Updated weights for policy 1, policy_version 115147 (0.0008) [2023-12-26 16:12:02,774][105692] Updated weights for policy 0, policy_version 114623 (0.0008) [2023-12-26 16:12:02,788][105620] Updated weights for policy 1, policy_version 115157 (0.0008) [2023-12-26 16:12:02,838][105620] Updated weights for policy 1, policy_version 115167 (0.0007) [2023-12-26 16:12:03,513][105692] Updated weights for policy 0, policy_version 114633 (0.0007) [2023-12-26 16:12:03,564][105692] Updated weights for policy 0, policy_version 114643 (0.0009) [2023-12-26 16:12:03,588][105620] Updated weights for policy 1, policy_version 115177 (0.0009) [2023-12-26 16:12:03,611][105692] Updated weights for policy 0, policy_version 114653 (0.0008) [2023-12-26 16:12:03,637][105620] Updated weights for policy 1, policy_version 115187 (0.0008) [2023-12-26 16:12:03,663][105692] Updated weights for policy 0, policy_version 114663 (0.0006) [2023-12-26 16:12:03,693][105620] Updated weights for policy 1, policy_version 115197 (0.0007) [2023-12-26 16:12:03,746][105620] Updated weights for policy 1, policy_version 115207 (0.0008) [2023-12-26 16:12:04,407][105692] Updated weights for policy 0, policy_version 114673 (0.0009) [2023-12-26 16:12:04,470][105692] Updated weights for policy 0, policy_version 114683 (0.0009) [2023-12-26 16:12:04,527][105620] Updated weights for policy 1, policy_version 115217 (0.0007) [2023-12-26 16:12:04,529][105692] Updated weights for policy 0, policy_version 114693 (0.0009) [2023-12-26 16:12:04,584][105620] Updated weights for policy 1, policy_version 115227 (0.0008) [2023-12-26 16:12:04,638][105620] Updated weights for policy 1, policy_version 115237 (0.0008) [2023-12-26 16:12:05,279][105692] Updated weights for policy 0, policy_version 114703 (0.0009) [2023-12-26 16:12:05,326][105692] Updated weights for policy 0, policy_version 114713 (0.0008) [2023-12-26 16:12:05,376][105692] Updated weights for policy 0, policy_version 114723 (0.0007) [2023-12-26 16:12:05,377][105620] Updated weights for policy 1, policy_version 115247 (0.0009) [2023-12-26 16:12:05,424][105620] Updated weights for policy 1, policy_version 115257 (0.0006) [2023-12-26 16:12:05,471][105620] Updated weights for policy 1, policy_version 115267 (0.0009) [2023-12-26 16:12:06,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19660.9, 300 sec: 19438.6). Total num frames: 58892288. Throughput: 0: 9849.4, 1: 9860.5. Samples: 58885032. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 16:12:06,063][104569] Avg episode reward: [(0, '8106.725'), (1, '7866.125')] [2023-12-26 16:12:06,178][105692] Updated weights for policy 0, policy_version 114733 (0.0008) [2023-12-26 16:12:06,242][105692] Updated weights for policy 0, policy_version 114743 (0.0008) [2023-12-26 16:12:06,251][105620] Updated weights for policy 1, policy_version 115277 (0.0010) [2023-12-26 16:12:06,306][105692] Updated weights for policy 0, policy_version 114753 (0.0010) [2023-12-26 16:12:06,314][105620] Updated weights for policy 1, policy_version 115287 (0.0010) [2023-12-26 16:12:06,374][105620] Updated weights for policy 1, policy_version 115297 (0.0009) [2023-12-26 16:12:07,013][105692] Updated weights for policy 0, policy_version 114763 (0.0008) [2023-12-26 16:12:07,061][105692] Updated weights for policy 0, policy_version 114773 (0.0010) [2023-12-26 16:12:07,117][105692] Updated weights for policy 0, policy_version 114783 (0.0008) [2023-12-26 16:12:07,127][105620] Updated weights for policy 1, policy_version 115307 (0.0008) [2023-12-26 16:12:07,180][105620] Updated weights for policy 1, policy_version 115317 (0.0006) [2023-12-26 16:12:07,227][105620] Updated weights for policy 1, policy_version 115327 (0.0006) [2023-12-26 16:12:07,851][105692] Updated weights for policy 0, policy_version 114793 (0.0010) [2023-12-26 16:12:07,908][105692] Updated weights for policy 0, policy_version 114803 (0.0005) [2023-12-26 16:12:07,937][105620] Updated weights for policy 1, policy_version 115337 (0.0009) [2023-12-26 16:12:07,966][105692] Updated weights for policy 0, policy_version 114813 (0.0007) [2023-12-26 16:12:08,004][105620] Updated weights for policy 1, policy_version 115347 (0.0010) [2023-12-26 16:12:08,026][105692] Updated weights for policy 0, policy_version 114823 (0.0007) [2023-12-26 16:12:08,064][105620] Updated weights for policy 1, policy_version 115357 (0.0010) [2023-12-26 16:12:08,125][105620] Updated weights for policy 1, policy_version 115367 (0.0007) [2023-12-26 16:12:08,680][105620] Updated weights for policy 1, policy_version 115377 (0.0008) [2023-12-26 16:12:08,714][105692] Updated weights for policy 0, policy_version 114833 (0.0008) [2023-12-26 16:12:08,735][105620] Updated weights for policy 1, policy_version 115387 (0.0005) [2023-12-26 16:12:08,771][105692] Updated weights for policy 0, policy_version 114843 (0.0009) [2023-12-26 16:12:08,786][105620] Updated weights for policy 1, policy_version 115397 (0.0007) [2023-12-26 16:12:08,829][105692] Updated weights for policy 0, policy_version 114853 (0.0007) [2023-12-26 16:12:09,535][105620] Updated weights for policy 1, policy_version 115407 (0.0011) [2023-12-26 16:12:09,598][105620] Updated weights for policy 1, policy_version 115417 (0.0011) [2023-12-26 16:12:09,607][105692] Updated weights for policy 0, policy_version 114863 (0.0008) [2023-12-26 16:12:09,656][105620] Updated weights for policy 1, policy_version 115427 (0.0009) [2023-12-26 16:12:09,668][105692] Updated weights for policy 0, policy_version 114873 (0.0007) [2023-12-26 16:12:09,723][105692] Updated weights for policy 0, policy_version 114883 (0.0009) [2023-12-26 16:12:10,381][105620] Updated weights for policy 1, policy_version 115437 (0.0008) [2023-12-26 16:12:10,445][105620] Updated weights for policy 1, policy_version 115447 (0.0011) [2023-12-26 16:12:10,505][105620] Updated weights for policy 1, policy_version 115457 (0.0010) [2023-12-26 16:12:10,539][105692] Updated weights for policy 0, policy_version 114893 (0.0008) [2023-12-26 16:12:10,596][105692] Updated weights for policy 0, policy_version 114903 (0.0008) [2023-12-26 16:12:10,651][105692] Updated weights for policy 0, policy_version 114913 (0.0007) [2023-12-26 16:12:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 58990592. Throughput: 0: 9773.0, 1: 9889.3. Samples: 59000660. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 16:12:11,063][104569] Avg episode reward: [(0, '8146.797'), (1, '8136.314')] [2023-12-26 16:12:11,185][105620] Updated weights for policy 1, policy_version 115467 (0.0010) [2023-12-26 16:12:11,258][105620] Updated weights for policy 1, policy_version 115477 (0.0010) [2023-12-26 16:12:11,317][105620] Updated weights for policy 1, policy_version 115487 (0.0010) [2023-12-26 16:12:11,352][105692] Updated weights for policy 0, policy_version 114923 (0.0006) [2023-12-26 16:12:11,423][105692] Updated weights for policy 0, policy_version 114933 (0.0009) [2023-12-26 16:12:11,483][105692] Updated weights for policy 0, policy_version 114943 (0.0008) [2023-12-26 16:12:12,043][105620] Updated weights for policy 1, policy_version 115497 (0.0012) [2023-12-26 16:12:12,106][105620] Updated weights for policy 1, policy_version 115507 (0.0009) [2023-12-26 16:12:12,160][105620] Updated weights for policy 1, policy_version 115517 (0.0009) [2023-12-26 16:12:12,210][105692] Updated weights for policy 0, policy_version 114953 (0.0005) [2023-12-26 16:12:12,219][105620] Updated weights for policy 1, policy_version 115527 (0.0009) [2023-12-26 16:12:12,278][105692] Updated weights for policy 0, policy_version 114963 (0.0010) [2023-12-26 16:12:12,333][105692] Updated weights for policy 0, policy_version 114973 (0.0009) [2023-12-26 16:12:12,393][105692] Updated weights for policy 0, policy_version 114983 (0.0009) [2023-12-26 16:12:12,902][105620] Updated weights for policy 1, policy_version 115537 (0.0008) [2023-12-26 16:12:12,965][105620] Updated weights for policy 1, policy_version 115547 (0.0008) [2023-12-26 16:12:13,034][105620] Updated weights for policy 1, policy_version 115557 (0.0008) [2023-12-26 16:12:13,190][105692] Updated weights for policy 0, policy_version 114993 (0.0011) [2023-12-26 16:12:13,251][105692] Updated weights for policy 0, policy_version 115003 (0.0010) [2023-12-26 16:12:13,313][105692] Updated weights for policy 0, policy_version 115013 (0.0010) [2023-12-26 16:12:13,600][105620] Updated weights for policy 1, policy_version 115567 (0.0006) [2023-12-26 16:12:13,653][105620] Updated weights for policy 1, policy_version 115577 (0.0005) [2023-12-26 16:12:13,708][105620] Updated weights for policy 1, policy_version 115587 (0.0005) [2023-12-26 16:12:13,949][105692] Updated weights for policy 0, policy_version 115023 (0.0011) [2023-12-26 16:12:14,008][105692] Updated weights for policy 0, policy_version 115033 (0.0010) [2023-12-26 16:12:14,066][105692] Updated weights for policy 0, policy_version 115043 (0.0010) [2023-12-26 16:12:14,360][105620] Updated weights for policy 1, policy_version 115597 (0.0007) [2023-12-26 16:12:14,421][105620] Updated weights for policy 1, policy_version 115607 (0.0008) [2023-12-26 16:12:14,482][105620] Updated weights for policy 1, policy_version 115617 (0.0008) [2023-12-26 16:12:14,729][105692] Updated weights for policy 0, policy_version 115053 (0.0008) [2023-12-26 16:12:14,789][105692] Updated weights for policy 0, policy_version 115063 (0.0007) [2023-12-26 16:12:14,809][105585] KL-divergence is very high: 118.6974 [2023-12-26 16:12:14,852][105692] Updated weights for policy 0, policy_version 115073 (0.0008) [2023-12-26 16:12:15,270][105620] Updated weights for policy 1, policy_version 115627 (0.0009) [2023-12-26 16:12:15,330][105620] Updated weights for policy 1, policy_version 115637 (0.0008) [2023-12-26 16:12:15,386][105620] Updated weights for policy 1, policy_version 115647 (0.0008) [2023-12-26 16:12:15,548][105692] Updated weights for policy 0, policy_version 115083 (0.0009) [2023-12-26 16:12:15,613][105692] Updated weights for policy 0, policy_version 115093 (0.0011) [2023-12-26 16:12:15,662][105692] Updated weights for policy 0, policy_version 115103 (0.0010) [2023-12-26 16:12:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 59088896. Throughput: 0: 9726.7, 1: 9881.1. Samples: 59060620. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 16:12:16,063][104569] Avg episode reward: [(0, '2067.475'), (1, '8658.369')] [2023-12-26 16:12:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000115112_29474816.pth... [2023-12-26 16:12:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000115656_29614080.pth... [2023-12-26 16:12:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000113992_29188096.pth [2023-12-26 16:12:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000114536_29327360.pth [2023-12-26 16:12:16,153][105620] Updated weights for policy 1, policy_version 115657 (0.0008) [2023-12-26 16:12:16,219][105620] Updated weights for policy 1, policy_version 115667 (0.0008) [2023-12-26 16:12:16,276][105620] Updated weights for policy 1, policy_version 115677 (0.0010) [2023-12-26 16:12:16,334][105620] Updated weights for policy 1, policy_version 115688 (0.0010) [2023-12-26 16:12:16,368][105692] Updated weights for policy 0, policy_version 115113 (0.0010) [2023-12-26 16:12:16,437][105692] Updated weights for policy 0, policy_version 115123 (0.0007) [2023-12-26 16:12:16,489][105692] Updated weights for policy 0, policy_version 115133 (0.0006) [2023-12-26 16:12:16,543][105692] Updated weights for policy 0, policy_version 115143 (0.0011) [2023-12-26 16:12:16,943][105620] Updated weights for policy 1, policy_version 115699 (0.0010) [2023-12-26 16:12:16,995][105620] Updated weights for policy 1, policy_version 115709 (0.0009) [2023-12-26 16:12:17,051][105620] Updated weights for policy 1, policy_version 115719 (0.0009) [2023-12-26 16:12:17,192][105692] Updated weights for policy 0, policy_version 115153 (0.0008) [2023-12-26 16:12:17,244][105692] Updated weights for policy 0, policy_version 115163 (0.0009) [2023-12-26 16:12:17,292][105692] Updated weights for policy 0, policy_version 115173 (0.0009) [2023-12-26 16:12:17,736][105620] Updated weights for policy 1, policy_version 115729 (0.0006) [2023-12-26 16:12:17,786][105620] Updated weights for policy 1, policy_version 115739 (0.0005) [2023-12-26 16:12:17,842][105620] Updated weights for policy 1, policy_version 115749 (0.0005) [2023-12-26 16:12:18,033][105692] Updated weights for policy 0, policy_version 115183 (0.0007) [2023-12-26 16:12:18,084][105692] Updated weights for policy 0, policy_version 115193 (0.0006) [2023-12-26 16:12:18,146][105692] Updated weights for policy 0, policy_version 115203 (0.0006) [2023-12-26 16:12:18,580][105620] Updated weights for policy 1, policy_version 115759 (0.0008) [2023-12-26 16:12:18,628][105620] Updated weights for policy 1, policy_version 115769 (0.0007) [2023-12-26 16:12:18,673][105586] KL-divergence is very high: 173.4092 [2023-12-26 16:12:18,678][105586] KL-divergence is very high: 173.5183 [2023-12-26 16:12:18,682][105620] Updated weights for policy 1, policy_version 115779 (0.0005) [2023-12-26 16:12:18,811][105692] Updated weights for policy 0, policy_version 115213 (0.0008) [2023-12-26 16:12:18,868][105692] Updated weights for policy 0, policy_version 115223 (0.0008) [2023-12-26 16:12:18,913][105692] Updated weights for policy 0, policy_version 115233 (0.0010) [2023-12-26 16:12:19,365][105620] Updated weights for policy 1, policy_version 115789 (0.0007) [2023-12-26 16:12:19,374][105586] KL-divergence is very high: 364.6818 [2023-12-26 16:12:19,421][105620] Updated weights for policy 1, policy_version 115799 (0.0008) [2023-12-26 16:12:19,423][105586] KL-divergence is very high: 379.0970 [2023-12-26 16:12:19,469][105586] KL-divergence is very high: 375.4354 [2023-12-26 16:12:19,484][105620] Updated weights for policy 1, policy_version 115809 (0.0008) [2023-12-26 16:12:19,520][105586] KL-divergence is very high: 362.1310 [2023-12-26 16:12:19,701][105692] Updated weights for policy 0, policy_version 115243 (0.0010) [2023-12-26 16:12:19,773][105692] Updated weights for policy 0, policy_version 115253 (0.0011) [2023-12-26 16:12:19,836][105692] Updated weights for policy 0, policy_version 115263 (0.0011) [2023-12-26 16:12:20,244][105620] Updated weights for policy 1, policy_version 115819 (0.0008) [2023-12-26 16:12:20,294][105620] Updated weights for policy 1, policy_version 115829 (0.0006) [2023-12-26 16:12:20,355][105620] Updated weights for policy 1, policy_version 115839 (0.0008) [2023-12-26 16:12:20,618][105692] Updated weights for policy 0, policy_version 115273 (0.0009) [2023-12-26 16:12:20,681][105692] Updated weights for policy 0, policy_version 115283 (0.0009) [2023-12-26 16:12:20,733][105692] Updated weights for policy 0, policy_version 115293 (0.0009) [2023-12-26 16:12:20,795][105692] Updated weights for policy 0, policy_version 115303 (0.0009) [2023-12-26 16:12:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 59187200. Throughput: 0: 9714.9, 1: 9907.5. Samples: 59179160. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 16:12:21,062][104569] Avg episode reward: [(0, '4295.389'), (1, '8566.568')] [2023-12-26 16:12:21,122][105620] Updated weights for policy 1, policy_version 115849 (0.0009) [2023-12-26 16:12:21,189][105620] Updated weights for policy 1, policy_version 115859 (0.0008) [2023-12-26 16:12:21,253][105620] Updated weights for policy 1, policy_version 115869 (0.0008) [2023-12-26 16:12:21,313][105620] Updated weights for policy 1, policy_version 115879 (0.0008) [2023-12-26 16:12:21,613][105692] Updated weights for policy 0, policy_version 115313 (0.0008) [2023-12-26 16:12:21,678][105692] Updated weights for policy 0, policy_version 115323 (0.0008) [2023-12-26 16:12:21,743][105692] Updated weights for policy 0, policy_version 115333 (0.0008) [2023-12-26 16:12:22,084][105620] Updated weights for policy 1, policy_version 115889 (0.0008) [2023-12-26 16:12:22,142][105620] Updated weights for policy 1, policy_version 115899 (0.0008) [2023-12-26 16:12:22,202][105620] Updated weights for policy 1, policy_version 115909 (0.0008) [2023-12-26 16:12:22,504][105692] Updated weights for policy 0, policy_version 115343 (0.0006) [2023-12-26 16:12:22,573][105692] Updated weights for policy 0, policy_version 115353 (0.0006) [2023-12-26 16:12:22,644][105692] Updated weights for policy 0, policy_version 115363 (0.0006) [2023-12-26 16:12:22,934][105620] Updated weights for policy 1, policy_version 115919 (0.0008) [2023-12-26 16:12:22,993][105620] Updated weights for policy 1, policy_version 115929 (0.0010) [2023-12-26 16:12:23,052][105620] Updated weights for policy 1, policy_version 115939 (0.0010) [2023-12-26 16:12:23,204][105692] Updated weights for policy 0, policy_version 115373 (0.0006) [2023-12-26 16:12:23,260][105692] Updated weights for policy 0, policy_version 115383 (0.0006) [2023-12-26 16:12:23,320][105692] Updated weights for policy 0, policy_version 115393 (0.0005) [2023-12-26 16:12:23,836][105620] Updated weights for policy 1, policy_version 115949 (0.0008) [2023-12-26 16:12:23,839][105692] Updated weights for policy 0, policy_version 115403 (0.0005) [2023-12-26 16:12:23,886][105692] Updated weights for policy 0, policy_version 115413 (0.0005) [2023-12-26 16:12:23,888][105620] Updated weights for policy 1, policy_version 115959 (0.0006) [2023-12-26 16:12:23,939][105692] Updated weights for policy 0, policy_version 115423 (0.0005) [2023-12-26 16:12:23,948][105620] Updated weights for policy 1, policy_version 115969 (0.0006) [2023-12-26 16:12:24,541][105620] Updated weights for policy 1, policy_version 115979 (0.0005) [2023-12-26 16:12:24,544][105692] Updated weights for policy 0, policy_version 115433 (0.0007) [2023-12-26 16:12:24,596][105692] Updated weights for policy 0, policy_version 115443 (0.0005) [2023-12-26 16:12:24,611][105620] Updated weights for policy 1, policy_version 115989 (0.0006) [2023-12-26 16:12:24,649][105692] Updated weights for policy 0, policy_version 115453 (0.0005) [2023-12-26 16:12:24,673][105620] Updated weights for policy 1, policy_version 115999 (0.0006) [2023-12-26 16:12:24,709][105692] Updated weights for policy 0, policy_version 115463 (0.0010) [2023-12-26 16:12:25,257][105620] Updated weights for policy 1, policy_version 116009 (0.0006) [2023-12-26 16:12:25,327][105620] Updated weights for policy 1, policy_version 116019 (0.0006) [2023-12-26 16:12:25,386][105620] Updated weights for policy 1, policy_version 116029 (0.0007) [2023-12-26 16:12:25,427][105692] Updated weights for policy 0, policy_version 115473 (0.0010) [2023-12-26 16:12:25,441][105620] Updated weights for policy 1, policy_version 116039 (0.0007) [2023-12-26 16:12:25,476][105692] Updated weights for policy 0, policy_version 115483 (0.0008) [2023-12-26 16:12:25,530][105692] Updated weights for policy 0, policy_version 115494 (0.0010) [2023-12-26 16:12:26,002][105620] Updated weights for policy 1, policy_version 116049 (0.0006) [2023-12-26 16:12:26,060][105620] Updated weights for policy 1, policy_version 116059 (0.0008) [2023-12-26 16:12:26,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.9, 300 sec: 19494.2). Total num frames: 59285504. Throughput: 0: 9757.5, 1: 9840.8. Samples: 59298804. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 16:12:26,063][104569] Avg episode reward: [(0, '7018.407'), (1, '9000.099')] [2023-12-26 16:12:26,117][105620] Updated weights for policy 1, policy_version 116069 (0.0008) [2023-12-26 16:12:26,272][105692] Updated weights for policy 0, policy_version 115504 (0.0008) [2023-12-26 16:12:26,332][105692] Updated weights for policy 0, policy_version 115514 (0.0005) [2023-12-26 16:12:26,381][105692] Updated weights for policy 0, policy_version 115524 (0.0010) [2023-12-26 16:12:26,798][105620] Updated weights for policy 1, policy_version 116079 (0.0008) [2023-12-26 16:12:26,846][105620] Updated weights for policy 1, policy_version 116089 (0.0008) [2023-12-26 16:12:26,902][105620] Updated weights for policy 1, policy_version 116099 (0.0009) [2023-12-26 16:12:26,919][105692] Updated weights for policy 0, policy_version 115534 (0.0009) [2023-12-26 16:12:26,966][105692] Updated weights for policy 0, policy_version 115544 (0.0010) [2023-12-26 16:12:27,017][105692] Updated weights for policy 0, policy_version 115554 (0.0010) [2023-12-26 16:12:27,680][105692] Updated weights for policy 0, policy_version 115564 (0.0008) [2023-12-26 16:12:27,717][105620] Updated weights for policy 1, policy_version 116109 (0.0008) [2023-12-26 16:12:27,731][105692] Updated weights for policy 0, policy_version 115574 (0.0005) [2023-12-26 16:12:27,772][105620] Updated weights for policy 1, policy_version 116119 (0.0006) [2023-12-26 16:12:27,779][105692] Updated weights for policy 0, policy_version 115584 (0.0010) [2023-12-26 16:12:27,816][105620] Updated weights for policy 1, policy_version 116129 (0.0007) [2023-12-26 16:12:28,381][105692] Updated weights for policy 0, policy_version 115594 (0.0007) [2023-12-26 16:12:28,432][105692] Updated weights for policy 0, policy_version 115604 (0.0005) [2023-12-26 16:12:28,481][105692] Updated weights for policy 0, policy_version 115614 (0.0005) [2023-12-26 16:12:28,529][105692] Updated weights for policy 0, policy_version 115624 (0.0005) [2023-12-26 16:12:28,644][105620] Updated weights for policy 1, policy_version 116139 (0.0008) [2023-12-26 16:12:28,697][105620] Updated weights for policy 1, policy_version 116149 (0.0010) [2023-12-26 16:12:28,750][105620] Updated weights for policy 1, policy_version 116159 (0.0009) [2023-12-26 16:12:29,155][105692] Updated weights for policy 0, policy_version 115634 (0.0009) [2023-12-26 16:12:29,213][105692] Updated weights for policy 0, policy_version 115644 (0.0009) [2023-12-26 16:12:29,275][105692] Updated weights for policy 0, policy_version 115654 (0.0009) [2023-12-26 16:12:29,519][105620] Updated weights for policy 1, policy_version 116169 (0.0009) [2023-12-26 16:12:29,581][105620] Updated weights for policy 1, policy_version 116179 (0.0005) [2023-12-26 16:12:29,638][105620] Updated weights for policy 1, policy_version 116189 (0.0005) [2023-12-26 16:12:29,693][105620] Updated weights for policy 1, policy_version 116199 (0.0005) [2023-12-26 16:12:30,056][105692] Updated weights for policy 0, policy_version 115664 (0.0006) [2023-12-26 16:12:30,112][105692] Updated weights for policy 0, policy_version 115674 (0.0006) [2023-12-26 16:12:30,169][105692] Updated weights for policy 0, policy_version 115684 (0.0006) [2023-12-26 16:12:30,291][105620] Updated weights for policy 1, policy_version 116209 (0.0006) [2023-12-26 16:12:30,347][105620] Updated weights for policy 1, policy_version 116219 (0.0010) [2023-12-26 16:12:30,397][105620] Updated weights for policy 1, policy_version 116229 (0.0009) [2023-12-26 16:12:30,759][105692] Updated weights for policy 0, policy_version 115694 (0.0007) [2023-12-26 16:12:30,806][105692] Updated weights for policy 0, policy_version 115704 (0.0007) [2023-12-26 16:12:30,857][105692] Updated weights for policy 0, policy_version 115714 (0.0008) [2023-12-26 16:12:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 59392000. Throughput: 0: 9841.5, 1: 9759.5. Samples: 59359696. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 16:12:31,062][104569] Avg episode reward: [(0, '8456.983'), (1, '9176.559')] [2023-12-26 16:12:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000115720_29630464.pth... [2023-12-26 16:12:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000116232_29761536.pth... [2023-12-26 16:12:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000114568_29335552.pth [2023-12-26 16:12:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000115112_29474816.pth [2023-12-26 16:12:31,143][105620] Updated weights for policy 1, policy_version 116239 (0.0009) [2023-12-26 16:12:31,208][105620] Updated weights for policy 1, policy_version 116249 (0.0010) [2023-12-26 16:12:31,273][105620] Updated weights for policy 1, policy_version 116259 (0.0010) [2023-12-26 16:12:31,604][105692] Updated weights for policy 0, policy_version 115724 (0.0008) [2023-12-26 16:12:31,670][105692] Updated weights for policy 0, policy_version 115734 (0.0008) [2023-12-26 16:12:31,734][105692] Updated weights for policy 0, policy_version 115744 (0.0008) [2023-12-26 16:12:31,943][105620] Updated weights for policy 1, policy_version 116269 (0.0008) [2023-12-26 16:12:32,000][105620] Updated weights for policy 1, policy_version 116279 (0.0006) [2023-12-26 16:12:32,060][105620] Updated weights for policy 1, policy_version 116289 (0.0005) [2023-12-26 16:12:32,501][105692] Updated weights for policy 0, policy_version 115754 (0.0010) [2023-12-26 16:12:32,554][105692] Updated weights for policy 0, policy_version 115764 (0.0009) [2023-12-26 16:12:32,599][105620] Updated weights for policy 1, policy_version 116299 (0.0006) [2023-12-26 16:12:32,616][105692] Updated weights for policy 0, policy_version 115774 (0.0010) [2023-12-26 16:12:32,655][105620] Updated weights for policy 1, policy_version 116309 (0.0006) [2023-12-26 16:12:32,673][105692] Updated weights for policy 0, policy_version 115784 (0.0009) [2023-12-26 16:12:32,713][105620] Updated weights for policy 1, policy_version 116319 (0.0008) [2023-12-26 16:12:33,368][105620] Updated weights for policy 1, policy_version 116329 (0.0008) [2023-12-26 16:12:33,430][105620] Updated weights for policy 1, policy_version 116339 (0.0006) [2023-12-26 16:12:33,486][105620] Updated weights for policy 1, policy_version 116349 (0.0007) [2023-12-26 16:12:33,489][105692] Updated weights for policy 0, policy_version 115794 (0.0006) [2023-12-26 16:12:33,538][105620] Updated weights for policy 1, policy_version 116359 (0.0010) [2023-12-26 16:12:33,540][105692] Updated weights for policy 0, policy_version 115804 (0.0005) [2023-12-26 16:12:33,594][105692] Updated weights for policy 0, policy_version 115814 (0.0008) [2023-12-26 16:12:34,245][105620] Updated weights for policy 1, policy_version 116369 (0.0006) [2023-12-26 16:12:34,310][105620] Updated weights for policy 1, policy_version 116379 (0.0010) [2023-12-26 16:12:34,372][105620] Updated weights for policy 1, policy_version 116389 (0.0010) [2023-12-26 16:12:34,391][105692] Updated weights for policy 0, policy_version 115824 (0.0006) [2023-12-26 16:12:34,440][105692] Updated weights for policy 0, policy_version 115834 (0.0008) [2023-12-26 16:12:34,489][105692] Updated weights for policy 0, policy_version 115844 (0.0008) [2023-12-26 16:12:35,069][105620] Updated weights for policy 1, policy_version 116399 (0.0008) [2023-12-26 16:12:35,125][105620] Updated weights for policy 1, policy_version 116409 (0.0010) [2023-12-26 16:12:35,176][105620] Updated weights for policy 1, policy_version 116419 (0.0010) [2023-12-26 16:12:35,247][105692] Updated weights for policy 0, policy_version 115854 (0.0008) [2023-12-26 16:12:35,304][105692] Updated weights for policy 0, policy_version 115864 (0.0006) [2023-12-26 16:12:35,351][105692] Updated weights for policy 0, policy_version 115874 (0.0008) [2023-12-26 16:12:35,910][105620] Updated weights for policy 1, policy_version 116429 (0.0008) [2023-12-26 16:12:35,967][105620] Updated weights for policy 1, policy_version 116439 (0.0010) [2023-12-26 16:12:36,022][105620] Updated weights for policy 1, policy_version 116449 (0.0010) [2023-12-26 16:12:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 59490304. Throughput: 0: 9748.2, 1: 9751.2. Samples: 59478936. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 16:12:36,063][104569] Avg episode reward: [(0, '8285.925'), (1, '9177.703')] [2023-12-26 16:12:36,101][105692] Updated weights for policy 0, policy_version 115884 (0.0008) [2023-12-26 16:12:36,164][105692] Updated weights for policy 0, policy_version 115894 (0.0009) [2023-12-26 16:12:36,218][105692] Updated weights for policy 0, policy_version 115904 (0.0006) [2023-12-26 16:12:36,778][105620] Updated weights for policy 1, policy_version 116459 (0.0010) [2023-12-26 16:12:36,829][105620] Updated weights for policy 1, policy_version 116469 (0.0010) [2023-12-26 16:12:36,881][105620] Updated weights for policy 1, policy_version 116479 (0.0010) [2023-12-26 16:12:36,888][105692] Updated weights for policy 0, policy_version 115914 (0.0007) [2023-12-26 16:12:36,941][105692] Updated weights for policy 0, policy_version 115924 (0.0007) [2023-12-26 16:12:37,010][105692] Updated weights for policy 0, policy_version 115934 (0.0008) [2023-12-26 16:12:37,059][105692] Updated weights for policy 0, policy_version 115944 (0.0008) [2023-12-26 16:12:37,640][105620] Updated weights for policy 1, policy_version 116489 (0.0010) [2023-12-26 16:12:37,685][105620] Updated weights for policy 1, policy_version 116499 (0.0010) [2023-12-26 16:12:37,738][105620] Updated weights for policy 1, policy_version 116509 (0.0010) [2023-12-26 16:12:37,787][105620] Updated weights for policy 1, policy_version 116519 (0.0010) [2023-12-26 16:12:37,813][105692] Updated weights for policy 0, policy_version 115954 (0.0006) [2023-12-26 16:12:37,874][105692] Updated weights for policy 0, policy_version 115964 (0.0009) [2023-12-26 16:12:37,927][105692] Updated weights for policy 0, policy_version 115974 (0.0008) [2023-12-26 16:12:38,571][105620] Updated weights for policy 1, policy_version 116529 (0.0010) [2023-12-26 16:12:38,625][105692] Updated weights for policy 0, policy_version 115984 (0.0008) [2023-12-26 16:12:38,631][105620] Updated weights for policy 1, policy_version 116539 (0.0011) [2023-12-26 16:12:38,685][105692] Updated weights for policy 0, policy_version 115994 (0.0010) [2023-12-26 16:12:38,689][105620] Updated weights for policy 1, policy_version 116549 (0.0010) [2023-12-26 16:12:38,739][105692] Updated weights for policy 0, policy_version 116004 (0.0009) [2023-12-26 16:12:39,297][105692] Updated weights for policy 0, policy_version 116014 (0.0007) [2023-12-26 16:12:39,349][105692] Updated weights for policy 0, policy_version 116024 (0.0008) [2023-12-26 16:12:39,419][105692] Updated weights for policy 0, policy_version 116034 (0.0009) [2023-12-26 16:12:39,467][105620] Updated weights for policy 1, policy_version 116559 (0.0009) [2023-12-26 16:12:39,527][105620] Updated weights for policy 1, policy_version 116569 (0.0009) [2023-12-26 16:12:39,581][105620] Updated weights for policy 1, policy_version 116579 (0.0010) [2023-12-26 16:12:40,085][105692] Updated weights for policy 0, policy_version 116044 (0.0007) [2023-12-26 16:12:40,147][105692] Updated weights for policy 0, policy_version 116054 (0.0008) [2023-12-26 16:12:40,206][105692] Updated weights for policy 0, policy_version 116064 (0.0008) [2023-12-26 16:12:40,423][105620] Updated weights for policy 1, policy_version 116589 (0.0010) [2023-12-26 16:12:40,478][105620] Updated weights for policy 1, policy_version 116599 (0.0009) [2023-12-26 16:12:40,538][105620] Updated weights for policy 1, policy_version 116609 (0.0009) [2023-12-26 16:12:40,886][105692] Updated weights for policy 0, policy_version 116074 (0.0009) [2023-12-26 16:12:40,943][105692] Updated weights for policy 0, policy_version 116084 (0.0010) [2023-12-26 16:12:41,001][105692] Updated weights for policy 0, policy_version 116094 (0.0008) [2023-12-26 16:12:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 59580416. Throughput: 0: 9822.5, 1: 9721.2. Samples: 59594084. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-26 16:12:41,062][104569] Avg episode reward: [(0, '7921.778'), (1, '9355.582')] [2023-12-26 16:12:41,064][105692] Updated weights for policy 0, policy_version 116104 (0.0009) [2023-12-26 16:12:41,374][105620] Updated weights for policy 1, policy_version 116619 (0.0009) [2023-12-26 16:12:41,444][105620] Updated weights for policy 1, policy_version 116629 (0.0007) [2023-12-26 16:12:41,505][105620] Updated weights for policy 1, policy_version 116639 (0.0009) [2023-12-26 16:12:41,847][105692] Updated weights for policy 0, policy_version 116114 (0.0009) [2023-12-26 16:12:41,905][105692] Updated weights for policy 0, policy_version 116124 (0.0009) [2023-12-26 16:12:41,964][105692] Updated weights for policy 0, policy_version 116134 (0.0009) [2023-12-26 16:12:42,253][105620] Updated weights for policy 1, policy_version 116649 (0.0008) [2023-12-26 16:12:42,317][105620] Updated weights for policy 1, policy_version 116659 (0.0009) [2023-12-26 16:12:42,382][105620] Updated weights for policy 1, policy_version 116669 (0.0010) [2023-12-26 16:12:42,442][105620] Updated weights for policy 1, policy_version 116679 (0.0009) [2023-12-26 16:12:42,662][105692] Updated weights for policy 0, policy_version 116144 (0.0006) [2023-12-26 16:12:42,713][105692] Updated weights for policy 0, policy_version 116154 (0.0005) [2023-12-26 16:12:42,775][105692] Updated weights for policy 0, policy_version 116164 (0.0009) [2023-12-26 16:12:43,215][105620] Updated weights for policy 1, policy_version 116689 (0.0009) [2023-12-26 16:12:43,276][105620] Updated weights for policy 1, policy_version 116699 (0.0008) [2023-12-26 16:12:43,337][105620] Updated weights for policy 1, policy_version 116709 (0.0009) [2023-12-26 16:12:43,443][105692] Updated weights for policy 0, policy_version 116174 (0.0007) [2023-12-26 16:12:43,497][105692] Updated weights for policy 0, policy_version 116184 (0.0008) [2023-12-26 16:12:43,548][105692] Updated weights for policy 0, policy_version 116194 (0.0009) [2023-12-26 16:12:44,083][105620] Updated weights for policy 1, policy_version 116719 (0.0008) [2023-12-26 16:12:44,140][105620] Updated weights for policy 1, policy_version 116729 (0.0008) [2023-12-26 16:12:44,194][105620] Updated weights for policy 1, policy_version 116739 (0.0009) [2023-12-26 16:12:44,252][105692] Updated weights for policy 0, policy_version 116204 (0.0009) [2023-12-26 16:12:44,314][105692] Updated weights for policy 0, policy_version 116214 (0.0006) [2023-12-26 16:12:44,364][105692] Updated weights for policy 0, policy_version 116224 (0.0008) [2023-12-26 16:12:44,929][105620] Updated weights for policy 1, policy_version 116749 (0.0007) [2023-12-26 16:12:44,998][105620] Updated weights for policy 1, policy_version 116759 (0.0007) [2023-12-26 16:12:45,041][105692] Updated weights for policy 0, policy_version 116234 (0.0009) [2023-12-26 16:12:45,063][105620] Updated weights for policy 1, policy_version 116769 (0.0006) [2023-12-26 16:12:45,099][105692] Updated weights for policy 0, policy_version 116244 (0.0009) [2023-12-26 16:12:45,160][105692] Updated weights for policy 0, policy_version 116254 (0.0009) [2023-12-26 16:12:45,214][105692] Updated weights for policy 0, policy_version 116264 (0.0010) [2023-12-26 16:12:45,749][105620] Updated weights for policy 1, policy_version 116779 (0.0006) [2023-12-26 16:12:45,809][105620] Updated weights for policy 1, policy_version 116789 (0.0006) [2023-12-26 16:12:45,858][105620] Updated weights for policy 1, policy_version 116799 (0.0008) [2023-12-26 16:12:45,931][105692] Updated weights for policy 0, policy_version 116274 (0.0006) [2023-12-26 16:12:45,986][105692] Updated weights for policy 0, policy_version 116284 (0.0006) [2023-12-26 16:12:46,045][105692] Updated weights for policy 0, policy_version 116294 (0.0008) [2023-12-26 16:12:46,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 59686912. Throughput: 0: 9798.4, 1: 9709.7. Samples: 59650188. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-26 16:12:46,062][104569] Avg episode reward: [(0, '8181.100'), (1, '9355.800')] [2023-12-26 16:12:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000116296_29777920.pth... [2023-12-26 16:12:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000116808_29908992.pth... [2023-12-26 16:12:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000115112_29474816.pth [2023-12-26 16:12:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000115656_29614080.pth [2023-12-26 16:12:46,629][105620] Updated weights for policy 1, policy_version 116809 (0.0008) [2023-12-26 16:12:46,677][105620] Updated weights for policy 1, policy_version 116819 (0.0009) [2023-12-26 16:12:46,686][105692] Updated weights for policy 0, policy_version 116304 (0.0006) [2023-12-26 16:12:46,733][105620] Updated weights for policy 1, policy_version 116829 (0.0009) [2023-12-26 16:12:46,735][105692] Updated weights for policy 0, policy_version 116314 (0.0008) [2023-12-26 16:12:46,782][105692] Updated weights for policy 0, policy_version 116324 (0.0007) [2023-12-26 16:12:46,798][105620] Updated weights for policy 1, policy_version 116839 (0.0008) [2023-12-26 16:12:47,450][105692] Updated weights for policy 0, policy_version 116334 (0.0006) [2023-12-26 16:12:47,506][105692] Updated weights for policy 0, policy_version 116344 (0.0006) [2023-12-26 16:12:47,558][105692] Updated weights for policy 0, policy_version 116354 (0.0006) [2023-12-26 16:12:47,608][105620] Updated weights for policy 1, policy_version 116849 (0.0009) [2023-12-26 16:12:47,655][105620] Updated weights for policy 1, policy_version 116860 (0.0008) [2023-12-26 16:12:47,706][105620] Updated weights for policy 1, policy_version 116870 (0.0007) [2023-12-26 16:12:48,209][105692] Updated weights for policy 0, policy_version 116364 (0.0006) [2023-12-26 16:12:48,269][105692] Updated weights for policy 0, policy_version 116374 (0.0006) [2023-12-26 16:12:48,329][105692] Updated weights for policy 0, policy_version 116384 (0.0006) [2023-12-26 16:12:48,495][105620] Updated weights for policy 1, policy_version 116880 (0.0008) [2023-12-26 16:12:48,546][105620] Updated weights for policy 1, policy_version 116890 (0.0005) [2023-12-26 16:12:48,604][105620] Updated weights for policy 1, policy_version 116900 (0.0006) [2023-12-26 16:12:48,981][105692] Updated weights for policy 0, policy_version 116394 (0.0007) [2023-12-26 16:12:49,043][105692] Updated weights for policy 0, policy_version 116404 (0.0009) [2023-12-26 16:12:49,092][105692] Updated weights for policy 0, policy_version 116414 (0.0008) [2023-12-26 16:12:49,142][105692] Updated weights for policy 0, policy_version 116424 (0.0005) [2023-12-26 16:12:49,332][105620] Updated weights for policy 1, policy_version 116910 (0.0008) [2023-12-26 16:12:49,402][105620] Updated weights for policy 1, policy_version 116920 (0.0008) [2023-12-26 16:12:49,468][105620] Updated weights for policy 1, policy_version 116930 (0.0009) [2023-12-26 16:12:49,824][105692] Updated weights for policy 0, policy_version 116434 (0.0009) [2023-12-26 16:12:49,891][105692] Updated weights for policy 0, policy_version 116444 (0.0008) [2023-12-26 16:12:49,959][105692] Updated weights for policy 0, policy_version 116454 (0.0008) [2023-12-26 16:12:50,254][105620] Updated weights for policy 1, policy_version 116940 (0.0009) [2023-12-26 16:12:50,311][105620] Updated weights for policy 1, policy_version 116950 (0.0008) [2023-12-26 16:12:50,373][105620] Updated weights for policy 1, policy_version 116960 (0.0008) [2023-12-26 16:12:50,708][105692] Updated weights for policy 0, policy_version 116464 (0.0009) [2023-12-26 16:12:50,769][105692] Updated weights for policy 0, policy_version 116474 (0.0009) [2023-12-26 16:12:50,834][105692] Updated weights for policy 0, policy_version 116484 (0.0009) [2023-12-26 16:12:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 59777024. Throughput: 0: 9948.2, 1: 9667.5. Samples: 59767736. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-26 16:12:51,063][104569] Avg episode reward: [(0, '8546.213'), (1, '9266.027')] [2023-12-26 16:12:51,088][105620] Updated weights for policy 1, policy_version 116970 (0.0009) [2023-12-26 16:12:51,144][105620] Updated weights for policy 1, policy_version 116980 (0.0009) [2023-12-26 16:12:51,201][105620] Updated weights for policy 1, policy_version 116990 (0.0009) [2023-12-26 16:12:51,248][105620] Updated weights for policy 1, policy_version 117000 (0.0008) [2023-12-26 16:12:51,659][105692] Updated weights for policy 0, policy_version 116494 (0.0009) [2023-12-26 16:12:51,714][105692] Updated weights for policy 0, policy_version 116504 (0.0010) [2023-12-26 16:12:51,783][105692] Updated weights for policy 0, policy_version 116514 (0.0010) [2023-12-26 16:12:51,953][105620] Updated weights for policy 1, policy_version 117010 (0.0010) [2023-12-26 16:12:52,006][105620] Updated weights for policy 1, policy_version 117020 (0.0008) [2023-12-26 16:12:52,063][105620] Updated weights for policy 1, policy_version 117030 (0.0008) [2023-12-26 16:12:52,551][105692] Updated weights for policy 0, policy_version 116524 (0.0008) [2023-12-26 16:12:52,610][105692] Updated weights for policy 0, policy_version 116534 (0.0009) [2023-12-26 16:12:52,668][105692] Updated weights for policy 0, policy_version 116544 (0.0009) [2023-12-26 16:12:52,817][105620] Updated weights for policy 1, policy_version 117040 (0.0006) [2023-12-26 16:12:52,869][105620] Updated weights for policy 1, policy_version 117050 (0.0008) [2023-12-26 16:12:52,923][105620] Updated weights for policy 1, policy_version 117060 (0.0008) [2023-12-26 16:12:53,363][105692] Updated weights for policy 0, policy_version 116554 (0.0009) [2023-12-26 16:12:53,417][105692] Updated weights for policy 0, policy_version 116564 (0.0010) [2023-12-26 16:12:53,488][105692] Updated weights for policy 0, policy_version 116574 (0.0010) [2023-12-26 16:12:53,541][105692] Updated weights for policy 0, policy_version 116584 (0.0010) [2023-12-26 16:12:53,639][105620] Updated weights for policy 1, policy_version 117070 (0.0009) [2023-12-26 16:12:53,697][105620] Updated weights for policy 1, policy_version 117080 (0.0009) [2023-12-26 16:12:53,754][105620] Updated weights for policy 1, policy_version 117090 (0.0009) [2023-12-26 16:12:54,300][105692] Updated weights for policy 0, policy_version 116594 (0.0007) [2023-12-26 16:12:54,348][105692] Updated weights for policy 0, policy_version 116604 (0.0005) [2023-12-26 16:12:54,401][105692] Updated weights for policy 0, policy_version 116614 (0.0005) [2023-12-26 16:12:54,546][105620] Updated weights for policy 1, policy_version 117100 (0.0009) [2023-12-26 16:12:54,605][105620] Updated weights for policy 1, policy_version 117110 (0.0010) [2023-12-26 16:12:54,659][105620] Updated weights for policy 1, policy_version 117120 (0.0010) [2023-12-26 16:12:54,987][105692] Updated weights for policy 0, policy_version 116624 (0.0006) [2023-12-26 16:12:55,048][105692] Updated weights for policy 0, policy_version 116634 (0.0005) [2023-12-26 16:12:55,108][105692] Updated weights for policy 0, policy_version 116644 (0.0005) [2023-12-26 16:12:55,485][105620] Updated weights for policy 1, policy_version 117130 (0.0010) [2023-12-26 16:12:55,538][105620] Updated weights for policy 1, policy_version 117140 (0.0009) [2023-12-26 16:12:55,591][105620] Updated weights for policy 1, policy_version 117150 (0.0008) [2023-12-26 16:12:55,643][105620] Updated weights for policy 1, policy_version 117160 (0.0008) [2023-12-26 16:12:55,659][105692] Updated weights for policy 0, policy_version 116654 (0.0007) [2023-12-26 16:12:55,718][105692] Updated weights for policy 0, policy_version 116664 (0.0009) [2023-12-26 16:12:55,781][105692] Updated weights for policy 0, policy_version 116674 (0.0009) [2023-12-26 16:12:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 59875328. Throughput: 0: 9999.5, 1: 9607.2. Samples: 59882960. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-26 16:12:56,062][104569] Avg episode reward: [(0, '8720.887'), (1, '9265.941')] [2023-12-26 16:12:56,379][105620] Updated weights for policy 1, policy_version 117170 (0.0005) [2023-12-26 16:12:56,449][105620] Updated weights for policy 1, policy_version 117180 (0.0005) [2023-12-26 16:12:56,499][105692] Updated weights for policy 0, policy_version 116684 (0.0009) [2023-12-26 16:12:56,513][105620] Updated weights for policy 1, policy_version 117190 (0.0008) [2023-12-26 16:12:56,548][105692] Updated weights for policy 0, policy_version 116694 (0.0007) [2023-12-26 16:12:56,599][105692] Updated weights for policy 0, policy_version 116704 (0.0007) [2023-12-26 16:12:57,120][105620] Updated weights for policy 1, policy_version 117200 (0.0008) [2023-12-26 16:12:57,178][105620] Updated weights for policy 1, policy_version 117210 (0.0010) [2023-12-26 16:12:57,237][105620] Updated weights for policy 1, policy_version 117220 (0.0011) [2023-12-26 16:12:57,368][105692] Updated weights for policy 0, policy_version 116714 (0.0008) [2023-12-26 16:12:57,424][105692] Updated weights for policy 0, policy_version 116724 (0.0005) [2023-12-26 16:12:57,479][105692] Updated weights for policy 0, policy_version 116734 (0.0005) [2023-12-26 16:12:57,532][105692] Updated weights for policy 0, policy_version 116744 (0.0005) [2023-12-26 16:12:57,914][105620] Updated weights for policy 1, policy_version 117230 (0.0010) [2023-12-26 16:12:57,958][105620] Updated weights for policy 1, policy_version 117240 (0.0010) [2023-12-26 16:12:58,002][105620] Updated weights for policy 1, policy_version 117250 (0.0010) [2023-12-26 16:12:58,116][105692] Updated weights for policy 0, policy_version 116754 (0.0009) [2023-12-26 16:12:58,177][105692] Updated weights for policy 0, policy_version 116764 (0.0009) [2023-12-26 16:12:58,238][105692] Updated weights for policy 0, policy_version 116774 (0.0008) [2023-12-26 16:12:58,781][105620] Updated weights for policy 1, policy_version 117260 (0.0009) [2023-12-26 16:12:58,848][105620] Updated weights for policy 1, policy_version 117270 (0.0007) [2023-12-26 16:12:58,914][105620] Updated weights for policy 1, policy_version 117280 (0.0007) [2023-12-26 16:12:59,025][105692] Updated weights for policy 0, policy_version 116784 (0.0009) [2023-12-26 16:12:59,078][105692] Updated weights for policy 0, policy_version 116794 (0.0008) [2023-12-26 16:12:59,140][105692] Updated weights for policy 0, policy_version 116804 (0.0009) [2023-12-26 16:12:59,625][105620] Updated weights for policy 1, policy_version 117290 (0.0007) [2023-12-26 16:12:59,684][105620] Updated weights for policy 1, policy_version 117300 (0.0010) [2023-12-26 16:12:59,741][105620] Updated weights for policy 1, policy_version 117310 (0.0011) [2023-12-26 16:12:59,809][105620] Updated weights for policy 1, policy_version 117320 (0.0007) [2023-12-26 16:12:59,863][105692] Updated weights for policy 0, policy_version 116814 (0.0009) [2023-12-26 16:12:59,927][105692] Updated weights for policy 0, policy_version 116824 (0.0008) [2023-12-26 16:12:59,990][105692] Updated weights for policy 0, policy_version 116834 (0.0008) [2023-12-26 16:13:00,534][105620] Updated weights for policy 1, policy_version 117330 (0.0010) [2023-12-26 16:13:00,587][105620] Updated weights for policy 1, policy_version 117340 (0.0010) [2023-12-26 16:13:00,644][105620] Updated weights for policy 1, policy_version 117350 (0.0010) [2023-12-26 16:13:00,733][105692] Updated weights for policy 0, policy_version 116844 (0.0008) [2023-12-26 16:13:00,784][105692] Updated weights for policy 0, policy_version 116854 (0.0007) [2023-12-26 16:13:00,828][105692] Updated weights for policy 0, policy_version 116864 (0.0008) [2023-12-26 16:13:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 59973632. Throughput: 0: 10025.1, 1: 9578.0. Samples: 59942756. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-26 16:13:01,062][104569] Avg episode reward: [(0, '8989.475'), (1, '9264.867')] [2023-12-26 16:13:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000116872_29925376.pth... [2023-12-26 16:13:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000117352_30048256.pth... [2023-12-26 16:13:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000115720_29630464.pth [2023-12-26 16:13:01,089][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000116232_29761536.pth [2023-12-26 16:13:01,355][105620] Updated weights for policy 1, policy_version 117360 (0.0008) [2023-12-26 16:13:01,426][105620] Updated weights for policy 1, policy_version 117370 (0.0006) [2023-12-26 16:13:01,486][105620] Updated weights for policy 1, policy_version 117380 (0.0007) [2023-12-26 16:13:01,575][105692] Updated weights for policy 0, policy_version 116874 (0.0007) [2023-12-26 16:13:01,638][105692] Updated weights for policy 0, policy_version 116884 (0.0006) [2023-12-26 16:13:01,692][105692] Updated weights for policy 0, policy_version 116894 (0.0008) [2023-12-26 16:13:01,759][105692] Updated weights for policy 0, policy_version 116904 (0.0007) [2023-12-26 16:13:02,122][105620] Updated weights for policy 1, policy_version 117390 (0.0009) [2023-12-26 16:13:02,177][105620] Updated weights for policy 1, policy_version 117400 (0.0010) [2023-12-26 16:13:02,226][105620] Updated weights for policy 1, policy_version 117410 (0.0010) [2023-12-26 16:13:02,469][105692] Updated weights for policy 0, policy_version 116914 (0.0008) [2023-12-26 16:13:02,513][105692] Updated weights for policy 0, policy_version 116924 (0.0008) [2023-12-26 16:13:02,569][105692] Updated weights for policy 0, policy_version 116934 (0.0008) [2023-12-26 16:13:02,952][105620] Updated weights for policy 1, policy_version 117420 (0.0010) [2023-12-26 16:13:02,997][105620] Updated weights for policy 1, policy_version 117430 (0.0008) [2023-12-26 16:13:03,040][105620] Updated weights for policy 1, policy_version 117440 (0.0005) [2023-12-26 16:13:03,399][105692] Updated weights for policy 0, policy_version 116944 (0.0008) [2023-12-26 16:13:03,444][105692] Updated weights for policy 0, policy_version 116954 (0.0008) [2023-12-26 16:13:03,498][105692] Updated weights for policy 0, policy_version 116964 (0.0007) [2023-12-26 16:13:03,750][105620] Updated weights for policy 1, policy_version 117450 (0.0005) [2023-12-26 16:13:03,798][105620] Updated weights for policy 1, policy_version 117460 (0.0005) [2023-12-26 16:13:03,867][105620] Updated weights for policy 1, policy_version 117470 (0.0007) [2023-12-26 16:13:03,932][105620] Updated weights for policy 1, policy_version 117480 (0.0008) [2023-12-26 16:13:04,206][105692] Updated weights for policy 0, policy_version 116974 (0.0008) [2023-12-26 16:13:04,267][105692] Updated weights for policy 0, policy_version 116984 (0.0012) [2023-12-26 16:13:04,333][105692] Updated weights for policy 0, policy_version 116994 (0.0009) [2023-12-26 16:13:04,576][105620] Updated weights for policy 1, policy_version 117490 (0.0008) [2023-12-26 16:13:04,633][105620] Updated weights for policy 1, policy_version 117500 (0.0008) [2023-12-26 16:13:04,687][105620] Updated weights for policy 1, policy_version 117510 (0.0007) [2023-12-26 16:13:05,121][105692] Updated weights for policy 0, policy_version 117004 (0.0009) [2023-12-26 16:13:05,169][105692] Updated weights for policy 0, policy_version 117014 (0.0009) [2023-12-26 16:13:05,227][105692] Updated weights for policy 0, policy_version 117024 (0.0009) [2023-12-26 16:13:05,404][105620] Updated weights for policy 1, policy_version 117520 (0.0009) [2023-12-26 16:13:05,457][105620] Updated weights for policy 1, policy_version 117530 (0.0009) [2023-12-26 16:13:05,504][105620] Updated weights for policy 1, policy_version 117540 (0.0009) [2023-12-26 16:13:05,962][105692] Updated weights for policy 0, policy_version 117034 (0.0009) [2023-12-26 16:13:06,009][105692] Updated weights for policy 0, policy_version 117044 (0.0008) [2023-12-26 16:13:06,053][105692] Updated weights for policy 0, policy_version 117054 (0.0008) [2023-12-26 16:13:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 60063744. Throughput: 0: 9926.6, 1: 9611.8. Samples: 60058388. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-26 16:13:06,062][104569] Avg episode reward: [(0, '8989.489'), (1, '9354.196')] [2023-12-26 16:13:06,107][105692] Updated weights for policy 0, policy_version 117064 (0.0009) [2023-12-26 16:13:06,180][105620] Updated weights for policy 1, policy_version 117550 (0.0008) [2023-12-26 16:13:06,234][105620] Updated weights for policy 1, policy_version 117560 (0.0006) [2023-12-26 16:13:06,301][105620] Updated weights for policy 1, policy_version 117570 (0.0007) [2023-12-26 16:13:06,920][105692] Updated weights for policy 0, policy_version 117074 (0.0008) [2023-12-26 16:13:06,964][105692] Updated weights for policy 0, policy_version 117084 (0.0007) [2023-12-26 16:13:07,015][105692] Updated weights for policy 0, policy_version 117094 (0.0007) [2023-12-26 16:13:07,025][105620] Updated weights for policy 1, policy_version 117580 (0.0011) [2023-12-26 16:13:07,077][105620] Updated weights for policy 1, policy_version 117590 (0.0010) [2023-12-26 16:13:07,125][105620] Updated weights for policy 1, policy_version 117600 (0.0010) [2023-12-26 16:13:07,743][105692] Updated weights for policy 0, policy_version 117104 (0.0005) [2023-12-26 16:13:07,795][105692] Updated weights for policy 0, policy_version 117114 (0.0006) [2023-12-26 16:13:07,851][105692] Updated weights for policy 0, policy_version 117124 (0.0008) [2023-12-26 16:13:07,896][105620] Updated weights for policy 1, policy_version 117610 (0.0010) [2023-12-26 16:13:07,946][105620] Updated weights for policy 1, policy_version 117620 (0.0009) [2023-12-26 16:13:07,997][105620] Updated weights for policy 1, policy_version 117630 (0.0010) [2023-12-26 16:13:08,048][105620] Updated weights for policy 1, policy_version 117640 (0.0010) [2023-12-26 16:13:08,559][105692] Updated weights for policy 0, policy_version 117134 (0.0008) [2023-12-26 16:13:08,628][105692] Updated weights for policy 0, policy_version 117144 (0.0008) [2023-12-26 16:13:08,692][105692] Updated weights for policy 0, policy_version 117154 (0.0008) [2023-12-26 16:13:08,835][105620] Updated weights for policy 1, policy_version 117650 (0.0010) [2023-12-26 16:13:08,891][105620] Updated weights for policy 1, policy_version 117660 (0.0010) [2023-12-26 16:13:08,942][105620] Updated weights for policy 1, policy_version 117670 (0.0010) [2023-12-26 16:13:09,545][105692] Updated weights for policy 0, policy_version 117164 (0.0009) [2023-12-26 16:13:09,566][105620] Updated weights for policy 1, policy_version 117680 (0.0006) [2023-12-26 16:13:09,600][105692] Updated weights for policy 0, policy_version 117174 (0.0009) [2023-12-26 16:13:09,614][105620] Updated weights for policy 1, policy_version 117690 (0.0005) [2023-12-26 16:13:09,655][105692] Updated weights for policy 0, policy_version 117184 (0.0009) [2023-12-26 16:13:09,663][105620] Updated weights for policy 1, policy_version 117700 (0.0005) [2023-12-26 16:13:10,263][105620] Updated weights for policy 1, policy_version 117710 (0.0008) [2023-12-26 16:13:10,316][105620] Updated weights for policy 1, policy_version 117720 (0.0008) [2023-12-26 16:13:10,378][105620] Updated weights for policy 1, policy_version 117730 (0.0005) [2023-12-26 16:13:10,509][105692] Updated weights for policy 0, policy_version 117194 (0.0009) [2023-12-26 16:13:10,577][105692] Updated weights for policy 0, policy_version 117204 (0.0007) [2023-12-26 16:13:10,647][105692] Updated weights for policy 0, policy_version 117214 (0.0008) [2023-12-26 16:13:10,732][105692] Updated weights for policy 0, policy_version 117224 (0.0005) [2023-12-26 16:13:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 60162048. Throughput: 0: 9812.3, 1: 9613.0. Samples: 60172940. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-26 16:13:11,062][104569] Avg episode reward: [(0, '8999.675'), (1, '9354.176')] [2023-12-26 16:13:11,236][105620] Updated weights for policy 1, policy_version 117740 (0.0008) [2023-12-26 16:13:11,298][105620] Updated weights for policy 1, policy_version 117750 (0.0007) [2023-12-26 16:13:11,316][105692] Updated weights for policy 0, policy_version 117234 (0.0009) [2023-12-26 16:13:11,362][105620] Updated weights for policy 1, policy_version 117760 (0.0008) [2023-12-26 16:13:11,383][105692] Updated weights for policy 0, policy_version 117244 (0.0011) [2023-12-26 16:13:11,446][105692] Updated weights for policy 0, policy_version 117254 (0.0009) [2023-12-26 16:13:12,122][105692] Updated weights for policy 0, policy_version 117264 (0.0008) [2023-12-26 16:13:12,154][105620] Updated weights for policy 1, policy_version 117770 (0.0009) [2023-12-26 16:13:12,185][105692] Updated weights for policy 0, policy_version 117274 (0.0010) [2023-12-26 16:13:12,217][105620] Updated weights for policy 1, policy_version 117780 (0.0011) [2023-12-26 16:13:12,249][105692] Updated weights for policy 0, policy_version 117284 (0.0011) [2023-12-26 16:13:12,281][105620] Updated weights for policy 1, policy_version 117790 (0.0011) [2023-12-26 16:13:12,346][105620] Updated weights for policy 1, policy_version 117800 (0.0011) [2023-12-26 16:13:12,855][105692] Updated weights for policy 0, policy_version 117294 (0.0011) [2023-12-26 16:13:12,919][105692] Updated weights for policy 0, policy_version 117304 (0.0008) [2023-12-26 16:13:12,986][105692] Updated weights for policy 0, policy_version 117314 (0.0010) [2023-12-26 16:13:13,005][105620] Updated weights for policy 1, policy_version 117810 (0.0006) [2023-12-26 16:13:13,056][105620] Updated weights for policy 1, policy_version 117820 (0.0005) [2023-12-26 16:13:13,114][105620] Updated weights for policy 1, policy_version 117830 (0.0005) [2023-12-26 16:13:13,564][105692] Updated weights for policy 0, policy_version 117324 (0.0008) [2023-12-26 16:13:13,610][105692] Updated weights for policy 0, policy_version 117334 (0.0005) [2023-12-26 16:13:13,656][105692] Updated weights for policy 0, policy_version 117344 (0.0007) [2023-12-26 16:13:13,730][105620] Updated weights for policy 1, policy_version 117840 (0.0009) [2023-12-26 16:13:13,785][105620] Updated weights for policy 1, policy_version 117850 (0.0006) [2023-12-26 16:13:13,841][105620] Updated weights for policy 1, policy_version 117860 (0.0005) [2023-12-26 16:13:14,314][105692] Updated weights for policy 0, policy_version 117354 (0.0009) [2023-12-26 16:13:14,370][105692] Updated weights for policy 0, policy_version 117364 (0.0008) [2023-12-26 16:13:14,427][105692] Updated weights for policy 0, policy_version 117374 (0.0009) [2023-12-26 16:13:14,452][105620] Updated weights for policy 1, policy_version 117870 (0.0006) [2023-12-26 16:13:14,496][105692] Updated weights for policy 0, policy_version 117384 (0.0010) [2023-12-26 16:13:14,513][105620] Updated weights for policy 1, policy_version 117880 (0.0005) [2023-12-26 16:13:14,584][105620] Updated weights for policy 1, policy_version 117890 (0.0008) [2023-12-26 16:13:15,051][105692] Updated weights for policy 0, policy_version 117394 (0.0006) [2023-12-26 16:13:15,107][105692] Updated weights for policy 0, policy_version 117404 (0.0007) [2023-12-26 16:13:15,167][105692] Updated weights for policy 0, policy_version 117414 (0.0011) [2023-12-26 16:13:15,221][105620] Updated weights for policy 1, policy_version 117900 (0.0008) [2023-12-26 16:13:15,275][105620] Updated weights for policy 1, policy_version 117910 (0.0008) [2023-12-26 16:13:15,333][105620] Updated weights for policy 1, policy_version 117920 (0.0009) [2023-12-26 16:13:15,760][105692] Updated weights for policy 0, policy_version 117424 (0.0006) [2023-12-26 16:13:15,820][105692] Updated weights for policy 0, policy_version 117434 (0.0005) [2023-12-26 16:13:15,871][105692] Updated weights for policy 0, policy_version 117444 (0.0005) [2023-12-26 16:13:15,988][105620] Updated weights for policy 1, policy_version 117930 (0.0009) [2023-12-26 16:13:16,049][105620] Updated weights for policy 1, policy_version 117940 (0.0009) [2023-12-26 16:13:16,062][104569] Fps is (10 sec: 20479.4, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 60268544. Throughput: 0: 9785.2, 1: 9665.1. Samples: 60234964. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-26 16:13:16,063][104569] Avg episode reward: [(0, '9176.301'), (1, '9262.653')] [2023-12-26 16:13:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000117448_30072832.pth... [2023-12-26 16:13:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000116296_29777920.pth [2023-12-26 16:13:16,102][105620] Updated weights for policy 1, policy_version 117950 (0.0009) [2023-12-26 16:13:16,164][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000117960_30203904.pth... [2023-12-26 16:13:16,167][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000116808_29908992.pth [2023-12-26 16:13:16,168][105620] Updated weights for policy 1, policy_version 117960 (0.0010) [2023-12-26 16:13:16,407][105692] Updated weights for policy 0, policy_version 117454 (0.0008) [2023-12-26 16:13:16,468][105692] Updated weights for policy 0, policy_version 117464 (0.0010) [2023-12-26 16:13:16,536][105692] Updated weights for policy 0, policy_version 117474 (0.0010) [2023-12-26 16:13:17,035][105620] Updated weights for policy 1, policy_version 117970 (0.0010) [2023-12-26 16:13:17,085][105620] Updated weights for policy 1, policy_version 117980 (0.0009) [2023-12-26 16:13:17,098][105692] Updated weights for policy 0, policy_version 117484 (0.0010) [2023-12-26 16:13:17,148][105692] Updated weights for policy 0, policy_version 117494 (0.0006) [2023-12-26 16:13:17,148][105620] Updated weights for policy 1, policy_version 117990 (0.0008) [2023-12-26 16:13:17,203][105692] Updated weights for policy 0, policy_version 117504 (0.0005) [2023-12-26 16:13:17,760][105692] Updated weights for policy 0, policy_version 117514 (0.0006) [2023-12-26 16:13:17,827][105692] Updated weights for policy 0, policy_version 117524 (0.0011) [2023-12-26 16:13:17,885][105692] Updated weights for policy 0, policy_version 117534 (0.0011) [2023-12-26 16:13:17,945][105692] Updated weights for policy 0, policy_version 117544 (0.0011) [2023-12-26 16:13:17,985][105620] Updated weights for policy 1, policy_version 118000 (0.0008) [2023-12-26 16:13:18,045][105620] Updated weights for policy 1, policy_version 118010 (0.0009) [2023-12-26 16:13:18,098][105620] Updated weights for policy 1, policy_version 118020 (0.0010) [2023-12-26 16:13:18,629][105692] Updated weights for policy 0, policy_version 117554 (0.0009) [2023-12-26 16:13:18,688][105692] Updated weights for policy 0, policy_version 117564 (0.0009) [2023-12-26 16:13:18,762][105692] Updated weights for policy 0, policy_version 117574 (0.0010) [2023-12-26 16:13:18,884][105620] Updated weights for policy 1, policy_version 118030 (0.0008) [2023-12-26 16:13:18,945][105620] Updated weights for policy 1, policy_version 118040 (0.0009) [2023-12-26 16:13:19,010][105620] Updated weights for policy 1, policy_version 118050 (0.0009) [2023-12-26 16:13:19,508][105692] Updated weights for policy 0, policy_version 117584 (0.0010) [2023-12-26 16:13:19,565][105692] Updated weights for policy 0, policy_version 117594 (0.0009) [2023-12-26 16:13:19,628][105692] Updated weights for policy 0, policy_version 117604 (0.0009) [2023-12-26 16:13:19,810][105620] Updated weights for policy 1, policy_version 118060 (0.0009) [2023-12-26 16:13:19,877][105620] Updated weights for policy 1, policy_version 118070 (0.0009) [2023-12-26 16:13:19,943][105620] Updated weights for policy 1, policy_version 118080 (0.0010) [2023-12-26 16:13:20,323][105692] Updated weights for policy 0, policy_version 117614 (0.0009) [2023-12-26 16:13:20,385][105692] Updated weights for policy 0, policy_version 117624 (0.0009) [2023-12-26 16:13:20,448][105692] Updated weights for policy 0, policy_version 117634 (0.0009) [2023-12-26 16:13:20,767][105620] Updated weights for policy 1, policy_version 118090 (0.0010) [2023-12-26 16:13:20,835][105620] Updated weights for policy 1, policy_version 118100 (0.0006) [2023-12-26 16:13:20,887][105620] Updated weights for policy 1, policy_version 118110 (0.0007) [2023-12-26 16:13:20,943][105620] Updated weights for policy 1, policy_version 118120 (0.0008) [2023-12-26 16:13:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 60366848. Throughput: 0: 9990.9, 1: 9515.0. Samples: 60356696. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-26 16:13:21,063][104569] Avg episode reward: [(0, '9176.475'), (1, '9175.177')] [2023-12-26 16:13:21,201][105692] Updated weights for policy 0, policy_version 117644 (0.0009) [2023-12-26 16:13:21,265][105692] Updated weights for policy 0, policy_version 117654 (0.0011) [2023-12-26 16:13:21,332][105692] Updated weights for policy 0, policy_version 117664 (0.0011) [2023-12-26 16:13:21,726][105620] Updated weights for policy 1, policy_version 118130 (0.0009) [2023-12-26 16:13:21,792][105620] Updated weights for policy 1, policy_version 118140 (0.0008) [2023-12-26 16:13:21,841][105620] Updated weights for policy 1, policy_version 118150 (0.0008) [2023-12-26 16:13:22,114][105692] Updated weights for policy 0, policy_version 117674 (0.0011) [2023-12-26 16:13:22,166][105692] Updated weights for policy 0, policy_version 117684 (0.0010) [2023-12-26 16:13:22,214][105692] Updated weights for policy 0, policy_version 117694 (0.0010) [2023-12-26 16:13:22,263][105692] Updated weights for policy 0, policy_version 117704 (0.0010) [2023-12-26 16:13:22,595][105620] Updated weights for policy 1, policy_version 118160 (0.0006) [2023-12-26 16:13:22,657][105620] Updated weights for policy 1, policy_version 118170 (0.0007) [2023-12-26 16:13:22,723][105620] Updated weights for policy 1, policy_version 118180 (0.0005) [2023-12-26 16:13:22,976][105692] Updated weights for policy 0, policy_version 117714 (0.0011) [2023-12-26 16:13:23,032][105692] Updated weights for policy 0, policy_version 117724 (0.0011) [2023-12-26 16:13:23,085][105692] Updated weights for policy 0, policy_version 117734 (0.0010) [2023-12-26 16:13:23,253][105620] Updated weights for policy 1, policy_version 118190 (0.0008) [2023-12-26 16:13:23,311][105620] Updated weights for policy 1, policy_version 118200 (0.0010) [2023-12-26 16:13:23,366][105620] Updated weights for policy 1, policy_version 118210 (0.0010) [2023-12-26 16:13:23,845][105692] Updated weights for policy 0, policy_version 117744 (0.0008) [2023-12-26 16:13:23,896][105692] Updated weights for policy 0, policy_version 117754 (0.0008) [2023-12-26 16:13:23,943][105692] Updated weights for policy 0, policy_version 117764 (0.0008) [2023-12-26 16:13:24,089][105620] Updated weights for policy 1, policy_version 118220 (0.0010) [2023-12-26 16:13:24,151][105620] Updated weights for policy 1, policy_version 118230 (0.0010) [2023-12-26 16:13:24,203][105620] Updated weights for policy 1, policy_version 118240 (0.0010) [2023-12-26 16:13:24,732][105692] Updated weights for policy 0, policy_version 117774 (0.0008) [2023-12-26 16:13:24,786][105692] Updated weights for policy 0, policy_version 117784 (0.0008) [2023-12-26 16:13:24,837][105692] Updated weights for policy 0, policy_version 117794 (0.0008) [2023-12-26 16:13:24,961][105620] Updated weights for policy 1, policy_version 118250 (0.0010) [2023-12-26 16:13:25,009][105620] Updated weights for policy 1, policy_version 118260 (0.0010) [2023-12-26 16:13:25,064][105620] Updated weights for policy 1, policy_version 118270 (0.0010) [2023-12-26 16:13:25,128][105620] Updated weights for policy 1, policy_version 118280 (0.0010) [2023-12-26 16:13:25,590][105692] Updated weights for policy 0, policy_version 117804 (0.0008) [2023-12-26 16:13:25,642][105692] Updated weights for policy 0, policy_version 117814 (0.0008) [2023-12-26 16:13:25,693][105692] Updated weights for policy 0, policy_version 117824 (0.0008) [2023-12-26 16:13:25,869][105620] Updated weights for policy 1, policy_version 118290 (0.0010) [2023-12-26 16:13:25,934][105620] Updated weights for policy 1, policy_version 118300 (0.0010) [2023-12-26 16:13:25,987][105620] Updated weights for policy 1, policy_version 118310 (0.0009) [2023-12-26 16:13:26,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 60465152. Throughput: 0: 9895.4, 1: 9577.6. Samples: 60470368. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-26 16:13:26,062][104569] Avg episode reward: [(0, '8829.360'), (1, '9176.760')] [2023-12-26 16:13:26,469][105692] Updated weights for policy 0, policy_version 117834 (0.0008) [2023-12-26 16:13:26,517][105692] Updated weights for policy 0, policy_version 117844 (0.0007) [2023-12-26 16:13:26,568][105692] Updated weights for policy 0, policy_version 117854 (0.0008) [2023-12-26 16:13:26,617][105692] Updated weights for policy 0, policy_version 117864 (0.0008) [2023-12-26 16:13:26,717][105620] Updated weights for policy 1, policy_version 118320 (0.0010) [2023-12-26 16:13:26,768][105620] Updated weights for policy 1, policy_version 118330 (0.0010) [2023-12-26 16:13:26,819][105620] Updated weights for policy 1, policy_version 118340 (0.0010) [2023-12-26 16:13:27,327][105692] Updated weights for policy 0, policy_version 117874 (0.0010) [2023-12-26 16:13:27,386][105692] Updated weights for policy 0, policy_version 117884 (0.0011) [2023-12-26 16:13:27,431][105692] Updated weights for policy 0, policy_version 117894 (0.0010) [2023-12-26 16:13:27,489][105620] Updated weights for policy 1, policy_version 118350 (0.0007) [2023-12-26 16:13:27,542][105620] Updated weights for policy 1, policy_version 118360 (0.0005) [2023-12-26 16:13:27,596][105620] Updated weights for policy 1, policy_version 118370 (0.0005) [2023-12-26 16:13:28,100][105692] Updated weights for policy 0, policy_version 117904 (0.0011) [2023-12-26 16:13:28,148][105692] Updated weights for policy 0, policy_version 117914 (0.0010) [2023-12-26 16:13:28,195][105692] Updated weights for policy 0, policy_version 117924 (0.0010) [2023-12-26 16:13:28,266][105620] Updated weights for policy 1, policy_version 118380 (0.0010) [2023-12-26 16:13:28,317][105620] Updated weights for policy 1, policy_version 118390 (0.0010) [2023-12-26 16:13:28,382][105620] Updated weights for policy 1, policy_version 118400 (0.0006) [2023-12-26 16:13:28,972][105620] Updated weights for policy 1, policy_version 118410 (0.0005) [2023-12-26 16:13:29,040][105692] Updated weights for policy 0, policy_version 117934 (0.0009) [2023-12-26 16:13:29,041][105620] Updated weights for policy 1, policy_version 118420 (0.0005) [2023-12-26 16:13:29,102][105620] Updated weights for policy 1, policy_version 118430 (0.0006) [2023-12-26 16:13:29,102][105692] Updated weights for policy 0, policy_version 117944 (0.0007) [2023-12-26 16:13:29,160][105692] Updated weights for policy 0, policy_version 117954 (0.0005) [2023-12-26 16:13:29,166][105620] Updated weights for policy 1, policy_version 118440 (0.0005) [2023-12-26 16:13:29,758][105620] Updated weights for policy 1, policy_version 118450 (0.0009) [2023-12-26 16:13:29,804][105620] Updated weights for policy 1, policy_version 118460 (0.0008) [2023-12-26 16:13:29,872][105620] Updated weights for policy 1, policy_version 118470 (0.0009) [2023-12-26 16:13:29,908][105692] Updated weights for policy 0, policy_version 117964 (0.0008) [2023-12-26 16:13:29,970][105692] Updated weights for policy 0, policy_version 117974 (0.0009) [2023-12-26 16:13:30,024][105692] Updated weights for policy 0, policy_version 117984 (0.0008) [2023-12-26 16:13:30,673][105620] Updated weights for policy 1, policy_version 118480 (0.0009) [2023-12-26 16:13:30,702][105692] Updated weights for policy 0, policy_version 117994 (0.0008) [2023-12-26 16:13:30,724][105620] Updated weights for policy 1, policy_version 118490 (0.0007) [2023-12-26 16:13:30,755][105692] Updated weights for policy 0, policy_version 118004 (0.0006) [2023-12-26 16:13:30,776][105620] Updated weights for policy 1, policy_version 118500 (0.0007) [2023-12-26 16:13:30,802][105692] Updated weights for policy 0, policy_version 118014 (0.0006) [2023-12-26 16:13:30,850][105692] Updated weights for policy 0, policy_version 118024 (0.0009) [2023-12-26 16:13:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 60563456. Throughput: 0: 9902.7, 1: 9665.8. Samples: 60530772. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-26 16:13:31,062][104569] Avg episode reward: [(0, '8478.900'), (1, '8831.106')] [2023-12-26 16:13:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000118024_30220288.pth... [2023-12-26 16:13:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000118504_30343168.pth... [2023-12-26 16:13:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000117352_30048256.pth [2023-12-26 16:13:31,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000116872_29925376.pth [2023-12-26 16:13:31,526][105620] Updated weights for policy 1, policy_version 118510 (0.0007) [2023-12-26 16:13:31,575][105620] Updated weights for policy 1, policy_version 118520 (0.0008) [2023-12-26 16:13:31,634][105620] Updated weights for policy 1, policy_version 118530 (0.0008) [2023-12-26 16:13:31,637][105692] Updated weights for policy 0, policy_version 118034 (0.0006) [2023-12-26 16:13:31,691][105692] Updated weights for policy 0, policy_version 118044 (0.0006) [2023-12-26 16:13:31,755][105692] Updated weights for policy 0, policy_version 118054 (0.0008) [2023-12-26 16:13:32,342][105620] Updated weights for policy 1, policy_version 118540 (0.0008) [2023-12-26 16:13:32,403][105620] Updated weights for policy 1, policy_version 118550 (0.0007) [2023-12-26 16:13:32,459][105620] Updated weights for policy 1, policy_version 118560 (0.0008) [2023-12-26 16:13:32,528][105692] Updated weights for policy 0, policy_version 118064 (0.0008) [2023-12-26 16:13:32,576][105692] Updated weights for policy 0, policy_version 118074 (0.0008) [2023-12-26 16:13:32,635][105692] Updated weights for policy 0, policy_version 118084 (0.0011) [2023-12-26 16:13:33,170][105620] Updated weights for policy 1, policy_version 118570 (0.0008) [2023-12-26 16:13:33,216][105620] Updated weights for policy 1, policy_version 118580 (0.0009) [2023-12-26 16:13:33,269][105620] Updated weights for policy 1, policy_version 118590 (0.0009) [2023-12-26 16:13:33,320][105620] Updated weights for policy 1, policy_version 118600 (0.0007) [2023-12-26 16:13:33,327][105692] Updated weights for policy 0, policy_version 118094 (0.0009) [2023-12-26 16:13:33,383][105692] Updated weights for policy 0, policy_version 118104 (0.0009) [2023-12-26 16:13:33,448][105692] Updated weights for policy 0, policy_version 118114 (0.0009) [2023-12-26 16:13:33,902][105620] Updated weights for policy 1, policy_version 118610 (0.0007) [2023-12-26 16:13:33,968][105620] Updated weights for policy 1, policy_version 118620 (0.0007) [2023-12-26 16:13:34,029][105620] Updated weights for policy 1, policy_version 118630 (0.0008) [2023-12-26 16:13:34,094][105692] Updated weights for policy 0, policy_version 118124 (0.0008) [2023-12-26 16:13:34,161][105692] Updated weights for policy 0, policy_version 118134 (0.0007) [2023-12-26 16:13:34,217][105692] Updated weights for policy 0, policy_version 118144 (0.0008) [2023-12-26 16:13:34,765][105620] Updated weights for policy 1, policy_version 118640 (0.0010) [2023-12-26 16:13:34,820][105620] Updated weights for policy 1, policy_version 118650 (0.0010) [2023-12-26 16:13:34,879][105620] Updated weights for policy 1, policy_version 118660 (0.0010) [2023-12-26 16:13:34,977][105692] Updated weights for policy 0, policy_version 118154 (0.0008) [2023-12-26 16:13:35,045][105692] Updated weights for policy 0, policy_version 118164 (0.0010) [2023-12-26 16:13:35,112][105692] Updated weights for policy 0, policy_version 118174 (0.0009) [2023-12-26 16:13:35,183][105692] Updated weights for policy 0, policy_version 118184 (0.0010) [2023-12-26 16:13:35,448][105620] Updated weights for policy 1, policy_version 118670 (0.0007) [2023-12-26 16:13:35,502][105620] Updated weights for policy 1, policy_version 118680 (0.0005) [2023-12-26 16:13:35,571][105620] Updated weights for policy 1, policy_version 118690 (0.0005) [2023-12-26 16:13:35,896][105692] Updated weights for policy 0, policy_version 118194 (0.0008) [2023-12-26 16:13:35,952][105692] Updated weights for policy 0, policy_version 118204 (0.0007) [2023-12-26 16:13:35,997][105692] Updated weights for policy 0, policy_version 118214 (0.0008) [2023-12-26 16:13:36,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 60661760. Throughput: 0: 9790.7, 1: 9746.1. Samples: 60646896. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-26 16:13:36,063][104569] Avg episode reward: [(0, '8200.238'), (1, '8833.485')] [2023-12-26 16:13:36,229][105620] Updated weights for policy 1, policy_version 118700 (0.0008) [2023-12-26 16:13:36,295][105620] Updated weights for policy 1, policy_version 118710 (0.0010) [2023-12-26 16:13:36,355][105620] Updated weights for policy 1, policy_version 118720 (0.0010) [2023-12-26 16:13:36,790][105692] Updated weights for policy 0, policy_version 118224 (0.0008) [2023-12-26 16:13:36,849][105692] Updated weights for policy 0, policy_version 118234 (0.0008) [2023-12-26 16:13:36,909][105692] Updated weights for policy 0, policy_version 118244 (0.0008) [2023-12-26 16:13:37,101][105620] Updated weights for policy 1, policy_version 118730 (0.0010) [2023-12-26 16:13:37,160][105620] Updated weights for policy 1, policy_version 118740 (0.0010) [2023-12-26 16:13:37,222][105620] Updated weights for policy 1, policy_version 118750 (0.0011) [2023-12-26 16:13:37,284][105620] Updated weights for policy 1, policy_version 118760 (0.0010) [2023-12-26 16:13:37,674][105692] Updated weights for policy 0, policy_version 118254 (0.0009) [2023-12-26 16:13:37,730][105692] Updated weights for policy 0, policy_version 118264 (0.0006) [2023-12-26 16:13:37,783][105692] Updated weights for policy 0, policy_version 118274 (0.0005) [2023-12-26 16:13:37,997][105620] Updated weights for policy 1, policy_version 118770 (0.0006) [2023-12-26 16:13:38,053][105620] Updated weights for policy 1, policy_version 118780 (0.0005) [2023-12-26 16:13:38,100][105620] Updated weights for policy 1, policy_version 118790 (0.0005) [2023-12-26 16:13:38,451][105692] Updated weights for policy 0, policy_version 118284 (0.0006) [2023-12-26 16:13:38,501][105692] Updated weights for policy 0, policy_version 118294 (0.0006) [2023-12-26 16:13:38,548][105692] Updated weights for policy 0, policy_version 118304 (0.0005) [2023-12-26 16:13:38,790][105620] Updated weights for policy 1, policy_version 118800 (0.0009) [2023-12-26 16:13:38,838][105620] Updated weights for policy 1, policy_version 118810 (0.0010) [2023-12-26 16:13:38,893][105620] Updated weights for policy 1, policy_version 118820 (0.0010) [2023-12-26 16:13:39,307][105692] Updated weights for policy 0, policy_version 118314 (0.0007) [2023-12-26 16:13:39,374][105692] Updated weights for policy 0, policy_version 118324 (0.0009) [2023-12-26 16:13:39,441][105692] Updated weights for policy 0, policy_version 118334 (0.0008) [2023-12-26 16:13:39,509][105692] Updated weights for policy 0, policy_version 118344 (0.0008) [2023-12-26 16:13:39,640][105620] Updated weights for policy 1, policy_version 118830 (0.0009) [2023-12-26 16:13:39,707][105620] Updated weights for policy 1, policy_version 118840 (0.0011) [2023-12-26 16:13:39,767][105620] Updated weights for policy 1, policy_version 118850 (0.0008) [2023-12-26 16:13:40,358][105692] Updated weights for policy 0, policy_version 118354 (0.0007) [2023-12-26 16:13:40,425][105692] Updated weights for policy 0, policy_version 118364 (0.0008) [2023-12-26 16:13:40,453][105620] Updated weights for policy 1, policy_version 118860 (0.0010) [2023-12-26 16:13:40,489][105692] Updated weights for policy 0, policy_version 118374 (0.0006) [2023-12-26 16:13:40,517][105620] Updated weights for policy 1, policy_version 118870 (0.0011) [2023-12-26 16:13:40,577][105620] Updated weights for policy 1, policy_version 118880 (0.0011) [2023-12-26 16:13:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 60751872. Throughput: 0: 9713.3, 1: 9856.2. Samples: 60763588. Policy #0 lag: (min: 28.0, avg: 52.6, max: 57.0) [2023-12-26 16:13:41,062][104569] Avg episode reward: [(0, '8377.813'), (1, '9179.531')] [2023-12-26 16:13:41,219][105692] Updated weights for policy 0, policy_version 118384 (0.0009) [2023-12-26 16:13:41,256][105620] Updated weights for policy 1, policy_version 118890 (0.0010) [2023-12-26 16:13:41,279][105692] Updated weights for policy 0, policy_version 118394 (0.0009) [2023-12-26 16:13:41,319][105620] Updated weights for policy 1, policy_version 118900 (0.0008) [2023-12-26 16:13:41,344][105692] Updated weights for policy 0, policy_version 118404 (0.0008) [2023-12-26 16:13:41,382][105620] Updated weights for policy 1, policy_version 118910 (0.0008) [2023-12-26 16:13:41,451][105620] Updated weights for policy 1, policy_version 118920 (0.0009) [2023-12-26 16:13:42,102][105620] Updated weights for policy 1, policy_version 118930 (0.0011) [2023-12-26 16:13:42,129][105692] Updated weights for policy 0, policy_version 118414 (0.0007) [2023-12-26 16:13:42,163][105620] Updated weights for policy 1, policy_version 118940 (0.0010) [2023-12-26 16:13:42,185][105692] Updated weights for policy 0, policy_version 118424 (0.0006) [2023-12-26 16:13:42,219][105620] Updated weights for policy 1, policy_version 118950 (0.0010) [2023-12-26 16:13:42,241][105692] Updated weights for policy 0, policy_version 118434 (0.0006) [2023-12-26 16:13:42,938][105620] Updated weights for policy 1, policy_version 118960 (0.0010) [2023-12-26 16:13:42,960][105692] Updated weights for policy 0, policy_version 118444 (0.0007) [2023-12-26 16:13:42,993][105620] Updated weights for policy 1, policy_version 118970 (0.0010) [2023-12-26 16:13:43,018][105692] Updated weights for policy 0, policy_version 118454 (0.0006) [2023-12-26 16:13:43,048][105620] Updated weights for policy 1, policy_version 118980 (0.0010) [2023-12-26 16:13:43,070][105692] Updated weights for policy 0, policy_version 118464 (0.0005) [2023-12-26 16:13:43,676][105620] Updated weights for policy 1, policy_version 118990 (0.0009) [2023-12-26 16:13:43,722][105620] Updated weights for policy 1, policy_version 119000 (0.0008) [2023-12-26 16:13:43,772][105620] Updated weights for policy 1, policy_version 119010 (0.0008) [2023-12-26 16:13:43,869][105692] Updated weights for policy 0, policy_version 118474 (0.0008) [2023-12-26 16:13:43,929][105692] Updated weights for policy 0, policy_version 118484 (0.0009) [2023-12-26 16:13:43,990][105692] Updated weights for policy 0, policy_version 118494 (0.0007) [2023-12-26 16:13:44,041][105692] Updated weights for policy 0, policy_version 118504 (0.0008) [2023-12-26 16:13:44,446][105620] Updated weights for policy 1, policy_version 119020 (0.0008) [2023-12-26 16:13:44,510][105620] Updated weights for policy 1, policy_version 119030 (0.0011) [2023-12-26 16:13:44,562][105620] Updated weights for policy 1, policy_version 119040 (0.0010) [2023-12-26 16:13:44,848][105692] Updated weights for policy 0, policy_version 118514 (0.0008) [2023-12-26 16:13:44,900][105692] Updated weights for policy 0, policy_version 118524 (0.0008) [2023-12-26 16:13:44,948][105692] Updated weights for policy 0, policy_version 118534 (0.0008) [2023-12-26 16:13:45,310][105620] Updated weights for policy 1, policy_version 119050 (0.0010) [2023-12-26 16:13:45,366][105620] Updated weights for policy 1, policy_version 119060 (0.0006) [2023-12-26 16:13:45,413][105620] Updated weights for policy 1, policy_version 119070 (0.0006) [2023-12-26 16:13:45,465][105620] Updated weights for policy 1, policy_version 119080 (0.0006) [2023-12-26 16:13:45,784][105692] Updated weights for policy 0, policy_version 118544 (0.0010) [2023-12-26 16:13:45,837][105692] Updated weights for policy 0, policy_version 118554 (0.0009) [2023-12-26 16:13:45,890][105692] Updated weights for policy 0, policy_version 118564 (0.0010) [2023-12-26 16:13:46,032][105620] Updated weights for policy 1, policy_version 119090 (0.0008) [2023-12-26 16:13:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 60850176. Throughput: 0: 9660.1, 1: 9846.2. Samples: 60820540. Policy #0 lag: (min: 28.0, avg: 52.6, max: 57.0) [2023-12-26 16:13:46,063][104569] Avg episode reward: [(0, '8556.383'), (1, '9086.310')] [2023-12-26 16:13:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000118568_30359552.pth... [2023-12-26 16:13:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000117448_30072832.pth [2023-12-26 16:13:46,090][105620] Updated weights for policy 1, policy_version 119100 (0.0010) [2023-12-26 16:13:46,153][105620] Updated weights for policy 1, policy_version 119110 (0.0011) [2023-12-26 16:13:46,168][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000119112_30498816.pth... [2023-12-26 16:13:46,172][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000117960_30203904.pth [2023-12-26 16:13:46,746][105692] Updated weights for policy 0, policy_version 118575 (0.0011) [2023-12-26 16:13:46,784][105620] Updated weights for policy 1, policy_version 119120 (0.0010) [2023-12-26 16:13:46,787][105692] Updated weights for policy 0, policy_version 118585 (0.0006) [2023-12-26 16:13:46,833][105692] Updated weights for policy 0, policy_version 118595 (0.0007) [2023-12-26 16:13:46,846][105620] Updated weights for policy 1, policy_version 119130 (0.0009) [2023-12-26 16:13:46,900][105620] Updated weights for policy 1, policy_version 119140 (0.0005) [2023-12-26 16:13:47,423][105620] Updated weights for policy 1, policy_version 119150 (0.0005) [2023-12-26 16:13:47,474][105620] Updated weights for policy 1, policy_version 119160 (0.0005) [2023-12-26 16:13:47,527][105620] Updated weights for policy 1, policy_version 119170 (0.0006) [2023-12-26 16:13:47,719][105692] Updated weights for policy 0, policy_version 118605 (0.0008) [2023-12-26 16:13:47,773][105692] Updated weights for policy 0, policy_version 118615 (0.0008) [2023-12-26 16:13:47,828][105692] Updated weights for policy 0, policy_version 118625 (0.0008) [2023-12-26 16:13:48,229][105620] Updated weights for policy 1, policy_version 119180 (0.0010) [2023-12-26 16:13:48,281][105620] Updated weights for policy 1, policy_version 119190 (0.0010) [2023-12-26 16:13:48,346][105620] Updated weights for policy 1, policy_version 119200 (0.0009) [2023-12-26 16:13:48,580][105692] Updated weights for policy 0, policy_version 118635 (0.0008) [2023-12-26 16:13:48,632][105692] Updated weights for policy 0, policy_version 118645 (0.0008) [2023-12-26 16:13:48,680][105692] Updated weights for policy 0, policy_version 118655 (0.0008) [2023-12-26 16:13:49,083][105620] Updated weights for policy 1, policy_version 119210 (0.0008) [2023-12-26 16:13:49,153][105620] Updated weights for policy 1, policy_version 119220 (0.0011) [2023-12-26 16:13:49,217][105620] Updated weights for policy 1, policy_version 119230 (0.0010) [2023-12-26 16:13:49,286][105620] Updated weights for policy 1, policy_version 119240 (0.0009) [2023-12-26 16:13:49,427][105692] Updated weights for policy 0, policy_version 118665 (0.0007) [2023-12-26 16:13:49,487][105692] Updated weights for policy 0, policy_version 118675 (0.0005) [2023-12-26 16:13:49,538][105692] Updated weights for policy 0, policy_version 118685 (0.0007) [2023-12-26 16:13:49,587][105692] Updated weights for policy 0, policy_version 118695 (0.0007) [2023-12-26 16:13:50,020][105620] Updated weights for policy 1, policy_version 119250 (0.0008) [2023-12-26 16:13:50,068][105620] Updated weights for policy 1, policy_version 119260 (0.0008) [2023-12-26 16:13:50,126][105620] Updated weights for policy 1, policy_version 119270 (0.0008) [2023-12-26 16:13:50,279][105692] Updated weights for policy 0, policy_version 118705 (0.0008) [2023-12-26 16:13:50,346][105692] Updated weights for policy 0, policy_version 118715 (0.0007) [2023-12-26 16:13:50,411][105692] Updated weights for policy 0, policy_version 118725 (0.0009) [2023-12-26 16:13:50,921][105620] Updated weights for policy 1, policy_version 119280 (0.0007) [2023-12-26 16:13:50,982][105620] Updated weights for policy 1, policy_version 119290 (0.0009) [2023-12-26 16:13:51,047][105620] Updated weights for policy 1, policy_version 119300 (0.0009) [2023-12-26 16:13:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 60940288. Throughput: 0: 9608.5, 1: 9916.4. Samples: 60937012. Policy #0 lag: (min: 28.0, avg: 52.6, max: 57.0) [2023-12-26 16:13:51,062][104569] Avg episode reward: [(0, '8374.386'), (1, '9086.338')] [2023-12-26 16:13:51,157][105692] Updated weights for policy 0, policy_version 118735 (0.0008) [2023-12-26 16:13:51,215][105692] Updated weights for policy 0, policy_version 118745 (0.0009) [2023-12-26 16:13:51,272][105692] Updated weights for policy 0, policy_version 118755 (0.0010) [2023-12-26 16:13:51,748][105620] Updated weights for policy 1, policy_version 119310 (0.0008) [2023-12-26 16:13:51,811][105620] Updated weights for policy 1, policy_version 119320 (0.0008) [2023-12-26 16:13:51,871][105620] Updated weights for policy 1, policy_version 119330 (0.0006) [2023-12-26 16:13:52,128][105692] Updated weights for policy 0, policy_version 118765 (0.0009) [2023-12-26 16:13:52,190][105692] Updated weights for policy 0, policy_version 118775 (0.0008) [2023-12-26 16:13:52,254][105692] Updated weights for policy 0, policy_version 118785 (0.0006) [2023-12-26 16:13:52,588][105620] Updated weights for policy 1, policy_version 119340 (0.0008) [2023-12-26 16:13:52,648][105620] Updated weights for policy 1, policy_version 119350 (0.0008) [2023-12-26 16:13:52,703][105620] Updated weights for policy 1, policy_version 119360 (0.0008) [2023-12-26 16:13:52,921][105692] Updated weights for policy 0, policy_version 118795 (0.0008) [2023-12-26 16:13:52,977][105692] Updated weights for policy 0, policy_version 118805 (0.0005) [2023-12-26 16:13:53,028][105692] Updated weights for policy 0, policy_version 118815 (0.0006) [2023-12-26 16:13:53,462][105620] Updated weights for policy 1, policy_version 119370 (0.0009) [2023-12-26 16:13:53,514][105620] Updated weights for policy 1, policy_version 119380 (0.0009) [2023-12-26 16:13:53,567][105620] Updated weights for policy 1, policy_version 119390 (0.0010) [2023-12-26 16:13:53,586][105692] Updated weights for policy 0, policy_version 118825 (0.0005) [2023-12-26 16:13:53,624][105620] Updated weights for policy 1, policy_version 119400 (0.0008) [2023-12-26 16:13:53,648][105692] Updated weights for policy 0, policy_version 118835 (0.0007) [2023-12-26 16:13:53,708][105692] Updated weights for policy 0, policy_version 118845 (0.0009) [2023-12-26 16:13:53,772][105692] Updated weights for policy 0, policy_version 118855 (0.0010) [2023-12-26 16:13:54,408][105620] Updated weights for policy 1, policy_version 119410 (0.0009) [2023-12-26 16:13:54,430][105692] Updated weights for policy 0, policy_version 118865 (0.0006) [2023-12-26 16:13:54,465][105620] Updated weights for policy 1, policy_version 119420 (0.0007) [2023-12-26 16:13:54,479][105692] Updated weights for policy 0, policy_version 118875 (0.0006) [2023-12-26 16:13:54,525][105620] Updated weights for policy 1, policy_version 119430 (0.0008) [2023-12-26 16:13:54,527][105692] Updated weights for policy 0, policy_version 118885 (0.0006) [2023-12-26 16:13:55,132][105620] Updated weights for policy 1, policy_version 119440 (0.0006) [2023-12-26 16:13:55,185][105620] Updated weights for policy 1, policy_version 119450 (0.0010) [2023-12-26 16:13:55,234][105620] Updated weights for policy 1, policy_version 119460 (0.0011) [2023-12-26 16:13:55,390][105692] Updated weights for policy 0, policy_version 118895 (0.0007) [2023-12-26 16:13:55,435][105692] Updated weights for policy 0, policy_version 118905 (0.0005) [2023-12-26 16:13:55,486][105692] Updated weights for policy 0, policy_version 118915 (0.0005) [2023-12-26 16:13:55,862][105620] Updated weights for policy 1, policy_version 119470 (0.0007) [2023-12-26 16:13:55,922][105620] Updated weights for policy 1, policy_version 119480 (0.0006) [2023-12-26 16:13:55,975][105620] Updated weights for policy 1, policy_version 119490 (0.0005) [2023-12-26 16:13:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 61046784. Throughput: 0: 9669.5, 1: 9896.8. Samples: 61053424. Policy #0 lag: (min: 28.0, avg: 52.6, max: 57.0) [2023-12-26 16:13:56,063][104569] Avg episode reward: [(0, '8364.237'), (1, '9353.869')] [2023-12-26 16:13:56,230][105692] Updated weights for policy 0, policy_version 118925 (0.0008) [2023-12-26 16:13:56,283][105692] Updated weights for policy 0, policy_version 118936 (0.0009) [2023-12-26 16:13:56,336][105692] Updated weights for policy 0, policy_version 118946 (0.0010) [2023-12-26 16:13:56,546][105620] Updated weights for policy 1, policy_version 119500 (0.0007) [2023-12-26 16:13:56,609][105620] Updated weights for policy 1, policy_version 119510 (0.0005) [2023-12-26 16:13:56,663][105620] Updated weights for policy 1, policy_version 119520 (0.0010) [2023-12-26 16:13:57,167][105692] Updated weights for policy 0, policy_version 118957 (0.0008) [2023-12-26 16:13:57,214][105692] Updated weights for policy 0, policy_version 118967 (0.0008) [2023-12-26 16:13:57,262][105692] Updated weights for policy 0, policy_version 118977 (0.0007) [2023-12-26 16:13:57,359][105620] Updated weights for policy 1, policy_version 119530 (0.0009) [2023-12-26 16:13:57,410][105620] Updated weights for policy 1, policy_version 119540 (0.0010) [2023-12-26 16:13:57,458][105620] Updated weights for policy 1, policy_version 119550 (0.0010) [2023-12-26 16:13:57,509][105620] Updated weights for policy 1, policy_version 119560 (0.0010) [2023-12-26 16:13:57,955][105692] Updated weights for policy 0, policy_version 118987 (0.0008) [2023-12-26 16:13:58,017][105692] Updated weights for policy 0, policy_version 118997 (0.0005) [2023-12-26 16:13:58,073][105692] Updated weights for policy 0, policy_version 119007 (0.0005) [2023-12-26 16:13:58,274][105620] Updated weights for policy 1, policy_version 119570 (0.0011) [2023-12-26 16:13:58,343][105620] Updated weights for policy 1, policy_version 119580 (0.0010) [2023-12-26 16:13:58,414][105620] Updated weights for policy 1, policy_version 119590 (0.0009) [2023-12-26 16:13:58,861][105692] Updated weights for policy 0, policy_version 119017 (0.0008) [2023-12-26 16:13:58,927][105692] Updated weights for policy 0, policy_version 119028 (0.0010) [2023-12-26 16:13:58,992][105692] Updated weights for policy 0, policy_version 119038 (0.0008) [2023-12-26 16:13:59,056][105692] Updated weights for policy 0, policy_version 119048 (0.0009) [2023-12-26 16:13:59,181][105620] Updated weights for policy 1, policy_version 119600 (0.0009) [2023-12-26 16:13:59,256][105620] Updated weights for policy 1, policy_version 119611 (0.0009) [2023-12-26 16:13:59,323][105620] Updated weights for policy 1, policy_version 119621 (0.0007) [2023-12-26 16:13:59,860][105692] Updated weights for policy 0, policy_version 119058 (0.0009) [2023-12-26 16:13:59,927][105692] Updated weights for policy 0, policy_version 119068 (0.0006) [2023-12-26 16:13:59,951][105620] Updated weights for policy 1, policy_version 119631 (0.0008) [2023-12-26 16:13:59,982][105692] Updated weights for policy 0, policy_version 119078 (0.0008) [2023-12-26 16:14:00,012][105620] Updated weights for policy 1, policy_version 119641 (0.0009) [2023-12-26 16:14:00,077][105620] Updated weights for policy 1, policy_version 119651 (0.0009) [2023-12-26 16:14:00,648][105692] Updated weights for policy 0, policy_version 119088 (0.0009) [2023-12-26 16:14:00,709][105692] Updated weights for policy 0, policy_version 119098 (0.0008) [2023-12-26 16:14:00,770][105692] Updated weights for policy 0, policy_version 119108 (0.0008) [2023-12-26 16:14:00,817][105620] Updated weights for policy 1, policy_version 119661 (0.0008) [2023-12-26 16:14:00,878][105620] Updated weights for policy 1, policy_version 119671 (0.0009) [2023-12-26 16:14:00,939][105620] Updated weights for policy 1, policy_version 119681 (0.0009) [2023-12-26 16:14:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 61145088. Throughput: 0: 9579.7, 1: 9892.0. Samples: 61111188. Policy #0 lag: (min: 28.0, avg: 52.6, max: 57.0) [2023-12-26 16:14:01,063][104569] Avg episode reward: [(0, '8805.139'), (1, '9271.383')] [2023-12-26 16:14:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000119112_30498816.pth... [2023-12-26 16:14:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000119688_30646272.pth... [2023-12-26 16:14:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000118504_30343168.pth [2023-12-26 16:14:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000118024_30220288.pth [2023-12-26 16:14:01,447][105692] Updated weights for policy 0, policy_version 119118 (0.0008) [2023-12-26 16:14:01,504][105692] Updated weights for policy 0, policy_version 119128 (0.0010) [2023-12-26 16:14:01,512][105585] KL-divergence is very high: 131.8774 [2023-12-26 16:14:01,562][105585] KL-divergence is very high: 120.7161 [2023-12-26 16:14:01,562][105692] Updated weights for policy 0, policy_version 119138 (0.0011) [2023-12-26 16:14:01,664][105620] Updated weights for policy 1, policy_version 119691 (0.0009) [2023-12-26 16:14:01,728][105620] Updated weights for policy 1, policy_version 119701 (0.0011) [2023-12-26 16:14:01,789][105620] Updated weights for policy 1, policy_version 119711 (0.0011) [2023-12-26 16:14:02,313][105692] Updated weights for policy 0, policy_version 119148 (0.0010) [2023-12-26 16:14:02,375][105692] Updated weights for policy 0, policy_version 119158 (0.0010) [2023-12-26 16:14:02,423][105620] Updated weights for policy 1, policy_version 119721 (0.0010) [2023-12-26 16:14:02,430][105692] Updated weights for policy 0, policy_version 119168 (0.0010) [2023-12-26 16:14:02,474][105620] Updated weights for policy 1, policy_version 119731 (0.0005) [2023-12-26 16:14:02,529][105620] Updated weights for policy 1, policy_version 119741 (0.0007) [2023-12-26 16:14:02,587][105620] Updated weights for policy 1, policy_version 119751 (0.0010) [2023-12-26 16:14:03,109][105692] Updated weights for policy 0, policy_version 119178 (0.0009) [2023-12-26 16:14:03,162][105692] Updated weights for policy 0, policy_version 119188 (0.0005) [2023-12-26 16:14:03,215][105692] Updated weights for policy 0, policy_version 119198 (0.0005) [2023-12-26 16:14:03,262][105692] Updated weights for policy 0, policy_version 119208 (0.0005) [2023-12-26 16:14:03,303][105620] Updated weights for policy 1, policy_version 119761 (0.0010) [2023-12-26 16:14:03,356][105620] Updated weights for policy 1, policy_version 119771 (0.0010) [2023-12-26 16:14:03,411][105620] Updated weights for policy 1, policy_version 119781 (0.0010) [2023-12-26 16:14:03,886][105692] Updated weights for policy 0, policy_version 119218 (0.0009) [2023-12-26 16:14:03,947][105692] Updated weights for policy 0, policy_version 119228 (0.0009) [2023-12-26 16:14:04,008][105692] Updated weights for policy 0, policy_version 119238 (0.0008) [2023-12-26 16:14:04,089][105620] Updated weights for policy 1, policy_version 119791 (0.0010) [2023-12-26 16:14:04,146][105620] Updated weights for policy 1, policy_version 119801 (0.0008) [2023-12-26 16:14:04,212][105620] Updated weights for policy 1, policy_version 119811 (0.0009) [2023-12-26 16:14:04,623][105692] Updated weights for policy 0, policy_version 119248 (0.0008) [2023-12-26 16:14:04,688][105692] Updated weights for policy 0, policy_version 119258 (0.0008) [2023-12-26 16:14:04,743][105692] Updated weights for policy 0, policy_version 119268 (0.0010) [2023-12-26 16:14:04,964][105620] Updated weights for policy 1, policy_version 119821 (0.0007) [2023-12-26 16:14:05,021][105620] Updated weights for policy 1, policy_version 119831 (0.0006) [2023-12-26 16:14:05,072][105620] Updated weights for policy 1, policy_version 119841 (0.0009) [2023-12-26 16:14:05,551][105692] Updated weights for policy 0, policy_version 119278 (0.0010) [2023-12-26 16:14:05,606][105692] Updated weights for policy 0, policy_version 119288 (0.0010) [2023-12-26 16:14:05,668][105692] Updated weights for policy 0, policy_version 119298 (0.0009) [2023-12-26 16:14:05,698][105620] Updated weights for policy 1, policy_version 119851 (0.0009) [2023-12-26 16:14:05,756][105620] Updated weights for policy 1, policy_version 119861 (0.0008) [2023-12-26 16:14:05,819][105620] Updated weights for policy 1, policy_version 119871 (0.0009) [2023-12-26 16:14:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 61243392. Throughput: 0: 9443.7, 1: 9979.0. Samples: 61230720. Policy #0 lag: (min: 28.0, avg: 52.6, max: 57.0) [2023-12-26 16:14:06,063][104569] Avg episode reward: [(0, '8899.416'), (1, '9182.019')] [2023-12-26 16:14:06,439][105692] Updated weights for policy 0, policy_version 119308 (0.0008) [2023-12-26 16:14:06,509][105692] Updated weights for policy 0, policy_version 119318 (0.0009) [2023-12-26 16:14:06,557][105620] Updated weights for policy 1, policy_version 119881 (0.0008) [2023-12-26 16:14:06,568][105692] Updated weights for policy 0, policy_version 119328 (0.0009) [2023-12-26 16:14:06,619][105620] Updated weights for policy 1, policy_version 119891 (0.0005) [2023-12-26 16:14:06,689][105620] Updated weights for policy 1, policy_version 119901 (0.0006) [2023-12-26 16:14:06,747][105620] Updated weights for policy 1, policy_version 119911 (0.0007) [2023-12-26 16:14:07,337][105692] Updated weights for policy 0, policy_version 119338 (0.0008) [2023-12-26 16:14:07,339][105620] Updated weights for policy 1, policy_version 119921 (0.0008) [2023-12-26 16:14:07,386][105692] Updated weights for policy 0, policy_version 119348 (0.0007) [2023-12-26 16:14:07,403][105620] Updated weights for policy 1, policy_version 119931 (0.0008) [2023-12-26 16:14:07,446][105692] Updated weights for policy 0, policy_version 119358 (0.0008) [2023-12-26 16:14:07,461][105620] Updated weights for policy 1, policy_version 119941 (0.0006) [2023-12-26 16:14:07,498][105692] Updated weights for policy 0, policy_version 119368 (0.0008) [2023-12-26 16:14:08,081][105620] Updated weights for policy 1, policy_version 119951 (0.0008) [2023-12-26 16:14:08,133][105620] Updated weights for policy 1, policy_version 119961 (0.0010) [2023-12-26 16:14:08,194][105620] Updated weights for policy 1, policy_version 119971 (0.0008) [2023-12-26 16:14:08,250][105692] Updated weights for policy 0, policy_version 119378 (0.0010) [2023-12-26 16:14:08,305][105692] Updated weights for policy 0, policy_version 119388 (0.0010) [2023-12-26 16:14:08,373][105692] Updated weights for policy 0, policy_version 119398 (0.0009) [2023-12-26 16:14:08,980][105620] Updated weights for policy 1, policy_version 119981 (0.0008) [2023-12-26 16:14:09,040][105620] Updated weights for policy 1, policy_version 119991 (0.0008) [2023-12-26 16:14:09,101][105620] Updated weights for policy 1, policy_version 120001 (0.0008) [2023-12-26 16:14:09,106][105692] Updated weights for policy 0, policy_version 119408 (0.0010) [2023-12-26 16:14:09,154][105692] Updated weights for policy 0, policy_version 119418 (0.0010) [2023-12-26 16:14:09,206][105692] Updated weights for policy 0, policy_version 119428 (0.0010) [2023-12-26 16:14:09,883][105692] Updated weights for policy 0, policy_version 119438 (0.0009) [2023-12-26 16:14:09,899][105620] Updated weights for policy 1, policy_version 120011 (0.0008) [2023-12-26 16:14:09,946][105692] Updated weights for policy 0, policy_version 119448 (0.0009) [2023-12-26 16:14:09,968][105620] Updated weights for policy 1, policy_version 120021 (0.0008) [2023-12-26 16:14:09,995][105692] Updated weights for policy 0, policy_version 119458 (0.0006) [2023-12-26 16:14:10,031][105620] Updated weights for policy 1, policy_version 120031 (0.0008) [2023-12-26 16:14:10,684][105692] Updated weights for policy 0, policy_version 119468 (0.0007) [2023-12-26 16:14:10,733][105692] Updated weights for policy 0, policy_version 119478 (0.0009) [2023-12-26 16:14:10,790][105692] Updated weights for policy 0, policy_version 119488 (0.0010) [2023-12-26 16:14:10,808][105620] Updated weights for policy 1, policy_version 120041 (0.0009) [2023-12-26 16:14:10,869][105620] Updated weights for policy 1, policy_version 120051 (0.0006) [2023-12-26 16:14:10,928][105620] Updated weights for policy 1, policy_version 120061 (0.0006) [2023-12-26 16:14:10,985][105620] Updated weights for policy 1, policy_version 120071 (0.0007) [2023-12-26 16:14:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 61341696. Throughput: 0: 9438.1, 1: 10008.0. Samples: 61345444. Policy #0 lag: (min: 28.0, avg: 52.6, max: 57.0) [2023-12-26 16:14:11,063][104569] Avg episode reward: [(0, '8898.463'), (1, '9264.170')] [2023-12-26 16:14:11,655][105620] Updated weights for policy 1, policy_version 120081 (0.0009) [2023-12-26 16:14:11,683][105692] Updated weights for policy 0, policy_version 119498 (0.0007) [2023-12-26 16:14:11,718][105620] Updated weights for policy 1, policy_version 120091 (0.0008) [2023-12-26 16:14:11,749][105692] Updated weights for policy 0, policy_version 119508 (0.0008) [2023-12-26 16:14:11,780][105620] Updated weights for policy 1, policy_version 120101 (0.0007) [2023-12-26 16:14:11,809][105692] Updated weights for policy 0, policy_version 119518 (0.0008) [2023-12-26 16:14:11,861][105692] Updated weights for policy 0, policy_version 119528 (0.0007) [2023-12-26 16:14:12,565][105692] Updated weights for policy 0, policy_version 119538 (0.0006) [2023-12-26 16:14:12,609][105620] Updated weights for policy 1, policy_version 120111 (0.0007) [2023-12-26 16:14:12,619][105692] Updated weights for policy 0, policy_version 119548 (0.0006) [2023-12-26 16:14:12,666][105620] Updated weights for policy 1, policy_version 120121 (0.0008) [2023-12-26 16:14:12,673][105692] Updated weights for policy 0, policy_version 119558 (0.0007) [2023-12-26 16:14:12,726][105620] Updated weights for policy 1, policy_version 120131 (0.0009) [2023-12-26 16:14:13,348][105692] Updated weights for policy 0, policy_version 119568 (0.0008) [2023-12-26 16:14:13,402][105692] Updated weights for policy 0, policy_version 119578 (0.0009) [2023-12-26 16:14:13,458][105692] Updated weights for policy 0, policy_version 119588 (0.0008) [2023-12-26 16:14:13,521][105620] Updated weights for policy 1, policy_version 120141 (0.0009) [2023-12-26 16:14:13,572][105620] Updated weights for policy 1, policy_version 120151 (0.0009) [2023-12-26 16:14:13,631][105620] Updated weights for policy 1, policy_version 120161 (0.0009) [2023-12-26 16:14:14,209][105692] Updated weights for policy 0, policy_version 119598 (0.0009) [2023-12-26 16:14:14,275][105692] Updated weights for policy 0, policy_version 119608 (0.0010) [2023-12-26 16:14:14,298][105620] Updated weights for policy 1, policy_version 120171 (0.0008) [2023-12-26 16:14:14,324][105692] Updated weights for policy 0, policy_version 119618 (0.0009) [2023-12-26 16:14:14,354][105620] Updated weights for policy 1, policy_version 120181 (0.0005) [2023-12-26 16:14:14,410][105620] Updated weights for policy 1, policy_version 120191 (0.0006) [2023-12-26 16:14:14,968][105692] Updated weights for policy 0, policy_version 119628 (0.0007) [2023-12-26 16:14:14,994][105620] Updated weights for policy 1, policy_version 120201 (0.0009) [2023-12-26 16:14:15,030][105692] Updated weights for policy 0, policy_version 119638 (0.0007) [2023-12-26 16:14:15,066][105620] Updated weights for policy 1, policy_version 120211 (0.0008) [2023-12-26 16:14:15,076][105692] Updated weights for policy 0, policy_version 119648 (0.0008) [2023-12-26 16:14:15,128][105620] Updated weights for policy 1, policy_version 120221 (0.0006) [2023-12-26 16:14:15,188][105620] Updated weights for policy 1, policy_version 120231 (0.0005) [2023-12-26 16:14:15,752][105692] Updated weights for policy 0, policy_version 119658 (0.0008) [2023-12-26 16:14:15,813][105692] Updated weights for policy 0, policy_version 119668 (0.0009) [2023-12-26 16:14:15,868][105692] Updated weights for policy 0, policy_version 119678 (0.0009) [2023-12-26 16:14:15,907][105620] Updated weights for policy 1, policy_version 120241 (0.0005) [2023-12-26 16:14:15,920][105692] Updated weights for policy 0, policy_version 119688 (0.0009) [2023-12-26 16:14:15,970][105620] Updated weights for policy 1, policy_version 120251 (0.0007) [2023-12-26 16:14:16,029][105620] Updated weights for policy 1, policy_version 120261 (0.0008) [2023-12-26 16:14:16,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 61440000. Throughput: 0: 9411.2, 1: 9924.9. Samples: 61400896. Policy #0 lag: (min: 28.0, avg: 52.6, max: 57.0) [2023-12-26 16:14:16,062][104569] Avg episode reward: [(0, '8892.066'), (1, '9083.479')] [2023-12-26 16:14:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000119688_30646272.pth... [2023-12-26 16:14:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000120264_30793728.pth... [2023-12-26 16:14:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000118568_30359552.pth [2023-12-26 16:14:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000119112_30498816.pth [2023-12-26 16:14:16,657][105620] Updated weights for policy 1, policy_version 120271 (0.0009) [2023-12-26 16:14:16,714][105620] Updated weights for policy 1, policy_version 120281 (0.0007) [2023-12-26 16:14:16,728][105692] Updated weights for policy 0, policy_version 119698 (0.0006) [2023-12-26 16:14:16,773][105620] Updated weights for policy 1, policy_version 120291 (0.0005) [2023-12-26 16:14:16,798][105692] Updated weights for policy 0, policy_version 119708 (0.0005) [2023-12-26 16:14:16,863][105692] Updated weights for policy 0, policy_version 119718 (0.0005) [2023-12-26 16:14:17,368][105692] Updated weights for policy 0, policy_version 119728 (0.0005) [2023-12-26 16:14:17,417][105692] Updated weights for policy 0, policy_version 119738 (0.0005) [2023-12-26 16:14:17,447][105620] Updated weights for policy 1, policy_version 120301 (0.0007) [2023-12-26 16:14:17,466][105692] Updated weights for policy 0, policy_version 119748 (0.0006) [2023-12-26 16:14:17,503][105620] Updated weights for policy 1, policy_version 120311 (0.0009) [2023-12-26 16:14:17,561][105620] Updated weights for policy 1, policy_version 120321 (0.0010) [2023-12-26 16:14:18,039][105692] Updated weights for policy 0, policy_version 119758 (0.0005) [2023-12-26 16:14:18,085][105692] Updated weights for policy 0, policy_version 119768 (0.0005) [2023-12-26 16:14:18,131][105692] Updated weights for policy 0, policy_version 119778 (0.0005) [2023-12-26 16:14:18,422][105620] Updated weights for policy 1, policy_version 120331 (0.0010) [2023-12-26 16:14:18,480][105620] Updated weights for policy 1, policy_version 120341 (0.0009) [2023-12-26 16:14:18,535][105620] Updated weights for policy 1, policy_version 120351 (0.0010) [2023-12-26 16:14:18,798][105692] Updated weights for policy 0, policy_version 119788 (0.0007) [2023-12-26 16:14:18,850][105692] Updated weights for policy 0, policy_version 119798 (0.0010) [2023-12-26 16:14:18,909][105692] Updated weights for policy 0, policy_version 119808 (0.0009) [2023-12-26 16:14:19,180][105620] Updated weights for policy 1, policy_version 120361 (0.0006) [2023-12-26 16:14:19,252][105620] Updated weights for policy 1, policy_version 120371 (0.0006) [2023-12-26 16:14:19,318][105620] Updated weights for policy 1, policy_version 120381 (0.0008) [2023-12-26 16:14:19,384][105620] Updated weights for policy 1, policy_version 120391 (0.0009) [2023-12-26 16:14:19,709][105692] Updated weights for policy 0, policy_version 119818 (0.0008) [2023-12-26 16:14:19,771][105692] Updated weights for policy 0, policy_version 119828 (0.0006) [2023-12-26 16:14:19,829][105692] Updated weights for policy 0, policy_version 119838 (0.0009) [2023-12-26 16:14:19,888][105692] Updated weights for policy 0, policy_version 119848 (0.0009) [2023-12-26 16:14:20,194][105620] Updated weights for policy 1, policy_version 120401 (0.0009) [2023-12-26 16:14:20,252][105620] Updated weights for policy 1, policy_version 120411 (0.0009) [2023-12-26 16:14:20,315][105620] Updated weights for policy 1, policy_version 120421 (0.0009) [2023-12-26 16:14:20,620][105692] Updated weights for policy 0, policy_version 119858 (0.0010) [2023-12-26 16:14:20,678][105692] Updated weights for policy 0, policy_version 119868 (0.0009) [2023-12-26 16:14:20,736][105692] Updated weights for policy 0, policy_version 119878 (0.0009) [2023-12-26 16:14:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 61530112. Throughput: 0: 9521.1, 1: 9933.2. Samples: 61522336. Policy #0 lag: (min: 28.0, avg: 52.6, max: 57.0) [2023-12-26 16:14:21,063][104569] Avg episode reward: [(0, '8989.215'), (1, '8908.820')] [2023-12-26 16:14:21,102][105620] Updated weights for policy 1, policy_version 120431 (0.0008) [2023-12-26 16:14:21,173][105620] Updated weights for policy 1, policy_version 120441 (0.0009) [2023-12-26 16:14:21,243][105620] Updated weights for policy 1, policy_version 120451 (0.0009) [2023-12-26 16:14:21,506][105692] Updated weights for policy 0, policy_version 119888 (0.0009) [2023-12-26 16:14:21,556][105692] Updated weights for policy 0, policy_version 119898 (0.0008) [2023-12-26 16:14:21,620][105692] Updated weights for policy 0, policy_version 119908 (0.0010) [2023-12-26 16:14:21,973][105620] Updated weights for policy 1, policy_version 120461 (0.0010) [2023-12-26 16:14:22,039][105620] Updated weights for policy 1, policy_version 120471 (0.0010) [2023-12-26 16:14:22,099][105620] Updated weights for policy 1, policy_version 120481 (0.0009) [2023-12-26 16:14:22,429][105692] Updated weights for policy 0, policy_version 119918 (0.0008) [2023-12-26 16:14:22,484][105692] Updated weights for policy 0, policy_version 119928 (0.0009) [2023-12-26 16:14:22,543][105692] Updated weights for policy 0, policy_version 119938 (0.0009) [2023-12-26 16:14:22,881][105620] Updated weights for policy 1, policy_version 120491 (0.0009) [2023-12-26 16:14:22,932][105620] Updated weights for policy 1, policy_version 120501 (0.0008) [2023-12-26 16:14:22,991][105620] Updated weights for policy 1, policy_version 120511 (0.0009) [2023-12-26 16:14:23,254][105692] Updated weights for policy 0, policy_version 119948 (0.0007) [2023-12-26 16:14:23,317][105692] Updated weights for policy 0, policy_version 119958 (0.0005) [2023-12-26 16:14:23,369][105692] Updated weights for policy 0, policy_version 119968 (0.0005) [2023-12-26 16:14:23,766][105620] Updated weights for policy 1, policy_version 120521 (0.0008) [2023-12-26 16:14:23,830][105620] Updated weights for policy 1, policy_version 120531 (0.0006) [2023-12-26 16:14:23,892][105620] Updated weights for policy 1, policy_version 120541 (0.0006) [2023-12-26 16:14:23,915][105692] Updated weights for policy 0, policy_version 119978 (0.0005) [2023-12-26 16:14:23,950][105620] Updated weights for policy 1, policy_version 120551 (0.0007) [2023-12-26 16:14:23,983][105692] Updated weights for policy 0, policy_version 119988 (0.0006) [2023-12-26 16:14:24,044][105692] Updated weights for policy 0, policy_version 119998 (0.0010) [2023-12-26 16:14:24,102][105692] Updated weights for policy 0, policy_version 120008 (0.0010) [2023-12-26 16:14:24,473][105620] Updated weights for policy 1, policy_version 120561 (0.0006) [2023-12-26 16:14:24,537][105620] Updated weights for policy 1, policy_version 120571 (0.0005) [2023-12-26 16:14:24,602][105620] Updated weights for policy 1, policy_version 120581 (0.0005) [2023-12-26 16:14:24,692][105692] Updated weights for policy 0, policy_version 120018 (0.0005) [2023-12-26 16:14:24,747][105692] Updated weights for policy 0, policy_version 120028 (0.0009) [2023-12-26 16:14:24,812][105692] Updated weights for policy 0, policy_version 120038 (0.0010) [2023-12-26 16:14:25,108][105620] Updated weights for policy 1, policy_version 120591 (0.0008) [2023-12-26 16:14:25,163][105620] Updated weights for policy 1, policy_version 120601 (0.0008) [2023-12-26 16:14:25,220][105620] Updated weights for policy 1, policy_version 120611 (0.0008) [2023-12-26 16:14:25,486][105692] Updated weights for policy 0, policy_version 120048 (0.0006) [2023-12-26 16:14:25,544][105692] Updated weights for policy 0, policy_version 120058 (0.0010) [2023-12-26 16:14:25,602][105692] Updated weights for policy 0, policy_version 120068 (0.0010) [2023-12-26 16:14:25,873][105620] Updated weights for policy 1, policy_version 120621 (0.0007) [2023-12-26 16:14:25,928][105620] Updated weights for policy 1, policy_version 120631 (0.0008) [2023-12-26 16:14:25,982][105620] Updated weights for policy 1, policy_version 120641 (0.0008) [2023-12-26 16:14:26,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.1, 300 sec: 19521.9). Total num frames: 61636608. Throughput: 0: 9626.3, 1: 9907.9. Samples: 61642636. Policy #0 lag: (min: 28.0, avg: 52.6, max: 57.0) [2023-12-26 16:14:26,063][104569] Avg episode reward: [(0, '8906.935'), (1, '9087.463')] [2023-12-26 16:14:26,331][105692] Updated weights for policy 0, policy_version 120078 (0.0010) [2023-12-26 16:14:26,386][105692] Updated weights for policy 0, policy_version 120088 (0.0010) [2023-12-26 16:14:26,451][105692] Updated weights for policy 0, policy_version 120098 (0.0010) [2023-12-26 16:14:26,754][105620] Updated weights for policy 1, policy_version 120651 (0.0009) [2023-12-26 16:14:26,813][105620] Updated weights for policy 1, policy_version 120661 (0.0010) [2023-12-26 16:14:26,867][105620] Updated weights for policy 1, policy_version 120671 (0.0010) [2023-12-26 16:14:27,104][105692] Updated weights for policy 0, policy_version 120108 (0.0009) [2023-12-26 16:14:27,170][105692] Updated weights for policy 0, policy_version 120118 (0.0005) [2023-12-26 16:14:27,218][105692] Updated weights for policy 0, policy_version 120128 (0.0005) [2023-12-26 16:14:27,523][105620] Updated weights for policy 1, policy_version 120681 (0.0010) [2023-12-26 16:14:27,573][105620] Updated weights for policy 1, policy_version 120691 (0.0005) [2023-12-26 16:14:27,625][105620] Updated weights for policy 1, policy_version 120701 (0.0005) [2023-12-26 16:14:27,673][105620] Updated weights for policy 1, policy_version 120711 (0.0005) [2023-12-26 16:14:27,830][105692] Updated weights for policy 0, policy_version 120138 (0.0005) [2023-12-26 16:14:27,894][105692] Updated weights for policy 0, policy_version 120148 (0.0005) [2023-12-26 16:14:27,938][105692] Updated weights for policy 0, policy_version 120158 (0.0005) [2023-12-26 16:14:27,988][105692] Updated weights for policy 0, policy_version 120168 (0.0005) [2023-12-26 16:14:28,271][105620] Updated weights for policy 1, policy_version 120721 (0.0007) [2023-12-26 16:14:28,339][105620] Updated weights for policy 1, policy_version 120731 (0.0007) [2023-12-26 16:14:28,401][105620] Updated weights for policy 1, policy_version 120741 (0.0008) [2023-12-26 16:14:28,641][105692] Updated weights for policy 0, policy_version 120178 (0.0010) [2023-12-26 16:14:28,689][105692] Updated weights for policy 0, policy_version 120188 (0.0009) [2023-12-26 16:14:28,736][105692] Updated weights for policy 0, policy_version 120198 (0.0009) [2023-12-26 16:14:29,046][105620] Updated weights for policy 1, policy_version 120751 (0.0009) [2023-12-26 16:14:29,111][105620] Updated weights for policy 1, policy_version 120761 (0.0009) [2023-12-26 16:14:29,163][105620] Updated weights for policy 1, policy_version 120771 (0.0005) [2023-12-26 16:14:29,480][105692] Updated weights for policy 0, policy_version 120208 (0.0008) [2023-12-26 16:14:29,547][105692] Updated weights for policy 0, policy_version 120218 (0.0010) [2023-12-26 16:14:29,618][105692] Updated weights for policy 0, policy_version 120228 (0.0010) [2023-12-26 16:14:29,881][105620] Updated weights for policy 1, policy_version 120781 (0.0007) [2023-12-26 16:14:29,946][105620] Updated weights for policy 1, policy_version 120791 (0.0008) [2023-12-26 16:14:30,013][105620] Updated weights for policy 1, policy_version 120801 (0.0009) [2023-12-26 16:14:30,302][105692] Updated weights for policy 0, policy_version 120238 (0.0008) [2023-12-26 16:14:30,357][105692] Updated weights for policy 0, policy_version 120248 (0.0006) [2023-12-26 16:14:30,411][105692] Updated weights for policy 0, policy_version 120258 (0.0006) [2023-12-26 16:14:30,872][105620] Updated weights for policy 1, policy_version 120811 (0.0010) [2023-12-26 16:14:30,928][105620] Updated weights for policy 1, policy_version 120821 (0.0008) [2023-12-26 16:14:30,975][105620] Updated weights for policy 1, policy_version 120831 (0.0009) [2023-12-26 16:14:30,988][105692] Updated weights for policy 0, policy_version 120268 (0.0005) [2023-12-26 16:14:31,046][105692] Updated weights for policy 0, policy_version 120278 (0.0007) [2023-12-26 16:14:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 61734912. Throughput: 0: 9712.8, 1: 9946.7. Samples: 61705216. Policy #0 lag: (min: 28.0, avg: 52.6, max: 57.0) [2023-12-26 16:14:31,062][104569] Avg episode reward: [(0, '8815.280'), (1, '9092.400')] [2023-12-26 16:14:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000120840_30941184.pth... [2023-12-26 16:14:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000119688_30646272.pth [2023-12-26 16:14:31,107][105692] Updated weights for policy 0, policy_version 120288 (0.0009) [2023-12-26 16:14:31,160][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000120296_30801920.pth... [2023-12-26 16:14:31,164][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000119112_30498816.pth [2023-12-26 16:14:31,799][105620] Updated weights for policy 1, policy_version 120841 (0.0009) [2023-12-26 16:14:31,815][105692] Updated weights for policy 0, policy_version 120298 (0.0008) [2023-12-26 16:14:31,861][105620] Updated weights for policy 1, policy_version 120851 (0.0007) [2023-12-26 16:14:31,865][105692] Updated weights for policy 0, policy_version 120308 (0.0008) [2023-12-26 16:14:31,920][105620] Updated weights for policy 1, policy_version 120861 (0.0007) [2023-12-26 16:14:31,921][105692] Updated weights for policy 0, policy_version 120318 (0.0008) [2023-12-26 16:14:31,981][105620] Updated weights for policy 1, policy_version 120871 (0.0008) [2023-12-26 16:14:31,981][105692] Updated weights for policy 0, policy_version 120328 (0.0008) [2023-12-26 16:14:32,672][105620] Updated weights for policy 1, policy_version 120881 (0.0008) [2023-12-26 16:14:32,708][105692] Updated weights for policy 0, policy_version 120338 (0.0010) [2023-12-26 16:14:32,727][105620] Updated weights for policy 1, policy_version 120891 (0.0008) [2023-12-26 16:14:32,776][105692] Updated weights for policy 0, policy_version 120348 (0.0010) [2023-12-26 16:14:32,777][105620] Updated weights for policy 1, policy_version 120901 (0.0007) [2023-12-26 16:14:32,838][105692] Updated weights for policy 0, policy_version 120358 (0.0010) [2023-12-26 16:14:33,427][105692] Updated weights for policy 0, policy_version 120368 (0.0006) [2023-12-26 16:14:33,472][105692] Updated weights for policy 0, policy_version 120378 (0.0010) [2023-12-26 16:14:33,518][105692] Updated weights for policy 0, policy_version 120388 (0.0010) [2023-12-26 16:14:33,549][105620] Updated weights for policy 1, policy_version 120911 (0.0006) [2023-12-26 16:14:33,599][105620] Updated weights for policy 1, policy_version 120921 (0.0005) [2023-12-26 16:14:33,650][105620] Updated weights for policy 1, policy_version 120931 (0.0005) [2023-12-26 16:14:34,055][105692] Updated weights for policy 0, policy_version 120398 (0.0007) [2023-12-26 16:14:34,102][105692] Updated weights for policy 0, policy_version 120408 (0.0010) [2023-12-26 16:14:34,161][105692] Updated weights for policy 0, policy_version 120418 (0.0011) [2023-12-26 16:14:34,449][105620] Updated weights for policy 1, policy_version 120941 (0.0010) [2023-12-26 16:14:34,506][105620] Updated weights for policy 1, policy_version 120951 (0.0008) [2023-12-26 16:14:34,569][105620] Updated weights for policy 1, policy_version 120961 (0.0009) [2023-12-26 16:14:34,909][105692] Updated weights for policy 0, policy_version 120428 (0.0009) [2023-12-26 16:14:34,967][105692] Updated weights for policy 0, policy_version 120438 (0.0010) [2023-12-26 16:14:35,018][105692] Updated weights for policy 0, policy_version 120448 (0.0010) [2023-12-26 16:14:35,330][105620] Updated weights for policy 1, policy_version 120971 (0.0008) [2023-12-26 16:14:35,392][105620] Updated weights for policy 1, policy_version 120981 (0.0008) [2023-12-26 16:14:35,447][105620] Updated weights for policy 1, policy_version 120991 (0.0008) [2023-12-26 16:14:35,765][105692] Updated weights for policy 0, policy_version 120458 (0.0010) [2023-12-26 16:14:35,819][105692] Updated weights for policy 0, policy_version 120468 (0.0010) [2023-12-26 16:14:35,876][105692] Updated weights for policy 0, policy_version 120478 (0.0010) [2023-12-26 16:14:35,933][105692] Updated weights for policy 0, policy_version 120488 (0.0010) [2023-12-26 16:14:36,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 61833216. Throughput: 0: 9924.5, 1: 9752.3. Samples: 61822472. Policy #0 lag: (min: 28.0, avg: 52.6, max: 57.0) [2023-12-26 16:14:36,063][104569] Avg episode reward: [(0, '8989.878'), (1, '8831.535')] [2023-12-26 16:14:36,221][105620] Updated weights for policy 1, policy_version 121001 (0.0008) [2023-12-26 16:14:36,291][105620] Updated weights for policy 1, policy_version 121011 (0.0007) [2023-12-26 16:14:36,356][105620] Updated weights for policy 1, policy_version 121021 (0.0008) [2023-12-26 16:14:36,424][105620] Updated weights for policy 1, policy_version 121031 (0.0008) [2023-12-26 16:14:36,622][105692] Updated weights for policy 0, policy_version 120498 (0.0011) [2023-12-26 16:14:36,685][105692] Updated weights for policy 0, policy_version 120508 (0.0011) [2023-12-26 16:14:36,740][105692] Updated weights for policy 0, policy_version 120518 (0.0010) [2023-12-26 16:14:37,142][105620] Updated weights for policy 1, policy_version 121041 (0.0008) [2023-12-26 16:14:37,207][105620] Updated weights for policy 1, policy_version 121051 (0.0008) [2023-12-26 16:14:37,272][105620] Updated weights for policy 1, policy_version 121061 (0.0007) [2023-12-26 16:14:37,492][105692] Updated weights for policy 0, policy_version 120528 (0.0010) [2023-12-26 16:14:37,541][105692] Updated weights for policy 0, policy_version 120538 (0.0010) [2023-12-26 16:14:37,604][105692] Updated weights for policy 0, policy_version 120548 (0.0010) [2023-12-26 16:14:38,023][105620] Updated weights for policy 1, policy_version 121071 (0.0008) [2023-12-26 16:14:38,082][105620] Updated weights for policy 1, policy_version 121081 (0.0008) [2023-12-26 16:14:38,144][105620] Updated weights for policy 1, policy_version 121091 (0.0008) [2023-12-26 16:14:38,354][105692] Updated weights for policy 0, policy_version 120558 (0.0009) [2023-12-26 16:14:38,408][105692] Updated weights for policy 0, policy_version 120568 (0.0006) [2023-12-26 16:14:38,462][105692] Updated weights for policy 0, policy_version 120578 (0.0010) [2023-12-26 16:14:38,872][105620] Updated weights for policy 1, policy_version 121101 (0.0008) [2023-12-26 16:14:38,938][105620] Updated weights for policy 1, policy_version 121111 (0.0008) [2023-12-26 16:14:39,004][105620] Updated weights for policy 1, policy_version 121121 (0.0008) [2023-12-26 16:14:39,197][105692] Updated weights for policy 0, policy_version 120588 (0.0010) [2023-12-26 16:14:39,265][105692] Updated weights for policy 0, policy_version 120598 (0.0009) [2023-12-26 16:14:39,331][105692] Updated weights for policy 0, policy_version 120608 (0.0012) [2023-12-26 16:14:39,718][105620] Updated weights for policy 1, policy_version 121131 (0.0007) [2023-12-26 16:14:39,779][105620] Updated weights for policy 1, policy_version 121141 (0.0005) [2023-12-26 16:14:39,850][105620] Updated weights for policy 1, policy_version 121151 (0.0008) [2023-12-26 16:14:40,093][105692] Updated weights for policy 0, policy_version 120618 (0.0008) [2023-12-26 16:14:40,157][105692] Updated weights for policy 0, policy_version 120628 (0.0009) [2023-12-26 16:14:40,209][105692] Updated weights for policy 0, policy_version 120638 (0.0008) [2023-12-26 16:14:40,262][105692] Updated weights for policy 0, policy_version 120648 (0.0007) [2023-12-26 16:14:40,523][105620] Updated weights for policy 1, policy_version 121161 (0.0008) [2023-12-26 16:14:40,588][105620] Updated weights for policy 1, policy_version 121171 (0.0010) [2023-12-26 16:14:40,660][105620] Updated weights for policy 1, policy_version 121181 (0.0008) [2023-12-26 16:14:40,727][105620] Updated weights for policy 1, policy_version 121191 (0.0006) [2023-12-26 16:14:40,970][105692] Updated weights for policy 0, policy_version 120658 (0.0009) [2023-12-26 16:14:41,034][105692] Updated weights for policy 0, policy_version 120668 (0.0009) [2023-12-26 16:14:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 61923328. Throughput: 0: 9892.0, 1: 9704.1. Samples: 61935248. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-12-26 16:14:41,063][104569] Avg episode reward: [(0, '8985.129'), (1, '9013.528')] [2023-12-26 16:14:41,101][105692] Updated weights for policy 0, policy_version 120678 (0.0006) [2023-12-26 16:14:41,429][105620] Updated weights for policy 1, policy_version 121201 (0.0009) [2023-12-26 16:14:41,484][105620] Updated weights for policy 1, policy_version 121211 (0.0009) [2023-12-26 16:14:41,533][105620] Updated weights for policy 1, policy_version 121221 (0.0009) [2023-12-26 16:14:41,807][105692] Updated weights for policy 0, policy_version 120688 (0.0009) [2023-12-26 16:14:41,857][105692] Updated weights for policy 0, policy_version 120698 (0.0009) [2023-12-26 16:14:41,907][105692] Updated weights for policy 0, policy_version 120708 (0.0009) [2023-12-26 16:14:42,437][105620] Updated weights for policy 1, policy_version 121231 (0.0009) [2023-12-26 16:14:42,514][105620] Updated weights for policy 1, policy_version 121241 (0.0009) [2023-12-26 16:14:42,576][105620] Updated weights for policy 1, policy_version 121251 (0.0009) [2023-12-26 16:14:42,807][105692] Updated weights for policy 0, policy_version 120718 (0.0008) [2023-12-26 16:14:42,863][105692] Updated weights for policy 0, policy_version 120728 (0.0008) [2023-12-26 16:14:42,928][105692] Updated weights for policy 0, policy_version 120738 (0.0009) [2023-12-26 16:14:43,260][105620] Updated weights for policy 1, policy_version 121261 (0.0007) [2023-12-26 16:14:43,322][105620] Updated weights for policy 1, policy_version 121271 (0.0006) [2023-12-26 16:14:43,381][105620] Updated weights for policy 1, policy_version 121281 (0.0009) [2023-12-26 16:14:43,584][105692] Updated weights for policy 0, policy_version 120748 (0.0008) [2023-12-26 16:14:43,627][105692] Updated weights for policy 0, policy_version 120758 (0.0005) [2023-12-26 16:14:43,673][105692] Updated weights for policy 0, policy_version 120768 (0.0006) [2023-12-26 16:14:44,157][105620] Updated weights for policy 1, policy_version 121291 (0.0009) [2023-12-26 16:14:44,215][105620] Updated weights for policy 1, policy_version 121301 (0.0009) [2023-12-26 16:14:44,281][105620] Updated weights for policy 1, policy_version 121311 (0.0010) [2023-12-26 16:14:44,348][105692] Updated weights for policy 0, policy_version 120778 (0.0006) [2023-12-26 16:14:44,415][105692] Updated weights for policy 0, policy_version 120788 (0.0006) [2023-12-26 16:14:44,475][105692] Updated weights for policy 0, policy_version 120798 (0.0005) [2023-12-26 16:14:44,535][105692] Updated weights for policy 0, policy_version 120808 (0.0006) [2023-12-26 16:14:44,962][105620] Updated weights for policy 1, policy_version 121321 (0.0009) [2023-12-26 16:14:45,030][105620] Updated weights for policy 1, policy_version 121331 (0.0008) [2023-12-26 16:14:45,095][105620] Updated weights for policy 1, policy_version 121341 (0.0008) [2023-12-26 16:14:45,156][105620] Updated weights for policy 1, policy_version 121351 (0.0008) [2023-12-26 16:14:45,262][105692] Updated weights for policy 0, policy_version 120818 (0.0009) [2023-12-26 16:14:45,321][105692] Updated weights for policy 0, policy_version 120828 (0.0010) [2023-12-26 16:14:45,377][105692] Updated weights for policy 0, policy_version 120838 (0.0009) [2023-12-26 16:14:45,841][105620] Updated weights for policy 1, policy_version 121361 (0.0009) [2023-12-26 16:14:45,893][105620] Updated weights for policy 1, policy_version 121371 (0.0008) [2023-12-26 16:14:45,945][105620] Updated weights for policy 1, policy_version 121381 (0.0006) [2023-12-26 16:14:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 62021632. Throughput: 0: 9892.7, 1: 9643.3. Samples: 61990308. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-12-26 16:14:46,063][104569] Avg episode reward: [(0, '8892.483'), (1, '9177.211')] [2023-12-26 16:14:46,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000121384_31080448.pth... [2023-12-26 16:14:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000120264_30793728.pth [2023-12-26 16:14:46,097][105692] Updated weights for policy 0, policy_version 120848 (0.0006) [2023-12-26 16:14:46,160][105692] Updated weights for policy 0, policy_version 120858 (0.0007) [2023-12-26 16:14:46,212][105692] Updated weights for policy 0, policy_version 120868 (0.0010) [2023-12-26 16:14:46,231][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000120872_30949376.pth... [2023-12-26 16:14:46,235][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000119688_30646272.pth [2023-12-26 16:14:46,518][105620] Updated weights for policy 1, policy_version 121391 (0.0005) [2023-12-26 16:14:46,565][105620] Updated weights for policy 1, policy_version 121401 (0.0007) [2023-12-26 16:14:46,618][105620] Updated weights for policy 1, policy_version 121411 (0.0010) [2023-12-26 16:14:46,758][105692] Updated weights for policy 0, policy_version 120878 (0.0007) [2023-12-26 16:14:46,812][105692] Updated weights for policy 0, policy_version 120888 (0.0006) [2023-12-26 16:14:46,877][105692] Updated weights for policy 0, policy_version 120898 (0.0010) [2023-12-26 16:14:47,339][105620] Updated weights for policy 1, policy_version 121421 (0.0010) [2023-12-26 16:14:47,394][105620] Updated weights for policy 1, policy_version 121431 (0.0010) [2023-12-26 16:14:47,442][105620] Updated weights for policy 1, policy_version 121441 (0.0010) [2023-12-26 16:14:47,477][105692] Updated weights for policy 0, policy_version 120908 (0.0010) [2023-12-26 16:14:47,535][105692] Updated weights for policy 0, policy_version 120918 (0.0010) [2023-12-26 16:14:47,594][105692] Updated weights for policy 0, policy_version 120928 (0.0010) [2023-12-26 16:14:48,194][105620] Updated weights for policy 1, policy_version 121451 (0.0010) [2023-12-26 16:14:48,255][105620] Updated weights for policy 1, policy_version 121461 (0.0010) [2023-12-26 16:14:48,313][105620] Updated weights for policy 1, policy_version 121471 (0.0010) [2023-12-26 16:14:48,329][105692] Updated weights for policy 0, policy_version 120938 (0.0010) [2023-12-26 16:14:48,396][105692] Updated weights for policy 0, policy_version 120948 (0.0007) [2023-12-26 16:14:48,461][105692] Updated weights for policy 0, policy_version 120958 (0.0006) [2023-12-26 16:14:48,531][105692] Updated weights for policy 0, policy_version 120968 (0.0005) [2023-12-26 16:14:49,003][105620] Updated weights for policy 1, policy_version 121481 (0.0010) [2023-12-26 16:14:49,062][105620] Updated weights for policy 1, policy_version 121491 (0.0009) [2023-12-26 16:14:49,087][105692] Updated weights for policy 0, policy_version 120978 (0.0008) [2023-12-26 16:14:49,108][105620] Updated weights for policy 1, policy_version 121501 (0.0005) [2023-12-26 16:14:49,151][105692] Updated weights for policy 0, policy_version 120988 (0.0008) [2023-12-26 16:14:49,164][105620] Updated weights for policy 1, policy_version 121511 (0.0008) [2023-12-26 16:14:49,222][105692] Updated weights for policy 0, policy_version 120998 (0.0007) [2023-12-26 16:14:49,936][105620] Updated weights for policy 1, policy_version 121521 (0.0009) [2023-12-26 16:14:49,939][105692] Updated weights for policy 0, policy_version 121008 (0.0007) [2023-12-26 16:14:49,998][105692] Updated weights for policy 0, policy_version 121018 (0.0009) [2023-12-26 16:14:50,003][105620] Updated weights for policy 1, policy_version 121531 (0.0009) [2023-12-26 16:14:50,064][105620] Updated weights for policy 1, policy_version 121541 (0.0009) [2023-12-26 16:14:50,066][105692] Updated weights for policy 0, policy_version 121028 (0.0007) [2023-12-26 16:14:50,665][105692] Updated weights for policy 0, policy_version 121038 (0.0009) [2023-12-26 16:14:50,714][105692] Updated weights for policy 0, policy_version 121048 (0.0010) [2023-12-26 16:14:50,745][105620] Updated weights for policy 1, policy_version 121551 (0.0007) [2023-12-26 16:14:50,770][105692] Updated weights for policy 0, policy_version 121058 (0.0011) [2023-12-26 16:14:50,808][105620] Updated weights for policy 1, policy_version 121561 (0.0006) [2023-12-26 16:14:50,875][105620] Updated weights for policy 1, policy_version 121571 (0.0008) [2023-12-26 16:14:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 62128128. Throughput: 0: 9940.9, 1: 9646.2. Samples: 62112136. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-12-26 16:14:51,062][104569] Avg episode reward: [(0, '8712.647'), (1, '8999.438')] [2023-12-26 16:14:51,577][105692] Updated weights for policy 0, policy_version 121068 (0.0010) [2023-12-26 16:14:51,583][105620] Updated weights for policy 1, policy_version 121581 (0.0008) [2023-12-26 16:14:51,647][105692] Updated weights for policy 0, policy_version 121078 (0.0006) [2023-12-26 16:14:51,656][105620] Updated weights for policy 1, policy_version 121591 (0.0009) [2023-12-26 16:14:51,704][105692] Updated weights for policy 0, policy_version 121088 (0.0006) [2023-12-26 16:14:51,716][105620] Updated weights for policy 1, policy_version 121601 (0.0011) [2023-12-26 16:14:52,457][105692] Updated weights for policy 0, policy_version 121098 (0.0008) [2023-12-26 16:14:52,476][105620] Updated weights for policy 1, policy_version 121611 (0.0009) [2023-12-26 16:14:52,513][105692] Updated weights for policy 0, policy_version 121108 (0.0007) [2023-12-26 16:14:52,539][105620] Updated weights for policy 1, policy_version 121621 (0.0008) [2023-12-26 16:14:52,570][105692] Updated weights for policy 0, policy_version 121118 (0.0007) [2023-12-26 16:14:52,590][105620] Updated weights for policy 1, policy_version 121631 (0.0007) [2023-12-26 16:14:52,627][105692] Updated weights for policy 0, policy_version 121128 (0.0006) [2023-12-26 16:14:53,309][105620] Updated weights for policy 1, policy_version 121641 (0.0007) [2023-12-26 16:14:53,361][105620] Updated weights for policy 1, policy_version 121651 (0.0009) [2023-12-26 16:14:53,363][105692] Updated weights for policy 0, policy_version 121138 (0.0007) [2023-12-26 16:14:53,418][105620] Updated weights for policy 1, policy_version 121661 (0.0006) [2023-12-26 16:14:53,420][105692] Updated weights for policy 0, policy_version 121148 (0.0006) [2023-12-26 16:14:53,464][105692] Updated weights for policy 0, policy_version 121158 (0.0007) [2023-12-26 16:14:53,474][105620] Updated weights for policy 1, policy_version 121671 (0.0008) [2023-12-26 16:14:54,169][105620] Updated weights for policy 1, policy_version 121681 (0.0006) [2023-12-26 16:14:54,231][105620] Updated weights for policy 1, policy_version 121691 (0.0006) [2023-12-26 16:14:54,264][105692] Updated weights for policy 0, policy_version 121168 (0.0008) [2023-12-26 16:14:54,284][105620] Updated weights for policy 1, policy_version 121701 (0.0006) [2023-12-26 16:14:54,314][105692] Updated weights for policy 0, policy_version 121178 (0.0008) [2023-12-26 16:14:54,360][105692] Updated weights for policy 0, policy_version 121188 (0.0006) [2023-12-26 16:14:54,853][105620] Updated weights for policy 1, policy_version 121711 (0.0005) [2023-12-26 16:14:54,906][105620] Updated weights for policy 1, policy_version 121721 (0.0005) [2023-12-26 16:14:54,956][105620] Updated weights for policy 1, policy_version 121731 (0.0006) [2023-12-26 16:14:55,209][105692] Updated weights for policy 0, policy_version 121198 (0.0009) [2023-12-26 16:14:55,269][105692] Updated weights for policy 0, policy_version 121208 (0.0008) [2023-12-26 16:14:55,324][105692] Updated weights for policy 0, policy_version 121218 (0.0008) [2023-12-26 16:14:55,611][105620] Updated weights for policy 1, policy_version 121741 (0.0011) [2023-12-26 16:14:55,670][105620] Updated weights for policy 1, policy_version 121751 (0.0010) [2023-12-26 16:14:55,728][105620] Updated weights for policy 1, policy_version 121761 (0.0010) [2023-12-26 16:14:56,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 62218240. Throughput: 0: 9932.9, 1: 9680.8. Samples: 62228060. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-12-26 16:14:56,062][104569] Avg episode reward: [(0, '8347.840'), (1, '3712.333')] [2023-12-26 16:14:56,088][105692] Updated weights for policy 0, policy_version 121228 (0.0009) [2023-12-26 16:14:56,151][105692] Updated weights for policy 0, policy_version 121238 (0.0008) [2023-12-26 16:14:56,210][105692] Updated weights for policy 0, policy_version 121248 (0.0008) [2023-12-26 16:14:56,472][105620] Updated weights for policy 1, policy_version 121771 (0.0009) [2023-12-26 16:14:56,520][105620] Updated weights for policy 1, policy_version 121781 (0.0005) [2023-12-26 16:14:56,576][105620] Updated weights for policy 1, policy_version 121791 (0.0005) [2023-12-26 16:14:56,902][105692] Updated weights for policy 0, policy_version 121258 (0.0008) [2023-12-26 16:14:56,949][105692] Updated weights for policy 0, policy_version 121268 (0.0007) [2023-12-26 16:14:56,999][105692] Updated weights for policy 0, policy_version 121278 (0.0009) [2023-12-26 16:14:57,113][105620] Updated weights for policy 1, policy_version 121801 (0.0006) [2023-12-26 16:14:57,161][105620] Updated weights for policy 1, policy_version 121811 (0.0010) [2023-12-26 16:14:57,206][105620] Updated weights for policy 1, policy_version 121821 (0.0008) [2023-12-26 16:14:57,252][105620] Updated weights for policy 1, policy_version 121831 (0.0005) [2023-12-26 16:14:57,718][105692] Updated weights for policy 0, policy_version 121289 (0.0010) [2023-12-26 16:14:57,773][105692] Updated weights for policy 0, policy_version 121299 (0.0011) [2023-12-26 16:14:57,832][105692] Updated weights for policy 0, policy_version 121310 (0.0009) [2023-12-26 16:14:57,863][105620] Updated weights for policy 1, policy_version 121841 (0.0005) [2023-12-26 16:14:57,882][105692] Updated weights for policy 0, policy_version 121320 (0.0008) [2023-12-26 16:14:57,910][105620] Updated weights for policy 1, policy_version 121851 (0.0005) [2023-12-26 16:14:57,966][105620] Updated weights for policy 1, policy_version 121861 (0.0005) [2023-12-26 16:14:58,678][105620] Updated weights for policy 1, policy_version 121871 (0.0009) [2023-12-26 16:14:58,742][105620] Updated weights for policy 1, policy_version 121881 (0.0010) [2023-12-26 16:14:58,767][105692] Updated weights for policy 0, policy_version 121330 (0.0008) [2023-12-26 16:14:58,802][105620] Updated weights for policy 1, policy_version 121891 (0.0009) [2023-12-26 16:14:58,840][105692] Updated weights for policy 0, policy_version 121340 (0.0008) [2023-12-26 16:14:58,908][105692] Updated weights for policy 0, policy_version 121350 (0.0009) [2023-12-26 16:14:59,551][105620] Updated weights for policy 1, policy_version 121901 (0.0008) [2023-12-26 16:14:59,609][105620] Updated weights for policy 1, policy_version 121911 (0.0010) [2023-12-26 16:14:59,667][105620] Updated weights for policy 1, policy_version 121921 (0.0007) [2023-12-26 16:14:59,767][105692] Updated weights for policy 0, policy_version 121360 (0.0008) [2023-12-26 16:14:59,825][105692] Updated weights for policy 0, policy_version 121370 (0.0008) [2023-12-26 16:14:59,868][105585] KL-divergence is very high: 104.0585 [2023-12-26 16:14:59,892][105692] Updated weights for policy 0, policy_version 121380 (0.0008) [2023-12-26 16:15:00,338][105620] Updated weights for policy 1, policy_version 121931 (0.0007) [2023-12-26 16:15:00,390][105620] Updated weights for policy 1, policy_version 121941 (0.0011) [2023-12-26 16:15:00,450][105620] Updated weights for policy 1, policy_version 121951 (0.0011) [2023-12-26 16:15:00,639][105692] Updated weights for policy 0, policy_version 121390 (0.0007) [2023-12-26 16:15:00,688][105692] Updated weights for policy 0, policy_version 121400 (0.0005) [2023-12-26 16:15:00,741][105692] Updated weights for policy 0, policy_version 121410 (0.0005) [2023-12-26 16:15:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 62316544. Throughput: 0: 9941.6, 1: 9775.6. Samples: 62288172. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-12-26 16:15:01,063][104569] Avg episode reward: [(0, '8803.675'), (1, '2893.205')] [2023-12-26 16:15:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000121416_31088640.pth... [2023-12-26 16:15:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000121960_31227904.pth... [2023-12-26 16:15:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000120296_30801920.pth [2023-12-26 16:15:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000120840_30941184.pth [2023-12-26 16:15:01,180][105620] Updated weights for policy 1, policy_version 121961 (0.0011) [2023-12-26 16:15:01,236][105620] Updated weights for policy 1, policy_version 121971 (0.0010) [2023-12-26 16:15:01,292][105620] Updated weights for policy 1, policy_version 121981 (0.0010) [2023-12-26 16:15:01,309][105692] Updated weights for policy 0, policy_version 121420 (0.0007) [2023-12-26 16:15:01,344][105620] Updated weights for policy 1, policy_version 121991 (0.0010) [2023-12-26 16:15:01,366][105692] Updated weights for policy 0, policy_version 121430 (0.0011) [2023-12-26 16:15:01,429][105692] Updated weights for policy 0, policy_version 121440 (0.0009) [2023-12-26 16:15:02,066][105620] Updated weights for policy 1, policy_version 122001 (0.0010) [2023-12-26 16:15:02,107][105692] Updated weights for policy 0, policy_version 121450 (0.0009) [2023-12-26 16:15:02,126][105620] Updated weights for policy 1, policy_version 122011 (0.0010) [2023-12-26 16:15:02,152][105692] Updated weights for policy 0, policy_version 121460 (0.0010) [2023-12-26 16:15:02,177][105620] Updated weights for policy 1, policy_version 122021 (0.0005) [2023-12-26 16:15:02,197][105692] Updated weights for policy 0, policy_version 121470 (0.0010) [2023-12-26 16:15:02,246][105692] Updated weights for policy 0, policy_version 121480 (0.0010) [2023-12-26 16:15:02,899][105620] Updated weights for policy 1, policy_version 122031 (0.0010) [2023-12-26 16:15:02,944][105620] Updated weights for policy 1, policy_version 122041 (0.0010) [2023-12-26 16:15:02,995][105620] Updated weights for policy 1, policy_version 122051 (0.0010) [2023-12-26 16:15:03,019][105692] Updated weights for policy 0, policy_version 121490 (0.0008) [2023-12-26 16:15:03,071][105692] Updated weights for policy 0, policy_version 121500 (0.0006) [2023-12-26 16:15:03,124][105692] Updated weights for policy 0, policy_version 121510 (0.0008) [2023-12-26 16:15:03,645][105620] Updated weights for policy 1, policy_version 122061 (0.0009) [2023-12-26 16:15:03,701][105620] Updated weights for policy 1, policy_version 122071 (0.0009) [2023-12-26 16:15:03,750][105620] Updated weights for policy 1, policy_version 122081 (0.0009) [2023-12-26 16:15:03,825][105692] Updated weights for policy 0, policy_version 121520 (0.0009) [2023-12-26 16:15:03,883][105692] Updated weights for policy 0, policy_version 121530 (0.0009) [2023-12-26 16:15:03,940][105692] Updated weights for policy 0, policy_version 121540 (0.0007) [2023-12-26 16:15:04,408][105620] Updated weights for policy 1, policy_version 122091 (0.0009) [2023-12-26 16:15:04,471][105620] Updated weights for policy 1, policy_version 122101 (0.0008) [2023-12-26 16:15:04,522][105620] Updated weights for policy 1, policy_version 122111 (0.0010) [2023-12-26 16:15:04,724][105692] Updated weights for policy 0, policy_version 121550 (0.0007) [2023-12-26 16:15:04,786][105692] Updated weights for policy 0, policy_version 121560 (0.0007) [2023-12-26 16:15:04,850][105692] Updated weights for policy 0, policy_version 121570 (0.0008) [2023-12-26 16:15:05,166][105620] Updated weights for policy 1, policy_version 122121 (0.0010) [2023-12-26 16:15:05,217][105620] Updated weights for policy 1, policy_version 122131 (0.0008) [2023-12-26 16:15:05,265][105620] Updated weights for policy 1, policy_version 122141 (0.0010) [2023-12-26 16:15:05,309][105620] Updated weights for policy 1, policy_version 122151 (0.0010) [2023-12-26 16:15:05,447][105692] Updated weights for policy 0, policy_version 121580 (0.0007) [2023-12-26 16:15:05,495][105692] Updated weights for policy 0, policy_version 121590 (0.0008) [2023-12-26 16:15:05,541][105692] Updated weights for policy 0, policy_version 121600 (0.0009) [2023-12-26 16:15:05,983][105620] Updated weights for policy 1, policy_version 122161 (0.0008) [2023-12-26 16:15:06,038][105620] Updated weights for policy 1, policy_version 122171 (0.0009) [2023-12-26 16:15:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 62414848. Throughput: 0: 9841.2, 1: 9804.2. Samples: 62406376. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-12-26 16:15:06,062][104569] Avg episode reward: [(0, '9079.771'), (1, '6928.932')] [2023-12-26 16:15:06,094][105620] Updated weights for policy 1, policy_version 122181 (0.0010) [2023-12-26 16:15:06,331][105692] Updated weights for policy 0, policy_version 121611 (0.0008) [2023-12-26 16:15:06,393][105692] Updated weights for policy 0, policy_version 121621 (0.0005) [2023-12-26 16:15:06,456][105692] Updated weights for policy 0, policy_version 121631 (0.0008) [2023-12-26 16:15:06,722][105620] Updated weights for policy 1, policy_version 122191 (0.0010) [2023-12-26 16:15:06,780][105620] Updated weights for policy 1, policy_version 122201 (0.0010) [2023-12-26 16:15:06,845][105620] Updated weights for policy 1, policy_version 122211 (0.0010) [2023-12-26 16:15:07,132][105692] Updated weights for policy 0, policy_version 121641 (0.0010) [2023-12-26 16:15:07,190][105692] Updated weights for policy 0, policy_version 121651 (0.0010) [2023-12-26 16:15:07,255][105692] Updated weights for policy 0, policy_version 121661 (0.0010) [2023-12-26 16:15:07,317][105692] Updated weights for policy 0, policy_version 121671 (0.0011) [2023-12-26 16:15:07,473][105620] Updated weights for policy 1, policy_version 122221 (0.0010) [2023-12-26 16:15:07,530][105620] Updated weights for policy 1, policy_version 122231 (0.0011) [2023-12-26 16:15:07,597][105620] Updated weights for policy 1, policy_version 122241 (0.0011) [2023-12-26 16:15:07,959][105692] Updated weights for policy 0, policy_version 121681 (0.0006) [2023-12-26 16:15:08,017][105692] Updated weights for policy 0, policy_version 121691 (0.0007) [2023-12-26 16:15:08,066][105692] Updated weights for policy 0, policy_version 121701 (0.0007) [2023-12-26 16:15:08,327][105620] Updated weights for policy 1, policy_version 122251 (0.0011) [2023-12-26 16:15:08,396][105620] Updated weights for policy 1, policy_version 122261 (0.0007) [2023-12-26 16:15:08,456][105620] Updated weights for policy 1, policy_version 122271 (0.0009) [2023-12-26 16:15:08,665][105692] Updated weights for policy 0, policy_version 121711 (0.0007) [2023-12-26 16:15:08,719][105692] Updated weights for policy 0, policy_version 121721 (0.0008) [2023-12-26 16:15:08,776][105692] Updated weights for policy 0, policy_version 121731 (0.0009) [2023-12-26 16:15:09,046][105620] Updated weights for policy 1, policy_version 122281 (0.0010) [2023-12-26 16:15:09,102][105620] Updated weights for policy 1, policy_version 122291 (0.0010) [2023-12-26 16:15:09,161][105620] Updated weights for policy 1, policy_version 122301 (0.0006) [2023-12-26 16:15:09,228][105620] Updated weights for policy 1, policy_version 122311 (0.0010) [2023-12-26 16:15:09,582][105692] Updated weights for policy 0, policy_version 121742 (0.0008) [2023-12-26 16:15:09,641][105692] Updated weights for policy 0, policy_version 121752 (0.0005) [2023-12-26 16:15:09,697][105692] Updated weights for policy 0, policy_version 121762 (0.0005) [2023-12-26 16:15:10,033][105620] Updated weights for policy 1, policy_version 122321 (0.0007) [2023-12-26 16:15:10,093][105620] Updated weights for policy 1, policy_version 122331 (0.0008) [2023-12-26 16:15:10,150][105620] Updated weights for policy 1, policy_version 122341 (0.0008) [2023-12-26 16:15:10,394][105692] Updated weights for policy 0, policy_version 121772 (0.0009) [2023-12-26 16:15:10,456][105692] Updated weights for policy 0, policy_version 121782 (0.0009) [2023-12-26 16:15:10,518][105692] Updated weights for policy 0, policy_version 121792 (0.0009) [2023-12-26 16:15:10,848][105620] Updated weights for policy 1, policy_version 122351 (0.0008) [2023-12-26 16:15:10,900][105620] Updated weights for policy 1, policy_version 122361 (0.0009) [2023-12-26 16:15:10,952][105620] Updated weights for policy 1, policy_version 122371 (0.0008) [2023-12-26 16:15:11,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 62521344. Throughput: 0: 9839.1, 1: 9832.8. Samples: 62527864. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-12-26 16:15:11,063][104569] Avg episode reward: [(0, '9170.164'), (1, '7584.282')] [2023-12-26 16:15:11,224][105692] Updated weights for policy 0, policy_version 121802 (0.0007) [2023-12-26 16:15:11,292][105692] Updated weights for policy 0, policy_version 121812 (0.0009) [2023-12-26 16:15:11,354][105692] Updated weights for policy 0, policy_version 121822 (0.0008) [2023-12-26 16:15:11,418][105692] Updated weights for policy 0, policy_version 121832 (0.0009) [2023-12-26 16:15:11,786][105620] Updated weights for policy 1, policy_version 122381 (0.0008) [2023-12-26 16:15:11,848][105620] Updated weights for policy 1, policy_version 122391 (0.0008) [2023-12-26 16:15:11,912][105620] Updated weights for policy 1, policy_version 122401 (0.0008) [2023-12-26 16:15:12,083][105692] Updated weights for policy 0, policy_version 121842 (0.0007) [2023-12-26 16:15:12,133][105692] Updated weights for policy 0, policy_version 121852 (0.0008) [2023-12-26 16:15:12,185][105692] Updated weights for policy 0, policy_version 121862 (0.0009) [2023-12-26 16:15:12,676][105620] Updated weights for policy 1, policy_version 122411 (0.0009) [2023-12-26 16:15:12,743][105620] Updated weights for policy 1, policy_version 122421 (0.0011) [2023-12-26 16:15:12,806][105620] Updated weights for policy 1, policy_version 122431 (0.0011) [2023-12-26 16:15:12,957][105692] Updated weights for policy 0, policy_version 121872 (0.0008) [2023-12-26 16:15:13,027][105692] Updated weights for policy 0, policy_version 121882 (0.0008) [2023-12-26 16:15:13,083][105692] Updated weights for policy 0, policy_version 121892 (0.0010) [2023-12-26 16:15:13,543][105620] Updated weights for policy 1, policy_version 122441 (0.0010) [2023-12-26 16:15:13,600][105620] Updated weights for policy 1, policy_version 122451 (0.0010) [2023-12-26 16:15:13,658][105620] Updated weights for policy 1, policy_version 122461 (0.0010) [2023-12-26 16:15:13,715][105620] Updated weights for policy 1, policy_version 122471 (0.0010) [2023-12-26 16:15:13,778][105692] Updated weights for policy 0, policy_version 121902 (0.0007) [2023-12-26 16:15:13,834][105692] Updated weights for policy 0, policy_version 121912 (0.0005) [2023-12-26 16:15:13,898][105692] Updated weights for policy 0, policy_version 121922 (0.0010) [2023-12-26 16:15:14,451][105620] Updated weights for policy 1, policy_version 122481 (0.0010) [2023-12-26 16:15:14,510][105620] Updated weights for policy 1, policy_version 122491 (0.0010) [2023-12-26 16:15:14,569][105620] Updated weights for policy 1, policy_version 122501 (0.0010) [2023-12-26 16:15:14,578][105692] Updated weights for policy 0, policy_version 121932 (0.0009) [2023-12-26 16:15:14,637][105692] Updated weights for policy 0, policy_version 121942 (0.0007) [2023-12-26 16:15:14,687][105692] Updated weights for policy 0, policy_version 121952 (0.0009) [2023-12-26 16:15:15,252][105620] Updated weights for policy 1, policy_version 122511 (0.0010) [2023-12-26 16:15:15,312][105620] Updated weights for policy 1, policy_version 122521 (0.0011) [2023-12-26 16:15:15,374][105620] Updated weights for policy 1, policy_version 122531 (0.0011) [2023-12-26 16:15:15,414][105692] Updated weights for policy 0, policy_version 121962 (0.0009) [2023-12-26 16:15:15,485][105692] Updated weights for policy 0, policy_version 121972 (0.0006) [2023-12-26 16:15:15,553][105692] Updated weights for policy 0, policy_version 121982 (0.0005) [2023-12-26 16:15:15,622][105692] Updated weights for policy 0, policy_version 121992 (0.0005) [2023-12-26 16:15:16,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 62611456. Throughput: 0: 9793.5, 1: 9736.0. Samples: 62584044. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-12-26 16:15:16,063][104569] Avg episode reward: [(0, '8894.244'), (1, '8676.655')] [2023-12-26 16:15:16,082][105620] Updated weights for policy 1, policy_version 122541 (0.0011) [2023-12-26 16:15:16,111][105692] Updated weights for policy 0, policy_version 122002 (0.0011) [2023-12-26 16:15:16,144][105620] Updated weights for policy 1, policy_version 122551 (0.0006) [2023-12-26 16:15:16,173][105692] Updated weights for policy 0, policy_version 122012 (0.0011) [2023-12-26 16:15:16,196][105620] Updated weights for policy 1, policy_version 122561 (0.0009) [2023-12-26 16:15:16,231][105692] Updated weights for policy 0, policy_version 122022 (0.0010) [2023-12-26 16:15:16,236][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000122568_31383552.pth... [2023-12-26 16:15:16,241][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000122024_31244288.pth... [2023-12-26 16:15:16,241][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000121384_31080448.pth [2023-12-26 16:15:16,245][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000120872_30949376.pth [2023-12-26 16:15:16,857][105620] Updated weights for policy 1, policy_version 122571 (0.0010) [2023-12-26 16:15:16,918][105620] Updated weights for policy 1, policy_version 122581 (0.0010) [2023-12-26 16:15:16,935][105692] Updated weights for policy 0, policy_version 122032 (0.0006) [2023-12-26 16:15:16,970][105620] Updated weights for policy 1, policy_version 122591 (0.0010) [2023-12-26 16:15:17,006][105692] Updated weights for policy 0, policy_version 122042 (0.0005) [2023-12-26 16:15:17,069][105692] Updated weights for policy 0, policy_version 122052 (0.0006) [2023-12-26 16:15:17,578][105620] Updated weights for policy 1, policy_version 122601 (0.0010) [2023-12-26 16:15:17,638][105692] Updated weights for policy 0, policy_version 122062 (0.0005) [2023-12-26 16:15:17,647][105620] Updated weights for policy 1, policy_version 122611 (0.0008) [2023-12-26 16:15:17,698][105692] Updated weights for policy 0, policy_version 122072 (0.0005) [2023-12-26 16:15:17,714][105620] Updated weights for policy 1, policy_version 122621 (0.0009) [2023-12-26 16:15:17,754][105692] Updated weights for policy 0, policy_version 122082 (0.0009) [2023-12-26 16:15:17,766][105620] Updated weights for policy 1, policy_version 122631 (0.0010) [2023-12-26 16:15:18,366][105692] Updated weights for policy 0, policy_version 122092 (0.0009) [2023-12-26 16:15:18,425][105692] Updated weights for policy 0, policy_version 122102 (0.0006) [2023-12-26 16:15:18,477][105692] Updated weights for policy 0, policy_version 122112 (0.0010) [2023-12-26 16:15:18,491][105620] Updated weights for policy 1, policy_version 122641 (0.0010) [2023-12-26 16:15:18,553][105620] Updated weights for policy 1, policy_version 122651 (0.0010) [2023-12-26 16:15:18,615][105620] Updated weights for policy 1, policy_version 122661 (0.0010) [2023-12-26 16:15:19,200][105692] Updated weights for policy 0, policy_version 122122 (0.0011) [2023-12-26 16:15:19,267][105692] Updated weights for policy 0, policy_version 122132 (0.0011) [2023-12-26 16:15:19,333][105692] Updated weights for policy 0, policy_version 122142 (0.0010) [2023-12-26 16:15:19,372][105620] Updated weights for policy 1, policy_version 122671 (0.0010) [2023-12-26 16:15:19,399][105692] Updated weights for policy 0, policy_version 122152 (0.0011) [2023-12-26 16:15:19,437][105620] Updated weights for policy 1, policy_version 122681 (0.0009) [2023-12-26 16:15:19,506][105620] Updated weights for policy 1, policy_version 122691 (0.0009) [2023-12-26 16:15:20,147][105620] Updated weights for policy 1, policy_version 122701 (0.0007) [2023-12-26 16:15:20,150][105692] Updated weights for policy 0, policy_version 122162 (0.0006) [2023-12-26 16:15:20,204][105620] Updated weights for policy 1, policy_version 122711 (0.0005) [2023-12-26 16:15:20,210][105692] Updated weights for policy 0, policy_version 122172 (0.0006) [2023-12-26 16:15:20,267][105692] Updated weights for policy 0, policy_version 122182 (0.0006) [2023-12-26 16:15:20,267][105620] Updated weights for policy 1, policy_version 122721 (0.0006) [2023-12-26 16:15:20,952][105620] Updated weights for policy 1, policy_version 122731 (0.0007) [2023-12-26 16:15:20,980][105692] Updated weights for policy 0, policy_version 122192 (0.0007) [2023-12-26 16:15:21,006][105620] Updated weights for policy 1, policy_version 122741 (0.0008) [2023-12-26 16:15:21,042][105692] Updated weights for policy 0, policy_version 122202 (0.0007) [2023-12-26 16:15:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 62709760. Throughput: 0: 9773.2, 1: 9863.0. Samples: 62706096. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-12-26 16:15:21,062][104569] Avg episode reward: [(0, '8710.449'), (1, '8340.417')] [2023-12-26 16:15:21,071][105620] Updated weights for policy 1, policy_version 122751 (0.0007) [2023-12-26 16:15:21,111][105692] Updated weights for policy 0, policy_version 122212 (0.0008) [2023-12-26 16:15:21,826][105620] Updated weights for policy 1, policy_version 122761 (0.0007) [2023-12-26 16:15:21,885][105620] Updated weights for policy 1, policy_version 122771 (0.0008) [2023-12-26 16:15:21,948][105692] Updated weights for policy 0, policy_version 122222 (0.0007) [2023-12-26 16:15:21,949][105620] Updated weights for policy 1, policy_version 122781 (0.0009) [2023-12-26 16:15:22,006][105620] Updated weights for policy 1, policy_version 122791 (0.0008) [2023-12-26 16:15:22,008][105692] Updated weights for policy 0, policy_version 122232 (0.0006) [2023-12-26 16:15:22,063][105692] Updated weights for policy 0, policy_version 122242 (0.0009) [2023-12-26 16:15:22,761][105620] Updated weights for policy 1, policy_version 122801 (0.0008) [2023-12-26 16:15:22,809][105620] Updated weights for policy 1, policy_version 122811 (0.0009) [2023-12-26 16:15:22,850][105692] Updated weights for policy 0, policy_version 122252 (0.0008) [2023-12-26 16:15:22,865][105620] Updated weights for policy 1, policy_version 122821 (0.0008) [2023-12-26 16:15:22,901][105692] Updated weights for policy 0, policy_version 122262 (0.0007) [2023-12-26 16:15:22,967][105692] Updated weights for policy 0, policy_version 122272 (0.0009) [2023-12-26 16:15:23,659][105620] Updated weights for policy 1, policy_version 122831 (0.0008) [2023-12-26 16:15:23,701][105692] Updated weights for policy 0, policy_version 122282 (0.0009) [2023-12-26 16:15:23,705][105620] Updated weights for policy 1, policy_version 122841 (0.0008) [2023-12-26 16:15:23,749][105692] Updated weights for policy 0, policy_version 122292 (0.0006) [2023-12-26 16:15:23,758][105620] Updated weights for policy 1, policy_version 122851 (0.0007) [2023-12-26 16:15:23,796][105692] Updated weights for policy 0, policy_version 122302 (0.0007) [2023-12-26 16:15:23,857][105692] Updated weights for policy 0, policy_version 122312 (0.0010) [2023-12-26 16:15:24,402][105620] Updated weights for policy 1, policy_version 122861 (0.0008) [2023-12-26 16:15:24,453][105620] Updated weights for policy 1, policy_version 122871 (0.0009) [2023-12-26 16:15:24,511][105620] Updated weights for policy 1, policy_version 122881 (0.0009) [2023-12-26 16:15:24,627][105692] Updated weights for policy 0, policy_version 122322 (0.0009) [2023-12-26 16:15:24,691][105692] Updated weights for policy 0, policy_version 122332 (0.0009) [2023-12-26 16:15:24,755][105692] Updated weights for policy 0, policy_version 122342 (0.0009) [2023-12-26 16:15:25,242][105620] Updated weights for policy 1, policy_version 122891 (0.0009) [2023-12-26 16:15:25,289][105620] Updated weights for policy 1, policy_version 122901 (0.0008) [2023-12-26 16:15:25,339][105620] Updated weights for policy 1, policy_version 122911 (0.0008) [2023-12-26 16:15:25,500][105692] Updated weights for policy 0, policy_version 122352 (0.0010) [2023-12-26 16:15:25,557][105692] Updated weights for policy 0, policy_version 122362 (0.0010) [2023-12-26 16:15:25,610][105692] Updated weights for policy 0, policy_version 122372 (0.0006) [2023-12-26 16:15:25,940][105620] Updated weights for policy 1, policy_version 122921 (0.0008) [2023-12-26 16:15:25,991][105620] Updated weights for policy 1, policy_version 122931 (0.0005) [2023-12-26 16:15:26,037][105620] Updated weights for policy 1, policy_version 122941 (0.0007) [2023-12-26 16:15:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19524.4, 300 sec: 19549.7). Total num frames: 62808064. Throughput: 0: 9746.9, 1: 9921.4. Samples: 62820316. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-12-26 16:15:26,062][104569] Avg episode reward: [(0, '8801.131'), (1, '7755.491')] [2023-12-26 16:15:26,094][105620] Updated weights for policy 1, policy_version 122951 (0.0007) [2023-12-26 16:15:26,218][105692] Updated weights for policy 0, policy_version 122382 (0.0005) [2023-12-26 16:15:26,277][105692] Updated weights for policy 0, policy_version 122392 (0.0006) [2023-12-26 16:15:26,350][105692] Updated weights for policy 0, policy_version 122402 (0.0005) [2023-12-26 16:15:26,778][105620] Updated weights for policy 1, policy_version 122961 (0.0009) [2023-12-26 16:15:26,850][105620] Updated weights for policy 1, policy_version 122971 (0.0006) [2023-12-26 16:15:26,910][105620] Updated weights for policy 1, policy_version 122981 (0.0005) [2023-12-26 16:15:26,955][105692] Updated weights for policy 0, policy_version 122412 (0.0008) [2023-12-26 16:15:27,009][105692] Updated weights for policy 0, policy_version 122422 (0.0006) [2023-12-26 16:15:27,054][105692] Updated weights for policy 0, policy_version 122432 (0.0005) [2023-12-26 16:15:27,446][105620] Updated weights for policy 1, policy_version 122991 (0.0005) [2023-12-26 16:15:27,497][105620] Updated weights for policy 1, policy_version 123001 (0.0005) [2023-12-26 16:15:27,550][105620] Updated weights for policy 1, policy_version 123011 (0.0005) [2023-12-26 16:15:27,582][105692] Updated weights for policy 0, policy_version 122442 (0.0005) [2023-12-26 16:15:27,631][105692] Updated weights for policy 0, policy_version 122452 (0.0007) [2023-12-26 16:15:27,676][105692] Updated weights for policy 0, policy_version 122462 (0.0008) [2023-12-26 16:15:27,729][105692] Updated weights for policy 0, policy_version 122472 (0.0008) [2023-12-26 16:15:28,192][105620] Updated weights for policy 1, policy_version 123021 (0.0008) [2023-12-26 16:15:28,256][105620] Updated weights for policy 1, policy_version 123031 (0.0005) [2023-12-26 16:15:28,309][105620] Updated weights for policy 1, policy_version 123041 (0.0008) [2023-12-26 16:15:28,320][105692] Updated weights for policy 0, policy_version 122482 (0.0007) [2023-12-26 16:15:28,374][105692] Updated weights for policy 0, policy_version 122492 (0.0007) [2023-12-26 16:15:28,433][105692] Updated weights for policy 0, policy_version 122502 (0.0005) [2023-12-26 16:15:29,044][105620] Updated weights for policy 1, policy_version 123051 (0.0007) [2023-12-26 16:15:29,050][105692] Updated weights for policy 0, policy_version 122512 (0.0009) [2023-12-26 16:15:29,103][105692] Updated weights for policy 0, policy_version 122522 (0.0006) [2023-12-26 16:15:29,104][105620] Updated weights for policy 1, policy_version 123061 (0.0008) [2023-12-26 16:15:29,151][105692] Updated weights for policy 0, policy_version 122532 (0.0008) [2023-12-26 16:15:29,170][105620] Updated weights for policy 1, policy_version 123071 (0.0007) [2023-12-26 16:15:29,862][105692] Updated weights for policy 0, policy_version 122542 (0.0009) [2023-12-26 16:15:29,863][105620] Updated weights for policy 1, policy_version 123081 (0.0006) [2023-12-26 16:15:29,913][105692] Updated weights for policy 0, policy_version 122552 (0.0009) [2023-12-26 16:15:29,928][105620] Updated weights for policy 1, policy_version 123091 (0.0006) [2023-12-26 16:15:29,977][105692] Updated weights for policy 0, policy_version 122562 (0.0009) [2023-12-26 16:15:29,980][105620] Updated weights for policy 1, policy_version 123101 (0.0009) [2023-12-26 16:15:30,033][105620] Updated weights for policy 1, policy_version 123111 (0.0009) [2023-12-26 16:15:30,708][105620] Updated weights for policy 1, policy_version 123121 (0.0009) [2023-12-26 16:15:30,758][105620] Updated weights for policy 1, policy_version 123131 (0.0008) [2023-12-26 16:15:30,762][105692] Updated weights for policy 0, policy_version 122573 (0.0008) [2023-12-26 16:15:30,810][105692] Updated weights for policy 0, policy_version 122583 (0.0009) [2023-12-26 16:15:30,812][105620] Updated weights for policy 1, policy_version 123141 (0.0005) [2023-12-26 16:15:30,861][105692] Updated weights for policy 0, policy_version 122593 (0.0009) [2023-12-26 16:15:31,062][104569] Fps is (10 sec: 21299.4, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 62922752. Throughput: 0: 9895.9, 1: 10029.2. Samples: 62886932. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-12-26 16:15:31,062][104569] Avg episode reward: [(0, '8984.445'), (1, '8655.105')] [2023-12-26 16:15:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000122600_31391744.pth... [2023-12-26 16:15:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000123144_31531008.pth... [2023-12-26 16:15:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000121416_31088640.pth [2023-12-26 16:15:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000121960_31227904.pth [2023-12-26 16:15:31,515][105620] Updated weights for policy 1, policy_version 123151 (0.0008) [2023-12-26 16:15:31,566][105620] Updated weights for policy 1, policy_version 123161 (0.0006) [2023-12-26 16:15:31,630][105620] Updated weights for policy 1, policy_version 123171 (0.0007) [2023-12-26 16:15:31,698][105692] Updated weights for policy 0, policy_version 122604 (0.0010) [2023-12-26 16:15:31,763][105692] Updated weights for policy 0, policy_version 122614 (0.0008) [2023-12-26 16:15:31,819][105692] Updated weights for policy 0, policy_version 122624 (0.0009) [2023-12-26 16:15:32,353][105620] Updated weights for policy 1, policy_version 123181 (0.0008) [2023-12-26 16:15:32,419][105620] Updated weights for policy 1, policy_version 123191 (0.0009) [2023-12-26 16:15:32,479][105620] Updated weights for policy 1, policy_version 123201 (0.0006) [2023-12-26 16:15:32,570][105692] Updated weights for policy 0, policy_version 122634 (0.0009) [2023-12-26 16:15:32,635][105692] Updated weights for policy 0, policy_version 122644 (0.0006) [2023-12-26 16:15:32,690][105692] Updated weights for policy 0, policy_version 122654 (0.0006) [2023-12-26 16:15:32,746][105692] Updated weights for policy 0, policy_version 122664 (0.0006) [2023-12-26 16:15:33,041][105620] Updated weights for policy 1, policy_version 123211 (0.0005) [2023-12-26 16:15:33,092][105620] Updated weights for policy 1, policy_version 123221 (0.0005) [2023-12-26 16:15:33,142][105620] Updated weights for policy 1, policy_version 123231 (0.0006) [2023-12-26 16:15:33,493][105692] Updated weights for policy 0, policy_version 122674 (0.0011) [2023-12-26 16:15:33,542][105692] Updated weights for policy 0, policy_version 122684 (0.0011) [2023-12-26 16:15:33,595][105692] Updated weights for policy 0, policy_version 122694 (0.0010) [2023-12-26 16:15:33,763][105620] Updated weights for policy 1, policy_version 123241 (0.0006) [2023-12-26 16:15:33,810][105620] Updated weights for policy 1, policy_version 123251 (0.0008) [2023-12-26 16:15:33,854][105620] Updated weights for policy 1, policy_version 123261 (0.0008) [2023-12-26 16:15:33,907][105620] Updated weights for policy 1, policy_version 123271 (0.0008) [2023-12-26 16:15:34,284][105692] Updated weights for policy 0, policy_version 122704 (0.0010) [2023-12-26 16:15:34,345][105692] Updated weights for policy 0, policy_version 122714 (0.0008) [2023-12-26 16:15:34,412][105692] Updated weights for policy 0, policy_version 122724 (0.0006) [2023-12-26 16:15:34,685][105620] Updated weights for policy 1, policy_version 123281 (0.0009) [2023-12-26 16:15:34,746][105620] Updated weights for policy 1, policy_version 123291 (0.0009) [2023-12-26 16:15:34,803][105620] Updated weights for policy 1, policy_version 123301 (0.0008) [2023-12-26 16:15:35,141][105692] Updated weights for policy 0, policy_version 122734 (0.0009) [2023-12-26 16:15:35,194][105692] Updated weights for policy 0, policy_version 122744 (0.0007) [2023-12-26 16:15:35,253][105692] Updated weights for policy 0, policy_version 122754 (0.0006) [2023-12-26 16:15:35,569][105620] Updated weights for policy 1, policy_version 123311 (0.0009) [2023-12-26 16:15:35,630][105620] Updated weights for policy 1, policy_version 123321 (0.0010) [2023-12-26 16:15:35,693][105620] Updated weights for policy 1, policy_version 123331 (0.0009) [2023-12-26 16:15:35,933][105692] Updated weights for policy 0, policy_version 122764 (0.0007) [2023-12-26 16:15:35,990][105692] Updated weights for policy 0, policy_version 122774 (0.0009) [2023-12-26 16:15:36,052][105692] Updated weights for policy 0, policy_version 122784 (0.0007) [2023-12-26 16:15:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 63012864. Throughput: 0: 9780.2, 1: 10052.1. Samples: 63004588. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-12-26 16:15:36,062][104569] Avg episode reward: [(0, '8994.974'), (1, '9013.258')] [2023-12-26 16:15:36,477][105620] Updated weights for policy 1, policy_version 123341 (0.0008) [2023-12-26 16:15:36,539][105620] Updated weights for policy 1, policy_version 123351 (0.0009) [2023-12-26 16:15:36,606][105620] Updated weights for policy 1, policy_version 123361 (0.0010) [2023-12-26 16:15:36,843][105692] Updated weights for policy 0, policy_version 122794 (0.0008) [2023-12-26 16:15:36,913][105692] Updated weights for policy 0, policy_version 122804 (0.0009) [2023-12-26 16:15:36,973][105692] Updated weights for policy 0, policy_version 122814 (0.0009) [2023-12-26 16:15:37,021][105692] Updated weights for policy 0, policy_version 122824 (0.0009) [2023-12-26 16:15:37,297][105620] Updated weights for policy 1, policy_version 123371 (0.0008) [2023-12-26 16:15:37,346][105620] Updated weights for policy 1, policy_version 123381 (0.0005) [2023-12-26 16:15:37,403][105620] Updated weights for policy 1, policy_version 123391 (0.0005) [2023-12-26 16:15:37,866][105692] Updated weights for policy 0, policy_version 122834 (0.0009) [2023-12-26 16:15:37,926][105692] Updated weights for policy 0, policy_version 122844 (0.0009) [2023-12-26 16:15:37,982][105692] Updated weights for policy 0, policy_version 122854 (0.0009) [2023-12-26 16:15:38,112][105620] Updated weights for policy 1, policy_version 123401 (0.0006) [2023-12-26 16:15:38,174][105620] Updated weights for policy 1, policy_version 123411 (0.0009) [2023-12-26 16:15:38,227][105620] Updated weights for policy 1, policy_version 123421 (0.0009) [2023-12-26 16:15:38,282][105620] Updated weights for policy 1, policy_version 123431 (0.0009) [2023-12-26 16:15:38,699][105692] Updated weights for policy 0, policy_version 122864 (0.0009) [2023-12-26 16:15:38,751][105692] Updated weights for policy 0, policy_version 122874 (0.0010) [2023-12-26 16:15:38,802][105692] Updated weights for policy 0, policy_version 122884 (0.0010) [2023-12-26 16:15:39,092][105620] Updated weights for policy 1, policy_version 123441 (0.0006) [2023-12-26 16:15:39,161][105620] Updated weights for policy 1, policy_version 123451 (0.0006) [2023-12-26 16:15:39,229][105620] Updated weights for policy 1, policy_version 123461 (0.0008) [2023-12-26 16:15:39,559][105692] Updated weights for policy 0, policy_version 122894 (0.0011) [2023-12-26 16:15:39,623][105692] Updated weights for policy 0, policy_version 122904 (0.0011) [2023-12-26 16:15:39,690][105692] Updated weights for policy 0, policy_version 122914 (0.0011) [2023-12-26 16:15:39,985][105620] Updated weights for policy 1, policy_version 123471 (0.0009) [2023-12-26 16:15:40,048][105620] Updated weights for policy 1, policy_version 123481 (0.0009) [2023-12-26 16:15:40,106][105620] Updated weights for policy 1, policy_version 123491 (0.0009) [2023-12-26 16:15:40,460][105692] Updated weights for policy 0, policy_version 122924 (0.0009) [2023-12-26 16:15:40,524][105692] Updated weights for policy 0, policy_version 122934 (0.0006) [2023-12-26 16:15:40,594][105692] Updated weights for policy 0, policy_version 122944 (0.0008) [2023-12-26 16:15:40,849][105620] Updated weights for policy 1, policy_version 123501 (0.0009) [2023-12-26 16:15:40,917][105620] Updated weights for policy 1, policy_version 123511 (0.0008) [2023-12-26 16:15:40,977][105620] Updated weights for policy 1, policy_version 123521 (0.0009) [2023-12-26 16:15:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 63111168. Throughput: 0: 9778.7, 1: 9959.0. Samples: 63116256. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-12-26 16:15:41,062][104569] Avg episode reward: [(0, '9085.633'), (1, '8580.847')] [2023-12-26 16:15:41,289][105692] Updated weights for policy 0, policy_version 122954 (0.0009) [2023-12-26 16:15:41,357][105692] Updated weights for policy 0, policy_version 122964 (0.0008) [2023-12-26 16:15:41,426][105692] Updated weights for policy 0, policy_version 122974 (0.0008) [2023-12-26 16:15:41,484][105692] Updated weights for policy 0, policy_version 122984 (0.0009) [2023-12-26 16:15:41,794][105620] Updated weights for policy 1, policy_version 123531 (0.0010) [2023-12-26 16:15:41,853][105620] Updated weights for policy 1, policy_version 123541 (0.0008) [2023-12-26 16:15:41,917][105620] Updated weights for policy 1, policy_version 123551 (0.0009) [2023-12-26 16:15:42,222][105692] Updated weights for policy 0, policy_version 122994 (0.0011) [2023-12-26 16:15:42,280][105692] Updated weights for policy 0, policy_version 123004 (0.0011) [2023-12-26 16:15:42,350][105692] Updated weights for policy 0, policy_version 123014 (0.0011) [2023-12-26 16:15:42,733][105620] Updated weights for policy 1, policy_version 123561 (0.0008) [2023-12-26 16:15:42,796][105620] Updated weights for policy 1, policy_version 123571 (0.0011) [2023-12-26 16:15:42,848][105620] Updated weights for policy 1, policy_version 123581 (0.0010) [2023-12-26 16:15:42,908][105620] Updated weights for policy 1, policy_version 123591 (0.0010) [2023-12-26 16:15:43,047][105692] Updated weights for policy 0, policy_version 123024 (0.0008) [2023-12-26 16:15:43,103][105692] Updated weights for policy 0, policy_version 123034 (0.0010) [2023-12-26 16:15:43,154][105692] Updated weights for policy 0, policy_version 123044 (0.0010) [2023-12-26 16:15:43,679][105620] Updated weights for policy 1, policy_version 123601 (0.0010) [2023-12-26 16:15:43,736][105620] Updated weights for policy 1, policy_version 123611 (0.0010) [2023-12-26 16:15:43,788][105620] Updated weights for policy 1, policy_version 123621 (0.0009) [2023-12-26 16:15:43,811][105692] Updated weights for policy 0, policy_version 123054 (0.0007) [2023-12-26 16:15:43,857][105692] Updated weights for policy 0, policy_version 123064 (0.0005) [2023-12-26 16:15:43,918][105692] Updated weights for policy 0, policy_version 123074 (0.0009) [2023-12-26 16:15:44,583][105620] Updated weights for policy 1, policy_version 123631 (0.0009) [2023-12-26 16:15:44,634][105620] Updated weights for policy 1, policy_version 123641 (0.0008) [2023-12-26 16:15:44,649][105692] Updated weights for policy 0, policy_version 123084 (0.0012) [2023-12-26 16:15:44,683][105620] Updated weights for policy 1, policy_version 123651 (0.0008) [2023-12-26 16:15:44,697][105692] Updated weights for policy 0, policy_version 123094 (0.0010) [2023-12-26 16:15:44,756][105692] Updated weights for policy 0, policy_version 123104 (0.0008) [2023-12-26 16:15:45,475][105620] Updated weights for policy 1, policy_version 123661 (0.0007) [2023-12-26 16:15:45,513][105692] Updated weights for policy 0, policy_version 123114 (0.0009) [2023-12-26 16:15:45,536][105620] Updated weights for policy 1, policy_version 123671 (0.0007) [2023-12-26 16:15:45,570][105692] Updated weights for policy 0, policy_version 123124 (0.0011) [2023-12-26 16:15:45,597][105620] Updated weights for policy 1, policy_version 123681 (0.0006) [2023-12-26 16:15:45,624][105692] Updated weights for policy 0, policy_version 123134 (0.0011) [2023-12-26 16:15:45,684][105692] Updated weights for policy 0, policy_version 123144 (0.0010) [2023-12-26 16:15:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 63201280. Throughput: 0: 9784.3, 1: 9839.1. Samples: 63171224. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-26 16:15:46,063][104569] Avg episode reward: [(0, '8707.715'), (1, '8664.642')] [2023-12-26 16:15:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000123688_31670272.pth... [2023-12-26 16:15:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000123144_31531008.pth... [2023-12-26 16:15:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000122568_31383552.pth [2023-12-26 16:15:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000122024_31244288.pth [2023-12-26 16:15:46,195][105620] Updated weights for policy 1, policy_version 123691 (0.0007) [2023-12-26 16:15:46,250][105620] Updated weights for policy 1, policy_version 123701 (0.0008) [2023-12-26 16:15:46,301][105620] Updated weights for policy 1, policy_version 123711 (0.0008) [2023-12-26 16:15:46,447][105692] Updated weights for policy 0, policy_version 123154 (0.0006) [2023-12-26 16:15:46,509][105692] Updated weights for policy 0, policy_version 123164 (0.0010) [2023-12-26 16:15:46,564][105692] Updated weights for policy 0, policy_version 123174 (0.0010) [2023-12-26 16:15:46,984][105620] Updated weights for policy 1, policy_version 123721 (0.0008) [2023-12-26 16:15:47,038][105620] Updated weights for policy 1, policy_version 123731 (0.0008) [2023-12-26 16:15:47,081][105620] Updated weights for policy 1, policy_version 123741 (0.0008) [2023-12-26 16:15:47,129][105620] Updated weights for policy 1, policy_version 123751 (0.0005) [2023-12-26 16:15:47,237][105692] Updated weights for policy 0, policy_version 123184 (0.0006) [2023-12-26 16:15:47,296][105692] Updated weights for policy 0, policy_version 123194 (0.0005) [2023-12-26 16:15:47,360][105692] Updated weights for policy 0, policy_version 123204 (0.0005) [2023-12-26 16:15:47,762][105620] Updated weights for policy 1, policy_version 123761 (0.0010) [2023-12-26 16:15:47,814][105620] Updated weights for policy 1, policy_version 123771 (0.0006) [2023-12-26 16:15:47,867][105620] Updated weights for policy 1, policy_version 123781 (0.0005) [2023-12-26 16:15:47,966][105692] Updated weights for policy 0, policy_version 123214 (0.0008) [2023-12-26 16:15:48,017][105692] Updated weights for policy 0, policy_version 123224 (0.0008) [2023-12-26 16:15:48,062][105692] Updated weights for policy 0, policy_version 123234 (0.0008) [2023-12-26 16:15:48,537][105620] Updated weights for policy 1, policy_version 123791 (0.0009) [2023-12-26 16:15:48,585][105620] Updated weights for policy 1, policy_version 123801 (0.0010) [2023-12-26 16:15:48,638][105620] Updated weights for policy 1, policy_version 123811 (0.0011) [2023-12-26 16:15:48,835][105692] Updated weights for policy 0, policy_version 123244 (0.0008) [2023-12-26 16:15:48,889][105692] Updated weights for policy 0, policy_version 123254 (0.0007) [2023-12-26 16:15:48,942][105692] Updated weights for policy 0, policy_version 123264 (0.0005) [2023-12-26 16:15:49,425][105620] Updated weights for policy 1, policy_version 123821 (0.0009) [2023-12-26 16:15:49,476][105620] Updated weights for policy 1, policy_version 123831 (0.0010) [2023-12-26 16:15:49,526][105620] Updated weights for policy 1, policy_version 123841 (0.0009) [2023-12-26 16:15:49,581][105692] Updated weights for policy 0, policy_version 123274 (0.0006) [2023-12-26 16:15:49,651][105692] Updated weights for policy 0, policy_version 123284 (0.0010) [2023-12-26 16:15:49,711][105692] Updated weights for policy 0, policy_version 123294 (0.0006) [2023-12-26 16:15:49,781][105692] Updated weights for policy 0, policy_version 123304 (0.0006) [2023-12-26 16:15:50,269][105620] Updated weights for policy 1, policy_version 123851 (0.0011) [2023-12-26 16:15:50,331][105620] Updated weights for policy 1, policy_version 123861 (0.0008) [2023-12-26 16:15:50,359][105692] Updated weights for policy 0, policy_version 123314 (0.0011) [2023-12-26 16:15:50,393][105620] Updated weights for policy 1, policy_version 123871 (0.0007) [2023-12-26 16:15:50,408][105692] Updated weights for policy 0, policy_version 123324 (0.0009) [2023-12-26 16:15:50,460][105692] Updated weights for policy 0, policy_version 123334 (0.0009) [2023-12-26 16:15:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 63299584. Throughput: 0: 9850.9, 1: 9832.0. Samples: 63292104. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-26 16:15:51,062][104569] Avg episode reward: [(0, '8621.910'), (1, '9353.487')] [2023-12-26 16:15:51,082][105620] Updated weights for policy 1, policy_version 123881 (0.0009) [2023-12-26 16:15:51,155][105620] Updated weights for policy 1, policy_version 123891 (0.0008) [2023-12-26 16:15:51,210][105620] Updated weights for policy 1, policy_version 123901 (0.0008) [2023-12-26 16:15:51,213][105692] Updated weights for policy 0, policy_version 123344 (0.0008) [2023-12-26 16:15:51,267][105620] Updated weights for policy 1, policy_version 123911 (0.0008) [2023-12-26 16:15:51,276][105692] Updated weights for policy 0, policy_version 123354 (0.0007) [2023-12-26 16:15:51,332][105692] Updated weights for policy 0, policy_version 123364 (0.0008) [2023-12-26 16:15:52,028][105620] Updated weights for policy 1, policy_version 123921 (0.0008) [2023-12-26 16:15:52,090][105620] Updated weights for policy 1, policy_version 123931 (0.0009) [2023-12-26 16:15:52,121][105692] Updated weights for policy 0, policy_version 123374 (0.0009) [2023-12-26 16:15:52,152][105620] Updated weights for policy 1, policy_version 123941 (0.0006) [2023-12-26 16:15:52,183][105692] Updated weights for policy 0, policy_version 123384 (0.0008) [2023-12-26 16:15:52,229][105692] Updated weights for policy 0, policy_version 123394 (0.0008) [2023-12-26 16:15:52,841][105620] Updated weights for policy 1, policy_version 123951 (0.0008) [2023-12-26 16:15:52,894][105620] Updated weights for policy 1, policy_version 123961 (0.0009) [2023-12-26 16:15:52,947][105620] Updated weights for policy 1, policy_version 123971 (0.0009) [2023-12-26 16:15:53,031][105692] Updated weights for policy 0, policy_version 123404 (0.0009) [2023-12-26 16:15:53,082][105692] Updated weights for policy 0, policy_version 123414 (0.0005) [2023-12-26 16:15:53,139][105692] Updated weights for policy 0, policy_version 123424 (0.0008) [2023-12-26 16:15:53,654][105620] Updated weights for policy 1, policy_version 123981 (0.0008) [2023-12-26 16:15:53,721][105620] Updated weights for policy 1, policy_version 123991 (0.0005) [2023-12-26 16:15:53,787][105620] Updated weights for policy 1, policy_version 124001 (0.0007) [2023-12-26 16:15:53,891][105692] Updated weights for policy 0, policy_version 123434 (0.0011) [2023-12-26 16:15:53,949][105692] Updated weights for policy 0, policy_version 123444 (0.0010) [2023-12-26 16:15:53,996][105692] Updated weights for policy 0, policy_version 123454 (0.0010) [2023-12-26 16:15:54,048][105692] Updated weights for policy 0, policy_version 123464 (0.0010) [2023-12-26 16:15:54,346][105620] Updated weights for policy 1, policy_version 124011 (0.0007) [2023-12-26 16:15:54,411][105620] Updated weights for policy 1, policy_version 124021 (0.0007) [2023-12-26 16:15:54,464][105620] Updated weights for policy 1, policy_version 124031 (0.0008) [2023-12-26 16:15:54,775][105692] Updated weights for policy 0, policy_version 123474 (0.0010) [2023-12-26 16:15:54,837][105692] Updated weights for policy 0, policy_version 123484 (0.0010) [2023-12-26 16:15:54,901][105692] Updated weights for policy 0, policy_version 123494 (0.0010) [2023-12-26 16:15:55,115][105620] Updated weights for policy 1, policy_version 124041 (0.0008) [2023-12-26 16:15:55,167][105620] Updated weights for policy 1, policy_version 124051 (0.0007) [2023-12-26 16:15:55,222][105620] Updated weights for policy 1, policy_version 124061 (0.0008) [2023-12-26 16:15:55,270][105620] Updated weights for policy 1, policy_version 124071 (0.0008) [2023-12-26 16:15:55,609][105692] Updated weights for policy 0, policy_version 123504 (0.0010) [2023-12-26 16:15:55,665][105692] Updated weights for policy 0, policy_version 123514 (0.0010) [2023-12-26 16:15:55,724][105692] Updated weights for policy 0, policy_version 123524 (0.0010) [2023-12-26 16:15:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 63397888. Throughput: 0: 9787.7, 1: 9786.2. Samples: 63408692. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-26 16:15:56,062][104569] Avg episode reward: [(0, '8808.585'), (1, '9009.825')] [2023-12-26 16:15:56,068][105620] Updated weights for policy 1, policy_version 124081 (0.0008) [2023-12-26 16:15:56,112][105620] Updated weights for policy 1, policy_version 124091 (0.0008) [2023-12-26 16:15:56,170][105620] Updated weights for policy 1, policy_version 124101 (0.0008) [2023-12-26 16:15:56,435][105692] Updated weights for policy 0, policy_version 123534 (0.0010) [2023-12-26 16:15:56,487][105692] Updated weights for policy 0, policy_version 123544 (0.0010) [2023-12-26 16:15:56,533][105692] Updated weights for policy 0, policy_version 123554 (0.0010) [2023-12-26 16:15:56,944][105620] Updated weights for policy 1, policy_version 124111 (0.0007) [2023-12-26 16:15:56,997][105620] Updated weights for policy 1, policy_version 124121 (0.0005) [2023-12-26 16:15:57,052][105620] Updated weights for policy 1, policy_version 124131 (0.0005) [2023-12-26 16:15:57,284][105692] Updated weights for policy 0, policy_version 123564 (0.0010) [2023-12-26 16:15:57,343][105692] Updated weights for policy 0, policy_version 123574 (0.0010) [2023-12-26 16:15:57,413][105692] Updated weights for policy 0, policy_version 123584 (0.0008) [2023-12-26 16:15:57,672][105620] Updated weights for policy 1, policy_version 124141 (0.0008) [2023-12-26 16:15:57,719][105620] Updated weights for policy 1, policy_version 124151 (0.0006) [2023-12-26 16:15:57,770][105620] Updated weights for policy 1, policy_version 124161 (0.0010) [2023-12-26 16:15:57,959][105692] Updated weights for policy 0, policy_version 123594 (0.0007) [2023-12-26 16:15:58,026][105692] Updated weights for policy 0, policy_version 123604 (0.0005) [2023-12-26 16:15:58,085][105692] Updated weights for policy 0, policy_version 123614 (0.0006) [2023-12-26 16:15:58,150][105692] Updated weights for policy 0, policy_version 123624 (0.0006) [2023-12-26 16:15:58,413][105620] Updated weights for policy 1, policy_version 124171 (0.0008) [2023-12-26 16:15:58,476][105620] Updated weights for policy 1, policy_version 124181 (0.0008) [2023-12-26 16:15:58,544][105620] Updated weights for policy 1, policy_version 124191 (0.0008) [2023-12-26 16:15:58,913][105692] Updated weights for policy 0, policy_version 123634 (0.0008) [2023-12-26 16:15:58,980][105692] Updated weights for policy 0, policy_version 123644 (0.0007) [2023-12-26 16:15:59,031][105692] Updated weights for policy 0, policy_version 123654 (0.0006) [2023-12-26 16:15:59,402][105620] Updated weights for policy 1, policy_version 124201 (0.0008) [2023-12-26 16:15:59,458][105620] Updated weights for policy 1, policy_version 124211 (0.0009) [2023-12-26 16:15:59,514][105620] Updated weights for policy 1, policy_version 124221 (0.0010) [2023-12-26 16:15:59,583][105620] Updated weights for policy 1, policy_version 124231 (0.0010) [2023-12-26 16:15:59,659][105692] Updated weights for policy 0, policy_version 123664 (0.0005) [2023-12-26 16:15:59,714][105692] Updated weights for policy 0, policy_version 123674 (0.0005) [2023-12-26 16:15:59,767][105692] Updated weights for policy 0, policy_version 123684 (0.0005) [2023-12-26 16:16:00,411][105620] Updated weights for policy 1, policy_version 124241 (0.0008) [2023-12-26 16:16:00,444][105692] Updated weights for policy 0, policy_version 123694 (0.0010) [2023-12-26 16:16:00,462][105620] Updated weights for policy 1, policy_version 124251 (0.0006) [2023-12-26 16:16:00,500][105692] Updated weights for policy 0, policy_version 123704 (0.0011) [2023-12-26 16:16:00,518][105620] Updated weights for policy 1, policy_version 124261 (0.0006) [2023-12-26 16:16:00,559][105692] Updated weights for policy 0, policy_version 123714 (0.0011) [2023-12-26 16:16:01,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 63496192. Throughput: 0: 9818.2, 1: 9842.5. Samples: 63468772. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-26 16:16:01,063][104569] Avg episode reward: [(0, '8376.733'), (1, '8922.863')] [2023-12-26 16:16:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000123720_31678464.pth... [2023-12-26 16:16:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000124264_31817728.pth... [2023-12-26 16:16:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000122600_31391744.pth [2023-12-26 16:16:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000123144_31531008.pth [2023-12-26 16:16:01,140][105692] Updated weights for policy 0, policy_version 123724 (0.0010) [2023-12-26 16:16:01,195][105692] Updated weights for policy 0, policy_version 123734 (0.0006) [2023-12-26 16:16:01,259][105692] Updated weights for policy 0, policy_version 123744 (0.0006) [2023-12-26 16:16:01,296][105620] Updated weights for policy 1, policy_version 124271 (0.0009) [2023-12-26 16:16:01,363][105620] Updated weights for policy 1, policy_version 124281 (0.0011) [2023-12-26 16:16:01,425][105620] Updated weights for policy 1, policy_version 124291 (0.0010) [2023-12-26 16:16:01,971][105692] Updated weights for policy 0, policy_version 123754 (0.0008) [2023-12-26 16:16:02,030][105692] Updated weights for policy 0, policy_version 123764 (0.0006) [2023-12-26 16:16:02,093][105692] Updated weights for policy 0, policy_version 123774 (0.0009) [2023-12-26 16:16:02,118][105620] Updated weights for policy 1, policy_version 124301 (0.0008) [2023-12-26 16:16:02,142][105692] Updated weights for policy 0, policy_version 123784 (0.0010) [2023-12-26 16:16:02,169][105620] Updated weights for policy 1, policy_version 124311 (0.0010) [2023-12-26 16:16:02,227][105620] Updated weights for policy 1, policy_version 124321 (0.0010) [2023-12-26 16:16:02,711][105692] Updated weights for policy 0, policy_version 123794 (0.0005) [2023-12-26 16:16:02,765][105692] Updated weights for policy 0, policy_version 123804 (0.0009) [2023-12-26 16:16:02,820][105692] Updated weights for policy 0, policy_version 123814 (0.0010) [2023-12-26 16:16:02,968][105620] Updated weights for policy 1, policy_version 124331 (0.0009) [2023-12-26 16:16:03,034][105620] Updated weights for policy 1, policy_version 124341 (0.0005) [2023-12-26 16:16:03,109][105620] Updated weights for policy 1, policy_version 124351 (0.0005) [2023-12-26 16:16:03,471][105692] Updated weights for policy 0, policy_version 123824 (0.0005) [2023-12-26 16:16:03,530][105692] Updated weights for policy 0, policy_version 123834 (0.0009) [2023-12-26 16:16:03,585][105692] Updated weights for policy 0, policy_version 123844 (0.0008) [2023-12-26 16:16:03,752][105620] Updated weights for policy 1, policy_version 124361 (0.0006) [2023-12-26 16:16:03,817][105620] Updated weights for policy 1, policy_version 124371 (0.0010) [2023-12-26 16:16:03,881][105620] Updated weights for policy 1, policy_version 124381 (0.0008) [2023-12-26 16:16:03,946][105620] Updated weights for policy 1, policy_version 124391 (0.0010) [2023-12-26 16:16:04,290][105692] Updated weights for policy 0, policy_version 123854 (0.0010) [2023-12-26 16:16:04,356][105692] Updated weights for policy 0, policy_version 123864 (0.0011) [2023-12-26 16:16:04,423][105692] Updated weights for policy 0, policy_version 123874 (0.0011) [2023-12-26 16:16:04,625][105620] Updated weights for policy 1, policy_version 124401 (0.0010) [2023-12-26 16:16:04,682][105620] Updated weights for policy 1, policy_version 124411 (0.0011) [2023-12-26 16:16:04,749][105620] Updated weights for policy 1, policy_version 124421 (0.0011) [2023-12-26 16:16:05,149][105692] Updated weights for policy 0, policy_version 123884 (0.0009) [2023-12-26 16:16:05,202][105692] Updated weights for policy 0, policy_version 123894 (0.0006) [2023-12-26 16:16:05,265][105692] Updated weights for policy 0, policy_version 123904 (0.0008) [2023-12-26 16:16:05,503][105620] Updated weights for policy 1, policy_version 124431 (0.0011) [2023-12-26 16:16:05,548][105620] Updated weights for policy 1, policy_version 124441 (0.0010) [2023-12-26 16:16:05,597][105620] Updated weights for policy 1, policy_version 124451 (0.0010) [2023-12-26 16:16:05,822][105692] Updated weights for policy 0, policy_version 123914 (0.0009) [2023-12-26 16:16:05,870][105692] Updated weights for policy 0, policy_version 123924 (0.0005) [2023-12-26 16:16:05,918][105692] Updated weights for policy 0, policy_version 123934 (0.0005) [2023-12-26 16:16:05,971][105692] Updated weights for policy 0, policy_version 123944 (0.0005) [2023-12-26 16:16:06,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 63602688. Throughput: 0: 9820.0, 1: 9754.3. Samples: 63586940. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-26 16:16:06,063][104569] Avg episode reward: [(0, '8121.862'), (1, '8741.638')] [2023-12-26 16:16:06,357][105620] Updated weights for policy 1, policy_version 124461 (0.0010) [2023-12-26 16:16:06,424][105620] Updated weights for policy 1, policy_version 124471 (0.0011) [2023-12-26 16:16:06,483][105620] Updated weights for policy 1, policy_version 124481 (0.0011) [2023-12-26 16:16:06,625][105692] Updated weights for policy 0, policy_version 123954 (0.0006) [2023-12-26 16:16:06,685][105692] Updated weights for policy 0, policy_version 123964 (0.0007) [2023-12-26 16:16:06,743][105692] Updated weights for policy 0, policy_version 123974 (0.0008) [2023-12-26 16:16:07,230][105620] Updated weights for policy 1, policy_version 124491 (0.0010) [2023-12-26 16:16:07,293][105620] Updated weights for policy 1, policy_version 124501 (0.0010) [2023-12-26 16:16:07,319][105692] Updated weights for policy 0, policy_version 123984 (0.0006) [2023-12-26 16:16:07,356][105620] Updated weights for policy 1, policy_version 124511 (0.0010) [2023-12-26 16:16:07,377][105692] Updated weights for policy 0, policy_version 123994 (0.0005) [2023-12-26 16:16:07,439][105692] Updated weights for policy 0, policy_version 124004 (0.0005) [2023-12-26 16:16:07,922][105620] Updated weights for policy 1, policy_version 124521 (0.0010) [2023-12-26 16:16:07,978][105620] Updated weights for policy 1, policy_version 124531 (0.0011) [2023-12-26 16:16:08,037][105620] Updated weights for policy 1, policy_version 124541 (0.0010) [2023-12-26 16:16:08,088][105620] Updated weights for policy 1, policy_version 124551 (0.0010) [2023-12-26 16:16:08,099][105692] Updated weights for policy 0, policy_version 124014 (0.0006) [2023-12-26 16:16:08,159][105692] Updated weights for policy 0, policy_version 124024 (0.0007) [2023-12-26 16:16:08,219][105692] Updated weights for policy 0, policy_version 124034 (0.0008) [2023-12-26 16:16:08,789][105620] Updated weights for policy 1, policy_version 124561 (0.0006) [2023-12-26 16:16:08,842][105620] Updated weights for policy 1, policy_version 124571 (0.0005) [2023-12-26 16:16:08,894][105620] Updated weights for policy 1, policy_version 124581 (0.0006) [2023-12-26 16:16:09,070][105692] Updated weights for policy 0, policy_version 124044 (0.0009) [2023-12-26 16:16:09,122][105692] Updated weights for policy 0, policy_version 124054 (0.0007) [2023-12-26 16:16:09,176][105692] Updated weights for policy 0, policy_version 124064 (0.0010) [2023-12-26 16:16:09,494][105620] Updated weights for policy 1, policy_version 124591 (0.0007) [2023-12-26 16:16:09,560][105620] Updated weights for policy 1, policy_version 124601 (0.0008) [2023-12-26 16:16:09,623][105620] Updated weights for policy 1, policy_version 124611 (0.0011) [2023-12-26 16:16:09,977][105692] Updated weights for policy 0, policy_version 124075 (0.0010) [2023-12-26 16:16:10,035][105692] Updated weights for policy 0, policy_version 124085 (0.0008) [2023-12-26 16:16:10,105][105692] Updated weights for policy 0, policy_version 124095 (0.0008) [2023-12-26 16:16:10,383][105620] Updated weights for policy 1, policy_version 124621 (0.0009) [2023-12-26 16:16:10,441][105620] Updated weights for policy 1, policy_version 124631 (0.0007) [2023-12-26 16:16:10,500][105620] Updated weights for policy 1, policy_version 124641 (0.0007) [2023-12-26 16:16:10,925][105692] Updated weights for policy 0, policy_version 124105 (0.0010) [2023-12-26 16:16:10,983][105692] Updated weights for policy 0, policy_version 124115 (0.0009) [2023-12-26 16:16:11,050][105692] Updated weights for policy 0, policy_version 124125 (0.0010) [2023-12-26 16:16:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 63692800. Throughput: 0: 9922.7, 1: 9780.8. Samples: 63706976. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-26 16:16:11,062][104569] Avg episode reward: [(0, '8221.045'), (1, '8739.874')] [2023-12-26 16:16:11,111][105692] Updated weights for policy 0, policy_version 124135 (0.0009) [2023-12-26 16:16:11,154][105620] Updated weights for policy 1, policy_version 124651 (0.0007) [2023-12-26 16:16:11,214][105620] Updated weights for policy 1, policy_version 124661 (0.0008) [2023-12-26 16:16:11,272][105620] Updated weights for policy 1, policy_version 124671 (0.0009) [2023-12-26 16:16:11,940][105620] Updated weights for policy 1, policy_version 124681 (0.0010) [2023-12-26 16:16:12,006][105620] Updated weights for policy 1, policy_version 124691 (0.0007) [2023-12-26 16:16:12,008][105692] Updated weights for policy 0, policy_version 124145 (0.0007) [2023-12-26 16:16:12,064][105692] Updated weights for policy 0, policy_version 124155 (0.0005) [2023-12-26 16:16:12,073][105620] Updated weights for policy 1, policy_version 124701 (0.0009) [2023-12-26 16:16:12,119][105692] Updated weights for policy 0, policy_version 124165 (0.0008) [2023-12-26 16:16:12,130][105620] Updated weights for policy 1, policy_version 124711 (0.0007) [2023-12-26 16:16:12,882][105620] Updated weights for policy 1, policy_version 124721 (0.0005) [2023-12-26 16:16:12,932][105620] Updated weights for policy 1, policy_version 124731 (0.0005) [2023-12-26 16:16:12,962][105692] Updated weights for policy 0, policy_version 124175 (0.0010) [2023-12-26 16:16:12,985][105620] Updated weights for policy 1, policy_version 124741 (0.0006) [2023-12-26 16:16:13,031][105692] Updated weights for policy 0, policy_version 124185 (0.0009) [2023-12-26 16:16:13,106][105692] Updated weights for policy 0, policy_version 124195 (0.0010) [2023-12-26 16:16:13,646][105620] Updated weights for policy 1, policy_version 124751 (0.0008) [2023-12-26 16:16:13,700][105620] Updated weights for policy 1, policy_version 124761 (0.0009) [2023-12-26 16:16:13,758][105620] Updated weights for policy 1, policy_version 124771 (0.0009) [2023-12-26 16:16:13,850][105692] Updated weights for policy 0, policy_version 124205 (0.0009) [2023-12-26 16:16:13,912][105692] Updated weights for policy 0, policy_version 124215 (0.0009) [2023-12-26 16:16:13,970][105692] Updated weights for policy 0, policy_version 124225 (0.0009) [2023-12-26 16:16:14,507][105620] Updated weights for policy 1, policy_version 124781 (0.0008) [2023-12-26 16:16:14,573][105620] Updated weights for policy 1, policy_version 124791 (0.0006) [2023-12-26 16:16:14,644][105620] Updated weights for policy 1, policy_version 124801 (0.0005) [2023-12-26 16:16:14,750][105692] Updated weights for policy 0, policy_version 124235 (0.0009) [2023-12-26 16:16:14,817][105692] Updated weights for policy 0, policy_version 124245 (0.0009) [2023-12-26 16:16:14,881][105692] Updated weights for policy 0, policy_version 124255 (0.0009) [2023-12-26 16:16:15,242][105620] Updated weights for policy 1, policy_version 124811 (0.0009) [2023-12-26 16:16:15,298][105620] Updated weights for policy 1, policy_version 124821 (0.0010) [2023-12-26 16:16:15,356][105620] Updated weights for policy 1, policy_version 124831 (0.0008) [2023-12-26 16:16:15,707][105692] Updated weights for policy 0, policy_version 124265 (0.0010) [2023-12-26 16:16:15,763][105692] Updated weights for policy 0, policy_version 124275 (0.0009) [2023-12-26 16:16:15,819][105692] Updated weights for policy 0, policy_version 124285 (0.0010) [2023-12-26 16:16:15,875][105692] Updated weights for policy 0, policy_version 124295 (0.0011) [2023-12-26 16:16:16,015][105620] Updated weights for policy 1, policy_version 124841 (0.0009) [2023-12-26 16:16:16,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 63791104. Throughput: 0: 9700.2, 1: 9742.9. Samples: 63761872. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-26 16:16:16,062][104569] Avg episode reward: [(0, '8479.881'), (1, '8152.469')] [2023-12-26 16:16:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000124296_31825920.pth... [2023-12-26 16:16:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000123144_31531008.pth [2023-12-26 16:16:16,076][105620] Updated weights for policy 1, policy_version 124851 (0.0009) [2023-12-26 16:16:16,131][105620] Updated weights for policy 1, policy_version 124861 (0.0010) [2023-12-26 16:16:16,197][105620] Updated weights for policy 1, policy_version 124871 (0.0010) [2023-12-26 16:16:16,200][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000124872_31973376.pth... [2023-12-26 16:16:16,204][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000123688_31670272.pth [2023-12-26 16:16:16,671][105692] Updated weights for policy 0, policy_version 124305 (0.0008) [2023-12-26 16:16:16,727][105692] Updated weights for policy 0, policy_version 124315 (0.0008) [2023-12-26 16:16:16,787][105692] Updated weights for policy 0, policy_version 124325 (0.0008) [2023-12-26 16:16:16,926][105620] Updated weights for policy 1, policy_version 124881 (0.0010) [2023-12-26 16:16:16,984][105620] Updated weights for policy 1, policy_version 124891 (0.0010) [2023-12-26 16:16:17,045][105620] Updated weights for policy 1, policy_version 124901 (0.0010) [2023-12-26 16:16:17,551][105692] Updated weights for policy 0, policy_version 124335 (0.0008) [2023-12-26 16:16:17,596][105692] Updated weights for policy 0, policy_version 124345 (0.0008) [2023-12-26 16:16:17,645][105692] Updated weights for policy 0, policy_version 124355 (0.0008) [2023-12-26 16:16:17,773][105620] Updated weights for policy 1, policy_version 124911 (0.0010) [2023-12-26 16:16:17,821][105620] Updated weights for policy 1, policy_version 124921 (0.0010) [2023-12-26 16:16:17,870][105620] Updated weights for policy 1, policy_version 124931 (0.0010) [2023-12-26 16:16:18,445][105692] Updated weights for policy 0, policy_version 124365 (0.0008) [2023-12-26 16:16:18,497][105692] Updated weights for policy 0, policy_version 124375 (0.0008) [2023-12-26 16:16:18,546][105692] Updated weights for policy 0, policy_version 124385 (0.0008) [2023-12-26 16:16:18,639][105620] Updated weights for policy 1, policy_version 124941 (0.0010) [2023-12-26 16:16:18,695][105620] Updated weights for policy 1, policy_version 124951 (0.0010) [2023-12-26 16:16:18,752][105620] Updated weights for policy 1, policy_version 124961 (0.0009) [2023-12-26 16:16:19,367][105692] Updated weights for policy 0, policy_version 124395 (0.0008) [2023-12-26 16:16:19,416][105692] Updated weights for policy 0, policy_version 124405 (0.0008) [2023-12-26 16:16:19,455][105620] Updated weights for policy 1, policy_version 124971 (0.0010) [2023-12-26 16:16:19,466][105692] Updated weights for policy 0, policy_version 124415 (0.0007) [2023-12-26 16:16:19,517][105620] Updated weights for policy 1, policy_version 124981 (0.0008) [2023-12-26 16:16:19,569][105620] Updated weights for policy 1, policy_version 124991 (0.0008) [2023-12-26 16:16:20,302][105620] Updated weights for policy 1, policy_version 125001 (0.0008) [2023-12-26 16:16:20,329][105692] Updated weights for policy 0, policy_version 124425 (0.0007) [2023-12-26 16:16:20,358][105620] Updated weights for policy 1, policy_version 125011 (0.0011) [2023-12-26 16:16:20,388][105692] Updated weights for policy 0, policy_version 124435 (0.0006) [2023-12-26 16:16:20,421][105620] Updated weights for policy 1, policy_version 125021 (0.0011) [2023-12-26 16:16:20,444][105692] Updated weights for policy 0, policy_version 124445 (0.0009) [2023-12-26 16:16:20,482][105620] Updated weights for policy 1, policy_version 125031 (0.0007) [2023-12-26 16:16:20,496][105692] Updated weights for policy 0, policy_version 124455 (0.0009) [2023-12-26 16:16:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 63881216. Throughput: 0: 9619.8, 1: 9713.0. Samples: 63874568. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-26 16:16:21,063][104569] Avg episode reward: [(0, '8087.095'), (1, '7783.684')] [2023-12-26 16:16:21,173][105620] Updated weights for policy 1, policy_version 125041 (0.0009) [2023-12-26 16:16:21,224][105620] Updated weights for policy 1, policy_version 125051 (0.0007) [2023-12-26 16:16:21,286][105620] Updated weights for policy 1, policy_version 125061 (0.0006) [2023-12-26 16:16:21,367][105692] Updated weights for policy 0, policy_version 124465 (0.0010) [2023-12-26 16:16:21,425][105692] Updated weights for policy 0, policy_version 124475 (0.0008) [2023-12-26 16:16:21,484][105692] Updated weights for policy 0, policy_version 124485 (0.0005) [2023-12-26 16:16:22,011][105620] Updated weights for policy 1, policy_version 125071 (0.0009) [2023-12-26 16:16:22,063][105620] Updated weights for policy 1, policy_version 125081 (0.0009) [2023-12-26 16:16:22,113][105620] Updated weights for policy 1, policy_version 125091 (0.0009) [2023-12-26 16:16:22,217][105692] Updated weights for policy 0, policy_version 124495 (0.0008) [2023-12-26 16:16:22,281][105692] Updated weights for policy 0, policy_version 124505 (0.0010) [2023-12-26 16:16:22,344][105692] Updated weights for policy 0, policy_version 124515 (0.0009) [2023-12-26 16:16:22,903][105620] Updated weights for policy 1, policy_version 125101 (0.0009) [2023-12-26 16:16:22,957][105620] Updated weights for policy 1, policy_version 125111 (0.0008) [2023-12-26 16:16:23,022][105620] Updated weights for policy 1, policy_version 125121 (0.0008) [2023-12-26 16:16:23,156][105692] Updated weights for policy 0, policy_version 124525 (0.0009) [2023-12-26 16:16:23,219][105692] Updated weights for policy 0, policy_version 124535 (0.0009) [2023-12-26 16:16:23,282][105692] Updated weights for policy 0, policy_version 124545 (0.0009) [2023-12-26 16:16:23,760][105620] Updated weights for policy 1, policy_version 125131 (0.0009) [2023-12-26 16:16:23,827][105620] Updated weights for policy 1, policy_version 125141 (0.0009) [2023-12-26 16:16:23,891][105620] Updated weights for policy 1, policy_version 125151 (0.0009) [2023-12-26 16:16:24,060][105692] Updated weights for policy 0, policy_version 124555 (0.0009) [2023-12-26 16:16:24,115][105692] Updated weights for policy 0, policy_version 124565 (0.0009) [2023-12-26 16:16:24,167][105692] Updated weights for policy 0, policy_version 124575 (0.0009) [2023-12-26 16:16:24,601][105620] Updated weights for policy 1, policy_version 125161 (0.0008) [2023-12-26 16:16:24,655][105620] Updated weights for policy 1, policy_version 125171 (0.0009) [2023-12-26 16:16:24,712][105620] Updated weights for policy 1, policy_version 125181 (0.0008) [2023-12-26 16:16:24,759][105620] Updated weights for policy 1, policy_version 125191 (0.0009) [2023-12-26 16:16:24,940][105692] Updated weights for policy 0, policy_version 124585 (0.0010) [2023-12-26 16:16:24,996][105692] Updated weights for policy 0, policy_version 124595 (0.0009) [2023-12-26 16:16:25,050][105692] Updated weights for policy 0, policy_version 124606 (0.0010) [2023-12-26 16:16:25,103][105692] Updated weights for policy 0, policy_version 124616 (0.0009) [2023-12-26 16:16:25,379][105620] Updated weights for policy 1, policy_version 125201 (0.0006) [2023-12-26 16:16:25,448][105620] Updated weights for policy 1, policy_version 125211 (0.0005) [2023-12-26 16:16:25,502][105620] Updated weights for policy 1, policy_version 125221 (0.0005) [2023-12-26 16:16:25,721][105692] Updated weights for policy 0, policy_version 124626 (0.0005) [2023-12-26 16:16:25,777][105692] Updated weights for policy 0, policy_version 124636 (0.0005) [2023-12-26 16:16:25,835][105692] Updated weights for policy 0, policy_version 124646 (0.0008) [2023-12-26 16:16:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 63979520. Throughput: 0: 9593.6, 1: 9776.8. Samples: 63987924. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-26 16:16:26,062][104569] Avg episode reward: [(0, '8257.551'), (1, '8034.891')] [2023-12-26 16:16:26,169][105620] Updated weights for policy 1, policy_version 125231 (0.0005) [2023-12-26 16:16:26,219][105620] Updated weights for policy 1, policy_version 125241 (0.0005) [2023-12-26 16:16:26,268][105620] Updated weights for policy 1, policy_version 125251 (0.0008) [2023-12-26 16:16:26,503][105692] Updated weights for policy 0, policy_version 124656 (0.0010) [2023-12-26 16:16:26,551][105692] Updated weights for policy 0, policy_version 124666 (0.0010) [2023-12-26 16:16:26,602][105692] Updated weights for policy 0, policy_version 124676 (0.0010) [2023-12-26 16:16:26,995][105620] Updated weights for policy 1, policy_version 125261 (0.0008) [2023-12-26 16:16:27,046][105620] Updated weights for policy 1, policy_version 125271 (0.0008) [2023-12-26 16:16:27,101][105620] Updated weights for policy 1, policy_version 125281 (0.0007) [2023-12-26 16:16:27,315][105692] Updated weights for policy 0, policy_version 124686 (0.0010) [2023-12-26 16:16:27,360][105692] Updated weights for policy 0, policy_version 124696 (0.0010) [2023-12-26 16:16:27,404][105692] Updated weights for policy 0, policy_version 124706 (0.0010) [2023-12-26 16:16:27,854][105620] Updated weights for policy 1, policy_version 125291 (0.0008) [2023-12-26 16:16:27,912][105620] Updated weights for policy 1, policy_version 125301 (0.0008) [2023-12-26 16:16:27,969][105620] Updated weights for policy 1, policy_version 125311 (0.0009) [2023-12-26 16:16:28,135][105692] Updated weights for policy 0, policy_version 124716 (0.0008) [2023-12-26 16:16:28,191][105692] Updated weights for policy 0, policy_version 124726 (0.0005) [2023-12-26 16:16:28,253][105692] Updated weights for policy 0, policy_version 124736 (0.0005) [2023-12-26 16:16:28,756][105620] Updated weights for policy 1, policy_version 125322 (0.0009) [2023-12-26 16:16:28,815][105620] Updated weights for policy 1, policy_version 125332 (0.0008) [2023-12-26 16:16:28,871][105620] Updated weights for policy 1, policy_version 125342 (0.0008) [2023-12-26 16:16:28,905][105692] Updated weights for policy 0, policy_version 124746 (0.0006) [2023-12-26 16:16:28,935][105620] Updated weights for policy 1, policy_version 125352 (0.0007) [2023-12-26 16:16:28,960][105692] Updated weights for policy 0, policy_version 124756 (0.0010) [2023-12-26 16:16:29,011][105692] Updated weights for policy 0, policy_version 124766 (0.0010) [2023-12-26 16:16:29,058][105692] Updated weights for policy 0, policy_version 124776 (0.0010) [2023-12-26 16:16:29,683][105620] Updated weights for policy 1, policy_version 125362 (0.0010) [2023-12-26 16:16:29,733][105620] Updated weights for policy 1, policy_version 125372 (0.0010) [2023-12-26 16:16:29,795][105620] Updated weights for policy 1, policy_version 125382 (0.0010) [2023-12-26 16:16:29,841][105692] Updated weights for policy 0, policy_version 124786 (0.0007) [2023-12-26 16:16:29,904][105692] Updated weights for policy 0, policy_version 124796 (0.0006) [2023-12-26 16:16:29,973][105692] Updated weights for policy 0, policy_version 124806 (0.0009) [2023-12-26 16:16:30,461][105620] Updated weights for policy 1, policy_version 125392 (0.0009) [2023-12-26 16:16:30,506][105620] Updated weights for policy 1, policy_version 125402 (0.0010) [2023-12-26 16:16:30,551][105620] Updated weights for policy 1, policy_version 125412 (0.0010) [2023-12-26 16:16:30,715][105692] Updated weights for policy 0, policy_version 124816 (0.0009) [2023-12-26 16:16:30,776][105692] Updated weights for policy 0, policy_version 124826 (0.0007) [2023-12-26 16:16:30,840][105692] Updated weights for policy 0, policy_version 124836 (0.0008) [2023-12-26 16:16:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 64077824. Throughput: 0: 9649.3, 1: 9811.1. Samples: 64046940. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-26 16:16:31,063][104569] Avg episode reward: [(0, '8716.646'), (1, '8666.215')] [2023-12-26 16:16:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000124840_31965184.pth... [2023-12-26 16:16:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000125416_32112640.pth... [2023-12-26 16:16:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000123720_31678464.pth [2023-12-26 16:16:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000124264_31817728.pth [2023-12-26 16:16:31,290][105620] Updated weights for policy 1, policy_version 125422 (0.0010) [2023-12-26 16:16:31,343][105620] Updated weights for policy 1, policy_version 125432 (0.0007) [2023-12-26 16:16:31,401][105620] Updated weights for policy 1, policy_version 125442 (0.0009) [2023-12-26 16:16:31,616][105692] Updated weights for policy 0, policy_version 124846 (0.0008) [2023-12-26 16:16:31,678][105692] Updated weights for policy 0, policy_version 124856 (0.0007) [2023-12-26 16:16:31,741][105692] Updated weights for policy 0, policy_version 124866 (0.0008) [2023-12-26 16:16:32,183][105620] Updated weights for policy 1, policy_version 125452 (0.0010) [2023-12-26 16:16:32,245][105620] Updated weights for policy 1, policy_version 125462 (0.0009) [2023-12-26 16:16:32,322][105620] Updated weights for policy 1, policy_version 125472 (0.0010) [2023-12-26 16:16:32,452][105692] Updated weights for policy 0, policy_version 124876 (0.0010) [2023-12-26 16:16:32,513][105692] Updated weights for policy 0, policy_version 124886 (0.0009) [2023-12-26 16:16:32,571][105692] Updated weights for policy 0, policy_version 124896 (0.0009) [2023-12-26 16:16:33,055][105620] Updated weights for policy 1, policy_version 125482 (0.0009) [2023-12-26 16:16:33,101][105620] Updated weights for policy 1, policy_version 125492 (0.0009) [2023-12-26 16:16:33,147][105620] Updated weights for policy 1, policy_version 125502 (0.0008) [2023-12-26 16:16:33,206][105620] Updated weights for policy 1, policy_version 125512 (0.0009) [2023-12-26 16:16:33,322][105692] Updated weights for policy 0, policy_version 124906 (0.0009) [2023-12-26 16:16:33,372][105692] Updated weights for policy 0, policy_version 124916 (0.0005) [2023-12-26 16:16:33,419][105692] Updated weights for policy 0, policy_version 124926 (0.0005) [2023-12-26 16:16:33,473][105692] Updated weights for policy 0, policy_version 124936 (0.0005) [2023-12-26 16:16:34,036][105620] Updated weights for policy 1, policy_version 125522 (0.0009) [2023-12-26 16:16:34,088][105620] Updated weights for policy 1, policy_version 125532 (0.0008) [2023-12-26 16:16:34,100][105692] Updated weights for policy 0, policy_version 124946 (0.0009) [2023-12-26 16:16:34,156][105620] Updated weights for policy 1, policy_version 125542 (0.0008) [2023-12-26 16:16:34,160][105692] Updated weights for policy 0, policy_version 124956 (0.0011) [2023-12-26 16:16:34,215][105692] Updated weights for policy 0, policy_version 124966 (0.0010) [2023-12-26 16:16:34,909][105692] Updated weights for policy 0, policy_version 124976 (0.0011) [2023-12-26 16:16:34,927][105620] Updated weights for policy 1, policy_version 125552 (0.0008) [2023-12-26 16:16:34,968][105692] Updated weights for policy 0, policy_version 124986 (0.0010) [2023-12-26 16:16:34,987][105620] Updated weights for policy 1, policy_version 125562 (0.0005) [2023-12-26 16:16:35,038][105692] Updated weights for policy 0, policy_version 124996 (0.0011) [2023-12-26 16:16:35,049][105620] Updated weights for policy 1, policy_version 125572 (0.0005) [2023-12-26 16:16:35,594][105620] Updated weights for policy 1, policy_version 125582 (0.0005) [2023-12-26 16:16:35,656][105620] Updated weights for policy 1, policy_version 125592 (0.0006) [2023-12-26 16:16:35,720][105620] Updated weights for policy 1, policy_version 125602 (0.0005) [2023-12-26 16:16:35,771][105692] Updated weights for policy 0, policy_version 125006 (0.0010) [2023-12-26 16:16:35,832][105692] Updated weights for policy 0, policy_version 125016 (0.0010) [2023-12-26 16:16:35,890][105692] Updated weights for policy 0, policy_version 125026 (0.0010) [2023-12-26 16:16:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 64176128. Throughput: 0: 9584.2, 1: 9713.0. Samples: 64160476. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-26 16:16:36,062][104569] Avg episode reward: [(0, '8897.715'), (1, '8714.246')] [2023-12-26 16:16:36,293][105620] Updated weights for policy 1, policy_version 125612 (0.0009) [2023-12-26 16:16:36,353][105620] Updated weights for policy 1, policy_version 125622 (0.0007) [2023-12-26 16:16:36,416][105620] Updated weights for policy 1, policy_version 125632 (0.0005) [2023-12-26 16:16:36,669][105692] Updated weights for policy 0, policy_version 125036 (0.0011) [2023-12-26 16:16:36,733][105692] Updated weights for policy 0, policy_version 125046 (0.0011) [2023-12-26 16:16:36,802][105692] Updated weights for policy 0, policy_version 125056 (0.0011) [2023-12-26 16:16:37,012][105620] Updated weights for policy 1, policy_version 125642 (0.0006) [2023-12-26 16:16:37,070][105620] Updated weights for policy 1, policy_version 125652 (0.0006) [2023-12-26 16:16:37,123][105620] Updated weights for policy 1, policy_version 125662 (0.0011) [2023-12-26 16:16:37,175][105620] Updated weights for policy 1, policy_version 125672 (0.0010) [2023-12-26 16:16:37,419][105692] Updated weights for policy 0, policy_version 125066 (0.0008) [2023-12-26 16:16:37,478][105692] Updated weights for policy 0, policy_version 125076 (0.0011) [2023-12-26 16:16:37,537][105692] Updated weights for policy 0, policy_version 125086 (0.0011) [2023-12-26 16:16:37,604][105692] Updated weights for policy 0, policy_version 125096 (0.0010) [2023-12-26 16:16:37,909][105620] Updated weights for policy 1, policy_version 125682 (0.0005) [2023-12-26 16:16:37,970][105620] Updated weights for policy 1, policy_version 125692 (0.0006) [2023-12-26 16:16:38,032][105620] Updated weights for policy 1, policy_version 125702 (0.0009) [2023-12-26 16:16:38,280][105692] Updated weights for policy 0, policy_version 125106 (0.0011) [2023-12-26 16:16:38,336][105692] Updated weights for policy 0, policy_version 125116 (0.0006) [2023-12-26 16:16:38,396][105692] Updated weights for policy 0, policy_version 125126 (0.0007) [2023-12-26 16:16:38,810][105620] Updated weights for policy 1, policy_version 125712 (0.0008) [2023-12-26 16:16:38,869][105620] Updated weights for policy 1, policy_version 125722 (0.0008) [2023-12-26 16:16:38,918][105620] Updated weights for policy 1, policy_version 125732 (0.0007) [2023-12-26 16:16:39,121][105692] Updated weights for policy 0, policy_version 125136 (0.0009) [2023-12-26 16:16:39,176][105692] Updated weights for policy 0, policy_version 125146 (0.0010) [2023-12-26 16:16:39,243][105692] Updated weights for policy 0, policy_version 125156 (0.0011) [2023-12-26 16:16:39,623][105620] Updated weights for policy 1, policy_version 125742 (0.0007) [2023-12-26 16:16:39,683][105620] Updated weights for policy 1, policy_version 125752 (0.0006) [2023-12-26 16:16:39,745][105620] Updated weights for policy 1, policy_version 125762 (0.0006) [2023-12-26 16:16:39,988][105692] Updated weights for policy 0, policy_version 125166 (0.0007) [2023-12-26 16:16:40,048][105692] Updated weights for policy 0, policy_version 125176 (0.0011) [2023-12-26 16:16:40,109][105692] Updated weights for policy 0, policy_version 125186 (0.0011) [2023-12-26 16:16:40,513][105620] Updated weights for policy 1, policy_version 125772 (0.0006) [2023-12-26 16:16:40,574][105620] Updated weights for policy 1, policy_version 125782 (0.0009) [2023-12-26 16:16:40,636][105620] Updated weights for policy 1, policy_version 125792 (0.0011) [2023-12-26 16:16:40,805][105692] Updated weights for policy 0, policy_version 125196 (0.0009) [2023-12-26 16:16:40,864][105692] Updated weights for policy 0, policy_version 125206 (0.0010) [2023-12-26 16:16:40,919][105692] Updated weights for policy 0, policy_version 125216 (0.0010) [2023-12-26 16:16:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 64274432. Throughput: 0: 9605.8, 1: 9753.3. Samples: 64279852. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-26 16:16:41,063][104569] Avg episode reward: [(0, '9170.552'), (1, '8709.179')] [2023-12-26 16:16:41,384][105620] Updated weights for policy 1, policy_version 125802 (0.0010) [2023-12-26 16:16:41,447][105620] Updated weights for policy 1, policy_version 125812 (0.0010) [2023-12-26 16:16:41,516][105620] Updated weights for policy 1, policy_version 125822 (0.0011) [2023-12-26 16:16:41,583][105620] Updated weights for policy 1, policy_version 125832 (0.0011) [2023-12-26 16:16:41,651][105692] Updated weights for policy 0, policy_version 125226 (0.0009) [2023-12-26 16:16:41,710][105692] Updated weights for policy 0, policy_version 125236 (0.0011) [2023-12-26 16:16:41,780][105692] Updated weights for policy 0, policy_version 125246 (0.0008) [2023-12-26 16:16:41,844][105692] Updated weights for policy 0, policy_version 125256 (0.0007) [2023-12-26 16:16:42,316][105620] Updated weights for policy 1, policy_version 125842 (0.0009) [2023-12-26 16:16:42,385][105620] Updated weights for policy 1, policy_version 125852 (0.0009) [2023-12-26 16:16:42,435][105620] Updated weights for policy 1, policy_version 125862 (0.0008) [2023-12-26 16:16:42,506][105692] Updated weights for policy 0, policy_version 125266 (0.0010) [2023-12-26 16:16:42,564][105692] Updated weights for policy 0, policy_version 125276 (0.0009) [2023-12-26 16:16:42,627][105692] Updated weights for policy 0, policy_version 125286 (0.0009) [2023-12-26 16:16:43,233][105692] Updated weights for policy 0, policy_version 125296 (0.0009) [2023-12-26 16:16:43,291][105620] Updated weights for policy 1, policy_version 125872 (0.0007) [2023-12-26 16:16:43,293][105692] Updated weights for policy 0, policy_version 125306 (0.0009) [2023-12-26 16:16:43,339][105620] Updated weights for policy 1, policy_version 125882 (0.0006) [2023-12-26 16:16:43,353][105692] Updated weights for policy 0, policy_version 125316 (0.0008) [2023-12-26 16:16:43,393][105620] Updated weights for policy 1, policy_version 125892 (0.0007) [2023-12-26 16:16:44,024][105620] Updated weights for policy 1, policy_version 125902 (0.0007) [2023-12-26 16:16:44,071][105692] Updated weights for policy 0, policy_version 125326 (0.0007) [2023-12-26 16:16:44,085][105620] Updated weights for policy 1, policy_version 125912 (0.0007) [2023-12-26 16:16:44,128][105692] Updated weights for policy 0, policy_version 125336 (0.0007) [2023-12-26 16:16:44,142][105620] Updated weights for policy 1, policy_version 125922 (0.0007) [2023-12-26 16:16:44,187][105692] Updated weights for policy 0, policy_version 125346 (0.0007) [2023-12-26 16:16:44,719][105620] Updated weights for policy 1, policy_version 125932 (0.0008) [2023-12-26 16:16:44,776][105620] Updated weights for policy 1, policy_version 125942 (0.0006) [2023-12-26 16:16:44,841][105620] Updated weights for policy 1, policy_version 125952 (0.0008) [2023-12-26 16:16:44,969][105692] Updated weights for policy 0, policy_version 125356 (0.0009) [2023-12-26 16:16:45,039][105692] Updated weights for policy 0, policy_version 125366 (0.0005) [2023-12-26 16:16:45,106][105692] Updated weights for policy 0, policy_version 125376 (0.0005) [2023-12-26 16:16:45,596][105620] Updated weights for policy 1, policy_version 125962 (0.0008) [2023-12-26 16:16:45,643][105620] Updated weights for policy 1, policy_version 125972 (0.0009) [2023-12-26 16:16:45,697][105620] Updated weights for policy 1, policy_version 125982 (0.0009) [2023-12-26 16:16:45,726][105692] Updated weights for policy 0, policy_version 125386 (0.0008) [2023-12-26 16:16:45,745][105620] Updated weights for policy 1, policy_version 125992 (0.0007) [2023-12-26 16:16:45,779][105692] Updated weights for policy 0, policy_version 125396 (0.0008) [2023-12-26 16:16:45,830][105692] Updated weights for policy 0, policy_version 125406 (0.0010) [2023-12-26 16:16:45,892][105692] Updated weights for policy 0, policy_version 125416 (0.0010) [2023-12-26 16:16:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 64372736. Throughput: 0: 9587.1, 1: 9720.1. Samples: 64337596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:16:46,062][104569] Avg episode reward: [(0, '9078.986'), (1, '8906.748')] [2023-12-26 16:16:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000125416_32112640.pth... [2023-12-26 16:16:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000125992_32260096.pth... [2023-12-26 16:16:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000124872_31973376.pth [2023-12-26 16:16:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000124296_31825920.pth [2023-12-26 16:16:46,348][105620] Updated weights for policy 1, policy_version 126002 (0.0006) [2023-12-26 16:16:46,408][105620] Updated weights for policy 1, policy_version 126012 (0.0005) [2023-12-26 16:16:46,455][105620] Updated weights for policy 1, policy_version 126022 (0.0005) [2023-12-26 16:16:46,819][105692] Updated weights for policy 0, policy_version 125426 (0.0009) [2023-12-26 16:16:46,874][105692] Updated weights for policy 0, policy_version 125436 (0.0009) [2023-12-26 16:16:46,920][105692] Updated weights for policy 0, policy_version 125446 (0.0009) [2023-12-26 16:16:47,042][105620] Updated weights for policy 1, policy_version 126032 (0.0007) [2023-12-26 16:16:47,103][105620] Updated weights for policy 1, policy_version 126042 (0.0009) [2023-12-26 16:16:47,169][105620] Updated weights for policy 1, policy_version 126052 (0.0009) [2023-12-26 16:16:47,702][105692] Updated weights for policy 0, policy_version 125456 (0.0008) [2023-12-26 16:16:47,767][105692] Updated weights for policy 0, policy_version 125466 (0.0009) [2023-12-26 16:16:47,814][105692] Updated weights for policy 0, policy_version 125476 (0.0010) [2023-12-26 16:16:47,864][105620] Updated weights for policy 1, policy_version 126062 (0.0009) [2023-12-26 16:16:47,920][105620] Updated weights for policy 1, policy_version 126072 (0.0008) [2023-12-26 16:16:47,984][105620] Updated weights for policy 1, policy_version 126082 (0.0005) [2023-12-26 16:16:48,613][105692] Updated weights for policy 0, policy_version 125486 (0.0008) [2023-12-26 16:16:48,649][105620] Updated weights for policy 1, policy_version 126092 (0.0008) [2023-12-26 16:16:48,667][105692] Updated weights for policy 0, policy_version 125496 (0.0009) [2023-12-26 16:16:48,710][105620] Updated weights for policy 1, policy_version 126102 (0.0007) [2023-12-26 16:16:48,724][105692] Updated weights for policy 0, policy_version 125506 (0.0006) [2023-12-26 16:16:48,767][105620] Updated weights for policy 1, policy_version 126112 (0.0007) [2023-12-26 16:16:49,432][105620] Updated weights for policy 1, policy_version 126122 (0.0009) [2023-12-26 16:16:49,495][105620] Updated weights for policy 1, policy_version 126132 (0.0008) [2023-12-26 16:16:49,536][105692] Updated weights for policy 0, policy_version 125516 (0.0007) [2023-12-26 16:16:49,564][105620] Updated weights for policy 1, policy_version 126142 (0.0007) [2023-12-26 16:16:49,593][105692] Updated weights for policy 0, policy_version 125526 (0.0008) [2023-12-26 16:16:49,619][105620] Updated weights for policy 1, policy_version 126152 (0.0007) [2023-12-26 16:16:49,652][105692] Updated weights for policy 0, policy_version 125536 (0.0008) [2023-12-26 16:16:50,304][105620] Updated weights for policy 1, policy_version 126162 (0.0008) [2023-12-26 16:16:50,350][105620] Updated weights for policy 1, policy_version 126172 (0.0009) [2023-12-26 16:16:50,397][105620] Updated weights for policy 1, policy_version 126182 (0.0009) [2023-12-26 16:16:50,439][105692] Updated weights for policy 0, policy_version 125546 (0.0009) [2023-12-26 16:16:50,505][105692] Updated weights for policy 0, policy_version 125556 (0.0009) [2023-12-26 16:16:50,579][105692] Updated weights for policy 0, policy_version 125566 (0.0009) [2023-12-26 16:16:50,637][105692] Updated weights for policy 0, policy_version 125576 (0.0009) [2023-12-26 16:16:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 64462848. Throughput: 0: 9419.9, 1: 9868.1. Samples: 64454900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:16:51,062][104569] Avg episode reward: [(0, '9168.898'), (1, '8553.532')] [2023-12-26 16:16:51,128][105620] Updated weights for policy 1, policy_version 126192 (0.0007) [2023-12-26 16:16:51,189][105620] Updated weights for policy 1, policy_version 126202 (0.0009) [2023-12-26 16:16:51,241][105620] Updated weights for policy 1, policy_version 126212 (0.0008) [2023-12-26 16:16:51,324][105692] Updated weights for policy 0, policy_version 125586 (0.0009) [2023-12-26 16:16:51,388][105692] Updated weights for policy 0, policy_version 125596 (0.0010) [2023-12-26 16:16:51,441][105692] Updated weights for policy 0, policy_version 125606 (0.0009) [2023-12-26 16:16:52,049][105620] Updated weights for policy 1, policy_version 126222 (0.0008) [2023-12-26 16:16:52,096][105620] Updated weights for policy 1, policy_version 126232 (0.0008) [2023-12-26 16:16:52,145][105620] Updated weights for policy 1, policy_version 126242 (0.0009) [2023-12-26 16:16:52,217][105692] Updated weights for policy 0, policy_version 125616 (0.0009) [2023-12-26 16:16:52,278][105692] Updated weights for policy 0, policy_version 125626 (0.0011) [2023-12-26 16:16:52,340][105692] Updated weights for policy 0, policy_version 125636 (0.0012) [2023-12-26 16:16:52,977][105620] Updated weights for policy 1, policy_version 126252 (0.0008) [2023-12-26 16:16:52,983][105692] Updated weights for policy 0, policy_version 125646 (0.0009) [2023-12-26 16:16:53,034][105620] Updated weights for policy 1, policy_version 126262 (0.0007) [2023-12-26 16:16:53,041][105692] Updated weights for policy 0, policy_version 125656 (0.0006) [2023-12-26 16:16:53,085][105620] Updated weights for policy 1, policy_version 126272 (0.0007) [2023-12-26 16:16:53,100][105692] Updated weights for policy 0, policy_version 125666 (0.0007) [2023-12-26 16:16:53,769][105692] Updated weights for policy 0, policy_version 125676 (0.0007) [2023-12-26 16:16:53,824][105692] Updated weights for policy 0, policy_version 125686 (0.0009) [2023-12-26 16:16:53,875][105692] Updated weights for policy 0, policy_version 125696 (0.0009) [2023-12-26 16:16:53,911][105620] Updated weights for policy 1, policy_version 126282 (0.0009) [2023-12-26 16:16:53,975][105620] Updated weights for policy 1, policy_version 126292 (0.0008) [2023-12-26 16:16:54,030][105620] Updated weights for policy 1, policy_version 126302 (0.0009) [2023-12-26 16:16:54,099][105620] Updated weights for policy 1, policy_version 126312 (0.0009) [2023-12-26 16:16:54,495][105692] Updated weights for policy 0, policy_version 125706 (0.0008) [2023-12-26 16:16:54,546][105692] Updated weights for policy 0, policy_version 125716 (0.0010) [2023-12-26 16:16:54,596][105692] Updated weights for policy 0, policy_version 125726 (0.0010) [2023-12-26 16:16:54,651][105692] Updated weights for policy 0, policy_version 125736 (0.0009) [2023-12-26 16:16:54,916][105620] Updated weights for policy 1, policy_version 126322 (0.0007) [2023-12-26 16:16:54,975][105620] Updated weights for policy 1, policy_version 126332 (0.0009) [2023-12-26 16:16:55,026][105620] Updated weights for policy 1, policy_version 126342 (0.0009) [2023-12-26 16:16:55,293][105692] Updated weights for policy 0, policy_version 125746 (0.0009) [2023-12-26 16:16:55,349][105692] Updated weights for policy 0, policy_version 125756 (0.0010) [2023-12-26 16:16:55,410][105692] Updated weights for policy 0, policy_version 125766 (0.0010) [2023-12-26 16:16:55,644][105620] Updated weights for policy 1, policy_version 126352 (0.0008) [2023-12-26 16:16:55,702][105620] Updated weights for policy 1, policy_version 126362 (0.0005) [2023-12-26 16:16:55,746][105620] Updated weights for policy 1, policy_version 126372 (0.0006) [2023-12-26 16:16:56,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 64561152. Throughput: 0: 9427.8, 1: 9779.7. Samples: 64571312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:16:56,062][104569] Avg episode reward: [(0, '9076.269'), (1, '8642.595')] [2023-12-26 16:16:56,138][105692] Updated weights for policy 0, policy_version 125776 (0.0010) [2023-12-26 16:16:56,200][105692] Updated weights for policy 0, policy_version 125786 (0.0009) [2023-12-26 16:16:56,261][105692] Updated weights for policy 0, policy_version 125796 (0.0010) [2023-12-26 16:16:56,436][105620] Updated weights for policy 1, policy_version 126382 (0.0010) [2023-12-26 16:16:56,487][105620] Updated weights for policy 1, policy_version 126392 (0.0010) [2023-12-26 16:16:56,532][105620] Updated weights for policy 1, policy_version 126402 (0.0010) [2023-12-26 16:16:56,865][105692] Updated weights for policy 0, policy_version 125806 (0.0009) [2023-12-26 16:16:56,912][105692] Updated weights for policy 0, policy_version 125816 (0.0008) [2023-12-26 16:16:56,956][105692] Updated weights for policy 0, policy_version 125826 (0.0008) [2023-12-26 16:16:57,260][105620] Updated weights for policy 1, policy_version 126412 (0.0008) [2023-12-26 16:16:57,318][105620] Updated weights for policy 1, policy_version 126422 (0.0005) [2023-12-26 16:16:57,376][105620] Updated weights for policy 1, policy_version 126432 (0.0008) [2023-12-26 16:16:57,382][105586] KL-divergence is very high: 144.8007 [2023-12-26 16:16:57,630][105692] Updated weights for policy 0, policy_version 125836 (0.0007) [2023-12-26 16:16:57,677][105692] Updated weights for policy 0, policy_version 125846 (0.0007) [2023-12-26 16:16:57,732][105692] Updated weights for policy 0, policy_version 125858 (0.0010) [2023-12-26 16:16:57,944][105620] Updated weights for policy 1, policy_version 126442 (0.0005) [2023-12-26 16:16:57,989][105620] Updated weights for policy 1, policy_version 126452 (0.0005) [2023-12-26 16:16:58,035][105620] Updated weights for policy 1, policy_version 126462 (0.0005) [2023-12-26 16:16:58,086][105620] Updated weights for policy 1, policy_version 126472 (0.0006) [2023-12-26 16:16:58,596][105692] Updated weights for policy 0, policy_version 125868 (0.0009) [2023-12-26 16:16:58,661][105692] Updated weights for policy 0, policy_version 125878 (0.0008) [2023-12-26 16:16:58,730][105692] Updated weights for policy 0, policy_version 125888 (0.0008) [2023-12-26 16:16:58,919][105620] Updated weights for policy 1, policy_version 126482 (0.0008) [2023-12-26 16:16:58,970][105620] Updated weights for policy 1, policy_version 126492 (0.0008) [2023-12-26 16:16:59,021][105620] Updated weights for policy 1, policy_version 126502 (0.0008) [2023-12-26 16:16:59,531][105692] Updated weights for policy 0, policy_version 125898 (0.0008) [2023-12-26 16:16:59,594][105692] Updated weights for policy 0, policy_version 125908 (0.0006) [2023-12-26 16:16:59,654][105692] Updated weights for policy 0, policy_version 125918 (0.0009) [2023-12-26 16:16:59,756][105620] Updated weights for policy 1, policy_version 126512 (0.0006) [2023-12-26 16:16:59,804][105620] Updated weights for policy 1, policy_version 126522 (0.0005) [2023-12-26 16:16:59,868][105620] Updated weights for policy 1, policy_version 126532 (0.0007) [2023-12-26 16:17:00,327][105692] Updated weights for policy 0, policy_version 125929 (0.0010) [2023-12-26 16:17:00,378][105692] Updated weights for policy 0, policy_version 125939 (0.0006) [2023-12-26 16:17:00,446][105692] Updated weights for policy 0, policy_version 125949 (0.0006) [2023-12-26 16:17:00,499][105692] Updated weights for policy 0, policy_version 125959 (0.0005) [2023-12-26 16:17:00,554][105620] Updated weights for policy 1, policy_version 126542 (0.0010) [2023-12-26 16:17:00,611][105620] Updated weights for policy 1, policy_version 126552 (0.0010) [2023-12-26 16:17:00,665][105620] Updated weights for policy 1, policy_version 126562 (0.0010) [2023-12-26 16:17:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 64659456. Throughput: 0: 9528.2, 1: 9790.8. Samples: 64631228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:17:01,062][104569] Avg episode reward: [(0, '9168.867'), (1, '8993.064')] [2023-12-26 16:17:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000126568_32407552.pth... [2023-12-26 16:17:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000125416_32112640.pth [2023-12-26 16:17:01,075][105692] Updated weights for policy 0, policy_version 125969 (0.0007) [2023-12-26 16:17:01,144][105692] Updated weights for policy 0, policy_version 125979 (0.0006) [2023-12-26 16:17:01,211][105692] Updated weights for policy 0, policy_version 125989 (0.0006) [2023-12-26 16:17:01,230][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000125992_32260096.pth... [2023-12-26 16:17:01,234][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000124840_31965184.pth [2023-12-26 16:17:01,430][105620] Updated weights for policy 1, policy_version 126572 (0.0009) [2023-12-26 16:17:01,503][105620] Updated weights for policy 1, policy_version 126582 (0.0005) [2023-12-26 16:17:01,572][105620] Updated weights for policy 1, policy_version 126592 (0.0006) [2023-12-26 16:17:01,944][105692] Updated weights for policy 0, policy_version 125999 (0.0006) [2023-12-26 16:17:01,992][105692] Updated weights for policy 0, policy_version 126009 (0.0005) [2023-12-26 16:17:02,042][105692] Updated weights for policy 0, policy_version 126019 (0.0006) [2023-12-26 16:17:02,190][105620] Updated weights for policy 1, policy_version 126602 (0.0008) [2023-12-26 16:17:02,247][105620] Updated weights for policy 1, policy_version 126612 (0.0008) [2023-12-26 16:17:02,305][105620] Updated weights for policy 1, policy_version 126622 (0.0009) [2023-12-26 16:17:02,367][105620] Updated weights for policy 1, policy_version 126632 (0.0009) [2023-12-26 16:17:02,743][105692] Updated weights for policy 0, policy_version 126029 (0.0007) [2023-12-26 16:17:02,801][105692] Updated weights for policy 0, policy_version 126039 (0.0009) [2023-12-26 16:17:02,869][105692] Updated weights for policy 0, policy_version 126049 (0.0010) [2023-12-26 16:17:03,082][105620] Updated weights for policy 1, policy_version 126642 (0.0006) [2023-12-26 16:17:03,135][105620] Updated weights for policy 1, policy_version 126652 (0.0006) [2023-12-26 16:17:03,192][105620] Updated weights for policy 1, policy_version 126662 (0.0005) [2023-12-26 16:17:03,710][105692] Updated weights for policy 0, policy_version 126059 (0.0010) [2023-12-26 16:17:03,757][105692] Updated weights for policy 0, policy_version 126069 (0.0009) [2023-12-26 16:17:03,782][105620] Updated weights for policy 1, policy_version 126672 (0.0008) [2023-12-26 16:17:03,806][105692] Updated weights for policy 0, policy_version 126079 (0.0006) [2023-12-26 16:17:03,832][105620] Updated weights for policy 1, policy_version 126682 (0.0007) [2023-12-26 16:17:03,900][105620] Updated weights for policy 1, policy_version 126692 (0.0007) [2023-12-26 16:17:04,560][105620] Updated weights for policy 1, policy_version 126702 (0.0007) [2023-12-26 16:17:04,618][105620] Updated weights for policy 1, policy_version 126712 (0.0009) [2023-12-26 16:17:04,637][105692] Updated weights for policy 0, policy_version 126089 (0.0007) [2023-12-26 16:17:04,669][105620] Updated weights for policy 1, policy_version 126722 (0.0007) [2023-12-26 16:17:04,693][105692] Updated weights for policy 0, policy_version 126099 (0.0008) [2023-12-26 16:17:04,746][105692] Updated weights for policy 0, policy_version 126109 (0.0008) [2023-12-26 16:17:04,804][105692] Updated weights for policy 0, policy_version 126119 (0.0009) [2023-12-26 16:17:05,360][105620] Updated weights for policy 1, policy_version 126732 (0.0009) [2023-12-26 16:17:05,409][105620] Updated weights for policy 1, policy_version 126742 (0.0010) [2023-12-26 16:17:05,463][105620] Updated weights for policy 1, policy_version 126752 (0.0010) [2023-12-26 16:17:05,604][105692] Updated weights for policy 0, policy_version 126129 (0.0009) [2023-12-26 16:17:05,661][105692] Updated weights for policy 0, policy_version 126139 (0.0009) [2023-12-26 16:17:05,714][105692] Updated weights for policy 0, policy_version 126149 (0.0009) [2023-12-26 16:17:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 64757760. Throughput: 0: 9593.3, 1: 9823.0. Samples: 64748304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:17:06,063][104569] Avg episode reward: [(0, '9168.623'), (1, '8552.383')] [2023-12-26 16:17:06,225][105620] Updated weights for policy 1, policy_version 126762 (0.0010) [2023-12-26 16:17:06,280][105620] Updated weights for policy 1, policy_version 126772 (0.0009) [2023-12-26 16:17:06,339][105620] Updated weights for policy 1, policy_version 126782 (0.0009) [2023-12-26 16:17:06,402][105620] Updated weights for policy 1, policy_version 126792 (0.0009) [2023-12-26 16:17:06,431][105692] Updated weights for policy 0, policy_version 126159 (0.0008) [2023-12-26 16:17:06,494][105692] Updated weights for policy 0, policy_version 126169 (0.0009) [2023-12-26 16:17:06,547][105692] Updated weights for policy 0, policy_version 126179 (0.0007) [2023-12-26 16:17:07,097][105620] Updated weights for policy 1, policy_version 126802 (0.0006) [2023-12-26 16:17:07,149][105620] Updated weights for policy 1, policy_version 126812 (0.0009) [2023-12-26 16:17:07,198][105620] Updated weights for policy 1, policy_version 126822 (0.0009) [2023-12-26 16:17:07,352][105692] Updated weights for policy 0, policy_version 126189 (0.0009) [2023-12-26 16:17:07,411][105692] Updated weights for policy 0, policy_version 126199 (0.0009) [2023-12-26 16:17:07,475][105692] Updated weights for policy 0, policy_version 126209 (0.0009) [2023-12-26 16:17:07,874][105620] Updated weights for policy 1, policy_version 126832 (0.0006) [2023-12-26 16:17:07,926][105620] Updated weights for policy 1, policy_version 126842 (0.0005) [2023-12-26 16:17:07,975][105620] Updated weights for policy 1, policy_version 126852 (0.0008) [2023-12-26 16:17:08,206][105692] Updated weights for policy 0, policy_version 126219 (0.0008) [2023-12-26 16:17:08,262][105692] Updated weights for policy 0, policy_version 126229 (0.0009) [2023-12-26 16:17:08,311][105692] Updated weights for policy 0, policy_version 126239 (0.0009) [2023-12-26 16:17:08,733][105620] Updated weights for policy 1, policy_version 126862 (0.0008) [2023-12-26 16:17:08,793][105620] Updated weights for policy 1, policy_version 126872 (0.0009) [2023-12-26 16:17:08,853][105620] Updated weights for policy 1, policy_version 126882 (0.0009) [2023-12-26 16:17:09,062][105692] Updated weights for policy 0, policy_version 126249 (0.0008) [2023-12-26 16:17:09,117][105692] Updated weights for policy 0, policy_version 126259 (0.0010) [2023-12-26 16:17:09,178][105692] Updated weights for policy 0, policy_version 126269 (0.0008) [2023-12-26 16:17:09,244][105692] Updated weights for policy 0, policy_version 126279 (0.0008) [2023-12-26 16:17:09,644][105620] Updated weights for policy 1, policy_version 126892 (0.0009) [2023-12-26 16:17:09,710][105620] Updated weights for policy 1, policy_version 126902 (0.0008) [2023-12-26 16:17:09,773][105620] Updated weights for policy 1, policy_version 126912 (0.0009) [2023-12-26 16:17:09,974][105692] Updated weights for policy 0, policy_version 126289 (0.0009) [2023-12-26 16:17:10,026][105692] Updated weights for policy 0, policy_version 126299 (0.0009) [2023-12-26 16:17:10,089][105692] Updated weights for policy 0, policy_version 126309 (0.0009) [2023-12-26 16:17:10,531][105620] Updated weights for policy 1, policy_version 126922 (0.0009) [2023-12-26 16:17:10,598][105620] Updated weights for policy 1, policy_version 126932 (0.0009) [2023-12-26 16:17:10,649][105620] Updated weights for policy 1, policy_version 126942 (0.0008) [2023-12-26 16:17:10,708][105620] Updated weights for policy 1, policy_version 126952 (0.0009) [2023-12-26 16:17:10,851][105692] Updated weights for policy 0, policy_version 126319 (0.0008) [2023-12-26 16:17:10,905][105692] Updated weights for policy 0, policy_version 126329 (0.0009) [2023-12-26 16:17:10,950][105692] Updated weights for policy 0, policy_version 126339 (0.0008) [2023-12-26 16:17:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 64856064. Throughput: 0: 9609.9, 1: 9783.3. Samples: 64860620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:17:11,062][104569] Avg episode reward: [(0, '9075.438'), (1, '8193.237')] [2023-12-26 16:17:11,478][105620] Updated weights for policy 1, policy_version 126962 (0.0008) [2023-12-26 16:17:11,533][105620] Updated weights for policy 1, policy_version 126972 (0.0008) [2023-12-26 16:17:11,597][105620] Updated weights for policy 1, policy_version 126982 (0.0008) [2023-12-26 16:17:11,799][105692] Updated weights for policy 0, policy_version 126349 (0.0008) [2023-12-26 16:17:11,853][105692] Updated weights for policy 0, policy_version 126359 (0.0009) [2023-12-26 16:17:11,915][105692] Updated weights for policy 0, policy_version 126369 (0.0009) [2023-12-26 16:17:12,356][105620] Updated weights for policy 1, policy_version 126992 (0.0009) [2023-12-26 16:17:12,421][105620] Updated weights for policy 1, policy_version 127002 (0.0009) [2023-12-26 16:17:12,479][105620] Updated weights for policy 1, policy_version 127012 (0.0009) [2023-12-26 16:17:12,699][105692] Updated weights for policy 0, policy_version 126379 (0.0009) [2023-12-26 16:17:12,750][105692] Updated weights for policy 0, policy_version 126389 (0.0009) [2023-12-26 16:17:12,801][105692] Updated weights for policy 0, policy_version 126399 (0.0009) [2023-12-26 16:17:13,144][105620] Updated weights for policy 1, policy_version 127022 (0.0007) [2023-12-26 16:17:13,207][105620] Updated weights for policy 1, policy_version 127032 (0.0005) [2023-12-26 16:17:13,273][105620] Updated weights for policy 1, policy_version 127042 (0.0005) [2023-12-26 16:17:13,698][105692] Updated weights for policy 0, policy_version 126409 (0.0010) [2023-12-26 16:17:13,761][105692] Updated weights for policy 0, policy_version 126419 (0.0008) [2023-12-26 16:17:13,770][105620] Updated weights for policy 1, policy_version 127052 (0.0006) [2023-12-26 16:17:13,820][105692] Updated weights for policy 0, policy_version 126429 (0.0008) [2023-12-26 16:17:13,822][105620] Updated weights for policy 1, policy_version 127062 (0.0006) [2023-12-26 16:17:13,875][105620] Updated weights for policy 1, policy_version 127072 (0.0007) [2023-12-26 16:17:13,880][105692] Updated weights for policy 0, policy_version 126439 (0.0008) [2023-12-26 16:17:14,559][105620] Updated weights for policy 1, policy_version 127082 (0.0007) [2023-12-26 16:17:14,612][105620] Updated weights for policy 1, policy_version 127092 (0.0009) [2023-12-26 16:17:14,647][105692] Updated weights for policy 0, policy_version 126449 (0.0008) [2023-12-26 16:17:14,670][105620] Updated weights for policy 1, policy_version 127102 (0.0008) [2023-12-26 16:17:14,690][105692] Updated weights for policy 0, policy_version 126459 (0.0008) [2023-12-26 16:17:14,730][105620] Updated weights for policy 1, policy_version 127112 (0.0008) [2023-12-26 16:17:14,732][105692] Updated weights for policy 0, policy_version 126469 (0.0007) [2023-12-26 16:17:15,497][105620] Updated weights for policy 1, policy_version 127122 (0.0008) [2023-12-26 16:17:15,512][105692] Updated weights for policy 0, policy_version 126479 (0.0007) [2023-12-26 16:17:15,550][105620] Updated weights for policy 1, policy_version 127132 (0.0007) [2023-12-26 16:17:15,571][105692] Updated weights for policy 0, policy_version 126489 (0.0007) [2023-12-26 16:17:15,606][105620] Updated weights for policy 1, policy_version 127142 (0.0006) [2023-12-26 16:17:15,625][105692] Updated weights for policy 0, policy_version 126499 (0.0007) [2023-12-26 16:17:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.1, 300 sec: 19521.9). Total num frames: 64946176. Throughput: 0: 9497.7, 1: 9844.8. Samples: 64917352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:17:16,063][104569] Avg episode reward: [(0, '9165.390'), (1, '8175.742')] [2023-12-26 16:17:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000126504_32391168.pth... [2023-12-26 16:17:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000127144_32555008.pth... [2023-12-26 16:17:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000125416_32112640.pth [2023-12-26 16:17:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000125992_32260096.pth [2023-12-26 16:17:16,252][105692] Updated weights for policy 0, policy_version 126509 (0.0007) [2023-12-26 16:17:16,305][105692] Updated weights for policy 0, policy_version 126519 (0.0005) [2023-12-26 16:17:16,362][105692] Updated weights for policy 0, policy_version 126529 (0.0005) [2023-12-26 16:17:16,386][105620] Updated weights for policy 1, policy_version 127152 (0.0005) [2023-12-26 16:17:16,438][105620] Updated weights for policy 1, policy_version 127162 (0.0008) [2023-12-26 16:17:16,487][105620] Updated weights for policy 1, policy_version 127172 (0.0005) [2023-12-26 16:17:17,059][105692] Updated weights for policy 0, policy_version 126539 (0.0007) [2023-12-26 16:17:17,110][105692] Updated weights for policy 0, policy_version 126549 (0.0007) [2023-12-26 16:17:17,117][105620] Updated weights for policy 1, policy_version 127182 (0.0007) [2023-12-26 16:17:17,163][105692] Updated weights for policy 0, policy_version 126559 (0.0006) [2023-12-26 16:17:17,173][105620] Updated weights for policy 1, policy_version 127192 (0.0007) [2023-12-26 16:17:17,226][105620] Updated weights for policy 1, policy_version 127202 (0.0007) [2023-12-26 16:17:17,852][105692] Updated weights for policy 0, policy_version 126569 (0.0007) [2023-12-26 16:17:17,910][105692] Updated weights for policy 0, policy_version 126579 (0.0010) [2023-12-26 16:17:17,917][105620] Updated weights for policy 1, policy_version 127212 (0.0007) [2023-12-26 16:17:17,965][105692] Updated weights for policy 0, policy_version 126589 (0.0010) [2023-12-26 16:17:17,966][105620] Updated weights for policy 1, policy_version 127222 (0.0005) [2023-12-26 16:17:18,015][105620] Updated weights for policy 1, policy_version 127232 (0.0006) [2023-12-26 16:17:18,019][105692] Updated weights for policy 0, policy_version 126599 (0.0009) [2023-12-26 16:17:18,574][105692] Updated weights for policy 0, policy_version 126609 (0.0010) [2023-12-26 16:17:18,630][105692] Updated weights for policy 0, policy_version 126619 (0.0011) [2023-12-26 16:17:18,689][105692] Updated weights for policy 0, policy_version 126629 (0.0011) [2023-12-26 16:17:18,768][105620] Updated weights for policy 1, policy_version 127242 (0.0006) [2023-12-26 16:17:18,827][105620] Updated weights for policy 1, policy_version 127252 (0.0008) [2023-12-26 16:17:18,894][105620] Updated weights for policy 1, policy_version 127262 (0.0009) [2023-12-26 16:17:18,952][105620] Updated weights for policy 1, policy_version 127272 (0.0008) [2023-12-26 16:17:19,451][105692] Updated weights for policy 0, policy_version 126639 (0.0009) [2023-12-26 16:17:19,516][105692] Updated weights for policy 0, policy_version 126649 (0.0008) [2023-12-26 16:17:19,562][105692] Updated weights for policy 0, policy_version 126659 (0.0008) [2023-12-26 16:17:19,690][105620] Updated weights for policy 1, policy_version 127282 (0.0008) [2023-12-26 16:17:19,753][105620] Updated weights for policy 1, policy_version 127292 (0.0005) [2023-12-26 16:17:19,810][105620] Updated weights for policy 1, policy_version 127302 (0.0006) [2023-12-26 16:17:20,272][105692] Updated weights for policy 0, policy_version 126669 (0.0009) [2023-12-26 16:17:20,332][105692] Updated weights for policy 0, policy_version 126679 (0.0009) [2023-12-26 16:17:20,390][105692] Updated weights for policy 0, policy_version 126689 (0.0010) [2023-12-26 16:17:20,544][105620] Updated weights for policy 1, policy_version 127312 (0.0008) [2023-12-26 16:17:20,612][105620] Updated weights for policy 1, policy_version 127322 (0.0008) [2023-12-26 16:17:20,676][105620] Updated weights for policy 1, policy_version 127332 (0.0008) [2023-12-26 16:17:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 65044480. Throughput: 0: 9548.1, 1: 9900.4. Samples: 65035660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:17:21,062][104569] Avg episode reward: [(0, '9073.383'), (1, '8191.550')] [2023-12-26 16:17:21,138][105692] Updated weights for policy 0, policy_version 126699 (0.0008) [2023-12-26 16:17:21,198][105692] Updated weights for policy 0, policy_version 126709 (0.0008) [2023-12-26 16:17:21,258][105692] Updated weights for policy 0, policy_version 126719 (0.0009) [2023-12-26 16:17:21,493][105620] Updated weights for policy 1, policy_version 127342 (0.0008) [2023-12-26 16:17:21,555][105620] Updated weights for policy 1, policy_version 127352 (0.0009) [2023-12-26 16:17:21,618][105620] Updated weights for policy 1, policy_version 127362 (0.0009) [2023-12-26 16:17:22,025][105692] Updated weights for policy 0, policy_version 126729 (0.0009) [2023-12-26 16:17:22,081][105692] Updated weights for policy 0, policy_version 126739 (0.0009) [2023-12-26 16:17:22,139][105692] Updated weights for policy 0, policy_version 126749 (0.0009) [2023-12-26 16:17:22,203][105692] Updated weights for policy 0, policy_version 126759 (0.0007) [2023-12-26 16:17:22,438][105620] Updated weights for policy 1, policy_version 127372 (0.0009) [2023-12-26 16:17:22,487][105620] Updated weights for policy 1, policy_version 127382 (0.0009) [2023-12-26 16:17:22,541][105620] Updated weights for policy 1, policy_version 127392 (0.0009) [2023-12-26 16:17:22,915][105692] Updated weights for policy 0, policy_version 126769 (0.0006) [2023-12-26 16:17:22,982][105692] Updated weights for policy 0, policy_version 126779 (0.0007) [2023-12-26 16:17:23,052][105692] Updated weights for policy 0, policy_version 126789 (0.0006) [2023-12-26 16:17:23,344][105620] Updated weights for policy 1, policy_version 127402 (0.0009) [2023-12-26 16:17:23,400][105620] Updated weights for policy 1, policy_version 127412 (0.0009) [2023-12-26 16:17:23,462][105620] Updated weights for policy 1, policy_version 127422 (0.0010) [2023-12-26 16:17:23,525][105620] Updated weights for policy 1, policy_version 127432 (0.0009) [2023-12-26 16:17:23,721][105692] Updated weights for policy 0, policy_version 126799 (0.0008) [2023-12-26 16:17:23,792][105692] Updated weights for policy 0, policy_version 126809 (0.0008) [2023-12-26 16:17:23,855][105692] Updated weights for policy 0, policy_version 126819 (0.0008) [2023-12-26 16:17:24,251][105620] Updated weights for policy 1, policy_version 127442 (0.0009) [2023-12-26 16:17:24,298][105620] Updated weights for policy 1, policy_version 127452 (0.0009) [2023-12-26 16:17:24,360][105620] Updated weights for policy 1, policy_version 127462 (0.0009) [2023-12-26 16:17:24,617][105692] Updated weights for policy 0, policy_version 126829 (0.0008) [2023-12-26 16:17:24,669][105692] Updated weights for policy 0, policy_version 126839 (0.0008) [2023-12-26 16:17:24,724][105692] Updated weights for policy 0, policy_version 126849 (0.0009) [2023-12-26 16:17:25,177][105620] Updated weights for policy 1, policy_version 127472 (0.0009) [2023-12-26 16:17:25,235][105620] Updated weights for policy 1, policy_version 127482 (0.0009) [2023-12-26 16:17:25,283][105620] Updated weights for policy 1, policy_version 127492 (0.0009) [2023-12-26 16:17:25,405][105692] Updated weights for policy 0, policy_version 126859 (0.0009) [2023-12-26 16:17:25,452][105692] Updated weights for policy 0, policy_version 126869 (0.0009) [2023-12-26 16:17:25,516][105692] Updated weights for policy 0, policy_version 126879 (0.0009) [2023-12-26 16:17:26,021][105620] Updated weights for policy 1, policy_version 127502 (0.0009) [2023-12-26 16:17:26,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 65134592. Throughput: 0: 9528.2, 1: 9750.0. Samples: 65147368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:17:26,062][104569] Avg episode reward: [(0, '9164.758'), (1, '8208.332')] [2023-12-26 16:17:26,083][105620] Updated weights for policy 1, policy_version 127512 (0.0009) [2023-12-26 16:17:26,148][105620] Updated weights for policy 1, policy_version 127522 (0.0009) [2023-12-26 16:17:26,261][105692] Updated weights for policy 0, policy_version 126889 (0.0008) [2023-12-26 16:17:26,318][105692] Updated weights for policy 0, policy_version 126899 (0.0005) [2023-12-26 16:17:26,373][105692] Updated weights for policy 0, policy_version 126909 (0.0006) [2023-12-26 16:17:26,432][105692] Updated weights for policy 0, policy_version 126919 (0.0006) [2023-12-26 16:17:26,981][105620] Updated weights for policy 1, policy_version 127532 (0.0008) [2023-12-26 16:17:27,015][105692] Updated weights for policy 0, policy_version 126929 (0.0010) [2023-12-26 16:17:27,036][105620] Updated weights for policy 1, policy_version 127542 (0.0006) [2023-12-26 16:17:27,063][105692] Updated weights for policy 0, policy_version 126939 (0.0010) [2023-12-26 16:17:27,088][105620] Updated weights for policy 1, policy_version 127552 (0.0006) [2023-12-26 16:17:27,107][105692] Updated weights for policy 0, policy_version 126949 (0.0010) [2023-12-26 16:17:27,771][105620] Updated weights for policy 1, policy_version 127562 (0.0009) [2023-12-26 16:17:27,828][105620] Updated weights for policy 1, policy_version 127572 (0.0007) [2023-12-26 16:17:27,838][105692] Updated weights for policy 0, policy_version 126959 (0.0009) [2023-12-26 16:17:27,879][105620] Updated weights for policy 1, policy_version 127582 (0.0008) [2023-12-26 16:17:27,890][105692] Updated weights for policy 0, policy_version 126969 (0.0005) [2023-12-26 16:17:27,931][105620] Updated weights for policy 1, policy_version 127592 (0.0008) [2023-12-26 16:17:27,942][105692] Updated weights for policy 0, policy_version 126979 (0.0009) [2023-12-26 16:17:28,690][105692] Updated weights for policy 0, policy_version 126989 (0.0008) [2023-12-26 16:17:28,714][105620] Updated weights for policy 1, policy_version 127602 (0.0009) [2023-12-26 16:17:28,742][105692] Updated weights for policy 0, policy_version 126999 (0.0006) [2023-12-26 16:17:28,768][105620] Updated weights for policy 1, policy_version 127612 (0.0008) [2023-12-26 16:17:28,787][105692] Updated weights for policy 0, policy_version 127009 (0.0005) [2023-12-26 16:17:28,827][105620] Updated weights for policy 1, policy_version 127622 (0.0009) [2023-12-26 16:17:29,483][105692] Updated weights for policy 0, policy_version 127019 (0.0007) [2023-12-26 16:17:29,535][105692] Updated weights for policy 0, policy_version 127029 (0.0009) [2023-12-26 16:17:29,584][105692] Updated weights for policy 0, policy_version 127039 (0.0008) [2023-12-26 16:17:29,612][105620] Updated weights for policy 1, policy_version 127632 (0.0007) [2023-12-26 16:17:29,666][105620] Updated weights for policy 1, policy_version 127642 (0.0008) [2023-12-26 16:17:29,723][105620] Updated weights for policy 1, policy_version 127652 (0.0009) [2023-12-26 16:17:30,289][105692] Updated weights for policy 0, policy_version 127049 (0.0007) [2023-12-26 16:17:30,340][105692] Updated weights for policy 0, policy_version 127059 (0.0005) [2023-12-26 16:17:30,397][105692] Updated weights for policy 0, policy_version 127069 (0.0005) [2023-12-26 16:17:30,462][105692] Updated weights for policy 0, policy_version 127079 (0.0005) [2023-12-26 16:17:30,572][105620] Updated weights for policy 1, policy_version 127663 (0.0010) [2023-12-26 16:17:30,625][105620] Updated weights for policy 1, policy_version 127673 (0.0009) [2023-12-26 16:17:30,669][105620] Updated weights for policy 1, policy_version 127683 (0.0008) [2023-12-26 16:17:31,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 65232896. Throughput: 0: 9553.5, 1: 9731.3. Samples: 65205412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:17:31,062][104569] Avg episode reward: [(0, '9347.254'), (1, '8991.231')] [2023-12-26 16:17:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000127688_32694272.pth... [2023-12-26 16:17:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000126568_32407552.pth [2023-12-26 16:17:31,077][105692] Updated weights for policy 0, policy_version 127089 (0.0010) [2023-12-26 16:17:31,142][105692] Updated weights for policy 0, policy_version 127099 (0.0010) [2023-12-26 16:17:31,208][105692] Updated weights for policy 0, policy_version 127109 (0.0009) [2023-12-26 16:17:31,226][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000127112_32546816.pth... [2023-12-26 16:17:31,230][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000125992_32260096.pth [2023-12-26 16:17:31,439][105620] Updated weights for policy 1, policy_version 127693 (0.0008) [2023-12-26 16:17:31,500][105620] Updated weights for policy 1, policy_version 127703 (0.0006) [2023-12-26 16:17:31,564][105620] Updated weights for policy 1, policy_version 127713 (0.0008) [2023-12-26 16:17:31,927][105692] Updated weights for policy 0, policy_version 127119 (0.0010) [2023-12-26 16:17:31,991][105692] Updated weights for policy 0, policy_version 127129 (0.0008) [2023-12-26 16:17:32,062][105692] Updated weights for policy 0, policy_version 127139 (0.0005) [2023-12-26 16:17:32,160][105620] Updated weights for policy 1, policy_version 127723 (0.0008) [2023-12-26 16:17:32,216][105620] Updated weights for policy 1, policy_version 127733 (0.0005) [2023-12-26 16:17:32,284][105620] Updated weights for policy 1, policy_version 127743 (0.0008) [2023-12-26 16:17:32,649][105692] Updated weights for policy 0, policy_version 127149 (0.0008) [2023-12-26 16:17:32,705][105692] Updated weights for policy 0, policy_version 127159 (0.0009) [2023-12-26 16:17:32,758][105692] Updated weights for policy 0, policy_version 127169 (0.0009) [2023-12-26 16:17:32,959][105620] Updated weights for policy 1, policy_version 127753 (0.0007) [2023-12-26 16:17:33,024][105620] Updated weights for policy 1, policy_version 127763 (0.0009) [2023-12-26 16:17:33,090][105620] Updated weights for policy 1, policy_version 127773 (0.0006) [2023-12-26 16:17:33,154][105620] Updated weights for policy 1, policy_version 127783 (0.0005) [2023-12-26 16:17:33,427][105692] Updated weights for policy 0, policy_version 127179 (0.0008) [2023-12-26 16:17:33,478][105692] Updated weights for policy 0, policy_version 127189 (0.0005) [2023-12-26 16:17:33,531][105692] Updated weights for policy 0, policy_version 127199 (0.0006) [2023-12-26 16:17:33,719][105620] Updated weights for policy 1, policy_version 127793 (0.0005) [2023-12-26 16:17:33,764][105620] Updated weights for policy 1, policy_version 127803 (0.0005) [2023-12-26 16:17:33,811][105620] Updated weights for policy 1, policy_version 127813 (0.0006) [2023-12-26 16:17:34,230][105692] Updated weights for policy 0, policy_version 127209 (0.0008) [2023-12-26 16:17:34,282][105692] Updated weights for policy 0, policy_version 127219 (0.0009) [2023-12-26 16:17:34,334][105692] Updated weights for policy 0, policy_version 127229 (0.0005) [2023-12-26 16:17:34,394][105692] Updated weights for policy 0, policy_version 127239 (0.0008) [2023-12-26 16:17:34,523][105620] Updated weights for policy 1, policy_version 127823 (0.0007) [2023-12-26 16:17:34,582][105620] Updated weights for policy 1, policy_version 127833 (0.0005) [2023-12-26 16:17:34,644][105620] Updated weights for policy 1, policy_version 127843 (0.0005) [2023-12-26 16:17:35,178][105692] Updated weights for policy 0, policy_version 127249 (0.0009) [2023-12-26 16:17:35,224][105692] Updated weights for policy 0, policy_version 127259 (0.0008) [2023-12-26 16:17:35,279][105692] Updated weights for policy 0, policy_version 127269 (0.0009) [2023-12-26 16:17:35,305][105620] Updated weights for policy 1, policy_version 127853 (0.0007) [2023-12-26 16:17:35,362][105620] Updated weights for policy 1, policy_version 127863 (0.0009) [2023-12-26 16:17:35,419][105620] Updated weights for policy 1, policy_version 127873 (0.0009) [2023-12-26 16:17:35,966][105692] Updated weights for policy 0, policy_version 127279 (0.0008) [2023-12-26 16:17:36,036][105692] Updated weights for policy 0, policy_version 127289 (0.0005) [2023-12-26 16:17:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 65331200. Throughput: 0: 9692.8, 1: 9661.1. Samples: 65325828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:17:36,062][104569] Avg episode reward: [(0, '9346.687'), (1, '8551.722')] [2023-12-26 16:17:36,093][105692] Updated weights for policy 0, policy_version 127299 (0.0005) [2023-12-26 16:17:36,169][105620] Updated weights for policy 1, policy_version 127883 (0.0009) [2023-12-26 16:17:36,225][105620] Updated weights for policy 1, policy_version 127893 (0.0008) [2023-12-26 16:17:36,283][105620] Updated weights for policy 1, policy_version 127903 (0.0010) [2023-12-26 16:17:36,675][105692] Updated weights for policy 0, policy_version 127309 (0.0006) [2023-12-26 16:17:36,722][105692] Updated weights for policy 0, policy_version 127319 (0.0005) [2023-12-26 16:17:36,774][105692] Updated weights for policy 0, policy_version 127329 (0.0005) [2023-12-26 16:17:37,179][105620] Updated weights for policy 1, policy_version 127913 (0.0010) [2023-12-26 16:17:37,231][105620] Updated weights for policy 1, policy_version 127923 (0.0009) [2023-12-26 16:17:37,282][105620] Updated weights for policy 1, policy_version 127934 (0.0009) [2023-12-26 16:17:37,341][105620] Updated weights for policy 1, policy_version 127944 (0.0009) [2023-12-26 16:17:37,342][105692] Updated weights for policy 0, policy_version 127339 (0.0007) [2023-12-26 16:17:37,389][105692] Updated weights for policy 0, policy_version 127349 (0.0009) [2023-12-26 16:17:37,453][105692] Updated weights for policy 0, policy_version 127359 (0.0009) [2023-12-26 16:17:38,020][105692] Updated weights for policy 0, policy_version 127369 (0.0009) [2023-12-26 16:17:38,079][105692] Updated weights for policy 0, policy_version 127379 (0.0009) [2023-12-26 16:17:38,135][105692] Updated weights for policy 0, policy_version 127389 (0.0006) [2023-12-26 16:17:38,201][105692] Updated weights for policy 0, policy_version 127399 (0.0005) [2023-12-26 16:17:38,229][105620] Updated weights for policy 1, policy_version 127954 (0.0009) [2023-12-26 16:17:38,282][105620] Updated weights for policy 1, policy_version 127964 (0.0009) [2023-12-26 16:17:38,343][105620] Updated weights for policy 1, policy_version 127974 (0.0009) [2023-12-26 16:17:38,774][105692] Updated weights for policy 0, policy_version 127409 (0.0010) [2023-12-26 16:17:38,832][105692] Updated weights for policy 0, policy_version 127419 (0.0010) [2023-12-26 16:17:38,894][105692] Updated weights for policy 0, policy_version 127429 (0.0010) [2023-12-26 16:17:39,135][105620] Updated weights for policy 1, policy_version 127984 (0.0009) [2023-12-26 16:17:39,192][105620] Updated weights for policy 1, policy_version 127994 (0.0009) [2023-12-26 16:17:39,255][105620] Updated weights for policy 1, policy_version 128004 (0.0009) [2023-12-26 16:17:39,526][105692] Updated weights for policy 0, policy_version 127439 (0.0006) [2023-12-26 16:17:39,593][105692] Updated weights for policy 0, policy_version 127449 (0.0006) [2023-12-26 16:17:39,663][105692] Updated weights for policy 0, policy_version 127459 (0.0006) [2023-12-26 16:17:40,080][105620] Updated weights for policy 1, policy_version 128014 (0.0007) [2023-12-26 16:17:40,133][105620] Updated weights for policy 1, policy_version 128024 (0.0008) [2023-12-26 16:17:40,182][105620] Updated weights for policy 1, policy_version 128034 (0.0008) [2023-12-26 16:17:40,334][105692] Updated weights for policy 0, policy_version 127469 (0.0008) [2023-12-26 16:17:40,403][105692] Updated weights for policy 0, policy_version 127479 (0.0011) [2023-12-26 16:17:40,469][105692] Updated weights for policy 0, policy_version 127489 (0.0011) [2023-12-26 16:17:40,950][105620] Updated weights for policy 1, policy_version 128044 (0.0008) [2023-12-26 16:17:41,019][105620] Updated weights for policy 1, policy_version 128054 (0.0010) [2023-12-26 16:17:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 65429504. Throughput: 0: 9793.1, 1: 9581.1. Samples: 65443148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:17:41,062][104569] Avg episode reward: [(0, '9346.629'), (1, '8466.202')] [2023-12-26 16:17:41,082][105620] Updated weights for policy 1, policy_version 128064 (0.0009) [2023-12-26 16:17:41,200][105692] Updated weights for policy 0, policy_version 127499 (0.0011) [2023-12-26 16:17:41,255][105692] Updated weights for policy 0, policy_version 127509 (0.0010) [2023-12-26 16:17:41,318][105692] Updated weights for policy 0, policy_version 127519 (0.0010) [2023-12-26 16:17:41,816][105620] Updated weights for policy 1, policy_version 128074 (0.0008) [2023-12-26 16:17:41,881][105620] Updated weights for policy 1, policy_version 128084 (0.0008) [2023-12-26 16:17:41,943][105620] Updated weights for policy 1, policy_version 128094 (0.0010) [2023-12-26 16:17:42,013][105620] Updated weights for policy 1, policy_version 128104 (0.0010) [2023-12-26 16:17:42,120][105692] Updated weights for policy 0, policy_version 127529 (0.0010) [2023-12-26 16:17:42,173][105692] Updated weights for policy 0, policy_version 127539 (0.0005) [2023-12-26 16:17:42,232][105692] Updated weights for policy 0, policy_version 127549 (0.0006) [2023-12-26 16:17:42,297][105692] Updated weights for policy 0, policy_version 127559 (0.0007) [2023-12-26 16:17:42,844][105620] Updated weights for policy 1, policy_version 128114 (0.0009) [2023-12-26 16:17:42,899][105620] Updated weights for policy 1, policy_version 128124 (0.0009) [2023-12-26 16:17:42,959][105620] Updated weights for policy 1, policy_version 128134 (0.0006) [2023-12-26 16:17:42,959][105692] Updated weights for policy 0, policy_version 127569 (0.0009) [2023-12-26 16:17:43,019][105692] Updated weights for policy 0, policy_version 127579 (0.0010) [2023-12-26 16:17:43,082][105692] Updated weights for policy 0, policy_version 127589 (0.0011) [2023-12-26 16:17:43,693][105620] Updated weights for policy 1, policy_version 128144 (0.0007) [2023-12-26 16:17:43,740][105620] Updated weights for policy 1, policy_version 128154 (0.0006) [2023-12-26 16:17:43,751][105692] Updated weights for policy 0, policy_version 127599 (0.0010) [2023-12-26 16:17:43,785][105620] Updated weights for policy 1, policy_version 128164 (0.0005) [2023-12-26 16:17:43,812][105692] Updated weights for policy 0, policy_version 127609 (0.0007) [2023-12-26 16:17:43,877][105692] Updated weights for policy 0, policy_version 127619 (0.0005) [2023-12-26 16:17:44,419][105620] Updated weights for policy 1, policy_version 128174 (0.0007) [2023-12-26 16:17:44,471][105620] Updated weights for policy 1, policy_version 128184 (0.0007) [2023-12-26 16:17:44,530][105620] Updated weights for policy 1, policy_version 128194 (0.0005) [2023-12-26 16:17:44,603][105692] Updated weights for policy 0, policy_version 127629 (0.0008) [2023-12-26 16:17:44,658][105692] Updated weights for policy 0, policy_version 127639 (0.0010) [2023-12-26 16:17:44,708][105692] Updated weights for policy 0, policy_version 127649 (0.0009) [2023-12-26 16:17:45,223][105620] Updated weights for policy 1, policy_version 128204 (0.0007) [2023-12-26 16:17:45,285][105620] Updated weights for policy 1, policy_version 128214 (0.0009) [2023-12-26 16:17:45,341][105620] Updated weights for policy 1, policy_version 128224 (0.0009) [2023-12-26 16:17:45,466][105692] Updated weights for policy 0, policy_version 127660 (0.0008) [2023-12-26 16:17:45,510][105692] Updated weights for policy 0, policy_version 127670 (0.0005) [2023-12-26 16:17:45,559][105692] Updated weights for policy 0, policy_version 127680 (0.0005) [2023-12-26 16:17:46,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19251.1, 300 sec: 19494.2). Total num frames: 65527808. Throughput: 0: 9791.8, 1: 9524.2. Samples: 65500456. Policy #0 lag: (min: 22.0, avg: 23.4, max: 54.0) [2023-12-26 16:17:46,063][104569] Avg episode reward: [(0, '9348.179'), (1, '8816.117')] [2023-12-26 16:17:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000127688_32694272.pth... [2023-12-26 16:17:46,074][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000128232_32833536.pth... [2023-12-26 16:17:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000126504_32391168.pth [2023-12-26 16:17:46,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000127144_32555008.pth [2023-12-26 16:17:46,139][105692] Updated weights for policy 0, policy_version 127690 (0.0006) [2023-12-26 16:17:46,173][105620] Updated weights for policy 1, policy_version 128234 (0.0009) [2023-12-26 16:17:46,194][105692] Updated weights for policy 0, policy_version 127700 (0.0006) [2023-12-26 16:17:46,230][105620] Updated weights for policy 1, policy_version 128244 (0.0006) [2023-12-26 16:17:46,244][105692] Updated weights for policy 0, policy_version 127710 (0.0009) [2023-12-26 16:17:46,282][105620] Updated weights for policy 1, policy_version 128254 (0.0007) [2023-12-26 16:17:46,289][105692] Updated weights for policy 0, policy_version 127720 (0.0006) [2023-12-26 16:17:46,331][105620] Updated weights for policy 1, policy_version 128264 (0.0008) [2023-12-26 16:17:47,004][105692] Updated weights for policy 0, policy_version 127730 (0.0009) [2023-12-26 16:17:47,034][105620] Updated weights for policy 1, policy_version 128274 (0.0006) [2023-12-26 16:17:47,052][105692] Updated weights for policy 0, policy_version 127740 (0.0010) [2023-12-26 16:17:47,099][105620] Updated weights for policy 1, policy_version 128284 (0.0007) [2023-12-26 16:17:47,110][105692] Updated weights for policy 0, policy_version 127750 (0.0010) [2023-12-26 16:17:47,157][105620] Updated weights for policy 1, policy_version 128294 (0.0008) [2023-12-26 16:17:47,759][105692] Updated weights for policy 0, policy_version 127760 (0.0007) [2023-12-26 16:17:47,819][105692] Updated weights for policy 0, policy_version 127770 (0.0006) [2023-12-26 16:17:47,881][105692] Updated weights for policy 0, policy_version 127780 (0.0005) [2023-12-26 16:17:47,941][105620] Updated weights for policy 1, policy_version 128304 (0.0007) [2023-12-26 16:17:48,011][105620] Updated weights for policy 1, policy_version 128314 (0.0008) [2023-12-26 16:17:48,076][105620] Updated weights for policy 1, policy_version 128324 (0.0008) [2023-12-26 16:17:48,476][105692] Updated weights for policy 0, policy_version 127790 (0.0008) [2023-12-26 16:17:48,529][105692] Updated weights for policy 0, policy_version 127800 (0.0009) [2023-12-26 16:17:48,576][105692] Updated weights for policy 0, policy_version 127810 (0.0008) [2023-12-26 16:17:48,825][105620] Updated weights for policy 1, policy_version 128334 (0.0007) [2023-12-26 16:17:48,878][105620] Updated weights for policy 1, policy_version 128344 (0.0005) [2023-12-26 16:17:48,939][105620] Updated weights for policy 1, policy_version 128354 (0.0006) [2023-12-26 16:17:49,459][105692] Updated weights for policy 0, policy_version 127820 (0.0009) [2023-12-26 16:17:49,512][105692] Updated weights for policy 0, policy_version 127830 (0.0009) [2023-12-26 16:17:49,542][105620] Updated weights for policy 1, policy_version 128364 (0.0006) [2023-12-26 16:17:49,570][105692] Updated weights for policy 0, policy_version 127840 (0.0009) [2023-12-26 16:17:49,610][105620] Updated weights for policy 1, policy_version 128374 (0.0006) [2023-12-26 16:17:49,669][105620] Updated weights for policy 1, policy_version 128384 (0.0009) [2023-12-26 16:17:50,351][105620] Updated weights for policy 1, policy_version 128394 (0.0009) [2023-12-26 16:17:50,389][105692] Updated weights for policy 0, policy_version 127850 (0.0008) [2023-12-26 16:17:50,404][105620] Updated weights for policy 1, policy_version 128404 (0.0008) [2023-12-26 16:17:50,443][105692] Updated weights for policy 0, policy_version 127860 (0.0008) [2023-12-26 16:17:50,461][105620] Updated weights for policy 1, policy_version 128414 (0.0007) [2023-12-26 16:17:50,503][105692] Updated weights for policy 0, policy_version 127870 (0.0007) [2023-12-26 16:17:50,521][105620] Updated weights for policy 1, policy_version 128424 (0.0006) [2023-12-26 16:17:50,554][105692] Updated weights for policy 0, policy_version 127880 (0.0008) [2023-12-26 16:17:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 65626112. Throughput: 0: 9864.6, 1: 9462.9. Samples: 65618036. Policy #0 lag: (min: 22.0, avg: 23.4, max: 54.0) [2023-12-26 16:17:51,062][104569] Avg episode reward: [(0, '9256.889'), (1, '8905.397')] [2023-12-26 16:17:51,234][105620] Updated weights for policy 1, policy_version 128434 (0.0009) [2023-12-26 16:17:51,294][105620] Updated weights for policy 1, policy_version 128444 (0.0008) [2023-12-26 16:17:51,338][105692] Updated weights for policy 0, policy_version 127890 (0.0007) [2023-12-26 16:17:51,344][105620] Updated weights for policy 1, policy_version 128454 (0.0006) [2023-12-26 16:17:51,406][105692] Updated weights for policy 0, policy_version 127900 (0.0008) [2023-12-26 16:17:51,458][105692] Updated weights for policy 0, policy_version 127910 (0.0005) [2023-12-26 16:17:52,114][105692] Updated weights for policy 0, policy_version 127920 (0.0008) [2023-12-26 16:17:52,184][105692] Updated weights for policy 0, policy_version 127930 (0.0005) [2023-12-26 16:17:52,200][105620] Updated weights for policy 1, policy_version 128464 (0.0009) [2023-12-26 16:17:52,248][105692] Updated weights for policy 0, policy_version 127940 (0.0007) [2023-12-26 16:17:52,259][105620] Updated weights for policy 1, policy_version 128474 (0.0008) [2023-12-26 16:17:52,323][105620] Updated weights for policy 1, policy_version 128484 (0.0008) [2023-12-26 16:17:52,946][105692] Updated weights for policy 0, policy_version 127950 (0.0008) [2023-12-26 16:17:53,009][105692] Updated weights for policy 0, policy_version 127960 (0.0009) [2023-12-26 16:17:53,073][105692] Updated weights for policy 0, policy_version 127970 (0.0008) [2023-12-26 16:17:53,079][105620] Updated weights for policy 1, policy_version 128494 (0.0007) [2023-12-26 16:17:53,137][105620] Updated weights for policy 1, policy_version 128504 (0.0007) [2023-12-26 16:17:53,195][105620] Updated weights for policy 1, policy_version 128514 (0.0009) [2023-12-26 16:17:53,858][105620] Updated weights for policy 1, policy_version 128524 (0.0009) [2023-12-26 16:17:53,868][105692] Updated weights for policy 0, policy_version 127980 (0.0007) [2023-12-26 16:17:53,913][105620] Updated weights for policy 1, policy_version 128534 (0.0011) [2023-12-26 16:17:53,928][105692] Updated weights for policy 0, policy_version 127990 (0.0005) [2023-12-26 16:17:53,962][105620] Updated weights for policy 1, policy_version 128544 (0.0010) [2023-12-26 16:17:53,981][105692] Updated weights for policy 0, policy_version 128000 (0.0006) [2023-12-26 16:17:54,579][105620] Updated weights for policy 1, policy_version 128554 (0.0009) [2023-12-26 16:17:54,635][105620] Updated weights for policy 1, policy_version 128564 (0.0007) [2023-12-26 16:17:54,693][105620] Updated weights for policy 1, policy_version 128574 (0.0007) [2023-12-26 16:17:54,747][105620] Updated weights for policy 1, policy_version 128584 (0.0005) [2023-12-26 16:17:54,768][105692] Updated weights for policy 0, policy_version 128010 (0.0007) [2023-12-26 16:17:54,822][105692] Updated weights for policy 0, policy_version 128020 (0.0005) [2023-12-26 16:17:54,876][105692] Updated weights for policy 0, policy_version 128030 (0.0005) [2023-12-26 16:17:54,927][105692] Updated weights for policy 0, policy_version 128040 (0.0005) [2023-12-26 16:17:55,451][105620] Updated weights for policy 1, policy_version 128594 (0.0005) [2023-12-26 16:17:55,519][105620] Updated weights for policy 1, policy_version 128604 (0.0005) [2023-12-26 16:17:55,574][105620] Updated weights for policy 1, policy_version 128614 (0.0005) [2023-12-26 16:17:55,685][105692] Updated weights for policy 0, policy_version 128050 (0.0010) [2023-12-26 16:17:55,745][105692] Updated weights for policy 0, policy_version 128060 (0.0009) [2023-12-26 16:17:55,811][105692] Updated weights for policy 0, policy_version 128070 (0.0009) [2023-12-26 16:17:56,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 65724416. Throughput: 0: 9872.8, 1: 9535.0. Samples: 65733972. Policy #0 lag: (min: 22.0, avg: 23.4, max: 54.0) [2023-12-26 16:17:56,063][104569] Avg episode reward: [(0, '9256.907'), (1, '8818.000')] [2023-12-26 16:17:56,120][105620] Updated weights for policy 1, policy_version 128624 (0.0005) [2023-12-26 16:17:56,171][105620] Updated weights for policy 1, policy_version 128634 (0.0006) [2023-12-26 16:17:56,221][105620] Updated weights for policy 1, policy_version 128644 (0.0005) [2023-12-26 16:17:56,607][105692] Updated weights for policy 0, policy_version 128080 (0.0010) [2023-12-26 16:17:56,661][105692] Updated weights for policy 0, policy_version 128090 (0.0010) [2023-12-26 16:17:56,712][105692] Updated weights for policy 0, policy_version 128100 (0.0009) [2023-12-26 16:17:56,736][105620] Updated weights for policy 1, policy_version 128654 (0.0008) [2023-12-26 16:17:56,796][105620] Updated weights for policy 1, policy_version 128664 (0.0010) [2023-12-26 16:17:56,851][105620] Updated weights for policy 1, policy_version 128674 (0.0006) [2023-12-26 16:17:57,400][105620] Updated weights for policy 1, policy_version 128684 (0.0007) [2023-12-26 16:17:57,453][105620] Updated weights for policy 1, policy_version 128694 (0.0009) [2023-12-26 16:17:57,513][105620] Updated weights for policy 1, policy_version 128704 (0.0005) [2023-12-26 16:17:57,607][105692] Updated weights for policy 0, policy_version 128110 (0.0007) [2023-12-26 16:17:57,659][105692] Updated weights for policy 0, policy_version 128120 (0.0006) [2023-12-26 16:17:57,713][105692] Updated weights for policy 0, policy_version 128131 (0.0010) [2023-12-26 16:17:58,035][105620] Updated weights for policy 1, policy_version 128714 (0.0005) [2023-12-26 16:17:58,107][105620] Updated weights for policy 1, policy_version 128724 (0.0007) [2023-12-26 16:17:58,174][105620] Updated weights for policy 1, policy_version 128734 (0.0011) [2023-12-26 16:17:58,236][105620] Updated weights for policy 1, policy_version 128744 (0.0011) [2023-12-26 16:17:58,520][105692] Updated weights for policy 0, policy_version 128141 (0.0008) [2023-12-26 16:17:58,578][105692] Updated weights for policy 0, policy_version 128151 (0.0008) [2023-12-26 16:17:58,643][105692] Updated weights for policy 0, policy_version 128161 (0.0008) [2023-12-26 16:17:58,945][105620] Updated weights for policy 1, policy_version 128754 (0.0006) [2023-12-26 16:17:59,011][105620] Updated weights for policy 1, policy_version 128764 (0.0006) [2023-12-26 16:17:59,076][105620] Updated weights for policy 1, policy_version 128774 (0.0006) [2023-12-26 16:17:59,455][105692] Updated weights for policy 0, policy_version 128171 (0.0008) [2023-12-26 16:17:59,508][105692] Updated weights for policy 0, policy_version 128181 (0.0009) [2023-12-26 16:17:59,560][105692] Updated weights for policy 0, policy_version 128191 (0.0010) [2023-12-26 16:17:59,668][105620] Updated weights for policy 1, policy_version 128784 (0.0006) [2023-12-26 16:17:59,715][105620] Updated weights for policy 1, policy_version 128794 (0.0006) [2023-12-26 16:17:59,769][105620] Updated weights for policy 1, policy_version 128804 (0.0009) [2023-12-26 16:18:00,381][105692] Updated weights for policy 0, policy_version 128201 (0.0009) [2023-12-26 16:18:00,415][105620] Updated weights for policy 1, policy_version 128814 (0.0007) [2023-12-26 16:18:00,439][105692] Updated weights for policy 0, policy_version 128211 (0.0009) [2023-12-26 16:18:00,474][105620] Updated weights for policy 1, policy_version 128824 (0.0007) [2023-12-26 16:18:00,486][105692] Updated weights for policy 0, policy_version 128221 (0.0007) [2023-12-26 16:18:00,528][105620] Updated weights for policy 1, policy_version 128834 (0.0005) [2023-12-26 16:18:00,534][105692] Updated weights for policy 0, policy_version 128231 (0.0008) [2023-12-26 16:18:01,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 65822720. Throughput: 0: 9863.4, 1: 9624.6. Samples: 65794312. Policy #0 lag: (min: 22.0, avg: 23.4, max: 54.0) [2023-12-26 16:18:01,063][104569] Avg episode reward: [(0, '9256.862'), (1, '8815.816')] [2023-12-26 16:18:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000128232_32833536.pth... [2023-12-26 16:18:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000128840_32989184.pth... [2023-12-26 16:18:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000127112_32546816.pth [2023-12-26 16:18:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000127688_32694272.pth [2023-12-26 16:18:01,116][105620] Updated weights for policy 1, policy_version 128844 (0.0006) [2023-12-26 16:18:01,178][105620] Updated weights for policy 1, policy_version 128854 (0.0009) [2023-12-26 16:18:01,233][105620] Updated weights for policy 1, policy_version 128864 (0.0009) [2023-12-26 16:18:01,379][105692] Updated weights for policy 0, policy_version 128241 (0.0010) [2023-12-26 16:18:01,436][105692] Updated weights for policy 0, policy_version 128251 (0.0009) [2023-12-26 16:18:01,497][105692] Updated weights for policy 0, policy_version 128261 (0.0009) [2023-12-26 16:18:02,029][105620] Updated weights for policy 1, policy_version 128874 (0.0009) [2023-12-26 16:18:02,087][105620] Updated weights for policy 1, policy_version 128884 (0.0009) [2023-12-26 16:18:02,136][105620] Updated weights for policy 1, policy_version 128894 (0.0008) [2023-12-26 16:18:02,187][105620] Updated weights for policy 1, policy_version 128904 (0.0009) [2023-12-26 16:18:02,251][105692] Updated weights for policy 0, policy_version 128271 (0.0009) [2023-12-26 16:18:02,308][105692] Updated weights for policy 0, policy_version 128281 (0.0006) [2023-12-26 16:18:02,376][105692] Updated weights for policy 0, policy_version 128291 (0.0008) [2023-12-26 16:18:02,852][105620] Updated weights for policy 1, policy_version 128914 (0.0008) [2023-12-26 16:18:02,906][105620] Updated weights for policy 1, policy_version 128924 (0.0009) [2023-12-26 16:18:02,960][105620] Updated weights for policy 1, policy_version 128934 (0.0009) [2023-12-26 16:18:03,126][105692] Updated weights for policy 0, policy_version 128301 (0.0009) [2023-12-26 16:18:03,180][105692] Updated weights for policy 0, policy_version 128311 (0.0009) [2023-12-26 16:18:03,230][105692] Updated weights for policy 0, policy_version 128321 (0.0009) [2023-12-26 16:18:03,655][105620] Updated weights for policy 1, policy_version 128944 (0.0009) [2023-12-26 16:18:03,708][105620] Updated weights for policy 1, policy_version 128954 (0.0008) [2023-12-26 16:18:03,775][105620] Updated weights for policy 1, policy_version 128964 (0.0006) [2023-12-26 16:18:04,018][105692] Updated weights for policy 0, policy_version 128331 (0.0009) [2023-12-26 16:18:04,089][105692] Updated weights for policy 0, policy_version 128341 (0.0009) [2023-12-26 16:18:04,151][105692] Updated weights for policy 0, policy_version 128351 (0.0009) [2023-12-26 16:18:04,442][105620] Updated weights for policy 1, policy_version 128974 (0.0006) [2023-12-26 16:18:04,511][105620] Updated weights for policy 1, policy_version 128984 (0.0006) [2023-12-26 16:18:04,578][105620] Updated weights for policy 1, policy_version 128994 (0.0008) [2023-12-26 16:18:05,002][105692] Updated weights for policy 0, policy_version 128361 (0.0010) [2023-12-26 16:18:05,056][105692] Updated weights for policy 0, policy_version 128372 (0.0010) [2023-12-26 16:18:05,113][105620] Updated weights for policy 1, policy_version 129004 (0.0005) [2023-12-26 16:18:05,116][105692] Updated weights for policy 0, policy_version 128382 (0.0010) [2023-12-26 16:18:05,161][105692] Updated weights for policy 0, policy_version 128392 (0.0008) [2023-12-26 16:18:05,161][105620] Updated weights for policy 1, policy_version 129014 (0.0007) [2023-12-26 16:18:05,223][105620] Updated weights for policy 1, policy_version 129024 (0.0010) [2023-12-26 16:18:05,854][105620] Updated weights for policy 1, policy_version 129034 (0.0008) [2023-12-26 16:18:05,915][105620] Updated weights for policy 1, policy_version 129044 (0.0008) [2023-12-26 16:18:05,970][105620] Updated weights for policy 1, policy_version 129054 (0.0007) [2023-12-26 16:18:05,993][105692] Updated weights for policy 0, policy_version 128402 (0.0007) [2023-12-26 16:18:06,032][105620] Updated weights for policy 1, policy_version 129064 (0.0007) [2023-12-26 16:18:06,036][105692] Updated weights for policy 0, policy_version 128412 (0.0008) [2023-12-26 16:18:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 65921024. Throughput: 0: 9717.2, 1: 9712.7. Samples: 65910004. Policy #0 lag: (min: 22.0, avg: 23.4, max: 54.0) [2023-12-26 16:18:06,063][104569] Avg episode reward: [(0, '9080.748'), (1, '8907.810')] [2023-12-26 16:18:06,089][105692] Updated weights for policy 0, policy_version 128422 (0.0009) [2023-12-26 16:18:06,681][105620] Updated weights for policy 1, policy_version 129074 (0.0006) [2023-12-26 16:18:06,751][105620] Updated weights for policy 1, policy_version 129084 (0.0005) [2023-12-26 16:18:06,819][105620] Updated weights for policy 1, policy_version 129094 (0.0005) [2023-12-26 16:18:06,968][105692] Updated weights for policy 0, policy_version 128432 (0.0009) [2023-12-26 16:18:07,024][105692] Updated weights for policy 0, policy_version 128442 (0.0009) [2023-12-26 16:18:07,086][105692] Updated weights for policy 0, policy_version 128452 (0.0009) [2023-12-26 16:18:07,425][105620] Updated weights for policy 1, policy_version 129104 (0.0005) [2023-12-26 16:18:07,484][105620] Updated weights for policy 1, policy_version 129114 (0.0005) [2023-12-26 16:18:07,548][105620] Updated weights for policy 1, policy_version 129124 (0.0005) [2023-12-26 16:18:07,935][105692] Updated weights for policy 0, policy_version 128462 (0.0009) [2023-12-26 16:18:08,002][105692] Updated weights for policy 0, policy_version 128472 (0.0009) [2023-12-26 16:18:08,065][105692] Updated weights for policy 0, policy_version 128482 (0.0008) [2023-12-26 16:18:08,089][105620] Updated weights for policy 1, policy_version 129134 (0.0008) [2023-12-26 16:18:08,146][105620] Updated weights for policy 1, policy_version 129144 (0.0009) [2023-12-26 16:18:08,203][105620] Updated weights for policy 1, policy_version 129154 (0.0009) [2023-12-26 16:18:08,771][105692] Updated weights for policy 0, policy_version 128492 (0.0009) [2023-12-26 16:18:08,837][105692] Updated weights for policy 0, policy_version 128502 (0.0009) [2023-12-26 16:18:08,906][105692] Updated weights for policy 0, policy_version 128512 (0.0008) [2023-12-26 16:18:08,994][105620] Updated weights for policy 1, policy_version 129164 (0.0009) [2023-12-26 16:18:09,059][105620] Updated weights for policy 1, policy_version 129174 (0.0006) [2023-12-26 16:18:09,105][105620] Updated weights for policy 1, policy_version 129184 (0.0005) [2023-12-26 16:18:09,702][105692] Updated weights for policy 0, policy_version 128522 (0.0008) [2023-12-26 16:18:09,714][105620] Updated weights for policy 1, policy_version 129194 (0.0005) [2023-12-26 16:18:09,763][105692] Updated weights for policy 0, policy_version 128532 (0.0009) [2023-12-26 16:18:09,781][105620] Updated weights for policy 1, policy_version 129204 (0.0005) [2023-12-26 16:18:09,827][105692] Updated weights for policy 0, policy_version 128542 (0.0009) [2023-12-26 16:18:09,846][105620] Updated weights for policy 1, policy_version 129214 (0.0007) [2023-12-26 16:18:09,886][105692] Updated weights for policy 0, policy_version 128552 (0.0007) [2023-12-26 16:18:09,911][105620] Updated weights for policy 1, policy_version 129224 (0.0009) [2023-12-26 16:18:10,641][105620] Updated weights for policy 1, policy_version 129234 (0.0007) [2023-12-26 16:18:10,667][105692] Updated weights for policy 0, policy_version 128562 (0.0008) [2023-12-26 16:18:10,698][105620] Updated weights for policy 1, policy_version 129244 (0.0006) [2023-12-26 16:18:10,729][105692] Updated weights for policy 0, policy_version 128572 (0.0007) [2023-12-26 16:18:10,754][105620] Updated weights for policy 1, policy_version 129254 (0.0006) [2023-12-26 16:18:10,793][105692] Updated weights for policy 0, policy_version 128582 (0.0009) [2023-12-26 16:18:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 66019328. Throughput: 0: 9604.8, 1: 9897.1. Samples: 66024960. Policy #0 lag: (min: 22.0, avg: 23.4, max: 54.0) [2023-12-26 16:18:11,063][104569] Avg episode reward: [(0, '9078.586'), (1, '8909.267')] [2023-12-26 16:18:11,498][105620] Updated weights for policy 1, policy_version 129264 (0.0008) [2023-12-26 16:18:11,555][105620] Updated weights for policy 1, policy_version 129274 (0.0010) [2023-12-26 16:18:11,598][105692] Updated weights for policy 0, policy_version 128592 (0.0008) [2023-12-26 16:18:11,613][105620] Updated weights for policy 1, policy_version 129284 (0.0008) [2023-12-26 16:18:11,668][105692] Updated weights for policy 0, policy_version 128602 (0.0009) [2023-12-26 16:18:11,739][105692] Updated weights for policy 0, policy_version 128612 (0.0009) [2023-12-26 16:18:12,413][105692] Updated weights for policy 0, policy_version 128622 (0.0009) [2023-12-26 16:18:12,462][105620] Updated weights for policy 1, policy_version 129294 (0.0008) [2023-12-26 16:18:12,469][105692] Updated weights for policy 0, policy_version 128632 (0.0007) [2023-12-26 16:18:12,516][105620] Updated weights for policy 1, policy_version 129304 (0.0006) [2023-12-26 16:18:12,519][105692] Updated weights for policy 0, policy_version 128642 (0.0006) [2023-12-26 16:18:12,567][105620] Updated weights for policy 1, policy_version 129314 (0.0007) [2023-12-26 16:18:13,286][105692] Updated weights for policy 0, policy_version 128652 (0.0009) [2023-12-26 16:18:13,321][105620] Updated weights for policy 1, policy_version 129324 (0.0009) [2023-12-26 16:18:13,342][105692] Updated weights for policy 0, policy_version 128662 (0.0008) [2023-12-26 16:18:13,370][105620] Updated weights for policy 1, policy_version 129334 (0.0007) [2023-12-26 16:18:13,395][105692] Updated weights for policy 0, policy_version 128672 (0.0007) [2023-12-26 16:18:13,420][105620] Updated weights for policy 1, policy_version 129344 (0.0005) [2023-12-26 16:18:14,076][105620] Updated weights for policy 1, policy_version 129354 (0.0008) [2023-12-26 16:18:14,085][105692] Updated weights for policy 0, policy_version 128682 (0.0007) [2023-12-26 16:18:14,126][105620] Updated weights for policy 1, policy_version 129364 (0.0005) [2023-12-26 16:18:14,148][105692] Updated weights for policy 0, policy_version 128692 (0.0009) [2023-12-26 16:18:14,173][105620] Updated weights for policy 1, policy_version 129374 (0.0006) [2023-12-26 16:18:14,204][105692] Updated weights for policy 0, policy_version 128702 (0.0009) [2023-12-26 16:18:14,221][105620] Updated weights for policy 1, policy_version 129384 (0.0006) [2023-12-26 16:18:14,259][105692] Updated weights for policy 0, policy_version 128712 (0.0009) [2023-12-26 16:18:14,881][105692] Updated weights for policy 0, policy_version 128722 (0.0009) [2023-12-26 16:18:14,946][105692] Updated weights for policy 0, policy_version 128732 (0.0009) [2023-12-26 16:18:14,997][105620] Updated weights for policy 1, policy_version 129394 (0.0008) [2023-12-26 16:18:15,011][105692] Updated weights for policy 0, policy_version 128742 (0.0009) [2023-12-26 16:18:15,056][105620] Updated weights for policy 1, policy_version 129404 (0.0007) [2023-12-26 16:18:15,122][105620] Updated weights for policy 1, policy_version 129414 (0.0009) [2023-12-26 16:18:15,700][105692] Updated weights for policy 0, policy_version 128752 (0.0009) [2023-12-26 16:18:15,761][105692] Updated weights for policy 0, policy_version 128762 (0.0008) [2023-12-26 16:18:15,826][105692] Updated weights for policy 0, policy_version 128772 (0.0008) [2023-12-26 16:18:15,910][105620] Updated weights for policy 1, policy_version 129424 (0.0009) [2023-12-26 16:18:15,962][105620] Updated weights for policy 1, policy_version 129434 (0.0009) [2023-12-26 16:18:16,016][105620] Updated weights for policy 1, policy_version 129444 (0.0009) [2023-12-26 16:18:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 66117632. Throughput: 0: 9539.7, 1: 9922.6. Samples: 66081216. Policy #0 lag: (min: 22.0, avg: 23.4, max: 54.0) [2023-12-26 16:18:16,062][104569] Avg episode reward: [(0, '9076.583'), (1, '8905.895')] [2023-12-26 16:18:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000128776_32972800.pth... [2023-12-26 16:18:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000129448_33144832.pth... [2023-12-26 16:18:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000128232_32833536.pth [2023-12-26 16:18:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000127688_32694272.pth [2023-12-26 16:18:16,462][105692] Updated weights for policy 0, policy_version 128782 (0.0006) [2023-12-26 16:18:16,524][105692] Updated weights for policy 0, policy_version 128792 (0.0006) [2023-12-26 16:18:16,577][105692] Updated weights for policy 0, policy_version 128802 (0.0005) [2023-12-26 16:18:16,897][105620] Updated weights for policy 1, policy_version 129454 (0.0009) [2023-12-26 16:18:16,953][105620] Updated weights for policy 1, policy_version 129465 (0.0010) [2023-12-26 16:18:17,001][105620] Updated weights for policy 1, policy_version 129475 (0.0009) [2023-12-26 16:18:17,147][105692] Updated weights for policy 0, policy_version 128812 (0.0005) [2023-12-26 16:18:17,207][105692] Updated weights for policy 0, policy_version 128822 (0.0008) [2023-12-26 16:18:17,269][105692] Updated weights for policy 0, policy_version 128832 (0.0010) [2023-12-26 16:18:17,719][105620] Updated weights for policy 1, policy_version 129485 (0.0007) [2023-12-26 16:18:17,780][105620] Updated weights for policy 1, policy_version 129495 (0.0005) [2023-12-26 16:18:17,837][105620] Updated weights for policy 1, policy_version 129505 (0.0005) [2023-12-26 16:18:17,913][105692] Updated weights for policy 0, policy_version 128842 (0.0009) [2023-12-26 16:18:17,968][105692] Updated weights for policy 0, policy_version 128852 (0.0010) [2023-12-26 16:18:18,026][105692] Updated weights for policy 0, policy_version 128862 (0.0009) [2023-12-26 16:18:18,079][105692] Updated weights for policy 0, policy_version 128872 (0.0010) [2023-12-26 16:18:18,484][105620] Updated weights for policy 1, policy_version 129515 (0.0007) [2023-12-26 16:18:18,540][105620] Updated weights for policy 1, policy_version 129525 (0.0011) [2023-12-26 16:18:18,603][105620] Updated weights for policy 1, policy_version 129535 (0.0011) [2023-12-26 16:18:18,708][105692] Updated weights for policy 0, policy_version 128882 (0.0006) [2023-12-26 16:18:18,764][105692] Updated weights for policy 0, policy_version 128892 (0.0010) [2023-12-26 16:18:18,813][105692] Updated weights for policy 0, policy_version 128902 (0.0010) [2023-12-26 16:18:19,386][105620] Updated weights for policy 1, policy_version 129545 (0.0011) [2023-12-26 16:18:19,442][105620] Updated weights for policy 1, policy_version 129555 (0.0009) [2023-12-26 16:18:19,502][105620] Updated weights for policy 1, policy_version 129565 (0.0009) [2023-12-26 16:18:19,534][105692] Updated weights for policy 0, policy_version 128912 (0.0008) [2023-12-26 16:18:19,562][105620] Updated weights for policy 1, policy_version 129575 (0.0007) [2023-12-26 16:18:19,592][105692] Updated weights for policy 0, policy_version 128922 (0.0008) [2023-12-26 16:18:19,646][105692] Updated weights for policy 0, policy_version 128932 (0.0008) [2023-12-26 16:18:20,349][105620] Updated weights for policy 1, policy_version 129585 (0.0009) [2023-12-26 16:18:20,397][105692] Updated weights for policy 0, policy_version 128942 (0.0008) [2023-12-26 16:18:20,411][105620] Updated weights for policy 1, policy_version 129595 (0.0008) [2023-12-26 16:18:20,447][105692] Updated weights for policy 0, policy_version 128952 (0.0006) [2023-12-26 16:18:20,473][105620] Updated weights for policy 1, policy_version 129605 (0.0009) [2023-12-26 16:18:20,495][105692] Updated weights for policy 0, policy_version 128962 (0.0005) [2023-12-26 16:18:21,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 66207744. Throughput: 0: 9603.8, 1: 9828.9. Samples: 66200296. Policy #0 lag: (min: 22.0, avg: 23.4, max: 54.0) [2023-12-26 16:18:21,062][104569] Avg episode reward: [(0, '9162.713'), (1, '8721.996')] [2023-12-26 16:18:21,210][105692] Updated weights for policy 0, policy_version 128972 (0.0006) [2023-12-26 16:18:21,270][105692] Updated weights for policy 0, policy_version 128982 (0.0008) [2023-12-26 16:18:21,329][105620] Updated weights for policy 1, policy_version 129615 (0.0010) [2023-12-26 16:18:21,333][105692] Updated weights for policy 0, policy_version 128992 (0.0008) [2023-12-26 16:18:21,400][105620] Updated weights for policy 1, policy_version 129625 (0.0010) [2023-12-26 16:18:21,461][105620] Updated weights for policy 1, policy_version 129635 (0.0011) [2023-12-26 16:18:22,115][105692] Updated weights for policy 0, policy_version 129002 (0.0008) [2023-12-26 16:18:22,168][105692] Updated weights for policy 0, policy_version 129012 (0.0008) [2023-12-26 16:18:22,219][105620] Updated weights for policy 1, policy_version 129645 (0.0011) [2023-12-26 16:18:22,222][105692] Updated weights for policy 0, policy_version 129022 (0.0008) [2023-12-26 16:18:22,280][105620] Updated weights for policy 1, policy_version 129655 (0.0009) [2023-12-26 16:18:22,283][105692] Updated weights for policy 0, policy_version 129032 (0.0007) [2023-12-26 16:18:22,344][105620] Updated weights for policy 1, policy_version 129665 (0.0010) [2023-12-26 16:18:22,994][105620] Updated weights for policy 1, policy_version 129675 (0.0009) [2023-12-26 16:18:23,055][105620] Updated weights for policy 1, policy_version 129685 (0.0006) [2023-12-26 16:18:23,125][105620] Updated weights for policy 1, policy_version 129695 (0.0005) [2023-12-26 16:18:23,130][105692] Updated weights for policy 0, policy_version 129042 (0.0009) [2023-12-26 16:18:23,192][105692] Updated weights for policy 0, policy_version 129052 (0.0008) [2023-12-26 16:18:23,245][105692] Updated weights for policy 0, policy_version 129062 (0.0009) [2023-12-26 16:18:23,703][105620] Updated weights for policy 1, policy_version 129705 (0.0009) [2023-12-26 16:18:23,754][105620] Updated weights for policy 1, policy_version 129715 (0.0010) [2023-12-26 16:18:23,815][105620] Updated weights for policy 1, policy_version 129725 (0.0010) [2023-12-26 16:18:23,879][105620] Updated weights for policy 1, policy_version 129735 (0.0010) [2023-12-26 16:18:24,068][105692] Updated weights for policy 0, policy_version 129072 (0.0011) [2023-12-26 16:18:24,135][105692] Updated weights for policy 0, policy_version 129082 (0.0011) [2023-12-26 16:18:24,191][105692] Updated weights for policy 0, policy_version 129092 (0.0010) [2023-12-26 16:18:24,640][105620] Updated weights for policy 1, policy_version 129745 (0.0011) [2023-12-26 16:18:24,704][105620] Updated weights for policy 1, policy_version 129755 (0.0011) [2023-12-26 16:18:24,763][105620] Updated weights for policy 1, policy_version 129765 (0.0011) [2023-12-26 16:18:24,959][105692] Updated weights for policy 0, policy_version 129102 (0.0010) [2023-12-26 16:18:25,014][105692] Updated weights for policy 0, policy_version 129112 (0.0010) [2023-12-26 16:18:25,056][105692] Updated weights for policy 0, policy_version 129122 (0.0009) [2023-12-26 16:18:25,514][105620] Updated weights for policy 1, policy_version 129775 (0.0011) [2023-12-26 16:18:25,562][105620] Updated weights for policy 1, policy_version 129785 (0.0010) [2023-12-26 16:18:25,618][105620] Updated weights for policy 1, policy_version 129795 (0.0010) [2023-12-26 16:18:25,714][105692] Updated weights for policy 0, policy_version 129132 (0.0007) [2023-12-26 16:18:25,758][105692] Updated weights for policy 0, policy_version 129142 (0.0010) [2023-12-26 16:18:25,806][105692] Updated weights for policy 0, policy_version 129152 (0.0010) [2023-12-26 16:18:26,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 66306048. Throughput: 0: 9399.7, 1: 9927.1. Samples: 66312856. Policy #0 lag: (min: 22.0, avg: 23.4, max: 54.0) [2023-12-26 16:18:26,063][104569] Avg episode reward: [(0, '9253.375'), (1, '8649.193')] [2023-12-26 16:18:26,354][105620] Updated weights for policy 1, policy_version 129805 (0.0010) [2023-12-26 16:18:26,419][105620] Updated weights for policy 1, policy_version 129815 (0.0010) [2023-12-26 16:18:26,467][105620] Updated weights for policy 1, policy_version 129825 (0.0010) [2023-12-26 16:18:26,540][105692] Updated weights for policy 0, policy_version 129162 (0.0010) [2023-12-26 16:18:26,591][105692] Updated weights for policy 0, policy_version 129172 (0.0008) [2023-12-26 16:18:26,636][105692] Updated weights for policy 0, policy_version 129182 (0.0008) [2023-12-26 16:18:26,686][105692] Updated weights for policy 0, policy_version 129192 (0.0008) [2023-12-26 16:18:27,206][105620] Updated weights for policy 1, policy_version 129835 (0.0009) [2023-12-26 16:18:27,268][105620] Updated weights for policy 1, policy_version 129845 (0.0005) [2023-12-26 16:18:27,325][105620] Updated weights for policy 1, policy_version 129855 (0.0006) [2023-12-26 16:18:27,432][105692] Updated weights for policy 0, policy_version 129202 (0.0008) [2023-12-26 16:18:27,476][105692] Updated weights for policy 0, policy_version 129212 (0.0007) [2023-12-26 16:18:27,520][105692] Updated weights for policy 0, policy_version 129222 (0.0008) [2023-12-26 16:18:28,012][105620] Updated weights for policy 1, policy_version 129866 (0.0010) [2023-12-26 16:18:28,062][105620] Updated weights for policy 1, policy_version 129876 (0.0007) [2023-12-26 16:18:28,115][105620] Updated weights for policy 1, policy_version 129886 (0.0006) [2023-12-26 16:18:28,166][105620] Updated weights for policy 1, policy_version 129896 (0.0005) [2023-12-26 16:18:28,385][105692] Updated weights for policy 0, policy_version 129232 (0.0008) [2023-12-26 16:18:28,436][105692] Updated weights for policy 0, policy_version 129242 (0.0009) [2023-12-26 16:18:28,492][105692] Updated weights for policy 0, policy_version 129252 (0.0008) [2023-12-26 16:18:28,750][105620] Updated weights for policy 1, policy_version 129906 (0.0005) [2023-12-26 16:18:28,807][105620] Updated weights for policy 1, policy_version 129916 (0.0005) [2023-12-26 16:18:28,864][105620] Updated weights for policy 1, policy_version 129926 (0.0006) [2023-12-26 16:18:29,381][105692] Updated weights for policy 0, policy_version 129262 (0.0010) [2023-12-26 16:18:29,402][105620] Updated weights for policy 1, policy_version 129936 (0.0005) [2023-12-26 16:18:29,427][105692] Updated weights for policy 0, policy_version 129272 (0.0008) [2023-12-26 16:18:29,458][105620] Updated weights for policy 1, policy_version 129946 (0.0007) [2023-12-26 16:18:29,480][105692] Updated weights for policy 0, policy_version 129282 (0.0006) [2023-12-26 16:18:29,514][105620] Updated weights for policy 1, policy_version 129956 (0.0007) [2023-12-26 16:18:30,199][105692] Updated weights for policy 0, policy_version 129292 (0.0006) [2023-12-26 16:18:30,255][105692] Updated weights for policy 0, policy_version 129302 (0.0006) [2023-12-26 16:18:30,278][105620] Updated weights for policy 1, policy_version 129966 (0.0008) [2023-12-26 16:18:30,304][105692] Updated weights for policy 0, policy_version 129312 (0.0006) [2023-12-26 16:18:30,330][105620] Updated weights for policy 1, policy_version 129976 (0.0008) [2023-12-26 16:18:30,387][105620] Updated weights for policy 1, policy_version 129986 (0.0010) [2023-12-26 16:18:30,961][105692] Updated weights for policy 0, policy_version 129322 (0.0006) [2023-12-26 16:18:31,029][105692] Updated weights for policy 0, policy_version 129332 (0.0008) [2023-12-26 16:18:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19438.7). Total num frames: 66396160. Throughput: 0: 9353.1, 1: 10004.5. Samples: 66371544. Policy #0 lag: (min: 22.0, avg: 23.4, max: 54.0) [2023-12-26 16:18:31,062][104569] Avg episode reward: [(0, '9253.999'), (1, '8655.941')] [2023-12-26 16:18:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000129992_33284096.pth... [2023-12-26 16:18:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000128840_32989184.pth [2023-12-26 16:18:31,092][105692] Updated weights for policy 0, policy_version 129342 (0.0009) [2023-12-26 16:18:31,108][105620] Updated weights for policy 1, policy_version 129996 (0.0009) [2023-12-26 16:18:31,154][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000129352_33120256.pth... [2023-12-26 16:18:31,156][105692] Updated weights for policy 0, policy_version 129352 (0.0009) [2023-12-26 16:18:31,159][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000128232_32833536.pth [2023-12-26 16:18:31,171][105620] Updated weights for policy 1, policy_version 130006 (0.0008) [2023-12-26 16:18:31,228][105620] Updated weights for policy 1, policy_version 130016 (0.0005) [2023-12-26 16:18:31,869][105620] Updated weights for policy 1, policy_version 130026 (0.0007) [2023-12-26 16:18:31,936][105620] Updated weights for policy 1, policy_version 130036 (0.0007) [2023-12-26 16:18:31,968][105692] Updated weights for policy 0, policy_version 129362 (0.0007) [2023-12-26 16:18:32,005][105620] Updated weights for policy 1, policy_version 130046 (0.0006) [2023-12-26 16:18:32,015][105692] Updated weights for policy 0, policy_version 129372 (0.0007) [2023-12-26 16:18:32,073][105620] Updated weights for policy 1, policy_version 130056 (0.0006) [2023-12-26 16:18:32,075][105692] Updated weights for policy 0, policy_version 129382 (0.0008) [2023-12-26 16:18:32,696][105620] Updated weights for policy 1, policy_version 130066 (0.0007) [2023-12-26 16:18:32,708][105692] Updated weights for policy 0, policy_version 129392 (0.0006) [2023-12-26 16:18:32,760][105620] Updated weights for policy 1, policy_version 130076 (0.0005) [2023-12-26 16:18:32,774][105692] Updated weights for policy 0, policy_version 129402 (0.0005) [2023-12-26 16:18:32,819][105620] Updated weights for policy 1, policy_version 130086 (0.0008) [2023-12-26 16:18:32,824][105692] Updated weights for policy 0, policy_version 129412 (0.0006) [2023-12-26 16:18:33,451][105692] Updated weights for policy 0, policy_version 129422 (0.0007) [2023-12-26 16:18:33,508][105692] Updated weights for policy 0, policy_version 129432 (0.0008) [2023-12-26 16:18:33,541][105620] Updated weights for policy 1, policy_version 130096 (0.0009) [2023-12-26 16:18:33,563][105692] Updated weights for policy 0, policy_version 129442 (0.0006) [2023-12-26 16:18:33,586][105620] Updated weights for policy 1, policy_version 130106 (0.0010) [2023-12-26 16:18:33,633][105620] Updated weights for policy 1, policy_version 130116 (0.0010) [2023-12-26 16:18:34,300][105692] Updated weights for policy 0, policy_version 129452 (0.0008) [2023-12-26 16:18:34,357][105692] Updated weights for policy 0, policy_version 129462 (0.0008) [2023-12-26 16:18:34,414][105692] Updated weights for policy 0, policy_version 129472 (0.0008) [2023-12-26 16:18:34,424][105620] Updated weights for policy 1, policy_version 130126 (0.0009) [2023-12-26 16:18:34,486][105620] Updated weights for policy 1, policy_version 130136 (0.0008) [2023-12-26 16:18:34,556][105620] Updated weights for policy 1, policy_version 130146 (0.0008) [2023-12-26 16:18:35,161][105692] Updated weights for policy 0, policy_version 129482 (0.0009) [2023-12-26 16:18:35,220][105692] Updated weights for policy 0, policy_version 129492 (0.0010) [2023-12-26 16:18:35,271][105620] Updated weights for policy 1, policy_version 130156 (0.0009) [2023-12-26 16:18:35,277][105692] Updated weights for policy 0, policy_version 129502 (0.0008) [2023-12-26 16:18:35,335][105620] Updated weights for policy 1, policy_version 130166 (0.0005) [2023-12-26 16:18:35,343][105692] Updated weights for policy 0, policy_version 129512 (0.0009) [2023-12-26 16:18:35,404][105620] Updated weights for policy 1, policy_version 130176 (0.0005) [2023-12-26 16:18:35,991][105620] Updated weights for policy 1, policy_version 130186 (0.0005) [2023-12-26 16:18:36,042][105620] Updated weights for policy 1, policy_version 130196 (0.0005) [2023-12-26 16:18:36,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 66494464. Throughput: 0: 9322.1, 1: 10035.7. Samples: 66489140. Policy #0 lag: (min: 22.0, avg: 23.4, max: 54.0) [2023-12-26 16:18:36,062][104569] Avg episode reward: [(0, '9254.340'), (1, '8998.019')] [2023-12-26 16:18:36,102][105620] Updated weights for policy 1, policy_version 130206 (0.0005) [2023-12-26 16:18:36,167][105620] Updated weights for policy 1, policy_version 130216 (0.0007) [2023-12-26 16:18:36,187][105692] Updated weights for policy 0, policy_version 129522 (0.0008) [2023-12-26 16:18:36,240][105692] Updated weights for policy 0, policy_version 129532 (0.0009) [2023-12-26 16:18:36,299][105692] Updated weights for policy 0, policy_version 129542 (0.0010) [2023-12-26 16:18:36,690][105620] Updated weights for policy 1, policy_version 130226 (0.0005) [2023-12-26 16:18:36,745][105620] Updated weights for policy 1, policy_version 130236 (0.0005) [2023-12-26 16:18:36,809][105620] Updated weights for policy 1, policy_version 130246 (0.0005) [2023-12-26 16:18:37,194][105692] Updated weights for policy 0, policy_version 129552 (0.0008) [2023-12-26 16:18:37,249][105692] Updated weights for policy 0, policy_version 129562 (0.0008) [2023-12-26 16:18:37,316][105692] Updated weights for policy 0, policy_version 129572 (0.0008) [2023-12-26 16:18:37,445][105620] Updated weights for policy 1, policy_version 130256 (0.0006) [2023-12-26 16:18:37,499][105620] Updated weights for policy 1, policy_version 130266 (0.0005) [2023-12-26 16:18:37,559][105620] Updated weights for policy 1, policy_version 130276 (0.0006) [2023-12-26 16:18:38,126][105692] Updated weights for policy 0, policy_version 129582 (0.0009) [2023-12-26 16:18:38,185][105692] Updated weights for policy 0, policy_version 129592 (0.0009) [2023-12-26 16:18:38,233][105620] Updated weights for policy 1, policy_version 130286 (0.0008) [2023-12-26 16:18:38,235][105692] Updated weights for policy 0, policy_version 129602 (0.0008) [2023-12-26 16:18:38,280][105620] Updated weights for policy 1, policy_version 130296 (0.0006) [2023-12-26 16:18:38,338][105620] Updated weights for policy 1, policy_version 130306 (0.0009) [2023-12-26 16:18:38,975][105692] Updated weights for policy 0, policy_version 129612 (0.0008) [2023-12-26 16:18:39,030][105692] Updated weights for policy 0, policy_version 129622 (0.0009) [2023-12-26 16:18:39,087][105692] Updated weights for policy 0, policy_version 129632 (0.0009) [2023-12-26 16:18:39,119][105620] Updated weights for policy 1, policy_version 130316 (0.0008) [2023-12-26 16:18:39,177][105620] Updated weights for policy 1, policy_version 130326 (0.0007) [2023-12-26 16:18:39,246][105620] Updated weights for policy 1, policy_version 130336 (0.0008) [2023-12-26 16:18:39,876][105692] Updated weights for policy 0, policy_version 129642 (0.0008) [2023-12-26 16:18:39,939][105692] Updated weights for policy 0, policy_version 129652 (0.0009) [2023-12-26 16:18:40,002][105692] Updated weights for policy 0, policy_version 129662 (0.0009) [2023-12-26 16:18:40,012][105620] Updated weights for policy 1, policy_version 130346 (0.0008) [2023-12-26 16:18:40,061][105692] Updated weights for policy 0, policy_version 129672 (0.0008) [2023-12-26 16:18:40,074][105620] Updated weights for policy 1, policy_version 130356 (0.0007) [2023-12-26 16:18:40,136][105620] Updated weights for policy 1, policy_version 130366 (0.0008) [2023-12-26 16:18:40,186][105620] Updated weights for policy 1, policy_version 130376 (0.0008) [2023-12-26 16:18:40,839][105692] Updated weights for policy 0, policy_version 129682 (0.0009) [2023-12-26 16:18:40,895][105692] Updated weights for policy 0, policy_version 129692 (0.0009) [2023-12-26 16:18:40,927][105620] Updated weights for policy 1, policy_version 130386 (0.0007) [2023-12-26 16:18:40,945][105692] Updated weights for policy 0, policy_version 129702 (0.0007) [2023-12-26 16:18:40,977][105620] Updated weights for policy 1, policy_version 130396 (0.0008) [2023-12-26 16:18:41,026][105620] Updated weights for policy 1, policy_version 130406 (0.0009) [2023-12-26 16:18:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 66600960. Throughput: 0: 9255.5, 1: 10057.0. Samples: 66603032. Policy #0 lag: (min: 22.0, avg: 23.4, max: 54.0) [2023-12-26 16:18:41,062][104569] Avg episode reward: [(0, '9254.296'), (1, '8906.566')] [2023-12-26 16:18:41,762][105692] Updated weights for policy 0, policy_version 129712 (0.0007) [2023-12-26 16:18:41,823][105620] Updated weights for policy 1, policy_version 130416 (0.0008) [2023-12-26 16:18:41,830][105692] Updated weights for policy 0, policy_version 129722 (0.0005) [2023-12-26 16:18:41,880][105620] Updated weights for policy 1, policy_version 130426 (0.0008) [2023-12-26 16:18:41,892][105692] Updated weights for policy 0, policy_version 129732 (0.0005) [2023-12-26 16:18:41,944][105620] Updated weights for policy 1, policy_version 130436 (0.0009) [2023-12-26 16:18:42,590][105692] Updated weights for policy 0, policy_version 129742 (0.0009) [2023-12-26 16:18:42,652][105692] Updated weights for policy 0, policy_version 129752 (0.0009) [2023-12-26 16:18:42,672][105620] Updated weights for policy 1, policy_version 130446 (0.0007) [2023-12-26 16:18:42,706][105692] Updated weights for policy 0, policy_version 129762 (0.0009) [2023-12-26 16:18:42,732][105620] Updated weights for policy 1, policy_version 130456 (0.0007) [2023-12-26 16:18:42,793][105620] Updated weights for policy 1, policy_version 130466 (0.0008) [2023-12-26 16:18:43,440][105620] Updated weights for policy 1, policy_version 130476 (0.0008) [2023-12-26 16:18:43,503][105620] Updated weights for policy 1, policy_version 130486 (0.0005) [2023-12-26 16:18:43,532][105692] Updated weights for policy 0, policy_version 129772 (0.0009) [2023-12-26 16:18:43,559][105620] Updated weights for policy 1, policy_version 130496 (0.0005) [2023-12-26 16:18:43,587][105692] Updated weights for policy 0, policy_version 129782 (0.0009) [2023-12-26 16:18:43,639][105692] Updated weights for policy 0, policy_version 129793 (0.0009) [2023-12-26 16:18:44,142][105620] Updated weights for policy 1, policy_version 130506 (0.0006) [2023-12-26 16:18:44,189][105620] Updated weights for policy 1, policy_version 130516 (0.0008) [2023-12-26 16:18:44,240][105620] Updated weights for policy 1, policy_version 130526 (0.0009) [2023-12-26 16:18:44,289][105620] Updated weights for policy 1, policy_version 130536 (0.0008) [2023-12-26 16:18:44,477][105692] Updated weights for policy 0, policy_version 129804 (0.0009) [2023-12-26 16:18:44,530][105692] Updated weights for policy 0, policy_version 129814 (0.0006) [2023-12-26 16:18:44,587][105692] Updated weights for policy 0, policy_version 129825 (0.0010) [2023-12-26 16:18:45,011][105620] Updated weights for policy 1, policy_version 130546 (0.0009) [2023-12-26 16:18:45,075][105620] Updated weights for policy 1, policy_version 130556 (0.0008) [2023-12-26 16:18:45,141][105620] Updated weights for policy 1, policy_version 130566 (0.0009) [2023-12-26 16:18:45,312][105692] Updated weights for policy 0, policy_version 129836 (0.0009) [2023-12-26 16:18:45,376][105692] Updated weights for policy 0, policy_version 129846 (0.0009) [2023-12-26 16:18:45,440][105692] Updated weights for policy 0, policy_version 129856 (0.0009) [2023-12-26 16:18:45,889][105620] Updated weights for policy 1, policy_version 130576 (0.0009) [2023-12-26 16:18:45,949][105620] Updated weights for policy 1, policy_version 130586 (0.0008) [2023-12-26 16:18:45,996][105620] Updated weights for policy 1, policy_version 130596 (0.0009) [2023-12-26 16:18:46,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 66691072. Throughput: 0: 9284.1, 1: 9955.5. Samples: 66660096. Policy #0 lag: (min: 22.0, avg: 23.4, max: 54.0) [2023-12-26 16:18:46,063][104569] Avg episode reward: [(0, '7399.524'), (1, '8987.371')] [2023-12-26 16:18:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000129864_33251328.pth... [2023-12-26 16:18:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000130600_33439744.pth... [2023-12-26 16:18:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000128776_32972800.pth [2023-12-26 16:18:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000129448_33144832.pth [2023-12-26 16:18:46,203][105692] Updated weights for policy 0, policy_version 129866 (0.0009) [2023-12-26 16:18:46,262][105692] Updated weights for policy 0, policy_version 129876 (0.0009) [2023-12-26 16:18:46,316][105692] Updated weights for policy 0, policy_version 129886 (0.0009) [2023-12-26 16:18:46,378][105692] Updated weights for policy 0, policy_version 129896 (0.0009) [2023-12-26 16:18:46,752][105620] Updated weights for policy 1, policy_version 130606 (0.0009) [2023-12-26 16:18:46,809][105620] Updated weights for policy 1, policy_version 130616 (0.0010) [2023-12-26 16:18:46,862][105620] Updated weights for policy 1, policy_version 130626 (0.0010) [2023-12-26 16:18:47,081][105692] Updated weights for policy 0, policy_version 129906 (0.0009) [2023-12-26 16:18:47,143][105692] Updated weights for policy 0, policy_version 129916 (0.0009) [2023-12-26 16:18:47,206][105692] Updated weights for policy 0, policy_version 129926 (0.0009) [2023-12-26 16:18:47,644][105620] Updated weights for policy 1, policy_version 130636 (0.0010) [2023-12-26 16:18:47,695][105620] Updated weights for policy 1, policy_version 130646 (0.0008) [2023-12-26 16:18:47,751][105620] Updated weights for policy 1, policy_version 130656 (0.0008) [2023-12-26 16:18:47,933][105692] Updated weights for policy 0, policy_version 129936 (0.0008) [2023-12-26 16:18:47,988][105692] Updated weights for policy 0, policy_version 129946 (0.0009) [2023-12-26 16:18:48,048][105692] Updated weights for policy 0, policy_version 129956 (0.0007) [2023-12-26 16:18:48,528][105620] Updated weights for policy 1, policy_version 130666 (0.0010) [2023-12-26 16:18:48,585][105620] Updated weights for policy 1, policy_version 130676 (0.0008) [2023-12-26 16:18:48,637][105620] Updated weights for policy 1, policy_version 130686 (0.0009) [2023-12-26 16:18:48,696][105620] Updated weights for policy 1, policy_version 130696 (0.0009) [2023-12-26 16:18:48,761][105692] Updated weights for policy 0, policy_version 129966 (0.0009) [2023-12-26 16:18:48,820][105692] Updated weights for policy 0, policy_version 129976 (0.0009) [2023-12-26 16:18:48,872][105692] Updated weights for policy 0, policy_version 129986 (0.0009) [2023-12-26 16:18:49,506][105620] Updated weights for policy 1, policy_version 130706 (0.0008) [2023-12-26 16:18:49,556][105620] Updated weights for policy 1, policy_version 130716 (0.0008) [2023-12-26 16:18:49,571][105692] Updated weights for policy 0, policy_version 129996 (0.0008) [2023-12-26 16:18:49,622][105620] Updated weights for policy 1, policy_version 130726 (0.0009) [2023-12-26 16:18:49,625][105692] Updated weights for policy 0, policy_version 130006 (0.0011) [2023-12-26 16:18:49,681][105692] Updated weights for policy 0, policy_version 130016 (0.0011) [2023-12-26 16:18:50,395][105620] Updated weights for policy 1, policy_version 130736 (0.0008) [2023-12-26 16:18:50,468][105620] Updated weights for policy 1, policy_version 130746 (0.0010) [2023-12-26 16:18:50,475][105692] Updated weights for policy 0, policy_version 130026 (0.0010) [2023-12-26 16:18:50,526][105620] Updated weights for policy 1, policy_version 130756 (0.0009) [2023-12-26 16:18:50,528][105692] Updated weights for policy 0, policy_version 130036 (0.0008) [2023-12-26 16:18:50,591][105692] Updated weights for policy 0, policy_version 130046 (0.0007) [2023-12-26 16:18:50,650][105692] Updated weights for policy 0, policy_version 130056 (0.0009) [2023-12-26 16:18:51,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19251.2, 300 sec: 19438.7). Total num frames: 66781184. Throughput: 0: 9373.8, 1: 9793.4. Samples: 66772528. Policy #0 lag: (min: 1.0, avg: 24.4, max: 33.0) [2023-12-26 16:18:51,062][104569] Avg episode reward: [(0, '5352.330'), (1, '8928.017')] [2023-12-26 16:18:51,334][105620] Updated weights for policy 1, policy_version 130766 (0.0006) [2023-12-26 16:18:51,401][105620] Updated weights for policy 1, policy_version 130776 (0.0008) [2023-12-26 16:18:51,421][105692] Updated weights for policy 0, policy_version 130066 (0.0008) [2023-12-26 16:18:51,453][105620] Updated weights for policy 1, policy_version 130786 (0.0006) [2023-12-26 16:18:51,468][105692] Updated weights for policy 0, policy_version 130076 (0.0006) [2023-12-26 16:18:51,528][105692] Updated weights for policy 0, policy_version 130086 (0.0007) [2023-12-26 16:18:52,219][105620] Updated weights for policy 1, policy_version 130796 (0.0008) [2023-12-26 16:18:52,285][105620] Updated weights for policy 1, policy_version 130806 (0.0008) [2023-12-26 16:18:52,291][105692] Updated weights for policy 0, policy_version 130096 (0.0009) [2023-12-26 16:18:52,357][105692] Updated weights for policy 0, policy_version 130106 (0.0008) [2023-12-26 16:18:52,364][105620] Updated weights for policy 1, policy_version 130816 (0.0007) [2023-12-26 16:18:52,413][105692] Updated weights for policy 0, policy_version 130116 (0.0008) [2023-12-26 16:18:53,107][105620] Updated weights for policy 1, policy_version 130826 (0.0007) [2023-12-26 16:18:53,151][105692] Updated weights for policy 0, policy_version 130126 (0.0007) [2023-12-26 16:18:53,162][105620] Updated weights for policy 1, policy_version 130836 (0.0009) [2023-12-26 16:18:53,200][105692] Updated weights for policy 0, policy_version 130136 (0.0008) [2023-12-26 16:18:53,210][105620] Updated weights for policy 1, policy_version 130846 (0.0007) [2023-12-26 16:18:53,249][105692] Updated weights for policy 0, policy_version 130146 (0.0006) [2023-12-26 16:18:53,255][105620] Updated weights for policy 1, policy_version 130856 (0.0006) [2023-12-26 16:18:53,893][105692] Updated weights for policy 0, policy_version 130156 (0.0008) [2023-12-26 16:18:53,947][105692] Updated weights for policy 0, policy_version 130166 (0.0009) [2023-12-26 16:18:54,002][105692] Updated weights for policy 0, policy_version 130176 (0.0009) [2023-12-26 16:18:54,078][105620] Updated weights for policy 1, policy_version 130866 (0.0008) [2023-12-26 16:18:54,135][105620] Updated weights for policy 1, policy_version 130876 (0.0008) [2023-12-26 16:18:54,196][105620] Updated weights for policy 1, policy_version 130886 (0.0009) [2023-12-26 16:18:54,670][105692] Updated weights for policy 0, policy_version 130186 (0.0007) [2023-12-26 16:18:54,724][105692] Updated weights for policy 0, policy_version 130196 (0.0008) [2023-12-26 16:18:54,775][105692] Updated weights for policy 0, policy_version 130206 (0.0009) [2023-12-26 16:18:54,829][105692] Updated weights for policy 0, policy_version 130216 (0.0005) [2023-12-26 16:18:55,035][105620] Updated weights for policy 1, policy_version 130896 (0.0009) [2023-12-26 16:18:55,090][105620] Updated weights for policy 1, policy_version 130906 (0.0008) [2023-12-26 16:18:55,146][105620] Updated weights for policy 1, policy_version 130916 (0.0008) [2023-12-26 16:18:55,448][105692] Updated weights for policy 0, policy_version 130226 (0.0006) [2023-12-26 16:18:55,514][105692] Updated weights for policy 0, policy_version 130236 (0.0005) [2023-12-26 16:18:55,579][105692] Updated weights for policy 0, policy_version 130246 (0.0006) [2023-12-26 16:18:56,006][105620] Updated weights for policy 1, policy_version 130926 (0.0008) [2023-12-26 16:18:56,062][104569] Fps is (10 sec: 18022.7, 60 sec: 19114.7, 300 sec: 19410.9). Total num frames: 66871296. Throughput: 0: 9532.8, 1: 9570.5. Samples: 66884608. Policy #0 lag: (min: 1.0, avg: 24.4, max: 33.0) [2023-12-26 16:18:56,063][104569] Avg episode reward: [(0, '7034.862'), (1, '8498.549')] [2023-12-26 16:18:56,065][105620] Updated weights for policy 1, policy_version 130936 (0.0008) [2023-12-26 16:18:56,125][105620] Updated weights for policy 1, policy_version 130946 (0.0008) [2023-12-26 16:18:56,158][105692] Updated weights for policy 0, policy_version 130256 (0.0010) [2023-12-26 16:18:56,205][105692] Updated weights for policy 0, policy_version 130266 (0.0010) [2023-12-26 16:18:56,249][105692] Updated weights for policy 0, policy_version 130276 (0.0010) [2023-12-26 16:18:56,827][105620] Updated weights for policy 1, policy_version 130956 (0.0007) [2023-12-26 16:18:56,882][105620] Updated weights for policy 1, policy_version 130966 (0.0008) [2023-12-26 16:18:56,936][105620] Updated weights for policy 1, policy_version 130976 (0.0008) [2023-12-26 16:18:57,015][105692] Updated weights for policy 0, policy_version 130286 (0.0010) [2023-12-26 16:18:57,066][105692] Updated weights for policy 0, policy_version 130296 (0.0010) [2023-12-26 16:18:57,131][105692] Updated weights for policy 0, policy_version 130306 (0.0010) [2023-12-26 16:18:57,588][105620] Updated weights for policy 1, policy_version 130986 (0.0008) [2023-12-26 16:18:57,643][105620] Updated weights for policy 1, policy_version 130996 (0.0009) [2023-12-26 16:18:57,701][105620] Updated weights for policy 1, policy_version 131006 (0.0009) [2023-12-26 16:18:57,758][105620] Updated weights for policy 1, policy_version 131016 (0.0010) [2023-12-26 16:18:57,808][105692] Updated weights for policy 0, policy_version 130316 (0.0008) [2023-12-26 16:18:57,858][105692] Updated weights for policy 0, policy_version 130326 (0.0005) [2023-12-26 16:18:57,911][105692] Updated weights for policy 0, policy_version 130336 (0.0005) [2023-12-26 16:18:58,542][105692] Updated weights for policy 0, policy_version 130346 (0.0006) [2023-12-26 16:18:58,605][105692] Updated weights for policy 0, policy_version 130356 (0.0010) [2023-12-26 16:18:58,616][105620] Updated weights for policy 1, policy_version 131026 (0.0008) [2023-12-26 16:18:58,669][105692] Updated weights for policy 0, policy_version 130366 (0.0011) [2023-12-26 16:18:58,678][105620] Updated weights for policy 1, policy_version 131036 (0.0008) [2023-12-26 16:18:58,734][105692] Updated weights for policy 0, policy_version 130376 (0.0010) [2023-12-26 16:18:58,744][105620] Updated weights for policy 1, policy_version 131046 (0.0008) [2023-12-26 16:18:59,359][105620] Updated weights for policy 1, policy_version 131056 (0.0008) [2023-12-26 16:18:59,412][105620] Updated weights for policy 1, policy_version 131066 (0.0005) [2023-12-26 16:18:59,463][105620] Updated weights for policy 1, policy_version 131076 (0.0008) [2023-12-26 16:18:59,487][105692] Updated weights for policy 0, policy_version 130386 (0.0008) [2023-12-26 16:18:59,553][105692] Updated weights for policy 0, policy_version 130396 (0.0007) [2023-12-26 16:18:59,611][105692] Updated weights for policy 0, policy_version 130406 (0.0010) [2023-12-26 16:19:00,197][105620] Updated weights for policy 1, policy_version 131086 (0.0009) [2023-12-26 16:19:00,244][105620] Updated weights for policy 1, policy_version 131096 (0.0008) [2023-12-26 16:19:00,299][105620] Updated weights for policy 1, policy_version 131106 (0.0007) [2023-12-26 16:19:00,318][105692] Updated weights for policy 0, policy_version 130416 (0.0008) [2023-12-26 16:19:00,367][105692] Updated weights for policy 0, policy_version 130426 (0.0008) [2023-12-26 16:19:00,419][105692] Updated weights for policy 0, policy_version 130436 (0.0009) [2023-12-26 16:19:01,039][105692] Updated weights for policy 0, policy_version 130446 (0.0007) [2023-12-26 16:19:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19114.7, 300 sec: 19410.9). Total num frames: 66969600. Throughput: 0: 9601.1, 1: 9564.2. Samples: 66943660. Policy #0 lag: (min: 1.0, avg: 24.4, max: 33.0) [2023-12-26 16:19:01,063][104569] Avg episode reward: [(0, '9088.738'), (1, '8646.452')] [2023-12-26 16:19:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000131112_33570816.pth... [2023-12-26 16:19:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000129992_33284096.pth [2023-12-26 16:19:01,111][105692] Updated weights for policy 0, policy_version 130456 (0.0007) [2023-12-26 16:19:01,128][105620] Updated weights for policy 1, policy_version 131116 (0.0007) [2023-12-26 16:19:01,172][105692] Updated weights for policy 0, policy_version 130466 (0.0010) [2023-12-26 16:19:01,190][105620] Updated weights for policy 1, policy_version 131126 (0.0007) [2023-12-26 16:19:01,205][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000130472_33406976.pth... [2023-12-26 16:19:01,208][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000129352_33120256.pth [2023-12-26 16:19:01,251][105620] Updated weights for policy 1, policy_version 131136 (0.0008) [2023-12-26 16:19:01,900][105692] Updated weights for policy 0, policy_version 130476 (0.0009) [2023-12-26 16:19:01,954][105620] Updated weights for policy 1, policy_version 131146 (0.0007) [2023-12-26 16:19:01,957][105692] Updated weights for policy 0, policy_version 130486 (0.0008) [2023-12-26 16:19:02,010][105692] Updated weights for policy 0, policy_version 130496 (0.0007) [2023-12-26 16:19:02,018][105620] Updated weights for policy 1, policy_version 131156 (0.0007) [2023-12-26 16:19:02,082][105620] Updated weights for policy 1, policy_version 131166 (0.0008) [2023-12-26 16:19:02,139][105620] Updated weights for policy 1, policy_version 131176 (0.0008) [2023-12-26 16:19:02,658][105692] Updated weights for policy 0, policy_version 130506 (0.0007) [2023-12-26 16:19:02,719][105692] Updated weights for policy 0, policy_version 130516 (0.0005) [2023-12-26 16:19:02,768][105692] Updated weights for policy 0, policy_version 130526 (0.0005) [2023-12-26 16:19:02,817][105692] Updated weights for policy 0, policy_version 130536 (0.0005) [2023-12-26 16:19:02,911][105620] Updated weights for policy 1, policy_version 131186 (0.0009) [2023-12-26 16:19:02,963][105620] Updated weights for policy 1, policy_version 131196 (0.0010) [2023-12-26 16:19:03,012][105620] Updated weights for policy 1, policy_version 131206 (0.0009) [2023-12-26 16:19:03,413][105692] Updated weights for policy 0, policy_version 130547 (0.0009) [2023-12-26 16:19:03,457][105692] Updated weights for policy 0, policy_version 130557 (0.0010) [2023-12-26 16:19:03,511][105692] Updated weights for policy 0, policy_version 130567 (0.0008) [2023-12-26 16:19:03,802][105620] Updated weights for policy 1, policy_version 131216 (0.0006) [2023-12-26 16:19:03,864][105620] Updated weights for policy 1, policy_version 131226 (0.0006) [2023-12-26 16:19:03,930][105620] Updated weights for policy 1, policy_version 131236 (0.0006) [2023-12-26 16:19:04,165][105692] Updated weights for policy 0, policy_version 130577 (0.0011) [2023-12-26 16:19:04,235][105692] Updated weights for policy 0, policy_version 130587 (0.0011) [2023-12-26 16:19:04,295][105692] Updated weights for policy 0, policy_version 130597 (0.0011) [2023-12-26 16:19:04,580][105620] Updated weights for policy 1, policy_version 131246 (0.0009) [2023-12-26 16:19:04,639][105620] Updated weights for policy 1, policy_version 131256 (0.0011) [2023-12-26 16:19:04,690][105620] Updated weights for policy 1, policy_version 131266 (0.0010) [2023-12-26 16:19:04,903][105692] Updated weights for policy 0, policy_version 130607 (0.0010) [2023-12-26 16:19:04,958][105692] Updated weights for policy 0, policy_version 130617 (0.0010) [2023-12-26 16:19:05,016][105692] Updated weights for policy 0, policy_version 130627 (0.0010) [2023-12-26 16:19:05,443][105620] Updated weights for policy 1, policy_version 131276 (0.0010) [2023-12-26 16:19:05,495][105620] Updated weights for policy 1, policy_version 131286 (0.0010) [2023-12-26 16:19:05,546][105620] Updated weights for policy 1, policy_version 131296 (0.0010) [2023-12-26 16:19:05,618][105692] Updated weights for policy 0, policy_version 130637 (0.0010) [2023-12-26 16:19:05,676][105692] Updated weights for policy 0, policy_version 130647 (0.0011) [2023-12-26 16:19:05,739][105692] Updated weights for policy 0, policy_version 130657 (0.0011) [2023-12-26 16:19:06,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 67076096. Throughput: 0: 9541.9, 1: 9619.4. Samples: 67062560. Policy #0 lag: (min: 1.0, avg: 24.4, max: 33.0) [2023-12-26 16:19:06,063][104569] Avg episode reward: [(0, '9255.407'), (1, '8737.341')] [2023-12-26 16:19:06,318][105620] Updated weights for policy 1, policy_version 131306 (0.0011) [2023-12-26 16:19:06,378][105620] Updated weights for policy 1, policy_version 131316 (0.0011) [2023-12-26 16:19:06,432][105692] Updated weights for policy 0, policy_version 130667 (0.0009) [2023-12-26 16:19:06,440][105620] Updated weights for policy 1, policy_version 131326 (0.0010) [2023-12-26 16:19:06,498][105692] Updated weights for policy 0, policy_version 130677 (0.0007) [2023-12-26 16:19:06,499][105620] Updated weights for policy 1, policy_version 131336 (0.0008) [2023-12-26 16:19:06,553][105692] Updated weights for policy 0, policy_version 130687 (0.0006) [2023-12-26 16:19:07,103][105692] Updated weights for policy 0, policy_version 130697 (0.0010) [2023-12-26 16:19:07,163][105692] Updated weights for policy 0, policy_version 130707 (0.0010) [2023-12-26 16:19:07,223][105692] Updated weights for policy 0, policy_version 130717 (0.0011) [2023-12-26 16:19:07,281][105692] Updated weights for policy 0, policy_version 130727 (0.0010) [2023-12-26 16:19:07,333][105620] Updated weights for policy 1, policy_version 131346 (0.0010) [2023-12-26 16:19:07,395][105620] Updated weights for policy 1, policy_version 131356 (0.0010) [2023-12-26 16:19:07,452][105620] Updated weights for policy 1, policy_version 131366 (0.0010) [2023-12-26 16:19:08,039][105692] Updated weights for policy 0, policy_version 130737 (0.0008) [2023-12-26 16:19:08,097][105692] Updated weights for policy 0, policy_version 130747 (0.0008) [2023-12-26 16:19:08,130][105620] Updated weights for policy 1, policy_version 131376 (0.0010) [2023-12-26 16:19:08,147][105692] Updated weights for policy 0, policy_version 130757 (0.0008) [2023-12-26 16:19:08,184][105620] Updated weights for policy 1, policy_version 131386 (0.0010) [2023-12-26 16:19:08,245][105620] Updated weights for policy 1, policy_version 131396 (0.0010) [2023-12-26 16:19:08,867][105692] Updated weights for policy 0, policy_version 130767 (0.0005) [2023-12-26 16:19:08,930][105692] Updated weights for policy 0, policy_version 130777 (0.0008) [2023-12-26 16:19:08,966][105620] Updated weights for policy 1, policy_version 131406 (0.0010) [2023-12-26 16:19:08,984][105692] Updated weights for policy 0, policy_version 130787 (0.0007) [2023-12-26 16:19:09,027][105620] Updated weights for policy 1, policy_version 131416 (0.0007) [2023-12-26 16:19:09,079][105620] Updated weights for policy 1, policy_version 131426 (0.0010) [2023-12-26 16:19:09,689][105692] Updated weights for policy 0, policy_version 130797 (0.0007) [2023-12-26 16:19:09,746][105692] Updated weights for policy 0, policy_version 130807 (0.0007) [2023-12-26 16:19:09,798][105692] Updated weights for policy 0, policy_version 130817 (0.0005) [2023-12-26 16:19:09,854][105620] Updated weights for policy 1, policy_version 131436 (0.0010) [2023-12-26 16:19:09,915][105620] Updated weights for policy 1, policy_version 131446 (0.0011) [2023-12-26 16:19:09,972][105620] Updated weights for policy 1, policy_version 131456 (0.0010) [2023-12-26 16:19:10,528][105692] Updated weights for policy 0, policy_version 130827 (0.0009) [2023-12-26 16:19:10,582][105692] Updated weights for policy 0, policy_version 130837 (0.0008) [2023-12-26 16:19:10,643][105692] Updated weights for policy 0, policy_version 130847 (0.0007) [2023-12-26 16:19:10,717][105620] Updated weights for policy 1, policy_version 131466 (0.0010) [2023-12-26 16:19:10,765][105620] Updated weights for policy 1, policy_version 131476 (0.0010) [2023-12-26 16:19:10,814][105620] Updated weights for policy 1, policy_version 131486 (0.0007) [2023-12-26 16:19:10,862][105620] Updated weights for policy 1, policy_version 131496 (0.0005) [2023-12-26 16:19:11,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19251.2, 300 sec: 19438.7). Total num frames: 67174400. Throughput: 0: 9683.9, 1: 9591.7. Samples: 67180252. Policy #0 lag: (min: 1.0, avg: 24.4, max: 33.0) [2023-12-26 16:19:11,062][104569] Avg episode reward: [(0, '9167.839'), (1, '4063.088')] [2023-12-26 16:19:11,391][105692] Updated weights for policy 0, policy_version 130857 (0.0008) [2023-12-26 16:19:11,458][105692] Updated weights for policy 0, policy_version 130867 (0.0009) [2023-12-26 16:19:11,520][105692] Updated weights for policy 0, policy_version 130877 (0.0011) [2023-12-26 16:19:11,586][105692] Updated weights for policy 0, policy_version 130887 (0.0011) [2023-12-26 16:19:11,629][105620] Updated weights for policy 1, policy_version 131506 (0.0010) [2023-12-26 16:19:11,690][105620] Updated weights for policy 1, policy_version 131516 (0.0007) [2023-12-26 16:19:11,753][105620] Updated weights for policy 1, policy_version 131526 (0.0012) [2023-12-26 16:19:12,289][105692] Updated weights for policy 0, policy_version 130897 (0.0009) [2023-12-26 16:19:12,357][105692] Updated weights for policy 0, policy_version 130907 (0.0008) [2023-12-26 16:19:12,417][105692] Updated weights for policy 0, policy_version 130917 (0.0008) [2023-12-26 16:19:12,425][105620] Updated weights for policy 1, policy_version 131536 (0.0007) [2023-12-26 16:19:12,476][105620] Updated weights for policy 1, policy_version 131546 (0.0005) [2023-12-26 16:19:12,523][105620] Updated weights for policy 1, policy_version 131556 (0.0010) [2023-12-26 16:19:13,073][105692] Updated weights for policy 0, policy_version 130927 (0.0006) [2023-12-26 16:19:13,121][105692] Updated weights for policy 0, policy_version 130937 (0.0006) [2023-12-26 16:19:13,174][105692] Updated weights for policy 0, policy_version 130947 (0.0006) [2023-12-26 16:19:13,206][105620] Updated weights for policy 1, policy_version 131566 (0.0010) [2023-12-26 16:19:13,276][105620] Updated weights for policy 1, policy_version 131576 (0.0010) [2023-12-26 16:19:13,341][105620] Updated weights for policy 1, policy_version 131586 (0.0010) [2023-12-26 16:19:13,847][105692] Updated weights for policy 0, policy_version 130957 (0.0007) [2023-12-26 16:19:13,899][105692] Updated weights for policy 0, policy_version 130967 (0.0008) [2023-12-26 16:19:13,948][105692] Updated weights for policy 0, policy_version 130977 (0.0008) [2023-12-26 16:19:14,050][105620] Updated weights for policy 1, policy_version 131596 (0.0011) [2023-12-26 16:19:14,112][105620] Updated weights for policy 1, policy_version 131606 (0.0010) [2023-12-26 16:19:14,170][105620] Updated weights for policy 1, policy_version 131616 (0.0010) [2023-12-26 16:19:14,677][105692] Updated weights for policy 0, policy_version 130987 (0.0007) [2023-12-26 16:19:14,754][105692] Updated weights for policy 0, policy_version 130997 (0.0006) [2023-12-26 16:19:14,782][105620] Updated weights for policy 1, policy_version 131626 (0.0010) [2023-12-26 16:19:14,824][105692] Updated weights for policy 0, policy_version 131007 (0.0008) [2023-12-26 16:19:14,847][105620] Updated weights for policy 1, policy_version 131636 (0.0008) [2023-12-26 16:19:14,906][105620] Updated weights for policy 1, policy_version 131646 (0.0008) [2023-12-26 16:19:14,977][105620] Updated weights for policy 1, policy_version 131656 (0.0008) [2023-12-26 16:19:15,400][105692] Updated weights for policy 0, policy_version 131017 (0.0007) [2023-12-26 16:19:15,468][105692] Updated weights for policy 0, policy_version 131027 (0.0006) [2023-12-26 16:19:15,533][105692] Updated weights for policy 0, policy_version 131037 (0.0008) [2023-12-26 16:19:15,590][105692] Updated weights for policy 0, policy_version 131047 (0.0009) [2023-12-26 16:19:15,693][105620] Updated weights for policy 1, policy_version 131666 (0.0010) [2023-12-26 16:19:15,745][105620] Updated weights for policy 1, policy_version 131676 (0.0010) [2023-12-26 16:19:15,800][105620] Updated weights for policy 1, policy_version 131686 (0.0010) [2023-12-26 16:19:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.1, 300 sec: 19466.4). Total num frames: 67272704. Throughput: 0: 9737.6, 1: 9558.3. Samples: 67239868. Policy #0 lag: (min: 1.0, avg: 24.4, max: 33.0) [2023-12-26 16:19:16,063][104569] Avg episode reward: [(0, '8900.265'), (1, '5029.742')] [2023-12-26 16:19:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000131048_33554432.pth... [2023-12-26 16:19:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000131688_33718272.pth... [2023-12-26 16:19:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000130600_33439744.pth [2023-12-26 16:19:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000129864_33251328.pth [2023-12-26 16:19:16,312][105692] Updated weights for policy 0, policy_version 131057 (0.0009) [2023-12-26 16:19:16,359][105692] Updated weights for policy 0, policy_version 131067 (0.0009) [2023-12-26 16:19:16,413][105692] Updated weights for policy 0, policy_version 131077 (0.0009) [2023-12-26 16:19:16,494][105620] Updated weights for policy 1, policy_version 131696 (0.0009) [2023-12-26 16:19:16,541][105620] Updated weights for policy 1, policy_version 131706 (0.0009) [2023-12-26 16:19:16,604][105620] Updated weights for policy 1, policy_version 131716 (0.0008) [2023-12-26 16:19:17,112][105692] Updated weights for policy 0, policy_version 131087 (0.0008) [2023-12-26 16:19:17,164][105692] Updated weights for policy 0, policy_version 131097 (0.0009) [2023-12-26 16:19:17,210][105692] Updated weights for policy 0, policy_version 131107 (0.0007) [2023-12-26 16:19:17,413][105620] Updated weights for policy 1, policy_version 131726 (0.0009) [2023-12-26 16:19:17,468][105620] Updated weights for policy 1, policy_version 131736 (0.0009) [2023-12-26 16:19:17,526][105620] Updated weights for policy 1, policy_version 131746 (0.0008) [2023-12-26 16:19:17,947][105692] Updated weights for policy 0, policy_version 131117 (0.0008) [2023-12-26 16:19:18,014][105692] Updated weights for policy 0, policy_version 131127 (0.0008) [2023-12-26 16:19:18,082][105692] Updated weights for policy 0, policy_version 131137 (0.0010) [2023-12-26 16:19:18,285][105620] Updated weights for policy 1, policy_version 131756 (0.0009) [2023-12-26 16:19:18,337][105620] Updated weights for policy 1, policy_version 131766 (0.0009) [2023-12-26 16:19:18,403][105620] Updated weights for policy 1, policy_version 131776 (0.0006) [2023-12-26 16:19:18,822][105692] Updated weights for policy 0, policy_version 131147 (0.0009) [2023-12-26 16:19:18,883][105692] Updated weights for policy 0, policy_version 131157 (0.0009) [2023-12-26 16:19:18,942][105692] Updated weights for policy 0, policy_version 131167 (0.0009) [2023-12-26 16:19:19,100][105620] Updated weights for policy 1, policy_version 131786 (0.0006) [2023-12-26 16:19:19,159][105620] Updated weights for policy 1, policy_version 131796 (0.0006) [2023-12-26 16:19:19,234][105620] Updated weights for policy 1, policy_version 131806 (0.0009) [2023-12-26 16:19:19,300][105620] Updated weights for policy 1, policy_version 131816 (0.0009) [2023-12-26 16:19:19,778][105692] Updated weights for policy 0, policy_version 131177 (0.0009) [2023-12-26 16:19:19,842][105692] Updated weights for policy 0, policy_version 131187 (0.0009) [2023-12-26 16:19:19,911][105692] Updated weights for policy 0, policy_version 131197 (0.0009) [2023-12-26 16:19:19,975][105692] Updated weights for policy 0, policy_version 131207 (0.0008) [2023-12-26 16:19:19,986][105620] Updated weights for policy 1, policy_version 131826 (0.0006) [2023-12-26 16:19:20,046][105620] Updated weights for policy 1, policy_version 131836 (0.0009) [2023-12-26 16:19:20,110][105620] Updated weights for policy 1, policy_version 131846 (0.0009) [2023-12-26 16:19:20,736][105692] Updated weights for policy 0, policy_version 131217 (0.0006) [2023-12-26 16:19:20,794][105692] Updated weights for policy 0, policy_version 131227 (0.0006) [2023-12-26 16:19:20,848][105692] Updated weights for policy 0, policy_version 131237 (0.0006) [2023-12-26 16:19:20,943][105620] Updated weights for policy 1, policy_version 131856 (0.0009) [2023-12-26 16:19:21,004][105620] Updated weights for policy 1, policy_version 131866 (0.0008) [2023-12-26 16:19:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 67362816. Throughput: 0: 9739.5, 1: 9530.6. Samples: 67356296. Policy #0 lag: (min: 1.0, avg: 24.4, max: 33.0) [2023-12-26 16:19:21,063][104569] Avg episode reward: [(0, '8901.559'), (1, '6937.871')] [2023-12-26 16:19:21,074][105620] Updated weights for policy 1, policy_version 131876 (0.0008) [2023-12-26 16:19:21,571][105692] Updated weights for policy 0, policy_version 131247 (0.0008) [2023-12-26 16:19:21,638][105692] Updated weights for policy 0, policy_version 131257 (0.0009) [2023-12-26 16:19:21,698][105692] Updated weights for policy 0, policy_version 131267 (0.0009) [2023-12-26 16:19:21,848][105620] Updated weights for policy 1, policy_version 131886 (0.0008) [2023-12-26 16:19:21,905][105620] Updated weights for policy 1, policy_version 131896 (0.0010) [2023-12-26 16:19:21,972][105620] Updated weights for policy 1, policy_version 131906 (0.0010) [2023-12-26 16:19:22,329][105692] Updated weights for policy 0, policy_version 131277 (0.0007) [2023-12-26 16:19:22,396][105692] Updated weights for policy 0, policy_version 131287 (0.0007) [2023-12-26 16:19:22,459][105692] Updated weights for policy 0, policy_version 131297 (0.0008) [2023-12-26 16:19:22,819][105620] Updated weights for policy 1, policy_version 131916 (0.0009) [2023-12-26 16:19:22,876][105620] Updated weights for policy 1, policy_version 131926 (0.0010) [2023-12-26 16:19:22,941][105620] Updated weights for policy 1, policy_version 131936 (0.0009) [2023-12-26 16:19:23,070][105692] Updated weights for policy 0, policy_version 131307 (0.0008) [2023-12-26 16:19:23,119][105692] Updated weights for policy 0, policy_version 131317 (0.0006) [2023-12-26 16:19:23,180][105692] Updated weights for policy 0, policy_version 131327 (0.0005) [2023-12-26 16:19:23,704][105620] Updated weights for policy 1, policy_version 131946 (0.0008) [2023-12-26 16:19:23,749][105620] Updated weights for policy 1, policy_version 131956 (0.0009) [2023-12-26 16:19:23,795][105620] Updated weights for policy 1, policy_version 131966 (0.0009) [2023-12-26 16:19:23,845][105620] Updated weights for policy 1, policy_version 131976 (0.0008) [2023-12-26 16:19:23,873][105692] Updated weights for policy 0, policy_version 131337 (0.0008) [2023-12-26 16:19:23,924][105692] Updated weights for policy 0, policy_version 131347 (0.0009) [2023-12-26 16:19:23,979][105692] Updated weights for policy 0, policy_version 131357 (0.0009) [2023-12-26 16:19:24,027][105692] Updated weights for policy 0, policy_version 131367 (0.0009) [2023-12-26 16:19:24,605][105620] Updated weights for policy 1, policy_version 131986 (0.0007) [2023-12-26 16:19:24,659][105620] Updated weights for policy 1, policy_version 131996 (0.0006) [2023-12-26 16:19:24,714][105620] Updated weights for policy 1, policy_version 132006 (0.0006) [2023-12-26 16:19:24,783][105692] Updated weights for policy 0, policy_version 131377 (0.0008) [2023-12-26 16:19:24,848][105692] Updated weights for policy 0, policy_version 131387 (0.0008) [2023-12-26 16:19:24,917][105692] Updated weights for policy 0, policy_version 131397 (0.0007) [2023-12-26 16:19:25,425][105620] Updated weights for policy 1, policy_version 132016 (0.0007) [2023-12-26 16:19:25,476][105620] Updated weights for policy 1, policy_version 132026 (0.0008) [2023-12-26 16:19:25,534][105620] Updated weights for policy 1, policy_version 132036 (0.0009) [2023-12-26 16:19:25,558][105692] Updated weights for policy 0, policy_version 131407 (0.0011) [2023-12-26 16:19:25,623][105692] Updated weights for policy 0, policy_version 131417 (0.0009) [2023-12-26 16:19:25,685][105692] Updated weights for policy 0, policy_version 131427 (0.0010) [2023-12-26 16:19:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 67461120. Throughput: 0: 9895.2, 1: 9369.0. Samples: 67469920. Policy #0 lag: (min: 1.0, avg: 24.4, max: 33.0) [2023-12-26 16:19:26,063][104569] Avg episode reward: [(0, '8993.062'), (1, '8340.275')] [2023-12-26 16:19:26,310][105620] Updated weights for policy 1, policy_version 132046 (0.0009) [2023-12-26 16:19:26,372][105620] Updated weights for policy 1, policy_version 132056 (0.0010) [2023-12-26 16:19:26,394][105692] Updated weights for policy 0, policy_version 131437 (0.0010) [2023-12-26 16:19:26,435][105620] Updated weights for policy 1, policy_version 132066 (0.0008) [2023-12-26 16:19:26,450][105692] Updated weights for policy 0, policy_version 131447 (0.0007) [2023-12-26 16:19:26,507][105692] Updated weights for policy 0, policy_version 131457 (0.0008) [2023-12-26 16:19:27,197][105692] Updated weights for policy 0, policy_version 131467 (0.0008) [2023-12-26 16:19:27,212][105620] Updated weights for policy 1, policy_version 132076 (0.0009) [2023-12-26 16:19:27,250][105692] Updated weights for policy 0, policy_version 131477 (0.0008) [2023-12-26 16:19:27,258][105620] Updated weights for policy 1, policy_version 132086 (0.0007) [2023-12-26 16:19:27,301][105692] Updated weights for policy 0, policy_version 131487 (0.0006) [2023-12-26 16:19:27,316][105620] Updated weights for policy 1, policy_version 132096 (0.0008) [2023-12-26 16:19:27,909][105692] Updated weights for policy 0, policy_version 131497 (0.0010) [2023-12-26 16:19:27,978][105692] Updated weights for policy 0, policy_version 131507 (0.0005) [2023-12-26 16:19:28,024][105692] Updated weights for policy 0, policy_version 131517 (0.0006) [2023-12-26 16:19:28,092][105692] Updated weights for policy 0, policy_version 131527 (0.0006) [2023-12-26 16:19:28,163][105620] Updated weights for policy 1, policy_version 132106 (0.0006) [2023-12-26 16:19:28,229][105620] Updated weights for policy 1, policy_version 132116 (0.0009) [2023-12-26 16:19:28,297][105620] Updated weights for policy 1, policy_version 132126 (0.0010) [2023-12-26 16:19:28,366][105620] Updated weights for policy 1, policy_version 132136 (0.0009) [2023-12-26 16:19:28,640][105692] Updated weights for policy 0, policy_version 131537 (0.0009) [2023-12-26 16:19:28,701][105692] Updated weights for policy 0, policy_version 131547 (0.0009) [2023-12-26 16:19:28,751][105692] Updated weights for policy 0, policy_version 131557 (0.0008) [2023-12-26 16:19:29,146][105620] Updated weights for policy 1, policy_version 132146 (0.0008) [2023-12-26 16:19:29,204][105620] Updated weights for policy 1, policy_version 132156 (0.0008) [2023-12-26 16:19:29,269][105620] Updated weights for policy 1, policy_version 132166 (0.0008) [2023-12-26 16:19:29,490][105692] Updated weights for policy 0, policy_version 131567 (0.0010) [2023-12-26 16:19:29,552][105692] Updated weights for policy 0, policy_version 131577 (0.0008) [2023-12-26 16:19:29,614][105692] Updated weights for policy 0, policy_version 131587 (0.0010) [2023-12-26 16:19:30,065][105620] Updated weights for policy 1, policy_version 132176 (0.0008) [2023-12-26 16:19:30,129][105620] Updated weights for policy 1, policy_version 132186 (0.0009) [2023-12-26 16:19:30,192][105620] Updated weights for policy 1, policy_version 132196 (0.0008) [2023-12-26 16:19:30,316][105692] Updated weights for policy 0, policy_version 131597 (0.0009) [2023-12-26 16:19:30,377][105692] Updated weights for policy 0, policy_version 131607 (0.0010) [2023-12-26 16:19:30,431][105692] Updated weights for policy 0, policy_version 131617 (0.0010) [2023-12-26 16:19:30,994][105620] Updated weights for policy 1, policy_version 132206 (0.0008) [2023-12-26 16:19:31,054][105692] Updated weights for policy 0, policy_version 131627 (0.0010) [2023-12-26 16:19:31,058][105620] Updated weights for policy 1, policy_version 132216 (0.0009) [2023-12-26 16:19:31,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 67551232. Throughput: 0: 10008.1, 1: 9288.0. Samples: 67528412. Policy #0 lag: (min: 1.0, avg: 24.4, max: 33.0) [2023-12-26 16:19:31,062][104569] Avg episode reward: [(0, '8995.928'), (1, '8782.263')] [2023-12-26 16:19:31,106][105692] Updated weights for policy 0, policy_version 131637 (0.0009) [2023-12-26 16:19:31,115][105620] Updated weights for policy 1, policy_version 132226 (0.0009) [2023-12-26 16:19:31,147][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000132232_33857536.pth... [2023-12-26 16:19:31,154][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000131112_33570816.pth [2023-12-26 16:19:31,163][105692] Updated weights for policy 0, policy_version 131647 (0.0010) [2023-12-26 16:19:31,215][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000131656_33710080.pth... [2023-12-26 16:19:31,218][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000130472_33406976.pth [2023-12-26 16:19:31,843][105692] Updated weights for policy 0, policy_version 131657 (0.0006) [2023-12-26 16:19:31,905][105692] Updated weights for policy 0, policy_version 131667 (0.0006) [2023-12-26 16:19:31,941][105620] Updated weights for policy 1, policy_version 132236 (0.0009) [2023-12-26 16:19:31,964][105692] Updated weights for policy 0, policy_version 131677 (0.0008) [2023-12-26 16:19:32,000][105620] Updated weights for policy 1, policy_version 132246 (0.0010) [2023-12-26 16:19:32,031][105692] Updated weights for policy 0, policy_version 131687 (0.0008) [2023-12-26 16:19:32,064][105620] Updated weights for policy 1, policy_version 132256 (0.0007) [2023-12-26 16:19:32,676][105620] Updated weights for policy 1, policy_version 132266 (0.0008) [2023-12-26 16:19:32,734][105620] Updated weights for policy 1, policy_version 132276 (0.0008) [2023-12-26 16:19:32,782][105620] Updated weights for policy 1, policy_version 132286 (0.0011) [2023-12-26 16:19:32,821][105692] Updated weights for policy 0, policy_version 131697 (0.0006) [2023-12-26 16:19:32,834][105620] Updated weights for policy 1, policy_version 132296 (0.0010) [2023-12-26 16:19:32,868][105585] KL-divergence is very high: 164.5803 [2023-12-26 16:19:32,879][105692] Updated weights for policy 0, policy_version 131707 (0.0008) [2023-12-26 16:19:32,915][105585] KL-divergence is very high: 175.1405 [2023-12-26 16:19:32,937][105692] Updated weights for policy 0, policy_version 131717 (0.0008) [2023-12-26 16:19:33,568][105620] Updated weights for policy 1, policy_version 132306 (0.0010) [2023-12-26 16:19:33,620][105620] Updated weights for policy 1, policy_version 132316 (0.0010) [2023-12-26 16:19:33,678][105620] Updated weights for policy 1, policy_version 132326 (0.0010) [2023-12-26 16:19:33,691][105692] Updated weights for policy 0, policy_version 131727 (0.0009) [2023-12-26 16:19:33,752][105692] Updated weights for policy 0, policy_version 131737 (0.0007) [2023-12-26 16:19:33,807][105692] Updated weights for policy 0, policy_version 131747 (0.0009) [2023-12-26 16:19:34,256][105620] Updated weights for policy 1, policy_version 132336 (0.0010) [2023-12-26 16:19:34,319][105620] Updated weights for policy 1, policy_version 132346 (0.0008) [2023-12-26 16:19:34,379][105620] Updated weights for policy 1, policy_version 132356 (0.0012) [2023-12-26 16:19:34,567][105692] Updated weights for policy 0, policy_version 131758 (0.0009) [2023-12-26 16:19:34,635][105692] Updated weights for policy 0, policy_version 131768 (0.0008) [2023-12-26 16:19:34,703][105692] Updated weights for policy 0, policy_version 131778 (0.0008) [2023-12-26 16:19:35,115][105620] Updated weights for policy 1, policy_version 132366 (0.0010) [2023-12-26 16:19:35,177][105620] Updated weights for policy 1, policy_version 132376 (0.0010) [2023-12-26 16:19:35,232][105620] Updated weights for policy 1, policy_version 132386 (0.0010) [2023-12-26 16:19:35,448][105692] Updated weights for policy 0, policy_version 131788 (0.0008) [2023-12-26 16:19:35,506][105692] Updated weights for policy 0, policy_version 131798 (0.0008) [2023-12-26 16:19:35,561][105692] Updated weights for policy 0, policy_version 131808 (0.0008) [2023-12-26 16:19:35,977][105620] Updated weights for policy 1, policy_version 132396 (0.0010) [2023-12-26 16:19:36,040][105620] Updated weights for policy 1, policy_version 132406 (0.0009) [2023-12-26 16:19:36,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 67649536. Throughput: 0: 10030.3, 1: 9325.3. Samples: 67643532. Policy #0 lag: (min: 1.0, avg: 24.4, max: 33.0) [2023-12-26 16:19:36,062][104569] Avg episode reward: [(0, '9180.425'), (1, '7985.667')] [2023-12-26 16:19:36,087][105620] Updated weights for policy 1, policy_version 132416 (0.0008) [2023-12-26 16:19:36,308][105692] Updated weights for policy 0, policy_version 131818 (0.0008) [2023-12-26 16:19:36,358][105692] Updated weights for policy 0, policy_version 131828 (0.0009) [2023-12-26 16:19:36,403][105692] Updated weights for policy 0, policy_version 131838 (0.0007) [2023-12-26 16:19:36,467][105692] Updated weights for policy 0, policy_version 131848 (0.0005) [2023-12-26 16:19:36,892][105620] Updated weights for policy 1, policy_version 132426 (0.0008) [2023-12-26 16:19:36,958][105620] Updated weights for policy 1, policy_version 132436 (0.0009) [2023-12-26 16:19:37,021][105620] Updated weights for policy 1, policy_version 132446 (0.0009) [2023-12-26 16:19:37,082][105620] Updated weights for policy 1, policy_version 132456 (0.0006) [2023-12-26 16:19:37,186][105692] Updated weights for policy 0, policy_version 131858 (0.0009) [2023-12-26 16:19:37,248][105692] Updated weights for policy 0, policy_version 131868 (0.0010) [2023-12-26 16:19:37,311][105692] Updated weights for policy 0, policy_version 131878 (0.0009) [2023-12-26 16:19:37,648][105620] Updated weights for policy 1, policy_version 132466 (0.0005) [2023-12-26 16:19:37,710][105620] Updated weights for policy 1, policy_version 132476 (0.0009) [2023-12-26 16:19:37,774][105620] Updated weights for policy 1, policy_version 132486 (0.0009) [2023-12-26 16:19:38,089][105692] Updated weights for policy 0, policy_version 131888 (0.0010) [2023-12-26 16:19:38,156][105692] Updated weights for policy 0, policy_version 131898 (0.0007) [2023-12-26 16:19:38,214][105692] Updated weights for policy 0, policy_version 131908 (0.0005) [2023-12-26 16:19:38,433][105620] Updated weights for policy 1, policy_version 132496 (0.0008) [2023-12-26 16:19:38,497][105620] Updated weights for policy 1, policy_version 132506 (0.0008) [2023-12-26 16:19:38,562][105620] Updated weights for policy 1, policy_version 132516 (0.0009) [2023-12-26 16:19:38,946][105692] Updated weights for policy 0, policy_version 131918 (0.0009) [2023-12-26 16:19:39,002][105692] Updated weights for policy 0, policy_version 131928 (0.0008) [2023-12-26 16:19:39,069][105692] Updated weights for policy 0, policy_version 131938 (0.0009) [2023-12-26 16:19:39,312][105620] Updated weights for policy 1, policy_version 132526 (0.0008) [2023-12-26 16:19:39,380][105620] Updated weights for policy 1, policy_version 132536 (0.0009) [2023-12-26 16:19:39,440][105620] Updated weights for policy 1, policy_version 132546 (0.0007) [2023-12-26 16:19:39,925][105692] Updated weights for policy 0, policy_version 131948 (0.0007) [2023-12-26 16:19:39,988][105692] Updated weights for policy 0, policy_version 131958 (0.0009) [2023-12-26 16:19:40,057][105692] Updated weights for policy 0, policy_version 131968 (0.0009) [2023-12-26 16:19:40,114][105620] Updated weights for policy 1, policy_version 132556 (0.0008) [2023-12-26 16:19:40,175][105620] Updated weights for policy 1, policy_version 132566 (0.0010) [2023-12-26 16:19:40,239][105620] Updated weights for policy 1, policy_version 132576 (0.0011) [2023-12-26 16:19:40,836][105620] Updated weights for policy 1, policy_version 132586 (0.0010) [2023-12-26 16:19:40,838][105692] Updated weights for policy 0, policy_version 131978 (0.0007) [2023-12-26 16:19:40,886][105620] Updated weights for policy 1, policy_version 132596 (0.0007) [2023-12-26 16:19:40,901][105692] Updated weights for policy 0, policy_version 131988 (0.0008) [2023-12-26 16:19:40,943][105620] Updated weights for policy 1, policy_version 132606 (0.0007) [2023-12-26 16:19:40,967][105692] Updated weights for policy 0, policy_version 131998 (0.0006) [2023-12-26 16:19:41,002][105620] Updated weights for policy 1, policy_version 132616 (0.0008) [2023-12-26 16:19:41,027][105692] Updated weights for policy 0, policy_version 132008 (0.0007) [2023-12-26 16:19:41,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19251.2, 300 sec: 19438.7). Total num frames: 67756032. Throughput: 0: 9934.2, 1: 9481.5. Samples: 67758312. Policy #0 lag: (min: 1.0, avg: 24.4, max: 33.0) [2023-12-26 16:19:41,063][104569] Avg episode reward: [(0, '7486.304'), (1, '1496.553')] [2023-12-26 16:19:41,797][105692] Updated weights for policy 0, policy_version 132018 (0.0009) [2023-12-26 16:19:41,854][105692] Updated weights for policy 0, policy_version 132028 (0.0006) [2023-12-26 16:19:41,856][105620] Updated weights for policy 1, policy_version 132626 (0.0008) [2023-12-26 16:19:41,911][105620] Updated weights for policy 1, policy_version 132636 (0.0005) [2023-12-26 16:19:41,917][105692] Updated weights for policy 0, policy_version 132038 (0.0008) [2023-12-26 16:19:41,973][105620] Updated weights for policy 1, policy_version 132646 (0.0005) [2023-12-26 16:19:42,566][105620] Updated weights for policy 1, policy_version 132656 (0.0005) [2023-12-26 16:19:42,638][105620] Updated weights for policy 1, policy_version 132666 (0.0005) [2023-12-26 16:19:42,690][105692] Updated weights for policy 0, policy_version 132048 (0.0007) [2023-12-26 16:19:42,697][105620] Updated weights for policy 1, policy_version 132676 (0.0008) [2023-12-26 16:19:42,747][105692] Updated weights for policy 0, policy_version 132058 (0.0007) [2023-12-26 16:19:42,806][105692] Updated weights for policy 0, policy_version 132068 (0.0006) [2023-12-26 16:19:43,355][105692] Updated weights for policy 0, policy_version 132078 (0.0005) [2023-12-26 16:19:43,409][105692] Updated weights for policy 0, policy_version 132088 (0.0005) [2023-12-26 16:19:43,472][105692] Updated weights for policy 0, policy_version 132098 (0.0005) [2023-12-26 16:19:43,478][105620] Updated weights for policy 1, policy_version 132686 (0.0008) [2023-12-26 16:19:43,538][105620] Updated weights for policy 1, policy_version 132696 (0.0009) [2023-12-26 16:19:43,593][105620] Updated weights for policy 1, policy_version 132706 (0.0010) [2023-12-26 16:19:44,178][105692] Updated weights for policy 0, policy_version 132108 (0.0007) [2023-12-26 16:19:44,181][105620] Updated weights for policy 1, policy_version 132716 (0.0007) [2023-12-26 16:19:44,238][105692] Updated weights for policy 0, policy_version 132118 (0.0009) [2023-12-26 16:19:44,240][105620] Updated weights for policy 1, policy_version 132726 (0.0010) [2023-12-26 16:19:44,292][105692] Updated weights for policy 0, policy_version 132128 (0.0007) [2023-12-26 16:19:44,297][105620] Updated weights for policy 1, policy_version 132736 (0.0010) [2023-12-26 16:19:44,985][105620] Updated weights for policy 1, policy_version 132746 (0.0010) [2023-12-26 16:19:45,041][105620] Updated weights for policy 1, policy_version 132756 (0.0010) [2023-12-26 16:19:45,071][105692] Updated weights for policy 0, policy_version 132138 (0.0009) [2023-12-26 16:19:45,094][105620] Updated weights for policy 1, policy_version 132766 (0.0008) [2023-12-26 16:19:45,132][105692] Updated weights for policy 0, policy_version 132148 (0.0007) [2023-12-26 16:19:45,158][105620] Updated weights for policy 1, policy_version 132776 (0.0007) [2023-12-26 16:19:45,195][105692] Updated weights for policy 0, policy_version 132158 (0.0007) [2023-12-26 16:19:45,256][105692] Updated weights for policy 0, policy_version 132168 (0.0009) [2023-12-26 16:19:45,864][105620] Updated weights for policy 1, policy_version 132786 (0.0010) [2023-12-26 16:19:45,912][105620] Updated weights for policy 1, policy_version 132796 (0.0010) [2023-12-26 16:19:45,968][105620] Updated weights for policy 1, policy_version 132806 (0.0010) [2023-12-26 16:19:46,031][105692] Updated weights for policy 0, policy_version 132178 (0.0009) [2023-12-26 16:19:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.3, 300 sec: 19383.1). Total num frames: 67846144. Throughput: 0: 9900.3, 1: 9518.1. Samples: 67817488. Policy #0 lag: (min: 1.0, avg: 24.4, max: 33.0) [2023-12-26 16:19:46,063][104569] Avg episode reward: [(0, '7403.924'), (1, '3523.521')] [2023-12-26 16:19:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000132808_34004992.pth... [2023-12-26 16:19:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000131688_33718272.pth [2023-12-26 16:19:46,087][105692] Updated weights for policy 0, policy_version 132188 (0.0008) [2023-12-26 16:19:46,143][105692] Updated weights for policy 0, policy_version 132198 (0.0008) [2023-12-26 16:19:46,152][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000132200_33849344.pth... [2023-12-26 16:19:46,156][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000131048_33554432.pth [2023-12-26 16:19:46,740][105620] Updated weights for policy 1, policy_version 132816 (0.0010) [2023-12-26 16:19:46,788][105620] Updated weights for policy 1, policy_version 132826 (0.0010) [2023-12-26 16:19:46,846][105620] Updated weights for policy 1, policy_version 132836 (0.0010) [2023-12-26 16:19:46,925][105692] Updated weights for policy 0, policy_version 132208 (0.0008) [2023-12-26 16:19:46,981][105692] Updated weights for policy 0, policy_version 132218 (0.0008) [2023-12-26 16:19:47,039][105692] Updated weights for policy 0, policy_version 132228 (0.0008) [2023-12-26 16:19:47,514][105620] Updated weights for policy 1, policy_version 132846 (0.0010) [2023-12-26 16:19:47,572][105620] Updated weights for policy 1, policy_version 132856 (0.0010) [2023-12-26 16:19:47,629][105620] Updated weights for policy 1, policy_version 132866 (0.0007) [2023-12-26 16:19:47,863][105692] Updated weights for policy 0, policy_version 132238 (0.0009) [2023-12-26 16:19:47,915][105692] Updated weights for policy 0, policy_version 132248 (0.0008) [2023-12-26 16:19:47,980][105692] Updated weights for policy 0, policy_version 132258 (0.0008) [2023-12-26 16:19:48,299][105620] Updated weights for policy 1, policy_version 132876 (0.0007) [2023-12-26 16:19:48,360][105620] Updated weights for policy 1, policy_version 132886 (0.0010) [2023-12-26 16:19:48,423][105620] Updated weights for policy 1, policy_version 132896 (0.0010) [2023-12-26 16:19:48,738][105692] Updated weights for policy 0, policy_version 132268 (0.0007) [2023-12-26 16:19:48,808][105692] Updated weights for policy 0, policy_version 132278 (0.0008) [2023-12-26 16:19:48,867][105692] Updated weights for policy 0, policy_version 132288 (0.0010) [2023-12-26 16:19:49,152][105620] Updated weights for policy 1, policy_version 132906 (0.0011) [2023-12-26 16:19:49,215][105620] Updated weights for policy 1, policy_version 132916 (0.0009) [2023-12-26 16:19:49,285][105620] Updated weights for policy 1, policy_version 132926 (0.0007) [2023-12-26 16:19:49,356][105620] Updated weights for policy 1, policy_version 132936 (0.0010) [2023-12-26 16:19:49,565][105692] Updated weights for policy 0, policy_version 132298 (0.0009) [2023-12-26 16:19:49,617][105692] Updated weights for policy 0, policy_version 132308 (0.0007) [2023-12-26 16:19:49,679][105692] Updated weights for policy 0, policy_version 132318 (0.0009) [2023-12-26 16:19:49,732][105692] Updated weights for policy 0, policy_version 132328 (0.0009) [2023-12-26 16:19:50,040][105620] Updated weights for policy 1, policy_version 132946 (0.0010) [2023-12-26 16:19:50,101][105620] Updated weights for policy 1, policy_version 132956 (0.0010) [2023-12-26 16:19:50,156][105620] Updated weights for policy 1, policy_version 132966 (0.0010) [2023-12-26 16:19:50,497][105692] Updated weights for policy 0, policy_version 132338 (0.0009) [2023-12-26 16:19:50,554][105692] Updated weights for policy 0, policy_version 132348 (0.0009) [2023-12-26 16:19:50,626][105692] Updated weights for policy 0, policy_version 132358 (0.0008) [2023-12-26 16:19:50,820][105620] Updated weights for policy 1, policy_version 132976 (0.0009) [2023-12-26 16:19:50,877][105620] Updated weights for policy 1, policy_version 132986 (0.0009) [2023-12-26 16:19:50,931][105620] Updated weights for policy 1, policy_version 132996 (0.0009) [2023-12-26 16:19:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 67944448. Throughput: 0: 9752.7, 1: 9559.5. Samples: 67931608. Policy #0 lag: (min: 31.0, avg: 45.9, max: 63.0) [2023-12-26 16:19:51,062][104569] Avg episode reward: [(0, '8316.632'), (1, '6730.560')] [2023-12-26 16:19:51,335][105692] Updated weights for policy 0, policy_version 132368 (0.0007) [2023-12-26 16:19:51,407][105692] Updated weights for policy 0, policy_version 132378 (0.0008) [2023-12-26 16:19:51,470][105692] Updated weights for policy 0, policy_version 132388 (0.0009) [2023-12-26 16:19:51,732][105620] Updated weights for policy 1, policy_version 133006 (0.0009) [2023-12-26 16:19:51,791][105620] Updated weights for policy 1, policy_version 133016 (0.0010) [2023-12-26 16:19:51,857][105620] Updated weights for policy 1, policy_version 133026 (0.0011) [2023-12-26 16:19:52,178][105692] Updated weights for policy 0, policy_version 132398 (0.0010) [2023-12-26 16:19:52,241][105692] Updated weights for policy 0, policy_version 132408 (0.0011) [2023-12-26 16:19:52,308][105692] Updated weights for policy 0, policy_version 132418 (0.0011) [2023-12-26 16:19:52,656][105620] Updated weights for policy 1, policy_version 133036 (0.0008) [2023-12-26 16:19:52,725][105620] Updated weights for policy 1, policy_version 133046 (0.0008) [2023-12-26 16:19:52,783][105620] Updated weights for policy 1, policy_version 133056 (0.0007) [2023-12-26 16:19:52,965][105692] Updated weights for policy 0, policy_version 132428 (0.0009) [2023-12-26 16:19:53,023][105692] Updated weights for policy 0, policy_version 132438 (0.0006) [2023-12-26 16:19:53,077][105692] Updated weights for policy 0, policy_version 132448 (0.0005) [2023-12-26 16:19:53,428][105620] Updated weights for policy 1, policy_version 133066 (0.0007) [2023-12-26 16:19:53,490][105620] Updated weights for policy 1, policy_version 133076 (0.0005) [2023-12-26 16:19:53,562][105620] Updated weights for policy 1, policy_version 133086 (0.0005) [2023-12-26 16:19:53,626][105620] Updated weights for policy 1, policy_version 133096 (0.0005) [2023-12-26 16:19:53,643][105692] Updated weights for policy 0, policy_version 132458 (0.0006) [2023-12-26 16:19:53,691][105692] Updated weights for policy 0, policy_version 132468 (0.0010) [2023-12-26 16:19:53,749][105692] Updated weights for policy 0, policy_version 132478 (0.0010) [2023-12-26 16:19:53,804][105692] Updated weights for policy 0, policy_version 132488 (0.0010) [2023-12-26 16:19:54,261][105620] Updated weights for policy 1, policy_version 133106 (0.0010) [2023-12-26 16:19:54,322][105620] Updated weights for policy 1, policy_version 133116 (0.0010) [2023-12-26 16:19:54,377][105620] Updated weights for policy 1, policy_version 133126 (0.0010) [2023-12-26 16:19:54,484][105692] Updated weights for policy 0, policy_version 132498 (0.0011) [2023-12-26 16:19:54,549][105692] Updated weights for policy 0, policy_version 132508 (0.0009) [2023-12-26 16:19:54,617][105692] Updated weights for policy 0, policy_version 132518 (0.0008) [2023-12-26 16:19:55,018][105620] Updated weights for policy 1, policy_version 133136 (0.0011) [2023-12-26 16:19:55,083][105620] Updated weights for policy 1, policy_version 133146 (0.0010) [2023-12-26 16:19:55,149][105620] Updated weights for policy 1, policy_version 133156 (0.0011) [2023-12-26 16:19:55,276][105692] Updated weights for policy 0, policy_version 132528 (0.0011) [2023-12-26 16:19:55,328][105692] Updated weights for policy 0, policy_version 132538 (0.0010) [2023-12-26 16:19:55,390][105692] Updated weights for policy 0, policy_version 132548 (0.0011) [2023-12-26 16:19:55,810][105620] Updated weights for policy 1, policy_version 133166 (0.0010) [2023-12-26 16:19:55,858][105620] Updated weights for policy 1, policy_version 133176 (0.0010) [2023-12-26 16:19:55,901][105620] Updated weights for policy 1, policy_version 133186 (0.0010) [2023-12-26 16:19:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 68042752. Throughput: 0: 9712.3, 1: 9641.4. Samples: 68051168. Policy #0 lag: (min: 31.0, avg: 45.9, max: 63.0) [2023-12-26 16:19:56,062][104569] Avg episode reward: [(0, '9171.695'), (1, '7411.340')] [2023-12-26 16:19:56,082][105692] Updated weights for policy 0, policy_version 132558 (0.0009) [2023-12-26 16:19:56,142][105692] Updated weights for policy 0, policy_version 132568 (0.0008) [2023-12-26 16:19:56,197][105692] Updated weights for policy 0, policy_version 132578 (0.0008) [2023-12-26 16:19:56,641][105620] Updated weights for policy 1, policy_version 133196 (0.0007) [2023-12-26 16:19:56,705][105620] Updated weights for policy 1, policy_version 133206 (0.0005) [2023-12-26 16:19:56,772][105620] Updated weights for policy 1, policy_version 133216 (0.0005) [2023-12-26 16:19:56,897][105692] Updated weights for policy 0, policy_version 132588 (0.0008) [2023-12-26 16:19:56,942][105692] Updated weights for policy 0, policy_version 132598 (0.0010) [2023-12-26 16:19:56,989][105692] Updated weights for policy 0, policy_version 132608 (0.0010) [2023-12-26 16:19:57,301][105620] Updated weights for policy 1, policy_version 133226 (0.0005) [2023-12-26 16:19:57,356][105620] Updated weights for policy 1, policy_version 133236 (0.0006) [2023-12-26 16:19:57,409][105620] Updated weights for policy 1, policy_version 133246 (0.0005) [2023-12-26 16:19:57,459][105620] Updated weights for policy 1, policy_version 133256 (0.0005) [2023-12-26 16:19:57,715][105692] Updated weights for policy 0, policy_version 132618 (0.0011) [2023-12-26 16:19:57,777][105692] Updated weights for policy 0, policy_version 132628 (0.0009) [2023-12-26 16:19:57,837][105692] Updated weights for policy 0, policy_version 132638 (0.0010) [2023-12-26 16:19:57,894][105692] Updated weights for policy 0, policy_version 132648 (0.0010) [2023-12-26 16:19:58,003][105620] Updated weights for policy 1, policy_version 133266 (0.0010) [2023-12-26 16:19:58,058][105620] Updated weights for policy 1, policy_version 133276 (0.0007) [2023-12-26 16:19:58,126][105620] Updated weights for policy 1, policy_version 133286 (0.0008) [2023-12-26 16:19:58,705][105692] Updated weights for policy 0, policy_version 132658 (0.0008) [2023-12-26 16:19:58,775][105692] Updated weights for policy 0, policy_version 132668 (0.0010) [2023-12-26 16:19:58,837][105692] Updated weights for policy 0, policy_version 132678 (0.0009) [2023-12-26 16:19:58,968][105620] Updated weights for policy 1, policy_version 133296 (0.0009) [2023-12-26 16:19:59,034][105620] Updated weights for policy 1, policy_version 133306 (0.0009) [2023-12-26 16:19:59,099][105620] Updated weights for policy 1, policy_version 133316 (0.0009) [2023-12-26 16:19:59,575][105692] Updated weights for policy 0, policy_version 132688 (0.0010) [2023-12-26 16:19:59,630][105692] Updated weights for policy 0, policy_version 132698 (0.0010) [2023-12-26 16:19:59,689][105692] Updated weights for policy 0, policy_version 132708 (0.0010) [2023-12-26 16:19:59,847][105620] Updated weights for policy 1, policy_version 133326 (0.0008) [2023-12-26 16:19:59,903][105620] Updated weights for policy 1, policy_version 133336 (0.0009) [2023-12-26 16:19:59,966][105620] Updated weights for policy 1, policy_version 133346 (0.0008) [2023-12-26 16:20:00,412][105692] Updated weights for policy 0, policy_version 132718 (0.0010) [2023-12-26 16:20:00,456][105692] Updated weights for policy 0, policy_version 132728 (0.0010) [2023-12-26 16:20:00,503][105692] Updated weights for policy 0, policy_version 132738 (0.0010) [2023-12-26 16:20:00,729][105620] Updated weights for policy 1, policy_version 133356 (0.0009) [2023-12-26 16:20:00,797][105620] Updated weights for policy 1, policy_version 133366 (0.0009) [2023-12-26 16:20:00,860][105620] Updated weights for policy 1, policy_version 133376 (0.0010) [2023-12-26 16:20:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 68141056. Throughput: 0: 9694.2, 1: 9681.4. Samples: 68111764. Policy #0 lag: (min: 31.0, avg: 45.9, max: 63.0) [2023-12-26 16:20:01,063][104569] Avg episode reward: [(0, '8812.673'), (1, '8041.794')] [2023-12-26 16:20:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000132744_33988608.pth... [2023-12-26 16:20:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000133384_34152448.pth... [2023-12-26 16:20:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000131656_33710080.pth [2023-12-26 16:20:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000132232_33857536.pth [2023-12-26 16:20:01,221][105692] Updated weights for policy 0, policy_version 132748 (0.0010) [2023-12-26 16:20:01,284][105692] Updated weights for policy 0, policy_version 132758 (0.0010) [2023-12-26 16:20:01,335][105692] Updated weights for policy 0, policy_version 132768 (0.0009) [2023-12-26 16:20:01,553][105620] Updated weights for policy 1, policy_version 133386 (0.0010) [2023-12-26 16:20:01,615][105620] Updated weights for policy 1, policy_version 133396 (0.0009) [2023-12-26 16:20:01,678][105620] Updated weights for policy 1, policy_version 133406 (0.0009) [2023-12-26 16:20:01,742][105620] Updated weights for policy 1, policy_version 133416 (0.0009) [2023-12-26 16:20:02,102][105692] Updated weights for policy 0, policy_version 132778 (0.0009) [2023-12-26 16:20:02,156][105692] Updated weights for policy 0, policy_version 132788 (0.0009) [2023-12-26 16:20:02,214][105692] Updated weights for policy 0, policy_version 132798 (0.0009) [2023-12-26 16:20:02,272][105692] Updated weights for policy 0, policy_version 132808 (0.0008) [2023-12-26 16:20:02,487][105620] Updated weights for policy 1, policy_version 133426 (0.0010) [2023-12-26 16:20:02,545][105620] Updated weights for policy 1, policy_version 133436 (0.0009) [2023-12-26 16:20:02,596][105620] Updated weights for policy 1, policy_version 133446 (0.0010) [2023-12-26 16:20:02,974][105692] Updated weights for policy 0, policy_version 132818 (0.0008) [2023-12-26 16:20:03,021][105692] Updated weights for policy 0, policy_version 132828 (0.0008) [2023-12-26 16:20:03,072][105692] Updated weights for policy 0, policy_version 132838 (0.0007) [2023-12-26 16:20:03,347][105620] Updated weights for policy 1, policy_version 133456 (0.0006) [2023-12-26 16:20:03,415][105620] Updated weights for policy 1, policy_version 133466 (0.0005) [2023-12-26 16:20:03,479][105620] Updated weights for policy 1, policy_version 133476 (0.0005) [2023-12-26 16:20:03,828][105692] Updated weights for policy 0, policy_version 132848 (0.0009) [2023-12-26 16:20:03,897][105692] Updated weights for policy 0, policy_version 132858 (0.0006) [2023-12-26 16:20:03,956][105692] Updated weights for policy 0, policy_version 132868 (0.0010) [2023-12-26 16:20:04,029][105620] Updated weights for policy 1, policy_version 133486 (0.0008) [2023-12-26 16:20:04,081][105620] Updated weights for policy 1, policy_version 133496 (0.0010) [2023-12-26 16:20:04,137][105620] Updated weights for policy 1, policy_version 133506 (0.0011) [2023-12-26 16:20:04,611][105692] Updated weights for policy 0, policy_version 132878 (0.0009) [2023-12-26 16:20:04,671][105692] Updated weights for policy 0, policy_version 132888 (0.0011) [2023-12-26 16:20:04,743][105692] Updated weights for policy 0, policy_version 132898 (0.0009) [2023-12-26 16:20:04,819][105620] Updated weights for policy 1, policy_version 133516 (0.0010) [2023-12-26 16:20:04,887][105620] Updated weights for policy 1, policy_version 133526 (0.0008) [2023-12-26 16:20:04,954][105620] Updated weights for policy 1, policy_version 133536 (0.0008) [2023-12-26 16:20:05,460][105692] Updated weights for policy 0, policy_version 132908 (0.0010) [2023-12-26 16:20:05,508][105692] Updated weights for policy 0, policy_version 132918 (0.0010) [2023-12-26 16:20:05,556][105692] Updated weights for policy 0, policy_version 132928 (0.0010) [2023-12-26 16:20:05,598][105620] Updated weights for policy 1, policy_version 133546 (0.0008) [2023-12-26 16:20:05,652][105620] Updated weights for policy 1, policy_version 133556 (0.0005) [2023-12-26 16:20:05,705][105620] Updated weights for policy 1, policy_version 133566 (0.0005) [2023-12-26 16:20:05,758][105620] Updated weights for policy 1, policy_version 133576 (0.0005) [2023-12-26 16:20:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 68239360. Throughput: 0: 9686.2, 1: 9677.7. Samples: 68227668. Policy #0 lag: (min: 31.0, avg: 45.9, max: 63.0) [2023-12-26 16:20:06,062][104569] Avg episode reward: [(0, '8992.225'), (1, '8808.857')] [2023-12-26 16:20:06,329][105692] Updated weights for policy 0, policy_version 132938 (0.0010) [2023-12-26 16:20:06,339][105620] Updated weights for policy 1, policy_version 133586 (0.0010) [2023-12-26 16:20:06,381][105692] Updated weights for policy 0, policy_version 132948 (0.0006) [2023-12-26 16:20:06,405][105620] Updated weights for policy 1, policy_version 133596 (0.0010) [2023-12-26 16:20:06,444][105692] Updated weights for policy 0, policy_version 132958 (0.0008) [2023-12-26 16:20:06,464][105620] Updated weights for policy 1, policy_version 133606 (0.0010) [2023-12-26 16:20:06,504][105692] Updated weights for policy 0, policy_version 132968 (0.0010) [2023-12-26 16:20:07,121][105620] Updated weights for policy 1, policy_version 133616 (0.0010) [2023-12-26 16:20:07,181][105620] Updated weights for policy 1, policy_version 133626 (0.0008) [2023-12-26 16:20:07,250][105620] Updated weights for policy 1, policy_version 133636 (0.0005) [2023-12-26 16:20:07,327][105692] Updated weights for policy 0, policy_version 132978 (0.0010) [2023-12-26 16:20:07,384][105692] Updated weights for policy 0, policy_version 132988 (0.0008) [2023-12-26 16:20:07,442][105692] Updated weights for policy 0, policy_version 132998 (0.0009) [2023-12-26 16:20:07,925][105620] Updated weights for policy 1, policy_version 133646 (0.0008) [2023-12-26 16:20:07,989][105620] Updated weights for policy 1, policy_version 133656 (0.0008) [2023-12-26 16:20:08,053][105620] Updated weights for policy 1, policy_version 133666 (0.0009) [2023-12-26 16:20:08,196][105692] Updated weights for policy 0, policy_version 133008 (0.0006) [2023-12-26 16:20:08,256][105692] Updated weights for policy 0, policy_version 133018 (0.0005) [2023-12-26 16:20:08,311][105692] Updated weights for policy 0, policy_version 133028 (0.0005) [2023-12-26 16:20:08,805][105620] Updated weights for policy 1, policy_version 133676 (0.0010) [2023-12-26 16:20:08,862][105620] Updated weights for policy 1, policy_version 133686 (0.0009) [2023-12-26 16:20:08,926][105620] Updated weights for policy 1, policy_version 133696 (0.0007) [2023-12-26 16:20:08,993][105692] Updated weights for policy 0, policy_version 133038 (0.0008) [2023-12-26 16:20:09,054][105692] Updated weights for policy 0, policy_version 133048 (0.0010) [2023-12-26 16:20:09,112][105692] Updated weights for policy 0, policy_version 133058 (0.0010) [2023-12-26 16:20:09,636][105620] Updated weights for policy 1, policy_version 133706 (0.0007) [2023-12-26 16:20:09,692][105620] Updated weights for policy 1, policy_version 133716 (0.0005) [2023-12-26 16:20:09,750][105620] Updated weights for policy 1, policy_version 133726 (0.0007) [2023-12-26 16:20:09,807][105620] Updated weights for policy 1, policy_version 133736 (0.0011) [2023-12-26 16:20:09,922][105692] Updated weights for policy 0, policy_version 133068 (0.0008) [2023-12-26 16:20:09,992][105692] Updated weights for policy 0, policy_version 133078 (0.0006) [2023-12-26 16:20:10,062][105692] Updated weights for policy 0, policy_version 133088 (0.0006) [2023-12-26 16:20:10,545][105620] Updated weights for policy 1, policy_version 133746 (0.0007) [2023-12-26 16:20:10,608][105620] Updated weights for policy 1, policy_version 133756 (0.0009) [2023-12-26 16:20:10,656][105620] Updated weights for policy 1, policy_version 133766 (0.0009) [2023-12-26 16:20:10,741][105692] Updated weights for policy 0, policy_version 133098 (0.0006) [2023-12-26 16:20:10,800][105692] Updated weights for policy 0, policy_version 133108 (0.0005) [2023-12-26 16:20:10,864][105692] Updated weights for policy 0, policy_version 133118 (0.0006) [2023-12-26 16:20:10,918][105692] Updated weights for policy 0, policy_version 133128 (0.0009) [2023-12-26 16:20:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 68337664. Throughput: 0: 9590.1, 1: 9808.4. Samples: 68342852. Policy #0 lag: (min: 31.0, avg: 45.9, max: 63.0) [2023-12-26 16:20:11,062][104569] Avg episode reward: [(0, '8992.034'), (1, '8759.157')] [2023-12-26 16:20:11,498][105620] Updated weights for policy 1, policy_version 133776 (0.0009) [2023-12-26 16:20:11,561][105620] Updated weights for policy 1, policy_version 133786 (0.0009) [2023-12-26 16:20:11,621][105620] Updated weights for policy 1, policy_version 133796 (0.0008) [2023-12-26 16:20:11,631][105692] Updated weights for policy 0, policy_version 133138 (0.0007) [2023-12-26 16:20:11,689][105692] Updated weights for policy 0, policy_version 133148 (0.0008) [2023-12-26 16:20:11,746][105692] Updated weights for policy 0, policy_version 133158 (0.0009) [2023-12-26 16:20:12,403][105620] Updated weights for policy 1, policy_version 133806 (0.0009) [2023-12-26 16:20:12,470][105620] Updated weights for policy 1, policy_version 133816 (0.0008) [2023-12-26 16:20:12,518][105692] Updated weights for policy 0, policy_version 133168 (0.0008) [2023-12-26 16:20:12,533][105620] Updated weights for policy 1, policy_version 133826 (0.0007) [2023-12-26 16:20:12,572][105692] Updated weights for policy 0, policy_version 133178 (0.0009) [2023-12-26 16:20:12,620][105692] Updated weights for policy 0, policy_version 133188 (0.0008) [2023-12-26 16:20:13,247][105620] Updated weights for policy 1, policy_version 133836 (0.0006) [2023-12-26 16:20:13,301][105620] Updated weights for policy 1, policy_version 133847 (0.0010) [2023-12-26 16:20:13,330][105692] Updated weights for policy 0, policy_version 133198 (0.0005) [2023-12-26 16:20:13,349][105620] Updated weights for policy 1, policy_version 133857 (0.0008) [2023-12-26 16:20:13,386][105692] Updated weights for policy 0, policy_version 133208 (0.0005) [2023-12-26 16:20:13,452][105692] Updated weights for policy 0, policy_version 133218 (0.0009) [2023-12-26 16:20:14,123][105620] Updated weights for policy 1, policy_version 133867 (0.0009) [2023-12-26 16:20:14,159][105692] Updated weights for policy 0, policy_version 133228 (0.0009) [2023-12-26 16:20:14,177][105620] Updated weights for policy 1, policy_version 133877 (0.0007) [2023-12-26 16:20:14,210][105692] Updated weights for policy 0, policy_version 133238 (0.0006) [2023-12-26 16:20:14,226][105620] Updated weights for policy 1, policy_version 133887 (0.0006) [2023-12-26 16:20:14,268][105692] Updated weights for policy 0, policy_version 133248 (0.0008) [2023-12-26 16:20:14,980][105620] Updated weights for policy 1, policy_version 133897 (0.0006) [2023-12-26 16:20:15,040][105620] Updated weights for policy 1, policy_version 133907 (0.0009) [2023-12-26 16:20:15,060][105692] Updated weights for policy 0, policy_version 133258 (0.0007) [2023-12-26 16:20:15,101][105620] Updated weights for policy 1, policy_version 133917 (0.0010) [2023-12-26 16:20:15,119][105692] Updated weights for policy 0, policy_version 133268 (0.0008) [2023-12-26 16:20:15,164][105620] Updated weights for policy 1, policy_version 133927 (0.0009) [2023-12-26 16:20:15,182][105692] Updated weights for policy 0, policy_version 133278 (0.0005) [2023-12-26 16:20:15,244][105692] Updated weights for policy 0, policy_version 133288 (0.0008) [2023-12-26 16:20:15,823][105620] Updated weights for policy 1, policy_version 133937 (0.0006) [2023-12-26 16:20:15,883][105620] Updated weights for policy 1, policy_version 133947 (0.0005) [2023-12-26 16:20:15,942][105620] Updated weights for policy 1, policy_version 133957 (0.0005) [2023-12-26 16:20:16,025][105692] Updated weights for policy 0, policy_version 133298 (0.0010) [2023-12-26 16:20:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.3, 300 sec: 19383.1). Total num frames: 68427776. Throughput: 0: 9519.1, 1: 9829.7. Samples: 68399108. Policy #0 lag: (min: 31.0, avg: 45.9, max: 63.0) [2023-12-26 16:20:16,062][104569] Avg episode reward: [(0, '9259.025'), (1, '8603.937')] [2023-12-26 16:20:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000133960_34299904.pth... [2023-12-26 16:20:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000132808_34004992.pth [2023-12-26 16:20:16,086][105692] Updated weights for policy 0, policy_version 133308 (0.0009) [2023-12-26 16:20:16,144][105692] Updated weights for policy 0, policy_version 133319 (0.0010) [2023-12-26 16:20:16,146][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000133320_34136064.pth... [2023-12-26 16:20:16,149][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000132200_33849344.pth [2023-12-26 16:20:16,473][105620] Updated weights for policy 1, policy_version 133967 (0.0005) [2023-12-26 16:20:16,536][105620] Updated weights for policy 1, policy_version 133977 (0.0005) [2023-12-26 16:20:16,591][105620] Updated weights for policy 1, policy_version 133987 (0.0005) [2023-12-26 16:20:17,023][105692] Updated weights for policy 0, policy_version 133329 (0.0009) [2023-12-26 16:20:17,077][105692] Updated weights for policy 0, policy_version 133339 (0.0009) [2023-12-26 16:20:17,127][105692] Updated weights for policy 0, policy_version 133349 (0.0008) [2023-12-26 16:20:17,183][105620] Updated weights for policy 1, policy_version 133997 (0.0005) [2023-12-26 16:20:17,236][105620] Updated weights for policy 1, policy_version 134007 (0.0005) [2023-12-26 16:20:17,297][105620] Updated weights for policy 1, policy_version 134017 (0.0005) [2023-12-26 16:20:17,878][105620] Updated weights for policy 1, policy_version 134027 (0.0007) [2023-12-26 16:20:17,938][105620] Updated weights for policy 1, policy_version 134037 (0.0009) [2023-12-26 16:20:17,977][105692] Updated weights for policy 0, policy_version 133359 (0.0007) [2023-12-26 16:20:17,995][105620] Updated weights for policy 1, policy_version 134047 (0.0008) [2023-12-26 16:20:18,036][105692] Updated weights for policy 0, policy_version 133369 (0.0007) [2023-12-26 16:20:18,098][105692] Updated weights for policy 0, policy_version 133379 (0.0009) [2023-12-26 16:20:18,662][105620] Updated weights for policy 1, policy_version 134057 (0.0008) [2023-12-26 16:20:18,710][105620] Updated weights for policy 1, policy_version 134067 (0.0011) [2023-12-26 16:20:18,772][105620] Updated weights for policy 1, policy_version 134077 (0.0010) [2023-12-26 16:20:18,838][105620] Updated weights for policy 1, policy_version 134087 (0.0010) [2023-12-26 16:20:18,902][105692] Updated weights for policy 0, policy_version 133389 (0.0009) [2023-12-26 16:20:18,963][105692] Updated weights for policy 0, policy_version 133399 (0.0008) [2023-12-26 16:20:19,020][105692] Updated weights for policy 0, policy_version 133409 (0.0008) [2023-12-26 16:20:19,580][105620] Updated weights for policy 1, policy_version 134097 (0.0010) [2023-12-26 16:20:19,649][105620] Updated weights for policy 1, policy_version 134107 (0.0010) [2023-12-26 16:20:19,718][105620] Updated weights for policy 1, policy_version 134117 (0.0010) [2023-12-26 16:20:19,791][105692] Updated weights for policy 0, policy_version 133419 (0.0009) [2023-12-26 16:20:19,856][105692] Updated weights for policy 0, policy_version 133429 (0.0009) [2023-12-26 16:20:19,922][105692] Updated weights for policy 0, policy_version 133439 (0.0009) [2023-12-26 16:20:20,408][105620] Updated weights for policy 1, policy_version 134127 (0.0011) [2023-12-26 16:20:20,469][105620] Updated weights for policy 1, policy_version 134137 (0.0007) [2023-12-26 16:20:20,534][105620] Updated weights for policy 1, policy_version 134147 (0.0006) [2023-12-26 16:20:20,799][105692] Updated weights for policy 0, policy_version 133449 (0.0008) [2023-12-26 16:20:20,866][105692] Updated weights for policy 0, policy_version 133459 (0.0007) [2023-12-26 16:20:20,929][105692] Updated weights for policy 0, policy_version 133469 (0.0009) [2023-12-26 16:20:21,001][105692] Updated weights for policy 0, policy_version 133479 (0.0008) [2023-12-26 16:20:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 68526080. Throughput: 0: 9394.6, 1: 9960.6. Samples: 68514520. Policy #0 lag: (min: 31.0, avg: 45.9, max: 63.0) [2023-12-26 16:20:21,063][104569] Avg episode reward: [(0, '9166.645'), (1, '8377.576')] [2023-12-26 16:20:21,200][105620] Updated weights for policy 1, policy_version 134157 (0.0009) [2023-12-26 16:20:21,270][105620] Updated weights for policy 1, policy_version 134167 (0.0011) [2023-12-26 16:20:21,338][105620] Updated weights for policy 1, policy_version 134177 (0.0011) [2023-12-26 16:20:21,774][105692] Updated weights for policy 0, policy_version 133489 (0.0008) [2023-12-26 16:20:21,838][105692] Updated weights for policy 0, policy_version 133499 (0.0009) [2023-12-26 16:20:21,901][105692] Updated weights for policy 0, policy_version 133509 (0.0009) [2023-12-26 16:20:22,129][105620] Updated weights for policy 1, policy_version 134187 (0.0009) [2023-12-26 16:20:22,191][105620] Updated weights for policy 1, policy_version 134197 (0.0009) [2023-12-26 16:20:22,249][105620] Updated weights for policy 1, policy_version 134207 (0.0009) [2023-12-26 16:20:22,688][105692] Updated weights for policy 0, policy_version 133519 (0.0009) [2023-12-26 16:20:22,747][105692] Updated weights for policy 0, policy_version 133529 (0.0009) [2023-12-26 16:20:22,806][105692] Updated weights for policy 0, policy_version 133539 (0.0009) [2023-12-26 16:20:23,017][105620] Updated weights for policy 1, policy_version 134217 (0.0008) [2023-12-26 16:20:23,070][105620] Updated weights for policy 1, policy_version 134227 (0.0008) [2023-12-26 16:20:23,119][105620] Updated weights for policy 1, policy_version 134237 (0.0006) [2023-12-26 16:20:23,173][105620] Updated weights for policy 1, policy_version 134247 (0.0005) [2023-12-26 16:20:23,635][105692] Updated weights for policy 0, policy_version 133549 (0.0008) [2023-12-26 16:20:23,701][105692] Updated weights for policy 0, policy_version 133559 (0.0005) [2023-12-26 16:20:23,756][105692] Updated weights for policy 0, policy_version 133569 (0.0005) [2023-12-26 16:20:23,769][105620] Updated weights for policy 1, policy_version 134257 (0.0010) [2023-12-26 16:20:23,828][105620] Updated weights for policy 1, policy_version 134267 (0.0010) [2023-12-26 16:20:23,877][105620] Updated weights for policy 1, policy_version 134277 (0.0007) [2023-12-26 16:20:24,306][105692] Updated weights for policy 0, policy_version 133579 (0.0005) [2023-12-26 16:20:24,362][105692] Updated weights for policy 0, policy_version 133589 (0.0005) [2023-12-26 16:20:24,425][105692] Updated weights for policy 0, policy_version 133599 (0.0005) [2023-12-26 16:20:24,441][105620] Updated weights for policy 1, policy_version 134287 (0.0009) [2023-12-26 16:20:24,507][105620] Updated weights for policy 1, policy_version 134297 (0.0010) [2023-12-26 16:20:24,571][105620] Updated weights for policy 1, policy_version 134307 (0.0010) [2023-12-26 16:20:25,074][105692] Updated weights for policy 0, policy_version 133609 (0.0006) [2023-12-26 16:20:25,133][105692] Updated weights for policy 0, policy_version 133619 (0.0008) [2023-12-26 16:20:25,193][105692] Updated weights for policy 0, policy_version 133629 (0.0008) [2023-12-26 16:20:25,256][105692] Updated weights for policy 0, policy_version 133639 (0.0008) [2023-12-26 16:20:25,304][105620] Updated weights for policy 1, policy_version 134317 (0.0008) [2023-12-26 16:20:25,369][105620] Updated weights for policy 1, policy_version 134327 (0.0006) [2023-12-26 16:20:25,434][105620] Updated weights for policy 1, policy_version 134337 (0.0009) [2023-12-26 16:20:25,884][105692] Updated weights for policy 0, policy_version 133649 (0.0005) [2023-12-26 16:20:25,942][105692] Updated weights for policy 0, policy_version 133659 (0.0006) [2023-12-26 16:20:25,994][105692] Updated weights for policy 0, policy_version 133669 (0.0005) [2023-12-26 16:20:26,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19387.7, 300 sec: 19327.5). Total num frames: 68624384. Throughput: 0: 9430.1, 1: 9967.2. Samples: 68631196. Policy #0 lag: (min: 31.0, avg: 45.9, max: 63.0) [2023-12-26 16:20:26,064][104569] Avg episode reward: [(0, '9165.985'), (1, '8712.512')] [2023-12-26 16:20:26,090][105620] Updated weights for policy 1, policy_version 134347 (0.0008) [2023-12-26 16:20:26,144][105620] Updated weights for policy 1, policy_version 134357 (0.0010) [2023-12-26 16:20:26,208][105620] Updated weights for policy 1, policy_version 134367 (0.0011) [2023-12-26 16:20:26,517][105692] Updated weights for policy 0, policy_version 133679 (0.0007) [2023-12-26 16:20:26,577][105692] Updated weights for policy 0, policy_version 133689 (0.0009) [2023-12-26 16:20:26,625][105692] Updated weights for policy 0, policy_version 133699 (0.0008) [2023-12-26 16:20:26,877][105620] Updated weights for policy 1, policy_version 134377 (0.0010) [2023-12-26 16:20:26,922][105620] Updated weights for policy 1, policy_version 134387 (0.0010) [2023-12-26 16:20:26,976][105620] Updated weights for policy 1, policy_version 134397 (0.0010) [2023-12-26 16:20:27,026][105620] Updated weights for policy 1, policy_version 134407 (0.0010) [2023-12-26 16:20:27,267][105692] Updated weights for policy 0, policy_version 133709 (0.0010) [2023-12-26 16:20:27,315][105692] Updated weights for policy 0, policy_version 133719 (0.0010) [2023-12-26 16:20:27,375][105692] Updated weights for policy 0, policy_version 133729 (0.0006) [2023-12-26 16:20:27,630][105620] Updated weights for policy 1, policy_version 134417 (0.0010) [2023-12-26 16:20:27,691][105620] Updated weights for policy 1, policy_version 134427 (0.0010) [2023-12-26 16:20:27,746][105620] Updated weights for policy 1, policy_version 134437 (0.0010) [2023-12-26 16:20:27,915][105692] Updated weights for policy 0, policy_version 133739 (0.0005) [2023-12-26 16:20:27,977][105692] Updated weights for policy 0, policy_version 133749 (0.0005) [2023-12-26 16:20:28,026][105692] Updated weights for policy 0, policy_version 133759 (0.0008) [2023-12-26 16:20:28,403][105620] Updated weights for policy 1, policy_version 134447 (0.0009) [2023-12-26 16:20:28,465][105620] Updated weights for policy 1, policy_version 134457 (0.0008) [2023-12-26 16:20:28,529][105620] Updated weights for policy 1, policy_version 134467 (0.0008) [2023-12-26 16:20:28,728][105692] Updated weights for policy 0, policy_version 133769 (0.0010) [2023-12-26 16:20:28,785][105692] Updated weights for policy 0, policy_version 133779 (0.0010) [2023-12-26 16:20:28,829][105692] Updated weights for policy 0, policy_version 133789 (0.0010) [2023-12-26 16:20:28,873][105692] Updated weights for policy 0, policy_version 133799 (0.0010) [2023-12-26 16:20:29,298][105620] Updated weights for policy 1, policy_version 134477 (0.0007) [2023-12-26 16:20:29,354][105620] Updated weights for policy 1, policy_version 134487 (0.0006) [2023-12-26 16:20:29,421][105620] Updated weights for policy 1, policy_version 134497 (0.0006) [2023-12-26 16:20:29,576][105692] Updated weights for policy 0, policy_version 133809 (0.0010) [2023-12-26 16:20:29,630][105692] Updated weights for policy 0, policy_version 133819 (0.0010) [2023-12-26 16:20:29,688][105692] Updated weights for policy 0, policy_version 133829 (0.0010) [2023-12-26 16:20:30,095][105620] Updated weights for policy 1, policy_version 134507 (0.0007) [2023-12-26 16:20:30,151][105620] Updated weights for policy 1, policy_version 134517 (0.0008) [2023-12-26 16:20:30,203][105620] Updated weights for policy 1, policy_version 134527 (0.0008) [2023-12-26 16:20:30,433][105692] Updated weights for policy 0, policy_version 133839 (0.0010) [2023-12-26 16:20:30,498][105692] Updated weights for policy 0, policy_version 133849 (0.0010) [2023-12-26 16:20:30,547][105692] Updated weights for policy 0, policy_version 133859 (0.0010) [2023-12-26 16:20:30,979][105620] Updated weights for policy 1, policy_version 134537 (0.0008) [2023-12-26 16:20:31,042][105620] Updated weights for policy 1, policy_version 134547 (0.0008) [2023-12-26 16:20:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19355.3). Total num frames: 68722688. Throughput: 0: 9537.6, 1: 10009.3. Samples: 68697100. Policy #0 lag: (min: 31.0, avg: 45.9, max: 63.0) [2023-12-26 16:20:31,063][104569] Avg episode reward: [(0, '9087.985'), (1, '9167.079')] [2023-12-26 16:20:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000133864_34275328.pth... [2023-12-26 16:20:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000132744_33988608.pth [2023-12-26 16:20:31,108][105620] Updated weights for policy 1, policy_version 134557 (0.0007) [2023-12-26 16:20:31,173][105620] Updated weights for policy 1, policy_version 134567 (0.0008) [2023-12-26 16:20:31,178][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000134568_34455552.pth... [2023-12-26 16:20:31,183][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000133384_34152448.pth [2023-12-26 16:20:31,273][105692] Updated weights for policy 0, policy_version 133869 (0.0010) [2023-12-26 16:20:31,331][105692] Updated weights for policy 0, policy_version 133879 (0.0010) [2023-12-26 16:20:31,394][105692] Updated weights for policy 0, policy_version 133889 (0.0011) [2023-12-26 16:20:31,955][105620] Updated weights for policy 1, policy_version 134577 (0.0009) [2023-12-26 16:20:32,004][105620] Updated weights for policy 1, policy_version 134588 (0.0009) [2023-12-26 16:20:32,060][105620] Updated weights for policy 1, policy_version 134598 (0.0009) [2023-12-26 16:20:32,066][105692] Updated weights for policy 0, policy_version 133899 (0.0010) [2023-12-26 16:20:32,119][105692] Updated weights for policy 0, policy_version 133909 (0.0008) [2023-12-26 16:20:32,169][105692] Updated weights for policy 0, policy_version 133919 (0.0008) [2023-12-26 16:20:32,793][105620] Updated weights for policy 1, policy_version 134608 (0.0009) [2023-12-26 16:20:32,852][105620] Updated weights for policy 1, policy_version 134618 (0.0010) [2023-12-26 16:20:32,904][105620] Updated weights for policy 1, policy_version 134628 (0.0010) [2023-12-26 16:20:32,922][105692] Updated weights for policy 0, policy_version 133929 (0.0008) [2023-12-26 16:20:32,980][105692] Updated weights for policy 0, policy_version 133939 (0.0005) [2023-12-26 16:20:33,037][105692] Updated weights for policy 0, policy_version 133949 (0.0006) [2023-12-26 16:20:33,091][105692] Updated weights for policy 0, policy_version 133959 (0.0006) [2023-12-26 16:20:33,691][105620] Updated weights for policy 1, policy_version 134638 (0.0009) [2023-12-26 16:20:33,740][105620] Updated weights for policy 1, policy_version 134648 (0.0007) [2023-12-26 16:20:33,750][105692] Updated weights for policy 0, policy_version 133969 (0.0007) [2023-12-26 16:20:33,792][105620] Updated weights for policy 1, policy_version 134658 (0.0007) [2023-12-26 16:20:33,804][105692] Updated weights for policy 0, policy_version 133979 (0.0006) [2023-12-26 16:20:33,859][105692] Updated weights for policy 0, policy_version 133989 (0.0008) [2023-12-26 16:20:34,549][105620] Updated weights for policy 1, policy_version 134668 (0.0009) [2023-12-26 16:20:34,603][105620] Updated weights for policy 1, policy_version 134678 (0.0006) [2023-12-26 16:20:34,620][105692] Updated weights for policy 0, policy_version 133999 (0.0009) [2023-12-26 16:20:34,665][105620] Updated weights for policy 1, policy_version 134688 (0.0007) [2023-12-26 16:20:34,686][105692] Updated weights for policy 0, policy_version 134009 (0.0006) [2023-12-26 16:20:34,748][105692] Updated weights for policy 0, policy_version 134019 (0.0010) [2023-12-26 16:20:35,362][105692] Updated weights for policy 0, policy_version 134029 (0.0009) [2023-12-26 16:20:35,426][105692] Updated weights for policy 0, policy_version 134039 (0.0008) [2023-12-26 16:20:35,430][105620] Updated weights for policy 1, policy_version 134698 (0.0007) [2023-12-26 16:20:35,475][105692] Updated weights for policy 0, policy_version 134049 (0.0005) [2023-12-26 16:20:35,482][105620] Updated weights for policy 1, policy_version 134708 (0.0009) [2023-12-26 16:20:35,540][105620] Updated weights for policy 1, policy_version 134718 (0.0009) [2023-12-26 16:20:35,594][105620] Updated weights for policy 1, policy_version 134728 (0.0009) [2023-12-26 16:20:36,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19524.3, 300 sec: 19355.3). Total num frames: 68820992. Throughput: 0: 9639.7, 1: 9934.6. Samples: 68812452. Policy #0 lag: (min: 31.0, avg: 45.9, max: 63.0) [2023-12-26 16:20:36,062][104569] Avg episode reward: [(0, '9180.767'), (1, '9257.581')] [2023-12-26 16:20:36,090][105692] Updated weights for policy 0, policy_version 134059 (0.0007) [2023-12-26 16:20:36,148][105692] Updated weights for policy 0, policy_version 134069 (0.0008) [2023-12-26 16:20:36,203][105692] Updated weights for policy 0, policy_version 134079 (0.0009) [2023-12-26 16:20:36,443][105620] Updated weights for policy 1, policy_version 134738 (0.0010) [2023-12-26 16:20:36,501][105620] Updated weights for policy 1, policy_version 134748 (0.0010) [2023-12-26 16:20:36,562][105620] Updated weights for policy 1, policy_version 134758 (0.0008) [2023-12-26 16:20:36,907][105692] Updated weights for policy 0, policy_version 134089 (0.0008) [2023-12-26 16:20:36,968][105692] Updated weights for policy 0, policy_version 134099 (0.0008) [2023-12-26 16:20:37,023][105692] Updated weights for policy 0, policy_version 134109 (0.0009) [2023-12-26 16:20:37,085][105692] Updated weights for policy 0, policy_version 134119 (0.0009) [2023-12-26 16:20:37,360][105620] Updated weights for policy 1, policy_version 134768 (0.0010) [2023-12-26 16:20:37,425][105620] Updated weights for policy 1, policy_version 134778 (0.0009) [2023-12-26 16:20:37,491][105620] Updated weights for policy 1, policy_version 134788 (0.0010) [2023-12-26 16:20:37,675][105692] Updated weights for policy 0, policy_version 134129 (0.0009) [2023-12-26 16:20:37,731][105692] Updated weights for policy 0, policy_version 134139 (0.0008) [2023-12-26 16:20:37,792][105692] Updated weights for policy 0, policy_version 134149 (0.0007) [2023-12-26 16:20:38,302][105620] Updated weights for policy 1, policy_version 134798 (0.0009) [2023-12-26 16:20:38,362][105620] Updated weights for policy 1, policy_version 134808 (0.0008) [2023-12-26 16:20:38,418][105620] Updated weights for policy 1, policy_version 134818 (0.0008) [2023-12-26 16:20:38,525][105692] Updated weights for policy 0, policy_version 134159 (0.0009) [2023-12-26 16:20:38,575][105692] Updated weights for policy 0, policy_version 134169 (0.0011) [2023-12-26 16:20:38,624][105692] Updated weights for policy 0, policy_version 134179 (0.0011) [2023-12-26 16:20:39,190][105620] Updated weights for policy 1, policy_version 134828 (0.0009) [2023-12-26 16:20:39,256][105620] Updated weights for policy 1, policy_version 134838 (0.0008) [2023-12-26 16:20:39,310][105620] Updated weights for policy 1, policy_version 134848 (0.0008) [2023-12-26 16:20:39,333][105692] Updated weights for policy 0, policy_version 134189 (0.0009) [2023-12-26 16:20:39,401][105692] Updated weights for policy 0, policy_version 134199 (0.0008) [2023-12-26 16:20:39,468][105692] Updated weights for policy 0, policy_version 134209 (0.0009) [2023-12-26 16:20:40,115][105620] Updated weights for policy 1, policy_version 134858 (0.0008) [2023-12-26 16:20:40,176][105620] Updated weights for policy 1, policy_version 134868 (0.0009) [2023-12-26 16:20:40,240][105620] Updated weights for policy 1, policy_version 134878 (0.0007) [2023-12-26 16:20:40,242][105692] Updated weights for policy 0, policy_version 134219 (0.0008) [2023-12-26 16:20:40,293][105692] Updated weights for policy 0, policy_version 134229 (0.0007) [2023-12-26 16:20:40,299][105620] Updated weights for policy 1, policy_version 134888 (0.0007) [2023-12-26 16:20:40,352][105692] Updated weights for policy 0, policy_version 134239 (0.0009) [2023-12-26 16:20:41,009][105620] Updated weights for policy 1, policy_version 134898 (0.0008) [2023-12-26 16:20:41,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19355.3). Total num frames: 68911104. Throughput: 0: 9637.8, 1: 9796.7. Samples: 68925720. Policy #0 lag: (min: 31.0, avg: 45.9, max: 63.0) [2023-12-26 16:20:41,062][104569] Avg episode reward: [(0, '7132.212'), (1, '9164.416')] [2023-12-26 16:20:41,072][105620] Updated weights for policy 1, policy_version 134908 (0.0009) [2023-12-26 16:20:41,130][105620] Updated weights for policy 1, policy_version 134918 (0.0009) [2023-12-26 16:20:41,174][105692] Updated weights for policy 0, policy_version 134249 (0.0009) [2023-12-26 16:20:41,235][105692] Updated weights for policy 0, policy_version 134259 (0.0009) [2023-12-26 16:20:41,291][105692] Updated weights for policy 0, policy_version 134269 (0.0009) [2023-12-26 16:20:41,354][105692] Updated weights for policy 0, policy_version 134279 (0.0009) [2023-12-26 16:20:41,981][105620] Updated weights for policy 1, policy_version 134928 (0.0008) [2023-12-26 16:20:42,037][105620] Updated weights for policy 1, policy_version 134938 (0.0010) [2023-12-26 16:20:42,093][105620] Updated weights for policy 1, policy_version 134948 (0.0007) [2023-12-26 16:20:42,116][105692] Updated weights for policy 0, policy_version 134289 (0.0008) [2023-12-26 16:20:42,179][105692] Updated weights for policy 0, policy_version 134299 (0.0009) [2023-12-26 16:20:42,238][105692] Updated weights for policy 0, policy_version 134309 (0.0009) [2023-12-26 16:20:42,871][105692] Updated weights for policy 0, policy_version 134319 (0.0009) [2023-12-26 16:20:42,931][105692] Updated weights for policy 0, policy_version 134329 (0.0008) [2023-12-26 16:20:42,958][105620] Updated weights for policy 1, policy_version 134958 (0.0008) [2023-12-26 16:20:42,992][105692] Updated weights for policy 0, policy_version 134339 (0.0009) [2023-12-26 16:20:43,017][105620] Updated weights for policy 1, policy_version 134968 (0.0009) [2023-12-26 16:20:43,075][105620] Updated weights for policy 1, policy_version 134978 (0.0008) [2023-12-26 16:20:43,729][105692] Updated weights for policy 0, policy_version 134349 (0.0007) [2023-12-26 16:20:43,779][105692] Updated weights for policy 0, policy_version 134359 (0.0008) [2023-12-26 16:20:43,815][105620] Updated weights for policy 1, policy_version 134988 (0.0010) [2023-12-26 16:20:43,826][105692] Updated weights for policy 0, policy_version 134369 (0.0007) [2023-12-26 16:20:43,867][105620] Updated weights for policy 1, policy_version 134998 (0.0007) [2023-12-26 16:20:43,914][105620] Updated weights for policy 1, policy_version 135009 (0.0008) [2023-12-26 16:20:44,595][105692] Updated weights for policy 0, policy_version 134379 (0.0008) [2023-12-26 16:20:44,654][105692] Updated weights for policy 0, policy_version 134389 (0.0009) [2023-12-26 16:20:44,681][105620] Updated weights for policy 1, policy_version 135019 (0.0008) [2023-12-26 16:20:44,702][105692] Updated weights for policy 0, policy_version 134399 (0.0009) [2023-12-26 16:20:44,732][105620] Updated weights for policy 1, policy_version 135029 (0.0007) [2023-12-26 16:20:44,795][105620] Updated weights for policy 1, policy_version 135039 (0.0008) [2023-12-26 16:20:45,418][105692] Updated weights for policy 0, policy_version 134409 (0.0008) [2023-12-26 16:20:45,480][105692] Updated weights for policy 0, policy_version 134419 (0.0009) [2023-12-26 16:20:45,535][105692] Updated weights for policy 0, policy_version 134429 (0.0009) [2023-12-26 16:20:45,590][105620] Updated weights for policy 1, policy_version 135049 (0.0009) [2023-12-26 16:20:45,593][105692] Updated weights for policy 0, policy_version 134439 (0.0008) [2023-12-26 16:20:45,647][105620] Updated weights for policy 1, policy_version 135059 (0.0009) [2023-12-26 16:20:45,705][105620] Updated weights for policy 1, policy_version 135069 (0.0009) [2023-12-26 16:20:45,767][105620] Updated weights for policy 1, policy_version 135079 (0.0009) [2023-12-26 16:20:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19355.3). Total num frames: 69009408. Throughput: 0: 9633.7, 1: 9682.3. Samples: 68980984. Policy #0 lag: (min: 31.0, avg: 45.9, max: 63.0) [2023-12-26 16:20:46,062][104569] Avg episode reward: [(0, '882.007'), (1, '8981.278')] [2023-12-26 16:20:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000134440_34422784.pth... [2023-12-26 16:20:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000135080_34586624.pth... [2023-12-26 16:20:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000133320_34136064.pth [2023-12-26 16:20:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000133960_34299904.pth [2023-12-26 16:20:46,348][105692] Updated weights for policy 0, policy_version 134449 (0.0009) [2023-12-26 16:20:46,395][105692] Updated weights for policy 0, policy_version 134459 (0.0009) [2023-12-26 16:20:46,450][105692] Updated weights for policy 0, policy_version 134469 (0.0009) [2023-12-26 16:20:46,515][105620] Updated weights for policy 1, policy_version 135089 (0.0008) [2023-12-26 16:20:46,568][105620] Updated weights for policy 1, policy_version 135099 (0.0009) [2023-12-26 16:20:46,621][105620] Updated weights for policy 1, policy_version 135109 (0.0009) [2023-12-26 16:20:47,220][105692] Updated weights for policy 0, policy_version 134479 (0.0009) [2023-12-26 16:20:47,270][105692] Updated weights for policy 0, policy_version 134489 (0.0009) [2023-12-26 16:20:47,321][105692] Updated weights for policy 0, policy_version 134499 (0.0009) [2023-12-26 16:20:47,364][105620] Updated weights for policy 1, policy_version 135119 (0.0009) [2023-12-26 16:20:47,425][105620] Updated weights for policy 1, policy_version 135129 (0.0009) [2023-12-26 16:20:47,482][105620] Updated weights for policy 1, policy_version 135139 (0.0008) [2023-12-26 16:20:48,086][105692] Updated weights for policy 0, policy_version 134509 (0.0008) [2023-12-26 16:20:48,141][105692] Updated weights for policy 0, policy_version 134519 (0.0009) [2023-12-26 16:20:48,199][105692] Updated weights for policy 0, policy_version 134529 (0.0010) [2023-12-26 16:20:48,231][105620] Updated weights for policy 1, policy_version 135149 (0.0008) [2023-12-26 16:20:48,291][105620] Updated weights for policy 1, policy_version 135159 (0.0009) [2023-12-26 16:20:48,352][105620] Updated weights for policy 1, policy_version 135169 (0.0009) [2023-12-26 16:20:48,959][105692] Updated weights for policy 0, policy_version 134539 (0.0008) [2023-12-26 16:20:49,015][105692] Updated weights for policy 0, policy_version 134549 (0.0009) [2023-12-26 16:20:49,074][105692] Updated weights for policy 0, policy_version 134559 (0.0009) [2023-12-26 16:20:49,075][105620] Updated weights for policy 1, policy_version 135179 (0.0008) [2023-12-26 16:20:49,128][105620] Updated weights for policy 1, policy_version 135189 (0.0009) [2023-12-26 16:20:49,176][105620] Updated weights for policy 1, policy_version 135199 (0.0010) [2023-12-26 16:20:49,840][105692] Updated weights for policy 0, policy_version 134569 (0.0008) [2023-12-26 16:20:49,903][105692] Updated weights for policy 0, policy_version 134579 (0.0011) [2023-12-26 16:20:49,944][105620] Updated weights for policy 1, policy_version 135209 (0.0008) [2023-12-26 16:20:49,975][105692] Updated weights for policy 0, policy_version 134589 (0.0010) [2023-12-26 16:20:50,009][105620] Updated weights for policy 1, policy_version 135219 (0.0010) [2023-12-26 16:20:50,039][105692] Updated weights for policy 0, policy_version 134599 (0.0010) [2023-12-26 16:20:50,068][105620] Updated weights for policy 1, policy_version 135229 (0.0011) [2023-12-26 16:20:50,129][105620] Updated weights for policy 1, policy_version 135239 (0.0008) [2023-12-26 16:20:50,690][105692] Updated weights for policy 0, policy_version 134609 (0.0011) [2023-12-26 16:20:50,747][105692] Updated weights for policy 0, policy_version 134619 (0.0011) [2023-12-26 16:20:50,790][105620] Updated weights for policy 1, policy_version 135249 (0.0008) [2023-12-26 16:20:50,807][105692] Updated weights for policy 0, policy_version 134629 (0.0011) [2023-12-26 16:20:50,851][105620] Updated weights for policy 1, policy_version 135259 (0.0008) [2023-12-26 16:20:50,909][105620] Updated weights for policy 1, policy_version 135269 (0.0008) [2023-12-26 16:20:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 69107712. Throughput: 0: 9588.4, 1: 9633.4. Samples: 69092652. Policy #0 lag: (min: 31.0, avg: 45.9, max: 63.0) [2023-12-26 16:20:51,062][104569] Avg episode reward: [(0, '1051.286'), (1, '8982.574')] [2023-12-26 16:20:51,552][105620] Updated weights for policy 1, policy_version 135279 (0.0006) [2023-12-26 16:20:51,588][105692] Updated weights for policy 0, policy_version 134639 (0.0009) [2023-12-26 16:20:51,616][105620] Updated weights for policy 1, policy_version 135289 (0.0006) [2023-12-26 16:20:51,653][105692] Updated weights for policy 0, policy_version 134649 (0.0008) [2023-12-26 16:20:51,712][105620] Updated weights for policy 1, policy_version 135299 (0.0007) [2023-12-26 16:20:51,713][105692] Updated weights for policy 0, policy_version 134659 (0.0008) [2023-12-26 16:20:52,343][105620] Updated weights for policy 1, policy_version 135309 (0.0009) [2023-12-26 16:20:52,397][105620] Updated weights for policy 1, policy_version 135319 (0.0009) [2023-12-26 16:20:52,447][105620] Updated weights for policy 1, policy_version 135329 (0.0010) [2023-12-26 16:20:52,499][105692] Updated weights for policy 0, policy_version 134669 (0.0008) [2023-12-26 16:20:52,556][105692] Updated weights for policy 0, policy_version 134679 (0.0010) [2023-12-26 16:20:52,609][105692] Updated weights for policy 0, policy_version 134689 (0.0007) [2023-12-26 16:20:53,141][105620] Updated weights for policy 1, policy_version 135339 (0.0008) [2023-12-26 16:20:53,191][105620] Updated weights for policy 1, policy_version 135349 (0.0008) [2023-12-26 16:20:53,252][105620] Updated weights for policy 1, policy_version 135359 (0.0005) [2023-12-26 16:20:53,375][105692] Updated weights for policy 0, policy_version 134699 (0.0007) [2023-12-26 16:20:53,453][105692] Updated weights for policy 0, policy_version 134709 (0.0009) [2023-12-26 16:20:53,518][105692] Updated weights for policy 0, policy_version 134719 (0.0009) [2023-12-26 16:20:53,882][105620] Updated weights for policy 1, policy_version 135369 (0.0006) [2023-12-26 16:20:53,942][105620] Updated weights for policy 1, policy_version 135379 (0.0009) [2023-12-26 16:20:53,998][105620] Updated weights for policy 1, policy_version 135389 (0.0008) [2023-12-26 16:20:54,058][105620] Updated weights for policy 1, policy_version 135399 (0.0009) [2023-12-26 16:20:54,271][105692] Updated weights for policy 0, policy_version 134729 (0.0009) [2023-12-26 16:20:54,326][105692] Updated weights for policy 0, policy_version 134739 (0.0008) [2023-12-26 16:20:54,385][105692] Updated weights for policy 0, policy_version 134750 (0.0007) [2023-12-26 16:20:54,447][105692] Updated weights for policy 0, policy_version 134760 (0.0005) [2023-12-26 16:20:54,744][105620] Updated weights for policy 1, policy_version 135409 (0.0006) [2023-12-26 16:20:54,808][105620] Updated weights for policy 1, policy_version 135419 (0.0010) [2023-12-26 16:20:54,870][105620] Updated weights for policy 1, policy_version 135429 (0.0010) [2023-12-26 16:20:55,246][105692] Updated weights for policy 0, policy_version 134770 (0.0009) [2023-12-26 16:20:55,312][105692] Updated weights for policy 0, policy_version 134780 (0.0009) [2023-12-26 16:20:55,376][105692] Updated weights for policy 0, policy_version 134790 (0.0009) [2023-12-26 16:20:55,473][105620] Updated weights for policy 1, policy_version 135439 (0.0007) [2023-12-26 16:20:55,527][105620] Updated weights for policy 1, policy_version 135449 (0.0005) [2023-12-26 16:20:55,585][105620] Updated weights for policy 1, policy_version 135459 (0.0005) [2023-12-26 16:20:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19327.6). Total num frames: 69197824. Throughput: 0: 9574.9, 1: 9708.9. Samples: 69210620. Policy #0 lag: (min: 2.0, avg: 18.3, max: 34.0) [2023-12-26 16:20:56,062][104569] Avg episode reward: [(0, '6418.192'), (1, '8900.192')] [2023-12-26 16:20:56,128][105620] Updated weights for policy 1, policy_version 135469 (0.0008) [2023-12-26 16:20:56,187][105620] Updated weights for policy 1, policy_version 135479 (0.0010) [2023-12-26 16:20:56,228][105692] Updated weights for policy 0, policy_version 134800 (0.0007) [2023-12-26 16:20:56,245][105620] Updated weights for policy 1, policy_version 135489 (0.0010) [2023-12-26 16:20:56,278][105692] Updated weights for policy 0, policy_version 134810 (0.0006) [2023-12-26 16:20:56,332][105692] Updated weights for policy 0, policy_version 134820 (0.0008) [2023-12-26 16:20:56,951][105620] Updated weights for policy 1, policy_version 135499 (0.0009) [2023-12-26 16:20:57,019][105620] Updated weights for policy 1, policy_version 135509 (0.0005) [2023-12-26 16:20:57,072][105692] Updated weights for policy 0, policy_version 134830 (0.0007) [2023-12-26 16:20:57,082][105620] Updated weights for policy 1, policy_version 135519 (0.0009) [2023-12-26 16:20:57,121][105692] Updated weights for policy 0, policy_version 134840 (0.0005) [2023-12-26 16:20:57,169][105692] Updated weights for policy 0, policy_version 134850 (0.0005) [2023-12-26 16:20:57,733][105620] Updated weights for policy 1, policy_version 135529 (0.0010) [2023-12-26 16:20:57,790][105620] Updated weights for policy 1, policy_version 135539 (0.0009) [2023-12-26 16:20:57,796][105692] Updated weights for policy 0, policy_version 134860 (0.0006) [2023-12-26 16:20:57,852][105692] Updated weights for policy 0, policy_version 134870 (0.0006) [2023-12-26 16:20:57,854][105620] Updated weights for policy 1, policy_version 135549 (0.0008) [2023-12-26 16:20:57,907][105620] Updated weights for policy 1, policy_version 135559 (0.0005) [2023-12-26 16:20:57,914][105692] Updated weights for policy 0, policy_version 134880 (0.0006) [2023-12-26 16:20:58,518][105692] Updated weights for policy 0, policy_version 134890 (0.0007) [2023-12-26 16:20:58,583][105692] Updated weights for policy 0, policy_version 134900 (0.0011) [2023-12-26 16:20:58,585][105620] Updated weights for policy 1, policy_version 135569 (0.0007) [2023-12-26 16:20:58,643][105692] Updated weights for policy 0, policy_version 134910 (0.0011) [2023-12-26 16:20:58,649][105620] Updated weights for policy 1, policy_version 135579 (0.0005) [2023-12-26 16:20:58,709][105620] Updated weights for policy 1, policy_version 135589 (0.0008) [2023-12-26 16:20:58,710][105692] Updated weights for policy 0, policy_version 134920 (0.0007) [2023-12-26 16:20:59,565][105620] Updated weights for policy 1, policy_version 135599 (0.0010) [2023-12-26 16:20:59,570][105692] Updated weights for policy 0, policy_version 134930 (0.0010) [2023-12-26 16:20:59,624][105620] Updated weights for policy 1, policy_version 135609 (0.0010) [2023-12-26 16:20:59,629][105692] Updated weights for policy 0, policy_version 134940 (0.0010) [2023-12-26 16:20:59,677][105620] Updated weights for policy 1, policy_version 135619 (0.0010) [2023-12-26 16:20:59,683][105692] Updated weights for policy 0, policy_version 134950 (0.0010) [2023-12-26 16:21:00,392][105620] Updated weights for policy 1, policy_version 135629 (0.0010) [2023-12-26 16:21:00,419][105692] Updated weights for policy 0, policy_version 134960 (0.0007) [2023-12-26 16:21:00,451][105620] Updated weights for policy 1, policy_version 135639 (0.0010) [2023-12-26 16:21:00,472][105692] Updated weights for policy 0, policy_version 134970 (0.0010) [2023-12-26 16:21:00,499][105620] Updated weights for policy 1, policy_version 135649 (0.0010) [2023-12-26 16:21:00,530][105692] Updated weights for policy 0, policy_version 134980 (0.0010) [2023-12-26 16:21:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19299.8). Total num frames: 69296128. Throughput: 0: 9614.9, 1: 9754.0. Samples: 69270708. Policy #0 lag: (min: 2.0, avg: 18.3, max: 34.0) [2023-12-26 16:21:01,062][104569] Avg episode reward: [(0, '8900.981'), (1, '8273.741')] [2023-12-26 16:21:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000134984_34562048.pth... [2023-12-26 16:21:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000135656_34734080.pth... [2023-12-26 16:21:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000133864_34275328.pth [2023-12-26 16:21:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000134568_34455552.pth [2023-12-26 16:21:01,106][105620] Updated weights for policy 1, policy_version 135659 (0.0009) [2023-12-26 16:21:01,166][105620] Updated weights for policy 1, policy_version 135669 (0.0008) [2023-12-26 16:21:01,216][105620] Updated weights for policy 1, policy_version 135679 (0.0007) [2023-12-26 16:21:01,272][105692] Updated weights for policy 0, policy_version 134990 (0.0008) [2023-12-26 16:21:01,342][105692] Updated weights for policy 0, policy_version 135000 (0.0006) [2023-12-26 16:21:01,418][105692] Updated weights for policy 0, policy_version 135010 (0.0007) [2023-12-26 16:21:01,855][105620] Updated weights for policy 1, policy_version 135689 (0.0009) [2023-12-26 16:21:01,920][105620] Updated weights for policy 1, policy_version 135699 (0.0010) [2023-12-26 16:21:01,986][105620] Updated weights for policy 1, policy_version 135709 (0.0011) [2023-12-26 16:21:02,049][105620] Updated weights for policy 1, policy_version 135719 (0.0011) [2023-12-26 16:21:02,077][105692] Updated weights for policy 0, policy_version 135020 (0.0005) [2023-12-26 16:21:02,125][105692] Updated weights for policy 0, policy_version 135030 (0.0006) [2023-12-26 16:21:02,183][105692] Updated weights for policy 0, policy_version 135040 (0.0005) [2023-12-26 16:21:02,717][105620] Updated weights for policy 1, policy_version 135729 (0.0011) [2023-12-26 16:21:02,759][105692] Updated weights for policy 0, policy_version 135050 (0.0006) [2023-12-26 16:21:02,775][105620] Updated weights for policy 1, policy_version 135739 (0.0010) [2023-12-26 16:21:02,805][105692] Updated weights for policy 0, policy_version 135060 (0.0009) [2023-12-26 16:21:02,837][105620] Updated weights for policy 1, policy_version 135749 (0.0011) [2023-12-26 16:21:02,855][105692] Updated weights for policy 0, policy_version 135070 (0.0007) [2023-12-26 16:21:02,906][105692] Updated weights for policy 0, policy_version 135080 (0.0008) [2023-12-26 16:21:03,565][105692] Updated weights for policy 0, policy_version 135090 (0.0005) [2023-12-26 16:21:03,571][105620] Updated weights for policy 1, policy_version 135759 (0.0009) [2023-12-26 16:21:03,613][105692] Updated weights for policy 0, policy_version 135100 (0.0005) [2023-12-26 16:21:03,633][105620] Updated weights for policy 1, policy_version 135769 (0.0010) [2023-12-26 16:21:03,665][105692] Updated weights for policy 0, policy_version 135110 (0.0007) [2023-12-26 16:21:03,691][105620] Updated weights for policy 1, policy_version 135779 (0.0010) [2023-12-26 16:21:04,427][105692] Updated weights for policy 0, policy_version 135120 (0.0008) [2023-12-26 16:21:04,430][105620] Updated weights for policy 1, policy_version 135789 (0.0010) [2023-12-26 16:21:04,484][105620] Updated weights for policy 1, policy_version 135799 (0.0011) [2023-12-26 16:21:04,487][105692] Updated weights for policy 0, policy_version 135130 (0.0010) [2023-12-26 16:21:04,540][105620] Updated weights for policy 1, policy_version 135809 (0.0011) [2023-12-26 16:21:04,543][105692] Updated weights for policy 0, policy_version 135140 (0.0007) [2023-12-26 16:21:05,287][105692] Updated weights for policy 0, policy_version 135150 (0.0008) [2023-12-26 16:21:05,290][105620] Updated weights for policy 1, policy_version 135819 (0.0009) [2023-12-26 16:21:05,338][105692] Updated weights for policy 0, policy_version 135160 (0.0008) [2023-12-26 16:21:05,341][105620] Updated weights for policy 1, policy_version 135829 (0.0006) [2023-12-26 16:21:05,390][105692] Updated weights for policy 0, policy_version 135170 (0.0008) [2023-12-26 16:21:05,404][105620] Updated weights for policy 1, policy_version 135839 (0.0005) [2023-12-26 16:21:05,984][105620] Updated weights for policy 1, policy_version 135849 (0.0005) [2023-12-26 16:21:06,050][105620] Updated weights for policy 1, policy_version 135859 (0.0006) [2023-12-26 16:21:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19327.6). Total num frames: 69394432. Throughput: 0: 9751.3, 1: 9673.5. Samples: 69388636. Policy #0 lag: (min: 2.0, avg: 18.3, max: 34.0) [2023-12-26 16:21:06,062][104569] Avg episode reward: [(0, '9080.555'), (1, '8354.428')] [2023-12-26 16:21:06,118][105620] Updated weights for policy 1, policy_version 135869 (0.0006) [2023-12-26 16:21:06,180][105620] Updated weights for policy 1, policy_version 135879 (0.0007) [2023-12-26 16:21:06,265][105692] Updated weights for policy 0, policy_version 135180 (0.0010) [2023-12-26 16:21:06,338][105692] Updated weights for policy 0, policy_version 135190 (0.0010) [2023-12-26 16:21:06,414][105692] Updated weights for policy 0, policy_version 135200 (0.0009) [2023-12-26 16:21:06,809][105620] Updated weights for policy 1, policy_version 135889 (0.0010) [2023-12-26 16:21:06,865][105620] Updated weights for policy 1, policy_version 135899 (0.0010) [2023-12-26 16:21:06,923][105620] Updated weights for policy 1, policy_version 135909 (0.0010) [2023-12-26 16:21:07,205][105692] Updated weights for policy 0, policy_version 135210 (0.0009) [2023-12-26 16:21:07,262][105692] Updated weights for policy 0, policy_version 135220 (0.0008) [2023-12-26 16:21:07,312][105692] Updated weights for policy 0, policy_version 135230 (0.0008) [2023-12-26 16:21:07,370][105692] Updated weights for policy 0, policy_version 135240 (0.0010) [2023-12-26 16:21:07,553][105620] Updated weights for policy 1, policy_version 135919 (0.0011) [2023-12-26 16:21:07,619][105620] Updated weights for policy 1, policy_version 135929 (0.0010) [2023-12-26 16:21:07,684][105620] Updated weights for policy 1, policy_version 135939 (0.0010) [2023-12-26 16:21:08,099][105692] Updated weights for policy 0, policy_version 135250 (0.0010) [2023-12-26 16:21:08,154][105692] Updated weights for policy 0, policy_version 135260 (0.0010) [2023-12-26 16:21:08,205][105692] Updated weights for policy 0, policy_version 135270 (0.0010) [2023-12-26 16:21:08,395][105620] Updated weights for policy 1, policy_version 135949 (0.0008) [2023-12-26 16:21:08,457][105620] Updated weights for policy 1, policy_version 135959 (0.0008) [2023-12-26 16:21:08,513][105620] Updated weights for policy 1, policy_version 135969 (0.0008) [2023-12-26 16:21:08,953][105692] Updated weights for policy 0, policy_version 135280 (0.0007) [2023-12-26 16:21:09,015][105692] Updated weights for policy 0, policy_version 135290 (0.0006) [2023-12-26 16:21:09,075][105692] Updated weights for policy 0, policy_version 135300 (0.0005) [2023-12-26 16:21:09,307][105620] Updated weights for policy 1, policy_version 135979 (0.0009) [2023-12-26 16:21:09,374][105620] Updated weights for policy 1, policy_version 135989 (0.0009) [2023-12-26 16:21:09,444][105620] Updated weights for policy 1, policy_version 135999 (0.0009) [2023-12-26 16:21:09,816][105692] Updated weights for policy 0, policy_version 135310 (0.0008) [2023-12-26 16:21:09,885][105692] Updated weights for policy 0, policy_version 135320 (0.0008) [2023-12-26 16:21:09,947][105692] Updated weights for policy 0, policy_version 135330 (0.0008) [2023-12-26 16:21:10,192][105620] Updated weights for policy 1, policy_version 136009 (0.0009) [2023-12-26 16:21:10,250][105620] Updated weights for policy 1, policy_version 136019 (0.0010) [2023-12-26 16:21:10,309][105620] Updated weights for policy 1, policy_version 136029 (0.0010) [2023-12-26 16:21:10,366][105620] Updated weights for policy 1, policy_version 136039 (0.0010) [2023-12-26 16:21:10,635][105692] Updated weights for policy 0, policy_version 135340 (0.0009) [2023-12-26 16:21:10,697][105692] Updated weights for policy 0, policy_version 135350 (0.0009) [2023-12-26 16:21:10,765][105692] Updated weights for policy 0, policy_version 135360 (0.0010) [2023-12-26 16:21:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19327.6). Total num frames: 69492736. Throughput: 0: 9724.3, 1: 9651.4. Samples: 69503096. Policy #0 lag: (min: 2.0, avg: 18.3, max: 34.0) [2023-12-26 16:21:11,062][104569] Avg episode reward: [(0, '8905.098'), (1, '8533.110')] [2023-12-26 16:21:11,209][105620] Updated weights for policy 1, policy_version 136049 (0.0009) [2023-12-26 16:21:11,276][105620] Updated weights for policy 1, policy_version 136059 (0.0010) [2023-12-26 16:21:11,330][105620] Updated weights for policy 1, policy_version 136069 (0.0010) [2023-12-26 16:21:11,436][105692] Updated weights for policy 0, policy_version 135370 (0.0009) [2023-12-26 16:21:11,498][105692] Updated weights for policy 0, policy_version 135380 (0.0005) [2023-12-26 16:21:11,548][105692] Updated weights for policy 0, policy_version 135390 (0.0005) [2023-12-26 16:21:11,597][105692] Updated weights for policy 0, policy_version 135400 (0.0005) [2023-12-26 16:21:12,192][105620] Updated weights for policy 1, policy_version 136079 (0.0009) [2023-12-26 16:21:12,250][105620] Updated weights for policy 1, policy_version 136089 (0.0007) [2023-12-26 16:21:12,256][105692] Updated weights for policy 0, policy_version 135410 (0.0006) [2023-12-26 16:21:12,309][105620] Updated weights for policy 1, policy_version 136099 (0.0006) [2023-12-26 16:21:12,324][105692] Updated weights for policy 0, policy_version 135420 (0.0008) [2023-12-26 16:21:12,393][105692] Updated weights for policy 0, policy_version 135430 (0.0008) [2023-12-26 16:21:12,992][105620] Updated weights for policy 1, policy_version 136109 (0.0007) [2023-12-26 16:21:13,059][105620] Updated weights for policy 1, policy_version 136119 (0.0005) [2023-12-26 16:21:13,124][105620] Updated weights for policy 1, policy_version 136129 (0.0006) [2023-12-26 16:21:13,149][105692] Updated weights for policy 0, policy_version 135440 (0.0008) [2023-12-26 16:21:13,202][105692] Updated weights for policy 0, policy_version 135450 (0.0010) [2023-12-26 16:21:13,253][105692] Updated weights for policy 0, policy_version 135460 (0.0010) [2023-12-26 16:21:13,716][105620] Updated weights for policy 1, policy_version 136139 (0.0006) [2023-12-26 16:21:13,789][105620] Updated weights for policy 1, policy_version 136149 (0.0005) [2023-12-26 16:21:13,849][105620] Updated weights for policy 1, policy_version 136159 (0.0006) [2023-12-26 16:21:13,879][105692] Updated weights for policy 0, policy_version 135470 (0.0007) [2023-12-26 16:21:13,939][105692] Updated weights for policy 0, policy_version 135480 (0.0005) [2023-12-26 16:21:13,998][105692] Updated weights for policy 0, policy_version 135490 (0.0005) [2023-12-26 16:21:14,442][105620] Updated weights for policy 1, policy_version 136169 (0.0010) [2023-12-26 16:21:14,511][105620] Updated weights for policy 1, policy_version 136179 (0.0006) [2023-12-26 16:21:14,582][105620] Updated weights for policy 1, policy_version 136189 (0.0006) [2023-12-26 16:21:14,643][105692] Updated weights for policy 0, policy_version 135500 (0.0005) [2023-12-26 16:21:14,651][105620] Updated weights for policy 1, policy_version 136199 (0.0010) [2023-12-26 16:21:14,690][105692] Updated weights for policy 0, policy_version 135510 (0.0009) [2023-12-26 16:21:14,738][105692] Updated weights for policy 0, policy_version 135520 (0.0008) [2023-12-26 16:21:15,354][105620] Updated weights for policy 1, policy_version 136209 (0.0010) [2023-12-26 16:21:15,375][105692] Updated weights for policy 0, policy_version 135530 (0.0006) [2023-12-26 16:21:15,421][105620] Updated weights for policy 1, policy_version 136219 (0.0009) [2023-12-26 16:21:15,441][105692] Updated weights for policy 0, policy_version 135540 (0.0006) [2023-12-26 16:21:15,477][105620] Updated weights for policy 1, policy_version 136229 (0.0008) [2023-12-26 16:21:15,500][105692] Updated weights for policy 0, policy_version 135550 (0.0007) [2023-12-26 16:21:15,556][105692] Updated weights for policy 0, policy_version 135560 (0.0006) [2023-12-26 16:21:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19355.3). Total num frames: 69591040. Throughput: 0: 9627.8, 1: 9587.2. Samples: 69561772. Policy #0 lag: (min: 2.0, avg: 18.3, max: 34.0) [2023-12-26 16:21:16,062][104569] Avg episode reward: [(0, '8999.783'), (1, '8635.516')] [2023-12-26 16:21:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000135560_34709504.pth... [2023-12-26 16:21:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000136232_34881536.pth... [2023-12-26 16:21:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000134440_34422784.pth [2023-12-26 16:21:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000135080_34586624.pth [2023-12-26 16:21:16,212][105692] Updated weights for policy 0, policy_version 135570 (0.0008) [2023-12-26 16:21:16,261][105692] Updated weights for policy 0, policy_version 135580 (0.0008) [2023-12-26 16:21:16,271][105620] Updated weights for policy 1, policy_version 136239 (0.0008) [2023-12-26 16:21:16,310][105692] Updated weights for policy 0, policy_version 135590 (0.0005) [2023-12-26 16:21:16,322][105620] Updated weights for policy 1, policy_version 136249 (0.0008) [2023-12-26 16:21:16,381][105620] Updated weights for policy 1, policy_version 136259 (0.0006) [2023-12-26 16:21:16,936][105692] Updated weights for policy 0, policy_version 135600 (0.0009) [2023-12-26 16:21:16,996][105620] Updated weights for policy 1, policy_version 136269 (0.0005) [2023-12-26 16:21:17,001][105692] Updated weights for policy 0, policy_version 135610 (0.0009) [2023-12-26 16:21:17,064][105692] Updated weights for policy 0, policy_version 135620 (0.0006) [2023-12-26 16:21:17,064][105620] Updated weights for policy 1, policy_version 136279 (0.0005) [2023-12-26 16:21:17,124][105620] Updated weights for policy 1, policy_version 136289 (0.0005) [2023-12-26 16:21:17,625][105692] Updated weights for policy 0, policy_version 135630 (0.0010) [2023-12-26 16:21:17,673][105692] Updated weights for policy 0, policy_version 135640 (0.0010) [2023-12-26 16:21:17,725][105692] Updated weights for policy 0, policy_version 135650 (0.0008) [2023-12-26 16:21:17,778][105620] Updated weights for policy 1, policy_version 136299 (0.0005) [2023-12-26 16:21:17,830][105620] Updated weights for policy 1, policy_version 136309 (0.0005) [2023-12-26 16:21:17,881][105620] Updated weights for policy 1, policy_version 136319 (0.0006) [2023-12-26 16:21:18,445][105692] Updated weights for policy 0, policy_version 135660 (0.0009) [2023-12-26 16:21:18,507][105692] Updated weights for policy 0, policy_version 135670 (0.0008) [2023-12-26 16:21:18,554][105620] Updated weights for policy 1, policy_version 136329 (0.0006) [2023-12-26 16:21:18,565][105692] Updated weights for policy 0, policy_version 135680 (0.0006) [2023-12-26 16:21:18,616][105620] Updated weights for policy 1, policy_version 136339 (0.0009) [2023-12-26 16:21:18,671][105620] Updated weights for policy 1, policy_version 136349 (0.0008) [2023-12-26 16:21:18,732][105620] Updated weights for policy 1, policy_version 136359 (0.0008) [2023-12-26 16:21:19,333][105692] Updated weights for policy 0, policy_version 135690 (0.0007) [2023-12-26 16:21:19,402][105692] Updated weights for policy 0, policy_version 135700 (0.0007) [2023-12-26 16:21:19,406][105620] Updated weights for policy 1, policy_version 136369 (0.0007) [2023-12-26 16:21:19,463][105692] Updated weights for policy 0, policy_version 135710 (0.0008) [2023-12-26 16:21:19,470][105620] Updated weights for policy 1, policy_version 136379 (0.0006) [2023-12-26 16:21:19,531][105692] Updated weights for policy 0, policy_version 135720 (0.0009) [2023-12-26 16:21:19,537][105620] Updated weights for policy 1, policy_version 136389 (0.0006) [2023-12-26 16:21:20,254][105692] Updated weights for policy 0, policy_version 135730 (0.0009) [2023-12-26 16:21:20,273][105620] Updated weights for policy 1, policy_version 136399 (0.0006) [2023-12-26 16:21:20,317][105692] Updated weights for policy 0, policy_version 135740 (0.0008) [2023-12-26 16:21:20,323][105620] Updated weights for policy 1, policy_version 136409 (0.0005) [2023-12-26 16:21:20,383][105692] Updated weights for policy 0, policy_version 135750 (0.0008) [2023-12-26 16:21:20,389][105620] Updated weights for policy 1, policy_version 136419 (0.0007) [2023-12-26 16:21:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19355.3). Total num frames: 69689344. Throughput: 0: 9705.0, 1: 9680.6. Samples: 69684804. Policy #0 lag: (min: 2.0, avg: 18.3, max: 34.0) [2023-12-26 16:21:21,062][104569] Avg episode reward: [(0, '8990.966'), (1, '8724.332')] [2023-12-26 16:21:21,151][105620] Updated weights for policy 1, policy_version 136429 (0.0009) [2023-12-26 16:21:21,170][105692] Updated weights for policy 0, policy_version 135760 (0.0008) [2023-12-26 16:21:21,215][105620] Updated weights for policy 1, policy_version 136439 (0.0007) [2023-12-26 16:21:21,225][105692] Updated weights for policy 0, policy_version 135770 (0.0009) [2023-12-26 16:21:21,281][105620] Updated weights for policy 1, policy_version 136449 (0.0007) [2023-12-26 16:21:21,294][105692] Updated weights for policy 0, policy_version 135780 (0.0008) [2023-12-26 16:21:21,999][105692] Updated weights for policy 0, policy_version 135790 (0.0007) [2023-12-26 16:21:22,050][105620] Updated weights for policy 1, policy_version 136459 (0.0008) [2023-12-26 16:21:22,059][105692] Updated weights for policy 0, policy_version 135800 (0.0009) [2023-12-26 16:21:22,110][105620] Updated weights for policy 1, policy_version 136469 (0.0011) [2023-12-26 16:21:22,118][105692] Updated weights for policy 0, policy_version 135810 (0.0010) [2023-12-26 16:21:22,170][105620] Updated weights for policy 1, policy_version 136479 (0.0010) [2023-12-26 16:21:22,904][105692] Updated weights for policy 0, policy_version 135820 (0.0008) [2023-12-26 16:21:22,941][105620] Updated weights for policy 1, policy_version 136489 (0.0010) [2023-12-26 16:21:22,954][105692] Updated weights for policy 0, policy_version 135830 (0.0008) [2023-12-26 16:21:23,007][105620] Updated weights for policy 1, policy_version 136499 (0.0006) [2023-12-26 16:21:23,008][105692] Updated weights for policy 0, policy_version 135840 (0.0007) [2023-12-26 16:21:23,072][105620] Updated weights for policy 1, policy_version 136509 (0.0005) [2023-12-26 16:21:23,132][105620] Updated weights for policy 1, policy_version 136519 (0.0006) [2023-12-26 16:21:23,646][105692] Updated weights for policy 0, policy_version 135850 (0.0005) [2023-12-26 16:21:23,697][105692] Updated weights for policy 0, policy_version 135860 (0.0005) [2023-12-26 16:21:23,748][105692] Updated weights for policy 0, policy_version 135870 (0.0005) [2023-12-26 16:21:23,807][105692] Updated weights for policy 0, policy_version 135880 (0.0005) [2023-12-26 16:21:23,823][105620] Updated weights for policy 1, policy_version 136529 (0.0010) [2023-12-26 16:21:23,894][105620] Updated weights for policy 1, policy_version 136539 (0.0010) [2023-12-26 16:21:23,960][105620] Updated weights for policy 1, policy_version 136549 (0.0010) [2023-12-26 16:21:24,425][105692] Updated weights for policy 0, policy_version 135890 (0.0008) [2023-12-26 16:21:24,481][105692] Updated weights for policy 0, policy_version 135900 (0.0010) [2023-12-26 16:21:24,535][105692] Updated weights for policy 0, policy_version 135910 (0.0010) [2023-12-26 16:21:24,634][105620] Updated weights for policy 1, policy_version 136559 (0.0009) [2023-12-26 16:21:24,702][105620] Updated weights for policy 1, policy_version 136569 (0.0010) [2023-12-26 16:21:24,767][105620] Updated weights for policy 1, policy_version 136579 (0.0010) [2023-12-26 16:21:25,165][105692] Updated weights for policy 0, policy_version 135920 (0.0006) [2023-12-26 16:21:25,231][105692] Updated weights for policy 0, policy_version 135930 (0.0005) [2023-12-26 16:21:25,294][105692] Updated weights for policy 0, policy_version 135940 (0.0006) [2023-12-26 16:21:25,301][105620] Updated weights for policy 1, policy_version 136589 (0.0006) [2023-12-26 16:21:25,366][105620] Updated weights for policy 1, policy_version 136599 (0.0010) [2023-12-26 16:21:25,427][105620] Updated weights for policy 1, policy_version 136609 (0.0010) [2023-12-26 16:21:25,848][105692] Updated weights for policy 0, policy_version 135950 (0.0008) [2023-12-26 16:21:25,916][105692] Updated weights for policy 0, policy_version 135960 (0.0009) [2023-12-26 16:21:25,984][105692] Updated weights for policy 0, policy_version 135970 (0.0008) [2023-12-26 16:21:26,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.4, 300 sec: 19383.1). Total num frames: 69795840. Throughput: 0: 9719.1, 1: 9790.0. Samples: 69803628. Policy #0 lag: (min: 2.0, avg: 18.3, max: 34.0) [2023-12-26 16:21:26,062][104569] Avg episode reward: [(0, '9080.631'), (1, '8295.818')] [2023-12-26 16:21:26,131][105620] Updated weights for policy 1, policy_version 136619 (0.0010) [2023-12-26 16:21:26,183][105620] Updated weights for policy 1, policy_version 136629 (0.0010) [2023-12-26 16:21:26,240][105620] Updated weights for policy 1, policy_version 136639 (0.0008) [2023-12-26 16:21:26,771][105692] Updated weights for policy 0, policy_version 135980 (0.0007) [2023-12-26 16:21:26,819][105692] Updated weights for policy 0, policy_version 135990 (0.0008) [2023-12-26 16:21:26,870][105692] Updated weights for policy 0, policy_version 136000 (0.0008) [2023-12-26 16:21:26,911][105620] Updated weights for policy 1, policy_version 136649 (0.0010) [2023-12-26 16:21:26,971][105620] Updated weights for policy 1, policy_version 136659 (0.0010) [2023-12-26 16:21:27,035][105620] Updated weights for policy 1, policy_version 136669 (0.0006) [2023-12-26 16:21:27,084][105620] Updated weights for policy 1, policy_version 136679 (0.0006) [2023-12-26 16:21:27,488][105692] Updated weights for policy 0, policy_version 136010 (0.0008) [2023-12-26 16:21:27,548][105692] Updated weights for policy 0, policy_version 136020 (0.0007) [2023-12-26 16:21:27,603][105692] Updated weights for policy 0, policy_version 136030 (0.0010) [2023-12-26 16:21:27,660][105692] Updated weights for policy 0, policy_version 136040 (0.0010) [2023-12-26 16:21:27,688][105620] Updated weights for policy 1, policy_version 136689 (0.0005) [2023-12-26 16:21:27,749][105620] Updated weights for policy 1, policy_version 136699 (0.0008) [2023-12-26 16:21:27,804][105620] Updated weights for policy 1, policy_version 136709 (0.0005) [2023-12-26 16:21:28,382][105692] Updated weights for policy 0, policy_version 136050 (0.0007) [2023-12-26 16:21:28,421][105620] Updated weights for policy 1, policy_version 136719 (0.0008) [2023-12-26 16:21:28,446][105692] Updated weights for policy 0, policy_version 136060 (0.0006) [2023-12-26 16:21:28,490][105620] Updated weights for policy 1, policy_version 136729 (0.0006) [2023-12-26 16:21:28,508][105692] Updated weights for policy 0, policy_version 136070 (0.0006) [2023-12-26 16:21:28,549][105620] Updated weights for policy 1, policy_version 136739 (0.0007) [2023-12-26 16:21:29,080][105692] Updated weights for policy 0, policy_version 136080 (0.0008) [2023-12-26 16:21:29,102][105620] Updated weights for policy 1, policy_version 136749 (0.0006) [2023-12-26 16:21:29,135][105692] Updated weights for policy 0, policy_version 136090 (0.0007) [2023-12-26 16:21:29,153][105620] Updated weights for policy 1, policy_version 136759 (0.0006) [2023-12-26 16:21:29,190][105692] Updated weights for policy 0, policy_version 136100 (0.0008) [2023-12-26 16:21:29,211][105620] Updated weights for policy 1, policy_version 136769 (0.0008) [2023-12-26 16:21:29,912][105620] Updated weights for policy 1, policy_version 136779 (0.0009) [2023-12-26 16:21:29,976][105620] Updated weights for policy 1, policy_version 136789 (0.0008) [2023-12-26 16:21:29,995][105692] Updated weights for policy 0, policy_version 136110 (0.0006) [2023-12-26 16:21:30,039][105620] Updated weights for policy 1, policy_version 136799 (0.0009) [2023-12-26 16:21:30,055][105692] Updated weights for policy 0, policy_version 136120 (0.0008) [2023-12-26 16:21:30,116][105692] Updated weights for policy 0, policy_version 136130 (0.0008) [2023-12-26 16:21:30,792][105692] Updated weights for policy 0, policy_version 136140 (0.0009) [2023-12-26 16:21:30,813][105620] Updated weights for policy 1, policy_version 136809 (0.0008) [2023-12-26 16:21:30,845][105692] Updated weights for policy 0, policy_version 136151 (0.0009) [2023-12-26 16:21:30,859][105620] Updated weights for policy 1, policy_version 136819 (0.0005) [2023-12-26 16:21:30,897][105692] Updated weights for policy 0, policy_version 136161 (0.0008) [2023-12-26 16:21:30,916][105620] Updated weights for policy 1, policy_version 136829 (0.0005) [2023-12-26 16:21:30,964][105620] Updated weights for policy 1, policy_version 136839 (0.0005) [2023-12-26 16:21:31,062][104569] Fps is (10 sec: 21299.1, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 69902336. Throughput: 0: 9732.0, 1: 9935.8. Samples: 69866036. Policy #0 lag: (min: 2.0, avg: 18.3, max: 34.0) [2023-12-26 16:21:31,062][104569] Avg episode reward: [(0, '8989.486'), (1, '7989.984')] [2023-12-26 16:21:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000136168_34865152.pth... [2023-12-26 16:21:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000136840_35037184.pth... [2023-12-26 16:21:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000134984_34562048.pth [2023-12-26 16:21:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000135656_34734080.pth [2023-12-26 16:21:31,650][105620] Updated weights for policy 1, policy_version 136849 (0.0008) [2023-12-26 16:21:31,704][105692] Updated weights for policy 0, policy_version 136171 (0.0010) [2023-12-26 16:21:31,710][105620] Updated weights for policy 1, policy_version 136859 (0.0009) [2023-12-26 16:21:31,767][105620] Updated weights for policy 1, policy_version 136869 (0.0006) [2023-12-26 16:21:31,768][105692] Updated weights for policy 0, policy_version 136181 (0.0008) [2023-12-26 16:21:31,831][105692] Updated weights for policy 0, policy_version 136191 (0.0008) [2023-12-26 16:21:32,333][105620] Updated weights for policy 1, policy_version 136879 (0.0006) [2023-12-26 16:21:32,398][105620] Updated weights for policy 1, policy_version 136889 (0.0008) [2023-12-26 16:21:32,464][105620] Updated weights for policy 1, policy_version 136899 (0.0007) [2023-12-26 16:21:32,649][105692] Updated weights for policy 0, policy_version 136201 (0.0010) [2023-12-26 16:21:32,711][105692] Updated weights for policy 0, policy_version 136211 (0.0007) [2023-12-26 16:21:32,762][105692] Updated weights for policy 0, policy_version 136221 (0.0005) [2023-12-26 16:21:32,811][105692] Updated weights for policy 0, policy_version 136231 (0.0005) [2023-12-26 16:21:33,197][105620] Updated weights for policy 1, policy_version 136909 (0.0009) [2023-12-26 16:21:33,253][105620] Updated weights for policy 1, policy_version 136919 (0.0009) [2023-12-26 16:21:33,300][105620] Updated weights for policy 1, policy_version 136929 (0.0009) [2023-12-26 16:21:33,507][105692] Updated weights for policy 0, policy_version 136241 (0.0008) [2023-12-26 16:21:33,560][105692] Updated weights for policy 0, policy_version 136251 (0.0009) [2023-12-26 16:21:33,607][105692] Updated weights for policy 0, policy_version 136261 (0.0008) [2023-12-26 16:21:33,925][105620] Updated weights for policy 1, policy_version 136939 (0.0008) [2023-12-26 16:21:33,977][105620] Updated weights for policy 1, policy_version 136949 (0.0010) [2023-12-26 16:21:34,028][105620] Updated weights for policy 1, policy_version 136959 (0.0010) [2023-12-26 16:21:34,369][105692] Updated weights for policy 0, policy_version 136271 (0.0010) [2023-12-26 16:21:34,425][105692] Updated weights for policy 0, policy_version 136281 (0.0011) [2023-12-26 16:21:34,495][105692] Updated weights for policy 0, policy_version 136291 (0.0011) [2023-12-26 16:21:34,726][105620] Updated weights for policy 1, policy_version 136969 (0.0010) [2023-12-26 16:21:34,784][105620] Updated weights for policy 1, policy_version 136979 (0.0011) [2023-12-26 16:21:34,839][105620] Updated weights for policy 1, policy_version 136989 (0.0009) [2023-12-26 16:21:34,906][105620] Updated weights for policy 1, policy_version 136999 (0.0011) [2023-12-26 16:21:35,125][105692] Updated weights for policy 0, policy_version 136301 (0.0007) [2023-12-26 16:21:35,177][105692] Updated weights for policy 0, policy_version 136311 (0.0008) [2023-12-26 16:21:35,222][105692] Updated weights for policy 0, policy_version 136321 (0.0007) [2023-12-26 16:21:35,651][105620] Updated weights for policy 1, policy_version 137009 (0.0010) [2023-12-26 16:21:35,696][105620] Updated weights for policy 1, policy_version 137019 (0.0009) [2023-12-26 16:21:35,752][105620] Updated weights for policy 1, policy_version 137029 (0.0008) [2023-12-26 16:21:35,950][105692] Updated weights for policy 0, policy_version 136331 (0.0007) [2023-12-26 16:21:36,008][105692] Updated weights for policy 0, policy_version 136342 (0.0010) [2023-12-26 16:21:36,061][105692] Updated weights for policy 0, policy_version 136352 (0.0010) [2023-12-26 16:21:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 69992448. Throughput: 0: 9756.3, 1: 10050.3. Samples: 69983948. Policy #0 lag: (min: 2.0, avg: 18.3, max: 34.0) [2023-12-26 16:21:36,063][104569] Avg episode reward: [(0, '8988.360'), (1, '8261.192')] [2023-12-26 16:21:36,345][105620] Updated weights for policy 1, policy_version 137039 (0.0011) [2023-12-26 16:21:36,405][105620] Updated weights for policy 1, policy_version 137049 (0.0011) [2023-12-26 16:21:36,465][105620] Updated weights for policy 1, policy_version 137059 (0.0011) [2023-12-26 16:21:36,865][105692] Updated weights for policy 0, policy_version 136362 (0.0006) [2023-12-26 16:21:36,915][105692] Updated weights for policy 0, policy_version 136372 (0.0008) [2023-12-26 16:21:36,974][105692] Updated weights for policy 0, policy_version 136382 (0.0008) [2023-12-26 16:21:37,038][105692] Updated weights for policy 0, policy_version 136392 (0.0008) [2023-12-26 16:21:37,196][105620] Updated weights for policy 1, policy_version 137069 (0.0011) [2023-12-26 16:21:37,245][105620] Updated weights for policy 1, policy_version 137079 (0.0010) [2023-12-26 16:21:37,301][105620] Updated weights for policy 1, policy_version 137089 (0.0011) [2023-12-26 16:21:37,760][105692] Updated weights for policy 0, policy_version 136402 (0.0005) [2023-12-26 16:21:37,807][105692] Updated weights for policy 0, policy_version 136412 (0.0005) [2023-12-26 16:21:37,853][105692] Updated weights for policy 0, policy_version 136422 (0.0005) [2023-12-26 16:21:37,942][105620] Updated weights for policy 1, policy_version 137099 (0.0009) [2023-12-26 16:21:38,005][105620] Updated weights for policy 1, policy_version 137109 (0.0006) [2023-12-26 16:21:38,062][105620] Updated weights for policy 1, policy_version 137119 (0.0005) [2023-12-26 16:21:38,459][105692] Updated weights for policy 0, policy_version 136432 (0.0007) [2023-12-26 16:21:38,520][105692] Updated weights for policy 0, policy_version 136442 (0.0008) [2023-12-26 16:21:38,585][105692] Updated weights for policy 0, policy_version 136452 (0.0008) [2023-12-26 16:21:38,622][105620] Updated weights for policy 1, policy_version 137129 (0.0006) [2023-12-26 16:21:38,681][105620] Updated weights for policy 1, policy_version 137139 (0.0005) [2023-12-26 16:21:38,748][105620] Updated weights for policy 1, policy_version 137149 (0.0005) [2023-12-26 16:21:38,821][105620] Updated weights for policy 1, policy_version 137159 (0.0006) [2023-12-26 16:21:39,390][105692] Updated weights for policy 0, policy_version 136462 (0.0008) [2023-12-26 16:21:39,458][105692] Updated weights for policy 0, policy_version 136472 (0.0008) [2023-12-26 16:21:39,515][105692] Updated weights for policy 0, policy_version 136482 (0.0007) [2023-12-26 16:21:39,520][105620] Updated weights for policy 1, policy_version 137169 (0.0010) [2023-12-26 16:21:39,585][105620] Updated weights for policy 1, policy_version 137179 (0.0011) [2023-12-26 16:21:39,648][105620] Updated weights for policy 1, policy_version 137189 (0.0011) [2023-12-26 16:21:40,266][105692] Updated weights for policy 0, policy_version 136492 (0.0009) [2023-12-26 16:21:40,325][105692] Updated weights for policy 0, policy_version 136502 (0.0008) [2023-12-26 16:21:40,357][105620] Updated weights for policy 1, policy_version 137199 (0.0011) [2023-12-26 16:21:40,387][105692] Updated weights for policy 0, policy_version 136512 (0.0006) [2023-12-26 16:21:40,410][105620] Updated weights for policy 1, policy_version 137209 (0.0011) [2023-12-26 16:21:40,460][105620] Updated weights for policy 1, policy_version 137219 (0.0011) [2023-12-26 16:21:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19383.1). Total num frames: 70090752. Throughput: 0: 9840.5, 1: 10010.9. Samples: 70103936. Policy #0 lag: (min: 2.0, avg: 18.3, max: 34.0) [2023-12-26 16:21:41,062][104569] Avg episode reward: [(0, '8804.099'), (1, '8475.717')] [2023-12-26 16:21:41,102][105692] Updated weights for policy 0, policy_version 136522 (0.0006) [2023-12-26 16:21:41,163][105692] Updated weights for policy 0, policy_version 136532 (0.0008) [2023-12-26 16:21:41,176][105620] Updated weights for policy 1, policy_version 137229 (0.0010) [2023-12-26 16:21:41,216][105692] Updated weights for policy 0, policy_version 136542 (0.0006) [2023-12-26 16:21:41,240][105620] Updated weights for policy 1, policy_version 137239 (0.0009) [2023-12-26 16:21:41,280][105692] Updated weights for policy 0, policy_version 136552 (0.0007) [2023-12-26 16:21:41,307][105620] Updated weights for policy 1, policy_version 137249 (0.0009) [2023-12-26 16:21:42,055][105692] Updated weights for policy 0, policy_version 136562 (0.0008) [2023-12-26 16:21:42,057][105620] Updated weights for policy 1, policy_version 137259 (0.0008) [2023-12-26 16:21:42,115][105620] Updated weights for policy 1, policy_version 137269 (0.0007) [2023-12-26 16:21:42,117][105692] Updated weights for policy 0, policy_version 136572 (0.0007) [2023-12-26 16:21:42,175][105692] Updated weights for policy 0, policy_version 136582 (0.0007) [2023-12-26 16:21:42,177][105620] Updated weights for policy 1, policy_version 137279 (0.0006) [2023-12-26 16:21:42,850][105620] Updated weights for policy 1, policy_version 137289 (0.0008) [2023-12-26 16:21:42,897][105620] Updated weights for policy 1, policy_version 137299 (0.0009) [2023-12-26 16:21:42,945][105620] Updated weights for policy 1, policy_version 137309 (0.0007) [2023-12-26 16:21:42,968][105692] Updated weights for policy 0, policy_version 136592 (0.0008) [2023-12-26 16:21:42,998][105620] Updated weights for policy 1, policy_version 137319 (0.0008) [2023-12-26 16:21:43,032][105692] Updated weights for policy 0, policy_version 136602 (0.0008) [2023-12-26 16:21:43,085][105692] Updated weights for policy 0, policy_version 136612 (0.0008) [2023-12-26 16:21:43,716][105620] Updated weights for policy 1, policy_version 137329 (0.0008) [2023-12-26 16:21:43,773][105620] Updated weights for policy 1, policy_version 137339 (0.0007) [2023-12-26 16:21:43,824][105620] Updated weights for policy 1, policy_version 137349 (0.0005) [2023-12-26 16:21:43,855][105692] Updated weights for policy 0, policy_version 136622 (0.0009) [2023-12-26 16:21:43,924][105692] Updated weights for policy 0, policy_version 136632 (0.0009) [2023-12-26 16:21:43,990][105692] Updated weights for policy 0, policy_version 136642 (0.0009) [2023-12-26 16:21:44,359][105620] Updated weights for policy 1, policy_version 137359 (0.0005) [2023-12-26 16:21:44,415][105620] Updated weights for policy 1, policy_version 137369 (0.0008) [2023-12-26 16:21:44,461][105620] Updated weights for policy 1, policy_version 137379 (0.0008) [2023-12-26 16:21:44,731][105692] Updated weights for policy 0, policy_version 136652 (0.0010) [2023-12-26 16:21:44,800][105692] Updated weights for policy 0, policy_version 136662 (0.0008) [2023-12-26 16:21:44,869][105692] Updated weights for policy 0, policy_version 136672 (0.0010) [2023-12-26 16:21:45,154][105620] Updated weights for policy 1, policy_version 137389 (0.0009) [2023-12-26 16:21:45,224][105620] Updated weights for policy 1, policy_version 137399 (0.0009) [2023-12-26 16:21:45,290][105620] Updated weights for policy 1, policy_version 137409 (0.0009) [2023-12-26 16:21:45,666][105692] Updated weights for policy 0, policy_version 136682 (0.0010) [2023-12-26 16:21:45,717][105692] Updated weights for policy 0, policy_version 136692 (0.0010) [2023-12-26 16:21:45,763][105692] Updated weights for policy 0, policy_version 136702 (0.0010) [2023-12-26 16:21:45,814][105692] Updated weights for policy 0, policy_version 136712 (0.0010) [2023-12-26 16:21:46,040][105620] Updated weights for policy 1, policy_version 137419 (0.0009) [2023-12-26 16:21:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 70189056. Throughput: 0: 9760.8, 1: 10022.1. Samples: 70160940. Policy #0 lag: (min: 2.0, avg: 18.3, max: 34.0) [2023-12-26 16:21:46,062][104569] Avg episode reward: [(0, '8895.912'), (1, '8006.256')] [2023-12-26 16:21:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000136712_35004416.pth... [2023-12-26 16:21:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000135560_34709504.pth [2023-12-26 16:21:46,073][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_000136712_35004416.pth [2023-12-26 16:21:46,097][105620] Updated weights for policy 1, policy_version 137429 (0.0009) [2023-12-26 16:21:46,151][105620] Updated weights for policy 1, policy_version 137439 (0.0009) [2023-12-26 16:21:46,209][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000137448_35192832.pth... [2023-12-26 16:21:46,214][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000136232_34881536.pth [2023-12-26 16:21:46,215][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_000137448_35192832.pth [2023-12-26 16:21:46,471][105692] Updated weights for policy 0, policy_version 136723 (0.0009) [2023-12-26 16:21:46,527][105692] Updated weights for policy 0, policy_version 136733 (0.0009) [2023-12-26 16:21:46,585][105692] Updated weights for policy 0, policy_version 136743 (0.0008) [2023-12-26 16:21:46,866][105620] Updated weights for policy 1, policy_version 137449 (0.0009) [2023-12-26 16:21:46,930][105620] Updated weights for policy 1, policy_version 137459 (0.0005) [2023-12-26 16:21:46,980][105620] Updated weights for policy 1, policy_version 137469 (0.0009) [2023-12-26 16:21:47,031][105620] Updated weights for policy 1, policy_version 137479 (0.0010) [2023-12-26 16:21:47,374][105692] Updated weights for policy 0, policy_version 136753 (0.0006) [2023-12-26 16:21:47,426][105692] Updated weights for policy 0, policy_version 136763 (0.0008) [2023-12-26 16:21:47,479][105692] Updated weights for policy 0, policy_version 136773 (0.0010) [2023-12-26 16:21:47,669][105620] Updated weights for policy 1, policy_version 137489 (0.0006) [2023-12-26 16:21:47,723][105620] Updated weights for policy 1, policy_version 137499 (0.0005) [2023-12-26 16:21:47,779][105620] Updated weights for policy 1, policy_version 137509 (0.0006) [2023-12-26 16:21:48,260][105692] Updated weights for policy 0, policy_version 136783 (0.0009) [2023-12-26 16:21:48,318][105692] Updated weights for policy 0, policy_version 136793 (0.0009) [2023-12-26 16:21:48,375][105620] Updated weights for policy 1, policy_version 137519 (0.0009) [2023-12-26 16:21:48,381][105692] Updated weights for policy 0, policy_version 136803 (0.0008) [2023-12-26 16:21:48,434][105620] Updated weights for policy 1, policy_version 137529 (0.0008) [2023-12-26 16:21:48,482][105620] Updated weights for policy 1, policy_version 137539 (0.0009) [2023-12-26 16:21:49,087][105692] Updated weights for policy 0, policy_version 136813 (0.0008) [2023-12-26 16:21:49,142][105692] Updated weights for policy 0, policy_version 136823 (0.0008) [2023-12-26 16:21:49,198][105692] Updated weights for policy 0, policy_version 136833 (0.0008) [2023-12-26 16:21:49,270][105620] Updated weights for policy 1, policy_version 137549 (0.0010) [2023-12-26 16:21:49,330][105620] Updated weights for policy 1, policy_version 137559 (0.0010) [2023-12-26 16:21:49,394][105620] Updated weights for policy 1, policy_version 137569 (0.0011) [2023-12-26 16:21:49,961][105692] Updated weights for policy 0, policy_version 136843 (0.0008) [2023-12-26 16:21:50,017][105692] Updated weights for policy 0, policy_version 136853 (0.0005) [2023-12-26 16:21:50,072][105692] Updated weights for policy 0, policy_version 136863 (0.0007) [2023-12-26 16:21:50,172][105620] Updated weights for policy 1, policy_version 137579 (0.0011) [2023-12-26 16:21:50,247][105620] Updated weights for policy 1, policy_version 137589 (0.0008) [2023-12-26 16:21:50,309][105620] Updated weights for policy 1, policy_version 137599 (0.0006) [2023-12-26 16:21:50,805][105692] Updated weights for policy 0, policy_version 136873 (0.0006) [2023-12-26 16:21:50,855][105620] Updated weights for policy 1, policy_version 137609 (0.0008) [2023-12-26 16:21:50,866][105692] Updated weights for policy 0, policy_version 136883 (0.0009) [2023-12-26 16:21:50,920][105620] Updated weights for policy 1, policy_version 137619 (0.0006) [2023-12-26 16:21:50,925][105692] Updated weights for policy 0, policy_version 136893 (0.0009) [2023-12-26 16:21:50,983][105620] Updated weights for policy 1, policy_version 137629 (0.0005) [2023-12-26 16:21:50,984][105692] Updated weights for policy 0, policy_version 136903 (0.0009) [2023-12-26 16:21:51,050][105620] Updated weights for policy 1, policy_version 137639 (0.0009) [2023-12-26 16:21:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19438.6). Total num frames: 70295552. Throughput: 0: 9710.2, 1: 10047.7. Samples: 70277740. Policy #0 lag: (min: 2.0, avg: 18.3, max: 34.0) [2023-12-26 16:21:51,062][104569] Avg episode reward: [(0, '8899.348'), (1, '5173.910')] [2023-12-26 16:21:51,759][105620] Updated weights for policy 1, policy_version 137649 (0.0008) [2023-12-26 16:21:51,783][105692] Updated weights for policy 0, policy_version 136913 (0.0009) [2023-12-26 16:21:51,830][105692] Updated weights for policy 0, policy_version 136923 (0.0010) [2023-12-26 16:21:51,831][105620] Updated weights for policy 1, policy_version 137659 (0.0008) [2023-12-26 16:21:51,888][105620] Updated weights for policy 1, policy_version 137669 (0.0008) [2023-12-26 16:21:51,889][105692] Updated weights for policy 0, policy_version 136933 (0.0009) [2023-12-26 16:21:52,519][105620] Updated weights for policy 1, policy_version 137679 (0.0010) [2023-12-26 16:21:52,589][105620] Updated weights for policy 1, policy_version 137689 (0.0011) [2023-12-26 16:21:52,647][105620] Updated weights for policy 1, policy_version 137699 (0.0010) [2023-12-26 16:21:52,695][105692] Updated weights for policy 0, policy_version 136943 (0.0008) [2023-12-26 16:21:52,755][105692] Updated weights for policy 0, policy_version 136953 (0.0008) [2023-12-26 16:21:52,816][105692] Updated weights for policy 0, policy_version 136963 (0.0008) [2023-12-26 16:21:53,385][105620] Updated weights for policy 1, policy_version 137709 (0.0011) [2023-12-26 16:21:53,440][105620] Updated weights for policy 1, policy_version 137719 (0.0010) [2023-12-26 16:21:53,495][105620] Updated weights for policy 1, policy_version 137729 (0.0010) [2023-12-26 16:21:53,590][105692] Updated weights for policy 0, policy_version 136973 (0.0009) [2023-12-26 16:21:53,639][105692] Updated weights for policy 0, policy_version 136983 (0.0008) [2023-12-26 16:21:53,699][105692] Updated weights for policy 0, policy_version 136993 (0.0009) [2023-12-26 16:21:54,248][105620] Updated weights for policy 1, policy_version 137739 (0.0009) [2023-12-26 16:21:54,309][105620] Updated weights for policy 1, policy_version 137749 (0.0007) [2023-12-26 16:21:54,360][105620] Updated weights for policy 1, policy_version 137759 (0.0005) [2023-12-26 16:21:54,441][105692] Updated weights for policy 0, policy_version 137003 (0.0010) [2023-12-26 16:21:54,506][105692] Updated weights for policy 0, policy_version 137013 (0.0006) [2023-12-26 16:21:54,576][105692] Updated weights for policy 0, policy_version 137023 (0.0008) [2023-12-26 16:21:54,929][105620] Updated weights for policy 1, policy_version 137769 (0.0006) [2023-12-26 16:21:54,996][105620] Updated weights for policy 1, policy_version 137779 (0.0009) [2023-12-26 16:21:55,065][105620] Updated weights for policy 1, policy_version 137789 (0.0010) [2023-12-26 16:21:55,124][105620] Updated weights for policy 1, policy_version 137799 (0.0010) [2023-12-26 16:21:55,342][105692] Updated weights for policy 0, policy_version 137033 (0.0009) [2023-12-26 16:21:55,389][105692] Updated weights for policy 0, policy_version 137043 (0.0008) [2023-12-26 16:21:55,433][105692] Updated weights for policy 0, policy_version 137053 (0.0007) [2023-12-26 16:21:55,477][105692] Updated weights for policy 0, policy_version 137063 (0.0008) [2023-12-26 16:21:55,829][105620] Updated weights for policy 1, policy_version 137809 (0.0010) [2023-12-26 16:21:55,885][105620] Updated weights for policy 1, policy_version 137819 (0.0011) [2023-12-26 16:21:55,943][105620] Updated weights for policy 1, policy_version 137829 (0.0011) [2023-12-26 16:21:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19410.9). Total num frames: 70385664. Throughput: 0: 9696.2, 1: 10084.6. Samples: 70393236. Policy #0 lag: (min: 2.0, avg: 18.3, max: 34.0) [2023-12-26 16:21:56,062][104569] Avg episode reward: [(0, '8099.879'), (1, '3799.191')] [2023-12-26 16:21:56,292][105692] Updated weights for policy 0, policy_version 137073 (0.0008) [2023-12-26 16:21:56,355][105692] Updated weights for policy 0, policy_version 137083 (0.0009) [2023-12-26 16:21:56,407][105692] Updated weights for policy 0, policy_version 137093 (0.0008) [2023-12-26 16:21:56,651][105620] Updated weights for policy 1, policy_version 137839 (0.0007) [2023-12-26 16:21:56,715][105620] Updated weights for policy 1, policy_version 137849 (0.0009) [2023-12-26 16:21:56,767][105620] Updated weights for policy 1, policy_version 137859 (0.0010) [2023-12-26 16:21:57,204][105692] Updated weights for policy 0, policy_version 137103 (0.0010) [2023-12-26 16:21:57,262][105692] Updated weights for policy 0, policy_version 137113 (0.0010) [2023-12-26 16:21:57,323][105692] Updated weights for policy 0, policy_version 137123 (0.0010) [2023-12-26 16:21:57,454][105620] Updated weights for policy 1, policy_version 137869 (0.0008) [2023-12-26 16:21:57,515][105620] Updated weights for policy 1, policy_version 137879 (0.0005) [2023-12-26 16:21:57,582][105620] Updated weights for policy 1, policy_version 137889 (0.0009) [2023-12-26 16:21:58,010][105692] Updated weights for policy 0, policy_version 137133 (0.0009) [2023-12-26 16:21:58,063][105692] Updated weights for policy 0, policy_version 137143 (0.0005) [2023-12-26 16:21:58,122][105692] Updated weights for policy 0, policy_version 137153 (0.0008) [2023-12-26 16:21:58,226][105620] Updated weights for policy 1, policy_version 137899 (0.0010) [2023-12-26 16:21:58,299][105620] Updated weights for policy 1, policy_version 137909 (0.0009) [2023-12-26 16:21:58,361][105620] Updated weights for policy 1, policy_version 137919 (0.0009) [2023-12-26 16:21:58,931][105692] Updated weights for policy 0, policy_version 137163 (0.0008) [2023-12-26 16:21:58,993][105692] Updated weights for policy 0, policy_version 137173 (0.0009) [2023-12-26 16:21:59,062][105692] Updated weights for policy 0, policy_version 137183 (0.0009) [2023-12-26 16:21:59,121][105620] Updated weights for policy 1, policy_version 137929 (0.0009) [2023-12-26 16:21:59,187][105620] Updated weights for policy 1, policy_version 137939 (0.0010) [2023-12-26 16:21:59,254][105620] Updated weights for policy 1, policy_version 137949 (0.0008) [2023-12-26 16:21:59,310][105620] Updated weights for policy 1, policy_version 137959 (0.0007) [2023-12-26 16:21:59,848][105692] Updated weights for policy 0, policy_version 137193 (0.0009) [2023-12-26 16:21:59,915][105692] Updated weights for policy 0, policy_version 137203 (0.0009) [2023-12-26 16:21:59,993][105692] Updated weights for policy 0, policy_version 137213 (0.0006) [2023-12-26 16:22:00,056][105620] Updated weights for policy 1, policy_version 137969 (0.0008) [2023-12-26 16:22:00,058][105692] Updated weights for policy 0, policy_version 137223 (0.0006) [2023-12-26 16:22:00,106][105620] Updated weights for policy 1, policy_version 137979 (0.0008) [2023-12-26 16:22:00,158][105620] Updated weights for policy 1, policy_version 137989 (0.0008) [2023-12-26 16:22:00,812][105620] Updated weights for policy 1, policy_version 137999 (0.0009) [2023-12-26 16:22:00,817][105692] Updated weights for policy 0, policy_version 137233 (0.0009) [2023-12-26 16:22:00,871][105620] Updated weights for policy 1, policy_version 138009 (0.0007) [2023-12-26 16:22:00,877][105692] Updated weights for policy 0, policy_version 137243 (0.0007) [2023-12-26 16:22:00,920][105620] Updated weights for policy 1, policy_version 138019 (0.0006) [2023-12-26 16:22:00,926][105692] Updated weights for policy 0, policy_version 137253 (0.0006) [2023-12-26 16:22:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19797.3, 300 sec: 19410.9). Total num frames: 70483968. Throughput: 0: 9656.9, 1: 10101.0. Samples: 70450876. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-26 16:22:01,063][104569] Avg episode reward: [(0, '8182.331'), (1, '6800.404')] [2023-12-26 16:22:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000137256_35143680.pth... [2023-12-26 16:22:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000138024_35340288.pth... [2023-12-26 16:22:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000136168_34865152.pth [2023-12-26 16:22:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000136840_35037184.pth [2023-12-26 16:22:01,694][105620] Updated weights for policy 1, policy_version 138029 (0.0009) [2023-12-26 16:22:01,708][105692] Updated weights for policy 0, policy_version 137263 (0.0007) [2023-12-26 16:22:01,753][105620] Updated weights for policy 1, policy_version 138039 (0.0008) [2023-12-26 16:22:01,766][105692] Updated weights for policy 0, policy_version 137273 (0.0006) [2023-12-26 16:22:01,809][105620] Updated weights for policy 1, policy_version 138049 (0.0006) [2023-12-26 16:22:01,829][105692] Updated weights for policy 0, policy_version 137283 (0.0009) [2023-12-26 16:22:02,447][105692] Updated weights for policy 0, policy_version 137293 (0.0008) [2023-12-26 16:22:02,493][105620] Updated weights for policy 1, policy_version 138059 (0.0005) [2023-12-26 16:22:02,498][105692] Updated weights for policy 0, policy_version 137303 (0.0005) [2023-12-26 16:22:02,551][105620] Updated weights for policy 1, policy_version 138069 (0.0008) [2023-12-26 16:22:02,560][105692] Updated weights for policy 0, policy_version 137313 (0.0005) [2023-12-26 16:22:02,611][105620] Updated weights for policy 1, policy_version 138079 (0.0009) [2023-12-26 16:22:03,167][105692] Updated weights for policy 0, policy_version 137323 (0.0007) [2023-12-26 16:22:03,218][105692] Updated weights for policy 0, policy_version 137333 (0.0010) [2023-12-26 16:22:03,278][105692] Updated weights for policy 0, policy_version 137343 (0.0008) [2023-12-26 16:22:03,354][105620] Updated weights for policy 1, policy_version 138089 (0.0009) [2023-12-26 16:22:03,407][105620] Updated weights for policy 1, policy_version 138099 (0.0005) [2023-12-26 16:22:03,455][105620] Updated weights for policy 1, policy_version 138109 (0.0006) [2023-12-26 16:22:03,507][105620] Updated weights for policy 1, policy_version 138120 (0.0010) [2023-12-26 16:22:03,837][105692] Updated weights for policy 0, policy_version 137353 (0.0006) [2023-12-26 16:22:03,905][105692] Updated weights for policy 0, policy_version 137363 (0.0007) [2023-12-26 16:22:03,971][105692] Updated weights for policy 0, policy_version 137373 (0.0011) [2023-12-26 16:22:04,030][105692] Updated weights for policy 0, policy_version 137383 (0.0008) [2023-12-26 16:22:04,179][105620] Updated weights for policy 1, policy_version 138130 (0.0006) [2023-12-26 16:22:04,246][105620] Updated weights for policy 1, policy_version 138140 (0.0006) [2023-12-26 16:22:04,312][105620] Updated weights for policy 1, policy_version 138150 (0.0006) [2023-12-26 16:22:04,694][105692] Updated weights for policy 0, policy_version 137393 (0.0010) [2023-12-26 16:22:04,756][105692] Updated weights for policy 0, policy_version 137403 (0.0009) [2023-12-26 16:22:04,812][105692] Updated weights for policy 0, policy_version 137413 (0.0008) [2023-12-26 16:22:04,966][105620] Updated weights for policy 1, policy_version 138160 (0.0008) [2023-12-26 16:22:05,031][105620] Updated weights for policy 1, policy_version 138170 (0.0009) [2023-12-26 16:22:05,094][105620] Updated weights for policy 1, policy_version 138180 (0.0009) [2023-12-26 16:22:05,510][105692] Updated weights for policy 0, policy_version 137423 (0.0006) [2023-12-26 16:22:05,575][105692] Updated weights for policy 0, policy_version 137433 (0.0009) [2023-12-26 16:22:05,623][105692] Updated weights for policy 0, policy_version 137443 (0.0010) [2023-12-26 16:22:05,842][105620] Updated weights for policy 1, policy_version 138190 (0.0007) [2023-12-26 16:22:05,902][105620] Updated weights for policy 1, policy_version 138200 (0.0008) [2023-12-26 16:22:05,964][105620] Updated weights for policy 1, policy_version 138210 (0.0009) [2023-12-26 16:22:06,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.3, 300 sec: 19410.9). Total num frames: 70582272. Throughput: 0: 9573.2, 1: 10076.8. Samples: 70569052. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-26 16:22:06,062][104569] Avg episode reward: [(0, '8363.143'), (1, '8434.984')] [2023-12-26 16:22:06,291][105692] Updated weights for policy 0, policy_version 137453 (0.0008) [2023-12-26 16:22:06,354][105692] Updated weights for policy 0, policy_version 137463 (0.0006) [2023-12-26 16:22:06,408][105692] Updated weights for policy 0, policy_version 137473 (0.0005) [2023-12-26 16:22:06,783][105620] Updated weights for policy 1, policy_version 138220 (0.0009) [2023-12-26 16:22:06,842][105620] Updated weights for policy 1, policy_version 138230 (0.0009) [2023-12-26 16:22:06,898][105620] Updated weights for policy 1, policy_version 138240 (0.0008) [2023-12-26 16:22:07,063][105692] Updated weights for policy 0, policy_version 137483 (0.0008) [2023-12-26 16:22:07,119][105692] Updated weights for policy 0, policy_version 137493 (0.0011) [2023-12-26 16:22:07,175][105692] Updated weights for policy 0, policy_version 137503 (0.0011) [2023-12-26 16:22:07,634][105620] Updated weights for policy 1, policy_version 138250 (0.0006) [2023-12-26 16:22:07,692][105620] Updated weights for policy 1, policy_version 138260 (0.0006) [2023-12-26 16:22:07,751][105620] Updated weights for policy 1, policy_version 138270 (0.0005) [2023-12-26 16:22:07,801][105620] Updated weights for policy 1, policy_version 138280 (0.0007) [2023-12-26 16:22:07,919][105692] Updated weights for policy 0, policy_version 137513 (0.0011) [2023-12-26 16:22:07,971][105692] Updated weights for policy 0, policy_version 137523 (0.0010) [2023-12-26 16:22:08,023][105692] Updated weights for policy 0, policy_version 137533 (0.0010) [2023-12-26 16:22:08,078][105692] Updated weights for policy 0, policy_version 137543 (0.0007) [2023-12-26 16:22:08,450][105620] Updated weights for policy 1, policy_version 138290 (0.0010) [2023-12-26 16:22:08,501][105620] Updated weights for policy 1, policy_version 138300 (0.0009) [2023-12-26 16:22:08,562][105620] Updated weights for policy 1, policy_version 138310 (0.0009) [2023-12-26 16:22:08,845][105692] Updated weights for policy 0, policy_version 137553 (0.0009) [2023-12-26 16:22:08,912][105692] Updated weights for policy 0, policy_version 137563 (0.0009) [2023-12-26 16:22:08,980][105692] Updated weights for policy 0, policy_version 137573 (0.0010) [2023-12-26 16:22:09,300][105620] Updated weights for policy 1, policy_version 138320 (0.0007) [2023-12-26 16:22:09,369][105620] Updated weights for policy 1, policy_version 138331 (0.0011) [2023-12-26 16:22:09,439][105620] Updated weights for policy 1, policy_version 138341 (0.0007) [2023-12-26 16:22:09,763][105692] Updated weights for policy 0, policy_version 137583 (0.0009) [2023-12-26 16:22:09,826][105692] Updated weights for policy 0, policy_version 137593 (0.0009) [2023-12-26 16:22:09,892][105692] Updated weights for policy 0, policy_version 137603 (0.0008) [2023-12-26 16:22:10,101][105620] Updated weights for policy 1, policy_version 138351 (0.0006) [2023-12-26 16:22:10,166][105620] Updated weights for policy 1, policy_version 138361 (0.0008) [2023-12-26 16:22:10,226][105620] Updated weights for policy 1, policy_version 138371 (0.0008) [2023-12-26 16:22:10,714][105692] Updated weights for policy 0, policy_version 137613 (0.0007) [2023-12-26 16:22:10,769][105692] Updated weights for policy 0, policy_version 137623 (0.0006) [2023-12-26 16:22:10,829][105692] Updated weights for policy 0, policy_version 137633 (0.0006) [2023-12-26 16:22:10,905][105620] Updated weights for policy 1, policy_version 138381 (0.0009) [2023-12-26 16:22:10,958][105620] Updated weights for policy 1, policy_version 138391 (0.0008) [2023-12-26 16:22:11,018][105620] Updated weights for policy 1, policy_version 138401 (0.0011) [2023-12-26 16:22:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19438.7). Total num frames: 70680576. Throughput: 0: 9502.3, 1: 10059.5. Samples: 70683912. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-26 16:22:11,062][104569] Avg episode reward: [(0, '8728.739'), (1, '8528.736')] [2023-12-26 16:22:11,526][105692] Updated weights for policy 0, policy_version 137643 (0.0009) [2023-12-26 16:22:11,584][105692] Updated weights for policy 0, policy_version 137653 (0.0009) [2023-12-26 16:22:11,641][105692] Updated weights for policy 0, policy_version 137663 (0.0008) [2023-12-26 16:22:11,858][105620] Updated weights for policy 1, policy_version 138411 (0.0009) [2023-12-26 16:22:11,925][105620] Updated weights for policy 1, policy_version 138421 (0.0009) [2023-12-26 16:22:11,985][105620] Updated weights for policy 1, policy_version 138431 (0.0009) [2023-12-26 16:22:12,391][105692] Updated weights for policy 0, policy_version 137673 (0.0008) [2023-12-26 16:22:12,446][105692] Updated weights for policy 0, policy_version 137683 (0.0005) [2023-12-26 16:22:12,498][105692] Updated weights for policy 0, policy_version 137693 (0.0005) [2023-12-26 16:22:12,568][105692] Updated weights for policy 0, policy_version 137703 (0.0010) [2023-12-26 16:22:12,711][105620] Updated weights for policy 1, policy_version 138441 (0.0009) [2023-12-26 16:22:12,764][105620] Updated weights for policy 1, policy_version 138451 (0.0010) [2023-12-26 16:22:12,817][105620] Updated weights for policy 1, policy_version 138461 (0.0011) [2023-12-26 16:22:12,876][105620] Updated weights for policy 1, policy_version 138471 (0.0008) [2023-12-26 16:22:13,207][105692] Updated weights for policy 0, policy_version 137713 (0.0009) [2023-12-26 16:22:13,267][105692] Updated weights for policy 0, policy_version 137723 (0.0009) [2023-12-26 16:22:13,323][105692] Updated weights for policy 0, policy_version 137733 (0.0007) [2023-12-26 16:22:13,437][105620] Updated weights for policy 1, policy_version 138481 (0.0006) [2023-12-26 16:22:13,495][105620] Updated weights for policy 1, policy_version 138491 (0.0010) [2023-12-26 16:22:13,554][105620] Updated weights for policy 1, policy_version 138501 (0.0010) [2023-12-26 16:22:14,049][105692] Updated weights for policy 0, policy_version 137743 (0.0009) [2023-12-26 16:22:14,104][105692] Updated weights for policy 0, policy_version 137753 (0.0006) [2023-12-26 16:22:14,113][105620] Updated weights for policy 1, policy_version 138511 (0.0011) [2023-12-26 16:22:14,158][105692] Updated weights for policy 0, policy_version 137763 (0.0005) [2023-12-26 16:22:14,164][105620] Updated weights for policy 1, policy_version 138521 (0.0010) [2023-12-26 16:22:14,213][105620] Updated weights for policy 1, policy_version 138531 (0.0010) [2023-12-26 16:22:14,910][105692] Updated weights for policy 0, policy_version 137773 (0.0007) [2023-12-26 16:22:14,972][105692] Updated weights for policy 0, policy_version 137783 (0.0007) [2023-12-26 16:22:14,978][105620] Updated weights for policy 1, policy_version 138541 (0.0011) [2023-12-26 16:22:15,032][105692] Updated weights for policy 0, policy_version 137793 (0.0005) [2023-12-26 16:22:15,038][105620] Updated weights for policy 1, policy_version 138551 (0.0011) [2023-12-26 16:22:15,100][105620] Updated weights for policy 1, policy_version 138561 (0.0011) [2023-12-26 16:22:15,772][105692] Updated weights for policy 0, policy_version 137803 (0.0007) [2023-12-26 16:22:15,791][105620] Updated weights for policy 1, policy_version 138571 (0.0009) [2023-12-26 16:22:15,831][105692] Updated weights for policy 0, policy_version 137813 (0.0007) [2023-12-26 16:22:15,846][105620] Updated weights for policy 1, policy_version 138581 (0.0006) [2023-12-26 16:22:15,889][105692] Updated weights for policy 0, policy_version 137823 (0.0005) [2023-12-26 16:22:15,916][105620] Updated weights for policy 1, policy_version 138591 (0.0009) [2023-12-26 16:22:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19797.2, 300 sec: 19438.6). Total num frames: 70778880. Throughput: 0: 9500.6, 1: 10007.7. Samples: 70743912. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-26 16:22:16,063][104569] Avg episode reward: [(0, '8996.275'), (1, '8361.772')] [2023-12-26 16:22:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000138600_35487744.pth... [2023-12-26 16:22:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000137832_35291136.pth... [2023-12-26 16:22:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000137448_35192832.pth [2023-12-26 16:22:16,081][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000136712_35004416.pth [2023-12-26 16:22:16,414][105692] Updated weights for policy 0, policy_version 137833 (0.0005) [2023-12-26 16:22:16,447][105620] Updated weights for policy 1, policy_version 138601 (0.0007) [2023-12-26 16:22:16,475][105692] Updated weights for policy 0, policy_version 137843 (0.0005) [2023-12-26 16:22:16,510][105620] Updated weights for policy 1, policy_version 138611 (0.0005) [2023-12-26 16:22:16,525][105692] Updated weights for policy 0, policy_version 137853 (0.0005) [2023-12-26 16:22:16,565][105620] Updated weights for policy 1, policy_version 138621 (0.0005) [2023-12-26 16:22:16,572][105692] Updated weights for policy 0, policy_version 137863 (0.0005) [2023-12-26 16:22:16,628][105620] Updated weights for policy 1, policy_version 138631 (0.0005) [2023-12-26 16:22:17,174][105620] Updated weights for policy 1, policy_version 138641 (0.0010) [2023-12-26 16:22:17,215][105692] Updated weights for policy 0, policy_version 137873 (0.0006) [2023-12-26 16:22:17,236][105620] Updated weights for policy 1, policy_version 138651 (0.0010) [2023-12-26 16:22:17,269][105692] Updated weights for policy 0, policy_version 137883 (0.0006) [2023-12-26 16:22:17,297][105620] Updated weights for policy 1, policy_version 138661 (0.0010) [2023-12-26 16:22:17,317][105692] Updated weights for policy 0, policy_version 137893 (0.0008) [2023-12-26 16:22:17,884][105692] Updated weights for policy 0, policy_version 137903 (0.0009) [2023-12-26 16:22:17,939][105620] Updated weights for policy 1, policy_version 138671 (0.0007) [2023-12-26 16:22:17,942][105692] Updated weights for policy 0, policy_version 137913 (0.0009) [2023-12-26 16:22:17,993][105620] Updated weights for policy 1, policy_version 138681 (0.0005) [2023-12-26 16:22:17,997][105692] Updated weights for policy 0, policy_version 137923 (0.0008) [2023-12-26 16:22:18,040][105620] Updated weights for policy 1, policy_version 138691 (0.0005) [2023-12-26 16:22:18,637][105620] Updated weights for policy 1, policy_version 138701 (0.0008) [2023-12-26 16:22:18,689][105620] Updated weights for policy 1, policy_version 138711 (0.0011) [2023-12-26 16:22:18,734][105620] Updated weights for policy 1, policy_version 138721 (0.0010) [2023-12-26 16:22:18,776][105692] Updated weights for policy 0, policy_version 137933 (0.0010) [2023-12-26 16:22:18,835][105692] Updated weights for policy 0, policy_version 137943 (0.0010) [2023-12-26 16:22:18,886][105692] Updated weights for policy 0, policy_version 137953 (0.0010) [2023-12-26 16:22:19,500][105620] Updated weights for policy 1, policy_version 138731 (0.0011) [2023-12-26 16:22:19,565][105620] Updated weights for policy 1, policy_version 138741 (0.0011) [2023-12-26 16:22:19,625][105620] Updated weights for policy 1, policy_version 138751 (0.0011) [2023-12-26 16:22:19,638][105692] Updated weights for policy 0, policy_version 137963 (0.0011) [2023-12-26 16:22:19,701][105692] Updated weights for policy 0, policy_version 137973 (0.0011) [2023-12-26 16:22:19,769][105692] Updated weights for policy 0, policy_version 137983 (0.0010) [2023-12-26 16:22:20,325][105620] Updated weights for policy 1, policy_version 138761 (0.0010) [2023-12-26 16:22:20,389][105620] Updated weights for policy 1, policy_version 138771 (0.0006) [2023-12-26 16:22:20,443][105692] Updated weights for policy 0, policy_version 137993 (0.0010) [2023-12-26 16:22:20,452][105620] Updated weights for policy 1, policy_version 138781 (0.0007) [2023-12-26 16:22:20,504][105692] Updated weights for policy 0, policy_version 138003 (0.0006) [2023-12-26 16:22:20,523][105620] Updated weights for policy 1, policy_version 138791 (0.0011) [2023-12-26 16:22:20,570][105692] Updated weights for policy 0, policy_version 138013 (0.0007) [2023-12-26 16:22:20,627][105692] Updated weights for policy 0, policy_version 138023 (0.0011) [2023-12-26 16:22:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 70877184. Throughput: 0: 9599.3, 1: 10058.9. Samples: 70868568. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-26 16:22:21,062][104569] Avg episode reward: [(0, '8907.194'), (1, '8631.001')] [2023-12-26 16:22:21,216][105620] Updated weights for policy 1, policy_version 138801 (0.0009) [2023-12-26 16:22:21,257][105692] Updated weights for policy 0, policy_version 138033 (0.0008) [2023-12-26 16:22:21,278][105620] Updated weights for policy 1, policy_version 138811 (0.0009) [2023-12-26 16:22:21,316][105692] Updated weights for policy 0, policy_version 138043 (0.0008) [2023-12-26 16:22:21,337][105620] Updated weights for policy 1, policy_version 138821 (0.0009) [2023-12-26 16:22:21,383][105692] Updated weights for policy 0, policy_version 138053 (0.0011) [2023-12-26 16:22:22,088][105620] Updated weights for policy 1, policy_version 138831 (0.0010) [2023-12-26 16:22:22,140][105620] Updated weights for policy 1, policy_version 138841 (0.0010) [2023-12-26 16:22:22,189][105692] Updated weights for policy 0, policy_version 138063 (0.0011) [2023-12-26 16:22:22,197][105620] Updated weights for policy 1, policy_version 138851 (0.0010) [2023-12-26 16:22:22,249][105692] Updated weights for policy 0, policy_version 138073 (0.0009) [2023-12-26 16:22:22,317][105692] Updated weights for policy 0, policy_version 138083 (0.0010) [2023-12-26 16:22:22,897][105620] Updated weights for policy 1, policy_version 138861 (0.0011) [2023-12-26 16:22:22,958][105620] Updated weights for policy 1, policy_version 138871 (0.0010) [2023-12-26 16:22:23,014][105620] Updated weights for policy 1, policy_version 138881 (0.0010) [2023-12-26 16:22:23,086][105692] Updated weights for policy 0, policy_version 138093 (0.0009) [2023-12-26 16:22:23,139][105692] Updated weights for policy 0, policy_version 138103 (0.0008) [2023-12-26 16:22:23,185][105692] Updated weights for policy 0, policy_version 138113 (0.0008) [2023-12-26 16:22:23,646][105620] Updated weights for policy 1, policy_version 138891 (0.0010) [2023-12-26 16:22:23,695][105620] Updated weights for policy 1, policy_version 138901 (0.0010) [2023-12-26 16:22:23,753][105620] Updated weights for policy 1, policy_version 138911 (0.0010) [2023-12-26 16:22:24,065][105692] Updated weights for policy 0, policy_version 138123 (0.0010) [2023-12-26 16:22:24,117][105692] Updated weights for policy 0, policy_version 138133 (0.0009) [2023-12-26 16:22:24,172][105692] Updated weights for policy 0, policy_version 138143 (0.0008) [2023-12-26 16:22:24,392][105620] Updated weights for policy 1, policy_version 138921 (0.0006) [2023-12-26 16:22:24,455][105620] Updated weights for policy 1, policy_version 138931 (0.0010) [2023-12-26 16:22:24,522][105620] Updated weights for policy 1, policy_version 138941 (0.0010) [2023-12-26 16:22:24,581][105620] Updated weights for policy 1, policy_version 138951 (0.0010) [2023-12-26 16:22:24,935][105692] Updated weights for policy 0, policy_version 138153 (0.0008) [2023-12-26 16:22:25,004][105692] Updated weights for policy 0, policy_version 138163 (0.0010) [2023-12-26 16:22:25,074][105692] Updated weights for policy 0, policy_version 138173 (0.0010) [2023-12-26 16:22:25,141][105692] Updated weights for policy 0, policy_version 138183 (0.0010) [2023-12-26 16:22:25,210][105620] Updated weights for policy 1, policy_version 138961 (0.0006) [2023-12-26 16:22:25,267][105620] Updated weights for policy 1, policy_version 138971 (0.0005) [2023-12-26 16:22:25,330][105620] Updated weights for policy 1, policy_version 138981 (0.0005) [2023-12-26 16:22:25,886][105620] Updated weights for policy 1, policy_version 138991 (0.0006) [2023-12-26 16:22:25,938][105620] Updated weights for policy 1, policy_version 139001 (0.0005) [2023-12-26 16:22:25,987][105692] Updated weights for policy 0, policy_version 138193 (0.0009) [2023-12-26 16:22:25,992][105620] Updated weights for policy 1, policy_version 139011 (0.0005) [2023-12-26 16:22:26,049][105692] Updated weights for policy 0, policy_version 138203 (0.0009) [2023-12-26 16:22:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 70975488. Throughput: 0: 9514.1, 1: 10082.5. Samples: 70985780. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-26 16:22:26,062][104569] Avg episode reward: [(0, '8907.095'), (1, '8989.465')] [2023-12-26 16:22:26,108][105692] Updated weights for policy 0, policy_version 138213 (0.0008) [2023-12-26 16:22:26,590][105620] Updated weights for policy 1, policy_version 139021 (0.0007) [2023-12-26 16:22:26,638][105620] Updated weights for policy 1, policy_version 139031 (0.0006) [2023-12-26 16:22:26,694][105620] Updated weights for policy 1, policy_version 139041 (0.0006) [2023-12-26 16:22:26,924][105692] Updated weights for policy 0, policy_version 138223 (0.0009) [2023-12-26 16:22:26,981][105692] Updated weights for policy 0, policy_version 138233 (0.0009) [2023-12-26 16:22:27,040][105692] Updated weights for policy 0, policy_version 138243 (0.0009) [2023-12-26 16:22:27,384][105620] Updated weights for policy 1, policy_version 139051 (0.0007) [2023-12-26 16:22:27,444][105620] Updated weights for policy 1, policy_version 139061 (0.0005) [2023-12-26 16:22:27,501][105620] Updated weights for policy 1, policy_version 139071 (0.0005) [2023-12-26 16:22:27,739][105692] Updated weights for policy 0, policy_version 138253 (0.0009) [2023-12-26 16:22:27,793][105692] Updated weights for policy 0, policy_version 138263 (0.0010) [2023-12-26 16:22:27,846][105692] Updated weights for policy 0, policy_version 138273 (0.0010) [2023-12-26 16:22:28,085][105620] Updated weights for policy 1, policy_version 139081 (0.0007) [2023-12-26 16:22:28,147][105620] Updated weights for policy 1, policy_version 139091 (0.0009) [2023-12-26 16:22:28,200][105620] Updated weights for policy 1, policy_version 139101 (0.0009) [2023-12-26 16:22:28,251][105620] Updated weights for policy 1, policy_version 139111 (0.0008) [2023-12-26 16:22:28,694][105692] Updated weights for policy 0, policy_version 138283 (0.0009) [2023-12-26 16:22:28,741][105692] Updated weights for policy 0, policy_version 138293 (0.0009) [2023-12-26 16:22:28,798][105692] Updated weights for policy 0, policy_version 138303 (0.0009) [2023-12-26 16:22:28,947][105620] Updated weights for policy 1, policy_version 139121 (0.0009) [2023-12-26 16:22:29,002][105620] Updated weights for policy 1, policy_version 139131 (0.0009) [2023-12-26 16:22:29,057][105620] Updated weights for policy 1, policy_version 139141 (0.0008) [2023-12-26 16:22:29,563][105692] Updated weights for policy 0, policy_version 138313 (0.0009) [2023-12-26 16:22:29,618][105692] Updated weights for policy 0, policy_version 138323 (0.0010) [2023-12-26 16:22:29,677][105692] Updated weights for policy 0, policy_version 138333 (0.0010) [2023-12-26 16:22:29,732][105692] Updated weights for policy 0, policy_version 138343 (0.0010) [2023-12-26 16:22:29,767][105620] Updated weights for policy 1, policy_version 139151 (0.0008) [2023-12-26 16:22:29,826][105620] Updated weights for policy 1, policy_version 139161 (0.0008) [2023-12-26 16:22:29,884][105620] Updated weights for policy 1, policy_version 139171 (0.0008) [2023-12-26 16:22:30,384][105692] Updated weights for policy 0, policy_version 138353 (0.0007) [2023-12-26 16:22:30,429][105692] Updated weights for policy 0, policy_version 138363 (0.0005) [2023-12-26 16:22:30,474][105692] Updated weights for policy 0, policy_version 138373 (0.0006) [2023-12-26 16:22:30,523][105620] Updated weights for policy 1, policy_version 139181 (0.0009) [2023-12-26 16:22:30,585][105620] Updated weights for policy 1, policy_version 139191 (0.0010) [2023-12-26 16:22:30,646][105620] Updated weights for policy 1, policy_version 139201 (0.0010) [2023-12-26 16:22:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 71073792. Throughput: 0: 9508.9, 1: 10134.6. Samples: 71044896. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-26 16:22:31,062][104569] Avg episode reward: [(0, '8910.635'), (1, '8630.227')] [2023-12-26 16:22:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000138376_35430400.pth... [2023-12-26 16:22:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000139208_35643392.pth... [2023-12-26 16:22:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000137256_35143680.pth [2023-12-26 16:22:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000138024_35340288.pth [2023-12-26 16:22:31,219][105692] Updated weights for policy 0, policy_version 138383 (0.0005) [2023-12-26 16:22:31,266][105620] Updated weights for policy 1, policy_version 139211 (0.0009) [2023-12-26 16:22:31,281][105692] Updated weights for policy 0, policy_version 138393 (0.0007) [2023-12-26 16:22:31,327][105620] Updated weights for policy 1, policy_version 139221 (0.0008) [2023-12-26 16:22:31,336][105692] Updated weights for policy 0, policy_version 138403 (0.0006) [2023-12-26 16:22:31,397][105620] Updated weights for policy 1, policy_version 139231 (0.0009) [2023-12-26 16:22:31,969][105692] Updated weights for policy 0, policy_version 138413 (0.0009) [2023-12-26 16:22:32,031][105692] Updated weights for policy 0, policy_version 138423 (0.0006) [2023-12-26 16:22:32,093][105692] Updated weights for policy 0, policy_version 138433 (0.0009) [2023-12-26 16:22:32,216][105620] Updated weights for policy 1, policy_version 139241 (0.0010) [2023-12-26 16:22:32,283][105620] Updated weights for policy 1, policy_version 139251 (0.0009) [2023-12-26 16:22:32,346][105620] Updated weights for policy 1, policy_version 139261 (0.0009) [2023-12-26 16:22:32,411][105620] Updated weights for policy 1, policy_version 139271 (0.0009) [2023-12-26 16:22:32,706][105692] Updated weights for policy 0, policy_version 138443 (0.0007) [2023-12-26 16:22:32,774][105692] Updated weights for policy 0, policy_version 138453 (0.0005) [2023-12-26 16:22:32,841][105692] Updated weights for policy 0, policy_version 138463 (0.0005) [2023-12-26 16:22:33,089][105620] Updated weights for policy 1, policy_version 139281 (0.0006) [2023-12-26 16:22:33,145][105620] Updated weights for policy 1, policy_version 139291 (0.0009) [2023-12-26 16:22:33,198][105620] Updated weights for policy 1, policy_version 139301 (0.0010) [2023-12-26 16:22:33,347][105692] Updated weights for policy 0, policy_version 138473 (0.0006) [2023-12-26 16:22:33,398][105692] Updated weights for policy 0, policy_version 138483 (0.0009) [2023-12-26 16:22:33,454][105692] Updated weights for policy 0, policy_version 138493 (0.0008) [2023-12-26 16:22:33,503][105692] Updated weights for policy 0, policy_version 138503 (0.0010) [2023-12-26 16:22:33,970][105620] Updated weights for policy 1, policy_version 139311 (0.0009) [2023-12-26 16:22:34,019][105620] Updated weights for policy 1, policy_version 139321 (0.0009) [2023-12-26 16:22:34,068][105620] Updated weights for policy 1, policy_version 139331 (0.0007) [2023-12-26 16:22:34,221][105692] Updated weights for policy 0, policy_version 138513 (0.0008) [2023-12-26 16:22:34,276][105692] Updated weights for policy 0, policy_version 138523 (0.0006) [2023-12-26 16:22:34,338][105692] Updated weights for policy 0, policy_version 138533 (0.0009) [2023-12-26 16:22:34,777][105620] Updated weights for policy 1, policy_version 139341 (0.0007) [2023-12-26 16:22:34,830][105620] Updated weights for policy 1, policy_version 139351 (0.0008) [2023-12-26 16:22:34,897][105620] Updated weights for policy 1, policy_version 139361 (0.0006) [2023-12-26 16:22:35,089][105692] Updated weights for policy 0, policy_version 138543 (0.0008) [2023-12-26 16:22:35,145][105692] Updated weights for policy 0, policy_version 138553 (0.0005) [2023-12-26 16:22:35,205][105692] Updated weights for policy 0, policy_version 138563 (0.0005) [2023-12-26 16:22:35,647][105620] Updated weights for policy 1, policy_version 139371 (0.0007) [2023-12-26 16:22:35,698][105620] Updated weights for policy 1, policy_version 139382 (0.0009) [2023-12-26 16:22:35,725][105692] Updated weights for policy 0, policy_version 138573 (0.0005) [2023-12-26 16:22:35,754][105620] Updated weights for policy 1, policy_version 139392 (0.0009) [2023-12-26 16:22:35,774][105692] Updated weights for policy 0, policy_version 138583 (0.0005) [2023-12-26 16:22:35,824][105692] Updated weights for policy 0, policy_version 138593 (0.0006) [2023-12-26 16:22:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19494.2). Total num frames: 71180288. Throughput: 0: 9637.5, 1: 10095.6. Samples: 71165728. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-26 16:22:36,062][104569] Avg episode reward: [(0, '9267.690'), (1, '8727.010')] [2023-12-26 16:22:36,385][105620] Updated weights for policy 1, policy_version 139402 (0.0009) [2023-12-26 16:22:36,451][105620] Updated weights for policy 1, policy_version 139412 (0.0008) [2023-12-26 16:22:36,521][105620] Updated weights for policy 1, policy_version 139422 (0.0008) [2023-12-26 16:22:36,588][105620] Updated weights for policy 1, policy_version 139432 (0.0008) [2023-12-26 16:22:36,629][105692] Updated weights for policy 0, policy_version 138603 (0.0008) [2023-12-26 16:22:36,694][105692] Updated weights for policy 0, policy_version 138613 (0.0009) [2023-12-26 16:22:36,759][105692] Updated weights for policy 0, policy_version 138623 (0.0008) [2023-12-26 16:22:37,347][105620] Updated weights for policy 1, policy_version 139442 (0.0008) [2023-12-26 16:22:37,370][105692] Updated weights for policy 0, policy_version 138633 (0.0006) [2023-12-26 16:22:37,417][105620] Updated weights for policy 1, policy_version 139452 (0.0006) [2023-12-26 16:22:37,433][105692] Updated weights for policy 0, policy_version 138643 (0.0008) [2023-12-26 16:22:37,483][105620] Updated weights for policy 1, policy_version 139462 (0.0007) [2023-12-26 16:22:37,486][105692] Updated weights for policy 0, policy_version 138653 (0.0007) [2023-12-26 16:22:37,535][105692] Updated weights for policy 0, policy_version 138663 (0.0005) [2023-12-26 16:22:38,153][105692] Updated weights for policy 0, policy_version 138673 (0.0008) [2023-12-26 16:22:38,204][105692] Updated weights for policy 0, policy_version 138683 (0.0009) [2023-12-26 16:22:38,261][105692] Updated weights for policy 0, policy_version 138693 (0.0008) [2023-12-26 16:22:38,264][105620] Updated weights for policy 1, policy_version 139472 (0.0007) [2023-12-26 16:22:38,325][105620] Updated weights for policy 1, policy_version 139482 (0.0008) [2023-12-26 16:22:38,389][105620] Updated weights for policy 1, policy_version 139492 (0.0009) [2023-12-26 16:22:39,049][105692] Updated weights for policy 0, policy_version 138703 (0.0009) [2023-12-26 16:22:39,103][105692] Updated weights for policy 0, policy_version 138713 (0.0010) [2023-12-26 16:22:39,137][105620] Updated weights for policy 1, policy_version 139502 (0.0009) [2023-12-26 16:22:39,163][105692] Updated weights for policy 0, policy_version 138723 (0.0010) [2023-12-26 16:22:39,184][105620] Updated weights for policy 1, policy_version 139512 (0.0005) [2023-12-26 16:22:39,249][105620] Updated weights for policy 1, policy_version 139522 (0.0008) [2023-12-26 16:22:39,941][105692] Updated weights for policy 0, policy_version 138733 (0.0010) [2023-12-26 16:22:39,976][105620] Updated weights for policy 1, policy_version 139532 (0.0007) [2023-12-26 16:22:40,009][105692] Updated weights for policy 0, policy_version 138743 (0.0009) [2023-12-26 16:22:40,037][105620] Updated weights for policy 1, policy_version 139542 (0.0009) [2023-12-26 16:22:40,075][105692] Updated weights for policy 0, policy_version 138753 (0.0009) [2023-12-26 16:22:40,099][105620] Updated weights for policy 1, policy_version 139552 (0.0008) [2023-12-26 16:22:40,778][105692] Updated weights for policy 0, policy_version 138763 (0.0009) [2023-12-26 16:22:40,836][105692] Updated weights for policy 0, policy_version 138773 (0.0005) [2023-12-26 16:22:40,894][105692] Updated weights for policy 0, policy_version 138783 (0.0005) [2023-12-26 16:22:40,919][105620] Updated weights for policy 1, policy_version 139562 (0.0008) [2023-12-26 16:22:40,973][105620] Updated weights for policy 1, policy_version 139572 (0.0009) [2023-12-26 16:22:41,034][105620] Updated weights for policy 1, policy_version 139582 (0.0008) [2023-12-26 16:22:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 71270400. Throughput: 0: 9741.4, 1: 9993.3. Samples: 71281300. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-26 16:22:41,062][104569] Avg episode reward: [(0, '2282.219'), (1, '8288.638')] [2023-12-26 16:22:41,098][105620] Updated weights for policy 1, policy_version 139592 (0.0007) [2023-12-26 16:22:41,630][105692] Updated weights for policy 0, policy_version 138793 (0.0006) [2023-12-26 16:22:41,697][105692] Updated weights for policy 0, policy_version 138803 (0.0011) [2023-12-26 16:22:41,768][105692] Updated weights for policy 0, policy_version 138813 (0.0011) [2023-12-26 16:22:41,833][105692] Updated weights for policy 0, policy_version 138823 (0.0010) [2023-12-26 16:22:41,893][105620] Updated weights for policy 1, policy_version 139602 (0.0006) [2023-12-26 16:22:41,939][105620] Updated weights for policy 1, policy_version 139612 (0.0007) [2023-12-26 16:22:41,986][105620] Updated weights for policy 1, policy_version 139622 (0.0008) [2023-12-26 16:22:42,564][105692] Updated weights for policy 0, policy_version 138833 (0.0010) [2023-12-26 16:22:42,616][105692] Updated weights for policy 0, policy_version 138843 (0.0010) [2023-12-26 16:22:42,676][105692] Updated weights for policy 0, policy_version 138853 (0.0010) [2023-12-26 16:22:42,778][105620] Updated weights for policy 1, policy_version 139632 (0.0008) [2023-12-26 16:22:42,826][105620] Updated weights for policy 1, policy_version 139642 (0.0008) [2023-12-26 16:22:42,870][105620] Updated weights for policy 1, policy_version 139652 (0.0008) [2023-12-26 16:22:43,362][105692] Updated weights for policy 0, policy_version 138863 (0.0008) [2023-12-26 16:22:43,413][105692] Updated weights for policy 0, policy_version 138873 (0.0008) [2023-12-26 16:22:43,471][105692] Updated weights for policy 0, policy_version 138883 (0.0009) [2023-12-26 16:22:43,690][105620] Updated weights for policy 1, policy_version 139662 (0.0008) [2023-12-26 16:22:43,742][105620] Updated weights for policy 1, policy_version 139672 (0.0009) [2023-12-26 16:22:43,799][105620] Updated weights for policy 1, policy_version 139682 (0.0008) [2023-12-26 16:22:44,138][105692] Updated weights for policy 0, policy_version 138893 (0.0007) [2023-12-26 16:22:44,191][105692] Updated weights for policy 0, policy_version 138903 (0.0005) [2023-12-26 16:22:44,243][105692] Updated weights for policy 0, policy_version 138913 (0.0007) [2023-12-26 16:22:44,615][105620] Updated weights for policy 1, policy_version 139692 (0.0009) [2023-12-26 16:22:44,681][105620] Updated weights for policy 1, policy_version 139702 (0.0009) [2023-12-26 16:22:44,747][105620] Updated weights for policy 1, policy_version 139712 (0.0010) [2023-12-26 16:22:44,924][105692] Updated weights for policy 0, policy_version 138923 (0.0009) [2023-12-26 16:22:44,986][105692] Updated weights for policy 0, policy_version 138933 (0.0009) [2023-12-26 16:22:45,041][105692] Updated weights for policy 0, policy_version 138943 (0.0009) [2023-12-26 16:22:45,558][105620] Updated weights for policy 1, policy_version 139722 (0.0008) [2023-12-26 16:22:45,607][105620] Updated weights for policy 1, policy_version 139732 (0.0005) [2023-12-26 16:22:45,653][105620] Updated weights for policy 1, policy_version 139742 (0.0005) [2023-12-26 16:22:45,680][105692] Updated weights for policy 0, policy_version 138953 (0.0007) [2023-12-26 16:22:45,712][105620] Updated weights for policy 1, policy_version 139752 (0.0006) [2023-12-26 16:22:45,741][105692] Updated weights for policy 0, policy_version 138963 (0.0005) [2023-12-26 16:22:45,804][105692] Updated weights for policy 0, policy_version 138973 (0.0005) [2023-12-26 16:22:45,858][105692] Updated weights for policy 0, policy_version 138983 (0.0005) [2023-12-26 16:22:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 71368704. Throughput: 0: 9759.8, 1: 9930.0. Samples: 71336916. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-26 16:22:46,062][104569] Avg episode reward: [(0, '4427.040'), (1, '8193.913')] [2023-12-26 16:22:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000138984_35586048.pth... [2023-12-26 16:22:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000139752_35782656.pth... [2023-12-26 16:22:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000137832_35291136.pth [2023-12-26 16:22:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000138600_35487744.pth [2023-12-26 16:22:46,389][105692] Updated weights for policy 0, policy_version 138993 (0.0005) [2023-12-26 16:22:46,449][105692] Updated weights for policy 0, policy_version 139003 (0.0006) [2023-12-26 16:22:46,491][105620] Updated weights for policy 1, policy_version 139762 (0.0009) [2023-12-26 16:22:46,506][105692] Updated weights for policy 0, policy_version 139013 (0.0005) [2023-12-26 16:22:46,538][105620] Updated weights for policy 1, policy_version 139772 (0.0009) [2023-12-26 16:22:46,593][105620] Updated weights for policy 1, policy_version 139782 (0.0009) [2023-12-26 16:22:47,129][105692] Updated weights for policy 0, policy_version 139023 (0.0008) [2023-12-26 16:22:47,183][105692] Updated weights for policy 0, policy_version 139033 (0.0009) [2023-12-26 16:22:47,243][105692] Updated weights for policy 0, policy_version 139043 (0.0007) [2023-12-26 16:22:47,398][105620] Updated weights for policy 1, policy_version 139792 (0.0010) [2023-12-26 16:22:47,443][105620] Updated weights for policy 1, policy_version 139802 (0.0008) [2023-12-26 16:22:47,499][105620] Updated weights for policy 1, policy_version 139812 (0.0008) [2023-12-26 16:22:47,894][105692] Updated weights for policy 0, policy_version 139053 (0.0007) [2023-12-26 16:22:47,948][105692] Updated weights for policy 0, policy_version 139063 (0.0009) [2023-12-26 16:22:48,018][105692] Updated weights for policy 0, policy_version 139073 (0.0006) [2023-12-26 16:22:48,294][105620] Updated weights for policy 1, policy_version 139822 (0.0009) [2023-12-26 16:22:48,360][105620] Updated weights for policy 1, policy_version 139832 (0.0009) [2023-12-26 16:22:48,424][105620] Updated weights for policy 1, policy_version 139842 (0.0009) [2023-12-26 16:22:48,729][105692] Updated weights for policy 0, policy_version 139083 (0.0009) [2023-12-26 16:22:48,781][105692] Updated weights for policy 0, policy_version 139093 (0.0009) [2023-12-26 16:22:48,838][105692] Updated weights for policy 0, policy_version 139103 (0.0010) [2023-12-26 16:22:49,180][105620] Updated weights for policy 1, policy_version 139852 (0.0009) [2023-12-26 16:22:49,240][105620] Updated weights for policy 1, policy_version 139862 (0.0009) [2023-12-26 16:22:49,298][105620] Updated weights for policy 1, policy_version 139872 (0.0009) [2023-12-26 16:22:49,569][105692] Updated weights for policy 0, policy_version 139113 (0.0008) [2023-12-26 16:22:49,625][105692] Updated weights for policy 0, policy_version 139123 (0.0010) [2023-12-26 16:22:49,677][105692] Updated weights for policy 0, policy_version 139133 (0.0010) [2023-12-26 16:22:49,723][105692] Updated weights for policy 0, policy_version 139143 (0.0008) [2023-12-26 16:22:50,164][105620] Updated weights for policy 1, policy_version 139882 (0.0009) [2023-12-26 16:22:50,220][105620] Updated weights for policy 1, policy_version 139892 (0.0008) [2023-12-26 16:22:50,271][105620] Updated weights for policy 1, policy_version 139902 (0.0007) [2023-12-26 16:22:50,326][105620] Updated weights for policy 1, policy_version 139912 (0.0008) [2023-12-26 16:22:50,378][105692] Updated weights for policy 0, policy_version 139153 (0.0010) [2023-12-26 16:22:50,424][105692] Updated weights for policy 0, policy_version 139163 (0.0007) [2023-12-26 16:22:50,468][105692] Updated weights for policy 0, policy_version 139173 (0.0005) [2023-12-26 16:22:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19438.7). Total num frames: 71458816. Throughput: 0: 9861.6, 1: 9794.6. Samples: 71453580. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-26 16:22:51,062][104569] Avg episode reward: [(0, '2734.782'), (1, '8899.396')] [2023-12-26 16:22:51,094][105620] Updated weights for policy 1, policy_version 139922 (0.0008) [2023-12-26 16:22:51,144][105620] Updated weights for policy 1, policy_version 139932 (0.0008) [2023-12-26 16:22:51,205][105620] Updated weights for policy 1, policy_version 139942 (0.0008) [2023-12-26 16:22:51,244][105692] Updated weights for policy 0, policy_version 139183 (0.0009) [2023-12-26 16:22:51,304][105692] Updated weights for policy 0, policy_version 139193 (0.0011) [2023-12-26 16:22:51,367][105692] Updated weights for policy 0, policy_version 139203 (0.0010) [2023-12-26 16:22:52,011][105620] Updated weights for policy 1, policy_version 139952 (0.0008) [2023-12-26 16:22:52,079][105620] Updated weights for policy 1, policy_version 139962 (0.0010) [2023-12-26 16:22:52,083][105692] Updated weights for policy 0, policy_version 139213 (0.0008) [2023-12-26 16:22:52,134][105692] Updated weights for policy 0, policy_version 139223 (0.0006) [2023-12-26 16:22:52,143][105620] Updated weights for policy 1, policy_version 139972 (0.0009) [2023-12-26 16:22:52,182][105692] Updated weights for policy 0, policy_version 139233 (0.0007) [2023-12-26 16:22:52,764][105692] Updated weights for policy 0, policy_version 139243 (0.0006) [2023-12-26 16:22:52,822][105692] Updated weights for policy 0, policy_version 139253 (0.0009) [2023-12-26 16:22:52,882][105692] Updated weights for policy 0, policy_version 139263 (0.0008) [2023-12-26 16:22:52,985][105620] Updated weights for policy 1, policy_version 139982 (0.0009) [2023-12-26 16:22:53,052][105620] Updated weights for policy 1, policy_version 139992 (0.0009) [2023-12-26 16:22:53,113][105620] Updated weights for policy 1, policy_version 140002 (0.0009) [2023-12-26 16:22:53,484][105692] Updated weights for policy 0, policy_version 139273 (0.0006) [2023-12-26 16:22:53,537][105692] Updated weights for policy 0, policy_version 139283 (0.0009) [2023-12-26 16:22:53,598][105692] Updated weights for policy 0, policy_version 139293 (0.0008) [2023-12-26 16:22:53,656][105692] Updated weights for policy 0, policy_version 139303 (0.0009) [2023-12-26 16:22:53,914][105620] Updated weights for policy 1, policy_version 140012 (0.0009) [2023-12-26 16:22:53,968][105620] Updated weights for policy 1, policy_version 140022 (0.0008) [2023-12-26 16:22:54,020][105620] Updated weights for policy 1, policy_version 140032 (0.0008) [2023-12-26 16:22:54,375][105692] Updated weights for policy 0, policy_version 139313 (0.0009) [2023-12-26 16:22:54,433][105692] Updated weights for policy 0, policy_version 139323 (0.0009) [2023-12-26 16:22:54,484][105692] Updated weights for policy 0, policy_version 139333 (0.0010) [2023-12-26 16:22:54,690][105620] Updated weights for policy 1, policy_version 140042 (0.0008) [2023-12-26 16:22:54,750][105620] Updated weights for policy 1, policy_version 140052 (0.0005) [2023-12-26 16:22:54,806][105620] Updated weights for policy 1, policy_version 140062 (0.0005) [2023-12-26 16:22:54,852][105620] Updated weights for policy 1, policy_version 140072 (0.0005) [2023-12-26 16:22:55,224][105692] Updated weights for policy 0, policy_version 139343 (0.0010) [2023-12-26 16:22:55,275][105692] Updated weights for policy 0, policy_version 139353 (0.0010) [2023-12-26 16:22:55,326][105692] Updated weights for policy 0, policy_version 139363 (0.0010) [2023-12-26 16:22:55,494][105620] Updated weights for policy 1, policy_version 140082 (0.0005) [2023-12-26 16:22:55,562][105620] Updated weights for policy 1, policy_version 140092 (0.0005) [2023-12-26 16:22:55,625][105620] Updated weights for policy 1, policy_version 140102 (0.0006) [2023-12-26 16:22:56,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 71557120. Throughput: 0: 9936.1, 1: 9771.5. Samples: 71570756. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-26 16:22:56,063][104569] Avg episode reward: [(0, '6841.007'), (1, '8985.153')] [2023-12-26 16:22:56,096][105692] Updated weights for policy 0, policy_version 139373 (0.0009) [2023-12-26 16:22:56,145][105692] Updated weights for policy 0, policy_version 139383 (0.0006) [2023-12-26 16:22:56,199][105692] Updated weights for policy 0, policy_version 139393 (0.0008) [2023-12-26 16:22:56,217][105620] Updated weights for policy 1, policy_version 140112 (0.0010) [2023-12-26 16:22:56,272][105620] Updated weights for policy 1, policy_version 140122 (0.0010) [2023-12-26 16:22:56,330][105620] Updated weights for policy 1, policy_version 140132 (0.0010) [2023-12-26 16:22:56,878][105692] Updated weights for policy 0, policy_version 139403 (0.0005) [2023-12-26 16:22:56,932][105692] Updated weights for policy 0, policy_version 139413 (0.0005) [2023-12-26 16:22:56,980][105620] Updated weights for policy 1, policy_version 140142 (0.0010) [2023-12-26 16:22:56,983][105692] Updated weights for policy 0, policy_version 139423 (0.0005) [2023-12-26 16:22:57,052][105620] Updated weights for policy 1, policy_version 140152 (0.0010) [2023-12-26 16:22:57,113][105620] Updated weights for policy 1, policy_version 140162 (0.0010) [2023-12-26 16:22:57,610][105692] Updated weights for policy 0, policy_version 139433 (0.0006) [2023-12-26 16:22:57,668][105692] Updated weights for policy 0, policy_version 139443 (0.0008) [2023-12-26 16:22:57,724][105692] Updated weights for policy 0, policy_version 139453 (0.0008) [2023-12-26 16:22:57,791][105692] Updated weights for policy 0, policy_version 139463 (0.0010) [2023-12-26 16:22:57,815][105620] Updated weights for policy 1, policy_version 140172 (0.0008) [2023-12-26 16:22:57,878][105620] Updated weights for policy 1, policy_version 140182 (0.0005) [2023-12-26 16:22:57,949][105620] Updated weights for policy 1, policy_version 140192 (0.0007) [2023-12-26 16:22:58,503][105692] Updated weights for policy 0, policy_version 139473 (0.0009) [2023-12-26 16:22:58,569][105692] Updated weights for policy 0, policy_version 139483 (0.0008) [2023-12-26 16:22:58,632][105692] Updated weights for policy 0, policy_version 139493 (0.0009) [2023-12-26 16:22:58,649][105620] Updated weights for policy 1, policy_version 140202 (0.0009) [2023-12-26 16:22:58,726][105620] Updated weights for policy 1, policy_version 140212 (0.0007) [2023-12-26 16:22:58,807][105620] Updated weights for policy 1, policy_version 140222 (0.0008) [2023-12-26 16:22:58,873][105620] Updated weights for policy 1, policy_version 140232 (0.0009) [2023-12-26 16:22:59,432][105692] Updated weights for policy 0, policy_version 139503 (0.0006) [2023-12-26 16:22:59,492][105692] Updated weights for policy 0, policy_version 139513 (0.0008) [2023-12-26 16:22:59,536][105692] Updated weights for policy 0, policy_version 139523 (0.0010) [2023-12-26 16:22:59,675][105620] Updated weights for policy 1, policy_version 140242 (0.0005) [2023-12-26 16:22:59,732][105620] Updated weights for policy 1, policy_version 140252 (0.0005) [2023-12-26 16:22:59,789][105620] Updated weights for policy 1, policy_version 140262 (0.0005) [2023-12-26 16:23:00,282][105692] Updated weights for policy 0, policy_version 139533 (0.0010) [2023-12-26 16:23:00,338][105692] Updated weights for policy 0, policy_version 139543 (0.0011) [2023-12-26 16:23:00,401][105692] Updated weights for policy 0, policy_version 139553 (0.0008) [2023-12-26 16:23:00,418][105620] Updated weights for policy 1, policy_version 140272 (0.0010) [2023-12-26 16:23:00,470][105620] Updated weights for policy 1, policy_version 140283 (0.0009) [2023-12-26 16:23:00,523][105620] Updated weights for policy 1, policy_version 140293 (0.0009) [2023-12-26 16:23:00,978][105692] Updated weights for policy 0, policy_version 139563 (0.0005) [2023-12-26 16:23:01,051][105692] Updated weights for policy 0, policy_version 139573 (0.0006) [2023-12-26 16:23:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 71655424. Throughput: 0: 9957.8, 1: 9746.5. Samples: 71630596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:23:01,062][104569] Avg episode reward: [(0, '8992.681'), (1, '8453.747')] [2023-12-26 16:23:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000140296_35921920.pth... [2023-12-26 16:23:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000139208_35643392.pth [2023-12-26 16:23:01,112][105692] Updated weights for policy 0, policy_version 139583 (0.0007) [2023-12-26 16:23:01,170][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000139592_35741696.pth... [2023-12-26 16:23:01,172][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000138376_35430400.pth [2023-12-26 16:23:01,370][105620] Updated weights for policy 1, policy_version 140303 (0.0008) [2023-12-26 16:23:01,430][105620] Updated weights for policy 1, policy_version 140313 (0.0006) [2023-12-26 16:23:01,486][105620] Updated weights for policy 1, policy_version 140323 (0.0005) [2023-12-26 16:23:01,770][105692] Updated weights for policy 0, policy_version 139593 (0.0010) [2023-12-26 16:23:01,821][105692] Updated weights for policy 0, policy_version 139603 (0.0010) [2023-12-26 16:23:01,869][105692] Updated weights for policy 0, policy_version 139613 (0.0010) [2023-12-26 16:23:01,923][105692] Updated weights for policy 0, policy_version 139623 (0.0010) [2023-12-26 16:23:02,217][105620] Updated weights for policy 1, policy_version 140333 (0.0010) [2023-12-26 16:23:02,280][105620] Updated weights for policy 1, policy_version 140343 (0.0010) [2023-12-26 16:23:02,346][105620] Updated weights for policy 1, policy_version 140353 (0.0011) [2023-12-26 16:23:02,700][105692] Updated weights for policy 0, policy_version 139633 (0.0010) [2023-12-26 16:23:02,764][105692] Updated weights for policy 0, policy_version 139643 (0.0010) [2023-12-26 16:23:02,819][105692] Updated weights for policy 0, policy_version 139653 (0.0010) [2023-12-26 16:23:02,923][105620] Updated weights for policy 1, policy_version 140363 (0.0010) [2023-12-26 16:23:02,982][105620] Updated weights for policy 1, policy_version 140373 (0.0007) [2023-12-26 16:23:03,033][105620] Updated weights for policy 1, policy_version 140383 (0.0007) [2023-12-26 16:23:03,518][105692] Updated weights for policy 0, policy_version 139663 (0.0006) [2023-12-26 16:23:03,578][105692] Updated weights for policy 0, policy_version 139673 (0.0005) [2023-12-26 16:23:03,626][105692] Updated weights for policy 0, policy_version 139683 (0.0005) [2023-12-26 16:23:03,753][105620] Updated weights for policy 1, policy_version 140393 (0.0010) [2023-12-26 16:23:03,811][105620] Updated weights for policy 1, policy_version 140403 (0.0010) [2023-12-26 16:23:03,870][105620] Updated weights for policy 1, policy_version 140413 (0.0011) [2023-12-26 16:23:03,921][105620] Updated weights for policy 1, policy_version 140423 (0.0010) [2023-12-26 16:23:04,217][105692] Updated weights for policy 0, policy_version 139693 (0.0008) [2023-12-26 16:23:04,283][105692] Updated weights for policy 0, policy_version 139703 (0.0009) [2023-12-26 16:23:04,345][105692] Updated weights for policy 0, policy_version 139713 (0.0010) [2023-12-26 16:23:04,702][105620] Updated weights for policy 1, policy_version 140433 (0.0010) [2023-12-26 16:23:04,753][105620] Updated weights for policy 1, policy_version 140443 (0.0010) [2023-12-26 16:23:04,805][105620] Updated weights for policy 1, policy_version 140453 (0.0007) [2023-12-26 16:23:05,027][105692] Updated weights for policy 0, policy_version 139723 (0.0010) [2023-12-26 16:23:05,082][105692] Updated weights for policy 0, policy_version 139733 (0.0010) [2023-12-26 16:23:05,141][105692] Updated weights for policy 0, policy_version 139743 (0.0011) [2023-12-26 16:23:05,358][105620] Updated weights for policy 1, policy_version 140463 (0.0009) [2023-12-26 16:23:05,402][105620] Updated weights for policy 1, policy_version 140473 (0.0010) [2023-12-26 16:23:05,449][105620] Updated weights for policy 1, policy_version 140483 (0.0010) [2023-12-26 16:23:05,854][105692] Updated weights for policy 0, policy_version 139753 (0.0010) [2023-12-26 16:23:05,901][105692] Updated weights for policy 0, policy_version 139763 (0.0007) [2023-12-26 16:23:05,951][105692] Updated weights for policy 0, policy_version 139773 (0.0005) [2023-12-26 16:23:06,004][105692] Updated weights for policy 0, policy_version 139783 (0.0010) [2023-12-26 16:23:06,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 71761920. Throughput: 0: 9920.6, 1: 9634.4. Samples: 71748544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:23:06,062][104569] Avg episode reward: [(0, '9079.504'), (1, '8458.440')] [2023-12-26 16:23:06,195][105620] Updated weights for policy 1, policy_version 140493 (0.0009) [2023-12-26 16:23:06,263][105620] Updated weights for policy 1, policy_version 140503 (0.0007) [2023-12-26 16:23:06,330][105620] Updated weights for policy 1, policy_version 140513 (0.0011) [2023-12-26 16:23:06,758][105692] Updated weights for policy 0, policy_version 139793 (0.0011) [2023-12-26 16:23:06,820][105692] Updated weights for policy 0, policy_version 139803 (0.0010) [2023-12-26 16:23:06,884][105692] Updated weights for policy 0, policy_version 139813 (0.0011) [2023-12-26 16:23:07,064][105620] Updated weights for policy 1, policy_version 140523 (0.0011) [2023-12-26 16:23:07,128][105620] Updated weights for policy 1, policy_version 140533 (0.0011) [2023-12-26 16:23:07,191][105620] Updated weights for policy 1, policy_version 140543 (0.0011) [2023-12-26 16:23:07,470][105692] Updated weights for policy 0, policy_version 139823 (0.0009) [2023-12-26 16:23:07,526][105692] Updated weights for policy 0, policy_version 139833 (0.0011) [2023-12-26 16:23:07,583][105692] Updated weights for policy 0, policy_version 139843 (0.0010) [2023-12-26 16:23:07,913][105620] Updated weights for policy 1, policy_version 140553 (0.0010) [2023-12-26 16:23:07,961][105620] Updated weights for policy 1, policy_version 140563 (0.0010) [2023-12-26 16:23:08,006][105620] Updated weights for policy 1, policy_version 140573 (0.0010) [2023-12-26 16:23:08,071][105620] Updated weights for policy 1, policy_version 140583 (0.0010) [2023-12-26 16:23:08,201][105692] Updated weights for policy 0, policy_version 139853 (0.0009) [2023-12-26 16:23:08,266][105692] Updated weights for policy 0, policy_version 139863 (0.0005) [2023-12-26 16:23:08,325][105692] Updated weights for policy 0, policy_version 139873 (0.0007) [2023-12-26 16:23:08,829][105620] Updated weights for policy 1, policy_version 140593 (0.0011) [2023-12-26 16:23:08,892][105620] Updated weights for policy 1, policy_version 140603 (0.0011) [2023-12-26 16:23:08,955][105620] Updated weights for policy 1, policy_version 140613 (0.0011) [2023-12-26 16:23:09,019][105692] Updated weights for policy 0, policy_version 139883 (0.0010) [2023-12-26 16:23:09,077][105692] Updated weights for policy 0, policy_version 139893 (0.0010) [2023-12-26 16:23:09,136][105692] Updated weights for policy 0, policy_version 139903 (0.0011) [2023-12-26 16:23:09,678][105620] Updated weights for policy 1, policy_version 140623 (0.0009) [2023-12-26 16:23:09,746][105620] Updated weights for policy 1, policy_version 140633 (0.0009) [2023-12-26 16:23:09,803][105620] Updated weights for policy 1, policy_version 140643 (0.0005) [2023-12-26 16:23:09,928][105692] Updated weights for policy 0, policy_version 139913 (0.0011) [2023-12-26 16:23:09,986][105692] Updated weights for policy 0, policy_version 139923 (0.0009) [2023-12-26 16:23:10,049][105692] Updated weights for policy 0, policy_version 139933 (0.0007) [2023-12-26 16:23:10,118][105692] Updated weights for policy 0, policy_version 139943 (0.0008) [2023-12-26 16:23:10,611][105620] Updated weights for policy 1, policy_version 140653 (0.0009) [2023-12-26 16:23:10,668][105620] Updated weights for policy 1, policy_version 140663 (0.0007) [2023-12-26 16:23:10,726][105620] Updated weights for policy 1, policy_version 140673 (0.0005) [2023-12-26 16:23:10,793][105692] Updated weights for policy 0, policy_version 139953 (0.0009) [2023-12-26 16:23:10,852][105692] Updated weights for policy 0, policy_version 139963 (0.0009) [2023-12-26 16:23:10,912][105692] Updated weights for policy 0, policy_version 139973 (0.0009) [2023-12-26 16:23:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 71860224. Throughput: 0: 10041.2, 1: 9530.6. Samples: 71866512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:23:11,062][104569] Avg episode reward: [(0, '8897.053'), (1, '8714.433')] [2023-12-26 16:23:11,500][105620] Updated weights for policy 1, policy_version 140683 (0.0007) [2023-12-26 16:23:11,554][105620] Updated weights for policy 1, policy_version 140693 (0.0009) [2023-12-26 16:23:11,608][105620] Updated weights for policy 1, policy_version 140703 (0.0009) [2023-12-26 16:23:11,754][105692] Updated weights for policy 0, policy_version 139983 (0.0008) [2023-12-26 16:23:11,819][105692] Updated weights for policy 0, policy_version 139993 (0.0006) [2023-12-26 16:23:11,876][105692] Updated weights for policy 0, policy_version 140003 (0.0006) [2023-12-26 16:23:12,475][105620] Updated weights for policy 1, policy_version 140713 (0.0007) [2023-12-26 16:23:12,503][105692] Updated weights for policy 0, policy_version 140013 (0.0006) [2023-12-26 16:23:12,522][105620] Updated weights for policy 1, policy_version 140723 (0.0008) [2023-12-26 16:23:12,552][105692] Updated weights for policy 0, policy_version 140023 (0.0007) [2023-12-26 16:23:12,571][105620] Updated weights for policy 1, policy_version 140733 (0.0006) [2023-12-26 16:23:12,602][105692] Updated weights for policy 0, policy_version 140033 (0.0006) [2023-12-26 16:23:12,621][105620] Updated weights for policy 1, policy_version 140743 (0.0006) [2023-12-26 16:23:13,320][105620] Updated weights for policy 1, policy_version 140753 (0.0009) [2023-12-26 16:23:13,360][105692] Updated weights for policy 0, policy_version 140043 (0.0006) [2023-12-26 16:23:13,371][105620] Updated weights for policy 1, policy_version 140764 (0.0009) [2023-12-26 16:23:13,414][105692] Updated weights for policy 0, policy_version 140053 (0.0008) [2023-12-26 16:23:13,423][105620] Updated weights for policy 1, policy_version 140774 (0.0005) [2023-12-26 16:23:13,467][105692] Updated weights for policy 0, policy_version 140063 (0.0009) [2023-12-26 16:23:14,028][105620] Updated weights for policy 1, policy_version 140784 (0.0008) [2023-12-26 16:23:14,077][105620] Updated weights for policy 1, policy_version 140794 (0.0009) [2023-12-26 16:23:14,126][105692] Updated weights for policy 0, policy_version 140074 (0.0009) [2023-12-26 16:23:14,144][105620] Updated weights for policy 1, policy_version 140804 (0.0008) [2023-12-26 16:23:14,177][105692] Updated weights for policy 0, policy_version 140084 (0.0005) [2023-12-26 16:23:14,222][105692] Updated weights for policy 0, policy_version 140094 (0.0005) [2023-12-26 16:23:14,270][105692] Updated weights for policy 0, policy_version 140104 (0.0005) [2023-12-26 16:23:14,858][105692] Updated weights for policy 0, policy_version 140114 (0.0007) [2023-12-26 16:23:14,868][105620] Updated weights for policy 1, policy_version 140814 (0.0010) [2023-12-26 16:23:14,923][105692] Updated weights for policy 0, policy_version 140124 (0.0006) [2023-12-26 16:23:14,929][105620] Updated weights for policy 1, policy_version 140824 (0.0011) [2023-12-26 16:23:14,984][105692] Updated weights for policy 0, policy_version 140134 (0.0006) [2023-12-26 16:23:14,990][105620] Updated weights for policy 1, policy_version 140834 (0.0011) [2023-12-26 16:23:15,686][105620] Updated weights for policy 1, policy_version 140844 (0.0008) [2023-12-26 16:23:15,688][105692] Updated weights for policy 0, policy_version 140144 (0.0008) [2023-12-26 16:23:15,744][105620] Updated weights for policy 1, policy_version 140854 (0.0006) [2023-12-26 16:23:15,751][105692] Updated weights for policy 0, policy_version 140154 (0.0007) [2023-12-26 16:23:15,801][105620] Updated weights for policy 1, policy_version 140864 (0.0007) [2023-12-26 16:23:15,808][105692] Updated weights for policy 0, policy_version 140164 (0.0006) [2023-12-26 16:23:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 71958528. Throughput: 0: 10076.7, 1: 9456.9. Samples: 71923916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:23:16,063][104569] Avg episode reward: [(0, '8717.194'), (1, '8888.055')] [2023-12-26 16:23:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000140168_35889152.pth... [2023-12-26 16:23:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000140872_36069376.pth... [2023-12-26 16:23:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000138984_35586048.pth [2023-12-26 16:23:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000139752_35782656.pth [2023-12-26 16:23:16,407][105620] Updated weights for policy 1, policy_version 140874 (0.0007) [2023-12-26 16:23:16,463][105620] Updated weights for policy 1, policy_version 140884 (0.0005) [2023-12-26 16:23:16,503][105692] Updated weights for policy 0, policy_version 140174 (0.0009) [2023-12-26 16:23:16,510][105620] Updated weights for policy 1, policy_version 140894 (0.0005) [2023-12-26 16:23:16,555][105620] Updated weights for policy 1, policy_version 140904 (0.0005) [2023-12-26 16:23:16,559][105692] Updated weights for policy 0, policy_version 140184 (0.0010) [2023-12-26 16:23:16,610][105692] Updated weights for policy 0, policy_version 140194 (0.0009) [2023-12-26 16:23:17,229][105692] Updated weights for policy 0, policy_version 140204 (0.0007) [2023-12-26 16:23:17,286][105692] Updated weights for policy 0, policy_version 140214 (0.0007) [2023-12-26 16:23:17,293][105620] Updated weights for policy 1, policy_version 140914 (0.0008) [2023-12-26 16:23:17,342][105692] Updated weights for policy 0, policy_version 140224 (0.0007) [2023-12-26 16:23:17,345][105620] Updated weights for policy 1, policy_version 140924 (0.0006) [2023-12-26 16:23:17,396][105620] Updated weights for policy 1, policy_version 140934 (0.0008) [2023-12-26 16:23:17,946][105692] Updated weights for policy 0, policy_version 140234 (0.0009) [2023-12-26 16:23:17,996][105692] Updated weights for policy 0, policy_version 140244 (0.0006) [2023-12-26 16:23:18,003][105620] Updated weights for policy 1, policy_version 140944 (0.0006) [2023-12-26 16:23:18,050][105692] Updated weights for policy 0, policy_version 140254 (0.0009) [2023-12-26 16:23:18,061][105620] Updated weights for policy 1, policy_version 140954 (0.0006) [2023-12-26 16:23:18,104][105692] Updated weights for policy 0, policy_version 140264 (0.0007) [2023-12-26 16:23:18,114][105620] Updated weights for policy 1, policy_version 140964 (0.0007) [2023-12-26 16:23:18,832][105620] Updated weights for policy 1, policy_version 140974 (0.0008) [2023-12-26 16:23:18,862][105692] Updated weights for policy 0, policy_version 140274 (0.0007) [2023-12-26 16:23:18,891][105620] Updated weights for policy 1, policy_version 140984 (0.0008) [2023-12-26 16:23:18,931][105692] Updated weights for policy 0, policy_version 140284 (0.0006) [2023-12-26 16:23:18,957][105620] Updated weights for policy 1, policy_version 140994 (0.0008) [2023-12-26 16:23:18,987][105692] Updated weights for policy 0, policy_version 140294 (0.0007) [2023-12-26 16:23:19,688][105620] Updated weights for policy 1, policy_version 141004 (0.0007) [2023-12-26 16:23:19,706][105692] Updated weights for policy 0, policy_version 140304 (0.0008) [2023-12-26 16:23:19,750][105620] Updated weights for policy 1, policy_version 141014 (0.0008) [2023-12-26 16:23:19,767][105692] Updated weights for policy 0, policy_version 140314 (0.0010) [2023-12-26 16:23:19,806][105620] Updated weights for policy 1, policy_version 141024 (0.0009) [2023-12-26 16:23:19,826][105692] Updated weights for policy 0, policy_version 140324 (0.0010) [2023-12-26 16:23:20,592][105620] Updated weights for policy 1, policy_version 141034 (0.0008) [2023-12-26 16:23:20,594][105692] Updated weights for policy 0, policy_version 140334 (0.0009) [2023-12-26 16:23:20,652][105692] Updated weights for policy 0, policy_version 140344 (0.0008) [2023-12-26 16:23:20,661][105620] Updated weights for policy 1, policy_version 141044 (0.0007) [2023-12-26 16:23:20,709][105692] Updated weights for policy 0, policy_version 140354 (0.0006) [2023-12-26 16:23:20,725][105620] Updated weights for policy 1, policy_version 141054 (0.0008) [2023-12-26 16:23:20,790][105620] Updated weights for policy 1, policy_version 141064 (0.0009) [2023-12-26 16:23:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 72056832. Throughput: 0: 10090.0, 1: 9483.8. Samples: 72046552. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:23:21,062][104569] Avg episode reward: [(0, '9083.226'), (1, '4783.605')] [2023-12-26 16:23:21,465][105692] Updated weights for policy 0, policy_version 140364 (0.0009) [2023-12-26 16:23:21,495][105620] Updated weights for policy 1, policy_version 141074 (0.0008) [2023-12-26 16:23:21,523][105692] Updated weights for policy 0, policy_version 140374 (0.0005) [2023-12-26 16:23:21,558][105620] Updated weights for policy 1, policy_version 141084 (0.0008) [2023-12-26 16:23:21,584][105692] Updated weights for policy 0, policy_version 140384 (0.0006) [2023-12-26 16:23:21,622][105620] Updated weights for policy 1, policy_version 141094 (0.0008) [2023-12-26 16:23:22,218][105620] Updated weights for policy 1, policy_version 141104 (0.0006) [2023-12-26 16:23:22,276][105620] Updated weights for policy 1, policy_version 141114 (0.0006) [2023-12-26 16:23:22,340][105620] Updated weights for policy 1, policy_version 141124 (0.0008) [2023-12-26 16:23:22,419][105692] Updated weights for policy 0, policy_version 140394 (0.0008) [2023-12-26 16:23:22,475][105692] Updated weights for policy 0, policy_version 140404 (0.0010) [2023-12-26 16:23:22,528][105692] Updated weights for policy 0, policy_version 140414 (0.0010) [2023-12-26 16:23:22,591][105692] Updated weights for policy 0, policy_version 140424 (0.0006) [2023-12-26 16:23:23,075][105620] Updated weights for policy 1, policy_version 141134 (0.0009) [2023-12-26 16:23:23,130][105620] Updated weights for policy 1, policy_version 141144 (0.0006) [2023-12-26 16:23:23,186][105620] Updated weights for policy 1, policy_version 141154 (0.0005) [2023-12-26 16:23:23,297][105692] Updated weights for policy 0, policy_version 140434 (0.0006) [2023-12-26 16:23:23,341][105692] Updated weights for policy 0, policy_version 140444 (0.0010) [2023-12-26 16:23:23,389][105692] Updated weights for policy 0, policy_version 140454 (0.0010) [2023-12-26 16:23:23,714][105620] Updated weights for policy 1, policy_version 141164 (0.0005) [2023-12-26 16:23:23,766][105620] Updated weights for policy 1, policy_version 141174 (0.0005) [2023-12-26 16:23:23,826][105620] Updated weights for policy 1, policy_version 141184 (0.0005) [2023-12-26 16:23:24,088][105692] Updated weights for policy 0, policy_version 140464 (0.0006) [2023-12-26 16:23:24,145][105692] Updated weights for policy 0, policy_version 140474 (0.0007) [2023-12-26 16:23:24,195][105692] Updated weights for policy 0, policy_version 140484 (0.0006) [2023-12-26 16:23:24,556][105620] Updated weights for policy 1, policy_version 141194 (0.0006) [2023-12-26 16:23:24,621][105620] Updated weights for policy 1, policy_version 141204 (0.0009) [2023-12-26 16:23:24,684][105620] Updated weights for policy 1, policy_version 141214 (0.0008) [2023-12-26 16:23:24,712][105692] Updated weights for policy 0, policy_version 140494 (0.0008) [2023-12-26 16:23:24,731][105620] Updated weights for policy 1, policy_version 141224 (0.0007) [2023-12-26 16:23:24,756][105692] Updated weights for policy 0, policy_version 140504 (0.0007) [2023-12-26 16:23:24,810][105692] Updated weights for policy 0, policy_version 140514 (0.0005) [2023-12-26 16:23:25,362][105692] Updated weights for policy 0, policy_version 140524 (0.0005) [2023-12-26 16:23:25,420][105692] Updated weights for policy 0, policy_version 140534 (0.0005) [2023-12-26 16:23:25,481][105692] Updated weights for policy 0, policy_version 140544 (0.0005) [2023-12-26 16:23:25,610][105620] Updated weights for policy 1, policy_version 141234 (0.0009) [2023-12-26 16:23:25,689][105620] Updated weights for policy 1, policy_version 141244 (0.0010) [2023-12-26 16:23:25,748][105620] Updated weights for policy 1, policy_version 141254 (0.0010) [2023-12-26 16:23:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 72155136. Throughput: 0: 10121.2, 1: 9534.6. Samples: 72165808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:23:26,062][104569] Avg episode reward: [(0, '9264.209'), (1, '6843.618')] [2023-12-26 16:23:26,124][105692] Updated weights for policy 0, policy_version 140554 (0.0006) [2023-12-26 16:23:26,182][105692] Updated weights for policy 0, policy_version 140564 (0.0009) [2023-12-26 16:23:26,243][105692] Updated weights for policy 0, policy_version 140574 (0.0009) [2023-12-26 16:23:26,304][105692] Updated weights for policy 0, policy_version 140584 (0.0009) [2023-12-26 16:23:26,390][105620] Updated weights for policy 1, policy_version 141264 (0.0009) [2023-12-26 16:23:26,436][105620] Updated weights for policy 1, policy_version 141274 (0.0008) [2023-12-26 16:23:26,483][105620] Updated weights for policy 1, policy_version 141284 (0.0008) [2023-12-26 16:23:27,058][105692] Updated weights for policy 0, policy_version 140594 (0.0009) [2023-12-26 16:23:27,116][105692] Updated weights for policy 0, policy_version 140604 (0.0008) [2023-12-26 16:23:27,172][105692] Updated weights for policy 0, policy_version 140614 (0.0010) [2023-12-26 16:23:27,289][105620] Updated weights for policy 1, policy_version 141294 (0.0009) [2023-12-26 16:23:27,342][105620] Updated weights for policy 1, policy_version 141304 (0.0008) [2023-12-26 16:23:27,400][105620] Updated weights for policy 1, policy_version 141314 (0.0009) [2023-12-26 16:23:27,901][105692] Updated weights for policy 0, policy_version 140624 (0.0006) [2023-12-26 16:23:27,969][105692] Updated weights for policy 0, policy_version 140634 (0.0005) [2023-12-26 16:23:28,049][105692] Updated weights for policy 0, policy_version 140644 (0.0005) [2023-12-26 16:23:28,099][105620] Updated weights for policy 1, policy_version 141324 (0.0008) [2023-12-26 16:23:28,156][105620] Updated weights for policy 1, policy_version 141334 (0.0009) [2023-12-26 16:23:28,211][105620] Updated weights for policy 1, policy_version 141344 (0.0009) [2023-12-26 16:23:28,663][105692] Updated weights for policy 0, policy_version 140654 (0.0008) [2023-12-26 16:23:28,717][105692] Updated weights for policy 0, policy_version 140664 (0.0007) [2023-12-26 16:23:28,773][105692] Updated weights for policy 0, policy_version 140674 (0.0008) [2023-12-26 16:23:28,849][105620] Updated weights for policy 1, policy_version 141354 (0.0008) [2023-12-26 16:23:28,900][105620] Updated weights for policy 1, policy_version 141364 (0.0009) [2023-12-26 16:23:28,957][105620] Updated weights for policy 1, policy_version 141374 (0.0009) [2023-12-26 16:23:29,011][105620] Updated weights for policy 1, policy_version 141384 (0.0009) [2023-12-26 16:23:29,451][105692] Updated weights for policy 0, policy_version 140684 (0.0009) [2023-12-26 16:23:29,498][105692] Updated weights for policy 0, policy_version 140694 (0.0008) [2023-12-26 16:23:29,556][105692] Updated weights for policy 0, policy_version 140704 (0.0007) [2023-12-26 16:23:29,831][105620] Updated weights for policy 1, policy_version 141394 (0.0009) [2023-12-26 16:23:29,893][105620] Updated weights for policy 1, policy_version 141404 (0.0009) [2023-12-26 16:23:29,956][105620] Updated weights for policy 1, policy_version 141414 (0.0009) [2023-12-26 16:23:30,321][105692] Updated weights for policy 0, policy_version 140714 (0.0009) [2023-12-26 16:23:30,377][105692] Updated weights for policy 0, policy_version 140724 (0.0010) [2023-12-26 16:23:30,435][105692] Updated weights for policy 0, policy_version 140734 (0.0008) [2023-12-26 16:23:30,493][105692] Updated weights for policy 0, policy_version 140744 (0.0009) [2023-12-26 16:23:30,696][105620] Updated weights for policy 1, policy_version 141424 (0.0009) [2023-12-26 16:23:30,743][105620] Updated weights for policy 1, policy_version 141434 (0.0009) [2023-12-26 16:23:30,805][105620] Updated weights for policy 1, policy_version 141444 (0.0009) [2023-12-26 16:23:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 72253440. Throughput: 0: 10131.3, 1: 9611.1. Samples: 72225324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:23:31,062][104569] Avg episode reward: [(0, '9080.119'), (1, '8479.563')] [2023-12-26 16:23:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000140744_36036608.pth... [2023-12-26 16:23:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000141448_36216832.pth... [2023-12-26 16:23:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000140296_35921920.pth [2023-12-26 16:23:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000139592_35741696.pth [2023-12-26 16:23:31,262][105692] Updated weights for policy 0, policy_version 140754 (0.0008) [2023-12-26 16:23:31,322][105692] Updated weights for policy 0, policy_version 140764 (0.0009) [2023-12-26 16:23:31,390][105692] Updated weights for policy 0, policy_version 140774 (0.0010) [2023-12-26 16:23:31,585][105620] Updated weights for policy 1, policy_version 141454 (0.0009) [2023-12-26 16:23:31,648][105620] Updated weights for policy 1, policy_version 141464 (0.0010) [2023-12-26 16:23:31,707][105620] Updated weights for policy 1, policy_version 141474 (0.0009) [2023-12-26 16:23:32,045][105692] Updated weights for policy 0, policy_version 140784 (0.0008) [2023-12-26 16:23:32,102][105692] Updated weights for policy 0, policy_version 140794 (0.0009) [2023-12-26 16:23:32,155][105692] Updated weights for policy 0, policy_version 140804 (0.0009) [2023-12-26 16:23:32,464][105620] Updated weights for policy 1, policy_version 141484 (0.0009) [2023-12-26 16:23:32,526][105620] Updated weights for policy 1, policy_version 141494 (0.0010) [2023-12-26 16:23:32,579][105620] Updated weights for policy 1, policy_version 141504 (0.0009) [2023-12-26 16:23:32,818][105692] Updated weights for policy 0, policy_version 140815 (0.0010) [2023-12-26 16:23:32,868][105692] Updated weights for policy 0, policy_version 140825 (0.0008) [2023-12-26 16:23:32,927][105692] Updated weights for policy 0, policy_version 140835 (0.0007) [2023-12-26 16:23:33,380][105620] Updated weights for policy 1, policy_version 141514 (0.0009) [2023-12-26 16:23:33,432][105620] Updated weights for policy 1, policy_version 141524 (0.0008) [2023-12-26 16:23:33,486][105620] Updated weights for policy 1, policy_version 141534 (0.0009) [2023-12-26 16:23:33,532][105620] Updated weights for policy 1, policy_version 141544 (0.0009) [2023-12-26 16:23:33,656][105692] Updated weights for policy 0, policy_version 140845 (0.0008) [2023-12-26 16:23:33,715][105692] Updated weights for policy 0, policy_version 140855 (0.0005) [2023-12-26 16:23:33,776][105692] Updated weights for policy 0, policy_version 140865 (0.0008) [2023-12-26 16:23:34,344][105620] Updated weights for policy 1, policy_version 141554 (0.0009) [2023-12-26 16:23:34,403][105692] Updated weights for policy 0, policy_version 140875 (0.0008) [2023-12-26 16:23:34,404][105620] Updated weights for policy 1, policy_version 141564 (0.0009) [2023-12-26 16:23:34,458][105692] Updated weights for policy 0, policy_version 140885 (0.0007) [2023-12-26 16:23:34,468][105620] Updated weights for policy 1, policy_version 141574 (0.0008) [2023-12-26 16:23:34,517][105692] Updated weights for policy 0, policy_version 140895 (0.0007) [2023-12-26 16:23:35,229][105620] Updated weights for policy 1, policy_version 141584 (0.0008) [2023-12-26 16:23:35,246][105692] Updated weights for policy 0, policy_version 140905 (0.0009) [2023-12-26 16:23:35,280][105620] Updated weights for policy 1, policy_version 141594 (0.0006) [2023-12-26 16:23:35,306][105692] Updated weights for policy 0, policy_version 140915 (0.0007) [2023-12-26 16:23:35,344][105620] Updated weights for policy 1, policy_version 141604 (0.0007) [2023-12-26 16:23:35,362][105692] Updated weights for policy 0, policy_version 140925 (0.0006) [2023-12-26 16:23:35,422][105692] Updated weights for policy 0, policy_version 140935 (0.0009) [2023-12-26 16:23:36,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 72343552. Throughput: 0: 10043.2, 1: 9633.1. Samples: 72339016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:23:36,062][104569] Avg episode reward: [(0, '8988.383'), (1, '8992.757')] [2023-12-26 16:23:36,089][105620] Updated weights for policy 1, policy_version 141614 (0.0008) [2023-12-26 16:23:36,157][105620] Updated weights for policy 1, policy_version 141624 (0.0009) [2023-12-26 16:23:36,167][105692] Updated weights for policy 0, policy_version 140945 (0.0007) [2023-12-26 16:23:36,208][105620] Updated weights for policy 1, policy_version 141634 (0.0009) [2023-12-26 16:23:36,225][105692] Updated weights for policy 0, policy_version 140955 (0.0008) [2023-12-26 16:23:36,294][105692] Updated weights for policy 0, policy_version 140965 (0.0007) [2023-12-26 16:23:36,963][105620] Updated weights for policy 1, policy_version 141644 (0.0009) [2023-12-26 16:23:37,017][105692] Updated weights for policy 0, policy_version 140975 (0.0007) [2023-12-26 16:23:37,019][105620] Updated weights for policy 1, policy_version 141654 (0.0008) [2023-12-26 16:23:37,066][105620] Updated weights for policy 1, policy_version 141664 (0.0007) [2023-12-26 16:23:37,068][105692] Updated weights for policy 0, policy_version 140985 (0.0006) [2023-12-26 16:23:37,123][105692] Updated weights for policy 0, policy_version 140995 (0.0007) [2023-12-26 16:23:37,710][105620] Updated weights for policy 1, policy_version 141674 (0.0007) [2023-12-26 16:23:37,763][105620] Updated weights for policy 1, policy_version 141684 (0.0008) [2023-12-26 16:23:37,825][105620] Updated weights for policy 1, policy_version 141694 (0.0009) [2023-12-26 16:23:37,881][105620] Updated weights for policy 1, policy_version 141704 (0.0009) [2023-12-26 16:23:37,926][105692] Updated weights for policy 0, policy_version 141005 (0.0008) [2023-12-26 16:23:37,988][105692] Updated weights for policy 0, policy_version 141015 (0.0009) [2023-12-26 16:23:38,050][105692] Updated weights for policy 0, policy_version 141025 (0.0007) [2023-12-26 16:23:38,630][105620] Updated weights for policy 1, policy_version 141714 (0.0007) [2023-12-26 16:23:38,682][105620] Updated weights for policy 1, policy_version 141724 (0.0009) [2023-12-26 16:23:38,738][105620] Updated weights for policy 1, policy_version 141734 (0.0009) [2023-12-26 16:23:38,753][105692] Updated weights for policy 0, policy_version 141035 (0.0006) [2023-12-26 16:23:38,818][105692] Updated weights for policy 0, policy_version 141045 (0.0009) [2023-12-26 16:23:38,882][105692] Updated weights for policy 0, policy_version 141055 (0.0010) [2023-12-26 16:23:39,473][105620] Updated weights for policy 1, policy_version 141744 (0.0009) [2023-12-26 16:23:39,536][105620] Updated weights for policy 1, policy_version 141754 (0.0009) [2023-12-26 16:23:39,599][105620] Updated weights for policy 1, policy_version 141764 (0.0009) [2023-12-26 16:23:39,694][105692] Updated weights for policy 0, policy_version 141065 (0.0010) [2023-12-26 16:23:39,757][105692] Updated weights for policy 0, policy_version 141075 (0.0010) [2023-12-26 16:23:39,811][105692] Updated weights for policy 0, policy_version 141085 (0.0009) [2023-12-26 16:23:39,876][105692] Updated weights for policy 0, policy_version 141095 (0.0009) [2023-12-26 16:23:40,281][105620] Updated weights for policy 1, policy_version 141774 (0.0009) [2023-12-26 16:23:40,349][105620] Updated weights for policy 1, policy_version 141784 (0.0009) [2023-12-26 16:23:40,412][105620] Updated weights for policy 1, policy_version 141794 (0.0009) [2023-12-26 16:23:40,663][105692] Updated weights for policy 0, policy_version 141105 (0.0009) [2023-12-26 16:23:40,720][105692] Updated weights for policy 0, policy_version 141115 (0.0008) [2023-12-26 16:23:40,774][105692] Updated weights for policy 0, policy_version 141125 (0.0009) [2023-12-26 16:23:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 72441856. Throughput: 0: 9924.9, 1: 9655.4. Samples: 72451868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:23:41,062][104569] Avg episode reward: [(0, '9172.453'), (1, '9086.227')] [2023-12-26 16:23:41,234][105620] Updated weights for policy 1, policy_version 141804 (0.0009) [2023-12-26 16:23:41,300][105620] Updated weights for policy 1, policy_version 141814 (0.0009) [2023-12-26 16:23:41,366][105620] Updated weights for policy 1, policy_version 141824 (0.0010) [2023-12-26 16:23:41,563][105692] Updated weights for policy 0, policy_version 141135 (0.0009) [2023-12-26 16:23:41,627][105692] Updated weights for policy 0, policy_version 141145 (0.0008) [2023-12-26 16:23:41,695][105692] Updated weights for policy 0, policy_version 141155 (0.0008) [2023-12-26 16:23:42,136][105620] Updated weights for policy 1, policy_version 141834 (0.0009) [2023-12-26 16:23:42,200][105620] Updated weights for policy 1, policy_version 141844 (0.0009) [2023-12-26 16:23:42,271][105620] Updated weights for policy 1, policy_version 141854 (0.0009) [2023-12-26 16:23:42,334][105620] Updated weights for policy 1, policy_version 141864 (0.0009) [2023-12-26 16:23:42,466][105692] Updated weights for policy 0, policy_version 141165 (0.0009) [2023-12-26 16:23:42,528][105692] Updated weights for policy 0, policy_version 141175 (0.0009) [2023-12-26 16:23:42,586][105692] Updated weights for policy 0, policy_version 141185 (0.0008) [2023-12-26 16:23:43,088][105620] Updated weights for policy 1, policy_version 141874 (0.0009) [2023-12-26 16:23:43,135][105620] Updated weights for policy 1, policy_version 141884 (0.0009) [2023-12-26 16:23:43,204][105620] Updated weights for policy 1, policy_version 141894 (0.0009) [2023-12-26 16:23:43,338][105692] Updated weights for policy 0, policy_version 141195 (0.0010) [2023-12-26 16:23:43,389][105692] Updated weights for policy 0, policy_version 141205 (0.0009) [2023-12-26 16:23:43,436][105692] Updated weights for policy 0, policy_version 141215 (0.0008) [2023-12-26 16:23:43,969][105620] Updated weights for policy 1, policy_version 141904 (0.0009) [2023-12-26 16:23:44,018][105620] Updated weights for policy 1, policy_version 141914 (0.0005) [2023-12-26 16:23:44,069][105620] Updated weights for policy 1, policy_version 141924 (0.0005) [2023-12-26 16:23:44,225][105692] Updated weights for policy 0, policy_version 141225 (0.0009) [2023-12-26 16:23:44,281][105692] Updated weights for policy 0, policy_version 141235 (0.0009) [2023-12-26 16:23:44,326][105692] Updated weights for policy 0, policy_version 141245 (0.0008) [2023-12-26 16:23:44,379][105692] Updated weights for policy 0, policy_version 141255 (0.0009) [2023-12-26 16:23:44,665][105620] Updated weights for policy 1, policy_version 141934 (0.0005) [2023-12-26 16:23:44,712][105620] Updated weights for policy 1, policy_version 141944 (0.0005) [2023-12-26 16:23:44,765][105620] Updated weights for policy 1, policy_version 141954 (0.0006) [2023-12-26 16:23:45,194][105692] Updated weights for policy 0, policy_version 141265 (0.0010) [2023-12-26 16:23:45,264][105692] Updated weights for policy 0, policy_version 141275 (0.0010) [2023-12-26 16:23:45,327][105692] Updated weights for policy 0, policy_version 141285 (0.0010) [2023-12-26 16:23:45,446][105620] Updated weights for policy 1, policy_version 141964 (0.0009) [2023-12-26 16:23:45,504][105620] Updated weights for policy 1, policy_version 141974 (0.0010) [2023-12-26 16:23:45,550][105620] Updated weights for policy 1, policy_version 141984 (0.0006) [2023-12-26 16:23:46,052][105692] Updated weights for policy 0, policy_version 141295 (0.0010) [2023-12-26 16:23:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 72531968. Throughput: 0: 9841.0, 1: 9599.4. Samples: 72505412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:23:46,062][104569] Avg episode reward: [(0, '9264.319'), (1, '8722.572')] [2023-12-26 16:23:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000141992_36356096.pth... [2023-12-26 16:23:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000140872_36069376.pth [2023-12-26 16:23:46,114][105692] Updated weights for policy 0, policy_version 141305 (0.0009) [2023-12-26 16:23:46,120][105620] Updated weights for policy 1, policy_version 141994 (0.0006) [2023-12-26 16:23:46,176][105692] Updated weights for policy 0, policy_version 141315 (0.0011) [2023-12-26 16:23:46,177][105620] Updated weights for policy 1, policy_version 142004 (0.0010) [2023-12-26 16:23:46,204][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000141320_36184064.pth... [2023-12-26 16:23:46,208][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000140168_35889152.pth [2023-12-26 16:23:46,238][105620] Updated weights for policy 1, policy_version 142014 (0.0010) [2023-12-26 16:23:46,299][105620] Updated weights for policy 1, policy_version 142024 (0.0010) [2023-12-26 16:23:46,895][105692] Updated weights for policy 0, policy_version 141325 (0.0010) [2023-12-26 16:23:46,948][105692] Updated weights for policy 0, policy_version 141335 (0.0010) [2023-12-26 16:23:46,950][105620] Updated weights for policy 1, policy_version 142034 (0.0005) [2023-12-26 16:23:46,997][105692] Updated weights for policy 0, policy_version 141345 (0.0010) [2023-12-26 16:23:47,006][105620] Updated weights for policy 1, policy_version 142044 (0.0006) [2023-12-26 16:23:47,058][105620] Updated weights for policy 1, policy_version 142054 (0.0006) [2023-12-26 16:23:47,642][105620] Updated weights for policy 1, policy_version 142064 (0.0007) [2023-12-26 16:23:47,653][105692] Updated weights for policy 0, policy_version 141355 (0.0010) [2023-12-26 16:23:47,702][105620] Updated weights for policy 1, policy_version 142074 (0.0007) [2023-12-26 16:23:47,720][105692] Updated weights for policy 0, policy_version 141365 (0.0009) [2023-12-26 16:23:47,769][105620] Updated weights for policy 1, policy_version 142084 (0.0005) [2023-12-26 16:23:47,779][105692] Updated weights for policy 0, policy_version 141375 (0.0010) [2023-12-26 16:23:48,421][105620] Updated weights for policy 1, policy_version 142094 (0.0007) [2023-12-26 16:23:48,481][105620] Updated weights for policy 1, policy_version 142104 (0.0007) [2023-12-26 16:23:48,487][105692] Updated weights for policy 0, policy_version 141385 (0.0010) [2023-12-26 16:23:48,529][105620] Updated weights for policy 1, policy_version 142114 (0.0006) [2023-12-26 16:23:48,549][105692] Updated weights for policy 0, policy_version 141395 (0.0010) [2023-12-26 16:23:48,614][105692] Updated weights for policy 0, policy_version 141405 (0.0010) [2023-12-26 16:23:48,665][105692] Updated weights for policy 0, policy_version 141415 (0.0010) [2023-12-26 16:23:49,260][105620] Updated weights for policy 1, policy_version 142124 (0.0009) [2023-12-26 16:23:49,326][105620] Updated weights for policy 1, policy_version 142134 (0.0011) [2023-12-26 16:23:49,391][105620] Updated weights for policy 1, policy_version 142144 (0.0010) [2023-12-26 16:23:49,412][105692] Updated weights for policy 0, policy_version 141425 (0.0008) [2023-12-26 16:23:49,470][105692] Updated weights for policy 0, policy_version 141435 (0.0008) [2023-12-26 16:23:49,533][105692] Updated weights for policy 0, policy_version 141445 (0.0008) [2023-12-26 16:23:50,097][105620] Updated weights for policy 1, policy_version 142154 (0.0006) [2023-12-26 16:23:50,150][105620] Updated weights for policy 1, policy_version 142164 (0.0011) [2023-12-26 16:23:50,205][105620] Updated weights for policy 1, policy_version 142174 (0.0010) [2023-12-26 16:23:50,267][105620] Updated weights for policy 1, policy_version 142184 (0.0010) [2023-12-26 16:23:50,303][105692] Updated weights for policy 0, policy_version 141455 (0.0008) [2023-12-26 16:23:50,354][105692] Updated weights for policy 0, policy_version 141465 (0.0008) [2023-12-26 16:23:50,397][105692] Updated weights for policy 0, policy_version 141475 (0.0007) [2023-12-26 16:23:50,969][105620] Updated weights for policy 1, policy_version 142194 (0.0008) [2023-12-26 16:23:51,021][105620] Updated weights for policy 1, policy_version 142204 (0.0010) [2023-12-26 16:23:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 72630272. Throughput: 0: 9779.3, 1: 9730.1. Samples: 72626472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:23:51,063][104569] Avg episode reward: [(0, '9172.555'), (1, '8444.633')] [2023-12-26 16:23:51,091][105620] Updated weights for policy 1, policy_version 142214 (0.0008) [2023-12-26 16:23:51,134][105692] Updated weights for policy 0, policy_version 141485 (0.0006) [2023-12-26 16:23:51,200][105692] Updated weights for policy 0, policy_version 141495 (0.0009) [2023-12-26 16:23:51,263][105692] Updated weights for policy 0, policy_version 141505 (0.0010) [2023-12-26 16:23:51,818][105620] Updated weights for policy 1, policy_version 142224 (0.0009) [2023-12-26 16:23:51,883][105620] Updated weights for policy 1, policy_version 142234 (0.0010) [2023-12-26 16:23:51,945][105620] Updated weights for policy 1, policy_version 142244 (0.0009) [2023-12-26 16:23:52,026][105692] Updated weights for policy 0, policy_version 141515 (0.0009) [2023-12-26 16:23:52,087][105692] Updated weights for policy 0, policy_version 141525 (0.0009) [2023-12-26 16:23:52,145][105692] Updated weights for policy 0, policy_version 141535 (0.0009) [2023-12-26 16:23:52,627][105620] Updated weights for policy 1, policy_version 142254 (0.0008) [2023-12-26 16:23:52,694][105620] Updated weights for policy 1, policy_version 142264 (0.0006) [2023-12-26 16:23:52,761][105620] Updated weights for policy 1, policy_version 142274 (0.0005) [2023-12-26 16:23:52,940][105692] Updated weights for policy 0, policy_version 141545 (0.0009) [2023-12-26 16:23:52,999][105692] Updated weights for policy 0, policy_version 141555 (0.0009) [2023-12-26 16:23:53,067][105692] Updated weights for policy 0, policy_version 141565 (0.0010) [2023-12-26 16:23:53,133][105692] Updated weights for policy 0, policy_version 141575 (0.0010) [2023-12-26 16:23:53,336][105620] Updated weights for policy 1, policy_version 142284 (0.0006) [2023-12-26 16:23:53,386][105620] Updated weights for policy 1, policy_version 142294 (0.0010) [2023-12-26 16:23:53,444][105620] Updated weights for policy 1, policy_version 142304 (0.0010) [2023-12-26 16:23:53,915][105692] Updated weights for policy 0, policy_version 141585 (0.0006) [2023-12-26 16:23:53,980][105692] Updated weights for policy 0, policy_version 141595 (0.0005) [2023-12-26 16:23:54,040][105692] Updated weights for policy 0, policy_version 141605 (0.0008) [2023-12-26 16:23:54,180][105620] Updated weights for policy 1, policy_version 142314 (0.0010) [2023-12-26 16:23:54,244][105620] Updated weights for policy 1, policy_version 142324 (0.0009) [2023-12-26 16:23:54,306][105620] Updated weights for policy 1, policy_version 142334 (0.0009) [2023-12-26 16:23:54,368][105620] Updated weights for policy 1, policy_version 142344 (0.0009) [2023-12-26 16:23:54,740][105692] Updated weights for policy 0, policy_version 141615 (0.0008) [2023-12-26 16:23:54,788][105692] Updated weights for policy 0, policy_version 141625 (0.0009) [2023-12-26 16:23:54,846][105692] Updated weights for policy 0, policy_version 141635 (0.0010) [2023-12-26 16:23:55,006][105620] Updated weights for policy 1, policy_version 142354 (0.0006) [2023-12-26 16:23:55,068][105620] Updated weights for policy 1, policy_version 142364 (0.0005) [2023-12-26 16:23:55,128][105620] Updated weights for policy 1, policy_version 142374 (0.0005) [2023-12-26 16:23:55,472][105692] Updated weights for policy 0, policy_version 141646 (0.0007) [2023-12-26 16:23:55,535][105692] Updated weights for policy 0, policy_version 141656 (0.0005) [2023-12-26 16:23:55,601][105692] Updated weights for policy 0, policy_version 141666 (0.0006) [2023-12-26 16:23:55,636][105620] Updated weights for policy 1, policy_version 142384 (0.0005) [2023-12-26 16:23:55,687][105620] Updated weights for policy 1, policy_version 142394 (0.0006) [2023-12-26 16:23:55,742][105620] Updated weights for policy 1, policy_version 142404 (0.0008) [2023-12-26 16:23:56,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 72736768. Throughput: 0: 9717.7, 1: 9815.7. Samples: 72745516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:23:56,062][104569] Avg episode reward: [(0, '9264.854'), (1, '8415.815')] [2023-12-26 16:23:56,252][105692] Updated weights for policy 0, policy_version 141676 (0.0009) [2023-12-26 16:23:56,303][105692] Updated weights for policy 0, policy_version 141686 (0.0010) [2023-12-26 16:23:56,364][105692] Updated weights for policy 0, policy_version 141696 (0.0010) [2023-12-26 16:23:56,418][105620] Updated weights for policy 1, policy_version 142414 (0.0007) [2023-12-26 16:23:56,477][105620] Updated weights for policy 1, policy_version 142424 (0.0008) [2023-12-26 16:23:56,536][105620] Updated weights for policy 1, policy_version 142434 (0.0008) [2023-12-26 16:23:56,996][105692] Updated weights for policy 0, policy_version 141706 (0.0009) [2023-12-26 16:23:57,061][105692] Updated weights for policy 0, policy_version 141716 (0.0006) [2023-12-26 16:23:57,111][105692] Updated weights for policy 0, policy_version 141726 (0.0006) [2023-12-26 16:23:57,113][105620] Updated weights for policy 1, policy_version 142444 (0.0007) [2023-12-26 16:23:57,157][105692] Updated weights for policy 0, policy_version 141736 (0.0005) [2023-12-26 16:23:57,170][105620] Updated weights for policy 1, policy_version 142454 (0.0008) [2023-12-26 16:23:57,231][105620] Updated weights for policy 1, policy_version 142464 (0.0008) [2023-12-26 16:23:57,860][105692] Updated weights for policy 0, policy_version 141746 (0.0009) [2023-12-26 16:23:57,926][105692] Updated weights for policy 0, policy_version 141756 (0.0009) [2023-12-26 16:23:57,986][105620] Updated weights for policy 1, policy_version 142474 (0.0008) [2023-12-26 16:23:57,995][105692] Updated weights for policy 0, policy_version 141766 (0.0009) [2023-12-26 16:23:58,048][105620] Updated weights for policy 1, policy_version 142484 (0.0009) [2023-12-26 16:23:58,108][105620] Updated weights for policy 1, policy_version 142494 (0.0010) [2023-12-26 16:23:58,170][105620] Updated weights for policy 1, policy_version 142504 (0.0009) [2023-12-26 16:23:58,782][105692] Updated weights for policy 0, policy_version 141776 (0.0008) [2023-12-26 16:23:58,848][105692] Updated weights for policy 0, policy_version 141786 (0.0007) [2023-12-26 16:23:58,918][105692] Updated weights for policy 0, policy_version 141796 (0.0010) [2023-12-26 16:23:59,057][105620] Updated weights for policy 1, policy_version 142514 (0.0008) [2023-12-26 16:23:59,116][105620] Updated weights for policy 1, policy_version 142524 (0.0007) [2023-12-26 16:23:59,182][105620] Updated weights for policy 1, policy_version 142534 (0.0007) [2023-12-26 16:23:59,747][105692] Updated weights for policy 0, policy_version 141806 (0.0010) [2023-12-26 16:23:59,795][105692] Updated weights for policy 0, policy_version 141816 (0.0010) [2023-12-26 16:23:59,856][105692] Updated weights for policy 0, policy_version 141826 (0.0010) [2023-12-26 16:23:59,894][105620] Updated weights for policy 1, policy_version 142544 (0.0006) [2023-12-26 16:23:59,961][105620] Updated weights for policy 1, policy_version 142554 (0.0007) [2023-12-26 16:24:00,017][105620] Updated weights for policy 1, policy_version 142564 (0.0008) [2023-12-26 16:24:00,601][105692] Updated weights for policy 0, policy_version 141836 (0.0008) [2023-12-26 16:24:00,624][105620] Updated weights for policy 1, policy_version 142574 (0.0007) [2023-12-26 16:24:00,663][105692] Updated weights for policy 0, policy_version 141846 (0.0007) [2023-12-26 16:24:00,690][105620] Updated weights for policy 1, policy_version 142584 (0.0005) [2023-12-26 16:24:00,726][105692] Updated weights for policy 0, policy_version 141856 (0.0009) [2023-12-26 16:24:00,740][105620] Updated weights for policy 1, policy_version 142594 (0.0006) [2023-12-26 16:24:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.7, 300 sec: 19522.0). Total num frames: 72835072. Throughput: 0: 9746.6, 1: 9812.4. Samples: 72804068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:24:01,063][104569] Avg episode reward: [(0, '9357.123'), (1, '4765.331')] [2023-12-26 16:24:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000141864_36323328.pth... [2023-12-26 16:24:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000142600_36511744.pth... [2023-12-26 16:24:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000140744_36036608.pth [2023-12-26 16:24:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000141448_36216832.pth [2023-12-26 16:24:01,357][105692] Updated weights for policy 0, policy_version 141866 (0.0011) [2023-12-26 16:24:01,414][105620] Updated weights for policy 1, policy_version 142604 (0.0006) [2023-12-26 16:24:01,426][105692] Updated weights for policy 0, policy_version 141876 (0.0007) [2023-12-26 16:24:01,476][105692] Updated weights for policy 0, policy_version 141886 (0.0009) [2023-12-26 16:24:01,479][105620] Updated weights for policy 1, policy_version 142614 (0.0005) [2023-12-26 16:24:01,528][105692] Updated weights for policy 0, policy_version 141896 (0.0008) [2023-12-26 16:24:01,540][105620] Updated weights for policy 1, policy_version 142624 (0.0006) [2023-12-26 16:24:02,245][105620] Updated weights for policy 1, policy_version 142634 (0.0007) [2023-12-26 16:24:02,272][105692] Updated weights for policy 0, policy_version 141906 (0.0007) [2023-12-26 16:24:02,303][105620] Updated weights for policy 1, policy_version 142644 (0.0007) [2023-12-26 16:24:02,338][105692] Updated weights for policy 0, policy_version 141916 (0.0006) [2023-12-26 16:24:02,364][105620] Updated weights for policy 1, policy_version 142654 (0.0008) [2023-12-26 16:24:02,407][105692] Updated weights for policy 0, policy_version 141926 (0.0007) [2023-12-26 16:24:02,424][105620] Updated weights for policy 1, policy_version 142664 (0.0010) [2023-12-26 16:24:03,027][105692] Updated weights for policy 0, policy_version 141936 (0.0006) [2023-12-26 16:24:03,080][105692] Updated weights for policy 0, policy_version 141946 (0.0009) [2023-12-26 16:24:03,132][105692] Updated weights for policy 0, policy_version 141957 (0.0009) [2023-12-26 16:24:03,178][105620] Updated weights for policy 1, policy_version 142674 (0.0010) [2023-12-26 16:24:03,223][105620] Updated weights for policy 1, policy_version 142684 (0.0010) [2023-12-26 16:24:03,267][105620] Updated weights for policy 1, policy_version 142694 (0.0010) [2023-12-26 16:24:03,872][105692] Updated weights for policy 0, policy_version 141967 (0.0007) [2023-12-26 16:24:03,929][105692] Updated weights for policy 0, policy_version 141977 (0.0008) [2023-12-26 16:24:03,990][105692] Updated weights for policy 0, policy_version 141987 (0.0007) [2023-12-26 16:24:04,024][105620] Updated weights for policy 1, policy_version 142704 (0.0010) [2023-12-26 16:24:04,088][105620] Updated weights for policy 1, policy_version 142714 (0.0008) [2023-12-26 16:24:04,149][105620] Updated weights for policy 1, policy_version 142724 (0.0011) [2023-12-26 16:24:04,685][105692] Updated weights for policy 0, policy_version 141997 (0.0007) [2023-12-26 16:24:04,737][105692] Updated weights for policy 0, policy_version 142007 (0.0008) [2023-12-26 16:24:04,793][105692] Updated weights for policy 0, policy_version 142017 (0.0008) [2023-12-26 16:24:04,846][105620] Updated weights for policy 1, policy_version 142734 (0.0009) [2023-12-26 16:24:04,900][105620] Updated weights for policy 1, policy_version 142744 (0.0005) [2023-12-26 16:24:04,948][105620] Updated weights for policy 1, policy_version 142754 (0.0005) [2023-12-26 16:24:05,468][105692] Updated weights for policy 0, policy_version 142027 (0.0007) [2023-12-26 16:24:05,502][105620] Updated weights for policy 1, policy_version 142764 (0.0006) [2023-12-26 16:24:05,522][105692] Updated weights for policy 0, policy_version 142037 (0.0005) [2023-12-26 16:24:05,561][105620] Updated weights for policy 1, policy_version 142774 (0.0006) [2023-12-26 16:24:05,582][105692] Updated weights for policy 0, policy_version 142047 (0.0010) [2023-12-26 16:24:05,619][105620] Updated weights for policy 1, policy_version 142784 (0.0005) [2023-12-26 16:24:06,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 72933376. Throughput: 0: 9634.0, 1: 9804.7. Samples: 72921296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:24:06,063][104569] Avg episode reward: [(0, '9357.414'), (1, '1829.387')] [2023-12-26 16:24:06,153][105692] Updated weights for policy 0, policy_version 142057 (0.0010) [2023-12-26 16:24:06,223][105692] Updated weights for policy 0, policy_version 142067 (0.0005) [2023-12-26 16:24:06,261][105620] Updated weights for policy 1, policy_version 142794 (0.0007) [2023-12-26 16:24:06,292][105692] Updated weights for policy 0, policy_version 142077 (0.0006) [2023-12-26 16:24:06,310][105620] Updated weights for policy 1, policy_version 142804 (0.0008) [2023-12-26 16:24:06,353][105692] Updated weights for policy 0, policy_version 142087 (0.0009) [2023-12-26 16:24:06,361][105620] Updated weights for policy 1, policy_version 142814 (0.0007) [2023-12-26 16:24:06,410][105620] Updated weights for policy 1, policy_version 142824 (0.0008) [2023-12-26 16:24:07,087][105692] Updated weights for policy 0, policy_version 142097 (0.0011) [2023-12-26 16:24:07,095][105620] Updated weights for policy 1, policy_version 142834 (0.0006) [2023-12-26 16:24:07,146][105620] Updated weights for policy 1, policy_version 142844 (0.0009) [2023-12-26 16:24:07,148][105692] Updated weights for policy 0, policy_version 142107 (0.0011) [2023-12-26 16:24:07,199][105620] Updated weights for policy 1, policy_version 142854 (0.0005) [2023-12-26 16:24:07,200][105692] Updated weights for policy 0, policy_version 142117 (0.0010) [2023-12-26 16:24:07,790][105620] Updated weights for policy 1, policy_version 142864 (0.0010) [2023-12-26 16:24:07,844][105620] Updated weights for policy 1, policy_version 142874 (0.0010) [2023-12-26 16:24:07,897][105620] Updated weights for policy 1, policy_version 142884 (0.0008) [2023-12-26 16:24:07,915][105692] Updated weights for policy 0, policy_version 142127 (0.0009) [2023-12-26 16:24:07,964][105692] Updated weights for policy 0, policy_version 142137 (0.0010) [2023-12-26 16:24:08,024][105692] Updated weights for policy 0, policy_version 142147 (0.0008) [2023-12-26 16:24:08,529][105620] Updated weights for policy 1, policy_version 142894 (0.0009) [2023-12-26 16:24:08,577][105620] Updated weights for policy 1, policy_version 142904 (0.0008) [2023-12-26 16:24:08,633][105620] Updated weights for policy 1, policy_version 142914 (0.0009) [2023-12-26 16:24:08,652][105692] Updated weights for policy 0, policy_version 142157 (0.0009) [2023-12-26 16:24:08,699][105692] Updated weights for policy 0, policy_version 142167 (0.0010) [2023-12-26 16:24:08,744][105692] Updated weights for policy 0, policy_version 142177 (0.0010) [2023-12-26 16:24:09,372][105620] Updated weights for policy 1, policy_version 142924 (0.0008) [2023-12-26 16:24:09,438][105620] Updated weights for policy 1, policy_version 142934 (0.0010) [2023-12-26 16:24:09,446][105692] Updated weights for policy 0, policy_version 142187 (0.0011) [2023-12-26 16:24:09,495][105620] Updated weights for policy 1, policy_version 142944 (0.0011) [2023-12-26 16:24:09,509][105692] Updated weights for policy 0, policy_version 142197 (0.0011) [2023-12-26 16:24:09,569][105692] Updated weights for policy 0, policy_version 142207 (0.0010) [2023-12-26 16:24:10,213][105620] Updated weights for policy 1, policy_version 142954 (0.0010) [2023-12-26 16:24:10,275][105620] Updated weights for policy 1, policy_version 142964 (0.0009) [2023-12-26 16:24:10,333][105620] Updated weights for policy 1, policy_version 142974 (0.0009) [2023-12-26 16:24:10,344][105692] Updated weights for policy 0, policy_version 142217 (0.0010) [2023-12-26 16:24:10,393][105620] Updated weights for policy 1, policy_version 142984 (0.0007) [2023-12-26 16:24:10,407][105692] Updated weights for policy 0, policy_version 142227 (0.0010) [2023-12-26 16:24:10,469][105692] Updated weights for policy 0, policy_version 142237 (0.0009) [2023-12-26 16:24:10,530][105692] Updated weights for policy 0, policy_version 142247 (0.0009) [2023-12-26 16:24:11,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 73031680. Throughput: 0: 9600.4, 1: 9901.0. Samples: 73043372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:24:11,062][104569] Avg episode reward: [(0, '9357.706'), (1, '1550.740')] [2023-12-26 16:24:11,214][105620] Updated weights for policy 1, policy_version 142994 (0.0008) [2023-12-26 16:24:11,233][105692] Updated weights for policy 0, policy_version 142257 (0.0006) [2023-12-26 16:24:11,276][105620] Updated weights for policy 1, policy_version 143004 (0.0007) [2023-12-26 16:24:11,300][105692] Updated weights for policy 0, policy_version 142267 (0.0006) [2023-12-26 16:24:11,336][105620] Updated weights for policy 1, policy_version 143014 (0.0008) [2023-12-26 16:24:11,364][105692] Updated weights for policy 0, policy_version 142277 (0.0007) [2023-12-26 16:24:12,077][105692] Updated weights for policy 0, policy_version 142287 (0.0007) [2023-12-26 16:24:12,090][105620] Updated weights for policy 1, policy_version 143024 (0.0010) [2023-12-26 16:24:12,139][105692] Updated weights for policy 0, policy_version 142297 (0.0009) [2023-12-26 16:24:12,153][105620] Updated weights for policy 1, policy_version 143034 (0.0011) [2023-12-26 16:24:12,199][105692] Updated weights for policy 0, policy_version 142307 (0.0006) [2023-12-26 16:24:12,213][105620] Updated weights for policy 1, policy_version 143044 (0.0011) [2023-12-26 16:24:12,830][105620] Updated weights for policy 1, policy_version 143054 (0.0011) [2023-12-26 16:24:12,883][105620] Updated weights for policy 1, policy_version 143064 (0.0010) [2023-12-26 16:24:12,933][105620] Updated weights for policy 1, policy_version 143074 (0.0011) [2023-12-26 16:24:13,036][105692] Updated weights for policy 0, policy_version 142317 (0.0010) [2023-12-26 16:24:13,093][105692] Updated weights for policy 0, policy_version 142327 (0.0011) [2023-12-26 16:24:13,146][105692] Updated weights for policy 0, policy_version 142337 (0.0011) [2023-12-26 16:24:13,693][105620] Updated weights for policy 1, policy_version 143084 (0.0009) [2023-12-26 16:24:13,720][105692] Updated weights for policy 0, policy_version 142347 (0.0010) [2023-12-26 16:24:13,747][105620] Updated weights for policy 1, policy_version 143094 (0.0005) [2023-12-26 16:24:13,785][105692] Updated weights for policy 0, policy_version 142357 (0.0009) [2023-12-26 16:24:13,800][105620] Updated weights for policy 1, policy_version 143104 (0.0005) [2023-12-26 16:24:13,848][105692] Updated weights for policy 0, policy_version 142367 (0.0008) [2023-12-26 16:24:14,466][105620] Updated weights for policy 1, policy_version 143114 (0.0009) [2023-12-26 16:24:14,515][105620] Updated weights for policy 1, policy_version 143124 (0.0005) [2023-12-26 16:24:14,534][105692] Updated weights for policy 0, policy_version 142377 (0.0008) [2023-12-26 16:24:14,577][105620] Updated weights for policy 1, policy_version 143134 (0.0009) [2023-12-26 16:24:14,595][105692] Updated weights for policy 0, policy_version 142387 (0.0006) [2023-12-26 16:24:14,629][105620] Updated weights for policy 1, policy_version 143144 (0.0010) [2023-12-26 16:24:14,651][105692] Updated weights for policy 0, policy_version 142397 (0.0005) [2023-12-26 16:24:14,710][105692] Updated weights for policy 0, policy_version 142407 (0.0006) [2023-12-26 16:24:15,374][105620] Updated weights for policy 1, policy_version 143154 (0.0008) [2023-12-26 16:24:15,415][105692] Updated weights for policy 0, policy_version 142417 (0.0011) [2023-12-26 16:24:15,430][105620] Updated weights for policy 1, policy_version 143164 (0.0006) [2023-12-26 16:24:15,475][105692] Updated weights for policy 0, policy_version 142427 (0.0011) [2023-12-26 16:24:15,486][105620] Updated weights for policy 1, policy_version 143174 (0.0007) [2023-12-26 16:24:15,535][105692] Updated weights for policy 0, policy_version 142437 (0.0011) [2023-12-26 16:24:16,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 73129984. Throughput: 0: 9594.0, 1: 9878.1. Samples: 73101572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:24:16,062][104569] Avg episode reward: [(0, '9357.699'), (1, '6485.521')] [2023-12-26 16:24:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000142440_36470784.pth... [2023-12-26 16:24:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000143176_36659200.pth... [2023-12-26 16:24:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000141320_36184064.pth [2023-12-26 16:24:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000141992_36356096.pth [2023-12-26 16:24:16,235][105620] Updated weights for policy 1, policy_version 143184 (0.0006) [2023-12-26 16:24:16,279][105692] Updated weights for policy 0, policy_version 142447 (0.0010) [2023-12-26 16:24:16,285][105620] Updated weights for policy 1, policy_version 143194 (0.0005) [2023-12-26 16:24:16,327][105692] Updated weights for policy 0, policy_version 142457 (0.0010) [2023-12-26 16:24:16,332][105620] Updated weights for policy 1, policy_version 143204 (0.0006) [2023-12-26 16:24:16,379][105692] Updated weights for policy 0, policy_version 142467 (0.0006) [2023-12-26 16:24:17,012][105692] Updated weights for policy 0, policy_version 142477 (0.0008) [2023-12-26 16:24:17,030][105620] Updated weights for policy 1, policy_version 143214 (0.0006) [2023-12-26 16:24:17,064][105692] Updated weights for policy 0, policy_version 142487 (0.0010) [2023-12-26 16:24:17,086][105620] Updated weights for policy 1, policy_version 143224 (0.0005) [2023-12-26 16:24:17,120][105692] Updated weights for policy 0, policy_version 142497 (0.0010) [2023-12-26 16:24:17,149][105620] Updated weights for policy 1, policy_version 143234 (0.0006) [2023-12-26 16:24:17,870][105692] Updated weights for policy 0, policy_version 142507 (0.0010) [2023-12-26 16:24:17,895][105620] Updated weights for policy 1, policy_version 143244 (0.0007) [2023-12-26 16:24:17,928][105692] Updated weights for policy 0, policy_version 142517 (0.0011) [2023-12-26 16:24:17,955][105620] Updated weights for policy 1, policy_version 143254 (0.0006) [2023-12-26 16:24:17,983][105692] Updated weights for policy 0, policy_version 142527 (0.0010) [2023-12-26 16:24:18,010][105620] Updated weights for policy 1, policy_version 143264 (0.0006) [2023-12-26 16:24:18,691][105692] Updated weights for policy 0, policy_version 142537 (0.0010) [2023-12-26 16:24:18,756][105692] Updated weights for policy 0, policy_version 142547 (0.0005) [2023-12-26 16:24:18,790][105620] Updated weights for policy 1, policy_version 143274 (0.0007) [2023-12-26 16:24:18,821][105692] Updated weights for policy 0, policy_version 142557 (0.0006) [2023-12-26 16:24:18,855][105620] Updated weights for policy 1, policy_version 143284 (0.0008) [2023-12-26 16:24:18,883][105692] Updated weights for policy 0, policy_version 142567 (0.0006) [2023-12-26 16:24:18,909][105620] Updated weights for policy 1, policy_version 143294 (0.0009) [2023-12-26 16:24:18,961][105620] Updated weights for policy 1, policy_version 143304 (0.0009) [2023-12-26 16:24:19,570][105692] Updated weights for policy 0, policy_version 142577 (0.0009) [2023-12-26 16:24:19,623][105692] Updated weights for policy 0, policy_version 142587 (0.0009) [2023-12-26 16:24:19,679][105692] Updated weights for policy 0, policy_version 142597 (0.0006) [2023-12-26 16:24:19,711][105620] Updated weights for policy 1, policy_version 143314 (0.0009) [2023-12-26 16:24:19,765][105620] Updated weights for policy 1, policy_version 143325 (0.0010) [2023-12-26 16:24:19,819][105620] Updated weights for policy 1, policy_version 143335 (0.0009) [2023-12-26 16:24:20,424][105692] Updated weights for policy 0, policy_version 142607 (0.0009) [2023-12-26 16:24:20,478][105692] Updated weights for policy 0, policy_version 142617 (0.0010) [2023-12-26 16:24:20,527][105692] Updated weights for policy 0, policy_version 142627 (0.0010) [2023-12-26 16:24:20,639][105620] Updated weights for policy 1, policy_version 143345 (0.0008) [2023-12-26 16:24:20,700][105620] Updated weights for policy 1, policy_version 143355 (0.0009) [2023-12-26 16:24:20,757][105620] Updated weights for policy 1, policy_version 143365 (0.0008) [2023-12-26 16:24:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 73228288. Throughput: 0: 9599.6, 1: 9928.6. Samples: 73217788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:24:21,062][104569] Avg episode reward: [(0, '9018.131'), (1, '7579.078')] [2023-12-26 16:24:21,261][105692] Updated weights for policy 0, policy_version 142637 (0.0009) [2023-12-26 16:24:21,324][105692] Updated weights for policy 0, policy_version 142647 (0.0010) [2023-12-26 16:24:21,392][105692] Updated weights for policy 0, policy_version 142657 (0.0009) [2023-12-26 16:24:21,487][105620] Updated weights for policy 1, policy_version 143375 (0.0010) [2023-12-26 16:24:21,546][105620] Updated weights for policy 1, policy_version 143385 (0.0010) [2023-12-26 16:24:21,605][105620] Updated weights for policy 1, policy_version 143395 (0.0011) [2023-12-26 16:24:22,120][105692] Updated weights for policy 0, policy_version 142667 (0.0006) [2023-12-26 16:24:22,175][105692] Updated weights for policy 0, policy_version 142677 (0.0010) [2023-12-26 16:24:22,228][105692] Updated weights for policy 0, policy_version 142687 (0.0009) [2023-12-26 16:24:22,304][105620] Updated weights for policy 1, policy_version 143405 (0.0009) [2023-12-26 16:24:22,374][105620] Updated weights for policy 1, policy_version 143415 (0.0008) [2023-12-26 16:24:22,440][105620] Updated weights for policy 1, policy_version 143425 (0.0008) [2023-12-26 16:24:23,068][105692] Updated weights for policy 0, policy_version 142697 (0.0008) [2023-12-26 16:24:23,119][105692] Updated weights for policy 0, policy_version 142707 (0.0006) [2023-12-26 16:24:23,128][105620] Updated weights for policy 1, policy_version 143435 (0.0009) [2023-12-26 16:24:23,169][105692] Updated weights for policy 0, policy_version 142717 (0.0008) [2023-12-26 16:24:23,187][105620] Updated weights for policy 1, policy_version 143445 (0.0010) [2023-12-26 16:24:23,222][105692] Updated weights for policy 0, policy_version 142727 (0.0005) [2023-12-26 16:24:23,242][105620] Updated weights for policy 1, policy_version 143455 (0.0010) [2023-12-26 16:24:23,933][105692] Updated weights for policy 0, policy_version 142737 (0.0008) [2023-12-26 16:24:23,973][105620] Updated weights for policy 1, policy_version 143465 (0.0010) [2023-12-26 16:24:23,992][105692] Updated weights for policy 0, policy_version 142747 (0.0008) [2023-12-26 16:24:24,031][105620] Updated weights for policy 1, policy_version 143475 (0.0007) [2023-12-26 16:24:24,049][105692] Updated weights for policy 0, policy_version 142757 (0.0007) [2023-12-26 16:24:24,089][105620] Updated weights for policy 1, policy_version 143485 (0.0007) [2023-12-26 16:24:24,143][105620] Updated weights for policy 1, policy_version 143495 (0.0008) [2023-12-26 16:24:24,807][105620] Updated weights for policy 1, policy_version 143505 (0.0006) [2023-12-26 16:24:24,837][105692] Updated weights for policy 0, policy_version 142767 (0.0007) [2023-12-26 16:24:24,854][105620] Updated weights for policy 1, policy_version 143515 (0.0005) [2023-12-26 16:24:24,895][105692] Updated weights for policy 0, policy_version 142777 (0.0005) [2023-12-26 16:24:24,905][105620] Updated weights for policy 1, policy_version 143525 (0.0006) [2023-12-26 16:24:24,952][105692] Updated weights for policy 0, policy_version 142787 (0.0005) [2023-12-26 16:24:25,496][105620] Updated weights for policy 1, policy_version 143535 (0.0007) [2023-12-26 16:24:25,542][105620] Updated weights for policy 1, policy_version 143545 (0.0008) [2023-12-26 16:24:25,592][105620] Updated weights for policy 1, policy_version 143555 (0.0009) [2023-12-26 16:24:25,662][105692] Updated weights for policy 0, policy_version 142797 (0.0007) [2023-12-26 16:24:25,709][105692] Updated weights for policy 0, policy_version 142807 (0.0009) [2023-12-26 16:24:25,762][105692] Updated weights for policy 0, policy_version 142817 (0.0009) [2023-12-26 16:24:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 73326592. Throughput: 0: 9619.9, 1: 9983.7. Samples: 73334032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:24:26,062][104569] Avg episode reward: [(0, '7990.239'), (1, '7418.322')] [2023-12-26 16:24:26,313][105620] Updated weights for policy 1, policy_version 143565 (0.0009) [2023-12-26 16:24:26,366][105620] Updated weights for policy 1, policy_version 143575 (0.0009) [2023-12-26 16:24:26,416][105620] Updated weights for policy 1, policy_version 143585 (0.0009) [2023-12-26 16:24:26,463][105692] Updated weights for policy 0, policy_version 142827 (0.0010) [2023-12-26 16:24:26,513][105692] Updated weights for policy 0, policy_version 142837 (0.0009) [2023-12-26 16:24:26,579][105692] Updated weights for policy 0, policy_version 142847 (0.0009) [2023-12-26 16:24:27,059][105620] Updated weights for policy 1, policy_version 143595 (0.0007) [2023-12-26 16:24:27,123][105620] Updated weights for policy 1, policy_version 143605 (0.0009) [2023-12-26 16:24:27,182][105620] Updated weights for policy 1, policy_version 143615 (0.0009) [2023-12-26 16:24:27,418][105692] Updated weights for policy 0, policy_version 142857 (0.0009) [2023-12-26 16:24:27,485][105692] Updated weights for policy 0, policy_version 142867 (0.0009) [2023-12-26 16:24:27,541][105692] Updated weights for policy 0, policy_version 142877 (0.0010) [2023-12-26 16:24:27,594][105692] Updated weights for policy 0, policy_version 142887 (0.0009) [2023-12-26 16:24:27,782][105620] Updated weights for policy 1, policy_version 143625 (0.0008) [2023-12-26 16:24:27,836][105620] Updated weights for policy 1, policy_version 143635 (0.0008) [2023-12-26 16:24:27,882][105620] Updated weights for policy 1, policy_version 143645 (0.0008) [2023-12-26 16:24:27,932][105620] Updated weights for policy 1, policy_version 143655 (0.0009) [2023-12-26 16:24:28,387][105692] Updated weights for policy 0, policy_version 142897 (0.0009) [2023-12-26 16:24:28,446][105692] Updated weights for policy 0, policy_version 142908 (0.0011) [2023-12-26 16:24:28,505][105692] Updated weights for policy 0, policy_version 142918 (0.0010) [2023-12-26 16:24:28,631][105620] Updated weights for policy 1, policy_version 143665 (0.0005) [2023-12-26 16:24:28,687][105620] Updated weights for policy 1, policy_version 143675 (0.0005) [2023-12-26 16:24:28,746][105620] Updated weights for policy 1, policy_version 143685 (0.0006) [2023-12-26 16:24:29,291][105620] Updated weights for policy 1, policy_version 143695 (0.0006) [2023-12-26 16:24:29,362][105620] Updated weights for policy 1, policy_version 143705 (0.0009) [2023-12-26 16:24:29,406][105692] Updated weights for policy 0, policy_version 142928 (0.0008) [2023-12-26 16:24:29,420][105620] Updated weights for policy 1, policy_version 143715 (0.0007) [2023-12-26 16:24:29,474][105692] Updated weights for policy 0, policy_version 142938 (0.0008) [2023-12-26 16:24:29,539][105692] Updated weights for policy 0, policy_version 142948 (0.0008) [2023-12-26 16:24:30,142][105620] Updated weights for policy 1, policy_version 143725 (0.0008) [2023-12-26 16:24:30,205][105620] Updated weights for policy 1, policy_version 143735 (0.0009) [2023-12-26 16:24:30,270][105620] Updated weights for policy 1, policy_version 143745 (0.0009) [2023-12-26 16:24:30,286][105692] Updated weights for policy 0, policy_version 142958 (0.0007) [2023-12-26 16:24:30,339][105692] Updated weights for policy 0, policy_version 142968 (0.0008) [2023-12-26 16:24:30,399][105692] Updated weights for policy 0, policy_version 142978 (0.0008) [2023-12-26 16:24:31,027][105692] Updated weights for policy 0, policy_version 142988 (0.0006) [2023-12-26 16:24:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 73416704. Throughput: 0: 9628.1, 1: 10098.3. Samples: 73393100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:24:31,063][104569] Avg episode reward: [(0, '6677.378'), (1, '6576.558')] [2023-12-26 16:24:31,068][105620] Updated weights for policy 1, policy_version 143755 (0.0008) [2023-12-26 16:24:31,090][105692] Updated weights for policy 0, policy_version 142998 (0.0007) [2023-12-26 16:24:31,132][105620] Updated weights for policy 1, policy_version 143765 (0.0011) [2023-12-26 16:24:31,157][105692] Updated weights for policy 0, policy_version 143008 (0.0008) [2023-12-26 16:24:31,199][105620] Updated weights for policy 1, policy_version 143775 (0.0011) [2023-12-26 16:24:31,203][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000143016_36618240.pth... [2023-12-26 16:24:31,207][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000141864_36323328.pth [2023-12-26 16:24:31,252][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000143784_36814848.pth... [2023-12-26 16:24:31,255][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000142600_36511744.pth [2023-12-26 16:24:31,923][105692] Updated weights for policy 0, policy_version 143018 (0.0006) [2023-12-26 16:24:31,946][105620] Updated weights for policy 1, policy_version 143785 (0.0011) [2023-12-26 16:24:31,970][105692] Updated weights for policy 0, policy_version 143028 (0.0007) [2023-12-26 16:24:32,008][105620] Updated weights for policy 1, policy_version 143795 (0.0010) [2023-12-26 16:24:32,022][105692] Updated weights for policy 0, policy_version 143038 (0.0006) [2023-12-26 16:24:32,063][105620] Updated weights for policy 1, policy_version 143805 (0.0010) [2023-12-26 16:24:32,072][105692] Updated weights for policy 0, policy_version 143048 (0.0009) [2023-12-26 16:24:32,123][105620] Updated weights for policy 1, policy_version 143815 (0.0010) [2023-12-26 16:24:32,780][105620] Updated weights for policy 1, policy_version 143825 (0.0007) [2023-12-26 16:24:32,829][105620] Updated weights for policy 1, policy_version 143835 (0.0008) [2023-12-26 16:24:32,877][105620] Updated weights for policy 1, policy_version 143845 (0.0008) [2023-12-26 16:24:32,908][105692] Updated weights for policy 0, policy_version 143058 (0.0008) [2023-12-26 16:24:32,962][105692] Updated weights for policy 0, policy_version 143068 (0.0010) [2023-12-26 16:24:33,020][105692] Updated weights for policy 0, policy_version 143078 (0.0010) [2023-12-26 16:24:33,596][105620] Updated weights for policy 1, policy_version 143855 (0.0006) [2023-12-26 16:24:33,642][105620] Updated weights for policy 1, policy_version 143865 (0.0005) [2023-12-26 16:24:33,691][105692] Updated weights for policy 0, policy_version 143088 (0.0006) [2023-12-26 16:24:33,699][105620] Updated weights for policy 1, policy_version 143875 (0.0005) [2023-12-26 16:24:33,737][105692] Updated weights for policy 0, policy_version 143098 (0.0006) [2023-12-26 16:24:33,793][105692] Updated weights for policy 0, policy_version 143108 (0.0005) [2023-12-26 16:24:34,230][105620] Updated weights for policy 1, policy_version 143885 (0.0007) [2023-12-26 16:24:34,287][105620] Updated weights for policy 1, policy_version 143895 (0.0008) [2023-12-26 16:24:34,341][105620] Updated weights for policy 1, policy_version 143905 (0.0008) [2023-12-26 16:24:34,541][105692] Updated weights for policy 0, policy_version 143118 (0.0006) [2023-12-26 16:24:34,608][105692] Updated weights for policy 0, policy_version 143128 (0.0006) [2023-12-26 16:24:34,675][105692] Updated weights for policy 0, policy_version 143138 (0.0006) [2023-12-26 16:24:35,214][105692] Updated weights for policy 0, policy_version 143148 (0.0006) [2023-12-26 16:24:35,222][105620] Updated weights for policy 1, policy_version 143915 (0.0007) [2023-12-26 16:24:35,270][105692] Updated weights for policy 0, policy_version 143158 (0.0010) [2023-12-26 16:24:35,281][105620] Updated weights for policy 1, policy_version 143925 (0.0009) [2023-12-26 16:24:35,331][105692] Updated weights for policy 0, policy_version 143168 (0.0008) [2023-12-26 16:24:35,333][105620] Updated weights for policy 1, policy_version 143935 (0.0006) [2023-12-26 16:24:36,001][105620] Updated weights for policy 1, policy_version 143945 (0.0006) [2023-12-26 16:24:36,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 73515008. Throughput: 0: 9615.9, 1: 9980.1. Samples: 73508292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:24:36,063][104569] Avg episode reward: [(0, '7394.691'), (1, '8476.477')] [2023-12-26 16:24:36,072][105620] Updated weights for policy 1, policy_version 143955 (0.0005) [2023-12-26 16:24:36,116][105692] Updated weights for policy 0, policy_version 143178 (0.0007) [2023-12-26 16:24:36,146][105620] Updated weights for policy 1, policy_version 143965 (0.0007) [2023-12-26 16:24:36,181][105692] Updated weights for policy 0, policy_version 143188 (0.0008) [2023-12-26 16:24:36,209][105620] Updated weights for policy 1, policy_version 143975 (0.0009) [2023-12-26 16:24:36,246][105692] Updated weights for policy 0, policy_version 143198 (0.0006) [2023-12-26 16:24:36,306][105692] Updated weights for policy 0, policy_version 143208 (0.0008) [2023-12-26 16:24:36,893][105692] Updated weights for policy 0, policy_version 143218 (0.0010) [2023-12-26 16:24:36,902][105620] Updated weights for policy 1, policy_version 143985 (0.0006) [2023-12-26 16:24:36,944][105692] Updated weights for policy 0, policy_version 143228 (0.0010) [2023-12-26 16:24:36,958][105620] Updated weights for policy 1, policy_version 143995 (0.0005) [2023-12-26 16:24:36,992][105692] Updated weights for policy 0, policy_version 143238 (0.0010) [2023-12-26 16:24:37,015][105620] Updated weights for policy 1, policy_version 144005 (0.0006) [2023-12-26 16:24:37,631][105692] Updated weights for policy 0, policy_version 143248 (0.0010) [2023-12-26 16:24:37,636][105620] Updated weights for policy 1, policy_version 144015 (0.0006) [2023-12-26 16:24:37,686][105620] Updated weights for policy 1, policy_version 144025 (0.0005) [2023-12-26 16:24:37,690][105692] Updated weights for policy 0, policy_version 143258 (0.0010) [2023-12-26 16:24:37,740][105620] Updated weights for policy 1, policy_version 144035 (0.0006) [2023-12-26 16:24:37,750][105692] Updated weights for policy 0, policy_version 143268 (0.0011) [2023-12-26 16:24:38,435][105692] Updated weights for policy 0, policy_version 143278 (0.0008) [2023-12-26 16:24:38,470][105620] Updated weights for policy 1, policy_version 144045 (0.0006) [2023-12-26 16:24:38,495][105692] Updated weights for policy 0, policy_version 143288 (0.0005) [2023-12-26 16:24:38,527][105620] Updated weights for policy 1, policy_version 144055 (0.0008) [2023-12-26 16:24:38,555][105692] Updated weights for policy 0, policy_version 143298 (0.0006) [2023-12-26 16:24:38,586][105620] Updated weights for policy 1, policy_version 144065 (0.0008) [2023-12-26 16:24:39,165][105692] Updated weights for policy 0, policy_version 143308 (0.0007) [2023-12-26 16:24:39,234][105692] Updated weights for policy 0, policy_version 143318 (0.0010) [2023-12-26 16:24:39,282][105620] Updated weights for policy 1, policy_version 144075 (0.0007) [2023-12-26 16:24:39,304][105692] Updated weights for policy 0, policy_version 143328 (0.0007) [2023-12-26 16:24:39,346][105620] Updated weights for policy 1, policy_version 144085 (0.0009) [2023-12-26 16:24:39,415][105620] Updated weights for policy 1, policy_version 144095 (0.0009) [2023-12-26 16:24:40,023][105692] Updated weights for policy 0, policy_version 143338 (0.0007) [2023-12-26 16:24:40,088][105692] Updated weights for policy 0, policy_version 143348 (0.0009) [2023-12-26 16:24:40,145][105692] Updated weights for policy 0, policy_version 143358 (0.0008) [2023-12-26 16:24:40,209][105692] Updated weights for policy 0, policy_version 143368 (0.0008) [2023-12-26 16:24:40,218][105620] Updated weights for policy 1, policy_version 144105 (0.0009) [2023-12-26 16:24:40,289][105620] Updated weights for policy 1, policy_version 144115 (0.0011) [2023-12-26 16:24:40,357][105620] Updated weights for policy 1, policy_version 144125 (0.0010) [2023-12-26 16:24:40,420][105620] Updated weights for policy 1, policy_version 144135 (0.0010) [2023-12-26 16:24:40,851][105692] Updated weights for policy 0, policy_version 143378 (0.0008) [2023-12-26 16:24:40,904][105692] Updated weights for policy 0, policy_version 143388 (0.0008) [2023-12-26 16:24:40,966][105692] Updated weights for policy 0, policy_version 143398 (0.0008) [2023-12-26 16:24:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 73621504. Throughput: 0: 9715.5, 1: 9906.8. Samples: 73628524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:24:41,062][104569] Avg episode reward: [(0, '9023.062'), (1, '8799.269')] [2023-12-26 16:24:41,143][105620] Updated weights for policy 1, policy_version 144145 (0.0010) [2023-12-26 16:24:41,196][105620] Updated weights for policy 1, policy_version 144155 (0.0010) [2023-12-26 16:24:41,263][105620] Updated weights for policy 1, policy_version 144165 (0.0010) [2023-12-26 16:24:41,771][105692] Updated weights for policy 0, policy_version 143408 (0.0009) [2023-12-26 16:24:41,837][105692] Updated weights for policy 0, policy_version 143418 (0.0010) [2023-12-26 16:24:41,897][105692] Updated weights for policy 0, policy_version 143428 (0.0011) [2023-12-26 16:24:42,001][105620] Updated weights for policy 1, policy_version 144175 (0.0008) [2023-12-26 16:24:42,065][105620] Updated weights for policy 1, policy_version 144185 (0.0008) [2023-12-26 16:24:42,125][105620] Updated weights for policy 1, policy_version 144195 (0.0006) [2023-12-26 16:24:42,650][105692] Updated weights for policy 0, policy_version 143438 (0.0008) [2023-12-26 16:24:42,707][105692] Updated weights for policy 0, policy_version 143448 (0.0008) [2023-12-26 16:24:42,766][105692] Updated weights for policy 0, policy_version 143458 (0.0008) [2023-12-26 16:24:42,789][105620] Updated weights for policy 1, policy_version 144205 (0.0006) [2023-12-26 16:24:42,844][105620] Updated weights for policy 1, policy_version 144215 (0.0006) [2023-12-26 16:24:42,903][105620] Updated weights for policy 1, policy_version 144225 (0.0006) [2023-12-26 16:24:43,367][105692] Updated weights for policy 0, policy_version 143468 (0.0009) [2023-12-26 16:24:43,424][105692] Updated weights for policy 0, policy_version 143478 (0.0009) [2023-12-26 16:24:43,487][105692] Updated weights for policy 0, policy_version 143488 (0.0008) [2023-12-26 16:24:43,567][105620] Updated weights for policy 1, policy_version 144235 (0.0009) [2023-12-26 16:24:43,618][105620] Updated weights for policy 1, policy_version 144245 (0.0010) [2023-12-26 16:24:43,679][105620] Updated weights for policy 1, policy_version 144255 (0.0010) [2023-12-26 16:24:44,157][105692] Updated weights for policy 0, policy_version 143498 (0.0008) [2023-12-26 16:24:44,214][105692] Updated weights for policy 0, policy_version 143508 (0.0006) [2023-12-26 16:24:44,279][105692] Updated weights for policy 0, policy_version 143518 (0.0005) [2023-12-26 16:24:44,342][105692] Updated weights for policy 0, policy_version 143528 (0.0008) [2023-12-26 16:24:44,435][105620] Updated weights for policy 1, policy_version 144265 (0.0010) [2023-12-26 16:24:44,497][105620] Updated weights for policy 1, policy_version 144275 (0.0008) [2023-12-26 16:24:44,556][105620] Updated weights for policy 1, policy_version 144285 (0.0009) [2023-12-26 16:24:44,619][105620] Updated weights for policy 1, policy_version 144295 (0.0010) [2023-12-26 16:24:45,046][105692] Updated weights for policy 0, policy_version 143538 (0.0008) [2023-12-26 16:24:45,114][105692] Updated weights for policy 0, policy_version 143548 (0.0009) [2023-12-26 16:24:45,178][105692] Updated weights for policy 0, policy_version 143558 (0.0008) [2023-12-26 16:24:45,380][105620] Updated weights for policy 1, policy_version 144305 (0.0006) [2023-12-26 16:24:45,447][105620] Updated weights for policy 1, policy_version 144315 (0.0005) [2023-12-26 16:24:45,494][105620] Updated weights for policy 1, policy_version 144325 (0.0005) [2023-12-26 16:24:45,925][105692] Updated weights for policy 0, policy_version 143568 (0.0008) [2023-12-26 16:24:45,977][105692] Updated weights for policy 0, policy_version 143578 (0.0009) [2023-12-26 16:24:46,038][105692] Updated weights for policy 0, policy_version 143589 (0.0008) [2023-12-26 16:24:46,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 73719808. Throughput: 0: 9705.0, 1: 9928.5. Samples: 73687572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:24:46,062][104569] Avg episode reward: [(0, '9107.396'), (1, '8896.660')] [2023-12-26 16:24:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000143592_36765696.pth... [2023-12-26 16:24:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000142440_36470784.pth [2023-12-26 16:24:46,077][105620] Updated weights for policy 1, policy_version 144335 (0.0005) [2023-12-26 16:24:46,131][105620] Updated weights for policy 1, policy_version 144345 (0.0006) [2023-12-26 16:24:46,182][105620] Updated weights for policy 1, policy_version 144355 (0.0005) [2023-12-26 16:24:46,205][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000144360_36962304.pth... [2023-12-26 16:24:46,208][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000143176_36659200.pth [2023-12-26 16:24:46,818][105692] Updated weights for policy 0, policy_version 143599 (0.0007) [2023-12-26 16:24:46,831][105620] Updated weights for policy 1, policy_version 144365 (0.0008) [2023-12-26 16:24:46,877][105692] Updated weights for policy 0, policy_version 143609 (0.0005) [2023-12-26 16:24:46,887][105620] Updated weights for policy 1, policy_version 144375 (0.0011) [2023-12-26 16:24:46,932][105692] Updated weights for policy 0, policy_version 143619 (0.0007) [2023-12-26 16:24:46,936][105620] Updated weights for policy 1, policy_version 144385 (0.0010) [2023-12-26 16:24:47,591][105692] Updated weights for policy 0, policy_version 143629 (0.0005) [2023-12-26 16:24:47,613][105620] Updated weights for policy 1, policy_version 144395 (0.0009) [2023-12-26 16:24:47,638][105692] Updated weights for policy 0, policy_version 143639 (0.0005) [2023-12-26 16:24:47,668][105620] Updated weights for policy 1, policy_version 144405 (0.0005) [2023-12-26 16:24:47,695][105692] Updated weights for policy 0, policy_version 143649 (0.0005) [2023-12-26 16:24:47,731][105620] Updated weights for policy 1, policy_version 144415 (0.0010) [2023-12-26 16:24:48,320][105692] Updated weights for policy 0, policy_version 143659 (0.0007) [2023-12-26 16:24:48,383][105692] Updated weights for policy 0, policy_version 143669 (0.0009) [2023-12-26 16:24:48,432][105692] Updated weights for policy 0, policy_version 143679 (0.0008) [2023-12-26 16:24:48,446][105620] Updated weights for policy 1, policy_version 144425 (0.0011) [2023-12-26 16:24:48,508][105620] Updated weights for policy 1, policy_version 144435 (0.0009) [2023-12-26 16:24:48,575][105620] Updated weights for policy 1, policy_version 144445 (0.0009) [2023-12-26 16:24:48,638][105620] Updated weights for policy 1, policy_version 144455 (0.0011) [2023-12-26 16:24:49,142][105692] Updated weights for policy 0, policy_version 143689 (0.0006) [2023-12-26 16:24:49,205][105692] Updated weights for policy 0, policy_version 143699 (0.0008) [2023-12-26 16:24:49,270][105692] Updated weights for policy 0, policy_version 143709 (0.0008) [2023-12-26 16:24:49,330][105692] Updated weights for policy 0, policy_version 143719 (0.0008) [2023-12-26 16:24:49,381][105620] Updated weights for policy 1, policy_version 144465 (0.0011) [2023-12-26 16:24:49,438][105620] Updated weights for policy 1, policy_version 144475 (0.0011) [2023-12-26 16:24:49,483][105620] Updated weights for policy 1, policy_version 144485 (0.0010) [2023-12-26 16:24:50,135][105692] Updated weights for policy 0, policy_version 143729 (0.0008) [2023-12-26 16:24:50,198][105692] Updated weights for policy 0, policy_version 143739 (0.0008) [2023-12-26 16:24:50,217][105620] Updated weights for policy 1, policy_version 144495 (0.0011) [2023-12-26 16:24:50,256][105692] Updated weights for policy 0, policy_version 143749 (0.0006) [2023-12-26 16:24:50,277][105620] Updated weights for policy 1, policy_version 144505 (0.0011) [2023-12-26 16:24:50,343][105620] Updated weights for policy 1, policy_version 144515 (0.0008) [2023-12-26 16:24:50,996][105620] Updated weights for policy 1, policy_version 144525 (0.0008) [2023-12-26 16:24:51,060][105620] Updated weights for policy 1, policy_version 144535 (0.0009) [2023-12-26 16:24:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 73809920. Throughput: 0: 9744.3, 1: 9930.4. Samples: 73806656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:24:51,062][104569] Avg episode reward: [(0, '9357.242'), (1, '8811.695')] [2023-12-26 16:24:51,091][105692] Updated weights for policy 0, policy_version 143759 (0.0007) [2023-12-26 16:24:51,119][105620] Updated weights for policy 1, policy_version 144545 (0.0008) [2023-12-26 16:24:51,164][105692] Updated weights for policy 0, policy_version 143769 (0.0009) [2023-12-26 16:24:51,232][105692] Updated weights for policy 0, policy_version 143779 (0.0010) [2023-12-26 16:24:51,828][105620] Updated weights for policy 1, policy_version 144555 (0.0007) [2023-12-26 16:24:51,882][105620] Updated weights for policy 1, policy_version 144565 (0.0007) [2023-12-26 16:24:51,946][105620] Updated weights for policy 1, policy_version 144575 (0.0005) [2023-12-26 16:24:52,032][105692] Updated weights for policy 0, policy_version 143789 (0.0010) [2023-12-26 16:24:52,098][105692] Updated weights for policy 0, policy_version 143799 (0.0010) [2023-12-26 16:24:52,164][105692] Updated weights for policy 0, policy_version 143809 (0.0009) [2023-12-26 16:24:52,586][105620] Updated weights for policy 1, policy_version 144585 (0.0006) [2023-12-26 16:24:52,643][105620] Updated weights for policy 1, policy_version 144595 (0.0008) [2023-12-26 16:24:52,699][105620] Updated weights for policy 1, policy_version 144605 (0.0009) [2023-12-26 16:24:52,767][105620] Updated weights for policy 1, policy_version 144615 (0.0006) [2023-12-26 16:24:52,953][105692] Updated weights for policy 0, policy_version 143819 (0.0009) [2023-12-26 16:24:52,999][105692] Updated weights for policy 0, policy_version 143829 (0.0008) [2023-12-26 16:24:53,051][105692] Updated weights for policy 0, policy_version 143839 (0.0009) [2023-12-26 16:24:53,370][105620] Updated weights for policy 1, policy_version 144625 (0.0005) [2023-12-26 16:24:53,420][105620] Updated weights for policy 1, policy_version 144635 (0.0007) [2023-12-26 16:24:53,473][105620] Updated weights for policy 1, policy_version 144645 (0.0005) [2023-12-26 16:24:53,900][105692] Updated weights for policy 0, policy_version 143849 (0.0010) [2023-12-26 16:24:53,951][105692] Updated weights for policy 0, policy_version 143859 (0.0009) [2023-12-26 16:24:53,999][105692] Updated weights for policy 0, policy_version 143869 (0.0009) [2023-12-26 16:24:54,052][105692] Updated weights for policy 0, policy_version 143880 (0.0010) [2023-12-26 16:24:54,155][105620] Updated weights for policy 1, policy_version 144655 (0.0008) [2023-12-26 16:24:54,206][105620] Updated weights for policy 1, policy_version 144665 (0.0009) [2023-12-26 16:24:54,262][105620] Updated weights for policy 1, policy_version 144675 (0.0009) [2023-12-26 16:24:54,810][105692] Updated weights for policy 0, policy_version 143890 (0.0009) [2023-12-26 16:24:54,858][105692] Updated weights for policy 0, policy_version 143900 (0.0009) [2023-12-26 16:24:54,911][105692] Updated weights for policy 0, policy_version 143910 (0.0009) [2023-12-26 16:24:55,031][105620] Updated weights for policy 1, policy_version 144685 (0.0009) [2023-12-26 16:24:55,079][105620] Updated weights for policy 1, policy_version 144695 (0.0007) [2023-12-26 16:24:55,138][105620] Updated weights for policy 1, policy_version 144705 (0.0005) [2023-12-26 16:24:55,755][105692] Updated weights for policy 0, policy_version 143920 (0.0008) [2023-12-26 16:24:55,757][105620] Updated weights for policy 1, policy_version 144715 (0.0007) [2023-12-26 16:24:55,807][105692] Updated weights for policy 0, policy_version 143930 (0.0006) [2023-12-26 16:24:55,812][105620] Updated weights for policy 1, policy_version 144725 (0.0010) [2023-12-26 16:24:55,860][105692] Updated weights for policy 0, policy_version 143940 (0.0005) [2023-12-26 16:24:55,869][105620] Updated weights for policy 1, policy_version 144735 (0.0010) [2023-12-26 16:24:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 73916416. Throughput: 0: 9592.7, 1: 9905.8. Samples: 73920808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:24:56,063][104569] Avg episode reward: [(0, '9357.181'), (1, '8726.604')] [2023-12-26 16:24:56,503][105692] Updated weights for policy 0, policy_version 143950 (0.0007) [2023-12-26 16:24:56,552][105692] Updated weights for policy 0, policy_version 143960 (0.0008) [2023-12-26 16:24:56,600][105692] Updated weights for policy 0, policy_version 143970 (0.0006) [2023-12-26 16:24:56,605][105620] Updated weights for policy 1, policy_version 144745 (0.0010) [2023-12-26 16:24:56,663][105620] Updated weights for policy 1, policy_version 144755 (0.0010) [2023-12-26 16:24:56,706][105620] Updated weights for policy 1, policy_version 144765 (0.0010) [2023-12-26 16:24:56,750][105620] Updated weights for policy 1, policy_version 144775 (0.0010) [2023-12-26 16:24:57,236][105692] Updated weights for policy 0, policy_version 143980 (0.0007) [2023-12-26 16:24:57,289][105692] Updated weights for policy 0, policy_version 143991 (0.0010) [2023-12-26 16:24:57,336][105620] Updated weights for policy 1, policy_version 144785 (0.0008) [2023-12-26 16:24:57,339][105692] Updated weights for policy 0, policy_version 144001 (0.0007) [2023-12-26 16:24:57,392][105620] Updated weights for policy 1, policy_version 144795 (0.0008) [2023-12-26 16:24:57,457][105620] Updated weights for policy 1, policy_version 144805 (0.0008) [2023-12-26 16:24:58,125][105692] Updated weights for policy 0, policy_version 144011 (0.0007) [2023-12-26 16:24:58,141][105620] Updated weights for policy 1, policy_version 144815 (0.0010) [2023-12-26 16:24:58,185][105692] Updated weights for policy 0, policy_version 144021 (0.0010) [2023-12-26 16:24:58,204][105620] Updated weights for policy 1, policy_version 144825 (0.0010) [2023-12-26 16:24:58,242][105692] Updated weights for policy 0, policy_version 144031 (0.0006) [2023-12-26 16:24:58,263][105620] Updated weights for policy 1, policy_version 144835 (0.0010) [2023-12-26 16:24:58,985][105620] Updated weights for policy 1, policy_version 144845 (0.0009) [2023-12-26 16:24:59,050][105620] Updated weights for policy 1, policy_version 144855 (0.0008) [2023-12-26 16:24:59,051][105692] Updated weights for policy 0, policy_version 144041 (0.0006) [2023-12-26 16:24:59,113][105692] Updated weights for policy 0, policy_version 144051 (0.0008) [2023-12-26 16:24:59,115][105620] Updated weights for policy 1, policy_version 144865 (0.0010) [2023-12-26 16:24:59,176][105692] Updated weights for policy 0, policy_version 144061 (0.0007) [2023-12-26 16:24:59,234][105692] Updated weights for policy 0, policy_version 144071 (0.0008) [2023-12-26 16:24:59,910][105620] Updated weights for policy 1, policy_version 144875 (0.0008) [2023-12-26 16:24:59,921][105692] Updated weights for policy 0, policy_version 144081 (0.0008) [2023-12-26 16:24:59,972][105620] Updated weights for policy 1, policy_version 144885 (0.0007) [2023-12-26 16:24:59,983][105692] Updated weights for policy 0, policy_version 144091 (0.0007) [2023-12-26 16:25:00,034][105620] Updated weights for policy 1, policy_version 144895 (0.0009) [2023-12-26 16:25:00,043][105692] Updated weights for policy 0, policy_version 144101 (0.0006) [2023-12-26 16:25:00,686][105620] Updated weights for policy 1, policy_version 144905 (0.0009) [2023-12-26 16:25:00,744][105620] Updated weights for policy 1, policy_version 144915 (0.0009) [2023-12-26 16:25:00,801][105620] Updated weights for policy 1, policy_version 144925 (0.0009) [2023-12-26 16:25:00,826][105692] Updated weights for policy 0, policy_version 144111 (0.0008) [2023-12-26 16:25:00,863][105620] Updated weights for policy 1, policy_version 144935 (0.0009) [2023-12-26 16:25:00,877][105692] Updated weights for policy 0, policy_version 144121 (0.0005) [2023-12-26 16:25:00,929][105692] Updated weights for policy 0, policy_version 144131 (0.0009) [2023-12-26 16:25:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 74014720. Throughput: 0: 9602.7, 1: 9931.1. Samples: 73980596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:25:01,063][104569] Avg episode reward: [(0, '9356.909'), (1, '8709.449')] [2023-12-26 16:25:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000144136_36904960.pth... [2023-12-26 16:25:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000144936_37109760.pth... [2023-12-26 16:25:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000143784_36814848.pth [2023-12-26 16:25:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000143016_36618240.pth [2023-12-26 16:25:01,620][105620] Updated weights for policy 1, policy_version 144945 (0.0009) [2023-12-26 16:25:01,681][105620] Updated weights for policy 1, policy_version 144955 (0.0007) [2023-12-26 16:25:01,703][105692] Updated weights for policy 0, policy_version 144141 (0.0010) [2023-12-26 16:25:01,748][105620] Updated weights for policy 1, policy_version 144965 (0.0006) [2023-12-26 16:25:01,768][105692] Updated weights for policy 0, policy_version 144151 (0.0010) [2023-12-26 16:25:01,821][105692] Updated weights for policy 0, policy_version 144161 (0.0010) [2023-12-26 16:25:02,493][105620] Updated weights for policy 1, policy_version 144975 (0.0008) [2023-12-26 16:25:02,543][105620] Updated weights for policy 1, policy_version 144985 (0.0008) [2023-12-26 16:25:02,603][105692] Updated weights for policy 0, policy_version 144171 (0.0010) [2023-12-26 16:25:02,605][105620] Updated weights for policy 1, policy_version 144995 (0.0008) [2023-12-26 16:25:02,655][105692] Updated weights for policy 0, policy_version 144181 (0.0007) [2023-12-26 16:25:02,706][105692] Updated weights for policy 0, policy_version 144191 (0.0007) [2023-12-26 16:25:03,318][105692] Updated weights for policy 0, policy_version 144201 (0.0005) [2023-12-26 16:25:03,376][105692] Updated weights for policy 0, policy_version 144211 (0.0005) [2023-12-26 16:25:03,428][105692] Updated weights for policy 0, policy_version 144221 (0.0005) [2023-12-26 16:25:03,429][105620] Updated weights for policy 1, policy_version 145005 (0.0007) [2023-12-26 16:25:03,484][105692] Updated weights for policy 0, policy_version 144231 (0.0005) [2023-12-26 16:25:03,485][105620] Updated weights for policy 1, policy_version 145015 (0.0009) [2023-12-26 16:25:03,537][105620] Updated weights for policy 1, policy_version 145025 (0.0010) [2023-12-26 16:25:04,020][105692] Updated weights for policy 0, policy_version 144241 (0.0005) [2023-12-26 16:25:04,078][105692] Updated weights for policy 0, policy_version 144251 (0.0006) [2023-12-26 16:25:04,141][105692] Updated weights for policy 0, policy_version 144261 (0.0006) [2023-12-26 16:25:04,402][105620] Updated weights for policy 1, policy_version 145036 (0.0011) [2023-12-26 16:25:04,465][105620] Updated weights for policy 1, policy_version 145046 (0.0007) [2023-12-26 16:25:04,526][105620] Updated weights for policy 1, policy_version 145056 (0.0007) [2023-12-26 16:25:04,747][105692] Updated weights for policy 0, policy_version 144271 (0.0009) [2023-12-26 16:25:04,807][105692] Updated weights for policy 0, policy_version 144281 (0.0010) [2023-12-26 16:25:04,863][105692] Updated weights for policy 0, policy_version 144291 (0.0010) [2023-12-26 16:25:05,343][105620] Updated weights for policy 1, policy_version 145066 (0.0008) [2023-12-26 16:25:05,395][105620] Updated weights for policy 1, policy_version 145076 (0.0008) [2023-12-26 16:25:05,448][105692] Updated weights for policy 0, policy_version 144301 (0.0011) [2023-12-26 16:25:05,453][105620] Updated weights for policy 1, policy_version 145086 (0.0007) [2023-12-26 16:25:05,497][105692] Updated weights for policy 0, policy_version 144311 (0.0008) [2023-12-26 16:25:05,506][105620] Updated weights for policy 1, policy_version 145096 (0.0007) [2023-12-26 16:25:05,549][105692] Updated weights for policy 0, policy_version 144321 (0.0005) [2023-12-26 16:25:06,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 74104832. Throughput: 0: 9608.3, 1: 9879.8. Samples: 74094752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:25:06,062][104569] Avg episode reward: [(0, '9356.309'), (1, '8794.551')] [2023-12-26 16:25:06,196][105692] Updated weights for policy 0, policy_version 144331 (0.0009) [2023-12-26 16:25:06,247][105692] Updated weights for policy 0, policy_version 144341 (0.0006) [2023-12-26 16:25:06,302][105692] Updated weights for policy 0, policy_version 144351 (0.0007) [2023-12-26 16:25:06,365][105620] Updated weights for policy 1, policy_version 145106 (0.0008) [2023-12-26 16:25:06,429][105620] Updated weights for policy 1, policy_version 145116 (0.0009) [2023-12-26 16:25:06,494][105620] Updated weights for policy 1, policy_version 145126 (0.0009) [2023-12-26 16:25:06,904][105692] Updated weights for policy 0, policy_version 144361 (0.0006) [2023-12-26 16:25:06,971][105692] Updated weights for policy 0, policy_version 144371 (0.0006) [2023-12-26 16:25:07,040][105692] Updated weights for policy 0, policy_version 144381 (0.0005) [2023-12-26 16:25:07,108][105692] Updated weights for policy 0, policy_version 144391 (0.0007) [2023-12-26 16:25:07,354][105620] Updated weights for policy 1, policy_version 145136 (0.0008) [2023-12-26 16:25:07,409][105620] Updated weights for policy 1, policy_version 145146 (0.0008) [2023-12-26 16:25:07,461][105620] Updated weights for policy 1, policy_version 145156 (0.0008) [2023-12-26 16:25:07,742][105692] Updated weights for policy 0, policy_version 144401 (0.0010) [2023-12-26 16:25:07,785][105692] Updated weights for policy 0, policy_version 144411 (0.0007) [2023-12-26 16:25:07,832][105692] Updated weights for policy 0, policy_version 144421 (0.0010) [2023-12-26 16:25:08,201][105620] Updated weights for policy 1, policy_version 145166 (0.0006) [2023-12-26 16:25:08,258][105620] Updated weights for policy 1, policy_version 145176 (0.0006) [2023-12-26 16:25:08,310][105620] Updated weights for policy 1, policy_version 145186 (0.0007) [2023-12-26 16:25:08,608][105692] Updated weights for policy 0, policy_version 144431 (0.0011) [2023-12-26 16:25:08,671][105692] Updated weights for policy 0, policy_version 144441 (0.0011) [2023-12-26 16:25:08,733][105692] Updated weights for policy 0, policy_version 144451 (0.0010) [2023-12-26 16:25:09,055][105620] Updated weights for policy 1, policy_version 145196 (0.0008) [2023-12-26 16:25:09,121][105620] Updated weights for policy 1, policy_version 145206 (0.0008) [2023-12-26 16:25:09,176][105620] Updated weights for policy 1, policy_version 145216 (0.0008) [2023-12-26 16:25:09,465][105692] Updated weights for policy 0, policy_version 144461 (0.0011) [2023-12-26 16:25:09,516][105692] Updated weights for policy 0, policy_version 144471 (0.0010) [2023-12-26 16:25:09,575][105692] Updated weights for policy 0, policy_version 144481 (0.0011) [2023-12-26 16:25:09,895][105620] Updated weights for policy 1, policy_version 145226 (0.0008) [2023-12-26 16:25:09,955][105620] Updated weights for policy 1, policy_version 145236 (0.0008) [2023-12-26 16:25:10,024][105620] Updated weights for policy 1, policy_version 145246 (0.0008) [2023-12-26 16:25:10,088][105620] Updated weights for policy 1, policy_version 145256 (0.0008) [2023-12-26 16:25:10,304][105692] Updated weights for policy 0, policy_version 144491 (0.0009) [2023-12-26 16:25:10,362][105692] Updated weights for policy 0, policy_version 144501 (0.0006) [2023-12-26 16:25:10,416][105692] Updated weights for policy 0, policy_version 144511 (0.0009) [2023-12-26 16:25:10,917][105620] Updated weights for policy 1, policy_version 145266 (0.0009) [2023-12-26 16:25:10,982][105620] Updated weights for policy 1, policy_version 145276 (0.0008) [2023-12-26 16:25:11,049][105620] Updated weights for policy 1, policy_version 145286 (0.0007) [2023-12-26 16:25:11,062][104569] Fps is (10 sec: 18022.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 74194944. Throughput: 0: 9724.5, 1: 9727.0. Samples: 74209352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:25:11,062][104569] Avg episode reward: [(0, '8915.020'), (1, '8870.359')] [2023-12-26 16:25:11,114][105692] Updated weights for policy 0, policy_version 144521 (0.0009) [2023-12-26 16:25:11,186][105692] Updated weights for policy 0, policy_version 144531 (0.0009) [2023-12-26 16:25:11,248][105692] Updated weights for policy 0, policy_version 144541 (0.0010) [2023-12-26 16:25:11,314][105692] Updated weights for policy 0, policy_version 144551 (0.0008) [2023-12-26 16:25:11,854][105620] Updated weights for policy 1, policy_version 145296 (0.0009) [2023-12-26 16:25:11,907][105620] Updated weights for policy 1, policy_version 145306 (0.0009) [2023-12-26 16:25:11,960][105620] Updated weights for policy 1, policy_version 145316 (0.0009) [2023-12-26 16:25:12,121][105692] Updated weights for policy 0, policy_version 144561 (0.0008) [2023-12-26 16:25:12,181][105692] Updated weights for policy 0, policy_version 144571 (0.0008) [2023-12-26 16:25:12,237][105692] Updated weights for policy 0, policy_version 144581 (0.0008) [2023-12-26 16:25:12,767][105620] Updated weights for policy 1, policy_version 145326 (0.0010) [2023-12-26 16:25:12,829][105620] Updated weights for policy 1, policy_version 145336 (0.0010) [2023-12-26 16:25:12,885][105620] Updated weights for policy 1, policy_version 145346 (0.0010) [2023-12-26 16:25:13,034][105692] Updated weights for policy 0, policy_version 144591 (0.0010) [2023-12-26 16:25:13,085][105692] Updated weights for policy 0, policy_version 144601 (0.0010) [2023-12-26 16:25:13,129][105692] Updated weights for policy 0, policy_version 144611 (0.0010) [2023-12-26 16:25:13,525][105620] Updated weights for policy 1, policy_version 145356 (0.0010) [2023-12-26 16:25:13,573][105620] Updated weights for policy 1, policy_version 145366 (0.0010) [2023-12-26 16:25:13,620][105620] Updated weights for policy 1, policy_version 145376 (0.0010) [2023-12-26 16:25:13,828][105692] Updated weights for policy 0, policy_version 144621 (0.0010) [2023-12-26 16:25:13,876][105692] Updated weights for policy 0, policy_version 144631 (0.0010) [2023-12-26 16:25:13,924][105692] Updated weights for policy 0, policy_version 144641 (0.0010) [2023-12-26 16:25:14,347][105620] Updated weights for policy 1, policy_version 145386 (0.0010) [2023-12-26 16:25:14,392][105620] Updated weights for policy 1, policy_version 145396 (0.0010) [2023-12-26 16:25:14,436][105620] Updated weights for policy 1, policy_version 145406 (0.0010) [2023-12-26 16:25:14,490][105620] Updated weights for policy 1, policy_version 145416 (0.0009) [2023-12-26 16:25:14,631][105692] Updated weights for policy 0, policy_version 144651 (0.0009) [2023-12-26 16:25:14,699][105692] Updated weights for policy 0, policy_version 144661 (0.0007) [2023-12-26 16:25:14,777][105692] Updated weights for policy 0, policy_version 144671 (0.0009) [2023-12-26 16:25:15,251][105620] Updated weights for policy 1, policy_version 145426 (0.0006) [2023-12-26 16:25:15,313][105620] Updated weights for policy 1, policy_version 145436 (0.0010) [2023-12-26 16:25:15,375][105620] Updated weights for policy 1, policy_version 145446 (0.0010) [2023-12-26 16:25:15,468][105692] Updated weights for policy 0, policy_version 144681 (0.0009) [2023-12-26 16:25:15,521][105692] Updated weights for policy 0, policy_version 144691 (0.0008) [2023-12-26 16:25:15,578][105692] Updated weights for policy 0, policy_version 144701 (0.0010) [2023-12-26 16:25:15,643][105692] Updated weights for policy 0, policy_version 144711 (0.0007) [2023-12-26 16:25:16,042][105620] Updated weights for policy 1, policy_version 145456 (0.0007) [2023-12-26 16:25:16,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 74293248. Throughput: 0: 9743.3, 1: 9637.9. Samples: 74265260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:25:16,063][104569] Avg episode reward: [(0, '8913.918'), (1, '9062.023')] [2023-12-26 16:25:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000144712_37052416.pth... [2023-12-26 16:25:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000143592_36765696.pth [2023-12-26 16:25:16,101][105620] Updated weights for policy 1, policy_version 145466 (0.0005) [2023-12-26 16:25:16,162][105620] Updated weights for policy 1, policy_version 145476 (0.0008) [2023-12-26 16:25:16,178][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000145480_37249024.pth... [2023-12-26 16:25:16,182][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000144360_36962304.pth [2023-12-26 16:25:16,269][105692] Updated weights for policy 0, policy_version 144721 (0.0010) [2023-12-26 16:25:16,317][105692] Updated weights for policy 0, policy_version 144731 (0.0010) [2023-12-26 16:25:16,367][105692] Updated weights for policy 0, policy_version 144741 (0.0010) [2023-12-26 16:25:16,803][105620] Updated weights for policy 1, policy_version 145486 (0.0010) [2023-12-26 16:25:16,855][105620] Updated weights for policy 1, policy_version 145496 (0.0010) [2023-12-26 16:25:16,911][105620] Updated weights for policy 1, policy_version 145506 (0.0010) [2023-12-26 16:25:17,121][105692] Updated weights for policy 0, policy_version 144751 (0.0009) [2023-12-26 16:25:17,176][105692] Updated weights for policy 0, policy_version 144761 (0.0008) [2023-12-26 16:25:17,225][105692] Updated weights for policy 0, policy_version 144771 (0.0008) [2023-12-26 16:25:17,627][105620] Updated weights for policy 1, policy_version 145516 (0.0008) [2023-12-26 16:25:17,682][105620] Updated weights for policy 1, policy_version 145526 (0.0008) [2023-12-26 16:25:17,741][105620] Updated weights for policy 1, policy_version 145536 (0.0010) [2023-12-26 16:25:17,957][105692] Updated weights for policy 0, policy_version 144781 (0.0007) [2023-12-26 16:25:18,005][105692] Updated weights for policy 0, policy_version 144791 (0.0010) [2023-12-26 16:25:18,054][105692] Updated weights for policy 0, policy_version 144801 (0.0010) [2023-12-26 16:25:18,297][105620] Updated weights for policy 1, policy_version 145546 (0.0005) [2023-12-26 16:25:18,351][105620] Updated weights for policy 1, policy_version 145556 (0.0007) [2023-12-26 16:25:18,413][105620] Updated weights for policy 1, policy_version 145566 (0.0010) [2023-12-26 16:25:18,465][105620] Updated weights for policy 1, policy_version 145576 (0.0010) [2023-12-26 16:25:18,720][105692] Updated weights for policy 0, policy_version 144811 (0.0010) [2023-12-26 16:25:18,784][105692] Updated weights for policy 0, policy_version 144821 (0.0009) [2023-12-26 16:25:18,847][105692] Updated weights for policy 0, policy_version 144831 (0.0010) [2023-12-26 16:25:19,181][105620] Updated weights for policy 1, policy_version 145586 (0.0008) [2023-12-26 16:25:19,249][105620] Updated weights for policy 1, policy_version 145596 (0.0007) [2023-12-26 16:25:19,303][105620] Updated weights for policy 1, policy_version 145606 (0.0008) [2023-12-26 16:25:19,597][105692] Updated weights for policy 0, policy_version 144841 (0.0011) [2023-12-26 16:25:19,645][105692] Updated weights for policy 0, policy_version 144851 (0.0010) [2023-12-26 16:25:19,704][105692] Updated weights for policy 0, policy_version 144861 (0.0010) [2023-12-26 16:25:19,763][105692] Updated weights for policy 0, policy_version 144871 (0.0010) [2023-12-26 16:25:20,085][105620] Updated weights for policy 1, policy_version 145616 (0.0008) [2023-12-26 16:25:20,152][105620] Updated weights for policy 1, policy_version 145626 (0.0008) [2023-12-26 16:25:20,203][105620] Updated weights for policy 1, policy_version 145636 (0.0008) [2023-12-26 16:25:20,515][105692] Updated weights for policy 0, policy_version 144881 (0.0009) [2023-12-26 16:25:20,573][105692] Updated weights for policy 0, policy_version 144891 (0.0009) [2023-12-26 16:25:20,627][105692] Updated weights for policy 0, policy_version 144901 (0.0007) [2023-12-26 16:25:21,015][105620] Updated weights for policy 1, policy_version 145646 (0.0009) [2023-12-26 16:25:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 74391552. Throughput: 0: 9812.4, 1: 9678.9. Samples: 74385404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:25:21,063][104569] Avg episode reward: [(0, '9090.404'), (1, '8985.260')] [2023-12-26 16:25:21,082][105620] Updated weights for policy 1, policy_version 145656 (0.0010) [2023-12-26 16:25:21,145][105620] Updated weights for policy 1, policy_version 145666 (0.0009) [2023-12-26 16:25:21,365][105692] Updated weights for policy 0, policy_version 144911 (0.0008) [2023-12-26 16:25:21,432][105692] Updated weights for policy 0, policy_version 144921 (0.0009) [2023-12-26 16:25:21,495][105692] Updated weights for policy 0, policy_version 144931 (0.0010) [2023-12-26 16:25:21,959][105620] Updated weights for policy 1, policy_version 145676 (0.0009) [2023-12-26 16:25:22,014][105620] Updated weights for policy 1, policy_version 145686 (0.0009) [2023-12-26 16:25:22,076][105620] Updated weights for policy 1, policy_version 145696 (0.0009) [2023-12-26 16:25:22,240][105692] Updated weights for policy 0, policy_version 144941 (0.0007) [2023-12-26 16:25:22,303][105692] Updated weights for policy 0, policy_version 144951 (0.0008) [2023-12-26 16:25:22,362][105692] Updated weights for policy 0, policy_version 144961 (0.0008) [2023-12-26 16:25:22,786][105620] Updated weights for policy 1, policy_version 145706 (0.0008) [2023-12-26 16:25:22,853][105620] Updated weights for policy 1, policy_version 145716 (0.0008) [2023-12-26 16:25:22,916][105620] Updated weights for policy 1, policy_version 145726 (0.0006) [2023-12-26 16:25:22,968][105620] Updated weights for policy 1, policy_version 145736 (0.0008) [2023-12-26 16:25:23,119][105692] Updated weights for policy 0, policy_version 144971 (0.0009) [2023-12-26 16:25:23,167][105692] Updated weights for policy 0, policy_version 144981 (0.0009) [2023-12-26 16:25:23,216][105692] Updated weights for policy 0, policy_version 144991 (0.0008) [2023-12-26 16:25:23,691][105620] Updated weights for policy 1, policy_version 145746 (0.0010) [2023-12-26 16:25:23,753][105620] Updated weights for policy 1, policy_version 145756 (0.0010) [2023-12-26 16:25:23,820][105620] Updated weights for policy 1, policy_version 145766 (0.0010) [2023-12-26 16:25:23,894][105692] Updated weights for policy 0, policy_version 145001 (0.0009) [2023-12-26 16:25:23,953][105692] Updated weights for policy 0, policy_version 145011 (0.0009) [2023-12-26 16:25:24,005][105692] Updated weights for policy 0, policy_version 145021 (0.0010) [2023-12-26 16:25:24,062][105692] Updated weights for policy 0, policy_version 145031 (0.0010) [2023-12-26 16:25:24,475][105620] Updated weights for policy 1, policy_version 145776 (0.0010) [2023-12-26 16:25:24,520][105620] Updated weights for policy 1, policy_version 145786 (0.0010) [2023-12-26 16:25:24,577][105620] Updated weights for policy 1, policy_version 145796 (0.0010) [2023-12-26 16:25:24,782][105692] Updated weights for policy 0, policy_version 145041 (0.0006) [2023-12-26 16:25:24,846][105692] Updated weights for policy 0, policy_version 145051 (0.0005) [2023-12-26 16:25:24,895][105692] Updated weights for policy 0, policy_version 145061 (0.0010) [2023-12-26 16:25:25,171][105620] Updated weights for policy 1, policy_version 145806 (0.0007) [2023-12-26 16:25:25,214][105620] Updated weights for policy 1, policy_version 145816 (0.0005) [2023-12-26 16:25:25,262][105620] Updated weights for policy 1, policy_version 145826 (0.0005) [2023-12-26 16:25:25,541][105692] Updated weights for policy 0, policy_version 145071 (0.0010) [2023-12-26 16:25:25,590][105692] Updated weights for policy 0, policy_version 145081 (0.0009) [2023-12-26 16:25:25,642][105692] Updated weights for policy 0, policy_version 145091 (0.0010) [2023-12-26 16:25:25,896][105620] Updated weights for policy 1, policy_version 145836 (0.0006) [2023-12-26 16:25:25,965][105620] Updated weights for policy 1, policy_version 145846 (0.0007) [2023-12-26 16:25:26,013][105620] Updated weights for policy 1, policy_version 145856 (0.0006) [2023-12-26 16:25:26,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 74498048. Throughput: 0: 9726.2, 1: 9694.4. Samples: 74502452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:25:26,063][104569] Avg episode reward: [(0, '9263.816'), (1, '8717.132')] [2023-12-26 16:25:26,295][105692] Updated weights for policy 0, policy_version 145101 (0.0008) [2023-12-26 16:25:26,355][105692] Updated weights for policy 0, policy_version 145111 (0.0008) [2023-12-26 16:25:26,358][105585] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000004 [2023-12-26 16:25:26,647][105620] Updated weights for policy 1, policy_version 145866 (0.0006) [2023-12-26 16:25:26,702][105620] Updated weights for policy 1, policy_version 145876 (0.0009) [2023-12-26 16:25:26,764][105620] Updated weights for policy 1, policy_version 145886 (0.0009) [2023-12-26 16:25:26,830][105620] Updated weights for policy 1, policy_version 145896 (0.0008) [2023-12-26 16:25:27,111][105692] Updated weights for policy 0, policy_version 145121 (0.0010) [2023-12-26 16:25:27,169][105692] Updated weights for policy 0, policy_version 145131 (0.0010) [2023-12-26 16:25:27,234][105692] Updated weights for policy 0, policy_version 145141 (0.0007) [2023-12-26 16:25:27,534][105620] Updated weights for policy 1, policy_version 145906 (0.0005) [2023-12-26 16:25:27,580][105620] Updated weights for policy 1, policy_version 145916 (0.0005) [2023-12-26 16:25:27,632][105620] Updated weights for policy 1, policy_version 145926 (0.0005) [2023-12-26 16:25:27,932][105692] Updated weights for policy 0, policy_version 145151 (0.0007) [2023-12-26 16:25:27,987][105692] Updated weights for policy 0, policy_version 145161 (0.0008) [2023-12-26 16:25:28,039][105692] Updated weights for policy 0, policy_version 145171 (0.0010) [2023-12-26 16:25:28,283][105620] Updated weights for policy 1, policy_version 145936 (0.0008) [2023-12-26 16:25:28,342][105620] Updated weights for policy 1, policy_version 145946 (0.0008) [2023-12-26 16:25:28,405][105620] Updated weights for policy 1, policy_version 145956 (0.0008) [2023-12-26 16:25:28,755][105692] Updated weights for policy 0, policy_version 145181 (0.0010) [2023-12-26 16:25:28,813][105692] Updated weights for policy 0, policy_version 145191 (0.0010) [2023-12-26 16:25:28,871][105692] Updated weights for policy 0, policy_version 145201 (0.0010) [2023-12-26 16:25:29,155][105620] Updated weights for policy 1, policy_version 145966 (0.0009) [2023-12-26 16:25:29,212][105620] Updated weights for policy 1, policy_version 145976 (0.0010) [2023-12-26 16:25:29,275][105620] Updated weights for policy 1, policy_version 145986 (0.0009) [2023-12-26 16:25:29,529][105692] Updated weights for policy 0, policy_version 145211 (0.0007) [2023-12-26 16:25:29,595][105692] Updated weights for policy 0, policy_version 145221 (0.0010) [2023-12-26 16:25:29,646][105692] Updated weights for policy 0, policy_version 145231 (0.0010) [2023-12-26 16:25:30,052][105620] Updated weights for policy 1, policy_version 145996 (0.0007) [2023-12-26 16:25:30,116][105620] Updated weights for policy 1, policy_version 146006 (0.0008) [2023-12-26 16:25:30,171][105620] Updated weights for policy 1, policy_version 146016 (0.0008) [2023-12-26 16:25:30,392][105692] Updated weights for policy 0, policy_version 145241 (0.0010) [2023-12-26 16:25:30,440][105692] Updated weights for policy 0, policy_version 145251 (0.0010) [2023-12-26 16:25:30,491][105692] Updated weights for policy 0, policy_version 145261 (0.0010) [2023-12-26 16:25:30,535][105692] Updated weights for policy 0, policy_version 145271 (0.0010) [2023-12-26 16:25:30,971][105620] Updated weights for policy 1, policy_version 146026 (0.0008) [2023-12-26 16:25:31,034][105620] Updated weights for policy 1, policy_version 146036 (0.0010) [2023-12-26 16:25:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 74588160. Throughput: 0: 9754.2, 1: 9707.7. Samples: 74563360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:25:31,063][104569] Avg episode reward: [(0, '9353.633'), (1, '8899.487')] [2023-12-26 16:25:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000145272_37199872.pth... [2023-12-26 16:25:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000144136_36904960.pth [2023-12-26 16:25:31,097][105620] Updated weights for policy 1, policy_version 146046 (0.0008) [2023-12-26 16:25:31,160][105692] Updated weights for policy 0, policy_version 145281 (0.0010) [2023-12-26 16:25:31,162][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000146056_37396480.pth... [2023-12-26 16:25:31,164][105620] Updated weights for policy 1, policy_version 146056 (0.0008) [2023-12-26 16:25:31,167][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000144936_37109760.pth [2023-12-26 16:25:31,222][105692] Updated weights for policy 0, policy_version 145291 (0.0010) [2023-12-26 16:25:31,287][105692] Updated weights for policy 0, policy_version 145301 (0.0009) [2023-12-26 16:25:31,929][105620] Updated weights for policy 1, policy_version 146066 (0.0008) [2023-12-26 16:25:31,983][105692] Updated weights for policy 0, policy_version 145311 (0.0009) [2023-12-26 16:25:31,985][105620] Updated weights for policy 1, policy_version 146076 (0.0008) [2023-12-26 16:25:32,037][105620] Updated weights for policy 1, policy_version 146086 (0.0006) [2023-12-26 16:25:32,038][105692] Updated weights for policy 0, policy_version 145321 (0.0010) [2023-12-26 16:25:32,100][105692] Updated weights for policy 0, policy_version 145331 (0.0010) [2023-12-26 16:25:32,754][105620] Updated weights for policy 1, policy_version 146096 (0.0006) [2023-12-26 16:25:32,783][105692] Updated weights for policy 0, policy_version 145341 (0.0008) [2023-12-26 16:25:32,820][105620] Updated weights for policy 1, policy_version 146106 (0.0005) [2023-12-26 16:25:32,852][105692] Updated weights for policy 0, policy_version 145351 (0.0009) [2023-12-26 16:25:32,873][105620] Updated weights for policy 1, policy_version 146116 (0.0010) [2023-12-26 16:25:32,915][105692] Updated weights for policy 0, policy_version 145361 (0.0007) [2023-12-26 16:25:33,484][105620] Updated weights for policy 1, policy_version 146126 (0.0007) [2023-12-26 16:25:33,543][105620] Updated weights for policy 1, policy_version 146136 (0.0006) [2023-12-26 16:25:33,602][105620] Updated weights for policy 1, policy_version 146146 (0.0007) [2023-12-26 16:25:33,672][105692] Updated weights for policy 0, policy_version 145371 (0.0007) [2023-12-26 16:25:33,726][105692] Updated weights for policy 0, policy_version 145383 (0.0010) [2023-12-26 16:25:33,795][105692] Updated weights for policy 0, policy_version 145393 (0.0010) [2023-12-26 16:25:34,177][105620] Updated weights for policy 1, policy_version 146156 (0.0007) [2023-12-26 16:25:34,229][105620] Updated weights for policy 1, policy_version 146166 (0.0007) [2023-12-26 16:25:34,293][105620] Updated weights for policy 1, policy_version 146176 (0.0006) [2023-12-26 16:25:34,575][105692] Updated weights for policy 0, policy_version 145403 (0.0010) [2023-12-26 16:25:34,639][105692] Updated weights for policy 0, policy_version 145413 (0.0008) [2023-12-26 16:25:34,703][105692] Updated weights for policy 0, policy_version 145423 (0.0008) [2023-12-26 16:25:34,969][105620] Updated weights for policy 1, policy_version 146186 (0.0007) [2023-12-26 16:25:35,021][105620] Updated weights for policy 1, policy_version 146196 (0.0010) [2023-12-26 16:25:35,069][105620] Updated weights for policy 1, policy_version 146206 (0.0010) [2023-12-26 16:25:35,122][105620] Updated weights for policy 1, policy_version 146216 (0.0010) [2023-12-26 16:25:35,390][105692] Updated weights for policy 0, policy_version 145433 (0.0008) [2023-12-26 16:25:35,454][105692] Updated weights for policy 0, policy_version 145443 (0.0006) [2023-12-26 16:25:35,512][105692] Updated weights for policy 0, policy_version 145453 (0.0009) [2023-12-26 16:25:35,571][105692] Updated weights for policy 0, policy_version 145463 (0.0008) [2023-12-26 16:25:35,880][105620] Updated weights for policy 1, policy_version 146226 (0.0010) [2023-12-26 16:25:35,932][105620] Updated weights for policy 1, policy_version 146236 (0.0010) [2023-12-26 16:25:35,988][105620] Updated weights for policy 1, policy_version 146246 (0.0010) [2023-12-26 16:25:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19605.2). Total num frames: 74694656. Throughput: 0: 9751.3, 1: 9689.9. Samples: 74681512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:25:36,063][104569] Avg episode reward: [(0, '9352.485'), (1, '8991.474')] [2023-12-26 16:25:36,316][105692] Updated weights for policy 0, policy_version 145473 (0.0008) [2023-12-26 16:25:36,377][105692] Updated weights for policy 0, policy_version 145483 (0.0008) [2023-12-26 16:25:36,440][105692] Updated weights for policy 0, policy_version 145493 (0.0008) [2023-12-26 16:25:36,698][105620] Updated weights for policy 1, policy_version 146256 (0.0007) [2023-12-26 16:25:36,757][105620] Updated weights for policy 1, policy_version 146266 (0.0010) [2023-12-26 16:25:36,820][105620] Updated weights for policy 1, policy_version 146276 (0.0011) [2023-12-26 16:25:37,137][105692] Updated weights for policy 0, policy_version 145503 (0.0006) [2023-12-26 16:25:37,190][105692] Updated weights for policy 0, policy_version 145513 (0.0008) [2023-12-26 16:25:37,239][105692] Updated weights for policy 0, policy_version 145523 (0.0010) [2023-12-26 16:25:37,542][105620] Updated weights for policy 1, policy_version 146286 (0.0010) [2023-12-26 16:25:37,603][105620] Updated weights for policy 1, policy_version 146296 (0.0010) [2023-12-26 16:25:37,667][105620] Updated weights for policy 1, policy_version 146306 (0.0011) [2023-12-26 16:25:37,908][105692] Updated weights for policy 0, policy_version 145533 (0.0008) [2023-12-26 16:25:37,966][105692] Updated weights for policy 0, policy_version 145543 (0.0008) [2023-12-26 16:25:38,022][105692] Updated weights for policy 0, policy_version 145553 (0.0011) [2023-12-26 16:25:38,403][105620] Updated weights for policy 1, policy_version 146316 (0.0010) [2023-12-26 16:25:38,455][105620] Updated weights for policy 1, policy_version 146326 (0.0010) [2023-12-26 16:25:38,507][105620] Updated weights for policy 1, policy_version 146336 (0.0010) [2023-12-26 16:25:38,757][105692] Updated weights for policy 0, policy_version 145563 (0.0010) [2023-12-26 16:25:38,805][105692] Updated weights for policy 0, policy_version 145573 (0.0008) [2023-12-26 16:25:38,862][105692] Updated weights for policy 0, policy_version 145583 (0.0009) [2023-12-26 16:25:39,270][105620] Updated weights for policy 1, policy_version 146346 (0.0010) [2023-12-26 16:25:39,331][105620] Updated weights for policy 1, policy_version 146356 (0.0009) [2023-12-26 16:25:39,395][105620] Updated weights for policy 1, policy_version 146366 (0.0007) [2023-12-26 16:25:39,456][105620] Updated weights for policy 1, policy_version 146376 (0.0010) [2023-12-26 16:25:39,704][105692] Updated weights for policy 0, policy_version 145593 (0.0009) [2023-12-26 16:25:39,766][105692] Updated weights for policy 0, policy_version 145603 (0.0010) [2023-12-26 16:25:39,824][105692] Updated weights for policy 0, policy_version 145613 (0.0008) [2023-12-26 16:25:39,888][105692] Updated weights for policy 0, policy_version 145623 (0.0009) [2023-12-26 16:25:40,161][105620] Updated weights for policy 1, policy_version 146386 (0.0010) [2023-12-26 16:25:40,224][105620] Updated weights for policy 1, policy_version 146396 (0.0010) [2023-12-26 16:25:40,283][105620] Updated weights for policy 1, policy_version 146406 (0.0010) [2023-12-26 16:25:40,679][105692] Updated weights for policy 0, policy_version 145633 (0.0010) [2023-12-26 16:25:40,735][105692] Updated weights for policy 0, policy_version 145643 (0.0011) [2023-12-26 16:25:40,786][105692] Updated weights for policy 0, policy_version 145653 (0.0011) [2023-12-26 16:25:40,894][105620] Updated weights for policy 1, policy_version 146416 (0.0009) [2023-12-26 16:25:40,945][105620] Updated weights for policy 1, policy_version 146426 (0.0005) [2023-12-26 16:25:40,996][105620] Updated weights for policy 1, policy_version 146436 (0.0005) [2023-12-26 16:25:41,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 74792960. Throughput: 0: 9832.6, 1: 9625.5. Samples: 74796416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:25:41,062][104569] Avg episode reward: [(0, '9352.371'), (1, '9257.324')] [2023-12-26 16:25:41,546][105692] Updated weights for policy 0, policy_version 145663 (0.0007) [2023-12-26 16:25:41,604][105692] Updated weights for policy 0, policy_version 145673 (0.0008) [2023-12-26 16:25:41,672][105692] Updated weights for policy 0, policy_version 145683 (0.0009) [2023-12-26 16:25:41,783][105620] Updated weights for policy 1, policy_version 146446 (0.0008) [2023-12-26 16:25:41,838][105620] Updated weights for policy 1, policy_version 146456 (0.0008) [2023-12-26 16:25:41,898][105620] Updated weights for policy 1, policy_version 146466 (0.0005) [2023-12-26 16:25:42,485][105692] Updated weights for policy 0, policy_version 145693 (0.0009) [2023-12-26 16:25:42,542][105692] Updated weights for policy 0, policy_version 145703 (0.0010) [2023-12-26 16:25:42,561][105620] Updated weights for policy 1, policy_version 146476 (0.0006) [2023-12-26 16:25:42,602][105692] Updated weights for policy 0, policy_version 145713 (0.0007) [2023-12-26 16:25:42,627][105620] Updated weights for policy 1, policy_version 146486 (0.0008) [2023-12-26 16:25:42,683][105620] Updated weights for policy 1, policy_version 146496 (0.0009) [2023-12-26 16:25:43,342][105620] Updated weights for policy 1, policy_version 146506 (0.0008) [2023-12-26 16:25:43,408][105620] Updated weights for policy 1, policy_version 146516 (0.0005) [2023-12-26 16:25:43,419][105692] Updated weights for policy 0, policy_version 145723 (0.0006) [2023-12-26 16:25:43,471][105692] Updated weights for policy 0, policy_version 145733 (0.0007) [2023-12-26 16:25:43,472][105620] Updated weights for policy 1, policy_version 146526 (0.0008) [2023-12-26 16:25:43,520][105692] Updated weights for policy 0, policy_version 145743 (0.0006) [2023-12-26 16:25:43,529][105620] Updated weights for policy 1, policy_version 146536 (0.0007) [2023-12-26 16:25:44,152][105620] Updated weights for policy 1, policy_version 146546 (0.0008) [2023-12-26 16:25:44,206][105620] Updated weights for policy 1, policy_version 146556 (0.0008) [2023-12-26 16:25:44,265][105620] Updated weights for policy 1, policy_version 146566 (0.0008) [2023-12-26 16:25:44,296][105692] Updated weights for policy 0, policy_version 145753 (0.0009) [2023-12-26 16:25:44,346][105692] Updated weights for policy 0, policy_version 145763 (0.0010) [2023-12-26 16:25:44,394][105692] Updated weights for policy 0, policy_version 145773 (0.0010) [2023-12-26 16:25:44,459][105692] Updated weights for policy 0, policy_version 145783 (0.0007) [2023-12-26 16:25:45,021][105620] Updated weights for policy 1, policy_version 146576 (0.0008) [2023-12-26 16:25:45,093][105620] Updated weights for policy 1, policy_version 146586 (0.0009) [2023-12-26 16:25:45,155][105620] Updated weights for policy 1, policy_version 146596 (0.0008) [2023-12-26 16:25:45,210][105692] Updated weights for policy 0, policy_version 145793 (0.0009) [2023-12-26 16:25:45,264][105692] Updated weights for policy 0, policy_version 145803 (0.0009) [2023-12-26 16:25:45,323][105692] Updated weights for policy 0, policy_version 145813 (0.0009) [2023-12-26 16:25:45,891][105620] Updated weights for policy 1, policy_version 146606 (0.0006) [2023-12-26 16:25:45,960][105620] Updated weights for policy 1, policy_version 146616 (0.0005) [2023-12-26 16:25:46,028][105620] Updated weights for policy 1, policy_version 146626 (0.0005) [2023-12-26 16:25:46,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19251.1, 300 sec: 19549.7). Total num frames: 74874880. Throughput: 0: 9772.7, 1: 9626.3. Samples: 74853552. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:25:46,063][104569] Avg episode reward: [(0, '9268.513'), (1, '9254.762')] [2023-12-26 16:25:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000145816_37339136.pth... [2023-12-26 16:25:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000146632_37543936.pth... [2023-12-26 16:25:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000145480_37249024.pth [2023-12-26 16:25:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000144712_37052416.pth [2023-12-26 16:25:46,152][105692] Updated weights for policy 0, policy_version 145823 (0.0009) [2023-12-26 16:25:46,204][105692] Updated weights for policy 0, policy_version 145833 (0.0009) [2023-12-26 16:25:46,250][105692] Updated weights for policy 0, policy_version 145843 (0.0008) [2023-12-26 16:25:46,552][105620] Updated weights for policy 1, policy_version 146636 (0.0005) [2023-12-26 16:25:46,606][105620] Updated weights for policy 1, policy_version 146646 (0.0006) [2023-12-26 16:25:46,663][105620] Updated weights for policy 1, policy_version 146656 (0.0008) [2023-12-26 16:25:46,968][105692] Updated weights for policy 0, policy_version 145853 (0.0009) [2023-12-26 16:25:47,030][105692] Updated weights for policy 0, policy_version 145863 (0.0006) [2023-12-26 16:25:47,091][105692] Updated weights for policy 0, policy_version 145873 (0.0008) [2023-12-26 16:25:47,260][105620] Updated weights for policy 1, policy_version 146666 (0.0006) [2023-12-26 16:25:47,331][105620] Updated weights for policy 1, policy_version 146676 (0.0009) [2023-12-26 16:25:47,384][105620] Updated weights for policy 1, policy_version 146686 (0.0009) [2023-12-26 16:25:47,440][105620] Updated weights for policy 1, policy_version 146696 (0.0008) [2023-12-26 16:25:47,773][105692] Updated weights for policy 0, policy_version 145883 (0.0008) [2023-12-26 16:25:47,818][105692] Updated weights for policy 0, policy_version 145893 (0.0005) [2023-12-26 16:25:47,863][105692] Updated weights for policy 0, policy_version 145903 (0.0005) [2023-12-26 16:25:48,204][105620] Updated weights for policy 1, policy_version 146706 (0.0009) [2023-12-26 16:25:48,263][105620] Updated weights for policy 1, policy_version 146716 (0.0009) [2023-12-26 16:25:48,319][105620] Updated weights for policy 1, policy_version 146726 (0.0010) [2023-12-26 16:25:48,477][105692] Updated weights for policy 0, policy_version 145913 (0.0005) [2023-12-26 16:25:48,544][105692] Updated weights for policy 0, policy_version 145923 (0.0006) [2023-12-26 16:25:48,610][105692] Updated weights for policy 0, policy_version 145933 (0.0005) [2023-12-26 16:25:48,671][105692] Updated weights for policy 0, policy_version 145943 (0.0006) [2023-12-26 16:25:49,166][105620] Updated weights for policy 1, policy_version 146736 (0.0008) [2023-12-26 16:25:49,217][105620] Updated weights for policy 1, policy_version 146746 (0.0009) [2023-12-26 16:25:49,280][105620] Updated weights for policy 1, policy_version 146756 (0.0007) [2023-12-26 16:25:49,290][105692] Updated weights for policy 0, policy_version 145953 (0.0009) [2023-12-26 16:25:49,355][105692] Updated weights for policy 0, policy_version 145963 (0.0008) [2023-12-26 16:25:49,414][105692] Updated weights for policy 0, policy_version 145973 (0.0009) [2023-12-26 16:25:50,064][105620] Updated weights for policy 1, policy_version 146766 (0.0007) [2023-12-26 16:25:50,120][105620] Updated weights for policy 1, policy_version 146776 (0.0007) [2023-12-26 16:25:50,180][105692] Updated weights for policy 0, policy_version 145983 (0.0009) [2023-12-26 16:25:50,183][105620] Updated weights for policy 1, policy_version 146786 (0.0007) [2023-12-26 16:25:50,228][105692] Updated weights for policy 0, policy_version 145993 (0.0009) [2023-12-26 16:25:50,280][105692] Updated weights for policy 0, policy_version 146003 (0.0010) [2023-12-26 16:25:50,952][105620] Updated weights for policy 1, policy_version 146796 (0.0008) [2023-12-26 16:25:51,014][105620] Updated weights for policy 1, policy_version 146806 (0.0009) [2023-12-26 16:25:51,028][105692] Updated weights for policy 0, policy_version 146013 (0.0010) [2023-12-26 16:25:51,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 74973184. Throughput: 0: 9743.8, 1: 9716.3. Samples: 74970460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:25:51,062][104569] Avg episode reward: [(0, '9353.624'), (1, '9254.377')] [2023-12-26 16:25:51,084][105620] Updated weights for policy 1, policy_version 146816 (0.0008) [2023-12-26 16:25:51,092][105692] Updated weights for policy 0, policy_version 146023 (0.0008) [2023-12-26 16:25:51,160][105692] Updated weights for policy 0, policy_version 146033 (0.0009) [2023-12-26 16:25:51,854][105620] Updated weights for policy 1, policy_version 146826 (0.0007) [2023-12-26 16:25:51,866][105692] Updated weights for policy 0, policy_version 146043 (0.0010) [2023-12-26 16:25:51,906][105620] Updated weights for policy 1, policy_version 146836 (0.0007) [2023-12-26 16:25:51,926][105692] Updated weights for policy 0, policy_version 146053 (0.0011) [2023-12-26 16:25:51,963][105620] Updated weights for policy 1, policy_version 146846 (0.0008) [2023-12-26 16:25:51,986][105692] Updated weights for policy 0, policy_version 146063 (0.0011) [2023-12-26 16:25:52,020][105620] Updated weights for policy 1, policy_version 146856 (0.0006) [2023-12-26 16:25:52,626][105620] Updated weights for policy 1, policy_version 146866 (0.0008) [2023-12-26 16:25:52,682][105620] Updated weights for policy 1, policy_version 146876 (0.0008) [2023-12-26 16:25:52,707][105692] Updated weights for policy 0, policy_version 146073 (0.0011) [2023-12-26 16:25:52,737][105620] Updated weights for policy 1, policy_version 146886 (0.0007) [2023-12-26 16:25:52,766][105692] Updated weights for policy 0, policy_version 146083 (0.0011) [2023-12-26 16:25:52,821][105692] Updated weights for policy 0, policy_version 146093 (0.0010) [2023-12-26 16:25:52,873][105692] Updated weights for policy 0, policy_version 146103 (0.0010) [2023-12-26 16:25:53,492][105620] Updated weights for policy 1, policy_version 146896 (0.0008) [2023-12-26 16:25:53,545][105620] Updated weights for policy 1, policy_version 146906 (0.0008) [2023-12-26 16:25:53,594][105692] Updated weights for policy 0, policy_version 146113 (0.0007) [2023-12-26 16:25:53,600][105620] Updated weights for policy 1, policy_version 146916 (0.0007) [2023-12-26 16:25:53,653][105692] Updated weights for policy 0, policy_version 146123 (0.0008) [2023-12-26 16:25:53,705][105692] Updated weights for policy 0, policy_version 146133 (0.0009) [2023-12-26 16:25:54,334][105620] Updated weights for policy 1, policy_version 146926 (0.0006) [2023-12-26 16:25:54,383][105620] Updated weights for policy 1, policy_version 146936 (0.0011) [2023-12-26 16:25:54,413][105692] Updated weights for policy 0, policy_version 146143 (0.0008) [2023-12-26 16:25:54,442][105620] Updated weights for policy 1, policy_version 146946 (0.0010) [2023-12-26 16:25:54,468][105692] Updated weights for policy 0, policy_version 146153 (0.0006) [2023-12-26 16:25:54,519][105692] Updated weights for policy 0, policy_version 146163 (0.0008) [2023-12-26 16:25:55,170][105620] Updated weights for policy 1, policy_version 146956 (0.0010) [2023-12-26 16:25:55,238][105620] Updated weights for policy 1, policy_version 146966 (0.0010) [2023-12-26 16:25:55,244][105692] Updated weights for policy 0, policy_version 146173 (0.0007) [2023-12-26 16:25:55,287][105620] Updated weights for policy 1, policy_version 146976 (0.0010) [2023-12-26 16:25:55,300][105692] Updated weights for policy 0, policy_version 146183 (0.0005) [2023-12-26 16:25:55,354][105692] Updated weights for policy 0, policy_version 146193 (0.0005) [2023-12-26 16:25:55,950][105692] Updated weights for policy 0, policy_version 146203 (0.0006) [2023-12-26 16:25:55,960][105620] Updated weights for policy 1, policy_version 146986 (0.0010) [2023-12-26 16:25:56,002][105692] Updated weights for policy 0, policy_version 146213 (0.0006) [2023-12-26 16:25:56,023][105620] Updated weights for policy 1, policy_version 146996 (0.0010) [2023-12-26 16:25:56,060][105692] Updated weights for policy 0, policy_version 146223 (0.0006) [2023-12-26 16:25:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 75071488. Throughput: 0: 9684.1, 1: 9824.0. Samples: 75087220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:25:56,063][104569] Avg episode reward: [(0, '9354.291'), (1, '9253.926')] [2023-12-26 16:25:56,079][105620] Updated weights for policy 1, policy_version 147006 (0.0010) [2023-12-26 16:25:56,130][105620] Updated weights for policy 1, policy_version 147016 (0.0010) [2023-12-26 16:25:56,758][105620] Updated weights for policy 1, policy_version 147026 (0.0008) [2023-12-26 16:25:56,821][105620] Updated weights for policy 1, policy_version 147036 (0.0008) [2023-12-26 16:25:56,836][105692] Updated weights for policy 0, policy_version 146233 (0.0005) [2023-12-26 16:25:56,866][105620] Updated weights for policy 1, policy_version 147046 (0.0007) [2023-12-26 16:25:56,896][105692] Updated weights for policy 0, policy_version 146243 (0.0008) [2023-12-26 16:25:56,957][105692] Updated weights for policy 0, policy_version 146253 (0.0007) [2023-12-26 16:25:57,014][105692] Updated weights for policy 0, policy_version 146263 (0.0009) [2023-12-26 16:25:57,568][105620] Updated weights for policy 1, policy_version 147056 (0.0009) [2023-12-26 16:25:57,628][105620] Updated weights for policy 1, policy_version 147066 (0.0010) [2023-12-26 16:25:57,658][105692] Updated weights for policy 0, policy_version 146273 (0.0006) [2023-12-26 16:25:57,682][105620] Updated weights for policy 1, policy_version 147076 (0.0010) [2023-12-26 16:25:57,709][105692] Updated weights for policy 0, policy_version 146283 (0.0008) [2023-12-26 16:25:57,753][105692] Updated weights for policy 0, policy_version 146293 (0.0007) [2023-12-26 16:25:58,517][105620] Updated weights for policy 1, policy_version 147087 (0.0009) [2023-12-26 16:25:58,548][105692] Updated weights for policy 0, policy_version 146303 (0.0008) [2023-12-26 16:25:58,580][105620] Updated weights for policy 1, policy_version 147097 (0.0008) [2023-12-26 16:25:58,616][105692] Updated weights for policy 0, policy_version 146313 (0.0008) [2023-12-26 16:25:58,644][105620] Updated weights for policy 1, policy_version 147107 (0.0008) [2023-12-26 16:25:58,683][105692] Updated weights for policy 0, policy_version 146323 (0.0009) [2023-12-26 16:25:59,438][105692] Updated weights for policy 0, policy_version 146333 (0.0008) [2023-12-26 16:25:59,469][105620] Updated weights for policy 1, policy_version 147117 (0.0008) [2023-12-26 16:25:59,495][105692] Updated weights for policy 0, policy_version 146343 (0.0010) [2023-12-26 16:25:59,525][105620] Updated weights for policy 1, policy_version 147127 (0.0007) [2023-12-26 16:25:59,551][105692] Updated weights for policy 0, policy_version 146353 (0.0006) [2023-12-26 16:25:59,583][105620] Updated weights for policy 1, policy_version 147137 (0.0007) [2023-12-26 16:26:00,294][105692] Updated weights for policy 0, policy_version 146363 (0.0008) [2023-12-26 16:26:00,302][105620] Updated weights for policy 1, policy_version 147147 (0.0008) [2023-12-26 16:26:00,348][105692] Updated weights for policy 0, policy_version 146373 (0.0009) [2023-12-26 16:26:00,362][105620] Updated weights for policy 1, policy_version 147157 (0.0008) [2023-12-26 16:26:00,398][105692] Updated weights for policy 0, policy_version 146383 (0.0008) [2023-12-26 16:26:00,422][105620] Updated weights for policy 1, policy_version 147167 (0.0008) [2023-12-26 16:26:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 75169792. Throughput: 0: 9717.7, 1: 9836.7. Samples: 75145204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:26:01,063][104569] Avg episode reward: [(0, '9354.447'), (1, '9069.346')] [2023-12-26 16:26:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000146392_37486592.pth... [2023-12-26 16:26:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000147176_37683200.pth... [2023-12-26 16:26:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000145272_37199872.pth [2023-12-26 16:26:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000146056_37396480.pth [2023-12-26 16:26:01,163][105692] Updated weights for policy 0, policy_version 146393 (0.0007) [2023-12-26 16:26:01,191][105620] Updated weights for policy 1, policy_version 147177 (0.0008) [2023-12-26 16:26:01,225][105692] Updated weights for policy 0, policy_version 146403 (0.0007) [2023-12-26 16:26:01,245][105620] Updated weights for policy 1, policy_version 147187 (0.0008) [2023-12-26 16:26:01,288][105692] Updated weights for policy 0, policy_version 146413 (0.0009) [2023-12-26 16:26:01,312][105620] Updated weights for policy 1, policy_version 147197 (0.0008) [2023-12-26 16:26:01,344][105692] Updated weights for policy 0, policy_version 146423 (0.0008) [2023-12-26 16:26:01,379][105620] Updated weights for policy 1, policy_version 147207 (0.0009) [2023-12-26 16:26:02,067][105692] Updated weights for policy 0, policy_version 146433 (0.0008) [2023-12-26 16:26:02,104][105620] Updated weights for policy 1, policy_version 147217 (0.0010) [2023-12-26 16:26:02,122][105692] Updated weights for policy 0, policy_version 146443 (0.0007) [2023-12-26 16:26:02,166][105620] Updated weights for policy 1, policy_version 147227 (0.0009) [2023-12-26 16:26:02,181][105692] Updated weights for policy 0, policy_version 146453 (0.0005) [2023-12-26 16:26:02,217][105620] Updated weights for policy 1, policy_version 147237 (0.0010) [2023-12-26 16:26:02,782][105692] Updated weights for policy 0, policy_version 146463 (0.0005) [2023-12-26 16:26:02,835][105692] Updated weights for policy 0, policy_version 146473 (0.0005) [2023-12-26 16:26:02,893][105692] Updated weights for policy 0, policy_version 146483 (0.0005) [2023-12-26 16:26:02,985][105620] Updated weights for policy 1, policy_version 147247 (0.0009) [2023-12-26 16:26:03,042][105620] Updated weights for policy 1, policy_version 147257 (0.0009) [2023-12-26 16:26:03,096][105620] Updated weights for policy 1, policy_version 147267 (0.0009) [2023-12-26 16:26:03,415][105692] Updated weights for policy 0, policy_version 146493 (0.0006) [2023-12-26 16:26:03,470][105692] Updated weights for policy 0, policy_version 146503 (0.0005) [2023-12-26 16:26:03,534][105692] Updated weights for policy 0, policy_version 146513 (0.0005) [2023-12-26 16:26:03,842][105620] Updated weights for policy 1, policy_version 147277 (0.0007) [2023-12-26 16:26:03,908][105620] Updated weights for policy 1, policy_version 147287 (0.0008) [2023-12-26 16:26:03,973][105620] Updated weights for policy 1, policy_version 147297 (0.0008) [2023-12-26 16:26:04,113][105692] Updated weights for policy 0, policy_version 146523 (0.0007) [2023-12-26 16:26:04,182][105692] Updated weights for policy 0, policy_version 146533 (0.0011) [2023-12-26 16:26:04,247][105692] Updated weights for policy 0, policy_version 146543 (0.0010) [2023-12-26 16:26:04,694][105620] Updated weights for policy 1, policy_version 147307 (0.0008) [2023-12-26 16:26:04,759][105620] Updated weights for policy 1, policy_version 147317 (0.0011) [2023-12-26 16:26:04,827][105620] Updated weights for policy 1, policy_version 147327 (0.0011) [2023-12-26 16:26:04,983][105692] Updated weights for policy 0, policy_version 146553 (0.0011) [2023-12-26 16:26:05,041][105692] Updated weights for policy 0, policy_version 146563 (0.0010) [2023-12-26 16:26:05,089][105692] Updated weights for policy 0, policy_version 146573 (0.0010) [2023-12-26 16:26:05,151][105692] Updated weights for policy 0, policy_version 146583 (0.0010) [2023-12-26 16:26:05,558][105620] Updated weights for policy 1, policy_version 147337 (0.0011) [2023-12-26 16:26:05,627][105620] Updated weights for policy 1, policy_version 147347 (0.0011) [2023-12-26 16:26:05,683][105620] Updated weights for policy 1, policy_version 147357 (0.0010) [2023-12-26 16:26:05,731][105620] Updated weights for policy 1, policy_version 147367 (0.0010) [2023-12-26 16:26:05,801][105692] Updated weights for policy 0, policy_version 146593 (0.0008) [2023-12-26 16:26:05,845][105692] Updated weights for policy 0, policy_version 146603 (0.0008) [2023-12-26 16:26:05,894][105692] Updated weights for policy 0, policy_version 146613 (0.0008) [2023-12-26 16:26:06,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 75276288. Throughput: 0: 9753.3, 1: 9729.6. Samples: 75262136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:26:06,063][104569] Avg episode reward: [(0, '9354.238'), (1, '9161.831')] [2023-12-26 16:26:06,461][105620] Updated weights for policy 1, policy_version 147377 (0.0006) [2023-12-26 16:26:06,526][105620] Updated weights for policy 1, policy_version 147387 (0.0006) [2023-12-26 16:26:06,598][105620] Updated weights for policy 1, policy_version 147397 (0.0006) [2023-12-26 16:26:06,663][105692] Updated weights for policy 0, policy_version 146623 (0.0009) [2023-12-26 16:26:06,722][105692] Updated weights for policy 0, policy_version 146633 (0.0009) [2023-12-26 16:26:06,787][105692] Updated weights for policy 0, policy_version 146643 (0.0009) [2023-12-26 16:26:07,301][105620] Updated weights for policy 1, policy_version 147407 (0.0006) [2023-12-26 16:26:07,358][105620] Updated weights for policy 1, policy_version 147417 (0.0006) [2023-12-26 16:26:07,404][105620] Updated weights for policy 1, policy_version 147427 (0.0008) [2023-12-26 16:26:07,563][105692] Updated weights for policy 0, policy_version 146653 (0.0009) [2023-12-26 16:26:07,616][105692] Updated weights for policy 0, policy_version 146663 (0.0011) [2023-12-26 16:26:07,663][105692] Updated weights for policy 0, policy_version 146673 (0.0009) [2023-12-26 16:26:08,076][105620] Updated weights for policy 1, policy_version 147437 (0.0009) [2023-12-26 16:26:08,128][105620] Updated weights for policy 1, policy_version 147447 (0.0008) [2023-12-26 16:26:08,183][105620] Updated weights for policy 1, policy_version 147457 (0.0008) [2023-12-26 16:26:08,388][105692] Updated weights for policy 0, policy_version 146683 (0.0006) [2023-12-26 16:26:08,453][105692] Updated weights for policy 0, policy_version 146693 (0.0005) [2023-12-26 16:26:08,517][105692] Updated weights for policy 0, policy_version 146703 (0.0008) [2023-12-26 16:26:08,856][105620] Updated weights for policy 1, policy_version 147467 (0.0008) [2023-12-26 16:26:08,918][105620] Updated weights for policy 1, policy_version 147477 (0.0008) [2023-12-26 16:26:08,985][105620] Updated weights for policy 1, policy_version 147487 (0.0008) [2023-12-26 16:26:09,237][105692] Updated weights for policy 0, policy_version 146713 (0.0010) [2023-12-26 16:26:09,302][105692] Updated weights for policy 0, policy_version 146723 (0.0008) [2023-12-26 16:26:09,372][105692] Updated weights for policy 0, policy_version 146733 (0.0009) [2023-12-26 16:26:09,437][105692] Updated weights for policy 0, policy_version 146743 (0.0010) [2023-12-26 16:26:09,714][105620] Updated weights for policy 1, policy_version 147497 (0.0009) [2023-12-26 16:26:09,771][105620] Updated weights for policy 1, policy_version 147507 (0.0009) [2023-12-26 16:26:09,827][105620] Updated weights for policy 1, policy_version 147517 (0.0009) [2023-12-26 16:26:09,890][105620] Updated weights for policy 1, policy_version 147527 (0.0008) [2023-12-26 16:26:10,091][105692] Updated weights for policy 0, policy_version 146753 (0.0009) [2023-12-26 16:26:10,144][105692] Updated weights for policy 0, policy_version 146763 (0.0009) [2023-12-26 16:26:10,197][105692] Updated weights for policy 0, policy_version 146773 (0.0009) [2023-12-26 16:26:10,693][105620] Updated weights for policy 1, policy_version 147537 (0.0009) [2023-12-26 16:26:10,757][105620] Updated weights for policy 1, policy_version 147547 (0.0008) [2023-12-26 16:26:10,823][105620] Updated weights for policy 1, policy_version 147557 (0.0008) [2023-12-26 16:26:10,966][105692] Updated weights for policy 0, policy_version 146783 (0.0009) [2023-12-26 16:26:11,026][105692] Updated weights for policy 0, policy_version 146793 (0.0010) [2023-12-26 16:26:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 75366400. Throughput: 0: 9740.2, 1: 9690.8. Samples: 75376844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:26:11,063][104569] Avg episode reward: [(0, '9354.334'), (1, '9162.481')] [2023-12-26 16:26:11,085][105692] Updated weights for policy 0, policy_version 146803 (0.0008) [2023-12-26 16:26:11,529][105620] Updated weights for policy 1, policy_version 147567 (0.0009) [2023-12-26 16:26:11,586][105620] Updated weights for policy 1, policy_version 147577 (0.0005) [2023-12-26 16:26:11,657][105620] Updated weights for policy 1, policy_version 147587 (0.0008) [2023-12-26 16:26:11,842][105692] Updated weights for policy 0, policy_version 146813 (0.0006) [2023-12-26 16:26:11,914][105692] Updated weights for policy 0, policy_version 146823 (0.0006) [2023-12-26 16:26:11,976][105692] Updated weights for policy 0, policy_version 146833 (0.0006) [2023-12-26 16:26:12,474][105620] Updated weights for policy 1, policy_version 147597 (0.0007) [2023-12-26 16:26:12,528][105620] Updated weights for policy 1, policy_version 147607 (0.0006) [2023-12-26 16:26:12,590][105620] Updated weights for policy 1, policy_version 147617 (0.0005) [2023-12-26 16:26:12,623][105692] Updated weights for policy 0, policy_version 146843 (0.0007) [2023-12-26 16:26:12,687][105692] Updated weights for policy 0, policy_version 146853 (0.0009) [2023-12-26 16:26:12,750][105692] Updated weights for policy 0, policy_version 146863 (0.0010) [2023-12-26 16:26:13,163][105620] Updated weights for policy 1, policy_version 147627 (0.0006) [2023-12-26 16:26:13,215][105620] Updated weights for policy 1, policy_version 147637 (0.0006) [2023-12-26 16:26:13,277][105620] Updated weights for policy 1, policy_version 147647 (0.0005) [2023-12-26 16:26:13,618][105692] Updated weights for policy 0, policy_version 146873 (0.0010) [2023-12-26 16:26:13,669][105692] Updated weights for policy 0, policy_version 146883 (0.0009) [2023-12-26 16:26:13,717][105692] Updated weights for policy 0, policy_version 146893 (0.0009) [2023-12-26 16:26:13,775][105692] Updated weights for policy 0, policy_version 146903 (0.0009) [2023-12-26 16:26:13,868][105620] Updated weights for policy 1, policy_version 147657 (0.0006) [2023-12-26 16:26:13,926][105620] Updated weights for policy 1, policy_version 147668 (0.0010) [2023-12-26 16:26:13,978][105620] Updated weights for policy 1, policy_version 147678 (0.0009) [2023-12-26 16:26:14,397][105692] Updated weights for policy 0, policy_version 146913 (0.0006) [2023-12-26 16:26:14,444][105692] Updated weights for policy 0, policy_version 146923 (0.0007) [2023-12-26 16:26:14,498][105692] Updated weights for policy 0, policy_version 146935 (0.0010) [2023-12-26 16:26:14,635][105620] Updated weights for policy 1, policy_version 147689 (0.0009) [2023-12-26 16:26:14,690][105620] Updated weights for policy 1, policy_version 147699 (0.0009) [2023-12-26 16:26:14,752][105620] Updated weights for policy 1, policy_version 147709 (0.0009) [2023-12-26 16:26:14,826][105620] Updated weights for policy 1, policy_version 147719 (0.0010) [2023-12-26 16:26:15,221][105692] Updated weights for policy 0, policy_version 146945 (0.0008) [2023-12-26 16:26:15,281][105692] Updated weights for policy 0, policy_version 146955 (0.0006) [2023-12-26 16:26:15,348][105692] Updated weights for policy 0, policy_version 146965 (0.0006) [2023-12-26 16:26:15,689][105620] Updated weights for policy 1, policy_version 147729 (0.0006) [2023-12-26 16:26:15,755][105620] Updated weights for policy 1, policy_version 147739 (0.0005) [2023-12-26 16:26:15,818][105620] Updated weights for policy 1, policy_version 147749 (0.0005) [2023-12-26 16:26:16,048][105692] Updated weights for policy 0, policy_version 146975 (0.0009) [2023-12-26 16:26:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 75464704. Throughput: 0: 9681.3, 1: 9705.0. Samples: 75435744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:26:16,063][104569] Avg episode reward: [(0, '9354.777'), (1, '8888.450')] [2023-12-26 16:26:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000147752_37830656.pth... [2023-12-26 16:26:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000146632_37543936.pth [2023-12-26 16:26:16,103][105692] Updated weights for policy 0, policy_version 146985 (0.0010) [2023-12-26 16:26:16,164][105692] Updated weights for policy 0, policy_version 146996 (0.0008) [2023-12-26 16:26:16,187][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000147000_37642240.pth... [2023-12-26 16:26:16,190][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000145816_37339136.pth [2023-12-26 16:26:16,335][105620] Updated weights for policy 1, policy_version 147759 (0.0006) [2023-12-26 16:26:16,387][105620] Updated weights for policy 1, policy_version 147769 (0.0005) [2023-12-26 16:26:16,443][105620] Updated weights for policy 1, policy_version 147779 (0.0005) [2023-12-26 16:26:16,922][105692] Updated weights for policy 0, policy_version 147006 (0.0009) [2023-12-26 16:26:16,958][105620] Updated weights for policy 1, policy_version 147789 (0.0006) [2023-12-26 16:26:16,971][105692] Updated weights for policy 0, policy_version 147016 (0.0008) [2023-12-26 16:26:17,010][105620] Updated weights for policy 1, policy_version 147799 (0.0005) [2023-12-26 16:26:17,017][105692] Updated weights for policy 0, policy_version 147026 (0.0009) [2023-12-26 16:26:17,057][105620] Updated weights for policy 1, policy_version 147809 (0.0006) [2023-12-26 16:26:17,732][105620] Updated weights for policy 1, policy_version 147819 (0.0009) [2023-12-26 16:26:17,781][105620] Updated weights for policy 1, policy_version 147829 (0.0009) [2023-12-26 16:26:17,821][105692] Updated weights for policy 0, policy_version 147036 (0.0006) [2023-12-26 16:26:17,836][105620] Updated weights for policy 1, policy_version 147839 (0.0008) [2023-12-26 16:26:17,881][105692] Updated weights for policy 0, policy_version 147046 (0.0008) [2023-12-26 16:26:17,940][105692] Updated weights for policy 0, policy_version 147056 (0.0005) [2023-12-26 16:26:18,511][105620] Updated weights for policy 1, policy_version 147849 (0.0008) [2023-12-26 16:26:18,538][105692] Updated weights for policy 0, policy_version 147066 (0.0006) [2023-12-26 16:26:18,579][105620] Updated weights for policy 1, policy_version 147859 (0.0008) [2023-12-26 16:26:18,586][105692] Updated weights for policy 0, policy_version 147076 (0.0006) [2023-12-26 16:26:18,646][105692] Updated weights for policy 0, policy_version 147086 (0.0006) [2023-12-26 16:26:18,646][105620] Updated weights for policy 1, policy_version 147869 (0.0009) [2023-12-26 16:26:18,702][105692] Updated weights for policy 0, policy_version 147096 (0.0006) [2023-12-26 16:26:18,718][105620] Updated weights for policy 1, policy_version 147879 (0.0009) [2023-12-26 16:26:19,362][105692] Updated weights for policy 0, policy_version 147106 (0.0010) [2023-12-26 16:26:19,431][105692] Updated weights for policy 0, policy_version 147116 (0.0010) [2023-12-26 16:26:19,498][105620] Updated weights for policy 1, policy_version 147889 (0.0007) [2023-12-26 16:26:19,499][105692] Updated weights for policy 0, policy_version 147126 (0.0009) [2023-12-26 16:26:19,559][105620] Updated weights for policy 1, policy_version 147899 (0.0008) [2023-12-26 16:26:19,616][105620] Updated weights for policy 1, policy_version 147909 (0.0006) [2023-12-26 16:26:20,149][105692] Updated weights for policy 0, policy_version 147136 (0.0010) [2023-12-26 16:26:20,202][105692] Updated weights for policy 0, policy_version 147146 (0.0011) [2023-12-26 16:26:20,264][105692] Updated weights for policy 0, policy_version 147156 (0.0011) [2023-12-26 16:26:20,432][105620] Updated weights for policy 1, policy_version 147919 (0.0010) [2023-12-26 16:26:20,495][105620] Updated weights for policy 1, policy_version 147929 (0.0007) [2023-12-26 16:26:20,557][105620] Updated weights for policy 1, policy_version 147939 (0.0006) [2023-12-26 16:26:20,970][105692] Updated weights for policy 0, policy_version 147166 (0.0011) [2023-12-26 16:26:21,031][105692] Updated weights for policy 0, policy_version 147176 (0.0008) [2023-12-26 16:26:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 75563008. Throughput: 0: 9702.6, 1: 9755.1. Samples: 75557104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:26:21,062][104569] Avg episode reward: [(0, '9017.654'), (1, '6438.631')] [2023-12-26 16:26:21,097][105692] Updated weights for policy 0, policy_version 147186 (0.0010) [2023-12-26 16:26:21,279][105620] Updated weights for policy 1, policy_version 147949 (0.0008) [2023-12-26 16:26:21,339][105620] Updated weights for policy 1, policy_version 147959 (0.0009) [2023-12-26 16:26:21,403][105620] Updated weights for policy 1, policy_version 147969 (0.0009) [2023-12-26 16:26:21,873][105692] Updated weights for policy 0, policy_version 147196 (0.0008) [2023-12-26 16:26:21,939][105692] Updated weights for policy 0, policy_version 147206 (0.0009) [2023-12-26 16:26:21,994][105692] Updated weights for policy 0, policy_version 147216 (0.0009) [2023-12-26 16:26:22,148][105620] Updated weights for policy 1, policy_version 147979 (0.0009) [2023-12-26 16:26:22,196][105620] Updated weights for policy 1, policy_version 147989 (0.0009) [2023-12-26 16:26:22,246][105620] Updated weights for policy 1, policy_version 147999 (0.0009) [2023-12-26 16:26:22,814][105692] Updated weights for policy 0, policy_version 147226 (0.0009) [2023-12-26 16:26:22,872][105692] Updated weights for policy 0, policy_version 147236 (0.0009) [2023-12-26 16:26:22,913][105620] Updated weights for policy 1, policy_version 148009 (0.0008) [2023-12-26 16:26:22,924][105692] Updated weights for policy 0, policy_version 147246 (0.0009) [2023-12-26 16:26:22,962][105620] Updated weights for policy 1, policy_version 148019 (0.0006) [2023-12-26 16:26:22,973][105692] Updated weights for policy 0, policy_version 147256 (0.0008) [2023-12-26 16:26:23,011][105620] Updated weights for policy 1, policy_version 148029 (0.0008) [2023-12-26 16:26:23,059][105620] Updated weights for policy 1, policy_version 148039 (0.0009) [2023-12-26 16:26:23,692][105620] Updated weights for policy 1, policy_version 148049 (0.0009) [2023-12-26 16:26:23,746][105620] Updated weights for policy 1, policy_version 148059 (0.0009) [2023-12-26 16:26:23,797][105620] Updated weights for policy 1, policy_version 148069 (0.0008) [2023-12-26 16:26:23,822][105692] Updated weights for policy 0, policy_version 147266 (0.0010) [2023-12-26 16:26:23,884][105692] Updated weights for policy 0, policy_version 147276 (0.0010) [2023-12-26 16:26:23,932][105692] Updated weights for policy 0, policy_version 147286 (0.0010) [2023-12-26 16:26:24,503][105692] Updated weights for policy 0, policy_version 147296 (0.0006) [2023-12-26 16:26:24,519][105620] Updated weights for policy 1, policy_version 148079 (0.0006) [2023-12-26 16:26:24,561][105692] Updated weights for policy 0, policy_version 147306 (0.0005) [2023-12-26 16:26:24,587][105620] Updated weights for policy 1, policy_version 148089 (0.0006) [2023-12-26 16:26:24,615][105692] Updated weights for policy 0, policy_version 147316 (0.0005) [2023-12-26 16:26:24,649][105620] Updated weights for policy 1, policy_version 148099 (0.0005) [2023-12-26 16:26:25,222][105620] Updated weights for policy 1, policy_version 148109 (0.0006) [2023-12-26 16:26:25,278][105620] Updated weights for policy 1, policy_version 148119 (0.0008) [2023-12-26 16:26:25,307][105692] Updated weights for policy 0, policy_version 147326 (0.0008) [2023-12-26 16:26:25,341][105620] Updated weights for policy 1, policy_version 148129 (0.0009) [2023-12-26 16:26:25,364][105692] Updated weights for policy 0, policy_version 147336 (0.0006) [2023-12-26 16:26:25,412][105692] Updated weights for policy 0, policy_version 147346 (0.0010) [2023-12-26 16:26:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 75661312. Throughput: 0: 9736.4, 1: 9783.1. Samples: 75674796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:26:26,063][104569] Avg episode reward: [(0, '8019.242'), (1, '6215.019')] [2023-12-26 16:26:26,102][105620] Updated weights for policy 1, policy_version 148139 (0.0007) [2023-12-26 16:26:26,147][105692] Updated weights for policy 0, policy_version 147356 (0.0010) [2023-12-26 16:26:26,157][105620] Updated weights for policy 1, policy_version 148149 (0.0007) [2023-12-26 16:26:26,192][105692] Updated weights for policy 0, policy_version 147366 (0.0010) [2023-12-26 16:26:26,223][105620] Updated weights for policy 1, policy_version 148159 (0.0007) [2023-12-26 16:26:26,250][105692] Updated weights for policy 0, policy_version 147376 (0.0010) [2023-12-26 16:26:26,945][105620] Updated weights for policy 1, policy_version 148169 (0.0009) [2023-12-26 16:26:27,000][105692] Updated weights for policy 0, policy_version 147386 (0.0010) [2023-12-26 16:26:27,007][105620] Updated weights for policy 1, policy_version 148179 (0.0008) [2023-12-26 16:26:27,060][105692] Updated weights for policy 0, policy_version 147396 (0.0010) [2023-12-26 16:26:27,062][105620] Updated weights for policy 1, policy_version 148189 (0.0005) [2023-12-26 16:26:27,110][105620] Updated weights for policy 1, policy_version 148199 (0.0005) [2023-12-26 16:26:27,111][105692] Updated weights for policy 0, policy_version 147406 (0.0010) [2023-12-26 16:26:27,158][105692] Updated weights for policy 0, policy_version 147416 (0.0010) [2023-12-26 16:26:27,830][105620] Updated weights for policy 1, policy_version 148209 (0.0007) [2023-12-26 16:26:27,863][105692] Updated weights for policy 0, policy_version 147426 (0.0010) [2023-12-26 16:26:27,888][105620] Updated weights for policy 1, policy_version 148219 (0.0006) [2023-12-26 16:26:27,914][105692] Updated weights for policy 0, policy_version 147436 (0.0010) [2023-12-26 16:26:27,943][105620] Updated weights for policy 1, policy_version 148229 (0.0006) [2023-12-26 16:26:27,968][105692] Updated weights for policy 0, policy_version 147446 (0.0010) [2023-12-26 16:26:28,673][105620] Updated weights for policy 1, policy_version 148239 (0.0008) [2023-12-26 16:26:28,711][105692] Updated weights for policy 0, policy_version 147456 (0.0010) [2023-12-26 16:26:28,726][105620] Updated weights for policy 1, policy_version 148249 (0.0005) [2023-12-26 16:26:28,766][105692] Updated weights for policy 0, policy_version 147466 (0.0010) [2023-12-26 16:26:28,778][105620] Updated weights for policy 1, policy_version 148259 (0.0008) [2023-12-26 16:26:28,817][105692] Updated weights for policy 0, policy_version 147476 (0.0010) [2023-12-26 16:26:29,551][105620] Updated weights for policy 1, policy_version 148269 (0.0007) [2023-12-26 16:26:29,568][105692] Updated weights for policy 0, policy_version 147486 (0.0010) [2023-12-26 16:26:29,605][105620] Updated weights for policy 1, policy_version 148279 (0.0005) [2023-12-26 16:26:29,626][105692] Updated weights for policy 0, policy_version 147496 (0.0010) [2023-12-26 16:26:29,666][105620] Updated weights for policy 1, policy_version 148289 (0.0006) [2023-12-26 16:26:29,684][105692] Updated weights for policy 0, policy_version 147506 (0.0010) [2023-12-26 16:26:30,417][105620] Updated weights for policy 1, policy_version 148299 (0.0005) [2023-12-26 16:26:30,422][105692] Updated weights for policy 0, policy_version 147516 (0.0010) [2023-12-26 16:26:30,471][105620] Updated weights for policy 1, policy_version 148309 (0.0005) [2023-12-26 16:26:30,478][105692] Updated weights for policy 0, policy_version 147526 (0.0008) [2023-12-26 16:26:30,526][105620] Updated weights for policy 1, policy_version 148319 (0.0007) [2023-12-26 16:26:30,540][105692] Updated weights for policy 0, policy_version 147536 (0.0007) [2023-12-26 16:26:31,062][104569] Fps is (10 sec: 19659.9, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 75759616. Throughput: 0: 9782.2, 1: 9745.4. Samples: 75732296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:26:31,063][104569] Avg episode reward: [(0, '8089.450'), (1, '7498.494')] [2023-12-26 16:26:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000147544_37781504.pth... [2023-12-26 16:26:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000148328_37978112.pth... [2023-12-26 16:26:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000146392_37486592.pth [2023-12-26 16:26:31,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000147176_37683200.pth [2023-12-26 16:26:31,271][105692] Updated weights for policy 0, policy_version 147546 (0.0006) [2023-12-26 16:26:31,302][105620] Updated weights for policy 1, policy_version 148329 (0.0008) [2023-12-26 16:26:31,333][105692] Updated weights for policy 0, policy_version 147556 (0.0008) [2023-12-26 16:26:31,363][105620] Updated weights for policy 1, policy_version 148339 (0.0007) [2023-12-26 16:26:31,400][105692] Updated weights for policy 0, policy_version 147566 (0.0008) [2023-12-26 16:26:31,427][105620] Updated weights for policy 1, policy_version 148349 (0.0008) [2023-12-26 16:26:31,457][105692] Updated weights for policy 0, policy_version 147576 (0.0008) [2023-12-26 16:26:31,481][105620] Updated weights for policy 1, policy_version 148359 (0.0007) [2023-12-26 16:26:32,096][105692] Updated weights for policy 0, policy_version 147586 (0.0005) [2023-12-26 16:26:32,145][105692] Updated weights for policy 0, policy_version 147596 (0.0006) [2023-12-26 16:26:32,199][105692] Updated weights for policy 0, policy_version 147606 (0.0010) [2023-12-26 16:26:32,303][105620] Updated weights for policy 1, policy_version 148369 (0.0007) [2023-12-26 16:26:32,372][105620] Updated weights for policy 1, policy_version 148379 (0.0007) [2023-12-26 16:26:32,429][105620] Updated weights for policy 1, policy_version 148389 (0.0009) [2023-12-26 16:26:32,835][105692] Updated weights for policy 0, policy_version 147616 (0.0006) [2023-12-26 16:26:32,886][105692] Updated weights for policy 0, policy_version 147626 (0.0006) [2023-12-26 16:26:32,934][105692] Updated weights for policy 0, policy_version 147636 (0.0009) [2023-12-26 16:26:33,238][105620] Updated weights for policy 1, policy_version 148399 (0.0010) [2023-12-26 16:26:33,297][105620] Updated weights for policy 1, policy_version 148409 (0.0010) [2023-12-26 16:26:33,354][105620] Updated weights for policy 1, policy_version 148419 (0.0009) [2023-12-26 16:26:33,506][105692] Updated weights for policy 0, policy_version 147646 (0.0007) [2023-12-26 16:26:33,549][105692] Updated weights for policy 0, policy_version 147656 (0.0005) [2023-12-26 16:26:33,597][105692] Updated weights for policy 0, policy_version 147666 (0.0005) [2023-12-26 16:26:34,195][105692] Updated weights for policy 0, policy_version 147676 (0.0008) [2023-12-26 16:26:34,222][105620] Updated weights for policy 1, policy_version 148429 (0.0009) [2023-12-26 16:26:34,256][105692] Updated weights for policy 0, policy_version 147686 (0.0011) [2023-12-26 16:26:34,289][105620] Updated weights for policy 1, policy_version 148439 (0.0007) [2023-12-26 16:26:34,309][105692] Updated weights for policy 0, policy_version 147696 (0.0010) [2023-12-26 16:26:34,352][105620] Updated weights for policy 1, policy_version 148449 (0.0008) [2023-12-26 16:26:34,979][105692] Updated weights for policy 0, policy_version 147706 (0.0011) [2023-12-26 16:26:35,036][105692] Updated weights for policy 0, policy_version 147716 (0.0011) [2023-12-26 16:26:35,036][105620] Updated weights for policy 1, policy_version 148459 (0.0008) [2023-12-26 16:26:35,085][105692] Updated weights for policy 0, policy_version 147726 (0.0010) [2023-12-26 16:26:35,098][105620] Updated weights for policy 1, policy_version 148469 (0.0008) [2023-12-26 16:26:35,133][105692] Updated weights for policy 0, policy_version 147736 (0.0010) [2023-12-26 16:26:35,159][105620] Updated weights for policy 1, policy_version 148479 (0.0008) [2023-12-26 16:26:35,816][105692] Updated weights for policy 0, policy_version 147746 (0.0008) [2023-12-26 16:26:35,871][105692] Updated weights for policy 0, policy_version 147756 (0.0009) [2023-12-26 16:26:35,874][105620] Updated weights for policy 1, policy_version 148489 (0.0008) [2023-12-26 16:26:35,924][105692] Updated weights for policy 0, policy_version 147766 (0.0006) [2023-12-26 16:26:35,933][105620] Updated weights for policy 1, policy_version 148499 (0.0010) [2023-12-26 16:26:35,990][105620] Updated weights for policy 1, policy_version 148509 (0.0010) [2023-12-26 16:26:36,052][105620] Updated weights for policy 1, policy_version 148519 (0.0006) [2023-12-26 16:26:36,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 75866112. Throughput: 0: 9854.8, 1: 9646.4. Samples: 75848020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:26:36,064][104569] Avg episode reward: [(0, '9175.377'), (1, '8034.337')] [2023-12-26 16:26:36,608][105692] Updated weights for policy 0, policy_version 147776 (0.0008) [2023-12-26 16:26:36,669][105692] Updated weights for policy 0, policy_version 147786 (0.0008) [2023-12-26 16:26:36,730][105692] Updated weights for policy 0, policy_version 147796 (0.0008) [2023-12-26 16:26:36,785][105620] Updated weights for policy 1, policy_version 148529 (0.0010) [2023-12-26 16:26:36,845][105620] Updated weights for policy 1, policy_version 148539 (0.0011) [2023-12-26 16:26:36,902][105620] Updated weights for policy 1, policy_version 148549 (0.0011) [2023-12-26 16:26:37,339][105692] Updated weights for policy 0, policy_version 147806 (0.0005) [2023-12-26 16:26:37,408][105692] Updated weights for policy 0, policy_version 147816 (0.0006) [2023-12-26 16:26:37,480][105692] Updated weights for policy 0, policy_version 147826 (0.0006) [2023-12-26 16:26:37,641][105620] Updated weights for policy 1, policy_version 148559 (0.0007) [2023-12-26 16:26:37,704][105620] Updated weights for policy 1, policy_version 148569 (0.0006) [2023-12-26 16:26:37,755][105620] Updated weights for policy 1, policy_version 148579 (0.0008) [2023-12-26 16:26:38,172][105692] Updated weights for policy 0, policy_version 147836 (0.0008) [2023-12-26 16:26:38,239][105692] Updated weights for policy 0, policy_version 147846 (0.0009) [2023-12-26 16:26:38,287][105692] Updated weights for policy 0, policy_version 147856 (0.0009) [2023-12-26 16:26:38,446][105620] Updated weights for policy 1, policy_version 148589 (0.0009) [2023-12-26 16:26:38,498][105620] Updated weights for policy 1, policy_version 148599 (0.0009) [2023-12-26 16:26:38,553][105620] Updated weights for policy 1, policy_version 148609 (0.0009) [2023-12-26 16:26:39,062][105692] Updated weights for policy 0, policy_version 147866 (0.0008) [2023-12-26 16:26:39,108][105692] Updated weights for policy 0, policy_version 147876 (0.0009) [2023-12-26 16:26:39,155][105692] Updated weights for policy 0, policy_version 147886 (0.0009) [2023-12-26 16:26:39,202][105692] Updated weights for policy 0, policy_version 147896 (0.0008) [2023-12-26 16:26:39,335][105620] Updated weights for policy 1, policy_version 148619 (0.0010) [2023-12-26 16:26:39,399][105620] Updated weights for policy 1, policy_version 148629 (0.0009) [2023-12-26 16:26:39,464][105620] Updated weights for policy 1, policy_version 148639 (0.0009) [2023-12-26 16:26:39,941][105692] Updated weights for policy 0, policy_version 147906 (0.0008) [2023-12-26 16:26:40,005][105692] Updated weights for policy 0, policy_version 147916 (0.0007) [2023-12-26 16:26:40,077][105692] Updated weights for policy 0, policy_version 147926 (0.0006) [2023-12-26 16:26:40,214][105620] Updated weights for policy 1, policy_version 148649 (0.0010) [2023-12-26 16:26:40,276][105620] Updated weights for policy 1, policy_version 148659 (0.0008) [2023-12-26 16:26:40,331][105620] Updated weights for policy 1, policy_version 148669 (0.0008) [2023-12-26 16:26:40,399][105620] Updated weights for policy 1, policy_version 148679 (0.0006) [2023-12-26 16:26:40,798][105692] Updated weights for policy 0, policy_version 147936 (0.0008) [2023-12-26 16:26:40,850][105692] Updated weights for policy 0, policy_version 147946 (0.0008) [2023-12-26 16:26:40,915][105692] Updated weights for policy 0, policy_version 147956 (0.0008) [2023-12-26 16:26:41,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 75956224. Throughput: 0: 9877.8, 1: 9631.6. Samples: 75965140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:26:41,062][104569] Avg episode reward: [(0, '9354.118'), (1, '8653.380')] [2023-12-26 16:26:41,120][105620] Updated weights for policy 1, policy_version 148689 (0.0010) [2023-12-26 16:26:41,194][105620] Updated weights for policy 1, policy_version 148699 (0.0010) [2023-12-26 16:26:41,262][105620] Updated weights for policy 1, policy_version 148709 (0.0011) [2023-12-26 16:26:41,682][105692] Updated weights for policy 0, policy_version 147966 (0.0008) [2023-12-26 16:26:41,750][105692] Updated weights for policy 0, policy_version 147976 (0.0008) [2023-12-26 16:26:41,815][105692] Updated weights for policy 0, policy_version 147986 (0.0008) [2023-12-26 16:26:41,989][105620] Updated weights for policy 1, policy_version 148719 (0.0007) [2023-12-26 16:26:42,058][105620] Updated weights for policy 1, policy_version 148729 (0.0010) [2023-12-26 16:26:42,129][105620] Updated weights for policy 1, policy_version 148739 (0.0011) [2023-12-26 16:26:42,494][105692] Updated weights for policy 0, policy_version 147996 (0.0009) [2023-12-26 16:26:42,550][105692] Updated weights for policy 0, policy_version 148006 (0.0008) [2023-12-26 16:26:42,599][105692] Updated weights for policy 0, policy_version 148016 (0.0008) [2023-12-26 16:26:42,834][105620] Updated weights for policy 1, policy_version 148749 (0.0010) [2023-12-26 16:26:42,886][105620] Updated weights for policy 1, policy_version 148759 (0.0010) [2023-12-26 16:26:42,951][105620] Updated weights for policy 1, policy_version 148769 (0.0010) [2023-12-26 16:26:43,259][105692] Updated weights for policy 0, policy_version 148026 (0.0007) [2023-12-26 16:26:43,305][105692] Updated weights for policy 0, policy_version 148036 (0.0005) [2023-12-26 16:26:43,348][105692] Updated weights for policy 0, policy_version 148046 (0.0005) [2023-12-26 16:26:43,409][105692] Updated weights for policy 0, policy_version 148056 (0.0008) [2023-12-26 16:26:43,683][105620] Updated weights for policy 1, policy_version 148779 (0.0010) [2023-12-26 16:26:43,743][105620] Updated weights for policy 1, policy_version 148789 (0.0007) [2023-12-26 16:26:43,809][105620] Updated weights for policy 1, policy_version 148799 (0.0007) [2023-12-26 16:26:44,137][105692] Updated weights for policy 0, policy_version 148067 (0.0009) [2023-12-26 16:26:44,194][105692] Updated weights for policy 0, policy_version 148077 (0.0010) [2023-12-26 16:26:44,251][105692] Updated weights for policy 0, policy_version 148087 (0.0010) [2023-12-26 16:26:44,363][105620] Updated weights for policy 1, policy_version 148809 (0.0007) [2023-12-26 16:26:44,425][105620] Updated weights for policy 1, policy_version 148819 (0.0007) [2023-12-26 16:26:44,481][105620] Updated weights for policy 1, policy_version 148829 (0.0005) [2023-12-26 16:26:44,534][105620] Updated weights for policy 1, policy_version 148839 (0.0005) [2023-12-26 16:26:45,124][105692] Updated weights for policy 0, policy_version 148097 (0.0008) [2023-12-26 16:26:45,128][105620] Updated weights for policy 1, policy_version 148849 (0.0006) [2023-12-26 16:26:45,189][105692] Updated weights for policy 0, policy_version 148107 (0.0009) [2023-12-26 16:26:45,190][105620] Updated weights for policy 1, policy_version 148859 (0.0007) [2023-12-26 16:26:45,252][105692] Updated weights for policy 0, policy_version 148117 (0.0008) [2023-12-26 16:26:45,255][105620] Updated weights for policy 1, policy_version 148869 (0.0008) [2023-12-26 16:26:45,898][105620] Updated weights for policy 1, policy_version 148879 (0.0005) [2023-12-26 16:26:45,963][105620] Updated weights for policy 1, policy_version 148889 (0.0006) [2023-12-26 16:26:46,022][105620] Updated weights for policy 1, policy_version 148899 (0.0006) [2023-12-26 16:26:46,023][105692] Updated weights for policy 0, policy_version 148127 (0.0008) [2023-12-26 16:26:46,062][104569] Fps is (10 sec: 18842.3, 60 sec: 19660.9, 300 sec: 19521.9). Total num frames: 76054528. Throughput: 0: 9884.4, 1: 9623.1. Samples: 76023040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:26:46,062][104569] Avg episode reward: [(0, '9353.984'), (1, '8657.365')] [2023-12-26 16:26:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000148904_38125568.pth... [2023-12-26 16:26:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000147752_37830656.pth [2023-12-26 16:26:46,086][105692] Updated weights for policy 0, policy_version 148137 (0.0007) [2023-12-26 16:26:46,147][105692] Updated weights for policy 0, policy_version 148147 (0.0009) [2023-12-26 16:26:46,177][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000148152_37937152.pth... [2023-12-26 16:26:46,182][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000147000_37642240.pth [2023-12-26 16:26:46,698][105620] Updated weights for policy 1, policy_version 148909 (0.0009) [2023-12-26 16:26:46,744][105620] Updated weights for policy 1, policy_version 148919 (0.0008) [2023-12-26 16:26:46,790][105620] Updated weights for policy 1, policy_version 148929 (0.0009) [2023-12-26 16:26:46,904][105692] Updated weights for policy 0, policy_version 148157 (0.0010) [2023-12-26 16:26:46,957][105692] Updated weights for policy 0, policy_version 148167 (0.0009) [2023-12-26 16:26:47,008][105692] Updated weights for policy 0, policy_version 148177 (0.0009) [2023-12-26 16:26:47,403][105620] Updated weights for policy 1, policy_version 148939 (0.0007) [2023-12-26 16:26:47,449][105620] Updated weights for policy 1, policy_version 148949 (0.0008) [2023-12-26 16:26:47,495][105620] Updated weights for policy 1, policy_version 148959 (0.0008) [2023-12-26 16:26:47,814][105692] Updated weights for policy 0, policy_version 148187 (0.0010) [2023-12-26 16:26:47,872][105692] Updated weights for policy 0, policy_version 148197 (0.0009) [2023-12-26 16:26:47,932][105692] Updated weights for policy 0, policy_version 148207 (0.0009) [2023-12-26 16:26:48,267][105620] Updated weights for policy 1, policy_version 148969 (0.0008) [2023-12-26 16:26:48,331][105620] Updated weights for policy 1, policy_version 148979 (0.0008) [2023-12-26 16:26:48,395][105620] Updated weights for policy 1, policy_version 148989 (0.0006) [2023-12-26 16:26:48,456][105620] Updated weights for policy 1, policy_version 148999 (0.0006) [2023-12-26 16:26:48,743][105692] Updated weights for policy 0, policy_version 148217 (0.0009) [2023-12-26 16:26:48,808][105692] Updated weights for policy 0, policy_version 148227 (0.0009) [2023-12-26 16:26:48,867][105692] Updated weights for policy 0, policy_version 148237 (0.0009) [2023-12-26 16:26:48,925][105692] Updated weights for policy 0, policy_version 148247 (0.0009) [2023-12-26 16:26:49,078][105620] Updated weights for policy 1, policy_version 149009 (0.0009) [2023-12-26 16:26:49,129][105620] Updated weights for policy 1, policy_version 149019 (0.0009) [2023-12-26 16:26:49,191][105620] Updated weights for policy 1, policy_version 149029 (0.0009) [2023-12-26 16:26:49,688][105692] Updated weights for policy 0, policy_version 148257 (0.0008) [2023-12-26 16:26:49,747][105692] Updated weights for policy 0, policy_version 148267 (0.0008) [2023-12-26 16:26:49,805][105692] Updated weights for policy 0, policy_version 148277 (0.0008) [2023-12-26 16:26:49,985][105620] Updated weights for policy 1, policy_version 149039 (0.0007) [2023-12-26 16:26:50,060][105620] Updated weights for policy 1, policy_version 149049 (0.0007) [2023-12-26 16:26:50,128][105620] Updated weights for policy 1, policy_version 149059 (0.0009) [2023-12-26 16:26:50,573][105692] Updated weights for policy 0, policy_version 148287 (0.0008) [2023-12-26 16:26:50,632][105692] Updated weights for policy 0, policy_version 148297 (0.0005) [2023-12-26 16:26:50,698][105692] Updated weights for policy 0, policy_version 148307 (0.0008) [2023-12-26 16:26:50,829][105620] Updated weights for policy 1, policy_version 149069 (0.0010) [2023-12-26 16:26:50,888][105620] Updated weights for policy 1, policy_version 149079 (0.0011) [2023-12-26 16:26:50,948][105620] Updated weights for policy 1, policy_version 149089 (0.0009) [2023-12-26 16:26:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 76152832. Throughput: 0: 9707.1, 1: 9772.9. Samples: 76138736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:26:51,063][104569] Avg episode reward: [(0, '9264.731'), (1, '8719.617')] [2023-12-26 16:26:51,429][105692] Updated weights for policy 0, policy_version 148317 (0.0010) [2023-12-26 16:26:51,495][105692] Updated weights for policy 0, policy_version 148327 (0.0009) [2023-12-26 16:26:51,562][105692] Updated weights for policy 0, policy_version 148337 (0.0007) [2023-12-26 16:26:51,615][105620] Updated weights for policy 1, policy_version 149099 (0.0006) [2023-12-26 16:26:51,681][105620] Updated weights for policy 1, policy_version 149109 (0.0009) [2023-12-26 16:26:51,746][105620] Updated weights for policy 1, policy_version 149119 (0.0009) [2023-12-26 16:26:52,288][105692] Updated weights for policy 0, policy_version 148347 (0.0008) [2023-12-26 16:26:52,347][105692] Updated weights for policy 0, policy_version 148357 (0.0008) [2023-12-26 16:26:52,398][105692] Updated weights for policy 0, policy_version 148367 (0.0008) [2023-12-26 16:26:52,506][105620] Updated weights for policy 1, policy_version 149129 (0.0009) [2023-12-26 16:26:52,566][105620] Updated weights for policy 1, policy_version 149139 (0.0009) [2023-12-26 16:26:52,629][105620] Updated weights for policy 1, policy_version 149150 (0.0011) [2023-12-26 16:26:52,696][105620] Updated weights for policy 1, policy_version 149160 (0.0010) [2023-12-26 16:26:52,994][105692] Updated weights for policy 0, policy_version 148377 (0.0008) [2023-12-26 16:26:53,048][105692] Updated weights for policy 0, policy_version 148387 (0.0008) [2023-12-26 16:26:53,097][105692] Updated weights for policy 0, policy_version 148397 (0.0008) [2023-12-26 16:26:53,145][105692] Updated weights for policy 0, policy_version 148407 (0.0005) [2023-12-26 16:26:53,541][105620] Updated weights for policy 1, policy_version 149170 (0.0009) [2023-12-26 16:26:53,594][105620] Updated weights for policy 1, policy_version 149180 (0.0010) [2023-12-26 16:26:53,653][105620] Updated weights for policy 1, policy_version 149191 (0.0011) [2023-12-26 16:26:53,751][105692] Updated weights for policy 0, policy_version 148417 (0.0005) [2023-12-26 16:26:53,805][105692] Updated weights for policy 0, policy_version 148427 (0.0008) [2023-12-26 16:26:53,856][105692] Updated weights for policy 0, policy_version 148437 (0.0009) [2023-12-26 16:26:54,478][105692] Updated weights for policy 0, policy_version 148447 (0.0006) [2023-12-26 16:26:54,530][105620] Updated weights for policy 1, policy_version 149201 (0.0008) [2023-12-26 16:26:54,534][105692] Updated weights for policy 0, policy_version 148457 (0.0005) [2023-12-26 16:26:54,584][105620] Updated weights for policy 1, policy_version 149211 (0.0007) [2023-12-26 16:26:54,592][105692] Updated weights for policy 0, policy_version 148467 (0.0006) [2023-12-26 16:26:54,641][105620] Updated weights for policy 1, policy_version 149221 (0.0008) [2023-12-26 16:26:55,145][105692] Updated weights for policy 0, policy_version 148477 (0.0007) [2023-12-26 16:26:55,193][105692] Updated weights for policy 0, policy_version 148487 (0.0009) [2023-12-26 16:26:55,241][105692] Updated weights for policy 0, policy_version 148497 (0.0009) [2023-12-26 16:26:55,455][105620] Updated weights for policy 1, policy_version 149231 (0.0010) [2023-12-26 16:26:55,508][105620] Updated weights for policy 1, policy_version 149241 (0.0009) [2023-12-26 16:26:55,564][105620] Updated weights for policy 1, policy_version 149252 (0.0009) [2023-12-26 16:26:55,922][105692] Updated weights for policy 0, policy_version 148507 (0.0009) [2023-12-26 16:26:55,980][105692] Updated weights for policy 0, policy_version 148517 (0.0009) [2023-12-26 16:26:56,040][105692] Updated weights for policy 0, policy_version 148527 (0.0009) [2023-12-26 16:26:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 76242944. Throughput: 0: 9839.0, 1: 9692.5. Samples: 76255760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:26:56,062][104569] Avg episode reward: [(0, '8929.293'), (1, '8915.456')] [2023-12-26 16:26:56,339][105620] Updated weights for policy 1, policy_version 149262 (0.0009) [2023-12-26 16:26:56,387][105620] Updated weights for policy 1, policy_version 149272 (0.0009) [2023-12-26 16:26:56,438][105620] Updated weights for policy 1, policy_version 149282 (0.0009) [2023-12-26 16:26:56,793][105692] Updated weights for policy 0, policy_version 148537 (0.0009) [2023-12-26 16:26:56,843][105692] Updated weights for policy 0, policy_version 148547 (0.0008) [2023-12-26 16:26:56,896][105692] Updated weights for policy 0, policy_version 148557 (0.0008) [2023-12-26 16:26:56,954][105692] Updated weights for policy 0, policy_version 148567 (0.0009) [2023-12-26 16:26:57,143][105620] Updated weights for policy 1, policy_version 149292 (0.0008) [2023-12-26 16:26:57,201][105620] Updated weights for policy 1, policy_version 149302 (0.0009) [2023-12-26 16:26:57,258][105620] Updated weights for policy 1, policy_version 149312 (0.0010) [2023-12-26 16:26:57,700][105692] Updated weights for policy 0, policy_version 148577 (0.0009) [2023-12-26 16:26:57,747][105692] Updated weights for policy 0, policy_version 148587 (0.0008) [2023-12-26 16:26:57,800][105692] Updated weights for policy 0, policy_version 148597 (0.0009) [2023-12-26 16:26:57,979][105620] Updated weights for policy 1, policy_version 149322 (0.0008) [2023-12-26 16:26:58,046][105620] Updated weights for policy 1, policy_version 149332 (0.0005) [2023-12-26 16:26:58,114][105620] Updated weights for policy 1, policy_version 149342 (0.0006) [2023-12-26 16:26:58,171][105620] Updated weights for policy 1, policy_version 149352 (0.0009) [2023-12-26 16:26:58,624][105692] Updated weights for policy 0, policy_version 148607 (0.0008) [2023-12-26 16:26:58,688][105692] Updated weights for policy 0, policy_version 148617 (0.0007) [2023-12-26 16:26:58,753][105692] Updated weights for policy 0, policy_version 148627 (0.0008) [2023-12-26 16:26:58,878][105620] Updated weights for policy 1, policy_version 149362 (0.0009) [2023-12-26 16:26:58,938][105620] Updated weights for policy 1, policy_version 149372 (0.0009) [2023-12-26 16:26:59,005][105620] Updated weights for policy 1, policy_version 149382 (0.0009) [2023-12-26 16:26:59,413][105692] Updated weights for policy 0, policy_version 148637 (0.0010) [2023-12-26 16:26:59,467][105692] Updated weights for policy 0, policy_version 148647 (0.0010) [2023-12-26 16:26:59,528][105692] Updated weights for policy 0, policy_version 148657 (0.0009) [2023-12-26 16:26:59,745][105620] Updated weights for policy 1, policy_version 149392 (0.0009) [2023-12-26 16:26:59,799][105620] Updated weights for policy 1, policy_version 149402 (0.0007) [2023-12-26 16:26:59,861][105620] Updated weights for policy 1, policy_version 149412 (0.0007) [2023-12-26 16:27:00,267][105692] Updated weights for policy 0, policy_version 148667 (0.0009) [2023-12-26 16:27:00,328][105692] Updated weights for policy 0, policy_version 148677 (0.0009) [2023-12-26 16:27:00,375][105692] Updated weights for policy 0, policy_version 148687 (0.0009) [2023-12-26 16:27:00,635][105620] Updated weights for policy 1, policy_version 149422 (0.0005) [2023-12-26 16:27:00,697][105620] Updated weights for policy 1, policy_version 149432 (0.0005) [2023-12-26 16:27:00,753][105620] Updated weights for policy 1, policy_version 149442 (0.0005) [2023-12-26 16:27:00,942][105692] Updated weights for policy 0, policy_version 148697 (0.0008) [2023-12-26 16:27:01,000][105692] Updated weights for policy 0, policy_version 148707 (0.0009) [2023-12-26 16:27:01,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 76341248. Throughput: 0: 9834.4, 1: 9652.2. Samples: 76312636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:27:01,062][105692] Updated weights for policy 0, policy_version 148717 (0.0008) [2023-12-26 16:27:01,062][104569] Avg episode reward: [(0, '1316.711'), (1, '8911.896')] [2023-12-26 16:27:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000149448_38264832.pth... [2023-12-26 16:27:01,084][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000148328_37978112.pth [2023-12-26 16:27:01,125][105692] Updated weights for policy 0, policy_version 148727 (0.0009) [2023-12-26 16:27:01,131][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000148728_38084608.pth... [2023-12-26 16:27:01,134][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000147544_37781504.pth [2023-12-26 16:27:01,390][105620] Updated weights for policy 1, policy_version 149452 (0.0006) [2023-12-26 16:27:01,454][105620] Updated weights for policy 1, policy_version 149462 (0.0008) [2023-12-26 16:27:01,516][105620] Updated weights for policy 1, policy_version 149472 (0.0009) [2023-12-26 16:27:01,904][105692] Updated weights for policy 0, policy_version 148737 (0.0006) [2023-12-26 16:27:01,960][105692] Updated weights for policy 0, policy_version 148747 (0.0006) [2023-12-26 16:27:02,023][105692] Updated weights for policy 0, policy_version 148757 (0.0005) [2023-12-26 16:27:02,241][105620] Updated weights for policy 1, policy_version 149482 (0.0009) [2023-12-26 16:27:02,307][105620] Updated weights for policy 1, policy_version 149492 (0.0010) [2023-12-26 16:27:02,368][105620] Updated weights for policy 1, policy_version 149502 (0.0011) [2023-12-26 16:27:02,430][105620] Updated weights for policy 1, policy_version 149512 (0.0010) [2023-12-26 16:27:02,670][105692] Updated weights for policy 0, policy_version 148767 (0.0005) [2023-12-26 16:27:02,731][105692] Updated weights for policy 0, policy_version 148777 (0.0007) [2023-12-26 16:27:02,776][105692] Updated weights for policy 0, policy_version 148787 (0.0010) [2023-12-26 16:27:03,102][105620] Updated weights for policy 1, policy_version 149522 (0.0007) [2023-12-26 16:27:03,155][105620] Updated weights for policy 1, policy_version 149532 (0.0005) [2023-12-26 16:27:03,202][105620] Updated weights for policy 1, policy_version 149542 (0.0010) [2023-12-26 16:27:03,434][105692] Updated weights for policy 0, policy_version 148797 (0.0010) [2023-12-26 16:27:03,478][105692] Updated weights for policy 0, policy_version 148807 (0.0010) [2023-12-26 16:27:03,524][105692] Updated weights for policy 0, policy_version 148817 (0.0009) [2023-12-26 16:27:03,886][105620] Updated weights for policy 1, policy_version 149552 (0.0011) [2023-12-26 16:27:03,944][105620] Updated weights for policy 1, policy_version 149562 (0.0010) [2023-12-26 16:27:03,999][105620] Updated weights for policy 1, policy_version 149572 (0.0010) [2023-12-26 16:27:04,261][105692] Updated weights for policy 0, policy_version 148827 (0.0008) [2023-12-26 16:27:04,322][105692] Updated weights for policy 0, policy_version 148837 (0.0006) [2023-12-26 16:27:04,384][105692] Updated weights for policy 0, policy_version 148847 (0.0005) [2023-12-26 16:27:04,701][105620] Updated weights for policy 1, policy_version 149582 (0.0010) [2023-12-26 16:27:04,760][105620] Updated weights for policy 1, policy_version 149592 (0.0011) [2023-12-26 16:27:04,822][105620] Updated weights for policy 1, policy_version 149602 (0.0011) [2023-12-26 16:27:05,063][105692] Updated weights for policy 0, policy_version 148857 (0.0006) [2023-12-26 16:27:05,111][105692] Updated weights for policy 0, policy_version 148867 (0.0008) [2023-12-26 16:27:05,159][105692] Updated weights for policy 0, policy_version 148877 (0.0008) [2023-12-26 16:27:05,211][105692] Updated weights for policy 0, policy_version 148887 (0.0008) [2023-12-26 16:27:05,574][105620] Updated weights for policy 1, policy_version 149612 (0.0010) [2023-12-26 16:27:05,619][105620] Updated weights for policy 1, policy_version 149622 (0.0010) [2023-12-26 16:27:05,668][105620] Updated weights for policy 1, policy_version 149632 (0.0010) [2023-12-26 16:27:05,970][105692] Updated weights for policy 0, policy_version 148897 (0.0007) [2023-12-26 16:27:06,027][105692] Updated weights for policy 0, policy_version 148907 (0.0006) [2023-12-26 16:27:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 76439552. Throughput: 0: 9859.3, 1: 9591.2. Samples: 76432376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:27:06,063][104569] Avg episode reward: [(0, '1643.533'), (1, '8910.465')] [2023-12-26 16:27:06,084][105692] Updated weights for policy 0, policy_version 148917 (0.0006) [2023-12-26 16:27:06,411][105620] Updated weights for policy 1, policy_version 149642 (0.0010) [2023-12-26 16:27:06,477][105620] Updated weights for policy 1, policy_version 149652 (0.0011) [2023-12-26 16:27:06,529][105620] Updated weights for policy 1, policy_version 149662 (0.0010) [2023-12-26 16:27:06,592][105620] Updated weights for policy 1, policy_version 149672 (0.0011) [2023-12-26 16:27:06,763][105692] Updated weights for policy 0, policy_version 148927 (0.0010) [2023-12-26 16:27:06,822][105692] Updated weights for policy 0, policy_version 148937 (0.0010) [2023-12-26 16:27:06,880][105692] Updated weights for policy 0, policy_version 148947 (0.0010) [2023-12-26 16:27:07,322][105620] Updated weights for policy 1, policy_version 149682 (0.0010) [2023-12-26 16:27:07,385][105620] Updated weights for policy 1, policy_version 149692 (0.0011) [2023-12-26 16:27:07,448][105620] Updated weights for policy 1, policy_version 149702 (0.0011) [2023-12-26 16:27:07,633][105692] Updated weights for policy 0, policy_version 148957 (0.0011) [2023-12-26 16:27:07,693][105692] Updated weights for policy 0, policy_version 148967 (0.0010) [2023-12-26 16:27:07,746][105692] Updated weights for policy 0, policy_version 148977 (0.0011) [2023-12-26 16:27:08,124][105620] Updated weights for policy 1, policy_version 149712 (0.0011) [2023-12-26 16:27:08,186][105620] Updated weights for policy 1, policy_version 149722 (0.0011) [2023-12-26 16:27:08,248][105620] Updated weights for policy 1, policy_version 149732 (0.0010) [2023-12-26 16:27:08,445][105692] Updated weights for policy 0, policy_version 148987 (0.0010) [2023-12-26 16:27:08,503][105692] Updated weights for policy 0, policy_version 148997 (0.0008) [2023-12-26 16:27:08,560][105692] Updated weights for policy 0, policy_version 149007 (0.0009) [2023-12-26 16:27:08,975][105620] Updated weights for policy 1, policy_version 149742 (0.0011) [2023-12-26 16:27:09,020][105620] Updated weights for policy 1, policy_version 149752 (0.0010) [2023-12-26 16:27:09,068][105620] Updated weights for policy 1, policy_version 149762 (0.0008) [2023-12-26 16:27:09,158][105692] Updated weights for policy 0, policy_version 149017 (0.0008) [2023-12-26 16:27:09,220][105692] Updated weights for policy 0, policy_version 149027 (0.0010) [2023-12-26 16:27:09,292][105692] Updated weights for policy 0, policy_version 149037 (0.0011) [2023-12-26 16:27:09,359][105692] Updated weights for policy 0, policy_version 149047 (0.0011) [2023-12-26 16:27:09,796][105620] Updated weights for policy 1, policy_version 149772 (0.0007) [2023-12-26 16:27:09,856][105620] Updated weights for policy 1, policy_version 149782 (0.0008) [2023-12-26 16:27:09,913][105620] Updated weights for policy 1, policy_version 149792 (0.0009) [2023-12-26 16:27:10,111][105692] Updated weights for policy 0, policy_version 149057 (0.0009) [2023-12-26 16:27:10,176][105692] Updated weights for policy 0, policy_version 149067 (0.0008) [2023-12-26 16:27:10,243][105692] Updated weights for policy 0, policy_version 149077 (0.0008) [2023-12-26 16:27:10,588][105620] Updated weights for policy 1, policy_version 149802 (0.0008) [2023-12-26 16:27:10,650][105620] Updated weights for policy 1, policy_version 149812 (0.0009) [2023-12-26 16:27:10,698][105620] Updated weights for policy 1, policy_version 149822 (0.0009) [2023-12-26 16:27:10,757][105620] Updated weights for policy 1, policy_version 149832 (0.0006) [2023-12-26 16:27:10,970][105692] Updated weights for policy 0, policy_version 149087 (0.0009) [2023-12-26 16:27:11,027][105692] Updated weights for policy 0, policy_version 149097 (0.0008) [2023-12-26 16:27:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 76537856. Throughput: 0: 9876.9, 1: 9557.6. Samples: 76549348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:27:11,063][104569] Avg episode reward: [(0, '6247.347'), (1, '8525.628')] [2023-12-26 16:27:11,095][105692] Updated weights for policy 0, policy_version 149107 (0.0009) [2023-12-26 16:27:11,533][105620] Updated weights for policy 1, policy_version 149842 (0.0008) [2023-12-26 16:27:11,592][105620] Updated weights for policy 1, policy_version 149852 (0.0008) [2023-12-26 16:27:11,661][105620] Updated weights for policy 1, policy_version 149862 (0.0008) [2023-12-26 16:27:11,878][105692] Updated weights for policy 0, policy_version 149117 (0.0008) [2023-12-26 16:27:11,927][105692] Updated weights for policy 0, policy_version 149127 (0.0008) [2023-12-26 16:27:11,976][105692] Updated weights for policy 0, policy_version 149137 (0.0008) [2023-12-26 16:27:12,371][105620] Updated weights for policy 1, policy_version 149872 (0.0008) [2023-12-26 16:27:12,427][105620] Updated weights for policy 1, policy_version 149882 (0.0006) [2023-12-26 16:27:12,487][105620] Updated weights for policy 1, policy_version 149892 (0.0008) [2023-12-26 16:27:12,710][105692] Updated weights for policy 0, policy_version 149147 (0.0009) [2023-12-26 16:27:12,765][105692] Updated weights for policy 0, policy_version 149157 (0.0009) [2023-12-26 16:27:12,824][105692] Updated weights for policy 0, policy_version 149167 (0.0010) [2023-12-26 16:27:13,129][105620] Updated weights for policy 1, policy_version 149902 (0.0007) [2023-12-26 16:27:13,181][105620] Updated weights for policy 1, policy_version 149912 (0.0006) [2023-12-26 16:27:13,242][105620] Updated weights for policy 1, policy_version 149922 (0.0009) [2023-12-26 16:27:13,610][105692] Updated weights for policy 0, policy_version 149177 (0.0010) [2023-12-26 16:27:13,671][105692] Updated weights for policy 0, policy_version 149187 (0.0008) [2023-12-26 16:27:13,732][105692] Updated weights for policy 0, policy_version 149198 (0.0009) [2023-12-26 16:27:13,795][105692] Updated weights for policy 0, policy_version 149208 (0.0009) [2023-12-26 16:27:13,946][105620] Updated weights for policy 1, policy_version 149932 (0.0008) [2023-12-26 16:27:14,012][105620] Updated weights for policy 1, policy_version 149942 (0.0005) [2023-12-26 16:27:14,075][105620] Updated weights for policy 1, policy_version 149952 (0.0007) [2023-12-26 16:27:14,635][105692] Updated weights for policy 0, policy_version 149218 (0.0009) [2023-12-26 16:27:14,683][105692] Updated weights for policy 0, policy_version 149228 (0.0009) [2023-12-26 16:27:14,706][105620] Updated weights for policy 1, policy_version 149962 (0.0007) [2023-12-26 16:27:14,728][105692] Updated weights for policy 0, policy_version 149238 (0.0008) [2023-12-26 16:27:14,764][105620] Updated weights for policy 1, policy_version 149972 (0.0008) [2023-12-26 16:27:14,833][105620] Updated weights for policy 1, policy_version 149982 (0.0009) [2023-12-26 16:27:14,904][105620] Updated weights for policy 1, policy_version 149992 (0.0009) [2023-12-26 16:27:15,521][105692] Updated weights for policy 0, policy_version 149248 (0.0005) [2023-12-26 16:27:15,580][105692] Updated weights for policy 0, policy_version 149258 (0.0006) [2023-12-26 16:27:15,595][105620] Updated weights for policy 1, policy_version 150002 (0.0006) [2023-12-26 16:27:15,647][105692] Updated weights for policy 0, policy_version 149268 (0.0006) [2023-12-26 16:27:15,656][105620] Updated weights for policy 1, policy_version 150012 (0.0009) [2023-12-26 16:27:15,715][105620] Updated weights for policy 1, policy_version 150022 (0.0007) [2023-12-26 16:27:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 76636160. Throughput: 0: 9853.7, 1: 9598.7. Samples: 76607648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:27:16,063][104569] Avg episode reward: [(0, '6146.375'), (1, '8794.787')] [2023-12-26 16:27:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000150024_38412288.pth... [2023-12-26 16:27:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000149272_38223872.pth... [2023-12-26 16:27:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000148152_37937152.pth [2023-12-26 16:27:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000148904_38125568.pth [2023-12-26 16:27:16,355][105620] Updated weights for policy 1, policy_version 150032 (0.0009) [2023-12-26 16:27:16,388][105692] Updated weights for policy 0, policy_version 149278 (0.0009) [2023-12-26 16:27:16,410][105620] Updated weights for policy 1, policy_version 150042 (0.0007) [2023-12-26 16:27:16,440][105692] Updated weights for policy 0, policy_version 149288 (0.0008) [2023-12-26 16:27:16,454][105620] Updated weights for policy 1, policy_version 150052 (0.0007) [2023-12-26 16:27:16,502][105692] Updated weights for policy 0, policy_version 149298 (0.0010) [2023-12-26 16:27:17,216][105620] Updated weights for policy 1, policy_version 150062 (0.0008) [2023-12-26 16:27:17,252][105692] Updated weights for policy 0, policy_version 149308 (0.0008) [2023-12-26 16:27:17,267][105620] Updated weights for policy 1, policy_version 150072 (0.0008) [2023-12-26 16:27:17,304][105692] Updated weights for policy 0, policy_version 149318 (0.0009) [2023-12-26 16:27:17,325][105620] Updated weights for policy 1, policy_version 150082 (0.0008) [2023-12-26 16:27:17,356][105692] Updated weights for policy 0, policy_version 149328 (0.0010) [2023-12-26 16:27:17,983][105692] Updated weights for policy 0, policy_version 149338 (0.0006) [2023-12-26 16:27:18,035][105692] Updated weights for policy 0, policy_version 149348 (0.0006) [2023-12-26 16:27:18,051][105620] Updated weights for policy 1, policy_version 150092 (0.0007) [2023-12-26 16:27:18,092][105692] Updated weights for policy 0, policy_version 149358 (0.0005) [2023-12-26 16:27:18,114][105620] Updated weights for policy 1, policy_version 150102 (0.0008) [2023-12-26 16:27:18,151][105692] Updated weights for policy 0, policy_version 149368 (0.0011) [2023-12-26 16:27:18,173][105620] Updated weights for policy 1, policy_version 150112 (0.0006) [2023-12-26 16:27:18,750][105692] Updated weights for policy 0, policy_version 149378 (0.0007) [2023-12-26 16:27:18,814][105692] Updated weights for policy 0, policy_version 149388 (0.0008) [2023-12-26 16:27:18,874][105692] Updated weights for policy 0, policy_version 149398 (0.0008) [2023-12-26 16:27:18,911][105620] Updated weights for policy 1, policy_version 150122 (0.0008) [2023-12-26 16:27:18,974][105620] Updated weights for policy 1, policy_version 150132 (0.0008) [2023-12-26 16:27:19,034][105620] Updated weights for policy 1, policy_version 150142 (0.0008) [2023-12-26 16:27:19,099][105620] Updated weights for policy 1, policy_version 150152 (0.0008) [2023-12-26 16:27:19,566][105692] Updated weights for policy 0, policy_version 149408 (0.0010) [2023-12-26 16:27:19,628][105692] Updated weights for policy 0, policy_version 149418 (0.0010) [2023-12-26 16:27:19,684][105692] Updated weights for policy 0, policy_version 149428 (0.0010) [2023-12-26 16:27:19,806][105620] Updated weights for policy 1, policy_version 150162 (0.0009) [2023-12-26 16:27:19,873][105620] Updated weights for policy 1, policy_version 150172 (0.0009) [2023-12-26 16:27:19,947][105620] Updated weights for policy 1, policy_version 150182 (0.0009) [2023-12-26 16:27:20,439][105692] Updated weights for policy 0, policy_version 149438 (0.0011) [2023-12-26 16:27:20,491][105692] Updated weights for policy 0, policy_version 149448 (0.0010) [2023-12-26 16:27:20,546][105692] Updated weights for policy 0, policy_version 149458 (0.0010) [2023-12-26 16:27:20,684][105620] Updated weights for policy 1, policy_version 150192 (0.0007) [2023-12-26 16:27:20,747][105620] Updated weights for policy 1, policy_version 150202 (0.0007) [2023-12-26 16:27:20,811][105620] Updated weights for policy 1, policy_version 150212 (0.0007) [2023-12-26 16:27:21,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 76734464. Throughput: 0: 9767.0, 1: 9698.1. Samples: 76723944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:27:21,062][104569] Avg episode reward: [(0, '7083.163'), (1, '8976.758')] [2023-12-26 16:27:21,256][105692] Updated weights for policy 0, policy_version 149468 (0.0010) [2023-12-26 16:27:21,322][105692] Updated weights for policy 0, policy_version 149478 (0.0011) [2023-12-26 16:27:21,390][105692] Updated weights for policy 0, policy_version 149488 (0.0012) [2023-12-26 16:27:21,460][105620] Updated weights for policy 1, policy_version 150222 (0.0007) [2023-12-26 16:27:21,512][105620] Updated weights for policy 1, policy_version 150232 (0.0008) [2023-12-26 16:27:21,572][105620] Updated weights for policy 1, policy_version 150242 (0.0009) [2023-12-26 16:27:22,150][105692] Updated weights for policy 0, policy_version 149498 (0.0011) [2023-12-26 16:27:22,223][105692] Updated weights for policy 0, policy_version 149508 (0.0010) [2023-12-26 16:27:22,250][105620] Updated weights for policy 1, policy_version 150252 (0.0007) [2023-12-26 16:27:22,287][105692] Updated weights for policy 0, policy_version 149518 (0.0011) [2023-12-26 16:27:22,313][105620] Updated weights for policy 1, policy_version 150262 (0.0009) [2023-12-26 16:27:22,350][105692] Updated weights for policy 0, policy_version 149528 (0.0011) [2023-12-26 16:27:22,379][105620] Updated weights for policy 1, policy_version 150272 (0.0008) [2023-12-26 16:27:23,072][105692] Updated weights for policy 0, policy_version 149538 (0.0010) [2023-12-26 16:27:23,127][105692] Updated weights for policy 0, policy_version 149548 (0.0010) [2023-12-26 16:27:23,143][105620] Updated weights for policy 1, policy_version 150282 (0.0008) [2023-12-26 16:27:23,182][105692] Updated weights for policy 0, policy_version 149558 (0.0010) [2023-12-26 16:27:23,197][105620] Updated weights for policy 1, policy_version 150292 (0.0006) [2023-12-26 16:27:23,254][105620] Updated weights for policy 1, policy_version 150302 (0.0006) [2023-12-26 16:27:23,316][105620] Updated weights for policy 1, policy_version 150312 (0.0008) [2023-12-26 16:27:23,949][105692] Updated weights for policy 0, policy_version 149568 (0.0010) [2023-12-26 16:27:24,000][105692] Updated weights for policy 0, policy_version 149578 (0.0010) [2023-12-26 16:27:24,030][105620] Updated weights for policy 1, policy_version 150322 (0.0008) [2023-12-26 16:27:24,061][105692] Updated weights for policy 0, policy_version 149588 (0.0010) [2023-12-26 16:27:24,075][105620] Updated weights for policy 1, policy_version 150332 (0.0007) [2023-12-26 16:27:24,122][105620] Updated weights for policy 1, policy_version 150342 (0.0008) [2023-12-26 16:27:24,807][105692] Updated weights for policy 0, policy_version 149598 (0.0010) [2023-12-26 16:27:24,855][105692] Updated weights for policy 0, policy_version 149608 (0.0010) [2023-12-26 16:27:24,885][105620] Updated weights for policy 1, policy_version 150352 (0.0007) [2023-12-26 16:27:24,912][105692] Updated weights for policy 0, policy_version 149618 (0.0010) [2023-12-26 16:27:24,939][105620] Updated weights for policy 1, policy_version 150362 (0.0008) [2023-12-26 16:27:24,996][105620] Updated weights for policy 1, policy_version 150372 (0.0010) [2023-12-26 16:27:25,461][105692] Updated weights for policy 0, policy_version 149628 (0.0005) [2023-12-26 16:27:25,520][105692] Updated weights for policy 0, policy_version 149638 (0.0008) [2023-12-26 16:27:25,575][105692] Updated weights for policy 0, policy_version 149648 (0.0010) [2023-12-26 16:27:25,771][105620] Updated weights for policy 1, policy_version 150382 (0.0009) [2023-12-26 16:27:25,820][105620] Updated weights for policy 1, policy_version 150392 (0.0007) [2023-12-26 16:27:25,878][105620] Updated weights for policy 1, policy_version 150402 (0.0005) [2023-12-26 16:27:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 76832768. Throughput: 0: 9726.2, 1: 9714.4. Samples: 76839968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:27:26,063][104569] Avg episode reward: [(0, '8213.331'), (1, '8975.182')] [2023-12-26 16:27:26,121][105692] Updated weights for policy 0, policy_version 149658 (0.0009) [2023-12-26 16:27:26,178][105692] Updated weights for policy 0, policy_version 149668 (0.0005) [2023-12-26 16:27:26,232][105692] Updated weights for policy 0, policy_version 149678 (0.0005) [2023-12-26 16:27:26,287][105692] Updated weights for policy 0, policy_version 149688 (0.0005) [2023-12-26 16:27:26,488][105620] Updated weights for policy 1, policy_version 150412 (0.0007) [2023-12-26 16:27:26,545][105620] Updated weights for policy 1, policy_version 150422 (0.0010) [2023-12-26 16:27:26,590][105620] Updated weights for policy 1, policy_version 150432 (0.0010) [2023-12-26 16:27:26,848][105692] Updated weights for policy 0, policy_version 149698 (0.0010) [2023-12-26 16:27:26,892][105692] Updated weights for policy 0, policy_version 149708 (0.0010) [2023-12-26 16:27:26,943][105692] Updated weights for policy 0, policy_version 149718 (0.0010) [2023-12-26 16:27:27,221][105620] Updated weights for policy 1, policy_version 150442 (0.0010) [2023-12-26 16:27:27,280][105620] Updated weights for policy 1, policy_version 150452 (0.0010) [2023-12-26 16:27:27,336][105620] Updated weights for policy 1, policy_version 150462 (0.0010) [2023-12-26 16:27:27,393][105620] Updated weights for policy 1, policy_version 150472 (0.0010) [2023-12-26 16:27:27,610][105692] Updated weights for policy 0, policy_version 149728 (0.0007) [2023-12-26 16:27:27,660][105692] Updated weights for policy 0, policy_version 149738 (0.0005) [2023-12-26 16:27:27,708][105692] Updated weights for policy 0, policy_version 149748 (0.0006) [2023-12-26 16:27:28,114][105620] Updated weights for policy 1, policy_version 150482 (0.0010) [2023-12-26 16:27:28,171][105620] Updated weights for policy 1, policy_version 150492 (0.0010) [2023-12-26 16:27:28,228][105620] Updated weights for policy 1, policy_version 150502 (0.0010) [2023-12-26 16:27:28,380][105692] Updated weights for policy 0, policy_version 149758 (0.0007) [2023-12-26 16:27:28,445][105692] Updated weights for policy 0, policy_version 149768 (0.0009) [2023-12-26 16:27:28,499][105692] Updated weights for policy 0, policy_version 149778 (0.0010) [2023-12-26 16:27:28,927][105620] Updated weights for policy 1, policy_version 150512 (0.0007) [2023-12-26 16:27:28,976][105620] Updated weights for policy 1, policy_version 150522 (0.0005) [2023-12-26 16:27:29,022][105620] Updated weights for policy 1, policy_version 150532 (0.0005) [2023-12-26 16:27:29,286][105692] Updated weights for policy 0, policy_version 149788 (0.0010) [2023-12-26 16:27:29,343][105692] Updated weights for policy 0, policy_version 149798 (0.0008) [2023-12-26 16:27:29,401][105692] Updated weights for policy 0, policy_version 149808 (0.0009) [2023-12-26 16:27:29,621][105620] Updated weights for policy 1, policy_version 150542 (0.0007) [2023-12-26 16:27:29,684][105620] Updated weights for policy 1, policy_version 150552 (0.0007) [2023-12-26 16:27:29,747][105620] Updated weights for policy 1, policy_version 150562 (0.0008) [2023-12-26 16:27:30,192][105692] Updated weights for policy 0, policy_version 149818 (0.0010) [2023-12-26 16:27:30,249][105692] Updated weights for policy 0, policy_version 149828 (0.0009) [2023-12-26 16:27:30,300][105692] Updated weights for policy 0, policy_version 149838 (0.0009) [2023-12-26 16:27:30,355][105692] Updated weights for policy 0, policy_version 149848 (0.0009) [2023-12-26 16:27:30,455][105620] Updated weights for policy 1, policy_version 150572 (0.0008) [2023-12-26 16:27:30,504][105620] Updated weights for policy 1, policy_version 150582 (0.0008) [2023-12-26 16:27:30,561][105620] Updated weights for policy 1, policy_version 150592 (0.0005) [2023-12-26 16:27:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.4, 300 sec: 19494.2). Total num frames: 76931072. Throughput: 0: 9804.0, 1: 9773.9. Samples: 76904044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:27:31,062][104569] Avg episode reward: [(0, '8915.102'), (1, '8708.762')] [2023-12-26 16:27:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000149848_38371328.pth... [2023-12-26 16:27:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000150600_38559744.pth... [2023-12-26 16:27:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000149448_38264832.pth [2023-12-26 16:27:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000148728_38084608.pth [2023-12-26 16:27:31,142][105620] Updated weights for policy 1, policy_version 150602 (0.0006) [2023-12-26 16:27:31,206][105620] Updated weights for policy 1, policy_version 150612 (0.0007) [2023-12-26 16:27:31,210][105692] Updated weights for policy 0, policy_version 149858 (0.0008) [2023-12-26 16:27:31,268][105620] Updated weights for policy 1, policy_version 150622 (0.0009) [2023-12-26 16:27:31,273][105692] Updated weights for policy 0, policy_version 149868 (0.0006) [2023-12-26 16:27:31,322][105620] Updated weights for policy 1, policy_version 150632 (0.0008) [2023-12-26 16:27:31,329][105692] Updated weights for policy 0, policy_version 149878 (0.0007) [2023-12-26 16:27:32,037][105620] Updated weights for policy 1, policy_version 150642 (0.0010) [2023-12-26 16:27:32,103][105620] Updated weights for policy 1, policy_version 150652 (0.0008) [2023-12-26 16:27:32,139][105692] Updated weights for policy 0, policy_version 149888 (0.0009) [2023-12-26 16:27:32,164][105620] Updated weights for policy 1, policy_version 150662 (0.0006) [2023-12-26 16:27:32,190][105692] Updated weights for policy 0, policy_version 149898 (0.0008) [2023-12-26 16:27:32,245][105692] Updated weights for policy 0, policy_version 149908 (0.0010) [2023-12-26 16:27:32,856][105620] Updated weights for policy 1, policy_version 150672 (0.0009) [2023-12-26 16:27:32,911][105620] Updated weights for policy 1, policy_version 150682 (0.0009) [2023-12-26 16:27:32,928][105692] Updated weights for policy 0, policy_version 149918 (0.0007) [2023-12-26 16:27:32,962][105620] Updated weights for policy 1, policy_version 150692 (0.0010) [2023-12-26 16:27:32,977][105692] Updated weights for policy 0, policy_version 149928 (0.0006) [2023-12-26 16:27:33,026][105692] Updated weights for policy 0, policy_version 149938 (0.0006) [2023-12-26 16:27:33,670][105620] Updated weights for policy 1, policy_version 150702 (0.0007) [2023-12-26 16:27:33,715][105692] Updated weights for policy 0, policy_version 149948 (0.0007) [2023-12-26 16:27:33,720][105620] Updated weights for policy 1, policy_version 150712 (0.0005) [2023-12-26 16:27:33,761][105692] Updated weights for policy 0, policy_version 149958 (0.0005) [2023-12-26 16:27:33,781][105620] Updated weights for policy 1, policy_version 150722 (0.0005) [2023-12-26 16:27:33,818][105692] Updated weights for policy 0, policy_version 149968 (0.0007) [2023-12-26 16:27:34,476][105620] Updated weights for policy 1, policy_version 150732 (0.0010) [2023-12-26 16:27:34,523][105692] Updated weights for policy 0, policy_version 149978 (0.0008) [2023-12-26 16:27:34,546][105620] Updated weights for policy 1, policy_version 150742 (0.0011) [2023-12-26 16:27:34,589][105692] Updated weights for policy 0, policy_version 149988 (0.0005) [2023-12-26 16:27:34,605][105620] Updated weights for policy 1, policy_version 150752 (0.0011) [2023-12-26 16:27:34,656][105692] Updated weights for policy 0, policy_version 149998 (0.0007) [2023-12-26 16:27:34,717][105692] Updated weights for policy 0, policy_version 150008 (0.0008) [2023-12-26 16:27:35,254][105620] Updated weights for policy 1, policy_version 150762 (0.0008) [2023-12-26 16:27:35,304][105620] Updated weights for policy 1, policy_version 150772 (0.0005) [2023-12-26 16:27:35,363][105620] Updated weights for policy 1, policy_version 150782 (0.0011) [2023-12-26 16:27:35,379][105692] Updated weights for policy 0, policy_version 150018 (0.0010) [2023-12-26 16:27:35,424][105620] Updated weights for policy 1, policy_version 150792 (0.0011) [2023-12-26 16:27:35,443][105692] Updated weights for policy 0, policy_version 150028 (0.0006) [2023-12-26 16:27:35,512][105692] Updated weights for policy 0, policy_version 150038 (0.0008) [2023-12-26 16:27:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.8, 300 sec: 19521.9). Total num frames: 77029376. Throughput: 0: 9869.8, 1: 9757.3. Samples: 77021956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:27:36,063][104569] Avg episode reward: [(0, '9175.912'), (1, '8446.488')] [2023-12-26 16:27:36,065][105620] Updated weights for policy 1, policy_version 150802 (0.0005) [2023-12-26 16:27:36,129][105620] Updated weights for policy 1, policy_version 150812 (0.0007) [2023-12-26 16:27:36,188][105620] Updated weights for policy 1, policy_version 150822 (0.0011) [2023-12-26 16:27:36,306][105692] Updated weights for policy 0, policy_version 150048 (0.0010) [2023-12-26 16:27:36,358][105692] Updated weights for policy 0, policy_version 150058 (0.0010) [2023-12-26 16:27:36,414][105692] Updated weights for policy 0, policy_version 150068 (0.0010) [2023-12-26 16:27:36,901][105620] Updated weights for policy 1, policy_version 150832 (0.0008) [2023-12-26 16:27:36,965][105620] Updated weights for policy 1, policy_version 150842 (0.0008) [2023-12-26 16:27:37,023][105620] Updated weights for policy 1, policy_version 150852 (0.0007) [2023-12-26 16:27:37,125][105692] Updated weights for policy 0, policy_version 150078 (0.0011) [2023-12-26 16:27:37,177][105692] Updated weights for policy 0, policy_version 150088 (0.0010) [2023-12-26 16:27:37,232][105692] Updated weights for policy 0, policy_version 150098 (0.0010) [2023-12-26 16:27:37,749][105620] Updated weights for policy 1, policy_version 150862 (0.0008) [2023-12-26 16:27:37,809][105620] Updated weights for policy 1, policy_version 150872 (0.0008) [2023-12-26 16:27:37,861][105620] Updated weights for policy 1, policy_version 150882 (0.0008) [2023-12-26 16:27:37,991][105692] Updated weights for policy 0, policy_version 150108 (0.0010) [2023-12-26 16:27:38,039][105692] Updated weights for policy 0, policy_version 150118 (0.0010) [2023-12-26 16:27:38,087][105692] Updated weights for policy 0, policy_version 150128 (0.0010) [2023-12-26 16:27:38,641][105620] Updated weights for policy 1, policy_version 150892 (0.0009) [2023-12-26 16:27:38,689][105620] Updated weights for policy 1, policy_version 150903 (0.0008) [2023-12-26 16:27:38,735][105620] Updated weights for policy 1, policy_version 150913 (0.0008) [2023-12-26 16:27:38,803][105692] Updated weights for policy 0, policy_version 150138 (0.0010) [2023-12-26 16:27:38,863][105692] Updated weights for policy 0, policy_version 150148 (0.0011) [2023-12-26 16:27:38,928][105692] Updated weights for policy 0, policy_version 150158 (0.0011) [2023-12-26 16:27:38,999][105692] Updated weights for policy 0, policy_version 150168 (0.0008) [2023-12-26 16:27:39,511][105620] Updated weights for policy 1, policy_version 150923 (0.0008) [2023-12-26 16:27:39,582][105620] Updated weights for policy 1, policy_version 150933 (0.0006) [2023-12-26 16:27:39,643][105692] Updated weights for policy 0, policy_version 150178 (0.0010) [2023-12-26 16:27:39,650][105620] Updated weights for policy 1, policy_version 150943 (0.0006) [2023-12-26 16:27:39,702][105692] Updated weights for policy 0, policy_version 150188 (0.0009) [2023-12-26 16:27:39,771][105692] Updated weights for policy 0, policy_version 150198 (0.0005) [2023-12-26 16:27:40,303][105620] Updated weights for policy 1, policy_version 150953 (0.0006) [2023-12-26 16:27:40,365][105620] Updated weights for policy 1, policy_version 150963 (0.0006) [2023-12-26 16:27:40,428][105620] Updated weights for policy 1, policy_version 150973 (0.0008) [2023-12-26 16:27:40,487][105620] Updated weights for policy 1, policy_version 150983 (0.0007) [2023-12-26 16:27:40,523][105692] Updated weights for policy 0, policy_version 150208 (0.0009) [2023-12-26 16:27:40,590][105692] Updated weights for policy 0, policy_version 150218 (0.0010) [2023-12-26 16:27:40,660][105692] Updated weights for policy 0, policy_version 150228 (0.0011) [2023-12-26 16:27:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 77127680. Throughput: 0: 9743.4, 1: 9875.6. Samples: 77138620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:27:41,063][104569] Avg episode reward: [(0, '9176.077'), (1, '8893.403')] [2023-12-26 16:27:41,096][105620] Updated weights for policy 1, policy_version 150993 (0.0010) [2023-12-26 16:27:41,156][105620] Updated weights for policy 1, policy_version 151003 (0.0008) [2023-12-26 16:27:41,216][105620] Updated weights for policy 1, policy_version 151013 (0.0008) [2023-12-26 16:27:41,391][105692] Updated weights for policy 0, policy_version 150238 (0.0009) [2023-12-26 16:27:41,454][105692] Updated weights for policy 0, policy_version 150248 (0.0006) [2023-12-26 16:27:41,514][105692] Updated weights for policy 0, policy_version 150258 (0.0007) [2023-12-26 16:27:42,025][105620] Updated weights for policy 1, policy_version 151023 (0.0009) [2023-12-26 16:27:42,091][105620] Updated weights for policy 1, policy_version 151033 (0.0010) [2023-12-26 16:27:42,125][105692] Updated weights for policy 0, policy_version 150268 (0.0007) [2023-12-26 16:27:42,155][105620] Updated weights for policy 1, policy_version 151043 (0.0011) [2023-12-26 16:27:42,186][105692] Updated weights for policy 0, policy_version 150278 (0.0006) [2023-12-26 16:27:42,246][105692] Updated weights for policy 0, policy_version 150288 (0.0008) [2023-12-26 16:27:42,936][105620] Updated weights for policy 1, policy_version 151053 (0.0011) [2023-12-26 16:27:42,991][105620] Updated weights for policy 1, policy_version 151063 (0.0010) [2023-12-26 16:27:43,021][105692] Updated weights for policy 0, policy_version 150298 (0.0008) [2023-12-26 16:27:43,054][105620] Updated weights for policy 1, policy_version 151073 (0.0011) [2023-12-26 16:27:43,080][105692] Updated weights for policy 0, policy_version 150308 (0.0006) [2023-12-26 16:27:43,135][105692] Updated weights for policy 0, policy_version 150318 (0.0007) [2023-12-26 16:27:43,179][105692] Updated weights for policy 0, policy_version 150328 (0.0008) [2023-12-26 16:27:43,792][105620] Updated weights for policy 1, policy_version 151083 (0.0011) [2023-12-26 16:27:43,854][105620] Updated weights for policy 1, policy_version 151093 (0.0011) [2023-12-26 16:27:43,917][105620] Updated weights for policy 1, policy_version 151103 (0.0010) [2023-12-26 16:27:43,954][105692] Updated weights for policy 0, policy_version 150338 (0.0007) [2023-12-26 16:27:44,005][105692] Updated weights for policy 0, policy_version 150348 (0.0008) [2023-12-26 16:27:44,063][105692] Updated weights for policy 0, policy_version 150358 (0.0008) [2023-12-26 16:27:44,691][105620] Updated weights for policy 1, policy_version 151113 (0.0011) [2023-12-26 16:27:44,749][105620] Updated weights for policy 1, policy_version 151123 (0.0010) [2023-12-26 16:27:44,809][105620] Updated weights for policy 1, policy_version 151133 (0.0010) [2023-12-26 16:27:44,837][105692] Updated weights for policy 0, policy_version 150368 (0.0010) [2023-12-26 16:27:44,865][105620] Updated weights for policy 1, policy_version 151143 (0.0011) [2023-12-26 16:27:44,893][105692] Updated weights for policy 0, policy_version 150378 (0.0011) [2023-12-26 16:27:44,960][105692] Updated weights for policy 0, policy_version 150388 (0.0008) [2023-12-26 16:27:45,583][105620] Updated weights for policy 1, policy_version 151153 (0.0009) [2023-12-26 16:27:45,639][105620] Updated weights for policy 1, policy_version 151163 (0.0010) [2023-12-26 16:27:45,699][105692] Updated weights for policy 0, policy_version 150398 (0.0008) [2023-12-26 16:27:45,701][105620] Updated weights for policy 1, policy_version 151173 (0.0011) [2023-12-26 16:27:45,749][105692] Updated weights for policy 0, policy_version 150408 (0.0007) [2023-12-26 16:27:45,796][105692] Updated weights for policy 0, policy_version 150418 (0.0008) [2023-12-26 16:27:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 77225984. Throughput: 0: 9767.8, 1: 9832.6. Samples: 77194656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:27:46,063][104569] Avg episode reward: [(0, '9266.669'), (1, '8983.542')] [2023-12-26 16:27:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000150424_38518784.pth... [2023-12-26 16:27:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000151176_38707200.pth... [2023-12-26 16:27:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000149272_38223872.pth [2023-12-26 16:27:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000150024_38412288.pth [2023-12-26 16:27:46,347][105620] Updated weights for policy 1, policy_version 151183 (0.0009) [2023-12-26 16:27:46,394][105620] Updated weights for policy 1, policy_version 151193 (0.0009) [2023-12-26 16:27:46,448][105620] Updated weights for policy 1, policy_version 151204 (0.0009) [2023-12-26 16:27:46,493][105692] Updated weights for policy 0, policy_version 150428 (0.0006) [2023-12-26 16:27:46,551][105692] Updated weights for policy 0, policy_version 150438 (0.0005) [2023-12-26 16:27:46,606][105692] Updated weights for policy 0, policy_version 150448 (0.0005) [2023-12-26 16:27:47,261][105620] Updated weights for policy 1, policy_version 151214 (0.0009) [2023-12-26 16:27:47,270][105692] Updated weights for policy 0, policy_version 150458 (0.0009) [2023-12-26 16:27:47,320][105620] Updated weights for policy 1, policy_version 151224 (0.0008) [2023-12-26 16:27:47,333][105692] Updated weights for policy 0, policy_version 150468 (0.0011) [2023-12-26 16:27:47,373][105620] Updated weights for policy 1, policy_version 151234 (0.0007) [2023-12-26 16:27:47,383][105692] Updated weights for policy 0, policy_version 150478 (0.0011) [2023-12-26 16:27:47,445][105692] Updated weights for policy 0, policy_version 150488 (0.0009) [2023-12-26 16:27:48,004][105620] Updated weights for policy 1, policy_version 151244 (0.0006) [2023-12-26 16:27:48,057][105620] Updated weights for policy 1, policy_version 151254 (0.0005) [2023-12-26 16:27:48,111][105620] Updated weights for policy 1, policy_version 151264 (0.0005) [2023-12-26 16:27:48,223][105692] Updated weights for policy 0, policy_version 150498 (0.0011) [2023-12-26 16:27:48,270][105692] Updated weights for policy 0, policy_version 150508 (0.0010) [2023-12-26 16:27:48,322][105692] Updated weights for policy 0, policy_version 150518 (0.0010) [2023-12-26 16:27:48,684][105620] Updated weights for policy 1, policy_version 151274 (0.0007) [2023-12-26 16:27:48,736][105620] Updated weights for policy 1, policy_version 151284 (0.0010) [2023-12-26 16:27:48,793][105620] Updated weights for policy 1, policy_version 151294 (0.0006) [2023-12-26 16:27:48,853][105620] Updated weights for policy 1, policy_version 151304 (0.0005) [2023-12-26 16:27:48,962][105692] Updated weights for policy 0, policy_version 150528 (0.0006) [2023-12-26 16:27:49,029][105692] Updated weights for policy 0, policy_version 150538 (0.0005) [2023-12-26 16:27:49,096][105692] Updated weights for policy 0, policy_version 150548 (0.0006) [2023-12-26 16:27:49,525][105620] Updated weights for policy 1, policy_version 151314 (0.0005) [2023-12-26 16:27:49,571][105620] Updated weights for policy 1, policy_version 151324 (0.0005) [2023-12-26 16:27:49,631][105620] Updated weights for policy 1, policy_version 151334 (0.0009) [2023-12-26 16:27:49,747][105692] Updated weights for policy 0, policy_version 150558 (0.0010) [2023-12-26 16:27:49,816][105692] Updated weights for policy 0, policy_version 150568 (0.0009) [2023-12-26 16:27:49,878][105692] Updated weights for policy 0, policy_version 150578 (0.0009) [2023-12-26 16:27:50,272][105620] Updated weights for policy 1, policy_version 151344 (0.0010) [2023-12-26 16:27:50,332][105620] Updated weights for policy 1, policy_version 151354 (0.0010) [2023-12-26 16:27:50,391][105620] Updated weights for policy 1, policy_version 151364 (0.0011) [2023-12-26 16:27:50,579][105692] Updated weights for policy 0, policy_version 150588 (0.0010) [2023-12-26 16:27:50,639][105692] Updated weights for policy 0, policy_version 150598 (0.0007) [2023-12-26 16:27:50,694][105692] Updated weights for policy 0, policy_version 150608 (0.0007) [2023-12-26 16:27:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 77324288. Throughput: 0: 9714.6, 1: 9885.4. Samples: 77314376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:27:51,062][104569] Avg episode reward: [(0, '9357.107'), (1, '8709.934')] [2023-12-26 16:27:51,077][105620] Updated weights for policy 1, policy_version 151374 (0.0011) [2023-12-26 16:27:51,148][105620] Updated weights for policy 1, policy_version 151384 (0.0011) [2023-12-26 16:27:51,217][105620] Updated weights for policy 1, policy_version 151394 (0.0009) [2023-12-26 16:27:51,276][105692] Updated weights for policy 0, policy_version 150618 (0.0005) [2023-12-26 16:27:51,333][105692] Updated weights for policy 0, policy_version 150628 (0.0006) [2023-12-26 16:27:51,400][105692] Updated weights for policy 0, policy_version 150638 (0.0009) [2023-12-26 16:27:51,452][105692] Updated weights for policy 0, policy_version 150648 (0.0009) [2023-12-26 16:27:51,973][105620] Updated weights for policy 1, policy_version 151404 (0.0010) [2023-12-26 16:27:52,033][105620] Updated weights for policy 1, policy_version 151414 (0.0009) [2023-12-26 16:27:52,091][105620] Updated weights for policy 1, policy_version 151424 (0.0009) [2023-12-26 16:27:52,197][105692] Updated weights for policy 0, policy_version 150658 (0.0009) [2023-12-26 16:27:52,267][105692] Updated weights for policy 0, policy_version 150668 (0.0007) [2023-12-26 16:27:52,337][105692] Updated weights for policy 0, policy_version 150678 (0.0008) [2023-12-26 16:27:52,794][105620] Updated weights for policy 1, policy_version 151434 (0.0008) [2023-12-26 16:27:52,863][105620] Updated weights for policy 1, policy_version 151444 (0.0005) [2023-12-26 16:27:52,927][105620] Updated weights for policy 1, policy_version 151454 (0.0008) [2023-12-26 16:27:52,945][105692] Updated weights for policy 0, policy_version 150688 (0.0006) [2023-12-26 16:27:52,990][105620] Updated weights for policy 1, policy_version 151464 (0.0009) [2023-12-26 16:27:52,991][105692] Updated weights for policy 0, policy_version 150698 (0.0005) [2023-12-26 16:27:53,049][105692] Updated weights for policy 0, policy_version 150708 (0.0006) [2023-12-26 16:27:53,723][105620] Updated weights for policy 1, policy_version 151474 (0.0007) [2023-12-26 16:27:53,724][105692] Updated weights for policy 0, policy_version 150718 (0.0010) [2023-12-26 16:27:53,779][105692] Updated weights for policy 0, policy_version 150728 (0.0010) [2023-12-26 16:27:53,781][105620] Updated weights for policy 1, policy_version 151484 (0.0006) [2023-12-26 16:27:53,832][105620] Updated weights for policy 1, policy_version 151494 (0.0007) [2023-12-26 16:27:53,836][105692] Updated weights for policy 0, policy_version 150738 (0.0010) [2023-12-26 16:27:54,585][105692] Updated weights for policy 0, policy_version 150748 (0.0010) [2023-12-26 16:27:54,601][105620] Updated weights for policy 1, policy_version 151504 (0.0006) [2023-12-26 16:27:54,641][105692] Updated weights for policy 0, policy_version 150758 (0.0010) [2023-12-26 16:27:54,647][105620] Updated weights for policy 1, policy_version 151514 (0.0006) [2023-12-26 16:27:54,693][105692] Updated weights for policy 0, policy_version 150768 (0.0010) [2023-12-26 16:27:54,702][105620] Updated weights for policy 1, policy_version 151524 (0.0005) [2023-12-26 16:27:55,345][105620] Updated weights for policy 1, policy_version 151534 (0.0007) [2023-12-26 16:27:55,408][105620] Updated weights for policy 1, policy_version 151544 (0.0009) [2023-12-26 16:27:55,451][105692] Updated weights for policy 0, policy_version 150778 (0.0010) [2023-12-26 16:27:55,466][105620] Updated weights for policy 1, policy_version 151554 (0.0010) [2023-12-26 16:27:55,505][105692] Updated weights for policy 0, policy_version 150788 (0.0006) [2023-12-26 16:27:55,555][105692] Updated weights for policy 0, policy_version 150798 (0.0008) [2023-12-26 16:27:55,609][105692] Updated weights for policy 0, policy_version 150808 (0.0005) [2023-12-26 16:27:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 77422592. Throughput: 0: 9733.6, 1: 9904.2. Samples: 77433052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:27:56,063][104569] Avg episode reward: [(0, '2907.248'), (1, '2589.188')] [2023-12-26 16:27:56,115][105620] Updated weights for policy 1, policy_version 151564 (0.0009) [2023-12-26 16:27:56,181][105620] Updated weights for policy 1, policy_version 151574 (0.0011) [2023-12-26 16:27:56,216][105692] Updated weights for policy 0, policy_version 150818 (0.0007) [2023-12-26 16:27:56,250][105620] Updated weights for policy 1, policy_version 151584 (0.0011) [2023-12-26 16:27:56,270][105692] Updated weights for policy 0, policy_version 150828 (0.0007) [2023-12-26 16:27:56,334][105692] Updated weights for policy 0, policy_version 150838 (0.0007) [2023-12-26 16:27:56,873][105692] Updated weights for policy 0, policy_version 150848 (0.0005) [2023-12-26 16:27:56,923][105692] Updated weights for policy 0, policy_version 150858 (0.0005) [2023-12-26 16:27:56,976][105620] Updated weights for policy 1, policy_version 151594 (0.0011) [2023-12-26 16:27:56,982][105692] Updated weights for policy 0, policy_version 150868 (0.0006) [2023-12-26 16:27:57,024][105620] Updated weights for policy 1, policy_version 151604 (0.0010) [2023-12-26 16:27:57,068][105620] Updated weights for policy 1, policy_version 151614 (0.0010) [2023-12-26 16:27:57,112][105620] Updated weights for policy 1, policy_version 151624 (0.0010) [2023-12-26 16:27:57,649][105692] Updated weights for policy 0, policy_version 150878 (0.0007) [2023-12-26 16:27:57,703][105692] Updated weights for policy 0, policy_version 150888 (0.0009) [2023-12-26 16:27:57,761][105692] Updated weights for policy 0, policy_version 150898 (0.0008) [2023-12-26 16:27:57,809][105620] Updated weights for policy 1, policy_version 151634 (0.0010) [2023-12-26 16:27:57,866][105620] Updated weights for policy 1, policy_version 151644 (0.0010) [2023-12-26 16:27:57,923][105620] Updated weights for policy 1, policy_version 151654 (0.0010) [2023-12-26 16:27:58,489][105692] Updated weights for policy 0, policy_version 150908 (0.0006) [2023-12-26 16:27:58,557][105692] Updated weights for policy 0, policy_version 150918 (0.0007) [2023-12-26 16:27:58,627][105692] Updated weights for policy 0, policy_version 150928 (0.0009) [2023-12-26 16:27:58,709][105620] Updated weights for policy 1, policy_version 151664 (0.0011) [2023-12-26 16:27:58,774][105620] Updated weights for policy 1, policy_version 151674 (0.0008) [2023-12-26 16:27:58,847][105620] Updated weights for policy 1, policy_version 151684 (0.0007) [2023-12-26 16:27:59,432][105692] Updated weights for policy 0, policy_version 150938 (0.0007) [2023-12-26 16:27:59,481][105692] Updated weights for policy 0, policy_version 150948 (0.0005) [2023-12-26 16:27:59,533][105692] Updated weights for policy 0, policy_version 150958 (0.0008) [2023-12-26 16:27:59,587][105692] Updated weights for policy 0, policy_version 150968 (0.0008) [2023-12-26 16:27:59,657][105620] Updated weights for policy 1, policy_version 151694 (0.0009) [2023-12-26 16:27:59,708][105620] Updated weights for policy 1, policy_version 151704 (0.0010) [2023-12-26 16:27:59,775][105620] Updated weights for policy 1, policy_version 151714 (0.0010) [2023-12-26 16:28:00,257][105692] Updated weights for policy 0, policy_version 150978 (0.0011) [2023-12-26 16:28:00,320][105692] Updated weights for policy 0, policy_version 150988 (0.0011) [2023-12-26 16:28:00,383][105692] Updated weights for policy 0, policy_version 150998 (0.0010) [2023-12-26 16:28:00,518][105620] Updated weights for policy 1, policy_version 151724 (0.0010) [2023-12-26 16:28:00,573][105620] Updated weights for policy 1, policy_version 151734 (0.0008) [2023-12-26 16:28:00,622][105620] Updated weights for policy 1, policy_version 151744 (0.0008) [2023-12-26 16:28:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 77520896. Throughput: 0: 9831.2, 1: 9854.1. Samples: 77493488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:28:01,062][104569] Avg episode reward: [(0, '1480.689'), (1, '2448.544')] [2023-12-26 16:28:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000151752_38854656.pth... [2023-12-26 16:28:01,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000150600_38559744.pth [2023-12-26 16:28:01,106][105692] Updated weights for policy 0, policy_version 151008 (0.0010) [2023-12-26 16:28:01,170][105692] Updated weights for policy 0, policy_version 151018 (0.0012) [2023-12-26 16:28:01,239][105692] Updated weights for policy 0, policy_version 151028 (0.0010) [2023-12-26 16:28:01,258][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000151032_38674432.pth... [2023-12-26 16:28:01,263][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000149848_38371328.pth [2023-12-26 16:28:01,339][105620] Updated weights for policy 1, policy_version 151754 (0.0008) [2023-12-26 16:28:01,407][105620] Updated weights for policy 1, policy_version 151764 (0.0010) [2023-12-26 16:28:01,467][105620] Updated weights for policy 1, policy_version 151774 (0.0008) [2023-12-26 16:28:01,526][105620] Updated weights for policy 1, policy_version 151784 (0.0007) [2023-12-26 16:28:01,958][105692] Updated weights for policy 0, policy_version 151038 (0.0011) [2023-12-26 16:28:02,020][105692] Updated weights for policy 0, policy_version 151048 (0.0010) [2023-12-26 16:28:02,078][105692] Updated weights for policy 0, policy_version 151058 (0.0011) [2023-12-26 16:28:02,176][105620] Updated weights for policy 1, policy_version 151794 (0.0006) [2023-12-26 16:28:02,228][105620] Updated weights for policy 1, policy_version 151804 (0.0008) [2023-12-26 16:28:02,288][105620] Updated weights for policy 1, policy_version 151814 (0.0007) [2023-12-26 16:28:02,814][105692] Updated weights for policy 0, policy_version 151068 (0.0010) [2023-12-26 16:28:02,848][105620] Updated weights for policy 1, policy_version 151824 (0.0005) [2023-12-26 16:28:02,876][105692] Updated weights for policy 0, policy_version 151078 (0.0010) [2023-12-26 16:28:02,898][105620] Updated weights for policy 1, policy_version 151834 (0.0007) [2023-12-26 16:28:02,934][105692] Updated weights for policy 0, policy_version 151088 (0.0010) [2023-12-26 16:28:02,948][105620] Updated weights for policy 1, policy_version 151844 (0.0007) [2023-12-26 16:28:03,609][105620] Updated weights for policy 1, policy_version 151854 (0.0005) [2023-12-26 16:28:03,652][105620] Updated weights for policy 1, policy_version 151864 (0.0006) [2023-12-26 16:28:03,659][105692] Updated weights for policy 0, policy_version 151098 (0.0010) [2023-12-26 16:28:03,709][105620] Updated weights for policy 1, policy_version 151874 (0.0005) [2023-12-26 16:28:03,723][105692] Updated weights for policy 0, policy_version 151108 (0.0008) [2023-12-26 16:28:03,785][105692] Updated weights for policy 0, policy_version 151118 (0.0010) [2023-12-26 16:28:03,846][105692] Updated weights for policy 0, policy_version 151128 (0.0010) [2023-12-26 16:28:04,401][105620] Updated weights for policy 1, policy_version 151884 (0.0007) [2023-12-26 16:28:04,461][105620] Updated weights for policy 1, policy_version 151895 (0.0010) [2023-12-26 16:28:04,486][105692] Updated weights for policy 0, policy_version 151138 (0.0006) [2023-12-26 16:28:04,513][105620] Updated weights for policy 1, policy_version 151905 (0.0008) [2023-12-26 16:28:04,537][105692] Updated weights for policy 0, policy_version 151148 (0.0005) [2023-12-26 16:28:04,582][105692] Updated weights for policy 0, policy_version 151158 (0.0005) [2023-12-26 16:28:05,214][105620] Updated weights for policy 1, policy_version 151915 (0.0008) [2023-12-26 16:28:05,258][105692] Updated weights for policy 0, policy_version 151168 (0.0009) [2023-12-26 16:28:05,274][105620] Updated weights for policy 1, policy_version 151925 (0.0005) [2023-12-26 16:28:05,324][105692] Updated weights for policy 0, policy_version 151178 (0.0010) [2023-12-26 16:28:05,331][105620] Updated weights for policy 1, policy_version 151935 (0.0005) [2023-12-26 16:28:05,389][105692] Updated weights for policy 0, policy_version 151188 (0.0009) [2023-12-26 16:28:05,925][105620] Updated weights for policy 1, policy_version 151945 (0.0006) [2023-12-26 16:28:05,969][105620] Updated weights for policy 1, policy_version 151955 (0.0008) [2023-12-26 16:28:06,016][105620] Updated weights for policy 1, policy_version 151965 (0.0008) [2023-12-26 16:28:06,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 77619200. Throughput: 0: 9845.7, 1: 9893.1. Samples: 77612188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:28:06,062][104569] Avg episode reward: [(0, '1698.214'), (1, '6711.484')] [2023-12-26 16:28:06,063][105620] Updated weights for policy 1, policy_version 151975 (0.0007) [2023-12-26 16:28:06,083][105692] Updated weights for policy 0, policy_version 151198 (0.0008) [2023-12-26 16:28:06,145][105692] Updated weights for policy 0, policy_version 151208 (0.0010) [2023-12-26 16:28:06,210][105692] Updated weights for policy 0, policy_version 151218 (0.0011) [2023-12-26 16:28:06,876][105620] Updated weights for policy 1, policy_version 151985 (0.0008) [2023-12-26 16:28:06,928][105620] Updated weights for policy 1, policy_version 151995 (0.0007) [2023-12-26 16:28:06,966][105692] Updated weights for policy 0, policy_version 151228 (0.0011) [2023-12-26 16:28:06,985][105620] Updated weights for policy 1, policy_version 152005 (0.0009) [2023-12-26 16:28:07,021][105692] Updated weights for policy 0, policy_version 151238 (0.0010) [2023-12-26 16:28:07,078][105692] Updated weights for policy 0, policy_version 151248 (0.0009) [2023-12-26 16:28:07,712][105692] Updated weights for policy 0, policy_version 151258 (0.0005) [2023-12-26 16:28:07,762][105692] Updated weights for policy 0, policy_version 151268 (0.0006) [2023-12-26 16:28:07,809][105620] Updated weights for policy 1, policy_version 152015 (0.0008) [2023-12-26 16:28:07,819][105692] Updated weights for policy 0, policy_version 151278 (0.0007) [2023-12-26 16:28:07,871][105620] Updated weights for policy 1, policy_version 152025 (0.0009) [2023-12-26 16:28:07,873][105692] Updated weights for policy 0, policy_version 151288 (0.0007) [2023-12-26 16:28:07,931][105620] Updated weights for policy 1, policy_version 152035 (0.0008) [2023-12-26 16:28:08,596][105692] Updated weights for policy 0, policy_version 151298 (0.0009) [2023-12-26 16:28:08,643][105692] Updated weights for policy 0, policy_version 151308 (0.0008) [2023-12-26 16:28:08,671][105620] Updated weights for policy 1, policy_version 152045 (0.0008) [2023-12-26 16:28:08,690][105692] Updated weights for policy 0, policy_version 151318 (0.0008) [2023-12-26 16:28:08,718][105620] Updated weights for policy 1, policy_version 152055 (0.0007) [2023-12-26 16:28:08,769][105620] Updated weights for policy 1, policy_version 152065 (0.0009) [2023-12-26 16:28:09,507][105692] Updated weights for policy 0, policy_version 151328 (0.0008) [2023-12-26 16:28:09,570][105692] Updated weights for policy 0, policy_version 151338 (0.0008) [2023-12-26 16:28:09,578][105620] Updated weights for policy 1, policy_version 152075 (0.0009) [2023-12-26 16:28:09,626][105692] Updated weights for policy 0, policy_version 151348 (0.0006) [2023-12-26 16:28:09,636][105620] Updated weights for policy 1, policy_version 152085 (0.0010) [2023-12-26 16:28:09,700][105620] Updated weights for policy 1, policy_version 152095 (0.0008) [2023-12-26 16:28:10,378][105692] Updated weights for policy 0, policy_version 151358 (0.0008) [2023-12-26 16:28:10,430][105620] Updated weights for policy 1, policy_version 152105 (0.0009) [2023-12-26 16:28:10,440][105692] Updated weights for policy 0, policy_version 151368 (0.0008) [2023-12-26 16:28:10,495][105620] Updated weights for policy 1, policy_version 152115 (0.0007) [2023-12-26 16:28:10,497][105692] Updated weights for policy 0, policy_version 151378 (0.0007) [2023-12-26 16:28:10,548][105620] Updated weights for policy 1, policy_version 152125 (0.0007) [2023-12-26 16:28:10,603][105620] Updated weights for policy 1, policy_version 152135 (0.0009) [2023-12-26 16:28:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 77717504. Throughput: 0: 9837.1, 1: 9876.1. Samples: 77727060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:28:11,063][104569] Avg episode reward: [(0, '5961.239'), (1, '9347.072')] [2023-12-26 16:28:11,318][105692] Updated weights for policy 0, policy_version 151388 (0.0009) [2023-12-26 16:28:11,375][105620] Updated weights for policy 1, policy_version 152145 (0.0009) [2023-12-26 16:28:11,389][105692] Updated weights for policy 0, policy_version 151398 (0.0008) [2023-12-26 16:28:11,441][105620] Updated weights for policy 1, policy_version 152155 (0.0006) [2023-12-26 16:28:11,456][105692] Updated weights for policy 0, policy_version 151408 (0.0008) [2023-12-26 16:28:11,506][105620] Updated weights for policy 1, policy_version 152165 (0.0011) [2023-12-26 16:28:12,227][105692] Updated weights for policy 0, policy_version 151418 (0.0009) [2023-12-26 16:28:12,262][105620] Updated weights for policy 1, policy_version 152175 (0.0011) [2023-12-26 16:28:12,292][105692] Updated weights for policy 0, policy_version 151428 (0.0006) [2023-12-26 16:28:12,323][105620] Updated weights for policy 1, policy_version 152185 (0.0010) [2023-12-26 16:28:12,360][105692] Updated weights for policy 0, policy_version 151438 (0.0007) [2023-12-26 16:28:12,393][105620] Updated weights for policy 1, policy_version 152195 (0.0010) [2023-12-26 16:28:12,424][105692] Updated weights for policy 0, policy_version 151448 (0.0006) [2023-12-26 16:28:13,017][105620] Updated weights for policy 1, policy_version 152205 (0.0010) [2023-12-26 16:28:13,079][105620] Updated weights for policy 1, policy_version 152215 (0.0009) [2023-12-26 16:28:13,137][105620] Updated weights for policy 1, policy_version 152225 (0.0009) [2023-12-26 16:28:13,223][105692] Updated weights for policy 0, policy_version 151458 (0.0008) [2023-12-26 16:28:13,288][105692] Updated weights for policy 0, policy_version 151468 (0.0010) [2023-12-26 16:28:13,336][105692] Updated weights for policy 0, policy_version 151478 (0.0009) [2023-12-26 16:28:13,846][105620] Updated weights for policy 1, policy_version 152235 (0.0008) [2023-12-26 16:28:13,896][105620] Updated weights for policy 1, policy_version 152245 (0.0009) [2023-12-26 16:28:13,955][105620] Updated weights for policy 1, policy_version 152255 (0.0010) [2023-12-26 16:28:14,044][105692] Updated weights for policy 0, policy_version 151488 (0.0006) [2023-12-26 16:28:14,093][105692] Updated weights for policy 0, policy_version 151498 (0.0007) [2023-12-26 16:28:14,148][105692] Updated weights for policy 0, policy_version 151508 (0.0009) [2023-12-26 16:28:14,747][105620] Updated weights for policy 1, policy_version 152265 (0.0010) [2023-12-26 16:28:14,810][105620] Updated weights for policy 1, policy_version 152275 (0.0008) [2023-12-26 16:28:14,848][105692] Updated weights for policy 0, policy_version 151518 (0.0008) [2023-12-26 16:28:14,870][105620] Updated weights for policy 1, policy_version 152285 (0.0009) [2023-12-26 16:28:14,905][105692] Updated weights for policy 0, policy_version 151528 (0.0007) [2023-12-26 16:28:14,928][105620] Updated weights for policy 1, policy_version 152295 (0.0008) [2023-12-26 16:28:14,968][105692] Updated weights for policy 0, policy_version 151538 (0.0007) [2023-12-26 16:28:15,649][105692] Updated weights for policy 0, policy_version 151548 (0.0010) [2023-12-26 16:28:15,697][105692] Updated weights for policy 0, policy_version 151558 (0.0010) [2023-12-26 16:28:15,698][105620] Updated weights for policy 1, policy_version 152305 (0.0010) [2023-12-26 16:28:15,742][105692] Updated weights for policy 0, policy_version 151568 (0.0010) [2023-12-26 16:28:15,755][105620] Updated weights for policy 1, policy_version 152315 (0.0006) [2023-12-26 16:28:15,811][105620] Updated weights for policy 1, policy_version 152325 (0.0005) [2023-12-26 16:28:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 77815808. Throughput: 0: 9691.8, 1: 9829.5. Samples: 77782504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:28:16,062][104569] Avg episode reward: [(0, '6738.672'), (1, '9347.112')] [2023-12-26 16:28:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000151576_38813696.pth... [2023-12-26 16:28:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000152328_39002112.pth... [2023-12-26 16:28:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000150424_38518784.pth [2023-12-26 16:28:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000151176_38707200.pth [2023-12-26 16:28:16,465][105692] Updated weights for policy 0, policy_version 151578 (0.0010) [2023-12-26 16:28:16,530][105692] Updated weights for policy 0, policy_version 151588 (0.0010) [2023-12-26 16:28:16,539][105620] Updated weights for policy 1, policy_version 152335 (0.0006) [2023-12-26 16:28:16,593][105692] Updated weights for policy 0, policy_version 151598 (0.0010) [2023-12-26 16:28:16,610][105620] Updated weights for policy 1, policy_version 152345 (0.0005) [2023-12-26 16:28:16,658][105692] Updated weights for policy 0, policy_version 151608 (0.0010) [2023-12-26 16:28:16,680][105620] Updated weights for policy 1, policy_version 152355 (0.0006) [2023-12-26 16:28:17,358][105692] Updated weights for policy 0, policy_version 151618 (0.0007) [2023-12-26 16:28:17,366][105620] Updated weights for policy 1, policy_version 152365 (0.0009) [2023-12-26 16:28:17,408][105620] Updated weights for policy 1, policy_version 152375 (0.0008) [2023-12-26 16:28:17,410][105692] Updated weights for policy 0, policy_version 151628 (0.0009) [2023-12-26 16:28:17,459][105692] Updated weights for policy 0, policy_version 151638 (0.0010) [2023-12-26 16:28:17,471][105620] Updated weights for policy 1, policy_version 152385 (0.0008) [2023-12-26 16:28:18,181][105620] Updated weights for policy 1, policy_version 152395 (0.0009) [2023-12-26 16:28:18,201][105692] Updated weights for policy 0, policy_version 151648 (0.0011) [2023-12-26 16:28:18,237][105620] Updated weights for policy 1, policy_version 152405 (0.0011) [2023-12-26 16:28:18,249][105692] Updated weights for policy 0, policy_version 151658 (0.0010) [2023-12-26 16:28:18,294][105692] Updated weights for policy 0, policy_version 151668 (0.0011) [2023-12-26 16:28:18,295][105620] Updated weights for policy 1, policy_version 152415 (0.0005) [2023-12-26 16:28:18,986][105620] Updated weights for policy 1, policy_version 152425 (0.0008) [2023-12-26 16:28:19,041][105620] Updated weights for policy 1, policy_version 152435 (0.0010) [2023-12-26 16:28:19,062][105692] Updated weights for policy 0, policy_version 151678 (0.0010) [2023-12-26 16:28:19,096][105620] Updated weights for policy 1, policy_version 152445 (0.0010) [2023-12-26 16:28:19,120][105692] Updated weights for policy 0, policy_version 151688 (0.0010) [2023-12-26 16:28:19,155][105620] Updated weights for policy 1, policy_version 152455 (0.0010) [2023-12-26 16:28:19,176][105692] Updated weights for policy 0, policy_version 151698 (0.0010) [2023-12-26 16:28:19,920][105620] Updated weights for policy 1, policy_version 152465 (0.0008) [2023-12-26 16:28:19,934][105692] Updated weights for policy 0, policy_version 151708 (0.0011) [2023-12-26 16:28:19,984][105620] Updated weights for policy 1, policy_version 152475 (0.0007) [2023-12-26 16:28:19,990][105692] Updated weights for policy 0, policy_version 151718 (0.0011) [2023-12-26 16:28:20,039][105692] Updated weights for policy 0, policy_version 151728 (0.0011) [2023-12-26 16:28:20,041][105620] Updated weights for policy 1, policy_version 152485 (0.0006) [2023-12-26 16:28:20,795][105620] Updated weights for policy 1, policy_version 152495 (0.0008) [2023-12-26 16:28:20,807][105692] Updated weights for policy 0, policy_version 151738 (0.0011) [2023-12-26 16:28:20,846][105620] Updated weights for policy 1, policy_version 152505 (0.0006) [2023-12-26 16:28:20,867][105692] Updated weights for policy 0, policy_version 151748 (0.0010) [2023-12-26 16:28:20,889][105620] Updated weights for policy 1, policy_version 152515 (0.0008) [2023-12-26 16:28:20,927][105692] Updated weights for policy 0, policy_version 151758 (0.0010) [2023-12-26 16:28:20,990][105692] Updated weights for policy 0, policy_version 151768 (0.0011) [2023-12-26 16:28:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 77914112. Throughput: 0: 9747.4, 1: 9719.2. Samples: 77897948. Policy #0 lag: (min: 27.0, avg: 38.3, max: 59.0) [2023-12-26 16:28:21,062][104569] Avg episode reward: [(0, '7793.969'), (1, '9257.325')] [2023-12-26 16:28:21,735][105620] Updated weights for policy 1, policy_version 152525 (0.0007) [2023-12-26 16:28:21,763][105692] Updated weights for policy 0, policy_version 151778 (0.0007) [2023-12-26 16:28:21,798][105620] Updated weights for policy 1, policy_version 152535 (0.0009) [2023-12-26 16:28:21,828][105692] Updated weights for policy 0, policy_version 151788 (0.0010) [2023-12-26 16:28:21,859][105620] Updated weights for policy 1, policy_version 152545 (0.0011) [2023-12-26 16:28:21,894][105692] Updated weights for policy 0, policy_version 151798 (0.0011) [2023-12-26 16:28:22,605][105620] Updated weights for policy 1, policy_version 152555 (0.0011) [2023-12-26 16:28:22,644][105692] Updated weights for policy 0, policy_version 151808 (0.0008) [2023-12-26 16:28:22,673][105620] Updated weights for policy 1, policy_version 152565 (0.0011) [2023-12-26 16:28:22,705][105692] Updated weights for policy 0, policy_version 151818 (0.0006) [2023-12-26 16:28:22,729][105620] Updated weights for policy 1, policy_version 152575 (0.0011) [2023-12-26 16:28:22,767][105692] Updated weights for policy 0, policy_version 151828 (0.0008) [2023-12-26 16:28:23,438][105692] Updated weights for policy 0, policy_version 151838 (0.0011) [2023-12-26 16:28:23,455][105620] Updated weights for policy 1, policy_version 152585 (0.0011) [2023-12-26 16:28:23,493][105692] Updated weights for policy 0, policy_version 151848 (0.0010) [2023-12-26 16:28:23,514][105620] Updated weights for policy 1, policy_version 152595 (0.0010) [2023-12-26 16:28:23,547][105692] Updated weights for policy 0, policy_version 151858 (0.0010) [2023-12-26 16:28:23,572][105620] Updated weights for policy 1, policy_version 152605 (0.0009) [2023-12-26 16:28:23,629][105620] Updated weights for policy 1, policy_version 152615 (0.0008) [2023-12-26 16:28:24,167][105620] Updated weights for policy 1, policy_version 152625 (0.0008) [2023-12-26 16:28:24,225][105620] Updated weights for policy 1, policy_version 152635 (0.0005) [2023-12-26 16:28:24,274][105620] Updated weights for policy 1, policy_version 152645 (0.0005) [2023-12-26 16:28:24,298][105692] Updated weights for policy 0, policy_version 151868 (0.0011) [2023-12-26 16:28:24,361][105692] Updated weights for policy 0, policy_version 151878 (0.0007) [2023-12-26 16:28:24,421][105692] Updated weights for policy 0, policy_version 151888 (0.0007) [2023-12-26 16:28:24,959][105620] Updated weights for policy 1, policy_version 152655 (0.0005) [2023-12-26 16:28:24,985][105692] Updated weights for policy 0, policy_version 151898 (0.0008) [2023-12-26 16:28:25,011][105620] Updated weights for policy 1, policy_version 152665 (0.0009) [2023-12-26 16:28:25,036][105692] Updated weights for policy 0, policy_version 151908 (0.0005) [2023-12-26 16:28:25,067][105620] Updated weights for policy 1, policy_version 152675 (0.0007) [2023-12-26 16:28:25,081][105692] Updated weights for policy 0, policy_version 151918 (0.0010) [2023-12-26 16:28:25,129][105692] Updated weights for policy 0, policy_version 151928 (0.0010) [2023-12-26 16:28:25,821][105620] Updated weights for policy 1, policy_version 152685 (0.0006) [2023-12-26 16:28:25,866][105620] Updated weights for policy 1, policy_version 152695 (0.0009) [2023-12-26 16:28:25,868][105692] Updated weights for policy 0, policy_version 151938 (0.0010) [2023-12-26 16:28:25,923][105692] Updated weights for policy 0, policy_version 151948 (0.0010) [2023-12-26 16:28:25,931][105620] Updated weights for policy 1, policy_version 152705 (0.0007) [2023-12-26 16:28:25,981][105692] Updated weights for policy 0, policy_version 151958 (0.0010) [2023-12-26 16:28:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 78012416. Throughput: 0: 9764.6, 1: 9691.3. Samples: 78014136. Policy #0 lag: (min: 27.0, avg: 38.3, max: 59.0) [2023-12-26 16:28:26,062][104569] Avg episode reward: [(0, '8224.197'), (1, '7302.112')] [2023-12-26 16:28:26,690][105620] Updated weights for policy 1, policy_version 152715 (0.0009) [2023-12-26 16:28:26,708][105692] Updated weights for policy 0, policy_version 151968 (0.0010) [2023-12-26 16:28:26,751][105620] Updated weights for policy 1, policy_version 152725 (0.0006) [2023-12-26 16:28:26,761][105692] Updated weights for policy 0, policy_version 151978 (0.0011) [2023-12-26 16:28:26,808][105620] Updated weights for policy 1, policy_version 152735 (0.0005) [2023-12-26 16:28:26,817][105692] Updated weights for policy 0, policy_version 151988 (0.0011) [2023-12-26 16:28:27,368][105620] Updated weights for policy 1, policy_version 152745 (0.0006) [2023-12-26 16:28:27,422][105620] Updated weights for policy 1, policy_version 152755 (0.0007) [2023-12-26 16:28:27,470][105620] Updated weights for policy 1, policy_version 152765 (0.0005) [2023-12-26 16:28:27,517][105620] Updated weights for policy 1, policy_version 152775 (0.0005) [2023-12-26 16:28:27,551][105692] Updated weights for policy 0, policy_version 151998 (0.0010) [2023-12-26 16:28:27,601][105692] Updated weights for policy 0, policy_version 152008 (0.0010) [2023-12-26 16:28:27,651][105692] Updated weights for policy 0, policy_version 152018 (0.0010) [2023-12-26 16:28:28,078][105620] Updated weights for policy 1, policy_version 152785 (0.0008) [2023-12-26 16:28:28,137][105620] Updated weights for policy 1, policy_version 152795 (0.0008) [2023-12-26 16:28:28,192][105620] Updated weights for policy 1, policy_version 152805 (0.0008) [2023-12-26 16:28:28,395][105692] Updated weights for policy 0, policy_version 152028 (0.0010) [2023-12-26 16:28:28,447][105692] Updated weights for policy 0, policy_version 152038 (0.0010) [2023-12-26 16:28:28,491][105692] Updated weights for policy 0, policy_version 152048 (0.0010) [2023-12-26 16:28:28,961][105620] Updated weights for policy 1, policy_version 152815 (0.0007) [2023-12-26 16:28:29,009][105620] Updated weights for policy 1, policy_version 152825 (0.0008) [2023-12-26 16:28:29,053][105620] Updated weights for policy 1, policy_version 152835 (0.0008) [2023-12-26 16:28:29,254][105692] Updated weights for policy 0, policy_version 152058 (0.0010) [2023-12-26 16:28:29,302][105692] Updated weights for policy 0, policy_version 152068 (0.0010) [2023-12-26 16:28:29,372][105692] Updated weights for policy 0, policy_version 152078 (0.0009) [2023-12-26 16:28:29,432][105692] Updated weights for policy 0, policy_version 152088 (0.0010) [2023-12-26 16:28:29,691][105620] Updated weights for policy 1, policy_version 152845 (0.0006) [2023-12-26 16:28:29,754][105620] Updated weights for policy 1, policy_version 152855 (0.0006) [2023-12-26 16:28:29,804][105620] Updated weights for policy 1, policy_version 152865 (0.0008) [2023-12-26 16:28:30,186][105692] Updated weights for policy 0, policy_version 152098 (0.0010) [2023-12-26 16:28:30,244][105692] Updated weights for policy 0, policy_version 152108 (0.0010) [2023-12-26 16:28:30,312][105692] Updated weights for policy 0, policy_version 152118 (0.0010) [2023-12-26 16:28:30,506][105620] Updated weights for policy 1, policy_version 152875 (0.0009) [2023-12-26 16:28:30,578][105620] Updated weights for policy 1, policy_version 152885 (0.0010) [2023-12-26 16:28:30,647][105620] Updated weights for policy 1, policy_version 152895 (0.0010) [2023-12-26 16:28:30,955][105692] Updated weights for policy 0, policy_version 152128 (0.0008) [2023-12-26 16:28:31,002][105692] Updated weights for policy 0, policy_version 152138 (0.0008) [2023-12-26 16:28:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 78102528. Throughput: 0: 9754.6, 1: 9797.8. Samples: 78074512. Policy #0 lag: (min: 27.0, avg: 38.3, max: 59.0) [2023-12-26 16:28:31,062][104569] Avg episode reward: [(0, '9085.338'), (1, '7956.711')] [2023-12-26 16:28:31,063][105692] Updated weights for policy 0, policy_version 152148 (0.0008) [2023-12-26 16:28:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000152904_39149568.pth... [2023-12-26 16:28:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000151752_38854656.pth [2023-12-26 16:28:31,087][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000152152_38961152.pth... [2023-12-26 16:28:31,091][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000151032_38674432.pth [2023-12-26 16:28:31,327][105620] Updated weights for policy 1, policy_version 152905 (0.0011) [2023-12-26 16:28:31,389][105620] Updated weights for policy 1, policy_version 152915 (0.0008) [2023-12-26 16:28:31,454][105620] Updated weights for policy 1, policy_version 152925 (0.0010) [2023-12-26 16:28:31,522][105620] Updated weights for policy 1, policy_version 152935 (0.0008) [2023-12-26 16:28:31,871][105692] Updated weights for policy 0, policy_version 152158 (0.0009) [2023-12-26 16:28:31,935][105692] Updated weights for policy 0, policy_version 152168 (0.0008) [2023-12-26 16:28:31,995][105692] Updated weights for policy 0, policy_version 152178 (0.0009) [2023-12-26 16:28:32,155][105620] Updated weights for policy 1, policy_version 152945 (0.0006) [2023-12-26 16:28:32,204][105620] Updated weights for policy 1, policy_version 152955 (0.0009) [2023-12-26 16:28:32,263][105620] Updated weights for policy 1, policy_version 152965 (0.0010) [2023-12-26 16:28:32,780][105692] Updated weights for policy 0, policy_version 152188 (0.0009) [2023-12-26 16:28:32,830][105692] Updated weights for policy 0, policy_version 152198 (0.0009) [2023-12-26 16:28:32,878][105692] Updated weights for policy 0, policy_version 152208 (0.0009) [2023-12-26 16:28:32,936][105620] Updated weights for policy 1, policy_version 152975 (0.0009) [2023-12-26 16:28:32,993][105620] Updated weights for policy 1, policy_version 152985 (0.0009) [2023-12-26 16:28:33,044][105620] Updated weights for policy 1, policy_version 152995 (0.0009) [2023-12-26 16:28:33,653][105692] Updated weights for policy 0, policy_version 152218 (0.0008) [2023-12-26 16:28:33,711][105692] Updated weights for policy 0, policy_version 152230 (0.0010) [2023-12-26 16:28:33,759][105692] Updated weights for policy 0, policy_version 152240 (0.0007) [2023-12-26 16:28:33,774][105620] Updated weights for policy 1, policy_version 153005 (0.0007) [2023-12-26 16:28:33,833][105620] Updated weights for policy 1, policy_version 153015 (0.0005) [2023-12-26 16:28:33,882][105620] Updated weights for policy 1, policy_version 153025 (0.0005) [2023-12-26 16:28:34,383][105692] Updated weights for policy 0, policy_version 152250 (0.0006) [2023-12-26 16:28:34,445][105692] Updated weights for policy 0, policy_version 152260 (0.0011) [2023-12-26 16:28:34,483][105620] Updated weights for policy 1, policy_version 153035 (0.0006) [2023-12-26 16:28:34,511][105692] Updated weights for policy 0, policy_version 152270 (0.0007) [2023-12-26 16:28:34,547][105620] Updated weights for policy 1, policy_version 153045 (0.0007) [2023-12-26 16:28:34,577][105692] Updated weights for policy 0, policy_version 152280 (0.0006) [2023-12-26 16:28:34,609][105620] Updated weights for policy 1, policy_version 153055 (0.0009) [2023-12-26 16:28:35,134][105692] Updated weights for policy 0, policy_version 152290 (0.0005) [2023-12-26 16:28:35,192][105692] Updated weights for policy 0, policy_version 152300 (0.0005) [2023-12-26 16:28:35,255][105692] Updated weights for policy 0, policy_version 152310 (0.0005) [2023-12-26 16:28:35,320][105620] Updated weights for policy 1, policy_version 153065 (0.0009) [2023-12-26 16:28:35,375][105620] Updated weights for policy 1, policy_version 153075 (0.0005) [2023-12-26 16:28:35,433][105620] Updated weights for policy 1, policy_version 153085 (0.0007) [2023-12-26 16:28:35,484][105620] Updated weights for policy 1, policy_version 153095 (0.0008) [2023-12-26 16:28:35,873][105692] Updated weights for policy 0, policy_version 152320 (0.0009) [2023-12-26 16:28:35,943][105692] Updated weights for policy 0, policy_version 152330 (0.0010) [2023-12-26 16:28:36,002][105692] Updated weights for policy 0, policy_version 152340 (0.0009) [2023-12-26 16:28:36,017][105620] Updated weights for policy 1, policy_version 153105 (0.0006) [2023-12-26 16:28:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 78209024. Throughput: 0: 9724.4, 1: 9805.4. Samples: 78193216. Policy #0 lag: (min: 27.0, avg: 38.3, max: 59.0) [2023-12-26 16:28:36,062][104569] Avg episode reward: [(0, '9175.571'), (1, '9258.356')] [2023-12-26 16:28:36,075][105620] Updated weights for policy 1, policy_version 153115 (0.0007) [2023-12-26 16:28:36,145][105620] Updated weights for policy 1, policy_version 153126 (0.0009) [2023-12-26 16:28:36,732][105620] Updated weights for policy 1, policy_version 153136 (0.0005) [2023-12-26 16:28:36,801][105620] Updated weights for policy 1, policy_version 153146 (0.0006) [2023-12-26 16:28:36,859][105692] Updated weights for policy 0, policy_version 152350 (0.0009) [2023-12-26 16:28:36,861][105620] Updated weights for policy 1, policy_version 153156 (0.0005) [2023-12-26 16:28:36,919][105692] Updated weights for policy 0, policy_version 152360 (0.0009) [2023-12-26 16:28:36,971][105692] Updated weights for policy 0, policy_version 152370 (0.0009) [2023-12-26 16:28:37,371][105620] Updated weights for policy 1, policy_version 153166 (0.0008) [2023-12-26 16:28:37,432][105620] Updated weights for policy 1, policy_version 153176 (0.0009) [2023-12-26 16:28:37,486][105620] Updated weights for policy 1, policy_version 153186 (0.0009) [2023-12-26 16:28:37,623][105692] Updated weights for policy 0, policy_version 152381 (0.0008) [2023-12-26 16:28:37,687][105692] Updated weights for policy 0, policy_version 152391 (0.0006) [2023-12-26 16:28:37,748][105692] Updated weights for policy 0, policy_version 152401 (0.0005) [2023-12-26 16:28:38,281][105692] Updated weights for policy 0, policy_version 152411 (0.0005) [2023-12-26 16:28:38,282][105620] Updated weights for policy 1, policy_version 153196 (0.0008) [2023-12-26 16:28:38,348][105620] Updated weights for policy 1, policy_version 153206 (0.0007) [2023-12-26 16:28:38,349][105692] Updated weights for policy 0, policy_version 152421 (0.0008) [2023-12-26 16:28:38,411][105620] Updated weights for policy 1, policy_version 153216 (0.0008) [2023-12-26 16:28:38,411][105692] Updated weights for policy 0, policy_version 152431 (0.0006) [2023-12-26 16:28:39,019][105620] Updated weights for policy 1, policy_version 153226 (0.0009) [2023-12-26 16:28:39,072][105620] Updated weights for policy 1, policy_version 153236 (0.0005) [2023-12-26 16:28:39,104][105692] Updated weights for policy 0, policy_version 152441 (0.0006) [2023-12-26 16:28:39,124][105620] Updated weights for policy 1, policy_version 153246 (0.0005) [2023-12-26 16:28:39,160][105692] Updated weights for policy 0, policy_version 152451 (0.0010) [2023-12-26 16:28:39,183][105620] Updated weights for policy 1, policy_version 153256 (0.0009) [2023-12-26 16:28:39,220][105692] Updated weights for policy 0, policy_version 152461 (0.0010) [2023-12-26 16:28:39,286][105692] Updated weights for policy 0, policy_version 152471 (0.0011) [2023-12-26 16:28:39,895][105620] Updated weights for policy 1, policy_version 153266 (0.0009) [2023-12-26 16:28:39,943][105692] Updated weights for policy 0, policy_version 152481 (0.0008) [2023-12-26 16:28:39,967][105620] Updated weights for policy 1, policy_version 153276 (0.0009) [2023-12-26 16:28:40,011][105692] Updated weights for policy 0, policy_version 152491 (0.0006) [2023-12-26 16:28:40,032][105620] Updated weights for policy 1, policy_version 153286 (0.0009) [2023-12-26 16:28:40,076][105692] Updated weights for policy 0, policy_version 152501 (0.0007) [2023-12-26 16:28:40,738][105692] Updated weights for policy 0, policy_version 152511 (0.0010) [2023-12-26 16:28:40,793][105620] Updated weights for policy 1, policy_version 153296 (0.0007) [2023-12-26 16:28:40,795][105692] Updated weights for policy 0, policy_version 152521 (0.0011) [2023-12-26 16:28:40,855][105692] Updated weights for policy 0, policy_version 152531 (0.0010) [2023-12-26 16:28:40,857][105620] Updated weights for policy 1, policy_version 153306 (0.0006) [2023-12-26 16:28:40,914][105620] Updated weights for policy 1, policy_version 153316 (0.0008) [2023-12-26 16:28:41,062][104569] Fps is (10 sec: 21299.0, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 78315520. Throughput: 0: 9789.8, 1: 9899.6. Samples: 78319072. Policy #0 lag: (min: 27.0, avg: 38.3, max: 59.0) [2023-12-26 16:28:41,063][104569] Avg episode reward: [(0, '9166.329'), (1, '9349.950')] [2023-12-26 16:28:41,616][105692] Updated weights for policy 0, policy_version 152541 (0.0011) [2023-12-26 16:28:41,676][105692] Updated weights for policy 0, policy_version 152551 (0.0010) [2023-12-26 16:28:41,698][105620] Updated weights for policy 1, policy_version 153326 (0.0007) [2023-12-26 16:28:41,744][105692] Updated weights for policy 0, policy_version 152561 (0.0009) [2023-12-26 16:28:41,758][105620] Updated weights for policy 1, policy_version 153336 (0.0009) [2023-12-26 16:28:41,814][105620] Updated weights for policy 1, policy_version 153346 (0.0007) [2023-12-26 16:28:42,489][105620] Updated weights for policy 1, policy_version 153356 (0.0007) [2023-12-26 16:28:42,555][105620] Updated weights for policy 1, policy_version 153366 (0.0009) [2023-12-26 16:28:42,575][105692] Updated weights for policy 0, policy_version 152571 (0.0008) [2023-12-26 16:28:42,610][105620] Updated weights for policy 1, policy_version 153376 (0.0006) [2023-12-26 16:28:42,649][105692] Updated weights for policy 0, policy_version 152581 (0.0008) [2023-12-26 16:28:42,715][105692] Updated weights for policy 0, policy_version 152591 (0.0006) [2023-12-26 16:28:43,347][105692] Updated weights for policy 0, policy_version 152601 (0.0006) [2023-12-26 16:28:43,384][105620] Updated weights for policy 1, policy_version 153386 (0.0006) [2023-12-26 16:28:43,405][105692] Updated weights for policy 0, policy_version 152611 (0.0010) [2023-12-26 16:28:43,435][105620] Updated weights for policy 1, policy_version 153396 (0.0005) [2023-12-26 16:28:43,467][105692] Updated weights for policy 0, policy_version 152621 (0.0010) [2023-12-26 16:28:43,481][105620] Updated weights for policy 1, policy_version 153406 (0.0007) [2023-12-26 16:28:43,524][105692] Updated weights for policy 0, policy_version 152631 (0.0010) [2023-12-26 16:28:43,526][105620] Updated weights for policy 1, policy_version 153416 (0.0008) [2023-12-26 16:28:44,198][105692] Updated weights for policy 0, policy_version 152641 (0.0006) [2023-12-26 16:28:44,205][105620] Updated weights for policy 1, policy_version 153426 (0.0007) [2023-12-26 16:28:44,248][105692] Updated weights for policy 0, policy_version 152651 (0.0007) [2023-12-26 16:28:44,267][105620] Updated weights for policy 1, policy_version 153436 (0.0009) [2023-12-26 16:28:44,294][105692] Updated weights for policy 0, policy_version 152661 (0.0007) [2023-12-26 16:28:44,325][105620] Updated weights for policy 1, policy_version 153446 (0.0008) [2023-12-26 16:28:45,006][105692] Updated weights for policy 0, policy_version 152671 (0.0006) [2023-12-26 16:28:45,074][105692] Updated weights for policy 0, policy_version 152681 (0.0006) [2023-12-26 16:28:45,124][105620] Updated weights for policy 1, policy_version 153456 (0.0009) [2023-12-26 16:28:45,139][105692] Updated weights for policy 0, policy_version 152691 (0.0006) [2023-12-26 16:28:45,188][105620] Updated weights for policy 1, policy_version 153466 (0.0007) [2023-12-26 16:28:45,244][105620] Updated weights for policy 1, policy_version 153476 (0.0010) [2023-12-26 16:28:45,805][105692] Updated weights for policy 0, policy_version 152701 (0.0006) [2023-12-26 16:28:45,859][105692] Updated weights for policy 0, policy_version 152711 (0.0005) [2023-12-26 16:28:45,909][105692] Updated weights for policy 0, policy_version 152721 (0.0005) [2023-12-26 16:28:45,979][105620] Updated weights for policy 1, policy_version 153486 (0.0009) [2023-12-26 16:28:46,040][105620] Updated weights for policy 1, policy_version 153496 (0.0006) [2023-12-26 16:28:46,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 78405632. Throughput: 0: 9689.3, 1: 9908.5. Samples: 78375396. Policy #0 lag: (min: 27.0, avg: 38.3, max: 59.0) [2023-12-26 16:28:46,063][104569] Avg episode reward: [(0, '9177.576'), (1, '9259.464')] [2023-12-26 16:28:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000152728_39108608.pth... [2023-12-26 16:28:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000151576_38813696.pth [2023-12-26 16:28:46,096][105620] Updated weights for policy 1, policy_version 153506 (0.0005) [2023-12-26 16:28:46,127][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000153512_39305216.pth... [2023-12-26 16:28:46,130][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000152328_39002112.pth [2023-12-26 16:28:46,599][105692] Updated weights for policy 0, policy_version 152731 (0.0007) [2023-12-26 16:28:46,646][105692] Updated weights for policy 0, policy_version 152741 (0.0010) [2023-12-26 16:28:46,700][105692] Updated weights for policy 0, policy_version 152751 (0.0010) [2023-12-26 16:28:46,730][105620] Updated weights for policy 1, policy_version 153516 (0.0006) [2023-12-26 16:28:46,787][105620] Updated weights for policy 1, policy_version 153526 (0.0007) [2023-12-26 16:28:46,841][105620] Updated weights for policy 1, policy_version 153536 (0.0008) [2023-12-26 16:28:47,387][105692] Updated weights for policy 0, policy_version 152761 (0.0010) [2023-12-26 16:28:47,438][105692] Updated weights for policy 0, policy_version 152771 (0.0006) [2023-12-26 16:28:47,501][105692] Updated weights for policy 0, policy_version 152781 (0.0008) [2023-12-26 16:28:47,562][105692] Updated weights for policy 0, policy_version 152791 (0.0009) [2023-12-26 16:28:47,613][105620] Updated weights for policy 1, policy_version 153546 (0.0008) [2023-12-26 16:28:47,673][105620] Updated weights for policy 1, policy_version 153556 (0.0007) [2023-12-26 16:28:47,720][105620] Updated weights for policy 1, policy_version 153566 (0.0009) [2023-12-26 16:28:47,783][105620] Updated weights for policy 1, policy_version 153576 (0.0011) [2023-12-26 16:28:48,232][105692] Updated weights for policy 0, policy_version 152801 (0.0005) [2023-12-26 16:28:48,287][105692] Updated weights for policy 0, policy_version 152811 (0.0005) [2023-12-26 16:28:48,355][105692] Updated weights for policy 0, policy_version 152821 (0.0007) [2023-12-26 16:28:48,500][105620] Updated weights for policy 1, policy_version 153586 (0.0009) [2023-12-26 16:28:48,572][105620] Updated weights for policy 1, policy_version 153596 (0.0009) [2023-12-26 16:28:48,638][105620] Updated weights for policy 1, policy_version 153606 (0.0010) [2023-12-26 16:28:48,921][105692] Updated weights for policy 0, policy_version 152831 (0.0005) [2023-12-26 16:28:48,990][105692] Updated weights for policy 0, policy_version 152841 (0.0005) [2023-12-26 16:28:49,044][105692] Updated weights for policy 0, policy_version 152851 (0.0007) [2023-12-26 16:28:49,455][105620] Updated weights for policy 1, policy_version 153616 (0.0009) [2023-12-26 16:28:49,521][105620] Updated weights for policy 1, policy_version 153626 (0.0009) [2023-12-26 16:28:49,581][105620] Updated weights for policy 1, policy_version 153636 (0.0008) [2023-12-26 16:28:49,744][105692] Updated weights for policy 0, policy_version 152861 (0.0009) [2023-12-26 16:28:49,807][105692] Updated weights for policy 0, policy_version 152871 (0.0010) [2023-12-26 16:28:49,883][105692] Updated weights for policy 0, policy_version 152882 (0.0010) [2023-12-26 16:28:50,223][105620] Updated weights for policy 1, policy_version 153646 (0.0009) [2023-12-26 16:28:50,286][105620] Updated weights for policy 1, policy_version 153656 (0.0008) [2023-12-26 16:28:50,345][105620] Updated weights for policy 1, policy_version 153666 (0.0009) [2023-12-26 16:28:50,697][105692] Updated weights for policy 0, policy_version 152892 (0.0008) [2023-12-26 16:28:50,746][105692] Updated weights for policy 0, policy_version 152902 (0.0008) [2023-12-26 16:28:50,794][105692] Updated weights for policy 0, policy_version 152912 (0.0008) [2023-12-26 16:28:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 78503936. Throughput: 0: 9772.4, 1: 9853.5. Samples: 78495356. Policy #0 lag: (min: 27.0, avg: 38.3, max: 59.0) [2023-12-26 16:28:51,062][104569] Avg episode reward: [(0, '8907.618'), (1, '9169.714')] [2023-12-26 16:28:51,097][105620] Updated weights for policy 1, policy_version 153676 (0.0010) [2023-12-26 16:28:51,164][105620] Updated weights for policy 1, policy_version 153686 (0.0010) [2023-12-26 16:28:51,215][105620] Updated weights for policy 1, policy_version 153696 (0.0010) [2023-12-26 16:28:51,584][105692] Updated weights for policy 0, policy_version 152922 (0.0007) [2023-12-26 16:28:51,644][105692] Updated weights for policy 0, policy_version 152932 (0.0007) [2023-12-26 16:28:51,710][105692] Updated weights for policy 0, policy_version 152942 (0.0006) [2023-12-26 16:28:51,778][105692] Updated weights for policy 0, policy_version 152952 (0.0007) [2023-12-26 16:28:51,930][105620] Updated weights for policy 1, policy_version 153706 (0.0011) [2023-12-26 16:28:51,999][105620] Updated weights for policy 1, policy_version 153716 (0.0011) [2023-12-26 16:28:52,052][105620] Updated weights for policy 1, policy_version 153726 (0.0010) [2023-12-26 16:28:52,122][105620] Updated weights for policy 1, policy_version 153736 (0.0009) [2023-12-26 16:28:52,380][105692] Updated weights for policy 0, policy_version 152962 (0.0008) [2023-12-26 16:28:52,429][105692] Updated weights for policy 0, policy_version 152972 (0.0008) [2023-12-26 16:28:52,484][105692] Updated weights for policy 0, policy_version 152982 (0.0008) [2023-12-26 16:28:52,820][105620] Updated weights for policy 1, policy_version 153746 (0.0005) [2023-12-26 16:28:52,883][105620] Updated weights for policy 1, policy_version 153756 (0.0009) [2023-12-26 16:28:52,949][105620] Updated weights for policy 1, policy_version 153766 (0.0009) [2023-12-26 16:28:53,332][105692] Updated weights for policy 0, policy_version 152992 (0.0008) [2023-12-26 16:28:53,387][105692] Updated weights for policy 0, policy_version 153002 (0.0008) [2023-12-26 16:28:53,442][105692] Updated weights for policy 0, policy_version 153012 (0.0008) [2023-12-26 16:28:53,602][105620] Updated weights for policy 1, policy_version 153776 (0.0010) [2023-12-26 16:28:53,663][105620] Updated weights for policy 1, policy_version 153786 (0.0010) [2023-12-26 16:28:53,728][105620] Updated weights for policy 1, policy_version 153796 (0.0010) [2023-12-26 16:28:54,206][105692] Updated weights for policy 0, policy_version 153022 (0.0006) [2023-12-26 16:28:54,265][105692] Updated weights for policy 0, policy_version 153032 (0.0007) [2023-12-26 16:28:54,309][105692] Updated weights for policy 0, policy_version 153042 (0.0008) [2023-12-26 16:28:54,356][105620] Updated weights for policy 1, policy_version 153806 (0.0007) [2023-12-26 16:28:54,426][105620] Updated weights for policy 1, policy_version 153816 (0.0006) [2023-12-26 16:28:54,493][105620] Updated weights for policy 1, policy_version 153826 (0.0005) [2023-12-26 16:28:54,952][105692] Updated weights for policy 0, policy_version 153052 (0.0008) [2023-12-26 16:28:55,012][105692] Updated weights for policy 0, policy_version 153062 (0.0010) [2023-12-26 16:28:55,026][105620] Updated weights for policy 1, policy_version 153836 (0.0008) [2023-12-26 16:28:55,067][105692] Updated weights for policy 0, policy_version 153072 (0.0011) [2023-12-26 16:28:55,083][105620] Updated weights for policy 1, policy_version 153846 (0.0007) [2023-12-26 16:28:55,139][105620] Updated weights for policy 1, policy_version 153856 (0.0010) [2023-12-26 16:28:55,834][105620] Updated weights for policy 1, policy_version 153866 (0.0010) [2023-12-26 16:28:55,838][105692] Updated weights for policy 0, policy_version 153082 (0.0011) [2023-12-26 16:28:55,892][105620] Updated weights for policy 1, policy_version 153876 (0.0010) [2023-12-26 16:28:55,896][105692] Updated weights for policy 0, policy_version 153092 (0.0010) [2023-12-26 16:28:55,947][105620] Updated weights for policy 1, policy_version 153886 (0.0010) [2023-12-26 16:28:55,957][105692] Updated weights for policy 0, policy_version 153102 (0.0010) [2023-12-26 16:28:56,002][105620] Updated weights for policy 1, policy_version 153896 (0.0010) [2023-12-26 16:28:56,019][105692] Updated weights for policy 0, policy_version 153112 (0.0010) [2023-12-26 16:28:56,062][104569] Fps is (10 sec: 20480.7, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 78610432. Throughput: 0: 9736.1, 1: 9948.0. Samples: 78612844. Policy #0 lag: (min: 27.0, avg: 38.3, max: 59.0) [2023-12-26 16:28:56,062][104569] Avg episode reward: [(0, '8816.450'), (1, '9259.707')] [2023-12-26 16:28:56,658][105620] Updated weights for policy 1, policy_version 153906 (0.0005) [2023-12-26 16:28:56,706][105620] Updated weights for policy 1, policy_version 153916 (0.0007) [2023-12-26 16:28:56,748][105692] Updated weights for policy 0, policy_version 153122 (0.0010) [2023-12-26 16:28:56,761][105620] Updated weights for policy 1, policy_version 153926 (0.0010) [2023-12-26 16:28:56,808][105692] Updated weights for policy 0, policy_version 153132 (0.0010) [2023-12-26 16:28:56,866][105692] Updated weights for policy 0, policy_version 153142 (0.0010) [2023-12-26 16:28:57,366][105620] Updated weights for policy 1, policy_version 153936 (0.0010) [2023-12-26 16:28:57,417][105620] Updated weights for policy 1, policy_version 153947 (0.0009) [2023-12-26 16:28:57,469][105620] Updated weights for policy 1, policy_version 153957 (0.0010) [2023-12-26 16:28:57,507][105692] Updated weights for policy 0, policy_version 153152 (0.0009) [2023-12-26 16:28:57,562][105692] Updated weights for policy 0, policy_version 153162 (0.0009) [2023-12-26 16:28:57,623][105692] Updated weights for policy 0, policy_version 153172 (0.0008) [2023-12-26 16:28:58,259][105692] Updated weights for policy 0, policy_version 153182 (0.0007) [2023-12-26 16:28:58,275][105620] Updated weights for policy 1, policy_version 153967 (0.0007) [2023-12-26 16:28:58,320][105692] Updated weights for policy 0, policy_version 153192 (0.0008) [2023-12-26 16:28:58,349][105620] Updated weights for policy 1, policy_version 153977 (0.0008) [2023-12-26 16:28:58,386][105692] Updated weights for policy 0, policy_version 153202 (0.0008) [2023-12-26 16:28:58,412][105620] Updated weights for policy 1, policy_version 153987 (0.0008) [2023-12-26 16:28:59,157][105692] Updated weights for policy 0, policy_version 153212 (0.0008) [2023-12-26 16:28:59,188][105620] Updated weights for policy 1, policy_version 153997 (0.0008) [2023-12-26 16:28:59,237][105692] Updated weights for policy 0, policy_version 153222 (0.0007) [2023-12-26 16:28:59,256][105620] Updated weights for policy 1, policy_version 154007 (0.0012) [2023-12-26 16:28:59,294][105692] Updated weights for policy 0, policy_version 153232 (0.0008) [2023-12-26 16:28:59,325][105620] Updated weights for policy 1, policy_version 154017 (0.0011) [2023-12-26 16:28:59,991][105692] Updated weights for policy 0, policy_version 153242 (0.0008) [2023-12-26 16:29:00,016][105620] Updated weights for policy 1, policy_version 154027 (0.0009) [2023-12-26 16:29:00,046][105692] Updated weights for policy 0, policy_version 153252 (0.0008) [2023-12-26 16:29:00,070][105620] Updated weights for policy 1, policy_version 154037 (0.0005) [2023-12-26 16:29:00,100][105692] Updated weights for policy 0, policy_version 153262 (0.0008) [2023-12-26 16:29:00,125][105620] Updated weights for policy 1, policy_version 154047 (0.0005) [2023-12-26 16:29:00,162][105692] Updated weights for policy 0, policy_version 153272 (0.0008) [2023-12-26 16:29:00,699][105620] Updated weights for policy 1, policy_version 154057 (0.0006) [2023-12-26 16:29:00,753][105620] Updated weights for policy 1, policy_version 154067 (0.0010) [2023-12-26 16:29:00,811][105620] Updated weights for policy 1, policy_version 154077 (0.0010) [2023-12-26 16:29:00,847][105692] Updated weights for policy 0, policy_version 153282 (0.0005) [2023-12-26 16:29:00,855][105620] Updated weights for policy 1, policy_version 154087 (0.0010) [2023-12-26 16:29:00,907][105692] Updated weights for policy 0, policy_version 153292 (0.0005) [2023-12-26 16:29:00,967][105692] Updated weights for policy 0, policy_version 153302 (0.0006) [2023-12-26 16:29:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 78708736. Throughput: 0: 9813.1, 1: 9950.2. Samples: 78671856. Policy #0 lag: (min: 27.0, avg: 38.3, max: 59.0) [2023-12-26 16:29:01,062][104569] Avg episode reward: [(0, '8995.483'), (1, '9167.872')] [2023-12-26 16:29:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000153304_39256064.pth... [2023-12-26 16:29:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000154088_39452672.pth... [2023-12-26 16:29:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000152152_38961152.pth [2023-12-26 16:29:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000152904_39149568.pth [2023-12-26 16:29:01,508][105620] Updated weights for policy 1, policy_version 154097 (0.0008) [2023-12-26 16:29:01,556][105620] Updated weights for policy 1, policy_version 154107 (0.0010) [2023-12-26 16:29:01,615][105620] Updated weights for policy 1, policy_version 154117 (0.0010) [2023-12-26 16:29:01,622][105692] Updated weights for policy 0, policy_version 153312 (0.0007) [2023-12-26 16:29:01,689][105692] Updated weights for policy 0, policy_version 153322 (0.0009) [2023-12-26 16:29:01,757][105692] Updated weights for policy 0, policy_version 153332 (0.0009) [2023-12-26 16:29:02,381][105620] Updated weights for policy 1, policy_version 154127 (0.0009) [2023-12-26 16:29:02,442][105620] Updated weights for policy 1, policy_version 154137 (0.0009) [2023-12-26 16:29:02,481][105692] Updated weights for policy 0, policy_version 153342 (0.0007) [2023-12-26 16:29:02,502][105620] Updated weights for policy 1, policy_version 154147 (0.0008) [2023-12-26 16:29:02,534][105692] Updated weights for policy 0, policy_version 153352 (0.0007) [2023-12-26 16:29:02,593][105692] Updated weights for policy 0, policy_version 153362 (0.0008) [2023-12-26 16:29:03,163][105692] Updated weights for policy 0, policy_version 153372 (0.0006) [2023-12-26 16:29:03,223][105692] Updated weights for policy 0, policy_version 153382 (0.0008) [2023-12-26 16:29:03,284][105692] Updated weights for policy 0, policy_version 153392 (0.0008) [2023-12-26 16:29:03,305][105620] Updated weights for policy 1, policy_version 154157 (0.0007) [2023-12-26 16:29:03,349][105620] Updated weights for policy 1, policy_version 154167 (0.0007) [2023-12-26 16:29:03,398][105620] Updated weights for policy 1, policy_version 154177 (0.0009) [2023-12-26 16:29:04,027][105692] Updated weights for policy 0, policy_version 153402 (0.0007) [2023-12-26 16:29:04,087][105692] Updated weights for policy 0, policy_version 153412 (0.0008) [2023-12-26 16:29:04,159][105692] Updated weights for policy 0, policy_version 153422 (0.0009) [2023-12-26 16:29:04,170][105620] Updated weights for policy 1, policy_version 154187 (0.0008) [2023-12-26 16:29:04,229][105692] Updated weights for policy 0, policy_version 153432 (0.0007) [2023-12-26 16:29:04,237][105620] Updated weights for policy 1, policy_version 154197 (0.0006) [2023-12-26 16:29:04,297][105620] Updated weights for policy 1, policy_version 154207 (0.0009) [2023-12-26 16:29:04,913][105692] Updated weights for policy 0, policy_version 153442 (0.0009) [2023-12-26 16:29:04,968][105692] Updated weights for policy 0, policy_version 153452 (0.0009) [2023-12-26 16:29:05,028][105692] Updated weights for policy 0, policy_version 153462 (0.0009) [2023-12-26 16:29:05,029][105620] Updated weights for policy 1, policy_version 154217 (0.0010) [2023-12-26 16:29:05,078][105620] Updated weights for policy 1, policy_version 154227 (0.0009) [2023-12-26 16:29:05,124][105620] Updated weights for policy 1, policy_version 154237 (0.0006) [2023-12-26 16:29:05,181][105620] Updated weights for policy 1, policy_version 154247 (0.0007) [2023-12-26 16:29:05,731][105692] Updated weights for policy 0, policy_version 153472 (0.0008) [2023-12-26 16:29:05,787][105692] Updated weights for policy 0, policy_version 153482 (0.0008) [2023-12-26 16:29:05,820][105620] Updated weights for policy 1, policy_version 154257 (0.0007) [2023-12-26 16:29:05,847][105692] Updated weights for policy 0, policy_version 153492 (0.0006) [2023-12-26 16:29:05,879][105620] Updated weights for policy 1, policy_version 154267 (0.0008) [2023-12-26 16:29:05,936][105620] Updated weights for policy 1, policy_version 154277 (0.0008) [2023-12-26 16:29:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 78807040. Throughput: 0: 9833.9, 1: 9997.8. Samples: 78790372. Policy #0 lag: (min: 27.0, avg: 38.3, max: 59.0) [2023-12-26 16:29:06,062][104569] Avg episode reward: [(0, '9178.126'), (1, '9257.989')] [2023-12-26 16:29:06,553][105692] Updated weights for policy 0, policy_version 153502 (0.0008) [2023-12-26 16:29:06,613][105692] Updated weights for policy 0, policy_version 153512 (0.0009) [2023-12-26 16:29:06,677][105692] Updated weights for policy 0, policy_version 153522 (0.0006) [2023-12-26 16:29:06,775][105620] Updated weights for policy 1, policy_version 154287 (0.0009) [2023-12-26 16:29:06,828][105620] Updated weights for policy 1, policy_version 154297 (0.0009) [2023-12-26 16:29:06,894][105620] Updated weights for policy 1, policy_version 154308 (0.0009) [2023-12-26 16:29:07,309][105692] Updated weights for policy 0, policy_version 153532 (0.0006) [2023-12-26 16:29:07,369][105692] Updated weights for policy 0, policy_version 153542 (0.0009) [2023-12-26 16:29:07,429][105692] Updated weights for policy 0, policy_version 153552 (0.0007) [2023-12-26 16:29:07,715][105620] Updated weights for policy 1, policy_version 154318 (0.0008) [2023-12-26 16:29:07,775][105620] Updated weights for policy 1, policy_version 154328 (0.0009) [2023-12-26 16:29:07,835][105620] Updated weights for policy 1, policy_version 154338 (0.0009) [2023-12-26 16:29:08,025][105692] Updated weights for policy 0, policy_version 153562 (0.0008) [2023-12-26 16:29:08,089][105692] Updated weights for policy 0, policy_version 153572 (0.0006) [2023-12-26 16:29:08,152][105692] Updated weights for policy 0, policy_version 153582 (0.0006) [2023-12-26 16:29:08,208][105692] Updated weights for policy 0, policy_version 153592 (0.0006) [2023-12-26 16:29:08,654][105620] Updated weights for policy 1, policy_version 154348 (0.0010) [2023-12-26 16:29:08,715][105620] Updated weights for policy 1, policy_version 154358 (0.0009) [2023-12-26 16:29:08,768][105620] Updated weights for policy 1, policy_version 154368 (0.0007) [2023-12-26 16:29:08,882][105692] Updated weights for policy 0, policy_version 153602 (0.0007) [2023-12-26 16:29:08,938][105692] Updated weights for policy 0, policy_version 153612 (0.0006) [2023-12-26 16:29:08,990][105692] Updated weights for policy 0, policy_version 153622 (0.0006) [2023-12-26 16:29:09,543][105620] Updated weights for policy 1, policy_version 154378 (0.0006) [2023-12-26 16:29:09,597][105620] Updated weights for policy 1, policy_version 154388 (0.0009) [2023-12-26 16:29:09,656][105620] Updated weights for policy 1, policy_version 154398 (0.0007) [2023-12-26 16:29:09,667][105692] Updated weights for policy 0, policy_version 153632 (0.0007) [2023-12-26 16:29:09,717][105620] Updated weights for policy 1, policy_version 154408 (0.0007) [2023-12-26 16:29:09,724][105692] Updated weights for policy 0, policy_version 153642 (0.0006) [2023-12-26 16:29:09,777][105692] Updated weights for policy 0, policy_version 153652 (0.0009) [2023-12-26 16:29:10,392][105620] Updated weights for policy 1, policy_version 154418 (0.0008) [2023-12-26 16:29:10,459][105620] Updated weights for policy 1, policy_version 154428 (0.0006) [2023-12-26 16:29:10,523][105620] Updated weights for policy 1, policy_version 154438 (0.0008) [2023-12-26 16:29:10,602][105692] Updated weights for policy 0, policy_version 153662 (0.0009) [2023-12-26 16:29:10,654][105692] Updated weights for policy 0, policy_version 153672 (0.0009) [2023-12-26 16:29:10,708][105692] Updated weights for policy 0, policy_version 153682 (0.0008) [2023-12-26 16:29:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 78897152. Throughput: 0: 9872.9, 1: 9958.0. Samples: 78906528. Policy #0 lag: (min: 27.0, avg: 38.3, max: 59.0) [2023-12-26 16:29:11,063][104569] Avg episode reward: [(0, '8343.883'), (1, '9164.060')] [2023-12-26 16:29:11,258][105620] Updated weights for policy 1, policy_version 154448 (0.0009) [2023-12-26 16:29:11,328][105620] Updated weights for policy 1, policy_version 154458 (0.0009) [2023-12-26 16:29:11,394][105620] Updated weights for policy 1, policy_version 154468 (0.0008) [2023-12-26 16:29:11,510][105692] Updated weights for policy 0, policy_version 153692 (0.0009) [2023-12-26 16:29:11,569][105692] Updated weights for policy 0, policy_version 153702 (0.0009) [2023-12-26 16:29:11,617][105692] Updated weights for policy 0, policy_version 153712 (0.0009) [2023-12-26 16:29:12,143][105620] Updated weights for policy 1, policy_version 154478 (0.0009) [2023-12-26 16:29:12,191][105620] Updated weights for policy 1, policy_version 154488 (0.0009) [2023-12-26 16:29:12,243][105620] Updated weights for policy 1, policy_version 154498 (0.0008) [2023-12-26 16:29:12,441][105692] Updated weights for policy 0, policy_version 153722 (0.0009) [2023-12-26 16:29:12,510][105692] Updated weights for policy 0, policy_version 153732 (0.0005) [2023-12-26 16:29:12,574][105692] Updated weights for policy 0, policy_version 153742 (0.0009) [2023-12-26 16:29:12,636][105692] Updated weights for policy 0, policy_version 153752 (0.0009) [2023-12-26 16:29:13,059][105620] Updated weights for policy 1, policy_version 154508 (0.0009) [2023-12-26 16:29:13,116][105620] Updated weights for policy 1, policy_version 154518 (0.0008) [2023-12-26 16:29:13,173][105620] Updated weights for policy 1, policy_version 154528 (0.0009) [2023-12-26 16:29:13,293][105692] Updated weights for policy 0, policy_version 153762 (0.0009) [2023-12-26 16:29:13,352][105692] Updated weights for policy 0, policy_version 153772 (0.0009) [2023-12-26 16:29:13,406][105692] Updated weights for policy 0, policy_version 153782 (0.0009) [2023-12-26 16:29:13,910][105620] Updated weights for policy 1, policy_version 154538 (0.0009) [2023-12-26 16:29:13,969][105620] Updated weights for policy 1, policy_version 154548 (0.0008) [2023-12-26 16:29:14,015][105620] Updated weights for policy 1, policy_version 154558 (0.0008) [2023-12-26 16:29:14,067][105620] Updated weights for policy 1, policy_version 154568 (0.0009) [2023-12-26 16:29:14,168][105692] Updated weights for policy 0, policy_version 153792 (0.0009) [2023-12-26 16:29:14,228][105692] Updated weights for policy 0, policy_version 153802 (0.0009) [2023-12-26 16:29:14,282][105692] Updated weights for policy 0, policy_version 153812 (0.0009) [2023-12-26 16:29:14,859][105620] Updated weights for policy 1, policy_version 154578 (0.0009) [2023-12-26 16:29:14,929][105620] Updated weights for policy 1, policy_version 154588 (0.0009) [2023-12-26 16:29:14,991][105620] Updated weights for policy 1, policy_version 154598 (0.0007) [2023-12-26 16:29:15,051][105692] Updated weights for policy 0, policy_version 153822 (0.0009) [2023-12-26 16:29:15,110][105692] Updated weights for policy 0, policy_version 153832 (0.0009) [2023-12-26 16:29:15,173][105692] Updated weights for policy 0, policy_version 153842 (0.0009) [2023-12-26 16:29:15,656][105620] Updated weights for policy 1, policy_version 154608 (0.0008) [2023-12-26 16:29:15,718][105620] Updated weights for policy 1, policy_version 154618 (0.0009) [2023-12-26 16:29:15,765][105620] Updated weights for policy 1, policy_version 154628 (0.0009) [2023-12-26 16:29:15,947][105692] Updated weights for policy 0, policy_version 153852 (0.0009) [2023-12-26 16:29:15,999][105692] Updated weights for policy 0, policy_version 153862 (0.0010) [2023-12-26 16:29:16,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 78987264. Throughput: 0: 9856.3, 1: 9853.8. Samples: 78961464. Policy #0 lag: (min: 27.0, avg: 38.3, max: 59.0) [2023-12-26 16:29:16,063][104569] Avg episode reward: [(0, '8052.583'), (1, '5858.975')] [2023-12-26 16:29:16,064][105692] Updated weights for policy 0, policy_version 153872 (0.0009) [2023-12-26 16:29:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000154632_39591936.pth... [2023-12-26 16:29:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000153512_39305216.pth [2023-12-26 16:29:16,104][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000153880_39403520.pth... [2023-12-26 16:29:16,108][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000152728_39108608.pth [2023-12-26 16:29:16,450][105620] Updated weights for policy 1, policy_version 154638 (0.0007) [2023-12-26 16:29:16,511][105620] Updated weights for policy 1, policy_version 154648 (0.0009) [2023-12-26 16:29:16,567][105620] Updated weights for policy 1, policy_version 154658 (0.0010) [2023-12-26 16:29:16,838][105692] Updated weights for policy 0, policy_version 153882 (0.0008) [2023-12-26 16:29:16,900][105692] Updated weights for policy 0, policy_version 153892 (0.0005) [2023-12-26 16:29:16,961][105692] Updated weights for policy 0, policy_version 153902 (0.0005) [2023-12-26 16:29:17,019][105692] Updated weights for policy 0, policy_version 153912 (0.0006) [2023-12-26 16:29:17,290][105620] Updated weights for policy 1, policy_version 154668 (0.0010) [2023-12-26 16:29:17,347][105620] Updated weights for policy 1, policy_version 154678 (0.0008) [2023-12-26 16:29:17,407][105620] Updated weights for policy 1, policy_version 154688 (0.0008) [2023-12-26 16:29:17,683][105692] Updated weights for policy 0, policy_version 153922 (0.0010) [2023-12-26 16:29:17,738][105692] Updated weights for policy 0, policy_version 153932 (0.0010) [2023-12-26 16:29:17,792][105692] Updated weights for policy 0, policy_version 153942 (0.0010) [2023-12-26 16:29:18,027][105620] Updated weights for policy 1, policy_version 154698 (0.0007) [2023-12-26 16:29:18,094][105620] Updated weights for policy 1, policy_version 154708 (0.0005) [2023-12-26 16:29:18,148][105620] Updated weights for policy 1, policy_version 154718 (0.0008) [2023-12-26 16:29:18,210][105620] Updated weights for policy 1, policy_version 154728 (0.0010) [2023-12-26 16:29:18,509][105692] Updated weights for policy 0, policy_version 153952 (0.0010) [2023-12-26 16:29:18,564][105692] Updated weights for policy 0, policy_version 153962 (0.0010) [2023-12-26 16:29:18,632][105692] Updated weights for policy 0, policy_version 153972 (0.0010) [2023-12-26 16:29:18,906][105620] Updated weights for policy 1, policy_version 154738 (0.0011) [2023-12-26 16:29:18,959][105620] Updated weights for policy 1, policy_version 154748 (0.0011) [2023-12-26 16:29:19,008][105620] Updated weights for policy 1, policy_version 154758 (0.0011) [2023-12-26 16:29:19,262][105692] Updated weights for policy 0, policy_version 153982 (0.0008) [2023-12-26 16:29:19,322][105692] Updated weights for policy 0, policy_version 153992 (0.0011) [2023-12-26 16:29:19,394][105692] Updated weights for policy 0, policy_version 154002 (0.0011) [2023-12-26 16:29:19,814][105620] Updated weights for policy 1, policy_version 154768 (0.0008) [2023-12-26 16:29:19,877][105620] Updated weights for policy 1, policy_version 154778 (0.0008) [2023-12-26 16:29:19,942][105620] Updated weights for policy 1, policy_version 154788 (0.0008) [2023-12-26 16:29:20,114][105692] Updated weights for policy 0, policy_version 154012 (0.0009) [2023-12-26 16:29:20,170][105692] Updated weights for policy 0, policy_version 154022 (0.0011) [2023-12-26 16:29:20,228][105692] Updated weights for policy 0, policy_version 154032 (0.0010) [2023-12-26 16:29:20,675][105620] Updated weights for policy 1, policy_version 154798 (0.0007) [2023-12-26 16:29:20,724][105620] Updated weights for policy 1, policy_version 154808 (0.0009) [2023-12-26 16:29:20,780][105620] Updated weights for policy 1, policy_version 154818 (0.0008) [2023-12-26 16:29:20,870][105692] Updated weights for policy 0, policy_version 154042 (0.0006) [2023-12-26 16:29:20,923][105692] Updated weights for policy 0, policy_version 154052 (0.0010) [2023-12-26 16:29:20,982][105692] Updated weights for policy 0, policy_version 154062 (0.0011) [2023-12-26 16:29:21,049][105692] Updated weights for policy 0, policy_version 154072 (0.0010) [2023-12-26 16:29:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 79093760. Throughput: 0: 9857.5, 1: 9792.0. Samples: 79077444. Policy #0 lag: (min: 27.0, avg: 38.3, max: 59.0) [2023-12-26 16:29:21,062][104569] Avg episode reward: [(0, '8227.164'), (1, '5957.413')] [2023-12-26 16:29:21,540][105620] Updated weights for policy 1, policy_version 154828 (0.0009) [2023-12-26 16:29:21,597][105620] Updated weights for policy 1, policy_version 154838 (0.0007) [2023-12-26 16:29:21,664][105620] Updated weights for policy 1, policy_version 154848 (0.0008) [2023-12-26 16:29:21,793][105692] Updated weights for policy 0, policy_version 154082 (0.0006) [2023-12-26 16:29:21,850][105692] Updated weights for policy 0, policy_version 154092 (0.0005) [2023-12-26 16:29:21,914][105692] Updated weights for policy 0, policy_version 154102 (0.0005) [2023-12-26 16:29:22,439][105620] Updated weights for policy 1, policy_version 154858 (0.0008) [2023-12-26 16:29:22,500][105620] Updated weights for policy 1, policy_version 154868 (0.0008) [2023-12-26 16:29:22,559][105620] Updated weights for policy 1, policy_version 154878 (0.0008) [2023-12-26 16:29:22,610][105692] Updated weights for policy 0, policy_version 154112 (0.0008) [2023-12-26 16:29:22,622][105620] Updated weights for policy 1, policy_version 154888 (0.0006) [2023-12-26 16:29:22,675][105692] Updated weights for policy 0, policy_version 154122 (0.0009) [2023-12-26 16:29:22,727][105692] Updated weights for policy 0, policy_version 154132 (0.0010) [2023-12-26 16:29:23,305][105620] Updated weights for policy 1, policy_version 154898 (0.0008) [2023-12-26 16:29:23,361][105620] Updated weights for policy 1, policy_version 154908 (0.0008) [2023-12-26 16:29:23,422][105620] Updated weights for policy 1, policy_version 154918 (0.0009) [2023-12-26 16:29:23,461][105692] Updated weights for policy 0, policy_version 154142 (0.0007) [2023-12-26 16:29:23,513][105692] Updated weights for policy 0, policy_version 154152 (0.0005) [2023-12-26 16:29:23,567][105692] Updated weights for policy 0, policy_version 154162 (0.0008) [2023-12-26 16:29:24,092][105620] Updated weights for policy 1, policy_version 154928 (0.0009) [2023-12-26 16:29:24,156][105620] Updated weights for policy 1, policy_version 154938 (0.0006) [2023-12-26 16:29:24,157][105692] Updated weights for policy 0, policy_version 154172 (0.0009) [2023-12-26 16:29:24,214][105692] Updated weights for policy 0, policy_version 154182 (0.0007) [2023-12-26 16:29:24,216][105620] Updated weights for policy 1, policy_version 154948 (0.0007) [2023-12-26 16:29:24,269][105692] Updated weights for policy 0, policy_version 154192 (0.0008) [2023-12-26 16:29:24,908][105620] Updated weights for policy 1, policy_version 154958 (0.0006) [2023-12-26 16:29:24,956][105620] Updated weights for policy 1, policy_version 154968 (0.0009) [2023-12-26 16:29:25,014][105620] Updated weights for policy 1, policy_version 154978 (0.0010) [2023-12-26 16:29:25,033][105692] Updated weights for policy 0, policy_version 154202 (0.0009) [2023-12-26 16:29:25,088][105692] Updated weights for policy 0, policy_version 154212 (0.0007) [2023-12-26 16:29:25,147][105692] Updated weights for policy 0, policy_version 154222 (0.0008) [2023-12-26 16:29:25,209][105692] Updated weights for policy 0, policy_version 154232 (0.0008) [2023-12-26 16:29:25,712][105620] Updated weights for policy 1, policy_version 154988 (0.0010) [2023-12-26 16:29:25,759][105620] Updated weights for policy 1, policy_version 154998 (0.0009) [2023-12-26 16:29:25,805][105620] Updated weights for policy 1, policy_version 155008 (0.0008) [2023-12-26 16:29:25,978][105692] Updated weights for policy 0, policy_version 154242 (0.0009) [2023-12-26 16:29:26,031][105692] Updated weights for policy 0, policy_version 154252 (0.0011) [2023-12-26 16:29:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 79183872. Throughput: 0: 9778.0, 1: 9669.0. Samples: 79194184. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:29:26,063][104569] Avg episode reward: [(0, '8994.708'), (1, '7133.397')] [2023-12-26 16:29:26,079][105692] Updated weights for policy 0, policy_version 154262 (0.0009) [2023-12-26 16:29:26,485][105620] Updated weights for policy 1, policy_version 155018 (0.0008) [2023-12-26 16:29:26,538][105620] Updated weights for policy 1, policy_version 155028 (0.0008) [2023-12-26 16:29:26,600][105620] Updated weights for policy 1, policy_version 155038 (0.0009) [2023-12-26 16:29:26,657][105620] Updated weights for policy 1, policy_version 155048 (0.0009) [2023-12-26 16:29:26,881][105692] Updated weights for policy 0, policy_version 154272 (0.0006) [2023-12-26 16:29:26,940][105692] Updated weights for policy 0, policy_version 154282 (0.0006) [2023-12-26 16:29:27,003][105692] Updated weights for policy 0, policy_version 154292 (0.0008) [2023-12-26 16:29:27,350][105620] Updated weights for policy 1, policy_version 155058 (0.0009) [2023-12-26 16:29:27,404][105620] Updated weights for policy 1, policy_version 155068 (0.0009) [2023-12-26 16:29:27,456][105620] Updated weights for policy 1, policy_version 155078 (0.0008) [2023-12-26 16:29:27,587][105692] Updated weights for policy 0, policy_version 154302 (0.0008) [2023-12-26 16:29:27,633][105692] Updated weights for policy 0, policy_version 154312 (0.0008) [2023-12-26 16:29:27,692][105692] Updated weights for policy 0, policy_version 154322 (0.0009) [2023-12-26 16:29:28,198][105620] Updated weights for policy 1, policy_version 155088 (0.0006) [2023-12-26 16:29:28,252][105620] Updated weights for policy 1, policy_version 155098 (0.0005) [2023-12-26 16:29:28,301][105620] Updated weights for policy 1, policy_version 155108 (0.0009) [2023-12-26 16:29:28,319][105692] Updated weights for policy 0, policy_version 154332 (0.0008) [2023-12-26 16:29:28,391][105692] Updated weights for policy 0, policy_version 154342 (0.0010) [2023-12-26 16:29:28,454][105692] Updated weights for policy 0, policy_version 154352 (0.0009) [2023-12-26 16:29:28,982][105620] Updated weights for policy 1, policy_version 155118 (0.0006) [2023-12-26 16:29:29,025][105620] Updated weights for policy 1, policy_version 155128 (0.0005) [2023-12-26 16:29:29,070][105620] Updated weights for policy 1, policy_version 155138 (0.0005) [2023-12-26 16:29:29,209][105692] Updated weights for policy 0, policy_version 154362 (0.0008) [2023-12-26 16:29:29,272][105692] Updated weights for policy 0, policy_version 154372 (0.0009) [2023-12-26 16:29:29,332][105692] Updated weights for policy 0, policy_version 154382 (0.0010) [2023-12-26 16:29:29,394][105692] Updated weights for policy 0, policy_version 154392 (0.0009) [2023-12-26 16:29:29,672][105620] Updated weights for policy 1, policy_version 155148 (0.0007) [2023-12-26 16:29:29,733][105620] Updated weights for policy 1, policy_version 155158 (0.0009) [2023-12-26 16:29:29,797][105620] Updated weights for policy 1, policy_version 155168 (0.0008) [2023-12-26 16:29:30,196][105692] Updated weights for policy 0, policy_version 154402 (0.0010) [2023-12-26 16:29:30,255][105692] Updated weights for policy 0, policy_version 154412 (0.0010) [2023-12-26 16:29:30,308][105692] Updated weights for policy 0, policy_version 154422 (0.0009) [2023-12-26 16:29:30,444][105620] Updated weights for policy 1, policy_version 155178 (0.0009) [2023-12-26 16:29:30,509][105620] Updated weights for policy 1, policy_version 155188 (0.0009) [2023-12-26 16:29:30,573][105620] Updated weights for policy 1, policy_version 155198 (0.0009) [2023-12-26 16:29:30,631][105620] Updated weights for policy 1, policy_version 155208 (0.0009) [2023-12-26 16:29:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 79282176. Throughput: 0: 9834.1, 1: 9712.1. Samples: 79254968. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:29:31,062][104569] Avg episode reward: [(0, '9266.154'), (1, '8806.403')] [2023-12-26 16:29:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000155208_39739392.pth... [2023-12-26 16:29:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000154088_39452672.pth [2023-12-26 16:29:31,091][105692] Updated weights for policy 0, policy_version 154432 (0.0009) [2023-12-26 16:29:31,158][105692] Updated weights for policy 0, policy_version 154442 (0.0007) [2023-12-26 16:29:31,222][105692] Updated weights for policy 0, policy_version 154452 (0.0005) [2023-12-26 16:29:31,246][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000154456_39550976.pth... [2023-12-26 16:29:31,252][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000153304_39256064.pth [2023-12-26 16:29:31,368][105620] Updated weights for policy 1, policy_version 155218 (0.0008) [2023-12-26 16:29:31,438][105620] Updated weights for policy 1, policy_version 155228 (0.0008) [2023-12-26 16:29:31,503][105620] Updated weights for policy 1, policy_version 155238 (0.0008) [2023-12-26 16:29:31,838][105692] Updated weights for policy 0, policy_version 154462 (0.0006) [2023-12-26 16:29:31,882][105692] Updated weights for policy 0, policy_version 154472 (0.0005) [2023-12-26 16:29:31,930][105692] Updated weights for policy 0, policy_version 154482 (0.0007) [2023-12-26 16:29:32,171][105620] Updated weights for policy 1, policy_version 155248 (0.0006) [2023-12-26 16:29:32,226][105620] Updated weights for policy 1, policy_version 155258 (0.0005) [2023-12-26 16:29:32,290][105620] Updated weights for policy 1, policy_version 155268 (0.0009) [2023-12-26 16:29:32,612][105692] Updated weights for policy 0, policy_version 154493 (0.0008) [2023-12-26 16:29:32,668][105692] Updated weights for policy 0, policy_version 154503 (0.0005) [2023-12-26 16:29:32,736][105692] Updated weights for policy 0, policy_version 154513 (0.0006) [2023-12-26 16:29:33,052][105620] Updated weights for policy 1, policy_version 155278 (0.0009) [2023-12-26 16:29:33,109][105620] Updated weights for policy 1, policy_version 155288 (0.0008) [2023-12-26 16:29:33,159][105620] Updated weights for policy 1, policy_version 155298 (0.0008) [2023-12-26 16:29:33,409][105692] Updated weights for policy 0, policy_version 154523 (0.0008) [2023-12-26 16:29:33,466][105692] Updated weights for policy 0, policy_version 154533 (0.0009) [2023-12-26 16:29:33,515][105692] Updated weights for policy 0, policy_version 154543 (0.0009) [2023-12-26 16:29:33,854][105620] Updated weights for policy 1, policy_version 155308 (0.0007) [2023-12-26 16:29:33,899][105620] Updated weights for policy 1, policy_version 155318 (0.0005) [2023-12-26 16:29:33,951][105620] Updated weights for policy 1, policy_version 155328 (0.0005) [2023-12-26 16:29:34,330][105692] Updated weights for policy 0, policy_version 154553 (0.0008) [2023-12-26 16:29:34,401][105692] Updated weights for policy 0, policy_version 154563 (0.0008) [2023-12-26 16:29:34,467][105692] Updated weights for policy 0, policy_version 154573 (0.0008) [2023-12-26 16:29:34,524][105692] Updated weights for policy 0, policy_version 154583 (0.0008) [2023-12-26 16:29:34,539][105620] Updated weights for policy 1, policy_version 155338 (0.0005) [2023-12-26 16:29:34,603][105620] Updated weights for policy 1, policy_version 155348 (0.0008) [2023-12-26 16:29:34,675][105620] Updated weights for policy 1, policy_version 155358 (0.0007) [2023-12-26 16:29:34,744][105620] Updated weights for policy 1, policy_version 155368 (0.0009) [2023-12-26 16:29:35,222][105692] Updated weights for policy 0, policy_version 154593 (0.0009) [2023-12-26 16:29:35,283][105692] Updated weights for policy 0, policy_version 154603 (0.0009) [2023-12-26 16:29:35,334][105692] Updated weights for policy 0, policy_version 154613 (0.0008) [2023-12-26 16:29:35,463][105620] Updated weights for policy 1, policy_version 155378 (0.0010) [2023-12-26 16:29:35,526][105620] Updated weights for policy 1, policy_version 155388 (0.0009) [2023-12-26 16:29:35,621][105620] Updated weights for policy 1, policy_version 155398 (0.0011) [2023-12-26 16:29:36,018][105692] Updated weights for policy 0, policy_version 154623 (0.0008) [2023-12-26 16:29:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 79380480. Throughput: 0: 9710.2, 1: 9790.2. Samples: 79372876. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:29:36,062][105692] Updated weights for policy 0, policy_version 154633 (0.0007) [2023-12-26 16:29:36,063][104569] Avg episode reward: [(0, '9357.324'), (1, '8903.643')] [2023-12-26 16:29:36,115][105692] Updated weights for policy 0, policy_version 154643 (0.0006) [2023-12-26 16:29:36,263][105620] Updated weights for policy 1, policy_version 155408 (0.0011) [2023-12-26 16:29:36,333][105620] Updated weights for policy 1, policy_version 155418 (0.0011) [2023-12-26 16:29:36,397][105620] Updated weights for policy 1, policy_version 155428 (0.0011) [2023-12-26 16:29:36,820][105692] Updated weights for policy 0, policy_version 154653 (0.0007) [2023-12-26 16:29:36,875][105692] Updated weights for policy 0, policy_version 154663 (0.0006) [2023-12-26 16:29:36,927][105692] Updated weights for policy 0, policy_version 154673 (0.0009) [2023-12-26 16:29:37,033][105620] Updated weights for policy 1, policy_version 155438 (0.0008) [2023-12-26 16:29:37,090][105620] Updated weights for policy 1, policy_version 155448 (0.0005) [2023-12-26 16:29:37,152][105620] Updated weights for policy 1, policy_version 155458 (0.0006) [2023-12-26 16:29:37,736][105692] Updated weights for policy 0, policy_version 154683 (0.0008) [2023-12-26 16:29:37,779][105620] Updated weights for policy 1, policy_version 155468 (0.0007) [2023-12-26 16:29:37,790][105692] Updated weights for policy 0, policy_version 154693 (0.0007) [2023-12-26 16:29:37,839][105620] Updated weights for policy 1, policy_version 155478 (0.0009) [2023-12-26 16:29:37,844][105692] Updated weights for policy 0, policy_version 154703 (0.0005) [2023-12-26 16:29:37,900][105620] Updated weights for policy 1, policy_version 155488 (0.0009) [2023-12-26 16:29:38,437][105692] Updated weights for policy 0, policy_version 154713 (0.0006) [2023-12-26 16:29:38,499][105692] Updated weights for policy 0, policy_version 154723 (0.0009) [2023-12-26 16:29:38,546][105692] Updated weights for policy 0, policy_version 154733 (0.0009) [2023-12-26 16:29:38,593][105692] Updated weights for policy 0, policy_version 154743 (0.0009) [2023-12-26 16:29:38,694][105620] Updated weights for policy 1, policy_version 155498 (0.0009) [2023-12-26 16:29:38,763][105620] Updated weights for policy 1, policy_version 155508 (0.0009) [2023-12-26 16:29:38,818][105620] Updated weights for policy 1, policy_version 155518 (0.0009) [2023-12-26 16:29:38,883][105620] Updated weights for policy 1, policy_version 155528 (0.0009) [2023-12-26 16:29:39,369][105692] Updated weights for policy 0, policy_version 154753 (0.0008) [2023-12-26 16:29:39,436][105692] Updated weights for policy 0, policy_version 154763 (0.0009) [2023-12-26 16:29:39,489][105692] Updated weights for policy 0, policy_version 154773 (0.0009) [2023-12-26 16:29:39,673][105620] Updated weights for policy 1, policy_version 155538 (0.0010) [2023-12-26 16:29:39,742][105620] Updated weights for policy 1, policy_version 155548 (0.0007) [2023-12-26 16:29:39,804][105620] Updated weights for policy 1, policy_version 155558 (0.0008) [2023-12-26 16:29:40,216][105692] Updated weights for policy 0, policy_version 154783 (0.0006) [2023-12-26 16:29:40,281][105692] Updated weights for policy 0, policy_version 154793 (0.0006) [2023-12-26 16:29:40,344][105692] Updated weights for policy 0, policy_version 154803 (0.0008) [2023-12-26 16:29:40,601][105620] Updated weights for policy 1, policy_version 155568 (0.0009) [2023-12-26 16:29:40,657][105620] Updated weights for policy 1, policy_version 155578 (0.0009) [2023-12-26 16:29:40,715][105620] Updated weights for policy 1, policy_version 155588 (0.0009) [2023-12-26 16:29:41,034][105692] Updated weights for policy 0, policy_version 154813 (0.0007) [2023-12-26 16:29:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 79478784. Throughput: 0: 9787.2, 1: 9689.6. Samples: 79489300. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:29:41,062][104569] Avg episode reward: [(0, '9357.265'), (1, '8727.008')] [2023-12-26 16:29:41,106][105692] Updated weights for policy 0, policy_version 154823 (0.0008) [2023-12-26 16:29:41,179][105692] Updated weights for policy 0, policy_version 154833 (0.0010) [2023-12-26 16:29:41,545][105620] Updated weights for policy 1, policy_version 155598 (0.0007) [2023-12-26 16:29:41,601][105620] Updated weights for policy 1, policy_version 155608 (0.0008) [2023-12-26 16:29:41,668][105620] Updated weights for policy 1, policy_version 155618 (0.0008) [2023-12-26 16:29:41,993][105692] Updated weights for policy 0, policy_version 154843 (0.0008) [2023-12-26 16:29:42,049][105692] Updated weights for policy 0, policy_version 154853 (0.0008) [2023-12-26 16:29:42,115][105692] Updated weights for policy 0, policy_version 154863 (0.0008) [2023-12-26 16:29:42,408][105620] Updated weights for policy 1, policy_version 155628 (0.0008) [2023-12-26 16:29:42,461][105620] Updated weights for policy 1, policy_version 155638 (0.0008) [2023-12-26 16:29:42,509][105620] Updated weights for policy 1, policy_version 155648 (0.0009) [2023-12-26 16:29:42,805][105692] Updated weights for policy 0, policy_version 154873 (0.0008) [2023-12-26 16:29:42,866][105692] Updated weights for policy 0, policy_version 154883 (0.0009) [2023-12-26 16:29:42,925][105692] Updated weights for policy 0, policy_version 154893 (0.0009) [2023-12-26 16:29:42,981][105692] Updated weights for policy 0, policy_version 154903 (0.0009) [2023-12-26 16:29:43,350][105620] Updated weights for policy 1, policy_version 155658 (0.0009) [2023-12-26 16:29:43,407][105620] Updated weights for policy 1, policy_version 155668 (0.0007) [2023-12-26 16:29:43,458][105620] Updated weights for policy 1, policy_version 155678 (0.0010) [2023-12-26 16:29:43,515][105620] Updated weights for policy 1, policy_version 155688 (0.0010) [2023-12-26 16:29:43,654][105692] Updated weights for policy 0, policy_version 154913 (0.0007) [2023-12-26 16:29:43,709][105692] Updated weights for policy 0, policy_version 154923 (0.0006) [2023-12-26 16:29:43,765][105692] Updated weights for policy 0, policy_version 154933 (0.0006) [2023-12-26 16:29:44,358][105692] Updated weights for policy 0, policy_version 154943 (0.0005) [2023-12-26 16:29:44,361][105620] Updated weights for policy 1, policy_version 155698 (0.0008) [2023-12-26 16:29:44,407][105692] Updated weights for policy 0, policy_version 154953 (0.0005) [2023-12-26 16:29:44,420][105620] Updated weights for policy 1, policy_version 155708 (0.0008) [2023-12-26 16:29:44,462][105692] Updated weights for policy 0, policy_version 154963 (0.0006) [2023-12-26 16:29:44,483][105620] Updated weights for policy 1, policy_version 155718 (0.0009) [2023-12-26 16:29:45,093][105692] Updated weights for policy 0, policy_version 154973 (0.0008) [2023-12-26 16:29:45,157][105692] Updated weights for policy 0, policy_version 154983 (0.0011) [2023-12-26 16:29:45,216][105692] Updated weights for policy 0, policy_version 154993 (0.0010) [2023-12-26 16:29:45,328][105620] Updated weights for policy 1, policy_version 155728 (0.0009) [2023-12-26 16:29:45,390][105620] Updated weights for policy 1, policy_version 155738 (0.0008) [2023-12-26 16:29:45,454][105620] Updated weights for policy 1, policy_version 155748 (0.0008) [2023-12-26 16:29:45,960][105692] Updated weights for policy 0, policy_version 155003 (0.0009) [2023-12-26 16:29:46,024][105692] Updated weights for policy 0, policy_version 155013 (0.0009) [2023-12-26 16:29:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19521.9). Total num frames: 79568896. Throughput: 0: 9753.6, 1: 9634.1. Samples: 79544304. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:29:46,062][104569] Avg episode reward: [(0, '3167.001'), (1, '8637.612')] [2023-12-26 16:29:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000155752_39878656.pth... [2023-12-26 16:29:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000154632_39591936.pth [2023-12-26 16:29:46,086][105692] Updated weights for policy 0, policy_version 155023 (0.0010) [2023-12-26 16:29:46,136][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000155032_39698432.pth... [2023-12-26 16:29:46,141][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000153880_39403520.pth [2023-12-26 16:29:46,229][105620] Updated weights for policy 1, policy_version 155758 (0.0009) [2023-12-26 16:29:46,277][105620] Updated weights for policy 1, policy_version 155768 (0.0007) [2023-12-26 16:29:46,326][105620] Updated weights for policy 1, policy_version 155778 (0.0008) [2023-12-26 16:29:46,793][105692] Updated weights for policy 0, policy_version 155033 (0.0009) [2023-12-26 16:29:46,840][105692] Updated weights for policy 0, policy_version 155043 (0.0010) [2023-12-26 16:29:46,891][105692] Updated weights for policy 0, policy_version 155053 (0.0010) [2023-12-26 16:29:46,943][105692] Updated weights for policy 0, policy_version 155063 (0.0005) [2023-12-26 16:29:47,128][105620] Updated weights for policy 1, policy_version 155788 (0.0008) [2023-12-26 16:29:47,177][105620] Updated weights for policy 1, policy_version 155798 (0.0008) [2023-12-26 16:29:47,221][105620] Updated weights for policy 1, policy_version 155808 (0.0008) [2023-12-26 16:29:47,635][105692] Updated weights for policy 0, policy_version 155073 (0.0005) [2023-12-26 16:29:47,690][105692] Updated weights for policy 0, policy_version 155083 (0.0005) [2023-12-26 16:29:47,752][105692] Updated weights for policy 0, policy_version 155093 (0.0009) [2023-12-26 16:29:48,003][105620] Updated weights for policy 1, policy_version 155818 (0.0008) [2023-12-26 16:29:48,065][105620] Updated weights for policy 1, policy_version 155828 (0.0010) [2023-12-26 16:29:48,134][105620] Updated weights for policy 1, policy_version 155838 (0.0010) [2023-12-26 16:29:48,192][105620] Updated weights for policy 1, policy_version 155848 (0.0010) [2023-12-26 16:29:48,456][105692] Updated weights for policy 0, policy_version 155103 (0.0009) [2023-12-26 16:29:48,506][105692] Updated weights for policy 0, policy_version 155113 (0.0010) [2023-12-26 16:29:48,558][105692] Updated weights for policy 0, policy_version 155123 (0.0010) [2023-12-26 16:29:48,829][105620] Updated weights for policy 1, policy_version 155858 (0.0011) [2023-12-26 16:29:48,891][105620] Updated weights for policy 1, policy_version 155868 (0.0011) [2023-12-26 16:29:48,956][105620] Updated weights for policy 1, policy_version 155878 (0.0011) [2023-12-26 16:29:49,217][105692] Updated weights for policy 0, policy_version 155133 (0.0009) [2023-12-26 16:29:49,285][105692] Updated weights for policy 0, policy_version 155143 (0.0010) [2023-12-26 16:29:49,350][105692] Updated weights for policy 0, policy_version 155153 (0.0011) [2023-12-26 16:29:49,635][105620] Updated weights for policy 1, policy_version 155888 (0.0010) [2023-12-26 16:29:49,695][105620] Updated weights for policy 1, policy_version 155898 (0.0011) [2023-12-26 16:29:49,755][105620] Updated weights for policy 1, policy_version 155908 (0.0011) [2023-12-26 16:29:50,097][105692] Updated weights for policy 0, policy_version 155163 (0.0012) [2023-12-26 16:29:50,148][105692] Updated weights for policy 0, policy_version 155173 (0.0007) [2023-12-26 16:29:50,206][105692] Updated weights for policy 0, policy_version 155183 (0.0006) [2023-12-26 16:29:50,538][105620] Updated weights for policy 1, policy_version 155918 (0.0011) [2023-12-26 16:29:50,607][105620] Updated weights for policy 1, policy_version 155928 (0.0011) [2023-12-26 16:29:50,655][105620] Updated weights for policy 1, policy_version 155938 (0.0010) [2023-12-26 16:29:50,894][105692] Updated weights for policy 0, policy_version 155193 (0.0005) [2023-12-26 16:29:50,954][105692] Updated weights for policy 0, policy_version 155203 (0.0008) [2023-12-26 16:29:51,008][105692] Updated weights for policy 0, policy_version 155213 (0.0008) [2023-12-26 16:29:51,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 79667200. Throughput: 0: 9789.6, 1: 9553.7. Samples: 79660828. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:29:51,063][104569] Avg episode reward: [(0, '936.270'), (1, '8901.515')] [2023-12-26 16:29:51,071][105692] Updated weights for policy 0, policy_version 155223 (0.0008) [2023-12-26 16:29:51,411][105620] Updated weights for policy 1, policy_version 155948 (0.0011) [2023-12-26 16:29:51,467][105620] Updated weights for policy 1, policy_version 155958 (0.0010) [2023-12-26 16:29:51,527][105620] Updated weights for policy 1, policy_version 155968 (0.0010) [2023-12-26 16:29:51,884][105692] Updated weights for policy 0, policy_version 155233 (0.0007) [2023-12-26 16:29:51,931][105692] Updated weights for policy 0, policy_version 155243 (0.0005) [2023-12-26 16:29:51,992][105692] Updated weights for policy 0, policy_version 155253 (0.0006) [2023-12-26 16:29:52,283][105620] Updated weights for policy 1, policy_version 155978 (0.0010) [2023-12-26 16:29:52,346][105620] Updated weights for policy 1, policy_version 155988 (0.0010) [2023-12-26 16:29:52,410][105620] Updated weights for policy 1, policy_version 155998 (0.0009) [2023-12-26 16:29:52,476][105620] Updated weights for policy 1, policy_version 156008 (0.0005) [2023-12-26 16:29:52,677][105692] Updated weights for policy 0, policy_version 155263 (0.0009) [2023-12-26 16:29:52,736][105692] Updated weights for policy 0, policy_version 155273 (0.0009) [2023-12-26 16:29:52,793][105692] Updated weights for policy 0, policy_version 155283 (0.0009) [2023-12-26 16:29:53,159][105620] Updated weights for policy 1, policy_version 156018 (0.0005) [2023-12-26 16:29:53,219][105620] Updated weights for policy 1, policy_version 156028 (0.0005) [2023-12-26 16:29:53,280][105620] Updated weights for policy 1, policy_version 156038 (0.0008) [2023-12-26 16:29:53,609][105692] Updated weights for policy 0, policy_version 155293 (0.0009) [2023-12-26 16:29:53,656][105692] Updated weights for policy 0, policy_version 155303 (0.0009) [2023-12-26 16:29:53,707][105692] Updated weights for policy 0, policy_version 155313 (0.0009) [2023-12-26 16:29:53,947][105620] Updated weights for policy 1, policy_version 156048 (0.0008) [2023-12-26 16:29:54,002][105620] Updated weights for policy 1, policy_version 156058 (0.0008) [2023-12-26 16:29:54,058][105620] Updated weights for policy 1, policy_version 156068 (0.0008) [2023-12-26 16:29:54,363][105692] Updated weights for policy 0, policy_version 155323 (0.0009) [2023-12-26 16:29:54,415][105692] Updated weights for policy 0, policy_version 155333 (0.0006) [2023-12-26 16:29:54,480][105692] Updated weights for policy 0, policy_version 155343 (0.0011) [2023-12-26 16:29:54,827][105620] Updated weights for policy 1, policy_version 156078 (0.0008) [2023-12-26 16:29:54,884][105620] Updated weights for policy 1, policy_version 156088 (0.0008) [2023-12-26 16:29:54,936][105620] Updated weights for policy 1, policy_version 156098 (0.0009) [2023-12-26 16:29:55,168][105692] Updated weights for policy 0, policy_version 155353 (0.0009) [2023-12-26 16:29:55,215][105692] Updated weights for policy 0, policy_version 155363 (0.0005) [2023-12-26 16:29:55,261][105692] Updated weights for policy 0, policy_version 155373 (0.0005) [2023-12-26 16:29:55,312][105692] Updated weights for policy 0, policy_version 155383 (0.0007) [2023-12-26 16:29:55,679][105620] Updated weights for policy 1, policy_version 156108 (0.0010) [2023-12-26 16:29:55,734][105620] Updated weights for policy 1, policy_version 156118 (0.0010) [2023-12-26 16:29:55,785][105620] Updated weights for policy 1, policy_version 156128 (0.0010) [2023-12-26 16:29:56,031][105692] Updated weights for policy 0, policy_version 155393 (0.0007) [2023-12-26 16:29:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 79765504. Throughput: 0: 9734.0, 1: 9578.5. Samples: 79775588. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:29:56,063][104569] Avg episode reward: [(0, '865.650'), (1, '8552.012')] [2023-12-26 16:29:56,089][105692] Updated weights for policy 0, policy_version 155403 (0.0005) [2023-12-26 16:29:56,146][105692] Updated weights for policy 0, policy_version 155413 (0.0005) [2023-12-26 16:29:56,542][105620] Updated weights for policy 1, policy_version 156138 (0.0010) [2023-12-26 16:29:56,603][105620] Updated weights for policy 1, policy_version 156148 (0.0007) [2023-12-26 16:29:56,667][105620] Updated weights for policy 1, policy_version 156158 (0.0008) [2023-12-26 16:29:56,724][105620] Updated weights for policy 1, policy_version 156168 (0.0010) [2023-12-26 16:29:56,760][105692] Updated weights for policy 0, policy_version 155423 (0.0005) [2023-12-26 16:29:56,810][105692] Updated weights for policy 0, policy_version 155433 (0.0005) [2023-12-26 16:29:56,861][105692] Updated weights for policy 0, policy_version 155443 (0.0005) [2023-12-26 16:29:57,381][105620] Updated weights for policy 1, policy_version 156178 (0.0006) [2023-12-26 16:29:57,437][105620] Updated weights for policy 1, policy_version 156188 (0.0008) [2023-12-26 16:29:57,484][105620] Updated weights for policy 1, policy_version 156198 (0.0009) [2023-12-26 16:29:57,524][105692] Updated weights for policy 0, policy_version 155453 (0.0007) [2023-12-26 16:29:57,582][105692] Updated weights for policy 0, policy_version 155465 (0.0010) [2023-12-26 16:29:57,634][105692] Updated weights for policy 0, policy_version 155475 (0.0010) [2023-12-26 16:29:58,074][105620] Updated weights for policy 1, policy_version 156208 (0.0006) [2023-12-26 16:29:58,143][105620] Updated weights for policy 1, policy_version 156218 (0.0010) [2023-12-26 16:29:58,202][105620] Updated weights for policy 1, policy_version 156228 (0.0010) [2023-12-26 16:29:58,431][105692] Updated weights for policy 0, policy_version 155485 (0.0011) [2023-12-26 16:29:58,490][105692] Updated weights for policy 0, policy_version 155495 (0.0010) [2023-12-26 16:29:58,551][105692] Updated weights for policy 0, policy_version 155505 (0.0011) [2023-12-26 16:29:59,011][105620] Updated weights for policy 1, policy_version 156238 (0.0010) [2023-12-26 16:29:59,074][105620] Updated weights for policy 1, policy_version 156248 (0.0008) [2023-12-26 16:29:59,139][105620] Updated weights for policy 1, policy_version 156258 (0.0007) [2023-12-26 16:29:59,435][105692] Updated weights for policy 0, policy_version 155515 (0.0010) [2023-12-26 16:29:59,483][105692] Updated weights for policy 0, policy_version 155525 (0.0010) [2023-12-26 16:29:59,530][105692] Updated weights for policy 0, policy_version 155535 (0.0010) [2023-12-26 16:29:59,893][105620] Updated weights for policy 1, policy_version 156268 (0.0007) [2023-12-26 16:29:59,949][105620] Updated weights for policy 1, policy_version 156278 (0.0008) [2023-12-26 16:29:59,996][105620] Updated weights for policy 1, policy_version 156288 (0.0008) [2023-12-26 16:30:00,309][105692] Updated weights for policy 0, policy_version 155545 (0.0010) [2023-12-26 16:30:00,364][105692] Updated weights for policy 0, policy_version 155555 (0.0010) [2023-12-26 16:30:00,423][105692] Updated weights for policy 0, policy_version 155565 (0.0010) [2023-12-26 16:30:00,481][105692] Updated weights for policy 0, policy_version 155575 (0.0010) [2023-12-26 16:30:00,674][105620] Updated weights for policy 1, policy_version 156298 (0.0007) [2023-12-26 16:30:00,728][105620] Updated weights for policy 1, policy_version 156308 (0.0006) [2023-12-26 16:30:00,781][105620] Updated weights for policy 1, policy_version 156318 (0.0005) [2023-12-26 16:30:00,854][105620] Updated weights for policy 1, policy_version 156328 (0.0005) [2023-12-26 16:30:01,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 79863808. Throughput: 0: 9789.4, 1: 9632.3. Samples: 79835444. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:30:01,063][104569] Avg episode reward: [(0, '5277.962'), (1, '8912.334')] [2023-12-26 16:30:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000155576_39837696.pth... [2023-12-26 16:30:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000156328_40026112.pth... [2023-12-26 16:30:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000154456_39550976.pth [2023-12-26 16:30:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000155208_39739392.pth [2023-12-26 16:30:01,215][105692] Updated weights for policy 0, policy_version 155585 (0.0009) [2023-12-26 16:30:01,272][105692] Updated weights for policy 0, policy_version 155595 (0.0008) [2023-12-26 16:30:01,335][105692] Updated weights for policy 0, policy_version 155605 (0.0008) [2023-12-26 16:30:01,426][105620] Updated weights for policy 1, policy_version 156338 (0.0008) [2023-12-26 16:30:01,476][105620] Updated weights for policy 1, policy_version 156348 (0.0008) [2023-12-26 16:30:01,536][105620] Updated weights for policy 1, policy_version 156358 (0.0009) [2023-12-26 16:30:02,073][105692] Updated weights for policy 0, policy_version 155615 (0.0006) [2023-12-26 16:30:02,135][105692] Updated weights for policy 0, policy_version 155625 (0.0006) [2023-12-26 16:30:02,200][105692] Updated weights for policy 0, policy_version 155635 (0.0006) [2023-12-26 16:30:02,294][105620] Updated weights for policy 1, policy_version 156368 (0.0010) [2023-12-26 16:30:02,363][105620] Updated weights for policy 1, policy_version 156378 (0.0010) [2023-12-26 16:30:02,421][105620] Updated weights for policy 1, policy_version 156388 (0.0010) [2023-12-26 16:30:02,873][105692] Updated weights for policy 0, policy_version 155645 (0.0008) [2023-12-26 16:30:02,928][105692] Updated weights for policy 0, policy_version 155655 (0.0010) [2023-12-26 16:30:02,982][105692] Updated weights for policy 0, policy_version 155665 (0.0010) [2023-12-26 16:30:03,085][105620] Updated weights for policy 1, policy_version 156398 (0.0007) [2023-12-26 16:30:03,147][105620] Updated weights for policy 1, policy_version 156408 (0.0006) [2023-12-26 16:30:03,213][105620] Updated weights for policy 1, policy_version 156418 (0.0010) [2023-12-26 16:30:03,669][105692] Updated weights for policy 0, policy_version 155675 (0.0010) [2023-12-26 16:30:03,713][105692] Updated weights for policy 0, policy_version 155685 (0.0008) [2023-12-26 16:30:03,771][105692] Updated weights for policy 0, policy_version 155696 (0.0010) [2023-12-26 16:30:03,792][105620] Updated weights for policy 1, policy_version 156428 (0.0005) [2023-12-26 16:30:03,860][105620] Updated weights for policy 1, policy_version 156438 (0.0007) [2023-12-26 16:30:03,916][105620] Updated weights for policy 1, policy_version 156448 (0.0006) [2023-12-26 16:30:04,540][105692] Updated weights for policy 0, policy_version 155706 (0.0008) [2023-12-26 16:30:04,590][105692] Updated weights for policy 0, policy_version 155716 (0.0008) [2023-12-26 16:30:04,638][105692] Updated weights for policy 0, policy_version 155726 (0.0008) [2023-12-26 16:30:04,644][105620] Updated weights for policy 1, policy_version 156458 (0.0010) [2023-12-26 16:30:04,694][105692] Updated weights for policy 0, policy_version 155736 (0.0006) [2023-12-26 16:30:04,699][105620] Updated weights for policy 1, policy_version 156468 (0.0010) [2023-12-26 16:30:04,750][105620] Updated weights for policy 1, policy_version 156478 (0.0010) [2023-12-26 16:30:04,816][105620] Updated weights for policy 1, policy_version 156488 (0.0010) [2023-12-26 16:30:05,481][105692] Updated weights for policy 0, policy_version 155746 (0.0006) [2023-12-26 16:30:05,530][105620] Updated weights for policy 1, policy_version 156498 (0.0005) [2023-12-26 16:30:05,548][105692] Updated weights for policy 0, policy_version 155756 (0.0005) [2023-12-26 16:30:05,592][105620] Updated weights for policy 1, policy_version 156508 (0.0005) [2023-12-26 16:30:05,613][105692] Updated weights for policy 0, policy_version 155766 (0.0006) [2023-12-26 16:30:05,638][105620] Updated weights for policy 1, policy_version 156518 (0.0005) [2023-12-26 16:30:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 79962112. Throughput: 0: 9765.1, 1: 9678.4. Samples: 79952404. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:30:06,063][104569] Avg episode reward: [(0, '8074.929'), (1, '9103.326')] [2023-12-26 16:30:06,151][105620] Updated weights for policy 1, policy_version 156528 (0.0007) [2023-12-26 16:30:06,207][105620] Updated weights for policy 1, policy_version 156538 (0.0008) [2023-12-26 16:30:06,270][105620] Updated weights for policy 1, policy_version 156548 (0.0009) [2023-12-26 16:30:06,276][105692] Updated weights for policy 0, policy_version 155776 (0.0007) [2023-12-26 16:30:06,333][105692] Updated weights for policy 0, policy_version 155786 (0.0008) [2023-12-26 16:30:06,396][105692] Updated weights for policy 0, policy_version 155796 (0.0009) [2023-12-26 16:30:06,981][105620] Updated weights for policy 1, policy_version 156558 (0.0007) [2023-12-26 16:30:07,037][105620] Updated weights for policy 1, policy_version 156568 (0.0011) [2023-12-26 16:30:07,086][105620] Updated weights for policy 1, policy_version 156578 (0.0009) [2023-12-26 16:30:07,142][105692] Updated weights for policy 0, policy_version 155806 (0.0009) [2023-12-26 16:30:07,200][105692] Updated weights for policy 0, policy_version 155816 (0.0008) [2023-12-26 16:30:07,255][105692] Updated weights for policy 0, policy_version 155826 (0.0009) [2023-12-26 16:30:07,806][105620] Updated weights for policy 1, policy_version 156588 (0.0011) [2023-12-26 16:30:07,865][105620] Updated weights for policy 1, policy_version 156598 (0.0010) [2023-12-26 16:30:07,907][105692] Updated weights for policy 0, policy_version 155836 (0.0007) [2023-12-26 16:30:07,920][105620] Updated weights for policy 1, policy_version 156608 (0.0010) [2023-12-26 16:30:07,954][105692] Updated weights for policy 0, policy_version 155846 (0.0005) [2023-12-26 16:30:08,004][105692] Updated weights for policy 0, policy_version 155856 (0.0005) [2023-12-26 16:30:08,586][105620] Updated weights for policy 1, policy_version 156618 (0.0010) [2023-12-26 16:30:08,587][105692] Updated weights for policy 0, policy_version 155866 (0.0007) [2023-12-26 16:30:08,639][105692] Updated weights for policy 0, policy_version 155876 (0.0006) [2023-12-26 16:30:08,648][105620] Updated weights for policy 1, policy_version 156628 (0.0010) [2023-12-26 16:30:08,693][105692] Updated weights for policy 0, policy_version 155886 (0.0008) [2023-12-26 16:30:08,705][105620] Updated weights for policy 1, policy_version 156638 (0.0008) [2023-12-26 16:30:08,756][105692] Updated weights for policy 0, policy_version 155896 (0.0007) [2023-12-26 16:30:08,767][105620] Updated weights for policy 1, policy_version 156648 (0.0006) [2023-12-26 16:30:09,385][105620] Updated weights for policy 1, policy_version 156658 (0.0008) [2023-12-26 16:30:09,458][105620] Updated weights for policy 1, policy_version 156668 (0.0009) [2023-12-26 16:30:09,522][105620] Updated weights for policy 1, policy_version 156678 (0.0011) [2023-12-26 16:30:09,542][105692] Updated weights for policy 0, policy_version 155906 (0.0011) [2023-12-26 16:30:09,605][105692] Updated weights for policy 0, policy_version 155916 (0.0011) [2023-12-26 16:30:09,671][105692] Updated weights for policy 0, policy_version 155926 (0.0011) [2023-12-26 16:30:10,239][105620] Updated weights for policy 1, policy_version 156688 (0.0007) [2023-12-26 16:30:10,302][105620] Updated weights for policy 1, policy_version 156698 (0.0006) [2023-12-26 16:30:10,360][105620] Updated weights for policy 1, policy_version 156708 (0.0010) [2023-12-26 16:30:10,396][105692] Updated weights for policy 0, policy_version 155936 (0.0008) [2023-12-26 16:30:10,460][105692] Updated weights for policy 0, policy_version 155946 (0.0008) [2023-12-26 16:30:10,512][105692] Updated weights for policy 0, policy_version 155956 (0.0008) [2023-12-26 16:30:11,044][105620] Updated weights for policy 1, policy_version 156718 (0.0010) [2023-12-26 16:30:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 80060416. Throughput: 0: 9761.5, 1: 9770.6. Samples: 80073128. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:30:11,063][104569] Avg episode reward: [(0, '8726.227'), (1, '9095.785')] [2023-12-26 16:30:11,108][105620] Updated weights for policy 1, policy_version 156728 (0.0007) [2023-12-26 16:30:11,176][105620] Updated weights for policy 1, policy_version 156738 (0.0008) [2023-12-26 16:30:11,314][105692] Updated weights for policy 0, policy_version 155966 (0.0009) [2023-12-26 16:30:11,383][105692] Updated weights for policy 0, policy_version 155976 (0.0009) [2023-12-26 16:30:11,448][105692] Updated weights for policy 0, policy_version 155986 (0.0009) [2023-12-26 16:30:11,956][105620] Updated weights for policy 1, policy_version 156748 (0.0009) [2023-12-26 16:30:12,024][105620] Updated weights for policy 1, policy_version 156758 (0.0006) [2023-12-26 16:30:12,091][105620] Updated weights for policy 1, policy_version 156768 (0.0008) [2023-12-26 16:30:12,294][105692] Updated weights for policy 0, policy_version 155996 (0.0009) [2023-12-26 16:30:12,345][105692] Updated weights for policy 0, policy_version 156006 (0.0009) [2023-12-26 16:30:12,410][105692] Updated weights for policy 0, policy_version 156016 (0.0009) [2023-12-26 16:30:12,758][105620] Updated weights for policy 1, policy_version 156778 (0.0007) [2023-12-26 16:30:12,823][105620] Updated weights for policy 1, policy_version 156788 (0.0005) [2023-12-26 16:30:12,892][105620] Updated weights for policy 1, policy_version 156798 (0.0007) [2023-12-26 16:30:12,947][105620] Updated weights for policy 1, policy_version 156808 (0.0006) [2023-12-26 16:30:13,166][105692] Updated weights for policy 0, policy_version 156026 (0.0010) [2023-12-26 16:30:13,218][105692] Updated weights for policy 0, policy_version 156036 (0.0007) [2023-12-26 16:30:13,277][105692] Updated weights for policy 0, policy_version 156046 (0.0006) [2023-12-26 16:30:13,321][105692] Updated weights for policy 0, policy_version 156056 (0.0005) [2023-12-26 16:30:13,605][105620] Updated weights for policy 1, policy_version 156818 (0.0007) [2023-12-26 16:30:13,652][105620] Updated weights for policy 1, policy_version 156828 (0.0010) [2023-12-26 16:30:13,709][105620] Updated weights for policy 1, policy_version 156838 (0.0010) [2023-12-26 16:30:13,939][105692] Updated weights for policy 0, policy_version 156066 (0.0005) [2023-12-26 16:30:13,999][105692] Updated weights for policy 0, policy_version 156076 (0.0006) [2023-12-26 16:30:14,047][105692] Updated weights for policy 0, policy_version 156086 (0.0008) [2023-12-26 16:30:14,370][105620] Updated weights for policy 1, policy_version 156848 (0.0006) [2023-12-26 16:30:14,436][105620] Updated weights for policy 1, policy_version 156858 (0.0008) [2023-12-26 16:30:14,497][105620] Updated weights for policy 1, policy_version 156868 (0.0007) [2023-12-26 16:30:14,769][105692] Updated weights for policy 0, policy_version 156096 (0.0008) [2023-12-26 16:30:14,834][105692] Updated weights for policy 0, policy_version 156106 (0.0008) [2023-12-26 16:30:14,900][105692] Updated weights for policy 0, policy_version 156116 (0.0008) [2023-12-26 16:30:15,207][105620] Updated weights for policy 1, policy_version 156878 (0.0009) [2023-12-26 16:30:15,264][105620] Updated weights for policy 1, policy_version 156888 (0.0008) [2023-12-26 16:30:15,317][105620] Updated weights for policy 1, policy_version 156898 (0.0008) [2023-12-26 16:30:15,585][105692] Updated weights for policy 0, policy_version 156126 (0.0009) [2023-12-26 16:30:15,634][105692] Updated weights for policy 0, policy_version 156136 (0.0010) [2023-12-26 16:30:15,681][105692] Updated weights for policy 0, policy_version 156146 (0.0010) [2023-12-26 16:30:16,046][105620] Updated weights for policy 1, policy_version 156908 (0.0007) [2023-12-26 16:30:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 80158720. Throughput: 0: 9696.4, 1: 9738.7. Samples: 80129552. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:30:16,062][104569] Avg episode reward: [(0, '8812.789'), (1, '8904.140')] [2023-12-26 16:30:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000156152_39985152.pth... [2023-12-26 16:30:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000155032_39698432.pth [2023-12-26 16:30:16,101][105620] Updated weights for policy 1, policy_version 156918 (0.0009) [2023-12-26 16:30:16,152][105620] Updated weights for policy 1, policy_version 156928 (0.0008) [2023-12-26 16:30:16,192][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000156936_40181760.pth... [2023-12-26 16:30:16,195][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000155752_39878656.pth [2023-12-26 16:30:16,444][105692] Updated weights for policy 0, policy_version 156156 (0.0008) [2023-12-26 16:30:16,502][105692] Updated weights for policy 0, policy_version 156166 (0.0008) [2023-12-26 16:30:16,549][105692] Updated weights for policy 0, policy_version 156176 (0.0005) [2023-12-26 16:30:16,997][105620] Updated weights for policy 1, policy_version 156938 (0.0009) [2023-12-26 16:30:17,060][105620] Updated weights for policy 1, policy_version 156948 (0.0009) [2023-12-26 16:30:17,120][105692] Updated weights for policy 0, policy_version 156186 (0.0006) [2023-12-26 16:30:17,122][105620] Updated weights for policy 1, policy_version 156958 (0.0008) [2023-12-26 16:30:17,174][105692] Updated weights for policy 0, policy_version 156196 (0.0008) [2023-12-26 16:30:17,189][105620] Updated weights for policy 1, policy_version 156968 (0.0006) [2023-12-26 16:30:17,228][105692] Updated weights for policy 0, policy_version 156206 (0.0006) [2023-12-26 16:30:17,280][105692] Updated weights for policy 0, policy_version 156216 (0.0008) [2023-12-26 16:30:17,909][105692] Updated weights for policy 0, policy_version 156226 (0.0005) [2023-12-26 16:30:17,958][105692] Updated weights for policy 0, policy_version 156236 (0.0005) [2023-12-26 16:30:17,977][105620] Updated weights for policy 1, policy_version 156978 (0.0005) [2023-12-26 16:30:18,013][105692] Updated weights for policy 0, policy_version 156246 (0.0005) [2023-12-26 16:30:18,031][105620] Updated weights for policy 1, policy_version 156988 (0.0007) [2023-12-26 16:30:18,091][105620] Updated weights for policy 1, policy_version 156998 (0.0005) [2023-12-26 16:30:18,579][105692] Updated weights for policy 0, policy_version 156256 (0.0007) [2023-12-26 16:30:18,649][105692] Updated weights for policy 0, policy_version 156266 (0.0009) [2023-12-26 16:30:18,720][105692] Updated weights for policy 0, policy_version 156276 (0.0009) [2023-12-26 16:30:18,823][105620] Updated weights for policy 1, policy_version 157008 (0.0005) [2023-12-26 16:30:18,880][105620] Updated weights for policy 1, policy_version 157018 (0.0005) [2023-12-26 16:30:18,944][105620] Updated weights for policy 1, policy_version 157028 (0.0008) [2023-12-26 16:30:19,477][105692] Updated weights for policy 0, policy_version 156286 (0.0007) [2023-12-26 16:30:19,540][105692] Updated weights for policy 0, policy_version 156296 (0.0008) [2023-12-26 16:30:19,610][105692] Updated weights for policy 0, policy_version 156306 (0.0008) [2023-12-26 16:30:19,648][105620] Updated weights for policy 1, policy_version 157038 (0.0010) [2023-12-26 16:30:19,705][105620] Updated weights for policy 1, policy_version 157048 (0.0010) [2023-12-26 16:30:19,754][105620] Updated weights for policy 1, policy_version 157058 (0.0011) [2023-12-26 16:30:20,392][105692] Updated weights for policy 0, policy_version 156316 (0.0007) [2023-12-26 16:30:20,454][105692] Updated weights for policy 0, policy_version 156326 (0.0008) [2023-12-26 16:30:20,515][105620] Updated weights for policy 1, policy_version 157068 (0.0011) [2023-12-26 16:30:20,517][105692] Updated weights for policy 0, policy_version 156336 (0.0008) [2023-12-26 16:30:20,577][105620] Updated weights for policy 1, policy_version 157078 (0.0011) [2023-12-26 16:30:20,648][105620] Updated weights for policy 1, policy_version 157088 (0.0011) [2023-12-26 16:30:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 80257024. Throughput: 0: 9812.9, 1: 9634.5. Samples: 80248008. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:30:21,063][104569] Avg episode reward: [(0, '8376.938'), (1, '9080.773')] [2023-12-26 16:30:21,175][105692] Updated weights for policy 0, policy_version 156346 (0.0007) [2023-12-26 16:30:21,228][105692] Updated weights for policy 0, policy_version 156356 (0.0008) [2023-12-26 16:30:21,284][105692] Updated weights for policy 0, policy_version 156366 (0.0008) [2023-12-26 16:30:21,356][105692] Updated weights for policy 0, policy_version 156376 (0.0007) [2023-12-26 16:30:21,406][105620] Updated weights for policy 1, policy_version 157098 (0.0010) [2023-12-26 16:30:21,469][105620] Updated weights for policy 1, policy_version 157108 (0.0011) [2023-12-26 16:30:21,525][105620] Updated weights for policy 1, policy_version 157118 (0.0011) [2023-12-26 16:30:21,594][105620] Updated weights for policy 1, policy_version 157128 (0.0010) [2023-12-26 16:30:22,058][105692] Updated weights for policy 0, policy_version 156386 (0.0008) [2023-12-26 16:30:22,115][105692] Updated weights for policy 0, policy_version 156396 (0.0009) [2023-12-26 16:30:22,175][105692] Updated weights for policy 0, policy_version 156406 (0.0008) [2023-12-26 16:30:22,393][105620] Updated weights for policy 1, policy_version 157138 (0.0009) [2023-12-26 16:30:22,454][105620] Updated weights for policy 1, policy_version 157148 (0.0010) [2023-12-26 16:30:22,517][105620] Updated weights for policy 1, policy_version 157158 (0.0011) [2023-12-26 16:30:22,976][105692] Updated weights for policy 0, policy_version 156416 (0.0008) [2023-12-26 16:30:23,025][105692] Updated weights for policy 0, policy_version 156426 (0.0008) [2023-12-26 16:30:23,071][105692] Updated weights for policy 0, policy_version 156436 (0.0005) [2023-12-26 16:30:23,229][105620] Updated weights for policy 1, policy_version 157168 (0.0010) [2023-12-26 16:30:23,293][105620] Updated weights for policy 1, policy_version 157178 (0.0008) [2023-12-26 16:30:23,362][105620] Updated weights for policy 1, policy_version 157188 (0.0010) [2023-12-26 16:30:23,807][105692] Updated weights for policy 0, policy_version 156446 (0.0010) [2023-12-26 16:30:23,857][105692] Updated weights for policy 0, policy_version 156456 (0.0010) [2023-12-26 16:30:23,921][105692] Updated weights for policy 0, policy_version 156466 (0.0010) [2023-12-26 16:30:24,072][105620] Updated weights for policy 1, policy_version 157198 (0.0010) [2023-12-26 16:30:24,130][105620] Updated weights for policy 1, policy_version 157208 (0.0010) [2023-12-26 16:30:24,190][105620] Updated weights for policy 1, policy_version 157218 (0.0010) [2023-12-26 16:30:24,588][105692] Updated weights for policy 0, policy_version 156476 (0.0009) [2023-12-26 16:30:24,638][105692] Updated weights for policy 0, policy_version 156486 (0.0007) [2023-12-26 16:30:24,687][105692] Updated weights for policy 0, policy_version 156496 (0.0005) [2023-12-26 16:30:24,854][105620] Updated weights for policy 1, policy_version 157228 (0.0010) [2023-12-26 16:30:24,902][105620] Updated weights for policy 1, policy_version 157238 (0.0010) [2023-12-26 16:30:24,950][105620] Updated weights for policy 1, policy_version 157248 (0.0010) [2023-12-26 16:30:25,352][105692] Updated weights for policy 0, policy_version 156506 (0.0006) [2023-12-26 16:30:25,407][105692] Updated weights for policy 0, policy_version 156516 (0.0010) [2023-12-26 16:30:25,461][105692] Updated weights for policy 0, policy_version 156526 (0.0009) [2023-12-26 16:30:25,514][105692] Updated weights for policy 0, policy_version 156536 (0.0005) [2023-12-26 16:30:25,542][105620] Updated weights for policy 1, policy_version 157258 (0.0009) [2023-12-26 16:30:25,601][105620] Updated weights for policy 1, policy_version 157268 (0.0005) [2023-12-26 16:30:25,652][105620] Updated weights for policy 1, policy_version 157278 (0.0005) [2023-12-26 16:30:25,704][105620] Updated weights for policy 1, policy_version 157288 (0.0005) [2023-12-26 16:30:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 80355328. Throughput: 0: 9794.5, 1: 9689.2. Samples: 80366072. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:30:26,063][104569] Avg episode reward: [(0, '8010.422'), (1, '8897.957')] [2023-12-26 16:30:26,161][105692] Updated weights for policy 0, policy_version 156546 (0.0011) [2023-12-26 16:30:26,217][105692] Updated weights for policy 0, policy_version 156556 (0.0010) [2023-12-26 16:30:26,226][105620] Updated weights for policy 1, policy_version 157298 (0.0007) [2023-12-26 16:30:26,272][105692] Updated weights for policy 0, policy_version 156566 (0.0010) [2023-12-26 16:30:26,284][105620] Updated weights for policy 1, policy_version 157308 (0.0010) [2023-12-26 16:30:26,345][105620] Updated weights for policy 1, policy_version 157318 (0.0010) [2023-12-26 16:30:26,930][105620] Updated weights for policy 1, policy_version 157328 (0.0007) [2023-12-26 16:30:26,982][105620] Updated weights for policy 1, policy_version 157338 (0.0009) [2023-12-26 16:30:27,028][105692] Updated weights for policy 0, policy_version 156576 (0.0010) [2023-12-26 16:30:27,034][105620] Updated weights for policy 1, policy_version 157348 (0.0010) [2023-12-26 16:30:27,075][105692] Updated weights for policy 0, policy_version 156586 (0.0010) [2023-12-26 16:30:27,119][105692] Updated weights for policy 0, policy_version 156596 (0.0010) [2023-12-26 16:30:27,703][105620] Updated weights for policy 1, policy_version 157358 (0.0007) [2023-12-26 16:30:27,757][105620] Updated weights for policy 1, policy_version 157368 (0.0005) [2023-12-26 16:30:27,814][105620] Updated weights for policy 1, policy_version 157378 (0.0010) [2023-12-26 16:30:27,863][105692] Updated weights for policy 0, policy_version 156606 (0.0007) [2023-12-26 16:30:27,915][105692] Updated weights for policy 0, policy_version 156616 (0.0005) [2023-12-26 16:30:27,972][105692] Updated weights for policy 0, policy_version 156626 (0.0005) [2023-12-26 16:30:28,463][105620] Updated weights for policy 1, policy_version 157388 (0.0006) [2023-12-26 16:30:28,518][105620] Updated weights for policy 1, policy_version 157398 (0.0006) [2023-12-26 16:30:28,571][105620] Updated weights for policy 1, policy_version 157408 (0.0005) [2023-12-26 16:30:28,622][105692] Updated weights for policy 0, policy_version 156636 (0.0008) [2023-12-26 16:30:28,683][105692] Updated weights for policy 0, policy_version 156646 (0.0010) [2023-12-26 16:30:28,737][105692] Updated weights for policy 0, policy_version 156656 (0.0010) [2023-12-26 16:30:29,146][105620] Updated weights for policy 1, policy_version 157418 (0.0007) [2023-12-26 16:30:29,198][105620] Updated weights for policy 1, policy_version 157428 (0.0007) [2023-12-26 16:30:29,256][105620] Updated weights for policy 1, policy_version 157438 (0.0006) [2023-12-26 16:30:29,310][105620] Updated weights for policy 1, policy_version 157448 (0.0006) [2023-12-26 16:30:29,497][105692] Updated weights for policy 0, policy_version 156666 (0.0009) [2023-12-26 16:30:29,551][105692] Updated weights for policy 0, policy_version 156676 (0.0007) [2023-12-26 16:30:29,602][105692] Updated weights for policy 0, policy_version 156686 (0.0010) [2023-12-26 16:30:29,664][105692] Updated weights for policy 0, policy_version 156696 (0.0010) [2023-12-26 16:30:30,022][105620] Updated weights for policy 1, policy_version 157458 (0.0010) [2023-12-26 16:30:30,077][105620] Updated weights for policy 1, policy_version 157468 (0.0009) [2023-12-26 16:30:30,132][105620] Updated weights for policy 1, policy_version 157478 (0.0008) [2023-12-26 16:30:30,397][105692] Updated weights for policy 0, policy_version 156706 (0.0009) [2023-12-26 16:30:30,456][105692] Updated weights for policy 0, policy_version 156716 (0.0010) [2023-12-26 16:30:30,514][105692] Updated weights for policy 0, policy_version 156726 (0.0010) [2023-12-26 16:30:30,855][105620] Updated weights for policy 1, policy_version 157488 (0.0009) [2023-12-26 16:30:30,907][105620] Updated weights for policy 1, policy_version 157498 (0.0008) [2023-12-26 16:30:30,955][105620] Updated weights for policy 1, policy_version 157508 (0.0009) [2023-12-26 16:30:31,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 80461824. Throughput: 0: 9834.0, 1: 9843.6. Samples: 80429800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:30:31,063][104569] Avg episode reward: [(0, '8264.572'), (1, '8897.159')] [2023-12-26 16:30:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000156728_40132608.pth... [2023-12-26 16:30:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000157512_40329216.pth... [2023-12-26 16:30:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000156328_40026112.pth [2023-12-26 16:30:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000155576_39837696.pth [2023-12-26 16:30:31,225][105692] Updated weights for policy 0, policy_version 156736 (0.0008) [2023-12-26 16:30:31,294][105692] Updated weights for policy 0, policy_version 156746 (0.0008) [2023-12-26 16:30:31,362][105692] Updated weights for policy 0, policy_version 156756 (0.0009) [2023-12-26 16:30:31,760][105620] Updated weights for policy 1, policy_version 157518 (0.0009) [2023-12-26 16:30:31,817][105620] Updated weights for policy 1, policy_version 157528 (0.0009) [2023-12-26 16:30:31,873][105620] Updated weights for policy 1, policy_version 157538 (0.0008) [2023-12-26 16:30:32,008][105692] Updated weights for policy 0, policy_version 156766 (0.0007) [2023-12-26 16:30:32,068][105692] Updated weights for policy 0, policy_version 156776 (0.0005) [2023-12-26 16:30:32,127][105692] Updated weights for policy 0, policy_version 156786 (0.0005) [2023-12-26 16:30:32,607][105620] Updated weights for policy 1, policy_version 157548 (0.0008) [2023-12-26 16:30:32,665][105620] Updated weights for policy 1, policy_version 157558 (0.0008) [2023-12-26 16:30:32,727][105620] Updated weights for policy 1, policy_version 157568 (0.0009) [2023-12-26 16:30:32,784][105692] Updated weights for policy 0, policy_version 156796 (0.0006) [2023-12-26 16:30:32,839][105692] Updated weights for policy 0, policy_version 156806 (0.0008) [2023-12-26 16:30:32,895][105692] Updated weights for policy 0, policy_version 156816 (0.0005) [2023-12-26 16:30:33,496][105692] Updated weights for policy 0, policy_version 156826 (0.0005) [2023-12-26 16:30:33,536][105620] Updated weights for policy 1, policy_version 157578 (0.0008) [2023-12-26 16:30:33,547][105692] Updated weights for policy 0, policy_version 156836 (0.0005) [2023-12-26 16:30:33,586][105620] Updated weights for policy 1, policy_version 157588 (0.0007) [2023-12-26 16:30:33,608][105692] Updated weights for policy 0, policy_version 156846 (0.0008) [2023-12-26 16:30:33,634][105620] Updated weights for policy 1, policy_version 157598 (0.0006) [2023-12-26 16:30:33,664][105692] Updated weights for policy 0, policy_version 156856 (0.0007) [2023-12-26 16:30:33,681][105620] Updated weights for policy 1, policy_version 157608 (0.0007) [2023-12-26 16:30:34,238][105692] Updated weights for policy 0, policy_version 156866 (0.0011) [2023-12-26 16:30:34,293][105692] Updated weights for policy 0, policy_version 156876 (0.0010) [2023-12-26 16:30:34,355][105692] Updated weights for policy 0, policy_version 156886 (0.0010) [2023-12-26 16:30:34,443][105620] Updated weights for policy 1, policy_version 157618 (0.0009) [2023-12-26 16:30:34,497][105620] Updated weights for policy 1, policy_version 157628 (0.0008) [2023-12-26 16:30:34,553][105620] Updated weights for policy 1, policy_version 157638 (0.0008) [2023-12-26 16:30:35,117][105692] Updated weights for policy 0, policy_version 156896 (0.0011) [2023-12-26 16:30:35,177][105692] Updated weights for policy 0, policy_version 156906 (0.0010) [2023-12-26 16:30:35,233][105692] Updated weights for policy 0, policy_version 156916 (0.0011) [2023-12-26 16:30:35,347][105620] Updated weights for policy 1, policy_version 157648 (0.0008) [2023-12-26 16:30:35,422][105620] Updated weights for policy 1, policy_version 157658 (0.0010) [2023-12-26 16:30:35,483][105620] Updated weights for policy 1, policy_version 157668 (0.0009) [2023-12-26 16:30:35,931][105692] Updated weights for policy 0, policy_version 156926 (0.0008) [2023-12-26 16:30:35,986][105692] Updated weights for policy 0, policy_version 156936 (0.0007) [2023-12-26 16:30:36,037][105692] Updated weights for policy 0, policy_version 156946 (0.0010) [2023-12-26 16:30:36,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 80551936. Throughput: 0: 9828.2, 1: 9866.6. Samples: 80547084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:30:36,062][104569] Avg episode reward: [(0, '8383.701'), (1, '9078.242')] [2023-12-26 16:30:36,270][105620] Updated weights for policy 1, policy_version 157678 (0.0008) [2023-12-26 16:30:36,335][105620] Updated weights for policy 1, policy_version 157688 (0.0009) [2023-12-26 16:30:36,400][105620] Updated weights for policy 1, policy_version 157698 (0.0008) [2023-12-26 16:30:36,785][105692] Updated weights for policy 0, policy_version 156956 (0.0009) [2023-12-26 16:30:36,844][105692] Updated weights for policy 0, policy_version 156966 (0.0009) [2023-12-26 16:30:36,907][105692] Updated weights for policy 0, policy_version 156976 (0.0006) [2023-12-26 16:30:37,184][105620] Updated weights for policy 1, policy_version 157708 (0.0009) [2023-12-26 16:30:37,243][105620] Updated weights for policy 1, policy_version 157718 (0.0010) [2023-12-26 16:30:37,297][105620] Updated weights for policy 1, policy_version 157728 (0.0008) [2023-12-26 16:30:37,572][105692] Updated weights for policy 0, policy_version 156986 (0.0006) [2023-12-26 16:30:37,626][105692] Updated weights for policy 0, policy_version 156996 (0.0009) [2023-12-26 16:30:37,688][105692] Updated weights for policy 0, policy_version 157006 (0.0009) [2023-12-26 16:30:37,741][105692] Updated weights for policy 0, policy_version 157016 (0.0009) [2023-12-26 16:30:38,108][105620] Updated weights for policy 1, policy_version 157738 (0.0009) [2023-12-26 16:30:38,165][105620] Updated weights for policy 1, policy_version 157748 (0.0009) [2023-12-26 16:30:38,225][105620] Updated weights for policy 1, policy_version 157758 (0.0009) [2023-12-26 16:30:38,281][105620] Updated weights for policy 1, policy_version 157768 (0.0008) [2023-12-26 16:30:38,471][105692] Updated weights for policy 0, policy_version 157026 (0.0008) [2023-12-26 16:30:38,529][105692] Updated weights for policy 0, policy_version 157036 (0.0010) [2023-12-26 16:30:38,593][105692] Updated weights for policy 0, policy_version 157046 (0.0009) [2023-12-26 16:30:39,107][105620] Updated weights for policy 1, policy_version 157778 (0.0009) [2023-12-26 16:30:39,163][105620] Updated weights for policy 1, policy_version 157788 (0.0009) [2023-12-26 16:30:39,224][105620] Updated weights for policy 1, policy_version 157798 (0.0010) [2023-12-26 16:30:39,352][105692] Updated weights for policy 0, policy_version 157056 (0.0009) [2023-12-26 16:30:39,412][105692] Updated weights for policy 0, policy_version 157066 (0.0007) [2023-12-26 16:30:39,480][105692] Updated weights for policy 0, policy_version 157076 (0.0009) [2023-12-26 16:30:40,083][105620] Updated weights for policy 1, policy_version 157808 (0.0009) [2023-12-26 16:30:40,146][105620] Updated weights for policy 1, policy_version 157818 (0.0010) [2023-12-26 16:30:40,204][105620] Updated weights for policy 1, policy_version 157828 (0.0008) [2023-12-26 16:30:40,254][105692] Updated weights for policy 0, policy_version 157086 (0.0008) [2023-12-26 16:30:40,310][105692] Updated weights for policy 0, policy_version 157096 (0.0009) [2023-12-26 16:30:40,378][105692] Updated weights for policy 0, policy_version 157106 (0.0009) [2023-12-26 16:30:40,976][105620] Updated weights for policy 1, policy_version 157838 (0.0009) [2023-12-26 16:30:41,030][105620] Updated weights for policy 1, policy_version 157848 (0.0009) [2023-12-26 16:30:41,062][104569] Fps is (10 sec: 18022.8, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 80642048. Throughput: 0: 9818.3, 1: 9765.5. Samples: 80656860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:30:41,062][104569] Avg episode reward: [(0, '8736.538'), (1, '9254.882')] [2023-12-26 16:30:41,098][105620] Updated weights for policy 1, policy_version 157858 (0.0009) [2023-12-26 16:30:41,186][105692] Updated weights for policy 0, policy_version 157116 (0.0010) [2023-12-26 16:30:41,237][105692] Updated weights for policy 0, policy_version 157126 (0.0008) [2023-12-26 16:30:41,303][105692] Updated weights for policy 0, policy_version 157136 (0.0009) [2023-12-26 16:30:41,917][105620] Updated weights for policy 1, policy_version 157868 (0.0008) [2023-12-26 16:30:41,980][105620] Updated weights for policy 1, policy_version 157878 (0.0009) [2023-12-26 16:30:42,045][105620] Updated weights for policy 1, policy_version 157888 (0.0009) [2023-12-26 16:30:42,114][105692] Updated weights for policy 0, policy_version 157146 (0.0008) [2023-12-26 16:30:42,181][105692] Updated weights for policy 0, policy_version 157156 (0.0007) [2023-12-26 16:30:42,253][105692] Updated weights for policy 0, policy_version 157166 (0.0007) [2023-12-26 16:30:42,328][105692] Updated weights for policy 0, policy_version 157176 (0.0009) [2023-12-26 16:30:42,775][105620] Updated weights for policy 1, policy_version 157898 (0.0008) [2023-12-26 16:30:42,838][105620] Updated weights for policy 1, policy_version 157908 (0.0009) [2023-12-26 16:30:42,897][105620] Updated weights for policy 1, policy_version 157918 (0.0008) [2023-12-26 16:30:42,958][105620] Updated weights for policy 1, policy_version 157928 (0.0007) [2023-12-26 16:30:43,083][105692] Updated weights for policy 0, policy_version 157186 (0.0009) [2023-12-26 16:30:43,138][105692] Updated weights for policy 0, policy_version 157196 (0.0010) [2023-12-26 16:30:43,192][105692] Updated weights for policy 0, policy_version 157206 (0.0010) [2023-12-26 16:30:43,520][105620] Updated weights for policy 1, policy_version 157938 (0.0008) [2023-12-26 16:30:43,579][105620] Updated weights for policy 1, policy_version 157948 (0.0010) [2023-12-26 16:30:43,638][105620] Updated weights for policy 1, policy_version 157958 (0.0011) [2023-12-26 16:30:44,020][105692] Updated weights for policy 0, policy_version 157216 (0.0007) [2023-12-26 16:30:44,088][105692] Updated weights for policy 0, policy_version 157226 (0.0010) [2023-12-26 16:30:44,154][105692] Updated weights for policy 0, policy_version 157236 (0.0011) [2023-12-26 16:30:44,323][105620] Updated weights for policy 1, policy_version 157968 (0.0010) [2023-12-26 16:30:44,388][105620] Updated weights for policy 1, policy_version 157978 (0.0010) [2023-12-26 16:30:44,447][105620] Updated weights for policy 1, policy_version 157988 (0.0010) [2023-12-26 16:30:44,863][105692] Updated weights for policy 0, policy_version 157246 (0.0008) [2023-12-26 16:30:44,929][105692] Updated weights for policy 0, policy_version 157256 (0.0011) [2023-12-26 16:30:44,992][105692] Updated weights for policy 0, policy_version 157266 (0.0011) [2023-12-26 16:30:45,122][105620] Updated weights for policy 1, policy_version 157998 (0.0011) [2023-12-26 16:30:45,185][105620] Updated weights for policy 1, policy_version 158008 (0.0011) [2023-12-26 16:30:45,249][105620] Updated weights for policy 1, policy_version 158018 (0.0011) [2023-12-26 16:30:45,716][105692] Updated weights for policy 0, policy_version 157276 (0.0010) [2023-12-26 16:30:45,777][105692] Updated weights for policy 0, policy_version 157286 (0.0011) [2023-12-26 16:30:45,834][105692] Updated weights for policy 0, policy_version 157296 (0.0011) [2023-12-26 16:30:45,996][105620] Updated weights for policy 1, policy_version 158028 (0.0008) [2023-12-26 16:30:46,056][105620] Updated weights for policy 1, policy_version 158038 (0.0005) [2023-12-26 16:30:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 80740352. Throughput: 0: 9720.1, 1: 9756.5. Samples: 80711888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:30:46,062][104569] Avg episode reward: [(0, '9093.716'), (1, '9308.187')] [2023-12-26 16:30:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000157304_40280064.pth... [2023-12-26 16:30:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000156152_39985152.pth [2023-12-26 16:30:46,106][105620] Updated weights for policy 1, policy_version 158048 (0.0006) [2023-12-26 16:30:46,140][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000158056_40468480.pth... [2023-12-26 16:30:46,143][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000156936_40181760.pth [2023-12-26 16:30:46,592][105692] Updated weights for policy 0, policy_version 157306 (0.0011) [2023-12-26 16:30:46,640][105692] Updated weights for policy 0, policy_version 157316 (0.0010) [2023-12-26 16:30:46,686][105692] Updated weights for policy 0, policy_version 157326 (0.0009) [2023-12-26 16:30:46,740][105692] Updated weights for policy 0, policy_version 157336 (0.0007) [2023-12-26 16:30:46,825][105620] Updated weights for policy 1, policy_version 158058 (0.0008) [2023-12-26 16:30:46,880][105620] Updated weights for policy 1, policy_version 158068 (0.0009) [2023-12-26 16:30:46,929][105620] Updated weights for policy 1, policy_version 158079 (0.0010) [2023-12-26 16:30:47,416][105692] Updated weights for policy 0, policy_version 157347 (0.0011) [2023-12-26 16:30:47,470][105692] Updated weights for policy 0, policy_version 157358 (0.0010) [2023-12-26 16:30:47,607][105620] Updated weights for policy 1, policy_version 158089 (0.0009) [2023-12-26 16:30:47,665][105620] Updated weights for policy 1, policy_version 158099 (0.0009) [2023-12-26 16:30:47,730][105620] Updated weights for policy 1, policy_version 158109 (0.0009) [2023-12-26 16:30:47,792][105620] Updated weights for policy 1, policy_version 158119 (0.0010) [2023-12-26 16:30:48,300][105692] Updated weights for policy 0, policy_version 157369 (0.0010) [2023-12-26 16:30:48,367][105692] Updated weights for policy 0, policy_version 157379 (0.0009) [2023-12-26 16:30:48,431][105692] Updated weights for policy 0, policy_version 157389 (0.0007) [2023-12-26 16:30:48,470][105620] Updated weights for policy 1, policy_version 158129 (0.0011) [2023-12-26 16:30:48,492][105692] Updated weights for policy 0, policy_version 157399 (0.0006) [2023-12-26 16:30:48,527][105620] Updated weights for policy 1, policy_version 158139 (0.0010) [2023-12-26 16:30:48,595][105620] Updated weights for policy 1, policy_version 158149 (0.0011) [2023-12-26 16:30:49,224][105692] Updated weights for policy 0, policy_version 157409 (0.0009) [2023-12-26 16:30:49,281][105692] Updated weights for policy 0, policy_version 157419 (0.0008) [2023-12-26 16:30:49,292][105620] Updated weights for policy 1, policy_version 158159 (0.0009) [2023-12-26 16:30:49,333][105692] Updated weights for policy 0, policy_version 157429 (0.0008) [2023-12-26 16:30:49,355][105620] Updated weights for policy 1, policy_version 158169 (0.0007) [2023-12-26 16:30:49,416][105620] Updated weights for policy 1, policy_version 158179 (0.0009) [2023-12-26 16:30:50,123][105620] Updated weights for policy 1, policy_version 158189 (0.0005) [2023-12-26 16:30:50,168][105692] Updated weights for policy 0, policy_version 157439 (0.0010) [2023-12-26 16:30:50,174][105620] Updated weights for policy 1, policy_version 158199 (0.0005) [2023-12-26 16:30:50,221][105620] Updated weights for policy 1, policy_version 158209 (0.0007) [2023-12-26 16:30:50,227][105692] Updated weights for policy 0, policy_version 157449 (0.0009) [2023-12-26 16:30:50,280][105692] Updated weights for policy 0, policy_version 157459 (0.0009) [2023-12-26 16:30:50,928][105620] Updated weights for policy 1, policy_version 158219 (0.0009) [2023-12-26 16:30:50,997][105620] Updated weights for policy 1, policy_version 158229 (0.0009) [2023-12-26 16:30:51,030][105692] Updated weights for policy 0, policy_version 157469 (0.0008) [2023-12-26 16:30:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 80830464. Throughput: 0: 9722.4, 1: 9728.3. Samples: 80827684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:30:51,062][104569] Avg episode reward: [(0, '9268.046'), (1, '9095.262')] [2023-12-26 16:30:51,066][105620] Updated weights for policy 1, policy_version 158239 (0.0008) [2023-12-26 16:30:51,093][105692] Updated weights for policy 0, policy_version 157479 (0.0010) [2023-12-26 16:30:51,161][105692] Updated weights for policy 0, policy_version 157489 (0.0009) [2023-12-26 16:30:51,740][105620] Updated weights for policy 1, policy_version 158249 (0.0007) [2023-12-26 16:30:51,798][105620] Updated weights for policy 1, policy_version 158259 (0.0006) [2023-12-26 16:30:51,855][105692] Updated weights for policy 0, policy_version 157499 (0.0010) [2023-12-26 16:30:51,859][105620] Updated weights for policy 1, policy_version 158269 (0.0006) [2023-12-26 16:30:51,910][105692] Updated weights for policy 0, policy_version 157509 (0.0008) [2023-12-26 16:30:51,927][105620] Updated weights for policy 1, policy_version 158279 (0.0006) [2023-12-26 16:30:51,974][105692] Updated weights for policy 0, policy_version 157519 (0.0009) [2023-12-26 16:30:52,565][105620] Updated weights for policy 1, policy_version 158289 (0.0009) [2023-12-26 16:30:52,628][105620] Updated weights for policy 1, policy_version 158299 (0.0009) [2023-12-26 16:30:52,690][105620] Updated weights for policy 1, policy_version 158309 (0.0009) [2023-12-26 16:30:52,730][105692] Updated weights for policy 0, policy_version 157529 (0.0010) [2023-12-26 16:30:52,798][105692] Updated weights for policy 0, policy_version 157539 (0.0009) [2023-12-26 16:30:52,855][105692] Updated weights for policy 0, policy_version 157549 (0.0009) [2023-12-26 16:30:52,918][105692] Updated weights for policy 0, policy_version 157559 (0.0009) [2023-12-26 16:30:53,411][105620] Updated weights for policy 1, policy_version 158319 (0.0009) [2023-12-26 16:30:53,462][105620] Updated weights for policy 1, policy_version 158329 (0.0009) [2023-12-26 16:30:53,520][105620] Updated weights for policy 1, policy_version 158339 (0.0009) [2023-12-26 16:30:53,662][105692] Updated weights for policy 0, policy_version 157569 (0.0009) [2023-12-26 16:30:53,721][105692] Updated weights for policy 0, policy_version 157579 (0.0009) [2023-12-26 16:30:53,781][105692] Updated weights for policy 0, policy_version 157589 (0.0008) [2023-12-26 16:30:54,217][105620] Updated weights for policy 1, policy_version 158349 (0.0009) [2023-12-26 16:30:54,277][105620] Updated weights for policy 1, policy_version 158359 (0.0010) [2023-12-26 16:30:54,339][105620] Updated weights for policy 1, policy_version 158369 (0.0009) [2023-12-26 16:30:54,579][105692] Updated weights for policy 0, policy_version 157599 (0.0007) [2023-12-26 16:30:54,655][105692] Updated weights for policy 0, policy_version 157609 (0.0007) [2023-12-26 16:30:54,708][105692] Updated weights for policy 0, policy_version 157620 (0.0010) [2023-12-26 16:30:54,944][105620] Updated weights for policy 1, policy_version 158379 (0.0007) [2023-12-26 16:30:54,995][105620] Updated weights for policy 1, policy_version 158389 (0.0006) [2023-12-26 16:30:55,057][105620] Updated weights for policy 1, policy_version 158399 (0.0005) [2023-12-26 16:30:55,428][105692] Updated weights for policy 0, policy_version 157630 (0.0007) [2023-12-26 16:30:55,482][105692] Updated weights for policy 0, policy_version 157640 (0.0005) [2023-12-26 16:30:55,536][105692] Updated weights for policy 0, policy_version 157650 (0.0005) [2023-12-26 16:30:55,808][105620] Updated weights for policy 1, policy_version 158409 (0.0008) [2023-12-26 16:30:55,864][105620] Updated weights for policy 1, policy_version 158419 (0.0005) [2023-12-26 16:30:55,925][105620] Updated weights for policy 1, policy_version 158429 (0.0005) [2023-12-26 16:30:55,987][105620] Updated weights for policy 1, policy_version 158439 (0.0008) [2023-12-26 16:30:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 80936960. Throughput: 0: 9669.7, 1: 9688.0. Samples: 80944224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:30:56,062][104569] Avg episode reward: [(0, '8467.730'), (1, '8535.155')] [2023-12-26 16:30:56,161][105692] Updated weights for policy 0, policy_version 157660 (0.0007) [2023-12-26 16:30:56,222][105692] Updated weights for policy 0, policy_version 157670 (0.0008) [2023-12-26 16:30:56,270][105692] Updated weights for policy 0, policy_version 157680 (0.0008) [2023-12-26 16:30:56,673][105620] Updated weights for policy 1, policy_version 158449 (0.0006) [2023-12-26 16:30:56,734][105620] Updated weights for policy 1, policy_version 158459 (0.0006) [2023-12-26 16:30:56,787][105620] Updated weights for policy 1, policy_version 158469 (0.0010) [2023-12-26 16:30:57,018][105692] Updated weights for policy 0, policy_version 157690 (0.0008) [2023-12-26 16:30:57,071][105692] Updated weights for policy 0, policy_version 157700 (0.0010) [2023-12-26 16:30:57,123][105692] Updated weights for policy 0, policy_version 157711 (0.0009) [2023-12-26 16:30:57,369][105620] Updated weights for policy 1, policy_version 158479 (0.0009) [2023-12-26 16:30:57,418][105620] Updated weights for policy 1, policy_version 158489 (0.0010) [2023-12-26 16:30:57,466][105620] Updated weights for policy 1, policy_version 158499 (0.0010) [2023-12-26 16:30:57,887][105692] Updated weights for policy 0, policy_version 157721 (0.0009) [2023-12-26 16:30:57,934][105692] Updated weights for policy 0, policy_version 157731 (0.0006) [2023-12-26 16:30:57,987][105692] Updated weights for policy 0, policy_version 157742 (0.0010) [2023-12-26 16:30:58,037][105692] Updated weights for policy 0, policy_version 157752 (0.0008) [2023-12-26 16:30:58,094][105620] Updated weights for policy 1, policy_version 158509 (0.0011) [2023-12-26 16:30:58,157][105620] Updated weights for policy 1, policy_version 158519 (0.0011) [2023-12-26 16:30:58,215][105620] Updated weights for policy 1, policy_version 158529 (0.0010) [2023-12-26 16:30:58,868][105692] Updated weights for policy 0, policy_version 157762 (0.0009) [2023-12-26 16:30:58,929][105692] Updated weights for policy 0, policy_version 157772 (0.0008) [2023-12-26 16:30:58,991][105692] Updated weights for policy 0, policy_version 157782 (0.0009) [2023-12-26 16:30:59,068][105620] Updated weights for policy 1, policy_version 158539 (0.0010) [2023-12-26 16:30:59,129][105620] Updated weights for policy 1, policy_version 158549 (0.0009) [2023-12-26 16:30:59,192][105620] Updated weights for policy 1, policy_version 158559 (0.0009) [2023-12-26 16:30:59,634][105692] Updated weights for policy 0, policy_version 157792 (0.0007) [2023-12-26 16:30:59,684][105692] Updated weights for policy 0, policy_version 157802 (0.0011) [2023-12-26 16:30:59,733][105692] Updated weights for policy 0, policy_version 157812 (0.0010) [2023-12-26 16:31:00,054][105620] Updated weights for policy 1, policy_version 158569 (0.0009) [2023-12-26 16:31:00,122][105620] Updated weights for policy 1, policy_version 158579 (0.0009) [2023-12-26 16:31:00,178][105620] Updated weights for policy 1, policy_version 158589 (0.0008) [2023-12-26 16:31:00,239][105620] Updated weights for policy 1, policy_version 158599 (0.0008) [2023-12-26 16:31:00,402][105692] Updated weights for policy 0, policy_version 157822 (0.0011) [2023-12-26 16:31:00,466][105692] Updated weights for policy 0, policy_version 157832 (0.0010) [2023-12-26 16:31:00,520][105692] Updated weights for policy 0, policy_version 157842 (0.0010) [2023-12-26 16:31:00,896][105620] Updated weights for policy 1, policy_version 158609 (0.0009) [2023-12-26 16:31:00,954][105620] Updated weights for policy 1, policy_version 158619 (0.0009) [2023-12-26 16:31:01,008][105620] Updated weights for policy 1, policy_version 158629 (0.0010) [2023-12-26 16:31:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 81035264. Throughput: 0: 9695.9, 1: 9729.3. Samples: 81003688. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:31:01,063][104569] Avg episode reward: [(0, '8560.254'), (1, '8806.741')] [2023-12-26 16:31:01,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000157848_40419328.pth... [2023-12-26 16:31:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000158632_40615936.pth... [2023-12-26 16:31:01,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000156728_40132608.pth [2023-12-26 16:31:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000157512_40329216.pth [2023-12-26 16:31:01,224][105692] Updated weights for policy 0, policy_version 157852 (0.0009) [2023-12-26 16:31:01,296][105692] Updated weights for policy 0, policy_version 157862 (0.0008) [2023-12-26 16:31:01,369][105692] Updated weights for policy 0, policy_version 157872 (0.0009) [2023-12-26 16:31:01,719][105620] Updated weights for policy 1, policy_version 158639 (0.0008) [2023-12-26 16:31:01,775][105620] Updated weights for policy 1, policy_version 158649 (0.0007) [2023-12-26 16:31:01,836][105620] Updated weights for policy 1, policy_version 158659 (0.0005) [2023-12-26 16:31:02,025][105692] Updated weights for policy 0, policy_version 157882 (0.0007) [2023-12-26 16:31:02,080][105692] Updated weights for policy 0, policy_version 157892 (0.0010) [2023-12-26 16:31:02,134][105692] Updated weights for policy 0, policy_version 157902 (0.0011) [2023-12-26 16:31:02,180][105585] KL-divergence is very high: 138.4302 [2023-12-26 16:31:02,191][105692] Updated weights for policy 0, policy_version 157912 (0.0008) [2023-12-26 16:31:02,483][105620] Updated weights for policy 1, policy_version 158669 (0.0006) [2023-12-26 16:31:02,549][105620] Updated weights for policy 1, policy_version 158679 (0.0005) [2023-12-26 16:31:02,609][105620] Updated weights for policy 1, policy_version 158689 (0.0005) [2023-12-26 16:31:02,923][105692] Updated weights for policy 0, policy_version 157922 (0.0009) [2023-12-26 16:31:02,980][105692] Updated weights for policy 0, policy_version 157932 (0.0008) [2023-12-26 16:31:03,040][105692] Updated weights for policy 0, policy_version 157942 (0.0010) [2023-12-26 16:31:03,211][105620] Updated weights for policy 1, policy_version 158699 (0.0005) [2023-12-26 16:31:03,271][105620] Updated weights for policy 1, policy_version 158709 (0.0008) [2023-12-26 16:31:03,329][105620] Updated weights for policy 1, policy_version 158719 (0.0008) [2023-12-26 16:31:03,755][105692] Updated weights for policy 0, policy_version 157952 (0.0010) [2023-12-26 16:31:03,816][105692] Updated weights for policy 0, policy_version 157962 (0.0005) [2023-12-26 16:31:03,878][105692] Updated weights for policy 0, policy_version 157972 (0.0008) [2023-12-26 16:31:03,942][105620] Updated weights for policy 1, policy_version 158729 (0.0008) [2023-12-26 16:31:04,008][105620] Updated weights for policy 1, policy_version 158739 (0.0009) [2023-12-26 16:31:04,058][105620] Updated weights for policy 1, policy_version 158749 (0.0008) [2023-12-26 16:31:04,115][105620] Updated weights for policy 1, policy_version 158759 (0.0010) [2023-12-26 16:31:04,616][105692] Updated weights for policy 0, policy_version 157982 (0.0010) [2023-12-26 16:31:04,682][105692] Updated weights for policy 0, policy_version 157992 (0.0010) [2023-12-26 16:31:04,745][105692] Updated weights for policy 0, policy_version 158002 (0.0008) [2023-12-26 16:31:04,877][105620] Updated weights for policy 1, policy_version 158769 (0.0010) [2023-12-26 16:31:04,937][105620] Updated weights for policy 1, policy_version 158779 (0.0010) [2023-12-26 16:31:04,995][105620] Updated weights for policy 1, policy_version 158789 (0.0010) [2023-12-26 16:31:05,436][105692] Updated weights for policy 0, policy_version 158012 (0.0010) [2023-12-26 16:31:05,484][105692] Updated weights for policy 0, policy_version 158022 (0.0010) [2023-12-26 16:31:05,532][105692] Updated weights for policy 0, policy_version 158032 (0.0010) [2023-12-26 16:31:05,657][105620] Updated weights for policy 1, policy_version 158799 (0.0007) [2023-12-26 16:31:05,719][105620] Updated weights for policy 1, policy_version 158809 (0.0006) [2023-12-26 16:31:05,774][105620] Updated weights for policy 1, policy_version 158819 (0.0008) [2023-12-26 16:31:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 81133568. Throughput: 0: 9626.2, 1: 9791.4. Samples: 81121800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:31:06,063][104569] Avg episode reward: [(0, '8646.418'), (1, '9348.300')] [2023-12-26 16:31:06,260][105692] Updated weights for policy 0, policy_version 158042 (0.0009) [2023-12-26 16:31:06,313][105692] Updated weights for policy 0, policy_version 158052 (0.0010) [2023-12-26 16:31:06,377][105692] Updated weights for policy 0, policy_version 158062 (0.0010) [2023-12-26 16:31:06,437][105692] Updated weights for policy 0, policy_version 158072 (0.0010) [2023-12-26 16:31:06,452][105620] Updated weights for policy 1, policy_version 158829 (0.0007) [2023-12-26 16:31:06,515][105620] Updated weights for policy 1, policy_version 158839 (0.0009) [2023-12-26 16:31:06,589][105620] Updated weights for policy 1, policy_version 158849 (0.0010) [2023-12-26 16:31:07,052][105692] Updated weights for policy 0, policy_version 158082 (0.0005) [2023-12-26 16:31:07,103][105692] Updated weights for policy 0, policy_version 158092 (0.0007) [2023-12-26 16:31:07,162][105692] Updated weights for policy 0, policy_version 158102 (0.0011) [2023-12-26 16:31:07,371][105620] Updated weights for policy 1, policy_version 158859 (0.0009) [2023-12-26 16:31:07,432][105620] Updated weights for policy 1, policy_version 158869 (0.0009) [2023-12-26 16:31:07,491][105620] Updated weights for policy 1, policy_version 158879 (0.0006) [2023-12-26 16:31:07,739][105692] Updated weights for policy 0, policy_version 158112 (0.0011) [2023-12-26 16:31:07,798][105692] Updated weights for policy 0, policy_version 158122 (0.0011) [2023-12-26 16:31:07,852][105692] Updated weights for policy 0, policy_version 158132 (0.0010) [2023-12-26 16:31:08,191][105620] Updated weights for policy 1, policy_version 158889 (0.0007) [2023-12-26 16:31:08,240][105620] Updated weights for policy 1, policy_version 158899 (0.0010) [2023-12-26 16:31:08,291][105620] Updated weights for policy 1, policy_version 158909 (0.0010) [2023-12-26 16:31:08,349][105620] Updated weights for policy 1, policy_version 158919 (0.0011) [2023-12-26 16:31:08,553][105692] Updated weights for policy 0, policy_version 158142 (0.0011) [2023-12-26 16:31:08,622][105692] Updated weights for policy 0, policy_version 158152 (0.0008) [2023-12-26 16:31:08,682][105692] Updated weights for policy 0, policy_version 158162 (0.0009) [2023-12-26 16:31:09,000][105620] Updated weights for policy 1, policy_version 158929 (0.0010) [2023-12-26 16:31:09,046][105620] Updated weights for policy 1, policy_version 158939 (0.0010) [2023-12-26 16:31:09,108][105620] Updated weights for policy 1, policy_version 158949 (0.0010) [2023-12-26 16:31:09,377][105692] Updated weights for policy 0, policy_version 158172 (0.0009) [2023-12-26 16:31:09,436][105692] Updated weights for policy 0, policy_version 158182 (0.0008) [2023-12-26 16:31:09,496][105692] Updated weights for policy 0, policy_version 158192 (0.0008) [2023-12-26 16:31:09,875][105620] Updated weights for policy 1, policy_version 158959 (0.0011) [2023-12-26 16:31:09,940][105620] Updated weights for policy 1, policy_version 158969 (0.0011) [2023-12-26 16:31:09,995][105620] Updated weights for policy 1, policy_version 158979 (0.0010) [2023-12-26 16:31:10,300][105692] Updated weights for policy 0, policy_version 158202 (0.0009) [2023-12-26 16:31:10,364][105692] Updated weights for policy 0, policy_version 158212 (0.0011) [2023-12-26 16:31:10,424][105692] Updated weights for policy 0, policy_version 158222 (0.0011) [2023-12-26 16:31:10,487][105692] Updated weights for policy 0, policy_version 158232 (0.0010) [2023-12-26 16:31:10,725][105620] Updated weights for policy 1, policy_version 158989 (0.0008) [2023-12-26 16:31:10,783][105620] Updated weights for policy 1, policy_version 158999 (0.0005) [2023-12-26 16:31:10,845][105620] Updated weights for policy 1, policy_version 159009 (0.0007) [2023-12-26 16:31:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 81231872. Throughput: 0: 9649.0, 1: 9768.0. Samples: 81239836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:31:11,062][104569] Avg episode reward: [(0, '8646.212'), (1, '9284.761')] [2023-12-26 16:31:11,200][105692] Updated weights for policy 0, policy_version 158242 (0.0009) [2023-12-26 16:31:11,267][105692] Updated weights for policy 0, policy_version 158252 (0.0009) [2023-12-26 16:31:11,325][105692] Updated weights for policy 0, policy_version 158262 (0.0008) [2023-12-26 16:31:11,534][105620] Updated weights for policy 1, policy_version 159019 (0.0008) [2023-12-26 16:31:11,603][105620] Updated weights for policy 1, policy_version 159029 (0.0008) [2023-12-26 16:31:11,674][105620] Updated weights for policy 1, policy_version 159039 (0.0008) [2023-12-26 16:31:12,104][105692] Updated weights for policy 0, policy_version 158272 (0.0010) [2023-12-26 16:31:12,169][105692] Updated weights for policy 0, policy_version 158282 (0.0010) [2023-12-26 16:31:12,233][105692] Updated weights for policy 0, policy_version 158292 (0.0010) [2023-12-26 16:31:12,400][105620] Updated weights for policy 1, policy_version 159049 (0.0008) [2023-12-26 16:31:12,469][105620] Updated weights for policy 1, policy_version 159059 (0.0011) [2023-12-26 16:31:12,528][105620] Updated weights for policy 1, policy_version 159069 (0.0010) [2023-12-26 16:31:12,583][105620] Updated weights for policy 1, policy_version 159079 (0.0010) [2023-12-26 16:31:12,953][105692] Updated weights for policy 0, policy_version 158302 (0.0010) [2023-12-26 16:31:13,018][105692] Updated weights for policy 0, policy_version 158312 (0.0008) [2023-12-26 16:31:13,087][105692] Updated weights for policy 0, policy_version 158322 (0.0009) [2023-12-26 16:31:13,318][105620] Updated weights for policy 1, policy_version 159089 (0.0010) [2023-12-26 16:31:13,369][105620] Updated weights for policy 1, policy_version 159099 (0.0009) [2023-12-26 16:31:13,420][105620] Updated weights for policy 1, policy_version 159109 (0.0009) [2023-12-26 16:31:13,747][105692] Updated weights for policy 0, policy_version 158332 (0.0008) [2023-12-26 16:31:13,796][105692] Updated weights for policy 0, policy_version 158342 (0.0005) [2023-12-26 16:31:13,849][105692] Updated weights for policy 0, policy_version 158352 (0.0005) [2023-12-26 16:31:14,222][105620] Updated weights for policy 1, policy_version 159119 (0.0006) [2023-12-26 16:31:14,274][105620] Updated weights for policy 1, policy_version 159129 (0.0005) [2023-12-26 16:31:14,328][105620] Updated weights for policy 1, policy_version 159139 (0.0007) [2023-12-26 16:31:14,497][105692] Updated weights for policy 0, policy_version 158362 (0.0006) [2023-12-26 16:31:14,557][105692] Updated weights for policy 0, policy_version 158372 (0.0009) [2023-12-26 16:31:14,610][105692] Updated weights for policy 0, policy_version 158382 (0.0010) [2023-12-26 16:31:14,661][105692] Updated weights for policy 0, policy_version 158392 (0.0009) [2023-12-26 16:31:14,986][105620] Updated weights for policy 1, policy_version 159149 (0.0009) [2023-12-26 16:31:15,044][105620] Updated weights for policy 1, policy_version 159159 (0.0009) [2023-12-26 16:31:15,092][105620] Updated weights for policy 1, policy_version 159169 (0.0009) [2023-12-26 16:31:15,388][105692] Updated weights for policy 0, policy_version 158402 (0.0009) [2023-12-26 16:31:15,442][105692] Updated weights for policy 0, policy_version 158412 (0.0008) [2023-12-26 16:31:15,498][105692] Updated weights for policy 0, policy_version 158422 (0.0010) [2023-12-26 16:31:15,881][105620] Updated weights for policy 1, policy_version 159179 (0.0009) [2023-12-26 16:31:15,940][105620] Updated weights for policy 1, policy_version 159189 (0.0009) [2023-12-26 16:31:16,003][105620] Updated weights for policy 1, policy_version 159199 (0.0009) [2023-12-26 16:31:16,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 81330176. Throughput: 0: 9615.9, 1: 9632.2. Samples: 81295960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:31:16,062][104569] Avg episode reward: [(0, '9085.209'), (1, '8329.840')] [2023-12-26 16:31:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000158424_40566784.pth... [2023-12-26 16:31:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000159208_40763392.pth... [2023-12-26 16:31:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000158056_40468480.pth [2023-12-26 16:31:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000157304_40280064.pth [2023-12-26 16:31:16,274][105692] Updated weights for policy 0, policy_version 158432 (0.0009) [2023-12-26 16:31:16,325][105692] Updated weights for policy 0, policy_version 158442 (0.0009) [2023-12-26 16:31:16,379][105692] Updated weights for policy 0, policy_version 158452 (0.0009) [2023-12-26 16:31:16,787][105620] Updated weights for policy 1, policy_version 159209 (0.0009) [2023-12-26 16:31:16,849][105620] Updated weights for policy 1, policy_version 159219 (0.0009) [2023-12-26 16:31:16,902][105620] Updated weights for policy 1, policy_version 159229 (0.0009) [2023-12-26 16:31:16,951][105620] Updated weights for policy 1, policy_version 159239 (0.0009) [2023-12-26 16:31:17,097][105692] Updated weights for policy 0, policy_version 158462 (0.0008) [2023-12-26 16:31:17,151][105692] Updated weights for policy 0, policy_version 158472 (0.0010) [2023-12-26 16:31:17,209][105692] Updated weights for policy 0, policy_version 158482 (0.0010) [2023-12-26 16:31:17,575][105620] Updated weights for policy 1, policy_version 159249 (0.0009) [2023-12-26 16:31:17,620][105620] Updated weights for policy 1, policy_version 159259 (0.0008) [2023-12-26 16:31:17,672][105620] Updated weights for policy 1, policy_version 159269 (0.0005) [2023-12-26 16:31:18,098][105692] Updated weights for policy 0, policy_version 158492 (0.0010) [2023-12-26 16:31:18,163][105692] Updated weights for policy 0, policy_version 158502 (0.0010) [2023-12-26 16:31:18,226][105692] Updated weights for policy 0, policy_version 158512 (0.0009) [2023-12-26 16:31:18,275][105620] Updated weights for policy 1, policy_version 159279 (0.0007) [2023-12-26 16:31:18,329][105620] Updated weights for policy 1, policy_version 159289 (0.0006) [2023-12-26 16:31:18,383][105620] Updated weights for policy 1, policy_version 159299 (0.0008) [2023-12-26 16:31:18,973][105692] Updated weights for policy 0, policy_version 158522 (0.0010) [2023-12-26 16:31:18,996][105620] Updated weights for policy 1, policy_version 159309 (0.0007) [2023-12-26 16:31:19,030][105692] Updated weights for policy 0, policy_version 158532 (0.0009) [2023-12-26 16:31:19,044][105620] Updated weights for policy 1, policy_version 159319 (0.0006) [2023-12-26 16:31:19,086][105692] Updated weights for policy 0, policy_version 158542 (0.0008) [2023-12-26 16:31:19,093][105620] Updated weights for policy 1, policy_version 159329 (0.0006) [2023-12-26 16:31:19,136][105692] Updated weights for policy 0, policy_version 158552 (0.0008) [2023-12-26 16:31:19,780][105620] Updated weights for policy 1, policy_version 159339 (0.0006) [2023-12-26 16:31:19,849][105620] Updated weights for policy 1, policy_version 159349 (0.0007) [2023-12-26 16:31:19,907][105620] Updated weights for policy 1, policy_version 159359 (0.0006) [2023-12-26 16:31:20,007][105692] Updated weights for policy 0, policy_version 158562 (0.0008) [2023-12-26 16:31:20,065][105692] Updated weights for policy 0, policy_version 158572 (0.0009) [2023-12-26 16:31:20,127][105692] Updated weights for policy 0, policy_version 158582 (0.0007) [2023-12-26 16:31:20,586][105620] Updated weights for policy 1, policy_version 159369 (0.0012) [2023-12-26 16:31:20,650][105620] Updated weights for policy 1, policy_version 159379 (0.0009) [2023-12-26 16:31:20,715][105620] Updated weights for policy 1, policy_version 159389 (0.0011) [2023-12-26 16:31:20,780][105620] Updated weights for policy 1, policy_version 159399 (0.0011) [2023-12-26 16:31:20,936][105692] Updated weights for policy 0, policy_version 158592 (0.0008) [2023-12-26 16:31:20,992][105692] Updated weights for policy 0, policy_version 158602 (0.0009) [2023-12-26 16:31:21,054][105692] Updated weights for policy 0, policy_version 158612 (0.0009) [2023-12-26 16:31:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 81420288. Throughput: 0: 9510.8, 1: 9751.1. Samples: 81413868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:31:21,062][104569] Avg episode reward: [(0, '9175.843'), (1, '8624.348')] [2023-12-26 16:31:21,548][105620] Updated weights for policy 1, policy_version 159409 (0.0008) [2023-12-26 16:31:21,608][105620] Updated weights for policy 1, policy_version 159419 (0.0008) [2023-12-26 16:31:21,676][105620] Updated weights for policy 1, policy_version 159429 (0.0009) [2023-12-26 16:31:21,940][105692] Updated weights for policy 0, policy_version 158622 (0.0008) [2023-12-26 16:31:22,003][105692] Updated weights for policy 0, policy_version 158632 (0.0010) [2023-12-26 16:31:22,071][105692] Updated weights for policy 0, policy_version 158642 (0.0007) [2023-12-26 16:31:22,315][105620] Updated weights for policy 1, policy_version 159439 (0.0008) [2023-12-26 16:31:22,383][105620] Updated weights for policy 1, policy_version 159449 (0.0008) [2023-12-26 16:31:22,445][105620] Updated weights for policy 1, policy_version 159459 (0.0009) [2023-12-26 16:31:22,809][105692] Updated weights for policy 0, policy_version 158652 (0.0008) [2023-12-26 16:31:22,861][105692] Updated weights for policy 0, policy_version 158662 (0.0009) [2023-12-26 16:31:22,924][105692] Updated weights for policy 0, policy_version 158672 (0.0009) [2023-12-26 16:31:23,053][105620] Updated weights for policy 1, policy_version 159469 (0.0008) [2023-12-26 16:31:23,100][105620] Updated weights for policy 1, policy_version 159479 (0.0009) [2023-12-26 16:31:23,153][105620] Updated weights for policy 1, policy_version 159489 (0.0008) [2023-12-26 16:31:23,712][105692] Updated weights for policy 0, policy_version 158682 (0.0010) [2023-12-26 16:31:23,769][105692] Updated weights for policy 0, policy_version 158692 (0.0010) [2023-12-26 16:31:23,832][105692] Updated weights for policy 0, policy_version 158703 (0.0011) [2023-12-26 16:31:23,870][105620] Updated weights for policy 1, policy_version 159499 (0.0007) [2023-12-26 16:31:23,918][105620] Updated weights for policy 1, policy_version 159509 (0.0005) [2023-12-26 16:31:23,966][105620] Updated weights for policy 1, policy_version 159519 (0.0005) [2023-12-26 16:31:24,586][105620] Updated weights for policy 1, policy_version 159529 (0.0009) [2023-12-26 16:31:24,643][105620] Updated weights for policy 1, policy_version 159539 (0.0005) [2023-12-26 16:31:24,687][105692] Updated weights for policy 0, policy_version 158714 (0.0009) [2023-12-26 16:31:24,689][105620] Updated weights for policy 1, policy_version 159549 (0.0007) [2023-12-26 16:31:24,739][105692] Updated weights for policy 0, policy_version 158724 (0.0007) [2023-12-26 16:31:24,749][105620] Updated weights for policy 1, policy_version 159559 (0.0006) [2023-12-26 16:31:24,791][105692] Updated weights for policy 0, policy_version 158734 (0.0008) [2023-12-26 16:31:24,837][105692] Updated weights for policy 0, policy_version 158744 (0.0008) [2023-12-26 16:31:25,320][105620] Updated weights for policy 1, policy_version 159569 (0.0006) [2023-12-26 16:31:25,380][105620] Updated weights for policy 1, policy_version 159579 (0.0005) [2023-12-26 16:31:25,438][105620] Updated weights for policy 1, policy_version 159589 (0.0005) [2023-12-26 16:31:25,696][105692] Updated weights for policy 0, policy_version 158754 (0.0005) [2023-12-26 16:31:25,757][105692] Updated weights for policy 0, policy_version 158764 (0.0005) [2023-12-26 16:31:25,820][105692] Updated weights for policy 0, policy_version 158774 (0.0007) [2023-12-26 16:31:25,963][105620] Updated weights for policy 1, policy_version 159599 (0.0005) [2023-12-26 16:31:26,026][105620] Updated weights for policy 1, policy_version 159609 (0.0006) [2023-12-26 16:31:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 81518592. Throughput: 0: 9383.6, 1: 10007.2. Samples: 81529448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:31:26,062][104569] Avg episode reward: [(0, '9266.305'), (1, '9345.519')] [2023-12-26 16:31:26,080][105620] Updated weights for policy 1, policy_version 159619 (0.0005) [2023-12-26 16:31:26,543][105692] Updated weights for policy 0, policy_version 158784 (0.0010) [2023-12-26 16:31:26,611][105692] Updated weights for policy 0, policy_version 158794 (0.0010) [2023-12-26 16:31:26,614][105620] Updated weights for policy 1, policy_version 159629 (0.0007) [2023-12-26 16:31:26,666][105620] Updated weights for policy 1, policy_version 159639 (0.0007) [2023-12-26 16:31:26,669][105692] Updated weights for policy 0, policy_version 158804 (0.0010) [2023-12-26 16:31:26,718][105620] Updated weights for policy 1, policy_version 159649 (0.0006) [2023-12-26 16:31:27,251][105692] Updated weights for policy 0, policy_version 158814 (0.0007) [2023-12-26 16:31:27,303][105692] Updated weights for policy 0, policy_version 158824 (0.0005) [2023-12-26 16:31:27,355][105692] Updated weights for policy 0, policy_version 158834 (0.0006) [2023-12-26 16:31:27,426][105620] Updated weights for policy 1, policy_version 159659 (0.0007) [2023-12-26 16:31:27,480][105620] Updated weights for policy 1, policy_version 159669 (0.0010) [2023-12-26 16:31:27,538][105620] Updated weights for policy 1, policy_version 159679 (0.0010) [2023-12-26 16:31:27,900][105692] Updated weights for policy 0, policy_version 158844 (0.0005) [2023-12-26 16:31:27,960][105692] Updated weights for policy 0, policy_version 158854 (0.0007) [2023-12-26 16:31:28,008][105692] Updated weights for policy 0, policy_version 158864 (0.0007) [2023-12-26 16:31:28,273][105620] Updated weights for policy 1, policy_version 159689 (0.0010) [2023-12-26 16:31:28,320][105620] Updated weights for policy 1, policy_version 159699 (0.0010) [2023-12-26 16:31:28,380][105620] Updated weights for policy 1, policy_version 159709 (0.0010) [2023-12-26 16:31:28,433][105620] Updated weights for policy 1, policy_version 159719 (0.0006) [2023-12-26 16:31:28,652][105692] Updated weights for policy 0, policy_version 158874 (0.0007) [2023-12-26 16:31:28,703][105692] Updated weights for policy 0, policy_version 158884 (0.0010) [2023-12-26 16:31:28,758][105692] Updated weights for policy 0, policy_version 158894 (0.0010) [2023-12-26 16:31:28,816][105692] Updated weights for policy 0, policy_version 158904 (0.0010) [2023-12-26 16:31:29,197][105620] Updated weights for policy 1, policy_version 159729 (0.0010) [2023-12-26 16:31:29,268][105620] Updated weights for policy 1, policy_version 159739 (0.0008) [2023-12-26 16:31:29,330][105620] Updated weights for policy 1, policy_version 159749 (0.0009) [2023-12-26 16:31:29,538][105692] Updated weights for policy 0, policy_version 158914 (0.0008) [2023-12-26 16:31:29,607][105692] Updated weights for policy 0, policy_version 158924 (0.0005) [2023-12-26 16:31:29,666][105692] Updated weights for policy 0, policy_version 158934 (0.0008) [2023-12-26 16:31:30,131][105620] Updated weights for policy 1, policy_version 159759 (0.0010) [2023-12-26 16:31:30,179][105620] Updated weights for policy 1, policy_version 159769 (0.0010) [2023-12-26 16:31:30,227][105620] Updated weights for policy 1, policy_version 159779 (0.0010) [2023-12-26 16:31:30,380][105692] Updated weights for policy 0, policy_version 158944 (0.0008) [2023-12-26 16:31:30,428][105692] Updated weights for policy 0, policy_version 158954 (0.0008) [2023-12-26 16:31:30,484][105692] Updated weights for policy 0, policy_version 158964 (0.0008) [2023-12-26 16:31:30,971][105620] Updated weights for policy 1, policy_version 159789 (0.0010) [2023-12-26 16:31:31,026][105620] Updated weights for policy 1, policy_version 159799 (0.0010) [2023-12-26 16:31:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 81616896. Throughput: 0: 9545.3, 1: 10035.6. Samples: 81593032. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:31:31,063][104569] Avg episode reward: [(0, '9054.028'), (1, '9167.885')] [2023-12-26 16:31:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000158968_40706048.pth... [2023-12-26 16:31:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000157848_40419328.pth [2023-12-26 16:31:31,082][105620] Updated weights for policy 1, policy_version 159809 (0.0010) [2023-12-26 16:31:31,128][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000159816_40919040.pth... [2023-12-26 16:31:31,132][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000158632_40615936.pth [2023-12-26 16:31:31,190][105692] Updated weights for policy 0, policy_version 158974 (0.0008) [2023-12-26 16:31:31,253][105692] Updated weights for policy 0, policy_version 158984 (0.0008) [2023-12-26 16:31:31,309][105692] Updated weights for policy 0, policy_version 158994 (0.0008) [2023-12-26 16:31:31,860][105620] Updated weights for policy 1, policy_version 159819 (0.0011) [2023-12-26 16:31:31,913][105620] Updated weights for policy 1, policy_version 159829 (0.0010) [2023-12-26 16:31:31,966][105620] Updated weights for policy 1, policy_version 159839 (0.0010) [2023-12-26 16:31:32,035][105692] Updated weights for policy 0, policy_version 159004 (0.0009) [2023-12-26 16:31:32,083][105692] Updated weights for policy 0, policy_version 159014 (0.0009) [2023-12-26 16:31:32,138][105692] Updated weights for policy 0, policy_version 159024 (0.0005) [2023-12-26 16:31:32,738][105620] Updated weights for policy 1, policy_version 159849 (0.0011) [2023-12-26 16:31:32,797][105620] Updated weights for policy 1, policy_version 159859 (0.0011) [2023-12-26 16:31:32,803][105692] Updated weights for policy 0, policy_version 159034 (0.0005) [2023-12-26 16:31:32,855][105620] Updated weights for policy 1, policy_version 159869 (0.0010) [2023-12-26 16:31:32,865][105692] Updated weights for policy 0, policy_version 159044 (0.0006) [2023-12-26 16:31:32,914][105620] Updated weights for policy 1, policy_version 159879 (0.0010) [2023-12-26 16:31:32,922][105692] Updated weights for policy 0, policy_version 159054 (0.0005) [2023-12-26 16:31:32,974][105692] Updated weights for policy 0, policy_version 159064 (0.0005) [2023-12-26 16:31:33,494][105692] Updated weights for policy 0, policy_version 159074 (0.0005) [2023-12-26 16:31:33,543][105692] Updated weights for policy 0, policy_version 159084 (0.0005) [2023-12-26 16:31:33,583][105620] Updated weights for policy 1, policy_version 159889 (0.0006) [2023-12-26 16:31:33,596][105692] Updated weights for policy 0, policy_version 159094 (0.0005) [2023-12-26 16:31:33,627][105620] Updated weights for policy 1, policy_version 159899 (0.0005) [2023-12-26 16:31:33,672][105620] Updated weights for policy 1, policy_version 159909 (0.0005) [2023-12-26 16:31:34,122][105692] Updated weights for policy 0, policy_version 159104 (0.0005) [2023-12-26 16:31:34,188][105692] Updated weights for policy 0, policy_version 159114 (0.0008) [2023-12-26 16:31:34,239][105620] Updated weights for policy 1, policy_version 159919 (0.0005) [2023-12-26 16:31:34,250][105692] Updated weights for policy 0, policy_version 159124 (0.0008) [2023-12-26 16:31:34,309][105620] Updated weights for policy 1, policy_version 159929 (0.0006) [2023-12-26 16:31:34,369][105620] Updated weights for policy 1, policy_version 159939 (0.0007) [2023-12-26 16:31:34,948][105692] Updated weights for policy 0, policy_version 159134 (0.0010) [2023-12-26 16:31:35,006][105692] Updated weights for policy 0, policy_version 159144 (0.0011) [2023-12-26 16:31:35,064][105692] Updated weights for policy 0, policy_version 159154 (0.0008) [2023-12-26 16:31:35,082][105620] Updated weights for policy 1, policy_version 159949 (0.0007) [2023-12-26 16:31:35,136][105620] Updated weights for policy 1, policy_version 159959 (0.0007) [2023-12-26 16:31:35,191][105620] Updated weights for policy 1, policy_version 159969 (0.0008) [2023-12-26 16:31:35,793][105692] Updated weights for policy 0, policy_version 159164 (0.0008) [2023-12-26 16:31:35,849][105692] Updated weights for policy 0, policy_version 159174 (0.0008) [2023-12-26 16:31:35,901][105692] Updated weights for policy 0, policy_version 159184 (0.0006) [2023-12-26 16:31:35,969][105620] Updated weights for policy 1, policy_version 159979 (0.0008) [2023-12-26 16:31:36,026][105620] Updated weights for policy 1, policy_version 159990 (0.0009) [2023-12-26 16:31:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 81723392. Throughput: 0: 9678.8, 1: 10019.4. Samples: 81714104. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:31:36,062][104569] Avg episode reward: [(0, '8781.711'), (1, '8986.809')] [2023-12-26 16:31:36,090][105620] Updated weights for policy 1, policy_version 160000 (0.0010) [2023-12-26 16:31:36,541][105692] Updated weights for policy 0, policy_version 159194 (0.0006) [2023-12-26 16:31:36,600][105692] Updated weights for policy 0, policy_version 159204 (0.0011) [2023-12-26 16:31:36,664][105692] Updated weights for policy 0, policy_version 159214 (0.0011) [2023-12-26 16:31:36,726][105692] Updated weights for policy 0, policy_version 159224 (0.0011) [2023-12-26 16:31:36,849][105620] Updated weights for policy 1, policy_version 160010 (0.0009) [2023-12-26 16:31:36,908][105620] Updated weights for policy 1, policy_version 160020 (0.0008) [2023-12-26 16:31:36,971][105620] Updated weights for policy 1, policy_version 160030 (0.0008) [2023-12-26 16:31:37,038][105620] Updated weights for policy 1, policy_version 160040 (0.0008) [2023-12-26 16:31:37,471][105692] Updated weights for policy 0, policy_version 159234 (0.0010) [2023-12-26 16:31:37,532][105692] Updated weights for policy 0, policy_version 159244 (0.0008) [2023-12-26 16:31:37,582][105692] Updated weights for policy 0, policy_version 159254 (0.0005) [2023-12-26 16:31:37,808][105620] Updated weights for policy 1, policy_version 160050 (0.0008) [2023-12-26 16:31:37,873][105620] Updated weights for policy 1, policy_version 160060 (0.0008) [2023-12-26 16:31:37,939][105620] Updated weights for policy 1, policy_version 160070 (0.0008) [2023-12-26 16:31:38,281][105692] Updated weights for policy 0, policy_version 159264 (0.0009) [2023-12-26 16:31:38,333][105692] Updated weights for policy 0, policy_version 159274 (0.0010) [2023-12-26 16:31:38,388][105692] Updated weights for policy 0, policy_version 159284 (0.0010) [2023-12-26 16:31:38,574][105620] Updated weights for policy 1, policy_version 160080 (0.0008) [2023-12-26 16:31:38,640][105620] Updated weights for policy 1, policy_version 160090 (0.0008) [2023-12-26 16:31:38,706][105620] Updated weights for policy 1, policy_version 160100 (0.0008) [2023-12-26 16:31:39,151][105692] Updated weights for policy 0, policy_version 159294 (0.0010) [2023-12-26 16:31:39,203][105692] Updated weights for policy 0, policy_version 159304 (0.0010) [2023-12-26 16:31:39,267][105692] Updated weights for policy 0, policy_version 159314 (0.0010) [2023-12-26 16:31:39,468][105620] Updated weights for policy 1, policy_version 160110 (0.0008) [2023-12-26 16:31:39,533][105620] Updated weights for policy 1, policy_version 160120 (0.0005) [2023-12-26 16:31:39,604][105620] Updated weights for policy 1, policy_version 160130 (0.0009) [2023-12-26 16:31:39,975][105692] Updated weights for policy 0, policy_version 159324 (0.0009) [2023-12-26 16:31:40,035][105692] Updated weights for policy 0, policy_version 159334 (0.0009) [2023-12-26 16:31:40,096][105692] Updated weights for policy 0, policy_version 159344 (0.0010) [2023-12-26 16:31:40,349][105620] Updated weights for policy 1, policy_version 160140 (0.0008) [2023-12-26 16:31:40,407][105620] Updated weights for policy 1, policy_version 160150 (0.0008) [2023-12-26 16:31:40,469][105620] Updated weights for policy 1, policy_version 160160 (0.0008) [2023-12-26 16:31:40,835][105692] Updated weights for policy 0, policy_version 159354 (0.0010) [2023-12-26 16:31:40,897][105692] Updated weights for policy 0, policy_version 159364 (0.0007) [2023-12-26 16:31:40,952][105692] Updated weights for policy 0, policy_version 159374 (0.0006) [2023-12-26 16:31:41,018][105692] Updated weights for policy 0, policy_version 159384 (0.0006) [2023-12-26 16:31:41,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 81821696. Throughput: 0: 9716.6, 1: 9925.0. Samples: 81828096. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:31:41,062][104569] Avg episode reward: [(0, '8233.709'), (1, '8895.751')] [2023-12-26 16:31:41,245][105620] Updated weights for policy 1, policy_version 160170 (0.0009) [2023-12-26 16:31:41,307][105620] Updated weights for policy 1, policy_version 160180 (0.0009) [2023-12-26 16:31:41,378][105620] Updated weights for policy 1, policy_version 160190 (0.0008) [2023-12-26 16:31:41,435][105620] Updated weights for policy 1, policy_version 160200 (0.0006) [2023-12-26 16:31:41,753][105692] Updated weights for policy 0, policy_version 159394 (0.0008) [2023-12-26 16:31:41,818][105692] Updated weights for policy 0, policy_version 159404 (0.0009) [2023-12-26 16:31:41,882][105692] Updated weights for policy 0, policy_version 159414 (0.0009) [2023-12-26 16:31:42,217][105620] Updated weights for policy 1, policy_version 160210 (0.0009) [2023-12-26 16:31:42,273][105620] Updated weights for policy 1, policy_version 160220 (0.0009) [2023-12-26 16:31:42,333][105620] Updated weights for policy 1, policy_version 160230 (0.0009) [2023-12-26 16:31:42,528][105692] Updated weights for policy 0, policy_version 159424 (0.0006) [2023-12-26 16:31:42,587][105692] Updated weights for policy 0, policy_version 159434 (0.0008) [2023-12-26 16:31:42,658][105692] Updated weights for policy 0, policy_version 159444 (0.0007) [2023-12-26 16:31:43,066][105620] Updated weights for policy 1, policy_version 160240 (0.0009) [2023-12-26 16:31:43,134][105620] Updated weights for policy 1, policy_version 160250 (0.0009) [2023-12-26 16:31:43,180][105620] Updated weights for policy 1, policy_version 160260 (0.0008) [2023-12-26 16:31:43,283][105692] Updated weights for policy 0, policy_version 159454 (0.0007) [2023-12-26 16:31:43,344][105692] Updated weights for policy 0, policy_version 159464 (0.0005) [2023-12-26 16:31:43,407][105692] Updated weights for policy 0, policy_version 159474 (0.0005) [2023-12-26 16:31:43,991][105620] Updated weights for policy 1, policy_version 160270 (0.0009) [2023-12-26 16:31:44,027][105692] Updated weights for policy 0, policy_version 159484 (0.0006) [2023-12-26 16:31:44,053][105620] Updated weights for policy 1, policy_version 160280 (0.0009) [2023-12-26 16:31:44,087][105692] Updated weights for policy 0, policy_version 159494 (0.0006) [2023-12-26 16:31:44,106][105620] Updated weights for policy 1, policy_version 160290 (0.0006) [2023-12-26 16:31:44,141][105692] Updated weights for policy 0, policy_version 159504 (0.0006) [2023-12-26 16:31:44,797][105620] Updated weights for policy 1, policy_version 160300 (0.0009) [2023-12-26 16:31:44,860][105620] Updated weights for policy 1, policy_version 160310 (0.0008) [2023-12-26 16:31:44,926][105692] Updated weights for policy 0, policy_version 159514 (0.0007) [2023-12-26 16:31:44,928][105620] Updated weights for policy 1, policy_version 160320 (0.0009) [2023-12-26 16:31:44,992][105692] Updated weights for policy 0, policy_version 159524 (0.0008) [2023-12-26 16:31:45,051][105692] Updated weights for policy 0, policy_version 159534 (0.0009) [2023-12-26 16:31:45,098][105692] Updated weights for policy 0, policy_version 159544 (0.0008) [2023-12-26 16:31:45,735][105620] Updated weights for policy 1, policy_version 160330 (0.0008) [2023-12-26 16:31:45,743][105692] Updated weights for policy 0, policy_version 159554 (0.0007) [2023-12-26 16:31:45,794][105620] Updated weights for policy 1, policy_version 160340 (0.0009) [2023-12-26 16:31:45,799][105692] Updated weights for policy 0, policy_version 159564 (0.0011) [2023-12-26 16:31:45,850][105620] Updated weights for policy 1, policy_version 160350 (0.0011) [2023-12-26 16:31:45,858][105692] Updated weights for policy 0, policy_version 159574 (0.0010) [2023-12-26 16:31:45,901][105620] Updated weights for policy 1, policy_version 160360 (0.0010) [2023-12-26 16:31:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 81920000. Throughput: 0: 9756.7, 1: 9841.6. Samples: 81885612. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:31:46,062][104569] Avg episode reward: [(0, '8315.685'), (1, '8991.070')] [2023-12-26 16:31:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000160360_41058304.pth... [2023-12-26 16:31:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000159576_40861696.pth... [2023-12-26 16:31:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000159208_40763392.pth [2023-12-26 16:31:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000158424_40566784.pth [2023-12-26 16:31:46,502][105692] Updated weights for policy 0, policy_version 159584 (0.0008) [2023-12-26 16:31:46,559][105692] Updated weights for policy 0, policy_version 159594 (0.0008) [2023-12-26 16:31:46,615][105692] Updated weights for policy 0, policy_version 159604 (0.0009) [2023-12-26 16:31:46,659][105620] Updated weights for policy 1, policy_version 160370 (0.0006) [2023-12-26 16:31:46,714][105620] Updated weights for policy 1, policy_version 160380 (0.0006) [2023-12-26 16:31:46,778][105620] Updated weights for policy 1, policy_version 160390 (0.0008) [2023-12-26 16:31:47,230][105692] Updated weights for policy 0, policy_version 159614 (0.0006) [2023-12-26 16:31:47,278][105692] Updated weights for policy 0, policy_version 159624 (0.0005) [2023-12-26 16:31:47,337][105692] Updated weights for policy 0, policy_version 159634 (0.0005) [2023-12-26 16:31:47,351][105620] Updated weights for policy 1, policy_version 160400 (0.0007) [2023-12-26 16:31:47,404][105620] Updated weights for policy 1, policy_version 160410 (0.0005) [2023-12-26 16:31:47,460][105620] Updated weights for policy 1, policy_version 160420 (0.0006) [2023-12-26 16:31:47,920][105692] Updated weights for policy 0, policy_version 159644 (0.0007) [2023-12-26 16:31:47,977][105692] Updated weights for policy 0, policy_version 159654 (0.0009) [2023-12-26 16:31:48,032][105692] Updated weights for policy 0, policy_version 159664 (0.0009) [2023-12-26 16:31:48,039][105620] Updated weights for policy 1, policy_version 160430 (0.0005) [2023-12-26 16:31:48,092][105620] Updated weights for policy 1, policy_version 160440 (0.0005) [2023-12-26 16:31:48,139][105620] Updated weights for policy 1, policy_version 160450 (0.0005) [2023-12-26 16:31:48,680][105692] Updated weights for policy 0, policy_version 159674 (0.0006) [2023-12-26 16:31:48,731][105692] Updated weights for policy 0, policy_version 159684 (0.0010) [2023-12-26 16:31:48,780][105692] Updated weights for policy 0, policy_version 159694 (0.0010) [2023-12-26 16:31:48,833][105692] Updated weights for policy 0, policy_version 159704 (0.0008) [2023-12-26 16:31:48,855][105620] Updated weights for policy 1, policy_version 160460 (0.0006) [2023-12-26 16:31:48,916][105620] Updated weights for policy 1, policy_version 160470 (0.0009) [2023-12-26 16:31:48,986][105620] Updated weights for policy 1, policy_version 160480 (0.0006) [2023-12-26 16:31:49,588][105692] Updated weights for policy 0, policy_version 159714 (0.0006) [2023-12-26 16:31:49,644][105692] Updated weights for policy 0, policy_version 159724 (0.0009) [2023-12-26 16:31:49,692][105692] Updated weights for policy 0, policy_version 159734 (0.0010) [2023-12-26 16:31:49,789][105620] Updated weights for policy 1, policy_version 160490 (0.0008) [2023-12-26 16:31:49,851][105620] Updated weights for policy 1, policy_version 160500 (0.0011) [2023-12-26 16:31:49,911][105620] Updated weights for policy 1, policy_version 160510 (0.0011) [2023-12-26 16:31:50,460][105692] Updated weights for policy 0, policy_version 159744 (0.0009) [2023-12-26 16:31:50,520][105692] Updated weights for policy 0, policy_version 159754 (0.0008) [2023-12-26 16:31:50,584][105692] Updated weights for policy 0, policy_version 159764 (0.0009) [2023-12-26 16:31:50,681][105620] Updated weights for policy 1, policy_version 160522 (0.0008) [2023-12-26 16:31:50,738][105620] Updated weights for policy 1, policy_version 160532 (0.0009) [2023-12-26 16:31:50,798][105620] Updated weights for policy 1, policy_version 160542 (0.0007) [2023-12-26 16:31:50,851][105620] Updated weights for policy 1, policy_version 160552 (0.0007) [2023-12-26 16:31:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 82018304. Throughput: 0: 9848.8, 1: 9840.0. Samples: 82007792. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:31:51,062][104569] Avg episode reward: [(0, '8447.702'), (1, '8902.390')] [2023-12-26 16:31:51,410][105692] Updated weights for policy 0, policy_version 159774 (0.0009) [2023-12-26 16:31:51,466][105692] Updated weights for policy 0, policy_version 159784 (0.0008) [2023-12-26 16:31:51,530][105692] Updated weights for policy 0, policy_version 159794 (0.0009) [2023-12-26 16:31:51,572][105620] Updated weights for policy 1, policy_version 160562 (0.0011) [2023-12-26 16:31:51,635][105620] Updated weights for policy 1, policy_version 160572 (0.0010) [2023-12-26 16:31:51,704][105620] Updated weights for policy 1, policy_version 160582 (0.0009) [2023-12-26 16:31:52,334][105692] Updated weights for policy 0, policy_version 159804 (0.0007) [2023-12-26 16:31:52,406][105692] Updated weights for policy 0, policy_version 159814 (0.0008) [2023-12-26 16:31:52,463][105620] Updated weights for policy 1, policy_version 160592 (0.0009) [2023-12-26 16:31:52,470][105692] Updated weights for policy 0, policy_version 159824 (0.0008) [2023-12-26 16:31:52,522][105620] Updated weights for policy 1, policy_version 160602 (0.0008) [2023-12-26 16:31:52,585][105620] Updated weights for policy 1, policy_version 160612 (0.0007) [2023-12-26 16:31:53,149][105692] Updated weights for policy 0, policy_version 159834 (0.0008) [2023-12-26 16:31:53,212][105692] Updated weights for policy 0, policy_version 159844 (0.0005) [2023-12-26 16:31:53,276][105692] Updated weights for policy 0, policy_version 159854 (0.0005) [2023-12-26 16:31:53,339][105692] Updated weights for policy 0, policy_version 159864 (0.0005) [2023-12-26 16:31:53,420][105620] Updated weights for policy 1, policy_version 160622 (0.0009) [2023-12-26 16:31:53,476][105620] Updated weights for policy 1, policy_version 160632 (0.0006) [2023-12-26 16:31:53,534][105620] Updated weights for policy 1, policy_version 160642 (0.0005) [2023-12-26 16:31:53,884][105692] Updated weights for policy 0, policy_version 159874 (0.0007) [2023-12-26 16:31:53,934][105692] Updated weights for policy 0, policy_version 159884 (0.0005) [2023-12-26 16:31:53,985][105692] Updated weights for policy 0, policy_version 159894 (0.0008) [2023-12-26 16:31:54,160][105620] Updated weights for policy 1, policy_version 160652 (0.0005) [2023-12-26 16:31:54,216][105620] Updated weights for policy 1, policy_version 160662 (0.0005) [2023-12-26 16:31:54,273][105620] Updated weights for policy 1, policy_version 160672 (0.0008) [2023-12-26 16:31:54,680][105692] Updated weights for policy 0, policy_version 159904 (0.0008) [2023-12-26 16:31:54,734][105692] Updated weights for policy 0, policy_version 159914 (0.0009) [2023-12-26 16:31:54,781][105692] Updated weights for policy 0, policy_version 159924 (0.0009) [2023-12-26 16:31:54,980][105620] Updated weights for policy 1, policy_version 160682 (0.0008) [2023-12-26 16:31:55,036][105620] Updated weights for policy 1, policy_version 160692 (0.0005) [2023-12-26 16:31:55,099][105620] Updated weights for policy 1, policy_version 160702 (0.0006) [2023-12-26 16:31:55,155][105620] Updated weights for policy 1, policy_version 160712 (0.0010) [2023-12-26 16:31:55,594][105692] Updated weights for policy 0, policy_version 159934 (0.0009) [2023-12-26 16:31:55,653][105692] Updated weights for policy 0, policy_version 159945 (0.0009) [2023-12-26 16:31:55,711][105692] Updated weights for policy 0, policy_version 159955 (0.0009) [2023-12-26 16:31:55,823][105620] Updated weights for policy 1, policy_version 160722 (0.0008) [2023-12-26 16:31:55,877][105620] Updated weights for policy 1, policy_version 160732 (0.0005) [2023-12-26 16:31:55,943][105620] Updated weights for policy 1, policy_version 160742 (0.0008) [2023-12-26 16:31:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 82116608. Throughput: 0: 9783.4, 1: 9819.5. Samples: 82121968. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:31:56,062][104569] Avg episode reward: [(0, '6608.618'), (1, '8992.013')] [2023-12-26 16:31:56,546][105620] Updated weights for policy 1, policy_version 160752 (0.0006) [2023-12-26 16:31:56,553][105692] Updated weights for policy 0, policy_version 159965 (0.0008) [2023-12-26 16:31:56,594][105620] Updated weights for policy 1, policy_version 160762 (0.0005) [2023-12-26 16:31:56,609][105692] Updated weights for policy 0, policy_version 159975 (0.0008) [2023-12-26 16:31:56,649][105620] Updated weights for policy 1, policy_version 160772 (0.0005) [2023-12-26 16:31:56,668][105692] Updated weights for policy 0, policy_version 159985 (0.0009) [2023-12-26 16:31:57,188][105620] Updated weights for policy 1, policy_version 160782 (0.0007) [2023-12-26 16:31:57,254][105620] Updated weights for policy 1, policy_version 160792 (0.0007) [2023-12-26 16:31:57,311][105620] Updated weights for policy 1, policy_version 160802 (0.0005) [2023-12-26 16:31:57,529][105692] Updated weights for policy 0, policy_version 159995 (0.0009) [2023-12-26 16:31:57,588][105692] Updated weights for policy 0, policy_version 160005 (0.0009) [2023-12-26 16:31:57,647][105692] Updated weights for policy 0, policy_version 160015 (0.0009) [2023-12-26 16:31:57,881][105620] Updated weights for policy 1, policy_version 160812 (0.0005) [2023-12-26 16:31:57,926][105620] Updated weights for policy 1, policy_version 160822 (0.0005) [2023-12-26 16:31:57,974][105620] Updated weights for policy 1, policy_version 160832 (0.0005) [2023-12-26 16:31:58,467][105692] Updated weights for policy 0, policy_version 160025 (0.0009) [2023-12-26 16:31:58,532][105692] Updated weights for policy 0, policy_version 160035 (0.0009) [2023-12-26 16:31:58,593][105692] Updated weights for policy 0, policy_version 160045 (0.0008) [2023-12-26 16:31:58,653][105692] Updated weights for policy 0, policy_version 160055 (0.0009) [2023-12-26 16:31:58,683][105620] Updated weights for policy 1, policy_version 160842 (0.0006) [2023-12-26 16:31:58,754][105620] Updated weights for policy 1, policy_version 160852 (0.0010) [2023-12-26 16:31:58,820][105620] Updated weights for policy 1, policy_version 160862 (0.0007) [2023-12-26 16:31:58,887][105620] Updated weights for policy 1, policy_version 160872 (0.0008) [2023-12-26 16:31:59,444][105692] Updated weights for policy 0, policy_version 160065 (0.0007) [2023-12-26 16:31:59,502][105692] Updated weights for policy 0, policy_version 160075 (0.0006) [2023-12-26 16:31:59,567][105692] Updated weights for policy 0, policy_version 160085 (0.0006) [2023-12-26 16:31:59,639][105620] Updated weights for policy 1, policy_version 160882 (0.0010) [2023-12-26 16:31:59,698][105620] Updated weights for policy 1, policy_version 160892 (0.0011) [2023-12-26 16:31:59,763][105620] Updated weights for policy 1, policy_version 160902 (0.0010) [2023-12-26 16:32:00,253][105692] Updated weights for policy 0, policy_version 160095 (0.0008) [2023-12-26 16:32:00,309][105692] Updated weights for policy 0, policy_version 160105 (0.0008) [2023-12-26 16:32:00,365][105692] Updated weights for policy 0, policy_version 160115 (0.0008) [2023-12-26 16:32:00,481][105620] Updated weights for policy 1, policy_version 160912 (0.0010) [2023-12-26 16:32:00,535][105620] Updated weights for policy 1, policy_version 160922 (0.0010) [2023-12-26 16:32:00,579][105620] Updated weights for policy 1, policy_version 160932 (0.0010) [2023-12-26 16:32:01,018][105692] Updated weights for policy 0, policy_version 160125 (0.0007) [2023-12-26 16:32:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 82206720. Throughput: 0: 9705.5, 1: 9952.3. Samples: 82180560. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:32:01,062][104569] Avg episode reward: [(0, '1780.716'), (1, '9040.164')] [2023-12-26 16:32:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000160936_41205760.pth... [2023-12-26 16:32:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000159816_40919040.pth [2023-12-26 16:32:01,076][105692] Updated weights for policy 0, policy_version 160135 (0.0008) [2023-12-26 16:32:01,132][105692] Updated weights for policy 0, policy_version 160145 (0.0006) [2023-12-26 16:32:01,178][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000160152_41009152.pth... [2023-12-26 16:32:01,182][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000158968_40706048.pth [2023-12-26 16:32:01,345][105620] Updated weights for policy 1, policy_version 160942 (0.0008) [2023-12-26 16:32:01,413][105620] Updated weights for policy 1, policy_version 160952 (0.0008) [2023-12-26 16:32:01,478][105620] Updated weights for policy 1, policy_version 160962 (0.0010) [2023-12-26 16:32:01,870][105692] Updated weights for policy 0, policy_version 160155 (0.0009) [2023-12-26 16:32:01,926][105692] Updated weights for policy 0, policy_version 160165 (0.0011) [2023-12-26 16:32:01,978][105692] Updated weights for policy 0, policy_version 160175 (0.0010) [2023-12-26 16:32:02,117][105620] Updated weights for policy 1, policy_version 160972 (0.0010) [2023-12-26 16:32:02,167][105620] Updated weights for policy 1, policy_version 160982 (0.0009) [2023-12-26 16:32:02,223][105620] Updated weights for policy 1, policy_version 160992 (0.0008) [2023-12-26 16:32:02,625][105692] Updated weights for policy 0, policy_version 160185 (0.0010) [2023-12-26 16:32:02,683][105692] Updated weights for policy 0, policy_version 160195 (0.0005) [2023-12-26 16:32:02,727][105692] Updated weights for policy 0, policy_version 160205 (0.0005) [2023-12-26 16:32:02,784][105692] Updated weights for policy 0, policy_version 160215 (0.0005) [2023-12-26 16:32:02,975][105620] Updated weights for policy 1, policy_version 161002 (0.0011) [2023-12-26 16:32:03,027][105620] Updated weights for policy 1, policy_version 161012 (0.0007) [2023-12-26 16:32:03,086][105620] Updated weights for policy 1, policy_version 161022 (0.0011) [2023-12-26 16:32:03,155][105620] Updated weights for policy 1, policy_version 161032 (0.0009) [2023-12-26 16:32:03,383][105692] Updated weights for policy 0, policy_version 160225 (0.0010) [2023-12-26 16:32:03,434][105692] Updated weights for policy 0, policy_version 160235 (0.0010) [2023-12-26 16:32:03,488][105692] Updated weights for policy 0, policy_version 160245 (0.0010) [2023-12-26 16:32:03,894][105620] Updated weights for policy 1, policy_version 161042 (0.0009) [2023-12-26 16:32:03,958][105620] Updated weights for policy 1, policy_version 161052 (0.0010) [2023-12-26 16:32:04,024][105620] Updated weights for policy 1, policy_version 161062 (0.0007) [2023-12-26 16:32:04,193][105692] Updated weights for policy 0, policy_version 160255 (0.0007) [2023-12-26 16:32:04,257][105692] Updated weights for policy 0, policy_version 160265 (0.0005) [2023-12-26 16:32:04,320][105692] Updated weights for policy 0, policy_version 160275 (0.0010) [2023-12-26 16:32:04,726][105620] Updated weights for policy 1, policy_version 161072 (0.0009) [2023-12-26 16:32:04,778][105620] Updated weights for policy 1, policy_version 161082 (0.0010) [2023-12-26 16:32:04,832][105620] Updated weights for policy 1, policy_version 161092 (0.0010) [2023-12-26 16:32:05,019][105692] Updated weights for policy 0, policy_version 160285 (0.0010) [2023-12-26 16:32:05,074][105692] Updated weights for policy 0, policy_version 160295 (0.0011) [2023-12-26 16:32:05,133][105692] Updated weights for policy 0, policy_version 160305 (0.0010) [2023-12-26 16:32:05,484][105620] Updated weights for policy 1, policy_version 161102 (0.0007) [2023-12-26 16:32:05,538][105620] Updated weights for policy 1, policy_version 161112 (0.0005) [2023-12-26 16:32:05,601][105620] Updated weights for policy 1, policy_version 161122 (0.0006) [2023-12-26 16:32:05,782][105692] Updated weights for policy 0, policy_version 160315 (0.0009) [2023-12-26 16:32:05,838][105692] Updated weights for policy 0, policy_version 160325 (0.0005) [2023-12-26 16:32:05,896][105692] Updated weights for policy 0, policy_version 160335 (0.0005) [2023-12-26 16:32:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 82313216. Throughput: 0: 9791.2, 1: 9858.6. Samples: 82298112. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:32:06,062][104569] Avg episode reward: [(0, '6271.105'), (1, '7569.252')] [2023-12-26 16:32:06,114][105620] Updated weights for policy 1, policy_version 161132 (0.0006) [2023-12-26 16:32:06,183][105620] Updated weights for policy 1, policy_version 161142 (0.0009) [2023-12-26 16:32:06,255][105620] Updated weights for policy 1, policy_version 161152 (0.0010) [2023-12-26 16:32:06,497][105692] Updated weights for policy 0, policy_version 160345 (0.0006) [2023-12-26 16:32:06,561][105692] Updated weights for policy 0, policy_version 160355 (0.0008) [2023-12-26 16:32:06,627][105692] Updated weights for policy 0, policy_version 160365 (0.0006) [2023-12-26 16:32:06,686][105692] Updated weights for policy 0, policy_version 160375 (0.0006) [2023-12-26 16:32:06,936][105620] Updated weights for policy 1, policy_version 161162 (0.0010) [2023-12-26 16:32:06,988][105620] Updated weights for policy 1, policy_version 161172 (0.0010) [2023-12-26 16:32:07,040][105620] Updated weights for policy 1, policy_version 161182 (0.0010) [2023-12-26 16:32:07,097][105620] Updated weights for policy 1, policy_version 161192 (0.0010) [2023-12-26 16:32:07,355][105692] Updated weights for policy 0, policy_version 160385 (0.0008) [2023-12-26 16:32:07,415][105692] Updated weights for policy 0, policy_version 160395 (0.0008) [2023-12-26 16:32:07,465][105692] Updated weights for policy 0, policy_version 160405 (0.0008) [2023-12-26 16:32:07,833][105620] Updated weights for policy 1, policy_version 161202 (0.0005) [2023-12-26 16:32:07,889][105620] Updated weights for policy 1, policy_version 161212 (0.0008) [2023-12-26 16:32:07,948][105620] Updated weights for policy 1, policy_version 161222 (0.0005) [2023-12-26 16:32:08,318][105692] Updated weights for policy 0, policy_version 160415 (0.0010) [2023-12-26 16:32:08,379][105692] Updated weights for policy 0, policy_version 160425 (0.0010) [2023-12-26 16:32:08,434][105692] Updated weights for policy 0, policy_version 160435 (0.0010) [2023-12-26 16:32:08,518][105620] Updated weights for policy 1, policy_version 161232 (0.0008) [2023-12-26 16:32:08,586][105620] Updated weights for policy 1, policy_version 161242 (0.0008) [2023-12-26 16:32:08,651][105620] Updated weights for policy 1, policy_version 161252 (0.0009) [2023-12-26 16:32:09,260][105692] Updated weights for policy 0, policy_version 160445 (0.0009) [2023-12-26 16:32:09,323][105692] Updated weights for policy 0, policy_version 160455 (0.0009) [2023-12-26 16:32:09,375][105620] Updated weights for policy 1, policy_version 161262 (0.0009) [2023-12-26 16:32:09,383][105692] Updated weights for policy 0, policy_version 160465 (0.0008) [2023-12-26 16:32:09,446][105620] Updated weights for policy 1, policy_version 161272 (0.0006) [2023-12-26 16:32:09,503][105620] Updated weights for policy 1, policy_version 161282 (0.0009) [2023-12-26 16:32:10,042][105692] Updated weights for policy 0, policy_version 160475 (0.0008) [2023-12-26 16:32:10,101][105692] Updated weights for policy 0, policy_version 160485 (0.0009) [2023-12-26 16:32:10,161][105692] Updated weights for policy 0, policy_version 160495 (0.0007) [2023-12-26 16:32:10,272][105620] Updated weights for policy 1, policy_version 161292 (0.0007) [2023-12-26 16:32:10,337][105620] Updated weights for policy 1, policy_version 161302 (0.0008) [2023-12-26 16:32:10,403][105620] Updated weights for policy 1, policy_version 161312 (0.0009) [2023-12-26 16:32:10,978][105692] Updated weights for policy 0, policy_version 160505 (0.0009) [2023-12-26 16:32:11,038][105692] Updated weights for policy 0, policy_version 160515 (0.0009) [2023-12-26 16:32:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 82403328. Throughput: 0: 9929.1, 1: 9815.0. Samples: 82417932. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:32:11,062][104569] Avg episode reward: [(0, '8311.333'), (1, '7686.529')] [2023-12-26 16:32:11,079][105620] Updated weights for policy 1, policy_version 161322 (0.0008) [2023-12-26 16:32:11,106][105692] Updated weights for policy 0, policy_version 160525 (0.0009) [2023-12-26 16:32:11,171][105692] Updated weights for policy 0, policy_version 160535 (0.0007) [2023-12-26 16:32:11,174][105620] Updated weights for policy 1, policy_version 161332 (0.0006) [2023-12-26 16:32:11,228][105620] Updated weights for policy 1, policy_version 161342 (0.0008) [2023-12-26 16:32:11,296][105620] Updated weights for policy 1, policy_version 161352 (0.0009) [2023-12-26 16:32:11,941][105692] Updated weights for policy 0, policy_version 160545 (0.0008) [2023-12-26 16:32:12,003][105692] Updated weights for policy 0, policy_version 160555 (0.0009) [2023-12-26 16:32:12,064][105692] Updated weights for policy 0, policy_version 160565 (0.0008) [2023-12-26 16:32:12,081][105620] Updated weights for policy 1, policy_version 161362 (0.0009) [2023-12-26 16:32:12,134][105620] Updated weights for policy 1, policy_version 161372 (0.0010) [2023-12-26 16:32:12,191][105620] Updated weights for policy 1, policy_version 161382 (0.0011) [2023-12-26 16:32:12,840][105620] Updated weights for policy 1, policy_version 161392 (0.0006) [2023-12-26 16:32:12,899][105620] Updated weights for policy 1, policy_version 161402 (0.0005) [2023-12-26 16:32:12,909][105692] Updated weights for policy 0, policy_version 160575 (0.0009) [2023-12-26 16:32:12,956][105620] Updated weights for policy 1, policy_version 161412 (0.0005) [2023-12-26 16:32:12,969][105692] Updated weights for policy 0, policy_version 160585 (0.0008) [2023-12-26 16:32:13,027][105692] Updated weights for policy 0, policy_version 160595 (0.0013) [2023-12-26 16:32:13,490][105620] Updated weights for policy 1, policy_version 161422 (0.0006) [2023-12-26 16:32:13,544][105620] Updated weights for policy 1, policy_version 161432 (0.0008) [2023-12-26 16:32:13,590][105620] Updated weights for policy 1, policy_version 161442 (0.0009) [2023-12-26 16:32:13,883][105692] Updated weights for policy 0, policy_version 160606 (0.0009) [2023-12-26 16:32:13,935][105692] Updated weights for policy 0, policy_version 160616 (0.0010) [2023-12-26 16:32:13,995][105692] Updated weights for policy 0, policy_version 160627 (0.0011) [2023-12-26 16:32:14,285][105620] Updated weights for policy 1, policy_version 161452 (0.0007) [2023-12-26 16:32:14,340][105620] Updated weights for policy 1, policy_version 161462 (0.0005) [2023-12-26 16:32:14,402][105620] Updated weights for policy 1, policy_version 161472 (0.0008) [2023-12-26 16:32:14,821][105692] Updated weights for policy 0, policy_version 160637 (0.0009) [2023-12-26 16:32:14,883][105692] Updated weights for policy 0, policy_version 160647 (0.0008) [2023-12-26 16:32:14,949][105692] Updated weights for policy 0, policy_version 160657 (0.0008) [2023-12-26 16:32:15,098][105620] Updated weights for policy 1, policy_version 161482 (0.0009) [2023-12-26 16:32:15,165][105620] Updated weights for policy 1, policy_version 161492 (0.0011) [2023-12-26 16:32:15,233][105620] Updated weights for policy 1, policy_version 161502 (0.0011) [2023-12-26 16:32:15,303][105620] Updated weights for policy 1, policy_version 161512 (0.0009) [2023-12-26 16:32:15,593][105692] Updated weights for policy 0, policy_version 160667 (0.0007) [2023-12-26 16:32:15,644][105692] Updated weights for policy 0, policy_version 160677 (0.0005) [2023-12-26 16:32:15,695][105692] Updated weights for policy 0, policy_version 160687 (0.0005) [2023-12-26 16:32:16,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 82501632. Throughput: 0: 9768.8, 1: 9812.6. Samples: 82474196. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:32:16,063][104569] Avg episode reward: [(0, '8638.102'), (1, '8342.209')] [2023-12-26 16:32:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000160696_41148416.pth... [2023-12-26 16:32:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000161512_41353216.pth... [2023-12-26 16:32:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000160360_41058304.pth [2023-12-26 16:32:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000159576_40861696.pth [2023-12-26 16:32:16,172][105620] Updated weights for policy 1, policy_version 161522 (0.0007) [2023-12-26 16:32:16,238][105620] Updated weights for policy 1, policy_version 161532 (0.0010) [2023-12-26 16:32:16,276][105692] Updated weights for policy 0, policy_version 160697 (0.0005) [2023-12-26 16:32:16,299][105620] Updated weights for policy 1, policy_version 161542 (0.0010) [2023-12-26 16:32:16,341][105692] Updated weights for policy 0, policy_version 160707 (0.0009) [2023-12-26 16:32:16,403][105692] Updated weights for policy 0, policy_version 160717 (0.0008) [2023-12-26 16:32:16,468][105692] Updated weights for policy 0, policy_version 160727 (0.0009) [2023-12-26 16:32:16,979][105620] Updated weights for policy 1, policy_version 161552 (0.0006) [2023-12-26 16:32:17,037][105620] Updated weights for policy 1, policy_version 161562 (0.0005) [2023-12-26 16:32:17,104][105620] Updated weights for policy 1, policy_version 161572 (0.0009) [2023-12-26 16:32:17,218][105692] Updated weights for policy 0, policy_version 160737 (0.0010) [2023-12-26 16:32:17,279][105692] Updated weights for policy 0, policy_version 160747 (0.0010) [2023-12-26 16:32:17,336][105692] Updated weights for policy 0, policy_version 160757 (0.0010) [2023-12-26 16:32:17,785][105620] Updated weights for policy 1, policy_version 161582 (0.0006) [2023-12-26 16:32:17,835][105620] Updated weights for policy 1, policy_version 161592 (0.0005) [2023-12-26 16:32:17,884][105620] Updated weights for policy 1, policy_version 161602 (0.0005) [2023-12-26 16:32:17,942][105692] Updated weights for policy 0, policy_version 160767 (0.0010) [2023-12-26 16:32:17,987][105692] Updated weights for policy 0, policy_version 160777 (0.0010) [2023-12-26 16:32:18,042][105692] Updated weights for policy 0, policy_version 160787 (0.0009) [2023-12-26 16:32:18,431][105620] Updated weights for policy 1, policy_version 161612 (0.0007) [2023-12-26 16:32:18,486][105620] Updated weights for policy 1, policy_version 161622 (0.0010) [2023-12-26 16:32:18,548][105620] Updated weights for policy 1, policy_version 161632 (0.0010) [2023-12-26 16:32:18,800][105692] Updated weights for policy 0, policy_version 160797 (0.0010) [2023-12-26 16:32:18,850][105692] Updated weights for policy 0, policy_version 160807 (0.0011) [2023-12-26 16:32:18,896][105692] Updated weights for policy 0, policy_version 160817 (0.0011) [2023-12-26 16:32:19,304][105620] Updated weights for policy 1, policy_version 161642 (0.0010) [2023-12-26 16:32:19,363][105620] Updated weights for policy 1, policy_version 161652 (0.0010) [2023-12-26 16:32:19,418][105620] Updated weights for policy 1, policy_version 161662 (0.0008) [2023-12-26 16:32:19,475][105620] Updated weights for policy 1, policy_version 161672 (0.0008) [2023-12-26 16:32:19,709][105692] Updated weights for policy 0, policy_version 160827 (0.0011) [2023-12-26 16:32:19,777][105692] Updated weights for policy 0, policy_version 160837 (0.0008) [2023-12-26 16:32:19,840][105692] Updated weights for policy 0, policy_version 160847 (0.0010) [2023-12-26 16:32:20,206][105620] Updated weights for policy 1, policy_version 161682 (0.0010) [2023-12-26 16:32:20,266][105620] Updated weights for policy 1, policy_version 161692 (0.0007) [2023-12-26 16:32:20,327][105620] Updated weights for policy 1, policy_version 161702 (0.0008) [2023-12-26 16:32:20,553][105692] Updated weights for policy 0, policy_version 160857 (0.0010) [2023-12-26 16:32:20,615][105692] Updated weights for policy 0, policy_version 160867 (0.0009) [2023-12-26 16:32:20,672][105692] Updated weights for policy 0, policy_version 160877 (0.0010) [2023-12-26 16:32:20,733][105692] Updated weights for policy 0, policy_version 160887 (0.0009) [2023-12-26 16:32:21,023][105620] Updated weights for policy 1, policy_version 161712 (0.0008) [2023-12-26 16:32:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 82599936. Throughput: 0: 9686.9, 1: 9820.9. Samples: 82591952. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:32:21,062][104569] Avg episode reward: [(0, '8372.550'), (1, '9259.392')] [2023-12-26 16:32:21,092][105620] Updated weights for policy 1, policy_version 161722 (0.0007) [2023-12-26 16:32:21,165][105620] Updated weights for policy 1, policy_version 161732 (0.0009) [2023-12-26 16:32:21,514][105692] Updated weights for policy 0, policy_version 160897 (0.0006) [2023-12-26 16:32:21,579][105692] Updated weights for policy 0, policy_version 160907 (0.0008) [2023-12-26 16:32:21,649][105692] Updated weights for policy 0, policy_version 160917 (0.0009) [2023-12-26 16:32:21,856][105620] Updated weights for policy 1, policy_version 161742 (0.0007) [2023-12-26 16:32:21,919][105620] Updated weights for policy 1, policy_version 161752 (0.0008) [2023-12-26 16:32:21,978][105620] Updated weights for policy 1, policy_version 161762 (0.0009) [2023-12-26 16:32:22,407][105692] Updated weights for policy 0, policy_version 160927 (0.0009) [2023-12-26 16:32:22,469][105692] Updated weights for policy 0, policy_version 160937 (0.0008) [2023-12-26 16:32:22,529][105692] Updated weights for policy 0, policy_version 160947 (0.0009) [2023-12-26 16:32:22,721][105620] Updated weights for policy 1, policy_version 161772 (0.0009) [2023-12-26 16:32:22,786][105620] Updated weights for policy 1, policy_version 161782 (0.0009) [2023-12-26 16:32:22,849][105620] Updated weights for policy 1, policy_version 161792 (0.0009) [2023-12-26 16:32:23,271][105692] Updated weights for policy 0, policy_version 160957 (0.0009) [2023-12-26 16:32:23,334][105692] Updated weights for policy 0, policy_version 160967 (0.0009) [2023-12-26 16:32:23,385][105692] Updated weights for policy 0, policy_version 160977 (0.0009) [2023-12-26 16:32:23,636][105620] Updated weights for policy 1, policy_version 161802 (0.0009) [2023-12-26 16:32:23,698][105620] Updated weights for policy 1, policy_version 161812 (0.0009) [2023-12-26 16:32:23,753][105620] Updated weights for policy 1, policy_version 161822 (0.0009) [2023-12-26 16:32:23,805][105620] Updated weights for policy 1, policy_version 161832 (0.0009) [2023-12-26 16:32:24,127][105692] Updated weights for policy 0, policy_version 160987 (0.0009) [2023-12-26 16:32:24,192][105692] Updated weights for policy 0, policy_version 160997 (0.0006) [2023-12-26 16:32:24,259][105692] Updated weights for policy 0, policy_version 161007 (0.0006) [2023-12-26 16:32:24,566][105620] Updated weights for policy 1, policy_version 161842 (0.0005) [2023-12-26 16:32:24,635][105620] Updated weights for policy 1, policy_version 161852 (0.0005) [2023-12-26 16:32:24,703][105620] Updated weights for policy 1, policy_version 161862 (0.0005) [2023-12-26 16:32:24,976][105692] Updated weights for policy 0, policy_version 161017 (0.0006) [2023-12-26 16:32:25,035][105692] Updated weights for policy 0, policy_version 161027 (0.0005) [2023-12-26 16:32:25,101][105692] Updated weights for policy 0, policy_version 161037 (0.0005) [2023-12-26 16:32:25,173][105692] Updated weights for policy 0, policy_version 161047 (0.0005) [2023-12-26 16:32:25,336][105620] Updated weights for policy 1, policy_version 161872 (0.0009) [2023-12-26 16:32:25,395][105620] Updated weights for policy 1, policy_version 161882 (0.0010) [2023-12-26 16:32:25,456][105620] Updated weights for policy 1, policy_version 161892 (0.0009) [2023-12-26 16:32:25,742][105692] Updated weights for policy 0, policy_version 161057 (0.0005) [2023-12-26 16:32:25,793][105692] Updated weights for policy 0, policy_version 161067 (0.0005) [2023-12-26 16:32:25,839][105692] Updated weights for policy 0, policy_version 161077 (0.0005) [2023-12-26 16:32:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 82698240. Throughput: 0: 9678.6, 1: 9867.7. Samples: 82707680. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:32:26,062][104569] Avg episode reward: [(0, '8909.702'), (1, '9170.627')] [2023-12-26 16:32:26,082][105620] Updated weights for policy 1, policy_version 161902 (0.0010) [2023-12-26 16:32:26,144][105620] Updated weights for policy 1, policy_version 161912 (0.0006) [2023-12-26 16:32:26,197][105620] Updated weights for policy 1, policy_version 161922 (0.0005) [2023-12-26 16:32:26,392][105692] Updated weights for policy 0, policy_version 161087 (0.0005) [2023-12-26 16:32:26,440][105692] Updated weights for policy 0, policy_version 161097 (0.0007) [2023-12-26 16:32:26,488][105692] Updated weights for policy 0, policy_version 161107 (0.0010) [2023-12-26 16:32:26,771][105620] Updated weights for policy 1, policy_version 161932 (0.0007) [2023-12-26 16:32:26,829][105620] Updated weights for policy 1, policy_version 161942 (0.0010) [2023-12-26 16:32:26,889][105620] Updated weights for policy 1, policy_version 161952 (0.0010) [2023-12-26 16:32:27,122][105692] Updated weights for policy 0, policy_version 161117 (0.0008) [2023-12-26 16:32:27,176][105692] Updated weights for policy 0, policy_version 161127 (0.0007) [2023-12-26 16:32:27,234][105692] Updated weights for policy 0, policy_version 161137 (0.0008) [2023-12-26 16:32:27,554][105620] Updated weights for policy 1, policy_version 161962 (0.0009) [2023-12-26 16:32:27,616][105620] Updated weights for policy 1, policy_version 161972 (0.0005) [2023-12-26 16:32:27,674][105620] Updated weights for policy 1, policy_version 161982 (0.0005) [2023-12-26 16:32:27,730][105620] Updated weights for policy 1, policy_version 161992 (0.0005) [2023-12-26 16:32:27,976][105692] Updated weights for policy 0, policy_version 161147 (0.0007) [2023-12-26 16:32:28,037][105692] Updated weights for policy 0, policy_version 161157 (0.0005) [2023-12-26 16:32:28,085][105692] Updated weights for policy 0, policy_version 161167 (0.0005) [2023-12-26 16:32:28,258][105620] Updated weights for policy 1, policy_version 162002 (0.0010) [2023-12-26 16:32:28,312][105620] Updated weights for policy 1, policy_version 162012 (0.0010) [2023-12-26 16:32:28,379][105620] Updated weights for policy 1, policy_version 162022 (0.0010) [2023-12-26 16:32:28,795][105692] Updated weights for policy 0, policy_version 161177 (0.0005) [2023-12-26 16:32:28,843][105692] Updated weights for policy 0, policy_version 161187 (0.0007) [2023-12-26 16:32:28,900][105692] Updated weights for policy 0, policy_version 161197 (0.0006) [2023-12-26 16:32:28,947][105692] Updated weights for policy 0, policy_version 161207 (0.0008) [2023-12-26 16:32:28,983][105620] Updated weights for policy 1, policy_version 162032 (0.0009) [2023-12-26 16:32:29,039][105620] Updated weights for policy 1, policy_version 162042 (0.0009) [2023-12-26 16:32:29,101][105620] Updated weights for policy 1, policy_version 162052 (0.0011) [2023-12-26 16:32:29,632][105692] Updated weights for policy 0, policy_version 161217 (0.0009) [2023-12-26 16:32:29,684][105692] Updated weights for policy 0, policy_version 161227 (0.0009) [2023-12-26 16:32:29,741][105692] Updated weights for policy 0, policy_version 161237 (0.0010) [2023-12-26 16:32:29,808][105620] Updated weights for policy 1, policy_version 162062 (0.0007) [2023-12-26 16:32:29,876][105620] Updated weights for policy 1, policy_version 162072 (0.0007) [2023-12-26 16:32:29,943][105620] Updated weights for policy 1, policy_version 162082 (0.0007) [2023-12-26 16:32:30,580][105692] Updated weights for policy 0, policy_version 161247 (0.0008) [2023-12-26 16:32:30,582][105620] Updated weights for policy 1, policy_version 162092 (0.0009) [2023-12-26 16:32:30,631][105620] Updated weights for policy 1, policy_version 162102 (0.0006) [2023-12-26 16:32:30,636][105692] Updated weights for policy 0, policy_version 161257 (0.0007) [2023-12-26 16:32:30,679][105620] Updated weights for policy 1, policy_version 162112 (0.0006) [2023-12-26 16:32:30,689][105692] Updated weights for policy 0, policy_version 161267 (0.0009) [2023-12-26 16:32:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 82804736. Throughput: 0: 9719.1, 1: 10019.2. Samples: 82773836. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:32:31,062][104569] Avg episode reward: [(0, '9176.958'), (1, '9254.954')] [2023-12-26 16:32:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000161272_41295872.pth... [2023-12-26 16:32:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000162120_41508864.pth... [2023-12-26 16:32:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000160152_41009152.pth [2023-12-26 16:32:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000160936_41205760.pth [2023-12-26 16:32:31,343][105620] Updated weights for policy 1, policy_version 162122 (0.0006) [2023-12-26 16:32:31,414][105620] Updated weights for policy 1, policy_version 162132 (0.0009) [2023-12-26 16:32:31,470][105620] Updated weights for policy 1, policy_version 162142 (0.0009) [2023-12-26 16:32:31,513][105692] Updated weights for policy 0, policy_version 161277 (0.0007) [2023-12-26 16:32:31,521][105620] Updated weights for policy 1, policy_version 162152 (0.0009) [2023-12-26 16:32:31,582][105692] Updated weights for policy 0, policy_version 161287 (0.0005) [2023-12-26 16:32:31,652][105692] Updated weights for policy 0, policy_version 161297 (0.0006) [2023-12-26 16:32:32,189][105620] Updated weights for policy 1, policy_version 162162 (0.0005) [2023-12-26 16:32:32,237][105620] Updated weights for policy 1, policy_version 162172 (0.0005) [2023-12-26 16:32:32,284][105692] Updated weights for policy 0, policy_version 161307 (0.0007) [2023-12-26 16:32:32,294][105620] Updated weights for policy 1, policy_version 162182 (0.0007) [2023-12-26 16:32:32,346][105692] Updated weights for policy 0, policy_version 161317 (0.0008) [2023-12-26 16:32:32,404][105692] Updated weights for policy 0, policy_version 161327 (0.0008) [2023-12-26 16:32:32,874][105620] Updated weights for policy 1, policy_version 162192 (0.0005) [2023-12-26 16:32:32,924][105620] Updated weights for policy 1, policy_version 162202 (0.0005) [2023-12-26 16:32:32,984][105620] Updated weights for policy 1, policy_version 162212 (0.0005) [2023-12-26 16:32:33,152][105692] Updated weights for policy 0, policy_version 161337 (0.0009) [2023-12-26 16:32:33,211][105692] Updated weights for policy 0, policy_version 161347 (0.0009) [2023-12-26 16:32:33,271][105692] Updated weights for policy 0, policy_version 161357 (0.0008) [2023-12-26 16:32:33,327][105692] Updated weights for policy 0, policy_version 161367 (0.0009) [2023-12-26 16:32:33,597][105620] Updated weights for policy 1, policy_version 162222 (0.0005) [2023-12-26 16:32:33,652][105620] Updated weights for policy 1, policy_version 162232 (0.0005) [2023-12-26 16:32:33,702][105620] Updated weights for policy 1, policy_version 162242 (0.0005) [2023-12-26 16:32:33,991][105692] Updated weights for policy 0, policy_version 161377 (0.0009) [2023-12-26 16:32:34,036][105692] Updated weights for policy 0, policy_version 161387 (0.0005) [2023-12-26 16:32:34,096][105692] Updated weights for policy 0, policy_version 161397 (0.0005) [2023-12-26 16:32:34,256][105620] Updated weights for policy 1, policy_version 162252 (0.0007) [2023-12-26 16:32:34,322][105620] Updated weights for policy 1, policy_version 162262 (0.0009) [2023-12-26 16:32:34,385][105620] Updated weights for policy 1, policy_version 162272 (0.0007) [2023-12-26 16:32:34,798][105692] Updated weights for policy 0, policy_version 161407 (0.0008) [2023-12-26 16:32:34,855][105692] Updated weights for policy 0, policy_version 161417 (0.0009) [2023-12-26 16:32:34,911][105692] Updated weights for policy 0, policy_version 161427 (0.0009) [2023-12-26 16:32:35,047][105620] Updated weights for policy 1, policy_version 162282 (0.0006) [2023-12-26 16:32:35,106][105620] Updated weights for policy 1, policy_version 162292 (0.0006) [2023-12-26 16:32:35,162][105620] Updated weights for policy 1, policy_version 162302 (0.0009) [2023-12-26 16:32:35,229][105620] Updated weights for policy 1, policy_version 162312 (0.0007) [2023-12-26 16:32:35,634][105692] Updated weights for policy 0, policy_version 161437 (0.0008) [2023-12-26 16:32:35,680][105692] Updated weights for policy 0, policy_version 161447 (0.0005) [2023-12-26 16:32:35,744][105692] Updated weights for policy 0, policy_version 161457 (0.0006) [2023-12-26 16:32:35,835][105620] Updated weights for policy 1, policy_version 162322 (0.0005) [2023-12-26 16:32:35,881][105620] Updated weights for policy 1, policy_version 162332 (0.0005) [2023-12-26 16:32:35,944][105620] Updated weights for policy 1, policy_version 162342 (0.0005) [2023-12-26 16:32:36,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 82911232. Throughput: 0: 9581.2, 1: 10142.0. Samples: 82895340. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 16:32:36,063][104569] Avg episode reward: [(0, '9265.787'), (1, '9341.314')] [2023-12-26 16:32:36,308][105692] Updated weights for policy 0, policy_version 161467 (0.0007) [2023-12-26 16:32:36,365][105692] Updated weights for policy 0, policy_version 161477 (0.0008) [2023-12-26 16:32:36,421][105692] Updated weights for policy 0, policy_version 161487 (0.0010) [2023-12-26 16:32:36,672][105620] Updated weights for policy 1, policy_version 162352 (0.0008) [2023-12-26 16:32:36,735][105620] Updated weights for policy 1, policy_version 162362 (0.0008) [2023-12-26 16:32:36,791][105620] Updated weights for policy 1, policy_version 162372 (0.0008) [2023-12-26 16:32:37,080][105692] Updated weights for policy 0, policy_version 161497 (0.0010) [2023-12-26 16:32:37,138][105692] Updated weights for policy 0, policy_version 161507 (0.0009) [2023-12-26 16:32:37,202][105692] Updated weights for policy 0, policy_version 161517 (0.0009) [2023-12-26 16:32:37,260][105692] Updated weights for policy 0, policy_version 161527 (0.0009) [2023-12-26 16:32:37,457][105620] Updated weights for policy 1, policy_version 162382 (0.0009) [2023-12-26 16:32:37,512][105620] Updated weights for policy 1, policy_version 162392 (0.0009) [2023-12-26 16:32:37,574][105620] Updated weights for policy 1, policy_version 162402 (0.0008) [2023-12-26 16:32:38,069][105692] Updated weights for policy 0, policy_version 161537 (0.0008) [2023-12-26 16:32:38,125][105692] Updated weights for policy 0, policy_version 161547 (0.0008) [2023-12-26 16:32:38,176][105692] Updated weights for policy 0, policy_version 161557 (0.0008) [2023-12-26 16:32:38,238][105620] Updated weights for policy 1, policy_version 162412 (0.0010) [2023-12-26 16:32:38,283][105620] Updated weights for policy 1, policy_version 162422 (0.0010) [2023-12-26 16:32:38,342][105620] Updated weights for policy 1, policy_version 162432 (0.0010) [2023-12-26 16:32:38,910][105692] Updated weights for policy 0, policy_version 161567 (0.0008) [2023-12-26 16:32:38,965][105692] Updated weights for policy 0, policy_version 161577 (0.0008) [2023-12-26 16:32:39,013][105692] Updated weights for policy 0, policy_version 161587 (0.0008) [2023-12-26 16:32:39,108][105620] Updated weights for policy 1, policy_version 162442 (0.0010) [2023-12-26 16:32:39,164][105620] Updated weights for policy 1, policy_version 162452 (0.0006) [2023-12-26 16:32:39,226][105620] Updated weights for policy 1, policy_version 162462 (0.0006) [2023-12-26 16:32:39,285][105620] Updated weights for policy 1, policy_version 162472 (0.0009) [2023-12-26 16:32:39,793][105692] Updated weights for policy 0, policy_version 161597 (0.0009) [2023-12-26 16:32:39,858][105692] Updated weights for policy 0, policy_version 161607 (0.0009) [2023-12-26 16:32:39,924][105692] Updated weights for policy 0, policy_version 161617 (0.0008) [2023-12-26 16:32:40,093][105620] Updated weights for policy 1, policy_version 162482 (0.0009) [2023-12-26 16:32:40,142][105620] Updated weights for policy 1, policy_version 162492 (0.0008) [2023-12-26 16:32:40,193][105620] Updated weights for policy 1, policy_version 162502 (0.0009) [2023-12-26 16:32:40,732][105692] Updated weights for policy 0, policy_version 161627 (0.0009) [2023-12-26 16:32:40,780][105692] Updated weights for policy 0, policy_version 161637 (0.0009) [2023-12-26 16:32:40,827][105692] Updated weights for policy 0, policy_version 161647 (0.0005) [2023-12-26 16:32:40,847][105620] Updated weights for policy 1, policy_version 162512 (0.0008) [2023-12-26 16:32:40,902][105620] Updated weights for policy 1, policy_version 162522 (0.0007) [2023-12-26 16:32:40,955][105620] Updated weights for policy 1, policy_version 162532 (0.0009) [2023-12-26 16:32:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 83009536. Throughput: 0: 9610.5, 1: 10188.6. Samples: 83012928. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 16:32:41,062][104569] Avg episode reward: [(0, '9262.506'), (1, '9340.885')] [2023-12-26 16:32:41,614][105692] Updated weights for policy 0, policy_version 161657 (0.0006) [2023-12-26 16:32:41,675][105692] Updated weights for policy 0, policy_version 161667 (0.0012) [2023-12-26 16:32:41,728][105620] Updated weights for policy 1, policy_version 162542 (0.0008) [2023-12-26 16:32:41,740][105692] Updated weights for policy 0, policy_version 161677 (0.0010) [2023-12-26 16:32:41,798][105620] Updated weights for policy 1, policy_version 162552 (0.0008) [2023-12-26 16:32:41,806][105692] Updated weights for policy 0, policy_version 161687 (0.0011) [2023-12-26 16:32:41,864][105620] Updated weights for policy 1, policy_version 162562 (0.0008) [2023-12-26 16:32:42,534][105692] Updated weights for policy 0, policy_version 161697 (0.0009) [2023-12-26 16:32:42,595][105692] Updated weights for policy 0, policy_version 161707 (0.0009) [2023-12-26 16:32:42,647][105620] Updated weights for policy 1, policy_version 162572 (0.0008) [2023-12-26 16:32:42,652][105692] Updated weights for policy 0, policy_version 161717 (0.0009) [2023-12-26 16:32:42,703][105620] Updated weights for policy 1, policy_version 162582 (0.0009) [2023-12-26 16:32:42,759][105620] Updated weights for policy 1, policy_version 162592 (0.0009) [2023-12-26 16:32:43,381][105692] Updated weights for policy 0, policy_version 161727 (0.0010) [2023-12-26 16:32:43,438][105692] Updated weights for policy 0, policy_version 161737 (0.0010) [2023-12-26 16:32:43,459][105620] Updated weights for policy 1, policy_version 162602 (0.0009) [2023-12-26 16:32:43,490][105692] Updated weights for policy 0, policy_version 161747 (0.0010) [2023-12-26 16:32:43,517][105620] Updated weights for policy 1, policy_version 162612 (0.0010) [2023-12-26 16:32:43,567][105620] Updated weights for policy 1, policy_version 162622 (0.0010) [2023-12-26 16:32:43,614][105620] Updated weights for policy 1, policy_version 162632 (0.0010) [2023-12-26 16:32:44,207][105692] Updated weights for policy 0, policy_version 161757 (0.0008) [2023-12-26 16:32:44,274][105692] Updated weights for policy 0, policy_version 161767 (0.0005) [2023-12-26 16:32:44,341][105692] Updated weights for policy 0, policy_version 161777 (0.0005) [2023-12-26 16:32:44,350][105620] Updated weights for policy 1, policy_version 162642 (0.0009) [2023-12-26 16:32:44,408][105620] Updated weights for policy 1, policy_version 162652 (0.0009) [2023-12-26 16:32:44,459][105620] Updated weights for policy 1, policy_version 162662 (0.0010) [2023-12-26 16:32:45,039][105692] Updated weights for policy 0, policy_version 161787 (0.0006) [2023-12-26 16:32:45,103][105692] Updated weights for policy 0, policy_version 161797 (0.0009) [2023-12-26 16:32:45,158][105692] Updated weights for policy 0, policy_version 161807 (0.0008) [2023-12-26 16:32:45,208][105620] Updated weights for policy 1, policy_version 162672 (0.0010) [2023-12-26 16:32:45,274][105620] Updated weights for policy 1, policy_version 162682 (0.0009) [2023-12-26 16:32:45,336][105620] Updated weights for policy 1, policy_version 162692 (0.0010) [2023-12-26 16:32:45,938][105692] Updated weights for policy 0, policy_version 161817 (0.0007) [2023-12-26 16:32:45,990][105692] Updated weights for policy 0, policy_version 161827 (0.0008) [2023-12-26 16:32:46,045][105692] Updated weights for policy 0, policy_version 161837 (0.0008) [2023-12-26 16:32:46,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 83091456. Throughput: 0: 9660.2, 1: 10070.2. Samples: 83068428. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 16:32:46,062][104569] Avg episode reward: [(0, '9351.119'), (1, '9342.805')] [2023-12-26 16:32:46,078][105620] Updated weights for policy 1, policy_version 162702 (0.0010) [2023-12-26 16:32:46,102][105692] Updated weights for policy 0, policy_version 161847 (0.0008) [2023-12-26 16:32:46,107][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000161848_41443328.pth... [2023-12-26 16:32:46,112][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000160696_41148416.pth [2023-12-26 16:32:46,142][105620] Updated weights for policy 1, policy_version 162712 (0.0011) [2023-12-26 16:32:46,197][105620] Updated weights for policy 1, policy_version 162722 (0.0010) [2023-12-26 16:32:46,231][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000162728_41664512.pth... [2023-12-26 16:32:46,234][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000161512_41353216.pth [2023-12-26 16:32:46,873][105692] Updated weights for policy 0, policy_version 161857 (0.0008) [2023-12-26 16:32:46,929][105692] Updated weights for policy 0, policy_version 161867 (0.0008) [2023-12-26 16:32:46,944][105620] Updated weights for policy 1, policy_version 162732 (0.0010) [2023-12-26 16:32:46,990][105692] Updated weights for policy 0, policy_version 161877 (0.0006) [2023-12-26 16:32:46,999][105620] Updated weights for policy 1, policy_version 162742 (0.0010) [2023-12-26 16:32:47,054][105620] Updated weights for policy 1, policy_version 162752 (0.0010) [2023-12-26 16:32:47,749][105692] Updated weights for policy 0, policy_version 161887 (0.0007) [2023-12-26 16:32:47,798][105620] Updated weights for policy 1, policy_version 162762 (0.0010) [2023-12-26 16:32:47,800][105692] Updated weights for policy 0, policy_version 161897 (0.0007) [2023-12-26 16:32:47,852][105620] Updated weights for policy 1, policy_version 162772 (0.0010) [2023-12-26 16:32:47,855][105692] Updated weights for policy 0, policy_version 161907 (0.0005) [2023-12-26 16:32:47,909][105620] Updated weights for policy 1, policy_version 162782 (0.0010) [2023-12-26 16:32:47,966][105620] Updated weights for policy 1, policy_version 162792 (0.0010) [2023-12-26 16:32:48,576][105692] Updated weights for policy 0, policy_version 161917 (0.0007) [2023-12-26 16:32:48,634][105692] Updated weights for policy 0, policy_version 161927 (0.0008) [2023-12-26 16:32:48,697][105692] Updated weights for policy 0, policy_version 161937 (0.0011) [2023-12-26 16:32:48,700][105620] Updated weights for policy 1, policy_version 162802 (0.0010) [2023-12-26 16:32:48,756][105620] Updated weights for policy 1, policy_version 162812 (0.0010) [2023-12-26 16:32:48,821][105620] Updated weights for policy 1, policy_version 162822 (0.0010) [2023-12-26 16:32:49,368][105692] Updated weights for policy 0, policy_version 161947 (0.0010) [2023-12-26 16:32:49,424][105692] Updated weights for policy 0, policy_version 161957 (0.0009) [2023-12-26 16:32:49,503][105692] Updated weights for policy 0, policy_version 161967 (0.0009) [2023-12-26 16:32:49,595][105620] Updated weights for policy 1, policy_version 162832 (0.0010) [2023-12-26 16:32:49,657][105620] Updated weights for policy 1, policy_version 162842 (0.0011) [2023-12-26 16:32:49,710][105620] Updated weights for policy 1, policy_version 162852 (0.0010) [2023-12-26 16:32:50,259][105692] Updated weights for policy 0, policy_version 161977 (0.0009) [2023-12-26 16:32:50,315][105692] Updated weights for policy 0, policy_version 161987 (0.0011) [2023-12-26 16:32:50,374][105620] Updated weights for policy 1, policy_version 162862 (0.0008) [2023-12-26 16:32:50,376][105692] Updated weights for policy 0, policy_version 161997 (0.0011) [2023-12-26 16:32:50,440][105692] Updated weights for policy 0, policy_version 162007 (0.0011) [2023-12-26 16:32:50,440][105620] Updated weights for policy 1, policy_version 162872 (0.0008) [2023-12-26 16:32:50,500][105620] Updated weights for policy 1, policy_version 162882 (0.0008) [2023-12-26 16:32:51,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 83189760. Throughput: 0: 9586.8, 1: 10036.5. Samples: 83181160. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 16:32:51,063][104569] Avg episode reward: [(0, '9351.514'), (1, '9344.139')] [2023-12-26 16:32:51,226][105620] Updated weights for policy 1, policy_version 162892 (0.0007) [2023-12-26 16:32:51,236][105692] Updated weights for policy 0, policy_version 162017 (0.0009) [2023-12-26 16:32:51,290][105620] Updated weights for policy 1, policy_version 162902 (0.0006) [2023-12-26 16:32:51,299][105692] Updated weights for policy 0, policy_version 162027 (0.0011) [2023-12-26 16:32:51,356][105620] Updated weights for policy 1, policy_version 162912 (0.0006) [2023-12-26 16:32:51,365][105692] Updated weights for policy 0, policy_version 162037 (0.0011) [2023-12-26 16:32:52,030][105620] Updated weights for policy 1, policy_version 162922 (0.0008) [2023-12-26 16:32:52,091][105620] Updated weights for policy 1, policy_version 162932 (0.0009) [2023-12-26 16:32:52,142][105620] Updated weights for policy 1, policy_version 162942 (0.0007) [2023-12-26 16:32:52,151][105692] Updated weights for policy 0, policy_version 162047 (0.0011) [2023-12-26 16:32:52,193][105620] Updated weights for policy 1, policy_version 162952 (0.0008) [2023-12-26 16:32:52,207][105692] Updated weights for policy 0, policy_version 162057 (0.0009) [2023-12-26 16:32:52,260][105692] Updated weights for policy 0, policy_version 162067 (0.0009) [2023-12-26 16:32:52,858][105620] Updated weights for policy 1, policy_version 162962 (0.0009) [2023-12-26 16:32:52,921][105620] Updated weights for policy 1, policy_version 162972 (0.0009) [2023-12-26 16:32:52,984][105620] Updated weights for policy 1, policy_version 162982 (0.0009) [2023-12-26 16:32:53,014][105692] Updated weights for policy 0, policy_version 162077 (0.0009) [2023-12-26 16:32:53,067][105692] Updated weights for policy 0, policy_version 162087 (0.0008) [2023-12-26 16:32:53,125][105692] Updated weights for policy 0, policy_version 162097 (0.0009) [2023-12-26 16:32:53,680][105620] Updated weights for policy 1, policy_version 162992 (0.0009) [2023-12-26 16:32:53,738][105620] Updated weights for policy 1, policy_version 163002 (0.0009) [2023-12-26 16:32:53,794][105620] Updated weights for policy 1, policy_version 163012 (0.0009) [2023-12-26 16:32:53,888][105692] Updated weights for policy 0, policy_version 162107 (0.0009) [2023-12-26 16:32:53,944][105692] Updated weights for policy 0, policy_version 162117 (0.0009) [2023-12-26 16:32:54,002][105692] Updated weights for policy 0, policy_version 162127 (0.0009) [2023-12-26 16:32:54,512][105620] Updated weights for policy 1, policy_version 163022 (0.0008) [2023-12-26 16:32:54,577][105620] Updated weights for policy 1, policy_version 163032 (0.0009) [2023-12-26 16:32:54,633][105620] Updated weights for policy 1, policy_version 163042 (0.0008) [2023-12-26 16:32:54,808][105692] Updated weights for policy 0, policy_version 162137 (0.0009) [2023-12-26 16:32:54,854][105692] Updated weights for policy 0, policy_version 162147 (0.0008) [2023-12-26 16:32:54,901][105692] Updated weights for policy 0, policy_version 162157 (0.0009) [2023-12-26 16:32:54,958][105692] Updated weights for policy 0, policy_version 162167 (0.0010) [2023-12-26 16:32:55,256][105620] Updated weights for policy 1, policy_version 163052 (0.0007) [2023-12-26 16:32:55,311][105620] Updated weights for policy 1, policy_version 163062 (0.0005) [2023-12-26 16:32:55,369][105620] Updated weights for policy 1, policy_version 163072 (0.0005) [2023-12-26 16:32:55,798][105692] Updated weights for policy 0, policy_version 162177 (0.0010) [2023-12-26 16:32:55,853][105692] Updated weights for policy 0, policy_version 162187 (0.0009) [2023-12-26 16:32:55,921][105692] Updated weights for policy 0, policy_version 162197 (0.0007) [2023-12-26 16:32:55,965][105620] Updated weights for policy 1, policy_version 163082 (0.0005) [2023-12-26 16:32:56,015][105620] Updated weights for policy 1, policy_version 163092 (0.0005) [2023-12-26 16:32:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 83288064. Throughput: 0: 9527.0, 1: 10036.0. Samples: 83298264. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 16:32:56,062][104569] Avg episode reward: [(0, '9351.760'), (1, '8993.752')] [2023-12-26 16:32:56,063][105620] Updated weights for policy 1, policy_version 163102 (0.0005) [2023-12-26 16:32:56,129][105620] Updated weights for policy 1, policy_version 163112 (0.0005) [2023-12-26 16:32:56,654][105692] Updated weights for policy 0, policy_version 162207 (0.0006) [2023-12-26 16:32:56,704][105692] Updated weights for policy 0, policy_version 162217 (0.0005) [2023-12-26 16:32:56,759][105692] Updated weights for policy 0, policy_version 162227 (0.0005) [2023-12-26 16:32:56,801][105620] Updated weights for policy 1, policy_version 163122 (0.0009) [2023-12-26 16:32:56,863][105620] Updated weights for policy 1, policy_version 163132 (0.0008) [2023-12-26 16:32:56,928][105620] Updated weights for policy 1, policy_version 163142 (0.0009) [2023-12-26 16:32:57,335][105692] Updated weights for policy 0, policy_version 162237 (0.0007) [2023-12-26 16:32:57,389][105692] Updated weights for policy 0, policy_version 162247 (0.0005) [2023-12-26 16:32:57,443][105692] Updated weights for policy 0, policy_version 162257 (0.0005) [2023-12-26 16:32:57,526][105620] Updated weights for policy 1, policy_version 163152 (0.0006) [2023-12-26 16:32:57,577][105620] Updated weights for policy 1, policy_version 163162 (0.0005) [2023-12-26 16:32:57,632][105620] Updated weights for policy 1, policy_version 163172 (0.0006) [2023-12-26 16:32:58,035][105692] Updated weights for policy 0, policy_version 162267 (0.0007) [2023-12-26 16:32:58,094][105692] Updated weights for policy 0, policy_version 162277 (0.0007) [2023-12-26 16:32:58,153][105692] Updated weights for policy 0, policy_version 162287 (0.0009) [2023-12-26 16:32:58,338][105620] Updated weights for policy 1, policy_version 163183 (0.0007) [2023-12-26 16:32:58,404][105620] Updated weights for policy 1, policy_version 163193 (0.0006) [2023-12-26 16:32:58,477][105620] Updated weights for policy 1, policy_version 163203 (0.0008) [2023-12-26 16:32:58,946][105692] Updated weights for policy 0, policy_version 162297 (0.0009) [2023-12-26 16:32:59,005][105692] Updated weights for policy 0, policy_version 162307 (0.0008) [2023-12-26 16:32:59,067][105692] Updated weights for policy 0, policy_version 162317 (0.0008) [2023-12-26 16:32:59,128][105692] Updated weights for policy 0, policy_version 162327 (0.0007) [2023-12-26 16:32:59,242][105620] Updated weights for policy 1, policy_version 163213 (0.0007) [2023-12-26 16:32:59,314][105620] Updated weights for policy 1, policy_version 163223 (0.0006) [2023-12-26 16:32:59,386][105620] Updated weights for policy 1, policy_version 163233 (0.0011) [2023-12-26 16:32:59,765][105692] Updated weights for policy 0, policy_version 162337 (0.0008) [2023-12-26 16:32:59,829][105692] Updated weights for policy 0, policy_version 162347 (0.0008) [2023-12-26 16:32:59,893][105692] Updated weights for policy 0, policy_version 162357 (0.0008) [2023-12-26 16:33:00,065][105620] Updated weights for policy 1, policy_version 163243 (0.0011) [2023-12-26 16:33:00,131][105620] Updated weights for policy 1, policy_version 163253 (0.0011) [2023-12-26 16:33:00,189][105620] Updated weights for policy 1, policy_version 163263 (0.0010) [2023-12-26 16:33:00,543][105692] Updated weights for policy 0, policy_version 162367 (0.0006) [2023-12-26 16:33:00,595][105692] Updated weights for policy 0, policy_version 162377 (0.0005) [2023-12-26 16:33:00,647][105692] Updated weights for policy 0, policy_version 162387 (0.0005) [2023-12-26 16:33:00,913][105620] Updated weights for policy 1, policy_version 163273 (0.0010) [2023-12-26 16:33:00,978][105620] Updated weights for policy 1, policy_version 163283 (0.0005) [2023-12-26 16:33:01,043][105620] Updated weights for policy 1, policy_version 163293 (0.0007) [2023-12-26 16:33:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 83386368. Throughput: 0: 9635.2, 1: 10031.7. Samples: 83359200. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 16:33:01,062][104569] Avg episode reward: [(0, '9351.668'), (1, '8996.688')] [2023-12-26 16:33:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000162392_41582592.pth... [2023-12-26 16:33:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000161272_41295872.pth [2023-12-26 16:33:01,103][105620] Updated weights for policy 1, policy_version 163303 (0.0006) [2023-12-26 16:33:01,109][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000163304_41811968.pth... [2023-12-26 16:33:01,114][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000162120_41508864.pth [2023-12-26 16:33:01,310][105692] Updated weights for policy 0, policy_version 162397 (0.0005) [2023-12-26 16:33:01,378][105692] Updated weights for policy 0, policy_version 162407 (0.0008) [2023-12-26 16:33:01,440][105692] Updated weights for policy 0, policy_version 162417 (0.0009) [2023-12-26 16:33:01,793][105620] Updated weights for policy 1, policy_version 163313 (0.0006) [2023-12-26 16:33:01,863][105620] Updated weights for policy 1, policy_version 163323 (0.0010) [2023-12-26 16:33:01,925][105620] Updated weights for policy 1, policy_version 163333 (0.0009) [2023-12-26 16:33:02,143][105692] Updated weights for policy 0, policy_version 162427 (0.0009) [2023-12-26 16:33:02,191][105692] Updated weights for policy 0, policy_version 162437 (0.0008) [2023-12-26 16:33:02,233][105692] Updated weights for policy 0, policy_version 162447 (0.0005) [2023-12-26 16:33:02,550][105620] Updated weights for policy 1, policy_version 163343 (0.0008) [2023-12-26 16:33:02,602][105620] Updated weights for policy 1, policy_version 163353 (0.0006) [2023-12-26 16:33:02,663][105620] Updated weights for policy 1, policy_version 163363 (0.0005) [2023-12-26 16:33:02,977][105692] Updated weights for policy 0, policy_version 162457 (0.0006) [2023-12-26 16:33:03,032][105692] Updated weights for policy 0, policy_version 162467 (0.0008) [2023-12-26 16:33:03,094][105692] Updated weights for policy 0, policy_version 162477 (0.0008) [2023-12-26 16:33:03,153][105692] Updated weights for policy 0, policy_version 162487 (0.0008) [2023-12-26 16:33:03,334][105620] Updated weights for policy 1, policy_version 163373 (0.0008) [2023-12-26 16:33:03,404][105620] Updated weights for policy 1, policy_version 163383 (0.0010) [2023-12-26 16:33:03,470][105620] Updated weights for policy 1, policy_version 163393 (0.0010) [2023-12-26 16:33:03,881][105692] Updated weights for policy 0, policy_version 162497 (0.0010) [2023-12-26 16:33:03,935][105692] Updated weights for policy 0, policy_version 162507 (0.0010) [2023-12-26 16:33:03,988][105692] Updated weights for policy 0, policy_version 162517 (0.0010) [2023-12-26 16:33:04,076][105620] Updated weights for policy 1, policy_version 163403 (0.0010) [2023-12-26 16:33:04,139][105620] Updated weights for policy 1, policy_version 163413 (0.0011) [2023-12-26 16:33:04,202][105620] Updated weights for policy 1, policy_version 163423 (0.0011) [2023-12-26 16:33:04,675][105692] Updated weights for policy 0, policy_version 162527 (0.0009) [2023-12-26 16:33:04,742][105692] Updated weights for policy 0, policy_version 162537 (0.0008) [2023-12-26 16:33:04,798][105692] Updated weights for policy 0, policy_version 162547 (0.0008) [2023-12-26 16:33:04,929][105620] Updated weights for policy 1, policy_version 163433 (0.0011) [2023-12-26 16:33:04,988][105620] Updated weights for policy 1, policy_version 163443 (0.0010) [2023-12-26 16:33:05,046][105620] Updated weights for policy 1, policy_version 163453 (0.0010) [2023-12-26 16:33:05,108][105620] Updated weights for policy 1, policy_version 163463 (0.0010) [2023-12-26 16:33:05,504][105692] Updated weights for policy 0, policy_version 162557 (0.0005) [2023-12-26 16:33:05,552][105692] Updated weights for policy 0, policy_version 162567 (0.0005) [2023-12-26 16:33:05,600][105692] Updated weights for policy 0, policy_version 162577 (0.0005) [2023-12-26 16:33:05,662][105620] Updated weights for policy 1, policy_version 163473 (0.0005) [2023-12-26 16:33:05,726][105620] Updated weights for policy 1, policy_version 163483 (0.0009) [2023-12-26 16:33:05,782][105620] Updated weights for policy 1, policy_version 163493 (0.0008) [2023-12-26 16:33:06,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 83492864. Throughput: 0: 9672.7, 1: 10053.9. Samples: 83479648. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 16:33:06,062][104569] Avg episode reward: [(0, '9351.824'), (1, '9175.011')] [2023-12-26 16:33:06,292][105692] Updated weights for policy 0, policy_version 162587 (0.0006) [2023-12-26 16:33:06,355][105692] Updated weights for policy 0, policy_version 162597 (0.0007) [2023-12-26 16:33:06,360][105620] Updated weights for policy 1, policy_version 163503 (0.0009) [2023-12-26 16:33:06,411][105692] Updated weights for policy 0, policy_version 162607 (0.0006) [2023-12-26 16:33:06,421][105620] Updated weights for policy 1, policy_version 163513 (0.0011) [2023-12-26 16:33:06,482][105620] Updated weights for policy 1, policy_version 163523 (0.0011) [2023-12-26 16:33:07,107][105620] Updated weights for policy 1, policy_version 163533 (0.0008) [2023-12-26 16:33:07,160][105620] Updated weights for policy 1, policy_version 163543 (0.0006) [2023-12-26 16:33:07,213][105620] Updated weights for policy 1, policy_version 163553 (0.0006) [2023-12-26 16:33:07,232][105692] Updated weights for policy 0, policy_version 162617 (0.0006) [2023-12-26 16:33:07,285][105692] Updated weights for policy 0, policy_version 162627 (0.0010) [2023-12-26 16:33:07,333][105692] Updated weights for policy 0, policy_version 162637 (0.0010) [2023-12-26 16:33:07,384][105692] Updated weights for policy 0, policy_version 162647 (0.0010) [2023-12-26 16:33:07,747][105620] Updated weights for policy 1, policy_version 163563 (0.0007) [2023-12-26 16:33:07,806][105620] Updated weights for policy 1, policy_version 163573 (0.0010) [2023-12-26 16:33:07,872][105620] Updated weights for policy 1, policy_version 163583 (0.0011) [2023-12-26 16:33:08,144][105692] Updated weights for policy 0, policy_version 162657 (0.0009) [2023-12-26 16:33:08,199][105692] Updated weights for policy 0, policy_version 162667 (0.0008) [2023-12-26 16:33:08,250][105692] Updated weights for policy 0, policy_version 162677 (0.0008) [2023-12-26 16:33:08,605][105620] Updated weights for policy 1, policy_version 163593 (0.0010) [2023-12-26 16:33:08,663][105620] Updated weights for policy 1, policy_version 163603 (0.0010) [2023-12-26 16:33:08,722][105620] Updated weights for policy 1, policy_version 163613 (0.0010) [2023-12-26 16:33:08,773][105620] Updated weights for policy 1, policy_version 163623 (0.0010) [2023-12-26 16:33:09,017][105692] Updated weights for policy 0, policy_version 162687 (0.0008) [2023-12-26 16:33:09,076][105692] Updated weights for policy 0, policy_version 162697 (0.0008) [2023-12-26 16:33:09,132][105692] Updated weights for policy 0, policy_version 162707 (0.0009) [2023-12-26 16:33:09,566][105620] Updated weights for policy 1, policy_version 163633 (0.0010) [2023-12-26 16:33:09,615][105620] Updated weights for policy 1, policy_version 163643 (0.0010) [2023-12-26 16:33:09,668][105620] Updated weights for policy 1, policy_version 163653 (0.0008) [2023-12-26 16:33:09,951][105692] Updated weights for policy 0, policy_version 162717 (0.0007) [2023-12-26 16:33:10,017][105692] Updated weights for policy 0, policy_version 162727 (0.0006) [2023-12-26 16:33:10,075][105692] Updated weights for policy 0, policy_version 162737 (0.0007) [2023-12-26 16:33:10,435][105620] Updated weights for policy 1, policy_version 163663 (0.0009) [2023-12-26 16:33:10,497][105620] Updated weights for policy 1, policy_version 163673 (0.0006) [2023-12-26 16:33:10,555][105620] Updated weights for policy 1, policy_version 163683 (0.0008) [2023-12-26 16:33:10,794][105692] Updated weights for policy 0, policy_version 162747 (0.0007) [2023-12-26 16:33:10,849][105692] Updated weights for policy 0, policy_version 162757 (0.0008) [2023-12-26 16:33:10,902][105692] Updated weights for policy 0, policy_version 162767 (0.0011) [2023-12-26 16:33:11,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 83591168. Throughput: 0: 9630.6, 1: 10151.6. Samples: 83597884. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 16:33:11,063][104569] Avg episode reward: [(0, '9262.289'), (1, '9092.854')] [2023-12-26 16:33:11,312][105620] Updated weights for policy 1, policy_version 163693 (0.0006) [2023-12-26 16:33:11,381][105620] Updated weights for policy 1, policy_version 163703 (0.0008) [2023-12-26 16:33:11,445][105620] Updated weights for policy 1, policy_version 163713 (0.0008) [2023-12-26 16:33:11,673][105692] Updated weights for policy 0, policy_version 162777 (0.0010) [2023-12-26 16:33:11,726][105692] Updated weights for policy 0, policy_version 162787 (0.0009) [2023-12-26 16:33:11,794][105692] Updated weights for policy 0, policy_version 162797 (0.0010) [2023-12-26 16:33:11,861][105692] Updated weights for policy 0, policy_version 162807 (0.0007) [2023-12-26 16:33:12,240][105620] Updated weights for policy 1, policy_version 163723 (0.0010) [2023-12-26 16:33:12,303][105620] Updated weights for policy 1, policy_version 163733 (0.0007) [2023-12-26 16:33:12,367][105620] Updated weights for policy 1, policy_version 163743 (0.0008) [2023-12-26 16:33:12,587][105692] Updated weights for policy 0, policy_version 162817 (0.0009) [2023-12-26 16:33:12,639][105692] Updated weights for policy 0, policy_version 162827 (0.0009) [2023-12-26 16:33:12,695][105692] Updated weights for policy 0, policy_version 162837 (0.0009) [2023-12-26 16:33:13,086][105620] Updated weights for policy 1, policy_version 163753 (0.0008) [2023-12-26 16:33:13,149][105620] Updated weights for policy 1, policy_version 163763 (0.0008) [2023-12-26 16:33:13,198][105620] Updated weights for policy 1, policy_version 163773 (0.0008) [2023-12-26 16:33:13,262][105620] Updated weights for policy 1, policy_version 163783 (0.0009) [2023-12-26 16:33:13,475][105692] Updated weights for policy 0, policy_version 162847 (0.0009) [2023-12-26 16:33:13,530][105692] Updated weights for policy 0, policy_version 162857 (0.0009) [2023-12-26 16:33:13,583][105692] Updated weights for policy 0, policy_version 162868 (0.0009) [2023-12-26 16:33:14,033][105620] Updated weights for policy 1, policy_version 163793 (0.0008) [2023-12-26 16:33:14,076][105620] Updated weights for policy 1, policy_version 163803 (0.0007) [2023-12-26 16:33:14,133][105620] Updated weights for policy 1, policy_version 163813 (0.0008) [2023-12-26 16:33:14,280][105692] Updated weights for policy 0, policy_version 162878 (0.0008) [2023-12-26 16:33:14,334][105692] Updated weights for policy 0, policy_version 162888 (0.0009) [2023-12-26 16:33:14,388][105692] Updated weights for policy 0, policy_version 162899 (0.0010) [2023-12-26 16:33:14,825][105620] Updated weights for policy 1, policy_version 163823 (0.0010) [2023-12-26 16:33:14,889][105620] Updated weights for policy 1, policy_version 163833 (0.0009) [2023-12-26 16:33:14,953][105620] Updated weights for policy 1, policy_version 163843 (0.0010) [2023-12-26 16:33:15,186][105692] Updated weights for policy 0, policy_version 162909 (0.0010) [2023-12-26 16:33:15,241][105692] Updated weights for policy 0, policy_version 162919 (0.0010) [2023-12-26 16:33:15,298][105692] Updated weights for policy 0, policy_version 162929 (0.0010) [2023-12-26 16:33:15,531][105620] Updated weights for policy 1, policy_version 163853 (0.0010) [2023-12-26 16:33:15,582][105620] Updated weights for policy 1, policy_version 163863 (0.0008) [2023-12-26 16:33:15,629][105620] Updated weights for policy 1, policy_version 163873 (0.0005) [2023-12-26 16:33:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 83681280. Throughput: 0: 9529.7, 1: 10004.6. Samples: 83652880. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 16:33:16,063][104569] Avg episode reward: [(0, '9261.968'), (1, '8922.950')] [2023-12-26 16:33:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000162936_41721856.pth... [2023-12-26 16:33:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000163880_41959424.pth... [2023-12-26 16:33:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000162728_41664512.pth [2023-12-26 16:33:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000161848_41443328.pth [2023-12-26 16:33:16,119][105692] Updated weights for policy 0, policy_version 162939 (0.0010) [2023-12-26 16:33:16,177][105692] Updated weights for policy 0, policy_version 162949 (0.0009) [2023-12-26 16:33:16,226][105692] Updated weights for policy 0, policy_version 162959 (0.0008) [2023-12-26 16:33:16,290][105620] Updated weights for policy 1, policy_version 163883 (0.0006) [2023-12-26 16:33:16,351][105620] Updated weights for policy 1, policy_version 163893 (0.0009) [2023-12-26 16:33:16,413][105620] Updated weights for policy 1, policy_version 163903 (0.0008) [2023-12-26 16:33:16,936][105692] Updated weights for policy 0, policy_version 162969 (0.0009) [2023-12-26 16:33:16,998][105692] Updated weights for policy 0, policy_version 162979 (0.0008) [2023-12-26 16:33:17,061][105692] Updated weights for policy 0, policy_version 162989 (0.0006) [2023-12-26 16:33:17,121][105692] Updated weights for policy 0, policy_version 162999 (0.0007) [2023-12-26 16:33:17,174][105620] Updated weights for policy 1, policy_version 163913 (0.0006) [2023-12-26 16:33:17,230][105620] Updated weights for policy 1, policy_version 163923 (0.0009) [2023-12-26 16:33:17,293][105620] Updated weights for policy 1, policy_version 163933 (0.0009) [2023-12-26 16:33:17,351][105620] Updated weights for policy 1, policy_version 163943 (0.0010) [2023-12-26 16:33:17,732][105692] Updated weights for policy 0, policy_version 163009 (0.0009) [2023-12-26 16:33:17,786][105692] Updated weights for policy 0, policy_version 163019 (0.0009) [2023-12-26 16:33:17,840][105692] Updated weights for policy 0, policy_version 163029 (0.0009) [2023-12-26 16:33:18,154][105620] Updated weights for policy 1, policy_version 163953 (0.0006) [2023-12-26 16:33:18,215][105620] Updated weights for policy 1, policy_version 163963 (0.0005) [2023-12-26 16:33:18,261][105620] Updated weights for policy 1, policy_version 163973 (0.0006) [2023-12-26 16:33:18,548][105692] Updated weights for policy 0, policy_version 163039 (0.0006) [2023-12-26 16:33:18,602][105692] Updated weights for policy 0, policy_version 163049 (0.0005) [2023-12-26 16:33:18,665][105692] Updated weights for policy 0, policy_version 163059 (0.0006) [2023-12-26 16:33:18,931][105620] Updated weights for policy 1, policy_version 163983 (0.0006) [2023-12-26 16:33:18,986][105620] Updated weights for policy 1, policy_version 163993 (0.0005) [2023-12-26 16:33:19,051][105620] Updated weights for policy 1, policy_version 164003 (0.0005) [2023-12-26 16:33:19,343][105692] Updated weights for policy 0, policy_version 163069 (0.0011) [2023-12-26 16:33:19,402][105692] Updated weights for policy 0, policy_version 163079 (0.0009) [2023-12-26 16:33:19,453][105692] Updated weights for policy 0, policy_version 163089 (0.0009) [2023-12-26 16:33:19,742][105620] Updated weights for policy 1, policy_version 164013 (0.0006) [2023-12-26 16:33:19,808][105620] Updated weights for policy 1, policy_version 164023 (0.0009) [2023-12-26 16:33:19,880][105620] Updated weights for policy 1, policy_version 164033 (0.0010) [2023-12-26 16:33:20,168][105692] Updated weights for policy 0, policy_version 163099 (0.0008) [2023-12-26 16:33:20,235][105692] Updated weights for policy 0, policy_version 163109 (0.0008) [2023-12-26 16:33:20,294][105692] Updated weights for policy 0, policy_version 163119 (0.0008) [2023-12-26 16:33:20,614][105620] Updated weights for policy 1, policy_version 164043 (0.0010) [2023-12-26 16:33:20,674][105620] Updated weights for policy 1, policy_version 164053 (0.0011) [2023-12-26 16:33:20,739][105620] Updated weights for policy 1, policy_version 164063 (0.0007) [2023-12-26 16:33:20,953][105692] Updated weights for policy 0, policy_version 163129 (0.0007) [2023-12-26 16:33:21,025][105692] Updated weights for policy 0, policy_version 163139 (0.0006) [2023-12-26 16:33:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 83779584. Throughput: 0: 9562.2, 1: 9891.8. Samples: 83770772. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 16:33:21,062][104569] Avg episode reward: [(0, '9351.955'), (1, '8943.450')] [2023-12-26 16:33:21,099][105692] Updated weights for policy 0, policy_version 163149 (0.0009) [2023-12-26 16:33:21,169][105692] Updated weights for policy 0, policy_version 163159 (0.0009) [2023-12-26 16:33:21,523][105620] Updated weights for policy 1, policy_version 164073 (0.0008) [2023-12-26 16:33:21,581][105620] Updated weights for policy 1, policy_version 164083 (0.0008) [2023-12-26 16:33:21,643][105620] Updated weights for policy 1, policy_version 164093 (0.0009) [2023-12-26 16:33:21,713][105620] Updated weights for policy 1, policy_version 164103 (0.0007) [2023-12-26 16:33:21,902][105692] Updated weights for policy 0, policy_version 163169 (0.0011) [2023-12-26 16:33:21,955][105692] Updated weights for policy 0, policy_version 163179 (0.0011) [2023-12-26 16:33:22,007][105692] Updated weights for policy 0, policy_version 163189 (0.0010) [2023-12-26 16:33:22,485][105620] Updated weights for policy 1, policy_version 164113 (0.0008) [2023-12-26 16:33:22,552][105620] Updated weights for policy 1, policy_version 164123 (0.0006) [2023-12-26 16:33:22,621][105620] Updated weights for policy 1, policy_version 164133 (0.0006) [2023-12-26 16:33:22,794][105692] Updated weights for policy 0, policy_version 163199 (0.0011) [2023-12-26 16:33:22,847][105692] Updated weights for policy 0, policy_version 163209 (0.0011) [2023-12-26 16:33:22,917][105692] Updated weights for policy 0, policy_version 163219 (0.0011) [2023-12-26 16:33:23,196][105620] Updated weights for policy 1, policy_version 164143 (0.0006) [2023-12-26 16:33:23,264][105620] Updated weights for policy 1, policy_version 164153 (0.0005) [2023-12-26 16:33:23,319][105620] Updated weights for policy 1, policy_version 164163 (0.0005) [2023-12-26 16:33:23,560][105692] Updated weights for policy 0, policy_version 163229 (0.0008) [2023-12-26 16:33:23,608][105692] Updated weights for policy 0, policy_version 163239 (0.0005) [2023-12-26 16:33:23,654][105692] Updated weights for policy 0, policy_version 163249 (0.0007) [2023-12-26 16:33:23,943][105620] Updated weights for policy 1, policy_version 164173 (0.0005) [2023-12-26 16:33:24,006][105620] Updated weights for policy 1, policy_version 164183 (0.0006) [2023-12-26 16:33:24,073][105620] Updated weights for policy 1, policy_version 164193 (0.0005) [2023-12-26 16:33:24,350][105692] Updated weights for policy 0, policy_version 163259 (0.0009) [2023-12-26 16:33:24,415][105692] Updated weights for policy 0, policy_version 163269 (0.0008) [2023-12-26 16:33:24,475][105692] Updated weights for policy 0, policy_version 163279 (0.0010) [2023-12-26 16:33:24,607][105620] Updated weights for policy 1, policy_version 164203 (0.0006) [2023-12-26 16:33:24,667][105620] Updated weights for policy 1, policy_version 164213 (0.0008) [2023-12-26 16:33:24,730][105620] Updated weights for policy 1, policy_version 164223 (0.0008) [2023-12-26 16:33:25,165][105692] Updated weights for policy 0, policy_version 163289 (0.0010) [2023-12-26 16:33:25,212][105692] Updated weights for policy 0, policy_version 163299 (0.0010) [2023-12-26 16:33:25,270][105692] Updated weights for policy 0, policy_version 163309 (0.0010) [2023-12-26 16:33:25,282][105620] Updated weights for policy 1, policy_version 164233 (0.0007) [2023-12-26 16:33:25,329][105692] Updated weights for policy 0, policy_version 163319 (0.0010) [2023-12-26 16:33:25,348][105620] Updated weights for policy 1, policy_version 164243 (0.0005) [2023-12-26 16:33:25,413][105620] Updated weights for policy 1, policy_version 164253 (0.0005) [2023-12-26 16:33:25,476][105620] Updated weights for policy 1, policy_version 164263 (0.0006) [2023-12-26 16:33:26,027][105620] Updated weights for policy 1, policy_version 164273 (0.0005) [2023-12-26 16:33:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 83877888. Throughput: 0: 9587.8, 1: 9973.0. Samples: 83893168. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 16:33:26,062][104569] Avg episode reward: [(0, '9352.940'), (1, '7847.738')] [2023-12-26 16:33:26,081][105692] Updated weights for policy 0, policy_version 163329 (0.0010) [2023-12-26 16:33:26,091][105620] Updated weights for policy 1, policy_version 164283 (0.0006) [2023-12-26 16:33:26,149][105692] Updated weights for policy 0, policy_version 163339 (0.0010) [2023-12-26 16:33:26,157][105620] Updated weights for policy 1, policy_version 164293 (0.0005) [2023-12-26 16:33:26,211][105692] Updated weights for policy 0, policy_version 163349 (0.0010) [2023-12-26 16:33:26,720][105620] Updated weights for policy 1, policy_version 164303 (0.0007) [2023-12-26 16:33:26,769][105620] Updated weights for policy 1, policy_version 164313 (0.0008) [2023-12-26 16:33:26,825][105620] Updated weights for policy 1, policy_version 164323 (0.0008) [2023-12-26 16:33:26,936][105692] Updated weights for policy 0, policy_version 163359 (0.0011) [2023-12-26 16:33:26,991][105692] Updated weights for policy 0, policy_version 163369 (0.0010) [2023-12-26 16:33:27,045][105692] Updated weights for policy 0, policy_version 163379 (0.0010) [2023-12-26 16:33:27,524][105620] Updated weights for policy 1, policy_version 164333 (0.0007) [2023-12-26 16:33:27,576][105620] Updated weights for policy 1, policy_version 164343 (0.0005) [2023-12-26 16:33:27,625][105620] Updated weights for policy 1, policy_version 164353 (0.0005) [2023-12-26 16:33:27,797][105692] Updated weights for policy 0, policy_version 163389 (0.0008) [2023-12-26 16:33:27,851][105692] Updated weights for policy 0, policy_version 163399 (0.0005) [2023-12-26 16:33:27,899][105692] Updated weights for policy 0, policy_version 163409 (0.0005) [2023-12-26 16:33:28,191][105620] Updated weights for policy 1, policy_version 164363 (0.0007) [2023-12-26 16:33:28,246][105620] Updated weights for policy 1, policy_version 164373 (0.0005) [2023-12-26 16:33:28,305][105620] Updated weights for policy 1, policy_version 164383 (0.0007) [2023-12-26 16:33:28,573][105692] Updated weights for policy 0, policy_version 163419 (0.0007) [2023-12-26 16:33:28,623][105692] Updated weights for policy 0, policy_version 163429 (0.0010) [2023-12-26 16:33:28,681][105692] Updated weights for policy 0, policy_version 163439 (0.0010) [2023-12-26 16:33:29,036][105620] Updated weights for policy 1, policy_version 164393 (0.0009) [2023-12-26 16:33:29,097][105620] Updated weights for policy 1, policy_version 164403 (0.0009) [2023-12-26 16:33:29,150][105620] Updated weights for policy 1, policy_version 164413 (0.0010) [2023-12-26 16:33:29,205][105620] Updated weights for policy 1, policy_version 164423 (0.0009) [2023-12-26 16:33:29,359][105692] Updated weights for policy 0, policy_version 163449 (0.0010) [2023-12-26 16:33:29,415][105692] Updated weights for policy 0, policy_version 163459 (0.0011) [2023-12-26 16:33:29,473][105692] Updated weights for policy 0, policy_version 163469 (0.0010) [2023-12-26 16:33:29,532][105692] Updated weights for policy 0, policy_version 163479 (0.0010) [2023-12-26 16:33:30,002][105620] Updated weights for policy 1, policy_version 164433 (0.0008) [2023-12-26 16:33:30,059][105620] Updated weights for policy 1, policy_version 164443 (0.0009) [2023-12-26 16:33:30,119][105620] Updated weights for policy 1, policy_version 164453 (0.0010) [2023-12-26 16:33:30,172][105692] Updated weights for policy 0, policy_version 163489 (0.0006) [2023-12-26 16:33:30,231][105692] Updated weights for policy 0, policy_version 163499 (0.0005) [2023-12-26 16:33:30,286][105692] Updated weights for policy 0, policy_version 163509 (0.0005) [2023-12-26 16:33:30,783][105692] Updated weights for policy 0, policy_version 163519 (0.0005) [2023-12-26 16:33:30,830][105692] Updated weights for policy 0, policy_version 163529 (0.0008) [2023-12-26 16:33:30,877][105692] Updated weights for policy 0, policy_version 163539 (0.0009) [2023-12-26 16:33:30,996][105620] Updated weights for policy 1, policy_version 164463 (0.0009) [2023-12-26 16:33:31,059][105620] Updated weights for policy 1, policy_version 164473 (0.0007) [2023-12-26 16:33:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 83984384. Throughput: 0: 9626.3, 1: 10073.8. Samples: 83954932. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 16:33:31,063][104569] Avg episode reward: [(0, '9352.150'), (1, '1159.745')] [2023-12-26 16:33:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000163544_41877504.pth... [2023-12-26 16:33:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000162392_41582592.pth [2023-12-26 16:33:31,115][105620] Updated weights for policy 1, policy_version 164483 (0.0009) [2023-12-26 16:33:31,149][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000164488_42115072.pth... [2023-12-26 16:33:31,154][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000163304_41811968.pth [2023-12-26 16:33:31,602][105692] Updated weights for policy 0, policy_version 163549 (0.0007) [2023-12-26 16:33:31,664][105692] Updated weights for policy 0, policy_version 163559 (0.0008) [2023-12-26 16:33:31,715][105692] Updated weights for policy 0, policy_version 163569 (0.0008) [2023-12-26 16:33:31,842][105620] Updated weights for policy 1, policy_version 164493 (0.0009) [2023-12-26 16:33:31,894][105620] Updated weights for policy 1, policy_version 164503 (0.0008) [2023-12-26 16:33:31,948][105620] Updated weights for policy 1, policy_version 164513 (0.0008) [2023-12-26 16:33:32,467][105692] Updated weights for policy 0, policy_version 163579 (0.0009) [2023-12-26 16:33:32,518][105692] Updated weights for policy 0, policy_version 163589 (0.0010) [2023-12-26 16:33:32,569][105692] Updated weights for policy 0, policy_version 163599 (0.0010) [2023-12-26 16:33:32,741][105620] Updated weights for policy 1, policy_version 164523 (0.0008) [2023-12-26 16:33:32,800][105620] Updated weights for policy 1, policy_version 164533 (0.0008) [2023-12-26 16:33:32,852][105620] Updated weights for policy 1, policy_version 164543 (0.0009) [2023-12-26 16:33:33,291][105692] Updated weights for policy 0, policy_version 163609 (0.0010) [2023-12-26 16:33:33,349][105692] Updated weights for policy 0, policy_version 163619 (0.0009) [2023-12-26 16:33:33,404][105692] Updated weights for policy 0, policy_version 163629 (0.0010) [2023-12-26 16:33:33,467][105692] Updated weights for policy 0, policy_version 163639 (0.0010) [2023-12-26 16:33:33,600][105620] Updated weights for policy 1, policy_version 164553 (0.0009) [2023-12-26 16:33:33,658][105620] Updated weights for policy 1, policy_version 164563 (0.0009) [2023-12-26 16:33:33,718][105620] Updated weights for policy 1, policy_version 164573 (0.0009) [2023-12-26 16:33:33,765][105620] Updated weights for policy 1, policy_version 164583 (0.0009) [2023-12-26 16:33:34,231][105692] Updated weights for policy 0, policy_version 163649 (0.0008) [2023-12-26 16:33:34,290][105692] Updated weights for policy 0, policy_version 163659 (0.0009) [2023-12-26 16:33:34,345][105692] Updated weights for policy 0, policy_version 163669 (0.0009) [2023-12-26 16:33:34,464][105620] Updated weights for policy 1, policy_version 164593 (0.0009) [2023-12-26 16:33:34,531][105620] Updated weights for policy 1, policy_version 164603 (0.0010) [2023-12-26 16:33:34,586][105620] Updated weights for policy 1, policy_version 164613 (0.0008) [2023-12-26 16:33:35,134][105692] Updated weights for policy 0, policy_version 163679 (0.0010) [2023-12-26 16:33:35,186][105692] Updated weights for policy 0, policy_version 163689 (0.0010) [2023-12-26 16:33:35,217][105620] Updated weights for policy 1, policy_version 164623 (0.0008) [2023-12-26 16:33:35,243][105692] Updated weights for policy 0, policy_version 163699 (0.0010) [2023-12-26 16:33:35,269][105620] Updated weights for policy 1, policy_version 164633 (0.0006) [2023-12-26 16:33:35,322][105620] Updated weights for policy 1, policy_version 164643 (0.0008) [2023-12-26 16:33:35,943][105692] Updated weights for policy 0, policy_version 163709 (0.0010) [2023-12-26 16:33:35,993][105692] Updated weights for policy 0, policy_version 163719 (0.0006) [2023-12-26 16:33:36,044][105692] Updated weights for policy 0, policy_version 163729 (0.0009) [2023-12-26 16:33:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 84074496. Throughput: 0: 9706.3, 1: 10055.6. Samples: 84070444. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 16:33:36,062][104569] Avg episode reward: [(0, '9351.148'), (1, '1213.358')] [2023-12-26 16:33:36,136][105620] Updated weights for policy 1, policy_version 164653 (0.0009) [2023-12-26 16:33:36,198][105620] Updated weights for policy 1, policy_version 164663 (0.0008) [2023-12-26 16:33:36,261][105620] Updated weights for policy 1, policy_version 164673 (0.0008) [2023-12-26 16:33:36,784][105692] Updated weights for policy 0, policy_version 163739 (0.0007) [2023-12-26 16:33:36,835][105692] Updated weights for policy 0, policy_version 163749 (0.0010) [2023-12-26 16:33:36,898][105692] Updated weights for policy 0, policy_version 163759 (0.0011) [2023-12-26 16:33:37,029][105620] Updated weights for policy 1, policy_version 164683 (0.0008) [2023-12-26 16:33:37,082][105620] Updated weights for policy 1, policy_version 164693 (0.0008) [2023-12-26 16:33:37,142][105620] Updated weights for policy 1, policy_version 164703 (0.0008) [2023-12-26 16:33:37,626][105692] Updated weights for policy 0, policy_version 163769 (0.0011) [2023-12-26 16:33:37,681][105692] Updated weights for policy 0, policy_version 163779 (0.0010) [2023-12-26 16:33:37,730][105692] Updated weights for policy 0, policy_version 163789 (0.0011) [2023-12-26 16:33:37,781][105692] Updated weights for policy 0, policy_version 163799 (0.0010) [2023-12-26 16:33:37,870][105620] Updated weights for policy 1, policy_version 164713 (0.0008) [2023-12-26 16:33:37,927][105620] Updated weights for policy 1, policy_version 164723 (0.0010) [2023-12-26 16:33:37,984][105620] Updated weights for policy 1, policy_version 164733 (0.0010) [2023-12-26 16:33:38,042][105620] Updated weights for policy 1, policy_version 164743 (0.0009) [2023-12-26 16:33:38,459][105692] Updated weights for policy 0, policy_version 163809 (0.0006) [2023-12-26 16:33:38,515][105692] Updated weights for policy 0, policy_version 163819 (0.0007) [2023-12-26 16:33:38,567][105692] Updated weights for policy 0, policy_version 163829 (0.0010) [2023-12-26 16:33:38,865][105620] Updated weights for policy 1, policy_version 164753 (0.0008) [2023-12-26 16:33:38,924][105620] Updated weights for policy 1, policy_version 164763 (0.0008) [2023-12-26 16:33:38,983][105620] Updated weights for policy 1, policy_version 164773 (0.0008) [2023-12-26 16:33:39,290][105692] Updated weights for policy 0, policy_version 163839 (0.0008) [2023-12-26 16:33:39,356][105692] Updated weights for policy 0, policy_version 163849 (0.0010) [2023-12-26 16:33:39,427][105692] Updated weights for policy 0, policy_version 163859 (0.0008) [2023-12-26 16:33:39,776][105620] Updated weights for policy 1, policy_version 164783 (0.0009) [2023-12-26 16:33:39,845][105620] Updated weights for policy 1, policy_version 164793 (0.0008) [2023-12-26 16:33:39,914][105620] Updated weights for policy 1, policy_version 164803 (0.0008) [2023-12-26 16:33:40,105][105692] Updated weights for policy 0, policy_version 163869 (0.0007) [2023-12-26 16:33:40,159][105692] Updated weights for policy 0, policy_version 163879 (0.0010) [2023-12-26 16:33:40,216][105692] Updated weights for policy 0, policy_version 163889 (0.0010) [2023-12-26 16:33:40,523][105620] Updated weights for policy 1, policy_version 164813 (0.0009) [2023-12-26 16:33:40,574][105620] Updated weights for policy 1, policy_version 164823 (0.0009) [2023-12-26 16:33:40,627][105620] Updated weights for policy 1, policy_version 164834 (0.0010) [2023-12-26 16:33:40,907][105692] Updated weights for policy 0, policy_version 163899 (0.0009) [2023-12-26 16:33:40,972][105692] Updated weights for policy 0, policy_version 163909 (0.0005) [2023-12-26 16:33:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 84172800. Throughput: 0: 9785.3, 1: 9908.7. Samples: 84184496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:33:41,062][104569] Avg episode reward: [(0, '9350.030'), (1, '1515.486')] [2023-12-26 16:33:41,075][105692] Updated weights for policy 0, policy_version 163919 (0.0006) [2023-12-26 16:33:41,486][105620] Updated weights for policy 1, policy_version 164845 (0.0010) [2023-12-26 16:33:41,533][105620] Updated weights for policy 1, policy_version 164855 (0.0008) [2023-12-26 16:33:41,594][105620] Updated weights for policy 1, policy_version 164865 (0.0007) [2023-12-26 16:33:41,697][105692] Updated weights for policy 0, policy_version 163929 (0.0008) [2023-12-26 16:33:41,764][105692] Updated weights for policy 0, policy_version 163939 (0.0009) [2023-12-26 16:33:41,813][105692] Updated weights for policy 0, policy_version 163949 (0.0009) [2023-12-26 16:33:41,861][105692] Updated weights for policy 0, policy_version 163959 (0.0009) [2023-12-26 16:33:42,386][105620] Updated weights for policy 1, policy_version 164875 (0.0008) [2023-12-26 16:33:42,453][105620] Updated weights for policy 1, policy_version 164885 (0.0007) [2023-12-26 16:33:42,525][105620] Updated weights for policy 1, policy_version 164895 (0.0005) [2023-12-26 16:33:42,655][105692] Updated weights for policy 0, policy_version 163969 (0.0010) [2023-12-26 16:33:42,707][105692] Updated weights for policy 0, policy_version 163980 (0.0009) [2023-12-26 16:33:42,758][105692] Updated weights for policy 0, policy_version 163990 (0.0009) [2023-12-26 16:33:43,060][105620] Updated weights for policy 1, policy_version 164905 (0.0005) [2023-12-26 16:33:43,118][105620] Updated weights for policy 1, policy_version 164915 (0.0006) [2023-12-26 16:33:43,183][105620] Updated weights for policy 1, policy_version 164925 (0.0007) [2023-12-26 16:33:43,241][105620] Updated weights for policy 1, policy_version 164936 (0.0010) [2023-12-26 16:33:43,345][105692] Updated weights for policy 0, policy_version 164000 (0.0005) [2023-12-26 16:33:43,399][105692] Updated weights for policy 0, policy_version 164010 (0.0006) [2023-12-26 16:33:43,458][105692] Updated weights for policy 0, policy_version 164020 (0.0006) [2023-12-26 16:33:44,001][105692] Updated weights for policy 0, policy_version 164030 (0.0009) [2023-12-26 16:33:44,060][105692] Updated weights for policy 0, policy_version 164040 (0.0010) [2023-12-26 16:33:44,062][105620] Updated weights for policy 1, policy_version 164946 (0.0006) [2023-12-26 16:33:44,122][105692] Updated weights for policy 0, policy_version 164050 (0.0010) [2023-12-26 16:33:44,125][105620] Updated weights for policy 1, policy_version 164956 (0.0006) [2023-12-26 16:33:44,171][105620] Updated weights for policy 1, policy_version 164966 (0.0005) [2023-12-26 16:33:44,749][105692] Updated weights for policy 0, policy_version 164060 (0.0008) [2023-12-26 16:33:44,809][105692] Updated weights for policy 0, policy_version 164070 (0.0009) [2023-12-26 16:33:44,863][105692] Updated weights for policy 0, policy_version 164080 (0.0010) [2023-12-26 16:33:44,960][105620] Updated weights for policy 1, policy_version 164976 (0.0009) [2023-12-26 16:33:45,014][105620] Updated weights for policy 1, policy_version 164986 (0.0009) [2023-12-26 16:33:45,083][105620] Updated weights for policy 1, policy_version 164996 (0.0007) [2023-12-26 16:33:45,526][105692] Updated weights for policy 0, policy_version 164090 (0.0007) [2023-12-26 16:33:45,584][105692] Updated weights for policy 0, policy_version 164100 (0.0010) [2023-12-26 16:33:45,654][105692] Updated weights for policy 0, policy_version 164110 (0.0010) [2023-12-26 16:33:45,715][105692] Updated weights for policy 0, policy_version 164120 (0.0010) [2023-12-26 16:33:45,860][105620] Updated weights for policy 1, policy_version 165006 (0.0008) [2023-12-26 16:33:45,911][105620] Updated weights for policy 1, policy_version 165016 (0.0007) [2023-12-26 16:33:45,963][105620] Updated weights for policy 1, policy_version 165026 (0.0005) [2023-12-26 16:33:46,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19797.2, 300 sec: 19577.5). Total num frames: 84279296. Throughput: 0: 9807.2, 1: 9878.8. Samples: 84245076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:33:46,063][104569] Avg episode reward: [(0, '9172.582'), (1, '6508.362')] [2023-12-26 16:33:46,073][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000164120_42024960.pth... [2023-12-26 16:33:46,075][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000165032_42254336.pth... [2023-12-26 16:33:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000162936_41721856.pth [2023-12-26 16:33:46,081][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000163880_41959424.pth [2023-12-26 16:33:46,334][105692] Updated weights for policy 0, policy_version 164130 (0.0007) [2023-12-26 16:33:46,384][105692] Updated weights for policy 0, policy_version 164140 (0.0005) [2023-12-26 16:33:46,433][105692] Updated weights for policy 0, policy_version 164150 (0.0007) [2023-12-26 16:33:46,558][105620] Updated weights for policy 1, policy_version 165036 (0.0006) [2023-12-26 16:33:46,603][105620] Updated weights for policy 1, policy_version 165046 (0.0005) [2023-12-26 16:33:46,662][105620] Updated weights for policy 1, policy_version 165056 (0.0010) [2023-12-26 16:33:47,115][105692] Updated weights for policy 0, policy_version 164160 (0.0007) [2023-12-26 16:33:47,174][105692] Updated weights for policy 0, policy_version 164170 (0.0011) [2023-12-26 16:33:47,236][105692] Updated weights for policy 0, policy_version 164180 (0.0010) [2023-12-26 16:33:47,288][105620] Updated weights for policy 1, policy_version 165066 (0.0009) [2023-12-26 16:33:47,354][105620] Updated weights for policy 1, policy_version 165076 (0.0010) [2023-12-26 16:33:47,420][105620] Updated weights for policy 1, policy_version 165086 (0.0010) [2023-12-26 16:33:47,486][105620] Updated weights for policy 1, policy_version 165096 (0.0011) [2023-12-26 16:33:47,954][105692] Updated weights for policy 0, policy_version 164190 (0.0010) [2023-12-26 16:33:48,012][105692] Updated weights for policy 0, policy_version 164200 (0.0010) [2023-12-26 16:33:48,066][105692] Updated weights for policy 0, policy_version 164210 (0.0010) [2023-12-26 16:33:48,215][105620] Updated weights for policy 1, policy_version 165106 (0.0010) [2023-12-26 16:33:48,277][105620] Updated weights for policy 1, policy_version 165116 (0.0010) [2023-12-26 16:33:48,343][105620] Updated weights for policy 1, policy_version 165126 (0.0009) [2023-12-26 16:33:48,816][105692] Updated weights for policy 0, policy_version 164220 (0.0010) [2023-12-26 16:33:48,882][105692] Updated weights for policy 0, policy_version 164230 (0.0010) [2023-12-26 16:33:48,945][105692] Updated weights for policy 0, policy_version 164240 (0.0010) [2023-12-26 16:33:49,071][105620] Updated weights for policy 1, policy_version 165136 (0.0008) [2023-12-26 16:33:49,125][105620] Updated weights for policy 1, policy_version 165146 (0.0007) [2023-12-26 16:33:49,179][105620] Updated weights for policy 1, policy_version 165156 (0.0006) [2023-12-26 16:33:49,691][105692] Updated weights for policy 0, policy_version 164250 (0.0010) [2023-12-26 16:33:49,750][105692] Updated weights for policy 0, policy_version 164260 (0.0011) [2023-12-26 16:33:49,771][105620] Updated weights for policy 1, policy_version 165166 (0.0005) [2023-12-26 16:33:49,802][105692] Updated weights for policy 0, policy_version 164270 (0.0011) [2023-12-26 16:33:49,831][105620] Updated weights for policy 1, policy_version 165176 (0.0009) [2023-12-26 16:33:49,860][105692] Updated weights for policy 0, policy_version 164280 (0.0011) [2023-12-26 16:33:49,894][105620] Updated weights for policy 1, policy_version 165186 (0.0010) [2023-12-26 16:33:50,553][105692] Updated weights for policy 0, policy_version 164290 (0.0009) [2023-12-26 16:33:50,620][105692] Updated weights for policy 0, policy_version 164300 (0.0006) [2023-12-26 16:33:50,681][105692] Updated weights for policy 0, policy_version 164310 (0.0007) [2023-12-26 16:33:50,686][105620] Updated weights for policy 1, policy_version 165196 (0.0010) [2023-12-26 16:33:50,751][105620] Updated weights for policy 1, policy_version 165206 (0.0009) [2023-12-26 16:33:50,811][105620] Updated weights for policy 1, policy_version 165216 (0.0009) [2023-12-26 16:33:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 84377600. Throughput: 0: 9829.3, 1: 9879.0. Samples: 84366520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:33:51,062][104569] Avg episode reward: [(0, '9172.882'), (1, '9176.333')] [2023-12-26 16:33:51,449][105692] Updated weights for policy 0, policy_version 164320 (0.0008) [2023-12-26 16:33:51,509][105692] Updated weights for policy 0, policy_version 164330 (0.0008) [2023-12-26 16:33:51,546][105620] Updated weights for policy 1, policy_version 165226 (0.0011) [2023-12-26 16:33:51,562][105692] Updated weights for policy 0, policy_version 164340 (0.0008) [2023-12-26 16:33:51,609][105620] Updated weights for policy 1, policy_version 165236 (0.0011) [2023-12-26 16:33:51,674][105620] Updated weights for policy 1, policy_version 165246 (0.0007) [2023-12-26 16:33:51,741][105620] Updated weights for policy 1, policy_version 165256 (0.0008) [2023-12-26 16:33:52,265][105692] Updated weights for policy 0, policy_version 164350 (0.0008) [2023-12-26 16:33:52,332][105692] Updated weights for policy 0, policy_version 164360 (0.0010) [2023-12-26 16:33:52,400][105692] Updated weights for policy 0, policy_version 164370 (0.0008) [2023-12-26 16:33:52,500][105620] Updated weights for policy 1, policy_version 165266 (0.0009) [2023-12-26 16:33:52,549][105620] Updated weights for policy 1, policy_version 165276 (0.0008) [2023-12-26 16:33:52,613][105620] Updated weights for policy 1, policy_version 165286 (0.0008) [2023-12-26 16:33:53,105][105692] Updated weights for policy 0, policy_version 164380 (0.0009) [2023-12-26 16:33:53,170][105692] Updated weights for policy 0, policy_version 164390 (0.0011) [2023-12-26 16:33:53,226][105692] Updated weights for policy 0, policy_version 164400 (0.0009) [2023-12-26 16:33:53,280][105620] Updated weights for policy 1, policy_version 165296 (0.0007) [2023-12-26 16:33:53,349][105620] Updated weights for policy 1, policy_version 165306 (0.0006) [2023-12-26 16:33:53,404][105620] Updated weights for policy 1, policy_version 165316 (0.0008) [2023-12-26 16:33:53,837][105692] Updated weights for policy 0, policy_version 164410 (0.0011) [2023-12-26 16:33:53,888][105692] Updated weights for policy 0, policy_version 164420 (0.0010) [2023-12-26 16:33:53,935][105692] Updated weights for policy 0, policy_version 164430 (0.0010) [2023-12-26 16:33:53,994][105692] Updated weights for policy 0, policy_version 164440 (0.0010) [2023-12-26 16:33:54,198][105620] Updated weights for policy 1, policy_version 165326 (0.0009) [2023-12-26 16:33:54,255][105620] Updated weights for policy 1, policy_version 165336 (0.0009) [2023-12-26 16:33:54,313][105620] Updated weights for policy 1, policy_version 165347 (0.0010) [2023-12-26 16:33:54,706][105692] Updated weights for policy 0, policy_version 164450 (0.0006) [2023-12-26 16:33:54,768][105692] Updated weights for policy 0, policy_version 164460 (0.0005) [2023-12-26 16:33:54,839][105692] Updated weights for policy 0, policy_version 164470 (0.0007) [2023-12-26 16:33:54,958][105620] Updated weights for policy 1, policy_version 165357 (0.0008) [2023-12-26 16:33:55,018][105620] Updated weights for policy 1, policy_version 165367 (0.0006) [2023-12-26 16:33:55,086][105620] Updated weights for policy 1, policy_version 165377 (0.0005) [2023-12-26 16:33:55,446][105692] Updated weights for policy 0, policy_version 164480 (0.0007) [2023-12-26 16:33:55,507][105692] Updated weights for policy 0, policy_version 164490 (0.0007) [2023-12-26 16:33:55,574][105692] Updated weights for policy 0, policy_version 164500 (0.0007) [2023-12-26 16:33:55,742][105620] Updated weights for policy 1, policy_version 165387 (0.0007) [2023-12-26 16:33:55,794][105620] Updated weights for policy 1, policy_version 165397 (0.0007) [2023-12-26 16:33:55,850][105620] Updated weights for policy 1, policy_version 165407 (0.0005) [2023-12-26 16:33:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 84475904. Throughput: 0: 9923.5, 1: 9776.7. Samples: 84484396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:33:56,063][104569] Avg episode reward: [(0, '9261.546'), (1, '9177.130')] [2023-12-26 16:33:56,134][105692] Updated weights for policy 0, policy_version 164510 (0.0008) [2023-12-26 16:33:56,182][105692] Updated weights for policy 0, policy_version 164520 (0.0010) [2023-12-26 16:33:56,229][105692] Updated weights for policy 0, policy_version 164530 (0.0010) [2023-12-26 16:33:56,507][105620] Updated weights for policy 1, policy_version 165417 (0.0006) [2023-12-26 16:33:56,558][105620] Updated weights for policy 1, policy_version 165427 (0.0010) [2023-12-26 16:33:56,607][105620] Updated weights for policy 1, policy_version 165437 (0.0010) [2023-12-26 16:33:56,661][105620] Updated weights for policy 1, policy_version 165447 (0.0008) [2023-12-26 16:33:56,978][105692] Updated weights for policy 0, policy_version 164540 (0.0010) [2023-12-26 16:33:57,037][105692] Updated weights for policy 0, policy_version 164550 (0.0011) [2023-12-26 16:33:57,087][105692] Updated weights for policy 0, policy_version 164560 (0.0008) [2023-12-26 16:33:57,228][105620] Updated weights for policy 1, policy_version 165457 (0.0010) [2023-12-26 16:33:57,286][105620] Updated weights for policy 1, policy_version 165467 (0.0010) [2023-12-26 16:33:57,349][105620] Updated weights for policy 1, policy_version 165477 (0.0011) [2023-12-26 16:33:57,819][105692] Updated weights for policy 0, policy_version 164570 (0.0007) [2023-12-26 16:33:57,870][105692] Updated weights for policy 0, policy_version 164580 (0.0010) [2023-12-26 16:33:57,920][105692] Updated weights for policy 0, policy_version 164590 (0.0010) [2023-12-26 16:33:57,975][105692] Updated weights for policy 0, policy_version 164600 (0.0010) [2023-12-26 16:33:57,987][105620] Updated weights for policy 1, policy_version 165487 (0.0007) [2023-12-26 16:33:58,041][105620] Updated weights for policy 1, policy_version 165497 (0.0005) [2023-12-26 16:33:58,099][105620] Updated weights for policy 1, policy_version 165507 (0.0006) [2023-12-26 16:33:58,785][105692] Updated weights for policy 0, policy_version 164610 (0.0008) [2023-12-26 16:33:58,852][105692] Updated weights for policy 0, policy_version 164620 (0.0008) [2023-12-26 16:33:58,879][105620] Updated weights for policy 1, policy_version 165517 (0.0008) [2023-12-26 16:33:58,915][105692] Updated weights for policy 0, policy_version 164630 (0.0008) [2023-12-26 16:33:58,942][105620] Updated weights for policy 1, policy_version 165527 (0.0008) [2023-12-26 16:33:59,002][105620] Updated weights for policy 1, policy_version 165537 (0.0005) [2023-12-26 16:33:59,574][105692] Updated weights for policy 0, policy_version 164640 (0.0009) [2023-12-26 16:33:59,628][105692] Updated weights for policy 0, policy_version 164650 (0.0009) [2023-12-26 16:33:59,656][105620] Updated weights for policy 1, policy_version 165547 (0.0006) [2023-12-26 16:33:59,675][105692] Updated weights for policy 0, policy_version 164660 (0.0007) [2023-12-26 16:33:59,713][105620] Updated weights for policy 1, policy_version 165557 (0.0007) [2023-12-26 16:33:59,765][105620] Updated weights for policy 1, policy_version 165567 (0.0008) [2023-12-26 16:34:00,391][105692] Updated weights for policy 0, policy_version 164670 (0.0006) [2023-12-26 16:34:00,453][105692] Updated weights for policy 0, policy_version 164680 (0.0008) [2023-12-26 16:34:00,514][105692] Updated weights for policy 0, policy_version 164690 (0.0009) [2023-12-26 16:34:00,551][105620] Updated weights for policy 1, policy_version 165578 (0.0008) [2023-12-26 16:34:00,608][105620] Updated weights for policy 1, policy_version 165588 (0.0005) [2023-12-26 16:34:00,661][105620] Updated weights for policy 1, policy_version 165598 (0.0005) [2023-12-26 16:34:00,718][105620] Updated weights for policy 1, policy_version 165608 (0.0005) [2023-12-26 16:34:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 84574208. Throughput: 0: 9969.9, 1: 9865.4. Samples: 84545468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:34:01,062][104569] Avg episode reward: [(0, '9351.783'), (1, '9350.017')] [2023-12-26 16:34:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000164696_42172416.pth... [2023-12-26 16:34:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000165608_42401792.pth... [2023-12-26 16:34:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000164488_42115072.pth [2023-12-26 16:34:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000163544_41877504.pth [2023-12-26 16:34:01,312][105620] Updated weights for policy 1, policy_version 165618 (0.0006) [2023-12-26 16:34:01,314][105692] Updated weights for policy 0, policy_version 164700 (0.0007) [2023-12-26 16:34:01,374][105692] Updated weights for policy 0, policy_version 164710 (0.0008) [2023-12-26 16:34:01,374][105620] Updated weights for policy 1, policy_version 165628 (0.0006) [2023-12-26 16:34:01,436][105620] Updated weights for policy 1, policy_version 165638 (0.0006) [2023-12-26 16:34:01,443][105692] Updated weights for policy 0, policy_version 164720 (0.0009) [2023-12-26 16:34:02,077][105692] Updated weights for policy 0, policy_version 164730 (0.0007) [2023-12-26 16:34:02,134][105692] Updated weights for policy 0, policy_version 164740 (0.0008) [2023-12-26 16:34:02,169][105620] Updated weights for policy 1, policy_version 165648 (0.0008) [2023-12-26 16:34:02,189][105692] Updated weights for policy 0, policy_version 164750 (0.0005) [2023-12-26 16:34:02,224][105620] Updated weights for policy 1, policy_version 165658 (0.0009) [2023-12-26 16:34:02,249][105692] Updated weights for policy 0, policy_version 164760 (0.0006) [2023-12-26 16:34:02,289][105620] Updated weights for policy 1, policy_version 165668 (0.0008) [2023-12-26 16:34:02,915][105692] Updated weights for policy 0, policy_version 164770 (0.0010) [2023-12-26 16:34:02,976][105692] Updated weights for policy 0, policy_version 164780 (0.0008) [2023-12-26 16:34:03,007][105620] Updated weights for policy 1, policy_version 165678 (0.0007) [2023-12-26 16:34:03,037][105692] Updated weights for policy 0, policy_version 164790 (0.0010) [2023-12-26 16:34:03,070][105620] Updated weights for policy 1, policy_version 165688 (0.0005) [2023-12-26 16:34:03,129][105620] Updated weights for policy 1, policy_version 165698 (0.0005) [2023-12-26 16:34:03,668][105620] Updated weights for policy 1, policy_version 165708 (0.0006) [2023-12-26 16:34:03,718][105620] Updated weights for policy 1, policy_version 165718 (0.0008) [2023-12-26 16:34:03,746][105692] Updated weights for policy 0, policy_version 164800 (0.0010) [2023-12-26 16:34:03,768][105620] Updated weights for policy 1, policy_version 165728 (0.0008) [2023-12-26 16:34:03,801][105692] Updated weights for policy 0, policy_version 164810 (0.0010) [2023-12-26 16:34:03,855][105692] Updated weights for policy 0, policy_version 164820 (0.0010) [2023-12-26 16:34:04,531][105620] Updated weights for policy 1, policy_version 165738 (0.0005) [2023-12-26 16:34:04,565][105692] Updated weights for policy 0, policy_version 164830 (0.0010) [2023-12-26 16:34:04,589][105620] Updated weights for policy 1, policy_version 165748 (0.0007) [2023-12-26 16:34:04,623][105692] Updated weights for policy 0, policy_version 164840 (0.0009) [2023-12-26 16:34:04,646][105620] Updated weights for policy 1, policy_version 165758 (0.0006) [2023-12-26 16:34:04,679][105692] Updated weights for policy 0, policy_version 164850 (0.0009) [2023-12-26 16:34:04,709][105620] Updated weights for policy 1, policy_version 165768 (0.0005) [2023-12-26 16:34:05,259][105620] Updated weights for policy 1, policy_version 165778 (0.0010) [2023-12-26 16:34:05,324][105620] Updated weights for policy 1, policy_version 165788 (0.0011) [2023-12-26 16:34:05,379][105620] Updated weights for policy 1, policy_version 165798 (0.0010) [2023-12-26 16:34:05,433][105692] Updated weights for policy 0, policy_version 164860 (0.0010) [2023-12-26 16:34:05,481][105692] Updated weights for policy 0, policy_version 164870 (0.0010) [2023-12-26 16:34:05,528][105692] Updated weights for policy 0, policy_version 164880 (0.0010) [2023-12-26 16:34:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 84672512. Throughput: 0: 9986.1, 1: 9905.6. Samples: 84665900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:34:06,063][104569] Avg episode reward: [(0, '9352.650'), (1, '9349.381')] [2023-12-26 16:34:06,071][105620] Updated weights for policy 1, policy_version 165808 (0.0006) [2023-12-26 16:34:06,136][105620] Updated weights for policy 1, policy_version 165818 (0.0007) [2023-12-26 16:34:06,203][105620] Updated weights for policy 1, policy_version 165828 (0.0006) [2023-12-26 16:34:06,292][105692] Updated weights for policy 0, policy_version 164890 (0.0010) [2023-12-26 16:34:06,359][105692] Updated weights for policy 0, policy_version 164900 (0.0010) [2023-12-26 16:34:06,431][105692] Updated weights for policy 0, policy_version 164910 (0.0010) [2023-12-26 16:34:06,491][105692] Updated weights for policy 0, policy_version 164920 (0.0010) [2023-12-26 16:34:06,743][105620] Updated weights for policy 1, policy_version 165838 (0.0007) [2023-12-26 16:34:06,801][105620] Updated weights for policy 1, policy_version 165848 (0.0006) [2023-12-26 16:34:06,861][105620] Updated weights for policy 1, policy_version 165858 (0.0006) [2023-12-26 16:34:07,167][105692] Updated weights for policy 0, policy_version 164930 (0.0008) [2023-12-26 16:34:07,237][105692] Updated weights for policy 0, policy_version 164940 (0.0009) [2023-12-26 16:34:07,303][105692] Updated weights for policy 0, policy_version 164950 (0.0007) [2023-12-26 16:34:07,478][105620] Updated weights for policy 1, policy_version 165868 (0.0007) [2023-12-26 16:34:07,540][105620] Updated weights for policy 1, policy_version 165878 (0.0010) [2023-12-26 16:34:07,598][105620] Updated weights for policy 1, policy_version 165888 (0.0010) [2023-12-26 16:34:07,861][105692] Updated weights for policy 0, policy_version 164960 (0.0009) [2023-12-26 16:34:07,919][105692] Updated weights for policy 0, policy_version 164970 (0.0010) [2023-12-26 16:34:07,975][105692] Updated weights for policy 0, policy_version 164980 (0.0008) [2023-12-26 16:34:08,334][105620] Updated weights for policy 1, policy_version 165898 (0.0010) [2023-12-26 16:34:08,404][105620] Updated weights for policy 1, policy_version 165908 (0.0007) [2023-12-26 16:34:08,469][105620] Updated weights for policy 1, policy_version 165918 (0.0009) [2023-12-26 16:34:08,531][105620] Updated weights for policy 1, policy_version 165928 (0.0009) [2023-12-26 16:34:08,606][105692] Updated weights for policy 0, policy_version 164990 (0.0005) [2023-12-26 16:34:08,666][105692] Updated weights for policy 0, policy_version 165000 (0.0008) [2023-12-26 16:34:08,724][105692] Updated weights for policy 0, policy_version 165010 (0.0009) [2023-12-26 16:34:09,180][105620] Updated weights for policy 1, policy_version 165938 (0.0005) [2023-12-26 16:34:09,244][105620] Updated weights for policy 1, policy_version 165948 (0.0008) [2023-12-26 16:34:09,307][105620] Updated weights for policy 1, policy_version 165958 (0.0008) [2023-12-26 16:34:09,465][105692] Updated weights for policy 0, policy_version 165020 (0.0008) [2023-12-26 16:34:09,521][105692] Updated weights for policy 0, policy_version 165030 (0.0009) [2023-12-26 16:34:09,575][105692] Updated weights for policy 0, policy_version 165040 (0.0007) [2023-12-26 16:34:10,074][105620] Updated weights for policy 1, policy_version 165968 (0.0009) [2023-12-26 16:34:10,141][105620] Updated weights for policy 1, policy_version 165978 (0.0009) [2023-12-26 16:34:10,205][105620] Updated weights for policy 1, policy_version 165988 (0.0009) [2023-12-26 16:34:10,315][105692] Updated weights for policy 0, policy_version 165050 (0.0009) [2023-12-26 16:34:10,382][105692] Updated weights for policy 0, policy_version 165060 (0.0009) [2023-12-26 16:34:10,431][105692] Updated weights for policy 0, policy_version 165070 (0.0009) [2023-12-26 16:34:10,483][105692] Updated weights for policy 0, policy_version 165080 (0.0009) [2023-12-26 16:34:10,917][105620] Updated weights for policy 1, policy_version 165998 (0.0009) [2023-12-26 16:34:10,963][105620] Updated weights for policy 1, policy_version 166008 (0.0008) [2023-12-26 16:34:11,016][105620] Updated weights for policy 1, policy_version 166018 (0.0009) [2023-12-26 16:34:11,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 84779008. Throughput: 0: 9978.7, 1: 9856.0. Samples: 84785728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:34:11,063][104569] Avg episode reward: [(0, '9351.524'), (1, '9168.767')] [2023-12-26 16:34:11,224][105692] Updated weights for policy 0, policy_version 165090 (0.0011) [2023-12-26 16:34:11,293][105692] Updated weights for policy 0, policy_version 165100 (0.0011) [2023-12-26 16:34:11,353][105692] Updated weights for policy 0, policy_version 165110 (0.0011) [2023-12-26 16:34:11,779][105620] Updated weights for policy 1, policy_version 166028 (0.0008) [2023-12-26 16:34:11,834][105620] Updated weights for policy 1, policy_version 166038 (0.0007) [2023-12-26 16:34:11,895][105620] Updated weights for policy 1, policy_version 166048 (0.0008) [2023-12-26 16:34:12,104][105692] Updated weights for policy 0, policy_version 165120 (0.0011) [2023-12-26 16:34:12,161][105692] Updated weights for policy 0, policy_version 165130 (0.0011) [2023-12-26 16:34:12,214][105692] Updated weights for policy 0, policy_version 165140 (0.0010) [2023-12-26 16:34:12,668][105620] Updated weights for policy 1, policy_version 166058 (0.0008) [2023-12-26 16:34:12,727][105620] Updated weights for policy 1, policy_version 166068 (0.0008) [2023-12-26 16:34:12,782][105620] Updated weights for policy 1, policy_version 166078 (0.0008) [2023-12-26 16:34:12,840][105620] Updated weights for policy 1, policy_version 166088 (0.0009) [2023-12-26 16:34:12,997][105692] Updated weights for policy 0, policy_version 165150 (0.0010) [2023-12-26 16:34:13,062][105692] Updated weights for policy 0, policy_version 165160 (0.0009) [2023-12-26 16:34:13,117][105692] Updated weights for policy 0, policy_version 165170 (0.0007) [2023-12-26 16:34:13,674][105620] Updated weights for policy 1, policy_version 166098 (0.0010) [2023-12-26 16:34:13,696][105692] Updated weights for policy 0, policy_version 165180 (0.0006) [2023-12-26 16:34:13,732][105620] Updated weights for policy 1, policy_version 166108 (0.0010) [2023-12-26 16:34:13,747][105692] Updated weights for policy 0, policy_version 165190 (0.0005) [2023-12-26 16:34:13,790][105620] Updated weights for policy 1, policy_version 166118 (0.0010) [2023-12-26 16:34:13,794][105692] Updated weights for policy 0, policy_version 165200 (0.0008) [2023-12-26 16:34:14,435][105692] Updated weights for policy 0, policy_version 165210 (0.0008) [2023-12-26 16:34:14,501][105692] Updated weights for policy 0, policy_version 165220 (0.0009) [2023-12-26 16:34:14,565][105692] Updated weights for policy 0, policy_version 165230 (0.0008) [2023-12-26 16:34:14,568][105620] Updated weights for policy 1, policy_version 166128 (0.0006) [2023-12-26 16:34:14,620][105692] Updated weights for policy 0, policy_version 165240 (0.0008) [2023-12-26 16:34:14,621][105620] Updated weights for policy 1, policy_version 166138 (0.0006) [2023-12-26 16:34:14,677][105620] Updated weights for policy 1, policy_version 166148 (0.0006) [2023-12-26 16:34:15,371][105620] Updated weights for policy 1, policy_version 166158 (0.0008) [2023-12-26 16:34:15,400][105692] Updated weights for policy 0, policy_version 165250 (0.0008) [2023-12-26 16:34:15,431][105620] Updated weights for policy 1, policy_version 166168 (0.0009) [2023-12-26 16:34:15,454][105692] Updated weights for policy 0, policy_version 165260 (0.0007) [2023-12-26 16:34:15,489][105620] Updated weights for policy 1, policy_version 166178 (0.0008) [2023-12-26 16:34:15,511][105692] Updated weights for policy 0, policy_version 165270 (0.0007) [2023-12-26 16:34:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 84869120. Throughput: 0: 9991.4, 1: 9734.2. Samples: 84842584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:34:16,063][104569] Avg episode reward: [(0, '9351.068'), (1, '9168.380')] [2023-12-26 16:34:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000166184_42549248.pth... [2023-12-26 16:34:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000165272_42319872.pth... [2023-12-26 16:34:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000165032_42254336.pth [2023-12-26 16:34:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000164120_42024960.pth [2023-12-26 16:34:16,216][105692] Updated weights for policy 0, policy_version 165280 (0.0008) [2023-12-26 16:34:16,216][105620] Updated weights for policy 1, policy_version 166188 (0.0007) [2023-12-26 16:34:16,260][105692] Updated weights for policy 0, policy_version 165290 (0.0008) [2023-12-26 16:34:16,271][105620] Updated weights for policy 1, policy_version 166198 (0.0007) [2023-12-26 16:34:16,306][105692] Updated weights for policy 0, policy_version 165300 (0.0006) [2023-12-26 16:34:16,319][105620] Updated weights for policy 1, policy_version 166208 (0.0007) [2023-12-26 16:34:16,951][105692] Updated weights for policy 0, policy_version 165310 (0.0007) [2023-12-26 16:34:17,012][105692] Updated weights for policy 0, policy_version 165320 (0.0009) [2023-12-26 16:34:17,066][105692] Updated weights for policy 0, policy_version 165332 (0.0010) [2023-12-26 16:34:17,101][105620] Updated weights for policy 1, policy_version 166218 (0.0007) [2023-12-26 16:34:17,148][105620] Updated weights for policy 1, policy_version 166228 (0.0005) [2023-12-26 16:34:17,193][105620] Updated weights for policy 1, policy_version 166238 (0.0005) [2023-12-26 16:34:17,246][105620] Updated weights for policy 1, policy_version 166248 (0.0006) [2023-12-26 16:34:17,894][105692] Updated weights for policy 0, policy_version 165342 (0.0009) [2023-12-26 16:34:17,949][105692] Updated weights for policy 0, policy_version 165352 (0.0008) [2023-12-26 16:34:17,953][105620] Updated weights for policy 1, policy_version 166258 (0.0008) [2023-12-26 16:34:18,006][105692] Updated weights for policy 0, policy_version 165362 (0.0009) [2023-12-26 16:34:18,014][105620] Updated weights for policy 1, policy_version 166268 (0.0006) [2023-12-26 16:34:18,065][105620] Updated weights for policy 1, policy_version 166278 (0.0010) [2023-12-26 16:34:18,753][105692] Updated weights for policy 0, policy_version 165372 (0.0007) [2023-12-26 16:34:18,777][105620] Updated weights for policy 1, policy_version 166288 (0.0007) [2023-12-26 16:34:18,820][105692] Updated weights for policy 0, policy_version 165382 (0.0005) [2023-12-26 16:34:18,822][105620] Updated weights for policy 1, policy_version 166298 (0.0007) [2023-12-26 16:34:18,869][105620] Updated weights for policy 1, policy_version 166308 (0.0007) [2023-12-26 16:34:18,883][105692] Updated weights for policy 0, policy_version 165392 (0.0006) [2023-12-26 16:34:19,571][105620] Updated weights for policy 1, policy_version 166318 (0.0009) [2023-12-26 16:34:19,617][105692] Updated weights for policy 0, policy_version 165402 (0.0007) [2023-12-26 16:34:19,630][105620] Updated weights for policy 1, policy_version 166328 (0.0010) [2023-12-26 16:34:19,678][105692] Updated weights for policy 0, policy_version 165412 (0.0005) [2023-12-26 16:34:19,690][105620] Updated weights for policy 1, policy_version 166338 (0.0010) [2023-12-26 16:34:19,744][105692] Updated weights for policy 0, policy_version 165422 (0.0006) [2023-12-26 16:34:19,810][105692] Updated weights for policy 0, policy_version 165432 (0.0008) [2023-12-26 16:34:20,443][105620] Updated weights for policy 1, policy_version 166348 (0.0009) [2023-12-26 16:34:20,507][105620] Updated weights for policy 1, policy_version 166358 (0.0011) [2023-12-26 16:34:20,538][105692] Updated weights for policy 0, policy_version 165442 (0.0006) [2023-12-26 16:34:20,565][105620] Updated weights for policy 1, policy_version 166368 (0.0011) [2023-12-26 16:34:20,602][105692] Updated weights for policy 0, policy_version 165452 (0.0008) [2023-12-26 16:34:20,652][105692] Updated weights for policy 0, policy_version 165462 (0.0007) [2023-12-26 16:34:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 84967424. Throughput: 0: 9931.3, 1: 9793.4. Samples: 84958052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:34:21,062][104569] Avg episode reward: [(0, '9350.787'), (1, '9258.163')] [2023-12-26 16:34:21,300][105620] Updated weights for policy 1, policy_version 166378 (0.0010) [2023-12-26 16:34:21,366][105620] Updated weights for policy 1, policy_version 166388 (0.0010) [2023-12-26 16:34:21,430][105692] Updated weights for policy 0, policy_version 165472 (0.0007) [2023-12-26 16:34:21,432][105620] Updated weights for policy 1, policy_version 166398 (0.0010) [2023-12-26 16:34:21,494][105692] Updated weights for policy 0, policy_version 165482 (0.0008) [2023-12-26 16:34:21,495][105620] Updated weights for policy 1, policy_version 166408 (0.0010) [2023-12-26 16:34:21,547][105692] Updated weights for policy 0, policy_version 165492 (0.0006) [2023-12-26 16:34:22,171][105692] Updated weights for policy 0, policy_version 165502 (0.0007) [2023-12-26 16:34:22,229][105692] Updated weights for policy 0, policy_version 165512 (0.0007) [2023-12-26 16:34:22,252][105620] Updated weights for policy 1, policy_version 166418 (0.0010) [2023-12-26 16:34:22,290][105692] Updated weights for policy 0, policy_version 165522 (0.0007) [2023-12-26 16:34:22,318][105620] Updated weights for policy 1, policy_version 166428 (0.0009) [2023-12-26 16:34:22,386][105620] Updated weights for policy 1, policy_version 166438 (0.0010) [2023-12-26 16:34:23,079][105692] Updated weights for policy 0, policy_version 165532 (0.0007) [2023-12-26 16:34:23,086][105620] Updated weights for policy 1, policy_version 166448 (0.0011) [2023-12-26 16:34:23,131][105620] Updated weights for policy 1, policy_version 166458 (0.0010) [2023-12-26 16:34:23,133][105692] Updated weights for policy 0, policy_version 165542 (0.0005) [2023-12-26 16:34:23,186][105692] Updated weights for policy 0, policy_version 165552 (0.0006) [2023-12-26 16:34:23,191][105620] Updated weights for policy 1, policy_version 166468 (0.0011) [2023-12-26 16:34:23,873][105620] Updated weights for policy 1, policy_version 166478 (0.0011) [2023-12-26 16:34:23,934][105620] Updated weights for policy 1, policy_version 166488 (0.0010) [2023-12-26 16:34:23,987][105692] Updated weights for policy 0, policy_version 165562 (0.0008) [2023-12-26 16:34:23,996][105620] Updated weights for policy 1, policy_version 166498 (0.0010) [2023-12-26 16:34:24,032][105692] Updated weights for policy 0, policy_version 165572 (0.0009) [2023-12-26 16:34:24,080][105692] Updated weights for policy 0, policy_version 165582 (0.0008) [2023-12-26 16:34:24,128][105692] Updated weights for policy 0, policy_version 165592 (0.0008) [2023-12-26 16:34:24,734][105620] Updated weights for policy 1, policy_version 166508 (0.0010) [2023-12-26 16:34:24,799][105620] Updated weights for policy 1, policy_version 166518 (0.0010) [2023-12-26 16:34:24,807][105692] Updated weights for policy 0, policy_version 165602 (0.0005) [2023-12-26 16:34:24,850][105620] Updated weights for policy 1, policy_version 166528 (0.0010) [2023-12-26 16:34:24,855][105692] Updated weights for policy 0, policy_version 165612 (0.0005) [2023-12-26 16:34:24,907][105692] Updated weights for policy 0, policy_version 165622 (0.0005) [2023-12-26 16:34:25,490][105692] Updated weights for policy 0, policy_version 165632 (0.0008) [2023-12-26 16:34:25,548][105692] Updated weights for policy 0, policy_version 165642 (0.0008) [2023-12-26 16:34:25,581][105620] Updated weights for policy 1, policy_version 166538 (0.0010) [2023-12-26 16:34:25,606][105692] Updated weights for policy 0, policy_version 165652 (0.0006) [2023-12-26 16:34:25,639][105620] Updated weights for policy 1, policy_version 166548 (0.0010) [2023-12-26 16:34:25,696][105620] Updated weights for policy 1, policy_version 166558 (0.0010) [2023-12-26 16:34:25,752][105620] Updated weights for policy 1, policy_version 166568 (0.0010) [2023-12-26 16:34:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 85065728. Throughput: 0: 9946.3, 1: 9832.8. Samples: 85074556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:34:26,063][104569] Avg episode reward: [(0, '9259.167'), (1, '9074.999')] [2023-12-26 16:34:26,263][105692] Updated weights for policy 0, policy_version 165662 (0.0007) [2023-12-26 16:34:26,315][105692] Updated weights for policy 0, policy_version 165672 (0.0008) [2023-12-26 16:34:26,366][105692] Updated weights for policy 0, policy_version 165682 (0.0008) [2023-12-26 16:34:26,463][105620] Updated weights for policy 1, policy_version 166578 (0.0010) [2023-12-26 16:34:26,516][105620] Updated weights for policy 1, policy_version 166588 (0.0010) [2023-12-26 16:34:26,573][105620] Updated weights for policy 1, policy_version 166598 (0.0010) [2023-12-26 16:34:27,073][105692] Updated weights for policy 0, policy_version 165692 (0.0008) [2023-12-26 16:34:27,133][105692] Updated weights for policy 0, policy_version 165702 (0.0009) [2023-12-26 16:34:27,187][105692] Updated weights for policy 0, policy_version 165712 (0.0009) [2023-12-26 16:34:27,242][105620] Updated weights for policy 1, policy_version 166608 (0.0007) [2023-12-26 16:34:27,288][105620] Updated weights for policy 1, policy_version 166618 (0.0008) [2023-12-26 16:34:27,346][105620] Updated weights for policy 1, policy_version 166628 (0.0008) [2023-12-26 16:34:27,944][105620] Updated weights for policy 1, policy_version 166638 (0.0009) [2023-12-26 16:34:27,984][105620] Updated weights for policy 1, policy_version 166648 (0.0006) [2023-12-26 16:34:28,027][105692] Updated weights for policy 0, policy_version 165722 (0.0008) [2023-12-26 16:34:28,032][105620] Updated weights for policy 1, policy_version 166658 (0.0008) [2023-12-26 16:34:28,085][105692] Updated weights for policy 0, policy_version 165732 (0.0008) [2023-12-26 16:34:28,138][105692] Updated weights for policy 0, policy_version 165742 (0.0010) [2023-12-26 16:34:28,194][105692] Updated weights for policy 0, policy_version 165752 (0.0009) [2023-12-26 16:34:28,594][105620] Updated weights for policy 1, policy_version 166668 (0.0008) [2023-12-26 16:34:28,649][105620] Updated weights for policy 1, policy_version 166678 (0.0009) [2023-12-26 16:34:28,705][105620] Updated weights for policy 1, policy_version 166688 (0.0005) [2023-12-26 16:34:29,085][105692] Updated weights for policy 0, policy_version 165762 (0.0009) [2023-12-26 16:34:29,140][105692] Updated weights for policy 0, policy_version 165772 (0.0008) [2023-12-26 16:34:29,187][105692] Updated weights for policy 0, policy_version 165782 (0.0008) [2023-12-26 16:34:29,312][105620] Updated weights for policy 1, policy_version 166698 (0.0006) [2023-12-26 16:34:29,385][105620] Updated weights for policy 1, policy_version 166708 (0.0009) [2023-12-26 16:34:29,443][105620] Updated weights for policy 1, policy_version 166718 (0.0008) [2023-12-26 16:34:29,509][105620] Updated weights for policy 1, policy_version 166728 (0.0007) [2023-12-26 16:34:30,005][105692] Updated weights for policy 0, policy_version 165792 (0.0009) [2023-12-26 16:34:30,069][105692] Updated weights for policy 0, policy_version 165802 (0.0008) [2023-12-26 16:34:30,127][105692] Updated weights for policy 0, policy_version 165812 (0.0008) [2023-12-26 16:34:30,211][105620] Updated weights for policy 1, policy_version 166738 (0.0010) [2023-12-26 16:34:30,270][105620] Updated weights for policy 1, policy_version 166748 (0.0010) [2023-12-26 16:34:30,328][105620] Updated weights for policy 1, policy_version 166758 (0.0010) [2023-12-26 16:34:30,756][105692] Updated weights for policy 0, policy_version 165822 (0.0009) [2023-12-26 16:34:30,809][105692] Updated weights for policy 0, policy_version 165832 (0.0010) [2023-12-26 16:34:30,866][105692] Updated weights for policy 0, policy_version 165842 (0.0010) [2023-12-26 16:34:30,936][105620] Updated weights for policy 1, policy_version 166768 (0.0007) [2023-12-26 16:34:30,993][105620] Updated weights for policy 1, policy_version 166778 (0.0008) [2023-12-26 16:34:31,060][105620] Updated weights for policy 1, policy_version 166788 (0.0008) [2023-12-26 16:34:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 85164032. Throughput: 0: 9857.9, 1: 9933.1. Samples: 85135664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:34:31,062][104569] Avg episode reward: [(0, '9173.098'), (1, '9257.240')] [2023-12-26 16:34:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000165848_42467328.pth... [2023-12-26 16:34:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000164696_42172416.pth [2023-12-26 16:34:31,084][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000166792_42704896.pth... [2023-12-26 16:34:31,089][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000165608_42401792.pth [2023-12-26 16:34:31,719][105620] Updated weights for policy 1, policy_version 166798 (0.0007) [2023-12-26 16:34:31,726][105692] Updated weights for policy 0, policy_version 165852 (0.0008) [2023-12-26 16:34:31,781][105620] Updated weights for policy 1, policy_version 166808 (0.0009) [2023-12-26 16:34:31,788][105692] Updated weights for policy 0, policy_version 165862 (0.0008) [2023-12-26 16:34:31,831][105620] Updated weights for policy 1, policy_version 166818 (0.0006) [2023-12-26 16:34:31,845][105692] Updated weights for policy 0, policy_version 165872 (0.0008) [2023-12-26 16:34:32,587][105620] Updated weights for policy 1, policy_version 166828 (0.0007) [2023-12-26 16:34:32,591][105692] Updated weights for policy 0, policy_version 165882 (0.0008) [2023-12-26 16:34:32,640][105692] Updated weights for policy 0, policy_version 165892 (0.0007) [2023-12-26 16:34:32,642][105620] Updated weights for policy 1, policy_version 166838 (0.0008) [2023-12-26 16:34:32,700][105692] Updated weights for policy 0, policy_version 165902 (0.0007) [2023-12-26 16:34:32,701][105620] Updated weights for policy 1, policy_version 166848 (0.0008) [2023-12-26 16:34:32,743][105692] Updated weights for policy 0, policy_version 165912 (0.0008) [2023-12-26 16:34:33,435][105620] Updated weights for policy 1, policy_version 166858 (0.0008) [2023-12-26 16:34:33,476][105692] Updated weights for policy 0, policy_version 165922 (0.0005) [2023-12-26 16:34:33,497][105620] Updated weights for policy 1, policy_version 166868 (0.0010) [2023-12-26 16:34:33,519][105692] Updated weights for policy 0, policy_version 165932 (0.0005) [2023-12-26 16:34:33,558][105620] Updated weights for policy 1, policy_version 166878 (0.0010) [2023-12-26 16:34:33,568][105692] Updated weights for policy 0, policy_version 165942 (0.0006) [2023-12-26 16:34:33,614][105620] Updated weights for policy 1, policy_version 166888 (0.0006) [2023-12-26 16:34:34,203][105620] Updated weights for policy 1, policy_version 166898 (0.0009) [2023-12-26 16:34:34,233][105692] Updated weights for policy 0, policy_version 165952 (0.0008) [2023-12-26 16:34:34,259][105620] Updated weights for policy 1, policy_version 166908 (0.0011) [2023-12-26 16:34:34,288][105692] Updated weights for policy 0, policy_version 165962 (0.0007) [2023-12-26 16:34:34,311][105620] Updated weights for policy 1, policy_version 166918 (0.0011) [2023-12-26 16:34:34,346][105692] Updated weights for policy 0, policy_version 165972 (0.0006) [2023-12-26 16:34:34,963][105620] Updated weights for policy 1, policy_version 166928 (0.0011) [2023-12-26 16:34:35,008][105620] Updated weights for policy 1, policy_version 166938 (0.0010) [2023-12-26 16:34:35,057][105620] Updated weights for policy 1, policy_version 166948 (0.0010) [2023-12-26 16:34:35,154][105692] Updated weights for policy 0, policy_version 165982 (0.0009) [2023-12-26 16:34:35,211][105692] Updated weights for policy 0, policy_version 165992 (0.0010) [2023-12-26 16:34:35,268][105692] Updated weights for policy 0, policy_version 166002 (0.0009) [2023-12-26 16:34:35,645][105620] Updated weights for policy 1, policy_version 166958 (0.0007) [2023-12-26 16:34:35,703][105620] Updated weights for policy 1, policy_version 166968 (0.0006) [2023-12-26 16:34:35,764][105620] Updated weights for policy 1, policy_version 166978 (0.0010) [2023-12-26 16:34:36,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 85262336. Throughput: 0: 9737.7, 1: 9969.1. Samples: 85253324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:34:36,062][104569] Avg episode reward: [(0, '9173.560'), (1, '9349.194')] [2023-12-26 16:34:36,131][105692] Updated weights for policy 0, policy_version 166012 (0.0010) [2023-12-26 16:34:36,196][105692] Updated weights for policy 0, policy_version 166022 (0.0010) [2023-12-26 16:34:36,262][105692] Updated weights for policy 0, policy_version 166032 (0.0009) [2023-12-26 16:34:36,437][105620] Updated weights for policy 1, policy_version 166988 (0.0008) [2023-12-26 16:34:36,501][105620] Updated weights for policy 1, policy_version 166998 (0.0010) [2023-12-26 16:34:36,574][105620] Updated weights for policy 1, policy_version 167008 (0.0008) [2023-12-26 16:34:36,921][105692] Updated weights for policy 0, policy_version 166042 (0.0008) [2023-12-26 16:34:36,991][105692] Updated weights for policy 0, policy_version 166052 (0.0010) [2023-12-26 16:34:37,057][105692] Updated weights for policy 0, policy_version 166062 (0.0011) [2023-12-26 16:34:37,117][105692] Updated weights for policy 0, policy_version 166072 (0.0010) [2023-12-26 16:34:37,133][105620] Updated weights for policy 1, policy_version 167018 (0.0006) [2023-12-26 16:34:37,187][105620] Updated weights for policy 1, policy_version 167028 (0.0008) [2023-12-26 16:34:37,243][105620] Updated weights for policy 1, policy_version 167038 (0.0008) [2023-12-26 16:34:37,307][105620] Updated weights for policy 1, policy_version 167048 (0.0006) [2023-12-26 16:34:37,841][105692] Updated weights for policy 0, policy_version 166082 (0.0010) [2023-12-26 16:34:37,891][105692] Updated weights for policy 0, policy_version 166092 (0.0009) [2023-12-26 16:34:37,951][105692] Updated weights for policy 0, policy_version 166102 (0.0006) [2023-12-26 16:34:37,965][105620] Updated weights for policy 1, policy_version 167058 (0.0008) [2023-12-26 16:34:38,030][105620] Updated weights for policy 1, policy_version 167068 (0.0009) [2023-12-26 16:34:38,087][105620] Updated weights for policy 1, policy_version 167078 (0.0008) [2023-12-26 16:34:38,718][105692] Updated weights for policy 0, policy_version 166112 (0.0008) [2023-12-26 16:34:38,773][105692] Updated weights for policy 0, policy_version 166122 (0.0009) [2023-12-26 16:34:38,823][105692] Updated weights for policy 0, policy_version 166132 (0.0008) [2023-12-26 16:34:38,849][105620] Updated weights for policy 1, policy_version 167088 (0.0006) [2023-12-26 16:34:38,913][105620] Updated weights for policy 1, policy_version 167098 (0.0009) [2023-12-26 16:34:38,975][105620] Updated weights for policy 1, policy_version 167108 (0.0009) [2023-12-26 16:34:39,643][105620] Updated weights for policy 1, policy_version 167118 (0.0007) [2023-12-26 16:34:39,644][105692] Updated weights for policy 0, policy_version 166142 (0.0007) [2023-12-26 16:34:39,696][105692] Updated weights for policy 0, policy_version 166152 (0.0008) [2023-12-26 16:34:39,705][105620] Updated weights for policy 1, policy_version 167128 (0.0005) [2023-12-26 16:34:39,747][105692] Updated weights for policy 0, policy_version 166162 (0.0008) [2023-12-26 16:34:39,769][105620] Updated weights for policy 1, policy_version 167138 (0.0008) [2023-12-26 16:34:40,487][105692] Updated weights for policy 0, policy_version 166172 (0.0009) [2023-12-26 16:34:40,501][105620] Updated weights for policy 1, policy_version 167148 (0.0008) [2023-12-26 16:34:40,544][105692] Updated weights for policy 0, policy_version 166182 (0.0010) [2023-12-26 16:34:40,554][105620] Updated weights for policy 1, policy_version 167158 (0.0006) [2023-12-26 16:34:40,603][105620] Updated weights for policy 1, policy_version 167168 (0.0006) [2023-12-26 16:34:40,605][105692] Updated weights for policy 0, policy_version 166192 (0.0010) [2023-12-26 16:34:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 85360640. Throughput: 0: 9631.5, 1: 10042.0. Samples: 85369700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:34:41,062][104569] Avg episode reward: [(0, '9175.044'), (1, '9258.940')] [2023-12-26 16:34:41,279][105692] Updated weights for policy 0, policy_version 166202 (0.0010) [2023-12-26 16:34:41,333][105692] Updated weights for policy 0, policy_version 166212 (0.0006) [2023-12-26 16:34:41,401][105692] Updated weights for policy 0, policy_version 166222 (0.0007) [2023-12-26 16:34:41,433][105620] Updated weights for policy 1, policy_version 167178 (0.0005) [2023-12-26 16:34:41,464][105692] Updated weights for policy 0, policy_version 166232 (0.0010) [2023-12-26 16:34:41,499][105620] Updated weights for policy 1, policy_version 167188 (0.0006) [2023-12-26 16:34:41,560][105620] Updated weights for policy 1, policy_version 167198 (0.0006) [2023-12-26 16:34:41,627][105620] Updated weights for policy 1, policy_version 167208 (0.0008) [2023-12-26 16:34:42,156][105692] Updated weights for policy 0, policy_version 166242 (0.0008) [2023-12-26 16:34:42,215][105692] Updated weights for policy 0, policy_version 166252 (0.0005) [2023-12-26 16:34:42,265][105620] Updated weights for policy 1, policy_version 167218 (0.0009) [2023-12-26 16:34:42,279][105692] Updated weights for policy 0, policy_version 166262 (0.0007) [2023-12-26 16:34:42,333][105620] Updated weights for policy 1, policy_version 167228 (0.0008) [2023-12-26 16:34:42,402][105620] Updated weights for policy 1, policy_version 167238 (0.0008) [2023-12-26 16:34:42,893][105692] Updated weights for policy 0, policy_version 166272 (0.0006) [2023-12-26 16:34:42,952][105692] Updated weights for policy 0, policy_version 166282 (0.0006) [2023-12-26 16:34:43,016][105692] Updated weights for policy 0, policy_version 166292 (0.0009) [2023-12-26 16:34:43,179][105620] Updated weights for policy 1, policy_version 167248 (0.0009) [2023-12-26 16:34:43,231][105620] Updated weights for policy 1, policy_version 167258 (0.0009) [2023-12-26 16:34:43,313][105620] Updated weights for policy 1, policy_version 167269 (0.0010) [2023-12-26 16:34:43,572][105692] Updated weights for policy 0, policy_version 166302 (0.0007) [2023-12-26 16:34:43,624][105692] Updated weights for policy 0, policy_version 166312 (0.0006) [2023-12-26 16:34:43,680][105692] Updated weights for policy 0, policy_version 166322 (0.0007) [2023-12-26 16:34:44,163][105620] Updated weights for policy 1, policy_version 167279 (0.0009) [2023-12-26 16:34:44,225][105620] Updated weights for policy 1, policy_version 167289 (0.0009) [2023-12-26 16:34:44,289][105620] Updated weights for policy 1, policy_version 167299 (0.0009) [2023-12-26 16:34:44,327][105692] Updated weights for policy 0, policy_version 166332 (0.0006) [2023-12-26 16:34:44,387][105692] Updated weights for policy 0, policy_version 166342 (0.0009) [2023-12-26 16:34:44,447][105692] Updated weights for policy 0, policy_version 166352 (0.0009) [2023-12-26 16:34:45,015][105620] Updated weights for policy 1, policy_version 167309 (0.0007) [2023-12-26 16:34:45,081][105620] Updated weights for policy 1, policy_version 167319 (0.0006) [2023-12-26 16:34:45,147][105620] Updated weights for policy 1, policy_version 167329 (0.0010) [2023-12-26 16:34:45,240][105692] Updated weights for policy 0, policy_version 166362 (0.0009) [2023-12-26 16:34:45,305][105692] Updated weights for policy 0, policy_version 166372 (0.0009) [2023-12-26 16:34:45,366][105692] Updated weights for policy 0, policy_version 166382 (0.0009) [2023-12-26 16:34:45,430][105692] Updated weights for policy 0, policy_version 166392 (0.0010) [2023-12-26 16:34:45,878][105620] Updated weights for policy 1, policy_version 167339 (0.0010) [2023-12-26 16:34:45,938][105620] Updated weights for policy 1, policy_version 167349 (0.0009) [2023-12-26 16:34:46,002][105620] Updated weights for policy 1, policy_version 167359 (0.0009) [2023-12-26 16:34:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.9, 300 sec: 19633.0). Total num frames: 85458944. Throughput: 0: 9681.5, 1: 9951.7. Samples: 85428964. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 16:34:46,062][104569] Avg episode reward: [(0, '9261.890'), (1, '9170.880')] [2023-12-26 16:34:46,065][105692] Updated weights for policy 0, policy_version 166402 (0.0006) [2023-12-26 16:34:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000167368_42852352.pth... [2023-12-26 16:34:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000166184_42549248.pth [2023-12-26 16:34:46,123][105692] Updated weights for policy 0, policy_version 166412 (0.0009) [2023-12-26 16:34:46,167][105692] Updated weights for policy 0, policy_version 166422 (0.0006) [2023-12-26 16:34:46,175][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000166424_42614784.pth... [2023-12-26 16:34:46,178][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000165272_42319872.pth [2023-12-26 16:34:46,796][105692] Updated weights for policy 0, policy_version 166432 (0.0008) [2023-12-26 16:34:46,802][105620] Updated weights for policy 1, policy_version 167369 (0.0008) [2023-12-26 16:34:46,861][105692] Updated weights for policy 0, policy_version 166442 (0.0006) [2023-12-26 16:34:46,863][105620] Updated weights for policy 1, policy_version 167379 (0.0007) [2023-12-26 16:34:46,913][105692] Updated weights for policy 0, policy_version 166452 (0.0006) [2023-12-26 16:34:46,924][105620] Updated weights for policy 1, policy_version 167389 (0.0008) [2023-12-26 16:34:46,987][105620] Updated weights for policy 1, policy_version 167399 (0.0009) [2023-12-26 16:34:47,619][105692] Updated weights for policy 0, policy_version 166462 (0.0009) [2023-12-26 16:34:47,679][105692] Updated weights for policy 0, policy_version 166472 (0.0011) [2023-12-26 16:34:47,739][105692] Updated weights for policy 0, policy_version 166482 (0.0011) [2023-12-26 16:34:47,741][105620] Updated weights for policy 1, policy_version 167409 (0.0010) [2023-12-26 16:34:47,803][105620] Updated weights for policy 1, policy_version 167419 (0.0011) [2023-12-26 16:34:47,863][105620] Updated weights for policy 1, policy_version 167429 (0.0008) [2023-12-26 16:34:48,458][105692] Updated weights for policy 0, policy_version 166492 (0.0009) [2023-12-26 16:34:48,498][105620] Updated weights for policy 1, policy_version 167439 (0.0009) [2023-12-26 16:34:48,510][105692] Updated weights for policy 0, policy_version 166502 (0.0010) [2023-12-26 16:34:48,561][105620] Updated weights for policy 1, policy_version 167449 (0.0006) [2023-12-26 16:34:48,573][105692] Updated weights for policy 0, policy_version 166512 (0.0011) [2023-12-26 16:34:48,628][105620] Updated weights for policy 1, policy_version 167459 (0.0006) [2023-12-26 16:34:49,169][105692] Updated weights for policy 0, policy_version 166522 (0.0010) [2023-12-26 16:34:49,241][105692] Updated weights for policy 0, policy_version 166532 (0.0008) [2023-12-26 16:34:49,305][105692] Updated weights for policy 0, policy_version 166542 (0.0007) [2023-12-26 16:34:49,305][105620] Updated weights for policy 1, policy_version 167469 (0.0009) [2023-12-26 16:34:49,366][105692] Updated weights for policy 0, policy_version 166552 (0.0007) [2023-12-26 16:34:49,368][105620] Updated weights for policy 1, policy_version 167479 (0.0010) [2023-12-26 16:34:49,429][105620] Updated weights for policy 1, policy_version 167489 (0.0008) [2023-12-26 16:34:50,092][105692] Updated weights for policy 0, policy_version 166562 (0.0010) [2023-12-26 16:34:50,107][105620] Updated weights for policy 1, policy_version 167499 (0.0008) [2023-12-26 16:34:50,147][105692] Updated weights for policy 0, policy_version 166572 (0.0010) [2023-12-26 16:34:50,165][105620] Updated weights for policy 1, policy_version 167509 (0.0008) [2023-12-26 16:34:50,206][105692] Updated weights for policy 0, policy_version 166582 (0.0011) [2023-12-26 16:34:50,231][105620] Updated weights for policy 1, policy_version 167519 (0.0006) [2023-12-26 16:34:50,921][105620] Updated weights for policy 1, policy_version 167529 (0.0008) [2023-12-26 16:34:50,976][105692] Updated weights for policy 0, policy_version 166592 (0.0011) [2023-12-26 16:34:50,978][105620] Updated weights for policy 1, policy_version 167539 (0.0006) [2023-12-26 16:34:51,032][105692] Updated weights for policy 0, policy_version 166602 (0.0010) [2023-12-26 16:34:51,039][105620] Updated weights for policy 1, policy_version 167549 (0.0006) [2023-12-26 16:34:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 85549056. Throughput: 0: 9716.8, 1: 9869.3. Samples: 85547272. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 16:34:51,062][104569] Avg episode reward: [(0, '6131.542'), (1, '9171.720')] [2023-12-26 16:34:51,092][105620] Updated weights for policy 1, policy_version 167559 (0.0006) [2023-12-26 16:34:51,093][105692] Updated weights for policy 0, policy_version 166612 (0.0010) [2023-12-26 16:34:51,797][105692] Updated weights for policy 0, policy_version 166622 (0.0011) [2023-12-26 16:34:51,861][105692] Updated weights for policy 0, policy_version 166632 (0.0009) [2023-12-26 16:34:51,873][105620] Updated weights for policy 1, policy_version 167569 (0.0005) [2023-12-26 16:34:51,914][105692] Updated weights for policy 0, policy_version 166642 (0.0010) [2023-12-26 16:34:51,928][105620] Updated weights for policy 1, policy_version 167579 (0.0006) [2023-12-26 16:34:51,982][105620] Updated weights for policy 1, policy_version 167589 (0.0007) [2023-12-26 16:34:52,707][105692] Updated weights for policy 0, policy_version 166652 (0.0010) [2023-12-26 16:34:52,742][105620] Updated weights for policy 1, policy_version 167599 (0.0008) [2023-12-26 16:34:52,766][105692] Updated weights for policy 0, policy_version 166662 (0.0009) [2023-12-26 16:34:52,792][105620] Updated weights for policy 1, policy_version 167609 (0.0008) [2023-12-26 16:34:52,827][105692] Updated weights for policy 0, policy_version 166672 (0.0007) [2023-12-26 16:34:52,849][105620] Updated weights for policy 1, policy_version 167619 (0.0009) [2023-12-26 16:34:53,444][105692] Updated weights for policy 0, policy_version 166682 (0.0007) [2023-12-26 16:34:53,508][105692] Updated weights for policy 0, policy_version 166692 (0.0005) [2023-12-26 16:34:53,523][105620] Updated weights for policy 1, policy_version 167629 (0.0007) [2023-12-26 16:34:53,563][105692] Updated weights for policy 0, policy_version 166702 (0.0005) [2023-12-26 16:34:53,578][105620] Updated weights for policy 1, policy_version 167639 (0.0009) [2023-12-26 16:34:53,629][105692] Updated weights for policy 0, policy_version 166712 (0.0005) [2023-12-26 16:34:53,631][105620] Updated weights for policy 1, policy_version 167649 (0.0008) [2023-12-26 16:34:54,326][105692] Updated weights for policy 0, policy_version 166722 (0.0005) [2023-12-26 16:34:54,337][105620] Updated weights for policy 1, policy_version 167659 (0.0009) [2023-12-26 16:34:54,391][105692] Updated weights for policy 0, policy_version 166732 (0.0005) [2023-12-26 16:34:54,391][105620] Updated weights for policy 1, policy_version 167669 (0.0008) [2023-12-26 16:34:54,457][105620] Updated weights for policy 1, policy_version 167679 (0.0008) [2023-12-26 16:34:54,458][105692] Updated weights for policy 0, policy_version 166742 (0.0006) [2023-12-26 16:34:55,043][105692] Updated weights for policy 0, policy_version 166752 (0.0010) [2023-12-26 16:34:55,091][105620] Updated weights for policy 1, policy_version 167689 (0.0006) [2023-12-26 16:34:55,101][105692] Updated weights for policy 0, policy_version 166762 (0.0010) [2023-12-26 16:34:55,144][105620] Updated weights for policy 1, policy_version 167699 (0.0006) [2023-12-26 16:34:55,155][105692] Updated weights for policy 0, policy_version 166772 (0.0007) [2023-12-26 16:34:55,200][105620] Updated weights for policy 1, policy_version 167709 (0.0005) [2023-12-26 16:34:55,254][105620] Updated weights for policy 1, policy_version 167719 (0.0009) [2023-12-26 16:34:55,746][105692] Updated weights for policy 0, policy_version 166782 (0.0007) [2023-12-26 16:34:55,802][105692] Updated weights for policy 0, policy_version 166792 (0.0005) [2023-12-26 16:34:55,835][105620] Updated weights for policy 1, policy_version 167729 (0.0006) [2023-12-26 16:34:55,866][105692] Updated weights for policy 0, policy_version 166802 (0.0007) [2023-12-26 16:34:55,891][105620] Updated weights for policy 1, policy_version 167739 (0.0005) [2023-12-26 16:34:55,946][105620] Updated weights for policy 1, policy_version 167749 (0.0005) [2023-12-26 16:34:56,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 85663744. Throughput: 0: 9739.3, 1: 9863.6. Samples: 85667864. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 16:34:56,063][104569] Avg episode reward: [(0, '7262.670'), (1, '9078.352')] [2023-12-26 16:34:56,470][105620] Updated weights for policy 1, policy_version 167759 (0.0008) [2023-12-26 16:34:56,527][105620] Updated weights for policy 1, policy_version 167769 (0.0008) [2023-12-26 16:34:56,529][105692] Updated weights for policy 0, policy_version 166812 (0.0007) [2023-12-26 16:34:56,581][105620] Updated weights for policy 1, policy_version 167779 (0.0006) [2023-12-26 16:34:56,590][105692] Updated weights for policy 0, policy_version 166822 (0.0008) [2023-12-26 16:34:56,646][105692] Updated weights for policy 0, policy_version 166832 (0.0009) [2023-12-26 16:34:57,308][105620] Updated weights for policy 1, policy_version 167789 (0.0008) [2023-12-26 16:34:57,359][105620] Updated weights for policy 1, policy_version 167799 (0.0008) [2023-12-26 16:34:57,403][105692] Updated weights for policy 0, policy_version 166842 (0.0009) [2023-12-26 16:34:57,411][105620] Updated weights for policy 1, policy_version 167809 (0.0008) [2023-12-26 16:34:57,459][105692] Updated weights for policy 0, policy_version 166852 (0.0007) [2023-12-26 16:34:57,519][105692] Updated weights for policy 0, policy_version 166862 (0.0009) [2023-12-26 16:34:57,580][105692] Updated weights for policy 0, policy_version 166872 (0.0009) [2023-12-26 16:34:58,126][105620] Updated weights for policy 1, policy_version 167819 (0.0007) [2023-12-26 16:34:58,184][105620] Updated weights for policy 1, policy_version 167829 (0.0006) [2023-12-26 16:34:58,246][105620] Updated weights for policy 1, policy_version 167839 (0.0006) [2023-12-26 16:34:58,368][105692] Updated weights for policy 0, policy_version 166882 (0.0009) [2023-12-26 16:34:58,429][105692] Updated weights for policy 0, policy_version 166892 (0.0009) [2023-12-26 16:34:58,495][105692] Updated weights for policy 0, policy_version 166902 (0.0007) [2023-12-26 16:34:58,965][105620] Updated weights for policy 1, policy_version 167849 (0.0008) [2023-12-26 16:34:59,034][105620] Updated weights for policy 1, policy_version 167859 (0.0005) [2023-12-26 16:34:59,104][105620] Updated weights for policy 1, policy_version 167869 (0.0006) [2023-12-26 16:34:59,151][105692] Updated weights for policy 0, policy_version 166912 (0.0006) [2023-12-26 16:34:59,160][105620] Updated weights for policy 1, policy_version 167879 (0.0007) [2023-12-26 16:34:59,215][105692] Updated weights for policy 0, policy_version 166922 (0.0007) [2023-12-26 16:34:59,282][105692] Updated weights for policy 0, policy_version 166932 (0.0010) [2023-12-26 16:34:59,832][105620] Updated weights for policy 1, policy_version 167889 (0.0010) [2023-12-26 16:34:59,888][105620] Updated weights for policy 1, policy_version 167899 (0.0009) [2023-12-26 16:34:59,911][105692] Updated weights for policy 0, policy_version 166942 (0.0008) [2023-12-26 16:34:59,948][105620] Updated weights for policy 1, policy_version 167909 (0.0008) [2023-12-26 16:34:59,978][105692] Updated weights for policy 0, policy_version 166952 (0.0008) [2023-12-26 16:35:00,037][105692] Updated weights for policy 0, policy_version 166962 (0.0008) [2023-12-26 16:35:00,642][105620] Updated weights for policy 1, policy_version 167919 (0.0010) [2023-12-26 16:35:00,693][105620] Updated weights for policy 1, policy_version 167929 (0.0010) [2023-12-26 16:35:00,749][105620] Updated weights for policy 1, policy_version 167939 (0.0006) [2023-12-26 16:35:00,797][105692] Updated weights for policy 0, policy_version 166972 (0.0009) [2023-12-26 16:35:00,849][105692] Updated weights for policy 0, policy_version 166982 (0.0010) [2023-12-26 16:35:00,911][105692] Updated weights for policy 0, policy_version 166992 (0.0010) [2023-12-26 16:35:01,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 85762048. Throughput: 0: 9716.9, 1: 9950.1. Samples: 85727596. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 16:35:01,062][104569] Avg episode reward: [(0, '7822.273'), (1, '8987.791')] [2023-12-26 16:35:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000167000_42762240.pth... [2023-12-26 16:35:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000167944_42999808.pth... [2023-12-26 16:35:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000165848_42467328.pth [2023-12-26 16:35:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000166792_42704896.pth [2023-12-26 16:35:01,461][105620] Updated weights for policy 1, policy_version 167949 (0.0008) [2023-12-26 16:35:01,528][105620] Updated weights for policy 1, policy_version 167959 (0.0005) [2023-12-26 16:35:01,596][105620] Updated weights for policy 1, policy_version 167969 (0.0009) [2023-12-26 16:35:01,679][105692] Updated weights for policy 0, policy_version 167002 (0.0009) [2023-12-26 16:35:01,746][105692] Updated weights for policy 0, policy_version 167012 (0.0009) [2023-12-26 16:35:01,812][105692] Updated weights for policy 0, policy_version 167022 (0.0008) [2023-12-26 16:35:01,877][105692] Updated weights for policy 0, policy_version 167032 (0.0011) [2023-12-26 16:35:02,285][105620] Updated weights for policy 1, policy_version 167979 (0.0010) [2023-12-26 16:35:02,340][105620] Updated weights for policy 1, policy_version 167989 (0.0009) [2023-12-26 16:35:02,398][105620] Updated weights for policy 1, policy_version 167999 (0.0006) [2023-12-26 16:35:02,529][105692] Updated weights for policy 0, policy_version 167042 (0.0010) [2023-12-26 16:35:02,586][105692] Updated weights for policy 0, policy_version 167052 (0.0010) [2023-12-26 16:35:02,647][105692] Updated weights for policy 0, policy_version 167062 (0.0011) [2023-12-26 16:35:03,028][105620] Updated weights for policy 1, policy_version 168009 (0.0006) [2023-12-26 16:35:03,085][105620] Updated weights for policy 1, policy_version 168019 (0.0005) [2023-12-26 16:35:03,137][105620] Updated weights for policy 1, policy_version 168029 (0.0005) [2023-12-26 16:35:03,191][105620] Updated weights for policy 1, policy_version 168039 (0.0005) [2023-12-26 16:35:03,366][105692] Updated weights for policy 0, policy_version 167072 (0.0009) [2023-12-26 16:35:03,417][105692] Updated weights for policy 0, policy_version 167083 (0.0009) [2023-12-26 16:35:03,475][105692] Updated weights for policy 0, policy_version 167093 (0.0009) [2023-12-26 16:35:03,764][105620] Updated weights for policy 1, policy_version 168049 (0.0008) [2023-12-26 16:35:03,818][105620] Updated weights for policy 1, policy_version 168059 (0.0009) [2023-12-26 16:35:03,880][105620] Updated weights for policy 1, policy_version 168069 (0.0008) [2023-12-26 16:35:04,290][105692] Updated weights for policy 0, policy_version 167103 (0.0009) [2023-12-26 16:35:04,349][105692] Updated weights for policy 0, policy_version 167113 (0.0010) [2023-12-26 16:35:04,401][105692] Updated weights for policy 0, policy_version 167123 (0.0009) [2023-12-26 16:35:04,599][105620] Updated weights for policy 1, policy_version 168079 (0.0009) [2023-12-26 16:35:04,654][105620] Updated weights for policy 1, policy_version 168089 (0.0009) [2023-12-26 16:35:04,705][105620] Updated weights for policy 1, policy_version 168099 (0.0008) [2023-12-26 16:35:05,107][105692] Updated weights for policy 0, policy_version 167133 (0.0007) [2023-12-26 16:35:05,155][105692] Updated weights for policy 0, policy_version 167143 (0.0005) [2023-12-26 16:35:05,208][105692] Updated weights for policy 0, policy_version 167153 (0.0009) [2023-12-26 16:35:05,517][105620] Updated weights for policy 1, policy_version 168109 (0.0008) [2023-12-26 16:35:05,572][105620] Updated weights for policy 1, policy_version 168119 (0.0008) [2023-12-26 16:35:05,635][105620] Updated weights for policy 1, policy_version 168129 (0.0008) [2023-12-26 16:35:05,857][105692] Updated weights for policy 0, policy_version 167163 (0.0010) [2023-12-26 16:35:05,919][105692] Updated weights for policy 0, policy_version 167173 (0.0010) [2023-12-26 16:35:05,977][105692] Updated weights for policy 0, policy_version 167183 (0.0011) [2023-12-26 16:35:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 85860352. Throughput: 0: 9709.8, 1: 10010.6. Samples: 85845476. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 16:35:06,063][104569] Avg episode reward: [(0, '8640.459'), (1, '8899.846')] [2023-12-26 16:35:06,370][105620] Updated weights for policy 1, policy_version 168139 (0.0007) [2023-12-26 16:35:06,423][105620] Updated weights for policy 1, policy_version 168149 (0.0009) [2023-12-26 16:35:06,483][105620] Updated weights for policy 1, policy_version 168159 (0.0009) [2023-12-26 16:35:06,761][105692] Updated weights for policy 0, policy_version 167193 (0.0010) [2023-12-26 16:35:06,811][105692] Updated weights for policy 0, policy_version 167203 (0.0008) [2023-12-26 16:35:06,862][105692] Updated weights for policy 0, policy_version 167213 (0.0009) [2023-12-26 16:35:06,924][105692] Updated weights for policy 0, policy_version 167223 (0.0010) [2023-12-26 16:35:07,229][105620] Updated weights for policy 1, policy_version 168169 (0.0009) [2023-12-26 16:35:07,280][105620] Updated weights for policy 1, policy_version 168179 (0.0009) [2023-12-26 16:35:07,328][105620] Updated weights for policy 1, policy_version 168189 (0.0009) [2023-12-26 16:35:07,375][105620] Updated weights for policy 1, policy_version 168199 (0.0009) [2023-12-26 16:35:07,712][105692] Updated weights for policy 0, policy_version 167233 (0.0009) [2023-12-26 16:35:07,763][105692] Updated weights for policy 0, policy_version 167243 (0.0010) [2023-12-26 16:35:07,822][105692] Updated weights for policy 0, policy_version 167253 (0.0008) [2023-12-26 16:35:08,136][105620] Updated weights for policy 1, policy_version 168209 (0.0009) [2023-12-26 16:35:08,193][105620] Updated weights for policy 1, policy_version 168219 (0.0009) [2023-12-26 16:35:08,248][105620] Updated weights for policy 1, policy_version 168229 (0.0008) [2023-12-26 16:35:08,585][105692] Updated weights for policy 0, policy_version 167263 (0.0008) [2023-12-26 16:35:08,651][105692] Updated weights for policy 0, policy_version 167273 (0.0010) [2023-12-26 16:35:08,712][105692] Updated weights for policy 0, policy_version 167283 (0.0010) [2023-12-26 16:35:08,958][105620] Updated weights for policy 1, policy_version 168239 (0.0008) [2023-12-26 16:35:09,005][105620] Updated weights for policy 1, policy_version 168249 (0.0008) [2023-12-26 16:35:09,057][105620] Updated weights for policy 1, policy_version 168259 (0.0005) [2023-12-26 16:35:09,528][105692] Updated weights for policy 0, policy_version 167293 (0.0009) [2023-12-26 16:35:09,592][105692] Updated weights for policy 0, policy_version 167303 (0.0009) [2023-12-26 16:35:09,654][105692] Updated weights for policy 0, policy_version 167313 (0.0009) [2023-12-26 16:35:09,738][105620] Updated weights for policy 1, policy_version 168269 (0.0007) [2023-12-26 16:35:09,803][105620] Updated weights for policy 1, policy_version 168279 (0.0009) [2023-12-26 16:35:09,869][105620] Updated weights for policy 1, policy_version 168289 (0.0010) [2023-12-26 16:35:10,511][105620] Updated weights for policy 1, policy_version 168299 (0.0008) [2023-12-26 16:35:10,513][105692] Updated weights for policy 0, policy_version 167323 (0.0009) [2023-12-26 16:35:10,569][105692] Updated weights for policy 0, policy_version 167333 (0.0010) [2023-12-26 16:35:10,571][105620] Updated weights for policy 1, policy_version 168309 (0.0005) [2023-12-26 16:35:10,626][105692] Updated weights for policy 0, policy_version 167343 (0.0009) [2023-12-26 16:35:10,631][105620] Updated weights for policy 1, policy_version 168319 (0.0006) [2023-12-26 16:35:11,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 85950464. Throughput: 0: 9630.9, 1: 10046.9. Samples: 85960060. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 16:35:11,063][104569] Avg episode reward: [(0, '8814.416'), (1, '8818.153')] [2023-12-26 16:35:11,253][105620] Updated weights for policy 1, policy_version 168329 (0.0009) [2023-12-26 16:35:11,320][105620] Updated weights for policy 1, policy_version 168340 (0.0010) [2023-12-26 16:35:11,388][105620] Updated weights for policy 1, policy_version 168350 (0.0010) [2023-12-26 16:35:11,427][105692] Updated weights for policy 0, policy_version 167353 (0.0007) [2023-12-26 16:35:11,453][105620] Updated weights for policy 1, policy_version 168360 (0.0008) [2023-12-26 16:35:11,479][105692] Updated weights for policy 0, policy_version 167363 (0.0009) [2023-12-26 16:35:11,539][105692] Updated weights for policy 0, policy_version 167373 (0.0007) [2023-12-26 16:35:11,602][105692] Updated weights for policy 0, policy_version 167383 (0.0008) [2023-12-26 16:35:12,222][105620] Updated weights for policy 1, policy_version 168370 (0.0007) [2023-12-26 16:35:12,288][105620] Updated weights for policy 1, policy_version 168380 (0.0008) [2023-12-26 16:35:12,330][105692] Updated weights for policy 0, policy_version 167393 (0.0008) [2023-12-26 16:35:12,354][105620] Updated weights for policy 1, policy_version 168390 (0.0007) [2023-12-26 16:35:12,399][105692] Updated weights for policy 0, policy_version 167404 (0.0007) [2023-12-26 16:35:12,466][105692] Updated weights for policy 0, policy_version 167414 (0.0005) [2023-12-26 16:35:13,063][105692] Updated weights for policy 0, policy_version 167424 (0.0005) [2023-12-26 16:35:13,126][105692] Updated weights for policy 0, policy_version 167434 (0.0009) [2023-12-26 16:35:13,163][105620] Updated weights for policy 1, policy_version 168400 (0.0008) [2023-12-26 16:35:13,184][105692] Updated weights for policy 0, policy_version 167444 (0.0008) [2023-12-26 16:35:13,218][105620] Updated weights for policy 1, policy_version 168410 (0.0009) [2023-12-26 16:35:13,273][105620] Updated weights for policy 1, policy_version 168420 (0.0009) [2023-12-26 16:35:13,858][105692] Updated weights for policy 0, policy_version 167454 (0.0007) [2023-12-26 16:35:13,917][105692] Updated weights for policy 0, policy_version 167464 (0.0006) [2023-12-26 16:35:13,970][105692] Updated weights for policy 0, policy_version 167474 (0.0011) [2023-12-26 16:35:14,044][105620] Updated weights for policy 1, policy_version 168430 (0.0010) [2023-12-26 16:35:14,101][105620] Updated weights for policy 1, policy_version 168440 (0.0010) [2023-12-26 16:35:14,155][105620] Updated weights for policy 1, policy_version 168450 (0.0010) [2023-12-26 16:35:14,512][105692] Updated weights for policy 0, policy_version 167484 (0.0008) [2023-12-26 16:35:14,559][105692] Updated weights for policy 0, policy_version 167494 (0.0005) [2023-12-26 16:35:14,616][105692] Updated weights for policy 0, policy_version 167504 (0.0007) [2023-12-26 16:35:14,774][105620] Updated weights for policy 1, policy_version 168460 (0.0009) [2023-12-26 16:35:14,830][105620] Updated weights for policy 1, policy_version 168470 (0.0009) [2023-12-26 16:35:14,892][105620] Updated weights for policy 1, policy_version 168480 (0.0010) [2023-12-26 16:35:15,371][105692] Updated weights for policy 0, policy_version 167514 (0.0010) [2023-12-26 16:35:15,440][105692] Updated weights for policy 0, policy_version 167524 (0.0011) [2023-12-26 16:35:15,499][105692] Updated weights for policy 0, policy_version 167534 (0.0010) [2023-12-26 16:35:15,561][105692] Updated weights for policy 0, policy_version 167544 (0.0011) [2023-12-26 16:35:15,608][105620] Updated weights for policy 1, policy_version 168490 (0.0009) [2023-12-26 16:35:15,664][105620] Updated weights for policy 1, policy_version 168500 (0.0010) [2023-12-26 16:35:15,720][105620] Updated weights for policy 1, policy_version 168510 (0.0010) [2023-12-26 16:35:15,768][105620] Updated weights for policy 1, policy_version 168520 (0.0010) [2023-12-26 16:35:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 86048768. Throughput: 0: 9669.2, 1: 9903.2. Samples: 86016428. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 16:35:16,063][104569] Avg episode reward: [(0, '9172.633'), (1, '8480.528')] [2023-12-26 16:35:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000167544_42901504.pth... [2023-12-26 16:35:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000168520_43147264.pth... [2023-12-26 16:35:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000166424_42614784.pth [2023-12-26 16:35:16,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000167368_42852352.pth [2023-12-26 16:35:16,198][105692] Updated weights for policy 0, policy_version 167554 (0.0010) [2023-12-26 16:35:16,266][105692] Updated weights for policy 0, policy_version 167564 (0.0010) [2023-12-26 16:35:16,328][105692] Updated weights for policy 0, policy_version 167574 (0.0010) [2023-12-26 16:35:16,415][105620] Updated weights for policy 1, policy_version 168530 (0.0007) [2023-12-26 16:35:16,466][105620] Updated weights for policy 1, policy_version 168540 (0.0010) [2023-12-26 16:35:16,514][105620] Updated weights for policy 1, policy_version 168550 (0.0010) [2023-12-26 16:35:16,979][105692] Updated weights for policy 0, policy_version 167584 (0.0010) [2023-12-26 16:35:17,030][105692] Updated weights for policy 0, policy_version 167594 (0.0010) [2023-12-26 16:35:17,076][105692] Updated weights for policy 0, policy_version 167604 (0.0006) [2023-12-26 16:35:17,220][105620] Updated weights for policy 1, policy_version 168560 (0.0010) [2023-12-26 16:35:17,274][105620] Updated weights for policy 1, policy_version 168570 (0.0010) [2023-12-26 16:35:17,332][105620] Updated weights for policy 1, policy_version 168580 (0.0010) [2023-12-26 16:35:17,720][105692] Updated weights for policy 0, policy_version 167614 (0.0008) [2023-12-26 16:35:17,789][105692] Updated weights for policy 0, policy_version 167624 (0.0010) [2023-12-26 16:35:17,850][105692] Updated weights for policy 0, policy_version 167634 (0.0009) [2023-12-26 16:35:18,028][105620] Updated weights for policy 1, policy_version 168590 (0.0009) [2023-12-26 16:35:18,092][105620] Updated weights for policy 1, policy_version 168600 (0.0008) [2023-12-26 16:35:18,157][105620] Updated weights for policy 1, policy_version 168610 (0.0008) [2023-12-26 16:35:18,600][105692] Updated weights for policy 0, policy_version 167644 (0.0008) [2023-12-26 16:35:18,651][105692] Updated weights for policy 0, policy_version 167654 (0.0006) [2023-12-26 16:35:18,708][105692] Updated weights for policy 0, policy_version 167664 (0.0005) [2023-12-26 16:35:18,929][105620] Updated weights for policy 1, policy_version 168620 (0.0008) [2023-12-26 16:35:18,977][105620] Updated weights for policy 1, policy_version 168630 (0.0008) [2023-12-26 16:35:19,028][105620] Updated weights for policy 1, policy_version 168640 (0.0008) [2023-12-26 16:35:19,411][105692] Updated weights for policy 0, policy_version 167674 (0.0007) [2023-12-26 16:35:19,476][105692] Updated weights for policy 0, policy_version 167684 (0.0011) [2023-12-26 16:35:19,541][105692] Updated weights for policy 0, policy_version 167694 (0.0011) [2023-12-26 16:35:19,607][105692] Updated weights for policy 0, policy_version 167704 (0.0010) [2023-12-26 16:35:19,873][105620] Updated weights for policy 1, policy_version 168650 (0.0008) [2023-12-26 16:35:19,934][105620] Updated weights for policy 1, policy_version 168660 (0.0009) [2023-12-26 16:35:19,985][105620] Updated weights for policy 1, policy_version 168670 (0.0009) [2023-12-26 16:35:20,038][105620] Updated weights for policy 1, policy_version 168680 (0.0009) [2023-12-26 16:35:20,343][105692] Updated weights for policy 0, policy_version 167714 (0.0006) [2023-12-26 16:35:20,410][105692] Updated weights for policy 0, policy_version 167724 (0.0007) [2023-12-26 16:35:20,478][105692] Updated weights for policy 0, policy_version 167734 (0.0009) [2023-12-26 16:35:20,817][105620] Updated weights for policy 1, policy_version 168690 (0.0009) [2023-12-26 16:35:20,881][105620] Updated weights for policy 1, policy_version 168700 (0.0008) [2023-12-26 16:35:20,945][105620] Updated weights for policy 1, policy_version 168710 (0.0005) [2023-12-26 16:35:21,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 86147072. Throughput: 0: 9785.9, 1: 9854.0. Samples: 86137116. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 16:35:21,062][104569] Avg episode reward: [(0, '9354.102'), (1, '7668.955')] [2023-12-26 16:35:21,230][105692] Updated weights for policy 0, policy_version 167744 (0.0009) [2023-12-26 16:35:21,298][105692] Updated weights for policy 0, policy_version 167754 (0.0010) [2023-12-26 16:35:21,363][105692] Updated weights for policy 0, policy_version 167764 (0.0008) [2023-12-26 16:35:21,624][105620] Updated weights for policy 1, policy_version 168720 (0.0009) [2023-12-26 16:35:21,682][105620] Updated weights for policy 1, policy_version 168730 (0.0009) [2023-12-26 16:35:21,747][105620] Updated weights for policy 1, policy_version 168740 (0.0008) [2023-12-26 16:35:22,181][105692] Updated weights for policy 0, policy_version 167774 (0.0009) [2023-12-26 16:35:22,237][105692] Updated weights for policy 0, policy_version 167784 (0.0009) [2023-12-26 16:35:22,298][105692] Updated weights for policy 0, policy_version 167794 (0.0009) [2023-12-26 16:35:22,521][105620] Updated weights for policy 1, policy_version 168750 (0.0009) [2023-12-26 16:35:22,582][105620] Updated weights for policy 1, policy_version 168760 (0.0009) [2023-12-26 16:35:22,612][105586] KL-divergence is very high: 106.2002 [2023-12-26 16:35:22,617][105586] KL-divergence is very high: 117.7676 [2023-12-26 16:35:22,621][105586] KL-divergence is very high: 137.1743 [2023-12-26 16:35:22,626][105586] KL-divergence is very high: 130.4284 [2023-12-26 16:35:22,637][105586] KL-divergence is very high: 131.3700 [2023-12-26 16:35:22,639][105620] Updated weights for policy 1, policy_version 168770 (0.0010) [2023-12-26 16:35:22,642][105586] KL-divergence is very high: 115.5403 [2023-12-26 16:35:22,653][105586] KL-divergence is very high: 108.0139 [2023-12-26 16:35:22,658][105586] KL-divergence is very high: 107.8363 [2023-12-26 16:35:22,663][105586] KL-divergence is very high: 113.9706 [2023-12-26 16:35:23,009][105692] Updated weights for policy 0, policy_version 167804 (0.0008) [2023-12-26 16:35:23,073][105692] Updated weights for policy 0, policy_version 167814 (0.0009) [2023-12-26 16:35:23,122][105692] Updated weights for policy 0, policy_version 167824 (0.0009) [2023-12-26 16:35:23,421][105620] Updated weights for policy 1, policy_version 168780 (0.0009) [2023-12-26 16:35:23,485][105620] Updated weights for policy 1, policy_version 168790 (0.0008) [2023-12-26 16:35:23,545][105620] Updated weights for policy 1, policy_version 168800 (0.0009) [2023-12-26 16:35:23,888][105692] Updated weights for policy 0, policy_version 167834 (0.0009) [2023-12-26 16:35:23,941][105692] Updated weights for policy 0, policy_version 167844 (0.0009) [2023-12-26 16:35:23,996][105692] Updated weights for policy 0, policy_version 167854 (0.0009) [2023-12-26 16:35:24,054][105692] Updated weights for policy 0, policy_version 167864 (0.0009) [2023-12-26 16:35:24,329][105620] Updated weights for policy 1, policy_version 168810 (0.0009) [2023-12-26 16:35:24,381][105620] Updated weights for policy 1, policy_version 168820 (0.0010) [2023-12-26 16:35:24,441][105620] Updated weights for policy 1, policy_version 168830 (0.0009) [2023-12-26 16:35:24,498][105620] Updated weights for policy 1, policy_version 168840 (0.0009) [2023-12-26 16:35:24,674][105692] Updated weights for policy 0, policy_version 167874 (0.0010) [2023-12-26 16:35:24,725][105692] Updated weights for policy 0, policy_version 167885 (0.0006) [2023-12-26 16:35:24,780][105692] Updated weights for policy 0, policy_version 167895 (0.0009) [2023-12-26 16:35:25,131][105620] Updated weights for policy 1, policy_version 168850 (0.0008) [2023-12-26 16:35:25,191][105620] Updated weights for policy 1, policy_version 168860 (0.0006) [2023-12-26 16:35:25,255][105620] Updated weights for policy 1, policy_version 168870 (0.0005) [2023-12-26 16:35:25,396][105692] Updated weights for policy 0, policy_version 167905 (0.0006) [2023-12-26 16:35:25,463][105692] Updated weights for policy 0, policy_version 167915 (0.0006) [2023-12-26 16:35:25,558][105692] Updated weights for policy 0, policy_version 167925 (0.0008) [2023-12-26 16:35:25,759][105620] Updated weights for policy 1, policy_version 168880 (0.0005) [2023-12-26 16:35:25,814][105620] Updated weights for policy 1, policy_version 168890 (0.0009) [2023-12-26 16:35:25,865][105620] Updated weights for policy 1, policy_version 168900 (0.0010) [2023-12-26 16:35:26,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 86245376. Throughput: 0: 9858.5, 1: 9800.6. Samples: 86254360. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 16:35:26,062][104569] Avg episode reward: [(0, '9353.046'), (1, '6999.402')] [2023-12-26 16:35:26,097][105692] Updated weights for policy 0, policy_version 167935 (0.0008) [2023-12-26 16:35:26,156][105692] Updated weights for policy 0, policy_version 167945 (0.0011) [2023-12-26 16:35:26,219][105692] Updated weights for policy 0, policy_version 167955 (0.0010) [2023-12-26 16:35:26,518][105620] Updated weights for policy 1, policy_version 168910 (0.0007) [2023-12-26 16:35:26,580][105620] Updated weights for policy 1, policy_version 168920 (0.0005) [2023-12-26 16:35:26,646][105620] Updated weights for policy 1, policy_version 168930 (0.0005) [2023-12-26 16:35:26,950][105692] Updated weights for policy 0, policy_version 167965 (0.0011) [2023-12-26 16:35:27,013][105692] Updated weights for policy 0, policy_version 167975 (0.0010) [2023-12-26 16:35:27,059][105692] Updated weights for policy 0, policy_version 167985 (0.0006) [2023-12-26 16:35:27,192][105620] Updated weights for policy 1, policy_version 168940 (0.0005) [2023-12-26 16:35:27,238][105620] Updated weights for policy 1, policy_version 168950 (0.0005) [2023-12-26 16:35:27,283][105620] Updated weights for policy 1, policy_version 168960 (0.0005) [2023-12-26 16:35:27,597][105692] Updated weights for policy 0, policy_version 167995 (0.0005) [2023-12-26 16:35:27,660][105692] Updated weights for policy 0, policy_version 168005 (0.0006) [2023-12-26 16:35:27,711][105692] Updated weights for policy 0, policy_version 168015 (0.0006) [2023-12-26 16:35:27,882][105620] Updated weights for policy 1, policy_version 168970 (0.0009) [2023-12-26 16:35:27,949][105620] Updated weights for policy 1, policy_version 168980 (0.0008) [2023-12-26 16:35:28,003][105620] Updated weights for policy 1, policy_version 168990 (0.0005) [2023-12-26 16:35:28,064][105620] Updated weights for policy 1, policy_version 169000 (0.0005) [2023-12-26 16:35:28,310][105692] Updated weights for policy 0, policy_version 168025 (0.0010) [2023-12-26 16:35:28,381][105692] Updated weights for policy 0, policy_version 168035 (0.0008) [2023-12-26 16:35:28,446][105692] Updated weights for policy 0, policy_version 168045 (0.0008) [2023-12-26 16:35:28,500][105692] Updated weights for policy 0, policy_version 168055 (0.0009) [2023-12-26 16:35:28,638][105620] Updated weights for policy 1, policy_version 169010 (0.0006) [2023-12-26 16:35:28,682][105620] Updated weights for policy 1, policy_version 169020 (0.0008) [2023-12-26 16:35:28,728][105620] Updated weights for policy 1, policy_version 169030 (0.0006) [2023-12-26 16:35:29,181][105692] Updated weights for policy 0, policy_version 168065 (0.0010) [2023-12-26 16:35:29,239][105692] Updated weights for policy 0, policy_version 168075 (0.0010) [2023-12-26 16:35:29,299][105692] Updated weights for policy 0, policy_version 168085 (0.0011) [2023-12-26 16:35:29,378][105620] Updated weights for policy 1, policy_version 169040 (0.0007) [2023-12-26 16:35:29,447][105620] Updated weights for policy 1, policy_version 169050 (0.0006) [2023-12-26 16:35:29,509][105620] Updated weights for policy 1, policy_version 169060 (0.0005) [2023-12-26 16:35:29,976][105692] Updated weights for policy 0, policy_version 168095 (0.0008) [2023-12-26 16:35:30,040][105692] Updated weights for policy 0, policy_version 168105 (0.0010) [2023-12-26 16:35:30,044][105620] Updated weights for policy 1, policy_version 169070 (0.0006) [2023-12-26 16:35:30,095][105692] Updated weights for policy 0, policy_version 168115 (0.0010) [2023-12-26 16:35:30,101][105620] Updated weights for policy 1, policy_version 169080 (0.0005) [2023-12-26 16:35:30,161][105620] Updated weights for policy 1, policy_version 169090 (0.0006) [2023-12-26 16:35:30,758][105692] Updated weights for policy 0, policy_version 168125 (0.0008) [2023-12-26 16:35:30,812][105692] Updated weights for policy 0, policy_version 168135 (0.0009) [2023-12-26 16:35:30,826][105620] Updated weights for policy 1, policy_version 169100 (0.0009) [2023-12-26 16:35:30,859][105692] Updated weights for policy 0, policy_version 168145 (0.0010) [2023-12-26 16:35:30,893][105620] Updated weights for policy 1, policy_version 169110 (0.0010) [2023-12-26 16:35:30,949][105620] Updated weights for policy 1, policy_version 169120 (0.0006) [2023-12-26 16:35:31,062][104569] Fps is (10 sec: 21298.7, 60 sec: 19933.8, 300 sec: 19688.6). Total num frames: 86360064. Throughput: 0: 9877.0, 1: 9963.4. Samples: 86321788. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 16:35:31,062][104569] Avg episode reward: [(0, '9350.596'), (1, '7640.269')] [2023-12-26 16:35:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000168152_43057152.pth... [2023-12-26 16:35:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000169128_43302912.pth... [2023-12-26 16:35:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000167000_42762240.pth [2023-12-26 16:35:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000167944_42999808.pth [2023-12-26 16:35:31,534][105692] Updated weights for policy 0, policy_version 168155 (0.0010) [2023-12-26 16:35:31,583][105692] Updated weights for policy 0, policy_version 168165 (0.0010) [2023-12-26 16:35:31,625][105620] Updated weights for policy 1, policy_version 169131 (0.0008) [2023-12-26 16:35:31,652][105692] Updated weights for policy 0, policy_version 168175 (0.0011) [2023-12-26 16:35:31,682][105620] Updated weights for policy 1, policy_version 169141 (0.0007) [2023-12-26 16:35:31,744][105620] Updated weights for policy 1, policy_version 169151 (0.0007) [2023-12-26 16:35:32,316][105692] Updated weights for policy 0, policy_version 168185 (0.0011) [2023-12-26 16:35:32,377][105692] Updated weights for policy 0, policy_version 168195 (0.0011) [2023-12-26 16:35:32,432][105692] Updated weights for policy 0, policy_version 168205 (0.0007) [2023-12-26 16:35:32,502][105692] Updated weights for policy 0, policy_version 168215 (0.0006) [2023-12-26 16:35:32,600][105620] Updated weights for policy 1, policy_version 169161 (0.0007) [2023-12-26 16:35:32,658][105620] Updated weights for policy 1, policy_version 169171 (0.0009) [2023-12-26 16:35:32,720][105620] Updated weights for policy 1, policy_version 169181 (0.0009) [2023-12-26 16:35:32,781][105620] Updated weights for policy 1, policy_version 169191 (0.0007) [2023-12-26 16:35:33,105][105692] Updated weights for policy 0, policy_version 168225 (0.0006) [2023-12-26 16:35:33,167][105692] Updated weights for policy 0, policy_version 168235 (0.0009) [2023-12-26 16:35:33,223][105692] Updated weights for policy 0, policy_version 168245 (0.0008) [2023-12-26 16:35:33,546][105620] Updated weights for policy 1, policy_version 169201 (0.0010) [2023-12-26 16:35:33,603][105620] Updated weights for policy 1, policy_version 169211 (0.0010) [2023-12-26 16:35:33,646][105620] Updated weights for policy 1, policy_version 169221 (0.0010) [2023-12-26 16:35:33,882][105692] Updated weights for policy 0, policy_version 168255 (0.0010) [2023-12-26 16:35:33,930][105692] Updated weights for policy 0, policy_version 168265 (0.0010) [2023-12-26 16:35:33,978][105692] Updated weights for policy 0, policy_version 168275 (0.0010) [2023-12-26 16:35:34,284][105620] Updated weights for policy 1, policy_version 169231 (0.0007) [2023-12-26 16:35:34,344][105620] Updated weights for policy 1, policy_version 169241 (0.0006) [2023-12-26 16:35:34,408][105620] Updated weights for policy 1, policy_version 169251 (0.0006) [2023-12-26 16:35:34,773][105692] Updated weights for policy 0, policy_version 168285 (0.0010) [2023-12-26 16:35:34,831][105692] Updated weights for policy 0, policy_version 168295 (0.0009) [2023-12-26 16:35:34,888][105692] Updated weights for policy 0, policy_version 168305 (0.0009) [2023-12-26 16:35:34,998][105620] Updated weights for policy 1, policy_version 169261 (0.0007) [2023-12-26 16:35:35,051][105620] Updated weights for policy 1, policy_version 169271 (0.0008) [2023-12-26 16:35:35,106][105620] Updated weights for policy 1, policy_version 169281 (0.0005) [2023-12-26 16:35:35,587][105692] Updated weights for policy 0, policy_version 168315 (0.0009) [2023-12-26 16:35:35,657][105692] Updated weights for policy 0, policy_version 168325 (0.0008) [2023-12-26 16:35:35,683][105620] Updated weights for policy 1, policy_version 169291 (0.0006) [2023-12-26 16:35:35,723][105692] Updated weights for policy 0, policy_version 168335 (0.0006) [2023-12-26 16:35:35,743][105620] Updated weights for policy 1, policy_version 169301 (0.0005) [2023-12-26 16:35:35,811][105620] Updated weights for policy 1, policy_version 169311 (0.0005) [2023-12-26 16:35:36,062][104569] Fps is (10 sec: 21298.7, 60 sec: 19933.8, 300 sec: 19716.3). Total num frames: 86458368. Throughput: 0: 9895.4, 1: 10037.3. Samples: 86444248. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 16:35:36,063][104569] Avg episode reward: [(0, '9261.369'), (1, '9101.513')] [2023-12-26 16:35:36,390][105692] Updated weights for policy 0, policy_version 168345 (0.0005) [2023-12-26 16:35:36,393][105620] Updated weights for policy 1, policy_version 169321 (0.0006) [2023-12-26 16:35:36,450][105692] Updated weights for policy 0, policy_version 168355 (0.0006) [2023-12-26 16:35:36,452][105620] Updated weights for policy 1, policy_version 169331 (0.0010) [2023-12-26 16:35:36,508][105620] Updated weights for policy 1, policy_version 169341 (0.0011) [2023-12-26 16:35:36,515][105692] Updated weights for policy 0, policy_version 168365 (0.0006) [2023-12-26 16:35:36,561][105620] Updated weights for policy 1, policy_version 169351 (0.0010) [2023-12-26 16:35:36,578][105692] Updated weights for policy 0, policy_version 168375 (0.0006) [2023-12-26 16:35:37,153][105692] Updated weights for policy 0, policy_version 168385 (0.0006) [2023-12-26 16:35:37,216][105692] Updated weights for policy 0, policy_version 168395 (0.0005) [2023-12-26 16:35:37,277][105692] Updated weights for policy 0, policy_version 168405 (0.0007) [2023-12-26 16:35:37,284][105620] Updated weights for policy 1, policy_version 169361 (0.0011) [2023-12-26 16:35:37,334][105620] Updated weights for policy 1, policy_version 169371 (0.0010) [2023-12-26 16:35:37,384][105620] Updated weights for policy 1, policy_version 169381 (0.0011) [2023-12-26 16:35:37,912][105692] Updated weights for policy 0, policy_version 168415 (0.0005) [2023-12-26 16:35:37,958][105692] Updated weights for policy 0, policy_version 168425 (0.0005) [2023-12-26 16:35:38,013][105692] Updated weights for policy 0, policy_version 168435 (0.0005) [2023-12-26 16:35:38,116][105620] Updated weights for policy 1, policy_version 169391 (0.0009) [2023-12-26 16:35:38,165][105620] Updated weights for policy 1, policy_version 169401 (0.0010) [2023-12-26 16:35:38,224][105620] Updated weights for policy 1, policy_version 169411 (0.0010) [2023-12-26 16:35:38,614][105692] Updated weights for policy 0, policy_version 168445 (0.0007) [2023-12-26 16:35:38,677][105692] Updated weights for policy 0, policy_version 168455 (0.0008) [2023-12-26 16:35:38,735][105692] Updated weights for policy 0, policy_version 168465 (0.0008) [2023-12-26 16:35:38,947][105620] Updated weights for policy 1, policy_version 169421 (0.0009) [2023-12-26 16:35:39,017][105620] Updated weights for policy 1, policy_version 169431 (0.0007) [2023-12-26 16:35:39,078][105620] Updated weights for policy 1, policy_version 169441 (0.0008) [2023-12-26 16:35:39,544][105692] Updated weights for policy 0, policy_version 168475 (0.0008) [2023-12-26 16:35:39,610][105692] Updated weights for policy 0, policy_version 168485 (0.0007) [2023-12-26 16:35:39,671][105692] Updated weights for policy 0, policy_version 168495 (0.0009) [2023-12-26 16:35:39,809][105620] Updated weights for policy 1, policy_version 169451 (0.0007) [2023-12-26 16:35:39,871][105620] Updated weights for policy 1, policy_version 169461 (0.0011) [2023-12-26 16:35:39,934][105620] Updated weights for policy 1, policy_version 169471 (0.0010) [2023-12-26 16:35:40,403][105692] Updated weights for policy 0, policy_version 168505 (0.0009) [2023-12-26 16:35:40,458][105692] Updated weights for policy 0, policy_version 168515 (0.0008) [2023-12-26 16:35:40,517][105692] Updated weights for policy 0, policy_version 168525 (0.0008) [2023-12-26 16:35:40,585][105692] Updated weights for policy 0, policy_version 168535 (0.0009) [2023-12-26 16:35:40,654][105620] Updated weights for policy 1, policy_version 169481 (0.0011) [2023-12-26 16:35:40,701][105620] Updated weights for policy 1, policy_version 169491 (0.0007) [2023-12-26 16:35:40,759][105620] Updated weights for policy 1, policy_version 169501 (0.0006) [2023-12-26 16:35:40,816][105620] Updated weights for policy 1, policy_version 169511 (0.0005) [2023-12-26 16:35:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19933.8, 300 sec: 19716.3). Total num frames: 86556672. Throughput: 0: 9901.9, 1: 10050.7. Samples: 86565728. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 16:35:41,063][104569] Avg episode reward: [(0, '9084.265'), (1, '8916.381')] [2023-12-26 16:35:41,411][105692] Updated weights for policy 0, policy_version 168545 (0.0008) [2023-12-26 16:35:41,467][105692] Updated weights for policy 0, policy_version 168555 (0.0008) [2023-12-26 16:35:41,484][105620] Updated weights for policy 1, policy_version 169521 (0.0007) [2023-12-26 16:35:41,529][105692] Updated weights for policy 0, policy_version 168565 (0.0008) [2023-12-26 16:35:41,544][105620] Updated weights for policy 1, policy_version 169531 (0.0009) [2023-12-26 16:35:41,595][105620] Updated weights for policy 1, policy_version 169541 (0.0008) [2023-12-26 16:35:42,225][105692] Updated weights for policy 0, policy_version 168575 (0.0008) [2023-12-26 16:35:42,281][105692] Updated weights for policy 0, policy_version 168585 (0.0008) [2023-12-26 16:35:42,344][105692] Updated weights for policy 0, policy_version 168595 (0.0008) [2023-12-26 16:35:42,383][105620] Updated weights for policy 1, policy_version 169551 (0.0010) [2023-12-26 16:35:42,446][105620] Updated weights for policy 1, policy_version 169561 (0.0011) [2023-12-26 16:35:42,513][105620] Updated weights for policy 1, policy_version 169571 (0.0011) [2023-12-26 16:35:43,124][105692] Updated weights for policy 0, policy_version 168605 (0.0008) [2023-12-26 16:35:43,175][105692] Updated weights for policy 0, policy_version 168615 (0.0005) [2023-12-26 16:35:43,197][105620] Updated weights for policy 1, policy_version 169581 (0.0008) [2023-12-26 16:35:43,228][105692] Updated weights for policy 0, policy_version 168625 (0.0008) [2023-12-26 16:35:43,256][105620] Updated weights for policy 1, policy_version 169591 (0.0007) [2023-12-26 16:35:43,306][105620] Updated weights for policy 1, policy_version 169601 (0.0009) [2023-12-26 16:35:43,917][105692] Updated weights for policy 0, policy_version 168635 (0.0008) [2023-12-26 16:35:43,964][105692] Updated weights for policy 0, policy_version 168645 (0.0009) [2023-12-26 16:35:44,013][105692] Updated weights for policy 0, policy_version 168655 (0.0008) [2023-12-26 16:35:44,079][105620] Updated weights for policy 1, policy_version 169611 (0.0009) [2023-12-26 16:35:44,141][105620] Updated weights for policy 1, policy_version 169621 (0.0009) [2023-12-26 16:35:44,199][105620] Updated weights for policy 1, policy_version 169631 (0.0009) [2023-12-26 16:35:44,725][105692] Updated weights for policy 0, policy_version 168665 (0.0009) [2023-12-26 16:35:44,784][105692] Updated weights for policy 0, policy_version 168675 (0.0010) [2023-12-26 16:35:44,839][105692] Updated weights for policy 0, policy_version 168685 (0.0009) [2023-12-26 16:35:44,906][105692] Updated weights for policy 0, policy_version 168695 (0.0009) [2023-12-26 16:35:44,939][105620] Updated weights for policy 1, policy_version 169641 (0.0009) [2023-12-26 16:35:45,006][105620] Updated weights for policy 1, policy_version 169651 (0.0009) [2023-12-26 16:35:45,064][105620] Updated weights for policy 1, policy_version 169661 (0.0009) [2023-12-26 16:35:45,130][105620] Updated weights for policy 1, policy_version 169671 (0.0008) [2023-12-26 16:35:45,653][105692] Updated weights for policy 0, policy_version 168705 (0.0006) [2023-12-26 16:35:45,699][105692] Updated weights for policy 0, policy_version 168715 (0.0007) [2023-12-26 16:35:45,747][105692] Updated weights for policy 0, policy_version 168725 (0.0010) [2023-12-26 16:35:45,875][105620] Updated weights for policy 1, policy_version 169681 (0.0010) [2023-12-26 16:35:45,933][105620] Updated weights for policy 1, policy_version 169691 (0.0010) [2023-12-26 16:35:45,986][105620] Updated weights for policy 1, policy_version 169701 (0.0010) [2023-12-26 16:35:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.8, 300 sec: 19744.1). Total num frames: 86654976. Throughput: 0: 9869.3, 1: 9984.4. Samples: 86621016. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 16:35:46,062][104569] Avg episode reward: [(0, '8997.159'), (1, '8919.670')] [2023-12-26 16:35:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000168728_43204608.pth... [2023-12-26 16:35:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000169704_43450368.pth... [2023-12-26 16:35:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000167544_42901504.pth [2023-12-26 16:35:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000168520_43147264.pth [2023-12-26 16:35:46,367][105692] Updated weights for policy 0, policy_version 168735 (0.0010) [2023-12-26 16:35:46,415][105692] Updated weights for policy 0, policy_version 168745 (0.0009) [2023-12-26 16:35:46,476][105692] Updated weights for policy 0, policy_version 168755 (0.0009) [2023-12-26 16:35:46,804][105620] Updated weights for policy 1, policy_version 169712 (0.0010) [2023-12-26 16:35:46,858][105620] Updated weights for policy 1, policy_version 169722 (0.0009) [2023-12-26 16:35:46,905][105620] Updated weights for policy 1, policy_version 169732 (0.0009) [2023-12-26 16:35:47,208][105692] Updated weights for policy 0, policy_version 168765 (0.0008) [2023-12-26 16:35:47,258][105692] Updated weights for policy 0, policy_version 168775 (0.0008) [2023-12-26 16:35:47,332][105692] Updated weights for policy 0, policy_version 168785 (0.0005) [2023-12-26 16:35:47,685][105620] Updated weights for policy 1, policy_version 169743 (0.0010) [2023-12-26 16:35:47,732][105620] Updated weights for policy 1, policy_version 169753 (0.0008) [2023-12-26 16:35:47,780][105620] Updated weights for policy 1, policy_version 169763 (0.0009) [2023-12-26 16:35:48,018][105692] Updated weights for policy 0, policy_version 168795 (0.0007) [2023-12-26 16:35:48,078][105692] Updated weights for policy 0, policy_version 168805 (0.0009) [2023-12-26 16:35:48,125][105692] Updated weights for policy 0, policy_version 168815 (0.0009) [2023-12-26 16:35:48,605][105620] Updated weights for policy 1, policy_version 169773 (0.0009) [2023-12-26 16:35:48,654][105620] Updated weights for policy 1, policy_version 169783 (0.0009) [2023-12-26 16:35:48,709][105620] Updated weights for policy 1, policy_version 169793 (0.0009) [2023-12-26 16:35:48,862][105692] Updated weights for policy 0, policy_version 168825 (0.0010) [2023-12-26 16:35:48,918][105692] Updated weights for policy 0, policy_version 168835 (0.0009) [2023-12-26 16:35:48,970][105692] Updated weights for policy 0, policy_version 168845 (0.0009) [2023-12-26 16:35:49,019][105692] Updated weights for policy 0, policy_version 168855 (0.0008) [2023-12-26 16:35:49,552][105620] Updated weights for policy 1, policy_version 169803 (0.0009) [2023-12-26 16:35:49,606][105620] Updated weights for policy 1, policy_version 169813 (0.0009) [2023-12-26 16:35:49,668][105620] Updated weights for policy 1, policy_version 169823 (0.0009) [2023-12-26 16:35:49,725][105692] Updated weights for policy 0, policy_version 168865 (0.0006) [2023-12-26 16:35:49,782][105692] Updated weights for policy 0, policy_version 168875 (0.0009) [2023-12-26 16:35:49,850][105692] Updated weights for policy 0, policy_version 168885 (0.0008) [2023-12-26 16:35:50,352][105620] Updated weights for policy 1, policy_version 169833 (0.0008) [2023-12-26 16:35:50,405][105620] Updated weights for policy 1, policy_version 169843 (0.0009) [2023-12-26 16:35:50,462][105620] Updated weights for policy 1, policy_version 169853 (0.0008) [2023-12-26 16:35:50,513][105620] Updated weights for policy 1, policy_version 169863 (0.0006) [2023-12-26 16:35:50,661][105692] Updated weights for policy 0, policy_version 168895 (0.0009) [2023-12-26 16:35:50,720][105692] Updated weights for policy 0, policy_version 168905 (0.0009) [2023-12-26 16:35:50,776][105692] Updated weights for policy 0, policy_version 168915 (0.0009) [2023-12-26 16:35:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19933.8, 300 sec: 19688.6). Total num frames: 86745088. Throughput: 0: 9925.2, 1: 9840.3. Samples: 86734928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:35:51,063][104569] Avg episode reward: [(0, '9172.759'), (1, '9087.075')] [2023-12-26 16:35:51,178][105620] Updated weights for policy 1, policy_version 169873 (0.0009) [2023-12-26 16:35:51,243][105620] Updated weights for policy 1, policy_version 169883 (0.0009) [2023-12-26 16:35:51,312][105620] Updated weights for policy 1, policy_version 169893 (0.0009) [2023-12-26 16:35:51,591][105692] Updated weights for policy 0, policy_version 168925 (0.0008) [2023-12-26 16:35:51,654][105692] Updated weights for policy 0, policy_version 168935 (0.0009) [2023-12-26 16:35:51,726][105692] Updated weights for policy 0, policy_version 168945 (0.0009) [2023-12-26 16:35:51,953][105620] Updated weights for policy 1, policy_version 169903 (0.0009) [2023-12-26 16:35:52,008][105620] Updated weights for policy 1, policy_version 169913 (0.0009) [2023-12-26 16:35:52,074][105620] Updated weights for policy 1, policy_version 169923 (0.0009) [2023-12-26 16:35:52,512][105692] Updated weights for policy 0, policy_version 168955 (0.0009) [2023-12-26 16:35:52,571][105692] Updated weights for policy 0, policy_version 168965 (0.0007) [2023-12-26 16:35:52,630][105692] Updated weights for policy 0, policy_version 168975 (0.0008) [2023-12-26 16:35:52,804][105620] Updated weights for policy 1, policy_version 169933 (0.0009) [2023-12-26 16:35:52,863][105620] Updated weights for policy 1, policy_version 169943 (0.0010) [2023-12-26 16:35:52,923][105620] Updated weights for policy 1, policy_version 169953 (0.0010) [2023-12-26 16:35:53,408][105692] Updated weights for policy 0, policy_version 168985 (0.0008) [2023-12-26 16:35:53,455][105692] Updated weights for policy 0, policy_version 168995 (0.0008) [2023-12-26 16:35:53,507][105692] Updated weights for policy 0, policy_version 169005 (0.0009) [2023-12-26 16:35:53,554][105692] Updated weights for policy 0, policy_version 169015 (0.0007) [2023-12-26 16:35:53,649][105620] Updated weights for policy 1, policy_version 169963 (0.0011) [2023-12-26 16:35:53,710][105620] Updated weights for policy 1, policy_version 169973 (0.0010) [2023-12-26 16:35:53,768][105620] Updated weights for policy 1, policy_version 169983 (0.0010) [2023-12-26 16:35:54,339][105692] Updated weights for policy 0, policy_version 169025 (0.0009) [2023-12-26 16:35:54,400][105692] Updated weights for policy 0, policy_version 169035 (0.0009) [2023-12-26 16:35:54,456][105620] Updated weights for policy 1, policy_version 169993 (0.0011) [2023-12-26 16:35:54,456][105692] Updated weights for policy 0, policy_version 169045 (0.0009) [2023-12-26 16:35:54,509][105620] Updated weights for policy 1, policy_version 170003 (0.0009) [2023-12-26 16:35:54,567][105620] Updated weights for policy 1, policy_version 170013 (0.0009) [2023-12-26 16:35:54,627][105620] Updated weights for policy 1, policy_version 170023 (0.0009) [2023-12-26 16:35:55,201][105692] Updated weights for policy 0, policy_version 169055 (0.0009) [2023-12-26 16:35:55,259][105692] Updated weights for policy 0, policy_version 169065 (0.0008) [2023-12-26 16:35:55,321][105692] Updated weights for policy 0, policy_version 169075 (0.0008) [2023-12-26 16:35:55,368][105620] Updated weights for policy 1, policy_version 170033 (0.0009) [2023-12-26 16:35:55,429][105620] Updated weights for policy 1, policy_version 170043 (0.0010) [2023-12-26 16:35:55,475][105620] Updated weights for policy 1, policy_version 170053 (0.0008) [2023-12-26 16:35:55,978][105692] Updated weights for policy 0, policy_version 169085 (0.0007) [2023-12-26 16:35:56,036][105692] Updated weights for policy 0, policy_version 169095 (0.0009) [2023-12-26 16:35:56,062][104569] Fps is (10 sec: 18022.8, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 86835200. Throughput: 0: 9888.4, 1: 9841.3. Samples: 86847892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:35:56,062][104569] Avg episode reward: [(0, '9257.453'), (1, '9262.687')] [2023-12-26 16:35:56,086][105692] Updated weights for policy 0, policy_version 169105 (0.0009) [2023-12-26 16:35:56,264][105620] Updated weights for policy 1, policy_version 170063 (0.0009) [2023-12-26 16:35:56,320][105620] Updated weights for policy 1, policy_version 170073 (0.0009) [2023-12-26 16:35:56,378][105620] Updated weights for policy 1, policy_version 170083 (0.0010) [2023-12-26 16:35:56,808][105692] Updated weights for policy 0, policy_version 169115 (0.0009) [2023-12-26 16:35:56,855][105692] Updated weights for policy 0, policy_version 169125 (0.0009) [2023-12-26 16:35:56,903][105692] Updated weights for policy 0, policy_version 169135 (0.0009) [2023-12-26 16:35:57,166][105620] Updated weights for policy 1, policy_version 170093 (0.0009) [2023-12-26 16:35:57,226][105620] Updated weights for policy 1, policy_version 170103 (0.0005) [2023-12-26 16:35:57,288][105620] Updated weights for policy 1, policy_version 170113 (0.0007) [2023-12-26 16:35:57,679][105692] Updated weights for policy 0, policy_version 169145 (0.0010) [2023-12-26 16:35:57,730][105692] Updated weights for policy 0, policy_version 169155 (0.0010) [2023-12-26 16:35:57,778][105692] Updated weights for policy 0, policy_version 169165 (0.0010) [2023-12-26 16:35:57,829][105692] Updated weights for policy 0, policy_version 169175 (0.0010) [2023-12-26 16:35:57,973][105620] Updated weights for policy 1, policy_version 170123 (0.0007) [2023-12-26 16:35:58,026][105620] Updated weights for policy 1, policy_version 170133 (0.0009) [2023-12-26 16:35:58,081][105620] Updated weights for policy 1, policy_version 170143 (0.0009) [2023-12-26 16:35:58,551][105692] Updated weights for policy 0, policy_version 169185 (0.0010) [2023-12-26 16:35:58,614][105692] Updated weights for policy 0, policy_version 169195 (0.0009) [2023-12-26 16:35:58,676][105692] Updated weights for policy 0, policy_version 169205 (0.0008) [2023-12-26 16:35:58,938][105620] Updated weights for policy 1, policy_version 170153 (0.0009) [2023-12-26 16:35:59,000][105620] Updated weights for policy 1, policy_version 170163 (0.0006) [2023-12-26 16:35:59,066][105620] Updated weights for policy 1, policy_version 170173 (0.0008) [2023-12-26 16:35:59,134][105620] Updated weights for policy 1, policy_version 170183 (0.0008) [2023-12-26 16:35:59,468][105692] Updated weights for policy 0, policy_version 169215 (0.0006) [2023-12-26 16:35:59,522][105692] Updated weights for policy 0, policy_version 169225 (0.0005) [2023-12-26 16:35:59,578][105692] Updated weights for policy 0, policy_version 169235 (0.0011) [2023-12-26 16:35:59,774][105620] Updated weights for policy 1, policy_version 170193 (0.0006) [2023-12-26 16:35:59,829][105620] Updated weights for policy 1, policy_version 170203 (0.0006) [2023-12-26 16:35:59,888][105620] Updated weights for policy 1, policy_version 170213 (0.0010) [2023-12-26 16:36:00,333][105692] Updated weights for policy 0, policy_version 169245 (0.0010) [2023-12-26 16:36:00,394][105692] Updated weights for policy 0, policy_version 169255 (0.0009) [2023-12-26 16:36:00,450][105692] Updated weights for policy 0, policy_version 169265 (0.0009) [2023-12-26 16:36:00,514][105620] Updated weights for policy 1, policy_version 170223 (0.0010) [2023-12-26 16:36:00,574][105620] Updated weights for policy 1, policy_version 170233 (0.0010) [2023-12-26 16:36:00,633][105620] Updated weights for policy 1, policy_version 170243 (0.0010) [2023-12-26 16:36:01,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 86933504. Throughput: 0: 9890.6, 1: 9849.8. Samples: 86904744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:36:01,062][104569] Avg episode reward: [(0, '9258.126'), (1, '9178.834')] [2023-12-26 16:36:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000169272_43343872.pth... [2023-12-26 16:36:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000170248_43589632.pth... [2023-12-26 16:36:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000168152_43057152.pth [2023-12-26 16:36:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000169128_43302912.pth [2023-12-26 16:36:01,139][105692] Updated weights for policy 0, policy_version 169275 (0.0008) [2023-12-26 16:36:01,193][105692] Updated weights for policy 0, policy_version 169285 (0.0010) [2023-12-26 16:36:01,246][105692] Updated weights for policy 0, policy_version 169295 (0.0010) [2023-12-26 16:36:01,313][105620] Updated weights for policy 1, policy_version 170253 (0.0010) [2023-12-26 16:36:01,377][105620] Updated weights for policy 1, policy_version 170263 (0.0010) [2023-12-26 16:36:01,428][105620] Updated weights for policy 1, policy_version 170273 (0.0010) [2023-12-26 16:36:01,985][105692] Updated weights for policy 0, policy_version 169305 (0.0010) [2023-12-26 16:36:02,032][105692] Updated weights for policy 0, policy_version 169315 (0.0006) [2023-12-26 16:36:02,079][105692] Updated weights for policy 0, policy_version 169325 (0.0005) [2023-12-26 16:36:02,104][105620] Updated weights for policy 1, policy_version 170283 (0.0009) [2023-12-26 16:36:02,133][105692] Updated weights for policy 0, policy_version 169335 (0.0005) [2023-12-26 16:36:02,166][105620] Updated weights for policy 1, policy_version 170293 (0.0006) [2023-12-26 16:36:02,228][105620] Updated weights for policy 1, policy_version 170303 (0.0009) [2023-12-26 16:36:02,817][105620] Updated weights for policy 1, policy_version 170313 (0.0009) [2023-12-26 16:36:02,865][105692] Updated weights for policy 0, policy_version 169345 (0.0009) [2023-12-26 16:36:02,874][105620] Updated weights for policy 1, policy_version 170323 (0.0005) [2023-12-26 16:36:02,917][105692] Updated weights for policy 0, policy_version 169355 (0.0009) [2023-12-26 16:36:02,921][105620] Updated weights for policy 1, policy_version 170333 (0.0005) [2023-12-26 16:36:02,969][105620] Updated weights for policy 1, policy_version 170343 (0.0005) [2023-12-26 16:36:02,970][105692] Updated weights for policy 0, policy_version 169365 (0.0009) [2023-12-26 16:36:03,580][105620] Updated weights for policy 1, policy_version 170353 (0.0007) [2023-12-26 16:36:03,627][105620] Updated weights for policy 1, policy_version 170363 (0.0009) [2023-12-26 16:36:03,674][105620] Updated weights for policy 1, policy_version 170373 (0.0009) [2023-12-26 16:36:03,836][105692] Updated weights for policy 0, policy_version 169375 (0.0009) [2023-12-26 16:36:03,897][105692] Updated weights for policy 0, policy_version 169385 (0.0009) [2023-12-26 16:36:03,946][105692] Updated weights for policy 0, policy_version 169395 (0.0010) [2023-12-26 16:36:04,364][105620] Updated weights for policy 1, policy_version 170383 (0.0009) [2023-12-26 16:36:04,429][105620] Updated weights for policy 1, policy_version 170393 (0.0008) [2023-12-26 16:36:04,488][105620] Updated weights for policy 1, policy_version 170403 (0.0009) [2023-12-26 16:36:04,762][105692] Updated weights for policy 0, policy_version 169405 (0.0009) [2023-12-26 16:36:04,810][105692] Updated weights for policy 0, policy_version 169415 (0.0009) [2023-12-26 16:36:04,857][105692] Updated weights for policy 0, policy_version 169425 (0.0009) [2023-12-26 16:36:05,225][105620] Updated weights for policy 1, policy_version 170413 (0.0009) [2023-12-26 16:36:05,289][105620] Updated weights for policy 1, policy_version 170423 (0.0008) [2023-12-26 16:36:05,350][105620] Updated weights for policy 1, policy_version 170433 (0.0009) [2023-12-26 16:36:05,622][105692] Updated weights for policy 0, policy_version 169435 (0.0009) [2023-12-26 16:36:05,676][105692] Updated weights for policy 0, policy_version 169445 (0.0009) [2023-12-26 16:36:05,732][105692] Updated weights for policy 0, policy_version 169455 (0.0010) [2023-12-26 16:36:06,055][105620] Updated weights for policy 1, policy_version 170443 (0.0008) [2023-12-26 16:36:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 87031808. Throughput: 0: 9753.7, 1: 9942.5. Samples: 87023444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:36:06,063][104569] Avg episode reward: [(0, '9350.248'), (1, '9177.490')] [2023-12-26 16:36:06,117][105620] Updated weights for policy 1, policy_version 170453 (0.0007) [2023-12-26 16:36:06,184][105620] Updated weights for policy 1, policy_version 170463 (0.0009) [2023-12-26 16:36:06,531][105692] Updated weights for policy 0, policy_version 169466 (0.0010) [2023-12-26 16:36:06,580][105692] Updated weights for policy 0, policy_version 169476 (0.0009) [2023-12-26 16:36:06,636][105692] Updated weights for policy 0, policy_version 169486 (0.0009) [2023-12-26 16:36:06,696][105692] Updated weights for policy 0, policy_version 169496 (0.0009) [2023-12-26 16:36:06,914][105620] Updated weights for policy 1, policy_version 170473 (0.0009) [2023-12-26 16:36:06,974][105620] Updated weights for policy 1, policy_version 170483 (0.0010) [2023-12-26 16:36:07,030][105620] Updated weights for policy 1, policy_version 170494 (0.0007) [2023-12-26 16:36:07,083][105620] Updated weights for policy 1, policy_version 170504 (0.0008) [2023-12-26 16:36:07,434][105692] Updated weights for policy 0, policy_version 169506 (0.0005) [2023-12-26 16:36:07,491][105692] Updated weights for policy 0, policy_version 169516 (0.0005) [2023-12-26 16:36:07,557][105692] Updated weights for policy 0, policy_version 169526 (0.0006) [2023-12-26 16:36:07,791][105620] Updated weights for policy 1, policy_version 170514 (0.0010) [2023-12-26 16:36:07,850][105620] Updated weights for policy 1, policy_version 170524 (0.0010) [2023-12-26 16:36:07,908][105620] Updated weights for policy 1, policy_version 170534 (0.0010) [2023-12-26 16:36:08,101][105692] Updated weights for policy 0, policy_version 169536 (0.0009) [2023-12-26 16:36:08,156][105692] Updated weights for policy 0, policy_version 169546 (0.0010) [2023-12-26 16:36:08,212][105692] Updated weights for policy 0, policy_version 169556 (0.0009) [2023-12-26 16:36:08,609][105620] Updated weights for policy 1, policy_version 170544 (0.0010) [2023-12-26 16:36:08,671][105620] Updated weights for policy 1, policy_version 170554 (0.0009) [2023-12-26 16:36:08,741][105620] Updated weights for policy 1, policy_version 170564 (0.0009) [2023-12-26 16:36:09,006][105692] Updated weights for policy 0, policy_version 169566 (0.0008) [2023-12-26 16:36:09,070][105692] Updated weights for policy 0, policy_version 169576 (0.0008) [2023-12-26 16:36:09,133][105692] Updated weights for policy 0, policy_version 169586 (0.0008) [2023-12-26 16:36:09,436][105620] Updated weights for policy 1, policy_version 170574 (0.0010) [2023-12-26 16:36:09,500][105620] Updated weights for policy 1, policy_version 170584 (0.0007) [2023-12-26 16:36:09,567][105620] Updated weights for policy 1, policy_version 170594 (0.0008) [2023-12-26 16:36:09,920][105692] Updated weights for policy 0, policy_version 169596 (0.0008) [2023-12-26 16:36:09,987][105692] Updated weights for policy 0, policy_version 169606 (0.0009) [2023-12-26 16:36:10,043][105692] Updated weights for policy 0, policy_version 169616 (0.0009) [2023-12-26 16:36:10,250][105620] Updated weights for policy 1, policy_version 170604 (0.0008) [2023-12-26 16:36:10,303][105620] Updated weights for policy 1, policy_version 170614 (0.0008) [2023-12-26 16:36:10,355][105620] Updated weights for policy 1, policy_version 170624 (0.0009) [2023-12-26 16:36:10,840][105692] Updated weights for policy 0, policy_version 169626 (0.0008) [2023-12-26 16:36:10,898][105692] Updated weights for policy 0, policy_version 169637 (0.0010) [2023-12-26 16:36:10,952][105692] Updated weights for policy 0, policy_version 169647 (0.0010) [2023-12-26 16:36:11,004][105620] Updated weights for policy 1, policy_version 170634 (0.0009) [2023-12-26 16:36:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.9, 300 sec: 19660.8). Total num frames: 87130112. Throughput: 0: 9718.6, 1: 9946.0. Samples: 87139268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:36:11,062][104569] Avg episode reward: [(0, '9351.287'), (1, '9263.590')] [2023-12-26 16:36:11,069][105620] Updated weights for policy 1, policy_version 170644 (0.0008) [2023-12-26 16:36:11,137][105620] Updated weights for policy 1, policy_version 170654 (0.0009) [2023-12-26 16:36:11,205][105620] Updated weights for policy 1, policy_version 170664 (0.0005) [2023-12-26 16:36:11,777][105692] Updated weights for policy 0, policy_version 169658 (0.0011) [2023-12-26 16:36:11,847][105692] Updated weights for policy 0, policy_version 169668 (0.0009) [2023-12-26 16:36:11,913][105692] Updated weights for policy 0, policy_version 169678 (0.0009) [2023-12-26 16:36:11,956][105620] Updated weights for policy 1, policy_version 170674 (0.0006) [2023-12-26 16:36:11,978][105692] Updated weights for policy 0, policy_version 169688 (0.0007) [2023-12-26 16:36:12,014][105620] Updated weights for policy 1, policy_version 170684 (0.0008) [2023-12-26 16:36:12,084][105620] Updated weights for policy 1, policy_version 170694 (0.0009) [2023-12-26 16:36:12,748][105692] Updated weights for policy 0, policy_version 169698 (0.0010) [2023-12-26 16:36:12,799][105692] Updated weights for policy 0, policy_version 169708 (0.0009) [2023-12-26 16:36:12,832][105620] Updated weights for policy 1, policy_version 170704 (0.0009) [2023-12-26 16:36:12,851][105692] Updated weights for policy 0, policy_version 169718 (0.0005) [2023-12-26 16:36:12,892][105620] Updated weights for policy 1, policy_version 170714 (0.0008) [2023-12-26 16:36:12,946][105620] Updated weights for policy 1, policy_version 170724 (0.0009) [2023-12-26 16:36:13,616][105692] Updated weights for policy 0, policy_version 169728 (0.0008) [2023-12-26 16:36:13,675][105692] Updated weights for policy 0, policy_version 169738 (0.0010) [2023-12-26 16:36:13,697][105620] Updated weights for policy 1, policy_version 170734 (0.0009) [2023-12-26 16:36:13,734][105692] Updated weights for policy 0, policy_version 169748 (0.0009) [2023-12-26 16:36:13,756][105620] Updated weights for policy 1, policy_version 170744 (0.0009) [2023-12-26 16:36:13,815][105620] Updated weights for policy 1, policy_version 170754 (0.0008) [2023-12-26 16:36:14,440][105692] Updated weights for policy 0, policy_version 169758 (0.0009) [2023-12-26 16:36:14,488][105692] Updated weights for policy 0, policy_version 169768 (0.0010) [2023-12-26 16:36:14,517][105620] Updated weights for policy 1, policy_version 170765 (0.0008) [2023-12-26 16:36:14,550][105692] Updated weights for policy 0, policy_version 169778 (0.0010) [2023-12-26 16:36:14,578][105620] Updated weights for policy 1, policy_version 170775 (0.0006) [2023-12-26 16:36:14,627][105620] Updated weights for policy 1, policy_version 170785 (0.0005) [2023-12-26 16:36:15,214][105620] Updated weights for policy 1, policy_version 170795 (0.0006) [2023-12-26 16:36:15,272][105692] Updated weights for policy 0, policy_version 169788 (0.0009) [2023-12-26 16:36:15,274][105620] Updated weights for policy 1, policy_version 170805 (0.0008) [2023-12-26 16:36:15,325][105692] Updated weights for policy 0, policy_version 169798 (0.0008) [2023-12-26 16:36:15,330][105620] Updated weights for policy 1, policy_version 170815 (0.0007) [2023-12-26 16:36:15,374][105692] Updated weights for policy 0, policy_version 169808 (0.0008) [2023-12-26 16:36:15,993][105620] Updated weights for policy 1, policy_version 170825 (0.0006) [2023-12-26 16:36:16,051][105620] Updated weights for policy 1, policy_version 170835 (0.0005) [2023-12-26 16:36:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 87220224. Throughput: 0: 9581.0, 1: 9790.1. Samples: 87193484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:36:16,063][104569] Avg episode reward: [(0, '9349.724'), (1, '9263.642')] [2023-12-26 16:36:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000169816_43483136.pth... [2023-12-26 16:36:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000168728_43204608.pth [2023-12-26 16:36:16,110][105620] Updated weights for policy 1, policy_version 170845 (0.0005) [2023-12-26 16:36:16,154][105620] Updated weights for policy 1, policy_version 170855 (0.0005) [2023-12-26 16:36:16,158][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000170856_43745280.pth... [2023-12-26 16:36:16,161][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000169704_43450368.pth [2023-12-26 16:36:16,202][105692] Updated weights for policy 0, policy_version 169818 (0.0007) [2023-12-26 16:36:16,260][105692] Updated weights for policy 0, policy_version 169828 (0.0005) [2023-12-26 16:36:16,311][105692] Updated weights for policy 0, policy_version 169838 (0.0006) [2023-12-26 16:36:16,359][105692] Updated weights for policy 0, policy_version 169848 (0.0005) [2023-12-26 16:36:16,744][105620] Updated weights for policy 1, policy_version 170865 (0.0010) [2023-12-26 16:36:16,804][105620] Updated weights for policy 1, policy_version 170875 (0.0011) [2023-12-26 16:36:16,857][105620] Updated weights for policy 1, policy_version 170885 (0.0010) [2023-12-26 16:36:16,976][105692] Updated weights for policy 0, policy_version 169858 (0.0007) [2023-12-26 16:36:17,030][105692] Updated weights for policy 0, policy_version 169868 (0.0005) [2023-12-26 16:36:17,087][105692] Updated weights for policy 0, policy_version 169878 (0.0007) [2023-12-26 16:36:17,545][105620] Updated weights for policy 1, policy_version 170895 (0.0007) [2023-12-26 16:36:17,603][105620] Updated weights for policy 1, policy_version 170905 (0.0005) [2023-12-26 16:36:17,660][105620] Updated weights for policy 1, policy_version 170915 (0.0005) [2023-12-26 16:36:17,821][105692] Updated weights for policy 0, policy_version 169888 (0.0006) [2023-12-26 16:36:17,867][105692] Updated weights for policy 0, policy_version 169898 (0.0005) [2023-12-26 16:36:17,915][105692] Updated weights for policy 0, policy_version 169908 (0.0005) [2023-12-26 16:36:18,197][105620] Updated weights for policy 1, policy_version 170925 (0.0008) [2023-12-26 16:36:18,261][105620] Updated weights for policy 1, policy_version 170935 (0.0011) [2023-12-26 16:36:18,330][105620] Updated weights for policy 1, policy_version 170945 (0.0011) [2023-12-26 16:36:18,569][105692] Updated weights for policy 0, policy_version 169918 (0.0005) [2023-12-26 16:36:18,639][105692] Updated weights for policy 0, policy_version 169928 (0.0009) [2023-12-26 16:36:18,706][105692] Updated weights for policy 0, policy_version 169938 (0.0011) [2023-12-26 16:36:18,977][105620] Updated weights for policy 1, policy_version 170955 (0.0007) [2023-12-26 16:36:19,034][105620] Updated weights for policy 1, policy_version 170965 (0.0005) [2023-12-26 16:36:19,094][105620] Updated weights for policy 1, policy_version 170975 (0.0007) [2023-12-26 16:36:19,451][105692] Updated weights for policy 0, policy_version 169948 (0.0011) [2023-12-26 16:36:19,518][105692] Updated weights for policy 0, policy_version 169958 (0.0011) [2023-12-26 16:36:19,578][105692] Updated weights for policy 0, policy_version 169968 (0.0010) [2023-12-26 16:36:19,769][105620] Updated weights for policy 1, policy_version 170985 (0.0008) [2023-12-26 16:36:19,826][105620] Updated weights for policy 1, policy_version 170995 (0.0011) [2023-12-26 16:36:19,880][105620] Updated weights for policy 1, policy_version 171005 (0.0011) [2023-12-26 16:36:19,954][105620] Updated weights for policy 1, policy_version 171015 (0.0011) [2023-12-26 16:36:20,341][105692] Updated weights for policy 0, policy_version 169978 (0.0011) [2023-12-26 16:36:20,399][105692] Updated weights for policy 0, policy_version 169988 (0.0010) [2023-12-26 16:36:20,465][105692] Updated weights for policy 0, policy_version 169998 (0.0010) [2023-12-26 16:36:20,533][105692] Updated weights for policy 0, policy_version 170008 (0.0010) [2023-12-26 16:36:20,711][105620] Updated weights for policy 1, policy_version 171025 (0.0006) [2023-12-26 16:36:20,768][105620] Updated weights for policy 1, policy_version 171035 (0.0006) [2023-12-26 16:36:20,829][105620] Updated weights for policy 1, policy_version 171045 (0.0006) [2023-12-26 16:36:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 87326720. Throughput: 0: 9517.6, 1: 9872.7. Samples: 87316808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:36:21,062][104569] Avg episode reward: [(0, '9349.242'), (1, '9176.452')] [2023-12-26 16:36:21,297][105692] Updated weights for policy 0, policy_version 170018 (0.0009) [2023-12-26 16:36:21,358][105692] Updated weights for policy 0, policy_version 170028 (0.0011) [2023-12-26 16:36:21,427][105692] Updated weights for policy 0, policy_version 170038 (0.0011) [2023-12-26 16:36:21,486][105620] Updated weights for policy 1, policy_version 171055 (0.0009) [2023-12-26 16:36:21,550][105620] Updated weights for policy 1, policy_version 171065 (0.0009) [2023-12-26 16:36:21,614][105620] Updated weights for policy 1, policy_version 171075 (0.0008) [2023-12-26 16:36:22,245][105620] Updated weights for policy 1, policy_version 171085 (0.0008) [2023-12-26 16:36:22,278][105692] Updated weights for policy 0, policy_version 170048 (0.0009) [2023-12-26 16:36:22,303][105620] Updated weights for policy 1, policy_version 171095 (0.0008) [2023-12-26 16:36:22,344][105692] Updated weights for policy 0, policy_version 170058 (0.0006) [2023-12-26 16:36:22,359][105620] Updated weights for policy 1, policy_version 171105 (0.0008) [2023-12-26 16:36:22,412][105692] Updated weights for policy 0, policy_version 170068 (0.0007) [2023-12-26 16:36:23,058][105692] Updated weights for policy 0, policy_version 170078 (0.0009) [2023-12-26 16:36:23,120][105692] Updated weights for policy 0, policy_version 170088 (0.0011) [2023-12-26 16:36:23,135][105620] Updated weights for policy 1, policy_version 171115 (0.0009) [2023-12-26 16:36:23,176][105692] Updated weights for policy 0, policy_version 170098 (0.0010) [2023-12-26 16:36:23,180][105620] Updated weights for policy 1, policy_version 171125 (0.0010) [2023-12-26 16:36:23,232][105620] Updated weights for policy 1, policy_version 171135 (0.0010) [2023-12-26 16:36:23,853][105692] Updated weights for policy 0, policy_version 170108 (0.0010) [2023-12-26 16:36:23,905][105692] Updated weights for policy 0, policy_version 170118 (0.0008) [2023-12-26 16:36:23,958][105692] Updated weights for policy 0, policy_version 170128 (0.0008) [2023-12-26 16:36:23,996][105620] Updated weights for policy 1, policy_version 171145 (0.0010) [2023-12-26 16:36:24,051][105620] Updated weights for policy 1, policy_version 171155 (0.0010) [2023-12-26 16:36:24,109][105620] Updated weights for policy 1, policy_version 171165 (0.0009) [2023-12-26 16:36:24,158][105620] Updated weights for policy 1, policy_version 171175 (0.0010) [2023-12-26 16:36:24,641][105692] Updated weights for policy 0, policy_version 170138 (0.0008) [2023-12-26 16:36:24,702][105692] Updated weights for policy 0, policy_version 170148 (0.0007) [2023-12-26 16:36:24,766][105692] Updated weights for policy 0, policy_version 170158 (0.0008) [2023-12-26 16:36:24,827][105692] Updated weights for policy 0, policy_version 170168 (0.0008) [2023-12-26 16:36:24,902][105620] Updated weights for policy 1, policy_version 171185 (0.0010) [2023-12-26 16:36:24,953][105620] Updated weights for policy 1, policy_version 171195 (0.0010) [2023-12-26 16:36:25,012][105620] Updated weights for policy 1, policy_version 171205 (0.0010) [2023-12-26 16:36:25,440][105692] Updated weights for policy 0, policy_version 170178 (0.0009) [2023-12-26 16:36:25,495][105692] Updated weights for policy 0, policy_version 170188 (0.0010) [2023-12-26 16:36:25,542][105692] Updated weights for policy 0, policy_version 170198 (0.0010) [2023-12-26 16:36:25,766][105620] Updated weights for policy 1, policy_version 171215 (0.0010) [2023-12-26 16:36:25,823][105620] Updated weights for policy 1, policy_version 171225 (0.0010) [2023-12-26 16:36:25,868][105620] Updated weights for policy 1, policy_version 171235 (0.0010) [2023-12-26 16:36:26,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 87425024. Throughput: 0: 9469.4, 1: 9784.6. Samples: 87432160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:36:26,062][104569] Avg episode reward: [(0, '9327.380'), (1, '9010.742')] [2023-12-26 16:36:26,157][105692] Updated weights for policy 0, policy_version 170208 (0.0006) [2023-12-26 16:36:26,204][105692] Updated weights for policy 0, policy_version 170218 (0.0005) [2023-12-26 16:36:26,253][105692] Updated weights for policy 0, policy_version 170228 (0.0005) [2023-12-26 16:36:26,578][105620] Updated weights for policy 1, policy_version 171245 (0.0008) [2023-12-26 16:36:26,636][105620] Updated weights for policy 1, policy_version 171255 (0.0005) [2023-12-26 16:36:26,685][105620] Updated weights for policy 1, policy_version 171265 (0.0009) [2023-12-26 16:36:26,856][105692] Updated weights for policy 0, policy_version 170238 (0.0007) [2023-12-26 16:36:26,911][105692] Updated weights for policy 0, policy_version 170248 (0.0007) [2023-12-26 16:36:26,962][105692] Updated weights for policy 0, policy_version 170258 (0.0005) [2023-12-26 16:36:27,337][105620] Updated weights for policy 1, policy_version 171275 (0.0007) [2023-12-26 16:36:27,392][105620] Updated weights for policy 1, policy_version 171285 (0.0010) [2023-12-26 16:36:27,450][105620] Updated weights for policy 1, policy_version 171295 (0.0010) [2023-12-26 16:36:27,562][105692] Updated weights for policy 0, policy_version 170268 (0.0005) [2023-12-26 16:36:27,620][105692] Updated weights for policy 0, policy_version 170278 (0.0008) [2023-12-26 16:36:27,683][105692] Updated weights for policy 0, policy_version 170288 (0.0008) [2023-12-26 16:36:28,142][105620] Updated weights for policy 1, policy_version 171305 (0.0010) [2023-12-26 16:36:28,209][105620] Updated weights for policy 1, policy_version 171315 (0.0006) [2023-12-26 16:36:28,267][105620] Updated weights for policy 1, policy_version 171325 (0.0010) [2023-12-26 16:36:28,311][105692] Updated weights for policy 0, policy_version 170298 (0.0009) [2023-12-26 16:36:28,322][105620] Updated weights for policy 1, policy_version 171335 (0.0009) [2023-12-26 16:36:28,369][105692] Updated weights for policy 0, policy_version 170308 (0.0008) [2023-12-26 16:36:28,433][105692] Updated weights for policy 0, policy_version 170318 (0.0008) [2023-12-26 16:36:28,489][105692] Updated weights for policy 0, policy_version 170328 (0.0008) [2023-12-26 16:36:29,044][105620] Updated weights for policy 1, policy_version 171345 (0.0010) [2023-12-26 16:36:29,108][105620] Updated weights for policy 1, policy_version 171355 (0.0010) [2023-12-26 16:36:29,133][105692] Updated weights for policy 0, policy_version 170338 (0.0007) [2023-12-26 16:36:29,162][105620] Updated weights for policy 1, policy_version 171365 (0.0010) [2023-12-26 16:36:29,182][105692] Updated weights for policy 0, policy_version 170348 (0.0010) [2023-12-26 16:36:29,244][105692] Updated weights for policy 0, policy_version 170358 (0.0008) [2023-12-26 16:36:29,895][105692] Updated weights for policy 0, policy_version 170368 (0.0007) [2023-12-26 16:36:29,902][105620] Updated weights for policy 1, policy_version 171375 (0.0007) [2023-12-26 16:36:29,959][105692] Updated weights for policy 0, policy_version 170378 (0.0008) [2023-12-26 16:36:29,963][105620] Updated weights for policy 1, policy_version 171385 (0.0008) [2023-12-26 16:36:30,019][105692] Updated weights for policy 0, policy_version 170388 (0.0006) [2023-12-26 16:36:30,023][105620] Updated weights for policy 1, policy_version 171395 (0.0008) [2023-12-26 16:36:30,562][105620] Updated weights for policy 1, policy_version 171405 (0.0007) [2023-12-26 16:36:30,608][105620] Updated weights for policy 1, policy_version 171415 (0.0005) [2023-12-26 16:36:30,663][105620] Updated weights for policy 1, policy_version 171425 (0.0005) [2023-12-26 16:36:30,684][105692] Updated weights for policy 0, policy_version 170398 (0.0008) [2023-12-26 16:36:30,741][105692] Updated weights for policy 0, policy_version 170408 (0.0008) [2023-12-26 16:36:30,796][105692] Updated weights for policy 0, policy_version 170418 (0.0006) [2023-12-26 16:36:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 87531520. Throughput: 0: 9605.7, 1: 9844.4. Samples: 87496268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:36:31,062][104569] Avg episode reward: [(0, '1188.497'), (1, '8963.987')] [2023-12-26 16:36:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000171432_43892736.pth... [2023-12-26 16:36:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000170424_43638784.pth... [2023-12-26 16:36:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000169272_43343872.pth [2023-12-26 16:36:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000170248_43589632.pth [2023-12-26 16:36:31,256][105620] Updated weights for policy 1, policy_version 171435 (0.0006) [2023-12-26 16:36:31,311][105620] Updated weights for policy 1, policy_version 171445 (0.0009) [2023-12-26 16:36:31,377][105620] Updated weights for policy 1, policy_version 171455 (0.0006) [2023-12-26 16:36:31,431][105692] Updated weights for policy 0, policy_version 170428 (0.0006) [2023-12-26 16:36:31,500][105692] Updated weights for policy 0, policy_version 170438 (0.0009) [2023-12-26 16:36:31,557][105692] Updated weights for policy 0, policy_version 170448 (0.0009) [2023-12-26 16:36:31,983][105620] Updated weights for policy 1, policy_version 171465 (0.0005) [2023-12-26 16:36:32,048][105620] Updated weights for policy 1, policy_version 171475 (0.0010) [2023-12-26 16:36:32,108][105620] Updated weights for policy 1, policy_version 171485 (0.0009) [2023-12-26 16:36:32,177][105620] Updated weights for policy 1, policy_version 171495 (0.0009) [2023-12-26 16:36:32,288][105692] Updated weights for policy 0, policy_version 170458 (0.0009) [2023-12-26 16:36:32,350][105692] Updated weights for policy 0, policy_version 170468 (0.0007) [2023-12-26 16:36:32,412][105692] Updated weights for policy 0, policy_version 170478 (0.0009) [2023-12-26 16:36:32,467][105692] Updated weights for policy 0, policy_version 170488 (0.0009) [2023-12-26 16:36:32,948][105620] Updated weights for policy 1, policy_version 171505 (0.0010) [2023-12-26 16:36:32,999][105620] Updated weights for policy 1, policy_version 171515 (0.0009) [2023-12-26 16:36:33,048][105620] Updated weights for policy 1, policy_version 171525 (0.0008) [2023-12-26 16:36:33,183][105692] Updated weights for policy 0, policy_version 170498 (0.0010) [2023-12-26 16:36:33,240][105692] Updated weights for policy 0, policy_version 170508 (0.0010) [2023-12-26 16:36:33,293][105692] Updated weights for policy 0, policy_version 170518 (0.0010) [2023-12-26 16:36:33,838][105620] Updated weights for policy 1, policy_version 171535 (0.0008) [2023-12-26 16:36:33,885][105692] Updated weights for policy 0, policy_version 170528 (0.0006) [2023-12-26 16:36:33,894][105620] Updated weights for policy 1, policy_version 171545 (0.0008) [2023-12-26 16:36:33,939][105692] Updated weights for policy 0, policy_version 170538 (0.0005) [2023-12-26 16:36:33,951][105620] Updated weights for policy 1, policy_version 171555 (0.0008) [2023-12-26 16:36:33,994][105692] Updated weights for policy 0, policy_version 170548 (0.0005) [2023-12-26 16:36:34,655][105692] Updated weights for policy 0, policy_version 170558 (0.0008) [2023-12-26 16:36:34,716][105692] Updated weights for policy 0, policy_version 170569 (0.0007) [2023-12-26 16:36:34,738][105620] Updated weights for policy 1, policy_version 171565 (0.0008) [2023-12-26 16:36:34,773][105692] Updated weights for policy 0, policy_version 170579 (0.0006) [2023-12-26 16:36:34,797][105620] Updated weights for policy 1, policy_version 171575 (0.0007) [2023-12-26 16:36:34,852][105620] Updated weights for policy 1, policy_version 171585 (0.0009) [2023-12-26 16:36:35,473][105692] Updated weights for policy 0, policy_version 170589 (0.0009) [2023-12-26 16:36:35,528][105692] Updated weights for policy 0, policy_version 170599 (0.0010) [2023-12-26 16:36:35,590][105692] Updated weights for policy 0, policy_version 170609 (0.0011) [2023-12-26 16:36:35,624][105620] Updated weights for policy 1, policy_version 171595 (0.0008) [2023-12-26 16:36:35,679][105620] Updated weights for policy 1, policy_version 171605 (0.0008) [2023-12-26 16:36:35,738][105620] Updated weights for policy 1, policy_version 171615 (0.0008) [2023-12-26 16:36:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 87629824. Throughput: 0: 9667.4, 1: 9964.2. Samples: 87618348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:36:36,062][104569] Avg episode reward: [(0, '3153.383'), (1, '9028.696')] [2023-12-26 16:36:36,361][105692] Updated weights for policy 0, policy_version 170619 (0.0011) [2023-12-26 16:36:36,424][105692] Updated weights for policy 0, policy_version 170629 (0.0011) [2023-12-26 16:36:36,483][105692] Updated weights for policy 0, policy_version 170639 (0.0010) [2023-12-26 16:36:36,504][105620] Updated weights for policy 1, policy_version 171625 (0.0008) [2023-12-26 16:36:36,566][105620] Updated weights for policy 1, policy_version 171635 (0.0007) [2023-12-26 16:36:36,626][105620] Updated weights for policy 1, policy_version 171645 (0.0008) [2023-12-26 16:36:36,675][105620] Updated weights for policy 1, policy_version 171655 (0.0008) [2023-12-26 16:36:37,226][105692] Updated weights for policy 0, policy_version 170649 (0.0011) [2023-12-26 16:36:37,286][105692] Updated weights for policy 0, policy_version 170659 (0.0011) [2023-12-26 16:36:37,344][105692] Updated weights for policy 0, policy_version 170669 (0.0010) [2023-12-26 16:36:37,406][105692] Updated weights for policy 0, policy_version 170679 (0.0010) [2023-12-26 16:36:37,446][105620] Updated weights for policy 1, policy_version 171665 (0.0007) [2023-12-26 16:36:37,501][105620] Updated weights for policy 1, policy_version 171675 (0.0008) [2023-12-26 16:36:37,559][105620] Updated weights for policy 1, policy_version 171685 (0.0008) [2023-12-26 16:36:38,135][105692] Updated weights for policy 0, policy_version 170689 (0.0006) [2023-12-26 16:36:38,184][105692] Updated weights for policy 0, policy_version 170699 (0.0006) [2023-12-26 16:36:38,239][105692] Updated weights for policy 0, policy_version 170709 (0.0006) [2023-12-26 16:36:38,256][105620] Updated weights for policy 1, policy_version 171695 (0.0007) [2023-12-26 16:36:38,331][105620] Updated weights for policy 1, policy_version 171705 (0.0008) [2023-12-26 16:36:38,392][105620] Updated weights for policy 1, policy_version 171715 (0.0009) [2023-12-26 16:36:38,845][105692] Updated weights for policy 0, policy_version 170719 (0.0007) [2023-12-26 16:36:38,904][105692] Updated weights for policy 0, policy_version 170729 (0.0010) [2023-12-26 16:36:38,969][105692] Updated weights for policy 0, policy_version 170739 (0.0011) [2023-12-26 16:36:39,003][105620] Updated weights for policy 1, policy_version 171725 (0.0011) [2023-12-26 16:36:39,058][105620] Updated weights for policy 1, policy_version 171735 (0.0010) [2023-12-26 16:36:39,102][105620] Updated weights for policy 1, policy_version 171745 (0.0010) [2023-12-26 16:36:39,710][105692] Updated weights for policy 0, policy_version 170749 (0.0008) [2023-12-26 16:36:39,781][105692] Updated weights for policy 0, policy_version 170759 (0.0007) [2023-12-26 16:36:39,810][105620] Updated weights for policy 1, policy_version 171755 (0.0009) [2023-12-26 16:36:39,853][105692] Updated weights for policy 0, policy_version 170769 (0.0008) [2023-12-26 16:36:39,874][105620] Updated weights for policy 1, policy_version 171765 (0.0007) [2023-12-26 16:36:39,942][105620] Updated weights for policy 1, policy_version 171775 (0.0007) [2023-12-26 16:36:40,587][105692] Updated weights for policy 0, policy_version 170779 (0.0009) [2023-12-26 16:36:40,630][105620] Updated weights for policy 1, policy_version 171785 (0.0008) [2023-12-26 16:36:40,639][105692] Updated weights for policy 0, policy_version 170789 (0.0009) [2023-12-26 16:36:40,680][105620] Updated weights for policy 1, policy_version 171795 (0.0005) [2023-12-26 16:36:40,684][105692] Updated weights for policy 0, policy_version 170799 (0.0008) [2023-12-26 16:36:40,728][105620] Updated weights for policy 1, policy_version 171805 (0.0007) [2023-12-26 16:36:40,777][105620] Updated weights for policy 1, policy_version 171815 (0.0005) [2023-12-26 16:36:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 87728128. Throughput: 0: 9758.8, 1: 9957.0. Samples: 87735104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:36:41,062][104569] Avg episode reward: [(0, '6497.525'), (1, '9070.084')] [2023-12-26 16:36:41,508][105692] Updated weights for policy 0, policy_version 170809 (0.0010) [2023-12-26 16:36:41,558][105620] Updated weights for policy 1, policy_version 171825 (0.0007) [2023-12-26 16:36:41,572][105692] Updated weights for policy 0, policy_version 170819 (0.0008) [2023-12-26 16:36:41,627][105620] Updated weights for policy 1, policy_version 171835 (0.0007) [2023-12-26 16:36:41,630][105692] Updated weights for policy 0, policy_version 170829 (0.0007) [2023-12-26 16:36:41,691][105620] Updated weights for policy 1, policy_version 171845 (0.0010) [2023-12-26 16:36:41,691][105692] Updated weights for policy 0, policy_version 170839 (0.0010) [2023-12-26 16:36:42,358][105620] Updated weights for policy 1, policy_version 171855 (0.0009) [2023-12-26 16:36:42,420][105620] Updated weights for policy 1, policy_version 171865 (0.0008) [2023-12-26 16:36:42,477][105620] Updated weights for policy 1, policy_version 171875 (0.0010) [2023-12-26 16:36:42,514][105692] Updated weights for policy 0, policy_version 170849 (0.0006) [2023-12-26 16:36:42,578][105692] Updated weights for policy 0, policy_version 170859 (0.0006) [2023-12-26 16:36:42,633][105692] Updated weights for policy 0, policy_version 170869 (0.0005) [2023-12-26 16:36:43,086][105620] Updated weights for policy 1, policy_version 171885 (0.0008) [2023-12-26 16:36:43,145][105620] Updated weights for policy 1, policy_version 171895 (0.0009) [2023-12-26 16:36:43,203][105620] Updated weights for policy 1, policy_version 171905 (0.0009) [2023-12-26 16:36:43,238][105692] Updated weights for policy 0, policy_version 170879 (0.0009) [2023-12-26 16:36:43,291][105692] Updated weights for policy 0, policy_version 170889 (0.0011) [2023-12-26 16:36:43,341][105692] Updated weights for policy 0, policy_version 170899 (0.0011) [2023-12-26 16:36:43,872][105620] Updated weights for policy 1, policy_version 171915 (0.0006) [2023-12-26 16:36:43,922][105620] Updated weights for policy 1, policy_version 171925 (0.0007) [2023-12-26 16:36:43,975][105620] Updated weights for policy 1, policy_version 171936 (0.0009) [2023-12-26 16:36:43,994][105692] Updated weights for policy 0, policy_version 170909 (0.0009) [2023-12-26 16:36:44,041][105692] Updated weights for policy 0, policy_version 170919 (0.0010) [2023-12-26 16:36:44,098][105692] Updated weights for policy 0, policy_version 170929 (0.0005) [2023-12-26 16:36:44,651][105692] Updated weights for policy 0, policy_version 170939 (0.0007) [2023-12-26 16:36:44,706][105692] Updated weights for policy 0, policy_version 170949 (0.0009) [2023-12-26 16:36:44,767][105692] Updated weights for policy 0, policy_version 170959 (0.0006) [2023-12-26 16:36:44,819][105620] Updated weights for policy 1, policy_version 171946 (0.0008) [2023-12-26 16:36:44,874][105620] Updated weights for policy 1, policy_version 171956 (0.0011) [2023-12-26 16:36:44,934][105620] Updated weights for policy 1, policy_version 171966 (0.0011) [2023-12-26 16:36:44,993][105620] Updated weights for policy 1, policy_version 171976 (0.0011) [2023-12-26 16:36:45,512][105692] Updated weights for policy 0, policy_version 170969 (0.0008) [2023-12-26 16:36:45,560][105692] Updated weights for policy 0, policy_version 170979 (0.0008) [2023-12-26 16:36:45,611][105692] Updated weights for policy 0, policy_version 170989 (0.0008) [2023-12-26 16:36:45,675][105692] Updated weights for policy 0, policy_version 170999 (0.0010) [2023-12-26 16:36:45,741][105620] Updated weights for policy 1, policy_version 171986 (0.0010) [2023-12-26 16:36:45,803][105620] Updated weights for policy 1, policy_version 171996 (0.0006) [2023-12-26 16:36:45,861][105620] Updated weights for policy 1, policy_version 172006 (0.0005) [2023-12-26 16:36:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 87826432. Throughput: 0: 9725.1, 1: 10025.4. Samples: 87793520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:36:46,063][104569] Avg episode reward: [(0, '9349.203'), (1, '8833.058')] [2023-12-26 16:36:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000171000_43786240.pth... [2023-12-26 16:36:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000172008_44040192.pth... [2023-12-26 16:36:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000169816_43483136.pth [2023-12-26 16:36:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000170856_43745280.pth [2023-12-26 16:36:46,387][105620] Updated weights for policy 1, policy_version 172016 (0.0009) [2023-12-26 16:36:46,432][105692] Updated weights for policy 0, policy_version 171009 (0.0006) [2023-12-26 16:36:46,436][105620] Updated weights for policy 1, policy_version 172026 (0.0010) [2023-12-26 16:36:46,476][105692] Updated weights for policy 0, policy_version 171019 (0.0005) [2023-12-26 16:36:46,484][105620] Updated weights for policy 1, policy_version 172036 (0.0010) [2023-12-26 16:36:46,522][105692] Updated weights for policy 0, policy_version 171029 (0.0005) [2023-12-26 16:36:47,120][105620] Updated weights for policy 1, policy_version 172046 (0.0007) [2023-12-26 16:36:47,165][105620] Updated weights for policy 1, policy_version 172056 (0.0005) [2023-12-26 16:36:47,227][105620] Updated weights for policy 1, policy_version 172066 (0.0008) [2023-12-26 16:36:47,299][105692] Updated weights for policy 0, policy_version 171039 (0.0007) [2023-12-26 16:36:47,361][105692] Updated weights for policy 0, policy_version 171049 (0.0010) [2023-12-26 16:36:47,431][105692] Updated weights for policy 0, policy_version 171059 (0.0009) [2023-12-26 16:36:47,805][105620] Updated weights for policy 1, policy_version 172076 (0.0010) [2023-12-26 16:36:47,852][105620] Updated weights for policy 1, policy_version 172086 (0.0010) [2023-12-26 16:36:47,904][105620] Updated weights for policy 1, policy_version 172096 (0.0010) [2023-12-26 16:36:48,194][105692] Updated weights for policy 0, policy_version 171069 (0.0007) [2023-12-26 16:36:48,252][105692] Updated weights for policy 0, policy_version 171079 (0.0007) [2023-12-26 16:36:48,315][105692] Updated weights for policy 0, policy_version 171090 (0.0010) [2023-12-26 16:36:48,635][105620] Updated weights for policy 1, policy_version 172106 (0.0010) [2023-12-26 16:36:48,700][105620] Updated weights for policy 1, policy_version 172116 (0.0009) [2023-12-26 16:36:48,764][105620] Updated weights for policy 1, policy_version 172126 (0.0011) [2023-12-26 16:36:48,830][105620] Updated weights for policy 1, policy_version 172136 (0.0009) [2023-12-26 16:36:49,004][105692] Updated weights for policy 0, policy_version 171100 (0.0008) [2023-12-26 16:36:49,055][105692] Updated weights for policy 0, policy_version 171110 (0.0008) [2023-12-26 16:36:49,111][105692] Updated weights for policy 0, policy_version 171120 (0.0008) [2023-12-26 16:36:49,536][105620] Updated weights for policy 1, policy_version 172146 (0.0009) [2023-12-26 16:36:49,597][105620] Updated weights for policy 1, policy_version 172156 (0.0007) [2023-12-26 16:36:49,651][105620] Updated weights for policy 1, policy_version 172166 (0.0009) [2023-12-26 16:36:49,889][105692] Updated weights for policy 0, policy_version 171130 (0.0008) [2023-12-26 16:36:49,950][105692] Updated weights for policy 0, policy_version 171140 (0.0008) [2023-12-26 16:36:50,004][105692] Updated weights for policy 0, policy_version 171150 (0.0007) [2023-12-26 16:36:50,065][105692] Updated weights for policy 0, policy_version 171160 (0.0010) [2023-12-26 16:36:50,364][105620] Updated weights for policy 1, policy_version 172176 (0.0010) [2023-12-26 16:36:50,428][105620] Updated weights for policy 1, policy_version 172186 (0.0011) [2023-12-26 16:36:50,490][105620] Updated weights for policy 1, policy_version 172196 (0.0011) [2023-12-26 16:36:50,861][105692] Updated weights for policy 0, policy_version 171170 (0.0008) [2023-12-26 16:36:50,925][105692] Updated weights for policy 0, policy_version 171180 (0.0008) [2023-12-26 16:36:50,990][105692] Updated weights for policy 0, policy_version 171190 (0.0008) [2023-12-26 16:36:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.9, 300 sec: 19688.6). Total num frames: 87924736. Throughput: 0: 9815.1, 1: 9959.8. Samples: 87913316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:36:51,062][104569] Avg episode reward: [(0, '9174.662'), (1, '8481.938')] [2023-12-26 16:36:51,201][105620] Updated weights for policy 1, policy_version 172206 (0.0008) [2023-12-26 16:36:51,264][105620] Updated weights for policy 1, policy_version 172216 (0.0011) [2023-12-26 16:36:51,316][105620] Updated weights for policy 1, policy_version 172226 (0.0010) [2023-12-26 16:36:51,359][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000009 [2023-12-26 16:36:51,770][105692] Updated weights for policy 0, policy_version 171200 (0.0008) [2023-12-26 16:36:51,830][105692] Updated weights for policy 0, policy_version 171210 (0.0008) [2023-12-26 16:36:51,893][105692] Updated weights for policy 0, policy_version 171220 (0.0008) [2023-12-26 16:36:52,083][105620] Updated weights for policy 1, policy_version 172236 (0.0011) [2023-12-26 16:36:52,152][105620] Updated weights for policy 1, policy_version 172246 (0.0010) [2023-12-26 16:36:52,207][105620] Updated weights for policy 1, policy_version 172256 (0.0010) [2023-12-26 16:36:52,666][105692] Updated weights for policy 0, policy_version 171230 (0.0008) [2023-12-26 16:36:52,722][105692] Updated weights for policy 0, policy_version 171240 (0.0008) [2023-12-26 16:36:52,781][105692] Updated weights for policy 0, policy_version 171250 (0.0008) [2023-12-26 16:36:52,950][105620] Updated weights for policy 1, policy_version 172266 (0.0010) [2023-12-26 16:36:53,000][105620] Updated weights for policy 1, policy_version 172276 (0.0010) [2023-12-26 16:36:53,048][105620] Updated weights for policy 1, policy_version 172286 (0.0010) [2023-12-26 16:36:53,108][105620] Updated weights for policy 1, policy_version 172296 (0.0010) [2023-12-26 16:36:53,454][105692] Updated weights for policy 0, policy_version 171260 (0.0007) [2023-12-26 16:36:53,505][105692] Updated weights for policy 0, policy_version 171270 (0.0005) [2023-12-26 16:36:53,561][105692] Updated weights for policy 0, policy_version 171280 (0.0010) [2023-12-26 16:36:53,853][105620] Updated weights for policy 1, policy_version 172306 (0.0010) [2023-12-26 16:36:53,910][105620] Updated weights for policy 1, policy_version 172316 (0.0009) [2023-12-26 16:36:53,962][105620] Updated weights for policy 1, policy_version 172326 (0.0005) [2023-12-26 16:36:54,291][105692] Updated weights for policy 0, policy_version 171290 (0.0010) [2023-12-26 16:36:54,342][105692] Updated weights for policy 0, policy_version 171300 (0.0007) [2023-12-26 16:36:54,394][105692] Updated weights for policy 0, policy_version 171310 (0.0007) [2023-12-26 16:36:54,448][105692] Updated weights for policy 0, policy_version 171320 (0.0010) [2023-12-26 16:36:54,600][105620] Updated weights for policy 1, policy_version 172336 (0.0009) [2023-12-26 16:36:54,665][105620] Updated weights for policy 1, policy_version 172346 (0.0010) [2023-12-26 16:36:54,722][105620] Updated weights for policy 1, policy_version 172356 (0.0010) [2023-12-26 16:36:55,136][105692] Updated weights for policy 0, policy_version 171330 (0.0007) [2023-12-26 16:36:55,183][105692] Updated weights for policy 0, policy_version 171340 (0.0008) [2023-12-26 16:36:55,229][105692] Updated weights for policy 0, policy_version 171350 (0.0008) [2023-12-26 16:36:55,460][105620] Updated weights for policy 1, policy_version 172366 (0.0010) [2023-12-26 16:36:55,518][105620] Updated weights for policy 1, policy_version 172376 (0.0010) [2023-12-26 16:36:55,570][105620] Updated weights for policy 1, policy_version 172386 (0.0010) [2023-12-26 16:36:55,869][105692] Updated weights for policy 0, policy_version 171360 (0.0006) [2023-12-26 16:36:55,934][105692] Updated weights for policy 0, policy_version 171370 (0.0008) [2023-12-26 16:36:55,989][105692] Updated weights for policy 0, policy_version 171380 (0.0008) [2023-12-26 16:36:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19797.3, 300 sec: 19716.3). Total num frames: 88023040. Throughput: 0: 9821.6, 1: 9939.7. Samples: 88028528. Policy #0 lag: (min: 4.0, avg: 8.6, max: 36.0) [2023-12-26 16:36:56,062][104569] Avg episode reward: [(0, '9002.918'), (1, '8730.970')] [2023-12-26 16:36:56,322][105620] Updated weights for policy 1, policy_version 172396 (0.0010) [2023-12-26 16:36:56,375][105620] Updated weights for policy 1, policy_version 172406 (0.0006) [2023-12-26 16:36:56,427][105620] Updated weights for policy 1, policy_version 172417 (0.0009) [2023-12-26 16:36:56,542][105692] Updated weights for policy 0, policy_version 171390 (0.0008) [2023-12-26 16:36:56,599][105692] Updated weights for policy 0, policy_version 171400 (0.0009) [2023-12-26 16:36:56,655][105692] Updated weights for policy 0, policy_version 171410 (0.0008) [2023-12-26 16:36:57,027][105620] Updated weights for policy 1, policy_version 172429 (0.0008) [2023-12-26 16:36:57,076][105620] Updated weights for policy 1, policy_version 172439 (0.0005) [2023-12-26 16:36:57,128][105620] Updated weights for policy 1, policy_version 172449 (0.0007) [2023-12-26 16:36:57,266][105692] Updated weights for policy 0, policy_version 171420 (0.0008) [2023-12-26 16:36:57,326][105692] Updated weights for policy 0, policy_version 171430 (0.0008) [2023-12-26 16:36:57,387][105692] Updated weights for policy 0, policy_version 171440 (0.0010) [2023-12-26 16:36:57,696][105620] Updated weights for policy 1, policy_version 172459 (0.0008) [2023-12-26 16:36:57,743][105620] Updated weights for policy 1, policy_version 172469 (0.0005) [2023-12-26 16:36:57,790][105620] Updated weights for policy 1, policy_version 172479 (0.0005) [2023-12-26 16:36:57,954][105692] Updated weights for policy 0, policy_version 171450 (0.0010) [2023-12-26 16:36:58,002][105692] Updated weights for policy 0, policy_version 171460 (0.0010) [2023-12-26 16:36:58,066][105692] Updated weights for policy 0, policy_version 171470 (0.0010) [2023-12-26 16:36:58,130][105692] Updated weights for policy 0, policy_version 171480 (0.0011) [2023-12-26 16:36:58,412][105620] Updated weights for policy 1, policy_version 172489 (0.0006) [2023-12-26 16:36:58,482][105620] Updated weights for policy 1, policy_version 172499 (0.0011) [2023-12-26 16:36:58,565][105620] Updated weights for policy 1, policy_version 172509 (0.0010) [2023-12-26 16:36:58,628][105620] Updated weights for policy 1, policy_version 172519 (0.0009) [2023-12-26 16:36:58,987][105692] Updated weights for policy 0, policy_version 171490 (0.0008) [2023-12-26 16:36:59,051][105692] Updated weights for policy 0, policy_version 171500 (0.0008) [2023-12-26 16:36:59,112][105692] Updated weights for policy 0, policy_version 171510 (0.0008) [2023-12-26 16:36:59,477][105620] Updated weights for policy 1, policy_version 172530 (0.0010) [2023-12-26 16:36:59,530][105620] Updated weights for policy 1, policy_version 172541 (0.0010) [2023-12-26 16:36:59,575][105620] Updated weights for policy 1, policy_version 172551 (0.0005) [2023-12-26 16:36:59,763][105692] Updated weights for policy 0, policy_version 171520 (0.0006) [2023-12-26 16:36:59,815][105692] Updated weights for policy 0, policy_version 171530 (0.0006) [2023-12-26 16:36:59,876][105692] Updated weights for policy 0, policy_version 171540 (0.0006) [2023-12-26 16:37:00,301][105620] Updated weights for policy 1, policy_version 172561 (0.0008) [2023-12-26 16:37:00,350][105620] Updated weights for policy 1, policy_version 172571 (0.0005) [2023-12-26 16:37:00,405][105620] Updated weights for policy 1, policy_version 172581 (0.0005) [2023-12-26 16:37:00,569][105692] Updated weights for policy 0, policy_version 171550 (0.0008) [2023-12-26 16:37:00,619][105692] Updated weights for policy 0, policy_version 171560 (0.0006) [2023-12-26 16:37:00,676][105692] Updated weights for policy 0, policy_version 171570 (0.0007) [2023-12-26 16:37:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 88121344. Throughput: 0: 9967.4, 1: 10032.4. Samples: 88093476. Policy #0 lag: (min: 4.0, avg: 8.6, max: 36.0) [2023-12-26 16:37:01,063][104569] Avg episode reward: [(0, '8495.907'), (1, '8906.902')] [2023-12-26 16:37:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000171576_43933696.pth... [2023-12-26 16:37:01,083][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000170424_43638784.pth [2023-12-26 16:37:01,118][105620] Updated weights for policy 1, policy_version 172591 (0.0008) [2023-12-26 16:37:01,187][105620] Updated weights for policy 1, policy_version 172601 (0.0008) [2023-12-26 16:37:01,250][105620] Updated weights for policy 1, policy_version 172611 (0.0009) [2023-12-26 16:37:01,281][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000172616_44195840.pth... [2023-12-26 16:37:01,285][105692] Updated weights for policy 0, policy_version 171580 (0.0008) [2023-12-26 16:37:01,286][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000171432_43892736.pth [2023-12-26 16:37:01,340][105692] Updated weights for policy 0, policy_version 171590 (0.0009) [2023-12-26 16:37:01,403][105692] Updated weights for policy 0, policy_version 171600 (0.0009) [2023-12-26 16:37:01,985][105620] Updated weights for policy 1, policy_version 172621 (0.0010) [2023-12-26 16:37:02,036][105620] Updated weights for policy 1, policy_version 172631 (0.0009) [2023-12-26 16:37:02,089][105620] Updated weights for policy 1, policy_version 172641 (0.0009) [2023-12-26 16:37:02,142][105692] Updated weights for policy 0, policy_version 171610 (0.0009) [2023-12-26 16:37:02,189][105692] Updated weights for policy 0, policy_version 171620 (0.0009) [2023-12-26 16:37:02,240][105692] Updated weights for policy 0, policy_version 171630 (0.0009) [2023-12-26 16:37:02,292][105692] Updated weights for policy 0, policy_version 171640 (0.0009) [2023-12-26 16:37:02,837][105620] Updated weights for policy 1, policy_version 172651 (0.0009) [2023-12-26 16:37:02,908][105620] Updated weights for policy 1, policy_version 172661 (0.0010) [2023-12-26 16:37:02,966][105620] Updated weights for policy 1, policy_version 172671 (0.0009) [2023-12-26 16:37:02,975][105692] Updated weights for policy 0, policy_version 171650 (0.0006) [2023-12-26 16:37:03,032][105692] Updated weights for policy 0, policy_version 171660 (0.0007) [2023-12-26 16:37:03,080][105692] Updated weights for policy 0, policy_version 171670 (0.0008) [2023-12-26 16:37:03,567][105620] Updated weights for policy 1, policy_version 172681 (0.0009) [2023-12-26 16:37:03,624][105620] Updated weights for policy 1, policy_version 172691 (0.0005) [2023-12-26 16:37:03,688][105620] Updated weights for policy 1, policy_version 172701 (0.0007) [2023-12-26 16:37:03,694][105692] Updated weights for policy 0, policy_version 171680 (0.0008) [2023-12-26 16:37:03,752][105692] Updated weights for policy 0, policy_version 171690 (0.0009) [2023-12-26 16:37:03,754][105620] Updated weights for policy 1, policy_version 172711 (0.0006) [2023-12-26 16:37:03,812][105692] Updated weights for policy 0, policy_version 171700 (0.0008) [2023-12-26 16:37:04,401][105620] Updated weights for policy 1, policy_version 172721 (0.0010) [2023-12-26 16:37:04,462][105620] Updated weights for policy 1, policy_version 172731 (0.0006) [2023-12-26 16:37:04,528][105620] Updated weights for policy 1, policy_version 172741 (0.0006) [2023-12-26 16:37:04,613][105692] Updated weights for policy 0, policy_version 171710 (0.0009) [2023-12-26 16:37:04,672][105692] Updated weights for policy 0, policy_version 171720 (0.0010) [2023-12-26 16:37:04,729][105692] Updated weights for policy 0, policy_version 171730 (0.0007) [2023-12-26 16:37:05,136][105620] Updated weights for policy 1, policy_version 172751 (0.0006) [2023-12-26 16:37:05,194][105620] Updated weights for policy 1, policy_version 172761 (0.0005) [2023-12-26 16:37:05,260][105620] Updated weights for policy 1, policy_version 172771 (0.0008) [2023-12-26 16:37:05,423][105692] Updated weights for policy 0, policy_version 171740 (0.0008) [2023-12-26 16:37:05,471][105692] Updated weights for policy 0, policy_version 171750 (0.0005) [2023-12-26 16:37:05,521][105692] Updated weights for policy 0, policy_version 171760 (0.0005) [2023-12-26 16:37:06,035][105692] Updated weights for policy 0, policy_version 171770 (0.0006) [2023-12-26 16:37:06,049][105620] Updated weights for policy 1, policy_version 172781 (0.0008) [2023-12-26 16:37:06,062][104569] Fps is (10 sec: 19660.0, 60 sec: 19797.2, 300 sec: 19716.3). Total num frames: 88219648. Throughput: 0: 10000.9, 1: 9921.4. Samples: 88213316. Policy #0 lag: (min: 4.0, avg: 8.6, max: 36.0) [2023-12-26 16:37:06,064][104569] Avg episode reward: [(0, '8669.360'), (1, '8995.414')] [2023-12-26 16:37:06,093][105692] Updated weights for policy 0, policy_version 171780 (0.0010) [2023-12-26 16:37:06,100][105620] Updated weights for policy 1, policy_version 172791 (0.0007) [2023-12-26 16:37:06,157][105620] Updated weights for policy 1, policy_version 172801 (0.0007) [2023-12-26 16:37:06,159][105692] Updated weights for policy 0, policy_version 171790 (0.0008) [2023-12-26 16:37:06,219][105692] Updated weights for policy 0, policy_version 171800 (0.0010) [2023-12-26 16:37:06,916][105620] Updated weights for policy 1, policy_version 172811 (0.0006) [2023-12-26 16:37:06,922][105692] Updated weights for policy 0, policy_version 171810 (0.0006) [2023-12-26 16:37:06,973][105620] Updated weights for policy 1, policy_version 172821 (0.0007) [2023-12-26 16:37:06,975][105692] Updated weights for policy 0, policy_version 171820 (0.0006) [2023-12-26 16:37:07,026][105620] Updated weights for policy 1, policy_version 172831 (0.0008) [2023-12-26 16:37:07,032][105692] Updated weights for policy 0, policy_version 171830 (0.0006) [2023-12-26 16:37:07,647][105692] Updated weights for policy 0, policy_version 171840 (0.0010) [2023-12-26 16:37:07,713][105692] Updated weights for policy 0, policy_version 171850 (0.0010) [2023-12-26 16:37:07,772][105692] Updated weights for policy 0, policy_version 171860 (0.0008) [2023-12-26 16:37:07,865][105620] Updated weights for policy 1, policy_version 172841 (0.0008) [2023-12-26 16:37:07,918][105620] Updated weights for policy 1, policy_version 172851 (0.0009) [2023-12-26 16:37:07,979][105620] Updated weights for policy 1, policy_version 172861 (0.0008) [2023-12-26 16:37:08,045][105620] Updated weights for policy 1, policy_version 172871 (0.0009) [2023-12-26 16:37:08,393][105692] Updated weights for policy 0, policy_version 171870 (0.0005) [2023-12-26 16:37:08,461][105692] Updated weights for policy 0, policy_version 171880 (0.0008) [2023-12-26 16:37:08,523][105692] Updated weights for policy 0, policy_version 171890 (0.0009) [2023-12-26 16:37:08,832][105620] Updated weights for policy 1, policy_version 172881 (0.0009) [2023-12-26 16:37:08,879][105620] Updated weights for policy 1, policy_version 172891 (0.0009) [2023-12-26 16:37:08,938][105620] Updated weights for policy 1, policy_version 172901 (0.0010) [2023-12-26 16:37:09,194][105692] Updated weights for policy 0, policy_version 171900 (0.0009) [2023-12-26 16:37:09,267][105692] Updated weights for policy 0, policy_version 171910 (0.0009) [2023-12-26 16:37:09,328][105692] Updated weights for policy 0, policy_version 171920 (0.0010) [2023-12-26 16:37:09,731][105620] Updated weights for policy 1, policy_version 172911 (0.0008) [2023-12-26 16:37:09,780][105620] Updated weights for policy 1, policy_version 172921 (0.0008) [2023-12-26 16:37:09,841][105620] Updated weights for policy 1, policy_version 172931 (0.0008) [2023-12-26 16:37:10,086][105692] Updated weights for policy 0, policy_version 171930 (0.0009) [2023-12-26 16:37:10,142][105692] Updated weights for policy 0, policy_version 171940 (0.0009) [2023-12-26 16:37:10,193][105692] Updated weights for policy 0, policy_version 171950 (0.0008) [2023-12-26 16:37:10,248][105692] Updated weights for policy 0, policy_version 171960 (0.0009) [2023-12-26 16:37:10,617][105620] Updated weights for policy 1, policy_version 172941 (0.0009) [2023-12-26 16:37:10,674][105620] Updated weights for policy 1, policy_version 172951 (0.0010) [2023-12-26 16:37:10,728][105620] Updated weights for policy 1, policy_version 172961 (0.0009) [2023-12-26 16:37:10,983][105692] Updated weights for policy 0, policy_version 171970 (0.0005) [2023-12-26 16:37:11,053][105692] Updated weights for policy 0, policy_version 171980 (0.0007) [2023-12-26 16:37:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19716.4). Total num frames: 88317952. Throughput: 0: 10077.8, 1: 9841.6. Samples: 88328532. Policy #0 lag: (min: 4.0, avg: 8.6, max: 36.0) [2023-12-26 16:37:11,062][104569] Avg episode reward: [(0, '9263.914'), (1, '9169.004')] [2023-12-26 16:37:11,113][105692] Updated weights for policy 0, policy_version 171990 (0.0006) [2023-12-26 16:37:11,586][105620] Updated weights for policy 1, policy_version 172972 (0.0010) [2023-12-26 16:37:11,652][105620] Updated weights for policy 1, policy_version 172982 (0.0009) [2023-12-26 16:37:11,714][105620] Updated weights for policy 1, policy_version 172992 (0.0009) [2023-12-26 16:37:11,810][105692] Updated weights for policy 0, policy_version 172000 (0.0007) [2023-12-26 16:37:11,878][105692] Updated weights for policy 0, policy_version 172010 (0.0006) [2023-12-26 16:37:11,930][105692] Updated weights for policy 0, policy_version 172020 (0.0009) [2023-12-26 16:37:12,489][105620] Updated weights for policy 1, policy_version 173002 (0.0008) [2023-12-26 16:37:12,545][105620] Updated weights for policy 1, policy_version 173012 (0.0009) [2023-12-26 16:37:12,570][105692] Updated weights for policy 0, policy_version 172030 (0.0007) [2023-12-26 16:37:12,610][105620] Updated weights for policy 1, policy_version 173022 (0.0007) [2023-12-26 16:37:12,624][105692] Updated weights for policy 0, policy_version 172040 (0.0008) [2023-12-26 16:37:12,659][105620] Updated weights for policy 1, policy_version 173032 (0.0007) [2023-12-26 16:37:12,680][105692] Updated weights for policy 0, policy_version 172050 (0.0008) [2023-12-26 16:37:13,333][105692] Updated weights for policy 0, policy_version 172060 (0.0009) [2023-12-26 16:37:13,385][105692] Updated weights for policy 0, policy_version 172070 (0.0010) [2023-12-26 16:37:13,391][105620] Updated weights for policy 1, policy_version 173042 (0.0006) [2023-12-26 16:37:13,440][105692] Updated weights for policy 0, policy_version 172080 (0.0010) [2023-12-26 16:37:13,450][105620] Updated weights for policy 1, policy_version 173052 (0.0005) [2023-12-26 16:37:13,499][105620] Updated weights for policy 1, policy_version 173062 (0.0005) [2023-12-26 16:37:14,144][105620] Updated weights for policy 1, policy_version 173072 (0.0005) [2023-12-26 16:37:14,171][105692] Updated weights for policy 0, policy_version 172090 (0.0011) [2023-12-26 16:37:14,202][105620] Updated weights for policy 1, policy_version 173082 (0.0006) [2023-12-26 16:37:14,228][105692] Updated weights for policy 0, policy_version 172100 (0.0011) [2023-12-26 16:37:14,255][105620] Updated weights for policy 1, policy_version 173092 (0.0007) [2023-12-26 16:37:14,284][105692] Updated weights for policy 0, policy_version 172110 (0.0010) [2023-12-26 16:37:14,346][105692] Updated weights for policy 0, policy_version 172120 (0.0006) [2023-12-26 16:37:14,850][105620] Updated weights for policy 1, policy_version 173102 (0.0008) [2023-12-26 16:37:14,910][105620] Updated weights for policy 1, policy_version 173112 (0.0009) [2023-12-26 16:37:14,974][105620] Updated weights for policy 1, policy_version 173122 (0.0009) [2023-12-26 16:37:15,115][105692] Updated weights for policy 0, policy_version 172130 (0.0007) [2023-12-26 16:37:15,176][105692] Updated weights for policy 0, policy_version 172140 (0.0009) [2023-12-26 16:37:15,233][105692] Updated weights for policy 0, policy_version 172150 (0.0009) [2023-12-26 16:37:15,664][105620] Updated weights for policy 1, policy_version 173132 (0.0008) [2023-12-26 16:37:15,728][105620] Updated weights for policy 1, policy_version 173142 (0.0008) [2023-12-26 16:37:15,795][105620] Updated weights for policy 1, policy_version 173152 (0.0009) [2023-12-26 16:37:15,953][105692] Updated weights for policy 0, policy_version 172160 (0.0010) [2023-12-26 16:37:16,001][105692] Updated weights for policy 0, policy_version 172170 (0.0010) [2023-12-26 16:37:16,047][105692] Updated weights for policy 0, policy_version 172180 (0.0007) [2023-12-26 16:37:16,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19933.9, 300 sec: 19716.3). Total num frames: 88416256. Throughput: 0: 10014.9, 1: 9782.9. Samples: 88387172. Policy #0 lag: (min: 4.0, avg: 8.6, max: 36.0) [2023-12-26 16:37:16,063][104569] Avg episode reward: [(0, '9170.251'), (1, '9170.035')] [2023-12-26 16:37:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000172184_44089344.pth... [2023-12-26 16:37:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000173160_44335104.pth... [2023-12-26 16:37:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000171000_43786240.pth [2023-12-26 16:37:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000172008_44040192.pth [2023-12-26 16:37:16,622][105620] Updated weights for policy 1, policy_version 173162 (0.0008) [2023-12-26 16:37:16,640][105692] Updated weights for policy 0, policy_version 172190 (0.0006) [2023-12-26 16:37:16,674][105620] Updated weights for policy 1, policy_version 173172 (0.0008) [2023-12-26 16:37:16,697][105692] Updated weights for policy 0, policy_version 172200 (0.0005) [2023-12-26 16:37:16,730][105620] Updated weights for policy 1, policy_version 173182 (0.0008) [2023-12-26 16:37:16,749][105692] Updated weights for policy 0, policy_version 172210 (0.0005) [2023-12-26 16:37:16,780][105620] Updated weights for policy 1, policy_version 173192 (0.0006) [2023-12-26 16:37:17,383][105620] Updated weights for policy 1, policy_version 173202 (0.0007) [2023-12-26 16:37:17,416][105692] Updated weights for policy 0, policy_version 172220 (0.0007) [2023-12-26 16:37:17,461][105620] Updated weights for policy 1, policy_version 173212 (0.0010) [2023-12-26 16:37:17,468][105692] Updated weights for policy 0, policy_version 172230 (0.0006) [2023-12-26 16:37:17,508][105620] Updated weights for policy 1, policy_version 173222 (0.0009) [2023-12-26 16:37:17,513][105692] Updated weights for policy 0, policy_version 172240 (0.0009) [2023-12-26 16:37:18,163][105692] Updated weights for policy 0, policy_version 172250 (0.0010) [2023-12-26 16:37:18,172][105620] Updated weights for policy 1, policy_version 173232 (0.0009) [2023-12-26 16:37:18,211][105692] Updated weights for policy 0, policy_version 172260 (0.0010) [2023-12-26 16:37:18,235][105620] Updated weights for policy 1, policy_version 173242 (0.0009) [2023-12-26 16:37:18,260][105692] Updated weights for policy 0, policy_version 172270 (0.0010) [2023-12-26 16:37:18,290][105620] Updated weights for policy 1, policy_version 173252 (0.0011) [2023-12-26 16:37:18,305][105692] Updated weights for policy 0, policy_version 172280 (0.0010) [2023-12-26 16:37:19,050][105620] Updated weights for policy 1, policy_version 173262 (0.0009) [2023-12-26 16:37:19,074][105692] Updated weights for policy 0, policy_version 172290 (0.0011) [2023-12-26 16:37:19,116][105620] Updated weights for policy 1, policy_version 173272 (0.0006) [2023-12-26 16:37:19,137][105692] Updated weights for policy 0, policy_version 172300 (0.0011) [2023-12-26 16:37:19,171][105620] Updated weights for policy 1, policy_version 173282 (0.0005) [2023-12-26 16:37:19,189][105692] Updated weights for policy 0, policy_version 172310 (0.0011) [2023-12-26 16:37:19,939][105620] Updated weights for policy 1, policy_version 173292 (0.0008) [2023-12-26 16:37:19,944][105692] Updated weights for policy 0, policy_version 172320 (0.0008) [2023-12-26 16:37:20,003][105620] Updated weights for policy 1, policy_version 173302 (0.0011) [2023-12-26 16:37:20,008][105692] Updated weights for policy 0, policy_version 172330 (0.0006) [2023-12-26 16:37:20,064][105692] Updated weights for policy 0, policy_version 172340 (0.0009) [2023-12-26 16:37:20,068][105620] Updated weights for policy 1, policy_version 173312 (0.0011) [2023-12-26 16:37:20,691][105692] Updated weights for policy 0, policy_version 172350 (0.0010) [2023-12-26 16:37:20,745][105692] Updated weights for policy 0, policy_version 172360 (0.0010) [2023-12-26 16:37:20,797][105620] Updated weights for policy 1, policy_version 173322 (0.0010) [2023-12-26 16:37:20,798][105692] Updated weights for policy 0, policy_version 172370 (0.0011) [2023-12-26 16:37:20,851][105620] Updated weights for policy 1, policy_version 173332 (0.0011) [2023-12-26 16:37:20,904][105620] Updated weights for policy 1, policy_version 173342 (0.0011) [2023-12-26 16:37:20,970][105620] Updated weights for policy 1, policy_version 173352 (0.0009) [2023-12-26 16:37:21,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19933.9, 300 sec: 19744.1). Total num frames: 88522752. Throughput: 0: 9956.7, 1: 9784.1. Samples: 88506684. Policy #0 lag: (min: 4.0, avg: 8.6, max: 36.0) [2023-12-26 16:37:21,063][104569] Avg episode reward: [(0, '9080.363'), (1, '9080.957')] [2023-12-26 16:37:21,581][105692] Updated weights for policy 0, policy_version 172380 (0.0009) [2023-12-26 16:37:21,646][105692] Updated weights for policy 0, policy_version 172390 (0.0008) [2023-12-26 16:37:21,718][105692] Updated weights for policy 0, policy_version 172400 (0.0009) [2023-12-26 16:37:21,740][105620] Updated weights for policy 1, policy_version 173362 (0.0009) [2023-12-26 16:37:21,798][105620] Updated weights for policy 1, policy_version 173372 (0.0010) [2023-12-26 16:37:21,852][105620] Updated weights for policy 1, policy_version 173382 (0.0011) [2023-12-26 16:37:22,489][105692] Updated weights for policy 0, policy_version 172410 (0.0008) [2023-12-26 16:37:22,553][105692] Updated weights for policy 0, policy_version 172420 (0.0008) [2023-12-26 16:37:22,607][105692] Updated weights for policy 0, policy_version 172430 (0.0006) [2023-12-26 16:37:22,612][105620] Updated weights for policy 1, policy_version 173392 (0.0011) [2023-12-26 16:37:22,667][105692] Updated weights for policy 0, policy_version 172440 (0.0008) [2023-12-26 16:37:22,668][105620] Updated weights for policy 1, policy_version 173402 (0.0008) [2023-12-26 16:37:22,735][105620] Updated weights for policy 1, policy_version 173412 (0.0009) [2023-12-26 16:37:23,346][105620] Updated weights for policy 1, policy_version 173422 (0.0005) [2023-12-26 16:37:23,409][105620] Updated weights for policy 1, policy_version 173432 (0.0005) [2023-12-26 16:37:23,461][105620] Updated weights for policy 1, policy_version 173442 (0.0005) [2023-12-26 16:37:23,505][105692] Updated weights for policy 0, policy_version 172450 (0.0008) [2023-12-26 16:37:23,569][105692] Updated weights for policy 0, policy_version 172460 (0.0009) [2023-12-26 16:37:23,638][105692] Updated weights for policy 0, policy_version 172470 (0.0008) [2023-12-26 16:37:24,195][105692] Updated weights for policy 0, policy_version 172480 (0.0005) [2023-12-26 16:37:24,205][105620] Updated weights for policy 1, policy_version 173452 (0.0008) [2023-12-26 16:37:24,254][105692] Updated weights for policy 0, policy_version 172490 (0.0005) [2023-12-26 16:37:24,262][105620] Updated weights for policy 1, policy_version 173462 (0.0009) [2023-12-26 16:37:24,313][105620] Updated weights for policy 1, policy_version 173472 (0.0006) [2023-12-26 16:37:24,319][105692] Updated weights for policy 0, policy_version 172500 (0.0007) [2023-12-26 16:37:24,836][105692] Updated weights for policy 0, policy_version 172510 (0.0005) [2023-12-26 16:37:24,887][105692] Updated weights for policy 0, policy_version 172520 (0.0008) [2023-12-26 16:37:24,888][105620] Updated weights for policy 1, policy_version 173482 (0.0006) [2023-12-26 16:37:24,938][105620] Updated weights for policy 1, policy_version 173492 (0.0006) [2023-12-26 16:37:24,940][105692] Updated weights for policy 0, policy_version 172530 (0.0007) [2023-12-26 16:37:24,996][105620] Updated weights for policy 1, policy_version 173502 (0.0005) [2023-12-26 16:37:25,047][105620] Updated weights for policy 1, policy_version 173512 (0.0005) [2023-12-26 16:37:25,504][105692] Updated weights for policy 0, policy_version 172540 (0.0009) [2023-12-26 16:37:25,563][105692] Updated weights for policy 0, policy_version 172550 (0.0008) [2023-12-26 16:37:25,632][105692] Updated weights for policy 0, policy_version 172560 (0.0006) [2023-12-26 16:37:25,735][105620] Updated weights for policy 1, policy_version 173522 (0.0010) [2023-12-26 16:37:25,793][105620] Updated weights for policy 1, policy_version 173532 (0.0010) [2023-12-26 16:37:25,841][105620] Updated weights for policy 1, policy_version 173542 (0.0010) [2023-12-26 16:37:26,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19933.8, 300 sec: 19716.3). Total num frames: 88621056. Throughput: 0: 10028.1, 1: 9807.8. Samples: 88627720. Policy #0 lag: (min: 4.0, avg: 8.6, max: 36.0) [2023-12-26 16:37:26,063][104569] Avg episode reward: [(0, '9079.977'), (1, '8909.458')] [2023-12-26 16:37:26,354][105692] Updated weights for policy 0, policy_version 172570 (0.0007) [2023-12-26 16:37:26,408][105692] Updated weights for policy 0, policy_version 172580 (0.0010) [2023-12-26 16:37:26,463][105692] Updated weights for policy 0, policy_version 172590 (0.0008) [2023-12-26 16:37:26,492][105620] Updated weights for policy 1, policy_version 173552 (0.0006) [2023-12-26 16:37:26,523][105692] Updated weights for policy 0, policy_version 172600 (0.0008) [2023-12-26 16:37:26,538][105620] Updated weights for policy 1, policy_version 173562 (0.0005) [2023-12-26 16:37:26,582][105620] Updated weights for policy 1, policy_version 173572 (0.0010) [2023-12-26 16:37:27,184][105692] Updated weights for policy 0, policy_version 172610 (0.0010) [2023-12-26 16:37:27,247][105692] Updated weights for policy 0, policy_version 172620 (0.0010) [2023-12-26 16:37:27,257][105620] Updated weights for policy 1, policy_version 173582 (0.0007) [2023-12-26 16:37:27,298][105692] Updated weights for policy 0, policy_version 172630 (0.0009) [2023-12-26 16:37:27,319][105620] Updated weights for policy 1, policy_version 173592 (0.0009) [2023-12-26 16:37:27,366][105620] Updated weights for policy 1, policy_version 173602 (0.0010) [2023-12-26 16:37:27,959][105620] Updated weights for policy 1, policy_version 173612 (0.0010) [2023-12-26 16:37:28,013][105620] Updated weights for policy 1, policy_version 173622 (0.0010) [2023-12-26 16:37:28,059][105620] Updated weights for policy 1, policy_version 173632 (0.0008) [2023-12-26 16:37:28,066][105692] Updated weights for policy 0, policy_version 172640 (0.0007) [2023-12-26 16:37:28,120][105692] Updated weights for policy 0, policy_version 172650 (0.0009) [2023-12-26 16:37:28,168][105692] Updated weights for policy 0, policy_version 172660 (0.0009) [2023-12-26 16:37:28,755][105620] Updated weights for policy 1, policy_version 173642 (0.0006) [2023-12-26 16:37:28,811][105620] Updated weights for policy 1, policy_version 173652 (0.0008) [2023-12-26 16:37:28,875][105620] Updated weights for policy 1, policy_version 173662 (0.0008) [2023-12-26 16:37:28,938][105620] Updated weights for policy 1, policy_version 173672 (0.0008) [2023-12-26 16:37:28,959][105692] Updated weights for policy 0, policy_version 172670 (0.0009) [2023-12-26 16:37:29,021][105692] Updated weights for policy 0, policy_version 172680 (0.0009) [2023-12-26 16:37:29,084][105692] Updated weights for policy 0, policy_version 172690 (0.0009) [2023-12-26 16:37:29,621][105620] Updated weights for policy 1, policy_version 173682 (0.0006) [2023-12-26 16:37:29,691][105620] Updated weights for policy 1, policy_version 173692 (0.0009) [2023-12-26 16:37:29,750][105620] Updated weights for policy 1, policy_version 173702 (0.0009) [2023-12-26 16:37:29,771][105692] Updated weights for policy 0, policy_version 172700 (0.0008) [2023-12-26 16:37:29,837][105692] Updated weights for policy 0, policy_version 172710 (0.0008) [2023-12-26 16:37:29,898][105692] Updated weights for policy 0, policy_version 172720 (0.0009) [2023-12-26 16:37:30,444][105620] Updated weights for policy 1, policy_version 173712 (0.0006) [2023-12-26 16:37:30,490][105620] Updated weights for policy 1, policy_version 173722 (0.0005) [2023-12-26 16:37:30,535][105620] Updated weights for policy 1, policy_version 173732 (0.0005) [2023-12-26 16:37:30,678][105692] Updated weights for policy 0, policy_version 172730 (0.0010) [2023-12-26 16:37:30,744][105692] Updated weights for policy 0, policy_version 172740 (0.0010) [2023-12-26 16:37:30,804][105692] Updated weights for policy 0, policy_version 172750 (0.0007) [2023-12-26 16:37:30,859][105692] Updated weights for policy 0, policy_version 172760 (0.0009) [2023-12-26 16:37:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 88719360. Throughput: 0: 10039.7, 1: 9849.3. Samples: 88688528. Policy #0 lag: (min: 4.0, avg: 8.6, max: 36.0) [2023-12-26 16:37:31,063][104569] Avg episode reward: [(0, '9347.742'), (1, '8832.128')] [2023-12-26 16:37:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000172760_44236800.pth... [2023-12-26 16:37:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000173736_44482560.pth... [2023-12-26 16:37:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000171576_43933696.pth [2023-12-26 16:37:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000172616_44195840.pth [2023-12-26 16:37:31,204][105620] Updated weights for policy 1, policy_version 173742 (0.0006) [2023-12-26 16:37:31,262][105620] Updated weights for policy 1, policy_version 173752 (0.0007) [2023-12-26 16:37:31,326][105620] Updated weights for policy 1, policy_version 173762 (0.0008) [2023-12-26 16:37:31,651][105692] Updated weights for policy 0, policy_version 172770 (0.0008) [2023-12-26 16:37:31,712][105692] Updated weights for policy 0, policy_version 172780 (0.0008) [2023-12-26 16:37:31,775][105692] Updated weights for policy 0, policy_version 172790 (0.0008) [2023-12-26 16:37:32,041][105620] Updated weights for policy 1, policy_version 173772 (0.0007) [2023-12-26 16:37:32,086][105620] Updated weights for policy 1, policy_version 173782 (0.0008) [2023-12-26 16:37:32,144][105620] Updated weights for policy 1, policy_version 173792 (0.0009) [2023-12-26 16:37:32,522][105692] Updated weights for policy 0, policy_version 172800 (0.0009) [2023-12-26 16:37:32,580][105692] Updated weights for policy 0, policy_version 172810 (0.0009) [2023-12-26 16:37:32,631][105692] Updated weights for policy 0, policy_version 172820 (0.0009) [2023-12-26 16:37:32,911][105620] Updated weights for policy 1, policy_version 173802 (0.0009) [2023-12-26 16:37:32,979][105620] Updated weights for policy 1, policy_version 173812 (0.0009) [2023-12-26 16:37:33,049][105620] Updated weights for policy 1, policy_version 173822 (0.0010) [2023-12-26 16:37:33,107][105620] Updated weights for policy 1, policy_version 173832 (0.0009) [2023-12-26 16:37:33,311][105692] Updated weights for policy 0, policy_version 172830 (0.0009) [2023-12-26 16:37:33,364][105692] Updated weights for policy 0, policy_version 172840 (0.0009) [2023-12-26 16:37:33,409][105692] Updated weights for policy 0, policy_version 172850 (0.0008) [2023-12-26 16:37:33,805][105620] Updated weights for policy 1, policy_version 173842 (0.0009) [2023-12-26 16:37:33,854][105620] Updated weights for policy 1, policy_version 173852 (0.0008) [2023-12-26 16:37:33,913][105620] Updated weights for policy 1, policy_version 173862 (0.0010) [2023-12-26 16:37:34,219][105692] Updated weights for policy 0, policy_version 172860 (0.0009) [2023-12-26 16:37:34,277][105692] Updated weights for policy 0, policy_version 172870 (0.0010) [2023-12-26 16:37:34,343][105692] Updated weights for policy 0, policy_version 172880 (0.0009) [2023-12-26 16:37:34,643][105620] Updated weights for policy 1, policy_version 173872 (0.0009) [2023-12-26 16:37:34,696][105620] Updated weights for policy 1, policy_version 173882 (0.0009) [2023-12-26 16:37:34,757][105620] Updated weights for policy 1, policy_version 173892 (0.0008) [2023-12-26 16:37:34,989][105692] Updated weights for policy 0, policy_version 172890 (0.0009) [2023-12-26 16:37:35,047][105692] Updated weights for policy 0, policy_version 172900 (0.0010) [2023-12-26 16:37:35,098][105692] Updated weights for policy 0, policy_version 172910 (0.0010) [2023-12-26 16:37:35,156][105692] Updated weights for policy 0, policy_version 172920 (0.0010) [2023-12-26 16:37:35,475][105620] Updated weights for policy 1, policy_version 173902 (0.0007) [2023-12-26 16:37:35,525][105620] Updated weights for policy 1, policy_version 173912 (0.0005) [2023-12-26 16:37:35,583][105620] Updated weights for policy 1, policy_version 173922 (0.0005) [2023-12-26 16:37:35,721][105692] Updated weights for policy 0, policy_version 172930 (0.0007) [2023-12-26 16:37:35,773][105692] Updated weights for policy 0, policy_version 172940 (0.0011) [2023-12-26 16:37:35,831][105692] Updated weights for policy 0, policy_version 172950 (0.0010) [2023-12-26 16:37:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 88817664. Throughput: 0: 9968.7, 1: 9795.5. Samples: 88802708. Policy #0 lag: (min: 4.0, avg: 8.6, max: 36.0) [2023-12-26 16:37:36,063][104569] Avg episode reward: [(0, '9348.152'), (1, '8805.738')] [2023-12-26 16:37:36,285][105620] Updated weights for policy 1, policy_version 173932 (0.0007) [2023-12-26 16:37:36,333][105620] Updated weights for policy 1, policy_version 173942 (0.0008) [2023-12-26 16:37:36,389][105620] Updated weights for policy 1, policy_version 173952 (0.0008) [2023-12-26 16:37:36,564][105692] Updated weights for policy 0, policy_version 172960 (0.0010) [2023-12-26 16:37:36,617][105692] Updated weights for policy 0, policy_version 172970 (0.0011) [2023-12-26 16:37:36,668][105692] Updated weights for policy 0, policy_version 172980 (0.0010) [2023-12-26 16:37:37,200][105620] Updated weights for policy 1, policy_version 173962 (0.0008) [2023-12-26 16:37:37,259][105620] Updated weights for policy 1, policy_version 173972 (0.0008) [2023-12-26 16:37:37,311][105620] Updated weights for policy 1, policy_version 173982 (0.0008) [2023-12-26 16:37:37,370][105620] Updated weights for policy 1, policy_version 173992 (0.0008) [2023-12-26 16:37:37,442][105692] Updated weights for policy 0, policy_version 172990 (0.0010) [2023-12-26 16:37:37,496][105692] Updated weights for policy 0, policy_version 173000 (0.0009) [2023-12-26 16:37:37,547][105692] Updated weights for policy 0, policy_version 173010 (0.0009) [2023-12-26 16:37:38,123][105620] Updated weights for policy 1, policy_version 174002 (0.0007) [2023-12-26 16:37:38,175][105620] Updated weights for policy 1, policy_version 174012 (0.0009) [2023-12-26 16:37:38,229][105620] Updated weights for policy 1, policy_version 174022 (0.0010) [2023-12-26 16:37:38,286][105692] Updated weights for policy 0, policy_version 173020 (0.0009) [2023-12-26 16:37:38,355][105692] Updated weights for policy 0, policy_version 173030 (0.0008) [2023-12-26 16:37:38,409][105692] Updated weights for policy 0, policy_version 173040 (0.0010) [2023-12-26 16:37:38,935][105620] Updated weights for policy 1, policy_version 174032 (0.0009) [2023-12-26 16:37:38,999][105620] Updated weights for policy 1, policy_version 174042 (0.0008) [2023-12-26 16:37:39,053][105620] Updated weights for policy 1, policy_version 174052 (0.0009) [2023-12-26 16:37:39,245][105692] Updated weights for policy 0, policy_version 173050 (0.0009) [2023-12-26 16:37:39,308][105692] Updated weights for policy 0, policy_version 173060 (0.0008) [2023-12-26 16:37:39,375][105692] Updated weights for policy 0, policy_version 173070 (0.0008) [2023-12-26 16:37:39,443][105692] Updated weights for policy 0, policy_version 173080 (0.0010) [2023-12-26 16:37:39,772][105620] Updated weights for policy 1, policy_version 174062 (0.0008) [2023-12-26 16:37:39,822][105620] Updated weights for policy 1, policy_version 174072 (0.0009) [2023-12-26 16:37:39,874][105620] Updated weights for policy 1, policy_version 174082 (0.0008) [2023-12-26 16:37:40,175][105692] Updated weights for policy 0, policy_version 173090 (0.0009) [2023-12-26 16:37:40,234][105692] Updated weights for policy 0, policy_version 173100 (0.0009) [2023-12-26 16:37:40,293][105692] Updated weights for policy 0, policy_version 173110 (0.0009) [2023-12-26 16:37:40,685][105620] Updated weights for policy 1, policy_version 174093 (0.0009) [2023-12-26 16:37:40,736][105620] Updated weights for policy 1, policy_version 174103 (0.0009) [2023-12-26 16:37:40,782][105620] Updated weights for policy 1, policy_version 174113 (0.0008) [2023-12-26 16:37:41,046][105692] Updated weights for policy 0, policy_version 173120 (0.0010) [2023-12-26 16:37:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19716.3). Total num frames: 88907776. Throughput: 0: 9993.7, 1: 9774.7. Samples: 88918108. Policy #0 lag: (min: 4.0, avg: 8.6, max: 36.0) [2023-12-26 16:37:41,063][104569] Avg episode reward: [(0, '9349.079'), (1, '9072.206')] [2023-12-26 16:37:41,103][105692] Updated weights for policy 0, policy_version 173130 (0.0008) [2023-12-26 16:37:41,163][105692] Updated weights for policy 0, policy_version 173140 (0.0008) [2023-12-26 16:37:41,599][105620] Updated weights for policy 1, policy_version 174123 (0.0009) [2023-12-26 16:37:41,669][105620] Updated weights for policy 1, policy_version 174133 (0.0008) [2023-12-26 16:37:41,735][105620] Updated weights for policy 1, policy_version 174143 (0.0008) [2023-12-26 16:37:41,919][105692] Updated weights for policy 0, policy_version 173150 (0.0008) [2023-12-26 16:37:41,971][105692] Updated weights for policy 0, policy_version 173160 (0.0008) [2023-12-26 16:37:42,028][105692] Updated weights for policy 0, policy_version 173170 (0.0008) [2023-12-26 16:37:42,493][105620] Updated weights for policy 1, policy_version 174153 (0.0008) [2023-12-26 16:37:42,551][105620] Updated weights for policy 1, policy_version 174163 (0.0009) [2023-12-26 16:37:42,605][105620] Updated weights for policy 1, policy_version 174173 (0.0009) [2023-12-26 16:37:42,658][105620] Updated weights for policy 1, policy_version 174183 (0.0009) [2023-12-26 16:37:42,801][105692] Updated weights for policy 0, policy_version 173180 (0.0008) [2023-12-26 16:37:42,862][105692] Updated weights for policy 0, policy_version 173190 (0.0009) [2023-12-26 16:37:42,924][105692] Updated weights for policy 0, policy_version 173200 (0.0006) [2023-12-26 16:37:43,448][105620] Updated weights for policy 1, policy_version 174193 (0.0009) [2023-12-26 16:37:43,495][105620] Updated weights for policy 1, policy_version 174203 (0.0009) [2023-12-26 16:37:43,544][105620] Updated weights for policy 1, policy_version 174213 (0.0008) [2023-12-26 16:37:43,626][105692] Updated weights for policy 0, policy_version 173210 (0.0007) [2023-12-26 16:37:43,679][105692] Updated weights for policy 0, policy_version 173220 (0.0008) [2023-12-26 16:37:43,730][105692] Updated weights for policy 0, policy_version 173230 (0.0009) [2023-12-26 16:37:43,780][105692] Updated weights for policy 0, policy_version 173240 (0.0009) [2023-12-26 16:37:44,255][105620] Updated weights for policy 1, policy_version 174223 (0.0007) [2023-12-26 16:37:44,319][105620] Updated weights for policy 1, policy_version 174233 (0.0006) [2023-12-26 16:37:44,375][105620] Updated weights for policy 1, policy_version 174243 (0.0005) [2023-12-26 16:37:44,614][105692] Updated weights for policy 0, policy_version 173250 (0.0009) [2023-12-26 16:37:44,675][105692] Updated weights for policy 0, policy_version 173260 (0.0009) [2023-12-26 16:37:44,726][105692] Updated weights for policy 0, policy_version 173270 (0.0009) [2023-12-26 16:37:45,015][105620] Updated weights for policy 1, policy_version 174253 (0.0007) [2023-12-26 16:37:45,070][105620] Updated weights for policy 1, policy_version 174263 (0.0009) [2023-12-26 16:37:45,132][105620] Updated weights for policy 1, policy_version 174273 (0.0010) [2023-12-26 16:37:45,486][105692] Updated weights for policy 0, policy_version 173280 (0.0009) [2023-12-26 16:37:45,552][105692] Updated weights for policy 0, policy_version 173290 (0.0009) [2023-12-26 16:37:45,611][105692] Updated weights for policy 0, policy_version 173300 (0.0009) [2023-12-26 16:37:45,888][105620] Updated weights for policy 1, policy_version 174283 (0.0009) [2023-12-26 16:37:45,943][105620] Updated weights for policy 1, policy_version 174293 (0.0009) [2023-12-26 16:37:46,002][105620] Updated weights for policy 1, policy_version 174303 (0.0009) [2023-12-26 16:37:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.9, 300 sec: 19716.3). Total num frames: 89006080. Throughput: 0: 9883.9, 1: 9676.4. Samples: 88973688. Policy #0 lag: (min: 4.0, avg: 8.6, max: 36.0) [2023-12-26 16:37:46,062][104569] Avg episode reward: [(0, '9170.015'), (1, '9166.381')] [2023-12-26 16:37:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000173304_44376064.pth... [2023-12-26 16:37:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000174312_44630016.pth... [2023-12-26 16:37:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000172184_44089344.pth [2023-12-26 16:37:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000173160_44335104.pth [2023-12-26 16:37:46,331][105692] Updated weights for policy 0, policy_version 173310 (0.0009) [2023-12-26 16:37:46,389][105692] Updated weights for policy 0, policy_version 173320 (0.0009) [2023-12-26 16:37:46,447][105692] Updated weights for policy 0, policy_version 173330 (0.0009) [2023-12-26 16:37:46,761][105620] Updated weights for policy 1, policy_version 174313 (0.0009) [2023-12-26 16:37:46,809][105620] Updated weights for policy 1, policy_version 174323 (0.0009) [2023-12-26 16:37:46,860][105620] Updated weights for policy 1, policy_version 174333 (0.0009) [2023-12-26 16:37:46,919][105620] Updated weights for policy 1, policy_version 174343 (0.0009) [2023-12-26 16:37:47,191][105692] Updated weights for policy 0, policy_version 173340 (0.0009) [2023-12-26 16:37:47,251][105692] Updated weights for policy 0, policy_version 173350 (0.0008) [2023-12-26 16:37:47,310][105692] Updated weights for policy 0, policy_version 173360 (0.0009) [2023-12-26 16:37:47,686][105620] Updated weights for policy 1, policy_version 174353 (0.0008) [2023-12-26 16:37:47,736][105620] Updated weights for policy 1, policy_version 174363 (0.0009) [2023-12-26 16:37:47,783][105620] Updated weights for policy 1, policy_version 174373 (0.0009) [2023-12-26 16:37:48,008][105692] Updated weights for policy 0, policy_version 173370 (0.0008) [2023-12-26 16:37:48,070][105692] Updated weights for policy 0, policy_version 173380 (0.0007) [2023-12-26 16:37:48,136][105692] Updated weights for policy 0, policy_version 173390 (0.0005) [2023-12-26 16:37:48,197][105692] Updated weights for policy 0, policy_version 173400 (0.0010) [2023-12-26 16:37:48,631][105620] Updated weights for policy 1, policy_version 174383 (0.0008) [2023-12-26 16:37:48,686][105620] Updated weights for policy 1, policy_version 174393 (0.0008) [2023-12-26 16:37:48,748][105620] Updated weights for policy 1, policy_version 174403 (0.0008) [2023-12-26 16:37:48,794][105692] Updated weights for policy 0, policy_version 173410 (0.0011) [2023-12-26 16:37:48,847][105692] Updated weights for policy 0, policy_version 173420 (0.0011) [2023-12-26 16:37:48,899][105692] Updated weights for policy 0, policy_version 173430 (0.0010) [2023-12-26 16:37:49,530][105620] Updated weights for policy 1, policy_version 174413 (0.0008) [2023-12-26 16:37:49,591][105620] Updated weights for policy 1, policy_version 174423 (0.0009) [2023-12-26 16:37:49,636][105692] Updated weights for policy 0, policy_version 173440 (0.0009) [2023-12-26 16:37:49,647][105620] Updated weights for policy 1, policy_version 174433 (0.0008) [2023-12-26 16:37:49,685][105692] Updated weights for policy 0, policy_version 173450 (0.0007) [2023-12-26 16:37:49,744][105692] Updated weights for policy 0, policy_version 173460 (0.0009) [2023-12-26 16:37:50,342][105620] Updated weights for policy 1, policy_version 174443 (0.0007) [2023-12-26 16:37:50,397][105620] Updated weights for policy 1, policy_version 174453 (0.0009) [2023-12-26 16:37:50,459][105620] Updated weights for policy 1, policy_version 174463 (0.0006) [2023-12-26 16:37:50,538][105692] Updated weights for policy 0, policy_version 173470 (0.0009) [2023-12-26 16:37:50,592][105692] Updated weights for policy 0, policy_version 173480 (0.0009) [2023-12-26 16:37:50,650][105692] Updated weights for policy 0, policy_version 173490 (0.0009) [2023-12-26 16:37:51,017][105620] Updated weights for policy 1, policy_version 174473 (0.0005) [2023-12-26 16:37:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 89096192. Throughput: 0: 9824.6, 1: 9597.3. Samples: 89087296. Policy #0 lag: (min: 4.0, avg: 8.6, max: 36.0) [2023-12-26 16:37:51,062][104569] Avg episode reward: [(0, '8996.037'), (1, '9349.397')] [2023-12-26 16:37:51,083][105620] Updated weights for policy 1, policy_version 174483 (0.0009) [2023-12-26 16:37:51,147][105620] Updated weights for policy 1, policy_version 174493 (0.0009) [2023-12-26 16:37:51,215][105620] Updated weights for policy 1, policy_version 174503 (0.0010) [2023-12-26 16:37:51,503][105692] Updated weights for policy 0, policy_version 173500 (0.0009) [2023-12-26 16:37:51,571][105692] Updated weights for policy 0, policy_version 173510 (0.0009) [2023-12-26 16:37:51,635][105692] Updated weights for policy 0, policy_version 173520 (0.0008) [2023-12-26 16:37:51,969][105620] Updated weights for policy 1, policy_version 174513 (0.0009) [2023-12-26 16:37:52,033][105620] Updated weights for policy 1, policy_version 174523 (0.0010) [2023-12-26 16:37:52,096][105620] Updated weights for policy 1, policy_version 174533 (0.0009) [2023-12-26 16:37:52,346][105692] Updated weights for policy 0, policy_version 173530 (0.0007) [2023-12-26 16:37:52,412][105692] Updated weights for policy 0, policy_version 173540 (0.0010) [2023-12-26 16:37:52,465][105692] Updated weights for policy 0, policy_version 173551 (0.0010) [2023-12-26 16:37:52,809][105620] Updated weights for policy 1, policy_version 174543 (0.0007) [2023-12-26 16:37:52,873][105620] Updated weights for policy 1, policy_version 174553 (0.0006) [2023-12-26 16:37:52,922][105620] Updated weights for policy 1, policy_version 174563 (0.0009) [2023-12-26 16:37:53,298][105692] Updated weights for policy 0, policy_version 173561 (0.0008) [2023-12-26 16:37:53,348][105692] Updated weights for policy 0, policy_version 173571 (0.0009) [2023-12-26 16:37:53,395][105692] Updated weights for policy 0, policy_version 173581 (0.0009) [2023-12-26 16:37:53,450][105692] Updated weights for policy 0, policy_version 173591 (0.0009) [2023-12-26 16:37:53,610][105620] Updated weights for policy 1, policy_version 174573 (0.0007) [2023-12-26 16:37:53,665][105620] Updated weights for policy 1, policy_version 174583 (0.0005) [2023-12-26 16:37:53,720][105620] Updated weights for policy 1, policy_version 174593 (0.0009) [2023-12-26 16:37:54,287][105692] Updated weights for policy 0, policy_version 173601 (0.0010) [2023-12-26 16:37:54,292][105620] Updated weights for policy 1, policy_version 174603 (0.0008) [2023-12-26 16:37:54,347][105692] Updated weights for policy 0, policy_version 173611 (0.0009) [2023-12-26 16:37:54,347][105620] Updated weights for policy 1, policy_version 174613 (0.0005) [2023-12-26 16:37:54,398][105692] Updated weights for policy 0, policy_version 173621 (0.0009) [2023-12-26 16:37:54,400][105620] Updated weights for policy 1, policy_version 174623 (0.0006) [2023-12-26 16:37:55,052][105620] Updated weights for policy 1, policy_version 174633 (0.0010) [2023-12-26 16:37:55,107][105620] Updated weights for policy 1, policy_version 174643 (0.0010) [2023-12-26 16:37:55,153][105692] Updated weights for policy 0, policy_version 173631 (0.0006) [2023-12-26 16:37:55,158][105620] Updated weights for policy 1, policy_version 174653 (0.0010) [2023-12-26 16:37:55,207][105620] Updated weights for policy 1, policy_version 174663 (0.0010) [2023-12-26 16:37:55,213][105692] Updated weights for policy 0, policy_version 173641 (0.0006) [2023-12-26 16:37:55,267][105692] Updated weights for policy 0, policy_version 173651 (0.0007) [2023-12-26 16:37:55,956][105620] Updated weights for policy 1, policy_version 174673 (0.0010) [2023-12-26 16:37:56,011][105620] Updated weights for policy 1, policy_version 174683 (0.0010) [2023-12-26 16:37:56,026][105692] Updated weights for policy 0, policy_version 173661 (0.0007) [2023-12-26 16:37:56,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 89186304. Throughput: 0: 9650.7, 1: 9776.3. Samples: 89202744. Policy #0 lag: (min: 4.0, avg: 8.6, max: 36.0) [2023-12-26 16:37:56,062][104569] Avg episode reward: [(0, '9176.590'), (1, '9174.402')] [2023-12-26 16:37:56,076][105620] Updated weights for policy 1, policy_version 174693 (0.0010) [2023-12-26 16:37:56,086][105692] Updated weights for policy 0, policy_version 173671 (0.0006) [2023-12-26 16:37:56,134][105692] Updated weights for policy 0, policy_version 173681 (0.0008) [2023-12-26 16:37:56,655][105620] Updated weights for policy 1, policy_version 174703 (0.0007) [2023-12-26 16:37:56,713][105620] Updated weights for policy 1, policy_version 174713 (0.0005) [2023-12-26 16:37:56,749][105692] Updated weights for policy 0, policy_version 173691 (0.0008) [2023-12-26 16:37:56,767][105620] Updated weights for policy 1, policy_version 174723 (0.0005) [2023-12-26 16:37:56,800][105692] Updated weights for policy 0, policy_version 173701 (0.0006) [2023-12-26 16:37:56,866][105692] Updated weights for policy 0, policy_version 173711 (0.0006) [2023-12-26 16:37:57,427][105692] Updated weights for policy 0, policy_version 173721 (0.0006) [2023-12-26 16:37:57,449][105620] Updated weights for policy 1, policy_version 174733 (0.0008) [2023-12-26 16:37:57,479][105692] Updated weights for policy 0, policy_version 173731 (0.0007) [2023-12-26 16:37:57,505][105620] Updated weights for policy 1, policy_version 174743 (0.0008) [2023-12-26 16:37:57,535][105692] Updated weights for policy 0, policy_version 173741 (0.0007) [2023-12-26 16:37:57,563][105620] Updated weights for policy 1, policy_version 174753 (0.0007) [2023-12-26 16:37:57,594][105692] Updated weights for policy 0, policy_version 173751 (0.0007) [2023-12-26 16:37:58,296][105692] Updated weights for policy 0, policy_version 173761 (0.0009) [2023-12-26 16:37:58,300][105620] Updated weights for policy 1, policy_version 174763 (0.0007) [2023-12-26 16:37:58,359][105692] Updated weights for policy 0, policy_version 173771 (0.0009) [2023-12-26 16:37:58,366][105620] Updated weights for policy 1, policy_version 174773 (0.0010) [2023-12-26 16:37:58,426][105692] Updated weights for policy 0, policy_version 173781 (0.0008) [2023-12-26 16:37:58,430][105620] Updated weights for policy 1, policy_version 174783 (0.0011) [2023-12-26 16:37:59,194][105692] Updated weights for policy 0, policy_version 173791 (0.0008) [2023-12-26 16:37:59,261][105620] Updated weights for policy 1, policy_version 174793 (0.0010) [2023-12-26 16:37:59,263][105692] Updated weights for policy 0, policy_version 173801 (0.0007) [2023-12-26 16:37:59,322][105620] Updated weights for policy 1, policy_version 174803 (0.0006) [2023-12-26 16:37:59,324][105692] Updated weights for policy 0, policy_version 173811 (0.0008) [2023-12-26 16:37:59,384][105620] Updated weights for policy 1, policy_version 174813 (0.0010) [2023-12-26 16:37:59,436][105620] Updated weights for policy 1, policy_version 174823 (0.0009) [2023-12-26 16:37:59,967][105692] Updated weights for policy 0, policy_version 173821 (0.0006) [2023-12-26 16:38:00,015][105692] Updated weights for policy 0, policy_version 173831 (0.0006) [2023-12-26 16:38:00,059][105692] Updated weights for policy 0, policy_version 173841 (0.0008) [2023-12-26 16:38:00,236][105620] Updated weights for policy 1, policy_version 174833 (0.0010) [2023-12-26 16:38:00,280][105620] Updated weights for policy 1, policy_version 174843 (0.0010) [2023-12-26 16:38:00,325][105620] Updated weights for policy 1, policy_version 174853 (0.0010) [2023-12-26 16:38:00,904][105692] Updated weights for policy 0, policy_version 173851 (0.0007) [2023-12-26 16:38:00,914][105620] Updated weights for policy 1, policy_version 174863 (0.0007) [2023-12-26 16:38:00,953][105692] Updated weights for policy 0, policy_version 173861 (0.0010) [2023-12-26 16:38:00,968][105620] Updated weights for policy 1, policy_version 174873 (0.0005) [2023-12-26 16:38:01,016][105692] Updated weights for policy 0, policy_version 173871 (0.0009) [2023-12-26 16:38:01,018][105620] Updated weights for policy 1, policy_version 174883 (0.0006) [2023-12-26 16:38:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 89292800. Throughput: 0: 9664.6, 1: 9809.8. Samples: 89263520. Policy #0 lag: (min: 4.0, avg: 8.6, max: 36.0) [2023-12-26 16:38:01,062][104569] Avg episode reward: [(0, '9351.232'), (1, '9174.815')] [2023-12-26 16:38:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000174888_44777472.pth... [2023-12-26 16:38:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000173736_44482560.pth [2023-12-26 16:38:01,078][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000173880_44523520.pth... [2023-12-26 16:38:01,084][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000172760_44236800.pth [2023-12-26 16:38:01,681][105620] Updated weights for policy 1, policy_version 174893 (0.0008) [2023-12-26 16:38:01,745][105620] Updated weights for policy 1, policy_version 174903 (0.0009) [2023-12-26 16:38:01,802][105692] Updated weights for policy 0, policy_version 173881 (0.0008) [2023-12-26 16:38:01,804][105620] Updated weights for policy 1, policy_version 174913 (0.0009) [2023-12-26 16:38:01,862][105692] Updated weights for policy 0, policy_version 173891 (0.0008) [2023-12-26 16:38:01,917][105692] Updated weights for policy 0, policy_version 173901 (0.0008) [2023-12-26 16:38:01,977][105692] Updated weights for policy 0, policy_version 173911 (0.0009) [2023-12-26 16:38:02,494][105620] Updated weights for policy 1, policy_version 174923 (0.0008) [2023-12-26 16:38:02,550][105620] Updated weights for policy 1, policy_version 174933 (0.0009) [2023-12-26 16:38:02,604][105620] Updated weights for policy 1, policy_version 174943 (0.0008) [2023-12-26 16:38:02,777][105692] Updated weights for policy 0, policy_version 173921 (0.0006) [2023-12-26 16:38:02,825][105692] Updated weights for policy 0, policy_version 173931 (0.0005) [2023-12-26 16:38:02,882][105692] Updated weights for policy 0, policy_version 173941 (0.0005) [2023-12-26 16:38:03,355][105620] Updated weights for policy 1, policy_version 174953 (0.0009) [2023-12-26 16:38:03,412][105620] Updated weights for policy 1, policy_version 174963 (0.0008) [2023-12-26 16:38:03,483][105620] Updated weights for policy 1, policy_version 174973 (0.0007) [2023-12-26 16:38:03,534][105692] Updated weights for policy 0, policy_version 173951 (0.0005) [2023-12-26 16:38:03,547][105620] Updated weights for policy 1, policy_version 174983 (0.0010) [2023-12-26 16:38:03,584][105692] Updated weights for policy 0, policy_version 173961 (0.0007) [2023-12-26 16:38:03,641][105692] Updated weights for policy 0, policy_version 173971 (0.0009) [2023-12-26 16:38:04,204][105620] Updated weights for policy 1, policy_version 174993 (0.0007) [2023-12-26 16:38:04,268][105620] Updated weights for policy 1, policy_version 175003 (0.0008) [2023-12-26 16:38:04,328][105620] Updated weights for policy 1, policy_version 175013 (0.0008) [2023-12-26 16:38:04,439][105692] Updated weights for policy 0, policy_version 173981 (0.0010) [2023-12-26 16:38:04,505][105692] Updated weights for policy 0, policy_version 173991 (0.0011) [2023-12-26 16:38:04,567][105692] Updated weights for policy 0, policy_version 174001 (0.0010) [2023-12-26 16:38:05,013][105620] Updated weights for policy 1, policy_version 175023 (0.0006) [2023-12-26 16:38:05,072][105620] Updated weights for policy 1, policy_version 175033 (0.0005) [2023-12-26 16:38:05,122][105620] Updated weights for policy 1, policy_version 175043 (0.0005) [2023-12-26 16:38:05,138][105692] Updated weights for policy 0, policy_version 174011 (0.0009) [2023-12-26 16:38:05,198][105692] Updated weights for policy 0, policy_version 174021 (0.0010) [2023-12-26 16:38:05,265][105692] Updated weights for policy 0, policy_version 174031 (0.0009) [2023-12-26 16:38:05,860][105620] Updated weights for policy 1, policy_version 175053 (0.0007) [2023-12-26 16:38:05,894][105692] Updated weights for policy 0, policy_version 174041 (0.0007) [2023-12-26 16:38:05,917][105620] Updated weights for policy 1, policy_version 175063 (0.0009) [2023-12-26 16:38:05,947][105692] Updated weights for policy 0, policy_version 174051 (0.0005) [2023-12-26 16:38:05,965][105620] Updated weights for policy 1, policy_version 175073 (0.0009) [2023-12-26 16:38:05,992][105692] Updated weights for policy 0, policy_version 174061 (0.0005) [2023-12-26 16:38:06,052][105692] Updated weights for policy 0, policy_version 174071 (0.0005) [2023-12-26 16:38:06,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19660.9, 300 sec: 19688.6). Total num frames: 89399296. Throughput: 0: 9577.0, 1: 9816.2. Samples: 89379376. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:38:06,062][104569] Avg episode reward: [(0, '9349.679'), (1, '9261.216')] [2023-12-26 16:38:06,665][105620] Updated weights for policy 1, policy_version 175083 (0.0008) [2023-12-26 16:38:06,726][105620] Updated weights for policy 1, policy_version 175093 (0.0005) [2023-12-26 16:38:06,751][105692] Updated weights for policy 0, policy_version 174081 (0.0010) [2023-12-26 16:38:06,779][105620] Updated weights for policy 1, policy_version 175103 (0.0005) [2023-12-26 16:38:06,798][105692] Updated weights for policy 0, policy_version 174091 (0.0007) [2023-12-26 16:38:06,849][105692] Updated weights for policy 0, policy_version 174101 (0.0005) [2023-12-26 16:38:07,445][105620] Updated weights for policy 1, policy_version 175113 (0.0006) [2023-12-26 16:38:07,497][105620] Updated weights for policy 1, policy_version 175123 (0.0005) [2023-12-26 16:38:07,551][105620] Updated weights for policy 1, policy_version 175133 (0.0005) [2023-12-26 16:38:07,594][105692] Updated weights for policy 0, policy_version 174111 (0.0009) [2023-12-26 16:38:07,601][105620] Updated weights for policy 1, policy_version 175143 (0.0005) [2023-12-26 16:38:07,651][105692] Updated weights for policy 0, policy_version 174121 (0.0010) [2023-12-26 16:38:07,715][105692] Updated weights for policy 0, policy_version 174131 (0.0011) [2023-12-26 16:38:08,298][105620] Updated weights for policy 1, policy_version 175153 (0.0008) [2023-12-26 16:38:08,358][105620] Updated weights for policy 1, policy_version 175163 (0.0008) [2023-12-26 16:38:08,373][105692] Updated weights for policy 0, policy_version 174141 (0.0008) [2023-12-26 16:38:08,419][105620] Updated weights for policy 1, policy_version 175173 (0.0008) [2023-12-26 16:38:08,430][105692] Updated weights for policy 0, policy_version 174151 (0.0006) [2023-12-26 16:38:08,494][105692] Updated weights for policy 0, policy_version 174161 (0.0005) [2023-12-26 16:38:09,067][105692] Updated weights for policy 0, policy_version 174171 (0.0006) [2023-12-26 16:38:09,118][105692] Updated weights for policy 0, policy_version 174181 (0.0006) [2023-12-26 16:38:09,184][105692] Updated weights for policy 0, policy_version 174191 (0.0005) [2023-12-26 16:38:09,333][105620] Updated weights for policy 1, policy_version 175183 (0.0008) [2023-12-26 16:38:09,398][105620] Updated weights for policy 1, policy_version 175193 (0.0007) [2023-12-26 16:38:09,463][105620] Updated weights for policy 1, policy_version 175203 (0.0009) [2023-12-26 16:38:09,859][105692] Updated weights for policy 0, policy_version 174201 (0.0008) [2023-12-26 16:38:09,919][105692] Updated weights for policy 0, policy_version 174211 (0.0008) [2023-12-26 16:38:09,986][105692] Updated weights for policy 0, policy_version 174221 (0.0008) [2023-12-26 16:38:10,043][105692] Updated weights for policy 0, policy_version 174231 (0.0008) [2023-12-26 16:38:10,268][105620] Updated weights for policy 1, policy_version 175213 (0.0009) [2023-12-26 16:38:10,318][105620] Updated weights for policy 1, policy_version 175223 (0.0008) [2023-12-26 16:38:10,370][105620] Updated weights for policy 1, policy_version 175233 (0.0009) [2023-12-26 16:38:10,822][105692] Updated weights for policy 0, policy_version 174241 (0.0009) [2023-12-26 16:38:10,881][105692] Updated weights for policy 0, policy_version 174253 (0.0010) [2023-12-26 16:38:10,932][105692] Updated weights for policy 0, policy_version 174263 (0.0010) [2023-12-26 16:38:11,052][105620] Updated weights for policy 1, policy_version 175243 (0.0008) [2023-12-26 16:38:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 89489408. Throughput: 0: 9598.5, 1: 9742.4. Samples: 89498060. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:38:11,062][104569] Avg episode reward: [(0, '9348.126'), (1, '9171.760')] [2023-12-26 16:38:11,118][105620] Updated weights for policy 1, policy_version 175253 (0.0009) [2023-12-26 16:38:11,188][105620] Updated weights for policy 1, policy_version 175263 (0.0008) [2023-12-26 16:38:11,752][105692] Updated weights for policy 0, policy_version 174273 (0.0010) [2023-12-26 16:38:11,808][105692] Updated weights for policy 0, policy_version 174283 (0.0009) [2023-12-26 16:38:11,860][105692] Updated weights for policy 0, policy_version 174293 (0.0009) [2023-12-26 16:38:11,996][105620] Updated weights for policy 1, policy_version 175273 (0.0009) [2023-12-26 16:38:12,055][105620] Updated weights for policy 1, policy_version 175283 (0.0010) [2023-12-26 16:38:12,124][105620] Updated weights for policy 1, policy_version 175293 (0.0010) [2023-12-26 16:38:12,179][105620] Updated weights for policy 1, policy_version 175303 (0.0009) [2023-12-26 16:38:12,518][105692] Updated weights for policy 0, policy_version 174303 (0.0009) [2023-12-26 16:38:12,582][105692] Updated weights for policy 0, policy_version 174313 (0.0007) [2023-12-26 16:38:12,646][105692] Updated weights for policy 0, policy_version 174323 (0.0006) [2023-12-26 16:38:13,040][105620] Updated weights for policy 1, policy_version 175313 (0.0009) [2023-12-26 16:38:13,105][105620] Updated weights for policy 1, policy_version 175323 (0.0009) [2023-12-26 16:38:13,167][105620] Updated weights for policy 1, policy_version 175333 (0.0009) [2023-12-26 16:38:13,226][105692] Updated weights for policy 0, policy_version 174333 (0.0006) [2023-12-26 16:38:13,293][105692] Updated weights for policy 0, policy_version 174343 (0.0010) [2023-12-26 16:38:13,351][105692] Updated weights for policy 0, policy_version 174353 (0.0010) [2023-12-26 16:38:13,824][105620] Updated weights for policy 1, policy_version 175343 (0.0006) [2023-12-26 16:38:13,883][105620] Updated weights for policy 1, policy_version 175353 (0.0005) [2023-12-26 16:38:13,949][105620] Updated weights for policy 1, policy_version 175363 (0.0008) [2023-12-26 16:38:14,179][105692] Updated weights for policy 0, policy_version 174363 (0.0009) [2023-12-26 16:38:14,239][105692] Updated weights for policy 0, policy_version 174373 (0.0008) [2023-12-26 16:38:14,296][105692] Updated weights for policy 0, policy_version 174383 (0.0008) [2023-12-26 16:38:14,608][105620] Updated weights for policy 1, policy_version 175373 (0.0010) [2023-12-26 16:38:14,667][105620] Updated weights for policy 1, policy_version 175383 (0.0010) [2023-12-26 16:38:14,725][105620] Updated weights for policy 1, policy_version 175393 (0.0010) [2023-12-26 16:38:15,036][105692] Updated weights for policy 0, policy_version 174393 (0.0006) [2023-12-26 16:38:15,102][105692] Updated weights for policy 0, policy_version 174403 (0.0005) [2023-12-26 16:38:15,159][105692] Updated weights for policy 0, policy_version 174413 (0.0005) [2023-12-26 16:38:15,214][105692] Updated weights for policy 0, policy_version 174423 (0.0006) [2023-12-26 16:38:15,480][105620] Updated weights for policy 1, policy_version 175403 (0.0011) [2023-12-26 16:38:15,543][105620] Updated weights for policy 1, policy_version 175413 (0.0011) [2023-12-26 16:38:15,600][105620] Updated weights for policy 1, policy_version 175423 (0.0011) [2023-12-26 16:38:15,951][105692] Updated weights for policy 0, policy_version 174433 (0.0007) [2023-12-26 16:38:16,014][105692] Updated weights for policy 0, policy_version 174443 (0.0007) [2023-12-26 16:38:16,062][104569] Fps is (10 sec: 18022.0, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 89579520. Throughput: 0: 9622.8, 1: 9637.9. Samples: 89555264. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:38:16,063][104569] Avg episode reward: [(0, '9349.313'), (1, '9260.145')] [2023-12-26 16:38:16,066][105692] Updated weights for policy 0, policy_version 174453 (0.0008) [2023-12-26 16:38:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000175432_44916736.pth... [2023-12-26 16:38:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000174312_44630016.pth [2023-12-26 16:38:16,079][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000174456_44670976.pth... [2023-12-26 16:38:16,082][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000173304_44376064.pth [2023-12-26 16:38:16,317][105620] Updated weights for policy 1, policy_version 175433 (0.0011) [2023-12-26 16:38:16,382][105620] Updated weights for policy 1, policy_version 175443 (0.0011) [2023-12-26 16:38:16,448][105620] Updated weights for policy 1, policy_version 175453 (0.0010) [2023-12-26 16:38:16,510][105620] Updated weights for policy 1, policy_version 175463 (0.0011) [2023-12-26 16:38:16,703][105692] Updated weights for policy 0, policy_version 174463 (0.0006) [2023-12-26 16:38:16,761][105692] Updated weights for policy 0, policy_version 174473 (0.0006) [2023-12-26 16:38:16,808][105692] Updated weights for policy 0, policy_version 174483 (0.0006) [2023-12-26 16:38:17,132][105620] Updated weights for policy 1, policy_version 175473 (0.0011) [2023-12-26 16:38:17,203][105620] Updated weights for policy 1, policy_version 175483 (0.0010) [2023-12-26 16:38:17,268][105620] Updated weights for policy 1, policy_version 175493 (0.0010) [2023-12-26 16:38:17,466][105692] Updated weights for policy 0, policy_version 174494 (0.0007) [2023-12-26 16:38:17,522][105692] Updated weights for policy 0, policy_version 174504 (0.0005) [2023-12-26 16:38:17,575][105692] Updated weights for policy 0, policy_version 174514 (0.0006) [2023-12-26 16:38:17,949][105620] Updated weights for policy 1, policy_version 175503 (0.0011) [2023-12-26 16:38:18,011][105620] Updated weights for policy 1, policy_version 175513 (0.0010) [2023-12-26 16:38:18,072][105620] Updated weights for policy 1, policy_version 175523 (0.0010) [2023-12-26 16:38:18,103][105692] Updated weights for policy 0, policy_version 174524 (0.0005) [2023-12-26 16:38:18,170][105692] Updated weights for policy 0, policy_version 174534 (0.0006) [2023-12-26 16:38:18,229][105692] Updated weights for policy 0, policy_version 174544 (0.0008) [2023-12-26 16:38:18,698][105620] Updated weights for policy 1, policy_version 175533 (0.0008) [2023-12-26 16:38:18,767][105620] Updated weights for policy 1, policy_version 175543 (0.0005) [2023-12-26 16:38:18,828][105620] Updated weights for policy 1, policy_version 175553 (0.0005) [2023-12-26 16:38:18,899][105692] Updated weights for policy 0, policy_version 174554 (0.0006) [2023-12-26 16:38:18,970][105692] Updated weights for policy 0, policy_version 174564 (0.0007) [2023-12-26 16:38:19,037][105692] Updated weights for policy 0, policy_version 174574 (0.0008) [2023-12-26 16:38:19,089][105692] Updated weights for policy 0, policy_version 174584 (0.0010) [2023-12-26 16:38:19,502][105620] Updated weights for policy 1, policy_version 175563 (0.0007) [2023-12-26 16:38:19,565][105620] Updated weights for policy 1, policy_version 175573 (0.0011) [2023-12-26 16:38:19,633][105620] Updated weights for policy 1, policy_version 175583 (0.0011) [2023-12-26 16:38:19,736][105692] Updated weights for policy 0, policy_version 174594 (0.0011) [2023-12-26 16:38:19,795][105692] Updated weights for policy 0, policy_version 174604 (0.0010) [2023-12-26 16:38:19,864][105692] Updated weights for policy 0, policy_version 174614 (0.0008) [2023-12-26 16:38:20,363][105620] Updated weights for policy 1, policy_version 175593 (0.0009) [2023-12-26 16:38:20,423][105620] Updated weights for policy 1, policy_version 175603 (0.0007) [2023-12-26 16:38:20,482][105620] Updated weights for policy 1, policy_version 175613 (0.0005) [2023-12-26 16:38:20,550][105620] Updated weights for policy 1, policy_version 175623 (0.0006) [2023-12-26 16:38:20,611][105692] Updated weights for policy 0, policy_version 174624 (0.0006) [2023-12-26 16:38:20,684][105692] Updated weights for policy 0, policy_version 174634 (0.0008) [2023-12-26 16:38:20,752][105692] Updated weights for policy 0, policy_version 174644 (0.0008) [2023-12-26 16:38:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19688.6). Total num frames: 89686016. Throughput: 0: 9738.3, 1: 9663.3. Samples: 89675780. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:38:21,062][104569] Avg episode reward: [(0, '9350.167'), (1, '9171.206')] [2023-12-26 16:38:21,256][105620] Updated weights for policy 1, policy_version 175633 (0.0007) [2023-12-26 16:38:21,322][105620] Updated weights for policy 1, policy_version 175643 (0.0006) [2023-12-26 16:38:21,388][105620] Updated weights for policy 1, policy_version 175653 (0.0008) [2023-12-26 16:38:21,389][105692] Updated weights for policy 0, policy_version 174654 (0.0009) [2023-12-26 16:38:21,452][105692] Updated weights for policy 0, policy_version 174664 (0.0010) [2023-12-26 16:38:21,512][105692] Updated weights for policy 0, policy_version 174674 (0.0008) [2023-12-26 16:38:22,152][105620] Updated weights for policy 1, policy_version 175663 (0.0010) [2023-12-26 16:38:22,212][105620] Updated weights for policy 1, policy_version 175673 (0.0008) [2023-12-26 16:38:22,223][105692] Updated weights for policy 0, policy_version 174684 (0.0009) [2023-12-26 16:38:22,278][105620] Updated weights for policy 1, policy_version 175683 (0.0008) [2023-12-26 16:38:22,287][105692] Updated weights for policy 0, policy_version 174694 (0.0009) [2023-12-26 16:38:22,346][105692] Updated weights for policy 0, policy_version 174704 (0.0008) [2023-12-26 16:38:22,984][105620] Updated weights for policy 1, policy_version 175693 (0.0006) [2023-12-26 16:38:23,044][105620] Updated weights for policy 1, policy_version 175703 (0.0006) [2023-12-26 16:38:23,111][105620] Updated weights for policy 1, policy_version 175713 (0.0006) [2023-12-26 16:38:23,200][105692] Updated weights for policy 0, policy_version 174714 (0.0009) [2023-12-26 16:38:23,253][105692] Updated weights for policy 0, policy_version 174724 (0.0008) [2023-12-26 16:38:23,316][105692] Updated weights for policy 0, policy_version 174734 (0.0008) [2023-12-26 16:38:23,376][105692] Updated weights for policy 0, policy_version 174744 (0.0008) [2023-12-26 16:38:23,800][105620] Updated weights for policy 1, policy_version 175723 (0.0006) [2023-12-26 16:38:23,858][105620] Updated weights for policy 1, policy_version 175733 (0.0005) [2023-12-26 16:38:23,923][105620] Updated weights for policy 1, policy_version 175743 (0.0005) [2023-12-26 16:38:24,082][105692] Updated weights for policy 0, policy_version 174754 (0.0008) [2023-12-26 16:38:24,134][105692] Updated weights for policy 0, policy_version 174764 (0.0008) [2023-12-26 16:38:24,198][105692] Updated weights for policy 0, policy_version 174774 (0.0008) [2023-12-26 16:38:24,566][105620] Updated weights for policy 1, policy_version 175753 (0.0009) [2023-12-26 16:38:24,609][105620] Updated weights for policy 1, policy_version 175763 (0.0010) [2023-12-26 16:38:24,654][105620] Updated weights for policy 1, policy_version 175773 (0.0010) [2023-12-26 16:38:24,698][105586] KL-divergence is very high: 101.6662 [2023-12-26 16:38:24,698][105620] Updated weights for policy 1, policy_version 175783 (0.0010) [2023-12-26 16:38:24,909][105692] Updated weights for policy 0, policy_version 174784 (0.0006) [2023-12-26 16:38:24,975][105692] Updated weights for policy 0, policy_version 174794 (0.0005) [2023-12-26 16:38:25,021][105692] Updated weights for policy 0, policy_version 174804 (0.0005) [2023-12-26 16:38:25,354][105620] Updated weights for policy 1, policy_version 175793 (0.0010) [2023-12-26 16:38:25,399][105620] Updated weights for policy 1, policy_version 175803 (0.0010) [2023-12-26 16:38:25,442][105620] Updated weights for policy 1, policy_version 175813 (0.0010) [2023-12-26 16:38:25,606][105692] Updated weights for policy 0, policy_version 174814 (0.0007) [2023-12-26 16:38:25,666][105692] Updated weights for policy 0, policy_version 174824 (0.0008) [2023-12-26 16:38:25,719][105692] Updated weights for policy 0, policy_version 174836 (0.0010) [2023-12-26 16:38:26,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 89784320. Throughput: 0: 9744.3, 1: 9742.8. Samples: 89795032. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:38:26,063][104569] Avg episode reward: [(0, '9350.312'), (1, '8800.487')] [2023-12-26 16:38:26,132][105620] Updated weights for policy 1, policy_version 175823 (0.0008) [2023-12-26 16:38:26,180][105620] Updated weights for policy 1, policy_version 175833 (0.0010) [2023-12-26 16:38:26,232][105620] Updated weights for policy 1, policy_version 175843 (0.0008) [2023-12-26 16:38:26,442][105692] Updated weights for policy 0, policy_version 174846 (0.0010) [2023-12-26 16:38:26,496][105692] Updated weights for policy 0, policy_version 174856 (0.0009) [2023-12-26 16:38:26,543][105692] Updated weights for policy 0, policy_version 174866 (0.0009) [2023-12-26 16:38:26,950][105620] Updated weights for policy 1, policy_version 175853 (0.0005) [2023-12-26 16:38:27,004][105620] Updated weights for policy 1, policy_version 175863 (0.0005) [2023-12-26 16:38:27,059][105620] Updated weights for policy 1, policy_version 175873 (0.0005) [2023-12-26 16:38:27,405][105692] Updated weights for policy 0, policy_version 174876 (0.0009) [2023-12-26 16:38:27,458][105692] Updated weights for policy 0, policy_version 174886 (0.0009) [2023-12-26 16:38:27,506][105692] Updated weights for policy 0, policy_version 174896 (0.0008) [2023-12-26 16:38:27,563][105620] Updated weights for policy 1, policy_version 175883 (0.0007) [2023-12-26 16:38:27,609][105620] Updated weights for policy 1, policy_version 175893 (0.0008) [2023-12-26 16:38:27,663][105620] Updated weights for policy 1, policy_version 175903 (0.0009) [2023-12-26 16:38:28,180][105692] Updated weights for policy 0, policy_version 174906 (0.0008) [2023-12-26 16:38:28,227][105692] Updated weights for policy 0, policy_version 174916 (0.0005) [2023-12-26 16:38:28,278][105692] Updated weights for policy 0, policy_version 174926 (0.0005) [2023-12-26 16:38:28,327][105692] Updated weights for policy 0, policy_version 174936 (0.0006) [2023-12-26 16:38:28,447][105620] Updated weights for policy 1, policy_version 175913 (0.0009) [2023-12-26 16:38:28,503][105620] Updated weights for policy 1, policy_version 175923 (0.0006) [2023-12-26 16:38:28,555][105620] Updated weights for policy 1, policy_version 175933 (0.0005) [2023-12-26 16:38:28,609][105620] Updated weights for policy 1, policy_version 175943 (0.0005) [2023-12-26 16:38:29,020][105692] Updated weights for policy 0, policy_version 174946 (0.0008) [2023-12-26 16:38:29,081][105692] Updated weights for policy 0, policy_version 174956 (0.0009) [2023-12-26 16:38:29,133][105692] Updated weights for policy 0, policy_version 174966 (0.0009) [2023-12-26 16:38:29,168][105620] Updated weights for policy 1, policy_version 175953 (0.0008) [2023-12-26 16:38:29,240][105620] Updated weights for policy 1, policy_version 175963 (0.0011) [2023-12-26 16:38:29,302][105620] Updated weights for policy 1, policy_version 175973 (0.0011) [2023-12-26 16:38:29,951][105692] Updated weights for policy 0, policy_version 174976 (0.0010) [2023-12-26 16:38:29,951][105620] Updated weights for policy 1, policy_version 175983 (0.0010) [2023-12-26 16:38:30,007][105692] Updated weights for policy 0, policy_version 174986 (0.0009) [2023-12-26 16:38:30,016][105620] Updated weights for policy 1, policy_version 175993 (0.0010) [2023-12-26 16:38:30,049][105692] Updated weights for policy 0, policy_version 174996 (0.0008) [2023-12-26 16:38:30,074][105620] Updated weights for policy 1, policy_version 176003 (0.0010) [2023-12-26 16:38:30,663][105692] Updated weights for policy 0, policy_version 175006 (0.0005) [2023-12-26 16:38:30,706][105692] Updated weights for policy 0, policy_version 175016 (0.0005) [2023-12-26 16:38:30,755][105620] Updated weights for policy 1, policy_version 176013 (0.0006) [2023-12-26 16:38:30,758][105692] Updated weights for policy 0, policy_version 175026 (0.0005) [2023-12-26 16:38:30,808][105620] Updated weights for policy 1, policy_version 176023 (0.0005) [2023-12-26 16:38:30,865][105620] Updated weights for policy 1, policy_version 176033 (0.0007) [2023-12-26 16:38:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19716.3). Total num frames: 89890816. Throughput: 0: 9752.4, 1: 9849.4. Samples: 89855768. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:38:31,062][104569] Avg episode reward: [(0, '8996.339'), (1, '8633.717')] [2023-12-26 16:38:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000175032_44818432.pth... [2023-12-26 16:38:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000176040_45072384.pth... [2023-12-26 16:38:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000174888_44777472.pth [2023-12-26 16:38:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000173880_44523520.pth [2023-12-26 16:38:31,484][105692] Updated weights for policy 0, policy_version 175036 (0.0007) [2023-12-26 16:38:31,488][105620] Updated weights for policy 1, policy_version 176043 (0.0009) [2023-12-26 16:38:31,538][105692] Updated weights for policy 0, policy_version 175046 (0.0007) [2023-12-26 16:38:31,549][105620] Updated weights for policy 1, policy_version 176053 (0.0010) [2023-12-26 16:38:31,594][105692] Updated weights for policy 0, policy_version 175056 (0.0005) [2023-12-26 16:38:31,611][105620] Updated weights for policy 1, policy_version 176063 (0.0010) [2023-12-26 16:38:32,270][105692] Updated weights for policy 0, policy_version 175066 (0.0008) [2023-12-26 16:38:32,291][105620] Updated weights for policy 1, policy_version 176073 (0.0008) [2023-12-26 16:38:32,333][105692] Updated weights for policy 0, policy_version 175076 (0.0007) [2023-12-26 16:38:32,351][105620] Updated weights for policy 1, policy_version 176083 (0.0010) [2023-12-26 16:38:32,399][105692] Updated weights for policy 0, policy_version 175086 (0.0007) [2023-12-26 16:38:32,416][105620] Updated weights for policy 1, policy_version 176093 (0.0010) [2023-12-26 16:38:32,455][105692] Updated weights for policy 0, policy_version 175096 (0.0006) [2023-12-26 16:38:32,459][105620] Updated weights for policy 1, policy_version 176103 (0.0006) [2023-12-26 16:38:33,030][105692] Updated weights for policy 0, policy_version 175106 (0.0010) [2023-12-26 16:38:33,038][105620] Updated weights for policy 1, policy_version 176113 (0.0010) [2023-12-26 16:38:33,088][105692] Updated weights for policy 0, policy_version 175116 (0.0010) [2023-12-26 16:38:33,095][105620] Updated weights for policy 1, policy_version 176123 (0.0010) [2023-12-26 16:38:33,140][105692] Updated weights for policy 0, policy_version 175126 (0.0007) [2023-12-26 16:38:33,153][105620] Updated weights for policy 1, policy_version 176133 (0.0010) [2023-12-26 16:38:33,841][105620] Updated weights for policy 1, policy_version 176143 (0.0007) [2023-12-26 16:38:33,886][105692] Updated weights for policy 0, policy_version 175136 (0.0006) [2023-12-26 16:38:33,894][105620] Updated weights for policy 1, policy_version 176153 (0.0005) [2023-12-26 16:38:33,933][105692] Updated weights for policy 0, policy_version 175146 (0.0009) [2023-12-26 16:38:33,946][105620] Updated weights for policy 1, policy_version 176163 (0.0005) [2023-12-26 16:38:33,982][105692] Updated weights for policy 0, policy_version 175156 (0.0008) [2023-12-26 16:38:34,532][105620] Updated weights for policy 1, policy_version 176173 (0.0005) [2023-12-26 16:38:34,583][105620] Updated weights for policy 1, policy_version 176183 (0.0008) [2023-12-26 16:38:34,633][105620] Updated weights for policy 1, policy_version 176193 (0.0008) [2023-12-26 16:38:34,828][105692] Updated weights for policy 0, policy_version 175167 (0.0009) [2023-12-26 16:38:34,877][105692] Updated weights for policy 0, policy_version 175177 (0.0009) [2023-12-26 16:38:34,929][105692] Updated weights for policy 0, policy_version 175187 (0.0009) [2023-12-26 16:38:35,240][105620] Updated weights for policy 1, policy_version 176203 (0.0005) [2023-12-26 16:38:35,286][105620] Updated weights for policy 1, policy_version 176213 (0.0005) [2023-12-26 16:38:35,332][105620] Updated weights for policy 1, policy_version 176223 (0.0005) [2023-12-26 16:38:35,621][105692] Updated weights for policy 0, policy_version 175198 (0.0009) [2023-12-26 16:38:35,678][105692] Updated weights for policy 0, policy_version 175208 (0.0009) [2023-12-26 16:38:35,724][105692] Updated weights for policy 0, policy_version 175218 (0.0008) [2023-12-26 16:38:35,961][105620] Updated weights for policy 1, policy_version 176233 (0.0007) [2023-12-26 16:38:36,019][105620] Updated weights for policy 1, policy_version 176244 (0.0010) [2023-12-26 16:38:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.2, 300 sec: 19716.3). Total num frames: 89989120. Throughput: 0: 9796.0, 1: 10008.7. Samples: 89978516. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:38:36,063][104569] Avg episode reward: [(0, '8996.182'), (1, '9008.445')] [2023-12-26 16:38:36,073][105620] Updated weights for policy 1, policy_version 176256 (0.0010) [2023-12-26 16:38:36,456][105692] Updated weights for policy 0, policy_version 175228 (0.0009) [2023-12-26 16:38:36,511][105692] Updated weights for policy 0, policy_version 175238 (0.0009) [2023-12-26 16:38:36,572][105692] Updated weights for policy 0, policy_version 175248 (0.0009) [2023-12-26 16:38:36,871][105620] Updated weights for policy 1, policy_version 176266 (0.0009) [2023-12-26 16:38:36,938][105620] Updated weights for policy 1, policy_version 176276 (0.0010) [2023-12-26 16:38:36,993][105620] Updated weights for policy 1, policy_version 176286 (0.0010) [2023-12-26 16:38:37,040][105620] Updated weights for policy 1, policy_version 176296 (0.0009) [2023-12-26 16:38:37,298][105692] Updated weights for policy 0, policy_version 175258 (0.0008) [2023-12-26 16:38:37,345][105692] Updated weights for policy 0, policy_version 175268 (0.0009) [2023-12-26 16:38:37,393][105692] Updated weights for policy 0, policy_version 175278 (0.0009) [2023-12-26 16:38:37,445][105692] Updated weights for policy 0, policy_version 175288 (0.0009) [2023-12-26 16:38:37,817][105620] Updated weights for policy 1, policy_version 176306 (0.0008) [2023-12-26 16:38:37,867][105620] Updated weights for policy 1, policy_version 176316 (0.0009) [2023-12-26 16:38:37,916][105620] Updated weights for policy 1, policy_version 176327 (0.0009) [2023-12-26 16:38:38,225][105692] Updated weights for policy 0, policy_version 175298 (0.0009) [2023-12-26 16:38:38,271][105692] Updated weights for policy 0, policy_version 175308 (0.0009) [2023-12-26 16:38:38,319][105692] Updated weights for policy 0, policy_version 175318 (0.0009) [2023-12-26 16:38:38,674][105620] Updated weights for policy 1, policy_version 176337 (0.0010) [2023-12-26 16:38:38,724][105620] Updated weights for policy 1, policy_version 176347 (0.0009) [2023-12-26 16:38:38,783][105620] Updated weights for policy 1, policy_version 176357 (0.0008) [2023-12-26 16:38:39,085][105692] Updated weights for policy 0, policy_version 175328 (0.0009) [2023-12-26 16:38:39,146][105692] Updated weights for policy 0, policy_version 175338 (0.0009) [2023-12-26 16:38:39,199][105692] Updated weights for policy 0, policy_version 175348 (0.0010) [2023-12-26 16:38:39,597][105620] Updated weights for policy 1, policy_version 176367 (0.0007) [2023-12-26 16:38:39,667][105620] Updated weights for policy 1, policy_version 176377 (0.0005) [2023-12-26 16:38:39,727][105620] Updated weights for policy 1, policy_version 176387 (0.0005) [2023-12-26 16:38:40,087][105692] Updated weights for policy 0, policy_version 175358 (0.0009) [2023-12-26 16:38:40,159][105692] Updated weights for policy 0, policy_version 175368 (0.0010) [2023-12-26 16:38:40,231][105692] Updated weights for policy 0, policy_version 175378 (0.0010) [2023-12-26 16:38:40,334][105620] Updated weights for policy 1, policy_version 176397 (0.0007) [2023-12-26 16:38:40,392][105620] Updated weights for policy 1, policy_version 176407 (0.0008) [2023-12-26 16:38:40,445][105620] Updated weights for policy 1, policy_version 176417 (0.0009) [2023-12-26 16:38:40,974][105692] Updated weights for policy 0, policy_version 175388 (0.0009) [2023-12-26 16:38:41,037][105692] Updated weights for policy 0, policy_version 175398 (0.0009) [2023-12-26 16:38:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 90079232. Throughput: 0: 9850.1, 1: 9952.3. Samples: 90093848. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:38:41,062][104569] Avg episode reward: [(0, '9261.889'), (1, '8829.327')] [2023-12-26 16:38:41,100][105692] Updated weights for policy 0, policy_version 175408 (0.0009) [2023-12-26 16:38:41,196][105620] Updated weights for policy 1, policy_version 176427 (0.0009) [2023-12-26 16:38:41,261][105620] Updated weights for policy 1, policy_version 176437 (0.0008) [2023-12-26 16:38:41,323][105620] Updated weights for policy 1, policy_version 176447 (0.0008) [2023-12-26 16:38:41,847][105692] Updated weights for policy 0, policy_version 175418 (0.0007) [2023-12-26 16:38:41,913][105692] Updated weights for policy 0, policy_version 175428 (0.0006) [2023-12-26 16:38:41,977][105692] Updated weights for policy 0, policy_version 175438 (0.0006) [2023-12-26 16:38:42,035][105692] Updated weights for policy 0, policy_version 175448 (0.0006) [2023-12-26 16:38:42,147][105620] Updated weights for policy 1, policy_version 176457 (0.0009) [2023-12-26 16:38:42,206][105620] Updated weights for policy 1, policy_version 176467 (0.0009) [2023-12-26 16:38:42,264][105620] Updated weights for policy 1, policy_version 176477 (0.0008) [2023-12-26 16:38:42,333][105620] Updated weights for policy 1, policy_version 176487 (0.0008) [2023-12-26 16:38:42,618][105692] Updated weights for policy 0, policy_version 175458 (0.0007) [2023-12-26 16:38:42,680][105692] Updated weights for policy 0, policy_version 175468 (0.0006) [2023-12-26 16:38:42,735][105692] Updated weights for policy 0, policy_version 175478 (0.0009) [2023-12-26 16:38:43,188][105620] Updated weights for policy 1, policy_version 176497 (0.0005) [2023-12-26 16:38:43,248][105620] Updated weights for policy 1, policy_version 176507 (0.0005) [2023-12-26 16:38:43,300][105620] Updated weights for policy 1, policy_version 176517 (0.0008) [2023-12-26 16:38:43,373][105692] Updated weights for policy 0, policy_version 175488 (0.0008) [2023-12-26 16:38:43,443][105692] Updated weights for policy 0, policy_version 175498 (0.0006) [2023-12-26 16:38:43,511][105692] Updated weights for policy 0, policy_version 175508 (0.0006) [2023-12-26 16:38:43,961][105620] Updated weights for policy 1, policy_version 176527 (0.0009) [2023-12-26 16:38:44,010][105620] Updated weights for policy 1, policy_version 176537 (0.0008) [2023-12-26 16:38:44,056][105620] Updated weights for policy 1, policy_version 176547 (0.0008) [2023-12-26 16:38:44,141][105692] Updated weights for policy 0, policy_version 175518 (0.0008) [2023-12-26 16:38:44,187][105692] Updated weights for policy 0, policy_version 175528 (0.0009) [2023-12-26 16:38:44,234][105692] Updated weights for policy 0, policy_version 175538 (0.0009) [2023-12-26 16:38:44,831][105620] Updated weights for policy 1, policy_version 176557 (0.0009) [2023-12-26 16:38:44,895][105620] Updated weights for policy 1, policy_version 176567 (0.0009) [2023-12-26 16:38:44,961][105620] Updated weights for policy 1, policy_version 176577 (0.0009) [2023-12-26 16:38:44,998][105692] Updated weights for policy 0, policy_version 175548 (0.0008) [2023-12-26 16:38:45,061][105692] Updated weights for policy 0, policy_version 175558 (0.0009) [2023-12-26 16:38:45,122][105692] Updated weights for policy 0, policy_version 175568 (0.0009) [2023-12-26 16:38:45,694][105620] Updated weights for policy 1, policy_version 176587 (0.0009) [2023-12-26 16:38:45,742][105620] Updated weights for policy 1, policy_version 176597 (0.0005) [2023-12-26 16:38:45,795][105620] Updated weights for policy 1, policy_version 176607 (0.0007) [2023-12-26 16:38:45,860][105692] Updated weights for policy 0, policy_version 175578 (0.0009) [2023-12-26 16:38:45,924][105692] Updated weights for policy 0, policy_version 175588 (0.0009) [2023-12-26 16:38:45,978][105692] Updated weights for policy 0, policy_version 175598 (0.0009) [2023-12-26 16:38:46,034][105692] Updated weights for policy 0, policy_version 175608 (0.0009) [2023-12-26 16:38:46,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 90185728. Throughput: 0: 9823.6, 1: 9893.4. Samples: 90150784. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:38:46,062][104569] Avg episode reward: [(0, '9349.129'), (1, '8825.436')] [2023-12-26 16:38:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000175608_44965888.pth... [2023-12-26 16:38:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000176616_45219840.pth... [2023-12-26 16:38:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000174456_44670976.pth [2023-12-26 16:38:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000175432_44916736.pth [2023-12-26 16:38:46,533][105620] Updated weights for policy 1, policy_version 176617 (0.0008) [2023-12-26 16:38:46,580][105620] Updated weights for policy 1, policy_version 176627 (0.0008) [2023-12-26 16:38:46,631][105620] Updated weights for policy 1, policy_version 176637 (0.0008) [2023-12-26 16:38:46,688][105620] Updated weights for policy 1, policy_version 176647 (0.0008) [2023-12-26 16:38:46,774][105692] Updated weights for policy 0, policy_version 175618 (0.0009) [2023-12-26 16:38:46,832][105692] Updated weights for policy 0, policy_version 175628 (0.0009) [2023-12-26 16:38:46,886][105692] Updated weights for policy 0, policy_version 175638 (0.0009) [2023-12-26 16:38:47,483][105620] Updated weights for policy 1, policy_version 176657 (0.0010) [2023-12-26 16:38:47,540][105620] Updated weights for policy 1, policy_version 176667 (0.0008) [2023-12-26 16:38:47,575][105692] Updated weights for policy 0, policy_version 175648 (0.0008) [2023-12-26 16:38:47,599][105620] Updated weights for policy 1, policy_version 176677 (0.0010) [2023-12-26 16:38:47,635][105692] Updated weights for policy 0, policy_version 175658 (0.0009) [2023-12-26 16:38:47,696][105692] Updated weights for policy 0, policy_version 175668 (0.0007) [2023-12-26 16:38:48,363][105620] Updated weights for policy 1, policy_version 176687 (0.0007) [2023-12-26 16:38:48,378][105692] Updated weights for policy 0, policy_version 175678 (0.0009) [2023-12-26 16:38:48,425][105620] Updated weights for policy 1, policy_version 176697 (0.0008) [2023-12-26 16:38:48,437][105692] Updated weights for policy 0, policy_version 175688 (0.0008) [2023-12-26 16:38:48,473][105620] Updated weights for policy 1, policy_version 176707 (0.0007) [2023-12-26 16:38:48,488][105692] Updated weights for policy 0, policy_version 175698 (0.0008) [2023-12-26 16:38:49,189][105620] Updated weights for policy 1, policy_version 176717 (0.0008) [2023-12-26 16:38:49,252][105620] Updated weights for policy 1, policy_version 176727 (0.0008) [2023-12-26 16:38:49,265][105692] Updated weights for policy 0, policy_version 175708 (0.0008) [2023-12-26 16:38:49,315][105620] Updated weights for policy 1, policy_version 176737 (0.0009) [2023-12-26 16:38:49,323][105692] Updated weights for policy 0, policy_version 175718 (0.0006) [2023-12-26 16:38:49,387][105692] Updated weights for policy 0, policy_version 175728 (0.0009) [2023-12-26 16:38:50,017][105620] Updated weights for policy 1, policy_version 176747 (0.0008) [2023-12-26 16:38:50,081][105620] Updated weights for policy 1, policy_version 176757 (0.0008) [2023-12-26 16:38:50,147][105620] Updated weights for policy 1, policy_version 176767 (0.0008) [2023-12-26 16:38:50,184][105692] Updated weights for policy 0, policy_version 175739 (0.0009) [2023-12-26 16:38:50,248][105692] Updated weights for policy 0, policy_version 175749 (0.0008) [2023-12-26 16:38:50,311][105692] Updated weights for policy 0, policy_version 175759 (0.0008) [2023-12-26 16:38:50,884][105620] Updated weights for policy 1, policy_version 176777 (0.0008) [2023-12-26 16:38:50,944][105620] Updated weights for policy 1, policy_version 176787 (0.0005) [2023-12-26 16:38:50,997][105620] Updated weights for policy 1, policy_version 176797 (0.0008) [2023-12-26 16:38:51,015][105692] Updated weights for policy 0, policy_version 175769 (0.0008) [2023-12-26 16:38:51,058][105620] Updated weights for policy 1, policy_version 176807 (0.0008) [2023-12-26 16:38:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 90267648. Throughput: 0: 9852.9, 1: 9819.4. Samples: 90264628. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:38:51,062][104569] Avg episode reward: [(0, '9261.508'), (1, '9172.515')] [2023-12-26 16:38:51,077][105692] Updated weights for policy 0, policy_version 175779 (0.0009) [2023-12-26 16:38:51,134][105692] Updated weights for policy 0, policy_version 175789 (0.0010) [2023-12-26 16:38:51,194][105692] Updated weights for policy 0, policy_version 175799 (0.0010) [2023-12-26 16:38:51,755][105620] Updated weights for policy 1, policy_version 176817 (0.0008) [2023-12-26 16:38:51,817][105620] Updated weights for policy 1, policy_version 176827 (0.0008) [2023-12-26 16:38:51,874][105620] Updated weights for policy 1, policy_version 176837 (0.0009) [2023-12-26 16:38:52,002][105692] Updated weights for policy 0, policy_version 175809 (0.0008) [2023-12-26 16:38:52,070][105692] Updated weights for policy 0, policy_version 175819 (0.0011) [2023-12-26 16:38:52,136][105692] Updated weights for policy 0, policy_version 175829 (0.0011) [2023-12-26 16:38:52,626][105620] Updated weights for policy 1, policy_version 176847 (0.0008) [2023-12-26 16:38:52,682][105620] Updated weights for policy 1, policy_version 176857 (0.0008) [2023-12-26 16:38:52,745][105620] Updated weights for policy 1, policy_version 176867 (0.0008) [2023-12-26 16:38:52,879][105692] Updated weights for policy 0, policy_version 175839 (0.0010) [2023-12-26 16:38:52,933][105692] Updated weights for policy 0, policy_version 175849 (0.0010) [2023-12-26 16:38:52,991][105692] Updated weights for policy 0, policy_version 175859 (0.0010) [2023-12-26 16:38:53,515][105620] Updated weights for policy 1, policy_version 176877 (0.0008) [2023-12-26 16:38:53,559][105620] Updated weights for policy 1, policy_version 176887 (0.0008) [2023-12-26 16:38:53,603][105620] Updated weights for policy 1, policy_version 176897 (0.0007) [2023-12-26 16:38:53,670][105692] Updated weights for policy 0, policy_version 175869 (0.0008) [2023-12-26 16:38:53,720][105692] Updated weights for policy 0, policy_version 175879 (0.0009) [2023-12-26 16:38:53,778][105692] Updated weights for policy 0, policy_version 175889 (0.0010) [2023-12-26 16:38:54,343][105620] Updated weights for policy 1, policy_version 176907 (0.0009) [2023-12-26 16:38:54,399][105620] Updated weights for policy 1, policy_version 176917 (0.0008) [2023-12-26 16:38:54,455][105620] Updated weights for policy 1, policy_version 176927 (0.0008) [2023-12-26 16:38:54,497][105692] Updated weights for policy 0, policy_version 175899 (0.0011) [2023-12-26 16:38:54,558][105692] Updated weights for policy 0, policy_version 175909 (0.0011) [2023-12-26 16:38:54,613][105692] Updated weights for policy 0, policy_version 175919 (0.0011) [2023-12-26 16:38:55,222][105620] Updated weights for policy 1, policy_version 176937 (0.0008) [2023-12-26 16:38:55,287][105620] Updated weights for policy 1, policy_version 176947 (0.0008) [2023-12-26 16:38:55,305][105692] Updated weights for policy 0, policy_version 175929 (0.0011) [2023-12-26 16:38:55,341][105620] Updated weights for policy 1, policy_version 176957 (0.0007) [2023-12-26 16:38:55,354][105692] Updated weights for policy 0, policy_version 175939 (0.0008) [2023-12-26 16:38:55,396][105620] Updated weights for policy 1, policy_version 176967 (0.0007) [2023-12-26 16:38:55,400][105692] Updated weights for policy 0, policy_version 175949 (0.0006) [2023-12-26 16:38:55,459][105692] Updated weights for policy 0, policy_version 175959 (0.0006) [2023-12-26 16:38:56,062][104569] Fps is (10 sec: 18022.0, 60 sec: 19660.7, 300 sec: 19633.0). Total num frames: 90365952. Throughput: 0: 9745.1, 1: 9817.7. Samples: 90378392. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:38:56,063][104569] Avg episode reward: [(0, '9258.787'), (1, '8916.610')] [2023-12-26 16:38:56,119][105692] Updated weights for policy 0, policy_version 175969 (0.0007) [2023-12-26 16:38:56,173][105692] Updated weights for policy 0, policy_version 175979 (0.0007) [2023-12-26 16:38:56,179][105620] Updated weights for policy 1, policy_version 176977 (0.0007) [2023-12-26 16:38:56,226][105620] Updated weights for policy 1, policy_version 176987 (0.0008) [2023-12-26 16:38:56,228][105692] Updated weights for policy 0, policy_version 175989 (0.0007) [2023-12-26 16:38:56,278][105620] Updated weights for policy 1, policy_version 176997 (0.0008) [2023-12-26 16:38:56,934][105692] Updated weights for policy 0, policy_version 175999 (0.0008) [2023-12-26 16:38:56,990][105692] Updated weights for policy 0, policy_version 176009 (0.0008) [2023-12-26 16:38:57,038][105620] Updated weights for policy 1, policy_version 177007 (0.0007) [2023-12-26 16:38:57,040][105692] Updated weights for policy 0, policy_version 176019 (0.0007) [2023-12-26 16:38:57,088][105620] Updated weights for policy 1, policy_version 177017 (0.0007) [2023-12-26 16:38:57,145][105620] Updated weights for policy 1, policy_version 177027 (0.0009) [2023-12-26 16:38:57,705][105692] Updated weights for policy 0, policy_version 176029 (0.0008) [2023-12-26 16:38:57,762][105692] Updated weights for policy 0, policy_version 176039 (0.0009) [2023-12-26 16:38:57,765][105620] Updated weights for policy 1, policy_version 177037 (0.0008) [2023-12-26 16:38:57,817][105692] Updated weights for policy 0, policy_version 176049 (0.0008) [2023-12-26 16:38:57,819][105620] Updated weights for policy 1, policy_version 177047 (0.0006) [2023-12-26 16:38:57,876][105620] Updated weights for policy 1, policy_version 177057 (0.0008) [2023-12-26 16:38:58,586][105692] Updated weights for policy 0, policy_version 176059 (0.0007) [2023-12-26 16:38:58,624][105620] Updated weights for policy 1, policy_version 177067 (0.0008) [2023-12-26 16:38:58,652][105692] Updated weights for policy 0, policy_version 176069 (0.0009) [2023-12-26 16:38:58,687][105620] Updated weights for policy 1, policy_version 177077 (0.0008) [2023-12-26 16:38:58,724][105692] Updated weights for policy 0, policy_version 176079 (0.0010) [2023-12-26 16:38:58,754][105620] Updated weights for policy 1, policy_version 177087 (0.0011) [2023-12-26 16:38:59,537][105692] Updated weights for policy 0, policy_version 176089 (0.0010) [2023-12-26 16:38:59,591][105692] Updated weights for policy 0, policy_version 176099 (0.0008) [2023-12-26 16:38:59,605][105620] Updated weights for policy 1, policy_version 177097 (0.0008) [2023-12-26 16:38:59,648][105692] Updated weights for policy 0, policy_version 176109 (0.0008) [2023-12-26 16:38:59,667][105620] Updated weights for policy 1, policy_version 177107 (0.0005) [2023-12-26 16:38:59,710][105692] Updated weights for policy 0, policy_version 176119 (0.0005) [2023-12-26 16:38:59,736][105620] Updated weights for policy 1, policy_version 177117 (0.0005) [2023-12-26 16:38:59,795][105620] Updated weights for policy 1, policy_version 177127 (0.0007) [2023-12-26 16:39:00,307][105692] Updated weights for policy 0, policy_version 176129 (0.0006) [2023-12-26 16:39:00,362][105692] Updated weights for policy 0, policy_version 176139 (0.0007) [2023-12-26 16:39:00,420][105620] Updated weights for policy 1, policy_version 177137 (0.0008) [2023-12-26 16:39:00,425][105692] Updated weights for policy 0, policy_version 176149 (0.0008) [2023-12-26 16:39:00,480][105620] Updated weights for policy 1, policy_version 177147 (0.0011) [2023-12-26 16:39:00,532][105620] Updated weights for policy 1, policy_version 177157 (0.0012) [2023-12-26 16:39:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 90464256. Throughput: 0: 9751.1, 1: 9845.8. Samples: 90437120. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:39:01,062][104569] Avg episode reward: [(0, '7900.906'), (1, '8745.900')] [2023-12-26 16:39:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000176152_45105152.pth... [2023-12-26 16:39:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000177160_45359104.pth... [2023-12-26 16:39:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000175032_44818432.pth [2023-12-26 16:39:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000176040_45072384.pth [2023-12-26 16:39:01,158][105692] Updated weights for policy 0, policy_version 176159 (0.0008) [2023-12-26 16:39:01,226][105692] Updated weights for policy 0, policy_version 176169 (0.0006) [2023-12-26 16:39:01,274][105620] Updated weights for policy 1, policy_version 177167 (0.0008) [2023-12-26 16:39:01,293][105692] Updated weights for policy 0, policy_version 176179 (0.0009) [2023-12-26 16:39:01,337][105620] Updated weights for policy 1, policy_version 177177 (0.0007) [2023-12-26 16:39:01,403][105620] Updated weights for policy 1, policy_version 177187 (0.0010) [2023-12-26 16:39:01,976][105692] Updated weights for policy 0, policy_version 176189 (0.0006) [2023-12-26 16:39:02,042][105692] Updated weights for policy 0, policy_version 176199 (0.0008) [2023-12-26 16:39:02,107][105692] Updated weights for policy 0, policy_version 176209 (0.0009) [2023-12-26 16:39:02,111][105620] Updated weights for policy 1, policy_version 177197 (0.0011) [2023-12-26 16:39:02,166][105620] Updated weights for policy 1, policy_version 177207 (0.0010) [2023-12-26 16:39:02,222][105620] Updated weights for policy 1, policy_version 177217 (0.0008) [2023-12-26 16:39:02,834][105692] Updated weights for policy 0, policy_version 176219 (0.0007) [2023-12-26 16:39:02,882][105692] Updated weights for policy 0, policy_version 176229 (0.0008) [2023-12-26 16:39:02,937][105692] Updated weights for policy 0, policy_version 176239 (0.0007) [2023-12-26 16:39:02,950][105620] Updated weights for policy 1, policy_version 177227 (0.0007) [2023-12-26 16:39:02,999][105620] Updated weights for policy 1, policy_version 177237 (0.0011) [2023-12-26 16:39:03,065][105620] Updated weights for policy 1, policy_version 177247 (0.0011) [2023-12-26 16:39:03,705][105692] Updated weights for policy 0, policy_version 176249 (0.0009) [2023-12-26 16:39:03,743][105620] Updated weights for policy 1, policy_version 177257 (0.0010) [2023-12-26 16:39:03,765][105692] Updated weights for policy 0, policy_version 176259 (0.0005) [2023-12-26 16:39:03,793][105620] Updated weights for policy 1, policy_version 177267 (0.0005) [2023-12-26 16:39:03,829][105692] Updated weights for policy 0, policy_version 176269 (0.0005) [2023-12-26 16:39:03,838][105620] Updated weights for policy 1, policy_version 177277 (0.0010) [2023-12-26 16:39:03,887][105692] Updated weights for policy 0, policy_version 176279 (0.0006) [2023-12-26 16:39:03,900][105620] Updated weights for policy 1, policy_version 177287 (0.0011) [2023-12-26 16:39:04,593][105692] Updated weights for policy 0, policy_version 176289 (0.0005) [2023-12-26 16:39:04,627][105620] Updated weights for policy 1, policy_version 177297 (0.0011) [2023-12-26 16:39:04,649][105692] Updated weights for policy 0, policy_version 176299 (0.0006) [2023-12-26 16:39:04,682][105620] Updated weights for policy 1, policy_version 177307 (0.0011) [2023-12-26 16:39:04,712][105692] Updated weights for policy 0, policy_version 176309 (0.0008) [2023-12-26 16:39:04,741][105620] Updated weights for policy 1, policy_version 177317 (0.0011) [2023-12-26 16:39:05,427][105692] Updated weights for policy 0, policy_version 176319 (0.0006) [2023-12-26 16:39:05,477][105692] Updated weights for policy 0, policy_version 176329 (0.0007) [2023-12-26 16:39:05,503][105620] Updated weights for policy 1, policy_version 177327 (0.0011) [2023-12-26 16:39:05,540][105692] Updated weights for policy 0, policy_version 176339 (0.0005) [2023-12-26 16:39:05,564][105620] Updated weights for policy 1, policy_version 177337 (0.0011) [2023-12-26 16:39:05,619][105620] Updated weights for policy 1, policy_version 177347 (0.0010) [2023-12-26 16:39:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.6, 300 sec: 19605.2). Total num frames: 90562560. Throughput: 0: 9670.2, 1: 9825.7. Samples: 90553100. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:39:06,063][104569] Avg episode reward: [(0, '1765.834'), (1, '9005.835')] [2023-12-26 16:39:06,254][105620] Updated weights for policy 1, policy_version 177357 (0.0011) [2023-12-26 16:39:06,311][105620] Updated weights for policy 1, policy_version 177367 (0.0011) [2023-12-26 16:39:06,320][105692] Updated weights for policy 0, policy_version 176349 (0.0007) [2023-12-26 16:39:06,370][105620] Updated weights for policy 1, policy_version 177377 (0.0011) [2023-12-26 16:39:06,376][105692] Updated weights for policy 0, policy_version 176359 (0.0006) [2023-12-26 16:39:06,441][105692] Updated weights for policy 0, policy_version 176369 (0.0008) [2023-12-26 16:39:07,024][105620] Updated weights for policy 1, policy_version 177387 (0.0009) [2023-12-26 16:39:07,091][105620] Updated weights for policy 1, policy_version 177397 (0.0009) [2023-12-26 16:39:07,157][105620] Updated weights for policy 1, policy_version 177407 (0.0011) [2023-12-26 16:39:07,251][105692] Updated weights for policy 0, policy_version 176379 (0.0008) [2023-12-26 16:39:07,304][105692] Updated weights for policy 0, policy_version 176389 (0.0008) [2023-12-26 16:39:07,353][105692] Updated weights for policy 0, policy_version 176399 (0.0008) [2023-12-26 16:39:07,855][105620] Updated weights for policy 1, policy_version 177417 (0.0010) [2023-12-26 16:39:07,917][105620] Updated weights for policy 1, policy_version 177427 (0.0010) [2023-12-26 16:39:07,971][105620] Updated weights for policy 1, policy_version 177437 (0.0010) [2023-12-26 16:39:08,035][105692] Updated weights for policy 0, policy_version 176409 (0.0008) [2023-12-26 16:39:08,037][105620] Updated weights for policy 1, policy_version 177447 (0.0010) [2023-12-26 16:39:08,091][105692] Updated weights for policy 0, policy_version 176419 (0.0008) [2023-12-26 16:39:08,144][105692] Updated weights for policy 0, policy_version 176429 (0.0009) [2023-12-26 16:39:08,198][105692] Updated weights for policy 0, policy_version 176440 (0.0010) [2023-12-26 16:39:08,684][105620] Updated weights for policy 1, policy_version 177457 (0.0011) [2023-12-26 16:39:08,749][105620] Updated weights for policy 1, policy_version 177467 (0.0011) [2023-12-26 16:39:08,808][105620] Updated weights for policy 1, policy_version 177477 (0.0010) [2023-12-26 16:39:09,034][105692] Updated weights for policy 0, policy_version 176450 (0.0008) [2023-12-26 16:39:09,089][105692] Updated weights for policy 0, policy_version 176460 (0.0008) [2023-12-26 16:39:09,140][105692] Updated weights for policy 0, policy_version 176470 (0.0008) [2023-12-26 16:39:09,553][105620] Updated weights for policy 1, policy_version 177487 (0.0011) [2023-12-26 16:39:09,617][105620] Updated weights for policy 1, policy_version 177497 (0.0011) [2023-12-26 16:39:09,674][105620] Updated weights for policy 1, policy_version 177507 (0.0011) [2023-12-26 16:39:09,957][105692] Updated weights for policy 0, policy_version 176480 (0.0008) [2023-12-26 16:39:10,021][105692] Updated weights for policy 0, policy_version 176490 (0.0008) [2023-12-26 16:39:10,076][105692] Updated weights for policy 0, policy_version 176500 (0.0009) [2023-12-26 16:39:10,356][105620] Updated weights for policy 1, policy_version 177517 (0.0008) [2023-12-26 16:39:10,417][105620] Updated weights for policy 1, policy_version 177527 (0.0007) [2023-12-26 16:39:10,484][105620] Updated weights for policy 1, policy_version 177537 (0.0011) [2023-12-26 16:39:10,915][105692] Updated weights for policy 0, policy_version 176510 (0.0009) [2023-12-26 16:39:10,970][105692] Updated weights for policy 0, policy_version 176520 (0.0010) [2023-12-26 16:39:11,036][105692] Updated weights for policy 0, policy_version 176530 (0.0010) [2023-12-26 16:39:11,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 90652672. Throughput: 0: 9573.1, 1: 9811.7. Samples: 90667348. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:39:11,063][104569] Avg episode reward: [(0, '3730.872'), (1, '9351.038')] [2023-12-26 16:39:11,153][105620] Updated weights for policy 1, policy_version 177547 (0.0010) [2023-12-26 16:39:11,216][105620] Updated weights for policy 1, policy_version 177557 (0.0007) [2023-12-26 16:39:11,281][105620] Updated weights for policy 1, policy_version 177567 (0.0009) [2023-12-26 16:39:11,813][105692] Updated weights for policy 0, policy_version 176540 (0.0009) [2023-12-26 16:39:11,879][105692] Updated weights for policy 0, policy_version 176550 (0.0009) [2023-12-26 16:39:11,934][105692] Updated weights for policy 0, policy_version 176560 (0.0009) [2023-12-26 16:39:12,037][105620] Updated weights for policy 1, policy_version 177577 (0.0008) [2023-12-26 16:39:12,092][105620] Updated weights for policy 1, policy_version 177587 (0.0009) [2023-12-26 16:39:12,154][105620] Updated weights for policy 1, policy_version 177597 (0.0008) [2023-12-26 16:39:12,217][105620] Updated weights for policy 1, policy_version 177607 (0.0008) [2023-12-26 16:39:12,627][105692] Updated weights for policy 0, policy_version 176570 (0.0009) [2023-12-26 16:39:12,679][105692] Updated weights for policy 0, policy_version 176580 (0.0010) [2023-12-26 16:39:12,736][105692] Updated weights for policy 0, policy_version 176590 (0.0011) [2023-12-26 16:39:12,785][105692] Updated weights for policy 0, policy_version 176600 (0.0010) [2023-12-26 16:39:12,974][105620] Updated weights for policy 1, policy_version 177617 (0.0005) [2023-12-26 16:39:13,043][105620] Updated weights for policy 1, policy_version 177627 (0.0005) [2023-12-26 16:39:13,107][105620] Updated weights for policy 1, policy_version 177637 (0.0007) [2023-12-26 16:39:13,528][105692] Updated weights for policy 0, policy_version 176610 (0.0010) [2023-12-26 16:39:13,587][105692] Updated weights for policy 0, policy_version 176620 (0.0010) [2023-12-26 16:39:13,645][105692] Updated weights for policy 0, policy_version 176630 (0.0010) [2023-12-26 16:39:13,704][105620] Updated weights for policy 1, policy_version 177647 (0.0008) [2023-12-26 16:39:13,761][105620] Updated weights for policy 1, policy_version 177657 (0.0007) [2023-12-26 16:39:13,821][105620] Updated weights for policy 1, policy_version 177667 (0.0005) [2023-12-26 16:39:14,244][105692] Updated weights for policy 0, policy_version 176640 (0.0010) [2023-12-26 16:39:14,302][105692] Updated weights for policy 0, policy_version 176650 (0.0010) [2023-12-26 16:39:14,369][105692] Updated weights for policy 0, policy_version 176660 (0.0010) [2023-12-26 16:39:14,452][105620] Updated weights for policy 1, policy_version 177677 (0.0008) [2023-12-26 16:39:14,504][105620] Updated weights for policy 1, policy_version 177687 (0.0010) [2023-12-26 16:39:14,559][105620] Updated weights for policy 1, policy_version 177697 (0.0010) [2023-12-26 16:39:15,043][105692] Updated weights for policy 0, policy_version 176670 (0.0010) [2023-12-26 16:39:15,105][105692] Updated weights for policy 0, policy_version 176680 (0.0009) [2023-12-26 16:39:15,156][105692] Updated weights for policy 0, policy_version 176690 (0.0008) [2023-12-26 16:39:15,311][105620] Updated weights for policy 1, policy_version 177707 (0.0010) [2023-12-26 16:39:15,367][105620] Updated weights for policy 1, policy_version 177717 (0.0011) [2023-12-26 16:39:15,427][105620] Updated weights for policy 1, policy_version 177727 (0.0011) [2023-12-26 16:39:15,947][105692] Updated weights for policy 0, policy_version 176700 (0.0009) [2023-12-26 16:39:16,001][105692] Updated weights for policy 0, policy_version 176710 (0.0010) [2023-12-26 16:39:16,058][105692] Updated weights for policy 0, policy_version 176720 (0.0010) [2023-12-26 16:39:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19605.2). Total num frames: 90750976. Throughput: 0: 9566.1, 1: 9758.4. Samples: 90725380. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:39:16,064][104569] Avg episode reward: [(0, '6713.646'), (1, '9350.831')] [2023-12-26 16:39:16,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000177736_45506560.pth... [2023-12-26 16:39:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000176616_45219840.pth [2023-12-26 16:39:16,094][105620] Updated weights for policy 1, policy_version 177737 (0.0010) [2023-12-26 16:39:16,109][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000176728_45252608.pth... [2023-12-26 16:39:16,114][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000175608_44965888.pth [2023-12-26 16:39:16,153][105620] Updated weights for policy 1, policy_version 177747 (0.0005) [2023-12-26 16:39:16,216][105620] Updated weights for policy 1, policy_version 177757 (0.0005) [2023-12-26 16:39:16,270][105620] Updated weights for policy 1, policy_version 177767 (0.0005) [2023-12-26 16:39:16,815][105620] Updated weights for policy 1, policy_version 177777 (0.0008) [2023-12-26 16:39:16,866][105692] Updated weights for policy 0, policy_version 176730 (0.0008) [2023-12-26 16:39:16,870][105620] Updated weights for policy 1, policy_version 177787 (0.0011) [2023-12-26 16:39:16,917][105692] Updated weights for policy 0, policy_version 176740 (0.0005) [2023-12-26 16:39:16,933][105620] Updated weights for policy 1, policy_version 177797 (0.0010) [2023-12-26 16:39:16,972][105692] Updated weights for policy 0, policy_version 176750 (0.0005) [2023-12-26 16:39:17,024][105692] Updated weights for policy 0, policy_version 176760 (0.0006) [2023-12-26 16:39:17,616][105692] Updated weights for policy 0, policy_version 176770 (0.0010) [2023-12-26 16:39:17,664][105620] Updated weights for policy 1, policy_version 177807 (0.0008) [2023-12-26 16:39:17,668][105692] Updated weights for policy 0, policy_version 176780 (0.0009) [2023-12-26 16:39:17,724][105620] Updated weights for policy 1, policy_version 177817 (0.0006) [2023-12-26 16:39:17,730][105692] Updated weights for policy 0, policy_version 176790 (0.0011) [2023-12-26 16:39:17,781][105620] Updated weights for policy 1, policy_version 177827 (0.0007) [2023-12-26 16:39:18,478][105692] Updated weights for policy 0, policy_version 176800 (0.0010) [2023-12-26 16:39:18,542][105692] Updated weights for policy 0, policy_version 176810 (0.0007) [2023-12-26 16:39:18,552][105620] Updated weights for policy 1, policy_version 177837 (0.0008) [2023-12-26 16:39:18,600][105692] Updated weights for policy 0, policy_version 176820 (0.0007) [2023-12-26 16:39:18,616][105620] Updated weights for policy 1, policy_version 177847 (0.0009) [2023-12-26 16:39:18,690][105620] Updated weights for policy 1, policy_version 177857 (0.0007) [2023-12-26 16:39:19,302][105692] Updated weights for policy 0, policy_version 176830 (0.0006) [2023-12-26 16:39:19,371][105692] Updated weights for policy 0, policy_version 176840 (0.0008) [2023-12-26 16:39:19,439][105692] Updated weights for policy 0, policy_version 176850 (0.0009) [2023-12-26 16:39:19,442][105620] Updated weights for policy 1, policy_version 177867 (0.0006) [2023-12-26 16:39:19,516][105620] Updated weights for policy 1, policy_version 177877 (0.0007) [2023-12-26 16:39:19,573][105620] Updated weights for policy 1, policy_version 177887 (0.0008) [2023-12-26 16:39:20,167][105692] Updated weights for policy 0, policy_version 176860 (0.0009) [2023-12-26 16:39:20,220][105692] Updated weights for policy 0, policy_version 176870 (0.0009) [2023-12-26 16:39:20,285][105692] Updated weights for policy 0, policy_version 176880 (0.0009) [2023-12-26 16:39:20,306][105620] Updated weights for policy 1, policy_version 177897 (0.0008) [2023-12-26 16:39:20,368][105620] Updated weights for policy 1, policy_version 177907 (0.0007) [2023-12-26 16:39:20,435][105620] Updated weights for policy 1, policy_version 177917 (0.0010) [2023-12-26 16:39:20,499][105620] Updated weights for policy 1, policy_version 177927 (0.0010) [2023-12-26 16:39:21,013][105692] Updated weights for policy 0, policy_version 176890 (0.0010) [2023-12-26 16:39:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 90849280. Throughput: 0: 9553.6, 1: 9671.8. Samples: 90843660. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:39:21,063][104569] Avg episode reward: [(0, '7840.476'), (1, '9350.642')] [2023-12-26 16:39:21,078][105692] Updated weights for policy 0, policy_version 176900 (0.0008) [2023-12-26 16:39:21,148][105692] Updated weights for policy 0, policy_version 176910 (0.0008) [2023-12-26 16:39:21,213][105692] Updated weights for policy 0, policy_version 176920 (0.0009) [2023-12-26 16:39:21,268][105620] Updated weights for policy 1, policy_version 177937 (0.0011) [2023-12-26 16:39:21,331][105620] Updated weights for policy 1, policy_version 177947 (0.0010) [2023-12-26 16:39:21,401][105620] Updated weights for policy 1, policy_version 177957 (0.0011) [2023-12-26 16:39:21,937][105692] Updated weights for policy 0, policy_version 176930 (0.0010) [2023-12-26 16:39:21,995][105692] Updated weights for policy 0, policy_version 176940 (0.0008) [2023-12-26 16:39:22,047][105692] Updated weights for policy 0, policy_version 176950 (0.0010) [2023-12-26 16:39:22,154][105620] Updated weights for policy 1, policy_version 177967 (0.0010) [2023-12-26 16:39:22,207][105620] Updated weights for policy 1, policy_version 177977 (0.0011) [2023-12-26 16:39:22,260][105620] Updated weights for policy 1, policy_version 177987 (0.0011) [2023-12-26 16:39:22,786][105692] Updated weights for policy 0, policy_version 176960 (0.0011) [2023-12-26 16:39:22,838][105692] Updated weights for policy 0, policy_version 176970 (0.0011) [2023-12-26 16:39:22,890][105692] Updated weights for policy 0, policy_version 176980 (0.0011) [2023-12-26 16:39:22,939][105620] Updated weights for policy 1, policy_version 177997 (0.0011) [2023-12-26 16:39:23,005][105620] Updated weights for policy 1, policy_version 178007 (0.0010) [2023-12-26 16:39:23,073][105620] Updated weights for policy 1, policy_version 178017 (0.0006) [2023-12-26 16:39:23,618][105692] Updated weights for policy 0, policy_version 176990 (0.0008) [2023-12-26 16:39:23,673][105692] Updated weights for policy 0, policy_version 177000 (0.0011) [2023-12-26 16:39:23,685][105620] Updated weights for policy 1, policy_version 178027 (0.0006) [2023-12-26 16:39:23,728][105692] Updated weights for policy 0, policy_version 177010 (0.0010) [2023-12-26 16:39:23,731][105620] Updated weights for policy 1, policy_version 178037 (0.0006) [2023-12-26 16:39:23,778][105620] Updated weights for policy 1, policy_version 178047 (0.0008) [2023-12-26 16:39:24,433][105692] Updated weights for policy 0, policy_version 177020 (0.0010) [2023-12-26 16:39:24,467][105620] Updated weights for policy 1, policy_version 178057 (0.0005) [2023-12-26 16:39:24,481][105692] Updated weights for policy 0, policy_version 177030 (0.0010) [2023-12-26 16:39:24,527][105620] Updated weights for policy 1, policy_version 178067 (0.0005) [2023-12-26 16:39:24,535][105692] Updated weights for policy 0, policy_version 177040 (0.0010) [2023-12-26 16:39:24,582][105620] Updated weights for policy 1, policy_version 178077 (0.0006) [2023-12-26 16:39:24,636][105620] Updated weights for policy 1, policy_version 178087 (0.0008) [2023-12-26 16:39:25,279][105692] Updated weights for policy 0, policy_version 177050 (0.0010) [2023-12-26 16:39:25,291][105620] Updated weights for policy 1, policy_version 178097 (0.0006) [2023-12-26 16:39:25,338][105692] Updated weights for policy 0, policy_version 177060 (0.0011) [2023-12-26 16:39:25,348][105620] Updated weights for policy 1, policy_version 178107 (0.0005) [2023-12-26 16:39:25,385][105692] Updated weights for policy 0, policy_version 177070 (0.0010) [2023-12-26 16:39:25,399][105620] Updated weights for policy 1, policy_version 178117 (0.0005) [2023-12-26 16:39:25,433][105692] Updated weights for policy 0, policy_version 177080 (0.0010) [2023-12-26 16:39:25,964][105620] Updated weights for policy 1, policy_version 178127 (0.0006) [2023-12-26 16:39:26,016][105620] Updated weights for policy 1, policy_version 178137 (0.0010) [2023-12-26 16:39:26,062][104569] Fps is (10 sec: 19661.7, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 90947584. Throughput: 0: 9585.8, 1: 9708.2. Samples: 90962080. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:39:26,063][104569] Avg episode reward: [(0, '8531.479'), (1, '9177.896')] [2023-12-26 16:39:26,071][105620] Updated weights for policy 1, policy_version 178147 (0.0010) [2023-12-26 16:39:26,147][105692] Updated weights for policy 0, policy_version 177090 (0.0005) [2023-12-26 16:39:26,210][105692] Updated weights for policy 0, policy_version 177100 (0.0005) [2023-12-26 16:39:26,268][105692] Updated weights for policy 0, policy_version 177110 (0.0006) [2023-12-26 16:39:26,742][105620] Updated weights for policy 1, policy_version 178157 (0.0011) [2023-12-26 16:39:26,789][105692] Updated weights for policy 0, policy_version 177120 (0.0005) [2023-12-26 16:39:26,800][105620] Updated weights for policy 1, policy_version 178167 (0.0010) [2023-12-26 16:39:26,840][105692] Updated weights for policy 0, policy_version 177130 (0.0005) [2023-12-26 16:39:26,849][105620] Updated weights for policy 1, policy_version 178177 (0.0010) [2023-12-26 16:39:26,893][105692] Updated weights for policy 0, policy_version 177140 (0.0005) [2023-12-26 16:39:27,410][105692] Updated weights for policy 0, policy_version 177150 (0.0008) [2023-12-26 16:39:27,454][105692] Updated weights for policy 0, policy_version 177160 (0.0010) [2023-12-26 16:39:27,497][105692] Updated weights for policy 0, policy_version 177170 (0.0010) [2023-12-26 16:39:27,597][105620] Updated weights for policy 1, policy_version 178187 (0.0011) [2023-12-26 16:39:27,658][105620] Updated weights for policy 1, policy_version 178198 (0.0010) [2023-12-26 16:39:27,714][105620] Updated weights for policy 1, policy_version 178208 (0.0009) [2023-12-26 16:39:28,079][105692] Updated weights for policy 0, policy_version 177180 (0.0006) [2023-12-26 16:39:28,125][105692] Updated weights for policy 0, policy_version 177190 (0.0005) [2023-12-26 16:39:28,171][105692] Updated weights for policy 0, policy_version 177200 (0.0005) [2023-12-26 16:39:28,406][105620] Updated weights for policy 1, policy_version 178218 (0.0010) [2023-12-26 16:39:28,468][105620] Updated weights for policy 1, policy_version 178228 (0.0010) [2023-12-26 16:39:28,529][105620] Updated weights for policy 1, policy_version 178238 (0.0010) [2023-12-26 16:39:28,587][105620] Updated weights for policy 1, policy_version 178248 (0.0010) [2023-12-26 16:39:28,771][105692] Updated weights for policy 0, policy_version 177210 (0.0005) [2023-12-26 16:39:28,818][105692] Updated weights for policy 0, policy_version 177220 (0.0005) [2023-12-26 16:39:28,869][105692] Updated weights for policy 0, policy_version 177230 (0.0005) [2023-12-26 16:39:28,923][105692] Updated weights for policy 0, policy_version 177240 (0.0005) [2023-12-26 16:39:29,273][105620] Updated weights for policy 1, policy_version 178258 (0.0008) [2023-12-26 16:39:29,336][105620] Updated weights for policy 1, policy_version 178268 (0.0007) [2023-12-26 16:39:29,402][105620] Updated weights for policy 1, policy_version 178278 (0.0008) [2023-12-26 16:39:29,499][105692] Updated weights for policy 0, policy_version 177250 (0.0009) [2023-12-26 16:39:29,570][105692] Updated weights for policy 0, policy_version 177260 (0.0010) [2023-12-26 16:39:29,625][105692] Updated weights for policy 0, policy_version 177270 (0.0010) [2023-12-26 16:39:30,131][105620] Updated weights for policy 1, policy_version 178288 (0.0009) [2023-12-26 16:39:30,190][105620] Updated weights for policy 1, policy_version 178298 (0.0005) [2023-12-26 16:39:30,252][105620] Updated weights for policy 1, policy_version 178308 (0.0005) [2023-12-26 16:39:30,347][105692] Updated weights for policy 0, policy_version 177280 (0.0010) [2023-12-26 16:39:30,405][105692] Updated weights for policy 0, policy_version 177290 (0.0010) [2023-12-26 16:39:30,460][105692] Updated weights for policy 0, policy_version 177300 (0.0010) [2023-12-26 16:39:30,893][105620] Updated weights for policy 1, policy_version 178318 (0.0005) [2023-12-26 16:39:30,960][105620] Updated weights for policy 1, policy_version 178328 (0.0005) [2023-12-26 16:39:31,021][105620] Updated weights for policy 1, policy_version 178338 (0.0007) [2023-12-26 16:39:31,062][104569] Fps is (10 sec: 21299.5, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 91062272. Throughput: 0: 9717.4, 1: 9758.9. Samples: 91027216. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:39:31,062][104569] Avg episode reward: [(0, '9257.574'), (1, '9178.530')] [2023-12-26 16:39:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000177304_45400064.pth... [2023-12-26 16:39:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000178344_45662208.pth... [2023-12-26 16:39:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000176152_45105152.pth [2023-12-26 16:39:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000177160_45359104.pth [2023-12-26 16:39:31,196][105692] Updated weights for policy 0, policy_version 177310 (0.0010) [2023-12-26 16:39:31,247][105692] Updated weights for policy 0, policy_version 177320 (0.0010) [2023-12-26 16:39:31,308][105692] Updated weights for policy 0, policy_version 177330 (0.0010) [2023-12-26 16:39:31,717][105620] Updated weights for policy 1, policy_version 178348 (0.0009) [2023-12-26 16:39:31,789][105620] Updated weights for policy 1, policy_version 178358 (0.0011) [2023-12-26 16:39:31,855][105620] Updated weights for policy 1, policy_version 178368 (0.0007) [2023-12-26 16:39:31,957][105692] Updated weights for policy 0, policy_version 177340 (0.0009) [2023-12-26 16:39:32,019][105692] Updated weights for policy 0, policy_version 177350 (0.0009) [2023-12-26 16:39:32,082][105692] Updated weights for policy 0, policy_version 177360 (0.0008) [2023-12-26 16:39:32,499][105620] Updated weights for policy 1, policy_version 178378 (0.0009) [2023-12-26 16:39:32,559][105620] Updated weights for policy 1, policy_version 178388 (0.0009) [2023-12-26 16:39:32,618][105620] Updated weights for policy 1, policy_version 178398 (0.0007) [2023-12-26 16:39:32,678][105620] Updated weights for policy 1, policy_version 178408 (0.0006) [2023-12-26 16:39:32,749][105692] Updated weights for policy 0, policy_version 177370 (0.0011) [2023-12-26 16:39:32,797][105692] Updated weights for policy 0, policy_version 177380 (0.0010) [2023-12-26 16:39:32,849][105692] Updated weights for policy 0, policy_version 177390 (0.0010) [2023-12-26 16:39:32,902][105692] Updated weights for policy 0, policy_version 177400 (0.0006) [2023-12-26 16:39:33,291][105620] Updated weights for policy 1, policy_version 178418 (0.0005) [2023-12-26 16:39:33,342][105620] Updated weights for policy 1, policy_version 178428 (0.0008) [2023-12-26 16:39:33,389][105620] Updated weights for policy 1, policy_version 178438 (0.0010) [2023-12-26 16:39:33,655][105692] Updated weights for policy 0, policy_version 177410 (0.0010) [2023-12-26 16:39:33,702][105692] Updated weights for policy 0, policy_version 177420 (0.0010) [2023-12-26 16:39:33,746][105692] Updated weights for policy 0, policy_version 177430 (0.0008) [2023-12-26 16:39:34,088][105620] Updated weights for policy 1, policy_version 178448 (0.0006) [2023-12-26 16:39:34,143][105620] Updated weights for policy 1, policy_version 178458 (0.0006) [2023-12-26 16:39:34,208][105620] Updated weights for policy 1, policy_version 178468 (0.0007) [2023-12-26 16:39:34,519][105692] Updated weights for policy 0, policy_version 177440 (0.0006) [2023-12-26 16:39:34,575][105692] Updated weights for policy 0, policy_version 177450 (0.0005) [2023-12-26 16:39:34,629][105692] Updated weights for policy 0, policy_version 177460 (0.0006) [2023-12-26 16:39:34,885][105620] Updated weights for policy 1, policy_version 178478 (0.0008) [2023-12-26 16:39:34,949][105620] Updated weights for policy 1, policy_version 178488 (0.0007) [2023-12-26 16:39:35,019][105620] Updated weights for policy 1, policy_version 178498 (0.0009) [2023-12-26 16:39:35,256][105692] Updated weights for policy 0, policy_version 177470 (0.0007) [2023-12-26 16:39:35,326][105692] Updated weights for policy 0, policy_version 177480 (0.0005) [2023-12-26 16:39:35,398][105692] Updated weights for policy 0, policy_version 177490 (0.0005) [2023-12-26 16:39:35,674][105620] Updated weights for policy 1, policy_version 178508 (0.0011) [2023-12-26 16:39:35,726][105620] Updated weights for policy 1, policy_version 178518 (0.0010) [2023-12-26 16:39:35,780][105620] Updated weights for policy 1, policy_version 178528 (0.0010) [2023-12-26 16:39:36,035][105692] Updated weights for policy 0, policy_version 177500 (0.0006) [2023-12-26 16:39:36,062][104569] Fps is (10 sec: 21298.9, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 91160576. Throughput: 0: 9786.0, 1: 9880.1. Samples: 91149608. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:39:36,063][104569] Avg episode reward: [(0, '9257.891'), (1, '3334.833')] [2023-12-26 16:39:36,090][105692] Updated weights for policy 0, policy_version 177510 (0.0008) [2023-12-26 16:39:36,159][105692] Updated weights for policy 0, policy_version 177520 (0.0010) [2023-12-26 16:39:36,508][105620] Updated weights for policy 1, policy_version 178538 (0.0010) [2023-12-26 16:39:36,559][105620] Updated weights for policy 1, policy_version 178548 (0.0009) [2023-12-26 16:39:36,609][105620] Updated weights for policy 1, policy_version 178558 (0.0010) [2023-12-26 16:39:36,676][105620] Updated weights for policy 1, policy_version 178568 (0.0010) [2023-12-26 16:39:36,834][105692] Updated weights for policy 0, policy_version 177530 (0.0010) [2023-12-26 16:39:36,896][105692] Updated weights for policy 0, policy_version 177540 (0.0009) [2023-12-26 16:39:36,943][105692] Updated weights for policy 0, policy_version 177550 (0.0009) [2023-12-26 16:39:36,990][105692] Updated weights for policy 0, policy_version 177560 (0.0009) [2023-12-26 16:39:37,492][105620] Updated weights for policy 1, policy_version 178578 (0.0008) [2023-12-26 16:39:37,540][105620] Updated weights for policy 1, policy_version 178588 (0.0008) [2023-12-26 16:39:37,587][105620] Updated weights for policy 1, policy_version 178598 (0.0009) [2023-12-26 16:39:37,706][105692] Updated weights for policy 0, policy_version 177570 (0.0008) [2023-12-26 16:39:37,772][105692] Updated weights for policy 0, policy_version 177580 (0.0008) [2023-12-26 16:39:37,830][105692] Updated weights for policy 0, policy_version 177590 (0.0009) [2023-12-26 16:39:38,324][105620] Updated weights for policy 1, policy_version 178608 (0.0010) [2023-12-26 16:39:38,383][105620] Updated weights for policy 1, policy_version 178618 (0.0009) [2023-12-26 16:39:38,442][105620] Updated weights for policy 1, policy_version 178628 (0.0009) [2023-12-26 16:39:38,563][105692] Updated weights for policy 0, policy_version 177600 (0.0009) [2023-12-26 16:39:38,617][105692] Updated weights for policy 0, policy_version 177610 (0.0009) [2023-12-26 16:39:38,675][105692] Updated weights for policy 0, policy_version 177620 (0.0009) [2023-12-26 16:39:39,285][105692] Updated weights for policy 0, policy_version 177630 (0.0007) [2023-12-26 16:39:39,296][105620] Updated weights for policy 1, policy_version 178638 (0.0007) [2023-12-26 16:39:39,357][105692] Updated weights for policy 0, policy_version 177640 (0.0008) [2023-12-26 16:39:39,361][105620] Updated weights for policy 1, policy_version 178648 (0.0008) [2023-12-26 16:39:39,423][105692] Updated weights for policy 0, policy_version 177650 (0.0008) [2023-12-26 16:39:39,424][105620] Updated weights for policy 1, policy_version 178658 (0.0008) [2023-12-26 16:39:40,068][105692] Updated weights for policy 0, policy_version 177660 (0.0008) [2023-12-26 16:39:40,135][105692] Updated weights for policy 0, policy_version 177670 (0.0009) [2023-12-26 16:39:40,167][105620] Updated weights for policy 1, policy_version 178668 (0.0007) [2023-12-26 16:39:40,197][105692] Updated weights for policy 0, policy_version 177680 (0.0009) [2023-12-26 16:39:40,224][105620] Updated weights for policy 1, policy_version 178678 (0.0007) [2023-12-26 16:39:40,283][105620] Updated weights for policy 1, policy_version 178688 (0.0008) [2023-12-26 16:39:40,956][105692] Updated weights for policy 0, policy_version 177690 (0.0007) [2023-12-26 16:39:41,016][105692] Updated weights for policy 0, policy_version 177700 (0.0009) [2023-12-26 16:39:41,034][105620] Updated weights for policy 1, policy_version 178698 (0.0008) [2023-12-26 16:39:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 91250688. Throughput: 0: 9853.2, 1: 9861.6. Samples: 91265552. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:39:41,062][104569] Avg episode reward: [(0, '9260.426'), (1, '2255.388')] [2023-12-26 16:39:41,087][105692] Updated weights for policy 0, policy_version 177710 (0.0008) [2023-12-26 16:39:41,106][105620] Updated weights for policy 1, policy_version 178708 (0.0007) [2023-12-26 16:39:41,155][105692] Updated weights for policy 0, policy_version 177720 (0.0008) [2023-12-26 16:39:41,175][105620] Updated weights for policy 1, policy_version 178718 (0.0007) [2023-12-26 16:39:41,234][105620] Updated weights for policy 1, policy_version 178728 (0.0009) [2023-12-26 16:39:41,940][105620] Updated weights for policy 1, policy_version 178738 (0.0007) [2023-12-26 16:39:41,954][105692] Updated weights for policy 0, policy_version 177730 (0.0005) [2023-12-26 16:39:42,001][105620] Updated weights for policy 1, policy_version 178749 (0.0007) [2023-12-26 16:39:42,011][105692] Updated weights for policy 0, policy_version 177740 (0.0007) [2023-12-26 16:39:42,061][105620] Updated weights for policy 1, policy_version 178759 (0.0006) [2023-12-26 16:39:42,075][105692] Updated weights for policy 0, policy_version 177750 (0.0007) [2023-12-26 16:39:42,787][105620] Updated weights for policy 1, policy_version 178769 (0.0008) [2023-12-26 16:39:42,840][105692] Updated weights for policy 0, policy_version 177760 (0.0006) [2023-12-26 16:39:42,849][105620] Updated weights for policy 1, policy_version 178779 (0.0008) [2023-12-26 16:39:42,900][105692] Updated weights for policy 0, policy_version 177770 (0.0007) [2023-12-26 16:39:42,910][105620] Updated weights for policy 1, policy_version 178789 (0.0008) [2023-12-26 16:39:42,961][105692] Updated weights for policy 0, policy_version 177780 (0.0008) [2023-12-26 16:39:43,576][105692] Updated weights for policy 0, policy_version 177790 (0.0007) [2023-12-26 16:39:43,636][105620] Updated weights for policy 1, policy_version 178799 (0.0006) [2023-12-26 16:39:43,637][105692] Updated weights for policy 0, policy_version 177800 (0.0005) [2023-12-26 16:39:43,693][105620] Updated weights for policy 1, policy_version 178809 (0.0005) [2023-12-26 16:39:43,696][105692] Updated weights for policy 0, policy_version 177810 (0.0005) [2023-12-26 16:39:43,758][105620] Updated weights for policy 1, policy_version 178819 (0.0005) [2023-12-26 16:39:44,195][105692] Updated weights for policy 0, policy_version 177820 (0.0006) [2023-12-26 16:39:44,261][105692] Updated weights for policy 0, policy_version 177830 (0.0009) [2023-12-26 16:39:44,322][105692] Updated weights for policy 0, policy_version 177840 (0.0008) [2023-12-26 16:39:44,328][105620] Updated weights for policy 1, policy_version 178829 (0.0006) [2023-12-26 16:39:44,378][105620] Updated weights for policy 1, policy_version 178839 (0.0007) [2023-12-26 16:39:44,428][105620] Updated weights for policy 1, policy_version 178849 (0.0009) [2023-12-26 16:39:45,107][105692] Updated weights for policy 0, policy_version 177850 (0.0007) [2023-12-26 16:39:45,170][105692] Updated weights for policy 0, policy_version 177860 (0.0009) [2023-12-26 16:39:45,212][105620] Updated weights for policy 1, policy_version 178859 (0.0008) [2023-12-26 16:39:45,235][105692] Updated weights for policy 0, policy_version 177870 (0.0008) [2023-12-26 16:39:45,269][105620] Updated weights for policy 1, policy_version 178869 (0.0008) [2023-12-26 16:39:45,297][105692] Updated weights for policy 0, policy_version 177880 (0.0006) [2023-12-26 16:39:45,322][105620] Updated weights for policy 1, policy_version 178879 (0.0008) [2023-12-26 16:39:45,972][105620] Updated weights for policy 1, policy_version 178889 (0.0009) [2023-12-26 16:39:46,036][105620] Updated weights for policy 1, policy_version 178899 (0.0005) [2023-12-26 16:39:46,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 91348992. Throughput: 0: 9832.3, 1: 9874.2. Samples: 91323912. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:39:46,062][104569] Avg episode reward: [(0, '9348.062'), (1, '4660.434')] [2023-12-26 16:39:46,099][105620] Updated weights for policy 1, policy_version 178909 (0.0006) [2023-12-26 16:39:46,128][105692] Updated weights for policy 0, policy_version 177890 (0.0009) [2023-12-26 16:39:46,152][105620] Updated weights for policy 1, policy_version 178919 (0.0005) [2023-12-26 16:39:46,155][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000178920_45809664.pth... [2023-12-26 16:39:46,159][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000177736_45506560.pth [2023-12-26 16:39:46,188][105692] Updated weights for policy 0, policy_version 177900 (0.0010) [2023-12-26 16:39:46,242][105692] Updated weights for policy 0, policy_version 177911 (0.0010) [2023-12-26 16:39:46,244][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000177912_45555712.pth... [2023-12-26 16:39:46,247][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000176728_45252608.pth [2023-12-26 16:39:46,775][105620] Updated weights for policy 1, policy_version 178929 (0.0008) [2023-12-26 16:39:46,829][105620] Updated weights for policy 1, policy_version 178939 (0.0008) [2023-12-26 16:39:46,886][105620] Updated weights for policy 1, policy_version 178949 (0.0008) [2023-12-26 16:39:46,997][105692] Updated weights for policy 0, policy_version 177921 (0.0006) [2023-12-26 16:39:47,065][105692] Updated weights for policy 0, policy_version 177931 (0.0009) [2023-12-26 16:39:47,136][105692] Updated weights for policy 0, policy_version 177941 (0.0006) [2023-12-26 16:39:47,448][105620] Updated weights for policy 1, policy_version 178959 (0.0005) [2023-12-26 16:39:47,506][105620] Updated weights for policy 1, policy_version 178969 (0.0005) [2023-12-26 16:39:47,565][105620] Updated weights for policy 1, policy_version 178979 (0.0005) [2023-12-26 16:39:47,763][105692] Updated weights for policy 0, policy_version 177951 (0.0009) [2023-12-26 16:39:47,819][105692] Updated weights for policy 0, policy_version 177962 (0.0010) [2023-12-26 16:39:47,871][105692] Updated weights for policy 0, policy_version 177972 (0.0009) [2023-12-26 16:39:48,138][105620] Updated weights for policy 1, policy_version 178989 (0.0005) [2023-12-26 16:39:48,192][105620] Updated weights for policy 1, policy_version 178999 (0.0007) [2023-12-26 16:39:48,240][105620] Updated weights for policy 1, policy_version 179009 (0.0009) [2023-12-26 16:39:48,556][105692] Updated weights for policy 0, policy_version 177982 (0.0008) [2023-12-26 16:39:48,622][105692] Updated weights for policy 0, policy_version 177992 (0.0008) [2023-12-26 16:39:48,682][105692] Updated weights for policy 0, policy_version 178002 (0.0009) [2023-12-26 16:39:48,944][105620] Updated weights for policy 1, policy_version 179019 (0.0008) [2023-12-26 16:39:49,012][105620] Updated weights for policy 1, policy_version 179029 (0.0011) [2023-12-26 16:39:49,075][105620] Updated weights for policy 1, policy_version 179039 (0.0006) [2023-12-26 16:39:49,349][105692] Updated weights for policy 0, policy_version 178012 (0.0009) [2023-12-26 16:39:49,416][105692] Updated weights for policy 0, policy_version 178022 (0.0008) [2023-12-26 16:39:49,475][105692] Updated weights for policy 0, policy_version 178032 (0.0008) [2023-12-26 16:39:49,721][105620] Updated weights for policy 1, policy_version 179049 (0.0006) [2023-12-26 16:39:49,788][105620] Updated weights for policy 1, policy_version 179059 (0.0011) [2023-12-26 16:39:49,856][105620] Updated weights for policy 1, policy_version 179069 (0.0011) [2023-12-26 16:39:49,920][105620] Updated weights for policy 1, policy_version 179079 (0.0007) [2023-12-26 16:39:50,278][105692] Updated weights for policy 0, policy_version 178042 (0.0008) [2023-12-26 16:39:50,341][105692] Updated weights for policy 0, policy_version 178052 (0.0008) [2023-12-26 16:39:50,394][105692] Updated weights for policy 0, policy_version 178062 (0.0008) [2023-12-26 16:39:50,450][105692] Updated weights for policy 0, policy_version 178072 (0.0008) [2023-12-26 16:39:50,619][105620] Updated weights for policy 1, policy_version 179089 (0.0007) [2023-12-26 16:39:50,678][105620] Updated weights for policy 1, policy_version 179099 (0.0006) [2023-12-26 16:39:50,740][105620] Updated weights for policy 1, policy_version 179109 (0.0006) [2023-12-26 16:39:51,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 91455488. Throughput: 0: 9857.1, 1: 9988.1. Samples: 91446124. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:39:51,062][104569] Avg episode reward: [(0, '9348.178'), (1, '7561.213')] [2023-12-26 16:39:51,268][105692] Updated weights for policy 0, policy_version 178082 (0.0008) [2023-12-26 16:39:51,321][105692] Updated weights for policy 0, policy_version 178092 (0.0008) [2023-12-26 16:39:51,384][105692] Updated weights for policy 0, policy_version 178102 (0.0009) [2023-12-26 16:39:51,441][105620] Updated weights for policy 1, policy_version 179119 (0.0011) [2023-12-26 16:39:51,502][105620] Updated weights for policy 1, policy_version 179129 (0.0011) [2023-12-26 16:39:51,565][105620] Updated weights for policy 1, policy_version 179139 (0.0010) [2023-12-26 16:39:52,168][105692] Updated weights for policy 0, policy_version 178112 (0.0009) [2023-12-26 16:39:52,230][105692] Updated weights for policy 0, policy_version 178122 (0.0009) [2023-12-26 16:39:52,291][105692] Updated weights for policy 0, policy_version 178132 (0.0009) [2023-12-26 16:39:52,319][105620] Updated weights for policy 1, policy_version 179149 (0.0008) [2023-12-26 16:39:52,384][105620] Updated weights for policy 1, policy_version 179159 (0.0008) [2023-12-26 16:39:52,438][105620] Updated weights for policy 1, policy_version 179169 (0.0010) [2023-12-26 16:39:52,946][105692] Updated weights for policy 0, policy_version 178142 (0.0009) [2023-12-26 16:39:53,001][105692] Updated weights for policy 0, policy_version 178152 (0.0008) [2023-12-26 16:39:53,053][105692] Updated weights for policy 0, policy_version 178162 (0.0008) [2023-12-26 16:39:53,141][105620] Updated weights for policy 1, policy_version 179179 (0.0011) [2023-12-26 16:39:53,189][105620] Updated weights for policy 1, policy_version 179189 (0.0008) [2023-12-26 16:39:53,235][105620] Updated weights for policy 1, policy_version 179199 (0.0005) [2023-12-26 16:39:53,705][105692] Updated weights for policy 0, policy_version 178172 (0.0008) [2023-12-26 16:39:53,753][105692] Updated weights for policy 0, policy_version 178182 (0.0009) [2023-12-26 16:39:53,804][105692] Updated weights for policy 0, policy_version 178192 (0.0009) [2023-12-26 16:39:54,044][105620] Updated weights for policy 1, policy_version 179209 (0.0010) [2023-12-26 16:39:54,103][105620] Updated weights for policy 1, policy_version 179219 (0.0010) [2023-12-26 16:39:54,161][105620] Updated weights for policy 1, policy_version 179229 (0.0010) [2023-12-26 16:39:54,209][105620] Updated weights for policy 1, policy_version 179239 (0.0010) [2023-12-26 16:39:54,588][105692] Updated weights for policy 0, policy_version 178202 (0.0009) [2023-12-26 16:39:54,641][105692] Updated weights for policy 0, policy_version 178212 (0.0009) [2023-12-26 16:39:54,689][105692] Updated weights for policy 0, policy_version 178222 (0.0009) [2023-12-26 16:39:54,736][105692] Updated weights for policy 0, policy_version 178232 (0.0009) [2023-12-26 16:39:54,884][105620] Updated weights for policy 1, policy_version 179249 (0.0009) [2023-12-26 16:39:54,930][105620] Updated weights for policy 1, policy_version 179259 (0.0008) [2023-12-26 16:39:54,990][105620] Updated weights for policy 1, policy_version 179269 (0.0009) [2023-12-26 16:39:55,515][105692] Updated weights for policy 0, policy_version 178242 (0.0009) [2023-12-26 16:39:55,567][105692] Updated weights for policy 0, policy_version 178252 (0.0009) [2023-12-26 16:39:55,620][105692] Updated weights for policy 0, policy_version 178262 (0.0009) [2023-12-26 16:39:55,755][105620] Updated weights for policy 1, policy_version 179279 (0.0010) [2023-12-26 16:39:55,803][105620] Updated weights for policy 1, policy_version 179289 (0.0008) [2023-12-26 16:39:55,856][105620] Updated weights for policy 1, policy_version 179299 (0.0009) [2023-12-26 16:39:56,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 91553792. Throughput: 0: 9915.7, 1: 9922.5. Samples: 91560068. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:39:56,063][104569] Avg episode reward: [(0, '9257.039'), (1, '9085.494')] [2023-12-26 16:39:56,295][105692] Updated weights for policy 0, policy_version 178272 (0.0006) [2023-12-26 16:39:56,353][105692] Updated weights for policy 0, policy_version 178282 (0.0010) [2023-12-26 16:39:56,407][105692] Updated weights for policy 0, policy_version 178293 (0.0010) [2023-12-26 16:39:56,626][105620] Updated weights for policy 1, policy_version 179309 (0.0009) [2023-12-26 16:39:56,679][105620] Updated weights for policy 1, policy_version 179319 (0.0008) [2023-12-26 16:39:56,732][105620] Updated weights for policy 1, policy_version 179329 (0.0009) [2023-12-26 16:39:57,154][105692] Updated weights for policy 0, policy_version 178304 (0.0009) [2023-12-26 16:39:57,200][105692] Updated weights for policy 0, policy_version 178314 (0.0008) [2023-12-26 16:39:57,253][105692] Updated weights for policy 0, policy_version 178324 (0.0009) [2023-12-26 16:39:57,473][105620] Updated weights for policy 1, policy_version 179339 (0.0008) [2023-12-26 16:39:57,522][105620] Updated weights for policy 1, policy_version 179349 (0.0008) [2023-12-26 16:39:57,566][105620] Updated weights for policy 1, policy_version 179359 (0.0006) [2023-12-26 16:39:57,935][105692] Updated weights for policy 0, policy_version 178334 (0.0007) [2023-12-26 16:39:57,986][105692] Updated weights for policy 0, policy_version 178344 (0.0008) [2023-12-26 16:39:58,042][105692] Updated weights for policy 0, policy_version 178354 (0.0009) [2023-12-26 16:39:58,208][105620] Updated weights for policy 1, policy_version 179369 (0.0006) [2023-12-26 16:39:58,269][105620] Updated weights for policy 1, policy_version 179379 (0.0008) [2023-12-26 16:39:58,342][105620] Updated weights for policy 1, policy_version 179389 (0.0008) [2023-12-26 16:39:58,413][105620] Updated weights for policy 1, policy_version 179399 (0.0008) [2023-12-26 16:39:58,880][105692] Updated weights for policy 0, policy_version 178364 (0.0009) [2023-12-26 16:39:58,949][105692] Updated weights for policy 0, policy_version 178374 (0.0009) [2023-12-26 16:39:59,015][105692] Updated weights for policy 0, policy_version 178384 (0.0007) [2023-12-26 16:39:59,183][105620] Updated weights for policy 1, policy_version 179409 (0.0011) [2023-12-26 16:39:59,250][105620] Updated weights for policy 1, policy_version 179419 (0.0009) [2023-12-26 16:39:59,311][105620] Updated weights for policy 1, policy_version 179429 (0.0009) [2023-12-26 16:39:59,730][105692] Updated weights for policy 0, policy_version 178394 (0.0008) [2023-12-26 16:39:59,783][105692] Updated weights for policy 0, policy_version 178404 (0.0010) [2023-12-26 16:39:59,847][105692] Updated weights for policy 0, policy_version 178414 (0.0010) [2023-12-26 16:39:59,905][105692] Updated weights for policy 0, policy_version 178424 (0.0011) [2023-12-26 16:39:59,999][105620] Updated weights for policy 1, policy_version 179439 (0.0010) [2023-12-26 16:40:00,048][105620] Updated weights for policy 1, policy_version 179449 (0.0010) [2023-12-26 16:40:00,108][105620] Updated weights for policy 1, policy_version 179459 (0.0010) [2023-12-26 16:40:00,694][105692] Updated weights for policy 0, policy_version 178434 (0.0010) [2023-12-26 16:40:00,742][105692] Updated weights for policy 0, policy_version 178444 (0.0010) [2023-12-26 16:40:00,761][105620] Updated weights for policy 1, policy_version 179469 (0.0008) [2023-12-26 16:40:00,787][105692] Updated weights for policy 0, policy_version 178454 (0.0010) [2023-12-26 16:40:00,821][105620] Updated weights for policy 1, policy_version 179479 (0.0007) [2023-12-26 16:40:00,868][105620] Updated weights for policy 1, policy_version 179489 (0.0010) [2023-12-26 16:40:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 91652096. Throughput: 0: 9951.4, 1: 9907.0. Samples: 91618996. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:40:01,062][104569] Avg episode reward: [(0, '9166.772'), (1, '8914.624')] [2023-12-26 16:40:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000179496_45957120.pth... [2023-12-26 16:40:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000178456_45694976.pth... [2023-12-26 16:40:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000178344_45662208.pth [2023-12-26 16:40:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000177304_45400064.pth [2023-12-26 16:40:01,538][105692] Updated weights for policy 0, policy_version 178464 (0.0009) [2023-12-26 16:40:01,586][105692] Updated weights for policy 0, policy_version 178474 (0.0008) [2023-12-26 16:40:01,611][105620] Updated weights for policy 1, policy_version 179499 (0.0009) [2023-12-26 16:40:01,651][105692] Updated weights for policy 0, policy_version 178484 (0.0009) [2023-12-26 16:40:01,674][105620] Updated weights for policy 1, policy_version 179509 (0.0008) [2023-12-26 16:40:01,742][105620] Updated weights for policy 1, policy_version 179519 (0.0009) [2023-12-26 16:40:02,321][105692] Updated weights for policy 0, policy_version 178494 (0.0008) [2023-12-26 16:40:02,379][105692] Updated weights for policy 0, policy_version 178504 (0.0009) [2023-12-26 16:40:02,435][105692] Updated weights for policy 0, policy_version 178514 (0.0008) [2023-12-26 16:40:02,514][105620] Updated weights for policy 1, policy_version 179529 (0.0009) [2023-12-26 16:40:02,576][105620] Updated weights for policy 1, policy_version 179539 (0.0009) [2023-12-26 16:40:02,637][105620] Updated weights for policy 1, policy_version 179549 (0.0010) [2023-12-26 16:40:02,692][105620] Updated weights for policy 1, policy_version 179559 (0.0009) [2023-12-26 16:40:03,179][105692] Updated weights for policy 0, policy_version 178524 (0.0009) [2023-12-26 16:40:03,234][105692] Updated weights for policy 0, policy_version 178534 (0.0008) [2023-12-26 16:40:03,279][105692] Updated weights for policy 0, policy_version 178544 (0.0008) [2023-12-26 16:40:03,459][105620] Updated weights for policy 1, policy_version 179569 (0.0009) [2023-12-26 16:40:03,513][105620] Updated weights for policy 1, policy_version 179580 (0.0009) [2023-12-26 16:40:03,570][105620] Updated weights for policy 1, policy_version 179590 (0.0009) [2023-12-26 16:40:03,912][105692] Updated weights for policy 0, policy_version 178554 (0.0008) [2023-12-26 16:40:03,972][105692] Updated weights for policy 0, policy_version 178564 (0.0006) [2023-12-26 16:40:04,043][105692] Updated weights for policy 0, policy_version 178574 (0.0005) [2023-12-26 16:40:04,106][105692] Updated weights for policy 0, policy_version 178584 (0.0007) [2023-12-26 16:40:04,329][105620] Updated weights for policy 1, policy_version 179600 (0.0011) [2023-12-26 16:40:04,392][105620] Updated weights for policy 1, policy_version 179610 (0.0011) [2023-12-26 16:40:04,455][105620] Updated weights for policy 1, policy_version 179620 (0.0011) [2023-12-26 16:40:04,746][105692] Updated weights for policy 0, policy_version 178594 (0.0008) [2023-12-26 16:40:04,801][105692] Updated weights for policy 0, policy_version 178604 (0.0007) [2023-12-26 16:40:04,849][105692] Updated weights for policy 0, policy_version 178614 (0.0005) [2023-12-26 16:40:05,138][105620] Updated weights for policy 1, policy_version 179630 (0.0011) [2023-12-26 16:40:05,184][105620] Updated weights for policy 1, policy_version 179640 (0.0010) [2023-12-26 16:40:05,250][105620] Updated weights for policy 1, policy_version 179650 (0.0011) [2023-12-26 16:40:05,455][105692] Updated weights for policy 0, policy_version 178624 (0.0009) [2023-12-26 16:40:05,513][105692] Updated weights for policy 0, policy_version 178634 (0.0006) [2023-12-26 16:40:05,575][105692] Updated weights for policy 0, policy_version 178644 (0.0009) [2023-12-26 16:40:06,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 91742208. Throughput: 0: 9937.4, 1: 9867.9. Samples: 91734904. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:40:06,063][104569] Avg episode reward: [(0, '8987.996'), (1, '9179.593')] [2023-12-26 16:40:06,068][105620] Updated weights for policy 1, policy_version 179660 (0.0011) [2023-12-26 16:40:06,134][105692] Updated weights for policy 0, policy_version 178654 (0.0009) [2023-12-26 16:40:06,134][105620] Updated weights for policy 1, policy_version 179670 (0.0011) [2023-12-26 16:40:06,191][105692] Updated weights for policy 0, policy_version 178664 (0.0010) [2023-12-26 16:40:06,196][105620] Updated weights for policy 1, policy_version 179680 (0.0011) [2023-12-26 16:40:06,254][105692] Updated weights for policy 0, policy_version 178674 (0.0010) [2023-12-26 16:40:06,881][105620] Updated weights for policy 1, policy_version 179690 (0.0009) [2023-12-26 16:40:06,936][105620] Updated weights for policy 1, policy_version 179700 (0.0005) [2023-12-26 16:40:06,989][105620] Updated weights for policy 1, policy_version 179710 (0.0009) [2023-12-26 16:40:06,992][105692] Updated weights for policy 0, policy_version 178684 (0.0010) [2023-12-26 16:40:07,033][105620] Updated weights for policy 1, policy_version 179720 (0.0005) [2023-12-26 16:40:07,048][105692] Updated weights for policy 0, policy_version 178694 (0.0010) [2023-12-26 16:40:07,111][105692] Updated weights for policy 0, policy_version 178704 (0.0008) [2023-12-26 16:40:07,583][105620] Updated weights for policy 1, policy_version 179730 (0.0010) [2023-12-26 16:40:07,641][105620] Updated weights for policy 1, policy_version 179740 (0.0010) [2023-12-26 16:40:07,686][105620] Updated weights for policy 1, policy_version 179750 (0.0010) [2023-12-26 16:40:07,741][105692] Updated weights for policy 0, policy_version 178714 (0.0005) [2023-12-26 16:40:07,804][105692] Updated weights for policy 0, policy_version 178724 (0.0005) [2023-12-26 16:40:07,859][105692] Updated weights for policy 0, policy_version 178734 (0.0006) [2023-12-26 16:40:07,914][105692] Updated weights for policy 0, policy_version 178744 (0.0010) [2023-12-26 16:40:08,294][105620] Updated weights for policy 1, policy_version 179760 (0.0006) [2023-12-26 16:40:08,352][105620] Updated weights for policy 1, policy_version 179770 (0.0008) [2023-12-26 16:40:08,404][105620] Updated weights for policy 1, policy_version 179780 (0.0006) [2023-12-26 16:40:08,604][105692] Updated weights for policy 0, policy_version 178754 (0.0011) [2023-12-26 16:40:08,662][105692] Updated weights for policy 0, policy_version 178764 (0.0010) [2023-12-26 16:40:08,724][105692] Updated weights for policy 0, policy_version 178774 (0.0010) [2023-12-26 16:40:09,065][105620] Updated weights for policy 1, policy_version 179790 (0.0008) [2023-12-26 16:40:09,109][105620] Updated weights for policy 1, policy_version 179800 (0.0010) [2023-12-26 16:40:09,158][105620] Updated weights for policy 1, policy_version 179810 (0.0010) [2023-12-26 16:40:09,462][105692] Updated weights for policy 0, policy_version 178784 (0.0011) [2023-12-26 16:40:09,519][105692] Updated weights for policy 0, policy_version 178794 (0.0010) [2023-12-26 16:40:09,571][105692] Updated weights for policy 0, policy_version 178804 (0.0010) [2023-12-26 16:40:09,904][105620] Updated weights for policy 1, policy_version 179820 (0.0008) [2023-12-26 16:40:09,968][105620] Updated weights for policy 1, policy_version 179830 (0.0007) [2023-12-26 16:40:10,027][105620] Updated weights for policy 1, policy_version 179840 (0.0007) [2023-12-26 16:40:10,309][105692] Updated weights for policy 0, policy_version 178814 (0.0010) [2023-12-26 16:40:10,378][105692] Updated weights for policy 0, policy_version 178824 (0.0011) [2023-12-26 16:40:10,434][105692] Updated weights for policy 0, policy_version 178834 (0.0010) [2023-12-26 16:40:10,804][105620] Updated weights for policy 1, policy_version 179850 (0.0009) [2023-12-26 16:40:10,863][105620] Updated weights for policy 1, policy_version 179860 (0.0010) [2023-12-26 16:40:10,922][105620] Updated weights for policy 1, policy_version 179870 (0.0010) [2023-12-26 16:40:10,970][105620] Updated weights for policy 1, policy_version 179880 (0.0010) [2023-12-26 16:40:11,006][105692] Updated weights for policy 0, policy_version 178844 (0.0011) [2023-12-26 16:40:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19660.8). Total num frames: 91848704. Throughput: 0: 10020.6, 1: 9857.1. Samples: 91856576. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-26 16:40:11,062][104569] Avg episode reward: [(0, '8679.356'), (1, '9177.202')] [2023-12-26 16:40:11,075][105692] Updated weights for policy 0, policy_version 178854 (0.0011) [2023-12-26 16:40:11,137][105692] Updated weights for policy 0, policy_version 178864 (0.0010) [2023-12-26 16:40:11,706][105620] Updated weights for policy 1, policy_version 179890 (0.0011) [2023-12-26 16:40:11,775][105620] Updated weights for policy 1, policy_version 179900 (0.0012) [2023-12-26 16:40:11,837][105620] Updated weights for policy 1, policy_version 179910 (0.0010) [2023-12-26 16:40:11,873][105692] Updated weights for policy 0, policy_version 178874 (0.0007) [2023-12-26 16:40:11,940][105692] Updated weights for policy 0, policy_version 178884 (0.0005) [2023-12-26 16:40:12,008][105692] Updated weights for policy 0, policy_version 178894 (0.0005) [2023-12-26 16:40:12,071][105692] Updated weights for policy 0, policy_version 178904 (0.0005) [2023-12-26 16:40:12,575][105620] Updated weights for policy 1, policy_version 179920 (0.0008) [2023-12-26 16:40:12,645][105620] Updated weights for policy 1, policy_version 179930 (0.0011) [2023-12-26 16:40:12,701][105692] Updated weights for policy 0, policy_version 178914 (0.0010) [2023-12-26 16:40:12,703][105620] Updated weights for policy 1, policy_version 179940 (0.0010) [2023-12-26 16:40:12,750][105692] Updated weights for policy 0, policy_version 178924 (0.0011) [2023-12-26 16:40:12,809][105692] Updated weights for policy 0, policy_version 178934 (0.0010) [2023-12-26 16:40:13,369][105620] Updated weights for policy 1, policy_version 179950 (0.0006) [2023-12-26 16:40:13,430][105620] Updated weights for policy 1, policy_version 179960 (0.0007) [2023-12-26 16:40:13,433][105692] Updated weights for policy 0, policy_version 178944 (0.0010) [2023-12-26 16:40:13,494][105620] Updated weights for policy 1, policy_version 179970 (0.0006) [2023-12-26 16:40:13,498][105692] Updated weights for policy 0, policy_version 178954 (0.0010) [2023-12-26 16:40:13,556][105692] Updated weights for policy 0, policy_version 178964 (0.0010) [2023-12-26 16:40:14,105][105692] Updated weights for policy 0, policy_version 178974 (0.0005) [2023-12-26 16:40:14,153][105620] Updated weights for policy 1, policy_version 179980 (0.0007) [2023-12-26 16:40:14,163][105692] Updated weights for policy 0, policy_version 178984 (0.0005) [2023-12-26 16:40:14,209][105620] Updated weights for policy 1, policy_version 179990 (0.0008) [2023-12-26 16:40:14,218][105692] Updated weights for policy 0, policy_version 178994 (0.0005) [2023-12-26 16:40:14,257][105620] Updated weights for policy 1, policy_version 180000 (0.0008) [2023-12-26 16:40:14,801][105692] Updated weights for policy 0, policy_version 179004 (0.0006) [2023-12-26 16:40:14,856][105692] Updated weights for policy 0, policy_version 179014 (0.0006) [2023-12-26 16:40:14,920][105692] Updated weights for policy 0, policy_version 179024 (0.0006) [2023-12-26 16:40:15,086][105620] Updated weights for policy 1, policy_version 180010 (0.0010) [2023-12-26 16:40:15,150][105620] Updated weights for policy 1, policy_version 180020 (0.0008) [2023-12-26 16:40:15,203][105620] Updated weights for policy 1, policy_version 180030 (0.0007) [2023-12-26 16:40:15,267][105620] Updated weights for policy 1, policy_version 180040 (0.0010) [2023-12-26 16:40:15,503][105692] Updated weights for policy 0, policy_version 179034 (0.0008) [2023-12-26 16:40:15,558][105692] Updated weights for policy 0, policy_version 179044 (0.0005) [2023-12-26 16:40:15,621][105692] Updated weights for policy 0, policy_version 179054 (0.0005) [2023-12-26 16:40:15,682][105692] Updated weights for policy 0, policy_version 179064 (0.0009) [2023-12-26 16:40:15,905][105620] Updated weights for policy 1, policy_version 180050 (0.0006) [2023-12-26 16:40:15,963][105620] Updated weights for policy 1, policy_version 180060 (0.0005) [2023-12-26 16:40:16,024][105620] Updated weights for policy 1, policy_version 180070 (0.0009) [2023-12-26 16:40:16,062][104569] Fps is (10 sec: 21299.8, 60 sec: 20070.5, 300 sec: 19688.6). Total num frames: 91955200. Throughput: 0: 9922.5, 1: 9876.8. Samples: 91918188. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-26 16:40:16,063][104569] Avg episode reward: [(0, '8856.838'), (1, '8450.115')] [2023-12-26 16:40:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000179064_45850624.pth... [2023-12-26 16:40:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000180072_46104576.pth... [2023-12-26 16:40:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000178920_45809664.pth [2023-12-26 16:40:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000177912_45555712.pth [2023-12-26 16:40:16,255][105692] Updated weights for policy 0, policy_version 179074 (0.0005) [2023-12-26 16:40:16,304][105692] Updated weights for policy 0, policy_version 179084 (0.0005) [2023-12-26 16:40:16,359][105692] Updated weights for policy 0, policy_version 179094 (0.0005) [2023-12-26 16:40:16,705][105620] Updated weights for policy 1, policy_version 180080 (0.0006) [2023-12-26 16:40:16,765][105620] Updated weights for policy 1, policy_version 180090 (0.0009) [2023-12-26 16:40:16,821][105620] Updated weights for policy 1, policy_version 180100 (0.0008) [2023-12-26 16:40:17,057][105692] Updated weights for policy 0, policy_version 179104 (0.0009) [2023-12-26 16:40:17,102][105692] Updated weights for policy 0, policy_version 179114 (0.0010) [2023-12-26 16:40:17,155][105692] Updated weights for policy 0, policy_version 179124 (0.0011) [2023-12-26 16:40:17,422][105620] Updated weights for policy 1, policy_version 180110 (0.0010) [2023-12-26 16:40:17,483][105620] Updated weights for policy 1, policy_version 180120 (0.0010) [2023-12-26 16:40:17,542][105620] Updated weights for policy 1, policy_version 180130 (0.0007) [2023-12-26 16:40:17,903][105692] Updated weights for policy 0, policy_version 179134 (0.0010) [2023-12-26 16:40:17,961][105692] Updated weights for policy 0, policy_version 179144 (0.0010) [2023-12-26 16:40:18,016][105692] Updated weights for policy 0, policy_version 179154 (0.0010) [2023-12-26 16:40:18,197][105620] Updated weights for policy 1, policy_version 180140 (0.0007) [2023-12-26 16:40:18,260][105620] Updated weights for policy 1, policy_version 180150 (0.0007) [2023-12-26 16:40:18,323][105620] Updated weights for policy 1, policy_version 180160 (0.0008) [2023-12-26 16:40:18,756][105692] Updated weights for policy 0, policy_version 179164 (0.0008) [2023-12-26 16:40:18,805][105692] Updated weights for policy 0, policy_version 179174 (0.0005) [2023-12-26 16:40:18,856][105692] Updated weights for policy 0, policy_version 179184 (0.0005) [2023-12-26 16:40:18,985][105620] Updated weights for policy 1, policy_version 180170 (0.0007) [2023-12-26 16:40:19,043][105620] Updated weights for policy 1, policy_version 180180 (0.0010) [2023-12-26 16:40:19,092][105620] Updated weights for policy 1, policy_version 180190 (0.0010) [2023-12-26 16:40:19,154][105620] Updated weights for policy 1, policy_version 180200 (0.0006) [2023-12-26 16:40:19,494][105692] Updated weights for policy 0, policy_version 179194 (0.0006) [2023-12-26 16:40:19,558][105692] Updated weights for policy 0, policy_version 179204 (0.0007) [2023-12-26 16:40:19,616][105692] Updated weights for policy 0, policy_version 179214 (0.0009) [2023-12-26 16:40:19,678][105692] Updated weights for policy 0, policy_version 179224 (0.0009) [2023-12-26 16:40:19,934][105620] Updated weights for policy 1, policy_version 180210 (0.0008) [2023-12-26 16:40:19,996][105620] Updated weights for policy 1, policy_version 180220 (0.0010) [2023-12-26 16:40:20,055][105620] Updated weights for policy 1, policy_version 180230 (0.0009) [2023-12-26 16:40:20,446][105692] Updated weights for policy 0, policy_version 179234 (0.0009) [2023-12-26 16:40:20,499][105692] Updated weights for policy 0, policy_version 179244 (0.0010) [2023-12-26 16:40:20,563][105692] Updated weights for policy 0, policy_version 179254 (0.0010) [2023-12-26 16:40:20,698][105620] Updated weights for policy 1, policy_version 180240 (0.0010) [2023-12-26 16:40:20,766][105620] Updated weights for policy 1, policy_version 180250 (0.0011) [2023-12-26 16:40:20,835][105620] Updated weights for policy 1, policy_version 180260 (0.0011) [2023-12-26 16:40:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 20070.4, 300 sec: 19688.6). Total num frames: 92053504. Throughput: 0: 10014.3, 1: 9829.4. Samples: 92042568. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-26 16:40:21,062][104569] Avg episode reward: [(0, '9022.139'), (1, '8630.541')] [2023-12-26 16:40:21,427][105692] Updated weights for policy 0, policy_version 179264 (0.0008) [2023-12-26 16:40:21,488][105692] Updated weights for policy 0, policy_version 179274 (0.0009) [2023-12-26 16:40:21,523][105620] Updated weights for policy 1, policy_version 180270 (0.0008) [2023-12-26 16:40:21,548][105692] Updated weights for policy 0, policy_version 179284 (0.0011) [2023-12-26 16:40:21,584][105620] Updated weights for policy 1, policy_version 180280 (0.0007) [2023-12-26 16:40:21,644][105620] Updated weights for policy 1, policy_version 180290 (0.0008) [2023-12-26 16:40:22,228][105692] Updated weights for policy 0, policy_version 179294 (0.0009) [2023-12-26 16:40:22,295][105692] Updated weights for policy 0, policy_version 179304 (0.0009) [2023-12-26 16:40:22,367][105692] Updated weights for policy 0, policy_version 179314 (0.0008) [2023-12-26 16:40:22,482][105620] Updated weights for policy 1, policy_version 180300 (0.0009) [2023-12-26 16:40:22,547][105620] Updated weights for policy 1, policy_version 180310 (0.0007) [2023-12-26 16:40:22,618][105620] Updated weights for policy 1, policy_version 180320 (0.0007) [2023-12-26 16:40:23,135][105692] Updated weights for policy 0, policy_version 179324 (0.0009) [2023-12-26 16:40:23,186][105692] Updated weights for policy 0, policy_version 179335 (0.0010) [2023-12-26 16:40:23,231][105692] Updated weights for policy 0, policy_version 179345 (0.0008) [2023-12-26 16:40:23,299][105620] Updated weights for policy 1, policy_version 180330 (0.0007) [2023-12-26 16:40:23,349][105620] Updated weights for policy 1, policy_version 180340 (0.0010) [2023-12-26 16:40:23,400][105620] Updated weights for policy 1, policy_version 180350 (0.0010) [2023-12-26 16:40:23,448][105620] Updated weights for policy 1, policy_version 180360 (0.0010) [2023-12-26 16:40:23,945][105692] Updated weights for policy 0, policy_version 179355 (0.0008) [2023-12-26 16:40:23,993][105692] Updated weights for policy 0, policy_version 179365 (0.0007) [2023-12-26 16:40:24,040][105692] Updated weights for policy 0, policy_version 179375 (0.0008) [2023-12-26 16:40:24,188][105620] Updated weights for policy 1, policy_version 180370 (0.0010) [2023-12-26 16:40:24,247][105620] Updated weights for policy 1, policy_version 180380 (0.0010) [2023-12-26 16:40:24,310][105620] Updated weights for policy 1, policy_version 180390 (0.0011) [2023-12-26 16:40:24,765][105692] Updated weights for policy 0, policy_version 179385 (0.0008) [2023-12-26 16:40:24,821][105692] Updated weights for policy 0, policy_version 179395 (0.0008) [2023-12-26 16:40:24,876][105692] Updated weights for policy 0, policy_version 179405 (0.0009) [2023-12-26 16:40:24,932][105692] Updated weights for policy 0, policy_version 179415 (0.0010) [2023-12-26 16:40:25,037][105620] Updated weights for policy 1, policy_version 180400 (0.0006) [2023-12-26 16:40:25,103][105620] Updated weights for policy 1, policy_version 180410 (0.0010) [2023-12-26 16:40:25,161][105620] Updated weights for policy 1, policy_version 180420 (0.0010) [2023-12-26 16:40:25,666][105692] Updated weights for policy 0, policy_version 179425 (0.0009) [2023-12-26 16:40:25,725][105692] Updated weights for policy 0, policy_version 179435 (0.0006) [2023-12-26 16:40:25,782][105692] Updated weights for policy 0, policy_version 179445 (0.0005) [2023-12-26 16:40:25,858][105620] Updated weights for policy 1, policy_version 180430 (0.0006) [2023-12-26 16:40:25,914][105620] Updated weights for policy 1, policy_version 180440 (0.0005) [2023-12-26 16:40:25,966][105620] Updated weights for policy 1, policy_version 180450 (0.0006) [2023-12-26 16:40:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 20070.4, 300 sec: 19633.0). Total num frames: 92151808. Throughput: 0: 9919.3, 1: 9882.7. Samples: 92156640. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-26 16:40:26,062][104569] Avg episode reward: [(0, '9085.264'), (1, '9087.248')] [2023-12-26 16:40:26,387][105692] Updated weights for policy 0, policy_version 179455 (0.0005) [2023-12-26 16:40:26,444][105692] Updated weights for policy 0, policy_version 179465 (0.0005) [2023-12-26 16:40:26,507][105620] Updated weights for policy 1, policy_version 180460 (0.0006) [2023-12-26 16:40:26,515][105692] Updated weights for policy 0, policy_version 179475 (0.0010) [2023-12-26 16:40:26,563][105620] Updated weights for policy 1, policy_version 180470 (0.0005) [2023-12-26 16:40:26,617][105620] Updated weights for policy 1, policy_version 180480 (0.0005) [2023-12-26 16:40:27,187][105692] Updated weights for policy 0, policy_version 179485 (0.0010) [2023-12-26 16:40:27,248][105692] Updated weights for policy 0, policy_version 179495 (0.0010) [2023-12-26 16:40:27,289][105620] Updated weights for policy 1, policy_version 180490 (0.0005) [2023-12-26 16:40:27,302][105692] Updated weights for policy 0, policy_version 179505 (0.0010) [2023-12-26 16:40:27,344][105620] Updated weights for policy 1, policy_version 180500 (0.0009) [2023-12-26 16:40:27,411][105620] Updated weights for policy 1, policy_version 180510 (0.0007) [2023-12-26 16:40:27,470][105620] Updated weights for policy 1, policy_version 180520 (0.0010) [2023-12-26 16:40:27,917][105692] Updated weights for policy 0, policy_version 179515 (0.0009) [2023-12-26 16:40:27,971][105692] Updated weights for policy 0, policy_version 179525 (0.0010) [2023-12-26 16:40:28,019][105692] Updated weights for policy 0, policy_version 179535 (0.0006) [2023-12-26 16:40:28,222][105620] Updated weights for policy 1, policy_version 180530 (0.0009) [2023-12-26 16:40:28,276][105620] Updated weights for policy 1, policy_version 180540 (0.0010) [2023-12-26 16:40:28,346][105620] Updated weights for policy 1, policy_version 180550 (0.0010) [2023-12-26 16:40:28,616][105692] Updated weights for policy 0, policy_version 179545 (0.0005) [2023-12-26 16:40:28,671][105692] Updated weights for policy 0, policy_version 179555 (0.0005) [2023-12-26 16:40:28,726][105692] Updated weights for policy 0, policy_version 179565 (0.0005) [2023-12-26 16:40:28,792][105692] Updated weights for policy 0, policy_version 179575 (0.0005) [2023-12-26 16:40:28,989][105620] Updated weights for policy 1, policy_version 180560 (0.0010) [2023-12-26 16:40:29,043][105620] Updated weights for policy 1, policy_version 180570 (0.0010) [2023-12-26 16:40:29,111][105620] Updated weights for policy 1, policy_version 180580 (0.0010) [2023-12-26 16:40:29,453][105692] Updated weights for policy 0, policy_version 179585 (0.0006) [2023-12-26 16:40:29,501][105692] Updated weights for policy 0, policy_version 179595 (0.0005) [2023-12-26 16:40:29,560][105692] Updated weights for policy 0, policy_version 179605 (0.0006) [2023-12-26 16:40:29,755][105620] Updated weights for policy 1, policy_version 180590 (0.0007) [2023-12-26 16:40:29,812][105620] Updated weights for policy 1, policy_version 180600 (0.0005) [2023-12-26 16:40:29,879][105620] Updated weights for policy 1, policy_version 180610 (0.0008) [2023-12-26 16:40:30,277][105692] Updated weights for policy 0, policy_version 179615 (0.0007) [2023-12-26 16:40:30,345][105692] Updated weights for policy 0, policy_version 179625 (0.0009) [2023-12-26 16:40:30,393][105692] Updated weights for policy 0, policy_version 179635 (0.0008) [2023-12-26 16:40:30,538][105620] Updated weights for policy 1, policy_version 180620 (0.0010) [2023-12-26 16:40:30,596][105620] Updated weights for policy 1, policy_version 180630 (0.0010) [2023-12-26 16:40:30,646][105620] Updated weights for policy 1, policy_version 180640 (0.0009) [2023-12-26 16:40:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 92250112. Throughput: 0: 10001.9, 1: 9924.3. Samples: 92220592. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-26 16:40:31,063][104569] Avg episode reward: [(0, '8826.676'), (1, '9085.420')] [2023-12-26 16:40:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000179640_45998080.pth... [2023-12-26 16:40:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000180648_46252032.pth... [2023-12-26 16:40:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000179496_45957120.pth [2023-12-26 16:40:31,082][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000178456_45694976.pth [2023-12-26 16:40:31,162][105692] Updated weights for policy 0, policy_version 179645 (0.0008) [2023-12-26 16:40:31,222][105692] Updated weights for policy 0, policy_version 179655 (0.0008) [2023-12-26 16:40:31,275][105620] Updated weights for policy 1, policy_version 180650 (0.0008) [2023-12-26 16:40:31,277][105692] Updated weights for policy 0, policy_version 179665 (0.0009) [2023-12-26 16:40:31,331][105620] Updated weights for policy 1, policy_version 180660 (0.0008) [2023-12-26 16:40:31,397][105620] Updated weights for policy 1, policy_version 180670 (0.0009) [2023-12-26 16:40:31,457][105620] Updated weights for policy 1, policy_version 180680 (0.0008) [2023-12-26 16:40:32,096][105692] Updated weights for policy 0, policy_version 179675 (0.0006) [2023-12-26 16:40:32,139][105620] Updated weights for policy 1, policy_version 180690 (0.0010) [2023-12-26 16:40:32,149][105692] Updated weights for policy 0, policy_version 179685 (0.0007) [2023-12-26 16:40:32,191][105620] Updated weights for policy 1, policy_version 180700 (0.0010) [2023-12-26 16:40:32,206][105692] Updated weights for policy 0, policy_version 179695 (0.0009) [2023-12-26 16:40:32,248][105620] Updated weights for policy 1, policy_version 180710 (0.0007) [2023-12-26 16:40:32,962][105692] Updated weights for policy 0, policy_version 179705 (0.0007) [2023-12-26 16:40:32,999][105620] Updated weights for policy 1, policy_version 180720 (0.0006) [2023-12-26 16:40:33,014][105692] Updated weights for policy 0, policy_version 179715 (0.0007) [2023-12-26 16:40:33,065][105620] Updated weights for policy 1, policy_version 180730 (0.0007) [2023-12-26 16:40:33,065][105692] Updated weights for policy 0, policy_version 179725 (0.0006) [2023-12-26 16:40:33,114][105692] Updated weights for policy 0, policy_version 179735 (0.0005) [2023-12-26 16:40:33,115][105620] Updated weights for policy 1, policy_version 180740 (0.0007) [2023-12-26 16:40:33,690][105692] Updated weights for policy 0, policy_version 179745 (0.0008) [2023-12-26 16:40:33,756][105692] Updated weights for policy 0, policy_version 179755 (0.0006) [2023-12-26 16:40:33,814][105692] Updated weights for policy 0, policy_version 179765 (0.0007) [2023-12-26 16:40:33,876][105620] Updated weights for policy 1, policy_version 180750 (0.0008) [2023-12-26 16:40:33,920][105620] Updated weights for policy 1, policy_version 180760 (0.0010) [2023-12-26 16:40:33,981][105620] Updated weights for policy 1, policy_version 180770 (0.0010) [2023-12-26 16:40:34,476][105692] Updated weights for policy 0, policy_version 179775 (0.0009) [2023-12-26 16:40:34,534][105692] Updated weights for policy 0, policy_version 179785 (0.0009) [2023-12-26 16:40:34,598][105692] Updated weights for policy 0, policy_version 179795 (0.0009) [2023-12-26 16:40:34,710][105620] Updated weights for policy 1, policy_version 180780 (0.0009) [2023-12-26 16:40:34,776][105620] Updated weights for policy 1, policy_version 180790 (0.0010) [2023-12-26 16:40:34,838][105620] Updated weights for policy 1, policy_version 180800 (0.0009) [2023-12-26 16:40:35,354][105692] Updated weights for policy 0, policy_version 179805 (0.0007) [2023-12-26 16:40:35,409][105692] Updated weights for policy 0, policy_version 179815 (0.0005) [2023-12-26 16:40:35,464][105692] Updated weights for policy 0, policy_version 179825 (0.0005) [2023-12-26 16:40:35,558][105620] Updated weights for policy 1, policy_version 180810 (0.0008) [2023-12-26 16:40:35,619][105620] Updated weights for policy 1, policy_version 180820 (0.0007) [2023-12-26 16:40:35,674][105620] Updated weights for policy 1, policy_version 180830 (0.0009) [2023-12-26 16:40:35,738][105620] Updated weights for policy 1, policy_version 180840 (0.0009) [2023-12-26 16:40:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 92348416. Throughput: 0: 10006.3, 1: 9834.9. Samples: 92338980. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-26 16:40:36,063][104569] Avg episode reward: [(0, '8736.984'), (1, '9173.932')] [2023-12-26 16:40:36,172][105692] Updated weights for policy 0, policy_version 179835 (0.0009) [2023-12-26 16:40:36,240][105692] Updated weights for policy 0, policy_version 179845 (0.0010) [2023-12-26 16:40:36,307][105692] Updated weights for policy 0, policy_version 179855 (0.0007) [2023-12-26 16:40:36,458][105620] Updated weights for policy 1, policy_version 180850 (0.0010) [2023-12-26 16:40:36,523][105620] Updated weights for policy 1, policy_version 180860 (0.0009) [2023-12-26 16:40:36,581][105620] Updated weights for policy 1, policy_version 180871 (0.0009) [2023-12-26 16:40:36,959][105692] Updated weights for policy 0, policy_version 179865 (0.0006) [2023-12-26 16:40:37,017][105692] Updated weights for policy 0, policy_version 179875 (0.0010) [2023-12-26 16:40:37,080][105692] Updated weights for policy 0, policy_version 179886 (0.0010) [2023-12-26 16:40:37,134][105692] Updated weights for policy 0, policy_version 179896 (0.0010) [2023-12-26 16:40:37,215][105620] Updated weights for policy 1, policy_version 180881 (0.0005) [2023-12-26 16:40:37,261][105620] Updated weights for policy 1, policy_version 180891 (0.0007) [2023-12-26 16:40:37,317][105620] Updated weights for policy 1, policy_version 180901 (0.0008) [2023-12-26 16:40:37,927][105692] Updated weights for policy 0, policy_version 179906 (0.0009) [2023-12-26 16:40:37,975][105692] Updated weights for policy 0, policy_version 179916 (0.0009) [2023-12-26 16:40:38,032][105692] Updated weights for policy 0, policy_version 179926 (0.0008) [2023-12-26 16:40:38,059][105620] Updated weights for policy 1, policy_version 180911 (0.0009) [2023-12-26 16:40:38,118][105620] Updated weights for policy 1, policy_version 180921 (0.0009) [2023-12-26 16:40:38,163][105620] Updated weights for policy 1, policy_version 180931 (0.0008) [2023-12-26 16:40:38,798][105692] Updated weights for policy 0, policy_version 179936 (0.0008) [2023-12-26 16:40:38,856][105692] Updated weights for policy 0, policy_version 179946 (0.0009) [2023-12-26 16:40:38,907][105692] Updated weights for policy 0, policy_version 179956 (0.0009) [2023-12-26 16:40:38,939][105620] Updated weights for policy 1, policy_version 180941 (0.0008) [2023-12-26 16:40:39,000][105620] Updated weights for policy 1, policy_version 180951 (0.0009) [2023-12-26 16:40:39,058][105620] Updated weights for policy 1, policy_version 180961 (0.0009) [2023-12-26 16:40:39,649][105692] Updated weights for policy 0, policy_version 179966 (0.0008) [2023-12-26 16:40:39,701][105692] Updated weights for policy 0, policy_version 179976 (0.0008) [2023-12-26 16:40:39,750][105692] Updated weights for policy 0, policy_version 179986 (0.0008) [2023-12-26 16:40:39,801][105620] Updated weights for policy 1, policy_version 180971 (0.0010) [2023-12-26 16:40:39,871][105620] Updated weights for policy 1, policy_version 180981 (0.0011) [2023-12-26 16:40:39,928][105620] Updated weights for policy 1, policy_version 180991 (0.0011) [2023-12-26 16:40:40,492][105692] Updated weights for policy 0, policy_version 179996 (0.0008) [2023-12-26 16:40:40,552][105692] Updated weights for policy 0, policy_version 180006 (0.0008) [2023-12-26 16:40:40,618][105692] Updated weights for policy 0, policy_version 180016 (0.0008) [2023-12-26 16:40:40,673][105620] Updated weights for policy 1, policy_version 181001 (0.0011) [2023-12-26 16:40:40,729][105620] Updated weights for policy 1, policy_version 181011 (0.0010) [2023-12-26 16:40:40,786][105620] Updated weights for policy 1, policy_version 181021 (0.0010) [2023-12-26 16:40:40,835][105620] Updated weights for policy 1, policy_version 181031 (0.0011) [2023-12-26 16:40:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 92446720. Throughput: 0: 10006.2, 1: 9843.6. Samples: 92453304. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-26 16:40:41,062][104569] Avg episode reward: [(0, '7913.716'), (1, '9174.949')] [2023-12-26 16:40:41,301][105692] Updated weights for policy 0, policy_version 180026 (0.0008) [2023-12-26 16:40:41,359][105692] Updated weights for policy 0, policy_version 180036 (0.0009) [2023-12-26 16:40:41,427][105692] Updated weights for policy 0, policy_version 180046 (0.0007) [2023-12-26 16:40:41,496][105692] Updated weights for policy 0, policy_version 180056 (0.0008) [2023-12-26 16:40:41,617][105620] Updated weights for policy 1, policy_version 181041 (0.0007) [2023-12-26 16:40:41,686][105620] Updated weights for policy 1, policy_version 181051 (0.0010) [2023-12-26 16:40:41,747][105620] Updated weights for policy 1, policy_version 181061 (0.0009) [2023-12-26 16:40:42,230][105692] Updated weights for policy 0, policy_version 180066 (0.0009) [2023-12-26 16:40:42,294][105692] Updated weights for policy 0, policy_version 180076 (0.0009) [2023-12-26 16:40:42,362][105692] Updated weights for policy 0, policy_version 180086 (0.0009) [2023-12-26 16:40:42,551][105620] Updated weights for policy 1, policy_version 181071 (0.0010) [2023-12-26 16:40:42,603][105620] Updated weights for policy 1, policy_version 181081 (0.0009) [2023-12-26 16:40:42,654][105620] Updated weights for policy 1, policy_version 181091 (0.0009) [2023-12-26 16:40:42,965][105692] Updated weights for policy 0, policy_version 180096 (0.0006) [2023-12-26 16:40:43,013][105692] Updated weights for policy 0, policy_version 180106 (0.0008) [2023-12-26 16:40:43,067][105692] Updated weights for policy 0, policy_version 180116 (0.0009) [2023-12-26 16:40:43,488][105620] Updated weights for policy 1, policy_version 181101 (0.0009) [2023-12-26 16:40:43,533][105620] Updated weights for policy 1, policy_version 181111 (0.0008) [2023-12-26 16:40:43,584][105620] Updated weights for policy 1, policy_version 181121 (0.0008) [2023-12-26 16:40:43,742][105692] Updated weights for policy 0, policy_version 180126 (0.0005) [2023-12-26 16:40:43,804][105692] Updated weights for policy 0, policy_version 180136 (0.0006) [2023-12-26 16:40:43,866][105692] Updated weights for policy 0, policy_version 180146 (0.0009) [2023-12-26 16:40:44,240][105620] Updated weights for policy 1, policy_version 181131 (0.0007) [2023-12-26 16:40:44,290][105620] Updated weights for policy 1, policy_version 181141 (0.0009) [2023-12-26 16:40:44,338][105620] Updated weights for policy 1, policy_version 181151 (0.0010) [2023-12-26 16:40:44,568][105692] Updated weights for policy 0, policy_version 180156 (0.0008) [2023-12-26 16:40:44,625][105692] Updated weights for policy 0, policy_version 180166 (0.0005) [2023-12-26 16:40:44,676][105692] Updated weights for policy 0, policy_version 180176 (0.0005) [2023-12-26 16:40:45,084][105620] Updated weights for policy 1, policy_version 181161 (0.0010) [2023-12-26 16:40:45,147][105620] Updated weights for policy 1, policy_version 181171 (0.0011) [2023-12-26 16:40:45,209][105620] Updated weights for policy 1, policy_version 181181 (0.0011) [2023-12-26 16:40:45,268][105620] Updated weights for policy 1, policy_version 181191 (0.0010) [2023-12-26 16:40:45,382][105692] Updated weights for policy 0, policy_version 180186 (0.0007) [2023-12-26 16:40:45,449][105692] Updated weights for policy 0, policy_version 180196 (0.0008) [2023-12-26 16:40:45,509][105692] Updated weights for policy 0, policy_version 180206 (0.0008) [2023-12-26 16:40:45,561][105692] Updated weights for policy 0, policy_version 180216 (0.0008) [2023-12-26 16:40:46,023][105620] Updated weights for policy 1, policy_version 181201 (0.0011) [2023-12-26 16:40:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 92536832. Throughput: 0: 10026.4, 1: 9792.3. Samples: 92510844. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-26 16:40:46,063][104569] Avg episode reward: [(0, '3497.174'), (1, '9174.029')] [2023-12-26 16:40:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000180216_46145536.pth... [2023-12-26 16:40:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000179064_45850624.pth [2023-12-26 16:40:46,079][105620] Updated weights for policy 1, policy_version 181211 (0.0010) [2023-12-26 16:40:46,137][105620] Updated weights for policy 1, policy_version 181221 (0.0010) [2023-12-26 16:40:46,151][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000181224_46399488.pth... [2023-12-26 16:40:46,156][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000180072_46104576.pth [2023-12-26 16:40:46,173][105692] Updated weights for policy 0, policy_version 180226 (0.0010) [2023-12-26 16:40:46,230][105692] Updated weights for policy 0, policy_version 180236 (0.0010) [2023-12-26 16:40:46,284][105692] Updated weights for policy 0, policy_version 180246 (0.0010) [2023-12-26 16:40:46,887][105620] Updated weights for policy 1, policy_version 181231 (0.0010) [2023-12-26 16:40:46,945][105620] Updated weights for policy 1, policy_version 181241 (0.0010) [2023-12-26 16:40:46,983][105692] Updated weights for policy 0, policy_version 180256 (0.0008) [2023-12-26 16:40:47,003][105620] Updated weights for policy 1, policy_version 181251 (0.0006) [2023-12-26 16:40:47,044][105692] Updated weights for policy 0, policy_version 180266 (0.0008) [2023-12-26 16:40:47,109][105692] Updated weights for policy 0, policy_version 180277 (0.0008) [2023-12-26 16:40:47,707][105620] Updated weights for policy 1, policy_version 181261 (0.0007) [2023-12-26 16:40:47,773][105620] Updated weights for policy 1, policy_version 181271 (0.0007) [2023-12-26 16:40:47,809][105692] Updated weights for policy 0, policy_version 180287 (0.0007) [2023-12-26 16:40:47,835][105620] Updated weights for policy 1, policy_version 181281 (0.0010) [2023-12-26 16:40:47,856][105692] Updated weights for policy 0, policy_version 180297 (0.0005) [2023-12-26 16:40:47,907][105692] Updated weights for policy 0, policy_version 180307 (0.0006) [2023-12-26 16:40:48,560][105620] Updated weights for policy 1, policy_version 181291 (0.0011) [2023-12-26 16:40:48,574][105692] Updated weights for policy 0, policy_version 180317 (0.0008) [2023-12-26 16:40:48,617][105620] Updated weights for policy 1, policy_version 181301 (0.0010) [2023-12-26 16:40:48,639][105692] Updated weights for policy 0, policy_version 180327 (0.0006) [2023-12-26 16:40:48,680][105620] Updated weights for policy 1, policy_version 181311 (0.0010) [2023-12-26 16:40:48,700][105692] Updated weights for policy 0, policy_version 180337 (0.0006) [2023-12-26 16:40:49,308][105692] Updated weights for policy 0, policy_version 180347 (0.0007) [2023-12-26 16:40:49,378][105692] Updated weights for policy 0, policy_version 180357 (0.0008) [2023-12-26 16:40:49,415][105620] Updated weights for policy 1, policy_version 181321 (0.0010) [2023-12-26 16:40:49,429][105692] Updated weights for policy 0, policy_version 180367 (0.0010) [2023-12-26 16:40:49,480][105620] Updated weights for policy 1, policy_version 181331 (0.0010) [2023-12-26 16:40:49,532][105620] Updated weights for policy 1, policy_version 181341 (0.0010) [2023-12-26 16:40:49,588][105620] Updated weights for policy 1, policy_version 181351 (0.0008) [2023-12-26 16:40:50,175][105692] Updated weights for policy 0, policy_version 180377 (0.0007) [2023-12-26 16:40:50,234][105692] Updated weights for policy 0, policy_version 180387 (0.0006) [2023-12-26 16:40:50,235][105620] Updated weights for policy 1, policy_version 181361 (0.0008) [2023-12-26 16:40:50,289][105692] Updated weights for policy 0, policy_version 180397 (0.0006) [2023-12-26 16:40:50,295][105620] Updated weights for policy 1, policy_version 181371 (0.0009) [2023-12-26 16:40:50,348][105692] Updated weights for policy 0, policy_version 180407 (0.0006) [2023-12-26 16:40:50,356][105620] Updated weights for policy 1, policy_version 181381 (0.0011) [2023-12-26 16:40:50,939][105692] Updated weights for policy 0, policy_version 180417 (0.0009) [2023-12-26 16:40:50,996][105692] Updated weights for policy 0, policy_version 180427 (0.0009) [2023-12-26 16:40:51,060][105692] Updated weights for policy 0, policy_version 180437 (0.0009) [2023-12-26 16:40:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 92635136. Throughput: 0: 10093.9, 1: 9812.2. Samples: 92630668. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-26 16:40:51,062][104569] Avg episode reward: [(0, '6169.307'), (1, '9263.074')] [2023-12-26 16:40:51,136][105620] Updated weights for policy 1, policy_version 181391 (0.0009) [2023-12-26 16:40:51,198][105620] Updated weights for policy 1, policy_version 181401 (0.0009) [2023-12-26 16:40:51,246][105620] Updated weights for policy 1, policy_version 181411 (0.0009) [2023-12-26 16:40:51,823][105692] Updated weights for policy 0, policy_version 180447 (0.0009) [2023-12-26 16:40:51,883][105692] Updated weights for policy 0, policy_version 180457 (0.0009) [2023-12-26 16:40:51,941][105692] Updated weights for policy 0, policy_version 180467 (0.0008) [2023-12-26 16:40:52,014][105620] Updated weights for policy 1, policy_version 181421 (0.0008) [2023-12-26 16:40:52,065][105620] Updated weights for policy 1, policy_version 181431 (0.0009) [2023-12-26 16:40:52,118][105620] Updated weights for policy 1, policy_version 181441 (0.0005) [2023-12-26 16:40:52,736][105692] Updated weights for policy 0, policy_version 180477 (0.0009) [2023-12-26 16:40:52,787][105692] Updated weights for policy 0, policy_version 180487 (0.0009) [2023-12-26 16:40:52,826][105620] Updated weights for policy 1, policy_version 181451 (0.0006) [2023-12-26 16:40:52,852][105692] Updated weights for policy 0, policy_version 180497 (0.0009) [2023-12-26 16:40:52,876][105620] Updated weights for policy 1, policy_version 181461 (0.0007) [2023-12-26 16:40:52,929][105620] Updated weights for policy 1, policy_version 181471 (0.0007) [2023-12-26 16:40:53,624][105620] Updated weights for policy 1, policy_version 181481 (0.0009) [2023-12-26 16:40:53,641][105692] Updated weights for policy 0, policy_version 180507 (0.0009) [2023-12-26 16:40:53,676][105620] Updated weights for policy 1, policy_version 181491 (0.0007) [2023-12-26 16:40:53,687][105692] Updated weights for policy 0, policy_version 180517 (0.0007) [2023-12-26 16:40:53,729][105620] Updated weights for policy 1, policy_version 181501 (0.0007) [2023-12-26 16:40:53,735][105692] Updated weights for policy 0, policy_version 180527 (0.0006) [2023-12-26 16:40:53,781][105620] Updated weights for policy 1, policy_version 181511 (0.0008) [2023-12-26 16:40:54,518][105692] Updated weights for policy 0, policy_version 180537 (0.0005) [2023-12-26 16:40:54,552][105620] Updated weights for policy 1, policy_version 181521 (0.0008) [2023-12-26 16:40:54,578][105692] Updated weights for policy 0, policy_version 180547 (0.0007) [2023-12-26 16:40:54,604][105620] Updated weights for policy 1, policy_version 181531 (0.0007) [2023-12-26 16:40:54,629][105692] Updated weights for policy 0, policy_version 180557 (0.0009) [2023-12-26 16:40:54,659][105620] Updated weights for policy 1, policy_version 181541 (0.0010) [2023-12-26 16:40:54,685][105692] Updated weights for policy 0, policy_version 180567 (0.0006) [2023-12-26 16:40:55,318][105620] Updated weights for policy 1, policy_version 181551 (0.0010) [2023-12-26 16:40:55,372][105620] Updated weights for policy 1, policy_version 181561 (0.0010) [2023-12-26 16:40:55,437][105620] Updated weights for policy 1, policy_version 181571 (0.0010) [2023-12-26 16:40:55,495][105692] Updated weights for policy 0, policy_version 180577 (0.0007) [2023-12-26 16:40:55,540][105692] Updated weights for policy 0, policy_version 180587 (0.0008) [2023-12-26 16:40:55,599][105692] Updated weights for policy 0, policy_version 180597 (0.0008) [2023-12-26 16:40:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 92733440. Throughput: 0: 9977.4, 1: 9751.9. Samples: 92744396. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-26 16:40:56,062][104569] Avg episode reward: [(0, '7752.894'), (1, '9263.172')] [2023-12-26 16:40:56,180][105620] Updated weights for policy 1, policy_version 181581 (0.0010) [2023-12-26 16:40:56,236][105620] Updated weights for policy 1, policy_version 181591 (0.0010) [2023-12-26 16:40:56,291][105620] Updated weights for policy 1, policy_version 181601 (0.0010) [2023-12-26 16:40:56,369][105692] Updated weights for policy 0, policy_version 180607 (0.0007) [2023-12-26 16:40:56,435][105692] Updated weights for policy 0, policy_version 180617 (0.0008) [2023-12-26 16:40:56,487][105692] Updated weights for policy 0, policy_version 180627 (0.0008) [2023-12-26 16:40:56,968][105620] Updated weights for policy 1, policy_version 181611 (0.0011) [2023-12-26 16:40:57,019][105620] Updated weights for policy 1, policy_version 181621 (0.0008) [2023-12-26 16:40:57,073][105620] Updated weights for policy 1, policy_version 181631 (0.0006) [2023-12-26 16:40:57,233][105692] Updated weights for policy 0, policy_version 180637 (0.0007) [2023-12-26 16:40:57,292][105692] Updated weights for policy 0, policy_version 180647 (0.0007) [2023-12-26 16:40:57,351][105692] Updated weights for policy 0, policy_version 180657 (0.0008) [2023-12-26 16:40:57,665][105620] Updated weights for policy 1, policy_version 181641 (0.0006) [2023-12-26 16:40:57,712][105620] Updated weights for policy 1, policy_version 181651 (0.0005) [2023-12-26 16:40:57,758][105620] Updated weights for policy 1, policy_version 181661 (0.0005) [2023-12-26 16:40:57,808][105620] Updated weights for policy 1, policy_version 181671 (0.0005) [2023-12-26 16:40:58,142][105692] Updated weights for policy 0, policy_version 180667 (0.0008) [2023-12-26 16:40:58,191][105692] Updated weights for policy 0, policy_version 180677 (0.0008) [2023-12-26 16:40:58,240][105692] Updated weights for policy 0, policy_version 180687 (0.0008) [2023-12-26 16:40:58,458][105620] Updated weights for policy 1, policy_version 181681 (0.0009) [2023-12-26 16:40:58,527][105620] Updated weights for policy 1, policy_version 181691 (0.0011) [2023-12-26 16:40:58,594][105620] Updated weights for policy 1, policy_version 181701 (0.0011) [2023-12-26 16:40:59,021][105692] Updated weights for policy 0, policy_version 180697 (0.0009) [2023-12-26 16:40:59,083][105692] Updated weights for policy 0, policy_version 180707 (0.0006) [2023-12-26 16:40:59,151][105692] Updated weights for policy 0, policy_version 180717 (0.0009) [2023-12-26 16:40:59,199][105692] Updated weights for policy 0, policy_version 180727 (0.0007) [2023-12-26 16:40:59,354][105620] Updated weights for policy 1, policy_version 181711 (0.0011) [2023-12-26 16:40:59,413][105620] Updated weights for policy 1, policy_version 181721 (0.0007) [2023-12-26 16:40:59,483][105620] Updated weights for policy 1, policy_version 181731 (0.0005) [2023-12-26 16:40:59,881][105692] Updated weights for policy 0, policy_version 180737 (0.0010) [2023-12-26 16:40:59,935][105692] Updated weights for policy 0, policy_version 180747 (0.0006) [2023-12-26 16:40:59,993][105692] Updated weights for policy 0, policy_version 180757 (0.0010) [2023-12-26 16:41:00,066][105620] Updated weights for policy 1, policy_version 181741 (0.0008) [2023-12-26 16:41:00,134][105620] Updated weights for policy 1, policy_version 181751 (0.0010) [2023-12-26 16:41:00,192][105620] Updated weights for policy 1, policy_version 181761 (0.0010) [2023-12-26 16:41:00,688][105692] Updated weights for policy 0, policy_version 180767 (0.0011) [2023-12-26 16:41:00,740][105692] Updated weights for policy 0, policy_version 180777 (0.0010) [2023-12-26 16:41:00,760][105620] Updated weights for policy 1, policy_version 181771 (0.0010) [2023-12-26 16:41:00,791][105692] Updated weights for policy 0, policy_version 180787 (0.0010) [2023-12-26 16:41:00,810][105620] Updated weights for policy 1, policy_version 181781 (0.0010) [2023-12-26 16:41:00,862][105620] Updated weights for policy 1, policy_version 181791 (0.0005) [2023-12-26 16:41:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 92839936. Throughput: 0: 9879.3, 1: 9778.9. Samples: 92802808. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-26 16:41:01,063][104569] Avg episode reward: [(0, '7928.729'), (1, '9352.720')] [2023-12-26 16:41:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000180792_46292992.pth... [2023-12-26 16:41:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000181800_46546944.pth... [2023-12-26 16:41:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000179640_45998080.pth [2023-12-26 16:41:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000180648_46252032.pth [2023-12-26 16:41:01,516][105620] Updated weights for policy 1, policy_version 181801 (0.0006) [2023-12-26 16:41:01,542][105692] Updated weights for policy 0, policy_version 180797 (0.0010) [2023-12-26 16:41:01,571][105620] Updated weights for policy 1, policy_version 181811 (0.0010) [2023-12-26 16:41:01,598][105692] Updated weights for policy 0, policy_version 180807 (0.0010) [2023-12-26 16:41:01,637][105620] Updated weights for policy 1, policy_version 181821 (0.0010) [2023-12-26 16:41:01,663][105692] Updated weights for policy 0, policy_version 180817 (0.0007) [2023-12-26 16:41:01,701][105620] Updated weights for policy 1, policy_version 181831 (0.0009) [2023-12-26 16:41:02,281][105692] Updated weights for policy 0, policy_version 180827 (0.0009) [2023-12-26 16:41:02,332][105692] Updated weights for policy 0, policy_version 180837 (0.0007) [2023-12-26 16:41:02,351][105620] Updated weights for policy 1, policy_version 181841 (0.0009) [2023-12-26 16:41:02,397][105692] Updated weights for policy 0, policy_version 180847 (0.0008) [2023-12-26 16:41:02,416][105620] Updated weights for policy 1, policy_version 181851 (0.0010) [2023-12-26 16:41:02,476][105620] Updated weights for policy 1, policy_version 181861 (0.0011) [2023-12-26 16:41:03,034][105692] Updated weights for policy 0, policy_version 180857 (0.0006) [2023-12-26 16:41:03,090][105692] Updated weights for policy 0, policy_version 180867 (0.0005) [2023-12-26 16:41:03,144][105692] Updated weights for policy 0, policy_version 180877 (0.0006) [2023-12-26 16:41:03,199][105692] Updated weights for policy 0, policy_version 180887 (0.0005) [2023-12-26 16:41:03,203][105620] Updated weights for policy 1, policy_version 181871 (0.0011) [2023-12-26 16:41:03,265][105620] Updated weights for policy 1, policy_version 181881 (0.0010) [2023-12-26 16:41:03,317][105620] Updated weights for policy 1, policy_version 181891 (0.0011) [2023-12-26 16:41:03,813][105692] Updated weights for policy 0, policy_version 180897 (0.0010) [2023-12-26 16:41:03,875][105692] Updated weights for policy 0, policy_version 180907 (0.0008) [2023-12-26 16:41:03,930][105692] Updated weights for policy 0, policy_version 180917 (0.0010) [2023-12-26 16:41:03,970][105620] Updated weights for policy 1, policy_version 181901 (0.0009) [2023-12-26 16:41:04,040][105620] Updated weights for policy 1, policy_version 181911 (0.0007) [2023-12-26 16:41:04,108][105620] Updated weights for policy 1, policy_version 181921 (0.0009) [2023-12-26 16:41:04,742][105692] Updated weights for policy 0, policy_version 180927 (0.0009) [2023-12-26 16:41:04,788][105692] Updated weights for policy 0, policy_version 180937 (0.0009) [2023-12-26 16:41:04,812][105620] Updated weights for policy 1, policy_version 181931 (0.0009) [2023-12-26 16:41:04,838][105692] Updated weights for policy 0, policy_version 180947 (0.0007) [2023-12-26 16:41:04,861][105620] Updated weights for policy 1, policy_version 181941 (0.0006) [2023-12-26 16:41:04,915][105620] Updated weights for policy 1, policy_version 181951 (0.0008) [2023-12-26 16:41:05,605][105692] Updated weights for policy 0, policy_version 180957 (0.0008) [2023-12-26 16:41:05,650][105692] Updated weights for policy 0, policy_version 180967 (0.0007) [2023-12-26 16:41:05,674][105620] Updated weights for policy 1, policy_version 181961 (0.0008) [2023-12-26 16:41:05,705][105692] Updated weights for policy 0, policy_version 180977 (0.0006) [2023-12-26 16:41:05,739][105620] Updated weights for policy 1, policy_version 181971 (0.0009) [2023-12-26 16:41:05,800][105620] Updated weights for policy 1, policy_version 181981 (0.0008) [2023-12-26 16:41:05,860][105620] Updated weights for policy 1, policy_version 181991 (0.0008) [2023-12-26 16:41:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19934.0, 300 sec: 19688.6). Total num frames: 92938240. Throughput: 0: 9795.0, 1: 9827.3. Samples: 92925572. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-26 16:41:06,063][104569] Avg episode reward: [(0, '8745.654'), (1, '8837.999')] [2023-12-26 16:41:06,469][105692] Updated weights for policy 0, policy_version 180987 (0.0008) [2023-12-26 16:41:06,527][105692] Updated weights for policy 0, policy_version 180997 (0.0010) [2023-12-26 16:41:06,581][105692] Updated weights for policy 0, policy_version 181007 (0.0006) [2023-12-26 16:41:06,587][105620] Updated weights for policy 1, policy_version 182001 (0.0008) [2023-12-26 16:41:06,641][105620] Updated weights for policy 1, policy_version 182011 (0.0007) [2023-12-26 16:41:06,699][105620] Updated weights for policy 1, policy_version 182021 (0.0009) [2023-12-26 16:41:07,260][105692] Updated weights for policy 0, policy_version 181017 (0.0007) [2023-12-26 16:41:07,321][105692] Updated weights for policy 0, policy_version 181027 (0.0009) [2023-12-26 16:41:07,379][105692] Updated weights for policy 0, policy_version 181037 (0.0009) [2023-12-26 16:41:07,441][105692] Updated weights for policy 0, policy_version 181047 (0.0009) [2023-12-26 16:41:07,483][105620] Updated weights for policy 1, policy_version 182031 (0.0008) [2023-12-26 16:41:07,534][105620] Updated weights for policy 1, policy_version 182041 (0.0009) [2023-12-26 16:41:07,592][105620] Updated weights for policy 1, policy_version 182051 (0.0008) [2023-12-26 16:41:08,157][105692] Updated weights for policy 0, policy_version 181057 (0.0009) [2023-12-26 16:41:08,203][105692] Updated weights for policy 0, policy_version 181067 (0.0009) [2023-12-26 16:41:08,264][105692] Updated weights for policy 0, policy_version 181077 (0.0007) [2023-12-26 16:41:08,417][105620] Updated weights for policy 1, policy_version 182061 (0.0009) [2023-12-26 16:41:08,478][105620] Updated weights for policy 1, policy_version 182071 (0.0010) [2023-12-26 16:41:08,543][105620] Updated weights for policy 1, policy_version 182081 (0.0009) [2023-12-26 16:41:08,957][105692] Updated weights for policy 0, policy_version 181087 (0.0008) [2023-12-26 16:41:09,014][105692] Updated weights for policy 0, policy_version 181097 (0.0009) [2023-12-26 16:41:09,064][105692] Updated weights for policy 0, policy_version 181107 (0.0009) [2023-12-26 16:41:09,358][105620] Updated weights for policy 1, policy_version 182091 (0.0008) [2023-12-26 16:41:09,426][105620] Updated weights for policy 1, policy_version 182101 (0.0010) [2023-12-26 16:41:09,488][105620] Updated weights for policy 1, policy_version 182111 (0.0007) [2023-12-26 16:41:09,817][105692] Updated weights for policy 0, policy_version 181117 (0.0008) [2023-12-26 16:41:09,881][105692] Updated weights for policy 0, policy_version 181127 (0.0009) [2023-12-26 16:41:09,949][105692] Updated weights for policy 0, policy_version 181137 (0.0008) [2023-12-26 16:41:10,151][105620] Updated weights for policy 1, policy_version 182121 (0.0010) [2023-12-26 16:41:10,211][105620] Updated weights for policy 1, policy_version 182131 (0.0009) [2023-12-26 16:41:10,269][105620] Updated weights for policy 1, policy_version 182141 (0.0010) [2023-12-26 16:41:10,330][105620] Updated weights for policy 1, policy_version 182151 (0.0009) [2023-12-26 16:41:10,712][105692] Updated weights for policy 0, policy_version 181147 (0.0009) [2023-12-26 16:41:10,772][105692] Updated weights for policy 0, policy_version 181157 (0.0009) [2023-12-26 16:41:10,832][105692] Updated weights for policy 0, policy_version 181167 (0.0009) [2023-12-26 16:41:11,012][105620] Updated weights for policy 1, policy_version 182161 (0.0006) [2023-12-26 16:41:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 93028352. Throughput: 0: 9819.3, 1: 9789.2. Samples: 93039024. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-26 16:41:11,062][104569] Avg episode reward: [(0, '9171.188'), (1, '8838.104')] [2023-12-26 16:41:11,078][105620] Updated weights for policy 1, policy_version 182171 (0.0007) [2023-12-26 16:41:11,149][105620] Updated weights for policy 1, policy_version 182181 (0.0009) [2023-12-26 16:41:11,657][105692] Updated weights for policy 0, policy_version 181177 (0.0009) [2023-12-26 16:41:11,720][105692] Updated weights for policy 0, policy_version 181187 (0.0009) [2023-12-26 16:41:11,786][105692] Updated weights for policy 0, policy_version 181197 (0.0009) [2023-12-26 16:41:11,852][105692] Updated weights for policy 0, policy_version 181207 (0.0009) [2023-12-26 16:41:11,914][105620] Updated weights for policy 1, policy_version 182191 (0.0009) [2023-12-26 16:41:11,969][105620] Updated weights for policy 1, policy_version 182201 (0.0009) [2023-12-26 16:41:12,020][105620] Updated weights for policy 1, policy_version 182211 (0.0009) [2023-12-26 16:41:12,625][105692] Updated weights for policy 0, policy_version 181217 (0.0008) [2023-12-26 16:41:12,686][105692] Updated weights for policy 0, policy_version 181227 (0.0009) [2023-12-26 16:41:12,730][105620] Updated weights for policy 1, policy_version 182221 (0.0008) [2023-12-26 16:41:12,756][105692] Updated weights for policy 0, policy_version 181237 (0.0008) [2023-12-26 16:41:12,789][105620] Updated weights for policy 1, policy_version 182231 (0.0006) [2023-12-26 16:41:12,837][105620] Updated weights for policy 1, policy_version 182241 (0.0005) [2023-12-26 16:41:13,454][105620] Updated weights for policy 1, policy_version 182251 (0.0006) [2023-12-26 16:41:13,491][105692] Updated weights for policy 0, policy_version 181247 (0.0008) [2023-12-26 16:41:13,507][105620] Updated weights for policy 1, policy_version 182261 (0.0005) [2023-12-26 16:41:13,544][105692] Updated weights for policy 0, policy_version 181257 (0.0009) [2023-12-26 16:41:13,567][105620] Updated weights for policy 1, policy_version 182271 (0.0005) [2023-12-26 16:41:13,602][105692] Updated weights for policy 0, policy_version 181267 (0.0007) [2023-12-26 16:41:14,105][105620] Updated weights for policy 1, policy_version 182281 (0.0007) [2023-12-26 16:41:14,152][105620] Updated weights for policy 1, policy_version 182291 (0.0005) [2023-12-26 16:41:14,208][105620] Updated weights for policy 1, policy_version 182301 (0.0006) [2023-12-26 16:41:14,250][105620] Updated weights for policy 1, policy_version 182311 (0.0006) [2023-12-26 16:41:14,385][105692] Updated weights for policy 0, policy_version 181277 (0.0006) [2023-12-26 16:41:14,446][105692] Updated weights for policy 0, policy_version 181287 (0.0005) [2023-12-26 16:41:14,502][105692] Updated weights for policy 0, policy_version 181297 (0.0005) [2023-12-26 16:41:14,927][105620] Updated weights for policy 1, policy_version 182321 (0.0010) [2023-12-26 16:41:14,986][105620] Updated weights for policy 1, policy_version 182331 (0.0010) [2023-12-26 16:41:15,046][105620] Updated weights for policy 1, policy_version 182341 (0.0011) [2023-12-26 16:41:15,132][105692] Updated weights for policy 0, policy_version 181307 (0.0006) [2023-12-26 16:41:15,197][105692] Updated weights for policy 0, policy_version 181317 (0.0008) [2023-12-26 16:41:15,253][105692] Updated weights for policy 0, policy_version 181327 (0.0008) [2023-12-26 16:41:15,793][105620] Updated weights for policy 1, policy_version 182351 (0.0010) [2023-12-26 16:41:15,855][105620] Updated weights for policy 1, policy_version 182361 (0.0010) [2023-12-26 16:41:15,910][105620] Updated weights for policy 1, policy_version 182371 (0.0010) [2023-12-26 16:41:15,975][105692] Updated weights for policy 0, policy_version 181337 (0.0008) [2023-12-26 16:41:16,027][105692] Updated weights for policy 0, policy_version 181347 (0.0008) [2023-12-26 16:41:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19660.8). Total num frames: 93126656. Throughput: 0: 9680.3, 1: 9775.6. Samples: 93096112. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-26 16:41:16,062][104569] Avg episode reward: [(0, '8569.184'), (1, '9185.730')] [2023-12-26 16:41:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000182376_46694400.pth... [2023-12-26 16:41:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000181224_46399488.pth [2023-12-26 16:41:16,080][105692] Updated weights for policy 0, policy_version 181357 (0.0007) [2023-12-26 16:41:16,134][105692] Updated weights for policy 0, policy_version 181367 (0.0005) [2023-12-26 16:41:16,137][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000181368_46440448.pth... [2023-12-26 16:41:16,141][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000180216_46145536.pth [2023-12-26 16:41:16,629][105620] Updated weights for policy 1, policy_version 182381 (0.0010) [2023-12-26 16:41:16,680][105692] Updated weights for policy 0, policy_version 181377 (0.0005) [2023-12-26 16:41:16,695][105620] Updated weights for policy 1, policy_version 182391 (0.0010) [2023-12-26 16:41:16,731][105692] Updated weights for policy 0, policy_version 181387 (0.0005) [2023-12-26 16:41:16,747][105620] Updated weights for policy 1, policy_version 182401 (0.0010) [2023-12-26 16:41:16,793][105692] Updated weights for policy 0, policy_version 181397 (0.0005) [2023-12-26 16:41:17,377][105692] Updated weights for policy 0, policy_version 181407 (0.0005) [2023-12-26 16:41:17,379][105620] Updated weights for policy 1, policy_version 182411 (0.0009) [2023-12-26 16:41:17,425][105620] Updated weights for policy 1, policy_version 182421 (0.0005) [2023-12-26 16:41:17,436][105692] Updated weights for policy 0, policy_version 181417 (0.0005) [2023-12-26 16:41:17,474][105620] Updated weights for policy 1, policy_version 182431 (0.0005) [2023-12-26 16:41:17,486][105692] Updated weights for policy 0, policy_version 181427 (0.0008) [2023-12-26 16:41:18,123][105692] Updated weights for policy 0, policy_version 181437 (0.0007) [2023-12-26 16:41:18,132][105620] Updated weights for policy 1, policy_version 182441 (0.0010) [2023-12-26 16:41:18,179][105692] Updated weights for policy 0, policy_version 181447 (0.0005) [2023-12-26 16:41:18,185][105620] Updated weights for policy 1, policy_version 182451 (0.0010) [2023-12-26 16:41:18,233][105692] Updated weights for policy 0, policy_version 181457 (0.0006) [2023-12-26 16:41:18,242][105620] Updated weights for policy 1, policy_version 182461 (0.0011) [2023-12-26 16:41:18,294][105620] Updated weights for policy 1, policy_version 182471 (0.0010) [2023-12-26 16:41:18,956][105692] Updated weights for policy 0, policy_version 181467 (0.0007) [2023-12-26 16:41:19,015][105620] Updated weights for policy 1, policy_version 182481 (0.0010) [2023-12-26 16:41:19,016][105692] Updated weights for policy 0, policy_version 181477 (0.0007) [2023-12-26 16:41:19,076][105620] Updated weights for policy 1, policy_version 182491 (0.0011) [2023-12-26 16:41:19,083][105692] Updated weights for policy 0, policy_version 181487 (0.0007) [2023-12-26 16:41:19,135][105620] Updated weights for policy 1, policy_version 182501 (0.0008) [2023-12-26 16:41:19,790][105692] Updated weights for policy 0, policy_version 181497 (0.0008) [2023-12-26 16:41:19,856][105692] Updated weights for policy 0, policy_version 181507 (0.0007) [2023-12-26 16:41:19,918][105620] Updated weights for policy 1, policy_version 182511 (0.0008) [2023-12-26 16:41:19,927][105692] Updated weights for policy 0, policy_version 181517 (0.0008) [2023-12-26 16:41:19,988][105620] Updated weights for policy 1, policy_version 182521 (0.0008) [2023-12-26 16:41:19,998][105692] Updated weights for policy 0, policy_version 181527 (0.0008) [2023-12-26 16:41:20,049][105620] Updated weights for policy 1, policy_version 182531 (0.0007) [2023-12-26 16:41:20,738][105692] Updated weights for policy 0, policy_version 181537 (0.0008) [2023-12-26 16:41:20,784][105620] Updated weights for policy 1, policy_version 182541 (0.0007) [2023-12-26 16:41:20,802][105692] Updated weights for policy 0, policy_version 181547 (0.0008) [2023-12-26 16:41:20,845][105620] Updated weights for policy 1, policy_version 182551 (0.0007) [2023-12-26 16:41:20,864][105692] Updated weights for policy 0, policy_version 181557 (0.0008) [2023-12-26 16:41:20,916][105620] Updated weights for policy 1, policy_version 182561 (0.0008) [2023-12-26 16:41:21,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 93233152. Throughput: 0: 9750.0, 1: 9794.7. Samples: 93218492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-12-26 16:41:21,063][104569] Avg episode reward: [(0, '8567.938'), (1, '9353.139')] [2023-12-26 16:41:21,599][105692] Updated weights for policy 0, policy_version 181567 (0.0008) [2023-12-26 16:41:21,668][105692] Updated weights for policy 0, policy_version 181577 (0.0009) [2023-12-26 16:41:21,672][105620] Updated weights for policy 1, policy_version 182571 (0.0009) [2023-12-26 16:41:21,734][105620] Updated weights for policy 1, policy_version 182581 (0.0008) [2023-12-26 16:41:21,736][105692] Updated weights for policy 0, policy_version 181587 (0.0007) [2023-12-26 16:41:21,792][105620] Updated weights for policy 1, policy_version 182591 (0.0008) [2023-12-26 16:41:22,464][105692] Updated weights for policy 0, policy_version 181597 (0.0008) [2023-12-26 16:41:22,511][105620] Updated weights for policy 1, policy_version 182601 (0.0008) [2023-12-26 16:41:22,512][105692] Updated weights for policy 0, policy_version 181607 (0.0009) [2023-12-26 16:41:22,560][105692] Updated weights for policy 0, policy_version 181617 (0.0006) [2023-12-26 16:41:22,573][105620] Updated weights for policy 1, policy_version 182611 (0.0009) [2023-12-26 16:41:22,646][105620] Updated weights for policy 1, policy_version 182621 (0.0008) [2023-12-26 16:41:22,704][105620] Updated weights for policy 1, policy_version 182631 (0.0008) [2023-12-26 16:41:23,283][105692] Updated weights for policy 0, policy_version 181627 (0.0007) [2023-12-26 16:41:23,340][105692] Updated weights for policy 0, policy_version 181637 (0.0005) [2023-12-26 16:41:23,399][105692] Updated weights for policy 0, policy_version 181647 (0.0009) [2023-12-26 16:41:23,500][105620] Updated weights for policy 1, policy_version 182641 (0.0007) [2023-12-26 16:41:23,559][105620] Updated weights for policy 1, policy_version 182651 (0.0008) [2023-12-26 16:41:23,621][105620] Updated weights for policy 1, policy_version 182661 (0.0008) [2023-12-26 16:41:24,115][105692] Updated weights for policy 0, policy_version 181657 (0.0011) [2023-12-26 16:41:24,178][105692] Updated weights for policy 0, policy_version 181667 (0.0011) [2023-12-26 16:41:24,234][105692] Updated weights for policy 0, policy_version 181677 (0.0011) [2023-12-26 16:41:24,283][105620] Updated weights for policy 1, policy_version 182671 (0.0009) [2023-12-26 16:41:24,283][105692] Updated weights for policy 0, policy_version 181687 (0.0011) [2023-12-26 16:41:24,341][105620] Updated weights for policy 1, policy_version 182681 (0.0010) [2023-12-26 16:41:24,399][105620] Updated weights for policy 1, policy_version 182691 (0.0007) [2023-12-26 16:41:24,955][105692] Updated weights for policy 0, policy_version 181697 (0.0006) [2023-12-26 16:41:25,011][105692] Updated weights for policy 0, policy_version 181707 (0.0008) [2023-12-26 16:41:25,062][105692] Updated weights for policy 0, policy_version 181717 (0.0010) [2023-12-26 16:41:25,088][105620] Updated weights for policy 1, policy_version 182701 (0.0010) [2023-12-26 16:41:25,146][105620] Updated weights for policy 1, policy_version 182711 (0.0010) [2023-12-26 16:41:25,210][105620] Updated weights for policy 1, policy_version 182721 (0.0008) [2023-12-26 16:41:25,797][105692] Updated weights for policy 0, policy_version 181727 (0.0007) [2023-12-26 16:41:25,862][105692] Updated weights for policy 0, policy_version 181737 (0.0006) [2023-12-26 16:41:25,921][105620] Updated weights for policy 1, policy_version 182731 (0.0010) [2023-12-26 16:41:25,925][105692] Updated weights for policy 0, policy_version 181747 (0.0006) [2023-12-26 16:41:25,966][105620] Updated weights for policy 1, policy_version 182741 (0.0010) [2023-12-26 16:41:26,016][105620] Updated weights for policy 1, policy_version 182751 (0.0009) [2023-12-26 16:41:26,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 93323264. Throughput: 0: 9770.0, 1: 9786.0. Samples: 93333324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-12-26 16:41:26,062][104569] Avg episode reward: [(0, '8993.075'), (1, '8774.033')] [2023-12-26 16:41:26,532][105692] Updated weights for policy 0, policy_version 181757 (0.0010) [2023-12-26 16:41:26,579][105692] Updated weights for policy 0, policy_version 181767 (0.0010) [2023-12-26 16:41:26,634][105692] Updated weights for policy 0, policy_version 181777 (0.0009) [2023-12-26 16:41:26,637][105620] Updated weights for policy 1, policy_version 182761 (0.0009) [2023-12-26 16:41:26,687][105620] Updated weights for policy 1, policy_version 182771 (0.0010) [2023-12-26 16:41:26,751][105620] Updated weights for policy 1, policy_version 182781 (0.0010) [2023-12-26 16:41:26,802][105620] Updated weights for policy 1, policy_version 182791 (0.0010) [2023-12-26 16:41:27,314][105692] Updated weights for policy 0, policy_version 181787 (0.0010) [2023-12-26 16:41:27,364][105692] Updated weights for policy 0, policy_version 181797 (0.0009) [2023-12-26 16:41:27,400][105620] Updated weights for policy 1, policy_version 182801 (0.0010) [2023-12-26 16:41:27,419][105692] Updated weights for policy 0, policy_version 181807 (0.0009) [2023-12-26 16:41:27,459][105620] Updated weights for policy 1, policy_version 182811 (0.0010) [2023-12-26 16:41:27,510][105620] Updated weights for policy 1, policy_version 182821 (0.0010) [2023-12-26 16:41:28,048][105692] Updated weights for policy 0, policy_version 181817 (0.0007) [2023-12-26 16:41:28,092][105692] Updated weights for policy 0, policy_version 181827 (0.0010) [2023-12-26 16:41:28,139][105692] Updated weights for policy 0, policy_version 181837 (0.0010) [2023-12-26 16:41:28,183][105692] Updated weights for policy 0, policy_version 181847 (0.0010) [2023-12-26 16:41:28,277][105620] Updated weights for policy 1, policy_version 182831 (0.0010) [2023-12-26 16:41:28,329][105620] Updated weights for policy 1, policy_version 182841 (0.0010) [2023-12-26 16:41:28,392][105620] Updated weights for policy 1, policy_version 182851 (0.0010) [2023-12-26 16:41:28,885][105692] Updated weights for policy 0, policy_version 181857 (0.0010) [2023-12-26 16:41:28,945][105692] Updated weights for policy 0, policy_version 181867 (0.0010) [2023-12-26 16:41:29,001][105692] Updated weights for policy 0, policy_version 181877 (0.0005) [2023-12-26 16:41:29,134][105620] Updated weights for policy 1, policy_version 182861 (0.0010) [2023-12-26 16:41:29,182][105620] Updated weights for policy 1, policy_version 182871 (0.0010) [2023-12-26 16:41:29,243][105620] Updated weights for policy 1, policy_version 182881 (0.0012) [2023-12-26 16:41:29,630][105692] Updated weights for policy 0, policy_version 181887 (0.0009) [2023-12-26 16:41:29,681][105692] Updated weights for policy 0, policy_version 181897 (0.0010) [2023-12-26 16:41:29,728][105692] Updated weights for policy 0, policy_version 181907 (0.0010) [2023-12-26 16:41:30,052][105620] Updated weights for policy 1, policy_version 182891 (0.0010) [2023-12-26 16:41:30,107][105620] Updated weights for policy 1, policy_version 182901 (0.0010) [2023-12-26 16:41:30,162][105620] Updated weights for policy 1, policy_version 182911 (0.0010) [2023-12-26 16:41:30,406][105692] Updated weights for policy 0, policy_version 181917 (0.0008) [2023-12-26 16:41:30,461][105692] Updated weights for policy 0, policy_version 181927 (0.0005) [2023-12-26 16:41:30,515][105692] Updated weights for policy 0, policy_version 181937 (0.0010) [2023-12-26 16:41:30,918][105620] Updated weights for policy 1, policy_version 182921 (0.0011) [2023-12-26 16:41:30,972][105620] Updated weights for policy 1, policy_version 182931 (0.0010) [2023-12-26 16:41:31,029][105620] Updated weights for policy 1, policy_version 182941 (0.0010) [2023-12-26 16:41:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 93421568. Throughput: 0: 9785.9, 1: 9875.8. Samples: 93395616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-12-26 16:41:31,063][104569] Avg episode reward: [(0, '9172.567'), (1, '8354.738')] [2023-12-26 16:41:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000181944_46587904.pth... [2023-12-26 16:41:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000180792_46292992.pth [2023-12-26 16:41:31,090][105620] Updated weights for policy 1, policy_version 182951 (0.0011) [2023-12-26 16:41:31,092][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000182952_46841856.pth... [2023-12-26 16:41:31,095][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000181800_46546944.pth [2023-12-26 16:41:31,184][105692] Updated weights for policy 0, policy_version 181947 (0.0010) [2023-12-26 16:41:31,256][105692] Updated weights for policy 0, policy_version 181957 (0.0008) [2023-12-26 16:41:31,316][105692] Updated weights for policy 0, policy_version 181967 (0.0008) [2023-12-26 16:41:31,768][105620] Updated weights for policy 1, policy_version 182961 (0.0010) [2023-12-26 16:41:31,827][105620] Updated weights for policy 1, policy_version 182971 (0.0010) [2023-12-26 16:41:31,892][105620] Updated weights for policy 1, policy_version 182981 (0.0010) [2023-12-26 16:41:32,017][105692] Updated weights for policy 0, policy_version 181977 (0.0008) [2023-12-26 16:41:32,070][105692] Updated weights for policy 0, policy_version 181987 (0.0005) [2023-12-26 16:41:32,119][105692] Updated weights for policy 0, policy_version 181997 (0.0006) [2023-12-26 16:41:32,169][105692] Updated weights for policy 0, policy_version 182007 (0.0007) [2023-12-26 16:41:32,633][105620] Updated weights for policy 1, policy_version 182991 (0.0010) [2023-12-26 16:41:32,681][105620] Updated weights for policy 1, policy_version 183001 (0.0010) [2023-12-26 16:41:32,730][105620] Updated weights for policy 1, policy_version 183011 (0.0010) [2023-12-26 16:41:32,930][105692] Updated weights for policy 0, policy_version 182017 (0.0010) [2023-12-26 16:41:32,988][105692] Updated weights for policy 0, policy_version 182027 (0.0010) [2023-12-26 16:41:33,045][105692] Updated weights for policy 0, policy_version 182037 (0.0010) [2023-12-26 16:41:33,466][105620] Updated weights for policy 1, policy_version 183021 (0.0010) [2023-12-26 16:41:33,524][105620] Updated weights for policy 1, policy_version 183031 (0.0010) [2023-12-26 16:41:33,578][105620] Updated weights for policy 1, policy_version 183041 (0.0010) [2023-12-26 16:41:33,645][105692] Updated weights for policy 0, policy_version 182047 (0.0008) [2023-12-26 16:41:33,713][105692] Updated weights for policy 0, policy_version 182057 (0.0005) [2023-12-26 16:41:33,766][105692] Updated weights for policy 0, policy_version 182067 (0.0005) [2023-12-26 16:41:34,300][105620] Updated weights for policy 1, policy_version 183051 (0.0010) [2023-12-26 16:41:34,365][105620] Updated weights for policy 1, policy_version 183061 (0.0008) [2023-12-26 16:41:34,394][105692] Updated weights for policy 0, policy_version 182077 (0.0010) [2023-12-26 16:41:34,432][105620] Updated weights for policy 1, policy_version 183071 (0.0008) [2023-12-26 16:41:34,455][105692] Updated weights for policy 0, policy_version 182087 (0.0006) [2023-12-26 16:41:34,513][105692] Updated weights for policy 0, policy_version 182097 (0.0007) [2023-12-26 16:41:35,178][105620] Updated weights for policy 1, policy_version 183081 (0.0009) [2023-12-26 16:41:35,181][105692] Updated weights for policy 0, policy_version 182107 (0.0010) [2023-12-26 16:41:35,227][105620] Updated weights for policy 1, policy_version 183091 (0.0005) [2023-12-26 16:41:35,229][105692] Updated weights for policy 0, policy_version 182117 (0.0010) [2023-12-26 16:41:35,281][105692] Updated weights for policy 0, policy_version 182127 (0.0010) [2023-12-26 16:41:35,286][105620] Updated weights for policy 1, policy_version 183101 (0.0007) [2023-12-26 16:41:35,339][105620] Updated weights for policy 1, policy_version 183111 (0.0007) [2023-12-26 16:41:35,994][105692] Updated weights for policy 0, policy_version 182137 (0.0010) [2023-12-26 16:41:36,015][105620] Updated weights for policy 1, policy_version 183121 (0.0008) [2023-12-26 16:41:36,053][105692] Updated weights for policy 0, policy_version 182147 (0.0007) [2023-12-26 16:41:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 93519872. Throughput: 0: 9806.0, 1: 9845.2. Samples: 93514976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-12-26 16:41:36,062][104569] Avg episode reward: [(0, '9258.772'), (1, '8434.338')] [2023-12-26 16:41:36,076][105620] Updated weights for policy 1, policy_version 183131 (0.0010) [2023-12-26 16:41:36,107][105692] Updated weights for policy 0, policy_version 182157 (0.0006) [2023-12-26 16:41:36,142][105620] Updated weights for policy 1, policy_version 183141 (0.0011) [2023-12-26 16:41:36,168][105692] Updated weights for policy 0, policy_version 182167 (0.0007) [2023-12-26 16:41:36,857][105620] Updated weights for policy 1, policy_version 183151 (0.0010) [2023-12-26 16:41:36,920][105620] Updated weights for policy 1, policy_version 183161 (0.0010) [2023-12-26 16:41:36,957][105692] Updated weights for policy 0, policy_version 182177 (0.0006) [2023-12-26 16:41:36,985][105620] Updated weights for policy 1, policy_version 183171 (0.0010) [2023-12-26 16:41:37,013][105692] Updated weights for policy 0, policy_version 182187 (0.0005) [2023-12-26 16:41:37,060][105692] Updated weights for policy 0, policy_version 182197 (0.0005) [2023-12-26 16:41:37,713][105692] Updated weights for policy 0, policy_version 182207 (0.0008) [2023-12-26 16:41:37,719][105620] Updated weights for policy 1, policy_version 183181 (0.0011) [2023-12-26 16:41:37,766][105692] Updated weights for policy 0, policy_version 182217 (0.0006) [2023-12-26 16:41:37,775][105620] Updated weights for policy 1, policy_version 183191 (0.0010) [2023-12-26 16:41:37,816][105692] Updated weights for policy 0, policy_version 182227 (0.0008) [2023-12-26 16:41:37,831][105620] Updated weights for policy 1, policy_version 183201 (0.0010) [2023-12-26 16:41:38,588][105620] Updated weights for policy 1, policy_version 183211 (0.0010) [2023-12-26 16:41:38,621][105692] Updated weights for policy 0, policy_version 182237 (0.0007) [2023-12-26 16:41:38,650][105620] Updated weights for policy 1, policy_version 183221 (0.0011) [2023-12-26 16:41:38,670][105692] Updated weights for policy 0, policy_version 182247 (0.0006) [2023-12-26 16:41:38,700][105620] Updated weights for policy 1, policy_version 183231 (0.0010) [2023-12-26 16:41:38,730][105692] Updated weights for policy 0, policy_version 182257 (0.0006) [2023-12-26 16:41:39,421][105620] Updated weights for policy 1, policy_version 183241 (0.0010) [2023-12-26 16:41:39,486][105620] Updated weights for policy 1, policy_version 183251 (0.0007) [2023-12-26 16:41:39,545][105620] Updated weights for policy 1, policy_version 183261 (0.0006) [2023-12-26 16:41:39,550][105692] Updated weights for policy 0, policy_version 182267 (0.0007) [2023-12-26 16:41:39,602][105620] Updated weights for policy 1, policy_version 183271 (0.0006) [2023-12-26 16:41:39,608][105692] Updated weights for policy 0, policy_version 182277 (0.0008) [2023-12-26 16:41:39,670][105692] Updated weights for policy 0, policy_version 182287 (0.0008) [2023-12-26 16:41:40,305][105620] Updated weights for policy 1, policy_version 183281 (0.0005) [2023-12-26 16:41:40,366][105620] Updated weights for policy 1, policy_version 183291 (0.0005) [2023-12-26 16:41:40,432][105620] Updated weights for policy 1, policy_version 183301 (0.0007) [2023-12-26 16:41:40,462][105692] Updated weights for policy 0, policy_version 182297 (0.0008) [2023-12-26 16:41:40,526][105692] Updated weights for policy 0, policy_version 182307 (0.0005) [2023-12-26 16:41:40,577][105692] Updated weights for policy 0, policy_version 182317 (0.0005) [2023-12-26 16:41:40,632][105692] Updated weights for policy 0, policy_version 182327 (0.0007) [2023-12-26 16:41:41,058][105620] Updated weights for policy 1, policy_version 183311 (0.0009) [2023-12-26 16:41:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 93618176. Throughput: 0: 9826.4, 1: 9862.6. Samples: 93630400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-12-26 16:41:41,063][104569] Avg episode reward: [(0, '9265.162'), (1, '8430.720')] [2023-12-26 16:41:41,120][105620] Updated weights for policy 1, policy_version 183321 (0.0007) [2023-12-26 16:41:41,187][105620] Updated weights for policy 1, policy_version 183331 (0.0008) [2023-12-26 16:41:41,271][105692] Updated weights for policy 0, policy_version 182337 (0.0008) [2023-12-26 16:41:41,327][105692] Updated weights for policy 0, policy_version 182347 (0.0008) [2023-12-26 16:41:41,399][105692] Updated weights for policy 0, policy_version 182357 (0.0008) [2023-12-26 16:41:42,015][105620] Updated weights for policy 1, policy_version 183341 (0.0009) [2023-12-26 16:41:42,071][105620] Updated weights for policy 1, policy_version 183351 (0.0011) [2023-12-26 16:41:42,132][105620] Updated weights for policy 1, policy_version 183361 (0.0011) [2023-12-26 16:41:42,173][105692] Updated weights for policy 0, policy_version 182367 (0.0007) [2023-12-26 16:41:42,237][105692] Updated weights for policy 0, policy_version 182377 (0.0009) [2023-12-26 16:41:42,305][105692] Updated weights for policy 0, policy_version 182387 (0.0007) [2023-12-26 16:41:42,885][105620] Updated weights for policy 1, policy_version 183371 (0.0007) [2023-12-26 16:41:42,937][105620] Updated weights for policy 1, policy_version 183381 (0.0010) [2023-12-26 16:41:42,990][105620] Updated weights for policy 1, policy_version 183391 (0.0006) [2023-12-26 16:41:43,068][105692] Updated weights for policy 0, policy_version 182397 (0.0010) [2023-12-26 16:41:43,123][105692] Updated weights for policy 0, policy_version 182407 (0.0009) [2023-12-26 16:41:43,170][105692] Updated weights for policy 0, policy_version 182417 (0.0009) [2023-12-26 16:41:43,583][105620] Updated weights for policy 1, policy_version 183401 (0.0006) [2023-12-26 16:41:43,632][105620] Updated weights for policy 1, policy_version 183411 (0.0010) [2023-12-26 16:41:43,683][105620] Updated weights for policy 1, policy_version 183421 (0.0007) [2023-12-26 16:41:43,730][105620] Updated weights for policy 1, policy_version 183431 (0.0005) [2023-12-26 16:41:43,979][105692] Updated weights for policy 0, policy_version 182427 (0.0010) [2023-12-26 16:41:44,046][105692] Updated weights for policy 0, policy_version 182437 (0.0006) [2023-12-26 16:41:44,110][105692] Updated weights for policy 0, policy_version 182447 (0.0005) [2023-12-26 16:41:44,352][105620] Updated weights for policy 1, policy_version 183441 (0.0006) [2023-12-26 16:41:44,421][105620] Updated weights for policy 1, policy_version 183451 (0.0007) [2023-12-26 16:41:44,493][105620] Updated weights for policy 1, policy_version 183461 (0.0005) [2023-12-26 16:41:44,670][105692] Updated weights for policy 0, policy_version 182457 (0.0010) [2023-12-26 16:41:44,719][105692] Updated weights for policy 0, policy_version 182467 (0.0010) [2023-12-26 16:41:44,772][105692] Updated weights for policy 0, policy_version 182477 (0.0008) [2023-12-26 16:41:44,833][105692] Updated weights for policy 0, policy_version 182487 (0.0010) [2023-12-26 16:41:45,123][105620] Updated weights for policy 1, policy_version 183471 (0.0009) [2023-12-26 16:41:45,190][105620] Updated weights for policy 1, policy_version 183481 (0.0011) [2023-12-26 16:41:45,253][105620] Updated weights for policy 1, policy_version 183491 (0.0011) [2023-12-26 16:41:45,615][105692] Updated weights for policy 0, policy_version 182497 (0.0010) [2023-12-26 16:41:45,667][105692] Updated weights for policy 0, policy_version 182507 (0.0010) [2023-12-26 16:41:45,722][105692] Updated weights for policy 0, policy_version 182517 (0.0011) [2023-12-26 16:41:45,859][105620] Updated weights for policy 1, policy_version 183501 (0.0007) [2023-12-26 16:41:45,916][105620] Updated weights for policy 1, policy_version 183511 (0.0005) [2023-12-26 16:41:45,970][105620] Updated weights for policy 1, policy_version 183521 (0.0005) [2023-12-26 16:41:46,062][104569] Fps is (10 sec: 20479.4, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 93724672. Throughput: 0: 9840.8, 1: 9821.8. Samples: 93687632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-12-26 16:41:46,063][104569] Avg episode reward: [(0, '9265.188'), (1, '8388.680')] [2023-12-26 16:41:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000183528_46989312.pth... [2023-12-26 16:41:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000182520_46735360.pth... [2023-12-26 16:41:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000182376_46694400.pth [2023-12-26 16:41:46,078][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_000183528_46989312.pth [2023-12-26 16:41:46,080][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000181368_46440448.pth [2023-12-26 16:41:46,081][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_000182520_46735360.pth [2023-12-26 16:41:46,520][105692] Updated weights for policy 0, policy_version 182527 (0.0010) [2023-12-26 16:41:46,545][105620] Updated weights for policy 1, policy_version 183531 (0.0005) [2023-12-26 16:41:46,588][105692] Updated weights for policy 0, policy_version 182537 (0.0008) [2023-12-26 16:41:46,606][105620] Updated weights for policy 1, policy_version 183541 (0.0005) [2023-12-26 16:41:46,648][105692] Updated weights for policy 0, policy_version 182547 (0.0006) [2023-12-26 16:41:46,660][105620] Updated weights for policy 1, policy_version 183551 (0.0005) [2023-12-26 16:41:47,250][105692] Updated weights for policy 0, policy_version 182557 (0.0008) [2023-12-26 16:41:47,254][105620] Updated weights for policy 1, policy_version 183561 (0.0007) [2023-12-26 16:41:47,295][105620] Updated weights for policy 1, policy_version 183571 (0.0010) [2023-12-26 16:41:47,300][105692] Updated weights for policy 0, policy_version 182567 (0.0005) [2023-12-26 16:41:47,356][105692] Updated weights for policy 0, policy_version 182577 (0.0005) [2023-12-26 16:41:47,364][105620] Updated weights for policy 1, policy_version 183581 (0.0010) [2023-12-26 16:41:47,421][105620] Updated weights for policy 1, policy_version 183591 (0.0005) [2023-12-26 16:41:47,966][105620] Updated weights for policy 1, policy_version 183601 (0.0005) [2023-12-26 16:41:47,996][105692] Updated weights for policy 0, policy_version 182587 (0.0007) [2023-12-26 16:41:48,030][105620] Updated weights for policy 1, policy_version 183611 (0.0005) [2023-12-26 16:41:48,055][105692] Updated weights for policy 0, policy_version 182597 (0.0007) [2023-12-26 16:41:48,080][105620] Updated weights for policy 1, policy_version 183621 (0.0005) [2023-12-26 16:41:48,101][105692] Updated weights for policy 0, policy_version 182607 (0.0009) [2023-12-26 16:41:48,722][105620] Updated weights for policy 1, policy_version 183631 (0.0009) [2023-12-26 16:41:48,781][105620] Updated weights for policy 1, policy_version 183641 (0.0011) [2023-12-26 16:41:48,830][105620] Updated weights for policy 1, policy_version 183651 (0.0010) [2023-12-26 16:41:48,853][105692] Updated weights for policy 0, policy_version 182617 (0.0010) [2023-12-26 16:41:48,898][105692] Updated weights for policy 0, policy_version 182627 (0.0007) [2023-12-26 16:41:48,946][105692] Updated weights for policy 0, policy_version 182637 (0.0005) [2023-12-26 16:41:49,006][105692] Updated weights for policy 0, policy_version 182647 (0.0005) [2023-12-26 16:41:49,563][105620] Updated weights for policy 1, policy_version 183661 (0.0008) [2023-12-26 16:41:49,611][105692] Updated weights for policy 0, policy_version 182657 (0.0005) [2023-12-26 16:41:49,619][105620] Updated weights for policy 1, policy_version 183671 (0.0011) [2023-12-26 16:41:49,672][105692] Updated weights for policy 0, policy_version 182667 (0.0006) [2023-12-26 16:41:49,682][105620] Updated weights for policy 1, policy_version 183681 (0.0011) [2023-12-26 16:41:49,735][105692] Updated weights for policy 0, policy_version 182677 (0.0011) [2023-12-26 16:41:50,452][105620] Updated weights for policy 1, policy_version 183691 (0.0010) [2023-12-26 16:41:50,454][105692] Updated weights for policy 0, policy_version 182687 (0.0008) [2023-12-26 16:41:50,501][105620] Updated weights for policy 1, policy_version 183701 (0.0008) [2023-12-26 16:41:50,504][105692] Updated weights for policy 0, policy_version 182697 (0.0007) [2023-12-26 16:41:50,553][105620] Updated weights for policy 1, policy_version 183711 (0.0007) [2023-12-26 16:41:50,559][105692] Updated weights for policy 0, policy_version 182707 (0.0008) [2023-12-26 16:41:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 93822976. Throughput: 0: 9852.0, 1: 9895.5. Samples: 93814208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-12-26 16:41:51,062][104569] Avg episode reward: [(0, '9180.072'), (1, '8825.864')] [2023-12-26 16:41:51,318][105692] Updated weights for policy 0, policy_version 182717 (0.0008) [2023-12-26 16:41:51,391][105692] Updated weights for policy 0, policy_version 182727 (0.0008) [2023-12-26 16:41:51,400][105620] Updated weights for policy 1, policy_version 183721 (0.0008) [2023-12-26 16:41:51,455][105692] Updated weights for policy 0, policy_version 182737 (0.0007) [2023-12-26 16:41:51,458][105620] Updated weights for policy 1, policy_version 183731 (0.0008) [2023-12-26 16:41:51,518][105620] Updated weights for policy 1, policy_version 183741 (0.0008) [2023-12-26 16:41:51,588][105620] Updated weights for policy 1, policy_version 183751 (0.0009) [2023-12-26 16:41:52,094][105692] Updated weights for policy 0, policy_version 182747 (0.0008) [2023-12-26 16:41:52,158][105692] Updated weights for policy 0, policy_version 182757 (0.0007) [2023-12-26 16:41:52,224][105692] Updated weights for policy 0, policy_version 182767 (0.0008) [2023-12-26 16:41:52,336][105620] Updated weights for policy 1, policy_version 183761 (0.0008) [2023-12-26 16:41:52,402][105620] Updated weights for policy 1, policy_version 183771 (0.0008) [2023-12-26 16:41:52,464][105620] Updated weights for policy 1, policy_version 183781 (0.0008) [2023-12-26 16:41:52,903][105692] Updated weights for policy 0, policy_version 182777 (0.0008) [2023-12-26 16:41:52,954][105692] Updated weights for policy 0, policy_version 182787 (0.0006) [2023-12-26 16:41:53,002][105692] Updated weights for policy 0, policy_version 182797 (0.0010) [2023-12-26 16:41:53,057][105692] Updated weights for policy 0, policy_version 182807 (0.0006) [2023-12-26 16:41:53,061][105585] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000007 [2023-12-26 16:41:53,258][105620] Updated weights for policy 1, policy_version 183791 (0.0008) [2023-12-26 16:41:53,322][105620] Updated weights for policy 1, policy_version 183801 (0.0009) [2023-12-26 16:41:53,380][105620] Updated weights for policy 1, policy_version 183811 (0.0009) [2023-12-26 16:41:53,750][105692] Updated weights for policy 0, policy_version 182817 (0.0009) [2023-12-26 16:41:53,810][105692] Updated weights for policy 0, policy_version 182827 (0.0009) [2023-12-26 16:41:53,863][105692] Updated weights for policy 0, policy_version 182837 (0.0011) [2023-12-26 16:41:53,980][105620] Updated weights for policy 1, policy_version 183821 (0.0008) [2023-12-26 16:41:54,038][105620] Updated weights for policy 1, policy_version 183831 (0.0008) [2023-12-26 16:41:54,092][105620] Updated weights for policy 1, policy_version 183841 (0.0009) [2023-12-26 16:41:54,656][105692] Updated weights for policy 0, policy_version 182847 (0.0007) [2023-12-26 16:41:54,713][105692] Updated weights for policy 0, policy_version 182857 (0.0008) [2023-12-26 16:41:54,759][105692] Updated weights for policy 0, policy_version 182867 (0.0008) [2023-12-26 16:41:54,826][105620] Updated weights for policy 1, policy_version 183851 (0.0008) [2023-12-26 16:41:54,876][105620] Updated weights for policy 1, policy_version 183861 (0.0008) [2023-12-26 16:41:54,933][105620] Updated weights for policy 1, policy_version 183871 (0.0008) [2023-12-26 16:41:55,509][105692] Updated weights for policy 0, policy_version 182877 (0.0008) [2023-12-26 16:41:55,566][105692] Updated weights for policy 0, policy_version 182887 (0.0006) [2023-12-26 16:41:55,626][105692] Updated weights for policy 0, policy_version 182897 (0.0009) [2023-12-26 16:41:55,666][105620] Updated weights for policy 1, policy_version 183881 (0.0009) [2023-12-26 16:41:55,726][105620] Updated weights for policy 1, policy_version 183891 (0.0009) [2023-12-26 16:41:55,771][105620] Updated weights for policy 1, policy_version 183901 (0.0008) [2023-12-26 16:41:55,820][105620] Updated weights for policy 1, policy_version 183911 (0.0008) [2023-12-26 16:41:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 93921280. Throughput: 0: 9866.5, 1: 9893.6. Samples: 93928232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-12-26 16:41:56,063][104569] Avg episode reward: [(0, '9169.767'), (1, '9091.631')] [2023-12-26 16:41:56,237][105692] Updated weights for policy 0, policy_version 182907 (0.0008) [2023-12-26 16:41:56,299][105692] Updated weights for policy 0, policy_version 182917 (0.0005) [2023-12-26 16:41:56,367][105692] Updated weights for policy 0, policy_version 182927 (0.0007) [2023-12-26 16:41:56,531][105620] Updated weights for policy 1, policy_version 183921 (0.0010) [2023-12-26 16:41:56,591][105620] Updated weights for policy 1, policy_version 183931 (0.0010) [2023-12-26 16:41:56,649][105620] Updated weights for policy 1, policy_version 183941 (0.0010) [2023-12-26 16:41:56,887][105692] Updated weights for policy 0, policy_version 182937 (0.0006) [2023-12-26 16:41:56,945][105692] Updated weights for policy 0, policy_version 182947 (0.0005) [2023-12-26 16:41:57,003][105692] Updated weights for policy 0, policy_version 182957 (0.0005) [2023-12-26 16:41:57,049][105692] Updated weights for policy 0, policy_version 182967 (0.0005) [2023-12-26 16:41:57,231][105620] Updated weights for policy 1, policy_version 183951 (0.0010) [2023-12-26 16:41:57,279][105620] Updated weights for policy 1, policy_version 183961 (0.0010) [2023-12-26 16:41:57,324][105620] Updated weights for policy 1, policy_version 183971 (0.0008) [2023-12-26 16:41:57,558][105692] Updated weights for policy 0, policy_version 182977 (0.0005) [2023-12-26 16:41:57,608][105692] Updated weights for policy 0, policy_version 182987 (0.0005) [2023-12-26 16:41:57,672][105692] Updated weights for policy 0, policy_version 182997 (0.0005) [2023-12-26 16:41:57,996][105620] Updated weights for policy 1, policy_version 183981 (0.0009) [2023-12-26 16:41:58,045][105620] Updated weights for policy 1, policy_version 183991 (0.0010) [2023-12-26 16:41:58,110][105620] Updated weights for policy 1, policy_version 184001 (0.0009) [2023-12-26 16:41:58,218][105692] Updated weights for policy 0, policy_version 183007 (0.0009) [2023-12-26 16:41:58,280][105692] Updated weights for policy 0, policy_version 183017 (0.0011) [2023-12-26 16:41:58,339][105692] Updated weights for policy 0, policy_version 183027 (0.0011) [2023-12-26 16:41:58,919][105620] Updated weights for policy 1, policy_version 184011 (0.0009) [2023-12-26 16:41:58,975][105620] Updated weights for policy 1, policy_version 184021 (0.0010) [2023-12-26 16:41:59,038][105620] Updated weights for policy 1, policy_version 184031 (0.0010) [2023-12-26 16:41:59,143][105692] Updated weights for policy 0, policy_version 183037 (0.0009) [2023-12-26 16:41:59,201][105692] Updated weights for policy 0, policy_version 183047 (0.0008) [2023-12-26 16:41:59,273][105692] Updated weights for policy 0, policy_version 183057 (0.0007) [2023-12-26 16:41:59,711][105620] Updated weights for policy 1, policy_version 184041 (0.0010) [2023-12-26 16:41:59,764][105620] Updated weights for policy 1, policy_version 184051 (0.0009) [2023-12-26 16:41:59,815][105620] Updated weights for policy 1, policy_version 184061 (0.0010) [2023-12-26 16:41:59,877][105620] Updated weights for policy 1, policy_version 184071 (0.0010) [2023-12-26 16:42:00,032][105692] Updated weights for policy 0, policy_version 183067 (0.0009) [2023-12-26 16:42:00,085][105692] Updated weights for policy 0, policy_version 183078 (0.0010) [2023-12-26 16:42:00,142][105692] Updated weights for policy 0, policy_version 183088 (0.0014) [2023-12-26 16:42:00,465][105620] Updated weights for policy 1, policy_version 184081 (0.0008) [2023-12-26 16:42:00,517][105620] Updated weights for policy 1, policy_version 184091 (0.0011) [2023-12-26 16:42:00,576][105620] Updated weights for policy 1, policy_version 184101 (0.0010) [2023-12-26 16:42:00,890][105692] Updated weights for policy 0, policy_version 183098 (0.0008) [2023-12-26 16:42:00,952][105692] Updated weights for policy 0, policy_version 183108 (0.0005) [2023-12-26 16:42:01,002][105692] Updated weights for policy 0, policy_version 183118 (0.0005) [2023-12-26 16:42:01,058][105692] Updated weights for policy 0, policy_version 183128 (0.0008) [2023-12-26 16:42:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.4, 300 sec: 19688.6). Total num frames: 94027776. Throughput: 0: 10061.9, 1: 9886.8. Samples: 93993800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-12-26 16:42:01,062][104569] Avg episode reward: [(0, '9258.163'), (1, '9003.057')] [2023-12-26 16:42:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000183128_46891008.pth... [2023-12-26 16:42:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000184104_47136768.pth... [2023-12-26 16:42:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000181944_46587904.pth [2023-12-26 16:42:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000182952_46841856.pth [2023-12-26 16:42:01,263][105620] Updated weights for policy 1, policy_version 184111 (0.0008) [2023-12-26 16:42:01,320][105620] Updated weights for policy 1, policy_version 184121 (0.0006) [2023-12-26 16:42:01,389][105620] Updated weights for policy 1, policy_version 184131 (0.0011) [2023-12-26 16:42:01,756][105692] Updated weights for policy 0, policy_version 183138 (0.0009) [2023-12-26 16:42:01,811][105692] Updated weights for policy 0, policy_version 183149 (0.0010) [2023-12-26 16:42:01,865][105692] Updated weights for policy 0, policy_version 183159 (0.0010) [2023-12-26 16:42:02,031][105620] Updated weights for policy 1, policy_version 184141 (0.0009) [2023-12-26 16:42:02,093][105620] Updated weights for policy 1, policy_version 184151 (0.0010) [2023-12-26 16:42:02,158][105620] Updated weights for policy 1, policy_version 184161 (0.0010) [2023-12-26 16:42:02,683][105692] Updated weights for policy 0, policy_version 183169 (0.0008) [2023-12-26 16:42:02,742][105692] Updated weights for policy 0, policy_version 183179 (0.0008) [2023-12-26 16:42:02,797][105692] Updated weights for policy 0, policy_version 183189 (0.0006) [2023-12-26 16:42:02,835][105620] Updated weights for policy 1, policy_version 184171 (0.0009) [2023-12-26 16:42:02,900][105620] Updated weights for policy 1, policy_version 184181 (0.0010) [2023-12-26 16:42:02,947][105620] Updated weights for policy 1, policy_version 184191 (0.0010) [2023-12-26 16:42:03,528][105692] Updated weights for policy 0, policy_version 183199 (0.0007) [2023-12-26 16:42:03,585][105692] Updated weights for policy 0, policy_version 183209 (0.0008) [2023-12-26 16:42:03,639][105692] Updated weights for policy 0, policy_version 183219 (0.0008) [2023-12-26 16:42:03,641][105620] Updated weights for policy 1, policy_version 184201 (0.0010) [2023-12-26 16:42:03,702][105620] Updated weights for policy 1, policy_version 184211 (0.0010) [2023-12-26 16:42:03,755][105620] Updated weights for policy 1, policy_version 184221 (0.0010) [2023-12-26 16:42:03,802][105620] Updated weights for policy 1, policy_version 184231 (0.0010) [2023-12-26 16:42:04,435][105692] Updated weights for policy 0, policy_version 183229 (0.0007) [2023-12-26 16:42:04,493][105692] Updated weights for policy 0, policy_version 183239 (0.0007) [2023-12-26 16:42:04,525][105620] Updated weights for policy 1, policy_version 184241 (0.0010) [2023-12-26 16:42:04,555][105692] Updated weights for policy 0, policy_version 183249 (0.0009) [2023-12-26 16:42:04,588][105620] Updated weights for policy 1, policy_version 184251 (0.0010) [2023-12-26 16:42:04,657][105620] Updated weights for policy 1, policy_version 184261 (0.0006) [2023-12-26 16:42:05,270][105620] Updated weights for policy 1, policy_version 184271 (0.0005) [2023-12-26 16:42:05,335][105620] Updated weights for policy 1, policy_version 184281 (0.0008) [2023-12-26 16:42:05,341][105692] Updated weights for policy 0, policy_version 183259 (0.0007) [2023-12-26 16:42:05,394][105620] Updated weights for policy 1, policy_version 184291 (0.0007) [2023-12-26 16:42:05,405][105692] Updated weights for policy 0, policy_version 183269 (0.0007) [2023-12-26 16:42:05,467][105692] Updated weights for policy 0, policy_version 183279 (0.0005) [2023-12-26 16:42:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 94117888. Throughput: 0: 9912.7, 1: 9916.5. Samples: 94110808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-12-26 16:42:06,063][104569] Avg episode reward: [(0, '9347.424'), (1, '9091.405')] [2023-12-26 16:42:06,077][105620] Updated weights for policy 1, policy_version 184301 (0.0010) [2023-12-26 16:42:06,106][105692] Updated weights for policy 0, policy_version 183289 (0.0005) [2023-12-26 16:42:06,145][105620] Updated weights for policy 1, policy_version 184311 (0.0011) [2023-12-26 16:42:06,167][105692] Updated weights for policy 0, policy_version 183299 (0.0007) [2023-12-26 16:42:06,201][105620] Updated weights for policy 1, policy_version 184321 (0.0011) [2023-12-26 16:42:06,230][105692] Updated weights for policy 0, policy_version 183309 (0.0006) [2023-12-26 16:42:06,289][105692] Updated weights for policy 0, policy_version 183319 (0.0008) [2023-12-26 16:42:06,834][105620] Updated weights for policy 1, policy_version 184331 (0.0010) [2023-12-26 16:42:06,882][105620] Updated weights for policy 1, policy_version 184341 (0.0010) [2023-12-26 16:42:06,927][105620] Updated weights for policy 1, policy_version 184351 (0.0010) [2023-12-26 16:42:07,077][105692] Updated weights for policy 0, policy_version 183329 (0.0011) [2023-12-26 16:42:07,126][105692] Updated weights for policy 0, policy_version 183339 (0.0010) [2023-12-26 16:42:07,185][105692] Updated weights for policy 0, policy_version 183349 (0.0010) [2023-12-26 16:42:07,610][105620] Updated weights for policy 1, policy_version 184361 (0.0010) [2023-12-26 16:42:07,666][105620] Updated weights for policy 1, policy_version 184371 (0.0005) [2023-12-26 16:42:07,722][105620] Updated weights for policy 1, policy_version 184381 (0.0005) [2023-12-26 16:42:07,776][105620] Updated weights for policy 1, policy_version 184391 (0.0005) [2023-12-26 16:42:07,777][105692] Updated weights for policy 0, policy_version 183359 (0.0007) [2023-12-26 16:42:07,835][105692] Updated weights for policy 0, policy_version 183369 (0.0005) [2023-12-26 16:42:07,882][105692] Updated weights for policy 0, policy_version 183379 (0.0007) [2023-12-26 16:42:08,277][105620] Updated weights for policy 1, policy_version 184401 (0.0005) [2023-12-26 16:42:08,330][105620] Updated weights for policy 1, policy_version 184411 (0.0006) [2023-12-26 16:42:08,390][105620] Updated weights for policy 1, policy_version 184421 (0.0010) [2023-12-26 16:42:08,599][105692] Updated weights for policy 0, policy_version 183389 (0.0010) [2023-12-26 16:42:08,650][105692] Updated weights for policy 0, policy_version 183399 (0.0010) [2023-12-26 16:42:08,709][105692] Updated weights for policy 0, policy_version 183409 (0.0010) [2023-12-26 16:42:09,107][105620] Updated weights for policy 1, policy_version 184431 (0.0010) [2023-12-26 16:42:09,162][105620] Updated weights for policy 1, policy_version 184441 (0.0011) [2023-12-26 16:42:09,211][105620] Updated weights for policy 1, policy_version 184451 (0.0010) [2023-12-26 16:42:09,509][105692] Updated weights for policy 0, policy_version 183419 (0.0011) [2023-12-26 16:42:09,573][105692] Updated weights for policy 0, policy_version 183429 (0.0011) [2023-12-26 16:42:09,626][105692] Updated weights for policy 0, policy_version 183439 (0.0011) [2023-12-26 16:42:10,026][105620] Updated weights for policy 1, policy_version 184461 (0.0011) [2023-12-26 16:42:10,090][105620] Updated weights for policy 1, policy_version 184471 (0.0010) [2023-12-26 16:42:10,153][105620] Updated weights for policy 1, policy_version 184481 (0.0011) [2023-12-26 16:42:10,376][105692] Updated weights for policy 0, policy_version 183449 (0.0010) [2023-12-26 16:42:10,427][105692] Updated weights for policy 0, policy_version 183459 (0.0009) [2023-12-26 16:42:10,472][105692] Updated weights for policy 0, policy_version 183469 (0.0010) [2023-12-26 16:42:10,521][105692] Updated weights for policy 0, policy_version 183479 (0.0010) [2023-12-26 16:42:10,916][105620] Updated weights for policy 1, policy_version 184491 (0.0011) [2023-12-26 16:42:10,979][105620] Updated weights for policy 1, policy_version 184501 (0.0011) [2023-12-26 16:42:11,045][105620] Updated weights for policy 1, policy_version 184511 (0.0009) [2023-12-26 16:42:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 94216192. Throughput: 0: 9914.0, 1: 10012.0. Samples: 94229996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-12-26 16:42:11,063][104569] Avg episode reward: [(0, '9346.389'), (1, '9354.557')] [2023-12-26 16:42:11,218][105692] Updated weights for policy 0, policy_version 183489 (0.0008) [2023-12-26 16:42:11,281][105692] Updated weights for policy 0, policy_version 183499 (0.0009) [2023-12-26 16:42:11,343][105692] Updated weights for policy 0, policy_version 183509 (0.0008) [2023-12-26 16:42:11,835][105620] Updated weights for policy 1, policy_version 184521 (0.0008) [2023-12-26 16:42:11,902][105620] Updated weights for policy 1, policy_version 184531 (0.0011) [2023-12-26 16:42:11,969][105620] Updated weights for policy 1, policy_version 184541 (0.0011) [2023-12-26 16:42:12,029][105620] Updated weights for policy 1, policy_version 184551 (0.0010) [2023-12-26 16:42:12,081][105692] Updated weights for policy 0, policy_version 183519 (0.0008) [2023-12-26 16:42:12,145][105692] Updated weights for policy 0, policy_version 183529 (0.0008) [2023-12-26 16:42:12,203][105692] Updated weights for policy 0, policy_version 183539 (0.0008) [2023-12-26 16:42:12,801][105620] Updated weights for policy 1, policy_version 184561 (0.0011) [2023-12-26 16:42:12,863][105620] Updated weights for policy 1, policy_version 184571 (0.0010) [2023-12-26 16:42:12,931][105620] Updated weights for policy 1, policy_version 184581 (0.0011) [2023-12-26 16:42:12,958][105692] Updated weights for policy 0, policy_version 183549 (0.0008) [2023-12-26 16:42:13,014][105692] Updated weights for policy 0, policy_version 183559 (0.0008) [2023-12-26 16:42:13,071][105692] Updated weights for policy 0, policy_version 183569 (0.0007) [2023-12-26 16:42:13,624][105620] Updated weights for policy 1, policy_version 184591 (0.0010) [2023-12-26 16:42:13,675][105620] Updated weights for policy 1, policy_version 184601 (0.0010) [2023-12-26 16:42:13,723][105620] Updated weights for policy 1, policy_version 184611 (0.0010) [2023-12-26 16:42:13,849][105692] Updated weights for policy 0, policy_version 183579 (0.0008) [2023-12-26 16:42:13,901][105692] Updated weights for policy 0, policy_version 183589 (0.0007) [2023-12-26 16:42:13,952][105692] Updated weights for policy 0, policy_version 183599 (0.0008) [2023-12-26 16:42:14,442][105620] Updated weights for policy 1, policy_version 184621 (0.0010) [2023-12-26 16:42:14,494][105620] Updated weights for policy 1, policy_version 184631 (0.0010) [2023-12-26 16:42:14,538][105620] Updated weights for policy 1, policy_version 184641 (0.0010) [2023-12-26 16:42:14,696][105692] Updated weights for policy 0, policy_version 183609 (0.0005) [2023-12-26 16:42:14,749][105692] Updated weights for policy 0, policy_version 183619 (0.0006) [2023-12-26 16:42:14,815][105692] Updated weights for policy 0, policy_version 183629 (0.0007) [2023-12-26 16:42:14,883][105692] Updated weights for policy 0, policy_version 183639 (0.0006) [2023-12-26 16:42:15,314][105620] Updated weights for policy 1, policy_version 184651 (0.0010) [2023-12-26 16:42:15,371][105620] Updated weights for policy 1, policy_version 184661 (0.0011) [2023-12-26 16:42:15,423][105620] Updated weights for policy 1, policy_version 184671 (0.0011) [2023-12-26 16:42:15,454][105692] Updated weights for policy 0, policy_version 183649 (0.0006) [2023-12-26 16:42:15,509][105692] Updated weights for policy 0, policy_version 183659 (0.0007) [2023-12-26 16:42:15,556][105692] Updated weights for policy 0, policy_version 183669 (0.0007) [2023-12-26 16:42:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 94314496. Throughput: 0: 9842.9, 1: 9943.5. Samples: 94286008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-12-26 16:42:16,063][104569] Avg episode reward: [(0, '9071.979'), (1, '9261.437')] [2023-12-26 16:42:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000183672_47030272.pth... [2023-12-26 16:42:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000184680_47284224.pth... [2023-12-26 16:42:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000182520_46735360.pth [2023-12-26 16:42:16,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000183528_46989312.pth [2023-12-26 16:42:16,182][105620] Updated weights for policy 1, policy_version 184681 (0.0010) [2023-12-26 16:42:16,215][105692] Updated weights for policy 0, policy_version 183679 (0.0009) [2023-12-26 16:42:16,245][105620] Updated weights for policy 1, policy_version 184691 (0.0011) [2023-12-26 16:42:16,272][105692] Updated weights for policy 0, policy_version 183689 (0.0006) [2023-12-26 16:42:16,305][105620] Updated weights for policy 1, policy_version 184701 (0.0011) [2023-12-26 16:42:16,324][105692] Updated weights for policy 0, policy_version 183699 (0.0007) [2023-12-26 16:42:16,361][105620] Updated weights for policy 1, policy_version 184711 (0.0010) [2023-12-26 16:42:17,047][105620] Updated weights for policy 1, policy_version 184721 (0.0007) [2023-12-26 16:42:17,091][105692] Updated weights for policy 0, policy_version 183709 (0.0008) [2023-12-26 16:42:17,104][105620] Updated weights for policy 1, policy_version 184731 (0.0007) [2023-12-26 16:42:17,148][105692] Updated weights for policy 0, policy_version 183719 (0.0009) [2023-12-26 16:42:17,152][105620] Updated weights for policy 1, policy_version 184741 (0.0005) [2023-12-26 16:42:17,216][105692] Updated weights for policy 0, policy_version 183729 (0.0009) [2023-12-26 16:42:17,773][105620] Updated weights for policy 1, policy_version 184751 (0.0005) [2023-12-26 16:42:17,831][105620] Updated weights for policy 1, policy_version 184761 (0.0005) [2023-12-26 16:42:17,892][105620] Updated weights for policy 1, policy_version 184771 (0.0005) [2023-12-26 16:42:18,007][105692] Updated weights for policy 0, policy_version 183739 (0.0009) [2023-12-26 16:42:18,074][105692] Updated weights for policy 0, policy_version 183749 (0.0010) [2023-12-26 16:42:18,121][105692] Updated weights for policy 0, policy_version 183759 (0.0005) [2023-12-26 16:42:18,566][105620] Updated weights for policy 1, policy_version 184781 (0.0007) [2023-12-26 16:42:18,634][105620] Updated weights for policy 1, policy_version 184791 (0.0005) [2023-12-26 16:42:18,700][105620] Updated weights for policy 1, policy_version 184801 (0.0009) [2023-12-26 16:42:18,768][105692] Updated weights for policy 0, policy_version 183769 (0.0008) [2023-12-26 16:42:18,835][105692] Updated weights for policy 0, policy_version 183779 (0.0008) [2023-12-26 16:42:18,900][105692] Updated weights for policy 0, policy_version 183789 (0.0008) [2023-12-26 16:42:18,964][105692] Updated weights for policy 0, policy_version 183799 (0.0008) [2023-12-26 16:42:19,513][105620] Updated weights for policy 1, policy_version 184811 (0.0009) [2023-12-26 16:42:19,585][105620] Updated weights for policy 1, policy_version 184821 (0.0006) [2023-12-26 16:42:19,648][105620] Updated weights for policy 1, policy_version 184831 (0.0006) [2023-12-26 16:42:19,652][105692] Updated weights for policy 0, policy_version 183809 (0.0010) [2023-12-26 16:42:19,718][105692] Updated weights for policy 0, policy_version 183819 (0.0011) [2023-12-26 16:42:19,775][105692] Updated weights for policy 0, policy_version 183829 (0.0010) [2023-12-26 16:42:20,216][105620] Updated weights for policy 1, policy_version 184841 (0.0005) [2023-12-26 16:42:20,269][105620] Updated weights for policy 1, policy_version 184851 (0.0006) [2023-12-26 16:42:20,321][105620] Updated weights for policy 1, policy_version 184861 (0.0009) [2023-12-26 16:42:20,375][105620] Updated weights for policy 1, policy_version 184871 (0.0009) [2023-12-26 16:42:20,515][105692] Updated weights for policy 0, policy_version 183839 (0.0008) [2023-12-26 16:42:20,572][105692] Updated weights for policy 0, policy_version 183849 (0.0006) [2023-12-26 16:42:20,641][105692] Updated weights for policy 0, policy_version 183859 (0.0009) [2023-12-26 16:42:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 94412800. Throughput: 0: 9787.8, 1: 9988.5. Samples: 94404912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-12-26 16:42:21,062][104569] Avg episode reward: [(0, '8980.030'), (1, '9169.800')] [2023-12-26 16:42:21,132][105620] Updated weights for policy 1, policy_version 184881 (0.0009) [2023-12-26 16:42:21,192][105620] Updated weights for policy 1, policy_version 184891 (0.0009) [2023-12-26 16:42:21,261][105620] Updated weights for policy 1, policy_version 184901 (0.0008) [2023-12-26 16:42:21,390][105692] Updated weights for policy 0, policy_version 183869 (0.0008) [2023-12-26 16:42:21,457][105692] Updated weights for policy 0, policy_version 183879 (0.0007) [2023-12-26 16:42:21,523][105692] Updated weights for policy 0, policy_version 183889 (0.0009) [2023-12-26 16:42:22,056][105620] Updated weights for policy 1, policy_version 184911 (0.0010) [2023-12-26 16:42:22,109][105620] Updated weights for policy 1, policy_version 184921 (0.0011) [2023-12-26 16:42:22,162][105620] Updated weights for policy 1, policy_version 184931 (0.0011) [2023-12-26 16:42:22,231][105692] Updated weights for policy 0, policy_version 183899 (0.0007) [2023-12-26 16:42:22,298][105692] Updated weights for policy 0, policy_version 183909 (0.0008) [2023-12-26 16:42:22,362][105692] Updated weights for policy 0, policy_version 183919 (0.0007) [2023-12-26 16:42:22,955][105620] Updated weights for policy 1, policy_version 184941 (0.0011) [2023-12-26 16:42:23,004][105620] Updated weights for policy 1, policy_version 184951 (0.0011) [2023-12-26 16:42:23,054][105620] Updated weights for policy 1, policy_version 184961 (0.0011) [2023-12-26 16:42:23,116][105692] Updated weights for policy 0, policy_version 183929 (0.0006) [2023-12-26 16:42:23,168][105692] Updated weights for policy 0, policy_version 183939 (0.0008) [2023-12-26 16:42:23,225][105692] Updated weights for policy 0, policy_version 183949 (0.0007) [2023-12-26 16:42:23,281][105692] Updated weights for policy 0, policy_version 183959 (0.0009) [2023-12-26 16:42:23,692][105620] Updated weights for policy 1, policy_version 184971 (0.0009) [2023-12-26 16:42:23,745][105620] Updated weights for policy 1, policy_version 184981 (0.0005) [2023-12-26 16:42:23,793][105620] Updated weights for policy 1, policy_version 184991 (0.0009) [2023-12-26 16:42:24,162][105692] Updated weights for policy 0, policy_version 183969 (0.0009) [2023-12-26 16:42:24,221][105692] Updated weights for policy 0, policy_version 183979 (0.0010) [2023-12-26 16:42:24,288][105692] Updated weights for policy 0, policy_version 183989 (0.0009) [2023-12-26 16:42:24,390][105620] Updated weights for policy 1, policy_version 185001 (0.0010) [2023-12-26 16:42:24,449][105620] Updated weights for policy 1, policy_version 185011 (0.0008) [2023-12-26 16:42:24,511][105620] Updated weights for policy 1, policy_version 185021 (0.0007) [2023-12-26 16:42:24,577][105620] Updated weights for policy 1, policy_version 185031 (0.0005) [2023-12-26 16:42:25,064][105692] Updated weights for policy 0, policy_version 183999 (0.0007) [2023-12-26 16:42:25,119][105692] Updated weights for policy 0, policy_version 184009 (0.0011) [2023-12-26 16:42:25,171][105692] Updated weights for policy 0, policy_version 184019 (0.0010) [2023-12-26 16:42:25,232][105620] Updated weights for policy 1, policy_version 185041 (0.0010) [2023-12-26 16:42:25,284][105620] Updated weights for policy 1, policy_version 185051 (0.0010) [2023-12-26 16:42:25,339][105620] Updated weights for policy 1, policy_version 185061 (0.0010) [2023-12-26 16:42:25,911][105692] Updated weights for policy 0, policy_version 184029 (0.0010) [2023-12-26 16:42:25,959][105692] Updated weights for policy 0, policy_version 184039 (0.0010) [2023-12-26 16:42:26,020][105692] Updated weights for policy 0, policy_version 184049 (0.0010) [2023-12-26 16:42:26,050][105620] Updated weights for policy 1, policy_version 185071 (0.0010) [2023-12-26 16:42:26,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 94511104. Throughput: 0: 9733.4, 1: 10010.6. Samples: 94518876. Policy #0 lag: (min: 26.0, avg: 35.4, max: 58.0) [2023-12-26 16:42:26,062][104569] Avg episode reward: [(0, '9253.892'), (1, '8730.993')] [2023-12-26 16:42:26,108][105620] Updated weights for policy 1, policy_version 185081 (0.0010) [2023-12-26 16:42:26,179][105620] Updated weights for policy 1, policy_version 185091 (0.0005) [2023-12-26 16:42:26,706][105620] Updated weights for policy 1, policy_version 185101 (0.0006) [2023-12-26 16:42:26,728][105692] Updated weights for policy 0, policy_version 184059 (0.0010) [2023-12-26 16:42:26,756][105620] Updated weights for policy 1, policy_version 185111 (0.0005) [2023-12-26 16:42:26,783][105692] Updated weights for policy 0, policy_version 184069 (0.0010) [2023-12-26 16:42:26,809][105620] Updated weights for policy 1, policy_version 185121 (0.0006) [2023-12-26 16:42:26,842][105692] Updated weights for policy 0, policy_version 184079 (0.0010) [2023-12-26 16:42:27,355][105620] Updated weights for policy 1, policy_version 185131 (0.0008) [2023-12-26 16:42:27,408][105620] Updated weights for policy 1, policy_version 185141 (0.0005) [2023-12-26 16:42:27,462][105620] Updated weights for policy 1, policy_version 185151 (0.0005) [2023-12-26 16:42:27,582][105692] Updated weights for policy 0, policy_version 184089 (0.0011) [2023-12-26 16:42:27,642][105692] Updated weights for policy 0, policy_version 184099 (0.0010) [2023-12-26 16:42:27,697][105692] Updated weights for policy 0, policy_version 184109 (0.0010) [2023-12-26 16:42:27,752][105692] Updated weights for policy 0, policy_version 184119 (0.0010) [2023-12-26 16:42:28,222][105620] Updated weights for policy 1, policy_version 185161 (0.0007) [2023-12-26 16:42:28,292][105620] Updated weights for policy 1, policy_version 185171 (0.0009) [2023-12-26 16:42:28,356][105620] Updated weights for policy 1, policy_version 185181 (0.0008) [2023-12-26 16:42:28,369][105692] Updated weights for policy 0, policy_version 184129 (0.0008) [2023-12-26 16:42:28,417][105620] Updated weights for policy 1, policy_version 185191 (0.0006) [2023-12-26 16:42:28,426][105692] Updated weights for policy 0, policy_version 184139 (0.0006) [2023-12-26 16:42:28,481][105692] Updated weights for policy 0, policy_version 184149 (0.0010) [2023-12-26 16:42:29,035][105620] Updated weights for policy 1, policy_version 185201 (0.0005) [2023-12-26 16:42:29,086][105620] Updated weights for policy 1, policy_version 185211 (0.0005) [2023-12-26 16:42:29,143][105620] Updated weights for policy 1, policy_version 185221 (0.0005) [2023-12-26 16:42:29,199][105692] Updated weights for policy 0, policy_version 184159 (0.0010) [2023-12-26 16:42:29,263][105692] Updated weights for policy 0, policy_version 184169 (0.0011) [2023-12-26 16:42:29,328][105692] Updated weights for policy 0, policy_version 184179 (0.0009) [2023-12-26 16:42:29,784][105620] Updated weights for policy 1, policy_version 185231 (0.0005) [2023-12-26 16:42:29,844][105620] Updated weights for policy 1, policy_version 185241 (0.0007) [2023-12-26 16:42:29,902][105620] Updated weights for policy 1, policy_version 185251 (0.0007) [2023-12-26 16:42:30,084][105692] Updated weights for policy 0, policy_version 184189 (0.0009) [2023-12-26 16:42:30,135][105692] Updated weights for policy 0, policy_version 184199 (0.0009) [2023-12-26 16:42:30,195][105692] Updated weights for policy 0, policy_version 184209 (0.0009) [2023-12-26 16:42:30,615][105620] Updated weights for policy 1, policy_version 185261 (0.0010) [2023-12-26 16:42:30,670][105620] Updated weights for policy 1, policy_version 185271 (0.0006) [2023-12-26 16:42:30,725][105620] Updated weights for policy 1, policy_version 185281 (0.0006) [2023-12-26 16:42:30,919][105692] Updated weights for policy 0, policy_version 184219 (0.0009) [2023-12-26 16:42:30,984][105692] Updated weights for policy 0, policy_version 184229 (0.0009) [2023-12-26 16:42:31,038][105692] Updated weights for policy 0, policy_version 184239 (0.0009) [2023-12-26 16:42:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 94609408. Throughput: 0: 9781.7, 1: 10089.3. Samples: 94581820. Policy #0 lag: (min: 26.0, avg: 35.4, max: 58.0) [2023-12-26 16:42:31,062][104569] Avg episode reward: [(0, '9254.321'), (1, '8823.763')] [2023-12-26 16:42:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000185288_47439872.pth... [2023-12-26 16:42:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000184104_47136768.pth [2023-12-26 16:42:31,097][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000184248_47177728.pth... [2023-12-26 16:42:31,101][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000183128_46891008.pth [2023-12-26 16:42:31,424][105620] Updated weights for policy 1, policy_version 185291 (0.0008) [2023-12-26 16:42:31,490][105620] Updated weights for policy 1, policy_version 185301 (0.0005) [2023-12-26 16:42:31,546][105620] Updated weights for policy 1, policy_version 185311 (0.0006) [2023-12-26 16:42:31,877][105692] Updated weights for policy 0, policy_version 184249 (0.0007) [2023-12-26 16:42:31,939][105692] Updated weights for policy 0, policy_version 184259 (0.0009) [2023-12-26 16:42:31,997][105692] Updated weights for policy 0, policy_version 184269 (0.0006) [2023-12-26 16:42:32,044][105692] Updated weights for policy 0, policy_version 184279 (0.0007) [2023-12-26 16:42:32,222][105620] Updated weights for policy 1, policy_version 185321 (0.0008) [2023-12-26 16:42:32,285][105620] Updated weights for policy 1, policy_version 185331 (0.0008) [2023-12-26 16:42:32,344][105620] Updated weights for policy 1, policy_version 185341 (0.0007) [2023-12-26 16:42:32,403][105620] Updated weights for policy 1, policy_version 185351 (0.0008) [2023-12-26 16:42:32,759][105692] Updated weights for policy 0, policy_version 184289 (0.0010) [2023-12-26 16:42:32,811][105692] Updated weights for policy 0, policy_version 184299 (0.0007) [2023-12-26 16:42:32,873][105692] Updated weights for policy 0, policy_version 184309 (0.0006) [2023-12-26 16:42:33,230][105620] Updated weights for policy 1, policy_version 185361 (0.0010) [2023-12-26 16:42:33,284][105620] Updated weights for policy 1, policy_version 185371 (0.0009) [2023-12-26 16:42:33,345][105620] Updated weights for policy 1, policy_version 185381 (0.0008) [2023-12-26 16:42:33,405][105692] Updated weights for policy 0, policy_version 184319 (0.0010) [2023-12-26 16:42:33,450][105692] Updated weights for policy 0, policy_version 184329 (0.0010) [2023-12-26 16:42:33,508][105692] Updated weights for policy 0, policy_version 184339 (0.0010) [2023-12-26 16:42:33,986][105620] Updated weights for policy 1, policy_version 185391 (0.0010) [2023-12-26 16:42:34,040][105620] Updated weights for policy 1, policy_version 185402 (0.0010) [2023-12-26 16:42:34,095][105620] Updated weights for policy 1, policy_version 185414 (0.0010) [2023-12-26 16:42:34,147][105692] Updated weights for policy 0, policy_version 184349 (0.0009) [2023-12-26 16:42:34,206][105692] Updated weights for policy 0, policy_version 184359 (0.0008) [2023-12-26 16:42:34,263][105692] Updated weights for policy 0, policy_version 184369 (0.0008) [2023-12-26 16:42:34,792][105620] Updated weights for policy 1, policy_version 185424 (0.0007) [2023-12-26 16:42:34,860][105620] Updated weights for policy 1, policy_version 185434 (0.0006) [2023-12-26 16:42:34,906][105620] Updated weights for policy 1, policy_version 185444 (0.0005) [2023-12-26 16:42:35,019][105692] Updated weights for policy 0, policy_version 184379 (0.0009) [2023-12-26 16:42:35,085][105692] Updated weights for policy 0, policy_version 184389 (0.0008) [2023-12-26 16:42:35,149][105692] Updated weights for policy 0, policy_version 184399 (0.0008) [2023-12-26 16:42:35,445][105620] Updated weights for policy 1, policy_version 185454 (0.0006) [2023-12-26 16:42:35,515][105620] Updated weights for policy 1, policy_version 185464 (0.0007) [2023-12-26 16:42:35,583][105620] Updated weights for policy 1, policy_version 185474 (0.0010) [2023-12-26 16:42:35,745][105692] Updated weights for policy 0, policy_version 184409 (0.0008) [2023-12-26 16:42:35,804][105692] Updated weights for policy 0, policy_version 184419 (0.0005) [2023-12-26 16:42:35,863][105692] Updated weights for policy 0, policy_version 184429 (0.0005) [2023-12-26 16:42:35,916][105692] Updated weights for policy 0, policy_version 184439 (0.0006) [2023-12-26 16:42:36,062][104569] Fps is (10 sec: 20479.3, 60 sec: 19933.8, 300 sec: 19688.6). Total num frames: 94715904. Throughput: 0: 9730.9, 1: 9976.7. Samples: 94701052. Policy #0 lag: (min: 26.0, avg: 35.4, max: 58.0) [2023-12-26 16:42:36,063][104569] Avg episode reward: [(0, '4436.005'), (1, '8854.013')] [2023-12-26 16:42:36,312][105620] Updated weights for policy 1, policy_version 185484 (0.0011) [2023-12-26 16:42:36,365][105620] Updated weights for policy 1, policy_version 185494 (0.0010) [2023-12-26 16:42:36,431][105620] Updated weights for policy 1, policy_version 185504 (0.0010) [2023-12-26 16:42:36,591][105692] Updated weights for policy 0, policy_version 184449 (0.0009) [2023-12-26 16:42:36,645][105692] Updated weights for policy 0, policy_version 184459 (0.0008) [2023-12-26 16:42:36,694][105692] Updated weights for policy 0, policy_version 184469 (0.0009) [2023-12-26 16:42:37,196][105620] Updated weights for policy 1, policy_version 185514 (0.0010) [2023-12-26 16:42:37,261][105620] Updated weights for policy 1, policy_version 185524 (0.0011) [2023-12-26 16:42:37,326][105620] Updated weights for policy 1, policy_version 185534 (0.0010) [2023-12-26 16:42:37,389][105620] Updated weights for policy 1, policy_version 185544 (0.0011) [2023-12-26 16:42:37,489][105692] Updated weights for policy 0, policy_version 184479 (0.0009) [2023-12-26 16:42:37,553][105692] Updated weights for policy 0, policy_version 184489 (0.0008) [2023-12-26 16:42:37,607][105692] Updated weights for policy 0, policy_version 184499 (0.0009) [2023-12-26 16:42:38,146][105620] Updated weights for policy 1, policy_version 185554 (0.0009) [2023-12-26 16:42:38,207][105620] Updated weights for policy 1, policy_version 185564 (0.0009) [2023-12-26 16:42:38,269][105620] Updated weights for policy 1, policy_version 185574 (0.0009) [2023-12-26 16:42:38,317][105692] Updated weights for policy 0, policy_version 184509 (0.0007) [2023-12-26 16:42:38,384][105692] Updated weights for policy 0, policy_version 184519 (0.0008) [2023-12-26 16:42:38,444][105692] Updated weights for policy 0, policy_version 184529 (0.0008) [2023-12-26 16:42:39,001][105620] Updated weights for policy 1, policy_version 185584 (0.0009) [2023-12-26 16:42:39,060][105620] Updated weights for policy 1, policy_version 185594 (0.0009) [2023-12-26 16:42:39,123][105620] Updated weights for policy 1, policy_version 185604 (0.0009) [2023-12-26 16:42:39,205][105692] Updated weights for policy 0, policy_version 184539 (0.0008) [2023-12-26 16:42:39,272][105692] Updated weights for policy 0, policy_version 184549 (0.0009) [2023-12-26 16:42:39,336][105692] Updated weights for policy 0, policy_version 184559 (0.0005) [2023-12-26 16:42:39,874][105620] Updated weights for policy 1, policy_version 185614 (0.0009) [2023-12-26 16:42:39,933][105620] Updated weights for policy 1, policy_version 185624 (0.0008) [2023-12-26 16:42:39,988][105620] Updated weights for policy 1, policy_version 185634 (0.0009) [2023-12-26 16:42:40,055][105692] Updated weights for policy 0, policy_version 184569 (0.0009) [2023-12-26 16:42:40,108][105692] Updated weights for policy 0, policy_version 184579 (0.0007) [2023-12-26 16:42:40,177][105692] Updated weights for policy 0, policy_version 184589 (0.0005) [2023-12-26 16:42:40,237][105692] Updated weights for policy 0, policy_version 184599 (0.0008) [2023-12-26 16:42:40,749][105620] Updated weights for policy 1, policy_version 185644 (0.0009) [2023-12-26 16:42:40,815][105620] Updated weights for policy 1, policy_version 185654 (0.0009) [2023-12-26 16:42:40,877][105620] Updated weights for policy 1, policy_version 185664 (0.0009) [2023-12-26 16:42:40,928][105692] Updated weights for policy 0, policy_version 184609 (0.0009) [2023-12-26 16:42:40,985][105692] Updated weights for policy 0, policy_version 184619 (0.0008) [2023-12-26 16:42:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 94806016. Throughput: 0: 9734.0, 1: 10005.6. Samples: 94816512. Policy #0 lag: (min: 26.0, avg: 35.4, max: 58.0) [2023-12-26 16:42:41,063][104569] Avg episode reward: [(0, '7192.709'), (1, '8665.274')] [2023-12-26 16:42:41,066][105692] Updated weights for policy 0, policy_version 184629 (0.0009) [2023-12-26 16:42:41,631][105620] Updated weights for policy 1, policy_version 185674 (0.0009) [2023-12-26 16:42:41,697][105620] Updated weights for policy 1, policy_version 185684 (0.0009) [2023-12-26 16:42:41,764][105620] Updated weights for policy 1, policy_version 185694 (0.0009) [2023-12-26 16:42:41,819][105620] Updated weights for policy 1, policy_version 185704 (0.0008) [2023-12-26 16:42:41,858][105692] Updated weights for policy 0, policy_version 184639 (0.0010) [2023-12-26 16:42:41,918][105692] Updated weights for policy 0, policy_version 184649 (0.0011) [2023-12-26 16:42:41,975][105692] Updated weights for policy 0, policy_version 184659 (0.0011) [2023-12-26 16:42:42,595][105620] Updated weights for policy 1, policy_version 185714 (0.0008) [2023-12-26 16:42:42,648][105620] Updated weights for policy 1, policy_version 185724 (0.0008) [2023-12-26 16:42:42,708][105620] Updated weights for policy 1, policy_version 185734 (0.0008) [2023-12-26 16:42:42,735][105692] Updated weights for policy 0, policy_version 184669 (0.0011) [2023-12-26 16:42:42,791][105692] Updated weights for policy 0, policy_version 184679 (0.0010) [2023-12-26 16:42:42,849][105692] Updated weights for policy 0, policy_version 184689 (0.0010) [2023-12-26 16:42:43,459][105620] Updated weights for policy 1, policy_version 185744 (0.0009) [2023-12-26 16:42:43,517][105620] Updated weights for policy 1, policy_version 185754 (0.0010) [2023-12-26 16:42:43,575][105620] Updated weights for policy 1, policy_version 185764 (0.0010) [2023-12-26 16:42:43,591][105692] Updated weights for policy 0, policy_version 184699 (0.0010) [2023-12-26 16:42:43,649][105692] Updated weights for policy 0, policy_version 184709 (0.0010) [2023-12-26 16:42:43,707][105692] Updated weights for policy 0, policy_version 184719 (0.0010) [2023-12-26 16:42:44,293][105620] Updated weights for policy 1, policy_version 185774 (0.0009) [2023-12-26 16:42:44,351][105620] Updated weights for policy 1, policy_version 185784 (0.0009) [2023-12-26 16:42:44,418][105620] Updated weights for policy 1, policy_version 185794 (0.0005) [2023-12-26 16:42:44,433][105692] Updated weights for policy 0, policy_version 184729 (0.0010) [2023-12-26 16:42:44,485][105692] Updated weights for policy 0, policy_version 184739 (0.0008) [2023-12-26 16:42:44,543][105692] Updated weights for policy 0, policy_version 184749 (0.0009) [2023-12-26 16:42:44,597][105692] Updated weights for policy 0, policy_version 184760 (0.0010) [2023-12-26 16:42:45,020][105620] Updated weights for policy 1, policy_version 185804 (0.0007) [2023-12-26 16:42:45,086][105620] Updated weights for policy 1, policy_version 185814 (0.0009) [2023-12-26 16:42:45,146][105620] Updated weights for policy 1, policy_version 185824 (0.0009) [2023-12-26 16:42:45,423][105692] Updated weights for policy 0, policy_version 184770 (0.0005) [2023-12-26 16:42:45,469][105692] Updated weights for policy 0, policy_version 184780 (0.0005) [2023-12-26 16:42:45,518][105692] Updated weights for policy 0, policy_version 184790 (0.0007) [2023-12-26 16:42:45,860][105620] Updated weights for policy 1, policy_version 185834 (0.0009) [2023-12-26 16:42:45,921][105620] Updated weights for policy 1, policy_version 185844 (0.0009) [2023-12-26 16:42:45,975][105620] Updated weights for policy 1, policy_version 185854 (0.0009) [2023-12-26 16:42:46,021][105620] Updated weights for policy 1, policy_version 185864 (0.0009) [2023-12-26 16:42:46,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19660.9, 300 sec: 19688.6). Total num frames: 94904320. Throughput: 0: 9563.2, 1: 9941.7. Samples: 94871520. Policy #0 lag: (min: 26.0, avg: 35.4, max: 58.0) [2023-12-26 16:42:46,062][104569] Avg episode reward: [(0, '7563.165'), (1, '8746.398')] [2023-12-26 16:42:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000185864_47587328.pth... [2023-12-26 16:42:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000184792_47316992.pth... [2023-12-26 16:42:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000184680_47284224.pth [2023-12-26 16:42:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000183672_47030272.pth [2023-12-26 16:42:46,259][105692] Updated weights for policy 0, policy_version 184800 (0.0009) [2023-12-26 16:42:46,312][105692] Updated weights for policy 0, policy_version 184810 (0.0010) [2023-12-26 16:42:46,367][105692] Updated weights for policy 0, policy_version 184822 (0.0010) [2023-12-26 16:42:46,616][105620] Updated weights for policy 1, policy_version 185874 (0.0007) [2023-12-26 16:42:46,680][105620] Updated weights for policy 1, policy_version 185884 (0.0009) [2023-12-26 16:42:46,739][105620] Updated weights for policy 1, policy_version 185894 (0.0009) [2023-12-26 16:42:47,193][105692] Updated weights for policy 0, policy_version 184832 (0.0009) [2023-12-26 16:42:47,252][105692] Updated weights for policy 0, policy_version 184842 (0.0008) [2023-12-26 16:42:47,312][105692] Updated weights for policy 0, policy_version 184852 (0.0007) [2023-12-26 16:42:47,490][105620] Updated weights for policy 1, policy_version 185904 (0.0010) [2023-12-26 16:42:47,539][105620] Updated weights for policy 1, policy_version 185914 (0.0010) [2023-12-26 16:42:47,598][105620] Updated weights for policy 1, policy_version 185924 (0.0007) [2023-12-26 16:42:48,075][105692] Updated weights for policy 0, policy_version 184862 (0.0008) [2023-12-26 16:42:48,126][105692] Updated weights for policy 0, policy_version 184872 (0.0008) [2023-12-26 16:42:48,183][105692] Updated weights for policy 0, policy_version 184883 (0.0010) [2023-12-26 16:42:48,224][105620] Updated weights for policy 1, policy_version 185934 (0.0006) [2023-12-26 16:42:48,283][105620] Updated weights for policy 1, policy_version 185944 (0.0006) [2023-12-26 16:42:48,344][105620] Updated weights for policy 1, policy_version 185954 (0.0009) [2023-12-26 16:42:49,017][105692] Updated weights for policy 0, policy_version 184893 (0.0008) [2023-12-26 16:42:49,064][105620] Updated weights for policy 1, policy_version 185964 (0.0009) [2023-12-26 16:42:49,069][105692] Updated weights for policy 0, policy_version 184903 (0.0009) [2023-12-26 16:42:49,118][105692] Updated weights for policy 0, policy_version 184913 (0.0006) [2023-12-26 16:42:49,120][105620] Updated weights for policy 1, policy_version 185974 (0.0010) [2023-12-26 16:42:49,182][105620] Updated weights for policy 1, policy_version 185984 (0.0010) [2023-12-26 16:42:49,842][105692] Updated weights for policy 0, policy_version 184923 (0.0006) [2023-12-26 16:42:49,905][105692] Updated weights for policy 0, policy_version 184933 (0.0009) [2023-12-26 16:42:49,971][105692] Updated weights for policy 0, policy_version 184943 (0.0008) [2023-12-26 16:42:49,985][105620] Updated weights for policy 1, policy_version 185994 (0.0008) [2023-12-26 16:42:50,049][105620] Updated weights for policy 1, policy_version 186004 (0.0008) [2023-12-26 16:42:50,109][105620] Updated weights for policy 1, policy_version 186014 (0.0006) [2023-12-26 16:42:50,168][105620] Updated weights for policy 1, policy_version 186024 (0.0005) [2023-12-26 16:42:50,765][105692] Updated weights for policy 0, policy_version 184953 (0.0008) [2023-12-26 16:42:50,807][105620] Updated weights for policy 1, policy_version 186034 (0.0009) [2023-12-26 16:42:50,824][105692] Updated weights for policy 0, policy_version 184963 (0.0006) [2023-12-26 16:42:50,863][105620] Updated weights for policy 1, policy_version 186044 (0.0008) [2023-12-26 16:42:50,877][105692] Updated weights for policy 0, policy_version 184973 (0.0007) [2023-12-26 16:42:50,923][105692] Updated weights for policy 0, policy_version 184983 (0.0007) [2023-12-26 16:42:50,928][105620] Updated weights for policy 1, policy_version 186054 (0.0008) [2023-12-26 16:42:51,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19716.3). Total num frames: 95002624. Throughput: 0: 9549.5, 1: 9915.8. Samples: 94986740. Policy #0 lag: (min: 26.0, avg: 35.4, max: 58.0) [2023-12-26 16:42:51,062][104569] Avg episode reward: [(0, '8788.240'), (1, '9004.008')] [2023-12-26 16:42:51,665][105692] Updated weights for policy 0, policy_version 184993 (0.0010) [2023-12-26 16:42:51,688][105620] Updated weights for policy 1, policy_version 186064 (0.0008) [2023-12-26 16:42:51,724][105692] Updated weights for policy 0, policy_version 185003 (0.0008) [2023-12-26 16:42:51,740][105620] Updated weights for policy 1, policy_version 186074 (0.0006) [2023-12-26 16:42:51,778][105692] Updated weights for policy 0, policy_version 185013 (0.0008) [2023-12-26 16:42:51,800][105620] Updated weights for policy 1, policy_version 186084 (0.0009) [2023-12-26 16:42:52,495][105692] Updated weights for policy 0, policy_version 185023 (0.0008) [2023-12-26 16:42:52,505][105620] Updated weights for policy 1, policy_version 186094 (0.0011) [2023-12-26 16:42:52,559][105692] Updated weights for policy 0, policy_version 185033 (0.0006) [2023-12-26 16:42:52,566][105620] Updated weights for policy 1, policy_version 186104 (0.0011) [2023-12-26 16:42:52,616][105692] Updated weights for policy 0, policy_version 185043 (0.0006) [2023-12-26 16:42:52,630][105620] Updated weights for policy 1, policy_version 186114 (0.0007) [2023-12-26 16:42:53,184][105620] Updated weights for policy 1, policy_version 186124 (0.0008) [2023-12-26 16:42:53,243][105620] Updated weights for policy 1, policy_version 186134 (0.0010) [2023-12-26 16:42:53,297][105692] Updated weights for policy 0, policy_version 185053 (0.0006) [2023-12-26 16:42:53,303][105620] Updated weights for policy 1, policy_version 186144 (0.0010) [2023-12-26 16:42:53,349][105692] Updated weights for policy 0, policy_version 185063 (0.0005) [2023-12-26 16:42:53,398][105692] Updated weights for policy 0, policy_version 185073 (0.0007) [2023-12-26 16:42:53,962][105620] Updated weights for policy 1, policy_version 186154 (0.0009) [2023-12-26 16:42:54,025][105692] Updated weights for policy 0, policy_version 185083 (0.0008) [2023-12-26 16:42:54,029][105620] Updated weights for policy 1, policy_version 186164 (0.0006) [2023-12-26 16:42:54,078][105692] Updated weights for policy 0, policy_version 185093 (0.0007) [2023-12-26 16:42:54,087][105620] Updated weights for policy 1, policy_version 186174 (0.0006) [2023-12-26 16:42:54,139][105692] Updated weights for policy 0, policy_version 185103 (0.0006) [2023-12-26 16:42:54,146][105620] Updated weights for policy 1, policy_version 186184 (0.0008) [2023-12-26 16:42:54,783][105620] Updated weights for policy 1, policy_version 186194 (0.0005) [2023-12-26 16:42:54,837][105620] Updated weights for policy 1, policy_version 186204 (0.0005) [2023-12-26 16:42:54,893][105620] Updated weights for policy 1, policy_version 186214 (0.0005) [2023-12-26 16:42:54,928][105692] Updated weights for policy 0, policy_version 185113 (0.0008) [2023-12-26 16:42:54,987][105692] Updated weights for policy 0, policy_version 185123 (0.0008) [2023-12-26 16:42:55,053][105692] Updated weights for policy 0, policy_version 185133 (0.0009) [2023-12-26 16:42:55,121][105692] Updated weights for policy 0, policy_version 185143 (0.0009) [2023-12-26 16:42:55,551][105620] Updated weights for policy 1, policy_version 186224 (0.0009) [2023-12-26 16:42:55,618][105620] Updated weights for policy 1, policy_version 186234 (0.0010) [2023-12-26 16:42:55,680][105620] Updated weights for policy 1, policy_version 186244 (0.0010) [2023-12-26 16:42:55,884][105692] Updated weights for policy 0, policy_version 185153 (0.0009) [2023-12-26 16:42:55,942][105692] Updated weights for policy 0, policy_version 185163 (0.0008) [2023-12-26 16:42:56,003][105692] Updated weights for policy 0, policy_version 185173 (0.0008) [2023-12-26 16:42:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.9, 300 sec: 19688.6). Total num frames: 95100928. Throughput: 0: 9549.9, 1: 9922.7. Samples: 95106260. Policy #0 lag: (min: 26.0, avg: 35.4, max: 58.0) [2023-12-26 16:42:56,062][104569] Avg episode reward: [(0, '9254.096'), (1, '9084.135')] [2023-12-26 16:42:56,400][105620] Updated weights for policy 1, policy_version 186254 (0.0010) [2023-12-26 16:42:56,447][105620] Updated weights for policy 1, policy_version 186264 (0.0010) [2023-12-26 16:42:56,515][105620] Updated weights for policy 1, policy_version 186274 (0.0010) [2023-12-26 16:42:56,722][105692] Updated weights for policy 0, policy_version 185183 (0.0008) [2023-12-26 16:42:56,776][105692] Updated weights for policy 0, policy_version 185193 (0.0010) [2023-12-26 16:42:56,827][105692] Updated weights for policy 0, policy_version 185204 (0.0009) [2023-12-26 16:42:57,209][105620] Updated weights for policy 1, policy_version 186284 (0.0010) [2023-12-26 16:42:57,270][105620] Updated weights for policy 1, policy_version 186294 (0.0010) [2023-12-26 16:42:57,325][105620] Updated weights for policy 1, policy_version 186304 (0.0010) [2023-12-26 16:42:57,603][105692] Updated weights for policy 0, policy_version 185214 (0.0008) [2023-12-26 16:42:57,647][105692] Updated weights for policy 0, policy_version 185224 (0.0008) [2023-12-26 16:42:57,702][105692] Updated weights for policy 0, policy_version 185234 (0.0008) [2023-12-26 16:42:58,000][105620] Updated weights for policy 1, policy_version 186314 (0.0010) [2023-12-26 16:42:58,047][105620] Updated weights for policy 1, policy_version 186324 (0.0008) [2023-12-26 16:42:58,104][105620] Updated weights for policy 1, policy_version 186334 (0.0009) [2023-12-26 16:42:58,161][105620] Updated weights for policy 1, policy_version 186344 (0.0008) [2023-12-26 16:42:58,485][105692] Updated weights for policy 0, policy_version 185244 (0.0008) [2023-12-26 16:42:58,556][105692] Updated weights for policy 0, policy_version 185254 (0.0009) [2023-12-26 16:42:58,619][105692] Updated weights for policy 0, policy_version 185264 (0.0008) [2023-12-26 16:42:59,034][105620] Updated weights for policy 1, policy_version 186354 (0.0006) [2023-12-26 16:42:59,101][105620] Updated weights for policy 1, policy_version 186364 (0.0006) [2023-12-26 16:42:59,165][105620] Updated weights for policy 1, policy_version 186374 (0.0010) [2023-12-26 16:42:59,431][105692] Updated weights for policy 0, policy_version 185274 (0.0009) [2023-12-26 16:42:59,501][105692] Updated weights for policy 0, policy_version 185284 (0.0008) [2023-12-26 16:42:59,571][105692] Updated weights for policy 0, policy_version 185294 (0.0008) [2023-12-26 16:42:59,632][105692] Updated weights for policy 0, policy_version 185304 (0.0010) [2023-12-26 16:42:59,825][105620] Updated weights for policy 1, policy_version 186384 (0.0009) [2023-12-26 16:42:59,882][105620] Updated weights for policy 1, policy_version 186394 (0.0010) [2023-12-26 16:42:59,953][105620] Updated weights for policy 1, policy_version 186404 (0.0009) [2023-12-26 16:43:00,368][105692] Updated weights for policy 0, policy_version 185314 (0.0008) [2023-12-26 16:43:00,427][105692] Updated weights for policy 0, policy_version 185324 (0.0008) [2023-12-26 16:43:00,482][105692] Updated weights for policy 0, policy_version 185334 (0.0008) [2023-12-26 16:43:00,656][105620] Updated weights for policy 1, policy_version 186414 (0.0010) [2023-12-26 16:43:00,713][105620] Updated weights for policy 1, policy_version 186424 (0.0010) [2023-12-26 16:43:00,770][105620] Updated weights for policy 1, policy_version 186434 (0.0010) [2023-12-26 16:43:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 95191040. Throughput: 0: 9539.5, 1: 9944.8. Samples: 95162796. Policy #0 lag: (min: 26.0, avg: 35.4, max: 58.0) [2023-12-26 16:43:01,062][104569] Avg episode reward: [(0, '9342.465'), (1, '8994.771')] [2023-12-26 16:43:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000186440_47734784.pth... [2023-12-26 16:43:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000185336_47456256.pth... [2023-12-26 16:43:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000184248_47177728.pth [2023-12-26 16:43:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000185288_47439872.pth [2023-12-26 16:43:01,200][105692] Updated weights for policy 0, policy_version 185344 (0.0007) [2023-12-26 16:43:01,254][105692] Updated weights for policy 0, policy_version 185354 (0.0007) [2023-12-26 16:43:01,310][105692] Updated weights for policy 0, policy_version 185364 (0.0009) [2023-12-26 16:43:01,475][105620] Updated weights for policy 1, policy_version 186444 (0.0010) [2023-12-26 16:43:01,535][105620] Updated weights for policy 1, policy_version 186454 (0.0010) [2023-12-26 16:43:01,587][105620] Updated weights for policy 1, policy_version 186464 (0.0005) [2023-12-26 16:43:02,130][105692] Updated weights for policy 0, policy_version 185375 (0.0010) [2023-12-26 16:43:02,192][105692] Updated weights for policy 0, policy_version 185385 (0.0010) [2023-12-26 16:43:02,254][105620] Updated weights for policy 1, policy_version 186474 (0.0010) [2023-12-26 16:43:02,259][105692] Updated weights for policy 0, policy_version 185395 (0.0009) [2023-12-26 16:43:02,318][105620] Updated weights for policy 1, policy_version 186484 (0.0010) [2023-12-26 16:43:02,382][105620] Updated weights for policy 1, policy_version 186494 (0.0007) [2023-12-26 16:43:02,443][105620] Updated weights for policy 1, policy_version 186504 (0.0006) [2023-12-26 16:43:02,999][105620] Updated weights for policy 1, policy_version 186514 (0.0005) [2023-12-26 16:43:03,057][105620] Updated weights for policy 1, policy_version 186524 (0.0005) [2023-12-26 16:43:03,072][105692] Updated weights for policy 0, policy_version 185405 (0.0007) [2023-12-26 16:43:03,104][105620] Updated weights for policy 1, policy_version 186534 (0.0005) [2023-12-26 16:43:03,126][105692] Updated weights for policy 0, policy_version 185415 (0.0007) [2023-12-26 16:43:03,183][105692] Updated weights for policy 0, policy_version 185425 (0.0008) [2023-12-26 16:43:03,721][105620] Updated weights for policy 1, policy_version 186544 (0.0010) [2023-12-26 16:43:03,777][105620] Updated weights for policy 1, policy_version 186554 (0.0009) [2023-12-26 16:43:03,809][105692] Updated weights for policy 0, policy_version 185435 (0.0007) [2023-12-26 16:43:03,843][105620] Updated weights for policy 1, policy_version 186564 (0.0006) [2023-12-26 16:43:03,877][105692] Updated weights for policy 0, policy_version 185445 (0.0007) [2023-12-26 16:43:03,943][105692] Updated weights for policy 0, policy_version 185455 (0.0007) [2023-12-26 16:43:04,588][105620] Updated weights for policy 1, policy_version 186574 (0.0007) [2023-12-26 16:43:04,659][105620] Updated weights for policy 1, policy_version 186584 (0.0008) [2023-12-26 16:43:04,663][105692] Updated weights for policy 0, policy_version 185465 (0.0008) [2023-12-26 16:43:04,709][105692] Updated weights for policy 0, policy_version 185475 (0.0006) [2023-12-26 16:43:04,715][105620] Updated weights for policy 1, policy_version 186594 (0.0007) [2023-12-26 16:43:04,757][105692] Updated weights for policy 0, policy_version 185485 (0.0005) [2023-12-26 16:43:04,820][105692] Updated weights for policy 0, policy_version 185495 (0.0005) [2023-12-26 16:43:05,432][105620] Updated weights for policy 1, policy_version 186604 (0.0007) [2023-12-26 16:43:05,476][105692] Updated weights for policy 0, policy_version 185505 (0.0006) [2023-12-26 16:43:05,488][105620] Updated weights for policy 1, policy_version 186614 (0.0007) [2023-12-26 16:43:05,539][105692] Updated weights for policy 0, policy_version 185515 (0.0008) [2023-12-26 16:43:05,545][105620] Updated weights for policy 1, policy_version 186624 (0.0008) [2023-12-26 16:43:05,603][105692] Updated weights for policy 0, policy_version 185525 (0.0005) [2023-12-26 16:43:06,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 95289344. Throughput: 0: 9446.2, 1: 9977.3. Samples: 95278972. Policy #0 lag: (min: 26.0, avg: 35.4, max: 58.0) [2023-12-26 16:43:06,063][104569] Avg episode reward: [(0, '9342.754'), (1, '8899.551')] [2023-12-26 16:43:06,138][105620] Updated weights for policy 1, policy_version 186634 (0.0007) [2023-12-26 16:43:06,207][105620] Updated weights for policy 1, policy_version 186644 (0.0011) [2023-12-26 16:43:06,278][105620] Updated weights for policy 1, policy_version 186654 (0.0011) [2023-12-26 16:43:06,315][105692] Updated weights for policy 0, policy_version 185535 (0.0007) [2023-12-26 16:43:06,321][105586] KL-divergence is very high: 208.0281 [2023-12-26 16:43:06,346][105620] Updated weights for policy 1, policy_version 186664 (0.0011) [2023-12-26 16:43:06,376][105692] Updated weights for policy 0, policy_version 185545 (0.0007) [2023-12-26 16:43:06,443][105692] Updated weights for policy 0, policy_version 185555 (0.0008) [2023-12-26 16:43:07,069][105620] Updated weights for policy 1, policy_version 186674 (0.0011) [2023-12-26 16:43:07,135][105620] Updated weights for policy 1, policy_version 186684 (0.0010) [2023-12-26 16:43:07,188][105692] Updated weights for policy 0, policy_version 185565 (0.0010) [2023-12-26 16:43:07,190][105620] Updated weights for policy 1, policy_version 186694 (0.0010) [2023-12-26 16:43:07,235][105692] Updated weights for policy 0, policy_version 185575 (0.0007) [2023-12-26 16:43:07,281][105692] Updated weights for policy 0, policy_version 185585 (0.0008) [2023-12-26 16:43:07,928][105620] Updated weights for policy 1, policy_version 186704 (0.0010) [2023-12-26 16:43:07,987][105620] Updated weights for policy 1, policy_version 186714 (0.0010) [2023-12-26 16:43:08,043][105620] Updated weights for policy 1, policy_version 186724 (0.0010) [2023-12-26 16:43:08,057][105692] Updated weights for policy 0, policy_version 185595 (0.0008) [2023-12-26 16:43:08,124][105692] Updated weights for policy 0, policy_version 185605 (0.0008) [2023-12-26 16:43:08,180][105692] Updated weights for policy 0, policy_version 185615 (0.0008) [2023-12-26 16:43:08,682][105620] Updated weights for policy 1, policy_version 186734 (0.0009) [2023-12-26 16:43:08,734][105620] Updated weights for policy 1, policy_version 186744 (0.0010) [2023-12-26 16:43:08,779][105620] Updated weights for policy 1, policy_version 186754 (0.0010) [2023-12-26 16:43:08,927][105692] Updated weights for policy 0, policy_version 185625 (0.0008) [2023-12-26 16:43:08,983][105692] Updated weights for policy 0, policy_version 185635 (0.0007) [2023-12-26 16:43:09,039][105692] Updated weights for policy 0, policy_version 185645 (0.0008) [2023-12-26 16:43:09,096][105692] Updated weights for policy 0, policy_version 185655 (0.0008) [2023-12-26 16:43:09,553][105620] Updated weights for policy 1, policy_version 186764 (0.0010) [2023-12-26 16:43:09,613][105620] Updated weights for policy 1, policy_version 186774 (0.0008) [2023-12-26 16:43:09,680][105620] Updated weights for policy 1, policy_version 186784 (0.0007) [2023-12-26 16:43:09,833][105692] Updated weights for policy 0, policy_version 185665 (0.0006) [2023-12-26 16:43:09,897][105692] Updated weights for policy 0, policy_version 185675 (0.0008) [2023-12-26 16:43:09,969][105692] Updated weights for policy 0, policy_version 185685 (0.0009) [2023-12-26 16:43:10,389][105620] Updated weights for policy 1, policy_version 186794 (0.0009) [2023-12-26 16:43:10,449][105620] Updated weights for policy 1, policy_version 186804 (0.0008) [2023-12-26 16:43:10,514][105620] Updated weights for policy 1, policy_version 186814 (0.0009) [2023-12-26 16:43:10,568][105620] Updated weights for policy 1, policy_version 186824 (0.0009) [2023-12-26 16:43:10,748][105692] Updated weights for policy 0, policy_version 185695 (0.0009) [2023-12-26 16:43:10,804][105692] Updated weights for policy 0, policy_version 185705 (0.0009) [2023-12-26 16:43:10,852][105692] Updated weights for policy 0, policy_version 185715 (0.0009) [2023-12-26 16:43:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 95387648. Throughput: 0: 9510.2, 1: 9967.1. Samples: 95395356. Policy #0 lag: (min: 26.0, avg: 35.4, max: 58.0) [2023-12-26 16:43:11,063][104569] Avg episode reward: [(0, '9343.472'), (1, '9174.225')] [2023-12-26 16:43:11,390][105620] Updated weights for policy 1, policy_version 186834 (0.0009) [2023-12-26 16:43:11,452][105620] Updated weights for policy 1, policy_version 186844 (0.0008) [2023-12-26 16:43:11,517][105620] Updated weights for policy 1, policy_version 186854 (0.0009) [2023-12-26 16:43:11,604][105692] Updated weights for policy 0, policy_version 185725 (0.0009) [2023-12-26 16:43:11,675][105692] Updated weights for policy 0, policy_version 185735 (0.0010) [2023-12-26 16:43:11,745][105692] Updated weights for policy 0, policy_version 185745 (0.0009) [2023-12-26 16:43:12,274][105620] Updated weights for policy 1, policy_version 186864 (0.0008) [2023-12-26 16:43:12,342][105620] Updated weights for policy 1, policy_version 186874 (0.0009) [2023-12-26 16:43:12,411][105620] Updated weights for policy 1, policy_version 186884 (0.0009) [2023-12-26 16:43:12,499][105692] Updated weights for policy 0, policy_version 185755 (0.0007) [2023-12-26 16:43:12,563][105692] Updated weights for policy 0, policy_version 185765 (0.0008) [2023-12-26 16:43:12,626][105692] Updated weights for policy 0, policy_version 185775 (0.0009) [2023-12-26 16:43:13,075][105620] Updated weights for policy 1, policy_version 186894 (0.0006) [2023-12-26 16:43:13,134][105620] Updated weights for policy 1, policy_version 186904 (0.0006) [2023-12-26 16:43:13,193][105620] Updated weights for policy 1, policy_version 186914 (0.0006) [2023-12-26 16:43:13,333][105692] Updated weights for policy 0, policy_version 185785 (0.0011) [2023-12-26 16:43:13,404][105692] Updated weights for policy 0, policy_version 185795 (0.0010) [2023-12-26 16:43:13,458][105692] Updated weights for policy 0, policy_version 185805 (0.0010) [2023-12-26 16:43:13,502][105692] Updated weights for policy 0, policy_version 185815 (0.0008) [2023-12-26 16:43:13,875][105620] Updated weights for policy 1, policy_version 186924 (0.0007) [2023-12-26 16:43:13,940][105620] Updated weights for policy 1, policy_version 186934 (0.0005) [2023-12-26 16:43:14,008][105620] Updated weights for policy 1, policy_version 186944 (0.0006) [2023-12-26 16:43:14,082][105692] Updated weights for policy 0, policy_version 185825 (0.0009) [2023-12-26 16:43:14,147][105692] Updated weights for policy 0, policy_version 185835 (0.0011) [2023-12-26 16:43:14,209][105692] Updated weights for policy 0, policy_version 185845 (0.0011) [2023-12-26 16:43:14,697][105620] Updated weights for policy 1, policy_version 186954 (0.0007) [2023-12-26 16:43:14,752][105620] Updated weights for policy 1, policy_version 186964 (0.0010) [2023-12-26 16:43:14,816][105620] Updated weights for policy 1, policy_version 186974 (0.0011) [2023-12-26 16:43:14,856][105692] Updated weights for policy 0, policy_version 185855 (0.0010) [2023-12-26 16:43:14,875][105620] Updated weights for policy 1, policy_version 186984 (0.0011) [2023-12-26 16:43:14,915][105692] Updated weights for policy 0, policy_version 185865 (0.0011) [2023-12-26 16:43:14,977][105692] Updated weights for policy 0, policy_version 185875 (0.0009) [2023-12-26 16:43:15,641][105620] Updated weights for policy 1, policy_version 186994 (0.0007) [2023-12-26 16:43:15,692][105620] Updated weights for policy 1, policy_version 187004 (0.0009) [2023-12-26 16:43:15,714][105692] Updated weights for policy 0, policy_version 185885 (0.0009) [2023-12-26 16:43:15,756][105620] Updated weights for policy 1, policy_version 187014 (0.0006) [2023-12-26 16:43:15,768][105692] Updated weights for policy 0, policy_version 185895 (0.0006) [2023-12-26 16:43:15,837][105692] Updated weights for policy 0, policy_version 185905 (0.0011) [2023-12-26 16:43:16,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.4, 300 sec: 19660.8). Total num frames: 95485952. Throughput: 0: 9470.1, 1: 9875.8. Samples: 95452388. Policy #0 lag: (min: 26.0, avg: 35.4, max: 58.0) [2023-12-26 16:43:16,062][104569] Avg episode reward: [(0, '9344.946'), (1, '9357.265')] [2023-12-26 16:43:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000185912_47603712.pth... [2023-12-26 16:43:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000187016_47882240.pth... [2023-12-26 16:43:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000184792_47316992.pth [2023-12-26 16:43:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000185864_47587328.pth [2023-12-26 16:43:16,304][105620] Updated weights for policy 1, policy_version 187024 (0.0006) [2023-12-26 16:43:16,364][105620] Updated weights for policy 1, policy_version 187034 (0.0006) [2023-12-26 16:43:16,419][105620] Updated weights for policy 1, policy_version 187044 (0.0008) [2023-12-26 16:43:16,487][105692] Updated weights for policy 0, policy_version 185915 (0.0009) [2023-12-26 16:43:16,535][105692] Updated weights for policy 0, policy_version 185925 (0.0005) [2023-12-26 16:43:16,593][105692] Updated weights for policy 0, policy_version 185935 (0.0005) [2023-12-26 16:43:17,015][105620] Updated weights for policy 1, policy_version 187054 (0.0011) [2023-12-26 16:43:17,060][105620] Updated weights for policy 1, policy_version 187064 (0.0010) [2023-12-26 16:43:17,112][105620] Updated weights for policy 1, policy_version 187074 (0.0010) [2023-12-26 16:43:17,144][105692] Updated weights for policy 0, policy_version 185945 (0.0006) [2023-12-26 16:43:17,202][105692] Updated weights for policy 0, policy_version 185955 (0.0005) [2023-12-26 16:43:17,262][105692] Updated weights for policy 0, policy_version 185965 (0.0005) [2023-12-26 16:43:17,320][105692] Updated weights for policy 0, policy_version 185975 (0.0005) [2023-12-26 16:43:17,813][105692] Updated weights for policy 0, policy_version 185985 (0.0005) [2023-12-26 16:43:17,857][105620] Updated weights for policy 1, policy_version 187084 (0.0008) [2023-12-26 16:43:17,867][105692] Updated weights for policy 0, policy_version 185995 (0.0007) [2023-12-26 16:43:17,911][105692] Updated weights for policy 0, policy_version 186005 (0.0007) [2023-12-26 16:43:17,925][105620] Updated weights for policy 1, policy_version 187094 (0.0005) [2023-12-26 16:43:17,990][105620] Updated weights for policy 1, policy_version 187104 (0.0005) [2023-12-26 16:43:18,528][105620] Updated weights for policy 1, policy_version 187114 (0.0007) [2023-12-26 16:43:18,577][105620] Updated weights for policy 1, policy_version 187124 (0.0010) [2023-12-26 16:43:18,602][105692] Updated weights for policy 0, policy_version 186015 (0.0010) [2023-12-26 16:43:18,622][105620] Updated weights for policy 1, policy_version 187134 (0.0010) [2023-12-26 16:43:18,651][105692] Updated weights for policy 0, policy_version 186025 (0.0010) [2023-12-26 16:43:18,673][105620] Updated weights for policy 1, policy_version 187144 (0.0010) [2023-12-26 16:43:18,728][105692] Updated weights for policy 0, policy_version 186035 (0.0008) [2023-12-26 16:43:19,279][105692] Updated weights for policy 0, policy_version 186045 (0.0006) [2023-12-26 16:43:19,345][105692] Updated weights for policy 0, policy_version 186055 (0.0011) [2023-12-26 16:43:19,354][105620] Updated weights for policy 1, policy_version 187154 (0.0013) [2023-12-26 16:43:19,405][105692] Updated weights for policy 0, policy_version 186065 (0.0011) [2023-12-26 16:43:19,412][105620] Updated weights for policy 1, policy_version 187164 (0.0008) [2023-12-26 16:43:19,472][105620] Updated weights for policy 1, policy_version 187174 (0.0006) [2023-12-26 16:43:20,174][105692] Updated weights for policy 0, policy_version 186075 (0.0010) [2023-12-26 16:43:20,226][105620] Updated weights for policy 1, policy_version 187184 (0.0007) [2023-12-26 16:43:20,232][105692] Updated weights for policy 0, policy_version 186085 (0.0007) [2023-12-26 16:43:20,284][105620] Updated weights for policy 1, policy_version 187194 (0.0008) [2023-12-26 16:43:20,289][105692] Updated weights for policy 0, policy_version 186095 (0.0006) [2023-12-26 16:43:20,342][105620] Updated weights for policy 1, policy_version 187204 (0.0007) [2023-12-26 16:43:21,056][105620] Updated weights for policy 1, policy_version 187214 (0.0007) [2023-12-26 16:43:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 95584256. Throughput: 0: 9618.3, 1: 9919.8. Samples: 95580260. Policy #0 lag: (min: 26.0, avg: 35.4, max: 58.0) [2023-12-26 16:43:21,062][104569] Avg episode reward: [(0, '9347.280'), (1, '9266.137')] [2023-12-26 16:43:21,105][105692] Updated weights for policy 0, policy_version 186105 (0.0008) [2023-12-26 16:43:21,117][105620] Updated weights for policy 1, policy_version 187224 (0.0006) [2023-12-26 16:43:21,162][105692] Updated weights for policy 0, policy_version 186115 (0.0008) [2023-12-26 16:43:21,182][105620] Updated weights for policy 1, policy_version 187234 (0.0008) [2023-12-26 16:43:21,217][105692] Updated weights for policy 0, policy_version 186125 (0.0008) [2023-12-26 16:43:21,279][105692] Updated weights for policy 0, policy_version 186135 (0.0009) [2023-12-26 16:43:21,915][105620] Updated weights for policy 1, policy_version 187244 (0.0007) [2023-12-26 16:43:21,985][105620] Updated weights for policy 1, policy_version 187254 (0.0006) [2023-12-26 16:43:22,048][105620] Updated weights for policy 1, policy_version 187264 (0.0009) [2023-12-26 16:43:22,076][105692] Updated weights for policy 0, policy_version 186145 (0.0009) [2023-12-26 16:43:22,129][105692] Updated weights for policy 0, policy_version 186155 (0.0009) [2023-12-26 16:43:22,181][105692] Updated weights for policy 0, policy_version 186165 (0.0009) [2023-12-26 16:43:22,741][105620] Updated weights for policy 1, policy_version 187274 (0.0009) [2023-12-26 16:43:22,797][105620] Updated weights for policy 1, policy_version 187284 (0.0009) [2023-12-26 16:43:22,859][105620] Updated weights for policy 1, policy_version 187294 (0.0009) [2023-12-26 16:43:22,923][105620] Updated weights for policy 1, policy_version 187304 (0.0009) [2023-12-26 16:43:22,999][105692] Updated weights for policy 0, policy_version 186175 (0.0009) [2023-12-26 16:43:23,068][105692] Updated weights for policy 0, policy_version 186185 (0.0006) [2023-12-26 16:43:23,127][105692] Updated weights for policy 0, policy_version 186195 (0.0006) [2023-12-26 16:43:23,641][105620] Updated weights for policy 1, policy_version 187315 (0.0009) [2023-12-26 16:43:23,702][105620] Updated weights for policy 1, policy_version 187325 (0.0009) [2023-12-26 16:43:23,756][105620] Updated weights for policy 1, policy_version 187335 (0.0009) [2023-12-26 16:43:23,869][105692] Updated weights for policy 0, policy_version 186205 (0.0007) [2023-12-26 16:43:23,916][105692] Updated weights for policy 0, policy_version 186215 (0.0009) [2023-12-26 16:43:23,964][105692] Updated weights for policy 0, policy_version 186225 (0.0009) [2023-12-26 16:43:24,462][105620] Updated weights for policy 1, policy_version 187345 (0.0007) [2023-12-26 16:43:24,513][105620] Updated weights for policy 1, policy_version 187355 (0.0008) [2023-12-26 16:43:24,565][105620] Updated weights for policy 1, policy_version 187365 (0.0006) [2023-12-26 16:43:24,804][105692] Updated weights for policy 0, policy_version 186235 (0.0010) [2023-12-26 16:43:24,866][105692] Updated weights for policy 0, policy_version 186245 (0.0009) [2023-12-26 16:43:24,928][105692] Updated weights for policy 0, policy_version 186255 (0.0009) [2023-12-26 16:43:25,166][105620] Updated weights for policy 1, policy_version 187375 (0.0005) [2023-12-26 16:43:25,220][105620] Updated weights for policy 1, policy_version 187385 (0.0005) [2023-12-26 16:43:25,272][105620] Updated weights for policy 1, policy_version 187395 (0.0005) [2023-12-26 16:43:25,726][105692] Updated weights for policy 0, policy_version 186265 (0.0008) [2023-12-26 16:43:25,776][105692] Updated weights for policy 0, policy_version 186275 (0.0007) [2023-12-26 16:43:25,821][105692] Updated weights for policy 0, policy_version 186285 (0.0010) [2023-12-26 16:43:25,875][105692] Updated weights for policy 0, policy_version 186295 (0.0010) [2023-12-26 16:43:25,974][105620] Updated weights for policy 1, policy_version 187405 (0.0005) [2023-12-26 16:43:26,043][105620] Updated weights for policy 1, policy_version 187415 (0.0005) [2023-12-26 16:43:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 95682560. Throughput: 0: 9516.6, 1: 9967.3. Samples: 95693288. Policy #0 lag: (min: 26.0, avg: 35.4, max: 58.0) [2023-12-26 16:43:26,062][104569] Avg episode reward: [(0, '9347.495'), (1, '9173.618')] [2023-12-26 16:43:26,098][105620] Updated weights for policy 1, policy_version 187425 (0.0009) [2023-12-26 16:43:26,559][105692] Updated weights for policy 0, policy_version 186305 (0.0009) [2023-12-26 16:43:26,621][105692] Updated weights for policy 0, policy_version 186315 (0.0009) [2023-12-26 16:43:26,682][105692] Updated weights for policy 0, policy_version 186325 (0.0007) [2023-12-26 16:43:26,770][105620] Updated weights for policy 1, policy_version 187435 (0.0008) [2023-12-26 16:43:26,826][105620] Updated weights for policy 1, policy_version 187445 (0.0008) [2023-12-26 16:43:26,878][105620] Updated weights for policy 1, policy_version 187455 (0.0008) [2023-12-26 16:43:27,325][105692] Updated weights for policy 0, policy_version 186335 (0.0008) [2023-12-26 16:43:27,383][105692] Updated weights for policy 0, policy_version 186345 (0.0010) [2023-12-26 16:43:27,441][105692] Updated weights for policy 0, policy_version 186355 (0.0010) [2023-12-26 16:43:27,583][105620] Updated weights for policy 1, policy_version 187465 (0.0008) [2023-12-26 16:43:27,634][105620] Updated weights for policy 1, policy_version 187475 (0.0005) [2023-12-26 16:43:27,683][105620] Updated weights for policy 1, policy_version 187485 (0.0006) [2023-12-26 16:43:27,730][105620] Updated weights for policy 1, policy_version 187495 (0.0008) [2023-12-26 16:43:28,042][105692] Updated weights for policy 0, policy_version 186365 (0.0010) [2023-12-26 16:43:28,092][105692] Updated weights for policy 0, policy_version 186375 (0.0010) [2023-12-26 16:43:28,140][105692] Updated weights for policy 0, policy_version 186385 (0.0010) [2023-12-26 16:43:28,421][105620] Updated weights for policy 1, policy_version 187505 (0.0010) [2023-12-26 16:43:28,480][105620] Updated weights for policy 1, policy_version 187515 (0.0011) [2023-12-26 16:43:28,536][105620] Updated weights for policy 1, policy_version 187525 (0.0011) [2023-12-26 16:43:28,866][105692] Updated weights for policy 0, policy_version 186395 (0.0009) [2023-12-26 16:43:28,930][105692] Updated weights for policy 0, policy_version 186405 (0.0010) [2023-12-26 16:43:28,984][105692] Updated weights for policy 0, policy_version 186415 (0.0010) [2023-12-26 16:43:29,276][105620] Updated weights for policy 1, policy_version 187535 (0.0009) [2023-12-26 16:43:29,321][105620] Updated weights for policy 1, policy_version 187545 (0.0010) [2023-12-26 16:43:29,384][105620] Updated weights for policy 1, policy_version 187555 (0.0011) [2023-12-26 16:43:29,654][105692] Updated weights for policy 0, policy_version 186425 (0.0010) [2023-12-26 16:43:29,717][105692] Updated weights for policy 0, policy_version 186435 (0.0006) [2023-12-26 16:43:29,774][105692] Updated weights for policy 0, policy_version 186445 (0.0005) [2023-12-26 16:43:29,826][105692] Updated weights for policy 0, policy_version 186455 (0.0006) [2023-12-26 16:43:30,079][105620] Updated weights for policy 1, policy_version 187565 (0.0009) [2023-12-26 16:43:30,131][105620] Updated weights for policy 1, policy_version 187575 (0.0011) [2023-12-26 16:43:30,177][105620] Updated weights for policy 1, policy_version 187585 (0.0010) [2023-12-26 16:43:30,497][105692] Updated weights for policy 0, policy_version 186465 (0.0005) [2023-12-26 16:43:30,559][105692] Updated weights for policy 0, policy_version 186475 (0.0005) [2023-12-26 16:43:30,611][105692] Updated weights for policy 0, policy_version 186485 (0.0005) [2023-12-26 16:43:30,766][105620] Updated weights for policy 1, policy_version 187595 (0.0009) [2023-12-26 16:43:30,818][105620] Updated weights for policy 1, policy_version 187605 (0.0008) [2023-12-26 16:43:30,872][105620] Updated weights for policy 1, policy_version 187615 (0.0010) [2023-12-26 16:43:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 95789056. Throughput: 0: 9605.6, 1: 10020.3. Samples: 95754684. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-26 16:43:31,062][104569] Avg episode reward: [(0, '9346.684'), (1, '9172.014')] [2023-12-26 16:43:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000186488_47751168.pth... [2023-12-26 16:43:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000187624_48037888.pth... [2023-12-26 16:43:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000186440_47734784.pth [2023-12-26 16:43:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000185336_47456256.pth [2023-12-26 16:43:31,232][105692] Updated weights for policy 0, policy_version 186495 (0.0006) [2023-12-26 16:43:31,295][105692] Updated weights for policy 0, policy_version 186505 (0.0009) [2023-12-26 16:43:31,352][105692] Updated weights for policy 0, policy_version 186515 (0.0009) [2023-12-26 16:43:31,589][105620] Updated weights for policy 1, policy_version 187625 (0.0010) [2023-12-26 16:43:31,658][105620] Updated weights for policy 1, policy_version 187635 (0.0008) [2023-12-26 16:43:31,728][105620] Updated weights for policy 1, policy_version 187645 (0.0008) [2023-12-26 16:43:31,792][105620] Updated weights for policy 1, policy_version 187655 (0.0008) [2023-12-26 16:43:32,109][105692] Updated weights for policy 0, policy_version 186525 (0.0010) [2023-12-26 16:43:32,176][105692] Updated weights for policy 0, policy_version 186535 (0.0009) [2023-12-26 16:43:32,245][105692] Updated weights for policy 0, policy_version 186545 (0.0009) [2023-12-26 16:43:32,469][105620] Updated weights for policy 1, policy_version 187665 (0.0006) [2023-12-26 16:43:32,521][105620] Updated weights for policy 1, policy_version 187675 (0.0007) [2023-12-26 16:43:32,587][105620] Updated weights for policy 1, policy_version 187685 (0.0009) [2023-12-26 16:43:33,036][105692] Updated weights for policy 0, policy_version 186555 (0.0009) [2023-12-26 16:43:33,097][105692] Updated weights for policy 0, policy_version 186565 (0.0009) [2023-12-26 16:43:33,144][105692] Updated weights for policy 0, policy_version 186575 (0.0009) [2023-12-26 16:43:33,250][105620] Updated weights for policy 1, policy_version 187695 (0.0008) [2023-12-26 16:43:33,300][105620] Updated weights for policy 1, policy_version 187705 (0.0009) [2023-12-26 16:43:33,355][105620] Updated weights for policy 1, policy_version 187715 (0.0008) [2023-12-26 16:43:33,839][105692] Updated weights for policy 0, policy_version 186585 (0.0008) [2023-12-26 16:43:33,894][105692] Updated weights for policy 0, policy_version 186595 (0.0005) [2023-12-26 16:43:33,951][105692] Updated weights for policy 0, policy_version 186605 (0.0005) [2023-12-26 16:43:34,002][105692] Updated weights for policy 0, policy_version 186615 (0.0005) [2023-12-26 16:43:34,178][105620] Updated weights for policy 1, policy_version 187725 (0.0009) [2023-12-26 16:43:34,241][105620] Updated weights for policy 1, policy_version 187735 (0.0010) [2023-12-26 16:43:34,297][105620] Updated weights for policy 1, policy_version 187745 (0.0009) [2023-12-26 16:43:34,568][105692] Updated weights for policy 0, policy_version 186625 (0.0007) [2023-12-26 16:43:34,632][105692] Updated weights for policy 0, policy_version 186635 (0.0007) [2023-12-26 16:43:34,691][105692] Updated weights for policy 0, policy_version 186645 (0.0010) [2023-12-26 16:43:35,142][105620] Updated weights for policy 1, policy_version 187755 (0.0009) [2023-12-26 16:43:35,194][105620] Updated weights for policy 1, policy_version 187765 (0.0008) [2023-12-26 16:43:35,247][105620] Updated weights for policy 1, policy_version 187775 (0.0009) [2023-12-26 16:43:35,330][105692] Updated weights for policy 0, policy_version 186655 (0.0011) [2023-12-26 16:43:35,378][105692] Updated weights for policy 0, policy_version 186665 (0.0010) [2023-12-26 16:43:35,429][105692] Updated weights for policy 0, policy_version 186675 (0.0010) [2023-12-26 16:43:36,019][105620] Updated weights for policy 1, policy_version 187785 (0.0008) [2023-12-26 16:43:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19660.8). Total num frames: 95879168. Throughput: 0: 9730.0, 1: 9994.6. Samples: 95874348. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-26 16:43:36,062][104569] Avg episode reward: [(0, '9344.370'), (1, '9264.872')] [2023-12-26 16:43:36,073][105620] Updated weights for policy 1, policy_version 187795 (0.0009) [2023-12-26 16:43:36,080][105692] Updated weights for policy 0, policy_version 186685 (0.0008) [2023-12-26 16:43:36,138][105620] Updated weights for policy 1, policy_version 187805 (0.0007) [2023-12-26 16:43:36,143][105692] Updated weights for policy 0, policy_version 186695 (0.0007) [2023-12-26 16:43:36,196][105620] Updated weights for policy 1, policy_version 187815 (0.0008) [2023-12-26 16:43:36,199][105692] Updated weights for policy 0, policy_version 186705 (0.0011) [2023-12-26 16:43:36,865][105692] Updated weights for policy 0, policy_version 186715 (0.0007) [2023-12-26 16:43:36,887][105620] Updated weights for policy 1, policy_version 187825 (0.0006) [2023-12-26 16:43:36,933][105692] Updated weights for policy 0, policy_version 186725 (0.0011) [2023-12-26 16:43:36,948][105620] Updated weights for policy 1, policy_version 187835 (0.0006) [2023-12-26 16:43:37,002][105692] Updated weights for policy 0, policy_version 186735 (0.0010) [2023-12-26 16:43:37,011][105620] Updated weights for policy 1, policy_version 187845 (0.0008) [2023-12-26 16:43:37,752][105692] Updated weights for policy 0, policy_version 186745 (0.0009) [2023-12-26 16:43:37,801][105620] Updated weights for policy 1, policy_version 187855 (0.0007) [2023-12-26 16:43:37,805][105692] Updated weights for policy 0, policy_version 186755 (0.0007) [2023-12-26 16:43:37,855][105620] Updated weights for policy 1, policy_version 187865 (0.0009) [2023-12-26 16:43:37,858][105692] Updated weights for policy 0, policy_version 186765 (0.0005) [2023-12-26 16:43:37,912][105692] Updated weights for policy 0, policy_version 186775 (0.0005) [2023-12-26 16:43:37,913][105620] Updated weights for policy 1, policy_version 187875 (0.0009) [2023-12-26 16:43:38,487][105692] Updated weights for policy 0, policy_version 186785 (0.0010) [2023-12-26 16:43:38,549][105692] Updated weights for policy 0, policy_version 186795 (0.0011) [2023-12-26 16:43:38,612][105692] Updated weights for policy 0, policy_version 186805 (0.0011) [2023-12-26 16:43:38,758][105620] Updated weights for policy 1, policy_version 187885 (0.0010) [2023-12-26 16:43:38,810][105620] Updated weights for policy 1, policy_version 187895 (0.0010) [2023-12-26 16:43:38,867][105620] Updated weights for policy 1, policy_version 187905 (0.0010) [2023-12-26 16:43:39,362][105692] Updated weights for policy 0, policy_version 186815 (0.0009) [2023-12-26 16:43:39,429][105692] Updated weights for policy 0, policy_version 186825 (0.0009) [2023-12-26 16:43:39,492][105692] Updated weights for policy 0, policy_version 186835 (0.0011) [2023-12-26 16:43:39,515][105620] Updated weights for policy 1, policy_version 187915 (0.0007) [2023-12-26 16:43:39,575][105620] Updated weights for policy 1, policy_version 187925 (0.0009) [2023-12-26 16:43:39,635][105620] Updated weights for policy 1, policy_version 187935 (0.0008) [2023-12-26 16:43:40,215][105692] Updated weights for policy 0, policy_version 186845 (0.0010) [2023-12-26 16:43:40,275][105692] Updated weights for policy 0, policy_version 186855 (0.0009) [2023-12-26 16:43:40,333][105692] Updated weights for policy 0, policy_version 186865 (0.0009) [2023-12-26 16:43:40,427][105620] Updated weights for policy 1, policy_version 187945 (0.0008) [2023-12-26 16:43:40,484][105620] Updated weights for policy 1, policy_version 187955 (0.0009) [2023-12-26 16:43:40,551][105620] Updated weights for policy 1, policy_version 187965 (0.0006) [2023-12-26 16:43:40,609][105620] Updated weights for policy 1, policy_version 187975 (0.0010) [2023-12-26 16:43:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 95977472. Throughput: 0: 9785.6, 1: 9862.6. Samples: 95990432. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-26 16:43:41,063][104569] Avg episode reward: [(0, '9345.211'), (1, '9265.653')] [2023-12-26 16:43:41,083][105692] Updated weights for policy 0, policy_version 186875 (0.0008) [2023-12-26 16:43:41,139][105692] Updated weights for policy 0, policy_version 186885 (0.0010) [2023-12-26 16:43:41,202][105692] Updated weights for policy 0, policy_version 186895 (0.0008) [2023-12-26 16:43:41,331][105620] Updated weights for policy 1, policy_version 187985 (0.0008) [2023-12-26 16:43:41,399][105620] Updated weights for policy 1, policy_version 187995 (0.0007) [2023-12-26 16:43:41,465][105620] Updated weights for policy 1, policy_version 188005 (0.0009) [2023-12-26 16:43:41,947][105692] Updated weights for policy 0, policy_version 186905 (0.0006) [2023-12-26 16:43:42,011][105692] Updated weights for policy 0, policy_version 186915 (0.0005) [2023-12-26 16:43:42,081][105692] Updated weights for policy 0, policy_version 186925 (0.0009) [2023-12-26 16:43:42,141][105692] Updated weights for policy 0, policy_version 186935 (0.0008) [2023-12-26 16:43:42,188][105620] Updated weights for policy 1, policy_version 188015 (0.0008) [2023-12-26 16:43:42,236][105620] Updated weights for policy 1, policy_version 188025 (0.0008) [2023-12-26 16:43:42,306][105620] Updated weights for policy 1, policy_version 188035 (0.0009) [2023-12-26 16:43:42,825][105692] Updated weights for policy 0, policy_version 186945 (0.0011) [2023-12-26 16:43:42,884][105692] Updated weights for policy 0, policy_version 186955 (0.0010) [2023-12-26 16:43:42,939][105692] Updated weights for policy 0, policy_version 186965 (0.0010) [2023-12-26 16:43:43,077][105620] Updated weights for policy 1, policy_version 188045 (0.0008) [2023-12-26 16:43:43,143][105620] Updated weights for policy 1, policy_version 188055 (0.0008) [2023-12-26 16:43:43,191][105620] Updated weights for policy 1, policy_version 188065 (0.0008) [2023-12-26 16:43:43,662][105692] Updated weights for policy 0, policy_version 186975 (0.0008) [2023-12-26 16:43:43,720][105692] Updated weights for policy 0, policy_version 186985 (0.0006) [2023-12-26 16:43:43,781][105692] Updated weights for policy 0, policy_version 186995 (0.0005) [2023-12-26 16:43:43,972][105620] Updated weights for policy 1, policy_version 188075 (0.0008) [2023-12-26 16:43:44,024][105620] Updated weights for policy 1, policy_version 188085 (0.0010) [2023-12-26 16:43:44,076][105620] Updated weights for policy 1, policy_version 188096 (0.0010) [2023-12-26 16:43:44,301][105692] Updated weights for policy 0, policy_version 187005 (0.0005) [2023-12-26 16:43:44,353][105692] Updated weights for policy 0, policy_version 187015 (0.0005) [2023-12-26 16:43:44,409][105692] Updated weights for policy 0, policy_version 187025 (0.0005) [2023-12-26 16:43:44,959][105692] Updated weights for policy 0, policy_version 187035 (0.0005) [2023-12-26 16:43:44,984][105620] Updated weights for policy 1, policy_version 188106 (0.0009) [2023-12-26 16:43:45,022][105692] Updated weights for policy 0, policy_version 187045 (0.0006) [2023-12-26 16:43:45,049][105620] Updated weights for policy 1, policy_version 188116 (0.0010) [2023-12-26 16:43:45,086][105692] Updated weights for policy 0, policy_version 187055 (0.0005) [2023-12-26 16:43:45,116][105620] Updated weights for policy 1, policy_version 188126 (0.0008) [2023-12-26 16:43:45,171][105620] Updated weights for policy 1, policy_version 188136 (0.0009) [2023-12-26 16:43:45,648][105692] Updated weights for policy 0, policy_version 187065 (0.0006) [2023-12-26 16:43:45,703][105692] Updated weights for policy 0, policy_version 187075 (0.0010) [2023-12-26 16:43:45,757][105692] Updated weights for policy 0, policy_version 187085 (0.0010) [2023-12-26 16:43:45,812][105692] Updated weights for policy 0, policy_version 187095 (0.0010) [2023-12-26 16:43:45,987][105620] Updated weights for policy 1, policy_version 188146 (0.0010) [2023-12-26 16:43:46,037][105620] Updated weights for policy 1, policy_version 188156 (0.0010) [2023-12-26 16:43:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19688.6). Total num frames: 96075776. Throughput: 0: 9796.6, 1: 9837.7. Samples: 96046340. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-26 16:43:46,063][104569] Avg episode reward: [(0, '9346.947'), (1, '9265.840')] [2023-12-26 16:43:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000187096_47906816.pth... [2023-12-26 16:43:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000185912_47603712.pth [2023-12-26 16:43:46,090][105620] Updated weights for policy 1, policy_version 188166 (0.0010) [2023-12-26 16:43:46,100][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000188168_48177152.pth... [2023-12-26 16:43:46,104][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000187016_47882240.pth [2023-12-26 16:43:46,500][105692] Updated weights for policy 0, policy_version 187105 (0.0009) [2023-12-26 16:43:46,555][105692] Updated weights for policy 0, policy_version 187115 (0.0005) [2023-12-26 16:43:46,603][105692] Updated weights for policy 0, policy_version 187125 (0.0005) [2023-12-26 16:43:46,802][105620] Updated weights for policy 1, policy_version 188176 (0.0006) [2023-12-26 16:43:46,863][105620] Updated weights for policy 1, policy_version 188186 (0.0005) [2023-12-26 16:43:46,922][105620] Updated weights for policy 1, policy_version 188196 (0.0008) [2023-12-26 16:43:47,217][105692] Updated weights for policy 0, policy_version 187135 (0.0009) [2023-12-26 16:43:47,269][105692] Updated weights for policy 0, policy_version 187145 (0.0009) [2023-12-26 16:43:47,318][105692] Updated weights for policy 0, policy_version 187155 (0.0009) [2023-12-26 16:43:47,525][105620] Updated weights for policy 1, policy_version 188206 (0.0007) [2023-12-26 16:43:47,578][105620] Updated weights for policy 1, policy_version 188216 (0.0005) [2023-12-26 16:43:47,636][105620] Updated weights for policy 1, policy_version 188226 (0.0005) [2023-12-26 16:43:48,070][105692] Updated weights for policy 0, policy_version 187165 (0.0010) [2023-12-26 16:43:48,136][105692] Updated weights for policy 0, policy_version 187175 (0.0010) [2023-12-26 16:43:48,151][105620] Updated weights for policy 1, policy_version 188236 (0.0007) [2023-12-26 16:43:48,188][105692] Updated weights for policy 0, policy_version 187185 (0.0010) [2023-12-26 16:43:48,203][105620] Updated weights for policy 1, policy_version 188246 (0.0010) [2023-12-26 16:43:48,250][105620] Updated weights for policy 1, policy_version 188256 (0.0008) [2023-12-26 16:43:48,936][105692] Updated weights for policy 0, policy_version 187195 (0.0010) [2023-12-26 16:43:48,964][105620] Updated weights for policy 1, policy_version 188266 (0.0006) [2023-12-26 16:43:48,995][105692] Updated weights for policy 0, policy_version 187205 (0.0006) [2023-12-26 16:43:49,026][105620] Updated weights for policy 1, policy_version 188276 (0.0010) [2023-12-26 16:43:49,053][105692] Updated weights for policy 0, policy_version 187215 (0.0007) [2023-12-26 16:43:49,088][105620] Updated weights for policy 1, policy_version 188286 (0.0010) [2023-12-26 16:43:49,149][105620] Updated weights for policy 1, policy_version 188296 (0.0010) [2023-12-26 16:43:49,729][105692] Updated weights for policy 0, policy_version 187225 (0.0007) [2023-12-26 16:43:49,784][105692] Updated weights for policy 0, policy_version 187235 (0.0007) [2023-12-26 16:43:49,847][105692] Updated weights for policy 0, policy_version 187245 (0.0007) [2023-12-26 16:43:49,887][105620] Updated weights for policy 1, policy_version 188306 (0.0010) [2023-12-26 16:43:49,898][105692] Updated weights for policy 0, policy_version 187255 (0.0006) [2023-12-26 16:43:49,951][105620] Updated weights for policy 1, policy_version 188316 (0.0010) [2023-12-26 16:43:50,003][105620] Updated weights for policy 1, policy_version 188326 (0.0010) [2023-12-26 16:43:50,692][105692] Updated weights for policy 0, policy_version 187265 (0.0009) [2023-12-26 16:43:50,746][105692] Updated weights for policy 0, policy_version 187276 (0.0010) [2023-12-26 16:43:50,780][105620] Updated weights for policy 1, policy_version 188336 (0.0010) [2023-12-26 16:43:50,799][105692] Updated weights for policy 0, policy_version 187286 (0.0006) [2023-12-26 16:43:50,837][105620] Updated weights for policy 1, policy_version 188346 (0.0010) [2023-12-26 16:43:50,902][105620] Updated weights for policy 1, policy_version 188356 (0.0010) [2023-12-26 16:43:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19716.4). Total num frames: 96182272. Throughput: 0: 9988.9, 1: 9781.3. Samples: 96168628. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-26 16:43:51,063][104569] Avg episode reward: [(0, '9256.531'), (1, '9266.323')] [2023-12-26 16:43:51,601][105620] Updated weights for policy 1, policy_version 188366 (0.0008) [2023-12-26 16:43:51,629][105692] Updated weights for policy 0, policy_version 187296 (0.0008) [2023-12-26 16:43:51,667][105620] Updated weights for policy 1, policy_version 188376 (0.0008) [2023-12-26 16:43:51,686][105692] Updated weights for policy 0, policy_version 187306 (0.0008) [2023-12-26 16:43:51,735][105620] Updated weights for policy 1, policy_version 188386 (0.0007) [2023-12-26 16:43:51,751][105692] Updated weights for policy 0, policy_version 187316 (0.0008) [2023-12-26 16:43:52,429][105620] Updated weights for policy 1, policy_version 188396 (0.0007) [2023-12-26 16:43:52,494][105620] Updated weights for policy 1, policy_version 188406 (0.0008) [2023-12-26 16:43:52,527][105692] Updated weights for policy 0, policy_version 187326 (0.0008) [2023-12-26 16:43:52,558][105620] Updated weights for policy 1, policy_version 188416 (0.0008) [2023-12-26 16:43:52,577][105692] Updated weights for policy 0, policy_version 187336 (0.0007) [2023-12-26 16:43:52,628][105692] Updated weights for policy 0, policy_version 187346 (0.0008) [2023-12-26 16:43:53,226][105692] Updated weights for policy 0, policy_version 187356 (0.0008) [2023-12-26 16:43:53,294][105692] Updated weights for policy 0, policy_version 187366 (0.0007) [2023-12-26 16:43:53,353][105692] Updated weights for policy 0, policy_version 187376 (0.0005) [2023-12-26 16:43:53,404][105620] Updated weights for policy 1, policy_version 188426 (0.0008) [2023-12-26 16:43:53,465][105620] Updated weights for policy 1, policy_version 188436 (0.0005) [2023-12-26 16:43:53,526][105620] Updated weights for policy 1, policy_version 188446 (0.0005) [2023-12-26 16:43:53,578][105620] Updated weights for policy 1, policy_version 188456 (0.0005) [2023-12-26 16:43:53,902][105692] Updated weights for policy 0, policy_version 187386 (0.0005) [2023-12-26 16:43:53,973][105692] Updated weights for policy 0, policy_version 187396 (0.0010) [2023-12-26 16:43:54,021][105692] Updated weights for policy 0, policy_version 187406 (0.0010) [2023-12-26 16:43:54,072][105692] Updated weights for policy 0, policy_version 187416 (0.0010) [2023-12-26 16:43:54,090][105620] Updated weights for policy 1, policy_version 188466 (0.0005) [2023-12-26 16:43:54,137][105620] Updated weights for policy 1, policy_version 188476 (0.0005) [2023-12-26 16:43:54,189][105620] Updated weights for policy 1, policy_version 188486 (0.0005) [2023-12-26 16:43:54,717][105692] Updated weights for policy 0, policy_version 187426 (0.0008) [2023-12-26 16:43:54,776][105692] Updated weights for policy 0, policy_version 187436 (0.0008) [2023-12-26 16:43:54,829][105692] Updated weights for policy 0, policy_version 187446 (0.0010) [2023-12-26 16:43:54,905][105620] Updated weights for policy 1, policy_version 188496 (0.0008) [2023-12-26 16:43:54,971][105620] Updated weights for policy 1, policy_version 188506 (0.0008) [2023-12-26 16:43:55,039][105620] Updated weights for policy 1, policy_version 188516 (0.0007) [2023-12-26 16:43:55,471][105692] Updated weights for policy 0, policy_version 187456 (0.0006) [2023-12-26 16:43:55,531][105692] Updated weights for policy 0, policy_version 187466 (0.0005) [2023-12-26 16:43:55,597][105692] Updated weights for policy 0, policy_version 187476 (0.0005) [2023-12-26 16:43:55,868][105620] Updated weights for policy 1, policy_version 188526 (0.0007) [2023-12-26 16:43:55,923][105620] Updated weights for policy 1, policy_version 188536 (0.0008) [2023-12-26 16:43:55,967][105620] Updated weights for policy 1, policy_version 188546 (0.0008) [2023-12-26 16:43:56,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.7, 300 sec: 19716.3). Total num frames: 96280576. Throughput: 0: 10077.5, 1: 9743.5. Samples: 96287304. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-26 16:43:56,063][104569] Avg episode reward: [(0, '9257.241'), (1, '9173.805')] [2023-12-26 16:43:56,171][105692] Updated weights for policy 0, policy_version 187486 (0.0005) [2023-12-26 16:43:56,233][105692] Updated weights for policy 0, policy_version 187496 (0.0005) [2023-12-26 16:43:56,283][105692] Updated weights for policy 0, policy_version 187506 (0.0005) [2023-12-26 16:43:56,760][105620] Updated weights for policy 1, policy_version 188556 (0.0008) [2023-12-26 16:43:56,814][105620] Updated weights for policy 1, policy_version 188566 (0.0009) [2023-12-26 16:43:56,864][105620] Updated weights for policy 1, policy_version 188576 (0.0009) [2023-12-26 16:43:56,940][105692] Updated weights for policy 0, policy_version 187516 (0.0007) [2023-12-26 16:43:57,000][105692] Updated weights for policy 0, policy_version 187526 (0.0009) [2023-12-26 16:43:57,060][105692] Updated weights for policy 0, policy_version 187536 (0.0008) [2023-12-26 16:43:57,575][105620] Updated weights for policy 1, policy_version 188586 (0.0009) [2023-12-26 16:43:57,632][105620] Updated weights for policy 1, policy_version 188596 (0.0010) [2023-12-26 16:43:57,687][105620] Updated weights for policy 1, policy_version 188606 (0.0010) [2023-12-26 16:43:57,784][105692] Updated weights for policy 0, policy_version 187546 (0.0009) [2023-12-26 16:43:57,835][105692] Updated weights for policy 0, policy_version 187556 (0.0009) [2023-12-26 16:43:57,880][105692] Updated weights for policy 0, policy_version 187566 (0.0009) [2023-12-26 16:43:57,939][105692] Updated weights for policy 0, policy_version 187576 (0.0007) [2023-12-26 16:43:58,443][105620] Updated weights for policy 1, policy_version 188617 (0.0010) [2023-12-26 16:43:58,504][105620] Updated weights for policy 1, policy_version 188627 (0.0007) [2023-12-26 16:43:58,560][105620] Updated weights for policy 1, policy_version 188637 (0.0008) [2023-12-26 16:43:58,626][105620] Updated weights for policy 1, policy_version 188647 (0.0008) [2023-12-26 16:43:58,685][105692] Updated weights for policy 0, policy_version 187586 (0.0008) [2023-12-26 16:43:58,754][105692] Updated weights for policy 0, policy_version 187596 (0.0008) [2023-12-26 16:43:58,818][105692] Updated weights for policy 0, policy_version 187606 (0.0010) [2023-12-26 16:43:59,292][105620] Updated weights for policy 1, policy_version 188658 (0.0009) [2023-12-26 16:43:59,358][105620] Updated weights for policy 1, policy_version 188668 (0.0009) [2023-12-26 16:43:59,408][105620] Updated weights for policy 1, policy_version 188678 (0.0007) [2023-12-26 16:43:59,608][105692] Updated weights for policy 0, policy_version 187616 (0.0009) [2023-12-26 16:43:59,667][105692] Updated weights for policy 0, policy_version 187626 (0.0009) [2023-12-26 16:43:59,738][105692] Updated weights for policy 0, policy_version 187636 (0.0010) [2023-12-26 16:44:00,127][105620] Updated weights for policy 1, policy_version 188688 (0.0009) [2023-12-26 16:44:00,188][105620] Updated weights for policy 1, policy_version 188698 (0.0009) [2023-12-26 16:44:00,238][105620] Updated weights for policy 1, policy_version 188708 (0.0008) [2023-12-26 16:44:00,540][105692] Updated weights for policy 0, policy_version 187646 (0.0010) [2023-12-26 16:44:00,589][105692] Updated weights for policy 0, policy_version 187656 (0.0009) [2023-12-26 16:44:00,637][105692] Updated weights for policy 0, policy_version 187666 (0.0009) [2023-12-26 16:44:00,949][105620] Updated weights for policy 1, policy_version 188718 (0.0009) [2023-12-26 16:44:00,996][105620] Updated weights for policy 1, policy_version 188728 (0.0009) [2023-12-26 16:44:01,054][105620] Updated weights for policy 1, policy_version 188738 (0.0009) [2023-12-26 16:44:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 96370688. Throughput: 0: 10128.1, 1: 9720.8. Samples: 96345588. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-26 16:44:01,062][104569] Avg episode reward: [(0, '9347.744'), (1, '8527.231')] [2023-12-26 16:44:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000187672_48054272.pth... [2023-12-26 16:44:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000186488_47751168.pth [2023-12-26 16:44:01,087][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000188744_48324608.pth... [2023-12-26 16:44:01,091][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000187624_48037888.pth [2023-12-26 16:44:01,392][105692] Updated weights for policy 0, policy_version 187676 (0.0007) [2023-12-26 16:44:01,437][105692] Updated weights for policy 0, policy_version 187686 (0.0008) [2023-12-26 16:44:01,487][105692] Updated weights for policy 0, policy_version 187696 (0.0009) [2023-12-26 16:44:01,859][105620] Updated weights for policy 1, policy_version 188748 (0.0008) [2023-12-26 16:44:01,933][105620] Updated weights for policy 1, policy_version 188758 (0.0010) [2023-12-26 16:44:01,992][105620] Updated weights for policy 1, policy_version 188768 (0.0009) [2023-12-26 16:44:02,154][105692] Updated weights for policy 0, policy_version 187706 (0.0006) [2023-12-26 16:44:02,201][105692] Updated weights for policy 0, policy_version 187716 (0.0005) [2023-12-26 16:44:02,248][105692] Updated weights for policy 0, policy_version 187726 (0.0005) [2023-12-26 16:44:02,312][105692] Updated weights for policy 0, policy_version 187736 (0.0006) [2023-12-26 16:44:02,764][105620] Updated weights for policy 1, policy_version 188778 (0.0008) [2023-12-26 16:44:02,819][105620] Updated weights for policy 1, policy_version 188788 (0.0006) [2023-12-26 16:44:02,874][105620] Updated weights for policy 1, policy_version 188798 (0.0008) [2023-12-26 16:44:02,928][105692] Updated weights for policy 0, policy_version 187746 (0.0005) [2023-12-26 16:44:02,938][105620] Updated weights for policy 1, policy_version 188808 (0.0009) [2023-12-26 16:44:02,984][105692] Updated weights for policy 0, policy_version 187756 (0.0008) [2023-12-26 16:44:03,045][105692] Updated weights for policy 0, policy_version 187766 (0.0007) [2023-12-26 16:44:03,573][105692] Updated weights for policy 0, policy_version 187776 (0.0007) [2023-12-26 16:44:03,585][105620] Updated weights for policy 1, policy_version 188818 (0.0006) [2023-12-26 16:44:03,633][105692] Updated weights for policy 0, policy_version 187786 (0.0006) [2023-12-26 16:44:03,648][105620] Updated weights for policy 1, policy_version 188828 (0.0005) [2023-12-26 16:44:03,689][105692] Updated weights for policy 0, policy_version 187796 (0.0010) [2023-12-26 16:44:03,708][105620] Updated weights for policy 1, policy_version 188838 (0.0007) [2023-12-26 16:44:04,341][105692] Updated weights for policy 0, policy_version 187806 (0.0010) [2023-12-26 16:44:04,368][105620] Updated weights for policy 1, policy_version 188848 (0.0006) [2023-12-26 16:44:04,393][105692] Updated weights for policy 0, policy_version 187816 (0.0011) [2023-12-26 16:44:04,435][105620] Updated weights for policy 1, policy_version 188858 (0.0006) [2023-12-26 16:44:04,457][105692] Updated weights for policy 0, policy_version 187826 (0.0011) [2023-12-26 16:44:04,503][105620] Updated weights for policy 1, policy_version 188868 (0.0007) [2023-12-26 16:44:05,154][105620] Updated weights for policy 1, policy_version 188878 (0.0009) [2023-12-26 16:44:05,162][105692] Updated weights for policy 0, policy_version 187836 (0.0009) [2023-12-26 16:44:05,208][105620] Updated weights for policy 1, policy_version 188888 (0.0010) [2023-12-26 16:44:05,219][105692] Updated weights for policy 0, policy_version 187846 (0.0005) [2023-12-26 16:44:05,263][105620] Updated weights for policy 1, policy_version 188898 (0.0010) [2023-12-26 16:44:05,276][105692] Updated weights for policy 0, policy_version 187856 (0.0007) [2023-12-26 16:44:05,897][105692] Updated weights for policy 0, policy_version 187866 (0.0009) [2023-12-26 16:44:05,914][105620] Updated weights for policy 1, policy_version 188908 (0.0009) [2023-12-26 16:44:05,963][105692] Updated weights for policy 0, policy_version 187876 (0.0010) [2023-12-26 16:44:05,966][105620] Updated weights for policy 1, policy_version 188918 (0.0005) [2023-12-26 16:44:06,013][105620] Updated weights for policy 1, policy_version 188928 (0.0005) [2023-12-26 16:44:06,024][105692] Updated weights for policy 0, policy_version 187886 (0.0005) [2023-12-26 16:44:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19744.1). Total num frames: 96477184. Throughput: 0: 10001.2, 1: 9651.4. Samples: 96464636. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-26 16:44:06,063][104569] Avg episode reward: [(0, '9347.405'), (1, '8620.585')] [2023-12-26 16:44:06,084][105692] Updated weights for policy 0, policy_version 187896 (0.0006) [2023-12-26 16:44:06,653][105692] Updated weights for policy 0, policy_version 187906 (0.0011) [2023-12-26 16:44:06,701][105692] Updated weights for policy 0, policy_version 187916 (0.0010) [2023-12-26 16:44:06,735][105620] Updated weights for policy 1, policy_version 188938 (0.0008) [2023-12-26 16:44:06,758][105692] Updated weights for policy 0, policy_version 187926 (0.0006) [2023-12-26 16:44:06,798][105620] Updated weights for policy 1, policy_version 188948 (0.0011) [2023-12-26 16:44:06,860][105620] Updated weights for policy 1, policy_version 188958 (0.0010) [2023-12-26 16:44:06,915][105620] Updated weights for policy 1, policy_version 188968 (0.0010) [2023-12-26 16:44:07,430][105692] Updated weights for policy 0, policy_version 187936 (0.0008) [2023-12-26 16:44:07,493][105692] Updated weights for policy 0, policy_version 187946 (0.0007) [2023-12-26 16:44:07,551][105692] Updated weights for policy 0, policy_version 187956 (0.0011) [2023-12-26 16:44:07,603][105620] Updated weights for policy 1, policy_version 188978 (0.0007) [2023-12-26 16:44:07,663][105620] Updated weights for policy 1, policy_version 188988 (0.0009) [2023-12-26 16:44:07,729][105620] Updated weights for policy 1, policy_version 188998 (0.0008) [2023-12-26 16:44:08,227][105692] Updated weights for policy 0, policy_version 187966 (0.0006) [2023-12-26 16:44:08,296][105692] Updated weights for policy 0, policy_version 187976 (0.0009) [2023-12-26 16:44:08,377][105692] Updated weights for policy 0, policy_version 187986 (0.0011) [2023-12-26 16:44:08,399][105620] Updated weights for policy 1, policy_version 189008 (0.0006) [2023-12-26 16:44:08,461][105620] Updated weights for policy 1, policy_version 189018 (0.0009) [2023-12-26 16:44:08,518][105620] Updated weights for policy 1, policy_version 189028 (0.0008) [2023-12-26 16:44:09,058][105692] Updated weights for policy 0, policy_version 187996 (0.0009) [2023-12-26 16:44:09,106][105692] Updated weights for policy 0, policy_version 188006 (0.0010) [2023-12-26 16:44:09,163][105692] Updated weights for policy 0, policy_version 188016 (0.0010) [2023-12-26 16:44:09,284][105620] Updated weights for policy 1, policy_version 189038 (0.0008) [2023-12-26 16:44:09,349][105620] Updated weights for policy 1, policy_version 189048 (0.0009) [2023-12-26 16:44:09,419][105620] Updated weights for policy 1, policy_version 189058 (0.0009) [2023-12-26 16:44:09,969][105692] Updated weights for policy 0, policy_version 188026 (0.0009) [2023-12-26 16:44:10,037][105692] Updated weights for policy 0, policy_version 188036 (0.0009) [2023-12-26 16:44:10,093][105692] Updated weights for policy 0, policy_version 188046 (0.0008) [2023-12-26 16:44:10,163][105692] Updated weights for policy 0, policy_version 188056 (0.0005) [2023-12-26 16:44:10,169][105620] Updated weights for policy 1, policy_version 189068 (0.0009) [2023-12-26 16:44:10,232][105620] Updated weights for policy 1, policy_version 189078 (0.0009) [2023-12-26 16:44:10,295][105620] Updated weights for policy 1, policy_version 189088 (0.0009) [2023-12-26 16:44:10,806][105692] Updated weights for policy 0, policy_version 188066 (0.0009) [2023-12-26 16:44:10,865][105692] Updated weights for policy 0, policy_version 188076 (0.0009) [2023-12-26 16:44:10,926][105692] Updated weights for policy 0, policy_version 188086 (0.0009) [2023-12-26 16:44:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19744.1). Total num frames: 96575488. Throughput: 0: 10169.2, 1: 9612.1. Samples: 96583448. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-26 16:44:11,063][104569] Avg episode reward: [(0, '9346.568'), (1, '8898.864')] [2023-12-26 16:44:11,115][105620] Updated weights for policy 1, policy_version 189098 (0.0009) [2023-12-26 16:44:11,182][105620] Updated weights for policy 1, policy_version 189108 (0.0009) [2023-12-26 16:44:11,244][105620] Updated weights for policy 1, policy_version 189118 (0.0008) [2023-12-26 16:44:11,310][105620] Updated weights for policy 1, policy_version 189128 (0.0009) [2023-12-26 16:44:11,638][105692] Updated weights for policy 0, policy_version 188096 (0.0010) [2023-12-26 16:44:11,701][105692] Updated weights for policy 0, policy_version 188106 (0.0009) [2023-12-26 16:44:11,772][105692] Updated weights for policy 0, policy_version 188116 (0.0009) [2023-12-26 16:44:12,059][105620] Updated weights for policy 1, policy_version 189138 (0.0009) [2023-12-26 16:44:12,107][105620] Updated weights for policy 1, policy_version 189148 (0.0009) [2023-12-26 16:44:12,158][105620] Updated weights for policy 1, policy_version 189158 (0.0009) [2023-12-26 16:44:12,537][105692] Updated weights for policy 0, policy_version 188126 (0.0009) [2023-12-26 16:44:12,589][105692] Updated weights for policy 0, policy_version 188136 (0.0009) [2023-12-26 16:44:12,649][105692] Updated weights for policy 0, policy_version 188146 (0.0009) [2023-12-26 16:44:12,967][105620] Updated weights for policy 1, policy_version 189168 (0.0009) [2023-12-26 16:44:13,018][105620] Updated weights for policy 1, policy_version 189178 (0.0009) [2023-12-26 16:44:13,073][105620] Updated weights for policy 1, policy_version 189188 (0.0008) [2023-12-26 16:44:13,313][105692] Updated weights for policy 0, policy_version 188156 (0.0009) [2023-12-26 16:44:13,375][105692] Updated weights for policy 0, policy_version 188166 (0.0010) [2023-12-26 16:44:13,421][105692] Updated weights for policy 0, policy_version 188176 (0.0008) [2023-12-26 16:44:13,866][105620] Updated weights for policy 1, policy_version 189198 (0.0010) [2023-12-26 16:44:13,926][105620] Updated weights for policy 1, policy_version 189208 (0.0009) [2023-12-26 16:44:13,993][105620] Updated weights for policy 1, policy_version 189218 (0.0009) [2023-12-26 16:44:14,150][105692] Updated weights for policy 0, policy_version 188186 (0.0008) [2023-12-26 16:44:14,211][105692] Updated weights for policy 0, policy_version 188196 (0.0006) [2023-12-26 16:44:14,260][105692] Updated weights for policy 0, policy_version 188206 (0.0005) [2023-12-26 16:44:14,315][105692] Updated weights for policy 0, policy_version 188216 (0.0005) [2023-12-26 16:44:14,803][105620] Updated weights for policy 1, policy_version 189228 (0.0008) [2023-12-26 16:44:14,859][105620] Updated weights for policy 1, policy_version 189238 (0.0009) [2023-12-26 16:44:14,910][105692] Updated weights for policy 0, policy_version 188226 (0.0007) [2023-12-26 16:44:14,917][105620] Updated weights for policy 1, policy_version 189248 (0.0008) [2023-12-26 16:44:14,971][105692] Updated weights for policy 0, policy_version 188236 (0.0011) [2023-12-26 16:44:15,032][105692] Updated weights for policy 0, policy_version 188246 (0.0008) [2023-12-26 16:44:15,594][105692] Updated weights for policy 0, policy_version 188256 (0.0005) [2023-12-26 16:44:15,652][105692] Updated weights for policy 0, policy_version 188266 (0.0005) [2023-12-26 16:44:15,705][105692] Updated weights for policy 0, policy_version 188276 (0.0005) [2023-12-26 16:44:15,768][105620] Updated weights for policy 1, policy_version 189258 (0.0006) [2023-12-26 16:44:15,826][105620] Updated weights for policy 1, policy_version 189268 (0.0009) [2023-12-26 16:44:15,885][105620] Updated weights for policy 1, policy_version 189279 (0.0010) [2023-12-26 16:44:16,062][104569] Fps is (10 sec: 19661.6, 60 sec: 19797.3, 300 sec: 19744.1). Total num frames: 96673792. Throughput: 0: 10111.8, 1: 9549.7. Samples: 96639452. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-26 16:44:16,062][104569] Avg episode reward: [(0, '8146.654'), (1, '8805.327')] [2023-12-26 16:44:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000188280_48209920.pth... [2023-12-26 16:44:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000189288_48463872.pth... [2023-12-26 16:44:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000187096_47906816.pth [2023-12-26 16:44:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000188168_48177152.pth [2023-12-26 16:44:16,231][105692] Updated weights for policy 0, policy_version 188286 (0.0006) [2023-12-26 16:44:16,289][105692] Updated weights for policy 0, policy_version 188296 (0.0009) [2023-12-26 16:44:16,351][105692] Updated weights for policy 0, policy_version 188306 (0.0010) [2023-12-26 16:44:16,724][105620] Updated weights for policy 1, policy_version 189289 (0.0010) [2023-12-26 16:44:16,778][105620] Updated weights for policy 1, policy_version 189299 (0.0009) [2023-12-26 16:44:16,830][105620] Updated weights for policy 1, policy_version 189309 (0.0008) [2023-12-26 16:44:16,882][105620] Updated weights for policy 1, policy_version 189319 (0.0008) [2023-12-26 16:44:17,053][105692] Updated weights for policy 0, policy_version 188316 (0.0010) [2023-12-26 16:44:17,115][105692] Updated weights for policy 0, policy_version 188326 (0.0010) [2023-12-26 16:44:17,170][105692] Updated weights for policy 0, policy_version 188336 (0.0010) [2023-12-26 16:44:17,674][105620] Updated weights for policy 1, policy_version 189329 (0.0009) [2023-12-26 16:44:17,741][105620] Updated weights for policy 1, policy_version 189339 (0.0008) [2023-12-26 16:44:17,805][105620] Updated weights for policy 1, policy_version 189349 (0.0008) [2023-12-26 16:44:17,877][105692] Updated weights for policy 0, policy_version 188346 (0.0010) [2023-12-26 16:44:17,932][105692] Updated weights for policy 0, policy_version 188356 (0.0010) [2023-12-26 16:44:17,983][105692] Updated weights for policy 0, policy_version 188366 (0.0010) [2023-12-26 16:44:18,045][105692] Updated weights for policy 0, policy_version 188376 (0.0010) [2023-12-26 16:44:18,573][105620] Updated weights for policy 1, policy_version 189359 (0.0008) [2023-12-26 16:44:18,632][105620] Updated weights for policy 1, policy_version 189369 (0.0009) [2023-12-26 16:44:18,693][105620] Updated weights for policy 1, policy_version 189379 (0.0009) [2023-12-26 16:44:18,772][105692] Updated weights for policy 0, policy_version 188386 (0.0008) [2023-12-26 16:44:18,833][105692] Updated weights for policy 0, policy_version 188396 (0.0009) [2023-12-26 16:44:18,888][105692] Updated weights for policy 0, policy_version 188406 (0.0009) [2023-12-26 16:44:19,419][105620] Updated weights for policy 1, policy_version 189389 (0.0009) [2023-12-26 16:44:19,485][105620] Updated weights for policy 1, policy_version 189399 (0.0008) [2023-12-26 16:44:19,546][105620] Updated weights for policy 1, policy_version 189409 (0.0009) [2023-12-26 16:44:19,612][105692] Updated weights for policy 0, policy_version 188416 (0.0006) [2023-12-26 16:44:19,667][105692] Updated weights for policy 0, policy_version 188426 (0.0006) [2023-12-26 16:44:19,724][105692] Updated weights for policy 0, policy_version 188436 (0.0005) [2023-12-26 16:44:20,378][105692] Updated weights for policy 0, policy_version 188446 (0.0008) [2023-12-26 16:44:20,401][105620] Updated weights for policy 1, policy_version 189419 (0.0007) [2023-12-26 16:44:20,441][105692] Updated weights for policy 0, policy_version 188456 (0.0006) [2023-12-26 16:44:20,463][105620] Updated weights for policy 1, policy_version 189429 (0.0008) [2023-12-26 16:44:20,503][105692] Updated weights for policy 0, policy_version 188466 (0.0005) [2023-12-26 16:44:20,513][105620] Updated weights for policy 1, policy_version 189439 (0.0009) [2023-12-26 16:44:21,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19660.8, 300 sec: 19716.3). Total num frames: 96763904. Throughput: 0: 10178.6, 1: 9396.3. Samples: 96755216. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-26 16:44:21,062][104569] Avg episode reward: [(0, '6910.921'), (1, '8991.665')] [2023-12-26 16:44:21,223][105692] Updated weights for policy 0, policy_version 188476 (0.0008) [2023-12-26 16:44:21,288][105692] Updated weights for policy 0, policy_version 188486 (0.0009) [2023-12-26 16:44:21,328][105620] Updated weights for policy 1, policy_version 189449 (0.0008) [2023-12-26 16:44:21,344][105692] Updated weights for policy 0, policy_version 188496 (0.0008) [2023-12-26 16:44:21,395][105620] Updated weights for policy 1, policy_version 189459 (0.0008) [2023-12-26 16:44:21,459][105620] Updated weights for policy 1, policy_version 189469 (0.0010) [2023-12-26 16:44:21,521][105620] Updated weights for policy 1, policy_version 189479 (0.0009) [2023-12-26 16:44:22,023][105692] Updated weights for policy 0, policy_version 188506 (0.0007) [2023-12-26 16:44:22,083][105692] Updated weights for policy 0, policy_version 188516 (0.0005) [2023-12-26 16:44:22,152][105692] Updated weights for policy 0, policy_version 188526 (0.0009) [2023-12-26 16:44:22,220][105692] Updated weights for policy 0, policy_version 188536 (0.0007) [2023-12-26 16:44:22,366][105620] Updated weights for policy 1, policy_version 189489 (0.0008) [2023-12-26 16:44:22,427][105620] Updated weights for policy 1, policy_version 189499 (0.0008) [2023-12-26 16:44:22,480][105620] Updated weights for policy 1, policy_version 189509 (0.0009) [2023-12-26 16:44:22,877][105692] Updated weights for policy 0, policy_version 188546 (0.0009) [2023-12-26 16:44:22,937][105692] Updated weights for policy 0, policy_version 188556 (0.0009) [2023-12-26 16:44:23,003][105692] Updated weights for policy 0, policy_version 188566 (0.0009) [2023-12-26 16:44:23,330][105620] Updated weights for policy 1, policy_version 189519 (0.0007) [2023-12-26 16:44:23,388][105620] Updated weights for policy 1, policy_version 189529 (0.0009) [2023-12-26 16:44:23,450][105620] Updated weights for policy 1, policy_version 189539 (0.0009) [2023-12-26 16:44:23,647][105692] Updated weights for policy 0, policy_version 188576 (0.0009) [2023-12-26 16:44:23,701][105692] Updated weights for policy 0, policy_version 188586 (0.0006) [2023-12-26 16:44:23,766][105692] Updated weights for policy 0, policy_version 188596 (0.0006) [2023-12-26 16:44:24,226][105620] Updated weights for policy 1, policy_version 189549 (0.0008) [2023-12-26 16:44:24,288][105620] Updated weights for policy 1, policy_version 189559 (0.0005) [2023-12-26 16:44:24,338][105620] Updated weights for policy 1, policy_version 189569 (0.0005) [2023-12-26 16:44:24,346][105692] Updated weights for policy 0, policy_version 188606 (0.0005) [2023-12-26 16:44:24,401][105692] Updated weights for policy 0, policy_version 188616 (0.0006) [2023-12-26 16:44:24,462][105692] Updated weights for policy 0, policy_version 188626 (0.0007) [2023-12-26 16:44:25,001][105620] Updated weights for policy 1, policy_version 189579 (0.0008) [2023-12-26 16:44:25,041][105692] Updated weights for policy 0, policy_version 188636 (0.0008) [2023-12-26 16:44:25,061][105620] Updated weights for policy 1, policy_version 189589 (0.0007) [2023-12-26 16:44:25,102][105692] Updated weights for policy 0, policy_version 188646 (0.0010) [2023-12-26 16:44:25,116][105620] Updated weights for policy 1, policy_version 189599 (0.0007) [2023-12-26 16:44:25,161][105692] Updated weights for policy 0, policy_version 188656 (0.0005) [2023-12-26 16:44:25,665][105620] Updated weights for policy 1, policy_version 189609 (0.0006) [2023-12-26 16:44:25,720][105692] Updated weights for policy 0, policy_version 188666 (0.0005) [2023-12-26 16:44:25,731][105620] Updated weights for policy 1, policy_version 189619 (0.0005) [2023-12-26 16:44:25,788][105692] Updated weights for policy 0, policy_version 188676 (0.0005) [2023-12-26 16:44:25,790][105620] Updated weights for policy 1, policy_version 189629 (0.0005) [2023-12-26 16:44:25,840][105692] Updated weights for policy 0, policy_version 188686 (0.0009) [2023-12-26 16:44:25,843][105620] Updated weights for policy 1, policy_version 189639 (0.0005) [2023-12-26 16:44:25,893][105692] Updated weights for policy 0, policy_version 188696 (0.0007) [2023-12-26 16:44:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 96870400. Throughput: 0: 10248.1, 1: 9410.8. Samples: 96875080. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-26 16:44:26,062][104569] Avg episode reward: [(0, '8548.164'), (1, '9084.062')] [2023-12-26 16:44:26,523][105620] Updated weights for policy 1, policy_version 189649 (0.0007) [2023-12-26 16:44:26,543][105692] Updated weights for policy 0, policy_version 188706 (0.0010) [2023-12-26 16:44:26,585][105620] Updated weights for policy 1, policy_version 189659 (0.0005) [2023-12-26 16:44:26,589][105692] Updated weights for policy 0, policy_version 188716 (0.0010) [2023-12-26 16:44:26,634][105692] Updated weights for policy 0, policy_version 188726 (0.0010) [2023-12-26 16:44:26,653][105620] Updated weights for policy 1, policy_version 189669 (0.0005) [2023-12-26 16:44:27,191][105620] Updated weights for policy 1, policy_version 189679 (0.0005) [2023-12-26 16:44:27,236][105620] Updated weights for policy 1, policy_version 189689 (0.0005) [2023-12-26 16:44:27,284][105692] Updated weights for policy 0, policy_version 188736 (0.0010) [2023-12-26 16:44:27,289][105620] Updated weights for policy 1, policy_version 189699 (0.0007) [2023-12-26 16:44:27,341][105692] Updated weights for policy 0, policy_version 188746 (0.0007) [2023-12-26 16:44:27,396][105692] Updated weights for policy 0, policy_version 188756 (0.0006) [2023-12-26 16:44:28,000][105620] Updated weights for policy 1, policy_version 189709 (0.0006) [2023-12-26 16:44:28,051][105620] Updated weights for policy 1, policy_version 189719 (0.0007) [2023-12-26 16:44:28,102][105620] Updated weights for policy 1, policy_version 189729 (0.0006) [2023-12-26 16:44:28,111][105692] Updated weights for policy 0, policy_version 188766 (0.0010) [2023-12-26 16:44:28,178][105692] Updated weights for policy 0, policy_version 188776 (0.0010) [2023-12-26 16:44:28,237][105692] Updated weights for policy 0, policy_version 188786 (0.0011) [2023-12-26 16:44:28,697][105620] Updated weights for policy 1, policy_version 189739 (0.0007) [2023-12-26 16:44:28,763][105620] Updated weights for policy 1, policy_version 189749 (0.0005) [2023-12-26 16:44:28,818][105620] Updated weights for policy 1, policy_version 189759 (0.0006) [2023-12-26 16:44:28,998][105692] Updated weights for policy 0, policy_version 188796 (0.0010) [2023-12-26 16:44:29,050][105692] Updated weights for policy 0, policy_version 188806 (0.0010) [2023-12-26 16:44:29,112][105692] Updated weights for policy 0, policy_version 188816 (0.0010) [2023-12-26 16:44:29,429][105620] Updated weights for policy 1, policy_version 189769 (0.0010) [2023-12-26 16:44:29,494][105620] Updated weights for policy 1, policy_version 189779 (0.0011) [2023-12-26 16:44:29,558][105620] Updated weights for policy 1, policy_version 189789 (0.0011) [2023-12-26 16:44:29,615][105620] Updated weights for policy 1, policy_version 189799 (0.0011) [2023-12-26 16:44:29,892][105692] Updated weights for policy 0, policy_version 188826 (0.0010) [2023-12-26 16:44:29,949][105692] Updated weights for policy 0, policy_version 188836 (0.0010) [2023-12-26 16:44:30,008][105692] Updated weights for policy 0, policy_version 188846 (0.0010) [2023-12-26 16:44:30,056][105692] Updated weights for policy 0, policy_version 188856 (0.0010) [2023-12-26 16:44:30,280][105620] Updated weights for policy 1, policy_version 189809 (0.0006) [2023-12-26 16:44:30,348][105620] Updated weights for policy 1, policy_version 189819 (0.0010) [2023-12-26 16:44:30,409][105620] Updated weights for policy 1, policy_version 189829 (0.0010) [2023-12-26 16:44:30,821][105692] Updated weights for policy 0, policy_version 188866 (0.0010) [2023-12-26 16:44:30,882][105692] Updated weights for policy 0, policy_version 188876 (0.0010) [2023-12-26 16:44:30,944][105692] Updated weights for policy 0, policy_version 188886 (0.0010) [2023-12-26 16:44:30,960][105620] Updated weights for policy 1, policy_version 189839 (0.0011) [2023-12-26 16:44:31,029][105620] Updated weights for policy 1, policy_version 189849 (0.0011) [2023-12-26 16:44:31,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 96968704. Throughput: 0: 10303.1, 1: 9519.5. Samples: 96938360. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-26 16:44:31,063][104569] Avg episode reward: [(0, '9348.469'), (1, '9267.362')] [2023-12-26 16:44:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000188888_48365568.pth... [2023-12-26 16:44:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000187672_48054272.pth [2023-12-26 16:44:31,093][105620] Updated weights for policy 1, policy_version 189859 (0.0011) [2023-12-26 16:44:31,126][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000189864_48611328.pth... [2023-12-26 16:44:31,133][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000188744_48324608.pth [2023-12-26 16:44:31,691][105692] Updated weights for policy 0, policy_version 188896 (0.0010) [2023-12-26 16:44:31,755][105692] Updated weights for policy 0, policy_version 188906 (0.0009) [2023-12-26 16:44:31,780][105620] Updated weights for policy 1, policy_version 189869 (0.0008) [2023-12-26 16:44:31,810][105692] Updated weights for policy 0, policy_version 188916 (0.0007) [2023-12-26 16:44:31,838][105620] Updated weights for policy 1, policy_version 189879 (0.0006) [2023-12-26 16:44:31,893][105620] Updated weights for policy 1, policy_version 189889 (0.0005) [2023-12-26 16:44:32,487][105620] Updated weights for policy 1, policy_version 189899 (0.0007) [2023-12-26 16:44:32,521][105692] Updated weights for policy 0, policy_version 188926 (0.0006) [2023-12-26 16:44:32,542][105620] Updated weights for policy 1, policy_version 189909 (0.0011) [2023-12-26 16:44:32,565][105692] Updated weights for policy 0, policy_version 188936 (0.0005) [2023-12-26 16:44:32,597][105620] Updated weights for policy 1, policy_version 189919 (0.0010) [2023-12-26 16:44:32,611][105692] Updated weights for policy 0, policy_version 188946 (0.0008) [2023-12-26 16:44:33,336][105692] Updated weights for policy 0, policy_version 188956 (0.0006) [2023-12-26 16:44:33,352][105620] Updated weights for policy 1, policy_version 189929 (0.0010) [2023-12-26 16:44:33,397][105692] Updated weights for policy 0, policy_version 188966 (0.0005) [2023-12-26 16:44:33,400][105620] Updated weights for policy 1, policy_version 189939 (0.0010) [2023-12-26 16:44:33,451][105692] Updated weights for policy 0, policy_version 188976 (0.0006) [2023-12-26 16:44:33,452][105620] Updated weights for policy 1, policy_version 189949 (0.0010) [2023-12-26 16:44:33,507][105620] Updated weights for policy 1, policy_version 189959 (0.0010) [2023-12-26 16:44:34,184][105692] Updated weights for policy 0, policy_version 188986 (0.0007) [2023-12-26 16:44:34,244][105692] Updated weights for policy 0, policy_version 188996 (0.0009) [2023-12-26 16:44:34,277][105620] Updated weights for policy 1, policy_version 189969 (0.0009) [2023-12-26 16:44:34,301][105692] Updated weights for policy 0, policy_version 189006 (0.0009) [2023-12-26 16:44:34,325][105620] Updated weights for policy 1, policy_version 189979 (0.0009) [2023-12-26 16:44:34,363][105692] Updated weights for policy 0, policy_version 189016 (0.0006) [2023-12-26 16:44:34,390][105620] Updated weights for policy 1, policy_version 189989 (0.0007) [2023-12-26 16:44:35,139][105692] Updated weights for policy 0, policy_version 189026 (0.0007) [2023-12-26 16:44:35,145][105620] Updated weights for policy 1, policy_version 189999 (0.0008) [2023-12-26 16:44:35,185][105692] Updated weights for policy 0, policy_version 189036 (0.0005) [2023-12-26 16:44:35,191][105620] Updated weights for policy 1, policy_version 190009 (0.0008) [2023-12-26 16:44:35,250][105692] Updated weights for policy 0, policy_version 189046 (0.0005) [2023-12-26 16:44:35,258][105620] Updated weights for policy 1, policy_version 190019 (0.0008) [2023-12-26 16:44:35,831][105692] Updated weights for policy 0, policy_version 189056 (0.0006) [2023-12-26 16:44:35,885][105692] Updated weights for policy 0, policy_version 189066 (0.0006) [2023-12-26 16:44:35,950][105692] Updated weights for policy 0, policy_version 189076 (0.0010) [2023-12-26 16:44:36,047][105620] Updated weights for policy 1, policy_version 190029 (0.0009) [2023-12-26 16:44:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19716.3). Total num frames: 97067008. Throughput: 0: 10134.1, 1: 9584.3. Samples: 97055960. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-26 16:44:36,063][104569] Avg episode reward: [(0, '9349.354'), (1, '9357.856')] [2023-12-26 16:44:36,106][105620] Updated weights for policy 1, policy_version 190039 (0.0008) [2023-12-26 16:44:36,169][105620] Updated weights for policy 1, policy_version 190049 (0.0008) [2023-12-26 16:44:36,627][105692] Updated weights for policy 0, policy_version 189086 (0.0011) [2023-12-26 16:44:36,686][105692] Updated weights for policy 0, policy_version 189096 (0.0010) [2023-12-26 16:44:36,735][105692] Updated weights for policy 0, policy_version 189106 (0.0010) [2023-12-26 16:44:36,890][105620] Updated weights for policy 1, policy_version 190059 (0.0008) [2023-12-26 16:44:36,944][105620] Updated weights for policy 1, policy_version 190069 (0.0008) [2023-12-26 16:44:36,993][105620] Updated weights for policy 1, policy_version 190079 (0.0008) [2023-12-26 16:44:37,482][105692] Updated weights for policy 0, policy_version 189116 (0.0010) [2023-12-26 16:44:37,534][105692] Updated weights for policy 0, policy_version 189126 (0.0011) [2023-12-26 16:44:37,593][105692] Updated weights for policy 0, policy_version 189136 (0.0010) [2023-12-26 16:44:37,712][105620] Updated weights for policy 1, policy_version 190089 (0.0008) [2023-12-26 16:44:37,778][105620] Updated weights for policy 1, policy_version 190099 (0.0008) [2023-12-26 16:44:37,851][105620] Updated weights for policy 1, policy_version 190109 (0.0007) [2023-12-26 16:44:37,920][105620] Updated weights for policy 1, policy_version 190119 (0.0008) [2023-12-26 16:44:38,347][105692] Updated weights for policy 0, policy_version 189146 (0.0009) [2023-12-26 16:44:38,396][105692] Updated weights for policy 0, policy_version 189156 (0.0008) [2023-12-26 16:44:38,448][105692] Updated weights for policy 0, policy_version 189166 (0.0007) [2023-12-26 16:44:38,497][105692] Updated weights for policy 0, policy_version 189176 (0.0008) [2023-12-26 16:44:38,546][105620] Updated weights for policy 1, policy_version 190129 (0.0010) [2023-12-26 16:44:38,604][105620] Updated weights for policy 1, policy_version 190139 (0.0010) [2023-12-26 16:44:38,656][105620] Updated weights for policy 1, policy_version 190149 (0.0010) [2023-12-26 16:44:39,112][105692] Updated weights for policy 0, policy_version 189186 (0.0005) [2023-12-26 16:44:39,168][105692] Updated weights for policy 0, policy_version 189196 (0.0005) [2023-12-26 16:44:39,228][105692] Updated weights for policy 0, policy_version 189206 (0.0006) [2023-12-26 16:44:39,430][105620] Updated weights for policy 1, policy_version 190159 (0.0007) [2023-12-26 16:44:39,488][105620] Updated weights for policy 1, policy_version 190169 (0.0009) [2023-12-26 16:44:39,540][105620] Updated weights for policy 1, policy_version 190179 (0.0009) [2023-12-26 16:44:39,906][105692] Updated weights for policy 0, policy_version 189216 (0.0009) [2023-12-26 16:44:39,974][105692] Updated weights for policy 0, policy_version 189226 (0.0006) [2023-12-26 16:44:40,047][105692] Updated weights for policy 0, policy_version 189236 (0.0007) [2023-12-26 16:44:40,234][105620] Updated weights for policy 1, policy_version 190189 (0.0008) [2023-12-26 16:44:40,296][105620] Updated weights for policy 1, policy_version 190199 (0.0009) [2023-12-26 16:44:40,356][105620] Updated weights for policy 1, policy_version 190209 (0.0008) [2023-12-26 16:44:40,677][105692] Updated weights for policy 0, policy_version 189246 (0.0008) [2023-12-26 16:44:40,724][105692] Updated weights for policy 0, policy_version 189256 (0.0008) [2023-12-26 16:44:40,777][105692] Updated weights for policy 0, policy_version 189266 (0.0006) [2023-12-26 16:44:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.4, 300 sec: 19716.3). Total num frames: 97165312. Throughput: 0: 10124.1, 1: 9571.1. Samples: 97173580. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 16:44:41,062][104569] Avg episode reward: [(0, '9349.001'), (1, '9357.929')] [2023-12-26 16:44:41,165][105620] Updated weights for policy 1, policy_version 190219 (0.0008) [2023-12-26 16:44:41,225][105620] Updated weights for policy 1, policy_version 190229 (0.0008) [2023-12-26 16:44:41,292][105620] Updated weights for policy 1, policy_version 190239 (0.0008) [2023-12-26 16:44:41,505][105692] Updated weights for policy 0, policy_version 189276 (0.0007) [2023-12-26 16:44:41,568][105692] Updated weights for policy 0, policy_version 189286 (0.0011) [2023-12-26 16:44:41,631][105692] Updated weights for policy 0, policy_version 189296 (0.0007) [2023-12-26 16:44:42,042][105620] Updated weights for policy 1, policy_version 190249 (0.0008) [2023-12-26 16:44:42,098][105620] Updated weights for policy 1, policy_version 190259 (0.0008) [2023-12-26 16:44:42,150][105620] Updated weights for policy 1, policy_version 190269 (0.0008) [2023-12-26 16:44:42,194][105620] Updated weights for policy 1, policy_version 190279 (0.0008) [2023-12-26 16:44:42,328][105692] Updated weights for policy 0, policy_version 189306 (0.0008) [2023-12-26 16:44:42,388][105692] Updated weights for policy 0, policy_version 189316 (0.0009) [2023-12-26 16:44:42,444][105692] Updated weights for policy 0, policy_version 189326 (0.0009) [2023-12-26 16:44:42,499][105692] Updated weights for policy 0, policy_version 189336 (0.0008) [2023-12-26 16:44:43,004][105620] Updated weights for policy 1, policy_version 190289 (0.0009) [2023-12-26 16:44:43,054][105620] Updated weights for policy 1, policy_version 190299 (0.0009) [2023-12-26 16:44:43,116][105620] Updated weights for policy 1, policy_version 190309 (0.0008) [2023-12-26 16:44:43,171][105692] Updated weights for policy 0, policy_version 189346 (0.0009) [2023-12-26 16:44:43,225][105692] Updated weights for policy 0, policy_version 189356 (0.0009) [2023-12-26 16:44:43,282][105692] Updated weights for policy 0, policy_version 189366 (0.0009) [2023-12-26 16:44:43,793][105620] Updated weights for policy 1, policy_version 190319 (0.0008) [2023-12-26 16:44:43,854][105620] Updated weights for policy 1, policy_version 190329 (0.0010) [2023-12-26 16:44:43,908][105620] Updated weights for policy 1, policy_version 190339 (0.0009) [2023-12-26 16:44:44,031][105692] Updated weights for policy 0, policy_version 189376 (0.0010) [2023-12-26 16:44:44,088][105692] Updated weights for policy 0, policy_version 189386 (0.0010) [2023-12-26 16:44:44,157][105692] Updated weights for policy 0, policy_version 189396 (0.0010) [2023-12-26 16:44:44,612][105620] Updated weights for policy 1, policy_version 190349 (0.0008) [2023-12-26 16:44:44,675][105620] Updated weights for policy 1, policy_version 190359 (0.0005) [2023-12-26 16:44:44,738][105620] Updated weights for policy 1, policy_version 190369 (0.0005) [2023-12-26 16:44:44,884][105692] Updated weights for policy 0, policy_version 189406 (0.0008) [2023-12-26 16:44:44,948][105692] Updated weights for policy 0, policy_version 189416 (0.0006) [2023-12-26 16:44:45,010][105692] Updated weights for policy 0, policy_version 189426 (0.0006) [2023-12-26 16:44:45,462][105620] Updated weights for policy 1, policy_version 190379 (0.0008) [2023-12-26 16:44:45,518][105620] Updated weights for policy 1, policy_version 190389 (0.0008) [2023-12-26 16:44:45,569][105620] Updated weights for policy 1, policy_version 190399 (0.0008) [2023-12-26 16:44:45,654][105692] Updated weights for policy 0, policy_version 189436 (0.0008) [2023-12-26 16:44:45,704][105692] Updated weights for policy 0, policy_version 189446 (0.0010) [2023-12-26 16:44:45,755][105692] Updated weights for policy 0, policy_version 189456 (0.0010) [2023-12-26 16:44:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19688.6). Total num frames: 97263616. Throughput: 0: 10100.9, 1: 9569.7. Samples: 97230764. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 16:44:46,062][104569] Avg episode reward: [(0, '9348.147'), (1, '9357.955')] [2023-12-26 16:44:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000189464_48513024.pth... [2023-12-26 16:44:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000190408_48750592.pth... [2023-12-26 16:44:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000188280_48209920.pth [2023-12-26 16:44:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000189288_48463872.pth [2023-12-26 16:44:46,345][105620] Updated weights for policy 1, policy_version 190409 (0.0008) [2023-12-26 16:44:46,403][105620] Updated weights for policy 1, policy_version 190419 (0.0009) [2023-12-26 16:44:46,470][105620] Updated weights for policy 1, policy_version 190429 (0.0009) [2023-12-26 16:44:46,509][105692] Updated weights for policy 0, policy_version 189466 (0.0010) [2023-12-26 16:44:46,520][105620] Updated weights for policy 1, policy_version 190439 (0.0007) [2023-12-26 16:44:46,571][105692] Updated weights for policy 0, policy_version 189476 (0.0008) [2023-12-26 16:44:46,631][105692] Updated weights for policy 0, policy_version 189486 (0.0008) [2023-12-26 16:44:46,696][105692] Updated weights for policy 0, policy_version 189496 (0.0009) [2023-12-26 16:44:47,281][105692] Updated weights for policy 0, policy_version 189506 (0.0005) [2023-12-26 16:44:47,329][105620] Updated weights for policy 1, policy_version 190449 (0.0005) [2023-12-26 16:44:47,337][105692] Updated weights for policy 0, policy_version 189516 (0.0008) [2023-12-26 16:44:47,388][105620] Updated weights for policy 1, policy_version 190459 (0.0005) [2023-12-26 16:44:47,399][105692] Updated weights for policy 0, policy_version 189526 (0.0010) [2023-12-26 16:44:47,454][105620] Updated weights for policy 1, policy_version 190469 (0.0005) [2023-12-26 16:44:48,046][105692] Updated weights for policy 0, policy_version 189536 (0.0006) [2023-12-26 16:44:48,072][105620] Updated weights for policy 1, policy_version 190479 (0.0007) [2023-12-26 16:44:48,110][105692] Updated weights for policy 0, policy_version 189546 (0.0005) [2023-12-26 16:44:48,128][105620] Updated weights for policy 1, policy_version 190489 (0.0008) [2023-12-26 16:44:48,167][105692] Updated weights for policy 0, policy_version 189556 (0.0005) [2023-12-26 16:44:48,178][105620] Updated weights for policy 1, policy_version 190499 (0.0008) [2023-12-26 16:44:48,745][105692] Updated weights for policy 0, policy_version 189566 (0.0007) [2023-12-26 16:44:48,805][105692] Updated weights for policy 0, policy_version 189576 (0.0009) [2023-12-26 16:44:48,861][105692] Updated weights for policy 0, policy_version 189586 (0.0009) [2023-12-26 16:44:49,038][105620] Updated weights for policy 1, policy_version 190509 (0.0009) [2023-12-26 16:44:49,095][105620] Updated weights for policy 1, policy_version 190519 (0.0009) [2023-12-26 16:44:49,150][105620] Updated weights for policy 1, policy_version 190529 (0.0009) [2023-12-26 16:44:49,564][105692] Updated weights for policy 0, policy_version 189596 (0.0009) [2023-12-26 16:44:49,627][105692] Updated weights for policy 0, policy_version 189606 (0.0010) [2023-12-26 16:44:49,689][105692] Updated weights for policy 0, policy_version 189616 (0.0011) [2023-12-26 16:44:49,950][105620] Updated weights for policy 1, policy_version 190539 (0.0008) [2023-12-26 16:44:50,017][105620] Updated weights for policy 1, policy_version 190549 (0.0006) [2023-12-26 16:44:50,084][105620] Updated weights for policy 1, policy_version 190559 (0.0005) [2023-12-26 16:44:50,437][105692] Updated weights for policy 0, policy_version 189626 (0.0007) [2023-12-26 16:44:50,494][105692] Updated weights for policy 0, policy_version 189636 (0.0008) [2023-12-26 16:44:50,552][105692] Updated weights for policy 0, policy_version 189646 (0.0009) [2023-12-26 16:44:50,619][105692] Updated weights for policy 0, policy_version 189656 (0.0009) [2023-12-26 16:44:50,739][105620] Updated weights for policy 1, policy_version 190569 (0.0006) [2023-12-26 16:44:50,804][105620] Updated weights for policy 1, policy_version 190579 (0.0009) [2023-12-26 16:44:50,870][105620] Updated weights for policy 1, policy_version 190589 (0.0009) [2023-12-26 16:44:50,928][105620] Updated weights for policy 1, policy_version 190599 (0.0008) [2023-12-26 16:44:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 97361920. Throughput: 0: 10146.3, 1: 9511.3. Samples: 97349220. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 16:44:51,062][104569] Avg episode reward: [(0, '9348.852'), (1, '9265.392')] [2023-12-26 16:44:51,371][105692] Updated weights for policy 0, policy_version 189666 (0.0012) [2023-12-26 16:44:51,428][105692] Updated weights for policy 0, policy_version 189676 (0.0010) [2023-12-26 16:44:51,477][105692] Updated weights for policy 0, policy_version 189686 (0.0010) [2023-12-26 16:44:51,644][105620] Updated weights for policy 1, policy_version 190609 (0.0008) [2023-12-26 16:44:51,705][105620] Updated weights for policy 1, policy_version 190619 (0.0009) [2023-12-26 16:44:51,767][105620] Updated weights for policy 1, policy_version 190629 (0.0009) [2023-12-26 16:44:52,305][105692] Updated weights for policy 0, policy_version 189696 (0.0009) [2023-12-26 16:44:52,369][105692] Updated weights for policy 0, policy_version 189706 (0.0009) [2023-12-26 16:44:52,424][105692] Updated weights for policy 0, policy_version 189716 (0.0008) [2023-12-26 16:44:52,443][105620] Updated weights for policy 1, policy_version 190639 (0.0007) [2023-12-26 16:44:52,515][105620] Updated weights for policy 1, policy_version 190649 (0.0008) [2023-12-26 16:44:52,569][105620] Updated weights for policy 1, policy_version 190659 (0.0008) [2023-12-26 16:44:53,166][105692] Updated weights for policy 0, policy_version 189726 (0.0009) [2023-12-26 16:44:53,230][105692] Updated weights for policy 0, policy_version 189736 (0.0010) [2023-12-26 16:44:53,291][105692] Updated weights for policy 0, policy_version 189746 (0.0010) [2023-12-26 16:44:53,326][105620] Updated weights for policy 1, policy_version 190669 (0.0008) [2023-12-26 16:44:53,397][105620] Updated weights for policy 1, policy_version 190679 (0.0006) [2023-12-26 16:44:53,451][105620] Updated weights for policy 1, policy_version 190689 (0.0005) [2023-12-26 16:44:54,034][105692] Updated weights for policy 0, policy_version 189756 (0.0010) [2023-12-26 16:44:54,095][105692] Updated weights for policy 0, policy_version 189766 (0.0010) [2023-12-26 16:44:54,107][105620] Updated weights for policy 1, policy_version 190699 (0.0006) [2023-12-26 16:44:54,150][105692] Updated weights for policy 0, policy_version 189776 (0.0010) [2023-12-26 16:44:54,157][105620] Updated weights for policy 1, policy_version 190709 (0.0005) [2023-12-26 16:44:54,204][105620] Updated weights for policy 1, policy_version 190719 (0.0007) [2023-12-26 16:44:54,779][105692] Updated weights for policy 0, policy_version 189786 (0.0010) [2023-12-26 16:44:54,827][105692] Updated weights for policy 0, policy_version 189796 (0.0010) [2023-12-26 16:44:54,870][105692] Updated weights for policy 0, policy_version 189806 (0.0006) [2023-12-26 16:44:54,918][105692] Updated weights for policy 0, policy_version 189816 (0.0005) [2023-12-26 16:44:54,959][105620] Updated weights for policy 1, policy_version 190729 (0.0009) [2023-12-26 16:44:55,006][105620] Updated weights for policy 1, policy_version 190739 (0.0008) [2023-12-26 16:44:55,058][105620] Updated weights for policy 1, policy_version 190749 (0.0005) [2023-12-26 16:44:55,114][105620] Updated weights for policy 1, policy_version 190759 (0.0007) [2023-12-26 16:44:55,611][105692] Updated weights for policy 0, policy_version 189826 (0.0009) [2023-12-26 16:44:55,659][105692] Updated weights for policy 0, policy_version 189836 (0.0010) [2023-12-26 16:44:55,713][105692] Updated weights for policy 0, policy_version 189846 (0.0010) [2023-12-26 16:44:55,831][105620] Updated weights for policy 1, policy_version 190769 (0.0007) [2023-12-26 16:44:55,883][105620] Updated weights for policy 1, policy_version 190780 (0.0010) [2023-12-26 16:44:55,937][105620] Updated weights for policy 1, policy_version 190791 (0.0010) [2023-12-26 16:44:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.9, 300 sec: 19688.6). Total num frames: 97460224. Throughput: 0: 10070.3, 1: 9536.1. Samples: 97465732. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 16:44:56,062][104569] Avg episode reward: [(0, '9350.122'), (1, '9265.390')] [2023-12-26 16:44:56,379][105692] Updated weights for policy 0, policy_version 189856 (0.0006) [2023-12-26 16:44:56,448][105692] Updated weights for policy 0, policy_version 189866 (0.0005) [2023-12-26 16:44:56,517][105692] Updated weights for policy 0, policy_version 189876 (0.0005) [2023-12-26 16:44:56,694][105620] Updated weights for policy 1, policy_version 190801 (0.0009) [2023-12-26 16:44:56,752][105620] Updated weights for policy 1, policy_version 190813 (0.0010) [2023-12-26 16:44:56,808][105620] Updated weights for policy 1, policy_version 190823 (0.0009) [2023-12-26 16:44:57,010][105692] Updated weights for policy 0, policy_version 189886 (0.0005) [2023-12-26 16:44:57,071][105692] Updated weights for policy 0, policy_version 189896 (0.0005) [2023-12-26 16:44:57,128][105692] Updated weights for policy 0, policy_version 189906 (0.0005) [2023-12-26 16:44:57,683][105620] Updated weights for policy 1, policy_version 190833 (0.0009) [2023-12-26 16:44:57,718][105692] Updated weights for policy 0, policy_version 189916 (0.0007) [2023-12-26 16:44:57,731][105620] Updated weights for policy 1, policy_version 190843 (0.0005) [2023-12-26 16:44:57,769][105692] Updated weights for policy 0, policy_version 189926 (0.0007) [2023-12-26 16:44:57,778][105620] Updated weights for policy 1, policy_version 190853 (0.0005) [2023-12-26 16:44:57,850][105692] Updated weights for policy 0, policy_version 189936 (0.0005) [2023-12-26 16:44:58,470][105692] Updated weights for policy 0, policy_version 189946 (0.0007) [2023-12-26 16:44:58,533][105692] Updated weights for policy 0, policy_version 189956 (0.0008) [2023-12-26 16:44:58,566][105620] Updated weights for policy 1, policy_version 190863 (0.0007) [2023-12-26 16:44:58,603][105692] Updated weights for policy 0, policy_version 189966 (0.0010) [2023-12-26 16:44:58,626][105620] Updated weights for policy 1, policy_version 190873 (0.0008) [2023-12-26 16:44:58,669][105692] Updated weights for policy 0, policy_version 189976 (0.0008) [2023-12-26 16:44:58,690][105620] Updated weights for policy 1, policy_version 190883 (0.0008) [2023-12-26 16:44:59,382][105692] Updated weights for policy 0, policy_version 189986 (0.0008) [2023-12-26 16:44:59,433][105692] Updated weights for policy 0, policy_version 189996 (0.0007) [2023-12-26 16:44:59,491][105692] Updated weights for policy 0, policy_version 190006 (0.0006) [2023-12-26 16:44:59,493][105620] Updated weights for policy 1, policy_version 190893 (0.0008) [2023-12-26 16:44:59,553][105620] Updated weights for policy 1, policy_version 190903 (0.0008) [2023-12-26 16:44:59,614][105620] Updated weights for policy 1, policy_version 190913 (0.0009) [2023-12-26 16:45:00,234][105692] Updated weights for policy 0, policy_version 190016 (0.0009) [2023-12-26 16:45:00,292][105692] Updated weights for policy 0, policy_version 190026 (0.0007) [2023-12-26 16:45:00,352][105692] Updated weights for policy 0, policy_version 190036 (0.0007) [2023-12-26 16:45:00,388][105620] Updated weights for policy 1, policy_version 190923 (0.0009) [2023-12-26 16:45:00,438][105620] Updated weights for policy 1, policy_version 190933 (0.0008) [2023-12-26 16:45:00,496][105620] Updated weights for policy 1, policy_version 190943 (0.0009) [2023-12-26 16:45:01,036][105692] Updated weights for policy 0, policy_version 190046 (0.0008) [2023-12-26 16:45:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 97550336. Throughput: 0: 10165.3, 1: 9521.4. Samples: 97525356. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 16:45:01,063][104569] Avg episode reward: [(0, '9350.384'), (1, '9357.959')] [2023-12-26 16:45:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000190952_48889856.pth... [2023-12-26 16:45:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000189864_48611328.pth [2023-12-26 16:45:01,100][105692] Updated weights for policy 0, policy_version 190056 (0.0009) [2023-12-26 16:45:01,161][105692] Updated weights for policy 0, policy_version 190066 (0.0008) [2023-12-26 16:45:01,193][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000190072_48668672.pth... [2023-12-26 16:45:01,199][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000188888_48365568.pth [2023-12-26 16:45:01,248][105620] Updated weights for policy 1, policy_version 190953 (0.0009) [2023-12-26 16:45:01,310][105620] Updated weights for policy 1, policy_version 190963 (0.0008) [2023-12-26 16:45:01,378][105620] Updated weights for policy 1, policy_version 190973 (0.0008) [2023-12-26 16:45:01,434][105620] Updated weights for policy 1, policy_version 190983 (0.0006) [2023-12-26 16:45:01,900][105692] Updated weights for policy 0, policy_version 190076 (0.0008) [2023-12-26 16:45:01,958][105692] Updated weights for policy 0, policy_version 190086 (0.0005) [2023-12-26 16:45:02,022][105692] Updated weights for policy 0, policy_version 190096 (0.0005) [2023-12-26 16:45:02,149][105620] Updated weights for policy 1, policy_version 190993 (0.0008) [2023-12-26 16:45:02,208][105620] Updated weights for policy 1, policy_version 191004 (0.0009) [2023-12-26 16:45:02,267][105620] Updated weights for policy 1, policy_version 191014 (0.0010) [2023-12-26 16:45:02,645][105692] Updated weights for policy 0, policy_version 190106 (0.0008) [2023-12-26 16:45:02,700][105692] Updated weights for policy 0, policy_version 190116 (0.0009) [2023-12-26 16:45:02,749][105692] Updated weights for policy 0, policy_version 190126 (0.0008) [2023-12-26 16:45:02,801][105692] Updated weights for policy 0, policy_version 190136 (0.0007) [2023-12-26 16:45:03,039][105620] Updated weights for policy 1, policy_version 191024 (0.0010) [2023-12-26 16:45:03,122][105620] Updated weights for policy 1, policy_version 191034 (0.0010) [2023-12-26 16:45:03,167][105620] Updated weights for policy 1, policy_version 191044 (0.0010) [2023-12-26 16:45:03,513][105692] Updated weights for policy 0, policy_version 190146 (0.0010) [2023-12-26 16:45:03,563][105692] Updated weights for policy 0, policy_version 190156 (0.0010) [2023-12-26 16:45:03,608][105692] Updated weights for policy 0, policy_version 190166 (0.0010) [2023-12-26 16:45:03,784][105620] Updated weights for policy 1, policy_version 191054 (0.0010) [2023-12-26 16:45:03,832][105620] Updated weights for policy 1, policy_version 191064 (0.0010) [2023-12-26 16:45:03,903][105620] Updated weights for policy 1, policy_version 191074 (0.0008) [2023-12-26 16:45:04,358][105692] Updated weights for policy 0, policy_version 190176 (0.0009) [2023-12-26 16:45:04,410][105692] Updated weights for policy 0, policy_version 190186 (0.0008) [2023-12-26 16:45:04,465][105692] Updated weights for policy 0, policy_version 190196 (0.0009) [2023-12-26 16:45:04,506][105620] Updated weights for policy 1, policy_version 191084 (0.0006) [2023-12-26 16:45:04,562][105620] Updated weights for policy 1, policy_version 191094 (0.0009) [2023-12-26 16:45:04,619][105620] Updated weights for policy 1, policy_version 191104 (0.0010) [2023-12-26 16:45:05,212][105692] Updated weights for policy 0, policy_version 190206 (0.0009) [2023-12-26 16:45:05,267][105692] Updated weights for policy 0, policy_version 190216 (0.0009) [2023-12-26 16:45:05,315][105692] Updated weights for policy 0, policy_version 190226 (0.0009) [2023-12-26 16:45:05,340][105620] Updated weights for policy 1, policy_version 191114 (0.0009) [2023-12-26 16:45:05,397][105620] Updated weights for policy 1, policy_version 191125 (0.0010) [2023-12-26 16:45:05,450][105620] Updated weights for policy 1, policy_version 191135 (0.0008) [2023-12-26 16:45:06,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 97648640. Throughput: 0: 10055.9, 1: 9637.4. Samples: 97641420. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 16:45:06,063][104569] Avg episode reward: [(0, '9349.681'), (1, '9265.801')] [2023-12-26 16:45:06,095][105692] Updated weights for policy 0, policy_version 190236 (0.0008) [2023-12-26 16:45:06,155][105692] Updated weights for policy 0, policy_version 190246 (0.0009) [2023-12-26 16:45:06,157][105620] Updated weights for policy 1, policy_version 191145 (0.0008) [2023-12-26 16:45:06,206][105692] Updated weights for policy 0, policy_version 190256 (0.0006) [2023-12-26 16:45:06,211][105620] Updated weights for policy 1, policy_version 191155 (0.0006) [2023-12-26 16:45:06,271][105620] Updated weights for policy 1, policy_version 191165 (0.0009) [2023-12-26 16:45:06,323][105620] Updated weights for policy 1, policy_version 191175 (0.0009) [2023-12-26 16:45:06,979][105692] Updated weights for policy 0, policy_version 190266 (0.0007) [2023-12-26 16:45:07,045][105692] Updated weights for policy 0, policy_version 190276 (0.0011) [2023-12-26 16:45:07,091][105620] Updated weights for policy 1, policy_version 191185 (0.0006) [2023-12-26 16:45:07,101][105692] Updated weights for policy 0, policy_version 190286 (0.0011) [2023-12-26 16:45:07,140][105620] Updated weights for policy 1, policy_version 191195 (0.0005) [2023-12-26 16:45:07,154][105692] Updated weights for policy 0, policy_version 190296 (0.0011) [2023-12-26 16:45:07,191][105620] Updated weights for policy 1, policy_version 191205 (0.0007) [2023-12-26 16:45:07,913][105692] Updated weights for policy 0, policy_version 190306 (0.0009) [2023-12-26 16:45:07,965][105620] Updated weights for policy 1, policy_version 191215 (0.0007) [2023-12-26 16:45:07,975][105692] Updated weights for policy 0, policy_version 190316 (0.0008) [2023-12-26 16:45:08,026][105620] Updated weights for policy 1, policy_version 191225 (0.0008) [2023-12-26 16:45:08,036][105692] Updated weights for policy 0, policy_version 190326 (0.0006) [2023-12-26 16:45:08,088][105620] Updated weights for policy 1, policy_version 191235 (0.0008) [2023-12-26 16:45:08,794][105620] Updated weights for policy 1, policy_version 191245 (0.0008) [2023-12-26 16:45:08,824][105692] Updated weights for policy 0, policy_version 190336 (0.0007) [2023-12-26 16:45:08,842][105620] Updated weights for policy 1, policy_version 191255 (0.0007) [2023-12-26 16:45:08,874][105692] Updated weights for policy 0, policy_version 190346 (0.0007) [2023-12-26 16:45:08,896][105620] Updated weights for policy 1, policy_version 191265 (0.0008) [2023-12-26 16:45:08,936][105692] Updated weights for policy 0, policy_version 190356 (0.0007) [2023-12-26 16:45:09,614][105692] Updated weights for policy 0, policy_version 190366 (0.0008) [2023-12-26 16:45:09,676][105692] Updated weights for policy 0, policy_version 190376 (0.0009) [2023-12-26 16:45:09,730][105692] Updated weights for policy 0, policy_version 190386 (0.0008) [2023-12-26 16:45:09,734][105620] Updated weights for policy 1, policy_version 191275 (0.0009) [2023-12-26 16:45:09,802][105620] Updated weights for policy 1, policy_version 191285 (0.0009) [2023-12-26 16:45:09,877][105620] Updated weights for policy 1, policy_version 191295 (0.0010) [2023-12-26 16:45:10,364][105692] Updated weights for policy 0, policy_version 190396 (0.0009) [2023-12-26 16:45:10,429][105692] Updated weights for policy 0, policy_version 190406 (0.0008) [2023-12-26 16:45:10,487][105692] Updated weights for policy 0, policy_version 190416 (0.0010) [2023-12-26 16:45:10,677][105620] Updated weights for policy 1, policy_version 191305 (0.0010) [2023-12-26 16:45:10,739][105620] Updated weights for policy 1, policy_version 191315 (0.0009) [2023-12-26 16:45:10,794][105620] Updated weights for policy 1, policy_version 191325 (0.0009) [2023-12-26 16:45:10,843][105620] Updated weights for policy 1, policy_version 191335 (0.0009) [2023-12-26 16:45:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 97746944. Throughput: 0: 9910.7, 1: 9615.3. Samples: 97753756. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 16:45:11,063][104569] Avg episode reward: [(0, '8464.048'), (1, '9265.780')] [2023-12-26 16:45:11,256][105692] Updated weights for policy 0, policy_version 190426 (0.0009) [2023-12-26 16:45:11,322][105692] Updated weights for policy 0, policy_version 190436 (0.0010) [2023-12-26 16:45:11,390][105692] Updated weights for policy 0, policy_version 190446 (0.0009) [2023-12-26 16:45:11,459][105692] Updated weights for policy 0, policy_version 190456 (0.0009) [2023-12-26 16:45:11,568][105620] Updated weights for policy 1, policy_version 191345 (0.0009) [2023-12-26 16:45:11,615][105620] Updated weights for policy 1, policy_version 191355 (0.0008) [2023-12-26 16:45:11,678][105620] Updated weights for policy 1, policy_version 191365 (0.0009) [2023-12-26 16:45:12,127][105692] Updated weights for policy 0, policy_version 190466 (0.0006) [2023-12-26 16:45:12,175][105692] Updated weights for policy 0, policy_version 190476 (0.0009) [2023-12-26 16:45:12,225][105692] Updated weights for policy 0, policy_version 190486 (0.0008) [2023-12-26 16:45:12,568][105620] Updated weights for policy 1, policy_version 191375 (0.0008) [2023-12-26 16:45:12,629][105620] Updated weights for policy 1, policy_version 191385 (0.0008) [2023-12-26 16:45:12,699][105620] Updated weights for policy 1, policy_version 191395 (0.0010) [2023-12-26 16:45:12,988][105692] Updated weights for policy 0, policy_version 190496 (0.0006) [2023-12-26 16:45:13,052][105692] Updated weights for policy 0, policy_version 190506 (0.0006) [2023-12-26 16:45:13,111][105692] Updated weights for policy 0, policy_version 190516 (0.0006) [2023-12-26 16:45:13,553][105620] Updated weights for policy 1, policy_version 191405 (0.0009) [2023-12-26 16:45:13,609][105692] Updated weights for policy 0, policy_version 190526 (0.0005) [2023-12-26 16:45:13,610][105620] Updated weights for policy 1, policy_version 191415 (0.0009) [2023-12-26 16:45:13,659][105620] Updated weights for policy 1, policy_version 191425 (0.0008) [2023-12-26 16:45:13,666][105692] Updated weights for policy 0, policy_version 190536 (0.0005) [2023-12-26 16:45:13,719][105692] Updated weights for policy 0, policy_version 190546 (0.0005) [2023-12-26 16:45:14,320][105692] Updated weights for policy 0, policy_version 190556 (0.0005) [2023-12-26 16:45:14,371][105692] Updated weights for policy 0, policy_version 190566 (0.0005) [2023-12-26 16:45:14,426][105692] Updated weights for policy 0, policy_version 190576 (0.0005) [2023-12-26 16:45:14,514][105620] Updated weights for policy 1, policy_version 191435 (0.0008) [2023-12-26 16:45:14,572][105620] Updated weights for policy 1, policy_version 191445 (0.0010) [2023-12-26 16:45:14,633][105620] Updated weights for policy 1, policy_version 191456 (0.0010) [2023-12-26 16:45:15,008][105692] Updated weights for policy 0, policy_version 190586 (0.0006) [2023-12-26 16:45:15,063][105692] Updated weights for policy 0, policy_version 190596 (0.0009) [2023-12-26 16:45:15,126][105692] Updated weights for policy 0, policy_version 190606 (0.0009) [2023-12-26 16:45:15,185][105692] Updated weights for policy 0, policy_version 190616 (0.0009) [2023-12-26 16:45:15,408][105620] Updated weights for policy 1, policy_version 191466 (0.0008) [2023-12-26 16:45:15,456][105620] Updated weights for policy 1, policy_version 191476 (0.0009) [2023-12-26 16:45:15,508][105620] Updated weights for policy 1, policy_version 191486 (0.0009) [2023-12-26 16:45:15,569][105620] Updated weights for policy 1, policy_version 191496 (0.0009) [2023-12-26 16:45:15,938][105692] Updated weights for policy 0, policy_version 190626 (0.0009) [2023-12-26 16:45:15,984][105692] Updated weights for policy 0, policy_version 190636 (0.0008) [2023-12-26 16:45:16,038][105692] Updated weights for policy 0, policy_version 190646 (0.0008) [2023-12-26 16:45:16,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 97845248. Throughput: 0: 9909.1, 1: 9470.6. Samples: 97810444. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 16:45:16,062][104569] Avg episode reward: [(0, '8501.937'), (1, '9357.769')] [2023-12-26 16:45:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000190648_48816128.pth... [2023-12-26 16:45:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000191496_49029120.pth... [2023-12-26 16:45:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000190408_48750592.pth [2023-12-26 16:45:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000189464_48513024.pth [2023-12-26 16:45:16,321][105620] Updated weights for policy 1, policy_version 191506 (0.0007) [2023-12-26 16:45:16,373][105620] Updated weights for policy 1, policy_version 191516 (0.0008) [2023-12-26 16:45:16,429][105620] Updated weights for policy 1, policy_version 191526 (0.0008) [2023-12-26 16:45:16,743][105692] Updated weights for policy 0, policy_version 190656 (0.0006) [2023-12-26 16:45:16,789][105692] Updated weights for policy 0, policy_version 190666 (0.0005) [2023-12-26 16:45:16,839][105692] Updated weights for policy 0, policy_version 190676 (0.0010) [2023-12-26 16:45:17,227][105620] Updated weights for policy 1, policy_version 191536 (0.0010) [2023-12-26 16:45:17,287][105620] Updated weights for policy 1, policy_version 191546 (0.0009) [2023-12-26 16:45:17,343][105620] Updated weights for policy 1, policy_version 191556 (0.0008) [2023-12-26 16:45:17,535][105692] Updated weights for policy 0, policy_version 190686 (0.0009) [2023-12-26 16:45:17,596][105692] Updated weights for policy 0, policy_version 190696 (0.0009) [2023-12-26 16:45:17,647][105692] Updated weights for policy 0, policy_version 190706 (0.0009) [2023-12-26 16:45:18,161][105620] Updated weights for policy 1, policy_version 191566 (0.0007) [2023-12-26 16:45:18,216][105620] Updated weights for policy 1, policy_version 191576 (0.0010) [2023-12-26 16:45:18,249][105692] Updated weights for policy 0, policy_version 190716 (0.0006) [2023-12-26 16:45:18,271][105620] Updated weights for policy 1, policy_version 191586 (0.0010) [2023-12-26 16:45:18,308][105692] Updated weights for policy 0, policy_version 190726 (0.0005) [2023-12-26 16:45:18,370][105692] Updated weights for policy 0, policy_version 190736 (0.0008) [2023-12-26 16:45:18,974][105692] Updated weights for policy 0, policy_version 190746 (0.0008) [2023-12-26 16:45:19,002][105620] Updated weights for policy 1, policy_version 191596 (0.0011) [2023-12-26 16:45:19,028][105692] Updated weights for policy 0, policy_version 190756 (0.0006) [2023-12-26 16:45:19,065][105620] Updated weights for policy 1, policy_version 191606 (0.0011) [2023-12-26 16:45:19,090][105692] Updated weights for policy 0, policy_version 190766 (0.0007) [2023-12-26 16:45:19,131][105620] Updated weights for policy 1, policy_version 191616 (0.0011) [2023-12-26 16:45:19,136][105692] Updated weights for policy 0, policy_version 190776 (0.0009) [2023-12-26 16:45:19,861][105692] Updated weights for policy 0, policy_version 190786 (0.0009) [2023-12-26 16:45:19,870][105620] Updated weights for policy 1, policy_version 191626 (0.0010) [2023-12-26 16:45:19,924][105692] Updated weights for policy 0, policy_version 190796 (0.0008) [2023-12-26 16:45:19,934][105620] Updated weights for policy 1, policy_version 191636 (0.0008) [2023-12-26 16:45:19,978][105692] Updated weights for policy 0, policy_version 190806 (0.0007) [2023-12-26 16:45:19,999][105620] Updated weights for policy 1, policy_version 191646 (0.0006) [2023-12-26 16:45:20,057][105620] Updated weights for policy 1, policy_version 191656 (0.0006) [2023-12-26 16:45:20,667][105620] Updated weights for policy 1, policy_version 191666 (0.0010) [2023-12-26 16:45:20,737][105620] Updated weights for policy 1, policy_version 191676 (0.0009) [2023-12-26 16:45:20,791][105620] Updated weights for policy 1, policy_version 191686 (0.0007) [2023-12-26 16:45:20,795][105692] Updated weights for policy 0, policy_version 190816 (0.0008) [2023-12-26 16:45:20,859][105692] Updated weights for policy 0, policy_version 190826 (0.0008) [2023-12-26 16:45:20,933][105692] Updated weights for policy 0, policy_version 190836 (0.0007) [2023-12-26 16:45:21,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 97943552. Throughput: 0: 10058.5, 1: 9333.6. Samples: 97928604. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 16:45:21,062][104569] Avg episode reward: [(0, '9169.743'), (1, '9267.852')] [2023-12-26 16:45:21,571][105620] Updated weights for policy 1, policy_version 191696 (0.0008) [2023-12-26 16:45:21,626][105620] Updated weights for policy 1, policy_version 191706 (0.0009) [2023-12-26 16:45:21,668][105692] Updated weights for policy 0, policy_version 190846 (0.0008) [2023-12-26 16:45:21,687][105620] Updated weights for policy 1, policy_version 191716 (0.0010) [2023-12-26 16:45:21,723][105692] Updated weights for policy 0, policy_version 190856 (0.0006) [2023-12-26 16:45:21,789][105692] Updated weights for policy 0, policy_version 190866 (0.0010) [2023-12-26 16:45:22,376][105620] Updated weights for policy 1, policy_version 191726 (0.0011) [2023-12-26 16:45:22,439][105620] Updated weights for policy 1, policy_version 191736 (0.0010) [2023-12-26 16:45:22,498][105620] Updated weights for policy 1, policy_version 191746 (0.0010) [2023-12-26 16:45:22,555][105692] Updated weights for policy 0, policy_version 190876 (0.0009) [2023-12-26 16:45:22,622][105692] Updated weights for policy 0, policy_version 190886 (0.0009) [2023-12-26 16:45:22,679][105692] Updated weights for policy 0, policy_version 190896 (0.0008) [2023-12-26 16:45:23,248][105620] Updated weights for policy 1, policy_version 191756 (0.0011) [2023-12-26 16:45:23,297][105620] Updated weights for policy 1, policy_version 191766 (0.0010) [2023-12-26 16:45:23,346][105620] Updated weights for policy 1, policy_version 191776 (0.0010) [2023-12-26 16:45:23,390][105692] Updated weights for policy 0, policy_version 190906 (0.0008) [2023-12-26 16:45:23,444][105692] Updated weights for policy 0, policy_version 190916 (0.0005) [2023-12-26 16:45:23,504][105692] Updated weights for policy 0, policy_version 190926 (0.0006) [2023-12-26 16:45:23,568][105692] Updated weights for policy 0, policy_version 190936 (0.0005) [2023-12-26 16:45:24,093][105692] Updated weights for policy 0, policy_version 190946 (0.0010) [2023-12-26 16:45:24,097][105620] Updated weights for policy 1, policy_version 191786 (0.0011) [2023-12-26 16:45:24,148][105692] Updated weights for policy 0, policy_version 190956 (0.0010) [2023-12-26 16:45:24,153][105620] Updated weights for policy 1, policy_version 191796 (0.0010) [2023-12-26 16:45:24,199][105692] Updated weights for policy 0, policy_version 190966 (0.0010) [2023-12-26 16:45:24,201][105620] Updated weights for policy 1, policy_version 191806 (0.0010) [2023-12-26 16:45:24,249][105620] Updated weights for policy 1, policy_version 191816 (0.0010) [2023-12-26 16:45:24,794][105692] Updated weights for policy 0, policy_version 190976 (0.0006) [2023-12-26 16:45:24,849][105692] Updated weights for policy 0, policy_version 190986 (0.0005) [2023-12-26 16:45:24,905][105692] Updated weights for policy 0, policy_version 190996 (0.0005) [2023-12-26 16:45:25,037][105620] Updated weights for policy 1, policy_version 191826 (0.0011) [2023-12-26 16:45:25,102][105620] Updated weights for policy 1, policy_version 191836 (0.0010) [2023-12-26 16:45:25,171][105620] Updated weights for policy 1, policy_version 191846 (0.0011) [2023-12-26 16:45:25,560][105692] Updated weights for policy 0, policy_version 191006 (0.0006) [2023-12-26 16:45:25,604][105692] Updated weights for policy 0, policy_version 191016 (0.0005) [2023-12-26 16:45:25,650][105692] Updated weights for policy 0, policy_version 191026 (0.0007) [2023-12-26 16:45:25,871][105620] Updated weights for policy 1, policy_version 191856 (0.0009) [2023-12-26 16:45:25,913][105620] Updated weights for policy 1, policy_version 191866 (0.0006) [2023-12-26 16:45:25,958][105620] Updated weights for policy 1, policy_version 191876 (0.0005) [2023-12-26 16:45:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 98041856. Throughput: 0: 10039.5, 1: 9342.8. Samples: 98045784. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 16:45:26,062][104569] Avg episode reward: [(0, '9081.896'), (1, '9267.744')] [2023-12-26 16:45:26,306][105692] Updated weights for policy 0, policy_version 191036 (0.0008) [2023-12-26 16:45:26,363][105692] Updated weights for policy 0, policy_version 191046 (0.0008) [2023-12-26 16:45:26,411][105692] Updated weights for policy 0, policy_version 191056 (0.0010) [2023-12-26 16:45:26,561][105620] Updated weights for policy 1, policy_version 191886 (0.0008) [2023-12-26 16:45:26,609][105620] Updated weights for policy 1, policy_version 191896 (0.0010) [2023-12-26 16:45:26,665][105620] Updated weights for policy 1, policy_version 191906 (0.0010) [2023-12-26 16:45:27,057][105692] Updated weights for policy 0, policy_version 191066 (0.0009) [2023-12-26 16:45:27,104][105692] Updated weights for policy 0, policy_version 191076 (0.0008) [2023-12-26 16:45:27,159][105692] Updated weights for policy 0, policy_version 191086 (0.0005) [2023-12-26 16:45:27,208][105692] Updated weights for policy 0, policy_version 191096 (0.0005) [2023-12-26 16:45:27,422][105620] Updated weights for policy 1, policy_version 191916 (0.0010) [2023-12-26 16:45:27,493][105620] Updated weights for policy 1, policy_version 191926 (0.0010) [2023-12-26 16:45:27,557][105620] Updated weights for policy 1, policy_version 191936 (0.0010) [2023-12-26 16:45:27,831][105692] Updated weights for policy 0, policy_version 191106 (0.0008) [2023-12-26 16:45:27,901][105692] Updated weights for policy 0, policy_version 191116 (0.0009) [2023-12-26 16:45:27,967][105692] Updated weights for policy 0, policy_version 191126 (0.0010) [2023-12-26 16:45:28,251][105620] Updated weights for policy 1, policy_version 191946 (0.0010) [2023-12-26 16:45:28,299][105620] Updated weights for policy 1, policy_version 191956 (0.0010) [2023-12-26 16:45:28,360][105620] Updated weights for policy 1, policy_version 191966 (0.0010) [2023-12-26 16:45:28,421][105620] Updated weights for policy 1, policy_version 191976 (0.0010) [2023-12-26 16:45:28,707][105692] Updated weights for policy 0, policy_version 191136 (0.0008) [2023-12-26 16:45:28,751][105692] Updated weights for policy 0, policy_version 191146 (0.0008) [2023-12-26 16:45:28,809][105692] Updated weights for policy 0, policy_version 191156 (0.0007) [2023-12-26 16:45:29,136][105620] Updated weights for policy 1, policy_version 191986 (0.0010) [2023-12-26 16:45:29,187][105620] Updated weights for policy 1, policy_version 191996 (0.0010) [2023-12-26 16:45:29,250][105620] Updated weights for policy 1, policy_version 192006 (0.0011) [2023-12-26 16:45:29,463][105692] Updated weights for policy 0, policy_version 191166 (0.0006) [2023-12-26 16:45:29,520][105692] Updated weights for policy 0, policy_version 191176 (0.0009) [2023-12-26 16:45:29,576][105692] Updated weights for policy 0, policy_version 191186 (0.0009) [2023-12-26 16:45:29,875][105620] Updated weights for policy 1, policy_version 192016 (0.0008) [2023-12-26 16:45:29,942][105620] Updated weights for policy 1, policy_version 192026 (0.0010) [2023-12-26 16:45:30,011][105620] Updated weights for policy 1, policy_version 192036 (0.0010) [2023-12-26 16:45:30,277][105692] Updated weights for policy 0, policy_version 191196 (0.0008) [2023-12-26 16:45:30,344][105692] Updated weights for policy 0, policy_version 191206 (0.0006) [2023-12-26 16:45:30,398][105692] Updated weights for policy 0, policy_version 191216 (0.0009) [2023-12-26 16:45:30,676][105620] Updated weights for policy 1, policy_version 192046 (0.0007) [2023-12-26 16:45:30,738][105620] Updated weights for policy 1, policy_version 192056 (0.0008) [2023-12-26 16:45:30,796][105620] Updated weights for policy 1, policy_version 192066 (0.0009) [2023-12-26 16:45:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 98140160. Throughput: 0: 10081.8, 1: 9402.0. Samples: 98107536. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 16:45:31,063][104569] Avg episode reward: [(0, '9098.729'), (1, '9267.465')] [2023-12-26 16:45:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000192072_49176576.pth... [2023-12-26 16:45:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000191224_48963584.pth... [2023-12-26 16:45:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000190952_48889856.pth [2023-12-26 16:45:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000190072_48668672.pth [2023-12-26 16:45:31,103][105692] Updated weights for policy 0, policy_version 191226 (0.0010) [2023-12-26 16:45:31,171][105692] Updated weights for policy 0, policy_version 191236 (0.0010) [2023-12-26 16:45:31,229][105692] Updated weights for policy 0, policy_version 191246 (0.0010) [2023-12-26 16:45:31,288][105692] Updated weights for policy 0, policy_version 191256 (0.0009) [2023-12-26 16:45:31,509][105620] Updated weights for policy 1, policy_version 192076 (0.0008) [2023-12-26 16:45:31,570][105620] Updated weights for policy 1, policy_version 192086 (0.0008) [2023-12-26 16:45:31,628][105620] Updated weights for policy 1, policy_version 192096 (0.0008) [2023-12-26 16:45:32,062][105692] Updated weights for policy 0, policy_version 191266 (0.0011) [2023-12-26 16:45:32,125][105692] Updated weights for policy 0, policy_version 191276 (0.0010) [2023-12-26 16:45:32,182][105692] Updated weights for policy 0, policy_version 191286 (0.0011) [2023-12-26 16:45:32,415][105620] Updated weights for policy 1, policy_version 192106 (0.0008) [2023-12-26 16:45:32,472][105620] Updated weights for policy 1, policy_version 192116 (0.0008) [2023-12-26 16:45:32,519][105620] Updated weights for policy 1, policy_version 192126 (0.0009) [2023-12-26 16:45:32,572][105620] Updated weights for policy 1, policy_version 192136 (0.0008) [2023-12-26 16:45:32,872][105692] Updated weights for policy 0, policy_version 191296 (0.0006) [2023-12-26 16:45:32,923][105692] Updated weights for policy 0, policy_version 191306 (0.0006) [2023-12-26 16:45:32,976][105692] Updated weights for policy 0, policy_version 191316 (0.0005) [2023-12-26 16:45:33,425][105620] Updated weights for policy 1, policy_version 192146 (0.0010) [2023-12-26 16:45:33,480][105620] Updated weights for policy 1, policy_version 192156 (0.0010) [2023-12-26 16:45:33,535][105620] Updated weights for policy 1, policy_version 192166 (0.0010) [2023-12-26 16:45:33,569][105692] Updated weights for policy 0, policy_version 191326 (0.0007) [2023-12-26 16:45:33,621][105692] Updated weights for policy 0, policy_version 191336 (0.0008) [2023-12-26 16:45:33,673][105692] Updated weights for policy 0, policy_version 191346 (0.0008) [2023-12-26 16:45:34,283][105620] Updated weights for policy 1, policy_version 192176 (0.0010) [2023-12-26 16:45:34,342][105620] Updated weights for policy 1, policy_version 192186 (0.0011) [2023-12-26 16:45:34,401][105620] Updated weights for policy 1, policy_version 192196 (0.0011) [2023-12-26 16:45:34,471][105692] Updated weights for policy 0, policy_version 191356 (0.0008) [2023-12-26 16:45:34,538][105692] Updated weights for policy 0, policy_version 191366 (0.0008) [2023-12-26 16:45:34,600][105692] Updated weights for policy 0, policy_version 191376 (0.0008) [2023-12-26 16:45:35,156][105620] Updated weights for policy 1, policy_version 192206 (0.0010) [2023-12-26 16:45:35,208][105620] Updated weights for policy 1, policy_version 192216 (0.0010) [2023-12-26 16:45:35,268][105620] Updated weights for policy 1, policy_version 192226 (0.0010) [2023-12-26 16:45:35,362][105692] Updated weights for policy 0, policy_version 191386 (0.0008) [2023-12-26 16:45:35,411][105692] Updated weights for policy 0, policy_version 191396 (0.0008) [2023-12-26 16:45:35,459][105692] Updated weights for policy 0, policy_version 191406 (0.0008) [2023-12-26 16:45:35,510][105692] Updated weights for policy 0, policy_version 191416 (0.0007) [2023-12-26 16:45:35,984][105620] Updated weights for policy 1, policy_version 192236 (0.0010) [2023-12-26 16:45:36,047][105620] Updated weights for policy 1, policy_version 192246 (0.0009) [2023-12-26 16:45:36,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 98230272. Throughput: 0: 10022.2, 1: 9405.0. Samples: 98223444. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 16:45:36,063][104569] Avg episode reward: [(0, '7655.867'), (1, '9085.532')] [2023-12-26 16:45:36,104][105620] Updated weights for policy 1, policy_version 192256 (0.0006) [2023-12-26 16:45:36,363][105692] Updated weights for policy 0, policy_version 191426 (0.0009) [2023-12-26 16:45:36,431][105692] Updated weights for policy 0, policy_version 191436 (0.0008) [2023-12-26 16:45:36,498][105692] Updated weights for policy 0, policy_version 191446 (0.0010) [2023-12-26 16:45:36,828][105620] Updated weights for policy 1, policy_version 192266 (0.0009) [2023-12-26 16:45:36,893][105620] Updated weights for policy 1, policy_version 192276 (0.0009) [2023-12-26 16:45:36,954][105620] Updated weights for policy 1, policy_version 192286 (0.0008) [2023-12-26 16:45:37,016][105620] Updated weights for policy 1, policy_version 192296 (0.0009) [2023-12-26 16:45:37,311][105692] Updated weights for policy 0, policy_version 191456 (0.0007) [2023-12-26 16:45:37,371][105692] Updated weights for policy 0, policy_version 191466 (0.0006) [2023-12-26 16:45:37,433][105692] Updated weights for policy 0, policy_version 191476 (0.0006) [2023-12-26 16:45:37,695][105620] Updated weights for policy 1, policy_version 192306 (0.0006) [2023-12-26 16:45:37,748][105620] Updated weights for policy 1, policy_version 192316 (0.0008) [2023-12-26 16:45:37,798][105620] Updated weights for policy 1, policy_version 192326 (0.0008) [2023-12-26 16:45:38,139][105692] Updated weights for policy 0, policy_version 191486 (0.0008) [2023-12-26 16:45:38,191][105692] Updated weights for policy 0, policy_version 191496 (0.0009) [2023-12-26 16:45:38,238][105692] Updated weights for policy 0, policy_version 191506 (0.0009) [2023-12-26 16:45:38,550][105620] Updated weights for policy 1, policy_version 192336 (0.0009) [2023-12-26 16:45:38,611][105620] Updated weights for policy 1, policy_version 192346 (0.0009) [2023-12-26 16:45:38,670][105620] Updated weights for policy 1, policy_version 192356 (0.0009) [2023-12-26 16:45:38,978][105692] Updated weights for policy 0, policy_version 191516 (0.0009) [2023-12-26 16:45:39,044][105692] Updated weights for policy 0, policy_version 191526 (0.0010) [2023-12-26 16:45:39,116][105692] Updated weights for policy 0, policy_version 191536 (0.0010) [2023-12-26 16:45:39,496][105620] Updated weights for policy 1, policy_version 192366 (0.0007) [2023-12-26 16:45:39,558][105620] Updated weights for policy 1, policy_version 192376 (0.0009) [2023-12-26 16:45:39,620][105620] Updated weights for policy 1, policy_version 192386 (0.0009) [2023-12-26 16:45:39,907][105692] Updated weights for policy 0, policy_version 191546 (0.0009) [2023-12-26 16:45:39,974][105692] Updated weights for policy 0, policy_version 191556 (0.0007) [2023-12-26 16:45:40,040][105692] Updated weights for policy 0, policy_version 191566 (0.0009) [2023-12-26 16:45:40,102][105692] Updated weights for policy 0, policy_version 191576 (0.0008) [2023-12-26 16:45:40,422][105620] Updated weights for policy 1, policy_version 192396 (0.0009) [2023-12-26 16:45:40,478][105620] Updated weights for policy 1, policy_version 192406 (0.0006) [2023-12-26 16:45:40,537][105620] Updated weights for policy 1, policy_version 192416 (0.0009) [2023-12-26 16:45:40,858][105692] Updated weights for policy 0, policy_version 191586 (0.0010) [2023-12-26 16:45:40,925][105692] Updated weights for policy 0, policy_version 191596 (0.0010) [2023-12-26 16:45:40,984][105692] Updated weights for policy 0, policy_version 191606 (0.0009) [2023-12-26 16:45:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 98328576. Throughput: 0: 9947.4, 1: 9354.9. Samples: 98334340. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 16:45:41,063][104569] Avg episode reward: [(0, '7279.039'), (1, '8903.381')] [2023-12-26 16:45:41,216][105620] Updated weights for policy 1, policy_version 192426 (0.0009) [2023-12-26 16:45:41,290][105620] Updated weights for policy 1, policy_version 192436 (0.0009) [2023-12-26 16:45:41,361][105620] Updated weights for policy 1, policy_version 192446 (0.0008) [2023-12-26 16:45:41,427][105620] Updated weights for policy 1, policy_version 192456 (0.0008) [2023-12-26 16:45:41,868][105692] Updated weights for policy 0, policy_version 191616 (0.0009) [2023-12-26 16:45:41,925][105692] Updated weights for policy 0, policy_version 191626 (0.0008) [2023-12-26 16:45:41,988][105692] Updated weights for policy 0, policy_version 191636 (0.0008) [2023-12-26 16:45:42,232][105620] Updated weights for policy 1, policy_version 192466 (0.0008) [2023-12-26 16:45:42,299][105620] Updated weights for policy 1, policy_version 192476 (0.0009) [2023-12-26 16:45:42,365][105620] Updated weights for policy 1, policy_version 192486 (0.0009) [2023-12-26 16:45:42,788][105692] Updated weights for policy 0, policy_version 191646 (0.0009) [2023-12-26 16:45:42,844][105692] Updated weights for policy 0, policy_version 191656 (0.0009) [2023-12-26 16:45:42,918][105692] Updated weights for policy 0, policy_version 191666 (0.0006) [2023-12-26 16:45:43,150][105620] Updated weights for policy 1, policy_version 192496 (0.0010) [2023-12-26 16:45:43,204][105620] Updated weights for policy 1, policy_version 192506 (0.0010) [2023-12-26 16:45:43,261][105620] Updated weights for policy 1, policy_version 192516 (0.0009) [2023-12-26 16:45:43,583][105692] Updated weights for policy 0, policy_version 191676 (0.0007) [2023-12-26 16:45:43,644][105692] Updated weights for policy 0, policy_version 191686 (0.0009) [2023-12-26 16:45:43,692][105692] Updated weights for policy 0, policy_version 191696 (0.0009) [2023-12-26 16:45:44,004][105620] Updated weights for policy 1, policy_version 192526 (0.0009) [2023-12-26 16:45:44,063][105620] Updated weights for policy 1, policy_version 192536 (0.0009) [2023-12-26 16:45:44,118][105620] Updated weights for policy 1, policy_version 192546 (0.0010) [2023-12-26 16:45:44,306][105692] Updated weights for policy 0, policy_version 191706 (0.0008) [2023-12-26 16:45:44,352][105692] Updated weights for policy 0, policy_version 191716 (0.0005) [2023-12-26 16:45:44,400][105692] Updated weights for policy 0, policy_version 191726 (0.0005) [2023-12-26 16:45:44,459][105692] Updated weights for policy 0, policy_version 191736 (0.0005) [2023-12-26 16:45:44,995][105620] Updated weights for policy 1, policy_version 192556 (0.0009) [2023-12-26 16:45:45,056][105620] Updated weights for policy 1, policy_version 192566 (0.0009) [2023-12-26 16:45:45,064][105692] Updated weights for policy 0, policy_version 191746 (0.0011) [2023-12-26 16:45:45,107][105620] Updated weights for policy 1, policy_version 192576 (0.0006) [2023-12-26 16:45:45,128][105692] Updated weights for policy 0, policy_version 191756 (0.0009) [2023-12-26 16:45:45,189][105692] Updated weights for policy 0, policy_version 191766 (0.0009) [2023-12-26 16:45:45,886][105620] Updated weights for policy 1, policy_version 192586 (0.0007) [2023-12-26 16:45:45,904][105692] Updated weights for policy 0, policy_version 191776 (0.0010) [2023-12-26 16:45:45,941][105620] Updated weights for policy 1, policy_version 192596 (0.0005) [2023-12-26 16:45:45,951][105692] Updated weights for policy 0, policy_version 191786 (0.0010) [2023-12-26 16:45:45,996][105620] Updated weights for policy 1, policy_version 192606 (0.0006) [2023-12-26 16:45:46,006][105692] Updated weights for policy 0, policy_version 191796 (0.0010) [2023-12-26 16:45:46,050][105620] Updated weights for policy 1, policy_version 192616 (0.0005) [2023-12-26 16:45:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 98426880. Throughput: 0: 9812.1, 1: 9374.9. Samples: 98388776. Policy #0 lag: (min: 25.0, avg: 45.3, max: 57.0) [2023-12-26 16:45:46,063][104569] Avg episode reward: [(0, '7820.897'), (1, '8812.287')] [2023-12-26 16:45:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000192616_49315840.pth... [2023-12-26 16:45:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000191800_49111040.pth... [2023-12-26 16:45:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000190648_48816128.pth [2023-12-26 16:45:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000191496_49029120.pth [2023-12-26 16:45:46,590][105620] Updated weights for policy 1, policy_version 192626 (0.0006) [2023-12-26 16:45:46,606][105692] Updated weights for policy 0, policy_version 191806 (0.0007) [2023-12-26 16:45:46,638][105620] Updated weights for policy 1, policy_version 192636 (0.0009) [2023-12-26 16:45:46,669][105692] Updated weights for policy 0, policy_version 191816 (0.0007) [2023-12-26 16:45:46,691][105620] Updated weights for policy 1, policy_version 192646 (0.0008) [2023-12-26 16:45:46,729][105692] Updated weights for policy 0, policy_version 191826 (0.0007) [2023-12-26 16:45:47,239][105692] Updated weights for policy 0, policy_version 191836 (0.0008) [2023-12-26 16:45:47,294][105692] Updated weights for policy 0, policy_version 191846 (0.0011) [2023-12-26 16:45:47,348][105620] Updated weights for policy 1, policy_version 192656 (0.0006) [2023-12-26 16:45:47,349][105692] Updated weights for policy 0, policy_version 191856 (0.0008) [2023-12-26 16:45:47,408][105620] Updated weights for policy 1, policy_version 192666 (0.0007) [2023-12-26 16:45:47,468][105620] Updated weights for policy 1, policy_version 192676 (0.0011) [2023-12-26 16:45:48,001][105692] Updated weights for policy 0, policy_version 191866 (0.0007) [2023-12-26 16:45:48,066][105692] Updated weights for policy 0, policy_version 191876 (0.0010) [2023-12-26 16:45:48,135][105692] Updated weights for policy 0, policy_version 191886 (0.0010) [2023-12-26 16:45:48,179][105620] Updated weights for policy 1, policy_version 192686 (0.0010) [2023-12-26 16:45:48,199][105692] Updated weights for policy 0, policy_version 191896 (0.0010) [2023-12-26 16:45:48,242][105620] Updated weights for policy 1, policy_version 192696 (0.0009) [2023-12-26 16:45:48,304][105620] Updated weights for policy 1, policy_version 192706 (0.0006) [2023-12-26 16:45:48,802][105692] Updated weights for policy 0, policy_version 191906 (0.0011) [2023-12-26 16:45:48,860][105692] Updated weights for policy 0, policy_version 191916 (0.0010) [2023-12-26 16:45:48,918][105692] Updated weights for policy 0, policy_version 191926 (0.0011) [2023-12-26 16:45:48,998][105620] Updated weights for policy 1, policy_version 192716 (0.0010) [2023-12-26 16:45:49,047][105620] Updated weights for policy 1, policy_version 192726 (0.0010) [2023-12-26 16:45:49,106][105620] Updated weights for policy 1, policy_version 192736 (0.0011) [2023-12-26 16:45:49,539][105692] Updated weights for policy 0, policy_version 191936 (0.0006) [2023-12-26 16:45:49,597][105692] Updated weights for policy 0, policy_version 191946 (0.0005) [2023-12-26 16:45:49,656][105692] Updated weights for policy 0, policy_version 191956 (0.0005) [2023-12-26 16:45:49,856][105620] Updated weights for policy 1, policy_version 192746 (0.0011) [2023-12-26 16:45:49,918][105620] Updated weights for policy 1, policy_version 192756 (0.0010) [2023-12-26 16:45:49,981][105620] Updated weights for policy 1, policy_version 192766 (0.0010) [2023-12-26 16:45:50,038][105620] Updated weights for policy 1, policy_version 192776 (0.0006) [2023-12-26 16:45:50,307][105692] Updated weights for policy 0, policy_version 191966 (0.0006) [2023-12-26 16:45:50,363][105692] Updated weights for policy 0, policy_version 191976 (0.0006) [2023-12-26 16:45:50,418][105692] Updated weights for policy 0, policy_version 191986 (0.0007) [2023-12-26 16:45:50,680][105620] Updated weights for policy 1, policy_version 192786 (0.0010) [2023-12-26 16:45:50,725][105620] Updated weights for policy 1, policy_version 192796 (0.0010) [2023-12-26 16:45:50,778][105620] Updated weights for policy 1, policy_version 192806 (0.0009) [2023-12-26 16:45:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 98525184. Throughput: 0: 9984.2, 1: 9385.3. Samples: 98513044. Policy #0 lag: (min: 25.0, avg: 45.3, max: 57.0) [2023-12-26 16:45:51,063][104569] Avg episode reward: [(0, '9102.277'), (1, '8814.191')] [2023-12-26 16:45:51,138][105692] Updated weights for policy 0, policy_version 191996 (0.0010) [2023-12-26 16:45:51,202][105692] Updated weights for policy 0, policy_version 192006 (0.0008) [2023-12-26 16:45:51,261][105692] Updated weights for policy 0, policy_version 192016 (0.0007) [2023-12-26 16:45:51,531][105620] Updated weights for policy 1, policy_version 192816 (0.0006) [2023-12-26 16:45:51,600][105620] Updated weights for policy 1, policy_version 192826 (0.0006) [2023-12-26 16:45:51,662][105620] Updated weights for policy 1, policy_version 192836 (0.0009) [2023-12-26 16:45:51,996][105692] Updated weights for policy 0, policy_version 192026 (0.0010) [2023-12-26 16:45:52,060][105692] Updated weights for policy 0, policy_version 192036 (0.0011) [2023-12-26 16:45:52,121][105692] Updated weights for policy 0, policy_version 192046 (0.0011) [2023-12-26 16:45:52,184][105692] Updated weights for policy 0, policy_version 192056 (0.0010) [2023-12-26 16:45:52,319][105620] Updated weights for policy 1, policy_version 192846 (0.0007) [2023-12-26 16:45:52,397][105620] Updated weights for policy 1, policy_version 192856 (0.0008) [2023-12-26 16:45:52,467][105620] Updated weights for policy 1, policy_version 192866 (0.0007) [2023-12-26 16:45:52,935][105692] Updated weights for policy 0, policy_version 192066 (0.0010) [2023-12-26 16:45:52,986][105692] Updated weights for policy 0, policy_version 192076 (0.0010) [2023-12-26 16:45:53,046][105692] Updated weights for policy 0, policy_version 192086 (0.0011) [2023-12-26 16:45:53,110][105620] Updated weights for policy 1, policy_version 192876 (0.0008) [2023-12-26 16:45:53,170][105620] Updated weights for policy 1, policy_version 192886 (0.0008) [2023-12-26 16:45:53,225][105620] Updated weights for policy 1, policy_version 192896 (0.0008) [2023-12-26 16:45:53,810][105692] Updated weights for policy 0, policy_version 192096 (0.0011) [2023-12-26 16:45:53,828][105620] Updated weights for policy 1, policy_version 192906 (0.0008) [2023-12-26 16:45:53,862][105692] Updated weights for policy 0, policy_version 192106 (0.0010) [2023-12-26 16:45:53,886][105620] Updated weights for policy 1, policy_version 192916 (0.0005) [2023-12-26 16:45:53,909][105692] Updated weights for policy 0, policy_version 192116 (0.0010) [2023-12-26 16:45:53,942][105620] Updated weights for policy 1, policy_version 192926 (0.0005) [2023-12-26 16:45:54,007][105620] Updated weights for policy 1, policy_version 192936 (0.0005) [2023-12-26 16:45:54,663][105692] Updated weights for policy 0, policy_version 192126 (0.0007) [2023-12-26 16:45:54,669][105620] Updated weights for policy 1, policy_version 192946 (0.0005) [2023-12-26 16:45:54,717][105692] Updated weights for policy 0, policy_version 192136 (0.0005) [2023-12-26 16:45:54,731][105620] Updated weights for policy 1, policy_version 192956 (0.0005) [2023-12-26 16:45:54,778][105692] Updated weights for policy 0, policy_version 192146 (0.0005) [2023-12-26 16:45:54,797][105620] Updated weights for policy 1, policy_version 192966 (0.0006) [2023-12-26 16:45:55,290][105692] Updated weights for policy 0, policy_version 192156 (0.0005) [2023-12-26 16:45:55,349][105692] Updated weights for policy 0, policy_version 192166 (0.0005) [2023-12-26 16:45:55,413][105692] Updated weights for policy 0, policy_version 192176 (0.0005) [2023-12-26 16:45:55,555][105620] Updated weights for policy 1, policy_version 192977 (0.0009) [2023-12-26 16:45:55,602][105620] Updated weights for policy 1, policy_version 192987 (0.0008) [2023-12-26 16:45:55,647][105620] Updated weights for policy 1, policy_version 192997 (0.0007) [2023-12-26 16:45:56,051][105692] Updated weights for policy 0, policy_version 192186 (0.0008) [2023-12-26 16:45:56,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 98623488. Throughput: 0: 10052.7, 1: 9504.7. Samples: 98633832. Policy #0 lag: (min: 25.0, avg: 45.3, max: 57.0) [2023-12-26 16:45:56,062][104569] Avg episode reward: [(0, '9089.946'), (1, '8275.659')] [2023-12-26 16:45:56,097][105692] Updated weights for policy 0, policy_version 192196 (0.0008) [2023-12-26 16:45:56,144][105692] Updated weights for policy 0, policy_version 192206 (0.0008) [2023-12-26 16:45:56,191][105692] Updated weights for policy 0, policy_version 192216 (0.0007) [2023-12-26 16:45:56,367][105620] Updated weights for policy 1, policy_version 193007 (0.0008) [2023-12-26 16:45:56,421][105620] Updated weights for policy 1, policy_version 193017 (0.0010) [2023-12-26 16:45:56,479][105620] Updated weights for policy 1, policy_version 193027 (0.0010) [2023-12-26 16:45:56,842][105692] Updated weights for policy 0, policy_version 192226 (0.0010) [2023-12-26 16:45:56,903][105692] Updated weights for policy 0, policy_version 192236 (0.0010) [2023-12-26 16:45:56,957][105692] Updated weights for policy 0, policy_version 192246 (0.0010) [2023-12-26 16:45:57,254][105620] Updated weights for policy 1, policy_version 193038 (0.0007) [2023-12-26 16:45:57,300][105620] Updated weights for policy 1, policy_version 193048 (0.0005) [2023-12-26 16:45:57,355][105620] Updated weights for policy 1, policy_version 193058 (0.0007) [2023-12-26 16:45:57,684][105692] Updated weights for policy 0, policy_version 192256 (0.0010) [2023-12-26 16:45:57,737][105692] Updated weights for policy 0, policy_version 192266 (0.0007) [2023-12-26 16:45:57,788][105692] Updated weights for policy 0, policy_version 192276 (0.0005) [2023-12-26 16:45:57,877][105620] Updated weights for policy 1, policy_version 193068 (0.0005) [2023-12-26 16:45:57,933][105620] Updated weights for policy 1, policy_version 193078 (0.0006) [2023-12-26 16:45:57,989][105620] Updated weights for policy 1, policy_version 193088 (0.0005) [2023-12-26 16:45:58,488][105692] Updated weights for policy 0, policy_version 192286 (0.0009) [2023-12-26 16:45:58,552][105692] Updated weights for policy 0, policy_version 192296 (0.0011) [2023-12-26 16:45:58,615][105692] Updated weights for policy 0, policy_version 192306 (0.0011) [2023-12-26 16:45:58,737][105620] Updated weights for policy 1, policy_version 193098 (0.0006) [2023-12-26 16:45:58,809][105620] Updated weights for policy 1, policy_version 193108 (0.0007) [2023-12-26 16:45:58,881][105620] Updated weights for policy 1, policy_version 193118 (0.0008) [2023-12-26 16:45:58,939][105620] Updated weights for policy 1, policy_version 193128 (0.0008) [2023-12-26 16:45:59,395][105692] Updated weights for policy 0, policy_version 192316 (0.0008) [2023-12-26 16:45:59,451][105692] Updated weights for policy 0, policy_version 192326 (0.0005) [2023-12-26 16:45:59,507][105692] Updated weights for policy 0, policy_version 192336 (0.0005) [2023-12-26 16:45:59,647][105620] Updated weights for policy 1, policy_version 193139 (0.0009) [2023-12-26 16:45:59,703][105620] Updated weights for policy 1, policy_version 193149 (0.0009) [2023-12-26 16:45:59,760][105620] Updated weights for policy 1, policy_version 193159 (0.0009) [2023-12-26 16:46:00,086][105692] Updated weights for policy 0, policy_version 192346 (0.0007) [2023-12-26 16:46:00,154][105692] Updated weights for policy 0, policy_version 192356 (0.0010) [2023-12-26 16:46:00,219][105692] Updated weights for policy 0, policy_version 192366 (0.0010) [2023-12-26 16:46:00,276][105692] Updated weights for policy 0, policy_version 192376 (0.0010) [2023-12-26 16:46:00,473][105620] Updated weights for policy 1, policy_version 193169 (0.0006) [2023-12-26 16:46:00,527][105620] Updated weights for policy 1, policy_version 193179 (0.0005) [2023-12-26 16:46:00,581][105620] Updated weights for policy 1, policy_version 193189 (0.0008) [2023-12-26 16:46:00,971][105692] Updated weights for policy 0, policy_version 192386 (0.0006) [2023-12-26 16:46:01,033][105692] Updated weights for policy 0, policy_version 192396 (0.0006) [2023-12-26 16:46:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 98721792. Throughput: 0: 10034.9, 1: 9597.3. Samples: 98693896. Policy #0 lag: (min: 25.0, avg: 45.3, max: 57.0) [2023-12-26 16:46:01,063][104569] Avg episode reward: [(0, '9090.756'), (1, '8547.815')] [2023-12-26 16:46:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000193192_49463296.pth... [2023-12-26 16:46:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000192072_49176576.pth [2023-12-26 16:46:01,102][105692] Updated weights for policy 0, policy_version 192406 (0.0007) [2023-12-26 16:46:01,113][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000192408_49266688.pth... [2023-12-26 16:46:01,116][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000191224_48963584.pth [2023-12-26 16:46:01,259][105620] Updated weights for policy 1, policy_version 193199 (0.0008) [2023-12-26 16:46:01,317][105620] Updated weights for policy 1, policy_version 193209 (0.0009) [2023-12-26 16:46:01,382][105620] Updated weights for policy 1, policy_version 193219 (0.0012) [2023-12-26 16:46:01,815][105692] Updated weights for policy 0, policy_version 192416 (0.0007) [2023-12-26 16:46:01,865][105692] Updated weights for policy 0, policy_version 192426 (0.0006) [2023-12-26 16:46:01,924][105692] Updated weights for policy 0, policy_version 192436 (0.0005) [2023-12-26 16:46:02,048][105620] Updated weights for policy 1, policy_version 193229 (0.0009) [2023-12-26 16:46:02,097][105620] Updated weights for policy 1, policy_version 193239 (0.0008) [2023-12-26 16:46:02,146][105620] Updated weights for policy 1, policy_version 193249 (0.0006) [2023-12-26 16:46:02,589][105692] Updated weights for policy 0, policy_version 192446 (0.0009) [2023-12-26 16:46:02,652][105692] Updated weights for policy 0, policy_version 192456 (0.0008) [2023-12-26 16:46:02,703][105692] Updated weights for policy 0, policy_version 192466 (0.0005) [2023-12-26 16:46:02,749][105620] Updated weights for policy 1, policy_version 193259 (0.0005) [2023-12-26 16:46:02,807][105620] Updated weights for policy 1, policy_version 193269 (0.0008) [2023-12-26 16:46:02,872][105620] Updated weights for policy 1, policy_version 193279 (0.0011) [2023-12-26 16:46:03,337][105692] Updated weights for policy 0, policy_version 192476 (0.0008) [2023-12-26 16:46:03,393][105692] Updated weights for policy 0, policy_version 192486 (0.0011) [2023-12-26 16:46:03,452][105692] Updated weights for policy 0, policy_version 192496 (0.0010) [2023-12-26 16:46:03,592][105620] Updated weights for policy 1, policy_version 193289 (0.0010) [2023-12-26 16:46:03,647][105620] Updated weights for policy 1, policy_version 193299 (0.0005) [2023-12-26 16:46:03,700][105620] Updated weights for policy 1, policy_version 193309 (0.0005) [2023-12-26 16:46:03,758][105620] Updated weights for policy 1, policy_version 193319 (0.0006) [2023-12-26 16:46:04,194][105692] Updated weights for policy 0, policy_version 192506 (0.0007) [2023-12-26 16:46:04,254][105692] Updated weights for policy 0, policy_version 192516 (0.0011) [2023-12-26 16:46:04,321][105692] Updated weights for policy 0, policy_version 192526 (0.0011) [2023-12-26 16:46:04,357][105620] Updated weights for policy 1, policy_version 193329 (0.0010) [2023-12-26 16:46:04,386][105692] Updated weights for policy 0, policy_version 192536 (0.0011) [2023-12-26 16:46:04,421][105620] Updated weights for policy 1, policy_version 193339 (0.0011) [2023-12-26 16:46:04,489][105620] Updated weights for policy 1, policy_version 193349 (0.0011) [2023-12-26 16:46:05,120][105692] Updated weights for policy 0, policy_version 192546 (0.0005) [2023-12-26 16:46:05,166][105620] Updated weights for policy 1, policy_version 193359 (0.0011) [2023-12-26 16:46:05,180][105692] Updated weights for policy 0, policy_version 192556 (0.0008) [2023-12-26 16:46:05,229][105620] Updated weights for policy 1, policy_version 193369 (0.0011) [2023-12-26 16:46:05,244][105692] Updated weights for policy 0, policy_version 192566 (0.0010) [2023-12-26 16:46:05,280][105620] Updated weights for policy 1, policy_version 193379 (0.0010) [2023-12-26 16:46:05,831][105692] Updated weights for policy 0, policy_version 192576 (0.0008) [2023-12-26 16:46:05,886][105692] Updated weights for policy 0, policy_version 192586 (0.0010) [2023-12-26 16:46:05,940][105692] Updated weights for policy 0, policy_version 192596 (0.0010) [2023-12-26 16:46:06,013][105620] Updated weights for policy 1, policy_version 193389 (0.0010) [2023-12-26 16:46:06,058][105620] Updated weights for policy 1, policy_version 193399 (0.0010) [2023-12-26 16:46:06,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.9, 300 sec: 19660.8). Total num frames: 98828288. Throughput: 0: 9964.0, 1: 9758.1. Samples: 98816096. Policy #0 lag: (min: 25.0, avg: 45.3, max: 57.0) [2023-12-26 16:46:06,062][104569] Avg episode reward: [(0, '9180.229'), (1, '8732.467')] [2023-12-26 16:46:06,104][105620] Updated weights for policy 1, policy_version 193409 (0.0008) [2023-12-26 16:46:06,647][105692] Updated weights for policy 0, policy_version 192606 (0.0008) [2023-12-26 16:46:06,707][105692] Updated weights for policy 0, policy_version 192616 (0.0008) [2023-12-26 16:46:06,769][105692] Updated weights for policy 0, policy_version 192626 (0.0009) [2023-12-26 16:46:06,876][105620] Updated weights for policy 1, policy_version 193419 (0.0010) [2023-12-26 16:46:06,936][105620] Updated weights for policy 1, policy_version 193429 (0.0010) [2023-12-26 16:46:07,003][105620] Updated weights for policy 1, policy_version 193439 (0.0011) [2023-12-26 16:46:07,542][105692] Updated weights for policy 0, policy_version 192636 (0.0007) [2023-12-26 16:46:07,596][105692] Updated weights for policy 0, policy_version 192646 (0.0005) [2023-12-26 16:46:07,659][105692] Updated weights for policy 0, policy_version 192656 (0.0005) [2023-12-26 16:46:07,743][105620] Updated weights for policy 1, policy_version 193449 (0.0011) [2023-12-26 16:46:07,790][105620] Updated weights for policy 1, policy_version 193459 (0.0009) [2023-12-26 16:46:07,844][105620] Updated weights for policy 1, policy_version 193469 (0.0009) [2023-12-26 16:46:07,888][105620] Updated weights for policy 1, policy_version 193479 (0.0010) [2023-12-26 16:46:08,190][105692] Updated weights for policy 0, policy_version 192666 (0.0006) [2023-12-26 16:46:08,251][105692] Updated weights for policy 0, policy_version 192676 (0.0007) [2023-12-26 16:46:08,311][105692] Updated weights for policy 0, policy_version 192686 (0.0008) [2023-12-26 16:46:08,376][105692] Updated weights for policy 0, policy_version 192696 (0.0009) [2023-12-26 16:46:08,640][105620] Updated weights for policy 1, policy_version 193489 (0.0009) [2023-12-26 16:46:08,699][105620] Updated weights for policy 1, policy_version 193499 (0.0009) [2023-12-26 16:46:08,768][105620] Updated weights for policy 1, policy_version 193509 (0.0009) [2023-12-26 16:46:09,046][105692] Updated weights for policy 0, policy_version 192706 (0.0005) [2023-12-26 16:46:09,108][105692] Updated weights for policy 0, policy_version 192716 (0.0007) [2023-12-26 16:46:09,170][105692] Updated weights for policy 0, policy_version 192726 (0.0009) [2023-12-26 16:46:09,616][105620] Updated weights for policy 1, policy_version 193519 (0.0010) [2023-12-26 16:46:09,683][105620] Updated weights for policy 1, policy_version 193529 (0.0008) [2023-12-26 16:46:09,746][105620] Updated weights for policy 1, policy_version 193539 (0.0010) [2023-12-26 16:46:09,908][105692] Updated weights for policy 0, policy_version 192736 (0.0008) [2023-12-26 16:46:09,975][105692] Updated weights for policy 0, policy_version 192746 (0.0008) [2023-12-26 16:46:10,035][105692] Updated weights for policy 0, policy_version 192756 (0.0008) [2023-12-26 16:46:10,442][105620] Updated weights for policy 1, policy_version 193549 (0.0009) [2023-12-26 16:46:10,502][105620] Updated weights for policy 1, policy_version 193559 (0.0005) [2023-12-26 16:46:10,551][105620] Updated weights for policy 1, policy_version 193569 (0.0009) [2023-12-26 16:46:10,846][105692] Updated weights for policy 0, policy_version 192766 (0.0009) [2023-12-26 16:46:10,910][105692] Updated weights for policy 0, policy_version 192776 (0.0008) [2023-12-26 16:46:10,966][105692] Updated weights for policy 0, policy_version 192786 (0.0008) [2023-12-26 16:46:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 98926592. Throughput: 0: 9963.8, 1: 9741.2. Samples: 98932512. Policy #0 lag: (min: 25.0, avg: 45.3, max: 57.0) [2023-12-26 16:46:11,063][104569] Avg episode reward: [(0, '9097.345'), (1, '8199.646')] [2023-12-26 16:46:11,294][105620] Updated weights for policy 1, policy_version 193579 (0.0010) [2023-12-26 16:46:11,365][105620] Updated weights for policy 1, policy_version 193589 (0.0008) [2023-12-26 16:46:11,428][105620] Updated weights for policy 1, policy_version 193599 (0.0008) [2023-12-26 16:46:11,816][105692] Updated weights for policy 0, policy_version 192796 (0.0007) [2023-12-26 16:46:11,876][105692] Updated weights for policy 0, policy_version 192806 (0.0010) [2023-12-26 16:46:11,937][105692] Updated weights for policy 0, policy_version 192816 (0.0009) [2023-12-26 16:46:12,181][105620] Updated weights for policy 1, policy_version 193609 (0.0010) [2023-12-26 16:46:12,250][105620] Updated weights for policy 1, policy_version 193619 (0.0006) [2023-12-26 16:46:12,319][105620] Updated weights for policy 1, policy_version 193629 (0.0008) [2023-12-26 16:46:12,388][105620] Updated weights for policy 1, policy_version 193639 (0.0007) [2023-12-26 16:46:12,663][105692] Updated weights for policy 0, policy_version 192826 (0.0008) [2023-12-26 16:46:12,726][105692] Updated weights for policy 0, policy_version 192836 (0.0006) [2023-12-26 16:46:12,782][105692] Updated weights for policy 0, policy_version 192846 (0.0005) [2023-12-26 16:46:12,845][105692] Updated weights for policy 0, policy_version 192856 (0.0009) [2023-12-26 16:46:13,094][105620] Updated weights for policy 1, policy_version 193649 (0.0009) [2023-12-26 16:46:13,147][105620] Updated weights for policy 1, policy_version 193659 (0.0005) [2023-12-26 16:46:13,203][105620] Updated weights for policy 1, policy_version 193669 (0.0005) [2023-12-26 16:46:13,402][105692] Updated weights for policy 0, policy_version 192866 (0.0011) [2023-12-26 16:46:13,449][105692] Updated weights for policy 0, policy_version 192876 (0.0010) [2023-12-26 16:46:13,504][105692] Updated weights for policy 0, policy_version 192886 (0.0010) [2023-12-26 16:46:13,848][105620] Updated weights for policy 1, policy_version 193679 (0.0007) [2023-12-26 16:46:13,916][105620] Updated weights for policy 1, policy_version 193689 (0.0009) [2023-12-26 16:46:13,971][105620] Updated weights for policy 1, policy_version 193699 (0.0008) [2023-12-26 16:46:14,187][105692] Updated weights for policy 0, policy_version 192896 (0.0011) [2023-12-26 16:46:14,239][105692] Updated weights for policy 0, policy_version 192906 (0.0010) [2023-12-26 16:46:14,298][105692] Updated weights for policy 0, policy_version 192916 (0.0011) [2023-12-26 16:46:14,706][105620] Updated weights for policy 1, policy_version 193709 (0.0009) [2023-12-26 16:46:14,774][105620] Updated weights for policy 1, policy_version 193719 (0.0007) [2023-12-26 16:46:14,832][105620] Updated weights for policy 1, policy_version 193729 (0.0008) [2023-12-26 16:46:15,050][105692] Updated weights for policy 0, policy_version 192926 (0.0009) [2023-12-26 16:46:15,100][105692] Updated weights for policy 0, policy_version 192936 (0.0009) [2023-12-26 16:46:15,148][105692] Updated weights for policy 0, policy_version 192946 (0.0005) [2023-12-26 16:46:15,535][105620] Updated weights for policy 1, policy_version 193739 (0.0009) [2023-12-26 16:46:15,597][105620] Updated weights for policy 1, policy_version 193749 (0.0009) [2023-12-26 16:46:15,651][105620] Updated weights for policy 1, policy_version 193759 (0.0009) [2023-12-26 16:46:15,917][105692] Updated weights for policy 0, policy_version 192956 (0.0008) [2023-12-26 16:46:15,966][105692] Updated weights for policy 0, policy_version 192966 (0.0008) [2023-12-26 16:46:16,014][105692] Updated weights for policy 0, policy_version 192976 (0.0008) [2023-12-26 16:46:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 99024896. Throughput: 0: 9915.5, 1: 9705.4. Samples: 98990480. Policy #0 lag: (min: 25.0, avg: 45.3, max: 57.0) [2023-12-26 16:46:16,063][104569] Avg episode reward: [(0, '9182.872'), (1, '8013.540')] [2023-12-26 16:46:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000192984_49414144.pth... [2023-12-26 16:46:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000193768_49610752.pth... [2023-12-26 16:46:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000191800_49111040.pth [2023-12-26 16:46:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000192616_49315840.pth [2023-12-26 16:46:16,435][105620] Updated weights for policy 1, policy_version 193769 (0.0009) [2023-12-26 16:46:16,484][105620] Updated weights for policy 1, policy_version 193779 (0.0010) [2023-12-26 16:46:16,540][105620] Updated weights for policy 1, policy_version 193789 (0.0010) [2023-12-26 16:46:16,597][105620] Updated weights for policy 1, policy_version 193799 (0.0006) [2023-12-26 16:46:16,769][105692] Updated weights for policy 0, policy_version 192986 (0.0008) [2023-12-26 16:46:16,819][105692] Updated weights for policy 0, policy_version 192996 (0.0005) [2023-12-26 16:46:16,878][105692] Updated weights for policy 0, policy_version 193006 (0.0005) [2023-12-26 16:46:16,939][105692] Updated weights for policy 0, policy_version 193016 (0.0007) [2023-12-26 16:46:17,293][105620] Updated weights for policy 1, policy_version 193809 (0.0006) [2023-12-26 16:46:17,357][105620] Updated weights for policy 1, policy_version 193819 (0.0005) [2023-12-26 16:46:17,426][105620] Updated weights for policy 1, policy_version 193829 (0.0005) [2023-12-26 16:46:17,499][105692] Updated weights for policy 0, policy_version 193026 (0.0005) [2023-12-26 16:46:17,550][105692] Updated weights for policy 0, policy_version 193036 (0.0005) [2023-12-26 16:46:17,603][105692] Updated weights for policy 0, policy_version 193046 (0.0005) [2023-12-26 16:46:17,987][105620] Updated weights for policy 1, policy_version 193839 (0.0007) [2023-12-26 16:46:18,049][105620] Updated weights for policy 1, policy_version 193849 (0.0006) [2023-12-26 16:46:18,113][105620] Updated weights for policy 1, policy_version 193859 (0.0005) [2023-12-26 16:46:18,157][105692] Updated weights for policy 0, policy_version 193056 (0.0010) [2023-12-26 16:46:18,222][105692] Updated weights for policy 0, policy_version 193066 (0.0010) [2023-12-26 16:46:18,284][105692] Updated weights for policy 0, policy_version 193076 (0.0010) [2023-12-26 16:46:18,785][105620] Updated weights for policy 1, policy_version 193869 (0.0007) [2023-12-26 16:46:18,846][105620] Updated weights for policy 1, policy_version 193879 (0.0008) [2023-12-26 16:46:18,909][105620] Updated weights for policy 1, policy_version 193889 (0.0010) [2023-12-26 16:46:18,946][105692] Updated weights for policy 0, policy_version 193086 (0.0008) [2023-12-26 16:46:19,005][105692] Updated weights for policy 0, policy_version 193096 (0.0011) [2023-12-26 16:46:19,067][105692] Updated weights for policy 0, policy_version 193106 (0.0010) [2023-12-26 16:46:19,640][105620] Updated weights for policy 1, policy_version 193899 (0.0010) [2023-12-26 16:46:19,706][105620] Updated weights for policy 1, policy_version 193909 (0.0011) [2023-12-26 16:46:19,775][105620] Updated weights for policy 1, policy_version 193919 (0.0008) [2023-12-26 16:46:19,821][105692] Updated weights for policy 0, policy_version 193116 (0.0010) [2023-12-26 16:46:19,890][105692] Updated weights for policy 0, policy_version 193126 (0.0008) [2023-12-26 16:46:19,959][105692] Updated weights for policy 0, policy_version 193136 (0.0008) [2023-12-26 16:46:20,526][105620] Updated weights for policy 1, policy_version 193929 (0.0009) [2023-12-26 16:46:20,598][105620] Updated weights for policy 1, policy_version 193939 (0.0006) [2023-12-26 16:46:20,659][105692] Updated weights for policy 0, policy_version 193146 (0.0007) [2023-12-26 16:46:20,661][105620] Updated weights for policy 1, policy_version 193949 (0.0006) [2023-12-26 16:46:20,710][105586] KL-divergence is very high: 109.9353 [2023-12-26 16:46:20,728][105692] Updated weights for policy 0, policy_version 193156 (0.0008) [2023-12-26 16:46:20,730][105620] Updated weights for policy 1, policy_version 193959 (0.0006) [2023-12-26 16:46:20,791][105692] Updated weights for policy 0, policy_version 193166 (0.0009) [2023-12-26 16:46:20,854][105692] Updated weights for policy 0, policy_version 193176 (0.0009) [2023-12-26 16:46:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 99123200. Throughput: 0: 9958.2, 1: 9768.4. Samples: 99111140. Policy #0 lag: (min: 25.0, avg: 45.3, max: 57.0) [2023-12-26 16:46:21,062][104569] Avg episode reward: [(0, '9086.673'), (1, '8007.071')] [2023-12-26 16:46:21,447][105620] Updated weights for policy 1, policy_version 193969 (0.0007) [2023-12-26 16:46:21,498][105692] Updated weights for policy 0, policy_version 193186 (0.0009) [2023-12-26 16:46:21,514][105620] Updated weights for policy 1, policy_version 193979 (0.0006) [2023-12-26 16:46:21,545][105692] Updated weights for policy 0, policy_version 193196 (0.0007) [2023-12-26 16:46:21,575][105620] Updated weights for policy 1, policy_version 193989 (0.0009) [2023-12-26 16:46:21,601][105692] Updated weights for policy 0, policy_version 193206 (0.0008) [2023-12-26 16:46:22,318][105692] Updated weights for policy 0, policy_version 193216 (0.0007) [2023-12-26 16:46:22,351][105620] Updated weights for policy 1, policy_version 193999 (0.0009) [2023-12-26 16:46:22,383][105692] Updated weights for policy 0, policy_version 193226 (0.0008) [2023-12-26 16:46:22,408][105620] Updated weights for policy 1, policy_version 194009 (0.0009) [2023-12-26 16:46:22,432][105692] Updated weights for policy 0, policy_version 193236 (0.0008) [2023-12-26 16:46:22,469][105620] Updated weights for policy 1, policy_version 194019 (0.0007) [2023-12-26 16:46:23,146][105692] Updated weights for policy 0, policy_version 193246 (0.0007) [2023-12-26 16:46:23,196][105692] Updated weights for policy 0, policy_version 193256 (0.0005) [2023-12-26 16:46:23,243][105692] Updated weights for policy 0, policy_version 193266 (0.0005) [2023-12-26 16:46:23,289][105620] Updated weights for policy 1, policy_version 194029 (0.0008) [2023-12-26 16:46:23,353][105620] Updated weights for policy 1, policy_version 194039 (0.0009) [2023-12-26 16:46:23,407][105620] Updated weights for policy 1, policy_version 194049 (0.0009) [2023-12-26 16:46:23,921][105692] Updated weights for policy 0, policy_version 193276 (0.0007) [2023-12-26 16:46:23,975][105692] Updated weights for policy 0, policy_version 193286 (0.0005) [2023-12-26 16:46:24,023][105692] Updated weights for policy 0, policy_version 193296 (0.0008) [2023-12-26 16:46:24,192][105620] Updated weights for policy 1, policy_version 194059 (0.0009) [2023-12-26 16:46:24,238][105620] Updated weights for policy 1, policy_version 194069 (0.0009) [2023-12-26 16:46:24,292][105620] Updated weights for policy 1, policy_version 194079 (0.0009) [2023-12-26 16:46:24,767][105692] Updated weights for policy 0, policy_version 193306 (0.0009) [2023-12-26 16:46:24,817][105692] Updated weights for policy 0, policy_version 193316 (0.0009) [2023-12-26 16:46:24,864][105692] Updated weights for policy 0, policy_version 193326 (0.0009) [2023-12-26 16:46:24,910][105692] Updated weights for policy 0, policy_version 193336 (0.0008) [2023-12-26 16:46:24,994][105620] Updated weights for policy 1, policy_version 194089 (0.0009) [2023-12-26 16:46:25,059][105620] Updated weights for policy 1, policy_version 194099 (0.0009) [2023-12-26 16:46:25,116][105620] Updated weights for policy 1, policy_version 194109 (0.0008) [2023-12-26 16:46:25,176][105620] Updated weights for policy 1, policy_version 194119 (0.0010) [2023-12-26 16:46:25,650][105692] Updated weights for policy 0, policy_version 193346 (0.0009) [2023-12-26 16:46:25,697][105692] Updated weights for policy 0, policy_version 193356 (0.0009) [2023-12-26 16:46:25,744][105692] Updated weights for policy 0, policy_version 193366 (0.0008) [2023-12-26 16:46:25,924][105620] Updated weights for policy 1, policy_version 194129 (0.0009) [2023-12-26 16:46:25,999][105620] Updated weights for policy 1, policy_version 194139 (0.0010) [2023-12-26 16:46:26,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 99213312. Throughput: 0: 10062.6, 1: 9732.4. Samples: 99225112. Policy #0 lag: (min: 25.0, avg: 45.3, max: 57.0) [2023-12-26 16:46:26,062][104569] Avg episode reward: [(0, '9087.379'), (1, '7825.072')] [2023-12-26 16:46:26,066][105620] Updated weights for policy 1, policy_version 194149 (0.0010) [2023-12-26 16:46:26,384][105692] Updated weights for policy 0, policy_version 193376 (0.0009) [2023-12-26 16:46:26,436][105692] Updated weights for policy 0, policy_version 193386 (0.0009) [2023-12-26 16:46:26,487][105692] Updated weights for policy 0, policy_version 193396 (0.0009) [2023-12-26 16:46:26,896][105620] Updated weights for policy 1, policy_version 194159 (0.0008) [2023-12-26 16:46:26,949][105620] Updated weights for policy 1, policy_version 194169 (0.0009) [2023-12-26 16:46:27,005][105620] Updated weights for policy 1, policy_version 194179 (0.0008) [2023-12-26 16:46:27,114][105692] Updated weights for policy 0, policy_version 193406 (0.0009) [2023-12-26 16:46:27,156][105692] Updated weights for policy 0, policy_version 193416 (0.0006) [2023-12-26 16:46:27,201][105692] Updated weights for policy 0, policy_version 193426 (0.0005) [2023-12-26 16:46:27,651][105620] Updated weights for policy 1, policy_version 194189 (0.0007) [2023-12-26 16:46:27,707][105620] Updated weights for policy 1, policy_version 194199 (0.0005) [2023-12-26 16:46:27,755][105620] Updated weights for policy 1, policy_version 194209 (0.0005) [2023-12-26 16:46:28,030][105692] Updated weights for policy 0, policy_version 193436 (0.0006) [2023-12-26 16:46:28,090][105692] Updated weights for policy 0, policy_version 193446 (0.0005) [2023-12-26 16:46:28,143][105692] Updated weights for policy 0, policy_version 193456 (0.0005) [2023-12-26 16:46:28,260][105620] Updated weights for policy 1, policy_version 194219 (0.0005) [2023-12-26 16:46:28,311][105620] Updated weights for policy 1, policy_version 194229 (0.0005) [2023-12-26 16:46:28,377][105620] Updated weights for policy 1, policy_version 194239 (0.0008) [2023-12-26 16:46:28,797][105692] Updated weights for policy 0, policy_version 193466 (0.0006) [2023-12-26 16:46:28,854][105692] Updated weights for policy 0, policy_version 193476 (0.0009) [2023-12-26 16:46:28,902][105692] Updated weights for policy 0, policy_version 193486 (0.0009) [2023-12-26 16:46:28,954][105692] Updated weights for policy 0, policy_version 193496 (0.0008) [2023-12-26 16:46:29,097][105620] Updated weights for policy 1, policy_version 194249 (0.0009) [2023-12-26 16:46:29,151][105620] Updated weights for policy 1, policy_version 194259 (0.0009) [2023-12-26 16:46:29,208][105620] Updated weights for policy 1, policy_version 194269 (0.0009) [2023-12-26 16:46:29,283][105620] Updated weights for policy 1, policy_version 194279 (0.0009) [2023-12-26 16:46:29,734][105692] Updated weights for policy 0, policy_version 193506 (0.0009) [2023-12-26 16:46:29,785][105692] Updated weights for policy 0, policy_version 193516 (0.0009) [2023-12-26 16:46:29,846][105692] Updated weights for policy 0, policy_version 193526 (0.0008) [2023-12-26 16:46:30,052][105620] Updated weights for policy 1, policy_version 194289 (0.0009) [2023-12-26 16:46:30,097][105586] KL-divergence is very high: 181.6287 [2023-12-26 16:46:30,110][105620] Updated weights for policy 1, policy_version 194299 (0.0009) [2023-12-26 16:46:30,139][105586] KL-divergence is very high: 206.5632 [2023-12-26 16:46:30,163][105620] Updated weights for policy 1, policy_version 194309 (0.0010) [2023-12-26 16:46:30,460][105692] Updated weights for policy 0, policy_version 193536 (0.0006) [2023-12-26 16:46:30,517][105692] Updated weights for policy 0, policy_version 193546 (0.0006) [2023-12-26 16:46:30,572][105692] Updated weights for policy 0, policy_version 193556 (0.0007) [2023-12-26 16:46:31,031][105620] Updated weights for policy 1, policy_version 194320 (0.0009) [2023-12-26 16:46:31,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 99311616. Throughput: 0: 10143.8, 1: 9803.0. Samples: 99286384. Policy #0 lag: (min: 25.0, avg: 45.3, max: 57.0) [2023-12-26 16:46:31,063][104569] Avg episode reward: [(0, '8994.771'), (1, '7555.292')] [2023-12-26 16:46:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000193560_49561600.pth... [2023-12-26 16:46:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000192408_49266688.pth [2023-12-26 16:46:31,091][105620] Updated weights for policy 1, policy_version 194330 (0.0009) [2023-12-26 16:46:31,154][105620] Updated weights for policy 1, policy_version 194340 (0.0008) [2023-12-26 16:46:31,179][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000194344_49758208.pth... [2023-12-26 16:46:31,183][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000193192_49463296.pth [2023-12-26 16:46:31,220][105692] Updated weights for policy 0, policy_version 193566 (0.0006) [2023-12-26 16:46:31,274][105692] Updated weights for policy 0, policy_version 193576 (0.0007) [2023-12-26 16:46:31,335][105692] Updated weights for policy 0, policy_version 193586 (0.0009) [2023-12-26 16:46:31,953][105620] Updated weights for policy 1, policy_version 194350 (0.0009) [2023-12-26 16:46:32,014][105620] Updated weights for policy 1, policy_version 194360 (0.0009) [2023-12-26 16:46:32,056][105692] Updated weights for policy 0, policy_version 193596 (0.0008) [2023-12-26 16:46:32,065][105620] Updated weights for policy 1, policy_version 194370 (0.0009) [2023-12-26 16:46:32,117][105692] Updated weights for policy 0, policy_version 193606 (0.0008) [2023-12-26 16:46:32,172][105692] Updated weights for policy 0, policy_version 193616 (0.0009) [2023-12-26 16:46:32,812][105692] Updated weights for policy 0, policy_version 193626 (0.0009) [2023-12-26 16:46:32,858][105692] Updated weights for policy 0, policy_version 193636 (0.0008) [2023-12-26 16:46:32,882][105620] Updated weights for policy 1, policy_version 194380 (0.0006) [2023-12-26 16:46:32,914][105692] Updated weights for policy 0, policy_version 193646 (0.0008) [2023-12-26 16:46:32,940][105620] Updated weights for policy 1, policy_version 194390 (0.0007) [2023-12-26 16:46:32,966][105692] Updated weights for policy 0, policy_version 193656 (0.0006) [2023-12-26 16:46:32,994][105620] Updated weights for policy 1, policy_version 194400 (0.0008) [2023-12-26 16:46:33,729][105692] Updated weights for policy 0, policy_version 193666 (0.0009) [2023-12-26 16:46:33,743][105620] Updated weights for policy 1, policy_version 194410 (0.0008) [2023-12-26 16:46:33,774][105692] Updated weights for policy 0, policy_version 193676 (0.0006) [2023-12-26 16:46:33,799][105620] Updated weights for policy 1, policy_version 194420 (0.0009) [2023-12-26 16:46:33,821][105692] Updated weights for policy 0, policy_version 193686 (0.0007) [2023-12-26 16:46:33,855][105620] Updated weights for policy 1, policy_version 194430 (0.0008) [2023-12-26 16:46:33,912][105620] Updated weights for policy 1, policy_version 194440 (0.0009) [2023-12-26 16:46:34,609][105692] Updated weights for policy 0, policy_version 193696 (0.0008) [2023-12-26 16:46:34,649][105620] Updated weights for policy 1, policy_version 194450 (0.0005) [2023-12-26 16:46:34,672][105692] Updated weights for policy 0, policy_version 193706 (0.0008) [2023-12-26 16:46:34,704][105620] Updated weights for policy 1, policy_version 194460 (0.0006) [2023-12-26 16:46:34,732][105692] Updated weights for policy 0, policy_version 193716 (0.0007) [2023-12-26 16:46:34,767][105620] Updated weights for policy 1, policy_version 194470 (0.0008) [2023-12-26 16:46:35,488][105692] Updated weights for policy 0, policy_version 193726 (0.0006) [2023-12-26 16:46:35,502][105620] Updated weights for policy 1, policy_version 194480 (0.0008) [2023-12-26 16:46:35,533][105692] Updated weights for policy 0, policy_version 193736 (0.0006) [2023-12-26 16:46:35,552][105620] Updated weights for policy 1, policy_version 194490 (0.0006) [2023-12-26 16:46:35,579][105692] Updated weights for policy 0, policy_version 193746 (0.0006) [2023-12-26 16:46:35,597][105620] Updated weights for policy 1, policy_version 194500 (0.0006) [2023-12-26 16:46:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 99409920. Throughput: 0: 9994.6, 1: 9716.6. Samples: 99400044. Policy #0 lag: (min: 25.0, avg: 45.3, max: 57.0) [2023-12-26 16:46:36,062][104569] Avg episode reward: [(0, '8994.136'), (1, '8277.461')] [2023-12-26 16:46:36,335][105620] Updated weights for policy 1, policy_version 194510 (0.0008) [2023-12-26 16:46:36,383][105692] Updated weights for policy 0, policy_version 193756 (0.0007) [2023-12-26 16:46:36,396][105620] Updated weights for policy 1, policy_version 194520 (0.0007) [2023-12-26 16:46:36,447][105692] Updated weights for policy 0, policy_version 193766 (0.0009) [2023-12-26 16:46:36,458][105620] Updated weights for policy 1, policy_version 194530 (0.0006) [2023-12-26 16:46:36,508][105692] Updated weights for policy 0, policy_version 193776 (0.0007) [2023-12-26 16:46:37,156][105692] Updated weights for policy 0, policy_version 193786 (0.0006) [2023-12-26 16:46:37,210][105692] Updated weights for policy 0, policy_version 193796 (0.0009) [2023-12-26 16:46:37,247][105620] Updated weights for policy 1, policy_version 194540 (0.0007) [2023-12-26 16:46:37,260][105692] Updated weights for policy 0, policy_version 193806 (0.0009) [2023-12-26 16:46:37,303][105620] Updated weights for policy 1, policy_version 194550 (0.0008) [2023-12-26 16:46:37,309][105692] Updated weights for policy 0, policy_version 193816 (0.0005) [2023-12-26 16:46:37,355][105620] Updated weights for policy 1, policy_version 194560 (0.0011) [2023-12-26 16:46:37,959][105620] Updated weights for policy 1, policy_version 194570 (0.0009) [2023-12-26 16:46:38,010][105620] Updated weights for policy 1, policy_version 194580 (0.0011) [2023-12-26 16:46:38,070][105620] Updated weights for policy 1, policy_version 194590 (0.0009) [2023-12-26 16:46:38,100][105692] Updated weights for policy 0, policy_version 193826 (0.0008) [2023-12-26 16:46:38,124][105620] Updated weights for policy 1, policy_version 194600 (0.0005) [2023-12-26 16:46:38,158][105692] Updated weights for policy 0, policy_version 193836 (0.0009) [2023-12-26 16:46:38,218][105692] Updated weights for policy 0, policy_version 193846 (0.0010) [2023-12-26 16:46:38,841][105620] Updated weights for policy 1, policy_version 194610 (0.0011) [2023-12-26 16:46:38,900][105620] Updated weights for policy 1, policy_version 194620 (0.0011) [2023-12-26 16:46:38,959][105620] Updated weights for policy 1, policy_version 194630 (0.0010) [2023-12-26 16:46:38,979][105692] Updated weights for policy 0, policy_version 193856 (0.0009) [2023-12-26 16:46:39,027][105692] Updated weights for policy 0, policy_version 193866 (0.0008) [2023-12-26 16:46:39,074][105692] Updated weights for policy 0, policy_version 193876 (0.0008) [2023-12-26 16:46:39,717][105620] Updated weights for policy 1, policy_version 194640 (0.0011) [2023-12-26 16:46:39,770][105620] Updated weights for policy 1, policy_version 194650 (0.0011) [2023-12-26 16:46:39,836][105620] Updated weights for policy 1, policy_version 194660 (0.0011) [2023-12-26 16:46:39,884][105692] Updated weights for policy 0, policy_version 193886 (0.0008) [2023-12-26 16:46:39,938][105692] Updated weights for policy 0, policy_version 193896 (0.0008) [2023-12-26 16:46:39,995][105692] Updated weights for policy 0, policy_version 193906 (0.0009) [2023-12-26 16:46:40,599][105620] Updated weights for policy 1, policy_version 194670 (0.0009) [2023-12-26 16:46:40,646][105620] Updated weights for policy 1, policy_version 194680 (0.0009) [2023-12-26 16:46:40,694][105620] Updated weights for policy 1, policy_version 194690 (0.0009) [2023-12-26 16:46:40,774][105692] Updated weights for policy 0, policy_version 193916 (0.0009) [2023-12-26 16:46:40,827][105692] Updated weights for policy 0, policy_version 193927 (0.0010) [2023-12-26 16:46:40,877][105692] Updated weights for policy 0, policy_version 193937 (0.0007) [2023-12-26 16:46:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 99508224. Throughput: 0: 9897.0, 1: 9646.8. Samples: 99513308. Policy #0 lag: (min: 25.0, avg: 45.3, max: 57.0) [2023-12-26 16:46:41,063][104569] Avg episode reward: [(0, '9351.816'), (1, '8369.251')] [2023-12-26 16:46:41,501][105620] Updated weights for policy 1, policy_version 194700 (0.0009) [2023-12-26 16:46:41,559][105620] Updated weights for policy 1, policy_version 194710 (0.0008) [2023-12-26 16:46:41,615][105620] Updated weights for policy 1, policy_version 194720 (0.0009) [2023-12-26 16:46:41,634][105692] Updated weights for policy 0, policy_version 193947 (0.0007) [2023-12-26 16:46:41,689][105692] Updated weights for policy 0, policy_version 193957 (0.0008) [2023-12-26 16:46:41,755][105692] Updated weights for policy 0, policy_version 193967 (0.0008) [2023-12-26 16:46:42,313][105620] Updated weights for policy 1, policy_version 194730 (0.0008) [2023-12-26 16:46:42,378][105620] Updated weights for policy 1, policy_version 194740 (0.0009) [2023-12-26 16:46:42,442][105620] Updated weights for policy 1, policy_version 194750 (0.0008) [2023-12-26 16:46:42,507][105620] Updated weights for policy 1, policy_version 194760 (0.0008) [2023-12-26 16:46:42,545][105692] Updated weights for policy 0, policy_version 193977 (0.0008) [2023-12-26 16:46:42,608][105692] Updated weights for policy 0, policy_version 193987 (0.0009) [2023-12-26 16:46:42,665][105692] Updated weights for policy 0, policy_version 193997 (0.0009) [2023-12-26 16:46:42,721][105692] Updated weights for policy 0, policy_version 194007 (0.0008) [2023-12-26 16:46:43,193][105620] Updated weights for policy 1, policy_version 194770 (0.0009) [2023-12-26 16:46:43,253][105620] Updated weights for policy 1, policy_version 194780 (0.0007) [2023-12-26 16:46:43,313][105620] Updated weights for policy 1, policy_version 194790 (0.0006) [2023-12-26 16:46:43,519][105692] Updated weights for policy 0, policy_version 194017 (0.0009) [2023-12-26 16:46:43,573][105692] Updated weights for policy 0, policy_version 194029 (0.0010) [2023-12-26 16:46:43,627][105692] Updated weights for policy 0, policy_version 194040 (0.0010) [2023-12-26 16:46:43,877][105620] Updated weights for policy 1, policy_version 194800 (0.0006) [2023-12-26 16:46:43,930][105620] Updated weights for policy 1, policy_version 194810 (0.0008) [2023-12-26 16:46:43,985][105620] Updated weights for policy 1, policy_version 194820 (0.0010) [2023-12-26 16:46:44,466][105692] Updated weights for policy 0, policy_version 194050 (0.0009) [2023-12-26 16:46:44,523][105692] Updated weights for policy 0, policy_version 194060 (0.0009) [2023-12-26 16:46:44,578][105692] Updated weights for policy 0, policy_version 194070 (0.0009) [2023-12-26 16:46:44,687][105620] Updated weights for policy 1, policy_version 194830 (0.0010) [2023-12-26 16:46:44,741][105620] Updated weights for policy 1, policy_version 194840 (0.0009) [2023-12-26 16:46:44,804][105620] Updated weights for policy 1, policy_version 194850 (0.0009) [2023-12-26 16:46:45,375][105692] Updated weights for policy 0, policy_version 194080 (0.0009) [2023-12-26 16:46:45,439][105692] Updated weights for policy 0, policy_version 194090 (0.0009) [2023-12-26 16:46:45,506][105692] Updated weights for policy 0, policy_version 194100 (0.0009) [2023-12-26 16:46:45,550][105620] Updated weights for policy 1, policy_version 194860 (0.0009) [2023-12-26 16:46:45,609][105620] Updated weights for policy 1, policy_version 194870 (0.0008) [2023-12-26 16:46:45,666][105620] Updated weights for policy 1, policy_version 194880 (0.0010) [2023-12-26 16:46:46,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 99598336. Throughput: 0: 9833.1, 1: 9653.3. Samples: 99570788. Policy #0 lag: (min: 25.0, avg: 45.3, max: 57.0) [2023-12-26 16:46:46,063][104569] Avg episode reward: [(0, '2991.264'), (1, '8193.392')] [2023-12-26 16:46:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000194104_49700864.pth... [2023-12-26 16:46:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000194888_49897472.pth... [2023-12-26 16:46:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000193768_49610752.pth [2023-12-26 16:46:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000192984_49414144.pth [2023-12-26 16:46:46,274][105692] Updated weights for policy 0, policy_version 194110 (0.0008) [2023-12-26 16:46:46,330][105692] Updated weights for policy 0, policy_version 194120 (0.0008) [2023-12-26 16:46:46,369][105620] Updated weights for policy 1, policy_version 194890 (0.0009) [2023-12-26 16:46:46,388][105692] Updated weights for policy 0, policy_version 194130 (0.0008) [2023-12-26 16:46:46,422][105620] Updated weights for policy 1, policy_version 194900 (0.0006) [2023-12-26 16:46:46,475][105620] Updated weights for policy 1, policy_version 194910 (0.0005) [2023-12-26 16:46:46,524][105620] Updated weights for policy 1, policy_version 194920 (0.0005) [2023-12-26 16:46:47,093][105692] Updated weights for policy 0, policy_version 194140 (0.0008) [2023-12-26 16:46:47,154][105692] Updated weights for policy 0, policy_version 194150 (0.0006) [2023-12-26 16:46:47,180][105620] Updated weights for policy 1, policy_version 194930 (0.0005) [2023-12-26 16:46:47,211][105692] Updated weights for policy 0, policy_version 194160 (0.0006) [2023-12-26 16:46:47,243][105620] Updated weights for policy 1, policy_version 194940 (0.0006) [2023-12-26 16:46:47,298][105620] Updated weights for policy 1, policy_version 194950 (0.0008) [2023-12-26 16:46:47,884][105620] Updated weights for policy 1, policy_version 194960 (0.0010) [2023-12-26 16:46:47,931][105620] Updated weights for policy 1, policy_version 194970 (0.0010) [2023-12-26 16:46:47,958][105692] Updated weights for policy 0, policy_version 194170 (0.0007) [2023-12-26 16:46:47,979][105620] Updated weights for policy 1, policy_version 194980 (0.0010) [2023-12-26 16:46:48,015][105692] Updated weights for policy 0, policy_version 194180 (0.0005) [2023-12-26 16:46:48,082][105692] Updated weights for policy 0, policy_version 194190 (0.0005) [2023-12-26 16:46:48,144][105692] Updated weights for policy 0, policy_version 194200 (0.0005) [2023-12-26 16:46:48,689][105692] Updated weights for policy 0, policy_version 194210 (0.0008) [2023-12-26 16:46:48,731][105620] Updated weights for policy 1, policy_version 194990 (0.0008) [2023-12-26 16:46:48,746][105692] Updated weights for policy 0, policy_version 194220 (0.0008) [2023-12-26 16:46:48,793][105620] Updated weights for policy 1, policy_version 195000 (0.0008) [2023-12-26 16:46:48,811][105692] Updated weights for policy 0, policy_version 194230 (0.0006) [2023-12-26 16:46:48,847][105620] Updated weights for policy 1, policy_version 195010 (0.0008) [2023-12-26 16:46:49,496][105692] Updated weights for policy 0, policy_version 194240 (0.0010) [2023-12-26 16:46:49,553][105692] Updated weights for policy 0, policy_version 194250 (0.0010) [2023-12-26 16:46:49,612][105620] Updated weights for policy 1, policy_version 195020 (0.0007) [2023-12-26 16:46:49,617][105692] Updated weights for policy 0, policy_version 194260 (0.0011) [2023-12-26 16:46:49,675][105620] Updated weights for policy 1, policy_version 195030 (0.0008) [2023-12-26 16:46:49,734][105620] Updated weights for policy 1, policy_version 195040 (0.0010) [2023-12-26 16:46:50,377][105692] Updated weights for policy 0, policy_version 194270 (0.0011) [2023-12-26 16:46:50,437][105692] Updated weights for policy 0, policy_version 194280 (0.0011) [2023-12-26 16:46:50,463][105620] Updated weights for policy 1, policy_version 195050 (0.0010) [2023-12-26 16:46:50,486][105692] Updated weights for policy 0, policy_version 194290 (0.0011) [2023-12-26 16:46:50,516][105620] Updated weights for policy 1, policy_version 195060 (0.0005) [2023-12-26 16:46:50,572][105620] Updated weights for policy 1, policy_version 195070 (0.0009) [2023-12-26 16:46:50,626][105620] Updated weights for policy 1, policy_version 195080 (0.0010) [2023-12-26 16:46:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 99696640. Throughput: 0: 9783.5, 1: 9592.3. Samples: 99688004. Policy #0 lag: (min: 3.0, avg: 7.3, max: 35.0) [2023-12-26 16:46:51,062][104569] Avg episode reward: [(0, '1597.589'), (1, '8464.166')] [2023-12-26 16:46:51,238][105692] Updated weights for policy 0, policy_version 194300 (0.0009) [2023-12-26 16:46:51,300][105692] Updated weights for policy 0, policy_version 194310 (0.0007) [2023-12-26 16:46:51,360][105692] Updated weights for policy 0, policy_version 194320 (0.0010) [2023-12-26 16:46:51,393][105620] Updated weights for policy 1, policy_version 195090 (0.0008) [2023-12-26 16:46:51,459][105620] Updated weights for policy 1, policy_version 195100 (0.0007) [2023-12-26 16:46:51,524][105620] Updated weights for policy 1, policy_version 195110 (0.0006) [2023-12-26 16:46:52,065][105692] Updated weights for policy 0, policy_version 194330 (0.0012) [2023-12-26 16:46:52,124][105692] Updated weights for policy 0, policy_version 194340 (0.0008) [2023-12-26 16:46:52,178][105692] Updated weights for policy 0, policy_version 194350 (0.0005) [2023-12-26 16:46:52,237][105692] Updated weights for policy 0, policy_version 194360 (0.0010) [2023-12-26 16:46:52,302][105620] Updated weights for policy 1, policy_version 195120 (0.0009) [2023-12-26 16:46:52,370][105620] Updated weights for policy 1, policy_version 195130 (0.0008) [2023-12-26 16:46:52,436][105620] Updated weights for policy 1, policy_version 195140 (0.0007) [2023-12-26 16:46:52,973][105692] Updated weights for policy 0, policy_version 194370 (0.0009) [2023-12-26 16:46:53,024][105692] Updated weights for policy 0, policy_version 194380 (0.0010) [2023-12-26 16:46:53,082][105692] Updated weights for policy 0, policy_version 194390 (0.0010) [2023-12-26 16:46:53,145][105620] Updated weights for policy 1, policy_version 195150 (0.0008) [2023-12-26 16:46:53,203][105620] Updated weights for policy 1, policy_version 195160 (0.0008) [2023-12-26 16:46:53,272][105620] Updated weights for policy 1, policy_version 195170 (0.0008) [2023-12-26 16:46:53,722][105692] Updated weights for policy 0, policy_version 194400 (0.0011) [2023-12-26 16:46:53,777][105692] Updated weights for policy 0, policy_version 194410 (0.0010) [2023-12-26 16:46:53,839][105692] Updated weights for policy 0, policy_version 194420 (0.0010) [2023-12-26 16:46:54,031][105620] Updated weights for policy 1, policy_version 195180 (0.0009) [2023-12-26 16:46:54,083][105620] Updated weights for policy 1, policy_version 195190 (0.0008) [2023-12-26 16:46:54,142][105620] Updated weights for policy 1, policy_version 195200 (0.0008) [2023-12-26 16:46:54,509][105692] Updated weights for policy 0, policy_version 194430 (0.0010) [2023-12-26 16:46:54,558][105692] Updated weights for policy 0, policy_version 194440 (0.0010) [2023-12-26 16:46:54,610][105692] Updated weights for policy 0, policy_version 194450 (0.0010) [2023-12-26 16:46:54,817][105620] Updated weights for policy 1, policy_version 195210 (0.0007) [2023-12-26 16:46:54,863][105620] Updated weights for policy 1, policy_version 195220 (0.0005) [2023-12-26 16:46:54,913][105620] Updated weights for policy 1, policy_version 195230 (0.0005) [2023-12-26 16:46:54,972][105620] Updated weights for policy 1, policy_version 195240 (0.0005) [2023-12-26 16:46:55,367][105692] Updated weights for policy 0, policy_version 194460 (0.0010) [2023-12-26 16:46:55,423][105692] Updated weights for policy 0, policy_version 194470 (0.0008) [2023-12-26 16:46:55,495][105692] Updated weights for policy 0, policy_version 194480 (0.0005) [2023-12-26 16:46:55,634][105620] Updated weights for policy 1, policy_version 195250 (0.0010) [2023-12-26 16:46:55,691][105620] Updated weights for policy 1, policy_version 195260 (0.0009) [2023-12-26 16:46:55,743][105620] Updated weights for policy 1, policy_version 195270 (0.0009) [2023-12-26 16:46:56,002][105692] Updated weights for policy 0, policy_version 194490 (0.0006) [2023-12-26 16:46:56,055][105692] Updated weights for policy 0, policy_version 194500 (0.0011) [2023-12-26 16:46:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 99794944. Throughput: 0: 9756.6, 1: 9601.8. Samples: 99803636. Policy #0 lag: (min: 3.0, avg: 7.3, max: 35.0) [2023-12-26 16:46:56,063][104569] Avg episode reward: [(0, '6286.636'), (1, '8459.730')] [2023-12-26 16:46:56,118][105692] Updated weights for policy 0, policy_version 194510 (0.0011) [2023-12-26 16:46:56,167][105692] Updated weights for policy 0, policy_version 194520 (0.0011) [2023-12-26 16:46:56,474][105620] Updated weights for policy 1, policy_version 195280 (0.0010) [2023-12-26 16:46:56,526][105620] Updated weights for policy 1, policy_version 195290 (0.0010) [2023-12-26 16:46:56,574][105620] Updated weights for policy 1, policy_version 195300 (0.0010) [2023-12-26 16:46:56,921][105692] Updated weights for policy 0, policy_version 194530 (0.0011) [2023-12-26 16:46:56,970][105692] Updated weights for policy 0, policy_version 194540 (0.0011) [2023-12-26 16:46:57,014][105692] Updated weights for policy 0, policy_version 194550 (0.0011) [2023-12-26 16:46:57,319][105620] Updated weights for policy 1, policy_version 195310 (0.0008) [2023-12-26 16:46:57,374][105620] Updated weights for policy 1, policy_version 195320 (0.0006) [2023-12-26 16:46:57,428][105620] Updated weights for policy 1, policy_version 195331 (0.0010) [2023-12-26 16:46:57,670][105692] Updated weights for policy 0, policy_version 194561 (0.0011) [2023-12-26 16:46:57,721][105692] Updated weights for policy 0, policy_version 194571 (0.0009) [2023-12-26 16:46:57,774][105692] Updated weights for policy 0, policy_version 194581 (0.0010) [2023-12-26 16:46:58,109][105620] Updated weights for policy 1, policy_version 195342 (0.0010) [2023-12-26 16:46:58,171][105620] Updated weights for policy 1, policy_version 195352 (0.0011) [2023-12-26 16:46:58,227][105620] Updated weights for policy 1, policy_version 195362 (0.0011) [2023-12-26 16:46:58,587][105692] Updated weights for policy 0, policy_version 194591 (0.0009) [2023-12-26 16:46:58,648][105692] Updated weights for policy 0, policy_version 194601 (0.0008) [2023-12-26 16:46:58,715][105692] Updated weights for policy 0, policy_version 194611 (0.0008) [2023-12-26 16:46:59,057][105620] Updated weights for policy 1, policy_version 195372 (0.0009) [2023-12-26 16:46:59,116][105620] Updated weights for policy 1, policy_version 195382 (0.0007) [2023-12-26 16:46:59,186][105620] Updated weights for policy 1, policy_version 195392 (0.0010) [2023-12-26 16:46:59,450][105692] Updated weights for policy 0, policy_version 194621 (0.0007) [2023-12-26 16:46:59,520][105692] Updated weights for policy 0, policy_version 194631 (0.0010) [2023-12-26 16:46:59,586][105692] Updated weights for policy 0, policy_version 194641 (0.0006) [2023-12-26 16:46:59,824][105620] Updated weights for policy 1, policy_version 195402 (0.0009) [2023-12-26 16:46:59,885][105620] Updated weights for policy 1, policy_version 195412 (0.0008) [2023-12-26 16:46:59,953][105620] Updated weights for policy 1, policy_version 195422 (0.0009) [2023-12-26 16:47:00,018][105620] Updated weights for policy 1, policy_version 195432 (0.0008) [2023-12-26 16:47:00,320][105692] Updated weights for policy 0, policy_version 194651 (0.0008) [2023-12-26 16:47:00,377][105692] Updated weights for policy 0, policy_version 194661 (0.0009) [2023-12-26 16:47:00,432][105692] Updated weights for policy 0, policy_version 194672 (0.0009) [2023-12-26 16:47:00,693][105620] Updated weights for policy 1, policy_version 195442 (0.0006) [2023-12-26 16:47:00,761][105620] Updated weights for policy 1, policy_version 195452 (0.0005) [2023-12-26 16:47:00,821][105620] Updated weights for policy 1, policy_version 195462 (0.0007) [2023-12-26 16:47:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 99893248. Throughput: 0: 9793.6, 1: 9611.9. Samples: 99863728. Policy #0 lag: (min: 3.0, avg: 7.3, max: 35.0) [2023-12-26 16:47:01,062][104569] Avg episode reward: [(0, '8580.267'), (1, '8456.452')] [2023-12-26 16:47:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000194680_49848320.pth... [2023-12-26 16:47:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000195464_50044928.pth... [2023-12-26 16:47:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000194344_49758208.pth [2023-12-26 16:47:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000193560_49561600.pth [2023-12-26 16:47:01,214][105692] Updated weights for policy 0, policy_version 194682 (0.0006) [2023-12-26 16:47:01,279][105692] Updated weights for policy 0, policy_version 194692 (0.0009) [2023-12-26 16:47:01,338][105692] Updated weights for policy 0, policy_version 194702 (0.0009) [2023-12-26 16:47:01,400][105692] Updated weights for policy 0, policy_version 194712 (0.0008) [2023-12-26 16:47:01,459][105620] Updated weights for policy 1, policy_version 195472 (0.0008) [2023-12-26 16:47:01,529][105620] Updated weights for policy 1, policy_version 195482 (0.0005) [2023-12-26 16:47:01,588][105620] Updated weights for policy 1, policy_version 195492 (0.0010) [2023-12-26 16:47:02,098][105692] Updated weights for policy 0, policy_version 194722 (0.0009) [2023-12-26 16:47:02,161][105692] Updated weights for policy 0, policy_version 194732 (0.0009) [2023-12-26 16:47:02,216][105692] Updated weights for policy 0, policy_version 194742 (0.0009) [2023-12-26 16:47:02,363][105620] Updated weights for policy 1, policy_version 195502 (0.0009) [2023-12-26 16:47:02,423][105620] Updated weights for policy 1, policy_version 195512 (0.0008) [2023-12-26 16:47:02,482][105620] Updated weights for policy 1, policy_version 195522 (0.0007) [2023-12-26 16:47:03,001][105692] Updated weights for policy 0, policy_version 194752 (0.0006) [2023-12-26 16:47:03,044][105692] Updated weights for policy 0, policy_version 194762 (0.0005) [2023-12-26 16:47:03,087][105692] Updated weights for policy 0, policy_version 194772 (0.0005) [2023-12-26 16:47:03,105][105620] Updated weights for policy 1, policy_version 195532 (0.0009) [2023-12-26 16:47:03,167][105620] Updated weights for policy 1, policy_version 195542 (0.0010) [2023-12-26 16:47:03,229][105620] Updated weights for policy 1, policy_version 195552 (0.0010) [2023-12-26 16:47:03,630][105692] Updated weights for policy 0, policy_version 194782 (0.0005) [2023-12-26 16:47:03,691][105692] Updated weights for policy 0, policy_version 194792 (0.0007) [2023-12-26 16:47:03,740][105692] Updated weights for policy 0, policy_version 194802 (0.0008) [2023-12-26 16:47:03,842][105620] Updated weights for policy 1, policy_version 195562 (0.0007) [2023-12-26 16:47:03,897][105620] Updated weights for policy 1, policy_version 195572 (0.0009) [2023-12-26 16:47:03,955][105620] Updated weights for policy 1, policy_version 195582 (0.0007) [2023-12-26 16:47:04,020][105620] Updated weights for policy 1, policy_version 195592 (0.0006) [2023-12-26 16:47:04,480][105692] Updated weights for policy 0, policy_version 194812 (0.0009) [2023-12-26 16:47:04,529][105692] Updated weights for policy 0, policy_version 194822 (0.0010) [2023-12-26 16:47:04,581][105692] Updated weights for policy 0, policy_version 194832 (0.0010) [2023-12-26 16:47:04,625][105620] Updated weights for policy 1, policy_version 195602 (0.0006) [2023-12-26 16:47:04,692][105620] Updated weights for policy 1, policy_version 195612 (0.0010) [2023-12-26 16:47:04,754][105620] Updated weights for policy 1, policy_version 195622 (0.0010) [2023-12-26 16:47:05,272][105692] Updated weights for policy 0, policy_version 194842 (0.0010) [2023-12-26 16:47:05,326][105692] Updated weights for policy 0, policy_version 194852 (0.0010) [2023-12-26 16:47:05,380][105692] Updated weights for policy 0, policy_version 194862 (0.0010) [2023-12-26 16:47:05,433][105692] Updated weights for policy 0, policy_version 194872 (0.0008) [2023-12-26 16:47:05,456][105620] Updated weights for policy 1, policy_version 195632 (0.0008) [2023-12-26 16:47:05,511][105620] Updated weights for policy 1, policy_version 195642 (0.0008) [2023-12-26 16:47:05,567][105620] Updated weights for policy 1, policy_version 195652 (0.0006) [2023-12-26 16:47:06,062][104569] Fps is (10 sec: 19660.0, 60 sec: 19387.6, 300 sec: 19577.5). Total num frames: 99991552. Throughput: 0: 9720.5, 1: 9660.9. Samples: 99983312. Policy #0 lag: (min: 3.0, avg: 7.3, max: 35.0) [2023-12-26 16:47:06,064][104569] Avg episode reward: [(0, '8746.239'), (1, '8546.215')] [2023-12-26 16:47:06,113][105620] Updated weights for policy 1, policy_version 195662 (0.0009) [2023-12-26 16:47:06,126][105692] Updated weights for policy 0, policy_version 194882 (0.0011) [2023-12-26 16:47:06,178][105620] Updated weights for policy 1, policy_version 195672 (0.0011) [2023-12-26 16:47:06,178][105692] Updated weights for policy 0, policy_version 194892 (0.0008) [2023-12-26 16:47:06,238][105620] Updated weights for policy 1, policy_version 195682 (0.0011) [2023-12-26 16:47:06,240][105692] Updated weights for policy 0, policy_version 194902 (0.0006) [2023-12-26 16:47:06,973][105692] Updated weights for policy 0, policy_version 194912 (0.0008) [2023-12-26 16:47:06,994][105620] Updated weights for policy 1, policy_version 195692 (0.0010) [2023-12-26 16:47:07,037][105692] Updated weights for policy 0, policy_version 194922 (0.0006) [2023-12-26 16:47:07,058][105620] Updated weights for policy 1, policy_version 195702 (0.0011) [2023-12-26 16:47:07,101][105692] Updated weights for policy 0, policy_version 194932 (0.0006) [2023-12-26 16:47:07,121][105620] Updated weights for policy 1, policy_version 195712 (0.0010) [2023-12-26 16:47:07,781][105620] Updated weights for policy 1, policy_version 195722 (0.0007) [2023-12-26 16:47:07,832][105620] Updated weights for policy 1, policy_version 195732 (0.0009) [2023-12-26 16:47:07,845][105692] Updated weights for policy 0, policy_version 194942 (0.0006) [2023-12-26 16:47:07,879][105620] Updated weights for policy 1, policy_version 195742 (0.0007) [2023-12-26 16:47:07,910][105692] Updated weights for policy 0, policy_version 194952 (0.0008) [2023-12-26 16:47:07,929][105620] Updated weights for policy 1, policy_version 195752 (0.0007) [2023-12-26 16:47:07,971][105692] Updated weights for policy 0, policy_version 194962 (0.0008) [2023-12-26 16:47:08,663][105692] Updated weights for policy 0, policy_version 194972 (0.0008) [2023-12-26 16:47:08,722][105692] Updated weights for policy 0, policy_version 194982 (0.0007) [2023-12-26 16:47:08,744][105620] Updated weights for policy 1, policy_version 195762 (0.0008) [2023-12-26 16:47:08,786][105692] Updated weights for policy 0, policy_version 194992 (0.0008) [2023-12-26 16:47:08,807][105620] Updated weights for policy 1, policy_version 195772 (0.0007) [2023-12-26 16:47:08,880][105620] Updated weights for policy 1, policy_version 195782 (0.0010) [2023-12-26 16:47:09,498][105620] Updated weights for policy 1, policy_version 195792 (0.0009) [2023-12-26 16:47:09,567][105620] Updated weights for policy 1, policy_version 195802 (0.0008) [2023-12-26 16:47:09,582][105692] Updated weights for policy 0, policy_version 195002 (0.0006) [2023-12-26 16:47:09,627][105620] Updated weights for policy 1, policy_version 195812 (0.0009) [2023-12-26 16:47:09,628][105692] Updated weights for policy 0, policy_version 195012 (0.0009) [2023-12-26 16:47:09,680][105692] Updated weights for policy 0, policy_version 195022 (0.0009) [2023-12-26 16:47:09,743][105692] Updated weights for policy 0, policy_version 195032 (0.0009) [2023-12-26 16:47:10,391][105620] Updated weights for policy 1, policy_version 195822 (0.0008) [2023-12-26 16:47:10,455][105620] Updated weights for policy 1, policy_version 195832 (0.0010) [2023-12-26 16:47:10,508][105620] Updated weights for policy 1, policy_version 195842 (0.0008) [2023-12-26 16:47:10,522][105692] Updated weights for policy 0, policy_version 195042 (0.0006) [2023-12-26 16:47:10,584][105692] Updated weights for policy 0, policy_version 195052 (0.0008) [2023-12-26 16:47:10,640][105692] Updated weights for policy 0, policy_version 195062 (0.0009) [2023-12-26 16:47:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 100089856. Throughput: 0: 9684.0, 1: 9751.5. Samples: 100099712. Policy #0 lag: (min: 3.0, avg: 7.3, max: 35.0) [2023-12-26 16:47:11,063][104569] Avg episode reward: [(0, '9096.661'), (1, '8548.146')] [2023-12-26 16:47:11,311][105620] Updated weights for policy 1, policy_version 195852 (0.0006) [2023-12-26 16:47:11,373][105620] Updated weights for policy 1, policy_version 195862 (0.0007) [2023-12-26 16:47:11,432][105620] Updated weights for policy 1, policy_version 195872 (0.0006) [2023-12-26 16:47:11,447][105692] Updated weights for policy 0, policy_version 195072 (0.0008) [2023-12-26 16:47:11,507][105692] Updated weights for policy 0, policy_version 195082 (0.0008) [2023-12-26 16:47:11,567][105692] Updated weights for policy 0, policy_version 195092 (0.0009) [2023-12-26 16:47:12,180][105620] Updated weights for policy 1, policy_version 195882 (0.0008) [2023-12-26 16:47:12,248][105620] Updated weights for policy 1, policy_version 195892 (0.0009) [2023-12-26 16:47:12,313][105620] Updated weights for policy 1, policy_version 195902 (0.0010) [2023-12-26 16:47:12,362][105692] Updated weights for policy 0, policy_version 195102 (0.0009) [2023-12-26 16:47:12,384][105620] Updated weights for policy 1, policy_version 195912 (0.0008) [2023-12-26 16:47:12,425][105692] Updated weights for policy 0, policy_version 195112 (0.0007) [2023-12-26 16:47:12,489][105692] Updated weights for policy 0, policy_version 195122 (0.0007) [2023-12-26 16:47:13,057][105620] Updated weights for policy 1, policy_version 195922 (0.0009) [2023-12-26 16:47:13,112][105620] Updated weights for policy 1, policy_version 195932 (0.0009) [2023-12-26 16:47:13,168][105620] Updated weights for policy 1, policy_version 195942 (0.0009) [2023-12-26 16:47:13,209][105692] Updated weights for policy 0, policy_version 195132 (0.0009) [2023-12-26 16:47:13,266][105692] Updated weights for policy 0, policy_version 195142 (0.0006) [2023-12-26 16:47:13,327][105692] Updated weights for policy 0, policy_version 195152 (0.0005) [2023-12-26 16:47:13,910][105692] Updated weights for policy 0, policy_version 195162 (0.0006) [2023-12-26 16:47:13,957][105620] Updated weights for policy 1, policy_version 195952 (0.0008) [2023-12-26 16:47:13,963][105692] Updated weights for policy 0, policy_version 195172 (0.0008) [2023-12-26 16:47:14,017][105620] Updated weights for policy 1, policy_version 195962 (0.0006) [2023-12-26 16:47:14,019][105692] Updated weights for policy 0, policy_version 195182 (0.0006) [2023-12-26 16:47:14,065][105620] Updated weights for policy 1, policy_version 195972 (0.0006) [2023-12-26 16:47:14,071][105692] Updated weights for policy 0, policy_version 195192 (0.0007) [2023-12-26 16:47:14,789][105692] Updated weights for policy 0, policy_version 195202 (0.0009) [2023-12-26 16:47:14,847][105692] Updated weights for policy 0, policy_version 195212 (0.0007) [2023-12-26 16:47:14,849][105620] Updated weights for policy 1, policy_version 195982 (0.0009) [2023-12-26 16:47:14,904][105692] Updated weights for policy 0, policy_version 195222 (0.0006) [2023-12-26 16:47:14,907][105620] Updated weights for policy 1, policy_version 195992 (0.0007) [2023-12-26 16:47:14,964][105620] Updated weights for policy 1, policy_version 196002 (0.0008) [2023-12-26 16:47:15,566][105692] Updated weights for policy 0, policy_version 195232 (0.0009) [2023-12-26 16:47:15,628][105692] Updated weights for policy 0, policy_version 195242 (0.0009) [2023-12-26 16:47:15,673][105692] Updated weights for policy 0, policy_version 195252 (0.0005) [2023-12-26 16:47:15,787][105620] Updated weights for policy 1, policy_version 196012 (0.0009) [2023-12-26 16:47:15,854][105620] Updated weights for policy 1, policy_version 196022 (0.0010) [2023-12-26 16:47:15,915][105620] Updated weights for policy 1, policy_version 196032 (0.0010) [2023-12-26 16:47:16,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 100188160. Throughput: 0: 9630.3, 1: 9695.8. Samples: 100156060. Policy #0 lag: (min: 3.0, avg: 7.3, max: 35.0) [2023-12-26 16:47:16,063][104569] Avg episode reward: [(0, '9006.334'), (1, '9000.463')] [2023-12-26 16:47:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000195256_49995776.pth... [2023-12-26 16:47:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000196040_50192384.pth... [2023-12-26 16:47:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000194104_49700864.pth [2023-12-26 16:47:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000194888_49897472.pth [2023-12-26 16:47:16,264][105692] Updated weights for policy 0, policy_version 195262 (0.0005) [2023-12-26 16:47:16,333][105692] Updated weights for policy 0, policy_version 195272 (0.0007) [2023-12-26 16:47:16,395][105692] Updated weights for policy 0, policy_version 195282 (0.0010) [2023-12-26 16:47:16,629][105620] Updated weights for policy 1, policy_version 196042 (0.0008) [2023-12-26 16:47:16,680][105620] Updated weights for policy 1, policy_version 196052 (0.0007) [2023-12-26 16:47:16,736][105620] Updated weights for policy 1, policy_version 196062 (0.0008) [2023-12-26 16:47:16,791][105620] Updated weights for policy 1, policy_version 196072 (0.0008) [2023-12-26 16:47:17,085][105692] Updated weights for policy 0, policy_version 195292 (0.0010) [2023-12-26 16:47:17,140][105692] Updated weights for policy 0, policy_version 195302 (0.0010) [2023-12-26 16:47:17,196][105692] Updated weights for policy 0, policy_version 195312 (0.0011) [2023-12-26 16:47:17,493][105620] Updated weights for policy 1, policy_version 196082 (0.0010) [2023-12-26 16:47:17,546][105620] Updated weights for policy 1, policy_version 196092 (0.0011) [2023-12-26 16:47:17,600][105620] Updated weights for policy 1, policy_version 196102 (0.0009) [2023-12-26 16:47:17,935][105692] Updated weights for policy 0, policy_version 195322 (0.0010) [2023-12-26 16:47:17,995][105692] Updated weights for policy 0, policy_version 195332 (0.0010) [2023-12-26 16:47:18,060][105692] Updated weights for policy 0, policy_version 195342 (0.0010) [2023-12-26 16:47:18,108][105692] Updated weights for policy 0, policy_version 195352 (0.0006) [2023-12-26 16:47:18,287][105620] Updated weights for policy 1, policy_version 196112 (0.0009) [2023-12-26 16:47:18,338][105620] Updated weights for policy 1, policy_version 196122 (0.0011) [2023-12-26 16:47:18,400][105620] Updated weights for policy 1, policy_version 196132 (0.0011) [2023-12-26 16:47:18,700][105692] Updated weights for policy 0, policy_version 195362 (0.0010) [2023-12-26 16:47:18,762][105692] Updated weights for policy 0, policy_version 195372 (0.0008) [2023-12-26 16:47:18,821][105692] Updated weights for policy 0, policy_version 195382 (0.0009) [2023-12-26 16:47:19,139][105620] Updated weights for policy 1, policy_version 196142 (0.0010) [2023-12-26 16:47:19,199][105620] Updated weights for policy 1, policy_version 196152 (0.0010) [2023-12-26 16:47:19,268][105620] Updated weights for policy 1, policy_version 196162 (0.0006) [2023-12-26 16:47:19,508][105692] Updated weights for policy 0, policy_version 195392 (0.0009) [2023-12-26 16:47:19,571][105692] Updated weights for policy 0, policy_version 195402 (0.0007) [2023-12-26 16:47:19,637][105692] Updated weights for policy 0, policy_version 195412 (0.0010) [2023-12-26 16:47:19,977][105620] Updated weights for policy 1, policy_version 196172 (0.0008) [2023-12-26 16:47:20,036][105620] Updated weights for policy 1, policy_version 196182 (0.0008) [2023-12-26 16:47:20,088][105620] Updated weights for policy 1, policy_version 196192 (0.0008) [2023-12-26 16:47:20,380][105692] Updated weights for policy 0, policy_version 195422 (0.0011) [2023-12-26 16:47:20,442][105692] Updated weights for policy 0, policy_version 195432 (0.0010) [2023-12-26 16:47:20,512][105692] Updated weights for policy 0, policy_version 195442 (0.0010) [2023-12-26 16:47:20,822][105620] Updated weights for policy 1, policy_version 196202 (0.0007) [2023-12-26 16:47:20,895][105620] Updated weights for policy 1, policy_version 196212 (0.0006) [2023-12-26 16:47:20,959][105620] Updated weights for policy 1, policy_version 196222 (0.0006) [2023-12-26 16:47:21,024][105620] Updated weights for policy 1, policy_version 196232 (0.0006) [2023-12-26 16:47:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 100286464. Throughput: 0: 9680.9, 1: 9753.7. Samples: 100274608. Policy #0 lag: (min: 3.0, avg: 7.3, max: 35.0) [2023-12-26 16:47:21,062][104569] Avg episode reward: [(0, '9094.997'), (1, '9090.296')] [2023-12-26 16:47:21,150][105692] Updated weights for policy 0, policy_version 195452 (0.0011) [2023-12-26 16:47:21,210][105692] Updated weights for policy 0, policy_version 195462 (0.0011) [2023-12-26 16:47:21,276][105692] Updated weights for policy 0, policy_version 195472 (0.0009) [2023-12-26 16:47:21,770][105620] Updated weights for policy 1, policy_version 196242 (0.0008) [2023-12-26 16:47:21,833][105620] Updated weights for policy 1, policy_version 196252 (0.0008) [2023-12-26 16:47:21,903][105620] Updated weights for policy 1, policy_version 196262 (0.0008) [2023-12-26 16:47:22,009][105692] Updated weights for policy 0, policy_version 195482 (0.0010) [2023-12-26 16:47:22,078][105692] Updated weights for policy 0, policy_version 195492 (0.0007) [2023-12-26 16:47:22,139][105692] Updated weights for policy 0, policy_version 195502 (0.0011) [2023-12-26 16:47:22,199][105692] Updated weights for policy 0, policy_version 195512 (0.0011) [2023-12-26 16:47:22,656][105620] Updated weights for policy 1, policy_version 196272 (0.0007) [2023-12-26 16:47:22,713][105620] Updated weights for policy 1, policy_version 196282 (0.0007) [2023-12-26 16:47:22,776][105620] Updated weights for policy 1, policy_version 196292 (0.0008) [2023-12-26 16:47:22,920][105692] Updated weights for policy 0, policy_version 195522 (0.0010) [2023-12-26 16:47:22,978][105692] Updated weights for policy 0, policy_version 195532 (0.0010) [2023-12-26 16:47:23,040][105692] Updated weights for policy 0, policy_version 195542 (0.0010) [2023-12-26 16:47:23,506][105620] Updated weights for policy 1, policy_version 196302 (0.0009) [2023-12-26 16:47:23,563][105620] Updated weights for policy 1, policy_version 196312 (0.0010) [2023-12-26 16:47:23,626][105620] Updated weights for policy 1, policy_version 196323 (0.0009) [2023-12-26 16:47:23,647][105692] Updated weights for policy 0, policy_version 195552 (0.0007) [2023-12-26 16:47:23,711][105692] Updated weights for policy 0, policy_version 195562 (0.0006) [2023-12-26 16:47:23,782][105692] Updated weights for policy 0, policy_version 195572 (0.0009) [2023-12-26 16:47:24,381][105692] Updated weights for policy 0, policy_version 195582 (0.0009) [2023-12-26 16:47:24,432][105692] Updated weights for policy 0, policy_version 195592 (0.0010) [2023-12-26 16:47:24,471][105620] Updated weights for policy 1, policy_version 196333 (0.0008) [2023-12-26 16:47:24,481][105692] Updated weights for policy 0, policy_version 195602 (0.0010) [2023-12-26 16:47:24,524][105620] Updated weights for policy 1, policy_version 196343 (0.0006) [2023-12-26 16:47:24,572][105620] Updated weights for policy 1, policy_version 196353 (0.0008) [2023-12-26 16:47:25,131][105692] Updated weights for policy 0, policy_version 195612 (0.0008) [2023-12-26 16:47:25,184][105692] Updated weights for policy 0, policy_version 195622 (0.0005) [2023-12-26 16:47:25,239][105692] Updated weights for policy 0, policy_version 195632 (0.0008) [2023-12-26 16:47:25,401][105620] Updated weights for policy 1, policy_version 196363 (0.0009) [2023-12-26 16:47:25,454][105620] Updated weights for policy 1, policy_version 196373 (0.0008) [2023-12-26 16:47:25,510][105620] Updated weights for policy 1, policy_version 196383 (0.0009) [2023-12-26 16:47:25,961][105692] Updated weights for policy 0, policy_version 195642 (0.0009) [2023-12-26 16:47:26,015][105692] Updated weights for policy 0, policy_version 195653 (0.0010) [2023-12-26 16:47:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 100376576. Throughput: 0: 9799.0, 1: 9704.6. Samples: 100390972. Policy #0 lag: (min: 3.0, avg: 7.3, max: 35.0) [2023-12-26 16:47:26,062][104569] Avg episode reward: [(0, '8651.331'), (1, '9087.372')] [2023-12-26 16:47:26,063][105692] Updated weights for policy 0, policy_version 195663 (0.0009) [2023-12-26 16:47:26,109][105620] Updated weights for policy 1, policy_version 196393 (0.0008) [2023-12-26 16:47:26,174][105620] Updated weights for policy 1, policy_version 196403 (0.0009) [2023-12-26 16:47:26,237][105620] Updated weights for policy 1, policy_version 196413 (0.0008) [2023-12-26 16:47:26,296][105620] Updated weights for policy 1, policy_version 196423 (0.0008) [2023-12-26 16:47:26,836][105692] Updated weights for policy 0, policy_version 195674 (0.0009) [2023-12-26 16:47:26,893][105692] Updated weights for policy 0, policy_version 195684 (0.0009) [2023-12-26 16:47:26,945][105692] Updated weights for policy 0, policy_version 195694 (0.0007) [2023-12-26 16:47:26,991][105620] Updated weights for policy 1, policy_version 196433 (0.0006) [2023-12-26 16:47:27,000][105692] Updated weights for policy 0, policy_version 195704 (0.0007) [2023-12-26 16:47:27,039][105620] Updated weights for policy 1, policy_version 196443 (0.0008) [2023-12-26 16:47:27,095][105620] Updated weights for policy 1, policy_version 196453 (0.0009) [2023-12-26 16:47:27,718][105620] Updated weights for policy 1, policy_version 196463 (0.0007) [2023-12-26 16:47:27,766][105620] Updated weights for policy 1, policy_version 196473 (0.0007) [2023-12-26 16:47:27,796][105692] Updated weights for policy 0, policy_version 195714 (0.0007) [2023-12-26 16:47:27,819][105620] Updated weights for policy 1, policy_version 196483 (0.0008) [2023-12-26 16:47:27,842][105692] Updated weights for policy 0, policy_version 195724 (0.0006) [2023-12-26 16:47:27,889][105692] Updated weights for policy 0, policy_version 195734 (0.0009) [2023-12-26 16:47:28,558][105692] Updated weights for policy 0, policy_version 195744 (0.0009) [2023-12-26 16:47:28,595][105620] Updated weights for policy 1, policy_version 196493 (0.0006) [2023-12-26 16:47:28,610][105692] Updated weights for policy 0, policy_version 195754 (0.0008) [2023-12-26 16:47:28,652][105620] Updated weights for policy 1, policy_version 196503 (0.0007) [2023-12-26 16:47:28,668][105692] Updated weights for policy 0, policy_version 195764 (0.0005) [2023-12-26 16:47:28,710][105620] Updated weights for policy 1, policy_version 196513 (0.0008) [2023-12-26 16:47:29,331][105692] Updated weights for policy 0, policy_version 195774 (0.0008) [2023-12-26 16:47:29,392][105692] Updated weights for policy 0, policy_version 195784 (0.0010) [2023-12-26 16:47:29,448][105692] Updated weights for policy 0, policy_version 195794 (0.0009) [2023-12-26 16:47:29,485][105620] Updated weights for policy 1, policy_version 196523 (0.0009) [2023-12-26 16:47:29,532][105620] Updated weights for policy 1, policy_version 196533 (0.0009) [2023-12-26 16:47:29,583][105620] Updated weights for policy 1, policy_version 196543 (0.0008) [2023-12-26 16:47:30,227][105620] Updated weights for policy 1, policy_version 196553 (0.0005) [2023-12-26 16:47:30,279][105620] Updated weights for policy 1, policy_version 196563 (0.0006) [2023-12-26 16:47:30,294][105692] Updated weights for policy 0, policy_version 195804 (0.0008) [2023-12-26 16:47:30,331][105620] Updated weights for policy 1, policy_version 196573 (0.0005) [2023-12-26 16:47:30,352][105692] Updated weights for policy 0, policy_version 195814 (0.0009) [2023-12-26 16:47:30,388][105620] Updated weights for policy 1, policy_version 196583 (0.0007) [2023-12-26 16:47:30,407][105692] Updated weights for policy 0, policy_version 195824 (0.0008) [2023-12-26 16:47:30,979][105620] Updated weights for policy 1, policy_version 196593 (0.0010) [2023-12-26 16:47:31,044][105620] Updated weights for policy 1, policy_version 196603 (0.0010) [2023-12-26 16:47:31,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 100474880. Throughput: 0: 9821.7, 1: 9683.7. Samples: 100448528. Policy #0 lag: (min: 3.0, avg: 7.3, max: 35.0) [2023-12-26 16:47:31,062][104569] Avg episode reward: [(0, '8488.964'), (1, '9086.003')] [2023-12-26 16:47:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000195832_50143232.pth... [2023-12-26 16:47:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000194680_49848320.pth [2023-12-26 16:47:31,108][105620] Updated weights for policy 1, policy_version 196613 (0.0010) [2023-12-26 16:47:31,119][105692] Updated weights for policy 0, policy_version 195834 (0.0010) [2023-12-26 16:47:31,128][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000196616_50339840.pth... [2023-12-26 16:47:31,133][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000195464_50044928.pth [2023-12-26 16:47:31,179][105692] Updated weights for policy 0, policy_version 195844 (0.0010) [2023-12-26 16:47:31,234][105692] Updated weights for policy 0, policy_version 195854 (0.0010) [2023-12-26 16:47:31,292][105692] Updated weights for policy 0, policy_version 195864 (0.0011) [2023-12-26 16:47:31,856][105620] Updated weights for policy 1, policy_version 196623 (0.0010) [2023-12-26 16:47:31,917][105620] Updated weights for policy 1, policy_version 196633 (0.0005) [2023-12-26 16:47:31,982][105620] Updated weights for policy 1, policy_version 196643 (0.0007) [2023-12-26 16:47:32,105][105692] Updated weights for policy 0, policy_version 195874 (0.0011) [2023-12-26 16:47:32,155][105692] Updated weights for policy 0, policy_version 195884 (0.0011) [2023-12-26 16:47:32,204][105692] Updated weights for policy 0, policy_version 195894 (0.0010) [2023-12-26 16:47:32,628][105620] Updated weights for policy 1, policy_version 196653 (0.0008) [2023-12-26 16:47:32,697][105620] Updated weights for policy 1, policy_version 196663 (0.0006) [2023-12-26 16:47:32,768][105620] Updated weights for policy 1, policy_version 196673 (0.0005) [2023-12-26 16:47:32,922][105692] Updated weights for policy 0, policy_version 195904 (0.0010) [2023-12-26 16:47:32,987][105692] Updated weights for policy 0, policy_version 195914 (0.0008) [2023-12-26 16:47:33,042][105692] Updated weights for policy 0, policy_version 195924 (0.0010) [2023-12-26 16:47:33,352][105620] Updated weights for policy 1, policy_version 196683 (0.0007) [2023-12-26 16:47:33,403][105620] Updated weights for policy 1, policy_version 196693 (0.0010) [2023-12-26 16:47:33,450][105620] Updated weights for policy 1, policy_version 196703 (0.0010) [2023-12-26 16:47:33,643][105692] Updated weights for policy 0, policy_version 195934 (0.0005) [2023-12-26 16:47:33,689][105692] Updated weights for policy 0, policy_version 195944 (0.0005) [2023-12-26 16:47:33,735][105692] Updated weights for policy 0, policy_version 195954 (0.0005) [2023-12-26 16:47:34,059][105620] Updated weights for policy 1, policy_version 196713 (0.0010) [2023-12-26 16:47:34,118][105620] Updated weights for policy 1, policy_version 196723 (0.0010) [2023-12-26 16:47:34,183][105620] Updated weights for policy 1, policy_version 196733 (0.0009) [2023-12-26 16:47:34,250][105620] Updated weights for policy 1, policy_version 196743 (0.0009) [2023-12-26 16:47:34,349][105692] Updated weights for policy 0, policy_version 195964 (0.0005) [2023-12-26 16:47:34,409][105692] Updated weights for policy 0, policy_version 195974 (0.0006) [2023-12-26 16:47:34,464][105692] Updated weights for policy 0, policy_version 195984 (0.0008) [2023-12-26 16:47:35,035][105620] Updated weights for policy 1, policy_version 196753 (0.0009) [2023-12-26 16:47:35,092][105620] Updated weights for policy 1, policy_version 196763 (0.0008) [2023-12-26 16:47:35,111][105692] Updated weights for policy 0, policy_version 195994 (0.0008) [2023-12-26 16:47:35,149][105620] Updated weights for policy 1, policy_version 196773 (0.0008) [2023-12-26 16:47:35,170][105692] Updated weights for policy 0, policy_version 196004 (0.0005) [2023-12-26 16:47:35,228][105692] Updated weights for policy 0, policy_version 196014 (0.0005) [2023-12-26 16:47:35,285][105692] Updated weights for policy 0, policy_version 196024 (0.0005) [2023-12-26 16:47:35,908][105620] Updated weights for policy 1, policy_version 196783 (0.0010) [2023-12-26 16:47:35,931][105692] Updated weights for policy 0, policy_version 196034 (0.0006) [2023-12-26 16:47:35,958][105620] Updated weights for policy 1, policy_version 196793 (0.0010) [2023-12-26 16:47:35,984][105692] Updated weights for policy 0, policy_version 196044 (0.0008) [2023-12-26 16:47:36,023][105620] Updated weights for policy 1, policy_version 196803 (0.0010) [2023-12-26 16:47:36,045][105692] Updated weights for policy 0, policy_version 196054 (0.0007) [2023-12-26 16:47:36,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 100589568. Throughput: 0: 9861.8, 1: 9748.5. Samples: 100570468. Policy #0 lag: (min: 3.0, avg: 7.3, max: 35.0) [2023-12-26 16:47:36,062][104569] Avg episode reward: [(0, '8925.358'), (1, '9174.255')] [2023-12-26 16:47:36,773][105620] Updated weights for policy 1, policy_version 196813 (0.0010) [2023-12-26 16:47:36,829][105620] Updated weights for policy 1, policy_version 196823 (0.0010) [2023-12-26 16:47:36,847][105692] Updated weights for policy 0, policy_version 196064 (0.0006) [2023-12-26 16:47:36,880][105620] Updated weights for policy 1, policy_version 196833 (0.0010) [2023-12-26 16:47:36,910][105692] Updated weights for policy 0, policy_version 196074 (0.0006) [2023-12-26 16:47:36,966][105692] Updated weights for policy 0, policy_version 196084 (0.0006) [2023-12-26 16:47:37,558][105692] Updated weights for policy 0, policy_version 196094 (0.0007) [2023-12-26 16:47:37,615][105692] Updated weights for policy 0, policy_version 196104 (0.0008) [2023-12-26 16:47:37,638][105620] Updated weights for policy 1, policy_version 196843 (0.0010) [2023-12-26 16:47:37,676][105692] Updated weights for policy 0, policy_version 196114 (0.0006) [2023-12-26 16:47:37,701][105620] Updated weights for policy 1, policy_version 196853 (0.0010) [2023-12-26 16:47:37,753][105620] Updated weights for policy 1, policy_version 196863 (0.0010) [2023-12-26 16:47:38,439][105692] Updated weights for policy 0, policy_version 196124 (0.0009) [2023-12-26 16:47:38,503][105692] Updated weights for policy 0, policy_version 196134 (0.0008) [2023-12-26 16:47:38,514][105620] Updated weights for policy 1, policy_version 196873 (0.0010) [2023-12-26 16:47:38,567][105692] Updated weights for policy 0, policy_version 196144 (0.0006) [2023-12-26 16:47:38,580][105620] Updated weights for policy 1, policy_version 196883 (0.0011) [2023-12-26 16:47:38,642][105620] Updated weights for policy 1, policy_version 196893 (0.0010) [2023-12-26 16:47:38,707][105620] Updated weights for policy 1, policy_version 196903 (0.0010) [2023-12-26 16:47:39,216][105692] Updated weights for policy 0, policy_version 196154 (0.0008) [2023-12-26 16:47:39,288][105692] Updated weights for policy 0, policy_version 196164 (0.0008) [2023-12-26 16:47:39,355][105692] Updated weights for policy 0, policy_version 196174 (0.0007) [2023-12-26 16:47:39,419][105692] Updated weights for policy 0, policy_version 196184 (0.0008) [2023-12-26 16:47:39,458][105620] Updated weights for policy 1, policy_version 196913 (0.0008) [2023-12-26 16:47:39,525][105620] Updated weights for policy 1, policy_version 196923 (0.0008) [2023-12-26 16:47:39,595][105620] Updated weights for policy 1, policy_version 196933 (0.0008) [2023-12-26 16:47:40,100][105692] Updated weights for policy 0, policy_version 196194 (0.0009) [2023-12-26 16:47:40,162][105692] Updated weights for policy 0, policy_version 196204 (0.0008) [2023-12-26 16:47:40,217][105692] Updated weights for policy 0, policy_version 196214 (0.0007) [2023-12-26 16:47:40,356][105620] Updated weights for policy 1, policy_version 196943 (0.0008) [2023-12-26 16:47:40,416][105620] Updated weights for policy 1, policy_version 196953 (0.0009) [2023-12-26 16:47:40,465][105620] Updated weights for policy 1, policy_version 196963 (0.0009) [2023-12-26 16:47:40,964][105692] Updated weights for policy 0, policy_version 196224 (0.0006) [2023-12-26 16:47:41,032][105692] Updated weights for policy 0, policy_version 196234 (0.0006) [2023-12-26 16:47:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 100671488. Throughput: 0: 9889.2, 1: 9710.9. Samples: 100685644. Policy #0 lag: (min: 3.0, avg: 7.3, max: 35.0) [2023-12-26 16:47:41,062][104569] Avg episode reward: [(0, '9173.874'), (1, '9173.005')] [2023-12-26 16:47:41,096][105692] Updated weights for policy 0, policy_version 196244 (0.0009) [2023-12-26 16:47:41,268][105620] Updated weights for policy 1, policy_version 196973 (0.0009) [2023-12-26 16:47:41,327][105620] Updated weights for policy 1, policy_version 196983 (0.0009) [2023-12-26 16:47:41,396][105620] Updated weights for policy 1, policy_version 196993 (0.0009) [2023-12-26 16:47:41,816][105692] Updated weights for policy 0, policy_version 196254 (0.0009) [2023-12-26 16:47:41,879][105692] Updated weights for policy 0, policy_version 196264 (0.0009) [2023-12-26 16:47:41,942][105692] Updated weights for policy 0, policy_version 196274 (0.0009) [2023-12-26 16:47:42,111][105620] Updated weights for policy 1, policy_version 197003 (0.0008) [2023-12-26 16:47:42,170][105620] Updated weights for policy 1, policy_version 197013 (0.0006) [2023-12-26 16:47:42,238][105620] Updated weights for policy 1, policy_version 197023 (0.0006) [2023-12-26 16:47:42,717][105692] Updated weights for policy 0, policy_version 196284 (0.0009) [2023-12-26 16:47:42,769][105692] Updated weights for policy 0, policy_version 196294 (0.0008) [2023-12-26 16:47:42,822][105692] Updated weights for policy 0, policy_version 196304 (0.0008) [2023-12-26 16:47:42,953][105620] Updated weights for policy 1, policy_version 197033 (0.0008) [2023-12-26 16:47:43,014][105620] Updated weights for policy 1, policy_version 197043 (0.0010) [2023-12-26 16:47:43,071][105620] Updated weights for policy 1, policy_version 197053 (0.0009) [2023-12-26 16:47:43,131][105620] Updated weights for policy 1, policy_version 197063 (0.0008) [2023-12-26 16:47:43,673][105692] Updated weights for policy 0, policy_version 196314 (0.0008) [2023-12-26 16:47:43,711][105620] Updated weights for policy 1, policy_version 197073 (0.0008) [2023-12-26 16:47:43,737][105692] Updated weights for policy 0, policy_version 196324 (0.0008) [2023-12-26 16:47:43,768][105620] Updated weights for policy 1, policy_version 197083 (0.0005) [2023-12-26 16:47:43,802][105692] Updated weights for policy 0, policy_version 196334 (0.0008) [2023-12-26 16:47:43,816][105620] Updated weights for policy 1, policy_version 197093 (0.0005) [2023-12-26 16:47:43,864][105692] Updated weights for policy 0, policy_version 196344 (0.0009) [2023-12-26 16:47:44,341][105620] Updated weights for policy 1, policy_version 197103 (0.0006) [2023-12-26 16:47:44,398][105620] Updated weights for policy 1, policy_version 197113 (0.0005) [2023-12-26 16:47:44,452][105620] Updated weights for policy 1, policy_version 197123 (0.0005) [2023-12-26 16:47:44,760][105692] Updated weights for policy 0, policy_version 196354 (0.0009) [2023-12-26 16:47:44,817][105692] Updated weights for policy 0, policy_version 196364 (0.0008) [2023-12-26 16:47:44,877][105692] Updated weights for policy 0, policy_version 196374 (0.0008) [2023-12-26 16:47:45,036][105620] Updated weights for policy 1, policy_version 197133 (0.0008) [2023-12-26 16:47:45,099][105620] Updated weights for policy 1, policy_version 197143 (0.0011) [2023-12-26 16:47:45,170][105620] Updated weights for policy 1, policy_version 197153 (0.0011) [2023-12-26 16:47:45,680][105692] Updated weights for policy 0, policy_version 196384 (0.0010) [2023-12-26 16:47:45,732][105692] Updated weights for policy 0, policy_version 196394 (0.0008) [2023-12-26 16:47:45,779][105620] Updated weights for policy 1, policy_version 197163 (0.0010) [2023-12-26 16:47:45,789][105692] Updated weights for policy 0, policy_version 196404 (0.0008) [2023-12-26 16:47:45,830][105620] Updated weights for policy 1, policy_version 197173 (0.0010) [2023-12-26 16:47:45,881][105620] Updated weights for policy 1, policy_version 197183 (0.0010) [2023-12-26 16:47:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 100777984. Throughput: 0: 9802.8, 1: 9732.7. Samples: 100742824. Policy #0 lag: (min: 3.0, avg: 7.3, max: 35.0) [2023-12-26 16:47:46,062][104569] Avg episode reward: [(0, '9090.789'), (1, '9175.340')] [2023-12-26 16:47:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000196408_50290688.pth... [2023-12-26 16:47:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000197192_50487296.pth... [2023-12-26 16:47:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000195256_49995776.pth [2023-12-26 16:47:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000196040_50192384.pth [2023-12-26 16:47:46,549][105620] Updated weights for policy 1, policy_version 197193 (0.0010) [2023-12-26 16:47:46,597][105620] Updated weights for policy 1, policy_version 197203 (0.0010) [2023-12-26 16:47:46,603][105692] Updated weights for policy 0, policy_version 196414 (0.0005) [2023-12-26 16:47:46,642][105620] Updated weights for policy 1, policy_version 197213 (0.0010) [2023-12-26 16:47:46,663][105692] Updated weights for policy 0, policy_version 196424 (0.0006) [2023-12-26 16:47:46,686][105620] Updated weights for policy 1, policy_version 197223 (0.0010) [2023-12-26 16:47:46,721][105692] Updated weights for policy 0, policy_version 196434 (0.0007) [2023-12-26 16:47:47,453][105620] Updated weights for policy 1, policy_version 197233 (0.0010) [2023-12-26 16:47:47,490][105692] Updated weights for policy 0, policy_version 196444 (0.0008) [2023-12-26 16:47:47,501][105620] Updated weights for policy 1, policy_version 197243 (0.0010) [2023-12-26 16:47:47,539][105692] Updated weights for policy 0, policy_version 196454 (0.0005) [2023-12-26 16:47:47,549][105620] Updated weights for policy 1, policy_version 197253 (0.0010) [2023-12-26 16:47:47,585][105692] Updated weights for policy 0, policy_version 196464 (0.0006) [2023-12-26 16:47:48,317][105620] Updated weights for policy 1, policy_version 197263 (0.0010) [2023-12-26 16:47:48,325][105692] Updated weights for policy 0, policy_version 196474 (0.0007) [2023-12-26 16:47:48,380][105620] Updated weights for policy 1, policy_version 197273 (0.0011) [2023-12-26 16:47:48,391][105692] Updated weights for policy 0, policy_version 196484 (0.0006) [2023-12-26 16:47:48,440][105620] Updated weights for policy 1, policy_version 197283 (0.0010) [2023-12-26 16:47:48,451][105692] Updated weights for policy 0, policy_version 196494 (0.0005) [2023-12-26 16:47:48,515][105692] Updated weights for policy 0, policy_version 196504 (0.0007) [2023-12-26 16:47:49,175][105620] Updated weights for policy 1, policy_version 197293 (0.0009) [2023-12-26 16:47:49,236][105620] Updated weights for policy 1, policy_version 197303 (0.0009) [2023-12-26 16:47:49,263][105692] Updated weights for policy 0, policy_version 196514 (0.0007) [2023-12-26 16:47:49,301][105620] Updated weights for policy 1, policy_version 197313 (0.0009) [2023-12-26 16:47:49,325][105692] Updated weights for policy 0, policy_version 196524 (0.0006) [2023-12-26 16:47:49,389][105692] Updated weights for policy 0, policy_version 196534 (0.0009) [2023-12-26 16:47:50,030][105620] Updated weights for policy 1, policy_version 197323 (0.0008) [2023-12-26 16:47:50,082][105620] Updated weights for policy 1, policy_version 197334 (0.0010) [2023-12-26 16:47:50,132][105620] Updated weights for policy 1, policy_version 197344 (0.0008) [2023-12-26 16:47:50,184][105692] Updated weights for policy 0, policy_version 196544 (0.0008) [2023-12-26 16:47:50,232][105692] Updated weights for policy 0, policy_version 196554 (0.0007) [2023-12-26 16:47:50,290][105692] Updated weights for policy 0, policy_version 196564 (0.0008) [2023-12-26 16:47:50,921][105620] Updated weights for policy 1, policy_version 197354 (0.0008) [2023-12-26 16:47:50,977][105620] Updated weights for policy 1, policy_version 197364 (0.0009) [2023-12-26 16:47:51,031][105692] Updated weights for policy 0, policy_version 196574 (0.0008) [2023-12-26 16:47:51,034][105620] Updated weights for policy 1, policy_version 197374 (0.0008) [2023-12-26 16:47:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 100859904. Throughput: 0: 9684.8, 1: 9736.7. Samples: 100857272. Policy #0 lag: (min: 3.0, avg: 7.3, max: 35.0) [2023-12-26 16:47:51,063][104569] Avg episode reward: [(0, '8913.019'), (1, '9175.231')] [2023-12-26 16:47:51,085][105692] Updated weights for policy 0, policy_version 196584 (0.0008) [2023-12-26 16:47:51,096][105620] Updated weights for policy 1, policy_version 197384 (0.0007) [2023-12-26 16:47:51,138][105692] Updated weights for policy 0, policy_version 196594 (0.0008) [2023-12-26 16:47:51,808][105620] Updated weights for policy 1, policy_version 197394 (0.0009) [2023-12-26 16:47:51,860][105620] Updated weights for policy 1, policy_version 197404 (0.0009) [2023-12-26 16:47:51,922][105620] Updated weights for policy 1, policy_version 197414 (0.0009) [2023-12-26 16:47:51,961][105692] Updated weights for policy 0, policy_version 196604 (0.0010) [2023-12-26 16:47:52,011][105692] Updated weights for policy 0, policy_version 196614 (0.0009) [2023-12-26 16:47:52,059][105692] Updated weights for policy 0, policy_version 196624 (0.0009) [2023-12-26 16:47:52,646][105620] Updated weights for policy 1, policy_version 197424 (0.0008) [2023-12-26 16:47:52,709][105620] Updated weights for policy 1, policy_version 197434 (0.0009) [2023-12-26 16:47:52,735][105692] Updated weights for policy 0, policy_version 196634 (0.0009) [2023-12-26 16:47:52,764][105620] Updated weights for policy 1, policy_version 197444 (0.0008) [2023-12-26 16:47:52,795][105692] Updated weights for policy 0, policy_version 196644 (0.0008) [2023-12-26 16:47:52,853][105692] Updated weights for policy 0, policy_version 196654 (0.0009) [2023-12-26 16:47:52,904][105692] Updated weights for policy 0, policy_version 196664 (0.0009) [2023-12-26 16:47:53,556][105620] Updated weights for policy 1, policy_version 197454 (0.0008) [2023-12-26 16:47:53,607][105692] Updated weights for policy 0, policy_version 196674 (0.0006) [2023-12-26 16:47:53,613][105620] Updated weights for policy 1, policy_version 197464 (0.0006) [2023-12-26 16:47:53,659][105692] Updated weights for policy 0, policy_version 196684 (0.0006) [2023-12-26 16:47:53,669][105620] Updated weights for policy 1, policy_version 197474 (0.0007) [2023-12-26 16:47:53,710][105692] Updated weights for policy 0, policy_version 196694 (0.0006) [2023-12-26 16:47:54,376][105692] Updated weights for policy 0, policy_version 196704 (0.0008) [2023-12-26 16:47:54,438][105692] Updated weights for policy 0, policy_version 196714 (0.0009) [2023-12-26 16:47:54,440][105620] Updated weights for policy 1, policy_version 197484 (0.0008) [2023-12-26 16:47:54,493][105620] Updated weights for policy 1, policy_version 197494 (0.0007) [2023-12-26 16:47:54,502][105692] Updated weights for policy 0, policy_version 196724 (0.0008) [2023-12-26 16:47:54,547][105620] Updated weights for policy 1, policy_version 197504 (0.0008) [2023-12-26 16:47:55,151][105620] Updated weights for policy 1, policy_version 197514 (0.0006) [2023-12-26 16:47:55,210][105620] Updated weights for policy 1, policy_version 197524 (0.0006) [2023-12-26 16:47:55,240][105692] Updated weights for policy 0, policy_version 196734 (0.0007) [2023-12-26 16:47:55,274][105620] Updated weights for policy 1, policy_version 197534 (0.0008) [2023-12-26 16:47:55,295][105692] Updated weights for policy 0, policy_version 196744 (0.0005) [2023-12-26 16:47:55,328][105620] Updated weights for policy 1, policy_version 197544 (0.0007) [2023-12-26 16:47:55,353][105692] Updated weights for policy 0, policy_version 196754 (0.0008) [2023-12-26 16:47:55,941][105620] Updated weights for policy 1, policy_version 197554 (0.0005) [2023-12-26 16:47:55,986][105692] Updated weights for policy 0, policy_version 196764 (0.0009) [2023-12-26 16:47:55,994][105620] Updated weights for policy 1, policy_version 197564 (0.0006) [2023-12-26 16:47:56,045][105620] Updated weights for policy 1, policy_version 197574 (0.0006) [2023-12-26 16:47:56,047][105692] Updated weights for policy 0, policy_version 196774 (0.0008) [2023-12-26 16:47:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 100966400. Throughput: 0: 9711.1, 1: 9722.6. Samples: 100974228. Policy #0 lag: (min: 3.0, avg: 7.3, max: 35.0) [2023-12-26 16:47:56,062][104569] Avg episode reward: [(0, '8912.023'), (1, '9264.885')] [2023-12-26 16:47:56,104][105692] Updated weights for policy 0, policy_version 196784 (0.0008) [2023-12-26 16:47:56,676][105620] Updated weights for policy 1, policy_version 197584 (0.0010) [2023-12-26 16:47:56,720][105620] Updated weights for policy 1, policy_version 197594 (0.0010) [2023-12-26 16:47:56,767][105620] Updated weights for policy 1, policy_version 197604 (0.0010) [2023-12-26 16:47:56,832][105692] Updated weights for policy 0, policy_version 196794 (0.0009) [2023-12-26 16:47:56,877][105692] Updated weights for policy 0, policy_version 196804 (0.0007) [2023-12-26 16:47:56,921][105692] Updated weights for policy 0, policy_version 196814 (0.0005) [2023-12-26 16:47:56,978][105692] Updated weights for policy 0, policy_version 196824 (0.0007) [2023-12-26 16:47:57,509][105620] Updated weights for policy 1, policy_version 197614 (0.0008) [2023-12-26 16:47:57,569][105620] Updated weights for policy 1, policy_version 197624 (0.0010) [2023-12-26 16:47:57,629][105620] Updated weights for policy 1, policy_version 197634 (0.0010) [2023-12-26 16:47:57,672][105692] Updated weights for policy 0, policy_version 196834 (0.0006) [2023-12-26 16:47:57,723][105692] Updated weights for policy 0, policy_version 196844 (0.0008) [2023-12-26 16:47:57,777][105692] Updated weights for policy 0, policy_version 196854 (0.0009) [2023-12-26 16:47:58,303][105620] Updated weights for policy 1, policy_version 197644 (0.0010) [2023-12-26 16:47:58,370][105620] Updated weights for policy 1, policy_version 197654 (0.0008) [2023-12-26 16:47:58,433][105620] Updated weights for policy 1, policy_version 197664 (0.0008) [2023-12-26 16:47:58,651][105692] Updated weights for policy 0, policy_version 196864 (0.0008) [2023-12-26 16:47:58,709][105692] Updated weights for policy 0, policy_version 196874 (0.0008) [2023-12-26 16:47:58,780][105692] Updated weights for policy 0, policy_version 196884 (0.0008) [2023-12-26 16:47:59,177][105620] Updated weights for policy 1, policy_version 197674 (0.0008) [2023-12-26 16:47:59,238][105620] Updated weights for policy 1, policy_version 197684 (0.0007) [2023-12-26 16:47:59,299][105620] Updated weights for policy 1, policy_version 197694 (0.0006) [2023-12-26 16:47:59,366][105620] Updated weights for policy 1, policy_version 197704 (0.0007) [2023-12-26 16:47:59,595][105692] Updated weights for policy 0, policy_version 196894 (0.0009) [2023-12-26 16:47:59,649][105692] Updated weights for policy 0, policy_version 196904 (0.0008) [2023-12-26 16:47:59,710][105692] Updated weights for policy 0, policy_version 196914 (0.0007) [2023-12-26 16:48:00,090][105620] Updated weights for policy 1, policy_version 197714 (0.0008) [2023-12-26 16:48:00,156][105620] Updated weights for policy 1, policy_version 197724 (0.0008) [2023-12-26 16:48:00,210][105620] Updated weights for policy 1, policy_version 197734 (0.0009) [2023-12-26 16:48:00,463][105692] Updated weights for policy 0, policy_version 196924 (0.0007) [2023-12-26 16:48:00,517][105692] Updated weights for policy 0, policy_version 196934 (0.0009) [2023-12-26 16:48:00,573][105692] Updated weights for policy 0, policy_version 196944 (0.0013) [2023-12-26 16:48:00,839][105620] Updated weights for policy 1, policy_version 197744 (0.0006) [2023-12-26 16:48:00,897][105620] Updated weights for policy 1, policy_version 197754 (0.0005) [2023-12-26 16:48:00,952][105620] Updated weights for policy 1, policy_version 197764 (0.0005) [2023-12-26 16:48:01,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 101064704. Throughput: 0: 9722.8, 1: 9752.5. Samples: 101032444. Policy #0 lag: (min: 23.0, avg: 31.8, max: 32.0) [2023-12-26 16:48:01,062][104569] Avg episode reward: [(0, '9090.868'), (1, '9173.228')] [2023-12-26 16:48:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000196952_50429952.pth... [2023-12-26 16:48:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000197768_50634752.pth... [2023-12-26 16:48:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000195832_50143232.pth [2023-12-26 16:48:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000196616_50339840.pth [2023-12-26 16:48:01,422][105692] Updated weights for policy 0, policy_version 196956 (0.0010) [2023-12-26 16:48:01,475][105692] Updated weights for policy 0, policy_version 196966 (0.0010) [2023-12-26 16:48:01,526][105692] Updated weights for policy 0, policy_version 196976 (0.0008) [2023-12-26 16:48:01,571][105620] Updated weights for policy 1, policy_version 197774 (0.0008) [2023-12-26 16:48:01,631][105620] Updated weights for policy 1, policy_version 197784 (0.0011) [2023-12-26 16:48:01,689][105620] Updated weights for policy 1, policy_version 197794 (0.0008) [2023-12-26 16:48:02,265][105620] Updated weights for policy 1, policy_version 197804 (0.0009) [2023-12-26 16:48:02,329][105620] Updated weights for policy 1, policy_version 197814 (0.0008) [2023-12-26 16:48:02,392][105620] Updated weights for policy 1, policy_version 197824 (0.0009) [2023-12-26 16:48:02,415][105692] Updated weights for policy 0, policy_version 196986 (0.0008) [2023-12-26 16:48:02,476][105692] Updated weights for policy 0, policy_version 196996 (0.0007) [2023-12-26 16:48:02,544][105692] Updated weights for policy 0, policy_version 197006 (0.0009) [2023-12-26 16:48:02,601][105692] Updated weights for policy 0, policy_version 197016 (0.0009) [2023-12-26 16:48:02,952][105620] Updated weights for policy 1, policy_version 197834 (0.0009) [2023-12-26 16:48:03,014][105620] Updated weights for policy 1, policy_version 197844 (0.0005) [2023-12-26 16:48:03,078][105620] Updated weights for policy 1, policy_version 197854 (0.0008) [2023-12-26 16:48:03,137][105620] Updated weights for policy 1, policy_version 197864 (0.0010) [2023-12-26 16:48:03,459][105692] Updated weights for policy 0, policy_version 197026 (0.0008) [2023-12-26 16:48:03,523][105692] Updated weights for policy 0, policy_version 197036 (0.0008) [2023-12-26 16:48:03,588][105692] Updated weights for policy 0, policy_version 197046 (0.0008) [2023-12-26 16:48:03,808][105620] Updated weights for policy 1, policy_version 197874 (0.0009) [2023-12-26 16:48:03,864][105620] Updated weights for policy 1, policy_version 197884 (0.0008) [2023-12-26 16:48:03,919][105620] Updated weights for policy 1, policy_version 197894 (0.0007) [2023-12-26 16:48:04,300][105692] Updated weights for policy 0, policy_version 197056 (0.0006) [2023-12-26 16:48:04,360][105692] Updated weights for policy 0, policy_version 197066 (0.0006) [2023-12-26 16:48:04,424][105692] Updated weights for policy 0, policy_version 197076 (0.0006) [2023-12-26 16:48:04,647][105620] Updated weights for policy 1, policy_version 197904 (0.0006) [2023-12-26 16:48:04,717][105620] Updated weights for policy 1, policy_version 197914 (0.0006) [2023-12-26 16:48:04,764][105620] Updated weights for policy 1, policy_version 197924 (0.0008) [2023-12-26 16:48:05,053][105692] Updated weights for policy 0, policy_version 197086 (0.0006) [2023-12-26 16:48:05,111][105692] Updated weights for policy 0, policy_version 197096 (0.0005) [2023-12-26 16:48:05,164][105692] Updated weights for policy 0, policy_version 197106 (0.0005) [2023-12-26 16:48:05,488][105620] Updated weights for policy 1, policy_version 197934 (0.0008) [2023-12-26 16:48:05,547][105620] Updated weights for policy 1, policy_version 197944 (0.0008) [2023-12-26 16:48:05,599][105620] Updated weights for policy 1, policy_version 197954 (0.0008) [2023-12-26 16:48:05,846][105692] Updated weights for policy 0, policy_version 197116 (0.0009) [2023-12-26 16:48:05,901][105692] Updated weights for policy 0, policy_version 197126 (0.0010) [2023-12-26 16:48:05,952][105692] Updated weights for policy 0, policy_version 197136 (0.0010) [2023-12-26 16:48:06,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 101163008. Throughput: 0: 9520.1, 1: 9904.6. Samples: 101148720. Policy #0 lag: (min: 23.0, avg: 31.8, max: 32.0) [2023-12-26 16:48:06,063][104569] Avg episode reward: [(0, '9067.940'), (1, '9173.077')] [2023-12-26 16:48:06,405][105620] Updated weights for policy 1, policy_version 197964 (0.0008) [2023-12-26 16:48:06,460][105620] Updated weights for policy 1, policy_version 197974 (0.0011) [2023-12-26 16:48:06,518][105620] Updated weights for policy 1, policy_version 197984 (0.0009) [2023-12-26 16:48:06,555][105692] Updated weights for policy 0, policy_version 197146 (0.0009) [2023-12-26 16:48:06,613][105692] Updated weights for policy 0, policy_version 197156 (0.0005) [2023-12-26 16:48:06,666][105692] Updated weights for policy 0, policy_version 197166 (0.0005) [2023-12-26 16:48:06,726][105692] Updated weights for policy 0, policy_version 197176 (0.0008) [2023-12-26 16:48:07,356][105620] Updated weights for policy 1, policy_version 197994 (0.0008) [2023-12-26 16:48:07,386][105692] Updated weights for policy 0, policy_version 197186 (0.0011) [2023-12-26 16:48:07,418][105620] Updated weights for policy 1, policy_version 198004 (0.0005) [2023-12-26 16:48:07,448][105692] Updated weights for policy 0, policy_version 197196 (0.0011) [2023-12-26 16:48:07,471][105620] Updated weights for policy 1, policy_version 198014 (0.0005) [2023-12-26 16:48:07,500][105692] Updated weights for policy 0, policy_version 197206 (0.0010) [2023-12-26 16:48:07,523][105620] Updated weights for policy 1, policy_version 198024 (0.0006) [2023-12-26 16:48:08,218][105620] Updated weights for policy 1, policy_version 198034 (0.0007) [2023-12-26 16:48:08,250][105692] Updated weights for policy 0, policy_version 197216 (0.0010) [2023-12-26 16:48:08,264][105620] Updated weights for policy 1, policy_version 198044 (0.0007) [2023-12-26 16:48:08,310][105620] Updated weights for policy 1, policy_version 198054 (0.0007) [2023-12-26 16:48:08,311][105692] Updated weights for policy 0, policy_version 197226 (0.0010) [2023-12-26 16:48:08,377][105692] Updated weights for policy 0, policy_version 197236 (0.0010) [2023-12-26 16:48:09,073][105620] Updated weights for policy 1, policy_version 198064 (0.0008) [2023-12-26 16:48:09,118][105692] Updated weights for policy 0, policy_version 197246 (0.0010) [2023-12-26 16:48:09,120][105620] Updated weights for policy 1, policy_version 198074 (0.0007) [2023-12-26 16:48:09,179][105692] Updated weights for policy 0, policy_version 197256 (0.0008) [2023-12-26 16:48:09,181][105620] Updated weights for policy 1, policy_version 198084 (0.0008) [2023-12-26 16:48:09,249][105692] Updated weights for policy 0, policy_version 197266 (0.0008) [2023-12-26 16:48:09,887][105692] Updated weights for policy 0, policy_version 197276 (0.0009) [2023-12-26 16:48:09,959][105692] Updated weights for policy 0, policy_version 197286 (0.0008) [2023-12-26 16:48:10,023][105692] Updated weights for policy 0, policy_version 197296 (0.0008) [2023-12-26 16:48:10,034][105620] Updated weights for policy 1, policy_version 198094 (0.0009) [2023-12-26 16:48:10,096][105620] Updated weights for policy 1, policy_version 198104 (0.0006) [2023-12-26 16:48:10,153][105620] Updated weights for policy 1, policy_version 198114 (0.0008) [2023-12-26 16:48:10,732][105692] Updated weights for policy 0, policy_version 197306 (0.0008) [2023-12-26 16:48:10,798][105692] Updated weights for policy 0, policy_version 197316 (0.0005) [2023-12-26 16:48:10,856][105692] Updated weights for policy 0, policy_version 197326 (0.0008) [2023-12-26 16:48:10,914][105692] Updated weights for policy 0, policy_version 197336 (0.0010) [2023-12-26 16:48:10,963][105620] Updated weights for policy 1, policy_version 198124 (0.0008) [2023-12-26 16:48:11,013][105620] Updated weights for policy 1, policy_version 198134 (0.0008) [2023-12-26 16:48:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 101253120. Throughput: 0: 9503.5, 1: 9870.0. Samples: 101262780. Policy #0 lag: (min: 23.0, avg: 31.8, max: 32.0) [2023-12-26 16:48:11,063][104569] Avg episode reward: [(0, '8206.388'), (1, '9172.983')] [2023-12-26 16:48:11,076][105620] Updated weights for policy 1, policy_version 198144 (0.0008) [2023-12-26 16:48:11,662][105692] Updated weights for policy 0, policy_version 197346 (0.0008) [2023-12-26 16:48:11,740][105692] Updated weights for policy 0, policy_version 197356 (0.0007) [2023-12-26 16:48:11,801][105692] Updated weights for policy 0, policy_version 197366 (0.0008) [2023-12-26 16:48:11,819][105620] Updated weights for policy 1, policy_version 198154 (0.0009) [2023-12-26 16:48:11,880][105620] Updated weights for policy 1, policy_version 198164 (0.0006) [2023-12-26 16:48:11,944][105620] Updated weights for policy 1, policy_version 198174 (0.0006) [2023-12-26 16:48:12,012][105620] Updated weights for policy 1, policy_version 198184 (0.0007) [2023-12-26 16:48:12,568][105692] Updated weights for policy 0, policy_version 197376 (0.0009) [2023-12-26 16:48:12,627][105692] Updated weights for policy 0, policy_version 197386 (0.0010) [2023-12-26 16:48:12,632][105620] Updated weights for policy 1, policy_version 198194 (0.0007) [2023-12-26 16:48:12,685][105692] Updated weights for policy 0, policy_version 197396 (0.0007) [2023-12-26 16:48:12,693][105620] Updated weights for policy 1, policy_version 198204 (0.0009) [2023-12-26 16:48:12,754][105620] Updated weights for policy 1, policy_version 198214 (0.0008) [2023-12-26 16:48:13,311][105692] Updated weights for policy 0, policy_version 197406 (0.0007) [2023-12-26 16:48:13,363][105620] Updated weights for policy 1, policy_version 198224 (0.0006) [2023-12-26 16:48:13,365][105692] Updated weights for policy 0, policy_version 197416 (0.0007) [2023-12-26 16:48:13,408][105620] Updated weights for policy 1, policy_version 198234 (0.0006) [2023-12-26 16:48:13,417][105692] Updated weights for policy 0, policy_version 197426 (0.0007) [2023-12-26 16:48:13,456][105620] Updated weights for policy 1, policy_version 198244 (0.0008) [2023-12-26 16:48:14,072][105692] Updated weights for policy 0, policy_version 197436 (0.0006) [2023-12-26 16:48:14,131][105692] Updated weights for policy 0, policy_version 197446 (0.0005) [2023-12-26 16:48:14,139][105620] Updated weights for policy 1, policy_version 198254 (0.0007) [2023-12-26 16:48:14,190][105692] Updated weights for policy 0, policy_version 197456 (0.0005) [2023-12-26 16:48:14,204][105620] Updated weights for policy 1, policy_version 198264 (0.0006) [2023-12-26 16:48:14,262][105620] Updated weights for policy 1, policy_version 198274 (0.0009) [2023-12-26 16:48:14,700][105692] Updated weights for policy 0, policy_version 197466 (0.0005) [2023-12-26 16:48:14,755][105692] Updated weights for policy 0, policy_version 197476 (0.0005) [2023-12-26 16:48:14,817][105692] Updated weights for policy 0, policy_version 197486 (0.0008) [2023-12-26 16:48:14,855][105620] Updated weights for policy 1, policy_version 198284 (0.0009) [2023-12-26 16:48:14,878][105692] Updated weights for policy 0, policy_version 197496 (0.0007) [2023-12-26 16:48:14,922][105620] Updated weights for policy 1, policy_version 198294 (0.0011) [2023-12-26 16:48:14,992][105620] Updated weights for policy 1, policy_version 198304 (0.0011) [2023-12-26 16:48:15,486][105692] Updated weights for policy 0, policy_version 197506 (0.0008) [2023-12-26 16:48:15,546][105692] Updated weights for policy 0, policy_version 197516 (0.0008) [2023-12-26 16:48:15,616][105692] Updated weights for policy 0, policy_version 197526 (0.0005) [2023-12-26 16:48:15,717][105620] Updated weights for policy 1, policy_version 198314 (0.0010) [2023-12-26 16:48:15,769][105620] Updated weights for policy 1, policy_version 198324 (0.0011) [2023-12-26 16:48:15,818][105620] Updated weights for policy 1, policy_version 198334 (0.0011) [2023-12-26 16:48:15,868][105620] Updated weights for policy 1, policy_version 198344 (0.0006) [2023-12-26 16:48:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 101359616. Throughput: 0: 9522.0, 1: 9904.7. Samples: 101322732. Policy #0 lag: (min: 23.0, avg: 31.8, max: 32.0) [2023-12-26 16:48:16,063][104569] Avg episode reward: [(0, '8481.159'), (1, '9355.043')] [2023-12-26 16:48:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000197528_50577408.pth... [2023-12-26 16:48:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000198344_50782208.pth... [2023-12-26 16:48:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000196408_50290688.pth [2023-12-26 16:48:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000197192_50487296.pth [2023-12-26 16:48:16,264][105692] Updated weights for policy 0, policy_version 197536 (0.0007) [2023-12-26 16:48:16,313][105692] Updated weights for policy 0, policy_version 197546 (0.0008) [2023-12-26 16:48:16,367][105692] Updated weights for policy 0, policy_version 197556 (0.0008) [2023-12-26 16:48:16,598][105620] Updated weights for policy 1, policy_version 198354 (0.0011) [2023-12-26 16:48:16,653][105620] Updated weights for policy 1, policy_version 198364 (0.0010) [2023-12-26 16:48:16,711][105620] Updated weights for policy 1, policy_version 198374 (0.0010) [2023-12-26 16:48:17,050][105692] Updated weights for policy 0, policy_version 197566 (0.0006) [2023-12-26 16:48:17,104][105692] Updated weights for policy 0, policy_version 197576 (0.0005) [2023-12-26 16:48:17,160][105692] Updated weights for policy 0, policy_version 197586 (0.0008) [2023-12-26 16:48:17,423][105620] Updated weights for policy 1, policy_version 198384 (0.0010) [2023-12-26 16:48:17,474][105620] Updated weights for policy 1, policy_version 198394 (0.0010) [2023-12-26 16:48:17,529][105620] Updated weights for policy 1, policy_version 198404 (0.0010) [2023-12-26 16:48:17,833][105692] Updated weights for policy 0, policy_version 197596 (0.0010) [2023-12-26 16:48:17,905][105692] Updated weights for policy 0, policy_version 197606 (0.0009) [2023-12-26 16:48:17,961][105692] Updated weights for policy 0, policy_version 197616 (0.0006) [2023-12-26 16:48:18,245][105620] Updated weights for policy 1, policy_version 198414 (0.0007) [2023-12-26 16:48:18,294][105620] Updated weights for policy 1, policy_version 198424 (0.0005) [2023-12-26 16:48:18,361][105620] Updated weights for policy 1, policy_version 198434 (0.0006) [2023-12-26 16:48:18,536][105692] Updated weights for policy 0, policy_version 197626 (0.0007) [2023-12-26 16:48:18,606][105692] Updated weights for policy 0, policy_version 197636 (0.0011) [2023-12-26 16:48:18,676][105692] Updated weights for policy 0, policy_version 197646 (0.0011) [2023-12-26 16:48:18,729][105692] Updated weights for policy 0, policy_version 197656 (0.0011) [2023-12-26 16:48:18,919][105620] Updated weights for policy 1, policy_version 198444 (0.0006) [2023-12-26 16:48:18,968][105620] Updated weights for policy 1, policy_version 198454 (0.0006) [2023-12-26 16:48:19,023][105620] Updated weights for policy 1, policy_version 198464 (0.0005) [2023-12-26 16:48:19,427][105692] Updated weights for policy 0, policy_version 197666 (0.0010) [2023-12-26 16:48:19,486][105692] Updated weights for policy 0, policy_version 197676 (0.0011) [2023-12-26 16:48:19,545][105692] Updated weights for policy 0, policy_version 197686 (0.0009) [2023-12-26 16:48:19,713][105620] Updated weights for policy 1, policy_version 198474 (0.0006) [2023-12-26 16:48:19,777][105620] Updated weights for policy 1, policy_version 198484 (0.0009) [2023-12-26 16:48:19,844][105620] Updated weights for policy 1, policy_version 198494 (0.0009) [2023-12-26 16:48:19,914][105620] Updated weights for policy 1, policy_version 198504 (0.0008) [2023-12-26 16:48:20,279][105692] Updated weights for policy 0, policy_version 197696 (0.0005) [2023-12-26 16:48:20,349][105692] Updated weights for policy 0, policy_version 197706 (0.0005) [2023-12-26 16:48:20,409][105692] Updated weights for policy 0, policy_version 197716 (0.0005) [2023-12-26 16:48:20,685][105620] Updated weights for policy 1, policy_version 198514 (0.0009) [2023-12-26 16:48:20,754][105620] Updated weights for policy 1, policy_version 198524 (0.0009) [2023-12-26 16:48:20,816][105620] Updated weights for policy 1, policy_version 198534 (0.0009) [2023-12-26 16:48:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 101457920. Throughput: 0: 9627.8, 1: 9875.6. Samples: 101448124. Policy #0 lag: (min: 23.0, avg: 31.8, max: 32.0) [2023-12-26 16:48:21,062][104569] Avg episode reward: [(0, '8732.168'), (1, '9082.362')] [2023-12-26 16:48:21,117][105692] Updated weights for policy 0, policy_version 197726 (0.0008) [2023-12-26 16:48:21,175][105692] Updated weights for policy 0, policy_version 197736 (0.0008) [2023-12-26 16:48:21,231][105692] Updated weights for policy 0, policy_version 197746 (0.0008) [2023-12-26 16:48:21,586][105620] Updated weights for policy 1, policy_version 198544 (0.0009) [2023-12-26 16:48:21,656][105620] Updated weights for policy 1, policy_version 198554 (0.0008) [2023-12-26 16:48:21,714][105620] Updated weights for policy 1, policy_version 198564 (0.0009) [2023-12-26 16:48:22,002][105692] Updated weights for policy 0, policy_version 197756 (0.0008) [2023-12-26 16:48:22,058][105692] Updated weights for policy 0, policy_version 197766 (0.0006) [2023-12-26 16:48:22,114][105692] Updated weights for policy 0, policy_version 197776 (0.0006) [2023-12-26 16:48:22,487][105620] Updated weights for policy 1, policy_version 198574 (0.0008) [2023-12-26 16:48:22,553][105620] Updated weights for policy 1, policy_version 198584 (0.0006) [2023-12-26 16:48:22,621][105620] Updated weights for policy 1, policy_version 198594 (0.0007) [2023-12-26 16:48:22,788][105692] Updated weights for policy 0, policy_version 197786 (0.0006) [2023-12-26 16:48:22,860][105692] Updated weights for policy 0, policy_version 197796 (0.0006) [2023-12-26 16:48:22,929][105692] Updated weights for policy 0, policy_version 197806 (0.0006) [2023-12-26 16:48:22,996][105692] Updated weights for policy 0, policy_version 197816 (0.0005) [2023-12-26 16:48:23,338][105620] Updated weights for policy 1, policy_version 198604 (0.0006) [2023-12-26 16:48:23,394][105620] Updated weights for policy 1, policy_version 198614 (0.0005) [2023-12-26 16:48:23,443][105620] Updated weights for policy 1, policy_version 198624 (0.0005) [2023-12-26 16:48:23,514][105692] Updated weights for policy 0, policy_version 197826 (0.0008) [2023-12-26 16:48:23,584][105692] Updated weights for policy 0, policy_version 197836 (0.0010) [2023-12-26 16:48:23,644][105692] Updated weights for policy 0, policy_version 197846 (0.0010) [2023-12-26 16:48:23,969][105620] Updated weights for policy 1, policy_version 198634 (0.0006) [2023-12-26 16:48:24,021][105620] Updated weights for policy 1, policy_version 198644 (0.0008) [2023-12-26 16:48:24,077][105620] Updated weights for policy 1, policy_version 198654 (0.0008) [2023-12-26 16:48:24,132][105620] Updated weights for policy 1, policy_version 198664 (0.0009) [2023-12-26 16:48:24,434][105692] Updated weights for policy 0, policy_version 197856 (0.0006) [2023-12-26 16:48:24,492][105692] Updated weights for policy 0, policy_version 197866 (0.0005) [2023-12-26 16:48:24,556][105692] Updated weights for policy 0, policy_version 197876 (0.0009) [2023-12-26 16:48:24,940][105620] Updated weights for policy 1, policy_version 198674 (0.0009) [2023-12-26 16:48:24,995][105620] Updated weights for policy 1, policy_version 198684 (0.0005) [2023-12-26 16:48:25,050][105620] Updated weights for policy 1, policy_version 198694 (0.0006) [2023-12-26 16:48:25,304][105692] Updated weights for policy 0, policy_version 197886 (0.0010) [2023-12-26 16:48:25,358][105692] Updated weights for policy 0, policy_version 197897 (0.0010) [2023-12-26 16:48:25,412][105692] Updated weights for policy 0, policy_version 197907 (0.0010) [2023-12-26 16:48:25,600][105620] Updated weights for policy 1, policy_version 198704 (0.0005) [2023-12-26 16:48:25,651][105620] Updated weights for policy 1, policy_version 198714 (0.0005) [2023-12-26 16:48:25,703][105620] Updated weights for policy 1, policy_version 198724 (0.0005) [2023-12-26 16:48:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 101556224. Throughput: 0: 9581.7, 1: 9974.8. Samples: 101565692. Policy #0 lag: (min: 23.0, avg: 31.8, max: 32.0) [2023-12-26 16:48:26,063][104569] Avg episode reward: [(0, '9081.381'), (1, '9174.260')] [2023-12-26 16:48:26,239][105692] Updated weights for policy 0, policy_version 197917 (0.0007) [2023-12-26 16:48:26,288][105692] Updated weights for policy 0, policy_version 197927 (0.0005) [2023-12-26 16:48:26,335][105692] Updated weights for policy 0, policy_version 197937 (0.0005) [2023-12-26 16:48:26,341][105620] Updated weights for policy 1, policy_version 198734 (0.0005) [2023-12-26 16:48:26,391][105620] Updated weights for policy 1, policy_version 198744 (0.0005) [2023-12-26 16:48:26,438][105620] Updated weights for policy 1, policy_version 198754 (0.0005) [2023-12-26 16:48:26,997][105620] Updated weights for policy 1, policy_version 198764 (0.0007) [2023-12-26 16:48:27,047][105692] Updated weights for policy 0, policy_version 197947 (0.0007) [2023-12-26 16:48:27,053][105620] Updated weights for policy 1, policy_version 198774 (0.0007) [2023-12-26 16:48:27,090][105692] Updated weights for policy 0, policy_version 197957 (0.0010) [2023-12-26 16:48:27,108][105620] Updated weights for policy 1, policy_version 198784 (0.0006) [2023-12-26 16:48:27,143][105692] Updated weights for policy 0, policy_version 197967 (0.0009) [2023-12-26 16:48:27,746][105692] Updated weights for policy 0, policy_version 197977 (0.0010) [2023-12-26 16:48:27,803][105692] Updated weights for policy 0, policy_version 197987 (0.0010) [2023-12-26 16:48:27,861][105692] Updated weights for policy 0, policy_version 197997 (0.0009) [2023-12-26 16:48:27,914][105692] Updated weights for policy 0, policy_version 198007 (0.0009) [2023-12-26 16:48:27,922][105620] Updated weights for policy 1, policy_version 198794 (0.0008) [2023-12-26 16:48:27,971][105620] Updated weights for policy 1, policy_version 198804 (0.0009) [2023-12-26 16:48:28,031][105620] Updated weights for policy 1, policy_version 198814 (0.0008) [2023-12-26 16:48:28,090][105620] Updated weights for policy 1, policy_version 198824 (0.0008) [2023-12-26 16:48:28,658][105692] Updated weights for policy 0, policy_version 198017 (0.0007) [2023-12-26 16:48:28,704][105692] Updated weights for policy 0, policy_version 198027 (0.0008) [2023-12-26 16:48:28,755][105692] Updated weights for policy 0, policy_version 198037 (0.0009) [2023-12-26 16:48:28,816][105620] Updated weights for policy 1, policy_version 198834 (0.0007) [2023-12-26 16:48:28,869][105620] Updated weights for policy 1, policy_version 198844 (0.0005) [2023-12-26 16:48:28,924][105620] Updated weights for policy 1, policy_version 198854 (0.0006) [2023-12-26 16:48:29,542][105620] Updated weights for policy 1, policy_version 198864 (0.0010) [2023-12-26 16:48:29,580][105692] Updated weights for policy 0, policy_version 198047 (0.0005) [2023-12-26 16:48:29,591][105620] Updated weights for policy 1, policy_version 198874 (0.0010) [2023-12-26 16:48:29,633][105692] Updated weights for policy 0, policy_version 198057 (0.0005) [2023-12-26 16:48:29,639][105620] Updated weights for policy 1, policy_version 198884 (0.0010) [2023-12-26 16:48:29,690][105692] Updated weights for policy 0, policy_version 198067 (0.0007) [2023-12-26 16:48:30,417][105620] Updated weights for policy 1, policy_version 198894 (0.0010) [2023-12-26 16:48:30,460][105692] Updated weights for policy 0, policy_version 198077 (0.0007) [2023-12-26 16:48:30,470][105620] Updated weights for policy 1, policy_version 198904 (0.0009) [2023-12-26 16:48:30,515][105620] Updated weights for policy 1, policy_version 198914 (0.0010) [2023-12-26 16:48:30,517][105692] Updated weights for policy 0, policy_version 198087 (0.0007) [2023-12-26 16:48:30,574][105692] Updated weights for policy 0, policy_version 198097 (0.0007) [2023-12-26 16:48:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 101654528. Throughput: 0: 9656.5, 1: 9978.0. Samples: 101626380. Policy #0 lag: (min: 23.0, avg: 31.8, max: 32.0) [2023-12-26 16:48:31,062][104569] Avg episode reward: [(0, '9350.414'), (1, '8990.409')] [2023-12-26 16:48:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000198104_50724864.pth... [2023-12-26 16:48:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000198920_50929664.pth... [2023-12-26 16:48:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000196952_50429952.pth [2023-12-26 16:48:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000197768_50634752.pth [2023-12-26 16:48:31,207][105620] Updated weights for policy 1, policy_version 198924 (0.0009) [2023-12-26 16:48:31,269][105620] Updated weights for policy 1, policy_version 198934 (0.0007) [2023-12-26 16:48:31,334][105620] Updated weights for policy 1, policy_version 198944 (0.0009) [2023-12-26 16:48:31,352][105692] Updated weights for policy 0, policy_version 198107 (0.0008) [2023-12-26 16:48:31,409][105692] Updated weights for policy 0, policy_version 198117 (0.0006) [2023-12-26 16:48:31,465][105692] Updated weights for policy 0, policy_version 198127 (0.0008) [2023-12-26 16:48:31,991][105620] Updated weights for policy 1, policy_version 198954 (0.0008) [2023-12-26 16:48:32,039][105620] Updated weights for policy 1, policy_version 198964 (0.0005) [2023-12-26 16:48:32,088][105620] Updated weights for policy 1, policy_version 198974 (0.0005) [2023-12-26 16:48:32,147][105620] Updated weights for policy 1, policy_version 198984 (0.0005) [2023-12-26 16:48:32,262][105692] Updated weights for policy 0, policy_version 198137 (0.0008) [2023-12-26 16:48:32,313][105692] Updated weights for policy 0, policy_version 198147 (0.0008) [2023-12-26 16:48:32,375][105692] Updated weights for policy 0, policy_version 198157 (0.0008) [2023-12-26 16:48:32,424][105692] Updated weights for policy 0, policy_version 198167 (0.0008) [2023-12-26 16:48:32,822][105620] Updated weights for policy 1, policy_version 198994 (0.0011) [2023-12-26 16:48:32,872][105620] Updated weights for policy 1, policy_version 199004 (0.0008) [2023-12-26 16:48:32,921][105620] Updated weights for policy 1, policy_version 199014 (0.0010) [2023-12-26 16:48:33,113][105692] Updated weights for policy 0, policy_version 198177 (0.0006) [2023-12-26 16:48:33,175][105692] Updated weights for policy 0, policy_version 198187 (0.0005) [2023-12-26 16:48:33,235][105692] Updated weights for policy 0, policy_version 198197 (0.0006) [2023-12-26 16:48:33,626][105620] Updated weights for policy 1, policy_version 199024 (0.0007) [2023-12-26 16:48:33,685][105620] Updated weights for policy 1, policy_version 199034 (0.0005) [2023-12-26 16:48:33,751][105620] Updated weights for policy 1, policy_version 199044 (0.0007) [2023-12-26 16:48:33,874][105692] Updated weights for policy 0, policy_version 198207 (0.0006) [2023-12-26 16:48:33,936][105692] Updated weights for policy 0, policy_version 198217 (0.0006) [2023-12-26 16:48:33,995][105692] Updated weights for policy 0, policy_version 198227 (0.0009) [2023-12-26 16:48:34,451][105620] Updated weights for policy 1, policy_version 199054 (0.0008) [2023-12-26 16:48:34,519][105620] Updated weights for policy 1, policy_version 199064 (0.0008) [2023-12-26 16:48:34,582][105620] Updated weights for policy 1, policy_version 199074 (0.0008) [2023-12-26 16:48:34,641][105692] Updated weights for policy 0, policy_version 198237 (0.0010) [2023-12-26 16:48:34,700][105692] Updated weights for policy 0, policy_version 198247 (0.0010) [2023-12-26 16:48:34,755][105692] Updated weights for policy 0, policy_version 198257 (0.0010) [2023-12-26 16:48:35,239][105620] Updated weights for policy 1, policy_version 199084 (0.0006) [2023-12-26 16:48:35,298][105620] Updated weights for policy 1, policy_version 199094 (0.0005) [2023-12-26 16:48:35,353][105620] Updated weights for policy 1, policy_version 199104 (0.0008) [2023-12-26 16:48:35,562][105692] Updated weights for policy 0, policy_version 198267 (0.0010) [2023-12-26 16:48:35,609][105692] Updated weights for policy 0, policy_version 198277 (0.0009) [2023-12-26 16:48:35,655][105692] Updated weights for policy 0, policy_version 198287 (0.0009) [2023-12-26 16:48:36,036][105620] Updated weights for policy 1, policy_version 199114 (0.0005) [2023-12-26 16:48:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 101752832. Throughput: 0: 9756.5, 1: 9954.9. Samples: 101744288. Policy #0 lag: (min: 23.0, avg: 31.8, max: 32.0) [2023-12-26 16:48:36,063][104569] Avg episode reward: [(0, '9352.021'), (1, '8989.760')] [2023-12-26 16:48:36,086][105620] Updated weights for policy 1, policy_version 199124 (0.0010) [2023-12-26 16:48:36,157][105620] Updated weights for policy 1, policy_version 199134 (0.0011) [2023-12-26 16:48:36,216][105620] Updated weights for policy 1, policy_version 199144 (0.0009) [2023-12-26 16:48:36,293][105692] Updated weights for policy 0, policy_version 198297 (0.0006) [2023-12-26 16:48:36,355][105692] Updated weights for policy 0, policy_version 198307 (0.0011) [2023-12-26 16:48:36,416][105692] Updated weights for policy 0, policy_version 198317 (0.0011) [2023-12-26 16:48:36,481][105692] Updated weights for policy 0, policy_version 198327 (0.0011) [2023-12-26 16:48:36,898][105620] Updated weights for policy 1, policy_version 199154 (0.0011) [2023-12-26 16:48:36,950][105620] Updated weights for policy 1, policy_version 199164 (0.0011) [2023-12-26 16:48:36,998][105620] Updated weights for policy 1, policy_version 199174 (0.0010) [2023-12-26 16:48:37,229][105692] Updated weights for policy 0, policy_version 198337 (0.0009) [2023-12-26 16:48:37,284][105692] Updated weights for policy 0, policy_version 198347 (0.0010) [2023-12-26 16:48:37,343][105692] Updated weights for policy 0, policy_version 198357 (0.0010) [2023-12-26 16:48:37,583][105620] Updated weights for policy 1, policy_version 199184 (0.0006) [2023-12-26 16:48:37,641][105620] Updated weights for policy 1, policy_version 199194 (0.0005) [2023-12-26 16:48:37,702][105620] Updated weights for policy 1, policy_version 199204 (0.0005) [2023-12-26 16:48:38,102][105692] Updated weights for policy 0, policy_version 198367 (0.0009) [2023-12-26 16:48:38,153][105692] Updated weights for policy 0, policy_version 198377 (0.0008) [2023-12-26 16:48:38,207][105692] Updated weights for policy 0, policy_version 198387 (0.0008) [2023-12-26 16:48:38,368][105620] Updated weights for policy 1, policy_version 199214 (0.0011) [2023-12-26 16:48:38,436][105620] Updated weights for policy 1, policy_version 199224 (0.0011) [2023-12-26 16:48:38,498][105620] Updated weights for policy 1, policy_version 199234 (0.0011) [2023-12-26 16:48:38,945][105692] Updated weights for policy 0, policy_version 198398 (0.0008) [2023-12-26 16:48:38,998][105692] Updated weights for policy 0, policy_version 198408 (0.0009) [2023-12-26 16:48:39,049][105692] Updated weights for policy 0, policy_version 198418 (0.0009) [2023-12-26 16:48:39,165][105620] Updated weights for policy 1, policy_version 199244 (0.0009) [2023-12-26 16:48:39,229][105620] Updated weights for policy 1, policy_version 199254 (0.0010) [2023-12-26 16:48:39,284][105620] Updated weights for policy 1, policy_version 199264 (0.0011) [2023-12-26 16:48:39,848][105692] Updated weights for policy 0, policy_version 198428 (0.0010) [2023-12-26 16:48:39,917][105692] Updated weights for policy 0, policy_version 198438 (0.0010) [2023-12-26 16:48:39,981][105692] Updated weights for policy 0, policy_version 198448 (0.0011) [2023-12-26 16:48:40,049][105620] Updated weights for policy 1, policy_version 199274 (0.0010) [2023-12-26 16:48:40,105][105620] Updated weights for policy 1, policy_version 199284 (0.0008) [2023-12-26 16:48:40,163][105620] Updated weights for policy 1, policy_version 199294 (0.0009) [2023-12-26 16:48:40,226][105620] Updated weights for policy 1, policy_version 199304 (0.0010) [2023-12-26 16:48:40,670][105692] Updated weights for policy 0, policy_version 198459 (0.0011) [2023-12-26 16:48:40,730][105692] Updated weights for policy 0, policy_version 198469 (0.0007) [2023-12-26 16:48:40,790][105692] Updated weights for policy 0, policy_version 198479 (0.0007) [2023-12-26 16:48:41,000][105620] Updated weights for policy 1, policy_version 199314 (0.0007) [2023-12-26 16:48:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 101851136. Throughput: 0: 9716.7, 1: 9996.5. Samples: 101861324. Policy #0 lag: (min: 23.0, avg: 31.8, max: 32.0) [2023-12-26 16:48:41,062][104569] Avg episode reward: [(0, '9354.009'), (1, '6908.601')] [2023-12-26 16:48:41,066][105620] Updated weights for policy 1, policy_version 199324 (0.0009) [2023-12-26 16:48:41,130][105620] Updated weights for policy 1, policy_version 199334 (0.0010) [2023-12-26 16:48:41,472][105692] Updated weights for policy 0, policy_version 198489 (0.0007) [2023-12-26 16:48:41,528][105692] Updated weights for policy 0, policy_version 198499 (0.0008) [2023-12-26 16:48:41,584][105692] Updated weights for policy 0, policy_version 198509 (0.0008) [2023-12-26 16:48:41,662][105692] Updated weights for policy 0, policy_version 198519 (0.0008) [2023-12-26 16:48:41,945][105620] Updated weights for policy 1, policy_version 199344 (0.0010) [2023-12-26 16:48:42,000][105620] Updated weights for policy 1, policy_version 199354 (0.0010) [2023-12-26 16:48:42,056][105620] Updated weights for policy 1, policy_version 199365 (0.0008) [2023-12-26 16:48:42,407][105692] Updated weights for policy 0, policy_version 198529 (0.0010) [2023-12-26 16:48:42,472][105692] Updated weights for policy 0, policy_version 198539 (0.0011) [2023-12-26 16:48:42,535][105692] Updated weights for policy 0, policy_version 198549 (0.0011) [2023-12-26 16:48:42,790][105620] Updated weights for policy 1, policy_version 199375 (0.0009) [2023-12-26 16:48:42,845][105620] Updated weights for policy 1, policy_version 199385 (0.0007) [2023-12-26 16:48:42,906][105620] Updated weights for policy 1, policy_version 199395 (0.0005) [2023-12-26 16:48:43,285][105692] Updated weights for policy 0, policy_version 198559 (0.0011) [2023-12-26 16:48:43,350][105692] Updated weights for policy 0, policy_version 198569 (0.0010) [2023-12-26 16:48:43,409][105692] Updated weights for policy 0, policy_version 198579 (0.0010) [2023-12-26 16:48:43,515][105620] Updated weights for policy 1, policy_version 199405 (0.0007) [2023-12-26 16:48:43,572][105620] Updated weights for policy 1, policy_version 199415 (0.0008) [2023-12-26 16:48:43,620][105620] Updated weights for policy 1, policy_version 199425 (0.0010) [2023-12-26 16:48:44,142][105692] Updated weights for policy 0, policy_version 198589 (0.0010) [2023-12-26 16:48:44,194][105692] Updated weights for policy 0, policy_version 198599 (0.0010) [2023-12-26 16:48:44,252][105692] Updated weights for policy 0, policy_version 198609 (0.0010) [2023-12-26 16:48:44,320][105620] Updated weights for policy 1, policy_version 199435 (0.0009) [2023-12-26 16:48:44,379][105620] Updated weights for policy 1, policy_version 199445 (0.0008) [2023-12-26 16:48:44,433][105620] Updated weights for policy 1, policy_version 199455 (0.0008) [2023-12-26 16:48:45,006][105692] Updated weights for policy 0, policy_version 198619 (0.0010) [2023-12-26 16:48:45,071][105692] Updated weights for policy 0, policy_version 198629 (0.0008) [2023-12-26 16:48:45,138][105692] Updated weights for policy 0, policy_version 198639 (0.0008) [2023-12-26 16:48:45,225][105620] Updated weights for policy 1, policy_version 199465 (0.0008) [2023-12-26 16:48:45,282][105620] Updated weights for policy 1, policy_version 199475 (0.0011) [2023-12-26 16:48:45,347][105620] Updated weights for policy 1, policy_version 199485 (0.0011) [2023-12-26 16:48:45,407][105620] Updated weights for policy 1, policy_version 199495 (0.0011) [2023-12-26 16:48:45,844][105692] Updated weights for policy 0, policy_version 198649 (0.0007) [2023-12-26 16:48:45,899][105692] Updated weights for policy 0, policy_version 198659 (0.0005) [2023-12-26 16:48:45,943][105692] Updated weights for policy 0, policy_version 198669 (0.0005) [2023-12-26 16:48:45,992][105692] Updated weights for policy 0, policy_version 198679 (0.0005) [2023-12-26 16:48:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 101949440. Throughput: 0: 9712.5, 1: 9990.5. Samples: 101919084. Policy #0 lag: (min: 23.0, avg: 31.8, max: 32.0) [2023-12-26 16:48:46,063][104569] Avg episode reward: [(0, '9176.613'), (1, '970.888')] [2023-12-26 16:48:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000198680_50872320.pth... [2023-12-26 16:48:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000199496_51077120.pth... [2023-12-26 16:48:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000197528_50577408.pth [2023-12-26 16:48:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000198344_50782208.pth [2023-12-26 16:48:46,174][105620] Updated weights for policy 1, policy_version 199505 (0.0011) [2023-12-26 16:48:46,227][105620] Updated weights for policy 1, policy_version 199515 (0.0010) [2023-12-26 16:48:46,271][105620] Updated weights for policy 1, policy_version 199525 (0.0010) [2023-12-26 16:48:46,544][105692] Updated weights for policy 0, policy_version 198689 (0.0005) [2023-12-26 16:48:46,607][105692] Updated weights for policy 0, policy_version 198699 (0.0005) [2023-12-26 16:48:46,670][105692] Updated weights for policy 0, policy_version 198709 (0.0005) [2023-12-26 16:48:47,028][105620] Updated weights for policy 1, policy_version 199535 (0.0007) [2023-12-26 16:48:47,092][105620] Updated weights for policy 1, policy_version 199545 (0.0011) [2023-12-26 16:48:47,155][105620] Updated weights for policy 1, policy_version 199555 (0.0011) [2023-12-26 16:48:47,217][105692] Updated weights for policy 0, policy_version 198719 (0.0006) [2023-12-26 16:48:47,279][105692] Updated weights for policy 0, policy_version 198729 (0.0010) [2023-12-26 16:48:47,326][105692] Updated weights for policy 0, policy_version 198739 (0.0010) [2023-12-26 16:48:47,880][105620] Updated weights for policy 1, policy_version 199565 (0.0011) [2023-12-26 16:48:47,938][105620] Updated weights for policy 1, policy_version 199575 (0.0008) [2023-12-26 16:48:47,956][105692] Updated weights for policy 0, policy_version 198749 (0.0009) [2023-12-26 16:48:48,009][105620] Updated weights for policy 1, policy_version 199585 (0.0006) [2023-12-26 16:48:48,023][105692] Updated weights for policy 0, policy_version 198759 (0.0005) [2023-12-26 16:48:48,076][105692] Updated weights for policy 0, policy_version 198769 (0.0005) [2023-12-26 16:48:48,689][105620] Updated weights for policy 1, policy_version 199595 (0.0007) [2023-12-26 16:48:48,740][105692] Updated weights for policy 0, policy_version 198779 (0.0010) [2023-12-26 16:48:48,752][105620] Updated weights for policy 1, policy_version 199605 (0.0011) [2023-12-26 16:48:48,785][105692] Updated weights for policy 0, policy_version 198789 (0.0010) [2023-12-26 16:48:48,812][105620] Updated weights for policy 1, policy_version 199615 (0.0011) [2023-12-26 16:48:48,834][105692] Updated weights for policy 0, policy_version 198799 (0.0011) [2023-12-26 16:48:49,482][105620] Updated weights for policy 1, policy_version 199625 (0.0011) [2023-12-26 16:48:49,546][105620] Updated weights for policy 1, policy_version 199635 (0.0011) [2023-12-26 16:48:49,609][105620] Updated weights for policy 1, policy_version 199645 (0.0011) [2023-12-26 16:48:49,616][105692] Updated weights for policy 0, policy_version 198809 (0.0011) [2023-12-26 16:48:49,669][105620] Updated weights for policy 1, policy_version 199655 (0.0011) [2023-12-26 16:48:49,676][105692] Updated weights for policy 0, policy_version 198819 (0.0008) [2023-12-26 16:48:49,728][105692] Updated weights for policy 0, policy_version 198829 (0.0010) [2023-12-26 16:48:49,790][105692] Updated weights for policy 0, policy_version 198839 (0.0010) [2023-12-26 16:48:50,375][105620] Updated weights for policy 1, policy_version 199665 (0.0007) [2023-12-26 16:48:50,444][105620] Updated weights for policy 1, policy_version 199675 (0.0009) [2023-12-26 16:48:50,503][105620] Updated weights for policy 1, policy_version 199685 (0.0010) [2023-12-26 16:48:50,508][105692] Updated weights for policy 0, policy_version 198849 (0.0006) [2023-12-26 16:48:50,558][105692] Updated weights for policy 0, policy_version 198859 (0.0006) [2023-12-26 16:48:50,626][105692] Updated weights for policy 0, policy_version 198869 (0.0008) [2023-12-26 16:48:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 102047744. Throughput: 0: 9922.3, 1: 9841.6. Samples: 102038088. Policy #0 lag: (min: 23.0, avg: 31.8, max: 32.0) [2023-12-26 16:48:51,062][104569] Avg episode reward: [(0, '9176.601'), (1, '1420.749')] [2023-12-26 16:48:51,217][105620] Updated weights for policy 1, policy_version 199695 (0.0011) [2023-12-26 16:48:51,282][105620] Updated weights for policy 1, policy_version 199705 (0.0011) [2023-12-26 16:48:51,294][105692] Updated weights for policy 0, policy_version 198879 (0.0009) [2023-12-26 16:48:51,334][105620] Updated weights for policy 1, policy_version 199715 (0.0011) [2023-12-26 16:48:51,348][105692] Updated weights for policy 0, policy_version 198889 (0.0008) [2023-12-26 16:48:51,414][105692] Updated weights for policy 0, policy_version 198899 (0.0007) [2023-12-26 16:48:52,112][105620] Updated weights for policy 1, policy_version 199725 (0.0010) [2023-12-26 16:48:52,158][105620] Updated weights for policy 1, policy_version 199735 (0.0010) [2023-12-26 16:48:52,189][105692] Updated weights for policy 0, policy_version 198909 (0.0007) [2023-12-26 16:48:52,203][105620] Updated weights for policy 1, policy_version 199745 (0.0010) [2023-12-26 16:48:52,249][105692] Updated weights for policy 0, policy_version 198919 (0.0006) [2023-12-26 16:48:52,312][105692] Updated weights for policy 0, policy_version 198929 (0.0008) [2023-12-26 16:48:52,938][105692] Updated weights for policy 0, policy_version 198939 (0.0007) [2023-12-26 16:48:52,991][105692] Updated weights for policy 0, policy_version 198949 (0.0008) [2023-12-26 16:48:52,994][105620] Updated weights for policy 1, policy_version 199755 (0.0011) [2023-12-26 16:48:53,049][105692] Updated weights for policy 0, policy_version 198959 (0.0005) [2023-12-26 16:48:53,056][105620] Updated weights for policy 1, policy_version 199765 (0.0010) [2023-12-26 16:48:53,115][105620] Updated weights for policy 1, policy_version 199775 (0.0010) [2023-12-26 16:48:53,592][105692] Updated weights for policy 0, policy_version 198969 (0.0005) [2023-12-26 16:48:53,649][105692] Updated weights for policy 0, policy_version 198979 (0.0008) [2023-12-26 16:48:53,712][105692] Updated weights for policy 0, policy_version 198989 (0.0005) [2023-12-26 16:48:53,775][105692] Updated weights for policy 0, policy_version 198999 (0.0005) [2023-12-26 16:48:53,855][105620] Updated weights for policy 1, policy_version 199785 (0.0010) [2023-12-26 16:48:53,920][105620] Updated weights for policy 1, policy_version 199795 (0.0010) [2023-12-26 16:48:53,968][105620] Updated weights for policy 1, policy_version 199805 (0.0010) [2023-12-26 16:48:54,027][105620] Updated weights for policy 1, policy_version 199815 (0.0010) [2023-12-26 16:48:54,334][105692] Updated weights for policy 0, policy_version 199009 (0.0005) [2023-12-26 16:48:54,396][105692] Updated weights for policy 0, policy_version 199019 (0.0008) [2023-12-26 16:48:54,473][105692] Updated weights for policy 0, policy_version 199029 (0.0008) [2023-12-26 16:48:54,775][105620] Updated weights for policy 1, policy_version 199825 (0.0010) [2023-12-26 16:48:54,830][105620] Updated weights for policy 1, policy_version 199835 (0.0010) [2023-12-26 16:48:54,892][105620] Updated weights for policy 1, policy_version 199845 (0.0010) [2023-12-26 16:48:55,200][105692] Updated weights for policy 0, policy_version 199039 (0.0009) [2023-12-26 16:48:55,262][105692] Updated weights for policy 0, policy_version 199049 (0.0006) [2023-12-26 16:48:55,311][105692] Updated weights for policy 0, policy_version 199059 (0.0010) [2023-12-26 16:48:55,644][105620] Updated weights for policy 1, policy_version 199855 (0.0010) [2023-12-26 16:48:55,706][105620] Updated weights for policy 1, policy_version 199865 (0.0011) [2023-12-26 16:48:55,761][105620] Updated weights for policy 1, policy_version 199875 (0.0010) [2023-12-26 16:48:55,837][105692] Updated weights for policy 0, policy_version 199069 (0.0005) [2023-12-26 16:48:55,899][105692] Updated weights for policy 0, policy_version 199079 (0.0005) [2023-12-26 16:48:55,958][105692] Updated weights for policy 0, policy_version 199089 (0.0005) [2023-12-26 16:48:56,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 102154240. Throughput: 0: 9993.4, 1: 9895.3. Samples: 102157772. Policy #0 lag: (min: 23.0, avg: 31.8, max: 32.0) [2023-12-26 16:48:56,062][104569] Avg episode reward: [(0, '9352.781'), (1, '3525.090')] [2023-12-26 16:48:56,486][105620] Updated weights for policy 1, policy_version 199885 (0.0008) [2023-12-26 16:48:56,551][105620] Updated weights for policy 1, policy_version 199895 (0.0006) [2023-12-26 16:48:56,570][105692] Updated weights for policy 0, policy_version 199099 (0.0005) [2023-12-26 16:48:56,596][105620] Updated weights for policy 1, policy_version 199905 (0.0010) [2023-12-26 16:48:56,622][105692] Updated weights for policy 0, policy_version 199109 (0.0006) [2023-12-26 16:48:56,669][105692] Updated weights for policy 0, policy_version 199119 (0.0010) [2023-12-26 16:48:57,263][105692] Updated weights for policy 0, policy_version 199129 (0.0010) [2023-12-26 16:48:57,297][105620] Updated weights for policy 1, policy_version 199915 (0.0010) [2023-12-26 16:48:57,312][105692] Updated weights for policy 0, policy_version 199139 (0.0005) [2023-12-26 16:48:57,356][105620] Updated weights for policy 1, policy_version 199925 (0.0009) [2023-12-26 16:48:57,364][105692] Updated weights for policy 0, policy_version 199149 (0.0010) [2023-12-26 16:48:57,414][105620] Updated weights for policy 1, policy_version 199935 (0.0010) [2023-12-26 16:48:57,419][105692] Updated weights for policy 0, policy_version 199159 (0.0010) [2023-12-26 16:48:58,084][105692] Updated weights for policy 0, policy_version 199169 (0.0009) [2023-12-26 16:48:58,150][105620] Updated weights for policy 1, policy_version 199945 (0.0010) [2023-12-26 16:48:58,151][105692] Updated weights for policy 0, policy_version 199179 (0.0010) [2023-12-26 16:48:58,207][105692] Updated weights for policy 0, policy_version 199189 (0.0008) [2023-12-26 16:48:58,211][105620] Updated weights for policy 1, policy_version 199955 (0.0007) [2023-12-26 16:48:58,261][105620] Updated weights for policy 1, policy_version 199965 (0.0006) [2023-12-26 16:48:58,330][105620] Updated weights for policy 1, policy_version 199975 (0.0008) [2023-12-26 16:48:58,950][105692] Updated weights for policy 0, policy_version 199199 (0.0007) [2023-12-26 16:48:59,013][105692] Updated weights for policy 0, policy_version 199209 (0.0007) [2023-12-26 16:48:59,076][105692] Updated weights for policy 0, policy_version 199219 (0.0009) [2023-12-26 16:48:59,100][105620] Updated weights for policy 1, policy_version 199985 (0.0006) [2023-12-26 16:48:59,153][105620] Updated weights for policy 1, policy_version 199995 (0.0008) [2023-12-26 16:48:59,202][105620] Updated weights for policy 1, policy_version 200005 (0.0008) [2023-12-26 16:48:59,808][105692] Updated weights for policy 0, policy_version 199229 (0.0006) [2023-12-26 16:48:59,875][105692] Updated weights for policy 0, policy_version 199239 (0.0009) [2023-12-26 16:48:59,934][105692] Updated weights for policy 0, policy_version 199249 (0.0009) [2023-12-26 16:48:59,981][105620] Updated weights for policy 1, policy_version 200015 (0.0008) [2023-12-26 16:49:00,038][105620] Updated weights for policy 1, policy_version 200025 (0.0005) [2023-12-26 16:49:00,089][105620] Updated weights for policy 1, policy_version 200035 (0.0005) [2023-12-26 16:49:00,689][105692] Updated weights for policy 0, policy_version 199259 (0.0006) [2023-12-26 16:49:00,728][105620] Updated weights for policy 1, policy_version 200045 (0.0007) [2023-12-26 16:49:00,745][105692] Updated weights for policy 0, policy_version 199269 (0.0005) [2023-12-26 16:49:00,787][105620] Updated weights for policy 1, policy_version 200055 (0.0009) [2023-12-26 16:49:00,803][105692] Updated weights for policy 0, policy_version 199279 (0.0008) [2023-12-26 16:49:00,839][105620] Updated weights for policy 1, policy_version 200065 (0.0007) [2023-12-26 16:49:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 102252544. Throughput: 0: 10070.7, 1: 9845.6. Samples: 102218964. Policy #0 lag: (min: 23.0, avg: 31.8, max: 32.0) [2023-12-26 16:49:01,062][104569] Avg episode reward: [(0, '9167.875'), (1, '2198.355')] [2023-12-26 16:49:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000199288_51027968.pth... [2023-12-26 16:49:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000200072_51224576.pth... [2023-12-26 16:49:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000198920_50929664.pth [2023-12-26 16:49:01,086][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000198104_50724864.pth [2023-12-26 16:49:01,532][105620] Updated weights for policy 1, policy_version 200075 (0.0008) [2023-12-26 16:49:01,544][105692] Updated weights for policy 0, policy_version 199289 (0.0009) [2023-12-26 16:49:01,594][105620] Updated weights for policy 1, policy_version 200085 (0.0005) [2023-12-26 16:49:01,597][105692] Updated weights for policy 0, policy_version 199299 (0.0008) [2023-12-26 16:49:01,658][105692] Updated weights for policy 0, policy_version 199309 (0.0009) [2023-12-26 16:49:01,660][105620] Updated weights for policy 1, policy_version 200095 (0.0008) [2023-12-26 16:49:01,719][105692] Updated weights for policy 0, policy_version 199319 (0.0010) [2023-12-26 16:49:02,282][105620] Updated weights for policy 1, policy_version 200105 (0.0006) [2023-12-26 16:49:02,340][105620] Updated weights for policy 1, policy_version 200115 (0.0010) [2023-12-26 16:49:02,421][105620] Updated weights for policy 1, policy_version 200125 (0.0011) [2023-12-26 16:49:02,480][105620] Updated weights for policy 1, policy_version 200135 (0.0010) [2023-12-26 16:49:02,539][105692] Updated weights for policy 0, policy_version 199329 (0.0008) [2023-12-26 16:49:02,598][105692] Updated weights for policy 0, policy_version 199339 (0.0007) [2023-12-26 16:49:02,655][105692] Updated weights for policy 0, policy_version 199349 (0.0007) [2023-12-26 16:49:03,130][105620] Updated weights for policy 1, policy_version 200145 (0.0006) [2023-12-26 16:49:03,178][105620] Updated weights for policy 1, policy_version 200155 (0.0007) [2023-12-26 16:49:03,225][105620] Updated weights for policy 1, policy_version 200165 (0.0005) [2023-12-26 16:49:03,346][105692] Updated weights for policy 0, policy_version 199359 (0.0006) [2023-12-26 16:49:03,405][105692] Updated weights for policy 0, policy_version 199369 (0.0007) [2023-12-26 16:49:03,460][105692] Updated weights for policy 0, policy_version 199379 (0.0008) [2023-12-26 16:49:03,789][105620] Updated weights for policy 1, policy_version 200175 (0.0005) [2023-12-26 16:49:03,843][105620] Updated weights for policy 1, policy_version 200185 (0.0006) [2023-12-26 16:49:03,894][105620] Updated weights for policy 1, policy_version 200195 (0.0007) [2023-12-26 16:49:04,288][105692] Updated weights for policy 0, policy_version 199389 (0.0009) [2023-12-26 16:49:04,349][105692] Updated weights for policy 0, policy_version 199399 (0.0009) [2023-12-26 16:49:04,406][105692] Updated weights for policy 0, policy_version 199409 (0.0008) [2023-12-26 16:49:04,525][105620] Updated weights for policy 1, policy_version 200205 (0.0008) [2023-12-26 16:49:04,592][105620] Updated weights for policy 1, policy_version 200215 (0.0010) [2023-12-26 16:49:04,645][105620] Updated weights for policy 1, policy_version 200225 (0.0010) [2023-12-26 16:49:05,171][105692] Updated weights for policy 0, policy_version 199419 (0.0008) [2023-12-26 16:49:05,216][105692] Updated weights for policy 0, policy_version 199429 (0.0008) [2023-12-26 16:49:05,268][105692] Updated weights for policy 0, policy_version 199439 (0.0008) [2023-12-26 16:49:05,389][105620] Updated weights for policy 1, policy_version 200235 (0.0010) [2023-12-26 16:49:05,444][105620] Updated weights for policy 1, policy_version 200245 (0.0010) [2023-12-26 16:49:05,505][105620] Updated weights for policy 1, policy_version 200255 (0.0010) [2023-12-26 16:49:06,055][105692] Updated weights for policy 0, policy_version 199449 (0.0008) [2023-12-26 16:49:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 102342656. Throughput: 0: 9866.0, 1: 9876.3. Samples: 102336524. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-26 16:49:06,062][104569] Avg episode reward: [(0, '9076.327'), (1, '6439.275')] [2023-12-26 16:49:06,118][105692] Updated weights for policy 0, policy_version 199459 (0.0008) [2023-12-26 16:49:06,180][105692] Updated weights for policy 0, policy_version 199469 (0.0008) [2023-12-26 16:49:06,245][105692] Updated weights for policy 0, policy_version 199479 (0.0008) [2023-12-26 16:49:06,246][105620] Updated weights for policy 1, policy_version 200265 (0.0010) [2023-12-26 16:49:06,300][105620] Updated weights for policy 1, policy_version 200275 (0.0007) [2023-12-26 16:49:06,352][105620] Updated weights for policy 1, policy_version 200285 (0.0009) [2023-12-26 16:49:06,405][105620] Updated weights for policy 1, policy_version 200295 (0.0010) [2023-12-26 16:49:07,009][105692] Updated weights for policy 0, policy_version 199489 (0.0008) [2023-12-26 16:49:07,058][105692] Updated weights for policy 0, policy_version 199499 (0.0008) [2023-12-26 16:49:07,119][105692] Updated weights for policy 0, policy_version 199509 (0.0008) [2023-12-26 16:49:07,164][105620] Updated weights for policy 1, policy_version 200305 (0.0011) [2023-12-26 16:49:07,214][105620] Updated weights for policy 1, policy_version 200315 (0.0010) [2023-12-26 16:49:07,276][105620] Updated weights for policy 1, policy_version 200325 (0.0010) [2023-12-26 16:49:07,908][105692] Updated weights for policy 0, policy_version 199519 (0.0007) [2023-12-26 16:49:07,959][105692] Updated weights for policy 0, policy_version 199529 (0.0008) [2023-12-26 16:49:08,010][105692] Updated weights for policy 0, policy_version 199539 (0.0007) [2023-12-26 16:49:08,027][105620] Updated weights for policy 1, policy_version 200335 (0.0010) [2023-12-26 16:49:08,089][105620] Updated weights for policy 1, policy_version 200345 (0.0010) [2023-12-26 16:49:08,159][105620] Updated weights for policy 1, policy_version 200355 (0.0011) [2023-12-26 16:49:08,731][105692] Updated weights for policy 0, policy_version 199549 (0.0007) [2023-12-26 16:49:08,788][105692] Updated weights for policy 0, policy_version 199559 (0.0008) [2023-12-26 16:49:08,837][105620] Updated weights for policy 1, policy_version 200365 (0.0011) [2023-12-26 16:49:08,847][105692] Updated weights for policy 0, policy_version 199569 (0.0009) [2023-12-26 16:49:08,897][105620] Updated weights for policy 1, policy_version 200375 (0.0011) [2023-12-26 16:49:08,956][105620] Updated weights for policy 1, policy_version 200385 (0.0011) [2023-12-26 16:49:09,629][105620] Updated weights for policy 1, policy_version 200395 (0.0010) [2023-12-26 16:49:09,651][105692] Updated weights for policy 0, policy_version 199579 (0.0007) [2023-12-26 16:49:09,683][105620] Updated weights for policy 1, policy_version 200405 (0.0007) [2023-12-26 16:49:09,709][105692] Updated weights for policy 0, policy_version 199589 (0.0009) [2023-12-26 16:49:09,740][105620] Updated weights for policy 1, policy_version 200415 (0.0007) [2023-12-26 16:49:09,771][105692] Updated weights for policy 0, policy_version 199599 (0.0006) [2023-12-26 16:49:10,490][105692] Updated weights for policy 0, policy_version 199609 (0.0008) [2023-12-26 16:49:10,534][105620] Updated weights for policy 1, policy_version 200425 (0.0008) [2023-12-26 16:49:10,548][105692] Updated weights for policy 0, policy_version 199619 (0.0007) [2023-12-26 16:49:10,593][105620] Updated weights for policy 1, policy_version 200435 (0.0010) [2023-12-26 16:49:10,607][105692] Updated weights for policy 0, policy_version 199629 (0.0008) [2023-12-26 16:49:10,645][105620] Updated weights for policy 1, policy_version 200445 (0.0010) [2023-12-26 16:49:10,667][105692] Updated weights for policy 0, policy_version 199639 (0.0005) [2023-12-26 16:49:10,690][105620] Updated weights for policy 1, policy_version 200455 (0.0010) [2023-12-26 16:49:11,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 102440960. Throughput: 0: 9816.7, 1: 9821.0. Samples: 102449388. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-26 16:49:11,063][104569] Avg episode reward: [(0, '9257.922'), (1, '8999.896')] [2023-12-26 16:49:11,418][105620] Updated weights for policy 1, policy_version 200465 (0.0008) [2023-12-26 16:49:11,474][105620] Updated weights for policy 1, policy_version 200475 (0.0006) [2023-12-26 16:49:11,534][105692] Updated weights for policy 0, policy_version 199649 (0.0006) [2023-12-26 16:49:11,536][105620] Updated weights for policy 1, policy_version 200485 (0.0011) [2023-12-26 16:49:11,593][105692] Updated weights for policy 0, policy_version 199659 (0.0007) [2023-12-26 16:49:11,659][105692] Updated weights for policy 0, policy_version 199669 (0.0009) [2023-12-26 16:49:12,295][105620] Updated weights for policy 1, policy_version 200495 (0.0012) [2023-12-26 16:49:12,359][105620] Updated weights for policy 1, policy_version 200505 (0.0009) [2023-12-26 16:49:12,422][105620] Updated weights for policy 1, policy_version 200515 (0.0011) [2023-12-26 16:49:12,452][105692] Updated weights for policy 0, policy_version 199679 (0.0010) [2023-12-26 16:49:12,513][105692] Updated weights for policy 0, policy_version 199689 (0.0010) [2023-12-26 16:49:12,574][105692] Updated weights for policy 0, policy_version 199699 (0.0011) [2023-12-26 16:49:13,169][105620] Updated weights for policy 1, policy_version 200525 (0.0009) [2023-12-26 16:49:13,234][105620] Updated weights for policy 1, policy_version 200535 (0.0010) [2023-12-26 16:49:13,300][105620] Updated weights for policy 1, policy_version 200545 (0.0008) [2023-12-26 16:49:13,321][105692] Updated weights for policy 0, policy_version 199709 (0.0008) [2023-12-26 16:49:13,380][105692] Updated weights for policy 0, policy_version 199719 (0.0005) [2023-12-26 16:49:13,438][105692] Updated weights for policy 0, policy_version 199729 (0.0005) [2023-12-26 16:49:13,918][105620] Updated weights for policy 1, policy_version 200555 (0.0010) [2023-12-26 16:49:13,987][105620] Updated weights for policy 1, policy_version 200565 (0.0010) [2023-12-26 16:49:14,048][105620] Updated weights for policy 1, policy_version 200575 (0.0010) [2023-12-26 16:49:14,069][105692] Updated weights for policy 0, policy_version 199739 (0.0009) [2023-12-26 16:49:14,123][105692] Updated weights for policy 0, policy_version 199749 (0.0010) [2023-12-26 16:49:14,195][105692] Updated weights for policy 0, policy_version 199759 (0.0006) [2023-12-26 16:49:14,619][105620] Updated weights for policy 1, policy_version 200585 (0.0010) [2023-12-26 16:49:14,681][105620] Updated weights for policy 1, policy_version 200595 (0.0006) [2023-12-26 16:49:14,745][105620] Updated weights for policy 1, policy_version 200605 (0.0008) [2023-12-26 16:49:14,812][105620] Updated weights for policy 1, policy_version 200615 (0.0008) [2023-12-26 16:49:14,941][105692] Updated weights for policy 0, policy_version 199769 (0.0008) [2023-12-26 16:49:15,012][105692] Updated weights for policy 0, policy_version 199779 (0.0006) [2023-12-26 16:49:15,084][105692] Updated weights for policy 0, policy_version 199789 (0.0006) [2023-12-26 16:49:15,145][105692] Updated weights for policy 0, policy_version 199799 (0.0006) [2023-12-26 16:49:15,420][105620] Updated weights for policy 1, policy_version 200625 (0.0006) [2023-12-26 16:49:15,474][105620] Updated weights for policy 1, policy_version 200635 (0.0005) [2023-12-26 16:49:15,523][105620] Updated weights for policy 1, policy_version 200645 (0.0005) [2023-12-26 16:49:15,930][105692] Updated weights for policy 0, policy_version 199809 (0.0007) [2023-12-26 16:49:15,994][105692] Updated weights for policy 0, policy_version 199819 (0.0010) [2023-12-26 16:49:16,059][105692] Updated weights for policy 0, policy_version 199829 (0.0010) [2023-12-26 16:49:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 102531072. Throughput: 0: 9740.9, 1: 9803.4. Samples: 102505872. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-26 16:49:16,062][104569] Avg episode reward: [(0, '9174.845'), (1, '9262.327')] [2023-12-26 16:49:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000199832_51167232.pth... [2023-12-26 16:49:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000198680_50872320.pth [2023-12-26 16:49:16,103][105620] Updated weights for policy 1, policy_version 200655 (0.0008) [2023-12-26 16:49:16,150][105620] Updated weights for policy 1, policy_version 200665 (0.0007) [2023-12-26 16:49:16,195][105620] Updated weights for policy 1, policy_version 200675 (0.0008) [2023-12-26 16:49:16,219][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000200680_51380224.pth... [2023-12-26 16:49:16,224][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000199496_51077120.pth [2023-12-26 16:49:16,775][105692] Updated weights for policy 0, policy_version 199839 (0.0008) [2023-12-26 16:49:16,820][105692] Updated weights for policy 0, policy_version 199849 (0.0005) [2023-12-26 16:49:16,871][105692] Updated weights for policy 0, policy_version 199859 (0.0005) [2023-12-26 16:49:16,933][105620] Updated weights for policy 1, policy_version 200685 (0.0009) [2023-12-26 16:49:17,001][105620] Updated weights for policy 1, policy_version 200695 (0.0008) [2023-12-26 16:49:17,056][105620] Updated weights for policy 1, policy_version 200705 (0.0009) [2023-12-26 16:49:17,423][105692] Updated weights for policy 0, policy_version 199869 (0.0009) [2023-12-26 16:49:17,481][105692] Updated weights for policy 0, policy_version 199879 (0.0010) [2023-12-26 16:49:17,538][105692] Updated weights for policy 0, policy_version 199889 (0.0010) [2023-12-26 16:49:17,664][105620] Updated weights for policy 1, policy_version 200715 (0.0008) [2023-12-26 16:49:17,719][105620] Updated weights for policy 1, policy_version 200725 (0.0007) [2023-12-26 16:49:17,768][105620] Updated weights for policy 1, policy_version 200735 (0.0006) [2023-12-26 16:49:18,181][105692] Updated weights for policy 0, policy_version 199899 (0.0010) [2023-12-26 16:49:18,228][105692] Updated weights for policy 0, policy_version 199909 (0.0010) [2023-12-26 16:49:18,293][105692] Updated weights for policy 0, policy_version 199919 (0.0010) [2023-12-26 16:49:18,553][105620] Updated weights for policy 1, policy_version 200745 (0.0008) [2023-12-26 16:49:18,621][105620] Updated weights for policy 1, policy_version 200755 (0.0008) [2023-12-26 16:49:18,680][105620] Updated weights for policy 1, policy_version 200765 (0.0010) [2023-12-26 16:49:18,741][105620] Updated weights for policy 1, policy_version 200775 (0.0010) [2023-12-26 16:49:19,054][105692] Updated weights for policy 0, policy_version 199929 (0.0010) [2023-12-26 16:49:19,118][105692] Updated weights for policy 0, policy_version 199939 (0.0008) [2023-12-26 16:49:19,166][105692] Updated weights for policy 0, policy_version 199949 (0.0008) [2023-12-26 16:49:19,217][105692] Updated weights for policy 0, policy_version 199959 (0.0008) [2023-12-26 16:49:19,486][105620] Updated weights for policy 1, policy_version 200785 (0.0011) [2023-12-26 16:49:19,546][105620] Updated weights for policy 1, policy_version 200795 (0.0011) [2023-12-26 16:49:19,598][105620] Updated weights for policy 1, policy_version 200805 (0.0011) [2023-12-26 16:49:20,011][105692] Updated weights for policy 0, policy_version 199969 (0.0006) [2023-12-26 16:49:20,059][105692] Updated weights for policy 0, policy_version 199979 (0.0005) [2023-12-26 16:49:20,116][105692] Updated weights for policy 0, policy_version 199989 (0.0006) [2023-12-26 16:49:20,357][105620] Updated weights for policy 1, policy_version 200815 (0.0011) [2023-12-26 16:49:20,417][105620] Updated weights for policy 1, policy_version 200825 (0.0011) [2023-12-26 16:49:20,486][105620] Updated weights for policy 1, policy_version 200835 (0.0011) [2023-12-26 16:49:20,793][105692] Updated weights for policy 0, policy_version 199999 (0.0008) [2023-12-26 16:49:20,853][105692] Updated weights for policy 0, policy_version 200009 (0.0009) [2023-12-26 16:49:20,911][105692] Updated weights for policy 0, policy_version 200019 (0.0010) [2023-12-26 16:49:21,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 102637568. Throughput: 0: 9777.6, 1: 9819.7. Samples: 102626164. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-26 16:49:21,062][104569] Avg episode reward: [(0, '9264.707'), (1, '9086.508')] [2023-12-26 16:49:21,268][105620] Updated weights for policy 1, policy_version 200845 (0.0010) [2023-12-26 16:49:21,330][105620] Updated weights for policy 1, policy_version 200855 (0.0008) [2023-12-26 16:49:21,402][105620] Updated weights for policy 1, policy_version 200865 (0.0010) [2023-12-26 16:49:21,628][105692] Updated weights for policy 0, policy_version 200029 (0.0010) [2023-12-26 16:49:21,690][105692] Updated weights for policy 0, policy_version 200039 (0.0010) [2023-12-26 16:49:21,756][105692] Updated weights for policy 0, policy_version 200049 (0.0010) [2023-12-26 16:49:22,194][105620] Updated weights for policy 1, policy_version 200875 (0.0008) [2023-12-26 16:49:22,240][105620] Updated weights for policy 1, policy_version 200885 (0.0006) [2023-12-26 16:49:22,302][105620] Updated weights for policy 1, policy_version 200895 (0.0007) [2023-12-26 16:49:22,339][105692] Updated weights for policy 0, policy_version 200059 (0.0008) [2023-12-26 16:49:22,404][105692] Updated weights for policy 0, policy_version 200069 (0.0009) [2023-12-26 16:49:22,466][105692] Updated weights for policy 0, policy_version 200079 (0.0010) [2023-12-26 16:49:22,935][105620] Updated weights for policy 1, policy_version 200905 (0.0007) [2023-12-26 16:49:22,999][105620] Updated weights for policy 1, policy_version 200915 (0.0007) [2023-12-26 16:49:23,063][105620] Updated weights for policy 1, policy_version 200925 (0.0008) [2023-12-26 16:49:23,117][105620] Updated weights for policy 1, policy_version 200935 (0.0008) [2023-12-26 16:49:23,171][105692] Updated weights for policy 0, policy_version 200089 (0.0011) [2023-12-26 16:49:23,240][105692] Updated weights for policy 0, policy_version 200099 (0.0011) [2023-12-26 16:49:23,295][105692] Updated weights for policy 0, policy_version 200109 (0.0010) [2023-12-26 16:49:23,347][105692] Updated weights for policy 0, policy_version 200119 (0.0010) [2023-12-26 16:49:23,796][105620] Updated weights for policy 1, policy_version 200945 (0.0008) [2023-12-26 16:49:23,861][105620] Updated weights for policy 1, policy_version 200955 (0.0008) [2023-12-26 16:49:23,910][105620] Updated weights for policy 1, policy_version 200965 (0.0009) [2023-12-26 16:49:24,060][105692] Updated weights for policy 0, policy_version 200129 (0.0006) [2023-12-26 16:49:24,116][105692] Updated weights for policy 0, policy_version 200139 (0.0008) [2023-12-26 16:49:24,175][105692] Updated weights for policy 0, policy_version 200149 (0.0009) [2023-12-26 16:49:24,676][105620] Updated weights for policy 1, policy_version 200975 (0.0009) [2023-12-26 16:49:24,728][105620] Updated weights for policy 1, policy_version 200985 (0.0008) [2023-12-26 16:49:24,793][105620] Updated weights for policy 1, policy_version 200995 (0.0008) [2023-12-26 16:49:24,880][105692] Updated weights for policy 0, policy_version 200159 (0.0007) [2023-12-26 16:49:24,941][105692] Updated weights for policy 0, policy_version 200169 (0.0005) [2023-12-26 16:49:25,001][105692] Updated weights for policy 0, policy_version 200179 (0.0008) [2023-12-26 16:49:25,557][105620] Updated weights for policy 1, policy_version 201005 (0.0009) [2023-12-26 16:49:25,610][105620] Updated weights for policy 1, policy_version 201015 (0.0008) [2023-12-26 16:49:25,664][105620] Updated weights for policy 1, policy_version 201025 (0.0009) [2023-12-26 16:49:25,715][105692] Updated weights for policy 0, policy_version 200189 (0.0007) [2023-12-26 16:49:25,771][105692] Updated weights for policy 0, policy_version 200199 (0.0005) [2023-12-26 16:49:25,837][105692] Updated weights for policy 0, policy_version 200209 (0.0009) [2023-12-26 16:49:26,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 102735872. Throughput: 0: 9840.1, 1: 9730.3. Samples: 102742000. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-26 16:49:26,063][104569] Avg episode reward: [(0, '9348.461'), (1, '8907.287')] [2023-12-26 16:49:26,309][105620] Updated weights for policy 1, policy_version 201035 (0.0008) [2023-12-26 16:49:26,364][105620] Updated weights for policy 1, policy_version 201045 (0.0005) [2023-12-26 16:49:26,430][105620] Updated weights for policy 1, policy_version 201055 (0.0008) [2023-12-26 16:49:26,605][105692] Updated weights for policy 0, policy_version 200219 (0.0008) [2023-12-26 16:49:26,658][105692] Updated weights for policy 0, policy_version 200229 (0.0006) [2023-12-26 16:49:26,707][105692] Updated weights for policy 0, policy_version 200239 (0.0005) [2023-12-26 16:49:27,237][105620] Updated weights for policy 1, policy_version 201065 (0.0009) [2023-12-26 16:49:27,239][105692] Updated weights for policy 0, policy_version 200249 (0.0006) [2023-12-26 16:49:27,297][105692] Updated weights for policy 0, policy_version 200259 (0.0007) [2023-12-26 16:49:27,298][105620] Updated weights for policy 1, policy_version 201075 (0.0008) [2023-12-26 16:49:27,350][105620] Updated weights for policy 1, policy_version 201085 (0.0005) [2023-12-26 16:49:27,356][105692] Updated weights for policy 0, policy_version 200269 (0.0010) [2023-12-26 16:49:27,397][105620] Updated weights for policy 1, policy_version 201095 (0.0006) [2023-12-26 16:49:27,417][105692] Updated weights for policy 0, policy_version 200279 (0.0010) [2023-12-26 16:49:27,966][105692] Updated weights for policy 0, policy_version 200289 (0.0006) [2023-12-26 16:49:28,016][105692] Updated weights for policy 0, policy_version 200299 (0.0008) [2023-12-26 16:49:28,062][105692] Updated weights for policy 0, policy_version 200309 (0.0008) [2023-12-26 16:49:28,216][105620] Updated weights for policy 1, policy_version 201105 (0.0010) [2023-12-26 16:49:28,265][105620] Updated weights for policy 1, policy_version 201115 (0.0009) [2023-12-26 16:49:28,314][105620] Updated weights for policy 1, policy_version 201125 (0.0008) [2023-12-26 16:49:28,754][105692] Updated weights for policy 0, policy_version 200319 (0.0007) [2023-12-26 16:49:28,821][105692] Updated weights for policy 0, policy_version 200329 (0.0005) [2023-12-26 16:49:28,880][105692] Updated weights for policy 0, policy_version 200339 (0.0007) [2023-12-26 16:49:29,103][105620] Updated weights for policy 1, policy_version 201135 (0.0009) [2023-12-26 16:49:29,154][105620] Updated weights for policy 1, policy_version 201145 (0.0008) [2023-12-26 16:49:29,205][105620] Updated weights for policy 1, policy_version 201155 (0.0008) [2023-12-26 16:49:29,495][105692] Updated weights for policy 0, policy_version 200349 (0.0008) [2023-12-26 16:49:29,554][105692] Updated weights for policy 0, policy_version 200359 (0.0010) [2023-12-26 16:49:29,610][105692] Updated weights for policy 0, policy_version 200369 (0.0011) [2023-12-26 16:49:30,036][105620] Updated weights for policy 1, policy_version 201165 (0.0008) [2023-12-26 16:49:30,090][105620] Updated weights for policy 1, policy_version 201175 (0.0010) [2023-12-26 16:49:30,149][105620] Updated weights for policy 1, policy_version 201185 (0.0008) [2023-12-26 16:49:30,292][105692] Updated weights for policy 0, policy_version 200379 (0.0010) [2023-12-26 16:49:30,339][105692] Updated weights for policy 0, policy_version 200389 (0.0009) [2023-12-26 16:49:30,400][105692] Updated weights for policy 0, policy_version 200399 (0.0009) [2023-12-26 16:49:30,895][105620] Updated weights for policy 1, policy_version 201195 (0.0009) [2023-12-26 16:49:30,952][105620] Updated weights for policy 1, policy_version 201205 (0.0008) [2023-12-26 16:49:31,015][105620] Updated weights for policy 1, policy_version 201215 (0.0008) [2023-12-26 16:49:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 102825984. Throughput: 0: 9931.3, 1: 9709.1. Samples: 102802900. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-26 16:49:31,063][104569] Avg episode reward: [(0, '9350.724'), (1, '4534.069')] [2023-12-26 16:49:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000200408_51314688.pth... [2023-12-26 16:49:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000199288_51027968.pth [2023-12-26 16:49:31,075][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000201224_51519488.pth... [2023-12-26 16:49:31,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000200072_51224576.pth [2023-12-26 16:49:31,149][105692] Updated weights for policy 0, policy_version 200409 (0.0009) [2023-12-26 16:49:31,199][105692] Updated weights for policy 0, policy_version 200419 (0.0008) [2023-12-26 16:49:31,250][105692] Updated weights for policy 0, policy_version 200429 (0.0008) [2023-12-26 16:49:31,318][105692] Updated weights for policy 0, policy_version 200439 (0.0006) [2023-12-26 16:49:31,796][105620] Updated weights for policy 1, policy_version 201225 (0.0008) [2023-12-26 16:49:31,851][105620] Updated weights for policy 1, policy_version 201235 (0.0008) [2023-12-26 16:49:31,901][105620] Updated weights for policy 1, policy_version 201245 (0.0008) [2023-12-26 16:49:31,959][105620] Updated weights for policy 1, policy_version 201255 (0.0007) [2023-12-26 16:49:32,020][105692] Updated weights for policy 0, policy_version 200449 (0.0006) [2023-12-26 16:49:32,067][105692] Updated weights for policy 0, policy_version 200459 (0.0008) [2023-12-26 16:49:32,113][105692] Updated weights for policy 0, policy_version 200469 (0.0008) [2023-12-26 16:49:32,751][105620] Updated weights for policy 1, policy_version 201265 (0.0008) [2023-12-26 16:49:32,809][105692] Updated weights for policy 0, policy_version 200479 (0.0010) [2023-12-26 16:49:32,811][105620] Updated weights for policy 1, policy_version 201275 (0.0006) [2023-12-26 16:49:32,859][105620] Updated weights for policy 1, policy_version 201285 (0.0006) [2023-12-26 16:49:32,861][105692] Updated weights for policy 0, policy_version 200489 (0.0010) [2023-12-26 16:49:32,919][105692] Updated weights for policy 0, policy_version 200499 (0.0010) [2023-12-26 16:49:33,563][105692] Updated weights for policy 0, policy_version 200509 (0.0010) [2023-12-26 16:49:33,615][105692] Updated weights for policy 0, policy_version 200519 (0.0008) [2023-12-26 16:49:33,637][105620] Updated weights for policy 1, policy_version 201295 (0.0008) [2023-12-26 16:49:33,663][105692] Updated weights for policy 0, policy_version 200529 (0.0007) [2023-12-26 16:49:33,689][105620] Updated weights for policy 1, policy_version 201305 (0.0007) [2023-12-26 16:49:33,741][105620] Updated weights for policy 1, policy_version 201315 (0.0008) [2023-12-26 16:49:34,414][105692] Updated weights for policy 0, policy_version 200539 (0.0006) [2023-12-26 16:49:34,469][105692] Updated weights for policy 0, policy_version 200549 (0.0009) [2023-12-26 16:49:34,517][105620] Updated weights for policy 1, policy_version 201325 (0.0009) [2023-12-26 16:49:34,527][105692] Updated weights for policy 0, policy_version 200559 (0.0007) [2023-12-26 16:49:34,580][105620] Updated weights for policy 1, policy_version 201335 (0.0007) [2023-12-26 16:49:34,642][105620] Updated weights for policy 1, policy_version 201345 (0.0009) [2023-12-26 16:49:35,284][105692] Updated weights for policy 0, policy_version 200569 (0.0007) [2023-12-26 16:49:35,344][105692] Updated weights for policy 0, policy_version 200579 (0.0009) [2023-12-26 16:49:35,397][105620] Updated weights for policy 1, policy_version 201355 (0.0008) [2023-12-26 16:49:35,399][105692] Updated weights for policy 0, policy_version 200589 (0.0008) [2023-12-26 16:49:35,453][105620] Updated weights for policy 1, policy_version 201365 (0.0007) [2023-12-26 16:49:35,455][105692] Updated weights for policy 0, policy_version 200599 (0.0006) [2023-12-26 16:49:35,509][105620] Updated weights for policy 1, policy_version 201375 (0.0008) [2023-12-26 16:49:36,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 102924288. Throughput: 0: 9878.9, 1: 9653.5. Samples: 102917048. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-26 16:49:36,062][104569] Avg episode reward: [(0, '9351.630'), (1, '1916.319')] [2023-12-26 16:49:36,172][105692] Updated weights for policy 0, policy_version 200609 (0.0007) [2023-12-26 16:49:36,242][105692] Updated weights for policy 0, policy_version 200619 (0.0009) [2023-12-26 16:49:36,271][105620] Updated weights for policy 1, policy_version 201385 (0.0009) [2023-12-26 16:49:36,312][105692] Updated weights for policy 0, policy_version 200629 (0.0007) [2023-12-26 16:49:36,334][105620] Updated weights for policy 1, policy_version 201395 (0.0007) [2023-12-26 16:49:36,403][105620] Updated weights for policy 1, policy_version 201405 (0.0006) [2023-12-26 16:49:36,474][105620] Updated weights for policy 1, policy_version 201415 (0.0006) [2023-12-26 16:49:37,019][105692] Updated weights for policy 0, policy_version 200639 (0.0008) [2023-12-26 16:49:37,070][105692] Updated weights for policy 0, policy_version 200649 (0.0008) [2023-12-26 16:49:37,106][105620] Updated weights for policy 1, policy_version 201425 (0.0008) [2023-12-26 16:49:37,122][105692] Updated weights for policy 0, policy_version 200659 (0.0009) [2023-12-26 16:49:37,172][105620] Updated weights for policy 1, policy_version 201435 (0.0006) [2023-12-26 16:49:37,232][105620] Updated weights for policy 1, policy_version 201445 (0.0008) [2023-12-26 16:49:37,900][105620] Updated weights for policy 1, policy_version 201455 (0.0008) [2023-12-26 16:49:37,928][105692] Updated weights for policy 0, policy_version 200669 (0.0007) [2023-12-26 16:49:37,961][105620] Updated weights for policy 1, policy_version 201465 (0.0009) [2023-12-26 16:49:37,995][105692] Updated weights for policy 0, policy_version 200679 (0.0005) [2023-12-26 16:49:38,014][105620] Updated weights for policy 1, policy_version 201475 (0.0008) [2023-12-26 16:49:38,058][105692] Updated weights for policy 0, policy_version 200689 (0.0005) [2023-12-26 16:49:38,624][105692] Updated weights for policy 0, policy_version 200699 (0.0007) [2023-12-26 16:49:38,686][105692] Updated weights for policy 0, policy_version 200709 (0.0010) [2023-12-26 16:49:38,753][105692] Updated weights for policy 0, policy_version 200719 (0.0010) [2023-12-26 16:49:38,795][105620] Updated weights for policy 1, policy_version 201485 (0.0007) [2023-12-26 16:49:38,858][105620] Updated weights for policy 1, policy_version 201495 (0.0007) [2023-12-26 16:49:38,918][105620] Updated weights for policy 1, policy_version 201505 (0.0008) [2023-12-26 16:49:39,515][105692] Updated weights for policy 0, policy_version 200729 (0.0010) [2023-12-26 16:49:39,567][105692] Updated weights for policy 0, policy_version 200739 (0.0010) [2023-12-26 16:49:39,627][105692] Updated weights for policy 0, policy_version 200749 (0.0010) [2023-12-26 16:49:39,645][105620] Updated weights for policy 1, policy_version 201515 (0.0008) [2023-12-26 16:49:39,686][105692] Updated weights for policy 0, policy_version 200759 (0.0010) [2023-12-26 16:49:39,691][105620] Updated weights for policy 1, policy_version 201525 (0.0007) [2023-12-26 16:49:39,741][105620] Updated weights for policy 1, policy_version 201535 (0.0008) [2023-12-26 16:49:40,446][105692] Updated weights for policy 0, policy_version 200769 (0.0009) [2023-12-26 16:49:40,499][105692] Updated weights for policy 0, policy_version 200779 (0.0009) [2023-12-26 16:49:40,548][105692] Updated weights for policy 0, policy_version 200789 (0.0009) [2023-12-26 16:49:40,592][105620] Updated weights for policy 1, policy_version 201545 (0.0009) [2023-12-26 16:49:40,650][105620] Updated weights for policy 1, policy_version 201555 (0.0008) [2023-12-26 16:49:40,711][105620] Updated weights for policy 1, policy_version 201565 (0.0008) [2023-12-26 16:49:40,775][105620] Updated weights for policy 1, policy_version 201575 (0.0008) [2023-12-26 16:49:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 103022592. Throughput: 0: 9740.7, 1: 9666.5. Samples: 103031096. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-26 16:49:41,062][104569] Avg episode reward: [(0, '9351.605'), (1, '6369.731')] [2023-12-26 16:49:41,369][105692] Updated weights for policy 0, policy_version 200799 (0.0010) [2023-12-26 16:49:41,437][105692] Updated weights for policy 0, policy_version 200809 (0.0009) [2023-12-26 16:49:41,493][105692] Updated weights for policy 0, policy_version 200819 (0.0007) [2023-12-26 16:49:41,542][105620] Updated weights for policy 1, policy_version 201585 (0.0008) [2023-12-26 16:49:41,597][105620] Updated weights for policy 1, policy_version 201595 (0.0010) [2023-12-26 16:49:41,659][105620] Updated weights for policy 1, policy_version 201605 (0.0009) [2023-12-26 16:49:42,157][105692] Updated weights for policy 0, policy_version 200829 (0.0008) [2023-12-26 16:49:42,212][105692] Updated weights for policy 0, policy_version 200839 (0.0012) [2023-12-26 16:49:42,278][105692] Updated weights for policy 0, policy_version 200849 (0.0010) [2023-12-26 16:49:42,361][105620] Updated weights for policy 1, policy_version 201615 (0.0009) [2023-12-26 16:49:42,425][105620] Updated weights for policy 1, policy_version 201625 (0.0009) [2023-12-26 16:49:42,487][105620] Updated weights for policy 1, policy_version 201635 (0.0010) [2023-12-26 16:49:42,986][105692] Updated weights for policy 0, policy_version 200859 (0.0008) [2023-12-26 16:49:43,044][105692] Updated weights for policy 0, policy_version 200869 (0.0005) [2023-12-26 16:49:43,099][105692] Updated weights for policy 0, policy_version 200879 (0.0008) [2023-12-26 16:49:43,274][105620] Updated weights for policy 1, policy_version 201645 (0.0010) [2023-12-26 16:49:43,339][105620] Updated weights for policy 1, policy_version 201655 (0.0008) [2023-12-26 16:49:43,410][105620] Updated weights for policy 1, policy_version 201665 (0.0005) [2023-12-26 16:49:43,698][105692] Updated weights for policy 0, policy_version 200889 (0.0009) [2023-12-26 16:49:43,764][105692] Updated weights for policy 0, policy_version 200899 (0.0011) [2023-12-26 16:49:43,823][105692] Updated weights for policy 0, policy_version 200909 (0.0010) [2023-12-26 16:49:43,881][105692] Updated weights for policy 0, policy_version 200919 (0.0010) [2023-12-26 16:49:44,034][105620] Updated weights for policy 1, policy_version 201675 (0.0009) [2023-12-26 16:49:44,089][105620] Updated weights for policy 1, policy_version 201685 (0.0010) [2023-12-26 16:49:44,151][105620] Updated weights for policy 1, policy_version 201695 (0.0011) [2023-12-26 16:49:44,619][105692] Updated weights for policy 0, policy_version 200929 (0.0010) [2023-12-26 16:49:44,667][105692] Updated weights for policy 0, policy_version 200939 (0.0011) [2023-12-26 16:49:44,720][105692] Updated weights for policy 0, policy_version 200949 (0.0011) [2023-12-26 16:49:44,919][105620] Updated weights for policy 1, policy_version 201705 (0.0010) [2023-12-26 16:49:44,985][105620] Updated weights for policy 1, policy_version 201715 (0.0010) [2023-12-26 16:49:45,050][105620] Updated weights for policy 1, policy_version 201725 (0.0010) [2023-12-26 16:49:45,113][105620] Updated weights for policy 1, policy_version 201735 (0.0010) [2023-12-26 16:49:45,461][105692] Updated weights for policy 0, policy_version 200959 (0.0011) [2023-12-26 16:49:45,525][105692] Updated weights for policy 0, policy_version 200969 (0.0009) [2023-12-26 16:49:45,581][105692] Updated weights for policy 0, policy_version 200979 (0.0009) [2023-12-26 16:49:45,818][105620] Updated weights for policy 1, policy_version 201745 (0.0010) [2023-12-26 16:49:45,877][105620] Updated weights for policy 1, policy_version 201755 (0.0010) [2023-12-26 16:49:45,933][105620] Updated weights for policy 1, policy_version 201765 (0.0010) [2023-12-26 16:49:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 103120896. Throughput: 0: 9688.0, 1: 9673.1. Samples: 103090212. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-26 16:49:46,062][104569] Avg episode reward: [(0, '9351.210'), (1, '8814.347')] [2023-12-26 16:49:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000200984_51462144.pth... [2023-12-26 16:49:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000201768_51658752.pth... [2023-12-26 16:49:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000199832_51167232.pth [2023-12-26 16:49:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000200680_51380224.pth [2023-12-26 16:49:46,249][105692] Updated weights for policy 0, policy_version 200989 (0.0008) [2023-12-26 16:49:46,304][105692] Updated weights for policy 0, policy_version 200999 (0.0008) [2023-12-26 16:49:46,358][105692] Updated weights for policy 0, policy_version 201009 (0.0005) [2023-12-26 16:49:46,657][105620] Updated weights for policy 1, policy_version 201775 (0.0007) [2023-12-26 16:49:46,712][105620] Updated weights for policy 1, policy_version 201785 (0.0005) [2023-12-26 16:49:46,764][105620] Updated weights for policy 1, policy_version 201795 (0.0005) [2023-12-26 16:49:46,961][105692] Updated weights for policy 0, policy_version 201019 (0.0005) [2023-12-26 16:49:47,009][105692] Updated weights for policy 0, policy_version 201029 (0.0005) [2023-12-26 16:49:47,066][105692] Updated weights for policy 0, policy_version 201039 (0.0006) [2023-12-26 16:49:47,363][105620] Updated weights for policy 1, policy_version 201805 (0.0005) [2023-12-26 16:49:47,428][105620] Updated weights for policy 1, policy_version 201815 (0.0009) [2023-12-26 16:49:47,494][105620] Updated weights for policy 1, policy_version 201825 (0.0010) [2023-12-26 16:49:47,659][105692] Updated weights for policy 0, policy_version 201049 (0.0010) [2023-12-26 16:49:47,723][105692] Updated weights for policy 0, policy_version 201059 (0.0007) [2023-12-26 16:49:47,769][105692] Updated weights for policy 0, policy_version 201069 (0.0005) [2023-12-26 16:49:47,817][105692] Updated weights for policy 0, policy_version 201079 (0.0005) [2023-12-26 16:49:48,340][105620] Updated weights for policy 1, policy_version 201835 (0.0009) [2023-12-26 16:49:48,342][105692] Updated weights for policy 0, policy_version 201089 (0.0006) [2023-12-26 16:49:48,403][105620] Updated weights for policy 1, policy_version 201845 (0.0008) [2023-12-26 16:49:48,404][105692] Updated weights for policy 0, policy_version 201099 (0.0006) [2023-12-26 16:49:48,460][105620] Updated weights for policy 1, policy_version 201855 (0.0007) [2023-12-26 16:49:48,464][105692] Updated weights for policy 0, policy_version 201109 (0.0008) [2023-12-26 16:49:49,061][105692] Updated weights for policy 0, policy_version 201119 (0.0009) [2023-12-26 16:49:49,118][105692] Updated weights for policy 0, policy_version 201129 (0.0011) [2023-12-26 16:49:49,170][105692] Updated weights for policy 0, policy_version 201139 (0.0011) [2023-12-26 16:49:49,243][105620] Updated weights for policy 1, policy_version 201865 (0.0008) [2023-12-26 16:49:49,303][105620] Updated weights for policy 1, policy_version 201875 (0.0008) [2023-12-26 16:49:49,365][105620] Updated weights for policy 1, policy_version 201885 (0.0008) [2023-12-26 16:49:49,432][105620] Updated weights for policy 1, policy_version 201895 (0.0010) [2023-12-26 16:49:49,794][105692] Updated weights for policy 0, policy_version 201149 (0.0009) [2023-12-26 16:49:49,866][105692] Updated weights for policy 0, policy_version 201159 (0.0008) [2023-12-26 16:49:49,930][105692] Updated weights for policy 0, policy_version 201169 (0.0008) [2023-12-26 16:49:50,231][105620] Updated weights for policy 1, policy_version 201905 (0.0006) [2023-12-26 16:49:50,289][105620] Updated weights for policy 1, policy_version 201915 (0.0005) [2023-12-26 16:49:50,354][105620] Updated weights for policy 1, policy_version 201925 (0.0006) [2023-12-26 16:49:50,586][105692] Updated weights for policy 0, policy_version 201179 (0.0008) [2023-12-26 16:49:50,657][105692] Updated weights for policy 0, policy_version 201189 (0.0008) [2023-12-26 16:49:50,712][105692] Updated weights for policy 0, policy_version 201199 (0.0008) [2023-12-26 16:49:50,948][105620] Updated weights for policy 1, policy_version 201936 (0.0010) [2023-12-26 16:49:51,004][105620] Updated weights for policy 1, policy_version 201947 (0.0011) [2023-12-26 16:49:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 103219200. Throughput: 0: 9895.2, 1: 9522.4. Samples: 103210316. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-26 16:49:51,062][104569] Avg episode reward: [(0, '9176.871'), (1, '9007.406')] [2023-12-26 16:49:51,073][105620] Updated weights for policy 1, policy_version 201957 (0.0007) [2023-12-26 16:49:51,372][105692] Updated weights for policy 0, policy_version 201209 (0.0010) [2023-12-26 16:49:51,428][105692] Updated weights for policy 0, policy_version 201219 (0.0010) [2023-12-26 16:49:51,476][105692] Updated weights for policy 0, policy_version 201229 (0.0010) [2023-12-26 16:49:51,524][105692] Updated weights for policy 0, policy_version 201239 (0.0010) [2023-12-26 16:49:51,784][105620] Updated weights for policy 1, policy_version 201967 (0.0008) [2023-12-26 16:49:51,841][105620] Updated weights for policy 1, policy_version 201977 (0.0008) [2023-12-26 16:49:51,893][105620] Updated weights for policy 1, policy_version 201987 (0.0008) [2023-12-26 16:49:52,318][105692] Updated weights for policy 0, policy_version 201249 (0.0010) [2023-12-26 16:49:52,375][105692] Updated weights for policy 0, policy_version 201259 (0.0011) [2023-12-26 16:49:52,436][105692] Updated weights for policy 0, policy_version 201269 (0.0005) [2023-12-26 16:49:52,710][105620] Updated weights for policy 1, policy_version 201997 (0.0009) [2023-12-26 16:49:52,773][105620] Updated weights for policy 1, policy_version 202007 (0.0008) [2023-12-26 16:49:52,826][105620] Updated weights for policy 1, policy_version 202017 (0.0010) [2023-12-26 16:49:53,100][105692] Updated weights for policy 0, policy_version 201279 (0.0006) [2023-12-26 16:49:53,159][105692] Updated weights for policy 0, policy_version 201289 (0.0010) [2023-12-26 16:49:53,221][105692] Updated weights for policy 0, policy_version 201299 (0.0010) [2023-12-26 16:49:53,649][105620] Updated weights for policy 1, policy_version 202027 (0.0009) [2023-12-26 16:49:53,702][105620] Updated weights for policy 1, policy_version 202037 (0.0010) [2023-12-26 16:49:53,754][105620] Updated weights for policy 1, policy_version 202047 (0.0009) [2023-12-26 16:49:53,799][105692] Updated weights for policy 0, policy_version 201309 (0.0008) [2023-12-26 16:49:53,868][105692] Updated weights for policy 0, policy_version 201319 (0.0005) [2023-12-26 16:49:53,928][105692] Updated weights for policy 0, policy_version 201329 (0.0005) [2023-12-26 16:49:54,446][105620] Updated weights for policy 1, policy_version 202057 (0.0009) [2023-12-26 16:49:54,491][105692] Updated weights for policy 0, policy_version 201339 (0.0007) [2023-12-26 16:49:54,501][105620] Updated weights for policy 1, policy_version 202067 (0.0006) [2023-12-26 16:49:54,550][105692] Updated weights for policy 0, policy_version 201349 (0.0011) [2023-12-26 16:49:54,556][105620] Updated weights for policy 1, policy_version 202077 (0.0006) [2023-12-26 16:49:54,611][105692] Updated weights for policy 0, policy_version 201359 (0.0010) [2023-12-26 16:49:54,621][105620] Updated weights for policy 1, policy_version 202087 (0.0005) [2023-12-26 16:49:55,325][105692] Updated weights for policy 0, policy_version 201369 (0.0010) [2023-12-26 16:49:55,384][105692] Updated weights for policy 0, policy_version 201379 (0.0010) [2023-12-26 16:49:55,393][105620] Updated weights for policy 1, policy_version 202097 (0.0010) [2023-12-26 16:49:55,443][105692] Updated weights for policy 0, policy_version 201389 (0.0010) [2023-12-26 16:49:55,452][105620] Updated weights for policy 1, policy_version 202107 (0.0008) [2023-12-26 16:49:55,505][105692] Updated weights for policy 0, policy_version 201399 (0.0010) [2023-12-26 16:49:55,512][105620] Updated weights for policy 1, policy_version 202117 (0.0008) [2023-12-26 16:49:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 103317504. Throughput: 0: 10041.3, 1: 9525.1. Samples: 103329876. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-26 16:49:56,063][104569] Avg episode reward: [(0, '9177.146'), (1, '568.399')] [2023-12-26 16:49:56,144][105620] Updated weights for policy 1, policy_version 202127 (0.0007) [2023-12-26 16:49:56,203][105620] Updated weights for policy 1, policy_version 202137 (0.0006) [2023-12-26 16:49:56,219][105692] Updated weights for policy 0, policy_version 201409 (0.0011) [2023-12-26 16:49:56,261][105620] Updated weights for policy 1, policy_version 202147 (0.0007) [2023-12-26 16:49:56,267][105692] Updated weights for policy 0, policy_version 201419 (0.0010) [2023-12-26 16:49:56,323][105692] Updated weights for policy 0, policy_version 201429 (0.0007) [2023-12-26 16:49:56,898][105620] Updated weights for policy 1, policy_version 202157 (0.0008) [2023-12-26 16:49:56,940][105692] Updated weights for policy 0, policy_version 201439 (0.0011) [2023-12-26 16:49:56,972][105620] Updated weights for policy 1, policy_version 202167 (0.0006) [2023-12-26 16:49:56,998][105692] Updated weights for policy 0, policy_version 201449 (0.0007) [2023-12-26 16:49:57,027][105620] Updated weights for policy 1, policy_version 202177 (0.0010) [2023-12-26 16:49:57,043][105692] Updated weights for policy 0, policy_version 201459 (0.0005) [2023-12-26 16:49:57,597][105620] Updated weights for policy 1, policy_version 202187 (0.0010) [2023-12-26 16:49:57,651][105620] Updated weights for policy 1, policy_version 202197 (0.0010) [2023-12-26 16:49:57,702][105620] Updated weights for policy 1, policy_version 202207 (0.0010) [2023-12-26 16:49:57,743][105692] Updated weights for policy 0, policy_version 201469 (0.0008) [2023-12-26 16:49:57,797][105692] Updated weights for policy 0, policy_version 201479 (0.0010) [2023-12-26 16:49:57,851][105692] Updated weights for policy 0, policy_version 201489 (0.0010) [2023-12-26 16:49:58,448][105620] Updated weights for policy 1, policy_version 202217 (0.0010) [2023-12-26 16:49:58,517][105620] Updated weights for policy 1, policy_version 202227 (0.0010) [2023-12-26 16:49:58,577][105692] Updated weights for policy 0, policy_version 201499 (0.0009) [2023-12-26 16:49:58,581][105620] Updated weights for policy 1, policy_version 202237 (0.0011) [2023-12-26 16:49:58,641][105620] Updated weights for policy 1, policy_version 202247 (0.0010) [2023-12-26 16:49:58,643][105692] Updated weights for policy 0, policy_version 201509 (0.0007) [2023-12-26 16:49:58,714][105692] Updated weights for policy 0, policy_version 201519 (0.0008) [2023-12-26 16:49:59,491][105620] Updated weights for policy 1, policy_version 202257 (0.0010) [2023-12-26 16:49:59,527][105692] Updated weights for policy 0, policy_version 201529 (0.0009) [2023-12-26 16:49:59,554][105620] Updated weights for policy 1, policy_version 202267 (0.0011) [2023-12-26 16:49:59,584][105692] Updated weights for policy 0, policy_version 201539 (0.0006) [2023-12-26 16:49:59,613][105620] Updated weights for policy 1, policy_version 202277 (0.0011) [2023-12-26 16:49:59,644][105692] Updated weights for policy 0, policy_version 201549 (0.0009) [2023-12-26 16:49:59,704][105692] Updated weights for policy 0, policy_version 201559 (0.0008) [2023-12-26 16:50:00,367][105620] Updated weights for policy 1, policy_version 202287 (0.0010) [2023-12-26 16:50:00,431][105620] Updated weights for policy 1, policy_version 202297 (0.0009) [2023-12-26 16:50:00,480][105692] Updated weights for policy 0, policy_version 201569 (0.0008) [2023-12-26 16:50:00,490][105620] Updated weights for policy 1, policy_version 202307 (0.0006) [2023-12-26 16:50:00,530][105692] Updated weights for policy 0, policy_version 201579 (0.0010) [2023-12-26 16:50:00,584][105692] Updated weights for policy 0, policy_version 201589 (0.0010) [2023-12-26 16:50:01,046][105620] Updated weights for policy 1, policy_version 202317 (0.0007) [2023-12-26 16:50:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 103415808. Throughput: 0: 10126.0, 1: 9547.8. Samples: 103391196. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-26 16:50:01,062][104569] Avg episode reward: [(0, '9266.570'), (1, '678.505')] [2023-12-26 16:50:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000201592_51617792.pth... [2023-12-26 16:50:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000200408_51314688.pth [2023-12-26 16:50:01,109][105620] Updated weights for policy 1, policy_version 202327 (0.0009) [2023-12-26 16:50:01,179][105620] Updated weights for policy 1, policy_version 202337 (0.0009) [2023-12-26 16:50:01,222][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000202344_51806208.pth... [2023-12-26 16:50:01,227][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000201224_51519488.pth [2023-12-26 16:50:01,470][105692] Updated weights for policy 0, policy_version 201599 (0.0009) [2023-12-26 16:50:01,534][105692] Updated weights for policy 0, policy_version 201609 (0.0008) [2023-12-26 16:50:01,599][105692] Updated weights for policy 0, policy_version 201619 (0.0009) [2023-12-26 16:50:01,972][105620] Updated weights for policy 1, policy_version 202347 (0.0009) [2023-12-26 16:50:02,033][105620] Updated weights for policy 1, policy_version 202357 (0.0009) [2023-12-26 16:50:02,091][105620] Updated weights for policy 1, policy_version 202367 (0.0009) [2023-12-26 16:50:02,317][105692] Updated weights for policy 0, policy_version 201629 (0.0008) [2023-12-26 16:50:02,381][105692] Updated weights for policy 0, policy_version 201639 (0.0007) [2023-12-26 16:50:02,426][105692] Updated weights for policy 0, policy_version 201649 (0.0006) [2023-12-26 16:50:02,818][105620] Updated weights for policy 1, policy_version 202377 (0.0009) [2023-12-26 16:50:02,880][105620] Updated weights for policy 1, policy_version 202387 (0.0010) [2023-12-26 16:50:02,948][105620] Updated weights for policy 1, policy_version 202397 (0.0010) [2023-12-26 16:50:03,013][105620] Updated weights for policy 1, policy_version 202407 (0.0010) [2023-12-26 16:50:03,129][105692] Updated weights for policy 0, policy_version 201659 (0.0006) [2023-12-26 16:50:03,190][105692] Updated weights for policy 0, policy_version 201669 (0.0010) [2023-12-26 16:50:03,256][105692] Updated weights for policy 0, policy_version 201679 (0.0010) [2023-12-26 16:50:03,717][105620] Updated weights for policy 1, policy_version 202417 (0.0010) [2023-12-26 16:50:03,769][105620] Updated weights for policy 1, policy_version 202427 (0.0010) [2023-12-26 16:50:03,783][105692] Updated weights for policy 0, policy_version 201689 (0.0005) [2023-12-26 16:50:03,814][105620] Updated weights for policy 1, policy_version 202437 (0.0010) [2023-12-26 16:50:03,828][105692] Updated weights for policy 0, policy_version 201699 (0.0005) [2023-12-26 16:50:03,889][105692] Updated weights for policy 0, policy_version 201709 (0.0008) [2023-12-26 16:50:03,964][105692] Updated weights for policy 0, policy_version 201719 (0.0011) [2023-12-26 16:50:04,588][105620] Updated weights for policy 1, policy_version 202447 (0.0010) [2023-12-26 16:50:04,644][105692] Updated weights for policy 0, policy_version 201729 (0.0007) [2023-12-26 16:50:04,648][105620] Updated weights for policy 1, policy_version 202457 (0.0010) [2023-12-26 16:50:04,690][105692] Updated weights for policy 0, policy_version 201739 (0.0011) [2023-12-26 16:50:04,705][105620] Updated weights for policy 1, policy_version 202467 (0.0006) [2023-12-26 16:50:04,743][105692] Updated weights for policy 0, policy_version 201749 (0.0010) [2023-12-26 16:50:05,262][105620] Updated weights for policy 1, policy_version 202477 (0.0006) [2023-12-26 16:50:05,331][105620] Updated weights for policy 1, policy_version 202487 (0.0010) [2023-12-26 16:50:05,380][105620] Updated weights for policy 1, policy_version 202497 (0.0010) [2023-12-26 16:50:05,435][105692] Updated weights for policy 0, policy_version 201759 (0.0009) [2023-12-26 16:50:05,479][105692] Updated weights for policy 0, policy_version 201769 (0.0010) [2023-12-26 16:50:05,523][105692] Updated weights for policy 0, policy_version 201779 (0.0010) [2023-12-26 16:50:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 103514112. Throughput: 0: 10091.1, 1: 9474.4. Samples: 103506612. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-26 16:50:06,063][104569] Avg episode reward: [(0, '9351.200'), (1, '4994.742')] [2023-12-26 16:50:06,112][105620] Updated weights for policy 1, policy_version 202507 (0.0007) [2023-12-26 16:50:06,168][105620] Updated weights for policy 1, policy_version 202517 (0.0008) [2023-12-26 16:50:06,225][105620] Updated weights for policy 1, policy_version 202527 (0.0006) [2023-12-26 16:50:06,283][105692] Updated weights for policy 0, policy_version 201789 (0.0010) [2023-12-26 16:50:06,347][105692] Updated weights for policy 0, policy_version 201799 (0.0010) [2023-12-26 16:50:06,408][105692] Updated weights for policy 0, policy_version 201809 (0.0011) [2023-12-26 16:50:06,954][105620] Updated weights for policy 1, policy_version 202537 (0.0006) [2023-12-26 16:50:07,015][105620] Updated weights for policy 1, policy_version 202547 (0.0008) [2023-12-26 16:50:07,075][105620] Updated weights for policy 1, policy_version 202557 (0.0008) [2023-12-26 16:50:07,133][105620] Updated weights for policy 1, policy_version 202567 (0.0008) [2023-12-26 16:50:07,161][105692] Updated weights for policy 0, policy_version 201819 (0.0011) [2023-12-26 16:50:07,222][105692] Updated weights for policy 0, policy_version 201829 (0.0007) [2023-12-26 16:50:07,271][105692] Updated weights for policy 0, policy_version 201839 (0.0011) [2023-12-26 16:50:07,913][105620] Updated weights for policy 1, policy_version 202577 (0.0008) [2023-12-26 16:50:07,930][105692] Updated weights for policy 0, policy_version 201849 (0.0010) [2023-12-26 16:50:07,973][105620] Updated weights for policy 1, policy_version 202587 (0.0008) [2023-12-26 16:50:07,982][105692] Updated weights for policy 0, policy_version 201859 (0.0005) [2023-12-26 16:50:08,027][105620] Updated weights for policy 1, policy_version 202597 (0.0008) [2023-12-26 16:50:08,036][105692] Updated weights for policy 0, policy_version 201869 (0.0006) [2023-12-26 16:50:08,093][105692] Updated weights for policy 0, policy_version 201879 (0.0006) [2023-12-26 16:50:08,730][105692] Updated weights for policy 0, policy_version 201889 (0.0010) [2023-12-26 16:50:08,792][105692] Updated weights for policy 0, policy_version 201899 (0.0010) [2023-12-26 16:50:08,811][105620] Updated weights for policy 1, policy_version 202607 (0.0006) [2023-12-26 16:50:08,850][105692] Updated weights for policy 0, policy_version 201909 (0.0007) [2023-12-26 16:50:08,861][105620] Updated weights for policy 1, policy_version 202617 (0.0005) [2023-12-26 16:50:08,929][105620] Updated weights for policy 1, policy_version 202627 (0.0006) [2023-12-26 16:50:09,560][105620] Updated weights for policy 1, policy_version 202637 (0.0005) [2023-12-26 16:50:09,582][105692] Updated weights for policy 0, policy_version 201919 (0.0007) [2023-12-26 16:50:09,631][105620] Updated weights for policy 1, policy_version 202647 (0.0007) [2023-12-26 16:50:09,639][105692] Updated weights for policy 0, policy_version 201929 (0.0006) [2023-12-26 16:50:09,690][105620] Updated weights for policy 1, policy_version 202657 (0.0007) [2023-12-26 16:50:09,700][105692] Updated weights for policy 0, policy_version 201939 (0.0007) [2023-12-26 16:50:10,432][105692] Updated weights for policy 0, policy_version 201949 (0.0007) [2023-12-26 16:50:10,439][105620] Updated weights for policy 1, policy_version 202667 (0.0008) [2023-12-26 16:50:10,494][105692] Updated weights for policy 0, policy_version 201959 (0.0007) [2023-12-26 16:50:10,499][105620] Updated weights for policy 1, policy_version 202677 (0.0008) [2023-12-26 16:50:10,541][105692] Updated weights for policy 0, policy_version 201969 (0.0008) [2023-12-26 16:50:10,566][105620] Updated weights for policy 1, policy_version 202687 (0.0008) [2023-12-26 16:50:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 103612416. Throughput: 0: 10076.0, 1: 9500.0. Samples: 103622916. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-26 16:50:11,062][104569] Avg episode reward: [(0, '9347.780'), (1, '6257.174')] [2023-12-26 16:50:11,262][105692] Updated weights for policy 0, policy_version 201979 (0.0009) [2023-12-26 16:50:11,316][105692] Updated weights for policy 0, policy_version 201989 (0.0006) [2023-12-26 16:50:11,352][105620] Updated weights for policy 1, policy_version 202697 (0.0008) [2023-12-26 16:50:11,386][105692] Updated weights for policy 0, policy_version 202000 (0.0007) [2023-12-26 16:50:11,418][105620] Updated weights for policy 1, policy_version 202707 (0.0007) [2023-12-26 16:50:11,479][105620] Updated weights for policy 1, policy_version 202717 (0.0008) [2023-12-26 16:50:11,550][105620] Updated weights for policy 1, policy_version 202727 (0.0010) [2023-12-26 16:50:12,072][105692] Updated weights for policy 0, policy_version 202010 (0.0007) [2023-12-26 16:50:12,135][105692] Updated weights for policy 0, policy_version 202020 (0.0008) [2023-12-26 16:50:12,212][105692] Updated weights for policy 0, policy_version 202030 (0.0005) [2023-12-26 16:50:12,281][105692] Updated weights for policy 0, policy_version 202040 (0.0006) [2023-12-26 16:50:12,365][105620] Updated weights for policy 1, policy_version 202737 (0.0009) [2023-12-26 16:50:12,423][105620] Updated weights for policy 1, policy_version 202747 (0.0007) [2023-12-26 16:50:12,482][105620] Updated weights for policy 1, policy_version 202757 (0.0009) [2023-12-26 16:50:12,958][105692] Updated weights for policy 0, policy_version 202050 (0.0011) [2023-12-26 16:50:13,023][105692] Updated weights for policy 0, policy_version 202060 (0.0011) [2023-12-26 16:50:13,078][105620] Updated weights for policy 1, policy_version 202767 (0.0007) [2023-12-26 16:50:13,082][105692] Updated weights for policy 0, policy_version 202070 (0.0010) [2023-12-26 16:50:13,131][105620] Updated weights for policy 1, policy_version 202777 (0.0005) [2023-12-26 16:50:13,189][105620] Updated weights for policy 1, policy_version 202787 (0.0007) [2023-12-26 16:50:13,723][105692] Updated weights for policy 0, policy_version 202080 (0.0009) [2023-12-26 16:50:13,782][105692] Updated weights for policy 0, policy_version 202090 (0.0006) [2023-12-26 16:50:13,851][105692] Updated weights for policy 0, policy_version 202100 (0.0009) [2023-12-26 16:50:13,882][105620] Updated weights for policy 1, policy_version 202797 (0.0007) [2023-12-26 16:50:13,945][105620] Updated weights for policy 1, policy_version 202807 (0.0005) [2023-12-26 16:50:14,003][105620] Updated weights for policy 1, policy_version 202817 (0.0005) [2023-12-26 16:50:14,495][105692] Updated weights for policy 0, policy_version 202110 (0.0008) [2023-12-26 16:50:14,560][105692] Updated weights for policy 0, policy_version 202120 (0.0010) [2023-12-26 16:50:14,618][105692] Updated weights for policy 0, policy_version 202130 (0.0007) [2023-12-26 16:50:14,638][105620] Updated weights for policy 1, policy_version 202827 (0.0009) [2023-12-26 16:50:14,693][105620] Updated weights for policy 1, policy_version 202837 (0.0008) [2023-12-26 16:50:14,754][105620] Updated weights for policy 1, policy_version 202847 (0.0009) [2023-12-26 16:50:15,256][105692] Updated weights for policy 0, policy_version 202140 (0.0007) [2023-12-26 16:50:15,325][105692] Updated weights for policy 0, policy_version 202150 (0.0010) [2023-12-26 16:50:15,386][105692] Updated weights for policy 0, policy_version 202160 (0.0010) [2023-12-26 16:50:15,492][105620] Updated weights for policy 1, policy_version 202857 (0.0010) [2023-12-26 16:50:15,555][105620] Updated weights for policy 1, policy_version 202867 (0.0011) [2023-12-26 16:50:15,610][105620] Updated weights for policy 1, policy_version 202877 (0.0011) [2023-12-26 16:50:15,655][105620] Updated weights for policy 1, policy_version 202887 (0.0010) [2023-12-26 16:50:16,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 103710720. Throughput: 0: 10017.4, 1: 9515.8. Samples: 103681892. Policy #0 lag: (min: 17.0, avg: 45.1, max: 49.0) [2023-12-26 16:50:16,062][104569] Avg episode reward: [(0, '9347.385'), (1, '1559.728')] [2023-12-26 16:50:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000202888_51945472.pth... [2023-12-26 16:50:16,071][105692] Updated weights for policy 0, policy_version 202170 (0.0006) [2023-12-26 16:50:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000201768_51658752.pth [2023-12-26 16:50:16,127][105692] Updated weights for policy 0, policy_version 202180 (0.0010) [2023-12-26 16:50:16,185][105692] Updated weights for policy 0, policy_version 202190 (0.0010) [2023-12-26 16:50:16,237][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000202200_51773440.pth... [2023-12-26 16:50:16,240][105692] Updated weights for policy 0, policy_version 202200 (0.0007) [2023-12-26 16:50:16,241][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000200984_51462144.pth [2023-12-26 16:50:16,293][105620] Updated weights for policy 1, policy_version 202897 (0.0006) [2023-12-26 16:50:16,359][105620] Updated weights for policy 1, policy_version 202907 (0.0007) [2023-12-26 16:50:16,417][105620] Updated weights for policy 1, policy_version 202917 (0.0010) [2023-12-26 16:50:16,890][105692] Updated weights for policy 0, policy_version 202210 (0.0009) [2023-12-26 16:50:16,955][105692] Updated weights for policy 0, policy_version 202220 (0.0009) [2023-12-26 16:50:17,019][105692] Updated weights for policy 0, policy_version 202230 (0.0008) [2023-12-26 16:50:17,057][105620] Updated weights for policy 1, policy_version 202927 (0.0007) [2023-12-26 16:50:17,115][105620] Updated weights for policy 1, policy_version 202937 (0.0005) [2023-12-26 16:50:17,174][105620] Updated weights for policy 1, policy_version 202947 (0.0006) [2023-12-26 16:50:17,655][105692] Updated weights for policy 0, policy_version 202240 (0.0010) [2023-12-26 16:50:17,716][105692] Updated weights for policy 0, policy_version 202250 (0.0009) [2023-12-26 16:50:17,767][105692] Updated weights for policy 0, policy_version 202260 (0.0010) [2023-12-26 16:50:17,857][105620] Updated weights for policy 1, policy_version 202957 (0.0007) [2023-12-26 16:50:17,922][105620] Updated weights for policy 1, policy_version 202967 (0.0009) [2023-12-26 16:50:17,982][105620] Updated weights for policy 1, policy_version 202977 (0.0006) [2023-12-26 16:50:18,391][105692] Updated weights for policy 0, policy_version 202270 (0.0008) [2023-12-26 16:50:18,441][105692] Updated weights for policy 0, policy_version 202280 (0.0011) [2023-12-26 16:50:18,485][105692] Updated weights for policy 0, policy_version 202290 (0.0010) [2023-12-26 16:50:18,621][105620] Updated weights for policy 1, policy_version 202987 (0.0009) [2023-12-26 16:50:18,681][105620] Updated weights for policy 1, policy_version 202997 (0.0007) [2023-12-26 16:50:18,736][105620] Updated weights for policy 1, policy_version 203007 (0.0005) [2023-12-26 16:50:19,244][105692] Updated weights for policy 0, policy_version 202300 (0.0011) [2023-12-26 16:50:19,299][105692] Updated weights for policy 0, policy_version 202310 (0.0010) [2023-12-26 16:50:19,362][105692] Updated weights for policy 0, policy_version 202320 (0.0010) [2023-12-26 16:50:19,503][105620] Updated weights for policy 1, policy_version 203017 (0.0008) [2023-12-26 16:50:19,567][105620] Updated weights for policy 1, policy_version 203027 (0.0009) [2023-12-26 16:50:19,622][105620] Updated weights for policy 1, policy_version 203037 (0.0008) [2023-12-26 16:50:19,683][105620] Updated weights for policy 1, policy_version 203047 (0.0009) [2023-12-26 16:50:20,131][105692] Updated weights for policy 0, policy_version 202330 (0.0010) [2023-12-26 16:50:20,199][105692] Updated weights for policy 0, policy_version 202340 (0.0009) [2023-12-26 16:50:20,269][105692] Updated weights for policy 0, policy_version 202350 (0.0011) [2023-12-26 16:50:20,335][105692] Updated weights for policy 0, policy_version 202360 (0.0011) [2023-12-26 16:50:20,416][105620] Updated weights for policy 1, policy_version 203057 (0.0009) [2023-12-26 16:50:20,477][105620] Updated weights for policy 1, policy_version 203067 (0.0008) [2023-12-26 16:50:20,532][105620] Updated weights for policy 1, policy_version 203077 (0.0008) [2023-12-26 16:50:21,001][105692] Updated weights for policy 0, policy_version 202370 (0.0006) [2023-12-26 16:50:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 103809024. Throughput: 0: 10056.3, 1: 9639.0. Samples: 103803336. Policy #0 lag: (min: 17.0, avg: 45.1, max: 49.0) [2023-12-26 16:50:21,062][104569] Avg episode reward: [(0, '9264.732'), (1, '6523.889')] [2023-12-26 16:50:21,077][105692] Updated weights for policy 0, policy_version 202380 (0.0008) [2023-12-26 16:50:21,143][105692] Updated weights for policy 0, policy_version 202390 (0.0010) [2023-12-26 16:50:21,355][105620] Updated weights for policy 1, policy_version 203087 (0.0009) [2023-12-26 16:50:21,427][105620] Updated weights for policy 1, policy_version 203097 (0.0009) [2023-12-26 16:50:21,490][105620] Updated weights for policy 1, policy_version 203107 (0.0009) [2023-12-26 16:50:21,870][105692] Updated weights for policy 0, policy_version 202400 (0.0008) [2023-12-26 16:50:21,939][105692] Updated weights for policy 0, policy_version 202410 (0.0008) [2023-12-26 16:50:22,015][105692] Updated weights for policy 0, policy_version 202420 (0.0009) [2023-12-26 16:50:22,214][105620] Updated weights for policy 1, policy_version 203117 (0.0009) [2023-12-26 16:50:22,272][105620] Updated weights for policy 1, policy_version 203127 (0.0009) [2023-12-26 16:50:22,328][105620] Updated weights for policy 1, policy_version 203137 (0.0009) [2023-12-26 16:50:22,768][105692] Updated weights for policy 0, policy_version 202430 (0.0009) [2023-12-26 16:50:22,840][105692] Updated weights for policy 0, policy_version 202440 (0.0010) [2023-12-26 16:50:22,904][105692] Updated weights for policy 0, policy_version 202450 (0.0010) [2023-12-26 16:50:22,937][105620] Updated weights for policy 1, policy_version 203147 (0.0007) [2023-12-26 16:50:23,007][105620] Updated weights for policy 1, policy_version 203157 (0.0008) [2023-12-26 16:50:23,064][105620] Updated weights for policy 1, policy_version 203167 (0.0009) [2023-12-26 16:50:23,618][105692] Updated weights for policy 0, policy_version 202460 (0.0007) [2023-12-26 16:50:23,669][105692] Updated weights for policy 0, policy_version 202470 (0.0010) [2023-12-26 16:50:23,720][105692] Updated weights for policy 0, policy_version 202480 (0.0010) [2023-12-26 16:50:23,823][105620] Updated weights for policy 1, policy_version 203177 (0.0008) [2023-12-26 16:50:23,875][105620] Updated weights for policy 1, policy_version 203187 (0.0010) [2023-12-26 16:50:23,937][105620] Updated weights for policy 1, policy_version 203197 (0.0010) [2023-12-26 16:50:23,995][105620] Updated weights for policy 1, policy_version 203207 (0.0010) [2023-12-26 16:50:24,298][105692] Updated weights for policy 0, policy_version 202490 (0.0006) [2023-12-26 16:50:24,347][105692] Updated weights for policy 0, policy_version 202500 (0.0005) [2023-12-26 16:50:24,397][105692] Updated weights for policy 0, policy_version 202510 (0.0005) [2023-12-26 16:50:24,444][105692] Updated weights for policy 0, policy_version 202520 (0.0005) [2023-12-26 16:50:24,629][105620] Updated weights for policy 1, policy_version 203217 (0.0006) [2023-12-26 16:50:24,683][105620] Updated weights for policy 1, policy_version 203227 (0.0005) [2023-12-26 16:50:24,734][105620] Updated weights for policy 1, policy_version 203237 (0.0005) [2023-12-26 16:50:25,081][105692] Updated weights for policy 0, policy_version 202530 (0.0008) [2023-12-26 16:50:25,127][105692] Updated weights for policy 0, policy_version 202540 (0.0008) [2023-12-26 16:50:25,182][105692] Updated weights for policy 0, policy_version 202550 (0.0008) [2023-12-26 16:50:25,413][105620] Updated weights for policy 1, policy_version 203247 (0.0009) [2023-12-26 16:50:25,478][105620] Updated weights for policy 1, policy_version 203257 (0.0010) [2023-12-26 16:50:25,527][105620] Updated weights for policy 1, policy_version 203267 (0.0010) [2023-12-26 16:50:25,968][105692] Updated weights for policy 0, policy_version 202560 (0.0008) [2023-12-26 16:50:26,030][105692] Updated weights for policy 0, policy_version 202570 (0.0008) [2023-12-26 16:50:26,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 103907328. Throughput: 0: 10096.6, 1: 9691.3. Samples: 103921560. Policy #0 lag: (min: 17.0, avg: 45.1, max: 49.0) [2023-12-26 16:50:26,063][104569] Avg episode reward: [(0, '9266.148'), (1, '9085.982')] [2023-12-26 16:50:26,081][105692] Updated weights for policy 0, policy_version 202580 (0.0008) [2023-12-26 16:50:26,276][105620] Updated weights for policy 1, policy_version 203277 (0.0010) [2023-12-26 16:50:26,323][105620] Updated weights for policy 1, policy_version 203287 (0.0010) [2023-12-26 16:50:26,368][105620] Updated weights for policy 1, policy_version 203297 (0.0010) [2023-12-26 16:50:26,836][105692] Updated weights for policy 0, policy_version 202590 (0.0008) [2023-12-26 16:50:26,887][105692] Updated weights for policy 0, policy_version 202600 (0.0008) [2023-12-26 16:50:26,934][105692] Updated weights for policy 0, policy_version 202610 (0.0008) [2023-12-26 16:50:27,136][105620] Updated weights for policy 1, policy_version 203307 (0.0010) [2023-12-26 16:50:27,197][105620] Updated weights for policy 1, policy_version 203317 (0.0010) [2023-12-26 16:50:27,245][105620] Updated weights for policy 1, policy_version 203327 (0.0010) [2023-12-26 16:50:27,718][105692] Updated weights for policy 0, policy_version 202620 (0.0008) [2023-12-26 16:50:27,766][105692] Updated weights for policy 0, policy_version 202630 (0.0008) [2023-12-26 16:50:27,810][105692] Updated weights for policy 0, policy_version 202640 (0.0008) [2023-12-26 16:50:27,987][105620] Updated weights for policy 1, policy_version 203337 (0.0010) [2023-12-26 16:50:28,038][105620] Updated weights for policy 1, policy_version 203347 (0.0010) [2023-12-26 16:50:28,085][105620] Updated weights for policy 1, policy_version 203357 (0.0010) [2023-12-26 16:50:28,137][105620] Updated weights for policy 1, policy_version 203367 (0.0010) [2023-12-26 16:50:28,589][105692] Updated weights for policy 0, policy_version 202650 (0.0008) [2023-12-26 16:50:28,645][105692] Updated weights for policy 0, policy_version 202660 (0.0008) [2023-12-26 16:50:28,689][105692] Updated weights for policy 0, policy_version 202670 (0.0008) [2023-12-26 16:50:28,734][105692] Updated weights for policy 0, policy_version 202680 (0.0008) [2023-12-26 16:50:28,904][105620] Updated weights for policy 1, policy_version 203377 (0.0010) [2023-12-26 16:50:28,956][105620] Updated weights for policy 1, policy_version 203387 (0.0010) [2023-12-26 16:50:29,008][105620] Updated weights for policy 1, policy_version 203397 (0.0010) [2023-12-26 16:50:29,549][105692] Updated weights for policy 0, policy_version 202690 (0.0006) [2023-12-26 16:50:29,609][105692] Updated weights for policy 0, policy_version 202700 (0.0008) [2023-12-26 16:50:29,659][105692] Updated weights for policy 0, policy_version 202711 (0.0008) [2023-12-26 16:50:29,755][105620] Updated weights for policy 1, policy_version 203407 (0.0010) [2023-12-26 16:50:29,820][105620] Updated weights for policy 1, policy_version 203417 (0.0008) [2023-12-26 16:50:29,877][105620] Updated weights for policy 1, policy_version 203427 (0.0008) [2023-12-26 16:50:30,318][105692] Updated weights for policy 0, policy_version 202721 (0.0008) [2023-12-26 16:50:30,386][105692] Updated weights for policy 0, policy_version 202731 (0.0009) [2023-12-26 16:50:30,451][105692] Updated weights for policy 0, policy_version 202741 (0.0010) [2023-12-26 16:50:30,554][105620] Updated weights for policy 1, policy_version 203437 (0.0005) [2023-12-26 16:50:30,613][105620] Updated weights for policy 1, policy_version 203447 (0.0005) [2023-12-26 16:50:30,668][105620] Updated weights for policy 1, policy_version 203457 (0.0005) [2023-12-26 16:50:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 104005632. Throughput: 0: 10045.8, 1: 9676.3. Samples: 103977704. Policy #0 lag: (min: 17.0, avg: 45.1, max: 49.0) [2023-12-26 16:50:31,062][104569] Avg episode reward: [(0, '9347.612'), (1, '9083.578')] [2023-12-26 16:50:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000202744_51912704.pth... [2023-12-26 16:50:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000203464_52092928.pth... [2023-12-26 16:50:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000201592_51617792.pth [2023-12-26 16:50:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000202344_51806208.pth [2023-12-26 16:50:31,213][105692] Updated weights for policy 0, policy_version 202751 (0.0007) [2023-12-26 16:50:31,277][105692] Updated weights for policy 0, policy_version 202761 (0.0007) [2023-12-26 16:50:31,336][105692] Updated weights for policy 0, policy_version 202771 (0.0008) [2023-12-26 16:50:31,358][105620] Updated weights for policy 1, policy_version 203467 (0.0008) [2023-12-26 16:50:31,423][105620] Updated weights for policy 1, policy_version 203477 (0.0008) [2023-12-26 16:50:31,480][105620] Updated weights for policy 1, policy_version 203487 (0.0008) [2023-12-26 16:50:32,008][105692] Updated weights for policy 0, policy_version 202781 (0.0006) [2023-12-26 16:50:32,063][105692] Updated weights for policy 0, policy_version 202791 (0.0006) [2023-12-26 16:50:32,113][105692] Updated weights for policy 0, policy_version 202801 (0.0009) [2023-12-26 16:50:32,236][105620] Updated weights for policy 1, policy_version 203497 (0.0008) [2023-12-26 16:50:32,303][105620] Updated weights for policy 1, policy_version 203507 (0.0007) [2023-12-26 16:50:32,370][105620] Updated weights for policy 1, policy_version 203517 (0.0008) [2023-12-26 16:50:32,437][105620] Updated weights for policy 1, policy_version 203527 (0.0009) [2023-12-26 16:50:32,810][105692] Updated weights for policy 0, policy_version 202811 (0.0008) [2023-12-26 16:50:32,858][105692] Updated weights for policy 0, policy_version 202821 (0.0009) [2023-12-26 16:50:32,906][105692] Updated weights for policy 0, policy_version 202831 (0.0008) [2023-12-26 16:50:33,159][105620] Updated weights for policy 1, policy_version 203537 (0.0010) [2023-12-26 16:50:33,220][105620] Updated weights for policy 1, policy_version 203547 (0.0009) [2023-12-26 16:50:33,270][105620] Updated weights for policy 1, policy_version 203557 (0.0009) [2023-12-26 16:50:33,696][105692] Updated weights for policy 0, policy_version 202841 (0.0009) [2023-12-26 16:50:33,755][105692] Updated weights for policy 0, policy_version 202851 (0.0010) [2023-12-26 16:50:33,807][105692] Updated weights for policy 0, policy_version 202861 (0.0009) [2023-12-26 16:50:33,857][105692] Updated weights for policy 0, policy_version 202871 (0.0008) [2023-12-26 16:50:33,932][105620] Updated weights for policy 1, policy_version 203567 (0.0010) [2023-12-26 16:50:33,992][105620] Updated weights for policy 1, policy_version 203577 (0.0010) [2023-12-26 16:50:34,049][105620] Updated weights for policy 1, policy_version 203587 (0.0010) [2023-12-26 16:50:34,579][105692] Updated weights for policy 0, policy_version 202881 (0.0008) [2023-12-26 16:50:34,629][105692] Updated weights for policy 0, policy_version 202891 (0.0006) [2023-12-26 16:50:34,689][105692] Updated weights for policy 0, policy_version 202901 (0.0007) [2023-12-26 16:50:34,786][105620] Updated weights for policy 1, policy_version 203597 (0.0008) [2023-12-26 16:50:34,837][105620] Updated weights for policy 1, policy_version 203607 (0.0009) [2023-12-26 16:50:34,886][105620] Updated weights for policy 1, policy_version 203617 (0.0008) [2023-12-26 16:50:35,415][105692] Updated weights for policy 0, policy_version 202911 (0.0009) [2023-12-26 16:50:35,473][105692] Updated weights for policy 0, policy_version 202921 (0.0005) [2023-12-26 16:50:35,485][105620] Updated weights for policy 1, policy_version 203628 (0.0009) [2023-12-26 16:50:35,532][105692] Updated weights for policy 0, policy_version 202931 (0.0005) [2023-12-26 16:50:35,534][105620] Updated weights for policy 1, policy_version 203638 (0.0010) [2023-12-26 16:50:35,582][105620] Updated weights for policy 1, policy_version 203648 (0.0010) [2023-12-26 16:50:36,062][105692] Updated weights for policy 0, policy_version 202941 (0.0008) [2023-12-26 16:50:36,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 104103936. Throughput: 0: 9898.7, 1: 9736.2. Samples: 104093892. Policy #0 lag: (min: 17.0, avg: 45.1, max: 49.0) [2023-12-26 16:50:36,063][104569] Avg episode reward: [(0, '9346.594'), (1, '9262.591')] [2023-12-26 16:50:36,124][105692] Updated weights for policy 0, policy_version 202951 (0.0010) [2023-12-26 16:50:36,182][105692] Updated weights for policy 0, policy_version 202961 (0.0011) [2023-12-26 16:50:36,404][105620] Updated weights for policy 1, policy_version 203658 (0.0010) [2023-12-26 16:50:36,456][105620] Updated weights for policy 1, policy_version 203668 (0.0008) [2023-12-26 16:50:36,506][105620] Updated weights for policy 1, policy_version 203678 (0.0008) [2023-12-26 16:50:36,563][105620] Updated weights for policy 1, policy_version 203688 (0.0009) [2023-12-26 16:50:36,920][105692] Updated weights for policy 0, policy_version 202971 (0.0010) [2023-12-26 16:50:36,975][105692] Updated weights for policy 0, policy_version 202981 (0.0010) [2023-12-26 16:50:37,037][105692] Updated weights for policy 0, policy_version 202991 (0.0010) [2023-12-26 16:50:37,348][105620] Updated weights for policy 1, policy_version 203698 (0.0010) [2023-12-26 16:50:37,410][105620] Updated weights for policy 1, policy_version 203708 (0.0010) [2023-12-26 16:50:37,471][105620] Updated weights for policy 1, policy_version 203718 (0.0010) [2023-12-26 16:50:37,782][105692] Updated weights for policy 0, policy_version 203001 (0.0010) [2023-12-26 16:50:37,847][105692] Updated weights for policy 0, policy_version 203011 (0.0010) [2023-12-26 16:50:37,900][105692] Updated weights for policy 0, policy_version 203021 (0.0009) [2023-12-26 16:50:37,960][105692] Updated weights for policy 0, policy_version 203031 (0.0006) [2023-12-26 16:50:38,113][105620] Updated weights for policy 1, policy_version 203728 (0.0006) [2023-12-26 16:50:38,162][105620] Updated weights for policy 1, policy_version 203738 (0.0005) [2023-12-26 16:50:38,208][105620] Updated weights for policy 1, policy_version 203748 (0.0005) [2023-12-26 16:50:38,771][105692] Updated weights for policy 0, policy_version 203041 (0.0005) [2023-12-26 16:50:38,820][105692] Updated weights for policy 0, policy_version 203051 (0.0008) [2023-12-26 16:50:38,873][105692] Updated weights for policy 0, policy_version 203062 (0.0009) [2023-12-26 16:50:38,894][105620] Updated weights for policy 1, policy_version 203758 (0.0006) [2023-12-26 16:50:38,949][105620] Updated weights for policy 1, policy_version 203768 (0.0009) [2023-12-26 16:50:39,009][105620] Updated weights for policy 1, policy_version 203778 (0.0009) [2023-12-26 16:50:39,620][105692] Updated weights for policy 0, policy_version 203072 (0.0008) [2023-12-26 16:50:39,668][105692] Updated weights for policy 0, policy_version 203082 (0.0009) [2023-12-26 16:50:39,730][105692] Updated weights for policy 0, policy_version 203092 (0.0008) [2023-12-26 16:50:39,740][105620] Updated weights for policy 1, policy_version 203788 (0.0008) [2023-12-26 16:50:39,800][105620] Updated weights for policy 1, policy_version 203798 (0.0008) [2023-12-26 16:50:39,868][105620] Updated weights for policy 1, policy_version 203808 (0.0010) [2023-12-26 16:50:40,510][105692] Updated weights for policy 0, policy_version 203102 (0.0009) [2023-12-26 16:50:40,574][105692] Updated weights for policy 0, policy_version 203112 (0.0011) [2023-12-26 16:50:40,589][105620] Updated weights for policy 1, policy_version 203818 (0.0007) [2023-12-26 16:50:40,637][105692] Updated weights for policy 0, policy_version 203122 (0.0011) [2023-12-26 16:50:40,646][105620] Updated weights for policy 1, policy_version 203828 (0.0010) [2023-12-26 16:50:40,708][105620] Updated weights for policy 1, policy_version 203838 (0.0007) [2023-12-26 16:50:40,780][105620] Updated weights for policy 1, policy_version 203848 (0.0008) [2023-12-26 16:50:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 104202240. Throughput: 0: 9798.8, 1: 9767.3. Samples: 104210352. Policy #0 lag: (min: 17.0, avg: 45.1, max: 49.0) [2023-12-26 16:50:41,063][104569] Avg episode reward: [(0, '9347.001'), (1, '9353.666')] [2023-12-26 16:50:41,393][105692] Updated weights for policy 0, policy_version 203132 (0.0010) [2023-12-26 16:50:41,450][105692] Updated weights for policy 0, policy_version 203142 (0.0006) [2023-12-26 16:50:41,507][105692] Updated weights for policy 0, policy_version 203152 (0.0009) [2023-12-26 16:50:41,518][105620] Updated weights for policy 1, policy_version 203858 (0.0006) [2023-12-26 16:50:41,578][105620] Updated weights for policy 1, policy_version 203868 (0.0006) [2023-12-26 16:50:41,641][105620] Updated weights for policy 1, policy_version 203878 (0.0007) [2023-12-26 16:50:42,130][105692] Updated weights for policy 0, policy_version 203162 (0.0008) [2023-12-26 16:50:42,188][105692] Updated weights for policy 0, policy_version 203172 (0.0006) [2023-12-26 16:50:42,240][105692] Updated weights for policy 0, policy_version 203182 (0.0009) [2023-12-26 16:50:42,301][105692] Updated weights for policy 0, policy_version 203192 (0.0009) [2023-12-26 16:50:42,368][105620] Updated weights for policy 1, policy_version 203888 (0.0009) [2023-12-26 16:50:42,424][105620] Updated weights for policy 1, policy_version 203898 (0.0008) [2023-12-26 16:50:42,486][105620] Updated weights for policy 1, policy_version 203908 (0.0009) [2023-12-26 16:50:43,062][105692] Updated weights for policy 0, policy_version 203202 (0.0009) [2023-12-26 16:50:43,117][105692] Updated weights for policy 0, policy_version 203212 (0.0009) [2023-12-26 16:50:43,164][105692] Updated weights for policy 0, policy_version 203222 (0.0008) [2023-12-26 16:50:43,214][105620] Updated weights for policy 1, policy_version 203918 (0.0009) [2023-12-26 16:50:43,270][105620] Updated weights for policy 1, policy_version 203928 (0.0009) [2023-12-26 16:50:43,331][105620] Updated weights for policy 1, policy_version 203938 (0.0008) [2023-12-26 16:50:43,911][105692] Updated weights for policy 0, policy_version 203232 (0.0007) [2023-12-26 16:50:43,973][105692] Updated weights for policy 0, policy_version 203242 (0.0008) [2023-12-26 16:50:44,040][105692] Updated weights for policy 0, policy_version 203252 (0.0008) [2023-12-26 16:50:44,116][105620] Updated weights for policy 1, policy_version 203948 (0.0010) [2023-12-26 16:50:44,171][105620] Updated weights for policy 1, policy_version 203958 (0.0009) [2023-12-26 16:50:44,231][105620] Updated weights for policy 1, policy_version 203968 (0.0009) [2023-12-26 16:50:44,643][105692] Updated weights for policy 0, policy_version 203262 (0.0009) [2023-12-26 16:50:44,696][105692] Updated weights for policy 0, policy_version 203272 (0.0010) [2023-12-26 16:50:44,754][105692] Updated weights for policy 0, policy_version 203282 (0.0010) [2023-12-26 16:50:44,998][105620] Updated weights for policy 1, policy_version 203978 (0.0009) [2023-12-26 16:50:45,068][105620] Updated weights for policy 1, policy_version 203988 (0.0008) [2023-12-26 16:50:45,135][105620] Updated weights for policy 1, policy_version 203998 (0.0009) [2023-12-26 16:50:45,200][105620] Updated weights for policy 1, policy_version 204008 (0.0009) [2023-12-26 16:50:45,593][105692] Updated weights for policy 0, policy_version 203292 (0.0011) [2023-12-26 16:50:45,653][105692] Updated weights for policy 0, policy_version 203302 (0.0010) [2023-12-26 16:50:45,701][105692] Updated weights for policy 0, policy_version 203312 (0.0009) [2023-12-26 16:50:45,925][105620] Updated weights for policy 1, policy_version 204018 (0.0006) [2023-12-26 16:50:45,977][105620] Updated weights for policy 1, policy_version 204028 (0.0008) [2023-12-26 16:50:46,027][105620] Updated weights for policy 1, policy_version 204038 (0.0008) [2023-12-26 16:50:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 104300544. Throughput: 0: 9755.8, 1: 9719.5. Samples: 104267592. Policy #0 lag: (min: 17.0, avg: 45.1, max: 49.0) [2023-12-26 16:50:46,063][104569] Avg episode reward: [(0, '9350.813'), (1, '9263.508')] [2023-12-26 16:50:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000203320_52060160.pth... [2023-12-26 16:50:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000204040_52240384.pth... [2023-12-26 16:50:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000202200_51773440.pth [2023-12-26 16:50:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000202888_51945472.pth [2023-12-26 16:50:46,426][105692] Updated weights for policy 0, policy_version 203322 (0.0010) [2023-12-26 16:50:46,484][105692] Updated weights for policy 0, policy_version 203332 (0.0011) [2023-12-26 16:50:46,545][105692] Updated weights for policy 0, policy_version 203342 (0.0010) [2023-12-26 16:50:46,594][105692] Updated weights for policy 0, policy_version 203352 (0.0010) [2023-12-26 16:50:46,780][105620] Updated weights for policy 1, policy_version 204048 (0.0007) [2023-12-26 16:50:46,826][105620] Updated weights for policy 1, policy_version 204058 (0.0005) [2023-12-26 16:50:46,880][105620] Updated weights for policy 1, policy_version 204068 (0.0008) [2023-12-26 16:50:47,237][105692] Updated weights for policy 0, policy_version 203362 (0.0007) [2023-12-26 16:50:47,295][105692] Updated weights for policy 0, policy_version 203372 (0.0009) [2023-12-26 16:50:47,356][105692] Updated weights for policy 0, policy_version 203382 (0.0008) [2023-12-26 16:50:47,681][105620] Updated weights for policy 1, policy_version 204078 (0.0009) [2023-12-26 16:50:47,732][105620] Updated weights for policy 1, policy_version 204088 (0.0009) [2023-12-26 16:50:47,789][105620] Updated weights for policy 1, policy_version 204098 (0.0008) [2023-12-26 16:50:48,019][105692] Updated weights for policy 0, policy_version 203392 (0.0009) [2023-12-26 16:50:48,078][105692] Updated weights for policy 0, policy_version 203402 (0.0009) [2023-12-26 16:50:48,129][105692] Updated weights for policy 0, policy_version 203412 (0.0009) [2023-12-26 16:50:48,611][105620] Updated weights for policy 1, policy_version 204108 (0.0009) [2023-12-26 16:50:48,668][105620] Updated weights for policy 1, policy_version 204118 (0.0008) [2023-12-26 16:50:48,731][105620] Updated weights for policy 1, policy_version 204128 (0.0009) [2023-12-26 16:50:48,848][105692] Updated weights for policy 0, policy_version 203422 (0.0007) [2023-12-26 16:50:48,915][105692] Updated weights for policy 0, policy_version 203432 (0.0006) [2023-12-26 16:50:48,973][105692] Updated weights for policy 0, policy_version 203442 (0.0007) [2023-12-26 16:50:49,482][105620] Updated weights for policy 1, policy_version 204138 (0.0010) [2023-12-26 16:50:49,546][105620] Updated weights for policy 1, policy_version 204148 (0.0010) [2023-12-26 16:50:49,610][105620] Updated weights for policy 1, policy_version 204158 (0.0009) [2023-12-26 16:50:49,637][105692] Updated weights for policy 0, policy_version 203452 (0.0008) [2023-12-26 16:50:49,667][105620] Updated weights for policy 1, policy_version 204168 (0.0007) [2023-12-26 16:50:49,699][105692] Updated weights for policy 0, policy_version 203463 (0.0009) [2023-12-26 16:50:49,759][105692] Updated weights for policy 0, policy_version 203473 (0.0005) [2023-12-26 16:50:50,361][105620] Updated weights for policy 1, policy_version 204178 (0.0008) [2023-12-26 16:50:50,416][105620] Updated weights for policy 1, policy_version 204188 (0.0009) [2023-12-26 16:50:50,471][105620] Updated weights for policy 1, policy_version 204198 (0.0009) [2023-12-26 16:50:50,472][105692] Updated weights for policy 0, policy_version 203483 (0.0006) [2023-12-26 16:50:50,525][105692] Updated weights for policy 0, policy_version 203493 (0.0009) [2023-12-26 16:50:50,587][105692] Updated weights for policy 0, policy_version 203503 (0.0007) [2023-12-26 16:50:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 104390656. Throughput: 0: 9813.6, 1: 9636.7. Samples: 104381872. Policy #0 lag: (min: 17.0, avg: 45.1, max: 49.0) [2023-12-26 16:50:51,062][104569] Avg episode reward: [(0, '9354.344'), (1, '9263.620')] [2023-12-26 16:50:51,213][105620] Updated weights for policy 1, policy_version 204208 (0.0009) [2023-12-26 16:50:51,277][105620] Updated weights for policy 1, policy_version 204218 (0.0008) [2023-12-26 16:50:51,337][105620] Updated weights for policy 1, policy_version 204228 (0.0009) [2023-12-26 16:50:51,369][105692] Updated weights for policy 0, policy_version 203513 (0.0010) [2023-12-26 16:50:51,431][105692] Updated weights for policy 0, policy_version 203523 (0.0008) [2023-12-26 16:50:51,493][105692] Updated weights for policy 0, policy_version 203533 (0.0009) [2023-12-26 16:50:51,552][105692] Updated weights for policy 0, policy_version 203543 (0.0009) [2023-12-26 16:50:52,111][105620] Updated weights for policy 1, policy_version 204238 (0.0009) [2023-12-26 16:50:52,176][105620] Updated weights for policy 1, policy_version 204248 (0.0009) [2023-12-26 16:50:52,238][105620] Updated weights for policy 1, policy_version 204258 (0.0009) [2023-12-26 16:50:52,311][105692] Updated weights for policy 0, policy_version 203553 (0.0009) [2023-12-26 16:50:52,378][105692] Updated weights for policy 0, policy_version 203563 (0.0008) [2023-12-26 16:50:52,442][105692] Updated weights for policy 0, policy_version 203573 (0.0009) [2023-12-26 16:50:52,892][105620] Updated weights for policy 1, policy_version 204268 (0.0007) [2023-12-26 16:50:52,957][105620] Updated weights for policy 1, policy_version 204278 (0.0010) [2023-12-26 16:50:53,024][105620] Updated weights for policy 1, policy_version 204288 (0.0010) [2023-12-26 16:50:53,220][105692] Updated weights for policy 0, policy_version 203583 (0.0010) [2023-12-26 16:50:53,265][105692] Updated weights for policy 0, policy_version 203593 (0.0010) [2023-12-26 16:50:53,324][105692] Updated weights for policy 0, policy_version 203603 (0.0009) [2023-12-26 16:50:53,630][105620] Updated weights for policy 1, policy_version 204298 (0.0009) [2023-12-26 16:50:53,681][105620] Updated weights for policy 1, policy_version 204308 (0.0009) [2023-12-26 16:50:53,738][105620] Updated weights for policy 1, policy_version 204318 (0.0010) [2023-12-26 16:50:53,786][105620] Updated weights for policy 1, policy_version 204328 (0.0010) [2023-12-26 16:50:54,129][105692] Updated weights for policy 0, policy_version 203613 (0.0009) [2023-12-26 16:50:54,191][105692] Updated weights for policy 0, policy_version 203623 (0.0010) [2023-12-26 16:50:54,241][105692] Updated weights for policy 0, policy_version 203633 (0.0010) [2023-12-26 16:50:54,544][105620] Updated weights for policy 1, policy_version 204338 (0.0005) [2023-12-26 16:50:54,611][105620] Updated weights for policy 1, policy_version 204348 (0.0007) [2023-12-26 16:50:54,666][105620] Updated weights for policy 1, policy_version 204358 (0.0011) [2023-12-26 16:50:54,856][105692] Updated weights for policy 0, policy_version 203643 (0.0010) [2023-12-26 16:50:54,903][105692] Updated weights for policy 0, policy_version 203653 (0.0010) [2023-12-26 16:50:54,963][105692] Updated weights for policy 0, policy_version 203663 (0.0010) [2023-12-26 16:50:55,404][105620] Updated weights for policy 1, policy_version 204368 (0.0008) [2023-12-26 16:50:55,458][105620] Updated weights for policy 1, policy_version 204378 (0.0010) [2023-12-26 16:50:55,514][105620] Updated weights for policy 1, policy_version 204388 (0.0010) [2023-12-26 16:50:55,721][105692] Updated weights for policy 0, policy_version 203673 (0.0010) [2023-12-26 16:50:55,786][105692] Updated weights for policy 0, policy_version 203683 (0.0010) [2023-12-26 16:50:55,841][105692] Updated weights for policy 0, policy_version 203693 (0.0010) [2023-12-26 16:50:55,889][105692] Updated weights for policy 0, policy_version 203703 (0.0010) [2023-12-26 16:50:56,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 104488960. Throughput: 0: 9755.8, 1: 9664.5. Samples: 104496832. Policy #0 lag: (min: 17.0, avg: 45.1, max: 49.0) [2023-12-26 16:50:56,063][104569] Avg episode reward: [(0, '9354.487'), (1, '9175.234')] [2023-12-26 16:50:56,286][105620] Updated weights for policy 1, policy_version 204398 (0.0009) [2023-12-26 16:50:56,341][105620] Updated weights for policy 1, policy_version 204408 (0.0008) [2023-12-26 16:50:56,396][105620] Updated weights for policy 1, policy_version 204418 (0.0008) [2023-12-26 16:50:56,629][105692] Updated weights for policy 0, policy_version 203713 (0.0010) [2023-12-26 16:50:56,673][105692] Updated weights for policy 0, policy_version 203723 (0.0010) [2023-12-26 16:50:56,720][105692] Updated weights for policy 0, policy_version 203733 (0.0010) [2023-12-26 16:50:57,093][105620] Updated weights for policy 1, policy_version 204428 (0.0007) [2023-12-26 16:50:57,153][105620] Updated weights for policy 1, policy_version 204438 (0.0008) [2023-12-26 16:50:57,216][105620] Updated weights for policy 1, policy_version 204448 (0.0008) [2023-12-26 16:50:57,463][105692] Updated weights for policy 0, policy_version 203743 (0.0010) [2023-12-26 16:50:57,523][105692] Updated weights for policy 0, policy_version 203753 (0.0010) [2023-12-26 16:50:57,577][105692] Updated weights for policy 0, policy_version 203763 (0.0010) [2023-12-26 16:50:57,931][105620] Updated weights for policy 1, policy_version 204458 (0.0008) [2023-12-26 16:50:57,983][105620] Updated weights for policy 1, policy_version 204468 (0.0008) [2023-12-26 16:50:58,030][105620] Updated weights for policy 1, policy_version 204478 (0.0006) [2023-12-26 16:50:58,081][105620] Updated weights for policy 1, policy_version 204488 (0.0008) [2023-12-26 16:50:58,291][105692] Updated weights for policy 0, policy_version 203773 (0.0010) [2023-12-26 16:50:58,362][105692] Updated weights for policy 0, policy_version 203783 (0.0008) [2023-12-26 16:50:58,424][105692] Updated weights for policy 0, policy_version 203793 (0.0009) [2023-12-26 16:50:58,954][105620] Updated weights for policy 1, policy_version 204498 (0.0008) [2023-12-26 16:50:59,015][105620] Updated weights for policy 1, policy_version 204508 (0.0008) [2023-12-26 16:50:59,068][105620] Updated weights for policy 1, policy_version 204518 (0.0008) [2023-12-26 16:50:59,242][105692] Updated weights for policy 0, policy_version 203803 (0.0009) [2023-12-26 16:50:59,307][105692] Updated weights for policy 0, policy_version 203813 (0.0010) [2023-12-26 16:50:59,377][105692] Updated weights for policy 0, policy_version 203823 (0.0008) [2023-12-26 16:50:59,865][105620] Updated weights for policy 1, policy_version 204528 (0.0008) [2023-12-26 16:50:59,931][105620] Updated weights for policy 1, policy_version 204538 (0.0008) [2023-12-26 16:50:59,982][105620] Updated weights for policy 1, policy_version 204548 (0.0008) [2023-12-26 16:51:00,038][105692] Updated weights for policy 0, policy_version 203833 (0.0005) [2023-12-26 16:51:00,097][105692] Updated weights for policy 0, policy_version 203843 (0.0006) [2023-12-26 16:51:00,155][105692] Updated weights for policy 0, policy_version 203853 (0.0006) [2023-12-26 16:51:00,223][105692] Updated weights for policy 0, policy_version 203863 (0.0006) [2023-12-26 16:51:00,687][105620] Updated weights for policy 1, policy_version 204558 (0.0006) [2023-12-26 16:51:00,756][105620] Updated weights for policy 1, policy_version 204568 (0.0005) [2023-12-26 16:51:00,817][105620] Updated weights for policy 1, policy_version 204578 (0.0005) [2023-12-26 16:51:00,864][105692] Updated weights for policy 0, policy_version 203873 (0.0010) [2023-12-26 16:51:00,915][105692] Updated weights for policy 0, policy_version 203883 (0.0010) [2023-12-26 16:51:00,963][105692] Updated weights for policy 0, policy_version 203893 (0.0010) [2023-12-26 16:51:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 104587264. Throughput: 0: 9718.2, 1: 9641.3. Samples: 104553072. Policy #0 lag: (min: 17.0, avg: 45.1, max: 49.0) [2023-12-26 16:51:01,063][104569] Avg episode reward: [(0, '9353.663'), (1, '8906.012')] [2023-12-26 16:51:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000203896_52207616.pth... [2023-12-26 16:51:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000204584_52379648.pth... [2023-12-26 16:51:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000203464_52092928.pth [2023-12-26 16:51:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000202744_51912704.pth [2023-12-26 16:51:01,425][105620] Updated weights for policy 1, policy_version 204588 (0.0006) [2023-12-26 16:51:01,483][105620] Updated weights for policy 1, policy_version 204598 (0.0005) [2023-12-26 16:51:01,536][105620] Updated weights for policy 1, policy_version 204608 (0.0006) [2023-12-26 16:51:01,728][105692] Updated weights for policy 0, policy_version 203903 (0.0009) [2023-12-26 16:51:01,778][105692] Updated weights for policy 0, policy_version 203913 (0.0008) [2023-12-26 16:51:01,832][105692] Updated weights for policy 0, policy_version 203923 (0.0008) [2023-12-26 16:51:02,297][105620] Updated weights for policy 1, policy_version 204618 (0.0006) [2023-12-26 16:51:02,360][105620] Updated weights for policy 1, policy_version 204628 (0.0008) [2023-12-26 16:51:02,415][105620] Updated weights for policy 1, policy_version 204638 (0.0008) [2023-12-26 16:51:02,470][105620] Updated weights for policy 1, policy_version 204648 (0.0008) [2023-12-26 16:51:02,530][105692] Updated weights for policy 0, policy_version 203933 (0.0007) [2023-12-26 16:51:02,578][105692] Updated weights for policy 0, policy_version 203943 (0.0005) [2023-12-26 16:51:02,625][105692] Updated weights for policy 0, policy_version 203953 (0.0010) [2023-12-26 16:51:03,175][105620] Updated weights for policy 1, policy_version 204658 (0.0010) [2023-12-26 16:51:03,236][105620] Updated weights for policy 1, policy_version 204668 (0.0010) [2023-12-26 16:51:03,284][105620] Updated weights for policy 1, policy_version 204678 (0.0008) [2023-12-26 16:51:03,311][105692] Updated weights for policy 0, policy_version 203963 (0.0011) [2023-12-26 16:51:03,368][105692] Updated weights for policy 0, policy_version 203973 (0.0010) [2023-12-26 16:51:03,436][105692] Updated weights for policy 0, policy_version 203983 (0.0010) [2023-12-26 16:51:03,922][105620] Updated weights for policy 1, policy_version 204688 (0.0007) [2023-12-26 16:51:03,991][105620] Updated weights for policy 1, policy_version 204698 (0.0006) [2023-12-26 16:51:04,061][105620] Updated weights for policy 1, policy_version 204708 (0.0006) [2023-12-26 16:51:04,184][105692] Updated weights for policy 0, policy_version 203993 (0.0010) [2023-12-26 16:51:04,254][105692] Updated weights for policy 0, policy_version 204003 (0.0008) [2023-12-26 16:51:04,321][105692] Updated weights for policy 0, policy_version 204013 (0.0008) [2023-12-26 16:51:04,380][105692] Updated weights for policy 0, policy_version 204023 (0.0008) [2023-12-26 16:51:04,682][105620] Updated weights for policy 1, policy_version 204718 (0.0011) [2023-12-26 16:51:04,738][105620] Updated weights for policy 1, policy_version 204728 (0.0011) [2023-12-26 16:51:04,794][105620] Updated weights for policy 1, policy_version 204738 (0.0011) [2023-12-26 16:51:05,124][105692] Updated weights for policy 0, policy_version 204033 (0.0010) [2023-12-26 16:51:05,185][105692] Updated weights for policy 0, policy_version 204043 (0.0010) [2023-12-26 16:51:05,229][105692] Updated weights for policy 0, policy_version 204053 (0.0010) [2023-12-26 16:51:05,558][105620] Updated weights for policy 1, policy_version 204748 (0.0011) [2023-12-26 16:51:05,629][105620] Updated weights for policy 1, policy_version 204758 (0.0010) [2023-12-26 16:51:05,683][105620] Updated weights for policy 1, policy_version 204768 (0.0010) [2023-12-26 16:51:05,921][105692] Updated weights for policy 0, policy_version 204063 (0.0009) [2023-12-26 16:51:05,973][105692] Updated weights for policy 0, policy_version 204073 (0.0008) [2023-12-26 16:51:06,020][105692] Updated weights for policy 0, policy_version 204083 (0.0007) [2023-12-26 16:51:06,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 104685568. Throughput: 0: 9654.3, 1: 9630.8. Samples: 104671168. Policy #0 lag: (min: 17.0, avg: 45.1, max: 49.0) [2023-12-26 16:51:06,063][104569] Avg episode reward: [(0, '9350.784'), (1, '9084.406')] [2023-12-26 16:51:06,427][105620] Updated weights for policy 1, policy_version 204778 (0.0010) [2023-12-26 16:51:06,483][105620] Updated weights for policy 1, policy_version 204788 (0.0011) [2023-12-26 16:51:06,536][105620] Updated weights for policy 1, policy_version 204798 (0.0011) [2023-12-26 16:51:06,593][105620] Updated weights for policy 1, policy_version 204808 (0.0011) [2023-12-26 16:51:06,774][105692] Updated weights for policy 0, policy_version 204093 (0.0008) [2023-12-26 16:51:06,829][105692] Updated weights for policy 0, policy_version 204103 (0.0008) [2023-12-26 16:51:06,883][105692] Updated weights for policy 0, policy_version 204113 (0.0009) [2023-12-26 16:51:07,329][105620] Updated weights for policy 1, policy_version 204818 (0.0009) [2023-12-26 16:51:07,381][105620] Updated weights for policy 1, policy_version 204828 (0.0007) [2023-12-26 16:51:07,454][105620] Updated weights for policy 1, policy_version 204838 (0.0007) [2023-12-26 16:51:07,612][105692] Updated weights for policy 0, policy_version 204123 (0.0007) [2023-12-26 16:51:07,671][105692] Updated weights for policy 0, policy_version 204133 (0.0008) [2023-12-26 16:51:07,732][105692] Updated weights for policy 0, policy_version 204143 (0.0009) [2023-12-26 16:51:08,148][105620] Updated weights for policy 1, policy_version 204848 (0.0006) [2023-12-26 16:51:08,211][105620] Updated weights for policy 1, policy_version 204858 (0.0008) [2023-12-26 16:51:08,260][105620] Updated weights for policy 1, policy_version 204868 (0.0008) [2023-12-26 16:51:08,421][105692] Updated weights for policy 0, policy_version 204153 (0.0007) [2023-12-26 16:51:08,485][105692] Updated weights for policy 0, policy_version 204163 (0.0008) [2023-12-26 16:51:08,542][105692] Updated weights for policy 0, policy_version 204173 (0.0008) [2023-12-26 16:51:08,606][105692] Updated weights for policy 0, policy_version 204183 (0.0008) [2023-12-26 16:51:08,983][105620] Updated weights for policy 1, policy_version 204878 (0.0008) [2023-12-26 16:51:09,041][105620] Updated weights for policy 1, policy_version 204888 (0.0010) [2023-12-26 16:51:09,099][105620] Updated weights for policy 1, policy_version 204898 (0.0010) [2023-12-26 16:51:09,252][105692] Updated weights for policy 0, policy_version 204193 (0.0008) [2023-12-26 16:51:09,312][105692] Updated weights for policy 0, policy_version 204203 (0.0008) [2023-12-26 16:51:09,385][105692] Updated weights for policy 0, policy_version 204213 (0.0009) [2023-12-26 16:51:09,881][105620] Updated weights for policy 1, policy_version 204908 (0.0009) [2023-12-26 16:51:09,950][105620] Updated weights for policy 1, policy_version 204918 (0.0008) [2023-12-26 16:51:10,020][105620] Updated weights for policy 1, policy_version 204928 (0.0009) [2023-12-26 16:51:10,189][105692] Updated weights for policy 0, policy_version 204223 (0.0007) [2023-12-26 16:51:10,257][105692] Updated weights for policy 0, policy_version 204233 (0.0006) [2023-12-26 16:51:10,321][105692] Updated weights for policy 0, policy_version 204243 (0.0007) [2023-12-26 16:51:10,763][105620] Updated weights for policy 1, policy_version 204938 (0.0008) [2023-12-26 16:51:10,824][105620] Updated weights for policy 1, policy_version 204948 (0.0006) [2023-12-26 16:51:10,889][105620] Updated weights for policy 1, policy_version 204958 (0.0007) [2023-12-26 16:51:10,954][105620] Updated weights for policy 1, policy_version 204968 (0.0009) [2023-12-26 16:51:10,989][105692] Updated weights for policy 0, policy_version 204253 (0.0009) [2023-12-26 16:51:11,060][105692] Updated weights for policy 0, policy_version 204263 (0.0007) [2023-12-26 16:51:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 104775680. Throughput: 0: 9635.5, 1: 9576.2. Samples: 104786076. Policy #0 lag: (min: 17.0, avg: 45.1, max: 49.0) [2023-12-26 16:51:11,062][104569] Avg episode reward: [(0, '9268.274'), (1, '9174.216')] [2023-12-26 16:51:11,120][105692] Updated weights for policy 0, policy_version 204273 (0.0008) [2023-12-26 16:51:11,661][105620] Updated weights for policy 1, policy_version 204978 (0.0006) [2023-12-26 16:51:11,730][105620] Updated weights for policy 1, policy_version 204988 (0.0007) [2023-12-26 16:51:11,804][105620] Updated weights for policy 1, policy_version 204998 (0.0009) [2023-12-26 16:51:11,854][105692] Updated weights for policy 0, policy_version 204283 (0.0009) [2023-12-26 16:51:11,907][105692] Updated weights for policy 0, policy_version 204293 (0.0010) [2023-12-26 16:51:11,970][105692] Updated weights for policy 0, policy_version 204303 (0.0009) [2023-12-26 16:51:12,479][105620] Updated weights for policy 1, policy_version 205008 (0.0009) [2023-12-26 16:51:12,540][105620] Updated weights for policy 1, policy_version 205018 (0.0008) [2023-12-26 16:51:12,602][105620] Updated weights for policy 1, policy_version 205028 (0.0006) [2023-12-26 16:51:12,749][105692] Updated weights for policy 0, policy_version 204313 (0.0008) [2023-12-26 16:51:12,817][105692] Updated weights for policy 0, policy_version 204323 (0.0006) [2023-12-26 16:51:12,880][105692] Updated weights for policy 0, policy_version 204333 (0.0006) [2023-12-26 16:51:12,949][105692] Updated weights for policy 0, policy_version 204343 (0.0006) [2023-12-26 16:51:13,294][105620] Updated weights for policy 1, policy_version 205038 (0.0007) [2023-12-26 16:51:13,360][105620] Updated weights for policy 1, policy_version 205048 (0.0005) [2023-12-26 16:51:13,425][105620] Updated weights for policy 1, policy_version 205058 (0.0005) [2023-12-26 16:51:13,618][105692] Updated weights for policy 0, policy_version 204353 (0.0010) [2023-12-26 16:51:13,676][105692] Updated weights for policy 0, policy_version 204363 (0.0010) [2023-12-26 16:51:13,743][105692] Updated weights for policy 0, policy_version 204373 (0.0010) [2023-12-26 16:51:13,920][105620] Updated weights for policy 1, policy_version 205068 (0.0005) [2023-12-26 16:51:13,981][105620] Updated weights for policy 1, policy_version 205078 (0.0006) [2023-12-26 16:51:14,043][105620] Updated weights for policy 1, policy_version 205088 (0.0010) [2023-12-26 16:51:14,473][105692] Updated weights for policy 0, policy_version 204383 (0.0009) [2023-12-26 16:51:14,530][105692] Updated weights for policy 0, policy_version 204393 (0.0010) [2023-12-26 16:51:14,592][105692] Updated weights for policy 0, policy_version 204403 (0.0010) [2023-12-26 16:51:14,632][105620] Updated weights for policy 1, policy_version 205098 (0.0010) [2023-12-26 16:51:14,677][105620] Updated weights for policy 1, policy_version 205108 (0.0010) [2023-12-26 16:51:14,727][105620] Updated weights for policy 1, policy_version 205118 (0.0010) [2023-12-26 16:51:14,788][105620] Updated weights for policy 1, policy_version 205128 (0.0011) [2023-12-26 16:51:15,349][105692] Updated weights for policy 0, policy_version 204413 (0.0010) [2023-12-26 16:51:15,408][105692] Updated weights for policy 0, policy_version 204423 (0.0008) [2023-12-26 16:51:15,469][105692] Updated weights for policy 0, policy_version 204433 (0.0005) [2023-12-26 16:51:15,575][105620] Updated weights for policy 1, policy_version 205138 (0.0008) [2023-12-26 16:51:15,637][105620] Updated weights for policy 1, policy_version 205148 (0.0007) [2023-12-26 16:51:15,695][105620] Updated weights for policy 1, policy_version 205158 (0.0008) [2023-12-26 16:51:16,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 104873984. Throughput: 0: 9651.7, 1: 9636.6. Samples: 104845680. Policy #0 lag: (min: 17.0, avg: 45.1, max: 49.0) [2023-12-26 16:51:16,062][104569] Avg episode reward: [(0, '9268.421'), (1, '9353.672')] [2023-12-26 16:51:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000204440_52346880.pth... [2023-12-26 16:51:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000205160_52527104.pth... [2023-12-26 16:51:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000204040_52240384.pth [2023-12-26 16:51:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000203320_52060160.pth [2023-12-26 16:51:16,241][105692] Updated weights for policy 0, policy_version 204443 (0.0007) [2023-12-26 16:51:16,299][105692] Updated weights for policy 0, policy_version 204453 (0.0009) [2023-12-26 16:51:16,352][105692] Updated weights for policy 0, policy_version 204463 (0.0007) [2023-12-26 16:51:16,366][105620] Updated weights for policy 1, policy_version 205168 (0.0007) [2023-12-26 16:51:16,419][105620] Updated weights for policy 1, policy_version 205178 (0.0006) [2023-12-26 16:51:16,477][105620] Updated weights for policy 1, policy_version 205188 (0.0009) [2023-12-26 16:51:17,099][105692] Updated weights for policy 0, policy_version 204473 (0.0009) [2023-12-26 16:51:17,150][105692] Updated weights for policy 0, policy_version 204483 (0.0010) [2023-12-26 16:51:17,191][105620] Updated weights for policy 1, policy_version 205198 (0.0009) [2023-12-26 16:51:17,210][105692] Updated weights for policy 0, policy_version 204493 (0.0006) [2023-12-26 16:51:17,243][105620] Updated weights for policy 1, policy_version 205208 (0.0008) [2023-12-26 16:51:17,265][105692] Updated weights for policy 0, policy_version 204503 (0.0006) [2023-12-26 16:51:17,313][105620] Updated weights for policy 1, policy_version 205218 (0.0009) [2023-12-26 16:51:17,802][105692] Updated weights for policy 0, policy_version 204513 (0.0005) [2023-12-26 16:51:17,851][105692] Updated weights for policy 0, policy_version 204523 (0.0005) [2023-12-26 16:51:17,905][105692] Updated weights for policy 0, policy_version 204533 (0.0011) [2023-12-26 16:51:18,017][105620] Updated weights for policy 1, policy_version 205228 (0.0008) [2023-12-26 16:51:18,080][105620] Updated weights for policy 1, policy_version 205238 (0.0007) [2023-12-26 16:51:18,150][105620] Updated weights for policy 1, policy_version 205248 (0.0008) [2023-12-26 16:51:18,588][105692] Updated weights for policy 0, policy_version 204543 (0.0011) [2023-12-26 16:51:18,643][105692] Updated weights for policy 0, policy_version 204553 (0.0010) [2023-12-26 16:51:18,701][105692] Updated weights for policy 0, policy_version 204563 (0.0011) [2023-12-26 16:51:18,857][105620] Updated weights for policy 1, policy_version 205258 (0.0007) [2023-12-26 16:51:18,913][105620] Updated weights for policy 1, policy_version 205268 (0.0008) [2023-12-26 16:51:18,972][105620] Updated weights for policy 1, policy_version 205278 (0.0008) [2023-12-26 16:51:19,028][105620] Updated weights for policy 1, policy_version 205288 (0.0008) [2023-12-26 16:51:19,449][105692] Updated weights for policy 0, policy_version 204573 (0.0010) [2023-12-26 16:51:19,518][105692] Updated weights for policy 0, policy_version 204583 (0.0008) [2023-12-26 16:51:19,582][105692] Updated weights for policy 0, policy_version 204593 (0.0008) [2023-12-26 16:51:19,831][105620] Updated weights for policy 1, policy_version 205298 (0.0008) [2023-12-26 16:51:19,899][105620] Updated weights for policy 1, policy_version 205308 (0.0008) [2023-12-26 16:51:19,970][105620] Updated weights for policy 1, policy_version 205318 (0.0010) [2023-12-26 16:51:20,370][105692] Updated weights for policy 0, policy_version 204603 (0.0010) [2023-12-26 16:51:20,433][105692] Updated weights for policy 0, policy_version 204613 (0.0008) [2023-12-26 16:51:20,502][105692] Updated weights for policy 0, policy_version 204623 (0.0008) [2023-12-26 16:51:20,768][105620] Updated weights for policy 1, policy_version 205328 (0.0009) [2023-12-26 16:51:20,819][105620] Updated weights for policy 1, policy_version 205338 (0.0009) [2023-12-26 16:51:20,881][105620] Updated weights for policy 1, policy_version 205348 (0.0009) [2023-12-26 16:51:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 104972288. Throughput: 0: 9660.3, 1: 9634.1. Samples: 104962136. Policy #0 lag: (min: 6.0, avg: 6.1, max: 12.0) [2023-12-26 16:51:21,062][104569] Avg episode reward: [(0, '9266.956'), (1, '9179.014')] [2023-12-26 16:51:21,240][105692] Updated weights for policy 0, policy_version 204633 (0.0006) [2023-12-26 16:51:21,307][105692] Updated weights for policy 0, policy_version 204643 (0.0010) [2023-12-26 16:51:21,375][105692] Updated weights for policy 0, policy_version 204653 (0.0008) [2023-12-26 16:51:21,445][105692] Updated weights for policy 0, policy_version 204663 (0.0006) [2023-12-26 16:51:21,693][105620] Updated weights for policy 1, policy_version 205358 (0.0009) [2023-12-26 16:51:21,769][105620] Updated weights for policy 1, policy_version 205368 (0.0008) [2023-12-26 16:51:21,832][105620] Updated weights for policy 1, policy_version 205378 (0.0009) [2023-12-26 16:51:22,098][105692] Updated weights for policy 0, policy_version 204673 (0.0006) [2023-12-26 16:51:22,157][105692] Updated weights for policy 0, policy_version 204683 (0.0006) [2023-12-26 16:51:22,217][105692] Updated weights for policy 0, policy_version 204693 (0.0005) [2023-12-26 16:51:22,656][105620] Updated weights for policy 1, policy_version 205388 (0.0009) [2023-12-26 16:51:22,718][105620] Updated weights for policy 1, policy_version 205398 (0.0009) [2023-12-26 16:51:22,770][105620] Updated weights for policy 1, policy_version 205408 (0.0009) [2023-12-26 16:51:22,856][105692] Updated weights for policy 0, policy_version 204703 (0.0007) [2023-12-26 16:51:22,911][105692] Updated weights for policy 0, policy_version 204713 (0.0008) [2023-12-26 16:51:22,963][105692] Updated weights for policy 0, policy_version 204723 (0.0007) [2023-12-26 16:51:23,546][105620] Updated weights for policy 1, policy_version 205418 (0.0008) [2023-12-26 16:51:23,596][105620] Updated weights for policy 1, policy_version 205428 (0.0006) [2023-12-26 16:51:23,650][105620] Updated weights for policy 1, policy_version 205438 (0.0007) [2023-12-26 16:51:23,709][105620] Updated weights for policy 1, policy_version 205448 (0.0005) [2023-12-26 16:51:23,718][105692] Updated weights for policy 0, policy_version 204733 (0.0010) [2023-12-26 16:51:23,776][105692] Updated weights for policy 0, policy_version 204743 (0.0010) [2023-12-26 16:51:23,825][105692] Updated weights for policy 0, policy_version 204753 (0.0010) [2023-12-26 16:51:24,324][105620] Updated weights for policy 1, policy_version 205458 (0.0005) [2023-12-26 16:51:24,381][105620] Updated weights for policy 1, policy_version 205468 (0.0006) [2023-12-26 16:51:24,440][105620] Updated weights for policy 1, policy_version 205478 (0.0010) [2023-12-26 16:51:24,589][105692] Updated weights for policy 0, policy_version 204763 (0.0010) [2023-12-26 16:51:24,651][105692] Updated weights for policy 0, policy_version 204773 (0.0010) [2023-12-26 16:51:24,695][105692] Updated weights for policy 0, policy_version 204783 (0.0010) [2023-12-26 16:51:25,031][105620] Updated weights for policy 1, policy_version 205488 (0.0007) [2023-12-26 16:51:25,089][105620] Updated weights for policy 1, policy_version 205498 (0.0007) [2023-12-26 16:51:25,140][105620] Updated weights for policy 1, policy_version 205508 (0.0009) [2023-12-26 16:51:25,443][105692] Updated weights for policy 0, policy_version 204793 (0.0010) [2023-12-26 16:51:25,512][105692] Updated weights for policy 0, policy_version 204803 (0.0007) [2023-12-26 16:51:25,571][105692] Updated weights for policy 0, policy_version 204813 (0.0009) [2023-12-26 16:51:25,622][105692] Updated weights for policy 0, policy_version 204823 (0.0005) [2023-12-26 16:51:25,908][105620] Updated weights for policy 1, policy_version 205518 (0.0007) [2023-12-26 16:51:25,965][105620] Updated weights for policy 1, policy_version 205528 (0.0006) [2023-12-26 16:51:26,024][105620] Updated weights for policy 1, policy_version 205538 (0.0006) [2023-12-26 16:51:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.9, 300 sec: 19522.0). Total num frames: 105070592. Throughput: 0: 9662.2, 1: 9595.1. Samples: 105076928. Policy #0 lag: (min: 6.0, avg: 6.1, max: 12.0) [2023-12-26 16:51:26,062][104569] Avg episode reward: [(0, '9255.862'), (1, '762.416')] [2023-12-26 16:51:26,333][105692] Updated weights for policy 0, policy_version 204833 (0.0005) [2023-12-26 16:51:26,396][105692] Updated weights for policy 0, policy_version 204843 (0.0007) [2023-12-26 16:51:26,458][105692] Updated weights for policy 0, policy_version 204853 (0.0009) [2023-12-26 16:51:26,686][105620] Updated weights for policy 1, policy_version 205548 (0.0006) [2023-12-26 16:51:26,737][105620] Updated weights for policy 1, policy_version 205558 (0.0007) [2023-12-26 16:51:26,804][105620] Updated weights for policy 1, policy_version 205568 (0.0006) [2023-12-26 16:51:27,173][105692] Updated weights for policy 0, policy_version 204863 (0.0009) [2023-12-26 16:51:27,233][105692] Updated weights for policy 0, policy_version 204873 (0.0009) [2023-12-26 16:51:27,295][105692] Updated weights for policy 0, policy_version 204883 (0.0009) [2023-12-26 16:51:27,443][105620] Updated weights for policy 1, policy_version 205578 (0.0008) [2023-12-26 16:51:27,506][105620] Updated weights for policy 1, policy_version 205588 (0.0006) [2023-12-26 16:51:27,562][105620] Updated weights for policy 1, policy_version 205598 (0.0007) [2023-12-26 16:51:27,607][105620] Updated weights for policy 1, policy_version 205608 (0.0010) [2023-12-26 16:51:27,869][105692] Updated weights for policy 0, policy_version 204893 (0.0005) [2023-12-26 16:51:27,932][105692] Updated weights for policy 0, policy_version 204903 (0.0005) [2023-12-26 16:51:27,993][105692] Updated weights for policy 0, policy_version 204913 (0.0005) [2023-12-26 16:51:28,268][105620] Updated weights for policy 1, policy_version 205618 (0.0005) [2023-12-26 16:51:28,323][105620] Updated weights for policy 1, policy_version 205628 (0.0005) [2023-12-26 16:51:28,382][105620] Updated weights for policy 1, policy_version 205638 (0.0007) [2023-12-26 16:51:28,702][105692] Updated weights for policy 0, policy_version 204923 (0.0008) [2023-12-26 16:51:28,755][105692] Updated weights for policy 0, policy_version 204933 (0.0010) [2023-12-26 16:51:28,812][105692] Updated weights for policy 0, policy_version 204943 (0.0010) [2023-12-26 16:51:28,909][105620] Updated weights for policy 1, policy_version 205648 (0.0005) [2023-12-26 16:51:28,965][105620] Updated weights for policy 1, policy_version 205658 (0.0006) [2023-12-26 16:51:29,020][105620] Updated weights for policy 1, policy_version 205668 (0.0005) [2023-12-26 16:51:29,532][105692] Updated weights for policy 0, policy_version 204953 (0.0009) [2023-12-26 16:51:29,583][105692] Updated weights for policy 0, policy_version 204963 (0.0008) [2023-12-26 16:51:29,628][105692] Updated weights for policy 0, policy_version 204973 (0.0008) [2023-12-26 16:51:29,692][105692] Updated weights for policy 0, policy_version 204983 (0.0006) [2023-12-26 16:51:29,724][105620] Updated weights for policy 1, policy_version 205678 (0.0008) [2023-12-26 16:51:29,780][105620] Updated weights for policy 1, policy_version 205688 (0.0011) [2023-12-26 16:51:29,841][105620] Updated weights for policy 1, policy_version 205698 (0.0011) [2023-12-26 16:51:30,400][105692] Updated weights for policy 0, policy_version 204993 (0.0010) [2023-12-26 16:51:30,466][105692] Updated weights for policy 0, policy_version 205003 (0.0007) [2023-12-26 16:51:30,478][105620] Updated weights for policy 1, policy_version 205708 (0.0009) [2023-12-26 16:51:30,518][105692] Updated weights for policy 0, policy_version 205013 (0.0006) [2023-12-26 16:51:30,534][105620] Updated weights for policy 1, policy_version 205718 (0.0005) [2023-12-26 16:51:30,589][105620] Updated weights for policy 1, policy_version 205728 (0.0005) [2023-12-26 16:51:31,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 105168896. Throughput: 0: 9702.2, 1: 9688.7. Samples: 105140180. Policy #0 lag: (min: 6.0, avg: 6.1, max: 12.0) [2023-12-26 16:51:31,063][104569] Avg episode reward: [(0, '9254.931'), (1, '2008.258')] [2023-12-26 16:51:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000205016_52494336.pth... [2023-12-26 16:51:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000205736_52674560.pth... [2023-12-26 16:51:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000204584_52379648.pth [2023-12-26 16:51:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000203896_52207616.pth [2023-12-26 16:51:31,185][105692] Updated weights for policy 0, policy_version 205023 (0.0007) [2023-12-26 16:51:31,199][105620] Updated weights for policy 1, policy_version 205738 (0.0006) [2023-12-26 16:51:31,238][105692] Updated weights for policy 0, policy_version 205033 (0.0005) [2023-12-26 16:51:31,258][105620] Updated weights for policy 1, policy_version 205748 (0.0010) [2023-12-26 16:51:31,296][105692] Updated weights for policy 0, policy_version 205043 (0.0008) [2023-12-26 16:51:31,317][105620] Updated weights for policy 1, policy_version 205758 (0.0011) [2023-12-26 16:51:31,381][105620] Updated weights for policy 1, policy_version 205768 (0.0010) [2023-12-26 16:51:31,900][105692] Updated weights for policy 0, policy_version 205053 (0.0007) [2023-12-26 16:51:31,960][105692] Updated weights for policy 0, policy_version 205063 (0.0009) [2023-12-26 16:51:32,019][105692] Updated weights for policy 0, policy_version 205073 (0.0009) [2023-12-26 16:51:32,122][105620] Updated weights for policy 1, policy_version 205778 (0.0009) [2023-12-26 16:51:32,172][105620] Updated weights for policy 1, policy_version 205788 (0.0009) [2023-12-26 16:51:32,228][105620] Updated weights for policy 1, policy_version 205798 (0.0009) [2023-12-26 16:51:32,736][105692] Updated weights for policy 0, policy_version 205083 (0.0008) [2023-12-26 16:51:32,784][105692] Updated weights for policy 0, policy_version 205093 (0.0009) [2023-12-26 16:51:32,831][105692] Updated weights for policy 0, policy_version 205103 (0.0008) [2023-12-26 16:51:32,992][105620] Updated weights for policy 1, policy_version 205808 (0.0007) [2023-12-26 16:51:33,050][105620] Updated weights for policy 1, policy_version 205818 (0.0005) [2023-12-26 16:51:33,105][105620] Updated weights for policy 1, policy_version 205828 (0.0006) [2023-12-26 16:51:33,619][105692] Updated weights for policy 0, policy_version 205113 (0.0009) [2023-12-26 16:51:33,676][105692] Updated weights for policy 0, policy_version 205123 (0.0010) [2023-12-26 16:51:33,720][105692] Updated weights for policy 0, policy_version 205133 (0.0010) [2023-12-26 16:51:33,776][105620] Updated weights for policy 1, policy_version 205838 (0.0009) [2023-12-26 16:51:33,783][105692] Updated weights for policy 0, policy_version 205143 (0.0010) [2023-12-26 16:51:33,833][105620] Updated weights for policy 1, policy_version 205848 (0.0010) [2023-12-26 16:51:33,891][105620] Updated weights for policy 1, policy_version 205858 (0.0010) [2023-12-26 16:51:34,503][105692] Updated weights for policy 0, policy_version 205153 (0.0010) [2023-12-26 16:51:34,563][105692] Updated weights for policy 0, policy_version 205163 (0.0010) [2023-12-26 16:51:34,625][105692] Updated weights for policy 0, policy_version 205173 (0.0010) [2023-12-26 16:51:34,667][105620] Updated weights for policy 1, policy_version 205868 (0.0008) [2023-12-26 16:51:34,718][105620] Updated weights for policy 1, policy_version 205878 (0.0008) [2023-12-26 16:51:34,774][105620] Updated weights for policy 1, policy_version 205888 (0.0008) [2023-12-26 16:51:35,370][105692] Updated weights for policy 0, policy_version 205183 (0.0010) [2023-12-26 16:51:35,428][105692] Updated weights for policy 0, policy_version 205193 (0.0010) [2023-12-26 16:51:35,459][105620] Updated weights for policy 1, policy_version 205898 (0.0007) [2023-12-26 16:51:35,480][105692] Updated weights for policy 0, policy_version 205203 (0.0010) [2023-12-26 16:51:35,520][105620] Updated weights for policy 1, policy_version 205908 (0.0006) [2023-12-26 16:51:35,565][105620] Updated weights for policy 1, policy_version 205918 (0.0005) [2023-12-26 16:51:35,613][105620] Updated weights for policy 1, policy_version 205928 (0.0005) [2023-12-26 16:51:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 105267200. Throughput: 0: 9688.3, 1: 9799.5. Samples: 105258820. Policy #0 lag: (min: 6.0, avg: 6.1, max: 12.0) [2023-12-26 16:51:36,062][104569] Avg episode reward: [(0, '9258.256'), (1, '6323.344')] [2023-12-26 16:51:36,219][105620] Updated weights for policy 1, policy_version 205938 (0.0008) [2023-12-26 16:51:36,226][105692] Updated weights for policy 0, policy_version 205213 (0.0011) [2023-12-26 16:51:36,275][105620] Updated weights for policy 1, policy_version 205948 (0.0008) [2023-12-26 16:51:36,288][105692] Updated weights for policy 0, policy_version 205223 (0.0010) [2023-12-26 16:51:36,338][105620] Updated weights for policy 1, policy_version 205958 (0.0008) [2023-12-26 16:51:36,348][105692] Updated weights for policy 0, policy_version 205233 (0.0006) [2023-12-26 16:51:36,956][105692] Updated weights for policy 0, policy_version 205243 (0.0005) [2023-12-26 16:51:37,013][105692] Updated weights for policy 0, policy_version 205253 (0.0005) [2023-12-26 16:51:37,062][105692] Updated weights for policy 0, policy_version 205263 (0.0007) [2023-12-26 16:51:37,088][105620] Updated weights for policy 1, policy_version 205968 (0.0007) [2023-12-26 16:51:37,159][105620] Updated weights for policy 1, policy_version 205978 (0.0005) [2023-12-26 16:51:37,215][105620] Updated weights for policy 1, policy_version 205988 (0.0009) [2023-12-26 16:51:37,740][105692] Updated weights for policy 0, policy_version 205273 (0.0010) [2023-12-26 16:51:37,804][105692] Updated weights for policy 0, policy_version 205283 (0.0008) [2023-12-26 16:51:37,869][105692] Updated weights for policy 0, policy_version 205293 (0.0007) [2023-12-26 16:51:37,929][105620] Updated weights for policy 1, policy_version 205998 (0.0007) [2023-12-26 16:51:37,932][105692] Updated weights for policy 0, policy_version 205303 (0.0007) [2023-12-26 16:51:37,987][105620] Updated weights for policy 1, policy_version 206008 (0.0005) [2023-12-26 16:51:38,040][105620] Updated weights for policy 1, policy_version 206018 (0.0006) [2023-12-26 16:51:38,622][105692] Updated weights for policy 0, policy_version 205313 (0.0006) [2023-12-26 16:51:38,676][105620] Updated weights for policy 1, policy_version 206028 (0.0006) [2023-12-26 16:51:38,678][105692] Updated weights for policy 0, policy_version 205323 (0.0006) [2023-12-26 16:51:38,737][105692] Updated weights for policy 0, policy_version 205333 (0.0006) [2023-12-26 16:51:38,744][105620] Updated weights for policy 1, policy_version 206038 (0.0006) [2023-12-26 16:51:38,802][105620] Updated weights for policy 1, policy_version 206048 (0.0006) [2023-12-26 16:51:39,333][105692] Updated weights for policy 0, policy_version 205343 (0.0006) [2023-12-26 16:51:39,398][105692] Updated weights for policy 0, policy_version 205353 (0.0008) [2023-12-26 16:51:39,456][105620] Updated weights for policy 1, policy_version 206058 (0.0008) [2023-12-26 16:51:39,460][105692] Updated weights for policy 0, policy_version 205363 (0.0008) [2023-12-26 16:51:39,519][105620] Updated weights for policy 1, policy_version 206068 (0.0011) [2023-12-26 16:51:39,578][105620] Updated weights for policy 1, policy_version 206078 (0.0011) [2023-12-26 16:51:39,634][105620] Updated weights for policy 1, policy_version 206088 (0.0011) [2023-12-26 16:51:40,173][105692] Updated weights for policy 0, policy_version 205373 (0.0008) [2023-12-26 16:51:40,227][105692] Updated weights for policy 0, policy_version 205383 (0.0009) [2023-12-26 16:51:40,283][105692] Updated weights for policy 0, policy_version 205393 (0.0009) [2023-12-26 16:51:40,371][105620] Updated weights for policy 1, policy_version 206098 (0.0010) [2023-12-26 16:51:40,433][105620] Updated weights for policy 1, policy_version 206108 (0.0009) [2023-12-26 16:51:40,501][105620] Updated weights for policy 1, policy_version 206118 (0.0009) [2023-12-26 16:51:41,008][105692] Updated weights for policy 0, policy_version 205403 (0.0010) [2023-12-26 16:51:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 105365504. Throughput: 0: 9774.7, 1: 9831.7. Samples: 105379120. Policy #0 lag: (min: 6.0, avg: 6.1, max: 12.0) [2023-12-26 16:51:41,063][104569] Avg episode reward: [(0, '9259.723'), (1, '9355.766')] [2023-12-26 16:51:41,079][105692] Updated weights for policy 0, policy_version 205413 (0.0008) [2023-12-26 16:51:41,130][105692] Updated weights for policy 0, policy_version 205423 (0.0008) [2023-12-26 16:51:41,281][105620] Updated weights for policy 1, policy_version 206128 (0.0010) [2023-12-26 16:51:41,338][105620] Updated weights for policy 1, policy_version 206138 (0.0009) [2023-12-26 16:51:41,413][105620] Updated weights for policy 1, policy_version 206148 (0.0008) [2023-12-26 16:51:41,921][105692] Updated weights for policy 0, policy_version 205433 (0.0009) [2023-12-26 16:51:41,979][105692] Updated weights for policy 0, policy_version 205443 (0.0010) [2023-12-26 16:51:42,039][105692] Updated weights for policy 0, policy_version 205453 (0.0010) [2023-12-26 16:51:42,096][105692] Updated weights for policy 0, policy_version 205463 (0.0009) [2023-12-26 16:51:42,133][105620] Updated weights for policy 1, policy_version 206158 (0.0006) [2023-12-26 16:51:42,182][105620] Updated weights for policy 1, policy_version 206168 (0.0008) [2023-12-26 16:51:42,234][105620] Updated weights for policy 1, policy_version 206178 (0.0009) [2023-12-26 16:51:42,935][105692] Updated weights for policy 0, policy_version 205473 (0.0009) [2023-12-26 16:51:42,974][105620] Updated weights for policy 1, policy_version 206188 (0.0007) [2023-12-26 16:51:42,993][105692] Updated weights for policy 0, policy_version 205483 (0.0006) [2023-12-26 16:51:43,028][105620] Updated weights for policy 1, policy_version 206198 (0.0006) [2023-12-26 16:51:43,054][105692] Updated weights for policy 0, policy_version 205493 (0.0008) [2023-12-26 16:51:43,083][105620] Updated weights for policy 1, policy_version 206208 (0.0007) [2023-12-26 16:51:43,695][105620] Updated weights for policy 1, policy_version 206218 (0.0008) [2023-12-26 16:51:43,754][105620] Updated weights for policy 1, policy_version 206228 (0.0009) [2023-12-26 16:51:43,806][105620] Updated weights for policy 1, policy_version 206238 (0.0008) [2023-12-26 16:51:43,834][105692] Updated weights for policy 0, policy_version 205503 (0.0009) [2023-12-26 16:51:43,866][105620] Updated weights for policy 1, policy_version 206248 (0.0007) [2023-12-26 16:51:43,886][105692] Updated weights for policy 0, policy_version 205513 (0.0005) [2023-12-26 16:51:43,940][105692] Updated weights for policy 0, policy_version 205523 (0.0005) [2023-12-26 16:51:44,556][105620] Updated weights for policy 1, policy_version 206258 (0.0008) [2023-12-26 16:51:44,617][105620] Updated weights for policy 1, policy_version 206268 (0.0008) [2023-12-26 16:51:44,636][105692] Updated weights for policy 0, policy_version 205533 (0.0009) [2023-12-26 16:51:44,673][105620] Updated weights for policy 1, policy_version 206278 (0.0007) [2023-12-26 16:51:44,698][105692] Updated weights for policy 0, policy_version 205543 (0.0010) [2023-12-26 16:51:44,759][105692] Updated weights for policy 0, policy_version 205553 (0.0010) [2023-12-26 16:51:45,410][105692] Updated weights for policy 0, policy_version 205563 (0.0011) [2023-12-26 16:51:45,462][105692] Updated weights for policy 0, policy_version 205573 (0.0010) [2023-12-26 16:51:45,471][105620] Updated weights for policy 1, policy_version 206288 (0.0006) [2023-12-26 16:51:45,513][105692] Updated weights for policy 0, policy_version 205583 (0.0010) [2023-12-26 16:51:45,523][105620] Updated weights for policy 1, policy_version 206298 (0.0005) [2023-12-26 16:51:45,584][105620] Updated weights for policy 1, policy_version 206308 (0.0010) [2023-12-26 16:51:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 105463808. Throughput: 0: 9733.1, 1: 9874.5. Samples: 105435416. Policy #0 lag: (min: 6.0, avg: 6.1, max: 12.0) [2023-12-26 16:51:46,063][104569] Avg episode reward: [(0, '9261.153'), (1, '8841.963')] [2023-12-26 16:51:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000206312_52822016.pth... [2023-12-26 16:51:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000205592_52641792.pth... [2023-12-26 16:51:46,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000205160_52527104.pth [2023-12-26 16:51:46,080][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000204440_52346880.pth [2023-12-26 16:51:46,239][105692] Updated weights for policy 0, policy_version 205593 (0.0007) [2023-12-26 16:51:46,292][105692] Updated weights for policy 0, policy_version 205603 (0.0007) [2023-12-26 16:51:46,337][105692] Updated weights for policy 0, policy_version 205613 (0.0005) [2023-12-26 16:51:46,338][105620] Updated weights for policy 1, policy_version 206318 (0.0008) [2023-12-26 16:51:46,392][105692] Updated weights for policy 0, policy_version 205623 (0.0006) [2023-12-26 16:51:46,394][105620] Updated weights for policy 1, policy_version 206328 (0.0007) [2023-12-26 16:51:46,452][105620] Updated weights for policy 1, policy_version 206338 (0.0008) [2023-12-26 16:51:47,090][105692] Updated weights for policy 0, policy_version 205633 (0.0009) [2023-12-26 16:51:47,152][105692] Updated weights for policy 0, policy_version 205643 (0.0008) [2023-12-26 16:51:47,214][105692] Updated weights for policy 0, policy_version 205653 (0.0006) [2023-12-26 16:51:47,236][105620] Updated weights for policy 1, policy_version 206348 (0.0008) [2023-12-26 16:51:47,283][105620] Updated weights for policy 1, policy_version 206358 (0.0009) [2023-12-26 16:51:47,334][105620] Updated weights for policy 1, policy_version 206368 (0.0009) [2023-12-26 16:51:47,927][105692] Updated weights for policy 0, policy_version 205663 (0.0009) [2023-12-26 16:51:47,984][105692] Updated weights for policy 0, policy_version 205673 (0.0009) [2023-12-26 16:51:48,030][105692] Updated weights for policy 0, policy_version 205683 (0.0009) [2023-12-26 16:51:48,102][105620] Updated weights for policy 1, policy_version 206378 (0.0010) [2023-12-26 16:51:48,155][105620] Updated weights for policy 1, policy_version 206388 (0.0009) [2023-12-26 16:51:48,206][105620] Updated weights for policy 1, policy_version 206398 (0.0009) [2023-12-26 16:51:48,260][105620] Updated weights for policy 1, policy_version 206408 (0.0010) [2023-12-26 16:51:48,754][105692] Updated weights for policy 0, policy_version 205693 (0.0008) [2023-12-26 16:51:48,817][105692] Updated weights for policy 0, policy_version 205703 (0.0010) [2023-12-26 16:51:48,874][105692] Updated weights for policy 0, policy_version 205713 (0.0010) [2023-12-26 16:51:48,941][105620] Updated weights for policy 1, policy_version 206418 (0.0005) [2023-12-26 16:51:48,996][105620] Updated weights for policy 1, policy_version 206428 (0.0007) [2023-12-26 16:51:49,048][105620] Updated weights for policy 1, policy_version 206438 (0.0009) [2023-12-26 16:51:49,663][105692] Updated weights for policy 0, policy_version 205723 (0.0009) [2023-12-26 16:51:49,729][105692] Updated weights for policy 0, policy_version 205733 (0.0008) [2023-12-26 16:51:49,791][105692] Updated weights for policy 0, policy_version 205743 (0.0007) [2023-12-26 16:51:49,801][105620] Updated weights for policy 1, policy_version 206448 (0.0007) [2023-12-26 16:51:49,862][105620] Updated weights for policy 1, policy_version 206458 (0.0007) [2023-12-26 16:51:49,921][105620] Updated weights for policy 1, policy_version 206468 (0.0009) [2023-12-26 16:51:50,567][105692] Updated weights for policy 0, policy_version 205753 (0.0008) [2023-12-26 16:51:50,599][105620] Updated weights for policy 1, policy_version 206478 (0.0008) [2023-12-26 16:51:50,634][105692] Updated weights for policy 0, policy_version 205763 (0.0009) [2023-12-26 16:51:50,659][105620] Updated weights for policy 1, policy_version 206488 (0.0009) [2023-12-26 16:51:50,701][105692] Updated weights for policy 0, policy_version 205773 (0.0008) [2023-12-26 16:51:50,724][105620] Updated weights for policy 1, policy_version 206498 (0.0005) [2023-12-26 16:51:50,764][105692] Updated weights for policy 0, policy_version 205783 (0.0008) [2023-12-26 16:51:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 105562112. Throughput: 0: 9732.0, 1: 9806.1. Samples: 105550380. Policy #0 lag: (min: 6.0, avg: 6.1, max: 12.0) [2023-12-26 16:51:51,062][104569] Avg episode reward: [(0, '8920.907'), (1, '8842.473')] [2023-12-26 16:51:51,445][105620] Updated weights for policy 1, policy_version 206508 (0.0007) [2023-12-26 16:51:51,507][105620] Updated weights for policy 1, policy_version 206518 (0.0010) [2023-12-26 16:51:51,541][105692] Updated weights for policy 0, policy_version 205793 (0.0005) [2023-12-26 16:51:51,572][105620] Updated weights for policy 1, policy_version 206528 (0.0009) [2023-12-26 16:51:51,600][105692] Updated weights for policy 0, policy_version 205803 (0.0006) [2023-12-26 16:51:51,668][105692] Updated weights for policy 0, policy_version 205813 (0.0009) [2023-12-26 16:51:52,367][105620] Updated weights for policy 1, policy_version 206538 (0.0008) [2023-12-26 16:51:52,373][105692] Updated weights for policy 0, policy_version 205823 (0.0010) [2023-12-26 16:51:52,432][105620] Updated weights for policy 1, policy_version 206548 (0.0006) [2023-12-26 16:51:52,436][105692] Updated weights for policy 0, policy_version 205833 (0.0011) [2023-12-26 16:51:52,490][105620] Updated weights for policy 1, policy_version 206558 (0.0006) [2023-12-26 16:51:52,499][105692] Updated weights for policy 0, policy_version 205843 (0.0010) [2023-12-26 16:51:52,542][105620] Updated weights for policy 1, policy_version 206568 (0.0007) [2023-12-26 16:51:53,179][105692] Updated weights for policy 0, policy_version 205853 (0.0010) [2023-12-26 16:51:53,182][105620] Updated weights for policy 1, policy_version 206578 (0.0010) [2023-12-26 16:51:53,230][105692] Updated weights for policy 0, policy_version 205863 (0.0010) [2023-12-26 16:51:53,238][105620] Updated weights for policy 1, policy_version 206588 (0.0010) [2023-12-26 16:51:53,287][105692] Updated weights for policy 0, policy_version 205873 (0.0010) [2023-12-26 16:51:53,296][105620] Updated weights for policy 1, policy_version 206598 (0.0010) [2023-12-26 16:51:53,989][105692] Updated weights for policy 0, policy_version 205883 (0.0009) [2023-12-26 16:51:54,042][105692] Updated weights for policy 0, policy_version 205893 (0.0006) [2023-12-26 16:51:54,051][105620] Updated weights for policy 1, policy_version 206608 (0.0010) [2023-12-26 16:51:54,095][105692] Updated weights for policy 0, policy_version 205903 (0.0006) [2023-12-26 16:51:54,113][105620] Updated weights for policy 1, policy_version 206618 (0.0010) [2023-12-26 16:51:54,173][105620] Updated weights for policy 1, policy_version 206628 (0.0010) [2023-12-26 16:51:54,799][105692] Updated weights for policy 0, policy_version 205913 (0.0006) [2023-12-26 16:51:54,832][105620] Updated weights for policy 1, policy_version 206638 (0.0009) [2023-12-26 16:51:54,847][105692] Updated weights for policy 0, policy_version 205923 (0.0009) [2023-12-26 16:51:54,883][105620] Updated weights for policy 1, policy_version 206648 (0.0005) [2023-12-26 16:51:54,904][105692] Updated weights for policy 0, policy_version 205933 (0.0006) [2023-12-26 16:51:54,946][105620] Updated weights for policy 1, policy_version 206658 (0.0006) [2023-12-26 16:51:54,967][105692] Updated weights for policy 0, policy_version 205943 (0.0006) [2023-12-26 16:51:55,567][105620] Updated weights for policy 1, policy_version 206668 (0.0006) [2023-12-26 16:51:55,626][105620] Updated weights for policy 1, policy_version 206678 (0.0005) [2023-12-26 16:51:55,627][105692] Updated weights for policy 0, policy_version 205953 (0.0005) [2023-12-26 16:51:55,691][105620] Updated weights for policy 1, policy_version 206688 (0.0006) [2023-12-26 16:51:55,693][105692] Updated weights for policy 0, policy_version 205963 (0.0005) [2023-12-26 16:51:55,744][105692] Updated weights for policy 0, policy_version 205973 (0.0005) [2023-12-26 16:51:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 105660416. Throughput: 0: 9735.5, 1: 9872.2. Samples: 105668424. Policy #0 lag: (min: 6.0, avg: 6.1, max: 12.0) [2023-12-26 16:51:56,063][104569] Avg episode reward: [(0, '8923.574'), (1, '8843.155')] [2023-12-26 16:51:56,310][105692] Updated weights for policy 0, policy_version 205983 (0.0005) [2023-12-26 16:51:56,363][105692] Updated weights for policy 0, policy_version 205993 (0.0006) [2023-12-26 16:51:56,364][105620] Updated weights for policy 1, policy_version 206698 (0.0007) [2023-12-26 16:51:56,422][105692] Updated weights for policy 0, policy_version 206003 (0.0008) [2023-12-26 16:51:56,433][105620] Updated weights for policy 1, policy_version 206708 (0.0006) [2023-12-26 16:51:56,495][105620] Updated weights for policy 1, policy_version 206718 (0.0007) [2023-12-26 16:51:56,551][105620] Updated weights for policy 1, policy_version 206728 (0.0008) [2023-12-26 16:51:57,121][105692] Updated weights for policy 0, policy_version 206013 (0.0010) [2023-12-26 16:51:57,167][105692] Updated weights for policy 0, policy_version 206023 (0.0010) [2023-12-26 16:51:57,215][105692] Updated weights for policy 0, policy_version 206033 (0.0010) [2023-12-26 16:51:57,252][105620] Updated weights for policy 1, policy_version 206738 (0.0006) [2023-12-26 16:51:57,311][105620] Updated weights for policy 1, policy_version 206748 (0.0007) [2023-12-26 16:51:57,371][105620] Updated weights for policy 1, policy_version 206758 (0.0008) [2023-12-26 16:51:57,970][105692] Updated weights for policy 0, policy_version 206043 (0.0010) [2023-12-26 16:51:58,024][105692] Updated weights for policy 0, policy_version 206053 (0.0010) [2023-12-26 16:51:58,046][105620] Updated weights for policy 1, policy_version 206768 (0.0006) [2023-12-26 16:51:58,082][105692] Updated weights for policy 0, policy_version 206063 (0.0010) [2023-12-26 16:51:58,110][105620] Updated weights for policy 1, policy_version 206778 (0.0007) [2023-12-26 16:51:58,169][105620] Updated weights for policy 1, policy_version 206788 (0.0007) [2023-12-26 16:51:58,831][105692] Updated weights for policy 0, policy_version 206073 (0.0008) [2023-12-26 16:51:58,895][105692] Updated weights for policy 0, policy_version 206083 (0.0009) [2023-12-26 16:51:58,947][105692] Updated weights for policy 0, policy_version 206093 (0.0005) [2023-12-26 16:51:58,957][105620] Updated weights for policy 1, policy_version 206798 (0.0008) [2023-12-26 16:51:59,003][105692] Updated weights for policy 0, policy_version 206103 (0.0008) [2023-12-26 16:51:59,009][105620] Updated weights for policy 1, policy_version 206808 (0.0006) [2023-12-26 16:51:59,063][105620] Updated weights for policy 1, policy_version 206818 (0.0009) [2023-12-26 16:51:59,731][105692] Updated weights for policy 0, policy_version 206113 (0.0009) [2023-12-26 16:51:59,772][105620] Updated weights for policy 1, policy_version 206828 (0.0008) [2023-12-26 16:51:59,779][105692] Updated weights for policy 0, policy_version 206123 (0.0007) [2023-12-26 16:51:59,831][105692] Updated weights for policy 0, policy_version 206133 (0.0009) [2023-12-26 16:51:59,836][105620] Updated weights for policy 1, policy_version 206838 (0.0007) [2023-12-26 16:51:59,895][105620] Updated weights for policy 1, policy_version 206848 (0.0008) [2023-12-26 16:52:00,521][105620] Updated weights for policy 1, policy_version 206858 (0.0007) [2023-12-26 16:52:00,546][105692] Updated weights for policy 0, policy_version 206143 (0.0006) [2023-12-26 16:52:00,570][105620] Updated weights for policy 1, policy_version 206868 (0.0009) [2023-12-26 16:52:00,609][105692] Updated weights for policy 0, policy_version 206153 (0.0009) [2023-12-26 16:52:00,627][105620] Updated weights for policy 1, policy_version 206878 (0.0006) [2023-12-26 16:52:00,658][105692] Updated weights for policy 0, policy_version 206163 (0.0010) [2023-12-26 16:52:00,690][105620] Updated weights for policy 1, policy_version 206888 (0.0006) [2023-12-26 16:52:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 105758720. Throughput: 0: 9770.1, 1: 9827.2. Samples: 105727560. Policy #0 lag: (min: 6.0, avg: 6.1, max: 12.0) [2023-12-26 16:52:01,062][104569] Avg episode reward: [(0, '9265.629'), (1, '9358.150')] [2023-12-26 16:52:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000206168_52789248.pth... [2023-12-26 16:52:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000206888_52969472.pth... [2023-12-26 16:52:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000205736_52674560.pth [2023-12-26 16:52:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000205016_52494336.pth [2023-12-26 16:52:01,307][105620] Updated weights for policy 1, policy_version 206898 (0.0008) [2023-12-26 16:52:01,314][105692] Updated weights for policy 0, policy_version 206173 (0.0010) [2023-12-26 16:52:01,365][105620] Updated weights for policy 1, policy_version 206908 (0.0006) [2023-12-26 16:52:01,374][105692] Updated weights for policy 0, policy_version 206183 (0.0011) [2023-12-26 16:52:01,426][105620] Updated weights for policy 1, policy_version 206918 (0.0007) [2023-12-26 16:52:01,436][105692] Updated weights for policy 0, policy_version 206193 (0.0010) [2023-12-26 16:52:02,106][105620] Updated weights for policy 1, policy_version 206928 (0.0007) [2023-12-26 16:52:02,165][105620] Updated weights for policy 1, policy_version 206938 (0.0008) [2023-12-26 16:52:02,191][105692] Updated weights for policy 0, policy_version 206203 (0.0010) [2023-12-26 16:52:02,221][105620] Updated weights for policy 1, policy_version 206948 (0.0006) [2023-12-26 16:52:02,249][105692] Updated weights for policy 0, policy_version 206213 (0.0010) [2023-12-26 16:52:02,312][105692] Updated weights for policy 0, policy_version 206223 (0.0010) [2023-12-26 16:52:02,932][105692] Updated weights for policy 0, policy_version 206233 (0.0010) [2023-12-26 16:52:03,002][105692] Updated weights for policy 0, policy_version 206243 (0.0010) [2023-12-26 16:52:03,053][105620] Updated weights for policy 1, policy_version 206958 (0.0006) [2023-12-26 16:52:03,059][105692] Updated weights for policy 0, policy_version 206253 (0.0010) [2023-12-26 16:52:03,109][105620] Updated weights for policy 1, policy_version 206968 (0.0006) [2023-12-26 16:52:03,119][105692] Updated weights for policy 0, policy_version 206263 (0.0011) [2023-12-26 16:52:03,174][105620] Updated weights for policy 1, policy_version 206978 (0.0007) [2023-12-26 16:52:03,782][105620] Updated weights for policy 1, policy_version 206988 (0.0007) [2023-12-26 16:52:03,845][105620] Updated weights for policy 1, policy_version 206998 (0.0006) [2023-12-26 16:52:03,854][105692] Updated weights for policy 0, policy_version 206273 (0.0007) [2023-12-26 16:52:03,913][105620] Updated weights for policy 1, policy_version 207008 (0.0006) [2023-12-26 16:52:03,914][105692] Updated weights for policy 0, policy_version 206283 (0.0009) [2023-12-26 16:52:03,981][105692] Updated weights for policy 0, policy_version 206293 (0.0010) [2023-12-26 16:52:04,561][105620] Updated weights for policy 1, policy_version 207018 (0.0008) [2023-12-26 16:52:04,606][105620] Updated weights for policy 1, policy_version 207028 (0.0010) [2023-12-26 16:52:04,642][105692] Updated weights for policy 0, policy_version 206303 (0.0007) [2023-12-26 16:52:04,658][105620] Updated weights for policy 1, policy_version 207038 (0.0010) [2023-12-26 16:52:04,705][105692] Updated weights for policy 0, policy_version 206313 (0.0005) [2023-12-26 16:52:04,707][105620] Updated weights for policy 1, policy_version 207048 (0.0010) [2023-12-26 16:52:04,763][105692] Updated weights for policy 0, policy_version 206323 (0.0007) [2023-12-26 16:52:05,429][105692] Updated weights for policy 0, policy_version 206333 (0.0010) [2023-12-26 16:52:05,440][105620] Updated weights for policy 1, policy_version 207058 (0.0010) [2023-12-26 16:52:05,477][105692] Updated weights for policy 0, policy_version 206343 (0.0010) [2023-12-26 16:52:05,495][105620] Updated weights for policy 1, policy_version 207068 (0.0010) [2023-12-26 16:52:05,528][105692] Updated weights for policy 0, policy_version 206353 (0.0010) [2023-12-26 16:52:05,543][105620] Updated weights for policy 1, policy_version 207078 (0.0010) [2023-12-26 16:52:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 105857024. Throughput: 0: 9791.1, 1: 9881.3. Samples: 105847396. Policy #0 lag: (min: 6.0, avg: 6.1, max: 12.0) [2023-12-26 16:52:06,063][104569] Avg episode reward: [(0, '9174.164'), (1, '9268.434')] [2023-12-26 16:52:06,235][105692] Updated weights for policy 0, policy_version 206363 (0.0010) [2023-12-26 16:52:06,297][105692] Updated weights for policy 0, policy_version 206373 (0.0008) [2023-12-26 16:52:06,312][105620] Updated weights for policy 1, policy_version 207088 (0.0008) [2023-12-26 16:52:06,352][105692] Updated weights for policy 0, policy_version 206383 (0.0009) [2023-12-26 16:52:06,377][105620] Updated weights for policy 1, policy_version 207098 (0.0009) [2023-12-26 16:52:06,440][105620] Updated weights for policy 1, policy_version 207108 (0.0011) [2023-12-26 16:52:06,993][105620] Updated weights for policy 1, policy_version 207118 (0.0008) [2023-12-26 16:52:07,054][105620] Updated weights for policy 1, policy_version 207128 (0.0006) [2023-12-26 16:52:07,110][105692] Updated weights for policy 0, policy_version 206393 (0.0008) [2023-12-26 16:52:07,125][105620] Updated weights for policy 1, policy_version 207138 (0.0008) [2023-12-26 16:52:07,170][105692] Updated weights for policy 0, policy_version 206403 (0.0010) [2023-12-26 16:52:07,225][105692] Updated weights for policy 0, policy_version 206413 (0.0010) [2023-12-26 16:52:07,288][105692] Updated weights for policy 0, policy_version 206423 (0.0010) [2023-12-26 16:52:07,840][105620] Updated weights for policy 1, policy_version 207148 (0.0010) [2023-12-26 16:52:07,904][105620] Updated weights for policy 1, policy_version 207158 (0.0009) [2023-12-26 16:52:07,960][105620] Updated weights for policy 1, policy_version 207168 (0.0006) [2023-12-26 16:52:07,982][105692] Updated weights for policy 0, policy_version 206433 (0.0008) [2023-12-26 16:52:08,033][105692] Updated weights for policy 0, policy_version 206443 (0.0009) [2023-12-26 16:52:08,084][105692] Updated weights for policy 0, policy_version 206453 (0.0010) [2023-12-26 16:52:08,674][105620] Updated weights for policy 1, policy_version 207178 (0.0007) [2023-12-26 16:52:08,740][105620] Updated weights for policy 1, policy_version 207188 (0.0009) [2023-12-26 16:52:08,789][105692] Updated weights for policy 0, policy_version 206463 (0.0010) [2023-12-26 16:52:08,800][105620] Updated weights for policy 1, policy_version 207198 (0.0007) [2023-12-26 16:52:08,854][105692] Updated weights for policy 0, policy_version 206473 (0.0008) [2023-12-26 16:52:08,867][105620] Updated weights for policy 1, policy_version 207208 (0.0006) [2023-12-26 16:52:08,916][105692] Updated weights for policy 0, policy_version 206483 (0.0009) [2023-12-26 16:52:09,621][105620] Updated weights for policy 1, policy_version 207218 (0.0010) [2023-12-26 16:52:09,635][105692] Updated weights for policy 0, policy_version 206493 (0.0008) [2023-12-26 16:52:09,687][105620] Updated weights for policy 1, policy_version 207228 (0.0009) [2023-12-26 16:52:09,700][105692] Updated weights for policy 0, policy_version 206503 (0.0006) [2023-12-26 16:52:09,755][105620] Updated weights for policy 1, policy_version 207238 (0.0009) [2023-12-26 16:52:09,764][105692] Updated weights for policy 0, policy_version 206513 (0.0006) [2023-12-26 16:52:10,429][105620] Updated weights for policy 1, policy_version 207248 (0.0011) [2023-12-26 16:52:10,471][105692] Updated weights for policy 0, policy_version 206523 (0.0007) [2023-12-26 16:52:10,490][105620] Updated weights for policy 1, policy_version 207258 (0.0011) [2023-12-26 16:52:10,528][105692] Updated weights for policy 0, policy_version 206533 (0.0006) [2023-12-26 16:52:10,545][105620] Updated weights for policy 1, policy_version 207268 (0.0011) [2023-12-26 16:52:10,589][105692] Updated weights for policy 0, policy_version 206543 (0.0007) [2023-12-26 16:52:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 105955328. Throughput: 0: 9805.8, 1: 9924.1. Samples: 105964772. Policy #0 lag: (min: 6.0, avg: 6.1, max: 12.0) [2023-12-26 16:52:11,063][104569] Avg episode reward: [(0, '9176.162'), (1, '8744.432')] [2023-12-26 16:52:11,312][105620] Updated weights for policy 1, policy_version 207278 (0.0011) [2023-12-26 16:52:11,331][105692] Updated weights for policy 0, policy_version 206553 (0.0009) [2023-12-26 16:52:11,386][105620] Updated weights for policy 1, policy_version 207288 (0.0007) [2023-12-26 16:52:11,391][105692] Updated weights for policy 0, policy_version 206563 (0.0011) [2023-12-26 16:52:11,445][105620] Updated weights for policy 1, policy_version 207298 (0.0009) [2023-12-26 16:52:11,451][105692] Updated weights for policy 0, policy_version 206573 (0.0011) [2023-12-26 16:52:11,510][105692] Updated weights for policy 0, policy_version 206583 (0.0008) [2023-12-26 16:52:12,184][105620] Updated weights for policy 1, policy_version 207308 (0.0010) [2023-12-26 16:52:12,246][105620] Updated weights for policy 1, policy_version 207318 (0.0010) [2023-12-26 16:52:12,280][105692] Updated weights for policy 0, policy_version 206593 (0.0011) [2023-12-26 16:52:12,310][105620] Updated weights for policy 1, policy_version 207328 (0.0010) [2023-12-26 16:52:12,341][105692] Updated weights for policy 0, policy_version 206603 (0.0011) [2023-12-26 16:52:12,409][105692] Updated weights for policy 0, policy_version 206613 (0.0011) [2023-12-26 16:52:12,995][105620] Updated weights for policy 1, policy_version 207338 (0.0012) [2023-12-26 16:52:13,053][105620] Updated weights for policy 1, policy_version 207348 (0.0007) [2023-12-26 16:52:13,054][105692] Updated weights for policy 0, policy_version 206623 (0.0011) [2023-12-26 16:52:13,111][105692] Updated weights for policy 0, policy_version 206633 (0.0011) [2023-12-26 16:52:13,119][105620] Updated weights for policy 1, policy_version 207358 (0.0007) [2023-12-26 16:52:13,175][105692] Updated weights for policy 0, policy_version 206643 (0.0011) [2023-12-26 16:52:13,179][105620] Updated weights for policy 1, policy_version 207368 (0.0010) [2023-12-26 16:52:13,841][105692] Updated weights for policy 0, policy_version 206653 (0.0008) [2023-12-26 16:52:13,899][105692] Updated weights for policy 0, policy_version 206663 (0.0006) [2023-12-26 16:52:13,909][105620] Updated weights for policy 1, policy_version 207378 (0.0010) [2023-12-26 16:52:13,965][105620] Updated weights for policy 1, policy_version 207388 (0.0007) [2023-12-26 16:52:13,969][105692] Updated weights for policy 0, policy_version 206673 (0.0006) [2023-12-26 16:52:14,014][105620] Updated weights for policy 1, policy_version 207398 (0.0005) [2023-12-26 16:52:14,525][105692] Updated weights for policy 0, policy_version 206683 (0.0006) [2023-12-26 16:52:14,581][105692] Updated weights for policy 0, policy_version 206693 (0.0005) [2023-12-26 16:52:14,642][105692] Updated weights for policy 0, policy_version 206703 (0.0005) [2023-12-26 16:52:14,655][105620] Updated weights for policy 1, policy_version 207408 (0.0009) [2023-12-26 16:52:14,706][105620] Updated weights for policy 1, policy_version 207418 (0.0010) [2023-12-26 16:52:14,765][105620] Updated weights for policy 1, policy_version 207428 (0.0010) [2023-12-26 16:52:15,258][105692] Updated weights for policy 0, policy_version 206713 (0.0005) [2023-12-26 16:52:15,312][105692] Updated weights for policy 0, policy_version 206723 (0.0009) [2023-12-26 16:52:15,364][105692] Updated weights for policy 0, policy_version 206733 (0.0008) [2023-12-26 16:52:15,413][105692] Updated weights for policy 0, policy_version 206743 (0.0007) [2023-12-26 16:52:15,528][105620] Updated weights for policy 1, policy_version 207438 (0.0005) [2023-12-26 16:52:15,583][105620] Updated weights for policy 1, policy_version 207448 (0.0005) [2023-12-26 16:52:15,629][105620] Updated weights for policy 1, policy_version 207458 (0.0005) [2023-12-26 16:52:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 106053632. Throughput: 0: 9779.2, 1: 9839.7. Samples: 106023028. Policy #0 lag: (min: 6.0, avg: 6.1, max: 12.0) [2023-12-26 16:52:16,062][104569] Avg episode reward: [(0, '9265.300'), (1, '6843.681')] [2023-12-26 16:52:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000207464_53116928.pth... [2023-12-26 16:52:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000206744_52936704.pth... [2023-12-26 16:52:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000206312_52822016.pth [2023-12-26 16:52:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000205592_52641792.pth [2023-12-26 16:52:16,144][105620] Updated weights for policy 1, policy_version 207468 (0.0005) [2023-12-26 16:52:16,197][105620] Updated weights for policy 1, policy_version 207478 (0.0005) [2023-12-26 16:52:16,261][105620] Updated weights for policy 1, policy_version 207488 (0.0005) [2023-12-26 16:52:16,326][105692] Updated weights for policy 0, policy_version 206753 (0.0010) [2023-12-26 16:52:16,378][105692] Updated weights for policy 0, policy_version 206763 (0.0011) [2023-12-26 16:52:16,426][105692] Updated weights for policy 0, policy_version 206773 (0.0010) [2023-12-26 16:52:16,780][105620] Updated weights for policy 1, policy_version 207498 (0.0006) [2023-12-26 16:52:16,844][105620] Updated weights for policy 1, policy_version 207508 (0.0008) [2023-12-26 16:52:16,895][105620] Updated weights for policy 1, policy_version 207518 (0.0010) [2023-12-26 16:52:16,951][105620] Updated weights for policy 1, policy_version 207528 (0.0010) [2023-12-26 16:52:17,002][105692] Updated weights for policy 0, policy_version 206783 (0.0006) [2023-12-26 16:52:17,059][105692] Updated weights for policy 0, policy_version 206793 (0.0005) [2023-12-26 16:52:17,128][105692] Updated weights for policy 0, policy_version 206803 (0.0011) [2023-12-26 16:52:17,666][105692] Updated weights for policy 0, policy_version 206813 (0.0010) [2023-12-26 16:52:17,695][105620] Updated weights for policy 1, policy_version 207538 (0.0010) [2023-12-26 16:52:17,725][105692] Updated weights for policy 0, policy_version 206823 (0.0010) [2023-12-26 16:52:17,748][105620] Updated weights for policy 1, policy_version 207548 (0.0008) [2023-12-26 16:52:17,788][105692] Updated weights for policy 0, policy_version 206833 (0.0006) [2023-12-26 16:52:17,799][105620] Updated weights for policy 1, policy_version 207558 (0.0005) [2023-12-26 16:52:18,336][105620] Updated weights for policy 1, policy_version 207568 (0.0007) [2023-12-26 16:52:18,399][105620] Updated weights for policy 1, policy_version 207578 (0.0010) [2023-12-26 16:52:18,459][105620] Updated weights for policy 1, policy_version 207588 (0.0011) [2023-12-26 16:52:18,491][105692] Updated weights for policy 0, policy_version 206843 (0.0007) [2023-12-26 16:52:18,555][105692] Updated weights for policy 0, policy_version 206853 (0.0011) [2023-12-26 16:52:18,614][105692] Updated weights for policy 0, policy_version 206863 (0.0007) [2023-12-26 16:52:19,246][105620] Updated weights for policy 1, policy_version 207598 (0.0010) [2023-12-26 16:52:19,256][105692] Updated weights for policy 0, policy_version 206873 (0.0006) [2023-12-26 16:52:19,302][105620] Updated weights for policy 1, policy_version 207608 (0.0011) [2023-12-26 16:52:19,309][105692] Updated weights for policy 0, policy_version 206883 (0.0010) [2023-12-26 16:52:19,367][105620] Updated weights for policy 1, policy_version 207618 (0.0010) [2023-12-26 16:52:19,373][105692] Updated weights for policy 0, policy_version 206893 (0.0011) [2023-12-26 16:52:19,445][105692] Updated weights for policy 0, policy_version 206903 (0.0011) [2023-12-26 16:52:20,125][105620] Updated weights for policy 1, policy_version 207628 (0.0009) [2023-12-26 16:52:20,178][105620] Updated weights for policy 1, policy_version 207638 (0.0011) [2023-12-26 16:52:20,206][105692] Updated weights for policy 0, policy_version 206913 (0.0011) [2023-12-26 16:52:20,228][105620] Updated weights for policy 1, policy_version 207648 (0.0011) [2023-12-26 16:52:20,260][105692] Updated weights for policy 0, policy_version 206923 (0.0011) [2023-12-26 16:52:20,306][105692] Updated weights for policy 0, policy_version 206933 (0.0010) [2023-12-26 16:52:21,005][105692] Updated weights for policy 0, policy_version 206943 (0.0009) [2023-12-26 16:52:21,025][105620] Updated weights for policy 1, policy_version 207658 (0.0010) [2023-12-26 16:52:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 106151936. Throughput: 0: 9850.9, 1: 9920.2. Samples: 106148524. Policy #0 lag: (min: 6.0, avg: 6.1, max: 12.0) [2023-12-26 16:52:21,063][104569] Avg episode reward: [(0, '9348.488'), (1, '7453.801')] [2023-12-26 16:52:21,073][105692] Updated weights for policy 0, policy_version 206953 (0.0009) [2023-12-26 16:52:21,096][105620] Updated weights for policy 1, policy_version 207668 (0.0008) [2023-12-26 16:52:21,138][105692] Updated weights for policy 0, policy_version 206963 (0.0007) [2023-12-26 16:52:21,160][105620] Updated weights for policy 1, policy_version 207678 (0.0008) [2023-12-26 16:52:21,228][105620] Updated weights for policy 1, policy_version 207688 (0.0006) [2023-12-26 16:52:21,891][105620] Updated weights for policy 1, policy_version 207698 (0.0005) [2023-12-26 16:52:21,959][105692] Updated weights for policy 0, policy_version 206973 (0.0009) [2023-12-26 16:52:21,960][105620] Updated weights for policy 1, policy_version 207708 (0.0010) [2023-12-26 16:52:22,016][105692] Updated weights for policy 0, policy_version 206983 (0.0009) [2023-12-26 16:52:22,021][105620] Updated weights for policy 1, policy_version 207718 (0.0007) [2023-12-26 16:52:22,066][105692] Updated weights for policy 0, policy_version 206993 (0.0010) [2023-12-26 16:52:22,721][105620] Updated weights for policy 1, policy_version 207728 (0.0010) [2023-12-26 16:52:22,781][105620] Updated weights for policy 1, policy_version 207738 (0.0011) [2023-12-26 16:52:22,831][105620] Updated weights for policy 1, policy_version 207748 (0.0011) [2023-12-26 16:52:22,868][105692] Updated weights for policy 0, policy_version 207003 (0.0009) [2023-12-26 16:52:22,923][105692] Updated weights for policy 0, policy_version 207013 (0.0008) [2023-12-26 16:52:22,987][105692] Updated weights for policy 0, policy_version 207023 (0.0007) [2023-12-26 16:52:23,614][105620] Updated weights for policy 1, policy_version 207758 (0.0010) [2023-12-26 16:52:23,656][105692] Updated weights for policy 0, policy_version 207033 (0.0007) [2023-12-26 16:52:23,673][105620] Updated weights for policy 1, policy_version 207768 (0.0010) [2023-12-26 16:52:23,716][105692] Updated weights for policy 0, policy_version 207043 (0.0007) [2023-12-26 16:52:23,726][105620] Updated weights for policy 1, policy_version 207778 (0.0010) [2023-12-26 16:52:23,778][105692] Updated weights for policy 0, policy_version 207053 (0.0006) [2023-12-26 16:52:23,836][105692] Updated weights for policy 0, policy_version 207063 (0.0008) [2023-12-26 16:52:24,446][105692] Updated weights for policy 0, policy_version 207073 (0.0007) [2023-12-26 16:52:24,460][105620] Updated weights for policy 1, policy_version 207788 (0.0010) [2023-12-26 16:52:24,503][105692] Updated weights for policy 0, policy_version 207083 (0.0007) [2023-12-26 16:52:24,509][105620] Updated weights for policy 1, policy_version 207798 (0.0006) [2023-12-26 16:52:24,564][105692] Updated weights for policy 0, policy_version 207093 (0.0007) [2023-12-26 16:52:24,564][105620] Updated weights for policy 1, policy_version 207808 (0.0008) [2023-12-26 16:52:25,224][105692] Updated weights for policy 0, policy_version 207103 (0.0007) [2023-12-26 16:52:25,225][105620] Updated weights for policy 1, policy_version 207818 (0.0006) [2023-12-26 16:52:25,274][105620] Updated weights for policy 1, policy_version 207828 (0.0010) [2023-12-26 16:52:25,285][105692] Updated weights for policy 0, policy_version 207113 (0.0005) [2023-12-26 16:52:25,330][105620] Updated weights for policy 1, policy_version 207838 (0.0011) [2023-12-26 16:52:25,338][105692] Updated weights for policy 0, policy_version 207123 (0.0005) [2023-12-26 16:52:25,376][105620] Updated weights for policy 1, policy_version 207848 (0.0010) [2023-12-26 16:52:25,932][105692] Updated weights for policy 0, policy_version 207133 (0.0008) [2023-12-26 16:52:25,979][105692] Updated weights for policy 0, policy_version 207143 (0.0010) [2023-12-26 16:52:26,034][105692] Updated weights for policy 0, policy_version 207153 (0.0010) [2023-12-26 16:52:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 106250240. Throughput: 0: 9838.9, 1: 9857.9. Samples: 106265472. Policy #0 lag: (min: 6.0, avg: 6.1, max: 12.0) [2023-12-26 16:52:26,062][104569] Avg episode reward: [(0, '9349.684'), (1, '9358.910')] [2023-12-26 16:52:26,100][105620] Updated weights for policy 1, policy_version 207858 (0.0007) [2023-12-26 16:52:26,146][105620] Updated weights for policy 1, policy_version 207868 (0.0005) [2023-12-26 16:52:26,203][105620] Updated weights for policy 1, policy_version 207878 (0.0005) [2023-12-26 16:52:26,726][105692] Updated weights for policy 0, policy_version 207163 (0.0010) [2023-12-26 16:52:26,783][105692] Updated weights for policy 0, policy_version 207173 (0.0010) [2023-12-26 16:52:26,831][105620] Updated weights for policy 1, policy_version 207888 (0.0006) [2023-12-26 16:52:26,843][105692] Updated weights for policy 0, policy_version 207183 (0.0011) [2023-12-26 16:52:26,885][105620] Updated weights for policy 1, policy_version 207898 (0.0007) [2023-12-26 16:52:26,946][105620] Updated weights for policy 1, policy_version 207908 (0.0009) [2023-12-26 16:52:27,495][105692] Updated weights for policy 0, policy_version 207193 (0.0010) [2023-12-26 16:52:27,559][105692] Updated weights for policy 0, policy_version 207203 (0.0011) [2023-12-26 16:52:27,591][105620] Updated weights for policy 1, policy_version 207918 (0.0007) [2023-12-26 16:52:27,621][105692] Updated weights for policy 0, policy_version 207213 (0.0011) [2023-12-26 16:52:27,649][105620] Updated weights for policy 1, policy_version 207928 (0.0005) [2023-12-26 16:52:27,676][105692] Updated weights for policy 0, policy_version 207223 (0.0009) [2023-12-26 16:52:27,712][105620] Updated weights for policy 1, policy_version 207938 (0.0005) [2023-12-26 16:52:28,278][105620] Updated weights for policy 1, policy_version 207948 (0.0005) [2023-12-26 16:52:28,320][105692] Updated weights for policy 0, policy_version 207233 (0.0008) [2023-12-26 16:52:28,335][105620] Updated weights for policy 1, policy_version 207958 (0.0008) [2023-12-26 16:52:28,380][105692] Updated weights for policy 0, policy_version 207243 (0.0007) [2023-12-26 16:52:28,394][105620] Updated weights for policy 1, policy_version 207968 (0.0008) [2023-12-26 16:52:28,443][105692] Updated weights for policy 0, policy_version 207253 (0.0009) [2023-12-26 16:52:29,075][105620] Updated weights for policy 1, policy_version 207978 (0.0008) [2023-12-26 16:52:29,133][105620] Updated weights for policy 1, policy_version 207988 (0.0009) [2023-12-26 16:52:29,144][105692] Updated weights for policy 0, policy_version 207263 (0.0007) [2023-12-26 16:52:29,187][105620] Updated weights for policy 1, policy_version 207998 (0.0008) [2023-12-26 16:52:29,190][105692] Updated weights for policy 0, policy_version 207273 (0.0007) [2023-12-26 16:52:29,248][105692] Updated weights for policy 0, policy_version 207283 (0.0007) [2023-12-26 16:52:29,252][105620] Updated weights for policy 1, policy_version 208008 (0.0008) [2023-12-26 16:52:29,890][105692] Updated weights for policy 0, policy_version 207293 (0.0006) [2023-12-26 16:52:29,958][105692] Updated weights for policy 0, policy_version 207303 (0.0007) [2023-12-26 16:52:30,016][105692] Updated weights for policy 0, policy_version 207313 (0.0009) [2023-12-26 16:52:30,065][105620] Updated weights for policy 1, policy_version 208018 (0.0008) [2023-12-26 16:52:30,129][105620] Updated weights for policy 1, policy_version 208028 (0.0008) [2023-12-26 16:52:30,195][105620] Updated weights for policy 1, policy_version 208038 (0.0009) [2023-12-26 16:52:30,724][105692] Updated weights for policy 0, policy_version 207323 (0.0006) [2023-12-26 16:52:30,775][105692] Updated weights for policy 0, policy_version 207333 (0.0008) [2023-12-26 16:52:30,823][105692] Updated weights for policy 0, policy_version 207343 (0.0008) [2023-12-26 16:52:30,949][105620] Updated weights for policy 1, policy_version 208048 (0.0007) [2023-12-26 16:52:31,007][105620] Updated weights for policy 1, policy_version 208058 (0.0006) [2023-12-26 16:52:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 106356736. Throughput: 0: 9928.6, 1: 9937.3. Samples: 106329376. Policy #0 lag: (min: 27.0, avg: 32.9, max: 59.0) [2023-12-26 16:52:31,063][104569] Avg episode reward: [(0, '9328.246'), (1, '9268.722')] [2023-12-26 16:52:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000207352_53092352.pth... [2023-12-26 16:52:31,070][105620] Updated weights for policy 1, policy_version 208068 (0.0010) [2023-12-26 16:52:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000206168_52789248.pth [2023-12-26 16:52:31,090][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000208072_53272576.pth... [2023-12-26 16:52:31,093][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000206888_52969472.pth [2023-12-26 16:52:31,565][105692] Updated weights for policy 0, policy_version 207353 (0.0008) [2023-12-26 16:52:31,626][105692] Updated weights for policy 0, policy_version 207363 (0.0007) [2023-12-26 16:52:31,679][105692] Updated weights for policy 0, policy_version 207373 (0.0007) [2023-12-26 16:52:31,754][105692] Updated weights for policy 0, policy_version 207383 (0.0008) [2023-12-26 16:52:31,790][105620] Updated weights for policy 1, policy_version 208078 (0.0008) [2023-12-26 16:52:31,857][105620] Updated weights for policy 1, policy_version 208088 (0.0005) [2023-12-26 16:52:31,923][105620] Updated weights for policy 1, policy_version 208098 (0.0007) [2023-12-26 16:52:32,403][105692] Updated weights for policy 0, policy_version 207393 (0.0009) [2023-12-26 16:52:32,469][105692] Updated weights for policy 0, policy_version 207403 (0.0009) [2023-12-26 16:52:32,504][105620] Updated weights for policy 1, policy_version 208108 (0.0008) [2023-12-26 16:52:32,529][105692] Updated weights for policy 0, policy_version 207413 (0.0009) [2023-12-26 16:52:32,557][105620] Updated weights for policy 1, policy_version 208118 (0.0005) [2023-12-26 16:52:32,609][105620] Updated weights for policy 1, policy_version 208128 (0.0005) [2023-12-26 16:52:33,154][105620] Updated weights for policy 1, policy_version 208138 (0.0005) [2023-12-26 16:52:33,210][105620] Updated weights for policy 1, policy_version 208148 (0.0005) [2023-12-26 16:52:33,261][105620] Updated weights for policy 1, policy_version 208158 (0.0005) [2023-12-26 16:52:33,311][105692] Updated weights for policy 0, policy_version 207423 (0.0006) [2023-12-26 16:52:33,315][105620] Updated weights for policy 1, policy_version 208168 (0.0005) [2023-12-26 16:52:33,359][105692] Updated weights for policy 0, policy_version 207433 (0.0005) [2023-12-26 16:52:33,416][105692] Updated weights for policy 0, policy_version 207443 (0.0008) [2023-12-26 16:52:34,003][105692] Updated weights for policy 0, policy_version 207453 (0.0010) [2023-12-26 16:52:34,005][105620] Updated weights for policy 1, policy_version 208178 (0.0005) [2023-12-26 16:52:34,050][105692] Updated weights for policy 0, policy_version 207463 (0.0010) [2023-12-26 16:52:34,053][105620] Updated weights for policy 1, policy_version 208188 (0.0005) [2023-12-26 16:52:34,099][105620] Updated weights for policy 1, policy_version 208198 (0.0008) [2023-12-26 16:52:34,101][105692] Updated weights for policy 0, policy_version 207473 (0.0010) [2023-12-26 16:52:34,878][105692] Updated weights for policy 0, policy_version 207483 (0.0010) [2023-12-26 16:52:34,888][105620] Updated weights for policy 1, policy_version 208208 (0.0008) [2023-12-26 16:52:34,929][105692] Updated weights for policy 0, policy_version 207493 (0.0010) [2023-12-26 16:52:34,942][105620] Updated weights for policy 1, policy_version 208218 (0.0007) [2023-12-26 16:52:34,980][105692] Updated weights for policy 0, policy_version 207503 (0.0010) [2023-12-26 16:52:35,001][105620] Updated weights for policy 1, policy_version 208228 (0.0010) [2023-12-26 16:52:35,604][105620] Updated weights for policy 1, policy_version 208238 (0.0007) [2023-12-26 16:52:35,669][105620] Updated weights for policy 1, policy_version 208248 (0.0005) [2023-12-26 16:52:35,733][105620] Updated weights for policy 1, policy_version 208258 (0.0005) [2023-12-26 16:52:35,739][105692] Updated weights for policy 0, policy_version 207513 (0.0010) [2023-12-26 16:52:35,794][105692] Updated weights for policy 0, policy_version 207523 (0.0009) [2023-12-26 16:52:35,849][105692] Updated weights for policy 0, policy_version 207533 (0.0010) [2023-12-26 16:52:35,897][105692] Updated weights for policy 0, policy_version 207543 (0.0010) [2023-12-26 16:52:36,062][104569] Fps is (10 sec: 21298.9, 60 sec: 19933.8, 300 sec: 19633.0). Total num frames: 106463232. Throughput: 0: 9961.9, 1: 9999.4. Samples: 106448644. Policy #0 lag: (min: 27.0, avg: 32.9, max: 59.0) [2023-12-26 16:52:36,063][104569] Avg episode reward: [(0, '1863.087'), (1, '8910.999')] [2023-12-26 16:52:36,392][105620] Updated weights for policy 1, policy_version 208268 (0.0008) [2023-12-26 16:52:36,451][105620] Updated weights for policy 1, policy_version 208278 (0.0010) [2023-12-26 16:52:36,505][105620] Updated weights for policy 1, policy_version 208288 (0.0010) [2023-12-26 16:52:36,556][105692] Updated weights for policy 0, policy_version 207553 (0.0009) [2023-12-26 16:52:36,616][105692] Updated weights for policy 0, policy_version 207563 (0.0008) [2023-12-26 16:52:36,681][105692] Updated weights for policy 0, policy_version 207573 (0.0009) [2023-12-26 16:52:37,269][105692] Updated weights for policy 0, policy_version 207583 (0.0006) [2023-12-26 16:52:37,328][105692] Updated weights for policy 0, policy_version 207593 (0.0010) [2023-12-26 16:52:37,369][105620] Updated weights for policy 1, policy_version 208298 (0.0007) [2023-12-26 16:52:37,390][105692] Updated weights for policy 0, policy_version 207603 (0.0010) [2023-12-26 16:52:37,424][105620] Updated weights for policy 1, policy_version 208308 (0.0005) [2023-12-26 16:52:37,480][105620] Updated weights for policy 1, policy_version 208318 (0.0008) [2023-12-26 16:52:37,532][105620] Updated weights for policy 1, policy_version 208328 (0.0009) [2023-12-26 16:52:37,987][105692] Updated weights for policy 0, policy_version 207613 (0.0010) [2023-12-26 16:52:38,052][105692] Updated weights for policy 0, policy_version 207623 (0.0006) [2023-12-26 16:52:38,120][105692] Updated weights for policy 0, policy_version 207633 (0.0008) [2023-12-26 16:52:38,272][105620] Updated weights for policy 1, policy_version 208338 (0.0006) [2023-12-26 16:52:38,339][105620] Updated weights for policy 1, policy_version 208348 (0.0007) [2023-12-26 16:52:38,401][105620] Updated weights for policy 1, policy_version 208358 (0.0008) [2023-12-26 16:52:38,838][105692] Updated weights for policy 0, policy_version 207643 (0.0010) [2023-12-26 16:52:38,897][105692] Updated weights for policy 0, policy_version 207653 (0.0008) [2023-12-26 16:52:38,963][105692] Updated weights for policy 0, policy_version 207663 (0.0007) [2023-12-26 16:52:39,086][105620] Updated weights for policy 1, policy_version 208368 (0.0010) [2023-12-26 16:52:39,141][105620] Updated weights for policy 1, policy_version 208378 (0.0011) [2023-12-26 16:52:39,197][105620] Updated weights for policy 1, policy_version 208388 (0.0010) [2023-12-26 16:52:39,727][105692] Updated weights for policy 0, policy_version 207673 (0.0007) [2023-12-26 16:52:39,795][105692] Updated weights for policy 0, policy_version 207683 (0.0010) [2023-12-26 16:52:39,860][105692] Updated weights for policy 0, policy_version 207693 (0.0008) [2023-12-26 16:52:39,868][105620] Updated weights for policy 1, policy_version 208398 (0.0010) [2023-12-26 16:52:39,929][105692] Updated weights for policy 0, policy_version 207703 (0.0007) [2023-12-26 16:52:39,933][105620] Updated weights for policy 1, policy_version 208408 (0.0011) [2023-12-26 16:52:39,983][105620] Updated weights for policy 1, policy_version 208418 (0.0011) [2023-12-26 16:52:40,657][105692] Updated weights for policy 0, policy_version 207713 (0.0008) [2023-12-26 16:52:40,709][105692] Updated weights for policy 0, policy_version 207723 (0.0006) [2023-12-26 16:52:40,760][105620] Updated weights for policy 1, policy_version 208428 (0.0009) [2023-12-26 16:52:40,770][105692] Updated weights for policy 0, policy_version 207733 (0.0007) [2023-12-26 16:52:40,817][105620] Updated weights for policy 1, policy_version 208438 (0.0006) [2023-12-26 16:52:40,869][105620] Updated weights for policy 1, policy_version 208448 (0.0006) [2023-12-26 16:52:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 106561536. Throughput: 0: 9994.3, 1: 9973.6. Samples: 106566980. Policy #0 lag: (min: 27.0, avg: 32.9, max: 59.0) [2023-12-26 16:52:41,063][104569] Avg episode reward: [(0, '2332.610'), (1, '8108.704')] [2023-12-26 16:52:41,524][105692] Updated weights for policy 0, policy_version 207743 (0.0009) [2023-12-26 16:52:41,558][105620] Updated weights for policy 1, policy_version 208458 (0.0008) [2023-12-26 16:52:41,581][105692] Updated weights for policy 0, policy_version 207753 (0.0008) [2023-12-26 16:52:41,625][105620] Updated weights for policy 1, policy_version 208468 (0.0008) [2023-12-26 16:52:41,642][105692] Updated weights for policy 0, policy_version 207763 (0.0008) [2023-12-26 16:52:41,695][105620] Updated weights for policy 1, policy_version 208478 (0.0008) [2023-12-26 16:52:41,761][105620] Updated weights for policy 1, policy_version 208488 (0.0009) [2023-12-26 16:52:42,423][105692] Updated weights for policy 0, policy_version 207773 (0.0008) [2023-12-26 16:52:42,475][105692] Updated weights for policy 0, policy_version 207783 (0.0010) [2023-12-26 16:52:42,527][105692] Updated weights for policy 0, policy_version 207793 (0.0010) [2023-12-26 16:52:42,533][105620] Updated weights for policy 1, policy_version 208498 (0.0008) [2023-12-26 16:52:42,597][105620] Updated weights for policy 1, policy_version 208508 (0.0010) [2023-12-26 16:52:42,657][105620] Updated weights for policy 1, policy_version 208518 (0.0011) [2023-12-26 16:52:43,228][105692] Updated weights for policy 0, policy_version 207803 (0.0009) [2023-12-26 16:52:43,282][105692] Updated weights for policy 0, policy_version 207813 (0.0006) [2023-12-26 16:52:43,330][105692] Updated weights for policy 0, policy_version 207823 (0.0010) [2023-12-26 16:52:43,343][105620] Updated weights for policy 1, policy_version 208528 (0.0006) [2023-12-26 16:52:43,391][105620] Updated weights for policy 1, policy_version 208538 (0.0008) [2023-12-26 16:52:43,445][105620] Updated weights for policy 1, policy_version 208548 (0.0005) [2023-12-26 16:52:43,916][105692] Updated weights for policy 0, policy_version 207833 (0.0010) [2023-12-26 16:52:43,964][105692] Updated weights for policy 0, policy_version 207843 (0.0010) [2023-12-26 16:52:44,011][105692] Updated weights for policy 0, policy_version 207853 (0.0010) [2023-12-26 16:52:44,063][105692] Updated weights for policy 0, policy_version 207863 (0.0008) [2023-12-26 16:52:44,078][105620] Updated weights for policy 1, policy_version 208558 (0.0005) [2023-12-26 16:52:44,141][105620] Updated weights for policy 1, policy_version 208568 (0.0007) [2023-12-26 16:52:44,202][105620] Updated weights for policy 1, policy_version 208578 (0.0010) [2023-12-26 16:52:44,679][105692] Updated weights for policy 0, policy_version 207873 (0.0010) [2023-12-26 16:52:44,746][105692] Updated weights for policy 0, policy_version 207883 (0.0006) [2023-12-26 16:52:44,766][105620] Updated weights for policy 1, policy_version 208588 (0.0010) [2023-12-26 16:52:44,814][105692] Updated weights for policy 0, policy_version 207893 (0.0008) [2023-12-26 16:52:44,837][105620] Updated weights for policy 1, policy_version 208598 (0.0007) [2023-12-26 16:52:44,893][105620] Updated weights for policy 1, policy_version 208608 (0.0009) [2023-12-26 16:52:45,484][105620] Updated weights for policy 1, policy_version 208618 (0.0008) [2023-12-26 16:52:45,540][105620] Updated weights for policy 1, policy_version 208628 (0.0005) [2023-12-26 16:52:45,583][105692] Updated weights for policy 0, policy_version 207903 (0.0008) [2023-12-26 16:52:45,597][105620] Updated weights for policy 1, policy_version 208638 (0.0005) [2023-12-26 16:52:45,636][105692] Updated weights for policy 0, policy_version 207913 (0.0008) [2023-12-26 16:52:45,649][105620] Updated weights for policy 1, policy_version 208648 (0.0005) [2023-12-26 16:52:45,692][105692] Updated weights for policy 0, policy_version 207923 (0.0009) [2023-12-26 16:52:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19933.9, 300 sec: 19660.8). Total num frames: 106659840. Throughput: 0: 9960.7, 1: 9989.5. Samples: 106625320. Policy #0 lag: (min: 27.0, avg: 32.9, max: 59.0) [2023-12-26 16:52:46,063][104569] Avg episode reward: [(0, '6743.393'), (1, '8631.462')] [2023-12-26 16:52:46,074][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000207928_53239808.pth... [2023-12-26 16:52:46,076][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000208648_53420032.pth... [2023-12-26 16:52:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000206744_52936704.pth [2023-12-26 16:52:46,081][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000207464_53116928.pth [2023-12-26 16:52:46,176][105620] Updated weights for policy 1, policy_version 208658 (0.0006) [2023-12-26 16:52:46,235][105620] Updated weights for policy 1, policy_version 208668 (0.0005) [2023-12-26 16:52:46,289][105620] Updated weights for policy 1, policy_version 208678 (0.0005) [2023-12-26 16:52:46,544][105692] Updated weights for policy 0, policy_version 207933 (0.0010) [2023-12-26 16:52:46,595][105692] Updated weights for policy 0, policy_version 207943 (0.0010) [2023-12-26 16:52:46,637][105692] Updated weights for policy 0, policy_version 207953 (0.0007) [2023-12-26 16:52:46,849][105620] Updated weights for policy 1, policy_version 208688 (0.0009) [2023-12-26 16:52:46,900][105620] Updated weights for policy 1, policy_version 208698 (0.0010) [2023-12-26 16:52:46,952][105620] Updated weights for policy 1, policy_version 208708 (0.0010) [2023-12-26 16:52:47,343][105692] Updated weights for policy 0, policy_version 207963 (0.0007) [2023-12-26 16:52:47,391][105692] Updated weights for policy 0, policy_version 207973 (0.0010) [2023-12-26 16:52:47,439][105692] Updated weights for policy 0, policy_version 207983 (0.0010) [2023-12-26 16:52:47,718][105620] Updated weights for policy 1, policy_version 208718 (0.0009) [2023-12-26 16:52:47,782][105620] Updated weights for policy 1, policy_version 208728 (0.0010) [2023-12-26 16:52:47,849][105620] Updated weights for policy 1, policy_version 208738 (0.0011) [2023-12-26 16:52:48,206][105692] Updated weights for policy 0, policy_version 207993 (0.0011) [2023-12-26 16:52:48,255][105692] Updated weights for policy 0, policy_version 208003 (0.0008) [2023-12-26 16:52:48,306][105692] Updated weights for policy 0, policy_version 208013 (0.0008) [2023-12-26 16:52:48,320][105585] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000000 [2023-12-26 16:52:48,558][105620] Updated weights for policy 1, policy_version 208748 (0.0008) [2023-12-26 16:52:48,623][105620] Updated weights for policy 1, policy_version 208758 (0.0006) [2023-12-26 16:52:48,681][105620] Updated weights for policy 1, policy_version 208768 (0.0011) [2023-12-26 16:52:49,117][105692] Updated weights for policy 0, policy_version 208023 (0.0008) [2023-12-26 16:52:49,173][105692] Updated weights for policy 0, policy_version 208033 (0.0008) [2023-12-26 16:52:49,238][105692] Updated weights for policy 0, policy_version 208043 (0.0008) [2023-12-26 16:52:49,386][105620] Updated weights for policy 1, policy_version 208778 (0.0010) [2023-12-26 16:52:49,438][105620] Updated weights for policy 1, policy_version 208788 (0.0010) [2023-12-26 16:52:49,483][105620] Updated weights for policy 1, policy_version 208798 (0.0010) [2023-12-26 16:52:49,535][105620] Updated weights for policy 1, policy_version 208808 (0.0010) [2023-12-26 16:52:49,970][105692] Updated weights for policy 0, policy_version 208053 (0.0006) [2023-12-26 16:52:50,031][105692] Updated weights for policy 0, policy_version 208063 (0.0007) [2023-12-26 16:52:50,092][105692] Updated weights for policy 0, policy_version 208073 (0.0008) [2023-12-26 16:52:50,273][105620] Updated weights for policy 1, policy_version 208818 (0.0007) [2023-12-26 16:52:50,325][105620] Updated weights for policy 1, policy_version 208828 (0.0006) [2023-12-26 16:52:50,379][105620] Updated weights for policy 1, policy_version 208838 (0.0009) [2023-12-26 16:52:50,863][105692] Updated weights for policy 0, policy_version 208083 (0.0008) [2023-12-26 16:52:50,924][105692] Updated weights for policy 0, policy_version 208093 (0.0006) [2023-12-26 16:52:50,979][105692] Updated weights for policy 0, policy_version 208103 (0.0006) [2023-12-26 16:52:51,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 106758144. Throughput: 0: 9935.6, 1: 10055.6. Samples: 106746996. Policy #0 lag: (min: 27.0, avg: 32.9, max: 59.0) [2023-12-26 16:52:51,062][104569] Avg episode reward: [(0, '9259.074'), (1, '9352.430')] [2023-12-26 16:52:51,121][105620] Updated weights for policy 1, policy_version 208848 (0.0007) [2023-12-26 16:52:51,188][105620] Updated weights for policy 1, policy_version 208858 (0.0007) [2023-12-26 16:52:51,244][105620] Updated weights for policy 1, policy_version 208868 (0.0007) [2023-12-26 16:52:51,646][105692] Updated weights for policy 0, policy_version 208113 (0.0007) [2023-12-26 16:52:51,703][105692] Updated weights for policy 0, policy_version 208123 (0.0006) [2023-12-26 16:52:51,766][105692] Updated weights for policy 0, policy_version 208133 (0.0008) [2023-12-26 16:52:51,828][105692] Updated weights for policy 0, policy_version 208143 (0.0009) [2023-12-26 16:52:51,945][105620] Updated weights for policy 1, policy_version 208878 (0.0007) [2023-12-26 16:52:52,007][105620] Updated weights for policy 1, policy_version 208888 (0.0009) [2023-12-26 16:52:52,067][105620] Updated weights for policy 1, policy_version 208898 (0.0009) [2023-12-26 16:52:52,591][105692] Updated weights for policy 0, policy_version 208153 (0.0009) [2023-12-26 16:52:52,642][105692] Updated weights for policy 0, policy_version 208163 (0.0009) [2023-12-26 16:52:52,704][105692] Updated weights for policy 0, policy_version 208173 (0.0009) [2023-12-26 16:52:52,795][105620] Updated weights for policy 1, policy_version 208908 (0.0008) [2023-12-26 16:52:52,846][105620] Updated weights for policy 1, policy_version 208918 (0.0009) [2023-12-26 16:52:52,897][105620] Updated weights for policy 1, policy_version 208928 (0.0009) [2023-12-26 16:52:53,484][105692] Updated weights for policy 0, policy_version 208183 (0.0010) [2023-12-26 16:52:53,541][105692] Updated weights for policy 0, policy_version 208193 (0.0010) [2023-12-26 16:52:53,589][105692] Updated weights for policy 0, policy_version 208203 (0.0010) [2023-12-26 16:52:53,613][105620] Updated weights for policy 1, policy_version 208938 (0.0009) [2023-12-26 16:52:53,660][105620] Updated weights for policy 1, policy_version 208948 (0.0008) [2023-12-26 16:52:53,706][105620] Updated weights for policy 1, policy_version 208958 (0.0007) [2023-12-26 16:52:53,768][105620] Updated weights for policy 1, policy_version 208968 (0.0005) [2023-12-26 16:52:54,254][105692] Updated weights for policy 0, policy_version 208213 (0.0010) [2023-12-26 16:52:54,313][105692] Updated weights for policy 0, policy_version 208223 (0.0010) [2023-12-26 16:52:54,369][105620] Updated weights for policy 1, policy_version 208978 (0.0006) [2023-12-26 16:52:54,373][105692] Updated weights for policy 0, policy_version 208233 (0.0011) [2023-12-26 16:52:54,435][105620] Updated weights for policy 1, policy_version 208988 (0.0009) [2023-12-26 16:52:54,496][105620] Updated weights for policy 1, policy_version 208998 (0.0008) [2023-12-26 16:52:55,018][105692] Updated weights for policy 0, policy_version 208243 (0.0010) [2023-12-26 16:52:55,066][105692] Updated weights for policy 0, policy_version 208253 (0.0010) [2023-12-26 16:52:55,115][105692] Updated weights for policy 0, policy_version 208263 (0.0010) [2023-12-26 16:52:55,302][105620] Updated weights for policy 1, policy_version 209008 (0.0008) [2023-12-26 16:52:55,363][105620] Updated weights for policy 1, policy_version 209018 (0.0006) [2023-12-26 16:52:55,416][105620] Updated weights for policy 1, policy_version 209028 (0.0005) [2023-12-26 16:52:55,750][105692] Updated weights for policy 0, policy_version 208273 (0.0007) [2023-12-26 16:52:55,800][105692] Updated weights for policy 0, policy_version 208283 (0.0009) [2023-12-26 16:52:55,843][105692] Updated weights for policy 0, policy_version 208293 (0.0008) [2023-12-26 16:52:55,893][105692] Updated weights for policy 0, policy_version 208303 (0.0005) [2023-12-26 16:52:56,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 106856448. Throughput: 0: 9952.0, 1: 10046.3. Samples: 106864692. Policy #0 lag: (min: 27.0, avg: 32.9, max: 59.0) [2023-12-26 16:52:56,062][104569] Avg episode reward: [(0, '9350.472'), (1, '9064.329')] [2023-12-26 16:52:56,149][105620] Updated weights for policy 1, policy_version 209038 (0.0008) [2023-12-26 16:52:56,221][105620] Updated weights for policy 1, policy_version 209048 (0.0009) [2023-12-26 16:52:56,281][105620] Updated weights for policy 1, policy_version 209058 (0.0006) [2023-12-26 16:52:56,612][105692] Updated weights for policy 0, policy_version 208313 (0.0010) [2023-12-26 16:52:56,675][105692] Updated weights for policy 0, policy_version 208323 (0.0010) [2023-12-26 16:52:56,737][105692] Updated weights for policy 0, policy_version 208333 (0.0011) [2023-12-26 16:52:56,965][105620] Updated weights for policy 1, policy_version 209068 (0.0007) [2023-12-26 16:52:57,024][105620] Updated weights for policy 1, policy_version 209078 (0.0008) [2023-12-26 16:52:57,080][105586] KL-divergence is very high: 106.7837 [2023-12-26 16:52:57,086][105586] KL-divergence is very high: 153.9676 [2023-12-26 16:52:57,086][105620] Updated weights for policy 1, policy_version 209088 (0.0008) [2023-12-26 16:52:57,468][105692] Updated weights for policy 0, policy_version 208343 (0.0010) [2023-12-26 16:52:57,515][105692] Updated weights for policy 0, policy_version 208353 (0.0010) [2023-12-26 16:52:57,563][105692] Updated weights for policy 0, policy_version 208363 (0.0010) [2023-12-26 16:52:57,836][105620] Updated weights for policy 1, policy_version 209098 (0.0008) [2023-12-26 16:52:57,886][105620] Updated weights for policy 1, policy_version 209108 (0.0008) [2023-12-26 16:52:57,931][105620] Updated weights for policy 1, policy_version 209118 (0.0007) [2023-12-26 16:52:57,980][105620] Updated weights for policy 1, policy_version 209128 (0.0008) [2023-12-26 16:52:58,344][105692] Updated weights for policy 0, policy_version 208373 (0.0010) [2023-12-26 16:52:58,412][105692] Updated weights for policy 0, policy_version 208383 (0.0010) [2023-12-26 16:52:58,477][105692] Updated weights for policy 0, policy_version 208393 (0.0009) [2023-12-26 16:52:58,790][105620] Updated weights for policy 1, policy_version 209138 (0.0007) [2023-12-26 16:52:58,852][105620] Updated weights for policy 1, policy_version 209148 (0.0008) [2023-12-26 16:52:58,913][105620] Updated weights for policy 1, policy_version 209158 (0.0007) [2023-12-26 16:52:59,195][105692] Updated weights for policy 0, policy_version 208403 (0.0009) [2023-12-26 16:52:59,257][105692] Updated weights for policy 0, policy_version 208413 (0.0011) [2023-12-26 16:52:59,312][105692] Updated weights for policy 0, policy_version 208423 (0.0010) [2023-12-26 16:52:59,600][105620] Updated weights for policy 1, policy_version 209168 (0.0008) [2023-12-26 16:52:59,648][105620] Updated weights for policy 1, policy_version 209178 (0.0008) [2023-12-26 16:52:59,703][105620] Updated weights for policy 1, policy_version 209188 (0.0008) [2023-12-26 16:53:00,067][105692] Updated weights for policy 0, policy_version 208433 (0.0011) [2023-12-26 16:53:00,119][105692] Updated weights for policy 0, policy_version 208443 (0.0010) [2023-12-26 16:53:00,163][105692] Updated weights for policy 0, policy_version 208453 (0.0010) [2023-12-26 16:53:00,218][105692] Updated weights for policy 0, policy_version 208463 (0.0010) [2023-12-26 16:53:00,476][105620] Updated weights for policy 1, policy_version 209198 (0.0008) [2023-12-26 16:53:00,531][105620] Updated weights for policy 1, policy_version 209208 (0.0008) [2023-12-26 16:53:00,586][105620] Updated weights for policy 1, policy_version 209218 (0.0008) [2023-12-26 16:53:00,991][105692] Updated weights for policy 0, policy_version 208473 (0.0010) [2023-12-26 16:53:01,046][105692] Updated weights for policy 0, policy_version 208483 (0.0010) [2023-12-26 16:53:01,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19797.2, 300 sec: 19605.3). Total num frames: 106946560. Throughput: 0: 9930.4, 1: 10021.5. Samples: 106920868. Policy #0 lag: (min: 27.0, avg: 32.9, max: 59.0) [2023-12-26 16:53:01,063][104569] Avg episode reward: [(0, '9350.367'), (1, '8237.343')] [2023-12-26 16:53:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000209224_53567488.pth... [2023-12-26 16:53:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000208072_53272576.pth [2023-12-26 16:53:01,112][105692] Updated weights for policy 0, policy_version 208493 (0.0010) [2023-12-26 16:53:01,131][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000208496_53387264.pth... [2023-12-26 16:53:01,136][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000207352_53092352.pth [2023-12-26 16:53:01,352][105620] Updated weights for policy 1, policy_version 209228 (0.0009) [2023-12-26 16:53:01,413][105620] Updated weights for policy 1, policy_version 209238 (0.0009) [2023-12-26 16:53:01,478][105620] Updated weights for policy 1, policy_version 209248 (0.0009) [2023-12-26 16:53:01,768][105692] Updated weights for policy 0, policy_version 208503 (0.0007) [2023-12-26 16:53:01,827][105692] Updated weights for policy 0, policy_version 208513 (0.0005) [2023-12-26 16:53:01,894][105692] Updated weights for policy 0, policy_version 208523 (0.0008) [2023-12-26 16:53:02,260][105620] Updated weights for policy 1, policy_version 209258 (0.0009) [2023-12-26 16:53:02,312][105620] Updated weights for policy 1, policy_version 209268 (0.0011) [2023-12-26 16:53:02,377][105620] Updated weights for policy 1, policy_version 209278 (0.0009) [2023-12-26 16:53:02,443][105620] Updated weights for policy 1, policy_version 209288 (0.0005) [2023-12-26 16:53:02,557][105692] Updated weights for policy 0, policy_version 208533 (0.0009) [2023-12-26 16:53:02,619][105692] Updated weights for policy 0, policy_version 208543 (0.0008) [2023-12-26 16:53:02,675][105692] Updated weights for policy 0, policy_version 208553 (0.0008) [2023-12-26 16:53:03,063][105620] Updated weights for policy 1, policy_version 209298 (0.0009) [2023-12-26 16:53:03,126][105620] Updated weights for policy 1, policy_version 209308 (0.0011) [2023-12-26 16:53:03,187][105620] Updated weights for policy 1, policy_version 209318 (0.0010) [2023-12-26 16:53:03,422][105692] Updated weights for policy 0, policy_version 208563 (0.0007) [2023-12-26 16:53:03,475][105692] Updated weights for policy 0, policy_version 208573 (0.0005) [2023-12-26 16:53:03,534][105692] Updated weights for policy 0, policy_version 208583 (0.0006) [2023-12-26 16:53:03,920][105620] Updated weights for policy 1, policy_version 209328 (0.0011) [2023-12-26 16:53:03,984][105620] Updated weights for policy 1, policy_version 209338 (0.0011) [2023-12-26 16:53:04,048][105620] Updated weights for policy 1, policy_version 209348 (0.0011) [2023-12-26 16:53:04,261][105692] Updated weights for policy 0, policy_version 208593 (0.0008) [2023-12-26 16:53:04,319][105692] Updated weights for policy 0, policy_version 208603 (0.0007) [2023-12-26 16:53:04,382][105692] Updated weights for policy 0, policy_version 208613 (0.0007) [2023-12-26 16:53:04,439][105692] Updated weights for policy 0, policy_version 208623 (0.0008) [2023-12-26 16:53:04,792][105620] Updated weights for policy 1, policy_version 209358 (0.0008) [2023-12-26 16:53:04,836][105620] Updated weights for policy 1, policy_version 209368 (0.0005) [2023-12-26 16:53:04,882][105620] Updated weights for policy 1, policy_version 209378 (0.0005) [2023-12-26 16:53:05,262][105692] Updated weights for policy 0, policy_version 208633 (0.0006) [2023-12-26 16:53:05,329][105692] Updated weights for policy 0, policy_version 208643 (0.0006) [2023-12-26 16:53:05,388][105692] Updated weights for policy 0, policy_version 208653 (0.0008) [2023-12-26 16:53:05,469][105620] Updated weights for policy 1, policy_version 209388 (0.0005) [2023-12-26 16:53:05,537][105620] Updated weights for policy 1, policy_version 209398 (0.0006) [2023-12-26 16:53:05,586][105620] Updated weights for policy 1, policy_version 209408 (0.0006) [2023-12-26 16:53:05,957][105692] Updated weights for policy 0, policy_version 208663 (0.0007) [2023-12-26 16:53:06,012][105692] Updated weights for policy 0, policy_version 208673 (0.0005) [2023-12-26 16:53:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 107044864. Throughput: 0: 9826.6, 1: 9891.9. Samples: 107035852. Policy #0 lag: (min: 27.0, avg: 32.9, max: 59.0) [2023-12-26 16:53:06,062][105692] Updated weights for policy 0, policy_version 208683 (0.0006) [2023-12-26 16:53:06,062][104569] Avg episode reward: [(0, '9258.202'), (1, '8449.617')] [2023-12-26 16:53:06,168][105620] Updated weights for policy 1, policy_version 209418 (0.0006) [2023-12-26 16:53:06,231][105620] Updated weights for policy 1, policy_version 209428 (0.0010) [2023-12-26 16:53:06,293][105620] Updated weights for policy 1, policy_version 209438 (0.0011) [2023-12-26 16:53:06,356][105620] Updated weights for policy 1, policy_version 209448 (0.0010) [2023-12-26 16:53:06,779][105692] Updated weights for policy 0, policy_version 208693 (0.0007) [2023-12-26 16:53:06,832][105692] Updated weights for policy 0, policy_version 208703 (0.0008) [2023-12-26 16:53:06,892][105692] Updated weights for policy 0, policy_version 208713 (0.0008) [2023-12-26 16:53:07,086][105620] Updated weights for policy 1, policy_version 209458 (0.0010) [2023-12-26 16:53:07,145][105620] Updated weights for policy 1, policy_version 209468 (0.0010) [2023-12-26 16:53:07,203][105620] Updated weights for policy 1, policy_version 209478 (0.0010) [2023-12-26 16:53:07,673][105692] Updated weights for policy 0, policy_version 208723 (0.0008) [2023-12-26 16:53:07,738][105692] Updated weights for policy 0, policy_version 208733 (0.0009) [2023-12-26 16:53:07,798][105692] Updated weights for policy 0, policy_version 208743 (0.0008) [2023-12-26 16:53:07,938][105620] Updated weights for policy 1, policy_version 209488 (0.0010) [2023-12-26 16:53:08,001][105620] Updated weights for policy 1, policy_version 209498 (0.0009) [2023-12-26 16:53:08,058][105620] Updated weights for policy 1, policy_version 209508 (0.0005) [2023-12-26 16:53:08,613][105620] Updated weights for policy 1, policy_version 209518 (0.0008) [2023-12-26 16:53:08,670][105620] Updated weights for policy 1, policy_version 209528 (0.0010) [2023-12-26 16:53:08,674][105692] Updated weights for policy 0, policy_version 208753 (0.0009) [2023-12-26 16:53:08,732][105620] Updated weights for policy 1, policy_version 209538 (0.0010) [2023-12-26 16:53:08,732][105692] Updated weights for policy 0, policy_version 208763 (0.0008) [2023-12-26 16:53:08,791][105692] Updated weights for policy 0, policy_version 208773 (0.0008) [2023-12-26 16:53:08,854][105692] Updated weights for policy 0, policy_version 208783 (0.0008) [2023-12-26 16:53:09,467][105620] Updated weights for policy 1, policy_version 209548 (0.0010) [2023-12-26 16:53:09,527][105620] Updated weights for policy 1, policy_version 209558 (0.0011) [2023-12-26 16:53:09,557][105692] Updated weights for policy 0, policy_version 208793 (0.0006) [2023-12-26 16:53:09,587][105620] Updated weights for policy 1, policy_version 209568 (0.0011) [2023-12-26 16:53:09,625][105692] Updated weights for policy 0, policy_version 208803 (0.0006) [2023-12-26 16:53:09,687][105692] Updated weights for policy 0, policy_version 208813 (0.0008) [2023-12-26 16:53:10,361][105620] Updated weights for policy 1, policy_version 209578 (0.0011) [2023-12-26 16:53:10,427][105620] Updated weights for policy 1, policy_version 209588 (0.0010) [2023-12-26 16:53:10,463][105692] Updated weights for policy 0, policy_version 208823 (0.0009) [2023-12-26 16:53:10,475][105620] Updated weights for policy 1, policy_version 209598 (0.0008) [2023-12-26 16:53:10,516][105692] Updated weights for policy 0, policy_version 208833 (0.0006) [2023-12-26 16:53:10,540][105620] Updated weights for policy 1, policy_version 209608 (0.0010) [2023-12-26 16:53:10,563][105692] Updated weights for policy 0, policy_version 208843 (0.0008) [2023-12-26 16:53:11,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 107143168. Throughput: 0: 9739.9, 1: 9971.4. Samples: 107152480. Policy #0 lag: (min: 27.0, avg: 32.9, max: 59.0) [2023-12-26 16:53:11,063][104569] Avg episode reward: [(0, '9257.097'), (1, '8443.934')] [2023-12-26 16:53:11,279][105586] KL-divergence is very high: 135.5793 [2023-12-26 16:53:11,292][105586] KL-divergence is very high: 173.7425 [2023-12-26 16:53:11,306][105620] Updated weights for policy 1, policy_version 209618 (0.0011) [2023-12-26 16:53:11,319][105586] KL-divergence is very high: 220.0200 [2023-12-26 16:53:11,332][105586] KL-divergence is very high: 217.0270 [2023-12-26 16:53:11,346][105586] KL-divergence is very high: 209.1129 [2023-12-26 16:53:11,375][105620] Updated weights for policy 1, policy_version 209628 (0.0010) [2023-12-26 16:53:11,376][105586] KL-divergence is very high: 180.0019 [2023-12-26 16:53:11,394][105586] KL-divergence is very high: 156.9114 [2023-12-26 16:53:11,407][105586] KL-divergence is very high: 135.7358 [2023-12-26 16:53:11,422][105692] Updated weights for policy 0, policy_version 208853 (0.0009) [2023-12-26 16:53:11,447][105620] Updated weights for policy 1, policy_version 209638 (0.0009) [2023-12-26 16:53:11,475][105692] Updated weights for policy 0, policy_version 208863 (0.0009) [2023-12-26 16:53:11,524][105692] Updated weights for policy 0, policy_version 208873 (0.0008) [2023-12-26 16:53:12,231][105620] Updated weights for policy 1, policy_version 209648 (0.0011) [2023-12-26 16:53:12,297][105620] Updated weights for policy 1, policy_version 209658 (0.0011) [2023-12-26 16:53:12,335][105692] Updated weights for policy 0, policy_version 208883 (0.0007) [2023-12-26 16:53:12,360][105620] Updated weights for policy 1, policy_version 209668 (0.0010) [2023-12-26 16:53:12,396][105692] Updated weights for policy 0, policy_version 208893 (0.0008) [2023-12-26 16:53:12,450][105692] Updated weights for policy 0, policy_version 208903 (0.0008) [2023-12-26 16:53:13,118][105620] Updated weights for policy 1, policy_version 209678 (0.0011) [2023-12-26 16:53:13,174][105620] Updated weights for policy 1, policy_version 209688 (0.0010) [2023-12-26 16:53:13,225][105620] Updated weights for policy 1, policy_version 209698 (0.0010) [2023-12-26 16:53:13,227][105692] Updated weights for policy 0, policy_version 208913 (0.0008) [2023-12-26 16:53:13,277][105692] Updated weights for policy 0, policy_version 208923 (0.0006) [2023-12-26 16:53:13,336][105692] Updated weights for policy 0, policy_version 208933 (0.0008) [2023-12-26 16:53:13,392][105692] Updated weights for policy 0, policy_version 208943 (0.0007) [2023-12-26 16:53:13,976][105620] Updated weights for policy 1, policy_version 209708 (0.0010) [2023-12-26 16:53:14,025][105620] Updated weights for policy 1, policy_version 209718 (0.0010) [2023-12-26 16:53:14,080][105620] Updated weights for policy 1, policy_version 209728 (0.0008) [2023-12-26 16:53:14,086][105692] Updated weights for policy 0, policy_version 208953 (0.0010) [2023-12-26 16:53:14,147][105692] Updated weights for policy 0, policy_version 208963 (0.0010) [2023-12-26 16:53:14,202][105692] Updated weights for policy 0, policy_version 208973 (0.0010) [2023-12-26 16:53:14,822][105620] Updated weights for policy 1, policy_version 209738 (0.0009) [2023-12-26 16:53:14,876][105620] Updated weights for policy 1, policy_version 209748 (0.0005) [2023-12-26 16:53:14,924][105620] Updated weights for policy 1, policy_version 209758 (0.0005) [2023-12-26 16:53:14,956][105692] Updated weights for policy 0, policy_version 208983 (0.0011) [2023-12-26 16:53:14,974][105620] Updated weights for policy 1, policy_version 209768 (0.0007) [2023-12-26 16:53:15,020][105692] Updated weights for policy 0, policy_version 208993 (0.0011) [2023-12-26 16:53:15,079][105692] Updated weights for policy 0, policy_version 209003 (0.0011) [2023-12-26 16:53:15,616][105620] Updated weights for policy 1, policy_version 209778 (0.0008) [2023-12-26 16:53:15,666][105620] Updated weights for policy 1, policy_version 209788 (0.0008) [2023-12-26 16:53:15,711][105620] Updated weights for policy 1, policy_version 209798 (0.0008) [2023-12-26 16:53:15,835][105692] Updated weights for policy 0, policy_version 209013 (0.0011) [2023-12-26 16:53:15,889][105692] Updated weights for policy 0, policy_version 209023 (0.0011) [2023-12-26 16:53:15,941][105692] Updated weights for policy 0, policy_version 209033 (0.0010) [2023-12-26 16:53:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 107241472. Throughput: 0: 9652.3, 1: 9844.2. Samples: 107206720. Policy #0 lag: (min: 27.0, avg: 32.9, max: 59.0) [2023-12-26 16:53:16,063][104569] Avg episode reward: [(0, '9346.229'), (1, '8526.577')] [2023-12-26 16:53:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000209040_53526528.pth... [2023-12-26 16:53:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000209800_53714944.pth... [2023-12-26 16:53:16,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000207928_53239808.pth [2023-12-26 16:53:16,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000208648_53420032.pth [2023-12-26 16:53:16,418][105620] Updated weights for policy 1, policy_version 209808 (0.0009) [2023-12-26 16:53:16,476][105620] Updated weights for policy 1, policy_version 209818 (0.0010) [2023-12-26 16:53:16,530][105620] Updated weights for policy 1, policy_version 209828 (0.0009) [2023-12-26 16:53:16,589][105692] Updated weights for policy 0, policy_version 209043 (0.0009) [2023-12-26 16:53:16,649][105692] Updated weights for policy 0, policy_version 209053 (0.0006) [2023-12-26 16:53:16,712][105692] Updated weights for policy 0, policy_version 209063 (0.0005) [2023-12-26 16:53:17,154][105620] Updated weights for policy 1, policy_version 209838 (0.0007) [2023-12-26 16:53:17,211][105620] Updated weights for policy 1, policy_version 209848 (0.0005) [2023-12-26 16:53:17,268][105620] Updated weights for policy 1, policy_version 209858 (0.0005) [2023-12-26 16:53:17,280][105692] Updated weights for policy 0, policy_version 209073 (0.0006) [2023-12-26 16:53:17,341][105692] Updated weights for policy 0, policy_version 209083 (0.0005) [2023-12-26 16:53:17,386][105692] Updated weights for policy 0, policy_version 209093 (0.0005) [2023-12-26 16:53:17,440][105692] Updated weights for policy 0, policy_version 209103 (0.0005) [2023-12-26 16:53:17,919][105620] Updated weights for policy 1, policy_version 209868 (0.0006) [2023-12-26 16:53:17,975][105620] Updated weights for policy 1, policy_version 209878 (0.0009) [2023-12-26 16:53:18,035][105692] Updated weights for policy 0, policy_version 209113 (0.0009) [2023-12-26 16:53:18,037][105620] Updated weights for policy 1, policy_version 209888 (0.0007) [2023-12-26 16:53:18,098][105692] Updated weights for policy 0, policy_version 209123 (0.0010) [2023-12-26 16:53:18,161][105692] Updated weights for policy 0, policy_version 209133 (0.0011) [2023-12-26 16:53:18,776][105620] Updated weights for policy 1, policy_version 209898 (0.0007) [2023-12-26 16:53:18,825][105620] Updated weights for policy 1, policy_version 209908 (0.0010) [2023-12-26 16:53:18,873][105620] Updated weights for policy 1, policy_version 209918 (0.0010) [2023-12-26 16:53:18,930][105692] Updated weights for policy 0, policy_version 209143 (0.0007) [2023-12-26 16:53:18,931][105620] Updated weights for policy 1, policy_version 209928 (0.0010) [2023-12-26 16:53:18,981][105692] Updated weights for policy 0, policy_version 209153 (0.0008) [2023-12-26 16:53:19,034][105692] Updated weights for policy 0, policy_version 209163 (0.0008) [2023-12-26 16:53:19,662][105620] Updated weights for policy 1, policy_version 209938 (0.0008) [2023-12-26 16:53:19,714][105620] Updated weights for policy 1, policy_version 209948 (0.0011) [2023-12-26 16:53:19,763][105620] Updated weights for policy 1, policy_version 209958 (0.0010) [2023-12-26 16:53:19,822][105692] Updated weights for policy 0, policy_version 209173 (0.0010) [2023-12-26 16:53:19,889][105692] Updated weights for policy 0, policy_version 209183 (0.0011) [2023-12-26 16:53:19,949][105692] Updated weights for policy 0, policy_version 209193 (0.0009) [2023-12-26 16:53:20,499][105620] Updated weights for policy 1, policy_version 209968 (0.0010) [2023-12-26 16:53:20,555][105620] Updated weights for policy 1, policy_version 209978 (0.0011) [2023-12-26 16:53:20,591][105692] Updated weights for policy 0, policy_version 209203 (0.0006) [2023-12-26 16:53:20,618][105620] Updated weights for policy 1, policy_version 209988 (0.0011) [2023-12-26 16:53:20,651][105692] Updated weights for policy 0, policy_version 209213 (0.0008) [2023-12-26 16:53:20,707][105692] Updated weights for policy 0, policy_version 209223 (0.0008) [2023-12-26 16:53:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 107339776. Throughput: 0: 9658.0, 1: 9875.2. Samples: 107327632. Policy #0 lag: (min: 27.0, avg: 32.9, max: 59.0) [2023-12-26 16:53:21,062][104569] Avg episode reward: [(0, '9345.381'), (1, '8866.887')] [2023-12-26 16:53:21,386][105692] Updated weights for policy 0, policy_version 209233 (0.0008) [2023-12-26 16:53:21,443][105692] Updated weights for policy 0, policy_version 209243 (0.0011) [2023-12-26 16:53:21,444][105620] Updated weights for policy 1, policy_version 209998 (0.0009) [2023-12-26 16:53:21,502][105692] Updated weights for policy 0, policy_version 209253 (0.0011) [2023-12-26 16:53:21,506][105620] Updated weights for policy 1, policy_version 210008 (0.0008) [2023-12-26 16:53:21,564][105692] Updated weights for policy 0, policy_version 209263 (0.0011) [2023-12-26 16:53:21,567][105620] Updated weights for policy 1, policy_version 210018 (0.0009) [2023-12-26 16:53:22,301][105692] Updated weights for policy 0, policy_version 209273 (0.0010) [2023-12-26 16:53:22,365][105692] Updated weights for policy 0, policy_version 209283 (0.0010) [2023-12-26 16:53:22,372][105620] Updated weights for policy 1, policy_version 210028 (0.0010) [2023-12-26 16:53:22,426][105692] Updated weights for policy 0, policy_version 209293 (0.0011) [2023-12-26 16:53:22,432][105620] Updated weights for policy 1, policy_version 210038 (0.0007) [2023-12-26 16:53:22,487][105620] Updated weights for policy 1, policy_version 210048 (0.0008) [2023-12-26 16:53:23,057][105692] Updated weights for policy 0, policy_version 209303 (0.0007) [2023-12-26 16:53:23,104][105692] Updated weights for policy 0, policy_version 209313 (0.0005) [2023-12-26 16:53:23,161][105692] Updated weights for policy 0, policy_version 209323 (0.0006) [2023-12-26 16:53:23,368][105620] Updated weights for policy 1, policy_version 210058 (0.0008) [2023-12-26 16:53:23,419][105620] Updated weights for policy 1, policy_version 210068 (0.0006) [2023-12-26 16:53:23,469][105620] Updated weights for policy 1, policy_version 210078 (0.0005) [2023-12-26 16:53:23,731][105692] Updated weights for policy 0, policy_version 209333 (0.0006) [2023-12-26 16:53:23,792][105692] Updated weights for policy 0, policy_version 209343 (0.0005) [2023-12-26 16:53:23,850][105692] Updated weights for policy 0, policy_version 209353 (0.0010) [2023-12-26 16:53:24,147][105620] Updated weights for policy 1, policy_version 210089 (0.0010) [2023-12-26 16:53:24,198][105620] Updated weights for policy 1, policy_version 210099 (0.0008) [2023-12-26 16:53:24,246][105620] Updated weights for policy 1, policy_version 210109 (0.0008) [2023-12-26 16:53:24,293][105620] Updated weights for policy 1, policy_version 210119 (0.0008) [2023-12-26 16:53:24,536][105692] Updated weights for policy 0, policy_version 209363 (0.0009) [2023-12-26 16:53:24,587][105692] Updated weights for policy 0, policy_version 209373 (0.0005) [2023-12-26 16:53:24,651][105692] Updated weights for policy 0, policy_version 209383 (0.0008) [2023-12-26 16:53:24,942][105620] Updated weights for policy 1, policy_version 210129 (0.0010) [2023-12-26 16:53:24,990][105620] Updated weights for policy 1, policy_version 210139 (0.0010) [2023-12-26 16:53:25,041][105620] Updated weights for policy 1, policy_version 210149 (0.0010) [2023-12-26 16:53:25,345][105692] Updated weights for policy 0, policy_version 209393 (0.0010) [2023-12-26 16:53:25,396][105692] Updated weights for policy 0, policy_version 209403 (0.0010) [2023-12-26 16:53:25,443][105692] Updated weights for policy 0, policy_version 209413 (0.0010) [2023-12-26 16:53:25,500][105692] Updated weights for policy 0, policy_version 209423 (0.0010) [2023-12-26 16:53:25,773][105620] Updated weights for policy 1, policy_version 210159 (0.0007) [2023-12-26 16:53:25,818][105620] Updated weights for policy 1, policy_version 210169 (0.0005) [2023-12-26 16:53:25,867][105620] Updated weights for policy 1, policy_version 210179 (0.0005) [2023-12-26 16:53:26,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19605.2). Total num frames: 107438080. Throughput: 0: 9697.8, 1: 9832.1. Samples: 107445828. Policy #0 lag: (min: 27.0, avg: 32.9, max: 59.0) [2023-12-26 16:53:26,063][104569] Avg episode reward: [(0, '9344.039'), (1, '9353.253')] [2023-12-26 16:53:26,241][105692] Updated weights for policy 0, policy_version 209433 (0.0011) [2023-12-26 16:53:26,295][105692] Updated weights for policy 0, policy_version 209443 (0.0009) [2023-12-26 16:53:26,358][105692] Updated weights for policy 0, policy_version 209453 (0.0005) [2023-12-26 16:53:26,612][105620] Updated weights for policy 1, policy_version 210189 (0.0005) [2023-12-26 16:53:26,659][105620] Updated weights for policy 1, policy_version 210199 (0.0005) [2023-12-26 16:53:26,708][105620] Updated weights for policy 1, policy_version 210209 (0.0005) [2023-12-26 16:53:26,982][105692] Updated weights for policy 0, policy_version 209463 (0.0009) [2023-12-26 16:53:27,035][105692] Updated weights for policy 0, policy_version 209473 (0.0011) [2023-12-26 16:53:27,091][105692] Updated weights for policy 0, policy_version 209483 (0.0011) [2023-12-26 16:53:27,309][105620] Updated weights for policy 1, policy_version 210219 (0.0006) [2023-12-26 16:53:27,367][105620] Updated weights for policy 1, policy_version 210229 (0.0006) [2023-12-26 16:53:27,426][105620] Updated weights for policy 1, policy_version 210239 (0.0007) [2023-12-26 16:53:27,834][105692] Updated weights for policy 0, policy_version 209493 (0.0010) [2023-12-26 16:53:27,888][105692] Updated weights for policy 0, policy_version 209503 (0.0010) [2023-12-26 16:53:27,950][105692] Updated weights for policy 0, policy_version 209513 (0.0010) [2023-12-26 16:53:28,014][105620] Updated weights for policy 1, policy_version 210249 (0.0008) [2023-12-26 16:53:28,063][105620] Updated weights for policy 1, policy_version 210259 (0.0006) [2023-12-26 16:53:28,122][105620] Updated weights for policy 1, policy_version 210269 (0.0005) [2023-12-26 16:53:28,175][105620] Updated weights for policy 1, policy_version 210279 (0.0009) [2023-12-26 16:53:28,546][105692] Updated weights for policy 0, policy_version 209523 (0.0010) [2023-12-26 16:53:28,611][105692] Updated weights for policy 0, policy_version 209533 (0.0009) [2023-12-26 16:53:28,678][105692] Updated weights for policy 0, policy_version 209543 (0.0008) [2023-12-26 16:53:28,851][105620] Updated weights for policy 1, policy_version 210289 (0.0006) [2023-12-26 16:53:28,901][105620] Updated weights for policy 1, policy_version 210300 (0.0006) [2023-12-26 16:53:28,950][105620] Updated weights for policy 1, policy_version 210310 (0.0008) [2023-12-26 16:53:29,312][105692] Updated weights for policy 0, policy_version 209553 (0.0007) [2023-12-26 16:53:29,379][105692] Updated weights for policy 0, policy_version 209563 (0.0009) [2023-12-26 16:53:29,437][105692] Updated weights for policy 0, policy_version 209573 (0.0010) [2023-12-26 16:53:29,492][105692] Updated weights for policy 0, policy_version 209583 (0.0010) [2023-12-26 16:53:29,717][105620] Updated weights for policy 1, policy_version 210320 (0.0008) [2023-12-26 16:53:29,774][105620] Updated weights for policy 1, policy_version 210330 (0.0009) [2023-12-26 16:53:29,831][105620] Updated weights for policy 1, policy_version 210340 (0.0008) [2023-12-26 16:53:30,223][105692] Updated weights for policy 0, policy_version 209593 (0.0010) [2023-12-26 16:53:30,271][105692] Updated weights for policy 0, policy_version 209603 (0.0010) [2023-12-26 16:53:30,319][105692] Updated weights for policy 0, policy_version 209613 (0.0010) [2023-12-26 16:53:30,625][105620] Updated weights for policy 1, policy_version 210350 (0.0008) [2023-12-26 16:53:30,673][105620] Updated weights for policy 1, policy_version 210360 (0.0008) [2023-12-26 16:53:30,723][105620] Updated weights for policy 1, policy_version 210370 (0.0008) [2023-12-26 16:53:31,027][105692] Updated weights for policy 0, policy_version 209623 (0.0008) [2023-12-26 16:53:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 107536384. Throughput: 0: 9757.3, 1: 9884.5. Samples: 107509196. Policy #0 lag: (min: 27.0, avg: 32.9, max: 59.0) [2023-12-26 16:53:31,063][104569] Avg episode reward: [(0, '9255.218'), (1, '9351.019')] [2023-12-26 16:53:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000210376_53862400.pth... [2023-12-26 16:53:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000209224_53567488.pth [2023-12-26 16:53:31,089][105692] Updated weights for policy 0, policy_version 209633 (0.0010) [2023-12-26 16:53:31,155][105692] Updated weights for policy 0, policy_version 209643 (0.0011) [2023-12-26 16:53:31,182][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000209648_53682176.pth... [2023-12-26 16:53:31,185][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000208496_53387264.pth [2023-12-26 16:53:31,588][105620] Updated weights for policy 1, policy_version 210380 (0.0009) [2023-12-26 16:53:31,654][105620] Updated weights for policy 1, policy_version 210390 (0.0008) [2023-12-26 16:53:31,710][105620] Updated weights for policy 1, policy_version 210400 (0.0008) [2023-12-26 16:53:31,790][105692] Updated weights for policy 0, policy_version 209653 (0.0008) [2023-12-26 16:53:31,850][105692] Updated weights for policy 0, policy_version 209663 (0.0008) [2023-12-26 16:53:31,901][105692] Updated weights for policy 0, policy_version 209673 (0.0005) [2023-12-26 16:53:32,434][105620] Updated weights for policy 1, policy_version 210410 (0.0008) [2023-12-26 16:53:32,486][105620] Updated weights for policy 1, policy_version 210420 (0.0008) [2023-12-26 16:53:32,543][105620] Updated weights for policy 1, policy_version 210430 (0.0006) [2023-12-26 16:53:32,602][105620] Updated weights for policy 1, policy_version 210440 (0.0008) [2023-12-26 16:53:32,655][105692] Updated weights for policy 0, policy_version 209683 (0.0008) [2023-12-26 16:53:32,710][105692] Updated weights for policy 0, policy_version 209693 (0.0008) [2023-12-26 16:53:32,760][105692] Updated weights for policy 0, policy_version 209703 (0.0008) [2023-12-26 16:53:33,322][105692] Updated weights for policy 0, policy_version 209713 (0.0007) [2023-12-26 16:53:33,381][105692] Updated weights for policy 0, policy_version 209723 (0.0006) [2023-12-26 16:53:33,430][105692] Updated weights for policy 0, policy_version 209733 (0.0008) [2023-12-26 16:53:33,436][105620] Updated weights for policy 1, policy_version 210450 (0.0005) [2023-12-26 16:53:33,489][105692] Updated weights for policy 0, policy_version 209743 (0.0006) [2023-12-26 16:53:33,494][105620] Updated weights for policy 1, policy_version 210460 (0.0006) [2023-12-26 16:53:33,540][105620] Updated weights for policy 1, policy_version 210470 (0.0008) [2023-12-26 16:53:34,141][105692] Updated weights for policy 0, policy_version 209753 (0.0008) [2023-12-26 16:53:34,205][105692] Updated weights for policy 0, policy_version 209763 (0.0010) [2023-12-26 16:53:34,267][105692] Updated weights for policy 0, policy_version 209773 (0.0008) [2023-12-26 16:53:34,290][105620] Updated weights for policy 1, policy_version 210480 (0.0009) [2023-12-26 16:53:34,352][105620] Updated weights for policy 1, policy_version 210490 (0.0008) [2023-12-26 16:53:34,412][105620] Updated weights for policy 1, policy_version 210500 (0.0007) [2023-12-26 16:53:35,015][105692] Updated weights for policy 0, policy_version 209783 (0.0008) [2023-12-26 16:53:35,085][105692] Updated weights for policy 0, policy_version 209793 (0.0008) [2023-12-26 16:53:35,096][105620] Updated weights for policy 1, policy_version 210510 (0.0008) [2023-12-26 16:53:35,147][105692] Updated weights for policy 0, policy_version 209803 (0.0008) [2023-12-26 16:53:35,149][105620] Updated weights for policy 1, policy_version 210520 (0.0006) [2023-12-26 16:53:35,208][105620] Updated weights for policy 1, policy_version 210530 (0.0007) [2023-12-26 16:53:35,867][105692] Updated weights for policy 0, policy_version 209813 (0.0006) [2023-12-26 16:53:35,919][105620] Updated weights for policy 1, policy_version 210540 (0.0008) [2023-12-26 16:53:35,931][105692] Updated weights for policy 0, policy_version 209823 (0.0005) [2023-12-26 16:53:35,977][105620] Updated weights for policy 1, policy_version 210550 (0.0010) [2023-12-26 16:53:35,991][105692] Updated weights for policy 0, policy_version 209833 (0.0006) [2023-12-26 16:53:36,038][105620] Updated weights for policy 1, policy_version 210560 (0.0009) [2023-12-26 16:53:36,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 107634688. Throughput: 0: 9826.3, 1: 9681.6. Samples: 107624852. Policy #0 lag: (min: 11.0, avg: 21.2, max: 43.0) [2023-12-26 16:53:36,063][104569] Avg episode reward: [(0, '9256.566'), (1, '9355.146')] [2023-12-26 16:53:36,572][105692] Updated weights for policy 0, policy_version 209843 (0.0005) [2023-12-26 16:53:36,629][105692] Updated weights for policy 0, policy_version 209853 (0.0006) [2023-12-26 16:53:36,691][105692] Updated weights for policy 0, policy_version 209863 (0.0006) [2023-12-26 16:53:36,743][105620] Updated weights for policy 1, policy_version 210570 (0.0009) [2023-12-26 16:53:36,814][105620] Updated weights for policy 1, policy_version 210580 (0.0011) [2023-12-26 16:53:36,870][105620] Updated weights for policy 1, policy_version 210590 (0.0011) [2023-12-26 16:53:36,926][105620] Updated weights for policy 1, policy_version 210600 (0.0010) [2023-12-26 16:53:37,336][105692] Updated weights for policy 0, policy_version 209873 (0.0006) [2023-12-26 16:53:37,392][105692] Updated weights for policy 0, policy_version 209883 (0.0008) [2023-12-26 16:53:37,458][105692] Updated weights for policy 0, policy_version 209893 (0.0005) [2023-12-26 16:53:37,522][105692] Updated weights for policy 0, policy_version 209903 (0.0005) [2023-12-26 16:53:37,616][105620] Updated weights for policy 1, policy_version 210610 (0.0009) [2023-12-26 16:53:37,669][105620] Updated weights for policy 1, policy_version 210620 (0.0008) [2023-12-26 16:53:37,743][105620] Updated weights for policy 1, policy_version 210630 (0.0008) [2023-12-26 16:53:38,152][105692] Updated weights for policy 0, policy_version 209913 (0.0006) [2023-12-26 16:53:38,211][105692] Updated weights for policy 0, policy_version 209923 (0.0010) [2023-12-26 16:53:38,271][105692] Updated weights for policy 0, policy_version 209933 (0.0011) [2023-12-26 16:53:38,547][105620] Updated weights for policy 1, policy_version 210640 (0.0008) [2023-12-26 16:53:38,608][105620] Updated weights for policy 1, policy_version 210650 (0.0010) [2023-12-26 16:53:38,667][105620] Updated weights for policy 1, policy_version 210660 (0.0010) [2023-12-26 16:53:38,948][105692] Updated weights for policy 0, policy_version 209943 (0.0010) [2023-12-26 16:53:39,005][105692] Updated weights for policy 0, policy_version 209953 (0.0009) [2023-12-26 16:53:39,064][105692] Updated weights for policy 0, policy_version 209963 (0.0010) [2023-12-26 16:53:39,396][105620] Updated weights for policy 1, policy_version 210670 (0.0009) [2023-12-26 16:53:39,466][105620] Updated weights for policy 1, policy_version 210680 (0.0009) [2023-12-26 16:53:39,531][105620] Updated weights for policy 1, policy_version 210690 (0.0010) [2023-12-26 16:53:39,904][105692] Updated weights for policy 0, policy_version 209973 (0.0007) [2023-12-26 16:53:39,967][105692] Updated weights for policy 0, policy_version 209983 (0.0007) [2023-12-26 16:53:40,020][105692] Updated weights for policy 0, policy_version 209993 (0.0008) [2023-12-26 16:53:40,289][105620] Updated weights for policy 1, policy_version 210700 (0.0011) [2023-12-26 16:53:40,352][105620] Updated weights for policy 1, policy_version 210710 (0.0011) [2023-12-26 16:53:40,413][105620] Updated weights for policy 1, policy_version 210720 (0.0009) [2023-12-26 16:53:40,756][105692] Updated weights for policy 0, policy_version 210003 (0.0007) [2023-12-26 16:53:40,815][105692] Updated weights for policy 0, policy_version 210013 (0.0005) [2023-12-26 16:53:40,871][105692] Updated weights for policy 0, policy_version 210023 (0.0005) [2023-12-26 16:53:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 107732992. Throughput: 0: 9832.0, 1: 9643.4. Samples: 107741092. Policy #0 lag: (min: 11.0, avg: 21.2, max: 43.0) [2023-12-26 16:53:41,063][104569] Avg episode reward: [(0, '9257.198'), (1, '9357.242')] [2023-12-26 16:53:41,172][105620] Updated weights for policy 1, policy_version 210730 (0.0011) [2023-12-26 16:53:41,217][105620] Updated weights for policy 1, policy_version 210740 (0.0010) [2023-12-26 16:53:41,276][105620] Updated weights for policy 1, policy_version 210750 (0.0009) [2023-12-26 16:53:41,334][105620] Updated weights for policy 1, policy_version 210760 (0.0011) [2023-12-26 16:53:41,551][105692] Updated weights for policy 0, policy_version 210033 (0.0005) [2023-12-26 16:53:41,611][105692] Updated weights for policy 0, policy_version 210043 (0.0010) [2023-12-26 16:53:41,680][105692] Updated weights for policy 0, policy_version 210053 (0.0009) [2023-12-26 16:53:41,747][105692] Updated weights for policy 0, policy_version 210063 (0.0008) [2023-12-26 16:53:42,115][105620] Updated weights for policy 1, policy_version 210770 (0.0010) [2023-12-26 16:53:42,174][105620] Updated weights for policy 1, policy_version 210780 (0.0010) [2023-12-26 16:53:42,231][105620] Updated weights for policy 1, policy_version 210790 (0.0010) [2023-12-26 16:53:42,508][105692] Updated weights for policy 0, policy_version 210073 (0.0006) [2023-12-26 16:53:42,571][105692] Updated weights for policy 0, policy_version 210083 (0.0005) [2023-12-26 16:53:42,631][105692] Updated weights for policy 0, policy_version 210093 (0.0005) [2023-12-26 16:53:42,989][105620] Updated weights for policy 1, policy_version 210800 (0.0009) [2023-12-26 16:53:43,045][105620] Updated weights for policy 1, policy_version 210810 (0.0007) [2023-12-26 16:53:43,101][105620] Updated weights for policy 1, policy_version 210820 (0.0005) [2023-12-26 16:53:43,229][105692] Updated weights for policy 0, policy_version 210103 (0.0008) [2023-12-26 16:53:43,277][105692] Updated weights for policy 0, policy_version 210113 (0.0009) [2023-12-26 16:53:43,335][105692] Updated weights for policy 0, policy_version 210123 (0.0010) [2023-12-26 16:53:43,821][105620] Updated weights for policy 1, policy_version 210830 (0.0007) [2023-12-26 16:53:43,877][105620] Updated weights for policy 1, policy_version 210840 (0.0009) [2023-12-26 16:53:43,924][105620] Updated weights for policy 1, policy_version 210850 (0.0005) [2023-12-26 16:53:43,990][105692] Updated weights for policy 0, policy_version 210133 (0.0009) [2023-12-26 16:53:44,044][105692] Updated weights for policy 0, policy_version 210143 (0.0010) [2023-12-26 16:53:44,106][105692] Updated weights for policy 0, policy_version 210153 (0.0007) [2023-12-26 16:53:44,579][105620] Updated weights for policy 1, policy_version 210860 (0.0005) [2023-12-26 16:53:44,638][105620] Updated weights for policy 1, policy_version 210870 (0.0005) [2023-12-26 16:53:44,695][105620] Updated weights for policy 1, policy_version 210880 (0.0005) [2023-12-26 16:53:44,819][105692] Updated weights for policy 0, policy_version 210163 (0.0010) [2023-12-26 16:53:44,885][105692] Updated weights for policy 0, policy_version 210173 (0.0009) [2023-12-26 16:53:44,947][105692] Updated weights for policy 0, policy_version 210183 (0.0009) [2023-12-26 16:53:45,441][105620] Updated weights for policy 1, policy_version 210890 (0.0009) [2023-12-26 16:53:45,501][105620] Updated weights for policy 1, policy_version 210900 (0.0010) [2023-12-26 16:53:45,555][105620] Updated weights for policy 1, policy_version 210910 (0.0010) [2023-12-26 16:53:45,609][105620] Updated weights for policy 1, policy_version 210920 (0.0010) [2023-12-26 16:53:45,641][105692] Updated weights for policy 0, policy_version 210193 (0.0008) [2023-12-26 16:53:45,694][105692] Updated weights for policy 0, policy_version 210203 (0.0005) [2023-12-26 16:53:45,751][105692] Updated weights for policy 0, policy_version 210213 (0.0006) [2023-12-26 16:53:45,804][105692] Updated weights for policy 0, policy_version 210223 (0.0010) [2023-12-26 16:53:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 107831296. Throughput: 0: 9863.7, 1: 9648.4. Samples: 107798908. Policy #0 lag: (min: 11.0, avg: 21.2, max: 43.0) [2023-12-26 16:53:46,062][104569] Avg episode reward: [(0, '9257.397'), (1, '9359.227')] [2023-12-26 16:53:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000210224_53829632.pth... [2023-12-26 16:53:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000210920_54001664.pth... [2023-12-26 16:53:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000209040_53526528.pth [2023-12-26 16:53:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000209800_53714944.pth [2023-12-26 16:53:46,331][105620] Updated weights for policy 1, policy_version 210930 (0.0009) [2023-12-26 16:53:46,378][105620] Updated weights for policy 1, policy_version 210940 (0.0008) [2023-12-26 16:53:46,431][105620] Updated weights for policy 1, policy_version 210950 (0.0008) [2023-12-26 16:53:46,482][105692] Updated weights for policy 0, policy_version 210233 (0.0009) [2023-12-26 16:53:46,536][105692] Updated weights for policy 0, policy_version 210243 (0.0009) [2023-12-26 16:53:46,583][105692] Updated weights for policy 0, policy_version 210253 (0.0008) [2023-12-26 16:53:47,148][105692] Updated weights for policy 0, policy_version 210263 (0.0005) [2023-12-26 16:53:47,202][105692] Updated weights for policy 0, policy_version 210273 (0.0005) [2023-12-26 16:53:47,249][105692] Updated weights for policy 0, policy_version 210283 (0.0008) [2023-12-26 16:53:47,314][105620] Updated weights for policy 1, policy_version 210960 (0.0008) [2023-12-26 16:53:47,360][105620] Updated weights for policy 1, policy_version 210970 (0.0009) [2023-12-26 16:53:47,407][105620] Updated weights for policy 1, policy_version 210980 (0.0008) [2023-12-26 16:53:47,883][105692] Updated weights for policy 0, policy_version 210293 (0.0007) [2023-12-26 16:53:47,942][105692] Updated weights for policy 0, policy_version 210303 (0.0006) [2023-12-26 16:53:48,004][105692] Updated weights for policy 0, policy_version 210313 (0.0009) [2023-12-26 16:53:48,232][105620] Updated weights for policy 1, policy_version 210990 (0.0009) [2023-12-26 16:53:48,284][105620] Updated weights for policy 1, policy_version 211000 (0.0009) [2023-12-26 16:53:48,347][105620] Updated weights for policy 1, policy_version 211010 (0.0007) [2023-12-26 16:53:48,725][105692] Updated weights for policy 0, policy_version 210323 (0.0009) [2023-12-26 16:53:48,782][105692] Updated weights for policy 0, policy_version 210333 (0.0009) [2023-12-26 16:53:48,837][105692] Updated weights for policy 0, policy_version 210343 (0.0009) [2023-12-26 16:53:49,075][105620] Updated weights for policy 1, policy_version 211020 (0.0009) [2023-12-26 16:53:49,133][105620] Updated weights for policy 1, policy_version 211030 (0.0009) [2023-12-26 16:53:49,200][105620] Updated weights for policy 1, policy_version 211040 (0.0009) [2023-12-26 16:53:49,595][105692] Updated weights for policy 0, policy_version 210353 (0.0009) [2023-12-26 16:53:49,657][105692] Updated weights for policy 0, policy_version 210363 (0.0009) [2023-12-26 16:53:49,714][105692] Updated weights for policy 0, policy_version 210373 (0.0008) [2023-12-26 16:53:49,765][105692] Updated weights for policy 0, policy_version 210383 (0.0009) [2023-12-26 16:53:50,013][105620] Updated weights for policy 1, policy_version 211050 (0.0009) [2023-12-26 16:53:50,074][105620] Updated weights for policy 1, policy_version 211060 (0.0009) [2023-12-26 16:53:50,133][105620] Updated weights for policy 1, policy_version 211070 (0.0009) [2023-12-26 16:53:50,194][105620] Updated weights for policy 1, policy_version 211080 (0.0009) [2023-12-26 16:53:50,583][105692] Updated weights for policy 0, policy_version 210393 (0.0009) [2023-12-26 16:53:50,647][105692] Updated weights for policy 0, policy_version 210403 (0.0006) [2023-12-26 16:53:50,714][105692] Updated weights for policy 0, policy_version 210413 (0.0007) [2023-12-26 16:53:50,845][105620] Updated weights for policy 1, policy_version 211090 (0.0005) [2023-12-26 16:53:50,909][105620] Updated weights for policy 1, policy_version 211100 (0.0005) [2023-12-26 16:53:50,979][105620] Updated weights for policy 1, policy_version 211110 (0.0005) [2023-12-26 16:53:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 107929600. Throughput: 0: 9948.0, 1: 9612.2. Samples: 107916068. Policy #0 lag: (min: 11.0, avg: 21.2, max: 43.0) [2023-12-26 16:53:51,063][104569] Avg episode reward: [(0, '9167.190'), (1, '9359.421')] [2023-12-26 16:53:51,319][105692] Updated weights for policy 0, policy_version 210423 (0.0008) [2023-12-26 16:53:51,388][105692] Updated weights for policy 0, policy_version 210433 (0.0007) [2023-12-26 16:53:51,444][105692] Updated weights for policy 0, policy_version 210443 (0.0009) [2023-12-26 16:53:51,596][105620] Updated weights for policy 1, policy_version 211120 (0.0008) [2023-12-26 16:53:51,661][105620] Updated weights for policy 1, policy_version 211130 (0.0008) [2023-12-26 16:53:51,723][105620] Updated weights for policy 1, policy_version 211140 (0.0009) [2023-12-26 16:53:52,222][105692] Updated weights for policy 0, policy_version 210453 (0.0009) [2023-12-26 16:53:52,278][105692] Updated weights for policy 0, policy_version 210463 (0.0009) [2023-12-26 16:53:52,345][105692] Updated weights for policy 0, policy_version 210473 (0.0007) [2023-12-26 16:53:52,517][105620] Updated weights for policy 1, policy_version 211150 (0.0010) [2023-12-26 16:53:52,569][105620] Updated weights for policy 1, policy_version 211160 (0.0010) [2023-12-26 16:53:52,618][105620] Updated weights for policy 1, policy_version 211170 (0.0009) [2023-12-26 16:53:52,951][105692] Updated weights for policy 0, policy_version 210483 (0.0009) [2023-12-26 16:53:53,001][105692] Updated weights for policy 0, policy_version 210493 (0.0009) [2023-12-26 16:53:53,053][105692] Updated weights for policy 0, policy_version 210503 (0.0009) [2023-12-26 16:53:53,512][105620] Updated weights for policy 1, policy_version 211180 (0.0009) [2023-12-26 16:53:53,574][105620] Updated weights for policy 1, policy_version 211190 (0.0009) [2023-12-26 16:53:53,632][105692] Updated weights for policy 0, policy_version 210513 (0.0008) [2023-12-26 16:53:53,636][105620] Updated weights for policy 1, policy_version 211200 (0.0009) [2023-12-26 16:53:53,685][105692] Updated weights for policy 0, policy_version 210523 (0.0007) [2023-12-26 16:53:53,731][105692] Updated weights for policy 0, policy_version 210533 (0.0009) [2023-12-26 16:53:53,786][105692] Updated weights for policy 0, policy_version 210543 (0.0009) [2023-12-26 16:53:54,379][105620] Updated weights for policy 1, policy_version 211210 (0.0008) [2023-12-26 16:53:54,441][105620] Updated weights for policy 1, policy_version 211220 (0.0009) [2023-12-26 16:53:54,505][105620] Updated weights for policy 1, policy_version 211230 (0.0008) [2023-12-26 16:53:54,554][105692] Updated weights for policy 0, policy_version 210553 (0.0008) [2023-12-26 16:53:54,563][105620] Updated weights for policy 1, policy_version 211240 (0.0009) [2023-12-26 16:53:54,612][105692] Updated weights for policy 0, policy_version 210563 (0.0007) [2023-12-26 16:53:54,667][105692] Updated weights for policy 0, policy_version 210573 (0.0005) [2023-12-26 16:53:55,319][105620] Updated weights for policy 1, policy_version 211250 (0.0009) [2023-12-26 16:53:55,363][105692] Updated weights for policy 0, policy_version 210583 (0.0008) [2023-12-26 16:53:55,368][105620] Updated weights for policy 1, policy_version 211260 (0.0009) [2023-12-26 16:53:55,413][105620] Updated weights for policy 1, policy_version 211270 (0.0007) [2023-12-26 16:53:55,415][105692] Updated weights for policy 0, policy_version 210593 (0.0007) [2023-12-26 16:53:55,468][105692] Updated weights for policy 0, policy_version 210603 (0.0008) [2023-12-26 16:53:56,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 108019712. Throughput: 0: 10037.9, 1: 9498.1. Samples: 108031604. Policy #0 lag: (min: 11.0, avg: 21.2, max: 43.0) [2023-12-26 16:53:56,063][104569] Avg episode reward: [(0, '9075.105'), (1, '9359.392')] [2023-12-26 16:53:56,134][105692] Updated weights for policy 0, policy_version 210613 (0.0009) [2023-12-26 16:53:56,194][105692] Updated weights for policy 0, policy_version 210623 (0.0008) [2023-12-26 16:53:56,208][105620] Updated weights for policy 1, policy_version 211280 (0.0006) [2023-12-26 16:53:56,253][105620] Updated weights for policy 1, policy_version 211290 (0.0006) [2023-12-26 16:53:56,254][105692] Updated weights for policy 0, policy_version 210633 (0.0008) [2023-12-26 16:53:56,305][105620] Updated weights for policy 1, policy_version 211300 (0.0006) [2023-12-26 16:53:57,011][105692] Updated weights for policy 0, policy_version 210643 (0.0009) [2023-12-26 16:53:57,064][105692] Updated weights for policy 0, policy_version 210653 (0.0008) [2023-12-26 16:53:57,070][105620] Updated weights for policy 1, policy_version 211310 (0.0007) [2023-12-26 16:53:57,108][105692] Updated weights for policy 0, policy_version 210663 (0.0005) [2023-12-26 16:53:57,125][105620] Updated weights for policy 1, policy_version 211320 (0.0007) [2023-12-26 16:53:57,191][105620] Updated weights for policy 1, policy_version 211330 (0.0007) [2023-12-26 16:53:57,861][105620] Updated weights for policy 1, policy_version 211340 (0.0008) [2023-12-26 16:53:57,878][105692] Updated weights for policy 0, policy_version 210673 (0.0008) [2023-12-26 16:53:57,908][105620] Updated weights for policy 1, policy_version 211350 (0.0007) [2023-12-26 16:53:57,938][105692] Updated weights for policy 0, policy_version 210683 (0.0008) [2023-12-26 16:53:57,957][105620] Updated weights for policy 1, policy_version 211360 (0.0006) [2023-12-26 16:53:57,998][105692] Updated weights for policy 0, policy_version 210693 (0.0007) [2023-12-26 16:53:58,059][105692] Updated weights for policy 0, policy_version 210703 (0.0007) [2023-12-26 16:53:58,704][105620] Updated weights for policy 1, policy_version 211370 (0.0006) [2023-12-26 16:53:58,772][105620] Updated weights for policy 1, policy_version 211380 (0.0010) [2023-12-26 16:53:58,839][105620] Updated weights for policy 1, policy_version 211390 (0.0009) [2023-12-26 16:53:58,895][105692] Updated weights for policy 0, policy_version 210713 (0.0007) [2023-12-26 16:53:58,901][105620] Updated weights for policy 1, policy_version 211400 (0.0008) [2023-12-26 16:53:58,956][105692] Updated weights for policy 0, policy_version 210723 (0.0009) [2023-12-26 16:53:59,016][105692] Updated weights for policy 0, policy_version 210733 (0.0006) [2023-12-26 16:53:59,601][105620] Updated weights for policy 1, policy_version 211410 (0.0009) [2023-12-26 16:53:59,655][105620] Updated weights for policy 1, policy_version 211420 (0.0009) [2023-12-26 16:53:59,718][105620] Updated weights for policy 1, policy_version 211430 (0.0009) [2023-12-26 16:53:59,787][105692] Updated weights for policy 0, policy_version 210743 (0.0008) [2023-12-26 16:53:59,853][105692] Updated weights for policy 0, policy_version 210753 (0.0009) [2023-12-26 16:53:59,918][105692] Updated weights for policy 0, policy_version 210763 (0.0009) [2023-12-26 16:54:00,385][105620] Updated weights for policy 1, policy_version 211440 (0.0010) [2023-12-26 16:54:00,451][105620] Updated weights for policy 1, policy_version 211450 (0.0010) [2023-12-26 16:54:00,512][105620] Updated weights for policy 1, policy_version 211460 (0.0010) [2023-12-26 16:54:00,755][105692] Updated weights for policy 0, policy_version 210773 (0.0008) [2023-12-26 16:54:00,807][105692] Updated weights for policy 0, policy_version 210783 (0.0008) [2023-12-26 16:54:00,850][105692] Updated weights for policy 0, policy_version 210793 (0.0007) [2023-12-26 16:54:01,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.4, 300 sec: 19577.5). Total num frames: 108118016. Throughput: 0: 10075.5, 1: 9525.1. Samples: 108088744. Policy #0 lag: (min: 11.0, avg: 21.2, max: 43.0) [2023-12-26 16:54:01,062][104569] Avg episode reward: [(0, '8755.662'), (1, '9359.356')] [2023-12-26 16:54:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000210800_53977088.pth... [2023-12-26 16:54:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000211464_54140928.pth... [2023-12-26 16:54:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000209648_53682176.pth [2023-12-26 16:54:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000210376_53862400.pth [2023-12-26 16:54:01,159][105620] Updated weights for policy 1, policy_version 211470 (0.0008) [2023-12-26 16:54:01,208][105620] Updated weights for policy 1, policy_version 211480 (0.0007) [2023-12-26 16:54:01,265][105620] Updated weights for policy 1, policy_version 211490 (0.0008) [2023-12-26 16:54:01,646][105692] Updated weights for policy 0, policy_version 210803 (0.0008) [2023-12-26 16:54:01,706][105692] Updated weights for policy 0, policy_version 210813 (0.0009) [2023-12-26 16:54:01,768][105692] Updated weights for policy 0, policy_version 210823 (0.0008) [2023-12-26 16:54:02,032][105620] Updated weights for policy 1, policy_version 211500 (0.0010) [2023-12-26 16:54:02,087][105620] Updated weights for policy 1, policy_version 211510 (0.0010) [2023-12-26 16:54:02,141][105620] Updated weights for policy 1, policy_version 211520 (0.0008) [2023-12-26 16:54:02,530][105692] Updated weights for policy 0, policy_version 210833 (0.0008) [2023-12-26 16:54:02,587][105692] Updated weights for policy 0, policy_version 210843 (0.0010) [2023-12-26 16:54:02,640][105692] Updated weights for policy 0, policy_version 210853 (0.0010) [2023-12-26 16:54:02,700][105692] Updated weights for policy 0, policy_version 210864 (0.0011) [2023-12-26 16:54:02,799][105620] Updated weights for policy 1, policy_version 211530 (0.0006) [2023-12-26 16:54:02,851][105620] Updated weights for policy 1, policy_version 211540 (0.0010) [2023-12-26 16:54:02,906][105620] Updated weights for policy 1, policy_version 211550 (0.0010) [2023-12-26 16:54:02,965][105620] Updated weights for policy 1, policy_version 211560 (0.0010) [2023-12-26 16:54:03,422][105692] Updated weights for policy 0, policy_version 210874 (0.0010) [2023-12-26 16:54:03,481][105692] Updated weights for policy 0, policy_version 210884 (0.0009) [2023-12-26 16:54:03,542][105692] Updated weights for policy 0, policy_version 210894 (0.0010) [2023-12-26 16:54:03,690][105620] Updated weights for policy 1, policy_version 211570 (0.0007) [2023-12-26 16:54:03,745][105620] Updated weights for policy 1, policy_version 211580 (0.0008) [2023-12-26 16:54:03,795][105620] Updated weights for policy 1, policy_version 211590 (0.0008) [2023-12-26 16:54:04,285][105692] Updated weights for policy 0, policy_version 210904 (0.0010) [2023-12-26 16:54:04,340][105692] Updated weights for policy 0, policy_version 210914 (0.0010) [2023-12-26 16:54:04,396][105692] Updated weights for policy 0, policy_version 210924 (0.0011) [2023-12-26 16:54:04,593][105620] Updated weights for policy 1, policy_version 211600 (0.0008) [2023-12-26 16:54:04,660][105620] Updated weights for policy 1, policy_version 211610 (0.0008) [2023-12-26 16:54:04,719][105620] Updated weights for policy 1, policy_version 211620 (0.0008) [2023-12-26 16:54:05,153][105692] Updated weights for policy 0, policy_version 210934 (0.0011) [2023-12-26 16:54:05,211][105692] Updated weights for policy 0, policy_version 210944 (0.0010) [2023-12-26 16:54:05,263][105692] Updated weights for policy 0, policy_version 210954 (0.0010) [2023-12-26 16:54:05,480][105620] Updated weights for policy 1, policy_version 211630 (0.0008) [2023-12-26 16:54:05,546][105620] Updated weights for policy 1, policy_version 211640 (0.0008) [2023-12-26 16:54:05,591][105620] Updated weights for policy 1, policy_version 211650 (0.0008) [2023-12-26 16:54:06,014][105692] Updated weights for policy 0, policy_version 210964 (0.0010) [2023-12-26 16:54:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 108208128. Throughput: 0: 9942.2, 1: 9485.5. Samples: 108201880. Policy #0 lag: (min: 11.0, avg: 21.2, max: 43.0) [2023-12-26 16:54:06,062][104569] Avg episode reward: [(0, '8440.959'), (1, '9359.259')] [2023-12-26 16:54:06,072][105692] Updated weights for policy 0, policy_version 210974 (0.0010) [2023-12-26 16:54:06,133][105692] Updated weights for policy 0, policy_version 210984 (0.0010) [2023-12-26 16:54:06,339][105620] Updated weights for policy 1, policy_version 211660 (0.0009) [2023-12-26 16:54:06,408][105620] Updated weights for policy 1, policy_version 211670 (0.0009) [2023-12-26 16:54:06,470][105620] Updated weights for policy 1, policy_version 211680 (0.0009) [2023-12-26 16:54:06,879][105692] Updated weights for policy 0, policy_version 210994 (0.0010) [2023-12-26 16:54:06,939][105692] Updated weights for policy 0, policy_version 211004 (0.0009) [2023-12-26 16:54:06,999][105692] Updated weights for policy 0, policy_version 211014 (0.0009) [2023-12-26 16:54:07,060][105692] Updated weights for policy 0, policy_version 211024 (0.0008) [2023-12-26 16:54:07,205][105620] Updated weights for policy 1, policy_version 211690 (0.0009) [2023-12-26 16:54:07,259][105620] Updated weights for policy 1, policy_version 211700 (0.0009) [2023-12-26 16:54:07,315][105620] Updated weights for policy 1, policy_version 211710 (0.0009) [2023-12-26 16:54:07,362][105620] Updated weights for policy 1, policy_version 211720 (0.0008) [2023-12-26 16:54:07,817][105692] Updated weights for policy 0, policy_version 211034 (0.0008) [2023-12-26 16:54:07,880][105692] Updated weights for policy 0, policy_version 211044 (0.0008) [2023-12-26 16:54:07,941][105692] Updated weights for policy 0, policy_version 211054 (0.0008) [2023-12-26 16:54:08,124][105620] Updated weights for policy 1, policy_version 211730 (0.0011) [2023-12-26 16:54:08,186][105620] Updated weights for policy 1, policy_version 211740 (0.0007) [2023-12-26 16:54:08,257][105620] Updated weights for policy 1, policy_version 211750 (0.0005) [2023-12-26 16:54:08,676][105692] Updated weights for policy 0, policy_version 211064 (0.0011) [2023-12-26 16:54:08,736][105692] Updated weights for policy 0, policy_version 211074 (0.0011) [2023-12-26 16:54:08,805][105692] Updated weights for policy 0, policy_version 211084 (0.0005) [2023-12-26 16:54:08,891][105620] Updated weights for policy 1, policy_version 211760 (0.0010) [2023-12-26 16:54:08,947][105620] Updated weights for policy 1, policy_version 211770 (0.0010) [2023-12-26 16:54:09,005][105620] Updated weights for policy 1, policy_version 211780 (0.0010) [2023-12-26 16:54:09,390][105692] Updated weights for policy 0, policy_version 211094 (0.0007) [2023-12-26 16:54:09,452][105692] Updated weights for policy 0, policy_version 211104 (0.0010) [2023-12-26 16:54:09,511][105692] Updated weights for policy 0, policy_version 211114 (0.0010) [2023-12-26 16:54:09,648][105620] Updated weights for policy 1, policy_version 211790 (0.0006) [2023-12-26 16:54:09,715][105620] Updated weights for policy 1, policy_version 211800 (0.0006) [2023-12-26 16:54:09,787][105620] Updated weights for policy 1, policy_version 211810 (0.0006) [2023-12-26 16:54:10,323][105692] Updated weights for policy 0, policy_version 211124 (0.0010) [2023-12-26 16:54:10,376][105692] Updated weights for policy 0, policy_version 211134 (0.0011) [2023-12-26 16:54:10,436][105692] Updated weights for policy 0, policy_version 211144 (0.0010) [2023-12-26 16:54:10,450][105620] Updated weights for policy 1, policy_version 211820 (0.0008) [2023-12-26 16:54:10,506][105620] Updated weights for policy 1, policy_version 211830 (0.0005) [2023-12-26 16:54:10,563][105620] Updated weights for policy 1, policy_version 211840 (0.0005) [2023-12-26 16:54:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 108306432. Throughput: 0: 9831.4, 1: 9556.2. Samples: 108318268. Policy #0 lag: (min: 11.0, avg: 21.2, max: 43.0) [2023-12-26 16:54:11,062][104569] Avg episode reward: [(0, '8677.583'), (1, '9359.197')] [2023-12-26 16:54:11,107][105692] Updated weights for policy 0, policy_version 211154 (0.0010) [2023-12-26 16:54:11,154][105620] Updated weights for policy 1, policy_version 211850 (0.0006) [2023-12-26 16:54:11,174][105692] Updated weights for policy 0, policy_version 211164 (0.0008) [2023-12-26 16:54:11,219][105620] Updated weights for policy 1, policy_version 211860 (0.0011) [2023-12-26 16:54:11,234][105692] Updated weights for policy 0, policy_version 211174 (0.0008) [2023-12-26 16:54:11,285][105620] Updated weights for policy 1, policy_version 211870 (0.0009) [2023-12-26 16:54:11,302][105692] Updated weights for policy 0, policy_version 211184 (0.0009) [2023-12-26 16:54:11,345][105620] Updated weights for policy 1, policy_version 211880 (0.0008) [2023-12-26 16:54:12,017][105620] Updated weights for policy 1, policy_version 211890 (0.0006) [2023-12-26 16:54:12,077][105620] Updated weights for policy 1, policy_version 211900 (0.0008) [2023-12-26 16:54:12,101][105692] Updated weights for policy 0, policy_version 211194 (0.0008) [2023-12-26 16:54:12,137][105620] Updated weights for policy 1, policy_version 211910 (0.0007) [2023-12-26 16:54:12,163][105692] Updated weights for policy 0, policy_version 211204 (0.0008) [2023-12-26 16:54:12,226][105692] Updated weights for policy 0, policy_version 211214 (0.0008) [2023-12-26 16:54:12,883][105620] Updated weights for policy 1, policy_version 211920 (0.0005) [2023-12-26 16:54:12,937][105620] Updated weights for policy 1, policy_version 211930 (0.0005) [2023-12-26 16:54:12,944][105692] Updated weights for policy 0, policy_version 211224 (0.0009) [2023-12-26 16:54:12,986][105620] Updated weights for policy 1, policy_version 211940 (0.0008) [2023-12-26 16:54:13,002][105692] Updated weights for policy 0, policy_version 211234 (0.0007) [2023-12-26 16:54:13,062][105692] Updated weights for policy 0, policy_version 211244 (0.0008) [2023-12-26 16:54:13,609][105620] Updated weights for policy 1, policy_version 211950 (0.0009) [2023-12-26 16:54:13,669][105620] Updated weights for policy 1, policy_version 211960 (0.0008) [2023-12-26 16:54:13,729][105620] Updated weights for policy 1, policy_version 211970 (0.0009) [2023-12-26 16:54:13,804][105692] Updated weights for policy 0, policy_version 211254 (0.0008) [2023-12-26 16:54:13,855][105692] Updated weights for policy 0, policy_version 211264 (0.0009) [2023-12-26 16:54:13,910][105692] Updated weights for policy 0, policy_version 211274 (0.0009) [2023-12-26 16:54:14,537][105620] Updated weights for policy 1, policy_version 211980 (0.0008) [2023-12-26 16:54:14,556][105692] Updated weights for policy 0, policy_version 211284 (0.0008) [2023-12-26 16:54:14,594][105620] Updated weights for policy 1, policy_version 211990 (0.0008) [2023-12-26 16:54:14,618][105692] Updated weights for policy 0, policy_version 211294 (0.0007) [2023-12-26 16:54:14,649][105620] Updated weights for policy 1, policy_version 212000 (0.0006) [2023-12-26 16:54:14,680][105692] Updated weights for policy 0, policy_version 211304 (0.0007) [2023-12-26 16:54:15,387][105620] Updated weights for policy 1, policy_version 212010 (0.0007) [2023-12-26 16:54:15,414][105692] Updated weights for policy 0, policy_version 211314 (0.0009) [2023-12-26 16:54:15,451][105620] Updated weights for policy 1, policy_version 212020 (0.0006) [2023-12-26 16:54:15,476][105692] Updated weights for policy 0, policy_version 211324 (0.0009) [2023-12-26 16:54:15,516][105620] Updated weights for policy 1, policy_version 212030 (0.0006) [2023-12-26 16:54:15,529][105692] Updated weights for policy 0, policy_version 211334 (0.0008) [2023-12-26 16:54:15,580][105620] Updated weights for policy 1, policy_version 212040 (0.0006) [2023-12-26 16:54:15,580][105692] Updated weights for policy 0, policy_version 211344 (0.0009) [2023-12-26 16:54:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 108404736. Throughput: 0: 9769.1, 1: 9508.9. Samples: 108376708. Policy #0 lag: (min: 11.0, avg: 21.2, max: 43.0) [2023-12-26 16:54:16,063][104569] Avg episode reward: [(0, '2850.491'), (1, '9359.037')] [2023-12-26 16:54:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000211344_54116352.pth... [2023-12-26 16:54:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000212040_54288384.pth... [2023-12-26 16:54:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000210920_54001664.pth [2023-12-26 16:54:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000210224_53829632.pth [2023-12-26 16:54:16,152][105620] Updated weights for policy 1, policy_version 212050 (0.0005) [2023-12-26 16:54:16,202][105620] Updated weights for policy 1, policy_version 212060 (0.0007) [2023-12-26 16:54:16,256][105620] Updated weights for policy 1, policy_version 212070 (0.0007) [2023-12-26 16:54:16,286][105692] Updated weights for policy 0, policy_version 211354 (0.0011) [2023-12-26 16:54:16,344][105692] Updated weights for policy 0, policy_version 211364 (0.0010) [2023-12-26 16:54:16,395][105692] Updated weights for policy 0, policy_version 211374 (0.0010) [2023-12-26 16:54:16,852][105620] Updated weights for policy 1, policy_version 212080 (0.0005) [2023-12-26 16:54:16,903][105620] Updated weights for policy 1, policy_version 212090 (0.0008) [2023-12-26 16:54:16,947][105620] Updated weights for policy 1, policy_version 212100 (0.0006) [2023-12-26 16:54:17,112][105692] Updated weights for policy 0, policy_version 211384 (0.0007) [2023-12-26 16:54:17,169][105692] Updated weights for policy 0, policy_version 211394 (0.0007) [2023-12-26 16:54:17,224][105692] Updated weights for policy 0, policy_version 211404 (0.0010) [2023-12-26 16:54:17,653][105620] Updated weights for policy 1, policy_version 212110 (0.0008) [2023-12-26 16:54:17,720][105620] Updated weights for policy 1, policy_version 212120 (0.0010) [2023-12-26 16:54:17,780][105620] Updated weights for policy 1, policy_version 212130 (0.0009) [2023-12-26 16:54:17,797][105692] Updated weights for policy 0, policy_version 211414 (0.0007) [2023-12-26 16:54:17,852][105692] Updated weights for policy 0, policy_version 211424 (0.0005) [2023-12-26 16:54:17,915][105692] Updated weights for policy 0, policy_version 211434 (0.0005) [2023-12-26 16:54:18,535][105692] Updated weights for policy 0, policy_version 211444 (0.0008) [2023-12-26 16:54:18,577][105620] Updated weights for policy 1, policy_version 212140 (0.0008) [2023-12-26 16:54:18,591][105692] Updated weights for policy 0, policy_version 211454 (0.0010) [2023-12-26 16:54:18,633][105620] Updated weights for policy 1, policy_version 212150 (0.0005) [2023-12-26 16:54:18,643][105692] Updated weights for policy 0, policy_version 211464 (0.0010) [2023-12-26 16:54:18,689][105620] Updated weights for policy 1, policy_version 212160 (0.0006) [2023-12-26 16:54:19,313][105692] Updated weights for policy 0, policy_version 211474 (0.0010) [2023-12-26 16:54:19,378][105692] Updated weights for policy 0, policy_version 211484 (0.0008) [2023-12-26 16:54:19,445][105692] Updated weights for policy 0, policy_version 211494 (0.0008) [2023-12-26 16:54:19,513][105692] Updated weights for policy 0, policy_version 211504 (0.0008) [2023-12-26 16:54:19,516][105620] Updated weights for policy 1, policy_version 212170 (0.0008) [2023-12-26 16:54:19,571][105620] Updated weights for policy 1, policy_version 212180 (0.0008) [2023-12-26 16:54:19,624][105620] Updated weights for policy 1, policy_version 212190 (0.0008) [2023-12-26 16:54:19,680][105620] Updated weights for policy 1, policy_version 212200 (0.0008) [2023-12-26 16:54:20,264][105692] Updated weights for policy 0, policy_version 211514 (0.0011) [2023-12-26 16:54:20,321][105692] Updated weights for policy 0, policy_version 211524 (0.0010) [2023-12-26 16:54:20,377][105692] Updated weights for policy 0, policy_version 211534 (0.0010) [2023-12-26 16:54:20,497][105620] Updated weights for policy 1, policy_version 212210 (0.0007) [2023-12-26 16:54:20,546][105620] Updated weights for policy 1, policy_version 212220 (0.0008) [2023-12-26 16:54:20,618][105620] Updated weights for policy 1, policy_version 212230 (0.0009) [2023-12-26 16:54:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 108503040. Throughput: 0: 9771.9, 1: 9576.0. Samples: 108495508. Policy #0 lag: (min: 11.0, avg: 21.2, max: 43.0) [2023-12-26 16:54:21,062][104569] Avg episode reward: [(0, '5440.940'), (1, '9358.944')] [2023-12-26 16:54:21,165][105692] Updated weights for policy 0, policy_version 211544 (0.0011) [2023-12-26 16:54:21,224][105692] Updated weights for policy 0, policy_version 211554 (0.0010) [2023-12-26 16:54:21,288][105692] Updated weights for policy 0, policy_version 211564 (0.0009) [2023-12-26 16:54:21,411][105620] Updated weights for policy 1, policy_version 212240 (0.0009) [2023-12-26 16:54:21,468][105620] Updated weights for policy 1, policy_version 212250 (0.0009) [2023-12-26 16:54:21,522][105620] Updated weights for policy 1, policy_version 212260 (0.0012) [2023-12-26 16:54:21,965][105692] Updated weights for policy 0, policy_version 211574 (0.0006) [2023-12-26 16:54:22,033][105692] Updated weights for policy 0, policy_version 211584 (0.0007) [2023-12-26 16:54:22,096][105692] Updated weights for policy 0, policy_version 211594 (0.0011) [2023-12-26 16:54:22,261][105620] Updated weights for policy 1, policy_version 212270 (0.0009) [2023-12-26 16:54:22,329][105620] Updated weights for policy 1, policy_version 212280 (0.0008) [2023-12-26 16:54:22,397][105620] Updated weights for policy 1, policy_version 212290 (0.0007) [2023-12-26 16:54:22,768][105692] Updated weights for policy 0, policy_version 211604 (0.0010) [2023-12-26 16:54:22,822][105692] Updated weights for policy 0, policy_version 211614 (0.0009) [2023-12-26 16:54:22,887][105692] Updated weights for policy 0, policy_version 211624 (0.0011) [2023-12-26 16:54:23,084][105620] Updated weights for policy 1, policy_version 212300 (0.0007) [2023-12-26 16:54:23,135][105620] Updated weights for policy 1, policy_version 212310 (0.0008) [2023-12-26 16:54:23,180][105620] Updated weights for policy 1, policy_version 212320 (0.0007) [2023-12-26 16:54:23,655][105692] Updated weights for policy 0, policy_version 211634 (0.0009) [2023-12-26 16:54:23,704][105692] Updated weights for policy 0, policy_version 211644 (0.0010) [2023-12-26 16:54:23,749][105692] Updated weights for policy 0, policy_version 211654 (0.0006) [2023-12-26 16:54:23,807][105692] Updated weights for policy 0, policy_version 211664 (0.0005) [2023-12-26 16:54:23,966][105620] Updated weights for policy 1, policy_version 212330 (0.0008) [2023-12-26 16:54:24,010][105620] Updated weights for policy 1, policy_version 212340 (0.0008) [2023-12-26 16:54:24,058][105620] Updated weights for policy 1, policy_version 212350 (0.0008) [2023-12-26 16:54:24,109][105620] Updated weights for policy 1, policy_version 212360 (0.0008) [2023-12-26 16:54:24,454][105692] Updated weights for policy 0, policy_version 211674 (0.0007) [2023-12-26 16:54:24,509][105692] Updated weights for policy 0, policy_version 211684 (0.0010) [2023-12-26 16:54:24,558][105692] Updated weights for policy 0, policy_version 211694 (0.0007) [2023-12-26 16:54:25,011][105620] Updated weights for policy 1, policy_version 212370 (0.0009) [2023-12-26 16:54:25,070][105620] Updated weights for policy 1, policy_version 212380 (0.0009) [2023-12-26 16:54:25,100][105692] Updated weights for policy 0, policy_version 211704 (0.0009) [2023-12-26 16:54:25,129][105620] Updated weights for policy 1, policy_version 212390 (0.0007) [2023-12-26 16:54:25,149][105692] Updated weights for policy 0, policy_version 211714 (0.0005) [2023-12-26 16:54:25,195][105692] Updated weights for policy 0, policy_version 211724 (0.0005) [2023-12-26 16:54:25,735][105620] Updated weights for policy 1, policy_version 212400 (0.0007) [2023-12-26 16:54:25,782][105620] Updated weights for policy 1, policy_version 212410 (0.0008) [2023-12-26 16:54:25,832][105692] Updated weights for policy 0, policy_version 211734 (0.0008) [2023-12-26 16:54:25,835][105620] Updated weights for policy 1, policy_version 212420 (0.0006) [2023-12-26 16:54:25,891][105692] Updated weights for policy 0, policy_version 211744 (0.0010) [2023-12-26 16:54:25,943][105692] Updated weights for policy 0, policy_version 211754 (0.0010) [2023-12-26 16:54:26,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 108609536. Throughput: 0: 9818.8, 1: 9563.2. Samples: 108613280. Policy #0 lag: (min: 11.0, avg: 21.2, max: 43.0) [2023-12-26 16:54:26,063][104569] Avg episode reward: [(0, '7070.804'), (1, '9358.791')] [2023-12-26 16:54:26,557][105692] Updated weights for policy 0, policy_version 211764 (0.0009) [2023-12-26 16:54:26,619][105692] Updated weights for policy 0, policy_version 211774 (0.0010) [2023-12-26 16:54:26,629][105620] Updated weights for policy 1, policy_version 212430 (0.0006) [2023-12-26 16:54:26,677][105692] Updated weights for policy 0, policy_version 211784 (0.0011) [2023-12-26 16:54:26,691][105620] Updated weights for policy 1, policy_version 212440 (0.0005) [2023-12-26 16:54:26,752][105620] Updated weights for policy 1, policy_version 212450 (0.0005) [2023-12-26 16:54:27,269][105620] Updated weights for policy 1, policy_version 212460 (0.0005) [2023-12-26 16:54:27,339][105620] Updated weights for policy 1, policy_version 212470 (0.0009) [2023-12-26 16:54:27,394][105620] Updated weights for policy 1, policy_version 212480 (0.0010) [2023-12-26 16:54:27,414][105692] Updated weights for policy 0, policy_version 211794 (0.0010) [2023-12-26 16:54:27,475][105692] Updated weights for policy 0, policy_version 211804 (0.0008) [2023-12-26 16:54:27,534][105692] Updated weights for policy 0, policy_version 211814 (0.0007) [2023-12-26 16:54:27,586][105692] Updated weights for policy 0, policy_version 211824 (0.0009) [2023-12-26 16:54:28,069][105620] Updated weights for policy 1, policy_version 212490 (0.0011) [2023-12-26 16:54:28,116][105620] Updated weights for policy 1, policy_version 212500 (0.0009) [2023-12-26 16:54:28,178][105620] Updated weights for policy 1, policy_version 212510 (0.0005) [2023-12-26 16:54:28,238][105620] Updated weights for policy 1, policy_version 212520 (0.0008) [2023-12-26 16:54:28,244][105692] Updated weights for policy 0, policy_version 211834 (0.0011) [2023-12-26 16:54:28,305][105692] Updated weights for policy 0, policy_version 211844 (0.0010) [2023-12-26 16:54:28,368][105692] Updated weights for policy 0, policy_version 211854 (0.0010) [2023-12-26 16:54:28,971][105620] Updated weights for policy 1, policy_version 212530 (0.0010) [2023-12-26 16:54:29,029][105620] Updated weights for policy 1, policy_version 212540 (0.0010) [2023-12-26 16:54:29,041][105692] Updated weights for policy 0, policy_version 211864 (0.0010) [2023-12-26 16:54:29,084][105620] Updated weights for policy 1, policy_version 212550 (0.0010) [2023-12-26 16:54:29,092][105692] Updated weights for policy 0, policy_version 211874 (0.0010) [2023-12-26 16:54:29,139][105692] Updated weights for policy 0, policy_version 211884 (0.0009) [2023-12-26 16:54:29,839][105692] Updated weights for policy 0, policy_version 211894 (0.0007) [2023-12-26 16:54:29,843][105620] Updated weights for policy 1, policy_version 212560 (0.0010) [2023-12-26 16:54:29,900][105692] Updated weights for policy 0, policy_version 211904 (0.0006) [2023-12-26 16:54:29,907][105620] Updated weights for policy 1, policy_version 212570 (0.0007) [2023-12-26 16:54:29,969][105620] Updated weights for policy 1, policy_version 212580 (0.0008) [2023-12-26 16:54:29,971][105692] Updated weights for policy 0, policy_version 211914 (0.0007) [2023-12-26 16:54:30,678][105692] Updated weights for policy 0, policy_version 211924 (0.0008) [2023-12-26 16:54:30,684][105620] Updated weights for policy 1, policy_version 212590 (0.0008) [2023-12-26 16:54:30,726][105692] Updated weights for policy 0, policy_version 211934 (0.0007) [2023-12-26 16:54:30,736][105620] Updated weights for policy 1, policy_version 212600 (0.0005) [2023-12-26 16:54:30,793][105692] Updated weights for policy 0, policy_version 211944 (0.0007) [2023-12-26 16:54:30,795][105620] Updated weights for policy 1, policy_version 212610 (0.0005) [2023-12-26 16:54:31,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 108707840. Throughput: 0: 9828.2, 1: 9623.9. Samples: 108674252. Policy #0 lag: (min: 11.0, avg: 21.2, max: 43.0) [2023-12-26 16:54:31,062][104569] Avg episode reward: [(0, '8493.727'), (1, '6179.642')] [2023-12-26 16:54:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000211952_54272000.pth... [2023-12-26 16:54:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000212616_54435840.pth... [2023-12-26 16:54:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000211464_54140928.pth [2023-12-26 16:54:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000210800_53977088.pth [2023-12-26 16:54:31,453][105620] Updated weights for policy 1, policy_version 212620 (0.0007) [2023-12-26 16:54:31,480][105692] Updated weights for policy 0, policy_version 211954 (0.0006) [2023-12-26 16:54:31,510][105620] Updated weights for policy 1, policy_version 212630 (0.0008) [2023-12-26 16:54:31,536][105692] Updated weights for policy 0, policy_version 211964 (0.0006) [2023-12-26 16:54:31,571][105620] Updated weights for policy 1, policy_version 212640 (0.0007) [2023-12-26 16:54:31,605][105692] Updated weights for policy 0, policy_version 211974 (0.0008) [2023-12-26 16:54:31,664][105692] Updated weights for policy 0, policy_version 211984 (0.0007) [2023-12-26 16:54:32,293][105692] Updated weights for policy 0, policy_version 211994 (0.0009) [2023-12-26 16:54:32,356][105692] Updated weights for policy 0, policy_version 212004 (0.0008) [2023-12-26 16:54:32,386][105620] Updated weights for policy 1, policy_version 212650 (0.0007) [2023-12-26 16:54:32,408][105692] Updated weights for policy 0, policy_version 212014 (0.0006) [2023-12-26 16:54:32,455][105620] Updated weights for policy 1, policy_version 212660 (0.0009) [2023-12-26 16:54:32,517][105620] Updated weights for policy 1, policy_version 212670 (0.0010) [2023-12-26 16:54:32,578][105620] Updated weights for policy 1, policy_version 212680 (0.0010) [2023-12-26 16:54:32,956][105692] Updated weights for policy 0, policy_version 212024 (0.0005) [2023-12-26 16:54:33,009][105692] Updated weights for policy 0, policy_version 212034 (0.0006) [2023-12-26 16:54:33,063][105692] Updated weights for policy 0, policy_version 212044 (0.0005) [2023-12-26 16:54:33,465][105620] Updated weights for policy 1, policy_version 212690 (0.0009) [2023-12-26 16:54:33,518][105620] Updated weights for policy 1, policy_version 212700 (0.0010) [2023-12-26 16:54:33,572][105620] Updated weights for policy 1, policy_version 212710 (0.0011) [2023-12-26 16:54:33,597][105692] Updated weights for policy 0, policy_version 212054 (0.0005) [2023-12-26 16:54:33,659][105692] Updated weights for policy 0, policy_version 212064 (0.0005) [2023-12-26 16:54:33,718][105692] Updated weights for policy 0, policy_version 212074 (0.0009) [2023-12-26 16:54:34,331][105692] Updated weights for policy 0, policy_version 212084 (0.0009) [2023-12-26 16:54:34,365][105620] Updated weights for policy 1, policy_version 212720 (0.0006) [2023-12-26 16:54:34,387][105692] Updated weights for policy 0, policy_version 212094 (0.0009) [2023-12-26 16:54:34,422][105620] Updated weights for policy 1, policy_version 212730 (0.0007) [2023-12-26 16:54:34,444][105692] Updated weights for policy 0, policy_version 212104 (0.0007) [2023-12-26 16:54:34,482][105620] Updated weights for policy 1, policy_version 212740 (0.0011) [2023-12-26 16:54:35,118][105692] Updated weights for policy 0, policy_version 212114 (0.0008) [2023-12-26 16:54:35,133][105620] Updated weights for policy 1, policy_version 212750 (0.0006) [2023-12-26 16:54:35,173][105692] Updated weights for policy 0, policy_version 212124 (0.0009) [2023-12-26 16:54:35,179][105620] Updated weights for policy 1, policy_version 212760 (0.0005) [2023-12-26 16:54:35,220][105692] Updated weights for policy 0, policy_version 212134 (0.0009) [2023-12-26 16:54:35,227][105620] Updated weights for policy 1, policy_version 212770 (0.0009) [2023-12-26 16:54:35,270][105692] Updated weights for policy 0, policy_version 212144 (0.0009) [2023-12-26 16:54:35,947][105620] Updated weights for policy 1, policy_version 212780 (0.0010) [2023-12-26 16:54:35,969][105586] KL-divergence is very high: 105.9363 [2023-12-26 16:54:35,974][105586] KL-divergence is very high: 104.8653 [2023-12-26 16:54:35,985][105586] KL-divergence is very high: 109.9910 [2023-12-26 16:54:36,001][105620] Updated weights for policy 1, policy_version 212790 (0.0010) [2023-12-26 16:54:36,002][105586] KL-divergence is very high: 101.4963 [2023-12-26 16:54:36,039][105692] Updated weights for policy 0, policy_version 212154 (0.0007) [2023-12-26 16:54:36,050][105620] Updated weights for policy 1, policy_version 212800 (0.0007) [2023-12-26 16:54:36,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 108797952. Throughput: 0: 9886.4, 1: 9614.3. Samples: 108793596. Policy #0 lag: (min: 11.0, avg: 21.2, max: 43.0) [2023-12-26 16:54:36,062][104569] Avg episode reward: [(0, '9089.525'), (1, '2525.349')] [2023-12-26 16:54:36,111][105692] Updated weights for policy 0, policy_version 212164 (0.0006) [2023-12-26 16:54:36,174][105692] Updated weights for policy 0, policy_version 212174 (0.0008) [2023-12-26 16:54:36,715][105620] Updated weights for policy 1, policy_version 212810 (0.0009) [2023-12-26 16:54:36,771][105620] Updated weights for policy 1, policy_version 212820 (0.0010) [2023-12-26 16:54:36,830][105620] Updated weights for policy 1, policy_version 212830 (0.0006) [2023-12-26 16:54:36,890][105620] Updated weights for policy 1, policy_version 212840 (0.0005) [2023-12-26 16:54:36,903][105692] Updated weights for policy 0, policy_version 212184 (0.0009) [2023-12-26 16:54:36,957][105692] Updated weights for policy 0, policy_version 212195 (0.0010) [2023-12-26 16:54:37,015][105692] Updated weights for policy 0, policy_version 212206 (0.0009) [2023-12-26 16:54:37,454][105620] Updated weights for policy 1, policy_version 212850 (0.0005) [2023-12-26 16:54:37,503][105620] Updated weights for policy 1, policy_version 212860 (0.0005) [2023-12-26 16:54:37,562][105620] Updated weights for policy 1, policy_version 212870 (0.0005) [2023-12-26 16:54:37,930][105692] Updated weights for policy 0, policy_version 212216 (0.0009) [2023-12-26 16:54:37,981][105692] Updated weights for policy 0, policy_version 212226 (0.0009) [2023-12-26 16:54:38,042][105692] Updated weights for policy 0, policy_version 212236 (0.0009) [2023-12-26 16:54:38,146][105620] Updated weights for policy 1, policy_version 212880 (0.0009) [2023-12-26 16:54:38,206][105620] Updated weights for policy 1, policy_version 212890 (0.0009) [2023-12-26 16:54:38,265][105620] Updated weights for policy 1, policy_version 212900 (0.0009) [2023-12-26 16:54:38,742][105692] Updated weights for policy 0, policy_version 212246 (0.0007) [2023-12-26 16:54:38,802][105692] Updated weights for policy 0, policy_version 212256 (0.0010) [2023-12-26 16:54:38,860][105692] Updated weights for policy 0, policy_version 212266 (0.0010) [2023-12-26 16:54:38,940][105620] Updated weights for policy 1, policy_version 212910 (0.0008) [2023-12-26 16:54:39,006][105620] Updated weights for policy 1, policy_version 212920 (0.0008) [2023-12-26 16:54:39,068][105620] Updated weights for policy 1, policy_version 212930 (0.0007) [2023-12-26 16:54:39,648][105692] Updated weights for policy 0, policy_version 212276 (0.0007) [2023-12-26 16:54:39,707][105692] Updated weights for policy 0, policy_version 212286 (0.0005) [2023-12-26 16:54:39,769][105692] Updated weights for policy 0, policy_version 212296 (0.0006) [2023-12-26 16:54:39,853][105620] Updated weights for policy 1, policy_version 212940 (0.0009) [2023-12-26 16:54:39,915][105620] Updated weights for policy 1, policy_version 212950 (0.0008) [2023-12-26 16:54:39,980][105620] Updated weights for policy 1, policy_version 212960 (0.0008) [2023-12-26 16:54:40,498][105692] Updated weights for policy 0, policy_version 212306 (0.0006) [2023-12-26 16:54:40,559][105692] Updated weights for policy 0, policy_version 212316 (0.0005) [2023-12-26 16:54:40,618][105692] Updated weights for policy 0, policy_version 212326 (0.0006) [2023-12-26 16:54:40,680][105692] Updated weights for policy 0, policy_version 212336 (0.0009) [2023-12-26 16:54:40,782][105620] Updated weights for policy 1, policy_version 212970 (0.0008) [2023-12-26 16:54:40,847][105620] Updated weights for policy 1, policy_version 212980 (0.0009) [2023-12-26 16:54:40,910][105620] Updated weights for policy 1, policy_version 212990 (0.0006) [2023-12-26 16:54:40,976][105620] Updated weights for policy 1, policy_version 213000 (0.0005) [2023-12-26 16:54:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 108904448. Throughput: 0: 9813.2, 1: 9721.0. Samples: 108910640. Policy #0 lag: (min: 11.0, avg: 21.2, max: 43.0) [2023-12-26 16:54:41,062][104569] Avg episode reward: [(0, '8248.140'), (1, '4734.901')] [2023-12-26 16:54:41,334][105692] Updated weights for policy 0, policy_version 212346 (0.0008) [2023-12-26 16:54:41,406][105692] Updated weights for policy 0, policy_version 212356 (0.0009) [2023-12-26 16:54:41,468][105692] Updated weights for policy 0, policy_version 212366 (0.0007) [2023-12-26 16:54:41,741][105620] Updated weights for policy 1, policy_version 213010 (0.0009) [2023-12-26 16:54:41,806][105620] Updated weights for policy 1, policy_version 213020 (0.0007) [2023-12-26 16:54:41,866][105620] Updated weights for policy 1, policy_version 213030 (0.0009) [2023-12-26 16:54:42,128][105692] Updated weights for policy 0, policy_version 212376 (0.0007) [2023-12-26 16:54:42,182][105692] Updated weights for policy 0, policy_version 212386 (0.0011) [2023-12-26 16:54:42,254][105692] Updated weights for policy 0, policy_version 212396 (0.0011) [2023-12-26 16:54:42,593][105620] Updated weights for policy 1, policy_version 213040 (0.0008) [2023-12-26 16:54:42,649][105620] Updated weights for policy 1, policy_version 213050 (0.0009) [2023-12-26 16:54:42,710][105620] Updated weights for policy 1, policy_version 213060 (0.0008) [2023-12-26 16:54:42,956][105692] Updated weights for policy 0, policy_version 212406 (0.0009) [2023-12-26 16:54:43,014][105692] Updated weights for policy 0, policy_version 212416 (0.0011) [2023-12-26 16:54:43,063][105692] Updated weights for policy 0, policy_version 212426 (0.0011) [2023-12-26 16:54:43,383][105620] Updated weights for policy 1, policy_version 213070 (0.0008) [2023-12-26 16:54:43,438][105620] Updated weights for policy 1, policy_version 213080 (0.0008) [2023-12-26 16:54:43,489][105620] Updated weights for policy 1, policy_version 213090 (0.0008) [2023-12-26 16:54:43,818][105692] Updated weights for policy 0, policy_version 212436 (0.0011) [2023-12-26 16:54:43,873][105692] Updated weights for policy 0, policy_version 212446 (0.0010) [2023-12-26 16:54:43,931][105692] Updated weights for policy 0, policy_version 212456 (0.0010) [2023-12-26 16:54:44,257][105620] Updated weights for policy 1, policy_version 213100 (0.0007) [2023-12-26 16:54:44,311][105620] Updated weights for policy 1, policy_version 213110 (0.0006) [2023-12-26 16:54:44,366][105620] Updated weights for policy 1, policy_version 213120 (0.0008) [2023-12-26 16:54:44,605][105692] Updated weights for policy 0, policy_version 212466 (0.0009) [2023-12-26 16:54:44,653][105692] Updated weights for policy 0, policy_version 212476 (0.0005) [2023-12-26 16:54:44,713][105692] Updated weights for policy 0, policy_version 212486 (0.0005) [2023-12-26 16:54:44,779][105692] Updated weights for policy 0, policy_version 212496 (0.0008) [2023-12-26 16:54:45,150][105620] Updated weights for policy 1, policy_version 213130 (0.0009) [2023-12-26 16:54:45,204][105620] Updated weights for policy 1, policy_version 213140 (0.0008) [2023-12-26 16:54:45,252][105620] Updated weights for policy 1, policy_version 213150 (0.0008) [2023-12-26 16:54:45,316][105620] Updated weights for policy 1, policy_version 213160 (0.0008) [2023-12-26 16:54:45,458][105692] Updated weights for policy 0, policy_version 212506 (0.0011) [2023-12-26 16:54:45,521][105692] Updated weights for policy 0, policy_version 212516 (0.0010) [2023-12-26 16:54:45,580][105692] Updated weights for policy 0, policy_version 212526 (0.0010) [2023-12-26 16:54:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 108994560. Throughput: 0: 9838.0, 1: 9716.4. Samples: 108968696. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:54:46,063][104569] Avg episode reward: [(0, '7373.711'), (1, '7069.629')] [2023-12-26 16:54:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000212528_54419456.pth... [2023-12-26 16:54:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000211344_54116352.pth [2023-12-26 16:54:46,082][105620] Updated weights for policy 1, policy_version 213170 (0.0006) [2023-12-26 16:54:46,145][105620] Updated weights for policy 1, policy_version 213180 (0.0006) [2023-12-26 16:54:46,158][105692] Updated weights for policy 0, policy_version 212536 (0.0007) [2023-12-26 16:54:46,211][105620] Updated weights for policy 1, policy_version 213190 (0.0008) [2023-12-26 16:54:46,213][105692] Updated weights for policy 0, policy_version 212546 (0.0010) [2023-12-26 16:54:46,219][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000213192_54583296.pth... [2023-12-26 16:54:46,223][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000212040_54288384.pth [2023-12-26 16:54:46,268][105692] Updated weights for policy 0, policy_version 212556 (0.0010) [2023-12-26 16:54:46,917][105692] Updated weights for policy 0, policy_version 212566 (0.0008) [2023-12-26 16:54:46,918][105620] Updated weights for policy 1, policy_version 213200 (0.0007) [2023-12-26 16:54:46,965][105692] Updated weights for policy 0, policy_version 212576 (0.0006) [2023-12-26 16:54:46,975][105620] Updated weights for policy 1, policy_version 213210 (0.0007) [2023-12-26 16:54:47,021][105692] Updated weights for policy 0, policy_version 212586 (0.0008) [2023-12-26 16:54:47,025][105620] Updated weights for policy 1, policy_version 213220 (0.0007) [2023-12-26 16:54:47,610][105620] Updated weights for policy 1, policy_version 213230 (0.0008) [2023-12-26 16:54:47,671][105620] Updated weights for policy 1, policy_version 213240 (0.0009) [2023-12-26 16:54:47,720][105620] Updated weights for policy 1, policy_version 213250 (0.0008) [2023-12-26 16:54:47,750][105692] Updated weights for policy 0, policy_version 212596 (0.0007) [2023-12-26 16:54:47,795][105692] Updated weights for policy 0, policy_version 212606 (0.0006) [2023-12-26 16:54:47,841][105692] Updated weights for policy 0, policy_version 212616 (0.0008) [2023-12-26 16:54:48,504][105620] Updated weights for policy 1, policy_version 213260 (0.0008) [2023-12-26 16:54:48,572][105620] Updated weights for policy 1, policy_version 213270 (0.0006) [2023-12-26 16:54:48,588][105692] Updated weights for policy 0, policy_version 212626 (0.0009) [2023-12-26 16:54:48,631][105620] Updated weights for policy 1, policy_version 213280 (0.0006) [2023-12-26 16:54:48,641][105692] Updated weights for policy 0, policy_version 212636 (0.0009) [2023-12-26 16:54:48,692][105692] Updated weights for policy 0, policy_version 212646 (0.0007) [2023-12-26 16:54:48,746][105692] Updated weights for policy 0, policy_version 212656 (0.0009) [2023-12-26 16:54:49,197][105620] Updated weights for policy 1, policy_version 213290 (0.0006) [2023-12-26 16:54:49,265][105620] Updated weights for policy 1, policy_version 213300 (0.0009) [2023-12-26 16:54:49,323][105620] Updated weights for policy 1, policy_version 213310 (0.0008) [2023-12-26 16:54:49,392][105620] Updated weights for policy 1, policy_version 213320 (0.0008) [2023-12-26 16:54:49,584][105692] Updated weights for policy 0, policy_version 212666 (0.0010) [2023-12-26 16:54:49,636][105692] Updated weights for policy 0, policy_version 212676 (0.0010) [2023-12-26 16:54:49,687][105692] Updated weights for policy 0, policy_version 212686 (0.0009) [2023-12-26 16:54:50,041][105620] Updated weights for policy 1, policy_version 213330 (0.0008) [2023-12-26 16:54:50,104][105620] Updated weights for policy 1, policy_version 213340 (0.0006) [2023-12-26 16:54:50,168][105620] Updated weights for policy 1, policy_version 213350 (0.0008) [2023-12-26 16:54:50,540][105692] Updated weights for policy 0, policy_version 212696 (0.0010) [2023-12-26 16:54:50,601][105692] Updated weights for policy 0, policy_version 212706 (0.0008) [2023-12-26 16:54:50,654][105692] Updated weights for policy 0, policy_version 212716 (0.0008) [2023-12-26 16:54:50,795][105620] Updated weights for policy 1, policy_version 213360 (0.0008) [2023-12-26 16:54:50,857][105620] Updated weights for policy 1, policy_version 213370 (0.0008) [2023-12-26 16:54:50,911][105620] Updated weights for policy 1, policy_version 213381 (0.0010) [2023-12-26 16:54:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 109101056. Throughput: 0: 9957.3, 1: 9737.4. Samples: 109088144. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:54:51,062][104569] Avg episode reward: [(0, '990.595'), (1, '9357.432')] [2023-12-26 16:54:51,431][105692] Updated weights for policy 0, policy_version 212726 (0.0009) [2023-12-26 16:54:51,494][105692] Updated weights for policy 0, policy_version 212736 (0.0009) [2023-12-26 16:54:51,557][105692] Updated weights for policy 0, policy_version 212746 (0.0009) [2023-12-26 16:54:51,683][105620] Updated weights for policy 1, policy_version 213391 (0.0008) [2023-12-26 16:54:51,749][105620] Updated weights for policy 1, policy_version 213401 (0.0008) [2023-12-26 16:54:51,803][105620] Updated weights for policy 1, policy_version 213411 (0.0009) [2023-12-26 16:54:52,319][105692] Updated weights for policy 0, policy_version 212756 (0.0008) [2023-12-26 16:54:52,379][105692] Updated weights for policy 0, policy_version 212766 (0.0010) [2023-12-26 16:54:52,435][105692] Updated weights for policy 0, policy_version 212776 (0.0009) [2023-12-26 16:54:52,575][105620] Updated weights for policy 1, policy_version 213421 (0.0008) [2023-12-26 16:54:52,637][105620] Updated weights for policy 1, policy_version 213431 (0.0008) [2023-12-26 16:54:52,699][105620] Updated weights for policy 1, policy_version 213441 (0.0005) [2023-12-26 16:54:53,214][105692] Updated weights for policy 0, policy_version 212786 (0.0008) [2023-12-26 16:54:53,269][105692] Updated weights for policy 0, policy_version 212796 (0.0005) [2023-12-26 16:54:53,327][105692] Updated weights for policy 0, policy_version 212806 (0.0005) [2023-12-26 16:54:53,328][105620] Updated weights for policy 1, policy_version 213451 (0.0006) [2023-12-26 16:54:53,373][105692] Updated weights for policy 0, policy_version 212816 (0.0005) [2023-12-26 16:54:53,387][105620] Updated weights for policy 1, policy_version 213461 (0.0007) [2023-12-26 16:54:53,435][105620] Updated weights for policy 1, policy_version 213471 (0.0005) [2023-12-26 16:54:54,027][105620] Updated weights for policy 1, policy_version 213481 (0.0006) [2023-12-26 16:54:54,072][105620] Updated weights for policy 1, policy_version 213491 (0.0006) [2023-12-26 16:54:54,094][105692] Updated weights for policy 0, policy_version 212826 (0.0011) [2023-12-26 16:54:54,120][105620] Updated weights for policy 1, policy_version 213501 (0.0006) [2023-12-26 16:54:54,143][105692] Updated weights for policy 0, policy_version 212836 (0.0011) [2023-12-26 16:54:54,181][105620] Updated weights for policy 1, policy_version 213511 (0.0007) [2023-12-26 16:54:54,199][105692] Updated weights for policy 0, policy_version 212846 (0.0010) [2023-12-26 16:54:54,864][105620] Updated weights for policy 1, policy_version 213521 (0.0007) [2023-12-26 16:54:54,912][105620] Updated weights for policy 1, policy_version 213531 (0.0008) [2023-12-26 16:54:54,949][105692] Updated weights for policy 0, policy_version 212856 (0.0007) [2023-12-26 16:54:54,984][105620] Updated weights for policy 1, policy_version 213542 (0.0007) [2023-12-26 16:54:55,006][105692] Updated weights for policy 0, policy_version 212866 (0.0008) [2023-12-26 16:54:55,057][105692] Updated weights for policy 0, policy_version 212876 (0.0010) [2023-12-26 16:54:55,669][105692] Updated weights for policy 0, policy_version 212886 (0.0010) [2023-12-26 16:54:55,701][105620] Updated weights for policy 1, policy_version 213552 (0.0006) [2023-12-26 16:54:55,724][105692] Updated weights for policy 0, policy_version 212896 (0.0010) [2023-12-26 16:54:55,748][105620] Updated weights for policy 1, policy_version 213562 (0.0005) [2023-12-26 16:54:55,769][105692] Updated weights for policy 0, policy_version 212906 (0.0010) [2023-12-26 16:54:55,798][105620] Updated weights for policy 1, policy_version 213572 (0.0005) [2023-12-26 16:54:56,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19605.2). Total num frames: 109199360. Throughput: 0: 9955.5, 1: 9761.0. Samples: 109205516. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:54:56,063][104569] Avg episode reward: [(0, '1231.380'), (1, '9358.168')] [2023-12-26 16:54:56,393][105692] Updated weights for policy 0, policy_version 212916 (0.0008) [2023-12-26 16:54:56,448][105692] Updated weights for policy 0, policy_version 212926 (0.0005) [2023-12-26 16:54:56,469][105620] Updated weights for policy 1, policy_version 213582 (0.0005) [2023-12-26 16:54:56,505][105692] Updated weights for policy 0, policy_version 212936 (0.0005) [2023-12-26 16:54:56,523][105620] Updated weights for policy 1, policy_version 213592 (0.0005) [2023-12-26 16:54:56,577][105620] Updated weights for policy 1, policy_version 213602 (0.0006) [2023-12-26 16:54:57,071][105692] Updated weights for policy 0, policy_version 212946 (0.0007) [2023-12-26 16:54:57,128][105692] Updated weights for policy 0, policy_version 212956 (0.0010) [2023-12-26 16:54:57,184][105692] Updated weights for policy 0, policy_version 212966 (0.0009) [2023-12-26 16:54:57,242][105692] Updated weights for policy 0, policy_version 212976 (0.0010) [2023-12-26 16:54:57,281][105620] Updated weights for policy 1, policy_version 213612 (0.0010) [2023-12-26 16:54:57,332][105620] Updated weights for policy 1, policy_version 213622 (0.0010) [2023-12-26 16:54:57,387][105620] Updated weights for policy 1, policy_version 213632 (0.0010) [2023-12-26 16:54:57,826][105692] Updated weights for policy 0, policy_version 212986 (0.0005) [2023-12-26 16:54:57,889][105692] Updated weights for policy 0, policy_version 212996 (0.0009) [2023-12-26 16:54:57,953][105692] Updated weights for policy 0, policy_version 213006 (0.0010) [2023-12-26 16:54:58,111][105620] Updated weights for policy 1, policy_version 213642 (0.0010) [2023-12-26 16:54:58,175][105620] Updated weights for policy 1, policy_version 213652 (0.0010) [2023-12-26 16:54:58,239][105620] Updated weights for policy 1, policy_version 213662 (0.0007) [2023-12-26 16:54:58,303][105620] Updated weights for policy 1, policy_version 213672 (0.0008) [2023-12-26 16:54:58,628][105692] Updated weights for policy 0, policy_version 213016 (0.0010) [2023-12-26 16:54:58,688][105692] Updated weights for policy 0, policy_version 213026 (0.0010) [2023-12-26 16:54:58,751][105692] Updated weights for policy 0, policy_version 213036 (0.0010) [2023-12-26 16:54:59,039][105620] Updated weights for policy 1, policy_version 213682 (0.0005) [2023-12-26 16:54:59,084][105620] Updated weights for policy 1, policy_version 213692 (0.0005) [2023-12-26 16:54:59,134][105620] Updated weights for policy 1, policy_version 213702 (0.0005) [2023-12-26 16:54:59,402][105692] Updated weights for policy 0, policy_version 213046 (0.0009) [2023-12-26 16:54:59,470][105692] Updated weights for policy 0, policy_version 213056 (0.0008) [2023-12-26 16:54:59,521][105692] Updated weights for policy 0, policy_version 213066 (0.0008) [2023-12-26 16:54:59,809][105620] Updated weights for policy 1, policy_version 213712 (0.0008) [2023-12-26 16:54:59,875][105620] Updated weights for policy 1, policy_version 213722 (0.0008) [2023-12-26 16:54:59,939][105620] Updated weights for policy 1, policy_version 213732 (0.0007) [2023-12-26 16:55:00,267][105692] Updated weights for policy 0, policy_version 213076 (0.0007) [2023-12-26 16:55:00,311][105692] Updated weights for policy 0, policy_version 213086 (0.0005) [2023-12-26 16:55:00,370][105692] Updated weights for policy 0, policy_version 213096 (0.0007) [2023-12-26 16:55:00,618][105620] Updated weights for policy 1, policy_version 213742 (0.0008) [2023-12-26 16:55:00,679][105620] Updated weights for policy 1, policy_version 213752 (0.0006) [2023-12-26 16:55:00,739][105620] Updated weights for policy 1, policy_version 213762 (0.0008) [2023-12-26 16:55:01,001][105692] Updated weights for policy 0, policy_version 213106 (0.0008) [2023-12-26 16:55:01,059][105692] Updated weights for policy 0, policy_version 213116 (0.0008) [2023-12-26 16:55:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 109297664. Throughput: 0: 10057.4, 1: 9742.3. Samples: 109267696. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:55:01,063][104569] Avg episode reward: [(0, '6406.659'), (1, '9358.150')] [2023-12-26 16:55:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000213768_54730752.pth... [2023-12-26 16:55:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000212616_54435840.pth [2023-12-26 16:55:01,120][105692] Updated weights for policy 0, policy_version 213126 (0.0010) [2023-12-26 16:55:01,186][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000213136_54575104.pth... [2023-12-26 16:55:01,186][105692] Updated weights for policy 0, policy_version 213136 (0.0011) [2023-12-26 16:55:01,189][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000211952_54272000.pth [2023-12-26 16:55:01,360][105620] Updated weights for policy 1, policy_version 213772 (0.0008) [2023-12-26 16:55:01,422][105620] Updated weights for policy 1, policy_version 213782 (0.0008) [2023-12-26 16:55:01,474][105620] Updated weights for policy 1, policy_version 213792 (0.0007) [2023-12-26 16:55:01,996][105692] Updated weights for policy 0, policy_version 213146 (0.0005) [2023-12-26 16:55:02,041][105692] Updated weights for policy 0, policy_version 213156 (0.0006) [2023-12-26 16:55:02,093][105692] Updated weights for policy 0, policy_version 213166 (0.0008) [2023-12-26 16:55:02,247][105620] Updated weights for policy 1, policy_version 213802 (0.0010) [2023-12-26 16:55:02,310][105620] Updated weights for policy 1, policy_version 213812 (0.0009) [2023-12-26 16:55:02,373][105620] Updated weights for policy 1, policy_version 213822 (0.0010) [2023-12-26 16:55:02,434][105620] Updated weights for policy 1, policy_version 213832 (0.0009) [2023-12-26 16:55:02,763][105692] Updated weights for policy 0, policy_version 213176 (0.0006) [2023-12-26 16:55:02,832][105692] Updated weights for policy 0, policy_version 213186 (0.0005) [2023-12-26 16:55:02,897][105692] Updated weights for policy 0, policy_version 213196 (0.0005) [2023-12-26 16:55:03,151][105620] Updated weights for policy 1, policy_version 213842 (0.0006) [2023-12-26 16:55:03,209][105620] Updated weights for policy 1, policy_version 213852 (0.0005) [2023-12-26 16:55:03,267][105620] Updated weights for policy 1, policy_version 213862 (0.0005) [2023-12-26 16:55:03,432][105692] Updated weights for policy 0, policy_version 213206 (0.0005) [2023-12-26 16:55:03,501][105692] Updated weights for policy 0, policy_version 213216 (0.0005) [2023-12-26 16:55:03,570][105692] Updated weights for policy 0, policy_version 213226 (0.0005) [2023-12-26 16:55:03,952][105620] Updated weights for policy 1, policy_version 213872 (0.0006) [2023-12-26 16:55:04,003][105620] Updated weights for policy 1, policy_version 213882 (0.0005) [2023-12-26 16:55:04,058][105620] Updated weights for policy 1, policy_version 213892 (0.0008) [2023-12-26 16:55:04,217][105692] Updated weights for policy 0, policy_version 213236 (0.0007) [2023-12-26 16:55:04,276][105692] Updated weights for policy 0, policy_version 213246 (0.0009) [2023-12-26 16:55:04,339][105692] Updated weights for policy 0, policy_version 213256 (0.0009) [2023-12-26 16:55:04,768][105620] Updated weights for policy 1, policy_version 213902 (0.0007) [2023-12-26 16:55:04,814][105620] Updated weights for policy 1, policy_version 213912 (0.0005) [2023-12-26 16:55:04,863][105620] Updated weights for policy 1, policy_version 213922 (0.0009) [2023-12-26 16:55:05,071][105692] Updated weights for policy 0, policy_version 213266 (0.0008) [2023-12-26 16:55:05,135][105692] Updated weights for policy 0, policy_version 213276 (0.0005) [2023-12-26 16:55:05,193][105692] Updated weights for policy 0, policy_version 213286 (0.0007) [2023-12-26 16:55:05,239][105692] Updated weights for policy 0, policy_version 213296 (0.0008) [2023-12-26 16:55:05,543][105620] Updated weights for policy 1, policy_version 213932 (0.0009) [2023-12-26 16:55:05,600][105620] Updated weights for policy 1, policy_version 213942 (0.0008) [2023-12-26 16:55:05,646][105620] Updated weights for policy 1, policy_version 213952 (0.0008) [2023-12-26 16:55:05,896][105692] Updated weights for policy 0, policy_version 213306 (0.0005) [2023-12-26 16:55:05,954][105692] Updated weights for policy 0, policy_version 213316 (0.0005) [2023-12-26 16:55:06,009][105692] Updated weights for policy 0, policy_version 213326 (0.0006) [2023-12-26 16:55:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19933.8, 300 sec: 19633.0). Total num frames: 109404160. Throughput: 0: 10049.1, 1: 9797.4. Samples: 109388604. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:55:06,063][104569] Avg episode reward: [(0, '8230.197'), (1, '9358.129')] [2023-12-26 16:55:06,477][105620] Updated weights for policy 1, policy_version 213962 (0.0008) [2023-12-26 16:55:06,539][105620] Updated weights for policy 1, policy_version 213972 (0.0009) [2023-12-26 16:55:06,600][105620] Updated weights for policy 1, policy_version 213982 (0.0009) [2023-12-26 16:55:06,663][105620] Updated weights for policy 1, policy_version 213992 (0.0008) [2023-12-26 16:55:06,669][105692] Updated weights for policy 0, policy_version 213336 (0.0007) [2023-12-26 16:55:06,731][105692] Updated weights for policy 0, policy_version 213346 (0.0006) [2023-12-26 16:55:06,791][105692] Updated weights for policy 0, policy_version 213356 (0.0005) [2023-12-26 16:55:07,423][105692] Updated weights for policy 0, policy_version 213366 (0.0007) [2023-12-26 16:55:07,455][105620] Updated weights for policy 1, policy_version 214002 (0.0009) [2023-12-26 16:55:07,473][105692] Updated weights for policy 0, policy_version 213376 (0.0007) [2023-12-26 16:55:07,515][105620] Updated weights for policy 1, policy_version 214012 (0.0008) [2023-12-26 16:55:07,532][105692] Updated weights for policy 0, policy_version 213386 (0.0007) [2023-12-26 16:55:07,564][105620] Updated weights for policy 1, policy_version 214022 (0.0008) [2023-12-26 16:55:08,284][105692] Updated weights for policy 0, policy_version 213396 (0.0008) [2023-12-26 16:55:08,342][105620] Updated weights for policy 1, policy_version 214032 (0.0008) [2023-12-26 16:55:08,359][105692] Updated weights for policy 0, policy_version 213406 (0.0012) [2023-12-26 16:55:08,397][105620] Updated weights for policy 1, policy_version 214042 (0.0007) [2023-12-26 16:55:08,423][105692] Updated weights for policy 0, policy_version 213416 (0.0009) [2023-12-26 16:55:08,450][105620] Updated weights for policy 1, policy_version 214052 (0.0006) [2023-12-26 16:55:09,168][105692] Updated weights for policy 0, policy_version 213426 (0.0010) [2023-12-26 16:55:09,224][105620] Updated weights for policy 1, policy_version 214062 (0.0007) [2023-12-26 16:55:09,228][105692] Updated weights for policy 0, policy_version 213436 (0.0011) [2023-12-26 16:55:09,290][105620] Updated weights for policy 1, policy_version 214072 (0.0007) [2023-12-26 16:55:09,291][105692] Updated weights for policy 0, policy_version 213446 (0.0011) [2023-12-26 16:55:09,355][105620] Updated weights for policy 1, policy_version 214082 (0.0009) [2023-12-26 16:55:09,358][105692] Updated weights for policy 0, policy_version 213456 (0.0009) [2023-12-26 16:55:10,112][105692] Updated weights for policy 0, policy_version 213466 (0.0010) [2023-12-26 16:55:10,142][105620] Updated weights for policy 1, policy_version 214092 (0.0007) [2023-12-26 16:55:10,171][105692] Updated weights for policy 0, policy_version 213476 (0.0011) [2023-12-26 16:55:10,202][105620] Updated weights for policy 1, policy_version 214102 (0.0006) [2023-12-26 16:55:10,227][105692] Updated weights for policy 0, policy_version 213486 (0.0010) [2023-12-26 16:55:10,255][105620] Updated weights for policy 1, policy_version 214112 (0.0007) [2023-12-26 16:55:10,978][105692] Updated weights for policy 0, policy_version 213496 (0.0010) [2023-12-26 16:55:11,016][105620] Updated weights for policy 1, policy_version 214122 (0.0008) [2023-12-26 16:55:11,035][105692] Updated weights for policy 0, policy_version 213506 (0.0010) [2023-12-26 16:55:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 109486080. Throughput: 0: 9986.6, 1: 9764.8. Samples: 109502092. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:55:11,063][104569] Avg episode reward: [(0, '8742.840'), (1, '9358.128')] [2023-12-26 16:55:11,077][105620] Updated weights for policy 1, policy_version 214132 (0.0007) [2023-12-26 16:55:11,099][105692] Updated weights for policy 0, policy_version 213516 (0.0011) [2023-12-26 16:55:11,143][105620] Updated weights for policy 1, policy_version 214142 (0.0007) [2023-12-26 16:55:11,211][105620] Updated weights for policy 1, policy_version 214152 (0.0008) [2023-12-26 16:55:11,889][105692] Updated weights for policy 0, policy_version 213526 (0.0011) [2023-12-26 16:55:11,952][105692] Updated weights for policy 0, policy_version 213536 (0.0010) [2023-12-26 16:55:12,000][105620] Updated weights for policy 1, policy_version 214162 (0.0008) [2023-12-26 16:55:12,013][105692] Updated weights for policy 0, policy_version 213546 (0.0010) [2023-12-26 16:55:12,056][105620] Updated weights for policy 1, policy_version 214172 (0.0007) [2023-12-26 16:55:12,105][105620] Updated weights for policy 1, policy_version 214182 (0.0008) [2023-12-26 16:55:12,772][105692] Updated weights for policy 0, policy_version 213556 (0.0010) [2023-12-26 16:55:12,835][105692] Updated weights for policy 0, policy_version 213566 (0.0010) [2023-12-26 16:55:12,885][105620] Updated weights for policy 1, policy_version 214192 (0.0008) [2023-12-26 16:55:12,901][105692] Updated weights for policy 0, policy_version 213576 (0.0009) [2023-12-26 16:55:12,948][105620] Updated weights for policy 1, policy_version 214202 (0.0006) [2023-12-26 16:55:13,002][105620] Updated weights for policy 1, policy_version 214212 (0.0009) [2023-12-26 16:55:13,638][105692] Updated weights for policy 0, policy_version 213586 (0.0009) [2023-12-26 16:55:13,709][105692] Updated weights for policy 0, policy_version 213596 (0.0005) [2023-12-26 16:55:13,772][105692] Updated weights for policy 0, policy_version 213606 (0.0005) [2023-12-26 16:55:13,800][105620] Updated weights for policy 1, policy_version 214222 (0.0010) [2023-12-26 16:55:13,835][105692] Updated weights for policy 0, policy_version 213616 (0.0006) [2023-12-26 16:55:13,856][105620] Updated weights for policy 1, policy_version 214232 (0.0010) [2023-12-26 16:55:13,915][105620] Updated weights for policy 1, policy_version 214242 (0.0010) [2023-12-26 16:55:14,480][105692] Updated weights for policy 0, policy_version 213626 (0.0007) [2023-12-26 16:55:14,534][105692] Updated weights for policy 0, policy_version 213636 (0.0008) [2023-12-26 16:55:14,598][105692] Updated weights for policy 0, policy_version 213646 (0.0008) [2023-12-26 16:55:14,644][105620] Updated weights for policy 1, policy_version 214252 (0.0010) [2023-12-26 16:55:14,699][105620] Updated weights for policy 1, policy_version 214262 (0.0010) [2023-12-26 16:55:14,761][105620] Updated weights for policy 1, policy_version 214272 (0.0010) [2023-12-26 16:55:15,330][105692] Updated weights for policy 0, policy_version 213656 (0.0007) [2023-12-26 16:55:15,379][105692] Updated weights for policy 0, policy_version 213666 (0.0005) [2023-12-26 16:55:15,430][105692] Updated weights for policy 0, policy_version 213676 (0.0005) [2023-12-26 16:55:15,436][105620] Updated weights for policy 1, policy_version 214282 (0.0009) [2023-12-26 16:55:15,490][105620] Updated weights for policy 1, policy_version 214292 (0.0010) [2023-12-26 16:55:15,545][105620] Updated weights for policy 1, policy_version 214302 (0.0010) [2023-12-26 16:55:15,603][105620] Updated weights for policy 1, policy_version 214312 (0.0010) [2023-12-26 16:55:16,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 109584384. Throughput: 0: 9920.5, 1: 9675.4. Samples: 109556072. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:55:16,063][104569] Avg episode reward: [(0, '8928.516'), (1, '9358.016')] [2023-12-26 16:55:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000214312_54870016.pth... [2023-12-26 16:55:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000213680_54714368.pth... [2023-12-26 16:55:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000213192_54583296.pth [2023-12-26 16:55:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000212528_54419456.pth [2023-12-26 16:55:16,115][105692] Updated weights for policy 0, policy_version 213686 (0.0007) [2023-12-26 16:55:16,162][105692] Updated weights for policy 0, policy_version 213696 (0.0006) [2023-12-26 16:55:16,217][105692] Updated weights for policy 0, policy_version 213706 (0.0006) [2023-12-26 16:55:16,344][105620] Updated weights for policy 1, policy_version 214322 (0.0006) [2023-12-26 16:55:16,396][105620] Updated weights for policy 1, policy_version 214333 (0.0010) [2023-12-26 16:55:16,450][105620] Updated weights for policy 1, policy_version 214343 (0.0009) [2023-12-26 16:55:16,806][105692] Updated weights for policy 0, policy_version 213716 (0.0007) [2023-12-26 16:55:16,862][105692] Updated weights for policy 0, policy_version 213726 (0.0009) [2023-12-26 16:55:16,920][105692] Updated weights for policy 0, policy_version 213736 (0.0005) [2023-12-26 16:55:17,211][105620] Updated weights for policy 1, policy_version 214353 (0.0010) [2023-12-26 16:55:17,266][105620] Updated weights for policy 1, policy_version 214363 (0.0008) [2023-12-26 16:55:17,326][105620] Updated weights for policy 1, policy_version 214373 (0.0008) [2023-12-26 16:55:17,545][105692] Updated weights for policy 0, policy_version 213746 (0.0006) [2023-12-26 16:55:17,604][105692] Updated weights for policy 0, policy_version 213756 (0.0007) [2023-12-26 16:55:17,662][105692] Updated weights for policy 0, policy_version 213766 (0.0010) [2023-12-26 16:55:17,721][105692] Updated weights for policy 0, policy_version 213776 (0.0010) [2023-12-26 16:55:18,038][105620] Updated weights for policy 1, policy_version 214383 (0.0006) [2023-12-26 16:55:18,103][105620] Updated weights for policy 1, policy_version 214393 (0.0006) [2023-12-26 16:55:18,173][105620] Updated weights for policy 1, policy_version 214403 (0.0009) [2023-12-26 16:55:18,350][105692] Updated weights for policy 0, policy_version 213786 (0.0007) [2023-12-26 16:55:18,406][105692] Updated weights for policy 0, policy_version 213796 (0.0007) [2023-12-26 16:55:18,466][105692] Updated weights for policy 0, policy_version 213806 (0.0009) [2023-12-26 16:55:18,906][105620] Updated weights for policy 1, policy_version 214413 (0.0010) [2023-12-26 16:55:18,963][105620] Updated weights for policy 1, policy_version 214423 (0.0008) [2023-12-26 16:55:19,020][105620] Updated weights for policy 1, policy_version 214433 (0.0008) [2023-12-26 16:55:19,205][105692] Updated weights for policy 0, policy_version 213816 (0.0008) [2023-12-26 16:55:19,272][105692] Updated weights for policy 0, policy_version 213826 (0.0009) [2023-12-26 16:55:19,334][105692] Updated weights for policy 0, policy_version 213836 (0.0009) [2023-12-26 16:55:19,838][105620] Updated weights for policy 1, policy_version 214443 (0.0009) [2023-12-26 16:55:19,907][105620] Updated weights for policy 1, policy_version 214453 (0.0008) [2023-12-26 16:55:19,972][105620] Updated weights for policy 1, policy_version 214463 (0.0006) [2023-12-26 16:55:19,978][105692] Updated weights for policy 0, policy_version 213846 (0.0009) [2023-12-26 16:55:20,039][105692] Updated weights for policy 0, policy_version 213856 (0.0008) [2023-12-26 16:55:20,103][105692] Updated weights for policy 0, policy_version 213866 (0.0008) [2023-12-26 16:55:20,700][105692] Updated weights for policy 0, policy_version 213876 (0.0010) [2023-12-26 16:55:20,765][105692] Updated weights for policy 0, policy_version 213886 (0.0009) [2023-12-26 16:55:20,806][105620] Updated weights for policy 1, policy_version 214473 (0.0007) [2023-12-26 16:55:20,827][105692] Updated weights for policy 0, policy_version 213896 (0.0009) [2023-12-26 16:55:20,869][105620] Updated weights for policy 1, policy_version 214483 (0.0006) [2023-12-26 16:55:20,936][105620] Updated weights for policy 1, policy_version 214493 (0.0010) [2023-12-26 16:55:20,993][105620] Updated weights for policy 1, policy_version 214503 (0.0007) [2023-12-26 16:55:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 109690880. Throughput: 0: 9864.2, 1: 9705.5. Samples: 109674236. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:55:21,063][104569] Avg episode reward: [(0, '9271.710'), (1, '9357.874')] [2023-12-26 16:55:21,585][105692] Updated weights for policy 0, policy_version 213906 (0.0007) [2023-12-26 16:55:21,651][105692] Updated weights for policy 0, policy_version 213916 (0.0009) [2023-12-26 16:55:21,713][105692] Updated weights for policy 0, policy_version 213926 (0.0009) [2023-12-26 16:55:21,805][105692] Updated weights for policy 0, policy_version 213936 (0.0007) [2023-12-26 16:55:21,820][105620] Updated weights for policy 1, policy_version 214513 (0.0008) [2023-12-26 16:55:21,883][105620] Updated weights for policy 1, policy_version 214523 (0.0009) [2023-12-26 16:55:21,943][105620] Updated weights for policy 1, policy_version 214533 (0.0008) [2023-12-26 16:55:22,425][105692] Updated weights for policy 0, policy_version 213946 (0.0009) [2023-12-26 16:55:22,488][105692] Updated weights for policy 0, policy_version 213956 (0.0009) [2023-12-26 16:55:22,552][105692] Updated weights for policy 0, policy_version 213966 (0.0009) [2023-12-26 16:55:22,763][105620] Updated weights for policy 1, policy_version 214543 (0.0008) [2023-12-26 16:55:22,817][105620] Updated weights for policy 1, policy_version 214553 (0.0009) [2023-12-26 16:55:22,880][105620] Updated weights for policy 1, policy_version 214563 (0.0008) [2023-12-26 16:55:23,276][105692] Updated weights for policy 0, policy_version 213976 (0.0010) [2023-12-26 16:55:23,348][105692] Updated weights for policy 0, policy_version 213986 (0.0010) [2023-12-26 16:55:23,413][105692] Updated weights for policy 0, policy_version 213996 (0.0010) [2023-12-26 16:55:23,474][105620] Updated weights for policy 1, policy_version 214573 (0.0007) [2023-12-26 16:55:23,535][105620] Updated weights for policy 1, policy_version 214583 (0.0009) [2023-12-26 16:55:23,596][105620] Updated weights for policy 1, policy_version 214593 (0.0008) [2023-12-26 16:55:24,183][105692] Updated weights for policy 0, policy_version 214006 (0.0010) [2023-12-26 16:55:24,241][105692] Updated weights for policy 0, policy_version 214016 (0.0010) [2023-12-26 16:55:24,300][105692] Updated weights for policy 0, policy_version 214026 (0.0010) [2023-12-26 16:55:24,357][105620] Updated weights for policy 1, policy_version 214603 (0.0008) [2023-12-26 16:55:24,427][105620] Updated weights for policy 1, policy_version 214613 (0.0006) [2023-12-26 16:55:24,495][105620] Updated weights for policy 1, policy_version 214623 (0.0005) [2023-12-26 16:55:24,917][105692] Updated weights for policy 0, policy_version 214036 (0.0009) [2023-12-26 16:55:24,968][105692] Updated weights for policy 0, policy_version 214046 (0.0010) [2023-12-26 16:55:25,030][105692] Updated weights for policy 0, policy_version 214056 (0.0010) [2023-12-26 16:55:25,163][105620] Updated weights for policy 1, policy_version 214633 (0.0007) [2023-12-26 16:55:25,228][105620] Updated weights for policy 1, policy_version 214643 (0.0010) [2023-12-26 16:55:25,272][105620] Updated weights for policy 1, policy_version 214653 (0.0010) [2023-12-26 16:55:25,321][105620] Updated weights for policy 1, policy_version 214663 (0.0009) [2023-12-26 16:55:25,766][105692] Updated weights for policy 0, policy_version 214066 (0.0009) [2023-12-26 16:55:25,822][105692] Updated weights for policy 0, policy_version 214076 (0.0005) [2023-12-26 16:55:25,889][105692] Updated weights for policy 0, policy_version 214086 (0.0005) [2023-12-26 16:55:25,918][105620] Updated weights for policy 1, policy_version 214673 (0.0009) [2023-12-26 16:55:25,945][105692] Updated weights for policy 0, policy_version 214096 (0.0005) [2023-12-26 16:55:25,976][105620] Updated weights for policy 1, policy_version 214683 (0.0005) [2023-12-26 16:55:26,036][105620] Updated weights for policy 1, policy_version 214693 (0.0005) [2023-12-26 16:55:26,062][104569] Fps is (10 sec: 20480.6, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 109789184. Throughput: 0: 9943.7, 1: 9644.1. Samples: 109792088. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:55:26,063][104569] Avg episode reward: [(0, '9260.315'), (1, '9357.833')] [2023-12-26 16:55:26,543][105692] Updated weights for policy 0, policy_version 214106 (0.0006) [2023-12-26 16:55:26,597][105692] Updated weights for policy 0, policy_version 214116 (0.0008) [2023-12-26 16:55:26,633][105620] Updated weights for policy 1, policy_version 214703 (0.0008) [2023-12-26 16:55:26,660][105692] Updated weights for policy 0, policy_version 214126 (0.0009) [2023-12-26 16:55:26,681][105620] Updated weights for policy 1, policy_version 214713 (0.0010) [2023-12-26 16:55:26,725][105620] Updated weights for policy 1, policy_version 214723 (0.0010) [2023-12-26 16:55:27,262][105692] Updated weights for policy 0, policy_version 214136 (0.0005) [2023-12-26 16:55:27,318][105692] Updated weights for policy 0, policy_version 214146 (0.0009) [2023-12-26 16:55:27,370][105692] Updated weights for policy 0, policy_version 214156 (0.0006) [2023-12-26 16:55:27,392][105620] Updated weights for policy 1, policy_version 214733 (0.0008) [2023-12-26 16:55:27,438][105620] Updated weights for policy 1, policy_version 214743 (0.0005) [2023-12-26 16:55:27,488][105620] Updated weights for policy 1, policy_version 214753 (0.0008) [2023-12-26 16:55:27,922][105692] Updated weights for policy 0, policy_version 214166 (0.0005) [2023-12-26 16:55:27,965][105692] Updated weights for policy 0, policy_version 214176 (0.0005) [2023-12-26 16:55:28,028][105692] Updated weights for policy 0, policy_version 214186 (0.0007) [2023-12-26 16:55:28,211][105620] Updated weights for policy 1, policy_version 214763 (0.0010) [2023-12-26 16:55:28,277][105620] Updated weights for policy 1, policy_version 214773 (0.0010) [2023-12-26 16:55:28,338][105620] Updated weights for policy 1, policy_version 214783 (0.0011) [2023-12-26 16:55:28,740][105692] Updated weights for policy 0, policy_version 214196 (0.0008) [2023-12-26 16:55:28,800][105692] Updated weights for policy 0, policy_version 214206 (0.0008) [2023-12-26 16:55:28,864][105692] Updated weights for policy 0, policy_version 214216 (0.0009) [2023-12-26 16:55:29,083][105620] Updated weights for policy 1, policy_version 214793 (0.0009) [2023-12-26 16:55:29,134][105620] Updated weights for policy 1, policy_version 214803 (0.0010) [2023-12-26 16:55:29,191][105620] Updated weights for policy 1, policy_version 214813 (0.0010) [2023-12-26 16:55:29,259][105620] Updated weights for policy 1, policy_version 214823 (0.0010) [2023-12-26 16:55:29,491][105692] Updated weights for policy 0, policy_version 214226 (0.0008) [2023-12-26 16:55:29,556][105692] Updated weights for policy 0, policy_version 214236 (0.0011) [2023-12-26 16:55:29,617][105692] Updated weights for policy 0, policy_version 214246 (0.0010) [2023-12-26 16:55:29,675][105692] Updated weights for policy 0, policy_version 214256 (0.0010) [2023-12-26 16:55:29,950][105620] Updated weights for policy 1, policy_version 214833 (0.0007) [2023-12-26 16:55:30,007][105620] Updated weights for policy 1, policy_version 214843 (0.0008) [2023-12-26 16:55:30,067][105620] Updated weights for policy 1, policy_version 214853 (0.0006) [2023-12-26 16:55:30,415][105692] Updated weights for policy 0, policy_version 214266 (0.0010) [2023-12-26 16:55:30,483][105692] Updated weights for policy 0, policy_version 214276 (0.0010) [2023-12-26 16:55:30,536][105692] Updated weights for policy 0, policy_version 214286 (0.0009) [2023-12-26 16:55:30,686][105620] Updated weights for policy 1, policy_version 214863 (0.0005) [2023-12-26 16:55:30,741][105620] Updated weights for policy 1, policy_version 214873 (0.0006) [2023-12-26 16:55:30,801][105620] Updated weights for policy 1, policy_version 214883 (0.0009) [2023-12-26 16:55:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 109887488. Throughput: 0: 10017.3, 1: 9694.9. Samples: 109855740. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:55:31,062][104569] Avg episode reward: [(0, '9173.963'), (1, '9266.495')] [2023-12-26 16:55:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000214288_54870016.pth... [2023-12-26 16:55:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000214888_55017472.pth... [2023-12-26 16:55:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000213136_54575104.pth [2023-12-26 16:55:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000213768_54730752.pth [2023-12-26 16:55:31,354][105692] Updated weights for policy 0, policy_version 214296 (0.0010) [2023-12-26 16:55:31,413][105620] Updated weights for policy 1, policy_version 214893 (0.0010) [2023-12-26 16:55:31,415][105692] Updated weights for policy 0, policy_version 214306 (0.0007) [2023-12-26 16:55:31,465][105620] Updated weights for policy 1, policy_version 214903 (0.0010) [2023-12-26 16:55:31,475][105692] Updated weights for policy 0, policy_version 214316 (0.0007) [2023-12-26 16:55:31,527][105620] Updated weights for policy 1, policy_version 214913 (0.0010) [2023-12-26 16:55:32,109][105620] Updated weights for policy 1, policy_version 214923 (0.0009) [2023-12-26 16:55:32,158][105620] Updated weights for policy 1, policy_version 214933 (0.0005) [2023-12-26 16:55:32,209][105620] Updated weights for policy 1, policy_version 214943 (0.0006) [2023-12-26 16:55:32,243][105692] Updated weights for policy 0, policy_version 214326 (0.0007) [2023-12-26 16:55:32,301][105692] Updated weights for policy 0, policy_version 214336 (0.0007) [2023-12-26 16:55:32,350][105692] Updated weights for policy 0, policy_version 214346 (0.0008) [2023-12-26 16:55:32,928][105620] Updated weights for policy 1, policy_version 214953 (0.0007) [2023-12-26 16:55:32,986][105620] Updated weights for policy 1, policy_version 214963 (0.0009) [2023-12-26 16:55:33,042][105620] Updated weights for policy 1, policy_version 214973 (0.0006) [2023-12-26 16:55:33,086][105692] Updated weights for policy 0, policy_version 214356 (0.0008) [2023-12-26 16:55:33,105][105620] Updated weights for policy 1, policy_version 214983 (0.0007) [2023-12-26 16:55:33,142][105692] Updated weights for policy 0, policy_version 214366 (0.0009) [2023-12-26 16:55:33,193][105692] Updated weights for policy 0, policy_version 214376 (0.0010) [2023-12-26 16:55:33,817][105692] Updated weights for policy 0, policy_version 214386 (0.0010) [2023-12-26 16:55:33,849][105620] Updated weights for policy 1, policy_version 214993 (0.0009) [2023-12-26 16:55:33,872][105692] Updated weights for policy 0, policy_version 214396 (0.0011) [2023-12-26 16:55:33,913][105620] Updated weights for policy 1, policy_version 215003 (0.0006) [2023-12-26 16:55:33,927][105692] Updated weights for policy 0, policy_version 214406 (0.0007) [2023-12-26 16:55:33,962][105620] Updated weights for policy 1, policy_version 215013 (0.0007) [2023-12-26 16:55:33,983][105692] Updated weights for policy 0, policy_version 214416 (0.0011) [2023-12-26 16:55:34,654][105620] Updated weights for policy 1, policy_version 215023 (0.0008) [2023-12-26 16:55:34,710][105620] Updated weights for policy 1, policy_version 215033 (0.0007) [2023-12-26 16:55:34,720][105692] Updated weights for policy 0, policy_version 214426 (0.0011) [2023-12-26 16:55:34,759][105620] Updated weights for policy 1, policy_version 215043 (0.0008) [2023-12-26 16:55:34,771][105692] Updated weights for policy 0, policy_version 214436 (0.0010) [2023-12-26 16:55:34,832][105692] Updated weights for policy 0, policy_version 214446 (0.0010) [2023-12-26 16:55:35,527][105620] Updated weights for policy 1, policy_version 215053 (0.0008) [2023-12-26 16:55:35,574][105692] Updated weights for policy 0, policy_version 214456 (0.0011) [2023-12-26 16:55:35,580][105620] Updated weights for policy 1, policy_version 215063 (0.0007) [2023-12-26 16:55:35,634][105620] Updated weights for policy 1, policy_version 215073 (0.0006) [2023-12-26 16:55:35,635][105692] Updated weights for policy 0, policy_version 214466 (0.0010) [2023-12-26 16:55:35,693][105692] Updated weights for policy 0, policy_version 214476 (0.0010) [2023-12-26 16:55:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 109985792. Throughput: 0: 9987.8, 1: 9729.3. Samples: 109975416. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:55:36,063][104569] Avg episode reward: [(0, '9173.573'), (1, '9266.020')] [2023-12-26 16:55:36,375][105692] Updated weights for policy 0, policy_version 214486 (0.0007) [2023-12-26 16:55:36,435][105620] Updated weights for policy 1, policy_version 215083 (0.0008) [2023-12-26 16:55:36,438][105692] Updated weights for policy 0, policy_version 214496 (0.0005) [2023-12-26 16:55:36,487][105620] Updated weights for policy 1, policy_version 215093 (0.0008) [2023-12-26 16:55:36,505][105692] Updated weights for policy 0, policy_version 214506 (0.0006) [2023-12-26 16:55:36,545][105620] Updated weights for policy 1, policy_version 215103 (0.0009) [2023-12-26 16:55:37,158][105692] Updated weights for policy 0, policy_version 214516 (0.0008) [2023-12-26 16:55:37,203][105692] Updated weights for policy 0, policy_version 214526 (0.0010) [2023-12-26 16:55:37,259][105692] Updated weights for policy 0, policy_version 214536 (0.0011) [2023-12-26 16:55:37,334][105620] Updated weights for policy 1, policy_version 215113 (0.0009) [2023-12-26 16:55:37,395][105620] Updated weights for policy 1, policy_version 215123 (0.0008) [2023-12-26 16:55:37,451][105620] Updated weights for policy 1, policy_version 215133 (0.0008) [2023-12-26 16:55:37,507][105620] Updated weights for policy 1, policy_version 215143 (0.0008) [2023-12-26 16:55:38,022][105692] Updated weights for policy 0, policy_version 214546 (0.0011) [2023-12-26 16:55:38,081][105692] Updated weights for policy 0, policy_version 214556 (0.0011) [2023-12-26 16:55:38,129][105692] Updated weights for policy 0, policy_version 214566 (0.0010) [2023-12-26 16:55:38,181][105692] Updated weights for policy 0, policy_version 214576 (0.0010) [2023-12-26 16:55:38,274][105620] Updated weights for policy 1, policy_version 215153 (0.0008) [2023-12-26 16:55:38,338][105620] Updated weights for policy 1, policy_version 215163 (0.0008) [2023-12-26 16:55:38,407][105620] Updated weights for policy 1, policy_version 215173 (0.0008) [2023-12-26 16:55:38,867][105692] Updated weights for policy 0, policy_version 214586 (0.0010) [2023-12-26 16:55:38,922][105692] Updated weights for policy 0, policy_version 214596 (0.0011) [2023-12-26 16:55:38,982][105692] Updated weights for policy 0, policy_version 214606 (0.0006) [2023-12-26 16:55:39,201][105620] Updated weights for policy 1, policy_version 215183 (0.0008) [2023-12-26 16:55:39,265][105620] Updated weights for policy 1, policy_version 215193 (0.0007) [2023-12-26 16:55:39,326][105620] Updated weights for policy 1, policy_version 215203 (0.0009) [2023-12-26 16:55:39,672][105692] Updated weights for policy 0, policy_version 214616 (0.0008) [2023-12-26 16:55:39,727][105692] Updated weights for policy 0, policy_version 214626 (0.0009) [2023-12-26 16:55:39,794][105692] Updated weights for policy 0, policy_version 214636 (0.0009) [2023-12-26 16:55:40,079][105620] Updated weights for policy 1, policy_version 215213 (0.0008) [2023-12-26 16:55:40,134][105620] Updated weights for policy 1, policy_version 215223 (0.0009) [2023-12-26 16:55:40,182][105620] Updated weights for policy 1, policy_version 215233 (0.0009) [2023-12-26 16:55:40,591][105692] Updated weights for policy 0, policy_version 214646 (0.0009) [2023-12-26 16:55:40,653][105692] Updated weights for policy 0, policy_version 214656 (0.0009) [2023-12-26 16:55:40,711][105692] Updated weights for policy 0, policy_version 214666 (0.0008) [2023-12-26 16:55:40,920][105620] Updated weights for policy 1, policy_version 215243 (0.0009) [2023-12-26 16:55:40,971][105620] Updated weights for policy 1, policy_version 215253 (0.0009) [2023-12-26 16:55:41,019][105620] Updated weights for policy 1, policy_version 215263 (0.0009) [2023-12-26 16:55:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 110075904. Throughput: 0: 10030.1, 1: 9582.1. Samples: 110088060. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:55:41,062][104569] Avg episode reward: [(0, '9263.042'), (1, '9356.769')] [2023-12-26 16:55:41,537][105692] Updated weights for policy 0, policy_version 214676 (0.0010) [2023-12-26 16:55:41,595][105692] Updated weights for policy 0, policy_version 214686 (0.0008) [2023-12-26 16:55:41,672][105692] Updated weights for policy 0, policy_version 214696 (0.0009) [2023-12-26 16:55:41,795][105620] Updated weights for policy 1, policy_version 215273 (0.0009) [2023-12-26 16:55:41,859][105620] Updated weights for policy 1, policy_version 215283 (0.0009) [2023-12-26 16:55:41,911][105620] Updated weights for policy 1, policy_version 215293 (0.0009) [2023-12-26 16:55:41,959][105620] Updated weights for policy 1, policy_version 215303 (0.0008) [2023-12-26 16:55:42,502][105692] Updated weights for policy 0, policy_version 214706 (0.0009) [2023-12-26 16:55:42,554][105692] Updated weights for policy 0, policy_version 214716 (0.0005) [2023-12-26 16:55:42,621][105692] Updated weights for policy 0, policy_version 214726 (0.0008) [2023-12-26 16:55:42,652][105620] Updated weights for policy 1, policy_version 215313 (0.0007) [2023-12-26 16:55:42,683][105692] Updated weights for policy 0, policy_version 214736 (0.0008) [2023-12-26 16:55:42,715][105620] Updated weights for policy 1, policy_version 215323 (0.0008) [2023-12-26 16:55:42,777][105620] Updated weights for policy 1, policy_version 215333 (0.0009) [2023-12-26 16:55:43,329][105692] Updated weights for policy 0, policy_version 214746 (0.0009) [2023-12-26 16:55:43,377][105692] Updated weights for policy 0, policy_version 214756 (0.0009) [2023-12-26 16:55:43,434][105692] Updated weights for policy 0, policy_version 214766 (0.0008) [2023-12-26 16:55:43,545][105620] Updated weights for policy 1, policy_version 215343 (0.0009) [2023-12-26 16:55:43,607][105620] Updated weights for policy 1, policy_version 215353 (0.0009) [2023-12-26 16:55:43,664][105620] Updated weights for policy 1, policy_version 215363 (0.0009) [2023-12-26 16:55:44,173][105692] Updated weights for policy 0, policy_version 214776 (0.0006) [2023-12-26 16:55:44,229][105692] Updated weights for policy 0, policy_version 214786 (0.0007) [2023-12-26 16:55:44,287][105692] Updated weights for policy 0, policy_version 214796 (0.0009) [2023-12-26 16:55:44,435][105620] Updated weights for policy 1, policy_version 215373 (0.0008) [2023-12-26 16:55:44,481][105620] Updated weights for policy 1, policy_version 215383 (0.0009) [2023-12-26 16:55:44,528][105620] Updated weights for policy 1, policy_version 215393 (0.0009) [2023-12-26 16:55:45,001][105692] Updated weights for policy 0, policy_version 214806 (0.0009) [2023-12-26 16:55:45,060][105692] Updated weights for policy 0, policy_version 214816 (0.0008) [2023-12-26 16:55:45,108][105692] Updated weights for policy 0, policy_version 214826 (0.0009) [2023-12-26 16:55:45,321][105620] Updated weights for policy 1, policy_version 215403 (0.0009) [2023-12-26 16:55:45,380][105620] Updated weights for policy 1, policy_version 215413 (0.0009) [2023-12-26 16:55:45,439][105620] Updated weights for policy 1, policy_version 215423 (0.0009) [2023-12-26 16:55:45,841][105692] Updated weights for policy 0, policy_version 214836 (0.0008) [2023-12-26 16:55:45,899][105692] Updated weights for policy 0, policy_version 214846 (0.0006) [2023-12-26 16:55:45,955][105692] Updated weights for policy 0, policy_version 214856 (0.0005) [2023-12-26 16:55:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19660.7, 300 sec: 19605.2). Total num frames: 110174208. Throughput: 0: 9892.9, 1: 9551.7. Samples: 110142708. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:55:46,063][104569] Avg episode reward: [(0, '9262.098'), (1, '9356.452')] [2023-12-26 16:55:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000214864_55017472.pth... [2023-12-26 16:55:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000215432_55156736.pth... [2023-12-26 16:55:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000214312_54870016.pth [2023-12-26 16:55:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000213680_54714368.pth [2023-12-26 16:55:46,259][105620] Updated weights for policy 1, policy_version 215433 (0.0008) [2023-12-26 16:55:46,318][105620] Updated weights for policy 1, policy_version 215443 (0.0009) [2023-12-26 16:55:46,373][105620] Updated weights for policy 1, policy_version 215453 (0.0008) [2023-12-26 16:55:46,428][105620] Updated weights for policy 1, policy_version 215463 (0.0008) [2023-12-26 16:55:46,617][105692] Updated weights for policy 0, policy_version 214866 (0.0010) [2023-12-26 16:55:46,667][105692] Updated weights for policy 0, policy_version 214876 (0.0009) [2023-12-26 16:55:46,727][105692] Updated weights for policy 0, policy_version 214886 (0.0009) [2023-12-26 16:55:46,784][105692] Updated weights for policy 0, policy_version 214896 (0.0007) [2023-12-26 16:55:47,147][105620] Updated weights for policy 1, policy_version 215473 (0.0006) [2023-12-26 16:55:47,200][105620] Updated weights for policy 1, policy_version 215483 (0.0008) [2023-12-26 16:55:47,245][105620] Updated weights for policy 1, policy_version 215493 (0.0008) [2023-12-26 16:55:47,455][105692] Updated weights for policy 0, policy_version 214906 (0.0008) [2023-12-26 16:55:47,507][105692] Updated weights for policy 0, policy_version 214916 (0.0010) [2023-12-26 16:55:47,565][105692] Updated weights for policy 0, policy_version 214926 (0.0010) [2023-12-26 16:55:47,905][105620] Updated weights for policy 1, policy_version 215503 (0.0008) [2023-12-26 16:55:47,962][105620] Updated weights for policy 1, policy_version 215513 (0.0010) [2023-12-26 16:55:48,014][105620] Updated weights for policy 1, policy_version 215523 (0.0010) [2023-12-26 16:55:48,166][105692] Updated weights for policy 0, policy_version 214936 (0.0010) [2023-12-26 16:55:48,229][105692] Updated weights for policy 0, policy_version 214946 (0.0006) [2023-12-26 16:55:48,290][105692] Updated weights for policy 0, policy_version 214956 (0.0010) [2023-12-26 16:55:48,743][105620] Updated weights for policy 1, policy_version 215533 (0.0011) [2023-12-26 16:55:48,806][105620] Updated weights for policy 1, policy_version 215543 (0.0011) [2023-12-26 16:55:48,868][105620] Updated weights for policy 1, policy_version 215553 (0.0011) [2023-12-26 16:55:48,995][105692] Updated weights for policy 0, policy_version 214966 (0.0007) [2023-12-26 16:55:49,056][105692] Updated weights for policy 0, policy_version 214976 (0.0005) [2023-12-26 16:55:49,114][105692] Updated weights for policy 0, policy_version 214986 (0.0006) [2023-12-26 16:55:49,648][105620] Updated weights for policy 1, policy_version 215563 (0.0009) [2023-12-26 16:55:49,723][105620] Updated weights for policy 1, policy_version 215573 (0.0006) [2023-12-26 16:55:49,762][105692] Updated weights for policy 0, policy_version 214996 (0.0006) [2023-12-26 16:55:49,782][105620] Updated weights for policy 1, policy_version 215583 (0.0008) [2023-12-26 16:55:49,813][105692] Updated weights for policy 0, policy_version 215006 (0.0007) [2023-12-26 16:55:49,879][105692] Updated weights for policy 0, policy_version 215016 (0.0008) [2023-12-26 16:55:50,394][105620] Updated weights for policy 1, policy_version 215593 (0.0008) [2023-12-26 16:55:50,462][105620] Updated weights for policy 1, policy_version 215603 (0.0006) [2023-12-26 16:55:50,521][105620] Updated weights for policy 1, policy_version 215613 (0.0009) [2023-12-26 16:55:50,591][105620] Updated weights for policy 1, policy_version 215623 (0.0009) [2023-12-26 16:55:50,639][105692] Updated weights for policy 0, policy_version 215026 (0.0007) [2023-12-26 16:55:50,701][105692] Updated weights for policy 0, policy_version 215036 (0.0009) [2023-12-26 16:55:50,761][105692] Updated weights for policy 0, policy_version 215046 (0.0005) [2023-12-26 16:55:50,819][105692] Updated weights for policy 0, policy_version 215056 (0.0010) [2023-12-26 16:55:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 110272512. Throughput: 0: 9907.7, 1: 9493.6. Samples: 110261656. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 16:55:51,062][104569] Avg episode reward: [(0, '8746.874'), (1, '9356.426')] [2023-12-26 16:55:51,288][105620] Updated weights for policy 1, policy_version 215633 (0.0009) [2023-12-26 16:55:51,352][105620] Updated weights for policy 1, policy_version 215643 (0.0009) [2023-12-26 16:55:51,412][105620] Updated weights for policy 1, policy_version 215653 (0.0008) [2023-12-26 16:55:51,463][105692] Updated weights for policy 0, policy_version 215066 (0.0009) [2023-12-26 16:55:51,521][105692] Updated weights for policy 0, policy_version 215076 (0.0010) [2023-12-26 16:55:51,581][105692] Updated weights for policy 0, policy_version 215086 (0.0010) [2023-12-26 16:55:52,136][105620] Updated weights for policy 1, policy_version 215663 (0.0008) [2023-12-26 16:55:52,192][105620] Updated weights for policy 1, policy_version 215673 (0.0008) [2023-12-26 16:55:52,250][105620] Updated weights for policy 1, policy_version 215683 (0.0008) [2023-12-26 16:55:52,340][105692] Updated weights for policy 0, policy_version 215096 (0.0008) [2023-12-26 16:55:52,413][105692] Updated weights for policy 0, policy_version 215106 (0.0011) [2023-12-26 16:55:52,475][105692] Updated weights for policy 0, policy_version 215116 (0.0010) [2023-12-26 16:55:53,048][105692] Updated weights for policy 0, policy_version 215126 (0.0007) [2023-12-26 16:55:53,082][105620] Updated weights for policy 1, policy_version 215693 (0.0008) [2023-12-26 16:55:53,092][105692] Updated weights for policy 0, policy_version 215136 (0.0005) [2023-12-26 16:55:53,137][105620] Updated weights for policy 1, policy_version 215703 (0.0009) [2023-12-26 16:55:53,139][105692] Updated weights for policy 0, policy_version 215146 (0.0006) [2023-12-26 16:55:53,196][105620] Updated weights for policy 1, policy_version 215713 (0.0009) [2023-12-26 16:55:53,745][105692] Updated weights for policy 0, policy_version 215156 (0.0007) [2023-12-26 16:55:53,797][105692] Updated weights for policy 0, policy_version 215166 (0.0008) [2023-12-26 16:55:53,850][105692] Updated weights for policy 0, policy_version 215176 (0.0008) [2023-12-26 16:55:53,983][105620] Updated weights for policy 1, policy_version 215723 (0.0010) [2023-12-26 16:55:54,027][105620] Updated weights for policy 1, policy_version 215733 (0.0010) [2023-12-26 16:55:54,072][105620] Updated weights for policy 1, policy_version 215743 (0.0010) [2023-12-26 16:55:54,626][105692] Updated weights for policy 0, policy_version 215186 (0.0008) [2023-12-26 16:55:54,688][105692] Updated weights for policy 0, policy_version 215196 (0.0008) [2023-12-26 16:55:54,739][105692] Updated weights for policy 0, policy_version 215206 (0.0008) [2023-12-26 16:55:54,801][105692] Updated weights for policy 0, policy_version 215216 (0.0008) [2023-12-26 16:55:54,841][105620] Updated weights for policy 1, policy_version 215753 (0.0010) [2023-12-26 16:55:54,905][105620] Updated weights for policy 1, policy_version 215763 (0.0008) [2023-12-26 16:55:54,967][105620] Updated weights for policy 1, policy_version 215773 (0.0006) [2023-12-26 16:55:55,019][105620] Updated weights for policy 1, policy_version 215783 (0.0005) [2023-12-26 16:55:55,551][105620] Updated weights for policy 1, policy_version 215793 (0.0005) [2023-12-26 16:55:55,617][105620] Updated weights for policy 1, policy_version 215803 (0.0006) [2023-12-26 16:55:55,666][105692] Updated weights for policy 0, policy_version 215226 (0.0007) [2023-12-26 16:55:55,678][105620] Updated weights for policy 1, policy_version 215813 (0.0010) [2023-12-26 16:55:55,722][105692] Updated weights for policy 0, policy_version 215236 (0.0009) [2023-12-26 16:55:55,770][105692] Updated weights for policy 0, policy_version 215246 (0.0008) [2023-12-26 16:55:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 110370816. Throughput: 0: 9900.9, 1: 9590.4. Samples: 110379200. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-12-26 16:55:56,063][104569] Avg episode reward: [(0, '8483.592'), (1, '9356.559')] [2023-12-26 16:55:56,282][105620] Updated weights for policy 1, policy_version 215823 (0.0008) [2023-12-26 16:55:56,334][105620] Updated weights for policy 1, policy_version 215833 (0.0009) [2023-12-26 16:55:56,398][105620] Updated weights for policy 1, policy_version 215843 (0.0006) [2023-12-26 16:55:56,430][105692] Updated weights for policy 0, policy_version 215256 (0.0008) [2023-12-26 16:55:56,488][105692] Updated weights for policy 0, policy_version 215266 (0.0005) [2023-12-26 16:55:56,537][105692] Updated weights for policy 0, policy_version 215276 (0.0005) [2023-12-26 16:55:56,948][105620] Updated weights for policy 1, policy_version 215853 (0.0005) [2023-12-26 16:55:57,001][105620] Updated weights for policy 1, policy_version 215863 (0.0005) [2023-12-26 16:55:57,063][105620] Updated weights for policy 1, policy_version 215873 (0.0005) [2023-12-26 16:55:57,080][105692] Updated weights for policy 0, policy_version 215286 (0.0007) [2023-12-26 16:55:57,140][105692] Updated weights for policy 0, policy_version 215297 (0.0010) [2023-12-26 16:55:57,193][105692] Updated weights for policy 0, policy_version 215307 (0.0009) [2023-12-26 16:55:57,573][105620] Updated weights for policy 1, policy_version 215883 (0.0005) [2023-12-26 16:55:57,621][105620] Updated weights for policy 1, policy_version 215893 (0.0005) [2023-12-26 16:55:57,678][105620] Updated weights for policy 1, policy_version 215903 (0.0005) [2023-12-26 16:55:58,051][105692] Updated weights for policy 0, policy_version 215317 (0.0008) [2023-12-26 16:55:58,109][105692] Updated weights for policy 0, policy_version 215327 (0.0008) [2023-12-26 16:55:58,169][105692] Updated weights for policy 0, policy_version 215337 (0.0008) [2023-12-26 16:55:58,238][105620] Updated weights for policy 1, policy_version 215913 (0.0005) [2023-12-26 16:55:58,308][105620] Updated weights for policy 1, policy_version 215923 (0.0009) [2023-12-26 16:55:58,378][105620] Updated weights for policy 1, policy_version 215933 (0.0007) [2023-12-26 16:55:58,443][105620] Updated weights for policy 1, policy_version 215943 (0.0009) [2023-12-26 16:55:59,037][105692] Updated weights for policy 0, policy_version 215347 (0.0008) [2023-12-26 16:55:59,100][105692] Updated weights for policy 0, policy_version 215357 (0.0009) [2023-12-26 16:55:59,158][105692] Updated weights for policy 0, policy_version 215367 (0.0009) [2023-12-26 16:55:59,197][105620] Updated weights for policy 1, policy_version 215953 (0.0008) [2023-12-26 16:55:59,267][105620] Updated weights for policy 1, policy_version 215963 (0.0007) [2023-12-26 16:55:59,329][105620] Updated weights for policy 1, policy_version 215973 (0.0009) [2023-12-26 16:55:59,894][105692] Updated weights for policy 0, policy_version 215377 (0.0006) [2023-12-26 16:55:59,961][105692] Updated weights for policy 0, policy_version 215387 (0.0009) [2023-12-26 16:56:00,024][105692] Updated weights for policy 0, policy_version 215397 (0.0008) [2023-12-26 16:56:00,063][105620] Updated weights for policy 1, policy_version 215983 (0.0008) [2023-12-26 16:56:00,069][105692] Updated weights for policy 0, policy_version 215407 (0.0005) [2023-12-26 16:56:00,110][105620] Updated weights for policy 1, policy_version 215993 (0.0008) [2023-12-26 16:56:00,171][105620] Updated weights for policy 1, policy_version 216003 (0.0008) [2023-12-26 16:56:00,763][105620] Updated weights for policy 1, policy_version 216013 (0.0006) [2023-12-26 16:56:00,809][105620] Updated weights for policy 1, policy_version 216023 (0.0005) [2023-12-26 16:56:00,878][105620] Updated weights for policy 1, policy_version 216033 (0.0005) [2023-12-26 16:56:00,906][105692] Updated weights for policy 0, policy_version 215417 (0.0009) [2023-12-26 16:56:00,958][105692] Updated weights for policy 0, policy_version 215427 (0.0009) [2023-12-26 16:56:01,011][105692] Updated weights for policy 0, policy_version 215437 (0.0009) [2023-12-26 16:56:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 110477312. Throughput: 0: 9944.1, 1: 9749.6. Samples: 110442284. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-12-26 16:56:01,063][104569] Avg episode reward: [(0, '8662.593'), (1, '9265.263')] [2023-12-26 16:56:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000215440_55164928.pth... [2023-12-26 16:56:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000216040_55312384.pth... [2023-12-26 16:56:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000214888_55017472.pth [2023-12-26 16:56:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000214288_54870016.pth [2023-12-26 16:56:01,510][105620] Updated weights for policy 1, policy_version 216043 (0.0007) [2023-12-26 16:56:01,568][105620] Updated weights for policy 1, policy_version 216053 (0.0010) [2023-12-26 16:56:01,624][105620] Updated weights for policy 1, policy_version 216063 (0.0009) [2023-12-26 16:56:01,842][105692] Updated weights for policy 0, policy_version 215447 (0.0009) [2023-12-26 16:56:01,895][105692] Updated weights for policy 0, policy_version 215457 (0.0009) [2023-12-26 16:56:01,951][105692] Updated weights for policy 0, policy_version 215468 (0.0010) [2023-12-26 16:56:02,297][105620] Updated weights for policy 1, policy_version 216073 (0.0006) [2023-12-26 16:56:02,357][105620] Updated weights for policy 1, policy_version 216083 (0.0010) [2023-12-26 16:56:02,427][105620] Updated weights for policy 1, policy_version 216093 (0.0007) [2023-12-26 16:56:02,490][105620] Updated weights for policy 1, policy_version 216103 (0.0007) [2023-12-26 16:56:02,730][105692] Updated weights for policy 0, policy_version 215478 (0.0007) [2023-12-26 16:56:02,779][105692] Updated weights for policy 0, policy_version 215488 (0.0006) [2023-12-26 16:56:02,841][105692] Updated weights for policy 0, policy_version 215498 (0.0008) [2023-12-26 16:56:03,179][105620] Updated weights for policy 1, policy_version 216113 (0.0008) [2023-12-26 16:56:03,236][105620] Updated weights for policy 1, policy_version 216123 (0.0008) [2023-12-26 16:56:03,294][105620] Updated weights for policy 1, policy_version 216133 (0.0009) [2023-12-26 16:56:03,474][105692] Updated weights for policy 0, policy_version 215508 (0.0008) [2023-12-26 16:56:03,531][105692] Updated weights for policy 0, policy_version 215518 (0.0007) [2023-12-26 16:56:03,602][105692] Updated weights for policy 0, policy_version 215528 (0.0010) [2023-12-26 16:56:03,999][105620] Updated weights for policy 1, policy_version 216143 (0.0007) [2023-12-26 16:56:04,063][105620] Updated weights for policy 1, policy_version 216153 (0.0007) [2023-12-26 16:56:04,125][105620] Updated weights for policy 1, policy_version 216163 (0.0008) [2023-12-26 16:56:04,292][105692] Updated weights for policy 0, policy_version 215538 (0.0008) [2023-12-26 16:56:04,365][105692] Updated weights for policy 0, policy_version 215548 (0.0008) [2023-12-26 16:56:04,430][105692] Updated weights for policy 0, policy_version 215558 (0.0008) [2023-12-26 16:56:04,489][105692] Updated weights for policy 0, policy_version 215568 (0.0009) [2023-12-26 16:56:04,801][105620] Updated weights for policy 1, policy_version 216173 (0.0006) [2023-12-26 16:56:04,857][105620] Updated weights for policy 1, policy_version 216183 (0.0007) [2023-12-26 16:56:04,913][105620] Updated weights for policy 1, policy_version 216193 (0.0008) [2023-12-26 16:56:05,208][105692] Updated weights for policy 0, policy_version 215578 (0.0010) [2023-12-26 16:56:05,265][105692] Updated weights for policy 0, policy_version 215588 (0.0010) [2023-12-26 16:56:05,322][105692] Updated weights for policy 0, policy_version 215600 (0.0009) [2023-12-26 16:56:05,568][105620] Updated weights for policy 1, policy_version 216203 (0.0007) [2023-12-26 16:56:05,627][105620] Updated weights for policy 1, policy_version 216213 (0.0005) [2023-12-26 16:56:05,694][105620] Updated weights for policy 1, policy_version 216223 (0.0005) [2023-12-26 16:56:06,005][105692] Updated weights for policy 0, policy_version 215610 (0.0009) [2023-12-26 16:56:06,055][105692] Updated weights for policy 0, policy_version 215620 (0.0009) [2023-12-26 16:56:06,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 110567424. Throughput: 0: 9809.9, 1: 9850.5. Samples: 110558952. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-12-26 16:56:06,062][104569] Avg episode reward: [(0, '8824.756'), (1, '9264.963')] [2023-12-26 16:56:06,106][105692] Updated weights for policy 0, policy_version 215630 (0.0008) [2023-12-26 16:56:06,313][105620] Updated weights for policy 1, policy_version 216233 (0.0008) [2023-12-26 16:56:06,387][105620] Updated weights for policy 1, policy_version 216243 (0.0006) [2023-12-26 16:56:06,453][105620] Updated weights for policy 1, policy_version 216253 (0.0008) [2023-12-26 16:56:06,515][105620] Updated weights for policy 1, policy_version 216263 (0.0005) [2023-12-26 16:56:06,882][105692] Updated weights for policy 0, policy_version 215640 (0.0007) [2023-12-26 16:56:06,936][105692] Updated weights for policy 0, policy_version 215650 (0.0005) [2023-12-26 16:56:06,992][105692] Updated weights for policy 0, policy_version 215660 (0.0005) [2023-12-26 16:56:07,178][105620] Updated weights for policy 1, policy_version 216273 (0.0008) [2023-12-26 16:56:07,230][105620] Updated weights for policy 1, policy_version 216283 (0.0009) [2023-12-26 16:56:07,277][105620] Updated weights for policy 1, policy_version 216293 (0.0009) [2023-12-26 16:56:07,653][105692] Updated weights for policy 0, policy_version 215670 (0.0008) [2023-12-26 16:56:07,711][105692] Updated weights for policy 0, policy_version 215680 (0.0009) [2023-12-26 16:56:07,763][105692] Updated weights for policy 0, policy_version 215690 (0.0009) [2023-12-26 16:56:08,045][105620] Updated weights for policy 1, policy_version 216303 (0.0009) [2023-12-26 16:56:08,098][105620] Updated weights for policy 1, policy_version 216313 (0.0008) [2023-12-26 16:56:08,155][105620] Updated weights for policy 1, policy_version 216323 (0.0009) [2023-12-26 16:56:08,513][105692] Updated weights for policy 0, policy_version 215700 (0.0009) [2023-12-26 16:56:08,576][105692] Updated weights for policy 0, policy_version 215710 (0.0009) [2023-12-26 16:56:08,641][105692] Updated weights for policy 0, policy_version 215720 (0.0008) [2023-12-26 16:56:08,897][105620] Updated weights for policy 1, policy_version 216333 (0.0008) [2023-12-26 16:56:08,963][105620] Updated weights for policy 1, policy_version 216343 (0.0010) [2023-12-26 16:56:09,030][105620] Updated weights for policy 1, policy_version 216353 (0.0009) [2023-12-26 16:56:09,278][105692] Updated weights for policy 0, policy_version 215730 (0.0009) [2023-12-26 16:56:09,342][105692] Updated weights for policy 0, policy_version 215740 (0.0006) [2023-12-26 16:56:09,407][105692] Updated weights for policy 0, policy_version 215750 (0.0008) [2023-12-26 16:56:09,472][105692] Updated weights for policy 0, policy_version 215760 (0.0009) [2023-12-26 16:56:09,807][105620] Updated weights for policy 1, policy_version 216363 (0.0009) [2023-12-26 16:56:09,879][105620] Updated weights for policy 1, policy_version 216373 (0.0010) [2023-12-26 16:56:09,947][105620] Updated weights for policy 1, policy_version 216383 (0.0009) [2023-12-26 16:56:10,201][105692] Updated weights for policy 0, policy_version 215770 (0.0009) [2023-12-26 16:56:10,260][105692] Updated weights for policy 0, policy_version 215780 (0.0009) [2023-12-26 16:56:10,324][105692] Updated weights for policy 0, policy_version 215790 (0.0009) [2023-12-26 16:56:10,701][105620] Updated weights for policy 1, policy_version 216393 (0.0009) [2023-12-26 16:56:10,762][105620] Updated weights for policy 1, policy_version 216403 (0.0009) [2023-12-26 16:56:10,818][105620] Updated weights for policy 1, policy_version 216413 (0.0005) [2023-12-26 16:56:10,874][105620] Updated weights for policy 1, policy_version 216423 (0.0007) [2023-12-26 16:56:10,996][105692] Updated weights for policy 0, policy_version 215800 (0.0008) [2023-12-26 16:56:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 110665728. Throughput: 0: 9780.4, 1: 9852.2. Samples: 110675556. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-12-26 16:56:11,062][104569] Avg episode reward: [(0, '8996.839'), (1, '9266.467')] [2023-12-26 16:56:11,064][105692] Updated weights for policy 0, policy_version 215810 (0.0008) [2023-12-26 16:56:11,134][105692] Updated weights for policy 0, policy_version 215820 (0.0008) [2023-12-26 16:56:11,630][105620] Updated weights for policy 1, policy_version 216433 (0.0007) [2023-12-26 16:56:11,699][105620] Updated weights for policy 1, policy_version 216443 (0.0008) [2023-12-26 16:56:11,770][105620] Updated weights for policy 1, policy_version 216453 (0.0008) [2023-12-26 16:56:11,942][105692] Updated weights for policy 0, policy_version 215830 (0.0010) [2023-12-26 16:56:11,996][105692] Updated weights for policy 0, policy_version 215840 (0.0008) [2023-12-26 16:56:12,053][105692] Updated weights for policy 0, policy_version 215850 (0.0005) [2023-12-26 16:56:12,519][105620] Updated weights for policy 1, policy_version 216463 (0.0006) [2023-12-26 16:56:12,578][105620] Updated weights for policy 1, policy_version 216473 (0.0006) [2023-12-26 16:56:12,637][105620] Updated weights for policy 1, policy_version 216483 (0.0006) [2023-12-26 16:56:12,726][105692] Updated weights for policy 0, policy_version 215860 (0.0007) [2023-12-26 16:56:12,787][105692] Updated weights for policy 0, policy_version 215870 (0.0005) [2023-12-26 16:56:12,855][105692] Updated weights for policy 0, policy_version 215880 (0.0005) [2023-12-26 16:56:13,341][105692] Updated weights for policy 0, policy_version 215890 (0.0005) [2023-12-26 16:56:13,367][105620] Updated weights for policy 1, policy_version 216493 (0.0009) [2023-12-26 16:56:13,406][105692] Updated weights for policy 0, policy_version 215900 (0.0006) [2023-12-26 16:56:13,418][105620] Updated weights for policy 1, policy_version 216503 (0.0009) [2023-12-26 16:56:13,462][105692] Updated weights for policy 0, policy_version 215910 (0.0009) [2023-12-26 16:56:13,476][105620] Updated weights for policy 1, policy_version 216513 (0.0007) [2023-12-26 16:56:13,524][105692] Updated weights for policy 0, policy_version 215920 (0.0010) [2023-12-26 16:56:14,251][105692] Updated weights for policy 0, policy_version 215930 (0.0010) [2023-12-26 16:56:14,281][105620] Updated weights for policy 1, policy_version 216523 (0.0006) [2023-12-26 16:56:14,302][105692] Updated weights for policy 0, policy_version 215940 (0.0010) [2023-12-26 16:56:14,336][105620] Updated weights for policy 1, policy_version 216533 (0.0005) [2023-12-26 16:56:14,360][105692] Updated weights for policy 0, policy_version 215950 (0.0010) [2023-12-26 16:56:14,395][105620] Updated weights for policy 1, policy_version 216543 (0.0007) [2023-12-26 16:56:15,074][105620] Updated weights for policy 1, policy_version 216553 (0.0008) [2023-12-26 16:56:15,096][105692] Updated weights for policy 0, policy_version 215960 (0.0010) [2023-12-26 16:56:15,138][105620] Updated weights for policy 1, policy_version 216563 (0.0006) [2023-12-26 16:56:15,148][105692] Updated weights for policy 0, policy_version 215970 (0.0011) [2023-12-26 16:56:15,202][105620] Updated weights for policy 1, policy_version 216573 (0.0007) [2023-12-26 16:56:15,216][105692] Updated weights for policy 0, policy_version 215980 (0.0007) [2023-12-26 16:56:15,261][105620] Updated weights for policy 1, policy_version 216583 (0.0009) [2023-12-26 16:56:15,886][105620] Updated weights for policy 1, policy_version 216593 (0.0005) [2023-12-26 16:56:15,939][105620] Updated weights for policy 1, policy_version 216603 (0.0007) [2023-12-26 16:56:15,951][105692] Updated weights for policy 0, policy_version 215990 (0.0011) [2023-12-26 16:56:15,992][105620] Updated weights for policy 1, policy_version 216613 (0.0009) [2023-12-26 16:56:16,010][105692] Updated weights for policy 0, policy_version 216000 (0.0010) [2023-12-26 16:56:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.9, 300 sec: 19633.0). Total num frames: 110764032. Throughput: 0: 9746.7, 1: 9776.1. Samples: 110734268. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-12-26 16:56:16,062][104569] Avg episode reward: [(0, '9180.273'), (1, '9266.572')] [2023-12-26 16:56:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000216616_55459840.pth... [2023-12-26 16:56:16,074][105692] Updated weights for policy 0, policy_version 216010 (0.0010) [2023-12-26 16:56:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000215432_55156736.pth [2023-12-26 16:56:16,106][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000216016_55312384.pth... [2023-12-26 16:56:16,110][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000214864_55017472.pth [2023-12-26 16:56:16,705][105620] Updated weights for policy 1, policy_version 216623 (0.0006) [2023-12-26 16:56:16,755][105620] Updated weights for policy 1, policy_version 216633 (0.0005) [2023-12-26 16:56:16,768][105692] Updated weights for policy 0, policy_version 216020 (0.0008) [2023-12-26 16:56:16,804][105620] Updated weights for policy 1, policy_version 216643 (0.0005) [2023-12-26 16:56:16,820][105692] Updated weights for policy 0, policy_version 216030 (0.0007) [2023-12-26 16:56:16,868][105692] Updated weights for policy 0, policy_version 216040 (0.0010) [2023-12-26 16:56:17,452][105692] Updated weights for policy 0, policy_version 216050 (0.0007) [2023-12-26 16:56:17,500][105692] Updated weights for policy 0, policy_version 216060 (0.0005) [2023-12-26 16:56:17,550][105692] Updated weights for policy 0, policy_version 216070 (0.0005) [2023-12-26 16:56:17,593][105620] Updated weights for policy 1, policy_version 216653 (0.0007) [2023-12-26 16:56:17,599][105692] Updated weights for policy 0, policy_version 216080 (0.0005) [2023-12-26 16:56:17,660][105620] Updated weights for policy 1, policy_version 216663 (0.0008) [2023-12-26 16:56:17,724][105620] Updated weights for policy 1, policy_version 216674 (0.0009) [2023-12-26 16:56:18,149][105692] Updated weights for policy 0, policy_version 216090 (0.0010) [2023-12-26 16:56:18,203][105692] Updated weights for policy 0, policy_version 216100 (0.0010) [2023-12-26 16:56:18,253][105692] Updated weights for policy 0, policy_version 216110 (0.0006) [2023-12-26 16:56:18,512][105620] Updated weights for policy 1, policy_version 216684 (0.0009) [2023-12-26 16:56:18,578][105620] Updated weights for policy 1, policy_version 216694 (0.0009) [2023-12-26 16:56:18,639][105620] Updated weights for policy 1, policy_version 216704 (0.0010) [2023-12-26 16:56:18,941][105692] Updated weights for policy 0, policy_version 216120 (0.0005) [2023-12-26 16:56:19,008][105692] Updated weights for policy 0, policy_version 216130 (0.0005) [2023-12-26 16:56:19,072][105692] Updated weights for policy 0, policy_version 216140 (0.0005) [2023-12-26 16:56:19,290][105620] Updated weights for policy 1, policy_version 216714 (0.0006) [2023-12-26 16:56:19,357][105620] Updated weights for policy 1, policy_version 216724 (0.0013) [2023-12-26 16:56:19,416][105620] Updated weights for policy 1, policy_version 216734 (0.0010) [2023-12-26 16:56:19,475][105620] Updated weights for policy 1, policy_version 216744 (0.0010) [2023-12-26 16:56:19,804][105692] Updated weights for policy 0, policy_version 216150 (0.0008) [2023-12-26 16:56:19,863][105692] Updated weights for policy 0, policy_version 216160 (0.0008) [2023-12-26 16:56:19,913][105692] Updated weights for policy 0, policy_version 216170 (0.0008) [2023-12-26 16:56:20,257][105620] Updated weights for policy 1, policy_version 216754 (0.0010) [2023-12-26 16:56:20,321][105620] Updated weights for policy 1, policy_version 216764 (0.0011) [2023-12-26 16:56:20,378][105620] Updated weights for policy 1, policy_version 216774 (0.0008) [2023-12-26 16:56:20,706][105692] Updated weights for policy 0, policy_version 216180 (0.0009) [2023-12-26 16:56:20,766][105692] Updated weights for policy 0, policy_version 216190 (0.0011) [2023-12-26 16:56:20,823][105692] Updated weights for policy 0, policy_version 216200 (0.0011) [2023-12-26 16:56:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 110862336. Throughput: 0: 9818.2, 1: 9709.9. Samples: 110854180. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-12-26 16:56:21,062][104569] Avg episode reward: [(0, '9095.738'), (1, '9091.327')] [2023-12-26 16:56:21,120][105620] Updated weights for policy 1, policy_version 216784 (0.0009) [2023-12-26 16:56:21,190][105620] Updated weights for policy 1, policy_version 216794 (0.0009) [2023-12-26 16:56:21,259][105620] Updated weights for policy 1, policy_version 216804 (0.0008) [2023-12-26 16:56:21,552][105692] Updated weights for policy 0, policy_version 216210 (0.0010) [2023-12-26 16:56:21,608][105692] Updated weights for policy 0, policy_version 216220 (0.0010) [2023-12-26 16:56:21,675][105692] Updated weights for policy 0, policy_version 216230 (0.0008) [2023-12-26 16:56:21,747][105692] Updated weights for policy 0, policy_version 216240 (0.0007) [2023-12-26 16:56:22,143][105620] Updated weights for policy 1, policy_version 216814 (0.0008) [2023-12-26 16:56:22,210][105620] Updated weights for policy 1, policy_version 216824 (0.0008) [2023-12-26 16:56:22,276][105620] Updated weights for policy 1, policy_version 216834 (0.0009) [2023-12-26 16:56:22,410][105692] Updated weights for policy 0, policy_version 216250 (0.0007) [2023-12-26 16:56:22,477][105692] Updated weights for policy 0, policy_version 216260 (0.0006) [2023-12-26 16:56:22,538][105692] Updated weights for policy 0, policy_version 216270 (0.0010) [2023-12-26 16:56:22,945][105620] Updated weights for policy 1, policy_version 216844 (0.0009) [2023-12-26 16:56:23,011][105620] Updated weights for policy 1, policy_version 216854 (0.0010) [2023-12-26 16:56:23,080][105620] Updated weights for policy 1, policy_version 216864 (0.0008) [2023-12-26 16:56:23,247][105692] Updated weights for policy 0, policy_version 216280 (0.0010) [2023-12-26 16:56:23,307][105692] Updated weights for policy 0, policy_version 216290 (0.0011) [2023-12-26 16:56:23,375][105692] Updated weights for policy 0, policy_version 216300 (0.0010) [2023-12-26 16:56:23,826][105620] Updated weights for policy 1, policy_version 216874 (0.0007) [2023-12-26 16:56:23,873][105620] Updated weights for policy 1, policy_version 216884 (0.0005) [2023-12-26 16:56:23,937][105620] Updated weights for policy 1, policy_version 216894 (0.0007) [2023-12-26 16:56:23,965][105692] Updated weights for policy 0, policy_version 216310 (0.0009) [2023-12-26 16:56:23,987][105620] Updated weights for policy 1, policy_version 216904 (0.0008) [2023-12-26 16:56:24,011][105692] Updated weights for policy 0, policy_version 216320 (0.0005) [2023-12-26 16:56:24,068][105692] Updated weights for policy 0, policy_version 216330 (0.0007) [2023-12-26 16:56:24,731][105620] Updated weights for policy 1, policy_version 216914 (0.0008) [2023-12-26 16:56:24,754][105692] Updated weights for policy 0, policy_version 216340 (0.0009) [2023-12-26 16:56:24,780][105620] Updated weights for policy 1, policy_version 216924 (0.0006) [2023-12-26 16:56:24,813][105692] Updated weights for policy 0, policy_version 216350 (0.0010) [2023-12-26 16:56:24,827][105620] Updated weights for policy 1, policy_version 216934 (0.0008) [2023-12-26 16:56:24,875][105692] Updated weights for policy 0, policy_version 216360 (0.0010) [2023-12-26 16:56:25,476][105620] Updated weights for policy 1, policy_version 216944 (0.0007) [2023-12-26 16:56:25,532][105620] Updated weights for policy 1, policy_version 216954 (0.0006) [2023-12-26 16:56:25,583][105620] Updated weights for policy 1, policy_version 216964 (0.0005) [2023-12-26 16:56:25,625][105692] Updated weights for policy 0, policy_version 216370 (0.0010) [2023-12-26 16:56:25,689][105692] Updated weights for policy 0, policy_version 216380 (0.0010) [2023-12-26 16:56:25,750][105692] Updated weights for policy 0, policy_version 216390 (0.0010) [2023-12-26 16:56:25,808][105692] Updated weights for policy 0, policy_version 216400 (0.0010) [2023-12-26 16:56:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 110960640. Throughput: 0: 9829.0, 1: 9774.9. Samples: 110970236. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-12-26 16:56:26,062][104569] Avg episode reward: [(0, '8913.517'), (1, '9180.383')] [2023-12-26 16:56:26,216][105620] Updated weights for policy 1, policy_version 216974 (0.0008) [2023-12-26 16:56:26,268][105620] Updated weights for policy 1, policy_version 216984 (0.0008) [2023-12-26 16:56:26,321][105620] Updated weights for policy 1, policy_version 216994 (0.0008) [2023-12-26 16:56:26,524][105692] Updated weights for policy 0, policy_version 216410 (0.0010) [2023-12-26 16:56:26,572][105692] Updated weights for policy 0, policy_version 216420 (0.0010) [2023-12-26 16:56:26,617][105692] Updated weights for policy 0, policy_version 216430 (0.0010) [2023-12-26 16:56:26,941][105620] Updated weights for policy 1, policy_version 217004 (0.0009) [2023-12-26 16:56:26,994][105620] Updated weights for policy 1, policy_version 217014 (0.0011) [2023-12-26 16:56:27,039][105620] Updated weights for policy 1, policy_version 217024 (0.0010) [2023-12-26 16:56:27,401][105692] Updated weights for policy 0, policy_version 216440 (0.0006) [2023-12-26 16:56:27,464][105692] Updated weights for policy 0, policy_version 216450 (0.0009) [2023-12-26 16:56:27,518][105692] Updated weights for policy 0, policy_version 216460 (0.0010) [2023-12-26 16:56:27,796][105620] Updated weights for policy 1, policy_version 217034 (0.0010) [2023-12-26 16:56:27,861][105620] Updated weights for policy 1, policy_version 217044 (0.0008) [2023-12-26 16:56:27,915][105620] Updated weights for policy 1, policy_version 217054 (0.0009) [2023-12-26 16:56:27,963][105620] Updated weights for policy 1, policy_version 217064 (0.0010) [2023-12-26 16:56:28,224][105692] Updated weights for policy 0, policy_version 216470 (0.0010) [2023-12-26 16:56:28,271][105692] Updated weights for policy 0, policy_version 216480 (0.0010) [2023-12-26 16:56:28,324][105692] Updated weights for policy 0, policy_version 216490 (0.0010) [2023-12-26 16:56:28,702][105620] Updated weights for policy 1, policy_version 217074 (0.0009) [2023-12-26 16:56:28,760][105620] Updated weights for policy 1, policy_version 217084 (0.0010) [2023-12-26 16:56:28,821][105620] Updated weights for policy 1, policy_version 217094 (0.0007) [2023-12-26 16:56:29,084][105692] Updated weights for policy 0, policy_version 216500 (0.0010) [2023-12-26 16:56:29,149][105692] Updated weights for policy 0, policy_version 216510 (0.0010) [2023-12-26 16:56:29,221][105692] Updated weights for policy 0, policy_version 216520 (0.0010) [2023-12-26 16:56:29,435][105620] Updated weights for policy 1, policy_version 217104 (0.0009) [2023-12-26 16:56:29,486][105620] Updated weights for policy 1, policy_version 217114 (0.0010) [2023-12-26 16:56:29,534][105620] Updated weights for policy 1, policy_version 217124 (0.0010) [2023-12-26 16:56:29,952][105692] Updated weights for policy 0, policy_version 216530 (0.0011) [2023-12-26 16:56:30,000][105692] Updated weights for policy 0, policy_version 216540 (0.0010) [2023-12-26 16:56:30,055][105692] Updated weights for policy 0, policy_version 216550 (0.0010) [2023-12-26 16:56:30,109][105692] Updated weights for policy 0, policy_version 216560 (0.0010) [2023-12-26 16:56:30,204][105620] Updated weights for policy 1, policy_version 217134 (0.0010) [2023-12-26 16:56:30,255][105620] Updated weights for policy 1, policy_version 217144 (0.0010) [2023-12-26 16:56:30,320][105620] Updated weights for policy 1, policy_version 217154 (0.0010) [2023-12-26 16:56:30,858][105692] Updated weights for policy 0, policy_version 216570 (0.0010) [2023-12-26 16:56:30,902][105692] Updated weights for policy 0, policy_version 216580 (0.0010) [2023-12-26 16:56:30,950][105692] Updated weights for policy 0, policy_version 216590 (0.0010) [2023-12-26 16:56:31,017][105620] Updated weights for policy 1, policy_version 217164 (0.0009) [2023-12-26 16:56:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 111058944. Throughput: 0: 9861.8, 1: 9835.5. Samples: 111029084. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-12-26 16:56:31,063][104569] Avg episode reward: [(0, '5715.267'), (1, '9180.305')] [2023-12-26 16:56:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000216592_55459840.pth... [2023-12-26 16:56:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000215440_55164928.pth [2023-12-26 16:56:31,076][105620] Updated weights for policy 1, policy_version 217174 (0.0008) [2023-12-26 16:56:31,139][105620] Updated weights for policy 1, policy_version 217184 (0.0008) [2023-12-26 16:56:31,190][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000217192_55607296.pth... [2023-12-26 16:56:31,194][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000216040_55312384.pth [2023-12-26 16:56:31,739][105692] Updated weights for policy 0, policy_version 216600 (0.0010) [2023-12-26 16:56:31,797][105692] Updated weights for policy 0, policy_version 216610 (0.0008) [2023-12-26 16:56:31,850][105692] Updated weights for policy 0, policy_version 216620 (0.0008) [2023-12-26 16:56:31,903][105620] Updated weights for policy 1, policy_version 217194 (0.0008) [2023-12-26 16:56:31,966][105620] Updated weights for policy 1, policy_version 217204 (0.0007) [2023-12-26 16:56:32,023][105620] Updated weights for policy 1, policy_version 217214 (0.0008) [2023-12-26 16:56:32,081][105620] Updated weights for policy 1, policy_version 217224 (0.0009) [2023-12-26 16:56:32,549][105692] Updated weights for policy 0, policy_version 216630 (0.0007) [2023-12-26 16:56:32,600][105692] Updated weights for policy 0, policy_version 216640 (0.0010) [2023-12-26 16:56:32,655][105692] Updated weights for policy 0, policy_version 216650 (0.0010) [2023-12-26 16:56:32,853][105620] Updated weights for policy 1, policy_version 217234 (0.0008) [2023-12-26 16:56:32,902][105620] Updated weights for policy 1, policy_version 217244 (0.0008) [2023-12-26 16:56:32,951][105620] Updated weights for policy 1, policy_version 217254 (0.0008) [2023-12-26 16:56:33,428][105692] Updated weights for policy 0, policy_version 216660 (0.0010) [2023-12-26 16:56:33,480][105692] Updated weights for policy 0, policy_version 216670 (0.0010) [2023-12-26 16:56:33,537][105692] Updated weights for policy 0, policy_version 216680 (0.0008) [2023-12-26 16:56:33,570][105620] Updated weights for policy 1, policy_version 217264 (0.0008) [2023-12-26 16:56:33,626][105620] Updated weights for policy 1, policy_version 217274 (0.0006) [2023-12-26 16:56:33,691][105620] Updated weights for policy 1, policy_version 217284 (0.0009) [2023-12-26 16:56:34,250][105692] Updated weights for policy 0, policy_version 216690 (0.0007) [2023-12-26 16:56:34,314][105692] Updated weights for policy 0, policy_version 216700 (0.0008) [2023-12-26 16:56:34,368][105692] Updated weights for policy 0, policy_version 216710 (0.0006) [2023-12-26 16:56:34,419][105620] Updated weights for policy 1, policy_version 217294 (0.0009) [2023-12-26 16:56:34,422][105692] Updated weights for policy 0, policy_version 216720 (0.0007) [2023-12-26 16:56:34,475][105620] Updated weights for policy 1, policy_version 217304 (0.0009) [2023-12-26 16:56:34,534][105620] Updated weights for policy 1, policy_version 217314 (0.0008) [2023-12-26 16:56:34,993][105692] Updated weights for policy 0, policy_version 216730 (0.0009) [2023-12-26 16:56:35,038][105692] Updated weights for policy 0, policy_version 216740 (0.0008) [2023-12-26 16:56:35,084][105692] Updated weights for policy 0, policy_version 216750 (0.0005) [2023-12-26 16:56:35,387][105620] Updated weights for policy 1, policy_version 217324 (0.0010) [2023-12-26 16:56:35,434][105620] Updated weights for policy 1, policy_version 217334 (0.0008) [2023-12-26 16:56:35,493][105620] Updated weights for policy 1, policy_version 217344 (0.0009) [2023-12-26 16:56:35,743][105692] Updated weights for policy 0, policy_version 216760 (0.0008) [2023-12-26 16:56:35,799][105692] Updated weights for policy 0, policy_version 216770 (0.0010) [2023-12-26 16:56:35,863][105692] Updated weights for policy 0, policy_version 216780 (0.0007) [2023-12-26 16:56:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 111157248. Throughput: 0: 9752.6, 1: 9866.6. Samples: 111144520. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-12-26 16:56:36,062][104569] Avg episode reward: [(0, '5517.904'), (1, '9266.426')] [2023-12-26 16:56:36,241][105620] Updated weights for policy 1, policy_version 217354 (0.0008) [2023-12-26 16:56:36,304][105620] Updated weights for policy 1, policy_version 217364 (0.0009) [2023-12-26 16:56:36,367][105620] Updated weights for policy 1, policy_version 217374 (0.0009) [2023-12-26 16:56:36,429][105620] Updated weights for policy 1, policy_version 217384 (0.0009) [2023-12-26 16:56:36,591][105692] Updated weights for policy 0, policy_version 216790 (0.0007) [2023-12-26 16:56:36,646][105692] Updated weights for policy 0, policy_version 216800 (0.0009) [2023-12-26 16:56:36,698][105692] Updated weights for policy 0, policy_version 216810 (0.0009) [2023-12-26 16:56:37,158][105620] Updated weights for policy 1, policy_version 217394 (0.0009) [2023-12-26 16:56:37,206][105620] Updated weights for policy 1, policy_version 217404 (0.0009) [2023-12-26 16:56:37,264][105620] Updated weights for policy 1, policy_version 217415 (0.0010) [2023-12-26 16:56:37,409][105692] Updated weights for policy 0, policy_version 216820 (0.0008) [2023-12-26 16:56:37,468][105692] Updated weights for policy 0, policy_version 216830 (0.0005) [2023-12-26 16:56:37,540][105692] Updated weights for policy 0, policy_version 216840 (0.0006) [2023-12-26 16:56:37,936][105620] Updated weights for policy 1, policy_version 217425 (0.0009) [2023-12-26 16:56:37,988][105620] Updated weights for policy 1, policy_version 217435 (0.0009) [2023-12-26 16:56:38,048][105620] Updated weights for policy 1, policy_version 217445 (0.0009) [2023-12-26 16:56:38,129][105692] Updated weights for policy 0, policy_version 216850 (0.0006) [2023-12-26 16:56:38,189][105692] Updated weights for policy 0, policy_version 216860 (0.0008) [2023-12-26 16:56:38,242][105692] Updated weights for policy 0, policy_version 216870 (0.0009) [2023-12-26 16:56:38,288][105692] Updated weights for policy 0, policy_version 216880 (0.0008) [2023-12-26 16:56:38,863][105620] Updated weights for policy 1, policy_version 217455 (0.0009) [2023-12-26 16:56:38,921][105620] Updated weights for policy 1, policy_version 217465 (0.0006) [2023-12-26 16:56:38,966][105692] Updated weights for policy 0, policy_version 216890 (0.0009) [2023-12-26 16:56:38,973][105620] Updated weights for policy 1, policy_version 217475 (0.0006) [2023-12-26 16:56:39,024][105692] Updated weights for policy 0, policy_version 216900 (0.0007) [2023-12-26 16:56:39,088][105692] Updated weights for policy 0, policy_version 216910 (0.0010) [2023-12-26 16:56:39,664][105620] Updated weights for policy 1, policy_version 217485 (0.0007) [2023-12-26 16:56:39,728][105620] Updated weights for policy 1, policy_version 217495 (0.0008) [2023-12-26 16:56:39,788][105620] Updated weights for policy 1, policy_version 217505 (0.0009) [2023-12-26 16:56:39,950][105692] Updated weights for policy 0, policy_version 216920 (0.0010) [2023-12-26 16:56:40,018][105692] Updated weights for policy 0, policy_version 216930 (0.0011) [2023-12-26 16:56:40,080][105692] Updated weights for policy 0, policy_version 216940 (0.0011) [2023-12-26 16:56:40,625][105620] Updated weights for policy 1, policy_version 217515 (0.0009) [2023-12-26 16:56:40,676][105620] Updated weights for policy 1, policy_version 217525 (0.0008) [2023-12-26 16:56:40,700][105692] Updated weights for policy 0, policy_version 216950 (0.0007) [2023-12-26 16:56:40,731][105620] Updated weights for policy 1, policy_version 217535 (0.0007) [2023-12-26 16:56:40,749][105692] Updated weights for policy 0, policy_version 216960 (0.0009) [2023-12-26 16:56:40,801][105692] Updated weights for policy 0, policy_version 216970 (0.0010) [2023-12-26 16:56:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19633.1). Total num frames: 111255552. Throughput: 0: 9809.7, 1: 9796.2. Samples: 111261464. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-12-26 16:56:41,062][104569] Avg episode reward: [(0, '7033.886'), (1, '9356.331')] [2023-12-26 16:56:41,435][105620] Updated weights for policy 1, policy_version 217545 (0.0006) [2023-12-26 16:56:41,487][105620] Updated weights for policy 1, policy_version 217555 (0.0008) [2023-12-26 16:56:41,536][105620] Updated weights for policy 1, policy_version 217565 (0.0008) [2023-12-26 16:56:41,591][105620] Updated weights for policy 1, policy_version 217575 (0.0007) [2023-12-26 16:56:41,593][105692] Updated weights for policy 0, policy_version 216980 (0.0010) [2023-12-26 16:56:41,657][105692] Updated weights for policy 0, policy_version 216990 (0.0009) [2023-12-26 16:56:41,712][105692] Updated weights for policy 0, policy_version 217000 (0.0008) [2023-12-26 16:56:42,416][105620] Updated weights for policy 1, policy_version 217585 (0.0008) [2023-12-26 16:56:42,456][105692] Updated weights for policy 0, policy_version 217010 (0.0008) [2023-12-26 16:56:42,479][105620] Updated weights for policy 1, policy_version 217595 (0.0008) [2023-12-26 16:56:42,520][105692] Updated weights for policy 0, policy_version 217020 (0.0007) [2023-12-26 16:56:42,535][105620] Updated weights for policy 1, policy_version 217605 (0.0007) [2023-12-26 16:56:42,580][105692] Updated weights for policy 0, policy_version 217030 (0.0008) [2023-12-26 16:56:42,639][105692] Updated weights for policy 0, policy_version 217040 (0.0009) [2023-12-26 16:56:43,263][105620] Updated weights for policy 1, policy_version 217615 (0.0007) [2023-12-26 16:56:43,307][105692] Updated weights for policy 0, policy_version 217050 (0.0005) [2023-12-26 16:56:43,318][105620] Updated weights for policy 1, policy_version 217625 (0.0009) [2023-12-26 16:56:43,356][105692] Updated weights for policy 0, policy_version 217060 (0.0005) [2023-12-26 16:56:43,366][105620] Updated weights for policy 1, policy_version 217635 (0.0010) [2023-12-26 16:56:43,407][105692] Updated weights for policy 0, policy_version 217070 (0.0005) [2023-12-26 16:56:43,962][105692] Updated weights for policy 0, policy_version 217080 (0.0005) [2023-12-26 16:56:43,994][105620] Updated weights for policy 1, policy_version 217645 (0.0007) [2023-12-26 16:56:44,030][105692] Updated weights for policy 0, policy_version 217090 (0.0006) [2023-12-26 16:56:44,064][105620] Updated weights for policy 1, policy_version 217655 (0.0006) [2023-12-26 16:56:44,079][105692] Updated weights for policy 0, policy_version 217100 (0.0008) [2023-12-26 16:56:44,125][105620] Updated weights for policy 1, policy_version 217665 (0.0005) [2023-12-26 16:56:44,685][105620] Updated weights for policy 1, policy_version 217675 (0.0006) [2023-12-26 16:56:44,743][105620] Updated weights for policy 1, policy_version 217685 (0.0010) [2023-12-26 16:56:44,811][105620] Updated weights for policy 1, policy_version 217696 (0.0007) [2023-12-26 16:56:44,829][105692] Updated weights for policy 0, policy_version 217110 (0.0007) [2023-12-26 16:56:44,885][105692] Updated weights for policy 0, policy_version 217120 (0.0008) [2023-12-26 16:56:44,946][105692] Updated weights for policy 0, policy_version 217131 (0.0010) [2023-12-26 16:56:45,568][105620] Updated weights for policy 1, policy_version 217706 (0.0009) [2023-12-26 16:56:45,620][105620] Updated weights for policy 1, policy_version 217716 (0.0010) [2023-12-26 16:56:45,666][105692] Updated weights for policy 0, policy_version 217141 (0.0007) [2023-12-26 16:56:45,674][105620] Updated weights for policy 1, policy_version 217726 (0.0009) [2023-12-26 16:56:45,716][105692] Updated weights for policy 0, policy_version 217151 (0.0005) [2023-12-26 16:56:45,726][105620] Updated weights for policy 1, policy_version 217736 (0.0008) [2023-12-26 16:56:45,767][105692] Updated weights for policy 0, policy_version 217161 (0.0005) [2023-12-26 16:56:46,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 111353856. Throughput: 0: 9828.4, 1: 9698.5. Samples: 111321000. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-12-26 16:56:46,063][104569] Avg episode reward: [(0, '7990.007'), (1, '9356.162')] [2023-12-26 16:56:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000217736_55746560.pth... [2023-12-26 16:56:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000217168_55607296.pth... [2023-12-26 16:56:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000216616_55459840.pth [2023-12-26 16:56:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000216016_55312384.pth [2023-12-26 16:56:46,448][105620] Updated weights for policy 1, policy_version 217746 (0.0009) [2023-12-26 16:56:46,493][105692] Updated weights for policy 0, policy_version 217171 (0.0007) [2023-12-26 16:56:46,496][105620] Updated weights for policy 1, policy_version 217756 (0.0008) [2023-12-26 16:56:46,539][105692] Updated weights for policy 0, policy_version 217181 (0.0006) [2023-12-26 16:56:46,541][105620] Updated weights for policy 1, policy_version 217766 (0.0006) [2023-12-26 16:56:46,589][105692] Updated weights for policy 0, policy_version 217191 (0.0009) [2023-12-26 16:56:47,175][105620] Updated weights for policy 1, policy_version 217776 (0.0007) [2023-12-26 16:56:47,223][105620] Updated weights for policy 1, policy_version 217786 (0.0009) [2023-12-26 16:56:47,269][105620] Updated weights for policy 1, policy_version 217796 (0.0008) [2023-12-26 16:56:47,402][105692] Updated weights for policy 0, policy_version 217201 (0.0008) [2023-12-26 16:56:47,454][105692] Updated weights for policy 0, policy_version 217211 (0.0007) [2023-12-26 16:56:47,500][105692] Updated weights for policy 0, policy_version 217221 (0.0005) [2023-12-26 16:56:47,551][105692] Updated weights for policy 0, policy_version 217231 (0.0005) [2023-12-26 16:56:47,991][105620] Updated weights for policy 1, policy_version 217806 (0.0007) [2023-12-26 16:56:48,049][105620] Updated weights for policy 1, policy_version 217816 (0.0010) [2023-12-26 16:56:48,108][105620] Updated weights for policy 1, policy_version 217826 (0.0010) [2023-12-26 16:56:48,230][105692] Updated weights for policy 0, policy_version 217241 (0.0008) [2023-12-26 16:56:48,277][105692] Updated weights for policy 0, policy_version 217251 (0.0008) [2023-12-26 16:56:48,337][105692] Updated weights for policy 0, policy_version 217261 (0.0008) [2023-12-26 16:56:48,817][105620] Updated weights for policy 1, policy_version 217836 (0.0010) [2023-12-26 16:56:48,879][105620] Updated weights for policy 1, policy_version 217846 (0.0009) [2023-12-26 16:56:48,943][105620] Updated weights for policy 1, policy_version 217856 (0.0008) [2023-12-26 16:56:49,113][105692] Updated weights for policy 0, policy_version 217271 (0.0009) [2023-12-26 16:56:49,171][105692] Updated weights for policy 0, policy_version 217281 (0.0008) [2023-12-26 16:56:49,232][105692] Updated weights for policy 0, policy_version 217291 (0.0008) [2023-12-26 16:56:49,697][105620] Updated weights for policy 1, policy_version 217866 (0.0009) [2023-12-26 16:56:49,745][105620] Updated weights for policy 1, policy_version 217876 (0.0009) [2023-12-26 16:56:49,797][105620] Updated weights for policy 1, policy_version 217886 (0.0009) [2023-12-26 16:56:49,864][105620] Updated weights for policy 1, policy_version 217896 (0.0009) [2023-12-26 16:56:49,997][105692] Updated weights for policy 0, policy_version 217301 (0.0009) [2023-12-26 16:56:50,053][105692] Updated weights for policy 0, policy_version 217311 (0.0009) [2023-12-26 16:56:50,100][105692] Updated weights for policy 0, policy_version 217321 (0.0009) [2023-12-26 16:56:50,584][105620] Updated weights for policy 1, policy_version 217906 (0.0007) [2023-12-26 16:56:50,646][105620] Updated weights for policy 1, policy_version 217916 (0.0006) [2023-12-26 16:56:50,708][105620] Updated weights for policy 1, policy_version 217926 (0.0009) [2023-12-26 16:56:50,940][105692] Updated weights for policy 0, policy_version 217331 (0.0009) [2023-12-26 16:56:51,002][105692] Updated weights for policy 0, policy_version 217341 (0.0009) [2023-12-26 16:56:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 111443968. Throughput: 0: 9876.0, 1: 9661.7. Samples: 111438148. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-12-26 16:56:51,062][104569] Avg episode reward: [(0, '8083.926'), (1, '9355.749')] [2023-12-26 16:56:51,070][105692] Updated weights for policy 0, policy_version 217351 (0.0008) [2023-12-26 16:56:51,475][105620] Updated weights for policy 1, policy_version 217936 (0.0009) [2023-12-26 16:56:51,536][105620] Updated weights for policy 1, policy_version 217946 (0.0009) [2023-12-26 16:56:51,600][105620] Updated weights for policy 1, policy_version 217956 (0.0010) [2023-12-26 16:56:51,787][105692] Updated weights for policy 0, policy_version 217361 (0.0008) [2023-12-26 16:56:51,847][105692] Updated weights for policy 0, policy_version 217371 (0.0009) [2023-12-26 16:56:51,908][105692] Updated weights for policy 0, policy_version 217381 (0.0008) [2023-12-26 16:56:51,969][105692] Updated weights for policy 0, policy_version 217391 (0.0008) [2023-12-26 16:56:52,419][105620] Updated weights for policy 1, policy_version 217966 (0.0009) [2023-12-26 16:56:52,478][105620] Updated weights for policy 1, policy_version 217976 (0.0009) [2023-12-26 16:56:52,531][105620] Updated weights for policy 1, policy_version 217986 (0.0006) [2023-12-26 16:56:52,720][105692] Updated weights for policy 0, policy_version 217401 (0.0009) [2023-12-26 16:56:52,774][105692] Updated weights for policy 0, policy_version 217411 (0.0009) [2023-12-26 16:56:52,835][105692] Updated weights for policy 0, policy_version 217421 (0.0010) [2023-12-26 16:56:53,161][105620] Updated weights for policy 1, policy_version 217996 (0.0007) [2023-12-26 16:56:53,213][105620] Updated weights for policy 1, policy_version 218006 (0.0006) [2023-12-26 16:56:53,270][105620] Updated weights for policy 1, policy_version 218016 (0.0006) [2023-12-26 16:56:53,586][105692] Updated weights for policy 0, policy_version 217431 (0.0009) [2023-12-26 16:56:53,652][105692] Updated weights for policy 0, policy_version 217441 (0.0010) [2023-12-26 16:56:53,709][105692] Updated weights for policy 0, policy_version 217451 (0.0010) [2023-12-26 16:56:53,882][105620] Updated weights for policy 1, policy_version 218026 (0.0007) [2023-12-26 16:56:53,943][105620] Updated weights for policy 1, policy_version 218036 (0.0007) [2023-12-26 16:56:54,000][105620] Updated weights for policy 1, policy_version 218046 (0.0008) [2023-12-26 16:56:54,061][105620] Updated weights for policy 1, policy_version 218056 (0.0008) [2023-12-26 16:56:54,454][105692] Updated weights for policy 0, policy_version 217461 (0.0008) [2023-12-26 16:56:54,498][105692] Updated weights for policy 0, policy_version 217471 (0.0005) [2023-12-26 16:56:54,555][105692] Updated weights for policy 0, policy_version 217481 (0.0005) [2023-12-26 16:56:54,830][105620] Updated weights for policy 1, policy_version 218066 (0.0009) [2023-12-26 16:56:54,878][105620] Updated weights for policy 1, policy_version 218076 (0.0007) [2023-12-26 16:56:54,927][105620] Updated weights for policy 1, policy_version 218086 (0.0007) [2023-12-26 16:56:55,124][105692] Updated weights for policy 0, policy_version 217491 (0.0007) [2023-12-26 16:56:55,173][105692] Updated weights for policy 0, policy_version 217501 (0.0010) [2023-12-26 16:56:55,228][105692] Updated weights for policy 0, policy_version 217511 (0.0010) [2023-12-26 16:56:55,521][105620] Updated weights for policy 1, policy_version 218096 (0.0006) [2023-12-26 16:56:55,575][105620] Updated weights for policy 1, policy_version 218106 (0.0005) [2023-12-26 16:56:55,626][105620] Updated weights for policy 1, policy_version 218116 (0.0005) [2023-12-26 16:56:55,811][105692] Updated weights for policy 0, policy_version 217521 (0.0007) [2023-12-26 16:56:55,864][105692] Updated weights for policy 0, policy_version 217531 (0.0008) [2023-12-26 16:56:55,912][105692] Updated weights for policy 0, policy_version 217541 (0.0010) [2023-12-26 16:56:55,958][105692] Updated weights for policy 0, policy_version 217551 (0.0006) [2023-12-26 16:56:56,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19660.9, 300 sec: 19633.0). Total num frames: 111550464. Throughput: 0: 9869.4, 1: 9739.9. Samples: 111557976. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-12-26 16:56:56,062][104569] Avg episode reward: [(0, '8272.729'), (1, '9356.376')] [2023-12-26 16:56:56,175][105620] Updated weights for policy 1, policy_version 218126 (0.0006) [2023-12-26 16:56:56,240][105620] Updated weights for policy 1, policy_version 218136 (0.0005) [2023-12-26 16:56:56,304][105620] Updated weights for policy 1, policy_version 218146 (0.0005) [2023-12-26 16:56:56,571][105692] Updated weights for policy 0, policy_version 217561 (0.0007) [2023-12-26 16:56:56,633][105692] Updated weights for policy 0, policy_version 217571 (0.0009) [2023-12-26 16:56:56,692][105692] Updated weights for policy 0, policy_version 217581 (0.0009) [2023-12-26 16:56:56,861][105620] Updated weights for policy 1, policy_version 218156 (0.0007) [2023-12-26 16:56:56,918][105620] Updated weights for policy 1, policy_version 218166 (0.0010) [2023-12-26 16:56:56,984][105620] Updated weights for policy 1, policy_version 218176 (0.0010) [2023-12-26 16:56:57,334][105692] Updated weights for policy 0, policy_version 217591 (0.0009) [2023-12-26 16:56:57,391][105692] Updated weights for policy 0, policy_version 217601 (0.0008) [2023-12-26 16:56:57,455][105692] Updated weights for policy 0, policy_version 217611 (0.0009) [2023-12-26 16:56:57,742][105620] Updated weights for policy 1, policy_version 218186 (0.0010) [2023-12-26 16:56:57,798][105620] Updated weights for policy 1, policy_version 218196 (0.0009) [2023-12-26 16:56:57,855][105620] Updated weights for policy 1, policy_version 218206 (0.0008) [2023-12-26 16:56:57,911][105620] Updated weights for policy 1, policy_version 218216 (0.0009) [2023-12-26 16:56:58,213][105692] Updated weights for policy 0, policy_version 217621 (0.0009) [2023-12-26 16:56:58,278][105692] Updated weights for policy 0, policy_version 217631 (0.0009) [2023-12-26 16:56:58,363][105692] Updated weights for policy 0, policy_version 217641 (0.0007) [2023-12-26 16:56:58,687][105620] Updated weights for policy 1, policy_version 218226 (0.0008) [2023-12-26 16:56:58,754][105620] Updated weights for policy 1, policy_version 218236 (0.0009) [2023-12-26 16:56:58,828][105620] Updated weights for policy 1, policy_version 218246 (0.0010) [2023-12-26 16:56:59,161][105692] Updated weights for policy 0, policy_version 217651 (0.0009) [2023-12-26 16:56:59,226][105692] Updated weights for policy 0, policy_version 217661 (0.0007) [2023-12-26 16:56:59,296][105692] Updated weights for policy 0, policy_version 217671 (0.0008) [2023-12-26 16:56:59,704][105620] Updated weights for policy 1, policy_version 218256 (0.0010) [2023-12-26 16:56:59,765][105620] Updated weights for policy 1, policy_version 218266 (0.0010) [2023-12-26 16:56:59,835][105620] Updated weights for policy 1, policy_version 218276 (0.0011) [2023-12-26 16:56:59,869][105692] Updated weights for policy 0, policy_version 217681 (0.0007) [2023-12-26 16:56:59,933][105692] Updated weights for policy 0, policy_version 217691 (0.0006) [2023-12-26 16:56:59,983][105692] Updated weights for policy 0, policy_version 217701 (0.0005) [2023-12-26 16:57:00,035][105692] Updated weights for policy 0, policy_version 217711 (0.0008) [2023-12-26 16:57:00,572][105620] Updated weights for policy 1, policy_version 218286 (0.0009) [2023-12-26 16:57:00,625][105620] Updated weights for policy 1, policy_version 218296 (0.0005) [2023-12-26 16:57:00,635][105586] KL-divergence is very high: 103.0315 [2023-12-26 16:57:00,675][105692] Updated weights for policy 0, policy_version 217721 (0.0008) [2023-12-26 16:57:00,678][105620] Updated weights for policy 1, policy_version 218306 (0.0005) [2023-12-26 16:57:00,729][105692] Updated weights for policy 0, policy_version 217732 (0.0009) [2023-12-26 16:57:00,783][105692] Updated weights for policy 0, policy_version 217743 (0.0009) [2023-12-26 16:57:01,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 111648768. Throughput: 0: 9856.8, 1: 9782.1. Samples: 111618020. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:57:01,062][104569] Avg episode reward: [(0, '8638.352'), (1, '2474.600')] [2023-12-26 16:57:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000217744_55754752.pth... [2023-12-26 16:57:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000218312_55894016.pth... [2023-12-26 16:57:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000217192_55607296.pth [2023-12-26 16:57:01,095][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000216592_55459840.pth [2023-12-26 16:57:01,299][105620] Updated weights for policy 1, policy_version 218316 (0.0007) [2023-12-26 16:57:01,367][105620] Updated weights for policy 1, policy_version 218326 (0.0010) [2023-12-26 16:57:01,426][105620] Updated weights for policy 1, policy_version 218336 (0.0006) [2023-12-26 16:57:01,646][105692] Updated weights for policy 0, policy_version 217753 (0.0009) [2023-12-26 16:57:01,720][105692] Updated weights for policy 0, policy_version 217764 (0.0010) [2023-12-26 16:57:01,778][105692] Updated weights for policy 0, policy_version 217774 (0.0008) [2023-12-26 16:57:02,052][105620] Updated weights for policy 1, policy_version 218346 (0.0006) [2023-12-26 16:57:02,109][105620] Updated weights for policy 1, policy_version 218356 (0.0009) [2023-12-26 16:57:02,161][105620] Updated weights for policy 1, policy_version 218366 (0.0009) [2023-12-26 16:57:02,212][105620] Updated weights for policy 1, policy_version 218376 (0.0009) [2023-12-26 16:57:02,582][105692] Updated weights for policy 0, policy_version 217784 (0.0008) [2023-12-26 16:57:02,629][105692] Updated weights for policy 0, policy_version 217794 (0.0008) [2023-12-26 16:57:02,696][105692] Updated weights for policy 0, policy_version 217804 (0.0010) [2023-12-26 16:57:02,989][105620] Updated weights for policy 1, policy_version 218386 (0.0009) [2023-12-26 16:57:03,049][105620] Updated weights for policy 1, policy_version 218396 (0.0009) [2023-12-26 16:57:03,110][105620] Updated weights for policy 1, policy_version 218406 (0.0009) [2023-12-26 16:57:03,325][105692] Updated weights for policy 0, policy_version 217814 (0.0009) [2023-12-26 16:57:03,372][105692] Updated weights for policy 0, policy_version 217824 (0.0008) [2023-12-26 16:57:03,429][105692] Updated weights for policy 0, policy_version 217834 (0.0009) [2023-12-26 16:57:03,835][105620] Updated weights for policy 1, policy_version 218416 (0.0009) [2023-12-26 16:57:03,901][105620] Updated weights for policy 1, policy_version 218426 (0.0007) [2023-12-26 16:57:03,969][105620] Updated weights for policy 1, policy_version 218436 (0.0008) [2023-12-26 16:57:04,096][105692] Updated weights for policy 0, policy_version 217844 (0.0009) [2023-12-26 16:57:04,153][105692] Updated weights for policy 0, policy_version 217854 (0.0008) [2023-12-26 16:57:04,208][105692] Updated weights for policy 0, policy_version 217864 (0.0008) [2023-12-26 16:57:04,717][105620] Updated weights for policy 1, policy_version 218446 (0.0008) [2023-12-26 16:57:04,766][105620] Updated weights for policy 1, policy_version 218456 (0.0008) [2023-12-26 16:57:04,821][105620] Updated weights for policy 1, policy_version 218466 (0.0009) [2023-12-26 16:57:04,997][105692] Updated weights for policy 0, policy_version 217874 (0.0009) [2023-12-26 16:57:05,063][105692] Updated weights for policy 0, policy_version 217884 (0.0009) [2023-12-26 16:57:05,122][105692] Updated weights for policy 0, policy_version 217894 (0.0009) [2023-12-26 16:57:05,173][105692] Updated weights for policy 0, policy_version 217904 (0.0009) [2023-12-26 16:57:05,501][105620] Updated weights for policy 1, policy_version 218476 (0.0009) [2023-12-26 16:57:05,547][105620] Updated weights for policy 1, policy_version 218486 (0.0009) [2023-12-26 16:57:05,594][105620] Updated weights for policy 1, policy_version 218496 (0.0008) [2023-12-26 16:57:05,927][105692] Updated weights for policy 0, policy_version 217914 (0.0010) [2023-12-26 16:57:05,984][105692] Updated weights for policy 0, policy_version 217924 (0.0009) [2023-12-26 16:57:06,034][105692] Updated weights for policy 0, policy_version 217934 (0.0008) [2023-12-26 16:57:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 111747072. Throughput: 0: 9781.7, 1: 9765.4. Samples: 111733800. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:57:06,062][104569] Avg episode reward: [(0, '2078.946'), (1, '1633.654')] [2023-12-26 16:57:06,322][105620] Updated weights for policy 1, policy_version 218506 (0.0009) [2023-12-26 16:57:06,381][105620] Updated weights for policy 1, policy_version 218516 (0.0008) [2023-12-26 16:57:06,441][105620] Updated weights for policy 1, policy_version 218526 (0.0008) [2023-12-26 16:57:06,501][105620] Updated weights for policy 1, policy_version 218536 (0.0008) [2023-12-26 16:57:06,843][105692] Updated weights for policy 0, policy_version 217944 (0.0010) [2023-12-26 16:57:06,899][105692] Updated weights for policy 0, policy_version 217954 (0.0008) [2023-12-26 16:57:06,960][105692] Updated weights for policy 0, policy_version 217964 (0.0011) [2023-12-26 16:57:07,267][105620] Updated weights for policy 1, policy_version 218546 (0.0008) [2023-12-26 16:57:07,322][105620] Updated weights for policy 1, policy_version 218556 (0.0008) [2023-12-26 16:57:07,378][105620] Updated weights for policy 1, policy_version 218566 (0.0008) [2023-12-26 16:57:07,735][105692] Updated weights for policy 0, policy_version 217974 (0.0010) [2023-12-26 16:57:07,799][105692] Updated weights for policy 0, policy_version 217984 (0.0006) [2023-12-26 16:57:07,860][105692] Updated weights for policy 0, policy_version 217994 (0.0006) [2023-12-26 16:57:08,130][105620] Updated weights for policy 1, policy_version 218576 (0.0008) [2023-12-26 16:57:08,192][105620] Updated weights for policy 1, policy_version 218586 (0.0009) [2023-12-26 16:57:08,247][105620] Updated weights for policy 1, policy_version 218596 (0.0009) [2023-12-26 16:57:08,516][105692] Updated weights for policy 0, policy_version 218004 (0.0009) [2023-12-26 16:57:08,590][105692] Updated weights for policy 0, policy_version 218014 (0.0008) [2023-12-26 16:57:08,648][105692] Updated weights for policy 0, policy_version 218024 (0.0010) [2023-12-26 16:57:09,066][105620] Updated weights for policy 1, policy_version 218606 (0.0008) [2023-12-26 16:57:09,118][105620] Updated weights for policy 1, policy_version 218616 (0.0008) [2023-12-26 16:57:09,172][105620] Updated weights for policy 1, policy_version 218626 (0.0008) [2023-12-26 16:57:09,378][105692] Updated weights for policy 0, policy_version 218034 (0.0010) [2023-12-26 16:57:09,446][105692] Updated weights for policy 0, policy_version 218044 (0.0011) [2023-12-26 16:57:09,510][105692] Updated weights for policy 0, policy_version 218054 (0.0011) [2023-12-26 16:57:09,574][105692] Updated weights for policy 0, policy_version 218064 (0.0008) [2023-12-26 16:57:10,037][105620] Updated weights for policy 1, policy_version 218636 (0.0008) [2023-12-26 16:57:10,101][105620] Updated weights for policy 1, policy_version 218646 (0.0008) [2023-12-26 16:57:10,164][105620] Updated weights for policy 1, policy_version 218656 (0.0008) [2023-12-26 16:57:10,177][105692] Updated weights for policy 0, policy_version 218074 (0.0009) [2023-12-26 16:57:10,236][105692] Updated weights for policy 0, policy_version 218084 (0.0009) [2023-12-26 16:57:10,305][105692] Updated weights for policy 0, policy_version 218094 (0.0009) [2023-12-26 16:57:10,881][105620] Updated weights for policy 1, policy_version 218666 (0.0008) [2023-12-26 16:57:10,933][105620] Updated weights for policy 1, policy_version 218676 (0.0009) [2023-12-26 16:57:10,995][105620] Updated weights for policy 1, policy_version 218686 (0.0005) [2023-12-26 16:57:11,058][105620] Updated weights for policy 1, policy_version 218696 (0.0008) [2023-12-26 16:57:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 111837184. Throughput: 0: 9728.4, 1: 9731.8. Samples: 111845948. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:57:11,062][104569] Avg episode reward: [(0, '3462.389'), (1, '6589.042')] [2023-12-26 16:57:11,100][105692] Updated weights for policy 0, policy_version 218104 (0.0008) [2023-12-26 16:57:11,163][105692] Updated weights for policy 0, policy_version 218114 (0.0009) [2023-12-26 16:57:11,227][105692] Updated weights for policy 0, policy_version 218124 (0.0009) [2023-12-26 16:57:11,809][105620] Updated weights for policy 1, policy_version 218706 (0.0009) [2023-12-26 16:57:11,876][105620] Updated weights for policy 1, policy_version 218716 (0.0008) [2023-12-26 16:57:11,941][105620] Updated weights for policy 1, policy_version 218726 (0.0009) [2023-12-26 16:57:12,015][105692] Updated weights for policy 0, policy_version 218134 (0.0009) [2023-12-26 16:57:12,070][105692] Updated weights for policy 0, policy_version 218144 (0.0009) [2023-12-26 16:57:12,127][105692] Updated weights for policy 0, policy_version 218154 (0.0009) [2023-12-26 16:57:12,692][105620] Updated weights for policy 1, policy_version 218736 (0.0010) [2023-12-26 16:57:12,762][105620] Updated weights for policy 1, policy_version 218746 (0.0008) [2023-12-26 16:57:12,821][105620] Updated weights for policy 1, policy_version 218756 (0.0009) [2023-12-26 16:57:12,867][105692] Updated weights for policy 0, policy_version 218164 (0.0009) [2023-12-26 16:57:12,922][105692] Updated weights for policy 0, policy_version 218174 (0.0009) [2023-12-26 16:57:12,975][105692] Updated weights for policy 0, policy_version 218184 (0.0009) [2023-12-26 16:57:13,588][105620] Updated weights for policy 1, policy_version 218766 (0.0009) [2023-12-26 16:57:13,634][105620] Updated weights for policy 1, policy_version 218776 (0.0009) [2023-12-26 16:57:13,684][105620] Updated weights for policy 1, policy_version 218786 (0.0009) [2023-12-26 16:57:13,694][105692] Updated weights for policy 0, policy_version 218194 (0.0008) [2023-12-26 16:57:13,749][105692] Updated weights for policy 0, policy_version 218204 (0.0007) [2023-12-26 16:57:13,805][105692] Updated weights for policy 0, policy_version 218214 (0.0009) [2023-12-26 16:57:13,857][105692] Updated weights for policy 0, policy_version 218224 (0.0009) [2023-12-26 16:57:14,391][105620] Updated weights for policy 1, policy_version 218796 (0.0010) [2023-12-26 16:57:14,457][105620] Updated weights for policy 1, policy_version 218806 (0.0010) [2023-12-26 16:57:14,516][105620] Updated weights for policy 1, policy_version 218816 (0.0010) [2023-12-26 16:57:14,656][105692] Updated weights for policy 0, policy_version 218234 (0.0010) [2023-12-26 16:57:14,714][105692] Updated weights for policy 0, policy_version 218245 (0.0011) [2023-12-26 16:57:14,770][105692] Updated weights for policy 0, policy_version 218255 (0.0010) [2023-12-26 16:57:15,214][105620] Updated weights for policy 1, policy_version 218826 (0.0010) [2023-12-26 16:57:15,274][105620] Updated weights for policy 1, policy_version 218836 (0.0009) [2023-12-26 16:57:15,329][105620] Updated weights for policy 1, policy_version 218846 (0.0009) [2023-12-26 16:57:15,377][105620] Updated weights for policy 1, policy_version 218856 (0.0008) [2023-12-26 16:57:15,563][105692] Updated weights for policy 0, policy_version 218265 (0.0009) [2023-12-26 16:57:15,625][105692] Updated weights for policy 0, policy_version 218275 (0.0009) [2023-12-26 16:57:15,685][105692] Updated weights for policy 0, policy_version 218285 (0.0009) [2023-12-26 16:57:16,062][104569] Fps is (10 sec: 18022.0, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 111927296. Throughput: 0: 9720.3, 1: 9670.3. Samples: 111901664. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:57:16,063][104569] Avg episode reward: [(0, '7142.020'), (1, '9264.244')] [2023-12-26 16:57:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000218288_55894016.pth... [2023-12-26 16:57:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000218856_56033280.pth... [2023-12-26 16:57:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000217168_55607296.pth [2023-12-26 16:57:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000217736_55746560.pth [2023-12-26 16:57:16,150][105620] Updated weights for policy 1, policy_version 218866 (0.0005) [2023-12-26 16:57:16,198][105620] Updated weights for policy 1, policy_version 218876 (0.0005) [2023-12-26 16:57:16,266][105620] Updated weights for policy 1, policy_version 218886 (0.0005) [2023-12-26 16:57:16,435][105692] Updated weights for policy 0, policy_version 218295 (0.0007) [2023-12-26 16:57:16,483][105692] Updated weights for policy 0, policy_version 218305 (0.0005) [2023-12-26 16:57:16,543][105692] Updated weights for policy 0, policy_version 218315 (0.0005) [2023-12-26 16:57:16,765][105620] Updated weights for policy 1, policy_version 218896 (0.0005) [2023-12-26 16:57:16,829][105620] Updated weights for policy 1, policy_version 218906 (0.0005) [2023-12-26 16:57:16,899][105620] Updated weights for policy 1, policy_version 218916 (0.0005) [2023-12-26 16:57:17,153][105692] Updated weights for policy 0, policy_version 218325 (0.0006) [2023-12-26 16:57:17,206][105692] Updated weights for policy 0, policy_version 218335 (0.0010) [2023-12-26 16:57:17,260][105692] Updated weights for policy 0, policy_version 218346 (0.0008) [2023-12-26 16:57:17,395][105620] Updated weights for policy 1, policy_version 218926 (0.0005) [2023-12-26 16:57:17,452][105620] Updated weights for policy 1, policy_version 218936 (0.0006) [2023-12-26 16:57:17,497][105620] Updated weights for policy 1, policy_version 218946 (0.0005) [2023-12-26 16:57:17,935][105692] Updated weights for policy 0, policy_version 218356 (0.0009) [2023-12-26 16:57:17,993][105692] Updated weights for policy 0, policy_version 218366 (0.0009) [2023-12-26 16:57:18,050][105692] Updated weights for policy 0, policy_version 218376 (0.0009) [2023-12-26 16:57:18,052][105620] Updated weights for policy 1, policy_version 218956 (0.0007) [2023-12-26 16:57:18,108][105620] Updated weights for policy 1, policy_version 218966 (0.0006) [2023-12-26 16:57:18,175][105620] Updated weights for policy 1, policy_version 218976 (0.0008) [2023-12-26 16:57:18,805][105620] Updated weights for policy 1, policy_version 218986 (0.0008) [2023-12-26 16:57:18,823][105692] Updated weights for policy 0, policy_version 218386 (0.0007) [2023-12-26 16:57:18,870][105620] Updated weights for policy 1, policy_version 218996 (0.0009) [2023-12-26 16:57:18,874][105692] Updated weights for policy 0, policy_version 218396 (0.0006) [2023-12-26 16:57:18,930][105620] Updated weights for policy 1, policy_version 219006 (0.0009) [2023-12-26 16:57:18,931][105692] Updated weights for policy 0, policy_version 218406 (0.0007) [2023-12-26 16:57:18,992][105620] Updated weights for policy 1, policy_version 219016 (0.0007) [2023-12-26 16:57:18,993][105692] Updated weights for policy 0, policy_version 218416 (0.0006) [2023-12-26 16:57:19,701][105692] Updated weights for policy 0, policy_version 218426 (0.0009) [2023-12-26 16:57:19,764][105692] Updated weights for policy 0, policy_version 218436 (0.0007) [2023-12-26 16:57:19,765][105620] Updated weights for policy 1, policy_version 219026 (0.0009) [2023-12-26 16:57:19,829][105692] Updated weights for policy 0, policy_version 218446 (0.0008) [2023-12-26 16:57:19,831][105620] Updated weights for policy 1, policy_version 219036 (0.0008) [2023-12-26 16:57:19,899][105620] Updated weights for policy 1, policy_version 219046 (0.0008) [2023-12-26 16:57:20,485][105620] Updated weights for policy 1, policy_version 219056 (0.0008) [2023-12-26 16:57:20,538][105620] Updated weights for policy 1, policy_version 219066 (0.0007) [2023-12-26 16:57:20,590][105692] Updated weights for policy 0, policy_version 218456 (0.0010) [2023-12-26 16:57:20,604][105620] Updated weights for policy 1, policy_version 219076 (0.0007) [2023-12-26 16:57:20,646][105692] Updated weights for policy 0, policy_version 218466 (0.0006) [2023-12-26 16:57:20,712][105692] Updated weights for policy 0, policy_version 218476 (0.0006) [2023-12-26 16:57:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 112033792. Throughput: 0: 9748.9, 1: 9771.3. Samples: 112022928. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:57:21,063][104569] Avg episode reward: [(0, '738.739'), (1, '9355.876')] [2023-12-26 16:57:21,415][105692] Updated weights for policy 0, policy_version 218486 (0.0008) [2023-12-26 16:57:21,428][105620] Updated weights for policy 1, policy_version 219086 (0.0008) [2023-12-26 16:57:21,474][105692] Updated weights for policy 0, policy_version 218496 (0.0009) [2023-12-26 16:57:21,482][105620] Updated weights for policy 1, policy_version 219096 (0.0008) [2023-12-26 16:57:21,525][105692] Updated weights for policy 0, policy_version 218506 (0.0009) [2023-12-26 16:57:21,543][105620] Updated weights for policy 1, policy_version 219106 (0.0007) [2023-12-26 16:57:22,299][105692] Updated weights for policy 0, policy_version 218516 (0.0008) [2023-12-26 16:57:22,313][105620] Updated weights for policy 1, policy_version 219116 (0.0009) [2023-12-26 16:57:22,357][105692] Updated weights for policy 0, policy_version 218526 (0.0008) [2023-12-26 16:57:22,380][105620] Updated weights for policy 1, policy_version 219126 (0.0011) [2023-12-26 16:57:22,424][105692] Updated weights for policy 0, policy_version 218536 (0.0008) [2023-12-26 16:57:22,447][105620] Updated weights for policy 1, policy_version 219136 (0.0008) [2023-12-26 16:57:23,185][105620] Updated weights for policy 1, policy_version 219146 (0.0008) [2023-12-26 16:57:23,222][105692] Updated weights for policy 0, policy_version 218546 (0.0009) [2023-12-26 16:57:23,232][105620] Updated weights for policy 1, policy_version 219156 (0.0008) [2023-12-26 16:57:23,263][105692] Updated weights for policy 0, policy_version 218556 (0.0007) [2023-12-26 16:57:23,285][105620] Updated weights for policy 1, policy_version 219166 (0.0008) [2023-12-26 16:57:23,326][105692] Updated weights for policy 0, policy_version 218566 (0.0008) [2023-12-26 16:57:23,343][105620] Updated weights for policy 1, policy_version 219176 (0.0007) [2023-12-26 16:57:23,381][105692] Updated weights for policy 0, policy_version 218576 (0.0008) [2023-12-26 16:57:24,018][105692] Updated weights for policy 0, policy_version 218586 (0.0006) [2023-12-26 16:57:24,076][105692] Updated weights for policy 0, policy_version 218596 (0.0008) [2023-12-26 16:57:24,110][105620] Updated weights for policy 1, policy_version 219186 (0.0011) [2023-12-26 16:57:24,136][105692] Updated weights for policy 0, policy_version 218606 (0.0006) [2023-12-26 16:57:24,174][105620] Updated weights for policy 1, policy_version 219196 (0.0010) [2023-12-26 16:57:24,228][105620] Updated weights for policy 1, policy_version 219206 (0.0007) [2023-12-26 16:57:24,878][105692] Updated weights for policy 0, policy_version 218616 (0.0007) [2023-12-26 16:57:24,899][105620] Updated weights for policy 1, policy_version 219216 (0.0009) [2023-12-26 16:57:24,924][105692] Updated weights for policy 0, policy_version 218626 (0.0005) [2023-12-26 16:57:24,962][105620] Updated weights for policy 1, policy_version 219226 (0.0009) [2023-12-26 16:57:24,972][105692] Updated weights for policy 0, policy_version 218636 (0.0005) [2023-12-26 16:57:25,023][105620] Updated weights for policy 1, policy_version 219236 (0.0009) [2023-12-26 16:57:25,560][105692] Updated weights for policy 0, policy_version 218646 (0.0005) [2023-12-26 16:57:25,625][105692] Updated weights for policy 0, policy_version 218656 (0.0009) [2023-12-26 16:57:25,683][105692] Updated weights for policy 0, policy_version 218666 (0.0010) [2023-12-26 16:57:25,854][105620] Updated weights for policy 1, policy_version 219246 (0.0008) [2023-12-26 16:57:25,901][105620] Updated weights for policy 1, policy_version 219256 (0.0008) [2023-12-26 16:57:25,950][105620] Updated weights for policy 1, policy_version 219266 (0.0008) [2023-12-26 16:57:26,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 112132096. Throughput: 0: 9690.1, 1: 9782.6. Samples: 112137736. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:57:26,062][104569] Avg episode reward: [(0, '990.568'), (1, '9354.874')] [2023-12-26 16:57:26,381][105692] Updated weights for policy 0, policy_version 218676 (0.0010) [2023-12-26 16:57:26,425][105692] Updated weights for policy 0, policy_version 218686 (0.0010) [2023-12-26 16:57:26,479][105692] Updated weights for policy 0, policy_version 218696 (0.0010) [2023-12-26 16:57:26,707][105620] Updated weights for policy 1, policy_version 219276 (0.0008) [2023-12-26 16:57:26,769][105620] Updated weights for policy 1, policy_version 219286 (0.0007) [2023-12-26 16:57:26,828][105620] Updated weights for policy 1, policy_version 219296 (0.0008) [2023-12-26 16:57:27,191][105692] Updated weights for policy 0, policy_version 218706 (0.0010) [2023-12-26 16:57:27,248][105692] Updated weights for policy 0, policy_version 218716 (0.0010) [2023-12-26 16:57:27,306][105692] Updated weights for policy 0, policy_version 218726 (0.0010) [2023-12-26 16:57:27,358][105692] Updated weights for policy 0, policy_version 218736 (0.0008) [2023-12-26 16:57:27,540][105620] Updated weights for policy 1, policy_version 219306 (0.0009) [2023-12-26 16:57:27,593][105620] Updated weights for policy 1, policy_version 219316 (0.0009) [2023-12-26 16:57:27,654][105620] Updated weights for policy 1, policy_version 219326 (0.0009) [2023-12-26 16:57:27,712][105620] Updated weights for policy 1, policy_version 219336 (0.0010) [2023-12-26 16:57:27,903][105692] Updated weights for policy 0, policy_version 218746 (0.0005) [2023-12-26 16:57:27,957][105692] Updated weights for policy 0, policy_version 218756 (0.0005) [2023-12-26 16:57:28,008][105692] Updated weights for policy 0, policy_version 218766 (0.0008) [2023-12-26 16:57:28,543][105620] Updated weights for policy 1, policy_version 219346 (0.0008) [2023-12-26 16:57:28,604][105620] Updated weights for policy 1, policy_version 219356 (0.0008) [2023-12-26 16:57:28,663][105620] Updated weights for policy 1, policy_version 219366 (0.0007) [2023-12-26 16:57:28,671][105692] Updated weights for policy 0, policy_version 218776 (0.0006) [2023-12-26 16:57:28,733][105692] Updated weights for policy 0, policy_version 218786 (0.0010) [2023-12-26 16:57:28,793][105692] Updated weights for policy 0, policy_version 218796 (0.0010) [2023-12-26 16:57:29,408][105692] Updated weights for policy 0, policy_version 218806 (0.0008) [2023-12-26 16:57:29,469][105692] Updated weights for policy 0, policy_version 218816 (0.0010) [2023-12-26 16:57:29,479][105620] Updated weights for policy 1, policy_version 219376 (0.0008) [2023-12-26 16:57:29,522][105692] Updated weights for policy 0, policy_version 218826 (0.0007) [2023-12-26 16:57:29,540][105620] Updated weights for policy 1, policy_version 219386 (0.0008) [2023-12-26 16:57:29,604][105620] Updated weights for policy 1, policy_version 219396 (0.0007) [2023-12-26 16:57:30,257][105692] Updated weights for policy 0, policy_version 218836 (0.0008) [2023-12-26 16:57:30,315][105620] Updated weights for policy 1, policy_version 219406 (0.0006) [2023-12-26 16:57:30,316][105692] Updated weights for policy 0, policy_version 218846 (0.0011) [2023-12-26 16:57:30,370][105620] Updated weights for policy 1, policy_version 219416 (0.0008) [2023-12-26 16:57:30,380][105692] Updated weights for policy 0, policy_version 218856 (0.0011) [2023-12-26 16:57:30,421][105620] Updated weights for policy 1, policy_version 219426 (0.0007) [2023-12-26 16:57:30,973][105692] Updated weights for policy 0, policy_version 218866 (0.0009) [2023-12-26 16:57:31,029][105692] Updated weights for policy 0, policy_version 218876 (0.0006) [2023-12-26 16:57:31,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 112222208. Throughput: 0: 9721.8, 1: 9736.9. Samples: 112196636. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:57:31,062][104569] Avg episode reward: [(0, '6337.334'), (1, '9353.836')] [2023-12-26 16:57:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000219432_56180736.pth... [2023-12-26 16:57:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000218312_55894016.pth [2023-12-26 16:57:31,087][105692] Updated weights for policy 0, policy_version 218886 (0.0010) [2023-12-26 16:57:31,150][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000218896_56049664.pth... [2023-12-26 16:57:31,153][105692] Updated weights for policy 0, policy_version 218896 (0.0011) [2023-12-26 16:57:31,154][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000217744_55754752.pth [2023-12-26 16:57:31,251][105620] Updated weights for policy 1, policy_version 219436 (0.0007) [2023-12-26 16:57:31,308][105620] Updated weights for policy 1, policy_version 219446 (0.0008) [2023-12-26 16:57:31,369][105620] Updated weights for policy 1, policy_version 219456 (0.0008) [2023-12-26 16:57:31,884][105692] Updated weights for policy 0, policy_version 218906 (0.0010) [2023-12-26 16:57:31,935][105692] Updated weights for policy 0, policy_version 218916 (0.0010) [2023-12-26 16:57:31,996][105692] Updated weights for policy 0, policy_version 218926 (0.0010) [2023-12-26 16:57:32,095][105620] Updated weights for policy 1, policy_version 219466 (0.0009) [2023-12-26 16:57:32,143][105620] Updated weights for policy 1, policy_version 219476 (0.0008) [2023-12-26 16:57:32,194][105620] Updated weights for policy 1, policy_version 219486 (0.0007) [2023-12-26 16:57:32,253][105620] Updated weights for policy 1, policy_version 219496 (0.0006) [2023-12-26 16:57:32,748][105692] Updated weights for policy 0, policy_version 218936 (0.0010) [2023-12-26 16:57:32,803][105692] Updated weights for policy 0, policy_version 218946 (0.0010) [2023-12-26 16:57:32,854][105692] Updated weights for policy 0, policy_version 218956 (0.0006) [2023-12-26 16:57:33,021][105620] Updated weights for policy 1, policy_version 219506 (0.0006) [2023-12-26 16:57:33,074][105620] Updated weights for policy 1, policy_version 219516 (0.0006) [2023-12-26 16:57:33,135][105620] Updated weights for policy 1, policy_version 219526 (0.0005) [2023-12-26 16:57:33,501][105692] Updated weights for policy 0, policy_version 218966 (0.0008) [2023-12-26 16:57:33,545][105692] Updated weights for policy 0, policy_version 218976 (0.0010) [2023-12-26 16:57:33,592][105692] Updated weights for policy 0, policy_version 218986 (0.0010) [2023-12-26 16:57:33,651][105620] Updated weights for policy 1, policy_version 219536 (0.0005) [2023-12-26 16:57:33,716][105620] Updated weights for policy 1, policy_version 219546 (0.0005) [2023-12-26 16:57:33,765][105620] Updated weights for policy 1, policy_version 219556 (0.0006) [2023-12-26 16:57:34,259][105692] Updated weights for policy 0, policy_version 218996 (0.0009) [2023-12-26 16:57:34,310][105692] Updated weights for policy 0, policy_version 219006 (0.0008) [2023-12-26 16:57:34,371][105692] Updated weights for policy 0, policy_version 219016 (0.0008) [2023-12-26 16:57:34,396][105620] Updated weights for policy 1, policy_version 219566 (0.0007) [2023-12-26 16:57:34,458][105620] Updated weights for policy 1, policy_version 219576 (0.0011) [2023-12-26 16:57:34,518][105620] Updated weights for policy 1, policy_version 219586 (0.0011) [2023-12-26 16:57:35,149][105620] Updated weights for policy 1, policy_version 219596 (0.0008) [2023-12-26 16:57:35,182][105692] Updated weights for policy 0, policy_version 219026 (0.0008) [2023-12-26 16:57:35,206][105620] Updated weights for policy 1, policy_version 219606 (0.0006) [2023-12-26 16:57:35,230][105692] Updated weights for policy 0, policy_version 219036 (0.0010) [2023-12-26 16:57:35,275][105620] Updated weights for policy 1, policy_version 219616 (0.0005) [2023-12-26 16:57:35,291][105692] Updated weights for policy 0, policy_version 219046 (0.0007) [2023-12-26 16:57:35,354][105692] Updated weights for policy 0, policy_version 219056 (0.0006) [2023-12-26 16:57:35,798][105620] Updated weights for policy 1, policy_version 219626 (0.0005) [2023-12-26 16:57:35,856][105620] Updated weights for policy 1, policy_version 219636 (0.0006) [2023-12-26 16:57:35,915][105620] Updated weights for policy 1, policy_version 219646 (0.0006) [2023-12-26 16:57:35,976][105620] Updated weights for policy 1, policy_version 219656 (0.0006) [2023-12-26 16:57:36,056][105692] Updated weights for policy 0, policy_version 219066 (0.0010) [2023-12-26 16:57:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 112328704. Throughput: 0: 9787.0, 1: 9736.5. Samples: 112316708. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:57:36,062][104569] Avg episode reward: [(0, '8663.787'), (1, '9263.296')] [2023-12-26 16:57:36,122][105692] Updated weights for policy 0, policy_version 219076 (0.0007) [2023-12-26 16:57:36,182][105692] Updated weights for policy 0, policy_version 219086 (0.0006) [2023-12-26 16:57:36,620][105620] Updated weights for policy 1, policy_version 219666 (0.0008) [2023-12-26 16:57:36,681][105620] Updated weights for policy 1, policy_version 219676 (0.0008) [2023-12-26 16:57:36,746][105620] Updated weights for policy 1, policy_version 219686 (0.0008) [2023-12-26 16:57:36,929][105692] Updated weights for policy 0, policy_version 219096 (0.0006) [2023-12-26 16:57:36,988][105692] Updated weights for policy 0, policy_version 219106 (0.0005) [2023-12-26 16:57:37,042][105692] Updated weights for policy 0, policy_version 219116 (0.0005) [2023-12-26 16:57:37,494][105620] Updated weights for policy 1, policy_version 219696 (0.0010) [2023-12-26 16:57:37,563][105620] Updated weights for policy 1, policy_version 219706 (0.0010) [2023-12-26 16:57:37,625][105620] Updated weights for policy 1, policy_version 219716 (0.0010) [2023-12-26 16:57:37,692][105692] Updated weights for policy 0, policy_version 219126 (0.0009) [2023-12-26 16:57:37,742][105692] Updated weights for policy 0, policy_version 219136 (0.0011) [2023-12-26 16:57:37,795][105692] Updated weights for policy 0, policy_version 219146 (0.0011) [2023-12-26 16:57:38,360][105620] Updated weights for policy 1, policy_version 219726 (0.0008) [2023-12-26 16:57:38,429][105620] Updated weights for policy 1, policy_version 219736 (0.0007) [2023-12-26 16:57:38,490][105620] Updated weights for policy 1, policy_version 219746 (0.0005) [2023-12-26 16:57:38,599][105692] Updated weights for policy 0, policy_version 219156 (0.0011) [2023-12-26 16:57:38,662][105692] Updated weights for policy 0, policy_version 219166 (0.0011) [2023-12-26 16:57:38,723][105692] Updated weights for policy 0, policy_version 219176 (0.0011) [2023-12-26 16:57:39,035][105620] Updated weights for policy 1, policy_version 219756 (0.0006) [2023-12-26 16:57:39,085][105620] Updated weights for policy 1, policy_version 219766 (0.0007) [2023-12-26 16:57:39,152][105620] Updated weights for policy 1, policy_version 219776 (0.0006) [2023-12-26 16:57:39,512][105692] Updated weights for policy 0, policy_version 219186 (0.0011) [2023-12-26 16:57:39,575][105692] Updated weights for policy 0, policy_version 219196 (0.0011) [2023-12-26 16:57:39,638][105692] Updated weights for policy 0, policy_version 219206 (0.0011) [2023-12-26 16:57:39,697][105692] Updated weights for policy 0, policy_version 219216 (0.0010) [2023-12-26 16:57:39,904][105620] Updated weights for policy 1, policy_version 219786 (0.0010) [2023-12-26 16:57:39,967][105620] Updated weights for policy 1, policy_version 219796 (0.0011) [2023-12-26 16:57:40,024][105620] Updated weights for policy 1, policy_version 219806 (0.0008) [2023-12-26 16:57:40,075][105620] Updated weights for policy 1, policy_version 219816 (0.0010) [2023-12-26 16:57:40,388][105692] Updated weights for policy 0, policy_version 219226 (0.0011) [2023-12-26 16:57:40,447][105692] Updated weights for policy 0, policy_version 219236 (0.0010) [2023-12-26 16:57:40,505][105692] Updated weights for policy 0, policy_version 219246 (0.0010) [2023-12-26 16:57:40,802][105620] Updated weights for policy 1, policy_version 219826 (0.0011) [2023-12-26 16:57:40,865][105620] Updated weights for policy 1, policy_version 219836 (0.0010) [2023-12-26 16:57:40,928][105620] Updated weights for policy 1, policy_version 219846 (0.0011) [2023-12-26 16:57:41,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 112427008. Throughput: 0: 9753.6, 1: 9729.1. Samples: 112434700. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:57:41,063][104569] Avg episode reward: [(0, '8088.672'), (1, '9039.860')] [2023-12-26 16:57:41,196][105692] Updated weights for policy 0, policy_version 219256 (0.0008) [2023-12-26 16:57:41,272][105692] Updated weights for policy 0, policy_version 219266 (0.0008) [2023-12-26 16:57:41,331][105692] Updated weights for policy 0, policy_version 219276 (0.0010) [2023-12-26 16:57:41,652][105620] Updated weights for policy 1, policy_version 219856 (0.0010) [2023-12-26 16:57:41,714][105620] Updated weights for policy 1, policy_version 219866 (0.0010) [2023-12-26 16:57:41,778][105620] Updated weights for policy 1, policy_version 219876 (0.0011) [2023-12-26 16:57:42,078][105692] Updated weights for policy 0, policy_version 219286 (0.0008) [2023-12-26 16:57:42,140][105692] Updated weights for policy 0, policy_version 219296 (0.0006) [2023-12-26 16:57:42,204][105692] Updated weights for policy 0, policy_version 219306 (0.0005) [2023-12-26 16:57:42,543][105620] Updated weights for policy 1, policy_version 219886 (0.0011) [2023-12-26 16:57:42,592][105620] Updated weights for policy 1, policy_version 219896 (0.0010) [2023-12-26 16:57:42,651][105620] Updated weights for policy 1, policy_version 219906 (0.0008) [2023-12-26 16:57:42,927][105692] Updated weights for policy 0, policy_version 219316 (0.0009) [2023-12-26 16:57:42,989][105692] Updated weights for policy 0, policy_version 219326 (0.0010) [2023-12-26 16:57:43,056][105692] Updated weights for policy 0, policy_version 219336 (0.0009) [2023-12-26 16:57:43,377][105620] Updated weights for policy 1, policy_version 219916 (0.0007) [2023-12-26 16:57:43,433][105620] Updated weights for policy 1, policy_version 219926 (0.0005) [2023-12-26 16:57:43,501][105620] Updated weights for policy 1, policy_version 219936 (0.0005) [2023-12-26 16:57:43,788][105692] Updated weights for policy 0, policy_version 219346 (0.0011) [2023-12-26 16:57:43,846][105692] Updated weights for policy 0, policy_version 219356 (0.0011) [2023-12-26 16:57:43,898][105692] Updated weights for policy 0, policy_version 219366 (0.0010) [2023-12-26 16:57:43,958][105692] Updated weights for policy 0, policy_version 219376 (0.0011) [2023-12-26 16:57:44,000][105620] Updated weights for policy 1, policy_version 219946 (0.0005) [2023-12-26 16:57:44,053][105620] Updated weights for policy 1, policy_version 219956 (0.0006) [2023-12-26 16:57:44,102][105620] Updated weights for policy 1, policy_version 219966 (0.0005) [2023-12-26 16:57:44,164][105620] Updated weights for policy 1, policy_version 219976 (0.0006) [2023-12-26 16:57:44,734][105692] Updated weights for policy 0, policy_version 219386 (0.0011) [2023-12-26 16:57:44,779][105620] Updated weights for policy 1, policy_version 219986 (0.0006) [2023-12-26 16:57:44,800][105692] Updated weights for policy 0, policy_version 219396 (0.0009) [2023-12-26 16:57:44,835][105620] Updated weights for policy 1, policy_version 219996 (0.0006) [2023-12-26 16:57:44,863][105692] Updated weights for policy 0, policy_version 219406 (0.0008) [2023-12-26 16:57:44,891][105620] Updated weights for policy 1, policy_version 220006 (0.0007) [2023-12-26 16:57:45,524][105620] Updated weights for policy 1, policy_version 220016 (0.0007) [2023-12-26 16:57:45,582][105620] Updated weights for policy 1, policy_version 220026 (0.0005) [2023-12-26 16:57:45,631][105692] Updated weights for policy 0, policy_version 219416 (0.0010) [2023-12-26 16:57:45,633][105620] Updated weights for policy 1, policy_version 220036 (0.0005) [2023-12-26 16:57:45,675][105692] Updated weights for policy 0, policy_version 219426 (0.0007) [2023-12-26 16:57:45,723][105692] Updated weights for policy 0, policy_version 219436 (0.0009) [2023-12-26 16:57:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.4, 300 sec: 19549.7). Total num frames: 112525312. Throughput: 0: 9705.2, 1: 9753.4. Samples: 112493652. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:57:46,062][104569] Avg episode reward: [(0, '8101.447'), (1, '9040.197')] [2023-12-26 16:57:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000219440_56188928.pth... [2023-12-26 16:57:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000220040_56336384.pth... [2023-12-26 16:57:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000218856_56033280.pth [2023-12-26 16:57:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000218288_55894016.pth [2023-12-26 16:57:46,297][105620] Updated weights for policy 1, policy_version 220046 (0.0008) [2023-12-26 16:57:46,345][105620] Updated weights for policy 1, policy_version 220056 (0.0009) [2023-12-26 16:57:46,396][105620] Updated weights for policy 1, policy_version 220066 (0.0008) [2023-12-26 16:57:46,436][105692] Updated weights for policy 0, policy_version 219446 (0.0008) [2023-12-26 16:57:46,486][105692] Updated weights for policy 0, policy_version 219456 (0.0009) [2023-12-26 16:57:46,544][105692] Updated weights for policy 0, policy_version 219466 (0.0009) [2023-12-26 16:57:47,153][105692] Updated weights for policy 0, policy_version 219476 (0.0009) [2023-12-26 16:57:47,201][105692] Updated weights for policy 0, policy_version 219486 (0.0010) [2023-12-26 16:57:47,239][105620] Updated weights for policy 1, policy_version 220076 (0.0006) [2023-12-26 16:57:47,249][105692] Updated weights for policy 0, policy_version 219496 (0.0010) [2023-12-26 16:57:47,299][105620] Updated weights for policy 1, policy_version 220086 (0.0006) [2023-12-26 16:57:47,350][105620] Updated weights for policy 1, policy_version 220097 (0.0008) [2023-12-26 16:57:47,857][105692] Updated weights for policy 0, policy_version 219506 (0.0009) [2023-12-26 16:57:47,909][105692] Updated weights for policy 0, policy_version 219516 (0.0005) [2023-12-26 16:57:47,970][105692] Updated weights for policy 0, policy_version 219526 (0.0005) [2023-12-26 16:57:48,034][105692] Updated weights for policy 0, policy_version 219536 (0.0005) [2023-12-26 16:57:48,239][105620] Updated weights for policy 1, policy_version 220107 (0.0008) [2023-12-26 16:57:48,299][105620] Updated weights for policy 1, policy_version 220117 (0.0007) [2023-12-26 16:57:48,366][105620] Updated weights for policy 1, policy_version 220127 (0.0006) [2023-12-26 16:57:48,661][105692] Updated weights for policy 0, policy_version 219546 (0.0011) [2023-12-26 16:57:48,719][105692] Updated weights for policy 0, policy_version 219556 (0.0010) [2023-12-26 16:57:48,782][105692] Updated weights for policy 0, policy_version 219566 (0.0011) [2023-12-26 16:57:48,950][105620] Updated weights for policy 1, policy_version 220137 (0.0007) [2023-12-26 16:57:49,012][105620] Updated weights for policy 1, policy_version 220147 (0.0008) [2023-12-26 16:57:49,075][105620] Updated weights for policy 1, policy_version 220157 (0.0010) [2023-12-26 16:57:49,137][105620] Updated weights for policy 1, policy_version 220167 (0.0011) [2023-12-26 16:57:49,468][105692] Updated weights for policy 0, policy_version 219576 (0.0009) [2023-12-26 16:57:49,520][105692] Updated weights for policy 0, policy_version 219586 (0.0008) [2023-12-26 16:57:49,577][105692] Updated weights for policy 0, policy_version 219596 (0.0008) [2023-12-26 16:57:49,807][105620] Updated weights for policy 1, policy_version 220177 (0.0010) [2023-12-26 16:57:49,876][105620] Updated weights for policy 1, policy_version 220187 (0.0009) [2023-12-26 16:57:49,948][105620] Updated weights for policy 1, policy_version 220197 (0.0009) [2023-12-26 16:57:50,313][105692] Updated weights for policy 0, policy_version 219606 (0.0009) [2023-12-26 16:57:50,368][105692] Updated weights for policy 0, policy_version 219616 (0.0009) [2023-12-26 16:57:50,430][105692] Updated weights for policy 0, policy_version 219626 (0.0009) [2023-12-26 16:57:50,695][105620] Updated weights for policy 1, policy_version 220207 (0.0011) [2023-12-26 16:57:50,757][105620] Updated weights for policy 1, policy_version 220217 (0.0010) [2023-12-26 16:57:50,821][105620] Updated weights for policy 1, policy_version 220227 (0.0009) [2023-12-26 16:57:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 112623616. Throughput: 0: 9752.1, 1: 9791.3. Samples: 112613252. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:57:51,062][104569] Avg episode reward: [(0, '9013.606'), (1, '5153.997')] [2023-12-26 16:57:51,081][105692] Updated weights for policy 0, policy_version 219636 (0.0006) [2023-12-26 16:57:51,134][105692] Updated weights for policy 0, policy_version 219646 (0.0008) [2023-12-26 16:57:51,194][105692] Updated weights for policy 0, policy_version 219656 (0.0009) [2023-12-26 16:57:51,588][105620] Updated weights for policy 1, policy_version 220237 (0.0009) [2023-12-26 16:57:51,652][105620] Updated weights for policy 1, policy_version 220247 (0.0008) [2023-12-26 16:57:51,710][105586] KL-divergence is very high: 146.7572 [2023-12-26 16:57:51,720][105620] Updated weights for policy 1, policy_version 220257 (0.0008) [2023-12-26 16:57:51,736][105586] KL-divergence is very high: 159.3375 [2023-12-26 16:57:51,999][105692] Updated weights for policy 0, policy_version 219666 (0.0009) [2023-12-26 16:57:52,059][105692] Updated weights for policy 0, policy_version 219676 (0.0008) [2023-12-26 16:57:52,123][105692] Updated weights for policy 0, policy_version 219686 (0.0008) [2023-12-26 16:57:52,186][105692] Updated weights for policy 0, policy_version 219696 (0.0008) [2023-12-26 16:57:52,472][105620] Updated weights for policy 1, policy_version 220267 (0.0009) [2023-12-26 16:57:52,520][105620] Updated weights for policy 1, policy_version 220277 (0.0010) [2023-12-26 16:57:52,565][105620] Updated weights for policy 1, policy_version 220287 (0.0010) [2023-12-26 16:57:52,956][105692] Updated weights for policy 0, policy_version 219706 (0.0010) [2023-12-26 16:57:53,014][105692] Updated weights for policy 0, policy_version 219716 (0.0010) [2023-12-26 16:57:53,062][105692] Updated weights for policy 0, policy_version 219726 (0.0010) [2023-12-26 16:57:53,297][105620] Updated weights for policy 1, policy_version 220297 (0.0010) [2023-12-26 16:57:53,348][105620] Updated weights for policy 1, policy_version 220307 (0.0009) [2023-12-26 16:57:53,405][105620] Updated weights for policy 1, policy_version 220317 (0.0009) [2023-12-26 16:57:53,458][105620] Updated weights for policy 1, policy_version 220328 (0.0010) [2023-12-26 16:57:53,723][105692] Updated weights for policy 0, policy_version 219736 (0.0006) [2023-12-26 16:57:53,782][105692] Updated weights for policy 0, policy_version 219746 (0.0005) [2023-12-26 16:57:53,840][105692] Updated weights for policy 0, policy_version 219756 (0.0005) [2023-12-26 16:57:54,318][105620] Updated weights for policy 1, policy_version 220338 (0.0009) [2023-12-26 16:57:54,380][105620] Updated weights for policy 1, policy_version 220348 (0.0010) [2023-12-26 16:57:54,439][105620] Updated weights for policy 1, policy_version 220358 (0.0005) [2023-12-26 16:57:54,440][105692] Updated weights for policy 0, policy_version 219766 (0.0007) [2023-12-26 16:57:54,491][105692] Updated weights for policy 0, policy_version 219776 (0.0009) [2023-12-26 16:57:54,546][105692] Updated weights for policy 0, policy_version 219786 (0.0009) [2023-12-26 16:57:55,115][105620] Updated weights for policy 1, policy_version 220368 (0.0008) [2023-12-26 16:57:55,165][105620] Updated weights for policy 1, policy_version 220378 (0.0009) [2023-12-26 16:57:55,226][105620] Updated weights for policy 1, policy_version 220388 (0.0010) [2023-12-26 16:57:55,308][105692] Updated weights for policy 0, policy_version 219796 (0.0008) [2023-12-26 16:57:55,367][105692] Updated weights for policy 0, policy_version 219806 (0.0005) [2023-12-26 16:57:55,416][105692] Updated weights for policy 0, policy_version 219816 (0.0005) [2023-12-26 16:57:55,948][105692] Updated weights for policy 0, policy_version 219826 (0.0006) [2023-12-26 16:57:55,996][105620] Updated weights for policy 1, policy_version 220398 (0.0007) [2023-12-26 16:57:56,012][105692] Updated weights for policy 0, policy_version 219836 (0.0008) [2023-12-26 16:57:56,054][105620] Updated weights for policy 1, policy_version 220408 (0.0006) [2023-12-26 16:57:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 112713728. Throughput: 0: 9821.1, 1: 9790.6. Samples: 112728472. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:57:56,063][104569] Avg episode reward: [(0, '9008.718'), (1, '4939.165')] [2023-12-26 16:57:56,074][105692] Updated weights for policy 0, policy_version 219846 (0.0008) [2023-12-26 16:57:56,103][105620] Updated weights for policy 1, policy_version 220418 (0.0008) [2023-12-26 16:57:56,140][105692] Updated weights for policy 0, policy_version 219856 (0.0008) [2023-12-26 16:57:56,686][105692] Updated weights for policy 0, policy_version 219866 (0.0008) [2023-12-26 16:57:56,743][105692] Updated weights for policy 0, policy_version 219876 (0.0007) [2023-12-26 16:57:56,770][105620] Updated weights for policy 1, policy_version 220428 (0.0008) [2023-12-26 16:57:56,800][105692] Updated weights for policy 0, policy_version 219886 (0.0007) [2023-12-26 16:57:56,833][105620] Updated weights for policy 1, policy_version 220438 (0.0011) [2023-12-26 16:57:56,884][105620] Updated weights for policy 1, policy_version 220448 (0.0010) [2023-12-26 16:57:57,334][105692] Updated weights for policy 0, policy_version 219896 (0.0007) [2023-12-26 16:57:57,391][105692] Updated weights for policy 0, policy_version 219906 (0.0006) [2023-12-26 16:57:57,450][105692] Updated weights for policy 0, policy_version 219916 (0.0005) [2023-12-26 16:57:57,527][105620] Updated weights for policy 1, policy_version 220458 (0.0009) [2023-12-26 16:57:57,585][105620] Updated weights for policy 1, policy_version 220468 (0.0005) [2023-12-26 16:57:57,631][105620] Updated weights for policy 1, policy_version 220478 (0.0005) [2023-12-26 16:57:57,685][105620] Updated weights for policy 1, policy_version 220488 (0.0005) [2023-12-26 16:57:58,142][105692] Updated weights for policy 0, policy_version 219926 (0.0011) [2023-12-26 16:57:58,205][105692] Updated weights for policy 0, policy_version 219936 (0.0011) [2023-12-26 16:57:58,272][105692] Updated weights for policy 0, policy_version 219946 (0.0011) [2023-12-26 16:57:58,325][105620] Updated weights for policy 1, policy_version 220498 (0.0006) [2023-12-26 16:57:58,395][105620] Updated weights for policy 1, policy_version 220508 (0.0007) [2023-12-26 16:57:58,454][105620] Updated weights for policy 1, policy_version 220518 (0.0010) [2023-12-26 16:57:59,074][105692] Updated weights for policy 0, policy_version 219956 (0.0010) [2023-12-26 16:57:59,138][105692] Updated weights for policy 0, policy_version 219966 (0.0009) [2023-12-26 16:57:59,192][105692] Updated weights for policy 0, policy_version 219976 (0.0007) [2023-12-26 16:57:59,291][105620] Updated weights for policy 1, policy_version 220528 (0.0009) [2023-12-26 16:57:59,358][105620] Updated weights for policy 1, policy_version 220538 (0.0009) [2023-12-26 16:57:59,416][105620] Updated weights for policy 1, policy_version 220548 (0.0007) [2023-12-26 16:57:59,929][105692] Updated weights for policy 0, policy_version 219986 (0.0008) [2023-12-26 16:57:59,987][105692] Updated weights for policy 0, policy_version 219996 (0.0010) [2023-12-26 16:58:00,047][105692] Updated weights for policy 0, policy_version 220006 (0.0006) [2023-12-26 16:58:00,056][105620] Updated weights for policy 1, policy_version 220558 (0.0007) [2023-12-26 16:58:00,114][105692] Updated weights for policy 0, policy_version 220016 (0.0005) [2023-12-26 16:58:00,119][105620] Updated weights for policy 1, policy_version 220568 (0.0008) [2023-12-26 16:58:00,173][105620] Updated weights for policy 1, policy_version 220578 (0.0007) [2023-12-26 16:58:00,795][105692] Updated weights for policy 0, policy_version 220026 (0.0010) [2023-12-26 16:58:00,845][105692] Updated weights for policy 0, policy_version 220036 (0.0010) [2023-12-26 16:58:00,870][105620] Updated weights for policy 1, policy_version 220588 (0.0006) [2023-12-26 16:58:00,899][105692] Updated weights for policy 0, policy_version 220046 (0.0010) [2023-12-26 16:58:00,934][105620] Updated weights for policy 1, policy_version 220598 (0.0007) [2023-12-26 16:58:01,001][105620] Updated weights for policy 1, policy_version 220608 (0.0007) [2023-12-26 16:58:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 112828416. Throughput: 0: 9936.4, 1: 9842.8. Samples: 112791724. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:58:01,062][104569] Avg episode reward: [(0, '9007.731'), (1, '7330.775')] [2023-12-26 16:58:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000220048_56344576.pth... [2023-12-26 16:58:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000220616_56483840.pth... [2023-12-26 16:58:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000218896_56049664.pth [2023-12-26 16:58:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000219432_56180736.pth [2023-12-26 16:58:01,602][105692] Updated weights for policy 0, policy_version 220056 (0.0006) [2023-12-26 16:58:01,634][105620] Updated weights for policy 1, policy_version 220618 (0.0008) [2023-12-26 16:58:01,668][105692] Updated weights for policy 0, policy_version 220066 (0.0008) [2023-12-26 16:58:01,703][105620] Updated weights for policy 1, policy_version 220628 (0.0010) [2023-12-26 16:58:01,731][105692] Updated weights for policy 0, policy_version 220076 (0.0008) [2023-12-26 16:58:01,756][105620] Updated weights for policy 1, policy_version 220638 (0.0006) [2023-12-26 16:58:01,805][105620] Updated weights for policy 1, policy_version 220648 (0.0005) [2023-12-26 16:58:02,445][105620] Updated weights for policy 1, policy_version 220658 (0.0007) [2023-12-26 16:58:02,498][105620] Updated weights for policy 1, policy_version 220668 (0.0005) [2023-12-26 16:58:02,517][105692] Updated weights for policy 0, policy_version 220086 (0.0007) [2023-12-26 16:58:02,547][105620] Updated weights for policy 1, policy_version 220678 (0.0010) [2023-12-26 16:58:02,565][105692] Updated weights for policy 0, policy_version 220096 (0.0006) [2023-12-26 16:58:02,624][105692] Updated weights for policy 0, policy_version 220106 (0.0008) [2023-12-26 16:58:03,192][105620] Updated weights for policy 1, policy_version 220688 (0.0010) [2023-12-26 16:58:03,242][105692] Updated weights for policy 0, policy_version 220116 (0.0007) [2023-12-26 16:58:03,243][105620] Updated weights for policy 1, policy_version 220698 (0.0010) [2023-12-26 16:58:03,305][105620] Updated weights for policy 1, policy_version 220708 (0.0010) [2023-12-26 16:58:03,305][105692] Updated weights for policy 0, policy_version 220126 (0.0006) [2023-12-26 16:58:03,362][105692] Updated weights for policy 0, policy_version 220136 (0.0006) [2023-12-26 16:58:03,990][105692] Updated weights for policy 0, policy_version 220146 (0.0009) [2023-12-26 16:58:04,020][105620] Updated weights for policy 1, policy_version 220718 (0.0009) [2023-12-26 16:58:04,045][105692] Updated weights for policy 0, policy_version 220156 (0.0010) [2023-12-26 16:58:04,082][105620] Updated weights for policy 1, policy_version 220728 (0.0010) [2023-12-26 16:58:04,097][105692] Updated weights for policy 0, policy_version 220166 (0.0010) [2023-12-26 16:58:04,141][105620] Updated weights for policy 1, policy_version 220738 (0.0010) [2023-12-26 16:58:04,150][105692] Updated weights for policy 0, policy_version 220176 (0.0011) [2023-12-26 16:58:04,857][105620] Updated weights for policy 1, policy_version 220748 (0.0010) [2023-12-26 16:58:04,874][105692] Updated weights for policy 0, policy_version 220186 (0.0008) [2023-12-26 16:58:04,909][105620] Updated weights for policy 1, policy_version 220758 (0.0010) [2023-12-26 16:58:04,934][105692] Updated weights for policy 0, policy_version 220196 (0.0011) [2023-12-26 16:58:04,967][105620] Updated weights for policy 1, policy_version 220768 (0.0010) [2023-12-26 16:58:04,992][105692] Updated weights for policy 0, policy_version 220206 (0.0010) [2023-12-26 16:58:05,642][105692] Updated weights for policy 0, policy_version 220216 (0.0010) [2023-12-26 16:58:05,697][105692] Updated weights for policy 0, policy_version 220226 (0.0010) [2023-12-26 16:58:05,723][105620] Updated weights for policy 1, policy_version 220778 (0.0009) [2023-12-26 16:58:05,754][105692] Updated weights for policy 0, policy_version 220236 (0.0010) [2023-12-26 16:58:05,777][105620] Updated weights for policy 1, policy_version 220788 (0.0006) [2023-12-26 16:58:05,836][105620] Updated weights for policy 1, policy_version 220798 (0.0006) [2023-12-26 16:58:05,883][105620] Updated weights for policy 1, policy_version 220808 (0.0005) [2023-12-26 16:58:06,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 112926720. Throughput: 0: 9957.3, 1: 9804.3. Samples: 112912196. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 16:58:06,063][104569] Avg episode reward: [(0, '9189.430'), (1, '9351.104')] [2023-12-26 16:58:06,475][105620] Updated weights for policy 1, policy_version 220818 (0.0011) [2023-12-26 16:58:06,490][105692] Updated weights for policy 0, policy_version 220246 (0.0011) [2023-12-26 16:58:06,535][105620] Updated weights for policy 1, policy_version 220828 (0.0011) [2023-12-26 16:58:06,550][105692] Updated weights for policy 0, policy_version 220256 (0.0011) [2023-12-26 16:58:06,593][105620] Updated weights for policy 1, policy_version 220838 (0.0008) [2023-12-26 16:58:06,607][105692] Updated weights for policy 0, policy_version 220266 (0.0011) [2023-12-26 16:58:07,247][105620] Updated weights for policy 1, policy_version 220848 (0.0010) [2023-12-26 16:58:07,251][105692] Updated weights for policy 0, policy_version 220276 (0.0009) [2023-12-26 16:58:07,307][105692] Updated weights for policy 0, policy_version 220286 (0.0010) [2023-12-26 16:58:07,308][105620] Updated weights for policy 1, policy_version 220858 (0.0011) [2023-12-26 16:58:07,371][105620] Updated weights for policy 1, policy_version 220868 (0.0009) [2023-12-26 16:58:07,371][105692] Updated weights for policy 0, policy_version 220296 (0.0011) [2023-12-26 16:58:07,964][105692] Updated weights for policy 0, policy_version 220306 (0.0010) [2023-12-26 16:58:08,019][105692] Updated weights for policy 0, policy_version 220316 (0.0007) [2023-12-26 16:58:08,080][105692] Updated weights for policy 0, policy_version 220326 (0.0008) [2023-12-26 16:58:08,093][105620] Updated weights for policy 1, policy_version 220878 (0.0011) [2023-12-26 16:58:08,144][105692] Updated weights for policy 0, policy_version 220336 (0.0010) [2023-12-26 16:58:08,145][105620] Updated weights for policy 1, policy_version 220888 (0.0010) [2023-12-26 16:58:08,203][105620] Updated weights for policy 1, policy_version 220898 (0.0010) [2023-12-26 16:58:08,808][105692] Updated weights for policy 0, policy_version 220346 (0.0008) [2023-12-26 16:58:08,874][105692] Updated weights for policy 0, policy_version 220356 (0.0008) [2023-12-26 16:58:08,906][105620] Updated weights for policy 1, policy_version 220908 (0.0010) [2023-12-26 16:58:08,932][105692] Updated weights for policy 0, policy_version 220366 (0.0007) [2023-12-26 16:58:08,962][105620] Updated weights for policy 1, policy_version 220918 (0.0011) [2023-12-26 16:58:09,025][105620] Updated weights for policy 1, policy_version 220928 (0.0011) [2023-12-26 16:58:09,690][105692] Updated weights for policy 0, policy_version 220376 (0.0008) [2023-12-26 16:58:09,745][105620] Updated weights for policy 1, policy_version 220938 (0.0010) [2023-12-26 16:58:09,758][105692] Updated weights for policy 0, policy_version 220386 (0.0008) [2023-12-26 16:58:09,793][105620] Updated weights for policy 1, policy_version 220948 (0.0006) [2023-12-26 16:58:09,816][105692] Updated weights for policy 0, policy_version 220396 (0.0008) [2023-12-26 16:58:09,854][105620] Updated weights for policy 1, policy_version 220958 (0.0008) [2023-12-26 16:58:09,914][105620] Updated weights for policy 1, policy_version 220968 (0.0006) [2023-12-26 16:58:10,549][105692] Updated weights for policy 0, policy_version 220406 (0.0006) [2023-12-26 16:58:10,614][105692] Updated weights for policy 0, policy_version 220416 (0.0006) [2023-12-26 16:58:10,625][105620] Updated weights for policy 1, policy_version 220978 (0.0009) [2023-12-26 16:58:10,669][105692] Updated weights for policy 0, policy_version 220426 (0.0006) [2023-12-26 16:58:10,671][105620] Updated weights for policy 1, policy_version 220988 (0.0011) [2023-12-26 16:58:10,729][105620] Updated weights for policy 1, policy_version 220998 (0.0009) [2023-12-26 16:58:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 113025024. Throughput: 0: 10000.5, 1: 9875.2. Samples: 113032144. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-26 16:58:11,062][104569] Avg episode reward: [(0, '9277.255'), (1, '9169.377')] [2023-12-26 16:58:11,409][105692] Updated weights for policy 0, policy_version 220436 (0.0008) [2023-12-26 16:58:11,460][105620] Updated weights for policy 1, policy_version 221008 (0.0010) [2023-12-26 16:58:11,473][105692] Updated weights for policy 0, policy_version 220446 (0.0007) [2023-12-26 16:58:11,520][105620] Updated weights for policy 1, policy_version 221018 (0.0010) [2023-12-26 16:58:11,533][105692] Updated weights for policy 0, policy_version 220456 (0.0007) [2023-12-26 16:58:11,580][105620] Updated weights for policy 1, policy_version 221028 (0.0007) [2023-12-26 16:58:12,266][105692] Updated weights for policy 0, policy_version 220466 (0.0008) [2023-12-26 16:58:12,301][105620] Updated weights for policy 1, policy_version 221038 (0.0006) [2023-12-26 16:58:12,326][105692] Updated weights for policy 0, policy_version 220476 (0.0008) [2023-12-26 16:58:12,369][105620] Updated weights for policy 1, policy_version 221048 (0.0010) [2023-12-26 16:58:12,389][105692] Updated weights for policy 0, policy_version 220486 (0.0007) [2023-12-26 16:58:12,432][105620] Updated weights for policy 1, policy_version 221058 (0.0008) [2023-12-26 16:58:12,445][105692] Updated weights for policy 0, policy_version 220496 (0.0005) [2023-12-26 16:58:13,060][105692] Updated weights for policy 0, policy_version 220506 (0.0007) [2023-12-26 16:58:13,108][105692] Updated weights for policy 0, policy_version 220516 (0.0007) [2023-12-26 16:58:13,156][105692] Updated weights for policy 0, policy_version 220526 (0.0007) [2023-12-26 16:58:13,248][105620] Updated weights for policy 1, policy_version 221068 (0.0009) [2023-12-26 16:58:13,299][105620] Updated weights for policy 1, policy_version 221078 (0.0010) [2023-12-26 16:58:13,357][105620] Updated weights for policy 1, policy_version 221088 (0.0010) [2023-12-26 16:58:13,843][105692] Updated weights for policy 0, policy_version 220536 (0.0007) [2023-12-26 16:58:13,905][105692] Updated weights for policy 0, policy_version 220546 (0.0008) [2023-12-26 16:58:13,963][105692] Updated weights for policy 0, policy_version 220556 (0.0007) [2023-12-26 16:58:14,109][105620] Updated weights for policy 1, policy_version 221098 (0.0010) [2023-12-26 16:58:14,163][105620] Updated weights for policy 1, policy_version 221108 (0.0010) [2023-12-26 16:58:14,214][105620] Updated weights for policy 1, policy_version 221118 (0.0010) [2023-12-26 16:58:14,266][105620] Updated weights for policy 1, policy_version 221128 (0.0010) [2023-12-26 16:58:14,570][105692] Updated weights for policy 0, policy_version 220566 (0.0008) [2023-12-26 16:58:14,626][105692] Updated weights for policy 0, policy_version 220576 (0.0008) [2023-12-26 16:58:14,684][105692] Updated weights for policy 0, policy_version 220586 (0.0009) [2023-12-26 16:58:15,037][105620] Updated weights for policy 1, policy_version 221138 (0.0009) [2023-12-26 16:58:15,094][105620] Updated weights for policy 1, policy_version 221148 (0.0008) [2023-12-26 16:58:15,145][105620] Updated weights for policy 1, policy_version 221158 (0.0008) [2023-12-26 16:58:15,477][105692] Updated weights for policy 0, policy_version 220596 (0.0009) [2023-12-26 16:58:15,533][105692] Updated weights for policy 0, policy_version 220606 (0.0009) [2023-12-26 16:58:15,585][105692] Updated weights for policy 0, policy_version 220616 (0.0009) [2023-12-26 16:58:15,892][105620] Updated weights for policy 1, policy_version 221168 (0.0008) [2023-12-26 16:58:15,943][105620] Updated weights for policy 1, policy_version 221178 (0.0009) [2023-12-26 16:58:15,989][105620] Updated weights for policy 1, policy_version 221188 (0.0008) [2023-12-26 16:58:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19933.9, 300 sec: 19605.2). Total num frames: 113123328. Throughput: 0: 9966.6, 1: 9886.3. Samples: 113090020. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-26 16:58:16,063][104569] Avg episode reward: [(0, '9358.506'), (1, '9135.332')] [2023-12-26 16:58:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000220624_56492032.pth... [2023-12-26 16:58:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000221192_56631296.pth... [2023-12-26 16:58:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000220040_56336384.pth [2023-12-26 16:58:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000219440_56188928.pth [2023-12-26 16:58:16,366][105692] Updated weights for policy 0, policy_version 220626 (0.0009) [2023-12-26 16:58:16,420][105692] Updated weights for policy 0, policy_version 220636 (0.0009) [2023-12-26 16:58:16,472][105692] Updated weights for policy 0, policy_version 220646 (0.0009) [2023-12-26 16:58:16,524][105692] Updated weights for policy 0, policy_version 220656 (0.0009) [2023-12-26 16:58:16,795][105620] Updated weights for policy 1, policy_version 221198 (0.0009) [2023-12-26 16:58:16,847][105620] Updated weights for policy 1, policy_version 221208 (0.0009) [2023-12-26 16:58:16,915][105620] Updated weights for policy 1, policy_version 221218 (0.0009) [2023-12-26 16:58:17,155][105692] Updated weights for policy 0, policy_version 220666 (0.0009) [2023-12-26 16:58:17,204][105692] Updated weights for policy 0, policy_version 220676 (0.0006) [2023-12-26 16:58:17,253][105692] Updated weights for policy 0, policy_version 220686 (0.0009) [2023-12-26 16:58:17,660][105620] Updated weights for policy 1, policy_version 221228 (0.0009) [2023-12-26 16:58:17,730][105620] Updated weights for policy 1, policy_version 221238 (0.0009) [2023-12-26 16:58:17,790][105620] Updated weights for policy 1, policy_version 221248 (0.0005) [2023-12-26 16:58:17,972][105692] Updated weights for policy 0, policy_version 220696 (0.0010) [2023-12-26 16:58:18,035][105692] Updated weights for policy 0, policy_version 220706 (0.0011) [2023-12-26 16:58:18,093][105692] Updated weights for policy 0, policy_version 220716 (0.0011) [2023-12-26 16:58:18,469][105620] Updated weights for policy 1, policy_version 221258 (0.0006) [2023-12-26 16:58:18,531][105620] Updated weights for policy 1, policy_version 221268 (0.0008) [2023-12-26 16:58:18,580][105620] Updated weights for policy 1, policy_version 221278 (0.0008) [2023-12-26 16:58:18,635][105620] Updated weights for policy 1, policy_version 221288 (0.0008) [2023-12-26 16:58:18,834][105692] Updated weights for policy 0, policy_version 220726 (0.0011) [2023-12-26 16:58:18,882][105692] Updated weights for policy 0, policy_version 220736 (0.0011) [2023-12-26 16:58:18,938][105692] Updated weights for policy 0, policy_version 220746 (0.0011) [2023-12-26 16:58:19,432][105620] Updated weights for policy 1, policy_version 221298 (0.0008) [2023-12-26 16:58:19,490][105620] Updated weights for policy 1, policy_version 221308 (0.0008) [2023-12-26 16:58:19,553][105620] Updated weights for policy 1, policy_version 221318 (0.0008) [2023-12-26 16:58:19,718][105692] Updated weights for policy 0, policy_version 220756 (0.0011) [2023-12-26 16:58:19,784][105692] Updated weights for policy 0, policy_version 220766 (0.0011) [2023-12-26 16:58:19,852][105692] Updated weights for policy 0, policy_version 220776 (0.0011) [2023-12-26 16:58:20,347][105620] Updated weights for policy 1, policy_version 221328 (0.0008) [2023-12-26 16:58:20,403][105620] Updated weights for policy 1, policy_version 221338 (0.0009) [2023-12-26 16:58:20,456][105620] Updated weights for policy 1, policy_version 221348 (0.0008) [2023-12-26 16:58:20,607][105692] Updated weights for policy 0, policy_version 220786 (0.0010) [2023-12-26 16:58:20,656][105692] Updated weights for policy 0, policy_version 220796 (0.0010) [2023-12-26 16:58:20,711][105692] Updated weights for policy 0, policy_version 220806 (0.0010) [2023-12-26 16:58:20,772][105692] Updated weights for policy 0, policy_version 220816 (0.0011) [2023-12-26 16:58:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 113213440. Throughput: 0: 9914.8, 1: 9802.8. Samples: 113204004. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-26 16:58:21,062][104569] Avg episode reward: [(0, '9269.195'), (1, '9226.676')] [2023-12-26 16:58:21,246][105620] Updated weights for policy 1, policy_version 221358 (0.0008) [2023-12-26 16:58:21,313][105620] Updated weights for policy 1, policy_version 221368 (0.0008) [2023-12-26 16:58:21,381][105620] Updated weights for policy 1, policy_version 221378 (0.0008) [2023-12-26 16:58:21,561][105692] Updated weights for policy 0, policy_version 220826 (0.0010) [2023-12-26 16:58:21,624][105692] Updated weights for policy 0, policy_version 220836 (0.0011) [2023-12-26 16:58:21,693][105692] Updated weights for policy 0, policy_version 220846 (0.0010) [2023-12-26 16:58:22,137][105620] Updated weights for policy 1, policy_version 221388 (0.0008) [2023-12-26 16:58:22,193][105620] Updated weights for policy 1, policy_version 221398 (0.0008) [2023-12-26 16:58:22,248][105620] Updated weights for policy 1, policy_version 221408 (0.0008) [2023-12-26 16:58:22,479][105692] Updated weights for policy 0, policy_version 220856 (0.0011) [2023-12-26 16:58:22,542][105692] Updated weights for policy 0, policy_version 220866 (0.0011) [2023-12-26 16:58:22,602][105692] Updated weights for policy 0, policy_version 220876 (0.0011) [2023-12-26 16:58:23,029][105620] Updated weights for policy 1, policy_version 221418 (0.0008) [2023-12-26 16:58:23,089][105620] Updated weights for policy 1, policy_version 221428 (0.0008) [2023-12-26 16:58:23,150][105620] Updated weights for policy 1, policy_version 221438 (0.0008) [2023-12-26 16:58:23,213][105620] Updated weights for policy 1, policy_version 221448 (0.0008) [2023-12-26 16:58:23,355][105692] Updated weights for policy 0, policy_version 220886 (0.0008) [2023-12-26 16:58:23,420][105692] Updated weights for policy 0, policy_version 220896 (0.0009) [2023-12-26 16:58:23,478][105692] Updated weights for policy 0, policy_version 220906 (0.0011) [2023-12-26 16:58:23,893][105620] Updated weights for policy 1, policy_version 221458 (0.0009) [2023-12-26 16:58:23,944][105620] Updated weights for policy 1, policy_version 221468 (0.0010) [2023-12-26 16:58:23,988][105620] Updated weights for policy 1, policy_version 221478 (0.0006) [2023-12-26 16:58:24,137][105692] Updated weights for policy 0, policy_version 220916 (0.0010) [2023-12-26 16:58:24,188][105692] Updated weights for policy 0, policy_version 220926 (0.0009) [2023-12-26 16:58:24,236][105692] Updated weights for policy 0, policy_version 220936 (0.0009) [2023-12-26 16:58:24,691][105620] Updated weights for policy 1, policy_version 221488 (0.0007) [2023-12-26 16:58:24,738][105620] Updated weights for policy 1, policy_version 221498 (0.0009) [2023-12-26 16:58:24,788][105620] Updated weights for policy 1, policy_version 221508 (0.0009) [2023-12-26 16:58:25,027][105692] Updated weights for policy 0, policy_version 220946 (0.0009) [2023-12-26 16:58:25,078][105692] Updated weights for policy 0, policy_version 220956 (0.0009) [2023-12-26 16:58:25,137][105692] Updated weights for policy 0, policy_version 220966 (0.0009) [2023-12-26 16:58:25,197][105692] Updated weights for policy 0, policy_version 220976 (0.0010) [2023-12-26 16:58:25,503][105620] Updated weights for policy 1, policy_version 221518 (0.0008) [2023-12-26 16:58:25,572][105620] Updated weights for policy 1, policy_version 221528 (0.0009) [2023-12-26 16:58:25,632][105620] Updated weights for policy 1, policy_version 221538 (0.0009) [2023-12-26 16:58:25,965][105692] Updated weights for policy 0, policy_version 220986 (0.0009) [2023-12-26 16:58:26,027][105692] Updated weights for policy 0, policy_version 220996 (0.0009) [2023-12-26 16:58:26,062][104569] Fps is (10 sec: 18022.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 113303552. Throughput: 0: 9881.4, 1: 9691.1. Samples: 113315460. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-26 16:58:26,062][104569] Avg episode reward: [(0, '9268.860'), (1, '3753.219')] [2023-12-26 16:58:26,085][105692] Updated weights for policy 0, policy_version 221006 (0.0009) [2023-12-26 16:58:26,385][105620] Updated weights for policy 1, policy_version 221548 (0.0008) [2023-12-26 16:58:26,435][105620] Updated weights for policy 1, policy_version 221558 (0.0009) [2023-12-26 16:58:26,481][105620] Updated weights for policy 1, policy_version 221568 (0.0008) [2023-12-26 16:58:26,831][105692] Updated weights for policy 0, policy_version 221016 (0.0009) [2023-12-26 16:58:26,882][105692] Updated weights for policy 0, policy_version 221026 (0.0009) [2023-12-26 16:58:26,934][105692] Updated weights for policy 0, policy_version 221036 (0.0009) [2023-12-26 16:58:27,222][105620] Updated weights for policy 1, policy_version 221578 (0.0009) [2023-12-26 16:58:27,268][105620] Updated weights for policy 1, policy_version 221588 (0.0008) [2023-12-26 16:58:27,318][105620] Updated weights for policy 1, policy_version 221598 (0.0008) [2023-12-26 16:58:27,372][105620] Updated weights for policy 1, policy_version 221608 (0.0009) [2023-12-26 16:58:27,676][105692] Updated weights for policy 0, policy_version 221046 (0.0007) [2023-12-26 16:58:27,732][105692] Updated weights for policy 0, policy_version 221056 (0.0005) [2023-12-26 16:58:27,785][105692] Updated weights for policy 0, policy_version 221066 (0.0009) [2023-12-26 16:58:28,150][105620] Updated weights for policy 1, policy_version 221618 (0.0008) [2023-12-26 16:58:28,204][105620] Updated weights for policy 1, policy_version 221628 (0.0009) [2023-12-26 16:58:28,253][105620] Updated weights for policy 1, policy_version 221638 (0.0009) [2023-12-26 16:58:28,461][105692] Updated weights for policy 0, policy_version 221076 (0.0008) [2023-12-26 16:58:28,532][105692] Updated weights for policy 0, policy_version 221086 (0.0006) [2023-12-26 16:58:28,593][105692] Updated weights for policy 0, policy_version 221096 (0.0008) [2023-12-26 16:58:29,087][105620] Updated weights for policy 1, policy_version 221648 (0.0008) [2023-12-26 16:58:29,144][105620] Updated weights for policy 1, policy_version 221658 (0.0009) [2023-12-26 16:58:29,193][105620] Updated weights for policy 1, policy_version 221668 (0.0008) [2023-12-26 16:58:29,238][105692] Updated weights for policy 0, policy_version 221106 (0.0008) [2023-12-26 16:58:29,304][105692] Updated weights for policy 0, policy_version 221116 (0.0010) [2023-12-26 16:58:29,368][105692] Updated weights for policy 0, policy_version 221126 (0.0010) [2023-12-26 16:58:29,435][105692] Updated weights for policy 0, policy_version 221136 (0.0007) [2023-12-26 16:58:29,962][105620] Updated weights for policy 1, policy_version 221678 (0.0009) [2023-12-26 16:58:30,022][105620] Updated weights for policy 1, policy_version 221688 (0.0009) [2023-12-26 16:58:30,065][105692] Updated weights for policy 0, policy_version 221146 (0.0007) [2023-12-26 16:58:30,074][105620] Updated weights for policy 1, policy_version 221698 (0.0007) [2023-12-26 16:58:30,122][105692] Updated weights for policy 0, policy_version 221156 (0.0006) [2023-12-26 16:58:30,176][105692] Updated weights for policy 0, policy_version 221166 (0.0009) [2023-12-26 16:58:30,855][105620] Updated weights for policy 1, policy_version 221708 (0.0007) [2023-12-26 16:58:30,881][105692] Updated weights for policy 0, policy_version 221176 (0.0008) [2023-12-26 16:58:30,921][105620] Updated weights for policy 1, policy_version 221718 (0.0009) [2023-12-26 16:58:30,946][105692] Updated weights for policy 0, policy_version 221186 (0.0006) [2023-12-26 16:58:30,979][105620] Updated weights for policy 1, policy_version 221728 (0.0007) [2023-12-26 16:58:30,994][105692] Updated weights for policy 0, policy_version 221196 (0.0006) [2023-12-26 16:58:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 113410048. Throughput: 0: 9903.1, 1: 9633.1. Samples: 113372780. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-26 16:58:31,062][104569] Avg episode reward: [(0, '9356.275'), (1, '3547.185')] [2023-12-26 16:58:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000221200_56639488.pth... [2023-12-26 16:58:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000221736_56770560.pth... [2023-12-26 16:58:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000220616_56483840.pth [2023-12-26 16:58:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000220048_56344576.pth [2023-12-26 16:58:31,726][105620] Updated weights for policy 1, policy_version 221738 (0.0007) [2023-12-26 16:58:31,773][105692] Updated weights for policy 0, policy_version 221206 (0.0008) [2023-12-26 16:58:31,792][105620] Updated weights for policy 1, policy_version 221748 (0.0006) [2023-12-26 16:58:31,828][105692] Updated weights for policy 0, policy_version 221216 (0.0008) [2023-12-26 16:58:31,856][105620] Updated weights for policy 1, policy_version 221758 (0.0006) [2023-12-26 16:58:31,891][105692] Updated weights for policy 0, policy_version 221226 (0.0008) [2023-12-26 16:58:31,923][105620] Updated weights for policy 1, policy_version 221768 (0.0005) [2023-12-26 16:58:32,593][105620] Updated weights for policy 1, policy_version 221778 (0.0006) [2023-12-26 16:58:32,659][105620] Updated weights for policy 1, policy_version 221788 (0.0006) [2023-12-26 16:58:32,678][105692] Updated weights for policy 0, policy_version 221236 (0.0007) [2023-12-26 16:58:32,720][105620] Updated weights for policy 1, policy_version 221798 (0.0008) [2023-12-26 16:58:32,730][105692] Updated weights for policy 0, policy_version 221246 (0.0008) [2023-12-26 16:58:32,777][105692] Updated weights for policy 0, policy_version 221256 (0.0008) [2023-12-26 16:58:33,275][105620] Updated weights for policy 1, policy_version 221808 (0.0006) [2023-12-26 16:58:33,328][105620] Updated weights for policy 1, policy_version 221818 (0.0005) [2023-12-26 16:58:33,395][105620] Updated weights for policy 1, policy_version 221828 (0.0005) [2023-12-26 16:58:33,521][105692] Updated weights for policy 0, policy_version 221266 (0.0007) [2023-12-26 16:58:33,566][105692] Updated weights for policy 0, policy_version 221276 (0.0005) [2023-12-26 16:58:33,611][105692] Updated weights for policy 0, policy_version 221286 (0.0005) [2023-12-26 16:58:33,664][105692] Updated weights for policy 0, policy_version 221296 (0.0005) [2023-12-26 16:58:33,948][105620] Updated weights for policy 1, policy_version 221838 (0.0005) [2023-12-26 16:58:34,006][105620] Updated weights for policy 1, policy_version 221848 (0.0005) [2023-12-26 16:58:34,057][105620] Updated weights for policy 1, policy_version 221858 (0.0005) [2023-12-26 16:58:34,330][105692] Updated weights for policy 0, policy_version 221306 (0.0006) [2023-12-26 16:58:34,382][105692] Updated weights for policy 0, policy_version 221316 (0.0008) [2023-12-26 16:58:34,443][105692] Updated weights for policy 0, policy_version 221326 (0.0008) [2023-12-26 16:58:34,703][105620] Updated weights for policy 1, policy_version 221868 (0.0008) [2023-12-26 16:58:34,763][105620] Updated weights for policy 1, policy_version 221878 (0.0011) [2023-12-26 16:58:34,822][105620] Updated weights for policy 1, policy_version 221888 (0.0011) [2023-12-26 16:58:35,077][105692] Updated weights for policy 0, policy_version 221336 (0.0008) [2023-12-26 16:58:35,131][105692] Updated weights for policy 0, policy_version 221346 (0.0008) [2023-12-26 16:58:35,183][105692] Updated weights for policy 0, policy_version 221356 (0.0006) [2023-12-26 16:58:35,558][105620] Updated weights for policy 1, policy_version 221898 (0.0010) [2023-12-26 16:58:35,610][105620] Updated weights for policy 1, policy_version 221908 (0.0011) [2023-12-26 16:58:35,665][105620] Updated weights for policy 1, policy_version 221918 (0.0010) [2023-12-26 16:58:35,720][105620] Updated weights for policy 1, policy_version 221928 (0.0010) [2023-12-26 16:58:35,743][105692] Updated weights for policy 0, policy_version 221366 (0.0006) [2023-12-26 16:58:35,798][105692] Updated weights for policy 0, policy_version 221376 (0.0008) [2023-12-26 16:58:35,841][105692] Updated weights for policy 0, policy_version 221386 (0.0006) [2023-12-26 16:58:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 113508352. Throughput: 0: 9853.5, 1: 9660.3. Samples: 113491376. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-26 16:58:36,063][104569] Avg episode reward: [(0, '8557.948'), (1, '5005.170')] [2023-12-26 16:58:36,464][105620] Updated weights for policy 1, policy_version 221938 (0.0011) [2023-12-26 16:58:36,513][105620] Updated weights for policy 1, policy_version 221948 (0.0011) [2023-12-26 16:58:36,514][105692] Updated weights for policy 0, policy_version 221396 (0.0007) [2023-12-26 16:58:36,568][105692] Updated weights for policy 0, policy_version 221406 (0.0009) [2023-12-26 16:58:36,573][105620] Updated weights for policy 1, policy_version 221958 (0.0011) [2023-12-26 16:58:36,627][105692] Updated weights for policy 0, policy_version 221416 (0.0010) [2023-12-26 16:58:37,316][105692] Updated weights for policy 0, policy_version 221426 (0.0010) [2023-12-26 16:58:37,329][105620] Updated weights for policy 1, policy_version 221968 (0.0008) [2023-12-26 16:58:37,367][105692] Updated weights for policy 0, policy_version 221436 (0.0010) [2023-12-26 16:58:37,388][105620] Updated weights for policy 1, policy_version 221978 (0.0005) [2023-12-26 16:58:37,422][105692] Updated weights for policy 0, policy_version 221446 (0.0010) [2023-12-26 16:58:37,450][105620] Updated weights for policy 1, policy_version 221988 (0.0005) [2023-12-26 16:58:37,481][105692] Updated weights for policy 0, policy_version 221456 (0.0010) [2023-12-26 16:58:38,086][105692] Updated weights for policy 0, policy_version 221466 (0.0007) [2023-12-26 16:58:38,135][105620] Updated weights for policy 1, policy_version 221998 (0.0006) [2023-12-26 16:58:38,141][105692] Updated weights for policy 0, policy_version 221476 (0.0010) [2023-12-26 16:58:38,194][105620] Updated weights for policy 1, policy_version 222008 (0.0005) [2023-12-26 16:58:38,199][105692] Updated weights for policy 0, policy_version 221486 (0.0010) [2023-12-26 16:58:38,254][105620] Updated weights for policy 1, policy_version 222018 (0.0007) [2023-12-26 16:58:38,863][105692] Updated weights for policy 0, policy_version 221496 (0.0010) [2023-12-26 16:58:38,912][105692] Updated weights for policy 0, policy_version 221506 (0.0010) [2023-12-26 16:58:38,965][105692] Updated weights for policy 0, policy_version 221516 (0.0010) [2023-12-26 16:58:39,007][105620] Updated weights for policy 1, policy_version 222028 (0.0008) [2023-12-26 16:58:39,063][105620] Updated weights for policy 1, policy_version 222038 (0.0008) [2023-12-26 16:58:39,125][105620] Updated weights for policy 1, policy_version 222048 (0.0009) [2023-12-26 16:58:39,707][105692] Updated weights for policy 0, policy_version 221526 (0.0011) [2023-12-26 16:58:39,770][105692] Updated weights for policy 0, policy_version 221536 (0.0011) [2023-12-26 16:58:39,833][105692] Updated weights for policy 0, policy_version 221546 (0.0011) [2023-12-26 16:58:39,866][105620] Updated weights for policy 1, policy_version 222058 (0.0008) [2023-12-26 16:58:39,933][105620] Updated weights for policy 1, policy_version 222068 (0.0008) [2023-12-26 16:58:39,997][105620] Updated weights for policy 1, policy_version 222078 (0.0009) [2023-12-26 16:58:40,060][105620] Updated weights for policy 1, policy_version 222088 (0.0009) [2023-12-26 16:58:40,538][105692] Updated weights for policy 0, policy_version 221556 (0.0009) [2023-12-26 16:58:40,588][105692] Updated weights for policy 0, policy_version 221566 (0.0010) [2023-12-26 16:58:40,642][105692] Updated weights for policy 0, policy_version 221576 (0.0011) [2023-12-26 16:58:40,841][105620] Updated weights for policy 1, policy_version 222098 (0.0008) [2023-12-26 16:58:40,899][105620] Updated weights for policy 1, policy_version 222108 (0.0010) [2023-12-26 16:58:40,960][105620] Updated weights for policy 1, policy_version 222118 (0.0009) [2023-12-26 16:58:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 113606656. Throughput: 0: 9946.9, 1: 9678.1. Samples: 113611596. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-26 16:58:41,062][104569] Avg episode reward: [(0, '8624.400'), (1, '7861.195')] [2023-12-26 16:58:41,355][105692] Updated weights for policy 0, policy_version 221586 (0.0009) [2023-12-26 16:58:41,429][105692] Updated weights for policy 0, policy_version 221596 (0.0007) [2023-12-26 16:58:41,495][105692] Updated weights for policy 0, policy_version 221606 (0.0006) [2023-12-26 16:58:41,562][105692] Updated weights for policy 0, policy_version 221616 (0.0007) [2023-12-26 16:58:41,719][105620] Updated weights for policy 1, policy_version 222128 (0.0009) [2023-12-26 16:58:41,786][105620] Updated weights for policy 1, policy_version 222138 (0.0008) [2023-12-26 16:58:41,838][105620] Updated weights for policy 1, policy_version 222148 (0.0009) [2023-12-26 16:58:42,232][105692] Updated weights for policy 0, policy_version 221626 (0.0006) [2023-12-26 16:58:42,301][105692] Updated weights for policy 0, policy_version 221636 (0.0007) [2023-12-26 16:58:42,370][105692] Updated weights for policy 0, policy_version 221646 (0.0007) [2023-12-26 16:58:42,600][105620] Updated weights for policy 1, policy_version 222158 (0.0007) [2023-12-26 16:58:42,660][105620] Updated weights for policy 1, policy_version 222168 (0.0007) [2023-12-26 16:58:42,717][105620] Updated weights for policy 1, policy_version 222178 (0.0007) [2023-12-26 16:58:42,974][105692] Updated weights for policy 0, policy_version 221656 (0.0009) [2023-12-26 16:58:43,024][105692] Updated weights for policy 0, policy_version 221666 (0.0007) [2023-12-26 16:58:43,083][105692] Updated weights for policy 0, policy_version 221676 (0.0009) [2023-12-26 16:58:43,327][105620] Updated weights for policy 1, policy_version 222188 (0.0009) [2023-12-26 16:58:43,390][105620] Updated weights for policy 1, policy_version 222198 (0.0009) [2023-12-26 16:58:43,451][105620] Updated weights for policy 1, policy_version 222208 (0.0009) [2023-12-26 16:58:43,769][105692] Updated weights for policy 0, policy_version 221686 (0.0009) [2023-12-26 16:58:43,823][105692] Updated weights for policy 0, policy_version 221696 (0.0010) [2023-12-26 16:58:43,877][105692] Updated weights for policy 0, policy_version 221706 (0.0010) [2023-12-26 16:58:44,013][105620] Updated weights for policy 1, policy_version 222218 (0.0008) [2023-12-26 16:58:44,058][105620] Updated weights for policy 1, policy_version 222228 (0.0005) [2023-12-26 16:58:44,114][105620] Updated weights for policy 1, policy_version 222238 (0.0005) [2023-12-26 16:58:44,169][105620] Updated weights for policy 1, policy_version 222248 (0.0005) [2023-12-26 16:58:44,736][105692] Updated weights for policy 0, policy_version 221716 (0.0009) [2023-12-26 16:58:44,804][105692] Updated weights for policy 0, policy_version 221726 (0.0009) [2023-12-26 16:58:44,866][105692] Updated weights for policy 0, policy_version 221736 (0.0010) [2023-12-26 16:58:44,872][105620] Updated weights for policy 1, policy_version 222258 (0.0008) [2023-12-26 16:58:44,941][105620] Updated weights for policy 1, policy_version 222268 (0.0006) [2023-12-26 16:58:45,008][105620] Updated weights for policy 1, policy_version 222278 (0.0008) [2023-12-26 16:58:45,554][105692] Updated weights for policy 0, policy_version 221746 (0.0010) [2023-12-26 16:58:45,610][105692] Updated weights for policy 0, policy_version 221756 (0.0005) [2023-12-26 16:58:45,667][105692] Updated weights for policy 0, policy_version 221766 (0.0005) [2023-12-26 16:58:45,727][105692] Updated weights for policy 0, policy_version 221776 (0.0006) [2023-12-26 16:58:45,816][105620] Updated weights for policy 1, policy_version 222288 (0.0005) [2023-12-26 16:58:45,887][105620] Updated weights for policy 1, policy_version 222298 (0.0005) [2023-12-26 16:58:45,950][105620] Updated weights for policy 1, policy_version 222308 (0.0005) [2023-12-26 16:58:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 113704960. Throughput: 0: 9888.0, 1: 9687.2. Samples: 113672616. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-26 16:58:46,063][104569] Avg episode reward: [(0, '9357.240'), (1, '9264.326')] [2023-12-26 16:58:46,073][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000221776_56786944.pth... [2023-12-26 16:58:46,074][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000222312_56918016.pth... [2023-12-26 16:58:46,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000220624_56492032.pth [2023-12-26 16:58:46,082][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000221192_56631296.pth [2023-12-26 16:58:46,264][105692] Updated weights for policy 0, policy_version 221786 (0.0005) [2023-12-26 16:58:46,321][105692] Updated weights for policy 0, policy_version 221796 (0.0008) [2023-12-26 16:58:46,375][105692] Updated weights for policy 0, policy_version 221806 (0.0010) [2023-12-26 16:58:46,650][105620] Updated weights for policy 1, policy_version 222318 (0.0008) [2023-12-26 16:58:46,712][105620] Updated weights for policy 1, policy_version 222328 (0.0010) [2023-12-26 16:58:46,766][105620] Updated weights for policy 1, policy_version 222338 (0.0010) [2023-12-26 16:58:46,950][105692] Updated weights for policy 0, policy_version 221816 (0.0010) [2023-12-26 16:58:47,011][105692] Updated weights for policy 0, policy_version 221826 (0.0010) [2023-12-26 16:58:47,072][105692] Updated weights for policy 0, policy_version 221836 (0.0006) [2023-12-26 16:58:47,493][105620] Updated weights for policy 1, policy_version 222348 (0.0010) [2023-12-26 16:58:47,545][105620] Updated weights for policy 1, policy_version 222358 (0.0006) [2023-12-26 16:58:47,594][105620] Updated weights for policy 1, policy_version 222368 (0.0006) [2023-12-26 16:58:47,657][105692] Updated weights for policy 0, policy_version 221846 (0.0007) [2023-12-26 16:58:47,716][105692] Updated weights for policy 0, policy_version 221856 (0.0005) [2023-12-26 16:58:47,780][105692] Updated weights for policy 0, policy_version 221866 (0.0005) [2023-12-26 16:58:48,270][105620] Updated weights for policy 1, policy_version 222378 (0.0007) [2023-12-26 16:58:48,327][105620] Updated weights for policy 1, policy_version 222388 (0.0011) [2023-12-26 16:58:48,390][105620] Updated weights for policy 1, policy_version 222398 (0.0011) [2023-12-26 16:58:48,412][105692] Updated weights for policy 0, policy_version 221876 (0.0006) [2023-12-26 16:58:48,456][105620] Updated weights for policy 1, policy_version 222408 (0.0011) [2023-12-26 16:58:48,461][105692] Updated weights for policy 0, policy_version 221886 (0.0007) [2023-12-26 16:58:48,510][105692] Updated weights for policy 0, policy_version 221896 (0.0007) [2023-12-26 16:58:49,145][105620] Updated weights for policy 1, policy_version 222418 (0.0005) [2023-12-26 16:58:49,199][105620] Updated weights for policy 1, policy_version 222428 (0.0006) [2023-12-26 16:58:49,254][105620] Updated weights for policy 1, policy_version 222438 (0.0009) [2023-12-26 16:58:49,342][105692] Updated weights for policy 0, policy_version 221906 (0.0008) [2023-12-26 16:58:49,409][105692] Updated weights for policy 0, policy_version 221916 (0.0008) [2023-12-26 16:58:49,477][105692] Updated weights for policy 0, policy_version 221926 (0.0009) [2023-12-26 16:58:49,542][105692] Updated weights for policy 0, policy_version 221936 (0.0008) [2023-12-26 16:58:49,970][105620] Updated weights for policy 1, policy_version 222448 (0.0011) [2023-12-26 16:58:50,023][105620] Updated weights for policy 1, policy_version 222458 (0.0011) [2023-12-26 16:58:50,082][105620] Updated weights for policy 1, policy_version 222468 (0.0011) [2023-12-26 16:58:50,287][105692] Updated weights for policy 0, policy_version 221946 (0.0007) [2023-12-26 16:58:50,360][105692] Updated weights for policy 0, policy_version 221956 (0.0009) [2023-12-26 16:58:50,412][105692] Updated weights for policy 0, policy_version 221966 (0.0010) [2023-12-26 16:58:50,753][105620] Updated weights for policy 1, policy_version 222478 (0.0007) [2023-12-26 16:58:50,802][105620] Updated weights for policy 1, policy_version 222488 (0.0005) [2023-12-26 16:58:50,847][105620] Updated weights for policy 1, policy_version 222498 (0.0005) [2023-12-26 16:58:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 113803264. Throughput: 0: 9928.2, 1: 9615.0. Samples: 113791640. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-26 16:58:51,062][104569] Avg episode reward: [(0, '9357.687'), (1, '9266.390')] [2023-12-26 16:58:51,225][105692] Updated weights for policy 0, policy_version 221976 (0.0011) [2023-12-26 16:58:51,282][105692] Updated weights for policy 0, policy_version 221986 (0.0011) [2023-12-26 16:58:51,339][105692] Updated weights for policy 0, policy_version 221996 (0.0011) [2023-12-26 16:58:51,500][105620] Updated weights for policy 1, policy_version 222508 (0.0006) [2023-12-26 16:58:51,551][105620] Updated weights for policy 1, policy_version 222518 (0.0007) [2023-12-26 16:58:51,608][105620] Updated weights for policy 1, policy_version 222528 (0.0008) [2023-12-26 16:58:52,101][105692] Updated weights for policy 0, policy_version 222006 (0.0011) [2023-12-26 16:58:52,160][105692] Updated weights for policy 0, policy_version 222016 (0.0011) [2023-12-26 16:58:52,220][105692] Updated weights for policy 0, policy_version 222026 (0.0010) [2023-12-26 16:58:52,384][105620] Updated weights for policy 1, policy_version 222538 (0.0008) [2023-12-26 16:58:52,449][105620] Updated weights for policy 1, policy_version 222548 (0.0006) [2023-12-26 16:58:52,509][105620] Updated weights for policy 1, policy_version 222558 (0.0005) [2023-12-26 16:58:52,573][105620] Updated weights for policy 1, policy_version 222568 (0.0006) [2023-12-26 16:58:52,991][105692] Updated weights for policy 0, policy_version 222036 (0.0010) [2023-12-26 16:58:53,053][105692] Updated weights for policy 0, policy_version 222046 (0.0010) [2023-12-26 16:58:53,111][105692] Updated weights for policy 0, policy_version 222056 (0.0010) [2023-12-26 16:58:53,256][105620] Updated weights for policy 1, policy_version 222578 (0.0008) [2023-12-26 16:58:53,316][105620] Updated weights for policy 1, policy_version 222588 (0.0007) [2023-12-26 16:58:53,363][105620] Updated weights for policy 1, policy_version 222598 (0.0006) [2023-12-26 16:58:53,833][105692] Updated weights for policy 0, policy_version 222066 (0.0010) [2023-12-26 16:58:53,886][105692] Updated weights for policy 0, policy_version 222076 (0.0009) [2023-12-26 16:58:53,931][105620] Updated weights for policy 1, policy_version 222608 (0.0006) [2023-12-26 16:58:53,942][105692] Updated weights for policy 0, policy_version 222086 (0.0008) [2023-12-26 16:58:53,977][105620] Updated weights for policy 1, policy_version 222618 (0.0009) [2023-12-26 16:58:53,999][105692] Updated weights for policy 0, policy_version 222096 (0.0006) [2023-12-26 16:58:54,026][105620] Updated weights for policy 1, policy_version 222628 (0.0007) [2023-12-26 16:58:54,584][105620] Updated weights for policy 1, policy_version 222638 (0.0008) [2023-12-26 16:58:54,648][105620] Updated weights for policy 1, policy_version 222648 (0.0010) [2023-12-26 16:58:54,713][105620] Updated weights for policy 1, policy_version 222658 (0.0010) [2023-12-26 16:58:54,719][105692] Updated weights for policy 0, policy_version 222106 (0.0010) [2023-12-26 16:58:54,778][105692] Updated weights for policy 0, policy_version 222116 (0.0011) [2023-12-26 16:58:54,840][105692] Updated weights for policy 0, policy_version 222126 (0.0010) [2023-12-26 16:58:55,437][105620] Updated weights for policy 1, policy_version 222668 (0.0010) [2023-12-26 16:58:55,486][105692] Updated weights for policy 0, policy_version 222136 (0.0011) [2023-12-26 16:58:55,495][105620] Updated weights for policy 1, policy_version 222678 (0.0010) [2023-12-26 16:58:55,538][105692] Updated weights for policy 0, policy_version 222146 (0.0011) [2023-12-26 16:58:55,556][105620] Updated weights for policy 1, policy_version 222688 (0.0010) [2023-12-26 16:58:55,594][105692] Updated weights for policy 0, policy_version 222156 (0.0011) [2023-12-26 16:58:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 113901568. Throughput: 0: 9843.4, 1: 9687.0. Samples: 113911016. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-26 16:58:56,062][104569] Avg episode reward: [(0, '9357.190'), (1, '9266.079')] [2023-12-26 16:58:56,258][105620] Updated weights for policy 1, policy_version 222698 (0.0009) [2023-12-26 16:58:56,297][105692] Updated weights for policy 0, policy_version 222166 (0.0011) [2023-12-26 16:58:56,311][105620] Updated weights for policy 1, policy_version 222708 (0.0006) [2023-12-26 16:58:56,342][105692] Updated weights for policy 0, policy_version 222176 (0.0011) [2023-12-26 16:58:56,368][105620] Updated weights for policy 1, policy_version 222718 (0.0006) [2023-12-26 16:58:56,394][105692] Updated weights for policy 0, policy_version 222186 (0.0010) [2023-12-26 16:58:56,414][105620] Updated weights for policy 1, policy_version 222728 (0.0005) [2023-12-26 16:58:56,976][105692] Updated weights for policy 0, policy_version 222196 (0.0008) [2023-12-26 16:58:56,997][105620] Updated weights for policy 1, policy_version 222738 (0.0008) [2023-12-26 16:58:57,024][105692] Updated weights for policy 0, policy_version 222206 (0.0005) [2023-12-26 16:58:57,057][105620] Updated weights for policy 1, policy_version 222748 (0.0008) [2023-12-26 16:58:57,081][105692] Updated weights for policy 0, policy_version 222216 (0.0005) [2023-12-26 16:58:57,109][105620] Updated weights for policy 1, policy_version 222758 (0.0009) [2023-12-26 16:58:57,731][105692] Updated weights for policy 0, policy_version 222226 (0.0005) [2023-12-26 16:58:57,753][105620] Updated weights for policy 1, policy_version 222768 (0.0006) [2023-12-26 16:58:57,782][105692] Updated weights for policy 0, policy_version 222236 (0.0007) [2023-12-26 16:58:57,810][105620] Updated weights for policy 1, policy_version 222778 (0.0008) [2023-12-26 16:58:57,830][105692] Updated weights for policy 0, policy_version 222246 (0.0011) [2023-12-26 16:58:57,859][105620] Updated weights for policy 1, policy_version 222788 (0.0008) [2023-12-26 16:58:57,874][105692] Updated weights for policy 0, policy_version 222256 (0.0008) [2023-12-26 16:58:58,615][105692] Updated weights for policy 0, policy_version 222266 (0.0008) [2023-12-26 16:58:58,622][105620] Updated weights for policy 1, policy_version 222798 (0.0009) [2023-12-26 16:58:58,681][105692] Updated weights for policy 0, policy_version 222276 (0.0008) [2023-12-26 16:58:58,689][105620] Updated weights for policy 1, policy_version 222808 (0.0008) [2023-12-26 16:58:58,754][105692] Updated weights for policy 0, policy_version 222286 (0.0008) [2023-12-26 16:58:58,760][105620] Updated weights for policy 1, policy_version 222818 (0.0008) [2023-12-26 16:58:59,511][105620] Updated weights for policy 1, policy_version 222828 (0.0008) [2023-12-26 16:58:59,556][105692] Updated weights for policy 0, policy_version 222296 (0.0009) [2023-12-26 16:58:59,572][105620] Updated weights for policy 1, policy_version 222838 (0.0006) [2023-12-26 16:58:59,617][105692] Updated weights for policy 0, policy_version 222306 (0.0008) [2023-12-26 16:58:59,626][105620] Updated weights for policy 1, policy_version 222848 (0.0007) [2023-12-26 16:58:59,675][105692] Updated weights for policy 0, policy_version 222316 (0.0007) [2023-12-26 16:59:00,363][105620] Updated weights for policy 1, policy_version 222858 (0.0007) [2023-12-26 16:59:00,407][105692] Updated weights for policy 0, policy_version 222326 (0.0007) [2023-12-26 16:59:00,427][105620] Updated weights for policy 1, policy_version 222868 (0.0009) [2023-12-26 16:59:00,466][105692] Updated weights for policy 0, policy_version 222336 (0.0007) [2023-12-26 16:59:00,484][105620] Updated weights for policy 1, policy_version 222878 (0.0009) [2023-12-26 16:59:00,519][105692] Updated weights for policy 0, policy_version 222346 (0.0006) [2023-12-26 16:59:00,546][105620] Updated weights for policy 1, policy_version 222888 (0.0007) [2023-12-26 16:59:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 113999872. Throughput: 0: 9882.6, 1: 9744.6. Samples: 113973240. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-26 16:59:01,062][104569] Avg episode reward: [(0, '9357.204'), (1, '9177.015')] [2023-12-26 16:59:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000222352_56934400.pth... [2023-12-26 16:59:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000222888_57065472.pth... [2023-12-26 16:59:01,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000221200_56639488.pth [2023-12-26 16:59:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000221736_56770560.pth [2023-12-26 16:59:01,217][105620] Updated weights for policy 1, policy_version 222898 (0.0005) [2023-12-26 16:59:01,279][105620] Updated weights for policy 1, policy_version 222908 (0.0006) [2023-12-26 16:59:01,294][105692] Updated weights for policy 0, policy_version 222356 (0.0006) [2023-12-26 16:59:01,344][105620] Updated weights for policy 1, policy_version 222918 (0.0006) [2023-12-26 16:59:01,355][105692] Updated weights for policy 0, policy_version 222366 (0.0008) [2023-12-26 16:59:01,427][105692] Updated weights for policy 0, policy_version 222376 (0.0007) [2023-12-26 16:59:02,018][105620] Updated weights for policy 1, policy_version 222928 (0.0009) [2023-12-26 16:59:02,083][105620] Updated weights for policy 1, policy_version 222938 (0.0008) [2023-12-26 16:59:02,149][105620] Updated weights for policy 1, policy_version 222948 (0.0008) [2023-12-26 16:59:02,150][105692] Updated weights for policy 0, policy_version 222386 (0.0007) [2023-12-26 16:59:02,205][105692] Updated weights for policy 0, policy_version 222396 (0.0010) [2023-12-26 16:59:02,267][105692] Updated weights for policy 0, policy_version 222406 (0.0010) [2023-12-26 16:59:02,333][105692] Updated weights for policy 0, policy_version 222416 (0.0010) [2023-12-26 16:59:02,928][105620] Updated weights for policy 1, policy_version 222958 (0.0005) [2023-12-26 16:59:02,945][105692] Updated weights for policy 0, policy_version 222426 (0.0010) [2023-12-26 16:59:02,978][105620] Updated weights for policy 1, policy_version 222968 (0.0007) [2023-12-26 16:59:03,000][105692] Updated weights for policy 0, policy_version 222436 (0.0007) [2023-12-26 16:59:03,025][105620] Updated weights for policy 1, policy_version 222978 (0.0008) [2023-12-26 16:59:03,062][105692] Updated weights for policy 0, policy_version 222446 (0.0006) [2023-12-26 16:59:03,681][105692] Updated weights for policy 0, policy_version 222456 (0.0010) [2023-12-26 16:59:03,731][105692] Updated weights for policy 0, policy_version 222466 (0.0010) [2023-12-26 16:59:03,782][105692] Updated weights for policy 0, policy_version 222476 (0.0010) [2023-12-26 16:59:03,804][105620] Updated weights for policy 1, policy_version 222989 (0.0008) [2023-12-26 16:59:03,869][105620] Updated weights for policy 1, policy_version 222999 (0.0009) [2023-12-26 16:59:03,926][105620] Updated weights for policy 1, policy_version 223010 (0.0010) [2023-12-26 16:59:04,440][105692] Updated weights for policy 0, policy_version 222486 (0.0009) [2023-12-26 16:59:04,494][105692] Updated weights for policy 0, policy_version 222496 (0.0010) [2023-12-26 16:59:04,542][105692] Updated weights for policy 0, policy_version 222506 (0.0010) [2023-12-26 16:59:04,727][105620] Updated weights for policy 1, policy_version 223020 (0.0009) [2023-12-26 16:59:04,781][105620] Updated weights for policy 1, policy_version 223030 (0.0009) [2023-12-26 16:59:04,830][105620] Updated weights for policy 1, policy_version 223040 (0.0009) [2023-12-26 16:59:05,271][105692] Updated weights for policy 0, policy_version 222516 (0.0008) [2023-12-26 16:59:05,320][105692] Updated weights for policy 0, policy_version 222526 (0.0009) [2023-12-26 16:59:05,373][105692] Updated weights for policy 0, policy_version 222536 (0.0010) [2023-12-26 16:59:05,569][105620] Updated weights for policy 1, policy_version 223050 (0.0008) [2023-12-26 16:59:05,623][105620] Updated weights for policy 1, policy_version 223060 (0.0005) [2023-12-26 16:59:05,681][105620] Updated weights for policy 1, policy_version 223070 (0.0005) [2023-12-26 16:59:05,741][105620] Updated weights for policy 1, policy_version 223080 (0.0008) [2023-12-26 16:59:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 114098176. Throughput: 0: 9892.8, 1: 9757.0. Samples: 114088244. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-26 16:59:06,062][104569] Avg episode reward: [(0, '9357.287'), (1, '8659.203')] [2023-12-26 16:59:06,169][105692] Updated weights for policy 0, policy_version 222547 (0.0009) [2023-12-26 16:59:06,229][105692] Updated weights for policy 0, policy_version 222557 (0.0008) [2023-12-26 16:59:06,294][105692] Updated weights for policy 0, policy_version 222567 (0.0009) [2023-12-26 16:59:06,419][105620] Updated weights for policy 1, policy_version 223090 (0.0011) [2023-12-26 16:59:06,468][105620] Updated weights for policy 1, policy_version 223100 (0.0010) [2023-12-26 16:59:06,526][105620] Updated weights for policy 1, policy_version 223110 (0.0011) [2023-12-26 16:59:07,035][105692] Updated weights for policy 0, policy_version 222577 (0.0008) [2023-12-26 16:59:07,090][105692] Updated weights for policy 0, policy_version 222587 (0.0008) [2023-12-26 16:59:07,146][105692] Updated weights for policy 0, policy_version 222597 (0.0008) [2023-12-26 16:59:07,194][105692] Updated weights for policy 0, policy_version 222607 (0.0007) [2023-12-26 16:59:07,304][105620] Updated weights for policy 1, policy_version 223120 (0.0011) [2023-12-26 16:59:07,363][105620] Updated weights for policy 1, policy_version 223130 (0.0008) [2023-12-26 16:59:07,427][105620] Updated weights for policy 1, policy_version 223140 (0.0005) [2023-12-26 16:59:07,984][105620] Updated weights for policy 1, policy_version 223150 (0.0008) [2023-12-26 16:59:08,029][105620] Updated weights for policy 1, policy_version 223160 (0.0010) [2023-12-26 16:59:08,043][105692] Updated weights for policy 0, policy_version 222617 (0.0008) [2023-12-26 16:59:08,084][105620] Updated weights for policy 1, policy_version 223170 (0.0010) [2023-12-26 16:59:08,091][105692] Updated weights for policy 0, policy_version 222627 (0.0006) [2023-12-26 16:59:08,148][105692] Updated weights for policy 0, policy_version 222637 (0.0009) [2023-12-26 16:59:08,750][105620] Updated weights for policy 1, policy_version 223180 (0.0008) [2023-12-26 16:59:08,816][105620] Updated weights for policy 1, policy_version 223190 (0.0011) [2023-12-26 16:59:08,876][105620] Updated weights for policy 1, policy_version 223200 (0.0010) [2023-12-26 16:59:08,925][105692] Updated weights for policy 0, policy_version 222647 (0.0008) [2023-12-26 16:59:08,991][105692] Updated weights for policy 0, policy_version 222657 (0.0008) [2023-12-26 16:59:09,058][105692] Updated weights for policy 0, policy_version 222667 (0.0008) [2023-12-26 16:59:09,624][105620] Updated weights for policy 1, policy_version 223210 (0.0010) [2023-12-26 16:59:09,688][105620] Updated weights for policy 1, policy_version 223220 (0.0008) [2023-12-26 16:59:09,752][105620] Updated weights for policy 1, policy_version 223230 (0.0008) [2023-12-26 16:59:09,823][105620] Updated weights for policy 1, policy_version 223240 (0.0006) [2023-12-26 16:59:09,865][105692] Updated weights for policy 0, policy_version 222677 (0.0008) [2023-12-26 16:59:09,913][105692] Updated weights for policy 0, policy_version 222687 (0.0006) [2023-12-26 16:59:09,975][105692] Updated weights for policy 0, policy_version 222697 (0.0007) [2023-12-26 16:59:10,483][105620] Updated weights for policy 1, policy_version 223250 (0.0005) [2023-12-26 16:59:10,546][105620] Updated weights for policy 1, policy_version 223260 (0.0006) [2023-12-26 16:59:10,619][105620] Updated weights for policy 1, policy_version 223270 (0.0006) [2023-12-26 16:59:10,810][105692] Updated weights for policy 0, policy_version 222707 (0.0007) [2023-12-26 16:59:10,868][105692] Updated weights for policy 0, policy_version 222717 (0.0010) [2023-12-26 16:59:10,919][105692] Updated weights for policy 0, policy_version 222727 (0.0009) [2023-12-26 16:59:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 114196480. Throughput: 0: 9860.3, 1: 9886.4. Samples: 114204060. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-26 16:59:11,062][104569] Avg episode reward: [(0, '9357.030'), (1, '8749.888')] [2023-12-26 16:59:11,186][105620] Updated weights for policy 1, policy_version 223280 (0.0009) [2023-12-26 16:59:11,246][105620] Updated weights for policy 1, policy_version 223290 (0.0009) [2023-12-26 16:59:11,304][105620] Updated weights for policy 1, policy_version 223300 (0.0010) [2023-12-26 16:59:11,775][105692] Updated weights for policy 0, policy_version 222737 (0.0009) [2023-12-26 16:59:11,843][105692] Updated weights for policy 0, policy_version 222747 (0.0008) [2023-12-26 16:59:11,901][105692] Updated weights for policy 0, policy_version 222757 (0.0009) [2023-12-26 16:59:11,956][105692] Updated weights for policy 0, policy_version 222767 (0.0009) [2023-12-26 16:59:12,076][105620] Updated weights for policy 1, policy_version 223310 (0.0009) [2023-12-26 16:59:12,129][105620] Updated weights for policy 1, policy_version 223320 (0.0009) [2023-12-26 16:59:12,194][105620] Updated weights for policy 1, policy_version 223330 (0.0009) [2023-12-26 16:59:12,680][105692] Updated weights for policy 0, policy_version 222777 (0.0008) [2023-12-26 16:59:12,685][105585] KL-divergence is very high: 104.0781 [2023-12-26 16:59:12,732][105585] KL-divergence is very high: 184.0569 [2023-12-26 16:59:12,739][105692] Updated weights for policy 0, policy_version 222787 (0.0009) [2023-12-26 16:59:12,778][105585] KL-divergence is very high: 190.7315 [2023-12-26 16:59:12,794][105692] Updated weights for policy 0, policy_version 222797 (0.0007) [2023-12-26 16:59:12,953][105620] Updated weights for policy 1, policy_version 223340 (0.0007) [2023-12-26 16:59:13,013][105620] Updated weights for policy 1, policy_version 223350 (0.0005) [2023-12-26 16:59:13,067][105620] Updated weights for policy 1, policy_version 223360 (0.0005) [2023-12-26 16:59:13,549][105692] Updated weights for policy 0, policy_version 222807 (0.0009) [2023-12-26 16:59:13,606][105692] Updated weights for policy 0, policy_version 222817 (0.0009) [2023-12-26 16:59:13,655][105692] Updated weights for policy 0, policy_version 222828 (0.0009) [2023-12-26 16:59:13,716][105620] Updated weights for policy 1, policy_version 223370 (0.0007) [2023-12-26 16:59:13,777][105620] Updated weights for policy 1, policy_version 223380 (0.0008) [2023-12-26 16:59:13,837][105620] Updated weights for policy 1, policy_version 223390 (0.0008) [2023-12-26 16:59:13,898][105620] Updated weights for policy 1, policy_version 223400 (0.0008) [2023-12-26 16:59:14,370][105692] Updated weights for policy 0, policy_version 222838 (0.0007) [2023-12-26 16:59:14,420][105692] Updated weights for policy 0, policy_version 222848 (0.0005) [2023-12-26 16:59:14,471][105692] Updated weights for policy 0, policy_version 222858 (0.0006) [2023-12-26 16:59:14,729][105620] Updated weights for policy 1, policy_version 223410 (0.0010) [2023-12-26 16:59:14,792][105620] Updated weights for policy 1, policy_version 223420 (0.0010) [2023-12-26 16:59:14,858][105620] Updated weights for policy 1, policy_version 223430 (0.0010) [2023-12-26 16:59:15,182][105692] Updated weights for policy 0, policy_version 222868 (0.0007) [2023-12-26 16:59:15,239][105692] Updated weights for policy 0, policy_version 222878 (0.0008) [2023-12-26 16:59:15,300][105692] Updated weights for policy 0, policy_version 222888 (0.0008) [2023-12-26 16:59:15,611][105620] Updated weights for policy 1, policy_version 223440 (0.0010) [2023-12-26 16:59:15,678][105620] Updated weights for policy 1, policy_version 223450 (0.0010) [2023-12-26 16:59:15,736][105620] Updated weights for policy 1, policy_version 223460 (0.0010) [2023-12-26 16:59:16,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19387.6, 300 sec: 19605.2). Total num frames: 114286592. Throughput: 0: 9803.7, 1: 9895.0. Samples: 114259232. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-26 16:59:16,064][104569] Avg episode reward: [(0, '9177.982'), (1, '9097.885')] [2023-12-26 16:59:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000222896_57073664.pth... [2023-12-26 16:59:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000223464_57212928.pth... [2023-12-26 16:59:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000221776_56786944.pth [2023-12-26 16:59:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000222312_56918016.pth [2023-12-26 16:59:16,097][105692] Updated weights for policy 0, policy_version 222898 (0.0008) [2023-12-26 16:59:16,149][105692] Updated weights for policy 0, policy_version 222908 (0.0008) [2023-12-26 16:59:16,209][105692] Updated weights for policy 0, policy_version 222918 (0.0008) [2023-12-26 16:59:16,262][105692] Updated weights for policy 0, policy_version 222928 (0.0008) [2023-12-26 16:59:16,477][105620] Updated weights for policy 1, policy_version 223470 (0.0010) [2023-12-26 16:59:16,528][105620] Updated weights for policy 1, policy_version 223480 (0.0010) [2023-12-26 16:59:16,589][105620] Updated weights for policy 1, policy_version 223490 (0.0010) [2023-12-26 16:59:17,036][105692] Updated weights for policy 0, policy_version 222938 (0.0008) [2023-12-26 16:59:17,085][105692] Updated weights for policy 0, policy_version 222948 (0.0008) [2023-12-26 16:59:17,135][105692] Updated weights for policy 0, policy_version 222958 (0.0008) [2023-12-26 16:59:17,334][105620] Updated weights for policy 1, policy_version 223500 (0.0010) [2023-12-26 16:59:17,392][105620] Updated weights for policy 1, policy_version 223510 (0.0010) [2023-12-26 16:59:17,454][105620] Updated weights for policy 1, policy_version 223520 (0.0010) [2023-12-26 16:59:17,796][105692] Updated weights for policy 0, policy_version 222968 (0.0006) [2023-12-26 16:59:17,850][105692] Updated weights for policy 0, policy_version 222978 (0.0009) [2023-12-26 16:59:17,911][105692] Updated weights for policy 0, policy_version 222988 (0.0010) [2023-12-26 16:59:18,184][105620] Updated weights for policy 1, policy_version 223530 (0.0010) [2023-12-26 16:59:18,239][105620] Updated weights for policy 1, policy_version 223540 (0.0010) [2023-12-26 16:59:18,283][105620] Updated weights for policy 1, policy_version 223550 (0.0010) [2023-12-26 16:59:18,335][105620] Updated weights for policy 1, policy_version 223560 (0.0010) [2023-12-26 16:59:18,501][105692] Updated weights for policy 0, policy_version 222998 (0.0011) [2023-12-26 16:59:18,557][105692] Updated weights for policy 0, policy_version 223008 (0.0011) [2023-12-26 16:59:18,617][105692] Updated weights for policy 0, policy_version 223018 (0.0011) [2023-12-26 16:59:19,038][105620] Updated weights for policy 1, policy_version 223570 (0.0009) [2023-12-26 16:59:19,093][105620] Updated weights for policy 1, policy_version 223580 (0.0010) [2023-12-26 16:59:19,158][105620] Updated weights for policy 1, policy_version 223590 (0.0010) [2023-12-26 16:59:19,387][105692] Updated weights for policy 0, policy_version 223028 (0.0009) [2023-12-26 16:59:19,443][105692] Updated weights for policy 0, policy_version 223038 (0.0005) [2023-12-26 16:59:19,500][105692] Updated weights for policy 0, policy_version 223048 (0.0008) [2023-12-26 16:59:19,954][105620] Updated weights for policy 1, policy_version 223600 (0.0009) [2023-12-26 16:59:20,015][105620] Updated weights for policy 1, policy_version 223610 (0.0007) [2023-12-26 16:59:20,079][105620] Updated weights for policy 1, policy_version 223620 (0.0006) [2023-12-26 16:59:20,259][105692] Updated weights for policy 0, policy_version 223058 (0.0008) [2023-12-26 16:59:20,332][105692] Updated weights for policy 0, policy_version 223068 (0.0011) [2023-12-26 16:59:20,388][105692] Updated weights for policy 0, policy_version 223078 (0.0010) [2023-12-26 16:59:20,447][105692] Updated weights for policy 0, policy_version 223088 (0.0009) [2023-12-26 16:59:20,697][105620] Updated weights for policy 1, policy_version 223630 (0.0007) [2023-12-26 16:59:20,751][105620] Updated weights for policy 1, policy_version 223640 (0.0008) [2023-12-26 16:59:20,801][105620] Updated weights for policy 1, policy_version 223650 (0.0008) [2023-12-26 16:59:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 114384896. Throughput: 0: 9814.5, 1: 9799.8. Samples: 114374020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:59:21,062][104569] Avg episode reward: [(0, '9175.940'), (1, '8919.454')] [2023-12-26 16:59:21,180][105692] Updated weights for policy 0, policy_version 223098 (0.0009) [2023-12-26 16:59:21,237][105692] Updated weights for policy 0, policy_version 223108 (0.0011) [2023-12-26 16:59:21,300][105692] Updated weights for policy 0, policy_version 223118 (0.0011) [2023-12-26 16:59:21,603][105620] Updated weights for policy 1, policy_version 223660 (0.0007) [2023-12-26 16:59:21,676][105620] Updated weights for policy 1, policy_version 223670 (0.0008) [2023-12-26 16:59:21,747][105620] Updated weights for policy 1, policy_version 223680 (0.0008) [2023-12-26 16:59:22,039][105692] Updated weights for policy 0, policy_version 223128 (0.0009) [2023-12-26 16:59:22,098][105692] Updated weights for policy 0, policy_version 223138 (0.0008) [2023-12-26 16:59:22,161][105692] Updated weights for policy 0, policy_version 223148 (0.0008) [2023-12-26 16:59:22,454][105620] Updated weights for policy 1, policy_version 223690 (0.0008) [2023-12-26 16:59:22,502][105620] Updated weights for policy 1, policy_version 223700 (0.0009) [2023-12-26 16:59:22,551][105620] Updated weights for policy 1, policy_version 223710 (0.0008) [2023-12-26 16:59:22,612][105620] Updated weights for policy 1, policy_version 223720 (0.0009) [2023-12-26 16:59:22,947][105692] Updated weights for policy 0, policy_version 223158 (0.0008) [2023-12-26 16:59:23,002][105692] Updated weights for policy 0, policy_version 223168 (0.0009) [2023-12-26 16:59:23,061][105692] Updated weights for policy 0, policy_version 223178 (0.0009) [2023-12-26 16:59:23,383][105620] Updated weights for policy 1, policy_version 223730 (0.0009) [2023-12-26 16:59:23,441][105620] Updated weights for policy 1, policy_version 223740 (0.0009) [2023-12-26 16:59:23,495][105620] Updated weights for policy 1, policy_version 223750 (0.0010) [2023-12-26 16:59:23,817][105692] Updated weights for policy 0, policy_version 223188 (0.0009) [2023-12-26 16:59:23,875][105692] Updated weights for policy 0, policy_version 223198 (0.0009) [2023-12-26 16:59:23,931][105692] Updated weights for policy 0, policy_version 223208 (0.0009) [2023-12-26 16:59:24,166][105620] Updated weights for policy 1, policy_version 223760 (0.0011) [2023-12-26 16:59:24,218][105620] Updated weights for policy 1, policy_version 223770 (0.0010) [2023-12-26 16:59:24,269][105620] Updated weights for policy 1, policy_version 223780 (0.0010) [2023-12-26 16:59:24,711][105692] Updated weights for policy 0, policy_version 223218 (0.0009) [2023-12-26 16:59:24,766][105692] Updated weights for policy 0, policy_version 223228 (0.0008) [2023-12-26 16:59:24,820][105692] Updated weights for policy 0, policy_version 223238 (0.0008) [2023-12-26 16:59:24,869][105692] Updated weights for policy 0, policy_version 223248 (0.0007) [2023-12-26 16:59:25,012][105620] Updated weights for policy 1, policy_version 223790 (0.0010) [2023-12-26 16:59:25,064][105620] Updated weights for policy 1, policy_version 223800 (0.0010) [2023-12-26 16:59:25,111][105620] Updated weights for policy 1, policy_version 223810 (0.0009) [2023-12-26 16:59:25,612][105692] Updated weights for policy 0, policy_version 223258 (0.0008) [2023-12-26 16:59:25,674][105692] Updated weights for policy 0, policy_version 223268 (0.0009) [2023-12-26 16:59:25,739][105692] Updated weights for policy 0, policy_version 223278 (0.0008) [2023-12-26 16:59:25,834][105620] Updated weights for policy 1, policy_version 223820 (0.0009) [2023-12-26 16:59:25,881][105620] Updated weights for policy 1, policy_version 223830 (0.0010) [2023-12-26 16:59:25,939][105620] Updated weights for policy 1, policy_version 223840 (0.0005) [2023-12-26 16:59:26,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 114483200. Throughput: 0: 9628.9, 1: 9836.6. Samples: 114487548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:59:26,063][104569] Avg episode reward: [(0, '9175.907'), (1, '9178.751')] [2023-12-26 16:59:26,458][105692] Updated weights for policy 0, policy_version 223288 (0.0009) [2023-12-26 16:59:26,512][105692] Updated weights for policy 0, policy_version 223299 (0.0009) [2023-12-26 16:59:26,565][105692] Updated weights for policy 0, policy_version 223309 (0.0008) [2023-12-26 16:59:26,598][105620] Updated weights for policy 1, policy_version 223850 (0.0007) [2023-12-26 16:59:26,656][105620] Updated weights for policy 1, policy_version 223860 (0.0010) [2023-12-26 16:59:26,718][105620] Updated weights for policy 1, policy_version 223870 (0.0010) [2023-12-26 16:59:26,773][105620] Updated weights for policy 1, policy_version 223880 (0.0010) [2023-12-26 16:59:27,226][105692] Updated weights for policy 0, policy_version 223319 (0.0006) [2023-12-26 16:59:27,279][105692] Updated weights for policy 0, policy_version 223329 (0.0005) [2023-12-26 16:59:27,334][105692] Updated weights for policy 0, policy_version 223339 (0.0007) [2023-12-26 16:59:27,523][105620] Updated weights for policy 1, policy_version 223890 (0.0010) [2023-12-26 16:59:27,578][105620] Updated weights for policy 1, policy_version 223900 (0.0008) [2023-12-26 16:59:27,632][105620] Updated weights for policy 1, policy_version 223910 (0.0006) [2023-12-26 16:59:27,932][105692] Updated weights for policy 0, policy_version 223349 (0.0007) [2023-12-26 16:59:27,988][105692] Updated weights for policy 0, policy_version 223359 (0.0006) [2023-12-26 16:59:28,035][105692] Updated weights for policy 0, policy_version 223369 (0.0008) [2023-12-26 16:59:28,272][105620] Updated weights for policy 1, policy_version 223920 (0.0006) [2023-12-26 16:59:28,334][105620] Updated weights for policy 1, policy_version 223930 (0.0006) [2023-12-26 16:59:28,397][105620] Updated weights for policy 1, policy_version 223940 (0.0007) [2023-12-26 16:59:28,842][105692] Updated weights for policy 0, policy_version 223379 (0.0008) [2023-12-26 16:59:28,892][105692] Updated weights for policy 0, policy_version 223389 (0.0007) [2023-12-26 16:59:28,937][105585] KL-divergence is very high: 123.8001 [2023-12-26 16:59:28,949][105692] Updated weights for policy 0, policy_version 223399 (0.0008) [2023-12-26 16:59:28,986][105585] KL-divergence is very high: 133.8041 [2023-12-26 16:59:29,020][105620] Updated weights for policy 1, policy_version 223950 (0.0008) [2023-12-26 16:59:29,090][105620] Updated weights for policy 1, policy_version 223960 (0.0010) [2023-12-26 16:59:29,154][105620] Updated weights for policy 1, policy_version 223970 (0.0010) [2023-12-26 16:59:29,747][105692] Updated weights for policy 0, policy_version 223409 (0.0009) [2023-12-26 16:59:29,800][105692] Updated weights for policy 0, policy_version 223419 (0.0009) [2023-12-26 16:59:29,835][105620] Updated weights for policy 1, policy_version 223980 (0.0010) [2023-12-26 16:59:29,856][105692] Updated weights for policy 0, policy_version 223429 (0.0007) [2023-12-26 16:59:29,897][105620] Updated weights for policy 1, policy_version 223990 (0.0010) [2023-12-26 16:59:29,915][105692] Updated weights for policy 0, policy_version 223439 (0.0006) [2023-12-26 16:59:29,964][105620] Updated weights for policy 1, policy_version 224000 (0.0009) [2023-12-26 16:59:30,573][105692] Updated weights for policy 0, policy_version 223449 (0.0005) [2023-12-26 16:59:30,632][105692] Updated weights for policy 0, policy_version 223459 (0.0007) [2023-12-26 16:59:30,686][105692] Updated weights for policy 0, policy_version 223469 (0.0011) [2023-12-26 16:59:30,720][105620] Updated weights for policy 1, policy_version 224010 (0.0008) [2023-12-26 16:59:30,770][105620] Updated weights for policy 1, policy_version 224020 (0.0005) [2023-12-26 16:59:30,822][105620] Updated weights for policy 1, policy_version 224030 (0.0005) [2023-12-26 16:59:30,875][105620] Updated weights for policy 1, policy_version 224040 (0.0005) [2023-12-26 16:59:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 114581504. Throughput: 0: 9624.8, 1: 9853.0. Samples: 114549112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:59:31,063][104569] Avg episode reward: [(0, '9084.742'), (1, '9005.946')] [2023-12-26 16:59:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000224040_57360384.pth... [2023-12-26 16:59:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000223472_57221120.pth... [2023-12-26 16:59:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000222888_57065472.pth [2023-12-26 16:59:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000222352_56934400.pth [2023-12-26 16:59:31,492][105692] Updated weights for policy 0, policy_version 223479 (0.0008) [2023-12-26 16:59:31,523][105620] Updated weights for policy 1, policy_version 224050 (0.0008) [2023-12-26 16:59:31,547][105692] Updated weights for policy 0, policy_version 223489 (0.0006) [2023-12-26 16:59:31,574][105620] Updated weights for policy 1, policy_version 224060 (0.0008) [2023-12-26 16:59:31,602][105692] Updated weights for policy 0, policy_version 223499 (0.0007) [2023-12-26 16:59:31,638][105620] Updated weights for policy 1, policy_version 224070 (0.0009) [2023-12-26 16:59:32,342][105620] Updated weights for policy 1, policy_version 224080 (0.0011) [2023-12-26 16:59:32,403][105620] Updated weights for policy 1, policy_version 224090 (0.0010) [2023-12-26 16:59:32,424][105692] Updated weights for policy 0, policy_version 223509 (0.0007) [2023-12-26 16:59:32,463][105620] Updated weights for policy 1, policy_version 224100 (0.0008) [2023-12-26 16:59:32,475][105692] Updated weights for policy 0, policy_version 223519 (0.0008) [2023-12-26 16:59:32,538][105692] Updated weights for policy 0, policy_version 223529 (0.0009) [2023-12-26 16:59:33,075][105620] Updated weights for policy 1, policy_version 224110 (0.0005) [2023-12-26 16:59:33,128][105620] Updated weights for policy 1, policy_version 224120 (0.0007) [2023-12-26 16:59:33,182][105620] Updated weights for policy 1, policy_version 224130 (0.0010) [2023-12-26 16:59:33,349][105692] Updated weights for policy 0, policy_version 223539 (0.0008) [2023-12-26 16:59:33,406][105692] Updated weights for policy 0, policy_version 223549 (0.0008) [2023-12-26 16:59:33,450][105692] Updated weights for policy 0, policy_version 223559 (0.0008) [2023-12-26 16:59:33,886][105620] Updated weights for policy 1, policy_version 224140 (0.0010) [2023-12-26 16:59:33,929][105620] Updated weights for policy 1, policy_version 224150 (0.0010) [2023-12-26 16:59:33,976][105620] Updated weights for policy 1, policy_version 224160 (0.0010) [2023-12-26 16:59:34,187][105692] Updated weights for policy 0, policy_version 223569 (0.0008) [2023-12-26 16:59:34,252][105692] Updated weights for policy 0, policy_version 223579 (0.0007) [2023-12-26 16:59:34,324][105692] Updated weights for policy 0, policy_version 223589 (0.0008) [2023-12-26 16:59:34,388][105692] Updated weights for policy 0, policy_version 223599 (0.0008) [2023-12-26 16:59:34,716][105620] Updated weights for policy 1, policy_version 224170 (0.0010) [2023-12-26 16:59:34,774][105620] Updated weights for policy 1, policy_version 224180 (0.0010) [2023-12-26 16:59:34,832][105620] Updated weights for policy 1, policy_version 224190 (0.0010) [2023-12-26 16:59:34,889][105620] Updated weights for policy 1, policy_version 224200 (0.0010) [2023-12-26 16:59:35,128][105692] Updated weights for policy 0, policy_version 223609 (0.0008) [2023-12-26 16:59:35,179][105692] Updated weights for policy 0, policy_version 223619 (0.0009) [2023-12-26 16:59:35,233][105692] Updated weights for policy 0, policy_version 223629 (0.0010) [2023-12-26 16:59:35,509][105620] Updated weights for policy 1, policy_version 224210 (0.0005) [2023-12-26 16:59:35,576][105620] Updated weights for policy 1, policy_version 224220 (0.0005) [2023-12-26 16:59:35,643][105620] Updated weights for policy 1, policy_version 224230 (0.0005) [2023-12-26 16:59:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 114671616. Throughput: 0: 9502.6, 1: 9883.9. Samples: 114664036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:59:36,063][104569] Avg episode reward: [(0, '9175.224'), (1, '8915.715')] [2023-12-26 16:59:36,079][105692] Updated weights for policy 0, policy_version 223639 (0.0008) [2023-12-26 16:59:36,139][105692] Updated weights for policy 0, policy_version 223649 (0.0008) [2023-12-26 16:59:36,205][105692] Updated weights for policy 0, policy_version 223659 (0.0008) [2023-12-26 16:59:36,273][105620] Updated weights for policy 1, policy_version 224240 (0.0010) [2023-12-26 16:59:36,328][105620] Updated weights for policy 1, policy_version 224250 (0.0010) [2023-12-26 16:59:36,384][105620] Updated weights for policy 1, policy_version 224260 (0.0010) [2023-12-26 16:59:36,983][105692] Updated weights for policy 0, policy_version 223669 (0.0007) [2023-12-26 16:59:37,040][105692] Updated weights for policy 0, policy_version 223679 (0.0008) [2023-12-26 16:59:37,095][105692] Updated weights for policy 0, policy_version 223689 (0.0008) [2023-12-26 16:59:37,110][105620] Updated weights for policy 1, policy_version 224270 (0.0010) [2023-12-26 16:59:37,172][105620] Updated weights for policy 1, policy_version 224280 (0.0010) [2023-12-26 16:59:37,240][105620] Updated weights for policy 1, policy_version 224290 (0.0010) [2023-12-26 16:59:37,871][105692] Updated weights for policy 0, policy_version 223699 (0.0007) [2023-12-26 16:59:37,904][105620] Updated weights for policy 1, policy_version 224300 (0.0008) [2023-12-26 16:59:37,924][105692] Updated weights for policy 0, policy_version 223709 (0.0007) [2023-12-26 16:59:37,959][105620] Updated weights for policy 1, policy_version 224310 (0.0007) [2023-12-26 16:59:37,978][105692] Updated weights for policy 0, policy_version 223719 (0.0006) [2023-12-26 16:59:38,019][105620] Updated weights for policy 1, policy_version 224320 (0.0007) [2023-12-26 16:59:38,708][105692] Updated weights for policy 0, policy_version 223729 (0.0011) [2023-12-26 16:59:38,765][105692] Updated weights for policy 0, policy_version 223739 (0.0006) [2023-12-26 16:59:38,795][105620] Updated weights for policy 1, policy_version 224330 (0.0007) [2023-12-26 16:59:38,828][105692] Updated weights for policy 0, policy_version 223749 (0.0009) [2023-12-26 16:59:38,853][105620] Updated weights for policy 1, policy_version 224340 (0.0006) [2023-12-26 16:59:38,884][105692] Updated weights for policy 0, policy_version 223759 (0.0009) [2023-12-26 16:59:38,910][105620] Updated weights for policy 1, policy_version 224350 (0.0005) [2023-12-26 16:59:38,966][105620] Updated weights for policy 1, policy_version 224360 (0.0007) [2023-12-26 16:59:39,634][105692] Updated weights for policy 0, policy_version 223769 (0.0008) [2023-12-26 16:59:39,646][105620] Updated weights for policy 1, policy_version 224370 (0.0006) [2023-12-26 16:59:39,681][105692] Updated weights for policy 0, policy_version 223779 (0.0005) [2023-12-26 16:59:39,710][105620] Updated weights for policy 1, policy_version 224380 (0.0009) [2023-12-26 16:59:39,737][105692] Updated weights for policy 0, policy_version 223789 (0.0009) [2023-12-26 16:59:39,771][105620] Updated weights for policy 1, policy_version 224390 (0.0009) [2023-12-26 16:59:40,496][105620] Updated weights for policy 1, policy_version 224400 (0.0007) [2023-12-26 16:59:40,561][105620] Updated weights for policy 1, policy_version 224410 (0.0006) [2023-12-26 16:59:40,590][105692] Updated weights for policy 0, policy_version 223799 (0.0009) [2023-12-26 16:59:40,617][105620] Updated weights for policy 1, policy_version 224420 (0.0007) [2023-12-26 16:59:40,638][105692] Updated weights for policy 0, policy_version 223809 (0.0007) [2023-12-26 16:59:40,690][105692] Updated weights for policy 0, policy_version 223819 (0.0009) [2023-12-26 16:59:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 114769920. Throughput: 0: 9453.9, 1: 9827.7. Samples: 114778692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:59:41,063][104569] Avg episode reward: [(0, '9266.445'), (1, '9092.745')] [2023-12-26 16:59:41,278][105620] Updated weights for policy 1, policy_version 224430 (0.0008) [2023-12-26 16:59:41,334][105620] Updated weights for policy 1, policy_version 224440 (0.0007) [2023-12-26 16:59:41,400][105620] Updated weights for policy 1, policy_version 224450 (0.0009) [2023-12-26 16:59:41,484][105692] Updated weights for policy 0, policy_version 223829 (0.0009) [2023-12-26 16:59:41,549][105692] Updated weights for policy 0, policy_version 223839 (0.0009) [2023-12-26 16:59:41,611][105692] Updated weights for policy 0, policy_version 223849 (0.0009) [2023-12-26 16:59:42,186][105620] Updated weights for policy 1, policy_version 224460 (0.0009) [2023-12-26 16:59:42,254][105620] Updated weights for policy 1, policy_version 224470 (0.0008) [2023-12-26 16:59:42,324][105692] Updated weights for policy 0, policy_version 223859 (0.0007) [2023-12-26 16:59:42,329][105620] Updated weights for policy 1, policy_version 224480 (0.0009) [2023-12-26 16:59:42,390][105692] Updated weights for policy 0, policy_version 223869 (0.0009) [2023-12-26 16:59:42,448][105692] Updated weights for policy 0, policy_version 223879 (0.0009) [2023-12-26 16:59:43,077][105620] Updated weights for policy 1, policy_version 224490 (0.0008) [2023-12-26 16:59:43,123][105620] Updated weights for policy 1, policy_version 224500 (0.0008) [2023-12-26 16:59:43,184][105620] Updated weights for policy 1, policy_version 224510 (0.0009) [2023-12-26 16:59:43,216][105692] Updated weights for policy 0, policy_version 223889 (0.0009) [2023-12-26 16:59:43,231][105620] Updated weights for policy 1, policy_version 224520 (0.0007) [2023-12-26 16:59:43,270][105692] Updated weights for policy 0, policy_version 223899 (0.0009) [2023-12-26 16:59:43,321][105692] Updated weights for policy 0, policy_version 223909 (0.0009) [2023-12-26 16:59:43,371][105692] Updated weights for policy 0, policy_version 223919 (0.0009) [2023-12-26 16:59:43,989][105620] Updated weights for policy 1, policy_version 224530 (0.0008) [2023-12-26 16:59:44,033][105620] Updated weights for policy 1, policy_version 224540 (0.0007) [2023-12-26 16:59:44,080][105620] Updated weights for policy 1, policy_version 224550 (0.0008) [2023-12-26 16:59:44,148][105692] Updated weights for policy 0, policy_version 223929 (0.0010) [2023-12-26 16:59:44,202][105692] Updated weights for policy 0, policy_version 223939 (0.0010) [2023-12-26 16:59:44,256][105692] Updated weights for policy 0, policy_version 223949 (0.0010) [2023-12-26 16:59:44,871][105620] Updated weights for policy 1, policy_version 224560 (0.0009) [2023-12-26 16:59:44,924][105620] Updated weights for policy 1, policy_version 224570 (0.0008) [2023-12-26 16:59:44,980][105620] Updated weights for policy 1, policy_version 224580 (0.0008) [2023-12-26 16:59:45,031][105692] Updated weights for policy 0, policy_version 223959 (0.0011) [2023-12-26 16:59:45,097][105692] Updated weights for policy 0, policy_version 223969 (0.0011) [2023-12-26 16:59:45,157][105692] Updated weights for policy 0, policy_version 223979 (0.0011) [2023-12-26 16:59:45,760][105620] Updated weights for policy 1, policy_version 224590 (0.0007) [2023-12-26 16:59:45,816][105620] Updated weights for policy 1, policy_version 224600 (0.0008) [2023-12-26 16:59:45,872][105620] Updated weights for policy 1, policy_version 224610 (0.0008) [2023-12-26 16:59:45,905][105692] Updated weights for policy 0, policy_version 223989 (0.0011) [2023-12-26 16:59:45,964][105692] Updated weights for policy 0, policy_version 223999 (0.0010) [2023-12-26 16:59:46,019][105692] Updated weights for policy 0, policy_version 224009 (0.0010) [2023-12-26 16:59:46,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 114868224. Throughput: 0: 9358.0, 1: 9760.4. Samples: 114833572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:59:46,062][104569] Avg episode reward: [(0, '9180.746'), (1, '9177.722')] [2023-12-26 16:59:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000224016_57360384.pth... [2023-12-26 16:59:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000224616_57507840.pth... [2023-12-26 16:59:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000223464_57212928.pth [2023-12-26 16:59:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000222896_57073664.pth [2023-12-26 16:59:46,546][105620] Updated weights for policy 1, policy_version 224620 (0.0008) [2023-12-26 16:59:46,593][105620] Updated weights for policy 1, policy_version 224630 (0.0006) [2023-12-26 16:59:46,646][105620] Updated weights for policy 1, policy_version 224640 (0.0007) [2023-12-26 16:59:46,732][105692] Updated weights for policy 0, policy_version 224019 (0.0009) [2023-12-26 16:59:46,788][105692] Updated weights for policy 0, policy_version 224029 (0.0006) [2023-12-26 16:59:46,844][105692] Updated weights for policy 0, policy_version 224039 (0.0005) [2023-12-26 16:59:47,260][105620] Updated weights for policy 1, policy_version 224650 (0.0008) [2023-12-26 16:59:47,323][105620] Updated weights for policy 1, policy_version 224660 (0.0008) [2023-12-26 16:59:47,378][105620] Updated weights for policy 1, policy_version 224670 (0.0008) [2023-12-26 16:59:47,468][105692] Updated weights for policy 0, policy_version 224049 (0.0006) [2023-12-26 16:59:47,531][105692] Updated weights for policy 0, policy_version 224059 (0.0010) [2023-12-26 16:59:47,580][105692] Updated weights for policy 0, policy_version 224069 (0.0011) [2023-12-26 16:59:47,639][105692] Updated weights for policy 0, policy_version 224079 (0.0010) [2023-12-26 16:59:48,091][105620] Updated weights for policy 1, policy_version 224681 (0.0010) [2023-12-26 16:59:48,135][105620] Updated weights for policy 1, policy_version 224691 (0.0010) [2023-12-26 16:59:48,189][105620] Updated weights for policy 1, policy_version 224701 (0.0010) [2023-12-26 16:59:48,237][105620] Updated weights for policy 1, policy_version 224711 (0.0010) [2023-12-26 16:59:48,405][105692] Updated weights for policy 0, policy_version 224089 (0.0011) [2023-12-26 16:59:48,468][105692] Updated weights for policy 0, policy_version 224099 (0.0010) [2023-12-26 16:59:48,530][105692] Updated weights for policy 0, policy_version 224109 (0.0010) [2023-12-26 16:59:49,014][105620] Updated weights for policy 1, policy_version 224721 (0.0010) [2023-12-26 16:59:49,069][105620] Updated weights for policy 1, policy_version 224731 (0.0010) [2023-12-26 16:59:49,121][105620] Updated weights for policy 1, policy_version 224741 (0.0010) [2023-12-26 16:59:49,193][105692] Updated weights for policy 0, policy_version 224119 (0.0008) [2023-12-26 16:59:49,263][105692] Updated weights for policy 0, policy_version 224129 (0.0010) [2023-12-26 16:59:49,331][105692] Updated weights for policy 0, policy_version 224139 (0.0011) [2023-12-26 16:59:49,835][105620] Updated weights for policy 1, policy_version 224751 (0.0009) [2023-12-26 16:59:49,897][105620] Updated weights for policy 1, policy_version 224761 (0.0008) [2023-12-26 16:59:49,962][105620] Updated weights for policy 1, policy_version 224771 (0.0010) [2023-12-26 16:59:50,037][105692] Updated weights for policy 0, policy_version 224149 (0.0011) [2023-12-26 16:59:50,095][105692] Updated weights for policy 0, policy_version 224159 (0.0007) [2023-12-26 16:59:50,160][105692] Updated weights for policy 0, policy_version 224169 (0.0005) [2023-12-26 16:59:50,649][105620] Updated weights for policy 1, policy_version 224781 (0.0011) [2023-12-26 16:59:50,702][105620] Updated weights for policy 1, policy_version 224791 (0.0011) [2023-12-26 16:59:50,758][105620] Updated weights for policy 1, policy_version 224801 (0.0011) [2023-12-26 16:59:50,878][105692] Updated weights for policy 0, policy_version 224179 (0.0010) [2023-12-26 16:59:50,941][105692] Updated weights for policy 0, policy_version 224189 (0.0011) [2023-12-26 16:59:51,008][105692] Updated weights for policy 0, policy_version 224199 (0.0011) [2023-12-26 16:59:51,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 114958336. Throughput: 0: 9342.4, 1: 9815.3. Samples: 114950340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:59:51,062][104569] Avg episode reward: [(0, '9091.057'), (1, '9268.030')] [2023-12-26 16:59:51,460][105620] Updated weights for policy 1, policy_version 224811 (0.0010) [2023-12-26 16:59:51,522][105620] Updated weights for policy 1, policy_version 224821 (0.0010) [2023-12-26 16:59:51,588][105620] Updated weights for policy 1, policy_version 224831 (0.0010) [2023-12-26 16:59:51,673][105692] Updated weights for policy 0, policy_version 224209 (0.0008) [2023-12-26 16:59:51,735][105692] Updated weights for policy 0, policy_version 224219 (0.0008) [2023-12-26 16:59:51,783][105692] Updated weights for policy 0, policy_version 224229 (0.0005) [2023-12-26 16:59:51,836][105692] Updated weights for policy 0, policy_version 224239 (0.0005) [2023-12-26 16:59:52,396][105692] Updated weights for policy 0, policy_version 224249 (0.0010) [2023-12-26 16:59:52,448][105692] Updated weights for policy 0, policy_version 224259 (0.0010) [2023-12-26 16:59:52,498][105692] Updated weights for policy 0, policy_version 224269 (0.0009) [2023-12-26 16:59:52,501][105620] Updated weights for policy 1, policy_version 224842 (0.0010) [2023-12-26 16:59:52,561][105620] Updated weights for policy 1, policy_version 224852 (0.0009) [2023-12-26 16:59:52,624][105620] Updated weights for policy 1, policy_version 224862 (0.0008) [2023-12-26 16:59:52,689][105620] Updated weights for policy 1, policy_version 224872 (0.0008) [2023-12-26 16:59:53,262][105692] Updated weights for policy 0, policy_version 224279 (0.0008) [2023-12-26 16:59:53,313][105692] Updated weights for policy 0, policy_version 224289 (0.0007) [2023-12-26 16:59:53,368][105692] Updated weights for policy 0, policy_version 224299 (0.0006) [2023-12-26 16:59:53,378][105620] Updated weights for policy 1, policy_version 224882 (0.0011) [2023-12-26 16:59:53,434][105620] Updated weights for policy 1, policy_version 224892 (0.0011) [2023-12-26 16:59:53,499][105620] Updated weights for policy 1, policy_version 224902 (0.0011) [2023-12-26 16:59:54,142][105692] Updated weights for policy 0, policy_version 224309 (0.0007) [2023-12-26 16:59:54,204][105692] Updated weights for policy 0, policy_version 224319 (0.0008) [2023-12-26 16:59:54,247][105620] Updated weights for policy 1, policy_version 224912 (0.0011) [2023-12-26 16:59:54,264][105692] Updated weights for policy 0, policy_version 224329 (0.0009) [2023-12-26 16:59:54,302][105620] Updated weights for policy 1, policy_version 224922 (0.0011) [2023-12-26 16:59:54,364][105620] Updated weights for policy 1, policy_version 224932 (0.0011) [2023-12-26 16:59:54,969][105692] Updated weights for policy 0, policy_version 224339 (0.0005) [2023-12-26 16:59:55,017][105692] Updated weights for policy 0, policy_version 224349 (0.0005) [2023-12-26 16:59:55,066][105692] Updated weights for policy 0, policy_version 224359 (0.0006) [2023-12-26 16:59:55,074][105620] Updated weights for policy 1, policy_version 224942 (0.0008) [2023-12-26 16:59:55,123][105620] Updated weights for policy 1, policy_version 224952 (0.0007) [2023-12-26 16:59:55,172][105620] Updated weights for policy 1, policy_version 224962 (0.0006) [2023-12-26 16:59:55,682][105692] Updated weights for policy 0, policy_version 224369 (0.0010) [2023-12-26 16:59:55,712][105620] Updated weights for policy 1, policy_version 224972 (0.0007) [2023-12-26 16:59:55,739][105692] Updated weights for policy 0, policy_version 224379 (0.0005) [2023-12-26 16:59:55,760][105620] Updated weights for policy 1, policy_version 224982 (0.0010) [2023-12-26 16:59:55,788][105692] Updated weights for policy 0, policy_version 224389 (0.0005) [2023-12-26 16:59:55,819][105620] Updated weights for policy 1, policy_version 224992 (0.0010) [2023-12-26 16:59:55,843][105692] Updated weights for policy 0, policy_version 224399 (0.0005) [2023-12-26 16:59:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 115064832. Throughput: 0: 9483.5, 1: 9732.5. Samples: 115068780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 16:59:56,062][104569] Avg episode reward: [(0, '9264.381'), (1, '9355.416')] [2023-12-26 16:59:56,448][105692] Updated weights for policy 0, policy_version 224409 (0.0006) [2023-12-26 16:59:56,509][105692] Updated weights for policy 0, policy_version 224419 (0.0006) [2023-12-26 16:59:56,526][105620] Updated weights for policy 1, policy_version 225002 (0.0009) [2023-12-26 16:59:56,559][105692] Updated weights for policy 0, policy_version 224429 (0.0010) [2023-12-26 16:59:56,572][105620] Updated weights for policy 1, policy_version 225012 (0.0005) [2023-12-26 16:59:56,618][105620] Updated weights for policy 1, policy_version 225022 (0.0008) [2023-12-26 16:59:56,666][105620] Updated weights for policy 1, policy_version 225032 (0.0010) [2023-12-26 16:59:57,282][105692] Updated weights for policy 0, policy_version 224439 (0.0009) [2023-12-26 16:59:57,341][105692] Updated weights for policy 0, policy_version 224449 (0.0006) [2023-12-26 16:59:57,359][105620] Updated weights for policy 1, policy_version 225042 (0.0010) [2023-12-26 16:59:57,401][105692] Updated weights for policy 0, policy_version 224459 (0.0006) [2023-12-26 16:59:57,415][105620] Updated weights for policy 1, policy_version 225052 (0.0010) [2023-12-26 16:59:57,477][105620] Updated weights for policy 1, policy_version 225062 (0.0010) [2023-12-26 16:59:58,164][105692] Updated weights for policy 0, policy_version 224469 (0.0007) [2023-12-26 16:59:58,183][105620] Updated weights for policy 1, policy_version 225072 (0.0008) [2023-12-26 16:59:58,221][105692] Updated weights for policy 0, policy_version 224479 (0.0006) [2023-12-26 16:59:58,243][105620] Updated weights for policy 1, policy_version 225082 (0.0008) [2023-12-26 16:59:58,270][105692] Updated weights for policy 0, policy_version 224489 (0.0006) [2023-12-26 16:59:58,303][105620] Updated weights for policy 1, policy_version 225092 (0.0008) [2023-12-26 16:59:59,052][105620] Updated weights for policy 1, policy_version 225102 (0.0010) [2023-12-26 16:59:59,107][105620] Updated weights for policy 1, policy_version 225112 (0.0007) [2023-12-26 16:59:59,129][105692] Updated weights for policy 0, policy_version 224499 (0.0007) [2023-12-26 16:59:59,167][105620] Updated weights for policy 1, policy_version 225122 (0.0006) [2023-12-26 16:59:59,185][105692] Updated weights for policy 0, policy_version 224509 (0.0008) [2023-12-26 16:59:59,247][105692] Updated weights for policy 0, policy_version 224519 (0.0008) [2023-12-26 16:59:59,881][105692] Updated weights for policy 0, policy_version 224529 (0.0007) [2023-12-26 16:59:59,891][105620] Updated weights for policy 1, policy_version 225132 (0.0007) [2023-12-26 16:59:59,943][105692] Updated weights for policy 0, policy_version 224539 (0.0007) [2023-12-26 16:59:59,957][105620] Updated weights for policy 1, policy_version 225142 (0.0007) [2023-12-26 16:59:59,999][105692] Updated weights for policy 0, policy_version 224549 (0.0009) [2023-12-26 17:00:00,017][105620] Updated weights for policy 1, policy_version 225152 (0.0006) [2023-12-26 17:00:00,059][105692] Updated weights for policy 0, policy_version 224559 (0.0011) [2023-12-26 17:00:00,641][105620] Updated weights for policy 1, policy_version 225162 (0.0006) [2023-12-26 17:00:00,713][105620] Updated weights for policy 1, policy_version 225172 (0.0006) [2023-12-26 17:00:00,767][105692] Updated weights for policy 0, policy_version 224569 (0.0006) [2023-12-26 17:00:00,783][105620] Updated weights for policy 1, policy_version 225182 (0.0005) [2023-12-26 17:00:00,818][105692] Updated weights for policy 0, policy_version 224579 (0.0007) [2023-12-26 17:00:00,836][105620] Updated weights for policy 1, policy_version 225192 (0.0006) [2023-12-26 17:00:00,877][105692] Updated weights for policy 0, policy_version 224589 (0.0011) [2023-12-26 17:00:01,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 115163136. Throughput: 0: 9539.8, 1: 9760.5. Samples: 115127740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:00:01,062][104569] Avg episode reward: [(0, '9264.472'), (1, '9355.341')] [2023-12-26 17:00:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000225192_57655296.pth... [2023-12-26 17:00:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000224592_57507840.pth... [2023-12-26 17:00:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000224040_57360384.pth [2023-12-26 17:00:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000223472_57221120.pth [2023-12-26 17:00:01,493][105620] Updated weights for policy 1, policy_version 225202 (0.0011) [2023-12-26 17:00:01,551][105620] Updated weights for policy 1, policy_version 225212 (0.0010) [2023-12-26 17:00:01,613][105620] Updated weights for policy 1, policy_version 225222 (0.0010) [2023-12-26 17:00:01,614][105692] Updated weights for policy 0, policy_version 224599 (0.0011) [2023-12-26 17:00:01,684][105692] Updated weights for policy 0, policy_version 224609 (0.0010) [2023-12-26 17:00:01,752][105692] Updated weights for policy 0, policy_version 224619 (0.0011) [2023-12-26 17:00:02,341][105620] Updated weights for policy 1, policy_version 225232 (0.0010) [2023-12-26 17:00:02,404][105620] Updated weights for policy 1, policy_version 225242 (0.0011) [2023-12-26 17:00:02,467][105620] Updated weights for policy 1, policy_version 225252 (0.0006) [2023-12-26 17:00:02,474][105692] Updated weights for policy 0, policy_version 224629 (0.0009) [2023-12-26 17:00:02,545][105692] Updated weights for policy 0, policy_version 224639 (0.0008) [2023-12-26 17:00:02,613][105692] Updated weights for policy 0, policy_version 224649 (0.0007) [2023-12-26 17:00:03,148][105692] Updated weights for policy 0, policy_version 224659 (0.0007) [2023-12-26 17:00:03,164][105620] Updated weights for policy 1, policy_version 225262 (0.0007) [2023-12-26 17:00:03,196][105692] Updated weights for policy 0, policy_version 224669 (0.0007) [2023-12-26 17:00:03,228][105620] Updated weights for policy 1, policy_version 225272 (0.0007) [2023-12-26 17:00:03,250][105692] Updated weights for policy 0, policy_version 224679 (0.0008) [2023-12-26 17:00:03,290][105620] Updated weights for policy 1, policy_version 225282 (0.0006) [2023-12-26 17:00:03,877][105692] Updated weights for policy 0, policy_version 224689 (0.0008) [2023-12-26 17:00:03,933][105692] Updated weights for policy 0, policy_version 224699 (0.0011) [2023-12-26 17:00:03,960][105620] Updated weights for policy 1, policy_version 225292 (0.0008) [2023-12-26 17:00:03,990][105692] Updated weights for policy 0, policy_version 224709 (0.0011) [2023-12-26 17:00:04,017][105620] Updated weights for policy 1, policy_version 225302 (0.0007) [2023-12-26 17:00:04,043][105692] Updated weights for policy 0, policy_version 224719 (0.0011) [2023-12-26 17:00:04,080][105620] Updated weights for policy 1, policy_version 225312 (0.0009) [2023-12-26 17:00:04,725][105620] Updated weights for policy 1, policy_version 225322 (0.0006) [2023-12-26 17:00:04,786][105620] Updated weights for policy 1, policy_version 225332 (0.0008) [2023-12-26 17:00:04,792][105692] Updated weights for policy 0, policy_version 224729 (0.0009) [2023-12-26 17:00:04,841][105692] Updated weights for policy 0, policy_version 224739 (0.0010) [2023-12-26 17:00:04,842][105620] Updated weights for policy 1, policy_version 225342 (0.0006) [2023-12-26 17:00:04,889][105692] Updated weights for policy 0, policy_version 224749 (0.0010) [2023-12-26 17:00:04,899][105620] Updated weights for policy 1, policy_version 225352 (0.0006) [2023-12-26 17:00:05,508][105620] Updated weights for policy 1, policy_version 225362 (0.0005) [2023-12-26 17:00:05,552][105620] Updated weights for policy 1, policy_version 225372 (0.0007) [2023-12-26 17:00:05,571][105692] Updated weights for policy 0, policy_version 224759 (0.0010) [2023-12-26 17:00:05,605][105620] Updated weights for policy 1, policy_version 225382 (0.0006) [2023-12-26 17:00:05,622][105692] Updated weights for policy 0, policy_version 224769 (0.0010) [2023-12-26 17:00:05,673][105692] Updated weights for policy 0, policy_version 224779 (0.0010) [2023-12-26 17:00:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 115261440. Throughput: 0: 9583.9, 1: 9842.1. Samples: 115248188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:00:06,062][104569] Avg episode reward: [(0, '9356.501'), (1, '9174.812')] [2023-12-26 17:00:06,381][105692] Updated weights for policy 0, policy_version 224789 (0.0010) [2023-12-26 17:00:06,414][105620] Updated weights for policy 1, policy_version 225392 (0.0007) [2023-12-26 17:00:06,440][105692] Updated weights for policy 0, policy_version 224799 (0.0010) [2023-12-26 17:00:06,478][105620] Updated weights for policy 1, policy_version 225402 (0.0006) [2023-12-26 17:00:06,489][105692] Updated weights for policy 0, policy_version 224809 (0.0011) [2023-12-26 17:00:06,539][105620] Updated weights for policy 1, policy_version 225412 (0.0006) [2023-12-26 17:00:07,216][105692] Updated weights for policy 0, policy_version 224819 (0.0011) [2023-12-26 17:00:07,278][105692] Updated weights for policy 0, policy_version 224829 (0.0010) [2023-12-26 17:00:07,293][105620] Updated weights for policy 1, policy_version 225422 (0.0006) [2023-12-26 17:00:07,332][105692] Updated weights for policy 0, policy_version 224839 (0.0010) [2023-12-26 17:00:07,348][105620] Updated weights for policy 1, policy_version 225432 (0.0005) [2023-12-26 17:00:07,416][105620] Updated weights for policy 1, policy_version 225442 (0.0005) [2023-12-26 17:00:08,075][105692] Updated weights for policy 0, policy_version 224849 (0.0010) [2023-12-26 17:00:08,093][105620] Updated weights for policy 1, policy_version 225452 (0.0007) [2023-12-26 17:00:08,134][105692] Updated weights for policy 0, policy_version 224859 (0.0007) [2023-12-26 17:00:08,161][105620] Updated weights for policy 1, policy_version 225462 (0.0007) [2023-12-26 17:00:08,189][105692] Updated weights for policy 0, policy_version 224869 (0.0009) [2023-12-26 17:00:08,216][105620] Updated weights for policy 1, policy_version 225472 (0.0006) [2023-12-26 17:00:08,244][105692] Updated weights for policy 0, policy_version 224879 (0.0010) [2023-12-26 17:00:08,811][105620] Updated weights for policy 1, policy_version 225482 (0.0008) [2023-12-26 17:00:08,868][105620] Updated weights for policy 1, policy_version 225492 (0.0006) [2023-12-26 17:00:08,925][105620] Updated weights for policy 1, policy_version 225502 (0.0006) [2023-12-26 17:00:08,983][105692] Updated weights for policy 0, policy_version 224889 (0.0010) [2023-12-26 17:00:08,984][105620] Updated weights for policy 1, policy_version 225512 (0.0008) [2023-12-26 17:00:09,038][105692] Updated weights for policy 0, policy_version 224899 (0.0010) [2023-12-26 17:00:09,089][105692] Updated weights for policy 0, policy_version 224909 (0.0010) [2023-12-26 17:00:09,688][105620] Updated weights for policy 1, policy_version 225522 (0.0009) [2023-12-26 17:00:09,750][105620] Updated weights for policy 1, policy_version 225532 (0.0009) [2023-12-26 17:00:09,817][105620] Updated weights for policy 1, policy_version 225542 (0.0009) [2023-12-26 17:00:09,880][105692] Updated weights for policy 0, policy_version 224919 (0.0009) [2023-12-26 17:00:09,952][105692] Updated weights for policy 0, policy_version 224929 (0.0009) [2023-12-26 17:00:10,022][105692] Updated weights for policy 0, policy_version 224939 (0.0008) [2023-12-26 17:00:10,629][105620] Updated weights for policy 1, policy_version 225552 (0.0009) [2023-12-26 17:00:10,643][105692] Updated weights for policy 0, policy_version 224949 (0.0007) [2023-12-26 17:00:10,690][105620] Updated weights for policy 1, policy_version 225562 (0.0007) [2023-12-26 17:00:10,707][105692] Updated weights for policy 0, policy_version 224959 (0.0008) [2023-12-26 17:00:10,743][105620] Updated weights for policy 1, policy_version 225572 (0.0007) [2023-12-26 17:00:10,764][105692] Updated weights for policy 0, policy_version 224969 (0.0006) [2023-12-26 17:00:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 115359744. Throughput: 0: 9653.8, 1: 9853.0. Samples: 115365352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:00:11,063][104569] Avg episode reward: [(0, '9356.767'), (1, '9174.433')] [2023-12-26 17:00:11,347][105692] Updated weights for policy 0, policy_version 224979 (0.0007) [2023-12-26 17:00:11,412][105692] Updated weights for policy 0, policy_version 224989 (0.0009) [2023-12-26 17:00:11,483][105692] Updated weights for policy 0, policy_version 224999 (0.0009) [2023-12-26 17:00:11,559][105620] Updated weights for policy 1, policy_version 225582 (0.0007) [2023-12-26 17:00:11,615][105620] Updated weights for policy 1, policy_version 225592 (0.0006) [2023-12-26 17:00:11,676][105620] Updated weights for policy 1, policy_version 225602 (0.0008) [2023-12-26 17:00:12,300][105692] Updated weights for policy 0, policy_version 225009 (0.0008) [2023-12-26 17:00:12,372][105692] Updated weights for policy 0, policy_version 225019 (0.0007) [2023-12-26 17:00:12,426][105620] Updated weights for policy 1, policy_version 225612 (0.0006) [2023-12-26 17:00:12,439][105692] Updated weights for policy 0, policy_version 225029 (0.0006) [2023-12-26 17:00:12,485][105620] Updated weights for policy 1, policy_version 225622 (0.0005) [2023-12-26 17:00:12,501][105692] Updated weights for policy 0, policy_version 225039 (0.0007) [2023-12-26 17:00:12,538][105620] Updated weights for policy 1, policy_version 225632 (0.0006) [2023-12-26 17:00:13,099][105692] Updated weights for policy 0, policy_version 225049 (0.0009) [2023-12-26 17:00:13,158][105692] Updated weights for policy 0, policy_version 225059 (0.0009) [2023-12-26 17:00:13,221][105692] Updated weights for policy 0, policy_version 225069 (0.0009) [2023-12-26 17:00:13,310][105620] Updated weights for policy 1, policy_version 225642 (0.0009) [2023-12-26 17:00:13,360][105620] Updated weights for policy 1, policy_version 225652 (0.0008) [2023-12-26 17:00:13,415][105620] Updated weights for policy 1, policy_version 225662 (0.0009) [2023-12-26 17:00:13,468][105620] Updated weights for policy 1, policy_version 225672 (0.0009) [2023-12-26 17:00:13,848][105692] Updated weights for policy 0, policy_version 225079 (0.0009) [2023-12-26 17:00:13,913][105692] Updated weights for policy 0, policy_version 225089 (0.0009) [2023-12-26 17:00:13,976][105692] Updated weights for policy 0, policy_version 225099 (0.0010) [2023-12-26 17:00:14,149][105620] Updated weights for policy 1, policy_version 225682 (0.0009) [2023-12-26 17:00:14,201][105620] Updated weights for policy 1, policy_version 225692 (0.0009) [2023-12-26 17:00:14,261][105620] Updated weights for policy 1, policy_version 225702 (0.0009) [2023-12-26 17:00:14,775][105692] Updated weights for policy 0, policy_version 225109 (0.0009) [2023-12-26 17:00:14,836][105692] Updated weights for policy 0, policy_version 225119 (0.0009) [2023-12-26 17:00:14,896][105692] Updated weights for policy 0, policy_version 225129 (0.0010) [2023-12-26 17:00:14,940][105620] Updated weights for policy 1, policy_version 225712 (0.0006) [2023-12-26 17:00:14,992][105620] Updated weights for policy 1, policy_version 225722 (0.0007) [2023-12-26 17:00:15,047][105620] Updated weights for policy 1, policy_version 225732 (0.0008) [2023-12-26 17:00:15,560][105692] Updated weights for policy 0, policy_version 225139 (0.0009) [2023-12-26 17:00:15,625][105692] Updated weights for policy 0, policy_version 225149 (0.0006) [2023-12-26 17:00:15,684][105692] Updated weights for policy 0, policy_version 225159 (0.0008) [2023-12-26 17:00:15,893][105620] Updated weights for policy 1, policy_version 225742 (0.0009) [2023-12-26 17:00:15,944][105620] Updated weights for policy 1, policy_version 225752 (0.0009) [2023-12-26 17:00:15,990][105620] Updated weights for policy 1, policy_version 225762 (0.0009) [2023-12-26 17:00:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.4, 300 sec: 19549.7). Total num frames: 115458048. Throughput: 0: 9657.0, 1: 9788.6. Samples: 115424164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:00:16,063][104569] Avg episode reward: [(0, '9356.614'), (1, '9266.320')] [2023-12-26 17:00:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000225168_57655296.pth... [2023-12-26 17:00:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000225768_57802752.pth... [2023-12-26 17:00:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000224616_57507840.pth [2023-12-26 17:00:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000224016_57360384.pth [2023-12-26 17:00:16,362][105692] Updated weights for policy 0, policy_version 225169 (0.0009) [2023-12-26 17:00:16,417][105692] Updated weights for policy 0, policy_version 225179 (0.0009) [2023-12-26 17:00:16,469][105692] Updated weights for policy 0, policy_version 225189 (0.0009) [2023-12-26 17:00:16,524][105692] Updated weights for policy 0, policy_version 225199 (0.0009) [2023-12-26 17:00:16,776][105620] Updated weights for policy 1, policy_version 225772 (0.0009) [2023-12-26 17:00:16,834][105620] Updated weights for policy 1, policy_version 225782 (0.0008) [2023-12-26 17:00:16,891][105620] Updated weights for policy 1, policy_version 225792 (0.0009) [2023-12-26 17:00:17,293][105692] Updated weights for policy 0, policy_version 225209 (0.0008) [2023-12-26 17:00:17,347][105692] Updated weights for policy 0, policy_version 225219 (0.0005) [2023-12-26 17:00:17,410][105692] Updated weights for policy 0, policy_version 225229 (0.0007) [2023-12-26 17:00:17,651][105620] Updated weights for policy 1, policy_version 225802 (0.0009) [2023-12-26 17:00:17,709][105620] Updated weights for policy 1, policy_version 225812 (0.0008) [2023-12-26 17:00:17,763][105620] Updated weights for policy 1, policy_version 225822 (0.0009) [2023-12-26 17:00:17,813][105620] Updated weights for policy 1, policy_version 225832 (0.0009) [2023-12-26 17:00:18,129][105692] Updated weights for policy 0, policy_version 225239 (0.0009) [2023-12-26 17:00:18,180][105692] Updated weights for policy 0, policy_version 225249 (0.0009) [2023-12-26 17:00:18,241][105692] Updated weights for policy 0, policy_version 225259 (0.0010) [2023-12-26 17:00:18,536][105620] Updated weights for policy 1, policy_version 225842 (0.0009) [2023-12-26 17:00:18,600][105620] Updated weights for policy 1, policy_version 225852 (0.0007) [2023-12-26 17:00:18,666][105620] Updated weights for policy 1, policy_version 225862 (0.0009) [2023-12-26 17:00:19,011][105692] Updated weights for policy 0, policy_version 225269 (0.0009) [2023-12-26 17:00:19,073][105692] Updated weights for policy 0, policy_version 225279 (0.0011) [2023-12-26 17:00:19,140][105692] Updated weights for policy 0, policy_version 225289 (0.0011) [2023-12-26 17:00:19,358][105620] Updated weights for policy 1, policy_version 225872 (0.0011) [2023-12-26 17:00:19,422][105620] Updated weights for policy 1, policy_version 225882 (0.0009) [2023-12-26 17:00:19,484][105620] Updated weights for policy 1, policy_version 225892 (0.0010) [2023-12-26 17:00:19,852][105692] Updated weights for policy 0, policy_version 225299 (0.0010) [2023-12-26 17:00:19,918][105692] Updated weights for policy 0, policy_version 225309 (0.0008) [2023-12-26 17:00:19,987][105692] Updated weights for policy 0, policy_version 225319 (0.0008) [2023-12-26 17:00:20,242][105620] Updated weights for policy 1, policy_version 225902 (0.0010) [2023-12-26 17:00:20,297][105620] Updated weights for policy 1, policy_version 225912 (0.0010) [2023-12-26 17:00:20,357][105620] Updated weights for policy 1, policy_version 225922 (0.0010) [2023-12-26 17:00:20,714][105692] Updated weights for policy 0, policy_version 225329 (0.0008) [2023-12-26 17:00:20,772][105692] Updated weights for policy 0, policy_version 225339 (0.0009) [2023-12-26 17:00:20,826][105692] Updated weights for policy 0, policy_version 225349 (0.0008) [2023-12-26 17:00:20,883][105692] Updated weights for policy 0, policy_version 225359 (0.0006) [2023-12-26 17:00:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 115548160. Throughput: 0: 9701.9, 1: 9721.4. Samples: 115538080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:00:21,063][104569] Avg episode reward: [(0, '9356.839'), (1, '9179.634')] [2023-12-26 17:00:21,117][105620] Updated weights for policy 1, policy_version 225932 (0.0010) [2023-12-26 17:00:21,184][105620] Updated weights for policy 1, policy_version 225942 (0.0010) [2023-12-26 17:00:21,246][105620] Updated weights for policy 1, policy_version 225952 (0.0009) [2023-12-26 17:00:21,742][105692] Updated weights for policy 0, policy_version 225369 (0.0008) [2023-12-26 17:00:21,807][105692] Updated weights for policy 0, policy_version 225379 (0.0008) [2023-12-26 17:00:21,871][105692] Updated weights for policy 0, policy_version 225389 (0.0010) [2023-12-26 17:00:22,010][105620] Updated weights for policy 1, policy_version 225962 (0.0009) [2023-12-26 17:00:22,058][105620] Updated weights for policy 1, policy_version 225972 (0.0010) [2023-12-26 17:00:22,107][105620] Updated weights for policy 1, policy_version 225982 (0.0010) [2023-12-26 17:00:22,156][105620] Updated weights for policy 1, policy_version 225992 (0.0010) [2023-12-26 17:00:22,593][105692] Updated weights for policy 0, policy_version 225399 (0.0009) [2023-12-26 17:00:22,651][105692] Updated weights for policy 0, policy_version 225409 (0.0010) [2023-12-26 17:00:22,713][105692] Updated weights for policy 0, policy_version 225419 (0.0011) [2023-12-26 17:00:22,936][105620] Updated weights for policy 1, policy_version 226002 (0.0011) [2023-12-26 17:00:22,997][105620] Updated weights for policy 1, policy_version 226012 (0.0011) [2023-12-26 17:00:23,050][105620] Updated weights for policy 1, policy_version 226022 (0.0011) [2023-12-26 17:00:23,390][105692] Updated weights for policy 0, policy_version 225429 (0.0008) [2023-12-26 17:00:23,453][105692] Updated weights for policy 0, policy_version 225439 (0.0010) [2023-12-26 17:00:23,509][105692] Updated weights for policy 0, policy_version 225449 (0.0007) [2023-12-26 17:00:23,772][105620] Updated weights for policy 1, policy_version 226032 (0.0010) [2023-12-26 17:00:23,831][105620] Updated weights for policy 1, policy_version 226042 (0.0010) [2023-12-26 17:00:23,889][105620] Updated weights for policy 1, policy_version 226052 (0.0010) [2023-12-26 17:00:24,074][105692] Updated weights for policy 0, policy_version 225459 (0.0006) [2023-12-26 17:00:24,137][105692] Updated weights for policy 0, policy_version 225469 (0.0008) [2023-12-26 17:00:24,200][105692] Updated weights for policy 0, policy_version 225479 (0.0009) [2023-12-26 17:00:24,611][105620] Updated weights for policy 1, policy_version 226062 (0.0010) [2023-12-26 17:00:24,662][105620] Updated weights for policy 1, policy_version 226072 (0.0009) [2023-12-26 17:00:24,714][105620] Updated weights for policy 1, policy_version 226082 (0.0008) [2023-12-26 17:00:24,919][105692] Updated weights for policy 0, policy_version 225490 (0.0008) [2023-12-26 17:00:24,984][105692] Updated weights for policy 0, policy_version 225500 (0.0005) [2023-12-26 17:00:25,047][105692] Updated weights for policy 0, policy_version 225510 (0.0008) [2023-12-26 17:00:25,100][105692] Updated weights for policy 0, policy_version 225520 (0.0008) [2023-12-26 17:00:25,447][105620] Updated weights for policy 1, policy_version 226092 (0.0010) [2023-12-26 17:00:25,495][105620] Updated weights for policy 1, policy_version 226103 (0.0008) [2023-12-26 17:00:25,546][105620] Updated weights for policy 1, policy_version 226113 (0.0005) [2023-12-26 17:00:25,732][105692] Updated weights for policy 0, policy_version 225530 (0.0005) [2023-12-26 17:00:25,783][105692] Updated weights for policy 0, policy_version 225540 (0.0009) [2023-12-26 17:00:25,830][105692] Updated weights for policy 0, policy_version 225550 (0.0010) [2023-12-26 17:00:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19521.9). Total num frames: 115646464. Throughput: 0: 9814.9, 1: 9655.2. Samples: 115654848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:00:26,062][104569] Avg episode reward: [(0, '9356.723'), (1, '9267.730')] [2023-12-26 17:00:26,157][105620] Updated weights for policy 1, policy_version 226123 (0.0005) [2023-12-26 17:00:26,225][105620] Updated weights for policy 1, policy_version 226133 (0.0005) [2023-12-26 17:00:26,288][105620] Updated weights for policy 1, policy_version 226143 (0.0005) [2023-12-26 17:00:26,575][105692] Updated weights for policy 0, policy_version 225560 (0.0010) [2023-12-26 17:00:26,644][105692] Updated weights for policy 0, policy_version 225570 (0.0009) [2023-12-26 17:00:26,710][105692] Updated weights for policy 0, policy_version 225580 (0.0005) [2023-12-26 17:00:26,823][105620] Updated weights for policy 1, policy_version 226153 (0.0005) [2023-12-26 17:00:26,883][105620] Updated weights for policy 1, policy_version 226163 (0.0006) [2023-12-26 17:00:26,952][105620] Updated weights for policy 1, policy_version 226173 (0.0005) [2023-12-26 17:00:27,010][105620] Updated weights for policy 1, policy_version 226183 (0.0005) [2023-12-26 17:00:27,327][105692] Updated weights for policy 0, policy_version 225590 (0.0008) [2023-12-26 17:00:27,382][105692] Updated weights for policy 0, policy_version 225600 (0.0011) [2023-12-26 17:00:27,426][105692] Updated weights for policy 0, policy_version 225610 (0.0010) [2023-12-26 17:00:27,577][105620] Updated weights for policy 1, policy_version 226193 (0.0008) [2023-12-26 17:00:27,636][105620] Updated weights for policy 1, policy_version 226203 (0.0008) [2023-12-26 17:00:27,687][105620] Updated weights for policy 1, policy_version 226213 (0.0010) [2023-12-26 17:00:28,082][105692] Updated weights for policy 0, policy_version 225620 (0.0009) [2023-12-26 17:00:28,145][105692] Updated weights for policy 0, policy_version 225630 (0.0009) [2023-12-26 17:00:28,208][105692] Updated weights for policy 0, policy_version 225640 (0.0009) [2023-12-26 17:00:28,467][105620] Updated weights for policy 1, policy_version 226223 (0.0009) [2023-12-26 17:00:28,533][105620] Updated weights for policy 1, policy_version 226233 (0.0007) [2023-12-26 17:00:28,597][105620] Updated weights for policy 1, policy_version 226243 (0.0006) [2023-12-26 17:00:28,992][105692] Updated weights for policy 0, policy_version 225650 (0.0009) [2023-12-26 17:00:29,039][105692] Updated weights for policy 0, policy_version 225660 (0.0009) [2023-12-26 17:00:29,091][105692] Updated weights for policy 0, policy_version 225670 (0.0009) [2023-12-26 17:00:29,142][105692] Updated weights for policy 0, policy_version 225680 (0.0008) [2023-12-26 17:00:29,267][105620] Updated weights for policy 1, policy_version 226253 (0.0008) [2023-12-26 17:00:29,337][105620] Updated weights for policy 1, policy_version 226263 (0.0006) [2023-12-26 17:00:29,401][105620] Updated weights for policy 1, policy_version 226273 (0.0009) [2023-12-26 17:00:29,976][105692] Updated weights for policy 0, policy_version 225690 (0.0009) [2023-12-26 17:00:30,040][105692] Updated weights for policy 0, policy_version 225700 (0.0009) [2023-12-26 17:00:30,099][105692] Updated weights for policy 0, policy_version 225710 (0.0009) [2023-12-26 17:00:30,117][105620] Updated weights for policy 1, policy_version 226283 (0.0009) [2023-12-26 17:00:30,170][105620] Updated weights for policy 1, policy_version 226293 (0.0008) [2023-12-26 17:00:30,221][105620] Updated weights for policy 1, policy_version 226303 (0.0009) [2023-12-26 17:00:30,833][105620] Updated weights for policy 1, policy_version 226313 (0.0009) [2023-12-26 17:00:30,865][105692] Updated weights for policy 0, policy_version 225720 (0.0007) [2023-12-26 17:00:30,892][105620] Updated weights for policy 1, policy_version 226323 (0.0008) [2023-12-26 17:00:30,911][105692] Updated weights for policy 0, policy_version 225730 (0.0010) [2023-12-26 17:00:30,948][105620] Updated weights for policy 1, policy_version 226333 (0.0007) [2023-12-26 17:00:30,967][105692] Updated weights for policy 0, policy_version 225740 (0.0009) [2023-12-26 17:00:31,004][105620] Updated weights for policy 1, policy_version 226343 (0.0006) [2023-12-26 17:00:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 115752960. Throughput: 0: 9881.2, 1: 9754.7. Samples: 115717184. Policy #0 lag: (min: 28.0, avg: 53.1, max: 60.0) [2023-12-26 17:00:31,062][104569] Avg episode reward: [(0, '9356.558'), (1, '9355.062')] [2023-12-26 17:00:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000226344_57950208.pth... [2023-12-26 17:00:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000225744_57802752.pth... [2023-12-26 17:00:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000225192_57655296.pth [2023-12-26 17:00:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000224592_57507840.pth [2023-12-26 17:00:31,704][105620] Updated weights for policy 1, policy_version 226353 (0.0008) [2023-12-26 17:00:31,764][105692] Updated weights for policy 0, policy_version 225750 (0.0009) [2023-12-26 17:00:31,766][105620] Updated weights for policy 1, policy_version 226363 (0.0007) [2023-12-26 17:00:31,827][105692] Updated weights for policy 0, policy_version 225760 (0.0008) [2023-12-26 17:00:31,829][105620] Updated weights for policy 1, policy_version 226373 (0.0006) [2023-12-26 17:00:31,883][105692] Updated weights for policy 0, policy_version 225770 (0.0008) [2023-12-26 17:00:32,457][105620] Updated weights for policy 1, policy_version 226383 (0.0009) [2023-12-26 17:00:32,516][105620] Updated weights for policy 1, policy_version 226393 (0.0009) [2023-12-26 17:00:32,573][105620] Updated weights for policy 1, policy_version 226403 (0.0005) [2023-12-26 17:00:32,732][105692] Updated weights for policy 0, policy_version 225780 (0.0009) [2023-12-26 17:00:32,793][105692] Updated weights for policy 0, policy_version 225790 (0.0009) [2023-12-26 17:00:32,848][105692] Updated weights for policy 0, policy_version 225800 (0.0009) [2023-12-26 17:00:33,230][105620] Updated weights for policy 1, policy_version 226413 (0.0007) [2023-12-26 17:00:33,275][105620] Updated weights for policy 1, policy_version 226423 (0.0009) [2023-12-26 17:00:33,321][105620] Updated weights for policy 1, policy_version 226433 (0.0008) [2023-12-26 17:00:33,620][105692] Updated weights for policy 0, policy_version 225810 (0.0009) [2023-12-26 17:00:33,681][105692] Updated weights for policy 0, policy_version 225820 (0.0009) [2023-12-26 17:00:33,743][105692] Updated weights for policy 0, policy_version 225830 (0.0009) [2023-12-26 17:00:33,801][105692] Updated weights for policy 0, policy_version 225840 (0.0009) [2023-12-26 17:00:34,101][105620] Updated weights for policy 1, policy_version 226443 (0.0009) [2023-12-26 17:00:34,169][105620] Updated weights for policy 1, policy_version 226453 (0.0009) [2023-12-26 17:00:34,226][105620] Updated weights for policy 1, policy_version 226463 (0.0008) [2023-12-26 17:00:34,546][105692] Updated weights for policy 0, policy_version 225850 (0.0008) [2023-12-26 17:00:34,615][105692] Updated weights for policy 0, policy_version 225860 (0.0007) [2023-12-26 17:00:34,673][105692] Updated weights for policy 0, policy_version 225870 (0.0006) [2023-12-26 17:00:35,034][105620] Updated weights for policy 1, policy_version 226473 (0.0008) [2023-12-26 17:00:35,093][105620] Updated weights for policy 1, policy_version 226483 (0.0007) [2023-12-26 17:00:35,154][105620] Updated weights for policy 1, policy_version 226493 (0.0008) [2023-12-26 17:00:35,212][105620] Updated weights for policy 1, policy_version 226503 (0.0010) [2023-12-26 17:00:35,370][105692] Updated weights for policy 0, policy_version 225880 (0.0008) [2023-12-26 17:00:35,425][105692] Updated weights for policy 0, policy_version 225890 (0.0008) [2023-12-26 17:00:35,477][105692] Updated weights for policy 0, policy_version 225900 (0.0008) [2023-12-26 17:00:35,928][105620] Updated weights for policy 1, policy_version 226513 (0.0009) [2023-12-26 17:00:35,991][105620] Updated weights for policy 1, policy_version 226523 (0.0008) [2023-12-26 17:00:36,054][105620] Updated weights for policy 1, policy_version 226533 (0.0009) [2023-12-26 17:00:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19521.9). Total num frames: 115834880. Throughput: 0: 9782.0, 1: 9783.4. Samples: 115830784. Policy #0 lag: (min: 28.0, avg: 53.1, max: 60.0) [2023-12-26 17:00:36,063][104569] Avg episode reward: [(0, '9266.725'), (1, '8996.093')] [2023-12-26 17:00:36,251][105692] Updated weights for policy 0, policy_version 225910 (0.0009) [2023-12-26 17:00:36,304][105692] Updated weights for policy 0, policy_version 225920 (0.0009) [2023-12-26 17:00:36,356][105692] Updated weights for policy 0, policy_version 225930 (0.0010) [2023-12-26 17:00:36,801][105620] Updated weights for policy 1, policy_version 226543 (0.0009) [2023-12-26 17:00:36,845][105586] KL-divergence is very high: 131.0115 [2023-12-26 17:00:36,859][105620] Updated weights for policy 1, policy_version 226553 (0.0009) [2023-12-26 17:00:36,871][105586] KL-divergence is very high: 149.2545 [2023-12-26 17:00:36,896][105586] KL-divergence is very high: 231.9479 [2023-12-26 17:00:36,921][105586] KL-divergence is very high: 195.3286 [2023-12-26 17:00:36,922][105620] Updated weights for policy 1, policy_version 226563 (0.0008) [2023-12-26 17:00:36,946][105586] KL-divergence is very high: 250.9959 [2023-12-26 17:00:37,177][105692] Updated weights for policy 0, policy_version 225940 (0.0009) [2023-12-26 17:00:37,236][105692] Updated weights for policy 0, policy_version 225950 (0.0009) [2023-12-26 17:00:37,300][105692] Updated weights for policy 0, policy_version 225960 (0.0010) [2023-12-26 17:00:37,665][105620] Updated weights for policy 1, policy_version 226573 (0.0009) [2023-12-26 17:00:37,727][105620] Updated weights for policy 1, policy_version 226583 (0.0009) [2023-12-26 17:00:37,795][105620] Updated weights for policy 1, policy_version 226593 (0.0010) [2023-12-26 17:00:37,887][105692] Updated weights for policy 0, policy_version 225970 (0.0005) [2023-12-26 17:00:37,942][105692] Updated weights for policy 0, policy_version 225980 (0.0005) [2023-12-26 17:00:38,003][105692] Updated weights for policy 0, policy_version 225990 (0.0006) [2023-12-26 17:00:38,071][105692] Updated weights for policy 0, policy_version 226000 (0.0008) [2023-12-26 17:00:38,633][105620] Updated weights for policy 1, policy_version 226603 (0.0010) [2023-12-26 17:00:38,687][105692] Updated weights for policy 0, policy_version 226010 (0.0011) [2023-12-26 17:00:38,693][105620] Updated weights for policy 1, policy_version 226613 (0.0005) [2023-12-26 17:00:38,744][105692] Updated weights for policy 0, policy_version 226020 (0.0011) [2023-12-26 17:00:38,754][105620] Updated weights for policy 1, policy_version 226623 (0.0006) [2023-12-26 17:00:38,805][105692] Updated weights for policy 0, policy_version 226030 (0.0011) [2023-12-26 17:00:39,544][105620] Updated weights for policy 1, policy_version 226633 (0.0006) [2023-12-26 17:00:39,594][105620] Updated weights for policy 1, policy_version 226643 (0.0007) [2023-12-26 17:00:39,608][105692] Updated weights for policy 0, policy_version 226040 (0.0011) [2023-12-26 17:00:39,647][105620] Updated weights for policy 1, policy_version 226653 (0.0006) [2023-12-26 17:00:39,664][105692] Updated weights for policy 0, policy_version 226050 (0.0011) [2023-12-26 17:00:39,707][105620] Updated weights for policy 1, policy_version 226663 (0.0006) [2023-12-26 17:00:39,720][105692] Updated weights for policy 0, policy_version 226060 (0.0010) [2023-12-26 17:00:40,526][105692] Updated weights for policy 0, policy_version 226070 (0.0009) [2023-12-26 17:00:40,531][105620] Updated weights for policy 1, policy_version 226673 (0.0010) [2023-12-26 17:00:40,582][105692] Updated weights for policy 0, policy_version 226080 (0.0008) [2023-12-26 17:00:40,588][105620] Updated weights for policy 1, policy_version 226683 (0.0010) [2023-12-26 17:00:40,636][105692] Updated weights for policy 0, policy_version 226090 (0.0007) [2023-12-26 17:00:40,645][105620] Updated weights for policy 1, policy_version 226693 (0.0006) [2023-12-26 17:00:41,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 115933184. Throughput: 0: 9714.2, 1: 9685.1. Samples: 115941752. Policy #0 lag: (min: 28.0, avg: 53.1, max: 60.0) [2023-12-26 17:00:41,063][104569] Avg episode reward: [(0, '9179.133'), (1, '8920.674')] [2023-12-26 17:00:41,328][105692] Updated weights for policy 0, policy_version 226100 (0.0006) [2023-12-26 17:00:41,403][105692] Updated weights for policy 0, policy_version 226110 (0.0009) [2023-12-26 17:00:41,466][105692] Updated weights for policy 0, policy_version 226120 (0.0007) [2023-12-26 17:00:41,480][105620] Updated weights for policy 1, policy_version 226703 (0.0008) [2023-12-26 17:00:41,544][105620] Updated weights for policy 1, policy_version 226713 (0.0008) [2023-12-26 17:00:41,606][105620] Updated weights for policy 1, policy_version 226723 (0.0008) [2023-12-26 17:00:42,254][105692] Updated weights for policy 0, policy_version 226130 (0.0008) [2023-12-26 17:00:42,279][105620] Updated weights for policy 1, policy_version 226733 (0.0008) [2023-12-26 17:00:42,318][105692] Updated weights for policy 0, policy_version 226140 (0.0008) [2023-12-26 17:00:42,342][105620] Updated weights for policy 1, policy_version 226743 (0.0007) [2023-12-26 17:00:42,388][105692] Updated weights for policy 0, policy_version 226150 (0.0008) [2023-12-26 17:00:42,408][105620] Updated weights for policy 1, policy_version 226753 (0.0008) [2023-12-26 17:00:42,451][105692] Updated weights for policy 0, policy_version 226160 (0.0008) [2023-12-26 17:00:43,087][105692] Updated weights for policy 0, policy_version 226170 (0.0006) [2023-12-26 17:00:43,147][105692] Updated weights for policy 0, policy_version 226180 (0.0010) [2023-12-26 17:00:43,199][105692] Updated weights for policy 0, policy_version 226190 (0.0010) [2023-12-26 17:00:43,217][105620] Updated weights for policy 1, policy_version 226763 (0.0007) [2023-12-26 17:00:43,266][105620] Updated weights for policy 1, policy_version 226773 (0.0008) [2023-12-26 17:00:43,317][105620] Updated weights for policy 1, policy_version 226783 (0.0008) [2023-12-26 17:00:43,914][105692] Updated weights for policy 0, policy_version 226200 (0.0008) [2023-12-26 17:00:43,972][105692] Updated weights for policy 0, policy_version 226210 (0.0010) [2023-12-26 17:00:44,032][105692] Updated weights for policy 0, policy_version 226220 (0.0010) [2023-12-26 17:00:44,093][105620] Updated weights for policy 1, policy_version 226793 (0.0008) [2023-12-26 17:00:44,154][105620] Updated weights for policy 1, policy_version 226803 (0.0008) [2023-12-26 17:00:44,203][105620] Updated weights for policy 1, policy_version 226813 (0.0008) [2023-12-26 17:00:44,266][105620] Updated weights for policy 1, policy_version 226823 (0.0008) [2023-12-26 17:00:44,796][105692] Updated weights for policy 0, policy_version 226230 (0.0010) [2023-12-26 17:00:44,848][105692] Updated weights for policy 0, policy_version 226240 (0.0010) [2023-12-26 17:00:44,905][105692] Updated weights for policy 0, policy_version 226250 (0.0010) [2023-12-26 17:00:45,067][105620] Updated weights for policy 1, policy_version 226833 (0.0008) [2023-12-26 17:00:45,130][105620] Updated weights for policy 1, policy_version 226843 (0.0008) [2023-12-26 17:00:45,191][105620] Updated weights for policy 1, policy_version 226853 (0.0008) [2023-12-26 17:00:45,699][105692] Updated weights for policy 0, policy_version 226260 (0.0010) [2023-12-26 17:00:45,768][105692] Updated weights for policy 0, policy_version 226270 (0.0011) [2023-12-26 17:00:45,832][105692] Updated weights for policy 0, policy_version 226280 (0.0011) [2023-12-26 17:00:45,993][105620] Updated weights for policy 1, policy_version 226863 (0.0008) [2023-12-26 17:00:46,050][105620] Updated weights for policy 1, policy_version 226873 (0.0008) [2023-12-26 17:00:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.1, 300 sec: 19494.2). Total num frames: 116023296. Throughput: 0: 9710.6, 1: 9643.3. Samples: 115998664. Policy #0 lag: (min: 28.0, avg: 53.1, max: 60.0) [2023-12-26 17:00:46,062][104569] Avg episode reward: [(0, '9269.068'), (1, '8838.421')] [2023-12-26 17:00:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000226288_57942016.pth... [2023-12-26 17:00:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000225168_57655296.pth [2023-12-26 17:00:46,106][105620] Updated weights for policy 1, policy_version 226883 (0.0008) [2023-12-26 17:00:46,137][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000226888_58089472.pth... [2023-12-26 17:00:46,141][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000225768_57802752.pth [2023-12-26 17:00:46,571][105692] Updated weights for policy 0, policy_version 226290 (0.0011) [2023-12-26 17:00:46,630][105692] Updated weights for policy 0, policy_version 226300 (0.0010) [2023-12-26 17:00:46,693][105692] Updated weights for policy 0, policy_version 226310 (0.0011) [2023-12-26 17:00:46,751][105692] Updated weights for policy 0, policy_version 226320 (0.0011) [2023-12-26 17:00:46,884][105620] Updated weights for policy 1, policy_version 226893 (0.0008) [2023-12-26 17:00:46,934][105620] Updated weights for policy 1, policy_version 226903 (0.0009) [2023-12-26 17:00:46,992][105620] Updated weights for policy 1, policy_version 226913 (0.0007) [2023-12-26 17:00:47,491][105692] Updated weights for policy 0, policy_version 226330 (0.0010) [2023-12-26 17:00:47,548][105692] Updated weights for policy 0, policy_version 226340 (0.0010) [2023-12-26 17:00:47,596][105692] Updated weights for policy 0, policy_version 226350 (0.0010) [2023-12-26 17:00:47,751][105620] Updated weights for policy 1, policy_version 226923 (0.0007) [2023-12-26 17:00:47,810][105620] Updated weights for policy 1, policy_version 226933 (0.0008) [2023-12-26 17:00:47,868][105620] Updated weights for policy 1, policy_version 226943 (0.0008) [2023-12-26 17:00:48,371][105692] Updated weights for policy 0, policy_version 226360 (0.0010) [2023-12-26 17:00:48,431][105692] Updated weights for policy 0, policy_version 226370 (0.0010) [2023-12-26 17:00:48,487][105692] Updated weights for policy 0, policy_version 226380 (0.0010) [2023-12-26 17:00:48,647][105620] Updated weights for policy 1, policy_version 226953 (0.0009) [2023-12-26 17:00:48,706][105620] Updated weights for policy 1, policy_version 226963 (0.0008) [2023-12-26 17:00:48,759][105620] Updated weights for policy 1, policy_version 226973 (0.0008) [2023-12-26 17:00:48,822][105620] Updated weights for policy 1, policy_version 226983 (0.0008) [2023-12-26 17:00:49,240][105692] Updated weights for policy 0, policy_version 226390 (0.0010) [2023-12-26 17:00:49,307][105692] Updated weights for policy 0, policy_version 226400 (0.0010) [2023-12-26 17:00:49,376][105692] Updated weights for policy 0, policy_version 226410 (0.0010) [2023-12-26 17:00:49,617][105620] Updated weights for policy 1, policy_version 226993 (0.0008) [2023-12-26 17:00:49,673][105620] Updated weights for policy 1, policy_version 227003 (0.0008) [2023-12-26 17:00:49,721][105620] Updated weights for policy 1, policy_version 227013 (0.0008) [2023-12-26 17:00:50,120][105692] Updated weights for policy 0, policy_version 226420 (0.0010) [2023-12-26 17:00:50,172][105692] Updated weights for policy 0, policy_version 226430 (0.0010) [2023-12-26 17:00:50,220][105692] Updated weights for policy 0, policy_version 226440 (0.0010) [2023-12-26 17:00:50,465][105620] Updated weights for policy 1, policy_version 227023 (0.0009) [2023-12-26 17:00:50,531][105620] Updated weights for policy 1, policy_version 227033 (0.0009) [2023-12-26 17:00:50,600][105620] Updated weights for policy 1, policy_version 227043 (0.0008) [2023-12-26 17:00:50,969][105692] Updated weights for policy 0, policy_version 226450 (0.0009) [2023-12-26 17:00:51,038][105692] Updated weights for policy 0, policy_version 226460 (0.0009) [2023-12-26 17:00:51,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 116113408. Throughput: 0: 9583.9, 1: 9512.5. Samples: 116107528. Policy #0 lag: (min: 28.0, avg: 53.1, max: 60.0) [2023-12-26 17:00:51,063][104569] Avg episode reward: [(0, '9269.177'), (1, '9272.683')] [2023-12-26 17:00:51,108][105692] Updated weights for policy 0, policy_version 226470 (0.0007) [2023-12-26 17:00:51,168][105692] Updated weights for policy 0, policy_version 226480 (0.0007) [2023-12-26 17:00:51,404][105620] Updated weights for policy 1, policy_version 227053 (0.0009) [2023-12-26 17:00:51,467][105620] Updated weights for policy 1, policy_version 227063 (0.0008) [2023-12-26 17:00:51,523][105620] Updated weights for policy 1, policy_version 227073 (0.0010) [2023-12-26 17:00:51,828][105692] Updated weights for policy 0, policy_version 226490 (0.0009) [2023-12-26 17:00:51,884][105692] Updated weights for policy 0, policy_version 226500 (0.0007) [2023-12-26 17:00:51,947][105692] Updated weights for policy 0, policy_version 226510 (0.0009) [2023-12-26 17:00:52,325][105620] Updated weights for policy 1, policy_version 227083 (0.0008) [2023-12-26 17:00:52,390][105620] Updated weights for policy 1, policy_version 227093 (0.0008) [2023-12-26 17:00:52,452][105620] Updated weights for policy 1, policy_version 227103 (0.0006) [2023-12-26 17:00:52,692][105692] Updated weights for policy 0, policy_version 226520 (0.0006) [2023-12-26 17:00:52,744][105692] Updated weights for policy 0, policy_version 226530 (0.0007) [2023-12-26 17:00:52,799][105692] Updated weights for policy 0, policy_version 226540 (0.0007) [2023-12-26 17:00:53,055][105620] Updated weights for policy 1, policy_version 227113 (0.0007) [2023-12-26 17:00:53,119][105620] Updated weights for policy 1, policy_version 227123 (0.0010) [2023-12-26 17:00:53,170][105620] Updated weights for policy 1, policy_version 227133 (0.0009) [2023-12-26 17:00:53,219][105620] Updated weights for policy 1, policy_version 227143 (0.0008) [2023-12-26 17:00:53,477][105692] Updated weights for policy 0, policy_version 226550 (0.0008) [2023-12-26 17:00:53,534][105692] Updated weights for policy 0, policy_version 226560 (0.0005) [2023-12-26 17:00:53,601][105692] Updated weights for policy 0, policy_version 226570 (0.0009) [2023-12-26 17:00:53,775][105620] Updated weights for policy 1, policy_version 227153 (0.0008) [2023-12-26 17:00:53,833][105620] Updated weights for policy 1, policy_version 227163 (0.0010) [2023-12-26 17:00:53,887][105620] Updated weights for policy 1, policy_version 227173 (0.0010) [2023-12-26 17:00:54,161][105692] Updated weights for policy 0, policy_version 226580 (0.0008) [2023-12-26 17:00:54,212][105692] Updated weights for policy 0, policy_version 226590 (0.0006) [2023-12-26 17:00:54,258][105692] Updated weights for policy 0, policy_version 226600 (0.0005) [2023-12-26 17:00:54,542][105620] Updated weights for policy 1, policy_version 227183 (0.0007) [2023-12-26 17:00:54,598][105620] Updated weights for policy 1, policy_version 227193 (0.0005) [2023-12-26 17:00:54,664][105620] Updated weights for policy 1, policy_version 227203 (0.0005) [2023-12-26 17:00:54,951][105692] Updated weights for policy 0, policy_version 226610 (0.0006) [2023-12-26 17:00:55,013][105692] Updated weights for policy 0, policy_version 226620 (0.0009) [2023-12-26 17:00:55,075][105692] Updated weights for policy 0, policy_version 226630 (0.0006) [2023-12-26 17:00:55,138][105692] Updated weights for policy 0, policy_version 226640 (0.0006) [2023-12-26 17:00:55,178][105620] Updated weights for policy 1, policy_version 227213 (0.0005) [2023-12-26 17:00:55,234][105620] Updated weights for policy 1, policy_version 227223 (0.0005) [2023-12-26 17:00:55,283][105620] Updated weights for policy 1, policy_version 227233 (0.0005) [2023-12-26 17:00:55,705][105692] Updated weights for policy 0, policy_version 226650 (0.0005) [2023-12-26 17:00:55,766][105692] Updated weights for policy 0, policy_version 226660 (0.0010) [2023-12-26 17:00:55,824][105692] Updated weights for policy 0, policy_version 226670 (0.0010) [2023-12-26 17:00:55,913][105620] Updated weights for policy 1, policy_version 227243 (0.0006) [2023-12-26 17:00:55,974][105620] Updated weights for policy 1, policy_version 227253 (0.0007) [2023-12-26 17:00:56,035][105620] Updated weights for policy 1, policy_version 227263 (0.0006) [2023-12-26 17:00:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 116219904. Throughput: 0: 9635.5, 1: 9609.3. Samples: 116231368. Policy #0 lag: (min: 28.0, avg: 53.1, max: 60.0) [2023-12-26 17:00:56,062][104569] Avg episode reward: [(0, '9356.919'), (1, '9354.669')] [2023-12-26 17:00:56,488][105692] Updated weights for policy 0, policy_version 226680 (0.0008) [2023-12-26 17:00:56,546][105692] Updated weights for policy 0, policy_version 226690 (0.0009) [2023-12-26 17:00:56,603][105692] Updated weights for policy 0, policy_version 226700 (0.0008) [2023-12-26 17:00:56,642][105620] Updated weights for policy 1, policy_version 227273 (0.0007) [2023-12-26 17:00:56,699][105620] Updated weights for policy 1, policy_version 227283 (0.0010) [2023-12-26 17:00:56,756][105620] Updated weights for policy 1, policy_version 227293 (0.0010) [2023-12-26 17:00:56,807][105620] Updated weights for policy 1, policy_version 227303 (0.0010) [2023-12-26 17:00:57,344][105692] Updated weights for policy 0, policy_version 226710 (0.0009) [2023-12-26 17:00:57,391][105692] Updated weights for policy 0, policy_version 226720 (0.0008) [2023-12-26 17:00:57,438][105692] Updated weights for policy 0, policy_version 226730 (0.0007) [2023-12-26 17:00:57,534][105620] Updated weights for policy 1, policy_version 227313 (0.0010) [2023-12-26 17:00:57,578][105620] Updated weights for policy 1, policy_version 227323 (0.0010) [2023-12-26 17:00:57,629][105620] Updated weights for policy 1, policy_version 227333 (0.0010) [2023-12-26 17:00:58,153][105692] Updated weights for policy 0, policy_version 226740 (0.0009) [2023-12-26 17:00:58,212][105692] Updated weights for policy 0, policy_version 226750 (0.0010) [2023-12-26 17:00:58,268][105692] Updated weights for policy 0, policy_version 226760 (0.0010) [2023-12-26 17:00:58,438][105620] Updated weights for policy 1, policy_version 227343 (0.0008) [2023-12-26 17:00:58,499][105620] Updated weights for policy 1, policy_version 227353 (0.0008) [2023-12-26 17:00:58,574][105620] Updated weights for policy 1, policy_version 227363 (0.0010) [2023-12-26 17:00:59,091][105692] Updated weights for policy 0, policy_version 226770 (0.0010) [2023-12-26 17:00:59,149][105692] Updated weights for policy 0, policy_version 226780 (0.0007) [2023-12-26 17:00:59,205][105692] Updated weights for policy 0, policy_version 226790 (0.0008) [2023-12-26 17:00:59,293][105692] Updated weights for policy 0, policy_version 226800 (0.0007) [2023-12-26 17:00:59,343][105620] Updated weights for policy 1, policy_version 227373 (0.0010) [2023-12-26 17:00:59,408][105620] Updated weights for policy 1, policy_version 227383 (0.0008) [2023-12-26 17:00:59,470][105620] Updated weights for policy 1, policy_version 227393 (0.0008) [2023-12-26 17:00:59,928][105692] Updated weights for policy 0, policy_version 226810 (0.0007) [2023-12-26 17:00:59,982][105692] Updated weights for policy 0, policy_version 226820 (0.0009) [2023-12-26 17:01:00,045][105692] Updated weights for policy 0, policy_version 226830 (0.0010) [2023-12-26 17:01:00,245][105620] Updated weights for policy 1, policy_version 227403 (0.0008) [2023-12-26 17:01:00,300][105620] Updated weights for policy 1, policy_version 227413 (0.0010) [2023-12-26 17:01:00,348][105620] Updated weights for policy 1, policy_version 227423 (0.0010) [2023-12-26 17:01:00,787][105692] Updated weights for policy 0, policy_version 226840 (0.0008) [2023-12-26 17:01:00,842][105692] Updated weights for policy 0, policy_version 226851 (0.0009) [2023-12-26 17:01:00,905][105692] Updated weights for policy 0, policy_version 226861 (0.0009) [2023-12-26 17:01:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 116318208. Throughput: 0: 9606.6, 1: 9605.2. Samples: 116288692. Policy #0 lag: (min: 28.0, avg: 53.1, max: 60.0) [2023-12-26 17:01:01,063][104569] Avg episode reward: [(0, '9265.510'), (1, '9354.307')] [2023-12-26 17:01:01,065][105620] Updated weights for policy 1, policy_version 227433 (0.0008) [2023-12-26 17:01:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000226864_58089472.pth... [2023-12-26 17:01:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000225744_57802752.pth [2023-12-26 17:01:01,126][105620] Updated weights for policy 1, policy_version 227443 (0.0009) [2023-12-26 17:01:01,180][105620] Updated weights for policy 1, policy_version 227453 (0.0011) [2023-12-26 17:01:01,242][105620] Updated weights for policy 1, policy_version 227463 (0.0011) [2023-12-26 17:01:01,248][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000227464_58236928.pth... [2023-12-26 17:01:01,251][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000226344_57950208.pth [2023-12-26 17:01:01,618][105692] Updated weights for policy 0, policy_version 226871 (0.0007) [2023-12-26 17:01:01,671][105692] Updated weights for policy 0, policy_version 226881 (0.0007) [2023-12-26 17:01:01,724][105692] Updated weights for policy 0, policy_version 226891 (0.0006) [2023-12-26 17:01:02,002][105620] Updated weights for policy 1, policy_version 227473 (0.0010) [2023-12-26 17:01:02,055][105620] Updated weights for policy 1, policy_version 227484 (0.0009) [2023-12-26 17:01:02,110][105620] Updated weights for policy 1, policy_version 227494 (0.0006) [2023-12-26 17:01:02,403][105692] Updated weights for policy 0, policy_version 226901 (0.0007) [2023-12-26 17:01:02,474][105692] Updated weights for policy 0, policy_version 226911 (0.0010) [2023-12-26 17:01:02,539][105692] Updated weights for policy 0, policy_version 226921 (0.0010) [2023-12-26 17:01:02,731][105620] Updated weights for policy 1, policy_version 227504 (0.0006) [2023-12-26 17:01:02,793][105620] Updated weights for policy 1, policy_version 227514 (0.0008) [2023-12-26 17:01:02,853][105620] Updated weights for policy 1, policy_version 227524 (0.0010) [2023-12-26 17:01:03,385][105692] Updated weights for policy 0, policy_version 226931 (0.0010) [2023-12-26 17:01:03,417][105620] Updated weights for policy 1, policy_version 227534 (0.0007) [2023-12-26 17:01:03,446][105692] Updated weights for policy 0, policy_version 226941 (0.0006) [2023-12-26 17:01:03,477][105620] Updated weights for policy 1, policy_version 227544 (0.0005) [2023-12-26 17:01:03,502][105692] Updated weights for policy 0, policy_version 226951 (0.0008) [2023-12-26 17:01:03,534][105620] Updated weights for policy 1, policy_version 227554 (0.0005) [2023-12-26 17:01:04,128][105620] Updated weights for policy 1, policy_version 227564 (0.0008) [2023-12-26 17:01:04,198][105620] Updated weights for policy 1, policy_version 227574 (0.0011) [2023-12-26 17:01:04,257][105692] Updated weights for policy 0, policy_version 226961 (0.0008) [2023-12-26 17:01:04,265][105620] Updated weights for policy 1, policy_version 227584 (0.0011) [2023-12-26 17:01:04,322][105692] Updated weights for policy 0, policy_version 226971 (0.0007) [2023-12-26 17:01:04,385][105692] Updated weights for policy 0, policy_version 226981 (0.0008) [2023-12-26 17:01:04,441][105692] Updated weights for policy 0, policy_version 226991 (0.0010) [2023-12-26 17:01:05,002][105620] Updated weights for policy 1, policy_version 227594 (0.0010) [2023-12-26 17:01:05,062][105620] Updated weights for policy 1, policy_version 227604 (0.0011) [2023-12-26 17:01:05,121][105620] Updated weights for policy 1, policy_version 227614 (0.0010) [2023-12-26 17:01:05,158][105692] Updated weights for policy 0, policy_version 227001 (0.0009) [2023-12-26 17:01:05,180][105620] Updated weights for policy 1, policy_version 227624 (0.0010) [2023-12-26 17:01:05,220][105692] Updated weights for policy 0, policy_version 227011 (0.0007) [2023-12-26 17:01:05,274][105692] Updated weights for policy 0, policy_version 227021 (0.0008) [2023-12-26 17:01:05,814][105620] Updated weights for policy 1, policy_version 227634 (0.0005) [2023-12-26 17:01:05,858][105620] Updated weights for policy 1, policy_version 227644 (0.0005) [2023-12-26 17:01:05,913][105620] Updated weights for policy 1, policy_version 227654 (0.0006) [2023-12-26 17:01:05,919][105692] Updated weights for policy 0, policy_version 227031 (0.0008) [2023-12-26 17:01:05,965][105692] Updated weights for policy 0, policy_version 227041 (0.0005) [2023-12-26 17:01:06,016][105692] Updated weights for policy 0, policy_version 227051 (0.0005) [2023-12-26 17:01:06,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 116424704. Throughput: 0: 9584.8, 1: 9696.6. Samples: 116405740. Policy #0 lag: (min: 28.0, avg: 53.1, max: 60.0) [2023-12-26 17:01:06,062][104569] Avg episode reward: [(0, '9265.329'), (1, '9265.900')] [2023-12-26 17:01:06,533][105620] Updated weights for policy 1, policy_version 227664 (0.0010) [2023-12-26 17:01:06,600][105620] Updated weights for policy 1, policy_version 227674 (0.0011) [2023-12-26 17:01:06,664][105620] Updated weights for policy 1, policy_version 227684 (0.0011) [2023-12-26 17:01:06,697][105692] Updated weights for policy 0, policy_version 227061 (0.0008) [2023-12-26 17:01:06,749][105692] Updated weights for policy 0, policy_version 227071 (0.0008) [2023-12-26 17:01:06,802][105692] Updated weights for policy 0, policy_version 227081 (0.0008) [2023-12-26 17:01:07,375][105620] Updated weights for policy 1, policy_version 227694 (0.0007) [2023-12-26 17:01:07,429][105620] Updated weights for policy 1, policy_version 227704 (0.0005) [2023-12-26 17:01:07,480][105620] Updated weights for policy 1, policy_version 227714 (0.0005) [2023-12-26 17:01:07,635][105692] Updated weights for policy 0, policy_version 227091 (0.0009) [2023-12-26 17:01:07,699][105692] Updated weights for policy 0, policy_version 227101 (0.0010) [2023-12-26 17:01:07,747][105692] Updated weights for policy 0, policy_version 227111 (0.0006) [2023-12-26 17:01:08,113][105620] Updated weights for policy 1, policy_version 227724 (0.0007) [2023-12-26 17:01:08,178][105620] Updated weights for policy 1, policy_version 227734 (0.0009) [2023-12-26 17:01:08,235][105620] Updated weights for policy 1, policy_version 227744 (0.0008) [2023-12-26 17:01:08,401][105692] Updated weights for policy 0, policy_version 227121 (0.0006) [2023-12-26 17:01:08,455][105692] Updated weights for policy 0, policy_version 227131 (0.0009) [2023-12-26 17:01:08,503][105692] Updated weights for policy 0, policy_version 227141 (0.0009) [2023-12-26 17:01:08,557][105692] Updated weights for policy 0, policy_version 227151 (0.0009) [2023-12-26 17:01:08,910][105620] Updated weights for policy 1, policy_version 227754 (0.0005) [2023-12-26 17:01:08,968][105620] Updated weights for policy 1, policy_version 227764 (0.0008) [2023-12-26 17:01:09,020][105620] Updated weights for policy 1, policy_version 227774 (0.0005) [2023-12-26 17:01:09,085][105620] Updated weights for policy 1, policy_version 227784 (0.0008) [2023-12-26 17:01:09,294][105692] Updated weights for policy 0, policy_version 227161 (0.0009) [2023-12-26 17:01:09,351][105692] Updated weights for policy 0, policy_version 227171 (0.0007) [2023-12-26 17:01:09,420][105692] Updated weights for policy 0, policy_version 227181 (0.0008) [2023-12-26 17:01:09,779][105620] Updated weights for policy 1, policy_version 227794 (0.0009) [2023-12-26 17:01:09,841][105620] Updated weights for policy 1, policy_version 227804 (0.0008) [2023-12-26 17:01:09,902][105620] Updated weights for policy 1, policy_version 227814 (0.0010) [2023-12-26 17:01:10,199][105692] Updated weights for policy 0, policy_version 227191 (0.0008) [2023-12-26 17:01:10,255][105692] Updated weights for policy 0, policy_version 227201 (0.0008) [2023-12-26 17:01:10,314][105692] Updated weights for policy 0, policy_version 227211 (0.0008) [2023-12-26 17:01:10,624][105620] Updated weights for policy 1, policy_version 227824 (0.0010) [2023-12-26 17:01:10,683][105620] Updated weights for policy 1, policy_version 227834 (0.0011) [2023-12-26 17:01:10,744][105620] Updated weights for policy 1, policy_version 227844 (0.0011) [2023-12-26 17:01:10,919][105692] Updated weights for policy 0, policy_version 227221 (0.0007) [2023-12-26 17:01:10,977][105692] Updated weights for policy 0, policy_version 227231 (0.0006) [2023-12-26 17:01:11,038][105692] Updated weights for policy 0, policy_version 227241 (0.0007) [2023-12-26 17:01:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 116514816. Throughput: 0: 9566.0, 1: 9783.3. Samples: 116525564. Policy #0 lag: (min: 28.0, avg: 53.1, max: 60.0) [2023-12-26 17:01:11,062][104569] Avg episode reward: [(0, '9265.457'), (1, '9264.860')] [2023-12-26 17:01:11,562][105620] Updated weights for policy 1, policy_version 227854 (0.0011) [2023-12-26 17:01:11,637][105620] Updated weights for policy 1, policy_version 227864 (0.0010) [2023-12-26 17:01:11,701][105620] Updated weights for policy 1, policy_version 227874 (0.0010) [2023-12-26 17:01:11,762][105692] Updated weights for policy 0, policy_version 227251 (0.0008) [2023-12-26 17:01:11,828][105692] Updated weights for policy 0, policy_version 227261 (0.0007) [2023-12-26 17:01:11,882][105692] Updated weights for policy 0, policy_version 227271 (0.0006) [2023-12-26 17:01:12,468][105620] Updated weights for policy 1, policy_version 227884 (0.0011) [2023-12-26 17:01:12,516][105620] Updated weights for policy 1, policy_version 227894 (0.0010) [2023-12-26 17:01:12,564][105620] Updated weights for policy 1, policy_version 227904 (0.0010) [2023-12-26 17:01:12,627][105692] Updated weights for policy 0, policy_version 227281 (0.0007) [2023-12-26 17:01:12,693][105692] Updated weights for policy 0, policy_version 227291 (0.0010) [2023-12-26 17:01:12,753][105692] Updated weights for policy 0, policy_version 227301 (0.0011) [2023-12-26 17:01:12,811][105692] Updated weights for policy 0, policy_version 227311 (0.0010) [2023-12-26 17:01:13,327][105620] Updated weights for policy 1, policy_version 227914 (0.0010) [2023-12-26 17:01:13,386][105620] Updated weights for policy 1, policy_version 227924 (0.0005) [2023-12-26 17:01:13,435][105620] Updated weights for policy 1, policy_version 227934 (0.0005) [2023-12-26 17:01:13,470][105692] Updated weights for policy 0, policy_version 227321 (0.0007) [2023-12-26 17:01:13,483][105620] Updated weights for policy 1, policy_version 227944 (0.0006) [2023-12-26 17:01:13,526][105692] Updated weights for policy 0, policy_version 227331 (0.0009) [2023-12-26 17:01:13,578][105692] Updated weights for policy 0, policy_version 227341 (0.0008) [2023-12-26 17:01:14,076][105620] Updated weights for policy 1, policy_version 227954 (0.0010) [2023-12-26 17:01:14,128][105620] Updated weights for policy 1, policy_version 227965 (0.0009) [2023-12-26 17:01:14,183][105620] Updated weights for policy 1, policy_version 227975 (0.0007) [2023-12-26 17:01:14,331][105692] Updated weights for policy 0, policy_version 227351 (0.0008) [2023-12-26 17:01:14,376][105692] Updated weights for policy 0, policy_version 227361 (0.0008) [2023-12-26 17:01:14,420][105692] Updated weights for policy 0, policy_version 227371 (0.0008) [2023-12-26 17:01:14,943][105620] Updated weights for policy 1, policy_version 227985 (0.0010) [2023-12-26 17:01:15,010][105620] Updated weights for policy 1, policy_version 227995 (0.0011) [2023-12-26 17:01:15,070][105620] Updated weights for policy 1, policy_version 228005 (0.0011) [2023-12-26 17:01:15,162][105692] Updated weights for policy 0, policy_version 227381 (0.0008) [2023-12-26 17:01:15,223][105692] Updated weights for policy 0, policy_version 227391 (0.0008) [2023-12-26 17:01:15,290][105692] Updated weights for policy 0, policy_version 227401 (0.0006) [2023-12-26 17:01:15,796][105620] Updated weights for policy 1, policy_version 228015 (0.0007) [2023-12-26 17:01:15,843][105620] Updated weights for policy 1, policy_version 228025 (0.0006) [2023-12-26 17:01:15,889][105620] Updated weights for policy 1, policy_version 228035 (0.0005) [2023-12-26 17:01:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 116613120. Throughput: 0: 9565.2, 1: 9717.3. Samples: 116584900. Policy #0 lag: (min: 28.0, avg: 53.1, max: 60.0) [2023-12-26 17:01:16,062][104569] Avg episode reward: [(0, '9356.894'), (1, '9262.903')] [2023-12-26 17:01:16,064][105692] Updated weights for policy 0, policy_version 227411 (0.0009) [2023-12-26 17:01:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000228040_58384384.pth... [2023-12-26 17:01:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000226888_58089472.pth [2023-12-26 17:01:16,125][105692] Updated weights for policy 0, policy_version 227421 (0.0009) [2023-12-26 17:01:16,184][105692] Updated weights for policy 0, policy_version 227431 (0.0009) [2023-12-26 17:01:16,235][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000227440_58236928.pth... [2023-12-26 17:01:16,240][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000226288_57942016.pth [2023-12-26 17:01:16,604][105620] Updated weights for policy 1, policy_version 228045 (0.0007) [2023-12-26 17:01:16,656][105620] Updated weights for policy 1, policy_version 228055 (0.0009) [2023-12-26 17:01:16,707][105620] Updated weights for policy 1, policy_version 228065 (0.0008) [2023-12-26 17:01:16,961][105692] Updated weights for policy 0, policy_version 227441 (0.0009) [2023-12-26 17:01:17,017][105692] Updated weights for policy 0, policy_version 227451 (0.0008) [2023-12-26 17:01:17,075][105692] Updated weights for policy 0, policy_version 227461 (0.0007) [2023-12-26 17:01:17,136][105692] Updated weights for policy 0, policy_version 227471 (0.0009) [2023-12-26 17:01:17,441][105620] Updated weights for policy 1, policy_version 228075 (0.0006) [2023-12-26 17:01:17,503][105620] Updated weights for policy 1, policy_version 228085 (0.0005) [2023-12-26 17:01:17,556][105620] Updated weights for policy 1, policy_version 228095 (0.0005) [2023-12-26 17:01:17,875][105692] Updated weights for policy 0, policy_version 227482 (0.0010) [2023-12-26 17:01:17,936][105692] Updated weights for policy 0, policy_version 227493 (0.0010) [2023-12-26 17:01:17,990][105692] Updated weights for policy 0, policy_version 227503 (0.0010) [2023-12-26 17:01:18,168][105620] Updated weights for policy 1, policy_version 228105 (0.0006) [2023-12-26 17:01:18,229][105620] Updated weights for policy 1, policy_version 228115 (0.0008) [2023-12-26 17:01:18,290][105620] Updated weights for policy 1, policy_version 228125 (0.0009) [2023-12-26 17:01:18,358][105620] Updated weights for policy 1, policy_version 228135 (0.0009) [2023-12-26 17:01:18,790][105692] Updated weights for policy 0, policy_version 227513 (0.0008) [2023-12-26 17:01:18,859][105692] Updated weights for policy 0, policy_version 227523 (0.0009) [2023-12-26 17:01:18,926][105692] Updated weights for policy 0, policy_version 227533 (0.0009) [2023-12-26 17:01:19,044][105620] Updated weights for policy 1, policy_version 228145 (0.0006) [2023-12-26 17:01:19,104][105620] Updated weights for policy 1, policy_version 228155 (0.0007) [2023-12-26 17:01:19,165][105620] Updated weights for policy 1, policy_version 228165 (0.0009) [2023-12-26 17:01:19,703][105692] Updated weights for policy 0, policy_version 227543 (0.0008) [2023-12-26 17:01:19,764][105692] Updated weights for policy 0, policy_version 227553 (0.0009) [2023-12-26 17:01:19,825][105692] Updated weights for policy 0, policy_version 227563 (0.0009) [2023-12-26 17:01:19,896][105620] Updated weights for policy 1, policy_version 228175 (0.0008) [2023-12-26 17:01:19,958][105620] Updated weights for policy 1, policy_version 228185 (0.0008) [2023-12-26 17:01:20,020][105620] Updated weights for policy 1, policy_version 228195 (0.0006) [2023-12-26 17:01:20,652][105692] Updated weights for policy 0, policy_version 227573 (0.0009) [2023-12-26 17:01:20,716][105692] Updated weights for policy 0, policy_version 227583 (0.0008) [2023-12-26 17:01:20,779][105692] Updated weights for policy 0, policy_version 227593 (0.0007) [2023-12-26 17:01:20,785][105620] Updated weights for policy 1, policy_version 228205 (0.0008) [2023-12-26 17:01:20,849][105620] Updated weights for policy 1, policy_version 228215 (0.0009) [2023-12-26 17:01:20,909][105620] Updated weights for policy 1, policy_version 228225 (0.0009) [2023-12-26 17:01:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 116711424. Throughput: 0: 9593.8, 1: 9697.3. Samples: 116698884. Policy #0 lag: (min: 28.0, avg: 53.1, max: 60.0) [2023-12-26 17:01:21,063][104569] Avg episode reward: [(0, '9356.664'), (1, '9091.862')] [2023-12-26 17:01:21,558][105692] Updated weights for policy 0, policy_version 227603 (0.0007) [2023-12-26 17:01:21,627][105692] Updated weights for policy 0, policy_version 227613 (0.0006) [2023-12-26 17:01:21,690][105692] Updated weights for policy 0, policy_version 227623 (0.0008) [2023-12-26 17:01:21,692][105620] Updated weights for policy 1, policy_version 228235 (0.0009) [2023-12-26 17:01:21,764][105620] Updated weights for policy 1, policy_version 228245 (0.0009) [2023-12-26 17:01:21,827][105620] Updated weights for policy 1, policy_version 228255 (0.0010) [2023-12-26 17:01:22,405][105692] Updated weights for policy 0, policy_version 227633 (0.0008) [2023-12-26 17:01:22,472][105692] Updated weights for policy 0, policy_version 227643 (0.0006) [2023-12-26 17:01:22,534][105692] Updated weights for policy 0, policy_version 227653 (0.0010) [2023-12-26 17:01:22,594][105692] Updated weights for policy 0, policy_version 227663 (0.0010) [2023-12-26 17:01:22,612][105620] Updated weights for policy 1, policy_version 228265 (0.0008) [2023-12-26 17:01:22,676][105620] Updated weights for policy 1, policy_version 228275 (0.0008) [2023-12-26 17:01:22,743][105620] Updated weights for policy 1, policy_version 228285 (0.0008) [2023-12-26 17:01:22,799][105620] Updated weights for policy 1, policy_version 228295 (0.0006) [2023-12-26 17:01:23,328][105692] Updated weights for policy 0, policy_version 227673 (0.0010) [2023-12-26 17:01:23,385][105692] Updated weights for policy 0, policy_version 227683 (0.0010) [2023-12-26 17:01:23,443][105692] Updated weights for policy 0, policy_version 227693 (0.0005) [2023-12-26 17:01:23,512][105620] Updated weights for policy 1, policy_version 228305 (0.0008) [2023-12-26 17:01:23,582][105620] Updated weights for policy 1, policy_version 228315 (0.0005) [2023-12-26 17:01:23,638][105620] Updated weights for policy 1, policy_version 228325 (0.0006) [2023-12-26 17:01:24,096][105692] Updated weights for policy 0, policy_version 227703 (0.0009) [2023-12-26 17:01:24,165][105692] Updated weights for policy 0, policy_version 227713 (0.0011) [2023-12-26 17:01:24,215][105620] Updated weights for policy 1, policy_version 228335 (0.0006) [2023-12-26 17:01:24,228][105692] Updated weights for policy 0, policy_version 227723 (0.0010) [2023-12-26 17:01:24,277][105620] Updated weights for policy 1, policy_version 228345 (0.0007) [2023-12-26 17:01:24,328][105620] Updated weights for policy 1, policy_version 228355 (0.0008) [2023-12-26 17:01:24,936][105692] Updated weights for policy 0, policy_version 227733 (0.0010) [2023-12-26 17:01:25,002][105692] Updated weights for policy 0, policy_version 227743 (0.0008) [2023-12-26 17:01:25,020][105620] Updated weights for policy 1, policy_version 228365 (0.0006) [2023-12-26 17:01:25,067][105692] Updated weights for policy 0, policy_version 227753 (0.0006) [2023-12-26 17:01:25,087][105620] Updated weights for policy 1, policy_version 228375 (0.0007) [2023-12-26 17:01:25,137][105620] Updated weights for policy 1, policy_version 228385 (0.0009) [2023-12-26 17:01:25,647][105692] Updated weights for policy 0, policy_version 227763 (0.0006) [2023-12-26 17:01:25,696][105692] Updated weights for policy 0, policy_version 227773 (0.0005) [2023-12-26 17:01:25,754][105692] Updated weights for policy 0, policy_version 227783 (0.0007) [2023-12-26 17:01:25,869][105620] Updated weights for policy 1, policy_version 228395 (0.0009) [2023-12-26 17:01:25,923][105620] Updated weights for policy 1, policy_version 228405 (0.0008) [2023-12-26 17:01:25,987][105620] Updated weights for policy 1, policy_version 228415 (0.0009) [2023-12-26 17:01:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 116809728. Throughput: 0: 9592.3, 1: 9774.5. Samples: 116813256. Policy #0 lag: (min: 28.0, avg: 53.1, max: 60.0) [2023-12-26 17:01:26,063][104569] Avg episode reward: [(0, '9264.425'), (1, '9001.220')] [2023-12-26 17:01:26,411][105692] Updated weights for policy 0, policy_version 227793 (0.0009) [2023-12-26 17:01:26,459][105692] Updated weights for policy 0, policy_version 227803 (0.0005) [2023-12-26 17:01:26,510][105692] Updated weights for policy 0, policy_version 227813 (0.0005) [2023-12-26 17:01:26,556][105692] Updated weights for policy 0, policy_version 227823 (0.0005) [2023-12-26 17:01:26,815][105620] Updated weights for policy 1, policy_version 228425 (0.0009) [2023-12-26 17:01:26,872][105620] Updated weights for policy 1, policy_version 228435 (0.0008) [2023-12-26 17:01:26,928][105620] Updated weights for policy 1, policy_version 228445 (0.0007) [2023-12-26 17:01:26,992][105620] Updated weights for policy 1, policy_version 228455 (0.0009) [2023-12-26 17:01:27,141][105692] Updated weights for policy 0, policy_version 227833 (0.0007) [2023-12-26 17:01:27,187][105692] Updated weights for policy 0, policy_version 227843 (0.0008) [2023-12-26 17:01:27,234][105692] Updated weights for policy 0, policy_version 227853 (0.0005) [2023-12-26 17:01:27,806][105620] Updated weights for policy 1, policy_version 228465 (0.0008) [2023-12-26 17:01:27,857][105620] Updated weights for policy 1, policy_version 228475 (0.0008) [2023-12-26 17:01:27,865][105692] Updated weights for policy 0, policy_version 227863 (0.0006) [2023-12-26 17:01:27,905][105620] Updated weights for policy 1, policy_version 228485 (0.0008) [2023-12-26 17:01:27,908][105692] Updated weights for policy 0, policy_version 227873 (0.0005) [2023-12-26 17:01:27,959][105692] Updated weights for policy 0, policy_version 227883 (0.0005) [2023-12-26 17:01:28,641][105620] Updated weights for policy 1, policy_version 228495 (0.0008) [2023-12-26 17:01:28,690][105692] Updated weights for policy 0, policy_version 227893 (0.0008) [2023-12-26 17:01:28,697][105620] Updated weights for policy 1, policy_version 228505 (0.0008) [2023-12-26 17:01:28,748][105692] Updated weights for policy 0, policy_version 227903 (0.0010) [2023-12-26 17:01:28,750][105620] Updated weights for policy 1, policy_version 228515 (0.0008) [2023-12-26 17:01:28,809][105692] Updated weights for policy 0, policy_version 227913 (0.0010) [2023-12-26 17:01:29,494][105620] Updated weights for policy 1, policy_version 228525 (0.0007) [2023-12-26 17:01:29,545][105692] Updated weights for policy 0, policy_version 227923 (0.0010) [2023-12-26 17:01:29,555][105620] Updated weights for policy 1, policy_version 228535 (0.0007) [2023-12-26 17:01:29,597][105692] Updated weights for policy 0, policy_version 227933 (0.0010) [2023-12-26 17:01:29,611][105620] Updated weights for policy 1, policy_version 228545 (0.0005) [2023-12-26 17:01:29,652][105692] Updated weights for policy 0, policy_version 227943 (0.0010) [2023-12-26 17:01:30,284][105620] Updated weights for policy 1, policy_version 228555 (0.0006) [2023-12-26 17:01:30,350][105692] Updated weights for policy 0, policy_version 227953 (0.0010) [2023-12-26 17:01:30,356][105620] Updated weights for policy 1, policy_version 228565 (0.0006) [2023-12-26 17:01:30,417][105692] Updated weights for policy 0, policy_version 227963 (0.0009) [2023-12-26 17:01:30,420][105620] Updated weights for policy 1, policy_version 228575 (0.0006) [2023-12-26 17:01:30,472][105692] Updated weights for policy 0, policy_version 227973 (0.0010) [2023-12-26 17:01:30,524][105692] Updated weights for policy 0, policy_version 227983 (0.0010) [2023-12-26 17:01:31,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 116899840. Throughput: 0: 9671.9, 1: 9752.5. Samples: 116872760. Policy #0 lag: (min: 28.0, avg: 53.1, max: 60.0) [2023-12-26 17:01:31,062][104569] Avg episode reward: [(0, '9264.211'), (1, '9174.469')] [2023-12-26 17:01:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000228584_58523648.pth... [2023-12-26 17:01:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000227464_58236928.pth [2023-12-26 17:01:31,088][105692] Updated weights for policy 0, policy_version 227993 (0.0010) [2023-12-26 17:01:31,126][105620] Updated weights for policy 1, policy_version 228585 (0.0006) [2023-12-26 17:01:31,152][105692] Updated weights for policy 0, policy_version 228003 (0.0010) [2023-12-26 17:01:31,193][105620] Updated weights for policy 1, policy_version 228595 (0.0009) [2023-12-26 17:01:31,212][105692] Updated weights for policy 0, policy_version 228013 (0.0006) [2023-12-26 17:01:31,228][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000228016_58384384.pth... [2023-12-26 17:01:31,232][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000226864_58089472.pth [2023-12-26 17:01:31,258][105620] Updated weights for policy 1, policy_version 228605 (0.0008) [2023-12-26 17:01:31,305][105620] Updated weights for policy 1, policy_version 228615 (0.0005) [2023-12-26 17:01:31,841][105692] Updated weights for policy 0, policy_version 228023 (0.0005) [2023-12-26 17:01:31,890][105692] Updated weights for policy 0, policy_version 228033 (0.0009) [2023-12-26 17:01:31,945][105692] Updated weights for policy 0, policy_version 228043 (0.0010) [2023-12-26 17:01:32,014][105620] Updated weights for policy 1, policy_version 228625 (0.0010) [2023-12-26 17:01:32,070][105620] Updated weights for policy 1, policy_version 228635 (0.0011) [2023-12-26 17:01:32,130][105620] Updated weights for policy 1, policy_version 228645 (0.0011) [2023-12-26 17:01:32,594][105692] Updated weights for policy 0, policy_version 228053 (0.0008) [2023-12-26 17:01:32,652][105692] Updated weights for policy 0, policy_version 228063 (0.0005) [2023-12-26 17:01:32,709][105692] Updated weights for policy 0, policy_version 228073 (0.0007) [2023-12-26 17:01:32,884][105620] Updated weights for policy 1, policy_version 228655 (0.0011) [2023-12-26 17:01:32,940][105620] Updated weights for policy 1, policy_version 228665 (0.0011) [2023-12-26 17:01:32,999][105620] Updated weights for policy 1, policy_version 228675 (0.0011) [2023-12-26 17:01:33,393][105692] Updated weights for policy 0, policy_version 228083 (0.0010) [2023-12-26 17:01:33,440][105692] Updated weights for policy 0, policy_version 228093 (0.0010) [2023-12-26 17:01:33,490][105692] Updated weights for policy 0, policy_version 228103 (0.0006) [2023-12-26 17:01:33,696][105620] Updated weights for policy 1, policy_version 228685 (0.0008) [2023-12-26 17:01:33,744][105620] Updated weights for policy 1, policy_version 228695 (0.0008) [2023-12-26 17:01:33,800][105620] Updated weights for policy 1, policy_version 228705 (0.0005) [2023-12-26 17:01:34,216][105692] Updated weights for policy 0, policy_version 228113 (0.0008) [2023-12-26 17:01:34,280][105692] Updated weights for policy 0, policy_version 228123 (0.0010) [2023-12-26 17:01:34,330][105620] Updated weights for policy 1, policy_version 228715 (0.0005) [2023-12-26 17:01:34,343][105692] Updated weights for policy 0, policy_version 228133 (0.0011) [2023-12-26 17:01:34,395][105620] Updated weights for policy 1, policy_version 228725 (0.0010) [2023-12-26 17:01:34,404][105692] Updated weights for policy 0, policy_version 228143 (0.0011) [2023-12-26 17:01:34,455][105620] Updated weights for policy 1, policy_version 228735 (0.0011) [2023-12-26 17:01:35,127][105692] Updated weights for policy 0, policy_version 228153 (0.0010) [2023-12-26 17:01:35,143][105620] Updated weights for policy 1, policy_version 228745 (0.0010) [2023-12-26 17:01:35,191][105692] Updated weights for policy 0, policy_version 228163 (0.0009) [2023-12-26 17:01:35,196][105620] Updated weights for policy 1, policy_version 228755 (0.0006) [2023-12-26 17:01:35,251][105692] Updated weights for policy 0, policy_version 228173 (0.0008) [2023-12-26 17:01:35,251][105620] Updated weights for policy 1, policy_version 228765 (0.0006) [2023-12-26 17:01:35,301][105620] Updated weights for policy 1, policy_version 228775 (0.0005) [2023-12-26 17:01:35,962][105692] Updated weights for policy 0, policy_version 228183 (0.0009) [2023-12-26 17:01:35,998][105620] Updated weights for policy 1, policy_version 228785 (0.0010) [2023-12-26 17:01:36,024][105692] Updated weights for policy 0, policy_version 228193 (0.0006) [2023-12-26 17:01:36,050][105620] Updated weights for policy 1, policy_version 228795 (0.0010) [2023-12-26 17:01:36,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 116998144. Throughput: 0: 9803.7, 1: 9892.8. Samples: 116993872. Policy #0 lag: (min: 28.0, avg: 53.1, max: 60.0) [2023-12-26 17:01:36,063][104569] Avg episode reward: [(0, '9265.381'), (1, '9175.542')] [2023-12-26 17:01:36,073][105692] Updated weights for policy 0, policy_version 228203 (0.0005) [2023-12-26 17:01:36,101][105620] Updated weights for policy 1, policy_version 228805 (0.0010) [2023-12-26 17:01:36,722][105692] Updated weights for policy 0, policy_version 228213 (0.0007) [2023-12-26 17:01:36,789][105692] Updated weights for policy 0, policy_version 228223 (0.0008) [2023-12-26 17:01:36,852][105692] Updated weights for policy 0, policy_version 228233 (0.0008) [2023-12-26 17:01:36,900][105620] Updated weights for policy 1, policy_version 228815 (0.0010) [2023-12-26 17:01:36,951][105620] Updated weights for policy 1, policy_version 228825 (0.0010) [2023-12-26 17:01:37,007][105620] Updated weights for policy 1, policy_version 228835 (0.0010) [2023-12-26 17:01:37,626][105692] Updated weights for policy 0, policy_version 228243 (0.0009) [2023-12-26 17:01:37,675][105692] Updated weights for policy 0, policy_version 228253 (0.0008) [2023-12-26 17:01:37,724][105692] Updated weights for policy 0, policy_version 228263 (0.0008) [2023-12-26 17:01:37,754][105620] Updated weights for policy 1, policy_version 228845 (0.0008) [2023-12-26 17:01:37,817][105620] Updated weights for policy 1, policy_version 228855 (0.0009) [2023-12-26 17:01:37,879][105620] Updated weights for policy 1, policy_version 228865 (0.0010) [2023-12-26 17:01:38,517][105692] Updated weights for policy 0, policy_version 228273 (0.0010) [2023-12-26 17:01:38,540][105620] Updated weights for policy 1, policy_version 228875 (0.0010) [2023-12-26 17:01:38,574][105692] Updated weights for policy 0, policy_version 228283 (0.0006) [2023-12-26 17:01:38,599][105620] Updated weights for policy 1, policy_version 228885 (0.0011) [2023-12-26 17:01:38,629][105692] Updated weights for policy 0, policy_version 228293 (0.0005) [2023-12-26 17:01:38,655][105620] Updated weights for policy 1, policy_version 228895 (0.0010) [2023-12-26 17:01:38,688][105692] Updated weights for policy 0, policy_version 228303 (0.0006) [2023-12-26 17:01:39,344][105620] Updated weights for policy 1, policy_version 228905 (0.0010) [2023-12-26 17:01:39,414][105620] Updated weights for policy 1, policy_version 228915 (0.0008) [2023-12-26 17:01:39,482][105620] Updated weights for policy 1, policy_version 228925 (0.0006) [2023-12-26 17:01:39,485][105692] Updated weights for policy 0, policy_version 228313 (0.0008) [2023-12-26 17:01:39,539][105692] Updated weights for policy 0, policy_version 228323 (0.0008) [2023-12-26 17:01:39,546][105620] Updated weights for policy 1, policy_version 228935 (0.0006) [2023-12-26 17:01:39,599][105692] Updated weights for policy 0, policy_version 228333 (0.0009) [2023-12-26 17:01:40,244][105620] Updated weights for policy 1, policy_version 228945 (0.0006) [2023-12-26 17:01:40,300][105620] Updated weights for policy 1, policy_version 228955 (0.0005) [2023-12-26 17:01:40,311][105692] Updated weights for policy 0, policy_version 228343 (0.0008) [2023-12-26 17:01:40,358][105620] Updated weights for policy 1, policy_version 228965 (0.0006) [2023-12-26 17:01:40,372][105692] Updated weights for policy 0, policy_version 228353 (0.0007) [2023-12-26 17:01:40,425][105692] Updated weights for policy 0, policy_version 228363 (0.0009) [2023-12-26 17:01:40,934][105620] Updated weights for policy 1, policy_version 228975 (0.0005) [2023-12-26 17:01:40,996][105620] Updated weights for policy 1, policy_version 228985 (0.0005) [2023-12-26 17:01:41,059][105620] Updated weights for policy 1, policy_version 228995 (0.0006) [2023-12-26 17:01:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 117096448. Throughput: 0: 9703.9, 1: 9836.6. Samples: 117110692. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-26 17:01:41,063][104569] Avg episode reward: [(0, '9264.652'), (1, '8993.213')] [2023-12-26 17:01:41,270][105692] Updated weights for policy 0, policy_version 228373 (0.0010) [2023-12-26 17:01:41,334][105692] Updated weights for policy 0, policy_version 228383 (0.0010) [2023-12-26 17:01:41,403][105692] Updated weights for policy 0, policy_version 228393 (0.0008) [2023-12-26 17:01:41,763][105620] Updated weights for policy 1, policy_version 229005 (0.0007) [2023-12-26 17:01:41,828][105620] Updated weights for policy 1, policy_version 229015 (0.0009) [2023-12-26 17:01:41,895][105620] Updated weights for policy 1, policy_version 229025 (0.0010) [2023-12-26 17:01:41,976][105692] Updated weights for policy 0, policy_version 228403 (0.0008) [2023-12-26 17:01:42,037][105692] Updated weights for policy 0, policy_version 228413 (0.0006) [2023-12-26 17:01:42,098][105692] Updated weights for policy 0, policy_version 228423 (0.0006) [2023-12-26 17:01:42,729][105620] Updated weights for policy 1, policy_version 229035 (0.0009) [2023-12-26 17:01:42,785][105692] Updated weights for policy 0, policy_version 228433 (0.0009) [2023-12-26 17:01:42,787][105620] Updated weights for policy 1, policy_version 229045 (0.0009) [2023-12-26 17:01:42,846][105620] Updated weights for policy 1, policy_version 229055 (0.0005) [2023-12-26 17:01:42,847][105692] Updated weights for policy 0, policy_version 228443 (0.0010) [2023-12-26 17:01:42,903][105692] Updated weights for policy 0, policy_version 228453 (0.0011) [2023-12-26 17:01:42,955][105692] Updated weights for policy 0, policy_version 228463 (0.0009) [2023-12-26 17:01:43,590][105692] Updated weights for policy 0, policy_version 228473 (0.0011) [2023-12-26 17:01:43,590][105620] Updated weights for policy 1, policy_version 229065 (0.0006) [2023-12-26 17:01:43,648][105692] Updated weights for policy 0, policy_version 228483 (0.0010) [2023-12-26 17:01:43,650][105620] Updated weights for policy 1, policy_version 229075 (0.0011) [2023-12-26 17:01:43,693][105620] Updated weights for policy 1, policy_version 229085 (0.0007) [2023-12-26 17:01:43,706][105692] Updated weights for policy 0, policy_version 228493 (0.0010) [2023-12-26 17:01:43,745][105620] Updated weights for policy 1, policy_version 229095 (0.0005) [2023-12-26 17:01:44,316][105692] Updated weights for policy 0, policy_version 228503 (0.0007) [2023-12-26 17:01:44,336][105620] Updated weights for policy 1, policy_version 229105 (0.0005) [2023-12-26 17:01:44,377][105692] Updated weights for policy 0, policy_version 228513 (0.0005) [2023-12-26 17:01:44,402][105620] Updated weights for policy 1, policy_version 229115 (0.0008) [2023-12-26 17:01:44,433][105692] Updated weights for policy 0, policy_version 228523 (0.0007) [2023-12-26 17:01:44,469][105620] Updated weights for policy 1, policy_version 229125 (0.0010) [2023-12-26 17:01:44,999][105692] Updated weights for policy 0, policy_version 228533 (0.0005) [2023-12-26 17:01:45,063][105692] Updated weights for policy 0, policy_version 228543 (0.0009) [2023-12-26 17:01:45,129][105692] Updated weights for policy 0, policy_version 228553 (0.0011) [2023-12-26 17:01:45,159][105620] Updated weights for policy 1, policy_version 229135 (0.0007) [2023-12-26 17:01:45,206][105620] Updated weights for policy 1, policy_version 229145 (0.0005) [2023-12-26 17:01:45,260][105620] Updated weights for policy 1, policy_version 229155 (0.0006) [2023-12-26 17:01:45,715][105692] Updated weights for policy 0, policy_version 228563 (0.0009) [2023-12-26 17:01:45,779][105692] Updated weights for policy 0, policy_version 228573 (0.0010) [2023-12-26 17:01:45,845][105692] Updated weights for policy 0, policy_version 228583 (0.0006) [2023-12-26 17:01:45,893][105620] Updated weights for policy 1, policy_version 229165 (0.0008) [2023-12-26 17:01:45,952][105620] Updated weights for policy 1, policy_version 229175 (0.0007) [2023-12-26 17:01:46,024][105620] Updated weights for policy 1, policy_version 229185 (0.0006) [2023-12-26 17:01:46,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 117202944. Throughput: 0: 9741.2, 1: 9829.9. Samples: 117169396. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-26 17:01:46,062][104569] Avg episode reward: [(0, '9264.932'), (1, '9083.048')] [2023-12-26 17:01:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000228592_58531840.pth... [2023-12-26 17:01:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000229192_58679296.pth... [2023-12-26 17:01:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000227440_58236928.pth [2023-12-26 17:01:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000228040_58384384.pth [2023-12-26 17:01:46,074][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_000228592_58531840.pth [2023-12-26 17:01:46,074][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_000229192_58679296.pth [2023-12-26 17:01:46,454][105692] Updated weights for policy 0, policy_version 228593 (0.0008) [2023-12-26 17:01:46,516][105692] Updated weights for policy 0, policy_version 228603 (0.0008) [2023-12-26 17:01:46,578][105692] Updated weights for policy 0, policy_version 228613 (0.0008) [2023-12-26 17:01:46,638][105692] Updated weights for policy 0, policy_version 228623 (0.0008) [2023-12-26 17:01:46,738][105620] Updated weights for policy 1, policy_version 229195 (0.0009) [2023-12-26 17:01:46,797][105620] Updated weights for policy 1, policy_version 229205 (0.0007) [2023-12-26 17:01:46,853][105620] Updated weights for policy 1, policy_version 229215 (0.0005) [2023-12-26 17:01:47,431][105620] Updated weights for policy 1, policy_version 229225 (0.0006) [2023-12-26 17:01:47,487][105692] Updated weights for policy 0, policy_version 228633 (0.0006) [2023-12-26 17:01:47,488][105620] Updated weights for policy 1, policy_version 229235 (0.0009) [2023-12-26 17:01:47,541][105692] Updated weights for policy 0, policy_version 228643 (0.0008) [2023-12-26 17:01:47,546][105620] Updated weights for policy 1, policy_version 229245 (0.0007) [2023-12-26 17:01:47,592][105692] Updated weights for policy 0, policy_version 228653 (0.0006) [2023-12-26 17:01:47,604][105620] Updated weights for policy 1, policy_version 229255 (0.0010) [2023-12-26 17:01:48,337][105620] Updated weights for policy 1, policy_version 229265 (0.0010) [2023-12-26 17:01:48,351][105692] Updated weights for policy 0, policy_version 228663 (0.0007) [2023-12-26 17:01:48,397][105620] Updated weights for policy 1, policy_version 229275 (0.0007) [2023-12-26 17:01:48,411][105692] Updated weights for policy 0, policy_version 228673 (0.0007) [2023-12-26 17:01:48,454][105620] Updated weights for policy 1, policy_version 229285 (0.0007) [2023-12-26 17:01:48,470][105692] Updated weights for policy 0, policy_version 228683 (0.0005) [2023-12-26 17:01:49,139][105692] Updated weights for policy 0, policy_version 228693 (0.0007) [2023-12-26 17:01:49,208][105692] Updated weights for policy 0, policy_version 228703 (0.0009) [2023-12-26 17:01:49,267][105620] Updated weights for policy 1, policy_version 229295 (0.0008) [2023-12-26 17:01:49,273][105692] Updated weights for policy 0, policy_version 228713 (0.0010) [2023-12-26 17:01:49,326][105620] Updated weights for policy 1, policy_version 229305 (0.0006) [2023-12-26 17:01:49,396][105620] Updated weights for policy 1, policy_version 229315 (0.0008) [2023-12-26 17:01:50,017][105692] Updated weights for policy 0, policy_version 228723 (0.0011) [2023-12-26 17:01:50,086][105692] Updated weights for policy 0, policy_version 228733 (0.0010) [2023-12-26 17:01:50,151][105692] Updated weights for policy 0, policy_version 228743 (0.0008) [2023-12-26 17:01:50,153][105620] Updated weights for policy 1, policy_version 229325 (0.0009) [2023-12-26 17:01:50,215][105620] Updated weights for policy 1, policy_version 229335 (0.0006) [2023-12-26 17:01:50,276][105620] Updated weights for policy 1, policy_version 229345 (0.0008) [2023-12-26 17:01:50,883][105692] Updated weights for policy 0, policy_version 228753 (0.0010) [2023-12-26 17:01:50,944][105692] Updated weights for policy 0, policy_version 228763 (0.0007) [2023-12-26 17:01:50,997][105692] Updated weights for policy 0, policy_version 228773 (0.0010) [2023-12-26 17:01:51,039][105620] Updated weights for policy 1, policy_version 229355 (0.0008) [2023-12-26 17:01:51,058][105692] Updated weights for policy 0, policy_version 228783 (0.0009) [2023-12-26 17:01:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 117301248. Throughput: 0: 9848.4, 1: 9817.0. Samples: 117290684. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-26 17:01:51,063][104569] Avg episode reward: [(0, '9356.266'), (1, '9084.403')] [2023-12-26 17:01:51,105][105620] Updated weights for policy 1, policy_version 229365 (0.0008) [2023-12-26 17:01:51,172][105620] Updated weights for policy 1, policy_version 229375 (0.0008) [2023-12-26 17:01:51,757][105692] Updated weights for policy 0, policy_version 228793 (0.0010) [2023-12-26 17:01:51,817][105692] Updated weights for policy 0, policy_version 228803 (0.0008) [2023-12-26 17:01:51,881][105692] Updated weights for policy 0, policy_version 228813 (0.0009) [2023-12-26 17:01:52,003][105620] Updated weights for policy 1, policy_version 229385 (0.0010) [2023-12-26 17:01:52,060][105620] Updated weights for policy 1, policy_version 229395 (0.0008) [2023-12-26 17:01:52,125][105620] Updated weights for policy 1, policy_version 229405 (0.0009) [2023-12-26 17:01:52,187][105620] Updated weights for policy 1, policy_version 229415 (0.0008) [2023-12-26 17:01:52,655][105692] Updated weights for policy 0, policy_version 228823 (0.0007) [2023-12-26 17:01:52,714][105692] Updated weights for policy 0, policy_version 228833 (0.0005) [2023-12-26 17:01:52,772][105692] Updated weights for policy 0, policy_version 228843 (0.0005) [2023-12-26 17:01:53,032][105620] Updated weights for policy 1, policy_version 229425 (0.0009) [2023-12-26 17:01:53,079][105620] Updated weights for policy 1, policy_version 229435 (0.0009) [2023-12-26 17:01:53,128][105620] Updated weights for policy 1, policy_version 229445 (0.0009) [2023-12-26 17:01:53,337][105692] Updated weights for policy 0, policy_version 228853 (0.0007) [2023-12-26 17:01:53,384][105692] Updated weights for policy 0, policy_version 228863 (0.0009) [2023-12-26 17:01:53,431][105692] Updated weights for policy 0, policy_version 228873 (0.0008) [2023-12-26 17:01:53,908][105620] Updated weights for policy 1, policy_version 229455 (0.0009) [2023-12-26 17:01:53,961][105620] Updated weights for policy 1, policy_version 229465 (0.0009) [2023-12-26 17:01:54,010][105620] Updated weights for policy 1, policy_version 229475 (0.0008) [2023-12-26 17:01:54,226][105692] Updated weights for policy 0, policy_version 228883 (0.0009) [2023-12-26 17:01:54,283][105692] Updated weights for policy 0, policy_version 228893 (0.0009) [2023-12-26 17:01:54,345][105692] Updated weights for policy 0, policy_version 228903 (0.0009) [2023-12-26 17:01:54,743][105620] Updated weights for policy 1, policy_version 229485 (0.0007) [2023-12-26 17:01:54,795][105620] Updated weights for policy 1, policy_version 229495 (0.0009) [2023-12-26 17:01:54,846][105620] Updated weights for policy 1, policy_version 229505 (0.0009) [2023-12-26 17:01:55,114][105692] Updated weights for policy 0, policy_version 228913 (0.0009) [2023-12-26 17:01:55,169][105692] Updated weights for policy 0, policy_version 228923 (0.0009) [2023-12-26 17:01:55,216][105692] Updated weights for policy 0, policy_version 228933 (0.0008) [2023-12-26 17:01:55,270][105692] Updated weights for policy 0, policy_version 228943 (0.0009) [2023-12-26 17:01:55,612][105620] Updated weights for policy 1, policy_version 229515 (0.0008) [2023-12-26 17:01:55,663][105620] Updated weights for policy 1, policy_version 229525 (0.0005) [2023-12-26 17:01:55,715][105620] Updated weights for policy 1, policy_version 229535 (0.0005) [2023-12-26 17:01:55,940][105692] Updated weights for policy 0, policy_version 228953 (0.0007) [2023-12-26 17:01:56,006][105692] Updated weights for policy 0, policy_version 228963 (0.0005) [2023-12-26 17:01:56,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 117391360. Throughput: 0: 9840.4, 1: 9664.4. Samples: 117403276. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-26 17:01:56,062][104569] Avg episode reward: [(0, '9356.484'), (1, '9263.983')] [2023-12-26 17:01:56,066][105692] Updated weights for policy 0, policy_version 228973 (0.0008) [2023-12-26 17:01:56,318][105620] Updated weights for policy 1, policy_version 229545 (0.0009) [2023-12-26 17:01:56,370][105620] Updated weights for policy 1, policy_version 229555 (0.0009) [2023-12-26 17:01:56,426][105620] Updated weights for policy 1, policy_version 229565 (0.0009) [2023-12-26 17:01:56,479][105620] Updated weights for policy 1, policy_version 229575 (0.0010) [2023-12-26 17:01:56,656][105692] Updated weights for policy 0, policy_version 228984 (0.0006) [2023-12-26 17:01:56,711][105692] Updated weights for policy 0, policy_version 228994 (0.0005) [2023-12-26 17:01:56,763][105692] Updated weights for policy 0, policy_version 229004 (0.0009) [2023-12-26 17:01:57,335][105692] Updated weights for policy 0, policy_version 229014 (0.0007) [2023-12-26 17:01:57,356][105620] Updated weights for policy 1, policy_version 229585 (0.0009) [2023-12-26 17:01:57,390][105692] Updated weights for policy 0, policy_version 229024 (0.0006) [2023-12-26 17:01:57,412][105620] Updated weights for policy 1, policy_version 229595 (0.0007) [2023-12-26 17:01:57,449][105692] Updated weights for policy 0, policy_version 229034 (0.0007) [2023-12-26 17:01:57,475][105620] Updated weights for policy 1, policy_version 229605 (0.0007) [2023-12-26 17:01:58,116][105620] Updated weights for policy 1, policy_version 229615 (0.0008) [2023-12-26 17:01:58,171][105620] Updated weights for policy 1, policy_version 229625 (0.0008) [2023-12-26 17:01:58,234][105692] Updated weights for policy 0, policy_version 229044 (0.0006) [2023-12-26 17:01:58,235][105620] Updated weights for policy 1, policy_version 229635 (0.0008) [2023-12-26 17:01:58,292][105692] Updated weights for policy 0, policy_version 229054 (0.0007) [2023-12-26 17:01:58,356][105692] Updated weights for policy 0, policy_version 229064 (0.0009) [2023-12-26 17:01:58,989][105620] Updated weights for policy 1, policy_version 229645 (0.0007) [2023-12-26 17:01:59,051][105620] Updated weights for policy 1, policy_version 229655 (0.0008) [2023-12-26 17:01:59,117][105620] Updated weights for policy 1, policy_version 229665 (0.0009) [2023-12-26 17:01:59,175][105692] Updated weights for policy 0, policy_version 229074 (0.0007) [2023-12-26 17:01:59,253][105692] Updated weights for policy 0, policy_version 229084 (0.0012) [2023-12-26 17:01:59,316][105692] Updated weights for policy 0, policy_version 229094 (0.0009) [2023-12-26 17:01:59,380][105692] Updated weights for policy 0, policy_version 229104 (0.0010) [2023-12-26 17:01:59,897][105620] Updated weights for policy 1, policy_version 229675 (0.0009) [2023-12-26 17:01:59,964][105620] Updated weights for policy 1, policy_version 229685 (0.0008) [2023-12-26 17:02:00,013][105620] Updated weights for policy 1, policy_version 229695 (0.0008) [2023-12-26 17:02:00,126][105692] Updated weights for policy 0, policy_version 229114 (0.0006) [2023-12-26 17:02:00,180][105692] Updated weights for policy 0, policy_version 229124 (0.0005) [2023-12-26 17:02:00,233][105692] Updated weights for policy 0, policy_version 229134 (0.0005) [2023-12-26 17:02:00,763][105620] Updated weights for policy 1, policy_version 229705 (0.0007) [2023-12-26 17:02:00,827][105620] Updated weights for policy 1, policy_version 229715 (0.0007) [2023-12-26 17:02:00,878][105620] Updated weights for policy 1, policy_version 229725 (0.0008) [2023-12-26 17:02:00,933][105620] Updated weights for policy 1, policy_version 229735 (0.0009) [2023-12-26 17:02:00,935][105692] Updated weights for policy 0, policy_version 229144 (0.0006) [2023-12-26 17:02:00,987][105692] Updated weights for policy 0, policy_version 229154 (0.0005) [2023-12-26 17:02:01,045][105692] Updated weights for policy 0, policy_version 229164 (0.0007) [2023-12-26 17:02:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 117489664. Throughput: 0: 9852.5, 1: 9653.5. Samples: 117462672. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-26 17:02:01,062][104569] Avg episode reward: [(0, '9356.816'), (1, '9263.712')] [2023-12-26 17:02:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000229736_58818560.pth... [2023-12-26 17:02:01,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000229168_58679296.pth... [2023-12-26 17:02:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000228584_58523648.pth [2023-12-26 17:02:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000228016_58384384.pth [2023-12-26 17:02:01,688][105692] Updated weights for policy 0, policy_version 229174 (0.0009) [2023-12-26 17:02:01,743][105620] Updated weights for policy 1, policy_version 229745 (0.0007) [2023-12-26 17:02:01,754][105692] Updated weights for policy 0, policy_version 229184 (0.0010) [2023-12-26 17:02:01,792][105620] Updated weights for policy 1, policy_version 229755 (0.0005) [2023-12-26 17:02:01,813][105692] Updated weights for policy 0, policy_version 229194 (0.0011) [2023-12-26 17:02:01,847][105620] Updated weights for policy 1, policy_version 229765 (0.0006) [2023-12-26 17:02:02,478][105620] Updated weights for policy 1, policy_version 229775 (0.0007) [2023-12-26 17:02:02,536][105620] Updated weights for policy 1, policy_version 229785 (0.0008) [2023-12-26 17:02:02,554][105692] Updated weights for policy 0, policy_version 229204 (0.0011) [2023-12-26 17:02:02,596][105620] Updated weights for policy 1, policy_version 229795 (0.0006) [2023-12-26 17:02:02,614][105692] Updated weights for policy 0, policy_version 229214 (0.0011) [2023-12-26 17:02:02,665][105692] Updated weights for policy 0, policy_version 229224 (0.0010) [2023-12-26 17:02:03,221][105620] Updated weights for policy 1, policy_version 229805 (0.0008) [2023-12-26 17:02:03,274][105620] Updated weights for policy 1, policy_version 229815 (0.0010) [2023-12-26 17:02:03,307][105692] Updated weights for policy 0, policy_version 229234 (0.0009) [2023-12-26 17:02:03,336][105620] Updated weights for policy 1, policy_version 229825 (0.0009) [2023-12-26 17:02:03,366][105692] Updated weights for policy 0, policy_version 229244 (0.0005) [2023-12-26 17:02:03,429][105692] Updated weights for policy 0, policy_version 229254 (0.0008) [2023-12-26 17:02:03,473][105692] Updated weights for policy 0, policy_version 229264 (0.0010) [2023-12-26 17:02:03,931][105620] Updated weights for policy 1, policy_version 229835 (0.0007) [2023-12-26 17:02:03,995][105620] Updated weights for policy 1, policy_version 229845 (0.0006) [2023-12-26 17:02:04,058][105620] Updated weights for policy 1, policy_version 229855 (0.0006) [2023-12-26 17:02:04,188][105692] Updated weights for policy 0, policy_version 229274 (0.0010) [2023-12-26 17:02:04,247][105692] Updated weights for policy 0, policy_version 229284 (0.0010) [2023-12-26 17:02:04,299][105692] Updated weights for policy 0, policy_version 229294 (0.0010) [2023-12-26 17:02:04,648][105620] Updated weights for policy 1, policy_version 229865 (0.0006) [2023-12-26 17:02:04,712][105620] Updated weights for policy 1, policy_version 229875 (0.0005) [2023-12-26 17:02:04,777][105620] Updated weights for policy 1, policy_version 229885 (0.0006) [2023-12-26 17:02:04,847][105620] Updated weights for policy 1, policy_version 229895 (0.0010) [2023-12-26 17:02:04,936][105692] Updated weights for policy 0, policy_version 229304 (0.0010) [2023-12-26 17:02:04,993][105692] Updated weights for policy 0, policy_version 229314 (0.0009) [2023-12-26 17:02:05,048][105692] Updated weights for policy 0, policy_version 229325 (0.0010) [2023-12-26 17:02:05,461][105620] Updated weights for policy 1, policy_version 229905 (0.0009) [2023-12-26 17:02:05,523][105620] Updated weights for policy 1, policy_version 229915 (0.0008) [2023-12-26 17:02:05,581][105620] Updated weights for policy 1, policy_version 229925 (0.0009) [2023-12-26 17:02:05,821][105692] Updated weights for policy 0, policy_version 229335 (0.0008) [2023-12-26 17:02:05,883][105692] Updated weights for policy 0, policy_version 229345 (0.0009) [2023-12-26 17:02:05,947][105692] Updated weights for policy 0, policy_version 229355 (0.0010) [2023-12-26 17:02:06,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 117596160. Throughput: 0: 9930.5, 1: 9681.9. Samples: 117581440. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-26 17:02:06,063][104569] Avg episode reward: [(0, '9176.862'), (1, '9263.747')] [2023-12-26 17:02:06,297][105620] Updated weights for policy 1, policy_version 229935 (0.0007) [2023-12-26 17:02:06,363][105620] Updated weights for policy 1, policy_version 229945 (0.0007) [2023-12-26 17:02:06,423][105620] Updated weights for policy 1, policy_version 229955 (0.0009) [2023-12-26 17:02:06,699][105692] Updated weights for policy 0, policy_version 229365 (0.0010) [2023-12-26 17:02:06,758][105692] Updated weights for policy 0, policy_version 229375 (0.0010) [2023-12-26 17:02:06,813][105692] Updated weights for policy 0, policy_version 229385 (0.0009) [2023-12-26 17:02:07,107][105620] Updated weights for policy 1, policy_version 229965 (0.0009) [2023-12-26 17:02:07,167][105620] Updated weights for policy 1, policy_version 229975 (0.0009) [2023-12-26 17:02:07,218][105620] Updated weights for policy 1, policy_version 229986 (0.0010) [2023-12-26 17:02:07,525][105692] Updated weights for policy 0, policy_version 229395 (0.0008) [2023-12-26 17:02:07,580][105692] Updated weights for policy 0, policy_version 229405 (0.0006) [2023-12-26 17:02:07,634][105692] Updated weights for policy 0, policy_version 229415 (0.0005) [2023-12-26 17:02:07,974][105620] Updated weights for policy 1, policy_version 229996 (0.0009) [2023-12-26 17:02:08,034][105620] Updated weights for policy 1, policy_version 230006 (0.0008) [2023-12-26 17:02:08,082][105620] Updated weights for policy 1, policy_version 230016 (0.0008) [2023-12-26 17:02:08,340][105692] Updated weights for policy 0, policy_version 229425 (0.0009) [2023-12-26 17:02:08,403][105692] Updated weights for policy 0, policy_version 229435 (0.0010) [2023-12-26 17:02:08,454][105692] Updated weights for policy 0, policy_version 229445 (0.0010) [2023-12-26 17:02:08,502][105692] Updated weights for policy 0, policy_version 229455 (0.0010) [2023-12-26 17:02:08,874][105620] Updated weights for policy 1, policy_version 230026 (0.0008) [2023-12-26 17:02:08,933][105620] Updated weights for policy 1, policy_version 230036 (0.0008) [2023-12-26 17:02:08,986][105620] Updated weights for policy 1, policy_version 230046 (0.0008) [2023-12-26 17:02:09,035][105620] Updated weights for policy 1, policy_version 230056 (0.0008) [2023-12-26 17:02:09,266][105692] Updated weights for policy 0, policy_version 229465 (0.0008) [2023-12-26 17:02:09,332][105692] Updated weights for policy 0, policy_version 229475 (0.0009) [2023-12-26 17:02:09,396][105692] Updated weights for policy 0, policy_version 229485 (0.0009) [2023-12-26 17:02:09,817][105620] Updated weights for policy 1, policy_version 230066 (0.0007) [2023-12-26 17:02:09,891][105620] Updated weights for policy 1, policy_version 230076 (0.0007) [2023-12-26 17:02:09,959][105620] Updated weights for policy 1, policy_version 230086 (0.0007) [2023-12-26 17:02:10,209][105692] Updated weights for policy 0, policy_version 229495 (0.0008) [2023-12-26 17:02:10,259][105692] Updated weights for policy 0, policy_version 229505 (0.0009) [2023-12-26 17:02:10,314][105692] Updated weights for policy 0, policy_version 229515 (0.0009) [2023-12-26 17:02:10,613][105620] Updated weights for policy 1, policy_version 230096 (0.0010) [2023-12-26 17:02:10,685][105620] Updated weights for policy 1, policy_version 230106 (0.0010) [2023-12-26 17:02:10,740][105620] Updated weights for policy 1, policy_version 230116 (0.0010) [2023-12-26 17:02:10,988][105692] Updated weights for policy 0, policy_version 229525 (0.0005) [2023-12-26 17:02:11,043][105692] Updated weights for policy 0, policy_version 229535 (0.0007) [2023-12-26 17:02:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 117686272. Throughput: 0: 9916.3, 1: 9700.1. Samples: 117695992. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-26 17:02:11,062][104569] Avg episode reward: [(0, '9177.102'), (1, '9264.267')] [2023-12-26 17:02:11,109][105692] Updated weights for policy 0, policy_version 229545 (0.0006) [2023-12-26 17:02:11,477][105620] Updated weights for policy 1, policy_version 230126 (0.0010) [2023-12-26 17:02:11,539][105620] Updated weights for policy 1, policy_version 230136 (0.0010) [2023-12-26 17:02:11,609][105620] Updated weights for policy 1, policy_version 230146 (0.0008) [2023-12-26 17:02:11,805][105692] Updated weights for policy 0, policy_version 229555 (0.0009) [2023-12-26 17:02:11,863][105692] Updated weights for policy 0, policy_version 229565 (0.0009) [2023-12-26 17:02:11,921][105692] Updated weights for policy 0, policy_version 229575 (0.0009) [2023-12-26 17:02:12,283][105620] Updated weights for policy 1, policy_version 230156 (0.0008) [2023-12-26 17:02:12,349][105620] Updated weights for policy 1, policy_version 230166 (0.0007) [2023-12-26 17:02:12,413][105620] Updated weights for policy 1, policy_version 230176 (0.0010) [2023-12-26 17:02:12,756][105692] Updated weights for policy 0, policy_version 229585 (0.0008) [2023-12-26 17:02:12,812][105692] Updated weights for policy 0, policy_version 229595 (0.0005) [2023-12-26 17:02:12,866][105692] Updated weights for policy 0, policy_version 229605 (0.0005) [2023-12-26 17:02:12,913][105692] Updated weights for policy 0, policy_version 229615 (0.0008) [2023-12-26 17:02:13,207][105620] Updated weights for policy 1, policy_version 230186 (0.0010) [2023-12-26 17:02:13,269][105620] Updated weights for policy 1, policy_version 230196 (0.0009) [2023-12-26 17:02:13,318][105620] Updated weights for policy 1, policy_version 230206 (0.0008) [2023-12-26 17:02:13,365][105620] Updated weights for policy 1, policy_version 230216 (0.0008) [2023-12-26 17:02:13,629][105692] Updated weights for policy 0, policy_version 229625 (0.0009) [2023-12-26 17:02:13,679][105692] Updated weights for policy 0, policy_version 229635 (0.0009) [2023-12-26 17:02:13,737][105692] Updated weights for policy 0, policy_version 229645 (0.0008) [2023-12-26 17:02:14,082][105620] Updated weights for policy 1, policy_version 230226 (0.0009) [2023-12-26 17:02:14,137][105620] Updated weights for policy 1, policy_version 230236 (0.0009) [2023-12-26 17:02:14,184][105620] Updated weights for policy 1, policy_version 230246 (0.0009) [2023-12-26 17:02:14,504][105692] Updated weights for policy 0, policy_version 229655 (0.0009) [2023-12-26 17:02:14,558][105692] Updated weights for policy 0, policy_version 229665 (0.0010) [2023-12-26 17:02:14,623][105692] Updated weights for policy 0, policy_version 229675 (0.0011) [2023-12-26 17:02:14,926][105620] Updated weights for policy 1, policy_version 230256 (0.0006) [2023-12-26 17:02:14,997][105620] Updated weights for policy 1, policy_version 230266 (0.0007) [2023-12-26 17:02:15,059][105620] Updated weights for policy 1, policy_version 230276 (0.0008) [2023-12-26 17:02:15,339][105692] Updated weights for policy 0, policy_version 229685 (0.0008) [2023-12-26 17:02:15,388][105692] Updated weights for policy 0, policy_version 229695 (0.0006) [2023-12-26 17:02:15,443][105692] Updated weights for policy 0, policy_version 229705 (0.0006) [2023-12-26 17:02:15,864][105620] Updated weights for policy 1, policy_version 230286 (0.0009) [2023-12-26 17:02:15,926][105620] Updated weights for policy 1, policy_version 230296 (0.0008) [2023-12-26 17:02:15,987][105620] Updated weights for policy 1, policy_version 230306 (0.0008) [2023-12-26 17:02:16,007][105692] Updated weights for policy 0, policy_version 229715 (0.0007) [2023-12-26 17:02:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 117784576. Throughput: 0: 9837.2, 1: 9742.5. Samples: 117753848. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-26 17:02:16,062][105692] Updated weights for policy 0, policy_version 229725 (0.0011) [2023-12-26 17:02:16,063][104569] Avg episode reward: [(0, '8599.909'), (1, '9265.210')] [2023-12-26 17:02:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000230312_58966016.pth... [2023-12-26 17:02:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000229192_58679296.pth [2023-12-26 17:02:16,126][105692] Updated weights for policy 0, policy_version 229735 (0.0011) [2023-12-26 17:02:16,180][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000229744_58826752.pth... [2023-12-26 17:02:16,183][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000228592_58531840.pth [2023-12-26 17:02:16,799][105620] Updated weights for policy 1, policy_version 230316 (0.0010) [2023-12-26 17:02:16,832][105692] Updated weights for policy 0, policy_version 229745 (0.0010) [2023-12-26 17:02:16,856][105620] Updated weights for policy 1, policy_version 230326 (0.0008) [2023-12-26 17:02:16,896][105692] Updated weights for policy 0, policy_version 229755 (0.0006) [2023-12-26 17:02:16,909][105620] Updated weights for policy 1, policy_version 230336 (0.0009) [2023-12-26 17:02:16,960][105692] Updated weights for policy 0, policy_version 229765 (0.0007) [2023-12-26 17:02:17,019][105692] Updated weights for policy 0, policy_version 229775 (0.0009) [2023-12-26 17:02:17,541][105620] Updated weights for policy 1, policy_version 230346 (0.0009) [2023-12-26 17:02:17,599][105620] Updated weights for policy 1, policy_version 230356 (0.0009) [2023-12-26 17:02:17,651][105620] Updated weights for policy 1, policy_version 230366 (0.0009) [2023-12-26 17:02:17,703][105620] Updated weights for policy 1, policy_version 230376 (0.0009) [2023-12-26 17:02:17,785][105692] Updated weights for policy 0, policy_version 229785 (0.0009) [2023-12-26 17:02:17,831][105692] Updated weights for policy 0, policy_version 229795 (0.0008) [2023-12-26 17:02:17,881][105692] Updated weights for policy 0, policy_version 229805 (0.0008) [2023-12-26 17:02:18,452][105620] Updated weights for policy 1, policy_version 230386 (0.0008) [2023-12-26 17:02:18,515][105620] Updated weights for policy 1, policy_version 230396 (0.0008) [2023-12-26 17:02:18,560][105620] Updated weights for policy 1, policy_version 230406 (0.0008) [2023-12-26 17:02:18,588][105692] Updated weights for policy 0, policy_version 229815 (0.0010) [2023-12-26 17:02:18,651][105692] Updated weights for policy 0, policy_version 229825 (0.0011) [2023-12-26 17:02:18,724][105692] Updated weights for policy 0, policy_version 229835 (0.0011) [2023-12-26 17:02:19,239][105620] Updated weights for policy 1, policy_version 230416 (0.0008) [2023-12-26 17:02:19,298][105620] Updated weights for policy 1, policy_version 230426 (0.0008) [2023-12-26 17:02:19,368][105620] Updated weights for policy 1, policy_version 230436 (0.0008) [2023-12-26 17:02:19,435][105692] Updated weights for policy 0, policy_version 229845 (0.0010) [2023-12-26 17:02:19,505][105692] Updated weights for policy 0, policy_version 229855 (0.0009) [2023-12-26 17:02:19,570][105692] Updated weights for policy 0, policy_version 229865 (0.0009) [2023-12-26 17:02:20,005][105620] Updated weights for policy 1, policy_version 230446 (0.0008) [2023-12-26 17:02:20,060][105620] Updated weights for policy 1, policy_version 230456 (0.0008) [2023-12-26 17:02:20,115][105620] Updated weights for policy 1, policy_version 230466 (0.0009) [2023-12-26 17:02:20,336][105692] Updated weights for policy 0, policy_version 229875 (0.0010) [2023-12-26 17:02:20,404][105692] Updated weights for policy 0, policy_version 229885 (0.0007) [2023-12-26 17:02:20,464][105692] Updated weights for policy 0, policy_version 229895 (0.0009) [2023-12-26 17:02:20,970][105620] Updated weights for policy 1, policy_version 230476 (0.0008) [2023-12-26 17:02:21,036][105620] Updated weights for policy 1, policy_version 230486 (0.0009) [2023-12-26 17:02:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 117874688. Throughput: 0: 9794.3, 1: 9684.7. Samples: 117870428. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-26 17:02:21,062][104569] Avg episode reward: [(0, '8372.269'), (1, '9266.179')] [2023-12-26 17:02:21,102][105620] Updated weights for policy 1, policy_version 230496 (0.0008) [2023-12-26 17:02:21,164][105692] Updated weights for policy 0, policy_version 229905 (0.0008) [2023-12-26 17:02:21,230][105692] Updated weights for policy 0, policy_version 229915 (0.0009) [2023-12-26 17:02:21,291][105692] Updated weights for policy 0, policy_version 229925 (0.0009) [2023-12-26 17:02:21,357][105692] Updated weights for policy 0, policy_version 229935 (0.0009) [2023-12-26 17:02:21,929][105620] Updated weights for policy 1, policy_version 230506 (0.0008) [2023-12-26 17:02:21,994][105620] Updated weights for policy 1, policy_version 230516 (0.0009) [2023-12-26 17:02:22,062][105620] Updated weights for policy 1, policy_version 230526 (0.0008) [2023-12-26 17:02:22,074][105692] Updated weights for policy 0, policy_version 229945 (0.0008) [2023-12-26 17:02:22,122][105620] Updated weights for policy 1, policy_version 230536 (0.0008) [2023-12-26 17:02:22,142][105692] Updated weights for policy 0, policy_version 229955 (0.0007) [2023-12-26 17:02:22,208][105692] Updated weights for policy 0, policy_version 229965 (0.0006) [2023-12-26 17:02:22,803][105620] Updated weights for policy 1, policy_version 230546 (0.0007) [2023-12-26 17:02:22,867][105620] Updated weights for policy 1, policy_version 230556 (0.0006) [2023-12-26 17:02:22,930][105620] Updated weights for policy 1, policy_version 230566 (0.0006) [2023-12-26 17:02:23,029][105692] Updated weights for policy 0, policy_version 229975 (0.0008) [2023-12-26 17:02:23,084][105692] Updated weights for policy 0, policy_version 229985 (0.0006) [2023-12-26 17:02:23,141][105692] Updated weights for policy 0, policy_version 229995 (0.0006) [2023-12-26 17:02:23,553][105620] Updated weights for policy 1, policy_version 230576 (0.0005) [2023-12-26 17:02:23,607][105620] Updated weights for policy 1, policy_version 230586 (0.0006) [2023-12-26 17:02:23,654][105620] Updated weights for policy 1, policy_version 230596 (0.0006) [2023-12-26 17:02:23,762][105692] Updated weights for policy 0, policy_version 230005 (0.0006) [2023-12-26 17:02:23,839][105692] Updated weights for policy 0, policy_version 230015 (0.0006) [2023-12-26 17:02:23,894][105692] Updated weights for policy 0, policy_version 230025 (0.0009) [2023-12-26 17:02:24,249][105620] Updated weights for policy 1, policy_version 230606 (0.0008) [2023-12-26 17:02:24,311][105620] Updated weights for policy 1, policy_version 230616 (0.0008) [2023-12-26 17:02:24,372][105620] Updated weights for policy 1, policy_version 230626 (0.0009) [2023-12-26 17:02:24,593][105692] Updated weights for policy 0, policy_version 230035 (0.0009) [2023-12-26 17:02:24,652][105692] Updated weights for policy 0, policy_version 230045 (0.0008) [2023-12-26 17:02:24,709][105692] Updated weights for policy 0, policy_version 230055 (0.0009) [2023-12-26 17:02:25,030][105620] Updated weights for policy 1, policy_version 230636 (0.0008) [2023-12-26 17:02:25,086][105620] Updated weights for policy 1, policy_version 230646 (0.0007) [2023-12-26 17:02:25,140][105620] Updated weights for policy 1, policy_version 230656 (0.0009) [2023-12-26 17:02:25,331][105692] Updated weights for policy 0, policy_version 230065 (0.0009) [2023-12-26 17:02:25,378][105692] Updated weights for policy 0, policy_version 230075 (0.0009) [2023-12-26 17:02:25,429][105692] Updated weights for policy 0, policy_version 230085 (0.0008) [2023-12-26 17:02:25,487][105692] Updated weights for policy 0, policy_version 230095 (0.0009) [2023-12-26 17:02:25,896][105620] Updated weights for policy 1, policy_version 230666 (0.0011) [2023-12-26 17:02:25,951][105620] Updated weights for policy 1, policy_version 230676 (0.0008) [2023-12-26 17:02:26,005][105620] Updated weights for policy 1, policy_version 230686 (0.0009) [2023-12-26 17:02:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 117981184. Throughput: 0: 9837.1, 1: 9641.5. Samples: 117987228. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-26 17:02:26,062][104569] Avg episode reward: [(0, '984.340'), (1, '9266.520')] [2023-12-26 17:02:26,281][105692] Updated weights for policy 0, policy_version 230105 (0.0009) [2023-12-26 17:02:26,335][105692] Updated weights for policy 0, policy_version 230115 (0.0009) [2023-12-26 17:02:26,387][105692] Updated weights for policy 0, policy_version 230125 (0.0009) [2023-12-26 17:02:26,707][105620] Updated weights for policy 1, policy_version 230697 (0.0008) [2023-12-26 17:02:26,764][105620] Updated weights for policy 1, policy_version 230707 (0.0009) [2023-12-26 17:02:26,814][105620] Updated weights for policy 1, policy_version 230717 (0.0009) [2023-12-26 17:02:26,861][105620] Updated weights for policy 1, policy_version 230727 (0.0009) [2023-12-26 17:02:27,154][105692] Updated weights for policy 0, policy_version 230135 (0.0009) [2023-12-26 17:02:27,214][105692] Updated weights for policy 0, policy_version 230145 (0.0009) [2023-12-26 17:02:27,261][105692] Updated weights for policy 0, policy_version 230155 (0.0009) [2023-12-26 17:02:27,586][105620] Updated weights for policy 1, policy_version 230737 (0.0008) [2023-12-26 17:02:27,646][105620] Updated weights for policy 1, policy_version 230747 (0.0009) [2023-12-26 17:02:27,706][105620] Updated weights for policy 1, policy_version 230757 (0.0008) [2023-12-26 17:02:28,070][105692] Updated weights for policy 0, policy_version 230165 (0.0010) [2023-12-26 17:02:28,117][105692] Updated weights for policy 0, policy_version 230175 (0.0010) [2023-12-26 17:02:28,163][105692] Updated weights for policy 0, policy_version 230185 (0.0007) [2023-12-26 17:02:28,315][105620] Updated weights for policy 1, policy_version 230767 (0.0008) [2023-12-26 17:02:28,375][105620] Updated weights for policy 1, policy_version 230777 (0.0008) [2023-12-26 17:02:28,424][105620] Updated weights for policy 1, policy_version 230787 (0.0006) [2023-12-26 17:02:28,925][105692] Updated weights for policy 0, policy_version 230195 (0.0007) [2023-12-26 17:02:28,976][105692] Updated weights for policy 0, policy_version 230205 (0.0010) [2023-12-26 17:02:29,014][105620] Updated weights for policy 1, policy_version 230797 (0.0005) [2023-12-26 17:02:29,018][105692] Updated weights for policy 0, policy_version 230215 (0.0009) [2023-12-26 17:02:29,068][105620] Updated weights for policy 1, policy_version 230807 (0.0006) [2023-12-26 17:02:29,116][105620] Updated weights for policy 1, policy_version 230817 (0.0008) [2023-12-26 17:02:29,636][105692] Updated weights for policy 0, policy_version 230225 (0.0010) [2023-12-26 17:02:29,698][105692] Updated weights for policy 0, policy_version 230235 (0.0006) [2023-12-26 17:02:29,759][105692] Updated weights for policy 0, policy_version 230245 (0.0008) [2023-12-26 17:02:29,809][105692] Updated weights for policy 0, policy_version 230255 (0.0007) [2023-12-26 17:02:30,015][105620] Updated weights for policy 1, policy_version 230827 (0.0008) [2023-12-26 17:02:30,083][105620] Updated weights for policy 1, policy_version 230837 (0.0009) [2023-12-26 17:02:30,145][105620] Updated weights for policy 1, policy_version 230847 (0.0009) [2023-12-26 17:02:30,431][105692] Updated weights for policy 0, policy_version 230265 (0.0010) [2023-12-26 17:02:30,489][105692] Updated weights for policy 0, policy_version 230275 (0.0009) [2023-12-26 17:02:30,543][105692] Updated weights for policy 0, policy_version 230285 (0.0008) [2023-12-26 17:02:30,923][105620] Updated weights for policy 1, policy_version 230857 (0.0009) [2023-12-26 17:02:30,973][105620] Updated weights for policy 1, policy_version 230867 (0.0005) [2023-12-26 17:02:31,030][105620] Updated weights for policy 1, policy_version 230877 (0.0007) [2023-12-26 17:02:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 118071296. Throughput: 0: 9773.8, 1: 9719.2. Samples: 118046580. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-26 17:02:31,063][104569] Avg episode reward: [(0, '1519.187'), (1, '9271.586')] [2023-12-26 17:02:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000230288_58966016.pth... [2023-12-26 17:02:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000229168_58679296.pth [2023-12-26 17:02:31,093][105620] Updated weights for policy 1, policy_version 230887 (0.0009) [2023-12-26 17:02:31,097][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000230888_59113472.pth... [2023-12-26 17:02:31,101][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000229736_58818560.pth [2023-12-26 17:02:31,308][105692] Updated weights for policy 0, policy_version 230295 (0.0009) [2023-12-26 17:02:31,366][105692] Updated weights for policy 0, policy_version 230305 (0.0008) [2023-12-26 17:02:31,423][105692] Updated weights for policy 0, policy_version 230315 (0.0009) [2023-12-26 17:02:31,814][105620] Updated weights for policy 1, policy_version 230897 (0.0006) [2023-12-26 17:02:31,882][105620] Updated weights for policy 1, policy_version 230907 (0.0007) [2023-12-26 17:02:31,944][105620] Updated weights for policy 1, policy_version 230917 (0.0007) [2023-12-26 17:02:32,271][105692] Updated weights for policy 0, policy_version 230325 (0.0009) [2023-12-26 17:02:32,330][105692] Updated weights for policy 0, policy_version 230335 (0.0009) [2023-12-26 17:02:32,393][105692] Updated weights for policy 0, policy_version 230345 (0.0008) [2023-12-26 17:02:32,578][105620] Updated weights for policy 1, policy_version 230927 (0.0006) [2023-12-26 17:02:32,629][105620] Updated weights for policy 1, policy_version 230937 (0.0009) [2023-12-26 17:02:32,682][105620] Updated weights for policy 1, policy_version 230947 (0.0009) [2023-12-26 17:02:33,157][105692] Updated weights for policy 0, policy_version 230355 (0.0008) [2023-12-26 17:02:33,209][105692] Updated weights for policy 0, policy_version 230365 (0.0005) [2023-12-26 17:02:33,254][105692] Updated weights for policy 0, policy_version 230375 (0.0005) [2023-12-26 17:02:33,449][105620] Updated weights for policy 1, policy_version 230957 (0.0009) [2023-12-26 17:02:33,503][105620] Updated weights for policy 1, policy_version 230967 (0.0009) [2023-12-26 17:02:33,567][105620] Updated weights for policy 1, policy_version 230977 (0.0009) [2023-12-26 17:02:33,804][105692] Updated weights for policy 0, policy_version 230385 (0.0006) [2023-12-26 17:02:33,872][105692] Updated weights for policy 0, policy_version 230395 (0.0010) [2023-12-26 17:02:33,930][105692] Updated weights for policy 0, policy_version 230405 (0.0010) [2023-12-26 17:02:33,977][105692] Updated weights for policy 0, policy_version 230415 (0.0010) [2023-12-26 17:02:34,277][105620] Updated weights for policy 1, policy_version 230987 (0.0007) [2023-12-26 17:02:34,332][105620] Updated weights for policy 1, policy_version 230997 (0.0005) [2023-12-26 17:02:34,383][105620] Updated weights for policy 1, policy_version 231007 (0.0008) [2023-12-26 17:02:34,719][105692] Updated weights for policy 0, policy_version 230425 (0.0010) [2023-12-26 17:02:34,771][105692] Updated weights for policy 0, policy_version 230435 (0.0010) [2023-12-26 17:02:34,830][105692] Updated weights for policy 0, policy_version 230445 (0.0010) [2023-12-26 17:02:35,117][105620] Updated weights for policy 1, policy_version 231017 (0.0008) [2023-12-26 17:02:35,164][105620] Updated weights for policy 1, policy_version 231027 (0.0008) [2023-12-26 17:02:35,209][105620] Updated weights for policy 1, policy_version 231037 (0.0008) [2023-12-26 17:02:35,265][105620] Updated weights for policy 1, policy_version 231047 (0.0009) [2023-12-26 17:02:35,537][105692] Updated weights for policy 0, policy_version 230455 (0.0010) [2023-12-26 17:02:35,582][105692] Updated weights for policy 0, policy_version 230465 (0.0010) [2023-12-26 17:02:35,627][105692] Updated weights for policy 0, policy_version 230475 (0.0010) [2023-12-26 17:02:35,931][105620] Updated weights for policy 1, policy_version 231057 (0.0009) [2023-12-26 17:02:35,979][105620] Updated weights for policy 1, policy_version 231067 (0.0008) [2023-12-26 17:02:36,026][105620] Updated weights for policy 1, policy_version 231078 (0.0009) [2023-12-26 17:02:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 118177792. Throughput: 0: 9731.0, 1: 9641.9. Samples: 118162464. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-26 17:02:36,062][104569] Avg episode reward: [(0, '576.246'), (1, '9181.715')] [2023-12-26 17:02:36,363][105692] Updated weights for policy 0, policy_version 230485 (0.0011) [2023-12-26 17:02:36,428][105692] Updated weights for policy 0, policy_version 230495 (0.0010) [2023-12-26 17:02:36,482][105692] Updated weights for policy 0, policy_version 230505 (0.0006) [2023-12-26 17:02:36,816][105620] Updated weights for policy 1, policy_version 231088 (0.0008) [2023-12-26 17:02:36,877][105620] Updated weights for policy 1, policy_version 231098 (0.0009) [2023-12-26 17:02:36,938][105620] Updated weights for policy 1, policy_version 231108 (0.0009) [2023-12-26 17:02:37,167][105692] Updated weights for policy 0, policy_version 230515 (0.0007) [2023-12-26 17:02:37,221][105692] Updated weights for policy 0, policy_version 230525 (0.0005) [2023-12-26 17:02:37,274][105692] Updated weights for policy 0, policy_version 230535 (0.0005) [2023-12-26 17:02:37,719][105620] Updated weights for policy 1, policy_version 231118 (0.0007) [2023-12-26 17:02:37,780][105620] Updated weights for policy 1, policy_version 231128 (0.0008) [2023-12-26 17:02:37,832][105692] Updated weights for policy 0, policy_version 230545 (0.0006) [2023-12-26 17:02:37,833][105620] Updated weights for policy 1, policy_version 231138 (0.0008) [2023-12-26 17:02:37,888][105692] Updated weights for policy 0, policy_version 230555 (0.0011) [2023-12-26 17:02:37,951][105692] Updated weights for policy 0, policy_version 230565 (0.0011) [2023-12-26 17:02:38,011][105692] Updated weights for policy 0, policy_version 230575 (0.0011) [2023-12-26 17:02:38,511][105620] Updated weights for policy 1, policy_version 231148 (0.0007) [2023-12-26 17:02:38,566][105620] Updated weights for policy 1, policy_version 231158 (0.0005) [2023-12-26 17:02:38,617][105620] Updated weights for policy 1, policy_version 231168 (0.0005) [2023-12-26 17:02:38,689][105692] Updated weights for policy 0, policy_version 230585 (0.0006) [2023-12-26 17:02:38,751][105692] Updated weights for policy 0, policy_version 230595 (0.0005) [2023-12-26 17:02:38,825][105692] Updated weights for policy 0, policy_version 230605 (0.0005) [2023-12-26 17:02:39,201][105620] Updated weights for policy 1, policy_version 231178 (0.0006) [2023-12-26 17:02:39,261][105620] Updated weights for policy 1, policy_version 231188 (0.0007) [2023-12-26 17:02:39,321][105620] Updated weights for policy 1, policy_version 231198 (0.0008) [2023-12-26 17:02:39,385][105620] Updated weights for policy 1, policy_version 231208 (0.0009) [2023-12-26 17:02:39,439][105692] Updated weights for policy 0, policy_version 230615 (0.0009) [2023-12-26 17:02:39,499][105692] Updated weights for policy 0, policy_version 230625 (0.0007) [2023-12-26 17:02:39,552][105692] Updated weights for policy 0, policy_version 230635 (0.0006) [2023-12-26 17:02:40,171][105620] Updated weights for policy 1, policy_version 231218 (0.0011) [2023-12-26 17:02:40,242][105620] Updated weights for policy 1, policy_version 231228 (0.0011) [2023-12-26 17:02:40,278][105692] Updated weights for policy 0, policy_version 230645 (0.0008) [2023-12-26 17:02:40,302][105620] Updated weights for policy 1, policy_version 231238 (0.0011) [2023-12-26 17:02:40,341][105692] Updated weights for policy 0, policy_version 230655 (0.0011) [2023-12-26 17:02:40,411][105692] Updated weights for policy 0, policy_version 230665 (0.0010) [2023-12-26 17:02:41,004][105620] Updated weights for policy 1, policy_version 231248 (0.0011) [2023-12-26 17:02:41,053][105692] Updated weights for policy 0, policy_version 230675 (0.0008) [2023-12-26 17:02:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 118267904. Throughput: 0: 9797.9, 1: 9741.3. Samples: 118282540. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-26 17:02:41,062][104569] Avg episode reward: [(0, '577.801'), (1, '9265.360')] [2023-12-26 17:02:41,072][105620] Updated weights for policy 1, policy_version 231258 (0.0011) [2023-12-26 17:02:41,113][105692] Updated weights for policy 0, policy_version 230685 (0.0011) [2023-12-26 17:02:41,132][105620] Updated weights for policy 1, policy_version 231268 (0.0011) [2023-12-26 17:02:41,178][105692] Updated weights for policy 0, policy_version 230695 (0.0009) [2023-12-26 17:02:41,944][105692] Updated weights for policy 0, policy_version 230705 (0.0009) [2023-12-26 17:02:41,950][105620] Updated weights for policy 1, policy_version 231278 (0.0007) [2023-12-26 17:02:42,005][105692] Updated weights for policy 0, policy_version 230715 (0.0009) [2023-12-26 17:02:42,016][105620] Updated weights for policy 1, policy_version 231288 (0.0005) [2023-12-26 17:02:42,073][105692] Updated weights for policy 0, policy_version 230725 (0.0008) [2023-12-26 17:02:42,081][105620] Updated weights for policy 1, policy_version 231298 (0.0006) [2023-12-26 17:02:42,134][105692] Updated weights for policy 0, policy_version 230735 (0.0009) [2023-12-26 17:02:42,739][105620] Updated weights for policy 1, policy_version 231308 (0.0008) [2023-12-26 17:02:42,799][105620] Updated weights for policy 1, policy_version 231318 (0.0011) [2023-12-26 17:02:42,802][105692] Updated weights for policy 0, policy_version 230745 (0.0006) [2023-12-26 17:02:42,858][105692] Updated weights for policy 0, policy_version 230755 (0.0005) [2023-12-26 17:02:42,860][105620] Updated weights for policy 1, policy_version 231328 (0.0011) [2023-12-26 17:02:42,916][105692] Updated weights for policy 0, policy_version 230765 (0.0008) [2023-12-26 17:02:43,496][105620] Updated weights for policy 1, policy_version 231338 (0.0008) [2023-12-26 17:02:43,514][105692] Updated weights for policy 0, policy_version 230775 (0.0006) [2023-12-26 17:02:43,544][105620] Updated weights for policy 1, policy_version 231348 (0.0010) [2023-12-26 17:02:43,570][105692] Updated weights for policy 0, policy_version 230785 (0.0006) [2023-12-26 17:02:43,603][105620] Updated weights for policy 1, policy_version 231358 (0.0010) [2023-12-26 17:02:43,627][105692] Updated weights for policy 0, policy_version 230795 (0.0005) [2023-12-26 17:02:43,648][105620] Updated weights for policy 1, policy_version 231368 (0.0010) [2023-12-26 17:02:44,145][105692] Updated weights for policy 0, policy_version 230805 (0.0008) [2023-12-26 17:02:44,203][105692] Updated weights for policy 0, policy_version 230815 (0.0005) [2023-12-26 17:02:44,262][105692] Updated weights for policy 0, policy_version 230825 (0.0005) [2023-12-26 17:02:44,404][105620] Updated weights for policy 1, policy_version 231378 (0.0005) [2023-12-26 17:02:44,463][105620] Updated weights for policy 1, policy_version 231388 (0.0006) [2023-12-26 17:02:44,517][105620] Updated weights for policy 1, policy_version 231398 (0.0009) [2023-12-26 17:02:44,784][105692] Updated weights for policy 0, policy_version 230835 (0.0006) [2023-12-26 17:02:44,846][105692] Updated weights for policy 0, policy_version 230845 (0.0008) [2023-12-26 17:02:44,905][105692] Updated weights for policy 0, policy_version 230855 (0.0008) [2023-12-26 17:02:45,166][105620] Updated weights for policy 1, policy_version 231408 (0.0010) [2023-12-26 17:02:45,234][105620] Updated weights for policy 1, policy_version 231418 (0.0007) [2023-12-26 17:02:45,293][105620] Updated weights for policy 1, policy_version 231428 (0.0011) [2023-12-26 17:02:45,606][105692] Updated weights for policy 0, policy_version 230865 (0.0008) [2023-12-26 17:02:45,694][105692] Updated weights for policy 0, policy_version 230875 (0.0005) [2023-12-26 17:02:45,737][105692] Updated weights for policy 0, policy_version 230885 (0.0005) [2023-12-26 17:02:45,792][105692] Updated weights for policy 0, policy_version 230895 (0.0005) [2023-12-26 17:02:45,993][105620] Updated weights for policy 1, policy_version 231438 (0.0010) [2023-12-26 17:02:46,047][105620] Updated weights for policy 1, policy_version 231448 (0.0010) [2023-12-26 17:02:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 118374400. Throughput: 0: 9791.7, 1: 9750.6. Samples: 118342076. Policy #0 lag: (min: 16.0, avg: 31.1, max: 32.0) [2023-12-26 17:02:46,063][104569] Avg episode reward: [(0, '773.416'), (1, '9265.440')] [2023-12-26 17:02:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000230896_59121664.pth... [2023-12-26 17:02:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000229744_58826752.pth [2023-12-26 17:02:46,109][105620] Updated weights for policy 1, policy_version 231458 (0.0010) [2023-12-26 17:02:46,136][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000231464_59260928.pth... [2023-12-26 17:02:46,139][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000230312_58966016.pth [2023-12-26 17:02:46,325][105692] Updated weights for policy 0, policy_version 230905 (0.0010) [2023-12-26 17:02:46,376][105692] Updated weights for policy 0, policy_version 230915 (0.0010) [2023-12-26 17:02:46,420][105692] Updated weights for policy 0, policy_version 230925 (0.0010) [2023-12-26 17:02:46,820][105620] Updated weights for policy 1, policy_version 231468 (0.0010) [2023-12-26 17:02:46,874][105620] Updated weights for policy 1, policy_version 231478 (0.0011) [2023-12-26 17:02:46,932][105620] Updated weights for policy 1, policy_version 231488 (0.0010) [2023-12-26 17:02:47,121][105692] Updated weights for policy 0, policy_version 230935 (0.0006) [2023-12-26 17:02:47,193][105692] Updated weights for policy 0, policy_version 230945 (0.0006) [2023-12-26 17:02:47,255][105692] Updated weights for policy 0, policy_version 230955 (0.0010) [2023-12-26 17:02:47,631][105620] Updated weights for policy 1, policy_version 231498 (0.0010) [2023-12-26 17:02:47,697][105620] Updated weights for policy 1, policy_version 231508 (0.0008) [2023-12-26 17:02:47,751][105620] Updated weights for policy 1, policy_version 231518 (0.0009) [2023-12-26 17:02:47,819][105620] Updated weights for policy 1, policy_version 231528 (0.0006) [2023-12-26 17:02:47,862][105692] Updated weights for policy 0, policy_version 230965 (0.0007) [2023-12-26 17:02:47,924][105692] Updated weights for policy 0, policy_version 230975 (0.0006) [2023-12-26 17:02:47,997][105692] Updated weights for policy 0, policy_version 230985 (0.0005) [2023-12-26 17:02:48,497][105620] Updated weights for policy 1, policy_version 231538 (0.0009) [2023-12-26 17:02:48,560][105692] Updated weights for policy 0, policy_version 230995 (0.0005) [2023-12-26 17:02:48,564][105620] Updated weights for policy 1, policy_version 231548 (0.0009) [2023-12-26 17:02:48,623][105692] Updated weights for policy 0, policy_version 231005 (0.0008) [2023-12-26 17:02:48,625][105620] Updated weights for policy 1, policy_version 231558 (0.0008) [2023-12-26 17:02:48,686][105692] Updated weights for policy 0, policy_version 231015 (0.0008) [2023-12-26 17:02:49,394][105620] Updated weights for policy 1, policy_version 231568 (0.0008) [2023-12-26 17:02:49,438][105692] Updated weights for policy 0, policy_version 231025 (0.0011) [2023-12-26 17:02:49,460][105620] Updated weights for policy 1, policy_version 231578 (0.0008) [2023-12-26 17:02:49,498][105692] Updated weights for policy 0, policy_version 231035 (0.0011) [2023-12-26 17:02:49,523][105620] Updated weights for policy 1, policy_version 231588 (0.0006) [2023-12-26 17:02:49,557][105692] Updated weights for policy 0, policy_version 231045 (0.0010) [2023-12-26 17:02:49,623][105692] Updated weights for policy 0, policy_version 231055 (0.0010) [2023-12-26 17:02:50,288][105620] Updated weights for policy 1, policy_version 231598 (0.0007) [2023-12-26 17:02:50,344][105620] Updated weights for policy 1, policy_version 231608 (0.0008) [2023-12-26 17:02:50,394][105620] Updated weights for policy 1, policy_version 231618 (0.0006) [2023-12-26 17:02:50,396][105692] Updated weights for policy 0, policy_version 231065 (0.0011) [2023-12-26 17:02:50,446][105692] Updated weights for policy 0, policy_version 231075 (0.0011) [2023-12-26 17:02:50,499][105692] Updated weights for policy 0, policy_version 231085 (0.0011) [2023-12-26 17:02:51,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 118472704. Throughput: 0: 9948.9, 1: 9710.0. Samples: 118466088. Policy #0 lag: (min: 16.0, avg: 31.1, max: 32.0) [2023-12-26 17:02:51,062][104569] Avg episode reward: [(0, '6215.985'), (1, '9354.644')] [2023-12-26 17:02:51,209][105620] Updated weights for policy 1, policy_version 231628 (0.0007) [2023-12-26 17:02:51,273][105620] Updated weights for policy 1, policy_version 231638 (0.0009) [2023-12-26 17:02:51,300][105692] Updated weights for policy 0, policy_version 231095 (0.0011) [2023-12-26 17:02:51,332][105620] Updated weights for policy 1, policy_version 231648 (0.0007) [2023-12-26 17:02:51,363][105692] Updated weights for policy 0, policy_version 231105 (0.0011) [2023-12-26 17:02:51,430][105692] Updated weights for policy 0, policy_version 231115 (0.0008) [2023-12-26 17:02:52,030][105620] Updated weights for policy 1, policy_version 231658 (0.0007) [2023-12-26 17:02:52,095][105620] Updated weights for policy 1, policy_version 231668 (0.0006) [2023-12-26 17:02:52,147][105620] Updated weights for policy 1, policy_version 231678 (0.0006) [2023-12-26 17:02:52,200][105620] Updated weights for policy 1, policy_version 231688 (0.0008) [2023-12-26 17:02:52,259][105692] Updated weights for policy 0, policy_version 231125 (0.0009) [2023-12-26 17:02:52,321][105692] Updated weights for policy 0, policy_version 231135 (0.0011) [2023-12-26 17:02:52,388][105692] Updated weights for policy 0, policy_version 231145 (0.0012) [2023-12-26 17:02:52,898][105620] Updated weights for policy 1, policy_version 231698 (0.0009) [2023-12-26 17:02:52,952][105620] Updated weights for policy 1, policy_version 231708 (0.0009) [2023-12-26 17:02:53,013][105620] Updated weights for policy 1, policy_version 231718 (0.0008) [2023-12-26 17:02:53,018][105692] Updated weights for policy 0, policy_version 231155 (0.0008) [2023-12-26 17:02:53,071][105692] Updated weights for policy 0, policy_version 231165 (0.0008) [2023-12-26 17:02:53,125][105692] Updated weights for policy 0, policy_version 231175 (0.0006) [2023-12-26 17:02:53,792][105692] Updated weights for policy 0, policy_version 231185 (0.0005) [2023-12-26 17:02:53,828][105620] Updated weights for policy 1, policy_version 231728 (0.0007) [2023-12-26 17:02:53,852][105692] Updated weights for policy 0, policy_version 231195 (0.0006) [2023-12-26 17:02:53,893][105620] Updated weights for policy 1, policy_version 231738 (0.0006) [2023-12-26 17:02:53,907][105692] Updated weights for policy 0, policy_version 231205 (0.0008) [2023-12-26 17:02:53,960][105620] Updated weights for policy 1, policy_version 231748 (0.0006) [2023-12-26 17:02:53,962][105692] Updated weights for policy 0, policy_version 231215 (0.0010) [2023-12-26 17:02:54,590][105692] Updated weights for policy 0, policy_version 231225 (0.0010) [2023-12-26 17:02:54,601][105620] Updated weights for policy 1, policy_version 231758 (0.0007) [2023-12-26 17:02:54,634][105692] Updated weights for policy 0, policy_version 231235 (0.0010) [2023-12-26 17:02:54,653][105620] Updated weights for policy 1, policy_version 231768 (0.0005) [2023-12-26 17:02:54,692][105692] Updated weights for policy 0, policy_version 231245 (0.0009) [2023-12-26 17:02:54,706][105620] Updated weights for policy 1, policy_version 231778 (0.0007) [2023-12-26 17:02:55,408][105620] Updated weights for policy 1, policy_version 231788 (0.0008) [2023-12-26 17:02:55,444][105692] Updated weights for policy 0, policy_version 231255 (0.0010) [2023-12-26 17:02:55,475][105620] Updated weights for policy 1, policy_version 231798 (0.0006) [2023-12-26 17:02:55,502][105692] Updated weights for policy 0, policy_version 231265 (0.0010) [2023-12-26 17:02:55,533][105620] Updated weights for policy 1, policy_version 231808 (0.0010) [2023-12-26 17:02:55,558][105692] Updated weights for policy 0, policy_version 231275 (0.0010) [2023-12-26 17:02:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 118571008. Throughput: 0: 9979.5, 1: 9712.1. Samples: 118582112. Policy #0 lag: (min: 16.0, avg: 31.1, max: 32.0) [2023-12-26 17:02:56,062][104569] Avg episode reward: [(0, '8221.375'), (1, '9354.953')] [2023-12-26 17:02:56,151][105620] Updated weights for policy 1, policy_version 231818 (0.0010) [2023-12-26 17:02:56,209][105620] Updated weights for policy 1, policy_version 231828 (0.0009) [2023-12-26 17:02:56,263][105692] Updated weights for policy 0, policy_version 231285 (0.0009) [2023-12-26 17:02:56,265][105620] Updated weights for policy 1, policy_version 231838 (0.0007) [2023-12-26 17:02:56,312][105692] Updated weights for policy 0, policy_version 231295 (0.0005) [2023-12-26 17:02:56,317][105620] Updated weights for policy 1, policy_version 231848 (0.0006) [2023-12-26 17:02:56,364][105692] Updated weights for policy 0, policy_version 231305 (0.0007) [2023-12-26 17:02:56,968][105692] Updated weights for policy 0, policy_version 231315 (0.0005) [2023-12-26 17:02:57,011][105620] Updated weights for policy 1, policy_version 231858 (0.0005) [2023-12-26 17:02:57,026][105692] Updated weights for policy 0, policy_version 231325 (0.0008) [2023-12-26 17:02:57,069][105620] Updated weights for policy 1, policy_version 231868 (0.0005) [2023-12-26 17:02:57,087][105692] Updated weights for policy 0, policy_version 231335 (0.0010) [2023-12-26 17:02:57,124][105620] Updated weights for policy 1, policy_version 231878 (0.0005) [2023-12-26 17:02:57,725][105692] Updated weights for policy 0, policy_version 231345 (0.0010) [2023-12-26 17:02:57,731][105620] Updated weights for policy 1, policy_version 231888 (0.0005) [2023-12-26 17:02:57,780][105620] Updated weights for policy 1, policy_version 231898 (0.0007) [2023-12-26 17:02:57,782][105692] Updated weights for policy 0, policy_version 231355 (0.0009) [2023-12-26 17:02:57,839][105692] Updated weights for policy 0, policy_version 231365 (0.0006) [2023-12-26 17:02:57,840][105620] Updated weights for policy 1, policy_version 231908 (0.0008) [2023-12-26 17:02:57,890][105692] Updated weights for policy 0, policy_version 231375 (0.0007) [2023-12-26 17:02:58,596][105620] Updated weights for policy 1, policy_version 231918 (0.0006) [2023-12-26 17:02:58,613][105692] Updated weights for policy 0, policy_version 231385 (0.0011) [2023-12-26 17:02:58,656][105620] Updated weights for policy 1, policy_version 231928 (0.0008) [2023-12-26 17:02:58,677][105692] Updated weights for policy 0, policy_version 231395 (0.0011) [2023-12-26 17:02:58,722][105620] Updated weights for policy 1, policy_version 231938 (0.0008) [2023-12-26 17:02:58,739][105692] Updated weights for policy 0, policy_version 231405 (0.0010) [2023-12-26 17:02:59,417][105692] Updated weights for policy 0, policy_version 231415 (0.0007) [2023-12-26 17:02:59,479][105620] Updated weights for policy 1, policy_version 231948 (0.0007) [2023-12-26 17:02:59,484][105692] Updated weights for policy 0, policy_version 231425 (0.0007) [2023-12-26 17:02:59,538][105620] Updated weights for policy 1, policy_version 231958 (0.0007) [2023-12-26 17:02:59,540][105692] Updated weights for policy 0, policy_version 231435 (0.0009) [2023-12-26 17:02:59,589][105620] Updated weights for policy 1, policy_version 231968 (0.0008) [2023-12-26 17:03:00,158][105692] Updated weights for policy 0, policy_version 231445 (0.0007) [2023-12-26 17:03:00,218][105692] Updated weights for policy 0, policy_version 231455 (0.0008) [2023-12-26 17:03:00,271][105692] Updated weights for policy 0, policy_version 231466 (0.0010) [2023-12-26 17:03:00,285][105620] Updated weights for policy 1, policy_version 231978 (0.0006) [2023-12-26 17:03:00,343][105620] Updated weights for policy 1, policy_version 231988 (0.0007) [2023-12-26 17:03:00,409][105620] Updated weights for policy 1, policy_version 231998 (0.0008) [2023-12-26 17:03:00,471][105620] Updated weights for policy 1, policy_version 232008 (0.0007) [2023-12-26 17:03:00,933][105692] Updated weights for policy 0, policy_version 231476 (0.0010) [2023-12-26 17:03:00,985][105692] Updated weights for policy 0, policy_version 231486 (0.0011) [2023-12-26 17:03:01,038][105692] Updated weights for policy 0, policy_version 231496 (0.0011) [2023-12-26 17:03:01,045][105620] Updated weights for policy 1, policy_version 232018 (0.0008) [2023-12-26 17:03:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 118669312. Throughput: 0: 10029.3, 1: 9755.4. Samples: 118644156. Policy #0 lag: (min: 16.0, avg: 31.1, max: 32.0) [2023-12-26 17:03:01,062][104569] Avg episode reward: [(0, '7828.196'), (1, '9353.773')] [2023-12-26 17:03:01,087][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000231504_59277312.pth... [2023-12-26 17:03:01,092][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000230288_58966016.pth [2023-12-26 17:03:01,105][105620] Updated weights for policy 1, policy_version 232028 (0.0009) [2023-12-26 17:03:01,168][105620] Updated weights for policy 1, policy_version 232038 (0.0011) [2023-12-26 17:03:01,174][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000232040_59408384.pth... [2023-12-26 17:03:01,177][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000230888_59113472.pth [2023-12-26 17:03:01,716][105692] Updated weights for policy 0, policy_version 231506 (0.0008) [2023-12-26 17:03:01,764][105692] Updated weights for policy 0, policy_version 231516 (0.0010) [2023-12-26 17:03:01,820][105692] Updated weights for policy 0, policy_version 231526 (0.0010) [2023-12-26 17:03:01,850][105620] Updated weights for policy 1, policy_version 232048 (0.0011) [2023-12-26 17:03:01,876][105692] Updated weights for policy 0, policy_version 231536 (0.0011) [2023-12-26 17:03:01,902][105620] Updated weights for policy 1, policy_version 232058 (0.0011) [2023-12-26 17:03:01,952][105620] Updated weights for policy 1, policy_version 232068 (0.0010) [2023-12-26 17:03:02,607][105692] Updated weights for policy 0, policy_version 231546 (0.0005) [2023-12-26 17:03:02,671][105692] Updated weights for policy 0, policy_version 231556 (0.0006) [2023-12-26 17:03:02,693][105620] Updated weights for policy 1, policy_version 232078 (0.0010) [2023-12-26 17:03:02,731][105692] Updated weights for policy 0, policy_version 231566 (0.0006) [2023-12-26 17:03:02,742][105620] Updated weights for policy 1, policy_version 232088 (0.0010) [2023-12-26 17:03:02,787][105620] Updated weights for policy 1, policy_version 232098 (0.0010) [2023-12-26 17:03:03,299][105692] Updated weights for policy 0, policy_version 231576 (0.0006) [2023-12-26 17:03:03,344][105692] Updated weights for policy 0, policy_version 231586 (0.0005) [2023-12-26 17:03:03,388][105692] Updated weights for policy 0, policy_version 231596 (0.0005) [2023-12-26 17:03:03,545][105620] Updated weights for policy 1, policy_version 232108 (0.0010) [2023-12-26 17:03:03,593][105620] Updated weights for policy 1, policy_version 232118 (0.0010) [2023-12-26 17:03:03,640][105620] Updated weights for policy 1, policy_version 232128 (0.0010) [2023-12-26 17:03:04,061][105692] Updated weights for policy 0, policy_version 231606 (0.0007) [2023-12-26 17:03:04,127][105692] Updated weights for policy 0, policy_version 231616 (0.0008) [2023-12-26 17:03:04,197][105692] Updated weights for policy 0, policy_version 231626 (0.0009) [2023-12-26 17:03:04,340][105620] Updated weights for policy 1, policy_version 232138 (0.0010) [2023-12-26 17:03:04,395][105620] Updated weights for policy 1, policy_version 232148 (0.0010) [2023-12-26 17:03:04,454][105620] Updated weights for policy 1, policy_version 232158 (0.0009) [2023-12-26 17:03:04,508][105620] Updated weights for policy 1, policy_version 232168 (0.0008) [2023-12-26 17:03:04,834][105692] Updated weights for policy 0, policy_version 231636 (0.0008) [2023-12-26 17:03:04,897][105692] Updated weights for policy 0, policy_version 231646 (0.0006) [2023-12-26 17:03:04,964][105692] Updated weights for policy 0, policy_version 231656 (0.0006) [2023-12-26 17:03:05,349][105620] Updated weights for policy 1, policy_version 232178 (0.0009) [2023-12-26 17:03:05,397][105620] Updated weights for policy 1, policy_version 232188 (0.0009) [2023-12-26 17:03:05,449][105620] Updated weights for policy 1, policy_version 232198 (0.0009) [2023-12-26 17:03:05,615][105692] Updated weights for policy 0, policy_version 231666 (0.0008) [2023-12-26 17:03:05,676][105692] Updated weights for policy 0, policy_version 231676 (0.0007) [2023-12-26 17:03:05,740][105692] Updated weights for policy 0, policy_version 231686 (0.0005) [2023-12-26 17:03:05,788][105692] Updated weights for policy 0, policy_version 231696 (0.0005) [2023-12-26 17:03:06,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 118775808. Throughput: 0: 10103.7, 1: 9767.3. Samples: 118764624. Policy #0 lag: (min: 16.0, avg: 31.1, max: 32.0) [2023-12-26 17:03:06,062][104569] Avg episode reward: [(0, '8069.452'), (1, '9353.354')] [2023-12-26 17:03:06,274][105620] Updated weights for policy 1, policy_version 232208 (0.0008) [2023-12-26 17:03:06,323][105620] Updated weights for policy 1, policy_version 232218 (0.0009) [2023-12-26 17:03:06,378][105620] Updated weights for policy 1, policy_version 232228 (0.0010) [2023-12-26 17:03:06,442][105692] Updated weights for policy 0, policy_version 231706 (0.0008) [2023-12-26 17:03:06,497][105692] Updated weights for policy 0, policy_version 231716 (0.0009) [2023-12-26 17:03:06,552][105692] Updated weights for policy 0, policy_version 231726 (0.0009) [2023-12-26 17:03:07,153][105620] Updated weights for policy 1, policy_version 232238 (0.0008) [2023-12-26 17:03:07,204][105620] Updated weights for policy 1, policy_version 232248 (0.0009) [2023-12-26 17:03:07,257][105620] Updated weights for policy 1, policy_version 232258 (0.0009) [2023-12-26 17:03:07,299][105692] Updated weights for policy 0, policy_version 231736 (0.0008) [2023-12-26 17:03:07,350][105692] Updated weights for policy 0, policy_version 231746 (0.0009) [2023-12-26 17:03:07,400][105692] Updated weights for policy 0, policy_version 231756 (0.0009) [2023-12-26 17:03:08,029][105620] Updated weights for policy 1, policy_version 232268 (0.0008) [2023-12-26 17:03:08,092][105620] Updated weights for policy 1, policy_version 232278 (0.0009) [2023-12-26 17:03:08,154][105620] Updated weights for policy 1, policy_version 232288 (0.0009) [2023-12-26 17:03:08,177][105692] Updated weights for policy 0, policy_version 231766 (0.0009) [2023-12-26 17:03:08,240][105692] Updated weights for policy 0, policy_version 231776 (0.0008) [2023-12-26 17:03:08,293][105692] Updated weights for policy 0, policy_version 231786 (0.0009) [2023-12-26 17:03:08,927][105620] Updated weights for policy 1, policy_version 232298 (0.0008) [2023-12-26 17:03:08,988][105620] Updated weights for policy 1, policy_version 232308 (0.0009) [2023-12-26 17:03:09,025][105692] Updated weights for policy 0, policy_version 231796 (0.0008) [2023-12-26 17:03:09,043][105620] Updated weights for policy 1, policy_version 232318 (0.0008) [2023-12-26 17:03:09,075][105692] Updated weights for policy 0, policy_version 231806 (0.0006) [2023-12-26 17:03:09,096][105620] Updated weights for policy 1, policy_version 232328 (0.0008) [2023-12-26 17:03:09,128][105692] Updated weights for policy 0, policy_version 231816 (0.0009) [2023-12-26 17:03:09,899][105620] Updated weights for policy 1, policy_version 232338 (0.0008) [2023-12-26 17:03:09,927][105692] Updated weights for policy 0, policy_version 231826 (0.0008) [2023-12-26 17:03:09,968][105620] Updated weights for policy 1, policy_version 232348 (0.0008) [2023-12-26 17:03:09,986][105692] Updated weights for policy 0, policy_version 231836 (0.0007) [2023-12-26 17:03:10,029][105620] Updated weights for policy 1, policy_version 232358 (0.0008) [2023-12-26 17:03:10,044][105692] Updated weights for policy 0, policy_version 231846 (0.0007) [2023-12-26 17:03:10,100][105692] Updated weights for policy 0, policy_version 231856 (0.0006) [2023-12-26 17:03:10,671][105620] Updated weights for policy 1, policy_version 232368 (0.0007) [2023-12-26 17:03:10,731][105620] Updated weights for policy 1, policy_version 232378 (0.0005) [2023-12-26 17:03:10,792][105620] Updated weights for policy 1, policy_version 232388 (0.0007) [2023-12-26 17:03:10,860][105692] Updated weights for policy 0, policy_version 231866 (0.0009) [2023-12-26 17:03:10,908][105692] Updated weights for policy 0, policy_version 231876 (0.0010) [2023-12-26 17:03:10,955][105692] Updated weights for policy 0, policy_version 231886 (0.0008) [2023-12-26 17:03:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 118874112. Throughput: 0: 10110.6, 1: 9709.2. Samples: 118879120. Policy #0 lag: (min: 16.0, avg: 31.1, max: 32.0) [2023-12-26 17:03:11,063][104569] Avg episode reward: [(0, '8678.916'), (1, '9353.967')] [2023-12-26 17:03:11,513][105620] Updated weights for policy 1, policy_version 232398 (0.0008) [2023-12-26 17:03:11,575][105620] Updated weights for policy 1, policy_version 232408 (0.0009) [2023-12-26 17:03:11,641][105620] Updated weights for policy 1, policy_version 232418 (0.0008) [2023-12-26 17:03:11,792][105692] Updated weights for policy 0, policy_version 231896 (0.0009) [2023-12-26 17:03:11,857][105692] Updated weights for policy 0, policy_version 231906 (0.0007) [2023-12-26 17:03:11,925][105692] Updated weights for policy 0, policy_version 231916 (0.0008) [2023-12-26 17:03:12,363][105620] Updated weights for policy 1, policy_version 232428 (0.0007) [2023-12-26 17:03:12,415][105620] Updated weights for policy 1, policy_version 232438 (0.0006) [2023-12-26 17:03:12,474][105620] Updated weights for policy 1, policy_version 232448 (0.0005) [2023-12-26 17:03:12,728][105692] Updated weights for policy 0, policy_version 231926 (0.0008) [2023-12-26 17:03:12,779][105692] Updated weights for policy 0, policy_version 231936 (0.0009) [2023-12-26 17:03:12,831][105692] Updated weights for policy 0, policy_version 231946 (0.0010) [2023-12-26 17:03:13,173][105620] Updated weights for policy 1, policy_version 232458 (0.0008) [2023-12-26 17:03:13,226][105620] Updated weights for policy 1, policy_version 232468 (0.0007) [2023-12-26 17:03:13,279][105620] Updated weights for policy 1, policy_version 232478 (0.0008) [2023-12-26 17:03:13,327][105620] Updated weights for policy 1, policy_version 232488 (0.0009) [2023-12-26 17:03:13,659][105692] Updated weights for policy 0, policy_version 231956 (0.0009) [2023-12-26 17:03:13,713][105692] Updated weights for policy 0, policy_version 231966 (0.0009) [2023-12-26 17:03:13,767][105692] Updated weights for policy 0, policy_version 231976 (0.0009) [2023-12-26 17:03:13,960][105620] Updated weights for policy 1, policy_version 232498 (0.0009) [2023-12-26 17:03:14,014][105620] Updated weights for policy 1, policy_version 232508 (0.0009) [2023-12-26 17:03:14,074][105620] Updated weights for policy 1, policy_version 232518 (0.0009) [2023-12-26 17:03:14,522][105692] Updated weights for policy 0, policy_version 231986 (0.0009) [2023-12-26 17:03:14,580][105692] Updated weights for policy 0, policy_version 231996 (0.0009) [2023-12-26 17:03:14,630][105692] Updated weights for policy 0, policy_version 232006 (0.0009) [2023-12-26 17:03:14,679][105692] Updated weights for policy 0, policy_version 232016 (0.0008) [2023-12-26 17:03:14,833][105620] Updated weights for policy 1, policy_version 232528 (0.0010) [2023-12-26 17:03:14,884][105620] Updated weights for policy 1, policy_version 232538 (0.0008) [2023-12-26 17:03:14,932][105620] Updated weights for policy 1, policy_version 232548 (0.0009) [2023-12-26 17:03:15,449][105692] Updated weights for policy 0, policy_version 232026 (0.0006) [2023-12-26 17:03:15,512][105692] Updated weights for policy 0, policy_version 232036 (0.0005) [2023-12-26 17:03:15,567][105692] Updated weights for policy 0, policy_version 232046 (0.0005) [2023-12-26 17:03:15,770][105620] Updated weights for policy 1, policy_version 232558 (0.0009) [2023-12-26 17:03:15,824][105620] Updated weights for policy 1, policy_version 232568 (0.0009) [2023-12-26 17:03:15,880][105620] Updated weights for policy 1, policy_version 232579 (0.0009) [2023-12-26 17:03:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 118964224. Throughput: 0: 10070.1, 1: 9682.9. Samples: 118935468. Policy #0 lag: (min: 16.0, avg: 31.1, max: 32.0) [2023-12-26 17:03:16,062][104569] Avg episode reward: [(0, '8277.692'), (1, '9355.239')] [2023-12-26 17:03:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000232048_59416576.pth... [2023-12-26 17:03:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000232584_59547648.pth... [2023-12-26 17:03:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000231464_59260928.pth [2023-12-26 17:03:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000230896_59121664.pth [2023-12-26 17:03:16,141][105692] Updated weights for policy 0, policy_version 232056 (0.0008) [2023-12-26 17:03:16,201][105692] Updated weights for policy 0, policy_version 232066 (0.0009) [2023-12-26 17:03:16,259][105692] Updated weights for policy 0, policy_version 232076 (0.0009) [2023-12-26 17:03:16,654][105620] Updated weights for policy 1, policy_version 232589 (0.0010) [2023-12-26 17:03:16,712][105620] Updated weights for policy 1, policy_version 232599 (0.0009) [2023-12-26 17:03:16,760][105620] Updated weights for policy 1, policy_version 232609 (0.0009) [2023-12-26 17:03:17,016][105692] Updated weights for policy 0, policy_version 232086 (0.0009) [2023-12-26 17:03:17,079][105692] Updated weights for policy 0, policy_version 232096 (0.0008) [2023-12-26 17:03:17,143][105692] Updated weights for policy 0, policy_version 232106 (0.0009) [2023-12-26 17:03:17,444][105620] Updated weights for policy 1, policy_version 232619 (0.0008) [2023-12-26 17:03:17,499][105620] Updated weights for policy 1, policy_version 232629 (0.0005) [2023-12-26 17:03:17,548][105620] Updated weights for policy 1, policy_version 232639 (0.0005) [2023-12-26 17:03:17,982][105692] Updated weights for policy 0, policy_version 232116 (0.0010) [2023-12-26 17:03:18,043][105692] Updated weights for policy 0, policy_version 232126 (0.0009) [2023-12-26 17:03:18,089][105692] Updated weights for policy 0, policy_version 232136 (0.0009) [2023-12-26 17:03:18,167][105620] Updated weights for policy 1, policy_version 232649 (0.0005) [2023-12-26 17:03:18,232][105620] Updated weights for policy 1, policy_version 232659 (0.0005) [2023-12-26 17:03:18,280][105620] Updated weights for policy 1, policy_version 232669 (0.0005) [2023-12-26 17:03:18,336][105620] Updated weights for policy 1, policy_version 232679 (0.0007) [2023-12-26 17:03:18,791][105692] Updated weights for policy 0, policy_version 232146 (0.0008) [2023-12-26 17:03:18,839][105692] Updated weights for policy 0, policy_version 232156 (0.0007) [2023-12-26 17:03:18,892][105692] Updated weights for policy 0, policy_version 232166 (0.0008) [2023-12-26 17:03:18,958][105692] Updated weights for policy 0, policy_version 232176 (0.0011) [2023-12-26 17:03:19,053][105620] Updated weights for policy 1, policy_version 232689 (0.0008) [2023-12-26 17:03:19,105][105620] Updated weights for policy 1, policy_version 232699 (0.0008) [2023-12-26 17:03:19,161][105620] Updated weights for policy 1, policy_version 232709 (0.0008) [2023-12-26 17:03:19,673][105692] Updated weights for policy 0, policy_version 232186 (0.0009) [2023-12-26 17:03:19,733][105692] Updated weights for policy 0, policy_version 232196 (0.0009) [2023-12-26 17:03:19,761][105585] KL-divergence is very high: 125.7247 [2023-12-26 17:03:19,799][105692] Updated weights for policy 0, policy_version 232206 (0.0009) [2023-12-26 17:03:19,926][105620] Updated weights for policy 1, policy_version 232719 (0.0009) [2023-12-26 17:03:19,989][105620] Updated weights for policy 1, policy_version 232729 (0.0010) [2023-12-26 17:03:20,058][105620] Updated weights for policy 1, policy_version 232739 (0.0008) [2023-12-26 17:03:20,636][105692] Updated weights for policy 0, policy_version 232216 (0.0009) [2023-12-26 17:03:20,706][105692] Updated weights for policy 0, policy_version 232226 (0.0007) [2023-12-26 17:03:20,732][105620] Updated weights for policy 1, policy_version 232749 (0.0009) [2023-12-26 17:03:20,772][105692] Updated weights for policy 0, policy_version 232236 (0.0007) [2023-12-26 17:03:20,796][105620] Updated weights for policy 1, policy_version 232759 (0.0008) [2023-12-26 17:03:20,865][105620] Updated weights for policy 1, policy_version 232769 (0.0007) [2023-12-26 17:03:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19797.3, 300 sec: 19521.9). Total num frames: 119062528. Throughput: 0: 10023.5, 1: 9694.7. Samples: 119049788. Policy #0 lag: (min: 16.0, avg: 31.1, max: 32.0) [2023-12-26 17:03:21,063][104569] Avg episode reward: [(0, '5400.886'), (1, '9355.001')] [2023-12-26 17:03:21,459][105692] Updated weights for policy 0, policy_version 232246 (0.0008) [2023-12-26 17:03:21,508][105692] Updated weights for policy 0, policy_version 232256 (0.0009) [2023-12-26 17:03:21,546][105620] Updated weights for policy 1, policy_version 232779 (0.0007) [2023-12-26 17:03:21,560][105692] Updated weights for policy 0, policy_version 232266 (0.0008) [2023-12-26 17:03:21,612][105620] Updated weights for policy 1, policy_version 232789 (0.0008) [2023-12-26 17:03:21,684][105620] Updated weights for policy 1, policy_version 232799 (0.0009) [2023-12-26 17:03:22,350][105620] Updated weights for policy 1, policy_version 232809 (0.0008) [2023-12-26 17:03:22,409][105620] Updated weights for policy 1, policy_version 232819 (0.0008) [2023-12-26 17:03:22,429][105692] Updated weights for policy 0, policy_version 232276 (0.0010) [2023-12-26 17:03:22,466][105620] Updated weights for policy 1, policy_version 232829 (0.0007) [2023-12-26 17:03:22,492][105692] Updated weights for policy 0, policy_version 232286 (0.0011) [2023-12-26 17:03:22,518][105620] Updated weights for policy 1, policy_version 232839 (0.0007) [2023-12-26 17:03:22,556][105692] Updated weights for policy 0, policy_version 232296 (0.0006) [2023-12-26 17:03:23,251][105692] Updated weights for policy 0, policy_version 232306 (0.0006) [2023-12-26 17:03:23,310][105692] Updated weights for policy 0, policy_version 232316 (0.0011) [2023-12-26 17:03:23,311][105620] Updated weights for policy 1, policy_version 232849 (0.0006) [2023-12-26 17:03:23,363][105620] Updated weights for policy 1, policy_version 232859 (0.0006) [2023-12-26 17:03:23,368][105692] Updated weights for policy 0, policy_version 232326 (0.0010) [2023-12-26 17:03:23,411][105620] Updated weights for policy 1, policy_version 232869 (0.0007) [2023-12-26 17:03:23,420][105692] Updated weights for policy 0, policy_version 232336 (0.0010) [2023-12-26 17:03:24,051][105692] Updated weights for policy 0, policy_version 232346 (0.0005) [2023-12-26 17:03:24,109][105692] Updated weights for policy 0, policy_version 232356 (0.0005) [2023-12-26 17:03:24,169][105692] Updated weights for policy 0, policy_version 232366 (0.0005) [2023-12-26 17:03:24,194][105620] Updated weights for policy 1, policy_version 232879 (0.0009) [2023-12-26 17:03:24,251][105620] Updated weights for policy 1, policy_version 232889 (0.0009) [2023-12-26 17:03:24,318][105620] Updated weights for policy 1, policy_version 232899 (0.0010) [2023-12-26 17:03:24,732][105692] Updated weights for policy 0, policy_version 232376 (0.0008) [2023-12-26 17:03:24,794][105692] Updated weights for policy 0, policy_version 232386 (0.0011) [2023-12-26 17:03:24,868][105692] Updated weights for policy 0, policy_version 232396 (0.0008) [2023-12-26 17:03:25,089][105620] Updated weights for policy 1, policy_version 232909 (0.0009) [2023-12-26 17:03:25,158][105620] Updated weights for policy 1, policy_version 232919 (0.0008) [2023-12-26 17:03:25,223][105620] Updated weights for policy 1, policy_version 232929 (0.0009) [2023-12-26 17:03:25,560][105692] Updated weights for policy 0, policy_version 232406 (0.0010) [2023-12-26 17:03:25,624][105692] Updated weights for policy 0, policy_version 232416 (0.0010) [2023-12-26 17:03:25,688][105692] Updated weights for policy 0, policy_version 232426 (0.0010) [2023-12-26 17:03:25,951][105620] Updated weights for policy 1, policy_version 232939 (0.0008) [2023-12-26 17:03:26,003][105620] Updated weights for policy 1, policy_version 232949 (0.0008) [2023-12-26 17:03:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 119152640. Throughput: 0: 9950.4, 1: 9652.6. Samples: 119164676. Policy #0 lag: (min: 16.0, avg: 31.1, max: 32.0) [2023-12-26 17:03:26,062][105620] Updated weights for policy 1, policy_version 232959 (0.0008) [2023-12-26 17:03:26,062][104569] Avg episode reward: [(0, '7085.856'), (1, '9354.288')] [2023-12-26 17:03:26,398][105692] Updated weights for policy 0, policy_version 232436 (0.0008) [2023-12-26 17:03:26,461][105692] Updated weights for policy 0, policy_version 232446 (0.0007) [2023-12-26 17:03:26,523][105692] Updated weights for policy 0, policy_version 232456 (0.0010) [2023-12-26 17:03:26,843][105620] Updated weights for policy 1, policy_version 232969 (0.0008) [2023-12-26 17:03:26,905][105620] Updated weights for policy 1, policy_version 232979 (0.0008) [2023-12-26 17:03:26,964][105620] Updated weights for policy 1, policy_version 232989 (0.0008) [2023-12-26 17:03:27,022][105620] Updated weights for policy 1, policy_version 232999 (0.0008) [2023-12-26 17:03:27,205][105692] Updated weights for policy 0, policy_version 232466 (0.0009) [2023-12-26 17:03:27,255][105692] Updated weights for policy 0, policy_version 232476 (0.0005) [2023-12-26 17:03:27,303][105692] Updated weights for policy 0, policy_version 232486 (0.0005) [2023-12-26 17:03:27,364][105692] Updated weights for policy 0, policy_version 232496 (0.0005) [2023-12-26 17:03:27,836][105620] Updated weights for policy 1, policy_version 233009 (0.0009) [2023-12-26 17:03:27,896][105620] Updated weights for policy 1, policy_version 233019 (0.0010) [2023-12-26 17:03:27,898][105692] Updated weights for policy 0, policy_version 232506 (0.0005) [2023-12-26 17:03:27,951][105692] Updated weights for policy 0, policy_version 232516 (0.0005) [2023-12-26 17:03:27,955][105620] Updated weights for policy 1, policy_version 233029 (0.0009) [2023-12-26 17:03:28,012][105692] Updated weights for policy 0, policy_version 232526 (0.0005) [2023-12-26 17:03:28,622][105692] Updated weights for policy 0, policy_version 232536 (0.0009) [2023-12-26 17:03:28,673][105692] Updated weights for policy 0, policy_version 232546 (0.0010) [2023-12-26 17:03:28,706][105620] Updated weights for policy 1, policy_version 233039 (0.0006) [2023-12-26 17:03:28,724][105692] Updated weights for policy 0, policy_version 232556 (0.0010) [2023-12-26 17:03:28,758][105620] Updated weights for policy 1, policy_version 233049 (0.0006) [2023-12-26 17:03:28,813][105620] Updated weights for policy 1, policy_version 233059 (0.0008) [2023-12-26 17:03:29,440][105692] Updated weights for policy 0, policy_version 232566 (0.0010) [2023-12-26 17:03:29,488][105692] Updated weights for policy 0, policy_version 232576 (0.0010) [2023-12-26 17:03:29,545][105692] Updated weights for policy 0, policy_version 232586 (0.0010) [2023-12-26 17:03:29,605][105620] Updated weights for policy 1, policy_version 233069 (0.0007) [2023-12-26 17:03:29,667][105620] Updated weights for policy 1, policy_version 233079 (0.0008) [2023-12-26 17:03:29,726][105620] Updated weights for policy 1, policy_version 233089 (0.0008) [2023-12-26 17:03:30,350][105692] Updated weights for policy 0, policy_version 232596 (0.0010) [2023-12-26 17:03:30,410][105692] Updated weights for policy 0, policy_version 232606 (0.0010) [2023-12-26 17:03:30,474][105692] Updated weights for policy 0, policy_version 232616 (0.0010) [2023-12-26 17:03:30,489][105620] Updated weights for policy 1, policy_version 233099 (0.0007) [2023-12-26 17:03:30,541][105620] Updated weights for policy 1, policy_version 233109 (0.0006) [2023-12-26 17:03:30,588][105620] Updated weights for policy 1, policy_version 233119 (0.0007) [2023-12-26 17:03:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 119250944. Throughput: 0: 9988.3, 1: 9607.8. Samples: 119223904. Policy #0 lag: (min: 16.0, avg: 31.1, max: 32.0) [2023-12-26 17:03:31,063][104569] Avg episode reward: [(0, '9166.819'), (1, '9265.425')] [2023-12-26 17:03:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000232624_59564032.pth... [2023-12-26 17:03:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000233128_59686912.pth... [2023-12-26 17:03:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000231504_59277312.pth [2023-12-26 17:03:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000232040_59408384.pth [2023-12-26 17:03:31,197][105692] Updated weights for policy 0, policy_version 232626 (0.0010) [2023-12-26 17:03:31,261][105692] Updated weights for policy 0, policy_version 232636 (0.0011) [2023-12-26 17:03:31,328][105692] Updated weights for policy 0, policy_version 232646 (0.0010) [2023-12-26 17:03:31,356][105620] Updated weights for policy 1, policy_version 233129 (0.0008) [2023-12-26 17:03:31,393][105692] Updated weights for policy 0, policy_version 232656 (0.0011) [2023-12-26 17:03:31,421][105620] Updated weights for policy 1, policy_version 233139 (0.0007) [2023-12-26 17:03:31,488][105620] Updated weights for policy 1, policy_version 233149 (0.0008) [2023-12-26 17:03:31,544][105620] Updated weights for policy 1, policy_version 233159 (0.0008) [2023-12-26 17:03:32,146][105692] Updated weights for policy 0, policy_version 232666 (0.0008) [2023-12-26 17:03:32,211][105692] Updated weights for policy 0, policy_version 232676 (0.0007) [2023-12-26 17:03:32,273][105620] Updated weights for policy 1, policy_version 233169 (0.0009) [2023-12-26 17:03:32,283][105692] Updated weights for policy 0, policy_version 232687 (0.0009) [2023-12-26 17:03:32,334][105620] Updated weights for policy 1, policy_version 233179 (0.0007) [2023-12-26 17:03:32,399][105620] Updated weights for policy 1, policy_version 233189 (0.0010) [2023-12-26 17:03:32,989][105692] Updated weights for policy 0, policy_version 232697 (0.0008) [2023-12-26 17:03:33,051][105692] Updated weights for policy 0, policy_version 232707 (0.0008) [2023-12-26 17:03:33,096][105620] Updated weights for policy 1, policy_version 233199 (0.0010) [2023-12-26 17:03:33,102][105692] Updated weights for policy 0, policy_version 232717 (0.0006) [2023-12-26 17:03:33,150][105620] Updated weights for policy 1, policy_version 233209 (0.0010) [2023-12-26 17:03:33,201][105620] Updated weights for policy 1, policy_version 233219 (0.0010) [2023-12-26 17:03:33,202][105586] KL-divergence is very high: 126.8603 [2023-12-26 17:03:33,794][105620] Updated weights for policy 1, policy_version 233229 (0.0008) [2023-12-26 17:03:33,810][105586] KL-divergence is very high: 139.9920 [2023-12-26 17:03:33,816][105586] KL-divergence is very high: 206.9206 [2023-12-26 17:03:33,822][105586] KL-divergence is very high: 264.5709 [2023-12-26 17:03:33,828][105586] KL-divergence is very high: 245.5500 [2023-12-26 17:03:33,832][105586] KL-divergence is very high: 164.7117 [2023-12-26 17:03:33,847][105692] Updated weights for policy 0, policy_version 232727 (0.0009) [2023-12-26 17:03:33,848][105620] Updated weights for policy 1, policy_version 233239 (0.0005) [2023-12-26 17:03:33,857][105586] KL-divergence is very high: 102.0405 [2023-12-26 17:03:33,862][105586] KL-divergence is very high: 111.8766 [2023-12-26 17:03:33,894][105692] Updated weights for policy 0, policy_version 232737 (0.0010) [2023-12-26 17:03:33,895][105620] Updated weights for policy 1, policy_version 233249 (0.0005) [2023-12-26 17:03:33,949][105692] Updated weights for policy 0, policy_version 232747 (0.0010) [2023-12-26 17:03:34,501][105586] KL-divergence is very high: 111.3008 [2023-12-26 17:03:34,520][105620] Updated weights for policy 1, policy_version 233259 (0.0006) [2023-12-26 17:03:34,532][105586] KL-divergence is very high: 370.0061 [2023-12-26 17:03:34,576][105586] KL-divergence is very high: 314.9417 [2023-12-26 17:03:34,577][105620] Updated weights for policy 1, policy_version 233269 (0.0006) [2023-12-26 17:03:34,619][105586] KL-divergence is very high: 138.1287 [2023-12-26 17:03:34,632][105620] Updated weights for policy 1, policy_version 233279 (0.0006) [2023-12-26 17:03:34,667][105586] KL-divergence is very high: 220.4457 [2023-12-26 17:03:34,673][105692] Updated weights for policy 0, policy_version 232757 (0.0009) [2023-12-26 17:03:34,736][105692] Updated weights for policy 0, policy_version 232767 (0.0007) [2023-12-26 17:03:34,804][105692] Updated weights for policy 0, policy_version 232777 (0.0007) [2023-12-26 17:03:35,252][105620] Updated weights for policy 1, policy_version 233289 (0.0008) [2023-12-26 17:03:35,314][105620] Updated weights for policy 1, policy_version 233299 (0.0008) [2023-12-26 17:03:35,369][105620] Updated weights for policy 1, policy_version 233309 (0.0005) [2023-12-26 17:03:35,422][105620] Updated weights for policy 1, policy_version 233319 (0.0005) [2023-12-26 17:03:35,593][105692] Updated weights for policy 0, policy_version 232787 (0.0008) [2023-12-26 17:03:35,653][105692] Updated weights for policy 0, policy_version 232797 (0.0006) [2023-12-26 17:03:35,710][105692] Updated weights for policy 0, policy_version 232807 (0.0005) [2023-12-26 17:03:35,985][105620] Updated weights for policy 1, policy_version 233329 (0.0005) [2023-12-26 17:03:36,043][105620] Updated weights for policy 1, policy_version 233339 (0.0005) [2023-12-26 17:03:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 119349248. Throughput: 0: 9795.8, 1: 9627.8. Samples: 119340152. Policy #0 lag: (min: 16.0, avg: 31.1, max: 32.0) [2023-12-26 17:03:36,063][104569] Avg episode reward: [(0, '9167.685'), (1, '5122.537')] [2023-12-26 17:03:36,108][105620] Updated weights for policy 1, policy_version 233349 (0.0006) [2023-12-26 17:03:36,383][105692] Updated weights for policy 0, policy_version 232817 (0.0006) [2023-12-26 17:03:36,448][105692] Updated weights for policy 0, policy_version 232827 (0.0011) [2023-12-26 17:03:36,504][105692] Updated weights for policy 0, policy_version 232837 (0.0010) [2023-12-26 17:03:36,565][105692] Updated weights for policy 0, policy_version 232847 (0.0008) [2023-12-26 17:03:36,754][105620] Updated weights for policy 1, policy_version 233359 (0.0009) [2023-12-26 17:03:36,814][105620] Updated weights for policy 1, policy_version 233369 (0.0008) [2023-12-26 17:03:36,870][105620] Updated weights for policy 1, policy_version 233379 (0.0008) [2023-12-26 17:03:37,264][105692] Updated weights for policy 0, policy_version 232857 (0.0010) [2023-12-26 17:03:37,322][105692] Updated weights for policy 0, policy_version 232867 (0.0010) [2023-12-26 17:03:37,373][105692] Updated weights for policy 0, policy_version 232877 (0.0010) [2023-12-26 17:03:37,626][105620] Updated weights for policy 1, policy_version 233389 (0.0009) [2023-12-26 17:03:37,674][105620] Updated weights for policy 1, policy_version 233399 (0.0010) [2023-12-26 17:03:37,731][105620] Updated weights for policy 1, policy_version 233409 (0.0010) [2023-12-26 17:03:37,991][105692] Updated weights for policy 0, policy_version 232887 (0.0007) [2023-12-26 17:03:38,045][105692] Updated weights for policy 0, policy_version 232897 (0.0005) [2023-12-26 17:03:38,095][105692] Updated weights for policy 0, policy_version 232907 (0.0009) [2023-12-26 17:03:38,491][105620] Updated weights for policy 1, policy_version 233419 (0.0009) [2023-12-26 17:03:38,547][105620] Updated weights for policy 1, policy_version 233429 (0.0005) [2023-12-26 17:03:38,612][105620] Updated weights for policy 1, policy_version 233439 (0.0007) [2023-12-26 17:03:38,674][105692] Updated weights for policy 0, policy_version 232917 (0.0008) [2023-12-26 17:03:38,722][105692] Updated weights for policy 0, policy_version 232927 (0.0011) [2023-12-26 17:03:38,780][105692] Updated weights for policy 0, policy_version 232937 (0.0010) [2023-12-26 17:03:39,311][105620] Updated weights for policy 1, policy_version 233449 (0.0009) [2023-12-26 17:03:39,384][105620] Updated weights for policy 1, policy_version 233459 (0.0010) [2023-12-26 17:03:39,449][105620] Updated weights for policy 1, policy_version 233469 (0.0010) [2023-12-26 17:03:39,511][105620] Updated weights for policy 1, policy_version 233479 (0.0011) [2023-12-26 17:03:39,552][105692] Updated weights for policy 0, policy_version 232947 (0.0010) [2023-12-26 17:03:39,613][105692] Updated weights for policy 0, policy_version 232957 (0.0008) [2023-12-26 17:03:39,670][105692] Updated weights for policy 0, policy_version 232967 (0.0008) [2023-12-26 17:03:40,225][105620] Updated weights for policy 1, policy_version 233489 (0.0008) [2023-12-26 17:03:40,294][105620] Updated weights for policy 1, policy_version 233499 (0.0006) [2023-12-26 17:03:40,359][105620] Updated weights for policy 1, policy_version 233509 (0.0007) [2023-12-26 17:03:40,474][105692] Updated weights for policy 0, policy_version 232977 (0.0008) [2023-12-26 17:03:40,534][105692] Updated weights for policy 0, policy_version 232987 (0.0005) [2023-12-26 17:03:40,598][105692] Updated weights for policy 0, policy_version 232997 (0.0008) [2023-12-26 17:03:40,654][105692] Updated weights for policy 0, policy_version 233007 (0.0010) [2023-12-26 17:03:40,946][105620] Updated weights for policy 1, policy_version 233519 (0.0010) [2023-12-26 17:03:41,004][105620] Updated weights for policy 1, policy_version 233529 (0.0010) [2023-12-26 17:03:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 119447552. Throughput: 0: 9826.0, 1: 9704.3. Samples: 119460980. Policy #0 lag: (min: 16.0, avg: 31.1, max: 32.0) [2023-12-26 17:03:41,063][104569] Avg episode reward: [(0, '9168.025'), (1, '6753.489')] [2023-12-26 17:03:41,070][105620] Updated weights for policy 1, policy_version 233539 (0.0010) [2023-12-26 17:03:41,405][105692] Updated weights for policy 0, policy_version 233017 (0.0008) [2023-12-26 17:03:41,457][105692] Updated weights for policy 0, policy_version 233027 (0.0008) [2023-12-26 17:03:41,504][105692] Updated weights for policy 0, policy_version 233037 (0.0008) [2023-12-26 17:03:41,886][105620] Updated weights for policy 1, policy_version 233549 (0.0008) [2023-12-26 17:03:41,940][105620] Updated weights for policy 1, policy_version 233559 (0.0008) [2023-12-26 17:03:41,998][105620] Updated weights for policy 1, policy_version 233569 (0.0009) [2023-12-26 17:03:42,320][105692] Updated weights for policy 0, policy_version 233047 (0.0009) [2023-12-26 17:03:42,387][105692] Updated weights for policy 0, policy_version 233057 (0.0009) [2023-12-26 17:03:42,457][105692] Updated weights for policy 0, policy_version 233067 (0.0007) [2023-12-26 17:03:42,711][105620] Updated weights for policy 1, policy_version 233579 (0.0008) [2023-12-26 17:03:42,779][105620] Updated weights for policy 1, policy_version 233589 (0.0005) [2023-12-26 17:03:42,847][105620] Updated weights for policy 1, policy_version 233599 (0.0005) [2023-12-26 17:03:43,070][105692] Updated weights for policy 0, policy_version 233077 (0.0009) [2023-12-26 17:03:43,118][105692] Updated weights for policy 0, policy_version 233087 (0.0009) [2023-12-26 17:03:43,165][105692] Updated weights for policy 0, policy_version 233097 (0.0009) [2023-12-26 17:03:43,435][105620] Updated weights for policy 1, policy_version 233609 (0.0005) [2023-12-26 17:03:43,482][105620] Updated weights for policy 1, policy_version 233619 (0.0006) [2023-12-26 17:03:43,530][105620] Updated weights for policy 1, policy_version 233629 (0.0005) [2023-12-26 17:03:43,590][105620] Updated weights for policy 1, policy_version 233639 (0.0005) [2023-12-26 17:03:44,050][105692] Updated weights for policy 0, policy_version 233107 (0.0009) [2023-12-26 17:03:44,097][105692] Updated weights for policy 0, policy_version 233117 (0.0009) [2023-12-26 17:03:44,162][105692] Updated weights for policy 0, policy_version 233127 (0.0006) [2023-12-26 17:03:44,164][105620] Updated weights for policy 1, policy_version 233649 (0.0008) [2023-12-26 17:03:44,220][105620] Updated weights for policy 1, policy_version 233659 (0.0007) [2023-12-26 17:03:44,276][105620] Updated weights for policy 1, policy_version 233669 (0.0009) [2023-12-26 17:03:44,813][105692] Updated weights for policy 0, policy_version 233137 (0.0006) [2023-12-26 17:03:44,863][105692] Updated weights for policy 0, policy_version 233147 (0.0008) [2023-12-26 17:03:44,922][105692] Updated weights for policy 0, policy_version 233157 (0.0009) [2023-12-26 17:03:44,986][105692] Updated weights for policy 0, policy_version 233167 (0.0009) [2023-12-26 17:03:45,065][105620] Updated weights for policy 1, policy_version 233679 (0.0007) [2023-12-26 17:03:45,124][105620] Updated weights for policy 1, policy_version 233689 (0.0009) [2023-12-26 17:03:45,184][105620] Updated weights for policy 1, policy_version 233699 (0.0011) [2023-12-26 17:03:45,790][105692] Updated weights for policy 0, policy_version 233177 (0.0008) [2023-12-26 17:03:45,849][105692] Updated weights for policy 0, policy_version 233187 (0.0008) [2023-12-26 17:03:45,892][105620] Updated weights for policy 1, policy_version 233709 (0.0008) [2023-12-26 17:03:45,904][105692] Updated weights for policy 0, policy_version 233197 (0.0009) [2023-12-26 17:03:45,954][105620] Updated weights for policy 1, policy_version 233719 (0.0008) [2023-12-26 17:03:46,012][105620] Updated weights for policy 1, policy_version 233729 (0.0010) [2023-12-26 17:03:46,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 119554048. Throughput: 0: 9744.8, 1: 9721.1. Samples: 119520124. Policy #0 lag: (min: 16.0, avg: 31.1, max: 32.0) [2023-12-26 17:03:46,063][104569] Avg episode reward: [(0, '9348.028'), (1, '8740.180')] [2023-12-26 17:03:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000233200_59711488.pth... [2023-12-26 17:03:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000233736_59842560.pth... [2023-12-26 17:03:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000232048_59416576.pth [2023-12-26 17:03:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000232584_59547648.pth [2023-12-26 17:03:46,628][105692] Updated weights for policy 0, policy_version 233207 (0.0008) [2023-12-26 17:03:46,684][105692] Updated weights for policy 0, policy_version 233217 (0.0008) [2023-12-26 17:03:46,713][105620] Updated weights for policy 1, policy_version 233739 (0.0009) [2023-12-26 17:03:46,734][105692] Updated weights for policy 0, policy_version 233228 (0.0009) [2023-12-26 17:03:46,762][105620] Updated weights for policy 1, policy_version 233749 (0.0006) [2023-12-26 17:03:46,808][105620] Updated weights for policy 1, policy_version 233759 (0.0008) [2023-12-26 17:03:47,426][105692] Updated weights for policy 0, policy_version 233238 (0.0007) [2023-12-26 17:03:47,500][105692] Updated weights for policy 0, policy_version 233248 (0.0009) [2023-12-26 17:03:47,514][105620] Updated weights for policy 1, policy_version 233769 (0.0008) [2023-12-26 17:03:47,562][105692] Updated weights for policy 0, policy_version 233258 (0.0007) [2023-12-26 17:03:47,572][105620] Updated weights for policy 1, policy_version 233779 (0.0009) [2023-12-26 17:03:47,619][105620] Updated weights for policy 1, policy_version 233789 (0.0010) [2023-12-26 17:03:47,671][105620] Updated weights for policy 1, policy_version 233799 (0.0010) [2023-12-26 17:03:48,296][105692] Updated weights for policy 0, policy_version 233268 (0.0009) [2023-12-26 17:03:48,359][105692] Updated weights for policy 0, policy_version 233278 (0.0011) [2023-12-26 17:03:48,423][105692] Updated weights for policy 0, policy_version 233288 (0.0011) [2023-12-26 17:03:48,464][105620] Updated weights for policy 1, policy_version 233809 (0.0010) [2023-12-26 17:03:48,523][105620] Updated weights for policy 1, policy_version 233819 (0.0008) [2023-12-26 17:03:48,575][105620] Updated weights for policy 1, policy_version 233829 (0.0010) [2023-12-26 17:03:49,182][105692] Updated weights for policy 0, policy_version 233298 (0.0011) [2023-12-26 17:03:49,182][105620] Updated weights for policy 1, policy_version 233839 (0.0011) [2023-12-26 17:03:49,245][105692] Updated weights for policy 0, policy_version 233308 (0.0010) [2023-12-26 17:03:49,248][105620] Updated weights for policy 1, policy_version 233849 (0.0010) [2023-12-26 17:03:49,306][105692] Updated weights for policy 0, policy_version 233318 (0.0011) [2023-12-26 17:03:49,307][105620] Updated weights for policy 1, policy_version 233859 (0.0009) [2023-12-26 17:03:49,372][105692] Updated weights for policy 0, policy_version 233328 (0.0012) [2023-12-26 17:03:50,048][105620] Updated weights for policy 1, policy_version 233869 (0.0007) [2023-12-26 17:03:50,115][105620] Updated weights for policy 1, policy_version 233879 (0.0010) [2023-12-26 17:03:50,129][105692] Updated weights for policy 0, policy_version 233338 (0.0008) [2023-12-26 17:03:50,176][105620] Updated weights for policy 1, policy_version 233889 (0.0011) [2023-12-26 17:03:50,192][105692] Updated weights for policy 0, policy_version 233348 (0.0008) [2023-12-26 17:03:50,261][105692] Updated weights for policy 0, policy_version 233358 (0.0008) [2023-12-26 17:03:50,892][105620] Updated weights for policy 1, policy_version 233899 (0.0011) [2023-12-26 17:03:50,948][105620] Updated weights for policy 1, policy_version 233909 (0.0011) [2023-12-26 17:03:51,003][105692] Updated weights for policy 0, policy_version 233368 (0.0010) [2023-12-26 17:03:51,007][105620] Updated weights for policy 1, policy_version 233919 (0.0007) [2023-12-26 17:03:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19438.7). Total num frames: 119635968. Throughput: 0: 9617.8, 1: 9724.5. Samples: 119635024. Policy #0 lag: (min: 16.0, avg: 31.1, max: 32.0) [2023-12-26 17:03:51,062][104569] Avg episode reward: [(0, '9172.004'), (1, '9091.141')] [2023-12-26 17:03:51,069][105692] Updated weights for policy 0, policy_version 233378 (0.0009) [2023-12-26 17:03:51,132][105692] Updated weights for policy 0, policy_version 233388 (0.0009) [2023-12-26 17:03:51,799][105620] Updated weights for policy 1, policy_version 233929 (0.0007) [2023-12-26 17:03:51,843][105692] Updated weights for policy 0, policy_version 233398 (0.0008) [2023-12-26 17:03:51,862][105620] Updated weights for policy 1, policy_version 233939 (0.0008) [2023-12-26 17:03:51,905][105692] Updated weights for policy 0, policy_version 233408 (0.0005) [2023-12-26 17:03:51,915][105620] Updated weights for policy 1, policy_version 233949 (0.0009) [2023-12-26 17:03:51,969][105620] Updated weights for policy 1, policy_version 233959 (0.0008) [2023-12-26 17:03:51,975][105692] Updated weights for policy 0, policy_version 233418 (0.0008) [2023-12-26 17:03:52,602][105692] Updated weights for policy 0, policy_version 233428 (0.0009) [2023-12-26 17:03:52,659][105692] Updated weights for policy 0, policy_version 233438 (0.0008) [2023-12-26 17:03:52,706][105692] Updated weights for policy 0, policy_version 233448 (0.0009) [2023-12-26 17:03:52,809][105620] Updated weights for policy 1, policy_version 233969 (0.0008) [2023-12-26 17:03:52,868][105620] Updated weights for policy 1, policy_version 233979 (0.0009) [2023-12-26 17:03:52,956][105620] Updated weights for policy 1, policy_version 233989 (0.0009) [2023-12-26 17:03:53,464][105692] Updated weights for policy 0, policy_version 233458 (0.0009) [2023-12-26 17:03:53,512][105692] Updated weights for policy 0, policy_version 233468 (0.0009) [2023-12-26 17:03:53,569][105692] Updated weights for policy 0, policy_version 233478 (0.0009) [2023-12-26 17:03:53,626][105692] Updated weights for policy 0, policy_version 233488 (0.0008) [2023-12-26 17:03:53,679][105620] Updated weights for policy 1, policy_version 233999 (0.0009) [2023-12-26 17:03:53,728][105620] Updated weights for policy 1, policy_version 234009 (0.0010) [2023-12-26 17:03:53,786][105620] Updated weights for policy 1, policy_version 234019 (0.0010) [2023-12-26 17:03:54,425][105692] Updated weights for policy 0, policy_version 233498 (0.0009) [2023-12-26 17:03:54,479][105692] Updated weights for policy 0, policy_version 233508 (0.0008) [2023-12-26 17:03:54,481][105620] Updated weights for policy 1, policy_version 234029 (0.0010) [2023-12-26 17:03:54,534][105620] Updated weights for policy 1, policy_version 234039 (0.0006) [2023-12-26 17:03:54,537][105692] Updated weights for policy 0, policy_version 233518 (0.0007) [2023-12-26 17:03:54,585][105620] Updated weights for policy 1, policy_version 234049 (0.0005) [2023-12-26 17:03:55,134][105620] Updated weights for policy 1, policy_version 234059 (0.0007) [2023-12-26 17:03:55,162][105586] KL-divergence is very high: 267.9768 [2023-12-26 17:03:55,185][105620] Updated weights for policy 1, policy_version 234069 (0.0010) [2023-12-26 17:03:55,202][105586] KL-divergence is very high: 495.1432 [2023-12-26 17:03:55,239][105620] Updated weights for policy 1, policy_version 234079 (0.0010) [2023-12-26 17:03:55,245][105586] KL-divergence is very high: 546.6788 [2023-12-26 17:03:55,300][105692] Updated weights for policy 0, policy_version 233528 (0.0006) [2023-12-26 17:03:55,356][105692] Updated weights for policy 0, policy_version 233538 (0.0007) [2023-12-26 17:03:55,413][105692] Updated weights for policy 0, policy_version 233548 (0.0008) [2023-12-26 17:03:55,945][105620] Updated weights for policy 1, policy_version 234089 (0.0010) [2023-12-26 17:03:56,010][105620] Updated weights for policy 1, policy_version 234099 (0.0009) [2023-12-26 17:03:56,062][104569] Fps is (10 sec: 18022.8, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 119734272. Throughput: 0: 9569.2, 1: 9774.9. Samples: 119749600. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 17:03:56,062][104569] Avg episode reward: [(0, '9172.243'), (1, '9087.899')] [2023-12-26 17:03:56,068][105620] Updated weights for policy 1, policy_version 234109 (0.0010) [2023-12-26 17:03:56,123][105620] Updated weights for policy 1, policy_version 234119 (0.0010) [2023-12-26 17:03:56,194][105692] Updated weights for policy 0, policy_version 233558 (0.0009) [2023-12-26 17:03:56,242][105692] Updated weights for policy 0, policy_version 233568 (0.0007) [2023-12-26 17:03:56,291][105692] Updated weights for policy 0, policy_version 233578 (0.0008) [2023-12-26 17:03:56,819][105620] Updated weights for policy 1, policy_version 234129 (0.0007) [2023-12-26 17:03:56,871][105620] Updated weights for policy 1, policy_version 234139 (0.0009) [2023-12-26 17:03:56,917][105620] Updated weights for policy 1, policy_version 234149 (0.0005) [2023-12-26 17:03:57,124][105692] Updated weights for policy 0, policy_version 233588 (0.0008) [2023-12-26 17:03:57,180][105692] Updated weights for policy 0, policy_version 233598 (0.0009) [2023-12-26 17:03:57,243][105692] Updated weights for policy 0, policy_version 233608 (0.0010) [2023-12-26 17:03:57,462][105620] Updated weights for policy 1, policy_version 234159 (0.0006) [2023-12-26 17:03:57,523][105620] Updated weights for policy 1, policy_version 234169 (0.0008) [2023-12-26 17:03:57,584][105620] Updated weights for policy 1, policy_version 234179 (0.0007) [2023-12-26 17:03:58,104][105692] Updated weights for policy 0, policy_version 233618 (0.0009) [2023-12-26 17:03:58,167][105692] Updated weights for policy 0, policy_version 233628 (0.0009) [2023-12-26 17:03:58,198][105620] Updated weights for policy 1, policy_version 234189 (0.0006) [2023-12-26 17:03:58,233][105692] Updated weights for policy 0, policy_version 233638 (0.0008) [2023-12-26 17:03:58,259][105620] Updated weights for policy 1, policy_version 234199 (0.0007) [2023-12-26 17:03:58,306][105692] Updated weights for policy 0, policy_version 233648 (0.0008) [2023-12-26 17:03:58,325][105620] Updated weights for policy 1, policy_version 234209 (0.0008) [2023-12-26 17:03:59,060][105692] Updated weights for policy 0, policy_version 233658 (0.0008) [2023-12-26 17:03:59,113][105620] Updated weights for policy 1, policy_version 234219 (0.0009) [2023-12-26 17:03:59,114][105692] Updated weights for policy 0, policy_version 233668 (0.0009) [2023-12-26 17:03:59,161][105620] Updated weights for policy 1, policy_version 234229 (0.0010) [2023-12-26 17:03:59,170][105692] Updated weights for policy 0, policy_version 233678 (0.0006) [2023-12-26 17:03:59,208][105620] Updated weights for policy 1, policy_version 234239 (0.0010) [2023-12-26 17:03:59,920][105692] Updated weights for policy 0, policy_version 233688 (0.0009) [2023-12-26 17:03:59,960][105620] Updated weights for policy 1, policy_version 234249 (0.0010) [2023-12-26 17:03:59,985][105692] Updated weights for policy 0, policy_version 233698 (0.0008) [2023-12-26 17:04:00,012][105620] Updated weights for policy 1, policy_version 234259 (0.0006) [2023-12-26 17:04:00,046][105692] Updated weights for policy 0, policy_version 233708 (0.0008) [2023-12-26 17:04:00,061][105620] Updated weights for policy 1, policy_version 234269 (0.0006) [2023-12-26 17:04:00,112][105620] Updated weights for policy 1, policy_version 234279 (0.0008) [2023-12-26 17:04:00,741][105692] Updated weights for policy 0, policy_version 233718 (0.0009) [2023-12-26 17:04:00,794][105692] Updated weights for policy 0, policy_version 233728 (0.0009) [2023-12-26 17:04:00,846][105620] Updated weights for policy 1, policy_version 234289 (0.0006) [2023-12-26 17:04:00,847][105692] Updated weights for policy 0, policy_version 233739 (0.0008) [2023-12-26 17:04:00,897][105620] Updated weights for policy 1, policy_version 234299 (0.0008) [2023-12-26 17:04:00,950][105620] Updated weights for policy 1, policy_version 234310 (0.0010) [2023-12-26 17:04:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 119840768. Throughput: 0: 9560.8, 1: 9797.7. Samples: 119806600. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 17:04:01,063][104569] Avg episode reward: [(0, '9349.361'), (1, '9094.202')] [2023-12-26 17:04:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000233744_59850752.pth... [2023-12-26 17:04:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000234312_59990016.pth... [2023-12-26 17:04:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000232624_59564032.pth [2023-12-26 17:04:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000233128_59686912.pth [2023-12-26 17:04:01,582][105692] Updated weights for policy 0, policy_version 233749 (0.0007) [2023-12-26 17:04:01,650][105692] Updated weights for policy 0, policy_version 233759 (0.0009) [2023-12-26 17:04:01,702][105692] Updated weights for policy 0, policy_version 233769 (0.0010) [2023-12-26 17:04:01,796][105620] Updated weights for policy 1, policy_version 234321 (0.0007) [2023-12-26 17:04:01,858][105620] Updated weights for policy 1, policy_version 234331 (0.0008) [2023-12-26 17:04:01,917][105620] Updated weights for policy 1, policy_version 234341 (0.0009) [2023-12-26 17:04:02,453][105692] Updated weights for policy 0, policy_version 233779 (0.0010) [2023-12-26 17:04:02,514][105692] Updated weights for policy 0, policy_version 233789 (0.0008) [2023-12-26 17:04:02,575][105692] Updated weights for policy 0, policy_version 233799 (0.0008) [2023-12-26 17:04:02,649][105620] Updated weights for policy 1, policy_version 234351 (0.0010) [2023-12-26 17:04:02,713][105620] Updated weights for policy 1, policy_version 234361 (0.0010) [2023-12-26 17:04:02,779][105620] Updated weights for policy 1, policy_version 234371 (0.0011) [2023-12-26 17:04:03,376][105692] Updated weights for policy 0, policy_version 233809 (0.0008) [2023-12-26 17:04:03,411][105620] Updated weights for policy 1, policy_version 234381 (0.0008) [2023-12-26 17:04:03,438][105692] Updated weights for policy 0, policy_version 233819 (0.0008) [2023-12-26 17:04:03,462][105620] Updated weights for policy 1, policy_version 234391 (0.0005) [2023-12-26 17:04:03,501][105692] Updated weights for policy 0, policy_version 233829 (0.0007) [2023-12-26 17:04:03,515][105620] Updated weights for policy 1, policy_version 234401 (0.0005) [2023-12-26 17:04:03,557][105692] Updated weights for policy 0, policy_version 233839 (0.0009) [2023-12-26 17:04:04,107][105620] Updated weights for policy 1, policy_version 234411 (0.0006) [2023-12-26 17:04:04,173][105620] Updated weights for policy 1, policy_version 234421 (0.0008) [2023-12-26 17:04:04,237][105620] Updated weights for policy 1, policy_version 234431 (0.0008) [2023-12-26 17:04:04,264][105692] Updated weights for policy 0, policy_version 233849 (0.0009) [2023-12-26 17:04:04,319][105692] Updated weights for policy 0, policy_version 233859 (0.0008) [2023-12-26 17:04:04,366][105692] Updated weights for policy 0, policy_version 233869 (0.0009) [2023-12-26 17:04:04,883][105620] Updated weights for policy 1, policy_version 234441 (0.0007) [2023-12-26 17:04:04,952][105620] Updated weights for policy 1, policy_version 234451 (0.0005) [2023-12-26 17:04:05,015][105620] Updated weights for policy 1, policy_version 234461 (0.0008) [2023-12-26 17:04:05,069][105620] Updated weights for policy 1, policy_version 234471 (0.0010) [2023-12-26 17:04:05,191][105692] Updated weights for policy 0, policy_version 233879 (0.0009) [2023-12-26 17:04:05,249][105692] Updated weights for policy 0, policy_version 233889 (0.0007) [2023-12-26 17:04:05,306][105692] Updated weights for policy 0, policy_version 233899 (0.0009) [2023-12-26 17:04:05,608][105620] Updated weights for policy 1, policy_version 234481 (0.0006) [2023-12-26 17:04:05,653][105620] Updated weights for policy 1, policy_version 234491 (0.0005) [2023-12-26 17:04:05,704][105620] Updated weights for policy 1, policy_version 234501 (0.0005) [2023-12-26 17:04:06,060][105692] Updated weights for policy 0, policy_version 233909 (0.0007) [2023-12-26 17:04:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 119930880. Throughput: 0: 9555.8, 1: 9840.5. Samples: 119922616. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 17:04:06,062][104569] Avg episode reward: [(0, '2334.293'), (1, '9093.915')] [2023-12-26 17:04:06,118][105692] Updated weights for policy 0, policy_version 233919 (0.0006) [2023-12-26 17:04:06,179][105692] Updated weights for policy 0, policy_version 233929 (0.0009) [2023-12-26 17:04:06,473][105620] Updated weights for policy 1, policy_version 234511 (0.0008) [2023-12-26 17:04:06,540][105620] Updated weights for policy 1, policy_version 234521 (0.0009) [2023-12-26 17:04:06,594][105620] Updated weights for policy 1, policy_version 234531 (0.0006) [2023-12-26 17:04:06,817][105692] Updated weights for policy 0, policy_version 233939 (0.0008) [2023-12-26 17:04:06,870][105692] Updated weights for policy 0, policy_version 233949 (0.0006) [2023-12-26 17:04:06,921][105692] Updated weights for policy 0, policy_version 233959 (0.0005) [2023-12-26 17:04:07,382][105620] Updated weights for policy 1, policy_version 234541 (0.0008) [2023-12-26 17:04:07,448][105620] Updated weights for policy 1, policy_version 234551 (0.0010) [2023-12-26 17:04:07,515][105620] Updated weights for policy 1, policy_version 234561 (0.0009) [2023-12-26 17:04:07,532][105692] Updated weights for policy 0, policy_version 233969 (0.0005) [2023-12-26 17:04:07,595][105692] Updated weights for policy 0, policy_version 233979 (0.0006) [2023-12-26 17:04:07,660][105692] Updated weights for policy 0, policy_version 233989 (0.0006) [2023-12-26 17:04:07,719][105692] Updated weights for policy 0, policy_version 233999 (0.0006) [2023-12-26 17:04:08,239][105620] Updated weights for policy 1, policy_version 234571 (0.0009) [2023-12-26 17:04:08,296][105620] Updated weights for policy 1, policy_version 234582 (0.0009) [2023-12-26 17:04:08,358][105620] Updated weights for policy 1, policy_version 234592 (0.0009) [2023-12-26 17:04:08,368][105692] Updated weights for policy 0, policy_version 234009 (0.0007) [2023-12-26 17:04:08,429][105692] Updated weights for policy 0, policy_version 234019 (0.0010) [2023-12-26 17:04:08,488][105692] Updated weights for policy 0, policy_version 234029 (0.0009) [2023-12-26 17:04:09,096][105620] Updated weights for policy 1, policy_version 234602 (0.0006) [2023-12-26 17:04:09,147][105620] Updated weights for policy 1, policy_version 234612 (0.0006) [2023-12-26 17:04:09,157][105692] Updated weights for policy 0, policy_version 234039 (0.0008) [2023-12-26 17:04:09,196][105620] Updated weights for policy 1, policy_version 234622 (0.0006) [2023-12-26 17:04:09,218][105692] Updated weights for policy 0, policy_version 234049 (0.0007) [2023-12-26 17:04:09,255][105620] Updated weights for policy 1, policy_version 234632 (0.0007) [2023-12-26 17:04:09,279][105692] Updated weights for policy 0, policy_version 234059 (0.0008) [2023-12-26 17:04:09,903][105620] Updated weights for policy 1, policy_version 234642 (0.0009) [2023-12-26 17:04:09,961][105620] Updated weights for policy 1, policy_version 234652 (0.0006) [2023-12-26 17:04:10,017][105620] Updated weights for policy 1, policy_version 234662 (0.0006) [2023-12-26 17:04:10,081][105692] Updated weights for policy 0, policy_version 234069 (0.0008) [2023-12-26 17:04:10,139][105692] Updated weights for policy 0, policy_version 234079 (0.0007) [2023-12-26 17:04:10,201][105692] Updated weights for policy 0, policy_version 234089 (0.0006) [2023-12-26 17:04:10,705][105620] Updated weights for policy 1, policy_version 234672 (0.0008) [2023-12-26 17:04:10,757][105620] Updated weights for policy 1, policy_version 234682 (0.0008) [2023-12-26 17:04:10,816][105620] Updated weights for policy 1, policy_version 234692 (0.0008) [2023-12-26 17:04:10,892][105692] Updated weights for policy 0, policy_version 234099 (0.0009) [2023-12-26 17:04:10,944][105692] Updated weights for policy 0, policy_version 234109 (0.0010) [2023-12-26 17:04:10,998][105692] Updated weights for policy 0, policy_version 234119 (0.0009) [2023-12-26 17:04:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 120037376. Throughput: 0: 9588.1, 1: 9904.8. Samples: 120041856. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 17:04:11,062][104569] Avg episode reward: [(0, '2148.388'), (1, '9181.747')] [2023-12-26 17:04:11,615][105620] Updated weights for policy 1, policy_version 234702 (0.0008) [2023-12-26 17:04:11,680][105620] Updated weights for policy 1, policy_version 234712 (0.0009) [2023-12-26 17:04:11,738][105620] Updated weights for policy 1, policy_version 234722 (0.0008) [2023-12-26 17:04:11,804][105692] Updated weights for policy 0, policy_version 234129 (0.0011) [2023-12-26 17:04:11,864][105692] Updated weights for policy 0, policy_version 234139 (0.0009) [2023-12-26 17:04:11,927][105692] Updated weights for policy 0, policy_version 234149 (0.0006) [2023-12-26 17:04:11,993][105692] Updated weights for policy 0, policy_version 234159 (0.0006) [2023-12-26 17:04:12,475][105620] Updated weights for policy 1, policy_version 234732 (0.0008) [2023-12-26 17:04:12,525][105620] Updated weights for policy 1, policy_version 234742 (0.0008) [2023-12-26 17:04:12,569][105620] Updated weights for policy 1, policy_version 234752 (0.0007) [2023-12-26 17:04:12,587][105692] Updated weights for policy 0, policy_version 234169 (0.0008) [2023-12-26 17:04:12,640][105692] Updated weights for policy 0, policy_version 234179 (0.0009) [2023-12-26 17:04:12,692][105692] Updated weights for policy 0, policy_version 234189 (0.0009) [2023-12-26 17:04:13,281][105692] Updated weights for policy 0, policy_version 234199 (0.0008) [2023-12-26 17:04:13,328][105692] Updated weights for policy 0, policy_version 234209 (0.0006) [2023-12-26 17:04:13,363][105620] Updated weights for policy 1, policy_version 234762 (0.0006) [2023-12-26 17:04:13,388][105692] Updated weights for policy 0, policy_version 234219 (0.0008) [2023-12-26 17:04:13,417][105620] Updated weights for policy 1, policy_version 234772 (0.0007) [2023-12-26 17:04:13,478][105620] Updated weights for policy 1, policy_version 234782 (0.0005) [2023-12-26 17:04:13,539][105620] Updated weights for policy 1, policy_version 234792 (0.0010) [2023-12-26 17:04:14,019][105692] Updated weights for policy 0, policy_version 234229 (0.0007) [2023-12-26 17:04:14,079][105692] Updated weights for policy 0, policy_version 234239 (0.0008) [2023-12-26 17:04:14,128][105692] Updated weights for policy 0, policy_version 234249 (0.0008) [2023-12-26 17:04:14,233][105620] Updated weights for policy 1, policy_version 234802 (0.0010) [2023-12-26 17:04:14,277][105620] Updated weights for policy 1, policy_version 234812 (0.0010) [2023-12-26 17:04:14,325][105620] Updated weights for policy 1, policy_version 234822 (0.0010) [2023-12-26 17:04:14,755][105692] Updated weights for policy 0, policy_version 234259 (0.0006) [2023-12-26 17:04:14,818][105692] Updated weights for policy 0, policy_version 234269 (0.0008) [2023-12-26 17:04:14,867][105692] Updated weights for policy 0, policy_version 234279 (0.0008) [2023-12-26 17:04:15,076][105620] Updated weights for policy 1, policy_version 234832 (0.0010) [2023-12-26 17:04:15,138][105620] Updated weights for policy 1, policy_version 234842 (0.0010) [2023-12-26 17:04:15,200][105620] Updated weights for policy 1, policy_version 234852 (0.0010) [2023-12-26 17:04:15,635][105692] Updated weights for policy 0, policy_version 234289 (0.0007) [2023-12-26 17:04:15,695][105692] Updated weights for policy 0, policy_version 234299 (0.0008) [2023-12-26 17:04:15,748][105692] Updated weights for policy 0, policy_version 234309 (0.0008) [2023-12-26 17:04:15,792][105692] Updated weights for policy 0, policy_version 234319 (0.0008) [2023-12-26 17:04:15,950][105620] Updated weights for policy 1, policy_version 234862 (0.0010) [2023-12-26 17:04:16,009][105620] Updated weights for policy 1, policy_version 234872 (0.0007) [2023-12-26 17:04:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 120127488. Throughput: 0: 9551.1, 1: 9932.9. Samples: 120100680. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 17:04:16,062][104569] Avg episode reward: [(0, '2905.876'), (1, '9178.584')] [2023-12-26 17:04:16,067][105620] Updated weights for policy 1, policy_version 234882 (0.0005) [2023-12-26 17:04:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000234320_59998208.pth... [2023-12-26 17:04:16,087][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000233200_59711488.pth [2023-12-26 17:04:16,110][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000234888_60137472.pth... [2023-12-26 17:04:16,114][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000233736_59842560.pth [2023-12-26 17:04:16,598][105620] Updated weights for policy 1, policy_version 234892 (0.0007) [2023-12-26 17:04:16,653][105620] Updated weights for policy 1, policy_version 234902 (0.0010) [2023-12-26 17:04:16,668][105692] Updated weights for policy 0, policy_version 234329 (0.0005) [2023-12-26 17:04:16,703][105620] Updated weights for policy 1, policy_version 234912 (0.0009) [2023-12-26 17:04:16,713][105692] Updated weights for policy 0, policy_version 234339 (0.0006) [2023-12-26 17:04:16,762][105692] Updated weights for policy 0, policy_version 234349 (0.0007) [2023-12-26 17:04:17,339][105620] Updated weights for policy 1, policy_version 234922 (0.0007) [2023-12-26 17:04:17,386][105620] Updated weights for policy 1, policy_version 234932 (0.0009) [2023-12-26 17:04:17,431][105620] Updated weights for policy 1, policy_version 234942 (0.0008) [2023-12-26 17:04:17,489][105620] Updated weights for policy 1, policy_version 234952 (0.0009) [2023-12-26 17:04:17,557][105692] Updated weights for policy 0, policy_version 234360 (0.0009) [2023-12-26 17:04:17,624][105692] Updated weights for policy 0, policy_version 234370 (0.0009) [2023-12-26 17:04:17,693][105692] Updated weights for policy 0, policy_version 234380 (0.0009) [2023-12-26 17:04:18,232][105620] Updated weights for policy 1, policy_version 234962 (0.0008) [2023-12-26 17:04:18,283][105620] Updated weights for policy 1, policy_version 234972 (0.0005) [2023-12-26 17:04:18,347][105620] Updated weights for policy 1, policy_version 234982 (0.0008) [2023-12-26 17:04:18,452][105692] Updated weights for policy 0, policy_version 234390 (0.0010) [2023-12-26 17:04:18,509][105692] Updated weights for policy 0, policy_version 234400 (0.0009) [2023-12-26 17:04:18,559][105692] Updated weights for policy 0, policy_version 234410 (0.0007) [2023-12-26 17:04:19,067][105620] Updated weights for policy 1, policy_version 234992 (0.0009) [2023-12-26 17:04:19,131][105620] Updated weights for policy 1, policy_version 235002 (0.0006) [2023-12-26 17:04:19,193][105620] Updated weights for policy 1, policy_version 235012 (0.0009) [2023-12-26 17:04:19,360][105692] Updated weights for policy 0, policy_version 234420 (0.0007) [2023-12-26 17:04:19,429][105692] Updated weights for policy 0, policy_version 234430 (0.0006) [2023-12-26 17:04:19,498][105692] Updated weights for policy 0, policy_version 234440 (0.0007) [2023-12-26 17:04:19,882][105620] Updated weights for policy 1, policy_version 235022 (0.0008) [2023-12-26 17:04:19,954][105620] Updated weights for policy 1, policy_version 235032 (0.0008) [2023-12-26 17:04:20,002][105620] Updated weights for policy 1, policy_version 235042 (0.0008) [2023-12-26 17:04:20,223][105692] Updated weights for policy 0, policy_version 234450 (0.0009) [2023-12-26 17:04:20,280][105692] Updated weights for policy 0, policy_version 234460 (0.0010) [2023-12-26 17:04:20,338][105692] Updated weights for policy 0, policy_version 234470 (0.0009) [2023-12-26 17:04:20,698][105620] Updated weights for policy 1, policy_version 235052 (0.0008) [2023-12-26 17:04:20,746][105620] Updated weights for policy 1, policy_version 235062 (0.0008) [2023-12-26 17:04:20,774][105586] KL-divergence is very high: 110.5964 [2023-12-26 17:04:20,785][105586] KL-divergence is very high: 109.6066 [2023-12-26 17:04:20,801][105620] Updated weights for policy 1, policy_version 235072 (0.0007) [2023-12-26 17:04:20,806][105586] KL-divergence is very high: 165.7539 [2023-12-26 17:04:20,812][105586] KL-divergence is very high: 185.4338 [2023-12-26 17:04:20,818][105586] KL-divergence is very high: 205.8142 [2023-12-26 17:04:20,829][105586] KL-divergence is very high: 168.3380 [2023-12-26 17:04:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 120225792. Throughput: 0: 9529.4, 1: 9957.9. Samples: 120217076. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 17:04:21,063][104569] Avg episode reward: [(0, '2345.801'), (1, '7599.509')] [2023-12-26 17:04:21,193][105692] Updated weights for policy 0, policy_version 234481 (0.0010) [2023-12-26 17:04:21,263][105692] Updated weights for policy 0, policy_version 234491 (0.0009) [2023-12-26 17:04:21,324][105692] Updated weights for policy 0, policy_version 234501 (0.0009) [2023-12-26 17:04:21,384][105692] Updated weights for policy 0, policy_version 234511 (0.0009) [2023-12-26 17:04:21,563][105620] Updated weights for policy 1, policy_version 235082 (0.0009) [2023-12-26 17:04:21,625][105620] Updated weights for policy 1, policy_version 235092 (0.0010) [2023-12-26 17:04:21,690][105620] Updated weights for policy 1, policy_version 235102 (0.0011) [2023-12-26 17:04:21,759][105620] Updated weights for policy 1, policy_version 235112 (0.0010) [2023-12-26 17:04:22,212][105692] Updated weights for policy 0, policy_version 234521 (0.0009) [2023-12-26 17:04:22,279][105692] Updated weights for policy 0, policy_version 234531 (0.0009) [2023-12-26 17:04:22,346][105692] Updated weights for policy 0, policy_version 234541 (0.0009) [2023-12-26 17:04:22,546][105620] Updated weights for policy 1, policy_version 235122 (0.0010) [2023-12-26 17:04:22,610][105620] Updated weights for policy 1, policy_version 235132 (0.0010) [2023-12-26 17:04:22,671][105620] Updated weights for policy 1, policy_version 235142 (0.0009) [2023-12-26 17:04:22,995][105692] Updated weights for policy 0, policy_version 234551 (0.0008) [2023-12-26 17:04:23,056][105692] Updated weights for policy 0, policy_version 234561 (0.0009) [2023-12-26 17:04:23,121][105692] Updated weights for policy 0, policy_version 234571 (0.0008) [2023-12-26 17:04:23,478][105620] Updated weights for policy 1, policy_version 235152 (0.0011) [2023-12-26 17:04:23,530][105620] Updated weights for policy 1, policy_version 235162 (0.0010) [2023-12-26 17:04:23,577][105620] Updated weights for policy 1, policy_version 235172 (0.0009) [2023-12-26 17:04:23,888][105692] Updated weights for policy 0, policy_version 234581 (0.0009) [2023-12-26 17:04:23,948][105692] Updated weights for policy 0, policy_version 234591 (0.0009) [2023-12-26 17:04:24,005][105692] Updated weights for policy 0, policy_version 234601 (0.0010) [2023-12-26 17:04:24,251][105620] Updated weights for policy 1, policy_version 235182 (0.0009) [2023-12-26 17:04:24,301][105620] Updated weights for policy 1, policy_version 235192 (0.0008) [2023-12-26 17:04:24,357][105620] Updated weights for policy 1, policy_version 235202 (0.0009) [2023-12-26 17:04:24,727][105692] Updated weights for policy 0, policy_version 234611 (0.0008) [2023-12-26 17:04:24,774][105692] Updated weights for policy 0, policy_version 234621 (0.0005) [2023-12-26 17:04:24,826][105692] Updated weights for policy 0, policy_version 234631 (0.0008) [2023-12-26 17:04:25,079][105620] Updated weights for policy 1, policy_version 235212 (0.0008) [2023-12-26 17:04:25,130][105620] Updated weights for policy 1, policy_version 235222 (0.0005) [2023-12-26 17:04:25,188][105620] Updated weights for policy 1, policy_version 235232 (0.0009) [2023-12-26 17:04:25,462][105692] Updated weights for policy 0, policy_version 234641 (0.0006) [2023-12-26 17:04:25,510][105692] Updated weights for policy 0, policy_version 234651 (0.0009) [2023-12-26 17:04:25,560][105692] Updated weights for policy 0, policy_version 234661 (0.0006) [2023-12-26 17:04:25,611][105692] Updated weights for policy 0, policy_version 234671 (0.0007) [2023-12-26 17:04:25,898][105620] Updated weights for policy 1, policy_version 235242 (0.0008) [2023-12-26 17:04:25,960][105620] Updated weights for policy 1, policy_version 235252 (0.0010) [2023-12-26 17:04:26,014][105620] Updated weights for policy 1, policy_version 235262 (0.0010) [2023-12-26 17:04:26,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 120315904. Throughput: 0: 9459.3, 1: 9849.8. Samples: 120329888. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 17:04:26,063][104569] Avg episode reward: [(0, '4879.872'), (1, '7506.819')] [2023-12-26 17:04:26,077][105620] Updated weights for policy 1, policy_version 235272 (0.0011) [2023-12-26 17:04:26,366][105692] Updated weights for policy 0, policy_version 234681 (0.0008) [2023-12-26 17:04:26,422][105692] Updated weights for policy 0, policy_version 234691 (0.0008) [2023-12-26 17:04:26,481][105692] Updated weights for policy 0, policy_version 234701 (0.0008) [2023-12-26 17:04:26,714][105620] Updated weights for policy 1, policy_version 235282 (0.0006) [2023-12-26 17:04:26,767][105620] Updated weights for policy 1, policy_version 235292 (0.0005) [2023-12-26 17:04:26,825][105620] Updated weights for policy 1, policy_version 235302 (0.0010) [2023-12-26 17:04:27,233][105692] Updated weights for policy 0, policy_version 234711 (0.0009) [2023-12-26 17:04:27,285][105692] Updated weights for policy 0, policy_version 234722 (0.0010) [2023-12-26 17:04:27,350][105692] Updated weights for policy 0, policy_version 234732 (0.0009) [2023-12-26 17:04:27,473][105620] Updated weights for policy 1, policy_version 235312 (0.0006) [2023-12-26 17:04:27,527][105620] Updated weights for policy 1, policy_version 235322 (0.0005) [2023-12-26 17:04:27,574][105620] Updated weights for policy 1, policy_version 235332 (0.0005) [2023-12-26 17:04:28,089][105620] Updated weights for policy 1, policy_version 235342 (0.0005) [2023-12-26 17:04:28,145][105620] Updated weights for policy 1, policy_version 235352 (0.0005) [2023-12-26 17:04:28,166][105692] Updated weights for policy 0, policy_version 234742 (0.0010) [2023-12-26 17:04:28,196][105620] Updated weights for policy 1, policy_version 235362 (0.0005) [2023-12-26 17:04:28,218][105692] Updated weights for policy 0, policy_version 234752 (0.0008) [2023-12-26 17:04:28,265][105692] Updated weights for policy 0, policy_version 234762 (0.0007) [2023-12-26 17:04:28,883][105620] Updated weights for policy 1, policy_version 235372 (0.0007) [2023-12-26 17:04:28,938][105692] Updated weights for policy 0, policy_version 234772 (0.0007) [2023-12-26 17:04:28,947][105620] Updated weights for policy 1, policy_version 235382 (0.0010) [2023-12-26 17:04:29,001][105692] Updated weights for policy 0, policy_version 234782 (0.0006) [2023-12-26 17:04:29,003][105620] Updated weights for policy 1, policy_version 235392 (0.0007) [2023-12-26 17:04:29,054][105692] Updated weights for policy 0, policy_version 234792 (0.0006) [2023-12-26 17:04:29,727][105620] Updated weights for policy 1, policy_version 235402 (0.0006) [2023-12-26 17:04:29,729][105692] Updated weights for policy 0, policy_version 234802 (0.0009) [2023-12-26 17:04:29,781][105692] Updated weights for policy 0, policy_version 234812 (0.0010) [2023-12-26 17:04:29,783][105620] Updated weights for policy 1, policy_version 235412 (0.0005) [2023-12-26 17:04:29,837][105692] Updated weights for policy 0, policy_version 234822 (0.0009) [2023-12-26 17:04:29,847][105620] Updated weights for policy 1, policy_version 235422 (0.0008) [2023-12-26 17:04:29,901][105620] Updated weights for policy 1, policy_version 235432 (0.0006) [2023-12-26 17:04:29,902][105692] Updated weights for policy 0, policy_version 234832 (0.0010) [2023-12-26 17:04:30,659][105692] Updated weights for policy 0, policy_version 234842 (0.0010) [2023-12-26 17:04:30,690][105620] Updated weights for policy 1, policy_version 235442 (0.0006) [2023-12-26 17:04:30,711][105692] Updated weights for policy 0, policy_version 234852 (0.0007) [2023-12-26 17:04:30,747][105620] Updated weights for policy 1, policy_version 235452 (0.0006) [2023-12-26 17:04:30,769][105692] Updated weights for policy 0, policy_version 234862 (0.0007) [2023-12-26 17:04:30,801][105620] Updated weights for policy 1, policy_version 235462 (0.0008) [2023-12-26 17:04:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 120422400. Throughput: 0: 9452.8, 1: 9886.8. Samples: 120390404. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 17:04:31,062][104569] Avg episode reward: [(0, '7426.700'), (1, '8043.549')] [2023-12-26 17:04:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000234864_60137472.pth... [2023-12-26 17:04:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000235464_60284928.pth... [2023-12-26 17:04:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000233744_59850752.pth [2023-12-26 17:04:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000234312_59990016.pth [2023-12-26 17:04:31,529][105692] Updated weights for policy 0, policy_version 234872 (0.0008) [2023-12-26 17:04:31,535][105620] Updated weights for policy 1, policy_version 235472 (0.0006) [2023-12-26 17:04:31,586][105620] Updated weights for policy 1, policy_version 235482 (0.0005) [2023-12-26 17:04:31,586][105692] Updated weights for policy 0, policy_version 234882 (0.0008) [2023-12-26 17:04:31,644][105620] Updated weights for policy 1, policy_version 235492 (0.0007) [2023-12-26 17:04:31,646][105692] Updated weights for policy 0, policy_version 234892 (0.0008) [2023-12-26 17:04:32,384][105692] Updated weights for policy 0, policy_version 234902 (0.0008) [2023-12-26 17:04:32,414][105620] Updated weights for policy 1, policy_version 235502 (0.0008) [2023-12-26 17:04:32,446][105692] Updated weights for policy 0, policy_version 234912 (0.0007) [2023-12-26 17:04:32,475][105620] Updated weights for policy 1, policy_version 235512 (0.0006) [2023-12-26 17:04:32,506][105692] Updated weights for policy 0, policy_version 234922 (0.0007) [2023-12-26 17:04:32,539][105620] Updated weights for policy 1, policy_version 235522 (0.0006) [2023-12-26 17:04:33,236][105692] Updated weights for policy 0, policy_version 234932 (0.0006) [2023-12-26 17:04:33,253][105620] Updated weights for policy 1, policy_version 235532 (0.0007) [2023-12-26 17:04:33,291][105692] Updated weights for policy 0, policy_version 234942 (0.0005) [2023-12-26 17:04:33,310][105620] Updated weights for policy 1, policy_version 235542 (0.0008) [2023-12-26 17:04:33,350][105692] Updated weights for policy 0, policy_version 234952 (0.0005) [2023-12-26 17:04:33,367][105620] Updated weights for policy 1, policy_version 235552 (0.0009) [2023-12-26 17:04:33,963][105692] Updated weights for policy 0, policy_version 234962 (0.0006) [2023-12-26 17:04:34,022][105692] Updated weights for policy 0, policy_version 234972 (0.0009) [2023-12-26 17:04:34,075][105692] Updated weights for policy 0, policy_version 234982 (0.0008) [2023-12-26 17:04:34,091][105620] Updated weights for policy 1, policy_version 235562 (0.0008) [2023-12-26 17:04:34,122][105692] Updated weights for policy 0, policy_version 234992 (0.0007) [2023-12-26 17:04:34,144][105620] Updated weights for policy 1, policy_version 235572 (0.0008) [2023-12-26 17:04:34,210][105620] Updated weights for policy 1, policy_version 235582 (0.0008) [2023-12-26 17:04:34,264][105620] Updated weights for policy 1, policy_version 235592 (0.0009) [2023-12-26 17:04:34,924][105692] Updated weights for policy 0, policy_version 235002 (0.0009) [2023-12-26 17:04:34,975][105692] Updated weights for policy 0, policy_version 235012 (0.0008) [2023-12-26 17:04:34,989][105620] Updated weights for policy 1, policy_version 235602 (0.0008) [2023-12-26 17:04:35,032][105692] Updated weights for policy 0, policy_version 235022 (0.0007) [2023-12-26 17:04:35,046][105620] Updated weights for policy 1, policy_version 235612 (0.0007) [2023-12-26 17:04:35,107][105620] Updated weights for policy 1, policy_version 235622 (0.0008) [2023-12-26 17:04:35,704][105692] Updated weights for policy 0, policy_version 235032 (0.0005) [2023-12-26 17:04:35,752][105692] Updated weights for policy 0, policy_version 235042 (0.0008) [2023-12-26 17:04:35,803][105692] Updated weights for policy 0, policy_version 235052 (0.0009) [2023-12-26 17:04:35,909][105620] Updated weights for policy 1, policy_version 235632 (0.0010) [2023-12-26 17:04:35,981][105620] Updated weights for policy 1, policy_version 235642 (0.0010) [2023-12-26 17:04:36,051][105620] Updated weights for policy 1, policy_version 235652 (0.0010) [2023-12-26 17:04:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 120512512. Throughput: 0: 9503.2, 1: 9850.0. Samples: 120505920. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 17:04:36,062][104569] Avg episode reward: [(0, '8739.396'), (1, '8126.658')] [2023-12-26 17:04:36,494][105692] Updated weights for policy 0, policy_version 235062 (0.0012) [2023-12-26 17:04:36,550][105692] Updated weights for policy 0, policy_version 235072 (0.0011) [2023-12-26 17:04:36,599][105692] Updated weights for policy 0, policy_version 235082 (0.0010) [2023-12-26 17:04:36,821][105620] Updated weights for policy 1, policy_version 235662 (0.0008) [2023-12-26 17:04:36,879][105620] Updated weights for policy 1, policy_version 235672 (0.0009) [2023-12-26 17:04:36,939][105620] Updated weights for policy 1, policy_version 235682 (0.0010) [2023-12-26 17:04:37,358][105692] Updated weights for policy 0, policy_version 235092 (0.0011) [2023-12-26 17:04:37,427][105692] Updated weights for policy 0, policy_version 235102 (0.0011) [2023-12-26 17:04:37,476][105692] Updated weights for policy 0, policy_version 235112 (0.0011) [2023-12-26 17:04:37,652][105620] Updated weights for policy 1, policy_version 235692 (0.0010) [2023-12-26 17:04:37,719][105620] Updated weights for policy 1, policy_version 235702 (0.0010) [2023-12-26 17:04:37,775][105620] Updated weights for policy 1, policy_version 235712 (0.0010) [2023-12-26 17:04:38,152][105692] Updated weights for policy 0, policy_version 235122 (0.0009) [2023-12-26 17:04:38,205][105692] Updated weights for policy 0, policy_version 235132 (0.0009) [2023-12-26 17:04:38,258][105692] Updated weights for policy 0, policy_version 235142 (0.0009) [2023-12-26 17:04:38,319][105692] Updated weights for policy 0, policy_version 235152 (0.0009) [2023-12-26 17:04:38,387][105620] Updated weights for policy 1, policy_version 235722 (0.0010) [2023-12-26 17:04:38,445][105620] Updated weights for policy 1, policy_version 235732 (0.0008) [2023-12-26 17:04:38,507][105620] Updated weights for policy 1, policy_version 235742 (0.0010) [2023-12-26 17:04:38,566][105620] Updated weights for policy 1, policy_version 235752 (0.0010) [2023-12-26 17:04:39,147][105692] Updated weights for policy 0, policy_version 235162 (0.0008) [2023-12-26 17:04:39,194][105692] Updated weights for policy 0, policy_version 235172 (0.0007) [2023-12-26 17:04:39,230][105620] Updated weights for policy 1, policy_version 235762 (0.0011) [2023-12-26 17:04:39,254][105692] Updated weights for policy 0, policy_version 235182 (0.0009) [2023-12-26 17:04:39,291][105620] Updated weights for policy 1, policy_version 235772 (0.0010) [2023-12-26 17:04:39,359][105620] Updated weights for policy 1, policy_version 235782 (0.0006) [2023-12-26 17:04:40,011][105620] Updated weights for policy 1, policy_version 235792 (0.0008) [2023-12-26 17:04:40,072][105620] Updated weights for policy 1, policy_version 235802 (0.0006) [2023-12-26 17:04:40,109][105692] Updated weights for policy 0, policy_version 235192 (0.0008) [2023-12-26 17:04:40,128][105620] Updated weights for policy 1, policy_version 235812 (0.0005) [2023-12-26 17:04:40,170][105692] Updated weights for policy 0, policy_version 235202 (0.0009) [2023-12-26 17:04:40,227][105692] Updated weights for policy 0, policy_version 235212 (0.0010) [2023-12-26 17:04:40,796][105620] Updated weights for policy 1, policy_version 235822 (0.0006) [2023-12-26 17:04:40,851][105620] Updated weights for policy 1, policy_version 235832 (0.0010) [2023-12-26 17:04:40,900][105620] Updated weights for policy 1, policy_version 235842 (0.0010) [2023-12-26 17:04:41,028][105692] Updated weights for policy 0, policy_version 235222 (0.0008) [2023-12-26 17:04:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 120610816. Throughput: 0: 9500.9, 1: 9865.0. Samples: 120621064. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 17:04:41,063][104569] Avg episode reward: [(0, '8625.246'), (1, '7584.877')] [2023-12-26 17:04:41,080][105692] Updated weights for policy 0, policy_version 235232 (0.0008) [2023-12-26 17:04:41,153][105692] Updated weights for policy 0, policy_version 235242 (0.0008) [2023-12-26 17:04:41,628][105620] Updated weights for policy 1, policy_version 235852 (0.0009) [2023-12-26 17:04:41,695][105620] Updated weights for policy 1, policy_version 235862 (0.0009) [2023-12-26 17:04:41,761][105620] Updated weights for policy 1, policy_version 235872 (0.0008) [2023-12-26 17:04:41,960][105692] Updated weights for policy 0, policy_version 235252 (0.0008) [2023-12-26 17:04:42,013][105692] Updated weights for policy 0, policy_version 235262 (0.0008) [2023-12-26 17:04:42,064][105692] Updated weights for policy 0, policy_version 235272 (0.0010) [2023-12-26 17:04:42,540][105620] Updated weights for policy 1, policy_version 235882 (0.0009) [2023-12-26 17:04:42,588][105620] Updated weights for policy 1, policy_version 235892 (0.0009) [2023-12-26 17:04:42,635][105620] Updated weights for policy 1, policy_version 235902 (0.0009) [2023-12-26 17:04:42,689][105620] Updated weights for policy 1, policy_version 235912 (0.0009) [2023-12-26 17:04:42,791][105692] Updated weights for policy 0, policy_version 235282 (0.0009) [2023-12-26 17:04:42,850][105692] Updated weights for policy 0, policy_version 235292 (0.0009) [2023-12-26 17:04:42,913][105692] Updated weights for policy 0, policy_version 235302 (0.0009) [2023-12-26 17:04:42,982][105692] Updated weights for policy 0, policy_version 235312 (0.0009) [2023-12-26 17:04:43,473][105620] Updated weights for policy 1, policy_version 235922 (0.0009) [2023-12-26 17:04:43,531][105620] Updated weights for policy 1, policy_version 235933 (0.0010) [2023-12-26 17:04:43,581][105620] Updated weights for policy 1, policy_version 235943 (0.0009) [2023-12-26 17:04:43,687][105692] Updated weights for policy 0, policy_version 235322 (0.0007) [2023-12-26 17:04:43,738][105692] Updated weights for policy 0, policy_version 235332 (0.0009) [2023-12-26 17:04:43,784][105692] Updated weights for policy 0, policy_version 235342 (0.0008) [2023-12-26 17:04:44,373][105620] Updated weights for policy 1, policy_version 235953 (0.0008) [2023-12-26 17:04:44,434][105620] Updated weights for policy 1, policy_version 235963 (0.0008) [2023-12-26 17:04:44,482][105620] Updated weights for policy 1, policy_version 235973 (0.0007) [2023-12-26 17:04:44,526][105692] Updated weights for policy 0, policy_version 235352 (0.0010) [2023-12-26 17:04:44,591][105692] Updated weights for policy 0, policy_version 235362 (0.0010) [2023-12-26 17:04:44,655][105692] Updated weights for policy 0, policy_version 235372 (0.0010) [2023-12-26 17:04:45,288][105692] Updated weights for policy 0, policy_version 235382 (0.0010) [2023-12-26 17:04:45,290][105620] Updated weights for policy 1, policy_version 235983 (0.0009) [2023-12-26 17:04:45,349][105620] Updated weights for policy 1, policy_version 235993 (0.0005) [2023-12-26 17:04:45,351][105692] Updated weights for policy 0, policy_version 235392 (0.0011) [2023-12-26 17:04:45,403][105692] Updated weights for policy 0, policy_version 235402 (0.0011) [2023-12-26 17:04:45,405][105620] Updated weights for policy 1, policy_version 236003 (0.0006) [2023-12-26 17:04:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 120700928. Throughput: 0: 9540.3, 1: 9785.6. Samples: 120676264. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 17:04:46,063][104569] Avg episode reward: [(0, '7703.276'), (1, '8211.995')] [2023-12-26 17:04:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000236008_60424192.pth... [2023-12-26 17:04:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000235408_60276736.pth... [2023-12-26 17:04:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000234888_60137472.pth [2023-12-26 17:04:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000234320_59998208.pth [2023-12-26 17:04:46,118][105692] Updated weights for policy 0, policy_version 235412 (0.0008) [2023-12-26 17:04:46,129][105620] Updated weights for policy 1, policy_version 236013 (0.0007) [2023-12-26 17:04:46,162][105692] Updated weights for policy 0, policy_version 235422 (0.0005) [2023-12-26 17:04:46,177][105620] Updated weights for policy 1, policy_version 236023 (0.0009) [2023-12-26 17:04:46,214][105692] Updated weights for policy 0, policy_version 235432 (0.0005) [2023-12-26 17:04:46,223][105620] Updated weights for policy 1, policy_version 236033 (0.0009) [2023-12-26 17:04:46,849][105692] Updated weights for policy 0, policy_version 235442 (0.0006) [2023-12-26 17:04:46,897][105620] Updated weights for policy 1, policy_version 236043 (0.0008) [2023-12-26 17:04:46,916][105692] Updated weights for policy 0, policy_version 235452 (0.0007) [2023-12-26 17:04:46,957][105620] Updated weights for policy 1, policy_version 236053 (0.0008) [2023-12-26 17:04:46,977][105692] Updated weights for policy 0, policy_version 235462 (0.0010) [2023-12-26 17:04:47,016][105620] Updated weights for policy 1, policy_version 236063 (0.0010) [2023-12-26 17:04:47,034][105692] Updated weights for policy 0, policy_version 235472 (0.0010) [2023-12-26 17:04:47,727][105620] Updated weights for policy 1, policy_version 236073 (0.0010) [2023-12-26 17:04:47,743][105692] Updated weights for policy 0, policy_version 235482 (0.0010) [2023-12-26 17:04:47,777][105620] Updated weights for policy 1, policy_version 236083 (0.0005) [2023-12-26 17:04:47,794][105692] Updated weights for policy 0, policy_version 235492 (0.0010) [2023-12-26 17:04:47,840][105620] Updated weights for policy 1, policy_version 236093 (0.0006) [2023-12-26 17:04:47,850][105692] Updated weights for policy 0, policy_version 235502 (0.0011) [2023-12-26 17:04:47,900][105620] Updated weights for policy 1, policy_version 236103 (0.0008) [2023-12-26 17:04:48,655][105585] KL-divergence is very high: 136.2580 [2023-12-26 17:04:48,660][105620] Updated weights for policy 1, policy_version 236113 (0.0010) [2023-12-26 17:04:48,661][105585] KL-divergence is very high: 127.4232 [2023-12-26 17:04:48,662][105692] Updated weights for policy 0, policy_version 235512 (0.0008) [2023-12-26 17:04:48,667][105585] KL-divergence is very high: 174.0705 [2023-12-26 17:04:48,673][105585] KL-divergence is very high: 167.2642 [2023-12-26 17:04:48,685][105585] KL-divergence is very high: 209.1633 [2023-12-26 17:04:48,702][105585] KL-divergence is very high: 217.5024 [2023-12-26 17:04:48,710][105585] KL-divergence is very high: 178.9548 [2023-12-26 17:04:48,712][105620] Updated weights for policy 1, policy_version 236123 (0.0010) [2023-12-26 17:04:48,716][105585] KL-divergence is very high: 215.9410 [2023-12-26 17:04:48,722][105692] Updated weights for policy 0, policy_version 235522 (0.0006) [2023-12-26 17:04:48,725][105585] KL-divergence is very high: 187.9473 [2023-12-26 17:04:48,737][105585] KL-divergence is very high: 199.7057 [2023-12-26 17:04:48,753][105585] KL-divergence is very high: 169.7826 [2023-12-26 17:04:48,760][105585] KL-divergence is very high: 130.2245 [2023-12-26 17:04:48,766][105585] KL-divergence is very high: 152.8106 [2023-12-26 17:04:48,772][105585] KL-divergence is very high: 124.3051 [2023-12-26 17:04:48,772][105620] Updated weights for policy 1, policy_version 236133 (0.0011) [2023-12-26 17:04:48,784][105585] KL-divergence is very high: 122.5085 [2023-12-26 17:04:48,785][105692] Updated weights for policy 0, policy_version 235532 (0.0005) [2023-12-26 17:04:49,440][105620] Updated weights for policy 1, policy_version 236143 (0.0009) [2023-12-26 17:04:49,491][105620] Updated weights for policy 1, policy_version 236153 (0.0008) [2023-12-26 17:04:49,542][105620] Updated weights for policy 1, policy_version 236163 (0.0008) [2023-12-26 17:04:49,551][105692] Updated weights for policy 0, policy_version 235542 (0.0007) [2023-12-26 17:04:49,610][105692] Updated weights for policy 0, policy_version 235552 (0.0008) [2023-12-26 17:04:49,656][105692] Updated weights for policy 0, policy_version 235562 (0.0008) [2023-12-26 17:04:50,281][105620] Updated weights for policy 1, policy_version 236173 (0.0008) [2023-12-26 17:04:50,331][105620] Updated weights for policy 1, policy_version 236183 (0.0008) [2023-12-26 17:04:50,391][105620] Updated weights for policy 1, policy_version 236193 (0.0007) [2023-12-26 17:04:50,443][105692] Updated weights for policy 0, policy_version 235572 (0.0008) [2023-12-26 17:04:50,505][105692] Updated weights for policy 0, policy_version 235582 (0.0010) [2023-12-26 17:04:50,560][105692] Updated weights for policy 0, policy_version 235592 (0.0010) [2023-12-26 17:04:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 120799232. Throughput: 0: 9585.7, 1: 9758.7. Samples: 120793116. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 17:04:51,063][104569] Avg episode reward: [(0, '7187.756'), (1, '8480.059')] [2023-12-26 17:04:51,063][105620] Updated weights for policy 1, policy_version 236203 (0.0009) [2023-12-26 17:04:51,122][105620] Updated weights for policy 1, policy_version 236213 (0.0009) [2023-12-26 17:04:51,183][105620] Updated weights for policy 1, policy_version 236223 (0.0006) [2023-12-26 17:04:51,306][105692] Updated weights for policy 0, policy_version 235602 (0.0009) [2023-12-26 17:04:51,368][105692] Updated weights for policy 0, policy_version 235612 (0.0009) [2023-12-26 17:04:51,434][105692] Updated weights for policy 0, policy_version 235622 (0.0009) [2023-12-26 17:04:51,498][105692] Updated weights for policy 0, policy_version 235632 (0.0009) [2023-12-26 17:04:51,941][105620] Updated weights for policy 1, policy_version 236233 (0.0009) [2023-12-26 17:04:52,003][105620] Updated weights for policy 1, policy_version 236243 (0.0009) [2023-12-26 17:04:52,051][105586] KL-divergence is very high: 193.2075 [2023-12-26 17:04:52,065][105620] Updated weights for policy 1, policy_version 236253 (0.0009) [2023-12-26 17:04:52,076][105586] KL-divergence is very high: 254.7685 [2023-12-26 17:04:52,102][105586] KL-divergence is very high: 279.4997 [2023-12-26 17:04:52,125][105620] Updated weights for policy 1, policy_version 236263 (0.0009) [2023-12-26 17:04:52,127][105586] KL-divergence is very high: 272.0497 [2023-12-26 17:04:52,247][105692] Updated weights for policy 0, policy_version 235642 (0.0010) [2023-12-26 17:04:52,309][105692] Updated weights for policy 0, policy_version 235652 (0.0009) [2023-12-26 17:04:52,364][105692] Updated weights for policy 0, policy_version 235662 (0.0009) [2023-12-26 17:04:52,913][105620] Updated weights for policy 1, policy_version 236273 (0.0008) [2023-12-26 17:04:52,976][105620] Updated weights for policy 1, policy_version 236283 (0.0008) [2023-12-26 17:04:53,013][105586] KL-divergence is very high: 107.6696 [2023-12-26 17:04:53,032][105620] Updated weights for policy 1, policy_version 236293 (0.0008) [2023-12-26 17:04:53,078][105692] Updated weights for policy 0, policy_version 235672 (0.0008) [2023-12-26 17:04:53,137][105692] Updated weights for policy 0, policy_version 235682 (0.0009) [2023-12-26 17:04:53,201][105692] Updated weights for policy 0, policy_version 235692 (0.0009) [2023-12-26 17:04:53,812][105620] Updated weights for policy 1, policy_version 236303 (0.0009) [2023-12-26 17:04:53,867][105620] Updated weights for policy 1, policy_version 236314 (0.0009) [2023-12-26 17:04:53,918][105620] Updated weights for policy 1, policy_version 236324 (0.0009) [2023-12-26 17:04:53,941][105692] Updated weights for policy 0, policy_version 235702 (0.0008) [2023-12-26 17:04:53,987][105692] Updated weights for policy 0, policy_version 235712 (0.0005) [2023-12-26 17:04:54,033][105692] Updated weights for policy 0, policy_version 235722 (0.0005) [2023-12-26 17:04:54,625][105692] Updated weights for policy 0, policy_version 235732 (0.0007) [2023-12-26 17:04:54,676][105692] Updated weights for policy 0, policy_version 235742 (0.0010) [2023-12-26 17:04:54,727][105692] Updated weights for policy 0, policy_version 235752 (0.0010) [2023-12-26 17:04:54,779][105620] Updated weights for policy 1, policy_version 236334 (0.0009) [2023-12-26 17:04:54,832][105620] Updated weights for policy 1, policy_version 236344 (0.0008) [2023-12-26 17:04:54,890][105620] Updated weights for policy 1, policy_version 236354 (0.0008) [2023-12-26 17:04:55,476][105692] Updated weights for policy 0, policy_version 235762 (0.0007) [2023-12-26 17:04:55,537][105692] Updated weights for policy 0, policy_version 235772 (0.0010) [2023-12-26 17:04:55,592][105692] Updated weights for policy 0, policy_version 235782 (0.0010) [2023-12-26 17:04:55,607][105620] Updated weights for policy 1, policy_version 236364 (0.0007) [2023-12-26 17:04:55,654][105692] Updated weights for policy 0, policy_version 235792 (0.0010) [2023-12-26 17:04:55,657][105620] Updated weights for policy 1, policy_version 236374 (0.0009) [2023-12-26 17:04:55,712][105620] Updated weights for policy 1, policy_version 236384 (0.0010) [2023-12-26 17:04:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 120897536. Throughput: 0: 9549.8, 1: 9670.0. Samples: 120906748. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 17:04:56,063][104569] Avg episode reward: [(0, '8380.197'), (1, '7427.412')] [2023-12-26 17:04:56,298][105692] Updated weights for policy 0, policy_version 235802 (0.0005) [2023-12-26 17:04:56,344][105692] Updated weights for policy 0, policy_version 235812 (0.0005) [2023-12-26 17:04:56,390][105692] Updated weights for policy 0, policy_version 235822 (0.0005) [2023-12-26 17:04:56,475][105620] Updated weights for policy 1, policy_version 236394 (0.0010) [2023-12-26 17:04:56,531][105620] Updated weights for policy 1, policy_version 236404 (0.0009) [2023-12-26 17:04:56,584][105620] Updated weights for policy 1, policy_version 236414 (0.0010) [2023-12-26 17:04:56,637][105620] Updated weights for policy 1, policy_version 236424 (0.0010) [2023-12-26 17:04:56,895][105692] Updated weights for policy 0, policy_version 235832 (0.0005) [2023-12-26 17:04:56,940][105692] Updated weights for policy 0, policy_version 235842 (0.0005) [2023-12-26 17:04:56,988][105692] Updated weights for policy 0, policy_version 235852 (0.0005) [2023-12-26 17:04:57,514][105620] Updated weights for policy 1, policy_version 236434 (0.0005) [2023-12-26 17:04:57,563][105620] Updated weights for policy 1, policy_version 236444 (0.0005) [2023-12-26 17:04:57,589][105692] Updated weights for policy 0, policy_version 235862 (0.0005) [2023-12-26 17:04:57,621][105620] Updated weights for policy 1, policy_version 236454 (0.0008) [2023-12-26 17:04:57,645][105692] Updated weights for policy 0, policy_version 235872 (0.0005) [2023-12-26 17:04:57,696][105692] Updated weights for policy 0, policy_version 235882 (0.0007) [2023-12-26 17:04:58,364][105692] Updated weights for policy 0, policy_version 235892 (0.0008) [2023-12-26 17:04:58,398][105620] Updated weights for policy 1, policy_version 236464 (0.0008) [2023-12-26 17:04:58,426][105692] Updated weights for policy 0, policy_version 235902 (0.0010) [2023-12-26 17:04:58,462][105620] Updated weights for policy 1, policy_version 236474 (0.0008) [2023-12-26 17:04:58,492][105692] Updated weights for policy 0, policy_version 235912 (0.0009) [2023-12-26 17:04:58,525][105620] Updated weights for policy 1, policy_version 236484 (0.0008) [2023-12-26 17:04:59,285][105692] Updated weights for policy 0, policy_version 235922 (0.0010) [2023-12-26 17:04:59,346][105692] Updated weights for policy 0, policy_version 235932 (0.0008) [2023-12-26 17:04:59,386][105620] Updated weights for policy 1, policy_version 236494 (0.0007) [2023-12-26 17:04:59,409][105692] Updated weights for policy 0, policy_version 235942 (0.0007) [2023-12-26 17:04:59,442][105620] Updated weights for policy 1, policy_version 236504 (0.0007) [2023-12-26 17:04:59,465][105692] Updated weights for policy 0, policy_version 235952 (0.0005) [2023-12-26 17:04:59,498][105620] Updated weights for policy 1, policy_version 236514 (0.0010) [2023-12-26 17:05:00,067][105692] Updated weights for policy 0, policy_version 235962 (0.0006) [2023-12-26 17:05:00,125][105692] Updated weights for policy 0, policy_version 235972 (0.0009) [2023-12-26 17:05:00,183][105692] Updated weights for policy 0, policy_version 235982 (0.0009) [2023-12-26 17:05:00,331][105620] Updated weights for policy 1, policy_version 236524 (0.0009) [2023-12-26 17:05:00,391][105620] Updated weights for policy 1, policy_version 236534 (0.0010) [2023-12-26 17:05:00,442][105620] Updated weights for policy 1, policy_version 236544 (0.0009) [2023-12-26 17:05:00,804][105692] Updated weights for policy 0, policy_version 235992 (0.0006) [2023-12-26 17:05:00,862][105692] Updated weights for policy 0, policy_version 236002 (0.0005) [2023-12-26 17:05:00,913][105692] Updated weights for policy 0, policy_version 236012 (0.0005) [2023-12-26 17:05:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 120995840. Throughput: 0: 9613.1, 1: 9613.5. Samples: 120965880. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 17:05:01,062][104569] Avg episode reward: [(0, '8912.767'), (1, '7337.069')] [2023-12-26 17:05:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000236016_60432384.pth... [2023-12-26 17:05:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000236552_60563456.pth... [2023-12-26 17:05:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000234864_60137472.pth [2023-12-26 17:05:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000235464_60284928.pth [2023-12-26 17:05:01,165][105620] Updated weights for policy 1, policy_version 236554 (0.0006) [2023-12-26 17:05:01,220][105620] Updated weights for policy 1, policy_version 236564 (0.0009) [2023-12-26 17:05:01,247][105586] KL-divergence is very high: 123.5292 [2023-12-26 17:05:01,253][105586] KL-divergence is very high: 133.0052 [2023-12-26 17:05:01,285][105620] Updated weights for policy 1, policy_version 236574 (0.0009) [2023-12-26 17:05:01,297][105586] KL-divergence is very high: 138.3912 [2023-12-26 17:05:01,303][105586] KL-divergence is very high: 126.3198 [2023-12-26 17:05:01,350][105620] Updated weights for policy 1, policy_version 236584 (0.0009) [2023-12-26 17:05:01,630][105692] Updated weights for policy 0, policy_version 236022 (0.0007) [2023-12-26 17:05:01,686][105692] Updated weights for policy 0, policy_version 236032 (0.0009) [2023-12-26 17:05:01,753][105692] Updated weights for policy 0, policy_version 236042 (0.0010) [2023-12-26 17:05:02,025][105620] Updated weights for policy 1, policy_version 236594 (0.0005) [2023-12-26 17:05:02,077][105620] Updated weights for policy 1, policy_version 236604 (0.0005) [2023-12-26 17:05:02,132][105620] Updated weights for policy 1, policy_version 236614 (0.0006) [2023-12-26 17:05:02,562][105692] Updated weights for policy 0, policy_version 236052 (0.0008) [2023-12-26 17:05:02,619][105692] Updated weights for policy 0, policy_version 236062 (0.0005) [2023-12-26 17:05:02,677][105692] Updated weights for policy 0, policy_version 236072 (0.0005) [2023-12-26 17:05:02,824][105620] Updated weights for policy 1, policy_version 236624 (0.0009) [2023-12-26 17:05:02,886][105620] Updated weights for policy 1, policy_version 236634 (0.0010) [2023-12-26 17:05:02,953][105620] Updated weights for policy 1, policy_version 236644 (0.0010) [2023-12-26 17:05:03,254][105692] Updated weights for policy 0, policy_version 236082 (0.0006) [2023-12-26 17:05:03,306][105692] Updated weights for policy 0, policy_version 236092 (0.0011) [2023-12-26 17:05:03,350][105692] Updated weights for policy 0, policy_version 236102 (0.0010) [2023-12-26 17:05:03,408][105692] Updated weights for policy 0, policy_version 236112 (0.0010) [2023-12-26 17:05:03,717][105620] Updated weights for policy 1, policy_version 236654 (0.0009) [2023-12-26 17:05:03,775][105620] Updated weights for policy 1, policy_version 236664 (0.0010) [2023-12-26 17:05:03,831][105620] Updated weights for policy 1, policy_version 236674 (0.0010) [2023-12-26 17:05:04,143][105692] Updated weights for policy 0, policy_version 236122 (0.0011) [2023-12-26 17:05:04,202][105692] Updated weights for policy 0, policy_version 236132 (0.0011) [2023-12-26 17:05:04,267][105692] Updated weights for policy 0, policy_version 236142 (0.0010) [2023-12-26 17:05:04,619][105620] Updated weights for policy 1, policy_version 236684 (0.0010) [2023-12-26 17:05:04,669][105620] Updated weights for policy 1, policy_version 236694 (0.0008) [2023-12-26 17:05:04,717][105620] Updated weights for policy 1, policy_version 236704 (0.0008) [2023-12-26 17:05:04,973][105692] Updated weights for policy 0, policy_version 236152 (0.0010) [2023-12-26 17:05:05,021][105692] Updated weights for policy 0, policy_version 236162 (0.0010) [2023-12-26 17:05:05,082][105692] Updated weights for policy 0, policy_version 236172 (0.0010) [2023-12-26 17:05:05,533][105620] Updated weights for policy 1, policy_version 236714 (0.0008) [2023-12-26 17:05:05,597][105620] Updated weights for policy 1, policy_version 236724 (0.0006) [2023-12-26 17:05:05,657][105620] Updated weights for policy 1, policy_version 236734 (0.0006) [2023-12-26 17:05:05,691][105692] Updated weights for policy 0, policy_version 236182 (0.0009) [2023-12-26 17:05:05,712][105620] Updated weights for policy 1, policy_version 236744 (0.0005) [2023-12-26 17:05:05,740][105692] Updated weights for policy 0, policy_version 236192 (0.0010) [2023-12-26 17:05:05,787][105692] Updated weights for policy 0, policy_version 236202 (0.0010) [2023-12-26 17:05:06,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19438.7). Total num frames: 121094144. Throughput: 0: 9706.2, 1: 9492.3. Samples: 121081008. Policy #0 lag: (min: 31.0, avg: 31.7, max: 52.0) [2023-12-26 17:05:06,062][104569] Avg episode reward: [(0, '8998.706'), (1, '7563.800')] [2023-12-26 17:05:06,232][105620] Updated weights for policy 1, policy_version 236754 (0.0008) [2023-12-26 17:05:06,285][105620] Updated weights for policy 1, policy_version 236764 (0.0011) [2023-12-26 17:05:06,345][105620] Updated weights for policy 1, policy_version 236774 (0.0011) [2023-12-26 17:05:06,551][105692] Updated weights for policy 0, policy_version 236212 (0.0010) [2023-12-26 17:05:06,610][105692] Updated weights for policy 0, policy_version 236222 (0.0010) [2023-12-26 17:05:06,669][105692] Updated weights for policy 0, policy_version 236232 (0.0010) [2023-12-26 17:05:06,975][105620] Updated weights for policy 1, policy_version 236784 (0.0006) [2023-12-26 17:05:07,036][105620] Updated weights for policy 1, policy_version 236794 (0.0006) [2023-12-26 17:05:07,098][105620] Updated weights for policy 1, policy_version 236804 (0.0005) [2023-12-26 17:05:07,381][105692] Updated weights for policy 0, policy_version 236242 (0.0010) [2023-12-26 17:05:07,435][105692] Updated weights for policy 0, policy_version 236252 (0.0010) [2023-12-26 17:05:07,490][105692] Updated weights for policy 0, policy_version 236262 (0.0010) [2023-12-26 17:05:07,534][105692] Updated weights for policy 0, policy_version 236272 (0.0010) [2023-12-26 17:05:07,648][105620] Updated weights for policy 1, policy_version 236814 (0.0008) [2023-12-26 17:05:07,701][105620] Updated weights for policy 1, policy_version 236824 (0.0006) [2023-12-26 17:05:07,755][105620] Updated weights for policy 1, policy_version 236834 (0.0010) [2023-12-26 17:05:08,127][105692] Updated weights for policy 0, policy_version 236282 (0.0006) [2023-12-26 17:05:08,178][105692] Updated weights for policy 0, policy_version 236292 (0.0006) [2023-12-26 17:05:08,232][105692] Updated weights for policy 0, policy_version 236302 (0.0005) [2023-12-26 17:05:08,501][105620] Updated weights for policy 1, policy_version 236844 (0.0010) [2023-12-26 17:05:08,563][105620] Updated weights for policy 1, policy_version 236854 (0.0010) [2023-12-26 17:05:08,619][105620] Updated weights for policy 1, policy_version 236864 (0.0009) [2023-12-26 17:05:08,792][105692] Updated weights for policy 0, policy_version 236312 (0.0006) [2023-12-26 17:05:08,852][105692] Updated weights for policy 0, policy_version 236322 (0.0005) [2023-12-26 17:05:08,903][105692] Updated weights for policy 0, policy_version 236332 (0.0010) [2023-12-26 17:05:09,372][105620] Updated weights for policy 1, policy_version 236874 (0.0010) [2023-12-26 17:05:09,437][105620] Updated weights for policy 1, policy_version 236884 (0.0009) [2023-12-26 17:05:09,494][105620] Updated weights for policy 1, policy_version 236894 (0.0006) [2023-12-26 17:05:09,553][105620] Updated weights for policy 1, policy_version 236904 (0.0006) [2023-12-26 17:05:09,586][105692] Updated weights for policy 0, policy_version 236342 (0.0009) [2023-12-26 17:05:09,646][105692] Updated weights for policy 0, policy_version 236352 (0.0010) [2023-12-26 17:05:09,712][105692] Updated weights for policy 0, policy_version 236362 (0.0009) [2023-12-26 17:05:10,287][105620] Updated weights for policy 1, policy_version 236914 (0.0008) [2023-12-26 17:05:10,345][105620] Updated weights for policy 1, policy_version 236924 (0.0007) [2023-12-26 17:05:10,393][105620] Updated weights for policy 1, policy_version 236934 (0.0008) [2023-12-26 17:05:10,445][105692] Updated weights for policy 0, policy_version 236372 (0.0010) [2023-12-26 17:05:10,497][105692] Updated weights for policy 0, policy_version 236382 (0.0007) [2023-12-26 17:05:10,567][105692] Updated weights for policy 0, policy_version 236392 (0.0005) [2023-12-26 17:05:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 121192448. Throughput: 0: 9847.0, 1: 9576.7. Samples: 121203952. Policy #0 lag: (min: 31.0, avg: 31.7, max: 52.0) [2023-12-26 17:05:11,063][104569] Avg episode reward: [(0, '9001.922'), (1, '4835.349')] [2023-12-26 17:05:11,123][105692] Updated weights for policy 0, policy_version 236402 (0.0007) [2023-12-26 17:05:11,195][105692] Updated weights for policy 0, policy_version 236412 (0.0009) [2023-12-26 17:05:11,258][105692] Updated weights for policy 0, policy_version 236422 (0.0010) [2023-12-26 17:05:11,279][105620] Updated weights for policy 1, policy_version 236944 (0.0007) [2023-12-26 17:05:11,317][105692] Updated weights for policy 0, policy_version 236432 (0.0010) [2023-12-26 17:05:11,334][105620] Updated weights for policy 1, policy_version 236954 (0.0009) [2023-12-26 17:05:11,404][105620] Updated weights for policy 1, policy_version 236964 (0.0009) [2023-12-26 17:05:12,110][105692] Updated weights for policy 0, policy_version 236442 (0.0011) [2023-12-26 17:05:12,176][105692] Updated weights for policy 0, policy_version 236452 (0.0011) [2023-12-26 17:05:12,179][105620] Updated weights for policy 1, policy_version 236974 (0.0007) [2023-12-26 17:05:12,236][105692] Updated weights for policy 0, policy_version 236462 (0.0010) [2023-12-26 17:05:12,244][105620] Updated weights for policy 1, policy_version 236984 (0.0006) [2023-12-26 17:05:12,314][105620] Updated weights for policy 1, policy_version 236994 (0.0008) [2023-12-26 17:05:12,984][105692] Updated weights for policy 0, policy_version 236472 (0.0010) [2023-12-26 17:05:13,036][105620] Updated weights for policy 1, policy_version 237004 (0.0007) [2023-12-26 17:05:13,053][105692] Updated weights for policy 0, policy_version 236482 (0.0011) [2023-12-26 17:05:13,091][105620] Updated weights for policy 1, policy_version 237014 (0.0005) [2023-12-26 17:05:13,109][105692] Updated weights for policy 0, policy_version 236492 (0.0011) [2023-12-26 17:05:13,156][105620] Updated weights for policy 1, policy_version 237024 (0.0006) [2023-12-26 17:05:13,852][105692] Updated weights for policy 0, policy_version 236502 (0.0010) [2023-12-26 17:05:13,900][105692] Updated weights for policy 0, policy_version 236512 (0.0010) [2023-12-26 17:05:13,903][105620] Updated weights for policy 1, policy_version 237034 (0.0008) [2023-12-26 17:05:13,958][105620] Updated weights for policy 1, policy_version 237044 (0.0009) [2023-12-26 17:05:13,963][105586] KL-divergence is very high: 215.9003 [2023-12-26 17:05:13,964][105692] Updated weights for policy 0, policy_version 236522 (0.0010) [2023-12-26 17:05:13,968][105586] KL-divergence is very high: 226.1892 [2023-12-26 17:05:13,973][105586] KL-divergence is very high: 231.0413 [2023-12-26 17:05:13,977][105586] KL-divergence is very high: 300.6422 [2023-12-26 17:05:13,982][105586] KL-divergence is very high: 176.3000 [2023-12-26 17:05:14,001][105586] KL-divergence is very high: 134.7167 [2023-12-26 17:05:14,006][105586] KL-divergence is very high: 106.5341 [2023-12-26 17:05:14,006][105620] Updated weights for policy 1, policy_version 237054 (0.0006) [2023-12-26 17:05:14,015][105586] KL-divergence is very high: 126.9814 [2023-12-26 17:05:14,057][105620] Updated weights for policy 1, policy_version 237064 (0.0007) [2023-12-26 17:05:14,702][105692] Updated weights for policy 0, policy_version 236532 (0.0010) [2023-12-26 17:05:14,751][105692] Updated weights for policy 0, policy_version 236542 (0.0010) [2023-12-26 17:05:14,817][105692] Updated weights for policy 0, policy_version 236552 (0.0010) [2023-12-26 17:05:14,825][105620] Updated weights for policy 1, policy_version 237074 (0.0011) [2023-12-26 17:05:14,888][105620] Updated weights for policy 1, policy_version 237084 (0.0011) [2023-12-26 17:05:14,954][105620] Updated weights for policy 1, policy_version 237094 (0.0010) [2023-12-26 17:05:15,564][105692] Updated weights for policy 0, policy_version 236562 (0.0010) [2023-12-26 17:05:15,612][105692] Updated weights for policy 0, policy_version 236572 (0.0005) [2023-12-26 17:05:15,665][105692] Updated weights for policy 0, policy_version 236582 (0.0009) [2023-12-26 17:05:15,674][105620] Updated weights for policy 1, policy_version 237104 (0.0005) [2023-12-26 17:05:15,716][105692] Updated weights for policy 0, policy_version 236592 (0.0010) [2023-12-26 17:05:15,734][105620] Updated weights for policy 1, policy_version 237114 (0.0009) [2023-12-26 17:05:15,801][105620] Updated weights for policy 1, policy_version 237124 (0.0010) [2023-12-26 17:05:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 121290752. Throughput: 0: 9880.4, 1: 9456.8. Samples: 121260576. Policy #0 lag: (min: 31.0, avg: 31.7, max: 52.0) [2023-12-26 17:05:16,062][104569] Avg episode reward: [(0, '7900.667'), (1, '5318.944')] [2023-12-26 17:05:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000236592_60579840.pth... [2023-12-26 17:05:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000237128_60710912.pth... [2023-12-26 17:05:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000235408_60276736.pth [2023-12-26 17:05:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000236008_60424192.pth [2023-12-26 17:05:16,347][105620] Updated weights for policy 1, policy_version 237134 (0.0007) [2023-12-26 17:05:16,408][105620] Updated weights for policy 1, policy_version 237144 (0.0005) [2023-12-26 17:05:16,464][105620] Updated weights for policy 1, policy_version 237154 (0.0005) [2023-12-26 17:05:16,495][105692] Updated weights for policy 0, policy_version 236602 (0.0005) [2023-12-26 17:05:16,553][105692] Updated weights for policy 0, policy_version 236612 (0.0005) [2023-12-26 17:05:16,607][105692] Updated weights for policy 0, policy_version 236622 (0.0005) [2023-12-26 17:05:16,964][105620] Updated weights for policy 1, policy_version 237164 (0.0007) [2023-12-26 17:05:17,025][105620] Updated weights for policy 1, policy_version 237174 (0.0010) [2023-12-26 17:05:17,093][105620] Updated weights for policy 1, policy_version 237184 (0.0007) [2023-12-26 17:05:17,168][105692] Updated weights for policy 0, policy_version 236632 (0.0010) [2023-12-26 17:05:17,219][105692] Updated weights for policy 0, policy_version 236642 (0.0010) [2023-12-26 17:05:17,270][105692] Updated weights for policy 0, policy_version 236652 (0.0010) [2023-12-26 17:05:17,727][105620] Updated weights for policy 1, policy_version 237194 (0.0008) [2023-12-26 17:05:17,775][105620] Updated weights for policy 1, policy_version 237204 (0.0010) [2023-12-26 17:05:17,820][105620] Updated weights for policy 1, policy_version 237214 (0.0010) [2023-12-26 17:05:17,868][105620] Updated weights for policy 1, policy_version 237224 (0.0010) [2023-12-26 17:05:17,954][105692] Updated weights for policy 0, policy_version 236662 (0.0007) [2023-12-26 17:05:18,011][105692] Updated weights for policy 0, policy_version 236672 (0.0005) [2023-12-26 17:05:18,067][105692] Updated weights for policy 0, policy_version 236682 (0.0007) [2023-12-26 17:05:18,558][105620] Updated weights for policy 1, policy_version 237234 (0.0011) [2023-12-26 17:05:18,620][105620] Updated weights for policy 1, policy_version 237244 (0.0010) [2023-12-26 17:05:18,681][105620] Updated weights for policy 1, policy_version 237254 (0.0010) [2023-12-26 17:05:18,748][105692] Updated weights for policy 0, policy_version 236692 (0.0008) [2023-12-26 17:05:18,808][105692] Updated weights for policy 0, policy_version 236702 (0.0006) [2023-12-26 17:05:18,860][105692] Updated weights for policy 0, policy_version 236712 (0.0008) [2023-12-26 17:05:19,266][105620] Updated weights for policy 1, policy_version 237264 (0.0008) [2023-12-26 17:05:19,337][105620] Updated weights for policy 1, policy_version 237274 (0.0008) [2023-12-26 17:05:19,403][105620] Updated weights for policy 1, policy_version 237284 (0.0010) [2023-12-26 17:05:19,454][105692] Updated weights for policy 0, policy_version 236722 (0.0006) [2023-12-26 17:05:19,524][105692] Updated weights for policy 0, policy_version 236732 (0.0009) [2023-12-26 17:05:19,588][105692] Updated weights for policy 0, policy_version 236742 (0.0006) [2023-12-26 17:05:19,653][105692] Updated weights for policy 0, policy_version 236752 (0.0006) [2023-12-26 17:05:20,267][105620] Updated weights for policy 1, policy_version 237295 (0.0009) [2023-12-26 17:05:20,315][105692] Updated weights for policy 0, policy_version 236762 (0.0007) [2023-12-26 17:05:20,331][105620] Updated weights for policy 1, policy_version 237305 (0.0008) [2023-12-26 17:05:20,375][105692] Updated weights for policy 0, policy_version 236772 (0.0008) [2023-12-26 17:05:20,381][105620] Updated weights for policy 1, policy_version 237315 (0.0009) [2023-12-26 17:05:20,438][105692] Updated weights for policy 0, policy_version 236782 (0.0007) [2023-12-26 17:05:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 121389056. Throughput: 0: 9923.7, 1: 9584.7. Samples: 121383796. Policy #0 lag: (min: 31.0, avg: 31.7, max: 52.0) [2023-12-26 17:05:21,062][104569] Avg episode reward: [(0, '7983.835'), (1, '6433.229')] [2023-12-26 17:05:21,142][105620] Updated weights for policy 1, policy_version 237325 (0.0008) [2023-12-26 17:05:21,213][105620] Updated weights for policy 1, policy_version 237335 (0.0009) [2023-12-26 17:05:21,238][105692] Updated weights for policy 0, policy_version 236792 (0.0010) [2023-12-26 17:05:21,285][105620] Updated weights for policy 1, policy_version 237345 (0.0009) [2023-12-26 17:05:21,309][105692] Updated weights for policy 0, policy_version 236802 (0.0009) [2023-12-26 17:05:21,376][105692] Updated weights for policy 0, policy_version 236812 (0.0007) [2023-12-26 17:05:22,070][105620] Updated weights for policy 1, policy_version 237355 (0.0007) [2023-12-26 17:05:22,128][105620] Updated weights for policy 1, policy_version 237365 (0.0009) [2023-12-26 17:05:22,153][105692] Updated weights for policy 0, policy_version 236822 (0.0007) [2023-12-26 17:05:22,183][105620] Updated weights for policy 1, policy_version 237375 (0.0008) [2023-12-26 17:05:22,210][105692] Updated weights for policy 0, policy_version 236832 (0.0006) [2023-12-26 17:05:22,268][105692] Updated weights for policy 0, policy_version 236842 (0.0008) [2023-12-26 17:05:22,960][105620] Updated weights for policy 1, policy_version 237385 (0.0008) [2023-12-26 17:05:23,015][105620] Updated weights for policy 1, policy_version 237395 (0.0009) [2023-12-26 17:05:23,055][105692] Updated weights for policy 0, policy_version 236852 (0.0008) [2023-12-26 17:05:23,072][105620] Updated weights for policy 1, policy_version 237405 (0.0008) [2023-12-26 17:05:23,106][105692] Updated weights for policy 0, policy_version 236862 (0.0008) [2023-12-26 17:05:23,128][105620] Updated weights for policy 1, policy_version 237415 (0.0008) [2023-12-26 17:05:23,162][105692] Updated weights for policy 0, policy_version 236872 (0.0007) [2023-12-26 17:05:23,822][105620] Updated weights for policy 1, policy_version 237425 (0.0008) [2023-12-26 17:05:23,844][105692] Updated weights for policy 0, policy_version 236882 (0.0008) [2023-12-26 17:05:23,876][105620] Updated weights for policy 1, policy_version 237435 (0.0008) [2023-12-26 17:05:23,903][105692] Updated weights for policy 0, policy_version 236892 (0.0007) [2023-12-26 17:05:23,935][105620] Updated weights for policy 1, policy_version 237445 (0.0009) [2023-12-26 17:05:23,962][105692] Updated weights for policy 0, policy_version 236902 (0.0008) [2023-12-26 17:05:24,014][105692] Updated weights for policy 0, policy_version 236912 (0.0009) [2023-12-26 17:05:24,665][105620] Updated weights for policy 1, policy_version 237455 (0.0008) [2023-12-26 17:05:24,688][105692] Updated weights for policy 0, policy_version 236922 (0.0005) [2023-12-26 17:05:24,721][105620] Updated weights for policy 1, policy_version 237465 (0.0009) [2023-12-26 17:05:24,740][105692] Updated weights for policy 0, policy_version 236932 (0.0007) [2023-12-26 17:05:24,781][105620] Updated weights for policy 1, policy_version 237475 (0.0010) [2023-12-26 17:05:24,800][105692] Updated weights for policy 0, policy_version 236942 (0.0006) [2023-12-26 17:05:25,357][105692] Updated weights for policy 0, policy_version 236952 (0.0010) [2023-12-26 17:05:25,417][105692] Updated weights for policy 0, policy_version 236962 (0.0009) [2023-12-26 17:05:25,442][105620] Updated weights for policy 1, policy_version 237485 (0.0009) [2023-12-26 17:05:25,476][105692] Updated weights for policy 0, policy_version 236972 (0.0008) [2023-12-26 17:05:25,493][105620] Updated weights for policy 1, policy_version 237495 (0.0005) [2023-12-26 17:05:25,544][105620] Updated weights for policy 1, policy_version 237505 (0.0005) [2023-12-26 17:05:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 121487360. Throughput: 0: 9970.6, 1: 9555.9. Samples: 121499756. Policy #0 lag: (min: 31.0, avg: 31.7, max: 52.0) [2023-12-26 17:05:26,062][104569] Avg episode reward: [(0, '8997.412'), (1, '7701.855')] [2023-12-26 17:05:26,163][105620] Updated weights for policy 1, policy_version 237515 (0.0006) [2023-12-26 17:05:26,223][105620] Updated weights for policy 1, policy_version 237525 (0.0008) [2023-12-26 17:05:26,268][105692] Updated weights for policy 0, policy_version 236982 (0.0009) [2023-12-26 17:05:26,286][105620] Updated weights for policy 1, policy_version 237535 (0.0008) [2023-12-26 17:05:26,321][105692] Updated weights for policy 0, policy_version 236992 (0.0007) [2023-12-26 17:05:26,367][105692] Updated weights for policy 0, policy_version 237002 (0.0005) [2023-12-26 17:05:26,896][105692] Updated weights for policy 0, policy_version 237012 (0.0005) [2023-12-26 17:05:26,951][105692] Updated weights for policy 0, policy_version 237022 (0.0005) [2023-12-26 17:05:26,997][105692] Updated weights for policy 0, policy_version 237032 (0.0005) [2023-12-26 17:05:27,079][105620] Updated weights for policy 1, policy_version 237545 (0.0008) [2023-12-26 17:05:27,135][105620] Updated weights for policy 1, policy_version 237555 (0.0010) [2023-12-26 17:05:27,192][105620] Updated weights for policy 1, policy_version 237565 (0.0008) [2023-12-26 17:05:27,256][105620] Updated weights for policy 1, policy_version 237575 (0.0006) [2023-12-26 17:05:27,565][105692] Updated weights for policy 0, policy_version 237042 (0.0005) [2023-12-26 17:05:27,638][105692] Updated weights for policy 0, policy_version 237052 (0.0006) [2023-12-26 17:05:27,699][105692] Updated weights for policy 0, policy_version 237062 (0.0010) [2023-12-26 17:05:27,760][105692] Updated weights for policy 0, policy_version 237072 (0.0010) [2023-12-26 17:05:27,880][105620] Updated weights for policy 1, policy_version 237585 (0.0006) [2023-12-26 17:05:27,934][105620] Updated weights for policy 1, policy_version 237595 (0.0006) [2023-12-26 17:05:28,001][105620] Updated weights for policy 1, policy_version 237605 (0.0006) [2023-12-26 17:05:28,353][105692] Updated weights for policy 0, policy_version 237082 (0.0008) [2023-12-26 17:05:28,409][105692] Updated weights for policy 0, policy_version 237092 (0.0009) [2023-12-26 17:05:28,456][105692] Updated weights for policy 0, policy_version 237102 (0.0010) [2023-12-26 17:05:28,642][105620] Updated weights for policy 1, policy_version 237615 (0.0009) [2023-12-26 17:05:28,709][105620] Updated weights for policy 1, policy_version 237625 (0.0011) [2023-12-26 17:05:28,772][105620] Updated weights for policy 1, policy_version 237635 (0.0011) [2023-12-26 17:05:29,149][105692] Updated weights for policy 0, policy_version 237112 (0.0009) [2023-12-26 17:05:29,212][105692] Updated weights for policy 0, policy_version 237122 (0.0008) [2023-12-26 17:05:29,275][105692] Updated weights for policy 0, policy_version 237132 (0.0008) [2023-12-26 17:05:29,515][105620] Updated weights for policy 1, policy_version 237645 (0.0010) [2023-12-26 17:05:29,571][105620] Updated weights for policy 1, policy_version 237655 (0.0009) [2023-12-26 17:05:29,635][105620] Updated weights for policy 1, policy_version 237665 (0.0006) [2023-12-26 17:05:30,032][105692] Updated weights for policy 0, policy_version 237142 (0.0007) [2023-12-26 17:05:30,078][105692] Updated weights for policy 0, policy_version 237152 (0.0005) [2023-12-26 17:05:30,128][105692] Updated weights for policy 0, policy_version 237162 (0.0007) [2023-12-26 17:05:30,355][105620] Updated weights for policy 1, policy_version 237675 (0.0007) [2023-12-26 17:05:30,416][105620] Updated weights for policy 1, policy_version 237685 (0.0005) [2023-12-26 17:05:30,476][105620] Updated weights for policy 1, policy_version 237695 (0.0006) [2023-12-26 17:05:30,998][105692] Updated weights for policy 0, policy_version 237173 (0.0010) [2023-12-26 17:05:31,030][105620] Updated weights for policy 1, policy_version 237705 (0.0007) [2023-12-26 17:05:31,062][105692] Updated weights for policy 0, policy_version 237183 (0.0009) [2023-12-26 17:05:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 121585664. Throughput: 0: 10093.2, 1: 9604.5. Samples: 121562664. Policy #0 lag: (min: 31.0, avg: 31.7, max: 52.0) [2023-12-26 17:05:31,063][104569] Avg episode reward: [(0, '9004.394'), (1, '7680.759')] [2023-12-26 17:05:31,089][105620] Updated weights for policy 1, policy_version 237715 (0.0007) [2023-12-26 17:05:31,120][105692] Updated weights for policy 0, policy_version 237193 (0.0008) [2023-12-26 17:05:31,154][105620] Updated weights for policy 1, policy_version 237725 (0.0007) [2023-12-26 17:05:31,163][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000237200_60735488.pth... [2023-12-26 17:05:31,168][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000236016_60432384.pth [2023-12-26 17:05:31,209][105620] Updated weights for policy 1, policy_version 237735 (0.0007) [2023-12-26 17:05:31,212][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000237736_60866560.pth... [2023-12-26 17:05:31,215][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000236552_60563456.pth [2023-12-26 17:05:31,890][105692] Updated weights for policy 0, policy_version 237203 (0.0009) [2023-12-26 17:05:31,950][105620] Updated weights for policy 1, policy_version 237745 (0.0007) [2023-12-26 17:05:31,952][105692] Updated weights for policy 0, policy_version 237213 (0.0011) [2023-12-26 17:05:32,007][105620] Updated weights for policy 1, policy_version 237755 (0.0007) [2023-12-26 17:05:32,017][105692] Updated weights for policy 0, policy_version 237223 (0.0008) [2023-12-26 17:05:32,079][105620] Updated weights for policy 1, policy_version 237765 (0.0007) [2023-12-26 17:05:32,690][105692] Updated weights for policy 0, policy_version 237233 (0.0006) [2023-12-26 17:05:32,727][105620] Updated weights for policy 1, policy_version 237775 (0.0009) [2023-12-26 17:05:32,753][105692] Updated weights for policy 0, policy_version 237243 (0.0009) [2023-12-26 17:05:32,776][105620] Updated weights for policy 1, policy_version 237785 (0.0010) [2023-12-26 17:05:32,813][105692] Updated weights for policy 0, policy_version 237253 (0.0008) [2023-12-26 17:05:32,828][105620] Updated weights for policy 1, policy_version 237795 (0.0010) [2023-12-26 17:05:32,870][105692] Updated weights for policy 0, policy_version 237263 (0.0006) [2023-12-26 17:05:33,554][105620] Updated weights for policy 1, policy_version 237805 (0.0010) [2023-12-26 17:05:33,611][105692] Updated weights for policy 0, policy_version 237273 (0.0006) [2023-12-26 17:05:33,613][105620] Updated weights for policy 1, policy_version 237815 (0.0010) [2023-12-26 17:05:33,673][105692] Updated weights for policy 0, policy_version 237283 (0.0006) [2023-12-26 17:05:33,680][105620] Updated weights for policy 1, policy_version 237825 (0.0011) [2023-12-26 17:05:33,731][105692] Updated weights for policy 0, policy_version 237293 (0.0006) [2023-12-26 17:05:34,366][105620] Updated weights for policy 1, policy_version 237835 (0.0010) [2023-12-26 17:05:34,386][105692] Updated weights for policy 0, policy_version 237303 (0.0007) [2023-12-26 17:05:34,423][105620] Updated weights for policy 1, policy_version 237845 (0.0009) [2023-12-26 17:05:34,449][105692] Updated weights for policy 0, policy_version 237313 (0.0010) [2023-12-26 17:05:34,484][105620] Updated weights for policy 1, policy_version 237855 (0.0006) [2023-12-26 17:05:34,513][105692] Updated weights for policy 0, policy_version 237323 (0.0011) [2023-12-26 17:05:35,181][105620] Updated weights for policy 1, policy_version 237865 (0.0006) [2023-12-26 17:05:35,237][105620] Updated weights for policy 1, policy_version 237875 (0.0006) [2023-12-26 17:05:35,242][105692] Updated weights for policy 0, policy_version 237333 (0.0009) [2023-12-26 17:05:35,283][105620] Updated weights for policy 1, policy_version 237885 (0.0005) [2023-12-26 17:05:35,313][105692] Updated weights for policy 0, policy_version 237343 (0.0006) [2023-12-26 17:05:35,349][105620] Updated weights for policy 1, policy_version 237896 (0.0006) [2023-12-26 17:05:35,368][105692] Updated weights for policy 0, policy_version 237353 (0.0005) [2023-12-26 17:05:35,912][105692] Updated weights for policy 0, policy_version 237363 (0.0006) [2023-12-26 17:05:35,956][105692] Updated weights for policy 0, policy_version 237373 (0.0008) [2023-12-26 17:05:36,015][105692] Updated weights for policy 0, policy_version 237383 (0.0007) [2023-12-26 17:05:36,061][105620] Updated weights for policy 1, policy_version 237906 (0.0010) [2023-12-26 17:05:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.9, 300 sec: 19522.0). Total num frames: 121692160. Throughput: 0: 10055.3, 1: 9630.7. Samples: 121678984. Policy #0 lag: (min: 31.0, avg: 31.7, max: 52.0) [2023-12-26 17:05:36,062][104569] Avg episode reward: [(0, '8909.690'), (1, '7138.962')] [2023-12-26 17:05:36,115][105620] Updated weights for policy 1, policy_version 237916 (0.0010) [2023-12-26 17:05:36,178][105620] Updated weights for policy 1, policy_version 237926 (0.0009) [2023-12-26 17:05:36,754][105692] Updated weights for policy 0, policy_version 237393 (0.0006) [2023-12-26 17:05:36,807][105692] Updated weights for policy 0, policy_version 237403 (0.0008) [2023-12-26 17:05:36,859][105692] Updated weights for policy 0, policy_version 237413 (0.0007) [2023-12-26 17:05:36,910][105692] Updated weights for policy 0, policy_version 237423 (0.0007) [2023-12-26 17:05:36,930][105620] Updated weights for policy 1, policy_version 237936 (0.0011) [2023-12-26 17:05:36,996][105620] Updated weights for policy 1, policy_version 237946 (0.0010) [2023-12-26 17:05:37,064][105620] Updated weights for policy 1, policy_version 237956 (0.0011) [2023-12-26 17:05:37,592][105692] Updated weights for policy 0, policy_version 237433 (0.0005) [2023-12-26 17:05:37,654][105692] Updated weights for policy 0, policy_version 237443 (0.0007) [2023-12-26 17:05:37,702][105692] Updated weights for policy 0, policy_version 237453 (0.0008) [2023-12-26 17:05:37,796][105620] Updated weights for policy 1, policy_version 237966 (0.0011) [2023-12-26 17:05:37,858][105620] Updated weights for policy 1, policy_version 237976 (0.0011) [2023-12-26 17:05:37,911][105620] Updated weights for policy 1, policy_version 237986 (0.0010) [2023-12-26 17:05:38,423][105692] Updated weights for policy 0, policy_version 237463 (0.0008) [2023-12-26 17:05:38,479][105692] Updated weights for policy 0, policy_version 237473 (0.0008) [2023-12-26 17:05:38,531][105692] Updated weights for policy 0, policy_version 237483 (0.0008) [2023-12-26 17:05:38,625][105620] Updated weights for policy 1, policy_version 237996 (0.0007) [2023-12-26 17:05:38,689][105620] Updated weights for policy 1, policy_version 238006 (0.0005) [2023-12-26 17:05:38,757][105620] Updated weights for policy 1, policy_version 238016 (0.0006) [2023-12-26 17:05:39,227][105692] Updated weights for policy 0, policy_version 237493 (0.0008) [2023-12-26 17:05:39,264][105620] Updated weights for policy 1, policy_version 238026 (0.0006) [2023-12-26 17:05:39,290][105692] Updated weights for policy 0, policy_version 237503 (0.0011) [2023-12-26 17:05:39,328][105620] Updated weights for policy 1, policy_version 238036 (0.0011) [2023-12-26 17:05:39,353][105692] Updated weights for policy 0, policy_version 237513 (0.0012) [2023-12-26 17:05:39,386][105620] Updated weights for policy 1, policy_version 238046 (0.0011) [2023-12-26 17:05:39,387][105586] KL-divergence is very high: 101.5642 [2023-12-26 17:05:39,446][105620] Updated weights for policy 1, policy_version 238056 (0.0010) [2023-12-26 17:05:40,087][105692] Updated weights for policy 0, policy_version 237523 (0.0007) [2023-12-26 17:05:40,149][105692] Updated weights for policy 0, policy_version 237533 (0.0007) [2023-12-26 17:05:40,155][105620] Updated weights for policy 1, policy_version 238066 (0.0010) [2023-12-26 17:05:40,206][105692] Updated weights for policy 0, policy_version 237543 (0.0005) [2023-12-26 17:05:40,212][105620] Updated weights for policy 1, policy_version 238076 (0.0011) [2023-12-26 17:05:40,264][105620] Updated weights for policy 1, policy_version 238086 (0.0010) [2023-12-26 17:05:40,839][105692] Updated weights for policy 0, policy_version 237553 (0.0006) [2023-12-26 17:05:40,902][105692] Updated weights for policy 0, policy_version 237563 (0.0011) [2023-12-26 17:05:40,962][105620] Updated weights for policy 1, policy_version 238096 (0.0011) [2023-12-26 17:05:40,965][105692] Updated weights for policy 0, policy_version 237573 (0.0011) [2023-12-26 17:05:41,026][105620] Updated weights for policy 1, policy_version 238106 (0.0009) [2023-12-26 17:05:41,031][105692] Updated weights for policy 0, policy_version 237583 (0.0010) [2023-12-26 17:05:41,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 121790464. Throughput: 0: 10120.6, 1: 9731.1. Samples: 121800072. Policy #0 lag: (min: 31.0, avg: 31.7, max: 52.0) [2023-12-26 17:05:41,063][104569] Avg episode reward: [(0, '8640.551'), (1, '7516.726')] [2023-12-26 17:05:41,096][105620] Updated weights for policy 1, policy_version 238116 (0.0009) [2023-12-26 17:05:41,808][105692] Updated weights for policy 0, policy_version 237593 (0.0010) [2023-12-26 17:05:41,856][105692] Updated weights for policy 0, policy_version 237603 (0.0010) [2023-12-26 17:05:41,883][105620] Updated weights for policy 1, policy_version 238126 (0.0008) [2023-12-26 17:05:41,912][105692] Updated weights for policy 0, policy_version 237613 (0.0010) [2023-12-26 17:05:41,952][105620] Updated weights for policy 1, policy_version 238136 (0.0006) [2023-12-26 17:05:42,015][105620] Updated weights for policy 1, policy_version 238146 (0.0008) [2023-12-26 17:05:42,686][105692] Updated weights for policy 0, policy_version 237623 (0.0010) [2023-12-26 17:05:42,748][105692] Updated weights for policy 0, policy_version 237633 (0.0010) [2023-12-26 17:05:42,798][105620] Updated weights for policy 1, policy_version 238156 (0.0007) [2023-12-26 17:05:42,807][105692] Updated weights for policy 0, policy_version 237643 (0.0010) [2023-12-26 17:05:42,859][105620] Updated weights for policy 1, policy_version 238166 (0.0007) [2023-12-26 17:05:42,918][105620] Updated weights for policy 1, policy_version 238176 (0.0009) [2023-12-26 17:05:43,448][105692] Updated weights for policy 0, policy_version 237653 (0.0010) [2023-12-26 17:05:43,500][105692] Updated weights for policy 0, policy_version 237664 (0.0010) [2023-12-26 17:05:43,552][105692] Updated weights for policy 0, policy_version 237674 (0.0008) [2023-12-26 17:05:43,616][105620] Updated weights for policy 1, policy_version 238186 (0.0010) [2023-12-26 17:05:43,678][105620] Updated weights for policy 1, policy_version 238196 (0.0008) [2023-12-26 17:05:43,739][105620] Updated weights for policy 1, policy_version 238206 (0.0008) [2023-12-26 17:05:43,805][105620] Updated weights for policy 1, policy_version 238216 (0.0008) [2023-12-26 17:05:44,293][105692] Updated weights for policy 0, policy_version 237684 (0.0008) [2023-12-26 17:05:44,351][105692] Updated weights for policy 0, policy_version 237694 (0.0010) [2023-12-26 17:05:44,369][105620] Updated weights for policy 1, policy_version 238226 (0.0006) [2023-12-26 17:05:44,414][105692] Updated weights for policy 0, policy_version 237704 (0.0011) [2023-12-26 17:05:44,424][105620] Updated weights for policy 1, policy_version 238236 (0.0006) [2023-12-26 17:05:44,476][105620] Updated weights for policy 1, policy_version 238246 (0.0008) [2023-12-26 17:05:45,113][105692] Updated weights for policy 0, policy_version 237714 (0.0009) [2023-12-26 17:05:45,165][105692] Updated weights for policy 0, policy_version 237724 (0.0010) [2023-12-26 17:05:45,211][105620] Updated weights for policy 1, policy_version 238256 (0.0007) [2023-12-26 17:05:45,227][105692] Updated weights for policy 0, policy_version 237734 (0.0009) [2023-12-26 17:05:45,276][105620] Updated weights for policy 1, policy_version 238266 (0.0011) [2023-12-26 17:05:45,282][105692] Updated weights for policy 0, policy_version 237744 (0.0006) [2023-12-26 17:05:45,336][105620] Updated weights for policy 1, policy_version 238276 (0.0010) [2023-12-26 17:05:45,993][105692] Updated weights for policy 0, policy_version 237754 (0.0009) [2023-12-26 17:05:46,026][105586] KL-divergence is very high: 114.3095 [2023-12-26 17:05:46,042][105692] Updated weights for policy 0, policy_version 237764 (0.0008) [2023-12-26 17:05:46,049][105620] Updated weights for policy 1, policy_version 238286 (0.0010) [2023-12-26 17:05:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 121880576. Throughput: 0: 10005.6, 1: 9777.5. Samples: 121856120. Policy #0 lag: (min: 31.0, avg: 31.7, max: 52.0) [2023-12-26 17:05:46,062][104569] Avg episode reward: [(0, '8908.847'), (1, '2528.896')] [2023-12-26 17:05:46,091][105692] Updated weights for policy 0, policy_version 237774 (0.0006) [2023-12-26 17:05:46,100][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000237776_60882944.pth... [2023-12-26 17:05:46,104][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000236592_60579840.pth [2023-12-26 17:05:46,108][105620] Updated weights for policy 1, policy_version 238296 (0.0010) [2023-12-26 17:05:46,117][105586] KL-divergence is very high: 118.4025 [2023-12-26 17:05:46,166][105620] Updated weights for policy 1, policy_version 238306 (0.0010) [2023-12-26 17:05:46,198][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000238312_61014016.pth... [2023-12-26 17:05:46,202][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000237128_60710912.pth [2023-12-26 17:05:46,816][105620] Updated weights for policy 1, policy_version 238316 (0.0008) [2023-12-26 17:05:46,861][105620] Updated weights for policy 1, policy_version 238326 (0.0005) [2023-12-26 17:05:46,920][105620] Updated weights for policy 1, policy_version 238336 (0.0006) [2023-12-26 17:05:46,928][105692] Updated weights for policy 0, policy_version 237784 (0.0008) [2023-12-26 17:05:46,992][105692] Updated weights for policy 0, policy_version 237794 (0.0008) [2023-12-26 17:05:47,020][105585] KL-divergence is very high: 123.5427 [2023-12-26 17:05:47,033][105585] KL-divergence is very high: 118.2601 [2023-12-26 17:05:47,055][105692] Updated weights for policy 0, policy_version 237804 (0.0009) [2023-12-26 17:05:47,064][105585] KL-divergence is very high: 136.1953 [2023-12-26 17:05:47,567][105620] Updated weights for policy 1, policy_version 238346 (0.0009) [2023-12-26 17:05:47,625][105620] Updated weights for policy 1, policy_version 238356 (0.0009) [2023-12-26 17:05:47,672][105620] Updated weights for policy 1, policy_version 238366 (0.0008) [2023-12-26 17:05:47,730][105620] Updated weights for policy 1, policy_version 238376 (0.0009) [2023-12-26 17:05:47,808][105692] Updated weights for policy 0, policy_version 237814 (0.0009) [2023-12-26 17:05:47,876][105692] Updated weights for policy 0, policy_version 237824 (0.0009) [2023-12-26 17:05:47,935][105692] Updated weights for policy 0, policy_version 237834 (0.0009) [2023-12-26 17:05:48,515][105620] Updated weights for policy 1, policy_version 238386 (0.0008) [2023-12-26 17:05:48,577][105620] Updated weights for policy 1, policy_version 238396 (0.0009) [2023-12-26 17:05:48,639][105620] Updated weights for policy 1, policy_version 238406 (0.0009) [2023-12-26 17:05:48,669][105692] Updated weights for policy 0, policy_version 237844 (0.0007) [2023-12-26 17:05:48,721][105692] Updated weights for policy 0, policy_version 237854 (0.0005) [2023-12-26 17:05:48,770][105692] Updated weights for policy 0, policy_version 237864 (0.0007) [2023-12-26 17:05:49,363][105620] Updated weights for policy 1, policy_version 238416 (0.0008) [2023-12-26 17:05:49,414][105620] Updated weights for policy 1, policy_version 238426 (0.0009) [2023-12-26 17:05:49,440][105586] KL-divergence is very high: 118.6432 [2023-12-26 17:05:49,446][105586] KL-divergence is very high: 193.8369 [2023-12-26 17:05:49,452][105586] KL-divergence is very high: 208.0150 [2023-12-26 17:05:49,457][105586] KL-divergence is very high: 169.5978 [2023-12-26 17:05:49,468][105620] Updated weights for policy 1, policy_version 238437 (0.0008) [2023-12-26 17:05:49,471][105586] KL-divergence is very high: 125.9398 [2023-12-26 17:05:49,499][105692] Updated weights for policy 0, policy_version 237874 (0.0008) [2023-12-26 17:05:49,550][105692] Updated weights for policy 0, policy_version 237884 (0.0005) [2023-12-26 17:05:49,605][105692] Updated weights for policy 0, policy_version 237894 (0.0006) [2023-12-26 17:05:49,664][105692] Updated weights for policy 0, policy_version 237904 (0.0005) [2023-12-26 17:05:50,295][105620] Updated weights for policy 1, policy_version 238447 (0.0008) [2023-12-26 17:05:50,311][105692] Updated weights for policy 0, policy_version 237914 (0.0007) [2023-12-26 17:05:50,354][105620] Updated weights for policy 1, policy_version 238457 (0.0009) [2023-12-26 17:05:50,373][105692] Updated weights for policy 0, policy_version 237924 (0.0006) [2023-12-26 17:05:50,412][105620] Updated weights for policy 1, policy_version 238467 (0.0008) [2023-12-26 17:05:50,431][105692] Updated weights for policy 0, policy_version 237934 (0.0006) [2023-12-26 17:05:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 121978880. Throughput: 0: 9937.8, 1: 9882.7. Samples: 121972932. Policy #0 lag: (min: 31.0, avg: 31.7, max: 52.0) [2023-12-26 17:05:51,062][104569] Avg episode reward: [(0, '8733.394'), (1, '1116.974')] [2023-12-26 17:05:51,173][105620] Updated weights for policy 1, policy_version 238477 (0.0008) [2023-12-26 17:05:51,195][105692] Updated weights for policy 0, policy_version 237944 (0.0007) [2023-12-26 17:05:51,239][105620] Updated weights for policy 1, policy_version 238487 (0.0007) [2023-12-26 17:05:51,245][105586] KL-divergence is very high: 111.5964 [2023-12-26 17:05:51,259][105692] Updated weights for policy 0, policy_version 237954 (0.0006) [2023-12-26 17:05:51,260][105586] KL-divergence is very high: 129.4967 [2023-12-26 17:05:51,280][105586] KL-divergence is very high: 116.8890 [2023-12-26 17:05:51,295][105586] KL-divergence is very high: 101.1076 [2023-12-26 17:05:51,299][105620] Updated weights for policy 1, policy_version 238497 (0.0007) [2023-12-26 17:05:51,305][105586] KL-divergence is very high: 102.7257 [2023-12-26 17:05:51,313][105692] Updated weights for policy 0, policy_version 237964 (0.0007) [2023-12-26 17:05:52,074][105620] Updated weights for policy 1, policy_version 238507 (0.0007) [2023-12-26 17:05:52,105][105692] Updated weights for policy 0, policy_version 237974 (0.0009) [2023-12-26 17:05:52,136][105620] Updated weights for policy 1, policy_version 238517 (0.0008) [2023-12-26 17:05:52,164][105692] Updated weights for policy 0, policy_version 237984 (0.0010) [2023-12-26 17:05:52,194][105620] Updated weights for policy 1, policy_version 238527 (0.0006) [2023-12-26 17:05:52,216][105692] Updated weights for policy 0, policy_version 237994 (0.0010) [2023-12-26 17:05:52,901][105692] Updated weights for policy 0, policy_version 238004 (0.0009) [2023-12-26 17:05:52,960][105620] Updated weights for policy 1, policy_version 238537 (0.0006) [2023-12-26 17:05:52,963][105692] Updated weights for policy 0, policy_version 238014 (0.0010) [2023-12-26 17:05:53,011][105620] Updated weights for policy 1, policy_version 238547 (0.0008) [2023-12-26 17:05:53,019][105692] Updated weights for policy 0, policy_version 238024 (0.0007) [2023-12-26 17:05:53,056][105620] Updated weights for policy 1, policy_version 238557 (0.0007) [2023-12-26 17:05:53,102][105620] Updated weights for policy 1, policy_version 238567 (0.0007) [2023-12-26 17:05:53,657][105692] Updated weights for policy 0, policy_version 238034 (0.0010) [2023-12-26 17:05:53,711][105692] Updated weights for policy 0, policy_version 238044 (0.0010) [2023-12-26 17:05:53,766][105692] Updated weights for policy 0, policy_version 238054 (0.0010) [2023-12-26 17:05:53,824][105692] Updated weights for policy 0, policy_version 238064 (0.0010) [2023-12-26 17:05:53,915][105620] Updated weights for policy 1, policy_version 238577 (0.0008) [2023-12-26 17:05:53,964][105620] Updated weights for policy 1, policy_version 238587 (0.0007) [2023-12-26 17:05:54,015][105620] Updated weights for policy 1, policy_version 238597 (0.0008) [2023-12-26 17:05:54,563][105692] Updated weights for policy 0, policy_version 238074 (0.0010) [2023-12-26 17:05:54,614][105692] Updated weights for policy 0, policy_version 238084 (0.0009) [2023-12-26 17:05:54,676][105692] Updated weights for policy 0, policy_version 238094 (0.0007) [2023-12-26 17:05:54,786][105620] Updated weights for policy 1, policy_version 238607 (0.0006) [2023-12-26 17:05:54,854][105620] Updated weights for policy 1, policy_version 238617 (0.0008) [2023-12-26 17:05:54,918][105620] Updated weights for policy 1, policy_version 238627 (0.0009) [2023-12-26 17:05:55,256][105692] Updated weights for policy 0, policy_version 238104 (0.0005) [2023-12-26 17:05:55,315][105692] Updated weights for policy 0, policy_version 238114 (0.0005) [2023-12-26 17:05:55,372][105692] Updated weights for policy 0, policy_version 238124 (0.0005) [2023-12-26 17:05:55,727][105620] Updated weights for policy 1, policy_version 238637 (0.0009) [2023-12-26 17:05:55,793][105620] Updated weights for policy 1, policy_version 238647 (0.0009) [2023-12-26 17:05:55,858][105620] Updated weights for policy 1, policy_version 238657 (0.0009) [2023-12-26 17:05:55,901][105692] Updated weights for policy 0, policy_version 238134 (0.0007) [2023-12-26 17:05:55,946][105692] Updated weights for policy 0, policy_version 238144 (0.0010) [2023-12-26 17:05:55,991][105692] Updated weights for policy 0, policy_version 238154 (0.0007) [2023-12-26 17:05:56,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 122085376. Throughput: 0: 9904.1, 1: 9723.5. Samples: 122087200. Policy #0 lag: (min: 31.0, avg: 31.7, max: 52.0) [2023-12-26 17:05:56,063][104569] Avg episode reward: [(0, '8822.068'), (1, '4147.408')] [2023-12-26 17:05:56,594][105692] Updated weights for policy 0, policy_version 238164 (0.0008) [2023-12-26 17:05:56,653][105692] Updated weights for policy 0, policy_version 238174 (0.0009) [2023-12-26 17:05:56,682][105620] Updated weights for policy 1, policy_version 238667 (0.0008) [2023-12-26 17:05:56,710][105692] Updated weights for policy 0, policy_version 238184 (0.0009) [2023-12-26 17:05:56,740][105620] Updated weights for policy 1, policy_version 238677 (0.0007) [2023-12-26 17:05:56,795][105620] Updated weights for policy 1, policy_version 238687 (0.0008) [2023-12-26 17:05:57,434][105692] Updated weights for policy 0, policy_version 238194 (0.0009) [2023-12-26 17:05:57,481][105692] Updated weights for policy 0, policy_version 238204 (0.0009) [2023-12-26 17:05:57,518][105620] Updated weights for policy 1, policy_version 238697 (0.0009) [2023-12-26 17:05:57,529][105692] Updated weights for policy 0, policy_version 238214 (0.0008) [2023-12-26 17:05:57,574][105620] Updated weights for policy 1, policy_version 238707 (0.0007) [2023-12-26 17:05:57,587][105692] Updated weights for policy 0, policy_version 238224 (0.0007) [2023-12-26 17:05:57,625][105620] Updated weights for policy 1, policy_version 238717 (0.0008) [2023-12-26 17:05:57,677][105620] Updated weights for policy 1, policy_version 238727 (0.0008) [2023-12-26 17:05:58,221][105692] Updated weights for policy 0, policy_version 238234 (0.0006) [2023-12-26 17:05:58,293][105692] Updated weights for policy 0, policy_version 238244 (0.0006) [2023-12-26 17:05:58,359][105692] Updated weights for policy 0, policy_version 238254 (0.0009) [2023-12-26 17:05:58,492][105620] Updated weights for policy 1, policy_version 238737 (0.0009) [2023-12-26 17:05:58,556][105620] Updated weights for policy 1, policy_version 238747 (0.0008) [2023-12-26 17:05:58,622][105620] Updated weights for policy 1, policy_version 238757 (0.0009) [2023-12-26 17:05:59,120][105692] Updated weights for policy 0, policy_version 238264 (0.0010) [2023-12-26 17:05:59,177][105692] Updated weights for policy 0, policy_version 238274 (0.0009) [2023-12-26 17:05:59,248][105692] Updated weights for policy 0, policy_version 238284 (0.0009) [2023-12-26 17:05:59,433][105620] Updated weights for policy 1, policy_version 238767 (0.0007) [2023-12-26 17:05:59,498][105620] Updated weights for policy 1, policy_version 238777 (0.0006) [2023-12-26 17:05:59,565][105620] Updated weights for policy 1, policy_version 238787 (0.0008) [2023-12-26 17:06:00,024][105692] Updated weights for policy 0, policy_version 238294 (0.0009) [2023-12-26 17:06:00,076][105692] Updated weights for policy 0, policy_version 238304 (0.0009) [2023-12-26 17:06:00,127][105692] Updated weights for policy 0, policy_version 238314 (0.0008) [2023-12-26 17:06:00,196][105620] Updated weights for policy 1, policy_version 238797 (0.0009) [2023-12-26 17:06:00,252][105620] Updated weights for policy 1, policy_version 238807 (0.0008) [2023-12-26 17:06:00,309][105620] Updated weights for policy 1, policy_version 238817 (0.0009) [2023-12-26 17:06:00,767][105692] Updated weights for policy 0, policy_version 238324 (0.0008) [2023-12-26 17:06:00,822][105692] Updated weights for policy 0, policy_version 238334 (0.0005) [2023-12-26 17:06:00,867][105692] Updated weights for policy 0, policy_version 238344 (0.0007) [2023-12-26 17:06:01,029][105620] Updated weights for policy 1, policy_version 238827 (0.0009) [2023-12-26 17:06:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 122175488. Throughput: 0: 9968.0, 1: 9708.2. Samples: 122146008. Policy #0 lag: (min: 31.0, avg: 31.7, max: 52.0) [2023-12-26 17:06:01,063][104569] Avg episode reward: [(0, '9266.736'), (1, '6221.324')] [2023-12-26 17:06:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000238352_61030400.pth... [2023-12-26 17:06:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000237200_60735488.pth [2023-12-26 17:06:01,091][105620] Updated weights for policy 1, policy_version 238837 (0.0007) [2023-12-26 17:06:01,155][105620] Updated weights for policy 1, policy_version 238847 (0.0007) [2023-12-26 17:06:01,211][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000238856_61153280.pth... [2023-12-26 17:06:01,215][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000237736_60866560.pth [2023-12-26 17:06:01,554][105692] Updated weights for policy 0, policy_version 238354 (0.0009) [2023-12-26 17:06:01,613][105692] Updated weights for policy 0, policy_version 238364 (0.0010) [2023-12-26 17:06:01,679][105692] Updated weights for policy 0, policy_version 238374 (0.0008) [2023-12-26 17:06:01,742][105692] Updated weights for policy 0, policy_version 238384 (0.0010) [2023-12-26 17:06:01,818][105620] Updated weights for policy 1, policy_version 238857 (0.0006) [2023-12-26 17:06:01,868][105620] Updated weights for policy 1, policy_version 238867 (0.0009) [2023-12-26 17:06:01,919][105620] Updated weights for policy 1, policy_version 238878 (0.0008) [2023-12-26 17:06:01,972][105620] Updated weights for policy 1, policy_version 238888 (0.0006) [2023-12-26 17:06:02,464][105692] Updated weights for policy 0, policy_version 238394 (0.0009) [2023-12-26 17:06:02,520][105692] Updated weights for policy 0, policy_version 238404 (0.0009) [2023-12-26 17:06:02,576][105692] Updated weights for policy 0, policy_version 238414 (0.0009) [2023-12-26 17:06:02,762][105620] Updated weights for policy 1, policy_version 238898 (0.0009) [2023-12-26 17:06:02,821][105620] Updated weights for policy 1, policy_version 238908 (0.0009) [2023-12-26 17:06:02,878][105620] Updated weights for policy 1, policy_version 238918 (0.0009) [2023-12-26 17:06:03,237][105692] Updated weights for policy 0, policy_version 238424 (0.0005) [2023-12-26 17:06:03,288][105692] Updated weights for policy 0, policy_version 238434 (0.0008) [2023-12-26 17:06:03,340][105692] Updated weights for policy 0, policy_version 238444 (0.0009) [2023-12-26 17:06:03,674][105620] Updated weights for policy 1, policy_version 238928 (0.0009) [2023-12-26 17:06:03,732][105620] Updated weights for policy 1, policy_version 238938 (0.0008) [2023-12-26 17:06:03,765][105586] KL-divergence is very high: 117.8305 [2023-12-26 17:06:03,792][105586] KL-divergence is very high: 124.8673 [2023-12-26 17:06:03,800][105620] Updated weights for policy 1, policy_version 238948 (0.0009) [2023-12-26 17:06:03,821][105586] KL-divergence is very high: 118.3189 [2023-12-26 17:06:04,134][105692] Updated weights for policy 0, policy_version 238454 (0.0007) [2023-12-26 17:06:04,197][105692] Updated weights for policy 0, policy_version 238464 (0.0008) [2023-12-26 17:06:04,260][105692] Updated weights for policy 0, policy_version 238474 (0.0009) [2023-12-26 17:06:04,558][105620] Updated weights for policy 1, policy_version 238958 (0.0009) [2023-12-26 17:06:04,619][105620] Updated weights for policy 1, policy_version 238968 (0.0008) [2023-12-26 17:06:04,685][105620] Updated weights for policy 1, policy_version 238978 (0.0009) [2023-12-26 17:06:04,988][105692] Updated weights for policy 0, policy_version 238484 (0.0009) [2023-12-26 17:06:05,047][105692] Updated weights for policy 0, policy_version 238494 (0.0009) [2023-12-26 17:06:05,108][105692] Updated weights for policy 0, policy_version 238504 (0.0009) [2023-12-26 17:06:05,446][105620] Updated weights for policy 1, policy_version 238988 (0.0009) [2023-12-26 17:06:05,506][105620] Updated weights for policy 1, policy_version 238998 (0.0009) [2023-12-26 17:06:05,564][105620] Updated weights for policy 1, policy_version 239008 (0.0009) [2023-12-26 17:06:05,799][105692] Updated weights for policy 0, policy_version 238514 (0.0008) [2023-12-26 17:06:05,862][105692] Updated weights for policy 0, policy_version 238524 (0.0007) [2023-12-26 17:06:05,925][105692] Updated weights for policy 0, policy_version 238534 (0.0008) [2023-12-26 17:06:05,983][105692] Updated weights for policy 0, policy_version 238544 (0.0005) [2023-12-26 17:06:06,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 122273792. Throughput: 0: 9896.4, 1: 9599.1. Samples: 122261096. Policy #0 lag: (min: 31.0, avg: 31.7, max: 52.0) [2023-12-26 17:06:06,062][104569] Avg episode reward: [(0, '9278.741'), (1, '7095.899')] [2023-12-26 17:06:06,359][105620] Updated weights for policy 1, policy_version 239018 (0.0009) [2023-12-26 17:06:06,425][105620] Updated weights for policy 1, policy_version 239028 (0.0005) [2023-12-26 17:06:06,496][105620] Updated weights for policy 1, policy_version 239038 (0.0005) [2023-12-26 17:06:06,557][105692] Updated weights for policy 0, policy_version 238554 (0.0010) [2023-12-26 17:06:06,561][105620] Updated weights for policy 1, policy_version 239048 (0.0005) [2023-12-26 17:06:06,616][105692] Updated weights for policy 0, policy_version 238564 (0.0010) [2023-12-26 17:06:06,671][105692] Updated weights for policy 0, policy_version 238574 (0.0010) [2023-12-26 17:06:07,129][105620] Updated weights for policy 1, policy_version 239058 (0.0009) [2023-12-26 17:06:07,190][105620] Updated weights for policy 1, policy_version 239068 (0.0009) [2023-12-26 17:06:07,251][105620] Updated weights for policy 1, policy_version 239078 (0.0009) [2023-12-26 17:06:07,473][105692] Updated weights for policy 0, policy_version 238584 (0.0009) [2023-12-26 17:06:07,521][105692] Updated weights for policy 0, policy_version 238594 (0.0009) [2023-12-26 17:06:07,590][105692] Updated weights for policy 0, policy_version 238604 (0.0007) [2023-12-26 17:06:07,950][105620] Updated weights for policy 1, policy_version 239088 (0.0009) [2023-12-26 17:06:08,005][105620] Updated weights for policy 1, policy_version 239099 (0.0010) [2023-12-26 17:06:08,059][105620] Updated weights for policy 1, policy_version 239111 (0.0010) [2023-12-26 17:06:08,209][105692] Updated weights for policy 0, policy_version 238614 (0.0008) [2023-12-26 17:06:08,269][105692] Updated weights for policy 0, policy_version 238624 (0.0009) [2023-12-26 17:06:08,334][105692] Updated weights for policy 0, policy_version 238634 (0.0009) [2023-12-26 17:06:08,874][105620] Updated weights for policy 1, policy_version 239121 (0.0009) [2023-12-26 17:06:08,942][105620] Updated weights for policy 1, policy_version 239131 (0.0008) [2023-12-26 17:06:09,004][105620] Updated weights for policy 1, policy_version 239141 (0.0008) [2023-12-26 17:06:09,063][105692] Updated weights for policy 0, policy_version 238644 (0.0009) [2023-12-26 17:06:09,122][105692] Updated weights for policy 0, policy_version 238654 (0.0008) [2023-12-26 17:06:09,192][105692] Updated weights for policy 0, policy_version 238664 (0.0005) [2023-12-26 17:06:09,765][105620] Updated weights for policy 1, policy_version 239151 (0.0008) [2023-12-26 17:06:09,830][105620] Updated weights for policy 1, policy_version 239161 (0.0010) [2023-12-26 17:06:09,890][105620] Updated weights for policy 1, policy_version 239171 (0.0009) [2023-12-26 17:06:09,916][105692] Updated weights for policy 0, policy_version 238674 (0.0008) [2023-12-26 17:06:09,985][105692] Updated weights for policy 0, policy_version 238684 (0.0009) [2023-12-26 17:06:10,051][105692] Updated weights for policy 0, policy_version 238694 (0.0010) [2023-12-26 17:06:10,108][105692] Updated weights for policy 0, policy_version 238704 (0.0009) [2023-12-26 17:06:10,655][105620] Updated weights for policy 1, policy_version 239181 (0.0008) [2023-12-26 17:06:10,717][105620] Updated weights for policy 1, policy_version 239191 (0.0007) [2023-12-26 17:06:10,787][105620] Updated weights for policy 1, policy_version 239201 (0.0009) [2023-12-26 17:06:10,846][105692] Updated weights for policy 0, policy_version 238714 (0.0007) [2023-12-26 17:06:10,908][105692] Updated weights for policy 0, policy_version 238724 (0.0008) [2023-12-26 17:06:10,971][105692] Updated weights for policy 0, policy_version 238734 (0.0008) [2023-12-26 17:06:11,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 122372096. Throughput: 0: 9908.3, 1: 9578.4. Samples: 122376656. Policy #0 lag: (min: 31.0, avg: 31.7, max: 52.0) [2023-12-26 17:06:11,062][104569] Avg episode reward: [(0, '9278.306'), (1, '8393.123')] [2023-12-26 17:06:11,543][105620] Updated weights for policy 1, policy_version 239211 (0.0009) [2023-12-26 17:06:11,600][105620] Updated weights for policy 1, policy_version 239221 (0.0009) [2023-12-26 17:06:11,661][105692] Updated weights for policy 0, policy_version 238744 (0.0008) [2023-12-26 17:06:11,663][105620] Updated weights for policy 1, policy_version 239231 (0.0007) [2023-12-26 17:06:11,729][105692] Updated weights for policy 0, policy_version 238754 (0.0008) [2023-12-26 17:06:11,790][105692] Updated weights for policy 0, policy_version 238764 (0.0008) [2023-12-26 17:06:12,420][105620] Updated weights for policy 1, policy_version 239241 (0.0008) [2023-12-26 17:06:12,476][105620] Updated weights for policy 1, policy_version 239251 (0.0010) [2023-12-26 17:06:12,520][105692] Updated weights for policy 0, policy_version 238774 (0.0007) [2023-12-26 17:06:12,536][105620] Updated weights for policy 1, policy_version 239261 (0.0007) [2023-12-26 17:06:12,579][105692] Updated weights for policy 0, policy_version 238784 (0.0007) [2023-12-26 17:06:12,597][105620] Updated weights for policy 1, policy_version 239271 (0.0007) [2023-12-26 17:06:12,638][105692] Updated weights for policy 0, policy_version 238794 (0.0008) [2023-12-26 17:06:13,332][105692] Updated weights for policy 0, policy_version 238804 (0.0006) [2023-12-26 17:06:13,334][105620] Updated weights for policy 1, policy_version 239281 (0.0008) [2023-12-26 17:06:13,382][105692] Updated weights for policy 0, policy_version 238814 (0.0006) [2023-12-26 17:06:13,384][105620] Updated weights for policy 1, policy_version 239291 (0.0006) [2023-12-26 17:06:13,427][105692] Updated weights for policy 0, policy_version 238824 (0.0006) [2023-12-26 17:06:13,429][105620] Updated weights for policy 1, policy_version 239301 (0.0006) [2023-12-26 17:06:14,121][105692] Updated weights for policy 0, policy_version 238834 (0.0007) [2023-12-26 17:06:14,176][105620] Updated weights for policy 1, policy_version 239311 (0.0006) [2023-12-26 17:06:14,177][105692] Updated weights for policy 0, policy_version 238844 (0.0008) [2023-12-26 17:06:14,225][105620] Updated weights for policy 1, policy_version 239321 (0.0009) [2023-12-26 17:06:14,232][105692] Updated weights for policy 0, policy_version 238854 (0.0010) [2023-12-26 17:06:14,280][105692] Updated weights for policy 0, policy_version 238864 (0.0010) [2023-12-26 17:06:14,282][105620] Updated weights for policy 1, policy_version 239331 (0.0006) [2023-12-26 17:06:14,957][105620] Updated weights for policy 1, policy_version 239341 (0.0006) [2023-12-26 17:06:14,963][105692] Updated weights for policy 0, policy_version 238874 (0.0010) [2023-12-26 17:06:15,017][105620] Updated weights for policy 1, policy_version 239351 (0.0006) [2023-12-26 17:06:15,019][105692] Updated weights for policy 0, policy_version 238884 (0.0011) [2023-12-26 17:06:15,078][105620] Updated weights for policy 1, policy_version 239361 (0.0007) [2023-12-26 17:06:15,082][105692] Updated weights for policy 0, policy_version 238894 (0.0011) [2023-12-26 17:06:15,833][105692] Updated weights for policy 0, policy_version 238904 (0.0010) [2023-12-26 17:06:15,836][105620] Updated weights for policy 1, policy_version 239371 (0.0007) [2023-12-26 17:06:15,891][105692] Updated weights for policy 0, policy_version 238914 (0.0010) [2023-12-26 17:06:15,893][105620] Updated weights for policy 1, policy_version 239381 (0.0005) [2023-12-26 17:06:15,955][105692] Updated weights for policy 0, policy_version 238924 (0.0010) [2023-12-26 17:06:15,961][105620] Updated weights for policy 1, policy_version 239391 (0.0006) [2023-12-26 17:06:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 122470400. Throughput: 0: 9824.1, 1: 9525.6. Samples: 122433400. Policy #0 lag: (min: 26.0, avg: 42.2, max: 58.0) [2023-12-26 17:06:16,062][104569] Avg episode reward: [(0, '9177.829'), (1, '8748.557')] [2023-12-26 17:06:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000238928_61177856.pth... [2023-12-26 17:06:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000239400_61292544.pth... [2023-12-26 17:06:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000237776_60882944.pth [2023-12-26 17:06:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000238312_61014016.pth [2023-12-26 17:06:16,537][105620] Updated weights for policy 1, policy_version 239401 (0.0006) [2023-12-26 17:06:16,588][105620] Updated weights for policy 1, policy_version 239411 (0.0005) [2023-12-26 17:06:16,641][105620] Updated weights for policy 1, policy_version 239421 (0.0006) [2023-12-26 17:06:16,686][105692] Updated weights for policy 0, policy_version 238934 (0.0007) [2023-12-26 17:06:16,704][105620] Updated weights for policy 1, policy_version 239431 (0.0005) [2023-12-26 17:06:16,751][105692] Updated weights for policy 0, policy_version 238944 (0.0008) [2023-12-26 17:06:16,810][105692] Updated weights for policy 0, policy_version 238954 (0.0007) [2023-12-26 17:06:17,324][105692] Updated weights for policy 0, policy_version 238964 (0.0005) [2023-12-26 17:06:17,337][105620] Updated weights for policy 1, policy_version 239441 (0.0008) [2023-12-26 17:06:17,377][105692] Updated weights for policy 0, policy_version 238974 (0.0010) [2023-12-26 17:06:17,386][105620] Updated weights for policy 1, policy_version 239451 (0.0009) [2023-12-26 17:06:17,434][105692] Updated weights for policy 0, policy_version 238984 (0.0009) [2023-12-26 17:06:17,441][105620] Updated weights for policy 1, policy_version 239461 (0.0007) [2023-12-26 17:06:17,965][105692] Updated weights for policy 0, policy_version 238994 (0.0005) [2023-12-26 17:06:18,030][105692] Updated weights for policy 0, policy_version 239004 (0.0005) [2023-12-26 17:06:18,084][105692] Updated weights for policy 0, policy_version 239014 (0.0005) [2023-12-26 17:06:18,151][105692] Updated weights for policy 0, policy_version 239024 (0.0008) [2023-12-26 17:06:18,170][105620] Updated weights for policy 1, policy_version 239471 (0.0008) [2023-12-26 17:06:18,221][105620] Updated weights for policy 1, policy_version 239481 (0.0007) [2023-12-26 17:06:18,267][105620] Updated weights for policy 1, policy_version 239491 (0.0005) [2023-12-26 17:06:18,822][105692] Updated weights for policy 0, policy_version 239034 (0.0010) [2023-12-26 17:06:18,880][105620] Updated weights for policy 1, policy_version 239501 (0.0008) [2023-12-26 17:06:18,881][105692] Updated weights for policy 0, policy_version 239044 (0.0011) [2023-12-26 17:06:18,932][105620] Updated weights for policy 1, policy_version 239511 (0.0010) [2023-12-26 17:06:18,940][105692] Updated weights for policy 0, policy_version 239054 (0.0010) [2023-12-26 17:06:18,980][105620] Updated weights for policy 1, policy_version 239521 (0.0010) [2023-12-26 17:06:19,647][105692] Updated weights for policy 0, policy_version 239064 (0.0010) [2023-12-26 17:06:19,676][105620] Updated weights for policy 1, policy_version 239531 (0.0009) [2023-12-26 17:06:19,710][105692] Updated weights for policy 0, policy_version 239074 (0.0010) [2023-12-26 17:06:19,745][105620] Updated weights for policy 1, policy_version 239541 (0.0006) [2023-12-26 17:06:19,776][105692] Updated weights for policy 0, policy_version 239084 (0.0011) [2023-12-26 17:06:19,809][105620] Updated weights for policy 1, policy_version 239551 (0.0006) [2023-12-26 17:06:20,471][105692] Updated weights for policy 0, policy_version 239094 (0.0008) [2023-12-26 17:06:20,527][105692] Updated weights for policy 0, policy_version 239104 (0.0009) [2023-12-26 17:06:20,554][105620] Updated weights for policy 1, policy_version 239561 (0.0008) [2023-12-26 17:06:20,589][105692] Updated weights for policy 0, policy_version 239114 (0.0009) [2023-12-26 17:06:20,626][105620] Updated weights for policy 1, policy_version 239571 (0.0008) [2023-12-26 17:06:20,682][105620] Updated weights for policy 1, policy_version 239581 (0.0008) [2023-12-26 17:06:20,739][105620] Updated weights for policy 1, policy_version 239591 (0.0007) [2023-12-26 17:06:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 122568704. Throughput: 0: 9949.4, 1: 9591.5. Samples: 122558328. Policy #0 lag: (min: 26.0, avg: 42.2, max: 58.0) [2023-12-26 17:06:21,062][104569] Avg episode reward: [(0, '9087.896'), (1, '8474.457')] [2023-12-26 17:06:21,343][105692] Updated weights for policy 0, policy_version 239124 (0.0007) [2023-12-26 17:06:21,412][105692] Updated weights for policy 0, policy_version 239134 (0.0008) [2023-12-26 17:06:21,480][105692] Updated weights for policy 0, policy_version 239144 (0.0008) [2023-12-26 17:06:21,499][105620] Updated weights for policy 1, policy_version 239601 (0.0006) [2023-12-26 17:06:21,559][105620] Updated weights for policy 1, policy_version 239611 (0.0006) [2023-12-26 17:06:21,628][105620] Updated weights for policy 1, policy_version 239621 (0.0006) [2023-12-26 17:06:22,155][105692] Updated weights for policy 0, policy_version 239154 (0.0010) [2023-12-26 17:06:22,211][105692] Updated weights for policy 0, policy_version 239164 (0.0011) [2023-12-26 17:06:22,271][105692] Updated weights for policy 0, policy_version 239174 (0.0008) [2023-12-26 17:06:22,329][105692] Updated weights for policy 0, policy_version 239184 (0.0005) [2023-12-26 17:06:22,419][105620] Updated weights for policy 1, policy_version 239631 (0.0008) [2023-12-26 17:06:22,475][105620] Updated weights for policy 1, policy_version 239641 (0.0009) [2023-12-26 17:06:22,534][105620] Updated weights for policy 1, policy_version 239651 (0.0008) [2023-12-26 17:06:22,970][105692] Updated weights for policy 0, policy_version 239194 (0.0006) [2023-12-26 17:06:23,030][105692] Updated weights for policy 0, policy_version 239204 (0.0005) [2023-12-26 17:06:23,088][105692] Updated weights for policy 0, policy_version 239214 (0.0006) [2023-12-26 17:06:23,368][105620] Updated weights for policy 1, policy_version 239661 (0.0008) [2023-12-26 17:06:23,424][105620] Updated weights for policy 1, policy_version 239671 (0.0008) [2023-12-26 17:06:23,477][105620] Updated weights for policy 1, policy_version 239681 (0.0008) [2023-12-26 17:06:23,731][105692] Updated weights for policy 0, policy_version 239224 (0.0009) [2023-12-26 17:06:23,791][105692] Updated weights for policy 0, policy_version 239234 (0.0010) [2023-12-26 17:06:23,843][105692] Updated weights for policy 0, policy_version 239244 (0.0010) [2023-12-26 17:06:24,168][105620] Updated weights for policy 1, policy_version 239691 (0.0009) [2023-12-26 17:06:24,232][105620] Updated weights for policy 1, policy_version 239701 (0.0008) [2023-12-26 17:06:24,277][105620] Updated weights for policy 1, policy_version 239711 (0.0008) [2023-12-26 17:06:24,576][105692] Updated weights for policy 0, policy_version 239254 (0.0010) [2023-12-26 17:06:24,632][105692] Updated weights for policy 0, policy_version 239264 (0.0010) [2023-12-26 17:06:24,680][105692] Updated weights for policy 0, policy_version 239274 (0.0010) [2023-12-26 17:06:25,003][105620] Updated weights for policy 1, policy_version 239721 (0.0008) [2023-12-26 17:06:25,061][105620] Updated weights for policy 1, policy_version 239731 (0.0008) [2023-12-26 17:06:25,117][105620] Updated weights for policy 1, policy_version 239741 (0.0008) [2023-12-26 17:06:25,164][105620] Updated weights for policy 1, policy_version 239751 (0.0008) [2023-12-26 17:06:25,429][105692] Updated weights for policy 0, policy_version 239284 (0.0010) [2023-12-26 17:06:25,478][105692] Updated weights for policy 0, policy_version 239294 (0.0007) [2023-12-26 17:06:25,534][105692] Updated weights for policy 0, policy_version 239304 (0.0005) [2023-12-26 17:06:25,994][105620] Updated weights for policy 1, policy_version 239761 (0.0009) [2023-12-26 17:06:26,052][105620] Updated weights for policy 1, policy_version 239771 (0.0008) [2023-12-26 17:06:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 122658816. Throughput: 0: 9907.8, 1: 9466.6. Samples: 122671920. Policy #0 lag: (min: 26.0, avg: 42.2, max: 58.0) [2023-12-26 17:06:26,062][104569] Avg episode reward: [(0, '9264.728'), (1, '8648.321')] [2023-12-26 17:06:26,107][105620] Updated weights for policy 1, policy_version 239781 (0.0008) [2023-12-26 17:06:26,115][105692] Updated weights for policy 0, policy_version 239314 (0.0006) [2023-12-26 17:06:26,178][105692] Updated weights for policy 0, policy_version 239324 (0.0011) [2023-12-26 17:06:26,234][105692] Updated weights for policy 0, policy_version 239334 (0.0011) [2023-12-26 17:06:26,286][105692] Updated weights for policy 0, policy_version 239344 (0.0011) [2023-12-26 17:06:26,852][105620] Updated weights for policy 1, policy_version 239791 (0.0008) [2023-12-26 17:06:26,905][105692] Updated weights for policy 0, policy_version 239354 (0.0005) [2023-12-26 17:06:26,907][105620] Updated weights for policy 1, policy_version 239801 (0.0008) [2023-12-26 17:06:26,953][105692] Updated weights for policy 0, policy_version 239364 (0.0006) [2023-12-26 17:06:26,963][105620] Updated weights for policy 1, policy_version 239811 (0.0008) [2023-12-26 17:06:27,008][105692] Updated weights for policy 0, policy_version 239374 (0.0006) [2023-12-26 17:06:27,595][105620] Updated weights for policy 1, policy_version 239821 (0.0009) [2023-12-26 17:06:27,644][105692] Updated weights for policy 0, policy_version 239384 (0.0006) [2023-12-26 17:06:27,647][105620] Updated weights for policy 1, policy_version 239831 (0.0009) [2023-12-26 17:06:27,691][105692] Updated weights for policy 0, policy_version 239394 (0.0005) [2023-12-26 17:06:27,694][105620] Updated weights for policy 1, policy_version 239841 (0.0008) [2023-12-26 17:06:27,744][105692] Updated weights for policy 0, policy_version 239404 (0.0005) [2023-12-26 17:06:28,328][105692] Updated weights for policy 0, policy_version 239414 (0.0008) [2023-12-26 17:06:28,394][105692] Updated weights for policy 0, policy_version 239424 (0.0011) [2023-12-26 17:06:28,454][105692] Updated weights for policy 0, policy_version 239434 (0.0009) [2023-12-26 17:06:28,490][105620] Updated weights for policy 1, policy_version 239851 (0.0009) [2023-12-26 17:06:28,544][105620] Updated weights for policy 1, policy_version 239861 (0.0008) [2023-12-26 17:06:28,606][105620] Updated weights for policy 1, policy_version 239871 (0.0008) [2023-12-26 17:06:29,082][105692] Updated weights for policy 0, policy_version 239444 (0.0008) [2023-12-26 17:06:29,143][105692] Updated weights for policy 0, policy_version 239454 (0.0005) [2023-12-26 17:06:29,189][105692] Updated weights for policy 0, policy_version 239464 (0.0005) [2023-12-26 17:06:29,456][105620] Updated weights for policy 1, policy_version 239881 (0.0009) [2023-12-26 17:06:29,517][105620] Updated weights for policy 1, policy_version 239891 (0.0009) [2023-12-26 17:06:29,581][105620] Updated weights for policy 1, policy_version 239901 (0.0009) [2023-12-26 17:06:29,642][105620] Updated weights for policy 1, policy_version 239912 (0.0010) [2023-12-26 17:06:29,801][105692] Updated weights for policy 0, policy_version 239474 (0.0008) [2023-12-26 17:06:29,869][105692] Updated weights for policy 0, policy_version 239484 (0.0008) [2023-12-26 17:06:29,937][105692] Updated weights for policy 0, policy_version 239494 (0.0007) [2023-12-26 17:06:29,996][105692] Updated weights for policy 0, policy_version 239504 (0.0006) [2023-12-26 17:06:30,510][105620] Updated weights for policy 1, policy_version 239922 (0.0008) [2023-12-26 17:06:30,527][105692] Updated weights for policy 0, policy_version 239514 (0.0007) [2023-12-26 17:06:30,563][105620] Updated weights for policy 1, policy_version 239932 (0.0006) [2023-12-26 17:06:30,594][105692] Updated weights for policy 0, policy_version 239524 (0.0005) [2023-12-26 17:06:30,612][105620] Updated weights for policy 1, policy_version 239942 (0.0006) [2023-12-26 17:06:30,658][105692] Updated weights for policy 0, policy_version 239534 (0.0005) [2023-12-26 17:06:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 122765312. Throughput: 0: 10021.2, 1: 9483.3. Samples: 122733824. Policy #0 lag: (min: 26.0, avg: 42.2, max: 58.0) [2023-12-26 17:06:31,062][104569] Avg episode reward: [(0, '9181.296'), (1, '9182.378')] [2023-12-26 17:06:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000239944_61431808.pth... [2023-12-26 17:06:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000239536_61333504.pth... [2023-12-26 17:06:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000238352_61030400.pth [2023-12-26 17:06:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000238856_61153280.pth [2023-12-26 17:06:31,292][105692] Updated weights for policy 0, policy_version 239544 (0.0007) [2023-12-26 17:06:31,298][105620] Updated weights for policy 1, policy_version 239952 (0.0007) [2023-12-26 17:06:31,341][105692] Updated weights for policy 0, policy_version 239554 (0.0007) [2023-12-26 17:06:31,361][105620] Updated weights for policy 1, policy_version 239962 (0.0008) [2023-12-26 17:06:31,397][105692] Updated weights for policy 0, policy_version 239564 (0.0008) [2023-12-26 17:06:31,424][105620] Updated weights for policy 1, policy_version 239972 (0.0007) [2023-12-26 17:06:32,126][105692] Updated weights for policy 0, policy_version 239574 (0.0008) [2023-12-26 17:06:32,165][105620] Updated weights for policy 1, policy_version 239982 (0.0008) [2023-12-26 17:06:32,178][105586] KL-divergence is very high: 112.7055 [2023-12-26 17:06:32,184][105692] Updated weights for policy 0, policy_version 239584 (0.0006) [2023-12-26 17:06:32,217][105586] KL-divergence is very high: 181.4057 [2023-12-26 17:06:32,217][105620] Updated weights for policy 1, policy_version 239992 (0.0010) [2023-12-26 17:06:32,248][105692] Updated weights for policy 0, policy_version 239594 (0.0006) [2023-12-26 17:06:32,266][105586] KL-divergence is very high: 162.1762 [2023-12-26 17:06:32,281][105620] Updated weights for policy 1, policy_version 240002 (0.0008) [2023-12-26 17:06:32,939][105692] Updated weights for policy 0, policy_version 239604 (0.0008) [2023-12-26 17:06:33,000][105692] Updated weights for policy 0, policy_version 239614 (0.0009) [2023-12-26 17:06:33,052][105620] Updated weights for policy 1, policy_version 240012 (0.0007) [2023-12-26 17:06:33,061][105692] Updated weights for policy 0, policy_version 239624 (0.0008) [2023-12-26 17:06:33,106][105620] Updated weights for policy 1, policy_version 240022 (0.0006) [2023-12-26 17:06:33,168][105620] Updated weights for policy 1, policy_version 240032 (0.0008) [2023-12-26 17:06:33,720][105692] Updated weights for policy 0, policy_version 239634 (0.0008) [2023-12-26 17:06:33,775][105692] Updated weights for policy 0, policy_version 239644 (0.0010) [2023-12-26 17:06:33,836][105692] Updated weights for policy 0, policy_version 239654 (0.0010) [2023-12-26 17:06:33,901][105692] Updated weights for policy 0, policy_version 239664 (0.0011) [2023-12-26 17:06:33,943][105620] Updated weights for policy 1, policy_version 240043 (0.0009) [2023-12-26 17:06:34,001][105620] Updated weights for policy 1, policy_version 240053 (0.0005) [2023-12-26 17:06:34,061][105620] Updated weights for policy 1, policy_version 240063 (0.0010) [2023-12-26 17:06:34,595][105692] Updated weights for policy 0, policy_version 239674 (0.0006) [2023-12-26 17:06:34,662][105692] Updated weights for policy 0, policy_version 239684 (0.0008) [2023-12-26 17:06:34,714][105692] Updated weights for policy 0, policy_version 239694 (0.0011) [2023-12-26 17:06:34,752][105620] Updated weights for policy 1, policy_version 240073 (0.0010) [2023-12-26 17:06:34,815][105620] Updated weights for policy 1, policy_version 240083 (0.0011) [2023-12-26 17:06:34,881][105620] Updated weights for policy 1, policy_version 240093 (0.0011) [2023-12-26 17:06:34,950][105620] Updated weights for policy 1, policy_version 240103 (0.0010) [2023-12-26 17:06:35,380][105692] Updated weights for policy 0, policy_version 239704 (0.0006) [2023-12-26 17:06:35,436][105692] Updated weights for policy 0, policy_version 239714 (0.0005) [2023-12-26 17:06:35,491][105692] Updated weights for policy 0, policy_version 239724 (0.0008) [2023-12-26 17:06:35,570][105620] Updated weights for policy 1, policy_version 240113 (0.0010) [2023-12-26 17:06:35,625][105620] Updated weights for policy 1, policy_version 240123 (0.0010) [2023-12-26 17:06:35,694][105620] Updated weights for policy 1, policy_version 240133 (0.0010) [2023-12-26 17:06:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 122863616. Throughput: 0: 10165.5, 1: 9374.6. Samples: 122852236. Policy #0 lag: (min: 26.0, avg: 42.2, max: 58.0) [2023-12-26 17:06:36,063][104569] Avg episode reward: [(0, '9005.268'), (1, '9000.382')] [2023-12-26 17:06:36,165][105692] Updated weights for policy 0, policy_version 239734 (0.0008) [2023-12-26 17:06:36,228][105692] Updated weights for policy 0, policy_version 239744 (0.0009) [2023-12-26 17:06:36,298][105692] Updated weights for policy 0, policy_version 239754 (0.0009) [2023-12-26 17:06:36,431][105620] Updated weights for policy 1, policy_version 240143 (0.0009) [2023-12-26 17:06:36,494][105620] Updated weights for policy 1, policy_version 240153 (0.0009) [2023-12-26 17:06:36,553][105620] Updated weights for policy 1, policy_version 240163 (0.0009) [2023-12-26 17:06:37,030][105692] Updated weights for policy 0, policy_version 239764 (0.0007) [2023-12-26 17:06:37,091][105692] Updated weights for policy 0, policy_version 239774 (0.0009) [2023-12-26 17:06:37,157][105692] Updated weights for policy 0, policy_version 239784 (0.0009) [2023-12-26 17:06:37,283][105620] Updated weights for policy 1, policy_version 240173 (0.0009) [2023-12-26 17:06:37,341][105620] Updated weights for policy 1, policy_version 240183 (0.0009) [2023-12-26 17:06:37,406][105620] Updated weights for policy 1, policy_version 240193 (0.0009) [2023-12-26 17:06:37,890][105692] Updated weights for policy 0, policy_version 239794 (0.0009) [2023-12-26 17:06:37,948][105692] Updated weights for policy 0, policy_version 239804 (0.0008) [2023-12-26 17:06:38,000][105692] Updated weights for policy 0, policy_version 239814 (0.0007) [2023-12-26 17:06:38,059][105692] Updated weights for policy 0, policy_version 239824 (0.0008) [2023-12-26 17:06:38,111][105620] Updated weights for policy 1, policy_version 240203 (0.0009) [2023-12-26 17:06:38,162][105620] Updated weights for policy 1, policy_version 240213 (0.0009) [2023-12-26 17:06:38,211][105620] Updated weights for policy 1, policy_version 240223 (0.0007) [2023-12-26 17:06:38,819][105692] Updated weights for policy 0, policy_version 239834 (0.0009) [2023-12-26 17:06:38,870][105692] Updated weights for policy 0, policy_version 239844 (0.0009) [2023-12-26 17:06:38,924][105692] Updated weights for policy 0, policy_version 239854 (0.0009) [2023-12-26 17:06:38,943][105620] Updated weights for policy 1, policy_version 240233 (0.0006) [2023-12-26 17:06:38,991][105620] Updated weights for policy 1, policy_version 240243 (0.0009) [2023-12-26 17:06:39,043][105620] Updated weights for policy 1, policy_version 240253 (0.0010) [2023-12-26 17:06:39,098][105620] Updated weights for policy 1, policy_version 240263 (0.0010) [2023-12-26 17:06:39,659][105692] Updated weights for policy 0, policy_version 239864 (0.0008) [2023-12-26 17:06:39,725][105692] Updated weights for policy 0, policy_version 239874 (0.0006) [2023-12-26 17:06:39,776][105692] Updated weights for policy 0, policy_version 239884 (0.0009) [2023-12-26 17:06:39,863][105620] Updated weights for policy 1, policy_version 240273 (0.0009) [2023-12-26 17:06:39,927][105620] Updated weights for policy 1, policy_version 240283 (0.0009) [2023-12-26 17:06:39,993][105620] Updated weights for policy 1, policy_version 240293 (0.0008) [2023-12-26 17:06:40,467][105692] Updated weights for policy 0, policy_version 239894 (0.0007) [2023-12-26 17:06:40,533][105692] Updated weights for policy 0, policy_version 239904 (0.0006) [2023-12-26 17:06:40,591][105692] Updated weights for policy 0, policy_version 239914 (0.0009) [2023-12-26 17:06:40,716][105620] Updated weights for policy 1, policy_version 240303 (0.0007) [2023-12-26 17:06:40,783][105620] Updated weights for policy 1, policy_version 240313 (0.0006) [2023-12-26 17:06:40,846][105620] Updated weights for policy 1, policy_version 240323 (0.0008) [2023-12-26 17:06:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 122961920. Throughput: 0: 10102.4, 1: 9487.2. Samples: 122968724. Policy #0 lag: (min: 26.0, avg: 42.2, max: 58.0) [2023-12-26 17:06:41,062][104569] Avg episode reward: [(0, '8831.772'), (1, '8821.075')] [2023-12-26 17:06:41,367][105692] Updated weights for policy 0, policy_version 239924 (0.0010) [2023-12-26 17:06:41,435][105692] Updated weights for policy 0, policy_version 239934 (0.0012) [2023-12-26 17:06:41,503][105692] Updated weights for policy 0, policy_version 239944 (0.0007) [2023-12-26 17:06:41,617][105620] Updated weights for policy 1, policy_version 240333 (0.0008) [2023-12-26 17:06:41,684][105620] Updated weights for policy 1, policy_version 240343 (0.0009) [2023-12-26 17:06:41,760][105620] Updated weights for policy 1, policy_version 240353 (0.0010) [2023-12-26 17:06:42,281][105692] Updated weights for policy 0, policy_version 239954 (0.0007) [2023-12-26 17:06:42,343][105692] Updated weights for policy 0, policy_version 239964 (0.0009) [2023-12-26 17:06:42,411][105692] Updated weights for policy 0, policy_version 239974 (0.0009) [2023-12-26 17:06:42,469][105692] Updated weights for policy 0, policy_version 239984 (0.0009) [2023-12-26 17:06:42,491][105620] Updated weights for policy 1, policy_version 240363 (0.0006) [2023-12-26 17:06:42,538][105620] Updated weights for policy 1, policy_version 240373 (0.0007) [2023-12-26 17:06:42,599][105620] Updated weights for policy 1, policy_version 240383 (0.0009) [2023-12-26 17:06:43,148][105692] Updated weights for policy 0, policy_version 239994 (0.0008) [2023-12-26 17:06:43,204][105692] Updated weights for policy 0, policy_version 240004 (0.0009) [2023-12-26 17:06:43,267][105692] Updated weights for policy 0, policy_version 240014 (0.0010) [2023-12-26 17:06:43,279][105620] Updated weights for policy 1, policy_version 240393 (0.0008) [2023-12-26 17:06:43,339][105620] Updated weights for policy 1, policy_version 240403 (0.0006) [2023-12-26 17:06:43,400][105620] Updated weights for policy 1, policy_version 240413 (0.0008) [2023-12-26 17:06:43,459][105586] KL-divergence is very high: 122.6202 [2023-12-26 17:06:43,465][105620] Updated weights for policy 1, policy_version 240423 (0.0009) [2023-12-26 17:06:44,012][105692] Updated weights for policy 0, policy_version 240024 (0.0009) [2023-12-26 17:06:44,064][105692] Updated weights for policy 0, policy_version 240034 (0.0008) [2023-12-26 17:06:44,086][105620] Updated weights for policy 1, policy_version 240433 (0.0005) [2023-12-26 17:06:44,122][105692] Updated weights for policy 0, policy_version 240044 (0.0007) [2023-12-26 17:06:44,139][105620] Updated weights for policy 1, policy_version 240443 (0.0005) [2023-12-26 17:06:44,187][105620] Updated weights for policy 1, policy_version 240453 (0.0009) [2023-12-26 17:06:44,903][105692] Updated weights for policy 0, policy_version 240054 (0.0008) [2023-12-26 17:06:44,929][105620] Updated weights for policy 1, policy_version 240463 (0.0009) [2023-12-26 17:06:44,936][105586] KL-divergence is very high: 280.4312 [2023-12-26 17:06:44,948][105586] KL-divergence is very high: 303.2917 [2023-12-26 17:06:44,954][105586] KL-divergence is very high: 186.1536 [2023-12-26 17:06:44,964][105692] Updated weights for policy 0, policy_version 240064 (0.0007) [2023-12-26 17:06:44,986][105586] KL-divergence is very high: 426.6007 [2023-12-26 17:06:44,993][105620] Updated weights for policy 1, policy_version 240473 (0.0010) [2023-12-26 17:06:45,000][105586] KL-divergence is very high: 368.0329 [2023-12-26 17:06:45,007][105586] KL-divergence is very high: 187.3943 [2023-12-26 17:06:45,022][105692] Updated weights for policy 0, policy_version 240074 (0.0007) [2023-12-26 17:06:45,037][105586] KL-divergence is very high: 365.1619 [2023-12-26 17:06:45,049][105586] KL-divergence is very high: 303.0868 [2023-12-26 17:06:45,057][105620] Updated weights for policy 1, policy_version 240483 (0.0007) [2023-12-26 17:06:45,057][105586] KL-divergence is very high: 111.7237 [2023-12-26 17:06:45,709][105692] Updated weights for policy 0, policy_version 240084 (0.0006) [2023-12-26 17:06:45,764][105692] Updated weights for policy 0, policy_version 240094 (0.0005) [2023-12-26 17:06:45,818][105692] Updated weights for policy 0, policy_version 240104 (0.0005) [2023-12-26 17:06:45,851][105620] Updated weights for policy 1, policy_version 240493 (0.0009) [2023-12-26 17:06:45,856][105586] KL-divergence is very high: 259.6224 [2023-12-26 17:06:45,903][105586] KL-divergence is very high: 243.6174 [2023-12-26 17:06:45,911][105620] Updated weights for policy 1, policy_version 240503 (0.0009) [2023-12-26 17:06:45,955][105586] KL-divergence is very high: 206.6920 [2023-12-26 17:06:45,972][105620] Updated weights for policy 1, policy_version 240513 (0.0009) [2023-12-26 17:06:45,999][105586] KL-divergence is very high: 186.7812 [2023-12-26 17:06:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 123060224. Throughput: 0: 10006.0, 1: 9535.5. Samples: 123025376. Policy #0 lag: (min: 26.0, avg: 42.2, max: 58.0) [2023-12-26 17:06:46,062][104569] Avg episode reward: [(0, '8744.367'), (1, '8287.537')] [2023-12-26 17:06:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000240520_61579264.pth... [2023-12-26 17:06:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000240112_61480960.pth... [2023-12-26 17:06:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000239400_61292544.pth [2023-12-26 17:06:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000238928_61177856.pth [2023-12-26 17:06:46,387][105692] Updated weights for policy 0, policy_version 240114 (0.0006) [2023-12-26 17:06:46,449][105692] Updated weights for policy 0, policy_version 240124 (0.0010) [2023-12-26 17:06:46,504][105692] Updated weights for policy 0, policy_version 240134 (0.0010) [2023-12-26 17:06:46,570][105692] Updated weights for policy 0, policy_version 240144 (0.0010) [2023-12-26 17:06:46,791][105620] Updated weights for policy 1, policy_version 240523 (0.0009) [2023-12-26 17:06:46,849][105620] Updated weights for policy 1, policy_version 240533 (0.0008) [2023-12-26 17:06:46,900][105620] Updated weights for policy 1, policy_version 240543 (0.0008) [2023-12-26 17:06:47,284][105692] Updated weights for policy 0, policy_version 240154 (0.0010) [2023-12-26 17:06:47,345][105692] Updated weights for policy 0, policy_version 240164 (0.0010) [2023-12-26 17:06:47,410][105692] Updated weights for policy 0, policy_version 240174 (0.0010) [2023-12-26 17:06:47,641][105620] Updated weights for policy 1, policy_version 240553 (0.0008) [2023-12-26 17:06:47,695][105620] Updated weights for policy 1, policy_version 240563 (0.0010) [2023-12-26 17:06:47,743][105620] Updated weights for policy 1, policy_version 240573 (0.0010) [2023-12-26 17:06:47,788][105620] Updated weights for policy 1, policy_version 240583 (0.0007) [2023-12-26 17:06:48,090][105692] Updated weights for policy 0, policy_version 240184 (0.0009) [2023-12-26 17:06:48,154][105692] Updated weights for policy 0, policy_version 240194 (0.0008) [2023-12-26 17:06:48,218][105692] Updated weights for policy 0, policy_version 240204 (0.0007) [2023-12-26 17:06:48,536][105620] Updated weights for policy 1, policy_version 240593 (0.0008) [2023-12-26 17:06:48,590][105620] Updated weights for policy 1, policy_version 240603 (0.0007) [2023-12-26 17:06:48,635][105620] Updated weights for policy 1, policy_version 240613 (0.0006) [2023-12-26 17:06:48,871][105692] Updated weights for policy 0, policy_version 240214 (0.0008) [2023-12-26 17:06:48,934][105692] Updated weights for policy 0, policy_version 240224 (0.0010) [2023-12-26 17:06:48,996][105692] Updated weights for policy 0, policy_version 240234 (0.0009) [2023-12-26 17:06:49,380][105620] Updated weights for policy 1, policy_version 240623 (0.0009) [2023-12-26 17:06:49,429][105620] Updated weights for policy 1, policy_version 240633 (0.0008) [2023-12-26 17:06:49,476][105620] Updated weights for policy 1, policy_version 240643 (0.0009) [2023-12-26 17:06:49,736][105692] Updated weights for policy 0, policy_version 240244 (0.0010) [2023-12-26 17:06:49,797][105692] Updated weights for policy 0, policy_version 240254 (0.0008) [2023-12-26 17:06:49,864][105692] Updated weights for policy 0, policy_version 240264 (0.0010) [2023-12-26 17:06:50,258][105620] Updated weights for policy 1, policy_version 240653 (0.0010) [2023-12-26 17:06:50,314][105620] Updated weights for policy 1, policy_version 240663 (0.0008) [2023-12-26 17:06:50,372][105620] Updated weights for policy 1, policy_version 240673 (0.0009) [2023-12-26 17:06:50,638][105692] Updated weights for policy 0, policy_version 240274 (0.0012) [2023-12-26 17:06:50,687][105692] Updated weights for policy 0, policy_version 240284 (0.0009) [2023-12-26 17:06:50,735][105692] Updated weights for policy 0, policy_version 240294 (0.0009) [2023-12-26 17:06:50,794][105692] Updated weights for policy 0, policy_version 240304 (0.0009) [2023-12-26 17:06:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 123150336. Throughput: 0: 10067.2, 1: 9495.2. Samples: 123141404. Policy #0 lag: (min: 26.0, avg: 42.2, max: 58.0) [2023-12-26 17:06:51,062][104569] Avg episode reward: [(0, '9091.754'), (1, '8196.278')] [2023-12-26 17:06:51,122][105620] Updated weights for policy 1, policy_version 240683 (0.0008) [2023-12-26 17:06:51,183][105620] Updated weights for policy 1, policy_version 240693 (0.0008) [2023-12-26 17:06:51,243][105620] Updated weights for policy 1, policy_version 240703 (0.0010) [2023-12-26 17:06:51,577][105692] Updated weights for policy 0, policy_version 240314 (0.0009) [2023-12-26 17:06:51,639][105692] Updated weights for policy 0, policy_version 240324 (0.0010) [2023-12-26 17:06:51,698][105692] Updated weights for policy 0, policy_version 240335 (0.0009) [2023-12-26 17:06:52,023][105620] Updated weights for policy 1, policy_version 240713 (0.0009) [2023-12-26 17:06:52,083][105620] Updated weights for policy 1, policy_version 240723 (0.0008) [2023-12-26 17:06:52,149][105620] Updated weights for policy 1, policy_version 240733 (0.0009) [2023-12-26 17:06:52,220][105620] Updated weights for policy 1, policy_version 240743 (0.0009) [2023-12-26 17:06:52,504][105692] Updated weights for policy 0, policy_version 240345 (0.0009) [2023-12-26 17:06:52,564][105692] Updated weights for policy 0, policy_version 240355 (0.0009) [2023-12-26 17:06:52,624][105692] Updated weights for policy 0, policy_version 240365 (0.0009) [2023-12-26 17:06:52,985][105620] Updated weights for policy 1, policy_version 240753 (0.0010) [2023-12-26 17:06:53,057][105620] Updated weights for policy 1, policy_version 240763 (0.0008) [2023-12-26 17:06:53,113][105620] Updated weights for policy 1, policy_version 240773 (0.0009) [2023-12-26 17:06:53,353][105692] Updated weights for policy 0, policy_version 240375 (0.0009) [2023-12-26 17:06:53,405][105692] Updated weights for policy 0, policy_version 240385 (0.0009) [2023-12-26 17:06:53,453][105692] Updated weights for policy 0, policy_version 240395 (0.0009) [2023-12-26 17:06:53,842][105620] Updated weights for policy 1, policy_version 240783 (0.0008) [2023-12-26 17:06:53,904][105620] Updated weights for policy 1, policy_version 240793 (0.0008) [2023-12-26 17:06:53,948][105620] Updated weights for policy 1, policy_version 240803 (0.0007) [2023-12-26 17:06:54,221][105692] Updated weights for policy 0, policy_version 240405 (0.0009) [2023-12-26 17:06:54,281][105692] Updated weights for policy 0, policy_version 240415 (0.0009) [2023-12-26 17:06:54,336][105692] Updated weights for policy 0, policy_version 240425 (0.0009) [2023-12-26 17:06:54,698][105620] Updated weights for policy 1, policy_version 240813 (0.0008) [2023-12-26 17:06:54,749][105620] Updated weights for policy 1, policy_version 240823 (0.0010) [2023-12-26 17:06:54,801][105620] Updated weights for policy 1, policy_version 240833 (0.0009) [2023-12-26 17:06:55,010][105692] Updated weights for policy 0, policy_version 240435 (0.0008) [2023-12-26 17:06:55,066][105692] Updated weights for policy 0, policy_version 240445 (0.0007) [2023-12-26 17:06:55,123][105692] Updated weights for policy 0, policy_version 240455 (0.0005) [2023-12-26 17:06:55,698][105620] Updated weights for policy 1, policy_version 240843 (0.0010) [2023-12-26 17:06:55,703][105692] Updated weights for policy 0, policy_version 240465 (0.0007) [2023-12-26 17:06:55,749][105620] Updated weights for policy 1, policy_version 240853 (0.0009) [2023-12-26 17:06:55,758][105692] Updated weights for policy 0, policy_version 240475 (0.0010) [2023-12-26 17:06:55,803][105620] Updated weights for policy 1, policy_version 240863 (0.0008) [2023-12-26 17:06:55,811][105692] Updated weights for policy 0, policy_version 240485 (0.0007) [2023-12-26 17:06:55,868][105692] Updated weights for policy 0, policy_version 240495 (0.0008) [2023-12-26 17:06:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 123248640. Throughput: 0: 10030.5, 1: 9438.5. Samples: 123252768. Policy #0 lag: (min: 26.0, avg: 42.2, max: 58.0) [2023-12-26 17:06:56,063][104569] Avg episode reward: [(0, '9178.530'), (1, '8108.310')] [2023-12-26 17:06:56,435][105692] Updated weights for policy 0, policy_version 240505 (0.0006) [2023-12-26 17:06:56,490][105692] Updated weights for policy 0, policy_version 240515 (0.0006) [2023-12-26 17:06:56,549][105692] Updated weights for policy 0, policy_version 240525 (0.0005) [2023-12-26 17:06:56,661][105620] Updated weights for policy 1, policy_version 240873 (0.0009) [2023-12-26 17:06:56,721][105620] Updated weights for policy 1, policy_version 240883 (0.0009) [2023-12-26 17:06:56,770][105620] Updated weights for policy 1, policy_version 240893 (0.0009) [2023-12-26 17:06:56,820][105620] Updated weights for policy 1, policy_version 240903 (0.0008) [2023-12-26 17:06:57,129][105692] Updated weights for policy 0, policy_version 240535 (0.0005) [2023-12-26 17:06:57,182][105692] Updated weights for policy 0, policy_version 240545 (0.0005) [2023-12-26 17:06:57,243][105692] Updated weights for policy 0, policy_version 240555 (0.0005) [2023-12-26 17:06:57,687][105620] Updated weights for policy 1, policy_version 240913 (0.0010) [2023-12-26 17:06:57,739][105692] Updated weights for policy 0, policy_version 240565 (0.0006) [2023-12-26 17:06:57,740][105620] Updated weights for policy 1, policy_version 240923 (0.0007) [2023-12-26 17:06:57,788][105692] Updated weights for policy 0, policy_version 240575 (0.0006) [2023-12-26 17:06:57,792][105620] Updated weights for policy 1, policy_version 240933 (0.0007) [2023-12-26 17:06:57,834][105692] Updated weights for policy 0, policy_version 240585 (0.0005) [2023-12-26 17:06:58,539][105692] Updated weights for policy 0, policy_version 240595 (0.0006) [2023-12-26 17:06:58,599][105692] Updated weights for policy 0, policy_version 240605 (0.0009) [2023-12-26 17:06:58,650][105620] Updated weights for policy 1, policy_version 240943 (0.0008) [2023-12-26 17:06:58,653][105692] Updated weights for policy 0, policy_version 240615 (0.0006) [2023-12-26 17:06:58,711][105620] Updated weights for policy 1, policy_version 240953 (0.0008) [2023-12-26 17:06:58,776][105620] Updated weights for policy 1, policy_version 240963 (0.0008) [2023-12-26 17:06:59,376][105692] Updated weights for policy 0, policy_version 240625 (0.0007) [2023-12-26 17:06:59,430][105692] Updated weights for policy 0, policy_version 240635 (0.0009) [2023-12-26 17:06:59,484][105692] Updated weights for policy 0, policy_version 240645 (0.0009) [2023-12-26 17:06:59,538][105692] Updated weights for policy 0, policy_version 240655 (0.0009) [2023-12-26 17:06:59,571][105620] Updated weights for policy 1, policy_version 240973 (0.0009) [2023-12-26 17:06:59,622][105620] Updated weights for policy 1, policy_version 240983 (0.0008) [2023-12-26 17:06:59,679][105620] Updated weights for policy 1, policy_version 240993 (0.0006) [2023-12-26 17:07:00,360][105620] Updated weights for policy 1, policy_version 241003 (0.0006) [2023-12-26 17:07:00,395][105692] Updated weights for policy 0, policy_version 240665 (0.0008) [2023-12-26 17:07:00,422][105620] Updated weights for policy 1, policy_version 241013 (0.0006) [2023-12-26 17:07:00,449][105692] Updated weights for policy 0, policy_version 240675 (0.0007) [2023-12-26 17:07:00,480][105620] Updated weights for policy 1, policy_version 241023 (0.0006) [2023-12-26 17:07:00,504][105692] Updated weights for policy 0, policy_version 240685 (0.0007) [2023-12-26 17:07:01,004][105620] Updated weights for policy 1, policy_version 241033 (0.0006) [2023-12-26 17:07:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 123338752. Throughput: 0: 10151.7, 1: 9382.6. Samples: 123312448. Policy #0 lag: (min: 26.0, avg: 42.2, max: 58.0) [2023-12-26 17:07:01,063][104569] Avg episode reward: [(0, '8641.984'), (1, '8906.030')] [2023-12-26 17:07:01,068][105620] Updated weights for policy 1, policy_version 241043 (0.0007) [2023-12-26 17:07:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000240688_61628416.pth... [2023-12-26 17:07:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000239536_61333504.pth [2023-12-26 17:07:01,130][105620] Updated weights for policy 1, policy_version 241053 (0.0010) [2023-12-26 17:07:01,180][105620] Updated weights for policy 1, policy_version 241063 (0.0012) [2023-12-26 17:07:01,183][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000241064_61718528.pth... [2023-12-26 17:07:01,187][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000239944_61431808.pth [2023-12-26 17:07:01,370][105692] Updated weights for policy 0, policy_version 240695 (0.0009) [2023-12-26 17:07:01,424][105692] Updated weights for policy 0, policy_version 240705 (0.0008) [2023-12-26 17:07:01,482][105692] Updated weights for policy 0, policy_version 240715 (0.0010) [2023-12-26 17:07:01,889][105620] Updated weights for policy 1, policy_version 241073 (0.0009) [2023-12-26 17:07:01,952][105620] Updated weights for policy 1, policy_version 241083 (0.0008) [2023-12-26 17:07:02,017][105620] Updated weights for policy 1, policy_version 241093 (0.0007) [2023-12-26 17:07:02,259][105692] Updated weights for policy 0, policy_version 240726 (0.0009) [2023-12-26 17:07:02,320][105692] Updated weights for policy 0, policy_version 240736 (0.0008) [2023-12-26 17:07:02,386][105692] Updated weights for policy 0, policy_version 240746 (0.0007) [2023-12-26 17:07:02,801][105620] Updated weights for policy 1, policy_version 241103 (0.0009) [2023-12-26 17:07:02,865][105620] Updated weights for policy 1, policy_version 241113 (0.0008) [2023-12-26 17:07:02,913][105620] Updated weights for policy 1, policy_version 241123 (0.0005) [2023-12-26 17:07:02,978][105692] Updated weights for policy 0, policy_version 240756 (0.0007) [2023-12-26 17:07:03,043][105692] Updated weights for policy 0, policy_version 240766 (0.0005) [2023-12-26 17:07:03,107][105692] Updated weights for policy 0, policy_version 240776 (0.0008) [2023-12-26 17:07:03,519][105620] Updated weights for policy 1, policy_version 241133 (0.0005) [2023-12-26 17:07:03,571][105620] Updated weights for policy 1, policy_version 241143 (0.0005) [2023-12-26 17:07:03,623][105620] Updated weights for policy 1, policy_version 241153 (0.0005) [2023-12-26 17:07:03,803][105692] Updated weights for policy 0, policy_version 240786 (0.0009) [2023-12-26 17:07:03,868][105692] Updated weights for policy 0, policy_version 240796 (0.0007) [2023-12-26 17:07:03,919][105692] Updated weights for policy 0, policy_version 240806 (0.0008) [2023-12-26 17:07:03,978][105692] Updated weights for policy 0, policy_version 240816 (0.0010) [2023-12-26 17:07:04,293][105620] Updated weights for policy 1, policy_version 241163 (0.0009) [2023-12-26 17:07:04,349][105620] Updated weights for policy 1, policy_version 241173 (0.0009) [2023-12-26 17:07:04,413][105620] Updated weights for policy 1, policy_version 241183 (0.0009) [2023-12-26 17:07:04,761][105692] Updated weights for policy 0, policy_version 240826 (0.0006) [2023-12-26 17:07:04,807][105692] Updated weights for policy 0, policy_version 240836 (0.0010) [2023-12-26 17:07:04,855][105692] Updated weights for policy 0, policy_version 240846 (0.0010) [2023-12-26 17:07:05,118][105620] Updated weights for policy 1, policy_version 241193 (0.0009) [2023-12-26 17:07:05,169][105620] Updated weights for policy 1, policy_version 241203 (0.0008) [2023-12-26 17:07:05,225][105620] Updated weights for policy 1, policy_version 241213 (0.0009) [2023-12-26 17:07:05,283][105620] Updated weights for policy 1, policy_version 241223 (0.0010) [2023-12-26 17:07:05,513][105692] Updated weights for policy 0, policy_version 240856 (0.0006) [2023-12-26 17:07:05,568][105692] Updated weights for policy 0, policy_version 240866 (0.0005) [2023-12-26 17:07:05,621][105692] Updated weights for policy 0, policy_version 240876 (0.0005) [2023-12-26 17:07:06,038][105620] Updated weights for policy 1, policy_version 241233 (0.0008) [2023-12-26 17:07:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 123437056. Throughput: 0: 9989.5, 1: 9371.0. Samples: 123429552. Policy #0 lag: (min: 26.0, avg: 42.2, max: 58.0) [2023-12-26 17:07:06,063][104569] Avg episode reward: [(0, '8273.509'), (1, '8027.634')] [2023-12-26 17:07:06,094][105620] Updated weights for policy 1, policy_version 241243 (0.0008) [2023-12-26 17:07:06,157][105620] Updated weights for policy 1, policy_version 241253 (0.0009) [2023-12-26 17:07:06,291][105692] Updated weights for policy 0, policy_version 240886 (0.0005) [2023-12-26 17:07:06,360][105692] Updated weights for policy 0, policy_version 240896 (0.0007) [2023-12-26 17:07:06,425][105692] Updated weights for policy 0, policy_version 240906 (0.0008) [2023-12-26 17:07:07,000][105692] Updated weights for policy 0, policy_version 240916 (0.0008) [2023-12-26 17:07:07,019][105620] Updated weights for policy 1, policy_version 241263 (0.0008) [2023-12-26 17:07:07,044][105692] Updated weights for policy 0, policy_version 240926 (0.0007) [2023-12-26 17:07:07,085][105620] Updated weights for policy 1, policy_version 241273 (0.0008) [2023-12-26 17:07:07,094][105692] Updated weights for policy 0, policy_version 240936 (0.0008) [2023-12-26 17:07:07,148][105620] Updated weights for policy 1, policy_version 241283 (0.0008) [2023-12-26 17:07:07,806][105620] Updated weights for policy 1, policy_version 241293 (0.0007) [2023-12-26 17:07:07,859][105620] Updated weights for policy 1, policy_version 241303 (0.0008) [2023-12-26 17:07:07,908][105620] Updated weights for policy 1, policy_version 241313 (0.0009) [2023-12-26 17:07:07,909][105692] Updated weights for policy 0, policy_version 240946 (0.0006) [2023-12-26 17:07:07,975][105692] Updated weights for policy 0, policy_version 240956 (0.0008) [2023-12-26 17:07:08,038][105692] Updated weights for policy 0, policy_version 240966 (0.0011) [2023-12-26 17:07:08,096][105692] Updated weights for policy 0, policy_version 240976 (0.0010) [2023-12-26 17:07:08,566][105620] Updated weights for policy 1, policy_version 241323 (0.0007) [2023-12-26 17:07:08,629][105620] Updated weights for policy 1, policy_version 241333 (0.0008) [2023-12-26 17:07:08,690][105620] Updated weights for policy 1, policy_version 241343 (0.0008) [2023-12-26 17:07:08,736][105692] Updated weights for policy 0, policy_version 240986 (0.0011) [2023-12-26 17:07:08,788][105692] Updated weights for policy 0, policy_version 240996 (0.0010) [2023-12-26 17:07:08,847][105692] Updated weights for policy 0, policy_version 241006 (0.0010) [2023-12-26 17:07:09,456][105620] Updated weights for policy 1, policy_version 241353 (0.0006) [2023-12-26 17:07:09,518][105620] Updated weights for policy 1, policy_version 241363 (0.0011) [2023-12-26 17:07:09,574][105620] Updated weights for policy 1, policy_version 241373 (0.0010) [2023-12-26 17:07:09,603][105692] Updated weights for policy 0, policy_version 241016 (0.0011) [2023-12-26 17:07:09,637][105620] Updated weights for policy 1, policy_version 241383 (0.0010) [2023-12-26 17:07:09,659][105692] Updated weights for policy 0, policy_version 241026 (0.0011) [2023-12-26 17:07:09,719][105692] Updated weights for policy 0, policy_version 241036 (0.0011) [2023-12-26 17:07:10,391][105620] Updated weights for policy 1, policy_version 241393 (0.0011) [2023-12-26 17:07:10,444][105620] Updated weights for policy 1, policy_version 241403 (0.0010) [2023-12-26 17:07:10,462][105586] KL-divergence is very high: 105.7347 [2023-12-26 17:07:10,478][105692] Updated weights for policy 0, policy_version 241046 (0.0009) [2023-12-26 17:07:10,497][105620] Updated weights for policy 1, policy_version 241413 (0.0010) [2023-12-26 17:07:10,501][105586] KL-divergence is very high: 108.9682 [2023-12-26 17:07:10,527][105692] Updated weights for policy 0, policy_version 241056 (0.0007) [2023-12-26 17:07:10,591][105692] Updated weights for policy 0, policy_version 241067 (0.0010) [2023-12-26 17:07:11,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 123535360. Throughput: 0: 10003.0, 1: 9449.2. Samples: 123547268. Policy #0 lag: (min: 26.0, avg: 42.2, max: 58.0) [2023-12-26 17:07:11,062][104569] Avg episode reward: [(0, '8819.753'), (1, '7846.014')] [2023-12-26 17:07:11,151][105620] Updated weights for policy 1, policy_version 241423 (0.0010) [2023-12-26 17:07:11,215][105620] Updated weights for policy 1, policy_version 241433 (0.0009) [2023-12-26 17:07:11,281][105620] Updated weights for policy 1, policy_version 241443 (0.0007) [2023-12-26 17:07:11,381][105692] Updated weights for policy 0, policy_version 241077 (0.0009) [2023-12-26 17:07:11,438][105692] Updated weights for policy 0, policy_version 241087 (0.0008) [2023-12-26 17:07:11,488][105692] Updated weights for policy 0, policy_version 241097 (0.0008) [2023-12-26 17:07:12,014][105620] Updated weights for policy 1, policy_version 241453 (0.0009) [2023-12-26 17:07:12,077][105620] Updated weights for policy 1, policy_version 241463 (0.0010) [2023-12-26 17:07:12,129][105620] Updated weights for policy 1, policy_version 241473 (0.0010) [2023-12-26 17:07:12,169][105692] Updated weights for policy 0, policy_version 241107 (0.0007) [2023-12-26 17:07:12,225][105692] Updated weights for policy 0, policy_version 241117 (0.0007) [2023-12-26 17:07:12,283][105692] Updated weights for policy 0, policy_version 241127 (0.0009) [2023-12-26 17:07:12,896][105620] Updated weights for policy 1, policy_version 241483 (0.0010) [2023-12-26 17:07:12,955][105620] Updated weights for policy 1, policy_version 241493 (0.0010) [2023-12-26 17:07:12,960][105692] Updated weights for policy 0, policy_version 241137 (0.0008) [2023-12-26 17:07:13,011][105620] Updated weights for policy 1, policy_version 241503 (0.0010) [2023-12-26 17:07:13,022][105692] Updated weights for policy 0, policy_version 241147 (0.0005) [2023-12-26 17:07:13,089][105692] Updated weights for policy 0, policy_version 241157 (0.0005) [2023-12-26 17:07:13,151][105692] Updated weights for policy 0, policy_version 241167 (0.0005) [2023-12-26 17:07:13,734][105620] Updated weights for policy 1, policy_version 241513 (0.0011) [2023-12-26 17:07:13,794][105586] KL-divergence is very high: 120.3603 [2023-12-26 17:07:13,798][105692] Updated weights for policy 0, policy_version 241177 (0.0005) [2023-12-26 17:07:13,799][105620] Updated weights for policy 1, policy_version 241523 (0.0010) [2023-12-26 17:07:13,845][105586] KL-divergence is very high: 135.1096 [2023-12-26 17:07:13,854][105692] Updated weights for policy 0, policy_version 241187 (0.0005) [2023-12-26 17:07:13,864][105620] Updated weights for policy 1, policy_version 241533 (0.0010) [2023-12-26 17:07:13,894][105586] KL-divergence is very high: 110.8511 [2023-12-26 17:07:13,905][105692] Updated weights for policy 0, policy_version 241197 (0.0005) [2023-12-26 17:07:13,919][105620] Updated weights for policy 1, policy_version 241543 (0.0010) [2023-12-26 17:07:14,464][105692] Updated weights for policy 0, policy_version 241207 (0.0005) [2023-12-26 17:07:14,517][105692] Updated weights for policy 0, policy_version 241217 (0.0005) [2023-12-26 17:07:14,568][105692] Updated weights for policy 0, policy_version 241227 (0.0005) [2023-12-26 17:07:14,644][105620] Updated weights for policy 1, policy_version 241553 (0.0010) [2023-12-26 17:07:14,697][105620] Updated weights for policy 1, policy_version 241563 (0.0010) [2023-12-26 17:07:14,749][105620] Updated weights for policy 1, policy_version 241573 (0.0010) [2023-12-26 17:07:15,286][105692] Updated weights for policy 0, policy_version 241237 (0.0006) [2023-12-26 17:07:15,344][105692] Updated weights for policy 0, policy_version 241247 (0.0010) [2023-12-26 17:07:15,398][105692] Updated weights for policy 0, policy_version 241257 (0.0010) [2023-12-26 17:07:15,468][105620] Updated weights for policy 1, policy_version 241583 (0.0009) [2023-12-26 17:07:15,533][105620] Updated weights for policy 1, policy_version 241593 (0.0008) [2023-12-26 17:07:15,598][105620] Updated weights for policy 1, policy_version 241603 (0.0009) [2023-12-26 17:07:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 123633664. Throughput: 0: 9928.4, 1: 9437.5. Samples: 123605288. Policy #0 lag: (min: 26.0, avg: 42.2, max: 58.0) [2023-12-26 17:07:16,062][104569] Avg episode reward: [(0, '8996.709'), (1, '8374.873')] [2023-12-26 17:07:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000241608_61857792.pth... [2023-12-26 17:07:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000241264_61775872.pth... [2023-12-26 17:07:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000240520_61579264.pth [2023-12-26 17:07:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000240112_61480960.pth [2023-12-26 17:07:16,222][105692] Updated weights for policy 0, policy_version 241267 (0.0008) [2023-12-26 17:07:16,237][105620] Updated weights for policy 1, policy_version 241613 (0.0009) [2023-12-26 17:07:16,284][105692] Updated weights for policy 0, policy_version 241277 (0.0006) [2023-12-26 17:07:16,298][105620] Updated weights for policy 1, policy_version 241623 (0.0010) [2023-12-26 17:07:16,345][105692] Updated weights for policy 0, policy_version 241287 (0.0007) [2023-12-26 17:07:16,355][105620] Updated weights for policy 1, policy_version 241633 (0.0007) [2023-12-26 17:07:16,895][105620] Updated weights for policy 1, policy_version 241643 (0.0005) [2023-12-26 17:07:16,961][105620] Updated weights for policy 1, policy_version 241653 (0.0005) [2023-12-26 17:07:17,001][105586] KL-divergence is very high: 128.8453 [2023-12-26 17:07:17,015][105586] KL-divergence is very high: 104.1158 [2023-12-26 17:07:17,027][105620] Updated weights for policy 1, policy_version 241663 (0.0007) [2023-12-26 17:07:17,054][105586] KL-divergence is very high: 190.9547 [2023-12-26 17:07:17,068][105586] KL-divergence is very high: 126.9025 [2023-12-26 17:07:17,179][105692] Updated weights for policy 0, policy_version 241297 (0.0008) [2023-12-26 17:07:17,238][105692] Updated weights for policy 0, policy_version 241307 (0.0008) [2023-12-26 17:07:17,298][105692] Updated weights for policy 0, policy_version 241317 (0.0010) [2023-12-26 17:07:17,365][105692] Updated weights for policy 0, policy_version 241327 (0.0005) [2023-12-26 17:07:17,731][105620] Updated weights for policy 1, policy_version 241673 (0.0006) [2023-12-26 17:07:17,793][105620] Updated weights for policy 1, policy_version 241683 (0.0008) [2023-12-26 17:07:17,854][105620] Updated weights for policy 1, policy_version 241693 (0.0008) [2023-12-26 17:07:17,902][105620] Updated weights for policy 1, policy_version 241703 (0.0008) [2023-12-26 17:07:17,998][105692] Updated weights for policy 0, policy_version 241337 (0.0006) [2023-12-26 17:07:18,057][105692] Updated weights for policy 0, policy_version 241347 (0.0009) [2023-12-26 17:07:18,117][105692] Updated weights for policy 0, policy_version 241357 (0.0009) [2023-12-26 17:07:18,699][105620] Updated weights for policy 1, policy_version 241713 (0.0009) [2023-12-26 17:07:18,755][105586] KL-divergence is very high: 554.6739 [2023-12-26 17:07:18,762][105620] Updated weights for policy 1, policy_version 241723 (0.0009) [2023-12-26 17:07:18,762][105692] Updated weights for policy 0, policy_version 241367 (0.0007) [2023-12-26 17:07:18,809][105586] KL-divergence is very high: 866.4030 [2023-12-26 17:07:18,811][105692] Updated weights for policy 0, policy_version 241377 (0.0005) [2023-12-26 17:07:18,829][105620] Updated weights for policy 1, policy_version 241733 (0.0009) [2023-12-26 17:07:18,859][105692] Updated weights for policy 0, policy_version 241387 (0.0006) [2023-12-26 17:07:19,546][105586] KL-divergence is very high: 991.8924 [2023-12-26 17:07:19,551][105586] KL-divergence is very high: 865.8730 [2023-12-26 17:07:19,579][105586] KL-divergence is very high: 858.2477 [2023-12-26 17:07:19,584][105620] Updated weights for policy 1, policy_version 241743 (0.0007) [2023-12-26 17:07:19,598][105586] KL-divergence is very high: 619.9960 [2023-12-26 17:07:19,603][105586] KL-divergence is very high: 525.9982 [2023-12-26 17:07:19,619][105692] Updated weights for policy 0, policy_version 241397 (0.0009) [2023-12-26 17:07:19,627][105586] KL-divergence is very high: 389.3459 [2023-12-26 17:07:19,644][105620] Updated weights for policy 1, policy_version 241753 (0.0009) [2023-12-26 17:07:19,644][105586] KL-divergence is very high: 224.6541 [2023-12-26 17:07:19,650][105586] KL-divergence is very high: 211.3463 [2023-12-26 17:07:19,675][105692] Updated weights for policy 0, policy_version 241407 (0.0006) [2023-12-26 17:07:19,675][105586] KL-divergence is very high: 122.9531 [2023-12-26 17:07:19,703][105620] Updated weights for policy 1, policy_version 241763 (0.0008) [2023-12-26 17:07:19,728][105692] Updated weights for policy 0, policy_version 241417 (0.0008) [2023-12-26 17:07:19,729][105586] KL-divergence is very high: 102.9170 [2023-12-26 17:07:20,485][105692] Updated weights for policy 0, policy_version 241427 (0.0007) [2023-12-26 17:07:20,491][105620] Updated weights for policy 1, policy_version 241773 (0.0007) [2023-12-26 17:07:20,542][105692] Updated weights for policy 0, policy_version 241437 (0.0006) [2023-12-26 17:07:20,552][105620] Updated weights for policy 1, policy_version 241783 (0.0008) [2023-12-26 17:07:20,607][105692] Updated weights for policy 0, policy_version 241447 (0.0009) [2023-12-26 17:07:20,621][105620] Updated weights for policy 1, policy_version 241793 (0.0008) [2023-12-26 17:07:21,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 123731968. Throughput: 0: 9819.5, 1: 9527.4. Samples: 123722844. Policy #0 lag: (min: 26.0, avg: 42.2, max: 58.0) [2023-12-26 17:07:21,063][104569] Avg episode reward: [(0, '8563.215'), (1, '7280.332')] [2023-12-26 17:07:21,353][105692] Updated weights for policy 0, policy_version 241457 (0.0008) [2023-12-26 17:07:21,413][105620] Updated weights for policy 1, policy_version 241803 (0.0007) [2023-12-26 17:07:21,425][105692] Updated weights for policy 0, policy_version 241467 (0.0008) [2023-12-26 17:07:21,470][105620] Updated weights for policy 1, policy_version 241813 (0.0007) [2023-12-26 17:07:21,486][105692] Updated weights for policy 0, policy_version 241477 (0.0009) [2023-12-26 17:07:21,531][105620] Updated weights for policy 1, policy_version 241823 (0.0007) [2023-12-26 17:07:21,546][105692] Updated weights for policy 0, policy_version 241487 (0.0010) [2023-12-26 17:07:22,290][105692] Updated weights for policy 0, policy_version 241497 (0.0010) [2023-12-26 17:07:22,304][105620] Updated weights for policy 1, policy_version 241833 (0.0009) [2023-12-26 17:07:22,353][105692] Updated weights for policy 0, policy_version 241507 (0.0009) [2023-12-26 17:07:22,371][105620] Updated weights for policy 1, policy_version 241843 (0.0007) [2023-12-26 17:07:22,419][105692] Updated weights for policy 0, policy_version 241517 (0.0010) [2023-12-26 17:07:22,437][105620] Updated weights for policy 1, policy_version 241853 (0.0008) [2023-12-26 17:07:22,501][105620] Updated weights for policy 1, policy_version 241863 (0.0008) [2023-12-26 17:07:23,176][105692] Updated weights for policy 0, policy_version 241527 (0.0008) [2023-12-26 17:07:23,242][105692] Updated weights for policy 0, policy_version 241537 (0.0005) [2023-12-26 17:07:23,276][105620] Updated weights for policy 1, policy_version 241873 (0.0008) [2023-12-26 17:07:23,297][105692] Updated weights for policy 0, policy_version 241547 (0.0007) [2023-12-26 17:07:23,325][105620] Updated weights for policy 1, policy_version 241883 (0.0010) [2023-12-26 17:07:23,371][105620] Updated weights for policy 1, policy_version 241893 (0.0010) [2023-12-26 17:07:23,843][105692] Updated weights for policy 0, policy_version 241557 (0.0008) [2023-12-26 17:07:23,896][105692] Updated weights for policy 0, policy_version 241567 (0.0005) [2023-12-26 17:07:23,952][105692] Updated weights for policy 0, policy_version 241577 (0.0008) [2023-12-26 17:07:23,987][105620] Updated weights for policy 1, policy_version 241903 (0.0007) [2023-12-26 17:07:24,045][105620] Updated weights for policy 1, policy_version 241913 (0.0005) [2023-12-26 17:07:24,114][105620] Updated weights for policy 1, policy_version 241923 (0.0010) [2023-12-26 17:07:24,544][105692] Updated weights for policy 0, policy_version 241587 (0.0010) [2023-12-26 17:07:24,609][105692] Updated weights for policy 0, policy_version 241597 (0.0010) [2023-12-26 17:07:24,674][105692] Updated weights for policy 0, policy_version 241607 (0.0010) [2023-12-26 17:07:24,697][105620] Updated weights for policy 1, policy_version 241933 (0.0008) [2023-12-26 17:07:24,757][105620] Updated weights for policy 1, policy_version 241943 (0.0005) [2023-12-26 17:07:24,818][105620] Updated weights for policy 1, policy_version 241953 (0.0008) [2023-12-26 17:07:25,259][105692] Updated weights for policy 0, policy_version 241617 (0.0010) [2023-12-26 17:07:25,306][105692] Updated weights for policy 0, policy_version 241627 (0.0005) [2023-12-26 17:07:25,353][105692] Updated weights for policy 0, policy_version 241637 (0.0005) [2023-12-26 17:07:25,400][105692] Updated weights for policy 0, policy_version 241647 (0.0006) [2023-12-26 17:07:25,572][105620] Updated weights for policy 1, policy_version 241963 (0.0010) [2023-12-26 17:07:25,619][105620] Updated weights for policy 1, policy_version 241973 (0.0010) [2023-12-26 17:07:25,667][105620] Updated weights for policy 1, policy_version 241983 (0.0010) [2023-12-26 17:07:25,948][105692] Updated weights for policy 0, policy_version 241657 (0.0006) [2023-12-26 17:07:26,019][105692] Updated weights for policy 0, policy_version 241667 (0.0009) [2023-12-26 17:07:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 123830272. Throughput: 0: 9895.8, 1: 9509.4. Samples: 123841960. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-26 17:07:26,062][104569] Avg episode reward: [(0, '7946.585'), (1, '7577.596')] [2023-12-26 17:07:26,080][105692] Updated weights for policy 0, policy_version 241677 (0.0010) [2023-12-26 17:07:26,340][105620] Updated weights for policy 1, policy_version 241993 (0.0009) [2023-12-26 17:07:26,391][105620] Updated weights for policy 1, policy_version 242003 (0.0005) [2023-12-26 17:07:26,416][105586] KL-divergence is very high: 103.2977 [2023-12-26 17:07:26,425][105586] KL-divergence is very high: 166.7101 [2023-12-26 17:07:26,440][105620] Updated weights for policy 1, policy_version 242013 (0.0005) [2023-12-26 17:07:26,452][105586] KL-divergence is very high: 122.4420 [2023-12-26 17:07:26,463][105586] KL-divergence is very high: 160.5115 [2023-12-26 17:07:26,497][105620] Updated weights for policy 1, policy_version 242023 (0.0006) [2023-12-26 17:07:26,779][105692] Updated weights for policy 0, policy_version 241687 (0.0007) [2023-12-26 17:07:26,829][105692] Updated weights for policy 0, policy_version 241697 (0.0005) [2023-12-26 17:07:26,877][105692] Updated weights for policy 0, policy_version 241707 (0.0005) [2023-12-26 17:07:27,079][105620] Updated weights for policy 1, policy_version 242033 (0.0010) [2023-12-26 17:07:27,123][105620] Updated weights for policy 1, policy_version 242043 (0.0010) [2023-12-26 17:07:27,170][105620] Updated weights for policy 1, policy_version 242053 (0.0010) [2023-12-26 17:07:27,493][105692] Updated weights for policy 0, policy_version 241717 (0.0008) [2023-12-26 17:07:27,534][105692] Updated weights for policy 0, policy_version 241727 (0.0010) [2023-12-26 17:07:27,581][105692] Updated weights for policy 0, policy_version 241737 (0.0010) [2023-12-26 17:07:27,939][105620] Updated weights for policy 1, policy_version 242063 (0.0010) [2023-12-26 17:07:27,990][105620] Updated weights for policy 1, policy_version 242073 (0.0010) [2023-12-26 17:07:28,037][105620] Updated weights for policy 1, policy_version 242083 (0.0010) [2023-12-26 17:07:28,193][105692] Updated weights for policy 0, policy_version 241747 (0.0009) [2023-12-26 17:07:28,241][105692] Updated weights for policy 0, policy_version 241757 (0.0005) [2023-12-26 17:07:28,288][105692] Updated weights for policy 0, policy_version 241767 (0.0005) [2023-12-26 17:07:28,756][105620] Updated weights for policy 1, policy_version 242093 (0.0010) [2023-12-26 17:07:28,811][105620] Updated weights for policy 1, policy_version 242103 (0.0010) [2023-12-26 17:07:28,860][105620] Updated weights for policy 1, policy_version 242113 (0.0010) [2023-12-26 17:07:29,018][105692] Updated weights for policy 0, policy_version 241777 (0.0008) [2023-12-26 17:07:29,076][105692] Updated weights for policy 0, policy_version 241787 (0.0010) [2023-12-26 17:07:29,134][105692] Updated weights for policy 0, policy_version 241797 (0.0010) [2023-12-26 17:07:29,189][105692] Updated weights for policy 0, policy_version 241807 (0.0010) [2023-12-26 17:07:29,610][105620] Updated weights for policy 1, policy_version 242123 (0.0010) [2023-12-26 17:07:29,661][105620] Updated weights for policy 1, policy_version 242133 (0.0010) [2023-12-26 17:07:29,712][105620] Updated weights for policy 1, policy_version 242143 (0.0010) [2023-12-26 17:07:29,908][105692] Updated weights for policy 0, policy_version 241817 (0.0006) [2023-12-26 17:07:29,976][105692] Updated weights for policy 0, policy_version 241827 (0.0007) [2023-12-26 17:07:30,037][105692] Updated weights for policy 0, policy_version 241837 (0.0008) [2023-12-26 17:07:30,418][105620] Updated weights for policy 1, policy_version 242153 (0.0010) [2023-12-26 17:07:30,473][105620] Updated weights for policy 1, policy_version 242163 (0.0010) [2023-12-26 17:07:30,536][105620] Updated weights for policy 1, policy_version 242173 (0.0010) [2023-12-26 17:07:30,581][105692] Updated weights for policy 0, policy_version 241847 (0.0008) [2023-12-26 17:07:30,595][105620] Updated weights for policy 1, policy_version 242183 (0.0010) [2023-12-26 17:07:30,641][105692] Updated weights for policy 0, policy_version 241857 (0.0008) [2023-12-26 17:07:30,700][105692] Updated weights for policy 0, policy_version 241867 (0.0008) [2023-12-26 17:07:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 123936768. Throughput: 0: 10011.8, 1: 9551.7. Samples: 123905736. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-26 17:07:31,063][104569] Avg episode reward: [(0, '8817.138'), (1, '7379.163')] [2023-12-26 17:07:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000241872_61931520.pth... [2023-12-26 17:07:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000242184_62005248.pth... [2023-12-26 17:07:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000240688_61628416.pth [2023-12-26 17:07:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000241064_61718528.pth [2023-12-26 17:07:31,351][105620] Updated weights for policy 1, policy_version 242193 (0.0010) [2023-12-26 17:07:31,401][105692] Updated weights for policy 0, policy_version 241877 (0.0009) [2023-12-26 17:07:31,416][105620] Updated weights for policy 1, policy_version 242203 (0.0009) [2023-12-26 17:07:31,460][105692] Updated weights for policy 0, policy_version 241887 (0.0010) [2023-12-26 17:07:31,475][105620] Updated weights for policy 1, policy_version 242213 (0.0010) [2023-12-26 17:07:31,523][105692] Updated weights for policy 0, policy_version 241897 (0.0010) [2023-12-26 17:07:32,138][105692] Updated weights for policy 0, policy_version 241907 (0.0008) [2023-12-26 17:07:32,155][105620] Updated weights for policy 1, policy_version 242223 (0.0009) [2023-12-26 17:07:32,187][105692] Updated weights for policy 0, policy_version 241917 (0.0010) [2023-12-26 17:07:32,214][105620] Updated weights for policy 1, policy_version 242233 (0.0009) [2023-12-26 17:07:32,236][105692] Updated weights for policy 0, policy_version 241927 (0.0009) [2023-12-26 17:07:32,274][105620] Updated weights for policy 1, policy_version 242243 (0.0010) [2023-12-26 17:07:32,946][105620] Updated weights for policy 1, policy_version 242253 (0.0008) [2023-12-26 17:07:33,005][105620] Updated weights for policy 1, policy_version 242263 (0.0006) [2023-12-26 17:07:33,009][105692] Updated weights for policy 0, policy_version 241937 (0.0010) [2023-12-26 17:07:33,057][105692] Updated weights for policy 0, policy_version 241947 (0.0010) [2023-12-26 17:07:33,065][105620] Updated weights for policy 1, policy_version 242273 (0.0008) [2023-12-26 17:07:33,106][105692] Updated weights for policy 0, policy_version 241957 (0.0010) [2023-12-26 17:07:33,154][105692] Updated weights for policy 0, policy_version 241967 (0.0010) [2023-12-26 17:07:33,678][105620] Updated weights for policy 1, policy_version 242283 (0.0008) [2023-12-26 17:07:33,738][105620] Updated weights for policy 1, policy_version 242293 (0.0010) [2023-12-26 17:07:33,805][105620] Updated weights for policy 1, policy_version 242303 (0.0010) [2023-12-26 17:07:33,916][105692] Updated weights for policy 0, policy_version 241977 (0.0010) [2023-12-26 17:07:33,973][105692] Updated weights for policy 0, policy_version 241987 (0.0010) [2023-12-26 17:07:34,031][105692] Updated weights for policy 0, policy_version 241997 (0.0010) [2023-12-26 17:07:34,417][105620] Updated weights for policy 1, policy_version 242313 (0.0010) [2023-12-26 17:07:34,482][105620] Updated weights for policy 1, policy_version 242323 (0.0010) [2023-12-26 17:07:34,542][105620] Updated weights for policy 1, policy_version 242333 (0.0009) [2023-12-26 17:07:34,606][105620] Updated weights for policy 1, policy_version 242343 (0.0010) [2023-12-26 17:07:34,736][105692] Updated weights for policy 0, policy_version 242007 (0.0007) [2023-12-26 17:07:34,798][105692] Updated weights for policy 0, policy_version 242017 (0.0006) [2023-12-26 17:07:34,856][105692] Updated weights for policy 0, policy_version 242027 (0.0009) [2023-12-26 17:07:35,283][105620] Updated weights for policy 1, policy_version 242353 (0.0010) [2023-12-26 17:07:35,331][105620] Updated weights for policy 1, policy_version 242363 (0.0010) [2023-12-26 17:07:35,398][105620] Updated weights for policy 1, policy_version 242373 (0.0010) [2023-12-26 17:07:35,528][105692] Updated weights for policy 0, policy_version 242037 (0.0010) [2023-12-26 17:07:35,585][105692] Updated weights for policy 0, policy_version 242047 (0.0010) [2023-12-26 17:07:35,653][105692] Updated weights for policy 0, policy_version 242057 (0.0010) [2023-12-26 17:07:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 124035072. Throughput: 0: 10000.7, 1: 9651.3. Samples: 124025744. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-26 17:07:36,062][104569] Avg episode reward: [(0, '9086.348'), (1, '7621.999')] [2023-12-26 17:07:36,152][105620] Updated weights for policy 1, policy_version 242383 (0.0009) [2023-12-26 17:07:36,211][105620] Updated weights for policy 1, policy_version 242393 (0.0008) [2023-12-26 17:07:36,273][105620] Updated weights for policy 1, policy_version 242403 (0.0010) [2023-12-26 17:07:36,348][105692] Updated weights for policy 0, policy_version 242067 (0.0008) [2023-12-26 17:07:36,415][105692] Updated weights for policy 0, policy_version 242077 (0.0009) [2023-12-26 17:07:36,473][105692] Updated weights for policy 0, policy_version 242087 (0.0009) [2023-12-26 17:07:36,972][105620] Updated weights for policy 1, policy_version 242413 (0.0008) [2023-12-26 17:07:37,025][105620] Updated weights for policy 1, policy_version 242423 (0.0005) [2023-12-26 17:07:37,089][105620] Updated weights for policy 1, policy_version 242433 (0.0006) [2023-12-26 17:07:37,244][105692] Updated weights for policy 0, policy_version 242097 (0.0010) [2023-12-26 17:07:37,304][105692] Updated weights for policy 0, policy_version 242107 (0.0011) [2023-12-26 17:07:37,366][105692] Updated weights for policy 0, policy_version 242117 (0.0011) [2023-12-26 17:07:37,433][105692] Updated weights for policy 0, policy_version 242127 (0.0011) [2023-12-26 17:07:37,741][105620] Updated weights for policy 1, policy_version 242443 (0.0006) [2023-12-26 17:07:37,789][105620] Updated weights for policy 1, policy_version 242453 (0.0008) [2023-12-26 17:07:37,844][105620] Updated weights for policy 1, policy_version 242463 (0.0008) [2023-12-26 17:07:38,185][105692] Updated weights for policy 0, policy_version 242137 (0.0011) [2023-12-26 17:07:38,230][105692] Updated weights for policy 0, policy_version 242147 (0.0010) [2023-12-26 17:07:38,289][105692] Updated weights for policy 0, policy_version 242157 (0.0010) [2023-12-26 17:07:38,529][105620] Updated weights for policy 1, policy_version 242473 (0.0008) [2023-12-26 17:07:38,581][105586] KL-divergence is very high: 107.9585 [2023-12-26 17:07:38,593][105620] Updated weights for policy 1, policy_version 242483 (0.0005) [2023-12-26 17:07:38,635][105586] KL-divergence is very high: 172.4791 [2023-12-26 17:07:38,661][105620] Updated weights for policy 1, policy_version 242493 (0.0010) [2023-12-26 17:07:38,686][105586] KL-divergence is very high: 180.4527 [2023-12-26 17:07:38,721][105620] Updated weights for policy 1, policy_version 242503 (0.0011) [2023-12-26 17:07:39,073][105692] Updated weights for policy 0, policy_version 242167 (0.0007) [2023-12-26 17:07:39,131][105692] Updated weights for policy 0, policy_version 242177 (0.0007) [2023-12-26 17:07:39,185][105692] Updated weights for policy 0, policy_version 242187 (0.0010) [2023-12-26 17:07:39,393][105586] KL-divergence is very high: 162.3662 [2023-12-26 17:07:39,442][105620] Updated weights for policy 1, policy_version 242513 (0.0012) [2023-12-26 17:07:39,443][105586] KL-divergence is very high: 125.1310 [2023-12-26 17:07:39,492][105586] KL-divergence is very high: 112.8761 [2023-12-26 17:07:39,502][105620] Updated weights for policy 1, policy_version 242523 (0.0011) [2023-12-26 17:07:39,532][105586] KL-divergence is very high: 149.2928 [2023-12-26 17:07:39,552][105620] Updated weights for policy 1, policy_version 242533 (0.0011) [2023-12-26 17:07:39,931][105692] Updated weights for policy 0, policy_version 242197 (0.0009) [2023-12-26 17:07:39,994][105692] Updated weights for policy 0, policy_version 242207 (0.0008) [2023-12-26 17:07:40,056][105692] Updated weights for policy 0, policy_version 242217 (0.0008) [2023-12-26 17:07:40,336][105620] Updated weights for policy 1, policy_version 242543 (0.0010) [2023-12-26 17:07:40,385][105620] Updated weights for policy 1, policy_version 242553 (0.0009) [2023-12-26 17:07:40,436][105620] Updated weights for policy 1, policy_version 242563 (0.0008) [2023-12-26 17:07:40,780][105692] Updated weights for policy 0, policy_version 242227 (0.0009) [2023-12-26 17:07:40,826][105692] Updated weights for policy 0, policy_version 242237 (0.0009) [2023-12-26 17:07:40,882][105692] Updated weights for policy 0, policy_version 242247 (0.0008) [2023-12-26 17:07:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 124133376. Throughput: 0: 10005.5, 1: 9741.3. Samples: 124141368. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-26 17:07:41,062][104569] Avg episode reward: [(0, '9266.644'), (1, '7804.652')] [2023-12-26 17:07:41,233][105620] Updated weights for policy 1, policy_version 242573 (0.0009) [2023-12-26 17:07:41,299][105620] Updated weights for policy 1, policy_version 242583 (0.0008) [2023-12-26 17:07:41,368][105620] Updated weights for policy 1, policy_version 242593 (0.0009) [2023-12-26 17:07:41,618][105692] Updated weights for policy 0, policy_version 242257 (0.0008) [2023-12-26 17:07:41,688][105692] Updated weights for policy 0, policy_version 242267 (0.0008) [2023-12-26 17:07:41,760][105692] Updated weights for policy 0, policy_version 242277 (0.0010) [2023-12-26 17:07:41,825][105692] Updated weights for policy 0, policy_version 242287 (0.0007) [2023-12-26 17:07:42,154][105620] Updated weights for policy 1, policy_version 242603 (0.0010) [2023-12-26 17:07:42,219][105620] Updated weights for policy 1, policy_version 242613 (0.0009) [2023-12-26 17:07:42,288][105620] Updated weights for policy 1, policy_version 242623 (0.0009) [2023-12-26 17:07:42,443][105692] Updated weights for policy 0, policy_version 242297 (0.0006) [2023-12-26 17:07:42,512][105692] Updated weights for policy 0, policy_version 242307 (0.0006) [2023-12-26 17:07:42,583][105692] Updated weights for policy 0, policy_version 242317 (0.0009) [2023-12-26 17:07:43,036][105620] Updated weights for policy 1, policy_version 242633 (0.0010) [2023-12-26 17:07:43,087][105620] Updated weights for policy 1, policy_version 242643 (0.0008) [2023-12-26 17:07:43,140][105620] Updated weights for policy 1, policy_version 242653 (0.0006) [2023-12-26 17:07:43,207][105620] Updated weights for policy 1, policy_version 242663 (0.0010) [2023-12-26 17:07:43,258][105692] Updated weights for policy 0, policy_version 242327 (0.0006) [2023-12-26 17:07:43,309][105692] Updated weights for policy 0, policy_version 242337 (0.0005) [2023-12-26 17:07:43,361][105692] Updated weights for policy 0, policy_version 242347 (0.0005) [2023-12-26 17:07:43,869][105620] Updated weights for policy 1, policy_version 242673 (0.0006) [2023-12-26 17:07:43,914][105692] Updated weights for policy 0, policy_version 242357 (0.0005) [2023-12-26 17:07:43,932][105620] Updated weights for policy 1, policy_version 242683 (0.0005) [2023-12-26 17:07:43,979][105692] Updated weights for policy 0, policy_version 242367 (0.0006) [2023-12-26 17:07:43,993][105620] Updated weights for policy 1, policy_version 242693 (0.0005) [2023-12-26 17:07:44,050][105692] Updated weights for policy 0, policy_version 242377 (0.0011) [2023-12-26 17:07:44,564][105620] Updated weights for policy 1, policy_version 242703 (0.0005) [2023-12-26 17:07:44,624][105620] Updated weights for policy 1, policy_version 242713 (0.0005) [2023-12-26 17:07:44,684][105620] Updated weights for policy 1, policy_version 242723 (0.0005) [2023-12-26 17:07:44,698][105692] Updated weights for policy 0, policy_version 242387 (0.0009) [2023-12-26 17:07:44,746][105692] Updated weights for policy 0, policy_version 242397 (0.0007) [2023-12-26 17:07:44,814][105692] Updated weights for policy 0, policy_version 242407 (0.0007) [2023-12-26 17:07:45,343][105620] Updated weights for policy 1, policy_version 242733 (0.0006) [2023-12-26 17:07:45,402][105620] Updated weights for policy 1, policy_version 242743 (0.0006) [2023-12-26 17:07:45,461][105620] Updated weights for policy 1, policy_version 242753 (0.0005) [2023-12-26 17:07:45,599][105692] Updated weights for policy 0, policy_version 242417 (0.0006) [2023-12-26 17:07:45,658][105692] Updated weights for policy 0, policy_version 242427 (0.0007) [2023-12-26 17:07:45,713][105692] Updated weights for policy 0, policy_version 242437 (0.0008) [2023-12-26 17:07:45,765][105692] Updated weights for policy 0, policy_version 242447 (0.0008) [2023-12-26 17:07:46,062][104569] Fps is (10 sec: 19659.9, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 124231680. Throughput: 0: 9916.8, 1: 9823.1. Samples: 124200752. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-26 17:07:46,063][104569] Avg episode reward: [(0, '9357.257'), (1, '7107.503')] [2023-12-26 17:07:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000242448_62078976.pth... [2023-12-26 17:07:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000241264_61775872.pth [2023-12-26 17:07:46,100][105620] Updated weights for policy 1, policy_version 242763 (0.0005) [2023-12-26 17:07:46,169][105620] Updated weights for policy 1, policy_version 242773 (0.0005) [2023-12-26 17:07:46,180][105586] KL-divergence is very high: 137.2861 [2023-12-26 17:07:46,224][105620] Updated weights for policy 1, policy_version 242783 (0.0005) [2023-12-26 17:07:46,226][105586] KL-divergence is very high: 238.5902 [2023-12-26 17:07:46,274][105586] KL-divergence is very high: 226.9718 [2023-12-26 17:07:46,279][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000242792_62160896.pth... [2023-12-26 17:07:46,283][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000241608_61857792.pth [2023-12-26 17:07:46,600][105692] Updated weights for policy 0, policy_version 242457 (0.0007) [2023-12-26 17:07:46,645][105692] Updated weights for policy 0, policy_version 242467 (0.0008) [2023-12-26 17:07:46,694][105692] Updated weights for policy 0, policy_version 242478 (0.0009) [2023-12-26 17:07:46,777][105620] Updated weights for policy 1, policy_version 242793 (0.0006) [2023-12-26 17:07:46,833][105620] Updated weights for policy 1, policy_version 242803 (0.0009) [2023-12-26 17:07:46,891][105620] Updated weights for policy 1, policy_version 242813 (0.0009) [2023-12-26 17:07:46,950][105620] Updated weights for policy 1, policy_version 242823 (0.0007) [2023-12-26 17:07:47,520][105692] Updated weights for policy 0, policy_version 242488 (0.0009) [2023-12-26 17:07:47,570][105692] Updated weights for policy 0, policy_version 242498 (0.0008) [2023-12-26 17:07:47,621][105620] Updated weights for policy 1, policy_version 242833 (0.0010) [2023-12-26 17:07:47,633][105692] Updated weights for policy 0, policy_version 242508 (0.0009) [2023-12-26 17:07:47,683][105620] Updated weights for policy 1, policy_version 242843 (0.0010) [2023-12-26 17:07:47,741][105620] Updated weights for policy 1, policy_version 242853 (0.0010) [2023-12-26 17:07:48,390][105692] Updated weights for policy 0, policy_version 242518 (0.0008) [2023-12-26 17:07:48,450][105692] Updated weights for policy 0, policy_version 242528 (0.0009) [2023-12-26 17:07:48,492][105620] Updated weights for policy 1, policy_version 242863 (0.0011) [2023-12-26 17:07:48,510][105692] Updated weights for policy 0, policy_version 242538 (0.0006) [2023-12-26 17:07:48,555][105620] Updated weights for policy 1, policy_version 242873 (0.0011) [2023-12-26 17:07:48,614][105620] Updated weights for policy 1, policy_version 242883 (0.0010) [2023-12-26 17:07:49,292][105620] Updated weights for policy 1, policy_version 242893 (0.0010) [2023-12-26 17:07:49,311][105692] Updated weights for policy 0, policy_version 242548 (0.0006) [2023-12-26 17:07:49,349][105620] Updated weights for policy 1, policy_version 242903 (0.0007) [2023-12-26 17:07:49,374][105692] Updated weights for policy 0, policy_version 242558 (0.0008) [2023-12-26 17:07:49,417][105620] Updated weights for policy 1, policy_version 242913 (0.0008) [2023-12-26 17:07:49,436][105692] Updated weights for policy 0, policy_version 242568 (0.0008) [2023-12-26 17:07:50,087][105620] Updated weights for policy 1, policy_version 242923 (0.0008) [2023-12-26 17:07:50,141][105620] Updated weights for policy 1, policy_version 242933 (0.0008) [2023-12-26 17:07:50,162][105692] Updated weights for policy 0, policy_version 242578 (0.0008) [2023-12-26 17:07:50,206][105620] Updated weights for policy 1, policy_version 242943 (0.0006) [2023-12-26 17:07:50,224][105692] Updated weights for policy 0, policy_version 242588 (0.0008) [2023-12-26 17:07:50,290][105692] Updated weights for policy 0, policy_version 242598 (0.0006) [2023-12-26 17:07:50,361][105692] Updated weights for policy 0, policy_version 242608 (0.0008) [2023-12-26 17:07:50,871][105620] Updated weights for policy 1, policy_version 242953 (0.0007) [2023-12-26 17:07:50,928][105620] Updated weights for policy 1, policy_version 242963 (0.0009) [2023-12-26 17:07:50,992][105620] Updated weights for policy 1, policy_version 242973 (0.0006) [2023-12-26 17:07:51,057][105620] Updated weights for policy 1, policy_version 242983 (0.0008) [2023-12-26 17:07:51,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 124321792. Throughput: 0: 9906.0, 1: 9843.2. Samples: 124318268. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-26 17:07:51,063][104569] Avg episode reward: [(0, '9357.526'), (1, '6210.733')] [2023-12-26 17:07:51,128][105692] Updated weights for policy 0, policy_version 242618 (0.0008) [2023-12-26 17:07:51,189][105692] Updated weights for policy 0, policy_version 242628 (0.0009) [2023-12-26 17:07:51,254][105692] Updated weights for policy 0, policy_version 242638 (0.0009) [2023-12-26 17:07:51,815][105620] Updated weights for policy 1, policy_version 242993 (0.0010) [2023-12-26 17:07:51,880][105620] Updated weights for policy 1, policy_version 243003 (0.0007) [2023-12-26 17:07:51,939][105620] Updated weights for policy 1, policy_version 243013 (0.0008) [2023-12-26 17:07:51,981][105692] Updated weights for policy 0, policy_version 242648 (0.0008) [2023-12-26 17:07:52,036][105692] Updated weights for policy 0, policy_version 242658 (0.0009) [2023-12-26 17:07:52,097][105692] Updated weights for policy 0, policy_version 242668 (0.0009) [2023-12-26 17:07:52,660][105620] Updated weights for policy 1, policy_version 243023 (0.0006) [2023-12-26 17:07:52,724][105620] Updated weights for policy 1, policy_version 243033 (0.0008) [2023-12-26 17:07:52,791][105620] Updated weights for policy 1, policy_version 243043 (0.0006) [2023-12-26 17:07:52,900][105692] Updated weights for policy 0, policy_version 242678 (0.0009) [2023-12-26 17:07:52,964][105692] Updated weights for policy 0, policy_version 242688 (0.0008) [2023-12-26 17:07:53,023][105692] Updated weights for policy 0, policy_version 242698 (0.0009) [2023-12-26 17:07:53,471][105620] Updated weights for policy 1, policy_version 243053 (0.0007) [2023-12-26 17:07:53,528][105620] Updated weights for policy 1, policy_version 243063 (0.0009) [2023-12-26 17:07:53,580][105620] Updated weights for policy 1, policy_version 243073 (0.0010) [2023-12-26 17:07:53,786][105692] Updated weights for policy 0, policy_version 242708 (0.0009) [2023-12-26 17:07:53,851][105692] Updated weights for policy 0, policy_version 242718 (0.0008) [2023-12-26 17:07:53,914][105692] Updated weights for policy 0, policy_version 242728 (0.0009) [2023-12-26 17:07:54,210][105620] Updated weights for policy 1, policy_version 243083 (0.0009) [2023-12-26 17:07:54,267][105620] Updated weights for policy 1, policy_version 243093 (0.0005) [2023-12-26 17:07:54,325][105620] Updated weights for policy 1, policy_version 243103 (0.0006) [2023-12-26 17:07:54,677][105692] Updated weights for policy 0, policy_version 242738 (0.0009) [2023-12-26 17:07:54,735][105692] Updated weights for policy 0, policy_version 242748 (0.0010) [2023-12-26 17:07:54,793][105692] Updated weights for policy 0, policy_version 242758 (0.0010) [2023-12-26 17:07:54,854][105692] Updated weights for policy 0, policy_version 242768 (0.0010) [2023-12-26 17:07:54,970][105620] Updated weights for policy 1, policy_version 243113 (0.0006) [2023-12-26 17:07:55,027][105620] Updated weights for policy 1, policy_version 243123 (0.0007) [2023-12-26 17:07:55,089][105620] Updated weights for policy 1, policy_version 243133 (0.0008) [2023-12-26 17:07:55,144][105620] Updated weights for policy 1, policy_version 243143 (0.0010) [2023-12-26 17:07:55,534][105692] Updated weights for policy 0, policy_version 242778 (0.0005) [2023-12-26 17:07:55,596][105692] Updated weights for policy 0, policy_version 242788 (0.0007) [2023-12-26 17:07:55,660][105692] Updated weights for policy 0, policy_version 242798 (0.0008) [2023-12-26 17:07:55,810][105620] Updated weights for policy 1, policy_version 243153 (0.0006) [2023-12-26 17:07:55,863][105620] Updated weights for policy 1, policy_version 243163 (0.0005) [2023-12-26 17:07:55,919][105620] Updated weights for policy 1, policy_version 243173 (0.0008) [2023-12-26 17:07:56,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 124428288. Throughput: 0: 9826.3, 1: 9899.2. Samples: 124434916. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-26 17:07:56,063][104569] Avg episode reward: [(0, '9357.260'), (1, '5969.045')] [2023-12-26 17:07:56,397][105692] Updated weights for policy 0, policy_version 242809 (0.0010) [2023-12-26 17:07:56,464][105692] Updated weights for policy 0, policy_version 242819 (0.0010) [2023-12-26 17:07:56,531][105692] Updated weights for policy 0, policy_version 242829 (0.0009) [2023-12-26 17:07:56,536][105620] Updated weights for policy 1, policy_version 243183 (0.0008) [2023-12-26 17:07:56,600][105620] Updated weights for policy 1, policy_version 243193 (0.0008) [2023-12-26 17:07:56,666][105620] Updated weights for policy 1, policy_version 243203 (0.0005) [2023-12-26 17:07:57,203][105620] Updated weights for policy 1, policy_version 243213 (0.0005) [2023-12-26 17:07:57,254][105620] Updated weights for policy 1, policy_version 243223 (0.0006) [2023-12-26 17:07:57,275][105692] Updated weights for policy 0, policy_version 242839 (0.0007) [2023-12-26 17:07:57,305][105620] Updated weights for policy 1, policy_version 243233 (0.0010) [2023-12-26 17:07:57,331][105692] Updated weights for policy 0, policy_version 242849 (0.0006) [2023-12-26 17:07:57,382][105692] Updated weights for policy 0, policy_version 242859 (0.0006) [2023-12-26 17:07:57,926][105620] Updated weights for policy 1, policy_version 243243 (0.0005) [2023-12-26 17:07:57,980][105692] Updated weights for policy 0, policy_version 242869 (0.0005) [2023-12-26 17:07:57,994][105620] Updated weights for policy 1, policy_version 243253 (0.0007) [2023-12-26 17:07:58,041][105692] Updated weights for policy 0, policy_version 242879 (0.0009) [2023-12-26 17:07:58,043][105620] Updated weights for policy 1, policy_version 243263 (0.0010) [2023-12-26 17:07:58,102][105692] Updated weights for policy 0, policy_version 242889 (0.0011) [2023-12-26 17:07:58,795][105620] Updated weights for policy 1, policy_version 243273 (0.0010) [2023-12-26 17:07:58,868][105620] Updated weights for policy 1, policy_version 243283 (0.0008) [2023-12-26 17:07:58,928][105620] Updated weights for policy 1, policy_version 243293 (0.0007) [2023-12-26 17:07:58,945][105692] Updated weights for policy 0, policy_version 242899 (0.0010) [2023-12-26 17:07:58,989][105620] Updated weights for policy 1, policy_version 243303 (0.0007) [2023-12-26 17:07:59,002][105692] Updated weights for policy 0, policy_version 242909 (0.0009) [2023-12-26 17:07:59,050][105692] Updated weights for policy 0, policy_version 242919 (0.0009) [2023-12-26 17:07:59,743][105620] Updated weights for policy 1, policy_version 243313 (0.0010) [2023-12-26 17:07:59,796][105620] Updated weights for policy 1, policy_version 243323 (0.0009) [2023-12-26 17:07:59,833][105692] Updated weights for policy 0, policy_version 242929 (0.0008) [2023-12-26 17:07:59,859][105620] Updated weights for policy 1, policy_version 243333 (0.0009) [2023-12-26 17:07:59,897][105692] Updated weights for policy 0, policy_version 242939 (0.0010) [2023-12-26 17:07:59,959][105692] Updated weights for policy 0, policy_version 242949 (0.0010) [2023-12-26 17:08:00,014][105692] Updated weights for policy 0, policy_version 242959 (0.0008) [2023-12-26 17:08:00,546][105620] Updated weights for policy 1, policy_version 243343 (0.0006) [2023-12-26 17:08:00,595][105620] Updated weights for policy 1, policy_version 243353 (0.0005) [2023-12-26 17:08:00,647][105620] Updated weights for policy 1, policy_version 243363 (0.0005) [2023-12-26 17:08:00,670][105692] Updated weights for policy 0, policy_version 242969 (0.0008) [2023-12-26 17:08:00,729][105692] Updated weights for policy 0, policy_version 242979 (0.0005) [2023-12-26 17:08:00,779][105692] Updated weights for policy 0, policy_version 242989 (0.0005) [2023-12-26 17:08:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 124526592. Throughput: 0: 9802.0, 1: 9984.3. Samples: 124495676. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-26 17:08:01,063][104569] Avg episode reward: [(0, '9175.738'), (1, '6644.833')] [2023-12-26 17:08:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000242992_62218240.pth... [2023-12-26 17:08:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000243368_62308352.pth... [2023-12-26 17:08:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000242184_62005248.pth [2023-12-26 17:08:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000241872_61931520.pth [2023-12-26 17:08:01,296][105620] Updated weights for policy 1, policy_version 243373 (0.0008) [2023-12-26 17:08:01,342][105620] Updated weights for policy 1, policy_version 243383 (0.0008) [2023-12-26 17:08:01,406][105620] Updated weights for policy 1, policy_version 243393 (0.0008) [2023-12-26 17:08:01,441][105692] Updated weights for policy 0, policy_version 242999 (0.0005) [2023-12-26 17:08:01,485][105692] Updated weights for policy 0, policy_version 243009 (0.0005) [2023-12-26 17:08:01,531][105692] Updated weights for policy 0, policy_version 243019 (0.0005) [2023-12-26 17:08:02,166][105692] Updated weights for policy 0, policy_version 243029 (0.0007) [2023-12-26 17:08:02,212][105620] Updated weights for policy 1, policy_version 243403 (0.0008) [2023-12-26 17:08:02,230][105692] Updated weights for policy 0, policy_version 243039 (0.0006) [2023-12-26 17:08:02,269][105620] Updated weights for policy 1, policy_version 243413 (0.0009) [2023-12-26 17:08:02,292][105692] Updated weights for policy 0, policy_version 243049 (0.0007) [2023-12-26 17:08:02,330][105620] Updated weights for policy 1, policy_version 243423 (0.0010) [2023-12-26 17:08:02,917][105692] Updated weights for policy 0, policy_version 243059 (0.0007) [2023-12-26 17:08:02,963][105620] Updated weights for policy 1, policy_version 243433 (0.0011) [2023-12-26 17:08:02,971][105692] Updated weights for policy 0, policy_version 243069 (0.0008) [2023-12-26 17:08:03,014][105620] Updated weights for policy 1, policy_version 243443 (0.0010) [2023-12-26 17:08:03,029][105692] Updated weights for policy 0, policy_version 243079 (0.0008) [2023-12-26 17:08:03,069][105620] Updated weights for policy 1, policy_version 243453 (0.0010) [2023-12-26 17:08:03,123][105620] Updated weights for policy 1, policy_version 243463 (0.0010) [2023-12-26 17:08:03,715][105692] Updated weights for policy 0, policy_version 243089 (0.0007) [2023-12-26 17:08:03,766][105692] Updated weights for policy 0, policy_version 243099 (0.0005) [2023-12-26 17:08:03,821][105692] Updated weights for policy 0, policy_version 243109 (0.0007) [2023-12-26 17:08:03,886][105692] Updated weights for policy 0, policy_version 243119 (0.0007) [2023-12-26 17:08:03,924][105620] Updated weights for policy 1, policy_version 243473 (0.0010) [2023-12-26 17:08:03,982][105620] Updated weights for policy 1, policy_version 243483 (0.0010) [2023-12-26 17:08:04,040][105620] Updated weights for policy 1, policy_version 243493 (0.0010) [2023-12-26 17:08:04,543][105692] Updated weights for policy 0, policy_version 243129 (0.0009) [2023-12-26 17:08:04,602][105692] Updated weights for policy 0, policy_version 243140 (0.0008) [2023-12-26 17:08:04,666][105692] Updated weights for policy 0, policy_version 243150 (0.0008) [2023-12-26 17:08:04,835][105620] Updated weights for policy 1, policy_version 243503 (0.0007) [2023-12-26 17:08:04,892][105620] Updated weights for policy 1, policy_version 243513 (0.0006) [2023-12-26 17:08:04,946][105620] Updated weights for policy 1, policy_version 243523 (0.0005) [2023-12-26 17:08:05,496][105620] Updated weights for policy 1, policy_version 243533 (0.0007) [2023-12-26 17:08:05,501][105692] Updated weights for policy 0, policy_version 243160 (0.0006) [2023-12-26 17:08:05,553][105692] Updated weights for policy 0, policy_version 243170 (0.0008) [2023-12-26 17:08:05,557][105620] Updated weights for policy 1, policy_version 243543 (0.0005) [2023-12-26 17:08:05,607][105620] Updated weights for policy 1, policy_version 243553 (0.0005) [2023-12-26 17:08:05,607][105692] Updated weights for policy 0, policy_version 243180 (0.0008) [2023-12-26 17:08:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19494.2). Total num frames: 124624896. Throughput: 0: 9845.5, 1: 9965.7. Samples: 124614348. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-26 17:08:06,062][104569] Avg episode reward: [(0, '9085.187'), (1, '6985.748')] [2023-12-26 17:08:06,263][105620] Updated weights for policy 1, policy_version 243563 (0.0006) [2023-12-26 17:08:06,314][105692] Updated weights for policy 0, policy_version 243190 (0.0007) [2023-12-26 17:08:06,326][105620] Updated weights for policy 1, policy_version 243573 (0.0008) [2023-12-26 17:08:06,372][105692] Updated weights for policy 0, policy_version 243200 (0.0005) [2023-12-26 17:08:06,391][105620] Updated weights for policy 1, policy_version 243583 (0.0010) [2023-12-26 17:08:06,420][105692] Updated weights for policy 0, policy_version 243210 (0.0005) [2023-12-26 17:08:07,083][105692] Updated weights for policy 0, policy_version 243220 (0.0007) [2023-12-26 17:08:07,145][105692] Updated weights for policy 0, policy_version 243230 (0.0009) [2023-12-26 17:08:07,175][105620] Updated weights for policy 1, policy_version 243593 (0.0008) [2023-12-26 17:08:07,197][105692] Updated weights for policy 0, policy_version 243240 (0.0009) [2023-12-26 17:08:07,237][105620] Updated weights for policy 1, policy_version 243603 (0.0006) [2023-12-26 17:08:07,299][105620] Updated weights for policy 1, policy_version 243613 (0.0006) [2023-12-26 17:08:07,360][105620] Updated weights for policy 1, policy_version 243623 (0.0006) [2023-12-26 17:08:07,951][105692] Updated weights for policy 0, policy_version 243250 (0.0009) [2023-12-26 17:08:07,999][105692] Updated weights for policy 0, policy_version 243260 (0.0010) [2023-12-26 17:08:08,048][105620] Updated weights for policy 1, policy_version 243633 (0.0007) [2023-12-26 17:08:08,050][105692] Updated weights for policy 0, policy_version 243270 (0.0009) [2023-12-26 17:08:08,107][105620] Updated weights for policy 1, policy_version 243643 (0.0008) [2023-12-26 17:08:08,110][105692] Updated weights for policy 0, policy_version 243280 (0.0007) [2023-12-26 17:08:08,155][105620] Updated weights for policy 1, policy_version 243653 (0.0008) [2023-12-26 17:08:08,847][105692] Updated weights for policy 0, policy_version 243290 (0.0011) [2023-12-26 17:08:08,909][105692] Updated weights for policy 0, policy_version 243300 (0.0010) [2023-12-26 17:08:08,923][105620] Updated weights for policy 1, policy_version 243663 (0.0006) [2023-12-26 17:08:08,965][105692] Updated weights for policy 0, policy_version 243310 (0.0011) [2023-12-26 17:08:08,980][105620] Updated weights for policy 1, policy_version 243673 (0.0006) [2023-12-26 17:08:09,032][105620] Updated weights for policy 1, policy_version 243683 (0.0008) [2023-12-26 17:08:09,728][105692] Updated weights for policy 0, policy_version 243320 (0.0011) [2023-12-26 17:08:09,785][105692] Updated weights for policy 0, policy_version 243330 (0.0011) [2023-12-26 17:08:09,847][105692] Updated weights for policy 0, policy_version 243340 (0.0011) [2023-12-26 17:08:09,848][105620] Updated weights for policy 1, policy_version 243693 (0.0009) [2023-12-26 17:08:09,915][105620] Updated weights for policy 1, policy_version 243703 (0.0009) [2023-12-26 17:08:09,985][105620] Updated weights for policy 1, policy_version 243713 (0.0007) [2023-12-26 17:08:10,607][105692] Updated weights for policy 0, policy_version 243350 (0.0009) [2023-12-26 17:08:10,641][105620] Updated weights for policy 1, policy_version 243723 (0.0006) [2023-12-26 17:08:10,660][105692] Updated weights for policy 0, policy_version 243360 (0.0011) [2023-12-26 17:08:10,704][105620] Updated weights for policy 1, policy_version 243733 (0.0007) [2023-12-26 17:08:10,716][105692] Updated weights for policy 0, policy_version 243370 (0.0008) [2023-12-26 17:08:10,761][105620] Updated weights for policy 1, policy_version 243743 (0.0009) [2023-12-26 17:08:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 124723200. Throughput: 0: 9742.2, 1: 9990.1. Samples: 124729916. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-26 17:08:11,062][104569] Avg episode reward: [(0, '9266.533'), (1, '7641.487')] [2023-12-26 17:08:11,452][105692] Updated weights for policy 0, policy_version 243380 (0.0006) [2023-12-26 17:08:11,520][105692] Updated weights for policy 0, policy_version 243390 (0.0007) [2023-12-26 17:08:11,541][105620] Updated weights for policy 1, policy_version 243753 (0.0008) [2023-12-26 17:08:11,586][105692] Updated weights for policy 0, policy_version 243400 (0.0009) [2023-12-26 17:08:11,605][105620] Updated weights for policy 1, policy_version 243763 (0.0009) [2023-12-26 17:08:11,665][105620] Updated weights for policy 1, policy_version 243773 (0.0008) [2023-12-26 17:08:11,692][105586] KL-divergence is very high: 141.4749 [2023-12-26 17:08:11,738][105620] Updated weights for policy 1, policy_version 243783 (0.0009) [2023-12-26 17:08:12,230][105692] Updated weights for policy 0, policy_version 243410 (0.0009) [2023-12-26 17:08:12,291][105692] Updated weights for policy 0, policy_version 243420 (0.0009) [2023-12-26 17:08:12,352][105692] Updated weights for policy 0, policy_version 243430 (0.0009) [2023-12-26 17:08:12,407][105692] Updated weights for policy 0, policy_version 243440 (0.0008) [2023-12-26 17:08:12,457][105620] Updated weights for policy 1, policy_version 243793 (0.0009) [2023-12-26 17:08:12,518][105620] Updated weights for policy 1, policy_version 243803 (0.0008) [2023-12-26 17:08:12,572][105620] Updated weights for policy 1, policy_version 243813 (0.0009) [2023-12-26 17:08:13,088][105692] Updated weights for policy 0, policy_version 243450 (0.0005) [2023-12-26 17:08:13,154][105692] Updated weights for policy 0, policy_version 243460 (0.0006) [2023-12-26 17:08:13,220][105692] Updated weights for policy 0, policy_version 243470 (0.0008) [2023-12-26 17:08:13,338][105620] Updated weights for policy 1, policy_version 243823 (0.0007) [2023-12-26 17:08:13,391][105620] Updated weights for policy 1, policy_version 243833 (0.0005) [2023-12-26 17:08:13,458][105620] Updated weights for policy 1, policy_version 243843 (0.0005) [2023-12-26 17:08:13,855][105692] Updated weights for policy 0, policy_version 243480 (0.0009) [2023-12-26 17:08:13,912][105692] Updated weights for policy 0, policy_version 243490 (0.0010) [2023-12-26 17:08:13,951][105620] Updated weights for policy 1, policy_version 243853 (0.0005) [2023-12-26 17:08:13,965][105692] Updated weights for policy 0, policy_version 243500 (0.0009) [2023-12-26 17:08:14,014][105620] Updated weights for policy 1, policy_version 243863 (0.0005) [2023-12-26 17:08:14,069][105620] Updated weights for policy 1, policy_version 243873 (0.0007) [2023-12-26 17:08:14,673][105620] Updated weights for policy 1, policy_version 243883 (0.0006) [2023-12-26 17:08:14,736][105620] Updated weights for policy 1, policy_version 243893 (0.0005) [2023-12-26 17:08:14,790][105692] Updated weights for policy 0, policy_version 243510 (0.0009) [2023-12-26 17:08:14,803][105620] Updated weights for policy 1, policy_version 243903 (0.0008) [2023-12-26 17:08:14,850][105692] Updated weights for policy 0, policy_version 243520 (0.0007) [2023-12-26 17:08:14,914][105692] Updated weights for policy 0, policy_version 243530 (0.0007) [2023-12-26 17:08:15,557][105620] Updated weights for policy 1, policy_version 243913 (0.0008) [2023-12-26 17:08:15,612][105620] Updated weights for policy 1, policy_version 243923 (0.0007) [2023-12-26 17:08:15,621][105692] Updated weights for policy 0, policy_version 243540 (0.0008) [2023-12-26 17:08:15,670][105692] Updated weights for policy 0, policy_version 243550 (0.0006) [2023-12-26 17:08:15,676][105620] Updated weights for policy 1, policy_version 243933 (0.0008) [2023-12-26 17:08:15,716][105692] Updated weights for policy 0, policy_version 243560 (0.0006) [2023-12-26 17:08:15,720][105620] Updated weights for policy 1, policy_version 243943 (0.0006) [2023-12-26 17:08:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 124821504. Throughput: 0: 9687.8, 1: 9971.6. Samples: 124790412. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-26 17:08:16,063][104569] Avg episode reward: [(0, '9356.975'), (1, '7709.631')] [2023-12-26 17:08:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000243568_62365696.pth... [2023-12-26 17:08:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000243944_62455808.pth... [2023-12-26 17:08:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000242792_62160896.pth [2023-12-26 17:08:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000242448_62078976.pth [2023-12-26 17:08:16,425][105620] Updated weights for policy 1, policy_version 243953 (0.0005) [2023-12-26 17:08:16,493][105620] Updated weights for policy 1, policy_version 243963 (0.0005) [2023-12-26 17:08:16,506][105692] Updated weights for policy 0, policy_version 243570 (0.0008) [2023-12-26 17:08:16,553][105620] Updated weights for policy 1, policy_version 243973 (0.0008) [2023-12-26 17:08:16,560][105692] Updated weights for policy 0, policy_version 243580 (0.0007) [2023-12-26 17:08:16,611][105692] Updated weights for policy 0, policy_version 243590 (0.0007) [2023-12-26 17:08:16,655][105692] Updated weights for policy 0, policy_version 243600 (0.0008) [2023-12-26 17:08:17,186][105620] Updated weights for policy 1, policy_version 243983 (0.0010) [2023-12-26 17:08:17,236][105620] Updated weights for policy 1, policy_version 243993 (0.0008) [2023-12-26 17:08:17,291][105620] Updated weights for policy 1, policy_version 244003 (0.0009) [2023-12-26 17:08:17,427][105692] Updated weights for policy 0, policy_version 243610 (0.0009) [2023-12-26 17:08:17,473][105692] Updated weights for policy 0, policy_version 243620 (0.0008) [2023-12-26 17:08:17,534][105692] Updated weights for policy 0, policy_version 243630 (0.0009) [2023-12-26 17:08:18,018][105620] Updated weights for policy 1, policy_version 244013 (0.0007) [2023-12-26 17:08:18,077][105620] Updated weights for policy 1, policy_version 244023 (0.0007) [2023-12-26 17:08:18,128][105620] Updated weights for policy 1, policy_version 244033 (0.0008) [2023-12-26 17:08:18,330][105692] Updated weights for policy 0, policy_version 243640 (0.0009) [2023-12-26 17:08:18,389][105692] Updated weights for policy 0, policy_version 243650 (0.0009) [2023-12-26 17:08:18,440][105692] Updated weights for policy 0, policy_version 243660 (0.0009) [2023-12-26 17:08:18,855][105620] Updated weights for policy 1, policy_version 244043 (0.0008) [2023-12-26 17:08:18,916][105620] Updated weights for policy 1, policy_version 244053 (0.0008) [2023-12-26 17:08:18,971][105620] Updated weights for policy 1, policy_version 244063 (0.0009) [2023-12-26 17:08:19,169][105692] Updated weights for policy 0, policy_version 243670 (0.0008) [2023-12-26 17:08:19,225][105692] Updated weights for policy 0, policy_version 243680 (0.0009) [2023-12-26 17:08:19,285][105692] Updated weights for policy 0, policy_version 243690 (0.0005) [2023-12-26 17:08:19,690][105620] Updated weights for policy 1, policy_version 244074 (0.0010) [2023-12-26 17:08:19,761][105620] Updated weights for policy 1, policy_version 244084 (0.0009) [2023-12-26 17:08:19,825][105620] Updated weights for policy 1, policy_version 244094 (0.0010) [2023-12-26 17:08:19,894][105620] Updated weights for policy 1, policy_version 244104 (0.0009) [2023-12-26 17:08:20,055][105692] Updated weights for policy 0, policy_version 243700 (0.0007) [2023-12-26 17:08:20,118][105692] Updated weights for policy 0, policy_version 243710 (0.0009) [2023-12-26 17:08:20,189][105692] Updated weights for policy 0, policy_version 243720 (0.0010) [2023-12-26 17:08:20,615][105620] Updated weights for policy 1, policy_version 244114 (0.0009) [2023-12-26 17:08:20,681][105620] Updated weights for policy 1, policy_version 244124 (0.0006) [2023-12-26 17:08:20,751][105620] Updated weights for policy 1, policy_version 244134 (0.0006) [2023-12-26 17:08:20,964][105692] Updated weights for policy 0, policy_version 243730 (0.0010) [2023-12-26 17:08:21,023][105692] Updated weights for policy 0, policy_version 243740 (0.0009) [2023-12-26 17:08:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 124911616. Throughput: 0: 9592.8, 1: 9937.8. Samples: 124904620. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-26 17:08:21,062][104569] Avg episode reward: [(0, '9356.796'), (1, '7253.781')] [2023-12-26 17:08:21,085][105692] Updated weights for policy 0, policy_version 243750 (0.0008) [2023-12-26 17:08:21,141][105692] Updated weights for policy 0, policy_version 243760 (0.0008) [2023-12-26 17:08:21,438][105620] Updated weights for policy 1, policy_version 244144 (0.0009) [2023-12-26 17:08:21,486][105620] Updated weights for policy 1, policy_version 244154 (0.0009) [2023-12-26 17:08:21,533][105620] Updated weights for policy 1, policy_version 244164 (0.0008) [2023-12-26 17:08:21,912][105692] Updated weights for policy 0, policy_version 243770 (0.0006) [2023-12-26 17:08:21,978][105692] Updated weights for policy 0, policy_version 243780 (0.0006) [2023-12-26 17:08:22,047][105692] Updated weights for policy 0, policy_version 243790 (0.0005) [2023-12-26 17:08:22,429][105620] Updated weights for policy 1, policy_version 244174 (0.0010) [2023-12-26 17:08:22,493][105620] Updated weights for policy 1, policy_version 244184 (0.0008) [2023-12-26 17:08:22,555][105620] Updated weights for policy 1, policy_version 244194 (0.0008) [2023-12-26 17:08:22,625][105692] Updated weights for policy 0, policy_version 243800 (0.0006) [2023-12-26 17:08:22,685][105692] Updated weights for policy 0, policy_version 243810 (0.0009) [2023-12-26 17:08:22,741][105692] Updated weights for policy 0, policy_version 243820 (0.0009) [2023-12-26 17:08:23,326][105620] Updated weights for policy 1, policy_version 244204 (0.0009) [2023-12-26 17:08:23,380][105620] Updated weights for policy 1, policy_version 244214 (0.0010) [2023-12-26 17:08:23,435][105620] Updated weights for policy 1, policy_version 244224 (0.0010) [2023-12-26 17:08:23,437][105692] Updated weights for policy 0, policy_version 243830 (0.0007) [2023-12-26 17:08:23,494][105692] Updated weights for policy 0, policy_version 243840 (0.0005) [2023-12-26 17:08:23,557][105692] Updated weights for policy 0, policy_version 243850 (0.0005) [2023-12-26 17:08:24,077][105692] Updated weights for policy 0, policy_version 243860 (0.0007) [2023-12-26 17:08:24,134][105692] Updated weights for policy 0, policy_version 243870 (0.0008) [2023-12-26 17:08:24,180][105620] Updated weights for policy 1, policy_version 244234 (0.0008) [2023-12-26 17:08:24,189][105692] Updated weights for policy 0, policy_version 243880 (0.0008) [2023-12-26 17:08:24,231][105620] Updated weights for policy 1, policy_version 244244 (0.0005) [2023-12-26 17:08:24,275][105620] Updated weights for policy 1, policy_version 244254 (0.0005) [2023-12-26 17:08:24,320][105620] Updated weights for policy 1, policy_version 244264 (0.0006) [2023-12-26 17:08:24,885][105692] Updated weights for policy 0, policy_version 243890 (0.0008) [2023-12-26 17:08:24,947][105692] Updated weights for policy 0, policy_version 243900 (0.0009) [2023-12-26 17:08:25,007][105692] Updated weights for policy 0, policy_version 243910 (0.0009) [2023-12-26 17:08:25,064][105620] Updated weights for policy 1, policy_version 244274 (0.0008) [2023-12-26 17:08:25,068][105692] Updated weights for policy 0, policy_version 243920 (0.0006) [2023-12-26 17:08:25,118][105620] Updated weights for policy 1, policy_version 244284 (0.0010) [2023-12-26 17:08:25,172][105620] Updated weights for policy 1, policy_version 244294 (0.0010) [2023-12-26 17:08:25,695][105692] Updated weights for policy 0, policy_version 243930 (0.0008) [2023-12-26 17:08:25,741][105692] Updated weights for policy 0, policy_version 243940 (0.0009) [2023-12-26 17:08:25,788][105692] Updated weights for policy 0, policy_version 243950 (0.0009) [2023-12-26 17:08:25,945][105620] Updated weights for policy 1, policy_version 244304 (0.0009) [2023-12-26 17:08:25,952][105586] KL-divergence is very high: 155.6987 [2023-12-26 17:08:26,003][105586] KL-divergence is very high: 252.2541 [2023-12-26 17:08:26,010][105620] Updated weights for policy 1, policy_version 244314 (0.0009) [2023-12-26 17:08:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 125009920. Throughput: 0: 9673.0, 1: 9883.6. Samples: 125021416. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-26 17:08:26,062][105586] KL-divergence is very high: 258.8132 [2023-12-26 17:08:26,062][104569] Avg episode reward: [(0, '9356.896'), (1, '7423.582')] [2023-12-26 17:08:26,083][105620] Updated weights for policy 1, policy_version 244324 (0.0009) [2023-12-26 17:08:26,454][105692] Updated weights for policy 0, policy_version 243960 (0.0007) [2023-12-26 17:08:26,508][105692] Updated weights for policy 0, policy_version 243970 (0.0006) [2023-12-26 17:08:26,559][105692] Updated weights for policy 0, policy_version 243980 (0.0005) [2023-12-26 17:08:26,841][105620] Updated weights for policy 1, policy_version 244334 (0.0007) [2023-12-26 17:08:26,898][105620] Updated weights for policy 1, policy_version 244344 (0.0005) [2023-12-26 17:08:26,961][105620] Updated weights for policy 1, policy_version 244354 (0.0005) [2023-12-26 17:08:27,151][105692] Updated weights for policy 0, policy_version 243990 (0.0005) [2023-12-26 17:08:27,200][105692] Updated weights for policy 0, policy_version 244000 (0.0005) [2023-12-26 17:08:27,246][105692] Updated weights for policy 0, policy_version 244010 (0.0006) [2023-12-26 17:08:27,563][105620] Updated weights for policy 1, policy_version 244364 (0.0007) [2023-12-26 17:08:27,615][105620] Updated weights for policy 1, policy_version 244374 (0.0009) [2023-12-26 17:08:27,668][105620] Updated weights for policy 1, policy_version 244385 (0.0010) [2023-12-26 17:08:27,798][105692] Updated weights for policy 0, policy_version 244020 (0.0008) [2023-12-26 17:08:27,854][105692] Updated weights for policy 0, policy_version 244030 (0.0005) [2023-12-26 17:08:27,909][105692] Updated weights for policy 0, policy_version 244040 (0.0005) [2023-12-26 17:08:28,430][105692] Updated weights for policy 0, policy_version 244050 (0.0006) [2023-12-26 17:08:28,482][105692] Updated weights for policy 0, policy_version 244060 (0.0007) [2023-12-26 17:08:28,535][105692] Updated weights for policy 0, policy_version 244070 (0.0005) [2023-12-26 17:08:28,540][105585] KL-divergence is very high: 107.3352 [2023-12-26 17:08:28,569][105620] Updated weights for policy 1, policy_version 244396 (0.0010) [2023-12-26 17:08:28,587][105585] KL-divergence is very high: 100.7129 [2023-12-26 17:08:28,592][105692] Updated weights for policy 0, policy_version 244080 (0.0005) [2023-12-26 17:08:28,635][105620] Updated weights for policy 1, policy_version 244406 (0.0009) [2023-12-26 17:08:28,702][105620] Updated weights for policy 1, policy_version 244416 (0.0010) [2023-12-26 17:08:29,139][105692] Updated weights for policy 0, policy_version 244090 (0.0006) [2023-12-26 17:08:29,187][105692] Updated weights for policy 0, policy_version 244100 (0.0005) [2023-12-26 17:08:29,247][105692] Updated weights for policy 0, policy_version 244110 (0.0006) [2023-12-26 17:08:29,499][105620] Updated weights for policy 1, policy_version 244426 (0.0008) [2023-12-26 17:08:29,563][105620] Updated weights for policy 1, policy_version 244436 (0.0006) [2023-12-26 17:08:29,622][105620] Updated weights for policy 1, policy_version 244446 (0.0008) [2023-12-26 17:08:29,805][105692] Updated weights for policy 0, policy_version 244120 (0.0006) [2023-12-26 17:08:29,869][105692] Updated weights for policy 0, policy_version 244130 (0.0009) [2023-12-26 17:08:29,921][105692] Updated weights for policy 0, policy_version 244140 (0.0010) [2023-12-26 17:08:30,312][105620] Updated weights for policy 1, policy_version 244457 (0.0010) [2023-12-26 17:08:30,383][105620] Updated weights for policy 1, policy_version 244467 (0.0009) [2023-12-26 17:08:30,452][105620] Updated weights for policy 1, policy_version 244477 (0.0006) [2023-12-26 17:08:30,514][105620] Updated weights for policy 1, policy_version 244487 (0.0005) [2023-12-26 17:08:30,635][105692] Updated weights for policy 0, policy_version 244150 (0.0008) [2023-12-26 17:08:30,683][105692] Updated weights for policy 0, policy_version 244160 (0.0006) [2023-12-26 17:08:30,738][105692] Updated weights for policy 0, policy_version 244170 (0.0010) [2023-12-26 17:08:31,001][105586] KL-divergence is very high: 102.7796 [2023-12-26 17:08:31,046][105586] KL-divergence is very high: 233.0087 [2023-12-26 17:08:31,049][105620] Updated weights for policy 1, policy_version 244497 (0.0007) [2023-12-26 17:08:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 125116416. Throughput: 0: 9771.8, 1: 9846.4. Samples: 125083564. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-26 17:08:31,062][104569] Avg episode reward: [(0, '9173.778'), (1, '7702.055')] [2023-12-26 17:08:31,065][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000244176_62521344.pth... [2023-12-26 17:08:31,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000242992_62218240.pth [2023-12-26 17:08:31,090][105586] KL-divergence is very high: 213.1373 [2023-12-26 17:08:31,111][105620] Updated weights for policy 1, policy_version 244507 (0.0008) [2023-12-26 17:08:31,144][105586] KL-divergence is very high: 180.9113 [2023-12-26 17:08:31,179][105620] Updated weights for policy 1, policy_version 244517 (0.0007) [2023-12-26 17:08:31,198][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000244520_62603264.pth... [2023-12-26 17:08:31,203][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000243368_62308352.pth [2023-12-26 17:08:31,527][105692] Updated weights for policy 0, policy_version 244180 (0.0010) [2023-12-26 17:08:31,582][105692] Updated weights for policy 0, policy_version 244190 (0.0009) [2023-12-26 17:08:31,634][105692] Updated weights for policy 0, policy_version 244200 (0.0009) [2023-12-26 17:08:31,881][105620] Updated weights for policy 1, policy_version 244527 (0.0005) [2023-12-26 17:08:31,940][105620] Updated weights for policy 1, policy_version 244537 (0.0008) [2023-12-26 17:08:31,941][105586] KL-divergence is very high: 112.4681 [2023-12-26 17:08:31,992][105586] KL-divergence is very high: 101.1898 [2023-12-26 17:08:32,003][105620] Updated weights for policy 1, policy_version 244547 (0.0009) [2023-12-26 17:08:32,458][105692] Updated weights for policy 0, policy_version 244210 (0.0009) [2023-12-26 17:08:32,524][105692] Updated weights for policy 0, policy_version 244220 (0.0009) [2023-12-26 17:08:32,592][105692] Updated weights for policy 0, policy_version 244230 (0.0009) [2023-12-26 17:08:32,651][105692] Updated weights for policy 0, policy_version 244240 (0.0008) [2023-12-26 17:08:32,653][105620] Updated weights for policy 1, policy_version 244557 (0.0008) [2023-12-26 17:08:32,714][105620] Updated weights for policy 1, policy_version 244567 (0.0007) [2023-12-26 17:08:32,782][105620] Updated weights for policy 1, policy_version 244577 (0.0009) [2023-12-26 17:08:33,408][105620] Updated weights for policy 1, policy_version 244587 (0.0008) [2023-12-26 17:08:33,458][105692] Updated weights for policy 0, policy_version 244250 (0.0009) [2023-12-26 17:08:33,471][105620] Updated weights for policy 1, policy_version 244597 (0.0006) [2023-12-26 17:08:33,505][105692] Updated weights for policy 0, policy_version 244260 (0.0006) [2023-12-26 17:08:33,527][105620] Updated weights for policy 1, policy_version 244607 (0.0007) [2023-12-26 17:08:33,559][105692] Updated weights for policy 0, policy_version 244270 (0.0008) [2023-12-26 17:08:34,239][105620] Updated weights for policy 1, policy_version 244617 (0.0009) [2023-12-26 17:08:34,297][105620] Updated weights for policy 1, policy_version 244627 (0.0008) [2023-12-26 17:08:34,323][105692] Updated weights for policy 0, policy_version 244280 (0.0009) [2023-12-26 17:08:34,362][105620] Updated weights for policy 1, policy_version 244637 (0.0008) [2023-12-26 17:08:34,384][105692] Updated weights for policy 0, policy_version 244290 (0.0009) [2023-12-26 17:08:34,430][105620] Updated weights for policy 1, policy_version 244647 (0.0008) [2023-12-26 17:08:34,437][105692] Updated weights for policy 0, policy_version 244300 (0.0005) [2023-12-26 17:08:35,177][105620] Updated weights for policy 1, policy_version 244657 (0.0008) [2023-12-26 17:08:35,209][105692] Updated weights for policy 0, policy_version 244310 (0.0008) [2023-12-26 17:08:35,228][105620] Updated weights for policy 1, policy_version 244667 (0.0007) [2023-12-26 17:08:35,274][105692] Updated weights for policy 0, policy_version 244320 (0.0009) [2023-12-26 17:08:35,281][105620] Updated weights for policy 1, policy_version 244677 (0.0006) [2023-12-26 17:08:35,331][105692] Updated weights for policy 0, policy_version 244330 (0.0009) [2023-12-26 17:08:36,057][105620] Updated weights for policy 1, policy_version 244687 (0.0009) [2023-12-26 17:08:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 125206528. Throughput: 0: 9840.4, 1: 9813.4. Samples: 125202684. Policy #0 lag: (min: 1.0, avg: 23.7, max: 33.0) [2023-12-26 17:08:36,062][104569] Avg episode reward: [(0, '8319.938'), (1, '7609.938')] [2023-12-26 17:08:36,081][105692] Updated weights for policy 0, policy_version 244340 (0.0008) [2023-12-26 17:08:36,121][105620] Updated weights for policy 1, policy_version 244697 (0.0009) [2023-12-26 17:08:36,141][105692] Updated weights for policy 0, policy_version 244350 (0.0008) [2023-12-26 17:08:36,180][105620] Updated weights for policy 1, policy_version 244707 (0.0008) [2023-12-26 17:08:36,199][105692] Updated weights for policy 0, policy_version 244360 (0.0010) [2023-12-26 17:08:36,833][105692] Updated weights for policy 0, policy_version 244370 (0.0008) [2023-12-26 17:08:36,898][105692] Updated weights for policy 0, policy_version 244380 (0.0008) [2023-12-26 17:08:36,964][105692] Updated weights for policy 0, policy_version 244390 (0.0008) [2023-12-26 17:08:36,999][105620] Updated weights for policy 1, policy_version 244717 (0.0007) [2023-12-26 17:08:37,016][105692] Updated weights for policy 0, policy_version 244400 (0.0006) [2023-12-26 17:08:37,061][105620] Updated weights for policy 1, policy_version 244727 (0.0008) [2023-12-26 17:08:37,120][105620] Updated weights for policy 1, policy_version 244737 (0.0009) [2023-12-26 17:08:37,740][105692] Updated weights for policy 0, policy_version 244410 (0.0010) [2023-12-26 17:08:37,767][105620] Updated weights for policy 1, policy_version 244747 (0.0008) [2023-12-26 17:08:37,796][105692] Updated weights for policy 0, policy_version 244420 (0.0011) [2023-12-26 17:08:37,819][105620] Updated weights for policy 1, policy_version 244757 (0.0007) [2023-12-26 17:08:37,856][105692] Updated weights for policy 0, policy_version 244430 (0.0011) [2023-12-26 17:08:37,878][105620] Updated weights for policy 1, policy_version 244767 (0.0008) [2023-12-26 17:08:38,614][105620] Updated weights for policy 1, policy_version 244777 (0.0008) [2023-12-26 17:08:38,653][105692] Updated weights for policy 0, policy_version 244440 (0.0007) [2023-12-26 17:08:38,675][105620] Updated weights for policy 1, policy_version 244787 (0.0008) [2023-12-26 17:08:38,714][105692] Updated weights for policy 0, policy_version 244450 (0.0006) [2023-12-26 17:08:38,729][105620] Updated weights for policy 1, policy_version 244797 (0.0007) [2023-12-26 17:08:38,771][105692] Updated weights for policy 0, policy_version 244460 (0.0007) [2023-12-26 17:08:38,782][105620] Updated weights for policy 1, policy_version 244807 (0.0007) [2023-12-26 17:08:39,477][105692] Updated weights for policy 0, policy_version 244470 (0.0009) [2023-12-26 17:08:39,536][105692] Updated weights for policy 0, policy_version 244480 (0.0009) [2023-12-26 17:08:39,595][105692] Updated weights for policy 0, policy_version 244490 (0.0008) [2023-12-26 17:08:39,609][105620] Updated weights for policy 1, policy_version 244817 (0.0007) [2023-12-26 17:08:39,672][105620] Updated weights for policy 1, policy_version 244827 (0.0008) [2023-12-26 17:08:39,727][105620] Updated weights for policy 1, policy_version 244837 (0.0009) [2023-12-26 17:08:40,331][105692] Updated weights for policy 0, policy_version 244500 (0.0007) [2023-12-26 17:08:40,389][105585] KL-divergence is very high: 105.9437 [2023-12-26 17:08:40,394][105692] Updated weights for policy 0, policy_version 244510 (0.0006) [2023-12-26 17:08:40,437][105585] KL-divergence is very high: 126.2532 [2023-12-26 17:08:40,456][105692] Updated weights for policy 0, policy_version 244520 (0.0005) [2023-12-26 17:08:40,486][105585] KL-divergence is very high: 106.6463 [2023-12-26 17:08:40,532][105620] Updated weights for policy 1, policy_version 244847 (0.0009) [2023-12-26 17:08:40,594][105620] Updated weights for policy 1, policy_version 244857 (0.0010) [2023-12-26 17:08:40,659][105620] Updated weights for policy 1, policy_version 244867 (0.0010) [2023-12-26 17:08:41,002][105692] Updated weights for policy 0, policy_version 244530 (0.0006) [2023-12-26 17:08:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 125304832. Throughput: 0: 9875.8, 1: 9673.0. Samples: 125314608. Policy #0 lag: (min: 1.0, avg: 23.7, max: 33.0) [2023-12-26 17:08:41,062][104569] Avg episode reward: [(0, '2059.477'), (1, '7979.489')] [2023-12-26 17:08:41,082][105692] Updated weights for policy 0, policy_version 244540 (0.0008) [2023-12-26 17:08:41,145][105692] Updated weights for policy 0, policy_version 244550 (0.0011) [2023-12-26 17:08:41,202][105692] Updated weights for policy 0, policy_version 244560 (0.0010) [2023-12-26 17:08:41,492][105620] Updated weights for policy 1, policy_version 244877 (0.0008) [2023-12-26 17:08:41,553][105620] Updated weights for policy 1, policy_version 244887 (0.0006) [2023-12-26 17:08:41,613][105620] Updated weights for policy 1, policy_version 244897 (0.0009) [2023-12-26 17:08:41,902][105692] Updated weights for policy 0, policy_version 244570 (0.0006) [2023-12-26 17:08:41,958][105692] Updated weights for policy 0, policy_version 244580 (0.0006) [2023-12-26 17:08:42,015][105692] Updated weights for policy 0, policy_version 244590 (0.0006) [2023-12-26 17:08:42,398][105620] Updated weights for policy 1, policy_version 244907 (0.0009) [2023-12-26 17:08:42,452][105620] Updated weights for policy 1, policy_version 244917 (0.0009) [2023-12-26 17:08:42,514][105620] Updated weights for policy 1, policy_version 244927 (0.0010) [2023-12-26 17:08:42,682][105692] Updated weights for policy 0, policy_version 244600 (0.0008) [2023-12-26 17:08:42,745][105692] Updated weights for policy 0, policy_version 244610 (0.0009) [2023-12-26 17:08:42,800][105692] Updated weights for policy 0, policy_version 244620 (0.0009) [2023-12-26 17:08:43,309][105620] Updated weights for policy 1, policy_version 244937 (0.0009) [2023-12-26 17:08:43,355][105620] Updated weights for policy 1, policy_version 244947 (0.0009) [2023-12-26 17:08:43,404][105620] Updated weights for policy 1, policy_version 244957 (0.0007) [2023-12-26 17:08:43,466][105620] Updated weights for policy 1, policy_version 244967 (0.0005) [2023-12-26 17:08:43,576][105692] Updated weights for policy 0, policy_version 244631 (0.0009) [2023-12-26 17:08:43,630][105692] Updated weights for policy 0, policy_version 244642 (0.0010) [2023-12-26 17:08:43,684][105692] Updated weights for policy 0, policy_version 244653 (0.0010) [2023-12-26 17:08:44,043][105620] Updated weights for policy 1, policy_version 244977 (0.0007) [2023-12-26 17:08:44,097][105620] Updated weights for policy 1, policy_version 244987 (0.0006) [2023-12-26 17:08:44,149][105620] Updated weights for policy 1, policy_version 244997 (0.0005) [2023-12-26 17:08:44,405][105692] Updated weights for policy 0, policy_version 244663 (0.0010) [2023-12-26 17:08:44,456][105692] Updated weights for policy 0, policy_version 244673 (0.0010) [2023-12-26 17:08:44,517][105692] Updated weights for policy 0, policy_version 244683 (0.0010) [2023-12-26 17:08:44,756][105620] Updated weights for policy 1, policy_version 245007 (0.0008) [2023-12-26 17:08:44,816][105620] Updated weights for policy 1, policy_version 245017 (0.0008) [2023-12-26 17:08:44,880][105620] Updated weights for policy 1, policy_version 245027 (0.0008) [2023-12-26 17:08:45,177][105692] Updated weights for policy 0, policy_version 244693 (0.0008) [2023-12-26 17:08:45,225][105692] Updated weights for policy 0, policy_version 244703 (0.0006) [2023-12-26 17:08:45,282][105692] Updated weights for policy 0, policy_version 244713 (0.0006) [2023-12-26 17:08:45,478][105620] Updated weights for policy 1, policy_version 245037 (0.0008) [2023-12-26 17:08:45,543][105620] Updated weights for policy 1, policy_version 245047 (0.0011) [2023-12-26 17:08:45,563][105586] KL-divergence is very high: 130.3034 [2023-12-26 17:08:45,609][105620] Updated weights for policy 1, policy_version 245057 (0.0011) [2023-12-26 17:08:45,617][105586] KL-divergence is very high: 130.5734 [2023-12-26 17:08:45,890][105692] Updated weights for policy 0, policy_version 244723 (0.0006) [2023-12-26 17:08:45,953][105692] Updated weights for policy 0, policy_version 244733 (0.0010) [2023-12-26 17:08:46,003][105692] Updated weights for policy 0, policy_version 244743 (0.0010) [2023-12-26 17:08:46,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 125411328. Throughput: 0: 9906.5, 1: 9601.4. Samples: 125373528. Policy #0 lag: (min: 1.0, avg: 23.7, max: 33.0) [2023-12-26 17:08:46,062][104569] Avg episode reward: [(0, '6486.589'), (1, '7424.127')] [2023-12-26 17:08:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000244752_62668800.pth... [2023-12-26 17:08:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000245064_62742528.pth... [2023-12-26 17:08:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000243944_62455808.pth [2023-12-26 17:08:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000243568_62365696.pth [2023-12-26 17:08:46,261][105620] Updated weights for policy 1, policy_version 245067 (0.0009) [2023-12-26 17:08:46,318][105620] Updated weights for policy 1, policy_version 245077 (0.0005) [2023-12-26 17:08:46,376][105620] Updated weights for policy 1, policy_version 245087 (0.0009) [2023-12-26 17:08:46,672][105692] Updated weights for policy 0, policy_version 244753 (0.0010) [2023-12-26 17:08:46,702][105585] KL-divergence is very high: 156.6804 [2023-12-26 17:08:46,723][105692] Updated weights for policy 0, policy_version 244763 (0.0010) [2023-12-26 17:08:46,761][105585] KL-divergence is very high: 275.4337 [2023-12-26 17:08:46,788][105692] Updated weights for policy 0, policy_version 244773 (0.0009) [2023-12-26 17:08:46,803][105585] KL-divergence is very high: 309.7824 [2023-12-26 17:08:46,840][105692] Updated weights for policy 0, policy_version 244783 (0.0009) [2023-12-26 17:08:47,074][105620] Updated weights for policy 1, policy_version 245097 (0.0010) [2023-12-26 17:08:47,122][105620] Updated weights for policy 1, policy_version 245107 (0.0010) [2023-12-26 17:08:47,181][105620] Updated weights for policy 1, policy_version 245117 (0.0010) [2023-12-26 17:08:47,242][105620] Updated weights for policy 1, policy_version 245127 (0.0010) [2023-12-26 17:08:47,572][105692] Updated weights for policy 0, policy_version 244793 (0.0006) [2023-12-26 17:08:47,635][105692] Updated weights for policy 0, policy_version 244803 (0.0005) [2023-12-26 17:08:47,690][105692] Updated weights for policy 0, policy_version 244813 (0.0011) [2023-12-26 17:08:47,910][105620] Updated weights for policy 1, policy_version 245137 (0.0010) [2023-12-26 17:08:47,961][105620] Updated weights for policy 1, policy_version 245147 (0.0010) [2023-12-26 17:08:48,017][105620] Updated weights for policy 1, policy_version 245157 (0.0009) [2023-12-26 17:08:48,376][105692] Updated weights for policy 0, policy_version 244823 (0.0008) [2023-12-26 17:08:48,434][105692] Updated weights for policy 0, policy_version 244833 (0.0008) [2023-12-26 17:08:48,494][105692] Updated weights for policy 0, policy_version 244843 (0.0009) [2023-12-26 17:08:48,605][105620] Updated weights for policy 1, policy_version 245167 (0.0008) [2023-12-26 17:08:48,677][105620] Updated weights for policy 1, policy_version 245177 (0.0010) [2023-12-26 17:08:48,735][105620] Updated weights for policy 1, policy_version 245187 (0.0011) [2023-12-26 17:08:49,122][105692] Updated weights for policy 0, policy_version 244853 (0.0010) [2023-12-26 17:08:49,187][105692] Updated weights for policy 0, policy_version 244863 (0.0011) [2023-12-26 17:08:49,250][105692] Updated weights for policy 0, policy_version 244874 (0.0010) [2023-12-26 17:08:49,405][105620] Updated weights for policy 1, policy_version 245197 (0.0010) [2023-12-26 17:08:49,457][105620] Updated weights for policy 1, policy_version 245207 (0.0008) [2023-12-26 17:08:49,515][105620] Updated weights for policy 1, policy_version 245217 (0.0010) [2023-12-26 17:08:49,886][105692] Updated weights for policy 0, policy_version 244884 (0.0008) [2023-12-26 17:08:49,954][105692] Updated weights for policy 0, policy_version 244894 (0.0007) [2023-12-26 17:08:50,019][105692] Updated weights for policy 0, policy_version 244904 (0.0009) [2023-12-26 17:08:50,352][105620] Updated weights for policy 1, policy_version 245227 (0.0009) [2023-12-26 17:08:50,402][105620] Updated weights for policy 1, policy_version 245237 (0.0008) [2023-12-26 17:08:50,464][105620] Updated weights for policy 1, policy_version 245247 (0.0009) [2023-12-26 17:08:50,687][105692] Updated weights for policy 0, policy_version 244914 (0.0009) [2023-12-26 17:08:50,753][105692] Updated weights for policy 0, policy_version 244924 (0.0008) [2023-12-26 17:08:50,818][105692] Updated weights for policy 0, policy_version 244934 (0.0007) [2023-12-26 17:08:50,886][105692] Updated weights for policy 0, policy_version 244944 (0.0007) [2023-12-26 17:08:51,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 125509632. Throughput: 0: 9919.3, 1: 9697.9. Samples: 125497124. Policy #0 lag: (min: 1.0, avg: 23.7, max: 33.0) [2023-12-26 17:08:51,063][104569] Avg episode reward: [(0, '8098.507'), (1, '7333.462')] [2023-12-26 17:08:51,195][105620] Updated weights for policy 1, policy_version 245257 (0.0009) [2023-12-26 17:08:51,259][105620] Updated weights for policy 1, policy_version 245267 (0.0009) [2023-12-26 17:08:51,326][105620] Updated weights for policy 1, policy_version 245277 (0.0009) [2023-12-26 17:08:51,399][105620] Updated weights for policy 1, policy_version 245287 (0.0010) [2023-12-26 17:08:51,696][105692] Updated weights for policy 0, policy_version 244954 (0.0009) [2023-12-26 17:08:51,761][105692] Updated weights for policy 0, policy_version 244964 (0.0010) [2023-12-26 17:08:51,818][105692] Updated weights for policy 0, policy_version 244975 (0.0010) [2023-12-26 17:08:52,041][105620] Updated weights for policy 1, policy_version 245297 (0.0009) [2023-12-26 17:08:52,094][105620] Updated weights for policy 1, policy_version 245307 (0.0009) [2023-12-26 17:08:52,156][105620] Updated weights for policy 1, policy_version 245317 (0.0009) [2023-12-26 17:08:52,609][105692] Updated weights for policy 0, policy_version 244985 (0.0009) [2023-12-26 17:08:52,663][105692] Updated weights for policy 0, policy_version 244995 (0.0009) [2023-12-26 17:08:52,711][105692] Updated weights for policy 0, policy_version 245005 (0.0009) [2023-12-26 17:08:52,870][105620] Updated weights for policy 1, policy_version 245327 (0.0009) [2023-12-26 17:08:52,924][105620] Updated weights for policy 1, policy_version 245337 (0.0010) [2023-12-26 17:08:52,986][105620] Updated weights for policy 1, policy_version 245348 (0.0010) [2023-12-26 17:08:53,509][105692] Updated weights for policy 0, policy_version 245015 (0.0007) [2023-12-26 17:08:53,562][105692] Updated weights for policy 0, policy_version 245025 (0.0005) [2023-12-26 17:08:53,615][105692] Updated weights for policy 0, policy_version 245035 (0.0005) [2023-12-26 17:08:53,743][105620] Updated weights for policy 1, policy_version 245358 (0.0009) [2023-12-26 17:08:53,804][105620] Updated weights for policy 1, policy_version 245368 (0.0010) [2023-12-26 17:08:53,869][105620] Updated weights for policy 1, policy_version 245378 (0.0009) [2023-12-26 17:08:54,150][105692] Updated weights for policy 0, policy_version 245045 (0.0007) [2023-12-26 17:08:54,208][105692] Updated weights for policy 0, policy_version 245055 (0.0009) [2023-12-26 17:08:54,270][105692] Updated weights for policy 0, policy_version 245065 (0.0010) [2023-12-26 17:08:54,606][105620] Updated weights for policy 1, policy_version 245388 (0.0010) [2023-12-26 17:08:54,653][105620] Updated weights for policy 1, policy_version 245398 (0.0008) [2023-12-26 17:08:54,704][105620] Updated weights for policy 1, policy_version 245409 (0.0010) [2023-12-26 17:08:54,948][105692] Updated weights for policy 0, policy_version 245075 (0.0009) [2023-12-26 17:08:54,999][105692] Updated weights for policy 0, policy_version 245085 (0.0005) [2023-12-26 17:08:55,048][105692] Updated weights for policy 0, policy_version 245095 (0.0005) [2023-12-26 17:08:55,413][105620] Updated weights for policy 1, policy_version 245420 (0.0010) [2023-12-26 17:08:55,464][105620] Updated weights for policy 1, policy_version 245430 (0.0010) [2023-12-26 17:08:55,509][105620] Updated weights for policy 1, policy_version 245440 (0.0010) [2023-12-26 17:08:55,612][105692] Updated weights for policy 0, policy_version 245105 (0.0008) [2023-12-26 17:08:55,675][105692] Updated weights for policy 0, policy_version 245115 (0.0005) [2023-12-26 17:08:55,735][105692] Updated weights for policy 0, policy_version 245125 (0.0005) [2023-12-26 17:08:55,788][105692] Updated weights for policy 0, policy_version 245135 (0.0005) [2023-12-26 17:08:56,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 125607936. Throughput: 0: 10003.0, 1: 9690.2. Samples: 125616112. Policy #0 lag: (min: 1.0, avg: 23.7, max: 33.0) [2023-12-26 17:08:56,063][104569] Avg episode reward: [(0, '9095.234'), (1, '7891.453')] [2023-12-26 17:08:56,216][105620] Updated weights for policy 1, policy_version 245450 (0.0009) [2023-12-26 17:08:56,270][105620] Updated weights for policy 1, policy_version 245460 (0.0009) [2023-12-26 17:08:56,332][105620] Updated weights for policy 1, policy_version 245470 (0.0011) [2023-12-26 17:08:56,390][105620] Updated weights for policy 1, policy_version 245480 (0.0006) [2023-12-26 17:08:56,455][105692] Updated weights for policy 0, policy_version 245145 (0.0008) [2023-12-26 17:08:56,502][105692] Updated weights for policy 0, policy_version 245155 (0.0008) [2023-12-26 17:08:56,561][105692] Updated weights for policy 0, policy_version 245165 (0.0007) [2023-12-26 17:08:57,098][105620] Updated weights for policy 1, policy_version 245490 (0.0010) [2023-12-26 17:08:57,156][105620] Updated weights for policy 1, policy_version 245500 (0.0010) [2023-12-26 17:08:57,203][105620] Updated weights for policy 1, policy_version 245510 (0.0010) [2023-12-26 17:08:57,293][105692] Updated weights for policy 0, policy_version 245175 (0.0008) [2023-12-26 17:08:57,356][105692] Updated weights for policy 0, policy_version 245185 (0.0007) [2023-12-26 17:08:57,410][105692] Updated weights for policy 0, policy_version 245195 (0.0008) [2023-12-26 17:08:57,957][105620] Updated weights for policy 1, policy_version 245520 (0.0009) [2023-12-26 17:08:57,998][105586] KL-divergence is very high: 150.8150 [2023-12-26 17:08:58,004][105586] KL-divergence is very high: 137.5354 [2023-12-26 17:08:58,017][105620] Updated weights for policy 1, policy_version 245530 (0.0007) [2023-12-26 17:08:58,024][105586] KL-divergence is very high: 143.6859 [2023-12-26 17:08:58,046][105586] KL-divergence is very high: 159.3857 [2023-12-26 17:08:58,051][105586] KL-divergence is very high: 143.5759 [2023-12-26 17:08:58,073][105620] Updated weights for policy 1, policy_version 245540 (0.0010) [2023-12-26 17:08:58,095][105586] KL-divergence is very high: 123.8765 [2023-12-26 17:08:58,171][105692] Updated weights for policy 0, policy_version 245205 (0.0008) [2023-12-26 17:08:58,231][105692] Updated weights for policy 0, policy_version 245215 (0.0008) [2023-12-26 17:08:58,292][105692] Updated weights for policy 0, policy_version 245225 (0.0009) [2023-12-26 17:08:58,850][105620] Updated weights for policy 1, policy_version 245550 (0.0011) [2023-12-26 17:08:58,909][105620] Updated weights for policy 1, policy_version 245560 (0.0011) [2023-12-26 17:08:58,968][105620] Updated weights for policy 1, policy_version 245570 (0.0010) [2023-12-26 17:08:59,091][105692] Updated weights for policy 0, policy_version 245235 (0.0009) [2023-12-26 17:08:59,138][105692] Updated weights for policy 0, policy_version 245245 (0.0005) [2023-12-26 17:08:59,194][105692] Updated weights for policy 0, policy_version 245255 (0.0005) [2023-12-26 17:08:59,709][105620] Updated weights for policy 1, policy_version 245580 (0.0009) [2023-12-26 17:08:59,757][105620] Updated weights for policy 1, policy_version 245590 (0.0007) [2023-12-26 17:08:59,810][105620] Updated weights for policy 1, policy_version 245600 (0.0008) [2023-12-26 17:08:59,973][105692] Updated weights for policy 0, policy_version 245265 (0.0007) [2023-12-26 17:09:00,027][105692] Updated weights for policy 0, policy_version 245275 (0.0009) [2023-12-26 17:09:00,078][105692] Updated weights for policy 0, policy_version 245285 (0.0009) [2023-12-26 17:09:00,140][105692] Updated weights for policy 0, policy_version 245295 (0.0009) [2023-12-26 17:09:00,529][105620] Updated weights for policy 1, policy_version 245610 (0.0008) [2023-12-26 17:09:00,581][105620] Updated weights for policy 1, policy_version 245620 (0.0005) [2023-12-26 17:09:00,634][105620] Updated weights for policy 1, policy_version 245630 (0.0005) [2023-12-26 17:09:00,679][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000003 [2023-12-26 17:09:00,680][105620] Updated weights for policy 1, policy_version 245640 (0.0005) [2023-12-26 17:09:00,982][105692] Updated weights for policy 0, policy_version 245305 (0.0009) [2023-12-26 17:09:01,046][105692] Updated weights for policy 0, policy_version 245315 (0.0009) [2023-12-26 17:09:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 125698048. Throughput: 0: 9953.7, 1: 9661.1. Samples: 125673076. Policy #0 lag: (min: 1.0, avg: 23.7, max: 33.0) [2023-12-26 17:09:01,062][104569] Avg episode reward: [(0, '9179.396'), (1, '7792.624')] [2023-12-26 17:09:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000245640_62889984.pth... [2023-12-26 17:09:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000244520_62603264.pth [2023-12-26 17:09:01,112][105692] Updated weights for policy 0, policy_version 245325 (0.0009) [2023-12-26 17:09:01,129][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000245328_62816256.pth... [2023-12-26 17:09:01,135][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000244176_62521344.pth [2023-12-26 17:09:01,313][105620] Updated weights for policy 1, policy_version 245650 (0.0008) [2023-12-26 17:09:01,387][105620] Updated weights for policy 1, policy_version 245660 (0.0009) [2023-12-26 17:09:01,438][105620] Updated weights for policy 1, policy_version 245670 (0.0008) [2023-12-26 17:09:01,886][105692] Updated weights for policy 0, policy_version 245335 (0.0008) [2023-12-26 17:09:01,950][105692] Updated weights for policy 0, policy_version 245345 (0.0010) [2023-12-26 17:09:02,020][105692] Updated weights for policy 0, policy_version 245355 (0.0010) [2023-12-26 17:09:02,176][105620] Updated weights for policy 1, policy_version 245680 (0.0008) [2023-12-26 17:09:02,229][105620] Updated weights for policy 1, policy_version 245690 (0.0009) [2023-12-26 17:09:02,284][105620] Updated weights for policy 1, policy_version 245700 (0.0009) [2023-12-26 17:09:02,780][105692] Updated weights for policy 0, policy_version 245365 (0.0008) [2023-12-26 17:09:02,827][105692] Updated weights for policy 0, policy_version 245375 (0.0005) [2023-12-26 17:09:02,875][105692] Updated weights for policy 0, policy_version 245385 (0.0005) [2023-12-26 17:09:03,054][105620] Updated weights for policy 1, policy_version 245710 (0.0010) [2023-12-26 17:09:03,106][105620] Updated weights for policy 1, policy_version 245720 (0.0006) [2023-12-26 17:09:03,173][105620] Updated weights for policy 1, policy_version 245730 (0.0005) [2023-12-26 17:09:03,394][105692] Updated weights for policy 0, policy_version 245395 (0.0005) [2023-12-26 17:09:03,467][105692] Updated weights for policy 0, policy_version 245405 (0.0006) [2023-12-26 17:09:03,528][105692] Updated weights for policy 0, policy_version 245415 (0.0009) [2023-12-26 17:09:03,807][105620] Updated weights for policy 1, policy_version 245740 (0.0006) [2023-12-26 17:09:03,859][105620] Updated weights for policy 1, policy_version 245750 (0.0008) [2023-12-26 17:09:03,921][105620] Updated weights for policy 1, policy_version 245760 (0.0008) [2023-12-26 17:09:04,257][105692] Updated weights for policy 0, policy_version 245425 (0.0009) [2023-12-26 17:09:04,310][105692] Updated weights for policy 0, policy_version 245435 (0.0010) [2023-12-26 17:09:04,365][105692] Updated weights for policy 0, policy_version 245445 (0.0009) [2023-12-26 17:09:04,418][105692] Updated weights for policy 0, policy_version 245455 (0.0009) [2023-12-26 17:09:04,620][105620] Updated weights for policy 1, policy_version 245770 (0.0009) [2023-12-26 17:09:04,680][105620] Updated weights for policy 1, policy_version 245780 (0.0008) [2023-12-26 17:09:04,735][105620] Updated weights for policy 1, policy_version 245790 (0.0008) [2023-12-26 17:09:04,786][105620] Updated weights for policy 1, policy_version 245800 (0.0008) [2023-12-26 17:09:05,209][105692] Updated weights for policy 0, policy_version 245465 (0.0010) [2023-12-26 17:09:05,269][105692] Updated weights for policy 0, policy_version 245475 (0.0011) [2023-12-26 17:09:05,327][105692] Updated weights for policy 0, policy_version 245485 (0.0010) [2023-12-26 17:09:05,561][105620] Updated weights for policy 1, policy_version 245810 (0.0008) [2023-12-26 17:09:05,624][105620] Updated weights for policy 1, policy_version 245820 (0.0008) [2023-12-26 17:09:05,675][105620] Updated weights for policy 1, policy_version 245830 (0.0008) [2023-12-26 17:09:06,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 125796352. Throughput: 0: 9990.8, 1: 9672.3. Samples: 125789456. Policy #0 lag: (min: 1.0, avg: 23.7, max: 33.0) [2023-12-26 17:09:06,063][104569] Avg episode reward: [(0, '9084.940'), (1, '7777.760')] [2023-12-26 17:09:06,072][105692] Updated weights for policy 0, policy_version 245495 (0.0010) [2023-12-26 17:09:06,134][105692] Updated weights for policy 0, policy_version 245505 (0.0010) [2023-12-26 17:09:06,200][105692] Updated weights for policy 0, policy_version 245515 (0.0011) [2023-12-26 17:09:06,400][105586] KL-divergence is very high: 126.8320 [2023-12-26 17:09:06,440][105620] Updated weights for policy 1, policy_version 245840 (0.0010) [2023-12-26 17:09:06,454][105586] KL-divergence is very high: 265.2985 [2023-12-26 17:09:06,509][105586] KL-divergence is very high: 299.9550 [2023-12-26 17:09:06,510][105620] Updated weights for policy 1, policy_version 245850 (0.0011) [2023-12-26 17:09:06,563][105586] KL-divergence is very high: 276.8515 [2023-12-26 17:09:06,577][105620] Updated weights for policy 1, policy_version 245860 (0.0011) [2023-12-26 17:09:06,843][105692] Updated weights for policy 0, policy_version 245525 (0.0008) [2023-12-26 17:09:06,899][105692] Updated weights for policy 0, policy_version 245535 (0.0005) [2023-12-26 17:09:06,961][105692] Updated weights for policy 0, policy_version 245545 (0.0011) [2023-12-26 17:09:07,238][105620] Updated weights for policy 1, policy_version 245870 (0.0010) [2023-12-26 17:09:07,289][105620] Updated weights for policy 1, policy_version 245880 (0.0006) [2023-12-26 17:09:07,339][105620] Updated weights for policy 1, policy_version 245890 (0.0010) [2023-12-26 17:09:07,674][105692] Updated weights for policy 0, policy_version 245555 (0.0009) [2023-12-26 17:09:07,734][105692] Updated weights for policy 0, policy_version 245565 (0.0008) [2023-12-26 17:09:07,789][105692] Updated weights for policy 0, policy_version 245575 (0.0006) [2023-12-26 17:09:07,957][105620] Updated weights for policy 1, policy_version 245900 (0.0009) [2023-12-26 17:09:08,008][105620] Updated weights for policy 1, policy_version 245910 (0.0010) [2023-12-26 17:09:08,063][105620] Updated weights for policy 1, policy_version 245920 (0.0010) [2023-12-26 17:09:08,331][105692] Updated weights for policy 0, policy_version 245585 (0.0006) [2023-12-26 17:09:08,395][105692] Updated weights for policy 0, policy_version 245595 (0.0011) [2023-12-26 17:09:08,462][105692] Updated weights for policy 0, policy_version 245605 (0.0011) [2023-12-26 17:09:08,514][105692] Updated weights for policy 0, policy_version 245615 (0.0010) [2023-12-26 17:09:08,725][105620] Updated weights for policy 1, policy_version 245930 (0.0009) [2023-12-26 17:09:08,787][105620] Updated weights for policy 1, policy_version 245940 (0.0009) [2023-12-26 17:09:08,835][105620] Updated weights for policy 1, policy_version 245950 (0.0010) [2023-12-26 17:09:08,894][105620] Updated weights for policy 1, policy_version 245960 (0.0007) [2023-12-26 17:09:09,248][105692] Updated weights for policy 0, policy_version 245625 (0.0010) [2023-12-26 17:09:09,308][105692] Updated weights for policy 0, policy_version 245635 (0.0010) [2023-12-26 17:09:09,379][105692] Updated weights for policy 0, policy_version 245645 (0.0010) [2023-12-26 17:09:09,570][105620] Updated weights for policy 1, policy_version 245970 (0.0011) [2023-12-26 17:09:09,629][105620] Updated weights for policy 1, policy_version 245980 (0.0011) [2023-12-26 17:09:09,690][105620] Updated weights for policy 1, policy_version 245990 (0.0011) [2023-12-26 17:09:10,004][105692] Updated weights for policy 0, policy_version 245655 (0.0011) [2023-12-26 17:09:10,057][105692] Updated weights for policy 0, policy_version 245665 (0.0011) [2023-12-26 17:09:10,112][105692] Updated weights for policy 0, policy_version 245675 (0.0009) [2023-12-26 17:09:10,456][105620] Updated weights for policy 1, policy_version 246000 (0.0010) [2023-12-26 17:09:10,499][105586] KL-divergence is very high: 101.7227 [2023-12-26 17:09:10,519][105620] Updated weights for policy 1, policy_version 246010 (0.0010) [2023-12-26 17:09:10,527][105586] KL-divergence is very high: 151.6348 [2023-12-26 17:09:10,533][105586] KL-divergence is very high: 258.4911 [2023-12-26 17:09:10,555][105586] KL-divergence is very high: 327.7669 [2023-12-26 17:09:10,584][105586] KL-divergence is very high: 219.8613 [2023-12-26 17:09:10,592][105586] KL-divergence is very high: 347.9490 [2023-12-26 17:09:10,593][105620] Updated weights for policy 1, policy_version 246020 (0.0006) [2023-12-26 17:09:10,612][105586] KL-divergence is very high: 343.2464 [2023-12-26 17:09:10,874][105692] Updated weights for policy 0, policy_version 245685 (0.0008) [2023-12-26 17:09:10,927][105692] Updated weights for policy 0, policy_version 245695 (0.0008) [2023-12-26 17:09:10,986][105692] Updated weights for policy 0, policy_version 245705 (0.0008) [2023-12-26 17:09:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 125902848. Throughput: 0: 9960.8, 1: 9754.7. Samples: 125908612. Policy #0 lag: (min: 1.0, avg: 23.7, max: 33.0) [2023-12-26 17:09:11,062][104569] Avg episode reward: [(0, '9174.452'), (1, '8149.791')] [2023-12-26 17:09:11,352][105620] Updated weights for policy 1, policy_version 246030 (0.0010) [2023-12-26 17:09:11,380][105586] KL-divergence is very high: 107.1289 [2023-12-26 17:09:11,409][105620] Updated weights for policy 1, policy_version 246040 (0.0009) [2023-12-26 17:09:11,428][105586] KL-divergence is very high: 182.5279 [2023-12-26 17:09:11,471][105620] Updated weights for policy 1, policy_version 246050 (0.0008) [2023-12-26 17:09:11,476][105586] KL-divergence is very high: 189.3001 [2023-12-26 17:09:11,738][105692] Updated weights for policy 0, policy_version 245715 (0.0008) [2023-12-26 17:09:11,808][105692] Updated weights for policy 0, policy_version 245725 (0.0007) [2023-12-26 17:09:11,876][105692] Updated weights for policy 0, policy_version 245735 (0.0009) [2023-12-26 17:09:12,271][105620] Updated weights for policy 1, policy_version 246060 (0.0008) [2023-12-26 17:09:12,343][105620] Updated weights for policy 1, policy_version 246070 (0.0009) [2023-12-26 17:09:12,412][105620] Updated weights for policy 1, policy_version 246080 (0.0010) [2023-12-26 17:09:12,663][105692] Updated weights for policy 0, policy_version 245745 (0.0008) [2023-12-26 17:09:12,719][105692] Updated weights for policy 0, policy_version 245755 (0.0008) [2023-12-26 17:09:12,783][105692] Updated weights for policy 0, policy_version 245765 (0.0008) [2023-12-26 17:09:12,845][105692] Updated weights for policy 0, policy_version 245775 (0.0008) [2023-12-26 17:09:13,052][105620] Updated weights for policy 1, policy_version 246090 (0.0007) [2023-12-26 17:09:13,110][105620] Updated weights for policy 1, policy_version 246100 (0.0011) [2023-12-26 17:09:13,176][105620] Updated weights for policy 1, policy_version 246110 (0.0010) [2023-12-26 17:09:13,235][105620] Updated weights for policy 1, policy_version 246120 (0.0010) [2023-12-26 17:09:13,636][105692] Updated weights for policy 0, policy_version 245785 (0.0010) [2023-12-26 17:09:13,690][105692] Updated weights for policy 0, policy_version 245797 (0.0010) [2023-12-26 17:09:13,737][105692] Updated weights for policy 0, policy_version 245807 (0.0007) [2023-12-26 17:09:13,875][105620] Updated weights for policy 1, policy_version 246130 (0.0005) [2023-12-26 17:09:13,947][105620] Updated weights for policy 1, policy_version 246140 (0.0005) [2023-12-26 17:09:14,018][105620] Updated weights for policy 1, policy_version 246150 (0.0005) [2023-12-26 17:09:14,426][105692] Updated weights for policy 0, policy_version 245817 (0.0009) [2023-12-26 17:09:14,488][105692] Updated weights for policy 0, policy_version 245827 (0.0011) [2023-12-26 17:09:14,491][105620] Updated weights for policy 1, policy_version 246160 (0.0006) [2023-12-26 17:09:14,555][105620] Updated weights for policy 1, policy_version 246170 (0.0007) [2023-12-26 17:09:14,555][105692] Updated weights for policy 0, policy_version 245837 (0.0010) [2023-12-26 17:09:14,614][105620] Updated weights for policy 1, policy_version 246180 (0.0006) [2023-12-26 17:09:15,253][105692] Updated weights for policy 0, policy_version 245847 (0.0009) [2023-12-26 17:09:15,267][105620] Updated weights for policy 1, policy_version 246190 (0.0009) [2023-12-26 17:09:15,313][105692] Updated weights for policy 0, policy_version 245857 (0.0011) [2023-12-26 17:09:15,327][105620] Updated weights for policy 1, policy_version 246200 (0.0011) [2023-12-26 17:09:15,372][105692] Updated weights for policy 0, policy_version 245867 (0.0011) [2023-12-26 17:09:15,390][105620] Updated weights for policy 1, policy_version 246210 (0.0011) [2023-12-26 17:09:16,040][105692] Updated weights for policy 0, policy_version 245877 (0.0010) [2023-12-26 17:09:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 125992960. Throughput: 0: 9781.6, 1: 9814.2. Samples: 125965376. Policy #0 lag: (min: 1.0, avg: 23.7, max: 33.0) [2023-12-26 17:09:16,062][104569] Avg episode reward: [(0, '9265.269'), (1, '7608.324')] [2023-12-26 17:09:16,099][105692] Updated weights for policy 0, policy_version 245887 (0.0010) [2023-12-26 17:09:16,102][105620] Updated weights for policy 1, policy_version 246220 (0.0011) [2023-12-26 17:09:16,157][105620] Updated weights for policy 1, policy_version 246230 (0.0010) [2023-12-26 17:09:16,157][105692] Updated weights for policy 0, policy_version 245897 (0.0010) [2023-12-26 17:09:16,195][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000245904_62963712.pth... [2023-12-26 17:09:16,199][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000244752_62668800.pth [2023-12-26 17:09:16,214][105620] Updated weights for policy 1, policy_version 246240 (0.0010) [2023-12-26 17:09:16,259][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000246248_63045632.pth... [2023-12-26 17:09:16,264][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000245064_62742528.pth [2023-12-26 17:09:16,828][105692] Updated weights for policy 0, policy_version 245907 (0.0009) [2023-12-26 17:09:16,897][105692] Updated weights for policy 0, policy_version 245917 (0.0007) [2023-12-26 17:09:16,944][105620] Updated weights for policy 1, policy_version 246250 (0.0010) [2023-12-26 17:09:16,955][105692] Updated weights for policy 0, policy_version 245927 (0.0008) [2023-12-26 17:09:16,992][105620] Updated weights for policy 1, policy_version 246260 (0.0010) [2023-12-26 17:09:17,040][105620] Updated weights for policy 1, policy_version 246270 (0.0010) [2023-12-26 17:09:17,095][105620] Updated weights for policy 1, policy_version 246280 (0.0010) [2023-12-26 17:09:17,521][105692] Updated weights for policy 0, policy_version 245937 (0.0005) [2023-12-26 17:09:17,587][105692] Updated weights for policy 0, policy_version 245947 (0.0006) [2023-12-26 17:09:17,632][105585] KL-divergence is very high: 108.4590 [2023-12-26 17:09:17,644][105692] Updated weights for policy 0, policy_version 245957 (0.0011) [2023-12-26 17:09:17,675][105585] KL-divergence is very high: 178.0127 [2023-12-26 17:09:17,696][105692] Updated weights for policy 0, policy_version 245967 (0.0010) [2023-12-26 17:09:17,705][105620] Updated weights for policy 1, policy_version 246290 (0.0005) [2023-12-26 17:09:17,758][105620] Updated weights for policy 1, policy_version 246300 (0.0007) [2023-12-26 17:09:17,810][105620] Updated weights for policy 1, policy_version 246310 (0.0010) [2023-12-26 17:09:18,412][105692] Updated weights for policy 0, policy_version 245977 (0.0008) [2023-12-26 17:09:18,479][105692] Updated weights for policy 0, policy_version 245987 (0.0007) [2023-12-26 17:09:18,528][105620] Updated weights for policy 1, policy_version 246320 (0.0010) [2023-12-26 17:09:18,541][105692] Updated weights for policy 0, policy_version 245997 (0.0008) [2023-12-26 17:09:18,589][105620] Updated weights for policy 1, policy_version 246330 (0.0010) [2023-12-26 17:09:18,658][105620] Updated weights for policy 1, policy_version 246340 (0.0010) [2023-12-26 17:09:19,172][105692] Updated weights for policy 0, policy_version 246007 (0.0007) [2023-12-26 17:09:19,236][105692] Updated weights for policy 0, policy_version 246017 (0.0008) [2023-12-26 17:09:19,295][105692] Updated weights for policy 0, policy_version 246027 (0.0008) [2023-12-26 17:09:19,403][105620] Updated weights for policy 1, policy_version 246350 (0.0011) [2023-12-26 17:09:19,466][105620] Updated weights for policy 1, policy_version 246360 (0.0011) [2023-12-26 17:09:19,529][105620] Updated weights for policy 1, policy_version 246370 (0.0011) [2023-12-26 17:09:20,001][105692] Updated weights for policy 0, policy_version 246037 (0.0009) [2023-12-26 17:09:20,068][105692] Updated weights for policy 0, policy_version 246047 (0.0009) [2023-12-26 17:09:20,129][105692] Updated weights for policy 0, policy_version 246057 (0.0009) [2023-12-26 17:09:20,277][105620] Updated weights for policy 1, policy_version 246380 (0.0010) [2023-12-26 17:09:20,336][105620] Updated weights for policy 1, policy_version 246390 (0.0006) [2023-12-26 17:09:20,395][105620] Updated weights for policy 1, policy_version 246400 (0.0010) [2023-12-26 17:09:20,935][105692] Updated weights for policy 0, policy_version 246067 (0.0009) [2023-12-26 17:09:20,990][105692] Updated weights for policy 0, policy_version 246077 (0.0009) [2023-12-26 17:09:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 126091264. Throughput: 0: 9847.7, 1: 9810.0. Samples: 126087280. Policy #0 lag: (min: 1.0, avg: 23.7, max: 33.0) [2023-12-26 17:09:21,062][104569] Avg episode reward: [(0, '9175.572'), (1, '7345.121')] [2023-12-26 17:09:21,066][105692] Updated weights for policy 0, policy_version 246087 (0.0008) [2023-12-26 17:09:21,139][105620] Updated weights for policy 1, policy_version 246410 (0.0008) [2023-12-26 17:09:21,202][105620] Updated weights for policy 1, policy_version 246420 (0.0009) [2023-12-26 17:09:21,263][105620] Updated weights for policy 1, policy_version 246430 (0.0009) [2023-12-26 17:09:21,326][105620] Updated weights for policy 1, policy_version 246440 (0.0009) [2023-12-26 17:09:21,841][105692] Updated weights for policy 0, policy_version 246097 (0.0007) [2023-12-26 17:09:21,909][105692] Updated weights for policy 0, policy_version 246107 (0.0009) [2023-12-26 17:09:21,974][105692] Updated weights for policy 0, policy_version 246117 (0.0009) [2023-12-26 17:09:22,040][105692] Updated weights for policy 0, policy_version 246127 (0.0011) [2023-12-26 17:09:22,116][105620] Updated weights for policy 1, policy_version 246450 (0.0009) [2023-12-26 17:09:22,165][105620] Updated weights for policy 1, policy_version 246460 (0.0008) [2023-12-26 17:09:22,227][105620] Updated weights for policy 1, policy_version 246470 (0.0009) [2023-12-26 17:09:22,818][105692] Updated weights for policy 0, policy_version 246137 (0.0010) [2023-12-26 17:09:22,878][105692] Updated weights for policy 0, policy_version 246147 (0.0011) [2023-12-26 17:09:22,938][105692] Updated weights for policy 0, policy_version 246157 (0.0010) [2023-12-26 17:09:23,021][105620] Updated weights for policy 1, policy_version 246480 (0.0008) [2023-12-26 17:09:23,071][105620] Updated weights for policy 1, policy_version 246490 (0.0008) [2023-12-26 17:09:23,120][105620] Updated weights for policy 1, policy_version 246500 (0.0008) [2023-12-26 17:09:23,624][105692] Updated weights for policy 0, policy_version 246167 (0.0007) [2023-12-26 17:09:23,690][105692] Updated weights for policy 0, policy_version 246177 (0.0005) [2023-12-26 17:09:23,748][105692] Updated weights for policy 0, policy_version 246187 (0.0007) [2023-12-26 17:09:23,817][105620] Updated weights for policy 1, policy_version 246510 (0.0007) [2023-12-26 17:09:23,863][105620] Updated weights for policy 1, policy_version 246520 (0.0005) [2023-12-26 17:09:23,909][105620] Updated weights for policy 1, policy_version 246530 (0.0005) [2023-12-26 17:09:24,426][105692] Updated weights for policy 0, policy_version 246197 (0.0010) [2023-12-26 17:09:24,463][105620] Updated weights for policy 1, policy_version 246540 (0.0005) [2023-12-26 17:09:24,492][105692] Updated weights for policy 0, policy_version 246207 (0.0010) [2023-12-26 17:09:24,518][105620] Updated weights for policy 1, policy_version 246550 (0.0008) [2023-12-26 17:09:24,574][105692] Updated weights for policy 0, policy_version 246217 (0.0010) [2023-12-26 17:09:24,578][105620] Updated weights for policy 1, policy_version 246560 (0.0010) [2023-12-26 17:09:25,207][105620] Updated weights for policy 1, policy_version 246570 (0.0011) [2023-12-26 17:09:25,225][105692] Updated weights for policy 0, policy_version 246227 (0.0010) [2023-12-26 17:09:25,264][105620] Updated weights for policy 1, policy_version 246580 (0.0006) [2023-12-26 17:09:25,278][105692] Updated weights for policy 0, policy_version 246237 (0.0007) [2023-12-26 17:09:25,321][105620] Updated weights for policy 1, policy_version 246590 (0.0006) [2023-12-26 17:09:25,341][105692] Updated weights for policy 0, policy_version 246247 (0.0008) [2023-12-26 17:09:25,374][105620] Updated weights for policy 1, policy_version 246600 (0.0008) [2023-12-26 17:09:26,041][105692] Updated weights for policy 0, policy_version 246257 (0.0009) [2023-12-26 17:09:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 126189568. Throughput: 0: 9827.9, 1: 9925.8. Samples: 126203524. Policy #0 lag: (min: 1.0, avg: 23.7, max: 33.0) [2023-12-26 17:09:26,063][104569] Avg episode reward: [(0, '9357.265'), (1, '7912.557')] [2023-12-26 17:09:26,075][105620] Updated weights for policy 1, policy_version 246610 (0.0010) [2023-12-26 17:09:26,097][105692] Updated weights for policy 0, policy_version 246267 (0.0006) [2023-12-26 17:09:26,130][105620] Updated weights for policy 1, policy_version 246620 (0.0009) [2023-12-26 17:09:26,145][105692] Updated weights for policy 0, policy_version 246277 (0.0006) [2023-12-26 17:09:26,174][105620] Updated weights for policy 1, policy_version 246630 (0.0010) [2023-12-26 17:09:26,193][105692] Updated weights for policy 0, policy_version 246287 (0.0006) [2023-12-26 17:09:26,870][105692] Updated weights for policy 0, policy_version 246297 (0.0006) [2023-12-26 17:09:26,872][105620] Updated weights for policy 1, policy_version 246640 (0.0010) [2023-12-26 17:09:26,920][105692] Updated weights for policy 0, policy_version 246307 (0.0006) [2023-12-26 17:09:26,926][105620] Updated weights for policy 1, policy_version 246650 (0.0010) [2023-12-26 17:09:26,969][105692] Updated weights for policy 0, policy_version 246317 (0.0005) [2023-12-26 17:09:26,980][105620] Updated weights for policy 1, policy_version 246660 (0.0009) [2023-12-26 17:09:27,488][105692] Updated weights for policy 0, policy_version 246327 (0.0005) [2023-12-26 17:09:27,535][105692] Updated weights for policy 0, policy_version 246337 (0.0005) [2023-12-26 17:09:27,577][105620] Updated weights for policy 1, policy_version 246670 (0.0005) [2023-12-26 17:09:27,596][105692] Updated weights for policy 0, policy_version 246347 (0.0005) [2023-12-26 17:09:27,629][105620] Updated weights for policy 1, policy_version 246680 (0.0005) [2023-12-26 17:09:27,634][105586] KL-divergence is very high: 133.8938 [2023-12-26 17:09:27,668][105586] KL-divergence is very high: 132.0074 [2023-12-26 17:09:27,675][105620] Updated weights for policy 1, policy_version 246690 (0.0010) [2023-12-26 17:09:28,107][105692] Updated weights for policy 0, policy_version 246357 (0.0008) [2023-12-26 17:09:28,169][105692] Updated weights for policy 0, policy_version 246367 (0.0010) [2023-12-26 17:09:28,236][105692] Updated weights for policy 0, policy_version 246377 (0.0010) [2023-12-26 17:09:28,398][105620] Updated weights for policy 1, policy_version 246700 (0.0010) [2023-12-26 17:09:28,457][105620] Updated weights for policy 1, policy_version 246710 (0.0008) [2023-12-26 17:09:28,519][105620] Updated weights for policy 1, policy_version 246720 (0.0008) [2023-12-26 17:09:29,003][105692] Updated weights for policy 0, policy_version 246387 (0.0009) [2023-12-26 17:09:29,049][105692] Updated weights for policy 0, policy_version 246397 (0.0005) [2023-12-26 17:09:29,096][105692] Updated weights for policy 0, policy_version 246407 (0.0009) [2023-12-26 17:09:29,297][105620] Updated weights for policy 1, policy_version 246730 (0.0008) [2023-12-26 17:09:29,360][105620] Updated weights for policy 1, policy_version 246740 (0.0008) [2023-12-26 17:09:29,382][105586] KL-divergence is very high: 122.0138 [2023-12-26 17:09:29,416][105620] Updated weights for policy 1, policy_version 246750 (0.0009) [2023-12-26 17:09:29,425][105586] KL-divergence is very high: 125.0252 [2023-12-26 17:09:29,473][105620] Updated weights for policy 1, policy_version 246760 (0.0009) [2023-12-26 17:09:29,767][105692] Updated weights for policy 0, policy_version 246417 (0.0008) [2023-12-26 17:09:29,824][105692] Updated weights for policy 0, policy_version 246427 (0.0008) [2023-12-26 17:09:29,880][105692] Updated weights for policy 0, policy_version 246437 (0.0009) [2023-12-26 17:09:29,926][105692] Updated weights for policy 0, policy_version 246447 (0.0008) [2023-12-26 17:09:30,226][105620] Updated weights for policy 1, policy_version 246770 (0.0009) [2023-12-26 17:09:30,284][105620] Updated weights for policy 1, policy_version 246780 (0.0009) [2023-12-26 17:09:30,335][105620] Updated weights for policy 1, policy_version 246790 (0.0009) [2023-12-26 17:09:30,671][105692] Updated weights for policy 0, policy_version 246457 (0.0009) [2023-12-26 17:09:30,721][105692] Updated weights for policy 0, policy_version 246467 (0.0008) [2023-12-26 17:09:30,767][105692] Updated weights for policy 0, policy_version 246477 (0.0009) [2023-12-26 17:09:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 126296064. Throughput: 0: 9903.9, 1: 9962.7. Samples: 126267524. Policy #0 lag: (min: 1.0, avg: 23.7, max: 33.0) [2023-12-26 17:09:31,063][104569] Avg episode reward: [(0, '9357.565'), (1, '7631.738')] [2023-12-26 17:09:31,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000246480_63111168.pth... [2023-12-26 17:09:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000245328_62816256.pth [2023-12-26 17:09:31,076][105620] Updated weights for policy 1, policy_version 246800 (0.0009) [2023-12-26 17:09:31,145][105620] Updated weights for policy 1, policy_version 246810 (0.0009) [2023-12-26 17:09:31,203][105620] Updated weights for policy 1, policy_version 246820 (0.0007) [2023-12-26 17:09:31,222][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000246824_63193088.pth... [2023-12-26 17:09:31,224][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000245640_62889984.pth [2023-12-26 17:09:31,582][105692] Updated weights for policy 0, policy_version 246487 (0.0009) [2023-12-26 17:09:31,639][105692] Updated weights for policy 0, policy_version 246497 (0.0009) [2023-12-26 17:09:31,686][105692] Updated weights for policy 0, policy_version 246507 (0.0008) [2023-12-26 17:09:31,951][105620] Updated weights for policy 1, policy_version 246830 (0.0008) [2023-12-26 17:09:32,007][105620] Updated weights for policy 1, policy_version 246840 (0.0009) [2023-12-26 17:09:32,068][105620] Updated weights for policy 1, policy_version 246850 (0.0008) [2023-12-26 17:09:32,489][105692] Updated weights for policy 0, policy_version 246517 (0.0009) [2023-12-26 17:09:32,536][105692] Updated weights for policy 0, policy_version 246527 (0.0009) [2023-12-26 17:09:32,583][105692] Updated weights for policy 0, policy_version 246537 (0.0009) [2023-12-26 17:09:32,805][105620] Updated weights for policy 1, policy_version 246860 (0.0010) [2023-12-26 17:09:32,862][105620] Updated weights for policy 1, policy_version 246870 (0.0008) [2023-12-26 17:09:32,930][105620] Updated weights for policy 1, policy_version 246880 (0.0005) [2023-12-26 17:09:33,295][105692] Updated weights for policy 0, policy_version 246547 (0.0008) [2023-12-26 17:09:33,354][105692] Updated weights for policy 0, policy_version 246557 (0.0007) [2023-12-26 17:09:33,415][105692] Updated weights for policy 0, policy_version 246567 (0.0009) [2023-12-26 17:09:33,597][105620] Updated weights for policy 1, policy_version 246890 (0.0006) [2023-12-26 17:09:33,650][105620] Updated weights for policy 1, policy_version 246900 (0.0009) [2023-12-26 17:09:33,702][105620] Updated weights for policy 1, policy_version 246910 (0.0009) [2023-12-26 17:09:33,759][105620] Updated weights for policy 1, policy_version 246920 (0.0008) [2023-12-26 17:09:34,032][105692] Updated weights for policy 0, policy_version 246577 (0.0008) [2023-12-26 17:09:34,079][105692] Updated weights for policy 0, policy_version 246587 (0.0005) [2023-12-26 17:09:34,143][105692] Updated weights for policy 0, policy_version 246597 (0.0006) [2023-12-26 17:09:34,207][105692] Updated weights for policy 0, policy_version 246607 (0.0008) [2023-12-26 17:09:34,557][105620] Updated weights for policy 1, policy_version 246930 (0.0009) [2023-12-26 17:09:34,619][105620] Updated weights for policy 1, policy_version 246940 (0.0009) [2023-12-26 17:09:34,677][105620] Updated weights for policy 1, policy_version 246950 (0.0009) [2023-12-26 17:09:34,924][105692] Updated weights for policy 0, policy_version 246617 (0.0009) [2023-12-26 17:09:34,970][105692] Updated weights for policy 0, policy_version 246627 (0.0008) [2023-12-26 17:09:35,020][105692] Updated weights for policy 0, policy_version 246637 (0.0009) [2023-12-26 17:09:35,434][105620] Updated weights for policy 1, policy_version 246960 (0.0009) [2023-12-26 17:09:35,492][105620] Updated weights for policy 1, policy_version 246970 (0.0009) [2023-12-26 17:09:35,550][105620] Updated weights for policy 1, policy_version 246980 (0.0009) [2023-12-26 17:09:35,785][105692] Updated weights for policy 0, policy_version 246647 (0.0009) [2023-12-26 17:09:35,846][105692] Updated weights for policy 0, policy_version 246657 (0.0009) [2023-12-26 17:09:35,893][105692] Updated weights for policy 0, policy_version 246667 (0.0008) [2023-12-26 17:09:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 126394368. Throughput: 0: 9849.6, 1: 9827.8. Samples: 126382608. Policy #0 lag: (min: 1.0, avg: 23.7, max: 33.0) [2023-12-26 17:09:36,063][104569] Avg episode reward: [(0, '9357.521'), (1, '7531.553')] [2023-12-26 17:09:36,322][105620] Updated weights for policy 1, policy_version 246990 (0.0009) [2023-12-26 17:09:36,384][105620] Updated weights for policy 1, policy_version 247000 (0.0009) [2023-12-26 17:09:36,445][105620] Updated weights for policy 1, policy_version 247010 (0.0009) [2023-12-26 17:09:36,620][105692] Updated weights for policy 0, policy_version 246677 (0.0009) [2023-12-26 17:09:36,674][105692] Updated weights for policy 0, policy_version 246687 (0.0009) [2023-12-26 17:09:36,733][105692] Updated weights for policy 0, policy_version 246697 (0.0009) [2023-12-26 17:09:37,196][105620] Updated weights for policy 1, policy_version 247020 (0.0009) [2023-12-26 17:09:37,261][105620] Updated weights for policy 1, policy_version 247030 (0.0009) [2023-12-26 17:09:37,323][105620] Updated weights for policy 1, policy_version 247040 (0.0009) [2023-12-26 17:09:37,486][105692] Updated weights for policy 0, policy_version 246707 (0.0009) [2023-12-26 17:09:37,546][105692] Updated weights for policy 0, policy_version 246717 (0.0010) [2023-12-26 17:09:37,599][105692] Updated weights for policy 0, policy_version 246727 (0.0010) [2023-12-26 17:09:38,086][105620] Updated weights for policy 1, policy_version 247050 (0.0009) [2023-12-26 17:09:38,146][105620] Updated weights for policy 1, policy_version 247060 (0.0008) [2023-12-26 17:09:38,209][105620] Updated weights for policy 1, policy_version 247070 (0.0008) [2023-12-26 17:09:38,256][105620] Updated weights for policy 1, policy_version 247080 (0.0008) [2023-12-26 17:09:38,360][105692] Updated weights for policy 0, policy_version 246737 (0.0011) [2023-12-26 17:09:38,420][105692] Updated weights for policy 0, policy_version 246747 (0.0010) [2023-12-26 17:09:38,478][105692] Updated weights for policy 0, policy_version 246757 (0.0010) [2023-12-26 17:09:38,532][105692] Updated weights for policy 0, policy_version 246767 (0.0010) [2023-12-26 17:09:39,034][105620] Updated weights for policy 1, policy_version 247090 (0.0008) [2023-12-26 17:09:39,091][105620] Updated weights for policy 1, policy_version 247100 (0.0008) [2023-12-26 17:09:39,139][105620] Updated weights for policy 1, policy_version 247110 (0.0008) [2023-12-26 17:09:39,321][105692] Updated weights for policy 0, policy_version 246777 (0.0011) [2023-12-26 17:09:39,384][105692] Updated weights for policy 0, policy_version 246787 (0.0009) [2023-12-26 17:09:39,455][105692] Updated weights for policy 0, policy_version 246797 (0.0009) [2023-12-26 17:09:39,960][105620] Updated weights for policy 1, policy_version 247120 (0.0008) [2023-12-26 17:09:40,019][105620] Updated weights for policy 1, policy_version 247130 (0.0009) [2023-12-26 17:09:40,078][105620] Updated weights for policy 1, policy_version 247140 (0.0009) [2023-12-26 17:09:40,122][105692] Updated weights for policy 0, policy_version 246807 (0.0009) [2023-12-26 17:09:40,185][105692] Updated weights for policy 0, policy_version 246817 (0.0011) [2023-12-26 17:09:40,248][105692] Updated weights for policy 0, policy_version 246827 (0.0011) [2023-12-26 17:09:40,840][105692] Updated weights for policy 0, policy_version 246837 (0.0009) [2023-12-26 17:09:40,864][105620] Updated weights for policy 1, policy_version 247150 (0.0008) [2023-12-26 17:09:40,889][105692] Updated weights for policy 0, policy_version 246847 (0.0008) [2023-12-26 17:09:40,919][105620] Updated weights for policy 1, policy_version 247160 (0.0006) [2023-12-26 17:09:40,940][105692] Updated weights for policy 0, policy_version 246857 (0.0010) [2023-12-26 17:09:40,983][105620] Updated weights for policy 1, policy_version 247170 (0.0008) [2023-12-26 17:09:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 126492672. Throughput: 0: 9773.1, 1: 9737.4. Samples: 126494076. Policy #0 lag: (min: 1.0, avg: 23.7, max: 33.0) [2023-12-26 17:09:41,062][104569] Avg episode reward: [(0, '9356.708'), (1, '7063.679')] [2023-12-26 17:09:41,736][105692] Updated weights for policy 0, policy_version 246867 (0.0011) [2023-12-26 17:09:41,781][105620] Updated weights for policy 1, policy_version 247180 (0.0008) [2023-12-26 17:09:41,803][105692] Updated weights for policy 0, policy_version 246877 (0.0009) [2023-12-26 17:09:41,833][105620] Updated weights for policy 1, policy_version 247190 (0.0008) [2023-12-26 17:09:41,866][105692] Updated weights for policy 0, policy_version 246887 (0.0011) [2023-12-26 17:09:41,889][105620] Updated weights for policy 1, policy_version 247200 (0.0008) [2023-12-26 17:09:42,616][105620] Updated weights for policy 1, policy_version 247210 (0.0008) [2023-12-26 17:09:42,634][105692] Updated weights for policy 0, policy_version 246897 (0.0011) [2023-12-26 17:09:42,680][105620] Updated weights for policy 1, policy_version 247220 (0.0006) [2023-12-26 17:09:42,686][105692] Updated weights for policy 0, policy_version 246907 (0.0011) [2023-12-26 17:09:42,744][105620] Updated weights for policy 1, policy_version 247230 (0.0006) [2023-12-26 17:09:42,755][105692] Updated weights for policy 0, policy_version 246917 (0.0010) [2023-12-26 17:09:42,816][105620] Updated weights for policy 1, policy_version 247240 (0.0006) [2023-12-26 17:09:42,818][105692] Updated weights for policy 0, policy_version 246927 (0.0011) [2023-12-26 17:09:43,450][105620] Updated weights for policy 1, policy_version 247250 (0.0008) [2023-12-26 17:09:43,498][105620] Updated weights for policy 1, policy_version 247260 (0.0008) [2023-12-26 17:09:43,546][105620] Updated weights for policy 1, policy_version 247270 (0.0007) [2023-12-26 17:09:43,547][105692] Updated weights for policy 0, policy_version 246937 (0.0010) [2023-12-26 17:09:43,592][105692] Updated weights for policy 0, policy_version 246947 (0.0010) [2023-12-26 17:09:43,643][105692] Updated weights for policy 0, policy_version 246957 (0.0010) [2023-12-26 17:09:44,235][105620] Updated weights for policy 1, policy_version 247280 (0.0005) [2023-12-26 17:09:44,289][105620] Updated weights for policy 1, policy_version 247290 (0.0005) [2023-12-26 17:09:44,346][105620] Updated weights for policy 1, policy_version 247300 (0.0008) [2023-12-26 17:09:44,385][105692] Updated weights for policy 0, policy_version 246967 (0.0010) [2023-12-26 17:09:44,443][105692] Updated weights for policy 0, policy_version 246977 (0.0008) [2023-12-26 17:09:44,497][105692] Updated weights for policy 0, policy_version 246987 (0.0010) [2023-12-26 17:09:45,088][105620] Updated weights for policy 1, policy_version 247310 (0.0009) [2023-12-26 17:09:45,148][105620] Updated weights for policy 1, policy_version 247320 (0.0007) [2023-12-26 17:09:45,156][105692] Updated weights for policy 0, policy_version 246997 (0.0011) [2023-12-26 17:09:45,214][105620] Updated weights for policy 1, policy_version 247330 (0.0007) [2023-12-26 17:09:45,216][105692] Updated weights for policy 0, policy_version 247007 (0.0011) [2023-12-26 17:09:45,279][105692] Updated weights for policy 0, policy_version 247017 (0.0011) [2023-12-26 17:09:45,978][105620] Updated weights for policy 1, policy_version 247340 (0.0007) [2023-12-26 17:09:46,021][105692] Updated weights for policy 0, policy_version 247027 (0.0011) [2023-12-26 17:09:46,033][105620] Updated weights for policy 1, policy_version 247350 (0.0008) [2023-12-26 17:09:46,062][104569] Fps is (10 sec: 18022.6, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 126574592. Throughput: 0: 9778.6, 1: 9745.8. Samples: 126551672. Policy #0 lag: (min: 15.0, avg: 16.7, max: 47.0) [2023-12-26 17:09:46,062][104569] Avg episode reward: [(0, '9355.444'), (1, '7624.429')] [2023-12-26 17:09:46,087][105692] Updated weights for policy 0, policy_version 247037 (0.0010) [2023-12-26 17:09:46,095][105620] Updated weights for policy 1, policy_version 247360 (0.0008) [2023-12-26 17:09:46,143][105692] Updated weights for policy 0, policy_version 247047 (0.0008) [2023-12-26 17:09:46,145][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000247368_63332352.pth... [2023-12-26 17:09:46,150][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000246248_63045632.pth [2023-12-26 17:09:46,199][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000247056_63258624.pth... [2023-12-26 17:09:46,204][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000245904_62963712.pth [2023-12-26 17:09:46,775][105620] Updated weights for policy 1, policy_version 247370 (0.0007) [2023-12-26 17:09:46,828][105692] Updated weights for policy 0, policy_version 247057 (0.0006) [2023-12-26 17:09:46,834][105620] Updated weights for policy 1, policy_version 247380 (0.0008) [2023-12-26 17:09:46,886][105692] Updated weights for policy 0, policy_version 247067 (0.0010) [2023-12-26 17:09:46,892][105620] Updated weights for policy 1, policy_version 247390 (0.0005) [2023-12-26 17:09:46,934][105692] Updated weights for policy 0, policy_version 247077 (0.0010) [2023-12-26 17:09:46,951][105620] Updated weights for policy 1, policy_version 247400 (0.0005) [2023-12-26 17:09:46,988][105692] Updated weights for policy 0, policy_version 247087 (0.0010) [2023-12-26 17:09:47,695][105620] Updated weights for policy 1, policy_version 247410 (0.0008) [2023-12-26 17:09:47,735][105692] Updated weights for policy 0, policy_version 247097 (0.0010) [2023-12-26 17:09:47,753][105620] Updated weights for policy 1, policy_version 247420 (0.0009) [2023-12-26 17:09:47,796][105692] Updated weights for policy 0, policy_version 247107 (0.0010) [2023-12-26 17:09:47,807][105620] Updated weights for policy 1, policy_version 247430 (0.0006) [2023-12-26 17:09:47,854][105692] Updated weights for policy 0, policy_version 247117 (0.0010) [2023-12-26 17:09:48,553][105692] Updated weights for policy 0, policy_version 247127 (0.0008) [2023-12-26 17:09:48,584][105620] Updated weights for policy 1, policy_version 247440 (0.0006) [2023-12-26 17:09:48,617][105692] Updated weights for policy 0, policy_version 247137 (0.0009) [2023-12-26 17:09:48,650][105620] Updated weights for policy 1, policy_version 247450 (0.0008) [2023-12-26 17:09:48,677][105692] Updated weights for policy 0, policy_version 247147 (0.0007) [2023-12-26 17:09:48,704][105620] Updated weights for policy 1, policy_version 247460 (0.0007) [2023-12-26 17:09:49,405][105692] Updated weights for policy 0, policy_version 247157 (0.0007) [2023-12-26 17:09:49,412][105620] Updated weights for policy 1, policy_version 247470 (0.0008) [2023-12-26 17:09:49,461][105692] Updated weights for policy 0, policy_version 247167 (0.0008) [2023-12-26 17:09:49,468][105620] Updated weights for policy 1, policy_version 247480 (0.0007) [2023-12-26 17:09:49,518][105620] Updated weights for policy 1, policy_version 247490 (0.0007) [2023-12-26 17:09:49,521][105692] Updated weights for policy 0, policy_version 247177 (0.0007) [2023-12-26 17:09:50,274][105620] Updated weights for policy 1, policy_version 247500 (0.0007) [2023-12-26 17:09:50,310][105692] Updated weights for policy 0, policy_version 247187 (0.0007) [2023-12-26 17:09:50,336][105620] Updated weights for policy 1, policy_version 247510 (0.0007) [2023-12-26 17:09:50,363][105692] Updated weights for policy 0, policy_version 247197 (0.0006) [2023-12-26 17:09:50,390][105620] Updated weights for policy 1, policy_version 247520 (0.0006) [2023-12-26 17:09:50,417][105692] Updated weights for policy 0, policy_version 247207 (0.0006) [2023-12-26 17:09:51,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 126672896. Throughput: 0: 9801.6, 1: 9705.6. Samples: 126667280. Policy #0 lag: (min: 15.0, avg: 16.7, max: 47.0) [2023-12-26 17:09:51,063][104569] Avg episode reward: [(0, '9355.260'), (1, '8446.797')] [2023-12-26 17:09:51,158][105692] Updated weights for policy 0, policy_version 247217 (0.0006) [2023-12-26 17:09:51,168][105620] Updated weights for policy 1, policy_version 247530 (0.0007) [2023-12-26 17:09:51,220][105692] Updated weights for policy 0, policy_version 247227 (0.0006) [2023-12-26 17:09:51,230][105620] Updated weights for policy 1, policy_version 247540 (0.0006) [2023-12-26 17:09:51,288][105692] Updated weights for policy 0, policy_version 247237 (0.0008) [2023-12-26 17:09:51,291][105620] Updated weights for policy 1, policy_version 247550 (0.0006) [2023-12-26 17:09:51,345][105692] Updated weights for policy 0, policy_version 247247 (0.0007) [2023-12-26 17:09:51,350][105620] Updated weights for policy 1, policy_version 247560 (0.0007) [2023-12-26 17:09:51,966][105692] Updated weights for policy 0, policy_version 247257 (0.0009) [2023-12-26 17:09:52,015][105692] Updated weights for policy 0, policy_version 247267 (0.0009) [2023-12-26 17:09:52,063][105692] Updated weights for policy 0, policy_version 247277 (0.0010) [2023-12-26 17:09:52,103][105620] Updated weights for policy 1, policy_version 247570 (0.0007) [2023-12-26 17:09:52,159][105620] Updated weights for policy 1, policy_version 247580 (0.0008) [2023-12-26 17:09:52,222][105620] Updated weights for policy 1, policy_version 247590 (0.0008) [2023-12-26 17:09:52,777][105692] Updated weights for policy 0, policy_version 247287 (0.0007) [2023-12-26 17:09:52,831][105692] Updated weights for policy 0, policy_version 247297 (0.0005) [2023-12-26 17:09:52,885][105692] Updated weights for policy 0, policy_version 247307 (0.0006) [2023-12-26 17:09:53,053][105620] Updated weights for policy 1, policy_version 247600 (0.0008) [2023-12-26 17:09:53,114][105620] Updated weights for policy 1, policy_version 247610 (0.0008) [2023-12-26 17:09:53,165][105620] Updated weights for policy 1, policy_version 247620 (0.0009) [2023-12-26 17:09:53,494][105692] Updated weights for policy 0, policy_version 247317 (0.0005) [2023-12-26 17:09:53,546][105692] Updated weights for policy 0, policy_version 247327 (0.0009) [2023-12-26 17:09:53,603][105692] Updated weights for policy 0, policy_version 247337 (0.0008) [2023-12-26 17:09:53,898][105620] Updated weights for policy 1, policy_version 247630 (0.0010) [2023-12-26 17:09:53,943][105620] Updated weights for policy 1, policy_version 247640 (0.0010) [2023-12-26 17:09:53,990][105620] Updated weights for policy 1, policy_version 247650 (0.0010) [2023-12-26 17:09:54,380][105692] Updated weights for policy 0, policy_version 247347 (0.0007) [2023-12-26 17:09:54,435][105692] Updated weights for policy 0, policy_version 247357 (0.0008) [2023-12-26 17:09:54,488][105692] Updated weights for policy 0, policy_version 247367 (0.0010) [2023-12-26 17:09:54,614][105620] Updated weights for policy 1, policy_version 247660 (0.0007) [2023-12-26 17:09:54,666][105620] Updated weights for policy 1, policy_version 247670 (0.0011) [2023-12-26 17:09:54,723][105620] Updated weights for policy 1, policy_version 247680 (0.0011) [2023-12-26 17:09:55,291][105692] Updated weights for policy 0, policy_version 247378 (0.0010) [2023-12-26 17:09:55,340][105692] Updated weights for policy 0, policy_version 247388 (0.0008) [2023-12-26 17:09:55,388][105692] Updated weights for policy 0, policy_version 247398 (0.0008) [2023-12-26 17:09:55,444][105692] Updated weights for policy 0, policy_version 247408 (0.0008) [2023-12-26 17:09:55,483][105620] Updated weights for policy 1, policy_version 247690 (0.0010) [2023-12-26 17:09:55,544][105620] Updated weights for policy 1, policy_version 247700 (0.0010) [2023-12-26 17:09:55,605][105620] Updated weights for policy 1, policy_version 247710 (0.0010) [2023-12-26 17:09:55,677][105620] Updated weights for policy 1, policy_version 247720 (0.0010) [2023-12-26 17:09:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 126771200. Throughput: 0: 9765.7, 1: 9630.8. Samples: 126781456. Policy #0 lag: (min: 15.0, avg: 16.7, max: 47.0) [2023-12-26 17:09:56,062][104569] Avg episode reward: [(0, '9355.395'), (1, '8632.928')] [2023-12-26 17:09:56,169][105692] Updated weights for policy 0, policy_version 247418 (0.0005) [2023-12-26 17:09:56,215][105692] Updated weights for policy 0, policy_version 247428 (0.0006) [2023-12-26 17:09:56,284][105692] Updated weights for policy 0, policy_version 247438 (0.0005) [2023-12-26 17:09:56,368][105620] Updated weights for policy 1, policy_version 247730 (0.0006) [2023-12-26 17:09:56,435][105620] Updated weights for policy 1, policy_version 247740 (0.0006) [2023-12-26 17:09:56,490][105620] Updated weights for policy 1, policy_version 247750 (0.0008) [2023-12-26 17:09:56,898][105692] Updated weights for policy 0, policy_version 247448 (0.0010) [2023-12-26 17:09:56,955][105692] Updated weights for policy 0, policy_version 247458 (0.0010) [2023-12-26 17:09:57,016][105692] Updated weights for policy 0, policy_version 247468 (0.0010) [2023-12-26 17:09:57,041][105620] Updated weights for policy 1, policy_version 247760 (0.0007) [2023-12-26 17:09:57,092][105620] Updated weights for policy 1, policy_version 247770 (0.0007) [2023-12-26 17:09:57,146][105620] Updated weights for policy 1, policy_version 247780 (0.0008) [2023-12-26 17:09:57,732][105692] Updated weights for policy 0, policy_version 247478 (0.0010) [2023-12-26 17:09:57,779][105692] Updated weights for policy 0, policy_version 247488 (0.0010) [2023-12-26 17:09:57,837][105692] Updated weights for policy 0, policy_version 247498 (0.0008) [2023-12-26 17:09:57,885][105620] Updated weights for policy 1, policy_version 247790 (0.0006) [2023-12-26 17:09:57,947][105620] Updated weights for policy 1, policy_version 247800 (0.0005) [2023-12-26 17:09:58,003][105620] Updated weights for policy 1, policy_version 247810 (0.0006) [2023-12-26 17:09:58,570][105692] Updated weights for policy 0, policy_version 247508 (0.0010) [2023-12-26 17:09:58,629][105620] Updated weights for policy 1, policy_version 247820 (0.0006) [2023-12-26 17:09:58,634][105692] Updated weights for policy 0, policy_version 247518 (0.0011) [2023-12-26 17:09:58,696][105692] Updated weights for policy 0, policy_version 247528 (0.0009) [2023-12-26 17:09:58,701][105620] Updated weights for policy 1, policy_version 247830 (0.0008) [2023-12-26 17:09:58,779][105620] Updated weights for policy 1, policy_version 247840 (0.0007) [2023-12-26 17:09:59,539][105620] Updated weights for policy 1, policy_version 247850 (0.0008) [2023-12-26 17:09:59,554][105692] Updated weights for policy 0, policy_version 247538 (0.0007) [2023-12-26 17:09:59,602][105620] Updated weights for policy 1, policy_version 247860 (0.0006) [2023-12-26 17:09:59,611][105692] Updated weights for policy 0, policy_version 247548 (0.0007) [2023-12-26 17:09:59,661][105692] Updated weights for policy 0, policy_version 247558 (0.0007) [2023-12-26 17:09:59,671][105620] Updated weights for policy 1, policy_version 247870 (0.0006) [2023-12-26 17:09:59,725][105692] Updated weights for policy 0, policy_version 247568 (0.0006) [2023-12-26 17:09:59,734][105620] Updated weights for policy 1, policy_version 247880 (0.0006) [2023-12-26 17:10:00,378][105620] Updated weights for policy 1, policy_version 247890 (0.0007) [2023-12-26 17:10:00,433][105620] Updated weights for policy 1, policy_version 247900 (0.0009) [2023-12-26 17:10:00,493][105620] Updated weights for policy 1, policy_version 247910 (0.0006) [2023-12-26 17:10:00,514][105692] Updated weights for policy 0, policy_version 247578 (0.0010) [2023-12-26 17:10:00,572][105692] Updated weights for policy 0, policy_version 247588 (0.0010) [2023-12-26 17:10:00,617][105692] Updated weights for policy 0, policy_version 247598 (0.0010) [2023-12-26 17:10:01,044][105620] Updated weights for policy 1, policy_version 247920 (0.0007) [2023-12-26 17:10:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 126869504. Throughput: 0: 9841.0, 1: 9661.8. Samples: 126843008. Policy #0 lag: (min: 15.0, avg: 16.7, max: 47.0) [2023-12-26 17:10:01,063][104569] Avg episode reward: [(0, '9356.160'), (1, '8544.332')] [2023-12-26 17:10:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000247600_63397888.pth... [2023-12-26 17:10:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000246480_63111168.pth [2023-12-26 17:10:01,104][105620] Updated weights for policy 1, policy_version 247930 (0.0007) [2023-12-26 17:10:01,168][105620] Updated weights for policy 1, policy_version 247940 (0.0007) [2023-12-26 17:10:01,189][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000247944_63479808.pth... [2023-12-26 17:10:01,193][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000246824_63193088.pth [2023-12-26 17:10:01,289][105692] Updated weights for policy 0, policy_version 247608 (0.0009) [2023-12-26 17:10:01,362][105692] Updated weights for policy 0, policy_version 247618 (0.0007) [2023-12-26 17:10:01,422][105692] Updated weights for policy 0, policy_version 247628 (0.0010) [2023-12-26 17:10:01,785][105620] Updated weights for policy 1, policy_version 247950 (0.0006) [2023-12-26 17:10:01,844][105620] Updated weights for policy 1, policy_version 247960 (0.0005) [2023-12-26 17:10:01,898][105620] Updated weights for policy 1, policy_version 247970 (0.0006) [2023-12-26 17:10:02,169][105692] Updated weights for policy 0, policy_version 247638 (0.0009) [2023-12-26 17:10:02,227][105692] Updated weights for policy 0, policy_version 247648 (0.0009) [2023-12-26 17:10:02,294][105692] Updated weights for policy 0, policy_version 247658 (0.0009) [2023-12-26 17:10:02,552][105620] Updated weights for policy 1, policy_version 247980 (0.0008) [2023-12-26 17:10:02,610][105620] Updated weights for policy 1, policy_version 247990 (0.0005) [2023-12-26 17:10:02,670][105620] Updated weights for policy 1, policy_version 248000 (0.0005) [2023-12-26 17:10:03,059][105692] Updated weights for policy 0, policy_version 247668 (0.0010) [2023-12-26 17:10:03,118][105692] Updated weights for policy 0, policy_version 247679 (0.0010) [2023-12-26 17:10:03,181][105620] Updated weights for policy 1, policy_version 248010 (0.0005) [2023-12-26 17:10:03,181][105692] Updated weights for policy 0, policy_version 247689 (0.0008) [2023-12-26 17:10:03,232][105620] Updated weights for policy 1, policy_version 248020 (0.0005) [2023-12-26 17:10:03,294][105620] Updated weights for policy 1, policy_version 248030 (0.0006) [2023-12-26 17:10:03,346][105620] Updated weights for policy 1, policy_version 248040 (0.0007) [2023-12-26 17:10:03,745][105692] Updated weights for policy 0, policy_version 247699 (0.0007) [2023-12-26 17:10:03,799][105692] Updated weights for policy 0, policy_version 247709 (0.0010) [2023-12-26 17:10:03,854][105692] Updated weights for policy 0, policy_version 247719 (0.0009) [2023-12-26 17:10:03,879][105620] Updated weights for policy 1, policy_version 248050 (0.0008) [2023-12-26 17:10:03,930][105620] Updated weights for policy 1, policy_version 248060 (0.0007) [2023-12-26 17:10:03,982][105620] Updated weights for policy 1, policy_version 248070 (0.0008) [2023-12-26 17:10:04,643][105692] Updated weights for policy 0, policy_version 247729 (0.0008) [2023-12-26 17:10:04,695][105692] Updated weights for policy 0, policy_version 247739 (0.0010) [2023-12-26 17:10:04,698][105620] Updated weights for policy 1, policy_version 248080 (0.0006) [2023-12-26 17:10:04,750][105692] Updated weights for policy 0, policy_version 247749 (0.0010) [2023-12-26 17:10:04,752][105620] Updated weights for policy 1, policy_version 248090 (0.0005) [2023-12-26 17:10:04,805][105692] Updated weights for policy 0, policy_version 247759 (0.0010) [2023-12-26 17:10:04,807][105620] Updated weights for policy 1, policy_version 248100 (0.0006) [2023-12-26 17:10:05,446][105620] Updated weights for policy 1, policy_version 248110 (0.0008) [2023-12-26 17:10:05,504][105620] Updated weights for policy 1, policy_version 248120 (0.0008) [2023-12-26 17:10:05,555][105692] Updated weights for policy 0, policy_version 247769 (0.0010) [2023-12-26 17:10:05,566][105620] Updated weights for policy 1, policy_version 248130 (0.0005) [2023-12-26 17:10:05,617][105692] Updated weights for policy 0, policy_version 247779 (0.0010) [2023-12-26 17:10:05,683][105692] Updated weights for policy 0, policy_version 247789 (0.0009) [2023-12-26 17:10:06,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 126976000. Throughput: 0: 9756.1, 1: 9775.2. Samples: 126966188. Policy #0 lag: (min: 15.0, avg: 16.7, max: 47.0) [2023-12-26 17:10:06,063][104569] Avg episode reward: [(0, '9266.312'), (1, '8084.428')] [2023-12-26 17:10:06,302][105620] Updated weights for policy 1, policy_version 248140 (0.0007) [2023-12-26 17:10:06,367][105620] Updated weights for policy 1, policy_version 248150 (0.0008) [2023-12-26 17:10:06,425][105692] Updated weights for policy 0, policy_version 247799 (0.0007) [2023-12-26 17:10:06,429][105620] Updated weights for policy 1, policy_version 248160 (0.0008) [2023-12-26 17:10:06,476][105692] Updated weights for policy 0, policy_version 247809 (0.0007) [2023-12-26 17:10:06,524][105692] Updated weights for policy 0, policy_version 247819 (0.0009) [2023-12-26 17:10:07,059][105620] Updated weights for policy 1, policy_version 248170 (0.0008) [2023-12-26 17:10:07,119][105620] Updated weights for policy 1, policy_version 248180 (0.0006) [2023-12-26 17:10:07,190][105620] Updated weights for policy 1, policy_version 248190 (0.0005) [2023-12-26 17:10:07,253][105620] Updated weights for policy 1, policy_version 248200 (0.0005) [2023-12-26 17:10:07,406][105692] Updated weights for policy 0, policy_version 247829 (0.0010) [2023-12-26 17:10:07,471][105692] Updated weights for policy 0, policy_version 247839 (0.0011) [2023-12-26 17:10:07,525][105692] Updated weights for policy 0, policy_version 247849 (0.0010) [2023-12-26 17:10:07,852][105620] Updated weights for policy 1, policy_version 248210 (0.0005) [2023-12-26 17:10:07,911][105620] Updated weights for policy 1, policy_version 248220 (0.0006) [2023-12-26 17:10:07,965][105620] Updated weights for policy 1, policy_version 248230 (0.0005) [2023-12-26 17:10:08,197][105692] Updated weights for policy 0, policy_version 247859 (0.0010) [2023-12-26 17:10:08,262][105692] Updated weights for policy 0, policy_version 247869 (0.0011) [2023-12-26 17:10:08,318][105692] Updated weights for policy 0, policy_version 247879 (0.0011) [2023-12-26 17:10:08,634][105620] Updated weights for policy 1, policy_version 248240 (0.0009) [2023-12-26 17:10:08,694][105620] Updated weights for policy 1, policy_version 248250 (0.0011) [2023-12-26 17:10:08,753][105620] Updated weights for policy 1, policy_version 248260 (0.0010) [2023-12-26 17:10:09,061][105692] Updated weights for policy 0, policy_version 247889 (0.0011) [2023-12-26 17:10:09,112][105692] Updated weights for policy 0, policy_version 247899 (0.0010) [2023-12-26 17:10:09,158][105692] Updated weights for policy 0, policy_version 247909 (0.0008) [2023-12-26 17:10:09,220][105692] Updated weights for policy 0, policy_version 247919 (0.0006) [2023-12-26 17:10:09,512][105620] Updated weights for policy 1, policy_version 248270 (0.0010) [2023-12-26 17:10:09,577][105620] Updated weights for policy 1, policy_version 248280 (0.0010) [2023-12-26 17:10:09,642][105620] Updated weights for policy 1, policy_version 248290 (0.0010) [2023-12-26 17:10:09,954][105692] Updated weights for policy 0, policy_version 247929 (0.0008) [2023-12-26 17:10:10,021][105692] Updated weights for policy 0, policy_version 247939 (0.0011) [2023-12-26 17:10:10,085][105692] Updated weights for policy 0, policy_version 247949 (0.0009) [2023-12-26 17:10:10,333][105620] Updated weights for policy 1, policy_version 248300 (0.0011) [2023-12-26 17:10:10,393][105620] Updated weights for policy 1, policy_version 248310 (0.0011) [2023-12-26 17:10:10,458][105620] Updated weights for policy 1, policy_version 248320 (0.0011) [2023-12-26 17:10:10,793][105692] Updated weights for policy 0, policy_version 247959 (0.0009) [2023-12-26 17:10:10,852][105692] Updated weights for policy 0, policy_version 247969 (0.0011) [2023-12-26 17:10:10,918][105692] Updated weights for policy 0, policy_version 247979 (0.0011) [2023-12-26 17:10:11,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 127074304. Throughput: 0: 9743.3, 1: 9791.9. Samples: 127082608. Policy #0 lag: (min: 15.0, avg: 16.7, max: 47.0) [2023-12-26 17:10:11,062][104569] Avg episode reward: [(0, '9266.685'), (1, '8438.199')] [2023-12-26 17:10:11,180][105620] Updated weights for policy 1, policy_version 248330 (0.0011) [2023-12-26 17:10:11,244][105620] Updated weights for policy 1, policy_version 248340 (0.0010) [2023-12-26 17:10:11,308][105620] Updated weights for policy 1, policy_version 248350 (0.0010) [2023-12-26 17:10:11,366][105620] Updated weights for policy 1, policy_version 248360 (0.0008) [2023-12-26 17:10:11,658][105692] Updated weights for policy 0, policy_version 247989 (0.0010) [2023-12-26 17:10:11,729][105692] Updated weights for policy 0, policy_version 247999 (0.0008) [2023-12-26 17:10:11,792][105692] Updated weights for policy 0, policy_version 248009 (0.0007) [2023-12-26 17:10:12,099][105620] Updated weights for policy 1, policy_version 248370 (0.0010) [2023-12-26 17:10:12,165][105620] Updated weights for policy 1, policy_version 248380 (0.0009) [2023-12-26 17:10:12,218][105620] Updated weights for policy 1, policy_version 248390 (0.0010) [2023-12-26 17:10:12,392][105692] Updated weights for policy 0, policy_version 248019 (0.0006) [2023-12-26 17:10:12,453][105692] Updated weights for policy 0, policy_version 248029 (0.0008) [2023-12-26 17:10:12,512][105692] Updated weights for policy 0, policy_version 248039 (0.0005) [2023-12-26 17:10:13,045][105620] Updated weights for policy 1, policy_version 248400 (0.0009) [2023-12-26 17:10:13,103][105620] Updated weights for policy 1, policy_version 248410 (0.0009) [2023-12-26 17:10:13,137][105692] Updated weights for policy 0, policy_version 248049 (0.0005) [2023-12-26 17:10:13,158][105620] Updated weights for policy 1, policy_version 248420 (0.0010) [2023-12-26 17:10:13,185][105692] Updated weights for policy 0, policy_version 248059 (0.0007) [2023-12-26 17:10:13,244][105692] Updated weights for policy 0, policy_version 248069 (0.0006) [2023-12-26 17:10:13,298][105692] Updated weights for policy 0, policy_version 248079 (0.0008) [2023-12-26 17:10:13,893][105620] Updated weights for policy 1, policy_version 248430 (0.0008) [2023-12-26 17:10:13,941][105620] Updated weights for policy 1, policy_version 248441 (0.0009) [2023-12-26 17:10:14,001][105620] Updated weights for policy 1, policy_version 248451 (0.0009) [2023-12-26 17:10:14,014][105692] Updated weights for policy 0, policy_version 248089 (0.0010) [2023-12-26 17:10:14,076][105692] Updated weights for policy 0, policy_version 248099 (0.0010) [2023-12-26 17:10:14,141][105692] Updated weights for policy 0, policy_version 248109 (0.0008) [2023-12-26 17:10:14,631][105620] Updated weights for policy 1, policy_version 248461 (0.0006) [2023-12-26 17:10:14,684][105620] Updated weights for policy 1, policy_version 248471 (0.0005) [2023-12-26 17:10:14,735][105620] Updated weights for policy 1, policy_version 248481 (0.0005) [2023-12-26 17:10:14,850][105692] Updated weights for policy 0, policy_version 248119 (0.0011) [2023-12-26 17:10:14,917][105692] Updated weights for policy 0, policy_version 248129 (0.0011) [2023-12-26 17:10:14,987][105692] Updated weights for policy 0, policy_version 248139 (0.0011) [2023-12-26 17:10:15,428][105620] Updated weights for policy 1, policy_version 248491 (0.0009) [2023-12-26 17:10:15,450][105586] KL-divergence is very high: 434.2661 [2023-12-26 17:10:15,469][105586] KL-divergence is very high: 120.3414 [2023-12-26 17:10:15,474][105620] Updated weights for policy 1, policy_version 248501 (0.0010) [2023-12-26 17:10:15,490][105586] KL-divergence is very high: 723.4046 [2023-12-26 17:10:15,510][105586] KL-divergence is very high: 102.4289 [2023-12-26 17:10:15,526][105620] Updated weights for policy 1, policy_version 248511 (0.0011) [2023-12-26 17:10:15,536][105586] KL-divergence is very high: 748.8741 [2023-12-26 17:10:15,733][105692] Updated weights for policy 0, policy_version 248149 (0.0010) [2023-12-26 17:10:15,799][105692] Updated weights for policy 0, policy_version 248159 (0.0010) [2023-12-26 17:10:15,848][105692] Updated weights for policy 0, policy_version 248169 (0.0009) [2023-12-26 17:10:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 127172608. Throughput: 0: 9671.5, 1: 9724.5. Samples: 127140344. Policy #0 lag: (min: 15.0, avg: 16.7, max: 47.0) [2023-12-26 17:10:16,062][104569] Avg episode reward: [(0, '9266.729'), (1, '8181.971')] [2023-12-26 17:10:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000248176_63545344.pth... [2023-12-26 17:10:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000248520_63627264.pth... [2023-12-26 17:10:16,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000247056_63258624.pth [2023-12-26 17:10:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000247368_63332352.pth [2023-12-26 17:10:16,272][105586] KL-divergence is very high: 112.7248 [2023-12-26 17:10:16,278][105620] Updated weights for policy 1, policy_version 248521 (0.0010) [2023-12-26 17:10:16,331][105620] Updated weights for policy 1, policy_version 248531 (0.0005) [2023-12-26 17:10:16,381][105620] Updated weights for policy 1, policy_version 248541 (0.0005) [2023-12-26 17:10:16,430][105620] Updated weights for policy 1, policy_version 248551 (0.0005) [2023-12-26 17:10:16,514][105692] Updated weights for policy 0, policy_version 248179 (0.0007) [2023-12-26 17:10:16,570][105692] Updated weights for policy 0, policy_version 248189 (0.0005) [2023-12-26 17:10:16,640][105692] Updated weights for policy 0, policy_version 248199 (0.0011) [2023-12-26 17:10:16,952][105620] Updated weights for policy 1, policy_version 248561 (0.0005) [2023-12-26 17:10:17,006][105620] Updated weights for policy 1, policy_version 248571 (0.0010) [2023-12-26 17:10:17,066][105620] Updated weights for policy 1, policy_version 248581 (0.0009) [2023-12-26 17:10:17,230][105692] Updated weights for policy 0, policy_version 248209 (0.0010) [2023-12-26 17:10:17,284][105692] Updated weights for policy 0, policy_version 248219 (0.0005) [2023-12-26 17:10:17,344][105692] Updated weights for policy 0, policy_version 248229 (0.0005) [2023-12-26 17:10:17,406][105692] Updated weights for policy 0, policy_version 248239 (0.0008) [2023-12-26 17:10:17,805][105620] Updated weights for policy 1, policy_version 248591 (0.0007) [2023-12-26 17:10:17,870][105620] Updated weights for policy 1, policy_version 248601 (0.0005) [2023-12-26 17:10:17,929][105620] Updated weights for policy 1, policy_version 248611 (0.0006) [2023-12-26 17:10:18,069][105692] Updated weights for policy 0, policy_version 248249 (0.0006) [2023-12-26 17:10:18,128][105692] Updated weights for policy 0, policy_version 248259 (0.0005) [2023-12-26 17:10:18,182][105692] Updated weights for policy 0, policy_version 248269 (0.0009) [2023-12-26 17:10:18,585][105620] Updated weights for policy 1, policy_version 248621 (0.0007) [2023-12-26 17:10:18,646][105620] Updated weights for policy 1, policy_version 248631 (0.0009) [2023-12-26 17:10:18,702][105620] Updated weights for policy 1, policy_version 248641 (0.0008) [2023-12-26 17:10:18,891][105692] Updated weights for policy 0, policy_version 248279 (0.0010) [2023-12-26 17:10:18,940][105692] Updated weights for policy 0, policy_version 248289 (0.0010) [2023-12-26 17:10:18,985][105692] Updated weights for policy 0, policy_version 248299 (0.0006) [2023-12-26 17:10:19,526][105620] Updated weights for policy 1, policy_version 248651 (0.0008) [2023-12-26 17:10:19,583][105620] Updated weights for policy 1, policy_version 248661 (0.0006) [2023-12-26 17:10:19,627][105692] Updated weights for policy 0, policy_version 248309 (0.0007) [2023-12-26 17:10:19,646][105620] Updated weights for policy 1, policy_version 248671 (0.0007) [2023-12-26 17:10:19,693][105692] Updated weights for policy 0, policy_version 248319 (0.0007) [2023-12-26 17:10:19,762][105692] Updated weights for policy 0, policy_version 248329 (0.0009) [2023-12-26 17:10:20,272][105620] Updated weights for policy 1, policy_version 248681 (0.0006) [2023-12-26 17:10:20,338][105620] Updated weights for policy 1, policy_version 248691 (0.0009) [2023-12-26 17:10:20,396][105620] Updated weights for policy 1, policy_version 248701 (0.0007) [2023-12-26 17:10:20,445][105620] Updated weights for policy 1, policy_version 248711 (0.0005) [2023-12-26 17:10:20,592][105692] Updated weights for policy 0, policy_version 248339 (0.0010) [2023-12-26 17:10:20,647][105692] Updated weights for policy 0, policy_version 248349 (0.0010) [2023-12-26 17:10:20,705][105692] Updated weights for policy 0, policy_version 248359 (0.0009) [2023-12-26 17:10:21,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19605.2). Total num frames: 127270912. Throughput: 0: 9729.7, 1: 9833.6. Samples: 127262956. Policy #0 lag: (min: 15.0, avg: 16.7, max: 47.0) [2023-12-26 17:10:21,063][104569] Avg episode reward: [(0, '9357.444'), (1, '7917.955')] [2023-12-26 17:10:21,155][105586] KL-divergence is very high: 294.4972 [2023-12-26 17:10:21,162][105620] Updated weights for policy 1, policy_version 248721 (0.0008) [2023-12-26 17:10:21,162][105586] KL-divergence is very high: 353.2846 [2023-12-26 17:10:21,204][105586] KL-divergence is very high: 475.2439 [2023-12-26 17:10:21,209][105586] KL-divergence is very high: 516.0961 [2023-12-26 17:10:21,220][105620] Updated weights for policy 1, policy_version 248731 (0.0006) [2023-12-26 17:10:21,248][105586] KL-divergence is very high: 426.4841 [2023-12-26 17:10:21,254][105586] KL-divergence is very high: 448.2358 [2023-12-26 17:10:21,277][105620] Updated weights for policy 1, policy_version 248741 (0.0008) [2023-12-26 17:10:21,508][105692] Updated weights for policy 0, policy_version 248369 (0.0008) [2023-12-26 17:10:21,576][105692] Updated weights for policy 0, policy_version 248379 (0.0008) [2023-12-26 17:10:21,640][105692] Updated weights for policy 0, policy_version 248389 (0.0008) [2023-12-26 17:10:21,705][105692] Updated weights for policy 0, policy_version 248399 (0.0007) [2023-12-26 17:10:22,101][105620] Updated weights for policy 1, policy_version 248751 (0.0008) [2023-12-26 17:10:22,158][105620] Updated weights for policy 1, policy_version 248761 (0.0009) [2023-12-26 17:10:22,220][105620] Updated weights for policy 1, policy_version 248771 (0.0009) [2023-12-26 17:10:22,357][105692] Updated weights for policy 0, policy_version 248409 (0.0009) [2023-12-26 17:10:22,424][105692] Updated weights for policy 0, policy_version 248419 (0.0008) [2023-12-26 17:10:22,496][105692] Updated weights for policy 0, policy_version 248429 (0.0009) [2023-12-26 17:10:22,939][105620] Updated weights for policy 1, policy_version 248781 (0.0009) [2023-12-26 17:10:22,998][105620] Updated weights for policy 1, policy_version 248791 (0.0007) [2023-12-26 17:10:23,046][105620] Updated weights for policy 1, policy_version 248801 (0.0005) [2023-12-26 17:10:23,308][105692] Updated weights for policy 0, policy_version 248439 (0.0010) [2023-12-26 17:10:23,357][105692] Updated weights for policy 0, policy_version 248449 (0.0010) [2023-12-26 17:10:23,405][105692] Updated weights for policy 0, policy_version 248459 (0.0009) [2023-12-26 17:10:23,578][105620] Updated weights for policy 1, policy_version 248811 (0.0007) [2023-12-26 17:10:23,623][105620] Updated weights for policy 1, policy_version 248821 (0.0010) [2023-12-26 17:10:23,671][105620] Updated weights for policy 1, policy_version 248831 (0.0010) [2023-12-26 17:10:23,966][105692] Updated weights for policy 0, policy_version 248469 (0.0007) [2023-12-26 17:10:24,019][105692] Updated weights for policy 0, policy_version 248479 (0.0010) [2023-12-26 17:10:24,085][105692] Updated weights for policy 0, policy_version 248489 (0.0011) [2023-12-26 17:10:24,358][105620] Updated weights for policy 1, policy_version 248841 (0.0010) [2023-12-26 17:10:24,417][105620] Updated weights for policy 1, policy_version 248851 (0.0011) [2023-12-26 17:10:24,479][105620] Updated weights for policy 1, policy_version 248861 (0.0005) [2023-12-26 17:10:24,545][105620] Updated weights for policy 1, policy_version 248871 (0.0005) [2023-12-26 17:10:24,740][105692] Updated weights for policy 0, policy_version 248499 (0.0010) [2023-12-26 17:10:24,784][105692] Updated weights for policy 0, policy_version 248509 (0.0010) [2023-12-26 17:10:24,829][105692] Updated weights for policy 0, policy_version 248519 (0.0010) [2023-12-26 17:10:25,071][105620] Updated weights for policy 1, policy_version 248881 (0.0009) [2023-12-26 17:10:25,120][105620] Updated weights for policy 1, policy_version 248891 (0.0010) [2023-12-26 17:10:25,168][105620] Updated weights for policy 1, policy_version 248901 (0.0010) [2023-12-26 17:10:25,600][105692] Updated weights for policy 0, policy_version 248529 (0.0010) [2023-12-26 17:10:25,664][105692] Updated weights for policy 0, policy_version 248539 (0.0010) [2023-12-26 17:10:25,738][105692] Updated weights for policy 0, policy_version 248549 (0.0010) [2023-12-26 17:10:25,803][105692] Updated weights for policy 0, policy_version 248559 (0.0010) [2023-12-26 17:10:25,867][105620] Updated weights for policy 1, policy_version 248911 (0.0008) [2023-12-26 17:10:25,925][105620] Updated weights for policy 1, policy_version 248921 (0.0010) [2023-12-26 17:10:25,976][105620] Updated weights for policy 1, policy_version 248931 (0.0010) [2023-12-26 17:10:26,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 127377408. Throughput: 0: 9731.2, 1: 10018.1. Samples: 127382796. Policy #0 lag: (min: 15.0, avg: 16.7, max: 47.0) [2023-12-26 17:10:26,063][104569] Avg episode reward: [(0, '9357.334'), (1, '7642.973')] [2023-12-26 17:10:26,519][105692] Updated weights for policy 0, policy_version 248569 (0.0008) [2023-12-26 17:10:26,579][105692] Updated weights for policy 0, policy_version 248579 (0.0009) [2023-12-26 17:10:26,634][105692] Updated weights for policy 0, policy_version 248589 (0.0008) [2023-12-26 17:10:26,685][105620] Updated weights for policy 1, policy_version 248941 (0.0009) [2023-12-26 17:10:26,734][105620] Updated weights for policy 1, policy_version 248951 (0.0006) [2023-12-26 17:10:26,779][105620] Updated weights for policy 1, policy_version 248961 (0.0010) [2023-12-26 17:10:27,387][105692] Updated weights for policy 0, policy_version 248599 (0.0006) [2023-12-26 17:10:27,447][105692] Updated weights for policy 0, policy_version 248609 (0.0005) [2023-12-26 17:10:27,490][105620] Updated weights for policy 1, policy_version 248971 (0.0009) [2023-12-26 17:10:27,512][105692] Updated weights for policy 0, policy_version 248619 (0.0005) [2023-12-26 17:10:27,534][105620] Updated weights for policy 1, policy_version 248981 (0.0007) [2023-12-26 17:10:27,579][105620] Updated weights for policy 1, policy_version 248991 (0.0008) [2023-12-26 17:10:28,201][105620] Updated weights for policy 1, policy_version 249001 (0.0009) [2023-12-26 17:10:28,217][105692] Updated weights for policy 0, policy_version 248629 (0.0006) [2023-12-26 17:10:28,261][105620] Updated weights for policy 1, policy_version 249011 (0.0008) [2023-12-26 17:10:28,276][105692] Updated weights for policy 0, policy_version 248639 (0.0005) [2023-12-26 17:10:28,321][105620] Updated weights for policy 1, policy_version 249021 (0.0009) [2023-12-26 17:10:28,338][105692] Updated weights for policy 0, policy_version 248649 (0.0006) [2023-12-26 17:10:28,380][105620] Updated weights for policy 1, policy_version 249031 (0.0007) [2023-12-26 17:10:28,987][105692] Updated weights for policy 0, policy_version 248659 (0.0008) [2023-12-26 17:10:29,048][105692] Updated weights for policy 0, policy_version 248669 (0.0008) [2023-12-26 17:10:29,091][105620] Updated weights for policy 1, policy_version 249041 (0.0006) [2023-12-26 17:10:29,108][105692] Updated weights for policy 0, policy_version 248679 (0.0006) [2023-12-26 17:10:29,154][105620] Updated weights for policy 1, policy_version 249051 (0.0006) [2023-12-26 17:10:29,201][105620] Updated weights for policy 1, policy_version 249061 (0.0005) [2023-12-26 17:10:29,832][105692] Updated weights for policy 0, policy_version 248689 (0.0008) [2023-12-26 17:10:29,894][105692] Updated weights for policy 0, policy_version 248699 (0.0007) [2023-12-26 17:10:29,946][105620] Updated weights for policy 1, policy_version 249071 (0.0008) [2023-12-26 17:10:29,965][105692] Updated weights for policy 0, policy_version 248709 (0.0007) [2023-12-26 17:10:29,998][105620] Updated weights for policy 1, policy_version 249081 (0.0008) [2023-12-26 17:10:30,023][105692] Updated weights for policy 0, policy_version 248719 (0.0010) [2023-12-26 17:10:30,057][105620] Updated weights for policy 1, policy_version 249091 (0.0008) [2023-12-26 17:10:30,628][105692] Updated weights for policy 0, policy_version 248729 (0.0006) [2023-12-26 17:10:30,677][105692] Updated weights for policy 0, policy_version 248739 (0.0006) [2023-12-26 17:10:30,699][105620] Updated weights for policy 1, policy_version 249101 (0.0010) [2023-12-26 17:10:30,731][105692] Updated weights for policy 0, policy_version 248749 (0.0007) [2023-12-26 17:10:30,752][105620] Updated weights for policy 1, policy_version 249111 (0.0005) [2023-12-26 17:10:30,809][105620] Updated weights for policy 1, policy_version 249121 (0.0005) [2023-12-26 17:10:31,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.7, 300 sec: 19605.2). Total num frames: 127475712. Throughput: 0: 9735.9, 1: 10052.7. Samples: 127442160. Policy #0 lag: (min: 15.0, avg: 16.7, max: 47.0) [2023-12-26 17:10:31,063][104569] Avg episode reward: [(0, '9357.123'), (1, '6842.400')] [2023-12-26 17:10:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000248752_63692800.pth... [2023-12-26 17:10:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000249128_63782912.pth... [2023-12-26 17:10:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000247944_63479808.pth [2023-12-26 17:10:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000247600_63397888.pth [2023-12-26 17:10:31,455][105692] Updated weights for policy 0, policy_version 248759 (0.0009) [2023-12-26 17:10:31,469][105620] Updated weights for policy 1, policy_version 249131 (0.0006) [2023-12-26 17:10:31,496][105586] KL-divergence is very high: 103.8521 [2023-12-26 17:10:31,513][105692] Updated weights for policy 0, policy_version 248769 (0.0010) [2023-12-26 17:10:31,523][105620] Updated weights for policy 1, policy_version 249141 (0.0006) [2023-12-26 17:10:31,540][105586] KL-divergence is very high: 171.6247 [2023-12-26 17:10:31,563][105692] Updated weights for policy 0, policy_version 248779 (0.0010) [2023-12-26 17:10:31,581][105620] Updated weights for policy 1, policy_version 249151 (0.0007) [2023-12-26 17:10:31,589][105586] KL-divergence is very high: 185.8029 [2023-12-26 17:10:32,220][105692] Updated weights for policy 0, policy_version 248789 (0.0007) [2023-12-26 17:10:32,277][105692] Updated weights for policy 0, policy_version 248799 (0.0009) [2023-12-26 17:10:32,305][105620] Updated weights for policy 1, policy_version 249161 (0.0008) [2023-12-26 17:10:32,342][105692] Updated weights for policy 0, policy_version 248809 (0.0007) [2023-12-26 17:10:32,371][105620] Updated weights for policy 1, policy_version 249171 (0.0010) [2023-12-26 17:10:32,429][105620] Updated weights for policy 1, policy_version 249181 (0.0007) [2023-12-26 17:10:32,483][105620] Updated weights for policy 1, policy_version 249191 (0.0005) [2023-12-26 17:10:32,913][105692] Updated weights for policy 0, policy_version 248819 (0.0007) [2023-12-26 17:10:32,959][105692] Updated weights for policy 0, policy_version 248829 (0.0005) [2023-12-26 17:10:33,004][105692] Updated weights for policy 0, policy_version 248839 (0.0010) [2023-12-26 17:10:33,196][105620] Updated weights for policy 1, policy_version 249201 (0.0010) [2023-12-26 17:10:33,265][105620] Updated weights for policy 1, policy_version 249211 (0.0009) [2023-12-26 17:10:33,331][105620] Updated weights for policy 1, policy_version 249221 (0.0009) [2023-12-26 17:10:33,631][105692] Updated weights for policy 0, policy_version 248849 (0.0010) [2023-12-26 17:10:33,690][105692] Updated weights for policy 0, policy_version 248859 (0.0006) [2023-12-26 17:10:33,746][105692] Updated weights for policy 0, policy_version 248869 (0.0006) [2023-12-26 17:10:33,799][105692] Updated weights for policy 0, policy_version 248879 (0.0005) [2023-12-26 17:10:34,002][105620] Updated weights for policy 1, policy_version 249231 (0.0007) [2023-12-26 17:10:34,055][105620] Updated weights for policy 1, policy_version 249241 (0.0005) [2023-12-26 17:10:34,113][105620] Updated weights for policy 1, policy_version 249251 (0.0009) [2023-12-26 17:10:34,365][105692] Updated weights for policy 0, policy_version 248889 (0.0010) [2023-12-26 17:10:34,426][105692] Updated weights for policy 0, policy_version 248899 (0.0010) [2023-12-26 17:10:34,485][105692] Updated weights for policy 0, policy_version 248909 (0.0009) [2023-12-26 17:10:34,819][105620] Updated weights for policy 1, policy_version 249261 (0.0009) [2023-12-26 17:10:34,880][105620] Updated weights for policy 1, policy_version 249271 (0.0008) [2023-12-26 17:10:34,941][105620] Updated weights for policy 1, policy_version 249281 (0.0008) [2023-12-26 17:10:35,268][105692] Updated weights for policy 0, policy_version 248919 (0.0009) [2023-12-26 17:10:35,325][105692] Updated weights for policy 0, policy_version 248929 (0.0009) [2023-12-26 17:10:35,378][105692] Updated weights for policy 0, policy_version 248939 (0.0009) [2023-12-26 17:10:35,649][105620] Updated weights for policy 1, policy_version 249291 (0.0009) [2023-12-26 17:10:35,702][105620] Updated weights for policy 1, policy_version 249301 (0.0012) [2023-12-26 17:10:35,757][105620] Updated weights for policy 1, policy_version 249311 (0.0007) [2023-12-26 17:10:36,002][105692] Updated weights for policy 0, policy_version 248949 (0.0007) [2023-12-26 17:10:36,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 127574016. Throughput: 0: 9846.0, 1: 10094.8. Samples: 127564612. Policy #0 lag: (min: 15.0, avg: 16.7, max: 47.0) [2023-12-26 17:10:36,062][104569] Avg episode reward: [(0, '9356.929'), (1, '7305.438')] [2023-12-26 17:10:36,069][105692] Updated weights for policy 0, policy_version 248959 (0.0006) [2023-12-26 17:10:36,141][105692] Updated weights for policy 0, policy_version 248969 (0.0008) [2023-12-26 17:10:36,528][105620] Updated weights for policy 1, policy_version 249321 (0.0009) [2023-12-26 17:10:36,584][105620] Updated weights for policy 1, policy_version 249331 (0.0009) [2023-12-26 17:10:36,642][105620] Updated weights for policy 1, policy_version 249341 (0.0009) [2023-12-26 17:10:36,699][105620] Updated weights for policy 1, policy_version 249351 (0.0009) [2023-12-26 17:10:36,703][105692] Updated weights for policy 0, policy_version 248979 (0.0006) [2023-12-26 17:10:36,756][105692] Updated weights for policy 0, policy_version 248989 (0.0005) [2023-12-26 17:10:36,818][105692] Updated weights for policy 0, policy_version 248999 (0.0007) [2023-12-26 17:10:37,466][105620] Updated weights for policy 1, policy_version 249361 (0.0006) [2023-12-26 17:10:37,519][105620] Updated weights for policy 1, policy_version 249371 (0.0007) [2023-12-26 17:10:37,580][105692] Updated weights for policy 0, policy_version 249009 (0.0009) [2023-12-26 17:10:37,582][105620] Updated weights for policy 1, policy_version 249381 (0.0009) [2023-12-26 17:10:37,636][105692] Updated weights for policy 0, policy_version 249019 (0.0009) [2023-12-26 17:10:37,698][105692] Updated weights for policy 0, policy_version 249029 (0.0010) [2023-12-26 17:10:37,757][105692] Updated weights for policy 0, policy_version 249039 (0.0011) [2023-12-26 17:10:38,355][105620] Updated weights for policy 1, policy_version 249391 (0.0006) [2023-12-26 17:10:38,388][105692] Updated weights for policy 0, policy_version 249049 (0.0009) [2023-12-26 17:10:38,415][105620] Updated weights for policy 1, policy_version 249401 (0.0006) [2023-12-26 17:10:38,446][105692] Updated weights for policy 0, policy_version 249059 (0.0008) [2023-12-26 17:10:38,473][105620] Updated weights for policy 1, policy_version 249411 (0.0009) [2023-12-26 17:10:38,506][105692] Updated weights for policy 0, policy_version 249069 (0.0006) [2023-12-26 17:10:39,178][105692] Updated weights for policy 0, policy_version 249079 (0.0006) [2023-12-26 17:10:39,233][105692] Updated weights for policy 0, policy_version 249089 (0.0008) [2023-12-26 17:10:39,264][105620] Updated weights for policy 1, policy_version 249421 (0.0008) [2023-12-26 17:10:39,292][105692] Updated weights for policy 0, policy_version 249099 (0.0008) [2023-12-26 17:10:39,322][105620] Updated weights for policy 1, policy_version 249431 (0.0008) [2023-12-26 17:10:39,380][105620] Updated weights for policy 1, policy_version 249441 (0.0008) [2023-12-26 17:10:40,089][105692] Updated weights for policy 0, policy_version 249109 (0.0007) [2023-12-26 17:10:40,102][105620] Updated weights for policy 1, policy_version 249451 (0.0008) [2023-12-26 17:10:40,149][105692] Updated weights for policy 0, policy_version 249119 (0.0006) [2023-12-26 17:10:40,158][105620] Updated weights for policy 1, policy_version 249461 (0.0008) [2023-12-26 17:10:40,199][105692] Updated weights for policy 0, policy_version 249129 (0.0006) [2023-12-26 17:10:40,206][105620] Updated weights for policy 1, policy_version 249471 (0.0008) [2023-12-26 17:10:40,928][105692] Updated weights for policy 0, policy_version 249139 (0.0009) [2023-12-26 17:10:40,972][105620] Updated weights for policy 1, policy_version 249481 (0.0008) [2023-12-26 17:10:40,993][105692] Updated weights for policy 0, policy_version 249149 (0.0010) [2023-12-26 17:10:41,035][105620] Updated weights for policy 1, policy_version 249491 (0.0007) [2023-12-26 17:10:41,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 127664128. Throughput: 0: 9894.4, 1: 10095.9. Samples: 127681020. Policy #0 lag: (min: 15.0, avg: 16.7, max: 47.0) [2023-12-26 17:10:41,063][104569] Avg episode reward: [(0, '9356.877'), (1, '7825.738')] [2023-12-26 17:10:41,069][105692] Updated weights for policy 0, policy_version 249159 (0.0009) [2023-12-26 17:10:41,101][105620] Updated weights for policy 1, policy_version 249501 (0.0007) [2023-12-26 17:10:41,173][105620] Updated weights for policy 1, policy_version 249511 (0.0009) [2023-12-26 17:10:41,847][105692] Updated weights for policy 0, policy_version 249169 (0.0008) [2023-12-26 17:10:41,910][105692] Updated weights for policy 0, policy_version 249179 (0.0009) [2023-12-26 17:10:41,960][105620] Updated weights for policy 1, policy_version 249521 (0.0009) [2023-12-26 17:10:41,967][105692] Updated weights for policy 0, policy_version 249189 (0.0008) [2023-12-26 17:10:42,020][105620] Updated weights for policy 1, policy_version 249531 (0.0007) [2023-12-26 17:10:42,026][105692] Updated weights for policy 0, policy_version 249199 (0.0008) [2023-12-26 17:10:42,073][105620] Updated weights for policy 1, policy_version 249541 (0.0008) [2023-12-26 17:10:42,783][105692] Updated weights for policy 0, policy_version 249209 (0.0009) [2023-12-26 17:10:42,831][105692] Updated weights for policy 0, policy_version 249219 (0.0007) [2023-12-26 17:10:42,840][105620] Updated weights for policy 1, policy_version 249551 (0.0009) [2023-12-26 17:10:42,881][105692] Updated weights for policy 0, policy_version 249229 (0.0009) [2023-12-26 17:10:42,901][105620] Updated weights for policy 1, policy_version 249561 (0.0009) [2023-12-26 17:10:42,962][105620] Updated weights for policy 1, policy_version 249571 (0.0008) [2023-12-26 17:10:43,640][105620] Updated weights for policy 1, policy_version 249581 (0.0008) [2023-12-26 17:10:43,687][105620] Updated weights for policy 1, policy_version 249591 (0.0008) [2023-12-26 17:10:43,689][105692] Updated weights for policy 0, policy_version 249239 (0.0008) [2023-12-26 17:10:43,747][105620] Updated weights for policy 1, policy_version 249601 (0.0007) [2023-12-26 17:10:43,750][105692] Updated weights for policy 0, policy_version 249249 (0.0006) [2023-12-26 17:10:43,803][105692] Updated weights for policy 0, policy_version 249259 (0.0006) [2023-12-26 17:10:44,422][105620] Updated weights for policy 1, policy_version 249611 (0.0007) [2023-12-26 17:10:44,482][105620] Updated weights for policy 1, policy_version 249621 (0.0005) [2023-12-26 17:10:44,545][105620] Updated weights for policy 1, policy_version 249631 (0.0007) [2023-12-26 17:10:44,607][105692] Updated weights for policy 0, policy_version 249269 (0.0008) [2023-12-26 17:10:44,654][105692] Updated weights for policy 0, policy_version 249279 (0.0009) [2023-12-26 17:10:44,701][105692] Updated weights for policy 0, policy_version 249289 (0.0006) [2023-12-26 17:10:45,134][105620] Updated weights for policy 1, policy_version 249641 (0.0008) [2023-12-26 17:10:45,198][105620] Updated weights for policy 1, policy_version 249651 (0.0006) [2023-12-26 17:10:45,254][105620] Updated weights for policy 1, policy_version 249661 (0.0008) [2023-12-26 17:10:45,307][105620] Updated weights for policy 1, policy_version 249671 (0.0008) [2023-12-26 17:10:45,487][105692] Updated weights for policy 0, policy_version 249299 (0.0009) [2023-12-26 17:10:45,539][105692] Updated weights for policy 0, policy_version 249309 (0.0009) [2023-12-26 17:10:45,598][105692] Updated weights for policy 0, policy_version 249319 (0.0009) [2023-12-26 17:10:46,002][105620] Updated weights for policy 1, policy_version 249681 (0.0008) [2023-12-26 17:10:46,050][105620] Updated weights for policy 1, policy_version 249691 (0.0010) [2023-12-26 17:10:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 127762432. Throughput: 0: 9822.9, 1: 10023.7. Samples: 127736100. Policy #0 lag: (min: 15.0, avg: 16.7, max: 47.0) [2023-12-26 17:10:46,062][104569] Avg episode reward: [(0, '9356.789'), (1, '8013.221')] [2023-12-26 17:10:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000249328_63840256.pth... [2023-12-26 17:10:46,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000248176_63545344.pth [2023-12-26 17:10:46,103][105620] Updated weights for policy 1, policy_version 249701 (0.0010) [2023-12-26 17:10:46,122][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000249704_63930368.pth... [2023-12-26 17:10:46,125][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000248520_63627264.pth [2023-12-26 17:10:46,435][105692] Updated weights for policy 0, policy_version 249329 (0.0009) [2023-12-26 17:10:46,493][105692] Updated weights for policy 0, policy_version 249339 (0.0010) [2023-12-26 17:10:46,552][105692] Updated weights for policy 0, policy_version 249349 (0.0010) [2023-12-26 17:10:46,609][105692] Updated weights for policy 0, policy_version 249359 (0.0010) [2023-12-26 17:10:46,699][105620] Updated weights for policy 1, policy_version 249711 (0.0007) [2023-12-26 17:10:46,750][105620] Updated weights for policy 1, policy_version 249721 (0.0005) [2023-12-26 17:10:46,797][105620] Updated weights for policy 1, policy_version 249731 (0.0005) [2023-12-26 17:10:47,298][105692] Updated weights for policy 0, policy_version 249369 (0.0008) [2023-12-26 17:10:47,347][105692] Updated weights for policy 0, policy_version 249379 (0.0006) [2023-12-26 17:10:47,398][105692] Updated weights for policy 0, policy_version 249389 (0.0005) [2023-12-26 17:10:47,400][105620] Updated weights for policy 1, policy_version 249741 (0.0008) [2023-12-26 17:10:47,455][105620] Updated weights for policy 1, policy_version 249751 (0.0010) [2023-12-26 17:10:47,508][105620] Updated weights for policy 1, policy_version 249761 (0.0010) [2023-12-26 17:10:47,989][105692] Updated weights for policy 0, policy_version 249399 (0.0009) [2023-12-26 17:10:48,037][105692] Updated weights for policy 0, policy_version 249409 (0.0010) [2023-12-26 17:10:48,085][105692] Updated weights for policy 0, policy_version 249419 (0.0010) [2023-12-26 17:10:48,268][105620] Updated weights for policy 1, policy_version 249771 (0.0010) [2023-12-26 17:10:48,323][105620] Updated weights for policy 1, policy_version 249781 (0.0009) [2023-12-26 17:10:48,390][105620] Updated weights for policy 1, policy_version 249791 (0.0009) [2023-12-26 17:10:48,843][105692] Updated weights for policy 0, policy_version 249429 (0.0010) [2023-12-26 17:10:48,892][105692] Updated weights for policy 0, policy_version 249439 (0.0010) [2023-12-26 17:10:48,940][105692] Updated weights for policy 0, policy_version 249449 (0.0010) [2023-12-26 17:10:49,158][105620] Updated weights for policy 1, policy_version 249801 (0.0008) [2023-12-26 17:10:49,219][105620] Updated weights for policy 1, policy_version 249811 (0.0008) [2023-12-26 17:10:49,282][105620] Updated weights for policy 1, policy_version 249821 (0.0008) [2023-12-26 17:10:49,345][105620] Updated weights for policy 1, policy_version 249831 (0.0008) [2023-12-26 17:10:49,691][105692] Updated weights for policy 0, policy_version 249459 (0.0009) [2023-12-26 17:10:49,750][105692] Updated weights for policy 0, policy_version 249469 (0.0011) [2023-12-26 17:10:49,816][105692] Updated weights for policy 0, policy_version 249479 (0.0010) [2023-12-26 17:10:50,095][105620] Updated weights for policy 1, policy_version 249841 (0.0009) [2023-12-26 17:10:50,151][105620] Updated weights for policy 1, policy_version 249851 (0.0008) [2023-12-26 17:10:50,219][105620] Updated weights for policy 1, policy_version 249861 (0.0008) [2023-12-26 17:10:50,569][105692] Updated weights for policy 0, policy_version 249489 (0.0011) [2023-12-26 17:10:50,631][105692] Updated weights for policy 0, policy_version 249499 (0.0011) [2023-12-26 17:10:50,687][105692] Updated weights for policy 0, policy_version 249509 (0.0011) [2023-12-26 17:10:50,738][105692] Updated weights for policy 0, policy_version 249519 (0.0010) [2023-12-26 17:10:50,962][105620] Updated weights for policy 1, policy_version 249871 (0.0007) [2023-12-26 17:10:51,029][105620] Updated weights for policy 1, policy_version 249881 (0.0008) [2023-12-26 17:10:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 127860736. Throughput: 0: 9809.7, 1: 9936.4. Samples: 127854760. Policy #0 lag: (min: 15.0, avg: 16.7, max: 47.0) [2023-12-26 17:10:51,062][104569] Avg episode reward: [(0, '9356.816'), (1, '7423.931')] [2023-12-26 17:10:51,088][105620] Updated weights for policy 1, policy_version 249891 (0.0008) [2023-12-26 17:10:51,494][105692] Updated weights for policy 0, policy_version 249529 (0.0009) [2023-12-26 17:10:51,550][105692] Updated weights for policy 0, policy_version 249539 (0.0011) [2023-12-26 17:10:51,606][105692] Updated weights for policy 0, policy_version 249549 (0.0011) [2023-12-26 17:10:51,830][105620] Updated weights for policy 1, policy_version 249901 (0.0008) [2023-12-26 17:10:51,894][105620] Updated weights for policy 1, policy_version 249911 (0.0008) [2023-12-26 17:10:51,965][105620] Updated weights for policy 1, policy_version 249921 (0.0007) [2023-12-26 17:10:52,380][105692] Updated weights for policy 0, policy_version 249559 (0.0009) [2023-12-26 17:10:52,431][105692] Updated weights for policy 0, policy_version 249569 (0.0008) [2023-12-26 17:10:52,479][105692] Updated weights for policy 0, policy_version 249579 (0.0008) [2023-12-26 17:10:52,660][105620] Updated weights for policy 1, policy_version 249931 (0.0008) [2023-12-26 17:10:52,709][105620] Updated weights for policy 1, policy_version 249941 (0.0008) [2023-12-26 17:10:52,769][105620] Updated weights for policy 1, policy_version 249951 (0.0008) [2023-12-26 17:10:53,243][105692] Updated weights for policy 0, policy_version 249589 (0.0007) [2023-12-26 17:10:53,289][105692] Updated weights for policy 0, policy_version 249599 (0.0005) [2023-12-26 17:10:53,335][105692] Updated weights for policy 0, policy_version 249609 (0.0005) [2023-12-26 17:10:53,450][105620] Updated weights for policy 1, policy_version 249961 (0.0009) [2023-12-26 17:10:53,509][105620] Updated weights for policy 1, policy_version 249971 (0.0005) [2023-12-26 17:10:53,578][105620] Updated weights for policy 1, policy_version 249981 (0.0005) [2023-12-26 17:10:53,627][105620] Updated weights for policy 1, policy_version 249991 (0.0005) [2023-12-26 17:10:53,994][105692] Updated weights for policy 0, policy_version 249619 (0.0005) [2023-12-26 17:10:54,057][105692] Updated weights for policy 0, policy_version 249629 (0.0006) [2023-12-26 17:10:54,124][105692] Updated weights for policy 0, policy_version 249639 (0.0005) [2023-12-26 17:10:54,142][105620] Updated weights for policy 1, policy_version 250001 (0.0008) [2023-12-26 17:10:54,200][105620] Updated weights for policy 1, policy_version 250011 (0.0009) [2023-12-26 17:10:54,251][105620] Updated weights for policy 1, policy_version 250021 (0.0008) [2023-12-26 17:10:54,646][105692] Updated weights for policy 0, policy_version 249649 (0.0006) [2023-12-26 17:10:54,697][105692] Updated weights for policy 0, policy_version 249659 (0.0010) [2023-12-26 17:10:54,746][105692] Updated weights for policy 0, policy_version 249669 (0.0009) [2023-12-26 17:10:54,807][105692] Updated weights for policy 0, policy_version 249679 (0.0007) [2023-12-26 17:10:54,863][105620] Updated weights for policy 1, policy_version 250031 (0.0009) [2023-12-26 17:10:54,920][105620] Updated weights for policy 1, policy_version 250041 (0.0009) [2023-12-26 17:10:54,980][105620] Updated weights for policy 1, policy_version 250051 (0.0007) [2023-12-26 17:10:55,413][105692] Updated weights for policy 0, policy_version 249689 (0.0007) [2023-12-26 17:10:55,484][105692] Updated weights for policy 0, policy_version 249699 (0.0006) [2023-12-26 17:10:55,538][105692] Updated weights for policy 0, policy_version 249709 (0.0005) [2023-12-26 17:10:55,619][105620] Updated weights for policy 1, policy_version 250061 (0.0008) [2023-12-26 17:10:55,668][105620] Updated weights for policy 1, policy_version 250071 (0.0009) [2023-12-26 17:10:55,726][105620] Updated weights for policy 1, policy_version 250081 (0.0005) [2023-12-26 17:10:56,054][105692] Updated weights for policy 0, policy_version 249719 (0.0007) [2023-12-26 17:10:56,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19933.8, 300 sec: 19633.0). Total num frames: 127967232. Throughput: 0: 9922.6, 1: 9968.2. Samples: 127977696. Policy #0 lag: (min: 2.0, avg: 20.2, max: 34.0) [2023-12-26 17:10:56,063][104569] Avg episode reward: [(0, '9356.783'), (1, '8311.586')] [2023-12-26 17:10:56,125][105692] Updated weights for policy 0, policy_version 249729 (0.0008) [2023-12-26 17:10:56,197][105692] Updated weights for policy 0, policy_version 249739 (0.0007) [2023-12-26 17:10:56,299][105620] Updated weights for policy 1, policy_version 250091 (0.0005) [2023-12-26 17:10:56,361][105620] Updated weights for policy 1, policy_version 250101 (0.0005) [2023-12-26 17:10:56,423][105620] Updated weights for policy 1, policy_version 250111 (0.0005) [2023-12-26 17:10:56,826][105692] Updated weights for policy 0, policy_version 249749 (0.0005) [2023-12-26 17:10:56,886][105692] Updated weights for policy 0, policy_version 249759 (0.0005) [2023-12-26 17:10:56,944][105692] Updated weights for policy 0, policy_version 249769 (0.0005) [2023-12-26 17:10:56,963][105620] Updated weights for policy 1, policy_version 250121 (0.0006) [2023-12-26 17:10:57,021][105620] Updated weights for policy 1, policy_version 250131 (0.0010) [2023-12-26 17:10:57,079][105620] Updated weights for policy 1, policy_version 250141 (0.0010) [2023-12-26 17:10:57,149][105620] Updated weights for policy 1, policy_version 250151 (0.0006) [2023-12-26 17:10:57,539][105692] Updated weights for policy 0, policy_version 249779 (0.0007) [2023-12-26 17:10:57,590][105692] Updated weights for policy 0, policy_version 249789 (0.0010) [2023-12-26 17:10:57,648][105692] Updated weights for policy 0, policy_version 249799 (0.0010) [2023-12-26 17:10:57,857][105620] Updated weights for policy 1, policy_version 250161 (0.0010) [2023-12-26 17:10:57,903][105620] Updated weights for policy 1, policy_version 250171 (0.0008) [2023-12-26 17:10:57,952][105620] Updated weights for policy 1, policy_version 250181 (0.0008) [2023-12-26 17:10:58,330][105692] Updated weights for policy 0, policy_version 249809 (0.0010) [2023-12-26 17:10:58,396][105692] Updated weights for policy 0, policy_version 249819 (0.0010) [2023-12-26 17:10:58,459][105692] Updated weights for policy 0, policy_version 249829 (0.0010) [2023-12-26 17:10:58,524][105692] Updated weights for policy 0, policy_version 249839 (0.0010) [2023-12-26 17:10:58,833][105620] Updated weights for policy 1, policy_version 250191 (0.0008) [2023-12-26 17:10:58,901][105620] Updated weights for policy 1, policy_version 250201 (0.0008) [2023-12-26 17:10:58,964][105620] Updated weights for policy 1, policy_version 250211 (0.0008) [2023-12-26 17:10:59,262][105692] Updated weights for policy 0, policy_version 249849 (0.0011) [2023-12-26 17:10:59,316][105692] Updated weights for policy 0, policy_version 249859 (0.0009) [2023-12-26 17:10:59,376][105692] Updated weights for policy 0, policy_version 249869 (0.0011) [2023-12-26 17:10:59,740][105620] Updated weights for policy 1, policy_version 250221 (0.0009) [2023-12-26 17:10:59,802][105620] Updated weights for policy 1, policy_version 250231 (0.0010) [2023-12-26 17:10:59,866][105620] Updated weights for policy 1, policy_version 250241 (0.0011) [2023-12-26 17:11:00,103][105692] Updated weights for policy 0, policy_version 249879 (0.0008) [2023-12-26 17:11:00,168][105692] Updated weights for policy 0, policy_version 249889 (0.0006) [2023-12-26 17:11:00,231][105692] Updated weights for policy 0, policy_version 249899 (0.0006) [2023-12-26 17:11:00,574][105620] Updated weights for policy 1, policy_version 250251 (0.0010) [2023-12-26 17:11:00,642][105620] Updated weights for policy 1, policy_version 250261 (0.0010) [2023-12-26 17:11:00,703][105620] Updated weights for policy 1, policy_version 250271 (0.0010) [2023-12-26 17:11:00,842][105692] Updated weights for policy 0, policy_version 249909 (0.0005) [2023-12-26 17:11:00,887][105692] Updated weights for policy 0, policy_version 249919 (0.0005) [2023-12-26 17:11:00,943][105692] Updated weights for policy 0, policy_version 249929 (0.0005) [2023-12-26 17:11:01,062][104569] Fps is (10 sec: 21299.1, 60 sec: 20070.4, 300 sec: 19660.8). Total num frames: 128073728. Throughput: 0: 9961.9, 1: 10036.1. Samples: 128040260. Policy #0 lag: (min: 2.0, avg: 20.2, max: 34.0) [2023-12-26 17:11:01,063][104569] Avg episode reward: [(0, '9356.793'), (1, '8530.369')] [2023-12-26 17:11:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000249936_63995904.pth... [2023-12-26 17:11:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000250280_64077824.pth... [2023-12-26 17:11:01,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000248752_63692800.pth [2023-12-26 17:11:01,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000249128_63782912.pth [2023-12-26 17:11:01,308][105620] Updated weights for policy 1, policy_version 250281 (0.0010) [2023-12-26 17:11:01,370][105620] Updated weights for policy 1, policy_version 250291 (0.0006) [2023-12-26 17:11:01,433][105620] Updated weights for policy 1, policy_version 250301 (0.0006) [2023-12-26 17:11:01,490][105620] Updated weights for policy 1, policy_version 250311 (0.0005) [2023-12-26 17:11:01,623][105692] Updated weights for policy 0, policy_version 249939 (0.0008) [2023-12-26 17:11:01,688][105692] Updated weights for policy 0, policy_version 249949 (0.0009) [2023-12-26 17:11:01,751][105692] Updated weights for policy 0, policy_version 249959 (0.0009) [2023-12-26 17:11:02,130][105620] Updated weights for policy 1, policy_version 250321 (0.0006) [2023-12-26 17:11:02,173][105620] Updated weights for policy 1, policy_version 250331 (0.0005) [2023-12-26 17:11:02,217][105620] Updated weights for policy 1, policy_version 250341 (0.0005) [2023-12-26 17:11:02,580][105692] Updated weights for policy 0, policy_version 249969 (0.0009) [2023-12-26 17:11:02,627][105692] Updated weights for policy 0, policy_version 249979 (0.0009) [2023-12-26 17:11:02,681][105692] Updated weights for policy 0, policy_version 249989 (0.0009) [2023-12-26 17:11:02,735][105692] Updated weights for policy 0, policy_version 249999 (0.0009) [2023-12-26 17:11:02,929][105620] Updated weights for policy 1, policy_version 250351 (0.0005) [2023-12-26 17:11:02,983][105620] Updated weights for policy 1, policy_version 250361 (0.0005) [2023-12-26 17:11:03,034][105620] Updated weights for policy 1, policy_version 250371 (0.0005) [2023-12-26 17:11:03,531][105692] Updated weights for policy 0, policy_version 250009 (0.0009) [2023-12-26 17:11:03,577][105692] Updated weights for policy 0, policy_version 250019 (0.0009) [2023-12-26 17:11:03,630][105692] Updated weights for policy 0, policy_version 250029 (0.0009) [2023-12-26 17:11:03,708][105620] Updated weights for policy 1, policy_version 250381 (0.0007) [2023-12-26 17:11:03,763][105620] Updated weights for policy 1, policy_version 250391 (0.0009) [2023-12-26 17:11:03,813][105620] Updated weights for policy 1, policy_version 250401 (0.0009) [2023-12-26 17:11:04,408][105692] Updated weights for policy 0, policy_version 250039 (0.0009) [2023-12-26 17:11:04,471][105692] Updated weights for policy 0, policy_version 250049 (0.0009) [2023-12-26 17:11:04,529][105692] Updated weights for policy 0, policy_version 250059 (0.0008) [2023-12-26 17:11:04,605][105620] Updated weights for policy 1, policy_version 250411 (0.0008) [2023-12-26 17:11:04,680][105620] Updated weights for policy 1, policy_version 250421 (0.0010) [2023-12-26 17:11:04,739][105620] Updated weights for policy 1, policy_version 250431 (0.0009) [2023-12-26 17:11:05,223][105692] Updated weights for policy 0, policy_version 250069 (0.0007) [2023-12-26 17:11:05,269][105692] Updated weights for policy 0, policy_version 250079 (0.0008) [2023-12-26 17:11:05,315][105692] Updated weights for policy 0, policy_version 250089 (0.0009) [2023-12-26 17:11:05,481][105620] Updated weights for policy 1, policy_version 250441 (0.0009) [2023-12-26 17:11:05,550][105620] Updated weights for policy 1, policy_version 250451 (0.0008) [2023-12-26 17:11:05,619][105620] Updated weights for policy 1, policy_version 250461 (0.0008) [2023-12-26 17:11:05,684][105620] Updated weights for policy 1, policy_version 250471 (0.0009) [2023-12-26 17:11:06,006][105692] Updated weights for policy 0, policy_version 250099 (0.0009) [2023-12-26 17:11:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 128163840. Throughput: 0: 9865.9, 1: 9987.1. Samples: 128156340. Policy #0 lag: (min: 2.0, avg: 20.2, max: 34.0) [2023-12-26 17:11:06,063][104569] Avg episode reward: [(0, '9356.850'), (1, '8713.477')] [2023-12-26 17:11:06,067][105692] Updated weights for policy 0, policy_version 250109 (0.0008) [2023-12-26 17:11:06,129][105692] Updated weights for policy 0, policy_version 250119 (0.0007) [2023-12-26 17:11:06,449][105620] Updated weights for policy 1, policy_version 250481 (0.0008) [2023-12-26 17:11:06,512][105620] Updated weights for policy 1, policy_version 250491 (0.0008) [2023-12-26 17:11:06,567][105620] Updated weights for policy 1, policy_version 250501 (0.0008) [2023-12-26 17:11:06,873][105692] Updated weights for policy 0, policy_version 250129 (0.0008) [2023-12-26 17:11:06,929][105692] Updated weights for policy 0, policy_version 250139 (0.0010) [2023-12-26 17:11:06,978][105692] Updated weights for policy 0, policy_version 250149 (0.0010) [2023-12-26 17:11:07,030][105692] Updated weights for policy 0, policy_version 250159 (0.0011) [2023-12-26 17:11:07,222][105620] Updated weights for policy 1, policy_version 250511 (0.0008) [2023-12-26 17:11:07,271][105620] Updated weights for policy 1, policy_version 250521 (0.0005) [2023-12-26 17:11:07,333][105620] Updated weights for policy 1, policy_version 250531 (0.0005) [2023-12-26 17:11:07,727][105692] Updated weights for policy 0, policy_version 250169 (0.0006) [2023-12-26 17:11:07,777][105692] Updated weights for policy 0, policy_version 250179 (0.0005) [2023-12-26 17:11:07,840][105692] Updated weights for policy 0, policy_version 250189 (0.0005) [2023-12-26 17:11:07,978][105620] Updated weights for policy 1, policy_version 250541 (0.0008) [2023-12-26 17:11:08,030][105620] Updated weights for policy 1, policy_version 250551 (0.0010) [2023-12-26 17:11:08,085][105620] Updated weights for policy 1, policy_version 250561 (0.0010) [2023-12-26 17:11:08,508][105692] Updated weights for policy 0, policy_version 250199 (0.0008) [2023-12-26 17:11:08,567][105692] Updated weights for policy 0, policy_version 250209 (0.0008) [2023-12-26 17:11:08,626][105692] Updated weights for policy 0, policy_version 250219 (0.0008) [2023-12-26 17:11:08,844][105620] Updated weights for policy 1, policy_version 250571 (0.0010) [2023-12-26 17:11:08,910][105620] Updated weights for policy 1, policy_version 250581 (0.0010) [2023-12-26 17:11:08,966][105620] Updated weights for policy 1, policy_version 250591 (0.0011) [2023-12-26 17:11:09,304][105692] Updated weights for policy 0, policy_version 250229 (0.0007) [2023-12-26 17:11:09,375][105692] Updated weights for policy 0, policy_version 250239 (0.0007) [2023-12-26 17:11:09,444][105692] Updated weights for policy 0, policy_version 250249 (0.0009) [2023-12-26 17:11:09,682][105620] Updated weights for policy 1, policy_version 250601 (0.0011) [2023-12-26 17:11:09,732][105620] Updated weights for policy 1, policy_version 250611 (0.0009) [2023-12-26 17:11:09,780][105620] Updated weights for policy 1, policy_version 250621 (0.0008) [2023-12-26 17:11:09,844][105620] Updated weights for policy 1, policy_version 250631 (0.0008) [2023-12-26 17:11:10,121][105692] Updated weights for policy 0, policy_version 250259 (0.0008) [2023-12-26 17:11:10,184][105692] Updated weights for policy 0, policy_version 250269 (0.0009) [2023-12-26 17:11:10,247][105692] Updated weights for policy 0, policy_version 250279 (0.0009) [2023-12-26 17:11:10,644][105620] Updated weights for policy 1, policy_version 250641 (0.0010) [2023-12-26 17:11:10,696][105620] Updated weights for policy 1, policy_version 250651 (0.0010) [2023-12-26 17:11:10,749][105620] Updated weights for policy 1, policy_version 250661 (0.0005) [2023-12-26 17:11:10,874][105692] Updated weights for policy 0, policy_version 250289 (0.0009) [2023-12-26 17:11:10,939][105692] Updated weights for policy 0, policy_version 250299 (0.0006) [2023-12-26 17:11:10,992][105692] Updated weights for policy 0, policy_version 250309 (0.0006) [2023-12-26 17:11:11,056][105692] Updated weights for policy 0, policy_version 250319 (0.0008) [2023-12-26 17:11:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19933.9, 300 sec: 19660.8). Total num frames: 128270336. Throughput: 0: 9938.3, 1: 9879.6. Samples: 128274600. Policy #0 lag: (min: 2.0, avg: 20.2, max: 34.0) [2023-12-26 17:11:11,062][104569] Avg episode reward: [(0, '9357.035'), (1, '8255.115')] [2023-12-26 17:11:11,539][105620] Updated weights for policy 1, policy_version 250671 (0.0007) [2023-12-26 17:11:11,611][105620] Updated weights for policy 1, policy_version 250681 (0.0006) [2023-12-26 17:11:11,681][105620] Updated weights for policy 1, policy_version 250691 (0.0009) [2023-12-26 17:11:11,844][105692] Updated weights for policy 0, policy_version 250329 (0.0008) [2023-12-26 17:11:11,914][105692] Updated weights for policy 0, policy_version 250339 (0.0006) [2023-12-26 17:11:11,986][105692] Updated weights for policy 0, policy_version 250349 (0.0006) [2023-12-26 17:11:12,344][105620] Updated weights for policy 1, policy_version 250701 (0.0008) [2023-12-26 17:11:12,408][105620] Updated weights for policy 1, policy_version 250711 (0.0007) [2023-12-26 17:11:12,476][105620] Updated weights for policy 1, policy_version 250721 (0.0009) [2023-12-26 17:11:12,587][105692] Updated weights for policy 0, policy_version 250359 (0.0007) [2023-12-26 17:11:12,647][105692] Updated weights for policy 0, policy_version 250369 (0.0008) [2023-12-26 17:11:12,703][105692] Updated weights for policy 0, policy_version 250379 (0.0008) [2023-12-26 17:11:13,241][105620] Updated weights for policy 1, policy_version 250731 (0.0010) [2023-12-26 17:11:13,289][105620] Updated weights for policy 1, policy_version 250741 (0.0010) [2023-12-26 17:11:13,341][105620] Updated weights for policy 1, policy_version 250751 (0.0010) [2023-12-26 17:11:13,471][105692] Updated weights for policy 0, policy_version 250389 (0.0008) [2023-12-26 17:11:13,518][105692] Updated weights for policy 0, policy_version 250399 (0.0008) [2023-12-26 17:11:13,564][105692] Updated weights for policy 0, policy_version 250409 (0.0008) [2023-12-26 17:11:14,094][105620] Updated weights for policy 1, policy_version 250761 (0.0010) [2023-12-26 17:11:14,154][105620] Updated weights for policy 1, policy_version 250771 (0.0008) [2023-12-26 17:11:14,208][105620] Updated weights for policy 1, policy_version 250781 (0.0008) [2023-12-26 17:11:14,269][105620] Updated weights for policy 1, policy_version 250791 (0.0008) [2023-12-26 17:11:14,337][105692] Updated weights for policy 0, policy_version 250419 (0.0008) [2023-12-26 17:11:14,399][105692] Updated weights for policy 0, policy_version 250429 (0.0009) [2023-12-26 17:11:14,455][105692] Updated weights for policy 0, policy_version 250439 (0.0009) [2023-12-26 17:11:14,848][105620] Updated weights for policy 1, policy_version 250801 (0.0008) [2023-12-26 17:11:14,898][105620] Updated weights for policy 1, policy_version 250811 (0.0008) [2023-12-26 17:11:14,959][105620] Updated weights for policy 1, policy_version 250821 (0.0009) [2023-12-26 17:11:15,258][105692] Updated weights for policy 0, policy_version 250449 (0.0009) [2023-12-26 17:11:15,313][105692] Updated weights for policy 0, policy_version 250459 (0.0009) [2023-12-26 17:11:15,372][105692] Updated weights for policy 0, policy_version 250469 (0.0009) [2023-12-26 17:11:15,427][105692] Updated weights for policy 0, policy_version 250479 (0.0009) [2023-12-26 17:11:15,677][105620] Updated weights for policy 1, policy_version 250831 (0.0006) [2023-12-26 17:11:15,728][105620] Updated weights for policy 1, policy_version 250841 (0.0005) [2023-12-26 17:11:15,776][105620] Updated weights for policy 1, policy_version 250851 (0.0005) [2023-12-26 17:11:16,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 128360448. Throughput: 0: 9943.7, 1: 9818.6. Samples: 128331456. Policy #0 lag: (min: 2.0, avg: 20.2, max: 34.0) [2023-12-26 17:11:16,062][104569] Avg episode reward: [(0, '9268.525'), (1, '7889.149')] [2023-12-26 17:11:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000250856_64225280.pth... [2023-12-26 17:11:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000250480_64135168.pth... [2023-12-26 17:11:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000249704_63930368.pth [2023-12-26 17:11:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000249328_63840256.pth [2023-12-26 17:11:16,249][105692] Updated weights for policy 0, policy_version 250489 (0.0009) [2023-12-26 17:11:16,299][105692] Updated weights for policy 0, policy_version 250499 (0.0009) [2023-12-26 17:11:16,356][105692] Updated weights for policy 0, policy_version 250509 (0.0010) [2023-12-26 17:11:16,403][105620] Updated weights for policy 1, policy_version 250861 (0.0005) [2023-12-26 17:11:16,455][105620] Updated weights for policy 1, policy_version 250871 (0.0007) [2023-12-26 17:11:16,513][105620] Updated weights for policy 1, policy_version 250881 (0.0009) [2023-12-26 17:11:17,131][105692] Updated weights for policy 0, policy_version 250519 (0.0009) [2023-12-26 17:11:17,193][105692] Updated weights for policy 0, policy_version 250529 (0.0009) [2023-12-26 17:11:17,216][105620] Updated weights for policy 1, policy_version 250891 (0.0008) [2023-12-26 17:11:17,254][105692] Updated weights for policy 0, policy_version 250539 (0.0009) [2023-12-26 17:11:17,268][105620] Updated weights for policy 1, policy_version 250901 (0.0006) [2023-12-26 17:11:17,320][105620] Updated weights for policy 1, policy_version 250911 (0.0008) [2023-12-26 17:11:17,949][105692] Updated weights for policy 0, policy_version 250549 (0.0008) [2023-12-26 17:11:17,960][105620] Updated weights for policy 1, policy_version 250921 (0.0008) [2023-12-26 17:11:17,998][105692] Updated weights for policy 0, policy_version 250559 (0.0009) [2023-12-26 17:11:18,011][105620] Updated weights for policy 1, policy_version 250931 (0.0005) [2023-12-26 17:11:18,050][105692] Updated weights for policy 0, policy_version 250569 (0.0007) [2023-12-26 17:11:18,064][105620] Updated weights for policy 1, policy_version 250941 (0.0007) [2023-12-26 17:11:18,119][105620] Updated weights for policy 1, policy_version 250951 (0.0008) [2023-12-26 17:11:18,751][105692] Updated weights for policy 0, policy_version 250579 (0.0007) [2023-12-26 17:11:18,812][105692] Updated weights for policy 0, policy_version 250589 (0.0005) [2023-12-26 17:11:18,863][105692] Updated weights for policy 0, policy_version 250599 (0.0007) [2023-12-26 17:11:18,905][105620] Updated weights for policy 1, policy_version 250961 (0.0006) [2023-12-26 17:11:18,973][105620] Updated weights for policy 1, policy_version 250971 (0.0008) [2023-12-26 17:11:19,031][105620] Updated weights for policy 1, policy_version 250981 (0.0009) [2023-12-26 17:11:19,587][105692] Updated weights for policy 0, policy_version 250609 (0.0007) [2023-12-26 17:11:19,647][105692] Updated weights for policy 0, policy_version 250619 (0.0008) [2023-12-26 17:11:19,720][105692] Updated weights for policy 0, policy_version 250629 (0.0008) [2023-12-26 17:11:19,786][105620] Updated weights for policy 1, policy_version 250991 (0.0007) [2023-12-26 17:11:19,787][105692] Updated weights for policy 0, policy_version 250639 (0.0007) [2023-12-26 17:11:19,853][105620] Updated weights for policy 1, policy_version 251001 (0.0008) [2023-12-26 17:11:19,915][105620] Updated weights for policy 1, policy_version 251011 (0.0009) [2023-12-26 17:11:20,530][105692] Updated weights for policy 0, policy_version 250649 (0.0009) [2023-12-26 17:11:20,586][105692] Updated weights for policy 0, policy_version 250659 (0.0009) [2023-12-26 17:11:20,601][105620] Updated weights for policy 1, policy_version 251021 (0.0009) [2023-12-26 17:11:20,652][105692] Updated weights for policy 0, policy_version 250669 (0.0008) [2023-12-26 17:11:20,662][105620] Updated weights for policy 1, policy_version 251031 (0.0007) [2023-12-26 17:11:20,729][105620] Updated weights for policy 1, policy_version 251041 (0.0009) [2023-12-26 17:11:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 128458752. Throughput: 0: 9790.1, 1: 9844.2. Samples: 128448156. Policy #0 lag: (min: 2.0, avg: 20.2, max: 34.0) [2023-12-26 17:11:21,063][104569] Avg episode reward: [(0, '9268.562'), (1, '7888.817')] [2023-12-26 17:11:21,457][105692] Updated weights for policy 0, policy_version 250679 (0.0006) [2023-12-26 17:11:21,523][105692] Updated weights for policy 0, policy_version 250689 (0.0006) [2023-12-26 17:11:21,538][105620] Updated weights for policy 1, policy_version 251051 (0.0008) [2023-12-26 17:11:21,588][105692] Updated weights for policy 0, policy_version 250699 (0.0011) [2023-12-26 17:11:21,601][105620] Updated weights for policy 1, policy_version 251061 (0.0006) [2023-12-26 17:11:21,660][105620] Updated weights for policy 1, policy_version 251071 (0.0007) [2023-12-26 17:11:22,253][105692] Updated weights for policy 0, policy_version 250709 (0.0010) [2023-12-26 17:11:22,314][105692] Updated weights for policy 0, policy_version 250719 (0.0006) [2023-12-26 17:11:22,377][105692] Updated weights for policy 0, policy_version 250729 (0.0006) [2023-12-26 17:11:22,493][105620] Updated weights for policy 1, policy_version 251081 (0.0008) [2023-12-26 17:11:22,544][105620] Updated weights for policy 1, policy_version 251091 (0.0010) [2023-12-26 17:11:22,605][105620] Updated weights for policy 1, policy_version 251101 (0.0009) [2023-12-26 17:11:22,671][105620] Updated weights for policy 1, policy_version 251111 (0.0010) [2023-12-26 17:11:23,000][105692] Updated weights for policy 0, policy_version 250739 (0.0008) [2023-12-26 17:11:23,065][105692] Updated weights for policy 0, policy_version 250749 (0.0006) [2023-12-26 17:11:23,130][105692] Updated weights for policy 0, policy_version 250759 (0.0007) [2023-12-26 17:11:23,498][105620] Updated weights for policy 1, policy_version 251121 (0.0009) [2023-12-26 17:11:23,571][105620] Updated weights for policy 1, policy_version 251131 (0.0010) [2023-12-26 17:11:23,637][105620] Updated weights for policy 1, policy_version 251141 (0.0010) [2023-12-26 17:11:23,732][105692] Updated weights for policy 0, policy_version 250769 (0.0007) [2023-12-26 17:11:23,783][105692] Updated weights for policy 0, policy_version 250779 (0.0008) [2023-12-26 17:11:23,845][105692] Updated weights for policy 0, policy_version 250789 (0.0006) [2023-12-26 17:11:23,903][105692] Updated weights for policy 0, policy_version 250799 (0.0005) [2023-12-26 17:11:24,434][105620] Updated weights for policy 1, policy_version 251151 (0.0009) [2023-12-26 17:11:24,485][105620] Updated weights for policy 1, policy_version 251161 (0.0009) [2023-12-26 17:11:24,528][105692] Updated weights for policy 0, policy_version 250809 (0.0008) [2023-12-26 17:11:24,538][105620] Updated weights for policy 1, policy_version 251171 (0.0008) [2023-12-26 17:11:24,585][105692] Updated weights for policy 0, policy_version 250819 (0.0009) [2023-12-26 17:11:24,632][105692] Updated weights for policy 0, policy_version 250829 (0.0009) [2023-12-26 17:11:25,258][105620] Updated weights for policy 1, policy_version 251181 (0.0008) [2023-12-26 17:11:25,325][105620] Updated weights for policy 1, policy_version 251191 (0.0009) [2023-12-26 17:11:25,379][105620] Updated weights for policy 1, policy_version 251201 (0.0008) [2023-12-26 17:11:25,401][105692] Updated weights for policy 0, policy_version 250839 (0.0008) [2023-12-26 17:11:25,449][105692] Updated weights for policy 0, policy_version 250849 (0.0008) [2023-12-26 17:11:25,498][105692] Updated weights for policy 0, policy_version 250859 (0.0008) [2023-12-26 17:11:26,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 128548864. Throughput: 0: 9757.1, 1: 9796.0. Samples: 128560912. Policy #0 lag: (min: 2.0, avg: 20.2, max: 34.0) [2023-12-26 17:11:26,063][104569] Avg episode reward: [(0, '9357.456'), (1, '8072.004')] [2023-12-26 17:11:26,164][105620] Updated weights for policy 1, policy_version 251211 (0.0007) [2023-12-26 17:11:26,198][105692] Updated weights for policy 0, policy_version 250869 (0.0009) [2023-12-26 17:11:26,213][105620] Updated weights for policy 1, policy_version 251221 (0.0006) [2023-12-26 17:11:26,251][105692] Updated weights for policy 0, policy_version 250879 (0.0008) [2023-12-26 17:11:26,265][105620] Updated weights for policy 1, policy_version 251231 (0.0006) [2023-12-26 17:11:26,304][105692] Updated weights for policy 0, policy_version 250889 (0.0007) [2023-12-26 17:11:26,990][105692] Updated weights for policy 0, policy_version 250899 (0.0006) [2023-12-26 17:11:27,053][105692] Updated weights for policy 0, policy_version 250909 (0.0005) [2023-12-26 17:11:27,080][105620] Updated weights for policy 1, policy_version 251241 (0.0007) [2023-12-26 17:11:27,114][105692] Updated weights for policy 0, policy_version 250919 (0.0008) [2023-12-26 17:11:27,133][105620] Updated weights for policy 1, policy_version 251251 (0.0006) [2023-12-26 17:11:27,187][105620] Updated weights for policy 1, policy_version 251261 (0.0009) [2023-12-26 17:11:27,241][105620] Updated weights for policy 1, policy_version 251271 (0.0008) [2023-12-26 17:11:27,720][105692] Updated weights for policy 0, policy_version 250929 (0.0008) [2023-12-26 17:11:27,780][105692] Updated weights for policy 0, policy_version 250939 (0.0009) [2023-12-26 17:11:27,830][105692] Updated weights for policy 0, policy_version 250949 (0.0009) [2023-12-26 17:11:27,871][105585] KL-divergence is very high: 433.9917 [2023-12-26 17:11:27,885][105692] Updated weights for policy 0, policy_version 250959 (0.0008) [2023-12-26 17:11:28,025][105620] Updated weights for policy 1, policy_version 251281 (0.0009) [2023-12-26 17:11:28,073][105620] Updated weights for policy 1, policy_version 251291 (0.0009) [2023-12-26 17:11:28,120][105620] Updated weights for policy 1, policy_version 251301 (0.0008) [2023-12-26 17:11:28,643][105692] Updated weights for policy 0, policy_version 250969 (0.0009) [2023-12-26 17:11:28,707][105692] Updated weights for policy 0, policy_version 250979 (0.0009) [2023-12-26 17:11:28,767][105692] Updated weights for policy 0, policy_version 250989 (0.0009) [2023-12-26 17:11:28,869][105620] Updated weights for policy 1, policy_version 251311 (0.0009) [2023-12-26 17:11:28,922][105620] Updated weights for policy 1, policy_version 251321 (0.0009) [2023-12-26 17:11:28,973][105620] Updated weights for policy 1, policy_version 251331 (0.0013) [2023-12-26 17:11:29,509][105692] Updated weights for policy 0, policy_version 250999 (0.0009) [2023-12-26 17:11:29,536][105585] KL-divergence is very high: 391.0721 [2023-12-26 17:11:29,567][105692] Updated weights for policy 0, policy_version 251009 (0.0009) [2023-12-26 17:11:29,585][105585] KL-divergence is very high: 585.2793 [2023-12-26 17:11:29,625][105692] Updated weights for policy 0, policy_version 251019 (0.0009) [2023-12-26 17:11:29,632][105585] KL-divergence is very high: 540.7512 [2023-12-26 17:11:29,765][105620] Updated weights for policy 1, policy_version 251341 (0.0009) [2023-12-26 17:11:29,829][105620] Updated weights for policy 1, policy_version 251351 (0.0009) [2023-12-26 17:11:29,882][105620] Updated weights for policy 1, policy_version 251361 (0.0008) [2023-12-26 17:11:30,395][105692] Updated weights for policy 0, policy_version 251029 (0.0009) [2023-12-26 17:11:30,454][105692] Updated weights for policy 0, policy_version 251039 (0.0009) [2023-12-26 17:11:30,515][105692] Updated weights for policy 0, policy_version 251049 (0.0009) [2023-12-26 17:11:30,628][105620] Updated weights for policy 1, policy_version 251371 (0.0008) [2023-12-26 17:11:30,681][105620] Updated weights for policy 1, policy_version 251381 (0.0009) [2023-12-26 17:11:30,734][105620] Updated weights for policy 1, policy_version 251391 (0.0008) [2023-12-26 17:11:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 128647168. Throughput: 0: 9835.8, 1: 9787.0. Samples: 128619128. Policy #0 lag: (min: 2.0, avg: 20.2, max: 34.0) [2023-12-26 17:11:31,063][104569] Avg episode reward: [(0, '9176.374'), (1, '8070.066')] [2023-12-26 17:11:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000251056_64282624.pth... [2023-12-26 17:11:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000251400_64364544.pth... [2023-12-26 17:11:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000249936_63995904.pth [2023-12-26 17:11:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000250280_64077824.pth [2023-12-26 17:11:31,208][105692] Updated weights for policy 0, policy_version 251059 (0.0008) [2023-12-26 17:11:31,268][105692] Updated weights for policy 0, policy_version 251069 (0.0007) [2023-12-26 17:11:31,329][105692] Updated weights for policy 0, policy_version 251079 (0.0007) [2023-12-26 17:11:31,443][105620] Updated weights for policy 1, policy_version 251401 (0.0009) [2023-12-26 17:11:31,498][105620] Updated weights for policy 1, policy_version 251411 (0.0011) [2023-12-26 17:11:31,559][105620] Updated weights for policy 1, policy_version 251421 (0.0006) [2023-12-26 17:11:31,619][105620] Updated weights for policy 1, policy_version 251431 (0.0008) [2023-12-26 17:11:31,977][105692] Updated weights for policy 0, policy_version 251089 (0.0007) [2023-12-26 17:11:32,026][105692] Updated weights for policy 0, policy_version 251099 (0.0008) [2023-12-26 17:11:32,076][105692] Updated weights for policy 0, policy_version 251109 (0.0009) [2023-12-26 17:11:32,133][105692] Updated weights for policy 0, policy_version 251119 (0.0008) [2023-12-26 17:11:32,315][105620] Updated weights for policy 1, policy_version 251441 (0.0009) [2023-12-26 17:11:32,373][105620] Updated weights for policy 1, policy_version 251451 (0.0009) [2023-12-26 17:11:32,427][105620] Updated weights for policy 1, policy_version 251461 (0.0008) [2023-12-26 17:11:32,934][105692] Updated weights for policy 0, policy_version 251129 (0.0009) [2023-12-26 17:11:32,998][105692] Updated weights for policy 0, policy_version 251139 (0.0007) [2023-12-26 17:11:33,055][105692] Updated weights for policy 0, policy_version 251149 (0.0009) [2023-12-26 17:11:33,202][105620] Updated weights for policy 1, policy_version 251471 (0.0009) [2023-12-26 17:11:33,260][105620] Updated weights for policy 1, policy_version 251481 (0.0009) [2023-12-26 17:11:33,306][105620] Updated weights for policy 1, policy_version 251491 (0.0008) [2023-12-26 17:11:33,797][105692] Updated weights for policy 0, policy_version 251159 (0.0009) [2023-12-26 17:11:33,844][105692] Updated weights for policy 0, policy_version 251169 (0.0008) [2023-12-26 17:11:33,887][105692] Updated weights for policy 0, policy_version 251179 (0.0007) [2023-12-26 17:11:34,030][105620] Updated weights for policy 1, policy_version 251501 (0.0010) [2023-12-26 17:11:34,081][105620] Updated weights for policy 1, policy_version 251511 (0.0005) [2023-12-26 17:11:34,127][105620] Updated weights for policy 1, policy_version 251521 (0.0005) [2023-12-26 17:11:34,730][105620] Updated weights for policy 1, policy_version 251531 (0.0009) [2023-12-26 17:11:34,732][105692] Updated weights for policy 0, policy_version 251189 (0.0007) [2023-12-26 17:11:34,795][105692] Updated weights for policy 0, policy_version 251199 (0.0006) [2023-12-26 17:11:34,795][105620] Updated weights for policy 1, policy_version 251541 (0.0011) [2023-12-26 17:11:34,855][105620] Updated weights for policy 1, policy_version 251551 (0.0010) [2023-12-26 17:11:34,860][105692] Updated weights for policy 0, policy_version 251209 (0.0006) [2023-12-26 17:11:35,580][105620] Updated weights for policy 1, policy_version 251561 (0.0009) [2023-12-26 17:11:35,595][105692] Updated weights for policy 0, policy_version 251219 (0.0006) [2023-12-26 17:11:35,645][105620] Updated weights for policy 1, policy_version 251571 (0.0005) [2023-12-26 17:11:35,654][105692] Updated weights for policy 0, policy_version 251229 (0.0005) [2023-12-26 17:11:35,705][105620] Updated weights for policy 1, policy_version 251581 (0.0005) [2023-12-26 17:11:35,714][105692] Updated weights for policy 0, policy_version 251239 (0.0006) [2023-12-26 17:11:35,757][105620] Updated weights for policy 1, policy_version 251591 (0.0010) [2023-12-26 17:11:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 128745472. Throughput: 0: 9833.3, 1: 9710.1. Samples: 128734212. Policy #0 lag: (min: 2.0, avg: 20.2, max: 34.0) [2023-12-26 17:11:36,063][104569] Avg episode reward: [(0, '9266.152'), (1, '8535.788')] [2023-12-26 17:11:36,375][105620] Updated weights for policy 1, policy_version 251601 (0.0010) [2023-12-26 17:11:36,438][105620] Updated weights for policy 1, policy_version 251611 (0.0011) [2023-12-26 17:11:36,457][105692] Updated weights for policy 0, policy_version 251249 (0.0007) [2023-12-26 17:11:36,505][105620] Updated weights for policy 1, policy_version 251621 (0.0009) [2023-12-26 17:11:36,523][105692] Updated weights for policy 0, policy_version 251259 (0.0008) [2023-12-26 17:11:36,593][105692] Updated weights for policy 0, policy_version 251269 (0.0009) [2023-12-26 17:11:36,669][105692] Updated weights for policy 0, policy_version 251279 (0.0010) [2023-12-26 17:11:37,183][105620] Updated weights for policy 1, policy_version 251631 (0.0010) [2023-12-26 17:11:37,250][105620] Updated weights for policy 1, policy_version 251641 (0.0011) [2023-12-26 17:11:37,305][105620] Updated weights for policy 1, policy_version 251651 (0.0010) [2023-12-26 17:11:37,405][105692] Updated weights for policy 0, policy_version 251289 (0.0008) [2023-12-26 17:11:37,451][105692] Updated weights for policy 0, policy_version 251299 (0.0008) [2023-12-26 17:11:37,504][105692] Updated weights for policy 0, policy_version 251309 (0.0009) [2023-12-26 17:11:37,898][105620] Updated weights for policy 1, policy_version 251661 (0.0008) [2023-12-26 17:11:37,955][105620] Updated weights for policy 1, policy_version 251671 (0.0005) [2023-12-26 17:11:38,003][105620] Updated weights for policy 1, policy_version 251681 (0.0005) [2023-12-26 17:11:38,339][105692] Updated weights for policy 0, policy_version 251319 (0.0010) [2023-12-26 17:11:38,399][105692] Updated weights for policy 0, policy_version 251329 (0.0010) [2023-12-26 17:11:38,462][105692] Updated weights for policy 0, policy_version 251340 (0.0010) [2023-12-26 17:11:38,611][105620] Updated weights for policy 1, policy_version 251691 (0.0008) [2023-12-26 17:11:38,661][105620] Updated weights for policy 1, policy_version 251701 (0.0005) [2023-12-26 17:11:38,710][105620] Updated weights for policy 1, policy_version 251711 (0.0006) [2023-12-26 17:11:39,237][105692] Updated weights for policy 0, policy_version 251350 (0.0008) [2023-12-26 17:11:39,304][105692] Updated weights for policy 0, policy_version 251360 (0.0008) [2023-12-26 17:11:39,366][105692] Updated weights for policy 0, policy_version 251370 (0.0008) [2023-12-26 17:11:39,400][105620] Updated weights for policy 1, policy_version 251721 (0.0006) [2023-12-26 17:11:39,470][105620] Updated weights for policy 1, policy_version 251731 (0.0008) [2023-12-26 17:11:39,521][105620] Updated weights for policy 1, policy_version 251741 (0.0009) [2023-12-26 17:11:39,584][105620] Updated weights for policy 1, policy_version 251751 (0.0009) [2023-12-26 17:11:40,116][105692] Updated weights for policy 0, policy_version 251380 (0.0008) [2023-12-26 17:11:40,177][105692] Updated weights for policy 0, policy_version 251390 (0.0008) [2023-12-26 17:11:40,229][105692] Updated weights for policy 0, policy_version 251400 (0.0009) [2023-12-26 17:11:40,348][105620] Updated weights for policy 1, policy_version 251761 (0.0007) [2023-12-26 17:11:40,401][105620] Updated weights for policy 1, policy_version 251771 (0.0008) [2023-12-26 17:11:40,451][105620] Updated weights for policy 1, policy_version 251781 (0.0007) [2023-12-26 17:11:40,892][105692] Updated weights for policy 0, policy_version 251410 (0.0008) [2023-12-26 17:11:40,952][105692] Updated weights for policy 0, policy_version 251420 (0.0008) [2023-12-26 17:11:40,999][105692] Updated weights for policy 0, policy_version 251430 (0.0009) [2023-12-26 17:11:41,059][105692] Updated weights for policy 0, policy_version 251440 (0.0008) [2023-12-26 17:11:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 128843776. Throughput: 0: 9683.8, 1: 9681.0. Samples: 128849108. Policy #0 lag: (min: 2.0, avg: 20.2, max: 34.0) [2023-12-26 17:11:41,062][104569] Avg episode reward: [(0, '9266.176'), (1, '8625.527')] [2023-12-26 17:11:41,291][105620] Updated weights for policy 1, policy_version 251791 (0.0008) [2023-12-26 17:11:41,374][105620] Updated weights for policy 1, policy_version 251801 (0.0008) [2023-12-26 17:11:41,428][105620] Updated weights for policy 1, policy_version 251811 (0.0008) [2023-12-26 17:11:41,864][105692] Updated weights for policy 0, policy_version 251450 (0.0009) [2023-12-26 17:11:41,928][105692] Updated weights for policy 0, policy_version 251460 (0.0009) [2023-12-26 17:11:41,991][105692] Updated weights for policy 0, policy_version 251470 (0.0010) [2023-12-26 17:11:42,186][105620] Updated weights for policy 1, policy_version 251821 (0.0008) [2023-12-26 17:11:42,249][105620] Updated weights for policy 1, policy_version 251831 (0.0009) [2023-12-26 17:11:42,320][105620] Updated weights for policy 1, policy_version 251841 (0.0009) [2023-12-26 17:11:42,815][105692] Updated weights for policy 0, policy_version 251480 (0.0009) [2023-12-26 17:11:42,874][105692] Updated weights for policy 0, policy_version 251490 (0.0008) [2023-12-26 17:11:42,926][105692] Updated weights for policy 0, policy_version 251500 (0.0007) [2023-12-26 17:11:42,995][105620] Updated weights for policy 1, policy_version 251851 (0.0010) [2023-12-26 17:11:43,047][105620] Updated weights for policy 1, policy_version 251861 (0.0010) [2023-12-26 17:11:43,106][105620] Updated weights for policy 1, policy_version 251871 (0.0010) [2023-12-26 17:11:43,712][105620] Updated weights for policy 1, policy_version 251881 (0.0006) [2023-12-26 17:11:43,746][105692] Updated weights for policy 0, policy_version 251510 (0.0005) [2023-12-26 17:11:43,764][105620] Updated weights for policy 1, policy_version 251891 (0.0010) [2023-12-26 17:11:43,806][105692] Updated weights for policy 0, policy_version 251520 (0.0008) [2023-12-26 17:11:43,826][105620] Updated weights for policy 1, policy_version 251902 (0.0009) [2023-12-26 17:11:43,868][105692] Updated weights for policy 0, policy_version 251530 (0.0009) [2023-12-26 17:11:43,887][105620] Updated weights for policy 1, policy_version 251912 (0.0007) [2023-12-26 17:11:44,485][105620] Updated weights for policy 1, policy_version 251922 (0.0007) [2023-12-26 17:11:44,547][105620] Updated weights for policy 1, policy_version 251932 (0.0006) [2023-12-26 17:11:44,559][105692] Updated weights for policy 0, policy_version 251540 (0.0006) [2023-12-26 17:11:44,603][105620] Updated weights for policy 1, policy_version 251942 (0.0005) [2023-12-26 17:11:44,622][105692] Updated weights for policy 0, policy_version 251550 (0.0005) [2023-12-26 17:11:44,674][105692] Updated weights for policy 0, policy_version 251560 (0.0005) [2023-12-26 17:11:45,218][105620] Updated weights for policy 1, policy_version 251952 (0.0006) [2023-12-26 17:11:45,283][105620] Updated weights for policy 1, policy_version 251962 (0.0008) [2023-12-26 17:11:45,331][105692] Updated weights for policy 0, policy_version 251570 (0.0007) [2023-12-26 17:11:45,349][105620] Updated weights for policy 1, policy_version 251972 (0.0009) [2023-12-26 17:11:45,381][105692] Updated weights for policy 0, policy_version 251580 (0.0011) [2023-12-26 17:11:45,430][105692] Updated weights for policy 0, policy_version 251590 (0.0010) [2023-12-26 17:11:45,475][105692] Updated weights for policy 0, policy_version 251600 (0.0010) [2023-12-26 17:11:45,992][105620] Updated weights for policy 1, policy_version 251982 (0.0008) [2023-12-26 17:11:46,049][105620] Updated weights for policy 1, policy_version 251992 (0.0009) [2023-12-26 17:11:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.2, 300 sec: 19605.2). Total num frames: 128933888. Throughput: 0: 9573.9, 1: 9662.5. Samples: 128905900. Policy #0 lag: (min: 2.0, avg: 20.2, max: 34.0) [2023-12-26 17:11:46,063][104569] Avg episode reward: [(0, '9357.452'), (1, '8620.588')] [2023-12-26 17:11:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000251600_64421888.pth... [2023-12-26 17:11:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000250480_64135168.pth [2023-12-26 17:11:46,106][105620] Updated weights for policy 1, policy_version 252002 (0.0008) [2023-12-26 17:11:46,138][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000252008_64520192.pth... [2023-12-26 17:11:46,142][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000250856_64225280.pth [2023-12-26 17:11:46,270][105692] Updated weights for policy 0, policy_version 251610 (0.0009) [2023-12-26 17:11:46,327][105692] Updated weights for policy 0, policy_version 251621 (0.0010) [2023-12-26 17:11:46,393][105692] Updated weights for policy 0, policy_version 251631 (0.0006) [2023-12-26 17:11:46,836][105620] Updated weights for policy 1, policy_version 252012 (0.0007) [2023-12-26 17:11:46,896][105620] Updated weights for policy 1, policy_version 252023 (0.0009) [2023-12-26 17:11:46,959][105620] Updated weights for policy 1, policy_version 252033 (0.0008) [2023-12-26 17:11:47,043][105692] Updated weights for policy 0, policy_version 251641 (0.0006) [2023-12-26 17:11:47,101][105692] Updated weights for policy 0, policy_version 251651 (0.0006) [2023-12-26 17:11:47,160][105692] Updated weights for policy 0, policy_version 251661 (0.0006) [2023-12-26 17:11:47,702][105620] Updated weights for policy 1, policy_version 252043 (0.0008) [2023-12-26 17:11:47,761][105620] Updated weights for policy 1, policy_version 252053 (0.0008) [2023-12-26 17:11:47,820][105620] Updated weights for policy 1, policy_version 252063 (0.0009) [2023-12-26 17:11:47,864][105692] Updated weights for policy 0, policy_version 251671 (0.0006) [2023-12-26 17:11:47,926][105692] Updated weights for policy 0, policy_version 251681 (0.0005) [2023-12-26 17:11:47,990][105692] Updated weights for policy 0, policy_version 251691 (0.0010) [2023-12-26 17:11:48,541][105620] Updated weights for policy 1, policy_version 252073 (0.0009) [2023-12-26 17:11:48,601][105620] Updated weights for policy 1, policy_version 252083 (0.0006) [2023-12-26 17:11:48,622][105692] Updated weights for policy 0, policy_version 251701 (0.0009) [2023-12-26 17:11:48,661][105620] Updated weights for policy 1, policy_version 252093 (0.0006) [2023-12-26 17:11:48,688][105692] Updated weights for policy 0, policy_version 251711 (0.0011) [2023-12-26 17:11:48,721][105620] Updated weights for policy 1, policy_version 252103 (0.0005) [2023-12-26 17:11:48,751][105692] Updated weights for policy 0, policy_version 251721 (0.0011) [2023-12-26 17:11:49,297][105620] Updated weights for policy 1, policy_version 252113 (0.0010) [2023-12-26 17:11:49,362][105620] Updated weights for policy 1, policy_version 252123 (0.0008) [2023-12-26 17:11:49,423][105620] Updated weights for policy 1, policy_version 252133 (0.0011) [2023-12-26 17:11:49,494][105692] Updated weights for policy 0, policy_version 251731 (0.0010) [2023-12-26 17:11:49,551][105692] Updated weights for policy 0, policy_version 251741 (0.0008) [2023-12-26 17:11:49,608][105692] Updated weights for policy 0, policy_version 251751 (0.0008) [2023-12-26 17:11:50,163][105620] Updated weights for policy 1, policy_version 252143 (0.0010) [2023-12-26 17:11:50,215][105620] Updated weights for policy 1, policy_version 252153 (0.0010) [2023-12-26 17:11:50,269][105620] Updated weights for policy 1, policy_version 252163 (0.0010) [2023-12-26 17:11:50,381][105692] Updated weights for policy 0, policy_version 251761 (0.0009) [2023-12-26 17:11:50,434][105692] Updated weights for policy 0, policy_version 251771 (0.0008) [2023-12-26 17:11:50,490][105692] Updated weights for policy 0, policy_version 251781 (0.0008) [2023-12-26 17:11:50,555][105692] Updated weights for policy 0, policy_version 251791 (0.0008) [2023-12-26 17:11:50,996][105620] Updated weights for policy 1, policy_version 252173 (0.0008) [2023-12-26 17:11:51,059][105620] Updated weights for policy 1, policy_version 252183 (0.0007) [2023-12-26 17:11:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 129032192. Throughput: 0: 9629.3, 1: 9728.5. Samples: 129027440. Policy #0 lag: (min: 2.0, avg: 20.2, max: 34.0) [2023-12-26 17:11:51,062][104569] Avg episode reward: [(0, '9357.629'), (1, '8988.603')] [2023-12-26 17:11:51,128][105620] Updated weights for policy 1, policy_version 252193 (0.0007) [2023-12-26 17:11:51,334][105692] Updated weights for policy 0, policy_version 251801 (0.0008) [2023-12-26 17:11:51,406][105692] Updated weights for policy 0, policy_version 251811 (0.0009) [2023-12-26 17:11:51,464][105692] Updated weights for policy 0, policy_version 251821 (0.0009) [2023-12-26 17:11:51,811][105620] Updated weights for policy 1, policy_version 252203 (0.0008) [2023-12-26 17:11:51,860][105620] Updated weights for policy 1, policy_version 252213 (0.0007) [2023-12-26 17:11:51,911][105620] Updated weights for policy 1, policy_version 252223 (0.0009) [2023-12-26 17:11:52,283][105692] Updated weights for policy 0, policy_version 251831 (0.0010) [2023-12-26 17:11:52,351][105692] Updated weights for policy 0, policy_version 251841 (0.0009) [2023-12-26 17:11:52,421][105692] Updated weights for policy 0, policy_version 251851 (0.0009) [2023-12-26 17:11:52,692][105620] Updated weights for policy 1, policy_version 252233 (0.0009) [2023-12-26 17:11:52,759][105620] Updated weights for policy 1, policy_version 252243 (0.0011) [2023-12-26 17:11:52,816][105620] Updated weights for policy 1, policy_version 252253 (0.0011) [2023-12-26 17:11:52,881][105620] Updated weights for policy 1, policy_version 252263 (0.0007) [2023-12-26 17:11:53,121][105692] Updated weights for policy 0, policy_version 251861 (0.0007) [2023-12-26 17:11:53,181][105692] Updated weights for policy 0, policy_version 251871 (0.0005) [2023-12-26 17:11:53,226][105692] Updated weights for policy 0, policy_version 251881 (0.0005) [2023-12-26 17:11:53,544][105620] Updated weights for policy 1, policy_version 252273 (0.0010) [2023-12-26 17:11:53,599][105620] Updated weights for policy 1, policy_version 252283 (0.0010) [2023-12-26 17:11:53,648][105620] Updated weights for policy 1, policy_version 252293 (0.0010) [2023-12-26 17:11:53,767][105692] Updated weights for policy 0, policy_version 251891 (0.0005) [2023-12-26 17:11:53,822][105692] Updated weights for policy 0, policy_version 251901 (0.0005) [2023-12-26 17:11:53,875][105692] Updated weights for policy 0, policy_version 251911 (0.0005) [2023-12-26 17:11:54,337][105620] Updated weights for policy 1, policy_version 252303 (0.0010) [2023-12-26 17:11:54,391][105620] Updated weights for policy 1, policy_version 252313 (0.0011) [2023-12-26 17:11:54,450][105620] Updated weights for policy 1, policy_version 252323 (0.0011) [2023-12-26 17:11:54,501][105692] Updated weights for policy 0, policy_version 251921 (0.0006) [2023-12-26 17:11:54,550][105692] Updated weights for policy 0, policy_version 251931 (0.0007) [2023-12-26 17:11:54,601][105692] Updated weights for policy 0, policy_version 251941 (0.0009) [2023-12-26 17:11:54,652][105692] Updated weights for policy 0, policy_version 251951 (0.0007) [2023-12-26 17:11:55,220][105620] Updated weights for policy 1, policy_version 252333 (0.0011) [2023-12-26 17:11:55,269][105620] Updated weights for policy 1, policy_version 252343 (0.0010) [2023-12-26 17:11:55,325][105620] Updated weights for policy 1, policy_version 252353 (0.0010) [2023-12-26 17:11:55,328][105692] Updated weights for policy 0, policy_version 251961 (0.0006) [2023-12-26 17:11:55,372][105692] Updated weights for policy 0, policy_version 251971 (0.0005) [2023-12-26 17:11:55,418][105692] Updated weights for policy 0, policy_version 251981 (0.0005) [2023-12-26 17:11:56,048][105620] Updated weights for policy 1, policy_version 252363 (0.0009) [2023-12-26 17:11:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 129130496. Throughput: 0: 9616.1, 1: 9724.4. Samples: 129144924. Policy #0 lag: (min: 2.0, avg: 20.2, max: 34.0) [2023-12-26 17:11:56,063][104569] Avg episode reward: [(0, '9357.492'), (1, '9088.779')] [2023-12-26 17:11:56,081][105692] Updated weights for policy 0, policy_version 251991 (0.0005) [2023-12-26 17:11:56,102][105620] Updated weights for policy 1, policy_version 252373 (0.0007) [2023-12-26 17:11:56,147][105692] Updated weights for policy 0, policy_version 252001 (0.0009) [2023-12-26 17:11:56,153][105620] Updated weights for policy 1, policy_version 252383 (0.0010) [2023-12-26 17:11:56,211][105692] Updated weights for policy 0, policy_version 252011 (0.0008) [2023-12-26 17:11:56,808][105692] Updated weights for policy 0, policy_version 252021 (0.0007) [2023-12-26 17:11:56,847][105620] Updated weights for policy 1, policy_version 252393 (0.0010) [2023-12-26 17:11:56,858][105692] Updated weights for policy 0, policy_version 252031 (0.0005) [2023-12-26 17:11:56,898][105620] Updated weights for policy 1, policy_version 252403 (0.0007) [2023-12-26 17:11:56,910][105692] Updated weights for policy 0, policy_version 252041 (0.0005) [2023-12-26 17:11:56,944][105620] Updated weights for policy 1, policy_version 252413 (0.0005) [2023-12-26 17:11:56,998][105620] Updated weights for policy 1, policy_version 252423 (0.0005) [2023-12-26 17:11:57,448][105692] Updated weights for policy 0, policy_version 252051 (0.0005) [2023-12-26 17:11:57,498][105692] Updated weights for policy 0, policy_version 252061 (0.0005) [2023-12-26 17:11:57,540][105620] Updated weights for policy 1, policy_version 252433 (0.0005) [2023-12-26 17:11:57,546][105692] Updated weights for policy 0, policy_version 252071 (0.0005) [2023-12-26 17:11:57,602][105620] Updated weights for policy 1, policy_version 252443 (0.0005) [2023-12-26 17:11:57,661][105620] Updated weights for policy 1, policy_version 252453 (0.0005) [2023-12-26 17:11:58,074][105692] Updated weights for policy 0, policy_version 252081 (0.0005) [2023-12-26 17:11:58,142][105692] Updated weights for policy 0, policy_version 252091 (0.0007) [2023-12-26 17:11:58,210][105692] Updated weights for policy 0, policy_version 252101 (0.0007) [2023-12-26 17:11:58,228][105620] Updated weights for policy 1, policy_version 252463 (0.0009) [2023-12-26 17:11:58,266][105692] Updated weights for policy 0, policy_version 252111 (0.0006) [2023-12-26 17:11:58,288][105620] Updated weights for policy 1, policy_version 252473 (0.0010) [2023-12-26 17:11:58,365][105620] Updated weights for policy 1, policy_version 252483 (0.0009) [2023-12-26 17:11:59,023][105692] Updated weights for policy 0, policy_version 252121 (0.0006) [2023-12-26 17:11:59,084][105692] Updated weights for policy 0, policy_version 252131 (0.0006) [2023-12-26 17:11:59,156][105692] Updated weights for policy 0, policy_version 252141 (0.0007) [2023-12-26 17:11:59,222][105620] Updated weights for policy 1, policy_version 252493 (0.0011) [2023-12-26 17:11:59,283][105620] Updated weights for policy 1, policy_version 252503 (0.0008) [2023-12-26 17:11:59,358][105620] Updated weights for policy 1, policy_version 252513 (0.0008) [2023-12-26 17:11:59,918][105692] Updated weights for policy 0, policy_version 252151 (0.0010) [2023-12-26 17:11:59,979][105692] Updated weights for policy 0, policy_version 252161 (0.0010) [2023-12-26 17:11:59,996][105620] Updated weights for policy 1, policy_version 252523 (0.0008) [2023-12-26 17:12:00,058][105620] Updated weights for policy 1, policy_version 252533 (0.0009) [2023-12-26 17:12:00,060][105692] Updated weights for policy 0, policy_version 252171 (0.0006) [2023-12-26 17:12:00,117][105620] Updated weights for policy 1, policy_version 252543 (0.0007) [2023-12-26 17:12:00,640][105692] Updated weights for policy 0, policy_version 252181 (0.0008) [2023-12-26 17:12:00,694][105692] Updated weights for policy 0, policy_version 252191 (0.0010) [2023-12-26 17:12:00,742][105692] Updated weights for policy 0, policy_version 252201 (0.0010) [2023-12-26 17:12:00,864][105620] Updated weights for policy 1, policy_version 252553 (0.0008) [2023-12-26 17:12:00,920][105620] Updated weights for policy 1, policy_version 252563 (0.0008) [2023-12-26 17:12:00,986][105620] Updated weights for policy 1, policy_version 252573 (0.0006) [2023-12-26 17:12:01,047][105620] Updated weights for policy 1, policy_version 252583 (0.0007) [2023-12-26 17:12:01,062][104569] Fps is (10 sec: 21298.9, 60 sec: 19524.2, 300 sec: 19688.6). Total num frames: 129245184. Throughput: 0: 9726.3, 1: 9815.5. Samples: 129210840. Policy #0 lag: (min: 2.0, avg: 20.2, max: 34.0) [2023-12-26 17:12:01,063][104569] Avg episode reward: [(0, '9357.059'), (1, '8644.318')] [2023-12-26 17:12:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000252208_64577536.pth... [2023-12-26 17:12:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000252584_64667648.pth... [2023-12-26 17:12:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000251056_64282624.pth [2023-12-26 17:12:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000251400_64364544.pth [2023-12-26 17:12:01,524][105692] Updated weights for policy 0, policy_version 252211 (0.0010) [2023-12-26 17:12:01,582][105692] Updated weights for policy 0, policy_version 252221 (0.0007) [2023-12-26 17:12:01,641][105692] Updated weights for policy 0, policy_version 252231 (0.0006) [2023-12-26 17:12:01,740][105620] Updated weights for policy 1, policy_version 252593 (0.0007) [2023-12-26 17:12:01,795][105620] Updated weights for policy 1, policy_version 252603 (0.0005) [2023-12-26 17:12:01,854][105620] Updated weights for policy 1, policy_version 252613 (0.0005) [2023-12-26 17:12:02,239][105692] Updated weights for policy 0, policy_version 252241 (0.0007) [2023-12-26 17:12:02,301][105692] Updated weights for policy 0, policy_version 252251 (0.0006) [2023-12-26 17:12:02,358][105692] Updated weights for policy 0, policy_version 252261 (0.0010) [2023-12-26 17:12:02,417][105692] Updated weights for policy 0, policy_version 252271 (0.0010) [2023-12-26 17:12:02,463][105620] Updated weights for policy 1, policy_version 252623 (0.0008) [2023-12-26 17:12:02,527][105620] Updated weights for policy 1, policy_version 252633 (0.0008) [2023-12-26 17:12:02,579][105620] Updated weights for policy 1, policy_version 252643 (0.0005) [2023-12-26 17:12:03,056][105692] Updated weights for policy 0, policy_version 252281 (0.0009) [2023-12-26 17:12:03,104][105692] Updated weights for policy 0, policy_version 252291 (0.0005) [2023-12-26 17:12:03,151][105692] Updated weights for policy 0, policy_version 252301 (0.0005) [2023-12-26 17:12:03,289][105620] Updated weights for policy 1, policy_version 252653 (0.0006) [2023-12-26 17:12:03,344][105620] Updated weights for policy 1, policy_version 252663 (0.0005) [2023-12-26 17:12:03,400][105620] Updated weights for policy 1, policy_version 252673 (0.0010) [2023-12-26 17:12:03,679][105692] Updated weights for policy 0, policy_version 252311 (0.0005) [2023-12-26 17:12:03,742][105692] Updated weights for policy 0, policy_version 252321 (0.0007) [2023-12-26 17:12:03,798][105692] Updated weights for policy 0, policy_version 252331 (0.0010) [2023-12-26 17:12:04,045][105620] Updated weights for policy 1, policy_version 252683 (0.0011) [2023-12-26 17:12:04,102][105620] Updated weights for policy 1, policy_version 252693 (0.0011) [2023-12-26 17:12:04,151][105620] Updated weights for policy 1, policy_version 252703 (0.0011) [2023-12-26 17:12:04,451][105692] Updated weights for policy 0, policy_version 252341 (0.0010) [2023-12-26 17:12:04,517][105692] Updated weights for policy 0, policy_version 252351 (0.0006) [2023-12-26 17:12:04,586][105692] Updated weights for policy 0, policy_version 252361 (0.0008) [2023-12-26 17:12:04,920][105620] Updated weights for policy 1, policy_version 252713 (0.0011) [2023-12-26 17:12:04,972][105620] Updated weights for policy 1, policy_version 252723 (0.0010) [2023-12-26 17:12:05,020][105620] Updated weights for policy 1, policy_version 252733 (0.0010) [2023-12-26 17:12:05,065][105620] Updated weights for policy 1, policy_version 252743 (0.0010) [2023-12-26 17:12:05,158][105692] Updated weights for policy 0, policy_version 252371 (0.0009) [2023-12-26 17:12:05,218][105692] Updated weights for policy 0, policy_version 252381 (0.0008) [2023-12-26 17:12:05,285][105692] Updated weights for policy 0, policy_version 252391 (0.0007) [2023-12-26 17:12:05,837][105620] Updated weights for policy 1, policy_version 252753 (0.0007) [2023-12-26 17:12:05,841][105692] Updated weights for policy 0, policy_version 252401 (0.0006) [2023-12-26 17:12:05,881][105620] Updated weights for policy 1, policy_version 252763 (0.0005) [2023-12-26 17:12:05,895][105692] Updated weights for policy 0, policy_version 252411 (0.0008) [2023-12-26 17:12:05,929][105620] Updated weights for policy 1, policy_version 252773 (0.0005) [2023-12-26 17:12:05,952][105692] Updated weights for policy 0, policy_version 252421 (0.0006) [2023-12-26 17:12:06,006][105692] Updated weights for policy 0, policy_version 252431 (0.0007) [2023-12-26 17:12:06,062][104569] Fps is (10 sec: 22118.5, 60 sec: 19797.4, 300 sec: 19716.3). Total num frames: 129351680. Throughput: 0: 9861.3, 1: 9802.9. Samples: 129333040. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-26 17:12:06,062][104569] Avg episode reward: [(0, '9356.816'), (1, '8383.817')] [2023-12-26 17:12:06,606][105620] Updated weights for policy 1, policy_version 252783 (0.0007) [2023-12-26 17:12:06,672][105620] Updated weights for policy 1, policy_version 252793 (0.0008) [2023-12-26 17:12:06,726][105692] Updated weights for policy 0, policy_version 252441 (0.0009) [2023-12-26 17:12:06,738][105620] Updated weights for policy 1, policy_version 252803 (0.0009) [2023-12-26 17:12:06,781][105692] Updated weights for policy 0, policy_version 252451 (0.0009) [2023-12-26 17:12:06,832][105692] Updated weights for policy 0, policy_version 252461 (0.0009) [2023-12-26 17:12:07,514][105620] Updated weights for policy 1, policy_version 252813 (0.0007) [2023-12-26 17:12:07,572][105620] Updated weights for policy 1, policy_version 252823 (0.0009) [2023-12-26 17:12:07,613][105692] Updated weights for policy 0, policy_version 252471 (0.0008) [2023-12-26 17:12:07,620][105620] Updated weights for policy 1, policy_version 252833 (0.0009) [2023-12-26 17:12:07,671][105692] Updated weights for policy 0, policy_version 252481 (0.0007) [2023-12-26 17:12:07,727][105692] Updated weights for policy 0, policy_version 252491 (0.0009) [2023-12-26 17:12:08,330][105692] Updated weights for policy 0, policy_version 252501 (0.0008) [2023-12-26 17:12:08,394][105692] Updated weights for policy 0, policy_version 252511 (0.0009) [2023-12-26 17:12:08,394][105620] Updated weights for policy 1, policy_version 252843 (0.0009) [2023-12-26 17:12:08,451][105620] Updated weights for policy 1, policy_version 252853 (0.0008) [2023-12-26 17:12:08,456][105692] Updated weights for policy 0, policy_version 252521 (0.0010) [2023-12-26 17:12:08,500][105620] Updated weights for policy 1, policy_version 252863 (0.0007) [2023-12-26 17:12:09,179][105692] Updated weights for policy 0, policy_version 252531 (0.0010) [2023-12-26 17:12:09,229][105692] Updated weights for policy 0, policy_version 252541 (0.0008) [2023-12-26 17:12:09,278][105620] Updated weights for policy 1, policy_version 252873 (0.0008) [2023-12-26 17:12:09,289][105692] Updated weights for policy 0, policy_version 252551 (0.0008) [2023-12-26 17:12:09,327][105620] Updated weights for policy 1, policy_version 252883 (0.0006) [2023-12-26 17:12:09,399][105620] Updated weights for policy 1, policy_version 252893 (0.0010) [2023-12-26 17:12:09,462][105620] Updated weights for policy 1, policy_version 252903 (0.0009) [2023-12-26 17:12:09,916][105692] Updated weights for policy 0, policy_version 252561 (0.0007) [2023-12-26 17:12:09,976][105692] Updated weights for policy 0, policy_version 252571 (0.0008) [2023-12-26 17:12:10,040][105692] Updated weights for policy 0, policy_version 252581 (0.0007) [2023-12-26 17:12:10,108][105692] Updated weights for policy 0, policy_version 252591 (0.0005) [2023-12-26 17:12:10,369][105620] Updated weights for policy 1, policy_version 252913 (0.0010) [2023-12-26 17:12:10,436][105620] Updated weights for policy 1, policy_version 252923 (0.0010) [2023-12-26 17:12:10,491][105620] Updated weights for policy 1, policy_version 252933 (0.0010) [2023-12-26 17:12:10,663][105692] Updated weights for policy 0, policy_version 252601 (0.0005) [2023-12-26 17:12:10,713][105692] Updated weights for policy 0, policy_version 252611 (0.0005) [2023-12-26 17:12:10,768][105692] Updated weights for policy 0, policy_version 252621 (0.0005) [2023-12-26 17:12:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 129441792. Throughput: 0: 9963.4, 1: 9800.2. Samples: 129450272. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-26 17:12:11,063][104569] Avg episode reward: [(0, '9356.668'), (1, '7457.309')] [2023-12-26 17:12:11,366][105620] Updated weights for policy 1, policy_version 252944 (0.0009) [2023-12-26 17:12:11,418][105692] Updated weights for policy 0, policy_version 252631 (0.0009) [2023-12-26 17:12:11,428][105620] Updated weights for policy 1, policy_version 252954 (0.0008) [2023-12-26 17:12:11,470][105692] Updated weights for policy 0, policy_version 252641 (0.0011) [2023-12-26 17:12:11,481][105620] Updated weights for policy 1, policy_version 252964 (0.0005) [2023-12-26 17:12:11,532][105692] Updated weights for policy 0, policy_version 252651 (0.0010) [2023-12-26 17:12:12,190][105620] Updated weights for policy 1, policy_version 252974 (0.0006) [2023-12-26 17:12:12,241][105620] Updated weights for policy 1, policy_version 252984 (0.0009) [2023-12-26 17:12:12,285][105692] Updated weights for policy 0, policy_version 252661 (0.0010) [2023-12-26 17:12:12,300][105620] Updated weights for policy 1, policy_version 252994 (0.0007) [2023-12-26 17:12:12,354][105692] Updated weights for policy 0, policy_version 252671 (0.0010) [2023-12-26 17:12:12,414][105692] Updated weights for policy 0, policy_version 252681 (0.0008) [2023-12-26 17:12:12,997][105692] Updated weights for policy 0, policy_version 252691 (0.0006) [2023-12-26 17:12:13,056][105692] Updated weights for policy 0, policy_version 252701 (0.0005) [2023-12-26 17:12:13,064][105620] Updated weights for policy 1, policy_version 253004 (0.0007) [2023-12-26 17:12:13,112][105586] KL-divergence is very high: 117.8186 [2023-12-26 17:12:13,113][105692] Updated weights for policy 0, policy_version 252711 (0.0006) [2023-12-26 17:12:13,125][105620] Updated weights for policy 1, policy_version 253014 (0.0011) [2023-12-26 17:12:13,162][105586] KL-divergence is very high: 271.3555 [2023-12-26 17:12:13,185][105620] Updated weights for policy 1, policy_version 253024 (0.0011) [2023-12-26 17:12:13,202][105586] KL-divergence is very high: 297.1155 [2023-12-26 17:12:13,748][105692] Updated weights for policy 0, policy_version 252721 (0.0009) [2023-12-26 17:12:13,797][105692] Updated weights for policy 0, policy_version 252731 (0.0010) [2023-12-26 17:12:13,842][105692] Updated weights for policy 0, policy_version 252741 (0.0010) [2023-12-26 17:12:13,890][105692] Updated weights for policy 0, policy_version 252751 (0.0010) [2023-12-26 17:12:13,925][105620] Updated weights for policy 1, policy_version 253034 (0.0010) [2023-12-26 17:12:13,984][105620] Updated weights for policy 1, policy_version 253044 (0.0010) [2023-12-26 17:12:14,043][105620] Updated weights for policy 1, policy_version 253054 (0.0010) [2023-12-26 17:12:14,111][105620] Updated weights for policy 1, policy_version 253064 (0.0010) [2023-12-26 17:12:14,624][105692] Updated weights for policy 0, policy_version 252761 (0.0006) [2023-12-26 17:12:14,692][105692] Updated weights for policy 0, policy_version 252771 (0.0005) [2023-12-26 17:12:14,734][105620] Updated weights for policy 1, policy_version 253074 (0.0007) [2023-12-26 17:12:14,749][105692] Updated weights for policy 0, policy_version 252781 (0.0009) [2023-12-26 17:12:14,786][105620] Updated weights for policy 1, policy_version 253084 (0.0011) [2023-12-26 17:12:14,843][105620] Updated weights for policy 1, policy_version 253094 (0.0011) [2023-12-26 17:12:15,447][105692] Updated weights for policy 0, policy_version 252791 (0.0009) [2023-12-26 17:12:15,497][105692] Updated weights for policy 0, policy_version 252801 (0.0010) [2023-12-26 17:12:15,542][105692] Updated weights for policy 0, policy_version 252811 (0.0010) [2023-12-26 17:12:15,601][105620] Updated weights for policy 1, policy_version 253104 (0.0011) [2023-12-26 17:12:15,653][105620] Updated weights for policy 1, policy_version 253114 (0.0010) [2023-12-26 17:12:15,698][105620] Updated weights for policy 1, policy_version 253124 (0.0010) [2023-12-26 17:12:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 129540096. Throughput: 0: 9976.8, 1: 9817.4. Samples: 129509868. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-26 17:12:16,062][104569] Avg episode reward: [(0, '9356.398'), (1, '7812.321')] [2023-12-26 17:12:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000252816_64733184.pth... [2023-12-26 17:12:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000253128_64806912.pth... [2023-12-26 17:12:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000251600_64421888.pth [2023-12-26 17:12:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000252008_64520192.pth [2023-12-26 17:12:16,156][105692] Updated weights for policy 0, policy_version 252821 (0.0006) [2023-12-26 17:12:16,211][105692] Updated weights for policy 0, policy_version 252831 (0.0006) [2023-12-26 17:12:16,264][105692] Updated weights for policy 0, policy_version 252841 (0.0005) [2023-12-26 17:12:16,483][105620] Updated weights for policy 1, policy_version 253134 (0.0011) [2023-12-26 17:12:16,539][105620] Updated weights for policy 1, policy_version 253144 (0.0011) [2023-12-26 17:12:16,597][105620] Updated weights for policy 1, policy_version 253154 (0.0011) [2023-12-26 17:12:16,783][105692] Updated weights for policy 0, policy_version 252851 (0.0006) [2023-12-26 17:12:16,835][105692] Updated weights for policy 0, policy_version 252861 (0.0008) [2023-12-26 17:12:16,895][105692] Updated weights for policy 0, policy_version 252871 (0.0008) [2023-12-26 17:12:17,347][105620] Updated weights for policy 1, policy_version 253164 (0.0010) [2023-12-26 17:12:17,401][105620] Updated weights for policy 1, policy_version 253174 (0.0010) [2023-12-26 17:12:17,456][105620] Updated weights for policy 1, policy_version 253184 (0.0010) [2023-12-26 17:12:17,498][105692] Updated weights for policy 0, policy_version 252881 (0.0007) [2023-12-26 17:12:17,547][105692] Updated weights for policy 0, policy_version 252891 (0.0005) [2023-12-26 17:12:17,601][105692] Updated weights for policy 0, policy_version 252901 (0.0005) [2023-12-26 17:12:17,647][105692] Updated weights for policy 0, policy_version 252911 (0.0005) [2023-12-26 17:12:18,084][105620] Updated weights for policy 1, policy_version 253194 (0.0009) [2023-12-26 17:12:18,149][105620] Updated weights for policy 1, policy_version 253204 (0.0006) [2023-12-26 17:12:18,176][105692] Updated weights for policy 0, policy_version 252921 (0.0008) [2023-12-26 17:12:18,212][105620] Updated weights for policy 1, policy_version 253214 (0.0006) [2023-12-26 17:12:18,243][105692] Updated weights for policy 0, policy_version 252931 (0.0005) [2023-12-26 17:12:18,274][105620] Updated weights for policy 1, policy_version 253224 (0.0008) [2023-12-26 17:12:18,308][105692] Updated weights for policy 0, policy_version 252941 (0.0005) [2023-12-26 17:12:18,875][105692] Updated weights for policy 0, policy_version 252951 (0.0006) [2023-12-26 17:12:18,933][105692] Updated weights for policy 0, policy_version 252961 (0.0009) [2023-12-26 17:12:18,992][105692] Updated weights for policy 0, policy_version 252971 (0.0009) [2023-12-26 17:12:19,044][105620] Updated weights for policy 1, policy_version 253234 (0.0008) [2023-12-26 17:12:19,108][105620] Updated weights for policy 1, policy_version 253244 (0.0009) [2023-12-26 17:12:19,172][105620] Updated weights for policy 1, policy_version 253254 (0.0009) [2023-12-26 17:12:19,734][105692] Updated weights for policy 0, policy_version 252981 (0.0007) [2023-12-26 17:12:19,789][105692] Updated weights for policy 0, policy_version 252991 (0.0009) [2023-12-26 17:12:19,849][105692] Updated weights for policy 0, policy_version 253001 (0.0010) [2023-12-26 17:12:19,918][105620] Updated weights for policy 1, policy_version 253264 (0.0009) [2023-12-26 17:12:19,976][105620] Updated weights for policy 1, policy_version 253274 (0.0008) [2023-12-26 17:12:20,033][105620] Updated weights for policy 1, policy_version 253284 (0.0009) [2023-12-26 17:12:20,651][105692] Updated weights for policy 0, policy_version 253011 (0.0008) [2023-12-26 17:12:20,699][105692] Updated weights for policy 0, policy_version 253021 (0.0009) [2023-12-26 17:12:20,750][105692] Updated weights for policy 0, policy_version 253031 (0.0009) [2023-12-26 17:12:20,783][105620] Updated weights for policy 1, policy_version 253294 (0.0007) [2023-12-26 17:12:20,837][105620] Updated weights for policy 1, policy_version 253304 (0.0009) [2023-12-26 17:12:20,885][105620] Updated weights for policy 1, policy_version 253314 (0.0008) [2023-12-26 17:12:21,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.3, 300 sec: 19716.3). Total num frames: 129646592. Throughput: 0: 10188.3, 1: 9802.0. Samples: 129633780. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-26 17:12:21,063][104569] Avg episode reward: [(0, '8722.358'), (1, '8740.848')] [2023-12-26 17:12:21,563][105692] Updated weights for policy 0, policy_version 253041 (0.0008) [2023-12-26 17:12:21,625][105692] Updated weights for policy 0, policy_version 253051 (0.0008) [2023-12-26 17:12:21,648][105620] Updated weights for policy 1, policy_version 253324 (0.0008) [2023-12-26 17:12:21,686][105692] Updated weights for policy 0, policy_version 253061 (0.0009) [2023-12-26 17:12:21,713][105620] Updated weights for policy 1, policy_version 253334 (0.0008) [2023-12-26 17:12:21,752][105692] Updated weights for policy 0, policy_version 253071 (0.0008) [2023-12-26 17:12:21,778][105620] Updated weights for policy 1, policy_version 253344 (0.0009) [2023-12-26 17:12:22,495][105620] Updated weights for policy 1, policy_version 253354 (0.0008) [2023-12-26 17:12:22,545][105620] Updated weights for policy 1, policy_version 253364 (0.0006) [2023-12-26 17:12:22,559][105692] Updated weights for policy 0, policy_version 253081 (0.0009) [2023-12-26 17:12:22,595][105620] Updated weights for policy 1, policy_version 253374 (0.0007) [2023-12-26 17:12:22,609][105692] Updated weights for policy 0, policy_version 253091 (0.0008) [2023-12-26 17:12:22,644][105620] Updated weights for policy 1, policy_version 253384 (0.0007) [2023-12-26 17:12:22,667][105692] Updated weights for policy 0, policy_version 253101 (0.0008) [2023-12-26 17:12:23,261][105620] Updated weights for policy 1, policy_version 253394 (0.0009) [2023-12-26 17:12:23,320][105620] Updated weights for policy 1, policy_version 253404 (0.0010) [2023-12-26 17:12:23,375][105620] Updated weights for policy 1, policy_version 253414 (0.0008) [2023-12-26 17:12:23,493][105692] Updated weights for policy 0, policy_version 253111 (0.0009) [2023-12-26 17:12:23,539][105692] Updated weights for policy 0, policy_version 253121 (0.0008) [2023-12-26 17:12:23,593][105692] Updated weights for policy 0, policy_version 253131 (0.0009) [2023-12-26 17:12:24,129][105620] Updated weights for policy 1, policy_version 253424 (0.0010) [2023-12-26 17:12:24,187][105620] Updated weights for policy 1, policy_version 253434 (0.0010) [2023-12-26 17:12:24,246][105620] Updated weights for policy 1, policy_version 253444 (0.0010) [2023-12-26 17:12:24,380][105692] Updated weights for policy 0, policy_version 253141 (0.0008) [2023-12-26 17:12:24,428][105692] Updated weights for policy 0, policy_version 253151 (0.0008) [2023-12-26 17:12:24,481][105692] Updated weights for policy 0, policy_version 253161 (0.0010) [2023-12-26 17:12:24,854][105620] Updated weights for policy 1, policy_version 253454 (0.0010) [2023-12-26 17:12:24,909][105620] Updated weights for policy 1, policy_version 253464 (0.0010) [2023-12-26 17:12:24,970][105620] Updated weights for policy 1, policy_version 253474 (0.0010) [2023-12-26 17:12:25,362][105692] Updated weights for policy 0, policy_version 253171 (0.0009) [2023-12-26 17:12:25,416][105692] Updated weights for policy 0, policy_version 253182 (0.0010) [2023-12-26 17:12:25,478][105692] Updated weights for policy 0, policy_version 253192 (0.0008) [2023-12-26 17:12:25,523][105620] Updated weights for policy 1, policy_version 253484 (0.0010) [2023-12-26 17:12:25,573][105620] Updated weights for policy 1, policy_version 253494 (0.0010) [2023-12-26 17:12:25,617][105620] Updated weights for policy 1, policy_version 253504 (0.0010) [2023-12-26 17:12:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 129736704. Throughput: 0: 10131.4, 1: 9824.3. Samples: 129747116. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-26 17:12:26,063][104569] Avg episode reward: [(0, '8543.650'), (1, '8725.121')] [2023-12-26 17:12:26,166][105692] Updated weights for policy 0, policy_version 253202 (0.0006) [2023-12-26 17:12:26,225][105692] Updated weights for policy 0, policy_version 253212 (0.0006) [2023-12-26 17:12:26,267][105585] KL-divergence is very high: 263.7257 [2023-12-26 17:12:26,284][105692] Updated weights for policy 0, policy_version 253222 (0.0009) [2023-12-26 17:12:26,313][105585] KL-divergence is very high: 224.3370 [2023-12-26 17:12:26,341][105692] Updated weights for policy 0, policy_version 253232 (0.0008) [2023-12-26 17:12:26,343][105620] Updated weights for policy 1, policy_version 253514 (0.0008) [2023-12-26 17:12:26,403][105620] Updated weights for policy 1, policy_version 253524 (0.0009) [2023-12-26 17:12:26,461][105620] Updated weights for policy 1, policy_version 253534 (0.0009) [2023-12-26 17:12:26,511][105620] Updated weights for policy 1, policy_version 253544 (0.0009) [2023-12-26 17:12:26,988][105585] KL-divergence is very high: 110.0268 [2023-12-26 17:12:27,045][105692] Updated weights for policy 0, policy_version 253242 (0.0009) [2023-12-26 17:12:27,102][105692] Updated weights for policy 0, policy_version 253252 (0.0009) [2023-12-26 17:12:27,153][105692] Updated weights for policy 0, policy_version 253262 (0.0008) [2023-12-26 17:12:27,284][105620] Updated weights for policy 1, policy_version 253554 (0.0005) [2023-12-26 17:12:27,340][105620] Updated weights for policy 1, policy_version 253564 (0.0006) [2023-12-26 17:12:27,398][105620] Updated weights for policy 1, policy_version 253574 (0.0008) [2023-12-26 17:12:27,946][105692] Updated weights for policy 0, policy_version 253272 (0.0008) [2023-12-26 17:12:27,996][105692] Updated weights for policy 0, policy_version 253282 (0.0009) [2023-12-26 17:12:28,039][105620] Updated weights for policy 1, policy_version 253584 (0.0007) [2023-12-26 17:12:28,048][105692] Updated weights for policy 0, policy_version 253292 (0.0007) [2023-12-26 17:12:28,088][105620] Updated weights for policy 1, policy_version 253594 (0.0007) [2023-12-26 17:12:28,146][105620] Updated weights for policy 1, policy_version 253604 (0.0008) [2023-12-26 17:12:28,841][105620] Updated weights for policy 1, policy_version 253614 (0.0008) [2023-12-26 17:12:28,867][105692] Updated weights for policy 0, policy_version 253302 (0.0007) [2023-12-26 17:12:28,901][105620] Updated weights for policy 1, policy_version 253624 (0.0007) [2023-12-26 17:12:28,918][105692] Updated weights for policy 0, policy_version 253312 (0.0006) [2023-12-26 17:12:28,952][105620] Updated weights for policy 1, policy_version 253634 (0.0008) [2023-12-26 17:12:28,974][105692] Updated weights for policy 0, policy_version 253322 (0.0008) [2023-12-26 17:12:29,648][105620] Updated weights for policy 1, policy_version 253644 (0.0007) [2023-12-26 17:12:29,699][105620] Updated weights for policy 1, policy_version 253654 (0.0005) [2023-12-26 17:12:29,767][105620] Updated weights for policy 1, policy_version 253664 (0.0006) [2023-12-26 17:12:29,789][105692] Updated weights for policy 0, policy_version 253332 (0.0009) [2023-12-26 17:12:29,845][105692] Updated weights for policy 0, policy_version 253342 (0.0007) [2023-12-26 17:12:29,902][105692] Updated weights for policy 0, policy_version 253352 (0.0009) [2023-12-26 17:12:30,414][105620] Updated weights for policy 1, policy_version 253674 (0.0008) [2023-12-26 17:12:30,478][105620] Updated weights for policy 1, policy_version 253684 (0.0008) [2023-12-26 17:12:30,537][105620] Updated weights for policy 1, policy_version 253694 (0.0009) [2023-12-26 17:12:30,594][105620] Updated weights for policy 1, policy_version 253704 (0.0009) [2023-12-26 17:12:30,679][105692] Updated weights for policy 0, policy_version 253362 (0.0008) [2023-12-26 17:12:30,732][105692] Updated weights for policy 0, policy_version 253372 (0.0009) [2023-12-26 17:12:30,787][105692] Updated weights for policy 0, policy_version 253382 (0.0007) [2023-12-26 17:12:30,795][105585] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000004 [2023-12-26 17:12:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 129835008. Throughput: 0: 10157.5, 1: 9815.1. Samples: 129804664. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-26 17:12:31,063][104569] Avg episode reward: [(0, '8637.081'), (1, '8812.329')] [2023-12-26 17:12:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000253384_64880640.pth... [2023-12-26 17:12:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000253704_64954368.pth... [2023-12-26 17:12:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000252208_64577536.pth [2023-12-26 17:12:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000252584_64667648.pth [2023-12-26 17:12:31,351][105620] Updated weights for policy 1, policy_version 253714 (0.0009) [2023-12-26 17:12:31,424][105620] Updated weights for policy 1, policy_version 253724 (0.0009) [2023-12-26 17:12:31,483][105620] Updated weights for policy 1, policy_version 253734 (0.0008) [2023-12-26 17:12:31,622][105692] Updated weights for policy 0, policy_version 253392 (0.0008) [2023-12-26 17:12:31,681][105692] Updated weights for policy 0, policy_version 253402 (0.0009) [2023-12-26 17:12:31,748][105692] Updated weights for policy 0, policy_version 253412 (0.0009) [2023-12-26 17:12:32,183][105620] Updated weights for policy 1, policy_version 253744 (0.0006) [2023-12-26 17:12:32,242][105620] Updated weights for policy 1, policy_version 253754 (0.0009) [2023-12-26 17:12:32,311][105620] Updated weights for policy 1, policy_version 253764 (0.0009) [2023-12-26 17:12:32,451][105692] Updated weights for policy 0, policy_version 253422 (0.0008) [2023-12-26 17:12:32,505][105692] Updated weights for policy 0, policy_version 253432 (0.0010) [2023-12-26 17:12:32,553][105692] Updated weights for policy 0, policy_version 253442 (0.0010) [2023-12-26 17:12:33,022][105620] Updated weights for policy 1, policy_version 253774 (0.0007) [2023-12-26 17:12:33,079][105620] Updated weights for policy 1, policy_version 253784 (0.0005) [2023-12-26 17:12:33,142][105620] Updated weights for policy 1, policy_version 253794 (0.0005) [2023-12-26 17:12:33,368][105692] Updated weights for policy 0, policy_version 253452 (0.0010) [2023-12-26 17:12:33,421][105692] Updated weights for policy 0, policy_version 253462 (0.0008) [2023-12-26 17:12:33,475][105692] Updated weights for policy 0, policy_version 253472 (0.0009) [2023-12-26 17:12:33,680][105620] Updated weights for policy 1, policy_version 253804 (0.0006) [2023-12-26 17:12:33,734][105620] Updated weights for policy 1, policy_version 253814 (0.0008) [2023-12-26 17:12:33,789][105620] Updated weights for policy 1, policy_version 253824 (0.0009) [2023-12-26 17:12:34,132][105692] Updated weights for policy 0, policy_version 253482 (0.0008) [2023-12-26 17:12:34,209][105692] Updated weights for policy 0, policy_version 253492 (0.0008) [2023-12-26 17:12:34,272][105692] Updated weights for policy 0, policy_version 253502 (0.0008) [2023-12-26 17:12:34,330][105692] Updated weights for policy 0, policy_version 253512 (0.0008) [2023-12-26 17:12:34,644][105620] Updated weights for policy 1, policy_version 253835 (0.0010) [2023-12-26 17:12:34,692][105620] Updated weights for policy 1, policy_version 253845 (0.0009) [2023-12-26 17:12:34,741][105620] Updated weights for policy 1, policy_version 253855 (0.0009) [2023-12-26 17:12:35,025][105692] Updated weights for policy 0, policy_version 253522 (0.0008) [2023-12-26 17:12:35,076][105692] Updated weights for policy 0, policy_version 253532 (0.0009) [2023-12-26 17:12:35,123][105692] Updated weights for policy 0, policy_version 253542 (0.0009) [2023-12-26 17:12:35,498][105620] Updated weights for policy 1, policy_version 253865 (0.0009) [2023-12-26 17:12:35,548][105620] Updated weights for policy 1, policy_version 253875 (0.0008) [2023-12-26 17:12:35,599][105620] Updated weights for policy 1, policy_version 253885 (0.0009) [2023-12-26 17:12:35,648][105620] Updated weights for policy 1, policy_version 253895 (0.0009) [2023-12-26 17:12:35,781][105692] Updated weights for policy 0, policy_version 253552 (0.0006) [2023-12-26 17:12:35,837][105692] Updated weights for policy 0, policy_version 253562 (0.0005) [2023-12-26 17:12:35,897][105692] Updated weights for policy 0, policy_version 253572 (0.0008) [2023-12-26 17:12:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 129933312. Throughput: 0: 10081.0, 1: 9739.5. Samples: 129919360. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-26 17:12:36,062][104569] Avg episode reward: [(0, '8999.400'), (1, '9001.336')] [2023-12-26 17:12:36,476][105620] Updated weights for policy 1, policy_version 253905 (0.0009) [2023-12-26 17:12:36,541][105620] Updated weights for policy 1, policy_version 253915 (0.0009) [2023-12-26 17:12:36,605][105620] Updated weights for policy 1, policy_version 253925 (0.0009) [2023-12-26 17:12:36,615][105692] Updated weights for policy 0, policy_version 253582 (0.0007) [2023-12-26 17:12:36,670][105692] Updated weights for policy 0, policy_version 253592 (0.0009) [2023-12-26 17:12:36,722][105692] Updated weights for policy 0, policy_version 253602 (0.0009) [2023-12-26 17:12:37,301][105620] Updated weights for policy 1, policy_version 253935 (0.0006) [2023-12-26 17:12:37,359][105620] Updated weights for policy 1, policy_version 253945 (0.0008) [2023-12-26 17:12:37,414][105620] Updated weights for policy 1, policy_version 253955 (0.0008) [2023-12-26 17:12:37,548][105692] Updated weights for policy 0, policy_version 253612 (0.0010) [2023-12-26 17:12:37,613][105692] Updated weights for policy 0, policy_version 253622 (0.0008) [2023-12-26 17:12:37,666][105692] Updated weights for policy 0, policy_version 253632 (0.0010) [2023-12-26 17:12:38,064][105620] Updated weights for policy 1, policy_version 253965 (0.0006) [2023-12-26 17:12:38,135][105620] Updated weights for policy 1, policy_version 253975 (0.0005) [2023-12-26 17:12:38,189][105620] Updated weights for policy 1, policy_version 253985 (0.0006) [2023-12-26 17:12:38,513][105692] Updated weights for policy 0, policy_version 253642 (0.0010) [2023-12-26 17:12:38,564][105692] Updated weights for policy 0, policy_version 253652 (0.0009) [2023-12-26 17:12:38,618][105692] Updated weights for policy 0, policy_version 253662 (0.0009) [2023-12-26 17:12:38,677][105692] Updated weights for policy 0, policy_version 253672 (0.0009) [2023-12-26 17:12:38,857][105620] Updated weights for policy 1, policy_version 253995 (0.0007) [2023-12-26 17:12:38,909][105620] Updated weights for policy 1, policy_version 254005 (0.0010) [2023-12-26 17:12:38,959][105620] Updated weights for policy 1, policy_version 254015 (0.0008) [2023-12-26 17:12:39,479][105692] Updated weights for policy 0, policy_version 253682 (0.0009) [2023-12-26 17:12:39,538][105692] Updated weights for policy 0, policy_version 253692 (0.0009) [2023-12-26 17:12:39,600][105692] Updated weights for policy 0, policy_version 253702 (0.0009) [2023-12-26 17:12:39,702][105620] Updated weights for policy 1, policy_version 254025 (0.0008) [2023-12-26 17:12:39,760][105620] Updated weights for policy 1, policy_version 254035 (0.0006) [2023-12-26 17:12:39,822][105620] Updated weights for policy 1, policy_version 254045 (0.0008) [2023-12-26 17:12:39,884][105620] Updated weights for policy 1, policy_version 254055 (0.0009) [2023-12-26 17:12:40,405][105692] Updated weights for policy 0, policy_version 253712 (0.0008) [2023-12-26 17:12:40,458][105692] Updated weights for policy 0, policy_version 253722 (0.0006) [2023-12-26 17:12:40,508][105692] Updated weights for policy 0, policy_version 253732 (0.0005) [2023-12-26 17:12:40,615][105620] Updated weights for policy 1, policy_version 254065 (0.0010) [2023-12-26 17:12:40,690][105620] Updated weights for policy 1, policy_version 254075 (0.0010) [2023-12-26 17:12:40,756][105620] Updated weights for policy 1, policy_version 254085 (0.0009) [2023-12-26 17:12:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19633.1). Total num frames: 130023424. Throughput: 0: 9971.5, 1: 9743.1. Samples: 130032080. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-26 17:12:41,063][104569] Avg episode reward: [(0, '9357.494'), (1, '7037.535')] [2023-12-26 17:12:41,190][105692] Updated weights for policy 0, policy_version 253742 (0.0008) [2023-12-26 17:12:41,249][105692] Updated weights for policy 0, policy_version 253752 (0.0009) [2023-12-26 17:12:41,307][105692] Updated weights for policy 0, policy_version 253762 (0.0009) [2023-12-26 17:12:41,530][105620] Updated weights for policy 1, policy_version 254095 (0.0009) [2023-12-26 17:12:41,582][105620] Updated weights for policy 1, policy_version 254105 (0.0009) [2023-12-26 17:12:41,642][105620] Updated weights for policy 1, policy_version 254115 (0.0009) [2023-12-26 17:12:42,054][105692] Updated weights for policy 0, policy_version 253772 (0.0009) [2023-12-26 17:12:42,106][105692] Updated weights for policy 0, policy_version 253782 (0.0008) [2023-12-26 17:12:42,162][105692] Updated weights for policy 0, policy_version 253792 (0.0006) [2023-12-26 17:12:42,425][105620] Updated weights for policy 1, policy_version 254125 (0.0009) [2023-12-26 17:12:42,476][105620] Updated weights for policy 1, policy_version 254135 (0.0008) [2023-12-26 17:12:42,538][105620] Updated weights for policy 1, policy_version 254145 (0.0008) [2023-12-26 17:12:42,903][105692] Updated weights for policy 0, policy_version 253802 (0.0009) [2023-12-26 17:12:42,949][105692] Updated weights for policy 0, policy_version 253812 (0.0009) [2023-12-26 17:12:42,997][105692] Updated weights for policy 0, policy_version 253822 (0.0008) [2023-12-26 17:12:43,062][105692] Updated weights for policy 0, policy_version 253832 (0.0009) [2023-12-26 17:12:43,289][105620] Updated weights for policy 1, policy_version 254155 (0.0009) [2023-12-26 17:12:43,343][105620] Updated weights for policy 1, policy_version 254165 (0.0009) [2023-12-26 17:12:43,396][105620] Updated weights for policy 1, policy_version 254175 (0.0009) [2023-12-26 17:12:43,761][105692] Updated weights for policy 0, policy_version 253842 (0.0005) [2023-12-26 17:12:43,825][105692] Updated weights for policy 0, policy_version 253852 (0.0005) [2023-12-26 17:12:43,882][105692] Updated weights for policy 0, policy_version 253862 (0.0009) [2023-12-26 17:12:44,073][105620] Updated weights for policy 1, policy_version 254185 (0.0010) [2023-12-26 17:12:44,139][105620] Updated weights for policy 1, policy_version 254195 (0.0009) [2023-12-26 17:12:44,201][105620] Updated weights for policy 1, policy_version 254205 (0.0009) [2023-12-26 17:12:44,260][105620] Updated weights for policy 1, policy_version 254215 (0.0009) [2023-12-26 17:12:44,527][105692] Updated weights for policy 0, policy_version 253872 (0.0006) [2023-12-26 17:12:44,580][105692] Updated weights for policy 0, policy_version 253882 (0.0005) [2023-12-26 17:12:44,629][105692] Updated weights for policy 0, policy_version 253892 (0.0008) [2023-12-26 17:12:45,013][105620] Updated weights for policy 1, policy_version 254225 (0.0010) [2023-12-26 17:12:45,076][105620] Updated weights for policy 1, policy_version 254235 (0.0009) [2023-12-26 17:12:45,139][105620] Updated weights for policy 1, policy_version 254245 (0.0009) [2023-12-26 17:12:45,287][105692] Updated weights for policy 0, policy_version 253902 (0.0009) [2023-12-26 17:12:45,342][105692] Updated weights for policy 0, policy_version 253912 (0.0009) [2023-12-26 17:12:45,390][105692] Updated weights for policy 0, policy_version 253922 (0.0009) [2023-12-26 17:12:45,910][105620] Updated weights for policy 1, policy_version 254255 (0.0008) [2023-12-26 17:12:45,971][105620] Updated weights for policy 1, policy_version 254265 (0.0009) [2023-12-26 17:12:46,034][105620] Updated weights for policy 1, policy_version 254275 (0.0008) [2023-12-26 17:12:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 130121728. Throughput: 0: 9862.1, 1: 9666.1. Samples: 130089604. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-26 17:12:46,062][104569] Avg episode reward: [(0, '9357.715'), (1, '8047.835')] [2023-12-26 17:12:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000253928_65019904.pth... [2023-12-26 17:12:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000254280_65101824.pth... [2023-12-26 17:12:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000252816_64733184.pth [2023-12-26 17:12:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000253128_64806912.pth [2023-12-26 17:12:46,119][105692] Updated weights for policy 0, policy_version 253932 (0.0008) [2023-12-26 17:12:46,166][105692] Updated weights for policy 0, policy_version 253942 (0.0009) [2023-12-26 17:12:46,214][105692] Updated weights for policy 0, policy_version 253952 (0.0007) [2023-12-26 17:12:46,777][105620] Updated weights for policy 1, policy_version 254285 (0.0009) [2023-12-26 17:12:46,824][105620] Updated weights for policy 1, policy_version 254295 (0.0009) [2023-12-26 17:12:46,870][105620] Updated weights for policy 1, policy_version 254305 (0.0008) [2023-12-26 17:12:46,950][105692] Updated weights for policy 0, policy_version 253962 (0.0006) [2023-12-26 17:12:47,008][105692] Updated weights for policy 0, policy_version 253972 (0.0009) [2023-12-26 17:12:47,069][105692] Updated weights for policy 0, policy_version 253982 (0.0008) [2023-12-26 17:12:47,127][105692] Updated weights for policy 0, policy_version 253992 (0.0009) [2023-12-26 17:12:47,663][105620] Updated weights for policy 1, policy_version 254315 (0.0008) [2023-12-26 17:12:47,714][105620] Updated weights for policy 1, policy_version 254325 (0.0009) [2023-12-26 17:12:47,761][105620] Updated weights for policy 1, policy_version 254335 (0.0009) [2023-12-26 17:12:47,837][105692] Updated weights for policy 0, policy_version 254002 (0.0005) [2023-12-26 17:12:47,900][105692] Updated weights for policy 0, policy_version 254012 (0.0008) [2023-12-26 17:12:47,956][105692] Updated weights for policy 0, policy_version 254022 (0.0008) [2023-12-26 17:12:48,514][105620] Updated weights for policy 1, policy_version 254345 (0.0009) [2023-12-26 17:12:48,570][105620] Updated weights for policy 1, policy_version 254355 (0.0011) [2023-12-26 17:12:48,629][105620] Updated weights for policy 1, policy_version 254365 (0.0011) [2023-12-26 17:12:48,659][105692] Updated weights for policy 0, policy_version 254032 (0.0006) [2023-12-26 17:12:48,688][105620] Updated weights for policy 1, policy_version 254375 (0.0010) [2023-12-26 17:12:48,719][105692] Updated weights for policy 0, policy_version 254042 (0.0007) [2023-12-26 17:12:48,785][105692] Updated weights for policy 0, policy_version 254052 (0.0009) [2023-12-26 17:12:49,392][105620] Updated weights for policy 1, policy_version 254385 (0.0009) [2023-12-26 17:12:49,454][105620] Updated weights for policy 1, policy_version 254395 (0.0010) [2023-12-26 17:12:49,482][105692] Updated weights for policy 0, policy_version 254062 (0.0008) [2023-12-26 17:12:49,513][105620] Updated weights for policy 1, policy_version 254405 (0.0010) [2023-12-26 17:12:49,544][105692] Updated weights for policy 0, policy_version 254072 (0.0008) [2023-12-26 17:12:49,599][105692] Updated weights for policy 0, policy_version 254082 (0.0008) [2023-12-26 17:12:50,284][105620] Updated weights for policy 1, policy_version 254415 (0.0009) [2023-12-26 17:12:50,339][105620] Updated weights for policy 1, policy_version 254425 (0.0009) [2023-12-26 17:12:50,368][105692] Updated weights for policy 0, policy_version 254092 (0.0007) [2023-12-26 17:12:50,387][105620] Updated weights for policy 1, policy_version 254435 (0.0008) [2023-12-26 17:12:50,417][105692] Updated weights for policy 0, policy_version 254102 (0.0006) [2023-12-26 17:12:50,470][105692] Updated weights for policy 0, policy_version 254112 (0.0009) [2023-12-26 17:12:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 130211840. Throughput: 0: 9802.2, 1: 9574.8. Samples: 130205004. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-26 17:12:51,062][104569] Avg episode reward: [(0, '9266.281'), (1, '8472.419')] [2023-12-26 17:12:51,077][105620] Updated weights for policy 1, policy_version 254445 (0.0008) [2023-12-26 17:12:51,141][105620] Updated weights for policy 1, policy_version 254455 (0.0010) [2023-12-26 17:12:51,216][105620] Updated weights for policy 1, policy_version 254465 (0.0010) [2023-12-26 17:12:51,232][105692] Updated weights for policy 0, policy_version 254122 (0.0008) [2023-12-26 17:12:51,299][105692] Updated weights for policy 0, policy_version 254132 (0.0008) [2023-12-26 17:12:51,371][105692] Updated weights for policy 0, policy_version 254142 (0.0008) [2023-12-26 17:12:51,443][105692] Updated weights for policy 0, policy_version 254152 (0.0010) [2023-12-26 17:12:51,871][105620] Updated weights for policy 1, policy_version 254475 (0.0008) [2023-12-26 17:12:51,920][105620] Updated weights for policy 1, policy_version 254485 (0.0007) [2023-12-26 17:12:51,976][105620] Updated weights for policy 1, policy_version 254495 (0.0007) [2023-12-26 17:12:52,162][105692] Updated weights for policy 0, policy_version 254162 (0.0006) [2023-12-26 17:12:52,213][105692] Updated weights for policy 0, policy_version 254172 (0.0005) [2023-12-26 17:12:52,266][105692] Updated weights for policy 0, policy_version 254182 (0.0008) [2023-12-26 17:12:52,806][105620] Updated weights for policy 1, policy_version 254505 (0.0010) [2023-12-26 17:12:52,875][105586] KL-divergence is very high: 137.4351 [2023-12-26 17:12:52,876][105620] Updated weights for policy 1, policy_version 254515 (0.0009) [2023-12-26 17:12:52,909][105586] KL-divergence is very high: 102.0784 [2023-12-26 17:12:52,926][105692] Updated weights for policy 0, policy_version 254192 (0.0006) [2023-12-26 17:12:52,929][105586] KL-divergence is very high: 206.0548 [2023-12-26 17:12:52,942][105620] Updated weights for policy 1, policy_version 254525 (0.0008) [2023-12-26 17:12:52,966][105586] KL-divergence is very high: 124.5859 [2023-12-26 17:12:52,973][105692] Updated weights for policy 0, policy_version 254202 (0.0007) [2023-12-26 17:12:52,985][105586] KL-divergence is very high: 235.3591 [2023-12-26 17:12:53,011][105620] Updated weights for policy 1, policy_version 254535 (0.0005) [2023-12-26 17:12:53,026][105692] Updated weights for policy 0, policy_version 254212 (0.0008) [2023-12-26 17:12:53,588][105692] Updated weights for policy 0, policy_version 254222 (0.0009) [2023-12-26 17:12:53,634][105692] Updated weights for policy 0, policy_version 254232 (0.0008) [2023-12-26 17:12:53,693][105692] Updated weights for policy 0, policy_version 254242 (0.0009) [2023-12-26 17:12:53,794][105620] Updated weights for policy 1, policy_version 254545 (0.0006) [2023-12-26 17:12:53,851][105620] Updated weights for policy 1, policy_version 254555 (0.0009) [2023-12-26 17:12:53,901][105620] Updated weights for policy 1, policy_version 254565 (0.0006) [2023-12-26 17:12:54,523][105692] Updated weights for policy 0, policy_version 254252 (0.0009) [2023-12-26 17:12:54,553][105620] Updated weights for policy 1, policy_version 254575 (0.0006) [2023-12-26 17:12:54,584][105692] Updated weights for policy 0, policy_version 254262 (0.0007) [2023-12-26 17:12:54,611][105620] Updated weights for policy 1, policy_version 254585 (0.0006) [2023-12-26 17:12:54,644][105692] Updated weights for policy 0, policy_version 254272 (0.0009) [2023-12-26 17:12:54,671][105620] Updated weights for policy 1, policy_version 254595 (0.0007) [2023-12-26 17:12:55,375][105692] Updated weights for policy 0, policy_version 254282 (0.0008) [2023-12-26 17:12:55,389][105620] Updated weights for policy 1, policy_version 254605 (0.0005) [2023-12-26 17:12:55,432][105692] Updated weights for policy 0, policy_version 254292 (0.0006) [2023-12-26 17:12:55,442][105620] Updated weights for policy 1, policy_version 254616 (0.0008) [2023-12-26 17:12:55,484][105692] Updated weights for policy 0, policy_version 254302 (0.0009) [2023-12-26 17:12:55,494][105620] Updated weights for policy 1, policy_version 254626 (0.0007) [2023-12-26 17:12:55,528][105692] Updated weights for policy 0, policy_version 254312 (0.0008) [2023-12-26 17:12:56,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19660.7, 300 sec: 19605.3). Total num frames: 130310144. Throughput: 0: 9673.7, 1: 9659.1. Samples: 130320252. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-26 17:12:56,063][104569] Avg episode reward: [(0, '9266.018'), (1, '8646.706')] [2023-12-26 17:12:56,176][105620] Updated weights for policy 1, policy_version 254636 (0.0006) [2023-12-26 17:12:56,196][105692] Updated weights for policy 0, policy_version 254322 (0.0009) [2023-12-26 17:12:56,230][105620] Updated weights for policy 1, policy_version 254646 (0.0008) [2023-12-26 17:12:56,253][105692] Updated weights for policy 0, policy_version 254332 (0.0010) [2023-12-26 17:12:56,287][105620] Updated weights for policy 1, policy_version 254656 (0.0007) [2023-12-26 17:12:56,307][105692] Updated weights for policy 0, policy_version 254342 (0.0010) [2023-12-26 17:12:56,913][105692] Updated weights for policy 0, policy_version 254352 (0.0006) [2023-12-26 17:12:56,961][105692] Updated weights for policy 0, policy_version 254362 (0.0005) [2023-12-26 17:12:57,013][105692] Updated weights for policy 0, policy_version 254372 (0.0005) [2023-12-26 17:12:57,139][105620] Updated weights for policy 1, policy_version 254666 (0.0007) [2023-12-26 17:12:57,192][105620] Updated weights for policy 1, policy_version 254676 (0.0009) [2023-12-26 17:12:57,250][105620] Updated weights for policy 1, policy_version 254687 (0.0011) [2023-12-26 17:12:57,556][105692] Updated weights for policy 0, policy_version 254382 (0.0008) [2023-12-26 17:12:57,612][105692] Updated weights for policy 0, policy_version 254392 (0.0007) [2023-12-26 17:12:57,674][105692] Updated weights for policy 0, policy_version 254402 (0.0006) [2023-12-26 17:12:58,098][105620] Updated weights for policy 1, policy_version 254697 (0.0009) [2023-12-26 17:12:58,158][105620] Updated weights for policy 1, policy_version 254707 (0.0008) [2023-12-26 17:12:58,219][105620] Updated weights for policy 1, policy_version 254717 (0.0006) [2023-12-26 17:12:58,278][105620] Updated weights for policy 1, policy_version 254727 (0.0007) [2023-12-26 17:12:58,282][105692] Updated weights for policy 0, policy_version 254412 (0.0007) [2023-12-26 17:12:58,342][105692] Updated weights for policy 0, policy_version 254422 (0.0009) [2023-12-26 17:12:58,399][105692] Updated weights for policy 0, policy_version 254432 (0.0011) [2023-12-26 17:12:59,035][105620] Updated weights for policy 1, policy_version 254737 (0.0007) [2023-12-26 17:12:59,089][105620] Updated weights for policy 1, policy_version 254747 (0.0006) [2023-12-26 17:12:59,148][105620] Updated weights for policy 1, policy_version 254757 (0.0008) [2023-12-26 17:12:59,242][105692] Updated weights for policy 0, policy_version 254442 (0.0009) [2023-12-26 17:12:59,303][105692] Updated weights for policy 0, policy_version 254452 (0.0008) [2023-12-26 17:12:59,362][105692] Updated weights for policy 0, policy_version 254462 (0.0009) [2023-12-26 17:12:59,420][105692] Updated weights for policy 0, policy_version 254472 (0.0009) [2023-12-26 17:12:59,946][105620] Updated weights for policy 1, policy_version 254767 (0.0009) [2023-12-26 17:13:00,017][105620] Updated weights for policy 1, policy_version 254777 (0.0008) [2023-12-26 17:13:00,074][105620] Updated weights for policy 1, policy_version 254787 (0.0009) [2023-12-26 17:13:00,181][105692] Updated weights for policy 0, policy_version 254482 (0.0009) [2023-12-26 17:13:00,235][105692] Updated weights for policy 0, policy_version 254493 (0.0010) [2023-12-26 17:13:00,284][105692] Updated weights for policy 0, policy_version 254503 (0.0009) [2023-12-26 17:13:00,735][105620] Updated weights for policy 1, policy_version 254797 (0.0009) [2023-12-26 17:13:00,786][105620] Updated weights for policy 1, policy_version 254807 (0.0007) [2023-12-26 17:13:00,847][105620] Updated weights for policy 1, policy_version 254817 (0.0005) [2023-12-26 17:13:00,993][105692] Updated weights for policy 0, policy_version 254513 (0.0006) [2023-12-26 17:13:01,053][105692] Updated weights for policy 0, policy_version 254523 (0.0006) [2023-12-26 17:13:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 130408448. Throughput: 0: 9713.9, 1: 9624.3. Samples: 130380088. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-26 17:13:01,062][104569] Avg episode reward: [(0, '9357.387'), (1, '8907.644')] [2023-12-26 17:13:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000254824_65241088.pth... [2023-12-26 17:13:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000253704_64954368.pth [2023-12-26 17:13:01,112][105692] Updated weights for policy 0, policy_version 254533 (0.0010) [2023-12-26 17:13:01,129][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000254536_65175552.pth... [2023-12-26 17:13:01,133][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000253384_64880640.pth [2023-12-26 17:13:01,604][105620] Updated weights for policy 1, policy_version 254827 (0.0006) [2023-12-26 17:13:01,668][105620] Updated weights for policy 1, policy_version 254837 (0.0007) [2023-12-26 17:13:01,741][105620] Updated weights for policy 1, policy_version 254847 (0.0007) [2023-12-26 17:13:01,830][105692] Updated weights for policy 0, policy_version 254543 (0.0009) [2023-12-26 17:13:01,879][105692] Updated weights for policy 0, policy_version 254553 (0.0009) [2023-12-26 17:13:01,925][105692] Updated weights for policy 0, policy_version 254563 (0.0008) [2023-12-26 17:13:02,361][105620] Updated weights for policy 1, policy_version 254857 (0.0006) [2023-12-26 17:13:02,425][105620] Updated weights for policy 1, policy_version 254867 (0.0010) [2023-12-26 17:13:02,479][105620] Updated weights for policy 1, policy_version 254877 (0.0008) [2023-12-26 17:13:02,538][105620] Updated weights for policy 1, policy_version 254887 (0.0010) [2023-12-26 17:13:02,798][105692] Updated weights for policy 0, policy_version 254573 (0.0009) [2023-12-26 17:13:02,853][105692] Updated weights for policy 0, policy_version 254583 (0.0008) [2023-12-26 17:13:02,918][105692] Updated weights for policy 0, policy_version 254593 (0.0008) [2023-12-26 17:13:03,258][105620] Updated weights for policy 1, policy_version 254897 (0.0006) [2023-12-26 17:13:03,325][105620] Updated weights for policy 1, policy_version 254907 (0.0005) [2023-12-26 17:13:03,397][105620] Updated weights for policy 1, policy_version 254917 (0.0005) [2023-12-26 17:13:03,563][105692] Updated weights for policy 0, policy_version 254603 (0.0008) [2023-12-26 17:13:03,619][105692] Updated weights for policy 0, policy_version 254613 (0.0009) [2023-12-26 17:13:03,672][105692] Updated weights for policy 0, policy_version 254623 (0.0009) [2023-12-26 17:13:03,916][105620] Updated weights for policy 1, policy_version 254927 (0.0007) [2023-12-26 17:13:03,974][105620] Updated weights for policy 1, policy_version 254937 (0.0006) [2023-12-26 17:13:04,037][105620] Updated weights for policy 1, policy_version 254947 (0.0006) [2023-12-26 17:13:04,407][105692] Updated weights for policy 0, policy_version 254633 (0.0008) [2023-12-26 17:13:04,459][105692] Updated weights for policy 0, policy_version 254643 (0.0011) [2023-12-26 17:13:04,515][105692] Updated weights for policy 0, policy_version 254653 (0.0010) [2023-12-26 17:13:04,572][105692] Updated weights for policy 0, policy_version 254663 (0.0010) [2023-12-26 17:13:04,592][105620] Updated weights for policy 1, policy_version 254957 (0.0008) [2023-12-26 17:13:04,655][105620] Updated weights for policy 1, policy_version 254967 (0.0011) [2023-12-26 17:13:04,718][105620] Updated weights for policy 1, policy_version 254977 (0.0011) [2023-12-26 17:13:05,334][105692] Updated weights for policy 0, policy_version 254673 (0.0010) [2023-12-26 17:13:05,365][105620] Updated weights for policy 1, policy_version 254987 (0.0009) [2023-12-26 17:13:05,385][105692] Updated weights for policy 0, policy_version 254683 (0.0010) [2023-12-26 17:13:05,420][105620] Updated weights for policy 1, policy_version 254997 (0.0005) [2023-12-26 17:13:05,447][105692] Updated weights for policy 0, policy_version 254693 (0.0010) [2023-12-26 17:13:05,476][105620] Updated weights for policy 1, policy_version 255007 (0.0005) [2023-12-26 17:13:06,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 130506752. Throughput: 0: 9521.7, 1: 9702.8. Samples: 130498880. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-26 17:13:06,063][104569] Avg episode reward: [(0, '9357.432'), (1, '9090.528')] [2023-12-26 17:13:06,148][105620] Updated weights for policy 1, policy_version 255017 (0.0006) [2023-12-26 17:13:06,212][105620] Updated weights for policy 1, policy_version 255027 (0.0009) [2023-12-26 17:13:06,213][105692] Updated weights for policy 0, policy_version 254703 (0.0008) [2023-12-26 17:13:06,278][105692] Updated weights for policy 0, policy_version 254713 (0.0007) [2023-12-26 17:13:06,278][105620] Updated weights for policy 1, policy_version 255037 (0.0008) [2023-12-26 17:13:06,344][105620] Updated weights for policy 1, policy_version 255047 (0.0010) [2023-12-26 17:13:06,346][105692] Updated weights for policy 0, policy_version 254723 (0.0008) [2023-12-26 17:13:07,027][105692] Updated weights for policy 0, policy_version 254733 (0.0009) [2023-12-26 17:13:07,085][105692] Updated weights for policy 0, policy_version 254743 (0.0007) [2023-12-26 17:13:07,091][105620] Updated weights for policy 1, policy_version 255057 (0.0009) [2023-12-26 17:13:07,135][105692] Updated weights for policy 0, policy_version 254753 (0.0007) [2023-12-26 17:13:07,149][105620] Updated weights for policy 1, policy_version 255067 (0.0010) [2023-12-26 17:13:07,153][105586] KL-divergence is very high: 106.0146 [2023-12-26 17:13:07,202][105586] KL-divergence is very high: 156.5730 [2023-12-26 17:13:07,208][105620] Updated weights for policy 1, policy_version 255077 (0.0010) [2023-12-26 17:13:07,905][105692] Updated weights for policy 0, policy_version 254763 (0.0006) [2023-12-26 17:13:07,952][105620] Updated weights for policy 1, policy_version 255087 (0.0010) [2023-12-26 17:13:07,967][105692] Updated weights for policy 0, policy_version 254773 (0.0007) [2023-12-26 17:13:08,007][105620] Updated weights for policy 1, policy_version 255097 (0.0010) [2023-12-26 17:13:08,016][105692] Updated weights for policy 0, policy_version 254783 (0.0008) [2023-12-26 17:13:08,065][105620] Updated weights for policy 1, policy_version 255107 (0.0010) [2023-12-26 17:13:08,795][105692] Updated weights for policy 0, policy_version 254793 (0.0006) [2023-12-26 17:13:08,815][105620] Updated weights for policy 1, policy_version 255117 (0.0010) [2023-12-26 17:13:08,849][105692] Updated weights for policy 0, policy_version 254803 (0.0006) [2023-12-26 17:13:08,874][105620] Updated weights for policy 1, policy_version 255127 (0.0010) [2023-12-26 17:13:08,901][105692] Updated weights for policy 0, policy_version 254813 (0.0008) [2023-12-26 17:13:08,929][105620] Updated weights for policy 1, policy_version 255137 (0.0010) [2023-12-26 17:13:08,959][105692] Updated weights for policy 0, policy_version 254823 (0.0006) [2023-12-26 17:13:09,695][105620] Updated weights for policy 1, policy_version 255147 (0.0010) [2023-12-26 17:13:09,752][105692] Updated weights for policy 0, policy_version 254833 (0.0007) [2023-12-26 17:13:09,765][105620] Updated weights for policy 1, policy_version 255157 (0.0011) [2023-12-26 17:13:09,820][105692] Updated weights for policy 0, policy_version 254843 (0.0006) [2023-12-26 17:13:09,822][105620] Updated weights for policy 1, policy_version 255167 (0.0011) [2023-12-26 17:13:09,905][105692] Updated weights for policy 0, policy_version 254853 (0.0007) [2023-12-26 17:13:10,571][105620] Updated weights for policy 1, policy_version 255177 (0.0010) [2023-12-26 17:13:10,582][105692] Updated weights for policy 0, policy_version 254863 (0.0007) [2023-12-26 17:13:10,634][105620] Updated weights for policy 1, policy_version 255187 (0.0006) [2023-12-26 17:13:10,639][105692] Updated weights for policy 0, policy_version 254873 (0.0005) [2023-12-26 17:13:10,695][105692] Updated weights for policy 0, policy_version 254883 (0.0007) [2023-12-26 17:13:10,697][105620] Updated weights for policy 1, policy_version 255197 (0.0006) [2023-12-26 17:13:10,753][105620] Updated weights for policy 1, policy_version 255207 (0.0010) [2023-12-26 17:13:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 130605056. Throughput: 0: 9621.4, 1: 9627.2. Samples: 130613304. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-26 17:13:11,063][104569] Avg episode reward: [(0, '9357.444'), (1, '8999.242')] [2023-12-26 17:13:11,432][105620] Updated weights for policy 1, policy_version 255217 (0.0009) [2023-12-26 17:13:11,474][105692] Updated weights for policy 0, policy_version 254893 (0.0009) [2023-12-26 17:13:11,493][105620] Updated weights for policy 1, policy_version 255227 (0.0005) [2023-12-26 17:13:11,537][105692] Updated weights for policy 0, policy_version 254903 (0.0008) [2023-12-26 17:13:11,556][105620] Updated weights for policy 1, policy_version 255237 (0.0006) [2023-12-26 17:13:11,601][105692] Updated weights for policy 0, policy_version 254913 (0.0009) [2023-12-26 17:13:12,206][105620] Updated weights for policy 1, policy_version 255247 (0.0006) [2023-12-26 17:13:12,272][105620] Updated weights for policy 1, policy_version 255257 (0.0007) [2023-12-26 17:13:12,329][105620] Updated weights for policy 1, policy_version 255267 (0.0008) [2023-12-26 17:13:12,394][105692] Updated weights for policy 0, policy_version 254923 (0.0008) [2023-12-26 17:13:12,452][105692] Updated weights for policy 0, policy_version 254933 (0.0008) [2023-12-26 17:13:12,513][105692] Updated weights for policy 0, policy_version 254943 (0.0008) [2023-12-26 17:13:13,002][105620] Updated weights for policy 1, policy_version 255277 (0.0007) [2023-12-26 17:13:13,065][105620] Updated weights for policy 1, policy_version 255287 (0.0006) [2023-12-26 17:13:13,131][105620] Updated weights for policy 1, policy_version 255297 (0.0010) [2023-12-26 17:13:13,167][105692] Updated weights for policy 0, policy_version 254953 (0.0008) [2023-12-26 17:13:13,230][105692] Updated weights for policy 0, policy_version 254963 (0.0006) [2023-12-26 17:13:13,289][105692] Updated weights for policy 0, policy_version 254973 (0.0010) [2023-12-26 17:13:13,346][105692] Updated weights for policy 0, policy_version 254983 (0.0010) [2023-12-26 17:13:13,877][105620] Updated weights for policy 1, policy_version 255307 (0.0009) [2023-12-26 17:13:13,940][105620] Updated weights for policy 1, policy_version 255317 (0.0009) [2023-12-26 17:13:13,997][105620] Updated weights for policy 1, policy_version 255327 (0.0008) [2023-12-26 17:13:14,003][105692] Updated weights for policy 0, policy_version 254993 (0.0006) [2023-12-26 17:13:14,060][105692] Updated weights for policy 0, policy_version 255003 (0.0006) [2023-12-26 17:13:14,119][105692] Updated weights for policy 0, policy_version 255013 (0.0009) [2023-12-26 17:13:14,709][105620] Updated weights for policy 1, policy_version 255337 (0.0008) [2023-12-26 17:13:14,778][105620] Updated weights for policy 1, policy_version 255347 (0.0009) [2023-12-26 17:13:14,839][105620] Updated weights for policy 1, policy_version 255357 (0.0008) [2023-12-26 17:13:14,900][105620] Updated weights for policy 1, policy_version 255367 (0.0008) [2023-12-26 17:13:14,922][105692] Updated weights for policy 0, policy_version 255023 (0.0008) [2023-12-26 17:13:14,991][105692] Updated weights for policy 0, policy_version 255033 (0.0008) [2023-12-26 17:13:15,040][105692] Updated weights for policy 0, policy_version 255043 (0.0008) [2023-12-26 17:13:15,629][105620] Updated weights for policy 1, policy_version 255377 (0.0005) [2023-12-26 17:13:15,678][105620] Updated weights for policy 1, policy_version 255387 (0.0006) [2023-12-26 17:13:15,733][105620] Updated weights for policy 1, policy_version 255397 (0.0006) [2023-12-26 17:13:15,814][105692] Updated weights for policy 0, policy_version 255053 (0.0009) [2023-12-26 17:13:15,867][105692] Updated weights for policy 0, policy_version 255064 (0.0010) [2023-12-26 17:13:15,914][105692] Updated weights for policy 0, policy_version 255074 (0.0009) [2023-12-26 17:13:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 130703360. Throughput: 0: 9629.1, 1: 9627.2. Samples: 130671196. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-26 17:13:16,062][104569] Avg episode reward: [(0, '9357.582'), (1, '8997.385')] [2023-12-26 17:13:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000255080_65314816.pth... [2023-12-26 17:13:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000255400_65388544.pth... [2023-12-26 17:13:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000254280_65101824.pth [2023-12-26 17:13:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000253928_65019904.pth [2023-12-26 17:13:16,420][105620] Updated weights for policy 1, policy_version 255407 (0.0008) [2023-12-26 17:13:16,483][105620] Updated weights for policy 1, policy_version 255417 (0.0006) [2023-12-26 17:13:16,538][105620] Updated weights for policy 1, policy_version 255427 (0.0006) [2023-12-26 17:13:16,671][105692] Updated weights for policy 0, policy_version 255084 (0.0009) [2023-12-26 17:13:16,720][105692] Updated weights for policy 0, policy_version 255094 (0.0006) [2023-12-26 17:13:16,780][105692] Updated weights for policy 0, policy_version 255104 (0.0006) [2023-12-26 17:13:17,257][105620] Updated weights for policy 1, policy_version 255437 (0.0007) [2023-12-26 17:13:17,310][105620] Updated weights for policy 1, policy_version 255447 (0.0005) [2023-12-26 17:13:17,360][105620] Updated weights for policy 1, policy_version 255457 (0.0005) [2023-12-26 17:13:17,426][105692] Updated weights for policy 0, policy_version 255114 (0.0006) [2023-12-26 17:13:17,481][105692] Updated weights for policy 0, policy_version 255124 (0.0009) [2023-12-26 17:13:17,529][105692] Updated weights for policy 0, policy_version 255134 (0.0005) [2023-12-26 17:13:17,584][105692] Updated weights for policy 0, policy_version 255144 (0.0005) [2023-12-26 17:13:17,906][105620] Updated weights for policy 1, policy_version 255467 (0.0006) [2023-12-26 17:13:17,956][105620] Updated weights for policy 1, policy_version 255477 (0.0005) [2023-12-26 17:13:18,015][105620] Updated weights for policy 1, policy_version 255487 (0.0005) [2023-12-26 17:13:18,327][105692] Updated weights for policy 0, policy_version 255154 (0.0009) [2023-12-26 17:13:18,378][105692] Updated weights for policy 0, policy_version 255164 (0.0008) [2023-12-26 17:13:18,440][105692] Updated weights for policy 0, policy_version 255174 (0.0008) [2023-12-26 17:13:18,679][105620] Updated weights for policy 1, policy_version 255497 (0.0005) [2023-12-26 17:13:18,749][105620] Updated weights for policy 1, policy_version 255507 (0.0006) [2023-12-26 17:13:18,818][105620] Updated weights for policy 1, policy_version 255517 (0.0008) [2023-12-26 17:13:18,877][105620] Updated weights for policy 1, policy_version 255527 (0.0009) [2023-12-26 17:13:19,221][105692] Updated weights for policy 0, policy_version 255184 (0.0010) [2023-12-26 17:13:19,297][105692] Updated weights for policy 0, policy_version 255194 (0.0009) [2023-12-26 17:13:19,362][105692] Updated weights for policy 0, policy_version 255204 (0.0009) [2023-12-26 17:13:19,572][105620] Updated weights for policy 1, policy_version 255537 (0.0009) [2023-12-26 17:13:19,639][105620] Updated weights for policy 1, policy_version 255547 (0.0005) [2023-12-26 17:13:19,705][105620] Updated weights for policy 1, policy_version 255557 (0.0006) [2023-12-26 17:13:20,084][105692] Updated weights for policy 0, policy_version 255214 (0.0009) [2023-12-26 17:13:20,149][105692] Updated weights for policy 0, policy_version 255224 (0.0009) [2023-12-26 17:13:20,214][105692] Updated weights for policy 0, policy_version 255234 (0.0009) [2023-12-26 17:13:20,402][105620] Updated weights for policy 1, policy_version 255567 (0.0008) [2023-12-26 17:13:20,462][105620] Updated weights for policy 1, policy_version 255577 (0.0008) [2023-12-26 17:13:20,527][105620] Updated weights for policy 1, policy_version 255587 (0.0006) [2023-12-26 17:13:20,988][105692] Updated weights for policy 0, policy_version 255244 (0.0009) [2023-12-26 17:13:21,060][105692] Updated weights for policy 0, policy_version 255254 (0.0008) [2023-12-26 17:13:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19605.3). Total num frames: 130793472. Throughput: 0: 9631.8, 1: 9682.0. Samples: 130788484. Policy #0 lag: (min: 20.0, avg: 27.0, max: 52.0) [2023-12-26 17:13:21,062][104569] Avg episode reward: [(0, '9268.042'), (1, '9086.869')] [2023-12-26 17:13:21,126][105692] Updated weights for policy 0, policy_version 255264 (0.0006) [2023-12-26 17:13:21,216][105620] Updated weights for policy 1, policy_version 255597 (0.0009) [2023-12-26 17:13:21,276][105620] Updated weights for policy 1, policy_version 255607 (0.0008) [2023-12-26 17:13:21,333][105620] Updated weights for policy 1, policy_version 255617 (0.0007) [2023-12-26 17:13:21,868][105692] Updated weights for policy 0, policy_version 255274 (0.0010) [2023-12-26 17:13:21,921][105692] Updated weights for policy 0, policy_version 255284 (0.0009) [2023-12-26 17:13:21,980][105692] Updated weights for policy 0, policy_version 255294 (0.0009) [2023-12-26 17:13:22,028][105620] Updated weights for policy 1, policy_version 255627 (0.0009) [2023-12-26 17:13:22,039][105692] Updated weights for policy 0, policy_version 255304 (0.0008) [2023-12-26 17:13:22,081][105620] Updated weights for policy 1, policy_version 255637 (0.0009) [2023-12-26 17:13:22,135][105620] Updated weights for policy 1, policy_version 255647 (0.0009) [2023-12-26 17:13:22,753][105692] Updated weights for policy 0, policy_version 255314 (0.0009) [2023-12-26 17:13:22,805][105692] Updated weights for policy 0, policy_version 255324 (0.0009) [2023-12-26 17:13:22,852][105692] Updated weights for policy 0, policy_version 255334 (0.0009) [2023-12-26 17:13:22,919][105620] Updated weights for policy 1, policy_version 255657 (0.0009) [2023-12-26 17:13:22,980][105620] Updated weights for policy 1, policy_version 255667 (0.0009) [2023-12-26 17:13:23,042][105620] Updated weights for policy 1, policy_version 255677 (0.0008) [2023-12-26 17:13:23,097][105620] Updated weights for policy 1, policy_version 255687 (0.0009) [2023-12-26 17:13:23,600][105692] Updated weights for policy 0, policy_version 255344 (0.0009) [2023-12-26 17:13:23,650][105692] Updated weights for policy 0, policy_version 255354 (0.0009) [2023-12-26 17:13:23,697][105692] Updated weights for policy 0, policy_version 255364 (0.0009) [2023-12-26 17:13:23,867][105620] Updated weights for policy 1, policy_version 255697 (0.0009) [2023-12-26 17:13:23,903][105586] KL-divergence is very high: 101.7961 [2023-12-26 17:13:23,916][105620] Updated weights for policy 1, policy_version 255707 (0.0008) [2023-12-26 17:13:23,938][105586] KL-divergence is very high: 130.6710 [2023-12-26 17:13:23,963][105620] Updated weights for policy 1, policy_version 255717 (0.0009) [2023-12-26 17:13:24,392][105692] Updated weights for policy 0, policy_version 255374 (0.0007) [2023-12-26 17:13:24,439][105692] Updated weights for policy 0, policy_version 255384 (0.0005) [2023-12-26 17:13:24,500][105692] Updated weights for policy 0, policy_version 255394 (0.0010) [2023-12-26 17:13:24,729][105620] Updated weights for policy 1, policy_version 255727 (0.0009) [2023-12-26 17:13:24,781][105620] Updated weights for policy 1, policy_version 255738 (0.0010) [2023-12-26 17:13:24,833][105620] Updated weights for policy 1, policy_version 255748 (0.0009) [2023-12-26 17:13:25,158][105692] Updated weights for policy 0, policy_version 255404 (0.0010) [2023-12-26 17:13:25,219][105692] Updated weights for policy 0, policy_version 255414 (0.0009) [2023-12-26 17:13:25,280][105692] Updated weights for policy 0, policy_version 255424 (0.0009) [2023-12-26 17:13:25,607][105620] Updated weights for policy 1, policy_version 255758 (0.0007) [2023-12-26 17:13:25,664][105620] Updated weights for policy 1, policy_version 255768 (0.0005) [2023-12-26 17:13:25,727][105620] Updated weights for policy 1, policy_version 255778 (0.0008) [2023-12-26 17:13:26,056][105692] Updated weights for policy 0, policy_version 255434 (0.0008) [2023-12-26 17:13:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 130891776. Throughput: 0: 9682.8, 1: 9660.6. Samples: 130902532. Policy #0 lag: (min: 20.0, avg: 27.0, max: 52.0) [2023-12-26 17:13:26,062][104569] Avg episode reward: [(0, '9267.839'), (1, '9098.462')] [2023-12-26 17:13:26,107][105692] Updated weights for policy 0, policy_version 255444 (0.0006) [2023-12-26 17:13:26,166][105692] Updated weights for policy 0, policy_version 255454 (0.0007) [2023-12-26 17:13:26,232][105692] Updated weights for policy 0, policy_version 255464 (0.0009) [2023-12-26 17:13:26,479][105620] Updated weights for policy 1, policy_version 255788 (0.0008) [2023-12-26 17:13:26,534][105620] Updated weights for policy 1, policy_version 255798 (0.0009) [2023-12-26 17:13:26,587][105620] Updated weights for policy 1, policy_version 255808 (0.0009) [2023-12-26 17:13:26,844][105692] Updated weights for policy 0, policy_version 255474 (0.0005) [2023-12-26 17:13:26,902][105692] Updated weights for policy 0, policy_version 255484 (0.0005) [2023-12-26 17:13:26,968][105692] Updated weights for policy 0, policy_version 255494 (0.0005) [2023-12-26 17:13:27,409][105620] Updated weights for policy 1, policy_version 255818 (0.0009) [2023-12-26 17:13:27,463][105620] Updated weights for policy 1, policy_version 255828 (0.0009) [2023-12-26 17:13:27,519][105620] Updated weights for policy 1, policy_version 255838 (0.0010) [2023-12-26 17:13:27,561][105692] Updated weights for policy 0, policy_version 255504 (0.0005) [2023-12-26 17:13:27,567][105620] Updated weights for policy 1, policy_version 255848 (0.0009) [2023-12-26 17:13:27,623][105692] Updated weights for policy 0, policy_version 255514 (0.0005) [2023-12-26 17:13:27,686][105692] Updated weights for policy 0, policy_version 255524 (0.0006) [2023-12-26 17:13:28,363][105620] Updated weights for policy 1, policy_version 255858 (0.0009) [2023-12-26 17:13:28,369][105692] Updated weights for policy 0, policy_version 255534 (0.0010) [2023-12-26 17:13:28,418][105692] Updated weights for policy 0, policy_version 255544 (0.0006) [2023-12-26 17:13:28,420][105620] Updated weights for policy 1, policy_version 255868 (0.0008) [2023-12-26 17:13:28,467][105692] Updated weights for policy 0, policy_version 255554 (0.0008) [2023-12-26 17:13:28,475][105620] Updated weights for policy 1, policy_version 255878 (0.0008) [2023-12-26 17:13:29,167][105620] Updated weights for policy 1, policy_version 255888 (0.0009) [2023-12-26 17:13:29,213][105620] Updated weights for policy 1, policy_version 255898 (0.0008) [2023-12-26 17:13:29,251][105692] Updated weights for policy 0, policy_version 255564 (0.0009) [2023-12-26 17:13:29,274][105620] Updated weights for policy 1, policy_version 255908 (0.0007) [2023-12-26 17:13:29,306][105692] Updated weights for policy 0, policy_version 255574 (0.0008) [2023-12-26 17:13:29,368][105692] Updated weights for policy 0, policy_version 255584 (0.0008) [2023-12-26 17:13:30,028][105692] Updated weights for policy 0, policy_version 255594 (0.0008) [2023-12-26 17:13:30,088][105620] Updated weights for policy 1, policy_version 255918 (0.0007) [2023-12-26 17:13:30,089][105692] Updated weights for policy 0, policy_version 255604 (0.0011) [2023-12-26 17:13:30,144][105692] Updated weights for policy 0, policy_version 255614 (0.0011) [2023-12-26 17:13:30,151][105620] Updated weights for policy 1, policy_version 255928 (0.0006) [2023-12-26 17:13:30,203][105620] Updated weights for policy 1, policy_version 255938 (0.0006) [2023-12-26 17:13:30,211][105692] Updated weights for policy 0, policy_version 255624 (0.0010) [2023-12-26 17:13:30,831][105620] Updated weights for policy 1, policy_version 255948 (0.0007) [2023-12-26 17:13:30,878][105620] Updated weights for policy 1, policy_version 255958 (0.0008) [2023-12-26 17:13:30,925][105620] Updated weights for policy 1, policy_version 255968 (0.0009) [2023-12-26 17:13:30,935][105692] Updated weights for policy 0, policy_version 255634 (0.0007) [2023-12-26 17:13:30,996][105692] Updated weights for policy 0, policy_version 255644 (0.0009) [2023-12-26 17:13:31,054][105692] Updated weights for policy 0, policy_version 255654 (0.0009) [2023-12-26 17:13:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 130990080. Throughput: 0: 9728.5, 1: 9630.9. Samples: 130960780. Policy #0 lag: (min: 20.0, avg: 27.0, max: 52.0) [2023-12-26 17:13:31,063][104569] Avg episode reward: [(0, '9267.697'), (1, '9280.226')] [2023-12-26 17:13:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000255656_65462272.pth... [2023-12-26 17:13:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000255976_65536000.pth... [2023-12-26 17:13:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000254536_65175552.pth [2023-12-26 17:13:31,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000254824_65241088.pth [2023-12-26 17:13:31,757][105620] Updated weights for policy 1, policy_version 255978 (0.0007) [2023-12-26 17:13:31,814][105620] Updated weights for policy 1, policy_version 255988 (0.0009) [2023-12-26 17:13:31,859][105692] Updated weights for policy 0, policy_version 255664 (0.0008) [2023-12-26 17:13:31,877][105620] Updated weights for policy 1, policy_version 255998 (0.0007) [2023-12-26 17:13:31,919][105692] Updated weights for policy 0, policy_version 255674 (0.0006) [2023-12-26 17:13:31,933][105620] Updated weights for policy 1, policy_version 256008 (0.0006) [2023-12-26 17:13:31,979][105692] Updated weights for policy 0, policy_version 255684 (0.0006) [2023-12-26 17:13:32,669][105692] Updated weights for policy 0, policy_version 255694 (0.0007) [2023-12-26 17:13:32,733][105620] Updated weights for policy 1, policy_version 256018 (0.0008) [2023-12-26 17:13:32,733][105692] Updated weights for policy 0, policy_version 255704 (0.0009) [2023-12-26 17:13:32,795][105692] Updated weights for policy 0, policy_version 255714 (0.0007) [2023-12-26 17:13:32,795][105620] Updated weights for policy 1, policy_version 256028 (0.0008) [2023-12-26 17:13:32,857][105620] Updated weights for policy 1, policy_version 256038 (0.0008) [2023-12-26 17:13:33,399][105692] Updated weights for policy 0, policy_version 255724 (0.0009) [2023-12-26 17:13:33,445][105692] Updated weights for policy 0, policy_version 255734 (0.0008) [2023-12-26 17:13:33,505][105692] Updated weights for policy 0, policy_version 255744 (0.0009) [2023-12-26 17:13:33,607][105620] Updated weights for policy 1, policy_version 256048 (0.0008) [2023-12-26 17:13:33,661][105620] Updated weights for policy 1, policy_version 256058 (0.0009) [2023-12-26 17:13:33,708][105620] Updated weights for policy 1, policy_version 256068 (0.0009) [2023-12-26 17:13:34,262][105692] Updated weights for policy 0, policy_version 255754 (0.0009) [2023-12-26 17:13:34,328][105692] Updated weights for policy 0, policy_version 255764 (0.0010) [2023-12-26 17:13:34,379][105692] Updated weights for policy 0, policy_version 255774 (0.0009) [2023-12-26 17:13:34,434][105620] Updated weights for policy 1, policy_version 256078 (0.0009) [2023-12-26 17:13:34,441][105692] Updated weights for policy 0, policy_version 255784 (0.0010) [2023-12-26 17:13:34,492][105620] Updated weights for policy 1, policy_version 256088 (0.0009) [2023-12-26 17:13:34,551][105620] Updated weights for policy 1, policy_version 256098 (0.0009) [2023-12-26 17:13:35,204][105692] Updated weights for policy 0, policy_version 255794 (0.0009) [2023-12-26 17:13:35,254][105692] Updated weights for policy 0, policy_version 255804 (0.0009) [2023-12-26 17:13:35,310][105620] Updated weights for policy 1, policy_version 256108 (0.0009) [2023-12-26 17:13:35,310][105692] Updated weights for policy 0, policy_version 255814 (0.0009) [2023-12-26 17:13:35,369][105620] Updated weights for policy 1, policy_version 256118 (0.0006) [2023-12-26 17:13:35,427][105620] Updated weights for policy 1, policy_version 256128 (0.0008) [2023-12-26 17:13:36,000][105692] Updated weights for policy 0, policy_version 255824 (0.0009) [2023-12-26 17:13:36,048][105692] Updated weights for policy 0, policy_version 255834 (0.0009) [2023-12-26 17:13:36,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19114.6, 300 sec: 19577.5). Total num frames: 131080192. Throughput: 0: 9697.6, 1: 9633.0. Samples: 131074888. Policy #0 lag: (min: 20.0, avg: 27.0, max: 52.0) [2023-12-26 17:13:36,063][104569] Avg episode reward: [(0, '9357.419'), (1, '9280.107')] [2023-12-26 17:13:36,106][105692] Updated weights for policy 0, policy_version 255844 (0.0009) [2023-12-26 17:13:36,153][105620] Updated weights for policy 1, policy_version 256138 (0.0007) [2023-12-26 17:13:36,208][105620] Updated weights for policy 1, policy_version 256148 (0.0009) [2023-12-26 17:13:36,267][105620] Updated weights for policy 1, policy_version 256158 (0.0009) [2023-12-26 17:13:36,322][105620] Updated weights for policy 1, policy_version 256168 (0.0009) [2023-12-26 17:13:36,788][105692] Updated weights for policy 0, policy_version 255854 (0.0007) [2023-12-26 17:13:36,836][105692] Updated weights for policy 0, policy_version 255864 (0.0009) [2023-12-26 17:13:36,893][105692] Updated weights for policy 0, policy_version 255874 (0.0008) [2023-12-26 17:13:37,181][105620] Updated weights for policy 1, policy_version 256178 (0.0009) [2023-12-26 17:13:37,239][105620] Updated weights for policy 1, policy_version 256189 (0.0010) [2023-12-26 17:13:37,291][105620] Updated weights for policy 1, policy_version 256199 (0.0009) [2023-12-26 17:13:37,558][105692] Updated weights for policy 0, policy_version 255884 (0.0009) [2023-12-26 17:13:37,618][105692] Updated weights for policy 0, policy_version 255894 (0.0009) [2023-12-26 17:13:37,682][105692] Updated weights for policy 0, policy_version 255904 (0.0007) [2023-12-26 17:13:38,152][105620] Updated weights for policy 1, policy_version 256209 (0.0009) [2023-12-26 17:13:38,201][105620] Updated weights for policy 1, policy_version 256219 (0.0009) [2023-12-26 17:13:38,254][105620] Updated weights for policy 1, policy_version 256230 (0.0010) [2023-12-26 17:13:38,309][105692] Updated weights for policy 0, policy_version 255914 (0.0006) [2023-12-26 17:13:38,371][105692] Updated weights for policy 0, policy_version 255924 (0.0008) [2023-12-26 17:13:38,423][105692] Updated weights for policy 0, policy_version 255934 (0.0008) [2023-12-26 17:13:38,472][105692] Updated weights for policy 0, policy_version 255944 (0.0009) [2023-12-26 17:13:39,074][105620] Updated weights for policy 1, policy_version 256240 (0.0008) [2023-12-26 17:13:39,097][105692] Updated weights for policy 0, policy_version 255954 (0.0005) [2023-12-26 17:13:39,141][105620] Updated weights for policy 1, policy_version 256250 (0.0008) [2023-12-26 17:13:39,148][105692] Updated weights for policy 0, policy_version 255964 (0.0005) [2023-12-26 17:13:39,195][105620] Updated weights for policy 1, policy_version 256260 (0.0008) [2023-12-26 17:13:39,197][105692] Updated weights for policy 0, policy_version 255974 (0.0009) [2023-12-26 17:13:39,893][105692] Updated weights for policy 0, policy_version 255984 (0.0010) [2023-12-26 17:13:39,950][105692] Updated weights for policy 0, policy_version 255994 (0.0009) [2023-12-26 17:13:39,992][105620] Updated weights for policy 1, policy_version 256270 (0.0006) [2023-12-26 17:13:40,002][105692] Updated weights for policy 0, policy_version 256004 (0.0011) [2023-12-26 17:13:40,051][105620] Updated weights for policy 1, policy_version 256280 (0.0007) [2023-12-26 17:13:40,100][105620] Updated weights for policy 1, policy_version 256290 (0.0007) [2023-12-26 17:13:40,700][105620] Updated weights for policy 1, policy_version 256300 (0.0006) [2023-12-26 17:13:40,752][105692] Updated weights for policy 0, policy_version 256014 (0.0011) [2023-12-26 17:13:40,757][105620] Updated weights for policy 1, policy_version 256310 (0.0006) [2023-12-26 17:13:40,802][105620] Updated weights for policy 1, policy_version 256320 (0.0006) [2023-12-26 17:13:40,811][105692] Updated weights for policy 0, policy_version 256024 (0.0009) [2023-12-26 17:13:40,873][105692] Updated weights for policy 0, policy_version 256034 (0.0007) [2023-12-26 17:13:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 131186688. Throughput: 0: 9769.9, 1: 9578.6. Samples: 131190932. Policy #0 lag: (min: 20.0, avg: 27.0, max: 52.0) [2023-12-26 17:13:41,063][104569] Avg episode reward: [(0, '9357.048'), (1, '8996.778')] [2023-12-26 17:13:41,476][105620] Updated weights for policy 1, policy_version 256330 (0.0007) [2023-12-26 17:13:41,532][105620] Updated weights for policy 1, policy_version 256340 (0.0008) [2023-12-26 17:13:41,584][105620] Updated weights for policy 1, policy_version 256350 (0.0009) [2023-12-26 17:13:41,649][105692] Updated weights for policy 0, policy_version 256044 (0.0010) [2023-12-26 17:13:41,650][105620] Updated weights for policy 1, policy_version 256360 (0.0009) [2023-12-26 17:13:41,712][105692] Updated weights for policy 0, policy_version 256054 (0.0008) [2023-12-26 17:13:41,785][105692] Updated weights for policy 0, policy_version 256064 (0.0007) [2023-12-26 17:13:42,415][105620] Updated weights for policy 1, policy_version 256370 (0.0008) [2023-12-26 17:13:42,475][105620] Updated weights for policy 1, policy_version 256380 (0.0008) [2023-12-26 17:13:42,507][105692] Updated weights for policy 0, policy_version 256074 (0.0008) [2023-12-26 17:13:42,535][105620] Updated weights for policy 1, policy_version 256390 (0.0009) [2023-12-26 17:13:42,568][105692] Updated weights for policy 0, policy_version 256084 (0.0005) [2023-12-26 17:13:42,632][105692] Updated weights for policy 0, policy_version 256094 (0.0005) [2023-12-26 17:13:42,691][105692] Updated weights for policy 0, policy_version 256104 (0.0005) [2023-12-26 17:13:43,263][105692] Updated weights for policy 0, policy_version 256114 (0.0009) [2023-12-26 17:13:43,308][105692] Updated weights for policy 0, policy_version 256124 (0.0009) [2023-12-26 17:13:43,359][105620] Updated weights for policy 1, policy_version 256400 (0.0008) [2023-12-26 17:13:43,361][105692] Updated weights for policy 0, policy_version 256134 (0.0008) [2023-12-26 17:13:43,415][105620] Updated weights for policy 1, policy_version 256410 (0.0008) [2023-12-26 17:13:43,474][105620] Updated weights for policy 1, policy_version 256420 (0.0011) [2023-12-26 17:13:44,138][105692] Updated weights for policy 0, policy_version 256144 (0.0009) [2023-12-26 17:13:44,191][105620] Updated weights for policy 1, policy_version 256430 (0.0008) [2023-12-26 17:13:44,191][105692] Updated weights for policy 0, policy_version 256154 (0.0008) [2023-12-26 17:13:44,247][105620] Updated weights for policy 1, policy_version 256440 (0.0010) [2023-12-26 17:13:44,251][105692] Updated weights for policy 0, policy_version 256164 (0.0008) [2023-12-26 17:13:44,273][105586] KL-divergence is very high: 107.0514 [2023-12-26 17:13:44,303][105620] Updated weights for policy 1, policy_version 256450 (0.0011) [2023-12-26 17:13:44,322][105586] KL-divergence is very high: 107.6360 [2023-12-26 17:13:45,016][105620] Updated weights for policy 1, policy_version 256460 (0.0011) [2023-12-26 17:13:45,082][105620] Updated weights for policy 1, policy_version 256470 (0.0007) [2023-12-26 17:13:45,086][105692] Updated weights for policy 0, policy_version 256174 (0.0010) [2023-12-26 17:13:45,145][105692] Updated weights for policy 0, policy_version 256184 (0.0011) [2023-12-26 17:13:45,149][105620] Updated weights for policy 1, policy_version 256480 (0.0010) [2023-12-26 17:13:45,206][105692] Updated weights for policy 0, policy_version 256194 (0.0011) [2023-12-26 17:13:45,823][105620] Updated weights for policy 1, policy_version 256490 (0.0010) [2023-12-26 17:13:45,889][105620] Updated weights for policy 1, policy_version 256500 (0.0006) [2023-12-26 17:13:45,954][105620] Updated weights for policy 1, policy_version 256510 (0.0007) [2023-12-26 17:13:45,971][105692] Updated weights for policy 0, policy_version 256204 (0.0011) [2023-12-26 17:13:46,018][105620] Updated weights for policy 1, policy_version 256520 (0.0005) [2023-12-26 17:13:46,033][105692] Updated weights for policy 0, policy_version 256214 (0.0010) [2023-12-26 17:13:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.1, 300 sec: 19549.7). Total num frames: 131276800. Throughput: 0: 9684.4, 1: 9600.8. Samples: 131247924. Policy #0 lag: (min: 20.0, avg: 27.0, max: 52.0) [2023-12-26 17:13:46,063][104569] Avg episode reward: [(0, '9356.707'), (1, '8907.307')] [2023-12-26 17:13:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000256520_65675264.pth... [2023-12-26 17:13:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000255400_65388544.pth [2023-12-26 17:13:46,099][105692] Updated weights for policy 0, policy_version 256224 (0.0010) [2023-12-26 17:13:46,147][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000256232_65609728.pth... [2023-12-26 17:13:46,151][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000255080_65314816.pth [2023-12-26 17:13:46,612][105620] Updated weights for policy 1, policy_version 256530 (0.0010) [2023-12-26 17:13:46,662][105620] Updated weights for policy 1, policy_version 256540 (0.0010) [2023-12-26 17:13:46,726][105620] Updated weights for policy 1, policy_version 256550 (0.0010) [2023-12-26 17:13:46,837][105692] Updated weights for policy 0, policy_version 256234 (0.0011) [2023-12-26 17:13:46,882][105692] Updated weights for policy 0, policy_version 256244 (0.0010) [2023-12-26 17:13:46,936][105692] Updated weights for policy 0, policy_version 256254 (0.0010) [2023-12-26 17:13:46,984][105692] Updated weights for policy 0, policy_version 256264 (0.0008) [2023-12-26 17:13:47,374][105620] Updated weights for policy 1, policy_version 256560 (0.0006) [2023-12-26 17:13:47,434][105620] Updated weights for policy 1, policy_version 256570 (0.0009) [2023-12-26 17:13:47,485][105620] Updated weights for policy 1, policy_version 256580 (0.0010) [2023-12-26 17:13:47,708][105692] Updated weights for policy 0, policy_version 256274 (0.0005) [2023-12-26 17:13:47,763][105692] Updated weights for policy 0, policy_version 256284 (0.0005) [2023-12-26 17:13:47,819][105692] Updated weights for policy 0, policy_version 256294 (0.0005) [2023-12-26 17:13:48,076][105620] Updated weights for policy 1, policy_version 256590 (0.0007) [2023-12-26 17:13:48,136][105620] Updated weights for policy 1, policy_version 256600 (0.0010) [2023-12-26 17:13:48,200][105620] Updated weights for policy 1, policy_version 256610 (0.0010) [2023-12-26 17:13:48,360][105692] Updated weights for policy 0, policy_version 256304 (0.0009) [2023-12-26 17:13:48,419][105692] Updated weights for policy 0, policy_version 256314 (0.0010) [2023-12-26 17:13:48,492][105692] Updated weights for policy 0, policy_version 256324 (0.0009) [2023-12-26 17:13:48,895][105620] Updated weights for policy 1, policy_version 256620 (0.0010) [2023-12-26 17:13:48,960][105620] Updated weights for policy 1, policy_version 256630 (0.0009) [2023-12-26 17:13:49,017][105620] Updated weights for policy 1, policy_version 256640 (0.0009) [2023-12-26 17:13:49,134][105692] Updated weights for policy 0, policy_version 256334 (0.0006) [2023-12-26 17:13:49,191][105692] Updated weights for policy 0, policy_version 256344 (0.0010) [2023-12-26 17:13:49,254][105692] Updated weights for policy 0, policy_version 256354 (0.0010) [2023-12-26 17:13:49,756][105620] Updated weights for policy 1, policy_version 256650 (0.0009) [2023-12-26 17:13:49,811][105620] Updated weights for policy 1, policy_version 256660 (0.0010) [2023-12-26 17:13:49,875][105620] Updated weights for policy 1, policy_version 256670 (0.0007) [2023-12-26 17:13:49,943][105620] Updated weights for policy 1, policy_version 256680 (0.0009) [2023-12-26 17:13:49,946][105692] Updated weights for policy 0, policy_version 256364 (0.0011) [2023-12-26 17:13:50,001][105692] Updated weights for policy 0, policy_version 256374 (0.0011) [2023-12-26 17:13:50,057][105692] Updated weights for policy 0, policy_version 256384 (0.0011) [2023-12-26 17:13:50,604][105620] Updated weights for policy 1, policy_version 256690 (0.0011) [2023-12-26 17:13:50,667][105620] Updated weights for policy 1, policy_version 256700 (0.0006) [2023-12-26 17:13:50,727][105620] Updated weights for policy 1, policy_version 256710 (0.0010) [2023-12-26 17:13:50,751][105692] Updated weights for policy 0, policy_version 256394 (0.0010) [2023-12-26 17:13:50,820][105692] Updated weights for policy 0, policy_version 256404 (0.0007) [2023-12-26 17:13:50,885][105692] Updated weights for policy 0, policy_version 256414 (0.0008) [2023-12-26 17:13:50,937][105692] Updated weights for policy 0, policy_version 256424 (0.0010) [2023-12-26 17:13:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 131383296. Throughput: 0: 9712.8, 1: 9600.1. Samples: 131367960. Policy #0 lag: (min: 20.0, avg: 27.0, max: 52.0) [2023-12-26 17:13:51,062][104569] Avg episode reward: [(0, '9356.709'), (1, '9179.925')] [2023-12-26 17:13:51,397][105620] Updated weights for policy 1, policy_version 256720 (0.0009) [2023-12-26 17:13:51,446][105620] Updated weights for policy 1, policy_version 256730 (0.0010) [2023-12-26 17:13:51,491][105620] Updated weights for policy 1, policy_version 256740 (0.0010) [2023-12-26 17:13:51,774][105692] Updated weights for policy 0, policy_version 256434 (0.0007) [2023-12-26 17:13:51,827][105692] Updated weights for policy 0, policy_version 256444 (0.0010) [2023-12-26 17:13:51,892][105692] Updated weights for policy 0, policy_version 256454 (0.0011) [2023-12-26 17:13:52,215][105620] Updated weights for policy 1, policy_version 256750 (0.0011) [2023-12-26 17:13:52,283][105620] Updated weights for policy 1, policy_version 256760 (0.0010) [2023-12-26 17:13:52,352][105620] Updated weights for policy 1, policy_version 256770 (0.0011) [2023-12-26 17:13:52,554][105692] Updated weights for policy 0, policy_version 256464 (0.0011) [2023-12-26 17:13:52,620][105692] Updated weights for policy 0, policy_version 256474 (0.0007) [2023-12-26 17:13:52,687][105692] Updated weights for policy 0, policy_version 256484 (0.0009) [2023-12-26 17:13:53,102][105620] Updated weights for policy 1, policy_version 256780 (0.0007) [2023-12-26 17:13:53,155][105620] Updated weights for policy 1, policy_version 256790 (0.0005) [2023-12-26 17:13:53,212][105620] Updated weights for policy 1, policy_version 256800 (0.0005) [2023-12-26 17:13:53,373][105692] Updated weights for policy 0, policy_version 256494 (0.0010) [2023-12-26 17:13:53,420][105692] Updated weights for policy 0, policy_version 256504 (0.0010) [2023-12-26 17:13:53,467][105692] Updated weights for policy 0, policy_version 256514 (0.0010) [2023-12-26 17:13:53,888][105620] Updated weights for policy 1, policy_version 256810 (0.0006) [2023-12-26 17:13:53,924][105586] KL-divergence is very high: 172.7187 [2023-12-26 17:13:53,943][105620] Updated weights for policy 1, policy_version 256820 (0.0008) [2023-12-26 17:13:53,969][105586] KL-divergence is very high: 314.3325 [2023-12-26 17:13:53,997][105620] Updated weights for policy 1, policy_version 256830 (0.0006) [2023-12-26 17:13:54,014][105586] KL-divergence is very high: 324.4080 [2023-12-26 17:13:54,054][105620] Updated weights for policy 1, policy_version 256840 (0.0005) [2023-12-26 17:13:54,227][105692] Updated weights for policy 0, policy_version 256524 (0.0010) [2023-12-26 17:13:54,285][105692] Updated weights for policy 0, policy_version 256534 (0.0010) [2023-12-26 17:13:54,337][105692] Updated weights for policy 0, policy_version 256544 (0.0010) [2023-12-26 17:13:54,697][105620] Updated weights for policy 1, policy_version 256850 (0.0007) [2023-12-26 17:13:54,751][105620] Updated weights for policy 1, policy_version 256860 (0.0009) [2023-12-26 17:13:54,804][105620] Updated weights for policy 1, policy_version 256870 (0.0009) [2023-12-26 17:13:55,028][105692] Updated weights for policy 0, policy_version 256554 (0.0010) [2023-12-26 17:13:55,096][105692] Updated weights for policy 0, policy_version 256564 (0.0008) [2023-12-26 17:13:55,161][105692] Updated weights for policy 0, policy_version 256574 (0.0007) [2023-12-26 17:13:55,224][105692] Updated weights for policy 0, policy_version 256584 (0.0005) [2023-12-26 17:13:55,627][105620] Updated weights for policy 1, policy_version 256880 (0.0009) [2023-12-26 17:13:55,679][105620] Updated weights for policy 1, policy_version 256890 (0.0007) [2023-12-26 17:13:55,739][105620] Updated weights for policy 1, policy_version 256900 (0.0005) [2023-12-26 17:13:55,809][105692] Updated weights for policy 0, policy_version 256594 (0.0008) [2023-12-26 17:13:55,857][105692] Updated weights for policy 0, policy_version 256604 (0.0005) [2023-12-26 17:13:55,908][105692] Updated weights for policy 0, policy_version 256614 (0.0005) [2023-12-26 17:13:56,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 131481600. Throughput: 0: 9785.1, 1: 9627.5. Samples: 131486868. Policy #0 lag: (min: 20.0, avg: 27.0, max: 52.0) [2023-12-26 17:13:56,062][104569] Avg episode reward: [(0, '9356.683'), (1, '8822.571')] [2023-12-26 17:13:56,404][105620] Updated weights for policy 1, policy_version 256910 (0.0008) [2023-12-26 17:13:56,457][105620] Updated weights for policy 1, policy_version 256920 (0.0010) [2023-12-26 17:13:56,513][105620] Updated weights for policy 1, policy_version 256930 (0.0010) [2023-12-26 17:13:56,556][105692] Updated weights for policy 0, policy_version 256624 (0.0006) [2023-12-26 17:13:56,611][105692] Updated weights for policy 0, policy_version 256634 (0.0010) [2023-12-26 17:13:56,667][105692] Updated weights for policy 0, policy_version 256644 (0.0010) [2023-12-26 17:13:57,273][105620] Updated weights for policy 1, policy_version 256940 (0.0010) [2023-12-26 17:13:57,327][105620] Updated weights for policy 1, policy_version 256950 (0.0005) [2023-12-26 17:13:57,375][105692] Updated weights for policy 0, policy_version 256654 (0.0009) [2023-12-26 17:13:57,378][105620] Updated weights for policy 1, policy_version 256960 (0.0006) [2023-12-26 17:13:57,433][105692] Updated weights for policy 0, policy_version 256664 (0.0010) [2023-12-26 17:13:57,488][105692] Updated weights for policy 0, policy_version 256674 (0.0010) [2023-12-26 17:13:57,991][105620] Updated weights for policy 1, policy_version 256970 (0.0005) [2023-12-26 17:13:58,038][105620] Updated weights for policy 1, policy_version 256980 (0.0005) [2023-12-26 17:13:58,089][105620] Updated weights for policy 1, policy_version 256990 (0.0006) [2023-12-26 17:13:58,127][105692] Updated weights for policy 0, policy_version 256684 (0.0009) [2023-12-26 17:13:58,150][105620] Updated weights for policy 1, policy_version 257000 (0.0007) [2023-12-26 17:13:58,190][105692] Updated weights for policy 0, policy_version 256694 (0.0008) [2023-12-26 17:13:58,228][105585] KL-divergence is very high: 186.5509 [2023-12-26 17:13:58,254][105692] Updated weights for policy 0, policy_version 256704 (0.0008) [2023-12-26 17:13:58,278][105585] KL-divergence is very high: 278.3432 [2023-12-26 17:13:58,932][105620] Updated weights for policy 1, policy_version 257010 (0.0006) [2023-12-26 17:13:58,998][105620] Updated weights for policy 1, policy_version 257020 (0.0006) [2023-12-26 17:13:59,033][105692] Updated weights for policy 0, policy_version 256714 (0.0009) [2023-12-26 17:13:59,056][105620] Updated weights for policy 1, policy_version 257030 (0.0006) [2023-12-26 17:13:59,087][105692] Updated weights for policy 0, policy_version 256724 (0.0010) [2023-12-26 17:13:59,145][105692] Updated weights for policy 0, policy_version 256734 (0.0010) [2023-12-26 17:13:59,207][105692] Updated weights for policy 0, policy_version 256744 (0.0010) [2023-12-26 17:13:59,682][105620] Updated weights for policy 1, policy_version 257040 (0.0005) [2023-12-26 17:13:59,728][105620] Updated weights for policy 1, policy_version 257050 (0.0005) [2023-12-26 17:13:59,784][105620] Updated weights for policy 1, policy_version 257060 (0.0005) [2023-12-26 17:13:59,965][105692] Updated weights for policy 0, policy_version 256754 (0.0008) [2023-12-26 17:14:00,022][105692] Updated weights for policy 0, policy_version 256764 (0.0006) [2023-12-26 17:14:00,067][105692] Updated weights for policy 0, policy_version 256774 (0.0005) [2023-12-26 17:14:00,356][105620] Updated weights for policy 1, policy_version 257070 (0.0005) [2023-12-26 17:14:00,420][105620] Updated weights for policy 1, policy_version 257080 (0.0005) [2023-12-26 17:14:00,480][105620] Updated weights for policy 1, policy_version 257090 (0.0005) [2023-12-26 17:14:00,765][105692] Updated weights for policy 0, policy_version 256784 (0.0007) [2023-12-26 17:14:00,830][105692] Updated weights for policy 0, policy_version 256794 (0.0010) [2023-12-26 17:14:00,884][105692] Updated weights for policy 0, policy_version 256804 (0.0010) [2023-12-26 17:14:01,051][105620] Updated weights for policy 1, policy_version 257100 (0.0007) [2023-12-26 17:14:01,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19605.2). Total num frames: 131579904. Throughput: 0: 9820.9, 1: 9650.2. Samples: 131547400. Policy #0 lag: (min: 20.0, avg: 27.0, max: 52.0) [2023-12-26 17:14:01,063][104569] Avg episode reward: [(0, '9266.984'), (1, '8828.479')] [2023-12-26 17:14:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000256808_65757184.pth... [2023-12-26 17:14:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000255656_65462272.pth [2023-12-26 17:14:01,114][105620] Updated weights for policy 1, policy_version 257110 (0.0008) [2023-12-26 17:14:01,169][105620] Updated weights for policy 1, policy_version 257120 (0.0007) [2023-12-26 17:14:01,219][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000257128_65830912.pth... [2023-12-26 17:14:01,223][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000255976_65536000.pth [2023-12-26 17:14:01,604][105692] Updated weights for policy 0, policy_version 256814 (0.0007) [2023-12-26 17:14:01,669][105692] Updated weights for policy 0, policy_version 256824 (0.0012) [2023-12-26 17:14:01,737][105692] Updated weights for policy 0, policy_version 256834 (0.0009) [2023-12-26 17:14:01,850][105620] Updated weights for policy 1, policy_version 257130 (0.0006) [2023-12-26 17:14:01,910][105620] Updated weights for policy 1, policy_version 257140 (0.0009) [2023-12-26 17:14:01,970][105620] Updated weights for policy 1, policy_version 257150 (0.0010) [2023-12-26 17:14:02,030][105620] Updated weights for policy 1, policy_version 257160 (0.0008) [2023-12-26 17:14:02,450][105692] Updated weights for policy 0, policy_version 256844 (0.0010) [2023-12-26 17:14:02,508][105692] Updated weights for policy 0, policy_version 256854 (0.0010) [2023-12-26 17:14:02,566][105692] Updated weights for policy 0, policy_version 256864 (0.0010) [2023-12-26 17:14:02,796][105620] Updated weights for policy 1, policy_version 257170 (0.0011) [2023-12-26 17:14:02,841][105620] Updated weights for policy 1, policy_version 257180 (0.0010) [2023-12-26 17:14:02,885][105620] Updated weights for policy 1, policy_version 257190 (0.0010) [2023-12-26 17:14:03,239][105692] Updated weights for policy 0, policy_version 256874 (0.0009) [2023-12-26 17:14:03,299][105692] Updated weights for policy 0, policy_version 256884 (0.0006) [2023-12-26 17:14:03,359][105692] Updated weights for policy 0, policy_version 256894 (0.0008) [2023-12-26 17:14:03,410][105692] Updated weights for policy 0, policy_version 256904 (0.0010) [2023-12-26 17:14:03,595][105620] Updated weights for policy 1, policy_version 257200 (0.0010) [2023-12-26 17:14:03,639][105620] Updated weights for policy 1, policy_version 257210 (0.0010) [2023-12-26 17:14:03,700][105620] Updated weights for policy 1, policy_version 257220 (0.0010) [2023-12-26 17:14:04,062][105692] Updated weights for policy 0, policy_version 256914 (0.0009) [2023-12-26 17:14:04,127][105692] Updated weights for policy 0, policy_version 256924 (0.0006) [2023-12-26 17:14:04,191][105692] Updated weights for policy 0, policy_version 256934 (0.0010) [2023-12-26 17:14:04,437][105620] Updated weights for policy 1, policy_version 257230 (0.0011) [2023-12-26 17:14:04,504][105620] Updated weights for policy 1, policy_version 257240 (0.0011) [2023-12-26 17:14:04,570][105620] Updated weights for policy 1, policy_version 257250 (0.0011) [2023-12-26 17:14:04,844][105692] Updated weights for policy 0, policy_version 256944 (0.0010) [2023-12-26 17:14:04,898][105692] Updated weights for policy 0, policy_version 256954 (0.0010) [2023-12-26 17:14:04,952][105692] Updated weights for policy 0, policy_version 256964 (0.0010) [2023-12-26 17:14:05,264][105620] Updated weights for policy 1, policy_version 257260 (0.0009) [2023-12-26 17:14:05,320][105620] Updated weights for policy 1, policy_version 257270 (0.0005) [2023-12-26 17:14:05,388][105620] Updated weights for policy 1, policy_version 257280 (0.0005) [2023-12-26 17:14:05,588][105692] Updated weights for policy 0, policy_version 256974 (0.0009) [2023-12-26 17:14:05,632][105692] Updated weights for policy 0, policy_version 256984 (0.0010) [2023-12-26 17:14:05,685][105692] Updated weights for policy 0, policy_version 256994 (0.0010) [2023-12-26 17:14:05,990][105620] Updated weights for policy 1, policy_version 257290 (0.0005) [2023-12-26 17:14:06,048][105620] Updated weights for policy 1, policy_version 257300 (0.0005) [2023-12-26 17:14:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 131678208. Throughput: 0: 9898.6, 1: 9662.6. Samples: 131668736. Policy #0 lag: (min: 20.0, avg: 27.0, max: 52.0) [2023-12-26 17:14:06,062][104569] Avg episode reward: [(0, '9356.582'), (1, '8877.637')] [2023-12-26 17:14:06,108][105620] Updated weights for policy 1, policy_version 257310 (0.0006) [2023-12-26 17:14:06,170][105620] Updated weights for policy 1, policy_version 257320 (0.0007) [2023-12-26 17:14:06,454][105692] Updated weights for policy 0, policy_version 257004 (0.0010) [2023-12-26 17:14:06,510][105692] Updated weights for policy 0, policy_version 257014 (0.0010) [2023-12-26 17:14:06,570][105692] Updated weights for policy 0, policy_version 257024 (0.0010) [2023-12-26 17:14:06,754][105620] Updated weights for policy 1, policy_version 257330 (0.0006) [2023-12-26 17:14:06,813][105620] Updated weights for policy 1, policy_version 257340 (0.0005) [2023-12-26 17:14:06,878][105620] Updated weights for policy 1, policy_version 257350 (0.0005) [2023-12-26 17:14:07,337][105692] Updated weights for policy 0, policy_version 257034 (0.0009) [2023-12-26 17:14:07,394][105692] Updated weights for policy 0, policy_version 257044 (0.0005) [2023-12-26 17:14:07,443][105692] Updated weights for policy 0, policy_version 257054 (0.0005) [2023-12-26 17:14:07,504][105620] Updated weights for policy 1, policy_version 257360 (0.0009) [2023-12-26 17:14:07,508][105692] Updated weights for policy 0, policy_version 257064 (0.0005) [2023-12-26 17:14:07,570][105620] Updated weights for policy 1, policy_version 257370 (0.0011) [2023-12-26 17:14:07,633][105620] Updated weights for policy 1, policy_version 257380 (0.0011) [2023-12-26 17:14:08,124][105692] Updated weights for policy 0, policy_version 257074 (0.0010) [2023-12-26 17:14:08,183][105692] Updated weights for policy 0, policy_version 257084 (0.0011) [2023-12-26 17:14:08,243][105692] Updated weights for policy 0, policy_version 257094 (0.0011) [2023-12-26 17:14:08,280][105620] Updated weights for policy 1, policy_version 257390 (0.0009) [2023-12-26 17:14:08,343][105620] Updated weights for policy 1, policy_version 257400 (0.0011) [2023-12-26 17:14:08,402][105620] Updated weights for policy 1, policy_version 257410 (0.0011) [2023-12-26 17:14:09,005][105692] Updated weights for policy 0, policy_version 257104 (0.0010) [2023-12-26 17:14:09,066][105692] Updated weights for policy 0, policy_version 257114 (0.0010) [2023-12-26 17:14:09,114][105620] Updated weights for policy 1, policy_version 257420 (0.0010) [2023-12-26 17:14:09,122][105692] Updated weights for policy 0, policy_version 257124 (0.0007) [2023-12-26 17:14:09,175][105620] Updated weights for policy 1, policy_version 257430 (0.0006) [2023-12-26 17:14:09,243][105620] Updated weights for policy 1, policy_version 257440 (0.0006) [2023-12-26 17:14:09,850][105692] Updated weights for policy 0, policy_version 257134 (0.0009) [2023-12-26 17:14:09,900][105692] Updated weights for policy 0, policy_version 257144 (0.0011) [2023-12-26 17:14:09,972][105692] Updated weights for policy 0, policy_version 257154 (0.0011) [2023-12-26 17:14:09,973][105620] Updated weights for policy 1, policy_version 257450 (0.0009) [2023-12-26 17:14:10,041][105620] Updated weights for policy 1, policy_version 257460 (0.0006) [2023-12-26 17:14:10,104][105620] Updated weights for policy 1, policy_version 257470 (0.0006) [2023-12-26 17:14:10,172][105620] Updated weights for policy 1, policy_version 257480 (0.0006) [2023-12-26 17:14:10,726][105692] Updated weights for policy 0, policy_version 257164 (0.0011) [2023-12-26 17:14:10,788][105692] Updated weights for policy 0, policy_version 257174 (0.0008) [2023-12-26 17:14:10,794][105620] Updated weights for policy 1, policy_version 257490 (0.0007) [2023-12-26 17:14:10,846][105620] Updated weights for policy 1, policy_version 257500 (0.0007) [2023-12-26 17:14:10,850][105692] Updated weights for policy 0, policy_version 257184 (0.0007) [2023-12-26 17:14:10,897][105620] Updated weights for policy 1, policy_version 257510 (0.0008) [2023-12-26 17:14:11,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 131784704. Throughput: 0: 9924.1, 1: 9787.7. Samples: 131789560. Policy #0 lag: (min: 20.0, avg: 27.0, max: 52.0) [2023-12-26 17:14:11,062][104569] Avg episode reward: [(0, '9356.673'), (1, '8697.608')] [2023-12-26 17:14:11,631][105692] Updated weights for policy 0, policy_version 257194 (0.0007) [2023-12-26 17:14:11,671][105620] Updated weights for policy 1, policy_version 257520 (0.0008) [2023-12-26 17:14:11,693][105692] Updated weights for policy 0, policy_version 257204 (0.0008) [2023-12-26 17:14:11,733][105620] Updated weights for policy 1, policy_version 257530 (0.0007) [2023-12-26 17:14:11,760][105692] Updated weights for policy 0, policy_version 257214 (0.0007) [2023-12-26 17:14:11,797][105620] Updated weights for policy 1, policy_version 257540 (0.0009) [2023-12-26 17:14:11,824][105692] Updated weights for policy 0, policy_version 257224 (0.0008) [2023-12-26 17:14:12,532][105620] Updated weights for policy 1, policy_version 257550 (0.0008) [2023-12-26 17:14:12,570][105692] Updated weights for policy 0, policy_version 257234 (0.0006) [2023-12-26 17:14:12,584][105620] Updated weights for policy 1, policy_version 257560 (0.0009) [2023-12-26 17:14:12,626][105692] Updated weights for policy 0, policy_version 257244 (0.0006) [2023-12-26 17:14:12,639][105620] Updated weights for policy 1, policy_version 257570 (0.0008) [2023-12-26 17:14:12,681][105692] Updated weights for policy 0, policy_version 257254 (0.0006) [2023-12-26 17:14:13,246][105620] Updated weights for policy 1, policy_version 257580 (0.0008) [2023-12-26 17:14:13,301][105620] Updated weights for policy 1, policy_version 257590 (0.0009) [2023-12-26 17:14:13,345][105620] Updated weights for policy 1, policy_version 257600 (0.0006) [2023-12-26 17:14:13,381][105692] Updated weights for policy 0, policy_version 257264 (0.0005) [2023-12-26 17:14:13,442][105692] Updated weights for policy 0, policy_version 257274 (0.0010) [2023-12-26 17:14:13,499][105692] Updated weights for policy 0, policy_version 257285 (0.0010) [2023-12-26 17:14:14,030][105620] Updated weights for policy 1, policy_version 257610 (0.0005) [2023-12-26 17:14:14,079][105620] Updated weights for policy 1, policy_version 257620 (0.0007) [2023-12-26 17:14:14,136][105620] Updated weights for policy 1, policy_version 257631 (0.0010) [2023-12-26 17:14:14,148][105692] Updated weights for policy 0, policy_version 257295 (0.0007) [2023-12-26 17:14:14,201][105692] Updated weights for policy 0, policy_version 257305 (0.0005) [2023-12-26 17:14:14,265][105692] Updated weights for policy 0, policy_version 257315 (0.0005) [2023-12-26 17:14:14,837][105692] Updated weights for policy 0, policy_version 257325 (0.0007) [2023-12-26 17:14:14,902][105692] Updated weights for policy 0, policy_version 257335 (0.0008) [2023-12-26 17:14:14,943][105620] Updated weights for policy 1, policy_version 257641 (0.0008) [2023-12-26 17:14:14,961][105692] Updated weights for policy 0, policy_version 257345 (0.0008) [2023-12-26 17:14:14,997][105620] Updated weights for policy 1, policy_version 257651 (0.0005) [2023-12-26 17:14:15,052][105620] Updated weights for policy 1, policy_version 257661 (0.0007) [2023-12-26 17:14:15,104][105620] Updated weights for policy 1, policy_version 257671 (0.0008) [2023-12-26 17:14:15,717][105692] Updated weights for policy 0, policy_version 257355 (0.0009) [2023-12-26 17:14:15,775][105692] Updated weights for policy 0, policy_version 257365 (0.0010) [2023-12-26 17:14:15,825][105692] Updated weights for policy 0, policy_version 257375 (0.0006) [2023-12-26 17:14:15,888][105620] Updated weights for policy 1, policy_version 257681 (0.0008) [2023-12-26 17:14:15,946][105620] Updated weights for policy 1, policy_version 257691 (0.0008) [2023-12-26 17:14:16,016][105620] Updated weights for policy 1, policy_version 257701 (0.0010) [2023-12-26 17:14:16,023][105586] KL-divergence is very high: 254.7423 [2023-12-26 17:14:16,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 131883008. Throughput: 0: 9846.7, 1: 9859.6. Samples: 131847560. Policy #0 lag: (min: 20.0, avg: 27.0, max: 52.0) [2023-12-26 17:14:16,062][104569] Avg episode reward: [(0, '9356.860'), (1, '8991.590')] [2023-12-26 17:14:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000257384_65904640.pth... [2023-12-26 17:14:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000257704_65978368.pth... [2023-12-26 17:14:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000256520_65675264.pth [2023-12-26 17:14:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000256232_65609728.pth [2023-12-26 17:14:16,470][105692] Updated weights for policy 0, policy_version 257385 (0.0005) [2023-12-26 17:14:16,523][105692] Updated weights for policy 0, policy_version 257395 (0.0007) [2023-12-26 17:14:16,576][105692] Updated weights for policy 0, policy_version 257405 (0.0005) [2023-12-26 17:14:16,639][105692] Updated weights for policy 0, policy_version 257415 (0.0007) [2023-12-26 17:14:16,793][105586] KL-divergence is very high: 380.1691 [2023-12-26 17:14:16,802][105620] Updated weights for policy 1, policy_version 257711 (0.0008) [2023-12-26 17:14:16,829][105586] KL-divergence is very high: 421.5755 [2023-12-26 17:14:16,849][105620] Updated weights for policy 1, policy_version 257721 (0.0007) [2023-12-26 17:14:16,865][105586] KL-divergence is very high: 366.7588 [2023-12-26 17:14:16,897][105620] Updated weights for policy 1, policy_version 257731 (0.0008) [2023-12-26 17:14:16,907][105586] KL-divergence is very high: 349.6291 [2023-12-26 17:14:17,353][105692] Updated weights for policy 0, policy_version 257425 (0.0010) [2023-12-26 17:14:17,411][105692] Updated weights for policy 0, policy_version 257435 (0.0010) [2023-12-26 17:14:17,458][105692] Updated weights for policy 0, policy_version 257445 (0.0010) [2023-12-26 17:14:17,696][105620] Updated weights for policy 1, policy_version 257741 (0.0008) [2023-12-26 17:14:17,751][105620] Updated weights for policy 1, policy_version 257751 (0.0008) [2023-12-26 17:14:17,809][105620] Updated weights for policy 1, policy_version 257761 (0.0008) [2023-12-26 17:14:18,201][105692] Updated weights for policy 0, policy_version 257455 (0.0010) [2023-12-26 17:14:18,256][105692] Updated weights for policy 0, policy_version 257465 (0.0010) [2023-12-26 17:14:18,317][105692] Updated weights for policy 0, policy_version 257475 (0.0010) [2023-12-26 17:14:18,494][105620] Updated weights for policy 1, policy_version 257771 (0.0007) [2023-12-26 17:14:18,552][105620] Updated weights for policy 1, policy_version 257781 (0.0006) [2023-12-26 17:14:18,606][105620] Updated weights for policy 1, policy_version 257791 (0.0005) [2023-12-26 17:14:19,017][105692] Updated weights for policy 0, policy_version 257485 (0.0008) [2023-12-26 17:14:19,066][105692] Updated weights for policy 0, policy_version 257495 (0.0005) [2023-12-26 17:14:19,115][105692] Updated weights for policy 0, policy_version 257505 (0.0005) [2023-12-26 17:14:19,347][105620] Updated weights for policy 1, policy_version 257801 (0.0006) [2023-12-26 17:14:19,414][105620] Updated weights for policy 1, policy_version 257811 (0.0009) [2023-12-26 17:14:19,467][105620] Updated weights for policy 1, policy_version 257821 (0.0006) [2023-12-26 17:14:19,531][105620] Updated weights for policy 1, policy_version 257831 (0.0007) [2023-12-26 17:14:19,855][105692] Updated weights for policy 0, policy_version 257515 (0.0006) [2023-12-26 17:14:19,914][105692] Updated weights for policy 0, policy_version 257525 (0.0009) [2023-12-26 17:14:19,984][105692] Updated weights for policy 0, policy_version 257535 (0.0006) [2023-12-26 17:14:20,225][105620] Updated weights for policy 1, policy_version 257841 (0.0009) [2023-12-26 17:14:20,274][105620] Updated weights for policy 1, policy_version 257851 (0.0010) [2023-12-26 17:14:20,337][105620] Updated weights for policy 1, policy_version 257861 (0.0011) [2023-12-26 17:14:20,734][105692] Updated weights for policy 0, policy_version 257545 (0.0008) [2023-12-26 17:14:20,785][105692] Updated weights for policy 0, policy_version 257555 (0.0008) [2023-12-26 17:14:20,843][105692] Updated weights for policy 0, policy_version 257565 (0.0009) [2023-12-26 17:14:20,905][105692] Updated weights for policy 0, policy_version 257575 (0.0009) [2023-12-26 17:14:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 131973120. Throughput: 0: 9910.0, 1: 9853.5. Samples: 131964240. Policy #0 lag: (min: 20.0, avg: 27.0, max: 52.0) [2023-12-26 17:14:21,063][104569] Avg episode reward: [(0, '9356.389'), (1, '8906.820')] [2023-12-26 17:14:21,070][105620] Updated weights for policy 1, policy_version 257871 (0.0009) [2023-12-26 17:14:21,124][105620] Updated weights for policy 1, policy_version 257881 (0.0007) [2023-12-26 17:14:21,186][105620] Updated weights for policy 1, policy_version 257891 (0.0009) [2023-12-26 17:14:21,664][105692] Updated weights for policy 0, policy_version 257585 (0.0010) [2023-12-26 17:14:21,733][105692] Updated weights for policy 0, policy_version 257595 (0.0008) [2023-12-26 17:14:21,799][105692] Updated weights for policy 0, policy_version 257605 (0.0009) [2023-12-26 17:14:21,970][105620] Updated weights for policy 1, policy_version 257901 (0.0009) [2023-12-26 17:14:22,043][105620] Updated weights for policy 1, policy_version 257911 (0.0009) [2023-12-26 17:14:22,091][105620] Updated weights for policy 1, policy_version 257921 (0.0009) [2023-12-26 17:14:22,532][105692] Updated weights for policy 0, policy_version 257615 (0.0009) [2023-12-26 17:14:22,593][105692] Updated weights for policy 0, policy_version 257625 (0.0006) [2023-12-26 17:14:22,660][105692] Updated weights for policy 0, policy_version 257635 (0.0005) [2023-12-26 17:14:22,932][105620] Updated weights for policy 1, policy_version 257931 (0.0009) [2023-12-26 17:14:22,990][105620] Updated weights for policy 1, policy_version 257941 (0.0009) [2023-12-26 17:14:23,049][105620] Updated weights for policy 1, policy_version 257951 (0.0008) [2023-12-26 17:14:23,338][105692] Updated weights for policy 0, policy_version 257645 (0.0008) [2023-12-26 17:14:23,389][105692] Updated weights for policy 0, policy_version 257655 (0.0010) [2023-12-26 17:14:23,444][105692] Updated weights for policy 0, policy_version 257665 (0.0010) [2023-12-26 17:14:23,818][105620] Updated weights for policy 1, policy_version 257961 (0.0008) [2023-12-26 17:14:23,871][105620] Updated weights for policy 1, policy_version 257971 (0.0008) [2023-12-26 17:14:23,933][105620] Updated weights for policy 1, policy_version 257981 (0.0007) [2023-12-26 17:14:23,996][105620] Updated weights for policy 1, policy_version 257991 (0.0008) [2023-12-26 17:14:24,198][105692] Updated weights for policy 0, policy_version 257675 (0.0010) [2023-12-26 17:14:24,253][105692] Updated weights for policy 0, policy_version 257685 (0.0011) [2023-12-26 17:14:24,308][105692] Updated weights for policy 0, policy_version 257695 (0.0010) [2023-12-26 17:14:24,754][105620] Updated weights for policy 1, policy_version 258001 (0.0008) [2023-12-26 17:14:24,813][105620] Updated weights for policy 1, policy_version 258011 (0.0008) [2023-12-26 17:14:24,871][105620] Updated weights for policy 1, policy_version 258021 (0.0008) [2023-12-26 17:14:25,059][105692] Updated weights for policy 0, policy_version 257705 (0.0010) [2023-12-26 17:14:25,121][105692] Updated weights for policy 0, policy_version 257715 (0.0006) [2023-12-26 17:14:25,178][105692] Updated weights for policy 0, policy_version 257725 (0.0005) [2023-12-26 17:14:25,246][105692] Updated weights for policy 0, policy_version 257735 (0.0005) [2023-12-26 17:14:25,514][105620] Updated weights for policy 1, policy_version 258031 (0.0006) [2023-12-26 17:14:25,582][105620] Updated weights for policy 1, policy_version 258041 (0.0006) [2023-12-26 17:14:25,649][105620] Updated weights for policy 1, policy_version 258051 (0.0009) [2023-12-26 17:14:25,856][105692] Updated weights for policy 0, policy_version 257745 (0.0009) [2023-12-26 17:14:25,903][105692] Updated weights for policy 0, policy_version 257755 (0.0009) [2023-12-26 17:14:25,953][105692] Updated weights for policy 0, policy_version 257765 (0.0009) [2023-12-26 17:14:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 132071424. Throughput: 0: 9828.4, 1: 9873.5. Samples: 132077516. Policy #0 lag: (min: 20.0, avg: 27.0, max: 52.0) [2023-12-26 17:14:26,062][104569] Avg episode reward: [(0, '9356.282'), (1, '8728.218')] [2023-12-26 17:14:26,374][105620] Updated weights for policy 1, policy_version 258061 (0.0009) [2023-12-26 17:14:26,426][105620] Updated weights for policy 1, policy_version 258071 (0.0005) [2023-12-26 17:14:26,494][105620] Updated weights for policy 1, policy_version 258081 (0.0006) [2023-12-26 17:14:26,584][105692] Updated weights for policy 0, policy_version 257775 (0.0009) [2023-12-26 17:14:26,632][105692] Updated weights for policy 0, policy_version 257785 (0.0006) [2023-12-26 17:14:26,684][105692] Updated weights for policy 0, policy_version 257795 (0.0005) [2023-12-26 17:14:27,114][105620] Updated weights for policy 1, policy_version 258091 (0.0009) [2023-12-26 17:14:27,159][105620] Updated weights for policy 1, policy_version 258101 (0.0006) [2023-12-26 17:14:27,212][105620] Updated weights for policy 1, policy_version 258111 (0.0005) [2023-12-26 17:14:27,334][105692] Updated weights for policy 0, policy_version 257805 (0.0006) [2023-12-26 17:14:27,399][105692] Updated weights for policy 0, policy_version 257815 (0.0006) [2023-12-26 17:14:27,463][105692] Updated weights for policy 0, policy_version 257825 (0.0005) [2023-12-26 17:14:27,974][105620] Updated weights for policy 1, policy_version 258121 (0.0006) [2023-12-26 17:14:27,994][105692] Updated weights for policy 0, policy_version 257835 (0.0005) [2023-12-26 17:14:28,022][105620] Updated weights for policy 1, policy_version 258131 (0.0008) [2023-12-26 17:14:28,044][105692] Updated weights for policy 0, policy_version 257845 (0.0010) [2023-12-26 17:14:28,070][105620] Updated weights for policy 1, policy_version 258141 (0.0005) [2023-12-26 17:14:28,099][105692] Updated weights for policy 0, policy_version 257855 (0.0010) [2023-12-26 17:14:28,124][105620] Updated weights for policy 1, policy_version 258151 (0.0005) [2023-12-26 17:14:28,789][105620] Updated weights for policy 1, policy_version 258161 (0.0006) [2023-12-26 17:14:28,835][105692] Updated weights for policy 0, policy_version 257865 (0.0010) [2023-12-26 17:14:28,849][105620] Updated weights for policy 1, policy_version 258171 (0.0006) [2023-12-26 17:14:28,897][105692] Updated weights for policy 0, policy_version 257875 (0.0009) [2023-12-26 17:14:28,908][105620] Updated weights for policy 1, policy_version 258181 (0.0009) [2023-12-26 17:14:28,955][105692] Updated weights for policy 0, policy_version 257885 (0.0010) [2023-12-26 17:14:29,020][105692] Updated weights for policy 0, policy_version 257895 (0.0010) [2023-12-26 17:14:29,553][105620] Updated weights for policy 1, policy_version 258191 (0.0006) [2023-12-26 17:14:29,626][105620] Updated weights for policy 1, policy_version 258202 (0.0008) [2023-12-26 17:14:29,692][105620] Updated weights for policy 1, policy_version 258212 (0.0008) [2023-12-26 17:14:29,798][105692] Updated weights for policy 0, policy_version 257905 (0.0010) [2023-12-26 17:14:29,866][105692] Updated weights for policy 0, policy_version 257915 (0.0008) [2023-12-26 17:14:29,930][105692] Updated weights for policy 0, policy_version 257925 (0.0009) [2023-12-26 17:14:30,325][105620] Updated weights for policy 1, policy_version 258222 (0.0006) [2023-12-26 17:14:30,378][105620] Updated weights for policy 1, policy_version 258232 (0.0009) [2023-12-26 17:14:30,433][105620] Updated weights for policy 1, policy_version 258242 (0.0009) [2023-12-26 17:14:30,707][105692] Updated weights for policy 0, policy_version 257935 (0.0009) [2023-12-26 17:14:30,757][105692] Updated weights for policy 0, policy_version 257945 (0.0009) [2023-12-26 17:14:30,818][105692] Updated weights for policy 0, policy_version 257955 (0.0007) [2023-12-26 17:14:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 132169728. Throughput: 0: 9898.6, 1: 9936.1. Samples: 132140484. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 17:14:31,063][104569] Avg episode reward: [(0, '9356.581'), (1, '8912.895')] [2023-12-26 17:14:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000257960_66052096.pth... [2023-12-26 17:14:31,071][105620] Updated weights for policy 1, policy_version 258252 (0.0008) [2023-12-26 17:14:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000256808_65757184.pth [2023-12-26 17:14:31,139][105620] Updated weights for policy 1, policy_version 258262 (0.0006) [2023-12-26 17:14:31,202][105620] Updated weights for policy 1, policy_version 258272 (0.0006) [2023-12-26 17:14:31,254][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000258280_66125824.pth... [2023-12-26 17:14:31,258][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000257128_65830912.pth [2023-12-26 17:14:31,526][105692] Updated weights for policy 0, policy_version 257965 (0.0007) [2023-12-26 17:14:31,589][105692] Updated weights for policy 0, policy_version 257975 (0.0009) [2023-12-26 17:14:31,654][105692] Updated weights for policy 0, policy_version 257985 (0.0008) [2023-12-26 17:14:31,873][105620] Updated weights for policy 1, policy_version 258282 (0.0006) [2023-12-26 17:14:31,934][105620] Updated weights for policy 1, policy_version 258292 (0.0009) [2023-12-26 17:14:31,994][105620] Updated weights for policy 1, policy_version 258302 (0.0009) [2023-12-26 17:14:32,049][105620] Updated weights for policy 1, policy_version 258312 (0.0008) [2023-12-26 17:14:32,333][105692] Updated weights for policy 0, policy_version 257995 (0.0008) [2023-12-26 17:14:32,386][105692] Updated weights for policy 0, policy_version 258005 (0.0011) [2023-12-26 17:14:32,442][105692] Updated weights for policy 0, policy_version 258015 (0.0006) [2023-12-26 17:14:32,834][105620] Updated weights for policy 1, policy_version 258322 (0.0008) [2023-12-26 17:14:32,893][105620] Updated weights for policy 1, policy_version 258332 (0.0008) [2023-12-26 17:14:32,956][105620] Updated weights for policy 1, policy_version 258342 (0.0008) [2023-12-26 17:14:33,133][105692] Updated weights for policy 0, policy_version 258025 (0.0006) [2023-12-26 17:14:33,181][105692] Updated weights for policy 0, policy_version 258035 (0.0010) [2023-12-26 17:14:33,236][105692] Updated weights for policy 0, policy_version 258045 (0.0010) [2023-12-26 17:14:33,283][105692] Updated weights for policy 0, policy_version 258055 (0.0010) [2023-12-26 17:14:33,655][105620] Updated weights for policy 1, policy_version 258352 (0.0006) [2023-12-26 17:14:33,711][105620] Updated weights for policy 1, policy_version 258362 (0.0005) [2023-12-26 17:14:33,758][105620] Updated weights for policy 1, policy_version 258372 (0.0005) [2023-12-26 17:14:34,034][105692] Updated weights for policy 0, policy_version 258065 (0.0006) [2023-12-26 17:14:34,090][105692] Updated weights for policy 0, policy_version 258075 (0.0007) [2023-12-26 17:14:34,145][105692] Updated weights for policy 0, policy_version 258085 (0.0007) [2023-12-26 17:14:34,321][105620] Updated weights for policy 1, policy_version 258382 (0.0007) [2023-12-26 17:14:34,386][105620] Updated weights for policy 1, policy_version 258392 (0.0008) [2023-12-26 17:14:34,450][105620] Updated weights for policy 1, policy_version 258402 (0.0008) [2023-12-26 17:14:34,845][105692] Updated weights for policy 0, policy_version 258095 (0.0008) [2023-12-26 17:14:34,898][105692] Updated weights for policy 0, policy_version 258105 (0.0009) [2023-12-26 17:14:34,981][105692] Updated weights for policy 0, policy_version 258115 (0.0010) [2023-12-26 17:14:35,140][105620] Updated weights for policy 1, policy_version 258412 (0.0007) [2023-12-26 17:14:35,203][105620] Updated weights for policy 1, policy_version 258422 (0.0005) [2023-12-26 17:14:35,263][105620] Updated weights for policy 1, policy_version 258432 (0.0009) [2023-12-26 17:14:35,639][105692] Updated weights for policy 0, policy_version 258125 (0.0009) [2023-12-26 17:14:35,698][105692] Updated weights for policy 0, policy_version 258135 (0.0008) [2023-12-26 17:14:35,742][105692] Updated weights for policy 0, policy_version 258145 (0.0008) [2023-12-26 17:14:35,977][105620] Updated weights for policy 1, policy_version 258442 (0.0009) [2023-12-26 17:14:36,047][105620] Updated weights for policy 1, policy_version 258452 (0.0006) [2023-12-26 17:14:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 132268032. Throughput: 0: 9867.9, 1: 9931.5. Samples: 132258932. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 17:14:36,062][104569] Avg episode reward: [(0, '9266.576'), (1, '9270.352')] [2023-12-26 17:14:36,111][105620] Updated weights for policy 1, policy_version 258462 (0.0008) [2023-12-26 17:14:36,182][105620] Updated weights for policy 1, policy_version 258472 (0.0009) [2023-12-26 17:14:36,549][105692] Updated weights for policy 0, policy_version 258155 (0.0009) [2023-12-26 17:14:36,612][105692] Updated weights for policy 0, policy_version 258165 (0.0009) [2023-12-26 17:14:36,675][105692] Updated weights for policy 0, policy_version 258175 (0.0009) [2023-12-26 17:14:36,846][105620] Updated weights for policy 1, policy_version 258482 (0.0008) [2023-12-26 17:14:36,904][105620] Updated weights for policy 1, policy_version 258492 (0.0007) [2023-12-26 17:14:36,965][105620] Updated weights for policy 1, policy_version 258502 (0.0006) [2023-12-26 17:14:37,467][105692] Updated weights for policy 0, policy_version 258185 (0.0008) [2023-12-26 17:14:37,526][105692] Updated weights for policy 0, policy_version 258195 (0.0009) [2023-12-26 17:14:37,587][105692] Updated weights for policy 0, policy_version 258205 (0.0008) [2023-12-26 17:14:37,594][105620] Updated weights for policy 1, policy_version 258512 (0.0007) [2023-12-26 17:14:37,640][105620] Updated weights for policy 1, policy_version 258522 (0.0005) [2023-12-26 17:14:37,645][105692] Updated weights for policy 0, policy_version 258215 (0.0009) [2023-12-26 17:14:37,684][105620] Updated weights for policy 1, policy_version 258532 (0.0005) [2023-12-26 17:14:38,357][105692] Updated weights for policy 0, policy_version 258225 (0.0009) [2023-12-26 17:14:38,421][105692] Updated weights for policy 0, policy_version 258235 (0.0009) [2023-12-26 17:14:38,433][105585] KL-divergence is very high: 112.6520 [2023-12-26 17:14:38,485][105692] Updated weights for policy 0, policy_version 258245 (0.0009) [2023-12-26 17:14:38,486][105585] KL-divergence is very high: 119.3936 [2023-12-26 17:14:38,499][105620] Updated weights for policy 1, policy_version 258542 (0.0007) [2023-12-26 17:14:38,564][105620] Updated weights for policy 1, policy_version 258552 (0.0007) [2023-12-26 17:14:38,630][105620] Updated weights for policy 1, policy_version 258562 (0.0008) [2023-12-26 17:14:39,100][105692] Updated weights for policy 0, policy_version 258255 (0.0006) [2023-12-26 17:14:39,153][105692] Updated weights for policy 0, policy_version 258265 (0.0008) [2023-12-26 17:14:39,184][105585] KL-divergence is very high: 121.4014 [2023-12-26 17:14:39,189][105585] KL-divergence is very high: 110.4426 [2023-12-26 17:14:39,200][105692] Updated weights for policy 0, policy_version 258275 (0.0009) [2023-12-26 17:14:39,427][105620] Updated weights for policy 1, policy_version 258572 (0.0008) [2023-12-26 17:14:39,489][105620] Updated weights for policy 1, policy_version 258582 (0.0005) [2023-12-26 17:14:39,554][105620] Updated weights for policy 1, policy_version 258592 (0.0007) [2023-12-26 17:14:39,970][105692] Updated weights for policy 0, policy_version 258285 (0.0010) [2023-12-26 17:14:40,040][105692] Updated weights for policy 0, policy_version 258295 (0.0009) [2023-12-26 17:14:40,108][105692] Updated weights for policy 0, policy_version 258305 (0.0010) [2023-12-26 17:14:40,229][105620] Updated weights for policy 1, policy_version 258602 (0.0008) [2023-12-26 17:14:40,287][105620] Updated weights for policy 1, policy_version 258612 (0.0006) [2023-12-26 17:14:40,346][105620] Updated weights for policy 1, policy_version 258622 (0.0009) [2023-12-26 17:14:40,401][105620] Updated weights for policy 1, policy_version 258632 (0.0009) [2023-12-26 17:14:40,801][105692] Updated weights for policy 0, policy_version 258315 (0.0009) [2023-12-26 17:14:40,859][105692] Updated weights for policy 0, policy_version 258325 (0.0005) [2023-12-26 17:14:40,913][105692] Updated weights for policy 0, policy_version 258335 (0.0005) [2023-12-26 17:14:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 132366336. Throughput: 0: 9811.3, 1: 9893.2. Samples: 132373568. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 17:14:41,062][104569] Avg episode reward: [(0, '8997.023'), (1, '9270.165')] [2023-12-26 17:14:41,224][105620] Updated weights for policy 1, policy_version 258642 (0.0009) [2023-12-26 17:14:41,290][105620] Updated weights for policy 1, policy_version 258652 (0.0009) [2023-12-26 17:14:41,358][105620] Updated weights for policy 1, policy_version 258662 (0.0008) [2023-12-26 17:14:41,610][105692] Updated weights for policy 0, policy_version 258345 (0.0009) [2023-12-26 17:14:41,670][105692] Updated weights for policy 0, policy_version 258355 (0.0011) [2023-12-26 17:14:41,734][105692] Updated weights for policy 0, policy_version 258365 (0.0011) [2023-12-26 17:14:41,791][105692] Updated weights for policy 0, policy_version 258375 (0.0011) [2023-12-26 17:14:42,211][105620] Updated weights for policy 1, policy_version 258672 (0.0009) [2023-12-26 17:14:42,275][105620] Updated weights for policy 1, policy_version 258682 (0.0008) [2023-12-26 17:14:42,343][105620] Updated weights for policy 1, policy_version 258692 (0.0009) [2023-12-26 17:14:42,424][105692] Updated weights for policy 0, policy_version 258385 (0.0011) [2023-12-26 17:14:42,473][105692] Updated weights for policy 0, policy_version 258395 (0.0009) [2023-12-26 17:14:42,538][105692] Updated weights for policy 0, policy_version 258405 (0.0009) [2023-12-26 17:14:43,074][105620] Updated weights for policy 1, policy_version 258702 (0.0007) [2023-12-26 17:14:43,135][105620] Updated weights for policy 1, policy_version 258712 (0.0006) [2023-12-26 17:14:43,149][105692] Updated weights for policy 0, policy_version 258415 (0.0010) [2023-12-26 17:14:43,192][105620] Updated weights for policy 1, policy_version 258722 (0.0005) [2023-12-26 17:14:43,198][105692] Updated weights for policy 0, policy_version 258425 (0.0007) [2023-12-26 17:14:43,252][105692] Updated weights for policy 0, policy_version 258435 (0.0009) [2023-12-26 17:14:43,789][105620] Updated weights for policy 1, policy_version 258732 (0.0005) [2023-12-26 17:14:43,852][105620] Updated weights for policy 1, policy_version 258742 (0.0005) [2023-12-26 17:14:43,853][105586] KL-divergence is very high: 110.9139 [2023-12-26 17:14:43,892][105586] KL-divergence is very high: 218.9558 [2023-12-26 17:14:43,902][105620] Updated weights for policy 1, policy_version 258752 (0.0006) [2023-12-26 17:14:43,926][105692] Updated weights for policy 0, policy_version 258445 (0.0009) [2023-12-26 17:14:43,934][105586] KL-divergence is very high: 201.6278 [2023-12-26 17:14:43,989][105692] Updated weights for policy 0, policy_version 258455 (0.0008) [2023-12-26 17:14:44,043][105692] Updated weights for policy 0, policy_version 258465 (0.0010) [2023-12-26 17:14:44,486][105620] Updated weights for policy 1, policy_version 258762 (0.0008) [2023-12-26 17:14:44,550][105620] Updated weights for policy 1, policy_version 258772 (0.0010) [2023-12-26 17:14:44,608][105620] Updated weights for policy 1, policy_version 258782 (0.0010) [2023-12-26 17:14:44,656][105620] Updated weights for policy 1, policy_version 258792 (0.0010) [2023-12-26 17:14:44,669][105692] Updated weights for policy 0, policy_version 258475 (0.0009) [2023-12-26 17:14:44,734][105692] Updated weights for policy 0, policy_version 258485 (0.0008) [2023-12-26 17:14:44,800][105692] Updated weights for policy 0, policy_version 258495 (0.0010) [2023-12-26 17:14:45,396][105620] Updated weights for policy 1, policy_version 258802 (0.0011) [2023-12-26 17:14:45,459][105620] Updated weights for policy 1, policy_version 258812 (0.0011) [2023-12-26 17:14:45,475][105692] Updated weights for policy 0, policy_version 258505 (0.0009) [2023-12-26 17:14:45,519][105620] Updated weights for policy 1, policy_version 258822 (0.0011) [2023-12-26 17:14:45,535][105692] Updated weights for policy 0, policy_version 258515 (0.0011) [2023-12-26 17:14:45,594][105692] Updated weights for policy 0, policy_version 258525 (0.0011) [2023-12-26 17:14:45,655][105692] Updated weights for policy 0, policy_version 258535 (0.0006) [2023-12-26 17:14:46,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 132464640. Throughput: 0: 9829.2, 1: 9863.3. Samples: 132433564. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 17:14:46,063][104569] Avg episode reward: [(0, '9177.489'), (1, '9083.293')] [2023-12-26 17:14:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000258536_66199552.pth... [2023-12-26 17:14:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000258824_66265088.pth... [2023-12-26 17:14:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000257704_65978368.pth [2023-12-26 17:14:46,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000257384_65904640.pth [2023-12-26 17:14:46,259][105620] Updated weights for policy 1, policy_version 258832 (0.0010) [2023-12-26 17:14:46,297][105692] Updated weights for policy 0, policy_version 258545 (0.0010) [2023-12-26 17:14:46,313][105620] Updated weights for policy 1, policy_version 258842 (0.0010) [2023-12-26 17:14:46,355][105692] Updated weights for policy 0, policy_version 258555 (0.0010) [2023-12-26 17:14:46,372][105620] Updated weights for policy 1, policy_version 258852 (0.0010) [2023-12-26 17:14:46,407][105692] Updated weights for policy 0, policy_version 258565 (0.0010) [2023-12-26 17:14:47,057][105620] Updated weights for policy 1, policy_version 258862 (0.0010) [2023-12-26 17:14:47,105][105620] Updated weights for policy 1, policy_version 258872 (0.0010) [2023-12-26 17:14:47,146][105692] Updated weights for policy 0, policy_version 258575 (0.0010) [2023-12-26 17:14:47,152][105620] Updated weights for policy 1, policy_version 258882 (0.0010) [2023-12-26 17:14:47,194][105692] Updated weights for policy 0, policy_version 258585 (0.0010) [2023-12-26 17:14:47,236][105692] Updated weights for policy 0, policy_version 258595 (0.0005) [2023-12-26 17:14:47,881][105620] Updated weights for policy 1, policy_version 258892 (0.0010) [2023-12-26 17:14:47,935][105692] Updated weights for policy 0, policy_version 258605 (0.0008) [2023-12-26 17:14:47,938][105620] Updated weights for policy 1, policy_version 258902 (0.0010) [2023-12-26 17:14:47,987][105692] Updated weights for policy 0, policy_version 258615 (0.0010) [2023-12-26 17:14:47,989][105620] Updated weights for policy 1, policy_version 258912 (0.0010) [2023-12-26 17:14:48,046][105692] Updated weights for policy 0, policy_version 258625 (0.0010) [2023-12-26 17:14:48,766][105620] Updated weights for policy 1, policy_version 258922 (0.0010) [2023-12-26 17:14:48,782][105692] Updated weights for policy 0, policy_version 258635 (0.0010) [2023-12-26 17:14:48,825][105620] Updated weights for policy 1, policy_version 258932 (0.0010) [2023-12-26 17:14:48,841][105692] Updated weights for policy 0, policy_version 258645 (0.0011) [2023-12-26 17:14:48,884][105620] Updated weights for policy 1, policy_version 258942 (0.0010) [2023-12-26 17:14:48,906][105692] Updated weights for policy 0, policy_version 258655 (0.0005) [2023-12-26 17:14:48,943][105620] Updated weights for policy 1, policy_version 258952 (0.0010) [2023-12-26 17:14:49,551][105692] Updated weights for policy 0, policy_version 258665 (0.0006) [2023-12-26 17:14:49,616][105692] Updated weights for policy 0, policy_version 258675 (0.0005) [2023-12-26 17:14:49,683][105692] Updated weights for policy 0, policy_version 258685 (0.0006) [2023-12-26 17:14:49,696][105620] Updated weights for policy 1, policy_version 258962 (0.0010) [2023-12-26 17:14:49,738][105692] Updated weights for policy 0, policy_version 258695 (0.0006) [2023-12-26 17:14:49,751][105620] Updated weights for policy 1, policy_version 258972 (0.0010) [2023-12-26 17:14:49,809][105620] Updated weights for policy 1, policy_version 258982 (0.0010) [2023-12-26 17:14:50,464][105692] Updated weights for policy 0, policy_version 258706 (0.0010) [2023-12-26 17:14:50,477][105585] KL-divergence is very high: 146.8599 [2023-12-26 17:14:50,483][105585] KL-divergence is very high: 168.0293 [2023-12-26 17:14:50,489][105585] KL-divergence is very high: 117.8195 [2023-12-26 17:14:50,522][105692] Updated weights for policy 0, policy_version 258716 (0.0007) [2023-12-26 17:14:50,529][105620] Updated weights for policy 1, policy_version 258992 (0.0009) [2023-12-26 17:14:50,588][105692] Updated weights for policy 0, policy_version 258726 (0.0007) [2023-12-26 17:14:50,594][105620] Updated weights for policy 1, policy_version 259002 (0.0007) [2023-12-26 17:14:50,659][105620] Updated weights for policy 1, policy_version 259012 (0.0008) [2023-12-26 17:14:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 132562944. Throughput: 0: 9873.7, 1: 9784.4. Samples: 132553348. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 17:14:51,062][104569] Avg episode reward: [(0, '8279.553'), (1, '8990.483')] [2023-12-26 17:14:51,327][105620] Updated weights for policy 1, policy_version 259022 (0.0009) [2023-12-26 17:14:51,394][105620] Updated weights for policy 1, policy_version 259032 (0.0009) [2023-12-26 17:14:51,412][105692] Updated weights for policy 0, policy_version 258736 (0.0007) [2023-12-26 17:14:51,423][105585] KL-divergence is very high: 112.7818 [2023-12-26 17:14:51,453][105620] Updated weights for policy 1, policy_version 259042 (0.0006) [2023-12-26 17:14:51,472][105692] Updated weights for policy 0, policy_version 258746 (0.0005) [2023-12-26 17:14:51,525][105692] Updated weights for policy 0, policy_version 258756 (0.0005) [2023-12-26 17:14:52,139][105620] Updated weights for policy 1, policy_version 259052 (0.0007) [2023-12-26 17:14:52,186][105620] Updated weights for policy 1, policy_version 259062 (0.0010) [2023-12-26 17:14:52,189][105692] Updated weights for policy 0, policy_version 258766 (0.0006) [2023-12-26 17:14:52,244][105692] Updated weights for policy 0, policy_version 258776 (0.0006) [2023-12-26 17:14:52,245][105620] Updated weights for policy 1, policy_version 259072 (0.0010) [2023-12-26 17:14:52,304][105692] Updated weights for policy 0, policy_version 258786 (0.0006) [2023-12-26 17:14:52,948][105692] Updated weights for policy 0, policy_version 258796 (0.0009) [2023-12-26 17:14:52,993][105620] Updated weights for policy 1, policy_version 259082 (0.0010) [2023-12-26 17:14:53,010][105692] Updated weights for policy 0, policy_version 258806 (0.0009) [2023-12-26 17:14:53,051][105620] Updated weights for policy 1, policy_version 259092 (0.0005) [2023-12-26 17:14:53,070][105692] Updated weights for policy 0, policy_version 258816 (0.0008) [2023-12-26 17:14:53,114][105620] Updated weights for policy 1, policy_version 259102 (0.0008) [2023-12-26 17:14:53,177][105620] Updated weights for policy 1, policy_version 259112 (0.0011) [2023-12-26 17:14:53,757][105692] Updated weights for policy 0, policy_version 258826 (0.0008) [2023-12-26 17:14:53,822][105692] Updated weights for policy 0, policy_version 258836 (0.0009) [2023-12-26 17:14:53,851][105620] Updated weights for policy 1, policy_version 259122 (0.0010) [2023-12-26 17:14:53,878][105692] Updated weights for policy 0, policy_version 258846 (0.0006) [2023-12-26 17:14:53,908][105620] Updated weights for policy 1, policy_version 259132 (0.0009) [2023-12-26 17:14:53,936][105692] Updated weights for policy 0, policy_version 258856 (0.0008) [2023-12-26 17:14:53,965][105620] Updated weights for policy 1, policy_version 259142 (0.0010) [2023-12-26 17:14:54,618][105692] Updated weights for policy 0, policy_version 258866 (0.0009) [2023-12-26 17:14:54,664][105692] Updated weights for policy 0, policy_version 258876 (0.0009) [2023-12-26 17:14:54,712][105620] Updated weights for policy 1, policy_version 259152 (0.0008) [2023-12-26 17:14:54,725][105692] Updated weights for policy 0, policy_version 258886 (0.0007) [2023-12-26 17:14:54,770][105620] Updated weights for policy 1, policy_version 259162 (0.0009) [2023-12-26 17:14:54,815][105620] Updated weights for policy 1, policy_version 259172 (0.0008) [2023-12-26 17:14:55,378][105692] Updated weights for policy 0, policy_version 258896 (0.0007) [2023-12-26 17:14:55,432][105692] Updated weights for policy 0, policy_version 258906 (0.0009) [2023-12-26 17:14:55,483][105692] Updated weights for policy 0, policy_version 258916 (0.0009) [2023-12-26 17:14:55,553][105620] Updated weights for policy 1, policy_version 259182 (0.0006) [2023-12-26 17:14:55,616][105620] Updated weights for policy 1, policy_version 259192 (0.0005) [2023-12-26 17:14:55,673][105620] Updated weights for policy 1, policy_version 259202 (0.0009) [2023-12-26 17:14:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 132661248. Throughput: 0: 9889.2, 1: 9701.2. Samples: 132671132. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 17:14:56,062][104569] Avg episode reward: [(0, '7928.101'), (1, '8800.931')] [2023-12-26 17:14:56,163][105692] Updated weights for policy 0, policy_version 258927 (0.0009) [2023-12-26 17:14:56,221][105692] Updated weights for policy 0, policy_version 258937 (0.0007) [2023-12-26 17:14:56,272][105692] Updated weights for policy 0, policy_version 258947 (0.0009) [2023-12-26 17:14:56,426][105620] Updated weights for policy 1, policy_version 259212 (0.0010) [2023-12-26 17:14:56,481][105620] Updated weights for policy 1, policy_version 259222 (0.0009) [2023-12-26 17:14:56,527][105620] Updated weights for policy 1, policy_version 259232 (0.0009) [2023-12-26 17:14:56,936][105692] Updated weights for policy 0, policy_version 258957 (0.0007) [2023-12-26 17:14:56,989][105692] Updated weights for policy 0, policy_version 258967 (0.0005) [2023-12-26 17:14:57,009][105585] KL-divergence is very high: 175.0795 [2023-12-26 17:14:57,014][105585] KL-divergence is very high: 133.6987 [2023-12-26 17:14:57,036][105692] Updated weights for policy 0, policy_version 258977 (0.0005) [2023-12-26 17:14:57,046][105585] KL-divergence is very high: 147.6093 [2023-12-26 17:14:57,261][105620] Updated weights for policy 1, policy_version 259242 (0.0009) [2023-12-26 17:14:57,317][105620] Updated weights for policy 1, policy_version 259252 (0.0010) [2023-12-26 17:14:57,361][105620] Updated weights for policy 1, policy_version 259262 (0.0010) [2023-12-26 17:14:57,409][105620] Updated weights for policy 1, policy_version 259272 (0.0010) [2023-12-26 17:14:57,664][105692] Updated weights for policy 0, policy_version 258987 (0.0006) [2023-12-26 17:14:57,711][105692] Updated weights for policy 0, policy_version 258997 (0.0008) [2023-12-26 17:14:57,774][105692] Updated weights for policy 0, policy_version 259007 (0.0007) [2023-12-26 17:14:58,063][105620] Updated weights for policy 1, policy_version 259282 (0.0010) [2023-12-26 17:14:58,134][105620] Updated weights for policy 1, policy_version 259292 (0.0010) [2023-12-26 17:14:58,200][105620] Updated weights for policy 1, policy_version 259302 (0.0010) [2023-12-26 17:14:58,405][105692] Updated weights for policy 0, policy_version 259017 (0.0005) [2023-12-26 17:14:58,467][105692] Updated weights for policy 0, policy_version 259027 (0.0008) [2023-12-26 17:14:58,537][105692] Updated weights for policy 0, policy_version 259037 (0.0009) [2023-12-26 17:14:58,601][105692] Updated weights for policy 0, policy_version 259047 (0.0009) [2023-12-26 17:14:59,002][105620] Updated weights for policy 1, policy_version 259312 (0.0008) [2023-12-26 17:14:59,054][105620] Updated weights for policy 1, policy_version 259322 (0.0008) [2023-12-26 17:14:59,106][105620] Updated weights for policy 1, policy_version 259332 (0.0008) [2023-12-26 17:14:59,457][105692] Updated weights for policy 0, policy_version 259057 (0.0010) [2023-12-26 17:14:59,515][105692] Updated weights for policy 0, policy_version 259067 (0.0010) [2023-12-26 17:14:59,571][105692] Updated weights for policy 0, policy_version 259077 (0.0009) [2023-12-26 17:14:59,891][105620] Updated weights for policy 1, policy_version 259342 (0.0008) [2023-12-26 17:14:59,957][105620] Updated weights for policy 1, policy_version 259352 (0.0007) [2023-12-26 17:15:00,021][105620] Updated weights for policy 1, policy_version 259362 (0.0007) [2023-12-26 17:15:00,316][105692] Updated weights for policy 0, policy_version 259087 (0.0007) [2023-12-26 17:15:00,373][105692] Updated weights for policy 0, policy_version 259097 (0.0008) [2023-12-26 17:15:00,432][105692] Updated weights for policy 0, policy_version 259107 (0.0005) [2023-12-26 17:15:00,669][105620] Updated weights for policy 1, policy_version 259372 (0.0007) [2023-12-26 17:15:00,717][105620] Updated weights for policy 1, policy_version 259382 (0.0009) [2023-12-26 17:15:00,763][105620] Updated weights for policy 1, policy_version 259392 (0.0008) [2023-12-26 17:15:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 132759552. Throughput: 0: 9988.3, 1: 9666.2. Samples: 132732016. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 17:15:01,062][104569] Avg episode reward: [(0, '8196.682'), (1, '8891.459')] [2023-12-26 17:15:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000259400_66412544.pth... [2023-12-26 17:15:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000258280_66125824.pth [2023-12-26 17:15:01,081][105692] Updated weights for policy 0, policy_version 259117 (0.0007) [2023-12-26 17:15:01,153][105692] Updated weights for policy 0, policy_version 259127 (0.0009) [2023-12-26 17:15:01,217][105692] Updated weights for policy 0, policy_version 259137 (0.0008) [2023-12-26 17:15:01,259][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000259144_66355200.pth... [2023-12-26 17:15:01,263][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000257960_66052096.pth [2023-12-26 17:15:01,490][105620] Updated weights for policy 1, policy_version 259402 (0.0008) [2023-12-26 17:15:01,552][105620] Updated weights for policy 1, policy_version 259412 (0.0010) [2023-12-26 17:15:01,612][105620] Updated weights for policy 1, policy_version 259422 (0.0010) [2023-12-26 17:15:01,672][105620] Updated weights for policy 1, policy_version 259432 (0.0009) [2023-12-26 17:15:01,939][105692] Updated weights for policy 0, policy_version 259147 (0.0009) [2023-12-26 17:15:02,005][105692] Updated weights for policy 0, policy_version 259157 (0.0008) [2023-12-26 17:15:02,060][105692] Updated weights for policy 0, policy_version 259167 (0.0006) [2023-12-26 17:15:02,370][105620] Updated weights for policy 1, policy_version 259442 (0.0007) [2023-12-26 17:15:02,438][105620] Updated weights for policy 1, policy_version 259452 (0.0010) [2023-12-26 17:15:02,499][105620] Updated weights for policy 1, policy_version 259462 (0.0010) [2023-12-26 17:15:02,814][105692] Updated weights for policy 0, policy_version 259177 (0.0007) [2023-12-26 17:15:02,873][105692] Updated weights for policy 0, policy_version 259187 (0.0010) [2023-12-26 17:15:02,924][105692] Updated weights for policy 0, policy_version 259197 (0.0010) [2023-12-26 17:15:02,976][105692] Updated weights for policy 0, policy_version 259207 (0.0010) [2023-12-26 17:15:03,195][105620] Updated weights for policy 1, policy_version 259472 (0.0008) [2023-12-26 17:15:03,250][105620] Updated weights for policy 1, policy_version 259482 (0.0008) [2023-12-26 17:15:03,312][105620] Updated weights for policy 1, policy_version 259492 (0.0008) [2023-12-26 17:15:03,740][105692] Updated weights for policy 0, policy_version 259217 (0.0008) [2023-12-26 17:15:03,791][105692] Updated weights for policy 0, policy_version 259227 (0.0008) [2023-12-26 17:15:03,860][105692] Updated weights for policy 0, policy_version 259237 (0.0008) [2023-12-26 17:15:04,007][105620] Updated weights for policy 1, policy_version 259502 (0.0006) [2023-12-26 17:15:04,077][105620] Updated weights for policy 1, policy_version 259512 (0.0005) [2023-12-26 17:15:04,145][105620] Updated weights for policy 1, policy_version 259522 (0.0006) [2023-12-26 17:15:04,572][105692] Updated weights for policy 0, policy_version 259247 (0.0008) [2023-12-26 17:15:04,632][105692] Updated weights for policy 0, policy_version 259257 (0.0008) [2023-12-26 17:15:04,683][105692] Updated weights for policy 0, policy_version 259267 (0.0010) [2023-12-26 17:15:04,804][105620] Updated weights for policy 1, policy_version 259532 (0.0006) [2023-12-26 17:15:04,852][105620] Updated weights for policy 1, policy_version 259542 (0.0005) [2023-12-26 17:15:04,912][105620] Updated weights for policy 1, policy_version 259552 (0.0005) [2023-12-26 17:15:05,310][105692] Updated weights for policy 0, policy_version 259277 (0.0008) [2023-12-26 17:15:05,378][105692] Updated weights for policy 0, policy_version 259287 (0.0005) [2023-12-26 17:15:05,444][105692] Updated weights for policy 0, policy_version 259297 (0.0006) [2023-12-26 17:15:05,555][105620] Updated weights for policy 1, policy_version 259562 (0.0006) [2023-12-26 17:15:05,610][105620] Updated weights for policy 1, policy_version 259572 (0.0010) [2023-12-26 17:15:05,669][105620] Updated weights for policy 1, policy_version 259582 (0.0010) [2023-12-26 17:15:05,728][105620] Updated weights for policy 1, policy_version 259592 (0.0010) [2023-12-26 17:15:05,984][105692] Updated weights for policy 0, policy_version 259307 (0.0006) [2023-12-26 17:15:06,043][105692] Updated weights for policy 0, policy_version 259317 (0.0006) [2023-12-26 17:15:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 132857856. Throughput: 0: 9895.6, 1: 9741.9. Samples: 132847924. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 17:15:06,062][104569] Avg episode reward: [(0, '9264.504'), (1, '9166.133')] [2023-12-26 17:15:06,087][105692] Updated weights for policy 0, policy_version 259327 (0.0005) [2023-12-26 17:15:06,426][105620] Updated weights for policy 1, policy_version 259602 (0.0011) [2023-12-26 17:15:06,493][105620] Updated weights for policy 1, policy_version 259612 (0.0011) [2023-12-26 17:15:06,560][105620] Updated weights for policy 1, policy_version 259622 (0.0010) [2023-12-26 17:15:06,734][105692] Updated weights for policy 0, policy_version 259337 (0.0007) [2023-12-26 17:15:06,783][105692] Updated weights for policy 0, policy_version 259347 (0.0011) [2023-12-26 17:15:06,833][105692] Updated weights for policy 0, policy_version 259357 (0.0011) [2023-12-26 17:15:06,889][105692] Updated weights for policy 0, policy_version 259367 (0.0011) [2023-12-26 17:15:07,162][105620] Updated weights for policy 1, policy_version 259632 (0.0006) [2023-12-26 17:15:07,226][105620] Updated weights for policy 1, policy_version 259642 (0.0005) [2023-12-26 17:15:07,289][105620] Updated weights for policy 1, policy_version 259652 (0.0005) [2023-12-26 17:15:07,643][105692] Updated weights for policy 0, policy_version 259377 (0.0011) [2023-12-26 17:15:07,694][105692] Updated weights for policy 0, policy_version 259387 (0.0010) [2023-12-26 17:15:07,743][105692] Updated weights for policy 0, policy_version 259397 (0.0010) [2023-12-26 17:15:07,830][105620] Updated weights for policy 1, policy_version 259662 (0.0006) [2023-12-26 17:15:07,879][105620] Updated weights for policy 1, policy_version 259672 (0.0008) [2023-12-26 17:15:07,927][105620] Updated weights for policy 1, policy_version 259682 (0.0009) [2023-12-26 17:15:08,393][105692] Updated weights for policy 0, policy_version 259407 (0.0007) [2023-12-26 17:15:08,454][105692] Updated weights for policy 0, policy_version 259417 (0.0009) [2023-12-26 17:15:08,500][105692] Updated weights for policy 0, policy_version 259427 (0.0008) [2023-12-26 17:15:08,595][105620] Updated weights for policy 1, policy_version 259692 (0.0008) [2023-12-26 17:15:08,646][105620] Updated weights for policy 1, policy_version 259702 (0.0009) [2023-12-26 17:15:08,698][105620] Updated weights for policy 1, policy_version 259712 (0.0008) [2023-12-26 17:15:09,209][105692] Updated weights for policy 0, policy_version 259437 (0.0009) [2023-12-26 17:15:09,265][105692] Updated weights for policy 0, policy_version 259447 (0.0008) [2023-12-26 17:15:09,320][105692] Updated weights for policy 0, policy_version 259457 (0.0008) [2023-12-26 17:15:09,465][105620] Updated weights for policy 1, policy_version 259722 (0.0008) [2023-12-26 17:15:09,514][105620] Updated weights for policy 1, policy_version 259732 (0.0008) [2023-12-26 17:15:09,568][105620] Updated weights for policy 1, policy_version 259742 (0.0008) [2023-12-26 17:15:09,616][105620] Updated weights for policy 1, policy_version 259752 (0.0008) [2023-12-26 17:15:10,189][105692] Updated weights for policy 0, policy_version 259467 (0.0009) [2023-12-26 17:15:10,251][105692] Updated weights for policy 0, policy_version 259477 (0.0009) [2023-12-26 17:15:10,308][105692] Updated weights for policy 0, policy_version 259487 (0.0008) [2023-12-26 17:15:10,374][105620] Updated weights for policy 1, policy_version 259762 (0.0008) [2023-12-26 17:15:10,444][105620] Updated weights for policy 1, policy_version 259772 (0.0007) [2023-12-26 17:15:10,507][105620] Updated weights for policy 1, policy_version 259782 (0.0009) [2023-12-26 17:15:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 132956160. Throughput: 0: 9967.3, 1: 9862.1. Samples: 132969840. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 17:15:11,062][104569] Avg episode reward: [(0, '9354.557'), (1, '9073.574')] [2023-12-26 17:15:11,075][105692] Updated weights for policy 0, policy_version 259497 (0.0007) [2023-12-26 17:15:11,140][105692] Updated weights for policy 0, policy_version 259507 (0.0009) [2023-12-26 17:15:11,205][105692] Updated weights for policy 0, policy_version 259517 (0.0008) [2023-12-26 17:15:11,211][105620] Updated weights for policy 1, policy_version 259792 (0.0007) [2023-12-26 17:15:11,267][105692] Updated weights for policy 0, policy_version 259527 (0.0007) [2023-12-26 17:15:11,276][105620] Updated weights for policy 1, policy_version 259802 (0.0008) [2023-12-26 17:15:11,355][105620] Updated weights for policy 1, policy_version 259812 (0.0008) [2023-12-26 17:15:12,019][105620] Updated weights for policy 1, policy_version 259822 (0.0007) [2023-12-26 17:15:12,038][105692] Updated weights for policy 0, policy_version 259537 (0.0006) [2023-12-26 17:15:12,080][105620] Updated weights for policy 1, policy_version 259832 (0.0009) [2023-12-26 17:15:12,100][105692] Updated weights for policy 0, policy_version 259547 (0.0007) [2023-12-26 17:15:12,139][105620] Updated weights for policy 1, policy_version 259842 (0.0007) [2023-12-26 17:15:12,157][105692] Updated weights for policy 0, policy_version 259557 (0.0006) [2023-12-26 17:15:12,841][105620] Updated weights for policy 1, policy_version 259852 (0.0008) [2023-12-26 17:15:12,894][105620] Updated weights for policy 1, policy_version 259862 (0.0008) [2023-12-26 17:15:12,944][105620] Updated weights for policy 1, policy_version 259872 (0.0008) [2023-12-26 17:15:12,962][105692] Updated weights for policy 0, policy_version 259567 (0.0007) [2023-12-26 17:15:13,017][105692] Updated weights for policy 0, policy_version 259577 (0.0008) [2023-12-26 17:15:13,067][105692] Updated weights for policy 0, policy_version 259587 (0.0009) [2023-12-26 17:15:13,621][105620] Updated weights for policy 1, policy_version 259882 (0.0006) [2023-12-26 17:15:13,675][105620] Updated weights for policy 1, policy_version 259892 (0.0009) [2023-12-26 17:15:13,732][105620] Updated weights for policy 1, policy_version 259902 (0.0009) [2023-12-26 17:15:13,797][105620] Updated weights for policy 1, policy_version 259912 (0.0009) [2023-12-26 17:15:13,858][105692] Updated weights for policy 0, policy_version 259597 (0.0007) [2023-12-26 17:15:13,918][105692] Updated weights for policy 0, policy_version 259607 (0.0005) [2023-12-26 17:15:13,982][105692] Updated weights for policy 0, policy_version 259617 (0.0005) [2023-12-26 17:15:14,524][105692] Updated weights for policy 0, policy_version 259627 (0.0006) [2023-12-26 17:15:14,586][105692] Updated weights for policy 0, policy_version 259637 (0.0008) [2023-12-26 17:15:14,633][105620] Updated weights for policy 1, policy_version 259922 (0.0007) [2023-12-26 17:15:14,643][105692] Updated weights for policy 0, policy_version 259647 (0.0008) [2023-12-26 17:15:14,693][105620] Updated weights for policy 1, policy_version 259932 (0.0007) [2023-12-26 17:15:14,750][105620] Updated weights for policy 1, policy_version 259942 (0.0009) [2023-12-26 17:15:15,351][105692] Updated weights for policy 0, policy_version 259657 (0.0007) [2023-12-26 17:15:15,407][105692] Updated weights for policy 0, policy_version 259667 (0.0009) [2023-12-26 17:15:15,470][105692] Updated weights for policy 0, policy_version 259677 (0.0009) [2023-12-26 17:15:15,514][105620] Updated weights for policy 1, policy_version 259952 (0.0008) [2023-12-26 17:15:15,529][105692] Updated weights for policy 0, policy_version 259687 (0.0008) [2023-12-26 17:15:15,576][105620] Updated weights for policy 1, policy_version 259962 (0.0008) [2023-12-26 17:15:15,636][105620] Updated weights for policy 1, policy_version 259972 (0.0009) [2023-12-26 17:15:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 133054464. Throughput: 0: 9836.5, 1: 9837.4. Samples: 133025804. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 17:15:16,062][104569] Avg episode reward: [(0, '9354.827'), (1, '9167.782')] [2023-12-26 17:15:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000259976_66560000.pth... [2023-12-26 17:15:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000259688_66494464.pth... [2023-12-26 17:15:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000258824_66265088.pth [2023-12-26 17:15:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000258536_66199552.pth [2023-12-26 17:15:16,230][105692] Updated weights for policy 0, policy_version 259697 (0.0008) [2023-12-26 17:15:16,276][105692] Updated weights for policy 0, policy_version 259707 (0.0005) [2023-12-26 17:15:16,325][105692] Updated weights for policy 0, policy_version 259717 (0.0005) [2023-12-26 17:15:16,381][105620] Updated weights for policy 1, policy_version 259982 (0.0007) [2023-12-26 17:15:16,426][105620] Updated weights for policy 1, policy_version 259992 (0.0005) [2023-12-26 17:15:16,472][105620] Updated weights for policy 1, policy_version 260002 (0.0006) [2023-12-26 17:15:17,033][105692] Updated weights for policy 0, policy_version 259727 (0.0008) [2023-12-26 17:15:17,097][105692] Updated weights for policy 0, policy_version 259737 (0.0009) [2023-12-26 17:15:17,162][105692] Updated weights for policy 0, policy_version 259747 (0.0009) [2023-12-26 17:15:17,204][105620] Updated weights for policy 1, policy_version 260012 (0.0008) [2023-12-26 17:15:17,272][105620] Updated weights for policy 1, policy_version 260022 (0.0009) [2023-12-26 17:15:17,334][105620] Updated weights for policy 1, policy_version 260032 (0.0009) [2023-12-26 17:15:17,900][105692] Updated weights for policy 0, policy_version 259757 (0.0009) [2023-12-26 17:15:17,947][105692] Updated weights for policy 0, policy_version 259767 (0.0009) [2023-12-26 17:15:18,002][105692] Updated weights for policy 0, policy_version 259777 (0.0009) [2023-12-26 17:15:18,073][105620] Updated weights for policy 1, policy_version 260042 (0.0009) [2023-12-26 17:15:18,127][105620] Updated weights for policy 1, policy_version 260052 (0.0008) [2023-12-26 17:15:18,174][105620] Updated weights for policy 1, policy_version 260062 (0.0009) [2023-12-26 17:15:18,221][105620] Updated weights for policy 1, policy_version 260072 (0.0009) [2023-12-26 17:15:18,770][105692] Updated weights for policy 0, policy_version 259787 (0.0009) [2023-12-26 17:15:18,832][105692] Updated weights for policy 0, policy_version 259797 (0.0008) [2023-12-26 17:15:18,891][105692] Updated weights for policy 0, policy_version 259807 (0.0009) [2023-12-26 17:15:18,986][105620] Updated weights for policy 1, policy_version 260082 (0.0008) [2023-12-26 17:15:19,038][105620] Updated weights for policy 1, policy_version 260092 (0.0008) [2023-12-26 17:15:19,088][105620] Updated weights for policy 1, policy_version 260102 (0.0010) [2023-12-26 17:15:19,635][105692] Updated weights for policy 0, policy_version 259817 (0.0009) [2023-12-26 17:15:19,690][105692] Updated weights for policy 0, policy_version 259827 (0.0009) [2023-12-26 17:15:19,757][105692] Updated weights for policy 0, policy_version 259837 (0.0009) [2023-12-26 17:15:19,817][105692] Updated weights for policy 0, policy_version 259847 (0.0009) [2023-12-26 17:15:19,908][105620] Updated weights for policy 1, policy_version 260112 (0.0008) [2023-12-26 17:15:19,982][105620] Updated weights for policy 1, policy_version 260122 (0.0009) [2023-12-26 17:15:20,050][105620] Updated weights for policy 1, policy_version 260132 (0.0009) [2023-12-26 17:15:20,506][105692] Updated weights for policy 0, policy_version 259857 (0.0008) [2023-12-26 17:15:20,569][105692] Updated weights for policy 0, policy_version 259867 (0.0006) [2023-12-26 17:15:20,627][105692] Updated weights for policy 0, policy_version 259877 (0.0007) [2023-12-26 17:15:20,852][105620] Updated weights for policy 1, policy_version 260142 (0.0010) [2023-12-26 17:15:20,907][105620] Updated weights for policy 1, policy_version 260152 (0.0009) [2023-12-26 17:15:20,970][105620] Updated weights for policy 1, policy_version 260162 (0.0010) [2023-12-26 17:15:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 133152768. Throughput: 0: 9879.2, 1: 9711.1. Samples: 133140504. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 17:15:21,063][104569] Avg episode reward: [(0, '9354.906'), (1, '9075.083')] [2023-12-26 17:15:21,284][105692] Updated weights for policy 0, policy_version 259887 (0.0007) [2023-12-26 17:15:21,354][105692] Updated weights for policy 0, policy_version 259897 (0.0007) [2023-12-26 17:15:21,419][105692] Updated weights for policy 0, policy_version 259907 (0.0009) [2023-12-26 17:15:21,818][105620] Updated weights for policy 1, policy_version 260172 (0.0009) [2023-12-26 17:15:21,876][105620] Updated weights for policy 1, policy_version 260182 (0.0008) [2023-12-26 17:15:21,928][105620] Updated weights for policy 1, policy_version 260192 (0.0008) [2023-12-26 17:15:22,120][105692] Updated weights for policy 0, policy_version 259917 (0.0008) [2023-12-26 17:15:22,183][105692] Updated weights for policy 0, policy_version 259927 (0.0008) [2023-12-26 17:15:22,251][105692] Updated weights for policy 0, policy_version 259937 (0.0010) [2023-12-26 17:15:22,702][105620] Updated weights for policy 1, policy_version 260202 (0.0008) [2023-12-26 17:15:22,755][105620] Updated weights for policy 1, policy_version 260212 (0.0009) [2023-12-26 17:15:22,814][105620] Updated weights for policy 1, policy_version 260222 (0.0009) [2023-12-26 17:15:22,878][105620] Updated weights for policy 1, policy_version 260232 (0.0010) [2023-12-26 17:15:23,016][105692] Updated weights for policy 0, policy_version 259947 (0.0009) [2023-12-26 17:15:23,067][105692] Updated weights for policy 0, policy_version 259957 (0.0009) [2023-12-26 17:15:23,119][105692] Updated weights for policy 0, policy_version 259967 (0.0008) [2023-12-26 17:15:23,636][105620] Updated weights for policy 1, policy_version 260242 (0.0009) [2023-12-26 17:15:23,696][105620] Updated weights for policy 1, policy_version 260252 (0.0009) [2023-12-26 17:15:23,763][105620] Updated weights for policy 1, policy_version 260262 (0.0008) [2023-12-26 17:15:23,891][105692] Updated weights for policy 0, policy_version 259977 (0.0008) [2023-12-26 17:15:23,946][105692] Updated weights for policy 0, policy_version 259987 (0.0010) [2023-12-26 17:15:24,004][105692] Updated weights for policy 0, policy_version 259997 (0.0010) [2023-12-26 17:15:24,072][105692] Updated weights for policy 0, policy_version 260007 (0.0010) [2023-12-26 17:15:24,432][105620] Updated weights for policy 1, policy_version 260272 (0.0006) [2023-12-26 17:15:24,482][105620] Updated weights for policy 1, policy_version 260282 (0.0006) [2023-12-26 17:15:24,531][105620] Updated weights for policy 1, policy_version 260292 (0.0005) [2023-12-26 17:15:24,811][105692] Updated weights for policy 0, policy_version 260017 (0.0010) [2023-12-26 17:15:24,869][105692] Updated weights for policy 0, policy_version 260027 (0.0010) [2023-12-26 17:15:24,928][105692] Updated weights for policy 0, policy_version 260037 (0.0010) [2023-12-26 17:15:25,131][105620] Updated weights for policy 1, policy_version 260302 (0.0007) [2023-12-26 17:15:25,182][105620] Updated weights for policy 1, policy_version 260312 (0.0005) [2023-12-26 17:15:25,234][105620] Updated weights for policy 1, policy_version 260322 (0.0005) [2023-12-26 17:15:25,801][105692] Updated weights for policy 0, policy_version 260047 (0.0008) [2023-12-26 17:15:25,861][105692] Updated weights for policy 0, policy_version 260057 (0.0008) [2023-12-26 17:15:25,881][105620] Updated weights for policy 1, policy_version 260332 (0.0007) [2023-12-26 17:15:25,918][105692] Updated weights for policy 0, policy_version 260067 (0.0007) [2023-12-26 17:15:25,934][105620] Updated weights for policy 1, policy_version 260342 (0.0010) [2023-12-26 17:15:25,994][105620] Updated weights for policy 1, policy_version 260352 (0.0010) [2023-12-26 17:15:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 133251072. Throughput: 0: 9858.1, 1: 9705.8. Samples: 133253948. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 17:15:26,063][104569] Avg episode reward: [(0, '9355.215'), (1, '9162.910')] [2023-12-26 17:15:26,662][105692] Updated weights for policy 0, policy_version 260077 (0.0010) [2023-12-26 17:15:26,702][105692] Updated weights for policy 0, policy_version 260087 (0.0010) [2023-12-26 17:15:26,732][105620] Updated weights for policy 1, policy_version 260362 (0.0010) [2023-12-26 17:15:26,762][105692] Updated weights for policy 0, policy_version 260097 (0.0006) [2023-12-26 17:15:26,783][105620] Updated weights for policy 1, policy_version 260372 (0.0010) [2023-12-26 17:15:26,837][105620] Updated weights for policy 1, policy_version 260382 (0.0010) [2023-12-26 17:15:26,892][105620] Updated weights for policy 1, policy_version 260392 (0.0007) [2023-12-26 17:15:27,383][105692] Updated weights for policy 0, policy_version 260107 (0.0008) [2023-12-26 17:15:27,446][105692] Updated weights for policy 0, policy_version 260117 (0.0005) [2023-12-26 17:15:27,513][105692] Updated weights for policy 0, policy_version 260127 (0.0005) [2023-12-26 17:15:27,575][105620] Updated weights for policy 1, policy_version 260402 (0.0006) [2023-12-26 17:15:27,630][105620] Updated weights for policy 1, policy_version 260412 (0.0005) [2023-12-26 17:15:27,683][105620] Updated weights for policy 1, policy_version 260422 (0.0006) [2023-12-26 17:15:28,036][105692] Updated weights for policy 0, policy_version 260137 (0.0006) [2023-12-26 17:15:28,096][105692] Updated weights for policy 0, policy_version 260147 (0.0010) [2023-12-26 17:15:28,157][105692] Updated weights for policy 0, policy_version 260157 (0.0010) [2023-12-26 17:15:28,217][105692] Updated weights for policy 0, policy_version 260167 (0.0010) [2023-12-26 17:15:28,292][105620] Updated weights for policy 1, policy_version 260432 (0.0005) [2023-12-26 17:15:28,349][105620] Updated weights for policy 1, policy_version 260442 (0.0008) [2023-12-26 17:15:28,410][105620] Updated weights for policy 1, policy_version 260452 (0.0009) [2023-12-26 17:15:28,887][105692] Updated weights for policy 0, policy_version 260177 (0.0010) [2023-12-26 17:15:28,945][105692] Updated weights for policy 0, policy_version 260187 (0.0009) [2023-12-26 17:15:29,001][105692] Updated weights for policy 0, policy_version 260197 (0.0010) [2023-12-26 17:15:29,110][105620] Updated weights for policy 1, policy_version 260462 (0.0011) [2023-12-26 17:15:29,171][105620] Updated weights for policy 1, policy_version 260472 (0.0011) [2023-12-26 17:15:29,230][105620] Updated weights for policy 1, policy_version 260482 (0.0011) [2023-12-26 17:15:29,773][105692] Updated weights for policy 0, policy_version 260207 (0.0008) [2023-12-26 17:15:29,825][105692] Updated weights for policy 0, policy_version 260217 (0.0010) [2023-12-26 17:15:29,888][105692] Updated weights for policy 0, policy_version 260227 (0.0010) [2023-12-26 17:15:30,028][105620] Updated weights for policy 1, policy_version 260492 (0.0010) [2023-12-26 17:15:30,084][105620] Updated weights for policy 1, policy_version 260502 (0.0011) [2023-12-26 17:15:30,136][105620] Updated weights for policy 1, policy_version 260512 (0.0010) [2023-12-26 17:15:30,557][105692] Updated weights for policy 0, policy_version 260237 (0.0008) [2023-12-26 17:15:30,603][105692] Updated weights for policy 0, policy_version 260247 (0.0005) [2023-12-26 17:15:30,648][105692] Updated weights for policy 0, policy_version 260257 (0.0005) [2023-12-26 17:15:30,855][105620] Updated weights for policy 1, policy_version 260522 (0.0009) [2023-12-26 17:15:30,914][105620] Updated weights for policy 1, policy_version 260532 (0.0005) [2023-12-26 17:15:30,970][105620] Updated weights for policy 1, policy_version 260542 (0.0008) [2023-12-26 17:15:31,028][105620] Updated weights for policy 1, policy_version 260552 (0.0010) [2023-12-26 17:15:31,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 133349376. Throughput: 0: 9867.6, 1: 9754.6. Samples: 133316560. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 17:15:31,062][104569] Avg episode reward: [(0, '9355.612'), (1, '9066.244')] [2023-12-26 17:15:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000260264_66641920.pth... [2023-12-26 17:15:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000260552_66707456.pth... [2023-12-26 17:15:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000259144_66355200.pth [2023-12-26 17:15:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000259400_66412544.pth [2023-12-26 17:15:31,358][105692] Updated weights for policy 0, policy_version 260267 (0.0008) [2023-12-26 17:15:31,421][105692] Updated weights for policy 0, policy_version 260277 (0.0008) [2023-12-26 17:15:31,485][105692] Updated weights for policy 0, policy_version 260287 (0.0008) [2023-12-26 17:15:31,678][105620] Updated weights for policy 1, policy_version 260562 (0.0010) [2023-12-26 17:15:31,744][105620] Updated weights for policy 1, policy_version 260572 (0.0009) [2023-12-26 17:15:31,794][105620] Updated weights for policy 1, policy_version 260582 (0.0009) [2023-12-26 17:15:32,187][105692] Updated weights for policy 0, policy_version 260297 (0.0008) [2023-12-26 17:15:32,240][105692] Updated weights for policy 0, policy_version 260307 (0.0008) [2023-12-26 17:15:32,291][105692] Updated weights for policy 0, policy_version 260317 (0.0008) [2023-12-26 17:15:32,354][105692] Updated weights for policy 0, policy_version 260327 (0.0009) [2023-12-26 17:15:32,532][105620] Updated weights for policy 1, policy_version 260592 (0.0010) [2023-12-26 17:15:32,587][105620] Updated weights for policy 1, policy_version 260602 (0.0010) [2023-12-26 17:15:32,634][105620] Updated weights for policy 1, policy_version 260612 (0.0010) [2023-12-26 17:15:33,120][105692] Updated weights for policy 0, policy_version 260337 (0.0008) [2023-12-26 17:15:33,166][105692] Updated weights for policy 0, policy_version 260347 (0.0005) [2023-12-26 17:15:33,215][105692] Updated weights for policy 0, policy_version 260357 (0.0006) [2023-12-26 17:15:33,237][105620] Updated weights for policy 1, policy_version 260622 (0.0007) [2023-12-26 17:15:33,288][105620] Updated weights for policy 1, policy_version 260632 (0.0010) [2023-12-26 17:15:33,339][105620] Updated weights for policy 1, policy_version 260642 (0.0010) [2023-12-26 17:15:33,779][105692] Updated weights for policy 0, policy_version 260367 (0.0005) [2023-12-26 17:15:33,838][105692] Updated weights for policy 0, policy_version 260377 (0.0011) [2023-12-26 17:15:33,892][105692] Updated weights for policy 0, policy_version 260387 (0.0010) [2023-12-26 17:15:34,058][105620] Updated weights for policy 1, policy_version 260652 (0.0010) [2023-12-26 17:15:34,110][105620] Updated weights for policy 1, policy_version 260662 (0.0008) [2023-12-26 17:15:34,171][105620] Updated weights for policy 1, policy_version 260672 (0.0008) [2023-12-26 17:15:34,623][105692] Updated weights for policy 0, policy_version 260397 (0.0010) [2023-12-26 17:15:34,683][105692] Updated weights for policy 0, policy_version 260407 (0.0011) [2023-12-26 17:15:34,736][105692] Updated weights for policy 0, policy_version 260417 (0.0010) [2023-12-26 17:15:34,960][105620] Updated weights for policy 1, policy_version 260682 (0.0008) [2023-12-26 17:15:35,021][105620] Updated weights for policy 1, policy_version 260692 (0.0009) [2023-12-26 17:15:35,079][105620] Updated weights for policy 1, policy_version 260702 (0.0008) [2023-12-26 17:15:35,132][105620] Updated weights for policy 1, policy_version 260712 (0.0008) [2023-12-26 17:15:35,496][105692] Updated weights for policy 0, policy_version 260427 (0.0010) [2023-12-26 17:15:35,557][105692] Updated weights for policy 0, policy_version 260437 (0.0010) [2023-12-26 17:15:35,622][105692] Updated weights for policy 0, policy_version 260447 (0.0010) [2023-12-26 17:15:35,924][105620] Updated weights for policy 1, policy_version 260722 (0.0008) [2023-12-26 17:15:35,980][105620] Updated weights for policy 1, policy_version 260732 (0.0008) [2023-12-26 17:15:36,041][105620] Updated weights for policy 1, policy_version 260742 (0.0009) [2023-12-26 17:15:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 133447680. Throughput: 0: 9825.4, 1: 9764.7. Samples: 133434904. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 17:15:36,062][104569] Avg episode reward: [(0, '9355.769'), (1, '9155.737')] [2023-12-26 17:15:36,376][105692] Updated weights for policy 0, policy_version 260457 (0.0010) [2023-12-26 17:15:36,436][105692] Updated weights for policy 0, policy_version 260467 (0.0010) [2023-12-26 17:15:36,489][105692] Updated weights for policy 0, policy_version 260477 (0.0010) [2023-12-26 17:15:36,542][105692] Updated weights for policy 0, policy_version 260487 (0.0011) [2023-12-26 17:15:36,841][105620] Updated weights for policy 1, policy_version 260752 (0.0008) [2023-12-26 17:15:36,910][105620] Updated weights for policy 1, policy_version 260762 (0.0008) [2023-12-26 17:15:36,978][105620] Updated weights for policy 1, policy_version 260772 (0.0008) [2023-12-26 17:15:37,338][105692] Updated weights for policy 0, policy_version 260497 (0.0011) [2023-12-26 17:15:37,387][105692] Updated weights for policy 0, policy_version 260507 (0.0010) [2023-12-26 17:15:37,450][105692] Updated weights for policy 0, policy_version 260517 (0.0011) [2023-12-26 17:15:37,757][105620] Updated weights for policy 1, policy_version 260782 (0.0009) [2023-12-26 17:15:37,825][105620] Updated weights for policy 1, policy_version 260792 (0.0008) [2023-12-26 17:15:37,891][105620] Updated weights for policy 1, policy_version 260802 (0.0009) [2023-12-26 17:15:38,222][105692] Updated weights for policy 0, policy_version 260527 (0.0011) [2023-12-26 17:15:38,289][105692] Updated weights for policy 0, policy_version 260537 (0.0011) [2023-12-26 17:15:38,356][105692] Updated weights for policy 0, policy_version 260547 (0.0011) [2023-12-26 17:15:38,675][105620] Updated weights for policy 1, policy_version 260812 (0.0008) [2023-12-26 17:15:38,732][105620] Updated weights for policy 1, policy_version 260822 (0.0008) [2023-12-26 17:15:38,788][105620] Updated weights for policy 1, policy_version 260832 (0.0009) [2023-12-26 17:15:39,123][105692] Updated weights for policy 0, policy_version 260557 (0.0010) [2023-12-26 17:15:39,174][105692] Updated weights for policy 0, policy_version 260567 (0.0009) [2023-12-26 17:15:39,239][105692] Updated weights for policy 0, policy_version 260577 (0.0009) [2023-12-26 17:15:39,582][105620] Updated weights for policy 1, policy_version 260842 (0.0008) [2023-12-26 17:15:39,648][105620] Updated weights for policy 1, policy_version 260852 (0.0009) [2023-12-26 17:15:39,715][105620] Updated weights for policy 1, policy_version 260862 (0.0008) [2023-12-26 17:15:39,778][105620] Updated weights for policy 1, policy_version 260872 (0.0009) [2023-12-26 17:15:39,990][105692] Updated weights for policy 0, policy_version 260587 (0.0010) [2023-12-26 17:15:40,051][105692] Updated weights for policy 0, policy_version 260597 (0.0010) [2023-12-26 17:15:40,105][105692] Updated weights for policy 0, policy_version 260607 (0.0009) [2023-12-26 17:15:40,586][105620] Updated weights for policy 1, policy_version 260882 (0.0009) [2023-12-26 17:15:40,650][105620] Updated weights for policy 1, policy_version 260892 (0.0008) [2023-12-26 17:15:40,713][105620] Updated weights for policy 1, policy_version 260902 (0.0008) [2023-12-26 17:15:40,843][105692] Updated weights for policy 0, policy_version 260617 (0.0009) [2023-12-26 17:15:40,899][105692] Updated weights for policy 0, policy_version 260627 (0.0007) [2023-12-26 17:15:40,957][105692] Updated weights for policy 0, policy_version 260637 (0.0008) [2023-12-26 17:15:41,020][105692] Updated weights for policy 0, policy_version 260647 (0.0009) [2023-12-26 17:15:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 133537792. Throughput: 0: 9729.3, 1: 9630.5. Samples: 133542324. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-26 17:15:41,063][104569] Avg episode reward: [(0, '9355.769'), (1, '9248.541')] [2023-12-26 17:15:41,563][105620] Updated weights for policy 1, policy_version 260912 (0.0008) [2023-12-26 17:15:41,651][105620] Updated weights for policy 1, policy_version 260922 (0.0009) [2023-12-26 17:15:41,717][105620] Updated weights for policy 1, policy_version 260932 (0.0007) [2023-12-26 17:15:41,854][105692] Updated weights for policy 0, policy_version 260657 (0.0009) [2023-12-26 17:15:41,903][105692] Updated weights for policy 0, policy_version 260667 (0.0009) [2023-12-26 17:15:41,964][105692] Updated weights for policy 0, policy_version 260677 (0.0008) [2023-12-26 17:15:42,503][105620] Updated weights for policy 1, policy_version 260942 (0.0008) [2023-12-26 17:15:42,565][105620] Updated weights for policy 1, policy_version 260952 (0.0007) [2023-12-26 17:15:42,623][105620] Updated weights for policy 1, policy_version 260962 (0.0008) [2023-12-26 17:15:42,805][105692] Updated weights for policy 0, policy_version 260687 (0.0008) [2023-12-26 17:15:42,858][105692] Updated weights for policy 0, policy_version 260697 (0.0009) [2023-12-26 17:15:42,911][105692] Updated weights for policy 0, policy_version 260707 (0.0010) [2023-12-26 17:15:43,225][105620] Updated weights for policy 1, policy_version 260972 (0.0007) [2023-12-26 17:15:43,270][105620] Updated weights for policy 1, policy_version 260982 (0.0008) [2023-12-26 17:15:43,330][105620] Updated weights for policy 1, policy_version 260992 (0.0008) [2023-12-26 17:15:43,642][105692] Updated weights for policy 0, policy_version 260717 (0.0011) [2023-12-26 17:15:43,690][105692] Updated weights for policy 0, policy_version 260727 (0.0010) [2023-12-26 17:15:43,739][105692] Updated weights for policy 0, policy_version 260737 (0.0009) [2023-12-26 17:15:44,145][105620] Updated weights for policy 1, policy_version 261002 (0.0008) [2023-12-26 17:15:44,216][105620] Updated weights for policy 1, policy_version 261012 (0.0008) [2023-12-26 17:15:44,275][105620] Updated weights for policy 1, policy_version 261022 (0.0008) [2023-12-26 17:15:44,326][105620] Updated weights for policy 1, policy_version 261032 (0.0007) [2023-12-26 17:15:44,468][105692] Updated weights for policy 0, policy_version 260747 (0.0009) [2023-12-26 17:15:44,535][105692] Updated weights for policy 0, policy_version 260757 (0.0010) [2023-12-26 17:15:44,588][105692] Updated weights for policy 0, policy_version 260767 (0.0011) [2023-12-26 17:15:45,083][105620] Updated weights for policy 1, policy_version 261042 (0.0010) [2023-12-26 17:15:45,145][105620] Updated weights for policy 1, policy_version 261052 (0.0008) [2023-12-26 17:15:45,210][105620] Updated weights for policy 1, policy_version 261062 (0.0008) [2023-12-26 17:15:45,338][105692] Updated weights for policy 0, policy_version 260777 (0.0010) [2023-12-26 17:15:45,396][105692] Updated weights for policy 0, policy_version 260787 (0.0009) [2023-12-26 17:15:45,452][105692] Updated weights for policy 0, policy_version 260797 (0.0010) [2023-12-26 17:15:45,509][105692] Updated weights for policy 0, policy_version 260807 (0.0011) [2023-12-26 17:15:45,867][105620] Updated weights for policy 1, policy_version 261072 (0.0010) [2023-12-26 17:15:45,915][105620] Updated weights for policy 1, policy_version 261082 (0.0009) [2023-12-26 17:15:45,966][105620] Updated weights for policy 1, policy_version 261092 (0.0005) [2023-12-26 17:15:46,062][104569] Fps is (10 sec: 18022.0, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 133627904. Throughput: 0: 9616.7, 1: 9613.5. Samples: 133597384. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-26 17:15:46,063][104569] Avg episode reward: [(0, '9355.886'), (1, '9252.330')] [2023-12-26 17:15:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000260808_66781184.pth... [2023-12-26 17:15:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000261096_66846720.pth... [2023-12-26 17:15:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000259688_66494464.pth [2023-12-26 17:15:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000259976_66560000.pth [2023-12-26 17:15:46,224][105692] Updated weights for policy 0, policy_version 260817 (0.0010) [2023-12-26 17:15:46,279][105692] Updated weights for policy 0, policy_version 260827 (0.0010) [2023-12-26 17:15:46,333][105692] Updated weights for policy 0, policy_version 260837 (0.0010) [2023-12-26 17:15:46,645][105620] Updated weights for policy 1, policy_version 261102 (0.0005) [2023-12-26 17:15:46,704][105620] Updated weights for policy 1, policy_version 261112 (0.0005) [2023-12-26 17:15:46,751][105620] Updated weights for policy 1, policy_version 261122 (0.0005) [2023-12-26 17:15:47,003][105692] Updated weights for policy 0, policy_version 260847 (0.0010) [2023-12-26 17:15:47,062][105692] Updated weights for policy 0, policy_version 260857 (0.0010) [2023-12-26 17:15:47,112][105692] Updated weights for policy 0, policy_version 260867 (0.0010) [2023-12-26 17:15:47,338][105620] Updated weights for policy 1, policy_version 261132 (0.0007) [2023-12-26 17:15:47,399][105620] Updated weights for policy 1, policy_version 261142 (0.0009) [2023-12-26 17:15:47,452][105620] Updated weights for policy 1, policy_version 261153 (0.0010) [2023-12-26 17:15:47,742][105692] Updated weights for policy 0, policy_version 260877 (0.0010) [2023-12-26 17:15:47,794][105692] Updated weights for policy 0, policy_version 260887 (0.0010) [2023-12-26 17:15:47,852][105692] Updated weights for policy 0, policy_version 260897 (0.0010) [2023-12-26 17:15:48,140][105620] Updated weights for policy 1, policy_version 261163 (0.0008) [2023-12-26 17:15:48,200][105620] Updated weights for policy 1, policy_version 261173 (0.0005) [2023-12-26 17:15:48,258][105620] Updated weights for policy 1, policy_version 261183 (0.0008) [2023-12-26 17:15:48,561][105692] Updated weights for policy 0, policy_version 260907 (0.0011) [2023-12-26 17:15:48,620][105692] Updated weights for policy 0, policy_version 260917 (0.0010) [2023-12-26 17:15:48,676][105692] Updated weights for policy 0, policy_version 260927 (0.0010) [2023-12-26 17:15:48,951][105620] Updated weights for policy 1, policy_version 261193 (0.0008) [2023-12-26 17:15:49,004][105620] Updated weights for policy 1, policy_version 261203 (0.0008) [2023-12-26 17:15:49,059][105620] Updated weights for policy 1, policy_version 261213 (0.0008) [2023-12-26 17:15:49,116][105620] Updated weights for policy 1, policy_version 261223 (0.0010) [2023-12-26 17:15:49,410][105692] Updated weights for policy 0, policy_version 260937 (0.0010) [2023-12-26 17:15:49,462][105692] Updated weights for policy 0, policy_version 260947 (0.0007) [2023-12-26 17:15:49,523][105692] Updated weights for policy 0, policy_version 260957 (0.0005) [2023-12-26 17:15:49,585][105692] Updated weights for policy 0, policy_version 260967 (0.0005) [2023-12-26 17:15:50,004][105620] Updated weights for policy 1, policy_version 261233 (0.0006) [2023-12-26 17:15:50,068][105620] Updated weights for policy 1, policy_version 261243 (0.0009) [2023-12-26 17:15:50,130][105620] Updated weights for policy 1, policy_version 261253 (0.0009) [2023-12-26 17:15:50,171][105692] Updated weights for policy 0, policy_version 260977 (0.0006) [2023-12-26 17:15:50,237][105692] Updated weights for policy 0, policy_version 260987 (0.0006) [2023-12-26 17:15:50,298][105692] Updated weights for policy 0, policy_version 260997 (0.0006) [2023-12-26 17:15:50,822][105620] Updated weights for policy 1, policy_version 261263 (0.0009) [2023-12-26 17:15:50,892][105620] Updated weights for policy 1, policy_version 261273 (0.0007) [2023-12-26 17:15:50,912][105692] Updated weights for policy 0, policy_version 261007 (0.0007) [2023-12-26 17:15:50,961][105620] Updated weights for policy 1, policy_version 261283 (0.0007) [2023-12-26 17:15:50,975][105692] Updated weights for policy 0, policy_version 261017 (0.0007) [2023-12-26 17:15:51,033][105692] Updated weights for policy 0, policy_version 261027 (0.0008) [2023-12-26 17:15:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 133726208. Throughput: 0: 9675.8, 1: 9613.3. Samples: 133715936. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-26 17:15:51,062][104569] Avg episode reward: [(0, '9356.319'), (1, '9256.913')] [2023-12-26 17:15:51,696][105620] Updated weights for policy 1, policy_version 261293 (0.0007) [2023-12-26 17:15:51,762][105620] Updated weights for policy 1, policy_version 261303 (0.0009) [2023-12-26 17:15:51,799][105692] Updated weights for policy 0, policy_version 261037 (0.0008) [2023-12-26 17:15:51,812][105620] Updated weights for policy 1, policy_version 261313 (0.0008) [2023-12-26 17:15:51,848][105692] Updated weights for policy 0, policy_version 261047 (0.0006) [2023-12-26 17:15:51,896][105692] Updated weights for policy 0, policy_version 261057 (0.0009) [2023-12-26 17:15:52,557][105620] Updated weights for policy 1, policy_version 261323 (0.0007) [2023-12-26 17:15:52,623][105620] Updated weights for policy 1, policy_version 261333 (0.0007) [2023-12-26 17:15:52,683][105620] Updated weights for policy 1, policy_version 261343 (0.0011) [2023-12-26 17:15:52,710][105692] Updated weights for policy 0, policy_version 261067 (0.0010) [2023-12-26 17:15:52,763][105692] Updated weights for policy 0, policy_version 261077 (0.0010) [2023-12-26 17:15:52,819][105692] Updated weights for policy 0, policy_version 261087 (0.0009) [2023-12-26 17:15:53,328][105620] Updated weights for policy 1, policy_version 261353 (0.0005) [2023-12-26 17:15:53,383][105620] Updated weights for policy 1, policy_version 261363 (0.0008) [2023-12-26 17:15:53,442][105620] Updated weights for policy 1, policy_version 261373 (0.0011) [2023-12-26 17:15:53,500][105620] Updated weights for policy 1, policy_version 261383 (0.0011) [2023-12-26 17:15:53,595][105692] Updated weights for policy 0, policy_version 261097 (0.0008) [2023-12-26 17:15:53,662][105692] Updated weights for policy 0, policy_version 261107 (0.0008) [2023-12-26 17:15:53,721][105692] Updated weights for policy 0, policy_version 261117 (0.0007) [2023-12-26 17:15:53,784][105692] Updated weights for policy 0, policy_version 261127 (0.0007) [2023-12-26 17:15:54,275][105620] Updated weights for policy 1, policy_version 261393 (0.0009) [2023-12-26 17:15:54,332][105620] Updated weights for policy 1, policy_version 261403 (0.0009) [2023-12-26 17:15:54,389][105620] Updated weights for policy 1, policy_version 261413 (0.0008) [2023-12-26 17:15:54,399][105692] Updated weights for policy 0, policy_version 261137 (0.0006) [2023-12-26 17:15:54,444][105692] Updated weights for policy 0, policy_version 261147 (0.0008) [2023-12-26 17:15:54,494][105692] Updated weights for policy 0, policy_version 261157 (0.0008) [2023-12-26 17:15:55,180][105692] Updated weights for policy 0, policy_version 261167 (0.0008) [2023-12-26 17:15:55,194][105620] Updated weights for policy 1, policy_version 261423 (0.0007) [2023-12-26 17:15:55,232][105692] Updated weights for policy 0, policy_version 261177 (0.0008) [2023-12-26 17:15:55,243][105620] Updated weights for policy 1, policy_version 261433 (0.0005) [2023-12-26 17:15:55,280][105692] Updated weights for policy 0, policy_version 261187 (0.0008) [2023-12-26 17:15:55,297][105620] Updated weights for policy 1, policy_version 261443 (0.0008) [2023-12-26 17:15:55,907][105692] Updated weights for policy 0, policy_version 261197 (0.0008) [2023-12-26 17:15:55,933][105585] KL-divergence is very high: 172.3901 [2023-12-26 17:15:55,938][105585] KL-divergence is very high: 1167.4213 [2023-12-26 17:15:55,953][105692] Updated weights for policy 0, policy_version 261207 (0.0007) [2023-12-26 17:15:55,978][105585] KL-divergence is very high: 333.1555 [2023-12-26 17:15:55,984][105585] KL-divergence is very high: 2063.4797 [2023-12-26 17:15:56,012][105692] Updated weights for policy 0, policy_version 261217 (0.0009) [2023-12-26 17:15:56,024][105585] KL-divergence is very high: 339.4998 [2023-12-26 17:15:56,030][105585] KL-divergence is very high: 2254.2949 [2023-12-26 17:15:56,062][104569] Fps is (10 sec: 19661.6, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 133824512. Throughput: 0: 9656.6, 1: 9488.6. Samples: 133831368. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-26 17:15:56,062][104569] Avg episode reward: [(0, '9264.263'), (1, '9258.579')] [2023-12-26 17:15:56,096][105620] Updated weights for policy 1, policy_version 261453 (0.0007) [2023-12-26 17:15:56,152][105620] Updated weights for policy 1, policy_version 261463 (0.0005) [2023-12-26 17:15:56,199][105620] Updated weights for policy 1, policy_version 261473 (0.0005) [2023-12-26 17:15:56,723][105692] Updated weights for policy 0, policy_version 261227 (0.0009) [2023-12-26 17:15:56,774][105692] Updated weights for policy 0, policy_version 261237 (0.0006) [2023-12-26 17:15:56,826][105692] Updated weights for policy 0, policy_version 261247 (0.0009) [2023-12-26 17:15:56,861][105620] Updated weights for policy 1, policy_version 261483 (0.0006) [2023-12-26 17:15:56,915][105620] Updated weights for policy 1, policy_version 261493 (0.0009) [2023-12-26 17:15:56,969][105620] Updated weights for policy 1, policy_version 261503 (0.0011) [2023-12-26 17:15:57,428][105692] Updated weights for policy 0, policy_version 261257 (0.0007) [2023-12-26 17:15:57,495][105692] Updated weights for policy 0, policy_version 261267 (0.0005) [2023-12-26 17:15:57,546][105692] Updated weights for policy 0, policy_version 261277 (0.0005) [2023-12-26 17:15:57,588][105692] Updated weights for policy 0, policy_version 261287 (0.0005) [2023-12-26 17:15:57,720][105620] Updated weights for policy 1, policy_version 261514 (0.0010) [2023-12-26 17:15:57,772][105620] Updated weights for policy 1, policy_version 261524 (0.0010) [2023-12-26 17:15:57,821][105620] Updated weights for policy 1, policy_version 261534 (0.0009) [2023-12-26 17:15:57,868][105620] Updated weights for policy 1, policy_version 261544 (0.0008) [2023-12-26 17:15:58,141][105692] Updated weights for policy 0, policy_version 261297 (0.0009) [2023-12-26 17:15:58,204][105692] Updated weights for policy 0, policy_version 261307 (0.0011) [2023-12-26 17:15:58,267][105692] Updated weights for policy 0, policy_version 261317 (0.0011) [2023-12-26 17:15:58,687][105620] Updated weights for policy 1, policy_version 261554 (0.0009) [2023-12-26 17:15:58,755][105620] Updated weights for policy 1, policy_version 261564 (0.0009) [2023-12-26 17:15:58,827][105620] Updated weights for policy 1, policy_version 261574 (0.0010) [2023-12-26 17:15:59,042][105692] Updated weights for policy 0, policy_version 261327 (0.0008) [2023-12-26 17:15:59,112][105692] Updated weights for policy 0, policy_version 261337 (0.0008) [2023-12-26 17:15:59,173][105692] Updated weights for policy 0, policy_version 261347 (0.0008) [2023-12-26 17:15:59,621][105620] Updated weights for policy 1, policy_version 261584 (0.0008) [2023-12-26 17:15:59,678][105620] Updated weights for policy 1, policy_version 261594 (0.0009) [2023-12-26 17:15:59,732][105620] Updated weights for policy 1, policy_version 261604 (0.0009) [2023-12-26 17:15:59,908][105692] Updated weights for policy 0, policy_version 261357 (0.0008) [2023-12-26 17:15:59,969][105692] Updated weights for policy 0, policy_version 261367 (0.0008) [2023-12-26 17:16:00,029][105692] Updated weights for policy 0, policy_version 261377 (0.0008) [2023-12-26 17:16:00,484][105620] Updated weights for policy 1, policy_version 261614 (0.0009) [2023-12-26 17:16:00,534][105620] Updated weights for policy 1, policy_version 261624 (0.0010) [2023-12-26 17:16:00,592][105620] Updated weights for policy 1, policy_version 261634 (0.0005) [2023-12-26 17:16:00,851][105692] Updated weights for policy 0, policy_version 261387 (0.0008) [2023-12-26 17:16:00,914][105692] Updated weights for policy 0, policy_version 261397 (0.0009) [2023-12-26 17:16:00,971][105692] Updated weights for policy 0, policy_version 261408 (0.0009) [2023-12-26 17:16:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 133922816. Throughput: 0: 9796.2, 1: 9460.7. Samples: 133892364. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-26 17:16:01,062][104569] Avg episode reward: [(0, '9264.119'), (1, '9257.542')] [2023-12-26 17:16:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000261640_66985984.pth... [2023-12-26 17:16:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000261416_66936832.pth... [2023-12-26 17:16:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000260264_66641920.pth [2023-12-26 17:16:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000260552_66707456.pth [2023-12-26 17:16:01,176][105620] Updated weights for policy 1, policy_version 261644 (0.0006) [2023-12-26 17:16:01,233][105620] Updated weights for policy 1, policy_version 261654 (0.0005) [2023-12-26 17:16:01,294][105620] Updated weights for policy 1, policy_version 261664 (0.0009) [2023-12-26 17:16:01,859][105692] Updated weights for policy 0, policy_version 261418 (0.0009) [2023-12-26 17:16:01,926][105692] Updated weights for policy 0, policy_version 261428 (0.0009) [2023-12-26 17:16:01,939][105620] Updated weights for policy 1, policy_version 261674 (0.0010) [2023-12-26 17:16:01,980][105692] Updated weights for policy 0, policy_version 261438 (0.0006) [2023-12-26 17:16:02,001][105620] Updated weights for policy 1, policy_version 261684 (0.0011) [2023-12-26 17:16:02,039][105692] Updated weights for policy 0, policy_version 261448 (0.0005) [2023-12-26 17:16:02,059][105620] Updated weights for policy 1, policy_version 261694 (0.0010) [2023-12-26 17:16:02,124][105620] Updated weights for policy 1, policy_version 261704 (0.0008) [2023-12-26 17:16:02,788][105620] Updated weights for policy 1, policy_version 261714 (0.0008) [2023-12-26 17:16:02,808][105692] Updated weights for policy 0, policy_version 261458 (0.0006) [2023-12-26 17:16:02,851][105620] Updated weights for policy 1, policy_version 261724 (0.0011) [2023-12-26 17:16:02,873][105692] Updated weights for policy 0, policy_version 261468 (0.0006) [2023-12-26 17:16:02,913][105620] Updated weights for policy 1, policy_version 261734 (0.0011) [2023-12-26 17:16:02,934][105692] Updated weights for policy 0, policy_version 261478 (0.0006) [2023-12-26 17:16:03,486][105620] Updated weights for policy 1, policy_version 261744 (0.0008) [2023-12-26 17:16:03,532][105692] Updated weights for policy 0, policy_version 261488 (0.0005) [2023-12-26 17:16:03,544][105620] Updated weights for policy 1, policy_version 261754 (0.0008) [2023-12-26 17:16:03,593][105692] Updated weights for policy 0, policy_version 261498 (0.0009) [2023-12-26 17:16:03,605][105620] Updated weights for policy 1, policy_version 261764 (0.0008) [2023-12-26 17:16:03,643][105692] Updated weights for policy 0, policy_version 261508 (0.0009) [2023-12-26 17:16:04,324][105692] Updated weights for policy 0, policy_version 261518 (0.0008) [2023-12-26 17:16:04,360][105620] Updated weights for policy 1, policy_version 261774 (0.0009) [2023-12-26 17:16:04,392][105692] Updated weights for policy 0, policy_version 261528 (0.0010) [2023-12-26 17:16:04,423][105620] Updated weights for policy 1, policy_version 261784 (0.0008) [2023-12-26 17:16:04,456][105692] Updated weights for policy 0, policy_version 261538 (0.0006) [2023-12-26 17:16:04,491][105620] Updated weights for policy 1, policy_version 261794 (0.0009) [2023-12-26 17:16:05,133][105692] Updated weights for policy 0, policy_version 261548 (0.0009) [2023-12-26 17:16:05,192][105692] Updated weights for policy 0, policy_version 261558 (0.0009) [2023-12-26 17:16:05,246][105692] Updated weights for policy 0, policy_version 261568 (0.0009) [2023-12-26 17:16:05,285][105620] Updated weights for policy 1, policy_version 261804 (0.0009) [2023-12-26 17:16:05,339][105620] Updated weights for policy 1, policy_version 261814 (0.0009) [2023-12-26 17:16:05,396][105620] Updated weights for policy 1, policy_version 261824 (0.0009) [2023-12-26 17:16:05,934][105692] Updated weights for policy 0, policy_version 261578 (0.0009) [2023-12-26 17:16:05,998][105692] Updated weights for policy 0, policy_version 261588 (0.0010) [2023-12-26 17:16:06,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 134012928. Throughput: 0: 9709.9, 1: 9566.9. Samples: 134007956. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-26 17:16:06,063][104569] Avg episode reward: [(0, '9264.025'), (1, '9256.113')] [2023-12-26 17:16:06,063][105692] Updated weights for policy 0, policy_version 261598 (0.0011) [2023-12-26 17:16:06,128][105692] Updated weights for policy 0, policy_version 261608 (0.0010) [2023-12-26 17:16:06,201][105620] Updated weights for policy 1, policy_version 261834 (0.0008) [2023-12-26 17:16:06,265][105620] Updated weights for policy 1, policy_version 261844 (0.0008) [2023-12-26 17:16:06,328][105620] Updated weights for policy 1, policy_version 261854 (0.0007) [2023-12-26 17:16:06,381][105620] Updated weights for policy 1, policy_version 261864 (0.0008) [2023-12-26 17:16:06,872][105692] Updated weights for policy 0, policy_version 261618 (0.0006) [2023-12-26 17:16:06,931][105692] Updated weights for policy 0, policy_version 261628 (0.0005) [2023-12-26 17:16:06,988][105692] Updated weights for policy 0, policy_version 261638 (0.0005) [2023-12-26 17:16:07,176][105620] Updated weights for policy 1, policy_version 261874 (0.0008) [2023-12-26 17:16:07,228][105620] Updated weights for policy 1, policy_version 261884 (0.0008) [2023-12-26 17:16:07,277][105620] Updated weights for policy 1, policy_version 261894 (0.0009) [2023-12-26 17:16:07,551][105692] Updated weights for policy 0, policy_version 261648 (0.0007) [2023-12-26 17:16:07,612][105692] Updated weights for policy 0, policy_version 261658 (0.0007) [2023-12-26 17:16:07,670][105692] Updated weights for policy 0, policy_version 261668 (0.0009) [2023-12-26 17:16:08,105][105620] Updated weights for policy 1, policy_version 261904 (0.0006) [2023-12-26 17:16:08,161][105620] Updated weights for policy 1, policy_version 261914 (0.0006) [2023-12-26 17:16:08,213][105620] Updated weights for policy 1, policy_version 261924 (0.0005) [2023-12-26 17:16:08,241][105692] Updated weights for policy 0, policy_version 261678 (0.0007) [2023-12-26 17:16:08,300][105692] Updated weights for policy 0, policy_version 261688 (0.0005) [2023-12-26 17:16:08,366][105692] Updated weights for policy 0, policy_version 261698 (0.0008) [2023-12-26 17:16:08,883][105620] Updated weights for policy 1, policy_version 261934 (0.0007) [2023-12-26 17:16:08,944][105620] Updated weights for policy 1, policy_version 261944 (0.0006) [2023-12-26 17:16:08,996][105692] Updated weights for policy 0, policy_version 261708 (0.0009) [2023-12-26 17:16:09,008][105620] Updated weights for policy 1, policy_version 261954 (0.0005) [2023-12-26 17:16:09,033][105585] KL-divergence is very high: 222.5010 [2023-12-26 17:16:09,042][105585] KL-divergence is very high: 211.5023 [2023-12-26 17:16:09,046][105692] Updated weights for policy 0, policy_version 261718 (0.0009) [2023-12-26 17:16:09,070][105585] KL-divergence is very high: 290.7747 [2023-12-26 17:16:09,080][105585] KL-divergence is very high: 172.7980 [2023-12-26 17:16:09,094][105692] Updated weights for policy 0, policy_version 261728 (0.0009) [2023-12-26 17:16:09,114][105585] KL-divergence is very high: 197.4736 [2023-12-26 17:16:09,679][105620] Updated weights for policy 1, policy_version 261964 (0.0008) [2023-12-26 17:16:09,742][105620] Updated weights for policy 1, policy_version 261974 (0.0009) [2023-12-26 17:16:09,799][105620] Updated weights for policy 1, policy_version 261984 (0.0008) [2023-12-26 17:16:09,860][105692] Updated weights for policy 0, policy_version 261738 (0.0009) [2023-12-26 17:16:09,923][105692] Updated weights for policy 0, policy_version 261748 (0.0008) [2023-12-26 17:16:09,988][105692] Updated weights for policy 0, policy_version 261758 (0.0008) [2023-12-26 17:16:10,059][105692] Updated weights for policy 0, policy_version 261768 (0.0008) [2023-12-26 17:16:10,596][105620] Updated weights for policy 1, policy_version 261994 (0.0010) [2023-12-26 17:16:10,658][105620] Updated weights for policy 1, policy_version 262004 (0.0009) [2023-12-26 17:16:10,720][105620] Updated weights for policy 1, policy_version 262014 (0.0009) [2023-12-26 17:16:10,783][105620] Updated weights for policy 1, policy_version 262024 (0.0009) [2023-12-26 17:16:10,809][105692] Updated weights for policy 0, policy_version 261778 (0.0005) [2023-12-26 17:16:10,867][105692] Updated weights for policy 0, policy_version 261788 (0.0006) [2023-12-26 17:16:10,923][105692] Updated weights for policy 0, policy_version 261798 (0.0007) [2023-12-26 17:16:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 134119424. Throughput: 0: 9822.4, 1: 9521.2. Samples: 134124408. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-26 17:16:11,062][104569] Avg episode reward: [(0, '9081.443'), (1, '9169.124')] [2023-12-26 17:16:11,584][105620] Updated weights for policy 1, policy_version 262034 (0.0009) [2023-12-26 17:16:11,642][105692] Updated weights for policy 0, policy_version 261808 (0.0008) [2023-12-26 17:16:11,650][105620] Updated weights for policy 1, policy_version 262044 (0.0006) [2023-12-26 17:16:11,708][105620] Updated weights for policy 1, policy_version 262054 (0.0006) [2023-12-26 17:16:11,710][105692] Updated weights for policy 0, policy_version 261818 (0.0007) [2023-12-26 17:16:11,785][105692] Updated weights for policy 0, policy_version 261828 (0.0007) [2023-12-26 17:16:12,470][105620] Updated weights for policy 1, policy_version 262064 (0.0009) [2023-12-26 17:16:12,519][105692] Updated weights for policy 0, policy_version 261838 (0.0006) [2023-12-26 17:16:12,531][105620] Updated weights for policy 1, policy_version 262074 (0.0007) [2023-12-26 17:16:12,579][105692] Updated weights for policy 0, policy_version 261848 (0.0008) [2023-12-26 17:16:12,581][105620] Updated weights for policy 1, policy_version 262084 (0.0008) [2023-12-26 17:16:12,637][105692] Updated weights for policy 0, policy_version 261858 (0.0009) [2023-12-26 17:16:13,306][105692] Updated weights for policy 0, policy_version 261868 (0.0008) [2023-12-26 17:16:13,372][105692] Updated weights for policy 0, policy_version 261878 (0.0007) [2023-12-26 17:16:13,388][105620] Updated weights for policy 1, policy_version 262094 (0.0008) [2023-12-26 17:16:13,437][105692] Updated weights for policy 0, policy_version 261888 (0.0008) [2023-12-26 17:16:13,440][105620] Updated weights for policy 1, policy_version 262104 (0.0006) [2023-12-26 17:16:13,486][105620] Updated weights for policy 1, policy_version 262114 (0.0008) [2023-12-26 17:16:14,107][105692] Updated weights for policy 0, policy_version 261898 (0.0009) [2023-12-26 17:16:14,158][105692] Updated weights for policy 0, policy_version 261908 (0.0010) [2023-12-26 17:16:14,212][105692] Updated weights for policy 0, policy_version 261918 (0.0010) [2023-12-26 17:16:14,279][105620] Updated weights for policy 1, policy_version 262124 (0.0008) [2023-12-26 17:16:14,280][105692] Updated weights for policy 0, policy_version 261928 (0.0010) [2023-12-26 17:16:14,333][105620] Updated weights for policy 1, policy_version 262134 (0.0007) [2023-12-26 17:16:14,395][105620] Updated weights for policy 1, policy_version 262144 (0.0008) [2023-12-26 17:16:15,033][105692] Updated weights for policy 0, policy_version 261938 (0.0009) [2023-12-26 17:16:15,092][105692] Updated weights for policy 0, policy_version 261948 (0.0009) [2023-12-26 17:16:15,153][105620] Updated weights for policy 1, policy_version 262154 (0.0008) [2023-12-26 17:16:15,160][105692] Updated weights for policy 0, policy_version 261958 (0.0008) [2023-12-26 17:16:15,213][105620] Updated weights for policy 1, policy_version 262164 (0.0008) [2023-12-26 17:16:15,270][105620] Updated weights for policy 1, policy_version 262174 (0.0009) [2023-12-26 17:16:15,330][105620] Updated weights for policy 1, policy_version 262184 (0.0009) [2023-12-26 17:16:15,771][105692] Updated weights for policy 0, policy_version 261968 (0.0009) [2023-12-26 17:16:15,780][105585] KL-divergence is very high: 105.8969 [2023-12-26 17:16:15,785][105585] KL-divergence is very high: 144.7758 [2023-12-26 17:16:15,801][105585] KL-divergence is very high: 1720.0979 [2023-12-26 17:16:15,817][105585] KL-divergence is very high: 214.9777 [2023-12-26 17:16:15,822][105585] KL-divergence is very high: 372.8235 [2023-12-26 17:16:15,822][105692] Updated weights for policy 0, policy_version 261978 (0.0005) [2023-12-26 17:16:15,828][105585] KL-divergence is very high: 357.6470 [2023-12-26 17:16:15,846][105585] KL-divergence is very high: 2783.8542 [2023-12-26 17:16:15,865][105585] KL-divergence is very high: 207.8521 [2023-12-26 17:16:15,872][105585] KL-divergence is very high: 364.1698 [2023-12-26 17:16:15,878][105585] KL-divergence is very high: 323.7413 [2023-12-26 17:16:15,884][105692] Updated weights for policy 0, policy_version 261988 (0.0005) [2023-12-26 17:16:15,896][105585] KL-divergence is very high: 2691.5278 [2023-12-26 17:16:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 134209536. Throughput: 0: 9771.6, 1: 9434.0. Samples: 134180812. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-26 17:16:16,062][104569] Avg episode reward: [(0, '8990.176'), (1, '9171.836')] [2023-12-26 17:16:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000261992_67084288.pth... [2023-12-26 17:16:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000262184_67125248.pth... [2023-12-26 17:16:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000260808_66781184.pth [2023-12-26 17:16:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000261096_66846720.pth [2023-12-26 17:16:16,204][105620] Updated weights for policy 1, policy_version 262194 (0.0010) [2023-12-26 17:16:16,258][105620] Updated weights for policy 1, policy_version 262205 (0.0010) [2023-12-26 17:16:16,320][105620] Updated weights for policy 1, policy_version 262215 (0.0010) [2023-12-26 17:16:16,407][105585] KL-divergence is very high: 137.7481 [2023-12-26 17:16:16,424][105692] Updated weights for policy 0, policy_version 261998 (0.0007) [2023-12-26 17:16:16,482][105692] Updated weights for policy 0, policy_version 262008 (0.0007) [2023-12-26 17:16:16,529][105692] Updated weights for policy 0, policy_version 262018 (0.0009) [2023-12-26 17:16:16,530][105585] KL-divergence is very high: 105.9032 [2023-12-26 17:16:17,128][105620] Updated weights for policy 1, policy_version 262225 (0.0008) [2023-12-26 17:16:17,187][105620] Updated weights for policy 1, policy_version 262235 (0.0009) [2023-12-26 17:16:17,235][105620] Updated weights for policy 1, policy_version 262245 (0.0009) [2023-12-26 17:16:17,257][105692] Updated weights for policy 0, policy_version 262028 (0.0008) [2023-12-26 17:16:17,317][105692] Updated weights for policy 0, policy_version 262038 (0.0008) [2023-12-26 17:16:17,375][105692] Updated weights for policy 0, policy_version 262048 (0.0009) [2023-12-26 17:16:17,991][105620] Updated weights for policy 1, policy_version 262255 (0.0008) [2023-12-26 17:16:18,043][105620] Updated weights for policy 1, policy_version 262265 (0.0007) [2023-12-26 17:16:18,091][105620] Updated weights for policy 1, policy_version 262275 (0.0009) [2023-12-26 17:16:18,123][105692] Updated weights for policy 0, policy_version 262058 (0.0009) [2023-12-26 17:16:18,188][105692] Updated weights for policy 0, policy_version 262068 (0.0009) [2023-12-26 17:16:18,246][105692] Updated weights for policy 0, policy_version 262078 (0.0009) [2023-12-26 17:16:18,304][105692] Updated weights for policy 0, policy_version 262088 (0.0008) [2023-12-26 17:16:18,912][105620] Updated weights for policy 1, policy_version 262285 (0.0009) [2023-12-26 17:16:18,961][105620] Updated weights for policy 1, policy_version 262295 (0.0009) [2023-12-26 17:16:18,965][105692] Updated weights for policy 0, policy_version 262098 (0.0010) [2023-12-26 17:16:19,008][105620] Updated weights for policy 1, policy_version 262305 (0.0008) [2023-12-26 17:16:19,014][105692] Updated weights for policy 0, policy_version 262108 (0.0007) [2023-12-26 17:16:19,072][105692] Updated weights for policy 0, policy_version 262118 (0.0008) [2023-12-26 17:16:19,817][105620] Updated weights for policy 1, policy_version 262315 (0.0006) [2023-12-26 17:16:19,850][105692] Updated weights for policy 0, policy_version 262128 (0.0008) [2023-12-26 17:16:19,880][105620] Updated weights for policy 1, policy_version 262325 (0.0007) [2023-12-26 17:16:19,917][105692] Updated weights for policy 0, policy_version 262138 (0.0008) [2023-12-26 17:16:19,946][105620] Updated weights for policy 1, policy_version 262335 (0.0008) [2023-12-26 17:16:19,981][105692] Updated weights for policy 0, policy_version 262148 (0.0006) [2023-12-26 17:16:20,686][105620] Updated weights for policy 1, policy_version 262345 (0.0007) [2023-12-26 17:16:20,739][105620] Updated weights for policy 1, policy_version 262355 (0.0007) [2023-12-26 17:16:20,749][105692] Updated weights for policy 0, policy_version 262158 (0.0008) [2023-12-26 17:16:20,791][105620] Updated weights for policy 1, policy_version 262365 (0.0009) [2023-12-26 17:16:20,805][105692] Updated weights for policy 0, policy_version 262168 (0.0008) [2023-12-26 17:16:20,856][105620] Updated weights for policy 1, policy_version 262375 (0.0007) [2023-12-26 17:16:20,862][105692] Updated weights for policy 0, policy_version 262178 (0.0007) [2023-12-26 17:16:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.3, 300 sec: 19522.0). Total num frames: 134307840. Throughput: 0: 9786.6, 1: 9313.0. Samples: 134294384. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-26 17:16:21,062][104569] Avg episode reward: [(0, '7901.462'), (1, '9171.290')] [2023-12-26 17:16:21,608][105620] Updated weights for policy 1, policy_version 262385 (0.0008) [2023-12-26 17:16:21,631][105692] Updated weights for policy 0, policy_version 262188 (0.0007) [2023-12-26 17:16:21,671][105620] Updated weights for policy 1, policy_version 262395 (0.0009) [2023-12-26 17:16:21,697][105692] Updated weights for policy 0, policy_version 262198 (0.0009) [2023-12-26 17:16:21,738][105620] Updated weights for policy 1, policy_version 262405 (0.0007) [2023-12-26 17:16:21,756][105585] KL-divergence is very high: 165.7916 [2023-12-26 17:16:21,770][105692] Updated weights for policy 0, policy_version 262208 (0.0008) [2023-12-26 17:16:21,779][105585] KL-divergence is very high: 970.5262 [2023-12-26 17:16:21,785][105585] KL-divergence is very high: 146.9408 [2023-12-26 17:16:21,805][105585] KL-divergence is very high: 474.0181 [2023-12-26 17:16:21,810][105585] KL-divergence is very high: 105.0779 [2023-12-26 17:16:22,482][105620] Updated weights for policy 1, policy_version 262415 (0.0009) [2023-12-26 17:16:22,484][105585] KL-divergence is very high: 990.4377 [2023-12-26 17:16:22,490][105585] KL-divergence is very high: 398.9142 [2023-12-26 17:16:22,494][105692] Updated weights for policy 0, policy_version 262218 (0.0008) [2023-12-26 17:16:22,528][105585] KL-divergence is very high: 706.2649 [2023-12-26 17:16:22,535][105585] KL-divergence is very high: 228.0729 [2023-12-26 17:16:22,544][105620] Updated weights for policy 1, policy_version 262425 (0.0006) [2023-12-26 17:16:22,552][105692] Updated weights for policy 0, policy_version 262228 (0.0008) [2023-12-26 17:16:22,576][105585] KL-divergence is very high: 691.8511 [2023-12-26 17:16:22,582][105585] KL-divergence is very high: 267.9427 [2023-12-26 17:16:22,602][105620] Updated weights for policy 1, policy_version 262435 (0.0006) [2023-12-26 17:16:22,611][105692] Updated weights for policy 0, policy_version 262238 (0.0009) [2023-12-26 17:16:22,622][105585] KL-divergence is very high: 623.9990 [2023-12-26 17:16:22,628][105585] KL-divergence is very high: 237.6422 [2023-12-26 17:16:22,663][105692] Updated weights for policy 0, policy_version 262248 (0.0008) [2023-12-26 17:16:23,308][105620] Updated weights for policy 1, policy_version 262445 (0.0008) [2023-12-26 17:16:23,326][105585] KL-divergence is very high: 101.3349 [2023-12-26 17:16:23,349][105692] Updated weights for policy 0, policy_version 262258 (0.0011) [2023-12-26 17:16:23,366][105620] Updated weights for policy 1, policy_version 262455 (0.0010) [2023-12-26 17:16:23,410][105692] Updated weights for policy 0, policy_version 262268 (0.0010) [2023-12-26 17:16:23,421][105620] Updated weights for policy 1, policy_version 262465 (0.0010) [2023-12-26 17:16:23,468][105692] Updated weights for policy 0, policy_version 262278 (0.0011) [2023-12-26 17:16:24,202][105620] Updated weights for policy 1, policy_version 262475 (0.0011) [2023-12-26 17:16:24,213][105692] Updated weights for policy 0, policy_version 262288 (0.0011) [2023-12-26 17:16:24,262][105620] Updated weights for policy 1, policy_version 262485 (0.0011) [2023-12-26 17:16:24,275][105692] Updated weights for policy 0, policy_version 262298 (0.0011) [2023-12-26 17:16:24,280][105585] KL-divergence is very high: 109.7593 [2023-12-26 17:16:24,318][105620] Updated weights for policy 1, policy_version 262495 (0.0011) [2023-12-26 17:16:24,335][105692] Updated weights for policy 0, policy_version 262308 (0.0011) [2023-12-26 17:16:25,035][105620] Updated weights for policy 1, policy_version 262505 (0.0011) [2023-12-26 17:16:25,041][105692] Updated weights for policy 0, policy_version 262318 (0.0010) [2023-12-26 17:16:25,089][105692] Updated weights for policy 0, policy_version 262328 (0.0010) [2023-12-26 17:16:25,097][105620] Updated weights for policy 1, policy_version 262515 (0.0010) [2023-12-26 17:16:25,140][105692] Updated weights for policy 0, policy_version 262338 (0.0010) [2023-12-26 17:16:25,155][105620] Updated weights for policy 1, policy_version 262525 (0.0010) [2023-12-26 17:16:25,216][105620] Updated weights for policy 1, policy_version 262535 (0.0010) [2023-12-26 17:16:25,864][105620] Updated weights for policy 1, policy_version 262545 (0.0006) [2023-12-26 17:16:25,897][105692] Updated weights for policy 0, policy_version 262348 (0.0010) [2023-12-26 17:16:25,911][105620] Updated weights for policy 1, policy_version 262555 (0.0005) [2023-12-26 17:16:25,961][105692] Updated weights for policy 0, policy_version 262358 (0.0010) [2023-12-26 17:16:25,974][105620] Updated weights for policy 1, policy_version 262565 (0.0005) [2023-12-26 17:16:26,016][105692] Updated weights for policy 0, policy_version 262368 (0.0010) [2023-12-26 17:16:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 134406144. Throughput: 0: 9822.6, 1: 9425.5. Samples: 134408492. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-26 17:16:26,063][104569] Avg episode reward: [(0, '4805.231'), (1, '9265.017')] [2023-12-26 17:16:26,608][105620] Updated weights for policy 1, policy_version 262575 (0.0006) [2023-12-26 17:16:26,667][105620] Updated weights for policy 1, policy_version 262585 (0.0007) [2023-12-26 17:16:26,725][105620] Updated weights for policy 1, policy_version 262595 (0.0005) [2023-12-26 17:16:26,749][105692] Updated weights for policy 0, policy_version 262378 (0.0009) [2023-12-26 17:16:26,799][105692] Updated weights for policy 0, policy_version 262388 (0.0008) [2023-12-26 17:16:26,857][105692] Updated weights for policy 0, policy_version 262398 (0.0009) [2023-12-26 17:16:26,915][105692] Updated weights for policy 0, policy_version 262408 (0.0008) [2023-12-26 17:16:27,285][105620] Updated weights for policy 1, policy_version 262605 (0.0008) [2023-12-26 17:16:27,344][105620] Updated weights for policy 1, policy_version 262615 (0.0006) [2023-12-26 17:16:27,394][105620] Updated weights for policy 1, policy_version 262625 (0.0005) [2023-12-26 17:16:27,570][105692] Updated weights for policy 0, policy_version 262418 (0.0008) [2023-12-26 17:16:27,621][105692] Updated weights for policy 0, policy_version 262428 (0.0010) [2023-12-26 17:16:27,669][105692] Updated weights for policy 0, policy_version 262438 (0.0009) [2023-12-26 17:16:28,076][105620] Updated weights for policy 1, policy_version 262635 (0.0008) [2023-12-26 17:16:28,123][105620] Updated weights for policy 1, policy_version 262645 (0.0010) [2023-12-26 17:16:28,181][105620] Updated weights for policy 1, policy_version 262655 (0.0010) [2023-12-26 17:16:28,366][105692] Updated weights for policy 0, policy_version 262448 (0.0008) [2023-12-26 17:16:28,412][105692] Updated weights for policy 0, policy_version 262458 (0.0006) [2023-12-26 17:16:28,477][105692] Updated weights for policy 0, policy_version 262468 (0.0005) [2023-12-26 17:16:28,860][105620] Updated weights for policy 1, policy_version 262665 (0.0008) [2023-12-26 17:16:28,909][105620] Updated weights for policy 1, policy_version 262675 (0.0009) [2023-12-26 17:16:28,954][105620] Updated weights for policy 1, policy_version 262685 (0.0010) [2023-12-26 17:16:29,003][105620] Updated weights for policy 1, policy_version 262695 (0.0008) [2023-12-26 17:16:29,082][105692] Updated weights for policy 0, policy_version 262478 (0.0006) [2023-12-26 17:16:29,139][105692] Updated weights for policy 0, policy_version 262488 (0.0010) [2023-12-26 17:16:29,194][105692] Updated weights for policy 0, policy_version 262498 (0.0010) [2023-12-26 17:16:29,791][105620] Updated weights for policy 1, policy_version 262705 (0.0009) [2023-12-26 17:16:29,853][105620] Updated weights for policy 1, policy_version 262715 (0.0009) [2023-12-26 17:16:29,872][105692] Updated weights for policy 0, policy_version 262508 (0.0010) [2023-12-26 17:16:29,914][105620] Updated weights for policy 1, policy_version 262725 (0.0006) [2023-12-26 17:16:29,937][105692] Updated weights for policy 0, policy_version 262518 (0.0011) [2023-12-26 17:16:29,996][105692] Updated weights for policy 0, policy_version 262528 (0.0009) [2023-12-26 17:16:30,616][105620] Updated weights for policy 1, policy_version 262735 (0.0006) [2023-12-26 17:16:30,683][105620] Updated weights for policy 1, policy_version 262745 (0.0008) [2023-12-26 17:16:30,734][105692] Updated weights for policy 0, policy_version 262538 (0.0009) [2023-12-26 17:16:30,744][105620] Updated weights for policy 1, policy_version 262755 (0.0007) [2023-12-26 17:16:30,789][105692] Updated weights for policy 0, policy_version 262548 (0.0010) [2023-12-26 17:16:30,844][105692] Updated weights for policy 0, policy_version 262558 (0.0010) [2023-12-26 17:16:30,902][105692] Updated weights for policy 0, policy_version 262568 (0.0010) [2023-12-26 17:16:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 134504448. Throughput: 0: 9898.6, 1: 9514.4. Samples: 134470968. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-26 17:16:31,062][104569] Avg episode reward: [(0, '5542.235'), (1, '9169.503')] [2023-12-26 17:16:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000262568_67231744.pth... [2023-12-26 17:16:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000262760_67272704.pth... [2023-12-26 17:16:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000261416_66936832.pth [2023-12-26 17:16:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000261640_66985984.pth [2023-12-26 17:16:31,407][105620] Updated weights for policy 1, policy_version 262765 (0.0008) [2023-12-26 17:16:31,470][105620] Updated weights for policy 1, policy_version 262775 (0.0011) [2023-12-26 17:16:31,532][105620] Updated weights for policy 1, policy_version 262785 (0.0010) [2023-12-26 17:16:31,684][105692] Updated weights for policy 0, policy_version 262578 (0.0011) [2023-12-26 17:16:31,752][105692] Updated weights for policy 0, policy_version 262588 (0.0009) [2023-12-26 17:16:31,813][105692] Updated weights for policy 0, policy_version 262598 (0.0006) [2023-12-26 17:16:32,273][105620] Updated weights for policy 1, policy_version 262795 (0.0010) [2023-12-26 17:16:32,328][105620] Updated weights for policy 1, policy_version 262805 (0.0008) [2023-12-26 17:16:32,373][105692] Updated weights for policy 0, policy_version 262608 (0.0009) [2023-12-26 17:16:32,384][105620] Updated weights for policy 1, policy_version 262815 (0.0006) [2023-12-26 17:16:32,432][105692] Updated weights for policy 0, policy_version 262618 (0.0009) [2023-12-26 17:16:32,494][105692] Updated weights for policy 0, policy_version 262628 (0.0010) [2023-12-26 17:16:33,155][105620] Updated weights for policy 1, policy_version 262825 (0.0007) [2023-12-26 17:16:33,187][105692] Updated weights for policy 0, policy_version 262638 (0.0007) [2023-12-26 17:16:33,216][105620] Updated weights for policy 1, policy_version 262835 (0.0010) [2023-12-26 17:16:33,240][105692] Updated weights for policy 0, policy_version 262648 (0.0005) [2023-12-26 17:16:33,267][105620] Updated weights for policy 1, policy_version 262845 (0.0010) [2023-12-26 17:16:33,290][105692] Updated weights for policy 0, policy_version 262658 (0.0005) [2023-12-26 17:16:33,321][105620] Updated weights for policy 1, policy_version 262855 (0.0010) [2023-12-26 17:16:33,880][105692] Updated weights for policy 0, policy_version 262668 (0.0005) [2023-12-26 17:16:33,894][105620] Updated weights for policy 1, policy_version 262865 (0.0006) [2023-12-26 17:16:33,933][105692] Updated weights for policy 0, policy_version 262678 (0.0005) [2023-12-26 17:16:33,943][105620] Updated weights for policy 1, policy_version 262875 (0.0008) [2023-12-26 17:16:33,982][105692] Updated weights for policy 0, policy_version 262688 (0.0006) [2023-12-26 17:16:34,002][105620] Updated weights for policy 1, policy_version 262885 (0.0005) [2023-12-26 17:16:34,651][105692] Updated weights for policy 0, policy_version 262698 (0.0006) [2023-12-26 17:16:34,693][105620] Updated weights for policy 1, policy_version 262895 (0.0006) [2023-12-26 17:16:34,711][105692] Updated weights for policy 0, policy_version 262708 (0.0011) [2023-12-26 17:16:34,747][105620] Updated weights for policy 1, policy_version 262905 (0.0007) [2023-12-26 17:16:34,770][105692] Updated weights for policy 0, policy_version 262718 (0.0011) [2023-12-26 17:16:34,796][105620] Updated weights for policy 1, policy_version 262915 (0.0005) [2023-12-26 17:16:34,829][105692] Updated weights for policy 0, policy_version 262728 (0.0010) [2023-12-26 17:16:35,454][105620] Updated weights for policy 1, policy_version 262925 (0.0010) [2023-12-26 17:16:35,513][105620] Updated weights for policy 1, policy_version 262935 (0.0011) [2023-12-26 17:16:35,537][105692] Updated weights for policy 0, policy_version 262738 (0.0011) [2023-12-26 17:16:35,574][105620] Updated weights for policy 1, policy_version 262945 (0.0011) [2023-12-26 17:16:35,591][105692] Updated weights for policy 0, policy_version 262748 (0.0010) [2023-12-26 17:16:35,641][105692] Updated weights for policy 0, policy_version 262758 (0.0008) [2023-12-26 17:16:36,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 134602752. Throughput: 0: 9958.1, 1: 9513.3. Samples: 134592148. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-26 17:16:36,062][104569] Avg episode reward: [(0, '606.604'), (1, '9256.794')] [2023-12-26 17:16:36,335][105620] Updated weights for policy 1, policy_version 262955 (0.0010) [2023-12-26 17:16:36,399][105620] Updated weights for policy 1, policy_version 262965 (0.0008) [2023-12-26 17:16:36,437][105692] Updated weights for policy 0, policy_version 262768 (0.0010) [2023-12-26 17:16:36,458][105585] KL-divergence is very high: 122.8880 [2023-12-26 17:16:36,460][105620] Updated weights for policy 1, policy_version 262975 (0.0006) [2023-12-26 17:16:36,463][105585] KL-divergence is very high: 101.8566 [2023-12-26 17:16:36,494][105692] Updated weights for policy 0, policy_version 262778 (0.0011) [2023-12-26 17:16:36,550][105692] Updated weights for policy 0, policy_version 262788 (0.0011) [2023-12-26 17:16:37,166][105620] Updated weights for policy 1, policy_version 262985 (0.0006) [2023-12-26 17:16:37,215][105620] Updated weights for policy 1, policy_version 262995 (0.0008) [2023-12-26 17:16:37,273][105620] Updated weights for policy 1, policy_version 263005 (0.0009) [2023-12-26 17:16:37,307][105692] Updated weights for policy 0, policy_version 262798 (0.0008) [2023-12-26 17:16:37,329][105620] Updated weights for policy 1, policy_version 263015 (0.0009) [2023-12-26 17:16:37,358][105692] Updated weights for policy 0, policy_version 262808 (0.0008) [2023-12-26 17:16:37,411][105692] Updated weights for policy 0, policy_version 262818 (0.0010) [2023-12-26 17:16:38,080][105620] Updated weights for policy 1, policy_version 263025 (0.0008) [2023-12-26 17:16:38,149][105620] Updated weights for policy 1, policy_version 263035 (0.0005) [2023-12-26 17:16:38,160][105692] Updated weights for policy 0, policy_version 262828 (0.0009) [2023-12-26 17:16:38,201][105620] Updated weights for policy 1, policy_version 263045 (0.0005) [2023-12-26 17:16:38,225][105692] Updated weights for policy 0, policy_version 262838 (0.0008) [2023-12-26 17:16:38,283][105692] Updated weights for policy 0, policy_version 262848 (0.0008) [2023-12-26 17:16:38,780][105620] Updated weights for policy 1, policy_version 263055 (0.0009) [2023-12-26 17:16:38,833][105620] Updated weights for policy 1, policy_version 263065 (0.0010) [2023-12-26 17:16:38,889][105620] Updated weights for policy 1, policy_version 263075 (0.0010) [2023-12-26 17:16:39,036][105692] Updated weights for policy 0, policy_version 262858 (0.0010) [2023-12-26 17:16:39,089][105692] Updated weights for policy 0, policy_version 262868 (0.0010) [2023-12-26 17:16:39,158][105692] Updated weights for policy 0, policy_version 262878 (0.0008) [2023-12-26 17:16:39,217][105692] Updated weights for policy 0, policy_version 262888 (0.0008) [2023-12-26 17:16:39,688][105620] Updated weights for policy 1, policy_version 263085 (0.0011) [2023-12-26 17:16:39,758][105620] Updated weights for policy 1, policy_version 263095 (0.0010) [2023-12-26 17:16:39,821][105620] Updated weights for policy 1, policy_version 263105 (0.0008) [2023-12-26 17:16:39,982][105692] Updated weights for policy 0, policy_version 262898 (0.0010) [2023-12-26 17:16:40,021][105585] KL-divergence is very high: 145.9761 [2023-12-26 17:16:40,029][105585] KL-divergence is very high: 160.5707 [2023-12-26 17:16:40,037][105585] KL-divergence is very high: 170.0743 [2023-12-26 17:16:40,045][105585] KL-divergence is very high: 162.1458 [2023-12-26 17:16:40,052][105692] Updated weights for policy 0, policy_version 262908 (0.0007) [2023-12-26 17:16:40,066][105585] KL-divergence is very high: 113.0461 [2023-12-26 17:16:40,118][105692] Updated weights for policy 0, policy_version 262918 (0.0009) [2023-12-26 17:16:40,578][105620] Updated weights for policy 1, policy_version 263115 (0.0009) [2023-12-26 17:16:40,640][105620] Updated weights for policy 1, policy_version 263125 (0.0010) [2023-12-26 17:16:40,689][105620] Updated weights for policy 1, policy_version 263135 (0.0010) [2023-12-26 17:16:40,794][105692] Updated weights for policy 0, policy_version 262928 (0.0006) [2023-12-26 17:16:40,853][105692] Updated weights for policy 0, policy_version 262938 (0.0005) [2023-12-26 17:16:40,906][105692] Updated weights for policy 0, policy_version 262948 (0.0005) [2023-12-26 17:16:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 134701056. Throughput: 0: 9883.9, 1: 9576.4. Samples: 134707084. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-26 17:16:41,062][104569] Avg episode reward: [(0, '1085.546'), (1, '9347.796')] [2023-12-26 17:16:41,511][105620] Updated weights for policy 1, policy_version 263145 (0.0009) [2023-12-26 17:16:41,585][105620] Updated weights for policy 1, policy_version 263155 (0.0010) [2023-12-26 17:16:41,614][105692] Updated weights for policy 0, policy_version 262958 (0.0009) [2023-12-26 17:16:41,647][105620] Updated weights for policy 1, policy_version 263165 (0.0009) [2023-12-26 17:16:41,678][105692] Updated weights for policy 0, policy_version 262968 (0.0009) [2023-12-26 17:16:41,713][105620] Updated weights for policy 1, policy_version 263175 (0.0007) [2023-12-26 17:16:41,742][105692] Updated weights for policy 0, policy_version 262978 (0.0010) [2023-12-26 17:16:42,424][105692] Updated weights for policy 0, policy_version 262988 (0.0009) [2023-12-26 17:16:42,482][105692] Updated weights for policy 0, policy_version 262998 (0.0010) [2023-12-26 17:16:42,496][105620] Updated weights for policy 1, policy_version 263185 (0.0008) [2023-12-26 17:16:42,546][105692] Updated weights for policy 0, policy_version 263008 (0.0010) [2023-12-26 17:16:42,567][105620] Updated weights for policy 1, policy_version 263195 (0.0008) [2023-12-26 17:16:42,621][105620] Updated weights for policy 1, policy_version 263205 (0.0006) [2023-12-26 17:16:43,249][105692] Updated weights for policy 0, policy_version 263018 (0.0010) [2023-12-26 17:16:43,302][105692] Updated weights for policy 0, policy_version 263028 (0.0005) [2023-12-26 17:16:43,358][105692] Updated weights for policy 0, policy_version 263038 (0.0005) [2023-12-26 17:16:43,372][105620] Updated weights for policy 1, policy_version 263215 (0.0008) [2023-12-26 17:16:43,414][105692] Updated weights for policy 0, policy_version 263048 (0.0005) [2023-12-26 17:16:43,425][105620] Updated weights for policy 1, policy_version 263225 (0.0009) [2023-12-26 17:16:43,479][105620] Updated weights for policy 1, policy_version 263235 (0.0009) [2023-12-26 17:16:43,926][105692] Updated weights for policy 0, policy_version 263058 (0.0006) [2023-12-26 17:16:43,969][105692] Updated weights for policy 0, policy_version 263068 (0.0005) [2023-12-26 17:16:44,027][105692] Updated weights for policy 0, policy_version 263078 (0.0007) [2023-12-26 17:16:44,377][105620] Updated weights for policy 1, policy_version 263245 (0.0009) [2023-12-26 17:16:44,435][105620] Updated weights for policy 1, policy_version 263255 (0.0008) [2023-12-26 17:16:44,496][105620] Updated weights for policy 1, policy_version 263265 (0.0008) [2023-12-26 17:16:44,691][105692] Updated weights for policy 0, policy_version 263088 (0.0010) [2023-12-26 17:16:44,742][105692] Updated weights for policy 0, policy_version 263098 (0.0009) [2023-12-26 17:16:44,803][105692] Updated weights for policy 0, policy_version 263108 (0.0008) [2023-12-26 17:16:45,294][105620] Updated weights for policy 1, policy_version 263275 (0.0008) [2023-12-26 17:16:45,354][105620] Updated weights for policy 1, policy_version 263285 (0.0008) [2023-12-26 17:16:45,410][105620] Updated weights for policy 1, policy_version 263295 (0.0008) [2023-12-26 17:16:45,476][105692] Updated weights for policy 0, policy_version 263118 (0.0009) [2023-12-26 17:16:45,531][105692] Updated weights for policy 0, policy_version 263128 (0.0010) [2023-12-26 17:16:45,545][105585] KL-divergence is very high: 111.6226 [2023-12-26 17:16:45,600][105692] Updated weights for policy 0, policy_version 263138 (0.0011) [2023-12-26 17:16:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 134791168. Throughput: 0: 9824.9, 1: 9529.9. Samples: 134763328. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-26 17:16:46,062][104569] Avg episode reward: [(0, '6147.691'), (1, '9167.876')] [2023-12-26 17:16:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000263144_67379200.pth... [2023-12-26 17:16:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000263304_67411968.pth... [2023-12-26 17:16:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000262184_67125248.pth [2023-12-26 17:16:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000261992_67084288.pth [2023-12-26 17:16:46,102][105620] Updated weights for policy 1, policy_version 263305 (0.0007) [2023-12-26 17:16:46,169][105620] Updated weights for policy 1, policy_version 263315 (0.0005) [2023-12-26 17:16:46,215][105692] Updated weights for policy 0, policy_version 263148 (0.0009) [2023-12-26 17:16:46,228][105620] Updated weights for policy 1, policy_version 263325 (0.0007) [2023-12-26 17:16:46,274][105692] Updated weights for policy 0, policy_version 263158 (0.0007) [2023-12-26 17:16:46,290][105620] Updated weights for policy 1, policy_version 263335 (0.0008) [2023-12-26 17:16:46,332][105692] Updated weights for policy 0, policy_version 263168 (0.0009) [2023-12-26 17:16:46,980][105620] Updated weights for policy 1, policy_version 263345 (0.0008) [2023-12-26 17:16:47,024][105692] Updated weights for policy 0, policy_version 263178 (0.0008) [2023-12-26 17:16:47,036][105620] Updated weights for policy 1, policy_version 263355 (0.0009) [2023-12-26 17:16:47,081][105692] Updated weights for policy 0, policy_version 263188 (0.0005) [2023-12-26 17:16:47,092][105620] Updated weights for policy 1, policy_version 263365 (0.0009) [2023-12-26 17:16:47,148][105692] Updated weights for policy 0, policy_version 263198 (0.0006) [2023-12-26 17:16:47,210][105692] Updated weights for policy 0, policy_version 263208 (0.0006) [2023-12-26 17:16:47,840][105692] Updated weights for policy 0, policy_version 263218 (0.0009) [2023-12-26 17:16:47,900][105620] Updated weights for policy 1, policy_version 263375 (0.0007) [2023-12-26 17:16:47,902][105692] Updated weights for policy 0, policy_version 263228 (0.0008) [2023-12-26 17:16:47,960][105620] Updated weights for policy 1, policy_version 263385 (0.0006) [2023-12-26 17:16:47,962][105692] Updated weights for policy 0, policy_version 263238 (0.0007) [2023-12-26 17:16:48,022][105620] Updated weights for policy 1, policy_version 263395 (0.0008) [2023-12-26 17:16:48,659][105692] Updated weights for policy 0, policy_version 263248 (0.0006) [2023-12-26 17:16:48,722][105620] Updated weights for policy 1, policy_version 263405 (0.0007) [2023-12-26 17:16:48,731][105692] Updated weights for policy 0, policy_version 263258 (0.0009) [2023-12-26 17:16:48,780][105620] Updated weights for policy 1, policy_version 263415 (0.0006) [2023-12-26 17:16:48,796][105692] Updated weights for policy 0, policy_version 263268 (0.0008) [2023-12-26 17:16:48,834][105620] Updated weights for policy 1, policy_version 263425 (0.0006) [2023-12-26 17:16:49,529][105692] Updated weights for policy 0, policy_version 263278 (0.0010) [2023-12-26 17:16:49,530][105620] Updated weights for policy 1, policy_version 263435 (0.0006) [2023-12-26 17:16:49,578][105692] Updated weights for policy 0, policy_version 263288 (0.0010) [2023-12-26 17:16:49,580][105620] Updated weights for policy 1, policy_version 263445 (0.0006) [2023-12-26 17:16:49,634][105692] Updated weights for policy 0, policy_version 263298 (0.0011) [2023-12-26 17:16:49,636][105620] Updated weights for policy 1, policy_version 263455 (0.0005) [2023-12-26 17:16:50,346][105620] Updated weights for policy 1, policy_version 263465 (0.0007) [2023-12-26 17:16:50,398][105692] Updated weights for policy 0, policy_version 263308 (0.0011) [2023-12-26 17:16:50,412][105620] Updated weights for policy 1, policy_version 263475 (0.0007) [2023-12-26 17:16:50,454][105692] Updated weights for policy 0, policy_version 263318 (0.0010) [2023-12-26 17:16:50,464][105620] Updated weights for policy 1, policy_version 263485 (0.0006) [2023-12-26 17:16:50,502][105692] Updated weights for policy 0, policy_version 263328 (0.0010) [2023-12-26 17:16:50,516][105620] Updated weights for policy 1, policy_version 263495 (0.0005) [2023-12-26 17:16:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 134889472. Throughput: 0: 9973.4, 1: 9447.7. Samples: 134881904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:16:51,063][104569] Avg episode reward: [(0, '7533.170'), (1, '8987.887')] [2023-12-26 17:16:51,235][105692] Updated weights for policy 0, policy_version 263338 (0.0010) [2023-12-26 17:16:51,294][105692] Updated weights for policy 0, policy_version 263348 (0.0007) [2023-12-26 17:16:51,301][105620] Updated weights for policy 1, policy_version 263505 (0.0008) [2023-12-26 17:16:51,344][105692] Updated weights for policy 0, policy_version 263358 (0.0006) [2023-12-26 17:16:51,368][105620] Updated weights for policy 1, policy_version 263515 (0.0007) [2023-12-26 17:16:51,410][105692] Updated weights for policy 0, policy_version 263368 (0.0009) [2023-12-26 17:16:51,430][105620] Updated weights for policy 1, policy_version 263525 (0.0006) [2023-12-26 17:16:52,097][105692] Updated weights for policy 0, policy_version 263378 (0.0011) [2023-12-26 17:16:52,156][105692] Updated weights for policy 0, policy_version 263388 (0.0006) [2023-12-26 17:16:52,176][105620] Updated weights for policy 1, policy_version 263535 (0.0008) [2023-12-26 17:16:52,219][105692] Updated weights for policy 0, policy_version 263398 (0.0008) [2023-12-26 17:16:52,229][105620] Updated weights for policy 1, policy_version 263545 (0.0009) [2023-12-26 17:16:52,288][105620] Updated weights for policy 1, policy_version 263555 (0.0009) [2023-12-26 17:16:52,939][105692] Updated weights for policy 0, policy_version 263408 (0.0009) [2023-12-26 17:16:52,994][105692] Updated weights for policy 0, policy_version 263418 (0.0008) [2023-12-26 17:16:53,000][105620] Updated weights for policy 1, policy_version 263565 (0.0008) [2023-12-26 17:16:53,055][105692] Updated weights for policy 0, policy_version 263428 (0.0008) [2023-12-26 17:16:53,057][105620] Updated weights for policy 1, policy_version 263575 (0.0006) [2023-12-26 17:16:53,119][105620] Updated weights for policy 1, policy_version 263585 (0.0008) [2023-12-26 17:16:53,752][105692] Updated weights for policy 0, policy_version 263438 (0.0006) [2023-12-26 17:16:53,797][105692] Updated weights for policy 0, policy_version 263448 (0.0005) [2023-12-26 17:16:53,839][105692] Updated weights for policy 0, policy_version 263458 (0.0005) [2023-12-26 17:16:53,889][105620] Updated weights for policy 1, policy_version 263595 (0.0009) [2023-12-26 17:16:53,936][105620] Updated weights for policy 1, policy_version 263605 (0.0010) [2023-12-26 17:16:53,995][105620] Updated weights for policy 1, policy_version 263615 (0.0005) [2023-12-26 17:16:54,488][105692] Updated weights for policy 0, policy_version 263468 (0.0008) [2023-12-26 17:16:54,552][105692] Updated weights for policy 0, policy_version 263478 (0.0007) [2023-12-26 17:16:54,601][105692] Updated weights for policy 0, policy_version 263488 (0.0008) [2023-12-26 17:16:54,723][105620] Updated weights for policy 1, policy_version 263625 (0.0006) [2023-12-26 17:16:54,781][105620] Updated weights for policy 1, policy_version 263635 (0.0011) [2023-12-26 17:16:54,830][105620] Updated weights for policy 1, policy_version 263646 (0.0011) [2023-12-26 17:16:54,881][105620] Updated weights for policy 1, policy_version 263656 (0.0007) [2023-12-26 17:16:55,240][105692] Updated weights for policy 0, policy_version 263498 (0.0008) [2023-12-26 17:16:55,285][105692] Updated weights for policy 0, policy_version 263508 (0.0005) [2023-12-26 17:16:55,335][105692] Updated weights for policy 0, policy_version 263518 (0.0006) [2023-12-26 17:16:55,385][105692] Updated weights for policy 0, policy_version 263528 (0.0005) [2023-12-26 17:16:55,582][105620] Updated weights for policy 1, policy_version 263666 (0.0008) [2023-12-26 17:16:55,638][105620] Updated weights for policy 1, policy_version 263676 (0.0007) [2023-12-26 17:16:55,720][105620] Updated weights for policy 1, policy_version 263686 (0.0006) [2023-12-26 17:16:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 134987776. Throughput: 0: 9943.7, 1: 9503.9. Samples: 134999552. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:16:56,062][104569] Avg episode reward: [(0, '8086.024'), (1, '8893.874')] [2023-12-26 17:16:56,112][105692] Updated weights for policy 0, policy_version 263538 (0.0005) [2023-12-26 17:16:56,174][105692] Updated weights for policy 0, policy_version 263548 (0.0005) [2023-12-26 17:16:56,225][105692] Updated weights for policy 0, policy_version 263558 (0.0005) [2023-12-26 17:16:56,408][105620] Updated weights for policy 1, policy_version 263696 (0.0005) [2023-12-26 17:16:56,455][105620] Updated weights for policy 1, policy_version 263706 (0.0005) [2023-12-26 17:16:56,504][105620] Updated weights for policy 1, policy_version 263716 (0.0005) [2023-12-26 17:16:56,771][105692] Updated weights for policy 0, policy_version 263568 (0.0005) [2023-12-26 17:16:56,817][105692] Updated weights for policy 0, policy_version 263578 (0.0005) [2023-12-26 17:16:56,870][105692] Updated weights for policy 0, policy_version 263588 (0.0005) [2023-12-26 17:16:57,161][105620] Updated weights for policy 1, policy_version 263726 (0.0006) [2023-12-26 17:16:57,214][105620] Updated weights for policy 1, policy_version 263736 (0.0007) [2023-12-26 17:16:57,275][105620] Updated weights for policy 1, policy_version 263746 (0.0009) [2023-12-26 17:16:57,531][105692] Updated weights for policy 0, policy_version 263598 (0.0005) [2023-12-26 17:16:57,584][105692] Updated weights for policy 0, policy_version 263608 (0.0005) [2023-12-26 17:16:57,631][105692] Updated weights for policy 0, policy_version 263618 (0.0005) [2023-12-26 17:16:58,071][105620] Updated weights for policy 1, policy_version 263756 (0.0008) [2023-12-26 17:16:58,133][105620] Updated weights for policy 1, policy_version 263766 (0.0008) [2023-12-26 17:16:58,155][105692] Updated weights for policy 0, policy_version 263628 (0.0007) [2023-12-26 17:16:58,189][105620] Updated weights for policy 1, policy_version 263776 (0.0007) [2023-12-26 17:16:58,215][105692] Updated weights for policy 0, policy_version 263638 (0.0007) [2023-12-26 17:16:58,272][105692] Updated weights for policy 0, policy_version 263648 (0.0007) [2023-12-26 17:16:58,982][105620] Updated weights for policy 1, policy_version 263786 (0.0008) [2023-12-26 17:16:59,048][105620] Updated weights for policy 1, policy_version 263796 (0.0009) [2023-12-26 17:16:59,105][105692] Updated weights for policy 0, policy_version 263658 (0.0008) [2023-12-26 17:16:59,112][105620] Updated weights for policy 1, policy_version 263806 (0.0006) [2023-12-26 17:16:59,160][105692] Updated weights for policy 0, policy_version 263668 (0.0007) [2023-12-26 17:16:59,175][105620] Updated weights for policy 1, policy_version 263816 (0.0007) [2023-12-26 17:16:59,217][105692] Updated weights for policy 0, policy_version 263678 (0.0007) [2023-12-26 17:16:59,277][105692] Updated weights for policy 0, policy_version 263688 (0.0008) [2023-12-26 17:16:59,833][105620] Updated weights for policy 1, policy_version 263826 (0.0006) [2023-12-26 17:16:59,895][105620] Updated weights for policy 1, policy_version 263836 (0.0007) [2023-12-26 17:16:59,960][105620] Updated weights for policy 1, policy_version 263846 (0.0006) [2023-12-26 17:17:00,012][105692] Updated weights for policy 0, policy_version 263698 (0.0009) [2023-12-26 17:17:00,069][105692] Updated weights for policy 0, policy_version 263708 (0.0009) [2023-12-26 17:17:00,128][105692] Updated weights for policy 0, policy_version 263718 (0.0009) [2023-12-26 17:17:00,523][105620] Updated weights for policy 1, policy_version 263856 (0.0005) [2023-12-26 17:17:00,582][105620] Updated weights for policy 1, policy_version 263866 (0.0005) [2023-12-26 17:17:00,637][105620] Updated weights for policy 1, policy_version 263876 (0.0005) [2023-12-26 17:17:00,981][105692] Updated weights for policy 0, policy_version 263728 (0.0009) [2023-12-26 17:17:01,034][105692] Updated weights for policy 0, policy_version 263738 (0.0009) [2023-12-26 17:17:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 135086080. Throughput: 0: 10017.1, 1: 9532.6. Samples: 135060548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:17:01,062][104569] Avg episode reward: [(0, '8167.562'), (1, '8892.459')] [2023-12-26 17:17:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000263880_67559424.pth... [2023-12-26 17:17:01,069][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000262760_67272704.pth [2023-12-26 17:17:01,093][105692] Updated weights for policy 0, policy_version 263748 (0.0009) [2023-12-26 17:17:01,117][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000263752_67534848.pth... [2023-12-26 17:17:01,122][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000262568_67231744.pth [2023-12-26 17:17:01,260][105620] Updated weights for policy 1, policy_version 263886 (0.0007) [2023-12-26 17:17:01,322][105620] Updated weights for policy 1, policy_version 263896 (0.0009) [2023-12-26 17:17:01,393][105620] Updated weights for policy 1, policy_version 263906 (0.0009) [2023-12-26 17:17:01,823][105692] Updated weights for policy 0, policy_version 263758 (0.0008) [2023-12-26 17:17:01,875][105692] Updated weights for policy 0, policy_version 263768 (0.0009) [2023-12-26 17:17:01,921][105692] Updated weights for policy 0, policy_version 263778 (0.0007) [2023-12-26 17:17:02,199][105620] Updated weights for policy 1, policy_version 263916 (0.0008) [2023-12-26 17:17:02,260][105620] Updated weights for policy 1, policy_version 263926 (0.0007) [2023-12-26 17:17:02,317][105620] Updated weights for policy 1, policy_version 263936 (0.0008) [2023-12-26 17:17:02,602][105692] Updated weights for policy 0, policy_version 263788 (0.0007) [2023-12-26 17:17:02,666][105692] Updated weights for policy 0, policy_version 263798 (0.0010) [2023-12-26 17:17:02,725][105692] Updated weights for policy 0, policy_version 263808 (0.0010) [2023-12-26 17:17:02,992][105620] Updated weights for policy 1, policy_version 263946 (0.0009) [2023-12-26 17:17:03,046][105620] Updated weights for policy 1, policy_version 263956 (0.0005) [2023-12-26 17:17:03,091][105620] Updated weights for policy 1, policy_version 263966 (0.0005) [2023-12-26 17:17:03,152][105620] Updated weights for policy 1, policy_version 263976 (0.0005) [2023-12-26 17:17:03,503][105692] Updated weights for policy 0, policy_version 263819 (0.0010) [2023-12-26 17:17:03,557][105692] Updated weights for policy 0, policy_version 263829 (0.0010) [2023-12-26 17:17:03,616][105692] Updated weights for policy 0, policy_version 263839 (0.0010) [2023-12-26 17:17:03,695][105620] Updated weights for policy 1, policy_version 263986 (0.0007) [2023-12-26 17:17:03,758][105620] Updated weights for policy 1, policy_version 263996 (0.0007) [2023-12-26 17:17:03,815][105620] Updated weights for policy 1, policy_version 264006 (0.0009) [2023-12-26 17:17:04,389][105692] Updated weights for policy 0, policy_version 263849 (0.0009) [2023-12-26 17:17:04,439][105692] Updated weights for policy 0, policy_version 263859 (0.0009) [2023-12-26 17:17:04,494][105692] Updated weights for policy 0, policy_version 263869 (0.0009) [2023-12-26 17:17:04,521][105620] Updated weights for policy 1, policy_version 264016 (0.0008) [2023-12-26 17:17:04,545][105692] Updated weights for policy 0, policy_version 263879 (0.0007) [2023-12-26 17:17:04,569][105620] Updated weights for policy 1, policy_version 264026 (0.0006) [2023-12-26 17:17:04,635][105620] Updated weights for policy 1, policy_version 264036 (0.0008) [2023-12-26 17:17:05,250][105620] Updated weights for policy 1, policy_version 264046 (0.0005) [2023-12-26 17:17:05,295][105620] Updated weights for policy 1, policy_version 264056 (0.0006) [2023-12-26 17:17:05,313][105692] Updated weights for policy 0, policy_version 263889 (0.0008) [2023-12-26 17:17:05,349][105620] Updated weights for policy 1, policy_version 264066 (0.0006) [2023-12-26 17:17:05,363][105692] Updated weights for policy 0, policy_version 263899 (0.0008) [2023-12-26 17:17:05,417][105692] Updated weights for policy 0, policy_version 263909 (0.0008) [2023-12-26 17:17:05,957][105620] Updated weights for policy 1, policy_version 264076 (0.0007) [2023-12-26 17:17:06,003][105620] Updated weights for policy 1, policy_version 264086 (0.0008) [2023-12-26 17:17:06,052][105620] Updated weights for policy 1, policy_version 264096 (0.0008) [2023-12-26 17:17:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 135184384. Throughput: 0: 9901.8, 1: 9738.2. Samples: 135178184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:17:06,062][104569] Avg episode reward: [(0, '8625.919'), (1, '9072.394')] [2023-12-26 17:17:06,094][105692] Updated weights for policy 0, policy_version 263919 (0.0010) [2023-12-26 17:17:06,158][105692] Updated weights for policy 0, policy_version 263929 (0.0011) [2023-12-26 17:17:06,218][105692] Updated weights for policy 0, policy_version 263939 (0.0010) [2023-12-26 17:17:06,852][105620] Updated weights for policy 1, policy_version 264106 (0.0007) [2023-12-26 17:17:06,906][105620] Updated weights for policy 1, policy_version 264116 (0.0008) [2023-12-26 17:17:06,956][105620] Updated weights for policy 1, policy_version 264126 (0.0008) [2023-12-26 17:17:06,994][105692] Updated weights for policy 0, policy_version 263949 (0.0010) [2023-12-26 17:17:07,009][105620] Updated weights for policy 1, policy_version 264136 (0.0007) [2023-12-26 17:17:07,051][105692] Updated weights for policy 0, policy_version 263959 (0.0010) [2023-12-26 17:17:07,103][105692] Updated weights for policy 0, policy_version 263969 (0.0010) [2023-12-26 17:17:07,754][105692] Updated weights for policy 0, policy_version 263979 (0.0010) [2023-12-26 17:17:07,810][105692] Updated weights for policy 0, policy_version 263989 (0.0009) [2023-12-26 17:17:07,836][105620] Updated weights for policy 1, policy_version 264146 (0.0007) [2023-12-26 17:17:07,868][105692] Updated weights for policy 0, policy_version 263999 (0.0006) [2023-12-26 17:17:07,898][105620] Updated weights for policy 1, policy_version 264156 (0.0007) [2023-12-26 17:17:07,971][105620] Updated weights for policy 1, policy_version 264166 (0.0010) [2023-12-26 17:17:08,514][105692] Updated weights for policy 0, policy_version 264009 (0.0007) [2023-12-26 17:17:08,567][105692] Updated weights for policy 0, policy_version 264019 (0.0006) [2023-12-26 17:17:08,624][105692] Updated weights for policy 0, policy_version 264029 (0.0007) [2023-12-26 17:17:08,679][105620] Updated weights for policy 1, policy_version 264176 (0.0011) [2023-12-26 17:17:08,681][105692] Updated weights for policy 0, policy_version 264039 (0.0006) [2023-12-26 17:17:08,742][105620] Updated weights for policy 1, policy_version 264186 (0.0010) [2023-12-26 17:17:08,805][105620] Updated weights for policy 1, policy_version 264196 (0.0011) [2023-12-26 17:17:09,336][105692] Updated weights for policy 0, policy_version 264049 (0.0008) [2023-12-26 17:17:09,402][105692] Updated weights for policy 0, policy_version 264059 (0.0009) [2023-12-26 17:17:09,485][105692] Updated weights for policy 0, policy_version 264069 (0.0007) [2023-12-26 17:17:09,502][105620] Updated weights for policy 1, policy_version 264206 (0.0009) [2023-12-26 17:17:09,554][105620] Updated weights for policy 1, policy_version 264216 (0.0008) [2023-12-26 17:17:09,602][105620] Updated weights for policy 1, policy_version 264226 (0.0008) [2023-12-26 17:17:10,266][105692] Updated weights for policy 0, policy_version 264079 (0.0009) [2023-12-26 17:17:10,317][105692] Updated weights for policy 0, policy_version 264089 (0.0007) [2023-12-26 17:17:10,382][105692] Updated weights for policy 0, policy_version 264099 (0.0007) [2023-12-26 17:17:10,392][105620] Updated weights for policy 1, policy_version 264236 (0.0008) [2023-12-26 17:17:10,444][105620] Updated weights for policy 1, policy_version 264246 (0.0010) [2023-12-26 17:17:10,493][105620] Updated weights for policy 1, policy_version 264256 (0.0010) [2023-12-26 17:17:11,025][105692] Updated weights for policy 0, policy_version 264109 (0.0006) [2023-12-26 17:17:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 135282688. Throughput: 0: 9964.7, 1: 9726.7. Samples: 135294600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:17:11,062][104569] Avg episode reward: [(0, '8992.557'), (1, '9257.994')] [2023-12-26 17:17:11,087][105692] Updated weights for policy 0, policy_version 264119 (0.0010) [2023-12-26 17:17:11,162][105692] Updated weights for policy 0, policy_version 264129 (0.0010) [2023-12-26 17:17:11,272][105620] Updated weights for policy 1, policy_version 264266 (0.0010) [2023-12-26 17:17:11,337][105620] Updated weights for policy 1, policy_version 264276 (0.0009) [2023-12-26 17:17:11,405][105620] Updated weights for policy 1, policy_version 264286 (0.0010) [2023-12-26 17:17:11,468][105620] Updated weights for policy 1, policy_version 264296 (0.0006) [2023-12-26 17:17:11,880][105692] Updated weights for policy 0, policy_version 264139 (0.0009) [2023-12-26 17:17:11,951][105692] Updated weights for policy 0, policy_version 264149 (0.0007) [2023-12-26 17:17:12,011][105692] Updated weights for policy 0, policy_version 264159 (0.0010) [2023-12-26 17:17:12,206][105620] Updated weights for policy 1, policy_version 264306 (0.0008) [2023-12-26 17:17:12,254][105620] Updated weights for policy 1, policy_version 264316 (0.0008) [2023-12-26 17:17:12,325][105620] Updated weights for policy 1, policy_version 264326 (0.0007) [2023-12-26 17:17:12,735][105692] Updated weights for policy 0, policy_version 264169 (0.0008) [2023-12-26 17:17:12,785][105692] Updated weights for policy 0, policy_version 264179 (0.0010) [2023-12-26 17:17:12,830][105692] Updated weights for policy 0, policy_version 264189 (0.0011) [2023-12-26 17:17:12,882][105692] Updated weights for policy 0, policy_version 264199 (0.0011) [2023-12-26 17:17:13,083][105620] Updated weights for policy 1, policy_version 264336 (0.0010) [2023-12-26 17:17:13,152][105620] Updated weights for policy 1, policy_version 264346 (0.0010) [2023-12-26 17:17:13,214][105620] Updated weights for policy 1, policy_version 264356 (0.0011) [2023-12-26 17:17:13,602][105692] Updated weights for policy 0, policy_version 264209 (0.0006) [2023-12-26 17:17:13,672][105692] Updated weights for policy 0, policy_version 264219 (0.0011) [2023-12-26 17:17:13,737][105692] Updated weights for policy 0, policy_version 264229 (0.0009) [2023-12-26 17:17:13,786][105620] Updated weights for policy 1, policy_version 264366 (0.0007) [2023-12-26 17:17:13,846][105620] Updated weights for policy 1, policy_version 264376 (0.0005) [2023-12-26 17:17:13,898][105620] Updated weights for policy 1, policy_version 264386 (0.0007) [2023-12-26 17:17:14,424][105692] Updated weights for policy 0, policy_version 264239 (0.0010) [2023-12-26 17:17:14,482][105692] Updated weights for policy 0, policy_version 264249 (0.0010) [2023-12-26 17:17:14,548][105692] Updated weights for policy 0, policy_version 264259 (0.0010) [2023-12-26 17:17:14,601][105620] Updated weights for policy 1, policy_version 264396 (0.0007) [2023-12-26 17:17:14,660][105620] Updated weights for policy 1, policy_version 264406 (0.0009) [2023-12-26 17:17:14,723][105620] Updated weights for policy 1, policy_version 264416 (0.0009) [2023-12-26 17:17:15,215][105692] Updated weights for policy 0, policy_version 264269 (0.0008) [2023-12-26 17:17:15,276][105692] Updated weights for policy 0, policy_version 264279 (0.0007) [2023-12-26 17:17:15,339][105692] Updated weights for policy 0, policy_version 264289 (0.0011) [2023-12-26 17:17:15,541][105620] Updated weights for policy 1, policy_version 264426 (0.0009) [2023-12-26 17:17:15,597][105620] Updated weights for policy 1, policy_version 264436 (0.0007) [2023-12-26 17:17:15,659][105620] Updated weights for policy 1, policy_version 264446 (0.0008) [2023-12-26 17:17:15,723][105620] Updated weights for policy 1, policy_version 264456 (0.0008) [2023-12-26 17:17:16,006][105692] Updated weights for policy 0, policy_version 264299 (0.0009) [2023-12-26 17:17:16,053][105692] Updated weights for policy 0, policy_version 264309 (0.0006) [2023-12-26 17:17:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 135380992. Throughput: 0: 9956.4, 1: 9673.3. Samples: 135354300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:17:16,062][104569] Avg episode reward: [(0, '9173.985'), (1, '9074.071')] [2023-12-26 17:17:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000264456_67706880.pth... [2023-12-26 17:17:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000263304_67411968.pth [2023-12-26 17:17:16,115][105692] Updated weights for policy 0, policy_version 264319 (0.0009) [2023-12-26 17:17:16,168][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000264328_67682304.pth... [2023-12-26 17:17:16,173][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000263144_67379200.pth [2023-12-26 17:17:16,560][105620] Updated weights for policy 1, policy_version 264467 (0.0009) [2023-12-26 17:17:16,616][105620] Updated weights for policy 1, policy_version 264477 (0.0010) [2023-12-26 17:17:16,647][105692] Updated weights for policy 0, policy_version 264329 (0.0008) [2023-12-26 17:17:16,666][105620] Updated weights for policy 1, policy_version 264488 (0.0010) [2023-12-26 17:17:16,693][105692] Updated weights for policy 0, policy_version 264339 (0.0005) [2023-12-26 17:17:16,737][105692] Updated weights for policy 0, policy_version 264349 (0.0005) [2023-12-26 17:17:16,785][105692] Updated weights for policy 0, policy_version 264359 (0.0007) [2023-12-26 17:17:17,447][105692] Updated weights for policy 0, policy_version 264369 (0.0006) [2023-12-26 17:17:17,497][105692] Updated weights for policy 0, policy_version 264379 (0.0005) [2023-12-26 17:17:17,505][105620] Updated weights for policy 1, policy_version 264498 (0.0009) [2023-12-26 17:17:17,540][105692] Updated weights for policy 0, policy_version 264389 (0.0005) [2023-12-26 17:17:17,560][105620] Updated weights for policy 1, policy_version 264508 (0.0009) [2023-12-26 17:17:17,619][105620] Updated weights for policy 1, policy_version 264519 (0.0010) [2023-12-26 17:17:18,178][105692] Updated weights for policy 0, policy_version 264399 (0.0009) [2023-12-26 17:17:18,227][105585] KL-divergence is very high: 139.8200 [2023-12-26 17:17:18,236][105692] Updated weights for policy 0, policy_version 264409 (0.0010) [2023-12-26 17:17:18,271][105585] KL-divergence is very high: 171.9462 [2023-12-26 17:17:18,284][105620] Updated weights for policy 1, policy_version 264529 (0.0006) [2023-12-26 17:17:18,300][105692] Updated weights for policy 0, policy_version 264419 (0.0011) [2023-12-26 17:17:18,328][105585] KL-divergence is very high: 110.1795 [2023-12-26 17:17:18,342][105620] Updated weights for policy 1, policy_version 264539 (0.0007) [2023-12-26 17:17:18,403][105620] Updated weights for policy 1, policy_version 264549 (0.0009) [2023-12-26 17:17:19,037][105692] Updated weights for policy 0, policy_version 264429 (0.0010) [2023-12-26 17:17:19,068][105620] Updated weights for policy 1, policy_version 264559 (0.0008) [2023-12-26 17:17:19,082][105692] Updated weights for policy 0, policy_version 264439 (0.0010) [2023-12-26 17:17:19,120][105620] Updated weights for policy 1, policy_version 264569 (0.0010) [2023-12-26 17:17:19,126][105692] Updated weights for policy 0, policy_version 264449 (0.0010) [2023-12-26 17:17:19,174][105620] Updated weights for policy 1, policy_version 264579 (0.0008) [2023-12-26 17:17:19,867][105620] Updated weights for policy 1, policy_version 264589 (0.0008) [2023-12-26 17:17:19,928][105692] Updated weights for policy 0, policy_version 264459 (0.0009) [2023-12-26 17:17:19,931][105620] Updated weights for policy 1, policy_version 264599 (0.0008) [2023-12-26 17:17:19,994][105692] Updated weights for policy 0, policy_version 264469 (0.0006) [2023-12-26 17:17:19,999][105620] Updated weights for policy 1, policy_version 264609 (0.0010) [2023-12-26 17:17:20,058][105692] Updated weights for policy 0, policy_version 264479 (0.0007) [2023-12-26 17:17:20,662][105620] Updated weights for policy 1, policy_version 264619 (0.0009) [2023-12-26 17:17:20,719][105620] Updated weights for policy 1, policy_version 264629 (0.0006) [2023-12-26 17:17:20,781][105620] Updated weights for policy 1, policy_version 264639 (0.0006) [2023-12-26 17:17:20,816][105692] Updated weights for policy 0, policy_version 264489 (0.0009) [2023-12-26 17:17:20,880][105692] Updated weights for policy 0, policy_version 264499 (0.0008) [2023-12-26 17:17:20,946][105692] Updated weights for policy 0, policy_version 264509 (0.0008) [2023-12-26 17:17:21,012][105692] Updated weights for policy 0, policy_version 264519 (0.0008) [2023-12-26 17:17:21,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 135487488. Throughput: 0: 9953.1, 1: 9612.8. Samples: 135472616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:17:21,062][104569] Avg episode reward: [(0, '8898.179'), (1, '9072.703')] [2023-12-26 17:17:21,504][105620] Updated weights for policy 1, policy_version 264649 (0.0008) [2023-12-26 17:17:21,570][105620] Updated weights for policy 1, policy_version 264659 (0.0011) [2023-12-26 17:17:21,635][105620] Updated weights for policy 1, policy_version 264669 (0.0010) [2023-12-26 17:17:21,701][105692] Updated weights for policy 0, policy_version 264529 (0.0008) [2023-12-26 17:17:21,702][105620] Updated weights for policy 1, policy_version 264679 (0.0007) [2023-12-26 17:17:21,765][105692] Updated weights for policy 0, policy_version 264539 (0.0009) [2023-12-26 17:17:21,819][105692] Updated weights for policy 0, policy_version 264549 (0.0008) [2023-12-26 17:17:22,328][105620] Updated weights for policy 1, policy_version 264689 (0.0009) [2023-12-26 17:17:22,401][105620] Updated weights for policy 1, policy_version 264699 (0.0008) [2023-12-26 17:17:22,463][105620] Updated weights for policy 1, policy_version 264709 (0.0008) [2023-12-26 17:17:22,640][105692] Updated weights for policy 0, policy_version 264559 (0.0009) [2023-12-26 17:17:22,706][105692] Updated weights for policy 0, policy_version 264569 (0.0009) [2023-12-26 17:17:22,769][105692] Updated weights for policy 0, policy_version 264579 (0.0009) [2023-12-26 17:17:23,229][105620] Updated weights for policy 1, policy_version 264719 (0.0009) [2023-12-26 17:17:23,294][105620] Updated weights for policy 1, policy_version 264729 (0.0009) [2023-12-26 17:17:23,349][105620] Updated weights for policy 1, policy_version 264739 (0.0007) [2023-12-26 17:17:23,519][105692] Updated weights for policy 0, policy_version 264589 (0.0009) [2023-12-26 17:17:23,572][105692] Updated weights for policy 0, policy_version 264599 (0.0009) [2023-12-26 17:17:23,634][105692] Updated weights for policy 0, policy_version 264609 (0.0009) [2023-12-26 17:17:24,097][105620] Updated weights for policy 1, policy_version 264749 (0.0009) [2023-12-26 17:17:24,155][105620] Updated weights for policy 1, policy_version 264759 (0.0010) [2023-12-26 17:17:24,217][105620] Updated weights for policy 1, policy_version 264769 (0.0009) [2023-12-26 17:17:24,292][105692] Updated weights for policy 0, policy_version 264619 (0.0009) [2023-12-26 17:17:24,355][105692] Updated weights for policy 0, policy_version 264629 (0.0009) [2023-12-26 17:17:24,421][105692] Updated weights for policy 0, policy_version 264639 (0.0008) [2023-12-26 17:17:24,969][105620] Updated weights for policy 1, policy_version 264779 (0.0009) [2023-12-26 17:17:25,015][105620] Updated weights for policy 1, policy_version 264789 (0.0008) [2023-12-26 17:17:25,062][105620] Updated weights for policy 1, policy_version 264799 (0.0008) [2023-12-26 17:17:25,187][105692] Updated weights for policy 0, policy_version 264649 (0.0009) [2023-12-26 17:17:25,232][105692] Updated weights for policy 0, policy_version 264659 (0.0005) [2023-12-26 17:17:25,284][105692] Updated weights for policy 0, policy_version 264669 (0.0005) [2023-12-26 17:17:25,312][105585] KL-divergence is very high: 148.2770 [2023-12-26 17:17:25,323][105585] KL-divergence is very high: 157.8339 [2023-12-26 17:17:25,339][105692] Updated weights for policy 0, policy_version 264679 (0.0005) [2023-12-26 17:17:25,831][105620] Updated weights for policy 1, policy_version 264809 (0.0009) [2023-12-26 17:17:25,894][105620] Updated weights for policy 1, policy_version 264819 (0.0007) [2023-12-26 17:17:25,938][105585] KL-divergence is very high: 112.1456 [2023-12-26 17:17:25,948][105620] Updated weights for policy 1, policy_version 264829 (0.0010) [2023-12-26 17:17:25,966][105692] Updated weights for policy 0, policy_version 264689 (0.0007) [2023-12-26 17:17:25,996][105620] Updated weights for policy 1, policy_version 264839 (0.0010) [2023-12-26 17:17:26,012][105692] Updated weights for policy 0, policy_version 264699 (0.0007) [2023-12-26 17:17:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 135577600. Throughput: 0: 9972.4, 1: 9588.0. Samples: 135587304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:17:26,062][104569] Avg episode reward: [(0, '8545.100'), (1, '9078.472')] [2023-12-26 17:17:26,069][105692] Updated weights for policy 0, policy_version 264709 (0.0008) [2023-12-26 17:17:26,735][105692] Updated weights for policy 0, policy_version 264719 (0.0006) [2023-12-26 17:17:26,740][105620] Updated weights for policy 1, policy_version 264849 (0.0010) [2023-12-26 17:17:26,787][105692] Updated weights for policy 0, policy_version 264729 (0.0006) [2023-12-26 17:17:26,798][105620] Updated weights for policy 1, policy_version 264859 (0.0010) [2023-12-26 17:17:26,837][105692] Updated weights for policy 0, policy_version 264739 (0.0006) [2023-12-26 17:17:26,853][105620] Updated weights for policy 1, policy_version 264869 (0.0010) [2023-12-26 17:17:27,500][105620] Updated weights for policy 1, policy_version 264879 (0.0007) [2023-12-26 17:17:27,562][105620] Updated weights for policy 1, policy_version 264889 (0.0007) [2023-12-26 17:17:27,601][105692] Updated weights for policy 0, policy_version 264749 (0.0008) [2023-12-26 17:17:27,608][105620] Updated weights for policy 1, policy_version 264899 (0.0005) [2023-12-26 17:17:27,655][105692] Updated weights for policy 0, policy_version 264759 (0.0008) [2023-12-26 17:17:27,712][105692] Updated weights for policy 0, policy_version 264770 (0.0010) [2023-12-26 17:17:28,230][105620] Updated weights for policy 1, policy_version 264909 (0.0007) [2023-12-26 17:17:28,282][105620] Updated weights for policy 1, policy_version 264919 (0.0008) [2023-12-26 17:17:28,333][105620] Updated weights for policy 1, policy_version 264929 (0.0009) [2023-12-26 17:17:28,482][105692] Updated weights for policy 0, policy_version 264780 (0.0009) [2023-12-26 17:17:28,536][105692] Updated weights for policy 0, policy_version 264790 (0.0009) [2023-12-26 17:17:28,586][105692] Updated weights for policy 0, policy_version 264801 (0.0010) [2023-12-26 17:17:29,108][105620] Updated weights for policy 1, policy_version 264939 (0.0009) [2023-12-26 17:17:29,172][105620] Updated weights for policy 1, policy_version 264949 (0.0009) [2023-12-26 17:17:29,231][105620] Updated weights for policy 1, policy_version 264959 (0.0008) [2023-12-26 17:17:29,354][105692] Updated weights for policy 0, policy_version 264811 (0.0009) [2023-12-26 17:17:29,398][105585] KL-divergence is very high: 106.4564 [2023-12-26 17:17:29,405][105585] KL-divergence is very high: 112.6060 [2023-12-26 17:17:29,411][105585] KL-divergence is very high: 110.1536 [2023-12-26 17:17:29,415][105692] Updated weights for policy 0, policy_version 264821 (0.0008) [2023-12-26 17:17:29,416][105585] KL-divergence is very high: 100.8316 [2023-12-26 17:17:29,469][105692] Updated weights for policy 0, policy_version 264831 (0.0009) [2023-12-26 17:17:29,994][105620] Updated weights for policy 1, policy_version 264969 (0.0008) [2023-12-26 17:17:30,049][105620] Updated weights for policy 1, policy_version 264979 (0.0009) [2023-12-26 17:17:30,100][105620] Updated weights for policy 1, policy_version 264989 (0.0009) [2023-12-26 17:17:30,164][105620] Updated weights for policy 1, policy_version 264999 (0.0009) [2023-12-26 17:17:30,221][105692] Updated weights for policy 0, policy_version 264841 (0.0009) [2023-12-26 17:17:30,252][105585] KL-divergence is very high: 162.1382 [2023-12-26 17:17:30,278][105692] Updated weights for policy 0, policy_version 264851 (0.0009) [2023-12-26 17:17:30,290][105585] KL-divergence is very high: 157.2986 [2023-12-26 17:17:30,325][105692] Updated weights for policy 0, policy_version 264861 (0.0009) [2023-12-26 17:17:30,330][105585] KL-divergence is very high: 134.5300 [2023-12-26 17:17:30,379][105692] Updated weights for policy 0, policy_version 264871 (0.0008) [2023-12-26 17:17:30,867][105620] Updated weights for policy 1, policy_version 265009 (0.0005) [2023-12-26 17:17:30,912][105620] Updated weights for policy 1, policy_version 265019 (0.0005) [2023-12-26 17:17:30,960][105620] Updated weights for policy 1, policy_version 265029 (0.0005) [2023-12-26 17:17:31,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 135675904. Throughput: 0: 9939.7, 1: 9684.4. Samples: 135646416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:17:31,063][104569] Avg episode reward: [(0, '5195.270'), (1, '8899.569')] [2023-12-26 17:17:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000264872_67821568.pth... [2023-12-26 17:17:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000265032_67854336.pth... [2023-12-26 17:17:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000263752_67534848.pth [2023-12-26 17:17:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000263880_67559424.pth [2023-12-26 17:17:31,207][105585] KL-divergence is very high: 130.6400 [2023-12-26 17:17:31,214][105585] KL-divergence is very high: 127.1754 [2023-12-26 17:17:31,219][105585] KL-divergence is very high: 119.6412 [2023-12-26 17:17:31,221][105692] Updated weights for policy 0, policy_version 264881 (0.0009) [2023-12-26 17:17:31,226][105585] KL-divergence is very high: 113.2161 [2023-12-26 17:17:31,281][105692] Updated weights for policy 0, policy_version 264891 (0.0009) [2023-12-26 17:17:31,345][105692] Updated weights for policy 0, policy_version 264901 (0.0009) [2023-12-26 17:17:31,580][105620] Updated weights for policy 1, policy_version 265039 (0.0005) [2023-12-26 17:17:31,639][105620] Updated weights for policy 1, policy_version 265049 (0.0007) [2023-12-26 17:17:31,697][105620] Updated weights for policy 1, policy_version 265059 (0.0008) [2023-12-26 17:17:32,123][105585] KL-divergence is very high: 146.6797 [2023-12-26 17:17:32,139][105692] Updated weights for policy 0, policy_version 264911 (0.0008) [2023-12-26 17:17:32,165][105585] KL-divergence is very high: 111.8475 [2023-12-26 17:17:32,186][105585] KL-divergence is very high: 170.1406 [2023-12-26 17:17:32,192][105692] Updated weights for policy 0, policy_version 264921 (0.0009) [2023-12-26 17:17:32,196][105585] KL-divergence is very high: 250.9723 [2023-12-26 17:17:32,208][105585] KL-divergence is very high: 190.2383 [2023-12-26 17:17:32,232][105585] KL-divergence is very high: 145.6641 [2023-12-26 17:17:32,249][105585] KL-divergence is very high: 169.4675 [2023-12-26 17:17:32,254][105692] Updated weights for policy 0, policy_version 264931 (0.0008) [2023-12-26 17:17:32,262][105585] KL-divergence is very high: 131.8245 [2023-12-26 17:17:32,364][105620] Updated weights for policy 1, policy_version 265069 (0.0008) [2023-12-26 17:17:32,430][105620] Updated weights for policy 1, policy_version 265079 (0.0008) [2023-12-26 17:17:32,490][105620] Updated weights for policy 1, policy_version 265089 (0.0008) [2023-12-26 17:17:32,855][105692] Updated weights for policy 0, policy_version 264941 (0.0006) [2023-12-26 17:17:32,913][105692] Updated weights for policy 0, policy_version 264951 (0.0005) [2023-12-26 17:17:32,976][105692] Updated weights for policy 0, policy_version 264961 (0.0006) [2023-12-26 17:17:33,235][105620] Updated weights for policy 1, policy_version 265099 (0.0008) [2023-12-26 17:17:33,284][105620] Updated weights for policy 1, policy_version 265109 (0.0008) [2023-12-26 17:17:33,331][105620] Updated weights for policy 1, policy_version 265119 (0.0009) [2023-12-26 17:17:33,638][105692] Updated weights for policy 0, policy_version 264971 (0.0008) [2023-12-26 17:17:33,683][105692] Updated weights for policy 0, policy_version 264981 (0.0008) [2023-12-26 17:17:33,727][105585] KL-divergence is very high: 103.4403 [2023-12-26 17:17:33,728][105692] Updated weights for policy 0, policy_version 264991 (0.0010) [2023-12-26 17:17:34,028][105620] Updated weights for policy 1, policy_version 265129 (0.0008) [2023-12-26 17:17:34,087][105620] Updated weights for policy 1, policy_version 265139 (0.0005) [2023-12-26 17:17:34,158][105620] Updated weights for policy 1, policy_version 265149 (0.0008) [2023-12-26 17:17:34,222][105620] Updated weights for policy 1, policy_version 265159 (0.0010) [2023-12-26 17:17:34,441][105692] Updated weights for policy 0, policy_version 265001 (0.0009) [2023-12-26 17:17:34,507][105692] Updated weights for policy 0, policy_version 265011 (0.0005) [2023-12-26 17:17:34,576][105692] Updated weights for policy 0, policy_version 265021 (0.0005) [2023-12-26 17:17:34,641][105692] Updated weights for policy 0, policy_version 265031 (0.0008) [2023-12-26 17:17:34,930][105620] Updated weights for policy 1, policy_version 265169 (0.0009) [2023-12-26 17:17:35,002][105620] Updated weights for policy 1, policy_version 265179 (0.0009) [2023-12-26 17:17:35,070][105620] Updated weights for policy 1, policy_version 265189 (0.0009) [2023-12-26 17:17:35,226][105692] Updated weights for policy 0, policy_version 265041 (0.0008) [2023-12-26 17:17:35,276][105692] Updated weights for policy 0, policy_version 265051 (0.0005) [2023-12-26 17:17:35,328][105692] Updated weights for policy 0, policy_version 265061 (0.0005) [2023-12-26 17:17:35,906][105692] Updated weights for policy 0, policy_version 265071 (0.0007) [2023-12-26 17:17:35,924][105620] Updated weights for policy 1, policy_version 265199 (0.0008) [2023-12-26 17:17:35,961][105692] Updated weights for policy 0, policy_version 265081 (0.0010) [2023-12-26 17:17:35,983][105620] Updated weights for policy 1, policy_version 265209 (0.0006) [2023-12-26 17:17:36,017][105692] Updated weights for policy 0, policy_version 265091 (0.0010) [2023-12-26 17:17:36,031][105620] Updated weights for policy 1, policy_version 265219 (0.0005) [2023-12-26 17:17:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 135782400. Throughput: 0: 9836.9, 1: 9742.5. Samples: 135762976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:17:36,063][104569] Avg episode reward: [(0, '815.907'), (1, '9079.798')] [2023-12-26 17:17:36,728][105620] Updated weights for policy 1, policy_version 265229 (0.0008) [2023-12-26 17:17:36,775][105692] Updated weights for policy 0, policy_version 265101 (0.0010) [2023-12-26 17:17:36,786][105620] Updated weights for policy 1, policy_version 265239 (0.0008) [2023-12-26 17:17:36,830][105692] Updated weights for policy 0, policy_version 265111 (0.0010) [2023-12-26 17:17:36,840][105620] Updated weights for policy 1, policy_version 265249 (0.0008) [2023-12-26 17:17:36,883][105692] Updated weights for policy 0, policy_version 265121 (0.0010) [2023-12-26 17:17:37,466][105692] Updated weights for policy 0, policy_version 265131 (0.0007) [2023-12-26 17:17:37,522][105692] Updated weights for policy 0, policy_version 265141 (0.0011) [2023-12-26 17:17:37,576][105692] Updated weights for policy 0, policy_version 265151 (0.0010) [2023-12-26 17:17:37,653][105620] Updated weights for policy 1, policy_version 265259 (0.0007) [2023-12-26 17:17:37,709][105620] Updated weights for policy 1, policy_version 265269 (0.0008) [2023-12-26 17:17:37,776][105620] Updated weights for policy 1, policy_version 265279 (0.0008) [2023-12-26 17:17:38,326][105692] Updated weights for policy 0, policy_version 265161 (0.0010) [2023-12-26 17:17:38,388][105692] Updated weights for policy 0, policy_version 265171 (0.0012) [2023-12-26 17:17:38,448][105692] Updated weights for policy 0, policy_version 265181 (0.0010) [2023-12-26 17:17:38,515][105692] Updated weights for policy 0, policy_version 265191 (0.0011) [2023-12-26 17:17:38,523][105620] Updated weights for policy 1, policy_version 265289 (0.0008) [2023-12-26 17:17:38,587][105620] Updated weights for policy 1, policy_version 265299 (0.0007) [2023-12-26 17:17:38,645][105620] Updated weights for policy 1, policy_version 265309 (0.0007) [2023-12-26 17:17:38,701][105620] Updated weights for policy 1, policy_version 265319 (0.0005) [2023-12-26 17:17:39,192][105692] Updated weights for policy 0, policy_version 265201 (0.0006) [2023-12-26 17:17:39,249][105692] Updated weights for policy 0, policy_version 265211 (0.0007) [2023-12-26 17:17:39,316][105692] Updated weights for policy 0, policy_version 265221 (0.0009) [2023-12-26 17:17:39,433][105620] Updated weights for policy 1, policy_version 265329 (0.0008) [2023-12-26 17:17:39,503][105620] Updated weights for policy 1, policy_version 265339 (0.0008) [2023-12-26 17:17:39,571][105620] Updated weights for policy 1, policy_version 265349 (0.0007) [2023-12-26 17:17:40,088][105692] Updated weights for policy 0, policy_version 265231 (0.0011) [2023-12-26 17:17:40,137][105692] Updated weights for policy 0, policy_version 265241 (0.0010) [2023-12-26 17:17:40,188][105692] Updated weights for policy 0, policy_version 265251 (0.0010) [2023-12-26 17:17:40,310][105620] Updated weights for policy 1, policy_version 265359 (0.0007) [2023-12-26 17:17:40,372][105620] Updated weights for policy 1, policy_version 265369 (0.0008) [2023-12-26 17:17:40,425][105620] Updated weights for policy 1, policy_version 265379 (0.0010) [2023-12-26 17:17:40,859][105692] Updated weights for policy 0, policy_version 265261 (0.0008) [2023-12-26 17:17:40,919][105692] Updated weights for policy 0, policy_version 265271 (0.0005) [2023-12-26 17:17:40,974][105692] Updated weights for policy 0, policy_version 265281 (0.0010) [2023-12-26 17:17:41,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 135872512. Throughput: 0: 9859.8, 1: 9668.9. Samples: 135878344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:17:41,062][104569] Avg episode reward: [(0, '1223.100'), (1, '2564.741')] [2023-12-26 17:17:41,248][105620] Updated weights for policy 1, policy_version 265389 (0.0009) [2023-12-26 17:17:41,310][105620] Updated weights for policy 1, policy_version 265399 (0.0008) [2023-12-26 17:17:41,373][105620] Updated weights for policy 1, policy_version 265409 (0.0007) [2023-12-26 17:17:41,646][105692] Updated weights for policy 0, policy_version 265291 (0.0010) [2023-12-26 17:17:41,721][105692] Updated weights for policy 0, policy_version 265301 (0.0007) [2023-12-26 17:17:41,781][105692] Updated weights for policy 0, policy_version 265311 (0.0007) [2023-12-26 17:17:42,162][105620] Updated weights for policy 1, policy_version 265419 (0.0008) [2023-12-26 17:17:42,219][105620] Updated weights for policy 1, policy_version 265429 (0.0008) [2023-12-26 17:17:42,274][105620] Updated weights for policy 1, policy_version 265439 (0.0009) [2023-12-26 17:17:42,489][105692] Updated weights for policy 0, policy_version 265321 (0.0006) [2023-12-26 17:17:42,552][105692] Updated weights for policy 0, policy_version 265331 (0.0011) [2023-12-26 17:17:42,614][105692] Updated weights for policy 0, policy_version 265341 (0.0010) [2023-12-26 17:17:42,680][105692] Updated weights for policy 0, policy_version 265351 (0.0011) [2023-12-26 17:17:42,995][105620] Updated weights for policy 1, policy_version 265449 (0.0009) [2023-12-26 17:17:43,048][105620] Updated weights for policy 1, policy_version 265459 (0.0009) [2023-12-26 17:17:43,126][105620] Updated weights for policy 1, policy_version 265469 (0.0009) [2023-12-26 17:17:43,182][105620] Updated weights for policy 1, policy_version 265479 (0.0008) [2023-12-26 17:17:43,343][105692] Updated weights for policy 0, policy_version 265361 (0.0010) [2023-12-26 17:17:43,388][105692] Updated weights for policy 0, policy_version 265371 (0.0010) [2023-12-26 17:17:43,445][105692] Updated weights for policy 0, policy_version 265381 (0.0010) [2023-12-26 17:17:43,919][105620] Updated weights for policy 1, policy_version 265489 (0.0007) [2023-12-26 17:17:43,971][105620] Updated weights for policy 1, policy_version 265499 (0.0008) [2023-12-26 17:17:44,023][105620] Updated weights for policy 1, policy_version 265509 (0.0008) [2023-12-26 17:17:44,164][105692] Updated weights for policy 0, policy_version 265391 (0.0010) [2023-12-26 17:17:44,218][105692] Updated weights for policy 0, policy_version 265401 (0.0010) [2023-12-26 17:17:44,278][105692] Updated weights for policy 0, policy_version 265411 (0.0011) [2023-12-26 17:17:44,698][105620] Updated weights for policy 1, policy_version 265519 (0.0006) [2023-12-26 17:17:44,755][105620] Updated weights for policy 1, policy_version 265529 (0.0009) [2023-12-26 17:17:44,823][105620] Updated weights for policy 1, policy_version 265539 (0.0008) [2023-12-26 17:17:44,903][105692] Updated weights for policy 0, policy_version 265421 (0.0009) [2023-12-26 17:17:44,960][105692] Updated weights for policy 0, policy_version 265431 (0.0011) [2023-12-26 17:17:45,018][105692] Updated weights for policy 0, policy_version 265441 (0.0011) [2023-12-26 17:17:45,593][105620] Updated weights for policy 1, policy_version 265549 (0.0009) [2023-12-26 17:17:45,648][105620] Updated weights for policy 1, policy_version 265559 (0.0009) [2023-12-26 17:17:45,711][105620] Updated weights for policy 1, policy_version 265569 (0.0006) [2023-12-26 17:17:45,758][105692] Updated weights for policy 0, policy_version 265451 (0.0009) [2023-12-26 17:17:45,823][105692] Updated weights for policy 0, policy_version 265461 (0.0009) [2023-12-26 17:17:45,871][105692] Updated weights for policy 0, policy_version 265471 (0.0009) [2023-12-26 17:17:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 135970816. Throughput: 0: 9797.5, 1: 9661.8. Samples: 135936216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:17:46,062][104569] Avg episode reward: [(0, '5417.002'), (1, '2274.592')] [2023-12-26 17:17:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000265480_67977216.pth... [2023-12-26 17:17:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000265576_67993600.pth... [2023-12-26 17:17:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000264456_67706880.pth [2023-12-26 17:17:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000264328_67682304.pth [2023-12-26 17:17:46,476][105692] Updated weights for policy 0, policy_version 265481 (0.0007) [2023-12-26 17:17:46,499][105620] Updated weights for policy 1, policy_version 265579 (0.0009) [2023-12-26 17:17:46,541][105692] Updated weights for policy 0, policy_version 265491 (0.0005) [2023-12-26 17:17:46,556][105620] Updated weights for policy 1, policy_version 265589 (0.0008) [2023-12-26 17:17:46,607][105692] Updated weights for policy 0, policy_version 265501 (0.0005) [2023-12-26 17:17:46,615][105620] Updated weights for policy 1, policy_version 265599 (0.0008) [2023-12-26 17:17:46,658][105692] Updated weights for policy 0, policy_version 265511 (0.0009) [2023-12-26 17:17:47,261][105620] Updated weights for policy 1, policy_version 265609 (0.0006) [2023-12-26 17:17:47,314][105620] Updated weights for policy 1, policy_version 265619 (0.0008) [2023-12-26 17:17:47,377][105620] Updated weights for policy 1, policy_version 265629 (0.0008) [2023-12-26 17:17:47,412][105692] Updated weights for policy 0, policy_version 265521 (0.0008) [2023-12-26 17:17:47,426][105620] Updated weights for policy 1, policy_version 265639 (0.0005) [2023-12-26 17:17:47,471][105692] Updated weights for policy 0, policy_version 265531 (0.0010) [2023-12-26 17:17:47,536][105692] Updated weights for policy 0, policy_version 265541 (0.0010) [2023-12-26 17:17:48,063][105620] Updated weights for policy 1, policy_version 265649 (0.0008) [2023-12-26 17:17:48,110][105620] Updated weights for policy 1, policy_version 265659 (0.0008) [2023-12-26 17:17:48,158][105620] Updated weights for policy 1, policy_version 265669 (0.0009) [2023-12-26 17:17:48,319][105692] Updated weights for policy 0, policy_version 265551 (0.0009) [2023-12-26 17:17:48,375][105692] Updated weights for policy 0, policy_version 265561 (0.0009) [2023-12-26 17:17:48,433][105692] Updated weights for policy 0, policy_version 265571 (0.0009) [2023-12-26 17:17:48,868][105620] Updated weights for policy 1, policy_version 265679 (0.0007) [2023-12-26 17:17:48,935][105620] Updated weights for policy 1, policy_version 265689 (0.0006) [2023-12-26 17:17:49,008][105620] Updated weights for policy 1, policy_version 265699 (0.0006) [2023-12-26 17:17:49,279][105692] Updated weights for policy 0, policy_version 265581 (0.0009) [2023-12-26 17:17:49,343][105692] Updated weights for policy 0, policy_version 265591 (0.0008) [2023-12-26 17:17:49,403][105692] Updated weights for policy 0, policy_version 265601 (0.0006) [2023-12-26 17:17:49,721][105620] Updated weights for policy 1, policy_version 265709 (0.0008) [2023-12-26 17:17:49,780][105620] Updated weights for policy 1, policy_version 265719 (0.0009) [2023-12-26 17:17:49,848][105620] Updated weights for policy 1, policy_version 265729 (0.0008) [2023-12-26 17:17:50,049][105692] Updated weights for policy 0, policy_version 265611 (0.0006) [2023-12-26 17:17:50,114][105692] Updated weights for policy 0, policy_version 265621 (0.0009) [2023-12-26 17:17:50,170][105692] Updated weights for policy 0, policy_version 265631 (0.0008) [2023-12-26 17:17:50,687][105620] Updated weights for policy 1, policy_version 265739 (0.0007) [2023-12-26 17:17:50,743][105620] Updated weights for policy 1, policy_version 265750 (0.0009) [2023-12-26 17:17:50,797][105620] Updated weights for policy 1, policy_version 265761 (0.0010) [2023-12-26 17:17:50,820][105692] Updated weights for policy 0, policy_version 265641 (0.0010) [2023-12-26 17:17:50,878][105692] Updated weights for policy 0, policy_version 265651 (0.0010) [2023-12-26 17:17:50,939][105692] Updated weights for policy 0, policy_version 265661 (0.0009) [2023-12-26 17:17:50,991][105692] Updated weights for policy 0, policy_version 265671 (0.0009) [2023-12-26 17:17:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 136069120. Throughput: 0: 9865.9, 1: 9579.5. Samples: 136053232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:17:51,062][104569] Avg episode reward: [(0, '6538.901'), (1, '6726.657')] [2023-12-26 17:17:51,597][105620] Updated weights for policy 1, policy_version 265771 (0.0007) [2023-12-26 17:17:51,660][105620] Updated weights for policy 1, policy_version 265781 (0.0008) [2023-12-26 17:17:51,722][105620] Updated weights for policy 1, policy_version 265791 (0.0008) [2023-12-26 17:17:51,752][105692] Updated weights for policy 0, policy_version 265681 (0.0008) [2023-12-26 17:17:51,811][105692] Updated weights for policy 0, policy_version 265691 (0.0008) [2023-12-26 17:17:51,861][105692] Updated weights for policy 0, policy_version 265701 (0.0008) [2023-12-26 17:17:52,356][105620] Updated weights for policy 1, policy_version 265801 (0.0008) [2023-12-26 17:17:52,415][105620] Updated weights for policy 1, policy_version 265811 (0.0008) [2023-12-26 17:17:52,473][105620] Updated weights for policy 1, policy_version 265821 (0.0009) [2023-12-26 17:17:52,531][105620] Updated weights for policy 1, policy_version 265831 (0.0007) [2023-12-26 17:17:52,733][105692] Updated weights for policy 0, policy_version 265711 (0.0009) [2023-12-26 17:17:52,786][105692] Updated weights for policy 0, policy_version 265721 (0.0008) [2023-12-26 17:17:52,851][105692] Updated weights for policy 0, policy_version 265731 (0.0005) [2023-12-26 17:17:53,253][105620] Updated weights for policy 1, policy_version 265841 (0.0008) [2023-12-26 17:17:53,317][105620] Updated weights for policy 1, policy_version 265851 (0.0008) [2023-12-26 17:17:53,383][105620] Updated weights for policy 1, policy_version 265861 (0.0009) [2023-12-26 17:17:53,463][105692] Updated weights for policy 0, policy_version 265741 (0.0005) [2023-12-26 17:17:53,511][105692] Updated weights for policy 0, policy_version 265751 (0.0005) [2023-12-26 17:17:53,561][105692] Updated weights for policy 0, policy_version 265761 (0.0005) [2023-12-26 17:17:54,000][105620] Updated weights for policy 1, policy_version 265871 (0.0007) [2023-12-26 17:17:54,053][105620] Updated weights for policy 1, policy_version 265881 (0.0005) [2023-12-26 17:17:54,106][105620] Updated weights for policy 1, policy_version 265891 (0.0007) [2023-12-26 17:17:54,224][105692] Updated weights for policy 0, policy_version 265771 (0.0008) [2023-12-26 17:17:54,279][105692] Updated weights for policy 0, policy_version 265781 (0.0010) [2023-12-26 17:17:54,344][105692] Updated weights for policy 0, policy_version 265791 (0.0010) [2023-12-26 17:17:54,770][105620] Updated weights for policy 1, policy_version 265901 (0.0008) [2023-12-26 17:17:54,830][105620] Updated weights for policy 1, policy_version 265911 (0.0009) [2023-12-26 17:17:54,890][105620] Updated weights for policy 1, policy_version 265921 (0.0007) [2023-12-26 17:17:55,074][105692] Updated weights for policy 0, policy_version 265801 (0.0010) [2023-12-26 17:17:55,136][105692] Updated weights for policy 0, policy_version 265811 (0.0009) [2023-12-26 17:17:55,201][105692] Updated weights for policy 0, policy_version 265822 (0.0010) [2023-12-26 17:17:55,547][105620] Updated weights for policy 1, policy_version 265931 (0.0007) [2023-12-26 17:17:55,598][105620] Updated weights for policy 1, policy_version 265941 (0.0008) [2023-12-26 17:17:55,647][105620] Updated weights for policy 1, policy_version 265951 (0.0005) [2023-12-26 17:17:55,958][105692] Updated weights for policy 0, policy_version 265833 (0.0009) [2023-12-26 17:17:56,009][105692] Updated weights for policy 0, policy_version 265843 (0.0005) [2023-12-26 17:17:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 136159232. Throughput: 0: 9845.5, 1: 9636.2. Samples: 136171276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:17:56,062][105692] Updated weights for policy 0, policy_version 265853 (0.0006) [2023-12-26 17:17:56,062][104569] Avg episode reward: [(0, '8917.064'), (1, '8823.505')] [2023-12-26 17:17:56,111][105692] Updated weights for policy 0, policy_version 265863 (0.0005) [2023-12-26 17:17:56,423][105620] Updated weights for policy 1, policy_version 265961 (0.0006) [2023-12-26 17:17:56,475][105620] Updated weights for policy 1, policy_version 265971 (0.0008) [2023-12-26 17:17:56,523][105620] Updated weights for policy 1, policy_version 265981 (0.0008) [2023-12-26 17:17:56,582][105620] Updated weights for policy 1, policy_version 265991 (0.0008) [2023-12-26 17:17:56,784][105692] Updated weights for policy 0, policy_version 265873 (0.0010) [2023-12-26 17:17:56,838][105692] Updated weights for policy 0, policy_version 265883 (0.0010) [2023-12-26 17:17:56,886][105692] Updated weights for policy 0, policy_version 265893 (0.0010) [2023-12-26 17:17:57,323][105620] Updated weights for policy 1, policy_version 266001 (0.0007) [2023-12-26 17:17:57,390][105620] Updated weights for policy 1, policy_version 266012 (0.0010) [2023-12-26 17:17:57,448][105620] Updated weights for policy 1, policy_version 266022 (0.0010) [2023-12-26 17:17:57,547][105692] Updated weights for policy 0, policy_version 265903 (0.0007) [2023-12-26 17:17:57,589][105692] Updated weights for policy 0, policy_version 265913 (0.0005) [2023-12-26 17:17:57,637][105692] Updated weights for policy 0, policy_version 265923 (0.0005) [2023-12-26 17:17:58,231][105620] Updated weights for policy 1, policy_version 266033 (0.0010) [2023-12-26 17:17:58,260][105692] Updated weights for policy 0, policy_version 265933 (0.0008) [2023-12-26 17:17:58,291][105620] Updated weights for policy 1, policy_version 266043 (0.0011) [2023-12-26 17:17:58,322][105692] Updated weights for policy 0, policy_version 265943 (0.0010) [2023-12-26 17:17:58,355][105620] Updated weights for policy 1, policy_version 266053 (0.0009) [2023-12-26 17:17:58,387][105692] Updated weights for policy 0, policy_version 265953 (0.0010) [2023-12-26 17:17:59,091][105620] Updated weights for policy 1, policy_version 266063 (0.0010) [2023-12-26 17:17:59,154][105620] Updated weights for policy 1, policy_version 266073 (0.0011) [2023-12-26 17:17:59,216][105620] Updated weights for policy 1, policy_version 266083 (0.0010) [2023-12-26 17:17:59,218][105692] Updated weights for policy 0, policy_version 265963 (0.0010) [2023-12-26 17:17:59,280][105692] Updated weights for policy 0, policy_version 265973 (0.0008) [2023-12-26 17:17:59,338][105692] Updated weights for policy 0, policy_version 265983 (0.0009) [2023-12-26 17:17:59,855][105620] Updated weights for policy 1, policy_version 266093 (0.0009) [2023-12-26 17:17:59,912][105620] Updated weights for policy 1, policy_version 266103 (0.0009) [2023-12-26 17:17:59,971][105620] Updated weights for policy 1, policy_version 266113 (0.0006) [2023-12-26 17:18:00,086][105692] Updated weights for policy 0, policy_version 265993 (0.0009) [2023-12-26 17:18:00,142][105692] Updated weights for policy 0, policy_version 266003 (0.0006) [2023-12-26 17:18:00,204][105692] Updated weights for policy 0, policy_version 266013 (0.0005) [2023-12-26 17:18:00,275][105692] Updated weights for policy 0, policy_version 266023 (0.0005) [2023-12-26 17:18:00,671][105620] Updated weights for policy 1, policy_version 266123 (0.0008) [2023-12-26 17:18:00,737][105620] Updated weights for policy 1, policy_version 266133 (0.0006) [2023-12-26 17:18:00,802][105620] Updated weights for policy 1, policy_version 266143 (0.0008) [2023-12-26 17:18:00,919][105692] Updated weights for policy 0, policy_version 266033 (0.0009) [2023-12-26 17:18:00,979][105692] Updated weights for policy 0, policy_version 266043 (0.0009) [2023-12-26 17:18:01,048][105692] Updated weights for policy 0, policy_version 266053 (0.0009) [2023-12-26 17:18:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 136257536. Throughput: 0: 9874.1, 1: 9590.8. Samples: 136230224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:18:01,062][104569] Avg episode reward: [(0, '9186.909'), (1, '8912.332')] [2023-12-26 17:18:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000266056_68124672.pth... [2023-12-26 17:18:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000266152_68141056.pth... [2023-12-26 17:18:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000264872_67821568.pth [2023-12-26 17:18:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000265032_67854336.pth [2023-12-26 17:18:01,499][105620] Updated weights for policy 1, policy_version 266153 (0.0008) [2023-12-26 17:18:01,559][105620] Updated weights for policy 1, policy_version 266163 (0.0009) [2023-12-26 17:18:01,627][105620] Updated weights for policy 1, policy_version 266173 (0.0009) [2023-12-26 17:18:01,685][105620] Updated weights for policy 1, policy_version 266183 (0.0009) [2023-12-26 17:18:01,800][105692] Updated weights for policy 0, policy_version 266063 (0.0009) [2023-12-26 17:18:01,853][105692] Updated weights for policy 0, policy_version 266073 (0.0005) [2023-12-26 17:18:01,905][105692] Updated weights for policy 0, policy_version 266083 (0.0005) [2023-12-26 17:18:02,303][105620] Updated weights for policy 1, policy_version 266193 (0.0008) [2023-12-26 17:18:02,366][105620] Updated weights for policy 1, policy_version 266203 (0.0008) [2023-12-26 17:18:02,423][105620] Updated weights for policy 1, policy_version 266213 (0.0008) [2023-12-26 17:18:02,548][105692] Updated weights for policy 0, policy_version 266093 (0.0006) [2023-12-26 17:18:02,601][105692] Updated weights for policy 0, policy_version 266103 (0.0007) [2023-12-26 17:18:02,648][105692] Updated weights for policy 0, policy_version 266113 (0.0009) [2023-12-26 17:18:03,074][105620] Updated weights for policy 1, policy_version 266223 (0.0005) [2023-12-26 17:18:03,128][105620] Updated weights for policy 1, policy_version 266233 (0.0006) [2023-12-26 17:18:03,171][105620] Updated weights for policy 1, policy_version 266243 (0.0005) [2023-12-26 17:18:03,445][105692] Updated weights for policy 0, policy_version 266123 (0.0008) [2023-12-26 17:18:03,507][105692] Updated weights for policy 0, policy_version 266133 (0.0009) [2023-12-26 17:18:03,566][105692] Updated weights for policy 0, policy_version 266143 (0.0009) [2023-12-26 17:18:03,733][105620] Updated weights for policy 1, policy_version 266253 (0.0008) [2023-12-26 17:18:03,777][105620] Updated weights for policy 1, policy_version 266263 (0.0006) [2023-12-26 17:18:03,837][105620] Updated weights for policy 1, policy_version 266273 (0.0007) [2023-12-26 17:18:04,136][105692] Updated weights for policy 0, policy_version 266153 (0.0006) [2023-12-26 17:18:04,199][105692] Updated weights for policy 0, policy_version 266163 (0.0010) [2023-12-26 17:18:04,253][105692] Updated weights for policy 0, policy_version 266173 (0.0009) [2023-12-26 17:18:04,305][105692] Updated weights for policy 0, policy_version 266183 (0.0010) [2023-12-26 17:18:04,513][105620] Updated weights for policy 1, policy_version 266283 (0.0007) [2023-12-26 17:18:04,574][105620] Updated weights for policy 1, policy_version 266293 (0.0005) [2023-12-26 17:18:04,639][105620] Updated weights for policy 1, policy_version 266303 (0.0006) [2023-12-26 17:18:05,055][105692] Updated weights for policy 0, policy_version 266193 (0.0010) [2023-12-26 17:18:05,106][105692] Updated weights for policy 0, policy_version 266203 (0.0010) [2023-12-26 17:18:05,154][105692] Updated weights for policy 0, policy_version 266213 (0.0010) [2023-12-26 17:18:05,285][105620] Updated weights for policy 1, policy_version 266313 (0.0005) [2023-12-26 17:18:05,338][105620] Updated weights for policy 1, policy_version 266323 (0.0007) [2023-12-26 17:18:05,393][105620] Updated weights for policy 1, policy_version 266333 (0.0009) [2023-12-26 17:18:05,454][105620] Updated weights for policy 1, policy_version 266343 (0.0006) [2023-12-26 17:18:05,793][105692] Updated weights for policy 0, policy_version 266223 (0.0007) [2023-12-26 17:18:05,855][105692] Updated weights for policy 0, policy_version 266233 (0.0008) [2023-12-26 17:18:05,909][105692] Updated weights for policy 0, policy_version 266243 (0.0007) [2023-12-26 17:18:06,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 136364032. Throughput: 0: 9797.5, 1: 9744.5. Samples: 136352008. Policy #0 lag: (min: 17.0, avg: 36.0, max: 49.0) [2023-12-26 17:18:06,063][104569] Avg episode reward: [(0, '8905.123'), (1, '8900.957')] [2023-12-26 17:18:06,102][105620] Updated weights for policy 1, policy_version 266353 (0.0010) [2023-12-26 17:18:06,167][105620] Updated weights for policy 1, policy_version 266363 (0.0011) [2023-12-26 17:18:06,224][105620] Updated weights for policy 1, policy_version 266373 (0.0012) [2023-12-26 17:18:06,640][105692] Updated weights for policy 0, policy_version 266253 (0.0007) [2023-12-26 17:18:06,701][105692] Updated weights for policy 0, policy_version 266263 (0.0006) [2023-12-26 17:18:06,774][105692] Updated weights for policy 0, policy_version 266273 (0.0006) [2023-12-26 17:18:06,805][105620] Updated weights for policy 1, policy_version 266383 (0.0009) [2023-12-26 17:18:06,853][105620] Updated weights for policy 1, policy_version 266393 (0.0010) [2023-12-26 17:18:06,902][105620] Updated weights for policy 1, policy_version 266403 (0.0010) [2023-12-26 17:18:07,379][105692] Updated weights for policy 0, policy_version 266283 (0.0006) [2023-12-26 17:18:07,435][105692] Updated weights for policy 0, policy_version 266293 (0.0009) [2023-12-26 17:18:07,484][105692] Updated weights for policy 0, policy_version 266303 (0.0010) [2023-12-26 17:18:07,640][105620] Updated weights for policy 1, policy_version 266413 (0.0009) [2023-12-26 17:18:07,695][105620] Updated weights for policy 1, policy_version 266423 (0.0005) [2023-12-26 17:18:07,746][105620] Updated weights for policy 1, policy_version 266433 (0.0006) [2023-12-26 17:18:08,105][105692] Updated weights for policy 0, policy_version 266313 (0.0010) [2023-12-26 17:18:08,159][105692] Updated weights for policy 0, policy_version 266323 (0.0010) [2023-12-26 17:18:08,224][105692] Updated weights for policy 0, policy_version 266333 (0.0010) [2023-12-26 17:18:08,282][105692] Updated weights for policy 0, policy_version 266343 (0.0010) [2023-12-26 17:18:08,316][105620] Updated weights for policy 1, policy_version 266443 (0.0006) [2023-12-26 17:18:08,378][105620] Updated weights for policy 1, policy_version 266453 (0.0009) [2023-12-26 17:18:08,439][105620] Updated weights for policy 1, policy_version 266463 (0.0008) [2023-12-26 17:18:09,043][105620] Updated weights for policy 1, policy_version 266473 (0.0005) [2023-12-26 17:18:09,077][105692] Updated weights for policy 0, policy_version 266353 (0.0007) [2023-12-26 17:18:09,097][105620] Updated weights for policy 1, policy_version 266483 (0.0006) [2023-12-26 17:18:09,135][105692] Updated weights for policy 0, policy_version 266363 (0.0008) [2023-12-26 17:18:09,150][105620] Updated weights for policy 1, policy_version 266493 (0.0005) [2023-12-26 17:18:09,194][105692] Updated weights for policy 0, policy_version 266373 (0.0009) [2023-12-26 17:18:09,211][105620] Updated weights for policy 1, policy_version 266503 (0.0006) [2023-12-26 17:18:09,858][105620] Updated weights for policy 1, policy_version 266513 (0.0010) [2023-12-26 17:18:09,931][105620] Updated weights for policy 1, policy_version 266523 (0.0007) [2023-12-26 17:18:10,000][105620] Updated weights for policy 1, policy_version 266533 (0.0008) [2023-12-26 17:18:10,007][105692] Updated weights for policy 0, policy_version 266383 (0.0009) [2023-12-26 17:18:10,065][105692] Updated weights for policy 0, policy_version 266393 (0.0009) [2023-12-26 17:18:10,124][105692] Updated weights for policy 0, policy_version 266403 (0.0009) [2023-12-26 17:18:10,720][105620] Updated weights for policy 1, policy_version 266543 (0.0009) [2023-12-26 17:18:10,771][105620] Updated weights for policy 1, policy_version 266553 (0.0006) [2023-12-26 17:18:10,826][105620] Updated weights for policy 1, policy_version 266563 (0.0005) [2023-12-26 17:18:10,905][105692] Updated weights for policy 0, policy_version 266413 (0.0009) [2023-12-26 17:18:10,969][105692] Updated weights for policy 0, policy_version 266423 (0.0008) [2023-12-26 17:18:11,043][105692] Updated weights for policy 0, policy_version 266433 (0.0009) [2023-12-26 17:18:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 136462336. Throughput: 0: 9832.4, 1: 9875.4. Samples: 136474156. Policy #0 lag: (min: 17.0, avg: 36.0, max: 49.0) [2023-12-26 17:18:11,062][104569] Avg episode reward: [(0, '8905.199'), (1, '9264.339')] [2023-12-26 17:18:11,643][105620] Updated weights for policy 1, policy_version 266573 (0.0007) [2023-12-26 17:18:11,713][105620] Updated weights for policy 1, policy_version 266583 (0.0007) [2023-12-26 17:18:11,764][105692] Updated weights for policy 0, policy_version 266443 (0.0008) [2023-12-26 17:18:11,780][105620] Updated weights for policy 1, policy_version 266593 (0.0010) [2023-12-26 17:18:11,816][105585] KL-divergence is very high: 209.1566 [2023-12-26 17:18:11,822][105692] Updated weights for policy 0, policy_version 266453 (0.0009) [2023-12-26 17:18:11,829][105585] KL-divergence is very high: 251.7615 [2023-12-26 17:18:11,867][105585] KL-divergence is very high: 341.0310 [2023-12-26 17:18:11,877][105585] KL-divergence is very high: 323.9052 [2023-12-26 17:18:11,883][105692] Updated weights for policy 0, policy_version 266463 (0.0010) [2023-12-26 17:18:11,913][105585] KL-divergence is very high: 303.3153 [2023-12-26 17:18:11,924][105585] KL-divergence is very high: 269.9346 [2023-12-26 17:18:12,420][105620] Updated weights for policy 1, policy_version 266603 (0.0006) [2023-12-26 17:18:12,490][105620] Updated weights for policy 1, policy_version 266613 (0.0011) [2023-12-26 17:18:12,548][105620] Updated weights for policy 1, policy_version 266623 (0.0011) [2023-12-26 17:18:12,645][105692] Updated weights for policy 0, policy_version 266473 (0.0009) [2023-12-26 17:18:12,709][105692] Updated weights for policy 0, policy_version 266483 (0.0011) [2023-12-26 17:18:12,771][105692] Updated weights for policy 0, policy_version 266493 (0.0011) [2023-12-26 17:18:12,836][105692] Updated weights for policy 0, policy_version 266503 (0.0011) [2023-12-26 17:18:13,368][105620] Updated weights for policy 1, policy_version 266633 (0.0010) [2023-12-26 17:18:13,383][105692] Updated weights for policy 0, policy_version 266513 (0.0006) [2023-12-26 17:18:13,424][105620] Updated weights for policy 1, policy_version 266643 (0.0009) [2023-12-26 17:18:13,434][105692] Updated weights for policy 0, policy_version 266523 (0.0005) [2023-12-26 17:18:13,478][105692] Updated weights for policy 0, policy_version 266533 (0.0005) [2023-12-26 17:18:13,480][105620] Updated weights for policy 1, policy_version 266653 (0.0009) [2023-12-26 17:18:13,529][105620] Updated weights for policy 1, policy_version 266663 (0.0008) [2023-12-26 17:18:14,137][105692] Updated weights for policy 0, policy_version 266543 (0.0009) [2023-12-26 17:18:14,192][105692] Updated weights for policy 0, policy_version 266553 (0.0010) [2023-12-26 17:18:14,257][105692] Updated weights for policy 0, policy_version 266563 (0.0011) [2023-12-26 17:18:14,259][105620] Updated weights for policy 1, policy_version 266673 (0.0006) [2023-12-26 17:18:14,313][105620] Updated weights for policy 1, policy_version 266683 (0.0007) [2023-12-26 17:18:14,361][105620] Updated weights for policy 1, policy_version 266693 (0.0008) [2023-12-26 17:18:15,023][105692] Updated weights for policy 0, policy_version 266573 (0.0010) [2023-12-26 17:18:15,088][105692] Updated weights for policy 0, policy_version 266583 (0.0009) [2023-12-26 17:18:15,116][105620] Updated weights for policy 1, policy_version 266703 (0.0007) [2023-12-26 17:18:15,155][105692] Updated weights for policy 0, policy_version 266593 (0.0008) [2023-12-26 17:18:15,171][105620] Updated weights for policy 1, policy_version 266713 (0.0006) [2023-12-26 17:18:15,225][105620] Updated weights for policy 1, policy_version 266723 (0.0007) [2023-12-26 17:18:15,879][105692] Updated weights for policy 0, policy_version 266603 (0.0008) [2023-12-26 17:18:15,934][105692] Updated weights for policy 0, policy_version 266613 (0.0005) [2023-12-26 17:18:15,991][105692] Updated weights for policy 0, policy_version 266623 (0.0007) [2023-12-26 17:18:16,013][105620] Updated weights for policy 1, policy_version 266733 (0.0008) [2023-12-26 17:18:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 136560640. Throughput: 0: 9860.3, 1: 9801.6. Samples: 136531200. Policy #0 lag: (min: 17.0, avg: 36.0, max: 49.0) [2023-12-26 17:18:16,063][104569] Avg episode reward: [(0, '8992.318'), (1, '9265.914')] [2023-12-26 17:18:16,064][105620] Updated weights for policy 1, policy_version 266743 (0.0007) [2023-12-26 17:18:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000266632_68272128.pth... [2023-12-26 17:18:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000265480_67977216.pth [2023-12-26 17:18:16,113][105620] Updated weights for policy 1, policy_version 266753 (0.0008) [2023-12-26 17:18:16,144][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000266760_68296704.pth... [2023-12-26 17:18:16,158][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000265576_67993600.pth [2023-12-26 17:18:16,699][105692] Updated weights for policy 0, policy_version 266633 (0.0006) [2023-12-26 17:18:16,758][105692] Updated weights for policy 0, policy_version 266643 (0.0008) [2023-12-26 17:18:16,810][105692] Updated weights for policy 0, policy_version 266653 (0.0009) [2023-12-26 17:18:16,868][105692] Updated weights for policy 0, policy_version 266663 (0.0009) [2023-12-26 17:18:16,898][105620] Updated weights for policy 1, policy_version 266763 (0.0009) [2023-12-26 17:18:16,946][105620] Updated weights for policy 1, policy_version 266773 (0.0009) [2023-12-26 17:18:17,010][105620] Updated weights for policy 1, policy_version 266783 (0.0009) [2023-12-26 17:18:17,490][105692] Updated weights for policy 0, policy_version 266673 (0.0007) [2023-12-26 17:18:17,540][105692] Updated weights for policy 0, policy_version 266683 (0.0006) [2023-12-26 17:18:17,588][105692] Updated weights for policy 0, policy_version 266693 (0.0005) [2023-12-26 17:18:17,880][105620] Updated weights for policy 1, policy_version 266793 (0.0008) [2023-12-26 17:18:17,931][105620] Updated weights for policy 1, policy_version 266803 (0.0008) [2023-12-26 17:18:17,982][105620] Updated weights for policy 1, policy_version 266813 (0.0007) [2023-12-26 17:18:18,038][105620] Updated weights for policy 1, policy_version 266823 (0.0008) [2023-12-26 17:18:18,258][105692] Updated weights for policy 0, policy_version 266703 (0.0009) [2023-12-26 17:18:18,313][105692] Updated weights for policy 0, policy_version 266713 (0.0010) [2023-12-26 17:18:18,373][105692] Updated weights for policy 0, policy_version 266723 (0.0010) [2023-12-26 17:18:18,388][105585] KL-divergence is very high: 107.4562 [2023-12-26 17:18:18,811][105620] Updated weights for policy 1, policy_version 266833 (0.0008) [2023-12-26 17:18:18,874][105620] Updated weights for policy 1, policy_version 266843 (0.0008) [2023-12-26 17:18:18,936][105620] Updated weights for policy 1, policy_version 266853 (0.0009) [2023-12-26 17:18:19,122][105692] Updated weights for policy 0, policy_version 266733 (0.0009) [2023-12-26 17:18:19,174][105692] Updated weights for policy 0, policy_version 266743 (0.0009) [2023-12-26 17:18:19,224][105692] Updated weights for policy 0, policy_version 266753 (0.0009) [2023-12-26 17:18:19,678][105620] Updated weights for policy 1, policy_version 266863 (0.0009) [2023-12-26 17:18:19,741][105620] Updated weights for policy 1, policy_version 266873 (0.0008) [2023-12-26 17:18:19,813][105620] Updated weights for policy 1, policy_version 266883 (0.0009) [2023-12-26 17:18:20,044][105692] Updated weights for policy 0, policy_version 266763 (0.0009) [2023-12-26 17:18:20,103][105692] Updated weights for policy 0, policy_version 266773 (0.0009) [2023-12-26 17:18:20,160][105692] Updated weights for policy 0, policy_version 266783 (0.0010) [2023-12-26 17:18:20,564][105620] Updated weights for policy 1, policy_version 266893 (0.0009) [2023-12-26 17:18:20,625][105620] Updated weights for policy 1, policy_version 266903 (0.0009) [2023-12-26 17:18:20,677][105620] Updated weights for policy 1, policy_version 266913 (0.0009) [2023-12-26 17:18:20,951][105692] Updated weights for policy 0, policy_version 266793 (0.0010) [2023-12-26 17:18:21,008][105692] Updated weights for policy 0, policy_version 266803 (0.0009) [2023-12-26 17:18:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 136650752. Throughput: 0: 9883.5, 1: 9711.4. Samples: 136644744. Policy #0 lag: (min: 17.0, avg: 36.0, max: 49.0) [2023-12-26 17:18:21,063][104569] Avg episode reward: [(0, '8991.959'), (1, '9356.247')] [2023-12-26 17:18:21,073][105692] Updated weights for policy 0, policy_version 266813 (0.0008) [2023-12-26 17:18:21,145][105692] Updated weights for policy 0, policy_version 266824 (0.0008) [2023-12-26 17:18:21,466][105620] Updated weights for policy 1, policy_version 266923 (0.0009) [2023-12-26 17:18:21,527][105620] Updated weights for policy 1, policy_version 266933 (0.0008) [2023-12-26 17:18:21,598][105620] Updated weights for policy 1, policy_version 266943 (0.0006) [2023-12-26 17:18:21,885][105692] Updated weights for policy 0, policy_version 266834 (0.0009) [2023-12-26 17:18:21,942][105692] Updated weights for policy 0, policy_version 266844 (0.0010) [2023-12-26 17:18:22,000][105692] Updated weights for policy 0, policy_version 266854 (0.0010) [2023-12-26 17:18:22,230][105620] Updated weights for policy 1, policy_version 266953 (0.0010) [2023-12-26 17:18:22,298][105620] Updated weights for policy 1, policy_version 266963 (0.0010) [2023-12-26 17:18:22,362][105620] Updated weights for policy 1, policy_version 266973 (0.0010) [2023-12-26 17:18:22,428][105620] Updated weights for policy 1, policy_version 266983 (0.0009) [2023-12-26 17:18:22,904][105692] Updated weights for policy 0, policy_version 266864 (0.0010) [2023-12-26 17:18:22,953][105692] Updated weights for policy 0, policy_version 266874 (0.0009) [2023-12-26 17:18:23,003][105692] Updated weights for policy 0, policy_version 266884 (0.0009) [2023-12-26 17:18:23,091][105620] Updated weights for policy 1, policy_version 266993 (0.0009) [2023-12-26 17:18:23,155][105620] Updated weights for policy 1, policy_version 267003 (0.0010) [2023-12-26 17:18:23,213][105620] Updated weights for policy 1, policy_version 267013 (0.0010) [2023-12-26 17:18:23,831][105692] Updated weights for policy 0, policy_version 266894 (0.0009) [2023-12-26 17:18:23,838][105620] Updated weights for policy 1, policy_version 267023 (0.0010) [2023-12-26 17:18:23,880][105692] Updated weights for policy 0, policy_version 266904 (0.0005) [2023-12-26 17:18:23,886][105620] Updated weights for policy 1, policy_version 267033 (0.0010) [2023-12-26 17:18:23,928][105692] Updated weights for policy 0, policy_version 266914 (0.0005) [2023-12-26 17:18:23,930][105620] Updated weights for policy 1, policy_version 267043 (0.0010) [2023-12-26 17:18:24,579][105692] Updated weights for policy 0, policy_version 266924 (0.0006) [2023-12-26 17:18:24,640][105692] Updated weights for policy 0, policy_version 266934 (0.0010) [2023-12-26 17:18:24,680][105620] Updated weights for policy 1, policy_version 267053 (0.0008) [2023-12-26 17:18:24,691][105692] Updated weights for policy 0, policy_version 266944 (0.0009) [2023-12-26 17:18:24,725][105620] Updated weights for policy 1, policy_version 267063 (0.0005) [2023-12-26 17:18:24,771][105620] Updated weights for policy 1, policy_version 267073 (0.0005) [2023-12-26 17:18:25,354][105620] Updated weights for policy 1, policy_version 267083 (0.0007) [2023-12-26 17:18:25,412][105620] Updated weights for policy 1, policy_version 267093 (0.0010) [2023-12-26 17:18:25,467][105620] Updated weights for policy 1, policy_version 267103 (0.0009) [2023-12-26 17:18:25,527][105692] Updated weights for policy 0, policy_version 266954 (0.0008) [2023-12-26 17:18:25,589][105692] Updated weights for policy 0, policy_version 266964 (0.0009) [2023-12-26 17:18:25,653][105692] Updated weights for policy 0, policy_version 266974 (0.0009) [2023-12-26 17:18:25,716][105692] Updated weights for policy 0, policy_version 266984 (0.0010) [2023-12-26 17:18:26,058][105620] Updated weights for policy 1, policy_version 267113 (0.0008) [2023-12-26 17:18:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 136749056. Throughput: 0: 9734.1, 1: 9861.3. Samples: 136760140. Policy #0 lag: (min: 17.0, avg: 36.0, max: 49.0) [2023-12-26 17:18:26,063][104569] Avg episode reward: [(0, '8532.882'), (1, '9356.383')] [2023-12-26 17:18:26,110][105620] Updated weights for policy 1, policy_version 267123 (0.0005) [2023-12-26 17:18:26,165][105620] Updated weights for policy 1, policy_version 267133 (0.0005) [2023-12-26 17:18:26,219][105620] Updated weights for policy 1, policy_version 267143 (0.0005) [2023-12-26 17:18:26,603][105692] Updated weights for policy 0, policy_version 266994 (0.0008) [2023-12-26 17:18:26,659][105692] Updated weights for policy 0, policy_version 267004 (0.0009) [2023-12-26 17:18:26,714][105692] Updated weights for policy 0, policy_version 267015 (0.0010) [2023-12-26 17:18:26,797][105620] Updated weights for policy 1, policy_version 267153 (0.0006) [2023-12-26 17:18:26,856][105620] Updated weights for policy 1, policy_version 267163 (0.0005) [2023-12-26 17:18:26,910][105620] Updated weights for policy 1, policy_version 267173 (0.0005) [2023-12-26 17:18:27,447][105620] Updated weights for policy 1, policy_version 267183 (0.0007) [2023-12-26 17:18:27,493][105620] Updated weights for policy 1, policy_version 267193 (0.0008) [2023-12-26 17:18:27,542][105620] Updated weights for policy 1, policy_version 267203 (0.0009) [2023-12-26 17:18:27,568][105692] Updated weights for policy 0, policy_version 267025 (0.0007) [2023-12-26 17:18:27,625][105692] Updated weights for policy 0, policy_version 267035 (0.0009) [2023-12-26 17:18:27,679][105692] Updated weights for policy 0, policy_version 267045 (0.0009) [2023-12-26 17:18:28,246][105620] Updated weights for policy 1, policy_version 267213 (0.0006) [2023-12-26 17:18:28,297][105620] Updated weights for policy 1, policy_version 267223 (0.0005) [2023-12-26 17:18:28,365][105620] Updated weights for policy 1, policy_version 267233 (0.0006) [2023-12-26 17:18:28,488][105692] Updated weights for policy 0, policy_version 267055 (0.0009) [2023-12-26 17:18:28,557][105692] Updated weights for policy 0, policy_version 267065 (0.0008) [2023-12-26 17:18:28,620][105692] Updated weights for policy 0, policy_version 267075 (0.0008) [2023-12-26 17:18:29,066][105620] Updated weights for policy 1, policy_version 267243 (0.0009) [2023-12-26 17:18:29,117][105620] Updated weights for policy 1, policy_version 267253 (0.0010) [2023-12-26 17:18:29,164][105620] Updated weights for policy 1, policy_version 267263 (0.0010) [2023-12-26 17:18:29,372][105692] Updated weights for policy 0, policy_version 267085 (0.0008) [2023-12-26 17:18:29,431][105692] Updated weights for policy 0, policy_version 267095 (0.0008) [2023-12-26 17:18:29,492][105692] Updated weights for policy 0, policy_version 267105 (0.0009) [2023-12-26 17:18:29,826][105620] Updated weights for policy 1, policy_version 267273 (0.0010) [2023-12-26 17:18:29,887][105620] Updated weights for policy 1, policy_version 267283 (0.0007) [2023-12-26 17:18:29,950][105620] Updated weights for policy 1, policy_version 267293 (0.0007) [2023-12-26 17:18:30,007][105620] Updated weights for policy 1, policy_version 267303 (0.0005) [2023-12-26 17:18:30,363][105692] Updated weights for policy 0, policy_version 267115 (0.0008) [2023-12-26 17:18:30,420][105692] Updated weights for policy 0, policy_version 267125 (0.0009) [2023-12-26 17:18:30,474][105692] Updated weights for policy 0, policy_version 267135 (0.0009) [2023-12-26 17:18:30,663][105620] Updated weights for policy 1, policy_version 267313 (0.0008) [2023-12-26 17:18:30,716][105620] Updated weights for policy 1, policy_version 267323 (0.0009) [2023-12-26 17:18:30,773][105620] Updated weights for policy 1, policy_version 267333 (0.0009) [2023-12-26 17:18:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 136847360. Throughput: 0: 9624.8, 1: 9984.5. Samples: 136818636. Policy #0 lag: (min: 17.0, avg: 36.0, max: 49.0) [2023-12-26 17:18:31,063][104569] Avg episode reward: [(0, '8624.315'), (1, '9266.159')] [2023-12-26 17:18:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000267144_68403200.pth... [2023-12-26 17:18:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000267336_68444160.pth... [2023-12-26 17:18:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000266056_68124672.pth [2023-12-26 17:18:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000266152_68141056.pth [2023-12-26 17:18:31,230][105692] Updated weights for policy 0, policy_version 267145 (0.0009) [2023-12-26 17:18:31,286][105692] Updated weights for policy 0, policy_version 267155 (0.0008) [2023-12-26 17:18:31,340][105692] Updated weights for policy 0, policy_version 267165 (0.0009) [2023-12-26 17:18:31,407][105692] Updated weights for policy 0, policy_version 267175 (0.0010) [2023-12-26 17:18:31,505][105620] Updated weights for policy 1, policy_version 267343 (0.0008) [2023-12-26 17:18:31,553][105620] Updated weights for policy 1, policy_version 267353 (0.0007) [2023-12-26 17:18:31,614][105620] Updated weights for policy 1, policy_version 267363 (0.0006) [2023-12-26 17:18:32,086][105692] Updated weights for policy 0, policy_version 267185 (0.0006) [2023-12-26 17:18:32,149][105692] Updated weights for policy 0, policy_version 267195 (0.0009) [2023-12-26 17:18:32,211][105692] Updated weights for policy 0, policy_version 267205 (0.0010) [2023-12-26 17:18:32,433][105620] Updated weights for policy 1, policy_version 267373 (0.0009) [2023-12-26 17:18:32,489][105620] Updated weights for policy 1, policy_version 267383 (0.0006) [2023-12-26 17:18:32,549][105620] Updated weights for policy 1, policy_version 267393 (0.0006) [2023-12-26 17:18:32,894][105692] Updated weights for policy 0, policy_version 267215 (0.0010) [2023-12-26 17:18:32,938][105692] Updated weights for policy 0, policy_version 267225 (0.0010) [2023-12-26 17:18:32,988][105692] Updated weights for policy 0, policy_version 267235 (0.0008) [2023-12-26 17:18:33,294][105620] Updated weights for policy 1, policy_version 267403 (0.0006) [2023-12-26 17:18:33,341][105620] Updated weights for policy 1, policy_version 267413 (0.0007) [2023-12-26 17:18:33,385][105620] Updated weights for policy 1, policy_version 267423 (0.0008) [2023-12-26 17:18:33,633][105692] Updated weights for policy 0, policy_version 267245 (0.0008) [2023-12-26 17:18:33,704][105692] Updated weights for policy 0, policy_version 267255 (0.0010) [2023-12-26 17:18:33,755][105692] Updated weights for policy 0, policy_version 267265 (0.0010) [2023-12-26 17:18:34,132][105620] Updated weights for policy 1, policy_version 267433 (0.0008) [2023-12-26 17:18:34,201][105620] Updated weights for policy 1, policy_version 267443 (0.0008) [2023-12-26 17:18:34,265][105620] Updated weights for policy 1, policy_version 267453 (0.0008) [2023-12-26 17:18:34,324][105620] Updated weights for policy 1, policy_version 267463 (0.0008) [2023-12-26 17:18:34,485][105692] Updated weights for policy 0, policy_version 267275 (0.0010) [2023-12-26 17:18:34,537][105692] Updated weights for policy 0, policy_version 267285 (0.0010) [2023-12-26 17:18:34,597][105692] Updated weights for policy 0, policy_version 267295 (0.0010) [2023-12-26 17:18:35,072][105620] Updated weights for policy 1, policy_version 267473 (0.0007) [2023-12-26 17:18:35,121][105620] Updated weights for policy 1, policy_version 267483 (0.0008) [2023-12-26 17:18:35,179][105620] Updated weights for policy 1, policy_version 267493 (0.0008) [2023-12-26 17:18:35,341][105692] Updated weights for policy 0, policy_version 267305 (0.0010) [2023-12-26 17:18:35,392][105692] Updated weights for policy 0, policy_version 267315 (0.0010) [2023-12-26 17:18:35,451][105692] Updated weights for policy 0, policy_version 267325 (0.0011) [2023-12-26 17:18:35,506][105692] Updated weights for policy 0, policy_version 267335 (0.0010) [2023-12-26 17:18:35,952][105620] Updated weights for policy 1, policy_version 267503 (0.0008) [2023-12-26 17:18:36,010][105620] Updated weights for policy 1, policy_version 267513 (0.0008) [2023-12-26 17:18:36,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 136937472. Throughput: 0: 9601.1, 1: 9960.2. Samples: 136933492. Policy #0 lag: (min: 17.0, avg: 36.0, max: 49.0) [2023-12-26 17:18:36,063][104569] Avg episode reward: [(0, '8991.088'), (1, '9354.258')] [2023-12-26 17:18:36,068][105620] Updated weights for policy 1, policy_version 267523 (0.0008) [2023-12-26 17:18:36,263][105692] Updated weights for policy 0, policy_version 267345 (0.0011) [2023-12-26 17:18:36,325][105692] Updated weights for policy 0, policy_version 267355 (0.0011) [2023-12-26 17:18:36,391][105692] Updated weights for policy 0, policy_version 267365 (0.0010) [2023-12-26 17:18:36,825][105620] Updated weights for policy 1, policy_version 267533 (0.0008) [2023-12-26 17:18:36,873][105620] Updated weights for policy 1, policy_version 267543 (0.0008) [2023-12-26 17:18:36,922][105620] Updated weights for policy 1, policy_version 267553 (0.0008) [2023-12-26 17:18:37,126][105692] Updated weights for policy 0, policy_version 267375 (0.0008) [2023-12-26 17:18:37,187][105692] Updated weights for policy 0, policy_version 267385 (0.0005) [2023-12-26 17:18:37,251][105692] Updated weights for policy 0, policy_version 267395 (0.0009) [2023-12-26 17:18:37,752][105620] Updated weights for policy 1, policy_version 267563 (0.0007) [2023-12-26 17:18:37,814][105620] Updated weights for policy 1, policy_version 267573 (0.0009) [2023-12-26 17:18:37,872][105620] Updated weights for policy 1, policy_version 267583 (0.0008) [2023-12-26 17:18:37,881][105692] Updated weights for policy 0, policy_version 267405 (0.0008) [2023-12-26 17:18:37,931][105692] Updated weights for policy 0, policy_version 267415 (0.0011) [2023-12-26 17:18:37,990][105692] Updated weights for policy 0, policy_version 267425 (0.0011) [2023-12-26 17:18:38,557][105620] Updated weights for policy 1, policy_version 267593 (0.0006) [2023-12-26 17:18:38,606][105620] Updated weights for policy 1, policy_version 267603 (0.0008) [2023-12-26 17:18:38,659][105620] Updated weights for policy 1, policy_version 267613 (0.0009) [2023-12-26 17:18:38,716][105620] Updated weights for policy 1, policy_version 267623 (0.0008) [2023-12-26 17:18:38,737][105692] Updated weights for policy 0, policy_version 267435 (0.0011) [2023-12-26 17:18:38,792][105692] Updated weights for policy 0, policy_version 267445 (0.0010) [2023-12-26 17:18:38,855][105692] Updated weights for policy 0, policy_version 267455 (0.0010) [2023-12-26 17:18:39,483][105620] Updated weights for policy 1, policy_version 267633 (0.0008) [2023-12-26 17:18:39,539][105620] Updated weights for policy 1, policy_version 267643 (0.0008) [2023-12-26 17:18:39,571][105692] Updated weights for policy 0, policy_version 267465 (0.0009) [2023-12-26 17:18:39,604][105620] Updated weights for policy 1, policy_version 267653 (0.0007) [2023-12-26 17:18:39,634][105692] Updated weights for policy 0, policy_version 267475 (0.0011) [2023-12-26 17:18:39,691][105692] Updated weights for policy 0, policy_version 267485 (0.0011) [2023-12-26 17:18:39,744][105692] Updated weights for policy 0, policy_version 267495 (0.0010) [2023-12-26 17:18:40,349][105620] Updated weights for policy 1, policy_version 267663 (0.0007) [2023-12-26 17:18:40,407][105620] Updated weights for policy 1, policy_version 267673 (0.0009) [2023-12-26 17:18:40,465][105620] Updated weights for policy 1, policy_version 267683 (0.0007) [2023-12-26 17:18:40,480][105692] Updated weights for policy 0, policy_version 267505 (0.0008) [2023-12-26 17:18:40,539][105692] Updated weights for policy 0, policy_version 267515 (0.0008) [2023-12-26 17:18:40,602][105692] Updated weights for policy 0, policy_version 267525 (0.0008) [2023-12-26 17:18:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 137035776. Throughput: 0: 9574.5, 1: 9864.6. Samples: 137046036. Policy #0 lag: (min: 17.0, avg: 36.0, max: 49.0) [2023-12-26 17:18:41,062][104569] Avg episode reward: [(0, '9082.749'), (1, '9261.973')] [2023-12-26 17:18:41,235][105620] Updated weights for policy 1, policy_version 267693 (0.0008) [2023-12-26 17:18:41,294][105620] Updated weights for policy 1, policy_version 267703 (0.0008) [2023-12-26 17:18:41,349][105620] Updated weights for policy 1, policy_version 267713 (0.0007) [2023-12-26 17:18:41,351][105692] Updated weights for policy 0, policy_version 267535 (0.0008) [2023-12-26 17:18:41,421][105692] Updated weights for policy 0, policy_version 267545 (0.0008) [2023-12-26 17:18:41,480][105692] Updated weights for policy 0, policy_version 267555 (0.0008) [2023-12-26 17:18:42,142][105620] Updated weights for policy 1, policy_version 267723 (0.0009) [2023-12-26 17:18:42,197][105692] Updated weights for policy 0, policy_version 267565 (0.0007) [2023-12-26 17:18:42,203][105620] Updated weights for policy 1, policy_version 267733 (0.0007) [2023-12-26 17:18:42,254][105692] Updated weights for policy 0, policy_version 267575 (0.0008) [2023-12-26 17:18:42,264][105620] Updated weights for policy 1, policy_version 267743 (0.0008) [2023-12-26 17:18:42,315][105692] Updated weights for policy 0, policy_version 267585 (0.0008) [2023-12-26 17:18:42,967][105620] Updated weights for policy 1, policy_version 267753 (0.0009) [2023-12-26 17:18:43,022][105620] Updated weights for policy 1, policy_version 267763 (0.0009) [2023-12-26 17:18:43,079][105620] Updated weights for policy 1, policy_version 267773 (0.0009) [2023-12-26 17:18:43,102][105692] Updated weights for policy 0, policy_version 267595 (0.0008) [2023-12-26 17:18:43,144][105620] Updated weights for policy 1, policy_version 267783 (0.0008) [2023-12-26 17:18:43,158][105692] Updated weights for policy 0, policy_version 267605 (0.0005) [2023-12-26 17:18:43,212][105692] Updated weights for policy 0, policy_version 267615 (0.0006) [2023-12-26 17:18:43,758][105692] Updated weights for policy 0, policy_version 267625 (0.0007) [2023-12-26 17:18:43,813][105692] Updated weights for policy 0, policy_version 267635 (0.0005) [2023-12-26 17:18:43,879][105692] Updated weights for policy 0, policy_version 267645 (0.0005) [2023-12-26 17:18:43,947][105692] Updated weights for policy 0, policy_version 267655 (0.0005) [2023-12-26 17:18:43,971][105620] Updated weights for policy 1, policy_version 267793 (0.0009) [2023-12-26 17:18:44,035][105620] Updated weights for policy 1, policy_version 267803 (0.0009) [2023-12-26 17:18:44,092][105620] Updated weights for policy 1, policy_version 267813 (0.0010) [2023-12-26 17:18:44,550][105692] Updated weights for policy 0, policy_version 267665 (0.0009) [2023-12-26 17:18:44,610][105692] Updated weights for policy 0, policy_version 267675 (0.0009) [2023-12-26 17:18:44,663][105692] Updated weights for policy 0, policy_version 267685 (0.0010) [2023-12-26 17:18:44,838][105620] Updated weights for policy 1, policy_version 267823 (0.0008) [2023-12-26 17:18:44,900][105620] Updated weights for policy 1, policy_version 267833 (0.0007) [2023-12-26 17:18:44,962][105620] Updated weights for policy 1, policy_version 267843 (0.0009) [2023-12-26 17:18:45,463][105692] Updated weights for policy 0, policy_version 267695 (0.0009) [2023-12-26 17:18:45,523][105692] Updated weights for policy 0, policy_version 267705 (0.0009) [2023-12-26 17:18:45,582][105692] Updated weights for policy 0, policy_version 267715 (0.0009) [2023-12-26 17:18:45,691][105620] Updated weights for policy 1, policy_version 267853 (0.0009) [2023-12-26 17:18:45,756][105620] Updated weights for policy 1, policy_version 267863 (0.0009) [2023-12-26 17:18:45,816][105620] Updated weights for policy 1, policy_version 267873 (0.0008) [2023-12-26 17:18:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 137134080. Throughput: 0: 9527.1, 1: 9844.4. Samples: 137101940. Policy #0 lag: (min: 17.0, avg: 36.0, max: 49.0) [2023-12-26 17:18:46,062][104569] Avg episode reward: [(0, '9174.068'), (1, '9170.215')] [2023-12-26 17:18:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000267880_68583424.pth... [2023-12-26 17:18:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000267720_68550656.pth... [2023-12-26 17:18:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000266760_68296704.pth [2023-12-26 17:18:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000266632_68272128.pth [2023-12-26 17:18:46,320][105692] Updated weights for policy 0, policy_version 267725 (0.0009) [2023-12-26 17:18:46,371][105692] Updated weights for policy 0, policy_version 267735 (0.0009) [2023-12-26 17:18:46,423][105692] Updated weights for policy 0, policy_version 267745 (0.0009) [2023-12-26 17:18:46,574][105620] Updated weights for policy 1, policy_version 267883 (0.0009) [2023-12-26 17:18:46,635][105620] Updated weights for policy 1, policy_version 267893 (0.0009) [2023-12-26 17:18:46,688][105620] Updated weights for policy 1, policy_version 267903 (0.0009) [2023-12-26 17:18:47,196][105692] Updated weights for policy 0, policy_version 267755 (0.0010) [2023-12-26 17:18:47,254][105692] Updated weights for policy 0, policy_version 267765 (0.0010) [2023-12-26 17:18:47,306][105692] Updated weights for policy 0, policy_version 267775 (0.0010) [2023-12-26 17:18:47,449][105620] Updated weights for policy 1, policy_version 267913 (0.0009) [2023-12-26 17:18:47,506][105620] Updated weights for policy 1, policy_version 267923 (0.0007) [2023-12-26 17:18:47,567][105620] Updated weights for policy 1, policy_version 267933 (0.0008) [2023-12-26 17:18:47,628][105620] Updated weights for policy 1, policy_version 267943 (0.0007) [2023-12-26 17:18:48,044][105692] Updated weights for policy 0, policy_version 267785 (0.0009) [2023-12-26 17:18:48,092][105692] Updated weights for policy 0, policy_version 267795 (0.0010) [2023-12-26 17:18:48,151][105692] Updated weights for policy 0, policy_version 267805 (0.0010) [2023-12-26 17:18:48,200][105692] Updated weights for policy 0, policy_version 267815 (0.0011) [2023-12-26 17:18:48,368][105620] Updated weights for policy 1, policy_version 267953 (0.0008) [2023-12-26 17:18:48,430][105620] Updated weights for policy 1, policy_version 267963 (0.0005) [2023-12-26 17:18:48,491][105620] Updated weights for policy 1, policy_version 267973 (0.0005) [2023-12-26 17:18:48,982][105692] Updated weights for policy 0, policy_version 267825 (0.0009) [2023-12-26 17:18:49,034][105620] Updated weights for policy 1, policy_version 267983 (0.0006) [2023-12-26 17:18:49,041][105692] Updated weights for policy 0, policy_version 267835 (0.0007) [2023-12-26 17:18:49,092][105620] Updated weights for policy 1, policy_version 267993 (0.0009) [2023-12-26 17:18:49,099][105692] Updated weights for policy 0, policy_version 267845 (0.0007) [2023-12-26 17:18:49,147][105620] Updated weights for policy 1, policy_version 268003 (0.0008) [2023-12-26 17:18:49,794][105620] Updated weights for policy 1, policy_version 268013 (0.0009) [2023-12-26 17:18:49,853][105620] Updated weights for policy 1, policy_version 268023 (0.0009) [2023-12-26 17:18:49,915][105620] Updated weights for policy 1, policy_version 268033 (0.0008) [2023-12-26 17:18:49,950][105692] Updated weights for policy 0, policy_version 267855 (0.0007) [2023-12-26 17:18:50,015][105692] Updated weights for policy 0, policy_version 267865 (0.0009) [2023-12-26 17:18:50,070][105692] Updated weights for policy 0, policy_version 267875 (0.0009) [2023-12-26 17:18:50,673][105620] Updated weights for policy 1, policy_version 268043 (0.0009) [2023-12-26 17:18:50,735][105620] Updated weights for policy 1, policy_version 268053 (0.0009) [2023-12-26 17:18:50,796][105620] Updated weights for policy 1, policy_version 268063 (0.0009) [2023-12-26 17:18:50,818][105692] Updated weights for policy 0, policy_version 267885 (0.0007) [2023-12-26 17:18:50,869][105692] Updated weights for policy 0, policy_version 267895 (0.0007) [2023-12-26 17:18:50,928][105692] Updated weights for policy 0, policy_version 267905 (0.0009) [2023-12-26 17:18:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 137232384. Throughput: 0: 9493.1, 1: 9741.3. Samples: 137217556. Policy #0 lag: (min: 17.0, avg: 36.0, max: 49.0) [2023-12-26 17:18:51,063][104569] Avg episode reward: [(0, '9173.947'), (1, '9078.149')] [2023-12-26 17:18:51,512][105620] Updated weights for policy 1, policy_version 268073 (0.0008) [2023-12-26 17:18:51,567][105620] Updated weights for policy 1, policy_version 268083 (0.0007) [2023-12-26 17:18:51,631][105620] Updated weights for policy 1, policy_version 268093 (0.0008) [2023-12-26 17:18:51,696][105620] Updated weights for policy 1, policy_version 268103 (0.0008) [2023-12-26 17:18:51,815][105692] Updated weights for policy 0, policy_version 267915 (0.0010) [2023-12-26 17:18:51,869][105692] Updated weights for policy 0, policy_version 267925 (0.0009) [2023-12-26 17:18:51,917][105692] Updated weights for policy 0, policy_version 267935 (0.0009) [2023-12-26 17:18:52,380][105620] Updated weights for policy 1, policy_version 268113 (0.0009) [2023-12-26 17:18:52,438][105620] Updated weights for policy 1, policy_version 268123 (0.0008) [2023-12-26 17:18:52,482][105620] Updated weights for policy 1, policy_version 268133 (0.0008) [2023-12-26 17:18:52,734][105692] Updated weights for policy 0, policy_version 267945 (0.0008) [2023-12-26 17:18:52,784][105585] KL-divergence is very high: 157.5534 [2023-12-26 17:18:52,791][105692] Updated weights for policy 0, policy_version 267955 (0.0010) [2023-12-26 17:18:52,831][105585] KL-divergence is very high: 271.0059 [2023-12-26 17:18:52,849][105692] Updated weights for policy 0, policy_version 267965 (0.0010) [2023-12-26 17:18:52,882][105585] KL-divergence is very high: 282.7663 [2023-12-26 17:18:52,910][105692] Updated weights for policy 0, policy_version 267975 (0.0009) [2023-12-26 17:18:53,123][105620] Updated weights for policy 1, policy_version 268143 (0.0009) [2023-12-26 17:18:53,169][105620] Updated weights for policy 1, policy_version 268153 (0.0008) [2023-12-26 17:18:53,220][105620] Updated weights for policy 1, policy_version 268164 (0.0009) [2023-12-26 17:18:53,708][105692] Updated weights for policy 0, policy_version 267985 (0.0009) [2023-12-26 17:18:53,761][105692] Updated weights for policy 0, policy_version 267995 (0.0006) [2023-12-26 17:18:53,813][105692] Updated weights for policy 0, policy_version 268005 (0.0005) [2023-12-26 17:18:54,002][105620] Updated weights for policy 1, policy_version 268174 (0.0009) [2023-12-26 17:18:54,061][105620] Updated weights for policy 1, policy_version 268184 (0.0010) [2023-12-26 17:18:54,123][105620] Updated weights for policy 1, policy_version 268194 (0.0009) [2023-12-26 17:18:54,426][105692] Updated weights for policy 0, policy_version 268015 (0.0008) [2023-12-26 17:18:54,483][105692] Updated weights for policy 0, policy_version 268025 (0.0006) [2023-12-26 17:18:54,551][105692] Updated weights for policy 0, policy_version 268035 (0.0008) [2023-12-26 17:18:54,903][105620] Updated weights for policy 1, policy_version 268204 (0.0010) [2023-12-26 17:18:54,955][105620] Updated weights for policy 1, policy_version 268214 (0.0010) [2023-12-26 17:18:55,007][105620] Updated weights for policy 1, policy_version 268224 (0.0010) [2023-12-26 17:18:55,275][105692] Updated weights for policy 0, policy_version 268045 (0.0009) [2023-12-26 17:18:55,328][105692] Updated weights for policy 0, policy_version 268055 (0.0009) [2023-12-26 17:18:55,376][105692] Updated weights for policy 0, policy_version 268065 (0.0006) [2023-12-26 17:18:55,667][105620] Updated weights for policy 1, policy_version 268234 (0.0009) [2023-12-26 17:18:55,734][105620] Updated weights for policy 1, policy_version 268244 (0.0008) [2023-12-26 17:18:55,790][105620] Updated weights for policy 1, policy_version 268254 (0.0011) [2023-12-26 17:18:55,843][105620] Updated weights for policy 1, policy_version 268264 (0.0011) [2023-12-26 17:18:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 137322496. Throughput: 0: 9412.4, 1: 9630.0. Samples: 137331068. Policy #0 lag: (min: 17.0, avg: 36.0, max: 49.0) [2023-12-26 17:18:56,063][104569] Avg episode reward: [(0, '9267.625'), (1, '9084.003')] [2023-12-26 17:18:56,158][105692] Updated weights for policy 0, policy_version 268075 (0.0006) [2023-12-26 17:18:56,209][105692] Updated weights for policy 0, policy_version 268085 (0.0008) [2023-12-26 17:18:56,274][105692] Updated weights for policy 0, policy_version 268095 (0.0008) [2023-12-26 17:18:56,570][105620] Updated weights for policy 1, policy_version 268274 (0.0010) [2023-12-26 17:18:56,617][105620] Updated weights for policy 1, policy_version 268284 (0.0010) [2023-12-26 17:18:56,665][105620] Updated weights for policy 1, policy_version 268294 (0.0010) [2023-12-26 17:18:56,870][105692] Updated weights for policy 0, policy_version 268105 (0.0007) [2023-12-26 17:18:56,925][105692] Updated weights for policy 0, policy_version 268115 (0.0010) [2023-12-26 17:18:56,969][105692] Updated weights for policy 0, policy_version 268125 (0.0005) [2023-12-26 17:18:57,015][105692] Updated weights for policy 0, policy_version 268135 (0.0007) [2023-12-26 17:18:57,359][105620] Updated weights for policy 1, policy_version 268304 (0.0009) [2023-12-26 17:18:57,415][105620] Updated weights for policy 1, policy_version 268314 (0.0010) [2023-12-26 17:18:57,469][105620] Updated weights for policy 1, policy_version 268324 (0.0010) [2023-12-26 17:18:57,607][105692] Updated weights for policy 0, policy_version 268145 (0.0006) [2023-12-26 17:18:57,670][105692] Updated weights for policy 0, policy_version 268155 (0.0008) [2023-12-26 17:18:57,727][105692] Updated weights for policy 0, policy_version 268165 (0.0009) [2023-12-26 17:18:58,224][105620] Updated weights for policy 1, policy_version 268334 (0.0008) [2023-12-26 17:18:58,287][105620] Updated weights for policy 1, policy_version 268344 (0.0010) [2023-12-26 17:18:58,354][105620] Updated weights for policy 1, policy_version 268354 (0.0010) [2023-12-26 17:18:58,428][105692] Updated weights for policy 0, policy_version 268175 (0.0009) [2023-12-26 17:18:58,486][105692] Updated weights for policy 0, policy_version 268185 (0.0009) [2023-12-26 17:18:58,547][105692] Updated weights for policy 0, policy_version 268195 (0.0008) [2023-12-26 17:18:59,187][105620] Updated weights for policy 1, policy_version 268364 (0.0011) [2023-12-26 17:18:59,254][105620] Updated weights for policy 1, policy_version 268374 (0.0010) [2023-12-26 17:18:59,306][105620] Updated weights for policy 1, policy_version 268384 (0.0009) [2023-12-26 17:18:59,353][105692] Updated weights for policy 0, policy_version 268205 (0.0009) [2023-12-26 17:18:59,413][105692] Updated weights for policy 0, policy_version 268215 (0.0011) [2023-12-26 17:18:59,460][105692] Updated weights for policy 0, policy_version 268225 (0.0008) [2023-12-26 17:18:59,472][105585] KL-divergence is very high: 148.4187 [2023-12-26 17:18:59,478][105585] KL-divergence is very high: 262.1525 [2023-12-26 17:19:00,069][105620] Updated weights for policy 1, policy_version 268394 (0.0010) [2023-12-26 17:19:00,095][105692] Updated weights for policy 0, policy_version 268235 (0.0010) [2023-12-26 17:19:00,125][105620] Updated weights for policy 1, policy_version 268404 (0.0010) [2023-12-26 17:19:00,143][105692] Updated weights for policy 0, policy_version 268245 (0.0008) [2023-12-26 17:19:00,178][105620] Updated weights for policy 1, policy_version 268414 (0.0010) [2023-12-26 17:19:00,194][105692] Updated weights for policy 0, policy_version 268255 (0.0008) [2023-12-26 17:19:00,238][105620] Updated weights for policy 1, policy_version 268424 (0.0011) [2023-12-26 17:19:00,949][105692] Updated weights for policy 0, policy_version 268265 (0.0009) [2023-12-26 17:19:00,981][105620] Updated weights for policy 1, policy_version 268434 (0.0010) [2023-12-26 17:19:01,004][105692] Updated weights for policy 0, policy_version 268275 (0.0010) [2023-12-26 17:19:01,037][105620] Updated weights for policy 1, policy_version 268444 (0.0009) [2023-12-26 17:19:01,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 137412608. Throughput: 0: 9445.3, 1: 9658.5. Samples: 137390868. Policy #0 lag: (min: 17.0, avg: 36.0, max: 49.0) [2023-12-26 17:19:01,062][104569] Avg episode reward: [(0, '9176.337'), (1, '9176.874')] [2023-12-26 17:19:01,062][105692] Updated weights for policy 0, policy_version 268285 (0.0010) [2023-12-26 17:19:01,095][105620] Updated weights for policy 1, policy_version 268454 (0.0011) [2023-12-26 17:19:01,102][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000268456_68730880.pth... [2023-12-26 17:19:01,105][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000267336_68444160.pth [2023-12-26 17:19:01,123][105692] Updated weights for policy 0, policy_version 268295 (0.0010) [2023-12-26 17:19:01,128][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000268296_68698112.pth... [2023-12-26 17:19:01,133][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000267144_68403200.pth [2023-12-26 17:19:01,856][105620] Updated weights for policy 1, policy_version 268464 (0.0007) [2023-12-26 17:19:01,892][105692] Updated weights for policy 0, policy_version 268305 (0.0008) [2023-12-26 17:19:01,919][105620] Updated weights for policy 1, policy_version 268474 (0.0006) [2023-12-26 17:19:01,950][105692] Updated weights for policy 0, policy_version 268315 (0.0010) [2023-12-26 17:19:01,970][105620] Updated weights for policy 1, policy_version 268484 (0.0005) [2023-12-26 17:19:02,019][105692] Updated weights for policy 0, policy_version 268325 (0.0010) [2023-12-26 17:19:02,622][105620] Updated weights for policy 1, policy_version 268494 (0.0007) [2023-12-26 17:19:02,677][105620] Updated weights for policy 1, policy_version 268504 (0.0008) [2023-12-26 17:19:02,739][105620] Updated weights for policy 1, policy_version 268514 (0.0008) [2023-12-26 17:19:02,755][105692] Updated weights for policy 0, policy_version 268335 (0.0011) [2023-12-26 17:19:02,806][105692] Updated weights for policy 0, policy_version 268345 (0.0010) [2023-12-26 17:19:02,871][105692] Updated weights for policy 0, policy_version 268355 (0.0010) [2023-12-26 17:19:03,492][105620] Updated weights for policy 1, policy_version 268525 (0.0009) [2023-12-26 17:19:03,527][105692] Updated weights for policy 0, policy_version 268365 (0.0008) [2023-12-26 17:19:03,541][105620] Updated weights for policy 1, policy_version 268535 (0.0009) [2023-12-26 17:19:03,577][105692] Updated weights for policy 0, policy_version 268375 (0.0005) [2023-12-26 17:19:03,593][105620] Updated weights for policy 1, policy_version 268545 (0.0008) [2023-12-26 17:19:03,629][105692] Updated weights for policy 0, policy_version 268385 (0.0006) [2023-12-26 17:19:04,275][105692] Updated weights for policy 0, policy_version 268395 (0.0007) [2023-12-26 17:19:04,280][105620] Updated weights for policy 1, policy_version 268555 (0.0007) [2023-12-26 17:19:04,338][105692] Updated weights for policy 0, policy_version 268405 (0.0011) [2023-12-26 17:19:04,344][105620] Updated weights for policy 1, policy_version 268565 (0.0008) [2023-12-26 17:19:04,404][105692] Updated weights for policy 0, policy_version 268415 (0.0011) [2023-12-26 17:19:04,410][105620] Updated weights for policy 1, policy_version 268575 (0.0009) [2023-12-26 17:19:05,091][105620] Updated weights for policy 1, policy_version 268585 (0.0007) [2023-12-26 17:19:05,148][105620] Updated weights for policy 1, policy_version 268595 (0.0006) [2023-12-26 17:19:05,160][105692] Updated weights for policy 0, policy_version 268425 (0.0011) [2023-12-26 17:19:05,202][105620] Updated weights for policy 1, policy_version 268605 (0.0008) [2023-12-26 17:19:05,215][105692] Updated weights for policy 0, policy_version 268435 (0.0010) [2023-12-26 17:19:05,232][105585] KL-divergence is very high: 166.8618 [2023-12-26 17:19:05,254][105620] Updated weights for policy 1, policy_version 268615 (0.0006) [2023-12-26 17:19:05,266][105692] Updated weights for policy 0, policy_version 268445 (0.0010) [2023-12-26 17:19:05,271][105585] KL-divergence is very high: 268.7307 [2023-12-26 17:19:05,315][105585] KL-divergence is very high: 249.2448 [2023-12-26 17:19:05,321][105692] Updated weights for policy 0, policy_version 268455 (0.0010) [2023-12-26 17:19:05,933][105620] Updated weights for policy 1, policy_version 268625 (0.0008) [2023-12-26 17:19:05,984][105620] Updated weights for policy 1, policy_version 268635 (0.0008) [2023-12-26 17:19:06,031][105620] Updated weights for policy 1, policy_version 268645 (0.0008) [2023-12-26 17:19:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 137519104. Throughput: 0: 9440.4, 1: 9720.9. Samples: 137507004. Policy #0 lag: (min: 17.0, avg: 36.0, max: 49.0) [2023-12-26 17:19:06,063][104569] Avg episode reward: [(0, '9083.700'), (1, '9356.318')] [2023-12-26 17:19:06,087][105692] Updated weights for policy 0, policy_version 268465 (0.0010) [2023-12-26 17:19:06,155][105692] Updated weights for policy 0, policy_version 268475 (0.0007) [2023-12-26 17:19:06,218][105692] Updated weights for policy 0, policy_version 268485 (0.0006) [2023-12-26 17:19:06,821][105692] Updated weights for policy 0, policy_version 268495 (0.0006) [2023-12-26 17:19:06,877][105692] Updated weights for policy 0, policy_version 268505 (0.0010) [2023-12-26 17:19:06,880][105620] Updated weights for policy 1, policy_version 268655 (0.0006) [2023-12-26 17:19:06,939][105620] Updated weights for policy 1, policy_version 268665 (0.0006) [2023-12-26 17:19:06,941][105692] Updated weights for policy 0, policy_version 268515 (0.0007) [2023-12-26 17:19:06,995][105620] Updated weights for policy 1, policy_version 268675 (0.0005) [2023-12-26 17:19:07,614][105692] Updated weights for policy 0, policy_version 268525 (0.0008) [2023-12-26 17:19:07,639][105620] Updated weights for policy 1, policy_version 268685 (0.0005) [2023-12-26 17:19:07,669][105692] Updated weights for policy 0, policy_version 268535 (0.0005) [2023-12-26 17:19:07,689][105620] Updated weights for policy 1, policy_version 268695 (0.0005) [2023-12-26 17:19:07,723][105692] Updated weights for policy 0, policy_version 268545 (0.0009) [2023-12-26 17:19:07,749][105620] Updated weights for policy 1, policy_version 268705 (0.0005) [2023-12-26 17:19:08,359][105620] Updated weights for policy 1, policy_version 268715 (0.0007) [2023-12-26 17:19:08,409][105692] Updated weights for policy 0, policy_version 268555 (0.0009) [2023-12-26 17:19:08,416][105620] Updated weights for policy 1, policy_version 268725 (0.0009) [2023-12-26 17:19:08,463][105692] Updated weights for policy 0, policy_version 268565 (0.0006) [2023-12-26 17:19:08,472][105620] Updated weights for policy 1, policy_version 268735 (0.0009) [2023-12-26 17:19:08,508][105692] Updated weights for policy 0, policy_version 268575 (0.0006) [2023-12-26 17:19:09,145][105692] Updated weights for policy 0, policy_version 268585 (0.0006) [2023-12-26 17:19:09,197][105620] Updated weights for policy 1, policy_version 268745 (0.0009) [2023-12-26 17:19:09,200][105692] Updated weights for policy 0, policy_version 268595 (0.0007) [2023-12-26 17:19:09,263][105692] Updated weights for policy 0, policy_version 268605 (0.0010) [2023-12-26 17:19:09,263][105620] Updated weights for policy 1, policy_version 268755 (0.0011) [2023-12-26 17:19:09,325][105620] Updated weights for policy 1, policy_version 268765 (0.0011) [2023-12-26 17:19:09,327][105692] Updated weights for policy 0, policy_version 268615 (0.0010) [2023-12-26 17:19:09,400][105620] Updated weights for policy 1, policy_version 268775 (0.0011) [2023-12-26 17:19:10,059][105692] Updated weights for policy 0, policy_version 268625 (0.0008) [2023-12-26 17:19:10,123][105692] Updated weights for policy 0, policy_version 268635 (0.0008) [2023-12-26 17:19:10,137][105620] Updated weights for policy 1, policy_version 268785 (0.0008) [2023-12-26 17:19:10,178][105692] Updated weights for policy 0, policy_version 268645 (0.0006) [2023-12-26 17:19:10,206][105620] Updated weights for policy 1, policy_version 268795 (0.0008) [2023-12-26 17:19:10,273][105620] Updated weights for policy 1, policy_version 268805 (0.0008) [2023-12-26 17:19:10,895][105620] Updated weights for policy 1, policy_version 268815 (0.0007) [2023-12-26 17:19:10,931][105692] Updated weights for policy 0, policy_version 268655 (0.0006) [2023-12-26 17:19:10,946][105620] Updated weights for policy 1, policy_version 268825 (0.0009) [2023-12-26 17:19:10,996][105692] Updated weights for policy 0, policy_version 268665 (0.0006) [2023-12-26 17:19:11,003][105620] Updated weights for policy 1, policy_version 268835 (0.0009) [2023-12-26 17:19:11,059][105692] Updated weights for policy 0, policy_version 268675 (0.0008) [2023-12-26 17:19:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 137617408. Throughput: 0: 9573.0, 1: 9691.8. Samples: 137627052. Policy #0 lag: (min: 17.0, avg: 36.0, max: 49.0) [2023-12-26 17:19:11,063][104569] Avg episode reward: [(0, '9083.729'), (1, '9263.160')] [2023-12-26 17:19:11,778][105692] Updated weights for policy 0, policy_version 268685 (0.0008) [2023-12-26 17:19:11,830][105620] Updated weights for policy 1, policy_version 268845 (0.0007) [2023-12-26 17:19:11,848][105692] Updated weights for policy 0, policy_version 268695 (0.0006) [2023-12-26 17:19:11,893][105620] Updated weights for policy 1, policy_version 268855 (0.0009) [2023-12-26 17:19:11,907][105692] Updated weights for policy 0, policy_version 268705 (0.0005) [2023-12-26 17:19:11,951][105620] Updated weights for policy 1, policy_version 268865 (0.0008) [2023-12-26 17:19:12,508][105692] Updated weights for policy 0, policy_version 268715 (0.0007) [2023-12-26 17:19:12,570][105692] Updated weights for policy 0, policy_version 268725 (0.0011) [2023-12-26 17:19:12,632][105692] Updated weights for policy 0, policy_version 268735 (0.0011) [2023-12-26 17:19:12,803][105620] Updated weights for policy 1, policy_version 268875 (0.0010) [2023-12-26 17:19:12,870][105620] Updated weights for policy 1, policy_version 268885 (0.0011) [2023-12-26 17:19:12,933][105620] Updated weights for policy 1, policy_version 268895 (0.0011) [2023-12-26 17:19:13,382][105692] Updated weights for policy 0, policy_version 268745 (0.0011) [2023-12-26 17:19:13,434][105692] Updated weights for policy 0, policy_version 268755 (0.0011) [2023-12-26 17:19:13,496][105692] Updated weights for policy 0, policy_version 268765 (0.0010) [2023-12-26 17:19:13,552][105692] Updated weights for policy 0, policy_version 268775 (0.0010) [2023-12-26 17:19:13,598][105620] Updated weights for policy 1, policy_version 268905 (0.0010) [2023-12-26 17:19:13,653][105620] Updated weights for policy 1, policy_version 268915 (0.0005) [2023-12-26 17:19:13,704][105620] Updated weights for policy 1, policy_version 268925 (0.0005) [2023-12-26 17:19:13,750][105620] Updated weights for policy 1, policy_version 268935 (0.0005) [2023-12-26 17:19:14,274][105692] Updated weights for policy 0, policy_version 268785 (0.0008) [2023-12-26 17:19:14,323][105692] Updated weights for policy 0, policy_version 268795 (0.0011) [2023-12-26 17:19:14,331][105620] Updated weights for policy 1, policy_version 268945 (0.0010) [2023-12-26 17:19:14,389][105692] Updated weights for policy 0, policy_version 268805 (0.0011) [2023-12-26 17:19:14,390][105620] Updated weights for policy 1, policy_version 268955 (0.0011) [2023-12-26 17:19:14,439][105620] Updated weights for policy 1, policy_version 268965 (0.0010) [2023-12-26 17:19:15,118][105692] Updated weights for policy 0, policy_version 268815 (0.0010) [2023-12-26 17:19:15,172][105692] Updated weights for policy 0, policy_version 268825 (0.0006) [2023-12-26 17:19:15,219][105620] Updated weights for policy 1, policy_version 268975 (0.0007) [2023-12-26 17:19:15,228][105692] Updated weights for policy 0, policy_version 268835 (0.0007) [2023-12-26 17:19:15,261][105586] KL-divergence is very high: 292.3478 [2023-12-26 17:19:15,279][105620] Updated weights for policy 1, policy_version 268985 (0.0008) [2023-12-26 17:19:15,280][105586] KL-divergence is very high: 106.1280 [2023-12-26 17:19:15,307][105586] KL-divergence is very high: 490.6822 [2023-12-26 17:19:15,323][105586] KL-divergence is very high: 141.4019 [2023-12-26 17:19:15,336][105620] Updated weights for policy 1, policy_version 268995 (0.0008) [2023-12-26 17:19:15,352][105586] KL-divergence is very high: 537.9609 [2023-12-26 17:19:15,943][105692] Updated weights for policy 0, policy_version 268845 (0.0008) [2023-12-26 17:19:15,990][105692] Updated weights for policy 0, policy_version 268855 (0.0007) [2023-12-26 17:19:16,042][105692] Updated weights for policy 0, policy_version 268865 (0.0008) [2023-12-26 17:19:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.7, 300 sec: 19438.6). Total num frames: 137707520. Throughput: 0: 9654.5, 1: 9573.6. Samples: 137683900. Policy #0 lag: (min: 9.0, avg: 22.6, max: 41.0) [2023-12-26 17:19:16,063][104569] Avg episode reward: [(0, '9359.180'), (1, '9077.546')] [2023-12-26 17:19:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000269000_68870144.pth... [2023-12-26 17:19:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000267880_68583424.pth [2023-12-26 17:19:16,087][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000268872_68845568.pth... [2023-12-26 17:19:16,091][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000267720_68550656.pth [2023-12-26 17:19:16,116][105620] Updated weights for policy 1, policy_version 269005 (0.0009) [2023-12-26 17:19:16,166][105620] Updated weights for policy 1, policy_version 269015 (0.0008) [2023-12-26 17:19:16,217][105620] Updated weights for policy 1, policy_version 269025 (0.0007) [2023-12-26 17:19:16,725][105692] Updated weights for policy 0, policy_version 268875 (0.0007) [2023-12-26 17:19:16,769][105692] Updated weights for policy 0, policy_version 268885 (0.0007) [2023-12-26 17:19:16,821][105692] Updated weights for policy 0, policy_version 268895 (0.0008) [2023-12-26 17:19:16,995][105620] Updated weights for policy 1, policy_version 269035 (0.0007) [2023-12-26 17:19:17,042][105620] Updated weights for policy 1, policy_version 269045 (0.0010) [2023-12-26 17:19:17,101][105620] Updated weights for policy 1, policy_version 269055 (0.0010) [2023-12-26 17:19:17,541][105692] Updated weights for policy 0, policy_version 268905 (0.0008) [2023-12-26 17:19:17,593][105692] Updated weights for policy 0, policy_version 268915 (0.0009) [2023-12-26 17:19:17,644][105692] Updated weights for policy 0, policy_version 268926 (0.0008) [2023-12-26 17:19:17,676][105620] Updated weights for policy 1, policy_version 269065 (0.0010) [2023-12-26 17:19:17,694][105692] Updated weights for policy 0, policy_version 268936 (0.0007) [2023-12-26 17:19:17,734][105620] Updated weights for policy 1, policy_version 269075 (0.0011) [2023-12-26 17:19:17,789][105620] Updated weights for policy 1, policy_version 269085 (0.0010) [2023-12-26 17:19:17,840][105620] Updated weights for policy 1, policy_version 269095 (0.0010) [2023-12-26 17:19:18,469][105692] Updated weights for policy 0, policy_version 268946 (0.0010) [2023-12-26 17:19:18,527][105692] Updated weights for policy 0, policy_version 268956 (0.0009) [2023-12-26 17:19:18,570][105620] Updated weights for policy 1, policy_version 269105 (0.0009) [2023-12-26 17:19:18,588][105692] Updated weights for policy 0, policy_version 268966 (0.0007) [2023-12-26 17:19:18,633][105620] Updated weights for policy 1, policy_version 269115 (0.0008) [2023-12-26 17:19:18,700][105620] Updated weights for policy 1, policy_version 269125 (0.0009) [2023-12-26 17:19:19,367][105620] Updated weights for policy 1, policy_version 269135 (0.0007) [2023-12-26 17:19:19,420][105692] Updated weights for policy 0, policy_version 268976 (0.0007) [2023-12-26 17:19:19,434][105620] Updated weights for policy 1, policy_version 269145 (0.0008) [2023-12-26 17:19:19,479][105692] Updated weights for policy 0, policy_version 268986 (0.0007) [2023-12-26 17:19:19,500][105620] Updated weights for policy 1, policy_version 269155 (0.0008) [2023-12-26 17:19:19,546][105692] Updated weights for policy 0, policy_version 268996 (0.0006) [2023-12-26 17:19:20,221][105692] Updated weights for policy 0, policy_version 269006 (0.0007) [2023-12-26 17:19:20,279][105692] Updated weights for policy 0, policy_version 269016 (0.0007) [2023-12-26 17:19:20,297][105620] Updated weights for policy 1, policy_version 269165 (0.0009) [2023-12-26 17:19:20,335][105692] Updated weights for policy 0, policy_version 269026 (0.0006) [2023-12-26 17:19:20,356][105620] Updated weights for policy 1, policy_version 269175 (0.0010) [2023-12-26 17:19:20,413][105620] Updated weights for policy 1, policy_version 269185 (0.0010) [2023-12-26 17:19:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 137805824. Throughput: 0: 9662.3, 1: 9589.8. Samples: 137799836. Policy #0 lag: (min: 9.0, avg: 22.6, max: 41.0) [2023-12-26 17:19:21,062][104569] Avg episode reward: [(0, '9086.368'), (1, '8708.139')] [2023-12-26 17:19:21,106][105692] Updated weights for policy 0, policy_version 269036 (0.0007) [2023-12-26 17:19:21,171][105692] Updated weights for policy 0, policy_version 269046 (0.0009) [2023-12-26 17:19:21,216][105620] Updated weights for policy 1, policy_version 269195 (0.0010) [2023-12-26 17:19:21,232][105692] Updated weights for policy 0, policy_version 269056 (0.0008) [2023-12-26 17:19:21,277][105620] Updated weights for policy 1, policy_version 269205 (0.0011) [2023-12-26 17:19:21,334][105620] Updated weights for policy 1, policy_version 269215 (0.0011) [2023-12-26 17:19:22,025][105692] Updated weights for policy 0, policy_version 269066 (0.0006) [2023-12-26 17:19:22,088][105620] Updated weights for policy 1, policy_version 269225 (0.0012) [2023-12-26 17:19:22,090][105692] Updated weights for policy 0, policy_version 269076 (0.0009) [2023-12-26 17:19:22,144][105620] Updated weights for policy 1, policy_version 269235 (0.0008) [2023-12-26 17:19:22,157][105692] Updated weights for policy 0, policy_version 269086 (0.0008) [2023-12-26 17:19:22,201][105620] Updated weights for policy 1, policy_version 269245 (0.0007) [2023-12-26 17:19:22,223][105692] Updated weights for policy 0, policy_version 269096 (0.0010) [2023-12-26 17:19:22,268][105620] Updated weights for policy 1, policy_version 269255 (0.0008) [2023-12-26 17:19:22,943][105692] Updated weights for policy 0, policy_version 269106 (0.0008) [2023-12-26 17:19:22,985][105620] Updated weights for policy 1, policy_version 269265 (0.0006) [2023-12-26 17:19:23,005][105692] Updated weights for policy 0, policy_version 269116 (0.0011) [2023-12-26 17:19:23,046][105620] Updated weights for policy 1, policy_version 269275 (0.0005) [2023-12-26 17:19:23,065][105692] Updated weights for policy 0, policy_version 269126 (0.0011) [2023-12-26 17:19:23,099][105620] Updated weights for policy 1, policy_version 269285 (0.0006) [2023-12-26 17:19:23,764][105692] Updated weights for policy 0, policy_version 269136 (0.0006) [2023-12-26 17:19:23,792][105620] Updated weights for policy 1, policy_version 269295 (0.0005) [2023-12-26 17:19:23,829][105692] Updated weights for policy 0, policy_version 269146 (0.0011) [2023-12-26 17:19:23,856][105620] Updated weights for policy 1, policy_version 269305 (0.0005) [2023-12-26 17:19:23,893][105692] Updated weights for policy 0, policy_version 269156 (0.0010) [2023-12-26 17:19:23,922][105620] Updated weights for policy 1, policy_version 269315 (0.0005) [2023-12-26 17:19:24,544][105692] Updated weights for policy 0, policy_version 269166 (0.0007) [2023-12-26 17:19:24,546][105620] Updated weights for policy 1, policy_version 269325 (0.0007) [2023-12-26 17:19:24,593][105692] Updated weights for policy 0, policy_version 269176 (0.0005) [2023-12-26 17:19:24,594][105620] Updated weights for policy 1, policy_version 269335 (0.0005) [2023-12-26 17:19:24,642][105692] Updated weights for policy 0, policy_version 269186 (0.0005) [2023-12-26 17:19:24,644][105620] Updated weights for policy 1, policy_version 269345 (0.0006) [2023-12-26 17:19:25,172][105692] Updated weights for policy 0, policy_version 269196 (0.0007) [2023-12-26 17:19:25,200][105620] Updated weights for policy 1, policy_version 269355 (0.0008) [2023-12-26 17:19:25,226][105692] Updated weights for policy 0, policy_version 269206 (0.0010) [2023-12-26 17:19:25,260][105620] Updated weights for policy 1, policy_version 269365 (0.0005) [2023-12-26 17:19:25,285][105692] Updated weights for policy 0, policy_version 269216 (0.0011) [2023-12-26 17:19:25,315][105620] Updated weights for policy 1, policy_version 269375 (0.0005) [2023-12-26 17:19:25,994][105692] Updated weights for policy 0, policy_version 269226 (0.0010) [2023-12-26 17:19:26,052][105692] Updated weights for policy 0, policy_version 269236 (0.0010) [2023-12-26 17:19:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.3, 300 sec: 19438.7). Total num frames: 137904128. Throughput: 0: 9705.4, 1: 9669.6. Samples: 137917912. Policy #0 lag: (min: 9.0, avg: 22.6, max: 41.0) [2023-12-26 17:19:26,062][104569] Avg episode reward: [(0, '8996.566'), (1, '8709.439')] [2023-12-26 17:19:26,105][105620] Updated weights for policy 1, policy_version 269385 (0.0008) [2023-12-26 17:19:26,115][105692] Updated weights for policy 0, policy_version 269246 (0.0009) [2023-12-26 17:19:26,165][105620] Updated weights for policy 1, policy_version 269395 (0.0007) [2023-12-26 17:19:26,177][105692] Updated weights for policy 0, policy_version 269256 (0.0005) [2023-12-26 17:19:26,216][105620] Updated weights for policy 1, policy_version 269405 (0.0009) [2023-12-26 17:19:26,269][105620] Updated weights for policy 1, policy_version 269415 (0.0009) [2023-12-26 17:19:26,799][105692] Updated weights for policy 0, policy_version 269266 (0.0005) [2023-12-26 17:19:26,852][105692] Updated weights for policy 0, policy_version 269276 (0.0005) [2023-12-26 17:19:26,900][105692] Updated weights for policy 0, policy_version 269286 (0.0005) [2023-12-26 17:19:27,110][105620] Updated weights for policy 1, policy_version 269426 (0.0010) [2023-12-26 17:19:27,163][105620] Updated weights for policy 1, policy_version 269436 (0.0010) [2023-12-26 17:19:27,218][105620] Updated weights for policy 1, policy_version 269446 (0.0010) [2023-12-26 17:19:27,467][105692] Updated weights for policy 0, policy_version 269296 (0.0009) [2023-12-26 17:19:27,521][105692] Updated weights for policy 0, policy_version 269306 (0.0010) [2023-12-26 17:19:27,579][105692] Updated weights for policy 0, policy_version 269316 (0.0010) [2023-12-26 17:19:27,987][105620] Updated weights for policy 1, policy_version 269456 (0.0010) [2023-12-26 17:19:28,045][105620] Updated weights for policy 1, policy_version 269466 (0.0010) [2023-12-26 17:19:28,099][105620] Updated weights for policy 1, policy_version 269476 (0.0010) [2023-12-26 17:19:28,163][105692] Updated weights for policy 0, policy_version 269326 (0.0007) [2023-12-26 17:19:28,214][105692] Updated weights for policy 0, policy_version 269336 (0.0005) [2023-12-26 17:19:28,270][105692] Updated weights for policy 0, policy_version 269346 (0.0007) [2023-12-26 17:19:28,859][105620] Updated weights for policy 1, policy_version 269486 (0.0010) [2023-12-26 17:19:28,907][105620] Updated weights for policy 1, policy_version 269496 (0.0010) [2023-12-26 17:19:28,959][105620] Updated weights for policy 1, policy_version 269506 (0.0010) [2023-12-26 17:19:28,966][105692] Updated weights for policy 0, policy_version 269356 (0.0007) [2023-12-26 17:19:29,010][105692] Updated weights for policy 0, policy_version 269366 (0.0010) [2023-12-26 17:19:29,061][105692] Updated weights for policy 0, policy_version 269376 (0.0010) [2023-12-26 17:19:29,702][105620] Updated weights for policy 1, policy_version 269516 (0.0010) [2023-12-26 17:19:29,723][105692] Updated weights for policy 0, policy_version 269386 (0.0010) [2023-12-26 17:19:29,750][105620] Updated weights for policy 1, policy_version 269526 (0.0010) [2023-12-26 17:19:29,783][105692] Updated weights for policy 0, policy_version 269396 (0.0010) [2023-12-26 17:19:29,804][105620] Updated weights for policy 1, policy_version 269536 (0.0010) [2023-12-26 17:19:29,838][105692] Updated weights for policy 0, policy_version 269406 (0.0010) [2023-12-26 17:19:29,902][105692] Updated weights for policy 0, policy_version 269416 (0.0011) [2023-12-26 17:19:30,473][105620] Updated weights for policy 1, policy_version 269546 (0.0009) [2023-12-26 17:19:30,510][105692] Updated weights for policy 0, policy_version 269426 (0.0011) [2023-12-26 17:19:30,536][105620] Updated weights for policy 1, policy_version 269556 (0.0011) [2023-12-26 17:19:30,569][105692] Updated weights for policy 0, policy_version 269436 (0.0010) [2023-12-26 17:19:30,595][105620] Updated weights for policy 1, policy_version 269566 (0.0011) [2023-12-26 17:19:30,627][105692] Updated weights for policy 0, policy_version 269446 (0.0010) [2023-12-26 17:19:30,646][105620] Updated weights for policy 1, policy_version 269576 (0.0010) [2023-12-26 17:19:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 138010624. Throughput: 0: 9808.6, 1: 9677.2. Samples: 137978800. Policy #0 lag: (min: 9.0, avg: 22.6, max: 41.0) [2023-12-26 17:19:31,062][104569] Avg episode reward: [(0, '9269.618'), (1, '8896.735')] [2023-12-26 17:19:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000269448_68993024.pth... [2023-12-26 17:19:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000269576_69017600.pth... [2023-12-26 17:19:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000268296_68698112.pth [2023-12-26 17:19:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000268456_68730880.pth [2023-12-26 17:19:31,231][105692] Updated weights for policy 0, policy_version 269456 (0.0010) [2023-12-26 17:19:31,293][105692] Updated weights for policy 0, policy_version 269466 (0.0008) [2023-12-26 17:19:31,314][105620] Updated weights for policy 1, policy_version 269586 (0.0010) [2023-12-26 17:19:31,351][105692] Updated weights for policy 0, policy_version 269476 (0.0008) [2023-12-26 17:19:31,379][105620] Updated weights for policy 1, policy_version 269596 (0.0009) [2023-12-26 17:19:31,432][105620] Updated weights for policy 1, policy_version 269606 (0.0006) [2023-12-26 17:19:32,085][105692] Updated weights for policy 0, policy_version 269486 (0.0010) [2023-12-26 17:19:32,124][105620] Updated weights for policy 1, policy_version 269616 (0.0009) [2023-12-26 17:19:32,137][105692] Updated weights for policy 0, policy_version 269496 (0.0010) [2023-12-26 17:19:32,185][105620] Updated weights for policy 1, policy_version 269626 (0.0010) [2023-12-26 17:19:32,195][105692] Updated weights for policy 0, policy_version 269506 (0.0010) [2023-12-26 17:19:32,247][105620] Updated weights for policy 1, policy_version 269636 (0.0010) [2023-12-26 17:19:32,900][105620] Updated weights for policy 1, policy_version 269646 (0.0007) [2023-12-26 17:19:32,961][105620] Updated weights for policy 1, policy_version 269656 (0.0005) [2023-12-26 17:19:32,984][105692] Updated weights for policy 0, policy_version 269516 (0.0009) [2023-12-26 17:19:33,011][105620] Updated weights for policy 1, policy_version 269666 (0.0010) [2023-12-26 17:19:33,045][105692] Updated weights for policy 0, policy_version 269526 (0.0005) [2023-12-26 17:19:33,111][105692] Updated weights for policy 0, policy_version 269536 (0.0010) [2023-12-26 17:19:33,629][105620] Updated weights for policy 1, policy_version 269676 (0.0010) [2023-12-26 17:19:33,686][105620] Updated weights for policy 1, policy_version 269686 (0.0010) [2023-12-26 17:19:33,726][105692] Updated weights for policy 0, policy_version 269546 (0.0009) [2023-12-26 17:19:33,742][105620] Updated weights for policy 1, policy_version 269696 (0.0010) [2023-12-26 17:19:33,776][105692] Updated weights for policy 0, policy_version 269556 (0.0005) [2023-12-26 17:19:33,837][105692] Updated weights for policy 0, policy_version 269566 (0.0005) [2023-12-26 17:19:33,883][105692] Updated weights for policy 0, policy_version 269576 (0.0009) [2023-12-26 17:19:34,326][105620] Updated weights for policy 1, policy_version 269706 (0.0005) [2023-12-26 17:19:34,386][105620] Updated weights for policy 1, policy_version 269716 (0.0005) [2023-12-26 17:19:34,443][105620] Updated weights for policy 1, policy_version 269726 (0.0006) [2023-12-26 17:19:34,504][105620] Updated weights for policy 1, policy_version 269736 (0.0007) [2023-12-26 17:19:34,591][105692] Updated weights for policy 0, policy_version 269586 (0.0010) [2023-12-26 17:19:34,650][105692] Updated weights for policy 0, policy_version 269596 (0.0010) [2023-12-26 17:19:34,720][105692] Updated weights for policy 0, policy_version 269606 (0.0011) [2023-12-26 17:19:35,061][105620] Updated weights for policy 1, policy_version 269746 (0.0010) [2023-12-26 17:19:35,115][105620] Updated weights for policy 1, policy_version 269756 (0.0010) [2023-12-26 17:19:35,173][105620] Updated weights for policy 1, policy_version 269766 (0.0010) [2023-12-26 17:19:35,389][105692] Updated weights for policy 0, policy_version 269616 (0.0006) [2023-12-26 17:19:35,444][105692] Updated weights for policy 0, policy_version 269626 (0.0005) [2023-12-26 17:19:35,505][105692] Updated weights for policy 0, policy_version 269636 (0.0005) [2023-12-26 17:19:35,889][105620] Updated weights for policy 1, policy_version 269776 (0.0008) [2023-12-26 17:19:35,955][105620] Updated weights for policy 1, policy_version 269786 (0.0008) [2023-12-26 17:19:36,021][105620] Updated weights for policy 1, policy_version 269796 (0.0008) [2023-12-26 17:19:36,062][104569] Fps is (10 sec: 21299.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 138117120. Throughput: 0: 9907.3, 1: 9788.5. Samples: 138103868. Policy #0 lag: (min: 9.0, avg: 22.6, max: 41.0) [2023-12-26 17:19:36,063][104569] Avg episode reward: [(0, '6588.067'), (1, '8804.938')] [2023-12-26 17:19:36,100][105692] Updated weights for policy 0, policy_version 269646 (0.0005) [2023-12-26 17:19:36,124][105585] KL-divergence is very high: 109.9947 [2023-12-26 17:19:36,161][105692] Updated weights for policy 0, policy_version 269656 (0.0007) [2023-12-26 17:19:36,223][105585] KL-divergence is very high: 124.6780 [2023-12-26 17:19:36,231][105692] Updated weights for policy 0, policy_version 269666 (0.0008) [2023-12-26 17:19:36,752][105620] Updated weights for policy 1, policy_version 269806 (0.0009) [2023-12-26 17:19:36,802][105620] Updated weights for policy 1, policy_version 269816 (0.0009) [2023-12-26 17:19:36,849][105620] Updated weights for policy 1, policy_version 269826 (0.0009) [2023-12-26 17:19:36,951][105692] Updated weights for policy 0, policy_version 269676 (0.0009) [2023-12-26 17:19:37,017][105692] Updated weights for policy 0, policy_version 269686 (0.0009) [2023-12-26 17:19:37,079][105692] Updated weights for policy 0, policy_version 269696 (0.0009) [2023-12-26 17:19:37,471][105620] Updated weights for policy 1, policy_version 269836 (0.0009) [2023-12-26 17:19:37,524][105620] Updated weights for policy 1, policy_version 269846 (0.0010) [2023-12-26 17:19:37,576][105620] Updated weights for policy 1, policy_version 269856 (0.0007) [2023-12-26 17:19:37,768][105692] Updated weights for policy 0, policy_version 269706 (0.0009) [2023-12-26 17:19:37,824][105692] Updated weights for policy 0, policy_version 269716 (0.0005) [2023-12-26 17:19:37,882][105692] Updated weights for policy 0, policy_version 269726 (0.0005) [2023-12-26 17:19:37,935][105585] KL-divergence is very high: 114.6064 [2023-12-26 17:19:37,940][105585] KL-divergence is very high: 197.9794 [2023-12-26 17:19:37,946][105585] KL-divergence is very high: 247.0270 [2023-12-26 17:19:37,954][105692] Updated weights for policy 0, policy_version 269736 (0.0006) [2023-12-26 17:19:38,192][105620] Updated weights for policy 1, policy_version 269866 (0.0005) [2023-12-26 17:19:38,243][105620] Updated weights for policy 1, policy_version 269876 (0.0005) [2023-12-26 17:19:38,291][105620] Updated weights for policy 1, policy_version 269886 (0.0005) [2023-12-26 17:19:38,357][105620] Updated weights for policy 1, policy_version 269896 (0.0007) [2023-12-26 17:19:38,449][105585] KL-divergence is very high: 285.2953 [2023-12-26 17:19:38,455][105585] KL-divergence is very high: 308.3803 [2023-12-26 17:19:38,480][105585] KL-divergence is very high: 338.3542 [2023-12-26 17:19:38,491][105585] KL-divergence is very high: 293.2571 [2023-12-26 17:19:38,496][105585] KL-divergence is very high: 240.4829 [2023-12-26 17:19:38,502][105585] KL-divergence is very high: 182.3002 [2023-12-26 17:19:38,509][105692] Updated weights for policy 0, policy_version 269746 (0.0006) [2023-12-26 17:19:38,530][105585] KL-divergence is very high: 231.9022 [2023-12-26 17:19:38,560][105692] Updated weights for policy 0, policy_version 269756 (0.0008) [2023-12-26 17:19:38,565][105585] KL-divergence is very high: 113.0132 [2023-12-26 17:19:38,570][105585] KL-divergence is very high: 398.3958 [2023-12-26 17:19:38,576][105585] KL-divergence is very high: 111.2017 [2023-12-26 17:19:38,614][105692] Updated weights for policy 0, policy_version 269766 (0.0009) [2023-12-26 17:19:38,616][105585] KL-divergence is very high: 189.0270 [2023-12-26 17:19:39,008][105620] Updated weights for policy 1, policy_version 269906 (0.0005) [2023-12-26 17:19:39,074][105620] Updated weights for policy 1, policy_version 269916 (0.0007) [2023-12-26 17:19:39,136][105620] Updated weights for policy 1, policy_version 269926 (0.0009) [2023-12-26 17:19:39,335][105585] KL-divergence is very high: 168.7611 [2023-12-26 17:19:39,386][105692] Updated weights for policy 0, policy_version 269776 (0.0008) [2023-12-26 17:19:39,388][105585] KL-divergence is very high: 108.4551 [2023-12-26 17:19:39,408][105585] KL-divergence is very high: 118.5402 [2023-12-26 17:19:39,447][105585] KL-divergence is very high: 102.1973 [2023-12-26 17:19:39,454][105692] Updated weights for policy 0, policy_version 269786 (0.0008) [2023-12-26 17:19:39,516][105692] Updated weights for policy 0, policy_version 269796 (0.0011) [2023-12-26 17:19:39,848][105620] Updated weights for policy 1, policy_version 269936 (0.0008) [2023-12-26 17:19:39,914][105620] Updated weights for policy 1, policy_version 269946 (0.0008) [2023-12-26 17:19:39,976][105620] Updated weights for policy 1, policy_version 269956 (0.0008) [2023-12-26 17:19:40,266][105692] Updated weights for policy 0, policy_version 269806 (0.0008) [2023-12-26 17:19:40,281][105585] KL-divergence is very high: 115.2420 [2023-12-26 17:19:40,286][105585] KL-divergence is very high: 126.3719 [2023-12-26 17:19:40,322][105692] Updated weights for policy 0, policy_version 269816 (0.0011) [2023-12-26 17:19:40,326][105585] KL-divergence is very high: 154.9861 [2023-12-26 17:19:40,332][105585] KL-divergence is very high: 186.9562 [2023-12-26 17:19:40,339][105585] KL-divergence is very high: 168.0566 [2023-12-26 17:19:40,365][105585] KL-divergence is very high: 106.1364 [2023-12-26 17:19:40,380][105585] KL-divergence is very high: 147.4898 [2023-12-26 17:19:40,386][105692] Updated weights for policy 0, policy_version 269826 (0.0007) [2023-12-26 17:19:40,387][105585] KL-divergence is very high: 143.3497 [2023-12-26 17:19:40,393][105585] KL-divergence is very high: 116.3240 [2023-12-26 17:19:40,762][105620] Updated weights for policy 1, policy_version 269966 (0.0008) [2023-12-26 17:19:40,835][105620] Updated weights for policy 1, policy_version 269976 (0.0009) [2023-12-26 17:19:40,897][105620] Updated weights for policy 1, policy_version 269986 (0.0010) [2023-12-26 17:19:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 138215424. Throughput: 0: 10017.2, 1: 9818.6. Samples: 138223676. Policy #0 lag: (min: 9.0, avg: 22.6, max: 41.0) [2023-12-26 17:19:41,062][104569] Avg episode reward: [(0, '4324.341'), (1, '9171.852')] [2023-12-26 17:19:41,078][105692] Updated weights for policy 0, policy_version 269836 (0.0010) [2023-12-26 17:19:41,146][105692] Updated weights for policy 0, policy_version 269846 (0.0008) [2023-12-26 17:19:41,196][105692] Updated weights for policy 0, policy_version 269856 (0.0006) [2023-12-26 17:19:41,781][105620] Updated weights for policy 1, policy_version 269996 (0.0010) [2023-12-26 17:19:41,839][105620] Updated weights for policy 1, policy_version 270006 (0.0008) [2023-12-26 17:19:41,901][105620] Updated weights for policy 1, policy_version 270016 (0.0009) [2023-12-26 17:19:41,921][105692] Updated weights for policy 0, policy_version 269866 (0.0006) [2023-12-26 17:19:41,981][105692] Updated weights for policy 0, policy_version 269876 (0.0008) [2023-12-26 17:19:42,046][105692] Updated weights for policy 0, policy_version 269886 (0.0008) [2023-12-26 17:19:42,111][105692] Updated weights for policy 0, policy_version 269896 (0.0008) [2023-12-26 17:19:42,659][105620] Updated weights for policy 1, policy_version 270026 (0.0009) [2023-12-26 17:19:42,724][105620] Updated weights for policy 1, policy_version 270036 (0.0010) [2023-12-26 17:19:42,786][105620] Updated weights for policy 1, policy_version 270046 (0.0009) [2023-12-26 17:19:42,810][105692] Updated weights for policy 0, policy_version 269906 (0.0006) [2023-12-26 17:19:42,846][105620] Updated weights for policy 1, policy_version 270056 (0.0006) [2023-12-26 17:19:42,872][105692] Updated weights for policy 0, policy_version 269916 (0.0008) [2023-12-26 17:19:42,928][105692] Updated weights for policy 0, policy_version 269926 (0.0007) [2023-12-26 17:19:43,543][105692] Updated weights for policy 0, policy_version 269936 (0.0008) [2023-12-26 17:19:43,593][105692] Updated weights for policy 0, policy_version 269946 (0.0010) [2023-12-26 17:19:43,623][105620] Updated weights for policy 1, policy_version 270066 (0.0011) [2023-12-26 17:19:43,645][105692] Updated weights for policy 0, policy_version 269956 (0.0010) [2023-12-26 17:19:43,679][105620] Updated weights for policy 1, policy_version 270076 (0.0010) [2023-12-26 17:19:43,740][105620] Updated weights for policy 1, policy_version 270086 (0.0010) [2023-12-26 17:19:44,319][105692] Updated weights for policy 0, policy_version 269966 (0.0010) [2023-12-26 17:19:44,376][105692] Updated weights for policy 0, policy_version 269976 (0.0010) [2023-12-26 17:19:44,432][105692] Updated weights for policy 0, policy_version 269986 (0.0010) [2023-12-26 17:19:44,482][105620] Updated weights for policy 1, policy_version 270096 (0.0011) [2023-12-26 17:19:44,545][105620] Updated weights for policy 1, policy_version 270106 (0.0011) [2023-12-26 17:19:44,615][105620] Updated weights for policy 1, policy_version 270116 (0.0006) [2023-12-26 17:19:45,144][105692] Updated weights for policy 0, policy_version 269996 (0.0010) [2023-12-26 17:19:45,207][105692] Updated weights for policy 0, policy_version 270006 (0.0010) [2023-12-26 17:19:45,272][105692] Updated weights for policy 0, policy_version 270016 (0.0009) [2023-12-26 17:19:45,278][105620] Updated weights for policy 1, policy_version 270126 (0.0009) [2023-12-26 17:19:45,342][105620] Updated weights for policy 1, policy_version 270136 (0.0011) [2023-12-26 17:19:45,406][105620] Updated weights for policy 1, policy_version 270146 (0.0011) [2023-12-26 17:19:45,965][105692] Updated weights for policy 0, policy_version 270026 (0.0011) [2023-12-26 17:19:46,019][105692] Updated weights for policy 0, policy_version 270036 (0.0009) [2023-12-26 17:19:46,027][105620] Updated weights for policy 1, policy_version 270156 (0.0009) [2023-12-26 17:19:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 138305536. Throughput: 0: 9987.1, 1: 9779.8. Samples: 138280380. Policy #0 lag: (min: 9.0, avg: 22.6, max: 41.0) [2023-12-26 17:19:46,062][104569] Avg episode reward: [(0, '7409.561'), (1, '9078.219')] [2023-12-26 17:19:46,077][105692] Updated weights for policy 0, policy_version 270046 (0.0010) [2023-12-26 17:19:46,091][105620] Updated weights for policy 1, policy_version 270166 (0.0005) [2023-12-26 17:19:46,132][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000270056_69148672.pth... [2023-12-26 17:19:46,133][105692] Updated weights for policy 0, policy_version 270056 (0.0010) [2023-12-26 17:19:46,136][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000268872_68845568.pth [2023-12-26 17:19:46,149][105620] Updated weights for policy 1, policy_version 270176 (0.0006) [2023-12-26 17:19:46,197][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000270184_69173248.pth... [2023-12-26 17:19:46,202][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000269000_68870144.pth [2023-12-26 17:19:46,826][105620] Updated weights for policy 1, policy_version 270186 (0.0009) [2023-12-26 17:19:46,864][105692] Updated weights for policy 0, policy_version 270066 (0.0006) [2023-12-26 17:19:46,879][105620] Updated weights for policy 1, policy_version 270196 (0.0010) [2023-12-26 17:19:46,931][105692] Updated weights for policy 0, policy_version 270076 (0.0005) [2023-12-26 17:19:46,943][105620] Updated weights for policy 1, policy_version 270206 (0.0005) [2023-12-26 17:19:46,995][105692] Updated weights for policy 0, policy_version 270086 (0.0007) [2023-12-26 17:19:47,000][105620] Updated weights for policy 1, policy_version 270216 (0.0005) [2023-12-26 17:19:47,633][105692] Updated weights for policy 0, policy_version 270096 (0.0010) [2023-12-26 17:19:47,664][105620] Updated weights for policy 1, policy_version 270226 (0.0005) [2023-12-26 17:19:47,685][105692] Updated weights for policy 0, policy_version 270106 (0.0010) [2023-12-26 17:19:47,724][105620] Updated weights for policy 1, policy_version 270236 (0.0007) [2023-12-26 17:19:47,729][105692] Updated weights for policy 0, policy_version 270116 (0.0009) [2023-12-26 17:19:47,786][105620] Updated weights for policy 1, policy_version 270246 (0.0011) [2023-12-26 17:19:48,396][105692] Updated weights for policy 0, policy_version 270126 (0.0009) [2023-12-26 17:19:48,451][105692] Updated weights for policy 0, policy_version 270136 (0.0009) [2023-12-26 17:19:48,467][105620] Updated weights for policy 1, policy_version 270256 (0.0011) [2023-12-26 17:19:48,507][105692] Updated weights for policy 0, policy_version 270146 (0.0011) [2023-12-26 17:19:48,523][105620] Updated weights for policy 1, policy_version 270266 (0.0011) [2023-12-26 17:19:48,579][105620] Updated weights for policy 1, policy_version 270276 (0.0011) [2023-12-26 17:19:49,203][105692] Updated weights for policy 0, policy_version 270156 (0.0011) [2023-12-26 17:19:49,274][105692] Updated weights for policy 0, policy_version 270166 (0.0009) [2023-12-26 17:19:49,334][105692] Updated weights for policy 0, policy_version 270176 (0.0008) [2023-12-26 17:19:49,350][105620] Updated weights for policy 1, policy_version 270286 (0.0008) [2023-12-26 17:19:49,410][105620] Updated weights for policy 1, policy_version 270296 (0.0007) [2023-12-26 17:19:49,468][105620] Updated weights for policy 1, policy_version 270306 (0.0006) [2023-12-26 17:19:50,059][105620] Updated weights for policy 1, policy_version 270316 (0.0008) [2023-12-26 17:19:50,110][105620] Updated weights for policy 1, policy_version 270326 (0.0009) [2023-12-26 17:19:50,170][105620] Updated weights for policy 1, policy_version 270336 (0.0008) [2023-12-26 17:19:50,182][105692] Updated weights for policy 0, policy_version 270186 (0.0008) [2023-12-26 17:19:50,241][105692] Updated weights for policy 0, policy_version 270196 (0.0009) [2023-12-26 17:19:50,300][105692] Updated weights for policy 0, policy_version 270206 (0.0010) [2023-12-26 17:19:50,366][105692] Updated weights for policy 0, policy_version 270216 (0.0010) [2023-12-26 17:19:50,810][105620] Updated weights for policy 1, policy_version 270346 (0.0006) [2023-12-26 17:19:50,868][105620] Updated weights for policy 1, policy_version 270356 (0.0009) [2023-12-26 17:19:50,927][105620] Updated weights for policy 1, policy_version 270366 (0.0009) [2023-12-26 17:19:50,987][105620] Updated weights for policy 1, policy_version 270376 (0.0008) [2023-12-26 17:19:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 138412032. Throughput: 0: 10024.0, 1: 9853.9. Samples: 138401512. Policy #0 lag: (min: 9.0, avg: 22.6, max: 41.0) [2023-12-26 17:19:51,063][104569] Avg episode reward: [(0, '8857.312'), (1, '8803.291')] [2023-12-26 17:19:51,188][105692] Updated weights for policy 0, policy_version 270226 (0.0008) [2023-12-26 17:19:51,249][105692] Updated weights for policy 0, policy_version 270236 (0.0009) [2023-12-26 17:19:51,301][105692] Updated weights for policy 0, policy_version 270246 (0.0009) [2023-12-26 17:19:51,788][105620] Updated weights for policy 1, policy_version 270386 (0.0011) [2023-12-26 17:19:51,851][105620] Updated weights for policy 1, policy_version 270396 (0.0011) [2023-12-26 17:19:51,913][105620] Updated weights for policy 1, policy_version 270406 (0.0010) [2023-12-26 17:19:52,084][105692] Updated weights for policy 0, policy_version 270256 (0.0006) [2023-12-26 17:19:52,140][105692] Updated weights for policy 0, policy_version 270266 (0.0006) [2023-12-26 17:19:52,205][105692] Updated weights for policy 0, policy_version 270276 (0.0005) [2023-12-26 17:19:52,636][105620] Updated weights for policy 1, policy_version 270416 (0.0007) [2023-12-26 17:19:52,704][105620] Updated weights for policy 1, policy_version 270426 (0.0009) [2023-12-26 17:19:52,772][105620] Updated weights for policy 1, policy_version 270436 (0.0008) [2023-12-26 17:19:52,871][105692] Updated weights for policy 0, policy_version 270286 (0.0007) [2023-12-26 17:19:52,929][105692] Updated weights for policy 0, policy_version 270296 (0.0007) [2023-12-26 17:19:52,993][105692] Updated weights for policy 0, policy_version 270306 (0.0006) [2023-12-26 17:19:53,450][105620] Updated weights for policy 1, policy_version 270446 (0.0007) [2023-12-26 17:19:53,501][105620] Updated weights for policy 1, policy_version 270456 (0.0005) [2023-12-26 17:19:53,557][105620] Updated weights for policy 1, policy_version 270466 (0.0005) [2023-12-26 17:19:53,718][105692] Updated weights for policy 0, policy_version 270316 (0.0008) [2023-12-26 17:19:53,784][105692] Updated weights for policy 0, policy_version 270326 (0.0010) [2023-12-26 17:19:53,845][105692] Updated weights for policy 0, policy_version 270336 (0.0010) [2023-12-26 17:19:54,252][105620] Updated weights for policy 1, policy_version 270476 (0.0010) [2023-12-26 17:19:54,308][105620] Updated weights for policy 1, policy_version 270486 (0.0010) [2023-12-26 17:19:54,360][105620] Updated weights for policy 1, policy_version 270496 (0.0010) [2023-12-26 17:19:54,456][105692] Updated weights for policy 0, policy_version 270346 (0.0010) [2023-12-26 17:19:54,520][105692] Updated weights for policy 0, policy_version 270356 (0.0008) [2023-12-26 17:19:54,575][105692] Updated weights for policy 0, policy_version 270366 (0.0010) [2023-12-26 17:19:54,631][105692] Updated weights for policy 0, policy_version 270376 (0.0010) [2023-12-26 17:19:55,038][105620] Updated weights for policy 1, policy_version 270506 (0.0010) [2023-12-26 17:19:55,090][105620] Updated weights for policy 1, policy_version 270516 (0.0010) [2023-12-26 17:19:55,145][105620] Updated weights for policy 1, policy_version 270526 (0.0010) [2023-12-26 17:19:55,200][105620] Updated weights for policy 1, policy_version 270536 (0.0010) [2023-12-26 17:19:55,397][105692] Updated weights for policy 0, policy_version 270386 (0.0011) [2023-12-26 17:19:55,455][105692] Updated weights for policy 0, policy_version 270396 (0.0010) [2023-12-26 17:19:55,516][105692] Updated weights for policy 0, policy_version 270406 (0.0006) [2023-12-26 17:19:55,958][105620] Updated weights for policy 1, policy_version 270546 (0.0010) [2023-12-26 17:19:56,009][105620] Updated weights for policy 1, policy_version 270556 (0.0010) [2023-12-26 17:19:56,058][105620] Updated weights for policy 1, policy_version 270566 (0.0010) [2023-12-26 17:19:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 138502144. Throughput: 0: 9940.0, 1: 9823.8. Samples: 138516420. Policy #0 lag: (min: 9.0, avg: 22.6, max: 41.0) [2023-12-26 17:19:56,062][104569] Avg episode reward: [(0, '9267.718'), (1, '9080.676')] [2023-12-26 17:19:56,216][105692] Updated weights for policy 0, policy_version 270416 (0.0010) [2023-12-26 17:19:56,267][105692] Updated weights for policy 0, policy_version 270426 (0.0010) [2023-12-26 17:19:56,319][105692] Updated weights for policy 0, policy_version 270436 (0.0010) [2023-12-26 17:19:56,746][105620] Updated weights for policy 1, policy_version 270576 (0.0007) [2023-12-26 17:19:56,800][105620] Updated weights for policy 1, policy_version 270586 (0.0007) [2023-12-26 17:19:56,860][105620] Updated weights for policy 1, policy_version 270596 (0.0009) [2023-12-26 17:19:57,095][105692] Updated weights for policy 0, policy_version 270446 (0.0010) [2023-12-26 17:19:57,166][105692] Updated weights for policy 0, policy_version 270456 (0.0010) [2023-12-26 17:19:57,214][105692] Updated weights for policy 0, policy_version 270466 (0.0010) [2023-12-26 17:19:57,485][105620] Updated weights for policy 1, policy_version 270606 (0.0006) [2023-12-26 17:19:57,549][105620] Updated weights for policy 1, policy_version 270616 (0.0007) [2023-12-26 17:19:57,612][105620] Updated weights for policy 1, policy_version 270626 (0.0005) [2023-12-26 17:19:57,873][105692] Updated weights for policy 0, policy_version 270476 (0.0010) [2023-12-26 17:19:57,933][105692] Updated weights for policy 0, policy_version 270486 (0.0010) [2023-12-26 17:19:57,981][105692] Updated weights for policy 0, policy_version 270496 (0.0010) [2023-12-26 17:19:58,270][105620] Updated weights for policy 1, policy_version 270636 (0.0007) [2023-12-26 17:19:58,355][105620] Updated weights for policy 1, policy_version 270646 (0.0008) [2023-12-26 17:19:58,419][105620] Updated weights for policy 1, policy_version 270656 (0.0008) [2023-12-26 17:19:58,722][105692] Updated weights for policy 0, policy_version 270506 (0.0010) [2023-12-26 17:19:58,788][105692] Updated weights for policy 0, policy_version 270516 (0.0010) [2023-12-26 17:19:58,852][105692] Updated weights for policy 0, policy_version 270526 (0.0011) [2023-12-26 17:19:58,909][105692] Updated weights for policy 0, policy_version 270536 (0.0011) [2023-12-26 17:19:59,107][105620] Updated weights for policy 1, policy_version 270666 (0.0008) [2023-12-26 17:19:59,162][105620] Updated weights for policy 1, policy_version 270676 (0.0005) [2023-12-26 17:19:59,238][105620] Updated weights for policy 1, policy_version 270686 (0.0007) [2023-12-26 17:19:59,304][105620] Updated weights for policy 1, policy_version 270696 (0.0008) [2023-12-26 17:19:59,584][105692] Updated weights for policy 0, policy_version 270546 (0.0010) [2023-12-26 17:19:59,633][105692] Updated weights for policy 0, policy_version 270556 (0.0009) [2023-12-26 17:19:59,685][105692] Updated weights for policy 0, policy_version 270566 (0.0007) [2023-12-26 17:19:59,950][105586] KL-divergence is very high: 102.4979 [2023-12-26 17:20:00,012][105620] Updated weights for policy 1, policy_version 270706 (0.0006) [2023-12-26 17:20:00,075][105620] Updated weights for policy 1, policy_version 270716 (0.0009) [2023-12-26 17:20:00,133][105620] Updated weights for policy 1, policy_version 270726 (0.0009) [2023-12-26 17:20:00,403][105692] Updated weights for policy 0, policy_version 270576 (0.0006) [2023-12-26 17:20:00,467][105692] Updated weights for policy 0, policy_version 270586 (0.0006) [2023-12-26 17:20:00,523][105692] Updated weights for policy 0, policy_version 270596 (0.0005) [2023-12-26 17:20:00,935][105620] Updated weights for policy 1, policy_version 270736 (0.0009) [2023-12-26 17:20:00,989][105620] Updated weights for policy 1, policy_version 270746 (0.0010) [2023-12-26 17:20:01,049][105620] Updated weights for policy 1, policy_version 270756 (0.0008) [2023-12-26 17:20:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 138600448. Throughput: 0: 9942.8, 1: 9875.8. Samples: 138575736. Policy #0 lag: (min: 9.0, avg: 22.6, max: 41.0) [2023-12-26 17:20:01,063][104569] Avg episode reward: [(0, '9267.973'), (1, '8898.627')] [2023-12-26 17:20:01,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000270760_69320704.pth... [2023-12-26 17:20:01,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000269576_69017600.pth [2023-12-26 17:20:01,079][105692] Updated weights for policy 0, policy_version 270606 (0.0006) [2023-12-26 17:20:01,145][105692] Updated weights for policy 0, policy_version 270616 (0.0011) [2023-12-26 17:20:01,197][105692] Updated weights for policy 0, policy_version 270626 (0.0010) [2023-12-26 17:20:01,223][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000270632_69296128.pth... [2023-12-26 17:20:01,226][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000269448_68993024.pth [2023-12-26 17:20:01,809][105620] Updated weights for policy 1, policy_version 270766 (0.0008) [2023-12-26 17:20:01,861][105620] Updated weights for policy 1, policy_version 270776 (0.0008) [2023-12-26 17:20:01,918][105620] Updated weights for policy 1, policy_version 270786 (0.0008) [2023-12-26 17:20:01,965][105692] Updated weights for policy 0, policy_version 270636 (0.0010) [2023-12-26 17:20:02,017][105692] Updated weights for policy 0, policy_version 270646 (0.0010) [2023-12-26 17:20:02,081][105692] Updated weights for policy 0, policy_version 270656 (0.0010) [2023-12-26 17:20:02,716][105620] Updated weights for policy 1, policy_version 270796 (0.0008) [2023-12-26 17:20:02,764][105620] Updated weights for policy 1, policy_version 270806 (0.0008) [2023-12-26 17:20:02,821][105620] Updated weights for policy 1, policy_version 270816 (0.0008) [2023-12-26 17:20:02,843][105692] Updated weights for policy 0, policy_version 270666 (0.0011) [2023-12-26 17:20:02,904][105692] Updated weights for policy 0, policy_version 270676 (0.0010) [2023-12-26 17:20:02,968][105692] Updated weights for policy 0, policy_version 270686 (0.0010) [2023-12-26 17:20:03,022][105692] Updated weights for policy 0, policy_version 270696 (0.0010) [2023-12-26 17:20:03,583][105620] Updated weights for policy 1, policy_version 270826 (0.0006) [2023-12-26 17:20:03,630][105620] Updated weights for policy 1, policy_version 270836 (0.0010) [2023-12-26 17:20:03,681][105620] Updated weights for policy 1, policy_version 270846 (0.0010) [2023-12-26 17:20:03,740][105620] Updated weights for policy 1, policy_version 270856 (0.0009) [2023-12-26 17:20:03,742][105692] Updated weights for policy 0, policy_version 270706 (0.0008) [2023-12-26 17:20:03,794][105692] Updated weights for policy 0, policy_version 270716 (0.0008) [2023-12-26 17:20:03,850][105692] Updated weights for policy 0, policy_version 270726 (0.0008) [2023-12-26 17:20:04,528][105620] Updated weights for policy 1, policy_version 270866 (0.0008) [2023-12-26 17:20:04,592][105620] Updated weights for policy 1, policy_version 270876 (0.0008) [2023-12-26 17:20:04,633][105692] Updated weights for policy 0, policy_version 270736 (0.0010) [2023-12-26 17:20:04,647][105620] Updated weights for policy 1, policy_version 270886 (0.0006) [2023-12-26 17:20:04,685][105692] Updated weights for policy 0, policy_version 270746 (0.0010) [2023-12-26 17:20:04,733][105692] Updated weights for policy 0, policy_version 270756 (0.0010) [2023-12-26 17:20:05,357][105692] Updated weights for policy 0, policy_version 270766 (0.0009) [2023-12-26 17:20:05,365][105620] Updated weights for policy 1, policy_version 270896 (0.0008) [2023-12-26 17:20:05,416][105692] Updated weights for policy 0, policy_version 270776 (0.0005) [2023-12-26 17:20:05,432][105620] Updated weights for policy 1, policy_version 270906 (0.0008) [2023-12-26 17:20:05,486][105692] Updated weights for policy 0, policy_version 270786 (0.0006) [2023-12-26 17:20:05,493][105620] Updated weights for policy 1, policy_version 270916 (0.0008) [2023-12-26 17:20:06,026][105692] Updated weights for policy 0, policy_version 270796 (0.0008) [2023-12-26 17:20:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 138698752. Throughput: 0: 9967.7, 1: 9835.7. Samples: 138690992. Policy #0 lag: (min: 9.0, avg: 22.6, max: 41.0) [2023-12-26 17:20:06,062][104569] Avg episode reward: [(0, '9358.860'), (1, '8807.699')] [2023-12-26 17:20:06,081][105692] Updated weights for policy 0, policy_version 270806 (0.0005) [2023-12-26 17:20:06,142][105692] Updated weights for policy 0, policy_version 270816 (0.0009) [2023-12-26 17:20:06,332][105620] Updated weights for policy 1, policy_version 270926 (0.0008) [2023-12-26 17:20:06,389][105620] Updated weights for policy 1, policy_version 270936 (0.0008) [2023-12-26 17:20:06,441][105620] Updated weights for policy 1, policy_version 270946 (0.0008) [2023-12-26 17:20:06,904][105692] Updated weights for policy 0, policy_version 270826 (0.0010) [2023-12-26 17:20:06,959][105692] Updated weights for policy 0, policy_version 270836 (0.0010) [2023-12-26 17:20:07,021][105692] Updated weights for policy 0, policy_version 270846 (0.0010) [2023-12-26 17:20:07,080][105692] Updated weights for policy 0, policy_version 270856 (0.0010) [2023-12-26 17:20:07,219][105620] Updated weights for policy 1, policy_version 270956 (0.0009) [2023-12-26 17:20:07,274][105620] Updated weights for policy 1, policy_version 270966 (0.0009) [2023-12-26 17:20:07,333][105620] Updated weights for policy 1, policy_version 270976 (0.0009) [2023-12-26 17:20:07,774][105692] Updated weights for policy 0, policy_version 270866 (0.0009) [2023-12-26 17:20:07,824][105692] Updated weights for policy 0, policy_version 270876 (0.0008) [2023-12-26 17:20:07,874][105692] Updated weights for policy 0, policy_version 270886 (0.0006) [2023-12-26 17:20:08,108][105620] Updated weights for policy 1, policy_version 270986 (0.0010) [2023-12-26 17:20:08,162][105620] Updated weights for policy 1, policy_version 270996 (0.0009) [2023-12-26 17:20:08,220][105620] Updated weights for policy 1, policy_version 271006 (0.0009) [2023-12-26 17:20:08,280][105620] Updated weights for policy 1, policy_version 271016 (0.0009) [2023-12-26 17:20:08,508][105692] Updated weights for policy 0, policy_version 270896 (0.0008) [2023-12-26 17:20:08,556][105692] Updated weights for policy 0, policy_version 270906 (0.0009) [2023-12-26 17:20:08,611][105692] Updated weights for policy 0, policy_version 270916 (0.0009) [2023-12-26 17:20:09,107][105620] Updated weights for policy 1, policy_version 271026 (0.0009) [2023-12-26 17:20:09,173][105620] Updated weights for policy 1, policy_version 271036 (0.0008) [2023-12-26 17:20:09,241][105620] Updated weights for policy 1, policy_version 271046 (0.0009) [2023-12-26 17:20:09,355][105692] Updated weights for policy 0, policy_version 270926 (0.0009) [2023-12-26 17:20:09,420][105692] Updated weights for policy 0, policy_version 270936 (0.0008) [2023-12-26 17:20:09,486][105692] Updated weights for policy 0, policy_version 270946 (0.0006) [2023-12-26 17:20:10,040][105620] Updated weights for policy 1, policy_version 271056 (0.0007) [2023-12-26 17:20:10,109][105620] Updated weights for policy 1, policy_version 271066 (0.0008) [2023-12-26 17:20:10,139][105692] Updated weights for policy 0, policy_version 270956 (0.0008) [2023-12-26 17:20:10,181][105620] Updated weights for policy 1, policy_version 271076 (0.0008) [2023-12-26 17:20:10,198][105692] Updated weights for policy 0, policy_version 270966 (0.0006) [2023-12-26 17:20:10,261][105692] Updated weights for policy 0, policy_version 270976 (0.0006) [2023-12-26 17:20:10,282][105585] KL-divergence is very high: 123.8894 [2023-12-26 17:20:10,935][105692] Updated weights for policy 0, policy_version 270986 (0.0008) [2023-12-26 17:20:10,957][105620] Updated weights for policy 1, policy_version 271086 (0.0008) [2023-12-26 17:20:11,004][105692] Updated weights for policy 0, policy_version 270996 (0.0008) [2023-12-26 17:20:11,022][105620] Updated weights for policy 1, policy_version 271096 (0.0008) [2023-12-26 17:20:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 138788864. Throughput: 0: 10034.6, 1: 9700.2. Samples: 138805976. Policy #0 lag: (min: 9.0, avg: 22.6, max: 41.0) [2023-12-26 17:20:11,063][104569] Avg episode reward: [(0, '9231.110'), (1, '8987.550')] [2023-12-26 17:20:11,072][105692] Updated weights for policy 0, policy_version 271006 (0.0010) [2023-12-26 17:20:11,092][105620] Updated weights for policy 1, policy_version 271106 (0.0009) [2023-12-26 17:20:11,129][105692] Updated weights for policy 0, policy_version 271016 (0.0010) [2023-12-26 17:20:11,829][105692] Updated weights for policy 0, policy_version 271026 (0.0010) [2023-12-26 17:20:11,882][105692] Updated weights for policy 0, policy_version 271036 (0.0010) [2023-12-26 17:20:11,902][105620] Updated weights for policy 1, policy_version 271116 (0.0009) [2023-12-26 17:20:11,940][105692] Updated weights for policy 0, policy_version 271046 (0.0011) [2023-12-26 17:20:11,969][105620] Updated weights for policy 1, policy_version 271126 (0.0006) [2023-12-26 17:20:12,034][105620] Updated weights for policy 1, policy_version 271136 (0.0008) [2023-12-26 17:20:12,698][105620] Updated weights for policy 1, policy_version 271146 (0.0008) [2023-12-26 17:20:12,721][105692] Updated weights for policy 0, policy_version 271056 (0.0011) [2023-12-26 17:20:12,762][105620] Updated weights for policy 1, policy_version 271156 (0.0009) [2023-12-26 17:20:12,779][105692] Updated weights for policy 0, policy_version 271066 (0.0011) [2023-12-26 17:20:12,829][105620] Updated weights for policy 1, policy_version 271166 (0.0006) [2023-12-26 17:20:12,842][105692] Updated weights for policy 0, policy_version 271076 (0.0011) [2023-12-26 17:20:12,889][105620] Updated weights for policy 1, policy_version 271176 (0.0006) [2023-12-26 17:20:13,460][105692] Updated weights for policy 0, policy_version 271086 (0.0009) [2023-12-26 17:20:13,527][105692] Updated weights for policy 0, policy_version 271096 (0.0005) [2023-12-26 17:20:13,581][105692] Updated weights for policy 0, policy_version 271106 (0.0008) [2023-12-26 17:20:13,640][105620] Updated weights for policy 1, policy_version 271186 (0.0009) [2023-12-26 17:20:13,694][105620] Updated weights for policy 1, policy_version 271197 (0.0010) [2023-12-26 17:20:13,747][105620] Updated weights for policy 1, policy_version 271208 (0.0010) [2023-12-26 17:20:14,174][105692] Updated weights for policy 0, policy_version 271116 (0.0006) [2023-12-26 17:20:14,232][105692] Updated weights for policy 0, policy_version 271126 (0.0005) [2023-12-26 17:20:14,285][105692] Updated weights for policy 0, policy_version 271136 (0.0009) [2023-12-26 17:20:14,592][105620] Updated weights for policy 1, policy_version 271218 (0.0008) [2023-12-26 17:20:14,643][105620] Updated weights for policy 1, policy_version 271228 (0.0008) [2023-12-26 17:20:14,708][105620] Updated weights for policy 1, policy_version 271238 (0.0008) [2023-12-26 17:20:14,989][105692] Updated weights for policy 0, policy_version 271146 (0.0010) [2023-12-26 17:20:15,042][105692] Updated weights for policy 0, policy_version 271156 (0.0010) [2023-12-26 17:20:15,099][105692] Updated weights for policy 0, policy_version 271166 (0.0011) [2023-12-26 17:20:15,164][105692] Updated weights for policy 0, policy_version 271176 (0.0011) [2023-12-26 17:20:15,481][105620] Updated weights for policy 1, policy_version 271248 (0.0008) [2023-12-26 17:20:15,536][105620] Updated weights for policy 1, policy_version 271258 (0.0008) [2023-12-26 17:20:15,580][105620] Updated weights for policy 1, policy_version 271268 (0.0008) [2023-12-26 17:20:15,918][105692] Updated weights for policy 0, policy_version 271186 (0.0005) [2023-12-26 17:20:15,976][105692] Updated weights for policy 0, policy_version 271196 (0.0005) [2023-12-26 17:20:16,036][105692] Updated weights for policy 0, policy_version 271206 (0.0008) [2023-12-26 17:20:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19466.4). Total num frames: 138895360. Throughput: 0: 9943.4, 1: 9694.2. Samples: 138862492. Policy #0 lag: (min: 9.0, avg: 22.6, max: 41.0) [2023-12-26 17:20:16,062][104569] Avg episode reward: [(0, '9230.050'), (1, '9077.094')] [2023-12-26 17:20:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000271208_69443584.pth... [2023-12-26 17:20:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000271272_69451776.pth... [2023-12-26 17:20:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000270056_69148672.pth [2023-12-26 17:20:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000270184_69173248.pth [2023-12-26 17:20:16,408][105620] Updated weights for policy 1, policy_version 271278 (0.0009) [2023-12-26 17:20:16,469][105620] Updated weights for policy 1, policy_version 271288 (0.0009) [2023-12-26 17:20:16,520][105620] Updated weights for policy 1, policy_version 271299 (0.0009) [2023-12-26 17:20:16,621][105692] Updated weights for policy 0, policy_version 271216 (0.0010) [2023-12-26 17:20:16,675][105692] Updated weights for policy 0, policy_version 271226 (0.0010) [2023-12-26 17:20:16,733][105692] Updated weights for policy 0, policy_version 271236 (0.0008) [2023-12-26 17:20:17,270][105620] Updated weights for policy 1, policy_version 271309 (0.0008) [2023-12-26 17:20:17,332][105620] Updated weights for policy 1, policy_version 271319 (0.0005) [2023-12-26 17:20:17,383][105692] Updated weights for policy 0, policy_version 271246 (0.0010) [2023-12-26 17:20:17,391][105620] Updated weights for policy 1, policy_version 271329 (0.0006) [2023-12-26 17:20:17,441][105692] Updated weights for policy 0, policy_version 271256 (0.0010) [2023-12-26 17:20:17,495][105692] Updated weights for policy 0, policy_version 271266 (0.0010) [2023-12-26 17:20:17,915][105620] Updated weights for policy 1, policy_version 271339 (0.0007) [2023-12-26 17:20:17,971][105620] Updated weights for policy 1, policy_version 271349 (0.0005) [2023-12-26 17:20:18,022][105620] Updated weights for policy 1, policy_version 271359 (0.0005) [2023-12-26 17:20:18,151][105692] Updated weights for policy 0, policy_version 271276 (0.0008) [2023-12-26 17:20:18,202][105692] Updated weights for policy 0, policy_version 271286 (0.0005) [2023-12-26 17:20:18,247][105692] Updated weights for policy 0, policy_version 271296 (0.0005) [2023-12-26 17:20:18,796][105620] Updated weights for policy 1, policy_version 271369 (0.0008) [2023-12-26 17:20:18,863][105620] Updated weights for policy 1, policy_version 271379 (0.0005) [2023-12-26 17:20:18,921][105692] Updated weights for policy 0, policy_version 271306 (0.0006) [2023-12-26 17:20:18,927][105620] Updated weights for policy 1, policy_version 271389 (0.0005) [2023-12-26 17:20:18,972][105692] Updated weights for policy 0, policy_version 271316 (0.0010) [2023-12-26 17:20:18,983][105620] Updated weights for policy 1, policy_version 271399 (0.0005) [2023-12-26 17:20:19,024][105692] Updated weights for policy 0, policy_version 271326 (0.0010) [2023-12-26 17:20:19,075][105692] Updated weights for policy 0, policy_version 271336 (0.0010) [2023-12-26 17:20:19,649][105620] Updated weights for policy 1, policy_version 271409 (0.0008) [2023-12-26 17:20:19,713][105620] Updated weights for policy 1, policy_version 271419 (0.0008) [2023-12-26 17:20:19,772][105620] Updated weights for policy 1, policy_version 271429 (0.0009) [2023-12-26 17:20:19,872][105692] Updated weights for policy 0, policy_version 271346 (0.0009) [2023-12-26 17:20:19,932][105692] Updated weights for policy 0, policy_version 271356 (0.0009) [2023-12-26 17:20:19,993][105692] Updated weights for policy 0, policy_version 271366 (0.0009) [2023-12-26 17:20:20,468][105620] Updated weights for policy 1, policy_version 271439 (0.0007) [2023-12-26 17:20:20,527][105620] Updated weights for policy 1, policy_version 271449 (0.0009) [2023-12-26 17:20:20,582][105620] Updated weights for policy 1, policy_version 271459 (0.0008) [2023-12-26 17:20:20,806][105692] Updated weights for policy 0, policy_version 271376 (0.0009) [2023-12-26 17:20:20,857][105692] Updated weights for policy 0, policy_version 271386 (0.0008) [2023-12-26 17:20:20,904][105692] Updated weights for policy 0, policy_version 271396 (0.0007) [2023-12-26 17:20:21,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 138993664. Throughput: 0: 9954.9, 1: 9569.0. Samples: 138982444. Policy #0 lag: (min: 9.0, avg: 22.6, max: 41.0) [2023-12-26 17:20:21,063][104569] Avg episode reward: [(0, '9356.550'), (1, '9082.410')] [2023-12-26 17:20:21,374][105620] Updated weights for policy 1, policy_version 271469 (0.0009) [2023-12-26 17:20:21,444][105620] Updated weights for policy 1, policy_version 271479 (0.0007) [2023-12-26 17:20:21,497][105620] Updated weights for policy 1, policy_version 271489 (0.0006) [2023-12-26 17:20:21,678][105692] Updated weights for policy 0, policy_version 271406 (0.0008) [2023-12-26 17:20:21,744][105692] Updated weights for policy 0, policy_version 271416 (0.0009) [2023-12-26 17:20:21,802][105692] Updated weights for policy 0, policy_version 271426 (0.0009) [2023-12-26 17:20:22,190][105620] Updated weights for policy 1, policy_version 271499 (0.0007) [2023-12-26 17:20:22,251][105620] Updated weights for policy 1, policy_version 271509 (0.0009) [2023-12-26 17:20:22,303][105620] Updated weights for policy 1, policy_version 271519 (0.0008) [2023-12-26 17:20:22,507][105692] Updated weights for policy 0, policy_version 271436 (0.0008) [2023-12-26 17:20:22,573][105692] Updated weights for policy 0, policy_version 271446 (0.0005) [2023-12-26 17:20:22,629][105692] Updated weights for policy 0, policy_version 271456 (0.0005) [2023-12-26 17:20:23,030][105620] Updated weights for policy 1, policy_version 271529 (0.0009) [2023-12-26 17:20:23,082][105620] Updated weights for policy 1, policy_version 271539 (0.0008) [2023-12-26 17:20:23,150][105620] Updated weights for policy 1, policy_version 271549 (0.0008) [2023-12-26 17:20:23,216][105620] Updated weights for policy 1, policy_version 271559 (0.0007) [2023-12-26 17:20:23,311][105692] Updated weights for policy 0, policy_version 271466 (0.0006) [2023-12-26 17:20:23,377][105692] Updated weights for policy 0, policy_version 271476 (0.0011) [2023-12-26 17:20:23,445][105692] Updated weights for policy 0, policy_version 271486 (0.0011) [2023-12-26 17:20:23,511][105692] Updated weights for policy 0, policy_version 271496 (0.0009) [2023-12-26 17:20:23,838][105620] Updated weights for policy 1, policy_version 271569 (0.0005) [2023-12-26 17:20:23,893][105620] Updated weights for policy 1, policy_version 271579 (0.0005) [2023-12-26 17:20:23,943][105620] Updated weights for policy 1, policy_version 271589 (0.0005) [2023-12-26 17:20:24,172][105692] Updated weights for policy 0, policy_version 271506 (0.0006) [2023-12-26 17:20:24,225][105692] Updated weights for policy 0, policy_version 271516 (0.0008) [2023-12-26 17:20:24,274][105692] Updated weights for policy 0, policy_version 271526 (0.0007) [2023-12-26 17:20:24,696][105620] Updated weights for policy 1, policy_version 271599 (0.0010) [2023-12-26 17:20:24,769][105620] Updated weights for policy 1, policy_version 271609 (0.0009) [2023-12-26 17:20:24,838][105620] Updated weights for policy 1, policy_version 271619 (0.0009) [2023-12-26 17:20:24,886][105692] Updated weights for policy 0, policy_version 271536 (0.0005) [2023-12-26 17:20:24,947][105692] Updated weights for policy 0, policy_version 271546 (0.0005) [2023-12-26 17:20:25,014][105692] Updated weights for policy 0, policy_version 271556 (0.0005) [2023-12-26 17:20:25,523][105692] Updated weights for policy 0, policy_version 271566 (0.0005) [2023-12-26 17:20:25,577][105692] Updated weights for policy 0, policy_version 271576 (0.0005) [2023-12-26 17:20:25,632][105692] Updated weights for policy 0, policy_version 271586 (0.0005) [2023-12-26 17:20:25,718][105620] Updated weights for policy 1, policy_version 271629 (0.0010) [2023-12-26 17:20:25,771][105620] Updated weights for policy 1, policy_version 271639 (0.0008) [2023-12-26 17:20:25,820][105620] Updated weights for policy 1, policy_version 271650 (0.0009) [2023-12-26 17:20:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 139091968. Throughput: 0: 9967.5, 1: 9489.1. Samples: 139099228. Policy #0 lag: (min: 13.0, avg: 15.6, max: 45.0) [2023-12-26 17:20:26,062][104569] Avg episode reward: [(0, '9174.124'), (1, '9082.273')] [2023-12-26 17:20:26,220][105692] Updated weights for policy 0, policy_version 271596 (0.0007) [2023-12-26 17:20:26,284][105692] Updated weights for policy 0, policy_version 271606 (0.0009) [2023-12-26 17:20:26,346][105692] Updated weights for policy 0, policy_version 271616 (0.0009) [2023-12-26 17:20:26,634][105620] Updated weights for policy 1, policy_version 271660 (0.0008) [2023-12-26 17:20:26,695][105620] Updated weights for policy 1, policy_version 271670 (0.0009) [2023-12-26 17:20:26,756][105620] Updated weights for policy 1, policy_version 271680 (0.0009) [2023-12-26 17:20:27,020][105692] Updated weights for policy 0, policy_version 271626 (0.0010) [2023-12-26 17:20:27,077][105692] Updated weights for policy 0, policy_version 271636 (0.0010) [2023-12-26 17:20:27,135][105692] Updated weights for policy 0, policy_version 271646 (0.0009) [2023-12-26 17:20:27,193][105692] Updated weights for policy 0, policy_version 271656 (0.0005) [2023-12-26 17:20:27,512][105620] Updated weights for policy 1, policy_version 271690 (0.0008) [2023-12-26 17:20:27,564][105620] Updated weights for policy 1, policy_version 271700 (0.0009) [2023-12-26 17:20:27,612][105620] Updated weights for policy 1, policy_version 271710 (0.0008) [2023-12-26 17:20:27,762][105692] Updated weights for policy 0, policy_version 271666 (0.0005) [2023-12-26 17:20:27,806][105692] Updated weights for policy 0, policy_version 271676 (0.0006) [2023-12-26 17:20:27,860][105692] Updated weights for policy 0, policy_version 271686 (0.0010) [2023-12-26 17:20:28,406][105620] Updated weights for policy 1, policy_version 271721 (0.0010) [2023-12-26 17:20:28,464][105620] Updated weights for policy 1, policy_version 271731 (0.0008) [2023-12-26 17:20:28,520][105620] Updated weights for policy 1, policy_version 271741 (0.0008) [2023-12-26 17:20:28,567][105692] Updated weights for policy 0, policy_version 271696 (0.0011) [2023-12-26 17:20:28,577][105620] Updated weights for policy 1, policy_version 271751 (0.0005) [2023-12-26 17:20:28,629][105692] Updated weights for policy 0, policy_version 271706 (0.0011) [2023-12-26 17:20:28,691][105692] Updated weights for policy 0, policy_version 271716 (0.0011) [2023-12-26 17:20:29,364][105620] Updated weights for policy 1, policy_version 271761 (0.0009) [2023-12-26 17:20:29,409][105692] Updated weights for policy 0, policy_version 271726 (0.0010) [2023-12-26 17:20:29,421][105620] Updated weights for policy 1, policy_version 271771 (0.0009) [2023-12-26 17:20:29,461][105692] Updated weights for policy 0, policy_version 271736 (0.0009) [2023-12-26 17:20:29,473][105620] Updated weights for policy 1, policy_version 271781 (0.0009) [2023-12-26 17:20:29,512][105692] Updated weights for policy 0, policy_version 271746 (0.0009) [2023-12-26 17:20:30,202][105620] Updated weights for policy 1, policy_version 271791 (0.0008) [2023-12-26 17:20:30,256][105620] Updated weights for policy 1, policy_version 271801 (0.0009) [2023-12-26 17:20:30,258][105692] Updated weights for policy 0, policy_version 271756 (0.0009) [2023-12-26 17:20:30,312][105692] Updated weights for policy 0, policy_version 271766 (0.0006) [2023-12-26 17:20:30,315][105620] Updated weights for policy 1, policy_version 271811 (0.0007) [2023-12-26 17:20:30,377][105692] Updated weights for policy 0, policy_version 271776 (0.0008) [2023-12-26 17:20:31,048][105692] Updated weights for policy 0, policy_version 271786 (0.0008) [2023-12-26 17:20:31,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 139182080. Throughput: 0: 10011.1, 1: 9515.0. Samples: 139159056. Policy #0 lag: (min: 13.0, avg: 15.6, max: 45.0) [2023-12-26 17:20:31,062][104569] Avg episode reward: [(0, '910.861'), (1, '8901.496')] [2023-12-26 17:20:31,067][105620] Updated weights for policy 1, policy_version 271821 (0.0009) [2023-12-26 17:20:31,107][105692] Updated weights for policy 0, policy_version 271796 (0.0009) [2023-12-26 17:20:31,132][105620] Updated weights for policy 1, policy_version 271831 (0.0006) [2023-12-26 17:20:31,172][105692] Updated weights for policy 0, policy_version 271806 (0.0007) [2023-12-26 17:20:31,189][105620] Updated weights for policy 1, policy_version 271841 (0.0008) [2023-12-26 17:20:31,223][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000271848_69599232.pth... [2023-12-26 17:20:31,227][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000270760_69320704.pth [2023-12-26 17:20:31,227][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000271816_69599232.pth... [2023-12-26 17:20:31,228][105692] Updated weights for policy 0, policy_version 271816 (0.0006) [2023-12-26 17:20:31,231][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000270632_69296128.pth [2023-12-26 17:20:31,893][105620] Updated weights for policy 1, policy_version 271851 (0.0006) [2023-12-26 17:20:31,948][105620] Updated weights for policy 1, policy_version 271861 (0.0005) [2023-12-26 17:20:31,997][105620] Updated weights for policy 1, policy_version 271871 (0.0005) [2023-12-26 17:20:32,048][105692] Updated weights for policy 0, policy_version 271826 (0.0009) [2023-12-26 17:20:32,110][105692] Updated weights for policy 0, policy_version 271836 (0.0006) [2023-12-26 17:20:32,178][105692] Updated weights for policy 0, policy_version 271846 (0.0008) [2023-12-26 17:20:32,621][105620] Updated weights for policy 1, policy_version 271881 (0.0007) [2023-12-26 17:20:32,681][105620] Updated weights for policy 1, policy_version 271891 (0.0009) [2023-12-26 17:20:32,738][105620] Updated weights for policy 1, policy_version 271901 (0.0008) [2023-12-26 17:20:32,798][105620] Updated weights for policy 1, policy_version 271911 (0.0008) [2023-12-26 17:20:32,940][105692] Updated weights for policy 0, policy_version 271856 (0.0009) [2023-12-26 17:20:33,002][105692] Updated weights for policy 0, policy_version 271866 (0.0008) [2023-12-26 17:20:33,076][105692] Updated weights for policy 0, policy_version 271876 (0.0005) [2023-12-26 17:20:33,555][105620] Updated weights for policy 1, policy_version 271921 (0.0006) [2023-12-26 17:20:33,624][105620] Updated weights for policy 1, policy_version 271931 (0.0005) [2023-12-26 17:20:33,683][105620] Updated weights for policy 1, policy_version 271941 (0.0006) [2023-12-26 17:20:33,685][105692] Updated weights for policy 0, policy_version 271886 (0.0005) [2023-12-26 17:20:33,733][105692] Updated weights for policy 0, policy_version 271896 (0.0005) [2023-12-26 17:20:33,786][105692] Updated weights for policy 0, policy_version 271906 (0.0008) [2023-12-26 17:20:34,189][105620] Updated weights for policy 1, policy_version 271951 (0.0009) [2023-12-26 17:20:34,251][105620] Updated weights for policy 1, policy_version 271961 (0.0009) [2023-12-26 17:20:34,311][105620] Updated weights for policy 1, policy_version 271971 (0.0010) [2023-12-26 17:20:34,590][105692] Updated weights for policy 0, policy_version 271916 (0.0009) [2023-12-26 17:20:34,649][105692] Updated weights for policy 0, policy_version 271926 (0.0009) [2023-12-26 17:20:34,714][105692] Updated weights for policy 0, policy_version 271936 (0.0009) [2023-12-26 17:20:35,001][105620] Updated weights for policy 1, policy_version 271981 (0.0008) [2023-12-26 17:20:35,050][105620] Updated weights for policy 1, policy_version 271991 (0.0005) [2023-12-26 17:20:35,099][105620] Updated weights for policy 1, policy_version 272001 (0.0006) [2023-12-26 17:20:35,448][105692] Updated weights for policy 0, policy_version 271946 (0.0009) [2023-12-26 17:20:35,499][105692] Updated weights for policy 0, policy_version 271956 (0.0008) [2023-12-26 17:20:35,550][105692] Updated weights for policy 0, policy_version 271966 (0.0008) [2023-12-26 17:20:35,598][105692] Updated weights for policy 0, policy_version 271976 (0.0008) [2023-12-26 17:20:35,780][105620] Updated weights for policy 1, policy_version 272011 (0.0010) [2023-12-26 17:20:35,825][105620] Updated weights for policy 1, policy_version 272021 (0.0010) [2023-12-26 17:20:35,866][105620] Updated weights for policy 1, policy_version 272031 (0.0010) [2023-12-26 17:20:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 139288576. Throughput: 0: 9940.5, 1: 9495.4. Samples: 139276128. Policy #0 lag: (min: 13.0, avg: 15.6, max: 45.0) [2023-12-26 17:20:36,063][104569] Avg episode reward: [(0, '926.454'), (1, '8443.206')] [2023-12-26 17:20:36,399][105692] Updated weights for policy 0, policy_version 271986 (0.0009) [2023-12-26 17:20:36,457][105692] Updated weights for policy 0, policy_version 271996 (0.0009) [2023-12-26 17:20:36,510][105692] Updated weights for policy 0, policy_version 272006 (0.0010) [2023-12-26 17:20:36,592][105620] Updated weights for policy 1, policy_version 272041 (0.0010) [2023-12-26 17:20:36,654][105620] Updated weights for policy 1, policy_version 272051 (0.0009) [2023-12-26 17:20:36,713][105620] Updated weights for policy 1, policy_version 272061 (0.0010) [2023-12-26 17:20:36,772][105620] Updated weights for policy 1, policy_version 272071 (0.0009) [2023-12-26 17:20:37,195][105692] Updated weights for policy 0, policy_version 272016 (0.0006) [2023-12-26 17:20:37,249][105692] Updated weights for policy 0, policy_version 272026 (0.0005) [2023-12-26 17:20:37,304][105692] Updated weights for policy 0, policy_version 272036 (0.0006) [2023-12-26 17:20:37,607][105620] Updated weights for policy 1, policy_version 272081 (0.0009) [2023-12-26 17:20:37,659][105620] Updated weights for policy 1, policy_version 272091 (0.0009) [2023-12-26 17:20:37,718][105620] Updated weights for policy 1, policy_version 272101 (0.0010) [2023-12-26 17:20:37,897][105692] Updated weights for policy 0, policy_version 272046 (0.0006) [2023-12-26 17:20:37,950][105692] Updated weights for policy 0, policy_version 272056 (0.0007) [2023-12-26 17:20:38,004][105692] Updated weights for policy 0, policy_version 272066 (0.0010) [2023-12-26 17:20:38,449][105620] Updated weights for policy 1, policy_version 272111 (0.0009) [2023-12-26 17:20:38,505][105620] Updated weights for policy 1, policy_version 272121 (0.0009) [2023-12-26 17:20:38,562][105620] Updated weights for policy 1, policy_version 272131 (0.0008) [2023-12-26 17:20:38,660][105692] Updated weights for policy 0, policy_version 272076 (0.0008) [2023-12-26 17:20:38,714][105692] Updated weights for policy 0, policy_version 272086 (0.0008) [2023-12-26 17:20:38,769][105692] Updated weights for policy 0, policy_version 272096 (0.0008) [2023-12-26 17:20:39,332][105620] Updated weights for policy 1, policy_version 272141 (0.0009) [2023-12-26 17:20:39,400][105620] Updated weights for policy 1, policy_version 272151 (0.0009) [2023-12-26 17:20:39,466][105620] Updated weights for policy 1, policy_version 272161 (0.0009) [2023-12-26 17:20:39,553][105692] Updated weights for policy 0, policy_version 272106 (0.0008) [2023-12-26 17:20:39,607][105692] Updated weights for policy 0, policy_version 272116 (0.0007) [2023-12-26 17:20:39,669][105692] Updated weights for policy 0, policy_version 272126 (0.0005) [2023-12-26 17:20:39,726][105692] Updated weights for policy 0, policy_version 272136 (0.0005) [2023-12-26 17:20:40,193][105620] Updated weights for policy 1, policy_version 272171 (0.0008) [2023-12-26 17:20:40,250][105620] Updated weights for policy 1, policy_version 272181 (0.0005) [2023-12-26 17:20:40,311][105620] Updated weights for policy 1, policy_version 272191 (0.0006) [2023-12-26 17:20:40,429][105692] Updated weights for policy 0, policy_version 272146 (0.0010) [2023-12-26 17:20:40,487][105692] Updated weights for policy 0, policy_version 272156 (0.0008) [2023-12-26 17:20:40,548][105692] Updated weights for policy 0, policy_version 272166 (0.0006) [2023-12-26 17:20:40,907][105620] Updated weights for policy 1, policy_version 272201 (0.0006) [2023-12-26 17:20:40,963][105620] Updated weights for policy 1, policy_version 272211 (0.0008) [2023-12-26 17:20:41,030][105620] Updated weights for policy 1, policy_version 272221 (0.0006) [2023-12-26 17:20:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 139378688. Throughput: 0: 9991.0, 1: 9479.6. Samples: 139392600. Policy #0 lag: (min: 13.0, avg: 15.6, max: 45.0) [2023-12-26 17:20:41,062][104569] Avg episode reward: [(0, '1448.655'), (1, '8712.126')] [2023-12-26 17:20:41,105][105620] Updated weights for policy 1, policy_version 272231 (0.0008) [2023-12-26 17:20:41,339][105692] Updated weights for policy 0, policy_version 272176 (0.0009) [2023-12-26 17:20:41,408][105692] Updated weights for policy 0, policy_version 272186 (0.0008) [2023-12-26 17:20:41,467][105692] Updated weights for policy 0, policy_version 272196 (0.0008) [2023-12-26 17:20:41,854][105620] Updated weights for policy 1, policy_version 272241 (0.0008) [2023-12-26 17:20:41,912][105620] Updated weights for policy 1, policy_version 272251 (0.0008) [2023-12-26 17:20:41,979][105620] Updated weights for policy 1, policy_version 272261 (0.0008) [2023-12-26 17:20:42,244][105692] Updated weights for policy 0, policy_version 272206 (0.0009) [2023-12-26 17:20:42,307][105692] Updated weights for policy 0, policy_version 272216 (0.0009) [2023-12-26 17:20:42,370][105692] Updated weights for policy 0, policy_version 272226 (0.0008) [2023-12-26 17:20:42,696][105620] Updated weights for policy 1, policy_version 272271 (0.0007) [2023-12-26 17:20:42,755][105620] Updated weights for policy 1, policy_version 272281 (0.0008) [2023-12-26 17:20:42,817][105620] Updated weights for policy 1, policy_version 272291 (0.0007) [2023-12-26 17:20:43,131][105692] Updated weights for policy 0, policy_version 272236 (0.0008) [2023-12-26 17:20:43,190][105692] Updated weights for policy 0, policy_version 272246 (0.0008) [2023-12-26 17:20:43,258][105692] Updated weights for policy 0, policy_version 272256 (0.0009) [2023-12-26 17:20:43,404][105620] Updated weights for policy 1, policy_version 272301 (0.0008) [2023-12-26 17:20:43,468][105620] Updated weights for policy 1, policy_version 272311 (0.0005) [2023-12-26 17:20:43,528][105620] Updated weights for policy 1, policy_version 272321 (0.0009) [2023-12-26 17:20:44,016][105692] Updated weights for policy 0, policy_version 272266 (0.0008) [2023-12-26 17:20:44,071][105692] Updated weights for policy 0, policy_version 272276 (0.0008) [2023-12-26 17:20:44,129][105692] Updated weights for policy 0, policy_version 272286 (0.0008) [2023-12-26 17:20:44,188][105692] Updated weights for policy 0, policy_version 272296 (0.0008) [2023-12-26 17:20:44,223][105620] Updated weights for policy 1, policy_version 272331 (0.0008) [2023-12-26 17:20:44,278][105620] Updated weights for policy 1, policy_version 272341 (0.0010) [2023-12-26 17:20:44,340][105620] Updated weights for policy 1, policy_version 272351 (0.0009) [2023-12-26 17:20:44,911][105692] Updated weights for policy 0, policy_version 272306 (0.0006) [2023-12-26 17:20:44,963][105692] Updated weights for policy 0, policy_version 272316 (0.0008) [2023-12-26 17:20:45,015][105692] Updated weights for policy 0, policy_version 272326 (0.0006) [2023-12-26 17:20:45,042][105620] Updated weights for policy 1, policy_version 272361 (0.0009) [2023-12-26 17:20:45,106][105620] Updated weights for policy 1, policy_version 272371 (0.0010) [2023-12-26 17:20:45,162][105620] Updated weights for policy 1, policy_version 272381 (0.0010) [2023-12-26 17:20:45,224][105620] Updated weights for policy 1, policy_version 272391 (0.0007) [2023-12-26 17:20:45,698][105692] Updated weights for policy 0, policy_version 272336 (0.0005) [2023-12-26 17:20:45,761][105692] Updated weights for policy 0, policy_version 272346 (0.0009) [2023-12-26 17:20:45,816][105692] Updated weights for policy 0, policy_version 272356 (0.0010) [2023-12-26 17:20:45,957][105620] Updated weights for policy 1, policy_version 272401 (0.0011) [2023-12-26 17:20:46,005][105620] Updated weights for policy 1, policy_version 272411 (0.0010) [2023-12-26 17:20:46,049][105620] Updated weights for policy 1, policy_version 272421 (0.0010) [2023-12-26 17:20:46,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 139485184. Throughput: 0: 9942.2, 1: 9465.9. Samples: 139449096. Policy #0 lag: (min: 13.0, avg: 15.6, max: 45.0) [2023-12-26 17:20:46,062][104569] Avg episode reward: [(0, '2985.052'), (1, '8899.921')] [2023-12-26 17:20:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000272360_69738496.pth... [2023-12-26 17:20:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000272424_69746688.pth... [2023-12-26 17:20:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000271272_69451776.pth [2023-12-26 17:20:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000271208_69443584.pth [2023-12-26 17:20:46,502][105692] Updated weights for policy 0, policy_version 272366 (0.0007) [2023-12-26 17:20:46,565][105692] Updated weights for policy 0, policy_version 272376 (0.0010) [2023-12-26 17:20:46,621][105692] Updated weights for policy 0, policy_version 272386 (0.0009) [2023-12-26 17:20:46,752][105620] Updated weights for policy 1, policy_version 272431 (0.0010) [2023-12-26 17:20:46,813][105620] Updated weights for policy 1, policy_version 272441 (0.0010) [2023-12-26 17:20:46,858][105620] Updated weights for policy 1, policy_version 272451 (0.0008) [2023-12-26 17:20:47,211][105692] Updated weights for policy 0, policy_version 272396 (0.0005) [2023-12-26 17:20:47,257][105692] Updated weights for policy 0, policy_version 272406 (0.0005) [2023-12-26 17:20:47,309][105692] Updated weights for policy 0, policy_version 272416 (0.0005) [2023-12-26 17:20:47,554][105620] Updated weights for policy 1, policy_version 272461 (0.0006) [2023-12-26 17:20:47,610][105620] Updated weights for policy 1, policy_version 272471 (0.0009) [2023-12-26 17:20:47,672][105620] Updated weights for policy 1, policy_version 272481 (0.0011) [2023-12-26 17:20:47,952][105692] Updated weights for policy 0, policy_version 272426 (0.0009) [2023-12-26 17:20:48,015][105692] Updated weights for policy 0, policy_version 272436 (0.0008) [2023-12-26 17:20:48,067][105692] Updated weights for policy 0, policy_version 272446 (0.0006) [2023-12-26 17:20:48,115][105692] Updated weights for policy 0, policy_version 272456 (0.0006) [2023-12-26 17:20:48,390][105620] Updated weights for policy 1, policy_version 272491 (0.0011) [2023-12-26 17:20:48,450][105620] Updated weights for policy 1, policy_version 272501 (0.0011) [2023-12-26 17:20:48,509][105620] Updated weights for policy 1, policy_version 272511 (0.0010) [2023-12-26 17:20:48,757][105692] Updated weights for policy 0, policy_version 272466 (0.0006) [2023-12-26 17:20:48,822][105692] Updated weights for policy 0, policy_version 272476 (0.0006) [2023-12-26 17:20:48,886][105692] Updated weights for policy 0, policy_version 272486 (0.0005) [2023-12-26 17:20:49,239][105620] Updated weights for policy 1, policy_version 272521 (0.0010) [2023-12-26 17:20:49,291][105620] Updated weights for policy 1, policy_version 272531 (0.0008) [2023-12-26 17:20:49,354][105620] Updated weights for policy 1, policy_version 272541 (0.0008) [2023-12-26 17:20:49,423][105620] Updated weights for policy 1, policy_version 272551 (0.0008) [2023-12-26 17:20:49,593][105692] Updated weights for policy 0, policy_version 272497 (0.0009) [2023-12-26 17:20:49,652][105692] Updated weights for policy 0, policy_version 272507 (0.0010) [2023-12-26 17:20:49,719][105692] Updated weights for policy 0, policy_version 272517 (0.0009) [2023-12-26 17:20:50,143][105620] Updated weights for policy 1, policy_version 272561 (0.0008) [2023-12-26 17:20:50,205][105620] Updated weights for policy 1, policy_version 272571 (0.0009) [2023-12-26 17:20:50,260][105620] Updated weights for policy 1, policy_version 272581 (0.0009) [2023-12-26 17:20:50,474][105692] Updated weights for policy 0, policy_version 272527 (0.0008) [2023-12-26 17:20:50,532][105692] Updated weights for policy 0, policy_version 272537 (0.0009) [2023-12-26 17:20:50,593][105692] Updated weights for policy 0, policy_version 272547 (0.0007) [2023-12-26 17:20:50,935][105620] Updated weights for policy 1, policy_version 272591 (0.0007) [2023-12-26 17:20:50,992][105620] Updated weights for policy 1, policy_version 272601 (0.0005) [2023-12-26 17:20:51,060][105620] Updated weights for policy 1, policy_version 272611 (0.0007) [2023-12-26 17:20:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 139575296. Throughput: 0: 10018.2, 1: 9507.2. Samples: 139569636. Policy #0 lag: (min: 13.0, avg: 15.6, max: 45.0) [2023-12-26 17:20:51,062][104569] Avg episode reward: [(0, '5745.307'), (1, '8995.336')] [2023-12-26 17:20:51,358][105692] Updated weights for policy 0, policy_version 272557 (0.0009) [2023-12-26 17:20:51,425][105692] Updated weights for policy 0, policy_version 272567 (0.0009) [2023-12-26 17:20:51,487][105692] Updated weights for policy 0, policy_version 272577 (0.0009) [2023-12-26 17:20:51,777][105620] Updated weights for policy 1, policy_version 272621 (0.0007) [2023-12-26 17:20:51,841][105620] Updated weights for policy 1, policy_version 272631 (0.0009) [2023-12-26 17:20:51,908][105620] Updated weights for policy 1, policy_version 272641 (0.0010) [2023-12-26 17:20:52,161][105692] Updated weights for policy 0, policy_version 272587 (0.0009) [2023-12-26 17:20:52,216][105692] Updated weights for policy 0, policy_version 272597 (0.0009) [2023-12-26 17:20:52,268][105692] Updated weights for policy 0, policy_version 272607 (0.0008) [2023-12-26 17:20:52,672][105620] Updated weights for policy 1, policy_version 272651 (0.0010) [2023-12-26 17:20:52,730][105620] Updated weights for policy 1, policy_version 272661 (0.0010) [2023-12-26 17:20:52,784][105620] Updated weights for policy 1, policy_version 272671 (0.0009) [2023-12-26 17:20:52,982][105692] Updated weights for policy 0, policy_version 272617 (0.0009) [2023-12-26 17:20:53,051][105692] Updated weights for policy 0, policy_version 272627 (0.0011) [2023-12-26 17:20:53,112][105692] Updated weights for policy 0, policy_version 272637 (0.0010) [2023-12-26 17:20:53,170][105692] Updated weights for policy 0, policy_version 272647 (0.0010) [2023-12-26 17:20:53,426][105620] Updated weights for policy 1, policy_version 272681 (0.0009) [2023-12-26 17:20:53,484][105620] Updated weights for policy 1, policy_version 272691 (0.0008) [2023-12-26 17:20:53,542][105620] Updated weights for policy 1, policy_version 272701 (0.0008) [2023-12-26 17:20:53,596][105620] Updated weights for policy 1, policy_version 272711 (0.0008) [2023-12-26 17:20:53,910][105692] Updated weights for policy 0, policy_version 272657 (0.0010) [2023-12-26 17:20:53,976][105692] Updated weights for policy 0, policy_version 272667 (0.0011) [2023-12-26 17:20:54,039][105692] Updated weights for policy 0, policy_version 272677 (0.0011) [2023-12-26 17:20:54,248][105620] Updated weights for policy 1, policy_version 272721 (0.0006) [2023-12-26 17:20:54,293][105620] Updated weights for policy 1, policy_version 272731 (0.0008) [2023-12-26 17:20:54,348][105620] Updated weights for policy 1, policy_version 272741 (0.0008) [2023-12-26 17:20:54,782][105692] Updated weights for policy 0, policy_version 272687 (0.0011) [2023-12-26 17:20:54,837][105692] Updated weights for policy 0, policy_version 272697 (0.0010) [2023-12-26 17:20:54,882][105692] Updated weights for policy 0, policy_version 272707 (0.0010) [2023-12-26 17:20:55,013][105620] Updated weights for policy 1, policy_version 272751 (0.0008) [2023-12-26 17:20:55,068][105620] Updated weights for policy 1, policy_version 272761 (0.0008) [2023-12-26 17:20:55,136][105620] Updated weights for policy 1, policy_version 272771 (0.0010) [2023-12-26 17:20:55,590][105692] Updated weights for policy 0, policy_version 272717 (0.0010) [2023-12-26 17:20:55,653][105692] Updated weights for policy 0, policy_version 272727 (0.0011) [2023-12-26 17:20:55,701][105692] Updated weights for policy 0, policy_version 272737 (0.0010) [2023-12-26 17:20:55,880][105620] Updated weights for policy 1, policy_version 272781 (0.0008) [2023-12-26 17:20:55,938][105620] Updated weights for policy 1, policy_version 272791 (0.0005) [2023-12-26 17:20:55,997][105620] Updated weights for policy 1, policy_version 272801 (0.0005) [2023-12-26 17:20:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 139681792. Throughput: 0: 9904.9, 1: 9660.1. Samples: 139686400. Policy #0 lag: (min: 13.0, avg: 15.6, max: 45.0) [2023-12-26 17:20:56,063][104569] Avg episode reward: [(0, '5935.392'), (1, '8906.143')] [2023-12-26 17:20:56,452][105692] Updated weights for policy 0, policy_version 272747 (0.0011) [2023-12-26 17:20:56,500][105692] Updated weights for policy 0, policy_version 272757 (0.0010) [2023-12-26 17:20:56,548][105692] Updated weights for policy 0, policy_version 272767 (0.0010) [2023-12-26 17:20:56,592][105620] Updated weights for policy 1, policy_version 272811 (0.0005) [2023-12-26 17:20:56,640][105620] Updated weights for policy 1, policy_version 272821 (0.0005) [2023-12-26 17:20:56,684][105620] Updated weights for policy 1, policy_version 272831 (0.0005) [2023-12-26 17:20:57,314][105692] Updated weights for policy 0, policy_version 272777 (0.0010) [2023-12-26 17:20:57,315][105620] Updated weights for policy 1, policy_version 272841 (0.0006) [2023-12-26 17:20:57,364][105620] Updated weights for policy 1, policy_version 272851 (0.0007) [2023-12-26 17:20:57,366][105692] Updated weights for policy 0, policy_version 272787 (0.0006) [2023-12-26 17:20:57,408][105620] Updated weights for policy 1, policy_version 272861 (0.0006) [2023-12-26 17:20:57,414][105692] Updated weights for policy 0, policy_version 272797 (0.0007) [2023-12-26 17:20:57,452][105620] Updated weights for policy 1, policy_version 272871 (0.0005) [2023-12-26 17:20:57,469][105692] Updated weights for policy 0, policy_version 272807 (0.0008) [2023-12-26 17:20:58,172][105692] Updated weights for policy 0, policy_version 272817 (0.0010) [2023-12-26 17:20:58,234][105692] Updated weights for policy 0, policy_version 272827 (0.0008) [2023-12-26 17:20:58,271][105620] Updated weights for policy 1, policy_version 272881 (0.0007) [2023-12-26 17:20:58,291][105692] Updated weights for policy 0, policy_version 272837 (0.0007) [2023-12-26 17:20:58,331][105620] Updated weights for policy 1, policy_version 272891 (0.0007) [2023-12-26 17:20:58,395][105620] Updated weights for policy 1, policy_version 272901 (0.0010) [2023-12-26 17:20:59,101][105692] Updated weights for policy 0, policy_version 272847 (0.0008) [2023-12-26 17:20:59,151][105692] Updated weights for policy 0, policy_version 272857 (0.0007) [2023-12-26 17:20:59,167][105620] Updated weights for policy 1, policy_version 272911 (0.0008) [2023-12-26 17:20:59,197][105692] Updated weights for policy 0, policy_version 272867 (0.0008) [2023-12-26 17:20:59,232][105620] Updated weights for policy 1, policy_version 272921 (0.0007) [2023-12-26 17:20:59,290][105620] Updated weights for policy 1, policy_version 272931 (0.0008) [2023-12-26 17:20:59,969][105620] Updated weights for policy 1, policy_version 272941 (0.0009) [2023-12-26 17:21:00,028][105692] Updated weights for policy 0, policy_version 272877 (0.0010) [2023-12-26 17:21:00,032][105620] Updated weights for policy 1, policy_version 272951 (0.0009) [2023-12-26 17:21:00,083][105692] Updated weights for policy 0, policy_version 272887 (0.0010) [2023-12-26 17:21:00,093][105620] Updated weights for policy 1, policy_version 272961 (0.0008) [2023-12-26 17:21:00,132][105692] Updated weights for policy 0, policy_version 272897 (0.0007) [2023-12-26 17:21:00,836][105692] Updated weights for policy 0, policy_version 272907 (0.0009) [2023-12-26 17:21:00,850][105620] Updated weights for policy 1, policy_version 272971 (0.0007) [2023-12-26 17:21:00,898][105692] Updated weights for policy 0, policy_version 272917 (0.0008) [2023-12-26 17:21:00,912][105620] Updated weights for policy 1, policy_version 272981 (0.0007) [2023-12-26 17:21:00,943][105692] Updated weights for policy 0, policy_version 272927 (0.0006) [2023-12-26 17:21:00,970][105620] Updated weights for policy 1, policy_version 272991 (0.0008) [2023-12-26 17:21:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 139780096. Throughput: 0: 9872.8, 1: 9724.1. Samples: 139744352. Policy #0 lag: (min: 13.0, avg: 15.6, max: 45.0) [2023-12-26 17:21:01,062][104569] Avg episode reward: [(0, '5757.919'), (1, '8993.404')] [2023-12-26 17:21:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000272936_69885952.pth... [2023-12-26 17:21:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000273000_69894144.pth... [2023-12-26 17:21:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000271816_69599232.pth [2023-12-26 17:21:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000271848_69599232.pth [2023-12-26 17:21:01,585][105620] Updated weights for policy 1, policy_version 273001 (0.0007) [2023-12-26 17:21:01,647][105620] Updated weights for policy 1, policy_version 273011 (0.0009) [2023-12-26 17:21:01,707][105620] Updated weights for policy 1, policy_version 273021 (0.0007) [2023-12-26 17:21:01,725][105692] Updated weights for policy 0, policy_version 272937 (0.0007) [2023-12-26 17:21:01,769][105620] Updated weights for policy 1, policy_version 273031 (0.0007) [2023-12-26 17:21:01,785][105692] Updated weights for policy 0, policy_version 272947 (0.0009) [2023-12-26 17:21:01,843][105692] Updated weights for policy 0, policy_version 272957 (0.0011) [2023-12-26 17:21:01,904][105692] Updated weights for policy 0, policy_version 272968 (0.0009) [2023-12-26 17:21:02,371][105620] Updated weights for policy 1, policy_version 273041 (0.0007) [2023-12-26 17:21:02,430][105620] Updated weights for policy 1, policy_version 273051 (0.0005) [2023-12-26 17:21:02,487][105620] Updated weights for policy 1, policy_version 273061 (0.0005) [2023-12-26 17:21:02,765][105692] Updated weights for policy 0, policy_version 272978 (0.0008) [2023-12-26 17:21:02,821][105692] Updated weights for policy 0, policy_version 272988 (0.0008) [2023-12-26 17:21:02,878][105692] Updated weights for policy 0, policy_version 272998 (0.0009) [2023-12-26 17:21:03,124][105620] Updated weights for policy 1, policy_version 273071 (0.0006) [2023-12-26 17:21:03,186][105620] Updated weights for policy 1, policy_version 273081 (0.0006) [2023-12-26 17:21:03,247][105620] Updated weights for policy 1, policy_version 273091 (0.0006) [2023-12-26 17:21:03,719][105692] Updated weights for policy 0, policy_version 273008 (0.0009) [2023-12-26 17:21:03,781][105692] Updated weights for policy 0, policy_version 273018 (0.0009) [2023-12-26 17:21:03,821][105620] Updated weights for policy 1, policy_version 273101 (0.0007) [2023-12-26 17:21:03,839][105692] Updated weights for policy 0, policy_version 273028 (0.0009) [2023-12-26 17:21:03,886][105620] Updated weights for policy 1, policy_version 273111 (0.0008) [2023-12-26 17:21:03,950][105620] Updated weights for policy 1, policy_version 273121 (0.0009) [2023-12-26 17:21:04,586][105692] Updated weights for policy 0, policy_version 273038 (0.0009) [2023-12-26 17:21:04,644][105692] Updated weights for policy 0, policy_version 273048 (0.0009) [2023-12-26 17:21:04,673][105620] Updated weights for policy 1, policy_version 273131 (0.0009) [2023-12-26 17:21:04,702][105692] Updated weights for policy 0, policy_version 273058 (0.0009) [2023-12-26 17:21:04,728][105620] Updated weights for policy 1, policy_version 273141 (0.0006) [2023-12-26 17:21:04,785][105620] Updated weights for policy 1, policy_version 273151 (0.0009) [2023-12-26 17:21:05,464][105692] Updated weights for policy 0, policy_version 273068 (0.0008) [2023-12-26 17:21:05,512][105692] Updated weights for policy 0, policy_version 273078 (0.0009) [2023-12-26 17:21:05,532][105620] Updated weights for policy 1, policy_version 273161 (0.0008) [2023-12-26 17:21:05,557][105692] Updated weights for policy 0, policy_version 273088 (0.0008) [2023-12-26 17:21:05,592][105620] Updated weights for policy 1, policy_version 273171 (0.0008) [2023-12-26 17:21:05,652][105620] Updated weights for policy 1, policy_version 273181 (0.0008) [2023-12-26 17:21:05,706][105620] Updated weights for policy 1, policy_version 273191 (0.0008) [2023-12-26 17:21:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 139870208. Throughput: 0: 9688.8, 1: 9812.2. Samples: 139859988. Policy #0 lag: (min: 13.0, avg: 15.6, max: 45.0) [2023-12-26 17:21:06,062][104569] Avg episode reward: [(0, '6746.011'), (1, '8904.923')] [2023-12-26 17:21:06,290][105620] Updated weights for policy 1, policy_version 273201 (0.0007) [2023-12-26 17:21:06,355][105620] Updated weights for policy 1, policy_version 273211 (0.0009) [2023-12-26 17:21:06,416][105620] Updated weights for policy 1, policy_version 273221 (0.0008) [2023-12-26 17:21:06,424][105692] Updated weights for policy 0, policy_version 273098 (0.0009) [2023-12-26 17:21:06,480][105692] Updated weights for policy 0, policy_version 273108 (0.0008) [2023-12-26 17:21:06,545][105692] Updated weights for policy 0, policy_version 273118 (0.0005) [2023-12-26 17:21:06,612][105692] Updated weights for policy 0, policy_version 273128 (0.0008) [2023-12-26 17:21:07,158][105620] Updated weights for policy 1, policy_version 273231 (0.0010) [2023-12-26 17:21:07,226][105620] Updated weights for policy 1, policy_version 273241 (0.0007) [2023-12-26 17:21:07,290][105620] Updated weights for policy 1, policy_version 273251 (0.0007) [2023-12-26 17:21:07,320][105692] Updated weights for policy 0, policy_version 273138 (0.0008) [2023-12-26 17:21:07,381][105692] Updated weights for policy 0, policy_version 273148 (0.0009) [2023-12-26 17:21:07,434][105692] Updated weights for policy 0, policy_version 273158 (0.0008) [2023-12-26 17:21:08,050][105620] Updated weights for policy 1, policy_version 273261 (0.0008) [2023-12-26 17:21:08,093][105692] Updated weights for policy 0, policy_version 273168 (0.0007) [2023-12-26 17:21:08,111][105620] Updated weights for policy 1, policy_version 273271 (0.0008) [2023-12-26 17:21:08,152][105692] Updated weights for policy 0, policy_version 273178 (0.0006) [2023-12-26 17:21:08,167][105620] Updated weights for policy 1, policy_version 273281 (0.0006) [2023-12-26 17:21:08,204][105692] Updated weights for policy 0, policy_version 273188 (0.0006) [2023-12-26 17:21:08,838][105620] Updated weights for policy 1, policy_version 273291 (0.0008) [2023-12-26 17:21:08,904][105620] Updated weights for policy 1, policy_version 273301 (0.0011) [2023-12-26 17:21:08,919][105692] Updated weights for policy 0, policy_version 273198 (0.0007) [2023-12-26 17:21:08,964][105620] Updated weights for policy 1, policy_version 273311 (0.0010) [2023-12-26 17:21:08,968][105692] Updated weights for policy 0, policy_version 273208 (0.0007) [2023-12-26 17:21:09,017][105692] Updated weights for policy 0, policy_version 273218 (0.0006) [2023-12-26 17:21:09,686][105620] Updated weights for policy 1, policy_version 273321 (0.0010) [2023-12-26 17:21:09,745][105620] Updated weights for policy 1, policy_version 273331 (0.0008) [2023-12-26 17:21:09,751][105692] Updated weights for policy 0, policy_version 273228 (0.0007) [2023-12-26 17:21:09,805][105620] Updated weights for policy 1, policy_version 273341 (0.0008) [2023-12-26 17:21:09,807][105692] Updated weights for policy 0, policy_version 273238 (0.0007) [2023-12-26 17:21:09,872][105692] Updated weights for policy 0, policy_version 273248 (0.0009) [2023-12-26 17:21:09,874][105620] Updated weights for policy 1, policy_version 273351 (0.0008) [2023-12-26 17:21:10,491][105692] Updated weights for policy 0, policy_version 273258 (0.0007) [2023-12-26 17:21:10,555][105692] Updated weights for policy 0, policy_version 273268 (0.0007) [2023-12-26 17:21:10,618][105692] Updated weights for policy 0, policy_version 273278 (0.0008) [2023-12-26 17:21:10,674][105692] Updated weights for policy 0, policy_version 273288 (0.0007) [2023-12-26 17:21:10,674][105620] Updated weights for policy 1, policy_version 273361 (0.0008) [2023-12-26 17:21:10,746][105620] Updated weights for policy 1, policy_version 273371 (0.0009) [2023-12-26 17:21:10,808][105620] Updated weights for policy 1, policy_version 273381 (0.0010) [2023-12-26 17:21:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 139968512. Throughput: 0: 9638.9, 1: 9845.7. Samples: 139976032. Policy #0 lag: (min: 13.0, avg: 15.6, max: 45.0) [2023-12-26 17:21:11,062][104569] Avg episode reward: [(0, '6927.662'), (1, '8725.826')] [2023-12-26 17:21:11,487][105692] Updated weights for policy 0, policy_version 273298 (0.0008) [2023-12-26 17:21:11,546][105692] Updated weights for policy 0, policy_version 273308 (0.0009) [2023-12-26 17:21:11,580][105620] Updated weights for policy 1, policy_version 273391 (0.0009) [2023-12-26 17:21:11,609][105692] Updated weights for policy 0, policy_version 273318 (0.0007) [2023-12-26 17:21:11,648][105620] Updated weights for policy 1, policy_version 273401 (0.0008) [2023-12-26 17:21:11,712][105620] Updated weights for policy 1, policy_version 273411 (0.0007) [2023-12-26 17:21:12,401][105620] Updated weights for policy 1, policy_version 273421 (0.0010) [2023-12-26 17:21:12,424][105692] Updated weights for policy 0, policy_version 273328 (0.0007) [2023-12-26 17:21:12,462][105620] Updated weights for policy 1, policy_version 273431 (0.0008) [2023-12-26 17:21:12,484][105692] Updated weights for policy 0, policy_version 273338 (0.0008) [2023-12-26 17:21:12,523][105620] Updated weights for policy 1, policy_version 273441 (0.0008) [2023-12-26 17:21:12,549][105692] Updated weights for policy 0, policy_version 273348 (0.0006) [2023-12-26 17:21:13,272][105620] Updated weights for policy 1, policy_version 273451 (0.0008) [2023-12-26 17:21:13,328][105620] Updated weights for policy 1, policy_version 273461 (0.0010) [2023-12-26 17:21:13,330][105692] Updated weights for policy 0, policy_version 273358 (0.0006) [2023-12-26 17:21:13,376][105620] Updated weights for policy 1, policy_version 273471 (0.0010) [2023-12-26 17:21:13,382][105692] Updated weights for policy 0, policy_version 273368 (0.0006) [2023-12-26 17:21:13,438][105692] Updated weights for policy 0, policy_version 273378 (0.0007) [2023-12-26 17:21:14,036][105620] Updated weights for policy 1, policy_version 273481 (0.0010) [2023-12-26 17:21:14,087][105620] Updated weights for policy 1, policy_version 273491 (0.0009) [2023-12-26 17:21:14,146][105620] Updated weights for policy 1, policy_version 273501 (0.0009) [2023-12-26 17:21:14,211][105620] Updated weights for policy 1, policy_version 273511 (0.0009) [2023-12-26 17:21:14,228][105692] Updated weights for policy 0, policy_version 273388 (0.0008) [2023-12-26 17:21:14,288][105692] Updated weights for policy 0, policy_version 273398 (0.0009) [2023-12-26 17:21:14,349][105692] Updated weights for policy 0, policy_version 273408 (0.0009) [2023-12-26 17:21:14,828][105620] Updated weights for policy 1, policy_version 273521 (0.0008) [2023-12-26 17:21:14,890][105620] Updated weights for policy 1, policy_version 273531 (0.0007) [2023-12-26 17:21:14,950][105620] Updated weights for policy 1, policy_version 273541 (0.0008) [2023-12-26 17:21:15,219][105692] Updated weights for policy 0, policy_version 273418 (0.0008) [2023-12-26 17:21:15,282][105692] Updated weights for policy 0, policy_version 273428 (0.0009) [2023-12-26 17:21:15,340][105692] Updated weights for policy 0, policy_version 273438 (0.0009) [2023-12-26 17:21:15,392][105692] Updated weights for policy 0, policy_version 273448 (0.0009) [2023-12-26 17:21:15,614][105620] Updated weights for policy 1, policy_version 273551 (0.0007) [2023-12-26 17:21:15,676][105620] Updated weights for policy 1, policy_version 273561 (0.0007) [2023-12-26 17:21:15,724][105620] Updated weights for policy 1, policy_version 273571 (0.0008) [2023-12-26 17:21:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 140058624. Throughput: 0: 9505.5, 1: 9874.7. Samples: 140031168. Policy #0 lag: (min: 13.0, avg: 15.6, max: 45.0) [2023-12-26 17:21:16,062][104569] Avg episode reward: [(0, '7705.924'), (1, '8809.763')] [2023-12-26 17:21:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000273576_70041600.pth... [2023-12-26 17:21:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000272424_69746688.pth [2023-12-26 17:21:16,094][105692] Updated weights for policy 0, policy_version 273458 (0.0008) [2023-12-26 17:21:16,162][105692] Updated weights for policy 0, policy_version 273468 (0.0006) [2023-12-26 17:21:16,232][105692] Updated weights for policy 0, policy_version 273478 (0.0007) [2023-12-26 17:21:16,243][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000273480_70025216.pth... [2023-12-26 17:21:16,247][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000272360_69738496.pth [2023-12-26 17:21:16,424][105620] Updated weights for policy 1, policy_version 273581 (0.0009) [2023-12-26 17:21:16,489][105620] Updated weights for policy 1, policy_version 273591 (0.0011) [2023-12-26 17:21:16,551][105620] Updated weights for policy 1, policy_version 273601 (0.0007) [2023-12-26 17:21:16,911][105692] Updated weights for policy 0, policy_version 273488 (0.0009) [2023-12-26 17:21:16,968][105692] Updated weights for policy 0, policy_version 273498 (0.0007) [2023-12-26 17:21:17,037][105692] Updated weights for policy 0, policy_version 273508 (0.0011) [2023-12-26 17:21:17,150][105620] Updated weights for policy 1, policy_version 273611 (0.0006) [2023-12-26 17:21:17,202][105620] Updated weights for policy 1, policy_version 273621 (0.0008) [2023-12-26 17:21:17,249][105620] Updated weights for policy 1, policy_version 273631 (0.0008) [2023-12-26 17:21:17,704][105692] Updated weights for policy 0, policy_version 273518 (0.0010) [2023-12-26 17:21:17,756][105692] Updated weights for policy 0, policy_version 273528 (0.0005) [2023-12-26 17:21:17,821][105692] Updated weights for policy 0, policy_version 273538 (0.0005) [2023-12-26 17:21:17,890][105620] Updated weights for policy 1, policy_version 273641 (0.0006) [2023-12-26 17:21:17,938][105620] Updated weights for policy 1, policy_version 273651 (0.0005) [2023-12-26 17:21:17,989][105620] Updated weights for policy 1, policy_version 273661 (0.0005) [2023-12-26 17:21:18,043][105620] Updated weights for policy 1, policy_version 273671 (0.0005) [2023-12-26 17:21:18,442][105692] Updated weights for policy 0, policy_version 273548 (0.0007) [2023-12-26 17:21:18,495][105692] Updated weights for policy 0, policy_version 273558 (0.0007) [2023-12-26 17:21:18,563][105692] Updated weights for policy 0, policy_version 273568 (0.0005) [2023-12-26 17:21:18,769][105620] Updated weights for policy 1, policy_version 273681 (0.0008) [2023-12-26 17:21:18,828][105620] Updated weights for policy 1, policy_version 273691 (0.0008) [2023-12-26 17:21:18,876][105620] Updated weights for policy 1, policy_version 273701 (0.0008) [2023-12-26 17:21:19,221][105692] Updated weights for policy 0, policy_version 273578 (0.0006) [2023-12-26 17:21:19,289][105692] Updated weights for policy 0, policy_version 273588 (0.0011) [2023-12-26 17:21:19,359][105692] Updated weights for policy 0, policy_version 273598 (0.0011) [2023-12-26 17:21:19,429][105692] Updated weights for policy 0, policy_version 273608 (0.0010) [2023-12-26 17:21:19,649][105620] Updated weights for policy 1, policy_version 273711 (0.0008) [2023-12-26 17:21:19,698][105620] Updated weights for policy 1, policy_version 273721 (0.0008) [2023-12-26 17:21:19,743][105620] Updated weights for policy 1, policy_version 273731 (0.0007) [2023-12-26 17:21:20,189][105692] Updated weights for policy 0, policy_version 273618 (0.0005) [2023-12-26 17:21:20,253][105692] Updated weights for policy 0, policy_version 273628 (0.0006) [2023-12-26 17:21:20,320][105692] Updated weights for policy 0, policy_version 273638 (0.0008) [2023-12-26 17:21:20,593][105620] Updated weights for policy 1, policy_version 273741 (0.0009) [2023-12-26 17:21:20,655][105620] Updated weights for policy 1, policy_version 273751 (0.0008) [2023-12-26 17:21:20,720][105620] Updated weights for policy 1, policy_version 273761 (0.0008) [2023-12-26 17:21:20,945][105692] Updated weights for policy 0, policy_version 273648 (0.0010) [2023-12-26 17:21:21,005][105692] Updated weights for policy 0, policy_version 273658 (0.0009) [2023-12-26 17:21:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 140156928. Throughput: 0: 9537.5, 1: 9893.8. Samples: 140150536. Policy #0 lag: (min: 13.0, avg: 15.6, max: 45.0) [2023-12-26 17:21:21,062][104569] Avg episode reward: [(0, '5572.653'), (1, '9174.776')] [2023-12-26 17:21:21,078][105692] Updated weights for policy 0, policy_version 273668 (0.0010) [2023-12-26 17:21:21,524][105620] Updated weights for policy 1, policy_version 273771 (0.0007) [2023-12-26 17:21:21,592][105620] Updated weights for policy 1, policy_version 273781 (0.0009) [2023-12-26 17:21:21,657][105620] Updated weights for policy 1, policy_version 273791 (0.0009) [2023-12-26 17:21:21,840][105692] Updated weights for policy 0, policy_version 273678 (0.0008) [2023-12-26 17:21:21,909][105692] Updated weights for policy 0, policy_version 273688 (0.0008) [2023-12-26 17:21:21,971][105692] Updated weights for policy 0, policy_version 273698 (0.0009) [2023-12-26 17:21:22,478][105620] Updated weights for policy 1, policy_version 273801 (0.0009) [2023-12-26 17:21:22,535][105620] Updated weights for policy 1, policy_version 273811 (0.0005) [2023-12-26 17:21:22,600][105620] Updated weights for policy 1, policy_version 273821 (0.0005) [2023-12-26 17:21:22,650][105620] Updated weights for policy 1, policy_version 273831 (0.0005) [2023-12-26 17:21:22,756][105692] Updated weights for policy 0, policy_version 273708 (0.0010) [2023-12-26 17:21:22,808][105692] Updated weights for policy 0, policy_version 273718 (0.0009) [2023-12-26 17:21:22,864][105692] Updated weights for policy 0, policy_version 273728 (0.0010) [2023-12-26 17:21:23,301][105620] Updated weights for policy 1, policy_version 273841 (0.0010) [2023-12-26 17:21:23,362][105620] Updated weights for policy 1, policy_version 273851 (0.0010) [2023-12-26 17:21:23,423][105620] Updated weights for policy 1, policy_version 273861 (0.0010) [2023-12-26 17:21:23,615][105692] Updated weights for policy 0, policy_version 273738 (0.0010) [2023-12-26 17:21:23,667][105692] Updated weights for policy 0, policy_version 273748 (0.0009) [2023-12-26 17:21:23,714][105692] Updated weights for policy 0, policy_version 273758 (0.0008) [2023-12-26 17:21:23,772][105692] Updated weights for policy 0, policy_version 273768 (0.0009) [2023-12-26 17:21:24,051][105620] Updated weights for policy 1, policy_version 273871 (0.0007) [2023-12-26 17:21:24,106][105620] Updated weights for policy 1, policy_version 273881 (0.0006) [2023-12-26 17:21:24,155][105620] Updated weights for policy 1, policy_version 273891 (0.0005) [2023-12-26 17:21:24,643][105692] Updated weights for policy 0, policy_version 273778 (0.0010) [2023-12-26 17:21:24,694][105692] Updated weights for policy 0, policy_version 273788 (0.0010) [2023-12-26 17:21:24,741][105620] Updated weights for policy 1, policy_version 273901 (0.0005) [2023-12-26 17:21:24,744][105692] Updated weights for policy 0, policy_version 273798 (0.0010) [2023-12-26 17:21:24,789][105620] Updated weights for policy 1, policy_version 273911 (0.0006) [2023-12-26 17:21:24,835][105620] Updated weights for policy 1, policy_version 273921 (0.0008) [2023-12-26 17:21:25,445][105620] Updated weights for policy 1, policy_version 273931 (0.0008) [2023-12-26 17:21:25,489][105620] Updated weights for policy 1, policy_version 273941 (0.0005) [2023-12-26 17:21:25,511][105692] Updated weights for policy 0, policy_version 273808 (0.0008) [2023-12-26 17:21:25,542][105620] Updated weights for policy 1, policy_version 273951 (0.0007) [2023-12-26 17:21:25,559][105692] Updated weights for policy 0, policy_version 273818 (0.0006) [2023-12-26 17:21:25,616][105692] Updated weights for policy 0, policy_version 273828 (0.0009) [2023-12-26 17:21:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 140255232. Throughput: 0: 9464.5, 1: 9968.1. Samples: 140267068. Policy #0 lag: (min: 13.0, avg: 15.6, max: 45.0) [2023-12-26 17:21:26,063][104569] Avg episode reward: [(0, '6525.178'), (1, '8897.190')] [2023-12-26 17:21:26,079][105620] Updated weights for policy 1, policy_version 273961 (0.0005) [2023-12-26 17:21:26,139][105620] Updated weights for policy 1, policy_version 273971 (0.0005) [2023-12-26 17:21:26,203][105620] Updated weights for policy 1, policy_version 273981 (0.0005) [2023-12-26 17:21:26,254][105620] Updated weights for policy 1, policy_version 273991 (0.0005) [2023-12-26 17:21:26,384][105692] Updated weights for policy 0, policy_version 273838 (0.0007) [2023-12-26 17:21:26,452][105692] Updated weights for policy 0, policy_version 273848 (0.0006) [2023-12-26 17:21:26,508][105692] Updated weights for policy 0, policy_version 273858 (0.0008) [2023-12-26 17:21:26,831][105620] Updated weights for policy 1, policy_version 274001 (0.0009) [2023-12-26 17:21:26,890][105620] Updated weights for policy 1, policy_version 274011 (0.0009) [2023-12-26 17:21:26,952][105620] Updated weights for policy 1, policy_version 274021 (0.0009) [2023-12-26 17:21:27,184][105692] Updated weights for policy 0, policy_version 273868 (0.0008) [2023-12-26 17:21:27,241][105692] Updated weights for policy 0, policy_version 273878 (0.0010) [2023-12-26 17:21:27,300][105692] Updated weights for policy 0, policy_version 273888 (0.0007) [2023-12-26 17:21:27,696][105620] Updated weights for policy 1, policy_version 274031 (0.0009) [2023-12-26 17:21:27,753][105620] Updated weights for policy 1, policy_version 274041 (0.0008) [2023-12-26 17:21:27,800][105620] Updated weights for policy 1, policy_version 274051 (0.0009) [2023-12-26 17:21:27,950][105692] Updated weights for policy 0, policy_version 273898 (0.0008) [2023-12-26 17:21:28,004][105692] Updated weights for policy 0, policy_version 273908 (0.0007) [2023-12-26 17:21:28,055][105692] Updated weights for policy 0, policy_version 273918 (0.0005) [2023-12-26 17:21:28,115][105692] Updated weights for policy 0, policy_version 273928 (0.0006) [2023-12-26 17:21:28,423][105620] Updated weights for policy 1, policy_version 274061 (0.0008) [2023-12-26 17:21:28,482][105620] Updated weights for policy 1, policy_version 274071 (0.0008) [2023-12-26 17:21:28,545][105620] Updated weights for policy 1, policy_version 274081 (0.0009) [2023-12-26 17:21:28,719][105692] Updated weights for policy 0, policy_version 273938 (0.0005) [2023-12-26 17:21:28,776][105692] Updated weights for policy 0, policy_version 273948 (0.0005) [2023-12-26 17:21:28,843][105692] Updated weights for policy 0, policy_version 273958 (0.0005) [2023-12-26 17:21:29,359][105620] Updated weights for policy 1, policy_version 274091 (0.0007) [2023-12-26 17:21:29,423][105620] Updated weights for policy 1, policy_version 274101 (0.0007) [2023-12-26 17:21:29,435][105692] Updated weights for policy 0, policy_version 273968 (0.0010) [2023-12-26 17:21:29,482][105620] Updated weights for policy 1, policy_version 274111 (0.0006) [2023-12-26 17:21:29,495][105692] Updated weights for policy 0, policy_version 273978 (0.0010) [2023-12-26 17:21:29,555][105692] Updated weights for policy 0, policy_version 273988 (0.0009) [2023-12-26 17:21:30,087][105620] Updated weights for policy 1, policy_version 274121 (0.0005) [2023-12-26 17:21:30,142][105620] Updated weights for policy 1, policy_version 274131 (0.0008) [2023-12-26 17:21:30,201][105620] Updated weights for policy 1, policy_version 274141 (0.0008) [2023-12-26 17:21:30,255][105692] Updated weights for policy 0, policy_version 273998 (0.0009) [2023-12-26 17:21:30,261][105620] Updated weights for policy 1, policy_version 274151 (0.0007) [2023-12-26 17:21:30,313][105692] Updated weights for policy 0, policy_version 274008 (0.0011) [2023-12-26 17:21:30,372][105692] Updated weights for policy 0, policy_version 274018 (0.0011) [2023-12-26 17:21:30,986][105620] Updated weights for policy 1, policy_version 274161 (0.0008) [2023-12-26 17:21:31,041][105620] Updated weights for policy 1, policy_version 274171 (0.0008) [2023-12-26 17:21:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 140353536. Throughput: 0: 9554.6, 1: 9992.0. Samples: 140328692. Policy #0 lag: (min: 13.0, avg: 15.6, max: 45.0) [2023-12-26 17:21:31,063][104569] Avg episode reward: [(0, '4355.330'), (1, '9077.342')] [2023-12-26 17:21:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000274024_70164480.pth... [2023-12-26 17:21:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000272936_69885952.pth [2023-12-26 17:21:31,097][105620] Updated weights for policy 1, policy_version 274181 (0.0008) [2023-12-26 17:21:31,104][105692] Updated weights for policy 0, policy_version 274028 (0.0010) [2023-12-26 17:21:31,110][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000274184_70197248.pth... [2023-12-26 17:21:31,115][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000273000_69894144.pth [2023-12-26 17:21:31,170][105692] Updated weights for policy 0, policy_version 274038 (0.0011) [2023-12-26 17:21:31,233][105692] Updated weights for policy 0, policy_version 274048 (0.0011) [2023-12-26 17:21:31,849][105620] Updated weights for policy 1, policy_version 274191 (0.0006) [2023-12-26 17:21:31,908][105620] Updated weights for policy 1, policy_version 274201 (0.0006) [2023-12-26 17:21:31,922][105692] Updated weights for policy 0, policy_version 274058 (0.0010) [2023-12-26 17:21:31,966][105620] Updated weights for policy 1, policy_version 274211 (0.0005) [2023-12-26 17:21:31,979][105692] Updated weights for policy 0, policy_version 274068 (0.0007) [2023-12-26 17:21:32,047][105692] Updated weights for policy 0, policy_version 274078 (0.0008) [2023-12-26 17:21:32,112][105692] Updated weights for policy 0, policy_version 274088 (0.0009) [2023-12-26 17:21:32,660][105620] Updated weights for policy 1, policy_version 274221 (0.0007) [2023-12-26 17:21:32,721][105620] Updated weights for policy 1, policy_version 274231 (0.0009) [2023-12-26 17:21:32,771][105692] Updated weights for policy 0, policy_version 274098 (0.0005) [2023-12-26 17:21:32,785][105620] Updated weights for policy 1, policy_version 274241 (0.0010) [2023-12-26 17:21:32,820][105692] Updated weights for policy 0, policy_version 274108 (0.0005) [2023-12-26 17:21:32,873][105692] Updated weights for policy 0, policy_version 274118 (0.0006) [2023-12-26 17:21:33,471][105692] Updated weights for policy 0, policy_version 274128 (0.0005) [2023-12-26 17:21:33,516][105692] Updated weights for policy 0, policy_version 274138 (0.0005) [2023-12-26 17:21:33,564][105692] Updated weights for policy 0, policy_version 274148 (0.0005) [2023-12-26 17:21:33,564][105620] Updated weights for policy 1, policy_version 274251 (0.0009) [2023-12-26 17:21:33,616][105620] Updated weights for policy 1, policy_version 274261 (0.0010) [2023-12-26 17:21:33,668][105620] Updated weights for policy 1, policy_version 274271 (0.0010) [2023-12-26 17:21:34,123][105692] Updated weights for policy 0, policy_version 274158 (0.0006) [2023-12-26 17:21:34,189][105692] Updated weights for policy 0, policy_version 274168 (0.0010) [2023-12-26 17:21:34,248][105692] Updated weights for policy 0, policy_version 274178 (0.0010) [2023-12-26 17:21:34,349][105620] Updated weights for policy 1, policy_version 274281 (0.0010) [2023-12-26 17:21:34,415][105620] Updated weights for policy 1, policy_version 274291 (0.0009) [2023-12-26 17:21:34,480][105620] Updated weights for policy 1, policy_version 274301 (0.0009) [2023-12-26 17:21:34,531][105620] Updated weights for policy 1, policy_version 274311 (0.0009) [2023-12-26 17:21:34,948][105692] Updated weights for policy 0, policy_version 274188 (0.0008) [2023-12-26 17:21:34,996][105692] Updated weights for policy 0, policy_version 274198 (0.0005) [2023-12-26 17:21:35,054][105692] Updated weights for policy 0, policy_version 274209 (0.0007) [2023-12-26 17:21:35,336][105620] Updated weights for policy 1, policy_version 274321 (0.0009) [2023-12-26 17:21:35,393][105620] Updated weights for policy 1, policy_version 274331 (0.0009) [2023-12-26 17:21:35,445][105620] Updated weights for policy 1, policy_version 274341 (0.0009) [2023-12-26 17:21:35,720][105692] Updated weights for policy 0, policy_version 274219 (0.0006) [2023-12-26 17:21:35,777][105692] Updated weights for policy 0, policy_version 274229 (0.0009) [2023-12-26 17:21:35,838][105692] Updated weights for policy 0, policy_version 274239 (0.0009) [2023-12-26 17:21:36,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 140460032. Throughput: 0: 9567.2, 1: 9984.3. Samples: 140449452. Policy #0 lag: (min: 13.0, avg: 15.6, max: 45.0) [2023-12-26 17:21:36,062][104569] Avg episode reward: [(0, '6563.304'), (1, '9262.483')] [2023-12-26 17:21:36,210][105620] Updated weights for policy 1, policy_version 274351 (0.0007) [2023-12-26 17:21:36,279][105620] Updated weights for policy 1, policy_version 274361 (0.0007) [2023-12-26 17:21:36,340][105620] Updated weights for policy 1, policy_version 274371 (0.0008) [2023-12-26 17:21:36,558][105692] Updated weights for policy 0, policy_version 274249 (0.0007) [2023-12-26 17:21:36,621][105692] Updated weights for policy 0, policy_version 274259 (0.0006) [2023-12-26 17:21:36,691][105692] Updated weights for policy 0, policy_version 274269 (0.0006) [2023-12-26 17:21:36,752][105692] Updated weights for policy 0, policy_version 274279 (0.0006) [2023-12-26 17:21:37,054][105620] Updated weights for policy 1, policy_version 274381 (0.0009) [2023-12-26 17:21:37,112][105620] Updated weights for policy 1, policy_version 274391 (0.0010) [2023-12-26 17:21:37,172][105620] Updated weights for policy 1, policy_version 274401 (0.0009) [2023-12-26 17:21:37,277][105692] Updated weights for policy 0, policy_version 274289 (0.0008) [2023-12-26 17:21:37,332][105692] Updated weights for policy 0, policy_version 274299 (0.0009) [2023-12-26 17:21:37,393][105692] Updated weights for policy 0, policy_version 274309 (0.0009) [2023-12-26 17:21:37,882][105620] Updated weights for policy 1, policy_version 274411 (0.0009) [2023-12-26 17:21:37,940][105620] Updated weights for policy 1, policy_version 274422 (0.0010) [2023-12-26 17:21:37,987][105692] Updated weights for policy 0, policy_version 274319 (0.0006) [2023-12-26 17:21:37,997][105620] Updated weights for policy 1, policy_version 274432 (0.0009) [2023-12-26 17:21:38,021][105586] KL-divergence is very high: 767.0042 [2023-12-26 17:21:38,051][105692] Updated weights for policy 0, policy_version 274329 (0.0006) [2023-12-26 17:21:38,114][105692] Updated weights for policy 0, policy_version 274339 (0.0006) [2023-12-26 17:21:38,736][105692] Updated weights for policy 0, policy_version 274349 (0.0007) [2023-12-26 17:21:38,804][105692] Updated weights for policy 0, policy_version 274359 (0.0007) [2023-12-26 17:21:38,840][105620] Updated weights for policy 1, policy_version 274442 (0.0008) [2023-12-26 17:21:38,867][105692] Updated weights for policy 0, policy_version 274369 (0.0009) [2023-12-26 17:21:38,902][105620] Updated weights for policy 1, policy_version 274452 (0.0008) [2023-12-26 17:21:38,964][105620] Updated weights for policy 1, policy_version 274462 (0.0008) [2023-12-26 17:21:39,030][105620] Updated weights for policy 1, policy_version 274472 (0.0007) [2023-12-26 17:21:39,563][105692] Updated weights for policy 0, policy_version 274379 (0.0009) [2023-12-26 17:21:39,626][105692] Updated weights for policy 0, policy_version 274389 (0.0006) [2023-12-26 17:21:39,694][105692] Updated weights for policy 0, policy_version 274399 (0.0010) [2023-12-26 17:21:39,694][105620] Updated weights for policy 1, policy_version 274482 (0.0005) [2023-12-26 17:21:39,758][105620] Updated weights for policy 1, policy_version 274492 (0.0006) [2023-12-26 17:21:39,816][105620] Updated weights for policy 1, policy_version 274502 (0.0006) [2023-12-26 17:21:40,421][105692] Updated weights for policy 0, policy_version 274409 (0.0011) [2023-12-26 17:21:40,428][105620] Updated weights for policy 1, policy_version 274512 (0.0008) [2023-12-26 17:21:40,470][105692] Updated weights for policy 0, policy_version 274419 (0.0010) [2023-12-26 17:21:40,477][105620] Updated weights for policy 1, policy_version 274522 (0.0006) [2023-12-26 17:21:40,526][105692] Updated weights for policy 0, policy_version 274429 (0.0010) [2023-12-26 17:21:40,530][105620] Updated weights for policy 1, policy_version 274532 (0.0005) [2023-12-26 17:21:40,579][105692] Updated weights for policy 0, policy_version 274439 (0.0010) [2023-12-26 17:21:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 140558336. Throughput: 0: 9675.2, 1: 9931.6. Samples: 140568704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:21:41,063][104569] Avg episode reward: [(0, '6788.504'), (1, '8896.497')] [2023-12-26 17:21:41,285][105620] Updated weights for policy 1, policy_version 274542 (0.0007) [2023-12-26 17:21:41,337][105692] Updated weights for policy 0, policy_version 274449 (0.0009) [2023-12-26 17:21:41,352][105620] Updated weights for policy 1, policy_version 274552 (0.0008) [2023-12-26 17:21:41,408][105692] Updated weights for policy 0, policy_version 274459 (0.0008) [2023-12-26 17:21:41,422][105620] Updated weights for policy 1, policy_version 274562 (0.0007) [2023-12-26 17:21:41,465][105692] Updated weights for policy 0, policy_version 274469 (0.0009) [2023-12-26 17:21:42,092][105620] Updated weights for policy 1, policy_version 274572 (0.0006) [2023-12-26 17:21:42,155][105620] Updated weights for policy 1, policy_version 274582 (0.0009) [2023-12-26 17:21:42,221][105620] Updated weights for policy 1, policy_version 274592 (0.0009) [2023-12-26 17:21:42,294][105692] Updated weights for policy 0, policy_version 274479 (0.0009) [2023-12-26 17:21:42,356][105692] Updated weights for policy 0, policy_version 274489 (0.0008) [2023-12-26 17:21:42,425][105692] Updated weights for policy 0, policy_version 274499 (0.0009) [2023-12-26 17:21:42,888][105620] Updated weights for policy 1, policy_version 274602 (0.0008) [2023-12-26 17:21:42,939][105620] Updated weights for policy 1, policy_version 274612 (0.0005) [2023-12-26 17:21:42,985][105620] Updated weights for policy 1, policy_version 274622 (0.0005) [2023-12-26 17:21:43,038][105620] Updated weights for policy 1, policy_version 274632 (0.0005) [2023-12-26 17:21:43,228][105692] Updated weights for policy 0, policy_version 274509 (0.0009) [2023-12-26 17:21:43,279][105692] Updated weights for policy 0, policy_version 274519 (0.0010) [2023-12-26 17:21:43,330][105692] Updated weights for policy 0, policy_version 274529 (0.0009) [2023-12-26 17:21:43,610][105620] Updated weights for policy 1, policy_version 274642 (0.0009) [2023-12-26 17:21:43,671][105620] Updated weights for policy 1, policy_version 274652 (0.0008) [2023-12-26 17:21:43,730][105620] Updated weights for policy 1, policy_version 274662 (0.0009) [2023-12-26 17:21:44,120][105692] Updated weights for policy 0, policy_version 274539 (0.0009) [2023-12-26 17:21:44,185][105692] Updated weights for policy 0, policy_version 274549 (0.0007) [2023-12-26 17:21:44,243][105692] Updated weights for policy 0, policy_version 274559 (0.0009) [2023-12-26 17:21:44,468][105620] Updated weights for policy 1, policy_version 274672 (0.0009) [2023-12-26 17:21:44,515][105620] Updated weights for policy 1, policy_version 274682 (0.0009) [2023-12-26 17:21:44,570][105620] Updated weights for policy 1, policy_version 274692 (0.0009) [2023-12-26 17:21:45,009][105692] Updated weights for policy 0, policy_version 274569 (0.0009) [2023-12-26 17:21:45,072][105692] Updated weights for policy 0, policy_version 274579 (0.0009) [2023-12-26 17:21:45,132][105692] Updated weights for policy 0, policy_version 274589 (0.0009) [2023-12-26 17:21:45,195][105692] Updated weights for policy 0, policy_version 274599 (0.0009) [2023-12-26 17:21:45,328][105620] Updated weights for policy 1, policy_version 274702 (0.0009) [2023-12-26 17:21:45,382][105620] Updated weights for policy 1, policy_version 274712 (0.0008) [2023-12-26 17:21:45,433][105620] Updated weights for policy 1, policy_version 274722 (0.0009) [2023-12-26 17:21:45,968][105692] Updated weights for policy 0, policy_version 274609 (0.0010) [2023-12-26 17:21:46,015][105692] Updated weights for policy 0, policy_version 274619 (0.0009) [2023-12-26 17:21:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 140648448. Throughput: 0: 9627.0, 1: 9960.3. Samples: 140625780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:21:46,062][104569] Avg episode reward: [(0, '7506.088'), (1, '8714.685')] [2023-12-26 17:21:46,063][105692] Updated weights for policy 0, policy_version 274630 (0.0009) [2023-12-26 17:21:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000274728_70336512.pth... [2023-12-26 17:21:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000273576_70041600.pth [2023-12-26 17:21:46,071][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_000274728_70336512.pth [2023-12-26 17:21:46,074][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000274632_70320128.pth... [2023-12-26 17:21:46,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000273480_70025216.pth [2023-12-26 17:21:46,079][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_000274632_70320128.pth [2023-12-26 17:21:46,173][105620] Updated weights for policy 1, policy_version 274732 (0.0007) [2023-12-26 17:21:46,231][105620] Updated weights for policy 1, policy_version 274742 (0.0009) [2023-12-26 17:21:46,280][105620] Updated weights for policy 1, policy_version 274752 (0.0008) [2023-12-26 17:21:46,887][105692] Updated weights for policy 0, policy_version 274640 (0.0009) [2023-12-26 17:21:46,946][105692] Updated weights for policy 0, policy_version 274650 (0.0006) [2023-12-26 17:21:46,991][105620] Updated weights for policy 1, policy_version 274762 (0.0010) [2023-12-26 17:21:47,010][105692] Updated weights for policy 0, policy_version 274660 (0.0007) [2023-12-26 17:21:47,046][105620] Updated weights for policy 1, policy_version 274772 (0.0008) [2023-12-26 17:21:47,099][105620] Updated weights for policy 1, policy_version 274782 (0.0008) [2023-12-26 17:21:47,157][105620] Updated weights for policy 1, policy_version 274792 (0.0009) [2023-12-26 17:21:47,724][105692] Updated weights for policy 0, policy_version 274670 (0.0006) [2023-12-26 17:21:47,775][105692] Updated weights for policy 0, policy_version 274680 (0.0009) [2023-12-26 17:21:47,827][105692] Updated weights for policy 0, policy_version 274690 (0.0009) [2023-12-26 17:21:47,898][105620] Updated weights for policy 1, policy_version 274802 (0.0008) [2023-12-26 17:21:47,949][105620] Updated weights for policy 1, policy_version 274812 (0.0009) [2023-12-26 17:21:48,004][105620] Updated weights for policy 1, policy_version 274822 (0.0009) [2023-12-26 17:21:48,590][105692] Updated weights for policy 0, policy_version 274700 (0.0009) [2023-12-26 17:21:48,639][105692] Updated weights for policy 0, policy_version 274710 (0.0009) [2023-12-26 17:21:48,687][105692] Updated weights for policy 0, policy_version 274720 (0.0009) [2023-12-26 17:21:48,781][105620] Updated weights for policy 1, policy_version 274832 (0.0009) [2023-12-26 17:21:48,839][105620] Updated weights for policy 1, policy_version 274842 (0.0009) [2023-12-26 17:21:48,899][105620] Updated weights for policy 1, policy_version 274852 (0.0008) [2023-12-26 17:21:49,485][105692] Updated weights for policy 0, policy_version 274730 (0.0009) [2023-12-26 17:21:49,537][105692] Updated weights for policy 0, policy_version 274740 (0.0010) [2023-12-26 17:21:49,583][105692] Updated weights for policy 0, policy_version 274750 (0.0010) [2023-12-26 17:21:49,642][105692] Updated weights for policy 0, policy_version 274760 (0.0010) [2023-12-26 17:21:49,649][105620] Updated weights for policy 1, policy_version 274862 (0.0008) [2023-12-26 17:21:49,715][105620] Updated weights for policy 1, policy_version 274872 (0.0008) [2023-12-26 17:21:49,777][105620] Updated weights for policy 1, policy_version 274882 (0.0009) [2023-12-26 17:21:50,452][105692] Updated weights for policy 0, policy_version 274770 (0.0010) [2023-12-26 17:21:50,510][105692] Updated weights for policy 0, policy_version 274780 (0.0009) [2023-12-26 17:21:50,552][105620] Updated weights for policy 1, policy_version 274892 (0.0008) [2023-12-26 17:21:50,570][105692] Updated weights for policy 0, policy_version 274790 (0.0010) [2023-12-26 17:21:50,619][105620] Updated weights for policy 1, policy_version 274902 (0.0008) [2023-12-26 17:21:50,673][105620] Updated weights for policy 1, policy_version 274912 (0.0008) [2023-12-26 17:21:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 140746752. Throughput: 0: 9664.8, 1: 9834.8. Samples: 140737468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:21:51,062][104569] Avg episode reward: [(0, '6811.103'), (1, '8621.621')] [2023-12-26 17:21:51,351][105692] Updated weights for policy 0, policy_version 274800 (0.0010) [2023-12-26 17:21:51,419][105692] Updated weights for policy 0, policy_version 274810 (0.0010) [2023-12-26 17:21:51,464][105620] Updated weights for policy 1, policy_version 274922 (0.0008) [2023-12-26 17:21:51,482][105692] Updated weights for policy 0, policy_version 274820 (0.0011) [2023-12-26 17:21:51,514][105620] Updated weights for policy 1, policy_version 274932 (0.0007) [2023-12-26 17:21:51,566][105620] Updated weights for policy 1, policy_version 274942 (0.0008) [2023-12-26 17:21:51,606][105586] KL-divergence is very high: 109.8695 [2023-12-26 17:21:51,627][105620] Updated weights for policy 1, policy_version 274952 (0.0008) [2023-12-26 17:21:52,215][105692] Updated weights for policy 0, policy_version 274830 (0.0011) [2023-12-26 17:21:52,267][105692] Updated weights for policy 0, policy_version 274840 (0.0010) [2023-12-26 17:21:52,330][105692] Updated weights for policy 0, policy_version 274850 (0.0011) [2023-12-26 17:21:52,409][105620] Updated weights for policy 1, policy_version 274962 (0.0011) [2023-12-26 17:21:52,458][105620] Updated weights for policy 1, policy_version 274972 (0.0010) [2023-12-26 17:21:52,512][105620] Updated weights for policy 1, policy_version 274983 (0.0007) [2023-12-26 17:21:53,071][105692] Updated weights for policy 0, policy_version 274860 (0.0010) [2023-12-26 17:21:53,123][105692] Updated weights for policy 0, policy_version 274871 (0.0010) [2023-12-26 17:21:53,175][105692] Updated weights for policy 0, policy_version 274881 (0.0006) [2023-12-26 17:21:53,257][105620] Updated weights for policy 1, policy_version 274993 (0.0008) [2023-12-26 17:21:53,316][105620] Updated weights for policy 1, policy_version 275003 (0.0008) [2023-12-26 17:21:53,344][105586] KL-divergence is very high: 234.6156 [2023-12-26 17:21:53,358][105586] KL-divergence is very high: 400.3628 [2023-12-26 17:21:53,384][105620] Updated weights for policy 1, policy_version 275013 (0.0008) [2023-12-26 17:21:53,396][105586] KL-divergence is very high: 385.3899 [2023-12-26 17:21:53,830][105692] Updated weights for policy 0, policy_version 274891 (0.0007) [2023-12-26 17:21:53,889][105692] Updated weights for policy 0, policy_version 274901 (0.0005) [2023-12-26 17:21:53,940][105692] Updated weights for policy 0, policy_version 274911 (0.0007) [2023-12-26 17:21:54,002][105620] Updated weights for policy 1, policy_version 275023 (0.0005) [2023-12-26 17:21:54,058][105620] Updated weights for policy 1, policy_version 275033 (0.0005) [2023-12-26 17:21:54,107][105620] Updated weights for policy 1, policy_version 275043 (0.0005) [2023-12-26 17:21:54,673][105692] Updated weights for policy 0, policy_version 274921 (0.0009) [2023-12-26 17:21:54,688][105620] Updated weights for policy 1, policy_version 275053 (0.0006) [2023-12-26 17:21:54,727][105692] Updated weights for policy 0, policy_version 274931 (0.0006) [2023-12-26 17:21:54,742][105620] Updated weights for policy 1, policy_version 275063 (0.0009) [2023-12-26 17:21:54,784][105692] Updated weights for policy 0, policy_version 274941 (0.0006) [2023-12-26 17:21:54,802][105620] Updated weights for policy 1, policy_version 275073 (0.0008) [2023-12-26 17:21:54,845][105692] Updated weights for policy 0, policy_version 274951 (0.0008) [2023-12-26 17:21:55,526][105692] Updated weights for policy 0, policy_version 274961 (0.0009) [2023-12-26 17:21:55,573][105692] Updated weights for policy 0, policy_version 274971 (0.0008) [2023-12-26 17:21:55,576][105620] Updated weights for policy 1, policy_version 275083 (0.0008) [2023-12-26 17:21:55,623][105692] Updated weights for policy 0, policy_version 274981 (0.0008) [2023-12-26 17:21:55,635][105620] Updated weights for policy 1, policy_version 275093 (0.0007) [2023-12-26 17:21:55,704][105620] Updated weights for policy 1, policy_version 275103 (0.0005) [2023-12-26 17:21:56,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 140845056. Throughput: 0: 9653.7, 1: 9859.7. Samples: 140854136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:21:56,063][104569] Avg episode reward: [(0, '5974.250'), (1, '8526.555')] [2023-12-26 17:21:56,224][105692] Updated weights for policy 0, policy_version 274991 (0.0008) [2023-12-26 17:21:56,283][105692] Updated weights for policy 0, policy_version 275001 (0.0009) [2023-12-26 17:21:56,331][105692] Updated weights for policy 0, policy_version 275011 (0.0009) [2023-12-26 17:21:56,392][105620] Updated weights for policy 1, policy_version 275113 (0.0006) [2023-12-26 17:21:56,438][105620] Updated weights for policy 1, policy_version 275123 (0.0009) [2023-12-26 17:21:56,485][105620] Updated weights for policy 1, policy_version 275133 (0.0008) [2023-12-26 17:21:56,537][105620] Updated weights for policy 1, policy_version 275144 (0.0010) [2023-12-26 17:21:56,965][105692] Updated weights for policy 0, policy_version 275021 (0.0009) [2023-12-26 17:21:57,030][105692] Updated weights for policy 0, policy_version 275031 (0.0010) [2023-12-26 17:21:57,097][105692] Updated weights for policy 0, policy_version 275041 (0.0010) [2023-12-26 17:21:57,163][105620] Updated weights for policy 1, policy_version 275154 (0.0005) [2023-12-26 17:21:57,217][105620] Updated weights for policy 1, policy_version 275164 (0.0005) [2023-12-26 17:21:57,278][105620] Updated weights for policy 1, policy_version 275174 (0.0005) [2023-12-26 17:21:57,814][105692] Updated weights for policy 0, policy_version 275051 (0.0008) [2023-12-26 17:21:57,879][105692] Updated weights for policy 0, policy_version 275061 (0.0005) [2023-12-26 17:21:57,948][105692] Updated weights for policy 0, policy_version 275071 (0.0006) [2023-12-26 17:21:58,005][105620] Updated weights for policy 1, policy_version 275184 (0.0005) [2023-12-26 17:21:58,070][105620] Updated weights for policy 1, policy_version 275194 (0.0005) [2023-12-26 17:21:58,140][105620] Updated weights for policy 1, policy_version 275204 (0.0008) [2023-12-26 17:21:58,544][105692] Updated weights for policy 0, policy_version 275081 (0.0006) [2023-12-26 17:21:58,610][105692] Updated weights for policy 0, policy_version 275091 (0.0008) [2023-12-26 17:21:58,674][105692] Updated weights for policy 0, policy_version 275101 (0.0008) [2023-12-26 17:21:58,748][105692] Updated weights for policy 0, policy_version 275111 (0.0007) [2023-12-26 17:21:59,000][105620] Updated weights for policy 1, policy_version 275214 (0.0008) [2023-12-26 17:21:59,055][105620] Updated weights for policy 1, policy_version 275224 (0.0009) [2023-12-26 17:21:59,103][105620] Updated weights for policy 1, policy_version 275234 (0.0009) [2023-12-26 17:21:59,509][105692] Updated weights for policy 0, policy_version 275121 (0.0008) [2023-12-26 17:21:59,568][105692] Updated weights for policy 0, policy_version 275131 (0.0009) [2023-12-26 17:21:59,619][105692] Updated weights for policy 0, policy_version 275141 (0.0010) [2023-12-26 17:21:59,812][105620] Updated weights for policy 1, policy_version 275244 (0.0007) [2023-12-26 17:21:59,882][105620] Updated weights for policy 1, policy_version 275254 (0.0007) [2023-12-26 17:21:59,946][105620] Updated weights for policy 1, policy_version 275264 (0.0008) [2023-12-26 17:22:00,364][105692] Updated weights for policy 0, policy_version 275151 (0.0011) [2023-12-26 17:22:00,422][105692] Updated weights for policy 0, policy_version 275161 (0.0010) [2023-12-26 17:22:00,487][105692] Updated weights for policy 0, policy_version 275171 (0.0010) [2023-12-26 17:22:00,614][105620] Updated weights for policy 1, policy_version 275274 (0.0009) [2023-12-26 17:22:00,667][105620] Updated weights for policy 1, policy_version 275284 (0.0009) [2023-12-26 17:22:00,722][105620] Updated weights for policy 1, policy_version 275294 (0.0010) [2023-12-26 17:22:00,780][105620] Updated weights for policy 1, policy_version 275304 (0.0010) [2023-12-26 17:22:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 140943360. Throughput: 0: 9782.7, 1: 9854.3. Samples: 140914832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:22:01,062][104569] Avg episode reward: [(0, '6557.334'), (1, '8802.920')] [2023-12-26 17:22:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000275176_70459392.pth... [2023-12-26 17:22:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000275304_70483968.pth... [2023-12-26 17:22:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000274024_70164480.pth [2023-12-26 17:22:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000274184_70197248.pth [2023-12-26 17:22:01,136][105692] Updated weights for policy 0, policy_version 275181 (0.0011) [2023-12-26 17:22:01,200][105692] Updated weights for policy 0, policy_version 275191 (0.0010) [2023-12-26 17:22:01,255][105692] Updated weights for policy 0, policy_version 275201 (0.0010) [2023-12-26 17:22:01,500][105620] Updated weights for policy 1, policy_version 275314 (0.0011) [2023-12-26 17:22:01,548][105620] Updated weights for policy 1, policy_version 275324 (0.0010) [2023-12-26 17:22:01,593][105620] Updated weights for policy 1, policy_version 275334 (0.0010) [2023-12-26 17:22:01,986][105692] Updated weights for policy 0, policy_version 275211 (0.0011) [2023-12-26 17:22:02,040][105692] Updated weights for policy 0, policy_version 275221 (0.0010) [2023-12-26 17:22:02,098][105692] Updated weights for policy 0, policy_version 275231 (0.0010) [2023-12-26 17:22:02,369][105620] Updated weights for policy 1, policy_version 275344 (0.0010) [2023-12-26 17:22:02,432][105620] Updated weights for policy 1, policy_version 275354 (0.0010) [2023-12-26 17:22:02,484][105620] Updated weights for policy 1, policy_version 275364 (0.0010) [2023-12-26 17:22:02,749][105692] Updated weights for policy 0, policy_version 275241 (0.0010) [2023-12-26 17:22:02,803][105692] Updated weights for policy 0, policy_version 275251 (0.0009) [2023-12-26 17:22:02,862][105692] Updated weights for policy 0, policy_version 275261 (0.0010) [2023-12-26 17:22:02,929][105692] Updated weights for policy 0, policy_version 275271 (0.0010) [2023-12-26 17:22:03,196][105620] Updated weights for policy 1, policy_version 275374 (0.0010) [2023-12-26 17:22:03,249][105620] Updated weights for policy 1, policy_version 275384 (0.0010) [2023-12-26 17:22:03,298][105620] Updated weights for policy 1, policy_version 275394 (0.0010) [2023-12-26 17:22:03,479][105692] Updated weights for policy 0, policy_version 275281 (0.0008) [2023-12-26 17:22:03,530][105692] Updated weights for policy 0, policy_version 275291 (0.0010) [2023-12-26 17:22:03,578][105692] Updated weights for policy 0, policy_version 275301 (0.0010) [2023-12-26 17:22:03,990][105620] Updated weights for policy 1, policy_version 275404 (0.0010) [2023-12-26 17:22:04,047][105620] Updated weights for policy 1, policy_version 275414 (0.0010) [2023-12-26 17:22:04,108][105620] Updated weights for policy 1, policy_version 275424 (0.0007) [2023-12-26 17:22:04,273][105692] Updated weights for policy 0, policy_version 275311 (0.0009) [2023-12-26 17:22:04,332][105692] Updated weights for policy 0, policy_version 275321 (0.0009) [2023-12-26 17:22:04,387][105692] Updated weights for policy 0, policy_version 275331 (0.0007) [2023-12-26 17:22:04,831][105620] Updated weights for policy 1, policy_version 275434 (0.0008) [2023-12-26 17:22:04,897][105620] Updated weights for policy 1, policy_version 275444 (0.0009) [2023-12-26 17:22:04,960][105620] Updated weights for policy 1, policy_version 275454 (0.0008) [2023-12-26 17:22:05,021][105620] Updated weights for policy 1, policy_version 275464 (0.0009) [2023-12-26 17:22:05,139][105692] Updated weights for policy 0, policy_version 275341 (0.0007) [2023-12-26 17:22:05,191][105692] Updated weights for policy 0, policy_version 275351 (0.0005) [2023-12-26 17:22:05,241][105692] Updated weights for policy 0, policy_version 275361 (0.0005) [2023-12-26 17:22:05,737][105620] Updated weights for policy 1, policy_version 275474 (0.0008) [2023-12-26 17:22:05,793][105620] Updated weights for policy 1, policy_version 275484 (0.0008) [2023-12-26 17:22:05,802][105692] Updated weights for policy 0, policy_version 275371 (0.0005) [2023-12-26 17:22:05,841][105620] Updated weights for policy 1, policy_version 275494 (0.0009) [2023-12-26 17:22:05,868][105692] Updated weights for policy 0, policy_version 275381 (0.0006) [2023-12-26 17:22:05,927][105692] Updated weights for policy 0, policy_version 275391 (0.0006) [2023-12-26 17:22:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 141049856. Throughput: 0: 9819.5, 1: 9792.9. Samples: 141033096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:22:06,063][104569] Avg episode reward: [(0, '6806.085'), (1, '9262.630')] [2023-12-26 17:22:06,552][105692] Updated weights for policy 0, policy_version 275401 (0.0008) [2023-12-26 17:22:06,589][105620] Updated weights for policy 1, policy_version 275504 (0.0009) [2023-12-26 17:22:06,606][105692] Updated weights for policy 0, policy_version 275411 (0.0010) [2023-12-26 17:22:06,645][105620] Updated weights for policy 1, policy_version 275514 (0.0008) [2023-12-26 17:22:06,663][105692] Updated weights for policy 0, policy_version 275421 (0.0011) [2023-12-26 17:22:06,702][105620] Updated weights for policy 1, policy_version 275524 (0.0006) [2023-12-26 17:22:06,720][105692] Updated weights for policy 0, policy_version 275431 (0.0010) [2023-12-26 17:22:07,426][105692] Updated weights for policy 0, policy_version 275441 (0.0009) [2023-12-26 17:22:07,487][105692] Updated weights for policy 0, policy_version 275451 (0.0010) [2023-12-26 17:22:07,505][105620] Updated weights for policy 1, policy_version 275534 (0.0007) [2023-12-26 17:22:07,536][105692] Updated weights for policy 0, policy_version 275461 (0.0006) [2023-12-26 17:22:07,557][105620] Updated weights for policy 1, policy_version 275544 (0.0008) [2023-12-26 17:22:07,608][105620] Updated weights for policy 1, policy_version 275554 (0.0009) [2023-12-26 17:22:08,081][105692] Updated weights for policy 0, policy_version 275471 (0.0005) [2023-12-26 17:22:08,140][105692] Updated weights for policy 0, policy_version 275481 (0.0005) [2023-12-26 17:22:08,193][105692] Updated weights for policy 0, policy_version 275491 (0.0005) [2023-12-26 17:22:08,522][105620] Updated weights for policy 1, policy_version 275564 (0.0009) [2023-12-26 17:22:08,577][105620] Updated weights for policy 1, policy_version 275574 (0.0008) [2023-12-26 17:22:08,632][105620] Updated weights for policy 1, policy_version 275584 (0.0008) [2023-12-26 17:22:08,791][105692] Updated weights for policy 0, policy_version 275501 (0.0008) [2023-12-26 17:22:08,855][105692] Updated weights for policy 0, policy_version 275511 (0.0008) [2023-12-26 17:22:08,919][105692] Updated weights for policy 0, policy_version 275521 (0.0009) [2023-12-26 17:22:09,476][105620] Updated weights for policy 1, policy_version 275594 (0.0008) [2023-12-26 17:22:09,526][105692] Updated weights for policy 0, policy_version 275531 (0.0009) [2023-12-26 17:22:09,539][105620] Updated weights for policy 1, policy_version 275604 (0.0010) [2023-12-26 17:22:09,593][105692] Updated weights for policy 0, policy_version 275541 (0.0008) [2023-12-26 17:22:09,605][105620] Updated weights for policy 1, policy_version 275614 (0.0010) [2023-12-26 17:22:09,653][105692] Updated weights for policy 0, policy_version 275551 (0.0010) [2023-12-26 17:22:09,671][105620] Updated weights for policy 1, policy_version 275624 (0.0010) [2023-12-26 17:22:10,289][105692] Updated weights for policy 0, policy_version 275561 (0.0010) [2023-12-26 17:22:10,355][105692] Updated weights for policy 0, policy_version 275571 (0.0011) [2023-12-26 17:22:10,406][105620] Updated weights for policy 1, policy_version 275634 (0.0011) [2023-12-26 17:22:10,411][105692] Updated weights for policy 0, policy_version 275581 (0.0011) [2023-12-26 17:22:10,458][105620] Updated weights for policy 1, policy_version 275644 (0.0010) [2023-12-26 17:22:10,477][105692] Updated weights for policy 0, policy_version 275591 (0.0011) [2023-12-26 17:22:10,517][105620] Updated weights for policy 1, policy_version 275654 (0.0010) [2023-12-26 17:22:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 141139968. Throughput: 0: 10033.0, 1: 9647.7. Samples: 141152696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:22:11,062][104569] Avg episode reward: [(0, '3873.614'), (1, '9171.309')] [2023-12-26 17:22:11,146][105620] Updated weights for policy 1, policy_version 275664 (0.0009) [2023-12-26 17:22:11,201][105620] Updated weights for policy 1, policy_version 275674 (0.0008) [2023-12-26 17:22:11,230][105692] Updated weights for policy 0, policy_version 275601 (0.0010) [2023-12-26 17:22:11,264][105620] Updated weights for policy 1, policy_version 275684 (0.0008) [2023-12-26 17:22:11,294][105692] Updated weights for policy 0, policy_version 275611 (0.0011) [2023-12-26 17:22:11,354][105692] Updated weights for policy 0, policy_version 275621 (0.0011) [2023-12-26 17:22:12,066][105620] Updated weights for policy 1, policy_version 275694 (0.0008) [2023-12-26 17:22:12,106][105692] Updated weights for policy 0, policy_version 275631 (0.0008) [2023-12-26 17:22:12,127][105620] Updated weights for policy 1, policy_version 275704 (0.0006) [2023-12-26 17:22:12,169][105692] Updated weights for policy 0, policy_version 275641 (0.0011) [2023-12-26 17:22:12,190][105620] Updated weights for policy 1, policy_version 275714 (0.0006) [2023-12-26 17:22:12,229][105692] Updated weights for policy 0, policy_version 275651 (0.0010) [2023-12-26 17:22:12,879][105620] Updated weights for policy 1, policy_version 275724 (0.0006) [2023-12-26 17:22:12,927][105620] Updated weights for policy 1, policy_version 275734 (0.0007) [2023-12-26 17:22:12,976][105620] Updated weights for policy 1, policy_version 275744 (0.0010) [2023-12-26 17:22:13,037][105692] Updated weights for policy 0, policy_version 275661 (0.0009) [2023-12-26 17:22:13,088][105692] Updated weights for policy 0, policy_version 275671 (0.0010) [2023-12-26 17:22:13,142][105692] Updated weights for policy 0, policy_version 275681 (0.0009) [2023-12-26 17:22:13,527][105620] Updated weights for policy 1, policy_version 275754 (0.0005) [2023-12-26 17:22:13,596][105620] Updated weights for policy 1, policy_version 275764 (0.0005) [2023-12-26 17:22:13,659][105620] Updated weights for policy 1, policy_version 275774 (0.0008) [2023-12-26 17:22:13,711][105620] Updated weights for policy 1, policy_version 275784 (0.0010) [2023-12-26 17:22:14,040][105692] Updated weights for policy 0, policy_version 275691 (0.0009) [2023-12-26 17:22:14,094][105692] Updated weights for policy 0, policy_version 275701 (0.0008) [2023-12-26 17:22:14,147][105692] Updated weights for policy 0, policy_version 275711 (0.0008) [2023-12-26 17:22:14,293][105620] Updated weights for policy 1, policy_version 275794 (0.0009) [2023-12-26 17:22:14,358][105620] Updated weights for policy 1, policy_version 275804 (0.0011) [2023-12-26 17:22:14,406][105620] Updated weights for policy 1, policy_version 275814 (0.0010) [2023-12-26 17:22:14,952][105692] Updated weights for policy 0, policy_version 275721 (0.0008) [2023-12-26 17:22:15,020][105692] Updated weights for policy 0, policy_version 275731 (0.0009) [2023-12-26 17:22:15,083][105692] Updated weights for policy 0, policy_version 275741 (0.0008) [2023-12-26 17:22:15,100][105620] Updated weights for policy 1, policy_version 275824 (0.0011) [2023-12-26 17:22:15,144][105692] Updated weights for policy 0, policy_version 275751 (0.0006) [2023-12-26 17:22:15,158][105620] Updated weights for policy 1, policy_version 275834 (0.0011) [2023-12-26 17:22:15,223][105620] Updated weights for policy 1, policy_version 275844 (0.0011) [2023-12-26 17:22:15,891][105692] Updated weights for policy 0, policy_version 275761 (0.0008) [2023-12-26 17:22:15,956][105692] Updated weights for policy 0, policy_version 275771 (0.0011) [2023-12-26 17:22:15,959][105620] Updated weights for policy 1, policy_version 275854 (0.0008) [2023-12-26 17:22:16,005][105620] Updated weights for policy 1, policy_version 275864 (0.0005) [2023-12-26 17:22:16,008][105692] Updated weights for policy 0, policy_version 275781 (0.0010) [2023-12-26 17:22:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 141238272. Throughput: 0: 9947.3, 1: 9659.7. Samples: 141211008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:22:16,062][104569] Avg episode reward: [(0, '3206.953'), (1, '9263.392')] [2023-12-26 17:22:16,068][105620] Updated weights for policy 1, policy_version 275874 (0.0006) [2023-12-26 17:22:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000275784_70615040.pth... [2023-12-26 17:22:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000274632_70320128.pth [2023-12-26 17:22:16,101][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000275880_70631424.pth... [2023-12-26 17:22:16,104][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000274728_70336512.pth [2023-12-26 17:22:16,685][105692] Updated weights for policy 0, policy_version 275791 (0.0006) [2023-12-26 17:22:16,710][105620] Updated weights for policy 1, policy_version 275884 (0.0008) [2023-12-26 17:22:16,734][105692] Updated weights for policy 0, policy_version 275801 (0.0009) [2023-12-26 17:22:16,770][105620] Updated weights for policy 1, policy_version 275894 (0.0005) [2023-12-26 17:22:16,782][105692] Updated weights for policy 0, policy_version 275811 (0.0010) [2023-12-26 17:22:16,830][105620] Updated weights for policy 1, policy_version 275904 (0.0005) [2023-12-26 17:22:17,483][105620] Updated weights for policy 1, policy_version 275914 (0.0006) [2023-12-26 17:22:17,495][105692] Updated weights for policy 0, policy_version 275821 (0.0010) [2023-12-26 17:22:17,540][105620] Updated weights for policy 1, policy_version 275924 (0.0011) [2023-12-26 17:22:17,547][105692] Updated weights for policy 0, policy_version 275831 (0.0010) [2023-12-26 17:22:17,588][105620] Updated weights for policy 1, policy_version 275934 (0.0010) [2023-12-26 17:22:17,595][105692] Updated weights for policy 0, policy_version 275841 (0.0010) [2023-12-26 17:22:17,633][105620] Updated weights for policy 1, policy_version 275944 (0.0010) [2023-12-26 17:22:18,346][105692] Updated weights for policy 0, policy_version 275851 (0.0010) [2023-12-26 17:22:18,363][105620] Updated weights for policy 1, policy_version 275954 (0.0008) [2023-12-26 17:22:18,402][105692] Updated weights for policy 0, policy_version 275861 (0.0011) [2023-12-26 17:22:18,426][105620] Updated weights for policy 1, policy_version 275964 (0.0010) [2023-12-26 17:22:18,451][105692] Updated weights for policy 0, policy_version 275871 (0.0010) [2023-12-26 17:22:18,471][105620] Updated weights for policy 1, policy_version 275974 (0.0010) [2023-12-26 17:22:19,213][105692] Updated weights for policy 0, policy_version 275881 (0.0010) [2023-12-26 17:22:19,215][105620] Updated weights for policy 1, policy_version 275984 (0.0011) [2023-12-26 17:22:19,275][105692] Updated weights for policy 0, policy_version 275891 (0.0008) [2023-12-26 17:22:19,279][105620] Updated weights for policy 1, policy_version 275994 (0.0011) [2023-12-26 17:22:19,343][105620] Updated weights for policy 1, policy_version 276004 (0.0010) [2023-12-26 17:22:19,349][105692] Updated weights for policy 0, policy_version 275901 (0.0006) [2023-12-26 17:22:19,416][105692] Updated weights for policy 0, policy_version 275911 (0.0009) [2023-12-26 17:22:20,132][105692] Updated weights for policy 0, policy_version 275921 (0.0008) [2023-12-26 17:22:20,142][105620] Updated weights for policy 1, policy_version 276014 (0.0007) [2023-12-26 17:22:20,184][105692] Updated weights for policy 0, policy_version 275931 (0.0008) [2023-12-26 17:22:20,194][105620] Updated weights for policy 1, policy_version 276024 (0.0005) [2023-12-26 17:22:20,236][105692] Updated weights for policy 0, policy_version 275941 (0.0008) [2023-12-26 17:22:20,251][105620] Updated weights for policy 1, policy_version 276034 (0.0006) [2023-12-26 17:22:20,879][105692] Updated weights for policy 0, policy_version 275951 (0.0006) [2023-12-26 17:22:20,942][105692] Updated weights for policy 0, policy_version 275961 (0.0006) [2023-12-26 17:22:21,010][105692] Updated weights for policy 0, policy_version 275971 (0.0009) [2023-12-26 17:22:21,038][105620] Updated weights for policy 1, policy_version 276044 (0.0006) [2023-12-26 17:22:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 141336576. Throughput: 0: 9795.3, 1: 9668.9. Samples: 141325340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:22:21,062][104569] Avg episode reward: [(0, '6340.957'), (1, '9174.503')] [2023-12-26 17:22:21,090][105620] Updated weights for policy 1, policy_version 276054 (0.0008) [2023-12-26 17:22:21,156][105620] Updated weights for policy 1, policy_version 276064 (0.0009) [2023-12-26 17:22:21,618][105692] Updated weights for policy 0, policy_version 275981 (0.0010) [2023-12-26 17:22:21,679][105692] Updated weights for policy 0, policy_version 275991 (0.0010) [2023-12-26 17:22:21,742][105692] Updated weights for policy 0, policy_version 276001 (0.0011) [2023-12-26 17:22:21,961][105620] Updated weights for policy 1, policy_version 276074 (0.0010) [2023-12-26 17:22:22,014][105620] Updated weights for policy 1, policy_version 276084 (0.0008) [2023-12-26 17:22:22,075][105620] Updated weights for policy 1, policy_version 276094 (0.0008) [2023-12-26 17:22:22,140][105620] Updated weights for policy 1, policy_version 276104 (0.0008) [2023-12-26 17:22:22,515][105692] Updated weights for policy 0, policy_version 276011 (0.0009) [2023-12-26 17:22:22,582][105692] Updated weights for policy 0, policy_version 276021 (0.0009) [2023-12-26 17:22:22,636][105692] Updated weights for policy 0, policy_version 276032 (0.0009) [2023-12-26 17:22:22,902][105620] Updated weights for policy 1, policy_version 276114 (0.0006) [2023-12-26 17:22:22,973][105620] Updated weights for policy 1, policy_version 276124 (0.0008) [2023-12-26 17:22:23,040][105620] Updated weights for policy 1, policy_version 276134 (0.0007) [2023-12-26 17:22:23,459][105692] Updated weights for policy 0, policy_version 276042 (0.0009) [2023-12-26 17:22:23,515][105692] Updated weights for policy 0, policy_version 276052 (0.0009) [2023-12-26 17:22:23,570][105692] Updated weights for policy 0, policy_version 276062 (0.0007) [2023-12-26 17:22:23,580][105620] Updated weights for policy 1, policy_version 276144 (0.0007) [2023-12-26 17:22:23,630][105692] Updated weights for policy 0, policy_version 276072 (0.0006) [2023-12-26 17:22:23,640][105620] Updated weights for policy 1, policy_version 276154 (0.0006) [2023-12-26 17:22:23,700][105620] Updated weights for policy 1, policy_version 276164 (0.0009) [2023-12-26 17:22:24,348][105692] Updated weights for policy 0, policy_version 276082 (0.0009) [2023-12-26 17:22:24,393][105620] Updated weights for policy 1, policy_version 276174 (0.0008) [2023-12-26 17:22:24,399][105692] Updated weights for policy 0, policy_version 276092 (0.0009) [2023-12-26 17:22:24,448][105692] Updated weights for policy 0, policy_version 276102 (0.0006) [2023-12-26 17:22:24,450][105620] Updated weights for policy 1, policy_version 276184 (0.0008) [2023-12-26 17:22:24,496][105620] Updated weights for policy 1, policy_version 276194 (0.0008) [2023-12-26 17:22:25,221][105692] Updated weights for policy 0, policy_version 276112 (0.0008) [2023-12-26 17:22:25,257][105620] Updated weights for policy 1, policy_version 276204 (0.0008) [2023-12-26 17:22:25,274][105692] Updated weights for policy 0, policy_version 276122 (0.0007) [2023-12-26 17:22:25,318][105620] Updated weights for policy 1, policy_version 276214 (0.0007) [2023-12-26 17:22:25,332][105692] Updated weights for policy 0, policy_version 276132 (0.0006) [2023-12-26 17:22:25,376][105620] Updated weights for policy 1, policy_version 276224 (0.0008) [2023-12-26 17:22:26,059][105692] Updated weights for policy 0, policy_version 276142 (0.0006) [2023-12-26 17:22:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 141426688. Throughput: 0: 9703.8, 1: 9665.8. Samples: 141440332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:22:26,062][104569] Avg episode reward: [(0, '8216.235'), (1, '9174.375')] [2023-12-26 17:22:26,113][105692] Updated weights for policy 0, policy_version 276152 (0.0006) [2023-12-26 17:22:26,166][105620] Updated weights for policy 1, policy_version 276234 (0.0009) [2023-12-26 17:22:26,170][105692] Updated weights for policy 0, policy_version 276162 (0.0005) [2023-12-26 17:22:26,215][105620] Updated weights for policy 1, policy_version 276244 (0.0009) [2023-12-26 17:22:26,269][105620] Updated weights for policy 1, policy_version 276255 (0.0010) [2023-12-26 17:22:26,727][105692] Updated weights for policy 0, policy_version 276172 (0.0005) [2023-12-26 17:22:26,788][105692] Updated weights for policy 0, policy_version 276182 (0.0006) [2023-12-26 17:22:26,845][105692] Updated weights for policy 0, policy_version 276192 (0.0010) [2023-12-26 17:22:27,134][105620] Updated weights for policy 1, policy_version 276265 (0.0010) [2023-12-26 17:22:27,197][105620] Updated weights for policy 1, policy_version 276275 (0.0005) [2023-12-26 17:22:27,264][105620] Updated weights for policy 1, policy_version 276285 (0.0007) [2023-12-26 17:22:27,327][105620] Updated weights for policy 1, policy_version 276295 (0.0009) [2023-12-26 17:22:27,425][105692] Updated weights for policy 0, policy_version 276202 (0.0009) [2023-12-26 17:22:27,472][105692] Updated weights for policy 0, policy_version 276212 (0.0005) [2023-12-26 17:22:27,530][105692] Updated weights for policy 0, policy_version 276222 (0.0005) [2023-12-26 17:22:27,585][105692] Updated weights for policy 0, policy_version 276232 (0.0009) [2023-12-26 17:22:28,058][105620] Updated weights for policy 1, policy_version 276305 (0.0010) [2023-12-26 17:22:28,119][105620] Updated weights for policy 1, policy_version 276315 (0.0009) [2023-12-26 17:22:28,123][105692] Updated weights for policy 0, policy_version 276242 (0.0005) [2023-12-26 17:22:28,171][105620] Updated weights for policy 1, policy_version 276325 (0.0009) [2023-12-26 17:22:28,181][105692] Updated weights for policy 0, policy_version 276252 (0.0007) [2023-12-26 17:22:28,242][105692] Updated weights for policy 0, policy_version 276262 (0.0010) [2023-12-26 17:22:28,950][105620] Updated weights for policy 1, policy_version 276335 (0.0007) [2023-12-26 17:22:28,956][105692] Updated weights for policy 0, policy_version 276272 (0.0010) [2023-12-26 17:22:29,003][105620] Updated weights for policy 1, policy_version 276345 (0.0007) [2023-12-26 17:22:29,013][105692] Updated weights for policy 0, policy_version 276282 (0.0006) [2023-12-26 17:22:29,063][105620] Updated weights for policy 1, policy_version 276355 (0.0006) [2023-12-26 17:22:29,069][105692] Updated weights for policy 0, policy_version 276292 (0.0008) [2023-12-26 17:22:29,785][105692] Updated weights for policy 0, policy_version 276302 (0.0008) [2023-12-26 17:22:29,797][105620] Updated weights for policy 1, policy_version 276365 (0.0007) [2023-12-26 17:22:29,847][105692] Updated weights for policy 0, policy_version 276312 (0.0009) [2023-12-26 17:22:29,863][105620] Updated weights for policy 1, policy_version 276375 (0.0007) [2023-12-26 17:22:29,906][105692] Updated weights for policy 0, policy_version 276322 (0.0009) [2023-12-26 17:22:29,930][105620] Updated weights for policy 1, policy_version 276385 (0.0008) [2023-12-26 17:22:30,554][105620] Updated weights for policy 1, policy_version 276395 (0.0011) [2023-12-26 17:22:30,621][105620] Updated weights for policy 1, policy_version 276405 (0.0010) [2023-12-26 17:22:30,639][105692] Updated weights for policy 0, policy_version 276332 (0.0008) [2023-12-26 17:22:30,686][105692] Updated weights for policy 0, policy_version 276342 (0.0008) [2023-12-26 17:22:30,688][105620] Updated weights for policy 1, policy_version 276415 (0.0010) [2023-12-26 17:22:30,736][105692] Updated weights for policy 0, policy_version 276352 (0.0008) [2023-12-26 17:22:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 141533184. Throughput: 0: 9881.1, 1: 9565.1. Samples: 141500856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:22:31,063][104569] Avg episode reward: [(0, '9176.767'), (1, '9171.980')] [2023-12-26 17:22:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000276360_70762496.pth... [2023-12-26 17:22:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000276424_70770688.pth... [2023-12-26 17:22:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000275176_70459392.pth [2023-12-26 17:22:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000275304_70483968.pth [2023-12-26 17:22:31,283][105620] Updated weights for policy 1, policy_version 276425 (0.0010) [2023-12-26 17:22:31,339][105620] Updated weights for policy 1, policy_version 276435 (0.0007) [2023-12-26 17:22:31,348][105586] KL-divergence is very high: 107.6553 [2023-12-26 17:22:31,410][105620] Updated weights for policy 1, policy_version 276445 (0.0008) [2023-12-26 17:22:31,410][105586] KL-divergence is very high: 131.4269 [2023-12-26 17:22:31,465][105692] Updated weights for policy 0, policy_version 276362 (0.0007) [2023-12-26 17:22:31,465][105586] KL-divergence is very high: 116.1557 [2023-12-26 17:22:31,475][105620] Updated weights for policy 1, policy_version 276455 (0.0010) [2023-12-26 17:22:31,526][105692] Updated weights for policy 0, policy_version 276372 (0.0007) [2023-12-26 17:22:31,582][105692] Updated weights for policy 0, policy_version 276382 (0.0010) [2023-12-26 17:22:31,651][105692] Updated weights for policy 0, policy_version 276392 (0.0011) [2023-12-26 17:22:32,179][105620] Updated weights for policy 1, policy_version 276465 (0.0008) [2023-12-26 17:22:32,233][105620] Updated weights for policy 1, policy_version 276475 (0.0008) [2023-12-26 17:22:32,289][105620] Updated weights for policy 1, policy_version 276485 (0.0008) [2023-12-26 17:22:32,379][105692] Updated weights for policy 0, policy_version 276402 (0.0011) [2023-12-26 17:22:32,438][105692] Updated weights for policy 0, policy_version 276412 (0.0011) [2023-12-26 17:22:32,502][105692] Updated weights for policy 0, policy_version 276422 (0.0010) [2023-12-26 17:22:33,065][105620] Updated weights for policy 1, policy_version 276495 (0.0009) [2023-12-26 17:22:33,116][105620] Updated weights for policy 1, policy_version 276505 (0.0010) [2023-12-26 17:22:33,171][105620] Updated weights for policy 1, policy_version 276515 (0.0010) [2023-12-26 17:22:33,204][105692] Updated weights for policy 0, policy_version 276432 (0.0006) [2023-12-26 17:22:33,264][105692] Updated weights for policy 0, policy_version 276442 (0.0005) [2023-12-26 17:22:33,331][105692] Updated weights for policy 0, policy_version 276452 (0.0005) [2023-12-26 17:22:33,826][105692] Updated weights for policy 0, policy_version 276462 (0.0007) [2023-12-26 17:22:33,876][105620] Updated weights for policy 1, policy_version 276525 (0.0008) [2023-12-26 17:22:33,880][105692] Updated weights for policy 0, policy_version 276472 (0.0010) [2023-12-26 17:22:33,927][105620] Updated weights for policy 1, policy_version 276535 (0.0005) [2023-12-26 17:22:33,931][105692] Updated weights for policy 0, policy_version 276482 (0.0010) [2023-12-26 17:22:33,987][105620] Updated weights for policy 1, policy_version 276545 (0.0005) [2023-12-26 17:22:34,536][105692] Updated weights for policy 0, policy_version 276492 (0.0008) [2023-12-26 17:22:34,560][105620] Updated weights for policy 1, policy_version 276555 (0.0007) [2023-12-26 17:22:34,597][105692] Updated weights for policy 0, policy_version 276502 (0.0007) [2023-12-26 17:22:34,623][105620] Updated weights for policy 1, policy_version 276565 (0.0010) [2023-12-26 17:22:34,659][105692] Updated weights for policy 0, policy_version 276512 (0.0010) [2023-12-26 17:22:34,682][105620] Updated weights for policy 1, policy_version 276575 (0.0008) [2023-12-26 17:22:35,381][105692] Updated weights for policy 0, policy_version 276522 (0.0011) [2023-12-26 17:22:35,401][105620] Updated weights for policy 1, policy_version 276585 (0.0010) [2023-12-26 17:22:35,439][105692] Updated weights for policy 0, policy_version 276532 (0.0006) [2023-12-26 17:22:35,465][105620] Updated weights for policy 1, policy_version 276595 (0.0009) [2023-12-26 17:22:35,496][105692] Updated weights for policy 0, policy_version 276542 (0.0007) [2023-12-26 17:22:35,524][105620] Updated weights for policy 1, policy_version 276605 (0.0010) [2023-12-26 17:22:35,557][105692] Updated weights for policy 0, policy_version 276552 (0.0010) [2023-12-26 17:22:35,575][105620] Updated weights for policy 1, policy_version 276615 (0.0010) [2023-12-26 17:22:36,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 141631488. Throughput: 0: 10010.7, 1: 9667.1. Samples: 141622972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:22:36,063][104569] Avg episode reward: [(0, '8457.963'), (1, '9172.274')] [2023-12-26 17:22:36,166][105692] Updated weights for policy 0, policy_version 276562 (0.0007) [2023-12-26 17:22:36,227][105692] Updated weights for policy 0, policy_version 276572 (0.0006) [2023-12-26 17:22:36,267][105620] Updated weights for policy 1, policy_version 276625 (0.0009) [2023-12-26 17:22:36,290][105692] Updated weights for policy 0, policy_version 276582 (0.0007) [2023-12-26 17:22:36,328][105620] Updated weights for policy 1, policy_version 276635 (0.0008) [2023-12-26 17:22:36,397][105620] Updated weights for policy 1, policy_version 276645 (0.0008) [2023-12-26 17:22:36,913][105692] Updated weights for policy 0, policy_version 276592 (0.0008) [2023-12-26 17:22:36,968][105692] Updated weights for policy 0, policy_version 276602 (0.0008) [2023-12-26 17:22:37,024][105692] Updated weights for policy 0, policy_version 276612 (0.0008) [2023-12-26 17:22:37,208][105620] Updated weights for policy 1, policy_version 276655 (0.0010) [2023-12-26 17:22:37,260][105620] Updated weights for policy 1, policy_version 276665 (0.0010) [2023-12-26 17:22:37,311][105620] Updated weights for policy 1, policy_version 276675 (0.0010) [2023-12-26 17:22:37,783][105692] Updated weights for policy 0, policy_version 276622 (0.0007) [2023-12-26 17:22:37,835][105692] Updated weights for policy 0, policy_version 276632 (0.0005) [2023-12-26 17:22:37,882][105692] Updated weights for policy 0, policy_version 276642 (0.0005) [2023-12-26 17:22:37,964][105620] Updated weights for policy 1, policy_version 276685 (0.0008) [2023-12-26 17:22:38,017][105620] Updated weights for policy 1, policy_version 276695 (0.0005) [2023-12-26 17:22:38,074][105620] Updated weights for policy 1, policy_version 276705 (0.0009) [2023-12-26 17:22:38,476][105692] Updated weights for policy 0, policy_version 276652 (0.0006) [2023-12-26 17:22:38,521][105692] Updated weights for policy 0, policy_version 276662 (0.0008) [2023-12-26 17:22:38,570][105692] Updated weights for policy 0, policy_version 276672 (0.0008) [2023-12-26 17:22:38,827][105620] Updated weights for policy 1, policy_version 276715 (0.0009) [2023-12-26 17:22:38,897][105620] Updated weights for policy 1, policy_version 276725 (0.0010) [2023-12-26 17:22:38,959][105620] Updated weights for policy 1, policy_version 276735 (0.0010) [2023-12-26 17:22:39,260][105692] Updated weights for policy 0, policy_version 276682 (0.0007) [2023-12-26 17:22:39,321][105692] Updated weights for policy 0, policy_version 276692 (0.0009) [2023-12-26 17:22:39,388][105692] Updated weights for policy 0, policy_version 276702 (0.0009) [2023-12-26 17:22:39,451][105692] Updated weights for policy 0, policy_version 276712 (0.0009) [2023-12-26 17:22:39,748][105620] Updated weights for policy 1, policy_version 276745 (0.0010) [2023-12-26 17:22:39,795][105620] Updated weights for policy 1, policy_version 276755 (0.0008) [2023-12-26 17:22:39,860][105620] Updated weights for policy 1, policy_version 276765 (0.0009) [2023-12-26 17:22:39,926][105620] Updated weights for policy 1, policy_version 276775 (0.0008) [2023-12-26 17:22:40,207][105692] Updated weights for policy 0, policy_version 276722 (0.0006) [2023-12-26 17:22:40,254][105692] Updated weights for policy 0, policy_version 276732 (0.0006) [2023-12-26 17:22:40,303][105692] Updated weights for policy 0, policy_version 276742 (0.0005) [2023-12-26 17:22:40,680][105620] Updated weights for policy 1, policy_version 276785 (0.0009) [2023-12-26 17:22:40,744][105620] Updated weights for policy 1, policy_version 276795 (0.0009) [2023-12-26 17:22:40,802][105620] Updated weights for policy 1, policy_version 276805 (0.0010) [2023-12-26 17:22:40,945][105692] Updated weights for policy 0, policy_version 276752 (0.0006) [2023-12-26 17:22:41,003][105692] Updated weights for policy 0, policy_version 276762 (0.0007) [2023-12-26 17:22:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 141729792. Throughput: 0: 10081.3, 1: 9611.5. Samples: 141740308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:22:41,063][104569] Avg episode reward: [(0, '7996.031'), (1, '9262.975')] [2023-12-26 17:22:41,065][105692] Updated weights for policy 0, policy_version 276772 (0.0010) [2023-12-26 17:22:41,617][105620] Updated weights for policy 1, policy_version 276815 (0.0009) [2023-12-26 17:22:41,680][105620] Updated weights for policy 1, policy_version 276825 (0.0009) [2023-12-26 17:22:41,742][105620] Updated weights for policy 1, policy_version 276835 (0.0008) [2023-12-26 17:22:41,872][105692] Updated weights for policy 0, policy_version 276782 (0.0009) [2023-12-26 17:22:41,931][105692] Updated weights for policy 0, policy_version 276792 (0.0006) [2023-12-26 17:22:41,993][105692] Updated weights for policy 0, policy_version 276802 (0.0008) [2023-12-26 17:22:42,457][105620] Updated weights for policy 1, policy_version 276845 (0.0007) [2023-12-26 17:22:42,503][105620] Updated weights for policy 1, policy_version 276855 (0.0008) [2023-12-26 17:22:42,553][105620] Updated weights for policy 1, policy_version 276865 (0.0008) [2023-12-26 17:22:42,736][105692] Updated weights for policy 0, policy_version 276812 (0.0008) [2023-12-26 17:22:42,786][105692] Updated weights for policy 0, policy_version 276822 (0.0009) [2023-12-26 17:22:42,835][105692] Updated weights for policy 0, policy_version 276832 (0.0009) [2023-12-26 17:22:43,265][105620] Updated weights for policy 1, policy_version 276875 (0.0009) [2023-12-26 17:22:43,319][105620] Updated weights for policy 1, policy_version 276885 (0.0009) [2023-12-26 17:22:43,368][105620] Updated weights for policy 1, policy_version 276895 (0.0009) [2023-12-26 17:22:43,616][105692] Updated weights for policy 0, policy_version 276842 (0.0009) [2023-12-26 17:22:43,677][105692] Updated weights for policy 0, policy_version 276852 (0.0008) [2023-12-26 17:22:43,738][105692] Updated weights for policy 0, policy_version 276862 (0.0009) [2023-12-26 17:22:43,785][105692] Updated weights for policy 0, policy_version 276872 (0.0009) [2023-12-26 17:22:44,128][105620] Updated weights for policy 1, policy_version 276905 (0.0008) [2023-12-26 17:22:44,183][105620] Updated weights for policy 1, policy_version 276915 (0.0009) [2023-12-26 17:22:44,248][105620] Updated weights for policy 1, policy_version 276925 (0.0009) [2023-12-26 17:22:44,302][105620] Updated weights for policy 1, policy_version 276935 (0.0009) [2023-12-26 17:22:44,534][105692] Updated weights for policy 0, policy_version 276882 (0.0007) [2023-12-26 17:22:44,599][105692] Updated weights for policy 0, policy_version 276892 (0.0005) [2023-12-26 17:22:44,663][105692] Updated weights for policy 0, policy_version 276902 (0.0005) [2023-12-26 17:22:45,094][105620] Updated weights for policy 1, policy_version 276945 (0.0010) [2023-12-26 17:22:45,103][105586] KL-divergence is very high: 103.8328 [2023-12-26 17:22:45,155][105586] KL-divergence is very high: 140.8512 [2023-12-26 17:22:45,161][105620] Updated weights for policy 1, policy_version 276955 (0.0009) [2023-12-26 17:22:45,205][105586] KL-divergence is very high: 125.6645 [2023-12-26 17:22:45,225][105620] Updated weights for policy 1, policy_version 276965 (0.0009) [2023-12-26 17:22:45,243][105692] Updated weights for policy 0, policy_version 276912 (0.0007) [2023-12-26 17:22:45,305][105692] Updated weights for policy 0, policy_version 276922 (0.0009) [2023-12-26 17:22:45,376][105692] Updated weights for policy 0, policy_version 276932 (0.0010) [2023-12-26 17:22:45,822][105620] Updated weights for policy 1, policy_version 276975 (0.0009) [2023-12-26 17:22:45,884][105620] Updated weights for policy 1, policy_version 276985 (0.0009) [2023-12-26 17:22:45,947][105620] Updated weights for policy 1, policy_version 276995 (0.0009) [2023-12-26 17:22:46,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 141828096. Throughput: 0: 10002.6, 1: 9608.2. Samples: 141797316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:22:46,062][104569] Avg episode reward: [(0, '8973.470'), (1, '9168.580')] [2023-12-26 17:22:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000277000_70918144.pth... [2023-12-26 17:22:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000276936_70909952.pth... [2023-12-26 17:22:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000275880_70631424.pth [2023-12-26 17:22:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000275784_70615040.pth [2023-12-26 17:22:46,123][105692] Updated weights for policy 0, policy_version 276942 (0.0009) [2023-12-26 17:22:46,182][105692] Updated weights for policy 0, policy_version 276952 (0.0010) [2023-12-26 17:22:46,237][105692] Updated weights for policy 0, policy_version 276962 (0.0009) [2023-12-26 17:22:46,702][105620] Updated weights for policy 1, policy_version 277005 (0.0007) [2023-12-26 17:22:46,768][105620] Updated weights for policy 1, policy_version 277015 (0.0005) [2023-12-26 17:22:46,837][105620] Updated weights for policy 1, policy_version 277025 (0.0005) [2023-12-26 17:22:46,887][105692] Updated weights for policy 0, policy_version 276972 (0.0008) [2023-12-26 17:22:46,941][105692] Updated weights for policy 0, policy_version 276982 (0.0006) [2023-12-26 17:22:46,992][105692] Updated weights for policy 0, policy_version 276992 (0.0009) [2023-12-26 17:22:47,426][105620] Updated weights for policy 1, policy_version 277035 (0.0006) [2023-12-26 17:22:47,486][105620] Updated weights for policy 1, policy_version 277045 (0.0005) [2023-12-26 17:22:47,538][105620] Updated weights for policy 1, policy_version 277055 (0.0005) [2023-12-26 17:22:47,598][105692] Updated weights for policy 0, policy_version 277002 (0.0008) [2023-12-26 17:22:47,649][105692] Updated weights for policy 0, policy_version 277012 (0.0005) [2023-12-26 17:22:47,706][105692] Updated weights for policy 0, policy_version 277022 (0.0005) [2023-12-26 17:22:47,757][105692] Updated weights for policy 0, policy_version 277032 (0.0006) [2023-12-26 17:22:48,223][105620] Updated weights for policy 1, policy_version 277065 (0.0005) [2023-12-26 17:22:48,287][105620] Updated weights for policy 1, policy_version 277075 (0.0009) [2023-12-26 17:22:48,344][105620] Updated weights for policy 1, policy_version 277085 (0.0007) [2023-12-26 17:22:48,364][105692] Updated weights for policy 0, policy_version 277042 (0.0009) [2023-12-26 17:22:48,393][105620] Updated weights for policy 1, policy_version 277095 (0.0007) [2023-12-26 17:22:48,429][105692] Updated weights for policy 0, policy_version 277052 (0.0008) [2023-12-26 17:22:48,493][105692] Updated weights for policy 0, policy_version 277062 (0.0009) [2023-12-26 17:22:49,167][105620] Updated weights for policy 1, policy_version 277105 (0.0007) [2023-12-26 17:22:49,186][105692] Updated weights for policy 0, policy_version 277072 (0.0008) [2023-12-26 17:22:49,219][105620] Updated weights for policy 1, policy_version 277115 (0.0007) [2023-12-26 17:22:49,249][105692] Updated weights for policy 0, policy_version 277082 (0.0008) [2023-12-26 17:22:49,282][105620] Updated weights for policy 1, policy_version 277125 (0.0009) [2023-12-26 17:22:49,310][105692] Updated weights for policy 0, policy_version 277092 (0.0008) [2023-12-26 17:22:50,003][105692] Updated weights for policy 0, policy_version 277102 (0.0008) [2023-12-26 17:22:50,042][105620] Updated weights for policy 1, policy_version 277135 (0.0006) [2023-12-26 17:22:50,063][105692] Updated weights for policy 0, policy_version 277112 (0.0008) [2023-12-26 17:22:50,099][105620] Updated weights for policy 1, policy_version 277145 (0.0008) [2023-12-26 17:22:50,129][105692] Updated weights for policy 0, policy_version 277122 (0.0007) [2023-12-26 17:22:50,160][105620] Updated weights for policy 1, policy_version 277155 (0.0006) [2023-12-26 17:22:50,854][105692] Updated weights for policy 0, policy_version 277132 (0.0008) [2023-12-26 17:22:50,910][105692] Updated weights for policy 0, policy_version 277142 (0.0008) [2023-12-26 17:22:50,925][105620] Updated weights for policy 1, policy_version 277165 (0.0009) [2023-12-26 17:22:50,969][105692] Updated weights for policy 0, policy_version 277152 (0.0007) [2023-12-26 17:22:50,980][105620] Updated weights for policy 1, policy_version 277175 (0.0007) [2023-12-26 17:22:51,044][105620] Updated weights for policy 1, policy_version 277185 (0.0007) [2023-12-26 17:22:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 141926400. Throughput: 0: 10028.4, 1: 9604.8. Samples: 141916592. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 17:22:51,062][104569] Avg episode reward: [(0, '9356.295'), (1, '9168.618')] [2023-12-26 17:22:51,684][105692] Updated weights for policy 0, policy_version 277162 (0.0008) [2023-12-26 17:22:51,755][105692] Updated weights for policy 0, policy_version 277172 (0.0009) [2023-12-26 17:22:51,810][105692] Updated weights for policy 0, policy_version 277182 (0.0009) [2023-12-26 17:22:51,856][105692] Updated weights for policy 0, policy_version 277192 (0.0008) [2023-12-26 17:22:51,907][105620] Updated weights for policy 1, policy_version 277195 (0.0009) [2023-12-26 17:22:51,962][105620] Updated weights for policy 1, policy_version 277205 (0.0008) [2023-12-26 17:22:52,014][105620] Updated weights for policy 1, policy_version 277215 (0.0009) [2023-12-26 17:22:52,631][105692] Updated weights for policy 0, policy_version 277202 (0.0009) [2023-12-26 17:22:52,693][105692] Updated weights for policy 0, policy_version 277212 (0.0009) [2023-12-26 17:22:52,753][105692] Updated weights for policy 0, policy_version 277222 (0.0008) [2023-12-26 17:22:52,778][105620] Updated weights for policy 1, policy_version 277225 (0.0009) [2023-12-26 17:22:52,836][105620] Updated weights for policy 1, policy_version 277235 (0.0009) [2023-12-26 17:22:52,889][105620] Updated weights for policy 1, policy_version 277245 (0.0009) [2023-12-26 17:22:52,942][105620] Updated weights for policy 1, policy_version 277255 (0.0008) [2023-12-26 17:22:53,424][105692] Updated weights for policy 0, policy_version 277232 (0.0005) [2023-12-26 17:22:53,496][105692] Updated weights for policy 0, policy_version 277242 (0.0005) [2023-12-26 17:22:53,565][105692] Updated weights for policy 0, policy_version 277252 (0.0005) [2023-12-26 17:22:53,620][105620] Updated weights for policy 1, policy_version 277265 (0.0006) [2023-12-26 17:22:53,667][105620] Updated weights for policy 1, policy_version 277275 (0.0005) [2023-12-26 17:22:53,723][105620] Updated weights for policy 1, policy_version 277285 (0.0005) [2023-12-26 17:22:54,154][105692] Updated weights for policy 0, policy_version 277262 (0.0008) [2023-12-26 17:22:54,213][105692] Updated weights for policy 0, policy_version 277272 (0.0009) [2023-12-26 17:22:54,259][105585] KL-divergence is very high: 105.6997 [2023-12-26 17:22:54,270][105692] Updated weights for policy 0, policy_version 277282 (0.0008) [2023-12-26 17:22:54,270][105585] KL-divergence is very high: 136.8116 [2023-12-26 17:22:54,276][105620] Updated weights for policy 1, policy_version 277295 (0.0007) [2023-12-26 17:22:54,319][105620] Updated weights for policy 1, policy_version 277305 (0.0006) [2023-12-26 17:22:54,373][105620] Updated weights for policy 1, policy_version 277315 (0.0009) [2023-12-26 17:22:55,028][105692] Updated weights for policy 0, policy_version 277292 (0.0007) [2023-12-26 17:22:55,084][105692] Updated weights for policy 0, policy_version 277302 (0.0008) [2023-12-26 17:22:55,140][105692] Updated weights for policy 0, policy_version 277312 (0.0008) [2023-12-26 17:22:55,173][105620] Updated weights for policy 1, policy_version 277325 (0.0010) [2023-12-26 17:22:55,222][105620] Updated weights for policy 1, policy_version 277335 (0.0009) [2023-12-26 17:22:55,270][105620] Updated weights for policy 1, policy_version 277345 (0.0005) [2023-12-26 17:22:55,821][105620] Updated weights for policy 1, policy_version 277355 (0.0005) [2023-12-26 17:22:55,884][105620] Updated weights for policy 1, policy_version 277365 (0.0008) [2023-12-26 17:22:55,950][105620] Updated weights for policy 1, policy_version 277375 (0.0010) [2023-12-26 17:22:56,016][105692] Updated weights for policy 0, policy_version 277322 (0.0007) [2023-12-26 17:22:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 142024704. Throughput: 0: 9885.5, 1: 9714.9. Samples: 142034716. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 17:22:56,062][104569] Avg episode reward: [(0, '9142.371'), (1, '9261.528')] [2023-12-26 17:22:56,068][105692] Updated weights for policy 0, policy_version 277332 (0.0005) [2023-12-26 17:22:56,126][105692] Updated weights for policy 0, policy_version 277342 (0.0007) [2023-12-26 17:22:56,173][105692] Updated weights for policy 0, policy_version 277352 (0.0007) [2023-12-26 17:22:56,552][105620] Updated weights for policy 1, policy_version 277385 (0.0010) [2023-12-26 17:22:56,602][105620] Updated weights for policy 1, policy_version 277395 (0.0007) [2023-12-26 17:22:56,660][105620] Updated weights for policy 1, policy_version 277405 (0.0010) [2023-12-26 17:22:56,712][105620] Updated weights for policy 1, policy_version 277415 (0.0010) [2023-12-26 17:22:56,811][105692] Updated weights for policy 0, policy_version 277362 (0.0005) [2023-12-26 17:22:56,862][105692] Updated weights for policy 0, policy_version 277372 (0.0005) [2023-12-26 17:22:56,911][105692] Updated weights for policy 0, policy_version 277382 (0.0008) [2023-12-26 17:22:57,423][105620] Updated weights for policy 1, policy_version 277425 (0.0006) [2023-12-26 17:22:57,475][105620] Updated weights for policy 1, policy_version 277435 (0.0005) [2023-12-26 17:22:57,491][105692] Updated weights for policy 0, policy_version 277392 (0.0005) [2023-12-26 17:22:57,538][105620] Updated weights for policy 1, policy_version 277445 (0.0006) [2023-12-26 17:22:57,550][105692] Updated weights for policy 0, policy_version 277402 (0.0006) [2023-12-26 17:22:57,612][105692] Updated weights for policy 0, policy_version 277412 (0.0005) [2023-12-26 17:22:58,219][105620] Updated weights for policy 1, policy_version 277455 (0.0009) [2023-12-26 17:22:58,272][105692] Updated weights for policy 0, policy_version 277422 (0.0006) [2023-12-26 17:22:58,276][105620] Updated weights for policy 1, policy_version 277465 (0.0008) [2023-12-26 17:22:58,330][105692] Updated weights for policy 0, policy_version 277432 (0.0008) [2023-12-26 17:22:58,338][105620] Updated weights for policy 1, policy_version 277475 (0.0007) [2023-12-26 17:22:58,397][105692] Updated weights for policy 0, policy_version 277442 (0.0008) [2023-12-26 17:22:59,104][105620] Updated weights for policy 1, policy_version 277485 (0.0008) [2023-12-26 17:22:59,162][105620] Updated weights for policy 1, policy_version 277495 (0.0008) [2023-12-26 17:22:59,219][105620] Updated weights for policy 1, policy_version 277505 (0.0008) [2023-12-26 17:22:59,262][105692] Updated weights for policy 0, policy_version 277452 (0.0008) [2023-12-26 17:22:59,329][105692] Updated weights for policy 0, policy_version 277462 (0.0011) [2023-12-26 17:22:59,392][105692] Updated weights for policy 0, policy_version 277472 (0.0011) [2023-12-26 17:22:59,938][105620] Updated weights for policy 1, policy_version 277515 (0.0008) [2023-12-26 17:22:59,990][105620] Updated weights for policy 1, policy_version 277525 (0.0008) [2023-12-26 17:23:00,048][105620] Updated weights for policy 1, policy_version 277535 (0.0008) [2023-12-26 17:23:00,154][105692] Updated weights for policy 0, policy_version 277482 (0.0011) [2023-12-26 17:23:00,214][105692] Updated weights for policy 0, policy_version 277492 (0.0011) [2023-12-26 17:23:00,266][105692] Updated weights for policy 0, policy_version 277502 (0.0010) [2023-12-26 17:23:00,317][105692] Updated weights for policy 0, policy_version 277512 (0.0010) [2023-12-26 17:23:00,831][105620] Updated weights for policy 1, policy_version 277545 (0.0008) [2023-12-26 17:23:00,882][105620] Updated weights for policy 1, policy_version 277555 (0.0009) [2023-12-26 17:23:00,923][105620] Updated weights for policy 1, policy_version 277565 (0.0005) [2023-12-26 17:23:00,974][105620] Updated weights for policy 1, policy_version 277575 (0.0005) [2023-12-26 17:23:00,987][105692] Updated weights for policy 0, policy_version 277522 (0.0006) [2023-12-26 17:23:01,039][105692] Updated weights for policy 0, policy_version 277532 (0.0007) [2023-12-26 17:23:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 142123008. Throughput: 0: 9971.8, 1: 9684.3. Samples: 142095536. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 17:23:01,063][104569] Avg episode reward: [(0, '9141.433'), (1, '9170.584')] [2023-12-26 17:23:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000277576_71065600.pth... [2023-12-26 17:23:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000276424_70770688.pth [2023-12-26 17:23:01,103][105692] Updated weights for policy 0, policy_version 277542 (0.0010) [2023-12-26 17:23:01,116][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000277544_71065600.pth... [2023-12-26 17:23:01,120][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000276360_70762496.pth [2023-12-26 17:23:01,701][105620] Updated weights for policy 1, policy_version 277585 (0.0009) [2023-12-26 17:23:01,756][105620] Updated weights for policy 1, policy_version 277595 (0.0006) [2023-12-26 17:23:01,821][105620] Updated weights for policy 1, policy_version 277605 (0.0007) [2023-12-26 17:23:01,848][105692] Updated weights for policy 0, policy_version 277552 (0.0008) [2023-12-26 17:23:01,919][105692] Updated weights for policy 0, policy_version 277562 (0.0009) [2023-12-26 17:23:01,993][105692] Updated weights for policy 0, policy_version 277572 (0.0010) [2023-12-26 17:23:02,509][105620] Updated weights for policy 1, policy_version 277615 (0.0009) [2023-12-26 17:23:02,569][105620] Updated weights for policy 1, policy_version 277625 (0.0009) [2023-12-26 17:23:02,634][105620] Updated weights for policy 1, policy_version 277635 (0.0009) [2023-12-26 17:23:02,738][105692] Updated weights for policy 0, policy_version 277582 (0.0010) [2023-12-26 17:23:02,785][105692] Updated weights for policy 0, policy_version 277592 (0.0009) [2023-12-26 17:23:02,841][105692] Updated weights for policy 0, policy_version 277602 (0.0008) [2023-12-26 17:23:03,439][105620] Updated weights for policy 1, policy_version 277645 (0.0008) [2023-12-26 17:23:03,454][105692] Updated weights for policy 0, policy_version 277612 (0.0009) [2023-12-26 17:23:03,493][105620] Updated weights for policy 1, policy_version 277655 (0.0005) [2023-12-26 17:23:03,509][105692] Updated weights for policy 0, policy_version 277622 (0.0008) [2023-12-26 17:23:03,545][105620] Updated weights for policy 1, policy_version 277665 (0.0005) [2023-12-26 17:23:03,555][105692] Updated weights for policy 0, policy_version 277632 (0.0008) [2023-12-26 17:23:04,159][105620] Updated weights for policy 1, policy_version 277675 (0.0005) [2023-12-26 17:23:04,216][105620] Updated weights for policy 1, policy_version 277685 (0.0007) [2023-12-26 17:23:04,264][105620] Updated weights for policy 1, policy_version 277695 (0.0009) [2023-12-26 17:23:04,349][105692] Updated weights for policy 0, policy_version 277642 (0.0009) [2023-12-26 17:23:04,407][105692] Updated weights for policy 0, policy_version 277652 (0.0009) [2023-12-26 17:23:04,466][105692] Updated weights for policy 0, policy_version 277662 (0.0008) [2023-12-26 17:23:04,523][105692] Updated weights for policy 0, policy_version 277672 (0.0008) [2023-12-26 17:23:05,021][105620] Updated weights for policy 1, policy_version 277705 (0.0010) [2023-12-26 17:23:05,072][105620] Updated weights for policy 1, policy_version 277715 (0.0010) [2023-12-26 17:23:05,126][105620] Updated weights for policy 1, policy_version 277725 (0.0008) [2023-12-26 17:23:05,181][105620] Updated weights for policy 1, policy_version 277735 (0.0010) [2023-12-26 17:23:05,294][105692] Updated weights for policy 0, policy_version 277682 (0.0008) [2023-12-26 17:23:05,347][105692] Updated weights for policy 0, policy_version 277692 (0.0009) [2023-12-26 17:23:05,406][105692] Updated weights for policy 0, policy_version 277702 (0.0008) [2023-12-26 17:23:05,831][105620] Updated weights for policy 1, policy_version 277745 (0.0008) [2023-12-26 17:23:05,898][105620] Updated weights for policy 1, policy_version 277755 (0.0009) [2023-12-26 17:23:05,960][105620] Updated weights for policy 1, policy_version 277765 (0.0011) [2023-12-26 17:23:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 142221312. Throughput: 0: 9990.8, 1: 9673.4. Samples: 142210228. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 17:23:06,062][104569] Avg episode reward: [(0, '9354.794'), (1, '8801.325')] [2023-12-26 17:23:06,123][105692] Updated weights for policy 0, policy_version 277712 (0.0008) [2023-12-26 17:23:06,184][105692] Updated weights for policy 0, policy_version 277722 (0.0008) [2023-12-26 17:23:06,249][105692] Updated weights for policy 0, policy_version 277732 (0.0010) [2023-12-26 17:23:06,583][105620] Updated weights for policy 1, policy_version 277775 (0.0009) [2023-12-26 17:23:06,642][105620] Updated weights for policy 1, policy_version 277785 (0.0011) [2023-12-26 17:23:06,704][105620] Updated weights for policy 1, policy_version 277795 (0.0006) [2023-12-26 17:23:07,123][105692] Updated weights for policy 0, policy_version 277742 (0.0010) [2023-12-26 17:23:07,185][105692] Updated weights for policy 0, policy_version 277752 (0.0010) [2023-12-26 17:23:07,238][105620] Updated weights for policy 1, policy_version 277805 (0.0007) [2023-12-26 17:23:07,243][105692] Updated weights for policy 0, policy_version 277762 (0.0009) [2023-12-26 17:23:07,289][105620] Updated weights for policy 1, policy_version 277815 (0.0006) [2023-12-26 17:23:07,351][105620] Updated weights for policy 1, policy_version 277825 (0.0007) [2023-12-26 17:23:07,954][105692] Updated weights for policy 0, policy_version 277772 (0.0009) [2023-12-26 17:23:07,990][105620] Updated weights for policy 1, policy_version 277835 (0.0008) [2023-12-26 17:23:08,001][105692] Updated weights for policy 0, policy_version 277782 (0.0007) [2023-12-26 17:23:08,048][105620] Updated weights for policy 1, policy_version 277845 (0.0005) [2023-12-26 17:23:08,054][105692] Updated weights for policy 0, policy_version 277792 (0.0008) [2023-12-26 17:23:08,111][105620] Updated weights for policy 1, policy_version 277855 (0.0005) [2023-12-26 17:23:08,817][105620] Updated weights for policy 1, policy_version 277865 (0.0007) [2023-12-26 17:23:08,827][105692] Updated weights for policy 0, policy_version 277802 (0.0009) [2023-12-26 17:23:08,877][105620] Updated weights for policy 1, policy_version 277875 (0.0007) [2023-12-26 17:23:08,884][105692] Updated weights for policy 0, policy_version 277812 (0.0006) [2023-12-26 17:23:08,935][105620] Updated weights for policy 1, policy_version 277885 (0.0006) [2023-12-26 17:23:08,949][105692] Updated weights for policy 0, policy_version 277822 (0.0007) [2023-12-26 17:23:08,990][105620] Updated weights for policy 1, policy_version 277895 (0.0005) [2023-12-26 17:23:09,003][105692] Updated weights for policy 0, policy_version 277832 (0.0008) [2023-12-26 17:23:09,721][105620] Updated weights for policy 1, policy_version 277905 (0.0009) [2023-12-26 17:23:09,776][105620] Updated weights for policy 1, policy_version 277915 (0.0008) [2023-12-26 17:23:09,801][105692] Updated weights for policy 0, policy_version 277842 (0.0010) [2023-12-26 17:23:09,837][105620] Updated weights for policy 1, policy_version 277925 (0.0007) [2023-12-26 17:23:09,868][105692] Updated weights for policy 0, policy_version 277852 (0.0011) [2023-12-26 17:23:09,931][105692] Updated weights for policy 0, policy_version 277862 (0.0010) [2023-12-26 17:23:10,522][105620] Updated weights for policy 1, policy_version 277935 (0.0006) [2023-12-26 17:23:10,576][105620] Updated weights for policy 1, policy_version 277945 (0.0007) [2023-12-26 17:23:10,638][105692] Updated weights for policy 0, policy_version 277872 (0.0010) [2023-12-26 17:23:10,638][105620] Updated weights for policy 1, policy_version 277955 (0.0007) [2023-12-26 17:23:10,702][105692] Updated weights for policy 0, policy_version 277882 (0.0005) [2023-12-26 17:23:10,783][105692] Updated weights for policy 0, policy_version 277892 (0.0008) [2023-12-26 17:23:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 142319616. Throughput: 0: 9938.4, 1: 9801.9. Samples: 142328648. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 17:23:11,062][104569] Avg episode reward: [(0, '9355.362'), (1, '8800.461')] [2023-12-26 17:23:11,326][105620] Updated weights for policy 1, policy_version 277965 (0.0009) [2023-12-26 17:23:11,391][105620] Updated weights for policy 1, policy_version 277975 (0.0008) [2023-12-26 17:23:11,438][105620] Updated weights for policy 1, policy_version 277985 (0.0009) [2023-12-26 17:23:11,503][105692] Updated weights for policy 0, policy_version 277902 (0.0010) [2023-12-26 17:23:11,562][105692] Updated weights for policy 0, policy_version 277912 (0.0009) [2023-12-26 17:23:11,631][105692] Updated weights for policy 0, policy_version 277922 (0.0009) [2023-12-26 17:23:12,216][105620] Updated weights for policy 1, policy_version 277995 (0.0009) [2023-12-26 17:23:12,284][105620] Updated weights for policy 1, policy_version 278005 (0.0009) [2023-12-26 17:23:12,348][105620] Updated weights for policy 1, policy_version 278015 (0.0008) [2023-12-26 17:23:12,411][105692] Updated weights for policy 0, policy_version 277932 (0.0009) [2023-12-26 17:23:12,468][105692] Updated weights for policy 0, policy_version 277942 (0.0008) [2023-12-26 17:23:12,525][105692] Updated weights for policy 0, policy_version 277952 (0.0008) [2023-12-26 17:23:13,098][105620] Updated weights for policy 1, policy_version 278025 (0.0008) [2023-12-26 17:23:13,172][105620] Updated weights for policy 1, policy_version 278035 (0.0008) [2023-12-26 17:23:13,237][105620] Updated weights for policy 1, policy_version 278045 (0.0008) [2023-12-26 17:23:13,298][105620] Updated weights for policy 1, policy_version 278055 (0.0008) [2023-12-26 17:23:13,314][105692] Updated weights for policy 0, policy_version 277962 (0.0008) [2023-12-26 17:23:13,379][105692] Updated weights for policy 0, policy_version 277972 (0.0010) [2023-12-26 17:23:13,452][105692] Updated weights for policy 0, policy_version 277982 (0.0011) [2023-12-26 17:23:13,500][105692] Updated weights for policy 0, policy_version 277992 (0.0010) [2023-12-26 17:23:13,902][105620] Updated weights for policy 1, policy_version 278065 (0.0005) [2023-12-26 17:23:13,953][105620] Updated weights for policy 1, policy_version 278075 (0.0006) [2023-12-26 17:23:14,000][105620] Updated weights for policy 1, policy_version 278085 (0.0009) [2023-12-26 17:23:14,156][105692] Updated weights for policy 0, policy_version 278002 (0.0005) [2023-12-26 17:23:14,226][105692] Updated weights for policy 0, policy_version 278012 (0.0007) [2023-12-26 17:23:14,297][105692] Updated weights for policy 0, policy_version 278022 (0.0010) [2023-12-26 17:23:14,635][105620] Updated weights for policy 1, policy_version 278095 (0.0006) [2023-12-26 17:23:14,695][105620] Updated weights for policy 1, policy_version 278105 (0.0005) [2023-12-26 17:23:14,747][105620] Updated weights for policy 1, policy_version 278115 (0.0008) [2023-12-26 17:23:14,995][105692] Updated weights for policy 0, policy_version 278032 (0.0011) [2023-12-26 17:23:15,061][105692] Updated weights for policy 0, policy_version 278042 (0.0011) [2023-12-26 17:23:15,124][105692] Updated weights for policy 0, policy_version 278052 (0.0011) [2023-12-26 17:23:15,442][105620] Updated weights for policy 1, policy_version 278125 (0.0007) [2023-12-26 17:23:15,506][105620] Updated weights for policy 1, policy_version 278135 (0.0006) [2023-12-26 17:23:15,560][105620] Updated weights for policy 1, policy_version 278145 (0.0008) [2023-12-26 17:23:15,852][105692] Updated weights for policy 0, policy_version 278062 (0.0009) [2023-12-26 17:23:15,897][105692] Updated weights for policy 0, policy_version 278072 (0.0010) [2023-12-26 17:23:15,948][105692] Updated weights for policy 0, policy_version 278082 (0.0010) [2023-12-26 17:23:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 142417920. Throughput: 0: 9782.5, 1: 9858.6. Samples: 142384708. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 17:23:16,063][104569] Avg episode reward: [(0, '9355.637'), (1, '8983.754')] [2023-12-26 17:23:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000278088_71204864.pth... [2023-12-26 17:23:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000278152_71213056.pth... [2023-12-26 17:23:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000276936_70909952.pth [2023-12-26 17:23:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000277000_70918144.pth [2023-12-26 17:23:16,156][105620] Updated weights for policy 1, policy_version 278155 (0.0009) [2023-12-26 17:23:16,220][105620] Updated weights for policy 1, policy_version 278165 (0.0008) [2023-12-26 17:23:16,287][105620] Updated weights for policy 1, policy_version 278175 (0.0008) [2023-12-26 17:23:16,690][105692] Updated weights for policy 0, policy_version 278092 (0.0008) [2023-12-26 17:23:16,747][105692] Updated weights for policy 0, policy_version 278102 (0.0006) [2023-12-26 17:23:16,800][105692] Updated weights for policy 0, policy_version 278112 (0.0005) [2023-12-26 17:23:17,085][105620] Updated weights for policy 1, policy_version 278185 (0.0009) [2023-12-26 17:23:17,135][105620] Updated weights for policy 1, policy_version 278195 (0.0008) [2023-12-26 17:23:17,191][105620] Updated weights for policy 1, policy_version 278205 (0.0008) [2023-12-26 17:23:17,254][105620] Updated weights for policy 1, policy_version 278215 (0.0008) [2023-12-26 17:23:17,458][105692] Updated weights for policy 0, policy_version 278122 (0.0006) [2023-12-26 17:23:17,508][105692] Updated weights for policy 0, policy_version 278132 (0.0008) [2023-12-26 17:23:17,569][105692] Updated weights for policy 0, policy_version 278142 (0.0009) [2023-12-26 17:23:17,629][105692] Updated weights for policy 0, policy_version 278152 (0.0009) [2023-12-26 17:23:18,015][105620] Updated weights for policy 1, policy_version 278225 (0.0009) [2023-12-26 17:23:18,081][105620] Updated weights for policy 1, policy_version 278235 (0.0011) [2023-12-26 17:23:18,143][105620] Updated weights for policy 1, policy_version 278245 (0.0011) [2023-12-26 17:23:18,412][105692] Updated weights for policy 0, policy_version 278162 (0.0010) [2023-12-26 17:23:18,475][105692] Updated weights for policy 0, policy_version 278172 (0.0010) [2023-12-26 17:23:18,541][105692] Updated weights for policy 0, policy_version 278182 (0.0005) [2023-12-26 17:23:18,785][105620] Updated weights for policy 1, policy_version 278255 (0.0010) [2023-12-26 17:23:18,844][105620] Updated weights for policy 1, policy_version 278265 (0.0010) [2023-12-26 17:23:18,912][105620] Updated weights for policy 1, policy_version 278275 (0.0010) [2023-12-26 17:23:19,147][105692] Updated weights for policy 0, policy_version 278192 (0.0007) [2023-12-26 17:23:19,212][105692] Updated weights for policy 0, policy_version 278202 (0.0010) [2023-12-26 17:23:19,275][105692] Updated weights for policy 0, policy_version 278212 (0.0008) [2023-12-26 17:23:19,580][105620] Updated weights for policy 1, policy_version 278285 (0.0011) [2023-12-26 17:23:19,635][105620] Updated weights for policy 1, policy_version 278295 (0.0010) [2023-12-26 17:23:19,684][105620] Updated weights for policy 1, policy_version 278305 (0.0010) [2023-12-26 17:23:20,040][105692] Updated weights for policy 0, policy_version 278222 (0.0007) [2023-12-26 17:23:20,111][105692] Updated weights for policy 0, policy_version 278232 (0.0006) [2023-12-26 17:23:20,178][105692] Updated weights for policy 0, policy_version 278242 (0.0006) [2023-12-26 17:23:20,319][105620] Updated weights for policy 1, policy_version 278315 (0.0009) [2023-12-26 17:23:20,381][105620] Updated weights for policy 1, policy_version 278325 (0.0011) [2023-12-26 17:23:20,440][105620] Updated weights for policy 1, policy_version 278335 (0.0011) [2023-12-26 17:23:20,820][105692] Updated weights for policy 0, policy_version 278252 (0.0007) [2023-12-26 17:23:20,885][105692] Updated weights for policy 0, policy_version 278262 (0.0009) [2023-12-26 17:23:20,945][105692] Updated weights for policy 0, policy_version 278272 (0.0009) [2023-12-26 17:23:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 142516224. Throughput: 0: 9733.8, 1: 9846.2. Samples: 142504068. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 17:23:21,062][104569] Avg episode reward: [(0, '9356.063'), (1, '9168.215')] [2023-12-26 17:23:21,124][105620] Updated weights for policy 1, policy_version 278345 (0.0011) [2023-12-26 17:23:21,182][105620] Updated weights for policy 1, policy_version 278355 (0.0009) [2023-12-26 17:23:21,233][105620] Updated weights for policy 1, policy_version 278365 (0.0010) [2023-12-26 17:23:21,291][105620] Updated weights for policy 1, policy_version 278375 (0.0010) [2023-12-26 17:23:21,695][105692] Updated weights for policy 0, policy_version 278282 (0.0009) [2023-12-26 17:23:21,764][105692] Updated weights for policy 0, policy_version 278292 (0.0009) [2023-12-26 17:23:21,836][105692] Updated weights for policy 0, policy_version 278302 (0.0006) [2023-12-26 17:23:21,907][105692] Updated weights for policy 0, policy_version 278312 (0.0006) [2023-12-26 17:23:22,065][105620] Updated weights for policy 1, policy_version 278385 (0.0006) [2023-12-26 17:23:22,133][105620] Updated weights for policy 1, policy_version 278395 (0.0006) [2023-12-26 17:23:22,196][105620] Updated weights for policy 1, policy_version 278405 (0.0005) [2023-12-26 17:23:22,693][105692] Updated weights for policy 0, policy_version 278322 (0.0009) [2023-12-26 17:23:22,757][105692] Updated weights for policy 0, policy_version 278332 (0.0008) [2023-12-26 17:23:22,809][105620] Updated weights for policy 1, policy_version 278415 (0.0008) [2023-12-26 17:23:22,824][105692] Updated weights for policy 0, policy_version 278342 (0.0007) [2023-12-26 17:23:22,870][105620] Updated weights for policy 1, policy_version 278425 (0.0010) [2023-12-26 17:23:22,935][105620] Updated weights for policy 1, policy_version 278435 (0.0011) [2023-12-26 17:23:23,547][105692] Updated weights for policy 0, policy_version 278352 (0.0009) [2023-12-26 17:23:23,599][105692] Updated weights for policy 0, policy_version 278362 (0.0008) [2023-12-26 17:23:23,657][105692] Updated weights for policy 0, policy_version 278372 (0.0008) [2023-12-26 17:23:23,672][105620] Updated weights for policy 1, policy_version 278445 (0.0011) [2023-12-26 17:23:23,723][105620] Updated weights for policy 1, policy_version 278455 (0.0010) [2023-12-26 17:23:23,774][105620] Updated weights for policy 1, policy_version 278465 (0.0010) [2023-12-26 17:23:24,409][105692] Updated weights for policy 0, policy_version 278382 (0.0007) [2023-12-26 17:23:24,467][105692] Updated weights for policy 0, policy_version 278392 (0.0008) [2023-12-26 17:23:24,526][105620] Updated weights for policy 1, policy_version 278475 (0.0010) [2023-12-26 17:23:24,528][105692] Updated weights for policy 0, policy_version 278402 (0.0007) [2023-12-26 17:23:24,588][105620] Updated weights for policy 1, policy_version 278485 (0.0010) [2023-12-26 17:23:24,646][105620] Updated weights for policy 1, policy_version 278495 (0.0010) [2023-12-26 17:23:25,262][105692] Updated weights for policy 0, policy_version 278412 (0.0008) [2023-12-26 17:23:25,291][105620] Updated weights for policy 1, policy_version 278505 (0.0010) [2023-12-26 17:23:25,321][105692] Updated weights for policy 0, policy_version 278422 (0.0010) [2023-12-26 17:23:25,349][105620] Updated weights for policy 1, policy_version 278515 (0.0006) [2023-12-26 17:23:25,384][105692] Updated weights for policy 0, policy_version 278432 (0.0011) [2023-12-26 17:23:25,411][105620] Updated weights for policy 1, policy_version 278525 (0.0005) [2023-12-26 17:23:25,471][105620] Updated weights for policy 1, policy_version 278535 (0.0005) [2023-12-26 17:23:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 142606336. Throughput: 0: 9623.8, 1: 9934.9. Samples: 142620444. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 17:23:26,062][104569] Avg episode reward: [(0, '9356.233'), (1, '9168.257')] [2023-12-26 17:23:26,131][105692] Updated weights for policy 0, policy_version 278442 (0.0010) [2023-12-26 17:23:26,134][105620] Updated weights for policy 1, policy_version 278545 (0.0010) [2023-12-26 17:23:26,186][105692] Updated weights for policy 0, policy_version 278452 (0.0010) [2023-12-26 17:23:26,196][105620] Updated weights for policy 1, policy_version 278555 (0.0006) [2023-12-26 17:23:26,247][105692] Updated weights for policy 0, policy_version 278462 (0.0010) [2023-12-26 17:23:26,288][105620] Updated weights for policy 1, policy_version 278565 (0.0011) [2023-12-26 17:23:26,306][105692] Updated weights for policy 0, policy_version 278472 (0.0011) [2023-12-26 17:23:26,949][105620] Updated weights for policy 1, policy_version 278575 (0.0011) [2023-12-26 17:23:27,001][105620] Updated weights for policy 1, policy_version 278585 (0.0010) [2023-12-26 17:23:27,051][105692] Updated weights for policy 0, policy_version 278482 (0.0010) [2023-12-26 17:23:27,053][105620] Updated weights for policy 1, policy_version 278595 (0.0010) [2023-12-26 17:23:27,102][105692] Updated weights for policy 0, policy_version 278492 (0.0010) [2023-12-26 17:23:27,156][105692] Updated weights for policy 0, policy_version 278502 (0.0010) [2023-12-26 17:23:27,808][105620] Updated weights for policy 1, policy_version 278605 (0.0010) [2023-12-26 17:23:27,867][105620] Updated weights for policy 1, policy_version 278615 (0.0011) [2023-12-26 17:23:27,878][105692] Updated weights for policy 0, policy_version 278512 (0.0008) [2023-12-26 17:23:27,926][105620] Updated weights for policy 1, policy_version 278625 (0.0011) [2023-12-26 17:23:27,944][105692] Updated weights for policy 0, policy_version 278522 (0.0007) [2023-12-26 17:23:28,001][105692] Updated weights for policy 0, policy_version 278532 (0.0005) [2023-12-26 17:23:28,646][105620] Updated weights for policy 1, policy_version 278635 (0.0010) [2023-12-26 17:23:28,708][105620] Updated weights for policy 1, policy_version 278645 (0.0009) [2023-12-26 17:23:28,710][105692] Updated weights for policy 0, policy_version 278542 (0.0007) [2023-12-26 17:23:28,764][105620] Updated weights for policy 1, policy_version 278655 (0.0006) [2023-12-26 17:23:28,770][105692] Updated weights for policy 0, policy_version 278552 (0.0007) [2023-12-26 17:23:28,828][105692] Updated weights for policy 0, policy_version 278562 (0.0007) [2023-12-26 17:23:29,511][105620] Updated weights for policy 1, policy_version 278665 (0.0008) [2023-12-26 17:23:29,558][105620] Updated weights for policy 1, policy_version 278675 (0.0009) [2023-12-26 17:23:29,583][105586] KL-divergence is very high: 107.3058 [2023-12-26 17:23:29,586][105692] Updated weights for policy 0, policy_version 278572 (0.0008) [2023-12-26 17:23:29,619][105620] Updated weights for policy 1, policy_version 278685 (0.0009) [2023-12-26 17:23:29,632][105586] KL-divergence is very high: 162.8682 [2023-12-26 17:23:29,634][105692] Updated weights for policy 0, policy_version 278582 (0.0006) [2023-12-26 17:23:29,671][105586] KL-divergence is very high: 158.7657 [2023-12-26 17:23:29,672][105620] Updated weights for policy 1, policy_version 278695 (0.0009) [2023-12-26 17:23:29,684][105692] Updated weights for policy 0, policy_version 278592 (0.0008) [2023-12-26 17:23:30,328][105692] Updated weights for policy 0, policy_version 278602 (0.0005) [2023-12-26 17:23:30,384][105692] Updated weights for policy 0, policy_version 278612 (0.0005) [2023-12-26 17:23:30,434][105692] Updated weights for policy 0, policy_version 278622 (0.0005) [2023-12-26 17:23:30,484][105692] Updated weights for policy 0, policy_version 278632 (0.0005) [2023-12-26 17:23:30,509][105620] Updated weights for policy 1, policy_version 278705 (0.0006) [2023-12-26 17:23:30,578][105620] Updated weights for policy 1, policy_version 278715 (0.0006) [2023-12-26 17:23:30,637][105620] Updated weights for policy 1, policy_version 278725 (0.0010) [2023-12-26 17:23:31,011][105692] Updated weights for policy 0, policy_version 278642 (0.0009) [2023-12-26 17:23:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 142704640. Throughput: 0: 9633.6, 1: 9938.0. Samples: 142678040. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 17:23:31,062][104569] Avg episode reward: [(0, '9355.685'), (1, '9260.606')] [2023-12-26 17:23:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000278728_71360512.pth... [2023-12-26 17:23:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000277576_71065600.pth [2023-12-26 17:23:31,075][105692] Updated weights for policy 0, policy_version 278652 (0.0008) [2023-12-26 17:23:31,140][105692] Updated weights for policy 0, policy_version 278662 (0.0009) [2023-12-26 17:23:31,155][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000278664_71352320.pth... [2023-12-26 17:23:31,159][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000277544_71065600.pth [2023-12-26 17:23:31,347][105620] Updated weights for policy 1, policy_version 278735 (0.0008) [2023-12-26 17:23:31,417][105620] Updated weights for policy 1, policy_version 278745 (0.0009) [2023-12-26 17:23:31,464][105620] Updated weights for policy 1, policy_version 278755 (0.0009) [2023-12-26 17:23:31,880][105692] Updated weights for policy 0, policy_version 278672 (0.0008) [2023-12-26 17:23:31,940][105692] Updated weights for policy 0, policy_version 278682 (0.0008) [2023-12-26 17:23:31,989][105692] Updated weights for policy 0, policy_version 278692 (0.0006) [2023-12-26 17:23:32,240][105620] Updated weights for policy 1, policy_version 278765 (0.0009) [2023-12-26 17:23:32,309][105620] Updated weights for policy 1, policy_version 278775 (0.0010) [2023-12-26 17:23:32,376][105620] Updated weights for policy 1, policy_version 278785 (0.0011) [2023-12-26 17:23:32,654][105692] Updated weights for policy 0, policy_version 278702 (0.0008) [2023-12-26 17:23:32,708][105692] Updated weights for policy 0, policy_version 278712 (0.0010) [2023-12-26 17:23:32,759][105692] Updated weights for policy 0, policy_version 278722 (0.0010) [2023-12-26 17:23:33,049][105620] Updated weights for policy 1, policy_version 278795 (0.0010) [2023-12-26 17:23:33,101][105620] Updated weights for policy 1, policy_version 278805 (0.0008) [2023-12-26 17:23:33,153][105620] Updated weights for policy 1, policy_version 278815 (0.0009) [2023-12-26 17:23:33,429][105692] Updated weights for policy 0, policy_version 278732 (0.0007) [2023-12-26 17:23:33,480][105692] Updated weights for policy 0, policy_version 278742 (0.0009) [2023-12-26 17:23:33,532][105692] Updated weights for policy 0, policy_version 278752 (0.0006) [2023-12-26 17:23:33,873][105620] Updated weights for policy 1, policy_version 278825 (0.0008) [2023-12-26 17:23:33,926][105620] Updated weights for policy 1, policy_version 278835 (0.0009) [2023-12-26 17:23:33,979][105620] Updated weights for policy 1, policy_version 278845 (0.0009) [2023-12-26 17:23:34,032][105620] Updated weights for policy 1, policy_version 278855 (0.0010) [2023-12-26 17:23:34,116][105692] Updated weights for policy 0, policy_version 278762 (0.0010) [2023-12-26 17:23:34,176][105692] Updated weights for policy 0, policy_version 278772 (0.0011) [2023-12-26 17:23:34,237][105692] Updated weights for policy 0, policy_version 278782 (0.0008) [2023-12-26 17:23:34,297][105692] Updated weights for policy 0, policy_version 278792 (0.0005) [2023-12-26 17:23:34,779][105620] Updated weights for policy 1, policy_version 278865 (0.0009) [2023-12-26 17:23:34,838][105620] Updated weights for policy 1, policy_version 278875 (0.0008) [2023-12-26 17:23:34,889][105620] Updated weights for policy 1, policy_version 278885 (0.0010) [2023-12-26 17:23:34,935][105692] Updated weights for policy 0, policy_version 278802 (0.0005) [2023-12-26 17:23:34,990][105692] Updated weights for policy 0, policy_version 278812 (0.0005) [2023-12-26 17:23:35,049][105692] Updated weights for policy 0, policy_version 278822 (0.0009) [2023-12-26 17:23:35,580][105620] Updated weights for policy 1, policy_version 278895 (0.0007) [2023-12-26 17:23:35,644][105620] Updated weights for policy 1, policy_version 278905 (0.0008) [2023-12-26 17:23:35,676][105692] Updated weights for policy 0, policy_version 278832 (0.0008) [2023-12-26 17:23:35,716][105620] Updated weights for policy 1, policy_version 278915 (0.0007) [2023-12-26 17:23:35,730][105692] Updated weights for policy 0, policy_version 278842 (0.0007) [2023-12-26 17:23:35,782][105692] Updated weights for policy 0, policy_version 278852 (0.0008) [2023-12-26 17:23:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 142811136. Throughput: 0: 9675.5, 1: 9886.0. Samples: 142796856. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 17:23:36,062][104569] Avg episode reward: [(0, '9355.293'), (1, '9169.470')] [2023-12-26 17:23:36,453][105620] Updated weights for policy 1, policy_version 278925 (0.0009) [2023-12-26 17:23:36,498][105692] Updated weights for policy 0, policy_version 278862 (0.0009) [2023-12-26 17:23:36,512][105620] Updated weights for policy 1, policy_version 278935 (0.0007) [2023-12-26 17:23:36,557][105692] Updated weights for policy 0, policy_version 278872 (0.0011) [2023-12-26 17:23:36,572][105620] Updated weights for policy 1, policy_version 278945 (0.0006) [2023-12-26 17:23:36,620][105692] Updated weights for policy 0, policy_version 278882 (0.0010) [2023-12-26 17:23:37,330][105692] Updated weights for policy 0, policy_version 278892 (0.0011) [2023-12-26 17:23:37,333][105620] Updated weights for policy 1, policy_version 278955 (0.0008) [2023-12-26 17:23:37,383][105692] Updated weights for policy 0, policy_version 278902 (0.0008) [2023-12-26 17:23:37,388][105620] Updated weights for policy 1, policy_version 278965 (0.0007) [2023-12-26 17:23:37,440][105692] Updated weights for policy 0, policy_version 278912 (0.0005) [2023-12-26 17:23:37,445][105620] Updated weights for policy 1, policy_version 278975 (0.0008) [2023-12-26 17:23:38,143][105692] Updated weights for policy 0, policy_version 278922 (0.0010) [2023-12-26 17:23:38,183][105620] Updated weights for policy 1, policy_version 278985 (0.0007) [2023-12-26 17:23:38,200][105692] Updated weights for policy 0, policy_version 278932 (0.0010) [2023-12-26 17:23:38,245][105620] Updated weights for policy 1, policy_version 278995 (0.0010) [2023-12-26 17:23:38,252][105692] Updated weights for policy 0, policy_version 278942 (0.0010) [2023-12-26 17:23:38,296][105620] Updated weights for policy 1, policy_version 279005 (0.0010) [2023-12-26 17:23:38,310][105692] Updated weights for policy 0, policy_version 278952 (0.0010) [2023-12-26 17:23:38,358][105620] Updated weights for policy 1, policy_version 279015 (0.0009) [2023-12-26 17:23:38,979][105620] Updated weights for policy 1, policy_version 279025 (0.0005) [2023-12-26 17:23:39,000][105692] Updated weights for policy 0, policy_version 278962 (0.0008) [2023-12-26 17:23:39,038][105620] Updated weights for policy 1, policy_version 279035 (0.0005) [2023-12-26 17:23:39,061][105692] Updated weights for policy 0, policy_version 278972 (0.0009) [2023-12-26 17:23:39,088][105620] Updated weights for policy 1, policy_version 279045 (0.0006) [2023-12-26 17:23:39,127][105692] Updated weights for policy 0, policy_version 278982 (0.0009) [2023-12-26 17:23:39,771][105620] Updated weights for policy 1, policy_version 279055 (0.0006) [2023-12-26 17:23:39,838][105620] Updated weights for policy 1, policy_version 279065 (0.0006) [2023-12-26 17:23:39,872][105692] Updated weights for policy 0, policy_version 278992 (0.0008) [2023-12-26 17:23:39,901][105620] Updated weights for policy 1, policy_version 279075 (0.0009) [2023-12-26 17:23:39,934][105692] Updated weights for policy 0, policy_version 279002 (0.0009) [2023-12-26 17:23:39,996][105692] Updated weights for policy 0, policy_version 279012 (0.0008) [2023-12-26 17:23:40,530][105620] Updated weights for policy 1, policy_version 279085 (0.0007) [2023-12-26 17:23:40,586][105620] Updated weights for policy 1, policy_version 279095 (0.0005) [2023-12-26 17:23:40,645][105620] Updated weights for policy 1, policy_version 279105 (0.0005) [2023-12-26 17:23:40,795][105692] Updated weights for policy 0, policy_version 279022 (0.0009) [2023-12-26 17:23:40,850][105692] Updated weights for policy 0, policy_version 279032 (0.0010) [2023-12-26 17:23:40,909][105692] Updated weights for policy 0, policy_version 279042 (0.0010) [2023-12-26 17:23:41,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 142909440. Throughput: 0: 9689.9, 1: 9907.5. Samples: 142916604. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 17:23:41,062][104569] Avg episode reward: [(0, '9355.459'), (1, '9169.923')] [2023-12-26 17:23:41,197][105620] Updated weights for policy 1, policy_version 279115 (0.0006) [2023-12-26 17:23:41,260][105620] Updated weights for policy 1, policy_version 279125 (0.0008) [2023-12-26 17:23:41,327][105620] Updated weights for policy 1, policy_version 279135 (0.0006) [2023-12-26 17:23:41,802][105692] Updated weights for policy 0, policy_version 279052 (0.0010) [2023-12-26 17:23:41,864][105692] Updated weights for policy 0, policy_version 279062 (0.0009) [2023-12-26 17:23:41,921][105692] Updated weights for policy 0, policy_version 279072 (0.0009) [2023-12-26 17:23:41,983][105620] Updated weights for policy 1, policy_version 279145 (0.0007) [2023-12-26 17:23:42,051][105620] Updated weights for policy 1, policy_version 279155 (0.0005) [2023-12-26 17:23:42,105][105620] Updated weights for policy 1, policy_version 279165 (0.0006) [2023-12-26 17:23:42,160][105620] Updated weights for policy 1, policy_version 279175 (0.0007) [2023-12-26 17:23:42,744][105620] Updated weights for policy 1, policy_version 279185 (0.0007) [2023-12-26 17:23:42,805][105692] Updated weights for policy 0, policy_version 279082 (0.0009) [2023-12-26 17:23:42,808][105620] Updated weights for policy 1, policy_version 279195 (0.0008) [2023-12-26 17:23:42,857][105692] Updated weights for policy 0, policy_version 279092 (0.0009) [2023-12-26 17:23:42,865][105620] Updated weights for policy 1, policy_version 279205 (0.0009) [2023-12-26 17:23:42,910][105692] Updated weights for policy 0, policy_version 279102 (0.0008) [2023-12-26 17:23:42,961][105692] Updated weights for policy 0, policy_version 279112 (0.0005) [2023-12-26 17:23:43,577][105692] Updated weights for policy 0, policy_version 279122 (0.0010) [2023-12-26 17:23:43,591][105620] Updated weights for policy 1, policy_version 279215 (0.0005) [2023-12-26 17:23:43,636][105692] Updated weights for policy 0, policy_version 279132 (0.0007) [2023-12-26 17:23:43,652][105620] Updated weights for policy 1, policy_version 279225 (0.0008) [2023-12-26 17:23:43,700][105692] Updated weights for policy 0, policy_version 279142 (0.0006) [2023-12-26 17:23:43,715][105620] Updated weights for policy 1, policy_version 279235 (0.0009) [2023-12-26 17:23:44,311][105692] Updated weights for policy 0, policy_version 279152 (0.0008) [2023-12-26 17:23:44,363][105692] Updated weights for policy 0, policy_version 279162 (0.0009) [2023-12-26 17:23:44,416][105692] Updated weights for policy 0, policy_version 279172 (0.0009) [2023-12-26 17:23:44,477][105620] Updated weights for policy 1, policy_version 279245 (0.0009) [2023-12-26 17:23:44,535][105620] Updated weights for policy 1, policy_version 279255 (0.0009) [2023-12-26 17:23:44,596][105620] Updated weights for policy 1, policy_version 279265 (0.0009) [2023-12-26 17:23:45,168][105692] Updated weights for policy 0, policy_version 279182 (0.0009) [2023-12-26 17:23:45,223][105692] Updated weights for policy 0, policy_version 279192 (0.0009) [2023-12-26 17:23:45,279][105692] Updated weights for policy 0, policy_version 279202 (0.0009) [2023-12-26 17:23:45,362][105620] Updated weights for policy 1, policy_version 279275 (0.0009) [2023-12-26 17:23:45,412][105620] Updated weights for policy 1, policy_version 279285 (0.0008) [2023-12-26 17:23:45,467][105620] Updated weights for policy 1, policy_version 279295 (0.0009) [2023-12-26 17:23:45,985][105692] Updated weights for policy 0, policy_version 279212 (0.0008) [2023-12-26 17:23:46,039][105692] Updated weights for policy 0, policy_version 279222 (0.0005) [2023-12-26 17:23:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 142999552. Throughput: 0: 9603.1, 1: 9918.3. Samples: 142974000. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 17:23:46,062][104569] Avg episode reward: [(0, '9356.092'), (1, '6935.391')] [2023-12-26 17:23:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000279304_71507968.pth... [2023-12-26 17:23:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000278152_71213056.pth [2023-12-26 17:23:46,095][105692] Updated weights for policy 0, policy_version 279232 (0.0007) [2023-12-26 17:23:46,131][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000279240_71499776.pth... [2023-12-26 17:23:46,134][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000278088_71204864.pth [2023-12-26 17:23:46,322][105620] Updated weights for policy 1, policy_version 279305 (0.0008) [2023-12-26 17:23:46,384][105620] Updated weights for policy 1, policy_version 279315 (0.0009) [2023-12-26 17:23:46,438][105620] Updated weights for policy 1, policy_version 279325 (0.0009) [2023-12-26 17:23:46,503][105620] Updated weights for policy 1, policy_version 279335 (0.0009) [2023-12-26 17:23:46,740][105692] Updated weights for policy 0, policy_version 279242 (0.0007) [2023-12-26 17:23:46,795][105692] Updated weights for policy 0, policy_version 279252 (0.0009) [2023-12-26 17:23:46,848][105692] Updated weights for policy 0, policy_version 279262 (0.0007) [2023-12-26 17:23:46,892][105692] Updated weights for policy 0, policy_version 279272 (0.0005) [2023-12-26 17:23:47,287][105620] Updated weights for policy 1, policy_version 279345 (0.0009) [2023-12-26 17:23:47,348][105620] Updated weights for policy 1, policy_version 279355 (0.0008) [2023-12-26 17:23:47,407][105620] Updated weights for policy 1, policy_version 279365 (0.0009) [2023-12-26 17:23:47,603][105692] Updated weights for policy 0, policy_version 279282 (0.0009) [2023-12-26 17:23:47,656][105692] Updated weights for policy 0, policy_version 279292 (0.0009) [2023-12-26 17:23:47,715][105692] Updated weights for policy 0, policy_version 279302 (0.0009) [2023-12-26 17:23:48,135][105620] Updated weights for policy 1, policy_version 279375 (0.0009) [2023-12-26 17:23:48,192][105620] Updated weights for policy 1, policy_version 279385 (0.0009) [2023-12-26 17:23:48,239][105620] Updated weights for policy 1, policy_version 279395 (0.0010) [2023-12-26 17:23:48,503][105692] Updated weights for policy 0, policy_version 279312 (0.0009) [2023-12-26 17:23:48,570][105692] Updated weights for policy 0, policy_version 279322 (0.0009) [2023-12-26 17:23:48,639][105692] Updated weights for policy 0, policy_version 279332 (0.0008) [2023-12-26 17:23:48,992][105620] Updated weights for policy 1, policy_version 279405 (0.0008) [2023-12-26 17:23:49,039][105620] Updated weights for policy 1, policy_version 279415 (0.0009) [2023-12-26 17:23:49,093][105620] Updated weights for policy 1, policy_version 279425 (0.0009) [2023-12-26 17:23:49,367][105692] Updated weights for policy 0, policy_version 279342 (0.0009) [2023-12-26 17:23:49,434][105692] Updated weights for policy 0, policy_version 279352 (0.0009) [2023-12-26 17:23:49,472][105585] KL-divergence is very high: 171.7087 [2023-12-26 17:23:49,497][105692] Updated weights for policy 0, policy_version 279362 (0.0009) [2023-12-26 17:23:49,514][105585] KL-divergence is very high: 213.0430 [2023-12-26 17:23:49,890][105620] Updated weights for policy 1, policy_version 279435 (0.0009) [2023-12-26 17:23:49,949][105620] Updated weights for policy 1, policy_version 279445 (0.0009) [2023-12-26 17:23:49,999][105620] Updated weights for policy 1, policy_version 279455 (0.0008) [2023-12-26 17:23:50,252][105585] KL-divergence is very high: 192.5745 [2023-12-26 17:23:50,259][105585] KL-divergence is very high: 159.5471 [2023-12-26 17:23:50,277][105692] Updated weights for policy 0, policy_version 279372 (0.0009) [2023-12-26 17:23:50,298][105585] KL-divergence is very high: 158.5813 [2023-12-26 17:23:50,311][105585] KL-divergence is very high: 246.4851 [2023-12-26 17:23:50,316][105585] KL-divergence is very high: 253.8753 [2023-12-26 17:23:50,340][105692] Updated weights for policy 0, policy_version 279382 (0.0009) [2023-12-26 17:23:50,340][105585] KL-divergence is very high: 181.6654 [2023-12-26 17:23:50,347][105585] KL-divergence is very high: 292.6077 [2023-12-26 17:23:50,359][105585] KL-divergence is very high: 254.3311 [2023-12-26 17:23:50,367][105585] KL-divergence is very high: 183.1666 [2023-12-26 17:23:50,406][105692] Updated weights for policy 0, policy_version 279392 (0.0009) [2023-12-26 17:23:50,410][105585] KL-divergence is very high: 101.7027 [2023-12-26 17:23:50,714][105620] Updated weights for policy 1, policy_version 279465 (0.0009) [2023-12-26 17:23:50,776][105620] Updated weights for policy 1, policy_version 279475 (0.0010) [2023-12-26 17:23:50,837][105620] Updated weights for policy 1, policy_version 279485 (0.0009) [2023-12-26 17:23:50,888][105620] Updated weights for policy 1, policy_version 279495 (0.0012) [2023-12-26 17:23:51,059][105692] Updated weights for policy 0, policy_version 279402 (0.0010) [2023-12-26 17:23:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 143097856. Throughput: 0: 9644.4, 1: 9842.3. Samples: 143087132. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 17:23:51,063][104569] Avg episode reward: [(0, '8637.846'), (1, '1512.444')] [2023-12-26 17:23:51,116][105692] Updated weights for policy 0, policy_version 279412 (0.0010) [2023-12-26 17:23:51,177][105692] Updated weights for policy 0, policy_version 279422 (0.0008) [2023-12-26 17:23:51,227][105692] Updated weights for policy 0, policy_version 279432 (0.0008) [2023-12-26 17:23:51,617][105620] Updated weights for policy 1, policy_version 279505 (0.0007) [2023-12-26 17:23:51,686][105620] Updated weights for policy 1, policy_version 279515 (0.0008) [2023-12-26 17:23:51,691][105586] KL-divergence is very high: 125.7379 [2023-12-26 17:23:51,717][105586] KL-divergence is very high: 165.1106 [2023-12-26 17:23:51,752][105586] KL-divergence is very high: 206.5079 [2023-12-26 17:23:51,760][105620] Updated weights for policy 1, policy_version 279525 (0.0009) [2023-12-26 17:23:51,771][105586] KL-divergence is very high: 211.2408 [2023-12-26 17:23:51,999][105692] Updated weights for policy 0, policy_version 279442 (0.0010) [2023-12-26 17:23:52,051][105692] Updated weights for policy 0, policy_version 279452 (0.0010) [2023-12-26 17:23:52,106][105692] Updated weights for policy 0, policy_version 279462 (0.0011) [2023-12-26 17:23:52,498][105620] Updated weights for policy 1, policy_version 279535 (0.0008) [2023-12-26 17:23:52,551][105620] Updated weights for policy 1, policy_version 279545 (0.0008) [2023-12-26 17:23:52,601][105620] Updated weights for policy 1, policy_version 279555 (0.0008) [2023-12-26 17:23:52,877][105692] Updated weights for policy 0, policy_version 279472 (0.0010) [2023-12-26 17:23:52,931][105692] Updated weights for policy 0, policy_version 279482 (0.0010) [2023-12-26 17:23:52,986][105692] Updated weights for policy 0, policy_version 279492 (0.0010) [2023-12-26 17:23:53,391][105620] Updated weights for policy 1, policy_version 279565 (0.0008) [2023-12-26 17:23:53,460][105620] Updated weights for policy 1, policy_version 279575 (0.0008) [2023-12-26 17:23:53,521][105620] Updated weights for policy 1, policy_version 279585 (0.0008) [2023-12-26 17:23:53,713][105692] Updated weights for policy 0, policy_version 279502 (0.0010) [2023-12-26 17:23:53,762][105692] Updated weights for policy 0, policy_version 279512 (0.0009) [2023-12-26 17:23:53,813][105692] Updated weights for policy 0, policy_version 279522 (0.0009) [2023-12-26 17:23:54,279][105620] Updated weights for policy 1, policy_version 279595 (0.0009) [2023-12-26 17:23:54,330][105620] Updated weights for policy 1, policy_version 279605 (0.0009) [2023-12-26 17:23:54,381][105620] Updated weights for policy 1, policy_version 279616 (0.0009) [2023-12-26 17:23:54,474][105692] Updated weights for policy 0, policy_version 279532 (0.0010) [2023-12-26 17:23:54,536][105692] Updated weights for policy 0, policy_version 279542 (0.0010) [2023-12-26 17:23:54,591][105692] Updated weights for policy 0, policy_version 279552 (0.0010) [2023-12-26 17:23:55,164][105620] Updated weights for policy 1, policy_version 279626 (0.0008) [2023-12-26 17:23:55,216][105620] Updated weights for policy 1, policy_version 279636 (0.0008) [2023-12-26 17:23:55,264][105620] Updated weights for policy 1, policy_version 279646 (0.0008) [2023-12-26 17:23:55,319][105620] Updated weights for policy 1, policy_version 279656 (0.0008) [2023-12-26 17:23:55,337][105692] Updated weights for policy 0, policy_version 279562 (0.0010) [2023-12-26 17:23:55,394][105692] Updated weights for policy 0, policy_version 279572 (0.0010) [2023-12-26 17:23:55,452][105692] Updated weights for policy 0, policy_version 279582 (0.0010) [2023-12-26 17:23:55,506][105692] Updated weights for policy 0, policy_version 279592 (0.0010) [2023-12-26 17:23:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 143187968. Throughput: 0: 9690.3, 1: 9681.4. Samples: 143200376. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 17:23:56,062][104569] Avg episode reward: [(0, '8637.494'), (1, '1196.394')] [2023-12-26 17:23:56,096][105620] Updated weights for policy 1, policy_version 279666 (0.0008) [2023-12-26 17:23:56,141][105620] Updated weights for policy 1, policy_version 279676 (0.0008) [2023-12-26 17:23:56,189][105620] Updated weights for policy 1, policy_version 279686 (0.0008) [2023-12-26 17:23:56,247][105692] Updated weights for policy 0, policy_version 279602 (0.0010) [2023-12-26 17:23:56,308][105692] Updated weights for policy 0, policy_version 279612 (0.0010) [2023-12-26 17:23:56,370][105692] Updated weights for policy 0, policy_version 279622 (0.0010) [2023-12-26 17:23:56,962][105620] Updated weights for policy 1, policy_version 279696 (0.0008) [2023-12-26 17:23:57,009][105620] Updated weights for policy 1, policy_version 279706 (0.0008) [2023-12-26 17:23:57,065][105620] Updated weights for policy 1, policy_version 279716 (0.0007) [2023-12-26 17:23:57,109][105692] Updated weights for policy 0, policy_version 279632 (0.0010) [2023-12-26 17:23:57,178][105692] Updated weights for policy 0, policy_version 279642 (0.0010) [2023-12-26 17:23:57,243][105692] Updated weights for policy 0, policy_version 279652 (0.0010) [2023-12-26 17:23:57,817][105620] Updated weights for policy 1, policy_version 279726 (0.0008) [2023-12-26 17:23:57,876][105620] Updated weights for policy 1, policy_version 279736 (0.0007) [2023-12-26 17:23:57,892][105692] Updated weights for policy 0, policy_version 279662 (0.0010) [2023-12-26 17:23:57,928][105620] Updated weights for policy 1, policy_version 279746 (0.0007) [2023-12-26 17:23:57,946][105692] Updated weights for policy 0, policy_version 279672 (0.0008) [2023-12-26 17:23:58,008][105692] Updated weights for policy 0, policy_version 279682 (0.0009) [2023-12-26 17:23:58,692][105620] Updated weights for policy 1, policy_version 279756 (0.0007) [2023-12-26 17:23:58,765][105620] Updated weights for policy 1, policy_version 279766 (0.0008) [2023-12-26 17:23:58,846][105620] Updated weights for policy 1, policy_version 279776 (0.0008) [2023-12-26 17:23:58,880][105692] Updated weights for policy 0, policy_version 279692 (0.0008) [2023-12-26 17:23:58,944][105692] Updated weights for policy 0, policy_version 279702 (0.0009) [2023-12-26 17:23:59,005][105692] Updated weights for policy 0, policy_version 279712 (0.0010) [2023-12-26 17:23:59,580][105620] Updated weights for policy 1, policy_version 279786 (0.0008) [2023-12-26 17:23:59,635][105620] Updated weights for policy 1, policy_version 279797 (0.0008) [2023-12-26 17:23:59,655][105692] Updated weights for policy 0, policy_version 279722 (0.0010) [2023-12-26 17:23:59,703][105620] Updated weights for policy 1, policy_version 279807 (0.0006) [2023-12-26 17:23:59,709][105692] Updated weights for policy 0, policy_version 279732 (0.0009) [2023-12-26 17:23:59,763][105692] Updated weights for policy 0, policy_version 279742 (0.0006) [2023-12-26 17:23:59,827][105692] Updated weights for policy 0, policy_version 279752 (0.0006) [2023-12-26 17:24:00,414][105620] Updated weights for policy 1, policy_version 279817 (0.0007) [2023-12-26 17:24:00,476][105620] Updated weights for policy 1, policy_version 279827 (0.0005) [2023-12-26 17:24:00,512][105692] Updated weights for policy 0, policy_version 279762 (0.0007) [2023-12-26 17:24:00,536][105620] Updated weights for policy 1, policy_version 279837 (0.0006) [2023-12-26 17:24:00,575][105692] Updated weights for policy 0, policy_version 279772 (0.0005) [2023-12-26 17:24:00,595][105620] Updated weights for policy 1, policy_version 279847 (0.0005) [2023-12-26 17:24:00,641][105692] Updated weights for policy 0, policy_version 279782 (0.0006) [2023-12-26 17:24:01,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 143286272. Throughput: 0: 9728.0, 1: 9648.8. Samples: 143256664. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 17:24:01,062][104569] Avg episode reward: [(0, '9079.600'), (1, '1457.136')] [2023-12-26 17:24:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000279784_71639040.pth... [2023-12-26 17:24:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000278664_71352320.pth [2023-12-26 17:24:01,116][105620] Updated weights for policy 1, policy_version 279857 (0.0006) [2023-12-26 17:24:01,183][105620] Updated weights for policy 1, policy_version 279867 (0.0007) [2023-12-26 17:24:01,251][105620] Updated weights for policy 1, policy_version 279877 (0.0008) [2023-12-26 17:24:01,270][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000279880_71655424.pth... [2023-12-26 17:24:01,276][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000278728_71360512.pth [2023-12-26 17:24:01,284][105692] Updated weights for policy 0, policy_version 279792 (0.0008) [2023-12-26 17:24:01,335][105692] Updated weights for policy 0, policy_version 279802 (0.0008) [2023-12-26 17:24:01,400][105692] Updated weights for policy 0, policy_version 279812 (0.0009) [2023-12-26 17:24:01,951][105620] Updated weights for policy 1, policy_version 279887 (0.0009) [2023-12-26 17:24:01,999][105620] Updated weights for policy 1, policy_version 279897 (0.0009) [2023-12-26 17:24:02,044][105620] Updated weights for policy 1, policy_version 279907 (0.0009) [2023-12-26 17:24:02,166][105692] Updated weights for policy 0, policy_version 279822 (0.0010) [2023-12-26 17:24:02,231][105692] Updated weights for policy 0, policy_version 279832 (0.0010) [2023-12-26 17:24:02,293][105692] Updated weights for policy 0, policy_version 279842 (0.0011) [2023-12-26 17:24:02,834][105620] Updated weights for policy 1, policy_version 279917 (0.0008) [2023-12-26 17:24:02,893][105620] Updated weights for policy 1, policy_version 279927 (0.0008) [2023-12-26 17:24:02,955][105620] Updated weights for policy 1, policy_version 279937 (0.0007) [2023-12-26 17:24:03,031][105692] Updated weights for policy 0, policy_version 279852 (0.0010) [2023-12-26 17:24:03,091][105692] Updated weights for policy 0, policy_version 279862 (0.0010) [2023-12-26 17:24:03,144][105692] Updated weights for policy 0, policy_version 279872 (0.0010) [2023-12-26 17:24:03,689][105620] Updated weights for policy 1, policy_version 279947 (0.0008) [2023-12-26 17:24:03,735][105620] Updated weights for policy 1, policy_version 279957 (0.0008) [2023-12-26 17:24:03,779][105620] Updated weights for policy 1, policy_version 279967 (0.0007) [2023-12-26 17:24:03,884][105692] Updated weights for policy 0, policy_version 279882 (0.0010) [2023-12-26 17:24:03,935][105692] Updated weights for policy 0, policy_version 279892 (0.0010) [2023-12-26 17:24:04,002][105692] Updated weights for policy 0, policy_version 279902 (0.0011) [2023-12-26 17:24:04,056][105692] Updated weights for policy 0, policy_version 279912 (0.0009) [2023-12-26 17:24:04,544][105620] Updated weights for policy 1, policy_version 279977 (0.0007) [2023-12-26 17:24:04,597][105620] Updated weights for policy 1, policy_version 279987 (0.0008) [2023-12-26 17:24:04,650][105620] Updated weights for policy 1, policy_version 279997 (0.0009) [2023-12-26 17:24:04,701][105620] Updated weights for policy 1, policy_version 280007 (0.0010) [2023-12-26 17:24:04,712][105692] Updated weights for policy 0, policy_version 279922 (0.0010) [2023-12-26 17:24:04,762][105692] Updated weights for policy 0, policy_version 279932 (0.0010) [2023-12-26 17:24:04,820][105692] Updated weights for policy 0, policy_version 279942 (0.0010) [2023-12-26 17:24:05,452][105620] Updated weights for policy 1, policy_version 280017 (0.0010) [2023-12-26 17:24:05,455][105692] Updated weights for policy 0, policy_version 279952 (0.0010) [2023-12-26 17:24:05,503][105620] Updated weights for policy 1, policy_version 280027 (0.0010) [2023-12-26 17:24:05,516][105692] Updated weights for policy 0, policy_version 279962 (0.0010) [2023-12-26 17:24:05,559][105620] Updated weights for policy 1, policy_version 280037 (0.0010) [2023-12-26 17:24:05,575][105692] Updated weights for policy 0, policy_version 279972 (0.0008) [2023-12-26 17:24:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 143384576. Throughput: 0: 9722.0, 1: 9619.0. Samples: 143374416. Policy #0 lag: (min: 17.0, avg: 31.1, max: 32.0) [2023-12-26 17:24:06,063][104569] Avg episode reward: [(0, '9080.469'), (1, '3704.208')] [2023-12-26 17:24:06,213][105620] Updated weights for policy 1, policy_version 280047 (0.0008) [2023-12-26 17:24:06,275][105620] Updated weights for policy 1, policy_version 280057 (0.0008) [2023-12-26 17:24:06,332][105692] Updated weights for policy 0, policy_version 279982 (0.0009) [2023-12-26 17:24:06,337][105620] Updated weights for policy 1, policy_version 280067 (0.0006) [2023-12-26 17:24:06,392][105692] Updated weights for policy 0, policy_version 279992 (0.0010) [2023-12-26 17:24:06,449][105692] Updated weights for policy 0, policy_version 280002 (0.0010) [2023-12-26 17:24:06,992][105620] Updated weights for policy 1, policy_version 280077 (0.0008) [2023-12-26 17:24:07,047][105620] Updated weights for policy 1, policy_version 280087 (0.0010) [2023-12-26 17:24:07,095][105620] Updated weights for policy 1, policy_version 280097 (0.0010) [2023-12-26 17:24:07,274][105692] Updated weights for policy 0, policy_version 280012 (0.0010) [2023-12-26 17:24:07,331][105692] Updated weights for policy 0, policy_version 280022 (0.0008) [2023-12-26 17:24:07,392][105692] Updated weights for policy 0, policy_version 280032 (0.0008) [2023-12-26 17:24:07,772][105620] Updated weights for policy 1, policy_version 280107 (0.0010) [2023-12-26 17:24:07,826][105620] Updated weights for policy 1, policy_version 280117 (0.0008) [2023-12-26 17:24:07,880][105620] Updated weights for policy 1, policy_version 280127 (0.0005) [2023-12-26 17:24:08,082][105692] Updated weights for policy 0, policy_version 280042 (0.0009) [2023-12-26 17:24:08,139][105692] Updated weights for policy 0, policy_version 280052 (0.0009) [2023-12-26 17:24:08,201][105692] Updated weights for policy 0, policy_version 280062 (0.0010) [2023-12-26 17:24:08,249][105692] Updated weights for policy 0, policy_version 280072 (0.0010) [2023-12-26 17:24:08,510][105620] Updated weights for policy 1, policy_version 280137 (0.0006) [2023-12-26 17:24:08,569][105620] Updated weights for policy 1, policy_version 280147 (0.0008) [2023-12-26 17:24:08,637][105620] Updated weights for policy 1, policy_version 280157 (0.0008) [2023-12-26 17:24:08,708][105620] Updated weights for policy 1, policy_version 280167 (0.0010) [2023-12-26 17:24:08,924][105692] Updated weights for policy 0, policy_version 280082 (0.0005) [2023-12-26 17:24:08,988][105692] Updated weights for policy 0, policy_version 280092 (0.0005) [2023-12-26 17:24:09,048][105692] Updated weights for policy 0, policy_version 280102 (0.0005) [2023-12-26 17:24:09,571][105620] Updated weights for policy 1, policy_version 280177 (0.0008) [2023-12-26 17:24:09,629][105620] Updated weights for policy 1, policy_version 280187 (0.0006) [2023-12-26 17:24:09,631][105692] Updated weights for policy 0, policy_version 280112 (0.0010) [2023-12-26 17:24:09,688][105620] Updated weights for policy 1, policy_version 280197 (0.0006) [2023-12-26 17:24:09,690][105692] Updated weights for policy 0, policy_version 280122 (0.0011) [2023-12-26 17:24:09,757][105692] Updated weights for policy 0, policy_version 280132 (0.0011) [2023-12-26 17:24:10,494][105620] Updated weights for policy 1, policy_version 280207 (0.0007) [2023-12-26 17:24:10,509][105692] Updated weights for policy 0, policy_version 280142 (0.0010) [2023-12-26 17:24:10,558][105620] Updated weights for policy 1, policy_version 280217 (0.0011) [2023-12-26 17:24:10,565][105692] Updated weights for policy 0, policy_version 280152 (0.0008) [2023-12-26 17:24:10,617][105620] Updated weights for policy 1, policy_version 280227 (0.0011) [2023-12-26 17:24:10,618][105692] Updated weights for policy 0, policy_version 280162 (0.0010) [2023-12-26 17:24:11,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 143482880. Throughput: 0: 9781.2, 1: 9583.9. Samples: 143491876. Policy #0 lag: (min: 17.0, avg: 31.1, max: 32.0) [2023-12-26 17:24:11,062][104569] Avg episode reward: [(0, '9172.291'), (1, '7407.156')] [2023-12-26 17:24:11,335][105692] Updated weights for policy 0, policy_version 280172 (0.0007) [2023-12-26 17:24:11,361][105620] Updated weights for policy 1, policy_version 280237 (0.0011) [2023-12-26 17:24:11,402][105692] Updated weights for policy 0, policy_version 280182 (0.0008) [2023-12-26 17:24:11,419][105620] Updated weights for policy 1, policy_version 280247 (0.0007) [2023-12-26 17:24:11,454][105692] Updated weights for policy 0, policy_version 280192 (0.0008) [2023-12-26 17:24:11,475][105620] Updated weights for policy 1, policy_version 280257 (0.0008) [2023-12-26 17:24:12,156][105692] Updated weights for policy 0, policy_version 280202 (0.0007) [2023-12-26 17:24:12,164][105620] Updated weights for policy 1, policy_version 280267 (0.0011) [2023-12-26 17:24:12,218][105692] Updated weights for policy 0, policy_version 280212 (0.0008) [2023-12-26 17:24:12,221][105620] Updated weights for policy 1, policy_version 280277 (0.0008) [2023-12-26 17:24:12,281][105620] Updated weights for policy 1, policy_version 280287 (0.0009) [2023-12-26 17:24:12,281][105692] Updated weights for policy 0, policy_version 280222 (0.0011) [2023-12-26 17:24:12,342][105692] Updated weights for policy 0, policy_version 280232 (0.0010) [2023-12-26 17:24:12,944][105620] Updated weights for policy 1, policy_version 280297 (0.0010) [2023-12-26 17:24:13,016][105620] Updated weights for policy 1, policy_version 280307 (0.0009) [2023-12-26 17:24:13,078][105692] Updated weights for policy 0, policy_version 280242 (0.0011) [2023-12-26 17:24:13,079][105620] Updated weights for policy 1, policy_version 280317 (0.0007) [2023-12-26 17:24:13,124][105692] Updated weights for policy 0, policy_version 280252 (0.0011) [2023-12-26 17:24:13,138][105620] Updated weights for policy 1, policy_version 280327 (0.0008) [2023-12-26 17:24:13,166][105692] Updated weights for policy 0, policy_version 280262 (0.0010) [2023-12-26 17:24:13,875][105620] Updated weights for policy 1, policy_version 280337 (0.0008) [2023-12-26 17:24:13,930][105620] Updated weights for policy 1, policy_version 280347 (0.0008) [2023-12-26 17:24:13,952][105692] Updated weights for policy 0, policy_version 280272 (0.0011) [2023-12-26 17:24:13,978][105620] Updated weights for policy 1, policy_version 280357 (0.0005) [2023-12-26 17:24:14,007][105692] Updated weights for policy 0, policy_version 280282 (0.0010) [2023-12-26 17:24:14,061][105692] Updated weights for policy 0, policy_version 280292 (0.0010) [2023-12-26 17:24:14,739][105620] Updated weights for policy 1, policy_version 280367 (0.0008) [2023-12-26 17:24:14,802][105620] Updated weights for policy 1, policy_version 280377 (0.0009) [2023-12-26 17:24:14,814][105692] Updated weights for policy 0, policy_version 280302 (0.0010) [2023-12-26 17:24:14,863][105620] Updated weights for policy 1, policy_version 280387 (0.0009) [2023-12-26 17:24:14,878][105692] Updated weights for policy 0, policy_version 280312 (0.0011) [2023-12-26 17:24:14,938][105692] Updated weights for policy 0, policy_version 280322 (0.0011) [2023-12-26 17:24:15,632][105620] Updated weights for policy 1, policy_version 280397 (0.0008) [2023-12-26 17:24:15,687][105620] Updated weights for policy 1, policy_version 280407 (0.0008) [2023-12-26 17:24:15,704][105692] Updated weights for policy 0, policy_version 280332 (0.0011) [2023-12-26 17:24:15,737][105620] Updated weights for policy 1, policy_version 280417 (0.0007) [2023-12-26 17:24:15,761][105692] Updated weights for policy 0, policy_version 280342 (0.0011) [2023-12-26 17:24:15,809][105692] Updated weights for policy 0, policy_version 280352 (0.0010) [2023-12-26 17:24:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 143581184. Throughput: 0: 9780.9, 1: 9598.5. Samples: 143550112. Policy #0 lag: (min: 17.0, avg: 31.1, max: 32.0) [2023-12-26 17:24:16,062][104569] Avg episode reward: [(0, '9355.373'), (1, '9262.110')] [2023-12-26 17:24:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000280360_71786496.pth... [2023-12-26 17:24:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000280424_71794688.pth... [2023-12-26 17:24:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000279240_71499776.pth [2023-12-26 17:24:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000279304_71507968.pth [2023-12-26 17:24:16,463][105620] Updated weights for policy 1, policy_version 280427 (0.0009) [2023-12-26 17:24:16,520][105620] Updated weights for policy 1, policy_version 280437 (0.0007) [2023-12-26 17:24:16,534][105692] Updated weights for policy 0, policy_version 280362 (0.0010) [2023-12-26 17:24:16,581][105620] Updated weights for policy 1, policy_version 280447 (0.0005) [2023-12-26 17:24:16,590][105692] Updated weights for policy 0, policy_version 280372 (0.0011) [2023-12-26 17:24:16,649][105692] Updated weights for policy 0, policy_version 280382 (0.0010) [2023-12-26 17:24:16,694][105692] Updated weights for policy 0, policy_version 280392 (0.0010) [2023-12-26 17:24:17,155][105620] Updated weights for policy 1, policy_version 280457 (0.0005) [2023-12-26 17:24:17,222][105620] Updated weights for policy 1, policy_version 280467 (0.0005) [2023-12-26 17:24:17,274][105620] Updated weights for policy 1, policy_version 280477 (0.0005) [2023-12-26 17:24:17,332][105620] Updated weights for policy 1, policy_version 280487 (0.0009) [2023-12-26 17:24:17,341][105692] Updated weights for policy 0, policy_version 280402 (0.0005) [2023-12-26 17:24:17,401][105692] Updated weights for policy 0, policy_version 280412 (0.0006) [2023-12-26 17:24:17,457][105692] Updated weights for policy 0, policy_version 280422 (0.0005) [2023-12-26 17:24:17,933][105620] Updated weights for policy 1, policy_version 280497 (0.0005) [2023-12-26 17:24:17,989][105620] Updated weights for policy 1, policy_version 280507 (0.0005) [2023-12-26 17:24:17,993][105692] Updated weights for policy 0, policy_version 280432 (0.0005) [2023-12-26 17:24:18,048][105620] Updated weights for policy 1, policy_version 280517 (0.0007) [2023-12-26 17:24:18,053][105692] Updated weights for policy 0, policy_version 280442 (0.0008) [2023-12-26 17:24:18,120][105692] Updated weights for policy 0, policy_version 280452 (0.0006) [2023-12-26 17:24:18,770][105620] Updated weights for policy 1, policy_version 280527 (0.0009) [2023-12-26 17:24:18,780][105692] Updated weights for policy 0, policy_version 280462 (0.0010) [2023-12-26 17:24:18,828][105620] Updated weights for policy 1, policy_version 280537 (0.0006) [2023-12-26 17:24:18,828][105692] Updated weights for policy 0, policy_version 280472 (0.0008) [2023-12-26 17:24:18,888][105620] Updated weights for policy 1, policy_version 280547 (0.0008) [2023-12-26 17:24:18,889][105692] Updated weights for policy 0, policy_version 280482 (0.0009) [2023-12-26 17:24:19,567][105620] Updated weights for policy 1, policy_version 280557 (0.0008) [2023-12-26 17:24:19,580][105692] Updated weights for policy 0, policy_version 280492 (0.0011) [2023-12-26 17:24:19,628][105620] Updated weights for policy 1, policy_version 280567 (0.0011) [2023-12-26 17:24:19,640][105692] Updated weights for policy 0, policy_version 280502 (0.0011) [2023-12-26 17:24:19,688][105620] Updated weights for policy 1, policy_version 280577 (0.0010) [2023-12-26 17:24:19,703][105692] Updated weights for policy 0, policy_version 280512 (0.0011) [2023-12-26 17:24:20,378][105620] Updated weights for policy 1, policy_version 280587 (0.0010) [2023-12-26 17:24:20,428][105620] Updated weights for policy 1, policy_version 280597 (0.0008) [2023-12-26 17:24:20,458][105692] Updated weights for policy 0, policy_version 280522 (0.0011) [2023-12-26 17:24:20,485][105620] Updated weights for policy 1, policy_version 280607 (0.0007) [2023-12-26 17:24:20,522][105692] Updated weights for policy 0, policy_version 280532 (0.0011) [2023-12-26 17:24:20,587][105692] Updated weights for policy 0, policy_version 280542 (0.0011) [2023-12-26 17:24:20,649][105692] Updated weights for policy 0, policy_version 280552 (0.0009) [2023-12-26 17:24:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 143679488. Throughput: 0: 9736.2, 1: 9685.0. Samples: 143670812. Policy #0 lag: (min: 17.0, avg: 31.1, max: 32.0) [2023-12-26 17:24:21,063][104569] Avg episode reward: [(0, '9355.707'), (1, '9354.626')] [2023-12-26 17:24:21,276][105620] Updated weights for policy 1, policy_version 280617 (0.0006) [2023-12-26 17:24:21,347][105620] Updated weights for policy 1, policy_version 280627 (0.0007) [2023-12-26 17:24:21,397][105692] Updated weights for policy 0, policy_version 280562 (0.0008) [2023-12-26 17:24:21,418][105620] Updated weights for policy 1, policy_version 280637 (0.0010) [2023-12-26 17:24:21,457][105692] Updated weights for policy 0, policy_version 280572 (0.0010) [2023-12-26 17:24:21,480][105620] Updated weights for policy 1, policy_version 280647 (0.0006) [2023-12-26 17:24:21,513][105692] Updated weights for policy 0, policy_version 280582 (0.0010) [2023-12-26 17:24:22,084][105620] Updated weights for policy 1, policy_version 280657 (0.0005) [2023-12-26 17:24:22,138][105620] Updated weights for policy 1, policy_version 280667 (0.0008) [2023-12-26 17:24:22,187][105620] Updated weights for policy 1, policy_version 280677 (0.0009) [2023-12-26 17:24:22,318][105692] Updated weights for policy 0, policy_version 280592 (0.0008) [2023-12-26 17:24:22,384][105692] Updated weights for policy 0, policy_version 280602 (0.0010) [2023-12-26 17:24:22,443][105692] Updated weights for policy 0, policy_version 280612 (0.0009) [2023-12-26 17:24:22,980][105620] Updated weights for policy 1, policy_version 280687 (0.0009) [2023-12-26 17:24:23,043][105620] Updated weights for policy 1, policy_version 280697 (0.0008) [2023-12-26 17:24:23,102][105620] Updated weights for policy 1, policy_version 280707 (0.0007) [2023-12-26 17:24:23,157][105692] Updated weights for policy 0, policy_version 280622 (0.0010) [2023-12-26 17:24:23,215][105692] Updated weights for policy 0, policy_version 280632 (0.0010) [2023-12-26 17:24:23,275][105692] Updated weights for policy 0, policy_version 280642 (0.0011) [2023-12-26 17:24:23,703][105620] Updated weights for policy 1, policy_version 280717 (0.0007) [2023-12-26 17:24:23,761][105620] Updated weights for policy 1, policy_version 280727 (0.0010) [2023-12-26 17:24:23,832][105620] Updated weights for policy 1, policy_version 280737 (0.0010) [2023-12-26 17:24:23,889][105692] Updated weights for policy 0, policy_version 280652 (0.0008) [2023-12-26 17:24:23,942][105692] Updated weights for policy 0, policy_version 280662 (0.0006) [2023-12-26 17:24:23,997][105692] Updated weights for policy 0, policy_version 280672 (0.0010) [2023-12-26 17:24:24,612][105620] Updated weights for policy 1, policy_version 280747 (0.0009) [2023-12-26 17:24:24,673][105620] Updated weights for policy 1, policy_version 280757 (0.0006) [2023-12-26 17:24:24,726][105620] Updated weights for policy 1, policy_version 280767 (0.0007) [2023-12-26 17:24:24,730][105692] Updated weights for policy 0, policy_version 280682 (0.0010) [2023-12-26 17:24:24,784][105692] Updated weights for policy 0, policy_version 280692 (0.0009) [2023-12-26 17:24:24,845][105692] Updated weights for policy 0, policy_version 280702 (0.0006) [2023-12-26 17:24:24,899][105692] Updated weights for policy 0, policy_version 280712 (0.0010) [2023-12-26 17:24:25,373][105620] Updated weights for policy 1, policy_version 280777 (0.0006) [2023-12-26 17:24:25,421][105620] Updated weights for policy 1, policy_version 280787 (0.0008) [2023-12-26 17:24:25,471][105620] Updated weights for policy 1, policy_version 280797 (0.0007) [2023-12-26 17:24:25,473][105692] Updated weights for policy 0, policy_version 280722 (0.0010) [2023-12-26 17:24:25,516][105620] Updated weights for policy 1, policy_version 280807 (0.0005) [2023-12-26 17:24:25,518][105692] Updated weights for policy 0, policy_version 280732 (0.0010) [2023-12-26 17:24:25,566][105692] Updated weights for policy 0, policy_version 280742 (0.0010) [2023-12-26 17:24:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 143777792. Throughput: 0: 9727.8, 1: 9649.3. Samples: 143788568. Policy #0 lag: (min: 17.0, avg: 31.1, max: 32.0) [2023-12-26 17:24:26,062][104569] Avg episode reward: [(0, '9356.229'), (1, '9172.740')] [2023-12-26 17:24:26,276][105620] Updated weights for policy 1, policy_version 280817 (0.0008) [2023-12-26 17:24:26,324][105692] Updated weights for policy 0, policy_version 280752 (0.0010) [2023-12-26 17:24:26,330][105620] Updated weights for policy 1, policy_version 280827 (0.0007) [2023-12-26 17:24:26,379][105692] Updated weights for policy 0, policy_version 280762 (0.0010) [2023-12-26 17:24:26,381][105620] Updated weights for policy 1, policy_version 280837 (0.0005) [2023-12-26 17:24:26,430][105692] Updated weights for policy 0, policy_version 280772 (0.0010) [2023-12-26 17:24:27,101][105620] Updated weights for policy 1, policy_version 280847 (0.0005) [2023-12-26 17:24:27,152][105620] Updated weights for policy 1, policy_version 280857 (0.0005) [2023-12-26 17:24:27,190][105692] Updated weights for policy 0, policy_version 280782 (0.0010) [2023-12-26 17:24:27,203][105620] Updated weights for policy 1, policy_version 280867 (0.0005) [2023-12-26 17:24:27,238][105692] Updated weights for policy 0, policy_version 280792 (0.0010) [2023-12-26 17:24:27,287][105692] Updated weights for policy 0, policy_version 280802 (0.0007) [2023-12-26 17:24:27,852][105692] Updated weights for policy 0, policy_version 280812 (0.0007) [2023-12-26 17:24:27,894][105620] Updated weights for policy 1, policy_version 280877 (0.0005) [2023-12-26 17:24:27,918][105692] Updated weights for policy 0, policy_version 280822 (0.0009) [2023-12-26 17:24:27,953][105620] Updated weights for policy 1, policy_version 280887 (0.0006) [2023-12-26 17:24:27,964][105692] Updated weights for policy 0, policy_version 280832 (0.0011) [2023-12-26 17:24:28,011][105620] Updated weights for policy 1, policy_version 280897 (0.0006) [2023-12-26 17:24:28,674][105620] Updated weights for policy 1, policy_version 280907 (0.0008) [2023-12-26 17:24:28,703][105692] Updated weights for policy 0, policy_version 280842 (0.0011) [2023-12-26 17:24:28,725][105620] Updated weights for policy 1, policy_version 280917 (0.0006) [2023-12-26 17:24:28,757][105692] Updated weights for policy 0, policy_version 280852 (0.0010) [2023-12-26 17:24:28,773][105620] Updated weights for policy 1, policy_version 280927 (0.0008) [2023-12-26 17:24:28,804][105692] Updated weights for policy 0, policy_version 280862 (0.0010) [2023-12-26 17:24:28,851][105692] Updated weights for policy 0, policy_version 280872 (0.0010) [2023-12-26 17:24:29,504][105692] Updated weights for policy 0, policy_version 280882 (0.0009) [2023-12-26 17:24:29,565][105692] Updated weights for policy 0, policy_version 280892 (0.0009) [2023-12-26 17:24:29,577][105620] Updated weights for policy 1, policy_version 280937 (0.0007) [2023-12-26 17:24:29,619][105692] Updated weights for policy 0, policy_version 280902 (0.0007) [2023-12-26 17:24:29,636][105620] Updated weights for policy 1, policy_version 280947 (0.0008) [2023-12-26 17:24:29,695][105620] Updated weights for policy 1, policy_version 280957 (0.0010) [2023-12-26 17:24:29,747][105620] Updated weights for policy 1, policy_version 280967 (0.0009) [2023-12-26 17:24:30,260][105692] Updated weights for policy 0, policy_version 280912 (0.0006) [2023-12-26 17:24:30,334][105692] Updated weights for policy 0, policy_version 280922 (0.0006) [2023-12-26 17:24:30,393][105692] Updated weights for policy 0, policy_version 280932 (0.0009) [2023-12-26 17:24:30,552][105620] Updated weights for policy 1, policy_version 280978 (0.0010) [2023-12-26 17:24:30,603][105620] Updated weights for policy 1, policy_version 280988 (0.0008) [2023-12-26 17:24:30,649][105620] Updated weights for policy 1, policy_version 280998 (0.0005) [2023-12-26 17:24:30,923][105692] Updated weights for policy 0, policy_version 280942 (0.0009) [2023-12-26 17:24:30,974][105692] Updated weights for policy 0, policy_version 280952 (0.0007) [2023-12-26 17:24:31,022][105692] Updated weights for policy 0, policy_version 280962 (0.0008) [2023-12-26 17:24:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 143884288. Throughput: 0: 9806.0, 1: 9634.3. Samples: 143848812. Policy #0 lag: (min: 17.0, avg: 31.1, max: 32.0) [2023-12-26 17:24:31,062][104569] Avg episode reward: [(0, '9266.718'), (1, '9172.869')] [2023-12-26 17:24:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000280968_71942144.pth... [2023-12-26 17:24:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000281000_71942144.pth... [2023-12-26 17:24:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000279784_71639040.pth [2023-12-26 17:24:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000279880_71655424.pth [2023-12-26 17:24:31,455][105620] Updated weights for policy 1, policy_version 281008 (0.0010) [2023-12-26 17:24:31,512][105620] Updated weights for policy 1, policy_version 281018 (0.0008) [2023-12-26 17:24:31,572][105620] Updated weights for policy 1, policy_version 281028 (0.0009) [2023-12-26 17:24:31,813][105692] Updated weights for policy 0, policy_version 280972 (0.0006) [2023-12-26 17:24:31,882][105692] Updated weights for policy 0, policy_version 280982 (0.0005) [2023-12-26 17:24:31,936][105692] Updated weights for policy 0, policy_version 280992 (0.0005) [2023-12-26 17:24:32,256][105620] Updated weights for policy 1, policy_version 281038 (0.0007) [2023-12-26 17:24:32,312][105620] Updated weights for policy 1, policy_version 281048 (0.0006) [2023-12-26 17:24:32,381][105620] Updated weights for policy 1, policy_version 281058 (0.0007) [2023-12-26 17:24:32,508][105692] Updated weights for policy 0, policy_version 281002 (0.0005) [2023-12-26 17:24:32,563][105692] Updated weights for policy 0, policy_version 281012 (0.0005) [2023-12-26 17:24:32,614][105692] Updated weights for policy 0, policy_version 281022 (0.0005) [2023-12-26 17:24:32,676][105692] Updated weights for policy 0, policy_version 281032 (0.0005) [2023-12-26 17:24:33,037][105620] Updated weights for policy 1, policy_version 281068 (0.0009) [2023-12-26 17:24:33,101][105620] Updated weights for policy 1, policy_version 281078 (0.0009) [2023-12-26 17:24:33,151][105620] Updated weights for policy 1, policy_version 281088 (0.0005) [2023-12-26 17:24:33,273][105692] Updated weights for policy 0, policy_version 281042 (0.0009) [2023-12-26 17:24:33,327][105692] Updated weights for policy 0, policy_version 281052 (0.0009) [2023-12-26 17:24:33,379][105692] Updated weights for policy 0, policy_version 281062 (0.0010) [2023-12-26 17:24:33,780][105620] Updated weights for policy 1, policy_version 281098 (0.0007) [2023-12-26 17:24:33,826][105620] Updated weights for policy 1, policy_version 281108 (0.0009) [2023-12-26 17:24:33,874][105620] Updated weights for policy 1, policy_version 281118 (0.0008) [2023-12-26 17:24:33,919][105620] Updated weights for policy 1, policy_version 281128 (0.0008) [2023-12-26 17:24:34,182][105692] Updated weights for policy 0, policy_version 281072 (0.0009) [2023-12-26 17:24:34,247][105692] Updated weights for policy 0, policy_version 281082 (0.0008) [2023-12-26 17:24:34,310][105692] Updated weights for policy 0, policy_version 281092 (0.0005) [2023-12-26 17:24:34,724][105620] Updated weights for policy 1, policy_version 281138 (0.0009) [2023-12-26 17:24:34,778][105620] Updated weights for policy 1, policy_version 281149 (0.0010) [2023-12-26 17:24:34,829][105620] Updated weights for policy 1, policy_version 281159 (0.0009) [2023-12-26 17:24:34,935][105692] Updated weights for policy 0, policy_version 281102 (0.0008) [2023-12-26 17:24:34,990][105692] Updated weights for policy 0, policy_version 281112 (0.0009) [2023-12-26 17:24:35,047][105692] Updated weights for policy 0, policy_version 281122 (0.0009) [2023-12-26 17:24:35,594][105620] Updated weights for policy 1, policy_version 281169 (0.0006) [2023-12-26 17:24:35,649][105620] Updated weights for policy 1, policy_version 281179 (0.0008) [2023-12-26 17:24:35,712][105620] Updated weights for policy 1, policy_version 281189 (0.0006) [2023-12-26 17:24:35,809][105692] Updated weights for policy 0, policy_version 281132 (0.0009) [2023-12-26 17:24:35,860][105692] Updated weights for policy 0, policy_version 281142 (0.0010) [2023-12-26 17:24:35,911][105692] Updated weights for policy 0, policy_version 281152 (0.0010) [2023-12-26 17:24:36,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 143982592. Throughput: 0: 9897.9, 1: 9680.9. Samples: 143968184. Policy #0 lag: (min: 17.0, avg: 31.1, max: 32.0) [2023-12-26 17:24:36,063][104569] Avg episode reward: [(0, '9266.888'), (1, '9264.303')] [2023-12-26 17:24:36,318][105620] Updated weights for policy 1, policy_version 281199 (0.0007) [2023-12-26 17:24:36,366][105620] Updated weights for policy 1, policy_version 281209 (0.0008) [2023-12-26 17:24:36,433][105620] Updated weights for policy 1, policy_version 281219 (0.0008) [2023-12-26 17:24:36,670][105692] Updated weights for policy 0, policy_version 281162 (0.0010) [2023-12-26 17:24:36,725][105692] Updated weights for policy 0, policy_version 281172 (0.0008) [2023-12-26 17:24:36,778][105692] Updated weights for policy 0, policy_version 281182 (0.0005) [2023-12-26 17:24:36,832][105692] Updated weights for policy 0, policy_version 281192 (0.0005) [2023-12-26 17:24:37,236][105620] Updated weights for policy 1, policy_version 281229 (0.0009) [2023-12-26 17:24:37,301][105620] Updated weights for policy 1, policy_version 281239 (0.0010) [2023-12-26 17:24:37,362][105620] Updated weights for policy 1, policy_version 281249 (0.0010) [2023-12-26 17:24:37,505][105692] Updated weights for policy 0, policy_version 281202 (0.0010) [2023-12-26 17:24:37,566][105692] Updated weights for policy 0, policy_version 281212 (0.0010) [2023-12-26 17:24:37,624][105692] Updated weights for policy 0, policy_version 281222 (0.0010) [2023-12-26 17:24:38,095][105620] Updated weights for policy 1, policy_version 281259 (0.0010) [2023-12-26 17:24:38,152][105620] Updated weights for policy 1, policy_version 281269 (0.0008) [2023-12-26 17:24:38,201][105620] Updated weights for policy 1, policy_version 281279 (0.0008) [2023-12-26 17:24:38,375][105692] Updated weights for policy 0, policy_version 281232 (0.0011) [2023-12-26 17:24:38,440][105692] Updated weights for policy 0, policy_version 281242 (0.0011) [2023-12-26 17:24:38,506][105692] Updated weights for policy 0, policy_version 281252 (0.0011) [2023-12-26 17:24:38,998][105620] Updated weights for policy 1, policy_version 281289 (0.0008) [2023-12-26 17:24:39,050][105620] Updated weights for policy 1, policy_version 281299 (0.0008) [2023-12-26 17:24:39,109][105620] Updated weights for policy 1, policy_version 281309 (0.0008) [2023-12-26 17:24:39,169][105620] Updated weights for policy 1, policy_version 281319 (0.0007) [2023-12-26 17:24:39,227][105692] Updated weights for policy 0, policy_version 281262 (0.0009) [2023-12-26 17:24:39,292][105692] Updated weights for policy 0, policy_version 281272 (0.0009) [2023-12-26 17:24:39,359][105692] Updated weights for policy 0, policy_version 281282 (0.0009) [2023-12-26 17:24:40,000][105620] Updated weights for policy 1, policy_version 281329 (0.0008) [2023-12-26 17:24:40,046][105692] Updated weights for policy 0, policy_version 281292 (0.0008) [2023-12-26 17:24:40,071][105620] Updated weights for policy 1, policy_version 281339 (0.0007) [2023-12-26 17:24:40,100][105692] Updated weights for policy 0, policy_version 281302 (0.0008) [2023-12-26 17:24:40,140][105620] Updated weights for policy 1, policy_version 281349 (0.0008) [2023-12-26 17:24:40,160][105692] Updated weights for policy 0, policy_version 281312 (0.0008) [2023-12-26 17:24:40,816][105620] Updated weights for policy 1, policy_version 281359 (0.0009) [2023-12-26 17:24:40,868][105620] Updated weights for policy 1, policy_version 281369 (0.0009) [2023-12-26 17:24:40,925][105692] Updated weights for policy 0, policy_version 281322 (0.0010) [2023-12-26 17:24:40,926][105620] Updated weights for policy 1, policy_version 281379 (0.0009) [2023-12-26 17:24:40,980][105692] Updated weights for policy 0, policy_version 281332 (0.0008) [2023-12-26 17:24:41,037][105692] Updated weights for policy 0, policy_version 281342 (0.0009) [2023-12-26 17:24:41,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 144072704. Throughput: 0: 9901.2, 1: 9694.7. Samples: 144082196. Policy #0 lag: (min: 17.0, avg: 31.1, max: 32.0) [2023-12-26 17:24:41,063][104569] Avg episode reward: [(0, '9266.575'), (1, '9080.495')] [2023-12-26 17:24:41,106][105692] Updated weights for policy 0, policy_version 281352 (0.0009) [2023-12-26 17:24:41,704][105620] Updated weights for policy 1, policy_version 281389 (0.0008) [2023-12-26 17:24:41,778][105620] Updated weights for policy 1, policy_version 281399 (0.0008) [2023-12-26 17:24:41,829][105620] Updated weights for policy 1, policy_version 281409 (0.0009) [2023-12-26 17:24:41,892][105692] Updated weights for policy 0, policy_version 281362 (0.0009) [2023-12-26 17:24:41,951][105692] Updated weights for policy 0, policy_version 281372 (0.0008) [2023-12-26 17:24:42,003][105692] Updated weights for policy 0, policy_version 281382 (0.0008) [2023-12-26 17:24:42,646][105620] Updated weights for policy 1, policy_version 281419 (0.0008) [2023-12-26 17:24:42,703][105620] Updated weights for policy 1, policy_version 281429 (0.0008) [2023-12-26 17:24:42,725][105692] Updated weights for policy 0, policy_version 281392 (0.0008) [2023-12-26 17:24:42,764][105620] Updated weights for policy 1, policy_version 281439 (0.0008) [2023-12-26 17:24:42,783][105692] Updated weights for policy 0, policy_version 281402 (0.0007) [2023-12-26 17:24:42,832][105692] Updated weights for policy 0, policy_version 281412 (0.0008) [2023-12-26 17:24:43,490][105620] Updated weights for policy 1, policy_version 281449 (0.0007) [2023-12-26 17:24:43,537][105692] Updated weights for policy 0, policy_version 281422 (0.0008) [2023-12-26 17:24:43,543][105620] Updated weights for policy 1, policy_version 281459 (0.0006) [2023-12-26 17:24:43,588][105620] Updated weights for policy 1, policy_version 281469 (0.0007) [2023-12-26 17:24:43,593][105692] Updated weights for policy 0, policy_version 281432 (0.0008) [2023-12-26 17:24:43,636][105620] Updated weights for policy 1, policy_version 281479 (0.0005) [2023-12-26 17:24:43,654][105692] Updated weights for policy 0, policy_version 281442 (0.0008) [2023-12-26 17:24:44,247][105692] Updated weights for policy 0, policy_version 281452 (0.0009) [2023-12-26 17:24:44,297][105692] Updated weights for policy 0, policy_version 281462 (0.0005) [2023-12-26 17:24:44,353][105692] Updated weights for policy 0, policy_version 281472 (0.0007) [2023-12-26 17:24:44,457][105620] Updated weights for policy 1, policy_version 281489 (0.0006) [2023-12-26 17:24:44,514][105620] Updated weights for policy 1, policy_version 281499 (0.0005) [2023-12-26 17:24:44,560][105620] Updated weights for policy 1, policy_version 281509 (0.0007) [2023-12-26 17:24:45,089][105692] Updated weights for policy 0, policy_version 281482 (0.0010) [2023-12-26 17:24:45,153][105692] Updated weights for policy 0, policy_version 281492 (0.0011) [2023-12-26 17:24:45,194][105620] Updated weights for policy 1, policy_version 281519 (0.0006) [2023-12-26 17:24:45,216][105692] Updated weights for policy 0, policy_version 281502 (0.0011) [2023-12-26 17:24:45,249][105620] Updated weights for policy 1, policy_version 281529 (0.0005) [2023-12-26 17:24:45,276][105692] Updated weights for policy 0, policy_version 281512 (0.0011) [2023-12-26 17:24:45,309][105620] Updated weights for policy 1, policy_version 281539 (0.0006) [2023-12-26 17:24:45,826][105620] Updated weights for policy 1, policy_version 281549 (0.0005) [2023-12-26 17:24:45,882][105620] Updated weights for policy 1, policy_version 281560 (0.0007) [2023-12-26 17:24:45,930][105620] Updated weights for policy 1, policy_version 281570 (0.0007) [2023-12-26 17:24:45,937][105692] Updated weights for policy 0, policy_version 281522 (0.0007) [2023-12-26 17:24:45,995][105692] Updated weights for policy 0, policy_version 281532 (0.0010) [2023-12-26 17:24:46,049][105692] Updated weights for policy 0, policy_version 281542 (0.0010) [2023-12-26 17:24:46,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 144179200. Throughput: 0: 9897.0, 1: 9683.7. Samples: 144137792. Policy #0 lag: (min: 17.0, avg: 31.1, max: 32.0) [2023-12-26 17:24:46,062][104569] Avg episode reward: [(0, '9355.486'), (1, '8621.035')] [2023-12-26 17:24:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000281576_72089600.pth... [2023-12-26 17:24:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000281544_72089600.pth... [2023-12-26 17:24:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000280424_71794688.pth [2023-12-26 17:24:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000280360_71786496.pth [2023-12-26 17:24:46,545][105620] Updated weights for policy 1, policy_version 281580 (0.0005) [2023-12-26 17:24:46,609][105620] Updated weights for policy 1, policy_version 281590 (0.0008) [2023-12-26 17:24:46,675][105620] Updated weights for policy 1, policy_version 281600 (0.0010) [2023-12-26 17:24:46,889][105692] Updated weights for policy 0, policy_version 281552 (0.0009) [2023-12-26 17:24:46,941][105692] Updated weights for policy 0, policy_version 281562 (0.0008) [2023-12-26 17:24:46,994][105692] Updated weights for policy 0, policy_version 281572 (0.0008) [2023-12-26 17:24:47,250][105620] Updated weights for policy 1, policy_version 281610 (0.0007) [2023-12-26 17:24:47,310][105620] Updated weights for policy 1, policy_version 281620 (0.0007) [2023-12-26 17:24:47,377][105620] Updated weights for policy 1, policy_version 281630 (0.0005) [2023-12-26 17:24:47,423][105620] Updated weights for policy 1, policy_version 281640 (0.0009) [2023-12-26 17:24:47,783][105692] Updated weights for policy 0, policy_version 281583 (0.0010) [2023-12-26 17:24:47,855][105692] Updated weights for policy 0, policy_version 281593 (0.0009) [2023-12-26 17:24:47,902][105692] Updated weights for policy 0, policy_version 281603 (0.0009) [2023-12-26 17:24:48,100][105620] Updated weights for policy 1, policy_version 281650 (0.0007) [2023-12-26 17:24:48,148][105620] Updated weights for policy 1, policy_version 281660 (0.0005) [2023-12-26 17:24:48,205][105620] Updated weights for policy 1, policy_version 281670 (0.0009) [2023-12-26 17:24:48,633][105692] Updated weights for policy 0, policy_version 281613 (0.0009) [2023-12-26 17:24:48,698][105692] Updated weights for policy 0, policy_version 281623 (0.0008) [2023-12-26 17:24:48,767][105692] Updated weights for policy 0, policy_version 281633 (0.0009) [2023-12-26 17:24:48,939][105620] Updated weights for policy 1, policy_version 281680 (0.0007) [2023-12-26 17:24:48,999][105620] Updated weights for policy 1, policy_version 281690 (0.0008) [2023-12-26 17:24:49,053][105620] Updated weights for policy 1, policy_version 281700 (0.0005) [2023-12-26 17:24:49,575][105692] Updated weights for policy 0, policy_version 281643 (0.0008) [2023-12-26 17:24:49,644][105692] Updated weights for policy 0, policy_version 281653 (0.0007) [2023-12-26 17:24:49,645][105620] Updated weights for policy 1, policy_version 281710 (0.0007) [2023-12-26 17:24:49,703][105620] Updated weights for policy 1, policy_version 281720 (0.0008) [2023-12-26 17:24:49,709][105692] Updated weights for policy 0, policy_version 281663 (0.0007) [2023-12-26 17:24:49,720][105586] KL-divergence is very high: 102.3014 [2023-12-26 17:24:49,761][105620] Updated weights for policy 1, policy_version 281730 (0.0008) [2023-12-26 17:24:49,766][105586] KL-divergence is very high: 100.7471 [2023-12-26 17:24:50,414][105692] Updated weights for policy 0, policy_version 281673 (0.0007) [2023-12-26 17:24:50,463][105692] Updated weights for policy 0, policy_version 281683 (0.0008) [2023-12-26 17:24:50,509][105692] Updated weights for policy 0, policy_version 281693 (0.0008) [2023-12-26 17:24:50,522][105620] Updated weights for policy 1, policy_version 281740 (0.0008) [2023-12-26 17:24:50,564][105692] Updated weights for policy 0, policy_version 281703 (0.0007) [2023-12-26 17:24:50,581][105620] Updated weights for policy 1, policy_version 281750 (0.0009) [2023-12-26 17:24:50,648][105620] Updated weights for policy 1, policy_version 281760 (0.0009) [2023-12-26 17:24:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 144269312. Throughput: 0: 9861.8, 1: 9807.8. Samples: 144259548. Policy #0 lag: (min: 17.0, avg: 31.1, max: 32.0) [2023-12-26 17:24:51,062][104569] Avg episode reward: [(0, '9266.210'), (1, '8623.829')] [2023-12-26 17:24:51,290][105692] Updated weights for policy 0, policy_version 281713 (0.0009) [2023-12-26 17:24:51,359][105692] Updated weights for policy 0, policy_version 281723 (0.0009) [2023-12-26 17:24:51,402][105620] Updated weights for policy 1, policy_version 281770 (0.0010) [2023-12-26 17:24:51,424][105692] Updated weights for policy 0, policy_version 281733 (0.0009) [2023-12-26 17:24:51,451][105620] Updated weights for policy 1, policy_version 281780 (0.0007) [2023-12-26 17:24:51,516][105620] Updated weights for policy 1, policy_version 281790 (0.0005) [2023-12-26 17:24:51,575][105620] Updated weights for policy 1, policy_version 281800 (0.0005) [2023-12-26 17:24:52,194][105692] Updated weights for policy 0, policy_version 281743 (0.0008) [2023-12-26 17:24:52,256][105692] Updated weights for policy 0, policy_version 281753 (0.0007) [2023-12-26 17:24:52,264][105620] Updated weights for policy 1, policy_version 281810 (0.0007) [2023-12-26 17:24:52,323][105692] Updated weights for policy 0, policy_version 281763 (0.0007) [2023-12-26 17:24:52,329][105620] Updated weights for policy 1, policy_version 281820 (0.0007) [2023-12-26 17:24:52,392][105620] Updated weights for policy 1, policy_version 281830 (0.0007) [2023-12-26 17:24:53,044][105620] Updated weights for policy 1, policy_version 281840 (0.0009) [2023-12-26 17:24:53,081][105692] Updated weights for policy 0, policy_version 281773 (0.0007) [2023-12-26 17:24:53,105][105620] Updated weights for policy 1, policy_version 281850 (0.0010) [2023-12-26 17:24:53,140][105692] Updated weights for policy 0, policy_version 281783 (0.0009) [2023-12-26 17:24:53,169][105620] Updated weights for policy 1, policy_version 281860 (0.0010) [2023-12-26 17:24:53,192][105692] Updated weights for policy 0, policy_version 281793 (0.0007) [2023-12-26 17:24:53,812][105620] Updated weights for policy 1, policy_version 281870 (0.0007) [2023-12-26 17:24:53,851][105692] Updated weights for policy 0, policy_version 281803 (0.0009) [2023-12-26 17:24:53,864][105620] Updated weights for policy 1, policy_version 281880 (0.0005) [2023-12-26 17:24:53,913][105692] Updated weights for policy 0, policy_version 281813 (0.0009) [2023-12-26 17:24:53,919][105620] Updated weights for policy 1, policy_version 281890 (0.0010) [2023-12-26 17:24:53,969][105692] Updated weights for policy 0, policy_version 281823 (0.0006) [2023-12-26 17:24:54,636][105620] Updated weights for policy 1, policy_version 281900 (0.0010) [2023-12-26 17:24:54,696][105620] Updated weights for policy 1, policy_version 281910 (0.0010) [2023-12-26 17:24:54,721][105692] Updated weights for policy 0, policy_version 281833 (0.0008) [2023-12-26 17:24:54,754][105620] Updated weights for policy 1, policy_version 281920 (0.0010) [2023-12-26 17:24:54,773][105692] Updated weights for policy 0, policy_version 281843 (0.0006) [2023-12-26 17:24:54,828][105692] Updated weights for policy 0, policy_version 281853 (0.0007) [2023-12-26 17:24:54,887][105692] Updated weights for policy 0, policy_version 281863 (0.0008) [2023-12-26 17:24:55,492][105620] Updated weights for policy 1, policy_version 281930 (0.0010) [2023-12-26 17:24:55,549][105620] Updated weights for policy 1, policy_version 281940 (0.0010) [2023-12-26 17:24:55,601][105620] Updated weights for policy 1, policy_version 281950 (0.0010) [2023-12-26 17:24:55,655][105620] Updated weights for policy 1, policy_version 281960 (0.0010) [2023-12-26 17:24:55,668][105692] Updated weights for policy 0, policy_version 281873 (0.0010) [2023-12-26 17:24:55,723][105692] Updated weights for policy 0, policy_version 281883 (0.0010) [2023-12-26 17:24:55,770][105692] Updated weights for policy 0, policy_version 281893 (0.0008) [2023-12-26 17:24:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 144367616. Throughput: 0: 9811.7, 1: 9800.7. Samples: 144374436. Policy #0 lag: (min: 17.0, avg: 31.1, max: 32.0) [2023-12-26 17:24:56,063][104569] Avg episode reward: [(0, '9176.034'), (1, '9173.926')] [2023-12-26 17:24:56,381][105620] Updated weights for policy 1, policy_version 281970 (0.0010) [2023-12-26 17:24:56,434][105620] Updated weights for policy 1, policy_version 281980 (0.0010) [2023-12-26 17:24:56,484][105620] Updated weights for policy 1, policy_version 281990 (0.0010) [2023-12-26 17:24:56,538][105692] Updated weights for policy 0, policy_version 281903 (0.0006) [2023-12-26 17:24:56,598][105692] Updated weights for policy 0, policy_version 281913 (0.0005) [2023-12-26 17:24:56,659][105692] Updated weights for policy 0, policy_version 281923 (0.0005) [2023-12-26 17:24:57,150][105620] Updated weights for policy 1, policy_version 282000 (0.0010) [2023-12-26 17:24:57,194][105692] Updated weights for policy 0, policy_version 281933 (0.0006) [2023-12-26 17:24:57,209][105620] Updated weights for policy 1, policy_version 282010 (0.0006) [2023-12-26 17:24:57,240][105692] Updated weights for policy 0, policy_version 281943 (0.0005) [2023-12-26 17:24:57,263][105620] Updated weights for policy 1, policy_version 282020 (0.0005) [2023-12-26 17:24:57,292][105692] Updated weights for policy 0, policy_version 281953 (0.0005) [2023-12-26 17:24:57,833][105692] Updated weights for policy 0, policy_version 281963 (0.0006) [2023-12-26 17:24:57,888][105692] Updated weights for policy 0, policy_version 281973 (0.0005) [2023-12-26 17:24:57,937][105692] Updated weights for policy 0, policy_version 281983 (0.0005) [2023-12-26 17:24:57,956][105620] Updated weights for policy 1, policy_version 282030 (0.0006) [2023-12-26 17:24:58,003][105620] Updated weights for policy 1, policy_version 282040 (0.0005) [2023-12-26 17:24:58,058][105620] Updated weights for policy 1, policy_version 282050 (0.0010) [2023-12-26 17:24:58,603][105692] Updated weights for policy 0, policy_version 281993 (0.0007) [2023-12-26 17:24:58,668][105692] Updated weights for policy 0, policy_version 282003 (0.0008) [2023-12-26 17:24:58,728][105692] Updated weights for policy 0, policy_version 282013 (0.0008) [2023-12-26 17:24:58,796][105692] Updated weights for policy 0, policy_version 282023 (0.0007) [2023-12-26 17:24:58,893][105620] Updated weights for policy 1, policy_version 282060 (0.0011) [2023-12-26 17:24:58,968][105620] Updated weights for policy 1, policy_version 282070 (0.0010) [2023-12-26 17:24:59,024][105620] Updated weights for policy 1, policy_version 282080 (0.0010) [2023-12-26 17:24:59,607][105692] Updated weights for policy 0, policy_version 282033 (0.0007) [2023-12-26 17:24:59,669][105692] Updated weights for policy 0, policy_version 282043 (0.0008) [2023-12-26 17:24:59,725][105692] Updated weights for policy 0, policy_version 282053 (0.0008) [2023-12-26 17:24:59,800][105620] Updated weights for policy 1, policy_version 282090 (0.0009) [2023-12-26 17:24:59,859][105620] Updated weights for policy 1, policy_version 282100 (0.0007) [2023-12-26 17:24:59,922][105620] Updated weights for policy 1, policy_version 282110 (0.0007) [2023-12-26 17:24:59,977][105620] Updated weights for policy 1, policy_version 282120 (0.0008) [2023-12-26 17:25:00,463][105692] Updated weights for policy 0, policy_version 282063 (0.0010) [2023-12-26 17:25:00,524][105692] Updated weights for policy 0, policy_version 282073 (0.0010) [2023-12-26 17:25:00,588][105692] Updated weights for policy 0, policy_version 282083 (0.0010) [2023-12-26 17:25:00,683][105620] Updated weights for policy 1, policy_version 282130 (0.0010) [2023-12-26 17:25:00,740][105620] Updated weights for policy 1, policy_version 282140 (0.0010) [2023-12-26 17:25:00,802][105620] Updated weights for policy 1, policy_version 282150 (0.0010) [2023-12-26 17:25:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 144465920. Throughput: 0: 9898.0, 1: 9794.5. Samples: 144436276. Policy #0 lag: (min: 17.0, avg: 31.1, max: 32.0) [2023-12-26 17:25:01,062][104569] Avg episode reward: [(0, '9265.036'), (1, '8803.389')] [2023-12-26 17:25:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000282088_72228864.pth... [2023-12-26 17:25:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000282152_72237056.pth... [2023-12-26 17:25:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000281000_71942144.pth [2023-12-26 17:25:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000280968_71942144.pth [2023-12-26 17:25:01,196][105692] Updated weights for policy 0, policy_version 282093 (0.0009) [2023-12-26 17:25:01,264][105692] Updated weights for policy 0, policy_version 282103 (0.0008) [2023-12-26 17:25:01,325][105692] Updated weights for policy 0, policy_version 282113 (0.0009) [2023-12-26 17:25:01,562][105620] Updated weights for policy 1, policy_version 282160 (0.0010) [2023-12-26 17:25:01,624][105620] Updated weights for policy 1, policy_version 282170 (0.0011) [2023-12-26 17:25:01,685][105620] Updated weights for policy 1, policy_version 282180 (0.0010) [2023-12-26 17:25:02,046][105692] Updated weights for policy 0, policy_version 282123 (0.0009) [2023-12-26 17:25:02,100][105692] Updated weights for policy 0, policy_version 282133 (0.0005) [2023-12-26 17:25:02,148][105692] Updated weights for policy 0, policy_version 282143 (0.0005) [2023-12-26 17:25:02,450][105620] Updated weights for policy 1, policy_version 282190 (0.0010) [2023-12-26 17:25:02,502][105620] Updated weights for policy 1, policy_version 282200 (0.0010) [2023-12-26 17:25:02,511][105586] KL-divergence is very high: 139.9747 [2023-12-26 17:25:02,549][105586] KL-divergence is very high: 155.4077 [2023-12-26 17:25:02,550][105620] Updated weights for policy 1, policy_version 282210 (0.0010) [2023-12-26 17:25:02,789][105692] Updated weights for policy 0, policy_version 282153 (0.0007) [2023-12-26 17:25:02,840][105692] Updated weights for policy 0, policy_version 282163 (0.0010) [2023-12-26 17:25:02,898][105692] Updated weights for policy 0, policy_version 282173 (0.0010) [2023-12-26 17:25:02,954][105692] Updated weights for policy 0, policy_version 282183 (0.0010) [2023-12-26 17:25:03,310][105620] Updated weights for policy 1, policy_version 282220 (0.0010) [2023-12-26 17:25:03,358][105620] Updated weights for policy 1, policy_version 282230 (0.0010) [2023-12-26 17:25:03,405][105620] Updated weights for policy 1, policy_version 282240 (0.0010) [2023-12-26 17:25:03,627][105692] Updated weights for policy 0, policy_version 282193 (0.0010) [2023-12-26 17:25:03,684][105692] Updated weights for policy 0, policy_version 282203 (0.0010) [2023-12-26 17:25:03,745][105692] Updated weights for policy 0, policy_version 282213 (0.0010) [2023-12-26 17:25:04,160][105620] Updated weights for policy 1, policy_version 282250 (0.0010) [2023-12-26 17:25:04,227][105620] Updated weights for policy 1, policy_version 282260 (0.0011) [2023-12-26 17:25:04,296][105620] Updated weights for policy 1, policy_version 282270 (0.0011) [2023-12-26 17:25:04,360][105620] Updated weights for policy 1, policy_version 282280 (0.0011) [2023-12-26 17:25:04,393][105692] Updated weights for policy 0, policy_version 282223 (0.0008) [2023-12-26 17:25:04,443][105692] Updated weights for policy 0, policy_version 282233 (0.0008) [2023-12-26 17:25:04,499][105692] Updated weights for policy 0, policy_version 282243 (0.0008) [2023-12-26 17:25:05,127][105620] Updated weights for policy 1, policy_version 282290 (0.0008) [2023-12-26 17:25:05,181][105620] Updated weights for policy 1, policy_version 282300 (0.0009) [2023-12-26 17:25:05,197][105692] Updated weights for policy 0, policy_version 282253 (0.0007) [2023-12-26 17:25:05,233][105620] Updated weights for policy 1, policy_version 282310 (0.0008) [2023-12-26 17:25:05,253][105692] Updated weights for policy 0, policy_version 282263 (0.0005) [2023-12-26 17:25:05,313][105692] Updated weights for policy 0, policy_version 282273 (0.0005) [2023-12-26 17:25:05,836][105692] Updated weights for policy 0, policy_version 282283 (0.0007) [2023-12-26 17:25:05,884][105692] Updated weights for policy 0, policy_version 282293 (0.0005) [2023-12-26 17:25:05,927][105692] Updated weights for policy 0, policy_version 282303 (0.0005) [2023-12-26 17:25:06,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 144564224. Throughput: 0: 9859.9, 1: 9717.5. Samples: 144551800. Policy #0 lag: (min: 17.0, avg: 31.1, max: 32.0) [2023-12-26 17:25:06,063][104569] Avg episode reward: [(0, '9082.589'), (1, '8800.367')] [2023-12-26 17:25:06,133][105620] Updated weights for policy 1, policy_version 282320 (0.0009) [2023-12-26 17:25:06,196][105620] Updated weights for policy 1, policy_version 282330 (0.0009) [2023-12-26 17:25:06,254][105620] Updated weights for policy 1, policy_version 282340 (0.0008) [2023-12-26 17:25:06,568][105692] Updated weights for policy 0, policy_version 282313 (0.0006) [2023-12-26 17:25:06,630][105692] Updated weights for policy 0, policy_version 282323 (0.0009) [2023-12-26 17:25:06,689][105692] Updated weights for policy 0, policy_version 282333 (0.0009) [2023-12-26 17:25:06,748][105692] Updated weights for policy 0, policy_version 282343 (0.0009) [2023-12-26 17:25:07,046][105620] Updated weights for policy 1, policy_version 282350 (0.0010) [2023-12-26 17:25:07,105][105620] Updated weights for policy 1, policy_version 282360 (0.0007) [2023-12-26 17:25:07,152][105620] Updated weights for policy 1, policy_version 282370 (0.0005) [2023-12-26 17:25:07,592][105692] Updated weights for policy 0, policy_version 282353 (0.0010) [2023-12-26 17:25:07,645][105692] Updated weights for policy 0, policy_version 282363 (0.0009) [2023-12-26 17:25:07,698][105692] Updated weights for policy 0, policy_version 282373 (0.0009) [2023-12-26 17:25:07,717][105620] Updated weights for policy 1, policy_version 282380 (0.0006) [2023-12-26 17:25:07,784][105620] Updated weights for policy 1, policy_version 282390 (0.0009) [2023-12-26 17:25:07,836][105620] Updated weights for policy 1, policy_version 282400 (0.0007) [2023-12-26 17:25:08,479][105620] Updated weights for policy 1, policy_version 282410 (0.0006) [2023-12-26 17:25:08,519][105692] Updated weights for policy 0, policy_version 282383 (0.0006) [2023-12-26 17:25:08,538][105620] Updated weights for policy 1, policy_version 282420 (0.0009) [2023-12-26 17:25:08,570][105692] Updated weights for policy 0, policy_version 282393 (0.0008) [2023-12-26 17:25:08,596][105620] Updated weights for policy 1, policy_version 282430 (0.0006) [2023-12-26 17:25:08,623][105692] Updated weights for policy 0, policy_version 282403 (0.0006) [2023-12-26 17:25:08,663][105620] Updated weights for policy 1, policy_version 282440 (0.0008) [2023-12-26 17:25:09,288][105620] Updated weights for policy 1, policy_version 282450 (0.0007) [2023-12-26 17:25:09,355][105620] Updated weights for policy 1, policy_version 282460 (0.0008) [2023-12-26 17:25:09,389][105692] Updated weights for policy 0, policy_version 282413 (0.0008) [2023-12-26 17:25:09,409][105585] KL-divergence is very high: 203.6272 [2023-12-26 17:25:09,426][105620] Updated weights for policy 1, policy_version 282470 (0.0009) [2023-12-26 17:25:09,456][105692] Updated weights for policy 0, policy_version 282423 (0.0007) [2023-12-26 17:25:09,463][105585] KL-divergence is very high: 309.8791 [2023-12-26 17:25:09,509][105585] KL-divergence is very high: 237.7417 [2023-12-26 17:25:09,515][105692] Updated weights for policy 0, policy_version 282433 (0.0009) [2023-12-26 17:25:10,217][105692] Updated weights for policy 0, policy_version 282443 (0.0010) [2023-12-26 17:25:10,232][105620] Updated weights for policy 1, policy_version 282480 (0.0007) [2023-12-26 17:25:10,270][105692] Updated weights for policy 0, policy_version 282453 (0.0011) [2023-12-26 17:25:10,293][105620] Updated weights for policy 1, policy_version 282490 (0.0006) [2023-12-26 17:25:10,326][105692] Updated weights for policy 0, policy_version 282463 (0.0011) [2023-12-26 17:25:10,361][105620] Updated weights for policy 1, policy_version 282500 (0.0006) [2023-12-26 17:25:11,042][105692] Updated weights for policy 0, policy_version 282473 (0.0011) [2023-12-26 17:25:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 144654336. Throughput: 0: 9865.6, 1: 9696.1. Samples: 144668844. Policy #0 lag: (min: 17.0, avg: 31.1, max: 32.0) [2023-12-26 17:25:11,063][104569] Avg episode reward: [(0, '8490.020'), (1, '8714.445')] [2023-12-26 17:25:11,102][105692] Updated weights for policy 0, policy_version 282483 (0.0008) [2023-12-26 17:25:11,107][105620] Updated weights for policy 1, policy_version 282510 (0.0008) [2023-12-26 17:25:11,170][105692] Updated weights for policy 0, policy_version 282493 (0.0008) [2023-12-26 17:25:11,174][105620] Updated weights for policy 1, policy_version 282520 (0.0007) [2023-12-26 17:25:11,225][105692] Updated weights for policy 0, policy_version 282503 (0.0009) [2023-12-26 17:25:11,234][105620] Updated weights for policy 1, policy_version 282530 (0.0008) [2023-12-26 17:25:11,947][105692] Updated weights for policy 0, policy_version 282513 (0.0008) [2023-12-26 17:25:12,000][105620] Updated weights for policy 1, policy_version 282540 (0.0008) [2023-12-26 17:25:12,007][105692] Updated weights for policy 0, policy_version 282523 (0.0008) [2023-12-26 17:25:12,059][105620] Updated weights for policy 1, policy_version 282550 (0.0007) [2023-12-26 17:25:12,065][105692] Updated weights for policy 0, policy_version 282533 (0.0008) [2023-12-26 17:25:12,124][105620] Updated weights for policy 1, policy_version 282560 (0.0008) [2023-12-26 17:25:12,850][105692] Updated weights for policy 0, policy_version 282543 (0.0009) [2023-12-26 17:25:12,903][105692] Updated weights for policy 0, policy_version 282553 (0.0010) [2023-12-26 17:25:12,907][105620] Updated weights for policy 1, policy_version 282570 (0.0008) [2023-12-26 17:25:12,955][105692] Updated weights for policy 0, policy_version 282563 (0.0009) [2023-12-26 17:25:12,969][105620] Updated weights for policy 1, policy_version 282580 (0.0006) [2023-12-26 17:25:13,015][105620] Updated weights for policy 1, policy_version 282590 (0.0009) [2023-12-26 17:25:13,063][105620] Updated weights for policy 1, policy_version 282600 (0.0009) [2023-12-26 17:25:13,726][105620] Updated weights for policy 1, policy_version 282610 (0.0009) [2023-12-26 17:25:13,772][105586] KL-divergence is very high: 124.0105 [2023-12-26 17:25:13,783][105620] Updated weights for policy 1, policy_version 282620 (0.0009) [2023-12-26 17:25:13,794][105692] Updated weights for policy 0, policy_version 282573 (0.0006) [2023-12-26 17:25:13,820][105586] KL-divergence is very high: 114.6983 [2023-12-26 17:25:13,841][105620] Updated weights for policy 1, policy_version 282630 (0.0007) [2023-12-26 17:25:13,847][105692] Updated weights for policy 0, policy_version 282583 (0.0006) [2023-12-26 17:25:13,893][105692] Updated weights for policy 0, policy_version 282593 (0.0008) [2023-12-26 17:25:14,576][105620] Updated weights for policy 1, policy_version 282640 (0.0006) [2023-12-26 17:25:14,628][105620] Updated weights for policy 1, policy_version 282650 (0.0005) [2023-12-26 17:25:14,687][105620] Updated weights for policy 1, policy_version 282660 (0.0008) [2023-12-26 17:25:14,721][105692] Updated weights for policy 0, policy_version 282603 (0.0008) [2023-12-26 17:25:14,790][105692] Updated weights for policy 0, policy_version 282613 (0.0008) [2023-12-26 17:25:14,853][105692] Updated weights for policy 0, policy_version 282623 (0.0006) [2023-12-26 17:25:15,346][105620] Updated weights for policy 1, policy_version 282670 (0.0011) [2023-12-26 17:25:15,408][105620] Updated weights for policy 1, policy_version 282680 (0.0010) [2023-12-26 17:25:15,463][105620] Updated weights for policy 1, policy_version 282690 (0.0010) [2023-12-26 17:25:15,515][105692] Updated weights for policy 0, policy_version 282633 (0.0006) [2023-12-26 17:25:15,574][105692] Updated weights for policy 0, policy_version 282643 (0.0008) [2023-12-26 17:25:15,626][105692] Updated weights for policy 0, policy_version 282653 (0.0008) [2023-12-26 17:25:15,676][105692] Updated weights for policy 0, policy_version 282663 (0.0008) [2023-12-26 17:25:16,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 144752640. Throughput: 0: 9813.2, 1: 9652.0. Samples: 144724748. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 17:25:16,062][104569] Avg episode reward: [(0, '8666.722'), (1, '8449.108')] [2023-12-26 17:25:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000282664_72376320.pth... [2023-12-26 17:25:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000282696_72376320.pth... [2023-12-26 17:25:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000281544_72089600.pth [2023-12-26 17:25:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000281576_72089600.pth [2023-12-26 17:25:16,195][105620] Updated weights for policy 1, policy_version 282700 (0.0011) [2023-12-26 17:25:16,260][105620] Updated weights for policy 1, policy_version 282710 (0.0011) [2023-12-26 17:25:16,314][105620] Updated weights for policy 1, policy_version 282720 (0.0010) [2023-12-26 17:25:16,393][105692] Updated weights for policy 0, policy_version 282673 (0.0009) [2023-12-26 17:25:16,451][105692] Updated weights for policy 0, policy_version 282683 (0.0008) [2023-12-26 17:25:16,503][105692] Updated weights for policy 0, policy_version 282693 (0.0008) [2023-12-26 17:25:17,086][105620] Updated weights for policy 1, policy_version 282730 (0.0010) [2023-12-26 17:25:17,145][105620] Updated weights for policy 1, policy_version 282740 (0.0010) [2023-12-26 17:25:17,203][105620] Updated weights for policy 1, policy_version 282750 (0.0010) [2023-12-26 17:25:17,260][105692] Updated weights for policy 0, policy_version 282703 (0.0007) [2023-12-26 17:25:17,265][105620] Updated weights for policy 1, policy_version 282760 (0.0010) [2023-12-26 17:25:17,318][105692] Updated weights for policy 0, policy_version 282713 (0.0008) [2023-12-26 17:25:17,383][105692] Updated weights for policy 0, policy_version 282723 (0.0008) [2023-12-26 17:25:18,006][105620] Updated weights for policy 1, policy_version 282770 (0.0009) [2023-12-26 17:25:18,028][105692] Updated weights for policy 0, policy_version 282733 (0.0007) [2023-12-26 17:25:18,071][105620] Updated weights for policy 1, policy_version 282780 (0.0008) [2023-12-26 17:25:18,082][105692] Updated weights for policy 0, policy_version 282743 (0.0009) [2023-12-26 17:25:18,133][105620] Updated weights for policy 1, policy_version 282790 (0.0009) [2023-12-26 17:25:18,135][105692] Updated weights for policy 0, policy_version 282753 (0.0005) [2023-12-26 17:25:18,853][105692] Updated weights for policy 0, policy_version 282763 (0.0006) [2023-12-26 17:25:18,872][105620] Updated weights for policy 1, policy_version 282800 (0.0010) [2023-12-26 17:25:18,909][105692] Updated weights for policy 0, policy_version 282773 (0.0007) [2023-12-26 17:25:18,928][105620] Updated weights for policy 1, policy_version 282810 (0.0010) [2023-12-26 17:25:18,966][105692] Updated weights for policy 0, policy_version 282783 (0.0006) [2023-12-26 17:25:18,984][105620] Updated weights for policy 1, policy_version 282820 (0.0010) [2023-12-26 17:25:19,675][105620] Updated weights for policy 1, policy_version 282830 (0.0011) [2023-12-26 17:25:19,739][105620] Updated weights for policy 1, policy_version 282840 (0.0006) [2023-12-26 17:25:19,760][105692] Updated weights for policy 0, policy_version 282793 (0.0006) [2023-12-26 17:25:19,803][105620] Updated weights for policy 1, policy_version 282850 (0.0010) [2023-12-26 17:25:19,820][105692] Updated weights for policy 0, policy_version 282803 (0.0008) [2023-12-26 17:25:19,884][105692] Updated weights for policy 0, policy_version 282813 (0.0008) [2023-12-26 17:25:19,942][105692] Updated weights for policy 0, policy_version 282823 (0.0008) [2023-12-26 17:25:20,530][105620] Updated weights for policy 1, policy_version 282860 (0.0011) [2023-12-26 17:25:20,588][105620] Updated weights for policy 1, policy_version 282870 (0.0009) [2023-12-26 17:25:20,640][105620] Updated weights for policy 1, policy_version 282880 (0.0006) [2023-12-26 17:25:20,724][105692] Updated weights for policy 0, policy_version 282833 (0.0010) [2023-12-26 17:25:20,788][105692] Updated weights for policy 0, policy_version 282843 (0.0010) [2023-12-26 17:25:20,848][105692] Updated weights for policy 0, policy_version 282853 (0.0009) [2023-12-26 17:25:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 144850944. Throughput: 0: 9698.5, 1: 9681.1. Samples: 144840264. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 17:25:21,062][104569] Avg episode reward: [(0, '9086.956'), (1, '8621.963')] [2023-12-26 17:25:21,366][105620] Updated weights for policy 1, policy_version 282890 (0.0008) [2023-12-26 17:25:21,429][105620] Updated weights for policy 1, policy_version 282900 (0.0008) [2023-12-26 17:25:21,491][105620] Updated weights for policy 1, policy_version 282910 (0.0008) [2023-12-26 17:25:21,553][105620] Updated weights for policy 1, policy_version 282920 (0.0010) [2023-12-26 17:25:21,628][105692] Updated weights for policy 0, policy_version 282863 (0.0009) [2023-12-26 17:25:21,689][105692] Updated weights for policy 0, policy_version 282873 (0.0009) [2023-12-26 17:25:21,760][105692] Updated weights for policy 0, policy_version 282883 (0.0008) [2023-12-26 17:25:22,215][105620] Updated weights for policy 1, policy_version 282930 (0.0005) [2023-12-26 17:25:22,278][105620] Updated weights for policy 1, policy_version 282940 (0.0007) [2023-12-26 17:25:22,339][105620] Updated weights for policy 1, policy_version 282950 (0.0006) [2023-12-26 17:25:22,430][105692] Updated weights for policy 0, policy_version 282893 (0.0007) [2023-12-26 17:25:22,498][105692] Updated weights for policy 0, policy_version 282903 (0.0009) [2023-12-26 17:25:22,554][105692] Updated weights for policy 0, policy_version 282913 (0.0010) [2023-12-26 17:25:22,903][105620] Updated weights for policy 1, policy_version 282960 (0.0007) [2023-12-26 17:25:22,967][105620] Updated weights for policy 1, policy_version 282970 (0.0010) [2023-12-26 17:25:23,029][105620] Updated weights for policy 1, policy_version 282980 (0.0010) [2023-12-26 17:25:23,399][105692] Updated weights for policy 0, policy_version 282924 (0.0010) [2023-12-26 17:25:23,453][105692] Updated weights for policy 0, policy_version 282934 (0.0010) [2023-12-26 17:25:23,513][105692] Updated weights for policy 0, policy_version 282944 (0.0010) [2023-12-26 17:25:23,580][105620] Updated weights for policy 1, policy_version 282990 (0.0007) [2023-12-26 17:25:23,630][105620] Updated weights for policy 1, policy_version 283000 (0.0009) [2023-12-26 17:25:23,678][105620] Updated weights for policy 1, policy_version 283010 (0.0010) [2023-12-26 17:25:24,310][105620] Updated weights for policy 1, policy_version 283020 (0.0008) [2023-12-26 17:25:24,372][105620] Updated weights for policy 1, policy_version 283030 (0.0007) [2023-12-26 17:25:24,379][105692] Updated weights for policy 0, policy_version 282954 (0.0010) [2023-12-26 17:25:24,433][105692] Updated weights for policy 0, policy_version 282964 (0.0009) [2023-12-26 17:25:24,435][105620] Updated weights for policy 1, policy_version 283040 (0.0010) [2023-12-26 17:25:24,489][105692] Updated weights for policy 0, policy_version 282974 (0.0006) [2023-12-26 17:25:24,538][105692] Updated weights for policy 0, policy_version 282984 (0.0008) [2023-12-26 17:25:25,037][105620] Updated weights for policy 1, policy_version 283050 (0.0009) [2023-12-26 17:25:25,090][105620] Updated weights for policy 1, policy_version 283060 (0.0008) [2023-12-26 17:25:25,139][105620] Updated weights for policy 1, policy_version 283070 (0.0006) [2023-12-26 17:25:25,191][105620] Updated weights for policy 1, policy_version 283080 (0.0005) [2023-12-26 17:25:25,398][105692] Updated weights for policy 0, policy_version 282995 (0.0010) [2023-12-26 17:25:25,453][105692] Updated weights for policy 0, policy_version 283005 (0.0010) [2023-12-26 17:25:25,520][105692] Updated weights for policy 0, policy_version 283015 (0.0010) [2023-12-26 17:25:25,734][105620] Updated weights for policy 1, policy_version 283090 (0.0005) [2023-12-26 17:25:25,795][105620] Updated weights for policy 1, policy_version 283100 (0.0006) [2023-12-26 17:25:25,844][105620] Updated weights for policy 1, policy_version 283110 (0.0005) [2023-12-26 17:25:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 144949248. Throughput: 0: 9577.3, 1: 9886.4. Samples: 144958060. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 17:25:26,062][104569] Avg episode reward: [(0, '9175.601'), (1, '8986.827')] [2023-12-26 17:25:26,372][105620] Updated weights for policy 1, policy_version 283120 (0.0005) [2023-12-26 17:25:26,425][105620] Updated weights for policy 1, policy_version 283130 (0.0005) [2023-12-26 17:25:26,469][105692] Updated weights for policy 0, policy_version 283025 (0.0008) [2023-12-26 17:25:26,479][105620] Updated weights for policy 1, policy_version 283140 (0.0007) [2023-12-26 17:25:26,534][105692] Updated weights for policy 0, policy_version 283035 (0.0008) [2023-12-26 17:25:26,593][105692] Updated weights for policy 0, policy_version 283045 (0.0010) [2023-12-26 17:25:27,011][105620] Updated weights for policy 1, policy_version 283150 (0.0010) [2023-12-26 17:25:27,068][105620] Updated weights for policy 1, policy_version 283160 (0.0009) [2023-12-26 17:25:27,123][105620] Updated weights for policy 1, policy_version 283170 (0.0005) [2023-12-26 17:25:27,426][105692] Updated weights for policy 0, policy_version 283055 (0.0006) [2023-12-26 17:25:27,486][105692] Updated weights for policy 0, policy_version 283065 (0.0007) [2023-12-26 17:25:27,540][105692] Updated weights for policy 0, policy_version 283075 (0.0010) [2023-12-26 17:25:27,658][105620] Updated weights for policy 1, policy_version 283180 (0.0007) [2023-12-26 17:25:27,707][105620] Updated weights for policy 1, policy_version 283190 (0.0006) [2023-12-26 17:25:27,764][105620] Updated weights for policy 1, policy_version 283200 (0.0009) [2023-12-26 17:25:28,334][105692] Updated weights for policy 0, policy_version 283085 (0.0009) [2023-12-26 17:25:28,391][105692] Updated weights for policy 0, policy_version 283095 (0.0009) [2023-12-26 17:25:28,420][105620] Updated weights for policy 1, policy_version 283210 (0.0009) [2023-12-26 17:25:28,443][105692] Updated weights for policy 0, policy_version 283105 (0.0008) [2023-12-26 17:25:28,473][105620] Updated weights for policy 1, policy_version 283220 (0.0006) [2023-12-26 17:25:28,518][105620] Updated weights for policy 1, policy_version 283230 (0.0005) [2023-12-26 17:25:28,565][105620] Updated weights for policy 1, policy_version 283240 (0.0005) [2023-12-26 17:25:29,258][105620] Updated weights for policy 1, policy_version 283250 (0.0010) [2023-12-26 17:25:29,301][105692] Updated weights for policy 0, policy_version 283115 (0.0009) [2023-12-26 17:25:29,311][105620] Updated weights for policy 1, policy_version 283261 (0.0008) [2023-12-26 17:25:29,365][105692] Updated weights for policy 0, policy_version 283125 (0.0007) [2023-12-26 17:25:29,378][105620] Updated weights for policy 1, policy_version 283271 (0.0008) [2023-12-26 17:25:29,427][105692] Updated weights for policy 0, policy_version 283135 (0.0009) [2023-12-26 17:25:30,126][105620] Updated weights for policy 1, policy_version 283281 (0.0009) [2023-12-26 17:25:30,186][105620] Updated weights for policy 1, policy_version 283291 (0.0009) [2023-12-26 17:25:30,205][105692] Updated weights for policy 0, policy_version 283145 (0.0009) [2023-12-26 17:25:30,246][105620] Updated weights for policy 1, policy_version 283301 (0.0009) [2023-12-26 17:25:30,266][105692] Updated weights for policy 0, policy_version 283155 (0.0006) [2023-12-26 17:25:30,326][105692] Updated weights for policy 0, policy_version 283165 (0.0006) [2023-12-26 17:25:30,382][105692] Updated weights for policy 0, policy_version 283175 (0.0009) [2023-12-26 17:25:30,994][105620] Updated weights for policy 1, policy_version 283311 (0.0009) [2023-12-26 17:25:31,051][105620] Updated weights for policy 1, policy_version 283321 (0.0008) [2023-12-26 17:25:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 145039360. Throughput: 0: 9501.6, 1: 10071.7. Samples: 145018588. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 17:25:31,062][104569] Avg episode reward: [(0, '8990.751'), (1, '8711.090')] [2023-12-26 17:25:31,077][105586] KL-divergence is very high: 126.1797 [2023-12-26 17:25:31,110][105692] Updated weights for policy 0, policy_version 283185 (0.0007) [2023-12-26 17:25:31,115][105620] Updated weights for policy 1, policy_version 283331 (0.0008) [2023-12-26 17:25:31,128][105586] KL-divergence is very high: 145.0908 [2023-12-26 17:25:31,145][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000283336_72540160.pth... [2023-12-26 17:25:31,148][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000282152_72237056.pth [2023-12-26 17:25:31,173][105692] Updated weights for policy 0, policy_version 283195 (0.0008) [2023-12-26 17:25:31,229][105692] Updated weights for policy 0, policy_version 283205 (0.0009) [2023-12-26 17:25:31,249][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000283208_72515584.pth... [2023-12-26 17:25:31,254][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000282088_72228864.pth [2023-12-26 17:25:31,774][105620] Updated weights for policy 1, policy_version 283341 (0.0009) [2023-12-26 17:25:31,834][105620] Updated weights for policy 1, policy_version 283351 (0.0007) [2023-12-26 17:25:31,889][105620] Updated weights for policy 1, policy_version 283361 (0.0009) [2023-12-26 17:25:32,067][105692] Updated weights for policy 0, policy_version 283215 (0.0009) [2023-12-26 17:25:32,121][105692] Updated weights for policy 0, policy_version 283226 (0.0009) [2023-12-26 17:25:32,176][105692] Updated weights for policy 0, policy_version 283238 (0.0011) [2023-12-26 17:25:32,552][105620] Updated weights for policy 1, policy_version 283371 (0.0008) [2023-12-26 17:25:32,599][105620] Updated weights for policy 1, policy_version 283381 (0.0008) [2023-12-26 17:25:32,646][105620] Updated weights for policy 1, policy_version 283391 (0.0009) [2023-12-26 17:25:32,965][105692] Updated weights for policy 0, policy_version 283248 (0.0009) [2023-12-26 17:25:33,023][105692] Updated weights for policy 0, policy_version 283258 (0.0009) [2023-12-26 17:25:33,087][105692] Updated weights for policy 0, policy_version 283268 (0.0009) [2023-12-26 17:25:33,385][105620] Updated weights for policy 1, policy_version 283401 (0.0008) [2023-12-26 17:25:33,445][105620] Updated weights for policy 1, policy_version 283411 (0.0005) [2023-12-26 17:25:33,498][105620] Updated weights for policy 1, policy_version 283421 (0.0008) [2023-12-26 17:25:33,558][105620] Updated weights for policy 1, policy_version 283431 (0.0009) [2023-12-26 17:25:33,862][105692] Updated weights for policy 0, policy_version 283278 (0.0009) [2023-12-26 17:25:33,917][105692] Updated weights for policy 0, policy_version 283288 (0.0009) [2023-12-26 17:25:33,964][105692] Updated weights for policy 0, policy_version 283298 (0.0009) [2023-12-26 17:25:34,249][105620] Updated weights for policy 1, policy_version 283441 (0.0006) [2023-12-26 17:25:34,308][105620] Updated weights for policy 1, policy_version 283451 (0.0005) [2023-12-26 17:25:34,350][105586] KL-divergence is very high: 210.6197 [2023-12-26 17:25:34,369][105620] Updated weights for policy 1, policy_version 283461 (0.0007) [2023-12-26 17:25:34,379][105586] KL-divergence is very high: 704.4705 [2023-12-26 17:25:34,723][105692] Updated weights for policy 0, policy_version 283308 (0.0009) [2023-12-26 17:25:34,780][105692] Updated weights for policy 0, policy_version 283318 (0.0008) [2023-12-26 17:25:34,836][105692] Updated weights for policy 0, policy_version 283328 (0.0008) [2023-12-26 17:25:34,999][105586] KL-divergence is very high: 919.0147 [2023-12-26 17:25:35,036][105620] Updated weights for policy 1, policy_version 283471 (0.0009) [2023-12-26 17:25:35,045][105586] KL-divergence is very high: 1214.6472 [2023-12-26 17:25:35,093][105586] KL-divergence is very high: 1219.7295 [2023-12-26 17:25:35,093][105620] Updated weights for policy 1, policy_version 283481 (0.0009) [2023-12-26 17:25:35,142][105586] KL-divergence is very high: 1115.0902 [2023-12-26 17:25:35,154][105620] Updated weights for policy 1, policy_version 283491 (0.0008) [2023-12-26 17:25:35,589][105692] Updated weights for policy 0, policy_version 283338 (0.0008) [2023-12-26 17:25:35,654][105692] Updated weights for policy 0, policy_version 283348 (0.0009) [2023-12-26 17:25:35,666][105585] KL-divergence is very high: 121.7510 [2023-12-26 17:25:35,704][105585] KL-divergence is very high: 183.9837 [2023-12-26 17:25:35,706][105692] Updated weights for policy 0, policy_version 283358 (0.0009) [2023-12-26 17:25:35,741][105585] KL-divergence is very high: 132.4817 [2023-12-26 17:25:35,753][105692] Updated weights for policy 0, policy_version 283368 (0.0009) [2023-12-26 17:25:35,904][105620] Updated weights for policy 1, policy_version 283501 (0.0009) [2023-12-26 17:25:35,954][105620] Updated weights for policy 1, policy_version 283511 (0.0009) [2023-12-26 17:25:36,016][105620] Updated weights for policy 1, policy_version 283521 (0.0009) [2023-12-26 17:25:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 145145856. Throughput: 0: 9434.4, 1: 9957.1. Samples: 145132164. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 17:25:36,062][104569] Avg episode reward: [(0, '8717.598'), (1, '8068.971')] [2023-12-26 17:25:36,512][105692] Updated weights for policy 0, policy_version 283378 (0.0005) [2023-12-26 17:25:36,534][105585] KL-divergence is very high: 119.4404 [2023-12-26 17:25:36,564][105585] KL-divergence is very high: 102.3666 [2023-12-26 17:25:36,576][105692] Updated weights for policy 0, policy_version 283388 (0.0005) [2023-12-26 17:25:36,584][105585] KL-divergence is very high: 125.7517 [2023-12-26 17:25:36,641][105692] Updated weights for policy 0, policy_version 283398 (0.0005) [2023-12-26 17:25:36,771][105620] Updated weights for policy 1, policy_version 283531 (0.0009) [2023-12-26 17:25:36,823][105620] Updated weights for policy 1, policy_version 283541 (0.0009) [2023-12-26 17:25:36,873][105620] Updated weights for policy 1, policy_version 283551 (0.0009) [2023-12-26 17:25:37,241][105585] KL-divergence is very high: 189.1774 [2023-12-26 17:25:37,282][105692] Updated weights for policy 0, policy_version 283408 (0.0010) [2023-12-26 17:25:37,287][105585] KL-divergence is very high: 355.6821 [2023-12-26 17:25:37,335][105585] KL-divergence is very high: 264.1959 [2023-12-26 17:25:37,340][105692] Updated weights for policy 0, policy_version 283418 (0.0009) [2023-12-26 17:25:37,380][105585] KL-divergence is very high: 221.8667 [2023-12-26 17:25:37,398][105692] Updated weights for policy 0, policy_version 283428 (0.0010) [2023-12-26 17:25:37,555][105620] Updated weights for policy 1, policy_version 283561 (0.0009) [2023-12-26 17:25:37,620][105620] Updated weights for policy 1, policy_version 283571 (0.0009) [2023-12-26 17:25:37,680][105620] Updated weights for policy 1, policy_version 283581 (0.0009) [2023-12-26 17:25:37,740][105620] Updated weights for policy 1, policy_version 283591 (0.0008) [2023-12-26 17:25:38,194][105692] Updated weights for policy 0, policy_version 283438 (0.0010) [2023-12-26 17:25:38,252][105692] Updated weights for policy 0, policy_version 283448 (0.0010) [2023-12-26 17:25:38,316][105692] Updated weights for policy 0, policy_version 283458 (0.0006) [2023-12-26 17:25:38,506][105620] Updated weights for policy 1, policy_version 283601 (0.0006) [2023-12-26 17:25:38,568][105620] Updated weights for policy 1, policy_version 283611 (0.0010) [2023-12-26 17:25:38,626][105620] Updated weights for policy 1, policy_version 283621 (0.0008) [2023-12-26 17:25:38,943][105692] Updated weights for policy 0, policy_version 283468 (0.0008) [2023-12-26 17:25:39,006][105692] Updated weights for policy 0, policy_version 283478 (0.0011) [2023-12-26 17:25:39,068][105692] Updated weights for policy 0, policy_version 283488 (0.0010) [2023-12-26 17:25:39,342][105620] Updated weights for policy 1, policy_version 283631 (0.0008) [2023-12-26 17:25:39,412][105620] Updated weights for policy 1, policy_version 283641 (0.0009) [2023-12-26 17:25:39,482][105620] Updated weights for policy 1, policy_version 283651 (0.0008) [2023-12-26 17:25:39,821][105692] Updated weights for policy 0, policy_version 283498 (0.0010) [2023-12-26 17:25:39,886][105692] Updated weights for policy 0, policy_version 283508 (0.0010) [2023-12-26 17:25:39,960][105692] Updated weights for policy 0, policy_version 283518 (0.0007) [2023-12-26 17:25:40,028][105692] Updated weights for policy 0, policy_version 283528 (0.0010) [2023-12-26 17:25:40,239][105620] Updated weights for policy 1, policy_version 283661 (0.0008) [2023-12-26 17:25:40,304][105620] Updated weights for policy 1, policy_version 283671 (0.0008) [2023-12-26 17:25:40,366][105620] Updated weights for policy 1, policy_version 283681 (0.0008) [2023-12-26 17:25:40,748][105692] Updated weights for policy 0, policy_version 283538 (0.0010) [2023-12-26 17:25:40,809][105692] Updated weights for policy 0, policy_version 283548 (0.0010) [2023-12-26 17:25:40,824][105585] KL-divergence is very high: 246.1158 [2023-12-26 17:25:40,877][105585] KL-divergence is very high: 485.9901 [2023-12-26 17:25:40,878][105692] Updated weights for policy 0, policy_version 283558 (0.0010) [2023-12-26 17:25:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 145235968. Throughput: 0: 9450.7, 1: 9908.7. Samples: 145245608. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 17:25:41,063][104569] Avg episode reward: [(0, '8084.428'), (1, '8344.383')] [2023-12-26 17:25:41,160][105620] Updated weights for policy 1, policy_version 283691 (0.0008) [2023-12-26 17:25:41,220][105620] Updated weights for policy 1, policy_version 283701 (0.0007) [2023-12-26 17:25:41,283][105620] Updated weights for policy 1, policy_version 283711 (0.0008) [2023-12-26 17:25:41,613][105692] Updated weights for policy 0, policy_version 283568 (0.0011) [2023-12-26 17:25:41,678][105692] Updated weights for policy 0, policy_version 283578 (0.0010) [2023-12-26 17:25:41,741][105692] Updated weights for policy 0, policy_version 283588 (0.0010) [2023-12-26 17:25:42,046][105620] Updated weights for policy 1, policy_version 283721 (0.0008) [2023-12-26 17:25:42,100][105620] Updated weights for policy 1, policy_version 283731 (0.0008) [2023-12-26 17:25:42,156][105620] Updated weights for policy 1, policy_version 283741 (0.0009) [2023-12-26 17:25:42,214][105620] Updated weights for policy 1, policy_version 283751 (0.0010) [2023-12-26 17:25:42,468][105692] Updated weights for policy 0, policy_version 283598 (0.0010) [2023-12-26 17:25:42,530][105692] Updated weights for policy 0, policy_version 283608 (0.0010) [2023-12-26 17:25:42,592][105692] Updated weights for policy 0, policy_version 283618 (0.0011) [2023-12-26 17:25:43,011][105620] Updated weights for policy 1, policy_version 283761 (0.0009) [2023-12-26 17:25:43,066][105620] Updated weights for policy 1, policy_version 283771 (0.0009) [2023-12-26 17:25:43,125][105620] Updated weights for policy 1, policy_version 283781 (0.0009) [2023-12-26 17:25:43,265][105692] Updated weights for policy 0, policy_version 283628 (0.0010) [2023-12-26 17:25:43,323][105692] Updated weights for policy 0, policy_version 283639 (0.0010) [2023-12-26 17:25:43,382][105692] Updated weights for policy 0, policy_version 283649 (0.0010) [2023-12-26 17:25:43,921][105620] Updated weights for policy 1, policy_version 283791 (0.0009) [2023-12-26 17:25:43,975][105620] Updated weights for policy 1, policy_version 283801 (0.0009) [2023-12-26 17:25:44,041][105620] Updated weights for policy 1, policy_version 283811 (0.0009) [2023-12-26 17:25:44,061][105692] Updated weights for policy 0, policy_version 283659 (0.0008) [2023-12-26 17:25:44,122][105692] Updated weights for policy 0, policy_version 283669 (0.0006) [2023-12-26 17:25:44,180][105692] Updated weights for policy 0, policy_version 283679 (0.0008) [2023-12-26 17:25:44,804][105620] Updated weights for policy 1, policy_version 283821 (0.0008) [2023-12-26 17:25:44,867][105620] Updated weights for policy 1, policy_version 283831 (0.0008) [2023-12-26 17:25:44,893][105692] Updated weights for policy 0, policy_version 283689 (0.0010) [2023-12-26 17:25:44,907][105586] KL-divergence is very high: 160.4196 [2023-12-26 17:25:44,932][105620] Updated weights for policy 1, policy_version 283841 (0.0006) [2023-12-26 17:25:44,937][105586] KL-divergence is very high: 379.5308 [2023-12-26 17:25:44,953][105692] Updated weights for policy 0, policy_version 283699 (0.0011) [2023-12-26 17:25:44,955][105586] KL-divergence is very high: 387.4694 [2023-12-26 17:25:45,014][105692] Updated weights for policy 0, policy_version 283709 (0.0010) [2023-12-26 17:25:45,075][105692] Updated weights for policy 0, policy_version 283719 (0.0006) [2023-12-26 17:25:45,655][105586] KL-divergence is very high: 366.6355 [2023-12-26 17:25:45,675][105620] Updated weights for policy 1, policy_version 283851 (0.0006) [2023-12-26 17:25:45,675][105586] KL-divergence is very high: 418.0497 [2023-12-26 17:25:45,678][105692] Updated weights for policy 0, policy_version 283729 (0.0005) [2023-12-26 17:25:45,705][105586] KL-divergence is very high: 328.3355 [2023-12-26 17:25:45,723][105586] KL-divergence is very high: 339.1170 [2023-12-26 17:25:45,731][105620] Updated weights for policy 1, policy_version 283861 (0.0008) [2023-12-26 17:25:45,740][105692] Updated weights for policy 0, policy_version 283739 (0.0008) [2023-12-26 17:25:45,746][105586] KL-divergence is very high: 253.9883 [2023-12-26 17:25:45,762][105586] KL-divergence is very high: 263.0749 [2023-12-26 17:25:45,781][105620] Updated weights for policy 1, policy_version 283871 (0.0006) [2023-12-26 17:25:45,786][105586] KL-divergence is very high: 187.1483 [2023-12-26 17:25:45,794][105692] Updated weights for policy 0, policy_version 283749 (0.0006) [2023-12-26 17:25:45,800][105586] KL-divergence is very high: 204.5693 [2023-12-26 17:25:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 145334272. Throughput: 0: 9362.6, 1: 9846.9. Samples: 145300704. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 17:25:46,062][104569] Avg episode reward: [(0, '8451.366'), (1, '8340.010')] [2023-12-26 17:25:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000283752_72654848.pth... [2023-12-26 17:25:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000283880_72679424.pth... [2023-12-26 17:25:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000282664_72376320.pth [2023-12-26 17:25:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000282696_72376320.pth [2023-12-26 17:25:46,333][105692] Updated weights for policy 0, policy_version 283759 (0.0009) [2023-12-26 17:25:46,392][105692] Updated weights for policy 0, policy_version 283769 (0.0010) [2023-12-26 17:25:46,447][105692] Updated weights for policy 0, policy_version 283779 (0.0010) [2023-12-26 17:25:46,592][105620] Updated weights for policy 1, policy_version 283881 (0.0008) [2023-12-26 17:25:46,640][105620] Updated weights for policy 1, policy_version 283891 (0.0008) [2023-12-26 17:25:46,692][105620] Updated weights for policy 1, policy_version 283901 (0.0008) [2023-12-26 17:25:46,751][105620] Updated weights for policy 1, policy_version 283911 (0.0008) [2023-12-26 17:25:47,184][105692] Updated weights for policy 0, policy_version 283789 (0.0010) [2023-12-26 17:25:47,241][105692] Updated weights for policy 0, policy_version 283799 (0.0010) [2023-12-26 17:25:47,299][105692] Updated weights for policy 0, policy_version 283809 (0.0010) [2023-12-26 17:25:47,510][105620] Updated weights for policy 1, policy_version 283921 (0.0007) [2023-12-26 17:25:47,561][105620] Updated weights for policy 1, policy_version 283931 (0.0007) [2023-12-26 17:25:47,615][105620] Updated weights for policy 1, policy_version 283941 (0.0005) [2023-12-26 17:25:47,977][105692] Updated weights for policy 0, policy_version 283819 (0.0009) [2023-12-26 17:25:48,038][105692] Updated weights for policy 0, policy_version 283829 (0.0007) [2023-12-26 17:25:48,085][105692] Updated weights for policy 0, policy_version 283839 (0.0009) [2023-12-26 17:25:48,369][105620] Updated weights for policy 1, policy_version 283951 (0.0009) [2023-12-26 17:25:48,419][105620] Updated weights for policy 1, policy_version 283961 (0.0011) [2023-12-26 17:25:48,468][105620] Updated weights for policy 1, policy_version 283971 (0.0010) [2023-12-26 17:25:48,817][105692] Updated weights for policy 0, policy_version 283849 (0.0009) [2023-12-26 17:25:48,883][105692] Updated weights for policy 0, policy_version 283859 (0.0007) [2023-12-26 17:25:48,938][105692] Updated weights for policy 0, policy_version 283869 (0.0010) [2023-12-26 17:25:49,000][105692] Updated weights for policy 0, policy_version 283879 (0.0010) [2023-12-26 17:25:49,140][105620] Updated weights for policy 1, policy_version 283981 (0.0010) [2023-12-26 17:25:49,194][105620] Updated weights for policy 1, policy_version 283991 (0.0010) [2023-12-26 17:25:49,262][105620] Updated weights for policy 1, policy_version 284001 (0.0008) [2023-12-26 17:25:49,648][105692] Updated weights for policy 0, policy_version 283889 (0.0009) [2023-12-26 17:25:49,710][105692] Updated weights for policy 0, policy_version 283899 (0.0011) [2023-12-26 17:25:49,771][105692] Updated weights for policy 0, policy_version 283909 (0.0007) [2023-12-26 17:25:49,893][105620] Updated weights for policy 1, policy_version 284011 (0.0009) [2023-12-26 17:25:49,959][105620] Updated weights for policy 1, policy_version 284021 (0.0006) [2023-12-26 17:25:50,028][105620] Updated weights for policy 1, policy_version 284031 (0.0007) [2023-12-26 17:25:50,384][105692] Updated weights for policy 0, policy_version 283919 (0.0007) [2023-12-26 17:25:50,444][105692] Updated weights for policy 0, policy_version 283929 (0.0008) [2023-12-26 17:25:50,498][105692] Updated weights for policy 0, policy_version 283939 (0.0010) [2023-12-26 17:25:50,756][105620] Updated weights for policy 1, policy_version 284041 (0.0011) [2023-12-26 17:25:50,819][105620] Updated weights for policy 1, policy_version 284051 (0.0010) [2023-12-26 17:25:50,885][105620] Updated weights for policy 1, policy_version 284061 (0.0010) [2023-12-26 17:25:50,947][105620] Updated weights for policy 1, policy_version 284071 (0.0009) [2023-12-26 17:25:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 145432576. Throughput: 0: 9426.9, 1: 9869.1. Samples: 145420116. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 17:25:51,063][104569] Avg episode reward: [(0, '9083.541'), (1, '8062.545')] [2023-12-26 17:25:51,261][105692] Updated weights for policy 0, policy_version 283949 (0.0010) [2023-12-26 17:25:51,327][105692] Updated weights for policy 0, policy_version 283959 (0.0009) [2023-12-26 17:25:51,392][105692] Updated weights for policy 0, policy_version 283969 (0.0008) [2023-12-26 17:25:51,732][105620] Updated weights for policy 1, policy_version 284081 (0.0009) [2023-12-26 17:25:51,796][105620] Updated weights for policy 1, policy_version 284091 (0.0009) [2023-12-26 17:25:51,857][105620] Updated weights for policy 1, policy_version 284101 (0.0009) [2023-12-26 17:25:52,104][105692] Updated weights for policy 0, policy_version 283979 (0.0009) [2023-12-26 17:25:52,168][105692] Updated weights for policy 0, policy_version 283989 (0.0009) [2023-12-26 17:25:52,216][105692] Updated weights for policy 0, policy_version 283999 (0.0009) [2023-12-26 17:25:52,609][105620] Updated weights for policy 1, policy_version 284111 (0.0009) [2023-12-26 17:25:52,667][105620] Updated weights for policy 1, policy_version 284121 (0.0009) [2023-12-26 17:25:52,720][105620] Updated weights for policy 1, policy_version 284131 (0.0009) [2023-12-26 17:25:52,939][105692] Updated weights for policy 0, policy_version 284009 (0.0009) [2023-12-26 17:25:52,993][105692] Updated weights for policy 0, policy_version 284019 (0.0009) [2023-12-26 17:25:53,054][105692] Updated weights for policy 0, policy_version 284029 (0.0009) [2023-12-26 17:25:53,112][105692] Updated weights for policy 0, policy_version 284039 (0.0009) [2023-12-26 17:25:53,421][105620] Updated weights for policy 1, policy_version 284141 (0.0009) [2023-12-26 17:25:53,484][105620] Updated weights for policy 1, policy_version 284151 (0.0009) [2023-12-26 17:25:53,528][105586] KL-divergence is very high: 151.2530 [2023-12-26 17:25:53,548][105620] Updated weights for policy 1, policy_version 284161 (0.0009) [2023-12-26 17:25:53,571][105586] KL-divergence is very high: 149.4448 [2023-12-26 17:25:53,894][105692] Updated weights for policy 0, policy_version 284049 (0.0009) [2023-12-26 17:25:53,942][105692] Updated weights for policy 0, policy_version 284060 (0.0009) [2023-12-26 17:25:54,001][105692] Updated weights for policy 0, policy_version 284070 (0.0009) [2023-12-26 17:25:54,260][105620] Updated weights for policy 1, policy_version 284171 (0.0009) [2023-12-26 17:25:54,313][105620] Updated weights for policy 1, policy_version 284181 (0.0008) [2023-12-26 17:25:54,367][105620] Updated weights for policy 1, policy_version 284191 (0.0009) [2023-12-26 17:25:54,804][105692] Updated weights for policy 0, policy_version 284080 (0.0009) [2023-12-26 17:25:54,862][105692] Updated weights for policy 0, policy_version 284090 (0.0010) [2023-12-26 17:25:54,920][105692] Updated weights for policy 0, policy_version 284101 (0.0010) [2023-12-26 17:25:55,032][105620] Updated weights for policy 1, policy_version 284201 (0.0009) [2023-12-26 17:25:55,087][105620] Updated weights for policy 1, policy_version 284211 (0.0008) [2023-12-26 17:25:55,140][105620] Updated weights for policy 1, policy_version 284221 (0.0008) [2023-12-26 17:25:55,192][105620] Updated weights for policy 1, policy_version 284231 (0.0010) [2023-12-26 17:25:55,619][105692] Updated weights for policy 0, policy_version 284111 (0.0006) [2023-12-26 17:25:55,664][105692] Updated weights for policy 0, policy_version 284121 (0.0005) [2023-12-26 17:25:55,719][105692] Updated weights for policy 0, policy_version 284131 (0.0010) [2023-12-26 17:25:56,000][105620] Updated weights for policy 1, policy_version 284241 (0.0008) [2023-12-26 17:25:56,048][105620] Updated weights for policy 1, policy_version 284251 (0.0008) [2023-12-26 17:25:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 145522688. Throughput: 0: 9389.9, 1: 9833.1. Samples: 145533876. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 17:25:56,062][104569] Avg episode reward: [(0, '9083.057'), (1, '7879.409')] [2023-12-26 17:25:56,103][105620] Updated weights for policy 1, policy_version 284261 (0.0008) [2023-12-26 17:25:56,427][105692] Updated weights for policy 0, policy_version 284141 (0.0010) [2023-12-26 17:25:56,483][105692] Updated weights for policy 0, policy_version 284151 (0.0008) [2023-12-26 17:25:56,538][105692] Updated weights for policy 0, policy_version 284161 (0.0006) [2023-12-26 17:25:56,911][105620] Updated weights for policy 1, policy_version 284271 (0.0008) [2023-12-26 17:25:56,961][105620] Updated weights for policy 1, policy_version 284281 (0.0008) [2023-12-26 17:25:57,004][105620] Updated weights for policy 1, policy_version 284291 (0.0008) [2023-12-26 17:25:57,223][105692] Updated weights for policy 0, policy_version 284171 (0.0008) [2023-12-26 17:25:57,285][105692] Updated weights for policy 0, policy_version 284181 (0.0010) [2023-12-26 17:25:57,344][105692] Updated weights for policy 0, policy_version 284191 (0.0006) [2023-12-26 17:25:57,854][105620] Updated weights for policy 1, policy_version 284301 (0.0009) [2023-12-26 17:25:57,860][105692] Updated weights for policy 0, policy_version 284201 (0.0005) [2023-12-26 17:25:57,911][105620] Updated weights for policy 1, policy_version 284311 (0.0006) [2023-12-26 17:25:57,913][105692] Updated weights for policy 0, policy_version 284211 (0.0010) [2023-12-26 17:25:57,962][105620] Updated weights for policy 1, policy_version 284321 (0.0005) [2023-12-26 17:25:57,967][105692] Updated weights for policy 0, policy_version 284221 (0.0010) [2023-12-26 17:25:58,030][105692] Updated weights for policy 0, policy_version 284231 (0.0010) [2023-12-26 17:25:58,824][105620] Updated weights for policy 1, policy_version 284331 (0.0006) [2023-12-26 17:25:58,867][105692] Updated weights for policy 0, policy_version 284241 (0.0008) [2023-12-26 17:25:58,885][105620] Updated weights for policy 1, policy_version 284341 (0.0008) [2023-12-26 17:25:58,937][105692] Updated weights for policy 0, policy_version 284251 (0.0008) [2023-12-26 17:25:58,951][105620] Updated weights for policy 1, policy_version 284351 (0.0008) [2023-12-26 17:25:58,999][105692] Updated weights for policy 0, policy_version 284261 (0.0010) [2023-12-26 17:25:59,700][105692] Updated weights for policy 0, policy_version 284271 (0.0009) [2023-12-26 17:25:59,743][105620] Updated weights for policy 1, policy_version 284361 (0.0008) [2023-12-26 17:25:59,756][105692] Updated weights for policy 0, policy_version 284281 (0.0008) [2023-12-26 17:25:59,802][105620] Updated weights for policy 1, policy_version 284371 (0.0008) [2023-12-26 17:25:59,809][105692] Updated weights for policy 0, policy_version 284291 (0.0005) [2023-12-26 17:25:59,869][105620] Updated weights for policy 1, policy_version 284381 (0.0008) [2023-12-26 17:25:59,936][105620] Updated weights for policy 1, policy_version 284391 (0.0009) [2023-12-26 17:26:00,625][105692] Updated weights for policy 0, policy_version 284301 (0.0008) [2023-12-26 17:26:00,645][105620] Updated weights for policy 1, policy_version 284402 (0.0009) [2023-12-26 17:26:00,686][105692] Updated weights for policy 0, policy_version 284311 (0.0005) [2023-12-26 17:26:00,697][105620] Updated weights for policy 1, policy_version 284412 (0.0008) [2023-12-26 17:26:00,744][105692] Updated weights for policy 0, policy_version 284321 (0.0005) [2023-12-26 17:26:00,749][105620] Updated weights for policy 1, policy_version 284422 (0.0009) [2023-12-26 17:26:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 145620992. Throughput: 0: 9460.8, 1: 9795.6. Samples: 145591284. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 17:26:01,063][104569] Avg episode reward: [(0, '9001.473'), (1, '7782.291')] [2023-12-26 17:26:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000284328_72802304.pth... [2023-12-26 17:26:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000284424_72818688.pth... [2023-12-26 17:26:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000283208_72515584.pth [2023-12-26 17:26:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000283336_72540160.pth [2023-12-26 17:26:01,385][105692] Updated weights for policy 0, policy_version 284331 (0.0007) [2023-12-26 17:26:01,447][105692] Updated weights for policy 0, policy_version 284341 (0.0008) [2023-12-26 17:26:01,498][105692] Updated weights for policy 0, policy_version 284351 (0.0009) [2023-12-26 17:26:01,556][105620] Updated weights for policy 1, policy_version 284432 (0.0007) [2023-12-26 17:26:01,617][105620] Updated weights for policy 1, policy_version 284442 (0.0006) [2023-12-26 17:26:01,680][105620] Updated weights for policy 1, policy_version 284452 (0.0008) [2023-12-26 17:26:02,270][105692] Updated weights for policy 0, policy_version 284361 (0.0009) [2023-12-26 17:26:02,334][105692] Updated weights for policy 0, policy_version 284371 (0.0008) [2023-12-26 17:26:02,397][105692] Updated weights for policy 0, policy_version 284381 (0.0009) [2023-12-26 17:26:02,433][105620] Updated weights for policy 1, policy_version 284462 (0.0010) [2023-12-26 17:26:02,453][105692] Updated weights for policy 0, policy_version 284391 (0.0010) [2023-12-26 17:26:02,498][105620] Updated weights for policy 1, policy_version 284472 (0.0009) [2023-12-26 17:26:02,562][105620] Updated weights for policy 1, policy_version 284482 (0.0010) [2023-12-26 17:26:03,181][105692] Updated weights for policy 0, policy_version 284401 (0.0008) [2023-12-26 17:26:03,235][105692] Updated weights for policy 0, policy_version 284411 (0.0008) [2023-12-26 17:26:03,286][105692] Updated weights for policy 0, policy_version 284421 (0.0007) [2023-12-26 17:26:03,288][105620] Updated weights for policy 1, policy_version 284492 (0.0010) [2023-12-26 17:26:03,360][105620] Updated weights for policy 1, policy_version 284502 (0.0010) [2023-12-26 17:26:03,410][105620] Updated weights for policy 1, policy_version 284512 (0.0010) [2023-12-26 17:26:03,997][105692] Updated weights for policy 0, policy_version 284431 (0.0009) [2023-12-26 17:26:04,052][105692] Updated weights for policy 0, policy_version 284441 (0.0009) [2023-12-26 17:26:04,108][105620] Updated weights for policy 1, policy_version 284522 (0.0009) [2023-12-26 17:26:04,110][105692] Updated weights for policy 0, policy_version 284451 (0.0008) [2023-12-26 17:26:04,171][105620] Updated weights for policy 1, policy_version 284532 (0.0008) [2023-12-26 17:26:04,230][105620] Updated weights for policy 1, policy_version 284542 (0.0008) [2023-12-26 17:26:04,287][105620] Updated weights for policy 1, policy_version 284552 (0.0010) [2023-12-26 17:26:04,860][105692] Updated weights for policy 0, policy_version 284461 (0.0008) [2023-12-26 17:26:04,916][105692] Updated weights for policy 0, policy_version 284471 (0.0008) [2023-12-26 17:26:04,970][105692] Updated weights for policy 0, policy_version 284481 (0.0009) [2023-12-26 17:26:05,052][105620] Updated weights for policy 1, policy_version 284562 (0.0008) [2023-12-26 17:26:05,100][105620] Updated weights for policy 1, policy_version 284572 (0.0006) [2023-12-26 17:26:05,151][105620] Updated weights for policy 1, policy_version 284582 (0.0005) [2023-12-26 17:26:05,763][105692] Updated weights for policy 0, policy_version 284491 (0.0009) [2023-12-26 17:26:05,814][105692] Updated weights for policy 0, policy_version 284501 (0.0010) [2023-12-26 17:26:05,834][105620] Updated weights for policy 1, policy_version 284592 (0.0007) [2023-12-26 17:26:05,863][105692] Updated weights for policy 0, policy_version 284511 (0.0010) [2023-12-26 17:26:05,885][105620] Updated weights for policy 1, policy_version 284602 (0.0005) [2023-12-26 17:26:05,936][105620] Updated weights for policy 1, policy_version 284612 (0.0005) [2023-12-26 17:26:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.3, 300 sec: 19494.2). Total num frames: 145719296. Throughput: 0: 9452.4, 1: 9741.7. Samples: 145704000. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 17:26:06,062][104569] Avg episode reward: [(0, '9183.659'), (1, '8060.043')] [2023-12-26 17:26:06,559][105692] Updated weights for policy 0, policy_version 284521 (0.0006) [2023-12-26 17:26:06,621][105692] Updated weights for policy 0, policy_version 284531 (0.0008) [2023-12-26 17:26:06,679][105692] Updated weights for policy 0, policy_version 284541 (0.0007) [2023-12-26 17:26:06,685][105620] Updated weights for policy 1, policy_version 284622 (0.0007) [2023-12-26 17:26:06,731][105692] Updated weights for policy 0, policy_version 284551 (0.0006) [2023-12-26 17:26:06,747][105620] Updated weights for policy 1, policy_version 284632 (0.0008) [2023-12-26 17:26:06,811][105620] Updated weights for policy 1, policy_version 284642 (0.0007) [2023-12-26 17:26:07,386][105692] Updated weights for policy 0, policy_version 284561 (0.0006) [2023-12-26 17:26:07,444][105692] Updated weights for policy 0, policy_version 284571 (0.0008) [2023-12-26 17:26:07,506][105692] Updated weights for policy 0, policy_version 284581 (0.0009) [2023-12-26 17:26:07,600][105620] Updated weights for policy 1, policy_version 284652 (0.0008) [2023-12-26 17:26:07,646][105620] Updated weights for policy 1, policy_version 284662 (0.0008) [2023-12-26 17:26:07,696][105620] Updated weights for policy 1, policy_version 284672 (0.0007) [2023-12-26 17:26:08,120][105692] Updated weights for policy 0, policy_version 284591 (0.0009) [2023-12-26 17:26:08,184][105692] Updated weights for policy 0, policy_version 284601 (0.0009) [2023-12-26 17:26:08,241][105692] Updated weights for policy 0, policy_version 284611 (0.0005) [2023-12-26 17:26:08,518][105620] Updated weights for policy 1, policy_version 284682 (0.0006) [2023-12-26 17:26:08,579][105620] Updated weights for policy 1, policy_version 284692 (0.0009) [2023-12-26 17:26:08,638][105620] Updated weights for policy 1, policy_version 284702 (0.0009) [2023-12-26 17:26:08,701][105620] Updated weights for policy 1, policy_version 284712 (0.0010) [2023-12-26 17:26:08,861][105692] Updated weights for policy 0, policy_version 284621 (0.0005) [2023-12-26 17:26:08,913][105692] Updated weights for policy 0, policy_version 284631 (0.0005) [2023-12-26 17:26:08,967][105692] Updated weights for policy 0, policy_version 284641 (0.0005) [2023-12-26 17:26:09,582][105692] Updated weights for policy 0, policy_version 284651 (0.0006) [2023-12-26 17:26:09,588][105620] Updated weights for policy 1, policy_version 284722 (0.0009) [2023-12-26 17:26:09,640][105692] Updated weights for policy 0, policy_version 284661 (0.0007) [2023-12-26 17:26:09,648][105620] Updated weights for policy 1, policy_version 284732 (0.0009) [2023-12-26 17:26:09,692][105692] Updated weights for policy 0, policy_version 284671 (0.0007) [2023-12-26 17:26:09,707][105620] Updated weights for policy 1, policy_version 284742 (0.0008) [2023-12-26 17:26:10,410][105692] Updated weights for policy 0, policy_version 284681 (0.0008) [2023-12-26 17:26:10,443][105620] Updated weights for policy 1, policy_version 284752 (0.0010) [2023-12-26 17:26:10,471][105692] Updated weights for policy 0, policy_version 284691 (0.0006) [2023-12-26 17:26:10,493][105620] Updated weights for policy 1, policy_version 284762 (0.0011) [2023-12-26 17:26:10,529][105692] Updated weights for policy 0, policy_version 284701 (0.0007) [2023-12-26 17:26:10,540][105620] Updated weights for policy 1, policy_version 284772 (0.0010) [2023-12-26 17:26:10,587][105692] Updated weights for policy 0, policy_version 284711 (0.0008) [2023-12-26 17:26:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 145809408. Throughput: 0: 9663.3, 1: 9514.1. Samples: 145821044. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 17:26:11,063][104569] Avg episode reward: [(0, '8992.222'), (1, '8063.713')] [2023-12-26 17:26:11,163][105620] Updated weights for policy 1, policy_version 284782 (0.0008) [2023-12-26 17:26:11,221][105620] Updated weights for policy 1, policy_version 284792 (0.0006) [2023-12-26 17:26:11,289][105620] Updated weights for policy 1, policy_version 284802 (0.0010) [2023-12-26 17:26:11,375][105692] Updated weights for policy 0, policy_version 284721 (0.0008) [2023-12-26 17:26:11,441][105692] Updated weights for policy 0, policy_version 284731 (0.0008) [2023-12-26 17:26:11,494][105692] Updated weights for policy 0, policy_version 284741 (0.0008) [2023-12-26 17:26:12,041][105620] Updated weights for policy 1, policy_version 284812 (0.0011) [2023-12-26 17:26:12,093][105620] Updated weights for policy 1, policy_version 284822 (0.0010) [2023-12-26 17:26:12,150][105620] Updated weights for policy 1, policy_version 284832 (0.0010) [2023-12-26 17:26:12,296][105692] Updated weights for policy 0, policy_version 284751 (0.0008) [2023-12-26 17:26:12,362][105692] Updated weights for policy 0, policy_version 284761 (0.0009) [2023-12-26 17:26:12,435][105692] Updated weights for policy 0, policy_version 284771 (0.0009) [2023-12-26 17:26:12,819][105620] Updated weights for policy 1, policy_version 284842 (0.0009) [2023-12-26 17:26:12,883][105620] Updated weights for policy 1, policy_version 284852 (0.0006) [2023-12-26 17:26:12,944][105620] Updated weights for policy 1, policy_version 284862 (0.0006) [2023-12-26 17:26:13,001][105620] Updated weights for policy 1, policy_version 284872 (0.0006) [2023-12-26 17:26:13,235][105692] Updated weights for policy 0, policy_version 284781 (0.0009) [2023-12-26 17:26:13,282][105692] Updated weights for policy 0, policy_version 284791 (0.0007) [2023-12-26 17:26:13,336][105692] Updated weights for policy 0, policy_version 284801 (0.0008) [2023-12-26 17:26:13,612][105620] Updated weights for policy 1, policy_version 284882 (0.0010) [2023-12-26 17:26:13,676][105620] Updated weights for policy 1, policy_version 284892 (0.0010) [2023-12-26 17:26:13,720][105620] Updated weights for policy 1, policy_version 284902 (0.0010) [2023-12-26 17:26:14,106][105692] Updated weights for policy 0, policy_version 284811 (0.0008) [2023-12-26 17:26:14,167][105692] Updated weights for policy 0, policy_version 284821 (0.0006) [2023-12-26 17:26:14,231][105692] Updated weights for policy 0, policy_version 284831 (0.0006) [2023-12-26 17:26:14,493][105620] Updated weights for policy 1, policy_version 284912 (0.0010) [2023-12-26 17:26:14,555][105620] Updated weights for policy 1, policy_version 284922 (0.0010) [2023-12-26 17:26:14,614][105620] Updated weights for policy 1, policy_version 284932 (0.0011) [2023-12-26 17:26:14,886][105692] Updated weights for policy 0, policy_version 284841 (0.0008) [2023-12-26 17:26:14,949][105692] Updated weights for policy 0, policy_version 284851 (0.0005) [2023-12-26 17:26:15,014][105692] Updated weights for policy 0, policy_version 284861 (0.0006) [2023-12-26 17:26:15,078][105692] Updated weights for policy 0, policy_version 284871 (0.0005) [2023-12-26 17:26:15,337][105620] Updated weights for policy 1, policy_version 284942 (0.0007) [2023-12-26 17:26:15,401][105620] Updated weights for policy 1, policy_version 284952 (0.0008) [2023-12-26 17:26:15,463][105620] Updated weights for policy 1, policy_version 284962 (0.0010) [2023-12-26 17:26:15,721][105692] Updated weights for policy 0, policy_version 284881 (0.0005) [2023-12-26 17:26:15,779][105692] Updated weights for policy 0, policy_version 284891 (0.0005) [2023-12-26 17:26:15,827][105692] Updated weights for policy 0, policy_version 284901 (0.0005) [2023-12-26 17:26:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 145907712. Throughput: 0: 9674.8, 1: 9399.3. Samples: 145876924. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 17:26:16,062][104569] Avg episode reward: [(0, '1566.983'), (1, '7508.078')] [2023-12-26 17:26:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000284968_72957952.pth... [2023-12-26 17:26:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000284904_72949760.pth... [2023-12-26 17:26:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000283752_72654848.pth [2023-12-26 17:26:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000283880_72679424.pth [2023-12-26 17:26:16,174][105620] Updated weights for policy 1, policy_version 284972 (0.0010) [2023-12-26 17:26:16,239][105620] Updated weights for policy 1, policy_version 284982 (0.0010) [2023-12-26 17:26:16,308][105620] Updated weights for policy 1, policy_version 284992 (0.0010) [2023-12-26 17:26:16,358][105692] Updated weights for policy 0, policy_version 284911 (0.0007) [2023-12-26 17:26:16,423][105692] Updated weights for policy 0, policy_version 284921 (0.0005) [2023-12-26 17:26:16,483][105692] Updated weights for policy 0, policy_version 284931 (0.0005) [2023-12-26 17:26:16,906][105620] Updated weights for policy 1, policy_version 285002 (0.0009) [2023-12-26 17:26:16,955][105620] Updated weights for policy 1, policy_version 285012 (0.0005) [2023-12-26 17:26:17,008][105620] Updated weights for policy 1, policy_version 285022 (0.0007) [2023-12-26 17:26:17,060][105620] Updated weights for policy 1, policy_version 285032 (0.0009) [2023-12-26 17:26:17,092][105692] Updated weights for policy 0, policy_version 284941 (0.0007) [2023-12-26 17:26:17,144][105692] Updated weights for policy 0, policy_version 284951 (0.0007) [2023-12-26 17:26:17,196][105692] Updated weights for policy 0, policy_version 284961 (0.0007) [2023-12-26 17:26:17,797][105620] Updated weights for policy 1, policy_version 285042 (0.0010) [2023-12-26 17:26:17,855][105620] Updated weights for policy 1, policy_version 285052 (0.0010) [2023-12-26 17:26:17,903][105692] Updated weights for policy 0, policy_version 284971 (0.0007) [2023-12-26 17:26:17,913][105620] Updated weights for policy 1, policy_version 285062 (0.0010) [2023-12-26 17:26:17,967][105692] Updated weights for policy 0, policy_version 284981 (0.0007) [2023-12-26 17:26:18,027][105692] Updated weights for policy 0, policy_version 284991 (0.0007) [2023-12-26 17:26:18,605][105620] Updated weights for policy 1, policy_version 285072 (0.0007) [2023-12-26 17:26:18,676][105620] Updated weights for policy 1, policy_version 285082 (0.0006) [2023-12-26 17:26:18,736][105692] Updated weights for policy 0, policy_version 285001 (0.0007) [2023-12-26 17:26:18,744][105620] Updated weights for policy 1, policy_version 285092 (0.0006) [2023-12-26 17:26:18,791][105692] Updated weights for policy 0, policy_version 285011 (0.0009) [2023-12-26 17:26:18,838][105692] Updated weights for policy 0, policy_version 285021 (0.0005) [2023-12-26 17:26:18,894][105692] Updated weights for policy 0, policy_version 285031 (0.0005) [2023-12-26 17:26:19,327][105620] Updated weights for policy 1, policy_version 285102 (0.0007) [2023-12-26 17:26:19,373][105586] KL-divergence is very high: 138.9471 [2023-12-26 17:26:19,390][105620] Updated weights for policy 1, policy_version 285112 (0.0009) [2023-12-26 17:26:19,406][105586] KL-divergence is very high: 133.4437 [2023-12-26 17:26:19,417][105586] KL-divergence is very high: 221.9267 [2023-12-26 17:26:19,447][105620] Updated weights for policy 1, policy_version 285122 (0.0009) [2023-12-26 17:26:19,453][105586] KL-divergence is very high: 134.3659 [2023-12-26 17:26:19,465][105586] KL-divergence is very high: 217.7396 [2023-12-26 17:26:19,561][105692] Updated weights for policy 0, policy_version 285041 (0.0009) [2023-12-26 17:26:19,627][105692] Updated weights for policy 0, policy_version 285051 (0.0010) [2023-12-26 17:26:19,695][105692] Updated weights for policy 0, policy_version 285061 (0.0009) [2023-12-26 17:26:20,109][105586] KL-divergence is very high: 186.2684 [2023-12-26 17:26:20,129][105620] Updated weights for policy 1, policy_version 285132 (0.0008) [2023-12-26 17:26:20,161][105586] KL-divergence is very high: 103.9560 [2023-12-26 17:26:20,174][105586] KL-divergence is very high: 145.9532 [2023-12-26 17:26:20,188][105586] KL-divergence is very high: 235.1588 [2023-12-26 17:26:20,193][105620] Updated weights for policy 1, policy_version 285142 (0.0009) [2023-12-26 17:26:20,202][105586] KL-divergence is very high: 186.7557 [2023-12-26 17:26:20,216][105586] KL-divergence is very high: 164.2515 [2023-12-26 17:26:20,229][105586] KL-divergence is very high: 171.1504 [2023-12-26 17:26:20,243][105586] KL-divergence is very high: 243.1637 [2023-12-26 17:26:20,257][105586] KL-divergence is very high: 157.7905 [2023-12-26 17:26:20,262][105620] Updated weights for policy 1, policy_version 285152 (0.0006) [2023-12-26 17:26:20,271][105586] KL-divergence is very high: 121.4930 [2023-12-26 17:26:20,285][105586] KL-divergence is very high: 112.5547 [2023-12-26 17:26:20,298][105586] KL-divergence is very high: 159.1223 [2023-12-26 17:26:20,528][105692] Updated weights for policy 0, policy_version 285071 (0.0009) [2023-12-26 17:26:20,588][105692] Updated weights for policy 0, policy_version 285081 (0.0007) [2023-12-26 17:26:20,644][105692] Updated weights for policy 0, policy_version 285091 (0.0009) [2023-12-26 17:26:20,978][105620] Updated weights for policy 1, policy_version 285162 (0.0006) [2023-12-26 17:26:21,042][105620] Updated weights for policy 1, policy_version 285172 (0.0008) [2023-12-26 17:26:21,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 146006016. Throughput: 0: 9864.6, 1: 9415.4. Samples: 145999768. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 17:26:21,063][104569] Avg episode reward: [(0, '1360.115'), (1, '7515.132')] [2023-12-26 17:26:21,109][105620] Updated weights for policy 1, policy_version 285182 (0.0010) [2023-12-26 17:26:21,178][105620] Updated weights for policy 1, policy_version 285192 (0.0009) [2023-12-26 17:26:21,307][105692] Updated weights for policy 0, policy_version 285101 (0.0009) [2023-12-26 17:26:21,371][105692] Updated weights for policy 0, policy_version 285111 (0.0010) [2023-12-26 17:26:21,441][105692] Updated weights for policy 0, policy_version 285121 (0.0009) [2023-12-26 17:26:21,981][105620] Updated weights for policy 1, policy_version 285202 (0.0009) [2023-12-26 17:26:22,043][105620] Updated weights for policy 1, policy_version 285212 (0.0009) [2023-12-26 17:26:22,112][105620] Updated weights for policy 1, policy_version 285222 (0.0009) [2023-12-26 17:26:22,240][105692] Updated weights for policy 0, policy_version 285131 (0.0009) [2023-12-26 17:26:22,312][105692] Updated weights for policy 0, policy_version 285141 (0.0008) [2023-12-26 17:26:22,379][105692] Updated weights for policy 0, policy_version 285151 (0.0008) [2023-12-26 17:26:22,917][105620] Updated weights for policy 1, policy_version 285232 (0.0008) [2023-12-26 17:26:22,962][105620] Updated weights for policy 1, policy_version 285242 (0.0008) [2023-12-26 17:26:23,008][105620] Updated weights for policy 1, policy_version 285252 (0.0006) [2023-12-26 17:26:23,018][105586] KL-divergence is very high: 113.6610 [2023-12-26 17:26:23,119][105692] Updated weights for policy 0, policy_version 285161 (0.0011) [2023-12-26 17:26:23,178][105692] Updated weights for policy 0, policy_version 285171 (0.0010) [2023-12-26 17:26:23,227][105692] Updated weights for policy 0, policy_version 285181 (0.0010) [2023-12-26 17:26:23,274][105692] Updated weights for policy 0, policy_version 285191 (0.0008) [2023-12-26 17:26:23,641][105620] Updated weights for policy 1, policy_version 285262 (0.0007) [2023-12-26 17:26:23,689][105620] Updated weights for policy 1, policy_version 285272 (0.0008) [2023-12-26 17:26:23,737][105620] Updated weights for policy 1, policy_version 285282 (0.0008) [2023-12-26 17:26:24,017][105692] Updated weights for policy 0, policy_version 285201 (0.0010) [2023-12-26 17:26:24,081][105692] Updated weights for policy 0, policy_version 285211 (0.0008) [2023-12-26 17:26:24,154][105692] Updated weights for policy 0, policy_version 285221 (0.0008) [2023-12-26 17:26:24,492][105620] Updated weights for policy 1, policy_version 285292 (0.0008) [2023-12-26 17:26:24,547][105620] Updated weights for policy 1, policy_version 285302 (0.0005) [2023-12-26 17:26:24,610][105620] Updated weights for policy 1, policy_version 285312 (0.0007) [2023-12-26 17:26:24,830][105692] Updated weights for policy 0, policy_version 285231 (0.0010) [2023-12-26 17:26:24,878][105692] Updated weights for policy 0, policy_version 285241 (0.0010) [2023-12-26 17:26:24,926][105692] Updated weights for policy 0, policy_version 285251 (0.0011) [2023-12-26 17:26:25,281][105620] Updated weights for policy 1, policy_version 285322 (0.0009) [2023-12-26 17:26:25,334][105620] Updated weights for policy 1, policy_version 285332 (0.0010) [2023-12-26 17:26:25,337][105586] KL-divergence is very high: 116.2100 [2023-12-26 17:26:25,359][105586] KL-divergence is very high: 195.1187 [2023-12-26 17:26:25,378][105586] KL-divergence is very high: 144.3136 [2023-12-26 17:26:25,381][105620] Updated weights for policy 1, policy_version 285342 (0.0009) [2023-12-26 17:26:25,397][105586] KL-divergence is very high: 174.2891 [2023-12-26 17:26:25,416][105586] KL-divergence is very high: 102.7913 [2023-12-26 17:26:25,431][105620] Updated weights for policy 1, policy_version 285352 (0.0008) [2023-12-26 17:26:25,638][105692] Updated weights for policy 0, policy_version 285261 (0.0011) [2023-12-26 17:26:25,692][105692] Updated weights for policy 0, policy_version 285271 (0.0010) [2023-12-26 17:26:25,753][105692] Updated weights for policy 0, policy_version 285281 (0.0010) [2023-12-26 17:26:26,058][105620] Updated weights for policy 1, policy_version 285362 (0.0005) [2023-12-26 17:26:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 146104320. Throughput: 0: 9842.8, 1: 9468.8. Samples: 146114628. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 17:26:26,062][104569] Avg episode reward: [(0, '6679.558'), (1, '6680.324')] [2023-12-26 17:26:26,108][105620] Updated weights for policy 1, policy_version 285372 (0.0008) [2023-12-26 17:26:26,151][105620] Updated weights for policy 1, policy_version 285382 (0.0007) [2023-12-26 17:26:26,495][105692] Updated weights for policy 0, policy_version 285291 (0.0010) [2023-12-26 17:26:26,554][105692] Updated weights for policy 0, policy_version 285301 (0.0010) [2023-12-26 17:26:26,609][105692] Updated weights for policy 0, policy_version 285311 (0.0011) [2023-12-26 17:26:26,942][105620] Updated weights for policy 1, policy_version 285392 (0.0006) [2023-12-26 17:26:27,005][105620] Updated weights for policy 1, policy_version 285402 (0.0008) [2023-12-26 17:26:27,051][105620] Updated weights for policy 1, policy_version 285412 (0.0008) [2023-12-26 17:26:27,267][105692] Updated weights for policy 0, policy_version 285321 (0.0006) [2023-12-26 17:26:27,313][105692] Updated weights for policy 0, policy_version 285331 (0.0008) [2023-12-26 17:26:27,374][105692] Updated weights for policy 0, policy_version 285341 (0.0009) [2023-12-26 17:26:27,433][105692] Updated weights for policy 0, policy_version 285351 (0.0008) [2023-12-26 17:26:27,735][105620] Updated weights for policy 1, policy_version 285422 (0.0009) [2023-12-26 17:26:27,783][105620] Updated weights for policy 1, policy_version 285432 (0.0009) [2023-12-26 17:26:27,833][105620] Updated weights for policy 1, policy_version 285442 (0.0010) [2023-12-26 17:26:28,116][105692] Updated weights for policy 0, policy_version 285361 (0.0009) [2023-12-26 17:26:28,181][105692] Updated weights for policy 0, policy_version 285371 (0.0007) [2023-12-26 17:26:28,250][105692] Updated weights for policy 0, policy_version 285381 (0.0007) [2023-12-26 17:26:28,629][105620] Updated weights for policy 1, policy_version 285452 (0.0009) [2023-12-26 17:26:28,679][105620] Updated weights for policy 1, policy_version 285462 (0.0008) [2023-12-26 17:26:28,731][105620] Updated weights for policy 1, policy_version 285472 (0.0007) [2023-12-26 17:26:28,898][105692] Updated weights for policy 0, policy_version 285391 (0.0008) [2023-12-26 17:26:28,953][105692] Updated weights for policy 0, policy_version 285401 (0.0010) [2023-12-26 17:26:29,004][105692] Updated weights for policy 0, policy_version 285411 (0.0010) [2023-12-26 17:26:29,515][105620] Updated weights for policy 1, policy_version 285483 (0.0010) [2023-12-26 17:26:29,577][105620] Updated weights for policy 1, policy_version 285493 (0.0008) [2023-12-26 17:26:29,632][105620] Updated weights for policy 1, policy_version 285503 (0.0008) [2023-12-26 17:26:29,744][105692] Updated weights for policy 0, policy_version 285421 (0.0009) [2023-12-26 17:26:29,763][105585] KL-divergence is very high: 120.6087 [2023-12-26 17:26:29,800][105692] Updated weights for policy 0, policy_version 285431 (0.0008) [2023-12-26 17:26:29,868][105692] Updated weights for policy 0, policy_version 285441 (0.0007) [2023-12-26 17:26:29,869][105585] KL-divergence is very high: 110.6966 [2023-12-26 17:26:30,363][105620] Updated weights for policy 1, policy_version 285513 (0.0009) [2023-12-26 17:26:30,419][105620] Updated weights for policy 1, policy_version 285523 (0.0010) [2023-12-26 17:26:30,480][105620] Updated weights for policy 1, policy_version 285533 (0.0008) [2023-12-26 17:26:30,481][105692] Updated weights for policy 0, policy_version 285451 (0.0006) [2023-12-26 17:26:30,534][105620] Updated weights for policy 1, policy_version 285543 (0.0007) [2023-12-26 17:26:30,542][105692] Updated weights for policy 0, policy_version 285461 (0.0006) [2023-12-26 17:26:30,606][105692] Updated weights for policy 0, policy_version 285471 (0.0006) [2023-12-26 17:26:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 146202624. Throughput: 0: 9867.5, 1: 9512.8. Samples: 146172816. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-12-26 17:26:31,062][104569] Avg episode reward: [(0, '8456.992'), (1, '7228.329')] [2023-12-26 17:26:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000285480_73097216.pth... [2023-12-26 17:26:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000285544_73105408.pth... [2023-12-26 17:26:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000284424_72818688.pth [2023-12-26 17:26:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000284328_72802304.pth [2023-12-26 17:26:31,241][105692] Updated weights for policy 0, policy_version 285481 (0.0006) [2023-12-26 17:26:31,247][105620] Updated weights for policy 1, policy_version 285553 (0.0006) [2023-12-26 17:26:31,304][105692] Updated weights for policy 0, policy_version 285491 (0.0011) [2023-12-26 17:26:31,309][105620] Updated weights for policy 1, policy_version 285563 (0.0008) [2023-12-26 17:26:31,364][105692] Updated weights for policy 0, policy_version 285501 (0.0011) [2023-12-26 17:26:31,370][105620] Updated weights for policy 1, policy_version 285573 (0.0009) [2023-12-26 17:26:31,429][105692] Updated weights for policy 0, policy_version 285511 (0.0011) [2023-12-26 17:26:32,048][105620] Updated weights for policy 1, policy_version 285583 (0.0008) [2023-12-26 17:26:32,111][105620] Updated weights for policy 1, policy_version 285593 (0.0008) [2023-12-26 17:26:32,137][105692] Updated weights for policy 0, policy_version 285521 (0.0011) [2023-12-26 17:26:32,165][105620] Updated weights for policy 1, policy_version 285603 (0.0006) [2023-12-26 17:26:32,194][105692] Updated weights for policy 0, policy_version 285531 (0.0009) [2023-12-26 17:26:32,263][105692] Updated weights for policy 0, policy_version 285541 (0.0010) [2023-12-26 17:26:32,889][105620] Updated weights for policy 1, policy_version 285613 (0.0006) [2023-12-26 17:26:32,943][105620] Updated weights for policy 1, policy_version 285623 (0.0006) [2023-12-26 17:26:32,996][105620] Updated weights for policy 1, policy_version 285633 (0.0005) [2023-12-26 17:26:33,030][105692] Updated weights for policy 0, policy_version 285551 (0.0009) [2023-12-26 17:26:33,093][105692] Updated weights for policy 0, policy_version 285561 (0.0010) [2023-12-26 17:26:33,149][105692] Updated weights for policy 0, policy_version 285571 (0.0010) [2023-12-26 17:26:33,614][105620] Updated weights for policy 1, policy_version 285643 (0.0007) [2023-12-26 17:26:33,671][105620] Updated weights for policy 1, policy_version 285653 (0.0006) [2023-12-26 17:26:33,729][105620] Updated weights for policy 1, policy_version 285663 (0.0005) [2023-12-26 17:26:33,843][105692] Updated weights for policy 0, policy_version 285581 (0.0009) [2023-12-26 17:26:33,894][105692] Updated weights for policy 0, policy_version 285591 (0.0008) [2023-12-26 17:26:33,941][105692] Updated weights for policy 0, policy_version 285601 (0.0008) [2023-12-26 17:26:34,425][105620] Updated weights for policy 1, policy_version 285673 (0.0010) [2023-12-26 17:26:34,478][105620] Updated weights for policy 1, policy_version 285683 (0.0011) [2023-12-26 17:26:34,530][105620] Updated weights for policy 1, policy_version 285693 (0.0011) [2023-12-26 17:26:34,583][105620] Updated weights for policy 1, policy_version 285703 (0.0011) [2023-12-26 17:26:34,606][105692] Updated weights for policy 0, policy_version 285611 (0.0007) [2023-12-26 17:26:34,661][105692] Updated weights for policy 0, policy_version 285621 (0.0005) [2023-12-26 17:26:34,708][105692] Updated weights for policy 0, policy_version 285631 (0.0005) [2023-12-26 17:26:35,165][105620] Updated weights for policy 1, policy_version 285713 (0.0006) [2023-12-26 17:26:35,220][105620] Updated weights for policy 1, policy_version 285723 (0.0005) [2023-12-26 17:26:35,265][105620] Updated weights for policy 1, policy_version 285733 (0.0005) [2023-12-26 17:26:35,443][105692] Updated weights for policy 0, policy_version 285641 (0.0005) [2023-12-26 17:26:35,494][105692] Updated weights for policy 0, policy_version 285651 (0.0006) [2023-12-26 17:26:35,539][105692] Updated weights for policy 0, policy_version 285661 (0.0005) [2023-12-26 17:26:35,597][105692] Updated weights for policy 0, policy_version 285671 (0.0005) [2023-12-26 17:26:35,916][105620] Updated weights for policy 1, policy_version 285743 (0.0009) [2023-12-26 17:26:35,961][105620] Updated weights for policy 1, policy_version 285753 (0.0010) [2023-12-26 17:26:36,006][105620] Updated weights for policy 1, policy_version 285763 (0.0010) [2023-12-26 17:26:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 146309120. Throughput: 0: 9826.0, 1: 9576.1. Samples: 146293212. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-12-26 17:26:36,063][104569] Avg episode reward: [(0, '8612.442'), (1, '7691.682')] [2023-12-26 17:26:36,274][105692] Updated weights for policy 0, policy_version 285681 (0.0008) [2023-12-26 17:26:36,346][105692] Updated weights for policy 0, policy_version 285691 (0.0008) [2023-12-26 17:26:36,412][105692] Updated weights for policy 0, policy_version 285701 (0.0010) [2023-12-26 17:26:36,711][105620] Updated weights for policy 1, policy_version 285773 (0.0008) [2023-12-26 17:26:36,766][105620] Updated weights for policy 1, policy_version 285783 (0.0005) [2023-12-26 17:26:36,823][105620] Updated weights for policy 1, policy_version 285793 (0.0005) [2023-12-26 17:26:37,303][105692] Updated weights for policy 0, policy_version 285711 (0.0009) [2023-12-26 17:26:37,357][105620] Updated weights for policy 1, policy_version 285803 (0.0007) [2023-12-26 17:26:37,370][105692] Updated weights for policy 0, policy_version 285721 (0.0006) [2023-12-26 17:26:37,411][105620] Updated weights for policy 1, policy_version 285813 (0.0010) [2023-12-26 17:26:37,434][105692] Updated weights for policy 0, policy_version 285731 (0.0006) [2023-12-26 17:26:37,461][105620] Updated weights for policy 1, policy_version 285823 (0.0011) [2023-12-26 17:26:38,104][105692] Updated weights for policy 0, policy_version 285741 (0.0007) [2023-12-26 17:26:38,162][105692] Updated weights for policy 0, policy_version 285751 (0.0009) [2023-12-26 17:26:38,200][105620] Updated weights for policy 1, policy_version 285833 (0.0011) [2023-12-26 17:26:38,221][105692] Updated weights for policy 0, policy_version 285761 (0.0008) [2023-12-26 17:26:38,252][105620] Updated weights for policy 1, policy_version 285843 (0.0011) [2023-12-26 17:26:38,308][105620] Updated weights for policy 1, policy_version 285853 (0.0010) [2023-12-26 17:26:38,372][105620] Updated weights for policy 1, policy_version 285863 (0.0011) [2023-12-26 17:26:39,033][105692] Updated weights for policy 0, policy_version 285771 (0.0009) [2023-12-26 17:26:39,069][105620] Updated weights for policy 1, policy_version 285873 (0.0010) [2023-12-26 17:26:39,088][105692] Updated weights for policy 0, policy_version 285781 (0.0007) [2023-12-26 17:26:39,127][105620] Updated weights for policy 1, policy_version 285883 (0.0008) [2023-12-26 17:26:39,148][105692] Updated weights for policy 0, policy_version 285791 (0.0008) [2023-12-26 17:26:39,184][105620] Updated weights for policy 1, policy_version 285893 (0.0005) [2023-12-26 17:26:39,924][105692] Updated weights for policy 0, policy_version 285801 (0.0009) [2023-12-26 17:26:39,924][105620] Updated weights for policy 1, policy_version 285903 (0.0007) [2023-12-26 17:26:39,983][105620] Updated weights for policy 1, policy_version 285913 (0.0007) [2023-12-26 17:26:39,986][105692] Updated weights for policy 0, policy_version 285811 (0.0008) [2023-12-26 17:26:40,038][105620] Updated weights for policy 1, policy_version 285923 (0.0008) [2023-12-26 17:26:40,046][105692] Updated weights for policy 0, policy_version 285821 (0.0007) [2023-12-26 17:26:40,104][105692] Updated weights for policy 0, policy_version 285831 (0.0006) [2023-12-26 17:26:40,705][105620] Updated weights for policy 1, policy_version 285933 (0.0007) [2023-12-26 17:26:40,764][105620] Updated weights for policy 1, policy_version 285943 (0.0005) [2023-12-26 17:26:40,828][105620] Updated weights for policy 1, policy_version 285953 (0.0005) [2023-12-26 17:26:40,848][105692] Updated weights for policy 0, policy_version 285841 (0.0008) [2023-12-26 17:26:40,904][105692] Updated weights for policy 0, policy_version 285851 (0.0009) [2023-12-26 17:26:40,956][105692] Updated weights for policy 0, policy_version 285861 (0.0009) [2023-12-26 17:26:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 146407424. Throughput: 0: 9786.0, 1: 9719.4. Samples: 146411620. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-12-26 17:26:41,062][104569] Avg episode reward: [(0, '9183.300'), (1, '7782.029')] [2023-12-26 17:26:41,500][105620] Updated weights for policy 1, policy_version 285963 (0.0006) [2023-12-26 17:26:41,563][105620] Updated weights for policy 1, policy_version 285973 (0.0009) [2023-12-26 17:26:41,628][105620] Updated weights for policy 1, policy_version 285983 (0.0011) [2023-12-26 17:26:41,781][105692] Updated weights for policy 0, policy_version 285872 (0.0009) [2023-12-26 17:26:41,849][105692] Updated weights for policy 0, policy_version 285882 (0.0011) [2023-12-26 17:26:41,907][105692] Updated weights for policy 0, policy_version 285892 (0.0011) [2023-12-26 17:26:42,308][105620] Updated weights for policy 1, policy_version 285993 (0.0008) [2023-12-26 17:26:42,379][105620] Updated weights for policy 1, policy_version 286003 (0.0011) [2023-12-26 17:26:42,441][105620] Updated weights for policy 1, policy_version 286013 (0.0011) [2023-12-26 17:26:42,500][105620] Updated weights for policy 1, policy_version 286023 (0.0010) [2023-12-26 17:26:42,660][105692] Updated weights for policy 0, policy_version 285902 (0.0010) [2023-12-26 17:26:42,724][105692] Updated weights for policy 0, policy_version 285912 (0.0010) [2023-12-26 17:26:42,783][105692] Updated weights for policy 0, policy_version 285922 (0.0007) [2023-12-26 17:26:43,217][105620] Updated weights for policy 1, policy_version 286033 (0.0011) [2023-12-26 17:26:43,262][105586] KL-divergence is very high: 113.2888 [2023-12-26 17:26:43,276][105620] Updated weights for policy 1, policy_version 286043 (0.0010) [2023-12-26 17:26:43,345][105620] Updated weights for policy 1, policy_version 286053 (0.0011) [2023-12-26 17:26:43,387][105692] Updated weights for policy 0, policy_version 285932 (0.0010) [2023-12-26 17:26:43,436][105692] Updated weights for policy 0, policy_version 285942 (0.0009) [2023-12-26 17:26:43,487][105692] Updated weights for policy 0, policy_version 285952 (0.0010) [2023-12-26 17:26:43,517][105585] KL-divergence is very high: 165.2762 [2023-12-26 17:26:44,083][105692] Updated weights for policy 0, policy_version 285962 (0.0008) [2023-12-26 17:26:44,097][105620] Updated weights for policy 1, policy_version 286063 (0.0010) [2023-12-26 17:26:44,134][105692] Updated weights for policy 0, policy_version 285972 (0.0006) [2023-12-26 17:26:44,150][105620] Updated weights for policy 1, policy_version 286073 (0.0011) [2023-12-26 17:26:44,183][105586] KL-divergence is very high: 109.7629 [2023-12-26 17:26:44,193][105692] Updated weights for policy 0, policy_version 285982 (0.0010) [2023-12-26 17:26:44,199][105620] Updated weights for policy 1, policy_version 286083 (0.0010) [2023-12-26 17:26:44,257][105692] Updated weights for policy 0, policy_version 285992 (0.0008) [2023-12-26 17:26:44,833][105620] Updated weights for policy 1, policy_version 286093 (0.0009) [2023-12-26 17:26:44,895][105620] Updated weights for policy 1, policy_version 286103 (0.0008) [2023-12-26 17:26:44,941][105692] Updated weights for policy 0, policy_version 286002 (0.0010) [2023-12-26 17:26:44,955][105620] Updated weights for policy 1, policy_version 286113 (0.0008) [2023-12-26 17:26:44,990][105692] Updated weights for policy 0, policy_version 286012 (0.0010) [2023-12-26 17:26:45,040][105692] Updated weights for policy 0, policy_version 286022 (0.0009) [2023-12-26 17:26:45,679][105692] Updated weights for policy 0, policy_version 286032 (0.0007) [2023-12-26 17:26:45,728][105692] Updated weights for policy 0, policy_version 286042 (0.0010) [2023-12-26 17:26:45,730][105620] Updated weights for policy 1, policy_version 286123 (0.0007) [2023-12-26 17:26:45,773][105692] Updated weights for policy 0, policy_version 286052 (0.0010) [2023-12-26 17:26:45,786][105620] Updated weights for policy 1, policy_version 286133 (0.0005) [2023-12-26 17:26:45,850][105620] Updated weights for policy 1, policy_version 286143 (0.0010) [2023-12-26 17:26:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 146505728. Throughput: 0: 9720.5, 1: 9777.3. Samples: 146468688. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-12-26 17:26:46,063][104569] Avg episode reward: [(0, '9262.629'), (1, '5359.869')] [2023-12-26 17:26:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000286152_73261056.pth... [2023-12-26 17:26:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000286056_73244672.pth... [2023-12-26 17:26:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000284968_72957952.pth [2023-12-26 17:26:46,082][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000284904_72949760.pth [2023-12-26 17:26:46,427][105692] Updated weights for policy 0, policy_version 286062 (0.0009) [2023-12-26 17:26:46,478][105692] Updated weights for policy 0, policy_version 286072 (0.0009) [2023-12-26 17:26:46,544][105692] Updated weights for policy 0, policy_version 286082 (0.0007) [2023-12-26 17:26:46,557][105620] Updated weights for policy 1, policy_version 286153 (0.0010) [2023-12-26 17:26:46,612][105620] Updated weights for policy 1, policy_version 286163 (0.0010) [2023-12-26 17:26:46,666][105620] Updated weights for policy 1, policy_version 286173 (0.0010) [2023-12-26 17:26:46,723][105620] Updated weights for policy 1, policy_version 286183 (0.0010) [2023-12-26 17:26:47,112][105692] Updated weights for policy 0, policy_version 286092 (0.0008) [2023-12-26 17:26:47,171][105692] Updated weights for policy 0, policy_version 286102 (0.0009) [2023-12-26 17:26:47,223][105692] Updated weights for policy 0, policy_version 286112 (0.0006) [2023-12-26 17:26:47,433][105620] Updated weights for policy 1, policy_version 286193 (0.0008) [2023-12-26 17:26:47,490][105620] Updated weights for policy 1, policy_version 286203 (0.0006) [2023-12-26 17:26:47,549][105620] Updated weights for policy 1, policy_version 286213 (0.0006) [2023-12-26 17:26:47,899][105692] Updated weights for policy 0, policy_version 286122 (0.0008) [2023-12-26 17:26:47,965][105692] Updated weights for policy 0, policy_version 286132 (0.0005) [2023-12-26 17:26:48,029][105692] Updated weights for policy 0, policy_version 286142 (0.0010) [2023-12-26 17:26:48,086][105692] Updated weights for policy 0, policy_version 286152 (0.0008) [2023-12-26 17:26:48,311][105620] Updated weights for policy 1, policy_version 286223 (0.0010) [2023-12-26 17:26:48,374][105620] Updated weights for policy 1, policy_version 286233 (0.0011) [2023-12-26 17:26:48,434][105620] Updated weights for policy 1, policy_version 286243 (0.0010) [2023-12-26 17:26:48,736][105692] Updated weights for policy 0, policy_version 286162 (0.0006) [2023-12-26 17:26:48,803][105692] Updated weights for policy 0, policy_version 286172 (0.0006) [2023-12-26 17:26:48,874][105692] Updated weights for policy 0, policy_version 286182 (0.0007) [2023-12-26 17:26:49,137][105620] Updated weights for policy 1, policy_version 286253 (0.0010) [2023-12-26 17:26:49,185][105620] Updated weights for policy 1, policy_version 286263 (0.0010) [2023-12-26 17:26:49,240][105620] Updated weights for policy 1, policy_version 286273 (0.0010) [2023-12-26 17:26:49,536][105692] Updated weights for policy 0, policy_version 286192 (0.0009) [2023-12-26 17:26:49,604][105692] Updated weights for policy 0, policy_version 286202 (0.0008) [2023-12-26 17:26:49,657][105692] Updated weights for policy 0, policy_version 286212 (0.0008) [2023-12-26 17:26:50,014][105620] Updated weights for policy 1, policy_version 286283 (0.0009) [2023-12-26 17:26:50,067][105620] Updated weights for policy 1, policy_version 286293 (0.0009) [2023-12-26 17:26:50,132][105620] Updated weights for policy 1, policy_version 286303 (0.0009) [2023-12-26 17:26:50,321][105692] Updated weights for policy 0, policy_version 286222 (0.0009) [2023-12-26 17:26:50,370][105692] Updated weights for policy 0, policy_version 286232 (0.0009) [2023-12-26 17:26:50,426][105692] Updated weights for policy 0, policy_version 286242 (0.0010) [2023-12-26 17:26:50,879][105620] Updated weights for policy 1, policy_version 286313 (0.0009) [2023-12-26 17:26:50,937][105620] Updated weights for policy 1, policy_version 286323 (0.0008) [2023-12-26 17:26:50,998][105620] Updated weights for policy 1, policy_version 286333 (0.0008) [2023-12-26 17:26:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 146595840. Throughput: 0: 9881.2, 1: 9828.5. Samples: 146590936. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-12-26 17:26:51,062][104569] Avg episode reward: [(0, '9262.626'), (1, '7339.324')] [2023-12-26 17:26:51,066][105620] Updated weights for policy 1, policy_version 286343 (0.0008) [2023-12-26 17:26:51,171][105692] Updated weights for policy 0, policy_version 286252 (0.0009) [2023-12-26 17:26:51,224][105692] Updated weights for policy 0, policy_version 286262 (0.0009) [2023-12-26 17:26:51,314][105692] Updated weights for policy 0, policy_version 286272 (0.0009) [2023-12-26 17:26:51,814][105620] Updated weights for policy 1, policy_version 286353 (0.0007) [2023-12-26 17:26:51,880][105620] Updated weights for policy 1, policy_version 286363 (0.0006) [2023-12-26 17:26:51,935][105620] Updated weights for policy 1, policy_version 286373 (0.0005) [2023-12-26 17:26:52,118][105692] Updated weights for policy 0, policy_version 286284 (0.0009) [2023-12-26 17:26:52,179][105692] Updated weights for policy 0, policy_version 286294 (0.0010) [2023-12-26 17:26:52,237][105692] Updated weights for policy 0, policy_version 286304 (0.0010) [2023-12-26 17:26:52,531][105620] Updated weights for policy 1, policy_version 286383 (0.0006) [2023-12-26 17:26:52,596][105620] Updated weights for policy 1, policy_version 286393 (0.0006) [2023-12-26 17:26:52,657][105620] Updated weights for policy 1, policy_version 286403 (0.0008) [2023-12-26 17:26:52,945][105692] Updated weights for policy 0, policy_version 286314 (0.0009) [2023-12-26 17:26:53,001][105692] Updated weights for policy 0, policy_version 286324 (0.0009) [2023-12-26 17:26:53,058][105692] Updated weights for policy 0, policy_version 286334 (0.0010) [2023-12-26 17:26:53,111][105692] Updated weights for policy 0, policy_version 286344 (0.0009) [2023-12-26 17:26:53,380][105620] Updated weights for policy 1, policy_version 286413 (0.0008) [2023-12-26 17:26:53,431][105620] Updated weights for policy 1, policy_version 286423 (0.0005) [2023-12-26 17:26:53,482][105620] Updated weights for policy 1, policy_version 286433 (0.0005) [2023-12-26 17:26:53,951][105692] Updated weights for policy 0, policy_version 286354 (0.0008) [2023-12-26 17:26:54,010][105692] Updated weights for policy 0, policy_version 286364 (0.0010) [2023-12-26 17:26:54,068][105692] Updated weights for policy 0, policy_version 286374 (0.0008) [2023-12-26 17:26:54,078][105620] Updated weights for policy 1, policy_version 286443 (0.0005) [2023-12-26 17:26:54,137][105620] Updated weights for policy 1, policy_version 286453 (0.0008) [2023-12-26 17:26:54,202][105620] Updated weights for policy 1, policy_version 286463 (0.0009) [2023-12-26 17:26:54,772][105692] Updated weights for policy 0, policy_version 286384 (0.0011) [2023-12-26 17:26:54,820][105692] Updated weights for policy 0, policy_version 286394 (0.0009) [2023-12-26 17:26:54,872][105692] Updated weights for policy 0, policy_version 286404 (0.0010) [2023-12-26 17:26:54,943][105620] Updated weights for policy 1, policy_version 286473 (0.0010) [2023-12-26 17:26:55,011][105620] Updated weights for policy 1, policy_version 286483 (0.0008) [2023-12-26 17:26:55,066][105620] Updated weights for policy 1, policy_version 286493 (0.0008) [2023-12-26 17:26:55,130][105620] Updated weights for policy 1, policy_version 286503 (0.0009) [2023-12-26 17:26:55,495][105692] Updated weights for policy 0, policy_version 286414 (0.0009) [2023-12-26 17:26:55,556][105692] Updated weights for policy 0, policy_version 286424 (0.0008) [2023-12-26 17:26:55,619][105692] Updated weights for policy 0, policy_version 286434 (0.0006) [2023-12-26 17:26:55,928][105620] Updated weights for policy 1, policy_version 286513 (0.0008) [2023-12-26 17:26:55,983][105620] Updated weights for policy 1, policy_version 286523 (0.0008) [2023-12-26 17:26:55,986][105586] KL-divergence is very high: 110.4419 [2023-12-26 17:26:56,034][105620] Updated weights for policy 1, policy_version 286533 (0.0008) [2023-12-26 17:26:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 146702336. Throughput: 0: 9789.0, 1: 9898.4. Samples: 146706980. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-12-26 17:26:56,063][104569] Avg episode reward: [(0, '5026.798'), (1, '7007.712')] [2023-12-26 17:26:56,260][105692] Updated weights for policy 0, policy_version 286444 (0.0007) [2023-12-26 17:26:56,329][105692] Updated weights for policy 0, policy_version 286454 (0.0011) [2023-12-26 17:26:56,391][105692] Updated weights for policy 0, policy_version 286464 (0.0010) [2023-12-26 17:26:56,654][105586] KL-divergence is very high: 181.9927 [2023-12-26 17:26:56,675][105586] KL-divergence is very high: 126.2236 [2023-12-26 17:26:56,705][105620] Updated weights for policy 1, policy_version 286543 (0.0006) [2023-12-26 17:26:56,713][105586] KL-divergence is very high: 264.4253 [2023-12-26 17:26:56,733][105586] KL-divergence is very high: 103.4275 [2023-12-26 17:26:56,766][105586] KL-divergence is very high: 143.8736 [2023-12-26 17:26:56,772][105620] Updated weights for policy 1, policy_version 286553 (0.0005) [2023-12-26 17:26:56,826][105620] Updated weights for policy 1, policy_version 286563 (0.0005) [2023-12-26 17:26:57,113][105692] Updated weights for policy 0, policy_version 286474 (0.0010) [2023-12-26 17:26:57,165][105692] Updated weights for policy 0, policy_version 286484 (0.0010) [2023-12-26 17:26:57,222][105692] Updated weights for policy 0, policy_version 286494 (0.0010) [2023-12-26 17:26:57,262][105585] KL-divergence is very high: 109.1598 [2023-12-26 17:26:57,266][105692] Updated weights for policy 0, policy_version 286504 (0.0010) [2023-12-26 17:26:57,322][105620] Updated weights for policy 1, policy_version 286573 (0.0006) [2023-12-26 17:26:57,377][105620] Updated weights for policy 1, policy_version 286583 (0.0008) [2023-12-26 17:26:57,427][105620] Updated weights for policy 1, policy_version 286593 (0.0006) [2023-12-26 17:26:58,026][105692] Updated weights for policy 0, policy_version 286514 (0.0010) [2023-12-26 17:26:58,064][105620] Updated weights for policy 1, policy_version 286603 (0.0008) [2023-12-26 17:26:58,072][105692] Updated weights for policy 0, policy_version 286524 (0.0006) [2023-12-26 17:26:58,127][105620] Updated weights for policy 1, policy_version 286613 (0.0008) [2023-12-26 17:26:58,140][105692] Updated weights for policy 0, policy_version 286534 (0.0009) [2023-12-26 17:26:58,190][105620] Updated weights for policy 1, policy_version 286623 (0.0008) [2023-12-26 17:26:58,959][105620] Updated weights for policy 1, policy_version 286633 (0.0008) [2023-12-26 17:26:58,960][105586] KL-divergence is very high: 139.5483 [2023-12-26 17:26:58,966][105586] KL-divergence is very high: 185.7043 [2023-12-26 17:26:58,984][105586] KL-divergence is very high: 166.8426 [2023-12-26 17:26:58,987][105692] Updated weights for policy 0, policy_version 286544 (0.0010) [2023-12-26 17:26:59,010][105586] KL-divergence is very high: 268.9738 [2023-12-26 17:26:59,017][105586] KL-divergence is very high: 248.4205 [2023-12-26 17:26:59,024][105620] Updated weights for policy 1, policy_version 286643 (0.0008) [2023-12-26 17:26:59,045][105692] Updated weights for policy 0, policy_version 286554 (0.0008) [2023-12-26 17:26:59,085][105620] Updated weights for policy 1, policy_version 286653 (0.0007) [2023-12-26 17:26:59,112][105692] Updated weights for policy 0, policy_version 286564 (0.0006) [2023-12-26 17:26:59,150][105620] Updated weights for policy 1, policy_version 286663 (0.0008) [2023-12-26 17:26:59,927][105620] Updated weights for policy 1, policy_version 286673 (0.0007) [2023-12-26 17:26:59,947][105692] Updated weights for policy 0, policy_version 286574 (0.0008) [2023-12-26 17:26:59,981][105620] Updated weights for policy 1, policy_version 286683 (0.0005) [2023-12-26 17:27:00,012][105692] Updated weights for policy 0, policy_version 286584 (0.0008) [2023-12-26 17:27:00,032][105620] Updated weights for policy 1, policy_version 286693 (0.0006) [2023-12-26 17:27:00,071][105692] Updated weights for policy 0, policy_version 286594 (0.0008) [2023-12-26 17:27:00,700][105692] Updated weights for policy 0, policy_version 286604 (0.0010) [2023-12-26 17:27:00,752][105620] Updated weights for policy 1, policy_version 286703 (0.0008) [2023-12-26 17:27:00,755][105692] Updated weights for policy 0, policy_version 286614 (0.0010) [2023-12-26 17:27:00,798][105620] Updated weights for policy 1, policy_version 286713 (0.0007) [2023-12-26 17:27:00,800][105692] Updated weights for policy 0, policy_version 286624 (0.0010) [2023-12-26 17:27:00,847][105620] Updated weights for policy 1, policy_version 286723 (0.0007) [2023-12-26 17:27:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 146800640. Throughput: 0: 9864.6, 1: 9932.9. Samples: 146767808. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-12-26 17:27:01,062][104569] Avg episode reward: [(0, '616.004'), (1, '1888.463')] [2023-12-26 17:27:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000286632_73392128.pth... [2023-12-26 17:27:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000286728_73408512.pth... [2023-12-26 17:27:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000285480_73097216.pth [2023-12-26 17:27:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000285544_73105408.pth [2023-12-26 17:27:01,541][105692] Updated weights for policy 0, policy_version 286634 (0.0010) [2023-12-26 17:27:01,551][105620] Updated weights for policy 1, policy_version 286733 (0.0007) [2023-12-26 17:27:01,603][105620] Updated weights for policy 1, policy_version 286743 (0.0006) [2023-12-26 17:27:01,605][105692] Updated weights for policy 0, policy_version 286644 (0.0008) [2023-12-26 17:27:01,669][105620] Updated weights for policy 1, policy_version 286753 (0.0008) [2023-12-26 17:27:01,673][105692] Updated weights for policy 0, policy_version 286654 (0.0008) [2023-12-26 17:27:01,742][105692] Updated weights for policy 0, policy_version 286664 (0.0008) [2023-12-26 17:27:02,300][105620] Updated weights for policy 1, policy_version 286763 (0.0009) [2023-12-26 17:27:02,361][105620] Updated weights for policy 1, policy_version 286773 (0.0008) [2023-12-26 17:27:02,393][105692] Updated weights for policy 0, policy_version 286674 (0.0008) [2023-12-26 17:27:02,415][105620] Updated weights for policy 1, policy_version 286783 (0.0007) [2023-12-26 17:27:02,449][105692] Updated weights for policy 0, policy_version 286684 (0.0007) [2023-12-26 17:27:02,502][105692] Updated weights for policy 0, policy_version 286694 (0.0009) [2023-12-26 17:27:03,122][105620] Updated weights for policy 1, policy_version 286793 (0.0006) [2023-12-26 17:27:03,179][105620] Updated weights for policy 1, policy_version 286803 (0.0006) [2023-12-26 17:27:03,230][105620] Updated weights for policy 1, policy_version 286813 (0.0006) [2023-12-26 17:27:03,237][105692] Updated weights for policy 0, policy_version 286704 (0.0006) [2023-12-26 17:27:03,282][105620] Updated weights for policy 1, policy_version 286823 (0.0006) [2023-12-26 17:27:03,300][105692] Updated weights for policy 0, policy_version 286714 (0.0009) [2023-12-26 17:27:03,370][105692] Updated weights for policy 0, policy_version 286724 (0.0010) [2023-12-26 17:27:03,865][105620] Updated weights for policy 1, policy_version 286833 (0.0010) [2023-12-26 17:27:03,934][105620] Updated weights for policy 1, policy_version 286843 (0.0010) [2023-12-26 17:27:03,986][105620] Updated weights for policy 1, policy_version 286853 (0.0011) [2023-12-26 17:27:04,130][105692] Updated weights for policy 0, policy_version 286734 (0.0010) [2023-12-26 17:27:04,186][105692] Updated weights for policy 0, policy_version 286744 (0.0011) [2023-12-26 17:27:04,250][105692] Updated weights for policy 0, policy_version 286754 (0.0010) [2023-12-26 17:27:04,732][105620] Updated weights for policy 1, policy_version 286863 (0.0011) [2023-12-26 17:27:04,786][105620] Updated weights for policy 1, policy_version 286873 (0.0008) [2023-12-26 17:27:04,854][105620] Updated weights for policy 1, policy_version 286883 (0.0010) [2023-12-26 17:27:04,967][105692] Updated weights for policy 0, policy_version 286764 (0.0007) [2023-12-26 17:27:05,030][105692] Updated weights for policy 0, policy_version 286774 (0.0009) [2023-12-26 17:27:05,089][105692] Updated weights for policy 0, policy_version 286784 (0.0008) [2023-12-26 17:27:05,590][105620] Updated weights for policy 1, policy_version 286893 (0.0010) [2023-12-26 17:27:05,635][105692] Updated weights for policy 0, policy_version 286794 (0.0006) [2023-12-26 17:27:05,637][105620] Updated weights for policy 1, policy_version 286903 (0.0010) [2023-12-26 17:27:05,696][105620] Updated weights for policy 1, policy_version 286913 (0.0010) [2023-12-26 17:27:05,703][105692] Updated weights for policy 0, policy_version 286804 (0.0005) [2023-12-26 17:27:05,761][105692] Updated weights for policy 0, policy_version 286814 (0.0006) [2023-12-26 17:27:05,812][105692] Updated weights for policy 0, policy_version 286824 (0.0009) [2023-12-26 17:27:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 146898944. Throughput: 0: 9736.8, 1: 9920.0. Samples: 146884324. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-12-26 17:27:06,062][104569] Avg episode reward: [(0, '623.341'), (1, '6035.437')] [2023-12-26 17:27:06,424][105692] Updated weights for policy 0, policy_version 286834 (0.0007) [2023-12-26 17:27:06,439][105620] Updated weights for policy 1, policy_version 286923 (0.0010) [2023-12-26 17:27:06,485][105692] Updated weights for policy 0, policy_version 286844 (0.0007) [2023-12-26 17:27:06,496][105620] Updated weights for policy 1, policy_version 286933 (0.0010) [2023-12-26 17:27:06,543][105692] Updated weights for policy 0, policy_version 286854 (0.0007) [2023-12-26 17:27:06,552][105620] Updated weights for policy 1, policy_version 286943 (0.0010) [2023-12-26 17:27:07,233][105692] Updated weights for policy 0, policy_version 286864 (0.0007) [2023-12-26 17:27:07,293][105692] Updated weights for policy 0, policy_version 286874 (0.0009) [2023-12-26 17:27:07,311][105620] Updated weights for policy 1, policy_version 286953 (0.0010) [2023-12-26 17:27:07,343][105692] Updated weights for policy 0, policy_version 286884 (0.0009) [2023-12-26 17:27:07,363][105620] Updated weights for policy 1, policy_version 286963 (0.0010) [2023-12-26 17:27:07,421][105620] Updated weights for policy 1, policy_version 286973 (0.0010) [2023-12-26 17:27:07,472][105620] Updated weights for policy 1, policy_version 286983 (0.0010) [2023-12-26 17:27:08,104][105692] Updated weights for policy 0, policy_version 286895 (0.0009) [2023-12-26 17:27:08,134][105586] KL-divergence is very high: 180.0372 [2023-12-26 17:27:08,156][105692] Updated weights for policy 0, policy_version 286905 (0.0007) [2023-12-26 17:27:08,166][105620] Updated weights for policy 1, policy_version 286993 (0.0007) [2023-12-26 17:27:08,175][105586] KL-divergence is very high: 136.8212 [2023-12-26 17:27:08,210][105692] Updated weights for policy 0, policy_version 286915 (0.0006) [2023-12-26 17:27:08,223][105620] Updated weights for policy 1, policy_version 287003 (0.0008) [2023-12-26 17:27:08,274][105620] Updated weights for policy 1, policy_version 287013 (0.0008) [2023-12-26 17:27:08,934][105692] Updated weights for policy 0, policy_version 286925 (0.0008) [2023-12-26 17:27:08,995][105692] Updated weights for policy 0, policy_version 286935 (0.0009) [2023-12-26 17:27:09,036][105620] Updated weights for policy 1, policy_version 287023 (0.0010) [2023-12-26 17:27:09,053][105692] Updated weights for policy 0, policy_version 286945 (0.0008) [2023-12-26 17:27:09,088][105586] KL-divergence is very high: 101.5616 [2023-12-26 17:27:09,095][105620] Updated weights for policy 1, policy_version 287033 (0.0008) [2023-12-26 17:27:09,152][105620] Updated weights for policy 1, policy_version 287043 (0.0009) [2023-12-26 17:27:09,753][105692] Updated weights for policy 0, policy_version 286955 (0.0007) [2023-12-26 17:27:09,812][105692] Updated weights for policy 0, policy_version 286965 (0.0009) [2023-12-26 17:27:09,876][105692] Updated weights for policy 0, policy_version 286975 (0.0008) [2023-12-26 17:27:09,962][105620] Updated weights for policy 1, policy_version 287053 (0.0008) [2023-12-26 17:27:10,019][105620] Updated weights for policy 1, policy_version 287063 (0.0007) [2023-12-26 17:27:10,068][105620] Updated weights for policy 1, policy_version 287073 (0.0009) [2023-12-26 17:27:10,569][105692] Updated weights for policy 0, policy_version 286985 (0.0007) [2023-12-26 17:27:10,620][105692] Updated weights for policy 0, policy_version 286995 (0.0007) [2023-12-26 17:27:10,668][105692] Updated weights for policy 0, policy_version 287005 (0.0008) [2023-12-26 17:27:10,717][105692] Updated weights for policy 0, policy_version 287015 (0.0008) [2023-12-26 17:27:10,779][105620] Updated weights for policy 1, policy_version 287083 (0.0008) [2023-12-26 17:27:10,830][105620] Updated weights for policy 1, policy_version 287093 (0.0005) [2023-12-26 17:27:10,890][105620] Updated weights for policy 1, policy_version 287103 (0.0006) [2023-12-26 17:27:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 146997248. Throughput: 0: 9853.5, 1: 9878.7. Samples: 147002580. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-12-26 17:27:11,062][104569] Avg episode reward: [(0, '612.820'), (1, '2187.275')] [2023-12-26 17:27:11,479][105692] Updated weights for policy 0, policy_version 287025 (0.0009) [2023-12-26 17:27:11,534][105692] Updated weights for policy 0, policy_version 287035 (0.0006) [2023-12-26 17:27:11,588][105620] Updated weights for policy 1, policy_version 287113 (0.0010) [2023-12-26 17:27:11,591][105692] Updated weights for policy 0, policy_version 287045 (0.0008) [2023-12-26 17:27:11,655][105620] Updated weights for policy 1, policy_version 287123 (0.0009) [2023-12-26 17:27:11,719][105620] Updated weights for policy 1, policy_version 287133 (0.0008) [2023-12-26 17:27:11,788][105620] Updated weights for policy 1, policy_version 287143 (0.0008) [2023-12-26 17:27:12,371][105692] Updated weights for policy 0, policy_version 287055 (0.0007) [2023-12-26 17:27:12,436][105692] Updated weights for policy 0, policy_version 287065 (0.0006) [2023-12-26 17:27:12,496][105692] Updated weights for policy 0, policy_version 287075 (0.0007) [2023-12-26 17:27:12,541][105620] Updated weights for policy 1, policy_version 287153 (0.0008) [2023-12-26 17:27:12,604][105620] Updated weights for policy 1, policy_version 287163 (0.0009) [2023-12-26 17:27:12,659][105620] Updated weights for policy 1, policy_version 287173 (0.0009) [2023-12-26 17:27:13,197][105692] Updated weights for policy 0, policy_version 287085 (0.0009) [2023-12-26 17:27:13,256][105692] Updated weights for policy 0, policy_version 287095 (0.0010) [2023-12-26 17:27:13,315][105692] Updated weights for policy 0, policy_version 287105 (0.0010) [2023-12-26 17:27:13,408][105620] Updated weights for policy 1, policy_version 287183 (0.0008) [2023-12-26 17:27:13,455][105620] Updated weights for policy 1, policy_version 287193 (0.0009) [2023-12-26 17:27:13,516][105620] Updated weights for policy 1, policy_version 287203 (0.0010) [2023-12-26 17:27:13,972][105692] Updated weights for policy 0, policy_version 287115 (0.0009) [2023-12-26 17:27:14,027][105692] Updated weights for policy 0, policy_version 287125 (0.0006) [2023-12-26 17:27:14,081][105620] Updated weights for policy 1, policy_version 287213 (0.0009) [2023-12-26 17:27:14,090][105692] Updated weights for policy 0, policy_version 287135 (0.0008) [2023-12-26 17:27:14,141][105620] Updated weights for policy 1, policy_version 287223 (0.0006) [2023-12-26 17:27:14,186][105586] KL-divergence is very high: 142.3525 [2023-12-26 17:27:14,195][105620] Updated weights for policy 1, policy_version 287233 (0.0005) [2023-12-26 17:27:14,231][105586] KL-divergence is very high: 117.6254 [2023-12-26 17:27:14,776][105692] Updated weights for policy 0, policy_version 287145 (0.0008) [2023-12-26 17:27:14,777][105620] Updated weights for policy 1, policy_version 287243 (0.0008) [2023-12-26 17:27:14,843][105620] Updated weights for policy 1, policy_version 287253 (0.0006) [2023-12-26 17:27:14,843][105692] Updated weights for policy 0, policy_version 287155 (0.0011) [2023-12-26 17:27:14,904][105620] Updated weights for policy 1, policy_version 287263 (0.0010) [2023-12-26 17:27:14,904][105692] Updated weights for policy 0, policy_version 287165 (0.0011) [2023-12-26 17:27:14,968][105692] Updated weights for policy 0, policy_version 287175 (0.0007) [2023-12-26 17:27:15,644][105620] Updated weights for policy 1, policy_version 287273 (0.0010) [2023-12-26 17:27:15,693][105620] Updated weights for policy 1, policy_version 287283 (0.0007) [2023-12-26 17:27:15,699][105692] Updated weights for policy 0, policy_version 287185 (0.0008) [2023-12-26 17:27:15,747][105692] Updated weights for policy 0, policy_version 287195 (0.0006) [2023-12-26 17:27:15,750][105620] Updated weights for policy 1, policy_version 287293 (0.0008) [2023-12-26 17:27:15,803][105620] Updated weights for policy 1, policy_version 287303 (0.0008) [2023-12-26 17:27:15,807][105692] Updated weights for policy 0, policy_version 287205 (0.0006) [2023-12-26 17:27:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.4, 300 sec: 19522.0). Total num frames: 147095552. Throughput: 0: 9807.2, 1: 9903.0. Samples: 147059776. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-12-26 17:27:16,063][104569] Avg episode reward: [(0, '612.984'), (1, '5845.451')] [2023-12-26 17:27:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000287208_73539584.pth... [2023-12-26 17:27:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000287304_73555968.pth... [2023-12-26 17:27:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000286056_73244672.pth [2023-12-26 17:27:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000286152_73261056.pth [2023-12-26 17:27:16,485][105692] Updated weights for policy 0, policy_version 287215 (0.0008) [2023-12-26 17:27:16,538][105692] Updated weights for policy 0, policy_version 287225 (0.0009) [2023-12-26 17:27:16,596][105620] Updated weights for policy 1, policy_version 287313 (0.0009) [2023-12-26 17:27:16,597][105692] Updated weights for policy 0, policy_version 287235 (0.0006) [2023-12-26 17:27:16,654][105620] Updated weights for policy 1, policy_version 287323 (0.0009) [2023-12-26 17:27:16,704][105620] Updated weights for policy 1, policy_version 287333 (0.0009) [2023-12-26 17:27:17,305][105692] Updated weights for policy 0, policy_version 287245 (0.0006) [2023-12-26 17:27:17,322][105620] Updated weights for policy 1, policy_version 287343 (0.0010) [2023-12-26 17:27:17,349][105692] Updated weights for policy 0, policy_version 287255 (0.0005) [2023-12-26 17:27:17,389][105620] Updated weights for policy 1, policy_version 287353 (0.0008) [2023-12-26 17:27:17,400][105692] Updated weights for policy 0, policy_version 287265 (0.0006) [2023-12-26 17:27:17,458][105620] Updated weights for policy 1, policy_version 287363 (0.0010) [2023-12-26 17:27:18,061][105692] Updated weights for policy 0, policy_version 287275 (0.0005) [2023-12-26 17:27:18,121][105692] Updated weights for policy 0, policy_version 287285 (0.0005) [2023-12-26 17:27:18,184][105692] Updated weights for policy 0, policy_version 287295 (0.0005) [2023-12-26 17:27:18,208][105620] Updated weights for policy 1, policy_version 287373 (0.0010) [2023-12-26 17:27:18,273][105620] Updated weights for policy 1, policy_version 287383 (0.0010) [2023-12-26 17:27:18,338][105620] Updated weights for policy 1, policy_version 287393 (0.0008) [2023-12-26 17:27:18,799][105692] Updated weights for policy 0, policy_version 287305 (0.0006) [2023-12-26 17:27:18,853][105692] Updated weights for policy 0, policy_version 287315 (0.0010) [2023-12-26 17:27:18,922][105692] Updated weights for policy 0, policy_version 287325 (0.0009) [2023-12-26 17:27:18,922][105620] Updated weights for policy 1, policy_version 287403 (0.0007) [2023-12-26 17:27:18,978][105620] Updated weights for policy 1, policy_version 287413 (0.0010) [2023-12-26 17:27:18,979][105692] Updated weights for policy 0, policy_version 287335 (0.0010) [2023-12-26 17:27:19,036][105620] Updated weights for policy 1, policy_version 287423 (0.0010) [2023-12-26 17:27:19,722][105620] Updated weights for policy 1, policy_version 287433 (0.0010) [2023-12-26 17:27:19,727][105692] Updated weights for policy 0, policy_version 287345 (0.0008) [2023-12-26 17:27:19,782][105692] Updated weights for policy 0, policy_version 287355 (0.0008) [2023-12-26 17:27:19,789][105620] Updated weights for policy 1, policy_version 287443 (0.0008) [2023-12-26 17:27:19,854][105692] Updated weights for policy 0, policy_version 287365 (0.0007) [2023-12-26 17:27:19,860][105620] Updated weights for policy 1, policy_version 287453 (0.0009) [2023-12-26 17:27:19,922][105620] Updated weights for policy 1, policy_version 287463 (0.0009) [2023-12-26 17:27:20,622][105692] Updated weights for policy 0, policy_version 287375 (0.0008) [2023-12-26 17:27:20,658][105620] Updated weights for policy 1, policy_version 287473 (0.0008) [2023-12-26 17:27:20,692][105692] Updated weights for policy 0, policy_version 287385 (0.0006) [2023-12-26 17:27:20,718][105620] Updated weights for policy 1, policy_version 287483 (0.0008) [2023-12-26 17:27:20,758][105692] Updated weights for policy 0, policy_version 287395 (0.0007) [2023-12-26 17:27:20,791][105620] Updated weights for policy 1, policy_version 287493 (0.0006) [2023-12-26 17:27:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 147193856. Throughput: 0: 9827.0, 1: 9911.7. Samples: 147181452. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-12-26 17:27:21,062][104569] Avg episode reward: [(0, '666.143'), (1, '7320.317')] [2023-12-26 17:27:21,467][105692] Updated weights for policy 0, policy_version 287405 (0.0009) [2023-12-26 17:27:21,524][105620] Updated weights for policy 1, policy_version 287503 (0.0007) [2023-12-26 17:27:21,529][105692] Updated weights for policy 0, policy_version 287415 (0.0007) [2023-12-26 17:27:21,549][105586] KL-divergence is very high: 176.4519 [2023-12-26 17:27:21,580][105586] KL-divergence is very high: 125.2244 [2023-12-26 17:27:21,586][105620] Updated weights for policy 1, policy_version 287513 (0.0010) [2023-12-26 17:27:21,588][105692] Updated weights for policy 0, policy_version 287425 (0.0008) [2023-12-26 17:27:21,599][105586] KL-divergence is very high: 178.2588 [2023-12-26 17:27:21,632][105586] KL-divergence is very high: 100.1309 [2023-12-26 17:27:21,649][105620] Updated weights for policy 1, policy_version 287523 (0.0008) [2023-12-26 17:27:21,652][105586] KL-divergence is very high: 130.0769 [2023-12-26 17:27:22,368][105692] Updated weights for policy 0, policy_version 287435 (0.0008) [2023-12-26 17:27:22,382][105620] Updated weights for policy 1, policy_version 287533 (0.0007) [2023-12-26 17:27:22,426][105692] Updated weights for policy 0, policy_version 287445 (0.0006) [2023-12-26 17:27:22,436][105620] Updated weights for policy 1, policy_version 287543 (0.0009) [2023-12-26 17:27:22,488][105692] Updated weights for policy 0, policy_version 287455 (0.0011) [2023-12-26 17:27:22,495][105620] Updated weights for policy 1, policy_version 287553 (0.0006) [2023-12-26 17:27:23,187][105692] Updated weights for policy 0, policy_version 287465 (0.0011) [2023-12-26 17:27:23,235][105692] Updated weights for policy 0, policy_version 287475 (0.0010) [2023-12-26 17:27:23,291][105620] Updated weights for policy 1, policy_version 287563 (0.0007) [2023-12-26 17:27:23,294][105692] Updated weights for policy 0, policy_version 287485 (0.0011) [2023-12-26 17:27:23,341][105620] Updated weights for policy 1, policy_version 287573 (0.0007) [2023-12-26 17:27:23,354][105692] Updated weights for policy 0, policy_version 287495 (0.0011) [2023-12-26 17:27:23,392][105620] Updated weights for policy 1, policy_version 287583 (0.0008) [2023-12-26 17:27:23,968][105692] Updated weights for policy 0, policy_version 287505 (0.0006) [2023-12-26 17:27:24,019][105692] Updated weights for policy 0, policy_version 287515 (0.0005) [2023-12-26 17:27:24,076][105692] Updated weights for policy 0, policy_version 287525 (0.0005) [2023-12-26 17:27:24,218][105620] Updated weights for policy 1, policy_version 287593 (0.0009) [2023-12-26 17:27:24,275][105620] Updated weights for policy 1, policy_version 287603 (0.0010) [2023-12-26 17:27:24,333][105620] Updated weights for policy 1, policy_version 287614 (0.0010) [2023-12-26 17:27:24,389][105620] Updated weights for policy 1, policy_version 287624 (0.0009) [2023-12-26 17:27:24,610][105692] Updated weights for policy 0, policy_version 287535 (0.0009) [2023-12-26 17:27:24,674][105692] Updated weights for policy 0, policy_version 287545 (0.0010) [2023-12-26 17:27:24,739][105692] Updated weights for policy 0, policy_version 287555 (0.0010) [2023-12-26 17:27:25,183][105620] Updated weights for policy 1, policy_version 287634 (0.0009) [2023-12-26 17:27:25,238][105620] Updated weights for policy 1, policy_version 287644 (0.0008) [2023-12-26 17:27:25,295][105620] Updated weights for policy 1, policy_version 287654 (0.0009) [2023-12-26 17:27:25,382][105692] Updated weights for policy 0, policy_version 287565 (0.0008) [2023-12-26 17:27:25,441][105692] Updated weights for policy 0, policy_version 287575 (0.0006) [2023-12-26 17:27:25,504][105692] Updated weights for policy 0, policy_version 287585 (0.0005) [2023-12-26 17:27:26,003][105692] Updated weights for policy 0, policy_version 287595 (0.0006) [2023-12-26 17:27:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 147283968. Throughput: 0: 9945.6, 1: 9711.2. Samples: 147296176. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-12-26 17:27:26,062][104569] Avg episode reward: [(0, '741.772'), (1, '7598.916')] [2023-12-26 17:27:26,071][105692] Updated weights for policy 0, policy_version 287605 (0.0006) [2023-12-26 17:27:26,134][105692] Updated weights for policy 0, policy_version 287615 (0.0008) [2023-12-26 17:27:26,153][105620] Updated weights for policy 1, policy_version 287664 (0.0009) [2023-12-26 17:27:26,205][105620] Updated weights for policy 1, policy_version 287674 (0.0010) [2023-12-26 17:27:26,253][105620] Updated weights for policy 1, policy_version 287684 (0.0010) [2023-12-26 17:27:26,792][105692] Updated weights for policy 0, policy_version 287625 (0.0008) [2023-12-26 17:27:26,853][105692] Updated weights for policy 0, policy_version 287635 (0.0010) [2023-12-26 17:27:26,898][105692] Updated weights for policy 0, policy_version 287645 (0.0010) [2023-12-26 17:27:26,900][105620] Updated weights for policy 1, policy_version 287694 (0.0007) [2023-12-26 17:27:26,942][105692] Updated weights for policy 0, policy_version 287655 (0.0010) [2023-12-26 17:27:26,957][105620] Updated weights for policy 1, policy_version 287704 (0.0005) [2023-12-26 17:27:27,019][105620] Updated weights for policy 1, policy_version 287714 (0.0005) [2023-12-26 17:27:27,559][105692] Updated weights for policy 0, policy_version 287665 (0.0006) [2023-12-26 17:27:27,608][105620] Updated weights for policy 1, policy_version 287724 (0.0007) [2023-12-26 17:27:27,620][105692] Updated weights for policy 0, policy_version 287675 (0.0007) [2023-12-26 17:27:27,653][105620] Updated weights for policy 1, policy_version 287734 (0.0010) [2023-12-26 17:27:27,653][105586] KL-divergence is very high: 180.6936 [2023-12-26 17:27:27,658][105586] KL-divergence is very high: 269.8045 [2023-12-26 17:27:27,662][105586] KL-divergence is very high: 334.8488 [2023-12-26 17:27:27,670][105692] Updated weights for policy 0, policy_version 287685 (0.0010) [2023-12-26 17:27:27,676][105586] KL-divergence is very high: 326.5605 [2023-12-26 17:27:27,693][105586] KL-divergence is very high: 313.7018 [2023-12-26 17:27:27,698][105586] KL-divergence is very high: 410.7176 [2023-12-26 17:27:27,703][105586] KL-divergence is very high: 422.7774 [2023-12-26 17:27:27,705][105620] Updated weights for policy 1, policy_version 287744 (0.0010) [2023-12-26 17:27:27,718][105586] KL-divergence is very high: 341.9393 [2023-12-26 17:27:27,731][105586] KL-divergence is very high: 270.8728 [2023-12-26 17:27:27,737][105586] KL-divergence is very high: 365.0566 [2023-12-26 17:27:28,363][105692] Updated weights for policy 0, policy_version 287695 (0.0009) [2023-12-26 17:27:28,404][105620] Updated weights for policy 1, policy_version 287754 (0.0010) [2023-12-26 17:27:28,421][105692] Updated weights for policy 0, policy_version 287705 (0.0009) [2023-12-26 17:27:28,437][105586] KL-divergence is very high: 129.6193 [2023-12-26 17:27:28,448][105586] KL-divergence is very high: 142.2332 [2023-12-26 17:27:28,457][105620] Updated weights for policy 1, policy_version 287764 (0.0008) [2023-12-26 17:27:28,463][105586] KL-divergence is very high: 191.2301 [2023-12-26 17:27:28,481][105586] KL-divergence is very high: 197.1620 [2023-12-26 17:27:28,483][105692] Updated weights for policy 0, policy_version 287715 (0.0009) [2023-12-26 17:27:28,494][105586] KL-divergence is very high: 185.6718 [2023-12-26 17:27:28,513][105586] KL-divergence is very high: 219.3875 [2023-12-26 17:27:28,520][105620] Updated weights for policy 1, policy_version 287774 (0.0006) [2023-12-26 17:27:28,532][105586] KL-divergence is very high: 206.2782 [2023-12-26 17:27:28,545][105586] KL-divergence is very high: 176.4151 [2023-12-26 17:27:28,565][105586] KL-divergence is very high: 200.3040 [2023-12-26 17:27:28,584][105620] Updated weights for policy 1, policy_version 287784 (0.0008) [2023-12-26 17:27:29,078][105692] Updated weights for policy 0, policy_version 287725 (0.0007) [2023-12-26 17:27:29,121][105692] Updated weights for policy 0, policy_version 287735 (0.0005) [2023-12-26 17:27:29,166][105692] Updated weights for policy 0, policy_version 287745 (0.0005) [2023-12-26 17:27:29,290][105586] KL-divergence is very high: 116.4897 [2023-12-26 17:27:29,352][105586] KL-divergence is very high: 107.8291 [2023-12-26 17:27:29,366][105620] Updated weights for policy 1, policy_version 287794 (0.0009) [2023-12-26 17:27:29,392][105586] KL-divergence is very high: 100.1608 [2023-12-26 17:27:29,412][105620] Updated weights for policy 1, policy_version 287804 (0.0010) [2023-12-26 17:27:29,467][105620] Updated weights for policy 1, policy_version 287814 (0.0010) [2023-12-26 17:27:29,808][105692] Updated weights for policy 0, policy_version 287755 (0.0006) [2023-12-26 17:27:29,867][105692] Updated weights for policy 0, policy_version 287765 (0.0007) [2023-12-26 17:27:29,928][105692] Updated weights for policy 0, policy_version 287775 (0.0007) [2023-12-26 17:27:30,232][105620] Updated weights for policy 1, policy_version 287825 (0.0009) [2023-12-26 17:27:30,287][105620] Updated weights for policy 1, policy_version 287836 (0.0010) [2023-12-26 17:27:30,345][105620] Updated weights for policy 1, policy_version 287846 (0.0008) [2023-12-26 17:27:30,617][105692] Updated weights for policy 0, policy_version 287785 (0.0008) [2023-12-26 17:27:30,676][105692] Updated weights for policy 0, policy_version 287795 (0.0008) [2023-12-26 17:27:30,722][105692] Updated weights for policy 0, policy_version 287805 (0.0006) [2023-12-26 17:27:30,769][105692] Updated weights for policy 0, policy_version 287815 (0.0005) [2023-12-26 17:27:30,959][105620] Updated weights for policy 1, policy_version 287856 (0.0006) [2023-12-26 17:27:31,022][105620] Updated weights for policy 1, policy_version 287866 (0.0006) [2023-12-26 17:27:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 147390464. Throughput: 0: 10035.4, 1: 9758.2. Samples: 147359396. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-12-26 17:27:31,063][104569] Avg episode reward: [(0, '1155.315'), (1, '7410.207')] [2023-12-26 17:27:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000287816_73695232.pth... [2023-12-26 17:27:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000286632_73392128.pth [2023-12-26 17:27:31,099][105620] Updated weights for policy 1, policy_version 287876 (0.0011) [2023-12-26 17:27:31,125][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000287880_73703424.pth... [2023-12-26 17:27:31,130][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000286728_73408512.pth [2023-12-26 17:27:31,534][105692] Updated weights for policy 0, policy_version 287825 (0.0007) [2023-12-26 17:27:31,596][105692] Updated weights for policy 0, policy_version 287835 (0.0009) [2023-12-26 17:27:31,660][105692] Updated weights for policy 0, policy_version 287845 (0.0009) [2023-12-26 17:27:31,765][105620] Updated weights for policy 1, policy_version 287886 (0.0009) [2023-12-26 17:27:31,828][105620] Updated weights for policy 1, policy_version 287896 (0.0005) [2023-12-26 17:27:31,892][105620] Updated weights for policy 1, policy_version 287906 (0.0006) [2023-12-26 17:27:32,386][105692] Updated weights for policy 0, policy_version 287855 (0.0010) [2023-12-26 17:27:32,443][105692] Updated weights for policy 0, policy_version 287865 (0.0011) [2023-12-26 17:27:32,497][105692] Updated weights for policy 0, policy_version 287875 (0.0010) [2023-12-26 17:27:32,608][105620] Updated weights for policy 1, policy_version 287917 (0.0009) [2023-12-26 17:27:32,667][105620] Updated weights for policy 1, policy_version 287927 (0.0005) [2023-12-26 17:27:32,724][105620] Updated weights for policy 1, policy_version 287937 (0.0006) [2023-12-26 17:27:33,152][105692] Updated weights for policy 0, policy_version 287885 (0.0008) [2023-12-26 17:27:33,202][105692] Updated weights for policy 0, policy_version 287895 (0.0005) [2023-12-26 17:27:33,254][105692] Updated weights for policy 0, policy_version 287905 (0.0005) [2023-12-26 17:27:33,400][105620] Updated weights for policy 1, policy_version 287947 (0.0009) [2023-12-26 17:27:33,462][105620] Updated weights for policy 1, policy_version 287957 (0.0010) [2023-12-26 17:27:33,529][105620] Updated weights for policy 1, policy_version 287967 (0.0010) [2023-12-26 17:27:33,790][105692] Updated weights for policy 0, policy_version 287915 (0.0006) [2023-12-26 17:27:33,837][105692] Updated weights for policy 0, policy_version 287925 (0.0008) [2023-12-26 17:27:33,881][105692] Updated weights for policy 0, policy_version 287935 (0.0008) [2023-12-26 17:27:34,147][105620] Updated weights for policy 1, policy_version 287977 (0.0010) [2023-12-26 17:27:34,203][105620] Updated weights for policy 1, policy_version 287987 (0.0007) [2023-12-26 17:27:34,250][105586] KL-divergence is very high: 123.4142 [2023-12-26 17:27:34,256][105620] Updated weights for policy 1, policy_version 287997 (0.0009) [2023-12-26 17:27:34,290][105586] KL-divergence is very high: 159.9554 [2023-12-26 17:27:34,295][105586] KL-divergence is very high: 251.4359 [2023-12-26 17:27:34,319][105620] Updated weights for policy 1, policy_version 288007 (0.0009) [2023-12-26 17:27:34,626][105692] Updated weights for policy 0, policy_version 287946 (0.0009) [2023-12-26 17:27:34,688][105692] Updated weights for policy 0, policy_version 287956 (0.0006) [2023-12-26 17:27:34,753][105692] Updated weights for policy 0, policy_version 287966 (0.0008) [2023-12-26 17:27:34,819][105692] Updated weights for policy 0, policy_version 287976 (0.0009) [2023-12-26 17:27:35,125][105620] Updated weights for policy 1, policy_version 288017 (0.0009) [2023-12-26 17:27:35,186][105620] Updated weights for policy 1, policy_version 288027 (0.0009) [2023-12-26 17:27:35,241][105620] Updated weights for policy 1, policy_version 288037 (0.0009) [2023-12-26 17:27:35,426][105692] Updated weights for policy 0, policy_version 287986 (0.0009) [2023-12-26 17:27:35,482][105692] Updated weights for policy 0, policy_version 287996 (0.0010) [2023-12-26 17:27:35,538][105692] Updated weights for policy 0, policy_version 288006 (0.0009) [2023-12-26 17:27:36,040][105620] Updated weights for policy 1, policy_version 288047 (0.0008) [2023-12-26 17:27:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 147488768. Throughput: 0: 9997.4, 1: 9801.7. Samples: 147481896. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-12-26 17:27:36,062][104569] Avg episode reward: [(0, '2824.356'), (1, '6859.984')] [2023-12-26 17:27:36,115][105620] Updated weights for policy 1, policy_version 288057 (0.0009) [2023-12-26 17:27:36,178][105620] Updated weights for policy 1, policy_version 288067 (0.0009) [2023-12-26 17:27:36,250][105692] Updated weights for policy 0, policy_version 288016 (0.0009) [2023-12-26 17:27:36,312][105692] Updated weights for policy 0, policy_version 288026 (0.0008) [2023-12-26 17:27:36,378][105692] Updated weights for policy 0, policy_version 288036 (0.0007) [2023-12-26 17:27:36,964][105692] Updated weights for policy 0, policy_version 288046 (0.0006) [2023-12-26 17:27:36,990][105620] Updated weights for policy 1, policy_version 288077 (0.0009) [2023-12-26 17:27:37,026][105692] Updated weights for policy 0, policy_version 288056 (0.0006) [2023-12-26 17:27:37,050][105620] Updated weights for policy 1, policy_version 288087 (0.0009) [2023-12-26 17:27:37,086][105692] Updated weights for policy 0, policy_version 288066 (0.0006) [2023-12-26 17:27:37,104][105620] Updated weights for policy 1, policy_version 288097 (0.0007) [2023-12-26 17:27:37,749][105692] Updated weights for policy 0, policy_version 288076 (0.0007) [2023-12-26 17:27:37,802][105692] Updated weights for policy 0, policy_version 288086 (0.0010) [2023-12-26 17:27:37,843][105620] Updated weights for policy 1, policy_version 288107 (0.0007) [2023-12-26 17:27:37,861][105692] Updated weights for policy 0, policy_version 288096 (0.0007) [2023-12-26 17:27:37,895][105620] Updated weights for policy 1, policy_version 288117 (0.0009) [2023-12-26 17:27:37,948][105620] Updated weights for policy 1, policy_version 288127 (0.0008) [2023-12-26 17:27:38,589][105692] Updated weights for policy 0, policy_version 288106 (0.0006) [2023-12-26 17:27:38,639][105692] Updated weights for policy 0, policy_version 288116 (0.0008) [2023-12-26 17:27:38,697][105692] Updated weights for policy 0, policy_version 288126 (0.0009) [2023-12-26 17:27:38,713][105620] Updated weights for policy 1, policy_version 288137 (0.0008) [2023-12-26 17:27:38,758][105692] Updated weights for policy 0, policy_version 288136 (0.0009) [2023-12-26 17:27:38,770][105620] Updated weights for policy 1, policy_version 288147 (0.0006) [2023-12-26 17:27:38,832][105620] Updated weights for policy 1, policy_version 288157 (0.0009) [2023-12-26 17:27:38,890][105620] Updated weights for policy 1, policy_version 288167 (0.0008) [2023-12-26 17:27:39,529][105692] Updated weights for policy 0, policy_version 288146 (0.0005) [2023-12-26 17:27:39,598][105692] Updated weights for policy 0, policy_version 288156 (0.0008) [2023-12-26 17:27:39,650][105620] Updated weights for policy 1, policy_version 288177 (0.0007) [2023-12-26 17:27:39,660][105692] Updated weights for policy 0, policy_version 288166 (0.0008) [2023-12-26 17:27:39,713][105620] Updated weights for policy 1, policy_version 288187 (0.0009) [2023-12-26 17:27:39,777][105620] Updated weights for policy 1, policy_version 288197 (0.0009) [2023-12-26 17:27:40,362][105692] Updated weights for policy 0, policy_version 288176 (0.0008) [2023-12-26 17:27:40,433][105692] Updated weights for policy 0, policy_version 288186 (0.0009) [2023-12-26 17:27:40,492][105692] Updated weights for policy 0, policy_version 288196 (0.0008) [2023-12-26 17:27:40,558][105620] Updated weights for policy 1, policy_version 288207 (0.0010) [2023-12-26 17:27:40,616][105620] Updated weights for policy 1, policy_version 288217 (0.0010) [2023-12-26 17:27:40,676][105620] Updated weights for policy 1, policy_version 288227 (0.0009) [2023-12-26 17:27:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 147587072. Throughput: 0: 10052.3, 1: 9711.6. Samples: 147596352. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-12-26 17:27:41,062][104569] Avg episode reward: [(0, '6457.711'), (1, '6490.286')] [2023-12-26 17:27:41,253][105692] Updated weights for policy 0, policy_version 288206 (0.0008) [2023-12-26 17:27:41,317][105692] Updated weights for policy 0, policy_version 288216 (0.0008) [2023-12-26 17:27:41,385][105692] Updated weights for policy 0, policy_version 288226 (0.0009) [2023-12-26 17:27:41,435][105620] Updated weights for policy 1, policy_version 288237 (0.0008) [2023-12-26 17:27:41,501][105620] Updated weights for policy 1, policy_version 288247 (0.0006) [2023-12-26 17:27:41,567][105620] Updated weights for policy 1, policy_version 288257 (0.0006) [2023-12-26 17:27:42,137][105620] Updated weights for policy 1, policy_version 288267 (0.0007) [2023-12-26 17:27:42,200][105620] Updated weights for policy 1, policy_version 288277 (0.0009) [2023-12-26 17:27:42,260][105620] Updated weights for policy 1, policy_version 288287 (0.0009) [2023-12-26 17:27:42,261][105692] Updated weights for policy 0, policy_version 288236 (0.0010) [2023-12-26 17:27:42,326][105692] Updated weights for policy 0, policy_version 288246 (0.0008) [2023-12-26 17:27:42,392][105692] Updated weights for policy 0, policy_version 288256 (0.0009) [2023-12-26 17:27:42,873][105620] Updated weights for policy 1, policy_version 288297 (0.0008) [2023-12-26 17:27:42,940][105620] Updated weights for policy 1, policy_version 288307 (0.0009) [2023-12-26 17:27:43,001][105620] Updated weights for policy 1, policy_version 288317 (0.0006) [2023-12-26 17:27:43,069][105620] Updated weights for policy 1, policy_version 288327 (0.0006) [2023-12-26 17:27:43,258][105692] Updated weights for policy 0, policy_version 288266 (0.0009) [2023-12-26 17:27:43,324][105692] Updated weights for policy 0, policy_version 288276 (0.0010) [2023-12-26 17:27:43,389][105692] Updated weights for policy 0, policy_version 288286 (0.0009) [2023-12-26 17:27:43,459][105692] Updated weights for policy 0, policy_version 288296 (0.0010) [2023-12-26 17:27:43,569][105620] Updated weights for policy 1, policy_version 288337 (0.0005) [2023-12-26 17:27:43,615][105620] Updated weights for policy 1, policy_version 288347 (0.0005) [2023-12-26 17:27:43,666][105620] Updated weights for policy 1, policy_version 288357 (0.0005) [2023-12-26 17:27:44,233][105620] Updated weights for policy 1, policy_version 288367 (0.0005) [2023-12-26 17:27:44,267][105692] Updated weights for policy 0, policy_version 288306 (0.0008) [2023-12-26 17:27:44,287][105620] Updated weights for policy 1, policy_version 288377 (0.0006) [2023-12-26 17:27:44,317][105692] Updated weights for policy 0, policy_version 288316 (0.0008) [2023-12-26 17:27:44,348][105620] Updated weights for policy 1, policy_version 288387 (0.0005) [2023-12-26 17:27:44,366][105692] Updated weights for policy 0, policy_version 288326 (0.0009) [2023-12-26 17:27:44,896][105620] Updated weights for policy 1, policy_version 288397 (0.0007) [2023-12-26 17:27:44,960][105620] Updated weights for policy 1, policy_version 288407 (0.0011) [2023-12-26 17:27:45,022][105620] Updated weights for policy 1, policy_version 288417 (0.0010) [2023-12-26 17:27:45,247][105692] Updated weights for policy 0, policy_version 288336 (0.0008) [2023-12-26 17:27:45,297][105692] Updated weights for policy 0, policy_version 288346 (0.0008) [2023-12-26 17:27:45,349][105692] Updated weights for policy 0, policy_version 288357 (0.0009) [2023-12-26 17:27:45,673][105620] Updated weights for policy 1, policy_version 288427 (0.0011) [2023-12-26 17:27:45,732][105620] Updated weights for policy 1, policy_version 288437 (0.0010) [2023-12-26 17:27:45,798][105620] Updated weights for policy 1, policy_version 288447 (0.0010) [2023-12-26 17:27:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.9, 300 sec: 19522.0). Total num frames: 147685376. Throughput: 0: 9968.4, 1: 9757.1. Samples: 147655456. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 17:27:46,062][104569] Avg episode reward: [(0, '8293.298'), (1, '6021.115')] [2023-12-26 17:27:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000288360_73834496.pth... [2023-12-26 17:27:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000288456_73850880.pth... [2023-12-26 17:27:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000287304_73555968.pth [2023-12-26 17:27:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000287208_73539584.pth [2023-12-26 17:27:46,230][105692] Updated weights for policy 0, policy_version 288367 (0.0008) [2023-12-26 17:27:46,282][105692] Updated weights for policy 0, policy_version 288377 (0.0008) [2023-12-26 17:27:46,343][105692] Updated weights for policy 0, policy_version 288387 (0.0008) [2023-12-26 17:27:46,450][105620] Updated weights for policy 1, policy_version 288457 (0.0006) [2023-12-26 17:27:46,500][105620] Updated weights for policy 1, policy_version 288467 (0.0010) [2023-12-26 17:27:46,545][105620] Updated weights for policy 1, policy_version 288477 (0.0009) [2023-12-26 17:27:46,598][105620] Updated weights for policy 1, policy_version 288487 (0.0005) [2023-12-26 17:27:47,168][105692] Updated weights for policy 0, policy_version 288397 (0.0008) [2023-12-26 17:27:47,222][105692] Updated weights for policy 0, policy_version 288407 (0.0007) [2023-12-26 17:27:47,250][105620] Updated weights for policy 1, policy_version 288497 (0.0010) [2023-12-26 17:27:47,284][105692] Updated weights for policy 0, policy_version 288417 (0.0008) [2023-12-26 17:27:47,305][105620] Updated weights for policy 1, policy_version 288507 (0.0010) [2023-12-26 17:27:47,363][105620] Updated weights for policy 1, policy_version 288517 (0.0010) [2023-12-26 17:27:48,049][105692] Updated weights for policy 0, policy_version 288427 (0.0007) [2023-12-26 17:27:48,102][105620] Updated weights for policy 1, policy_version 288527 (0.0010) [2023-12-26 17:27:48,108][105692] Updated weights for policy 0, policy_version 288437 (0.0009) [2023-12-26 17:27:48,163][105692] Updated weights for policy 0, policy_version 288447 (0.0006) [2023-12-26 17:27:48,164][105620] Updated weights for policy 1, policy_version 288537 (0.0010) [2023-12-26 17:27:48,218][105620] Updated weights for policy 1, policy_version 288547 (0.0009) [2023-12-26 17:27:48,936][105692] Updated weights for policy 0, policy_version 288457 (0.0008) [2023-12-26 17:27:48,942][105620] Updated weights for policy 1, policy_version 288557 (0.0008) [2023-12-26 17:27:48,995][105620] Updated weights for policy 1, policy_version 288567 (0.0010) [2023-12-26 17:27:48,996][105692] Updated weights for policy 0, policy_version 288467 (0.0006) [2023-12-26 17:27:49,050][105620] Updated weights for policy 1, policy_version 288577 (0.0010) [2023-12-26 17:27:49,056][105692] Updated weights for policy 0, policy_version 288477 (0.0005) [2023-12-26 17:27:49,120][105692] Updated weights for policy 0, policy_version 288487 (0.0007) [2023-12-26 17:27:49,809][105692] Updated weights for policy 0, policy_version 288497 (0.0010) [2023-12-26 17:27:49,819][105620] Updated weights for policy 1, policy_version 288587 (0.0010) [2023-12-26 17:27:49,872][105692] Updated weights for policy 0, policy_version 288507 (0.0010) [2023-12-26 17:27:49,879][105620] Updated weights for policy 1, policy_version 288597 (0.0009) [2023-12-26 17:27:49,940][105692] Updated weights for policy 0, policy_version 288517 (0.0010) [2023-12-26 17:27:49,943][105620] Updated weights for policy 1, policy_version 288607 (0.0009) [2023-12-26 17:27:50,658][105620] Updated weights for policy 1, policy_version 288617 (0.0008) [2023-12-26 17:27:50,687][105692] Updated weights for policy 0, policy_version 288527 (0.0009) [2023-12-26 17:27:50,719][105620] Updated weights for policy 1, policy_version 288627 (0.0007) [2023-12-26 17:27:50,743][105692] Updated weights for policy 0, policy_version 288537 (0.0007) [2023-12-26 17:27:50,784][105620] Updated weights for policy 1, policy_version 288637 (0.0007) [2023-12-26 17:27:50,805][105692] Updated weights for policy 0, policy_version 288547 (0.0008) [2023-12-26 17:27:50,832][105620] Updated weights for policy 1, policy_version 288647 (0.0008) [2023-12-26 17:27:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 147783680. Throughput: 0: 9865.1, 1: 9797.2. Samples: 147769132. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 17:27:51,062][104569] Avg episode reward: [(0, '8641.862'), (1, '6574.448')] [2023-12-26 17:27:51,534][105692] Updated weights for policy 0, policy_version 288557 (0.0008) [2023-12-26 17:27:51,587][105692] Updated weights for policy 0, policy_version 288567 (0.0006) [2023-12-26 17:27:51,617][105620] Updated weights for policy 1, policy_version 288657 (0.0011) [2023-12-26 17:27:51,654][105692] Updated weights for policy 0, policy_version 288577 (0.0009) [2023-12-26 17:27:51,678][105620] Updated weights for policy 1, policy_version 288667 (0.0011) [2023-12-26 17:27:51,745][105620] Updated weights for policy 1, policy_version 288677 (0.0011) [2023-12-26 17:27:52,281][105692] Updated weights for policy 0, policy_version 288587 (0.0006) [2023-12-26 17:27:52,352][105692] Updated weights for policy 0, policy_version 288597 (0.0008) [2023-12-26 17:27:52,407][105585] KL-divergence is very high: 126.4686 [2023-12-26 17:27:52,413][105692] Updated weights for policy 0, policy_version 288607 (0.0011) [2023-12-26 17:27:52,429][105620] Updated weights for policy 1, policy_version 288687 (0.0010) [2023-12-26 17:27:52,450][105585] KL-divergence is very high: 150.4762 [2023-12-26 17:27:52,494][105620] Updated weights for policy 1, policy_version 288697 (0.0009) [2023-12-26 17:27:52,564][105620] Updated weights for policy 1, policy_version 288707 (0.0008) [2023-12-26 17:27:53,152][105620] Updated weights for policy 1, policy_version 288717 (0.0006) [2023-12-26 17:27:53,211][105620] Updated weights for policy 1, policy_version 288727 (0.0010) [2023-12-26 17:27:53,233][105692] Updated weights for policy 0, policy_version 288617 (0.0009) [2023-12-26 17:27:53,239][105585] KL-divergence is very high: 121.9273 [2023-12-26 17:27:53,269][105620] Updated weights for policy 1, policy_version 288737 (0.0010) [2023-12-26 17:27:53,297][105585] KL-divergence is very high: 106.1263 [2023-12-26 17:27:53,304][105692] Updated weights for policy 0, policy_version 288627 (0.0008) [2023-12-26 17:27:53,359][105692] Updated weights for policy 0, policy_version 288637 (0.0009) [2023-12-26 17:27:53,411][105692] Updated weights for policy 0, policy_version 288648 (0.0009) [2023-12-26 17:27:53,820][105620] Updated weights for policy 1, policy_version 288747 (0.0007) [2023-12-26 17:27:53,858][105586] KL-divergence is very high: 168.5776 [2023-12-26 17:27:53,864][105586] KL-divergence is very high: 108.7533 [2023-12-26 17:27:53,875][105620] Updated weights for policy 1, policy_version 288757 (0.0010) [2023-12-26 17:27:53,900][105586] KL-divergence is very high: 179.2494 [2023-12-26 17:27:53,906][105586] KL-divergence is very high: 266.9843 [2023-12-26 17:27:53,912][105586] KL-divergence is very high: 151.6477 [2023-12-26 17:27:53,940][105620] Updated weights for policy 1, policy_version 288767 (0.0010) [2023-12-26 17:27:53,954][105586] KL-divergence is very high: 169.5304 [2023-12-26 17:27:53,960][105586] KL-divergence is very high: 247.8414 [2023-12-26 17:27:53,967][105586] KL-divergence is very high: 133.4700 [2023-12-26 17:27:54,171][105692] Updated weights for policy 0, policy_version 288658 (0.0006) [2023-12-26 17:27:54,231][105692] Updated weights for policy 0, policy_version 288668 (0.0005) [2023-12-26 17:27:54,281][105692] Updated weights for policy 0, policy_version 288678 (0.0005) [2023-12-26 17:27:54,672][105586] KL-divergence is very high: 118.6144 [2023-12-26 17:27:54,679][105620] Updated weights for policy 1, policy_version 288777 (0.0011) [2023-12-26 17:27:54,719][105586] KL-divergence is very high: 106.6828 [2023-12-26 17:27:54,737][105620] Updated weights for policy 1, policy_version 288787 (0.0010) [2023-12-26 17:27:54,766][105586] KL-divergence is very high: 101.7611 [2023-12-26 17:27:54,792][105620] Updated weights for policy 1, policy_version 288797 (0.0010) [2023-12-26 17:27:54,843][105692] Updated weights for policy 0, policy_version 288688 (0.0005) [2023-12-26 17:27:54,851][105620] Updated weights for policy 1, policy_version 288807 (0.0011) [2023-12-26 17:27:54,901][105692] Updated weights for policy 0, policy_version 288698 (0.0005) [2023-12-26 17:27:54,969][105692] Updated weights for policy 0, policy_version 288708 (0.0008) [2023-12-26 17:27:55,580][105620] Updated weights for policy 1, policy_version 288817 (0.0006) [2023-12-26 17:27:55,611][105692] Updated weights for policy 0, policy_version 288718 (0.0008) [2023-12-26 17:27:55,632][105620] Updated weights for policy 1, policy_version 288827 (0.0005) [2023-12-26 17:27:55,664][105692] Updated weights for policy 0, policy_version 288728 (0.0005) [2023-12-26 17:27:55,693][105620] Updated weights for policy 1, policy_version 288837 (0.0007) [2023-12-26 17:27:55,715][105692] Updated weights for policy 0, policy_version 288738 (0.0005) [2023-12-26 17:27:56,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 147881984. Throughput: 0: 9817.3, 1: 9895.8. Samples: 147889672. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 17:27:56,063][104569] Avg episode reward: [(0, '8395.345'), (1, '6395.484')] [2023-12-26 17:27:56,269][105692] Updated weights for policy 0, policy_version 288748 (0.0007) [2023-12-26 17:27:56,317][105620] Updated weights for policy 1, policy_version 288847 (0.0010) [2023-12-26 17:27:56,328][105692] Updated weights for policy 0, policy_version 288758 (0.0010) [2023-12-26 17:27:56,377][105692] Updated weights for policy 0, policy_version 288768 (0.0009) [2023-12-26 17:27:56,380][105620] Updated weights for policy 1, policy_version 288857 (0.0007) [2023-12-26 17:27:56,436][105620] Updated weights for policy 1, policy_version 288867 (0.0005) [2023-12-26 17:27:56,920][105692] Updated weights for policy 0, policy_version 288778 (0.0005) [2023-12-26 17:27:56,962][105620] Updated weights for policy 1, policy_version 288877 (0.0005) [2023-12-26 17:27:56,979][105692] Updated weights for policy 0, policy_version 288788 (0.0006) [2023-12-26 17:27:57,023][105620] Updated weights for policy 1, policy_version 288887 (0.0005) [2023-12-26 17:27:57,042][105692] Updated weights for policy 0, policy_version 288798 (0.0010) [2023-12-26 17:27:57,079][105620] Updated weights for policy 1, policy_version 288897 (0.0005) [2023-12-26 17:27:57,099][105692] Updated weights for policy 0, policy_version 288808 (0.0010) [2023-12-26 17:27:57,729][105620] Updated weights for policy 1, policy_version 288907 (0.0007) [2023-12-26 17:27:57,774][105692] Updated weights for policy 0, policy_version 288818 (0.0005) [2023-12-26 17:27:57,776][105620] Updated weights for policy 1, policy_version 288917 (0.0010) [2023-12-26 17:27:57,827][105620] Updated weights for policy 1, policy_version 288927 (0.0010) [2023-12-26 17:27:57,833][105692] Updated weights for policy 0, policy_version 288828 (0.0007) [2023-12-26 17:27:57,884][105692] Updated weights for policy 0, policy_version 288838 (0.0005) [2023-12-26 17:27:58,539][105692] Updated weights for policy 0, policy_version 288848 (0.0008) [2023-12-26 17:27:58,596][105692] Updated weights for policy 0, policy_version 288858 (0.0008) [2023-12-26 17:27:58,628][105620] Updated weights for policy 1, policy_version 288937 (0.0010) [2023-12-26 17:27:58,654][105692] Updated weights for policy 0, policy_version 288868 (0.0007) [2023-12-26 17:27:58,688][105620] Updated weights for policy 1, policy_version 288947 (0.0010) [2023-12-26 17:27:58,754][105620] Updated weights for policy 1, policy_version 288957 (0.0010) [2023-12-26 17:27:58,807][105620] Updated weights for policy 1, policy_version 288967 (0.0010) [2023-12-26 17:27:59,461][105692] Updated weights for policy 0, policy_version 288878 (0.0006) [2023-12-26 17:27:59,514][105692] Updated weights for policy 0, policy_version 288888 (0.0006) [2023-12-26 17:27:59,527][105620] Updated weights for policy 1, policy_version 288977 (0.0010) [2023-12-26 17:27:59,571][105692] Updated weights for policy 0, policy_version 288898 (0.0008) [2023-12-26 17:27:59,585][105620] Updated weights for policy 1, policy_version 288987 (0.0010) [2023-12-26 17:27:59,640][105620] Updated weights for policy 1, policy_version 288997 (0.0010) [2023-12-26 17:28:00,204][105692] Updated weights for policy 0, policy_version 288908 (0.0005) [2023-12-26 17:28:00,270][105692] Updated weights for policy 0, policy_version 288918 (0.0005) [2023-12-26 17:28:00,319][105692] Updated weights for policy 0, policy_version 288928 (0.0007) [2023-12-26 17:28:00,361][105620] Updated weights for policy 1, policy_version 289007 (0.0008) [2023-12-26 17:28:00,423][105620] Updated weights for policy 1, policy_version 289017 (0.0009) [2023-12-26 17:28:00,479][105620] Updated weights for policy 1, policy_version 289027 (0.0008) [2023-12-26 17:28:00,854][105692] Updated weights for policy 0, policy_version 288938 (0.0007) [2023-12-26 17:28:00,898][105692] Updated weights for policy 0, policy_version 288948 (0.0005) [2023-12-26 17:28:00,951][105692] Updated weights for policy 0, policy_version 288958 (0.0005) [2023-12-26 17:28:01,011][105692] Updated weights for policy 0, policy_version 288968 (0.0006) [2023-12-26 17:28:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 147988480. Throughput: 0: 9939.8, 1: 9924.9. Samples: 147953688. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 17:28:01,062][104569] Avg episode reward: [(0, '8665.589'), (1, '6299.884')] [2023-12-26 17:28:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000288968_73990144.pth... [2023-12-26 17:28:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000289032_73998336.pth... [2023-12-26 17:28:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000287816_73695232.pth [2023-12-26 17:28:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000287880_73703424.pth [2023-12-26 17:28:01,146][105620] Updated weights for policy 1, policy_version 289037 (0.0009) [2023-12-26 17:28:01,201][105620] Updated weights for policy 1, policy_version 289047 (0.0008) [2023-12-26 17:28:01,256][105620] Updated weights for policy 1, policy_version 289057 (0.0008) [2023-12-26 17:28:01,736][105692] Updated weights for policy 0, policy_version 288978 (0.0010) [2023-12-26 17:28:01,791][105692] Updated weights for policy 0, policy_version 288988 (0.0009) [2023-12-26 17:28:01,843][105692] Updated weights for policy 0, policy_version 288998 (0.0009) [2023-12-26 17:28:02,038][105620] Updated weights for policy 1, policy_version 289067 (0.0008) [2023-12-26 17:28:02,089][105620] Updated weights for policy 1, policy_version 289077 (0.0009) [2023-12-26 17:28:02,149][105620] Updated weights for policy 1, policy_version 289087 (0.0008) [2023-12-26 17:28:02,604][105692] Updated weights for policy 0, policy_version 289008 (0.0008) [2023-12-26 17:28:02,668][105692] Updated weights for policy 0, policy_version 289018 (0.0007) [2023-12-26 17:28:02,729][105692] Updated weights for policy 0, policy_version 289028 (0.0007) [2023-12-26 17:28:02,892][105620] Updated weights for policy 1, policy_version 289097 (0.0008) [2023-12-26 17:28:02,938][105620] Updated weights for policy 1, policy_version 289107 (0.0005) [2023-12-26 17:28:02,995][105620] Updated weights for policy 1, policy_version 289117 (0.0005) [2023-12-26 17:28:03,053][105620] Updated weights for policy 1, policy_version 289127 (0.0005) [2023-12-26 17:28:03,469][105692] Updated weights for policy 0, policy_version 289038 (0.0010) [2023-12-26 17:28:03,522][105692] Updated weights for policy 0, policy_version 289048 (0.0010) [2023-12-26 17:28:03,572][105692] Updated weights for policy 0, policy_version 289058 (0.0006) [2023-12-26 17:28:03,771][105620] Updated weights for policy 1, policy_version 289137 (0.0009) [2023-12-26 17:28:03,832][105620] Updated weights for policy 1, policy_version 289147 (0.0010) [2023-12-26 17:28:03,898][105620] Updated weights for policy 1, policy_version 289157 (0.0008) [2023-12-26 17:28:04,219][105692] Updated weights for policy 0, policy_version 289068 (0.0007) [2023-12-26 17:28:04,283][105692] Updated weights for policy 0, policy_version 289078 (0.0010) [2023-12-26 17:28:04,337][105692] Updated weights for policy 0, policy_version 289088 (0.0010) [2023-12-26 17:28:04,567][105620] Updated weights for policy 1, policy_version 289167 (0.0008) [2023-12-26 17:28:04,621][105620] Updated weights for policy 1, policy_version 289177 (0.0009) [2023-12-26 17:28:04,677][105620] Updated weights for policy 1, policy_version 289187 (0.0010) [2023-12-26 17:28:05,039][105692] Updated weights for policy 0, policy_version 289098 (0.0009) [2023-12-26 17:28:05,085][105692] Updated weights for policy 0, policy_version 289108 (0.0005) [2023-12-26 17:28:05,133][105692] Updated weights for policy 0, policy_version 289118 (0.0006) [2023-12-26 17:28:05,196][105692] Updated weights for policy 0, policy_version 289128 (0.0008) [2023-12-26 17:28:05,427][105620] Updated weights for policy 1, policy_version 289197 (0.0009) [2023-12-26 17:28:05,480][105620] Updated weights for policy 1, policy_version 289207 (0.0009) [2023-12-26 17:28:05,532][105620] Updated weights for policy 1, policy_version 289217 (0.0008) [2023-12-26 17:28:05,925][105692] Updated weights for policy 0, policy_version 289138 (0.0009) [2023-12-26 17:28:05,986][105692] Updated weights for policy 0, policy_version 289148 (0.0009) [2023-12-26 17:28:06,043][105692] Updated weights for policy 0, policy_version 289158 (0.0009) [2023-12-26 17:28:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 148086784. Throughput: 0: 9901.8, 1: 9866.2. Samples: 148071012. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 17:28:06,062][104569] Avg episode reward: [(0, '9088.204'), (1, '6113.375')] [2023-12-26 17:28:06,257][105620] Updated weights for policy 1, policy_version 289227 (0.0009) [2023-12-26 17:28:06,315][105620] Updated weights for policy 1, policy_version 289237 (0.0006) [2023-12-26 17:28:06,355][105586] KL-divergence is very high: 107.8319 [2023-12-26 17:28:06,373][105620] Updated weights for policy 1, policy_version 289247 (0.0005) [2023-12-26 17:28:06,379][105586] KL-divergence is very high: 253.6546 [2023-12-26 17:28:06,392][105586] KL-divergence is very high: 348.3148 [2023-12-26 17:28:06,405][105586] KL-divergence is very high: 311.8771 [2023-12-26 17:28:06,880][105692] Updated weights for policy 0, policy_version 289168 (0.0010) [2023-12-26 17:28:06,946][105692] Updated weights for policy 0, policy_version 289178 (0.0010) [2023-12-26 17:28:07,008][105620] Updated weights for policy 1, policy_version 289257 (0.0006) [2023-12-26 17:28:07,010][105692] Updated weights for policy 0, policy_version 289188 (0.0009) [2023-12-26 17:28:07,067][105620] Updated weights for policy 1, policy_version 289267 (0.0005) [2023-12-26 17:28:07,117][105620] Updated weights for policy 1, policy_version 289277 (0.0005) [2023-12-26 17:28:07,173][105620] Updated weights for policy 1, policy_version 289287 (0.0007) [2023-12-26 17:28:07,776][105620] Updated weights for policy 1, policy_version 289297 (0.0008) [2023-12-26 17:28:07,778][105692] Updated weights for policy 0, policy_version 289198 (0.0006) [2023-12-26 17:28:07,824][105692] Updated weights for policy 0, policy_version 289208 (0.0005) [2023-12-26 17:28:07,824][105620] Updated weights for policy 1, policy_version 289307 (0.0010) [2023-12-26 17:28:07,873][105620] Updated weights for policy 1, policy_version 289317 (0.0010) [2023-12-26 17:28:07,876][105692] Updated weights for policy 0, policy_version 289218 (0.0005) [2023-12-26 17:28:08,442][105692] Updated weights for policy 0, policy_version 289228 (0.0006) [2023-12-26 17:28:08,500][105692] Updated weights for policy 0, policy_version 289238 (0.0010) [2023-12-26 17:28:08,567][105692] Updated weights for policy 0, policy_version 289248 (0.0009) [2023-12-26 17:28:08,593][105620] Updated weights for policy 1, policy_version 289327 (0.0010) [2023-12-26 17:28:08,648][105620] Updated weights for policy 1, policy_version 289337 (0.0010) [2023-12-26 17:28:08,709][105620] Updated weights for policy 1, policy_version 289347 (0.0010) [2023-12-26 17:28:09,203][105692] Updated weights for policy 0, policy_version 289258 (0.0008) [2023-12-26 17:28:09,270][105692] Updated weights for policy 0, policy_version 289268 (0.0009) [2023-12-26 17:28:09,330][105692] Updated weights for policy 0, policy_version 289278 (0.0009) [2023-12-26 17:28:09,398][105692] Updated weights for policy 0, policy_version 289288 (0.0009) [2023-12-26 17:28:09,465][105620] Updated weights for policy 1, policy_version 289357 (0.0008) [2023-12-26 17:28:09,520][105620] Updated weights for policy 1, policy_version 289367 (0.0006) [2023-12-26 17:28:09,572][105620] Updated weights for policy 1, policy_version 289377 (0.0006) [2023-12-26 17:28:10,194][105692] Updated weights for policy 0, policy_version 289298 (0.0009) [2023-12-26 17:28:10,235][105620] Updated weights for policy 1, policy_version 289387 (0.0006) [2023-12-26 17:28:10,255][105692] Updated weights for policy 0, policy_version 289308 (0.0009) [2023-12-26 17:28:10,288][105620] Updated weights for policy 1, policy_version 289397 (0.0007) [2023-12-26 17:28:10,311][105692] Updated weights for policy 0, policy_version 289318 (0.0007) [2023-12-26 17:28:10,344][105620] Updated weights for policy 1, policy_version 289407 (0.0006) [2023-12-26 17:28:11,050][105692] Updated weights for policy 0, policy_version 289328 (0.0008) [2023-12-26 17:28:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 148176896. Throughput: 0: 9848.7, 1: 10023.9. Samples: 148190444. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 17:28:11,063][104569] Avg episode reward: [(0, '9265.070'), (1, '6206.939')] [2023-12-26 17:28:11,106][105620] Updated weights for policy 1, policy_version 289417 (0.0009) [2023-12-26 17:28:11,108][105692] Updated weights for policy 0, policy_version 289338 (0.0009) [2023-12-26 17:28:11,172][105620] Updated weights for policy 1, policy_version 289427 (0.0006) [2023-12-26 17:28:11,176][105692] Updated weights for policy 0, policy_version 289348 (0.0008) [2023-12-26 17:28:11,233][105620] Updated weights for policy 1, policy_version 289437 (0.0009) [2023-12-26 17:28:11,300][105620] Updated weights for policy 1, policy_version 289447 (0.0010) [2023-12-26 17:28:11,945][105620] Updated weights for policy 1, policy_version 289457 (0.0007) [2023-12-26 17:28:12,008][105620] Updated weights for policy 1, policy_version 289467 (0.0006) [2023-12-26 17:28:12,024][105692] Updated weights for policy 0, policy_version 289358 (0.0007) [2023-12-26 17:28:12,073][105620] Updated weights for policy 1, policy_version 289477 (0.0007) [2023-12-26 17:28:12,084][105692] Updated weights for policy 0, policy_version 289368 (0.0007) [2023-12-26 17:28:12,137][105692] Updated weights for policy 0, policy_version 289378 (0.0009) [2023-12-26 17:28:12,802][105620] Updated weights for policy 1, policy_version 289487 (0.0009) [2023-12-26 17:28:12,856][105620] Updated weights for policy 1, policy_version 289497 (0.0005) [2023-12-26 17:28:12,910][105620] Updated weights for policy 1, policy_version 289507 (0.0005) [2023-12-26 17:28:12,955][105692] Updated weights for policy 0, policy_version 289388 (0.0009) [2023-12-26 17:28:13,004][105692] Updated weights for policy 0, policy_version 289398 (0.0009) [2023-12-26 17:28:13,050][105692] Updated weights for policy 0, policy_version 289408 (0.0008) [2023-12-26 17:28:13,609][105620] Updated weights for policy 1, policy_version 289517 (0.0005) [2023-12-26 17:28:13,666][105620] Updated weights for policy 1, policy_version 289527 (0.0005) [2023-12-26 17:28:13,701][105586] KL-divergence is very high: 104.8389 [2023-12-26 17:28:13,719][105620] Updated weights for policy 1, policy_version 289537 (0.0005) [2023-12-26 17:28:13,730][105586] KL-divergence is very high: 171.2797 [2023-12-26 17:28:13,747][105586] KL-divergence is very high: 160.7560 [2023-12-26 17:28:13,827][105692] Updated weights for policy 0, policy_version 289418 (0.0009) [2023-12-26 17:28:13,881][105692] Updated weights for policy 0, policy_version 289428 (0.0009) [2023-12-26 17:28:13,942][105692] Updated weights for policy 0, policy_version 289438 (0.0009) [2023-12-26 17:28:14,004][105692] Updated weights for policy 0, policy_version 289448 (0.0009) [2023-12-26 17:28:14,364][105586] KL-divergence is very high: 111.1637 [2023-12-26 17:28:14,370][105586] KL-divergence is very high: 103.8632 [2023-12-26 17:28:14,379][105620] Updated weights for policy 1, policy_version 289547 (0.0006) [2023-12-26 17:28:14,431][105620] Updated weights for policy 1, policy_version 289557 (0.0008) [2023-12-26 17:28:14,489][105620] Updated weights for policy 1, policy_version 289567 (0.0009) [2023-12-26 17:28:14,750][105692] Updated weights for policy 0, policy_version 289458 (0.0009) [2023-12-26 17:28:14,809][105692] Updated weights for policy 0, policy_version 289468 (0.0008) [2023-12-26 17:28:14,860][105692] Updated weights for policy 0, policy_version 289478 (0.0009) [2023-12-26 17:28:15,192][105620] Updated weights for policy 1, policy_version 289577 (0.0008) [2023-12-26 17:28:15,254][105620] Updated weights for policy 1, policy_version 289587 (0.0008) [2023-12-26 17:28:15,311][105586] KL-divergence is very high: 125.0812 [2023-12-26 17:28:15,326][105620] Updated weights for policy 1, policy_version 289597 (0.0010) [2023-12-26 17:28:15,361][105586] KL-divergence is very high: 146.2198 [2023-12-26 17:28:15,386][105620] Updated weights for policy 1, policy_version 289607 (0.0010) [2023-12-26 17:28:15,581][105692] Updated weights for policy 0, policy_version 289488 (0.0006) [2023-12-26 17:28:15,642][105692] Updated weights for policy 0, policy_version 289498 (0.0005) [2023-12-26 17:28:15,691][105692] Updated weights for policy 0, policy_version 289508 (0.0005) [2023-12-26 17:28:16,028][105620] Updated weights for policy 1, policy_version 289617 (0.0006) [2023-12-26 17:28:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 148275200. Throughput: 0: 9727.8, 1: 10021.2. Samples: 148248104. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 17:28:16,063][104569] Avg episode reward: [(0, '9355.769'), (1, '6015.142')] [2023-12-26 17:28:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000289512_74129408.pth... [2023-12-26 17:28:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000288360_73834496.pth [2023-12-26 17:28:16,095][105620] Updated weights for policy 1, policy_version 289627 (0.0006) [2023-12-26 17:28:16,142][105620] Updated weights for policy 1, policy_version 289637 (0.0010) [2023-12-26 17:28:16,154][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000289640_74153984.pth... [2023-12-26 17:28:16,157][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000288456_73850880.pth [2023-12-26 17:28:16,238][105692] Updated weights for policy 0, policy_version 289518 (0.0006) [2023-12-26 17:28:16,292][105692] Updated weights for policy 0, policy_version 289528 (0.0005) [2023-12-26 17:28:16,340][105692] Updated weights for policy 0, policy_version 289538 (0.0005) [2023-12-26 17:28:16,822][105620] Updated weights for policy 1, policy_version 289647 (0.0007) [2023-12-26 17:28:16,888][105620] Updated weights for policy 1, policy_version 289657 (0.0005) [2023-12-26 17:28:16,950][105620] Updated weights for policy 1, policy_version 289667 (0.0008) [2023-12-26 17:28:17,036][105692] Updated weights for policy 0, policy_version 289548 (0.0007) [2023-12-26 17:28:17,095][105692] Updated weights for policy 0, policy_version 289558 (0.0008) [2023-12-26 17:28:17,154][105692] Updated weights for policy 0, policy_version 289568 (0.0008) [2023-12-26 17:28:17,646][105620] Updated weights for policy 1, policy_version 289677 (0.0010) [2023-12-26 17:28:17,709][105620] Updated weights for policy 1, policy_version 289687 (0.0010) [2023-12-26 17:28:17,757][105620] Updated weights for policy 1, policy_version 289697 (0.0010) [2023-12-26 17:28:17,932][105692] Updated weights for policy 0, policy_version 289578 (0.0008) [2023-12-26 17:28:17,994][105692] Updated weights for policy 0, policy_version 289588 (0.0008) [2023-12-26 17:28:18,054][105692] Updated weights for policy 0, policy_version 289598 (0.0008) [2023-12-26 17:28:18,109][105692] Updated weights for policy 0, policy_version 289608 (0.0009) [2023-12-26 17:28:18,416][105620] Updated weights for policy 1, policy_version 289707 (0.0010) [2023-12-26 17:28:18,475][105620] Updated weights for policy 1, policy_version 289717 (0.0010) [2023-12-26 17:28:18,536][105620] Updated weights for policy 1, policy_version 289727 (0.0010) [2023-12-26 17:28:18,783][105692] Updated weights for policy 0, policy_version 289618 (0.0010) [2023-12-26 17:28:18,842][105692] Updated weights for policy 0, policy_version 289628 (0.0011) [2023-12-26 17:28:18,896][105692] Updated weights for policy 0, policy_version 289638 (0.0007) [2023-12-26 17:28:19,172][105620] Updated weights for policy 1, policy_version 289737 (0.0006) [2023-12-26 17:28:19,227][105620] Updated weights for policy 1, policy_version 289747 (0.0010) [2023-12-26 17:28:19,294][105620] Updated weights for policy 1, policy_version 289757 (0.0010) [2023-12-26 17:28:19,358][105620] Updated weights for policy 1, policy_version 289767 (0.0010) [2023-12-26 17:28:19,553][105692] Updated weights for policy 0, policy_version 289648 (0.0009) [2023-12-26 17:28:19,608][105692] Updated weights for policy 0, policy_version 289658 (0.0010) [2023-12-26 17:28:19,669][105692] Updated weights for policy 0, policy_version 289668 (0.0010) [2023-12-26 17:28:20,148][105620] Updated weights for policy 1, policy_version 289777 (0.0011) [2023-12-26 17:28:20,214][105620] Updated weights for policy 1, policy_version 289787 (0.0011) [2023-12-26 17:28:20,278][105620] Updated weights for policy 1, policy_version 289797 (0.0011) [2023-12-26 17:28:20,392][105692] Updated weights for policy 0, policy_version 289678 (0.0009) [2023-12-26 17:28:20,440][105692] Updated weights for policy 0, policy_version 289688 (0.0010) [2023-12-26 17:28:20,489][105692] Updated weights for policy 0, policy_version 289698 (0.0010) [2023-12-26 17:28:21,051][105620] Updated weights for policy 1, policy_version 289807 (0.0010) [2023-12-26 17:28:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 148373504. Throughput: 0: 9670.1, 1: 10028.6. Samples: 148368340. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 17:28:21,062][104569] Avg episode reward: [(0, '9356.235'), (1, '6571.517')] [2023-12-26 17:28:21,116][105620] Updated weights for policy 1, policy_version 289817 (0.0010) [2023-12-26 17:28:21,187][105620] Updated weights for policy 1, policy_version 289827 (0.0007) [2023-12-26 17:28:21,272][105692] Updated weights for policy 0, policy_version 289708 (0.0010) [2023-12-26 17:28:21,334][105692] Updated weights for policy 0, policy_version 289718 (0.0006) [2023-12-26 17:28:21,407][105692] Updated weights for policy 0, policy_version 289728 (0.0008) [2023-12-26 17:28:21,834][105620] Updated weights for policy 1, policy_version 289837 (0.0008) [2023-12-26 17:28:21,897][105620] Updated weights for policy 1, policy_version 289847 (0.0011) [2023-12-26 17:28:21,949][105620] Updated weights for policy 1, policy_version 289857 (0.0010) [2023-12-26 17:28:22,233][105692] Updated weights for policy 0, policy_version 289738 (0.0009) [2023-12-26 17:28:22,296][105692] Updated weights for policy 0, policy_version 289748 (0.0008) [2023-12-26 17:28:22,362][105692] Updated weights for policy 0, policy_version 289758 (0.0007) [2023-12-26 17:28:22,423][105692] Updated weights for policy 0, policy_version 289768 (0.0009) [2023-12-26 17:28:22,698][105620] Updated weights for policy 1, policy_version 289867 (0.0010) [2023-12-26 17:28:22,765][105620] Updated weights for policy 1, policy_version 289877 (0.0011) [2023-12-26 17:28:22,828][105620] Updated weights for policy 1, policy_version 289887 (0.0011) [2023-12-26 17:28:23,135][105692] Updated weights for policy 0, policy_version 289778 (0.0006) [2023-12-26 17:28:23,189][105692] Updated weights for policy 0, policy_version 289788 (0.0005) [2023-12-26 17:28:23,247][105692] Updated weights for policy 0, policy_version 289798 (0.0008) [2023-12-26 17:28:23,553][105620] Updated weights for policy 1, policy_version 289897 (0.0007) [2023-12-26 17:28:23,618][105620] Updated weights for policy 1, policy_version 289907 (0.0008) [2023-12-26 17:28:23,689][105620] Updated weights for policy 1, policy_version 289917 (0.0005) [2023-12-26 17:28:23,746][105620] Updated weights for policy 1, policy_version 289927 (0.0008) [2023-12-26 17:28:23,819][105692] Updated weights for policy 0, policy_version 289808 (0.0005) [2023-12-26 17:28:23,875][105692] Updated weights for policy 0, policy_version 289818 (0.0010) [2023-12-26 17:28:23,925][105692] Updated weights for policy 0, policy_version 289828 (0.0006) [2023-12-26 17:28:24,457][105620] Updated weights for policy 1, policy_version 289937 (0.0008) [2023-12-26 17:28:24,511][105620] Updated weights for policy 1, policy_version 289947 (0.0007) [2023-12-26 17:28:24,571][105620] Updated weights for policy 1, policy_version 289957 (0.0005) [2023-12-26 17:28:24,586][105692] Updated weights for policy 0, policy_version 289838 (0.0009) [2023-12-26 17:28:24,644][105692] Updated weights for policy 0, policy_version 289848 (0.0011) [2023-12-26 17:28:24,714][105692] Updated weights for policy 0, policy_version 289858 (0.0007) [2023-12-26 17:28:25,150][105620] Updated weights for policy 1, policy_version 289967 (0.0005) [2023-12-26 17:28:25,196][105620] Updated weights for policy 1, policy_version 289977 (0.0005) [2023-12-26 17:28:25,209][105586] KL-divergence is very high: 351.6458 [2023-12-26 17:28:25,219][105586] KL-divergence is very high: 237.0161 [2023-12-26 17:28:25,241][105620] Updated weights for policy 1, policy_version 289987 (0.0005) [2023-12-26 17:28:25,246][105586] KL-divergence is very high: 584.3760 [2023-12-26 17:28:25,255][105586] KL-divergence is very high: 325.1302 [2023-12-26 17:28:25,405][105692] Updated weights for policy 0, policy_version 289868 (0.0007) [2023-12-26 17:28:25,449][105692] Updated weights for policy 0, policy_version 289878 (0.0010) [2023-12-26 17:28:25,506][105692] Updated weights for policy 0, policy_version 289888 (0.0010) [2023-12-26 17:28:25,896][105586] KL-divergence is very high: 294.9163 [2023-12-26 17:28:25,911][105586] KL-divergence is very high: 510.8491 [2023-12-26 17:28:25,929][105620] Updated weights for policy 1, policy_version 289997 (0.0008) [2023-12-26 17:28:25,948][105586] KL-divergence is very high: 270.1506 [2023-12-26 17:28:25,958][105586] KL-divergence is very high: 515.0746 [2023-12-26 17:28:25,983][105620] Updated weights for policy 1, policy_version 290007 (0.0008) [2023-12-26 17:28:25,989][105586] KL-divergence is very high: 283.5713 [2023-12-26 17:28:26,002][105586] KL-divergence is very high: 528.2154 [2023-12-26 17:28:26,043][105586] KL-divergence is very high: 305.5458 [2023-12-26 17:28:26,049][105620] Updated weights for policy 1, policy_version 290017 (0.0008) [2023-12-26 17:28:26,055][105586] KL-divergence is very high: 565.0137 [2023-12-26 17:28:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 148471808. Throughput: 0: 9642.8, 1: 10126.5. Samples: 148485972. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 17:28:26,062][104569] Avg episode reward: [(0, '9356.847'), (1, '5923.412')] [2023-12-26 17:28:26,221][105692] Updated weights for policy 0, policy_version 289898 (0.0009) [2023-12-26 17:28:26,285][105692] Updated weights for policy 0, policy_version 289908 (0.0007) [2023-12-26 17:28:26,336][105692] Updated weights for policy 0, policy_version 289918 (0.0010) [2023-12-26 17:28:26,381][105692] Updated weights for policy 0, policy_version 289928 (0.0010) [2023-12-26 17:28:26,705][105620] Updated weights for policy 1, policy_version 290027 (0.0008) [2023-12-26 17:28:26,749][105620] Updated weights for policy 1, policy_version 290037 (0.0008) [2023-12-26 17:28:26,789][105620] Updated weights for policy 1, policy_version 290047 (0.0006) [2023-12-26 17:28:27,116][105692] Updated weights for policy 0, policy_version 289938 (0.0010) [2023-12-26 17:28:27,173][105692] Updated weights for policy 0, policy_version 289948 (0.0009) [2023-12-26 17:28:27,235][105692] Updated weights for policy 0, policy_version 289958 (0.0007) [2023-12-26 17:28:27,543][105620] Updated weights for policy 1, policy_version 290057 (0.0008) [2023-12-26 17:28:27,589][105620] Updated weights for policy 1, policy_version 290067 (0.0008) [2023-12-26 17:28:27,641][105620] Updated weights for policy 1, policy_version 290077 (0.0009) [2023-12-26 17:28:27,687][105620] Updated weights for policy 1, policy_version 290087 (0.0008) [2023-12-26 17:28:27,878][105692] Updated weights for policy 0, policy_version 289968 (0.0008) [2023-12-26 17:28:27,930][105692] Updated weights for policy 0, policy_version 289978 (0.0010) [2023-12-26 17:28:27,988][105692] Updated weights for policy 0, policy_version 289989 (0.0010) [2023-12-26 17:28:28,309][105620] Updated weights for policy 1, policy_version 290097 (0.0006) [2023-12-26 17:28:28,376][105620] Updated weights for policy 1, policy_version 290107 (0.0008) [2023-12-26 17:28:28,434][105620] Updated weights for policy 1, policy_version 290117 (0.0005) [2023-12-26 17:28:28,735][105692] Updated weights for policy 0, policy_version 290000 (0.0006) [2023-12-26 17:28:28,798][105692] Updated weights for policy 0, policy_version 290010 (0.0008) [2023-12-26 17:28:28,853][105692] Updated weights for policy 0, policy_version 290020 (0.0009) [2023-12-26 17:28:28,997][105620] Updated weights for policy 1, policy_version 290127 (0.0005) [2023-12-26 17:28:29,050][105620] Updated weights for policy 1, policy_version 290137 (0.0006) [2023-12-26 17:28:29,107][105620] Updated weights for policy 1, policy_version 290147 (0.0005) [2023-12-26 17:28:29,567][105692] Updated weights for policy 0, policy_version 290030 (0.0009) [2023-12-26 17:28:29,627][105692] Updated weights for policy 0, policy_version 290040 (0.0008) [2023-12-26 17:28:29,689][105692] Updated weights for policy 0, policy_version 290050 (0.0008) [2023-12-26 17:28:29,810][105620] Updated weights for policy 1, policy_version 290157 (0.0009) [2023-12-26 17:28:29,870][105620] Updated weights for policy 1, policy_version 290167 (0.0008) [2023-12-26 17:28:29,930][105620] Updated weights for policy 1, policy_version 290177 (0.0008) [2023-12-26 17:28:30,390][105692] Updated weights for policy 0, policy_version 290060 (0.0009) [2023-12-26 17:28:30,446][105692] Updated weights for policy 0, policy_version 290070 (0.0011) [2023-12-26 17:28:30,505][105692] Updated weights for policy 0, policy_version 290080 (0.0009) [2023-12-26 17:28:30,579][105620] Updated weights for policy 1, policy_version 290187 (0.0010) [2023-12-26 17:28:30,645][105620] Updated weights for policy 1, policy_version 290197 (0.0011) [2023-12-26 17:28:30,710][105620] Updated weights for policy 1, policy_version 290207 (0.0010) [2023-12-26 17:28:31,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 148578304. Throughput: 0: 9738.5, 1: 10087.0. Samples: 148547608. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 17:28:31,063][104569] Avg episode reward: [(0, '9003.539'), (1, '5922.342')] [2023-12-26 17:28:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000290088_74276864.pth... [2023-12-26 17:28:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000290216_74301440.pth... [2023-12-26 17:28:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000288968_73990144.pth [2023-12-26 17:28:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000289032_73998336.pth [2023-12-26 17:28:31,220][105692] Updated weights for policy 0, policy_version 290090 (0.0006) [2023-12-26 17:28:31,279][105692] Updated weights for policy 0, policy_version 290100 (0.0008) [2023-12-26 17:28:31,334][105692] Updated weights for policy 0, policy_version 290110 (0.0008) [2023-12-26 17:28:31,400][105692] Updated weights for policy 0, policy_version 290120 (0.0008) [2023-12-26 17:28:31,410][105620] Updated weights for policy 1, policy_version 290217 (0.0009) [2023-12-26 17:28:31,471][105620] Updated weights for policy 1, policy_version 290227 (0.0006) [2023-12-26 17:28:31,518][105620] Updated weights for policy 1, policy_version 290237 (0.0007) [2023-12-26 17:28:31,564][105586] KL-divergence is very high: 106.3294 [2023-12-26 17:28:31,569][105620] Updated weights for policy 1, policy_version 290247 (0.0009) [2023-12-26 17:28:32,076][105692] Updated weights for policy 0, policy_version 290130 (0.0008) [2023-12-26 17:28:32,134][105692] Updated weights for policy 0, policy_version 290140 (0.0008) [2023-12-26 17:28:32,185][105692] Updated weights for policy 0, policy_version 290150 (0.0008) [2023-12-26 17:28:32,198][105586] KL-divergence is very high: 100.1196 [2023-12-26 17:28:32,232][105620] Updated weights for policy 1, policy_version 290257 (0.0007) [2023-12-26 17:28:32,243][105586] KL-divergence is very high: 168.3157 [2023-12-26 17:28:32,293][105586] KL-divergence is very high: 148.3496 [2023-12-26 17:28:32,293][105620] Updated weights for policy 1, policy_version 290267 (0.0008) [2023-12-26 17:28:32,340][105586] KL-divergence is very high: 108.5952 [2023-12-26 17:28:32,352][105620] Updated weights for policy 1, policy_version 290277 (0.0010) [2023-12-26 17:28:32,959][105692] Updated weights for policy 0, policy_version 290160 (0.0008) [2023-12-26 17:28:33,006][105692] Updated weights for policy 0, policy_version 290170 (0.0007) [2023-12-26 17:28:33,054][105692] Updated weights for policy 0, policy_version 290180 (0.0008) [2023-12-26 17:28:33,068][105620] Updated weights for policy 1, policy_version 290287 (0.0009) [2023-12-26 17:28:33,122][105620] Updated weights for policy 1, policy_version 290297 (0.0010) [2023-12-26 17:28:33,165][105620] Updated weights for policy 1, policy_version 290307 (0.0010) [2023-12-26 17:28:33,174][105586] KL-divergence is very high: 135.2079 [2023-12-26 17:28:33,801][105586] KL-divergence is very high: 160.6769 [2023-12-26 17:28:33,828][105620] Updated weights for policy 1, policy_version 290317 (0.0009) [2023-12-26 17:28:33,842][105586] KL-divergence is very high: 154.0092 [2023-12-26 17:28:33,846][105692] Updated weights for policy 0, policy_version 290190 (0.0007) [2023-12-26 17:28:33,875][105620] Updated weights for policy 1, policy_version 290327 (0.0006) [2023-12-26 17:28:33,880][105586] KL-divergence is very high: 135.5748 [2023-12-26 17:28:33,898][105692] Updated weights for policy 0, policy_version 290200 (0.0007) [2023-12-26 17:28:33,921][105586] KL-divergence is very high: 124.1010 [2023-12-26 17:28:33,927][105620] Updated weights for policy 1, policy_version 290337 (0.0008) [2023-12-26 17:28:33,949][105692] Updated weights for policy 0, policy_version 290210 (0.0007) [2023-12-26 17:28:34,580][105620] Updated weights for policy 1, policy_version 290347 (0.0005) [2023-12-26 17:28:34,636][105620] Updated weights for policy 1, policy_version 290357 (0.0005) [2023-12-26 17:28:34,690][105620] Updated weights for policy 1, policy_version 290367 (0.0005) [2023-12-26 17:28:34,819][105692] Updated weights for policy 0, policy_version 290220 (0.0010) [2023-12-26 17:28:34,886][105692] Updated weights for policy 0, policy_version 290230 (0.0010) [2023-12-26 17:28:34,952][105692] Updated weights for policy 0, policy_version 290240 (0.0009) [2023-12-26 17:28:35,262][105620] Updated weights for policy 1, policy_version 290377 (0.0006) [2023-12-26 17:28:35,317][105620] Updated weights for policy 1, policy_version 290387 (0.0010) [2023-12-26 17:28:35,385][105620] Updated weights for policy 1, policy_version 290397 (0.0010) [2023-12-26 17:28:35,442][105620] Updated weights for policy 1, policy_version 290407 (0.0010) [2023-12-26 17:28:35,795][105692] Updated weights for policy 0, policy_version 290250 (0.0010) [2023-12-26 17:28:35,860][105692] Updated weights for policy 0, policy_version 290260 (0.0009) [2023-12-26 17:28:35,920][105692] Updated weights for policy 0, policy_version 290270 (0.0009) [2023-12-26 17:28:35,985][105692] Updated weights for policy 0, policy_version 290280 (0.0009) [2023-12-26 17:28:36,036][105620] Updated weights for policy 1, policy_version 290417 (0.0008) [2023-12-26 17:28:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 148676608. Throughput: 0: 9845.6, 1: 10102.6. Samples: 148666800. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 17:28:36,062][104569] Avg episode reward: [(0, '9004.038'), (1, '5733.553')] [2023-12-26 17:28:36,087][105620] Updated weights for policy 1, policy_version 290427 (0.0005) [2023-12-26 17:28:36,150][105620] Updated weights for policy 1, policy_version 290437 (0.0007) [2023-12-26 17:28:36,789][105620] Updated weights for policy 1, policy_version 290447 (0.0009) [2023-12-26 17:28:36,799][105692] Updated weights for policy 0, policy_version 290290 (0.0007) [2023-12-26 17:28:36,838][105620] Updated weights for policy 1, policy_version 290457 (0.0010) [2023-12-26 17:28:36,857][105692] Updated weights for policy 0, policy_version 290300 (0.0006) [2023-12-26 17:28:36,888][105620] Updated weights for policy 1, policy_version 290467 (0.0006) [2023-12-26 17:28:36,918][105692] Updated weights for policy 0, policy_version 290310 (0.0008) [2023-12-26 17:28:37,596][105620] Updated weights for policy 1, policy_version 290477 (0.0005) [2023-12-26 17:28:37,663][105620] Updated weights for policy 1, policy_version 290487 (0.0009) [2023-12-26 17:28:37,714][105692] Updated weights for policy 0, policy_version 290320 (0.0006) [2023-12-26 17:28:37,725][105620] Updated weights for policy 1, policy_version 290497 (0.0008) [2023-12-26 17:28:37,776][105692] Updated weights for policy 0, policy_version 290330 (0.0008) [2023-12-26 17:28:37,843][105692] Updated weights for policy 0, policy_version 290340 (0.0009) [2023-12-26 17:28:38,509][105620] Updated weights for policy 1, policy_version 290507 (0.0006) [2023-12-26 17:28:38,527][105692] Updated weights for policy 0, policy_version 290350 (0.0009) [2023-12-26 17:28:38,560][105620] Updated weights for policy 1, policy_version 290517 (0.0005) [2023-12-26 17:28:38,576][105692] Updated weights for policy 0, policy_version 290360 (0.0008) [2023-12-26 17:28:38,589][105586] KL-divergence is very high: 130.4755 [2023-12-26 17:28:38,619][105620] Updated weights for policy 1, policy_version 290527 (0.0005) [2023-12-26 17:28:38,635][105692] Updated weights for policy 0, policy_version 290370 (0.0007) [2023-12-26 17:28:38,638][105586] KL-divergence is very high: 121.4526 [2023-12-26 17:28:39,200][105620] Updated weights for policy 1, policy_version 290537 (0.0007) [2023-12-26 17:28:39,261][105620] Updated weights for policy 1, policy_version 290547 (0.0009) [2023-12-26 17:28:39,333][105620] Updated weights for policy 1, policy_version 290557 (0.0009) [2023-12-26 17:28:39,393][105692] Updated weights for policy 0, policy_version 290380 (0.0007) [2023-12-26 17:28:39,403][105620] Updated weights for policy 1, policy_version 290567 (0.0009) [2023-12-26 17:28:39,464][105692] Updated weights for policy 0, policy_version 290390 (0.0008) [2023-12-26 17:28:39,535][105692] Updated weights for policy 0, policy_version 290400 (0.0008) [2023-12-26 17:28:40,155][105620] Updated weights for policy 1, policy_version 290577 (0.0006) [2023-12-26 17:28:40,214][105620] Updated weights for policy 1, policy_version 290587 (0.0006) [2023-12-26 17:28:40,282][105620] Updated weights for policy 1, policy_version 290597 (0.0005) [2023-12-26 17:28:40,323][105692] Updated weights for policy 0, policy_version 290410 (0.0009) [2023-12-26 17:28:40,379][105692] Updated weights for policy 0, policy_version 290420 (0.0009) [2023-12-26 17:28:40,439][105692] Updated weights for policy 0, policy_version 290430 (0.0009) [2023-12-26 17:28:40,493][105692] Updated weights for policy 0, policy_version 290440 (0.0009) [2023-12-26 17:28:40,875][105620] Updated weights for policy 1, policy_version 290607 (0.0006) [2023-12-26 17:28:40,931][105620] Updated weights for policy 1, policy_version 290617 (0.0005) [2023-12-26 17:28:40,991][105620] Updated weights for policy 1, policy_version 290627 (0.0005) [2023-12-26 17:28:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 148774912. Throughput: 0: 9698.1, 1: 10126.5. Samples: 148781776. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 17:28:41,062][104569] Avg episode reward: [(0, '8824.027'), (1, '6105.915')] [2023-12-26 17:28:41,358][105692] Updated weights for policy 0, policy_version 290450 (0.0010) [2023-12-26 17:28:41,426][105692] Updated weights for policy 0, policy_version 290460 (0.0009) [2023-12-26 17:28:41,478][105692] Updated weights for policy 0, policy_version 290470 (0.0009) [2023-12-26 17:28:41,628][105620] Updated weights for policy 1, policy_version 290637 (0.0007) [2023-12-26 17:28:41,699][105620] Updated weights for policy 1, policy_version 290647 (0.0009) [2023-12-26 17:28:41,768][105620] Updated weights for policy 1, policy_version 290657 (0.0010) [2023-12-26 17:28:42,363][105692] Updated weights for policy 0, policy_version 290480 (0.0010) [2023-12-26 17:28:42,414][105620] Updated weights for policy 1, policy_version 290667 (0.0008) [2023-12-26 17:28:42,415][105692] Updated weights for policy 0, policy_version 290490 (0.0008) [2023-12-26 17:28:42,471][105620] Updated weights for policy 1, policy_version 290677 (0.0007) [2023-12-26 17:28:42,473][105692] Updated weights for policy 0, policy_version 290500 (0.0006) [2023-12-26 17:28:42,529][105620] Updated weights for policy 1, policy_version 290687 (0.0007) [2023-12-26 17:28:43,198][105620] Updated weights for policy 1, policy_version 290697 (0.0006) [2023-12-26 17:28:43,252][105620] Updated weights for policy 1, policy_version 290707 (0.0009) [2023-12-26 17:28:43,278][105692] Updated weights for policy 0, policy_version 290510 (0.0007) [2023-12-26 17:28:43,310][105620] Updated weights for policy 1, policy_version 290717 (0.0006) [2023-12-26 17:28:43,324][105692] Updated weights for policy 0, policy_version 290520 (0.0007) [2023-12-26 17:28:43,367][105620] Updated weights for policy 1, policy_version 290727 (0.0008) [2023-12-26 17:28:43,373][105692] Updated weights for policy 0, policy_version 290530 (0.0007) [2023-12-26 17:28:43,997][105620] Updated weights for policy 1, policy_version 290737 (0.0008) [2023-12-26 17:28:44,044][105620] Updated weights for policy 1, policy_version 290747 (0.0008) [2023-12-26 17:28:44,091][105620] Updated weights for policy 1, policy_version 290757 (0.0008) [2023-12-26 17:28:44,204][105692] Updated weights for policy 0, policy_version 290540 (0.0009) [2023-12-26 17:28:44,250][105692] Updated weights for policy 0, policy_version 290550 (0.0008) [2023-12-26 17:28:44,298][105692] Updated weights for policy 0, policy_version 290560 (0.0009) [2023-12-26 17:28:44,737][105620] Updated weights for policy 1, policy_version 290767 (0.0007) [2023-12-26 17:28:44,804][105620] Updated weights for policy 1, policy_version 290777 (0.0008) [2023-12-26 17:28:44,858][105620] Updated weights for policy 1, policy_version 290787 (0.0007) [2023-12-26 17:28:44,965][105692] Updated weights for policy 0, policy_version 290570 (0.0009) [2023-12-26 17:28:45,021][105692] Updated weights for policy 0, policy_version 290580 (0.0009) [2023-12-26 17:28:45,079][105692] Updated weights for policy 0, policy_version 290590 (0.0009) [2023-12-26 17:28:45,140][105692] Updated weights for policy 0, policy_version 290600 (0.0009) [2023-12-26 17:28:45,569][105620] Updated weights for policy 1, policy_version 290797 (0.0009) [2023-12-26 17:28:45,632][105620] Updated weights for policy 1, policy_version 290807 (0.0009) [2023-12-26 17:28:45,683][105620] Updated weights for policy 1, policy_version 290817 (0.0009) [2023-12-26 17:28:45,939][105692] Updated weights for policy 0, policy_version 290610 (0.0009) [2023-12-26 17:28:45,987][105692] Updated weights for policy 0, policy_version 290620 (0.0009) [2023-12-26 17:28:46,034][105692] Updated weights for policy 0, policy_version 290630 (0.0009) [2023-12-26 17:28:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 148873216. Throughput: 0: 9527.3, 1: 10140.3. Samples: 148838732. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 17:28:46,062][104569] Avg episode reward: [(0, '9177.956'), (1, '6664.604')] [2023-12-26 17:28:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000290632_74416128.pth... [2023-12-26 17:28:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000290824_74457088.pth... [2023-12-26 17:28:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000289640_74153984.pth [2023-12-26 17:28:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000289512_74129408.pth [2023-12-26 17:28:46,399][105620] Updated weights for policy 1, policy_version 290827 (0.0009) [2023-12-26 17:28:46,456][105620] Updated weights for policy 1, policy_version 290837 (0.0009) [2023-12-26 17:28:46,518][105620] Updated weights for policy 1, policy_version 290847 (0.0009) [2023-12-26 17:28:46,801][105692] Updated weights for policy 0, policy_version 290640 (0.0010) [2023-12-26 17:28:46,852][105692] Updated weights for policy 0, policy_version 290650 (0.0010) [2023-12-26 17:28:46,910][105692] Updated weights for policy 0, policy_version 290660 (0.0010) [2023-12-26 17:28:47,253][105620] Updated weights for policy 1, policy_version 290857 (0.0009) [2023-12-26 17:28:47,305][105620] Updated weights for policy 1, policy_version 290867 (0.0007) [2023-12-26 17:28:47,351][105620] Updated weights for policy 1, policy_version 290877 (0.0005) [2023-12-26 17:28:47,404][105620] Updated weights for policy 1, policy_version 290887 (0.0005) [2023-12-26 17:28:47,572][105692] Updated weights for policy 0, policy_version 290670 (0.0008) [2023-12-26 17:28:47,628][105692] Updated weights for policy 0, policy_version 290680 (0.0009) [2023-12-26 17:28:47,692][105692] Updated weights for policy 0, policy_version 290690 (0.0010) [2023-12-26 17:28:48,075][105620] Updated weights for policy 1, policy_version 290897 (0.0009) [2023-12-26 17:28:48,135][105620] Updated weights for policy 1, policy_version 290907 (0.0009) [2023-12-26 17:28:48,197][105620] Updated weights for policy 1, policy_version 290917 (0.0009) [2023-12-26 17:28:48,257][105692] Updated weights for policy 0, policy_version 290700 (0.0009) [2023-12-26 17:28:48,318][105692] Updated weights for policy 0, policy_version 290710 (0.0008) [2023-12-26 17:28:48,377][105692] Updated weights for policy 0, policy_version 290720 (0.0010) [2023-12-26 17:28:48,868][105620] Updated weights for policy 1, policy_version 290927 (0.0006) [2023-12-26 17:28:48,919][105620] Updated weights for policy 1, policy_version 290937 (0.0005) [2023-12-26 17:28:48,970][105620] Updated weights for policy 1, policy_version 290947 (0.0005) [2023-12-26 17:28:49,261][105692] Updated weights for policy 0, policy_version 290730 (0.0008) [2023-12-26 17:28:49,321][105692] Updated weights for policy 0, policy_version 290740 (0.0008) [2023-12-26 17:28:49,388][105692] Updated weights for policy 0, policy_version 290750 (0.0009) [2023-12-26 17:28:49,446][105692] Updated weights for policy 0, policy_version 290760 (0.0009) [2023-12-26 17:28:49,594][105620] Updated weights for policy 1, policy_version 290957 (0.0007) [2023-12-26 17:28:49,652][105620] Updated weights for policy 1, policy_version 290967 (0.0009) [2023-12-26 17:28:49,702][105620] Updated weights for policy 1, policy_version 290977 (0.0008) [2023-12-26 17:28:50,209][105692] Updated weights for policy 0, policy_version 290770 (0.0009) [2023-12-26 17:28:50,273][105692] Updated weights for policy 0, policy_version 290780 (0.0009) [2023-12-26 17:28:50,338][105692] Updated weights for policy 0, policy_version 290790 (0.0009) [2023-12-26 17:28:50,494][105620] Updated weights for policy 1, policy_version 290987 (0.0009) [2023-12-26 17:28:50,549][105620] Updated weights for policy 1, policy_version 290997 (0.0009) [2023-12-26 17:28:50,581][105586] KL-divergence is very high: 111.4399 [2023-12-26 17:28:50,611][105620] Updated weights for policy 1, policy_version 291007 (0.0009) [2023-12-26 17:28:50,631][105586] KL-divergence is very high: 120.7617 [2023-12-26 17:28:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 148963328. Throughput: 0: 9479.7, 1: 10203.1. Samples: 148956740. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-26 17:28:51,063][104569] Avg episode reward: [(0, '9358.202'), (1, '5367.319')] [2023-12-26 17:28:51,080][105692] Updated weights for policy 0, policy_version 290800 (0.0010) [2023-12-26 17:28:51,148][105692] Updated weights for policy 0, policy_version 290810 (0.0010) [2023-12-26 17:28:51,209][105692] Updated weights for policy 0, policy_version 290820 (0.0011) [2023-12-26 17:28:51,299][105620] Updated weights for policy 1, policy_version 291017 (0.0009) [2023-12-26 17:28:51,363][105620] Updated weights for policy 1, policy_version 291027 (0.0010) [2023-12-26 17:28:51,427][105620] Updated weights for policy 1, policy_version 291037 (0.0012) [2023-12-26 17:28:51,476][105620] Updated weights for policy 1, policy_version 291047 (0.0010) [2023-12-26 17:28:51,895][105692] Updated weights for policy 0, policy_version 290830 (0.0007) [2023-12-26 17:28:51,948][105692] Updated weights for policy 0, policy_version 290840 (0.0010) [2023-12-26 17:28:52,004][105692] Updated weights for policy 0, policy_version 290850 (0.0010) [2023-12-26 17:28:52,166][105620] Updated weights for policy 1, policy_version 291057 (0.0007) [2023-12-26 17:28:52,229][105620] Updated weights for policy 1, policy_version 291067 (0.0010) [2023-12-26 17:28:52,285][105620] Updated weights for policy 1, policy_version 291077 (0.0009) [2023-12-26 17:28:52,686][105692] Updated weights for policy 0, policy_version 290860 (0.0008) [2023-12-26 17:28:52,746][105692] Updated weights for policy 0, policy_version 290870 (0.0005) [2023-12-26 17:28:52,802][105692] Updated weights for policy 0, policy_version 290880 (0.0006) [2023-12-26 17:28:52,993][105620] Updated weights for policy 1, policy_version 291087 (0.0010) [2023-12-26 17:28:53,052][105620] Updated weights for policy 1, policy_version 291097 (0.0010) [2023-12-26 17:28:53,104][105620] Updated weights for policy 1, policy_version 291107 (0.0010) [2023-12-26 17:28:53,446][105692] Updated weights for policy 0, policy_version 290890 (0.0006) [2023-12-26 17:28:53,515][105692] Updated weights for policy 0, policy_version 290900 (0.0009) [2023-12-26 17:28:53,578][105692] Updated weights for policy 0, policy_version 290910 (0.0009) [2023-12-26 17:28:53,649][105692] Updated weights for policy 0, policy_version 290920 (0.0005) [2023-12-26 17:28:53,701][105620] Updated weights for policy 1, policy_version 291117 (0.0008) [2023-12-26 17:28:53,763][105620] Updated weights for policy 1, policy_version 291127 (0.0005) [2023-12-26 17:28:53,814][105620] Updated weights for policy 1, policy_version 291137 (0.0005) [2023-12-26 17:28:54,252][105692] Updated weights for policy 0, policy_version 290930 (0.0008) [2023-12-26 17:28:54,304][105692] Updated weights for policy 0, policy_version 290940 (0.0006) [2023-12-26 17:28:54,367][105692] Updated weights for policy 0, policy_version 290950 (0.0006) [2023-12-26 17:28:54,439][105620] Updated weights for policy 1, policy_version 291147 (0.0006) [2023-12-26 17:28:54,507][105620] Updated weights for policy 1, policy_version 291157 (0.0009) [2023-12-26 17:28:54,562][105620] Updated weights for policy 1, policy_version 291167 (0.0010) [2023-12-26 17:28:55,012][105692] Updated weights for policy 0, policy_version 290960 (0.0009) [2023-12-26 17:28:55,064][105692] Updated weights for policy 0, policy_version 290970 (0.0009) [2023-12-26 17:28:55,118][105692] Updated weights for policy 0, policy_version 290980 (0.0008) [2023-12-26 17:28:55,208][105620] Updated weights for policy 1, policy_version 291177 (0.0009) [2023-12-26 17:28:55,260][105620] Updated weights for policy 1, policy_version 291187 (0.0010) [2023-12-26 17:28:55,317][105620] Updated weights for policy 1, policy_version 291197 (0.0010) [2023-12-26 17:28:55,376][105620] Updated weights for policy 1, policy_version 291207 (0.0010) [2023-12-26 17:28:55,699][105692] Updated weights for policy 0, policy_version 290990 (0.0006) [2023-12-26 17:28:55,753][105692] Updated weights for policy 0, policy_version 291002 (0.0011) [2023-12-26 17:28:55,811][105692] Updated weights for policy 0, policy_version 291013 (0.0011) [2023-12-26 17:28:56,007][105620] Updated weights for policy 1, policy_version 291217 (0.0006) [2023-12-26 17:28:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 149069824. Throughput: 0: 9546.1, 1: 10223.4. Samples: 149080072. Policy #0 lag: (min: 31.0, avg: 32.4, max: 60.0) [2023-12-26 17:28:56,063][104569] Avg episode reward: [(0, '9357.904'), (1, '5645.809')] [2023-12-26 17:28:56,076][105620] Updated weights for policy 1, policy_version 291227 (0.0011) [2023-12-26 17:28:56,133][105620] Updated weights for policy 1, policy_version 291237 (0.0011) [2023-12-26 17:28:56,479][105692] Updated weights for policy 0, policy_version 291023 (0.0006) [2023-12-26 17:28:56,526][105692] Updated weights for policy 0, policy_version 291033 (0.0007) [2023-12-26 17:28:56,579][105692] Updated weights for policy 0, policy_version 291043 (0.0009) [2023-12-26 17:28:56,826][105620] Updated weights for policy 1, policy_version 291247 (0.0010) [2023-12-26 17:28:56,870][105620] Updated weights for policy 1, policy_version 291257 (0.0010) [2023-12-26 17:28:56,914][105620] Updated weights for policy 1, policy_version 291267 (0.0010) [2023-12-26 17:28:57,165][105692] Updated weights for policy 0, policy_version 291053 (0.0005) [2023-12-26 17:28:57,210][105692] Updated weights for policy 0, policy_version 291063 (0.0005) [2023-12-26 17:28:57,257][105692] Updated weights for policy 0, policy_version 291073 (0.0005) [2023-12-26 17:28:57,636][105620] Updated weights for policy 1, policy_version 291277 (0.0008) [2023-12-26 17:28:57,706][105620] Updated weights for policy 1, policy_version 291287 (0.0005) [2023-12-26 17:28:57,756][105620] Updated weights for policy 1, policy_version 291297 (0.0005) [2023-12-26 17:28:57,812][105692] Updated weights for policy 0, policy_version 291083 (0.0007) [2023-12-26 17:28:57,869][105692] Updated weights for policy 0, policy_version 291093 (0.0010) [2023-12-26 17:28:57,920][105692] Updated weights for policy 0, policy_version 291103 (0.0010) [2023-12-26 17:28:58,420][105620] Updated weights for policy 1, policy_version 291307 (0.0009) [2023-12-26 17:28:58,480][105620] Updated weights for policy 1, policy_version 291317 (0.0011) [2023-12-26 17:28:58,553][105620] Updated weights for policy 1, policy_version 291327 (0.0010) [2023-12-26 17:28:58,693][105692] Updated weights for policy 0, policy_version 291113 (0.0010) [2023-12-26 17:28:58,762][105692] Updated weights for policy 0, policy_version 291123 (0.0007) [2023-12-26 17:28:58,836][105692] Updated weights for policy 0, policy_version 291133 (0.0008) [2023-12-26 17:28:58,894][105692] Updated weights for policy 0, policy_version 291143 (0.0007) [2023-12-26 17:28:59,437][105620] Updated weights for policy 1, policy_version 291337 (0.0011) [2023-12-26 17:28:59,496][105620] Updated weights for policy 1, policy_version 291347 (0.0008) [2023-12-26 17:28:59,521][105692] Updated weights for policy 0, policy_version 291153 (0.0008) [2023-12-26 17:28:59,549][105620] Updated weights for policy 1, policy_version 291357 (0.0008) [2023-12-26 17:28:59,551][105586] KL-divergence is very high: 214.3848 [2023-12-26 17:28:59,582][105692] Updated weights for policy 0, policy_version 291163 (0.0008) [2023-12-26 17:28:59,594][105586] KL-divergence is very high: 406.4977 [2023-12-26 17:28:59,600][105586] KL-divergence is very high: 168.1194 [2023-12-26 17:28:59,603][105620] Updated weights for policy 1, policy_version 291367 (0.0006) [2023-12-26 17:28:59,635][105692] Updated weights for policy 0, policy_version 291173 (0.0008) [2023-12-26 17:29:00,384][105620] Updated weights for policy 1, policy_version 291377 (0.0005) [2023-12-26 17:29:00,445][105692] Updated weights for policy 0, policy_version 291183 (0.0009) [2023-12-26 17:29:00,449][105620] Updated weights for policy 1, policy_version 291387 (0.0008) [2023-12-26 17:29:00,500][105692] Updated weights for policy 0, policy_version 291193 (0.0008) [2023-12-26 17:29:00,511][105620] Updated weights for policy 1, policy_version 291397 (0.0007) [2023-12-26 17:29:00,558][105692] Updated weights for policy 0, policy_version 291203 (0.0009) [2023-12-26 17:29:01,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 149168128. Throughput: 0: 9681.4, 1: 10180.6. Samples: 149141892. Policy #0 lag: (min: 31.0, avg: 32.4, max: 60.0) [2023-12-26 17:29:01,062][104569] Avg episode reward: [(0, '9357.712'), (1, '7131.998')] [2023-12-26 17:29:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000291208_74563584.pth... [2023-12-26 17:29:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000291400_74604544.pth... [2023-12-26 17:29:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000290088_74276864.pth [2023-12-26 17:29:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000290216_74301440.pth [2023-12-26 17:29:01,183][105620] Updated weights for policy 1, policy_version 291407 (0.0007) [2023-12-26 17:29:01,236][105620] Updated weights for policy 1, policy_version 291417 (0.0008) [2023-12-26 17:29:01,243][105692] Updated weights for policy 0, policy_version 291213 (0.0010) [2023-12-26 17:29:01,291][105620] Updated weights for policy 1, policy_version 291427 (0.0007) [2023-12-26 17:29:01,304][105692] Updated weights for policy 0, policy_version 291223 (0.0011) [2023-12-26 17:29:01,366][105692] Updated weights for policy 0, policy_version 291233 (0.0009) [2023-12-26 17:29:02,012][105692] Updated weights for policy 0, policy_version 291243 (0.0009) [2023-12-26 17:29:02,067][105692] Updated weights for policy 0, policy_version 291253 (0.0010) [2023-12-26 17:29:02,104][105620] Updated weights for policy 1, policy_version 291437 (0.0007) [2023-12-26 17:29:02,120][105692] Updated weights for policy 0, policy_version 291263 (0.0011) [2023-12-26 17:29:02,149][105620] Updated weights for policy 1, policy_version 291447 (0.0005) [2023-12-26 17:29:02,198][105620] Updated weights for policy 1, policy_version 291457 (0.0006) [2023-12-26 17:29:02,791][105692] Updated weights for policy 0, policy_version 291273 (0.0010) [2023-12-26 17:29:02,852][105692] Updated weights for policy 0, policy_version 291283 (0.0005) [2023-12-26 17:29:02,910][105692] Updated weights for policy 0, policy_version 291293 (0.0009) [2023-12-26 17:29:02,951][105620] Updated weights for policy 1, policy_version 291467 (0.0006) [2023-12-26 17:29:02,964][105692] Updated weights for policy 0, policy_version 291303 (0.0010) [2023-12-26 17:29:03,008][105620] Updated weights for policy 1, policy_version 291477 (0.0007) [2023-12-26 17:29:03,053][105620] Updated weights for policy 1, policy_version 291487 (0.0008) [2023-12-26 17:29:03,676][105692] Updated weights for policy 0, policy_version 291313 (0.0010) [2023-12-26 17:29:03,736][105692] Updated weights for policy 0, policy_version 291323 (0.0010) [2023-12-26 17:29:03,800][105692] Updated weights for policy 0, policy_version 291333 (0.0010) [2023-12-26 17:29:03,831][105620] Updated weights for policy 1, policy_version 291497 (0.0008) [2023-12-26 17:29:03,900][105620] Updated weights for policy 1, policy_version 291507 (0.0007) [2023-12-26 17:29:03,960][105620] Updated weights for policy 1, policy_version 291517 (0.0009) [2023-12-26 17:29:04,022][105620] Updated weights for policy 1, policy_version 291527 (0.0010) [2023-12-26 17:29:04,475][105692] Updated weights for policy 0, policy_version 291343 (0.0009) [2023-12-26 17:29:04,534][105692] Updated weights for policy 0, policy_version 291353 (0.0007) [2023-12-26 17:29:04,597][105692] Updated weights for policy 0, policy_version 291363 (0.0006) [2023-12-26 17:29:04,795][105620] Updated weights for policy 1, policy_version 291537 (0.0008) [2023-12-26 17:29:04,860][105620] Updated weights for policy 1, policy_version 291547 (0.0009) [2023-12-26 17:29:04,919][105620] Updated weights for policy 1, policy_version 291557 (0.0009) [2023-12-26 17:29:04,927][105586] KL-divergence is very high: 142.7362 [2023-12-26 17:29:05,232][105692] Updated weights for policy 0, policy_version 291373 (0.0010) [2023-12-26 17:29:05,294][105692] Updated weights for policy 0, policy_version 291383 (0.0010) [2023-12-26 17:29:05,347][105692] Updated weights for policy 0, policy_version 291393 (0.0010) [2023-12-26 17:29:05,542][105620] Updated weights for policy 1, policy_version 291567 (0.0006) [2023-12-26 17:29:05,607][105620] Updated weights for policy 1, policy_version 291577 (0.0009) [2023-12-26 17:29:05,639][105586] KL-divergence is very high: 118.3400 [2023-12-26 17:29:05,645][105586] KL-divergence is very high: 114.0847 [2023-12-26 17:29:05,668][105620] Updated weights for policy 1, policy_version 291587 (0.0010) [2023-12-26 17:29:05,684][105586] KL-divergence is very high: 116.1933 [2023-12-26 17:29:05,689][105586] KL-divergence is very high: 104.4978 [2023-12-26 17:29:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 149266432. Throughput: 0: 9666.9, 1: 10066.6. Samples: 149256352. Policy #0 lag: (min: 31.0, avg: 32.4, max: 60.0) [2023-12-26 17:29:06,063][104569] Avg episode reward: [(0, '9357.908'), (1, '6204.128')] [2023-12-26 17:29:06,124][105692] Updated weights for policy 0, policy_version 291403 (0.0010) [2023-12-26 17:29:06,178][105692] Updated weights for policy 0, policy_version 291413 (0.0008) [2023-12-26 17:29:06,242][105692] Updated weights for policy 0, policy_version 291423 (0.0008) [2023-12-26 17:29:06,346][105586] KL-divergence is very high: 133.0240 [2023-12-26 17:29:06,367][105620] Updated weights for policy 1, policy_version 291597 (0.0010) [2023-12-26 17:29:06,416][105620] Updated weights for policy 1, policy_version 291607 (0.0010) [2023-12-26 17:29:06,474][105620] Updated weights for policy 1, policy_version 291617 (0.0010) [2023-12-26 17:29:06,911][105692] Updated weights for policy 0, policy_version 291433 (0.0008) [2023-12-26 17:29:06,969][105692] Updated weights for policy 0, policy_version 291443 (0.0005) [2023-12-26 17:29:07,031][105692] Updated weights for policy 0, policy_version 291453 (0.0005) [2023-12-26 17:29:07,097][105692] Updated weights for policy 0, policy_version 291463 (0.0006) [2023-12-26 17:29:07,231][105620] Updated weights for policy 1, policy_version 291627 (0.0010) [2023-12-26 17:29:07,279][105620] Updated weights for policy 1, policy_version 291637 (0.0010) [2023-12-26 17:29:07,327][105620] Updated weights for policy 1, policy_version 291647 (0.0010) [2023-12-26 17:29:07,717][105692] Updated weights for policy 0, policy_version 291473 (0.0008) [2023-12-26 17:29:07,765][105692] Updated weights for policy 0, policy_version 291483 (0.0008) [2023-12-26 17:29:07,812][105692] Updated weights for policy 0, policy_version 291493 (0.0008) [2023-12-26 17:29:08,088][105620] Updated weights for policy 1, policy_version 291657 (0.0010) [2023-12-26 17:29:08,146][105620] Updated weights for policy 1, policy_version 291667 (0.0010) [2023-12-26 17:29:08,200][105620] Updated weights for policy 1, policy_version 291677 (0.0010) [2023-12-26 17:29:08,257][105620] Updated weights for policy 1, policy_version 291687 (0.0010) [2023-12-26 17:29:08,575][105692] Updated weights for policy 0, policy_version 291503 (0.0008) [2023-12-26 17:29:08,624][105692] Updated weights for policy 0, policy_version 291513 (0.0008) [2023-12-26 17:29:08,677][105692] Updated weights for policy 0, policy_version 291523 (0.0008) [2023-12-26 17:29:08,997][105620] Updated weights for policy 1, policy_version 291697 (0.0010) [2023-12-26 17:29:09,052][105620] Updated weights for policy 1, policy_version 291707 (0.0010) [2023-12-26 17:29:09,108][105620] Updated weights for policy 1, policy_version 291717 (0.0010) [2023-12-26 17:29:09,458][105692] Updated weights for policy 0, policy_version 291533 (0.0009) [2023-12-26 17:29:09,508][105692] Updated weights for policy 0, policy_version 291543 (0.0009) [2023-12-26 17:29:09,558][105692] Updated weights for policy 0, policy_version 291553 (0.0009) [2023-12-26 17:29:09,889][105620] Updated weights for policy 1, policy_version 291727 (0.0009) [2023-12-26 17:29:09,954][105620] Updated weights for policy 1, policy_version 291737 (0.0011) [2023-12-26 17:29:10,022][105620] Updated weights for policy 1, policy_version 291747 (0.0011) [2023-12-26 17:29:10,354][105692] Updated weights for policy 0, policy_version 291563 (0.0009) [2023-12-26 17:29:10,409][105692] Updated weights for policy 0, policy_version 291573 (0.0010) [2023-12-26 17:29:10,468][105692] Updated weights for policy 0, policy_version 291583 (0.0007) [2023-12-26 17:29:10,763][105620] Updated weights for policy 1, policy_version 291757 (0.0010) [2023-12-26 17:29:10,768][105586] KL-divergence is very high: 179.7713 [2023-12-26 17:29:10,778][105586] KL-divergence is very high: 213.2121 [2023-12-26 17:29:10,783][105586] KL-divergence is very high: 341.3498 [2023-12-26 17:29:10,798][105586] KL-divergence is very high: 183.6221 [2023-12-26 17:29:10,809][105586] KL-divergence is very high: 427.2587 [2023-12-26 17:29:10,814][105586] KL-divergence is very high: 130.0342 [2023-12-26 17:29:10,815][105620] Updated weights for policy 1, policy_version 291767 (0.0010) [2023-12-26 17:29:10,820][105586] KL-divergence is very high: 291.1812 [2023-12-26 17:29:10,826][105586] KL-divergence is very high: 472.3409 [2023-12-26 17:29:10,842][105586] KL-divergence is very high: 164.2446 [2023-12-26 17:29:10,853][105586] KL-divergence is very high: 336.7491 [2023-12-26 17:29:10,865][105586] KL-divergence is very high: 191.7036 [2023-12-26 17:29:10,871][105586] KL-divergence is very high: 395.1623 [2023-12-26 17:29:10,873][105620] Updated weights for policy 1, policy_version 291777 (0.0010) [2023-12-26 17:29:10,903][105586] KL-divergence is very high: 174.1909 [2023-12-26 17:29:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 149364736. Throughput: 0: 9658.2, 1: 10035.7. Samples: 149372196. Policy #0 lag: (min: 31.0, avg: 32.4, max: 60.0) [2023-12-26 17:29:11,062][104569] Avg episode reward: [(0, '9358.396'), (1, '5925.109')] [2023-12-26 17:29:11,209][105692] Updated weights for policy 0, policy_version 291593 (0.0008) [2023-12-26 17:29:11,275][105692] Updated weights for policy 0, policy_version 291603 (0.0008) [2023-12-26 17:29:11,341][105692] Updated weights for policy 0, policy_version 291613 (0.0008) [2023-12-26 17:29:11,411][105692] Updated weights for policy 0, policy_version 291623 (0.0008) [2023-12-26 17:29:11,626][105586] KL-divergence is very high: 143.3878 [2023-12-26 17:29:11,646][105620] Updated weights for policy 1, policy_version 291787 (0.0010) [2023-12-26 17:29:11,715][105620] Updated weights for policy 1, policy_version 291797 (0.0009) [2023-12-26 17:29:11,790][105620] Updated weights for policy 1, policy_version 291807 (0.0010) [2023-12-26 17:29:12,048][105692] Updated weights for policy 0, policy_version 291633 (0.0010) [2023-12-26 17:29:12,105][105692] Updated weights for policy 0, policy_version 291643 (0.0009) [2023-12-26 17:29:12,153][105692] Updated weights for policy 0, policy_version 291653 (0.0010) [2023-12-26 17:29:12,582][105620] Updated weights for policy 1, policy_version 291817 (0.0010) [2023-12-26 17:29:12,649][105620] Updated weights for policy 1, policy_version 291827 (0.0008) [2023-12-26 17:29:12,721][105620] Updated weights for policy 1, policy_version 291837 (0.0010) [2023-12-26 17:29:12,785][105620] Updated weights for policy 1, policy_version 291847 (0.0010) [2023-12-26 17:29:12,865][105692] Updated weights for policy 0, policy_version 291663 (0.0007) [2023-12-26 17:29:12,927][105692] Updated weights for policy 0, policy_version 291673 (0.0009) [2023-12-26 17:29:12,985][105692] Updated weights for policy 0, policy_version 291683 (0.0010) [2023-12-26 17:29:13,536][105620] Updated weights for policy 1, policy_version 291857 (0.0006) [2023-12-26 17:29:13,593][105620] Updated weights for policy 1, policy_version 291867 (0.0005) [2023-12-26 17:29:13,648][105692] Updated weights for policy 0, policy_version 291693 (0.0010) [2023-12-26 17:29:13,651][105620] Updated weights for policy 1, policy_version 291877 (0.0005) [2023-12-26 17:29:13,703][105692] Updated weights for policy 0, policy_version 291703 (0.0010) [2023-12-26 17:29:13,767][105692] Updated weights for policy 0, policy_version 291713 (0.0010) [2023-12-26 17:29:14,209][105620] Updated weights for policy 1, policy_version 291887 (0.0005) [2023-12-26 17:29:14,274][105620] Updated weights for policy 1, policy_version 291897 (0.0005) [2023-12-26 17:29:14,337][105620] Updated weights for policy 1, policy_version 291907 (0.0006) [2023-12-26 17:29:14,424][105692] Updated weights for policy 0, policy_version 291723 (0.0007) [2023-12-26 17:29:14,480][105692] Updated weights for policy 0, policy_version 291733 (0.0009) [2023-12-26 17:29:14,542][105692] Updated weights for policy 0, policy_version 291743 (0.0009) [2023-12-26 17:29:14,864][105620] Updated weights for policy 1, policy_version 291917 (0.0008) [2023-12-26 17:29:14,926][105620] Updated weights for policy 1, policy_version 291927 (0.0010) [2023-12-26 17:29:14,986][105620] Updated weights for policy 1, policy_version 291937 (0.0011) [2023-12-26 17:29:15,239][105692] Updated weights for policy 0, policy_version 291753 (0.0010) [2023-12-26 17:29:15,306][105692] Updated weights for policy 0, policy_version 291763 (0.0011) [2023-12-26 17:29:15,368][105692] Updated weights for policy 0, policy_version 291773 (0.0011) [2023-12-26 17:29:15,424][105692] Updated weights for policy 0, policy_version 291783 (0.0009) [2023-12-26 17:29:15,649][105620] Updated weights for policy 1, policy_version 291947 (0.0010) [2023-12-26 17:29:15,711][105620] Updated weights for policy 1, policy_version 291957 (0.0008) [2023-12-26 17:29:15,777][105620] Updated weights for policy 1, policy_version 291967 (0.0007) [2023-12-26 17:29:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 149463040. Throughput: 0: 9668.1, 1: 9943.0. Samples: 149430108. Policy #0 lag: (min: 31.0, avg: 32.4, max: 60.0) [2023-12-26 17:29:16,062][104569] Avg episode reward: [(0, '9358.544'), (1, '6016.354')] [2023-12-26 17:29:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000291976_74752000.pth... [2023-12-26 17:29:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000290824_74457088.pth [2023-12-26 17:29:16,090][105692] Updated weights for policy 0, policy_version 291793 (0.0009) [2023-12-26 17:29:16,143][105692] Updated weights for policy 0, policy_version 291803 (0.0010) [2023-12-26 17:29:16,191][105692] Updated weights for policy 0, policy_version 291813 (0.0009) [2023-12-26 17:29:16,205][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000291816_74719232.pth... [2023-12-26 17:29:16,208][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000290632_74416128.pth [2023-12-26 17:29:16,486][105620] Updated weights for policy 1, policy_version 291977 (0.0009) [2023-12-26 17:29:16,536][105620] Updated weights for policy 1, policy_version 291987 (0.0009) [2023-12-26 17:29:16,582][105620] Updated weights for policy 1, policy_version 291997 (0.0009) [2023-12-26 17:29:16,637][105620] Updated weights for policy 1, policy_version 292007 (0.0009) [2023-12-26 17:29:17,003][105692] Updated weights for policy 0, policy_version 291823 (0.0007) [2023-12-26 17:29:17,061][105692] Updated weights for policy 0, policy_version 291833 (0.0005) [2023-12-26 17:29:17,125][105692] Updated weights for policy 0, policy_version 291843 (0.0005) [2023-12-26 17:29:17,380][105620] Updated weights for policy 1, policy_version 292017 (0.0006) [2023-12-26 17:29:17,445][105620] Updated weights for policy 1, policy_version 292027 (0.0005) [2023-12-26 17:29:17,500][105620] Updated weights for policy 1, policy_version 292037 (0.0005) [2023-12-26 17:29:17,837][105692] Updated weights for policy 0, policy_version 291853 (0.0007) [2023-12-26 17:29:17,896][105692] Updated weights for policy 0, policy_version 291863 (0.0007) [2023-12-26 17:29:17,942][105692] Updated weights for policy 0, policy_version 291873 (0.0005) [2023-12-26 17:29:18,114][105620] Updated weights for policy 1, policy_version 292047 (0.0009) [2023-12-26 17:29:18,166][105620] Updated weights for policy 1, policy_version 292057 (0.0011) [2023-12-26 17:29:18,215][105620] Updated weights for policy 1, policy_version 292067 (0.0010) [2023-12-26 17:29:18,601][105692] Updated weights for policy 0, policy_version 291883 (0.0005) [2023-12-26 17:29:18,659][105692] Updated weights for policy 0, policy_version 291893 (0.0006) [2023-12-26 17:29:18,711][105692] Updated weights for policy 0, policy_version 291903 (0.0008) [2023-12-26 17:29:18,972][105620] Updated weights for policy 1, policy_version 292077 (0.0011) [2023-12-26 17:29:19,032][105620] Updated weights for policy 1, policy_version 292087 (0.0011) [2023-12-26 17:29:19,088][105620] Updated weights for policy 1, policy_version 292097 (0.0010) [2023-12-26 17:29:19,482][105692] Updated weights for policy 0, policy_version 291913 (0.0008) [2023-12-26 17:29:19,547][105692] Updated weights for policy 0, policy_version 291923 (0.0008) [2023-12-26 17:29:19,603][105692] Updated weights for policy 0, policy_version 291933 (0.0009) [2023-12-26 17:29:19,663][105692] Updated weights for policy 0, policy_version 291943 (0.0007) [2023-12-26 17:29:19,796][105620] Updated weights for policy 1, policy_version 292107 (0.0011) [2023-12-26 17:29:19,856][105620] Updated weights for policy 1, policy_version 292117 (0.0009) [2023-12-26 17:29:19,910][105620] Updated weights for policy 1, policy_version 292127 (0.0009) [2023-12-26 17:29:20,335][105692] Updated weights for policy 0, policy_version 291953 (0.0006) [2023-12-26 17:29:20,393][105692] Updated weights for policy 0, policy_version 291963 (0.0008) [2023-12-26 17:29:20,457][105692] Updated weights for policy 0, policy_version 291973 (0.0005) [2023-12-26 17:29:20,664][105620] Updated weights for policy 1, policy_version 292137 (0.0007) [2023-12-26 17:29:20,730][105620] Updated weights for policy 1, policy_version 292147 (0.0010) [2023-12-26 17:29:20,794][105620] Updated weights for policy 1, policy_version 292157 (0.0009) [2023-12-26 17:29:20,862][105620] Updated weights for policy 1, policy_version 292167 (0.0010) [2023-12-26 17:29:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 149561344. Throughput: 0: 9695.9, 1: 9932.3. Samples: 149550068. Policy #0 lag: (min: 31.0, avg: 32.4, max: 60.0) [2023-12-26 17:29:21,063][104569] Avg episode reward: [(0, '9358.538'), (1, '6202.419')] [2023-12-26 17:29:21,191][105692] Updated weights for policy 0, policy_version 291983 (0.0007) [2023-12-26 17:29:21,259][105692] Updated weights for policy 0, policy_version 291993 (0.0009) [2023-12-26 17:29:21,317][105692] Updated weights for policy 0, policy_version 292003 (0.0008) [2023-12-26 17:29:21,674][105620] Updated weights for policy 1, policy_version 292177 (0.0008) [2023-12-26 17:29:21,736][105620] Updated weights for policy 1, policy_version 292187 (0.0009) [2023-12-26 17:29:21,800][105620] Updated weights for policy 1, policy_version 292197 (0.0010) [2023-12-26 17:29:22,034][105692] Updated weights for policy 0, policy_version 292013 (0.0008) [2023-12-26 17:29:22,097][105692] Updated weights for policy 0, policy_version 292023 (0.0009) [2023-12-26 17:29:22,154][105692] Updated weights for policy 0, policy_version 292033 (0.0009) [2023-12-26 17:29:22,555][105620] Updated weights for policy 1, policy_version 292207 (0.0008) [2023-12-26 17:29:22,618][105620] Updated weights for policy 1, policy_version 292217 (0.0008) [2023-12-26 17:29:22,677][105620] Updated weights for policy 1, policy_version 292227 (0.0006) [2023-12-26 17:29:22,942][105692] Updated weights for policy 0, policy_version 292043 (0.0010) [2023-12-26 17:29:23,011][105692] Updated weights for policy 0, policy_version 292053 (0.0007) [2023-12-26 17:29:23,075][105692] Updated weights for policy 0, policy_version 292063 (0.0005) [2023-12-26 17:29:23,479][105620] Updated weights for policy 1, policy_version 292237 (0.0010) [2023-12-26 17:29:23,537][105586] KL-divergence is very high: 133.0798 [2023-12-26 17:29:23,537][105620] Updated weights for policy 1, policy_version 292248 (0.0010) [2023-12-26 17:29:23,578][105586] KL-divergence is very high: 315.5455 [2023-12-26 17:29:23,588][105586] KL-divergence is very high: 120.5566 [2023-12-26 17:29:23,588][105620] Updated weights for policy 1, policy_version 292258 (0.0009) [2023-12-26 17:29:23,691][105692] Updated weights for policy 0, policy_version 292073 (0.0006) [2023-12-26 17:29:23,755][105692] Updated weights for policy 0, policy_version 292083 (0.0006) [2023-12-26 17:29:23,813][105692] Updated weights for policy 0, policy_version 292093 (0.0006) [2023-12-26 17:29:23,870][105692] Updated weights for policy 0, policy_version 292103 (0.0008) [2023-12-26 17:29:24,276][105620] Updated weights for policy 1, policy_version 292268 (0.0010) [2023-12-26 17:29:24,345][105620] Updated weights for policy 1, policy_version 292278 (0.0010) [2023-12-26 17:29:24,414][105620] Updated weights for policy 1, policy_version 292288 (0.0009) [2023-12-26 17:29:24,415][105586] KL-divergence is very high: 182.2801 [2023-12-26 17:29:24,451][105586] KL-divergence is very high: 114.4577 [2023-12-26 17:29:24,582][105692] Updated weights for policy 0, policy_version 292113 (0.0010) [2023-12-26 17:29:24,636][105692] Updated weights for policy 0, policy_version 292123 (0.0009) [2023-12-26 17:29:24,689][105692] Updated weights for policy 0, policy_version 292134 (0.0009) [2023-12-26 17:29:25,015][105620] Updated weights for policy 1, policy_version 292298 (0.0008) [2023-12-26 17:29:25,069][105620] Updated weights for policy 1, policy_version 292308 (0.0005) [2023-12-26 17:29:25,126][105620] Updated weights for policy 1, policy_version 292318 (0.0006) [2023-12-26 17:29:25,181][105620] Updated weights for policy 1, policy_version 292328 (0.0006) [2023-12-26 17:29:25,501][105692] Updated weights for policy 0, policy_version 292144 (0.0008) [2023-12-26 17:29:25,562][105692] Updated weights for policy 0, policy_version 292154 (0.0009) [2023-12-26 17:29:25,613][105692] Updated weights for policy 0, policy_version 292164 (0.0009) [2023-12-26 17:29:25,771][105620] Updated weights for policy 1, policy_version 292338 (0.0010) [2023-12-26 17:29:25,824][105620] Updated weights for policy 1, policy_version 292348 (0.0009) [2023-12-26 17:29:25,877][105620] Updated weights for policy 1, policy_version 292358 (0.0010) [2023-12-26 17:29:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 149659648. Throughput: 0: 9805.3, 1: 9831.5. Samples: 149665432. Policy #0 lag: (min: 31.0, avg: 32.4, max: 60.0) [2023-12-26 17:29:26,062][104569] Avg episode reward: [(0, '9268.586'), (1, '6756.835')] [2023-12-26 17:29:26,235][105692] Updated weights for policy 0, policy_version 292174 (0.0007) [2023-12-26 17:29:26,303][105692] Updated weights for policy 0, policy_version 292184 (0.0008) [2023-12-26 17:29:26,356][105692] Updated weights for policy 0, policy_version 292194 (0.0009) [2023-12-26 17:29:26,673][105620] Updated weights for policy 1, policy_version 292369 (0.0009) [2023-12-26 17:29:26,722][105620] Updated weights for policy 1, policy_version 292379 (0.0008) [2023-12-26 17:29:26,769][105620] Updated weights for policy 1, policy_version 292389 (0.0009) [2023-12-26 17:29:26,987][105692] Updated weights for policy 0, policy_version 292204 (0.0007) [2023-12-26 17:29:27,074][105692] Updated weights for policy 0, policy_version 292214 (0.0005) [2023-12-26 17:29:27,134][105692] Updated weights for policy 0, policy_version 292224 (0.0008) [2023-12-26 17:29:27,520][105620] Updated weights for policy 1, policy_version 292399 (0.0006) [2023-12-26 17:29:27,573][105620] Updated weights for policy 1, policy_version 292409 (0.0005) [2023-12-26 17:29:27,635][105692] Updated weights for policy 0, policy_version 292234 (0.0009) [2023-12-26 17:29:27,653][105620] Updated weights for policy 1, policy_version 292419 (0.0009) [2023-12-26 17:29:27,689][105692] Updated weights for policy 0, policy_version 292244 (0.0006) [2023-12-26 17:29:27,739][105692] Updated weights for policy 0, policy_version 292254 (0.0007) [2023-12-26 17:29:27,794][105692] Updated weights for policy 0, policy_version 292264 (0.0010) [2023-12-26 17:29:28,268][105620] Updated weights for policy 1, policy_version 292429 (0.0009) [2023-12-26 17:29:28,312][105620] Updated weights for policy 1, policy_version 292439 (0.0010) [2023-12-26 17:29:28,371][105620] Updated weights for policy 1, policy_version 292449 (0.0008) [2023-12-26 17:29:28,453][105692] Updated weights for policy 0, policy_version 292274 (0.0008) [2023-12-26 17:29:28,500][105692] Updated weights for policy 0, policy_version 292284 (0.0008) [2023-12-26 17:29:28,552][105692] Updated weights for policy 0, policy_version 292294 (0.0008) [2023-12-26 17:29:29,082][105620] Updated weights for policy 1, policy_version 292459 (0.0008) [2023-12-26 17:29:29,127][105620] Updated weights for policy 1, policy_version 292469 (0.0006) [2023-12-26 17:29:29,182][105620] Updated weights for policy 1, policy_version 292479 (0.0006) [2023-12-26 17:29:29,331][105692] Updated weights for policy 0, policy_version 292304 (0.0007) [2023-12-26 17:29:29,396][105692] Updated weights for policy 0, policy_version 292314 (0.0008) [2023-12-26 17:29:29,459][105692] Updated weights for policy 0, policy_version 292324 (0.0008) [2023-12-26 17:29:29,912][105620] Updated weights for policy 1, policy_version 292489 (0.0010) [2023-12-26 17:29:29,973][105620] Updated weights for policy 1, policy_version 292499 (0.0009) [2023-12-26 17:29:30,035][105620] Updated weights for policy 1, policy_version 292509 (0.0009) [2023-12-26 17:29:30,096][105620] Updated weights for policy 1, policy_version 292519 (0.0009) [2023-12-26 17:29:30,142][105692] Updated weights for policy 0, policy_version 292334 (0.0008) [2023-12-26 17:29:30,203][105692] Updated weights for policy 0, policy_version 292344 (0.0009) [2023-12-26 17:29:30,256][105692] Updated weights for policy 0, policy_version 292354 (0.0009) [2023-12-26 17:29:30,793][105620] Updated weights for policy 1, policy_version 292529 (0.0009) [2023-12-26 17:29:30,851][105620] Updated weights for policy 1, policy_version 292539 (0.0009) [2023-12-26 17:29:30,908][105620] Updated weights for policy 1, policy_version 292549 (0.0006) [2023-12-26 17:29:31,025][105692] Updated weights for policy 0, policy_version 292364 (0.0009) [2023-12-26 17:29:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 149757952. Throughput: 0: 9968.4, 1: 9792.7. Samples: 149727976. Policy #0 lag: (min: 31.0, avg: 32.4, max: 60.0) [2023-12-26 17:29:31,063][104569] Avg episode reward: [(0, '9268.212'), (1, '6665.148')] [2023-12-26 17:29:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000292552_74899456.pth... [2023-12-26 17:29:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000291400_74604544.pth [2023-12-26 17:29:31,079][105692] Updated weights for policy 0, policy_version 292374 (0.0009) [2023-12-26 17:29:31,137][105692] Updated weights for policy 0, policy_version 292384 (0.0008) [2023-12-26 17:29:31,173][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000292392_74866688.pth... [2023-12-26 17:29:31,176][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000291208_74563584.pth [2023-12-26 17:29:31,640][105620] Updated weights for policy 1, policy_version 292559 (0.0006) [2023-12-26 17:29:31,693][105620] Updated weights for policy 1, policy_version 292569 (0.0006) [2023-12-26 17:29:31,756][105620] Updated weights for policy 1, policy_version 292579 (0.0009) [2023-12-26 17:29:31,912][105692] Updated weights for policy 0, policy_version 292394 (0.0006) [2023-12-26 17:29:31,976][105692] Updated weights for policy 0, policy_version 292404 (0.0009) [2023-12-26 17:29:32,027][105692] Updated weights for policy 0, policy_version 292414 (0.0009) [2023-12-26 17:29:32,073][105692] Updated weights for policy 0, policy_version 292424 (0.0008) [2023-12-26 17:29:32,474][105620] Updated weights for policy 1, policy_version 292589 (0.0009) [2023-12-26 17:29:32,541][105620] Updated weights for policy 1, policy_version 292599 (0.0009) [2023-12-26 17:29:32,600][105620] Updated weights for policy 1, policy_version 292609 (0.0009) [2023-12-26 17:29:32,856][105692] Updated weights for policy 0, policy_version 292434 (0.0008) [2023-12-26 17:29:32,906][105692] Updated weights for policy 0, policy_version 292444 (0.0010) [2023-12-26 17:29:32,947][105692] Updated weights for policy 0, policy_version 292454 (0.0009) [2023-12-26 17:29:33,329][105620] Updated weights for policy 1, policy_version 292619 (0.0008) [2023-12-26 17:29:33,390][105620] Updated weights for policy 1, policy_version 292629 (0.0008) [2023-12-26 17:29:33,457][105620] Updated weights for policy 1, policy_version 292639 (0.0008) [2023-12-26 17:29:33,608][105692] Updated weights for policy 0, policy_version 292464 (0.0009) [2023-12-26 17:29:33,652][105692] Updated weights for policy 0, policy_version 292474 (0.0010) [2023-12-26 17:29:33,695][105692] Updated weights for policy 0, policy_version 292484 (0.0007) [2023-12-26 17:29:34,053][105620] Updated weights for policy 1, policy_version 292649 (0.0008) [2023-12-26 17:29:34,101][105620] Updated weights for policy 1, policy_version 292659 (0.0005) [2023-12-26 17:29:34,158][105620] Updated weights for policy 1, policy_version 292669 (0.0007) [2023-12-26 17:29:34,216][105620] Updated weights for policy 1, policy_version 292679 (0.0009) [2023-12-26 17:29:34,333][105692] Updated weights for policy 0, policy_version 292494 (0.0006) [2023-12-26 17:29:34,381][105585] KL-divergence is very high: 156.3258 [2023-12-26 17:29:34,395][105692] Updated weights for policy 0, policy_version 292504 (0.0008) [2023-12-26 17:29:34,435][105585] KL-divergence is very high: 226.3889 [2023-12-26 17:29:34,460][105692] Updated weights for policy 0, policy_version 292514 (0.0011) [2023-12-26 17:29:34,484][105585] KL-divergence is very high: 170.9903 [2023-12-26 17:29:34,968][105620] Updated weights for policy 1, policy_version 292689 (0.0010) [2023-12-26 17:29:35,020][105620] Updated weights for policy 1, policy_version 292699 (0.0006) [2023-12-26 17:29:35,091][105620] Updated weights for policy 1, policy_version 292709 (0.0006) [2023-12-26 17:29:35,121][105692] Updated weights for policy 0, policy_version 292524 (0.0009) [2023-12-26 17:29:35,189][105692] Updated weights for policy 0, policy_version 292534 (0.0006) [2023-12-26 17:29:35,254][105692] Updated weights for policy 0, policy_version 292544 (0.0006) [2023-12-26 17:29:35,725][105620] Updated weights for policy 1, policy_version 292719 (0.0006) [2023-12-26 17:29:35,781][105620] Updated weights for policy 1, policy_version 292729 (0.0005) [2023-12-26 17:29:35,814][105692] Updated weights for policy 0, policy_version 292554 (0.0006) [2023-12-26 17:29:35,827][105620] Updated weights for policy 1, policy_version 292739 (0.0006) [2023-12-26 17:29:35,873][105692] Updated weights for policy 0, policy_version 292564 (0.0006) [2023-12-26 17:29:35,927][105692] Updated weights for policy 0, policy_version 292574 (0.0008) [2023-12-26 17:29:35,973][105692] Updated weights for policy 0, policy_version 292584 (0.0006) [2023-12-26 17:29:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 149864448. Throughput: 0: 10005.6, 1: 9757.2. Samples: 149846068. Policy #0 lag: (min: 31.0, avg: 32.4, max: 60.0) [2023-12-26 17:29:36,062][104569] Avg episode reward: [(0, '9177.020'), (1, '6662.940')] [2023-12-26 17:29:36,438][105620] Updated weights for policy 1, policy_version 292749 (0.0007) [2023-12-26 17:29:36,465][105586] KL-divergence is very high: 114.9947 [2023-12-26 17:29:36,499][105620] Updated weights for policy 1, policy_version 292759 (0.0006) [2023-12-26 17:29:36,512][105586] KL-divergence is very high: 117.1730 [2023-12-26 17:29:36,549][105586] KL-divergence is very high: 169.4357 [2023-12-26 17:29:36,561][105620] Updated weights for policy 1, policy_version 292769 (0.0006) [2023-12-26 17:29:36,563][105586] KL-divergence is very high: 122.1853 [2023-12-26 17:29:36,600][105586] KL-divergence is very high: 181.6426 [2023-12-26 17:29:36,627][105692] Updated weights for policy 0, policy_version 292594 (0.0011) [2023-12-26 17:29:36,696][105692] Updated weights for policy 0, policy_version 292604 (0.0006) [2023-12-26 17:29:36,768][105692] Updated weights for policy 0, policy_version 292614 (0.0005) [2023-12-26 17:29:37,237][105620] Updated weights for policy 1, policy_version 292779 (0.0006) [2023-12-26 17:29:37,298][105620] Updated weights for policy 1, policy_version 292789 (0.0005) [2023-12-26 17:29:37,319][105586] KL-divergence is very high: 138.2656 [2023-12-26 17:29:37,347][105586] KL-divergence is very high: 264.9675 [2023-12-26 17:29:37,353][105586] KL-divergence is very high: 322.6241 [2023-12-26 17:29:37,359][105620] Updated weights for policy 1, policy_version 292799 (0.0005) [2023-12-26 17:29:37,367][105586] KL-divergence is very high: 538.6675 [2023-12-26 17:29:37,400][105586] KL-divergence is very high: 448.6613 [2023-12-26 17:29:37,405][105586] KL-divergence is very high: 430.3435 [2023-12-26 17:29:37,448][105692] Updated weights for policy 0, policy_version 292624 (0.0010) [2023-12-26 17:29:37,500][105692] Updated weights for policy 0, policy_version 292634 (0.0010) [2023-12-26 17:29:37,548][105692] Updated weights for policy 0, policy_version 292644 (0.0010) [2023-12-26 17:29:37,939][105620] Updated weights for policy 1, policy_version 292809 (0.0006) [2023-12-26 17:29:37,995][105620] Updated weights for policy 1, policy_version 292819 (0.0008) [2023-12-26 17:29:38,052][105620] Updated weights for policy 1, policy_version 292829 (0.0008) [2023-12-26 17:29:38,104][105620] Updated weights for policy 1, policy_version 292839 (0.0007) [2023-12-26 17:29:38,304][105692] Updated weights for policy 0, policy_version 292654 (0.0010) [2023-12-26 17:29:38,370][105692] Updated weights for policy 0, policy_version 292664 (0.0008) [2023-12-26 17:29:38,428][105692] Updated weights for policy 0, policy_version 292674 (0.0007) [2023-12-26 17:29:38,837][105620] Updated weights for policy 1, policy_version 292849 (0.0007) [2023-12-26 17:29:38,897][105620] Updated weights for policy 1, policy_version 292859 (0.0008) [2023-12-26 17:29:38,956][105620] Updated weights for policy 1, policy_version 292869 (0.0008) [2023-12-26 17:29:39,143][105692] Updated weights for policy 0, policy_version 292684 (0.0007) [2023-12-26 17:29:39,202][105692] Updated weights for policy 0, policy_version 292694 (0.0005) [2023-12-26 17:29:39,267][105692] Updated weights for policy 0, policy_version 292704 (0.0008) [2023-12-26 17:29:39,682][105620] Updated weights for policy 1, policy_version 292879 (0.0007) [2023-12-26 17:29:39,739][105620] Updated weights for policy 1, policy_version 292889 (0.0008) [2023-12-26 17:29:39,801][105620] Updated weights for policy 1, policy_version 292899 (0.0007) [2023-12-26 17:29:39,981][105692] Updated weights for policy 0, policy_version 292714 (0.0008) [2023-12-26 17:29:40,046][105692] Updated weights for policy 0, policy_version 292724 (0.0010) [2023-12-26 17:29:40,110][105692] Updated weights for policy 0, policy_version 292734 (0.0011) [2023-12-26 17:29:40,172][105692] Updated weights for policy 0, policy_version 292744 (0.0006) [2023-12-26 17:29:40,562][105620] Updated weights for policy 1, policy_version 292909 (0.0009) [2023-12-26 17:29:40,614][105620] Updated weights for policy 1, policy_version 292919 (0.0009) [2023-12-26 17:29:40,667][105620] Updated weights for policy 1, policy_version 292929 (0.0010) [2023-12-26 17:29:40,799][105692] Updated weights for policy 0, policy_version 292754 (0.0006) [2023-12-26 17:29:40,869][105692] Updated weights for policy 0, policy_version 292764 (0.0006) [2023-12-26 17:29:40,937][105692] Updated weights for policy 0, policy_version 292774 (0.0005) [2023-12-26 17:29:41,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 149962752. Throughput: 0: 9989.8, 1: 9754.1. Samples: 149968544. Policy #0 lag: (min: 31.0, avg: 32.4, max: 60.0) [2023-12-26 17:29:41,062][104569] Avg episode reward: [(0, '9085.666'), (1, '6757.248')] [2023-12-26 17:29:41,318][105620] Updated weights for policy 1, policy_version 292939 (0.0008) [2023-12-26 17:29:41,391][105620] Updated weights for policy 1, policy_version 292950 (0.0010) [2023-12-26 17:29:41,451][105620] Updated weights for policy 1, policy_version 292960 (0.0011) [2023-12-26 17:29:41,582][105692] Updated weights for policy 0, policy_version 292784 (0.0006) [2023-12-26 17:29:41,646][105692] Updated weights for policy 0, policy_version 292794 (0.0008) [2023-12-26 17:29:41,704][105692] Updated weights for policy 0, policy_version 292804 (0.0006) [2023-12-26 17:29:42,273][105620] Updated weights for policy 1, policy_version 292970 (0.0008) [2023-12-26 17:29:42,333][105620] Updated weights for policy 1, policy_version 292980 (0.0009) [2023-12-26 17:29:42,339][105692] Updated weights for policy 0, policy_version 292814 (0.0008) [2023-12-26 17:29:42,393][105620] Updated weights for policy 1, policy_version 292990 (0.0009) [2023-12-26 17:29:42,404][105692] Updated weights for policy 0, policy_version 292824 (0.0007) [2023-12-26 17:29:42,447][105620] Updated weights for policy 1, policy_version 293000 (0.0007) [2023-12-26 17:29:42,460][105692] Updated weights for policy 0, policy_version 292834 (0.0008) [2023-12-26 17:29:43,133][105620] Updated weights for policy 1, policy_version 293010 (0.0005) [2023-12-26 17:29:43,176][105692] Updated weights for policy 0, policy_version 292844 (0.0008) [2023-12-26 17:29:43,187][105620] Updated weights for policy 1, policy_version 293020 (0.0005) [2023-12-26 17:29:43,226][105692] Updated weights for policy 0, policy_version 292854 (0.0006) [2023-12-26 17:29:43,246][105620] Updated weights for policy 1, policy_version 293030 (0.0007) [2023-12-26 17:29:43,271][105692] Updated weights for policy 0, policy_version 292864 (0.0005) [2023-12-26 17:29:43,864][105692] Updated weights for policy 0, policy_version 292874 (0.0009) [2023-12-26 17:29:43,912][105692] Updated weights for policy 0, policy_version 292884 (0.0005) [2023-12-26 17:29:43,965][105620] Updated weights for policy 1, policy_version 293040 (0.0006) [2023-12-26 17:29:43,966][105692] Updated weights for policy 0, policy_version 292894 (0.0005) [2023-12-26 17:29:44,012][105692] Updated weights for policy 0, policy_version 292904 (0.0005) [2023-12-26 17:29:44,035][105620] Updated weights for policy 1, policy_version 293050 (0.0005) [2023-12-26 17:29:44,103][105620] Updated weights for policy 1, policy_version 293060 (0.0006) [2023-12-26 17:29:44,666][105620] Updated weights for policy 1, policy_version 293070 (0.0007) [2023-12-26 17:29:44,700][105692] Updated weights for policy 0, policy_version 292914 (0.0006) [2023-12-26 17:29:44,725][105620] Updated weights for policy 1, policy_version 293080 (0.0007) [2023-12-26 17:29:44,731][105586] KL-divergence is very high: 133.4372 [2023-12-26 17:29:44,756][105692] Updated weights for policy 0, policy_version 292924 (0.0006) [2023-12-26 17:29:44,791][105620] Updated weights for policy 1, policy_version 293090 (0.0007) [2023-12-26 17:29:44,818][105692] Updated weights for policy 0, policy_version 292934 (0.0009) [2023-12-26 17:29:45,483][105620] Updated weights for policy 1, policy_version 293100 (0.0006) [2023-12-26 17:29:45,532][105692] Updated weights for policy 0, policy_version 292944 (0.0008) [2023-12-26 17:29:45,537][105620] Updated weights for policy 1, policy_version 293110 (0.0005) [2023-12-26 17:29:45,590][105692] Updated weights for policy 0, policy_version 292954 (0.0006) [2023-12-26 17:29:45,601][105620] Updated weights for policy 1, policy_version 293120 (0.0007) [2023-12-26 17:29:45,642][105692] Updated weights for policy 0, policy_version 292964 (0.0005) [2023-12-26 17:29:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 150061056. Throughput: 0: 9949.0, 1: 9742.7. Samples: 150028016. Policy #0 lag: (min: 31.0, avg: 32.4, max: 60.0) [2023-12-26 17:29:46,063][104569] Avg episode reward: [(0, '9176.615'), (1, '6478.392')] [2023-12-26 17:29:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000292968_75014144.pth... [2023-12-26 17:29:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000293128_75046912.pth... [2023-12-26 17:29:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000291816_74719232.pth [2023-12-26 17:29:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000291976_74752000.pth [2023-12-26 17:29:46,309][105692] Updated weights for policy 0, policy_version 292974 (0.0009) [2023-12-26 17:29:46,353][105620] Updated weights for policy 1, policy_version 293130 (0.0008) [2023-12-26 17:29:46,371][105692] Updated weights for policy 0, policy_version 292984 (0.0010) [2023-12-26 17:29:46,415][105620] Updated weights for policy 1, policy_version 293140 (0.0009) [2023-12-26 17:29:46,419][105692] Updated weights for policy 0, policy_version 292994 (0.0010) [2023-12-26 17:29:46,473][105620] Updated weights for policy 1, policy_version 293150 (0.0010) [2023-12-26 17:29:46,532][105620] Updated weights for policy 1, policy_version 293160 (0.0010) [2023-12-26 17:29:47,154][105692] Updated weights for policy 0, policy_version 293004 (0.0011) [2023-12-26 17:29:47,173][105620] Updated weights for policy 1, policy_version 293170 (0.0006) [2023-12-26 17:29:47,205][105692] Updated weights for policy 0, policy_version 293014 (0.0010) [2023-12-26 17:29:47,224][105620] Updated weights for policy 1, policy_version 293180 (0.0005) [2023-12-26 17:29:47,267][105692] Updated weights for policy 0, policy_version 293024 (0.0010) [2023-12-26 17:29:47,278][105620] Updated weights for policy 1, policy_version 293190 (0.0006) [2023-12-26 17:29:47,983][105620] Updated weights for policy 1, policy_version 293200 (0.0007) [2023-12-26 17:29:48,008][105692] Updated weights for policy 0, policy_version 293034 (0.0010) [2023-12-26 17:29:48,035][105620] Updated weights for policy 1, policy_version 293210 (0.0008) [2023-12-26 17:29:48,070][105692] Updated weights for policy 0, policy_version 293044 (0.0010) [2023-12-26 17:29:48,084][105620] Updated weights for policy 1, policy_version 293220 (0.0009) [2023-12-26 17:29:48,125][105692] Updated weights for policy 0, policy_version 293054 (0.0010) [2023-12-26 17:29:48,183][105692] Updated weights for policy 0, policy_version 293064 (0.0010) [2023-12-26 17:29:48,826][105620] Updated weights for policy 1, policy_version 293230 (0.0007) [2023-12-26 17:29:48,883][105620] Updated weights for policy 1, policy_version 293240 (0.0008) [2023-12-26 17:29:48,929][105620] Updated weights for policy 1, policy_version 293250 (0.0008) [2023-12-26 17:29:48,986][105692] Updated weights for policy 0, policy_version 293074 (0.0005) [2023-12-26 17:29:49,033][105692] Updated weights for policy 0, policy_version 293084 (0.0005) [2023-12-26 17:29:49,077][105692] Updated weights for policy 0, policy_version 293094 (0.0005) [2023-12-26 17:29:49,655][105620] Updated weights for policy 1, policy_version 293260 (0.0008) [2023-12-26 17:29:49,702][105620] Updated weights for policy 1, policy_version 293270 (0.0008) [2023-12-26 17:29:49,751][105620] Updated weights for policy 1, policy_version 293280 (0.0008) [2023-12-26 17:29:49,796][105692] Updated weights for policy 0, policy_version 293104 (0.0009) [2023-12-26 17:29:49,860][105692] Updated weights for policy 0, policy_version 293114 (0.0010) [2023-12-26 17:29:49,927][105692] Updated weights for policy 0, policy_version 293124 (0.0011) [2023-12-26 17:29:50,552][105620] Updated weights for policy 1, policy_version 293290 (0.0007) [2023-12-26 17:29:50,619][105620] Updated weights for policy 1, policy_version 293300 (0.0009) [2023-12-26 17:29:50,628][105692] Updated weights for policy 0, policy_version 293134 (0.0008) [2023-12-26 17:29:50,669][105586] KL-divergence is very high: 105.9122 [2023-12-26 17:29:50,674][105620] Updated weights for policy 1, policy_version 293310 (0.0008) [2023-12-26 17:29:50,679][105586] KL-divergence is very high: 242.7905 [2023-12-26 17:29:50,684][105586] KL-divergence is very high: 111.2209 [2023-12-26 17:29:50,689][105586] KL-divergence is very high: 133.4079 [2023-12-26 17:29:50,691][105692] Updated weights for policy 0, policy_version 293144 (0.0006) [2023-12-26 17:29:50,694][105586] KL-divergence is very high: 125.9194 [2023-12-26 17:29:50,700][105586] KL-divergence is very high: 222.8450 [2023-12-26 17:29:50,707][105586] KL-divergence is very high: 165.9955 [2023-12-26 17:29:50,713][105586] KL-divergence is very high: 105.9577 [2023-12-26 17:29:50,724][105586] KL-divergence is very high: 146.1826 [2023-12-26 17:29:50,729][105620] Updated weights for policy 1, policy_version 293320 (0.0007) [2023-12-26 17:29:50,753][105692] Updated weights for policy 0, policy_version 293154 (0.0010) [2023-12-26 17:29:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 150159360. Throughput: 0: 9947.5, 1: 9843.9. Samples: 150146964. Policy #0 lag: (min: 31.0, avg: 32.4, max: 60.0) [2023-12-26 17:29:51,062][104569] Avg episode reward: [(0, '9087.297'), (1, '1316.152')] [2023-12-26 17:29:51,335][105692] Updated weights for policy 0, policy_version 293164 (0.0007) [2023-12-26 17:29:51,403][105692] Updated weights for policy 0, policy_version 293174 (0.0008) [2023-12-26 17:29:51,453][105692] Updated weights for policy 0, policy_version 293184 (0.0009) [2023-12-26 17:29:51,586][105620] Updated weights for policy 1, policy_version 293330 (0.0009) [2023-12-26 17:29:51,640][105620] Updated weights for policy 1, policy_version 293340 (0.0009) [2023-12-26 17:29:51,703][105620] Updated weights for policy 1, policy_version 293350 (0.0009) [2023-12-26 17:29:52,193][105692] Updated weights for policy 0, policy_version 293194 (0.0009) [2023-12-26 17:29:52,240][105692] Updated weights for policy 0, policy_version 293204 (0.0009) [2023-12-26 17:29:52,302][105692] Updated weights for policy 0, policy_version 293214 (0.0009) [2023-12-26 17:29:52,364][105692] Updated weights for policy 0, policy_version 293224 (0.0009) [2023-12-26 17:29:52,503][105620] Updated weights for policy 1, policy_version 293360 (0.0008) [2023-12-26 17:29:52,564][105620] Updated weights for policy 1, policy_version 293370 (0.0007) [2023-12-26 17:29:52,625][105620] Updated weights for policy 1, policy_version 293380 (0.0009) [2023-12-26 17:29:53,204][105692] Updated weights for policy 0, policy_version 293234 (0.0008) [2023-12-26 17:29:53,264][105692] Updated weights for policy 0, policy_version 293244 (0.0009) [2023-12-26 17:29:53,285][105620] Updated weights for policy 1, policy_version 293390 (0.0008) [2023-12-26 17:29:53,324][105692] Updated weights for policy 0, policy_version 293254 (0.0006) [2023-12-26 17:29:53,338][105620] Updated weights for policy 1, policy_version 293400 (0.0008) [2023-12-26 17:29:53,384][105620] Updated weights for policy 1, policy_version 293410 (0.0009) [2023-12-26 17:29:54,081][105620] Updated weights for policy 1, policy_version 293420 (0.0008) [2023-12-26 17:29:54,092][105692] Updated weights for policy 0, policy_version 293264 (0.0007) [2023-12-26 17:29:54,107][105586] KL-divergence is very high: 128.2561 [2023-12-26 17:29:54,121][105586] KL-divergence is very high: 136.2023 [2023-12-26 17:29:54,127][105586] KL-divergence is very high: 148.0968 [2023-12-26 17:29:54,133][105586] KL-divergence is very high: 111.5555 [2023-12-26 17:29:54,141][105692] Updated weights for policy 0, policy_version 293274 (0.0007) [2023-12-26 17:29:54,146][105620] Updated weights for policy 1, policy_version 293430 (0.0009) [2023-12-26 17:29:54,198][105692] Updated weights for policy 0, policy_version 293284 (0.0007) [2023-12-26 17:29:54,205][105620] Updated weights for policy 1, policy_version 293440 (0.0007) [2023-12-26 17:29:54,889][105620] Updated weights for policy 1, policy_version 293450 (0.0009) [2023-12-26 17:29:54,940][105620] Updated weights for policy 1, policy_version 293460 (0.0009) [2023-12-26 17:29:54,984][105692] Updated weights for policy 0, policy_version 293294 (0.0008) [2023-12-26 17:29:54,998][105620] Updated weights for policy 1, policy_version 293470 (0.0008) [2023-12-26 17:29:55,036][105692] Updated weights for policy 0, policy_version 293304 (0.0006) [2023-12-26 17:29:55,057][105620] Updated weights for policy 1, policy_version 293480 (0.0009) [2023-12-26 17:29:55,089][105692] Updated weights for policy 0, policy_version 293314 (0.0007) [2023-12-26 17:29:55,705][105692] Updated weights for policy 0, policy_version 293324 (0.0008) [2023-12-26 17:29:55,768][105692] Updated weights for policy 0, policy_version 293334 (0.0006) [2023-12-26 17:29:55,824][105692] Updated weights for policy 0, policy_version 293344 (0.0009) [2023-12-26 17:29:55,883][105620] Updated weights for policy 1, policy_version 293490 (0.0006) [2023-12-26 17:29:55,954][105620] Updated weights for policy 1, policy_version 293500 (0.0006) [2023-12-26 17:29:56,019][105620] Updated weights for policy 1, policy_version 293510 (0.0005) [2023-12-26 17:29:56,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 150257664. Throughput: 0: 9938.3, 1: 9804.9. Samples: 150260648. Policy #0 lag: (min: 31.0, avg: 32.4, max: 60.0) [2023-12-26 17:29:56,063][104569] Avg episode reward: [(0, '9177.766'), (1, '820.729')] [2023-12-26 17:29:56,406][105692] Updated weights for policy 0, policy_version 293354 (0.0007) [2023-12-26 17:29:56,456][105692] Updated weights for policy 0, policy_version 293364 (0.0005) [2023-12-26 17:29:56,509][105692] Updated weights for policy 0, policy_version 293374 (0.0005) [2023-12-26 17:29:56,563][105692] Updated weights for policy 0, policy_version 293384 (0.0005) [2023-12-26 17:29:56,750][105620] Updated weights for policy 1, policy_version 293520 (0.0007) [2023-12-26 17:29:56,808][105620] Updated weights for policy 1, policy_version 293530 (0.0008) [2023-12-26 17:29:56,862][105620] Updated weights for policy 1, policy_version 293540 (0.0008) [2023-12-26 17:29:57,135][105692] Updated weights for policy 0, policy_version 293394 (0.0005) [2023-12-26 17:29:57,192][105692] Updated weights for policy 0, policy_version 293404 (0.0006) [2023-12-26 17:29:57,256][105692] Updated weights for policy 0, policy_version 293414 (0.0007) [2023-12-26 17:29:57,680][105620] Updated weights for policy 1, policy_version 293551 (0.0009) [2023-12-26 17:29:57,713][105586] KL-divergence is very high: 118.3701 [2023-12-26 17:29:57,735][105620] Updated weights for policy 1, policy_version 293561 (0.0010) [2023-12-26 17:29:57,735][105586] KL-divergence is very high: 122.8731 [2023-12-26 17:29:57,754][105586] KL-divergence is very high: 106.8870 [2023-12-26 17:29:57,783][105620] Updated weights for policy 1, policy_version 293571 (0.0009) [2023-12-26 17:29:57,847][105692] Updated weights for policy 0, policy_version 293424 (0.0008) [2023-12-26 17:29:57,908][105692] Updated weights for policy 0, policy_version 293434 (0.0009) [2023-12-26 17:29:57,964][105692] Updated weights for policy 0, policy_version 293444 (0.0009) [2023-12-26 17:29:58,574][105620] Updated weights for policy 1, policy_version 293582 (0.0008) [2023-12-26 17:29:58,635][105620] Updated weights for policy 1, policy_version 293592 (0.0008) [2023-12-26 17:29:58,698][105620] Updated weights for policy 1, policy_version 293602 (0.0008) [2023-12-26 17:29:58,727][105692] Updated weights for policy 0, policy_version 293454 (0.0008) [2023-12-26 17:29:58,808][105692] Updated weights for policy 0, policy_version 293464 (0.0010) [2023-12-26 17:29:58,877][105692] Updated weights for policy 0, policy_version 293474 (0.0010) [2023-12-26 17:29:59,515][105620] Updated weights for policy 1, policy_version 293612 (0.0009) [2023-12-26 17:29:59,574][105692] Updated weights for policy 0, policy_version 293484 (0.0009) [2023-12-26 17:29:59,580][105620] Updated weights for policy 1, policy_version 293622 (0.0008) [2023-12-26 17:29:59,630][105692] Updated weights for policy 0, policy_version 293494 (0.0008) [2023-12-26 17:29:59,639][105620] Updated weights for policy 1, policy_version 293632 (0.0006) [2023-12-26 17:29:59,682][105692] Updated weights for policy 0, policy_version 293504 (0.0006) [2023-12-26 17:30:00,352][105620] Updated weights for policy 1, policy_version 293642 (0.0007) [2023-12-26 17:30:00,414][105620] Updated weights for policy 1, policy_version 293652 (0.0009) [2023-12-26 17:30:00,420][105692] Updated weights for policy 0, policy_version 293514 (0.0008) [2023-12-26 17:30:00,476][105692] Updated weights for policy 0, policy_version 293524 (0.0006) [2023-12-26 17:30:00,478][105620] Updated weights for policy 1, policy_version 293662 (0.0008) [2023-12-26 17:30:00,527][105620] Updated weights for policy 1, policy_version 293672 (0.0006) [2023-12-26 17:30:00,537][105692] Updated weights for policy 0, policy_version 293534 (0.0008) [2023-12-26 17:30:00,600][105692] Updated weights for policy 0, policy_version 293544 (0.0008) [2023-12-26 17:30:01,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 150347776. Throughput: 0: 10002.4, 1: 9774.9. Samples: 150320084. Policy #0 lag: (min: 31.0, avg: 32.4, max: 60.0) [2023-12-26 17:30:01,062][104569] Avg episode reward: [(0, '9358.862'), (1, '2357.575')] [2023-12-26 17:30:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000293544_75161600.pth... [2023-12-26 17:30:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000293672_75186176.pth... [2023-12-26 17:30:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000292392_74866688.pth [2023-12-26 17:30:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000292552_74899456.pth [2023-12-26 17:30:01,276][105620] Updated weights for policy 1, policy_version 293682 (0.0009) [2023-12-26 17:30:01,307][105692] Updated weights for policy 0, policy_version 293554 (0.0009) [2023-12-26 17:30:01,325][105620] Updated weights for policy 1, policy_version 293692 (0.0009) [2023-12-26 17:30:01,354][105586] KL-divergence is very high: 102.6802 [2023-12-26 17:30:01,359][105692] Updated weights for policy 0, policy_version 293564 (0.0007) [2023-12-26 17:30:01,386][105620] Updated weights for policy 1, policy_version 293702 (0.0007) [2023-12-26 17:30:01,420][105692] Updated weights for policy 0, policy_version 293574 (0.0008) [2023-12-26 17:30:02,107][105692] Updated weights for policy 0, policy_version 293584 (0.0006) [2023-12-26 17:30:02,162][105692] Updated weights for policy 0, policy_version 293594 (0.0006) [2023-12-26 17:30:02,195][105620] Updated weights for policy 1, policy_version 293712 (0.0009) [2023-12-26 17:30:02,213][105692] Updated weights for policy 0, policy_version 293604 (0.0008) [2023-12-26 17:30:02,244][105620] Updated weights for policy 1, policy_version 293722 (0.0006) [2023-12-26 17:30:02,302][105620] Updated weights for policy 1, policy_version 293732 (0.0009) [2023-12-26 17:30:02,823][105692] Updated weights for policy 0, policy_version 293614 (0.0007) [2023-12-26 17:30:02,877][105692] Updated weights for policy 0, policy_version 293624 (0.0005) [2023-12-26 17:30:02,928][105692] Updated weights for policy 0, policy_version 293634 (0.0005) [2023-12-26 17:30:03,160][105620] Updated weights for policy 1, policy_version 293742 (0.0008) [2023-12-26 17:30:03,207][105620] Updated weights for policy 1, policy_version 293752 (0.0009) [2023-12-26 17:30:03,271][105620] Updated weights for policy 1, policy_version 293762 (0.0009) [2023-12-26 17:30:03,500][105692] Updated weights for policy 0, policy_version 293644 (0.0006) [2023-12-26 17:30:03,545][105692] Updated weights for policy 0, policy_version 293654 (0.0005) [2023-12-26 17:30:03,595][105692] Updated weights for policy 0, policy_version 293664 (0.0007) [2023-12-26 17:30:03,912][105620] Updated weights for policy 1, policy_version 293772 (0.0009) [2023-12-26 17:30:03,969][105620] Updated weights for policy 1, policy_version 293782 (0.0009) [2023-12-26 17:30:04,016][105620] Updated weights for policy 1, policy_version 293792 (0.0009) [2023-12-26 17:30:04,390][105692] Updated weights for policy 0, policy_version 293675 (0.0010) [2023-12-26 17:30:04,447][105692] Updated weights for policy 0, policy_version 293685 (0.0008) [2023-12-26 17:30:04,504][105692] Updated weights for policy 0, policy_version 293695 (0.0009) [2023-12-26 17:30:04,794][105620] Updated weights for policy 1, policy_version 293802 (0.0009) [2023-12-26 17:30:04,844][105620] Updated weights for policy 1, policy_version 293812 (0.0006) [2023-12-26 17:30:04,898][105620] Updated weights for policy 1, policy_version 293823 (0.0010) [2023-12-26 17:30:05,162][105692] Updated weights for policy 0, policy_version 293705 (0.0008) [2023-12-26 17:30:05,225][105692] Updated weights for policy 0, policy_version 293715 (0.0007) [2023-12-26 17:30:05,292][105692] Updated weights for policy 0, policy_version 293725 (0.0005) [2023-12-26 17:30:05,353][105692] Updated weights for policy 0, policy_version 293735 (0.0006) [2023-12-26 17:30:05,593][105620] Updated weights for policy 1, policy_version 293834 (0.0009) [2023-12-26 17:30:05,650][105620] Updated weights for policy 1, policy_version 293844 (0.0005) [2023-12-26 17:30:05,704][105620] Updated weights for policy 1, policy_version 293854 (0.0005) [2023-12-26 17:30:05,772][105620] Updated weights for policy 1, policy_version 293864 (0.0005) [2023-12-26 17:30:05,951][105692] Updated weights for policy 0, policy_version 293745 (0.0008) [2023-12-26 17:30:06,013][105692] Updated weights for policy 0, policy_version 293755 (0.0006) [2023-12-26 17:30:06,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 150446080. Throughput: 0: 10027.0, 1: 9643.7. Samples: 150435248. Policy #0 lag: (min: 31.0, avg: 32.4, max: 60.0) [2023-12-26 17:30:06,062][104569] Avg episode reward: [(0, '9359.104'), (1, '6565.979')] [2023-12-26 17:30:06,081][105692] Updated weights for policy 0, policy_version 293765 (0.0007) [2023-12-26 17:30:06,493][105620] Updated weights for policy 1, policy_version 293874 (0.0009) [2023-12-26 17:30:06,568][105620] Updated weights for policy 1, policy_version 293884 (0.0010) [2023-12-26 17:30:06,639][105620] Updated weights for policy 1, policy_version 293894 (0.0009) [2023-12-26 17:30:06,699][105692] Updated weights for policy 0, policy_version 293775 (0.0007) [2023-12-26 17:30:06,755][105692] Updated weights for policy 0, policy_version 293785 (0.0010) [2023-12-26 17:30:06,807][105692] Updated weights for policy 0, policy_version 293795 (0.0010) [2023-12-26 17:30:07,409][105620] Updated weights for policy 1, policy_version 293904 (0.0006) [2023-12-26 17:30:07,470][105620] Updated weights for policy 1, policy_version 293914 (0.0005) [2023-12-26 17:30:07,513][105692] Updated weights for policy 0, policy_version 293805 (0.0009) [2023-12-26 17:30:07,519][105620] Updated weights for policy 1, policy_version 293924 (0.0006) [2023-12-26 17:30:07,579][105692] Updated weights for policy 0, policy_version 293815 (0.0008) [2023-12-26 17:30:07,634][105692] Updated weights for policy 0, policy_version 293825 (0.0009) [2023-12-26 17:30:08,153][105620] Updated weights for policy 1, policy_version 293934 (0.0008) [2023-12-26 17:30:08,216][105620] Updated weights for policy 1, policy_version 293944 (0.0009) [2023-12-26 17:30:08,267][105620] Updated weights for policy 1, policy_version 293954 (0.0007) [2023-12-26 17:30:08,430][105692] Updated weights for policy 0, policy_version 293835 (0.0010) [2023-12-26 17:30:08,483][105692] Updated weights for policy 0, policy_version 293845 (0.0009) [2023-12-26 17:30:08,536][105692] Updated weights for policy 0, policy_version 293856 (0.0011) [2023-12-26 17:30:08,992][105620] Updated weights for policy 1, policy_version 293964 (0.0007) [2023-12-26 17:30:09,039][105620] Updated weights for policy 1, policy_version 293974 (0.0008) [2023-12-26 17:30:09,094][105620] Updated weights for policy 1, policy_version 293984 (0.0008) [2023-12-26 17:30:09,321][105692] Updated weights for policy 0, policy_version 293866 (0.0010) [2023-12-26 17:30:09,394][105692] Updated weights for policy 0, policy_version 293876 (0.0010) [2023-12-26 17:30:09,459][105692] Updated weights for policy 0, policy_version 293886 (0.0011) [2023-12-26 17:30:09,516][105692] Updated weights for policy 0, policy_version 293896 (0.0011) [2023-12-26 17:30:09,889][105620] Updated weights for policy 1, policy_version 293994 (0.0008) [2023-12-26 17:30:09,953][105620] Updated weights for policy 1, policy_version 294004 (0.0008) [2023-12-26 17:30:09,963][105586] KL-divergence is very high: 122.5022 [2023-12-26 17:30:09,969][105586] KL-divergence is very high: 147.7243 [2023-12-26 17:30:09,976][105586] KL-divergence is very high: 167.9973 [2023-12-26 17:30:10,002][105586] KL-divergence is very high: 140.7242 [2023-12-26 17:30:10,014][105586] KL-divergence is very high: 134.5182 [2023-12-26 17:30:10,021][105586] KL-divergence is very high: 136.8644 [2023-12-26 17:30:10,021][105620] Updated weights for policy 1, policy_version 294014 (0.0007) [2023-12-26 17:30:10,028][105586] KL-divergence is very high: 139.4614 [2023-12-26 17:30:10,089][105620] Updated weights for policy 1, policy_version 294024 (0.0010) [2023-12-26 17:30:10,148][105692] Updated weights for policy 0, policy_version 293906 (0.0006) [2023-12-26 17:30:10,211][105692] Updated weights for policy 0, policy_version 293916 (0.0007) [2023-12-26 17:30:10,267][105692] Updated weights for policy 0, policy_version 293926 (0.0009) [2023-12-26 17:30:10,819][105620] Updated weights for policy 1, policy_version 294034 (0.0005) [2023-12-26 17:30:10,873][105620] Updated weights for policy 1, policy_version 294044 (0.0008) [2023-12-26 17:30:10,921][105620] Updated weights for policy 1, policy_version 294054 (0.0008) [2023-12-26 17:30:10,948][105692] Updated weights for policy 0, policy_version 293936 (0.0009) [2023-12-26 17:30:11,008][105692] Updated weights for policy 0, policy_version 293946 (0.0008) [2023-12-26 17:30:11,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19660.7, 300 sec: 19633.0). Total num frames: 150544384. Throughput: 0: 10099.9, 1: 9649.6. Samples: 150554168. Policy #0 lag: (min: 11.0, avg: 23.4, max: 43.0) [2023-12-26 17:30:11,063][104569] Avg episode reward: [(0, '9359.242'), (1, '5464.462')] [2023-12-26 17:30:11,070][105692] Updated weights for policy 0, policy_version 293956 (0.0008) [2023-12-26 17:30:11,670][105620] Updated weights for policy 1, policy_version 294064 (0.0008) [2023-12-26 17:30:11,734][105620] Updated weights for policy 1, policy_version 294074 (0.0008) [2023-12-26 17:30:11,797][105620] Updated weights for policy 1, policy_version 294084 (0.0009) [2023-12-26 17:30:11,846][105692] Updated weights for policy 0, policy_version 293966 (0.0007) [2023-12-26 17:30:11,913][105692] Updated weights for policy 0, policy_version 293976 (0.0009) [2023-12-26 17:30:11,977][105692] Updated weights for policy 0, policy_version 293986 (0.0010) [2023-12-26 17:30:12,542][105620] Updated weights for policy 1, policy_version 294094 (0.0007) [2023-12-26 17:30:12,595][105620] Updated weights for policy 1, policy_version 294104 (0.0009) [2023-12-26 17:30:12,658][105620] Updated weights for policy 1, policy_version 294114 (0.0010) [2023-12-26 17:30:12,680][105692] Updated weights for policy 0, policy_version 293996 (0.0009) [2023-12-26 17:30:12,745][105692] Updated weights for policy 0, policy_version 294006 (0.0008) [2023-12-26 17:30:12,810][105692] Updated weights for policy 0, policy_version 294016 (0.0008) [2023-12-26 17:30:13,390][105620] Updated weights for policy 1, policy_version 294124 (0.0008) [2023-12-26 17:30:13,437][105620] Updated weights for policy 1, policy_version 294134 (0.0010) [2023-12-26 17:30:13,485][105620] Updated weights for policy 1, policy_version 294144 (0.0010) [2023-12-26 17:30:13,546][105692] Updated weights for policy 0, policy_version 294026 (0.0009) [2023-12-26 17:30:13,600][105692] Updated weights for policy 0, policy_version 294036 (0.0010) [2023-12-26 17:30:13,607][105585] KL-divergence is very high: 892.8536 [2023-12-26 17:30:13,646][105585] KL-divergence is very high: 1537.0953 [2023-12-26 17:30:13,648][105692] Updated weights for policy 0, policy_version 294046 (0.0010) [2023-12-26 17:30:13,684][105585] KL-divergence is very high: 1638.0144 [2023-12-26 17:30:13,695][105692] Updated weights for policy 0, policy_version 294056 (0.0010) [2023-12-26 17:30:14,243][105620] Updated weights for policy 1, policy_version 294154 (0.0010) [2023-12-26 17:30:14,303][105620] Updated weights for policy 1, policy_version 294164 (0.0010) [2023-12-26 17:30:14,348][105620] Updated weights for policy 1, policy_version 294174 (0.0010) [2023-12-26 17:30:14,363][105585] KL-divergence is very high: 106.6133 [2023-12-26 17:30:14,402][105620] Updated weights for policy 1, policy_version 294184 (0.0009) [2023-12-26 17:30:14,420][105692] Updated weights for policy 0, policy_version 294066 (0.0007) [2023-12-26 17:30:14,476][105692] Updated weights for policy 0, policy_version 294076 (0.0005) [2023-12-26 17:30:14,544][105692] Updated weights for policy 0, policy_version 294086 (0.0008) [2023-12-26 17:30:15,203][105692] Updated weights for policy 0, policy_version 294096 (0.0009) [2023-12-26 17:30:15,233][105620] Updated weights for policy 1, policy_version 294194 (0.0006) [2023-12-26 17:30:15,264][105692] Updated weights for policy 0, policy_version 294106 (0.0010) [2023-12-26 17:30:15,292][105620] Updated weights for policy 1, policy_version 294204 (0.0007) [2023-12-26 17:30:15,323][105692] Updated weights for policy 0, policy_version 294116 (0.0006) [2023-12-26 17:30:15,346][105620] Updated weights for policy 1, policy_version 294214 (0.0007) [2023-12-26 17:30:15,939][105620] Updated weights for policy 1, policy_version 294224 (0.0008) [2023-12-26 17:30:15,988][105620] Updated weights for policy 1, policy_version 294234 (0.0008) [2023-12-26 17:30:16,053][105620] Updated weights for policy 1, policy_version 294244 (0.0010) [2023-12-26 17:30:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 150634496. Throughput: 0: 9986.7, 1: 9621.9. Samples: 150610364. Policy #0 lag: (min: 11.0, avg: 23.4, max: 43.0) [2023-12-26 17:30:16,063][104569] Avg episode reward: [(0, '9178.163'), (1, '5403.538')] [2023-12-26 17:30:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000294120_75309056.pth... [2023-12-26 17:30:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000292968_75014144.pth [2023-12-26 17:30:16,078][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000294248_75333632.pth... [2023-12-26 17:30:16,082][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000293128_75046912.pth [2023-12-26 17:30:16,150][105692] Updated weights for policy 0, policy_version 294126 (0.0008) [2023-12-26 17:30:16,204][105692] Updated weights for policy 0, policy_version 294136 (0.0007) [2023-12-26 17:30:16,268][105692] Updated weights for policy 0, policy_version 294146 (0.0006) [2023-12-26 17:30:16,695][105620] Updated weights for policy 1, policy_version 294254 (0.0007) [2023-12-26 17:30:16,756][105620] Updated weights for policy 1, policy_version 294264 (0.0006) [2023-12-26 17:30:16,812][105620] Updated weights for policy 1, policy_version 294274 (0.0005) [2023-12-26 17:30:17,113][105692] Updated weights for policy 0, policy_version 294156 (0.0009) [2023-12-26 17:30:17,177][105692] Updated weights for policy 0, policy_version 294166 (0.0009) [2023-12-26 17:30:17,230][105692] Updated weights for policy 0, policy_version 294176 (0.0010) [2023-12-26 17:30:17,354][105620] Updated weights for policy 1, policy_version 294284 (0.0005) [2023-12-26 17:30:17,407][105620] Updated weights for policy 1, policy_version 294294 (0.0006) [2023-12-26 17:30:17,455][105620] Updated weights for policy 1, policy_version 294304 (0.0007) [2023-12-26 17:30:18,006][105692] Updated weights for policy 0, policy_version 294186 (0.0009) [2023-12-26 17:30:18,067][105692] Updated weights for policy 0, policy_version 294196 (0.0010) [2023-12-26 17:30:18,069][105620] Updated weights for policy 1, policy_version 294314 (0.0008) [2023-12-26 17:30:18,124][105692] Updated weights for policy 0, policy_version 294206 (0.0009) [2023-12-26 17:30:18,125][105620] Updated weights for policy 1, policy_version 294324 (0.0005) [2023-12-26 17:30:18,172][105692] Updated weights for policy 0, policy_version 294216 (0.0008) [2023-12-26 17:30:18,182][105620] Updated weights for policy 1, policy_version 294334 (0.0006) [2023-12-26 17:30:18,237][105620] Updated weights for policy 1, policy_version 294344 (0.0009) [2023-12-26 17:30:18,926][105692] Updated weights for policy 0, policy_version 294226 (0.0009) [2023-12-26 17:30:18,961][105620] Updated weights for policy 1, policy_version 294354 (0.0006) [2023-12-26 17:30:18,984][105692] Updated weights for policy 0, policy_version 294236 (0.0008) [2023-12-26 17:30:19,010][105620] Updated weights for policy 1, policy_version 294364 (0.0006) [2023-12-26 17:30:19,041][105692] Updated weights for policy 0, policy_version 294246 (0.0008) [2023-12-26 17:30:19,069][105620] Updated weights for policy 1, policy_version 294374 (0.0007) [2023-12-26 17:30:19,783][105692] Updated weights for policy 0, policy_version 294256 (0.0008) [2023-12-26 17:30:19,847][105620] Updated weights for policy 1, policy_version 294384 (0.0008) [2023-12-26 17:30:19,849][105692] Updated weights for policy 0, policy_version 294266 (0.0008) [2023-12-26 17:30:19,907][105620] Updated weights for policy 1, policy_version 294394 (0.0008) [2023-12-26 17:30:19,910][105692] Updated weights for policy 0, policy_version 294276 (0.0007) [2023-12-26 17:30:19,968][105620] Updated weights for policy 1, policy_version 294404 (0.0008) [2023-12-26 17:30:20,709][105692] Updated weights for policy 0, policy_version 294286 (0.0007) [2023-12-26 17:30:20,715][105620] Updated weights for policy 1, policy_version 294414 (0.0007) [2023-12-26 17:30:20,771][105620] Updated weights for policy 1, policy_version 294424 (0.0006) [2023-12-26 17:30:20,771][105692] Updated weights for policy 0, policy_version 294296 (0.0009) [2023-12-26 17:30:20,831][105620] Updated weights for policy 1, policy_version 294434 (0.0006) [2023-12-26 17:30:20,832][105692] Updated weights for policy 0, policy_version 294306 (0.0009) [2023-12-26 17:30:21,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 150740992. Throughput: 0: 9911.0, 1: 9671.5. Samples: 150727280. Policy #0 lag: (min: 11.0, avg: 23.4, max: 43.0) [2023-12-26 17:30:21,063][104569] Avg episode reward: [(0, '8996.993'), (1, '6949.034')] [2023-12-26 17:30:21,542][105620] Updated weights for policy 1, policy_version 294444 (0.0006) [2023-12-26 17:30:21,600][105620] Updated weights for policy 1, policy_version 294454 (0.0009) [2023-12-26 17:30:21,670][105692] Updated weights for policy 0, policy_version 294316 (0.0008) [2023-12-26 17:30:21,672][105620] Updated weights for policy 1, policy_version 294464 (0.0007) [2023-12-26 17:30:21,740][105692] Updated weights for policy 0, policy_version 294326 (0.0007) [2023-12-26 17:30:21,798][105585] KL-divergence is very high: 100.2746 [2023-12-26 17:30:21,801][105692] Updated weights for policy 0, policy_version 294336 (0.0009) [2023-12-26 17:30:22,289][105620] Updated weights for policy 1, policy_version 294474 (0.0008) [2023-12-26 17:30:22,349][105620] Updated weights for policy 1, policy_version 294484 (0.0009) [2023-12-26 17:30:22,404][105620] Updated weights for policy 1, policy_version 294494 (0.0009) [2023-12-26 17:30:22,459][105620] Updated weights for policy 1, policy_version 294504 (0.0009) [2023-12-26 17:30:22,659][105692] Updated weights for policy 0, policy_version 294346 (0.0009) [2023-12-26 17:30:22,713][105692] Updated weights for policy 0, policy_version 294356 (0.0010) [2023-12-26 17:30:22,722][105585] KL-divergence is very high: 119.6156 [2023-12-26 17:30:22,761][105585] KL-divergence is very high: 206.3824 [2023-12-26 17:30:22,762][105692] Updated weights for policy 0, policy_version 294366 (0.0009) [2023-12-26 17:30:22,803][105585] KL-divergence is very high: 242.5660 [2023-12-26 17:30:22,814][105692] Updated weights for policy 0, policy_version 294376 (0.0009) [2023-12-26 17:30:23,150][105620] Updated weights for policy 1, policy_version 294514 (0.0009) [2023-12-26 17:30:23,200][105620] Updated weights for policy 1, policy_version 294524 (0.0009) [2023-12-26 17:30:23,248][105620] Updated weights for policy 1, policy_version 294534 (0.0009) [2023-12-26 17:30:23,550][105692] Updated weights for policy 0, policy_version 294386 (0.0007) [2023-12-26 17:30:23,612][105692] Updated weights for policy 0, policy_version 294396 (0.0009) [2023-12-26 17:30:23,674][105692] Updated weights for policy 0, policy_version 294406 (0.0007) [2023-12-26 17:30:24,134][105620] Updated weights for policy 1, policy_version 294544 (0.0008) [2023-12-26 17:30:24,190][105620] Updated weights for policy 1, policy_version 294554 (0.0008) [2023-12-26 17:30:24,235][105692] Updated weights for policy 0, policy_version 294416 (0.0009) [2023-12-26 17:30:24,251][105620] Updated weights for policy 1, policy_version 294564 (0.0006) [2023-12-26 17:30:24,288][105692] Updated weights for policy 0, policy_version 294426 (0.0009) [2023-12-26 17:30:24,342][105692] Updated weights for policy 0, policy_version 294436 (0.0008) [2023-12-26 17:30:25,013][105620] Updated weights for policy 1, policy_version 294574 (0.0007) [2023-12-26 17:30:25,061][105620] Updated weights for policy 1, policy_version 294584 (0.0009) [2023-12-26 17:30:25,082][105692] Updated weights for policy 0, policy_version 294446 (0.0009) [2023-12-26 17:30:25,110][105620] Updated weights for policy 1, policy_version 294594 (0.0006) [2023-12-26 17:30:25,140][105692] Updated weights for policy 0, policy_version 294456 (0.0010) [2023-12-26 17:30:25,191][105692] Updated weights for policy 0, policy_version 294466 (0.0010) [2023-12-26 17:30:25,862][105692] Updated weights for policy 0, policy_version 294476 (0.0008) [2023-12-26 17:30:25,899][105620] Updated weights for policy 1, policy_version 294604 (0.0006) [2023-12-26 17:30:25,923][105692] Updated weights for policy 0, policy_version 294486 (0.0006) [2023-12-26 17:30:25,951][105620] Updated weights for policy 1, policy_version 294614 (0.0005) [2023-12-26 17:30:25,976][105692] Updated weights for policy 0, policy_version 294496 (0.0008) [2023-12-26 17:30:25,997][105620] Updated weights for policy 1, policy_version 294624 (0.0005) [2023-12-26 17:30:26,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 150839296. Throughput: 0: 9791.0, 1: 9567.7. Samples: 150839680. Policy #0 lag: (min: 11.0, avg: 23.4, max: 43.0) [2023-12-26 17:30:26,062][104569] Avg episode reward: [(0, '8724.950'), (1, '6576.331')] [2023-12-26 17:30:26,547][105692] Updated weights for policy 0, policy_version 294506 (0.0005) [2023-12-26 17:30:26,597][105692] Updated weights for policy 0, policy_version 294516 (0.0008) [2023-12-26 17:30:26,650][105692] Updated weights for policy 0, policy_version 294526 (0.0005) [2023-12-26 17:30:26,706][105692] Updated weights for policy 0, policy_version 294536 (0.0007) [2023-12-26 17:30:26,801][105620] Updated weights for policy 1, policy_version 294634 (0.0009) [2023-12-26 17:30:26,850][105620] Updated weights for policy 1, policy_version 294644 (0.0009) [2023-12-26 17:30:26,898][105620] Updated weights for policy 1, policy_version 294654 (0.0009) [2023-12-26 17:30:26,946][105620] Updated weights for policy 1, policy_version 294664 (0.0008) [2023-12-26 17:30:27,410][105692] Updated weights for policy 0, policy_version 294546 (0.0006) [2023-12-26 17:30:27,457][105692] Updated weights for policy 0, policy_version 294556 (0.0005) [2023-12-26 17:30:27,505][105692] Updated weights for policy 0, policy_version 294566 (0.0005) [2023-12-26 17:30:27,681][105620] Updated weights for policy 1, policy_version 294674 (0.0007) [2023-12-26 17:30:27,734][105620] Updated weights for policy 1, policy_version 294684 (0.0009) [2023-12-26 17:30:27,793][105620] Updated weights for policy 1, policy_version 294694 (0.0005) [2023-12-26 17:30:28,089][105692] Updated weights for policy 0, policy_version 294576 (0.0008) [2023-12-26 17:30:28,140][105692] Updated weights for policy 0, policy_version 294586 (0.0009) [2023-12-26 17:30:28,187][105692] Updated weights for policy 0, policy_version 294596 (0.0010) [2023-12-26 17:30:28,453][105620] Updated weights for policy 1, policy_version 294704 (0.0007) [2023-12-26 17:30:28,498][105620] Updated weights for policy 1, policy_version 294714 (0.0008) [2023-12-26 17:30:28,550][105620] Updated weights for policy 1, policy_version 294724 (0.0008) [2023-12-26 17:30:28,882][105692] Updated weights for policy 0, policy_version 294606 (0.0008) [2023-12-26 17:30:28,912][105585] KL-divergence is very high: 304.3998 [2023-12-26 17:30:28,930][105692] Updated weights for policy 0, policy_version 294616 (0.0005) [2023-12-26 17:30:28,948][105585] KL-divergence is very high: 320.4311 [2023-12-26 17:30:28,976][105692] Updated weights for policy 0, policy_version 294626 (0.0005) [2023-12-26 17:30:28,985][105585] KL-divergence is very high: 302.9565 [2023-12-26 17:30:29,340][105620] Updated weights for policy 1, policy_version 294734 (0.0009) [2023-12-26 17:30:29,406][105620] Updated weights for policy 1, policy_version 294744 (0.0007) [2023-12-26 17:30:29,465][105620] Updated weights for policy 1, policy_version 294754 (0.0008) [2023-12-26 17:30:29,597][105692] Updated weights for policy 0, policy_version 294636 (0.0007) [2023-12-26 17:30:29,667][105692] Updated weights for policy 0, policy_version 294646 (0.0011) [2023-12-26 17:30:29,725][105692] Updated weights for policy 0, policy_version 294656 (0.0006) [2023-12-26 17:30:30,249][105620] Updated weights for policy 1, policy_version 294764 (0.0007) [2023-12-26 17:30:30,314][105620] Updated weights for policy 1, policy_version 294774 (0.0005) [2023-12-26 17:30:30,338][105692] Updated weights for policy 0, policy_version 294666 (0.0006) [2023-12-26 17:30:30,381][105620] Updated weights for policy 1, policy_version 294784 (0.0007) [2023-12-26 17:30:30,402][105692] Updated weights for policy 0, policy_version 294676 (0.0006) [2023-12-26 17:30:30,425][105585] KL-divergence is very high: 193.3428 [2023-12-26 17:30:30,463][105692] Updated weights for policy 0, policy_version 294686 (0.0007) [2023-12-26 17:30:30,474][105585] KL-divergence is very high: 167.2485 [2023-12-26 17:30:30,523][105692] Updated weights for policy 0, policy_version 294696 (0.0011) [2023-12-26 17:30:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 150929408. Throughput: 0: 9827.8, 1: 9570.5. Samples: 150900940. Policy #0 lag: (min: 11.0, avg: 23.4, max: 43.0) [2023-12-26 17:30:31,062][104569] Avg episode reward: [(0, '7903.580'), (1, '1289.535')] [2023-12-26 17:30:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000294792_75472896.pth... [2023-12-26 17:30:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000293672_75186176.pth [2023-12-26 17:30:31,085][105692] Updated weights for policy 0, policy_version 294706 (0.0010) [2023-12-26 17:30:31,153][105692] Updated weights for policy 0, policy_version 294716 (0.0009) [2023-12-26 17:30:31,161][105620] Updated weights for policy 1, policy_version 294794 (0.0009) [2023-12-26 17:30:31,220][105692] Updated weights for policy 0, policy_version 294726 (0.0011) [2023-12-26 17:30:31,223][105620] Updated weights for policy 1, policy_version 294804 (0.0007) [2023-12-26 17:30:31,232][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000294728_75464704.pth... [2023-12-26 17:30:31,235][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000293544_75161600.pth [2023-12-26 17:30:31,275][105620] Updated weights for policy 1, policy_version 294814 (0.0009) [2023-12-26 17:30:31,346][105620] Updated weights for policy 1, policy_version 294824 (0.0008) [2023-12-26 17:30:31,950][105692] Updated weights for policy 0, policy_version 294736 (0.0006) [2023-12-26 17:30:32,006][105692] Updated weights for policy 0, policy_version 294746 (0.0006) [2023-12-26 17:30:32,067][105692] Updated weights for policy 0, policy_version 294756 (0.0009) [2023-12-26 17:30:32,132][105620] Updated weights for policy 1, policy_version 294834 (0.0008) [2023-12-26 17:30:32,197][105620] Updated weights for policy 1, policy_version 294844 (0.0008) [2023-12-26 17:30:32,256][105620] Updated weights for policy 1, policy_version 294854 (0.0009) [2023-12-26 17:30:32,841][105692] Updated weights for policy 0, policy_version 294766 (0.0008) [2023-12-26 17:30:32,899][105692] Updated weights for policy 0, policy_version 294776 (0.0007) [2023-12-26 17:30:32,910][105620] Updated weights for policy 1, policy_version 294864 (0.0006) [2023-12-26 17:30:32,967][105692] Updated weights for policy 0, policy_version 294786 (0.0005) [2023-12-26 17:30:32,978][105620] Updated weights for policy 1, policy_version 294874 (0.0005) [2023-12-26 17:30:33,045][105620] Updated weights for policy 1, policy_version 294884 (0.0005) [2023-12-26 17:30:33,485][105692] Updated weights for policy 0, policy_version 294796 (0.0006) [2023-12-26 17:30:33,527][105620] Updated weights for policy 1, policy_version 294894 (0.0005) [2023-12-26 17:30:33,539][105692] Updated weights for policy 0, policy_version 294806 (0.0006) [2023-12-26 17:30:33,572][105620] Updated weights for policy 1, policy_version 294904 (0.0005) [2023-12-26 17:30:33,592][105692] Updated weights for policy 0, policy_version 294816 (0.0006) [2023-12-26 17:30:33,615][105620] Updated weights for policy 1, policy_version 294914 (0.0005) [2023-12-26 17:30:34,259][105620] Updated weights for policy 1, policy_version 294924 (0.0007) [2023-12-26 17:30:34,261][105692] Updated weights for policy 0, policy_version 294826 (0.0006) [2023-12-26 17:30:34,318][105692] Updated weights for policy 0, policy_version 294836 (0.0006) [2023-12-26 17:30:34,324][105620] Updated weights for policy 1, policy_version 294934 (0.0007) [2023-12-26 17:30:34,375][105692] Updated weights for policy 0, policy_version 294846 (0.0006) [2023-12-26 17:30:34,385][105620] Updated weights for policy 1, policy_version 294944 (0.0008) [2023-12-26 17:30:34,441][105692] Updated weights for policy 0, policy_version 294856 (0.0008) [2023-12-26 17:30:35,084][105620] Updated weights for policy 1, policy_version 294954 (0.0006) [2023-12-26 17:30:35,131][105620] Updated weights for policy 1, policy_version 294964 (0.0008) [2023-12-26 17:30:35,186][105620] Updated weights for policy 1, policy_version 294974 (0.0007) [2023-12-26 17:30:35,199][105692] Updated weights for policy 0, policy_version 294866 (0.0009) [2023-12-26 17:30:35,238][105620] Updated weights for policy 1, policy_version 294984 (0.0006) [2023-12-26 17:30:35,249][105692] Updated weights for policy 0, policy_version 294876 (0.0007) [2023-12-26 17:30:35,310][105692] Updated weights for policy 0, policy_version 294886 (0.0009) [2023-12-26 17:30:35,997][105620] Updated weights for policy 1, policy_version 294994 (0.0011) [2023-12-26 17:30:36,027][105692] Updated weights for policy 0, policy_version 294896 (0.0007) [2023-12-26 17:30:36,062][105620] Updated weights for policy 1, policy_version 295004 (0.0011) [2023-12-26 17:30:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 151027712. Throughput: 0: 9919.1, 1: 9557.4. Samples: 151023408. Policy #0 lag: (min: 11.0, avg: 23.4, max: 43.0) [2023-12-26 17:30:36,062][104569] Avg episode reward: [(0, '8276.503'), (1, '4683.186')] [2023-12-26 17:30:36,084][105692] Updated weights for policy 0, policy_version 294906 (0.0006) [2023-12-26 17:30:36,123][105620] Updated weights for policy 1, policy_version 295014 (0.0010) [2023-12-26 17:30:36,147][105692] Updated weights for policy 0, policy_version 294916 (0.0008) [2023-12-26 17:30:36,893][105620] Updated weights for policy 1, policy_version 295024 (0.0009) [2023-12-26 17:30:36,932][105692] Updated weights for policy 0, policy_version 294926 (0.0008) [2023-12-26 17:30:36,953][105620] Updated weights for policy 1, policy_version 295034 (0.0008) [2023-12-26 17:30:37,001][105692] Updated weights for policy 0, policy_version 294936 (0.0008) [2023-12-26 17:30:37,008][105620] Updated weights for policy 1, policy_version 295044 (0.0007) [2023-12-26 17:30:37,070][105692] Updated weights for policy 0, policy_version 294946 (0.0008) [2023-12-26 17:30:37,761][105620] Updated weights for policy 1, policy_version 295054 (0.0009) [2023-12-26 17:30:37,791][105692] Updated weights for policy 0, policy_version 294956 (0.0009) [2023-12-26 17:30:37,815][105620] Updated weights for policy 1, policy_version 295064 (0.0007) [2023-12-26 17:30:37,855][105692] Updated weights for policy 0, policy_version 294966 (0.0008) [2023-12-26 17:30:37,869][105620] Updated weights for policy 1, policy_version 295074 (0.0007) [2023-12-26 17:30:37,911][105692] Updated weights for policy 0, policy_version 294976 (0.0010) [2023-12-26 17:30:38,576][105620] Updated weights for policy 1, policy_version 295084 (0.0008) [2023-12-26 17:30:38,629][105620] Updated weights for policy 1, policy_version 295094 (0.0009) [2023-12-26 17:30:38,684][105620] Updated weights for policy 1, policy_version 295104 (0.0009) [2023-12-26 17:30:38,702][105692] Updated weights for policy 0, policy_version 294986 (0.0011) [2023-12-26 17:30:38,757][105692] Updated weights for policy 0, policy_version 294996 (0.0009) [2023-12-26 17:30:38,815][105692] Updated weights for policy 0, policy_version 295006 (0.0009) [2023-12-26 17:30:38,874][105692] Updated weights for policy 0, policy_version 295016 (0.0009) [2023-12-26 17:30:39,332][105620] Updated weights for policy 1, policy_version 295114 (0.0009) [2023-12-26 17:30:39,405][105620] Updated weights for policy 1, policy_version 295124 (0.0008) [2023-12-26 17:30:39,472][105620] Updated weights for policy 1, policy_version 295134 (0.0009) [2023-12-26 17:30:39,533][105620] Updated weights for policy 1, policy_version 295144 (0.0009) [2023-12-26 17:30:39,646][105692] Updated weights for policy 0, policy_version 295026 (0.0009) [2023-12-26 17:30:39,712][105692] Updated weights for policy 0, policy_version 295036 (0.0009) [2023-12-26 17:30:39,771][105692] Updated weights for policy 0, policy_version 295046 (0.0009) [2023-12-26 17:30:40,330][105620] Updated weights for policy 1, policy_version 295154 (0.0009) [2023-12-26 17:30:40,394][105620] Updated weights for policy 1, policy_version 295164 (0.0009) [2023-12-26 17:30:40,446][105620] Updated weights for policy 1, policy_version 295174 (0.0009) [2023-12-26 17:30:40,506][105692] Updated weights for policy 0, policy_version 295056 (0.0009) [2023-12-26 17:30:40,570][105692] Updated weights for policy 0, policy_version 295066 (0.0008) [2023-12-26 17:30:40,630][105692] Updated weights for policy 0, policy_version 295076 (0.0009) [2023-12-26 17:30:41,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 151126016. Throughput: 0: 9864.6, 1: 9573.5. Samples: 151135364. Policy #0 lag: (min: 11.0, avg: 23.4, max: 43.0) [2023-12-26 17:30:41,063][104569] Avg episode reward: [(0, '8912.782'), (1, '6662.347')] [2023-12-26 17:30:41,283][105620] Updated weights for policy 1, policy_version 295184 (0.0009) [2023-12-26 17:30:41,348][105620] Updated weights for policy 1, policy_version 295194 (0.0009) [2023-12-26 17:30:41,413][105692] Updated weights for policy 0, policy_version 295086 (0.0009) [2023-12-26 17:30:41,418][105620] Updated weights for policy 1, policy_version 295204 (0.0007) [2023-12-26 17:30:41,480][105692] Updated weights for policy 0, policy_version 295096 (0.0009) [2023-12-26 17:30:41,549][105692] Updated weights for policy 0, policy_version 295106 (0.0009) [2023-12-26 17:30:42,272][105620] Updated weights for policy 1, policy_version 295214 (0.0007) [2023-12-26 17:30:42,305][105692] Updated weights for policy 0, policy_version 295116 (0.0008) [2023-12-26 17:30:42,344][105620] Updated weights for policy 1, policy_version 295224 (0.0008) [2023-12-26 17:30:42,371][105692] Updated weights for policy 0, policy_version 295126 (0.0011) [2023-12-26 17:30:42,417][105620] Updated weights for policy 1, policy_version 295234 (0.0008) [2023-12-26 17:30:42,438][105692] Updated weights for policy 0, policy_version 295136 (0.0009) [2023-12-26 17:30:43,116][105620] Updated weights for policy 1, policy_version 295244 (0.0009) [2023-12-26 17:30:43,144][105586] KL-divergence is very high: 116.1581 [2023-12-26 17:30:43,175][105620] Updated weights for policy 1, policy_version 295254 (0.0009) [2023-12-26 17:30:43,189][105586] KL-divergence is very high: 160.7894 [2023-12-26 17:30:43,215][105692] Updated weights for policy 0, policy_version 295146 (0.0009) [2023-12-26 17:30:43,233][105620] Updated weights for policy 1, policy_version 295264 (0.0008) [2023-12-26 17:30:43,239][105586] KL-divergence is very high: 150.6186 [2023-12-26 17:30:43,273][105692] Updated weights for policy 0, policy_version 295156 (0.0007) [2023-12-26 17:30:43,332][105692] Updated weights for policy 0, policy_version 295166 (0.0008) [2023-12-26 17:30:43,398][105692] Updated weights for policy 0, policy_version 295176 (0.0009) [2023-12-26 17:30:43,985][105620] Updated weights for policy 1, policy_version 295274 (0.0008) [2023-12-26 17:30:44,049][105620] Updated weights for policy 1, policy_version 295284 (0.0009) [2023-12-26 17:30:44,104][105620] Updated weights for policy 1, policy_version 295294 (0.0009) [2023-12-26 17:30:44,167][105620] Updated weights for policy 1, policy_version 295304 (0.0007) [2023-12-26 17:30:44,173][105692] Updated weights for policy 0, policy_version 295186 (0.0007) [2023-12-26 17:30:44,228][105692] Updated weights for policy 0, policy_version 295196 (0.0008) [2023-12-26 17:30:44,290][105692] Updated weights for policy 0, policy_version 295206 (0.0007) [2023-12-26 17:30:44,918][105620] Updated weights for policy 1, policy_version 295314 (0.0008) [2023-12-26 17:30:44,970][105620] Updated weights for policy 1, policy_version 295324 (0.0008) [2023-12-26 17:30:45,024][105620] Updated weights for policy 1, policy_version 295334 (0.0007) [2023-12-26 17:30:45,045][105692] Updated weights for policy 0, policy_version 295216 (0.0009) [2023-12-26 17:30:45,112][105692] Updated weights for policy 0, policy_version 295226 (0.0009) [2023-12-26 17:30:45,179][105692] Updated weights for policy 0, policy_version 295236 (0.0005) [2023-12-26 17:30:45,777][105620] Updated weights for policy 1, policy_version 295344 (0.0008) [2023-12-26 17:30:45,839][105620] Updated weights for policy 1, policy_version 295354 (0.0009) [2023-12-26 17:30:45,883][105692] Updated weights for policy 0, policy_version 295246 (0.0007) [2023-12-26 17:30:45,895][105620] Updated weights for policy 1, policy_version 295364 (0.0008) [2023-12-26 17:30:45,935][105692] Updated weights for policy 0, policy_version 295256 (0.0009) [2023-12-26 17:30:45,998][105692] Updated weights for policy 0, policy_version 295266 (0.0010) [2023-12-26 17:30:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 151224320. Throughput: 0: 9734.8, 1: 9579.6. Samples: 151189236. Policy #0 lag: (min: 11.0, avg: 23.4, max: 43.0) [2023-12-26 17:30:46,062][104569] Avg episode reward: [(0, '9001.091'), (1, '6948.898')] [2023-12-26 17:30:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000295368_75620352.pth... [2023-12-26 17:30:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000295272_75603968.pth... [2023-12-26 17:30:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000294120_75309056.pth [2023-12-26 17:30:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000294248_75333632.pth [2023-12-26 17:30:46,686][105620] Updated weights for policy 1, policy_version 295374 (0.0008) [2023-12-26 17:30:46,748][105620] Updated weights for policy 1, policy_version 295384 (0.0009) [2023-12-26 17:30:46,767][105692] Updated weights for policy 0, policy_version 295276 (0.0008) [2023-12-26 17:30:46,801][105620] Updated weights for policy 1, policy_version 295394 (0.0007) [2023-12-26 17:30:46,823][105692] Updated weights for policy 0, policy_version 295286 (0.0008) [2023-12-26 17:30:46,884][105692] Updated weights for policy 0, policy_version 295296 (0.0008) [2023-12-26 17:30:47,581][105620] Updated weights for policy 1, policy_version 295404 (0.0006) [2023-12-26 17:30:47,603][105692] Updated weights for policy 0, policy_version 295306 (0.0008) [2023-12-26 17:30:47,646][105620] Updated weights for policy 1, policy_version 295414 (0.0006) [2023-12-26 17:30:47,658][105692] Updated weights for policy 0, policy_version 295316 (0.0011) [2023-12-26 17:30:47,695][105620] Updated weights for policy 1, policy_version 295424 (0.0005) [2023-12-26 17:30:47,707][105692] Updated weights for policy 0, policy_version 295326 (0.0010) [2023-12-26 17:30:48,320][105620] Updated weights for policy 1, policy_version 295434 (0.0005) [2023-12-26 17:30:48,382][105692] Updated weights for policy 0, policy_version 295337 (0.0010) [2023-12-26 17:30:48,387][105620] Updated weights for policy 1, policy_version 295444 (0.0008) [2023-12-26 17:30:48,437][105692] Updated weights for policy 0, policy_version 295347 (0.0007) [2023-12-26 17:30:48,446][105620] Updated weights for policy 1, policy_version 295454 (0.0009) [2023-12-26 17:30:48,486][105692] Updated weights for policy 0, policy_version 295357 (0.0008) [2023-12-26 17:30:48,507][105620] Updated weights for policy 1, policy_version 295464 (0.0011) [2023-12-26 17:30:48,551][105692] Updated weights for policy 0, policy_version 295367 (0.0008) [2023-12-26 17:30:49,199][105620] Updated weights for policy 1, policy_version 295474 (0.0010) [2023-12-26 17:30:49,231][105692] Updated weights for policy 0, policy_version 295377 (0.0007) [2023-12-26 17:30:49,262][105620] Updated weights for policy 1, policy_version 295484 (0.0010) [2023-12-26 17:30:49,292][105692] Updated weights for policy 0, policy_version 295387 (0.0007) [2023-12-26 17:30:49,325][105620] Updated weights for policy 1, policy_version 295494 (0.0010) [2023-12-26 17:30:49,357][105692] Updated weights for policy 0, policy_version 295397 (0.0008) [2023-12-26 17:30:49,944][105620] Updated weights for policy 1, policy_version 295504 (0.0009) [2023-12-26 17:30:50,011][105620] Updated weights for policy 1, policy_version 295514 (0.0008) [2023-12-26 17:30:50,029][105692] Updated weights for policy 0, policy_version 295407 (0.0008) [2023-12-26 17:30:50,060][105620] Updated weights for policy 1, policy_version 295524 (0.0006) [2023-12-26 17:30:50,082][105692] Updated weights for policy 0, policy_version 295417 (0.0007) [2023-12-26 17:30:50,133][105692] Updated weights for policy 0, policy_version 295427 (0.0009) [2023-12-26 17:30:50,709][105620] Updated weights for policy 1, policy_version 295534 (0.0007) [2023-12-26 17:30:50,778][105620] Updated weights for policy 1, policy_version 295544 (0.0009) [2023-12-26 17:30:50,839][105620] Updated weights for policy 1, policy_version 295554 (0.0008) [2023-12-26 17:30:50,951][105692] Updated weights for policy 0, policy_version 295437 (0.0009) [2023-12-26 17:30:51,014][105692] Updated weights for policy 0, policy_version 295447 (0.0009) [2023-12-26 17:30:51,062][104569] Fps is (10 sec: 18842.3, 60 sec: 19251.2, 300 sec: 19633.0). Total num frames: 151314432. Throughput: 0: 9689.3, 1: 9629.0. Samples: 151304572. Policy #0 lag: (min: 11.0, avg: 23.4, max: 43.0) [2023-12-26 17:30:51,062][104569] Avg episode reward: [(0, '9268.867'), (1, '6949.178')] [2023-12-26 17:30:51,078][105692] Updated weights for policy 0, policy_version 295457 (0.0009) [2023-12-26 17:30:51,629][105620] Updated weights for policy 1, policy_version 295564 (0.0009) [2023-12-26 17:30:51,687][105620] Updated weights for policy 1, policy_version 295574 (0.0008) [2023-12-26 17:30:51,760][105620] Updated weights for policy 1, policy_version 295584 (0.0009) [2023-12-26 17:30:51,790][105692] Updated weights for policy 0, policy_version 295467 (0.0009) [2023-12-26 17:30:51,846][105692] Updated weights for policy 0, policy_version 295477 (0.0008) [2023-12-26 17:30:51,910][105692] Updated weights for policy 0, policy_version 295487 (0.0008) [2023-12-26 17:30:52,498][105620] Updated weights for policy 1, policy_version 295594 (0.0008) [2023-12-26 17:30:52,565][105620] Updated weights for policy 1, policy_version 295604 (0.0009) [2023-12-26 17:30:52,627][105620] Updated weights for policy 1, policy_version 295614 (0.0009) [2023-12-26 17:30:52,672][105692] Updated weights for policy 0, policy_version 295497 (0.0008) [2023-12-26 17:30:52,687][105620] Updated weights for policy 1, policy_version 295624 (0.0008) [2023-12-26 17:30:52,738][105692] Updated weights for policy 0, policy_version 295507 (0.0009) [2023-12-26 17:30:52,809][105692] Updated weights for policy 0, policy_version 295517 (0.0010) [2023-12-26 17:30:52,866][105692] Updated weights for policy 0, policy_version 295527 (0.0009) [2023-12-26 17:30:53,337][105620] Updated weights for policy 1, policy_version 295634 (0.0005) [2023-12-26 17:30:53,394][105620] Updated weights for policy 1, policy_version 295644 (0.0007) [2023-12-26 17:30:53,440][105620] Updated weights for policy 1, policy_version 295654 (0.0008) [2023-12-26 17:30:53,679][105692] Updated weights for policy 0, policy_version 295537 (0.0009) [2023-12-26 17:30:53,748][105692] Updated weights for policy 0, policy_version 295547 (0.0009) [2023-12-26 17:30:53,802][105692] Updated weights for policy 0, policy_version 295557 (0.0009) [2023-12-26 17:30:54,155][105620] Updated weights for policy 1, policy_version 295664 (0.0006) [2023-12-26 17:30:54,223][105620] Updated weights for policy 1, policy_version 295674 (0.0005) [2023-12-26 17:30:54,279][105620] Updated weights for policy 1, policy_version 295684 (0.0008) [2023-12-26 17:30:54,564][105692] Updated weights for policy 0, policy_version 295567 (0.0009) [2023-12-26 17:30:54,615][105692] Updated weights for policy 0, policy_version 295577 (0.0009) [2023-12-26 17:30:54,661][105692] Updated weights for policy 0, policy_version 295587 (0.0008) [2023-12-26 17:30:54,953][105620] Updated weights for policy 1, policy_version 295694 (0.0009) [2023-12-26 17:30:55,001][105620] Updated weights for policy 1, policy_version 295704 (0.0009) [2023-12-26 17:30:55,055][105620] Updated weights for policy 1, policy_version 295714 (0.0008) [2023-12-26 17:30:55,409][105692] Updated weights for policy 0, policy_version 295597 (0.0008) [2023-12-26 17:30:55,456][105692] Updated weights for policy 0, policy_version 295607 (0.0008) [2023-12-26 17:30:55,511][105692] Updated weights for policy 0, policy_version 295617 (0.0009) [2023-12-26 17:30:55,790][105620] Updated weights for policy 1, policy_version 295724 (0.0009) [2023-12-26 17:30:55,837][105620] Updated weights for policy 1, policy_version 295734 (0.0009) [2023-12-26 17:30:55,868][105586] KL-divergence is very high: 144.7792 [2023-12-26 17:30:55,885][105620] Updated weights for policy 1, policy_version 295744 (0.0009) [2023-12-26 17:30:55,906][105586] KL-divergence is very high: 137.8043 [2023-12-26 17:30:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.3, 300 sec: 19633.0). Total num frames: 151412736. Throughput: 0: 9566.8, 1: 9633.1. Samples: 151418160. Policy #0 lag: (min: 11.0, avg: 23.4, max: 43.0) [2023-12-26 17:30:56,063][104569] Avg episode reward: [(0, '8906.815'), (1, '7412.276')] [2023-12-26 17:30:56,326][105692] Updated weights for policy 0, policy_version 295627 (0.0009) [2023-12-26 17:30:56,387][105692] Updated weights for policy 0, policy_version 295637 (0.0009) [2023-12-26 17:30:56,434][105692] Updated weights for policy 0, policy_version 295647 (0.0008) [2023-12-26 17:30:56,551][105620] Updated weights for policy 1, policy_version 295754 (0.0009) [2023-12-26 17:30:56,606][105620] Updated weights for policy 1, policy_version 295764 (0.0009) [2023-12-26 17:30:56,656][105620] Updated weights for policy 1, policy_version 295774 (0.0008) [2023-12-26 17:30:56,707][105620] Updated weights for policy 1, policy_version 295784 (0.0009) [2023-12-26 17:30:57,163][105692] Updated weights for policy 0, policy_version 295657 (0.0008) [2023-12-26 17:30:57,224][105692] Updated weights for policy 0, policy_version 295667 (0.0009) [2023-12-26 17:30:57,278][105692] Updated weights for policy 0, policy_version 295677 (0.0010) [2023-12-26 17:30:57,333][105692] Updated weights for policy 0, policy_version 295687 (0.0009) [2023-12-26 17:30:57,436][105620] Updated weights for policy 1, policy_version 295794 (0.0009) [2023-12-26 17:30:57,492][105620] Updated weights for policy 1, policy_version 295804 (0.0009) [2023-12-26 17:30:57,541][105620] Updated weights for policy 1, policy_version 295814 (0.0008) [2023-12-26 17:30:58,079][105692] Updated weights for policy 0, policy_version 295697 (0.0009) [2023-12-26 17:30:58,141][105692] Updated weights for policy 0, policy_version 295707 (0.0009) [2023-12-26 17:30:58,194][105692] Updated weights for policy 0, policy_version 295717 (0.0009) [2023-12-26 17:30:58,360][105620] Updated weights for policy 1, policy_version 295824 (0.0008) [2023-12-26 17:30:58,423][105620] Updated weights for policy 1, policy_version 295834 (0.0009) [2023-12-26 17:30:58,486][105620] Updated weights for policy 1, policy_version 295844 (0.0008) [2023-12-26 17:30:59,067][105692] Updated weights for policy 0, policy_version 295727 (0.0010) [2023-12-26 17:30:59,128][105692] Updated weights for policy 0, policy_version 295737 (0.0010) [2023-12-26 17:30:59,189][105692] Updated weights for policy 0, policy_version 295747 (0.0008) [2023-12-26 17:30:59,316][105620] Updated weights for policy 1, policy_version 295854 (0.0008) [2023-12-26 17:30:59,382][105620] Updated weights for policy 1, policy_version 295864 (0.0007) [2023-12-26 17:30:59,442][105620] Updated weights for policy 1, policy_version 295874 (0.0008) [2023-12-26 17:30:59,978][105692] Updated weights for policy 0, policy_version 295757 (0.0008) [2023-12-26 17:31:00,036][105692] Updated weights for policy 0, policy_version 295767 (0.0010) [2023-12-26 17:31:00,042][105620] Updated weights for policy 1, policy_version 295884 (0.0006) [2023-12-26 17:31:00,104][105620] Updated weights for policy 1, policy_version 295894 (0.0006) [2023-12-26 17:31:00,104][105692] Updated weights for policy 0, policy_version 295777 (0.0010) [2023-12-26 17:31:00,159][105620] Updated weights for policy 1, policy_version 295904 (0.0007) [2023-12-26 17:31:00,798][105692] Updated weights for policy 0, policy_version 295787 (0.0009) [2023-12-26 17:31:00,826][105620] Updated weights for policy 1, policy_version 295914 (0.0008) [2023-12-26 17:31:00,848][105692] Updated weights for policy 0, policy_version 295797 (0.0006) [2023-12-26 17:31:00,882][105620] Updated weights for policy 1, policy_version 295924 (0.0009) [2023-12-26 17:31:00,897][105692] Updated weights for policy 0, policy_version 295807 (0.0006) [2023-12-26 17:31:00,940][105620] Updated weights for policy 1, policy_version 295934 (0.0009) [2023-12-26 17:31:00,993][105620] Updated weights for policy 1, policy_version 295944 (0.0008) [2023-12-26 17:31:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 151511040. Throughput: 0: 9553.4, 1: 9633.8. Samples: 151473788. Policy #0 lag: (min: 11.0, avg: 23.4, max: 43.0) [2023-12-26 17:31:01,063][104569] Avg episode reward: [(0, '8816.755'), (1, '7227.293')] [2023-12-26 17:31:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000295816_75743232.pth... [2023-12-26 17:31:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000295944_75767808.pth... [2023-12-26 17:31:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000294728_75464704.pth [2023-12-26 17:31:01,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000294792_75472896.pth [2023-12-26 17:31:01,553][105692] Updated weights for policy 0, policy_version 295817 (0.0005) [2023-12-26 17:31:01,612][105692] Updated weights for policy 0, policy_version 295827 (0.0009) [2023-12-26 17:31:01,675][105692] Updated weights for policy 0, policy_version 295837 (0.0008) [2023-12-26 17:31:01,744][105692] Updated weights for policy 0, policy_version 295847 (0.0007) [2023-12-26 17:31:01,775][105620] Updated weights for policy 1, policy_version 295954 (0.0008) [2023-12-26 17:31:01,842][105620] Updated weights for policy 1, policy_version 295964 (0.0010) [2023-12-26 17:31:01,898][105620] Updated weights for policy 1, policy_version 295974 (0.0010) [2023-12-26 17:31:02,437][105692] Updated weights for policy 0, policy_version 295857 (0.0009) [2023-12-26 17:31:02,493][105692] Updated weights for policy 0, policy_version 295867 (0.0008) [2023-12-26 17:31:02,549][105692] Updated weights for policy 0, policy_version 295877 (0.0008) [2023-12-26 17:31:02,661][105620] Updated weights for policy 1, policy_version 295984 (0.0010) [2023-12-26 17:31:02,709][105620] Updated weights for policy 1, policy_version 295994 (0.0010) [2023-12-26 17:31:02,767][105620] Updated weights for policy 1, policy_version 296004 (0.0010) [2023-12-26 17:31:03,307][105692] Updated weights for policy 0, policy_version 295887 (0.0008) [2023-12-26 17:31:03,358][105692] Updated weights for policy 0, policy_version 295897 (0.0007) [2023-12-26 17:31:03,406][105692] Updated weights for policy 0, policy_version 295907 (0.0008) [2023-12-26 17:31:03,509][105620] Updated weights for policy 1, policy_version 296014 (0.0010) [2023-12-26 17:31:03,566][105620] Updated weights for policy 1, policy_version 296024 (0.0010) [2023-12-26 17:31:03,621][105620] Updated weights for policy 1, policy_version 296034 (0.0010) [2023-12-26 17:31:03,646][105586] KL-divergence is very high: 135.7876 [2023-12-26 17:31:04,193][105692] Updated weights for policy 0, policy_version 295917 (0.0008) [2023-12-26 17:31:04,249][105692] Updated weights for policy 0, policy_version 295927 (0.0009) [2023-12-26 17:31:04,314][105692] Updated weights for policy 0, policy_version 295937 (0.0009) [2023-12-26 17:31:04,400][105620] Updated weights for policy 1, policy_version 296044 (0.0010) [2023-12-26 17:31:04,453][105620] Updated weights for policy 1, policy_version 296054 (0.0007) [2023-12-26 17:31:04,518][105620] Updated weights for policy 1, policy_version 296064 (0.0009) [2023-12-26 17:31:05,094][105692] Updated weights for policy 0, policy_version 295947 (0.0008) [2023-12-26 17:31:05,143][105692] Updated weights for policy 0, policy_version 295957 (0.0008) [2023-12-26 17:31:05,187][105692] Updated weights for policy 0, policy_version 295967 (0.0008) [2023-12-26 17:31:05,251][105620] Updated weights for policy 1, policy_version 296074 (0.0010) [2023-12-26 17:31:05,302][105620] Updated weights for policy 1, policy_version 296084 (0.0010) [2023-12-26 17:31:05,353][105620] Updated weights for policy 1, policy_version 296094 (0.0010) [2023-12-26 17:31:05,404][105620] Updated weights for policy 1, policy_version 296104 (0.0010) [2023-12-26 17:31:05,991][105692] Updated weights for policy 0, policy_version 295977 (0.0008) [2023-12-26 17:31:06,042][105692] Updated weights for policy 0, policy_version 295987 (0.0008) [2023-12-26 17:31:06,062][104569] Fps is (10 sec: 18022.7, 60 sec: 19114.7, 300 sec: 19605.3). Total num frames: 151592960. Throughput: 0: 9572.0, 1: 9557.2. Samples: 151588092. Policy #0 lag: (min: 11.0, avg: 23.4, max: 43.0) [2023-12-26 17:31:06,062][104569] Avg episode reward: [(0, '8996.610'), (1, '7597.280')] [2023-12-26 17:31:06,097][105692] Updated weights for policy 0, policy_version 295997 (0.0007) [2023-12-26 17:31:06,110][105620] Updated weights for policy 1, policy_version 296114 (0.0008) [2023-12-26 17:31:06,160][105692] Updated weights for policy 0, policy_version 296007 (0.0008) [2023-12-26 17:31:06,169][105620] Updated weights for policy 1, policy_version 296124 (0.0008) [2023-12-26 17:31:06,230][105620] Updated weights for policy 1, policy_version 296134 (0.0007) [2023-12-26 17:31:06,939][105692] Updated weights for policy 0, policy_version 296017 (0.0009) [2023-12-26 17:31:06,974][105620] Updated weights for policy 1, policy_version 296144 (0.0008) [2023-12-26 17:31:06,998][105692] Updated weights for policy 0, policy_version 296027 (0.0008) [2023-12-26 17:31:07,021][105620] Updated weights for policy 1, policy_version 296154 (0.0007) [2023-12-26 17:31:07,053][105692] Updated weights for policy 0, policy_version 296037 (0.0008) [2023-12-26 17:31:07,072][105620] Updated weights for policy 1, policy_version 296164 (0.0006) [2023-12-26 17:31:07,737][105620] Updated weights for policy 1, policy_version 296174 (0.0009) [2023-12-26 17:31:07,796][105620] Updated weights for policy 1, policy_version 296184 (0.0009) [2023-12-26 17:31:07,855][105692] Updated weights for policy 0, policy_version 296047 (0.0009) [2023-12-26 17:31:07,857][105620] Updated weights for policy 1, policy_version 296194 (0.0006) [2023-12-26 17:31:07,914][105692] Updated weights for policy 0, policy_version 296057 (0.0008) [2023-12-26 17:31:07,965][105692] Updated weights for policy 0, policy_version 296067 (0.0009) [2023-12-26 17:31:08,549][105620] Updated weights for policy 1, policy_version 296204 (0.0008) [2023-12-26 17:31:08,604][105620] Updated weights for policy 1, policy_version 296214 (0.0009) [2023-12-26 17:31:08,655][105620] Updated weights for policy 1, policy_version 296224 (0.0009) [2023-12-26 17:31:08,770][105692] Updated weights for policy 0, policy_version 296077 (0.0009) [2023-12-26 17:31:08,832][105692] Updated weights for policy 0, policy_version 296087 (0.0009) [2023-12-26 17:31:08,886][105692] Updated weights for policy 0, policy_version 296097 (0.0009) [2023-12-26 17:31:09,477][105620] Updated weights for policy 1, policy_version 296234 (0.0009) [2023-12-26 17:31:09,536][105620] Updated weights for policy 1, policy_version 296244 (0.0009) [2023-12-26 17:31:09,592][105620] Updated weights for policy 1, policy_version 296254 (0.0009) [2023-12-26 17:31:09,631][105692] Updated weights for policy 0, policy_version 296107 (0.0008) [2023-12-26 17:31:09,658][105620] Updated weights for policy 1, policy_version 296264 (0.0007) [2023-12-26 17:31:09,691][105692] Updated weights for policy 0, policy_version 296117 (0.0007) [2023-12-26 17:31:09,749][105692] Updated weights for policy 0, policy_version 296127 (0.0009) [2023-12-26 17:31:10,345][105620] Updated weights for policy 1, policy_version 296274 (0.0008) [2023-12-26 17:31:10,411][105620] Updated weights for policy 1, policy_version 296284 (0.0006) [2023-12-26 17:31:10,474][105620] Updated weights for policy 1, policy_version 296294 (0.0007) [2023-12-26 17:31:10,486][105692] Updated weights for policy 0, policy_version 296137 (0.0008) [2023-12-26 17:31:10,545][105692] Updated weights for policy 0, policy_version 296147 (0.0011) [2023-12-26 17:31:10,607][105692] Updated weights for policy 0, policy_version 296157 (0.0010) [2023-12-26 17:31:10,666][105692] Updated weights for policy 0, policy_version 296167 (0.0010) [2023-12-26 17:31:11,062][104569] Fps is (10 sec: 18022.6, 60 sec: 19114.8, 300 sec: 19605.3). Total num frames: 151691264. Throughput: 0: 9557.9, 1: 9586.5. Samples: 151701176. Policy #0 lag: (min: 11.0, avg: 23.4, max: 43.0) [2023-12-26 17:31:11,062][104569] Avg episode reward: [(0, '9087.187'), (1, '7043.395')] [2023-12-26 17:31:11,201][105620] Updated weights for policy 1, policy_version 296304 (0.0008) [2023-12-26 17:31:11,261][105620] Updated weights for policy 1, policy_version 296314 (0.0008) [2023-12-26 17:31:11,321][105620] Updated weights for policy 1, policy_version 296324 (0.0008) [2023-12-26 17:31:11,420][105692] Updated weights for policy 0, policy_version 296177 (0.0008) [2023-12-26 17:31:11,483][105692] Updated weights for policy 0, policy_version 296187 (0.0011) [2023-12-26 17:31:11,545][105692] Updated weights for policy 0, policy_version 296197 (0.0011) [2023-12-26 17:31:12,054][105620] Updated weights for policy 1, policy_version 296334 (0.0007) [2023-12-26 17:31:12,121][105620] Updated weights for policy 1, policy_version 296344 (0.0006) [2023-12-26 17:31:12,191][105620] Updated weights for policy 1, policy_version 296354 (0.0005) [2023-12-26 17:31:12,322][105692] Updated weights for policy 0, policy_version 296207 (0.0009) [2023-12-26 17:31:12,387][105692] Updated weights for policy 0, policy_version 296217 (0.0008) [2023-12-26 17:31:12,448][105692] Updated weights for policy 0, policy_version 296227 (0.0009) [2023-12-26 17:31:12,807][105620] Updated weights for policy 1, policy_version 296364 (0.0008) [2023-12-26 17:31:12,873][105620] Updated weights for policy 1, policy_version 296374 (0.0008) [2023-12-26 17:31:12,924][105620] Updated weights for policy 1, policy_version 296384 (0.0009) [2023-12-26 17:31:13,235][105692] Updated weights for policy 0, policy_version 296237 (0.0009) [2023-12-26 17:31:13,287][105692] Updated weights for policy 0, policy_version 296247 (0.0010) [2023-12-26 17:31:13,353][105692] Updated weights for policy 0, policy_version 296257 (0.0007) [2023-12-26 17:31:13,561][105620] Updated weights for policy 1, policy_version 296394 (0.0008) [2023-12-26 17:31:13,621][105620] Updated weights for policy 1, policy_version 296404 (0.0006) [2023-12-26 17:31:13,676][105620] Updated weights for policy 1, policy_version 296414 (0.0005) [2023-12-26 17:31:13,743][105620] Updated weights for policy 1, policy_version 296424 (0.0005) [2023-12-26 17:31:13,944][105692] Updated weights for policy 0, policy_version 296267 (0.0009) [2023-12-26 17:31:13,988][105692] Updated weights for policy 0, policy_version 296277 (0.0005) [2023-12-26 17:31:14,034][105692] Updated weights for policy 0, policy_version 296287 (0.0005) [2023-12-26 17:31:14,334][105620] Updated weights for policy 1, policy_version 296434 (0.0008) [2023-12-26 17:31:14,393][105620] Updated weights for policy 1, policy_version 296444 (0.0008) [2023-12-26 17:31:14,452][105620] Updated weights for policy 1, policy_version 296454 (0.0008) [2023-12-26 17:31:14,765][105692] Updated weights for policy 0, policy_version 296297 (0.0009) [2023-12-26 17:31:14,829][105692] Updated weights for policy 0, policy_version 296307 (0.0008) [2023-12-26 17:31:14,889][105692] Updated weights for policy 0, policy_version 296317 (0.0008) [2023-12-26 17:31:14,945][105692] Updated weights for policy 0, policy_version 296327 (0.0011) [2023-12-26 17:31:15,223][105620] Updated weights for policy 1, policy_version 296464 (0.0010) [2023-12-26 17:31:15,283][105620] Updated weights for policy 1, policy_version 296474 (0.0010) [2023-12-26 17:31:15,348][105620] Updated weights for policy 1, policy_version 296484 (0.0010) [2023-12-26 17:31:15,659][105692] Updated weights for policy 0, policy_version 296337 (0.0006) [2023-12-26 17:31:15,715][105692] Updated weights for policy 0, policy_version 296347 (0.0005) [2023-12-26 17:31:15,769][105692] Updated weights for policy 0, policy_version 296357 (0.0005) [2023-12-26 17:31:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 151789568. Throughput: 0: 9436.0, 1: 9643.7. Samples: 151759528. Policy #0 lag: (min: 11.0, avg: 23.4, max: 43.0) [2023-12-26 17:31:16,062][104569] Avg episode reward: [(0, '8998.513'), (1, '7226.931')] [2023-12-26 17:31:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000296360_75882496.pth... [2023-12-26 17:31:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000295272_75603968.pth [2023-12-26 17:31:16,087][105620] Updated weights for policy 1, policy_version 296494 (0.0010) [2023-12-26 17:31:16,139][105620] Updated weights for policy 1, policy_version 296504 (0.0010) [2023-12-26 17:31:16,200][105620] Updated weights for policy 1, policy_version 296514 (0.0010) [2023-12-26 17:31:16,240][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000296520_75915264.pth... [2023-12-26 17:31:16,245][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000295368_75620352.pth [2023-12-26 17:31:16,289][105692] Updated weights for policy 0, policy_version 296367 (0.0005) [2023-12-26 17:31:16,333][105692] Updated weights for policy 0, policy_version 296377 (0.0005) [2023-12-26 17:31:16,377][105692] Updated weights for policy 0, policy_version 296387 (0.0005) [2023-12-26 17:31:16,797][105620] Updated weights for policy 1, policy_version 296524 (0.0009) [2023-12-26 17:31:16,853][105620] Updated weights for policy 1, policy_version 296534 (0.0011) [2023-12-26 17:31:16,914][105620] Updated weights for policy 1, policy_version 296544 (0.0010) [2023-12-26 17:31:17,023][105692] Updated weights for policy 0, policy_version 296397 (0.0008) [2023-12-26 17:31:17,089][105692] Updated weights for policy 0, policy_version 296407 (0.0011) [2023-12-26 17:31:17,155][105692] Updated weights for policy 0, policy_version 296417 (0.0010) [2023-12-26 17:31:17,663][105620] Updated weights for policy 1, policy_version 296554 (0.0010) [2023-12-26 17:31:17,714][105620] Updated weights for policy 1, policy_version 296564 (0.0010) [2023-12-26 17:31:17,769][105620] Updated weights for policy 1, policy_version 296574 (0.0006) [2023-12-26 17:31:17,821][105692] Updated weights for policy 0, policy_version 296427 (0.0009) [2023-12-26 17:31:17,833][105620] Updated weights for policy 1, policy_version 296584 (0.0006) [2023-12-26 17:31:17,869][105692] Updated weights for policy 0, policy_version 296437 (0.0007) [2023-12-26 17:31:17,934][105692] Updated weights for policy 0, policy_version 296447 (0.0007) [2023-12-26 17:31:18,550][105620] Updated weights for policy 1, policy_version 296594 (0.0010) [2023-12-26 17:31:18,606][105692] Updated weights for policy 0, policy_version 296457 (0.0008) [2023-12-26 17:31:18,608][105620] Updated weights for policy 1, policy_version 296604 (0.0010) [2023-12-26 17:31:18,665][105692] Updated weights for policy 0, policy_version 296467 (0.0008) [2023-12-26 17:31:18,668][105620] Updated weights for policy 1, policy_version 296614 (0.0006) [2023-12-26 17:31:18,720][105692] Updated weights for policy 0, policy_version 296477 (0.0008) [2023-12-26 17:31:18,775][105692] Updated weights for policy 0, policy_version 296487 (0.0009) [2023-12-26 17:31:19,270][105620] Updated weights for policy 1, policy_version 296624 (0.0007) [2023-12-26 17:31:19,332][105620] Updated weights for policy 1, policy_version 296634 (0.0008) [2023-12-26 17:31:19,397][105620] Updated weights for policy 1, policy_version 296644 (0.0009) [2023-12-26 17:31:19,565][105692] Updated weights for policy 0, policy_version 296497 (0.0009) [2023-12-26 17:31:19,625][105692] Updated weights for policy 0, policy_version 296507 (0.0009) [2023-12-26 17:31:19,679][105692] Updated weights for policy 0, policy_version 296517 (0.0008) [2023-12-26 17:31:20,081][105620] Updated weights for policy 1, policy_version 296654 (0.0010) [2023-12-26 17:31:20,148][105620] Updated weights for policy 1, policy_version 296664 (0.0011) [2023-12-26 17:31:20,218][105620] Updated weights for policy 1, policy_version 296674 (0.0011) [2023-12-26 17:31:20,373][105692] Updated weights for policy 0, policy_version 296527 (0.0006) [2023-12-26 17:31:20,443][105692] Updated weights for policy 0, policy_version 296537 (0.0005) [2023-12-26 17:31:20,511][105692] Updated weights for policy 0, policy_version 296547 (0.0006) [2023-12-26 17:31:20,951][105620] Updated weights for policy 1, policy_version 296684 (0.0011) [2023-12-26 17:31:21,017][105620] Updated weights for policy 1, policy_version 296694 (0.0011) [2023-12-26 17:31:21,048][105692] Updated weights for policy 0, policy_version 296557 (0.0007) [2023-12-26 17:31:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19114.7, 300 sec: 19605.3). Total num frames: 151887872. Throughput: 0: 9407.5, 1: 9663.7. Samples: 151881616. Policy #0 lag: (min: 11.0, avg: 23.4, max: 43.0) [2023-12-26 17:31:21,063][104569] Avg episode reward: [(0, '9089.454'), (1, '7873.664')] [2023-12-26 17:31:21,084][105620] Updated weights for policy 1, policy_version 296704 (0.0007) [2023-12-26 17:31:21,119][105692] Updated weights for policy 0, policy_version 296567 (0.0008) [2023-12-26 17:31:21,190][105692] Updated weights for policy 0, policy_version 296577 (0.0008) [2023-12-26 17:31:21,805][105620] Updated weights for policy 1, policy_version 296714 (0.0007) [2023-12-26 17:31:21,852][105620] Updated weights for policy 1, policy_version 296724 (0.0009) [2023-12-26 17:31:21,902][105620] Updated weights for policy 1, policy_version 296734 (0.0005) [2023-12-26 17:31:21,959][105620] Updated weights for policy 1, policy_version 296744 (0.0005) [2023-12-26 17:31:21,988][105692] Updated weights for policy 0, policy_version 296587 (0.0008) [2023-12-26 17:31:22,060][105692] Updated weights for policy 0, policy_version 296597 (0.0009) [2023-12-26 17:31:22,133][105692] Updated weights for policy 0, policy_version 296607 (0.0008) [2023-12-26 17:31:22,616][105586] KL-divergence is very high: 105.5105 [2023-12-26 17:31:22,622][105586] KL-divergence is very high: 142.4832 [2023-12-26 17:31:22,628][105586] KL-divergence is very high: 304.7016 [2023-12-26 17:31:22,629][105620] Updated weights for policy 1, policy_version 296754 (0.0011) [2023-12-26 17:31:22,642][105586] KL-divergence is very high: 325.2167 [2023-12-26 17:31:22,649][105586] KL-divergence is very high: 188.5700 [2023-12-26 17:31:22,663][105586] KL-divergence is very high: 403.5307 [2023-12-26 17:31:22,670][105586] KL-divergence is very high: 217.6070 [2023-12-26 17:31:22,677][105586] KL-divergence is very high: 218.6034 [2023-12-26 17:31:22,685][105586] KL-divergence is very high: 484.6998 [2023-12-26 17:31:22,698][105620] Updated weights for policy 1, policy_version 296764 (0.0010) [2023-12-26 17:31:22,699][105586] KL-divergence is very high: 410.1936 [2023-12-26 17:31:22,705][105586] KL-divergence is very high: 184.6546 [2023-12-26 17:31:22,718][105586] KL-divergence is very high: 368.3170 [2023-12-26 17:31:22,724][105586] KL-divergence is very high: 130.7413 [2023-12-26 17:31:22,730][105586] KL-divergence is very high: 117.3299 [2023-12-26 17:31:22,736][105586] KL-divergence is very high: 419.4573 [2023-12-26 17:31:22,748][105586] KL-divergence is very high: 335.4674 [2023-12-26 17:31:22,754][105586] KL-divergence is very high: 130.1161 [2023-12-26 17:31:22,760][105620] Updated weights for policy 1, policy_version 296774 (0.0010) [2023-12-26 17:31:22,766][105586] KL-divergence is very high: 277.9153 [2023-12-26 17:31:22,809][105692] Updated weights for policy 0, policy_version 296617 (0.0008) [2023-12-26 17:31:22,866][105692] Updated weights for policy 0, policy_version 296627 (0.0010) [2023-12-26 17:31:22,920][105692] Updated weights for policy 0, policy_version 296637 (0.0010) [2023-12-26 17:31:22,975][105692] Updated weights for policy 0, policy_version 296647 (0.0010) [2023-12-26 17:31:23,318][105620] Updated weights for policy 1, policy_version 296784 (0.0005) [2023-12-26 17:31:23,362][105586] KL-divergence is very high: 109.6665 [2023-12-26 17:31:23,386][105620] Updated weights for policy 1, policy_version 296794 (0.0006) [2023-12-26 17:31:23,410][105586] KL-divergence is very high: 110.5423 [2023-12-26 17:31:23,415][105586] KL-divergence is very high: 139.3029 [2023-12-26 17:31:23,430][105586] KL-divergence is very high: 100.3862 [2023-12-26 17:31:23,447][105620] Updated weights for policy 1, policy_version 296804 (0.0010) [2023-12-26 17:31:23,453][105586] KL-divergence is very high: 100.3601 [2023-12-26 17:31:23,459][105586] KL-divergence is very high: 116.6885 [2023-12-26 17:31:23,828][105692] Updated weights for policy 0, policy_version 296657 (0.0008) [2023-12-26 17:31:23,883][105692] Updated weights for policy 0, policy_version 296667 (0.0008) [2023-12-26 17:31:23,947][105692] Updated weights for policy 0, policy_version 296677 (0.0007) [2023-12-26 17:31:24,070][105620] Updated weights for policy 1, policy_version 296814 (0.0007) [2023-12-26 17:31:24,131][105620] Updated weights for policy 1, policy_version 296824 (0.0005) [2023-12-26 17:31:24,205][105620] Updated weights for policy 1, policy_version 296834 (0.0009) [2023-12-26 17:31:24,705][105692] Updated weights for policy 0, policy_version 296687 (0.0009) [2023-12-26 17:31:24,776][105692] Updated weights for policy 0, policy_version 296697 (0.0005) [2023-12-26 17:31:24,821][105620] Updated weights for policy 1, policy_version 296844 (0.0007) [2023-12-26 17:31:24,844][105692] Updated weights for policy 0, policy_version 296707 (0.0011) [2023-12-26 17:31:24,873][105620] Updated weights for policy 1, policy_version 296854 (0.0006) [2023-12-26 17:31:24,925][105620] Updated weights for policy 1, policy_version 296864 (0.0006) [2023-12-26 17:31:25,392][105692] Updated weights for policy 0, policy_version 296717 (0.0008) [2023-12-26 17:31:25,459][105692] Updated weights for policy 0, policy_version 296727 (0.0011) [2023-12-26 17:31:25,523][105692] Updated weights for policy 0, policy_version 296737 (0.0010) [2023-12-26 17:31:25,690][105620] Updated weights for policy 1, policy_version 296875 (0.0008) [2023-12-26 17:31:25,744][105620] Updated weights for policy 1, policy_version 296885 (0.0005) [2023-12-26 17:31:25,806][105620] Updated weights for policy 1, policy_version 296895 (0.0005) [2023-12-26 17:31:26,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19251.2, 300 sec: 19633.0). Total num frames: 151994368. Throughput: 0: 9487.3, 1: 9775.5. Samples: 152002180. Policy #0 lag: (min: 26.0, avg: 44.0, max: 58.0) [2023-12-26 17:31:26,062][104569] Avg episode reward: [(0, '9179.872'), (1, '7504.975')] [2023-12-26 17:31:26,177][105692] Updated weights for policy 0, policy_version 296747 (0.0010) [2023-12-26 17:31:26,225][105692] Updated weights for policy 0, policy_version 296757 (0.0010) [2023-12-26 17:31:26,270][105692] Updated weights for policy 0, policy_version 296767 (0.0010) [2023-12-26 17:31:26,321][105620] Updated weights for policy 1, policy_version 296905 (0.0006) [2023-12-26 17:31:26,374][105620] Updated weights for policy 1, policy_version 296915 (0.0005) [2023-12-26 17:31:26,434][105620] Updated weights for policy 1, policy_version 296925 (0.0005) [2023-12-26 17:31:26,489][105620] Updated weights for policy 1, policy_version 296935 (0.0007) [2023-12-26 17:31:26,897][105692] Updated weights for policy 0, policy_version 296777 (0.0010) [2023-12-26 17:31:26,951][105692] Updated weights for policy 0, policy_version 296787 (0.0006) [2023-12-26 17:31:27,011][105692] Updated weights for policy 0, policy_version 296797 (0.0007) [2023-12-26 17:31:27,061][105620] Updated weights for policy 1, policy_version 296945 (0.0006) [2023-12-26 17:31:27,066][105692] Updated weights for policy 0, policy_version 296807 (0.0010) [2023-12-26 17:31:27,113][105620] Updated weights for policy 1, policy_version 296955 (0.0005) [2023-12-26 17:31:27,166][105620] Updated weights for policy 1, policy_version 296965 (0.0005) [2023-12-26 17:31:27,691][105692] Updated weights for policy 0, policy_version 296817 (0.0009) [2023-12-26 17:31:27,739][105692] Updated weights for policy 0, policy_version 296827 (0.0010) [2023-12-26 17:31:27,783][105692] Updated weights for policy 0, policy_version 296837 (0.0010) [2023-12-26 17:31:27,875][105620] Updated weights for policy 1, policy_version 296975 (0.0007) [2023-12-26 17:31:27,940][105620] Updated weights for policy 1, policy_version 296985 (0.0008) [2023-12-26 17:31:28,005][105620] Updated weights for policy 1, policy_version 296995 (0.0008) [2023-12-26 17:31:28,469][105692] Updated weights for policy 0, policy_version 296847 (0.0010) [2023-12-26 17:31:28,530][105692] Updated weights for policy 0, policy_version 296857 (0.0010) [2023-12-26 17:31:28,593][105692] Updated weights for policy 0, policy_version 296867 (0.0009) [2023-12-26 17:31:28,772][105620] Updated weights for policy 1, policy_version 297005 (0.0008) [2023-12-26 17:31:28,838][105620] Updated weights for policy 1, policy_version 297015 (0.0005) [2023-12-26 17:31:28,901][105620] Updated weights for policy 1, policy_version 297025 (0.0006) [2023-12-26 17:31:29,282][105692] Updated weights for policy 0, policy_version 296877 (0.0010) [2023-12-26 17:31:29,344][105692] Updated weights for policy 0, policy_version 296887 (0.0010) [2023-12-26 17:31:29,404][105692] Updated weights for policy 0, policy_version 296897 (0.0009) [2023-12-26 17:31:29,558][105620] Updated weights for policy 1, policy_version 297035 (0.0005) [2023-12-26 17:31:29,611][105620] Updated weights for policy 1, policy_version 297045 (0.0005) [2023-12-26 17:31:29,665][105620] Updated weights for policy 1, policy_version 297055 (0.0008) [2023-12-26 17:31:30,158][105692] Updated weights for policy 0, policy_version 296907 (0.0009) [2023-12-26 17:31:30,212][105692] Updated weights for policy 0, policy_version 296917 (0.0005) [2023-12-26 17:31:30,260][105692] Updated weights for policy 0, policy_version 296927 (0.0005) [2023-12-26 17:31:30,405][105620] Updated weights for policy 1, policy_version 297065 (0.0008) [2023-12-26 17:31:30,464][105620] Updated weights for policy 1, policy_version 297075 (0.0005) [2023-12-26 17:31:30,518][105620] Updated weights for policy 1, policy_version 297085 (0.0007) [2023-12-26 17:31:30,572][105620] Updated weights for policy 1, policy_version 297096 (0.0010) [2023-12-26 17:31:30,920][105692] Updated weights for policy 0, policy_version 296937 (0.0010) [2023-12-26 17:31:30,970][105692] Updated weights for policy 0, policy_version 296947 (0.0010) [2023-12-26 17:31:31,025][105692] Updated weights for policy 0, policy_version 296957 (0.0010) [2023-12-26 17:31:31,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 152092672. Throughput: 0: 9593.8, 1: 9884.6. Samples: 152065764. Policy #0 lag: (min: 26.0, avg: 44.0, max: 58.0) [2023-12-26 17:31:31,063][104569] Avg episode reward: [(0, '9270.203'), (1, '8062.629')] [2023-12-26 17:31:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000297096_76062720.pth... [2023-12-26 17:31:31,090][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000295944_75767808.pth [2023-12-26 17:31:31,092][105692] Updated weights for policy 0, policy_version 296967 (0.0010) [2023-12-26 17:31:31,097][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000296968_76038144.pth... [2023-12-26 17:31:31,100][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000295816_75743232.pth [2023-12-26 17:31:31,192][105620] Updated weights for policy 1, policy_version 297106 (0.0008) [2023-12-26 17:31:31,251][105620] Updated weights for policy 1, policy_version 297116 (0.0008) [2023-12-26 17:31:31,312][105620] Updated weights for policy 1, policy_version 297126 (0.0008) [2023-12-26 17:31:31,755][105692] Updated weights for policy 0, policy_version 296977 (0.0007) [2023-12-26 17:31:31,803][105692] Updated weights for policy 0, policy_version 296987 (0.0005) [2023-12-26 17:31:31,853][105692] Updated weights for policy 0, policy_version 296997 (0.0010) [2023-12-26 17:31:32,131][105620] Updated weights for policy 1, policy_version 297136 (0.0008) [2023-12-26 17:31:32,181][105620] Updated weights for policy 1, policy_version 297146 (0.0008) [2023-12-26 17:31:32,232][105620] Updated weights for policy 1, policy_version 297156 (0.0008) [2023-12-26 17:31:32,598][105692] Updated weights for policy 0, policy_version 297007 (0.0010) [2023-12-26 17:31:32,658][105692] Updated weights for policy 0, policy_version 297017 (0.0011) [2023-12-26 17:31:32,724][105692] Updated weights for policy 0, policy_version 297027 (0.0011) [2023-12-26 17:31:33,013][105620] Updated weights for policy 1, policy_version 297166 (0.0009) [2023-12-26 17:31:33,060][105620] Updated weights for policy 1, policy_version 297176 (0.0007) [2023-12-26 17:31:33,110][105620] Updated weights for policy 1, policy_version 297186 (0.0008) [2023-12-26 17:31:33,433][105692] Updated weights for policy 0, policy_version 297037 (0.0010) [2023-12-26 17:31:33,486][105692] Updated weights for policy 0, policy_version 297047 (0.0008) [2023-12-26 17:31:33,543][105692] Updated weights for policy 0, policy_version 297057 (0.0009) [2023-12-26 17:31:33,867][105620] Updated weights for policy 1, policy_version 297196 (0.0007) [2023-12-26 17:31:33,917][105620] Updated weights for policy 1, policy_version 297206 (0.0005) [2023-12-26 17:31:33,970][105620] Updated weights for policy 1, policy_version 297216 (0.0005) [2023-12-26 17:31:34,002][105586] KL-divergence is very high: 315.7920 [2023-12-26 17:31:34,354][105692] Updated weights for policy 0, policy_version 297067 (0.0009) [2023-12-26 17:31:34,416][105692] Updated weights for policy 0, policy_version 297077 (0.0009) [2023-12-26 17:31:34,479][105692] Updated weights for policy 0, policy_version 297087 (0.0009) [2023-12-26 17:31:34,687][105620] Updated weights for policy 1, policy_version 297226 (0.0009) [2023-12-26 17:31:34,737][105620] Updated weights for policy 1, policy_version 297236 (0.0008) [2023-12-26 17:31:34,784][105620] Updated weights for policy 1, policy_version 297246 (0.0009) [2023-12-26 17:31:34,831][105620] Updated weights for policy 1, policy_version 297256 (0.0009) [2023-12-26 17:31:35,217][105692] Updated weights for policy 0, policy_version 297097 (0.0009) [2023-12-26 17:31:35,287][105692] Updated weights for policy 0, policy_version 297107 (0.0005) [2023-12-26 17:31:35,358][105585] KL-divergence is very high: 208.9076 [2023-12-26 17:31:35,359][105692] Updated weights for policy 0, policy_version 297117 (0.0006) [2023-12-26 17:31:35,405][105585] KL-divergence is very high: 175.5461 [2023-12-26 17:31:35,417][105692] Updated weights for policy 0, policy_version 297127 (0.0010) [2023-12-26 17:31:35,521][105620] Updated weights for policy 1, policy_version 297266 (0.0005) [2023-12-26 17:31:35,576][105620] Updated weights for policy 1, policy_version 297276 (0.0005) [2023-12-26 17:31:35,625][105620] Updated weights for policy 1, policy_version 297286 (0.0005) [2023-12-26 17:31:36,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19387.7, 300 sec: 19605.2). Total num frames: 152190976. Throughput: 0: 9604.7, 1: 9877.8. Samples: 152181288. Policy #0 lag: (min: 26.0, avg: 44.0, max: 58.0) [2023-12-26 17:31:36,063][104569] Avg episode reward: [(0, '9269.235'), (1, '7227.189')] [2023-12-26 17:31:36,137][105692] Updated weights for policy 0, policy_version 297137 (0.0007) [2023-12-26 17:31:36,201][105692] Updated weights for policy 0, policy_version 297147 (0.0007) [2023-12-26 17:31:36,207][105586] KL-divergence is very high: 101.3425 [2023-12-26 17:31:36,224][105586] KL-divergence is very high: 229.4637 [2023-12-26 17:31:36,231][105620] Updated weights for policy 1, policy_version 297296 (0.0005) [2023-12-26 17:31:36,235][105586] KL-divergence is very high: 373.2816 [2023-12-26 17:31:36,242][105586] KL-divergence is very high: 332.5505 [2023-12-26 17:31:36,247][105586] KL-divergence is very high: 344.3350 [2023-12-26 17:31:36,253][105586] KL-divergence is very high: 295.9516 [2023-12-26 17:31:36,266][105692] Updated weights for policy 0, policy_version 297157 (0.0008) [2023-12-26 17:31:36,270][105586] KL-divergence is very high: 515.6653 [2023-12-26 17:31:36,282][105586] KL-divergence is very high: 575.6711 [2023-12-26 17:31:36,288][105586] KL-divergence is very high: 469.5480 [2023-12-26 17:31:36,289][105620] Updated weights for policy 1, policy_version 297306 (0.0008) [2023-12-26 17:31:36,295][105586] KL-divergence is very high: 413.6118 [2023-12-26 17:31:36,302][105586] KL-divergence is very high: 344.4796 [2023-12-26 17:31:36,323][105586] KL-divergence is very high: 481.2878 [2023-12-26 17:31:36,336][105586] KL-divergence is very high: 489.7852 [2023-12-26 17:31:36,343][105586] KL-divergence is very high: 369.5437 [2023-12-26 17:31:36,351][105586] KL-divergence is very high: 317.4269 [2023-12-26 17:31:36,358][105620] Updated weights for policy 1, policy_version 297316 (0.0009) [2023-12-26 17:31:36,358][105586] KL-divergence is very high: 249.0020 [2023-12-26 17:31:36,380][105586] KL-divergence is very high: 331.6029 [2023-12-26 17:31:36,974][105692] Updated weights for policy 0, policy_version 297167 (0.0008) [2023-12-26 17:31:37,024][105692] Updated weights for policy 0, policy_version 297177 (0.0009) [2023-12-26 17:31:37,071][105692] Updated weights for policy 0, policy_version 297187 (0.0009) [2023-12-26 17:31:37,106][105620] Updated weights for policy 1, policy_version 297326 (0.0008) [2023-12-26 17:31:37,175][105620] Updated weights for policy 1, policy_version 297336 (0.0009) [2023-12-26 17:31:37,237][105620] Updated weights for policy 1, policy_version 297346 (0.0010) [2023-12-26 17:31:37,796][105692] Updated weights for policy 0, policy_version 297197 (0.0007) [2023-12-26 17:31:37,852][105692] Updated weights for policy 0, policy_version 297207 (0.0009) [2023-12-26 17:31:37,918][105692] Updated weights for policy 0, policy_version 297217 (0.0009) [2023-12-26 17:31:37,997][105620] Updated weights for policy 1, policy_version 297356 (0.0010) [2023-12-26 17:31:38,063][105620] Updated weights for policy 1, policy_version 297366 (0.0009) [2023-12-26 17:31:38,125][105620] Updated weights for policy 1, policy_version 297376 (0.0009) [2023-12-26 17:31:38,674][105692] Updated weights for policy 0, policy_version 297227 (0.0009) [2023-12-26 17:31:38,740][105692] Updated weights for policy 0, policy_version 297237 (0.0009) [2023-12-26 17:31:38,801][105692] Updated weights for policy 0, policy_version 297247 (0.0008) [2023-12-26 17:31:38,810][105620] Updated weights for policy 1, policy_version 297386 (0.0008) [2023-12-26 17:31:38,881][105620] Updated weights for policy 1, policy_version 297396 (0.0006) [2023-12-26 17:31:38,941][105620] Updated weights for policy 1, policy_version 297406 (0.0006) [2023-12-26 17:31:39,000][105620] Updated weights for policy 1, policy_version 297416 (0.0011) [2023-12-26 17:31:39,522][105692] Updated weights for policy 0, policy_version 297257 (0.0007) [2023-12-26 17:31:39,574][105692] Updated weights for policy 0, policy_version 297267 (0.0009) [2023-12-26 17:31:39,622][105692] Updated weights for policy 0, policy_version 297277 (0.0009) [2023-12-26 17:31:39,678][105692] Updated weights for policy 0, policy_version 297287 (0.0008) [2023-12-26 17:31:39,722][105620] Updated weights for policy 1, policy_version 297426 (0.0010) [2023-12-26 17:31:39,785][105620] Updated weights for policy 1, policy_version 297436 (0.0009) [2023-12-26 17:31:39,854][105620] Updated weights for policy 1, policy_version 297446 (0.0009) [2023-12-26 17:31:40,348][105692] Updated weights for policy 0, policy_version 297297 (0.0010) [2023-12-26 17:31:40,407][105692] Updated weights for policy 0, policy_version 297307 (0.0009) [2023-12-26 17:31:40,474][105692] Updated weights for policy 0, policy_version 297317 (0.0007) [2023-12-26 17:31:40,694][105620] Updated weights for policy 1, policy_version 297456 (0.0008) [2023-12-26 17:31:40,743][105620] Updated weights for policy 1, policy_version 297466 (0.0007) [2023-12-26 17:31:40,799][105620] Updated weights for policy 1, policy_version 297477 (0.0009) [2023-12-26 17:31:41,046][105692] Updated weights for policy 0, policy_version 297327 (0.0008) [2023-12-26 17:31:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 152289280. Throughput: 0: 9663.4, 1: 9851.4. Samples: 152296324. Policy #0 lag: (min: 26.0, avg: 44.0, max: 58.0) [2023-12-26 17:31:41,062][104569] Avg episode reward: [(0, '9269.126'), (1, '7502.895')] [2023-12-26 17:31:41,115][105692] Updated weights for policy 0, policy_version 297337 (0.0010) [2023-12-26 17:31:41,182][105692] Updated weights for policy 0, policy_version 297347 (0.0008) [2023-12-26 17:31:41,654][105620] Updated weights for policy 1, policy_version 297487 (0.0010) [2023-12-26 17:31:41,728][105620] Updated weights for policy 1, policy_version 297497 (0.0010) [2023-12-26 17:31:41,789][105620] Updated weights for policy 1, policy_version 297507 (0.0009) [2023-12-26 17:31:41,933][105692] Updated weights for policy 0, policy_version 297357 (0.0009) [2023-12-26 17:31:41,992][105692] Updated weights for policy 0, policy_version 297367 (0.0009) [2023-12-26 17:31:42,051][105692] Updated weights for policy 0, policy_version 297377 (0.0009) [2023-12-26 17:31:42,575][105620] Updated weights for policy 1, policy_version 297517 (0.0009) [2023-12-26 17:31:42,612][105586] KL-divergence is very high: 104.4690 [2023-12-26 17:31:42,630][105620] Updated weights for policy 1, policy_version 297527 (0.0009) [2023-12-26 17:31:42,658][105586] KL-divergence is very high: 220.5012 [2023-12-26 17:31:42,688][105620] Updated weights for policy 1, policy_version 297537 (0.0009) [2023-12-26 17:31:42,709][105586] KL-divergence is very high: 224.5992 [2023-12-26 17:31:42,835][105692] Updated weights for policy 0, policy_version 297387 (0.0009) [2023-12-26 17:31:42,891][105692] Updated weights for policy 0, policy_version 297397 (0.0008) [2023-12-26 17:31:42,946][105692] Updated weights for policy 0, policy_version 297407 (0.0008) [2023-12-26 17:31:43,459][105620] Updated weights for policy 1, policy_version 297547 (0.0010) [2023-12-26 17:31:43,507][105620] Updated weights for policy 1, policy_version 297557 (0.0010) [2023-12-26 17:31:43,560][105620] Updated weights for policy 1, policy_version 297567 (0.0010) [2023-12-26 17:31:43,734][105692] Updated weights for policy 0, policy_version 297417 (0.0009) [2023-12-26 17:31:43,792][105692] Updated weights for policy 0, policy_version 297427 (0.0010) [2023-12-26 17:31:43,847][105692] Updated weights for policy 0, policy_version 297437 (0.0010) [2023-12-26 17:31:43,901][105692] Updated weights for policy 0, policy_version 297447 (0.0010) [2023-12-26 17:31:44,268][105620] Updated weights for policy 1, policy_version 297577 (0.0010) [2023-12-26 17:31:44,313][105620] Updated weights for policy 1, policy_version 297587 (0.0008) [2023-12-26 17:31:44,360][105620] Updated weights for policy 1, policy_version 297597 (0.0007) [2023-12-26 17:31:44,365][105586] KL-divergence is very high: 138.9162 [2023-12-26 17:31:44,404][105586] KL-divergence is very high: 155.7729 [2023-12-26 17:31:44,408][105620] Updated weights for policy 1, policy_version 297607 (0.0009) [2023-12-26 17:31:44,618][105692] Updated weights for policy 0, policy_version 297457 (0.0008) [2023-12-26 17:31:44,683][105692] Updated weights for policy 0, policy_version 297467 (0.0007) [2023-12-26 17:31:44,746][105692] Updated weights for policy 0, policy_version 297477 (0.0007) [2023-12-26 17:31:45,109][105620] Updated weights for policy 1, policy_version 297617 (0.0011) [2023-12-26 17:31:45,173][105620] Updated weights for policy 1, policy_version 297627 (0.0011) [2023-12-26 17:31:45,236][105620] Updated weights for policy 1, policy_version 297637 (0.0011) [2023-12-26 17:31:45,454][105692] Updated weights for policy 0, policy_version 297487 (0.0006) [2023-12-26 17:31:45,521][105692] Updated weights for policy 0, policy_version 297497 (0.0008) [2023-12-26 17:31:45,588][105692] Updated weights for policy 0, policy_version 297507 (0.0006) [2023-12-26 17:31:45,978][105620] Updated weights for policy 1, policy_version 297647 (0.0007) [2023-12-26 17:31:46,022][105620] Updated weights for policy 1, policy_version 297657 (0.0005) [2023-12-26 17:31:46,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 152379392. Throughput: 0: 9702.9, 1: 9836.3. Samples: 152353048. Policy #0 lag: (min: 26.0, avg: 44.0, max: 58.0) [2023-12-26 17:31:46,063][104569] Avg episode reward: [(0, '9269.117'), (1, '7411.502')] [2023-12-26 17:31:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000297512_76177408.pth... [2023-12-26 17:31:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000296360_75882496.pth [2023-12-26 17:31:46,080][105620] Updated weights for policy 1, policy_version 297667 (0.0006) [2023-12-26 17:31:46,109][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000297672_76210176.pth... [2023-12-26 17:31:46,114][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000296520_75915264.pth [2023-12-26 17:31:46,283][105692] Updated weights for policy 0, policy_version 297517 (0.0006) [2023-12-26 17:31:46,351][105692] Updated weights for policy 0, policy_version 297527 (0.0005) [2023-12-26 17:31:46,413][105692] Updated weights for policy 0, policy_version 297537 (0.0009) [2023-12-26 17:31:46,725][105620] Updated weights for policy 1, policy_version 297677 (0.0009) [2023-12-26 17:31:46,778][105620] Updated weights for policy 1, policy_version 297687 (0.0009) [2023-12-26 17:31:46,844][105620] Updated weights for policy 1, policy_version 297697 (0.0007) [2023-12-26 17:31:47,053][105692] Updated weights for policy 0, policy_version 297547 (0.0007) [2023-12-26 17:31:47,110][105692] Updated weights for policy 0, policy_version 297557 (0.0010) [2023-12-26 17:31:47,167][105692] Updated weights for policy 0, policy_version 297568 (0.0009) [2023-12-26 17:31:47,497][105620] Updated weights for policy 1, policy_version 297707 (0.0009) [2023-12-26 17:31:47,562][105620] Updated weights for policy 1, policy_version 297717 (0.0008) [2023-12-26 17:31:47,625][105620] Updated weights for policy 1, policy_version 297727 (0.0005) [2023-12-26 17:31:47,850][105692] Updated weights for policy 0, policy_version 297578 (0.0009) [2023-12-26 17:31:47,909][105692] Updated weights for policy 0, policy_version 297588 (0.0008) [2023-12-26 17:31:47,971][105692] Updated weights for policy 0, policy_version 297599 (0.0011) [2023-12-26 17:31:48,218][105620] Updated weights for policy 1, policy_version 297737 (0.0005) [2023-12-26 17:31:48,270][105620] Updated weights for policy 1, policy_version 297747 (0.0005) [2023-12-26 17:31:48,276][105586] KL-divergence is very high: 127.9673 [2023-12-26 17:31:48,327][105586] KL-divergence is very high: 159.0185 [2023-12-26 17:31:48,334][105620] Updated weights for policy 1, policy_version 297757 (0.0006) [2023-12-26 17:31:48,381][105586] KL-divergence is very high: 133.9021 [2023-12-26 17:31:48,401][105620] Updated weights for policy 1, policy_version 297767 (0.0009) [2023-12-26 17:31:48,702][105692] Updated weights for policy 0, policy_version 297610 (0.0009) [2023-12-26 17:31:48,750][105692] Updated weights for policy 0, policy_version 297620 (0.0005) [2023-12-26 17:31:48,805][105692] Updated weights for policy 0, policy_version 297630 (0.0005) [2023-12-26 17:31:48,868][105692] Updated weights for policy 0, policy_version 297640 (0.0005) [2023-12-26 17:31:49,181][105620] Updated weights for policy 1, policy_version 297777 (0.0009) [2023-12-26 17:31:49,249][105620] Updated weights for policy 1, policy_version 297787 (0.0008) [2023-12-26 17:31:49,319][105620] Updated weights for policy 1, policy_version 297797 (0.0009) [2023-12-26 17:31:49,539][105692] Updated weights for policy 0, policy_version 297650 (0.0009) [2023-12-26 17:31:49,607][105692] Updated weights for policy 0, policy_version 297660 (0.0009) [2023-12-26 17:31:49,672][105692] Updated weights for policy 0, policy_version 297670 (0.0009) [2023-12-26 17:31:49,975][105620] Updated weights for policy 1, policy_version 297807 (0.0009) [2023-12-26 17:31:50,032][105620] Updated weights for policy 1, policy_version 297817 (0.0011) [2023-12-26 17:31:50,096][105620] Updated weights for policy 1, policy_version 297827 (0.0011) [2023-12-26 17:31:50,445][105692] Updated weights for policy 0, policy_version 297680 (0.0007) [2023-12-26 17:31:50,500][105692] Updated weights for policy 0, policy_version 297690 (0.0005) [2023-12-26 17:31:50,557][105692] Updated weights for policy 0, policy_version 297700 (0.0006) [2023-12-26 17:31:50,846][105620] Updated weights for policy 1, policy_version 297837 (0.0011) [2023-12-26 17:31:50,906][105620] Updated weights for policy 1, policy_version 297847 (0.0011) [2023-12-26 17:31:50,969][105620] Updated weights for policy 1, policy_version 297857 (0.0011) [2023-12-26 17:31:51,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 152485888. Throughput: 0: 9774.9, 1: 9896.7. Samples: 152473316. Policy #0 lag: (min: 26.0, avg: 44.0, max: 58.0) [2023-12-26 17:31:51,063][104569] Avg episode reward: [(0, '9359.340'), (1, '6308.078')] [2023-12-26 17:31:51,265][105692] Updated weights for policy 0, policy_version 297710 (0.0009) [2023-12-26 17:31:51,316][105692] Updated weights for policy 0, policy_version 297720 (0.0009) [2023-12-26 17:31:51,383][105692] Updated weights for policy 0, policy_version 297730 (0.0007) [2023-12-26 17:31:51,688][105620] Updated weights for policy 1, policy_version 297867 (0.0010) [2023-12-26 17:31:51,756][105620] Updated weights for policy 1, policy_version 297877 (0.0008) [2023-12-26 17:31:51,816][105620] Updated weights for policy 1, policy_version 297887 (0.0008) [2023-12-26 17:31:52,149][105692] Updated weights for policy 0, policy_version 297740 (0.0009) [2023-12-26 17:31:52,210][105692] Updated weights for policy 0, policy_version 297750 (0.0008) [2023-12-26 17:31:52,268][105692] Updated weights for policy 0, policy_version 297760 (0.0008) [2023-12-26 17:31:52,562][105620] Updated weights for policy 1, policy_version 297897 (0.0008) [2023-12-26 17:31:52,618][105620] Updated weights for policy 1, policy_version 297907 (0.0009) [2023-12-26 17:31:52,669][105620] Updated weights for policy 1, policy_version 297917 (0.0009) [2023-12-26 17:31:52,721][105620] Updated weights for policy 1, policy_version 297927 (0.0009) [2023-12-26 17:31:53,070][105692] Updated weights for policy 0, policy_version 297770 (0.0009) [2023-12-26 17:31:53,124][105692] Updated weights for policy 0, policy_version 297780 (0.0010) [2023-12-26 17:31:53,174][105692] Updated weights for policy 0, policy_version 297790 (0.0009) [2023-12-26 17:31:53,230][105692] Updated weights for policy 0, policy_version 297800 (0.0010) [2023-12-26 17:31:53,350][105620] Updated weights for policy 1, policy_version 297937 (0.0009) [2023-12-26 17:31:53,405][105620] Updated weights for policy 1, policy_version 297947 (0.0009) [2023-12-26 17:31:53,455][105620] Updated weights for policy 1, policy_version 297957 (0.0009) [2023-12-26 17:31:53,989][105692] Updated weights for policy 0, policy_version 297810 (0.0009) [2023-12-26 17:31:54,035][105692] Updated weights for policy 0, policy_version 297820 (0.0009) [2023-12-26 17:31:54,082][105692] Updated weights for policy 0, policy_version 297830 (0.0008) [2023-12-26 17:31:54,218][105620] Updated weights for policy 1, policy_version 297967 (0.0008) [2023-12-26 17:31:54,252][105586] KL-divergence is very high: 275.1781 [2023-12-26 17:31:54,258][105586] KL-divergence is very high: 164.6497 [2023-12-26 17:31:54,271][105586] KL-divergence is very high: 195.5504 [2023-12-26 17:31:54,273][105620] Updated weights for policy 1, policy_version 297977 (0.0007) [2023-12-26 17:31:54,290][105586] KL-divergence is very high: 340.6601 [2023-12-26 17:31:54,295][105586] KL-divergence is very high: 196.8970 [2023-12-26 17:31:54,313][105586] KL-divergence is very high: 172.9743 [2023-12-26 17:31:54,323][105620] Updated weights for policy 1, policy_version 297987 (0.0006) [2023-12-26 17:31:54,336][105586] KL-divergence is very high: 249.3410 [2023-12-26 17:31:54,342][105586] KL-divergence is very high: 128.3627 [2023-12-26 17:31:54,871][105620] Updated weights for policy 1, policy_version 297997 (0.0006) [2023-12-26 17:31:54,922][105620] Updated weights for policy 1, policy_version 298007 (0.0005) [2023-12-26 17:31:54,985][105620] Updated weights for policy 1, policy_version 298017 (0.0006) [2023-12-26 17:31:54,985][105692] Updated weights for policy 0, policy_version 297840 (0.0009) [2023-12-26 17:31:55,047][105692] Updated weights for policy 0, policy_version 297850 (0.0009) [2023-12-26 17:31:55,101][105692] Updated weights for policy 0, policy_version 297860 (0.0010) [2023-12-26 17:31:55,611][105620] Updated weights for policy 1, policy_version 298027 (0.0006) [2023-12-26 17:31:55,670][105620] Updated weights for policy 1, policy_version 298037 (0.0010) [2023-12-26 17:31:55,728][105620] Updated weights for policy 1, policy_version 298047 (0.0009) [2023-12-26 17:31:55,877][105692] Updated weights for policy 0, policy_version 297871 (0.0009) [2023-12-26 17:31:55,933][105692] Updated weights for policy 0, policy_version 297881 (0.0008) [2023-12-26 17:31:55,987][105692] Updated weights for policy 0, policy_version 297891 (0.0009) [2023-12-26 17:31:56,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19524.2, 300 sec: 19605.2). Total num frames: 152584192. Throughput: 0: 9750.2, 1: 9930.3. Samples: 152586800. Policy #0 lag: (min: 26.0, avg: 44.0, max: 58.0) [2023-12-26 17:31:56,063][104569] Avg episode reward: [(0, '9358.792'), (1, '6388.676')] [2023-12-26 17:31:56,463][105620] Updated weights for policy 1, policy_version 298057 (0.0009) [2023-12-26 17:31:56,531][105620] Updated weights for policy 1, policy_version 298067 (0.0006) [2023-12-26 17:31:56,597][105620] Updated weights for policy 1, policy_version 298077 (0.0006) [2023-12-26 17:31:56,652][105620] Updated weights for policy 1, policy_version 298087 (0.0005) [2023-12-26 17:31:56,675][105692] Updated weights for policy 0, policy_version 297901 (0.0007) [2023-12-26 17:31:56,727][105692] Updated weights for policy 0, policy_version 297911 (0.0009) [2023-12-26 17:31:56,778][105692] Updated weights for policy 0, policy_version 297921 (0.0010) [2023-12-26 17:31:57,237][105620] Updated weights for policy 1, policy_version 298097 (0.0010) [2023-12-26 17:31:57,281][105620] Updated weights for policy 1, policy_version 298107 (0.0010) [2023-12-26 17:31:57,332][105620] Updated weights for policy 1, policy_version 298117 (0.0010) [2023-12-26 17:31:57,406][105692] Updated weights for policy 0, policy_version 297931 (0.0010) [2023-12-26 17:31:57,456][105692] Updated weights for policy 0, policy_version 297941 (0.0008) [2023-12-26 17:31:57,513][105692] Updated weights for policy 0, policy_version 297951 (0.0009) [2023-12-26 17:31:58,035][105620] Updated weights for policy 1, policy_version 298127 (0.0008) [2023-12-26 17:31:58,085][105620] Updated weights for policy 1, policy_version 298137 (0.0005) [2023-12-26 17:31:58,152][105620] Updated weights for policy 1, policy_version 298147 (0.0008) [2023-12-26 17:31:58,253][105692] Updated weights for policy 0, policy_version 297961 (0.0009) [2023-12-26 17:31:58,311][105692] Updated weights for policy 0, policy_version 297971 (0.0007) [2023-12-26 17:31:58,387][105692] Updated weights for policy 0, policy_version 297981 (0.0009) [2023-12-26 17:31:58,455][105692] Updated weights for policy 0, policy_version 297991 (0.0008) [2023-12-26 17:31:58,867][105620] Updated weights for policy 1, policy_version 298157 (0.0009) [2023-12-26 17:31:58,930][105620] Updated weights for policy 1, policy_version 298167 (0.0010) [2023-12-26 17:31:58,998][105620] Updated weights for policy 1, policy_version 298177 (0.0010) [2023-12-26 17:31:59,162][105692] Updated weights for policy 0, policy_version 298001 (0.0006) [2023-12-26 17:31:59,227][105692] Updated weights for policy 0, policy_version 298011 (0.0006) [2023-12-26 17:31:59,298][105692] Updated weights for policy 0, policy_version 298021 (0.0008) [2023-12-26 17:31:59,663][105620] Updated weights for policy 1, policy_version 298187 (0.0010) [2023-12-26 17:31:59,721][105620] Updated weights for policy 1, policy_version 298197 (0.0010) [2023-12-26 17:31:59,778][105620] Updated weights for policy 1, policy_version 298207 (0.0006) [2023-12-26 17:32:00,008][105692] Updated weights for policy 0, policy_version 298031 (0.0008) [2023-12-26 17:32:00,064][105692] Updated weights for policy 0, policy_version 298041 (0.0008) [2023-12-26 17:32:00,128][105692] Updated weights for policy 0, policy_version 298051 (0.0009) [2023-12-26 17:32:00,368][105620] Updated weights for policy 1, policy_version 298217 (0.0005) [2023-12-26 17:32:00,422][105620] Updated weights for policy 1, policy_version 298227 (0.0005) [2023-12-26 17:32:00,472][105620] Updated weights for policy 1, policy_version 298237 (0.0005) [2023-12-26 17:32:00,530][105620] Updated weights for policy 1, policy_version 298247 (0.0006) [2023-12-26 17:32:00,954][105692] Updated weights for policy 0, policy_version 298061 (0.0009) [2023-12-26 17:32:01,005][105692] Updated weights for policy 0, policy_version 298071 (0.0008) [2023-12-26 17:32:01,060][105692] Updated weights for policy 0, policy_version 298081 (0.0009) [2023-12-26 17:32:01,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 152674304. Throughput: 0: 9813.3, 1: 9926.1. Samples: 152647804. Policy #0 lag: (min: 26.0, avg: 44.0, max: 58.0) [2023-12-26 17:32:01,062][104569] Avg episode reward: [(0, '9358.532'), (1, '6390.566')] [2023-12-26 17:32:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000298248_76357632.pth... [2023-12-26 17:32:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000297096_76062720.pth [2023-12-26 17:32:01,102][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000298088_76324864.pth... [2023-12-26 17:32:01,106][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000296968_76038144.pth [2023-12-26 17:32:01,224][105586] KL-divergence is very high: 117.7360 [2023-12-26 17:32:01,227][105620] Updated weights for policy 1, policy_version 298257 (0.0010) [2023-12-26 17:32:01,270][105586] KL-divergence is very high: 181.0718 [2023-12-26 17:32:01,282][105620] Updated weights for policy 1, policy_version 298267 (0.0010) [2023-12-26 17:32:01,315][105586] KL-divergence is very high: 175.9500 [2023-12-26 17:32:01,336][105620] Updated weights for policy 1, policy_version 298277 (0.0010) [2023-12-26 17:32:01,837][105692] Updated weights for policy 0, policy_version 298091 (0.0008) [2023-12-26 17:32:01,891][105692] Updated weights for policy 0, policy_version 298101 (0.0010) [2023-12-26 17:32:01,952][105692] Updated weights for policy 0, policy_version 298111 (0.0010) [2023-12-26 17:32:02,116][105620] Updated weights for policy 1, policy_version 298287 (0.0008) [2023-12-26 17:32:02,166][105620] Updated weights for policy 1, policy_version 298297 (0.0009) [2023-12-26 17:32:02,217][105620] Updated weights for policy 1, policy_version 298307 (0.0009) [2023-12-26 17:32:02,688][105692] Updated weights for policy 0, policy_version 298121 (0.0010) [2023-12-26 17:32:02,761][105692] Updated weights for policy 0, policy_version 298131 (0.0010) [2023-12-26 17:32:02,830][105692] Updated weights for policy 0, policy_version 298141 (0.0007) [2023-12-26 17:32:02,894][105620] Updated weights for policy 1, policy_version 298317 (0.0008) [2023-12-26 17:32:02,895][105692] Updated weights for policy 0, policy_version 298151 (0.0005) [2023-12-26 17:32:02,953][105620] Updated weights for policy 1, policy_version 298327 (0.0009) [2023-12-26 17:32:03,014][105620] Updated weights for policy 1, policy_version 298337 (0.0009) [2023-12-26 17:32:03,459][105692] Updated weights for policy 0, policy_version 298161 (0.0006) [2023-12-26 17:32:03,510][105692] Updated weights for policy 0, policy_version 298171 (0.0006) [2023-12-26 17:32:03,566][105692] Updated weights for policy 0, policy_version 298181 (0.0005) [2023-12-26 17:32:03,681][105620] Updated weights for policy 1, policy_version 298347 (0.0009) [2023-12-26 17:32:03,747][105620] Updated weights for policy 1, policy_version 298357 (0.0007) [2023-12-26 17:32:03,805][105620] Updated weights for policy 1, policy_version 298367 (0.0008) [2023-12-26 17:32:04,170][105692] Updated weights for policy 0, policy_version 298191 (0.0008) [2023-12-26 17:32:04,233][105692] Updated weights for policy 0, policy_version 298201 (0.0010) [2023-12-26 17:32:04,300][105692] Updated weights for policy 0, policy_version 298211 (0.0008) [2023-12-26 17:32:04,453][105620] Updated weights for policy 1, policy_version 298377 (0.0010) [2023-12-26 17:32:04,511][105620] Updated weights for policy 1, policy_version 298387 (0.0010) [2023-12-26 17:32:04,568][105620] Updated weights for policy 1, policy_version 298397 (0.0009) [2023-12-26 17:32:04,635][105620] Updated weights for policy 1, policy_version 298407 (0.0007) [2023-12-26 17:32:05,151][105692] Updated weights for policy 0, policy_version 298221 (0.0009) [2023-12-26 17:32:05,201][105692] Updated weights for policy 0, policy_version 298231 (0.0006) [2023-12-26 17:32:05,210][105620] Updated weights for policy 1, policy_version 298417 (0.0010) [2023-12-26 17:32:05,248][105692] Updated weights for policy 0, policy_version 298241 (0.0005) [2023-12-26 17:32:05,259][105620] Updated weights for policy 1, policy_version 298427 (0.0009) [2023-12-26 17:32:05,318][105620] Updated weights for policy 1, policy_version 298437 (0.0010) [2023-12-26 17:32:06,006][105620] Updated weights for policy 1, policy_version 298447 (0.0007) [2023-12-26 17:32:06,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 152772608. Throughput: 0: 9730.8, 1: 9974.4. Samples: 152768348. Policy #0 lag: (min: 26.0, avg: 44.0, max: 58.0) [2023-12-26 17:32:06,062][104569] Avg episode reward: [(0, '9268.178'), (1, '7036.339')] [2023-12-26 17:32:06,071][105620] Updated weights for policy 1, policy_version 298457 (0.0005) [2023-12-26 17:32:06,072][105692] Updated weights for policy 0, policy_version 298251 (0.0008) [2023-12-26 17:32:06,131][105692] Updated weights for policy 0, policy_version 298261 (0.0008) [2023-12-26 17:32:06,144][105620] Updated weights for policy 1, policy_version 298467 (0.0008) [2023-12-26 17:32:06,192][105692] Updated weights for policy 0, policy_version 298271 (0.0007) [2023-12-26 17:32:06,657][105620] Updated weights for policy 1, policy_version 298477 (0.0006) [2023-12-26 17:32:06,720][105620] Updated weights for policy 1, policy_version 298487 (0.0005) [2023-12-26 17:32:06,759][105586] KL-divergence is very high: 139.9835 [2023-12-26 17:32:06,766][105586] KL-divergence is very high: 144.0864 [2023-12-26 17:32:06,784][105620] Updated weights for policy 1, policy_version 298497 (0.0008) [2023-12-26 17:32:06,812][105586] KL-divergence is very high: 128.2407 [2023-12-26 17:32:06,819][105586] KL-divergence is very high: 124.5712 [2023-12-26 17:32:07,046][105692] Updated weights for policy 0, policy_version 298281 (0.0009) [2023-12-26 17:32:07,105][105692] Updated weights for policy 0, policy_version 298291 (0.0008) [2023-12-26 17:32:07,168][105692] Updated weights for policy 0, policy_version 298301 (0.0008) [2023-12-26 17:32:07,235][105692] Updated weights for policy 0, policy_version 298311 (0.0008) [2023-12-26 17:32:07,423][105620] Updated weights for policy 1, policy_version 298507 (0.0009) [2023-12-26 17:32:07,481][105620] Updated weights for policy 1, policy_version 298517 (0.0005) [2023-12-26 17:32:07,516][105586] KL-divergence is very high: 123.1498 [2023-12-26 17:32:07,524][105620] Updated weights for policy 1, policy_version 298527 (0.0005) [2023-12-26 17:32:07,529][105586] KL-divergence is very high: 153.3769 [2023-12-26 17:32:07,539][105586] KL-divergence is very high: 220.3464 [2023-12-26 17:32:07,557][105586] KL-divergence is very high: 189.6215 [2023-12-26 17:32:07,881][105692] Updated weights for policy 0, policy_version 298321 (0.0009) [2023-12-26 17:32:07,946][105692] Updated weights for policy 0, policy_version 298331 (0.0010) [2023-12-26 17:32:08,010][105692] Updated weights for policy 0, policy_version 298341 (0.0010) [2023-12-26 17:32:08,194][105586] KL-divergence is very high: 179.9217 [2023-12-26 17:32:08,200][105620] Updated weights for policy 1, policy_version 298537 (0.0006) [2023-12-26 17:32:08,201][105586] KL-divergence is very high: 171.0659 [2023-12-26 17:32:08,218][105586] KL-divergence is very high: 120.6451 [2023-12-26 17:32:08,245][105586] KL-divergence is very high: 134.4852 [2023-12-26 17:32:08,251][105586] KL-divergence is very high: 129.4068 [2023-12-26 17:32:08,261][105620] Updated weights for policy 1, policy_version 298547 (0.0010) [2023-12-26 17:32:08,320][105620] Updated weights for policy 1, policy_version 298557 (0.0010) [2023-12-26 17:32:08,386][105620] Updated weights for policy 1, policy_version 298567 (0.0009) [2023-12-26 17:32:08,695][105692] Updated weights for policy 0, policy_version 298351 (0.0010) [2023-12-26 17:32:08,751][105692] Updated weights for policy 0, policy_version 298361 (0.0009) [2023-12-26 17:32:08,803][105692] Updated weights for policy 0, policy_version 298371 (0.0009) [2023-12-26 17:32:09,037][105620] Updated weights for policy 1, policy_version 298577 (0.0009) [2023-12-26 17:32:09,090][105620] Updated weights for policy 1, policy_version 298587 (0.0008) [2023-12-26 17:32:09,151][105620] Updated weights for policy 1, policy_version 298597 (0.0009) [2023-12-26 17:32:09,602][105692] Updated weights for policy 0, policy_version 298381 (0.0008) [2023-12-26 17:32:09,665][105692] Updated weights for policy 0, policy_version 298391 (0.0006) [2023-12-26 17:32:09,741][105692] Updated weights for policy 0, policy_version 298401 (0.0006) [2023-12-26 17:32:09,897][105620] Updated weights for policy 1, policy_version 298607 (0.0009) [2023-12-26 17:32:09,960][105620] Updated weights for policy 1, policy_version 298617 (0.0008) [2023-12-26 17:32:10,018][105620] Updated weights for policy 1, policy_version 298627 (0.0008) [2023-12-26 17:32:10,527][105692] Updated weights for policy 0, policy_version 298411 (0.0010) [2023-12-26 17:32:10,589][105692] Updated weights for policy 0, policy_version 298421 (0.0010) [2023-12-26 17:32:10,645][105692] Updated weights for policy 0, policy_version 298432 (0.0010) [2023-12-26 17:32:10,749][105620] Updated weights for policy 1, policy_version 298637 (0.0008) [2023-12-26 17:32:10,802][105620] Updated weights for policy 1, policy_version 298647 (0.0008) [2023-12-26 17:32:10,859][105620] Updated weights for policy 1, policy_version 298657 (0.0009) [2023-12-26 17:32:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 152879104. Throughput: 0: 9621.1, 1: 9982.1. Samples: 152884328. Policy #0 lag: (min: 26.0, avg: 44.0, max: 58.0) [2023-12-26 17:32:11,062][104569] Avg episode reward: [(0, '9267.657'), (1, '6683.893')] [2023-12-26 17:32:11,432][105692] Updated weights for policy 0, policy_version 298442 (0.0009) [2023-12-26 17:32:11,498][105692] Updated weights for policy 0, policy_version 298452 (0.0009) [2023-12-26 17:32:11,559][105692] Updated weights for policy 0, policy_version 298462 (0.0006) [2023-12-26 17:32:11,623][105692] Updated weights for policy 0, policy_version 298472 (0.0006) [2023-12-26 17:32:11,714][105620] Updated weights for policy 1, policy_version 298667 (0.0009) [2023-12-26 17:32:11,772][105620] Updated weights for policy 1, policy_version 298677 (0.0009) [2023-12-26 17:32:11,829][105620] Updated weights for policy 1, policy_version 298687 (0.0009) [2023-12-26 17:32:12,255][105692] Updated weights for policy 0, policy_version 298482 (0.0009) [2023-12-26 17:32:12,313][105692] Updated weights for policy 0, policy_version 298492 (0.0009) [2023-12-26 17:32:12,381][105692] Updated weights for policy 0, policy_version 298502 (0.0009) [2023-12-26 17:32:12,537][105620] Updated weights for policy 1, policy_version 298697 (0.0009) [2023-12-26 17:32:12,588][105620] Updated weights for policy 1, policy_version 298707 (0.0007) [2023-12-26 17:32:12,613][105586] KL-divergence is very high: 106.0865 [2023-12-26 17:32:12,623][105586] KL-divergence is very high: 150.0911 [2023-12-26 17:32:12,629][105586] KL-divergence is very high: 113.6693 [2023-12-26 17:32:12,641][105620] Updated weights for policy 1, policy_version 298717 (0.0008) [2023-12-26 17:32:12,656][105586] KL-divergence is very high: 111.3710 [2023-12-26 17:32:12,666][105586] KL-divergence is very high: 149.5824 [2023-12-26 17:32:12,672][105586] KL-divergence is very high: 111.2342 [2023-12-26 17:32:12,695][105620] Updated weights for policy 1, policy_version 298727 (0.0006) [2023-12-26 17:32:13,165][105692] Updated weights for policy 0, policy_version 298512 (0.0009) [2023-12-26 17:32:13,223][105692] Updated weights for policy 0, policy_version 298522 (0.0009) [2023-12-26 17:32:13,280][105692] Updated weights for policy 0, policy_version 298532 (0.0009) [2023-12-26 17:32:13,331][105620] Updated weights for policy 1, policy_version 298737 (0.0008) [2023-12-26 17:32:13,379][105620] Updated weights for policy 1, policy_version 298747 (0.0009) [2023-12-26 17:32:13,434][105620] Updated weights for policy 1, policy_version 298757 (0.0009) [2023-12-26 17:32:14,073][105692] Updated weights for policy 0, policy_version 298542 (0.0008) [2023-12-26 17:32:14,121][105692] Updated weights for policy 0, policy_version 298552 (0.0008) [2023-12-26 17:32:14,131][105620] Updated weights for policy 1, policy_version 298767 (0.0008) [2023-12-26 17:32:14,174][105692] Updated weights for policy 0, policy_version 298562 (0.0006) [2023-12-26 17:32:14,179][105620] Updated weights for policy 1, policy_version 298777 (0.0007) [2023-12-26 17:32:14,234][105620] Updated weights for policy 1, policy_version 298787 (0.0007) [2023-12-26 17:32:14,873][105620] Updated weights for policy 1, policy_version 298797 (0.0009) [2023-12-26 17:32:14,890][105586] KL-divergence is very high: 109.0760 [2023-12-26 17:32:14,900][105586] KL-divergence is very high: 106.7214 [2023-12-26 17:32:14,912][105586] KL-divergence is very high: 213.8395 [2023-12-26 17:32:14,924][105620] Updated weights for policy 1, policy_version 298807 (0.0008) [2023-12-26 17:32:14,984][105620] Updated weights for policy 1, policy_version 298817 (0.0008) [2023-12-26 17:32:14,986][105692] Updated weights for policy 0, policy_version 298572 (0.0007) [2023-12-26 17:32:15,048][105692] Updated weights for policy 0, policy_version 298582 (0.0007) [2023-12-26 17:32:15,110][105692] Updated weights for policy 0, policy_version 298592 (0.0009) [2023-12-26 17:32:15,731][105620] Updated weights for policy 1, policy_version 298827 (0.0008) [2023-12-26 17:32:15,784][105620] Updated weights for policy 1, policy_version 298837 (0.0011) [2023-12-26 17:32:15,799][105586] KL-divergence is very high: 102.1764 [2023-12-26 17:32:15,853][105620] Updated weights for policy 1, policy_version 298847 (0.0009) [2023-12-26 17:32:15,879][105692] Updated weights for policy 0, policy_version 298602 (0.0008) [2023-12-26 17:32:15,935][105692] Updated weights for policy 0, policy_version 298612 (0.0007) [2023-12-26 17:32:15,986][105692] Updated weights for policy 0, policy_version 298622 (0.0006) [2023-12-26 17:32:16,038][105692] Updated weights for policy 0, policy_version 298632 (0.0006) [2023-12-26 17:32:16,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 152977408. Throughput: 0: 9549.7, 1: 9935.3. Samples: 152942588. Policy #0 lag: (min: 26.0, avg: 44.0, max: 58.0) [2023-12-26 17:32:16,062][104569] Avg episode reward: [(0, '9266.736'), (1, '5998.404')] [2023-12-26 17:32:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000298632_76464128.pth... [2023-12-26 17:32:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000298856_76513280.pth... [2023-12-26 17:32:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000297672_76210176.pth [2023-12-26 17:32:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000297512_76177408.pth [2023-12-26 17:32:16,463][105620] Updated weights for policy 1, policy_version 298857 (0.0010) [2023-12-26 17:32:16,512][105620] Updated weights for policy 1, policy_version 298867 (0.0010) [2023-12-26 17:32:16,536][105586] KL-divergence is very high: 144.1649 [2023-12-26 17:32:16,561][105620] Updated weights for policy 1, policy_version 298877 (0.0010) [2023-12-26 17:32:16,581][105586] KL-divergence is very high: 174.9171 [2023-12-26 17:32:16,588][105586] KL-divergence is very high: 120.8771 [2023-12-26 17:32:16,621][105620] Updated weights for policy 1, policy_version 298887 (0.0011) [2023-12-26 17:32:16,643][105692] Updated weights for policy 0, policy_version 298642 (0.0005) [2023-12-26 17:32:16,698][105692] Updated weights for policy 0, policy_version 298652 (0.0005) [2023-12-26 17:32:16,753][105692] Updated weights for policy 0, policy_version 298662 (0.0007) [2023-12-26 17:32:17,377][105620] Updated weights for policy 1, policy_version 298897 (0.0006) [2023-12-26 17:32:17,439][105620] Updated weights for policy 1, policy_version 298907 (0.0007) [2023-12-26 17:32:17,495][105620] Updated weights for policy 1, policy_version 298917 (0.0006) [2023-12-26 17:32:17,495][105692] Updated weights for policy 0, policy_version 298672 (0.0009) [2023-12-26 17:32:17,557][105692] Updated weights for policy 0, policy_version 298682 (0.0009) [2023-12-26 17:32:17,623][105692] Updated weights for policy 0, policy_version 298692 (0.0010) [2023-12-26 17:32:18,149][105620] Updated weights for policy 1, policy_version 298927 (0.0005) [2023-12-26 17:32:18,214][105620] Updated weights for policy 1, policy_version 298937 (0.0005) [2023-12-26 17:32:18,267][105620] Updated weights for policy 1, policy_version 298947 (0.0007) [2023-12-26 17:32:18,412][105692] Updated weights for policy 0, policy_version 298702 (0.0007) [2023-12-26 17:32:18,468][105692] Updated weights for policy 0, policy_version 298712 (0.0005) [2023-12-26 17:32:18,523][105692] Updated weights for policy 0, policy_version 298722 (0.0009) [2023-12-26 17:32:18,994][105620] Updated weights for policy 1, policy_version 298957 (0.0009) [2023-12-26 17:32:19,057][105620] Updated weights for policy 1, policy_version 298967 (0.0009) [2023-12-26 17:32:19,122][105620] Updated weights for policy 1, policy_version 298977 (0.0009) [2023-12-26 17:32:19,242][105692] Updated weights for policy 0, policy_version 298732 (0.0009) [2023-12-26 17:32:19,311][105692] Updated weights for policy 0, policy_version 298742 (0.0009) [2023-12-26 17:32:19,378][105692] Updated weights for policy 0, policy_version 298752 (0.0009) [2023-12-26 17:32:19,883][105620] Updated weights for policy 1, policy_version 298987 (0.0009) [2023-12-26 17:32:19,952][105620] Updated weights for policy 1, policy_version 298997 (0.0009) [2023-12-26 17:32:20,018][105586] KL-divergence is very high: 109.3269 [2023-12-26 17:32:20,019][105620] Updated weights for policy 1, policy_version 299007 (0.0009) [2023-12-26 17:32:20,031][105586] KL-divergence is very high: 100.5306 [2023-12-26 17:32:20,052][105692] Updated weights for policy 0, policy_version 298762 (0.0008) [2023-12-26 17:32:20,110][105692] Updated weights for policy 0, policy_version 298772 (0.0007) [2023-12-26 17:32:20,173][105692] Updated weights for policy 0, policy_version 298782 (0.0005) [2023-12-26 17:32:20,232][105692] Updated weights for policy 0, policy_version 298792 (0.0005) [2023-12-26 17:32:20,786][105620] Updated weights for policy 1, policy_version 299017 (0.0008) [2023-12-26 17:32:20,826][105586] KL-divergence is very high: 216.9304 [2023-12-26 17:32:20,832][105586] KL-divergence is very high: 153.6935 [2023-12-26 17:32:20,839][105586] KL-divergence is very high: 132.6876 [2023-12-26 17:32:20,843][105620] Updated weights for policy 1, policy_version 299027 (0.0008) [2023-12-26 17:32:20,845][105586] KL-divergence is very high: 172.3528 [2023-12-26 17:32:20,852][105586] KL-divergence is very high: 136.4266 [2023-12-26 17:32:20,857][105586] KL-divergence is very high: 153.9601 [2023-12-26 17:32:20,868][105586] KL-divergence is very high: 183.2737 [2023-12-26 17:32:20,873][105586] KL-divergence is very high: 259.5843 [2023-12-26 17:32:20,878][105586] KL-divergence is very high: 168.4192 [2023-12-26 17:32:20,884][105586] KL-divergence is very high: 181.1692 [2023-12-26 17:32:20,888][105586] KL-divergence is very high: 200.7369 [2023-12-26 17:32:20,894][105586] KL-divergence is very high: 133.1507 [2023-12-26 17:32:20,900][105586] KL-divergence is very high: 130.4356 [2023-12-26 17:32:20,901][105620] Updated weights for policy 1, policy_version 299037 (0.0007) [2023-12-26 17:32:20,910][105586] KL-divergence is very high: 120.4466 [2023-12-26 17:32:20,918][105586] KL-divergence is very high: 221.0549 [2023-12-26 17:32:20,924][105586] KL-divergence is very high: 114.1992 [2023-12-26 17:32:20,928][105692] Updated weights for policy 0, policy_version 298802 (0.0006) [2023-12-26 17:32:20,930][105586] KL-divergence is very high: 100.3176 [2023-12-26 17:32:20,960][105620] Updated weights for policy 1, policy_version 299047 (0.0008) [2023-12-26 17:32:20,981][105692] Updated weights for policy 0, policy_version 298812 (0.0006) [2023-12-26 17:32:21,043][105692] Updated weights for policy 0, policy_version 298822 (0.0008) [2023-12-26 17:32:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 153075712. Throughput: 0: 9511.1, 1: 9973.0. Samples: 153058068. Policy #0 lag: (min: 26.0, avg: 44.0, max: 58.0) [2023-12-26 17:32:21,063][104569] Avg episode reward: [(0, '9267.121'), (1, '4361.321')] [2023-12-26 17:32:21,748][105586] KL-divergence is very high: 107.4565 [2023-12-26 17:32:21,756][105620] Updated weights for policy 1, policy_version 299057 (0.0008) [2023-12-26 17:32:21,789][105692] Updated weights for policy 0, policy_version 298832 (0.0009) [2023-12-26 17:32:21,824][105620] Updated weights for policy 1, policy_version 299067 (0.0009) [2023-12-26 17:32:21,856][105692] Updated weights for policy 0, policy_version 298842 (0.0007) [2023-12-26 17:32:21,872][105586] KL-divergence is very high: 100.4293 [2023-12-26 17:32:21,883][105620] Updated weights for policy 1, policy_version 299077 (0.0007) [2023-12-26 17:32:21,915][105692] Updated weights for policy 0, policy_version 298852 (0.0006) [2023-12-26 17:32:22,635][105692] Updated weights for policy 0, policy_version 298862 (0.0007) [2023-12-26 17:32:22,651][105586] KL-divergence is very high: 241.6075 [2023-12-26 17:32:22,671][105620] Updated weights for policy 1, policy_version 299087 (0.0006) [2023-12-26 17:32:22,688][105692] Updated weights for policy 0, policy_version 298872 (0.0007) [2023-12-26 17:32:22,692][105586] KL-divergence is very high: 225.7503 [2023-12-26 17:32:22,705][105586] KL-divergence is very high: 311.0708 [2023-12-26 17:32:22,735][105620] Updated weights for policy 1, policy_version 299097 (0.0005) [2023-12-26 17:32:22,745][105692] Updated weights for policy 0, policy_version 298882 (0.0007) [2023-12-26 17:32:22,756][105586] KL-divergence is very high: 202.8615 [2023-12-26 17:32:22,795][105586] KL-divergence is very high: 117.5723 [2023-12-26 17:32:22,803][105620] Updated weights for policy 1, policy_version 299107 (0.0006) [2023-12-26 17:32:22,810][105586] KL-divergence is very high: 183.1886 [2023-12-26 17:32:23,429][105692] Updated weights for policy 0, policy_version 298892 (0.0008) [2023-12-26 17:32:23,493][105692] Updated weights for policy 0, policy_version 298902 (0.0009) [2023-12-26 17:32:23,557][105692] Updated weights for policy 0, policy_version 298912 (0.0009) [2023-12-26 17:32:23,571][105620] Updated weights for policy 1, policy_version 299117 (0.0007) [2023-12-26 17:32:23,576][105586] KL-divergence is very high: 187.1737 [2023-12-26 17:32:23,583][105586] KL-divergence is very high: 206.7174 [2023-12-26 17:32:23,589][105586] KL-divergence is very high: 363.4875 [2023-12-26 17:32:23,598][105586] KL-divergence is very high: 446.8741 [2023-12-26 17:32:23,603][105586] KL-divergence is very high: 535.8934 [2023-12-26 17:32:23,609][105586] KL-divergence is very high: 304.9995 [2023-12-26 17:32:23,616][105586] KL-divergence is very high: 283.9132 [2023-12-26 17:32:23,622][105586] KL-divergence is very high: 353.0797 [2023-12-26 17:32:23,629][105620] Updated weights for policy 1, policy_version 299127 (0.0007) [2023-12-26 17:32:23,629][105586] KL-divergence is very high: 242.5253 [2023-12-26 17:32:23,636][105586] KL-divergence is very high: 338.3398 [2023-12-26 17:32:23,649][105586] KL-divergence is very high: 264.7539 [2023-12-26 17:32:23,655][105586] KL-divergence is very high: 277.8258 [2023-12-26 17:32:23,661][105586] KL-divergence is very high: 107.7820 [2023-12-26 17:32:23,687][105620] Updated weights for policy 1, policy_version 299137 (0.0009) [2023-12-26 17:32:24,256][105692] Updated weights for policy 0, policy_version 298922 (0.0007) [2023-12-26 17:32:24,305][105692] Updated weights for policy 0, policy_version 298932 (0.0009) [2023-12-26 17:32:24,355][105692] Updated weights for policy 0, policy_version 298942 (0.0009) [2023-12-26 17:32:24,387][105586] KL-divergence is very high: 132.0458 [2023-12-26 17:32:24,410][105620] Updated weights for policy 1, policy_version 299147 (0.0009) [2023-12-26 17:32:24,413][105692] Updated weights for policy 0, policy_version 298952 (0.0008) [2023-12-26 17:32:24,440][105586] KL-divergence is very high: 263.3496 [2023-12-26 17:32:24,466][105620] Updated weights for policy 1, policy_version 299157 (0.0009) [2023-12-26 17:32:24,482][105586] KL-divergence is very high: 130.6055 [2023-12-26 17:32:24,517][105620] Updated weights for policy 1, policy_version 299167 (0.0009) [2023-12-26 17:32:24,521][105586] KL-divergence is very high: 374.2479 [2023-12-26 17:32:25,131][105692] Updated weights for policy 0, policy_version 298962 (0.0009) [2023-12-26 17:32:25,186][105692] Updated weights for policy 0, policy_version 298972 (0.0009) [2023-12-26 17:32:25,246][105692] Updated weights for policy 0, policy_version 298982 (0.0009) [2023-12-26 17:32:25,285][105620] Updated weights for policy 1, policy_version 299177 (0.0008) [2023-12-26 17:32:25,341][105620] Updated weights for policy 1, policy_version 299187 (0.0006) [2023-12-26 17:32:25,392][105620] Updated weights for policy 1, policy_version 299197 (0.0009) [2023-12-26 17:32:25,443][105620] Updated weights for policy 1, policy_version 299207 (0.0009) [2023-12-26 17:32:25,978][105692] Updated weights for policy 0, policy_version 298992 (0.0006) [2023-12-26 17:32:26,029][105692] Updated weights for policy 0, policy_version 299002 (0.0005) [2023-12-26 17:32:26,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 153157632. Throughput: 0: 9513.6, 1: 9947.3. Samples: 153172064. Policy #0 lag: (min: 26.0, avg: 44.0, max: 58.0) [2023-12-26 17:32:26,062][104569] Avg episode reward: [(0, '9267.274'), (1, '2394.345')] [2023-12-26 17:32:26,089][105692] Updated weights for policy 0, policy_version 299012 (0.0005) [2023-12-26 17:32:26,114][105620] Updated weights for policy 1, policy_version 299217 (0.0009) [2023-12-26 17:32:26,174][105620] Updated weights for policy 1, policy_version 299227 (0.0009) [2023-12-26 17:32:26,177][105586] KL-divergence is very high: 120.8409 [2023-12-26 17:32:26,182][105586] KL-divergence is very high: 116.9290 [2023-12-26 17:32:26,205][105586] KL-divergence is very high: 104.5103 [2023-12-26 17:32:26,214][105586] KL-divergence is very high: 142.1352 [2023-12-26 17:32:26,219][105586] KL-divergence is very high: 126.5146 [2023-12-26 17:32:26,220][105620] Updated weights for policy 1, policy_version 299237 (0.0009) [2023-12-26 17:32:26,661][105692] Updated weights for policy 0, policy_version 299022 (0.0006) [2023-12-26 17:32:26,724][105692] Updated weights for policy 0, policy_version 299032 (0.0005) [2023-12-26 17:32:26,782][105692] Updated weights for policy 0, policy_version 299042 (0.0008) [2023-12-26 17:32:27,047][105586] KL-divergence is very high: 112.6609 [2023-12-26 17:32:27,077][105620] Updated weights for policy 1, policy_version 299247 (0.0009) [2023-12-26 17:32:27,127][105586] KL-divergence is very high: 107.6760 [2023-12-26 17:32:27,128][105620] Updated weights for policy 1, policy_version 299257 (0.0009) [2023-12-26 17:32:27,142][105586] KL-divergence is very high: 223.8527 [2023-12-26 17:32:27,148][105586] KL-divergence is very high: 199.5429 [2023-12-26 17:32:27,153][105586] KL-divergence is very high: 167.8817 [2023-12-26 17:32:27,173][105586] KL-divergence is very high: 137.2491 [2023-12-26 17:32:27,186][105620] Updated weights for policy 1, policy_version 299267 (0.0010) [2023-12-26 17:32:27,194][105586] KL-divergence is very high: 195.4969 [2023-12-26 17:32:27,201][105586] KL-divergence is very high: 156.2511 [2023-12-26 17:32:27,208][105586] KL-divergence is very high: 112.3941 [2023-12-26 17:32:27,378][105692] Updated weights for policy 0, policy_version 299052 (0.0008) [2023-12-26 17:32:27,435][105692] Updated weights for policy 0, policy_version 299062 (0.0009) [2023-12-26 17:32:27,489][105692] Updated weights for policy 0, policy_version 299072 (0.0009) [2023-12-26 17:32:27,907][105620] Updated weights for policy 1, policy_version 299277 (0.0007) [2023-12-26 17:32:27,957][105620] Updated weights for policy 1, policy_version 299287 (0.0005) [2023-12-26 17:32:27,970][105586] KL-divergence is very high: 136.8282 [2023-12-26 17:32:27,975][105586] KL-divergence is very high: 126.6811 [2023-12-26 17:32:28,005][105620] Updated weights for policy 1, policy_version 299297 (0.0006) [2023-12-26 17:32:28,008][105586] KL-divergence is very high: 225.5672 [2023-12-26 17:32:28,013][105586] KL-divergence is very high: 152.1986 [2023-12-26 17:32:28,137][105692] Updated weights for policy 0, policy_version 299082 (0.0009) [2023-12-26 17:32:28,195][105692] Updated weights for policy 0, policy_version 299092 (0.0006) [2023-12-26 17:32:28,248][105692] Updated weights for policy 0, policy_version 299102 (0.0005) [2023-12-26 17:32:28,301][105692] Updated weights for policy 0, policy_version 299112 (0.0005) [2023-12-26 17:32:28,625][105620] Updated weights for policy 1, policy_version 299307 (0.0009) [2023-12-26 17:32:28,690][105620] Updated weights for policy 1, policy_version 299317 (0.0008) [2023-12-26 17:32:28,749][105620] Updated weights for policy 1, policy_version 299327 (0.0007) [2023-12-26 17:32:28,943][105692] Updated weights for policy 0, policy_version 299122 (0.0009) [2023-12-26 17:32:28,997][105692] Updated weights for policy 0, policy_version 299132 (0.0010) [2023-12-26 17:32:29,050][105692] Updated weights for policy 0, policy_version 299142 (0.0009) [2023-12-26 17:32:29,352][105620] Updated weights for policy 1, policy_version 299337 (0.0006) [2023-12-26 17:32:29,411][105620] Updated weights for policy 1, policy_version 299347 (0.0007) [2023-12-26 17:32:29,445][105586] KL-divergence is very high: 141.8773 [2023-12-26 17:32:29,462][105586] KL-divergence is very high: 110.4630 [2023-12-26 17:32:29,473][105620] Updated weights for policy 1, policy_version 299357 (0.0005) [2023-12-26 17:32:29,478][105586] KL-divergence is very high: 123.8291 [2023-12-26 17:32:29,490][105586] KL-divergence is very high: 136.9827 [2023-12-26 17:32:29,530][105620] Updated weights for policy 1, policy_version 299367 (0.0005) [2023-12-26 17:32:29,952][105692] Updated weights for policy 0, policy_version 299152 (0.0010) [2023-12-26 17:32:30,002][105692] Updated weights for policy 0, policy_version 299162 (0.0009) [2023-12-26 17:32:30,060][105692] Updated weights for policy 0, policy_version 299172 (0.0008) [2023-12-26 17:32:30,106][105620] Updated weights for policy 1, policy_version 299377 (0.0008) [2023-12-26 17:32:30,168][105620] Updated weights for policy 1, policy_version 299387 (0.0009) [2023-12-26 17:32:30,227][105620] Updated weights for policy 1, policy_version 299397 (0.0009) [2023-12-26 17:32:30,810][105692] Updated weights for policy 0, policy_version 299182 (0.0010) [2023-12-26 17:32:30,861][105692] Updated weights for policy 0, policy_version 299192 (0.0010) [2023-12-26 17:32:30,865][105620] Updated weights for policy 1, policy_version 299407 (0.0006) [2023-12-26 17:32:30,915][105692] Updated weights for policy 0, policy_version 299202 (0.0009) [2023-12-26 17:32:30,919][105620] Updated weights for policy 1, policy_version 299417 (0.0005) [2023-12-26 17:32:30,972][105620] Updated weights for policy 1, policy_version 299427 (0.0005) [2023-12-26 17:32:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 153272320. Throughput: 0: 9601.2, 1: 9981.0. Samples: 153234252. Policy #0 lag: (min: 26.0, avg: 44.0, max: 58.0) [2023-12-26 17:32:31,063][104569] Avg episode reward: [(0, '9268.044'), (1, '4949.839')] [2023-12-26 17:32:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000299208_76611584.pth... [2023-12-26 17:32:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000299432_76660736.pth... [2023-12-26 17:32:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000298088_76324864.pth [2023-12-26 17:32:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000298248_76357632.pth [2023-12-26 17:32:31,645][105620] Updated weights for policy 1, policy_version 299437 (0.0005) [2023-12-26 17:32:31,692][105692] Updated weights for policy 0, policy_version 299212 (0.0010) [2023-12-26 17:32:31,708][105620] Updated weights for policy 1, policy_version 299447 (0.0007) [2023-12-26 17:32:31,759][105692] Updated weights for policy 0, policy_version 299222 (0.0007) [2023-12-26 17:32:31,769][105620] Updated weights for policy 1, policy_version 299457 (0.0006) [2023-12-26 17:32:31,822][105692] Updated weights for policy 0, policy_version 299232 (0.0009) [2023-12-26 17:32:32,442][105620] Updated weights for policy 1, policy_version 299467 (0.0006) [2023-12-26 17:32:32,497][105620] Updated weights for policy 1, policy_version 299477 (0.0006) [2023-12-26 17:32:32,552][105620] Updated weights for policy 1, policy_version 299487 (0.0006) [2023-12-26 17:32:32,565][105692] Updated weights for policy 0, policy_version 299242 (0.0008) [2023-12-26 17:32:32,623][105692] Updated weights for policy 0, policy_version 299252 (0.0009) [2023-12-26 17:32:32,680][105692] Updated weights for policy 0, policy_version 299262 (0.0009) [2023-12-26 17:32:32,728][105692] Updated weights for policy 0, policy_version 299272 (0.0008) [2023-12-26 17:32:33,152][105620] Updated weights for policy 1, policy_version 299497 (0.0006) [2023-12-26 17:32:33,204][105620] Updated weights for policy 1, policy_version 299507 (0.0009) [2023-12-26 17:32:33,255][105620] Updated weights for policy 1, policy_version 299517 (0.0009) [2023-12-26 17:32:33,304][105620] Updated weights for policy 1, policy_version 299527 (0.0008) [2023-12-26 17:32:33,479][105692] Updated weights for policy 0, policy_version 299282 (0.0009) [2023-12-26 17:32:33,539][105692] Updated weights for policy 0, policy_version 299292 (0.0009) [2023-12-26 17:32:33,597][105692] Updated weights for policy 0, policy_version 299302 (0.0009) [2023-12-26 17:32:34,067][105620] Updated weights for policy 1, policy_version 299537 (0.0009) [2023-12-26 17:32:34,127][105620] Updated weights for policy 1, policy_version 299547 (0.0008) [2023-12-26 17:32:34,191][105620] Updated weights for policy 1, policy_version 299557 (0.0007) [2023-12-26 17:32:34,362][105692] Updated weights for policy 0, policy_version 299312 (0.0009) [2023-12-26 17:32:34,420][105692] Updated weights for policy 0, policy_version 299322 (0.0009) [2023-12-26 17:32:34,482][105692] Updated weights for policy 0, policy_version 299332 (0.0009) [2023-12-26 17:32:34,859][105620] Updated weights for policy 1, policy_version 299567 (0.0005) [2023-12-26 17:32:34,921][105620] Updated weights for policy 1, policy_version 299577 (0.0008) [2023-12-26 17:32:34,982][105620] Updated weights for policy 1, policy_version 299587 (0.0009) [2023-12-26 17:32:35,308][105692] Updated weights for policy 0, policy_version 299343 (0.0010) [2023-12-26 17:32:35,359][105692] Updated weights for policy 0, policy_version 299353 (0.0005) [2023-12-26 17:32:35,413][105692] Updated weights for policy 0, policy_version 299363 (0.0005) [2023-12-26 17:32:35,534][105620] Updated weights for policy 1, policy_version 299597 (0.0007) [2023-12-26 17:32:35,596][105620] Updated weights for policy 1, policy_version 299607 (0.0006) [2023-12-26 17:32:35,647][105620] Updated weights for policy 1, policy_version 299617 (0.0009) [2023-12-26 17:32:36,046][105692] Updated weights for policy 0, policy_version 299373 (0.0008) [2023-12-26 17:32:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 153362432. Throughput: 0: 9490.3, 1: 10041.5. Samples: 153352244. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-26 17:32:36,062][104569] Avg episode reward: [(0, '9267.345'), (1, '5680.028')] [2023-12-26 17:32:36,100][105692] Updated weights for policy 0, policy_version 299383 (0.0009) [2023-12-26 17:32:36,168][105692] Updated weights for policy 0, policy_version 299393 (0.0009) [2023-12-26 17:32:36,414][105620] Updated weights for policy 1, policy_version 299627 (0.0009) [2023-12-26 17:32:36,469][105620] Updated weights for policy 1, policy_version 299637 (0.0008) [2023-12-26 17:32:36,532][105620] Updated weights for policy 1, policy_version 299647 (0.0010) [2023-12-26 17:32:36,950][105692] Updated weights for policy 0, policy_version 299403 (0.0009) [2023-12-26 17:32:37,001][105692] Updated weights for policy 0, policy_version 299413 (0.0009) [2023-12-26 17:32:37,054][105692] Updated weights for policy 0, policy_version 299424 (0.0010) [2023-12-26 17:32:37,182][105620] Updated weights for policy 1, policy_version 299657 (0.0009) [2023-12-26 17:32:37,229][105620] Updated weights for policy 1, policy_version 299667 (0.0005) [2023-12-26 17:32:37,273][105620] Updated weights for policy 1, policy_version 299677 (0.0005) [2023-12-26 17:32:37,327][105620] Updated weights for policy 1, policy_version 299687 (0.0006) [2023-12-26 17:32:37,883][105620] Updated weights for policy 1, policy_version 299697 (0.0009) [2023-12-26 17:32:37,932][105620] Updated weights for policy 1, policy_version 299707 (0.0009) [2023-12-26 17:32:37,951][105692] Updated weights for policy 0, policy_version 299434 (0.0009) [2023-12-26 17:32:37,987][105620] Updated weights for policy 1, policy_version 299717 (0.0008) [2023-12-26 17:32:38,006][105692] Updated weights for policy 0, policy_version 299444 (0.0008) [2023-12-26 17:32:38,063][105692] Updated weights for policy 0, policy_version 299454 (0.0009) [2023-12-26 17:32:38,124][105692] Updated weights for policy 0, policy_version 299464 (0.0008) [2023-12-26 17:32:38,772][105620] Updated weights for policy 1, policy_version 299727 (0.0007) [2023-12-26 17:32:38,831][105620] Updated weights for policy 1, policy_version 299737 (0.0009) [2023-12-26 17:32:38,858][105692] Updated weights for policy 0, policy_version 299474 (0.0006) [2023-12-26 17:32:38,861][105586] KL-divergence is very high: 144.2618 [2023-12-26 17:32:38,866][105586] KL-divergence is very high: 145.9613 [2023-12-26 17:32:38,881][105620] Updated weights for policy 1, policy_version 299747 (0.0007) [2023-12-26 17:32:38,901][105586] KL-divergence is very high: 146.4375 [2023-12-26 17:32:38,917][105692] Updated weights for policy 0, policy_version 299484 (0.0008) [2023-12-26 17:32:38,974][105692] Updated weights for policy 0, policy_version 299494 (0.0009) [2023-12-26 17:32:39,545][105586] KL-divergence is very high: 125.8006 [2023-12-26 17:32:39,580][105620] Updated weights for policy 1, policy_version 299757 (0.0008) [2023-12-26 17:32:39,600][105586] KL-divergence is very high: 137.1785 [2023-12-26 17:32:39,653][105620] Updated weights for policy 1, policy_version 299767 (0.0006) [2023-12-26 17:32:39,660][105586] KL-divergence is very high: 125.1333 [2023-12-26 17:32:39,722][105620] Updated weights for policy 1, policy_version 299777 (0.0006) [2023-12-26 17:32:39,785][105692] Updated weights for policy 0, policy_version 299504 (0.0007) [2023-12-26 17:32:39,845][105692] Updated weights for policy 0, policy_version 299514 (0.0008) [2023-12-26 17:32:39,900][105692] Updated weights for policy 0, policy_version 299524 (0.0009) [2023-12-26 17:32:40,282][105620] Updated weights for policy 1, policy_version 299787 (0.0007) [2023-12-26 17:32:40,344][105620] Updated weights for policy 1, policy_version 299797 (0.0007) [2023-12-26 17:32:40,413][105620] Updated weights for policy 1, policy_version 299807 (0.0006) [2023-12-26 17:32:40,721][105692] Updated weights for policy 0, policy_version 299534 (0.0008) [2023-12-26 17:32:40,773][105692] Updated weights for policy 0, policy_version 299544 (0.0010) [2023-12-26 17:32:40,834][105692] Updated weights for policy 0, policy_version 299554 (0.0009) [2023-12-26 17:32:41,032][105620] Updated weights for policy 1, policy_version 299817 (0.0005) [2023-12-26 17:32:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 153460736. Throughput: 0: 9492.6, 1: 10118.5. Samples: 153469296. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-26 17:32:41,063][104569] Avg episode reward: [(0, '9177.331'), (1, '6143.349')] [2023-12-26 17:32:41,108][105620] Updated weights for policy 1, policy_version 299827 (0.0010) [2023-12-26 17:32:41,171][105620] Updated weights for policy 1, policy_version 299837 (0.0011) [2023-12-26 17:32:41,238][105620] Updated weights for policy 1, policy_version 299847 (0.0010) [2023-12-26 17:32:41,650][105692] Updated weights for policy 0, policy_version 299564 (0.0010) [2023-12-26 17:32:41,710][105692] Updated weights for policy 0, policy_version 299574 (0.0010) [2023-12-26 17:32:41,771][105692] Updated weights for policy 0, policy_version 299584 (0.0008) [2023-12-26 17:32:41,891][105620] Updated weights for policy 1, policy_version 299857 (0.0009) [2023-12-26 17:32:41,939][105620] Updated weights for policy 1, policy_version 299867 (0.0009) [2023-12-26 17:32:42,001][105620] Updated weights for policy 1, policy_version 299877 (0.0009) [2023-12-26 17:32:42,519][105692] Updated weights for policy 0, policy_version 299594 (0.0010) [2023-12-26 17:32:42,583][105692] Updated weights for policy 0, policy_version 299604 (0.0009) [2023-12-26 17:32:42,644][105692] Updated weights for policy 0, policy_version 299614 (0.0009) [2023-12-26 17:32:42,701][105692] Updated weights for policy 0, policy_version 299624 (0.0008) [2023-12-26 17:32:42,729][105620] Updated weights for policy 1, policy_version 299887 (0.0008) [2023-12-26 17:32:42,791][105620] Updated weights for policy 1, policy_version 299897 (0.0008) [2023-12-26 17:32:42,857][105620] Updated weights for policy 1, policy_version 299907 (0.0010) [2023-12-26 17:32:43,306][105692] Updated weights for policy 0, policy_version 299634 (0.0008) [2023-12-26 17:32:43,376][105692] Updated weights for policy 0, policy_version 299644 (0.0009) [2023-12-26 17:32:43,438][105692] Updated weights for policy 0, policy_version 299654 (0.0009) [2023-12-26 17:32:43,587][105620] Updated weights for policy 1, policy_version 299917 (0.0008) [2023-12-26 17:32:43,648][105620] Updated weights for policy 1, policy_version 299927 (0.0005) [2023-12-26 17:32:43,710][105620] Updated weights for policy 1, policy_version 299937 (0.0007) [2023-12-26 17:32:44,176][105692] Updated weights for policy 0, policy_version 299664 (0.0010) [2023-12-26 17:32:44,230][105692] Updated weights for policy 0, policy_version 299674 (0.0010) [2023-12-26 17:32:44,292][105692] Updated weights for policy 0, policy_version 299684 (0.0010) [2023-12-26 17:32:44,334][105620] Updated weights for policy 1, policy_version 299947 (0.0009) [2023-12-26 17:32:44,396][105620] Updated weights for policy 1, policy_version 299957 (0.0006) [2023-12-26 17:32:44,447][105620] Updated weights for policy 1, policy_version 299967 (0.0009) [2023-12-26 17:32:45,105][105620] Updated weights for policy 1, policy_version 299977 (0.0008) [2023-12-26 17:32:45,112][105692] Updated weights for policy 0, policy_version 299694 (0.0009) [2023-12-26 17:32:45,164][105620] Updated weights for policy 1, policy_version 299987 (0.0008) [2023-12-26 17:32:45,170][105692] Updated weights for policy 0, policy_version 299704 (0.0010) [2023-12-26 17:32:45,195][105586] KL-divergence is very high: 114.5309 [2023-12-26 17:32:45,227][105620] Updated weights for policy 1, policy_version 299997 (0.0006) [2023-12-26 17:32:45,227][105692] Updated weights for policy 0, policy_version 299714 (0.0007) [2023-12-26 17:32:45,246][105586] KL-divergence is very high: 129.9704 [2023-12-26 17:32:45,286][105620] Updated weights for policy 1, policy_version 300007 (0.0008) [2023-12-26 17:32:45,975][105692] Updated weights for policy 0, policy_version 299724 (0.0008) [2023-12-26 17:32:46,017][105620] Updated weights for policy 1, policy_version 300017 (0.0007) [2023-12-26 17:32:46,027][105692] Updated weights for policy 0, policy_version 299734 (0.0007) [2023-12-26 17:32:46,062][104569] Fps is (10 sec: 18840.5, 60 sec: 19524.1, 300 sec: 19549.7). Total num frames: 153550848. Throughput: 0: 9448.6, 1: 10077.7. Samples: 153526496. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-26 17:32:46,064][104569] Avg episode reward: [(0, '9267.836'), (1, '6420.647')] [2023-12-26 17:32:46,079][105692] Updated weights for policy 0, policy_version 299744 (0.0006) [2023-12-26 17:32:46,081][105620] Updated weights for policy 1, policy_version 300027 (0.0007) [2023-12-26 17:32:46,126][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000299752_76750848.pth... [2023-12-26 17:32:46,130][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000298632_76464128.pth [2023-12-26 17:32:46,136][105620] Updated weights for policy 1, policy_version 300037 (0.0007) [2023-12-26 17:32:46,150][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000300040_76816384.pth... [2023-12-26 17:32:46,154][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000298856_76513280.pth [2023-12-26 17:32:46,780][105620] Updated weights for policy 1, policy_version 300047 (0.0008) [2023-12-26 17:32:46,832][105620] Updated weights for policy 1, policy_version 300057 (0.0007) [2023-12-26 17:32:46,882][105620] Updated weights for policy 1, policy_version 300067 (0.0009) [2023-12-26 17:32:46,889][105692] Updated weights for policy 0, policy_version 299754 (0.0008) [2023-12-26 17:32:46,942][105692] Updated weights for policy 0, policy_version 299765 (0.0008) [2023-12-26 17:32:46,990][105692] Updated weights for policy 0, policy_version 299775 (0.0009) [2023-12-26 17:32:47,553][105620] Updated weights for policy 1, policy_version 300077 (0.0010) [2023-12-26 17:32:47,605][105620] Updated weights for policy 1, policy_version 300087 (0.0010) [2023-12-26 17:32:47,658][105620] Updated weights for policy 1, policy_version 300097 (0.0011) [2023-12-26 17:32:47,807][105692] Updated weights for policy 0, policy_version 299785 (0.0009) [2023-12-26 17:32:47,859][105692] Updated weights for policy 0, policy_version 299795 (0.0010) [2023-12-26 17:32:47,904][105692] Updated weights for policy 0, policy_version 299805 (0.0008) [2023-12-26 17:32:47,959][105692] Updated weights for policy 0, policy_version 299815 (0.0008) [2023-12-26 17:32:48,358][105620] Updated weights for policy 1, policy_version 300107 (0.0010) [2023-12-26 17:32:48,420][105620] Updated weights for policy 1, policy_version 300117 (0.0008) [2023-12-26 17:32:48,480][105620] Updated weights for policy 1, policy_version 300127 (0.0009) [2023-12-26 17:32:48,659][105692] Updated weights for policy 0, policy_version 299825 (0.0009) [2023-12-26 17:32:48,724][105692] Updated weights for policy 0, policy_version 299835 (0.0009) [2023-12-26 17:32:48,788][105692] Updated weights for policy 0, policy_version 299845 (0.0008) [2023-12-26 17:32:49,172][105620] Updated weights for policy 1, policy_version 300137 (0.0009) [2023-12-26 17:32:49,236][105620] Updated weights for policy 1, policy_version 300147 (0.0011) [2023-12-26 17:32:49,300][105620] Updated weights for policy 1, policy_version 300157 (0.0010) [2023-12-26 17:32:49,370][105620] Updated weights for policy 1, policy_version 300167 (0.0011) [2023-12-26 17:32:49,582][105692] Updated weights for policy 0, policy_version 299855 (0.0009) [2023-12-26 17:32:49,633][105585] KL-divergence is very high: 229.1916 [2023-12-26 17:32:49,636][105692] Updated weights for policy 0, policy_version 299865 (0.0010) [2023-12-26 17:32:49,671][105585] KL-divergence is very high: 318.7647 [2023-12-26 17:32:49,690][105692] Updated weights for policy 0, policy_version 299876 (0.0010) [2023-12-26 17:32:50,049][105620] Updated weights for policy 1, policy_version 300177 (0.0009) [2023-12-26 17:32:50,105][105620] Updated weights for policy 1, policy_version 300187 (0.0009) [2023-12-26 17:32:50,163][105620] Updated weights for policy 1, policy_version 300197 (0.0008) [2023-12-26 17:32:50,486][105692] Updated weights for policy 0, policy_version 299886 (0.0010) [2023-12-26 17:32:50,545][105692] Updated weights for policy 0, policy_version 299896 (0.0011) [2023-12-26 17:32:50,606][105692] Updated weights for policy 0, policy_version 299906 (0.0009) [2023-12-26 17:32:50,972][105620] Updated weights for policy 1, policy_version 300207 (0.0010) [2023-12-26 17:32:51,035][105620] Updated weights for policy 1, policy_version 300217 (0.0009) [2023-12-26 17:32:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 153649152. Throughput: 0: 9367.4, 1: 10062.5. Samples: 153642696. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-26 17:32:51,062][104569] Avg episode reward: [(0, '9084.648'), (1, '6868.535')] [2023-12-26 17:32:51,106][105620] Updated weights for policy 1, policy_version 300227 (0.0007) [2023-12-26 17:32:51,412][105692] Updated weights for policy 0, policy_version 299916 (0.0008) [2023-12-26 17:32:51,478][105692] Updated weights for policy 0, policy_version 299926 (0.0009) [2023-12-26 17:32:51,536][105692] Updated weights for policy 0, policy_version 299936 (0.0009) [2023-12-26 17:32:51,863][105620] Updated weights for policy 1, policy_version 300237 (0.0008) [2023-12-26 17:32:51,919][105620] Updated weights for policy 1, policy_version 300248 (0.0010) [2023-12-26 17:32:51,977][105620] Updated weights for policy 1, policy_version 300258 (0.0010) [2023-12-26 17:32:52,161][105692] Updated weights for policy 0, policy_version 299946 (0.0008) [2023-12-26 17:32:52,212][105692] Updated weights for policy 0, policy_version 299956 (0.0005) [2023-12-26 17:32:52,269][105692] Updated weights for policy 0, policy_version 299966 (0.0006) [2023-12-26 17:32:52,329][105692] Updated weights for policy 0, policy_version 299976 (0.0006) [2023-12-26 17:32:52,761][105620] Updated weights for policy 1, policy_version 300268 (0.0011) [2023-12-26 17:32:52,817][105620] Updated weights for policy 1, policy_version 300278 (0.0011) [2023-12-26 17:32:52,870][105692] Updated weights for policy 0, policy_version 299986 (0.0005) [2023-12-26 17:32:52,870][105620] Updated weights for policy 1, policy_version 300288 (0.0010) [2023-12-26 17:32:52,926][105692] Updated weights for policy 0, policy_version 299996 (0.0006) [2023-12-26 17:32:52,984][105692] Updated weights for policy 0, policy_version 300006 (0.0006) [2023-12-26 17:32:53,585][105692] Updated weights for policy 0, policy_version 300016 (0.0008) [2023-12-26 17:32:53,634][105620] Updated weights for policy 1, policy_version 300298 (0.0010) [2023-12-26 17:32:53,648][105692] Updated weights for policy 0, policy_version 300026 (0.0007) [2023-12-26 17:32:53,692][105620] Updated weights for policy 1, policy_version 300308 (0.0010) [2023-12-26 17:32:53,709][105692] Updated weights for policy 0, policy_version 300036 (0.0006) [2023-12-26 17:32:53,743][105620] Updated weights for policy 1, policy_version 300318 (0.0010) [2023-12-26 17:32:53,785][105586] KL-divergence is very high: 113.5033 [2023-12-26 17:32:53,796][105620] Updated weights for policy 1, policy_version 300328 (0.0005) [2023-12-26 17:32:54,471][105620] Updated weights for policy 1, policy_version 300338 (0.0010) [2023-12-26 17:32:54,501][105692] Updated weights for policy 0, policy_version 300046 (0.0006) [2023-12-26 17:32:54,526][105620] Updated weights for policy 1, policy_version 300348 (0.0010) [2023-12-26 17:32:54,560][105692] Updated weights for policy 0, policy_version 300056 (0.0005) [2023-12-26 17:32:54,585][105620] Updated weights for policy 1, policy_version 300358 (0.0010) [2023-12-26 17:32:54,616][105692] Updated weights for policy 0, policy_version 300066 (0.0006) [2023-12-26 17:32:55,325][105692] Updated weights for policy 0, policy_version 300076 (0.0007) [2023-12-26 17:32:55,339][105620] Updated weights for policy 1, policy_version 300368 (0.0010) [2023-12-26 17:32:55,376][105692] Updated weights for policy 0, policy_version 300086 (0.0005) [2023-12-26 17:32:55,401][105620] Updated weights for policy 1, policy_version 300378 (0.0010) [2023-12-26 17:32:55,434][105692] Updated weights for policy 0, policy_version 300096 (0.0006) [2023-12-26 17:32:55,452][105620] Updated weights for policy 1, policy_version 300388 (0.0010) [2023-12-26 17:32:56,062][104569] Fps is (10 sec: 19661.7, 60 sec: 19387.8, 300 sec: 19521.9). Total num frames: 153747456. Throughput: 0: 9503.2, 1: 9928.7. Samples: 153758764. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-26 17:32:56,063][104569] Avg episode reward: [(0, '9175.791'), (1, '7695.798')] [2023-12-26 17:32:56,181][105692] Updated weights for policy 0, policy_version 300106 (0.0007) [2023-12-26 17:32:56,183][105620] Updated weights for policy 1, policy_version 300398 (0.0009) [2023-12-26 17:32:56,235][105692] Updated weights for policy 0, policy_version 300116 (0.0006) [2023-12-26 17:32:56,248][105620] Updated weights for policy 1, policy_version 300408 (0.0009) [2023-12-26 17:32:56,262][105586] KL-divergence is very high: 193.8743 [2023-12-26 17:32:56,270][105586] KL-divergence is very high: 117.7435 [2023-12-26 17:32:56,287][105692] Updated weights for policy 0, policy_version 300126 (0.0005) [2023-12-26 17:32:56,303][105620] Updated weights for policy 1, policy_version 300418 (0.0008) [2023-12-26 17:32:56,307][105586] KL-divergence is very high: 216.8078 [2023-12-26 17:32:56,311][105586] KL-divergence is very high: 124.3968 [2023-12-26 17:32:56,341][105692] Updated weights for policy 0, policy_version 300136 (0.0005) [2023-12-26 17:32:56,967][105692] Updated weights for policy 0, policy_version 300146 (0.0007) [2023-12-26 17:32:57,020][105692] Updated weights for policy 0, policy_version 300156 (0.0007) [2023-12-26 17:32:57,072][105692] Updated weights for policy 0, policy_version 300166 (0.0005) [2023-12-26 17:32:57,072][105620] Updated weights for policy 1, policy_version 300428 (0.0008) [2023-12-26 17:32:57,126][105620] Updated weights for policy 1, policy_version 300438 (0.0009) [2023-12-26 17:32:57,176][105620] Updated weights for policy 1, policy_version 300448 (0.0008) [2023-12-26 17:32:57,774][105692] Updated weights for policy 0, policy_version 300176 (0.0009) [2023-12-26 17:32:57,834][105692] Updated weights for policy 0, policy_version 300186 (0.0009) [2023-12-26 17:32:57,872][105620] Updated weights for policy 1, policy_version 300458 (0.0010) [2023-12-26 17:32:57,890][105692] Updated weights for policy 0, policy_version 300196 (0.0008) [2023-12-26 17:32:57,917][105620] Updated weights for policy 1, policy_version 300468 (0.0007) [2023-12-26 17:32:57,964][105620] Updated weights for policy 1, policy_version 300478 (0.0008) [2023-12-26 17:32:58,017][105620] Updated weights for policy 1, policy_version 300488 (0.0008) [2023-12-26 17:32:58,683][105692] Updated weights for policy 0, policy_version 300206 (0.0007) [2023-12-26 17:32:58,754][105692] Updated weights for policy 0, policy_version 300216 (0.0008) [2023-12-26 17:32:58,787][105620] Updated weights for policy 1, policy_version 300498 (0.0007) [2023-12-26 17:32:58,823][105692] Updated weights for policy 0, policy_version 300226 (0.0007) [2023-12-26 17:32:58,860][105620] Updated weights for policy 1, policy_version 300508 (0.0009) [2023-12-26 17:32:58,880][105586] KL-divergence is very high: 128.9402 [2023-12-26 17:32:58,926][105620] Updated weights for policy 1, policy_version 300518 (0.0009) [2023-12-26 17:32:58,936][105586] KL-divergence is very high: 110.3532 [2023-12-26 17:32:59,657][105692] Updated weights for policy 0, policy_version 300236 (0.0008) [2023-12-26 17:32:59,708][105692] Updated weights for policy 0, policy_version 300246 (0.0009) [2023-12-26 17:32:59,763][105692] Updated weights for policy 0, policy_version 300256 (0.0009) [2023-12-26 17:32:59,824][105620] Updated weights for policy 1, policy_version 300528 (0.0009) [2023-12-26 17:32:59,889][105620] Updated weights for policy 1, policy_version 300538 (0.0009) [2023-12-26 17:32:59,950][105620] Updated weights for policy 1, policy_version 300548 (0.0009) [2023-12-26 17:33:00,492][105692] Updated weights for policy 0, policy_version 300266 (0.0007) [2023-12-26 17:33:00,549][105692] Updated weights for policy 0, policy_version 300276 (0.0007) [2023-12-26 17:33:00,608][105692] Updated weights for policy 0, policy_version 300286 (0.0008) [2023-12-26 17:33:00,666][105692] Updated weights for policy 0, policy_version 300296 (0.0008) [2023-12-26 17:33:00,701][105620] Updated weights for policy 1, policy_version 300558 (0.0009) [2023-12-26 17:33:00,749][105620] Updated weights for policy 1, policy_version 300568 (0.0010) [2023-12-26 17:33:00,797][105620] Updated weights for policy 1, policy_version 300578 (0.0010) [2023-12-26 17:33:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 153845760. Throughput: 0: 9526.4, 1: 9890.2. Samples: 153816336. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-26 17:33:01,062][104569] Avg episode reward: [(0, '9266.005'), (1, '7527.545')] [2023-12-26 17:33:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000300296_76890112.pth... [2023-12-26 17:33:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000300584_76955648.pth... [2023-12-26 17:33:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000299208_76611584.pth [2023-12-26 17:33:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000299432_76660736.pth [2023-12-26 17:33:01,470][105692] Updated weights for policy 0, policy_version 300306 (0.0009) [2023-12-26 17:33:01,509][105620] Updated weights for policy 1, policy_version 300588 (0.0009) [2023-12-26 17:33:01,530][105692] Updated weights for policy 0, policy_version 300316 (0.0008) [2023-12-26 17:33:01,558][105620] Updated weights for policy 1, policy_version 300598 (0.0009) [2023-12-26 17:33:01,590][105692] Updated weights for policy 0, policy_version 300326 (0.0010) [2023-12-26 17:33:01,606][105620] Updated weights for policy 1, policy_version 300608 (0.0006) [2023-12-26 17:33:01,636][105586] KL-divergence is very high: 104.0136 [2023-12-26 17:33:02,324][105620] Updated weights for policy 1, policy_version 300618 (0.0008) [2023-12-26 17:33:02,371][105692] Updated weights for policy 0, policy_version 300336 (0.0007) [2023-12-26 17:33:02,382][105620] Updated weights for policy 1, policy_version 300628 (0.0008) [2023-12-26 17:33:02,433][105692] Updated weights for policy 0, policy_version 300346 (0.0007) [2023-12-26 17:33:02,438][105620] Updated weights for policy 1, policy_version 300638 (0.0008) [2023-12-26 17:33:02,495][105692] Updated weights for policy 0, policy_version 300356 (0.0010) [2023-12-26 17:33:02,502][105620] Updated weights for policy 1, policy_version 300648 (0.0007) [2023-12-26 17:33:03,245][105692] Updated weights for policy 0, policy_version 300366 (0.0008) [2023-12-26 17:33:03,247][105620] Updated weights for policy 1, policy_version 300658 (0.0008) [2023-12-26 17:33:03,298][105692] Updated weights for policy 0, policy_version 300376 (0.0007) [2023-12-26 17:33:03,303][105620] Updated weights for policy 1, policy_version 300668 (0.0006) [2023-12-26 17:33:03,355][105620] Updated weights for policy 1, policy_version 300678 (0.0007) [2023-12-26 17:33:03,357][105692] Updated weights for policy 0, policy_version 300386 (0.0005) [2023-12-26 17:33:04,039][105620] Updated weights for policy 1, policy_version 300688 (0.0008) [2023-12-26 17:33:04,101][105620] Updated weights for policy 1, policy_version 300698 (0.0009) [2023-12-26 17:33:04,133][105692] Updated weights for policy 0, policy_version 300396 (0.0009) [2023-12-26 17:33:04,163][105620] Updated weights for policy 1, policy_version 300708 (0.0006) [2023-12-26 17:33:04,192][105692] Updated weights for policy 0, policy_version 300406 (0.0010) [2023-12-26 17:33:04,247][105692] Updated weights for policy 0, policy_version 300416 (0.0009) [2023-12-26 17:33:04,813][105620] Updated weights for policy 1, policy_version 300718 (0.0007) [2023-12-26 17:33:04,842][105586] KL-divergence is very high: 136.0293 [2023-12-26 17:33:04,884][105620] Updated weights for policy 1, policy_version 300728 (0.0005) [2023-12-26 17:33:04,900][105586] KL-divergence is very high: 181.3203 [2023-12-26 17:33:04,954][105620] Updated weights for policy 1, policy_version 300738 (0.0005) [2023-12-26 17:33:04,956][105586] KL-divergence is very high: 152.9331 [2023-12-26 17:33:05,104][105692] Updated weights for policy 0, policy_version 300426 (0.0008) [2023-12-26 17:33:05,163][105692] Updated weights for policy 0, policy_version 300436 (0.0005) [2023-12-26 17:33:05,211][105692] Updated weights for policy 0, policy_version 300446 (0.0006) [2023-12-26 17:33:05,254][105692] Updated weights for policy 0, policy_version 300456 (0.0008) [2023-12-26 17:33:05,574][105620] Updated weights for policy 1, policy_version 300748 (0.0007) [2023-12-26 17:33:05,625][105620] Updated weights for policy 1, policy_version 300758 (0.0010) [2023-12-26 17:33:05,676][105620] Updated weights for policy 1, policy_version 300768 (0.0010) [2023-12-26 17:33:05,925][105692] Updated weights for policy 0, policy_version 300466 (0.0008) [2023-12-26 17:33:05,970][105692] Updated weights for policy 0, policy_version 300476 (0.0008) [2023-12-26 17:33:06,014][105692] Updated weights for policy 0, policy_version 300486 (0.0008) [2023-12-26 17:33:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 153944064. Throughput: 0: 9459.4, 1: 9865.8. Samples: 153927708. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-26 17:33:06,063][104569] Avg episode reward: [(0, '9265.592'), (1, '8174.490')] [2023-12-26 17:33:06,482][105620] Updated weights for policy 1, policy_version 300778 (0.0010) [2023-12-26 17:33:06,551][105620] Updated weights for policy 1, policy_version 300788 (0.0011) [2023-12-26 17:33:06,611][105620] Updated weights for policy 1, policy_version 300798 (0.0010) [2023-12-26 17:33:06,672][105620] Updated weights for policy 1, policy_version 300808 (0.0009) [2023-12-26 17:33:06,810][105692] Updated weights for policy 0, policy_version 300496 (0.0010) [2023-12-26 17:33:06,861][105692] Updated weights for policy 0, policy_version 300506 (0.0010) [2023-12-26 17:33:06,912][105692] Updated weights for policy 0, policy_version 300516 (0.0010) [2023-12-26 17:33:07,342][105620] Updated weights for policy 1, policy_version 300818 (0.0006) [2023-12-26 17:33:07,392][105620] Updated weights for policy 1, policy_version 300828 (0.0005) [2023-12-26 17:33:07,443][105620] Updated weights for policy 1, policy_version 300838 (0.0006) [2023-12-26 17:33:07,629][105692] Updated weights for policy 0, policy_version 300526 (0.0007) [2023-12-26 17:33:07,684][105692] Updated weights for policy 0, policy_version 300536 (0.0006) [2023-12-26 17:33:07,735][105692] Updated weights for policy 0, policy_version 300546 (0.0005) [2023-12-26 17:33:08,125][105620] Updated weights for policy 1, policy_version 300848 (0.0008) [2023-12-26 17:33:08,178][105620] Updated weights for policy 1, policy_version 300858 (0.0009) [2023-12-26 17:33:08,231][105620] Updated weights for policy 1, policy_version 300868 (0.0009) [2023-12-26 17:33:08,325][105692] Updated weights for policy 0, policy_version 300556 (0.0007) [2023-12-26 17:33:08,384][105692] Updated weights for policy 0, policy_version 300566 (0.0008) [2023-12-26 17:33:08,442][105692] Updated weights for policy 0, policy_version 300576 (0.0008) [2023-12-26 17:33:09,009][105620] Updated weights for policy 1, policy_version 300878 (0.0009) [2023-12-26 17:33:09,067][105620] Updated weights for policy 1, policy_version 300888 (0.0009) [2023-12-26 17:33:09,129][105620] Updated weights for policy 1, policy_version 300898 (0.0008) [2023-12-26 17:33:09,188][105692] Updated weights for policy 0, policy_version 300586 (0.0008) [2023-12-26 17:33:09,248][105692] Updated weights for policy 0, policy_version 300596 (0.0007) [2023-12-26 17:33:09,311][105692] Updated weights for policy 0, policy_version 300606 (0.0008) [2023-12-26 17:33:09,376][105692] Updated weights for policy 0, policy_version 300616 (0.0008) [2023-12-26 17:33:09,970][105620] Updated weights for policy 1, policy_version 300908 (0.0008) [2023-12-26 17:33:10,025][105620] Updated weights for policy 1, policy_version 300918 (0.0007) [2023-12-26 17:33:10,092][105620] Updated weights for policy 1, policy_version 300928 (0.0006) [2023-12-26 17:33:10,121][105692] Updated weights for policy 0, policy_version 300626 (0.0010) [2023-12-26 17:33:10,183][105692] Updated weights for policy 0, policy_version 300636 (0.0008) [2023-12-26 17:33:10,246][105692] Updated weights for policy 0, policy_version 300646 (0.0009) [2023-12-26 17:33:10,793][105620] Updated weights for policy 1, policy_version 300938 (0.0008) [2023-12-26 17:33:10,847][105620] Updated weights for policy 1, policy_version 300948 (0.0009) [2023-12-26 17:33:10,904][105620] Updated weights for policy 1, policy_version 300958 (0.0009) [2023-12-26 17:33:10,935][105692] Updated weights for policy 0, policy_version 300656 (0.0007) [2023-12-26 17:33:10,960][105620] Updated weights for policy 1, policy_version 300968 (0.0006) [2023-12-26 17:33:10,981][105692] Updated weights for policy 0, policy_version 300666 (0.0008) [2023-12-26 17:33:11,048][105692] Updated weights for policy 0, policy_version 300676 (0.0008) [2023-12-26 17:33:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 154034176. Throughput: 0: 9464.7, 1: 9893.4. Samples: 154043180. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-26 17:33:11,063][104569] Avg episode reward: [(0, '9265.596'), (1, '8167.261')] [2023-12-26 17:33:11,785][105620] Updated weights for policy 1, policy_version 300978 (0.0008) [2023-12-26 17:33:11,785][105692] Updated weights for policy 0, policy_version 300686 (0.0009) [2023-12-26 17:33:11,835][105692] Updated weights for policy 0, policy_version 300696 (0.0006) [2023-12-26 17:33:11,849][105620] Updated weights for policy 1, policy_version 300988 (0.0008) [2023-12-26 17:33:11,893][105692] Updated weights for policy 0, policy_version 300706 (0.0006) [2023-12-26 17:33:11,911][105620] Updated weights for policy 1, policy_version 300998 (0.0007) [2023-12-26 17:33:12,630][105620] Updated weights for policy 1, policy_version 301008 (0.0009) [2023-12-26 17:33:12,684][105692] Updated weights for policy 0, policy_version 300716 (0.0006) [2023-12-26 17:33:12,692][105620] Updated weights for policy 1, policy_version 301018 (0.0008) [2023-12-26 17:33:12,748][105620] Updated weights for policy 1, policy_version 301028 (0.0008) [2023-12-26 17:33:12,750][105692] Updated weights for policy 0, policy_version 300726 (0.0008) [2023-12-26 17:33:12,803][105692] Updated weights for policy 0, policy_version 300736 (0.0009) [2023-12-26 17:33:13,443][105692] Updated weights for policy 0, policy_version 300746 (0.0009) [2023-12-26 17:33:13,497][105692] Updated weights for policy 0, policy_version 300756 (0.0009) [2023-12-26 17:33:13,557][105692] Updated weights for policy 0, policy_version 300766 (0.0007) [2023-12-26 17:33:13,560][105620] Updated weights for policy 1, policy_version 301038 (0.0008) [2023-12-26 17:33:13,616][105620] Updated weights for policy 1, policy_version 301048 (0.0009) [2023-12-26 17:33:13,619][105692] Updated weights for policy 0, policy_version 300776 (0.0005) [2023-12-26 17:33:13,667][105620] Updated weights for policy 1, policy_version 301058 (0.0009) [2023-12-26 17:33:14,248][105692] Updated weights for policy 0, policy_version 300786 (0.0009) [2023-12-26 17:33:14,312][105692] Updated weights for policy 0, policy_version 300796 (0.0009) [2023-12-26 17:33:14,363][105692] Updated weights for policy 0, policy_version 300806 (0.0009) [2023-12-26 17:33:14,432][105620] Updated weights for policy 1, policy_version 301068 (0.0010) [2023-12-26 17:33:14,482][105620] Updated weights for policy 1, policy_version 301078 (0.0009) [2023-12-26 17:33:14,529][105620] Updated weights for policy 1, policy_version 301088 (0.0009) [2023-12-26 17:33:15,079][105692] Updated weights for policy 0, policy_version 300816 (0.0009) [2023-12-26 17:33:15,131][105692] Updated weights for policy 0, policy_version 300826 (0.0009) [2023-12-26 17:33:15,194][105692] Updated weights for policy 0, policy_version 300836 (0.0009) [2023-12-26 17:33:15,308][105620] Updated weights for policy 1, policy_version 301098 (0.0009) [2023-12-26 17:33:15,370][105620] Updated weights for policy 1, policy_version 301108 (0.0009) [2023-12-26 17:33:15,440][105620] Updated weights for policy 1, policy_version 301118 (0.0010) [2023-12-26 17:33:15,501][105620] Updated weights for policy 1, policy_version 301128 (0.0009) [2023-12-26 17:33:15,908][105692] Updated weights for policy 0, policy_version 300846 (0.0009) [2023-12-26 17:33:15,960][105692] Updated weights for policy 0, policy_version 300856 (0.0009) [2023-12-26 17:33:16,006][105692] Updated weights for policy 0, policy_version 300866 (0.0008) [2023-12-26 17:33:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.1, 300 sec: 19521.9). Total num frames: 154132480. Throughput: 0: 9385.8, 1: 9834.2. Samples: 154099152. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-26 17:33:16,063][104569] Avg episode reward: [(0, '9355.815'), (1, '8075.351')] [2023-12-26 17:33:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000300872_77037568.pth... [2023-12-26 17:33:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000301128_77094912.pth... [2023-12-26 17:33:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000300040_76816384.pth [2023-12-26 17:33:16,081][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000299752_76750848.pth [2023-12-26 17:33:16,269][105620] Updated weights for policy 1, policy_version 301138 (0.0009) [2023-12-26 17:33:16,330][105620] Updated weights for policy 1, policy_version 301148 (0.0009) [2023-12-26 17:33:16,377][105586] KL-divergence is very high: 126.8016 [2023-12-26 17:33:16,389][105620] Updated weights for policy 1, policy_version 301158 (0.0009) [2023-12-26 17:33:16,751][105692] Updated weights for policy 0, policy_version 300876 (0.0009) [2023-12-26 17:33:16,797][105692] Updated weights for policy 0, policy_version 300886 (0.0008) [2023-12-26 17:33:16,839][105692] Updated weights for policy 0, policy_version 300896 (0.0006) [2023-12-26 17:33:17,147][105586] KL-divergence is very high: 117.1662 [2023-12-26 17:33:17,200][105586] KL-divergence is very high: 107.5729 [2023-12-26 17:33:17,201][105620] Updated weights for policy 1, policy_version 301168 (0.0009) [2023-12-26 17:33:17,246][105586] KL-divergence is very high: 103.6196 [2023-12-26 17:33:17,262][105620] Updated weights for policy 1, policy_version 301178 (0.0010) [2023-12-26 17:33:17,300][105586] KL-divergence is very high: 101.0146 [2023-12-26 17:33:17,317][105620] Updated weights for policy 1, policy_version 301188 (0.0009) [2023-12-26 17:33:17,452][105692] Updated weights for policy 0, policy_version 300906 (0.0006) [2023-12-26 17:33:17,510][105692] Updated weights for policy 0, policy_version 300916 (0.0010) [2023-12-26 17:33:17,572][105692] Updated weights for policy 0, policy_version 300926 (0.0010) [2023-12-26 17:33:17,632][105692] Updated weights for policy 0, policy_version 300936 (0.0011) [2023-12-26 17:33:18,098][105620] Updated weights for policy 1, policy_version 301198 (0.0009) [2023-12-26 17:33:18,157][105620] Updated weights for policy 1, policy_version 301208 (0.0011) [2023-12-26 17:33:18,213][105620] Updated weights for policy 1, policy_version 301218 (0.0011) [2023-12-26 17:33:18,275][105692] Updated weights for policy 0, policy_version 300946 (0.0005) [2023-12-26 17:33:18,328][105692] Updated weights for policy 0, policy_version 300956 (0.0006) [2023-12-26 17:33:18,391][105692] Updated weights for policy 0, policy_version 300966 (0.0007) [2023-12-26 17:33:18,868][105620] Updated weights for policy 1, policy_version 301228 (0.0009) [2023-12-26 17:33:18,917][105620] Updated weights for policy 1, policy_version 301238 (0.0006) [2023-12-26 17:33:18,969][105620] Updated weights for policy 1, policy_version 301248 (0.0006) [2023-12-26 17:33:19,000][105692] Updated weights for policy 0, policy_version 300976 (0.0010) [2023-12-26 17:33:19,059][105692] Updated weights for policy 0, policy_version 300986 (0.0011) [2023-12-26 17:33:19,124][105692] Updated weights for policy 0, policy_version 300996 (0.0011) [2023-12-26 17:33:19,697][105620] Updated weights for policy 1, policy_version 301258 (0.0010) [2023-12-26 17:33:19,746][105620] Updated weights for policy 1, policy_version 301268 (0.0008) [2023-12-26 17:33:19,795][105620] Updated weights for policy 1, policy_version 301278 (0.0008) [2023-12-26 17:33:19,853][105620] Updated weights for policy 1, policy_version 301288 (0.0008) [2023-12-26 17:33:19,875][105692] Updated weights for policy 0, policy_version 301006 (0.0009) [2023-12-26 17:33:19,940][105692] Updated weights for policy 0, policy_version 301016 (0.0009) [2023-12-26 17:33:20,009][105692] Updated weights for policy 0, policy_version 301026 (0.0011) [2023-12-26 17:33:20,674][105620] Updated weights for policy 1, policy_version 301298 (0.0010) [2023-12-26 17:33:20,730][105620] Updated weights for policy 1, policy_version 301308 (0.0009) [2023-12-26 17:33:20,752][105692] Updated weights for policy 0, policy_version 301036 (0.0010) [2023-12-26 17:33:20,787][105620] Updated weights for policy 1, policy_version 301318 (0.0006) [2023-12-26 17:33:20,810][105692] Updated weights for policy 0, policy_version 301046 (0.0007) [2023-12-26 17:33:20,862][105692] Updated weights for policy 0, policy_version 301056 (0.0009) [2023-12-26 17:33:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 154230784. Throughput: 0: 9545.2, 1: 9667.2. Samples: 154216800. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-26 17:33:21,062][104569] Avg episode reward: [(0, '9355.891'), (1, '8077.791')] [2023-12-26 17:33:21,570][105620] Updated weights for policy 1, policy_version 301328 (0.0008) [2023-12-26 17:33:21,634][105620] Updated weights for policy 1, policy_version 301338 (0.0009) [2023-12-26 17:33:21,659][105692] Updated weights for policy 0, policy_version 301066 (0.0009) [2023-12-26 17:33:21,699][105620] Updated weights for policy 1, policy_version 301348 (0.0008) [2023-12-26 17:33:21,718][105692] Updated weights for policy 0, policy_version 301076 (0.0007) [2023-12-26 17:33:21,783][105692] Updated weights for policy 0, policy_version 301086 (0.0008) [2023-12-26 17:33:21,843][105692] Updated weights for policy 0, policy_version 301096 (0.0010) [2023-12-26 17:33:22,415][105620] Updated weights for policy 1, policy_version 301358 (0.0008) [2023-12-26 17:33:22,480][105620] Updated weights for policy 1, policy_version 301368 (0.0008) [2023-12-26 17:33:22,531][105692] Updated weights for policy 0, policy_version 301106 (0.0008) [2023-12-26 17:33:22,541][105620] Updated weights for policy 1, policy_version 301378 (0.0008) [2023-12-26 17:33:22,583][105692] Updated weights for policy 0, policy_version 301116 (0.0008) [2023-12-26 17:33:22,643][105692] Updated weights for policy 0, policy_version 301126 (0.0009) [2023-12-26 17:33:23,222][105620] Updated weights for policy 1, policy_version 301388 (0.0009) [2023-12-26 17:33:23,282][105620] Updated weights for policy 1, policy_version 301398 (0.0008) [2023-12-26 17:33:23,341][105620] Updated weights for policy 1, policy_version 301408 (0.0008) [2023-12-26 17:33:23,416][105692] Updated weights for policy 0, policy_version 301136 (0.0010) [2023-12-26 17:33:23,472][105692] Updated weights for policy 0, policy_version 301146 (0.0010) [2023-12-26 17:33:23,534][105692] Updated weights for policy 0, policy_version 301156 (0.0010) [2023-12-26 17:33:24,100][105620] Updated weights for policy 1, policy_version 301418 (0.0008) [2023-12-26 17:33:24,162][105620] Updated weights for policy 1, policy_version 301428 (0.0008) [2023-12-26 17:33:24,229][105620] Updated weights for policy 1, policy_version 301438 (0.0009) [2023-12-26 17:33:24,255][105692] Updated weights for policy 0, policy_version 301166 (0.0009) [2023-12-26 17:33:24,284][105620] Updated weights for policy 1, policy_version 301448 (0.0006) [2023-12-26 17:33:24,321][105692] Updated weights for policy 0, policy_version 301176 (0.0008) [2023-12-26 17:33:24,373][105692] Updated weights for policy 0, policy_version 301186 (0.0005) [2023-12-26 17:33:24,962][105692] Updated weights for policy 0, policy_version 301196 (0.0006) [2023-12-26 17:33:25,021][105692] Updated weights for policy 0, policy_version 301206 (0.0005) [2023-12-26 17:33:25,079][105692] Updated weights for policy 0, policy_version 301216 (0.0009) [2023-12-26 17:33:25,102][105620] Updated weights for policy 1, policy_version 301458 (0.0006) [2023-12-26 17:33:25,158][105620] Updated weights for policy 1, policy_version 301468 (0.0007) [2023-12-26 17:33:25,218][105620] Updated weights for policy 1, policy_version 301478 (0.0009) [2023-12-26 17:33:25,747][105692] Updated weights for policy 0, policy_version 301226 (0.0010) [2023-12-26 17:33:25,802][105692] Updated weights for policy 0, policy_version 301236 (0.0005) [2023-12-26 17:33:25,867][105692] Updated weights for policy 0, policy_version 301246 (0.0006) [2023-12-26 17:33:25,924][105692] Updated weights for policy 0, policy_version 301256 (0.0006) [2023-12-26 17:33:26,025][105620] Updated weights for policy 1, policy_version 301489 (0.0010) [2023-12-26 17:33:26,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 154320896. Throughput: 0: 9640.5, 1: 9478.5. Samples: 154329648. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-26 17:33:26,062][104569] Avg episode reward: [(0, '9355.937'), (1, '7897.648')] [2023-12-26 17:33:26,074][105620] Updated weights for policy 1, policy_version 301499 (0.0009) [2023-12-26 17:33:26,125][105620] Updated weights for policy 1, policy_version 301509 (0.0009) [2023-12-26 17:33:26,561][105692] Updated weights for policy 0, policy_version 301266 (0.0009) [2023-12-26 17:33:26,618][105692] Updated weights for policy 0, policy_version 301276 (0.0009) [2023-12-26 17:33:26,679][105692] Updated weights for policy 0, policy_version 301286 (0.0009) [2023-12-26 17:33:26,938][105620] Updated weights for policy 1, policy_version 301519 (0.0010) [2023-12-26 17:33:27,003][105620] Updated weights for policy 1, policy_version 301529 (0.0010) [2023-12-26 17:33:27,068][105586] KL-divergence is very high: 114.6259 [2023-12-26 17:33:27,078][105586] KL-divergence is very high: 114.3491 [2023-12-26 17:33:27,078][105620] Updated weights for policy 1, policy_version 301539 (0.0010) [2023-12-26 17:33:27,104][105586] KL-divergence is very high: 132.5284 [2023-12-26 17:33:27,298][105692] Updated weights for policy 0, policy_version 301296 (0.0011) [2023-12-26 17:33:27,356][105692] Updated weights for policy 0, policy_version 301306 (0.0010) [2023-12-26 17:33:27,414][105692] Updated weights for policy 0, policy_version 301316 (0.0010) [2023-12-26 17:33:27,878][105620] Updated weights for policy 1, policy_version 301549 (0.0010) [2023-12-26 17:33:27,936][105620] Updated weights for policy 1, policy_version 301559 (0.0007) [2023-12-26 17:33:27,942][105586] KL-divergence is very high: 121.3648 [2023-12-26 17:33:27,987][105586] KL-divergence is very high: 100.3549 [2023-12-26 17:33:27,996][105620] Updated weights for policy 1, policy_version 301569 (0.0009) [2023-12-26 17:33:28,044][105692] Updated weights for policy 0, policy_version 301326 (0.0010) [2023-12-26 17:33:28,099][105692] Updated weights for policy 0, policy_version 301336 (0.0010) [2023-12-26 17:33:28,151][105692] Updated weights for policy 0, policy_version 301346 (0.0010) [2023-12-26 17:33:28,645][105620] Updated weights for policy 1, policy_version 301579 (0.0008) [2023-12-26 17:33:28,706][105620] Updated weights for policy 1, policy_version 301589 (0.0008) [2023-12-26 17:33:28,766][105620] Updated weights for policy 1, policy_version 301599 (0.0008) [2023-12-26 17:33:28,918][105692] Updated weights for policy 0, policy_version 301356 (0.0010) [2023-12-26 17:33:28,962][105692] Updated weights for policy 0, policy_version 301366 (0.0010) [2023-12-26 17:33:29,009][105692] Updated weights for policy 0, policy_version 301376 (0.0010) [2023-12-26 17:33:29,556][105620] Updated weights for policy 1, policy_version 301609 (0.0008) [2023-12-26 17:33:29,617][105620] Updated weights for policy 1, policy_version 301619 (0.0008) [2023-12-26 17:33:29,682][105620] Updated weights for policy 1, policy_version 301629 (0.0009) [2023-12-26 17:33:29,690][105692] Updated weights for policy 0, policy_version 301386 (0.0009) [2023-12-26 17:33:29,738][105692] Updated weights for policy 0, policy_version 301396 (0.0005) [2023-12-26 17:33:29,741][105620] Updated weights for policy 1, policy_version 301639 (0.0008) [2023-12-26 17:33:29,754][105585] KL-divergence is very high: 235.7802 [2023-12-26 17:33:29,791][105692] Updated weights for policy 0, policy_version 301406 (0.0008) [2023-12-26 17:33:29,797][105585] KL-divergence is very high: 281.5329 [2023-12-26 17:33:29,847][105585] KL-divergence is very high: 174.3184 [2023-12-26 17:33:29,853][105692] Updated weights for policy 0, policy_version 301416 (0.0011) [2023-12-26 17:33:30,523][105692] Updated weights for policy 0, policy_version 301426 (0.0008) [2023-12-26 17:33:30,572][105692] Updated weights for policy 0, policy_version 301436 (0.0006) [2023-12-26 17:33:30,589][105620] Updated weights for policy 1, policy_version 301649 (0.0008) [2023-12-26 17:33:30,624][105692] Updated weights for policy 0, policy_version 301446 (0.0005) [2023-12-26 17:33:30,638][105620] Updated weights for policy 1, policy_version 301659 (0.0008) [2023-12-26 17:33:30,691][105620] Updated weights for policy 1, policy_version 301670 (0.0010) [2023-12-26 17:33:31,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 154419200. Throughput: 0: 9712.6, 1: 9448.4. Samples: 154388732. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-26 17:33:31,063][104569] Avg episode reward: [(0, '9179.508'), (1, '7436.176')] [2023-12-26 17:33:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000301448_77185024.pth... [2023-12-26 17:33:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000301672_77234176.pth... [2023-12-26 17:33:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000300296_76890112.pth [2023-12-26 17:33:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000300584_76955648.pth [2023-12-26 17:33:31,293][105692] Updated weights for policy 0, policy_version 301456 (0.0008) [2023-12-26 17:33:31,355][105692] Updated weights for policy 0, policy_version 301466 (0.0008) [2023-12-26 17:33:31,420][105692] Updated weights for policy 0, policy_version 301476 (0.0009) [2023-12-26 17:33:31,459][105620] Updated weights for policy 1, policy_version 301680 (0.0006) [2023-12-26 17:33:31,514][105620] Updated weights for policy 1, policy_version 301690 (0.0005) [2023-12-26 17:33:31,563][105620] Updated weights for policy 1, policy_version 301700 (0.0005) [2023-12-26 17:33:32,231][105620] Updated weights for policy 1, policy_version 301710 (0.0008) [2023-12-26 17:33:32,233][105692] Updated weights for policy 0, policy_version 301486 (0.0009) [2023-12-26 17:33:32,290][105620] Updated weights for policy 1, policy_version 301720 (0.0011) [2023-12-26 17:33:32,293][105692] Updated weights for policy 0, policy_version 301496 (0.0006) [2023-12-26 17:33:32,345][105692] Updated weights for policy 0, policy_version 301506 (0.0008) [2023-12-26 17:33:32,349][105620] Updated weights for policy 1, policy_version 301730 (0.0009) [2023-12-26 17:33:33,041][105692] Updated weights for policy 0, policy_version 301516 (0.0009) [2023-12-26 17:33:33,050][105620] Updated weights for policy 1, policy_version 301740 (0.0008) [2023-12-26 17:33:33,093][105692] Updated weights for policy 0, policy_version 301526 (0.0008) [2023-12-26 17:33:33,112][105620] Updated weights for policy 1, policy_version 301750 (0.0006) [2023-12-26 17:33:33,149][105692] Updated weights for policy 0, policy_version 301536 (0.0008) [2023-12-26 17:33:33,162][105620] Updated weights for policy 1, policy_version 301760 (0.0006) [2023-12-26 17:33:33,720][105620] Updated weights for policy 1, policy_version 301770 (0.0006) [2023-12-26 17:33:33,767][105620] Updated weights for policy 1, policy_version 301780 (0.0010) [2023-12-26 17:33:33,818][105620] Updated weights for policy 1, policy_version 301790 (0.0010) [2023-12-26 17:33:33,883][105620] Updated weights for policy 1, policy_version 301800 (0.0010) [2023-12-26 17:33:33,903][105692] Updated weights for policy 0, policy_version 301546 (0.0009) [2023-12-26 17:33:33,969][105692] Updated weights for policy 0, policy_version 301556 (0.0005) [2023-12-26 17:33:34,030][105692] Updated weights for policy 0, policy_version 301566 (0.0005) [2023-12-26 17:33:34,088][105692] Updated weights for policy 0, policy_version 301576 (0.0005) [2023-12-26 17:33:34,594][105620] Updated weights for policy 1, policy_version 301810 (0.0008) [2023-12-26 17:33:34,644][105692] Updated weights for policy 0, policy_version 301586 (0.0008) [2023-12-26 17:33:34,657][105620] Updated weights for policy 1, policy_version 301820 (0.0008) [2023-12-26 17:33:34,700][105692] Updated weights for policy 0, policy_version 301596 (0.0007) [2023-12-26 17:33:34,715][105620] Updated weights for policy 1, policy_version 301830 (0.0008) [2023-12-26 17:33:34,760][105692] Updated weights for policy 0, policy_version 301606 (0.0009) [2023-12-26 17:33:35,379][105620] Updated weights for policy 1, policy_version 301840 (0.0008) [2023-12-26 17:33:35,437][105692] Updated weights for policy 0, policy_version 301616 (0.0008) [2023-12-26 17:33:35,439][105620] Updated weights for policy 1, policy_version 301850 (0.0008) [2023-12-26 17:33:35,488][105620] Updated weights for policy 1, policy_version 301860 (0.0007) [2023-12-26 17:33:35,502][105692] Updated weights for policy 0, policy_version 301626 (0.0005) [2023-12-26 17:33:35,565][105692] Updated weights for policy 0, policy_version 301636 (0.0006) [2023-12-26 17:33:36,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19251.1, 300 sec: 19466.4). Total num frames: 154517504. Throughput: 0: 9835.7, 1: 9379.8. Samples: 154507396. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-26 17:33:36,063][104569] Avg episode reward: [(0, '8999.620'), (1, '7967.053')] [2023-12-26 17:33:36,115][105692] Updated weights for policy 0, policy_version 301646 (0.0007) [2023-12-26 17:33:36,125][105620] Updated weights for policy 1, policy_version 301870 (0.0007) [2023-12-26 17:33:36,175][105692] Updated weights for policy 0, policy_version 301656 (0.0007) [2023-12-26 17:33:36,189][105620] Updated weights for policy 1, policy_version 301880 (0.0007) [2023-12-26 17:33:36,229][105692] Updated weights for policy 0, policy_version 301666 (0.0006) [2023-12-26 17:33:36,251][105620] Updated weights for policy 1, policy_version 301890 (0.0008) [2023-12-26 17:33:36,898][105620] Updated weights for policy 1, policy_version 301900 (0.0008) [2023-12-26 17:33:36,945][105620] Updated weights for policy 1, policy_version 301910 (0.0009) [2023-12-26 17:33:36,997][105620] Updated weights for policy 1, policy_version 301920 (0.0009) [2023-12-26 17:33:36,997][105692] Updated weights for policy 0, policy_version 301676 (0.0008) [2023-12-26 17:33:37,048][105692] Updated weights for policy 0, policy_version 301686 (0.0010) [2023-12-26 17:33:37,104][105692] Updated weights for policy 0, policy_version 301696 (0.0011) [2023-12-26 17:33:37,764][105620] Updated weights for policy 1, policy_version 301930 (0.0006) [2023-12-26 17:33:37,816][105620] Updated weights for policy 1, policy_version 301940 (0.0005) [2023-12-26 17:33:37,876][105692] Updated weights for policy 0, policy_version 301706 (0.0007) [2023-12-26 17:33:37,878][105620] Updated weights for policy 1, policy_version 301950 (0.0006) [2023-12-26 17:33:37,931][105620] Updated weights for policy 1, policy_version 301960 (0.0007) [2023-12-26 17:33:37,934][105692] Updated weights for policy 0, policy_version 301716 (0.0010) [2023-12-26 17:33:37,992][105692] Updated weights for policy 0, policy_version 301726 (0.0010) [2023-12-26 17:33:38,050][105692] Updated weights for policy 0, policy_version 301736 (0.0010) [2023-12-26 17:33:38,641][105620] Updated weights for policy 1, policy_version 301970 (0.0008) [2023-12-26 17:33:38,693][105620] Updated weights for policy 1, policy_version 301980 (0.0007) [2023-12-26 17:33:38,747][105620] Updated weights for policy 1, policy_version 301990 (0.0009) [2023-12-26 17:33:38,790][105692] Updated weights for policy 0, policy_version 301746 (0.0011) [2023-12-26 17:33:38,853][105692] Updated weights for policy 0, policy_version 301756 (0.0011) [2023-12-26 17:33:38,921][105692] Updated weights for policy 0, policy_version 301766 (0.0009) [2023-12-26 17:33:39,566][105620] Updated weights for policy 1, policy_version 302000 (0.0009) [2023-12-26 17:33:39,597][105692] Updated weights for policy 0, policy_version 301776 (0.0010) [2023-12-26 17:33:39,620][105620] Updated weights for policy 1, policy_version 302010 (0.0006) [2023-12-26 17:33:39,657][105692] Updated weights for policy 0, policy_version 301786 (0.0011) [2023-12-26 17:33:39,679][105620] Updated weights for policy 1, policy_version 302020 (0.0005) [2023-12-26 17:33:39,712][105692] Updated weights for policy 0, policy_version 301796 (0.0010) [2023-12-26 17:33:40,341][105620] Updated weights for policy 1, policy_version 302030 (0.0006) [2023-12-26 17:33:40,401][105620] Updated weights for policy 1, policy_version 302040 (0.0006) [2023-12-26 17:33:40,463][105692] Updated weights for policy 0, policy_version 301806 (0.0010) [2023-12-26 17:33:40,464][105620] Updated weights for policy 1, policy_version 302050 (0.0007) [2023-12-26 17:33:40,520][105692] Updated weights for policy 0, policy_version 301816 (0.0010) [2023-12-26 17:33:40,581][105692] Updated weights for policy 0, policy_version 301826 (0.0010) [2023-12-26 17:33:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 154615808. Throughput: 0: 9811.8, 1: 9489.3. Samples: 154627308. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-26 17:33:41,062][104569] Avg episode reward: [(0, '9087.687'), (1, '7692.917')] [2023-12-26 17:33:41,094][105620] Updated weights for policy 1, policy_version 302060 (0.0011) [2023-12-26 17:33:41,159][105620] Updated weights for policy 1, policy_version 302070 (0.0011) [2023-12-26 17:33:41,222][105620] Updated weights for policy 1, policy_version 302080 (0.0010) [2023-12-26 17:33:41,289][105692] Updated weights for policy 0, policy_version 301836 (0.0010) [2023-12-26 17:33:41,356][105692] Updated weights for policy 0, policy_version 301846 (0.0008) [2023-12-26 17:33:41,423][105692] Updated weights for policy 0, policy_version 301856 (0.0008) [2023-12-26 17:33:41,927][105620] Updated weights for policy 1, policy_version 302090 (0.0010) [2023-12-26 17:33:41,985][105620] Updated weights for policy 1, policy_version 302100 (0.0006) [2023-12-26 17:33:42,036][105620] Updated weights for policy 1, policy_version 302110 (0.0005) [2023-12-26 17:33:42,084][105620] Updated weights for policy 1, policy_version 302120 (0.0005) [2023-12-26 17:33:42,183][105692] Updated weights for policy 0, policy_version 301866 (0.0007) [2023-12-26 17:33:42,237][105692] Updated weights for policy 0, policy_version 301876 (0.0008) [2023-12-26 17:33:42,306][105692] Updated weights for policy 0, policy_version 301886 (0.0009) [2023-12-26 17:33:42,363][105692] Updated weights for policy 0, policy_version 301896 (0.0009) [2023-12-26 17:33:42,773][105620] Updated weights for policy 1, policy_version 302130 (0.0010) [2023-12-26 17:33:42,840][105620] Updated weights for policy 1, policy_version 302140 (0.0010) [2023-12-26 17:33:42,902][105620] Updated weights for policy 1, policy_version 302150 (0.0009) [2023-12-26 17:33:43,015][105692] Updated weights for policy 0, policy_version 301906 (0.0009) [2023-12-26 17:33:43,063][105692] Updated weights for policy 0, policy_version 301916 (0.0009) [2023-12-26 17:33:43,125][105692] Updated weights for policy 0, policy_version 301926 (0.0009) [2023-12-26 17:33:43,660][105620] Updated weights for policy 1, policy_version 302160 (0.0009) [2023-12-26 17:33:43,721][105620] Updated weights for policy 1, policy_version 302170 (0.0009) [2023-12-26 17:33:43,779][105620] Updated weights for policy 1, policy_version 302180 (0.0009) [2023-12-26 17:33:43,890][105692] Updated weights for policy 0, policy_version 301936 (0.0009) [2023-12-26 17:33:43,936][105692] Updated weights for policy 0, policy_version 301946 (0.0008) [2023-12-26 17:33:43,994][105692] Updated weights for policy 0, policy_version 301956 (0.0009) [2023-12-26 17:33:44,516][105620] Updated weights for policy 1, policy_version 302190 (0.0009) [2023-12-26 17:33:44,563][105620] Updated weights for policy 1, policy_version 302200 (0.0009) [2023-12-26 17:33:44,611][105620] Updated weights for policy 1, policy_version 302210 (0.0009) [2023-12-26 17:33:44,772][105692] Updated weights for policy 0, policy_version 301966 (0.0008) [2023-12-26 17:33:44,836][105692] Updated weights for policy 0, policy_version 301976 (0.0008) [2023-12-26 17:33:44,888][105692] Updated weights for policy 0, policy_version 301986 (0.0009) [2023-12-26 17:33:45,418][105620] Updated weights for policy 1, policy_version 302220 (0.0009) [2023-12-26 17:33:45,465][105620] Updated weights for policy 1, policy_version 302230 (0.0008) [2023-12-26 17:33:45,520][105620] Updated weights for policy 1, policy_version 302240 (0.0009) [2023-12-26 17:33:45,606][105692] Updated weights for policy 0, policy_version 301996 (0.0009) [2023-12-26 17:33:45,659][105692] Updated weights for policy 0, policy_version 302006 (0.0009) [2023-12-26 17:33:45,714][105692] Updated weights for policy 0, policy_version 302016 (0.0009) [2023-12-26 17:33:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.9, 300 sec: 19494.2). Total num frames: 154714112. Throughput: 0: 9794.6, 1: 9495.0. Samples: 154684372. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-26 17:33:46,063][104569] Avg episode reward: [(0, '9088.080'), (1, '7510.136')] [2023-12-26 17:33:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000302024_77332480.pth... [2023-12-26 17:33:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000302248_77381632.pth... [2023-12-26 17:33:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000301128_77094912.pth [2023-12-26 17:33:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000300872_77037568.pth [2023-12-26 17:33:46,287][105620] Updated weights for policy 1, policy_version 302250 (0.0009) [2023-12-26 17:33:46,338][105620] Updated weights for policy 1, policy_version 302260 (0.0009) [2023-12-26 17:33:46,390][105620] Updated weights for policy 1, policy_version 302270 (0.0009) [2023-12-26 17:33:46,441][105620] Updated weights for policy 1, policy_version 302280 (0.0009) [2023-12-26 17:33:46,461][105692] Updated weights for policy 0, policy_version 302026 (0.0009) [2023-12-26 17:33:46,511][105692] Updated weights for policy 0, policy_version 302036 (0.0008) [2023-12-26 17:33:46,572][105692] Updated weights for policy 0, policy_version 302046 (0.0007) [2023-12-26 17:33:46,623][105692] Updated weights for policy 0, policy_version 302056 (0.0008) [2023-12-26 17:33:47,149][105620] Updated weights for policy 1, policy_version 302290 (0.0009) [2023-12-26 17:33:47,206][105620] Updated weights for policy 1, policy_version 302300 (0.0009) [2023-12-26 17:33:47,266][105620] Updated weights for policy 1, policy_version 302310 (0.0008) [2023-12-26 17:33:47,415][105692] Updated weights for policy 0, policy_version 302067 (0.0010) [2023-12-26 17:33:47,474][105692] Updated weights for policy 0, policy_version 302078 (0.0010) [2023-12-26 17:33:47,532][105692] Updated weights for policy 0, policy_version 302088 (0.0010) [2023-12-26 17:33:47,875][105620] Updated weights for policy 1, policy_version 302320 (0.0008) [2023-12-26 17:33:47,933][105620] Updated weights for policy 1, policy_version 302330 (0.0007) [2023-12-26 17:33:47,986][105620] Updated weights for policy 1, policy_version 302340 (0.0011) [2023-12-26 17:33:48,277][105692] Updated weights for policy 0, policy_version 302098 (0.0006) [2023-12-26 17:33:48,330][105692] Updated weights for policy 0, policy_version 302108 (0.0006) [2023-12-26 17:33:48,387][105692] Updated weights for policy 0, policy_version 302118 (0.0010) [2023-12-26 17:33:48,722][105620] Updated weights for policy 1, policy_version 302350 (0.0011) [2023-12-26 17:33:48,794][105620] Updated weights for policy 1, policy_version 302360 (0.0011) [2023-12-26 17:33:48,853][105620] Updated weights for policy 1, policy_version 302370 (0.0010) [2023-12-26 17:33:49,101][105692] Updated weights for policy 0, policy_version 302128 (0.0010) [2023-12-26 17:33:49,164][105692] Updated weights for policy 0, policy_version 302138 (0.0007) [2023-12-26 17:33:49,232][105692] Updated weights for policy 0, policy_version 302148 (0.0007) [2023-12-26 17:33:49,497][105620] Updated weights for policy 1, policy_version 302380 (0.0009) [2023-12-26 17:33:49,546][105620] Updated weights for policy 1, policy_version 302390 (0.0006) [2023-12-26 17:33:49,601][105620] Updated weights for policy 1, policy_version 302400 (0.0007) [2023-12-26 17:33:49,951][105692] Updated weights for policy 0, policy_version 302158 (0.0007) [2023-12-26 17:33:50,011][105692] Updated weights for policy 0, policy_version 302168 (0.0005) [2023-12-26 17:33:50,067][105692] Updated weights for policy 0, policy_version 302178 (0.0005) [2023-12-26 17:33:50,399][105620] Updated weights for policy 1, policy_version 302410 (0.0009) [2023-12-26 17:33:50,472][105620] Updated weights for policy 1, policy_version 302420 (0.0009) [2023-12-26 17:33:50,540][105620] Updated weights for policy 1, policy_version 302430 (0.0010) [2023-12-26 17:33:50,600][105620] Updated weights for policy 1, policy_version 302440 (0.0008) [2023-12-26 17:33:50,620][105692] Updated weights for policy 0, policy_version 302188 (0.0005) [2023-12-26 17:33:50,678][105692] Updated weights for policy 0, policy_version 302198 (0.0006) [2023-12-26 17:33:50,733][105692] Updated weights for policy 0, policy_version 302208 (0.0005) [2023-12-26 17:33:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 154812416. Throughput: 0: 9867.3, 1: 9511.2. Samples: 154799732. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-26 17:33:51,062][104569] Avg episode reward: [(0, '9268.402'), (1, '7513.257')] [2023-12-26 17:33:51,305][105692] Updated weights for policy 0, policy_version 302218 (0.0006) [2023-12-26 17:33:51,372][105692] Updated weights for policy 0, policy_version 302228 (0.0007) [2023-12-26 17:33:51,440][105692] Updated weights for policy 0, policy_version 302238 (0.0009) [2023-12-26 17:33:51,506][105692] Updated weights for policy 0, policy_version 302248 (0.0006) [2023-12-26 17:33:51,513][105620] Updated weights for policy 1, policy_version 302450 (0.0008) [2023-12-26 17:33:51,566][105620] Updated weights for policy 1, policy_version 302460 (0.0010) [2023-12-26 17:33:51,621][105620] Updated weights for policy 1, policy_version 302470 (0.0010) [2023-12-26 17:33:52,248][105692] Updated weights for policy 0, policy_version 302258 (0.0009) [2023-12-26 17:33:52,311][105692] Updated weights for policy 0, policy_version 302268 (0.0011) [2023-12-26 17:33:52,377][105692] Updated weights for policy 0, policy_version 302278 (0.0009) [2023-12-26 17:33:52,383][105620] Updated weights for policy 1, policy_version 302480 (0.0008) [2023-12-26 17:33:52,448][105620] Updated weights for policy 1, policy_version 302490 (0.0008) [2023-12-26 17:33:52,511][105620] Updated weights for policy 1, policy_version 302500 (0.0010) [2023-12-26 17:33:53,148][105620] Updated weights for policy 1, policy_version 302510 (0.0007) [2023-12-26 17:33:53,210][105620] Updated weights for policy 1, policy_version 302520 (0.0005) [2023-12-26 17:33:53,213][105692] Updated weights for policy 0, policy_version 302288 (0.0008) [2023-12-26 17:33:53,260][105692] Updated weights for policy 0, policy_version 302298 (0.0008) [2023-12-26 17:33:53,265][105620] Updated weights for policy 1, policy_version 302530 (0.0005) [2023-12-26 17:33:53,309][105692] Updated weights for policy 0, policy_version 302308 (0.0009) [2023-12-26 17:33:53,784][105620] Updated weights for policy 1, policy_version 302540 (0.0007) [2023-12-26 17:33:53,833][105620] Updated weights for policy 1, policy_version 302550 (0.0006) [2023-12-26 17:33:53,889][105620] Updated weights for policy 1, policy_version 302560 (0.0009) [2023-12-26 17:33:54,127][105692] Updated weights for policy 0, policy_version 302319 (0.0009) [2023-12-26 17:33:54,179][105692] Updated weights for policy 0, policy_version 302329 (0.0009) [2023-12-26 17:33:54,224][105692] Updated weights for policy 0, policy_version 302339 (0.0008) [2023-12-26 17:33:54,693][105620] Updated weights for policy 1, policy_version 302570 (0.0008) [2023-12-26 17:33:54,754][105620] Updated weights for policy 1, policy_version 302580 (0.0009) [2023-12-26 17:33:54,819][105620] Updated weights for policy 1, policy_version 302590 (0.0009) [2023-12-26 17:33:54,876][105692] Updated weights for policy 0, policy_version 302349 (0.0006) [2023-12-26 17:33:54,878][105620] Updated weights for policy 1, policy_version 302600 (0.0009) [2023-12-26 17:33:54,931][105692] Updated weights for policy 0, policy_version 302359 (0.0006) [2023-12-26 17:33:54,994][105692] Updated weights for policy 0, policy_version 302369 (0.0007) [2023-12-26 17:33:55,571][105692] Updated weights for policy 0, policy_version 302379 (0.0006) [2023-12-26 17:33:55,629][105692] Updated weights for policy 0, policy_version 302389 (0.0009) [2023-12-26 17:33:55,686][105692] Updated weights for policy 0, policy_version 302399 (0.0008) [2023-12-26 17:33:55,700][105620] Updated weights for policy 1, policy_version 302610 (0.0007) [2023-12-26 17:33:55,755][105620] Updated weights for policy 1, policy_version 302620 (0.0007) [2023-12-26 17:33:55,812][105620] Updated weights for policy 1, policy_version 302630 (0.0009) [2023-12-26 17:33:56,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 154910720. Throughput: 0: 9932.7, 1: 9477.5. Samples: 154916644. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-26 17:33:56,063][104569] Avg episode reward: [(0, '9177.222'), (1, '6855.967')] [2023-12-26 17:33:56,350][105692] Updated weights for policy 0, policy_version 302409 (0.0009) [2023-12-26 17:33:56,406][105692] Updated weights for policy 0, policy_version 302419 (0.0005) [2023-12-26 17:33:56,469][105692] Updated weights for policy 0, policy_version 302429 (0.0005) [2023-12-26 17:33:56,533][105692] Updated weights for policy 0, policy_version 302439 (0.0007) [2023-12-26 17:33:56,631][105620] Updated weights for policy 1, policy_version 302640 (0.0010) [2023-12-26 17:33:56,686][105620] Updated weights for policy 1, policy_version 302650 (0.0009) [2023-12-26 17:33:56,734][105620] Updated weights for policy 1, policy_version 302660 (0.0010) [2023-12-26 17:33:57,052][105692] Updated weights for policy 0, policy_version 302449 (0.0006) [2023-12-26 17:33:57,100][105692] Updated weights for policy 0, policy_version 302459 (0.0007) [2023-12-26 17:33:57,151][105692] Updated weights for policy 0, policy_version 302469 (0.0009) [2023-12-26 17:33:57,464][105620] Updated weights for policy 1, policy_version 302670 (0.0007) [2023-12-26 17:33:57,515][105620] Updated weights for policy 1, policy_version 302680 (0.0007) [2023-12-26 17:33:57,573][105620] Updated weights for policy 1, policy_version 302690 (0.0005) [2023-12-26 17:33:57,887][105692] Updated weights for policy 0, policy_version 302479 (0.0009) [2023-12-26 17:33:57,938][105692] Updated weights for policy 0, policy_version 302489 (0.0008) [2023-12-26 17:33:58,003][105692] Updated weights for policy 0, policy_version 302499 (0.0008) [2023-12-26 17:33:58,286][105620] Updated weights for policy 1, policy_version 302700 (0.0007) [2023-12-26 17:33:58,359][105620] Updated weights for policy 1, policy_version 302710 (0.0008) [2023-12-26 17:33:58,420][105620] Updated weights for policy 1, policy_version 302720 (0.0010) [2023-12-26 17:33:58,792][105692] Updated weights for policy 0, policy_version 302509 (0.0009) [2023-12-26 17:33:58,860][105692] Updated weights for policy 0, policy_version 302519 (0.0011) [2023-12-26 17:33:58,923][105692] Updated weights for policy 0, policy_version 302529 (0.0011) [2023-12-26 17:33:59,102][105620] Updated weights for policy 1, policy_version 302730 (0.0010) [2023-12-26 17:33:59,154][105620] Updated weights for policy 1, policy_version 302740 (0.0005) [2023-12-26 17:33:59,216][105620] Updated weights for policy 1, policy_version 302750 (0.0006) [2023-12-26 17:33:59,280][105620] Updated weights for policy 1, policy_version 302760 (0.0009) [2023-12-26 17:33:59,591][105692] Updated weights for policy 0, policy_version 302539 (0.0010) [2023-12-26 17:33:59,639][105692] Updated weights for policy 0, policy_version 302549 (0.0010) [2023-12-26 17:33:59,682][105692] Updated weights for policy 0, policy_version 302559 (0.0007) [2023-12-26 17:34:00,040][105620] Updated weights for policy 1, policy_version 302770 (0.0009) [2023-12-26 17:34:00,105][105620] Updated weights for policy 1, policy_version 302781 (0.0009) [2023-12-26 17:34:00,169][105620] Updated weights for policy 1, policy_version 302791 (0.0008) [2023-12-26 17:34:00,363][105692] Updated weights for policy 0, policy_version 302569 (0.0006) [2023-12-26 17:34:00,415][105585] KL-divergence is very high: 125.8401 [2023-12-26 17:34:00,424][105692] Updated weights for policy 0, policy_version 302579 (0.0010) [2023-12-26 17:34:00,491][105692] Updated weights for policy 0, policy_version 302589 (0.0011) [2023-12-26 17:34:00,554][105692] Updated weights for policy 0, policy_version 302599 (0.0011) [2023-12-26 17:34:00,909][105620] Updated weights for policy 1, policy_version 302801 (0.0006) [2023-12-26 17:34:00,979][105620] Updated weights for policy 1, policy_version 302811 (0.0005) [2023-12-26 17:34:01,049][105620] Updated weights for policy 1, policy_version 302821 (0.0007) [2023-12-26 17:34:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 155000832. Throughput: 0: 9967.9, 1: 9515.8. Samples: 154975916. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-26 17:34:01,062][104569] Avg episode reward: [(0, '9088.894'), (1, '7504.458')] [2023-12-26 17:34:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000302600_77479936.pth... [2023-12-26 17:34:01,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000302824_77529088.pth... [2023-12-26 17:34:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000301448_77185024.pth [2023-12-26 17:34:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000301672_77234176.pth [2023-12-26 17:34:01,277][105692] Updated weights for policy 0, policy_version 302609 (0.0009) [2023-12-26 17:34:01,308][105585] KL-divergence is very high: 133.1403 [2023-12-26 17:34:01,332][105692] Updated weights for policy 0, policy_version 302619 (0.0009) [2023-12-26 17:34:01,356][105585] KL-divergence is very high: 150.8388 [2023-12-26 17:34:01,390][105692] Updated weights for policy 0, policy_version 302629 (0.0009) [2023-12-26 17:34:01,403][105585] KL-divergence is very high: 108.1744 [2023-12-26 17:34:01,739][105620] Updated weights for policy 1, policy_version 302831 (0.0008) [2023-12-26 17:34:01,788][105620] Updated weights for policy 1, policy_version 302841 (0.0008) [2023-12-26 17:34:01,835][105620] Updated weights for policy 1, policy_version 302851 (0.0007) [2023-12-26 17:34:02,140][105692] Updated weights for policy 0, policy_version 302639 (0.0010) [2023-12-26 17:34:02,184][105692] Updated weights for policy 0, policy_version 302649 (0.0010) [2023-12-26 17:34:02,228][105692] Updated weights for policy 0, policy_version 302659 (0.0010) [2023-12-26 17:34:02,562][105620] Updated weights for policy 1, policy_version 302861 (0.0006) [2023-12-26 17:34:02,610][105620] Updated weights for policy 1, policy_version 302871 (0.0009) [2023-12-26 17:34:02,655][105620] Updated weights for policy 1, policy_version 302881 (0.0008) [2023-12-26 17:34:03,009][105692] Updated weights for policy 0, policy_version 302669 (0.0010) [2023-12-26 17:34:03,057][105692] Updated weights for policy 0, policy_version 302679 (0.0010) [2023-12-26 17:34:03,106][105692] Updated weights for policy 0, policy_version 302689 (0.0011) [2023-12-26 17:34:03,260][105620] Updated weights for policy 1, policy_version 302891 (0.0005) [2023-12-26 17:34:03,311][105620] Updated weights for policy 1, policy_version 302901 (0.0005) [2023-12-26 17:34:03,355][105620] Updated weights for policy 1, policy_version 302911 (0.0005) [2023-12-26 17:34:03,877][105692] Updated weights for policy 0, policy_version 302699 (0.0011) [2023-12-26 17:34:03,936][105692] Updated weights for policy 0, policy_version 302709 (0.0009) [2023-12-26 17:34:03,938][105620] Updated weights for policy 1, policy_version 302921 (0.0006) [2023-12-26 17:34:03,999][105692] Updated weights for policy 0, policy_version 302719 (0.0011) [2023-12-26 17:34:04,003][105620] Updated weights for policy 1, policy_version 302931 (0.0008) [2023-12-26 17:34:04,062][105620] Updated weights for policy 1, policy_version 302941 (0.0007) [2023-12-26 17:34:04,127][105620] Updated weights for policy 1, policy_version 302951 (0.0009) [2023-12-26 17:34:04,743][105692] Updated weights for policy 0, policy_version 302729 (0.0011) [2023-12-26 17:34:04,767][105620] Updated weights for policy 1, policy_version 302961 (0.0007) [2023-12-26 17:34:04,808][105692] Updated weights for policy 0, policy_version 302739 (0.0011) [2023-12-26 17:34:04,814][105620] Updated weights for policy 1, policy_version 302971 (0.0007) [2023-12-26 17:34:04,860][105692] Updated weights for policy 0, policy_version 302749 (0.0010) [2023-12-26 17:34:04,876][105620] Updated weights for policy 1, policy_version 302981 (0.0005) [2023-12-26 17:34:04,912][105692] Updated weights for policy 0, policy_version 302759 (0.0010) [2023-12-26 17:34:05,428][105620] Updated weights for policy 1, policy_version 302991 (0.0005) [2023-12-26 17:34:05,485][105620] Updated weights for policy 1, policy_version 303001 (0.0005) [2023-12-26 17:34:05,523][105692] Updated weights for policy 0, policy_version 302769 (0.0006) [2023-12-26 17:34:05,540][105620] Updated weights for policy 1, policy_version 303011 (0.0005) [2023-12-26 17:34:05,577][105692] Updated weights for policy 0, policy_version 302779 (0.0011) [2023-12-26 17:34:05,625][105692] Updated weights for policy 0, policy_version 302789 (0.0010) [2023-12-26 17:34:06,057][105620] Updated weights for policy 1, policy_version 303021 (0.0006) [2023-12-26 17:34:06,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 155107328. Throughput: 0: 9865.5, 1: 9648.8. Samples: 155094944. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-26 17:34:06,062][104569] Avg episode reward: [(0, '9179.102'), (1, '7782.257')] [2023-12-26 17:34:06,117][105620] Updated weights for policy 1, policy_version 303031 (0.0008) [2023-12-26 17:34:06,179][105620] Updated weights for policy 1, policy_version 303041 (0.0010) [2023-12-26 17:34:06,327][105692] Updated weights for policy 0, policy_version 302799 (0.0011) [2023-12-26 17:34:06,376][105692] Updated weights for policy 0, policy_version 302809 (0.0010) [2023-12-26 17:34:06,428][105692] Updated weights for policy 0, policy_version 302819 (0.0005) [2023-12-26 17:34:06,898][105620] Updated weights for policy 1, policy_version 303051 (0.0011) [2023-12-26 17:34:06,957][105620] Updated weights for policy 1, policy_version 303061 (0.0011) [2023-12-26 17:34:07,009][105620] Updated weights for policy 1, policy_version 303071 (0.0011) [2023-12-26 17:34:07,173][105692] Updated weights for policy 0, policy_version 302829 (0.0008) [2023-12-26 17:34:07,234][105692] Updated weights for policy 0, policy_version 302839 (0.0011) [2023-12-26 17:34:07,292][105692] Updated weights for policy 0, policy_version 302849 (0.0011) [2023-12-26 17:34:07,767][105620] Updated weights for policy 1, policy_version 303081 (0.0011) [2023-12-26 17:34:07,821][105620] Updated weights for policy 1, policy_version 303091 (0.0007) [2023-12-26 17:34:07,875][105620] Updated weights for policy 1, policy_version 303101 (0.0005) [2023-12-26 17:34:07,932][105620] Updated weights for policy 1, policy_version 303111 (0.0005) [2023-12-26 17:34:08,046][105692] Updated weights for policy 0, policy_version 302859 (0.0011) [2023-12-26 17:34:08,098][105692] Updated weights for policy 0, policy_version 302869 (0.0010) [2023-12-26 17:34:08,160][105692] Updated weights for policy 0, policy_version 302879 (0.0010) [2023-12-26 17:34:08,510][105620] Updated weights for policy 1, policy_version 303121 (0.0005) [2023-12-26 17:34:08,553][105586] KL-divergence is very high: 108.8921 [2023-12-26 17:34:08,565][105620] Updated weights for policy 1, policy_version 303131 (0.0007) [2023-12-26 17:34:08,600][105586] KL-divergence is very high: 104.4968 [2023-12-26 17:34:08,623][105620] Updated weights for policy 1, policy_version 303141 (0.0006) [2023-12-26 17:34:08,913][105692] Updated weights for policy 0, policy_version 302889 (0.0010) [2023-12-26 17:34:08,977][105692] Updated weights for policy 0, policy_version 302899 (0.0011) [2023-12-26 17:34:09,041][105692] Updated weights for policy 0, policy_version 302909 (0.0010) [2023-12-26 17:34:09,094][105692] Updated weights for policy 0, policy_version 302919 (0.0010) [2023-12-26 17:34:09,257][105620] Updated weights for policy 1, policy_version 303151 (0.0007) [2023-12-26 17:34:09,316][105620] Updated weights for policy 1, policy_version 303161 (0.0008) [2023-12-26 17:34:09,382][105620] Updated weights for policy 1, policy_version 303171 (0.0010) [2023-12-26 17:34:09,780][105692] Updated weights for policy 0, policy_version 302929 (0.0011) [2023-12-26 17:34:09,837][105692] Updated weights for policy 0, policy_version 302939 (0.0011) [2023-12-26 17:34:09,894][105692] Updated weights for policy 0, policy_version 302949 (0.0011) [2023-12-26 17:34:10,155][105620] Updated weights for policy 1, policy_version 303181 (0.0010) [2023-12-26 17:34:10,225][105620] Updated weights for policy 1, policy_version 303191 (0.0010) [2023-12-26 17:34:10,288][105620] Updated weights for policy 1, policy_version 303201 (0.0009) [2023-12-26 17:34:10,621][105692] Updated weights for policy 0, policy_version 302959 (0.0010) [2023-12-26 17:34:10,673][105692] Updated weights for policy 0, policy_version 302969 (0.0010) [2023-12-26 17:34:10,721][105692] Updated weights for policy 0, policy_version 302979 (0.0010) [2023-12-26 17:34:11,001][105620] Updated weights for policy 1, policy_version 303211 (0.0011) [2023-12-26 17:34:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 155205632. Throughput: 0: 9886.3, 1: 9821.5. Samples: 155216500. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-26 17:34:11,063][104569] Avg episode reward: [(0, '9178.119'), (1, '7783.528')] [2023-12-26 17:34:11,074][105620] Updated weights for policy 1, policy_version 303221 (0.0011) [2023-12-26 17:34:11,142][105620] Updated weights for policy 1, policy_version 303231 (0.0009) [2023-12-26 17:34:11,529][105692] Updated weights for policy 0, policy_version 302989 (0.0009) [2023-12-26 17:34:11,591][105692] Updated weights for policy 0, policy_version 302999 (0.0008) [2023-12-26 17:34:11,659][105692] Updated weights for policy 0, policy_version 303009 (0.0008) [2023-12-26 17:34:11,908][105620] Updated weights for policy 1, policy_version 303241 (0.0008) [2023-12-26 17:34:11,970][105620] Updated weights for policy 1, policy_version 303251 (0.0011) [2023-12-26 17:34:12,027][105620] Updated weights for policy 1, policy_version 303261 (0.0011) [2023-12-26 17:34:12,085][105620] Updated weights for policy 1, policy_version 303271 (0.0011) [2023-12-26 17:34:12,321][105692] Updated weights for policy 0, policy_version 303019 (0.0007) [2023-12-26 17:34:12,393][105692] Updated weights for policy 0, policy_version 303029 (0.0008) [2023-12-26 17:34:12,460][105692] Updated weights for policy 0, policy_version 303039 (0.0008) [2023-12-26 17:34:12,850][105620] Updated weights for policy 1, policy_version 303281 (0.0009) [2023-12-26 17:34:12,916][105620] Updated weights for policy 1, policy_version 303291 (0.0009) [2023-12-26 17:34:12,978][105620] Updated weights for policy 1, policy_version 303301 (0.0009) [2023-12-26 17:34:13,236][105692] Updated weights for policy 0, policy_version 303049 (0.0009) [2023-12-26 17:34:13,298][105692] Updated weights for policy 0, policy_version 303059 (0.0006) [2023-12-26 17:34:13,363][105692] Updated weights for policy 0, policy_version 303069 (0.0007) [2023-12-26 17:34:13,414][105692] Updated weights for policy 0, policy_version 303079 (0.0009) [2023-12-26 17:34:13,633][105620] Updated weights for policy 1, policy_version 303311 (0.0006) [2023-12-26 17:34:13,690][105620] Updated weights for policy 1, policy_version 303321 (0.0005) [2023-12-26 17:34:13,741][105620] Updated weights for policy 1, policy_version 303331 (0.0007) [2023-12-26 17:34:14,176][105692] Updated weights for policy 0, policy_version 303089 (0.0010) [2023-12-26 17:34:14,230][105692] Updated weights for policy 0, policy_version 303099 (0.0010) [2023-12-26 17:34:14,286][105692] Updated weights for policy 0, policy_version 303109 (0.0010) [2023-12-26 17:34:14,337][105620] Updated weights for policy 1, policy_version 303341 (0.0009) [2023-12-26 17:34:14,397][105620] Updated weights for policy 1, policy_version 303352 (0.0012) [2023-12-26 17:34:14,449][105620] Updated weights for policy 1, policy_version 303362 (0.0010) [2023-12-26 17:34:14,969][105692] Updated weights for policy 0, policy_version 303119 (0.0006) [2023-12-26 17:34:15,033][105692] Updated weights for policy 0, policy_version 303129 (0.0008) [2023-12-26 17:34:15,099][105692] Updated weights for policy 0, policy_version 303139 (0.0006) [2023-12-26 17:34:15,295][105620] Updated weights for policy 1, policy_version 303372 (0.0008) [2023-12-26 17:34:15,355][105620] Updated weights for policy 1, policy_version 303382 (0.0009) [2023-12-26 17:34:15,421][105620] Updated weights for policy 1, policy_version 303392 (0.0008) [2023-12-26 17:34:15,693][105692] Updated weights for policy 0, policy_version 303149 (0.0008) [2023-12-26 17:34:15,756][105692] Updated weights for policy 0, policy_version 303159 (0.0010) [2023-12-26 17:34:15,807][105692] Updated weights for policy 0, policy_version 303169 (0.0009) [2023-12-26 17:34:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 155303936. Throughput: 0: 9809.2, 1: 9846.5. Samples: 155273240. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-26 17:34:16,063][104569] Avg episode reward: [(0, '9175.023'), (1, '7605.424')] [2023-12-26 17:34:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000303176_77627392.pth... [2023-12-26 17:34:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000303400_77676544.pth... [2023-12-26 17:34:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000302248_77381632.pth [2023-12-26 17:34:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000302024_77332480.pth [2023-12-26 17:34:16,149][105620] Updated weights for policy 1, policy_version 303402 (0.0006) [2023-12-26 17:34:16,203][105620] Updated weights for policy 1, policy_version 303412 (0.0009) [2023-12-26 17:34:16,248][105620] Updated weights for policy 1, policy_version 303422 (0.0008) [2023-12-26 17:34:16,301][105620] Updated weights for policy 1, policy_version 303432 (0.0008) [2023-12-26 17:34:16,437][105692] Updated weights for policy 0, policy_version 303179 (0.0008) [2023-12-26 17:34:16,497][105692] Updated weights for policy 0, policy_version 303189 (0.0005) [2023-12-26 17:34:16,564][105692] Updated weights for policy 0, policy_version 303199 (0.0007) [2023-12-26 17:34:17,130][105620] Updated weights for policy 1, policy_version 303442 (0.0009) [2023-12-26 17:34:17,187][105620] Updated weights for policy 1, policy_version 303452 (0.0009) [2023-12-26 17:34:17,194][105692] Updated weights for policy 0, policy_version 303209 (0.0009) [2023-12-26 17:34:17,237][105620] Updated weights for policy 1, policy_version 303462 (0.0008) [2023-12-26 17:34:17,238][105692] Updated weights for policy 0, policy_version 303219 (0.0006) [2023-12-26 17:34:17,284][105692] Updated weights for policy 0, policy_version 303229 (0.0005) [2023-12-26 17:34:17,329][105692] Updated weights for policy 0, policy_version 303239 (0.0005) [2023-12-26 17:34:17,959][105692] Updated weights for policy 0, policy_version 303249 (0.0008) [2023-12-26 17:34:18,006][105620] Updated weights for policy 1, policy_version 303472 (0.0007) [2023-12-26 17:34:18,012][105692] Updated weights for policy 0, policy_version 303259 (0.0007) [2023-12-26 17:34:18,052][105620] Updated weights for policy 1, policy_version 303482 (0.0007) [2023-12-26 17:34:18,070][105692] Updated weights for policy 0, policy_version 303269 (0.0009) [2023-12-26 17:34:18,110][105620] Updated weights for policy 1, policy_version 303492 (0.0007) [2023-12-26 17:34:18,809][105692] Updated weights for policy 0, policy_version 303279 (0.0008) [2023-12-26 17:34:18,869][105692] Updated weights for policy 0, policy_version 303289 (0.0008) [2023-12-26 17:34:18,871][105620] Updated weights for policy 1, policy_version 303502 (0.0007) [2023-12-26 17:34:18,882][105585] KL-divergence is very high: 160.0187 [2023-12-26 17:34:18,889][105585] KL-divergence is very high: 208.2103 [2023-12-26 17:34:18,900][105585] KL-divergence is very high: 138.7970 [2023-12-26 17:34:18,907][105585] KL-divergence is very high: 312.0581 [2023-12-26 17:34:18,914][105585] KL-divergence is very high: 284.0012 [2023-12-26 17:34:18,919][105585] KL-divergence is very high: 303.0644 [2023-12-26 17:34:18,928][105620] Updated weights for policy 1, policy_version 303512 (0.0006) [2023-12-26 17:34:18,930][105692] Updated weights for policy 0, policy_version 303299 (0.0008) [2023-12-26 17:34:18,932][105585] KL-divergence is very high: 446.7176 [2023-12-26 17:34:18,938][105585] KL-divergence is very high: 376.4171 [2023-12-26 17:34:18,950][105585] KL-divergence is very high: 247.7847 [2023-12-26 17:34:18,956][105585] KL-divergence is very high: 320.5196 [2023-12-26 17:34:18,993][105620] Updated weights for policy 1, policy_version 303522 (0.0008) [2023-12-26 17:34:19,684][105585] KL-divergence is very high: 137.7862 [2023-12-26 17:34:19,691][105585] KL-divergence is very high: 218.3815 [2023-12-26 17:34:19,717][105692] Updated weights for policy 0, policy_version 303309 (0.0007) [2023-12-26 17:34:19,719][105620] Updated weights for policy 1, policy_version 303532 (0.0008) [2023-12-26 17:34:19,740][105585] KL-divergence is very high: 123.6553 [2023-12-26 17:34:19,775][105692] Updated weights for policy 0, policy_version 303319 (0.0006) [2023-12-26 17:34:19,782][105620] Updated weights for policy 1, policy_version 303542 (0.0007) [2023-12-26 17:34:19,832][105692] Updated weights for policy 0, policy_version 303329 (0.0007) [2023-12-26 17:34:19,844][105620] Updated weights for policy 1, policy_version 303552 (0.0009) [2023-12-26 17:34:19,849][105585] KL-divergence is very high: 110.4275 [2023-12-26 17:34:20,583][105585] KL-divergence is very high: 152.8543 [2023-12-26 17:34:20,594][105620] Updated weights for policy 1, policy_version 303562 (0.0009) [2023-12-26 17:34:20,604][105692] Updated weights for policy 0, policy_version 303339 (0.0007) [2023-12-26 17:34:20,626][105585] KL-divergence is very high: 116.9143 [2023-12-26 17:34:20,653][105692] Updated weights for policy 0, policy_version 303349 (0.0006) [2023-12-26 17:34:20,654][105620] Updated weights for policy 1, policy_version 303572 (0.0007) [2023-12-26 17:34:20,719][105620] Updated weights for policy 1, policy_version 303582 (0.0008) [2023-12-26 17:34:20,721][105692] Updated weights for policy 0, policy_version 303359 (0.0006) [2023-12-26 17:34:20,778][105620] Updated weights for policy 1, policy_version 303592 (0.0006) [2023-12-26 17:34:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 155402240. Throughput: 0: 9828.3, 1: 9782.4. Samples: 155389872. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-26 17:34:21,062][104569] Avg episode reward: [(0, '7990.129'), (1, '7791.007')] [2023-12-26 17:34:21,460][105692] Updated weights for policy 0, policy_version 303369 (0.0008) [2023-12-26 17:34:21,516][105620] Updated weights for policy 1, policy_version 303602 (0.0009) [2023-12-26 17:34:21,524][105692] Updated weights for policy 0, policy_version 303379 (0.0010) [2023-12-26 17:34:21,569][105620] Updated weights for policy 1, policy_version 303612 (0.0006) [2023-12-26 17:34:21,582][105692] Updated weights for policy 0, policy_version 303389 (0.0010) [2023-12-26 17:34:21,625][105620] Updated weights for policy 1, policy_version 303622 (0.0006) [2023-12-26 17:34:21,646][105692] Updated weights for policy 0, policy_version 303399 (0.0011) [2023-12-26 17:34:22,410][105692] Updated weights for policy 0, policy_version 303409 (0.0011) [2023-12-26 17:34:22,413][105620] Updated weights for policy 1, policy_version 303632 (0.0005) [2023-12-26 17:34:22,476][105620] Updated weights for policy 1, policy_version 303642 (0.0006) [2023-12-26 17:34:22,477][105692] Updated weights for policy 0, policy_version 303419 (0.0011) [2023-12-26 17:34:22,539][105620] Updated weights for policy 1, policy_version 303652 (0.0006) [2023-12-26 17:34:22,544][105692] Updated weights for policy 0, policy_version 303429 (0.0011) [2023-12-26 17:34:23,288][105692] Updated weights for policy 0, policy_version 303439 (0.0010) [2023-12-26 17:34:23,294][105620] Updated weights for policy 1, policy_version 303662 (0.0007) [2023-12-26 17:34:23,339][105692] Updated weights for policy 0, policy_version 303449 (0.0010) [2023-12-26 17:34:23,349][105620] Updated weights for policy 1, policy_version 303672 (0.0005) [2023-12-26 17:34:23,393][105692] Updated weights for policy 0, policy_version 303459 (0.0010) [2023-12-26 17:34:23,403][105620] Updated weights for policy 1, policy_version 303682 (0.0005) [2023-12-26 17:34:24,005][105620] Updated weights for policy 1, policy_version 303692 (0.0008) [2023-12-26 17:34:24,057][105620] Updated weights for policy 1, policy_version 303702 (0.0010) [2023-12-26 17:34:24,119][105620] Updated weights for policy 1, policy_version 303712 (0.0010) [2023-12-26 17:34:24,155][105692] Updated weights for policy 0, policy_version 303469 (0.0010) [2023-12-26 17:34:24,210][105692] Updated weights for policy 0, policy_version 303479 (0.0010) [2023-12-26 17:34:24,264][105692] Updated weights for policy 0, policy_version 303489 (0.0010) [2023-12-26 17:34:24,831][105620] Updated weights for policy 1, policy_version 303722 (0.0010) [2023-12-26 17:34:24,872][105692] Updated weights for policy 0, policy_version 303499 (0.0010) [2023-12-26 17:34:24,890][105620] Updated weights for policy 1, policy_version 303732 (0.0007) [2023-12-26 17:34:24,916][105692] Updated weights for policy 0, policy_version 303509 (0.0010) [2023-12-26 17:34:24,942][105620] Updated weights for policy 1, policy_version 303742 (0.0006) [2023-12-26 17:34:24,968][105692] Updated weights for policy 0, policy_version 303519 (0.0010) [2023-12-26 17:34:24,994][105620] Updated weights for policy 1, policy_version 303752 (0.0005) [2023-12-26 17:34:25,654][105692] Updated weights for policy 0, policy_version 303529 (0.0010) [2023-12-26 17:34:25,705][105692] Updated weights for policy 0, policy_version 303539 (0.0005) [2023-12-26 17:34:25,752][105620] Updated weights for policy 1, policy_version 303762 (0.0005) [2023-12-26 17:34:25,756][105692] Updated weights for policy 0, policy_version 303549 (0.0006) [2023-12-26 17:34:25,762][105586] KL-divergence is very high: 164.7084 [2023-12-26 17:34:25,800][105620] Updated weights for policy 1, policy_version 303772 (0.0006) [2023-12-26 17:34:25,800][105586] KL-divergence is very high: 235.8256 [2023-12-26 17:34:25,808][105692] Updated weights for policy 0, policy_version 303559 (0.0010) [2023-12-26 17:34:25,840][105586] KL-divergence is very high: 198.1204 [2023-12-26 17:34:25,851][105620] Updated weights for policy 1, policy_version 303782 (0.0006) [2023-12-26 17:34:26,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.7, 300 sec: 19466.4). Total num frames: 155500544. Throughput: 0: 9777.5, 1: 9720.3. Samples: 155504716. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-26 17:34:26,063][104569] Avg episode reward: [(0, '8354.130'), (1, '7883.898')] [2023-12-26 17:34:26,409][105692] Updated weights for policy 0, policy_version 303569 (0.0007) [2023-12-26 17:34:26,456][105692] Updated weights for policy 0, policy_version 303579 (0.0008) [2023-12-26 17:34:26,520][105692] Updated weights for policy 0, policy_version 303589 (0.0010) [2023-12-26 17:34:26,540][105620] Updated weights for policy 1, policy_version 303792 (0.0006) [2023-12-26 17:34:26,594][105620] Updated weights for policy 1, policy_version 303802 (0.0005) [2023-12-26 17:34:26,642][105620] Updated weights for policy 1, policy_version 303812 (0.0005) [2023-12-26 17:34:27,166][105620] Updated weights for policy 1, policy_version 303822 (0.0005) [2023-12-26 17:34:27,217][105692] Updated weights for policy 0, policy_version 303599 (0.0010) [2023-12-26 17:34:27,223][105620] Updated weights for policy 1, policy_version 303832 (0.0006) [2023-12-26 17:34:27,275][105692] Updated weights for policy 0, policy_version 303609 (0.0010) [2023-12-26 17:34:27,281][105620] Updated weights for policy 1, policy_version 303842 (0.0005) [2023-12-26 17:34:27,333][105692] Updated weights for policy 0, policy_version 303619 (0.0010) [2023-12-26 17:34:27,861][105620] Updated weights for policy 1, policy_version 303852 (0.0007) [2023-12-26 17:34:27,910][105620] Updated weights for policy 1, policy_version 303862 (0.0008) [2023-12-26 17:34:27,957][105586] KL-divergence is very high: 123.3326 [2023-12-26 17:34:27,963][105620] Updated weights for policy 1, policy_version 303872 (0.0006) [2023-12-26 17:34:27,996][105692] Updated weights for policy 0, policy_version 303629 (0.0007) [2023-12-26 17:34:28,003][105586] KL-divergence is very high: 106.3563 [2023-12-26 17:34:28,060][105692] Updated weights for policy 0, policy_version 303639 (0.0010) [2023-12-26 17:34:28,120][105692] Updated weights for policy 0, policy_version 303649 (0.0010) [2023-12-26 17:34:28,716][105620] Updated weights for policy 1, policy_version 303882 (0.0010) [2023-12-26 17:34:28,771][105620] Updated weights for policy 1, policy_version 303892 (0.0009) [2023-12-26 17:34:28,823][105692] Updated weights for policy 0, policy_version 303659 (0.0010) [2023-12-26 17:34:28,836][105620] Updated weights for policy 1, policy_version 303902 (0.0006) [2023-12-26 17:34:28,875][105692] Updated weights for policy 0, policy_version 303669 (0.0007) [2023-12-26 17:34:28,896][105620] Updated weights for policy 1, policy_version 303912 (0.0008) [2023-12-26 17:34:28,934][105692] Updated weights for policy 0, policy_version 303679 (0.0009) [2023-12-26 17:34:29,578][105620] Updated weights for policy 1, policy_version 303922 (0.0008) [2023-12-26 17:34:29,634][105620] Updated weights for policy 1, policy_version 303932 (0.0008) [2023-12-26 17:34:29,693][105620] Updated weights for policy 1, policy_version 303942 (0.0008) [2023-12-26 17:34:29,714][105692] Updated weights for policy 0, policy_version 303689 (0.0010) [2023-12-26 17:34:29,772][105692] Updated weights for policy 0, policy_version 303699 (0.0010) [2023-12-26 17:34:29,835][105692] Updated weights for policy 0, policy_version 303709 (0.0011) [2023-12-26 17:34:29,904][105692] Updated weights for policy 0, policy_version 303719 (0.0008) [2023-12-26 17:34:30,463][105620] Updated weights for policy 1, policy_version 303952 (0.0009) [2023-12-26 17:34:30,519][105620] Updated weights for policy 1, policy_version 303962 (0.0009) [2023-12-26 17:34:30,573][105620] Updated weights for policy 1, policy_version 303972 (0.0008) [2023-12-26 17:34:30,594][105692] Updated weights for policy 0, policy_version 303729 (0.0009) [2023-12-26 17:34:30,648][105692] Updated weights for policy 0, policy_version 303739 (0.0010) [2023-12-26 17:34:30,702][105692] Updated weights for policy 0, policy_version 303749 (0.0010) [2023-12-26 17:34:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 155598848. Throughput: 0: 9831.7, 1: 9809.9. Samples: 155568240. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-26 17:34:31,062][104569] Avg episode reward: [(0, '8501.610'), (1, '7334.869')] [2023-12-26 17:34:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000303752_77774848.pth... [2023-12-26 17:34:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000303976_77824000.pth... [2023-12-26 17:34:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000302824_77529088.pth [2023-12-26 17:34:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000302600_77479936.pth [2023-12-26 17:34:31,332][105692] Updated weights for policy 0, policy_version 303759 (0.0009) [2023-12-26 17:34:31,398][105692] Updated weights for policy 0, policy_version 303769 (0.0008) [2023-12-26 17:34:31,413][105620] Updated weights for policy 1, policy_version 303982 (0.0007) [2023-12-26 17:34:31,459][105692] Updated weights for policy 0, policy_version 303779 (0.0006) [2023-12-26 17:34:31,461][105620] Updated weights for policy 1, policy_version 303992 (0.0009) [2023-12-26 17:34:31,519][105620] Updated weights for policy 1, policy_version 304002 (0.0009) [2023-12-26 17:34:32,197][105692] Updated weights for policy 0, policy_version 303789 (0.0007) [2023-12-26 17:34:32,219][105620] Updated weights for policy 1, policy_version 304012 (0.0008) [2023-12-26 17:34:32,250][105692] Updated weights for policy 0, policy_version 303799 (0.0007) [2023-12-26 17:34:32,276][105620] Updated weights for policy 1, policy_version 304022 (0.0008) [2023-12-26 17:34:32,315][105692] Updated weights for policy 0, policy_version 303809 (0.0008) [2023-12-26 17:34:32,337][105620] Updated weights for policy 1, policy_version 304032 (0.0007) [2023-12-26 17:34:33,002][105620] Updated weights for policy 1, policy_version 304042 (0.0008) [2023-12-26 17:34:33,050][105620] Updated weights for policy 1, policy_version 304052 (0.0008) [2023-12-26 17:34:33,069][105586] KL-divergence is very high: 133.9227 [2023-12-26 17:34:33,080][105586] KL-divergence is very high: 121.3946 [2023-12-26 17:34:33,092][105586] KL-divergence is very high: 280.7198 [2023-12-26 17:34:33,099][105586] KL-divergence is very high: 227.4636 [2023-12-26 17:34:33,110][105620] Updated weights for policy 1, policy_version 304062 (0.0009) [2023-12-26 17:34:33,117][105692] Updated weights for policy 0, policy_version 303819 (0.0006) [2023-12-26 17:34:33,117][105586] KL-divergence is very high: 429.6148 [2023-12-26 17:34:33,126][105586] KL-divergence is very high: 240.4893 [2023-12-26 17:34:33,137][105586] KL-divergence is very high: 401.9443 [2023-12-26 17:34:33,142][105586] KL-divergence is very high: 255.7026 [2023-12-26 17:34:33,156][105586] KL-divergence is very high: 407.2274 [2023-12-26 17:34:33,163][105620] Updated weights for policy 1, policy_version 304072 (0.0006) [2023-12-26 17:34:33,169][105692] Updated weights for policy 0, policy_version 303829 (0.0006) [2023-12-26 17:34:33,223][105692] Updated weights for policy 0, policy_version 303839 (0.0009) [2023-12-26 17:34:33,836][105692] Updated weights for policy 0, policy_version 303849 (0.0007) [2023-12-26 17:34:33,904][105692] Updated weights for policy 0, policy_version 303859 (0.0005) [2023-12-26 17:34:33,952][105620] Updated weights for policy 1, policy_version 304082 (0.0008) [2023-12-26 17:34:33,956][105692] Updated weights for policy 0, policy_version 303869 (0.0006) [2023-12-26 17:34:34,012][105620] Updated weights for policy 1, policy_version 304092 (0.0009) [2023-12-26 17:34:34,022][105692] Updated weights for policy 0, policy_version 303879 (0.0007) [2023-12-26 17:34:34,063][105620] Updated weights for policy 1, policy_version 304102 (0.0007) [2023-12-26 17:34:34,667][105692] Updated weights for policy 0, policy_version 303889 (0.0010) [2023-12-26 17:34:34,722][105692] Updated weights for policy 0, policy_version 303899 (0.0010) [2023-12-26 17:34:34,776][105692] Updated weights for policy 0, policy_version 303909 (0.0010) [2023-12-26 17:34:34,878][105620] Updated weights for policy 1, policy_version 304112 (0.0006) [2023-12-26 17:34:34,931][105620] Updated weights for policy 1, policy_version 304122 (0.0005) [2023-12-26 17:34:34,994][105620] Updated weights for policy 1, policy_version 304132 (0.0009) [2023-12-26 17:34:35,433][105692] Updated weights for policy 0, policy_version 303919 (0.0010) [2023-12-26 17:34:35,483][105692] Updated weights for policy 0, policy_version 303929 (0.0010) [2023-12-26 17:34:35,534][105692] Updated weights for policy 0, policy_version 303939 (0.0010) [2023-12-26 17:34:35,676][105620] Updated weights for policy 1, policy_version 304142 (0.0008) [2023-12-26 17:34:35,741][105620] Updated weights for policy 1, policy_version 304152 (0.0008) [2023-12-26 17:34:35,799][105620] Updated weights for policy 1, policy_version 304162 (0.0005) [2023-12-26 17:34:36,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.9, 300 sec: 19438.6). Total num frames: 155697152. Throughput: 0: 9896.4, 1: 9762.7. Samples: 155684396. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-26 17:34:36,062][104569] Avg episode reward: [(0, '8996.965'), (1, '7513.802')] [2023-12-26 17:34:36,155][105692] Updated weights for policy 0, policy_version 303949 (0.0007) [2023-12-26 17:34:36,218][105692] Updated weights for policy 0, policy_version 303959 (0.0010) [2023-12-26 17:34:36,284][105692] Updated weights for policy 0, policy_version 303969 (0.0011) [2023-12-26 17:34:36,372][105620] Updated weights for policy 1, policy_version 304172 (0.0005) [2023-12-26 17:34:36,437][105620] Updated weights for policy 1, policy_version 304182 (0.0008) [2023-12-26 17:34:36,455][105586] KL-divergence is very high: 100.0354 [2023-12-26 17:34:36,497][105620] Updated weights for policy 1, policy_version 304192 (0.0008) [2023-12-26 17:34:36,502][105586] KL-divergence is very high: 207.9540 [2023-12-26 17:34:37,031][105692] Updated weights for policy 0, policy_version 303979 (0.0010) [2023-12-26 17:34:37,067][105620] Updated weights for policy 1, policy_version 304202 (0.0007) [2023-12-26 17:34:37,088][105692] Updated weights for policy 0, policy_version 303989 (0.0008) [2023-12-26 17:34:37,114][105620] Updated weights for policy 1, policy_version 304212 (0.0007) [2023-12-26 17:34:37,143][105692] Updated weights for policy 0, policy_version 303999 (0.0009) [2023-12-26 17:34:37,175][105620] Updated weights for policy 1, policy_version 304222 (0.0009) [2023-12-26 17:34:37,227][105620] Updated weights for policy 1, policy_version 304232 (0.0008) [2023-12-26 17:34:37,768][105692] Updated weights for policy 0, policy_version 304009 (0.0011) [2023-12-26 17:34:37,816][105692] Updated weights for policy 0, policy_version 304019 (0.0010) [2023-12-26 17:34:37,868][105692] Updated weights for policy 0, policy_version 304029 (0.0010) [2023-12-26 17:34:37,923][105692] Updated weights for policy 0, policy_version 304039 (0.0010) [2023-12-26 17:34:38,059][105620] Updated weights for policy 1, policy_version 304242 (0.0008) [2023-12-26 17:34:38,115][105620] Updated weights for policy 1, policy_version 304252 (0.0008) [2023-12-26 17:34:38,175][105620] Updated weights for policy 1, policy_version 304262 (0.0008) [2023-12-26 17:34:38,684][105692] Updated weights for policy 0, policy_version 304049 (0.0011) [2023-12-26 17:34:38,740][105692] Updated weights for policy 0, policy_version 304059 (0.0010) [2023-12-26 17:34:38,805][105692] Updated weights for policy 0, policy_version 304069 (0.0011) [2023-12-26 17:34:38,879][105620] Updated weights for policy 1, policy_version 304272 (0.0008) [2023-12-26 17:34:38,943][105620] Updated weights for policy 1, policy_version 304282 (0.0008) [2023-12-26 17:34:39,003][105620] Updated weights for policy 1, policy_version 304292 (0.0008) [2023-12-26 17:34:39,024][105586] KL-divergence is very high: 117.5418 [2023-12-26 17:34:39,562][105692] Updated weights for policy 0, policy_version 304079 (0.0011) [2023-12-26 17:34:39,615][105692] Updated weights for policy 0, policy_version 304089 (0.0011) [2023-12-26 17:34:39,677][105692] Updated weights for policy 0, policy_version 304099 (0.0011) [2023-12-26 17:34:39,770][105620] Updated weights for policy 1, policy_version 304302 (0.0009) [2023-12-26 17:34:39,830][105620] Updated weights for policy 1, policy_version 304312 (0.0008) [2023-12-26 17:34:39,871][105586] KL-divergence is very high: 127.3901 [2023-12-26 17:34:39,898][105620] Updated weights for policy 1, policy_version 304322 (0.0008) [2023-12-26 17:34:39,927][105586] KL-divergence is very high: 179.2104 [2023-12-26 17:34:40,424][105692] Updated weights for policy 0, policy_version 304109 (0.0011) [2023-12-26 17:34:40,486][105692] Updated weights for policy 0, policy_version 304119 (0.0010) [2023-12-26 17:34:40,553][105692] Updated weights for policy 0, policy_version 304129 (0.0010) [2023-12-26 17:34:40,571][105620] Updated weights for policy 1, policy_version 304332 (0.0007) [2023-12-26 17:34:40,633][105620] Updated weights for policy 1, policy_version 304342 (0.0007) [2023-12-26 17:34:40,681][105620] Updated weights for policy 1, policy_version 304352 (0.0008) [2023-12-26 17:34:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 155795456. Throughput: 0: 9853.4, 1: 9846.9. Samples: 155803152. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-26 17:34:41,062][104569] Avg episode reward: [(0, '9179.348'), (1, '7970.115')] [2023-12-26 17:34:41,289][105692] Updated weights for policy 0, policy_version 304139 (0.0009) [2023-12-26 17:34:41,341][105692] Updated weights for policy 0, policy_version 304149 (0.0008) [2023-12-26 17:34:41,411][105692] Updated weights for policy 0, policy_version 304159 (0.0007) [2023-12-26 17:34:41,521][105620] Updated weights for policy 1, policy_version 304362 (0.0008) [2023-12-26 17:34:41,588][105620] Updated weights for policy 1, policy_version 304372 (0.0008) [2023-12-26 17:34:41,654][105620] Updated weights for policy 1, policy_version 304382 (0.0008) [2023-12-26 17:34:41,714][105620] Updated weights for policy 1, policy_version 304392 (0.0008) [2023-12-26 17:34:42,149][105692] Updated weights for policy 0, policy_version 304169 (0.0009) [2023-12-26 17:34:42,216][105692] Updated weights for policy 0, policy_version 304179 (0.0009) [2023-12-26 17:34:42,278][105692] Updated weights for policy 0, policy_version 304189 (0.0011) [2023-12-26 17:34:42,342][105692] Updated weights for policy 0, policy_version 304199 (0.0011) [2023-12-26 17:34:42,471][105620] Updated weights for policy 1, policy_version 304402 (0.0006) [2023-12-26 17:34:42,534][105620] Updated weights for policy 1, policy_version 304412 (0.0006) [2023-12-26 17:34:42,607][105620] Updated weights for policy 1, policy_version 304422 (0.0006) [2023-12-26 17:34:42,990][105692] Updated weights for policy 0, policy_version 304209 (0.0009) [2023-12-26 17:34:43,053][105692] Updated weights for policy 0, policy_version 304219 (0.0008) [2023-12-26 17:34:43,116][105692] Updated weights for policy 0, policy_version 304229 (0.0008) [2023-12-26 17:34:43,215][105620] Updated weights for policy 1, policy_version 304432 (0.0009) [2023-12-26 17:34:43,270][105620] Updated weights for policy 1, policy_version 304442 (0.0010) [2023-12-26 17:34:43,325][105620] Updated weights for policy 1, policy_version 304452 (0.0010) [2023-12-26 17:34:43,788][105692] Updated weights for policy 0, policy_version 304239 (0.0009) [2023-12-26 17:34:43,840][105692] Updated weights for policy 0, policy_version 304249 (0.0008) [2023-12-26 17:34:43,897][105692] Updated weights for policy 0, policy_version 304259 (0.0006) [2023-12-26 17:34:43,944][105620] Updated weights for policy 1, policy_version 304462 (0.0007) [2023-12-26 17:34:43,988][105620] Updated weights for policy 1, policy_version 304472 (0.0005) [2023-12-26 17:34:44,036][105620] Updated weights for policy 1, policy_version 304482 (0.0006) [2023-12-26 17:34:44,572][105692] Updated weights for policy 0, policy_version 304269 (0.0007) [2023-12-26 17:34:44,630][105692] Updated weights for policy 0, policy_version 304279 (0.0007) [2023-12-26 17:34:44,687][105692] Updated weights for policy 0, policy_version 304289 (0.0009) [2023-12-26 17:34:44,824][105620] Updated weights for policy 1, policy_version 304492 (0.0006) [2023-12-26 17:34:44,890][105620] Updated weights for policy 1, policy_version 304502 (0.0007) [2023-12-26 17:34:44,956][105620] Updated weights for policy 1, policy_version 304512 (0.0005) [2023-12-26 17:34:45,355][105692] Updated weights for policy 0, policy_version 304299 (0.0009) [2023-12-26 17:34:45,417][105692] Updated weights for policy 0, policy_version 304309 (0.0010) [2023-12-26 17:34:45,481][105692] Updated weights for policy 0, policy_version 304319 (0.0009) [2023-12-26 17:34:45,641][105620] Updated weights for policy 1, policy_version 304522 (0.0006) [2023-12-26 17:34:45,706][105620] Updated weights for policy 1, policy_version 304532 (0.0009) [2023-12-26 17:34:45,770][105620] Updated weights for policy 1, policy_version 304542 (0.0009) [2023-12-26 17:34:45,826][105620] Updated weights for policy 1, policy_version 304552 (0.0005) [2023-12-26 17:34:46,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.9, 300 sec: 19438.6). Total num frames: 155893760. Throughput: 0: 9822.6, 1: 9886.4. Samples: 155862820. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-26 17:34:46,062][104569] Avg episode reward: [(0, '9268.081'), (1, '7880.016')] [2023-12-26 17:34:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000304328_77922304.pth... [2023-12-26 17:34:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000304552_77971456.pth... [2023-12-26 17:34:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000303176_77627392.pth [2023-12-26 17:34:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000303400_77676544.pth [2023-12-26 17:34:46,261][105692] Updated weights for policy 0, policy_version 304329 (0.0009) [2023-12-26 17:34:46,315][105692] Updated weights for policy 0, policy_version 304340 (0.0010) [2023-12-26 17:34:46,367][105692] Updated weights for policy 0, policy_version 304351 (0.0009) [2023-12-26 17:34:46,458][105620] Updated weights for policy 1, policy_version 304562 (0.0009) [2023-12-26 17:34:46,504][105620] Updated weights for policy 1, policy_version 304572 (0.0008) [2023-12-26 17:34:46,562][105620] Updated weights for policy 1, policy_version 304582 (0.0010) [2023-12-26 17:34:47,111][105692] Updated weights for policy 0, policy_version 304361 (0.0009) [2023-12-26 17:34:47,157][105620] Updated weights for policy 1, policy_version 304592 (0.0007) [2023-12-26 17:34:47,160][105692] Updated weights for policy 0, policy_version 304371 (0.0009) [2023-12-26 17:34:47,207][105620] Updated weights for policy 1, policy_version 304602 (0.0006) [2023-12-26 17:34:47,211][105692] Updated weights for policy 0, policy_version 304381 (0.0009) [2023-12-26 17:34:47,266][105620] Updated weights for policy 1, policy_version 304612 (0.0006) [2023-12-26 17:34:47,279][105692] Updated weights for policy 0, policy_version 304391 (0.0009) [2023-12-26 17:34:47,914][105620] Updated weights for policy 1, policy_version 304622 (0.0007) [2023-12-26 17:34:47,929][105692] Updated weights for policy 0, policy_version 304401 (0.0007) [2023-12-26 17:34:47,970][105620] Updated weights for policy 1, policy_version 304632 (0.0009) [2023-12-26 17:34:47,981][105692] Updated weights for policy 0, policy_version 304411 (0.0005) [2023-12-26 17:34:48,023][105620] Updated weights for policy 1, policy_version 304642 (0.0008) [2023-12-26 17:34:48,037][105692] Updated weights for policy 0, policy_version 304421 (0.0005) [2023-12-26 17:34:48,637][105692] Updated weights for policy 0, policy_version 304431 (0.0009) [2023-12-26 17:34:48,692][105692] Updated weights for policy 0, policy_version 304441 (0.0010) [2023-12-26 17:34:48,743][105692] Updated weights for policy 0, policy_version 304451 (0.0009) [2023-12-26 17:34:48,757][105620] Updated weights for policy 1, policy_version 304652 (0.0008) [2023-12-26 17:34:48,818][105620] Updated weights for policy 1, policy_version 304662 (0.0008) [2023-12-26 17:34:48,878][105620] Updated weights for policy 1, policy_version 304672 (0.0008) [2023-12-26 17:34:49,566][105692] Updated weights for policy 0, policy_version 304461 (0.0007) [2023-12-26 17:34:49,612][105620] Updated weights for policy 1, policy_version 304682 (0.0008) [2023-12-26 17:34:49,629][105692] Updated weights for policy 0, policy_version 304471 (0.0009) [2023-12-26 17:34:49,664][105620] Updated weights for policy 1, policy_version 304692 (0.0007) [2023-12-26 17:34:49,689][105692] Updated weights for policy 0, policy_version 304481 (0.0010) [2023-12-26 17:34:49,716][105620] Updated weights for policy 1, policy_version 304702 (0.0006) [2023-12-26 17:34:49,776][105620] Updated weights for policy 1, policy_version 304712 (0.0008) [2023-12-26 17:34:50,467][105620] Updated weights for policy 1, policy_version 304722 (0.0006) [2023-12-26 17:34:50,474][105692] Updated weights for policy 0, policy_version 304491 (0.0006) [2023-12-26 17:34:50,527][105620] Updated weights for policy 1, policy_version 304732 (0.0007) [2023-12-26 17:34:50,537][105692] Updated weights for policy 0, policy_version 304501 (0.0007) [2023-12-26 17:34:50,585][105620] Updated weights for policy 1, policy_version 304742 (0.0008) [2023-12-26 17:34:50,603][105692] Updated weights for policy 0, policy_version 304511 (0.0009) [2023-12-26 17:34:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19438.7). Total num frames: 155992064. Throughput: 0: 9873.2, 1: 9856.8. Samples: 155982796. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-26 17:34:51,062][104569] Avg episode reward: [(0, '9267.725'), (1, '7971.238')] [2023-12-26 17:34:51,306][105620] Updated weights for policy 1, policy_version 304752 (0.0009) [2023-12-26 17:34:51,363][105620] Updated weights for policy 1, policy_version 304762 (0.0008) [2023-12-26 17:34:51,370][105692] Updated weights for policy 0, policy_version 304521 (0.0009) [2023-12-26 17:34:51,432][105620] Updated weights for policy 1, policy_version 304772 (0.0008) [2023-12-26 17:34:51,434][105692] Updated weights for policy 0, policy_version 304531 (0.0006) [2023-12-26 17:34:51,491][105692] Updated weights for policy 0, policy_version 304541 (0.0008) [2023-12-26 17:34:51,549][105692] Updated weights for policy 0, policy_version 304551 (0.0007) [2023-12-26 17:34:52,217][105620] Updated weights for policy 1, policy_version 304782 (0.0007) [2023-12-26 17:34:52,243][105692] Updated weights for policy 0, policy_version 304561 (0.0007) [2023-12-26 17:34:52,276][105620] Updated weights for policy 1, policy_version 304792 (0.0008) [2023-12-26 17:34:52,300][105692] Updated weights for policy 0, policy_version 304571 (0.0008) [2023-12-26 17:34:52,328][105620] Updated weights for policy 1, policy_version 304802 (0.0005) [2023-12-26 17:34:52,363][105692] Updated weights for policy 0, policy_version 304581 (0.0007) [2023-12-26 17:34:53,045][105692] Updated weights for policy 0, policy_version 304591 (0.0009) [2023-12-26 17:34:53,101][105692] Updated weights for policy 0, policy_version 304601 (0.0008) [2023-12-26 17:34:53,128][105620] Updated weights for policy 1, policy_version 304812 (0.0008) [2023-12-26 17:34:53,158][105692] Updated weights for policy 0, policy_version 304611 (0.0007) [2023-12-26 17:34:53,188][105620] Updated weights for policy 1, policy_version 304822 (0.0007) [2023-12-26 17:34:53,241][105620] Updated weights for policy 1, policy_version 304832 (0.0008) [2023-12-26 17:34:53,909][105692] Updated weights for policy 0, policy_version 304621 (0.0008) [2023-12-26 17:34:53,957][105692] Updated weights for policy 0, policy_version 304631 (0.0009) [2023-12-26 17:34:53,989][105620] Updated weights for policy 1, policy_version 304842 (0.0008) [2023-12-26 17:34:54,019][105692] Updated weights for policy 0, policy_version 304641 (0.0008) [2023-12-26 17:34:54,045][105620] Updated weights for policy 1, policy_version 304852 (0.0006) [2023-12-26 17:34:54,105][105620] Updated weights for policy 1, policy_version 304862 (0.0009) [2023-12-26 17:34:54,163][105620] Updated weights for policy 1, policy_version 304872 (0.0008) [2023-12-26 17:34:54,739][105692] Updated weights for policy 0, policy_version 304651 (0.0008) [2023-12-26 17:34:54,797][105692] Updated weights for policy 0, policy_version 304661 (0.0009) [2023-12-26 17:34:54,858][105692] Updated weights for policy 0, policy_version 304671 (0.0009) [2023-12-26 17:34:54,923][105620] Updated weights for policy 1, policy_version 304882 (0.0005) [2023-12-26 17:34:54,994][105620] Updated weights for policy 1, policy_version 304892 (0.0008) [2023-12-26 17:34:55,052][105620] Updated weights for policy 1, policy_version 304902 (0.0009) [2023-12-26 17:34:55,598][105692] Updated weights for policy 0, policy_version 304681 (0.0010) [2023-12-26 17:34:55,659][105692] Updated weights for policy 0, policy_version 304691 (0.0009) [2023-12-26 17:34:55,723][105692] Updated weights for policy 0, policy_version 304701 (0.0008) [2023-12-26 17:34:55,773][105620] Updated weights for policy 1, policy_version 304912 (0.0007) [2023-12-26 17:34:55,779][105692] Updated weights for policy 0, policy_version 304711 (0.0007) [2023-12-26 17:34:55,833][105620] Updated weights for policy 1, policy_version 304922 (0.0008) [2023-12-26 17:34:55,894][105620] Updated weights for policy 1, policy_version 304932 (0.0007) [2023-12-26 17:34:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.9, 300 sec: 19466.4). Total num frames: 156090368. Throughput: 0: 9821.8, 1: 9717.4. Samples: 156095764. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-26 17:34:56,062][104569] Avg episode reward: [(0, '9267.561'), (1, '8154.864')] [2023-12-26 17:34:56,539][105692] Updated weights for policy 0, policy_version 304721 (0.0009) [2023-12-26 17:34:56,562][105620] Updated weights for policy 1, policy_version 304942 (0.0006) [2023-12-26 17:34:56,587][105692] Updated weights for policy 0, policy_version 304731 (0.0008) [2023-12-26 17:34:56,606][105620] Updated weights for policy 1, policy_version 304952 (0.0007) [2023-12-26 17:34:56,642][105692] Updated weights for policy 0, policy_version 304741 (0.0008) [2023-12-26 17:34:56,651][105620] Updated weights for policy 1, policy_version 304962 (0.0007) [2023-12-26 17:34:57,390][105692] Updated weights for policy 0, policy_version 304751 (0.0007) [2023-12-26 17:34:57,412][105620] Updated weights for policy 1, policy_version 304972 (0.0007) [2023-12-26 17:34:57,448][105692] Updated weights for policy 0, policy_version 304761 (0.0007) [2023-12-26 17:34:57,472][105620] Updated weights for policy 1, policy_version 304982 (0.0005) [2023-12-26 17:34:57,504][105692] Updated weights for policy 0, policy_version 304771 (0.0010) [2023-12-26 17:34:57,528][105620] Updated weights for policy 1, policy_version 304992 (0.0006) [2023-12-26 17:34:58,210][105692] Updated weights for policy 0, policy_version 304781 (0.0009) [2023-12-26 17:34:58,256][105620] Updated weights for policy 1, policy_version 305002 (0.0009) [2023-12-26 17:34:58,272][105692] Updated weights for policy 0, policy_version 304791 (0.0009) [2023-12-26 17:34:58,325][105620] Updated weights for policy 1, policy_version 305012 (0.0008) [2023-12-26 17:34:58,338][105692] Updated weights for policy 0, policy_version 304801 (0.0009) [2023-12-26 17:34:58,389][105620] Updated weights for policy 1, policy_version 305022 (0.0008) [2023-12-26 17:34:58,458][105620] Updated weights for policy 1, policy_version 305032 (0.0008) [2023-12-26 17:34:59,087][105692] Updated weights for policy 0, policy_version 304811 (0.0010) [2023-12-26 17:34:59,151][105692] Updated weights for policy 0, policy_version 304821 (0.0007) [2023-12-26 17:34:59,221][105692] Updated weights for policy 0, policy_version 304831 (0.0008) [2023-12-26 17:34:59,251][105620] Updated weights for policy 1, policy_version 305042 (0.0009) [2023-12-26 17:34:59,316][105620] Updated weights for policy 1, policy_version 305052 (0.0010) [2023-12-26 17:34:59,381][105620] Updated weights for policy 1, policy_version 305062 (0.0009) [2023-12-26 17:34:59,877][105692] Updated weights for policy 0, policy_version 304841 (0.0009) [2023-12-26 17:34:59,935][105692] Updated weights for policy 0, policy_version 304851 (0.0009) [2023-12-26 17:34:59,996][105692] Updated weights for policy 0, policy_version 304861 (0.0009) [2023-12-26 17:35:00,048][105692] Updated weights for policy 0, policy_version 304871 (0.0005) [2023-12-26 17:35:00,234][105620] Updated weights for policy 1, policy_version 305072 (0.0010) [2023-12-26 17:35:00,268][105586] KL-divergence is very high: 164.8334 [2023-12-26 17:35:00,286][105620] Updated weights for policy 1, policy_version 305082 (0.0009) [2023-12-26 17:35:00,307][105586] KL-divergence is very high: 269.2869 [2023-12-26 17:35:00,340][105620] Updated weights for policy 1, policy_version 305093 (0.0010) [2023-12-26 17:35:00,348][105586] KL-divergence is very high: 291.6336 [2023-12-26 17:35:00,648][105692] Updated weights for policy 0, policy_version 304881 (0.0008) [2023-12-26 17:35:00,709][105692] Updated weights for policy 0, policy_version 304891 (0.0009) [2023-12-26 17:35:00,774][105692] Updated weights for policy 0, policy_version 304901 (0.0006) [2023-12-26 17:35:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 156180480. Throughput: 0: 9825.2, 1: 9705.4. Samples: 156152116. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-26 17:35:01,063][104569] Avg episode reward: [(0, '9267.730'), (1, '8153.074')] [2023-12-26 17:35:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000304904_78069760.pth... [2023-12-26 17:35:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000305096_78110720.pth... [2023-12-26 17:35:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000303752_77774848.pth [2023-12-26 17:35:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000303976_77824000.pth [2023-12-26 17:35:01,173][105620] Updated weights for policy 1, policy_version 305103 (0.0010) [2023-12-26 17:35:01,231][105620] Updated weights for policy 1, policy_version 305113 (0.0008) [2023-12-26 17:35:01,297][105620] Updated weights for policy 1, policy_version 305123 (0.0009) [2023-12-26 17:35:01,490][105692] Updated weights for policy 0, policy_version 304911 (0.0005) [2023-12-26 17:35:01,558][105692] Updated weights for policy 0, policy_version 304921 (0.0009) [2023-12-26 17:35:01,625][105692] Updated weights for policy 0, policy_version 304931 (0.0010) [2023-12-26 17:35:02,086][105620] Updated weights for policy 1, policy_version 305133 (0.0009) [2023-12-26 17:35:02,143][105620] Updated weights for policy 1, policy_version 305143 (0.0008) [2023-12-26 17:35:02,204][105620] Updated weights for policy 1, policy_version 305153 (0.0009) [2023-12-26 17:35:02,291][105692] Updated weights for policy 0, policy_version 304941 (0.0009) [2023-12-26 17:35:02,347][105692] Updated weights for policy 0, policy_version 304951 (0.0009) [2023-12-26 17:35:02,406][105692] Updated weights for policy 0, policy_version 304961 (0.0009) [2023-12-26 17:35:02,955][105620] Updated weights for policy 1, policy_version 305163 (0.0009) [2023-12-26 17:35:03,016][105620] Updated weights for policy 1, policy_version 305173 (0.0008) [2023-12-26 17:35:03,077][105620] Updated weights for policy 1, policy_version 305183 (0.0008) [2023-12-26 17:35:03,157][105692] Updated weights for policy 0, policy_version 304971 (0.0009) [2023-12-26 17:35:03,211][105692] Updated weights for policy 0, policy_version 304981 (0.0009) [2023-12-26 17:35:03,261][105692] Updated weights for policy 0, policy_version 304991 (0.0009) [2023-12-26 17:35:03,791][105620] Updated weights for policy 1, policy_version 305193 (0.0009) [2023-12-26 17:35:03,841][105620] Updated weights for policy 1, policy_version 305203 (0.0008) [2023-12-26 17:35:03,901][105620] Updated weights for policy 1, policy_version 305213 (0.0009) [2023-12-26 17:35:03,959][105620] Updated weights for policy 1, policy_version 305223 (0.0009) [2023-12-26 17:35:04,048][105692] Updated weights for policy 0, policy_version 305001 (0.0009) [2023-12-26 17:35:04,109][105692] Updated weights for policy 0, policy_version 305011 (0.0010) [2023-12-26 17:35:04,168][105692] Updated weights for policy 0, policy_version 305021 (0.0010) [2023-12-26 17:35:04,228][105692] Updated weights for policy 0, policy_version 305031 (0.0009) [2023-12-26 17:35:04,628][105620] Updated weights for policy 1, policy_version 305233 (0.0008) [2023-12-26 17:35:04,691][105620] Updated weights for policy 1, policy_version 305243 (0.0008) [2023-12-26 17:35:04,748][105620] Updated weights for policy 1, policy_version 305253 (0.0009) [2023-12-26 17:35:05,040][105692] Updated weights for policy 0, policy_version 305041 (0.0010) [2023-12-26 17:35:05,087][105692] Updated weights for policy 0, policy_version 305052 (0.0009) [2023-12-26 17:35:05,145][105692] Updated weights for policy 0, policy_version 305062 (0.0009) [2023-12-26 17:35:05,454][105620] Updated weights for policy 1, policy_version 305263 (0.0009) [2023-12-26 17:35:05,518][105620] Updated weights for policy 1, policy_version 305273 (0.0009) [2023-12-26 17:35:05,576][105620] Updated weights for policy 1, policy_version 305283 (0.0009) [2023-12-26 17:35:05,906][105692] Updated weights for policy 0, policy_version 305072 (0.0009) [2023-12-26 17:35:05,968][105692] Updated weights for policy 0, policy_version 305082 (0.0009) [2023-12-26 17:35:06,030][105692] Updated weights for policy 0, policy_version 305092 (0.0009) [2023-12-26 17:35:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19438.7). Total num frames: 156278784. Throughput: 0: 9752.4, 1: 9694.3. Samples: 156264976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:35:06,062][104569] Avg episode reward: [(0, '9359.023'), (1, '8153.969')] [2023-12-26 17:35:06,332][105620] Updated weights for policy 1, policy_version 305293 (0.0009) [2023-12-26 17:35:06,390][105620] Updated weights for policy 1, policy_version 305303 (0.0009) [2023-12-26 17:35:06,452][105620] Updated weights for policy 1, policy_version 305313 (0.0009) [2023-12-26 17:35:06,823][105692] Updated weights for policy 0, policy_version 305102 (0.0009) [2023-12-26 17:35:06,871][105692] Updated weights for policy 0, policy_version 305112 (0.0009) [2023-12-26 17:35:06,919][105692] Updated weights for policy 0, policy_version 305122 (0.0009) [2023-12-26 17:35:07,216][105620] Updated weights for policy 1, policy_version 305323 (0.0009) [2023-12-26 17:35:07,270][105620] Updated weights for policy 1, policy_version 305333 (0.0009) [2023-12-26 17:35:07,320][105620] Updated weights for policy 1, policy_version 305343 (0.0008) [2023-12-26 17:35:07,739][105692] Updated weights for policy 0, policy_version 305132 (0.0008) [2023-12-26 17:35:07,797][105692] Updated weights for policy 0, policy_version 305142 (0.0005) [2023-12-26 17:35:07,850][105692] Updated weights for policy 0, policy_version 305152 (0.0005) [2023-12-26 17:35:08,046][105620] Updated weights for policy 1, policy_version 305353 (0.0009) [2023-12-26 17:35:08,103][105620] Updated weights for policy 1, policy_version 305363 (0.0006) [2023-12-26 17:35:08,151][105620] Updated weights for policy 1, policy_version 305373 (0.0005) [2023-12-26 17:35:08,216][105620] Updated weights for policy 1, policy_version 305383 (0.0008) [2023-12-26 17:35:08,536][105692] Updated weights for policy 0, policy_version 305162 (0.0006) [2023-12-26 17:35:08,595][105692] Updated weights for policy 0, policy_version 305172 (0.0010) [2023-12-26 17:35:08,649][105692] Updated weights for policy 0, policy_version 305182 (0.0008) [2023-12-26 17:35:08,699][105692] Updated weights for policy 0, policy_version 305192 (0.0009) [2023-12-26 17:35:08,913][105586] KL-divergence is very high: 496.3917 [2023-12-26 17:35:08,919][105620] Updated weights for policy 1, policy_version 305393 (0.0008) [2023-12-26 17:35:08,955][105586] KL-divergence is very high: 785.9022 [2023-12-26 17:35:08,974][105620] Updated weights for policy 1, policy_version 305403 (0.0009) [2023-12-26 17:35:08,998][105586] KL-divergence is very high: 691.6540 [2023-12-26 17:35:09,024][105620] Updated weights for policy 1, policy_version 305413 (0.0009) [2023-12-26 17:35:09,484][105692] Updated weights for policy 0, policy_version 305202 (0.0009) [2023-12-26 17:35:09,538][105692] Updated weights for policy 0, policy_version 305212 (0.0008) [2023-12-26 17:35:09,590][105692] Updated weights for policy 0, policy_version 305222 (0.0008) [2023-12-26 17:35:09,793][105620] Updated weights for policy 1, policy_version 305423 (0.0007) [2023-12-26 17:35:09,810][105586] KL-divergence is very high: 102.3708 [2023-12-26 17:35:09,824][105586] KL-divergence is very high: 280.1314 [2023-12-26 17:35:09,858][105620] Updated weights for policy 1, policy_version 305433 (0.0007) [2023-12-26 17:35:09,865][105586] KL-divergence is very high: 137.5631 [2023-12-26 17:35:09,879][105586] KL-divergence is very high: 243.2510 [2023-12-26 17:35:09,927][105620] Updated weights for policy 1, policy_version 305443 (0.0006) [2023-12-26 17:35:10,446][105692] Updated weights for policy 0, policy_version 305232 (0.0009) [2023-12-26 17:35:10,505][105692] Updated weights for policy 0, policy_version 305242 (0.0010) [2023-12-26 17:35:10,562][105692] Updated weights for policy 0, policy_version 305252 (0.0007) [2023-12-26 17:35:10,606][105620] Updated weights for policy 1, policy_version 305453 (0.0008) [2023-12-26 17:35:10,662][105620] Updated weights for policy 1, policy_version 305463 (0.0007) [2023-12-26 17:35:10,718][105620] Updated weights for policy 1, policy_version 305473 (0.0009) [2023-12-26 17:35:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19438.7). Total num frames: 156368896. Throughput: 0: 9704.4, 1: 9675.0. Samples: 156376784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:35:11,063][104569] Avg episode reward: [(0, '9359.109'), (1, '7441.136')] [2023-12-26 17:35:11,241][105692] Updated weights for policy 0, policy_version 305262 (0.0007) [2023-12-26 17:35:11,302][105692] Updated weights for policy 0, policy_version 305272 (0.0008) [2023-12-26 17:35:11,370][105692] Updated weights for policy 0, policy_version 305282 (0.0008) [2023-12-26 17:35:11,486][105620] Updated weights for policy 1, policy_version 305483 (0.0009) [2023-12-26 17:35:11,543][105620] Updated weights for policy 1, policy_version 305493 (0.0005) [2023-12-26 17:35:11,605][105620] Updated weights for policy 1, policy_version 305503 (0.0006) [2023-12-26 17:35:12,112][105692] Updated weights for policy 0, policy_version 305292 (0.0008) [2023-12-26 17:35:12,173][105692] Updated weights for policy 0, policy_version 305302 (0.0006) [2023-12-26 17:35:12,228][105692] Updated weights for policy 0, policy_version 305312 (0.0007) [2023-12-26 17:35:12,278][105620] Updated weights for policy 1, policy_version 305513 (0.0007) [2023-12-26 17:35:12,343][105620] Updated weights for policy 1, policy_version 305523 (0.0007) [2023-12-26 17:35:12,405][105620] Updated weights for policy 1, policy_version 305533 (0.0007) [2023-12-26 17:35:12,462][105620] Updated weights for policy 1, policy_version 305543 (0.0005) [2023-12-26 17:35:12,855][105692] Updated weights for policy 0, policy_version 305322 (0.0008) [2023-12-26 17:35:12,917][105692] Updated weights for policy 0, policy_version 305332 (0.0005) [2023-12-26 17:35:12,974][105692] Updated weights for policy 0, policy_version 305342 (0.0005) [2023-12-26 17:35:13,038][105692] Updated weights for policy 0, policy_version 305352 (0.0005) [2023-12-26 17:35:13,169][105620] Updated weights for policy 1, policy_version 305553 (0.0008) [2023-12-26 17:35:13,219][105586] KL-divergence is very high: 522.9240 [2023-12-26 17:35:13,223][105620] Updated weights for policy 1, policy_version 305563 (0.0008) [2023-12-26 17:35:13,242][105586] KL-divergence is very high: 864.0264 [2023-12-26 17:35:13,254][105586] KL-divergence is very high: 252.3201 [2023-12-26 17:35:13,266][105586] KL-divergence is very high: 1328.9393 [2023-12-26 17:35:13,284][105620] Updated weights for policy 1, policy_version 305573 (0.0008) [2023-12-26 17:35:13,290][105586] KL-divergence is very high: 993.7063 [2023-12-26 17:35:13,610][105692] Updated weights for policy 0, policy_version 305362 (0.0010) [2023-12-26 17:35:13,672][105692] Updated weights for policy 0, policy_version 305372 (0.0007) [2023-12-26 17:35:13,740][105692] Updated weights for policy 0, policy_version 305382 (0.0005) [2023-12-26 17:35:14,110][105586] KL-divergence is very high: 419.2524 [2023-12-26 17:35:14,115][105586] KL-divergence is very high: 406.0024 [2023-12-26 17:35:14,143][105620] Updated weights for policy 1, policy_version 305583 (0.0010) [2023-12-26 17:35:14,153][105586] KL-divergence is very high: 266.5293 [2023-12-26 17:35:14,159][105586] KL-divergence is very high: 247.7982 [2023-12-26 17:35:14,165][105586] KL-divergence is very high: 101.9293 [2023-12-26 17:35:14,170][105586] KL-divergence is very high: 143.8008 [2023-12-26 17:35:14,175][105586] KL-divergence is very high: 100.6486 [2023-12-26 17:35:14,196][105620] Updated weights for policy 1, policy_version 305593 (0.0009) [2023-12-26 17:35:14,197][105586] KL-divergence is very high: 194.2335 [2023-12-26 17:35:14,202][105586] KL-divergence is very high: 197.9225 [2023-12-26 17:35:14,207][105586] KL-divergence is very high: 116.1381 [2023-12-26 17:35:14,211][105586] KL-divergence is very high: 141.0933 [2023-12-26 17:35:14,239][105586] KL-divergence is very high: 241.6833 [2023-12-26 17:35:14,245][105586] KL-divergence is very high: 246.2546 [2023-12-26 17:35:14,251][105620] Updated weights for policy 1, policy_version 305603 (0.0009) [2023-12-26 17:35:14,282][105692] Updated weights for policy 0, policy_version 305392 (0.0009) [2023-12-26 17:35:14,347][105692] Updated weights for policy 0, policy_version 305402 (0.0010) [2023-12-26 17:35:14,406][105692] Updated weights for policy 0, policy_version 305412 (0.0007) [2023-12-26 17:35:14,964][105692] Updated weights for policy 0, policy_version 305422 (0.0006) [2023-12-26 17:35:15,025][105692] Updated weights for policy 0, policy_version 305432 (0.0006) [2023-12-26 17:35:15,081][105692] Updated weights for policy 0, policy_version 305442 (0.0005) [2023-12-26 17:35:15,142][105620] Updated weights for policy 1, policy_version 305613 (0.0006) [2023-12-26 17:35:15,199][105620] Updated weights for policy 1, policy_version 305623 (0.0005) [2023-12-26 17:35:15,257][105620] Updated weights for policy 1, policy_version 305633 (0.0010) [2023-12-26 17:35:15,761][105692] Updated weights for policy 0, policy_version 305452 (0.0006) [2023-12-26 17:35:15,814][105692] Updated weights for policy 0, policy_version 305462 (0.0005) [2023-12-26 17:35:15,867][105692] Updated weights for policy 0, policy_version 305472 (0.0005) [2023-12-26 17:35:15,907][105620] Updated weights for policy 1, policy_version 305643 (0.0011) [2023-12-26 17:35:15,964][105620] Updated weights for policy 1, policy_version 305653 (0.0010) [2023-12-26 17:35:16,033][105620] Updated weights for policy 1, policy_version 305663 (0.0011) [2023-12-26 17:35:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 156467200. Throughput: 0: 9717.5, 1: 9587.0. Samples: 156436944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:35:16,062][104569] Avg episode reward: [(0, '9359.263'), (1, '6288.381')] [2023-12-26 17:35:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000305480_78217216.pth... [2023-12-26 17:35:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000304328_77922304.pth [2023-12-26 17:35:16,089][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000305672_78258176.pth... [2023-12-26 17:35:16,092][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000304552_77971456.pth [2023-12-26 17:35:16,396][105692] Updated weights for policy 0, policy_version 305482 (0.0005) [2023-12-26 17:35:16,448][105692] Updated weights for policy 0, policy_version 305492 (0.0007) [2023-12-26 17:35:16,510][105692] Updated weights for policy 0, policy_version 305502 (0.0006) [2023-12-26 17:35:16,579][105692] Updated weights for policy 0, policy_version 305512 (0.0005) [2023-12-26 17:35:16,729][105620] Updated weights for policy 1, policy_version 305673 (0.0011) [2023-12-26 17:35:16,782][105620] Updated weights for policy 1, policy_version 305683 (0.0011) [2023-12-26 17:35:16,815][105586] KL-divergence is very high: 104.4648 [2023-12-26 17:35:16,830][105620] Updated weights for policy 1, policy_version 305693 (0.0010) [2023-12-26 17:35:16,878][105620] Updated weights for policy 1, policy_version 305703 (0.0010) [2023-12-26 17:35:17,187][105692] Updated weights for policy 0, policy_version 305522 (0.0009) [2023-12-26 17:35:17,242][105692] Updated weights for policy 0, policy_version 305532 (0.0011) [2023-12-26 17:35:17,290][105692] Updated weights for policy 0, policy_version 305542 (0.0010) [2023-12-26 17:35:17,624][105620] Updated weights for policy 1, policy_version 305713 (0.0010) [2023-12-26 17:35:17,683][105620] Updated weights for policy 1, policy_version 305723 (0.0010) [2023-12-26 17:35:17,744][105620] Updated weights for policy 1, policy_version 305733 (0.0007) [2023-12-26 17:35:17,883][105692] Updated weights for policy 0, policy_version 305552 (0.0006) [2023-12-26 17:35:17,924][105585] KL-divergence is very high: 169.8522 [2023-12-26 17:35:17,946][105692] Updated weights for policy 0, policy_version 305562 (0.0005) [2023-12-26 17:35:17,982][105585] KL-divergence is very high: 128.7236 [2023-12-26 17:35:18,012][105692] Updated weights for policy 0, policy_version 305572 (0.0005) [2023-12-26 17:35:18,307][105620] Updated weights for policy 1, policy_version 305743 (0.0005) [2023-12-26 17:35:18,369][105620] Updated weights for policy 1, policy_version 305753 (0.0010) [2023-12-26 17:35:18,428][105620] Updated weights for policy 1, policy_version 305763 (0.0011) [2023-12-26 17:35:18,554][105692] Updated weights for policy 0, policy_version 305582 (0.0005) [2023-12-26 17:35:18,611][105692] Updated weights for policy 0, policy_version 305592 (0.0006) [2023-12-26 17:35:18,670][105692] Updated weights for policy 0, policy_version 305602 (0.0008) [2023-12-26 17:35:19,047][105620] Updated weights for policy 1, policy_version 305773 (0.0006) [2023-12-26 17:35:19,106][105620] Updated weights for policy 1, policy_version 305783 (0.0009) [2023-12-26 17:35:19,165][105620] Updated weights for policy 1, policy_version 305793 (0.0011) [2023-12-26 17:35:19,437][105692] Updated weights for policy 0, policy_version 305612 (0.0009) [2023-12-26 17:35:19,503][105692] Updated weights for policy 0, policy_version 305622 (0.0007) [2023-12-26 17:35:19,564][105692] Updated weights for policy 0, policy_version 305632 (0.0008) [2023-12-26 17:35:19,903][105620] Updated weights for policy 1, policy_version 305803 (0.0009) [2023-12-26 17:35:19,963][105620] Updated weights for policy 1, policy_version 305813 (0.0010) [2023-12-26 17:35:20,020][105620] Updated weights for policy 1, policy_version 305823 (0.0011) [2023-12-26 17:35:20,285][105692] Updated weights for policy 0, policy_version 305642 (0.0008) [2023-12-26 17:35:20,350][105692] Updated weights for policy 0, policy_version 305652 (0.0007) [2023-12-26 17:35:20,410][105692] Updated weights for policy 0, policy_version 305662 (0.0009) [2023-12-26 17:35:20,470][105692] Updated weights for policy 0, policy_version 305672 (0.0008) [2023-12-26 17:35:20,786][105620] Updated weights for policy 1, policy_version 305833 (0.0011) [2023-12-26 17:35:20,832][105620] Updated weights for policy 1, policy_version 305843 (0.0008) [2023-12-26 17:35:20,884][105620] Updated weights for policy 1, policy_version 305853 (0.0005) [2023-12-26 17:35:20,939][105620] Updated weights for policy 1, policy_version 305863 (0.0005) [2023-12-26 17:35:21,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 156573696. Throughput: 0: 9876.8, 1: 9643.1. Samples: 156562792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:35:21,062][104569] Avg episode reward: [(0, '9269.398'), (1, '4074.871')] [2023-12-26 17:35:21,244][105692] Updated weights for policy 0, policy_version 305682 (0.0008) [2023-12-26 17:35:21,305][105692] Updated weights for policy 0, policy_version 305692 (0.0009) [2023-12-26 17:35:21,376][105692] Updated weights for policy 0, policy_version 305702 (0.0009) [2023-12-26 17:35:21,634][105620] Updated weights for policy 1, policy_version 305873 (0.0010) [2023-12-26 17:35:21,664][105586] KL-divergence is very high: 175.3681 [2023-12-26 17:35:21,670][105586] KL-divergence is very high: 103.2291 [2023-12-26 17:35:21,686][105586] KL-divergence is very high: 177.4306 [2023-12-26 17:35:21,694][105620] Updated weights for policy 1, policy_version 305883 (0.0011) [2023-12-26 17:35:21,719][105586] KL-divergence is very high: 270.0304 [2023-12-26 17:35:21,727][105586] KL-divergence is very high: 132.1208 [2023-12-26 17:35:21,749][105586] KL-divergence is very high: 182.9284 [2023-12-26 17:35:21,772][105620] Updated weights for policy 1, policy_version 305893 (0.0011) [2023-12-26 17:35:21,781][105586] KL-divergence is very high: 238.5120 [2023-12-26 17:35:22,076][105692] Updated weights for policy 0, policy_version 305712 (0.0008) [2023-12-26 17:35:22,140][105692] Updated weights for policy 0, policy_version 305722 (0.0006) [2023-12-26 17:35:22,199][105692] Updated weights for policy 0, policy_version 305732 (0.0006) [2023-12-26 17:35:22,517][105586] KL-divergence is very high: 154.2444 [2023-12-26 17:35:22,546][105586] KL-divergence is very high: 141.3877 [2023-12-26 17:35:22,557][105620] Updated weights for policy 1, policy_version 305903 (0.0011) [2023-12-26 17:35:22,559][105586] KL-divergence is very high: 267.7776 [2023-12-26 17:35:22,565][105586] KL-divergence is very high: 628.0237 [2023-12-26 17:35:22,571][105586] KL-divergence is very high: 763.6458 [2023-12-26 17:35:22,578][105586] KL-divergence is very high: 397.7040 [2023-12-26 17:35:22,593][105586] KL-divergence is very high: 561.1190 [2023-12-26 17:35:22,600][105586] KL-divergence is very high: 424.7270 [2023-12-26 17:35:22,613][105586] KL-divergence is very high: 457.0060 [2023-12-26 17:35:22,619][105586] KL-divergence is very high: 846.1761 [2023-12-26 17:35:22,625][105620] Updated weights for policy 1, policy_version 305913 (0.0011) [2023-12-26 17:35:22,626][105586] KL-divergence is very high: 950.8881 [2023-12-26 17:35:22,632][105586] KL-divergence is very high: 394.0221 [2023-12-26 17:35:22,644][105586] KL-divergence is very high: 401.2211 [2023-12-26 17:35:22,650][105586] KL-divergence is very high: 308.7336 [2023-12-26 17:35:22,662][105586] KL-divergence is very high: 240.7800 [2023-12-26 17:35:22,669][105586] KL-divergence is very high: 543.6417 [2023-12-26 17:35:22,677][105586] KL-divergence is very high: 562.8765 [2023-12-26 17:35:22,684][105586] KL-divergence is very high: 206.7939 [2023-12-26 17:35:22,690][105620] Updated weights for policy 1, policy_version 305923 (0.0008) [2023-12-26 17:35:22,699][105586] KL-divergence is very high: 118.0648 [2023-12-26 17:35:22,705][105586] KL-divergence is very high: 134.1004 [2023-12-26 17:35:22,873][105692] Updated weights for policy 0, policy_version 305742 (0.0007) [2023-12-26 17:35:22,925][105692] Updated weights for policy 0, policy_version 305752 (0.0008) [2023-12-26 17:35:22,971][105692] Updated weights for policy 0, policy_version 305762 (0.0008) [2023-12-26 17:35:23,416][105586] KL-divergence is very high: 166.6363 [2023-12-26 17:35:23,435][105586] KL-divergence is very high: 132.8131 [2023-12-26 17:35:23,445][105620] Updated weights for policy 1, policy_version 305933 (0.0011) [2023-12-26 17:35:23,482][105586] KL-divergence is very high: 208.3898 [2023-12-26 17:35:23,511][105620] Updated weights for policy 1, policy_version 305943 (0.0011) [2023-12-26 17:35:23,529][105586] KL-divergence is very high: 189.5532 [2023-12-26 17:35:23,560][105620] Updated weights for policy 1, policy_version 305953 (0.0010) [2023-12-26 17:35:23,571][105586] KL-divergence is very high: 150.4007 [2023-12-26 17:35:23,602][105692] Updated weights for policy 0, policy_version 305772 (0.0008) [2023-12-26 17:35:23,657][105692] Updated weights for policy 0, policy_version 305782 (0.0007) [2023-12-26 17:35:23,715][105692] Updated weights for policy 0, policy_version 305792 (0.0006) [2023-12-26 17:35:24,266][105692] Updated weights for policy 0, policy_version 305802 (0.0008) [2023-12-26 17:35:24,320][105692] Updated weights for policy 0, policy_version 305812 (0.0006) [2023-12-26 17:35:24,355][105620] Updated weights for policy 1, policy_version 305963 (0.0008) [2023-12-26 17:35:24,376][105692] Updated weights for policy 0, policy_version 305822 (0.0007) [2023-12-26 17:35:24,420][105620] Updated weights for policy 1, policy_version 305973 (0.0009) [2023-12-26 17:35:24,431][105692] Updated weights for policy 0, policy_version 305832 (0.0007) [2023-12-26 17:35:24,475][105620] Updated weights for policy 1, policy_version 305983 (0.0009) [2023-12-26 17:35:25,020][105692] Updated weights for policy 0, policy_version 305842 (0.0005) [2023-12-26 17:35:25,066][105692] Updated weights for policy 0, policy_version 305852 (0.0005) [2023-12-26 17:35:25,112][105692] Updated weights for policy 0, policy_version 305862 (0.0005) [2023-12-26 17:35:25,202][105620] Updated weights for policy 1, policy_version 305993 (0.0008) [2023-12-26 17:35:25,253][105620] Updated weights for policy 1, policy_version 306003 (0.0010) [2023-12-26 17:35:25,306][105620] Updated weights for policy 1, policy_version 306013 (0.0010) [2023-12-26 17:35:25,354][105620] Updated weights for policy 1, policy_version 306023 (0.0010) [2023-12-26 17:35:25,758][105692] Updated weights for policy 0, policy_version 305872 (0.0010) [2023-12-26 17:35:25,816][105692] Updated weights for policy 0, policy_version 305882 (0.0010) [2023-12-26 17:35:25,875][105692] Updated weights for policy 0, policy_version 305892 (0.0010) [2023-12-26 17:35:25,931][105620] Updated weights for policy 1, policy_version 306033 (0.0006) [2023-12-26 17:35:25,990][105620] Updated weights for policy 1, policy_version 306043 (0.0005) [2023-12-26 17:35:26,047][105620] Updated weights for policy 1, policy_version 306053 (0.0005) [2023-12-26 17:35:26,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 156672000. Throughput: 0: 9948.1, 1: 9608.5. Samples: 156683200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:35:26,063][104569] Avg episode reward: [(0, '9269.352'), (1, '3828.920')] [2023-12-26 17:35:26,573][105620] Updated weights for policy 1, policy_version 306063 (0.0006) [2023-12-26 17:35:26,611][105692] Updated weights for policy 0, policy_version 305902 (0.0010) [2023-12-26 17:35:26,626][105620] Updated weights for policy 1, policy_version 306073 (0.0008) [2023-12-26 17:35:26,661][105692] Updated weights for policy 0, policy_version 305912 (0.0010) [2023-12-26 17:35:26,683][105620] Updated weights for policy 1, policy_version 306083 (0.0010) [2023-12-26 17:35:26,716][105692] Updated weights for policy 0, policy_version 305922 (0.0010) [2023-12-26 17:35:27,335][105620] Updated weights for policy 1, policy_version 306093 (0.0006) [2023-12-26 17:35:27,390][105620] Updated weights for policy 1, policy_version 306103 (0.0008) [2023-12-26 17:35:27,432][105692] Updated weights for policy 0, policy_version 305932 (0.0010) [2023-12-26 17:35:27,437][105620] Updated weights for policy 1, policy_version 306113 (0.0008) [2023-12-26 17:35:27,491][105692] Updated weights for policy 0, policy_version 305942 (0.0010) [2023-12-26 17:35:27,558][105692] Updated weights for policy 0, policy_version 305952 (0.0010) [2023-12-26 17:35:28,075][105620] Updated weights for policy 1, policy_version 306123 (0.0007) [2023-12-26 17:35:28,121][105620] Updated weights for policy 1, policy_version 306133 (0.0007) [2023-12-26 17:35:28,169][105620] Updated weights for policy 1, policy_version 306143 (0.0005) [2023-12-26 17:35:28,289][105692] Updated weights for policy 0, policy_version 305962 (0.0010) [2023-12-26 17:35:28,360][105692] Updated weights for policy 0, policy_version 305972 (0.0010) [2023-12-26 17:35:28,418][105692] Updated weights for policy 0, policy_version 305982 (0.0010) [2023-12-26 17:35:28,466][105692] Updated weights for policy 0, policy_version 305992 (0.0010) [2023-12-26 17:35:28,780][105620] Updated weights for policy 1, policy_version 306153 (0.0005) [2023-12-26 17:35:28,840][105620] Updated weights for policy 1, policy_version 306163 (0.0006) [2023-12-26 17:35:28,893][105620] Updated weights for policy 1, policy_version 306173 (0.0008) [2023-12-26 17:35:28,944][105620] Updated weights for policy 1, policy_version 306183 (0.0008) [2023-12-26 17:35:29,208][105692] Updated weights for policy 0, policy_version 306002 (0.0010) [2023-12-26 17:35:29,270][105692] Updated weights for policy 0, policy_version 306012 (0.0009) [2023-12-26 17:35:29,324][105692] Updated weights for policy 0, policy_version 306022 (0.0008) [2023-12-26 17:35:29,639][105620] Updated weights for policy 1, policy_version 306193 (0.0007) [2023-12-26 17:35:29,696][105620] Updated weights for policy 1, policy_version 306203 (0.0006) [2023-12-26 17:35:29,758][105620] Updated weights for policy 1, policy_version 306213 (0.0008) [2023-12-26 17:35:30,084][105692] Updated weights for policy 0, policy_version 306032 (0.0007) [2023-12-26 17:35:30,142][105692] Updated weights for policy 0, policy_version 306042 (0.0006) [2023-12-26 17:35:30,200][105692] Updated weights for policy 0, policy_version 306052 (0.0006) [2023-12-26 17:35:30,566][105620] Updated weights for policy 1, policy_version 306223 (0.0008) [2023-12-26 17:35:30,622][105620] Updated weights for policy 1, policy_version 306233 (0.0008) [2023-12-26 17:35:30,679][105620] Updated weights for policy 1, policy_version 306243 (0.0007) [2023-12-26 17:35:30,784][105692] Updated weights for policy 0, policy_version 306062 (0.0008) [2023-12-26 17:35:30,842][105692] Updated weights for policy 0, policy_version 306072 (0.0011) [2023-12-26 17:35:30,901][105692] Updated weights for policy 0, policy_version 306082 (0.0011) [2023-12-26 17:35:31,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 156778496. Throughput: 0: 9920.7, 1: 9690.1. Samples: 156745308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:35:31,063][104569] Avg episode reward: [(0, '9359.428'), (1, '6050.541')] [2023-12-26 17:35:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000306088_78372864.pth... [2023-12-26 17:35:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000306248_78405632.pth... [2023-12-26 17:35:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000305096_78110720.pth [2023-12-26 17:35:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000304904_78069760.pth [2023-12-26 17:35:31,404][105620] Updated weights for policy 1, policy_version 306253 (0.0009) [2023-12-26 17:35:31,460][105620] Updated weights for policy 1, policy_version 306263 (0.0008) [2023-12-26 17:35:31,512][105620] Updated weights for policy 1, policy_version 306273 (0.0008) [2023-12-26 17:35:31,636][105692] Updated weights for policy 0, policy_version 306092 (0.0011) [2023-12-26 17:35:31,696][105692] Updated weights for policy 0, policy_version 306102 (0.0011) [2023-12-26 17:35:31,764][105692] Updated weights for policy 0, policy_version 306112 (0.0010) [2023-12-26 17:35:32,160][105620] Updated weights for policy 1, policy_version 306283 (0.0008) [2023-12-26 17:35:32,213][105620] Updated weights for policy 1, policy_version 306293 (0.0005) [2023-12-26 17:35:32,279][105620] Updated weights for policy 1, policy_version 306303 (0.0007) [2023-12-26 17:35:32,506][105692] Updated weights for policy 0, policy_version 306122 (0.0011) [2023-12-26 17:35:32,569][105692] Updated weights for policy 0, policy_version 306132 (0.0011) [2023-12-26 17:35:32,635][105692] Updated weights for policy 0, policy_version 306142 (0.0011) [2023-12-26 17:35:32,705][105692] Updated weights for policy 0, policy_version 306152 (0.0011) [2023-12-26 17:35:32,923][105620] Updated weights for policy 1, policy_version 306313 (0.0009) [2023-12-26 17:35:32,972][105620] Updated weights for policy 1, policy_version 306323 (0.0005) [2023-12-26 17:35:33,030][105620] Updated weights for policy 1, policy_version 306333 (0.0005) [2023-12-26 17:35:33,049][105586] KL-divergence is very high: 422.7333 [2023-12-26 17:35:33,088][105586] KL-divergence is very high: 128.1731 [2023-12-26 17:35:33,089][105620] Updated weights for policy 1, policy_version 306343 (0.0008) [2023-12-26 17:35:33,376][105692] Updated weights for policy 0, policy_version 306162 (0.0010) [2023-12-26 17:35:33,424][105692] Updated weights for policy 0, policy_version 306172 (0.0010) [2023-12-26 17:35:33,467][105692] Updated weights for policy 0, policy_version 306182 (0.0010) [2023-12-26 17:35:33,773][105620] Updated weights for policy 1, policy_version 306353 (0.0009) [2023-12-26 17:35:33,824][105620] Updated weights for policy 1, policy_version 306363 (0.0008) [2023-12-26 17:35:33,892][105620] Updated weights for policy 1, policy_version 306373 (0.0009) [2023-12-26 17:35:34,210][105692] Updated weights for policy 0, policy_version 306192 (0.0010) [2023-12-26 17:35:34,268][105692] Updated weights for policy 0, policy_version 306202 (0.0009) [2023-12-26 17:35:34,323][105692] Updated weights for policy 0, policy_version 306212 (0.0009) [2023-12-26 17:35:34,651][105620] Updated weights for policy 1, policy_version 306383 (0.0009) [2023-12-26 17:35:34,710][105620] Updated weights for policy 1, policy_version 306393 (0.0008) [2023-12-26 17:35:34,767][105620] Updated weights for policy 1, policy_version 306403 (0.0009) [2023-12-26 17:35:35,081][105692] Updated weights for policy 0, policy_version 306222 (0.0009) [2023-12-26 17:35:35,139][105692] Updated weights for policy 0, policy_version 306232 (0.0010) [2023-12-26 17:35:35,191][105692] Updated weights for policy 0, policy_version 306242 (0.0009) [2023-12-26 17:35:35,399][105620] Updated weights for policy 1, policy_version 306413 (0.0009) [2023-12-26 17:35:35,446][105620] Updated weights for policy 1, policy_version 306423 (0.0009) [2023-12-26 17:35:35,493][105620] Updated weights for policy 1, policy_version 306433 (0.0008) [2023-12-26 17:35:35,937][105692] Updated weights for policy 0, policy_version 306252 (0.0008) [2023-12-26 17:35:35,993][105692] Updated weights for policy 0, policy_version 306262 (0.0007) [2023-12-26 17:35:36,050][105692] Updated weights for policy 0, policy_version 306272 (0.0005) [2023-12-26 17:35:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 156868608. Throughput: 0: 9890.0, 1: 9639.4. Samples: 156861620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:35:36,062][104569] Avg episode reward: [(0, '9359.437'), (1, '6092.931')] [2023-12-26 17:35:36,317][105620] Updated weights for policy 1, policy_version 306443 (0.0009) [2023-12-26 17:35:36,384][105620] Updated weights for policy 1, policy_version 306453 (0.0006) [2023-12-26 17:35:36,455][105620] Updated weights for policy 1, policy_version 306463 (0.0005) [2023-12-26 17:35:36,644][105692] Updated weights for policy 0, policy_version 306282 (0.0006) [2023-12-26 17:35:36,702][105692] Updated weights for policy 0, policy_version 306292 (0.0009) [2023-12-26 17:35:36,765][105692] Updated weights for policy 0, policy_version 306302 (0.0010) [2023-12-26 17:35:36,965][105620] Updated weights for policy 1, policy_version 306473 (0.0005) [2023-12-26 17:35:37,020][105620] Updated weights for policy 1, policy_version 306483 (0.0006) [2023-12-26 17:35:37,074][105620] Updated weights for policy 1, policy_version 306493 (0.0009) [2023-12-26 17:35:37,136][105620] Updated weights for policy 1, policy_version 306503 (0.0008) [2023-12-26 17:35:37,572][105692] Updated weights for policy 0, policy_version 306313 (0.0009) [2023-12-26 17:35:37,639][105692] Updated weights for policy 0, policy_version 306323 (0.0010) [2023-12-26 17:35:37,699][105692] Updated weights for policy 0, policy_version 306333 (0.0009) [2023-12-26 17:35:37,756][105692] Updated weights for policy 0, policy_version 306343 (0.0010) [2023-12-26 17:35:37,836][105620] Updated weights for policy 1, policy_version 306513 (0.0007) [2023-12-26 17:35:37,892][105620] Updated weights for policy 1, policy_version 306523 (0.0008) [2023-12-26 17:35:37,958][105620] Updated weights for policy 1, policy_version 306533 (0.0009) [2023-12-26 17:35:38,345][105692] Updated weights for policy 0, policy_version 306353 (0.0010) [2023-12-26 17:35:38,404][105692] Updated weights for policy 0, policy_version 306363 (0.0011) [2023-12-26 17:35:38,463][105692] Updated weights for policy 0, policy_version 306373 (0.0010) [2023-12-26 17:35:38,718][105620] Updated weights for policy 1, policy_version 306543 (0.0008) [2023-12-26 17:35:38,773][105620] Updated weights for policy 1, policy_version 306553 (0.0009) [2023-12-26 17:35:38,830][105620] Updated weights for policy 1, policy_version 306563 (0.0009) [2023-12-26 17:35:39,084][105692] Updated weights for policy 0, policy_version 306383 (0.0008) [2023-12-26 17:35:39,136][105692] Updated weights for policy 0, policy_version 306393 (0.0008) [2023-12-26 17:35:39,181][105692] Updated weights for policy 0, policy_version 306403 (0.0010) [2023-12-26 17:35:39,683][105620] Updated weights for policy 1, policy_version 306573 (0.0010) [2023-12-26 17:35:39,750][105620] Updated weights for policy 1, policy_version 306583 (0.0009) [2023-12-26 17:35:39,809][105620] Updated weights for policy 1, policy_version 306593 (0.0010) [2023-12-26 17:35:39,913][105692] Updated weights for policy 0, policy_version 306413 (0.0008) [2023-12-26 17:35:39,976][105692] Updated weights for policy 0, policy_version 306423 (0.0008) [2023-12-26 17:35:40,042][105692] Updated weights for policy 0, policy_version 306433 (0.0008) [2023-12-26 17:35:40,530][105620] Updated weights for policy 1, policy_version 306603 (0.0006) [2023-12-26 17:35:40,538][105586] KL-divergence is very high: 105.4977 [2023-12-26 17:35:40,584][105620] Updated weights for policy 1, policy_version 306613 (0.0005) [2023-12-26 17:35:40,638][105620] Updated weights for policy 1, policy_version 306623 (0.0005) [2023-12-26 17:35:40,789][105692] Updated weights for policy 0, policy_version 306443 (0.0008) [2023-12-26 17:35:40,849][105692] Updated weights for policy 0, policy_version 306453 (0.0008) [2023-12-26 17:35:40,906][105692] Updated weights for policy 0, policy_version 306463 (0.0009) [2023-12-26 17:35:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 156975104. Throughput: 0: 9963.7, 1: 9704.3. Samples: 156980824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:35:41,063][104569] Avg episode reward: [(0, '9359.345'), (1, '6554.728')] [2023-12-26 17:35:41,313][105620] Updated weights for policy 1, policy_version 306633 (0.0007) [2023-12-26 17:35:41,387][105620] Updated weights for policy 1, policy_version 306643 (0.0009) [2023-12-26 17:35:41,459][105620] Updated weights for policy 1, policy_version 306653 (0.0009) [2023-12-26 17:35:41,513][105620] Updated weights for policy 1, policy_version 306663 (0.0009) [2023-12-26 17:35:41,700][105692] Updated weights for policy 0, policy_version 306473 (0.0009) [2023-12-26 17:35:41,772][105692] Updated weights for policy 0, policy_version 306483 (0.0009) [2023-12-26 17:35:41,836][105692] Updated weights for policy 0, policy_version 306493 (0.0008) [2023-12-26 17:35:41,893][105692] Updated weights for policy 0, policy_version 306503 (0.0008) [2023-12-26 17:35:42,314][105620] Updated weights for policy 1, policy_version 306673 (0.0009) [2023-12-26 17:35:42,378][105620] Updated weights for policy 1, policy_version 306683 (0.0009) [2023-12-26 17:35:42,438][105620] Updated weights for policy 1, policy_version 306693 (0.0008) [2023-12-26 17:35:42,671][105692] Updated weights for policy 0, policy_version 306513 (0.0010) [2023-12-26 17:35:42,740][105692] Updated weights for policy 0, policy_version 306523 (0.0010) [2023-12-26 17:35:42,813][105692] Updated weights for policy 0, policy_version 306533 (0.0010) [2023-12-26 17:35:43,108][105620] Updated weights for policy 1, policy_version 306703 (0.0009) [2023-12-26 17:35:43,170][105620] Updated weights for policy 1, policy_version 306713 (0.0009) [2023-12-26 17:35:43,228][105620] Updated weights for policy 1, policy_version 306723 (0.0010) [2023-12-26 17:35:43,538][105692] Updated weights for policy 0, policy_version 306543 (0.0008) [2023-12-26 17:35:43,591][105692] Updated weights for policy 0, policy_version 306553 (0.0007) [2023-12-26 17:35:43,649][105692] Updated weights for policy 0, policy_version 306563 (0.0006) [2023-12-26 17:35:44,068][105620] Updated weights for policy 1, policy_version 306733 (0.0008) [2023-12-26 17:35:44,134][105620] Updated weights for policy 1, policy_version 306743 (0.0007) [2023-12-26 17:35:44,181][105620] Updated weights for policy 1, policy_version 306753 (0.0008) [2023-12-26 17:35:44,195][105692] Updated weights for policy 0, policy_version 306573 (0.0006) [2023-12-26 17:35:44,255][105692] Updated weights for policy 0, policy_version 306583 (0.0008) [2023-12-26 17:35:44,316][105692] Updated weights for policy 0, policy_version 306593 (0.0008) [2023-12-26 17:35:44,915][105620] Updated weights for policy 1, policy_version 306763 (0.0006) [2023-12-26 17:35:44,951][105692] Updated weights for policy 0, policy_version 306603 (0.0007) [2023-12-26 17:35:44,980][105620] Updated weights for policy 1, policy_version 306773 (0.0006) [2023-12-26 17:35:45,017][105692] Updated weights for policy 0, policy_version 306613 (0.0006) [2023-12-26 17:35:45,040][105620] Updated weights for policy 1, policy_version 306783 (0.0010) [2023-12-26 17:35:45,078][105692] Updated weights for policy 0, policy_version 306623 (0.0007) [2023-12-26 17:35:45,718][105692] Updated weights for policy 0, policy_version 306633 (0.0008) [2023-12-26 17:35:45,747][105620] Updated weights for policy 1, policy_version 306793 (0.0006) [2023-12-26 17:35:45,763][105692] Updated weights for policy 0, policy_version 306643 (0.0008) [2023-12-26 17:35:45,805][105620] Updated weights for policy 1, policy_version 306803 (0.0008) [2023-12-26 17:35:45,823][105692] Updated weights for policy 0, policy_version 306653 (0.0007) [2023-12-26 17:35:45,866][105620] Updated weights for policy 1, policy_version 306813 (0.0008) [2023-12-26 17:35:45,873][105692] Updated weights for policy 0, policy_version 306663 (0.0007) [2023-12-26 17:35:45,918][105620] Updated weights for policy 1, policy_version 306823 (0.0009) [2023-12-26 17:35:46,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 157073408. Throughput: 0: 9941.8, 1: 9685.3. Samples: 157035340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:35:46,063][104569] Avg episode reward: [(0, '9359.270'), (1, '7357.502')] [2023-12-26 17:35:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000306664_78520320.pth... [2023-12-26 17:35:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000306824_78553088.pth... [2023-12-26 17:35:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000305672_78258176.pth [2023-12-26 17:35:46,098][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000305480_78217216.pth [2023-12-26 17:35:46,616][105620] Updated weights for policy 1, policy_version 306833 (0.0008) [2023-12-26 17:35:46,654][105692] Updated weights for policy 0, policy_version 306673 (0.0006) [2023-12-26 17:35:46,671][105620] Updated weights for policy 1, policy_version 306843 (0.0007) [2023-12-26 17:35:46,706][105692] Updated weights for policy 0, policy_version 306683 (0.0007) [2023-12-26 17:35:46,722][105620] Updated weights for policy 1, policy_version 306853 (0.0007) [2023-12-26 17:35:46,759][105692] Updated weights for policy 0, policy_version 306693 (0.0006) [2023-12-26 17:35:47,364][105620] Updated weights for policy 1, policy_version 306863 (0.0006) [2023-12-26 17:35:47,413][105620] Updated weights for policy 1, policy_version 306873 (0.0005) [2023-12-26 17:35:47,464][105620] Updated weights for policy 1, policy_version 306883 (0.0005) [2023-12-26 17:35:47,600][105692] Updated weights for policy 0, policy_version 306703 (0.0009) [2023-12-26 17:35:47,668][105692] Updated weights for policy 0, policy_version 306713 (0.0008) [2023-12-26 17:35:47,732][105692] Updated weights for policy 0, policy_version 306723 (0.0007) [2023-12-26 17:35:48,012][105620] Updated weights for policy 1, policy_version 306893 (0.0008) [2023-12-26 17:35:48,067][105620] Updated weights for policy 1, policy_version 306903 (0.0010) [2023-12-26 17:35:48,119][105620] Updated weights for policy 1, policy_version 306913 (0.0010) [2023-12-26 17:35:48,376][105692] Updated weights for policy 0, policy_version 306733 (0.0007) [2023-12-26 17:35:48,436][105692] Updated weights for policy 0, policy_version 306743 (0.0008) [2023-12-26 17:35:48,499][105692] Updated weights for policy 0, policy_version 306753 (0.0008) [2023-12-26 17:35:48,881][105620] Updated weights for policy 1, policy_version 306923 (0.0010) [2023-12-26 17:35:48,948][105620] Updated weights for policy 1, policy_version 306933 (0.0011) [2023-12-26 17:35:49,005][105620] Updated weights for policy 1, policy_version 306943 (0.0011) [2023-12-26 17:35:49,163][105692] Updated weights for policy 0, policy_version 306763 (0.0008) [2023-12-26 17:35:49,215][105692] Updated weights for policy 0, policy_version 306773 (0.0008) [2023-12-26 17:35:49,271][105692] Updated weights for policy 0, policy_version 306783 (0.0007) [2023-12-26 17:35:49,801][105620] Updated weights for policy 1, policy_version 306953 (0.0010) [2023-12-26 17:35:49,866][105620] Updated weights for policy 1, policy_version 306963 (0.0009) [2023-12-26 17:35:49,928][105620] Updated weights for policy 1, policy_version 306973 (0.0009) [2023-12-26 17:35:49,980][105620] Updated weights for policy 1, policy_version 306983 (0.0009) [2023-12-26 17:35:49,994][105692] Updated weights for policy 0, policy_version 306793 (0.0009) [2023-12-26 17:35:50,052][105692] Updated weights for policy 0, policy_version 306803 (0.0009) [2023-12-26 17:35:50,105][105692] Updated weights for policy 0, policy_version 306813 (0.0009) [2023-12-26 17:35:50,161][105692] Updated weights for policy 0, policy_version 306823 (0.0009) [2023-12-26 17:35:50,725][105620] Updated weights for policy 1, policy_version 306993 (0.0009) [2023-12-26 17:35:50,784][105620] Updated weights for policy 1, policy_version 307003 (0.0009) [2023-12-26 17:35:50,840][105620] Updated weights for policy 1, policy_version 307013 (0.0009) [2023-12-26 17:35:50,924][105692] Updated weights for policy 0, policy_version 306833 (0.0009) [2023-12-26 17:35:50,982][105692] Updated weights for policy 0, policy_version 306843 (0.0009) [2023-12-26 17:35:51,045][105692] Updated weights for policy 0, policy_version 306853 (0.0009) [2023-12-26 17:35:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 157171712. Throughput: 0: 9999.0, 1: 9794.5. Samples: 157155680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:35:51,062][104569] Avg episode reward: [(0, '9359.328'), (1, '6778.074')] [2023-12-26 17:35:51,612][105620] Updated weights for policy 1, policy_version 307023 (0.0009) [2023-12-26 17:35:51,669][105586] KL-divergence is very high: 146.5388 [2023-12-26 17:35:51,680][105620] Updated weights for policy 1, policy_version 307033 (0.0007) [2023-12-26 17:35:51,723][105586] KL-divergence is very high: 275.8808 [2023-12-26 17:35:51,748][105620] Updated weights for policy 1, policy_version 307043 (0.0008) [2023-12-26 17:35:51,768][105586] KL-divergence is very high: 320.5377 [2023-12-26 17:35:51,882][105692] Updated weights for policy 0, policy_version 306863 (0.0010) [2023-12-26 17:35:51,944][105692] Updated weights for policy 0, policy_version 306873 (0.0009) [2023-12-26 17:35:51,997][105692] Updated weights for policy 0, policy_version 306883 (0.0009) [2023-12-26 17:35:52,447][105586] KL-divergence is very high: 260.1971 [2023-12-26 17:35:52,466][105620] Updated weights for policy 1, policy_version 307053 (0.0009) [2023-12-26 17:35:52,496][105586] KL-divergence is very high: 265.1426 [2023-12-26 17:35:52,528][105620] Updated weights for policy 1, policy_version 307063 (0.0009) [2023-12-26 17:35:52,545][105586] KL-divergence is very high: 255.7881 [2023-12-26 17:35:52,589][105620] Updated weights for policy 1, policy_version 307073 (0.0008) [2023-12-26 17:35:52,595][105586] KL-divergence is very high: 258.3245 [2023-12-26 17:35:52,779][105692] Updated weights for policy 0, policy_version 306893 (0.0009) [2023-12-26 17:35:52,842][105692] Updated weights for policy 0, policy_version 306903 (0.0009) [2023-12-26 17:35:52,902][105692] Updated weights for policy 0, policy_version 306913 (0.0010) [2023-12-26 17:35:53,259][105620] Updated weights for policy 1, policy_version 307083 (0.0009) [2023-12-26 17:35:53,317][105620] Updated weights for policy 1, policy_version 307093 (0.0009) [2023-12-26 17:35:53,377][105620] Updated weights for policy 1, policy_version 307103 (0.0009) [2023-12-26 17:35:53,555][105692] Updated weights for policy 0, policy_version 306923 (0.0008) [2023-12-26 17:35:53,611][105692] Updated weights for policy 0, policy_version 306933 (0.0009) [2023-12-26 17:35:53,659][105692] Updated weights for policy 0, policy_version 306943 (0.0009) [2023-12-26 17:35:54,152][105620] Updated weights for policy 1, policy_version 307113 (0.0010) [2023-12-26 17:35:54,199][105620] Updated weights for policy 1, policy_version 307123 (0.0008) [2023-12-26 17:35:54,245][105620] Updated weights for policy 1, policy_version 307133 (0.0008) [2023-12-26 17:35:54,293][105620] Updated weights for policy 1, policy_version 307143 (0.0009) [2023-12-26 17:35:54,432][105692] Updated weights for policy 0, policy_version 306953 (0.0009) [2023-12-26 17:35:54,494][105692] Updated weights for policy 0, policy_version 306963 (0.0009) [2023-12-26 17:35:54,543][105692] Updated weights for policy 0, policy_version 306973 (0.0009) [2023-12-26 17:35:54,604][105692] Updated weights for policy 0, policy_version 306983 (0.0008) [2023-12-26 17:35:55,105][105620] Updated weights for policy 1, policy_version 307153 (0.0009) [2023-12-26 17:35:55,163][105620] Updated weights for policy 1, policy_version 307163 (0.0008) [2023-12-26 17:35:55,220][105620] Updated weights for policy 1, policy_version 307173 (0.0005) [2023-12-26 17:35:55,311][105692] Updated weights for policy 0, policy_version 306993 (0.0009) [2023-12-26 17:35:55,370][105692] Updated weights for policy 0, policy_version 307003 (0.0009) [2023-12-26 17:35:55,418][105692] Updated weights for policy 0, policy_version 307013 (0.0009) [2023-12-26 17:35:55,839][105620] Updated weights for policy 1, policy_version 307183 (0.0008) [2023-12-26 17:35:55,894][105620] Updated weights for policy 1, policy_version 307193 (0.0010) [2023-12-26 17:35:55,951][105620] Updated weights for policy 1, policy_version 307204 (0.0009) [2023-12-26 17:35:56,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 157261824. Throughput: 0: 10036.1, 1: 9795.9. Samples: 157269224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:35:56,062][104569] Avg episode reward: [(0, '9359.354'), (1, '6883.602')] [2023-12-26 17:35:56,083][105692] Updated weights for policy 0, policy_version 307023 (0.0007) [2023-12-26 17:35:56,144][105692] Updated weights for policy 0, policy_version 307033 (0.0006) [2023-12-26 17:35:56,200][105692] Updated weights for policy 0, policy_version 307043 (0.0006) [2023-12-26 17:35:56,642][105620] Updated weights for policy 1, policy_version 307214 (0.0007) [2023-12-26 17:35:56,688][105620] Updated weights for policy 1, policy_version 307224 (0.0005) [2023-12-26 17:35:56,742][105620] Updated weights for policy 1, policy_version 307234 (0.0005) [2023-12-26 17:35:56,845][105692] Updated weights for policy 0, policy_version 307053 (0.0008) [2023-12-26 17:35:56,907][105692] Updated weights for policy 0, policy_version 307063 (0.0010) [2023-12-26 17:35:56,958][105692] Updated weights for policy 0, policy_version 307073 (0.0011) [2023-12-26 17:35:57,279][105620] Updated weights for policy 1, policy_version 307244 (0.0007) [2023-12-26 17:35:57,339][105620] Updated weights for policy 1, policy_version 307254 (0.0010) [2023-12-26 17:35:57,390][105620] Updated weights for policy 1, policy_version 307264 (0.0010) [2023-12-26 17:35:57,686][105692] Updated weights for policy 0, policy_version 307083 (0.0010) [2023-12-26 17:35:57,739][105692] Updated weights for policy 0, policy_version 307093 (0.0008) [2023-12-26 17:35:57,780][105692] Updated weights for policy 0, policy_version 307103 (0.0005) [2023-12-26 17:35:58,067][105620] Updated weights for policy 1, policy_version 307274 (0.0006) [2023-12-26 17:35:58,128][105620] Updated weights for policy 1, policy_version 307284 (0.0009) [2023-12-26 17:35:58,185][105620] Updated weights for policy 1, policy_version 307294 (0.0008) [2023-12-26 17:35:58,242][105620] Updated weights for policy 1, policy_version 307304 (0.0007) [2023-12-26 17:35:58,460][105692] Updated weights for policy 0, policy_version 307113 (0.0006) [2023-12-26 17:35:58,516][105692] Updated weights for policy 0, policy_version 307123 (0.0010) [2023-12-26 17:35:58,582][105692] Updated weights for policy 0, policy_version 307133 (0.0010) [2023-12-26 17:35:58,644][105692] Updated weights for policy 0, policy_version 307143 (0.0010) [2023-12-26 17:35:58,958][105620] Updated weights for policy 1, policy_version 307314 (0.0007) [2023-12-26 17:35:59,013][105620] Updated weights for policy 1, policy_version 307324 (0.0010) [2023-12-26 17:35:59,066][105620] Updated weights for policy 1, policy_version 307335 (0.0010) [2023-12-26 17:35:59,350][105692] Updated weights for policy 0, policy_version 307153 (0.0008) [2023-12-26 17:35:59,417][105692] Updated weights for policy 0, policy_version 307163 (0.0008) [2023-12-26 17:35:59,480][105692] Updated weights for policy 0, policy_version 307173 (0.0008) [2023-12-26 17:35:59,830][105620] Updated weights for policy 1, policy_version 307345 (0.0009) [2023-12-26 17:35:59,887][105586] KL-divergence is very high: 113.6742 [2023-12-26 17:35:59,895][105620] Updated weights for policy 1, policy_version 307355 (0.0009) [2023-12-26 17:35:59,939][105586] KL-divergence is very high: 127.6895 [2023-12-26 17:35:59,958][105620] Updated weights for policy 1, policy_version 307365 (0.0009) [2023-12-26 17:36:00,160][105692] Updated weights for policy 0, policy_version 307183 (0.0006) [2023-12-26 17:36:00,228][105692] Updated weights for policy 0, policy_version 307193 (0.0005) [2023-12-26 17:36:00,299][105692] Updated weights for policy 0, policy_version 307203 (0.0006) [2023-12-26 17:36:00,774][105620] Updated weights for policy 1, policy_version 307375 (0.0009) [2023-12-26 17:36:00,825][105620] Updated weights for policy 1, policy_version 307385 (0.0009) [2023-12-26 17:36:00,884][105620] Updated weights for policy 1, policy_version 307395 (0.0007) [2023-12-26 17:36:00,978][105692] Updated weights for policy 0, policy_version 307213 (0.0009) [2023-12-26 17:36:01,026][105692] Updated weights for policy 0, policy_version 307223 (0.0008) [2023-12-26 17:36:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 157360128. Throughput: 0: 10021.2, 1: 9886.1. Samples: 157332776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:36:01,062][104569] Avg episode reward: [(0, '9359.400'), (1, '7712.092')] [2023-12-26 17:36:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000307400_78700544.pth... [2023-12-26 17:36:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000306248_78405632.pth [2023-12-26 17:36:01,090][105692] Updated weights for policy 0, policy_version 307233 (0.0008) [2023-12-26 17:36:01,128][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000307240_78667776.pth... [2023-12-26 17:36:01,133][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000306088_78372864.pth [2023-12-26 17:36:01,624][105620] Updated weights for policy 1, policy_version 307405 (0.0007) [2023-12-26 17:36:01,684][105620] Updated weights for policy 1, policy_version 307415 (0.0009) [2023-12-26 17:36:01,758][105620] Updated weights for policy 1, policy_version 307425 (0.0009) [2023-12-26 17:36:01,851][105692] Updated weights for policy 0, policy_version 307243 (0.0007) [2023-12-26 17:36:01,913][105692] Updated weights for policy 0, policy_version 307253 (0.0005) [2023-12-26 17:36:01,971][105692] Updated weights for policy 0, policy_version 307263 (0.0006) [2023-12-26 17:36:02,554][105692] Updated weights for policy 0, policy_version 307273 (0.0008) [2023-12-26 17:36:02,606][105692] Updated weights for policy 0, policy_version 307283 (0.0005) [2023-12-26 17:36:02,618][105620] Updated weights for policy 1, policy_version 307435 (0.0008) [2023-12-26 17:36:02,662][105692] Updated weights for policy 0, policy_version 307293 (0.0006) [2023-12-26 17:36:02,672][105620] Updated weights for policy 1, policy_version 307445 (0.0010) [2023-12-26 17:36:02,718][105692] Updated weights for policy 0, policy_version 307303 (0.0006) [2023-12-26 17:36:02,732][105620] Updated weights for policy 1, policy_version 307455 (0.0007) [2023-12-26 17:36:03,370][105692] Updated weights for policy 0, policy_version 307313 (0.0009) [2023-12-26 17:36:03,422][105692] Updated weights for policy 0, policy_version 307323 (0.0009) [2023-12-26 17:36:03,473][105692] Updated weights for policy 0, policy_version 307333 (0.0008) [2023-12-26 17:36:03,496][105620] Updated weights for policy 1, policy_version 307465 (0.0009) [2023-12-26 17:36:03,548][105620] Updated weights for policy 1, policy_version 307475 (0.0009) [2023-12-26 17:36:03,600][105620] Updated weights for policy 1, policy_version 307486 (0.0010) [2023-12-26 17:36:03,648][105620] Updated weights for policy 1, policy_version 307496 (0.0010) [2023-12-26 17:36:04,222][105692] Updated weights for policy 0, policy_version 307343 (0.0006) [2023-12-26 17:36:04,294][105692] Updated weights for policy 0, policy_version 307353 (0.0006) [2023-12-26 17:36:04,309][105620] Updated weights for policy 1, policy_version 307506 (0.0008) [2023-12-26 17:36:04,362][105692] Updated weights for policy 0, policy_version 307363 (0.0007) [2023-12-26 17:36:04,377][105620] Updated weights for policy 1, policy_version 307516 (0.0009) [2023-12-26 17:36:04,444][105620] Updated weights for policy 1, policy_version 307526 (0.0011) [2023-12-26 17:36:05,077][105692] Updated weights for policy 0, policy_version 307373 (0.0008) [2023-12-26 17:36:05,123][105620] Updated weights for policy 1, policy_version 307536 (0.0006) [2023-12-26 17:36:05,131][105692] Updated weights for policy 0, policy_version 307383 (0.0009) [2023-12-26 17:36:05,179][105620] Updated weights for policy 1, policy_version 307546 (0.0005) [2023-12-26 17:36:05,188][105692] Updated weights for policy 0, policy_version 307393 (0.0009) [2023-12-26 17:36:05,237][105620] Updated weights for policy 1, policy_version 307556 (0.0005) [2023-12-26 17:36:05,772][105620] Updated weights for policy 1, policy_version 307566 (0.0005) [2023-12-26 17:36:05,807][105692] Updated weights for policy 0, policy_version 307403 (0.0009) [2023-12-26 17:36:05,818][105620] Updated weights for policy 1, policy_version 307576 (0.0005) [2023-12-26 17:36:05,862][105692] Updated weights for policy 0, policy_version 307413 (0.0007) [2023-12-26 17:36:05,878][105620] Updated weights for policy 1, policy_version 307586 (0.0006) [2023-12-26 17:36:05,922][105692] Updated weights for policy 0, policy_version 307423 (0.0011) [2023-12-26 17:36:06,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 157466624. Throughput: 0: 9882.5, 1: 9801.5. Samples: 157448572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:36:06,063][104569] Avg episode reward: [(0, '9359.348'), (1, '8070.839')] [2023-12-26 17:36:06,509][105620] Updated weights for policy 1, policy_version 307596 (0.0008) [2023-12-26 17:36:06,561][105692] Updated weights for policy 0, policy_version 307433 (0.0005) [2023-12-26 17:36:06,567][105620] Updated weights for policy 1, policy_version 307606 (0.0009) [2023-12-26 17:36:06,615][105692] Updated weights for policy 0, policy_version 307443 (0.0006) [2023-12-26 17:36:06,632][105620] Updated weights for policy 1, policy_version 307616 (0.0009) [2023-12-26 17:36:06,672][105692] Updated weights for policy 0, policy_version 307453 (0.0007) [2023-12-26 17:36:06,727][105692] Updated weights for policy 0, policy_version 307463 (0.0006) [2023-12-26 17:36:07,410][105620] Updated weights for policy 1, policy_version 307626 (0.0008) [2023-12-26 17:36:07,463][105692] Updated weights for policy 0, policy_version 307473 (0.0007) [2023-12-26 17:36:07,474][105620] Updated weights for policy 1, policy_version 307636 (0.0006) [2023-12-26 17:36:07,521][105692] Updated weights for policy 0, policy_version 307483 (0.0008) [2023-12-26 17:36:07,527][105620] Updated weights for policy 1, policy_version 307646 (0.0006) [2023-12-26 17:36:07,568][105692] Updated weights for policy 0, policy_version 307493 (0.0007) [2023-12-26 17:36:07,574][105620] Updated weights for policy 1, policy_version 307656 (0.0006) [2023-12-26 17:36:08,273][105620] Updated weights for policy 1, policy_version 307666 (0.0010) [2023-12-26 17:36:08,298][105692] Updated weights for policy 0, policy_version 307503 (0.0006) [2023-12-26 17:36:08,329][105620] Updated weights for policy 1, policy_version 307676 (0.0009) [2023-12-26 17:36:08,352][105692] Updated weights for policy 0, policy_version 307513 (0.0006) [2023-12-26 17:36:08,390][105620] Updated weights for policy 1, policy_version 307686 (0.0009) [2023-12-26 17:36:08,409][105692] Updated weights for policy 0, policy_version 307523 (0.0007) [2023-12-26 17:36:09,045][105692] Updated weights for policy 0, policy_version 307533 (0.0007) [2023-12-26 17:36:09,092][105692] Updated weights for policy 0, policy_version 307543 (0.0005) [2023-12-26 17:36:09,137][105692] Updated weights for policy 0, policy_version 307553 (0.0006) [2023-12-26 17:36:09,241][105620] Updated weights for policy 1, policy_version 307696 (0.0009) [2023-12-26 17:36:09,307][105620] Updated weights for policy 1, policy_version 307706 (0.0008) [2023-12-26 17:36:09,377][105620] Updated weights for policy 1, policy_version 307716 (0.0008) [2023-12-26 17:36:09,858][105692] Updated weights for policy 0, policy_version 307563 (0.0007) [2023-12-26 17:36:09,924][105692] Updated weights for policy 0, policy_version 307573 (0.0011) [2023-12-26 17:36:09,993][105692] Updated weights for policy 0, policy_version 307583 (0.0011) [2023-12-26 17:36:10,153][105620] Updated weights for policy 1, policy_version 307726 (0.0008) [2023-12-26 17:36:10,221][105620] Updated weights for policy 1, policy_version 307736 (0.0008) [2023-12-26 17:36:10,286][105620] Updated weights for policy 1, policy_version 307746 (0.0009) [2023-12-26 17:36:10,751][105692] Updated weights for policy 0, policy_version 307593 (0.0010) [2023-12-26 17:36:10,806][105692] Updated weights for policy 0, policy_version 307603 (0.0010) [2023-12-26 17:36:10,865][105692] Updated weights for policy 0, policy_version 307613 (0.0010) [2023-12-26 17:36:10,923][105692] Updated weights for policy 0, policy_version 307623 (0.0011) [2023-12-26 17:36:11,041][105620] Updated weights for policy 1, policy_version 307756 (0.0008) [2023-12-26 17:36:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 157556736. Throughput: 0: 9826.9, 1: 9806.1. Samples: 157566680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:36:11,062][104569] Avg episode reward: [(0, '9359.289'), (1, '8168.573')] [2023-12-26 17:36:11,091][105620] Updated weights for policy 1, policy_version 307766 (0.0008) [2023-12-26 17:36:11,155][105620] Updated weights for policy 1, policy_version 307776 (0.0008) [2023-12-26 17:36:11,716][105692] Updated weights for policy 0, policy_version 307633 (0.0011) [2023-12-26 17:36:11,785][105692] Updated weights for policy 0, policy_version 307643 (0.0008) [2023-12-26 17:36:11,845][105692] Updated weights for policy 0, policy_version 307653 (0.0007) [2023-12-26 17:36:11,944][105620] Updated weights for policy 1, policy_version 307786 (0.0008) [2023-12-26 17:36:12,010][105620] Updated weights for policy 1, policy_version 307796 (0.0010) [2023-12-26 17:36:12,079][105620] Updated weights for policy 1, policy_version 307806 (0.0010) [2023-12-26 17:36:12,145][105620] Updated weights for policy 1, policy_version 307816 (0.0010) [2023-12-26 17:36:12,689][105692] Updated weights for policy 0, policy_version 307663 (0.0010) [2023-12-26 17:36:12,752][105692] Updated weights for policy 0, policy_version 307673 (0.0008) [2023-12-26 17:36:12,769][105620] Updated weights for policy 1, policy_version 307826 (0.0007) [2023-12-26 17:36:12,814][105692] Updated weights for policy 0, policy_version 307683 (0.0008) [2023-12-26 17:36:12,827][105620] Updated weights for policy 1, policy_version 307836 (0.0005) [2023-12-26 17:36:12,886][105620] Updated weights for policy 1, policy_version 307846 (0.0005) [2023-12-26 17:36:13,411][105620] Updated weights for policy 1, policy_version 307856 (0.0008) [2023-12-26 17:36:13,473][105620] Updated weights for policy 1, policy_version 307866 (0.0010) [2023-12-26 17:36:13,531][105620] Updated weights for policy 1, policy_version 307876 (0.0010) [2023-12-26 17:36:13,693][105692] Updated weights for policy 0, policy_version 307693 (0.0008) [2023-12-26 17:36:13,742][105692] Updated weights for policy 0, policy_version 307703 (0.0008) [2023-12-26 17:36:13,802][105692] Updated weights for policy 0, policy_version 307713 (0.0008) [2023-12-26 17:36:14,229][105620] Updated weights for policy 1, policy_version 307886 (0.0010) [2023-12-26 17:36:14,287][105620] Updated weights for policy 1, policy_version 307896 (0.0010) [2023-12-26 17:36:14,345][105620] Updated weights for policy 1, policy_version 307906 (0.0010) [2023-12-26 17:36:14,593][105692] Updated weights for policy 0, policy_version 307723 (0.0009) [2023-12-26 17:36:14,653][105692] Updated weights for policy 0, policy_version 307733 (0.0008) [2023-12-26 17:36:14,707][105692] Updated weights for policy 0, policy_version 307743 (0.0008) [2023-12-26 17:36:15,100][105620] Updated weights for policy 1, policy_version 307916 (0.0010) [2023-12-26 17:36:15,157][105620] Updated weights for policy 1, policy_version 307926 (0.0007) [2023-12-26 17:36:15,216][105620] Updated weights for policy 1, policy_version 307936 (0.0010) [2023-12-26 17:36:15,420][105692] Updated weights for policy 0, policy_version 307753 (0.0008) [2023-12-26 17:36:15,466][105692] Updated weights for policy 0, policy_version 307763 (0.0006) [2023-12-26 17:36:15,515][105692] Updated weights for policy 0, policy_version 307773 (0.0006) [2023-12-26 17:36:15,576][105692] Updated weights for policy 0, policy_version 307783 (0.0008) [2023-12-26 17:36:15,953][105620] Updated weights for policy 1, policy_version 307946 (0.0011) [2023-12-26 17:36:16,020][105620] Updated weights for policy 1, policy_version 307956 (0.0010) [2023-12-26 17:36:16,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 157646848. Throughput: 0: 9759.8, 1: 9742.9. Samples: 157622932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:36:16,063][104569] Avg episode reward: [(0, '9359.289'), (1, '8173.158')] [2023-12-26 17:36:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000307784_78807040.pth... [2023-12-26 17:36:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000306664_78520320.pth [2023-12-26 17:36:16,075][105620] Updated weights for policy 1, policy_version 307966 (0.0010) [2023-12-26 17:36:16,124][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000307976_78848000.pth... [2023-12-26 17:36:16,125][105620] Updated weights for policy 1, policy_version 307976 (0.0009) [2023-12-26 17:36:16,127][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000306824_78553088.pth [2023-12-26 17:36:16,245][105692] Updated weights for policy 0, policy_version 307793 (0.0008) [2023-12-26 17:36:16,293][105692] Updated weights for policy 0, policy_version 307803 (0.0008) [2023-12-26 17:36:16,356][105692] Updated weights for policy 0, policy_version 307813 (0.0010) [2023-12-26 17:36:16,749][105620] Updated weights for policy 1, policy_version 307986 (0.0006) [2023-12-26 17:36:16,814][105620] Updated weights for policy 1, policy_version 307996 (0.0006) [2023-12-26 17:36:16,865][105620] Updated weights for policy 1, policy_version 308006 (0.0009) [2023-12-26 17:36:16,999][105692] Updated weights for policy 0, policy_version 307823 (0.0009) [2023-12-26 17:36:17,049][105692] Updated weights for policy 0, policy_version 307833 (0.0008) [2023-12-26 17:36:17,106][105692] Updated weights for policy 0, policy_version 307843 (0.0009) [2023-12-26 17:36:17,489][105620] Updated weights for policy 1, policy_version 308016 (0.0006) [2023-12-26 17:36:17,545][105620] Updated weights for policy 1, policy_version 308026 (0.0005) [2023-12-26 17:36:17,607][105620] Updated weights for policy 1, policy_version 308036 (0.0007) [2023-12-26 17:36:17,737][105692] Updated weights for policy 0, policy_version 307853 (0.0007) [2023-12-26 17:36:17,793][105692] Updated weights for policy 0, policy_version 307863 (0.0005) [2023-12-26 17:36:17,844][105692] Updated weights for policy 0, policy_version 307873 (0.0005) [2023-12-26 17:36:18,370][105620] Updated weights for policy 1, policy_version 308046 (0.0009) [2023-12-26 17:36:18,424][105692] Updated weights for policy 0, policy_version 307883 (0.0007) [2023-12-26 17:36:18,428][105620] Updated weights for policy 1, policy_version 308056 (0.0008) [2023-12-26 17:36:18,477][105692] Updated weights for policy 0, policy_version 307893 (0.0010) [2023-12-26 17:36:18,483][105620] Updated weights for policy 1, policy_version 308066 (0.0008) [2023-12-26 17:36:18,533][105692] Updated weights for policy 0, policy_version 307903 (0.0010) [2023-12-26 17:36:19,241][105620] Updated weights for policy 1, policy_version 308076 (0.0008) [2023-12-26 17:36:19,305][105692] Updated weights for policy 0, policy_version 307913 (0.0011) [2023-12-26 17:36:19,312][105620] Updated weights for policy 1, policy_version 308086 (0.0007) [2023-12-26 17:36:19,364][105692] Updated weights for policy 0, policy_version 307923 (0.0009) [2023-12-26 17:36:19,375][105620] Updated weights for policy 1, policy_version 308096 (0.0007) [2023-12-26 17:36:19,411][105692] Updated weights for policy 0, policy_version 307933 (0.0007) [2023-12-26 17:36:19,472][105692] Updated weights for policy 0, policy_version 307943 (0.0009) [2023-12-26 17:36:20,137][105620] Updated weights for policy 1, policy_version 308106 (0.0009) [2023-12-26 17:36:20,197][105620] Updated weights for policy 1, policy_version 308116 (0.0009) [2023-12-26 17:36:20,254][105620] Updated weights for policy 1, policy_version 308126 (0.0009) [2023-12-26 17:36:20,274][105692] Updated weights for policy 0, policy_version 307953 (0.0007) [2023-12-26 17:36:20,310][105620] Updated weights for policy 1, policy_version 308136 (0.0009) [2023-12-26 17:36:20,333][105692] Updated weights for policy 0, policy_version 307963 (0.0006) [2023-12-26 17:36:20,386][105692] Updated weights for policy 0, policy_version 307973 (0.0005) [2023-12-26 17:36:21,022][105692] Updated weights for policy 0, policy_version 307983 (0.0008) [2023-12-26 17:36:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 157745152. Throughput: 0: 9805.7, 1: 9744.4. Samples: 157741372. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 17:36:21,062][104569] Avg episode reward: [(0, '9359.267'), (1, '8899.865')] [2023-12-26 17:36:21,091][105692] Updated weights for policy 0, policy_version 307993 (0.0009) [2023-12-26 17:36:21,155][105620] Updated weights for policy 1, policy_version 308146 (0.0007) [2023-12-26 17:36:21,156][105692] Updated weights for policy 0, policy_version 308003 (0.0007) [2023-12-26 17:36:21,218][105620] Updated weights for policy 1, policy_version 308156 (0.0006) [2023-12-26 17:36:21,281][105620] Updated weights for policy 1, policy_version 308166 (0.0009) [2023-12-26 17:36:21,847][105692] Updated weights for policy 0, policy_version 308013 (0.0007) [2023-12-26 17:36:21,917][105692] Updated weights for policy 0, policy_version 308023 (0.0008) [2023-12-26 17:36:21,968][105692] Updated weights for policy 0, policy_version 308033 (0.0009) [2023-12-26 17:36:22,051][105620] Updated weights for policy 1, policy_version 308176 (0.0007) [2023-12-26 17:36:22,106][105620] Updated weights for policy 1, policy_version 308186 (0.0006) [2023-12-26 17:36:22,167][105620] Updated weights for policy 1, policy_version 308196 (0.0009) [2023-12-26 17:36:22,738][105692] Updated weights for policy 0, policy_version 308043 (0.0009) [2023-12-26 17:36:22,798][105692] Updated weights for policy 0, policy_version 308053 (0.0008) [2023-12-26 17:36:22,853][105692] Updated weights for policy 0, policy_version 308063 (0.0008) [2023-12-26 17:36:22,875][105620] Updated weights for policy 1, policy_version 308206 (0.0010) [2023-12-26 17:36:22,932][105620] Updated weights for policy 1, policy_version 308216 (0.0010) [2023-12-26 17:36:22,990][105620] Updated weights for policy 1, policy_version 308226 (0.0010) [2023-12-26 17:36:23,492][105692] Updated weights for policy 0, policy_version 308073 (0.0006) [2023-12-26 17:36:23,558][105692] Updated weights for policy 0, policy_version 308083 (0.0005) [2023-12-26 17:36:23,620][105692] Updated weights for policy 0, policy_version 308093 (0.0005) [2023-12-26 17:36:23,679][105692] Updated weights for policy 0, policy_version 308103 (0.0006) [2023-12-26 17:36:23,745][105620] Updated weights for policy 1, policy_version 308236 (0.0010) [2023-12-26 17:36:23,796][105620] Updated weights for policy 1, policy_version 308246 (0.0010) [2023-12-26 17:36:23,803][105586] KL-divergence is very high: 128.9217 [2023-12-26 17:36:23,844][105620] Updated weights for policy 1, policy_version 308256 (0.0010) [2023-12-26 17:36:24,265][105692] Updated weights for policy 0, policy_version 308113 (0.0006) [2023-12-26 17:36:24,333][105692] Updated weights for policy 0, policy_version 308123 (0.0005) [2023-12-26 17:36:24,395][105692] Updated weights for policy 0, policy_version 308133 (0.0006) [2023-12-26 17:36:24,611][105620] Updated weights for policy 1, policy_version 308266 (0.0010) [2023-12-26 17:36:24,659][105620] Updated weights for policy 1, policy_version 308276 (0.0010) [2023-12-26 17:36:24,705][105620] Updated weights for policy 1, policy_version 308286 (0.0009) [2023-12-26 17:36:24,754][105620] Updated weights for policy 1, policy_version 308296 (0.0005) [2023-12-26 17:36:24,896][105692] Updated weights for policy 0, policy_version 308143 (0.0005) [2023-12-26 17:36:24,954][105692] Updated weights for policy 0, policy_version 308153 (0.0008) [2023-12-26 17:36:25,012][105692] Updated weights for policy 0, policy_version 308163 (0.0008) [2023-12-26 17:36:25,455][105620] Updated weights for policy 1, policy_version 308306 (0.0010) [2023-12-26 17:36:25,520][105620] Updated weights for policy 1, policy_version 308316 (0.0006) [2023-12-26 17:36:25,568][105620] Updated weights for policy 1, policy_version 308326 (0.0005) [2023-12-26 17:36:25,684][105692] Updated weights for policy 0, policy_version 308173 (0.0007) [2023-12-26 17:36:25,740][105692] Updated weights for policy 0, policy_version 308183 (0.0005) [2023-12-26 17:36:25,804][105692] Updated weights for policy 0, policy_version 308193 (0.0005) [2023-12-26 17:36:26,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 157851648. Throughput: 0: 9851.4, 1: 9696.2. Samples: 157860464. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 17:36:26,063][104569] Avg episode reward: [(0, '9359.237'), (1, '8540.146')] [2023-12-26 17:36:26,124][105620] Updated weights for policy 1, policy_version 308336 (0.0006) [2023-12-26 17:36:26,186][105620] Updated weights for policy 1, policy_version 308346 (0.0006) [2023-12-26 17:36:26,241][105620] Updated weights for policy 1, policy_version 308356 (0.0010) [2023-12-26 17:36:26,384][105692] Updated weights for policy 0, policy_version 308203 (0.0006) [2023-12-26 17:36:26,434][105692] Updated weights for policy 0, policy_version 308213 (0.0009) [2023-12-26 17:36:26,478][105692] Updated weights for policy 0, policy_version 308223 (0.0010) [2023-12-26 17:36:26,916][105620] Updated weights for policy 1, policy_version 308366 (0.0010) [2023-12-26 17:36:26,974][105620] Updated weights for policy 1, policy_version 308376 (0.0009) [2023-12-26 17:36:27,036][105620] Updated weights for policy 1, policy_version 308386 (0.0005) [2023-12-26 17:36:27,216][105692] Updated weights for policy 0, policy_version 308233 (0.0010) [2023-12-26 17:36:27,282][105692] Updated weights for policy 0, policy_version 308243 (0.0006) [2023-12-26 17:36:27,343][105692] Updated weights for policy 0, policy_version 308253 (0.0008) [2023-12-26 17:36:27,401][105692] Updated weights for policy 0, policy_version 308263 (0.0010) [2023-12-26 17:36:27,679][105620] Updated weights for policy 1, policy_version 308396 (0.0007) [2023-12-26 17:36:27,736][105620] Updated weights for policy 1, policy_version 308406 (0.0011) [2023-12-26 17:36:27,782][105620] Updated weights for policy 1, policy_version 308416 (0.0010) [2023-12-26 17:36:28,031][105692] Updated weights for policy 0, policy_version 308273 (0.0009) [2023-12-26 17:36:28,084][105692] Updated weights for policy 0, policy_version 308283 (0.0008) [2023-12-26 17:36:28,137][105692] Updated weights for policy 0, policy_version 308293 (0.0005) [2023-12-26 17:36:28,513][105620] Updated weights for policy 1, policy_version 308426 (0.0007) [2023-12-26 17:36:28,586][105620] Updated weights for policy 1, policy_version 308436 (0.0006) [2023-12-26 17:36:28,642][105620] Updated weights for policy 1, policy_version 308446 (0.0010) [2023-12-26 17:36:28,698][105620] Updated weights for policy 1, policy_version 308456 (0.0009) [2023-12-26 17:36:28,815][105692] Updated weights for policy 0, policy_version 308303 (0.0008) [2023-12-26 17:36:28,879][105692] Updated weights for policy 0, policy_version 308313 (0.0009) [2023-12-26 17:36:28,938][105692] Updated weights for policy 0, policy_version 308323 (0.0006) [2023-12-26 17:36:29,376][105620] Updated weights for policy 1, policy_version 308466 (0.0009) [2023-12-26 17:36:29,432][105620] Updated weights for policy 1, policy_version 308476 (0.0008) [2023-12-26 17:36:29,491][105620] Updated weights for policy 1, policy_version 308486 (0.0009) [2023-12-26 17:36:29,704][105692] Updated weights for policy 0, policy_version 308334 (0.0009) [2023-12-26 17:36:29,765][105692] Updated weights for policy 0, policy_version 308344 (0.0009) [2023-12-26 17:36:29,811][105692] Updated weights for policy 0, policy_version 308354 (0.0009) [2023-12-26 17:36:30,118][105620] Updated weights for policy 1, policy_version 308496 (0.0009) [2023-12-26 17:36:30,169][105620] Updated weights for policy 1, policy_version 308506 (0.0007) [2023-12-26 17:36:30,222][105620] Updated weights for policy 1, policy_version 308516 (0.0005) [2023-12-26 17:36:30,606][105692] Updated weights for policy 0, policy_version 308364 (0.0007) [2023-12-26 17:36:30,659][105692] Updated weights for policy 0, policy_version 308374 (0.0005) [2023-12-26 17:36:30,713][105692] Updated weights for policy 0, policy_version 308384 (0.0005) [2023-12-26 17:36:30,931][105620] Updated weights for policy 1, policy_version 308526 (0.0009) [2023-12-26 17:36:30,984][105620] Updated weights for policy 1, policy_version 308536 (0.0009) [2023-12-26 17:36:31,045][105620] Updated weights for policy 1, policy_version 308546 (0.0007) [2023-12-26 17:36:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 157949952. Throughput: 0: 9953.1, 1: 9774.8. Samples: 157923092. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 17:36:31,063][104569] Avg episode reward: [(0, '9175.200'), (1, '7997.944')] [2023-12-26 17:36:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000308392_78962688.pth... [2023-12-26 17:36:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000307240_78667776.pth [2023-12-26 17:36:31,080][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000308552_78995456.pth... [2023-12-26 17:36:31,084][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000307400_78700544.pth [2023-12-26 17:36:31,400][105692] Updated weights for policy 0, policy_version 308394 (0.0006) [2023-12-26 17:36:31,462][105692] Updated weights for policy 0, policy_version 308404 (0.0007) [2023-12-26 17:36:31,527][105692] Updated weights for policy 0, policy_version 308414 (0.0009) [2023-12-26 17:36:31,591][105692] Updated weights for policy 0, policy_version 308424 (0.0008) [2023-12-26 17:36:31,814][105620] Updated weights for policy 1, policy_version 308556 (0.0009) [2023-12-26 17:36:31,864][105620] Updated weights for policy 1, policy_version 308566 (0.0011) [2023-12-26 17:36:31,917][105620] Updated weights for policy 1, policy_version 308576 (0.0010) [2023-12-26 17:36:32,391][105692] Updated weights for policy 0, policy_version 308434 (0.0011) [2023-12-26 17:36:32,436][105692] Updated weights for policy 0, policy_version 308444 (0.0008) [2023-12-26 17:36:32,484][105692] Updated weights for policy 0, policy_version 308454 (0.0008) [2023-12-26 17:36:32,599][105620] Updated weights for policy 1, policy_version 308586 (0.0010) [2023-12-26 17:36:32,659][105620] Updated weights for policy 1, policy_version 308596 (0.0005) [2023-12-26 17:36:32,715][105586] KL-divergence is very high: 210.0955 [2023-12-26 17:36:32,720][105620] Updated weights for policy 1, policy_version 308606 (0.0008) [2023-12-26 17:36:32,765][105586] KL-divergence is very high: 380.2867 [2023-12-26 17:36:32,783][105620] Updated weights for policy 1, policy_version 308616 (0.0007) [2023-12-26 17:36:33,159][105692] Updated weights for policy 0, policy_version 308464 (0.0006) [2023-12-26 17:36:33,212][105692] Updated weights for policy 0, policy_version 308474 (0.0005) [2023-12-26 17:36:33,266][105692] Updated weights for policy 0, policy_version 308484 (0.0007) [2023-12-26 17:36:33,337][105620] Updated weights for policy 1, policy_version 308626 (0.0006) [2023-12-26 17:36:33,401][105620] Updated weights for policy 1, policy_version 308636 (0.0005) [2023-12-26 17:36:33,472][105620] Updated weights for policy 1, policy_version 308646 (0.0005) [2023-12-26 17:36:33,882][105692] Updated weights for policy 0, policy_version 308494 (0.0008) [2023-12-26 17:36:33,934][105692] Updated weights for policy 0, policy_version 308504 (0.0009) [2023-12-26 17:36:33,976][105620] Updated weights for policy 1, policy_version 308656 (0.0005) [2023-12-26 17:36:33,983][105692] Updated weights for policy 0, policy_version 308514 (0.0009) [2023-12-26 17:36:34,037][105620] Updated weights for policy 1, policy_version 308666 (0.0011) [2023-12-26 17:36:34,102][105620] Updated weights for policy 1, policy_version 308676 (0.0010) [2023-12-26 17:36:34,699][105620] Updated weights for policy 1, policy_version 308686 (0.0008) [2023-12-26 17:36:34,755][105620] Updated weights for policy 1, policy_version 308696 (0.0010) [2023-12-26 17:36:34,804][105620] Updated weights for policy 1, policy_version 308706 (0.0010) [2023-12-26 17:36:34,852][105692] Updated weights for policy 0, policy_version 308524 (0.0008) [2023-12-26 17:36:34,909][105692] Updated weights for policy 0, policy_version 308534 (0.0009) [2023-12-26 17:36:34,962][105692] Updated weights for policy 0, policy_version 308544 (0.0010) [2023-12-26 17:36:35,385][105620] Updated weights for policy 1, policy_version 308716 (0.0009) [2023-12-26 17:36:35,432][105620] Updated weights for policy 1, policy_version 308726 (0.0010) [2023-12-26 17:36:35,480][105620] Updated weights for policy 1, policy_version 308736 (0.0010) [2023-12-26 17:36:35,787][105692] Updated weights for policy 0, policy_version 308554 (0.0009) [2023-12-26 17:36:35,835][105692] Updated weights for policy 0, policy_version 308564 (0.0008) [2023-12-26 17:36:35,884][105692] Updated weights for policy 0, policy_version 308574 (0.0008) [2023-12-26 17:36:35,896][105585] KL-divergence is very high: 112.2120 [2023-12-26 17:36:35,940][105692] Updated weights for policy 0, policy_version 308584 (0.0008) [2023-12-26 17:36:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 158056448. Throughput: 0: 9879.2, 1: 9854.7. Samples: 158043708. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 17:36:36,063][104569] Avg episode reward: [(0, '8815.845'), (1, '8550.997')] [2023-12-26 17:36:36,253][105620] Updated weights for policy 1, policy_version 308746 (0.0010) [2023-12-26 17:36:36,323][105620] Updated weights for policy 1, policy_version 308756 (0.0010) [2023-12-26 17:36:36,385][105620] Updated weights for policy 1, policy_version 308766 (0.0011) [2023-12-26 17:36:36,444][105620] Updated weights for policy 1, policy_version 308776 (0.0010) [2023-12-26 17:36:36,702][105692] Updated weights for policy 0, policy_version 308594 (0.0006) [2023-12-26 17:36:36,757][105692] Updated weights for policy 0, policy_version 308604 (0.0009) [2023-12-26 17:36:36,821][105692] Updated weights for policy 0, policy_version 308614 (0.0006) [2023-12-26 17:36:37,201][105620] Updated weights for policy 1, policy_version 308786 (0.0010) [2023-12-26 17:36:37,267][105620] Updated weights for policy 1, policy_version 308796 (0.0010) [2023-12-26 17:36:37,322][105620] Updated weights for policy 1, policy_version 308806 (0.0010) [2023-12-26 17:36:37,509][105692] Updated weights for policy 0, policy_version 308624 (0.0008) [2023-12-26 17:36:37,561][105692] Updated weights for policy 0, policy_version 308634 (0.0009) [2023-12-26 17:36:37,619][105692] Updated weights for policy 0, policy_version 308644 (0.0009) [2023-12-26 17:36:38,018][105620] Updated weights for policy 1, policy_version 308816 (0.0010) [2023-12-26 17:36:38,073][105620] Updated weights for policy 1, policy_version 308826 (0.0009) [2023-12-26 17:36:38,124][105620] Updated weights for policy 1, policy_version 308836 (0.0009) [2023-12-26 17:36:38,394][105692] Updated weights for policy 0, policy_version 308654 (0.0010) [2023-12-26 17:36:38,446][105692] Updated weights for policy 0, policy_version 308664 (0.0008) [2023-12-26 17:36:38,497][105692] Updated weights for policy 0, policy_version 308674 (0.0009) [2023-12-26 17:36:38,895][105620] Updated weights for policy 1, policy_version 308846 (0.0010) [2023-12-26 17:36:38,958][105620] Updated weights for policy 1, policy_version 308856 (0.0011) [2023-12-26 17:36:39,010][105620] Updated weights for policy 1, policy_version 308866 (0.0011) [2023-12-26 17:36:39,296][105692] Updated weights for policy 0, policy_version 308684 (0.0008) [2023-12-26 17:36:39,361][105692] Updated weights for policy 0, policy_version 308694 (0.0009) [2023-12-26 17:36:39,413][105585] KL-divergence is very high: 102.7718 [2023-12-26 17:36:39,420][105585] KL-divergence is very high: 114.9104 [2023-12-26 17:36:39,427][105692] Updated weights for policy 0, policy_version 308704 (0.0008) [2023-12-26 17:36:39,461][105585] KL-divergence is very high: 115.3651 [2023-12-26 17:36:39,466][105585] KL-divergence is very high: 154.2394 [2023-12-26 17:36:39,762][105620] Updated weights for policy 1, policy_version 308876 (0.0011) [2023-12-26 17:36:39,828][105620] Updated weights for policy 1, policy_version 308886 (0.0010) [2023-12-26 17:36:39,842][105586] KL-divergence is very high: 100.8514 [2023-12-26 17:36:39,850][105586] KL-divergence is very high: 117.6261 [2023-12-26 17:36:39,896][105620] Updated weights for policy 1, policy_version 308896 (0.0010) [2023-12-26 17:36:39,898][105586] KL-divergence is very high: 104.2004 [2023-12-26 17:36:39,905][105586] KL-divergence is very high: 116.0429 [2023-12-26 17:36:40,277][105585] KL-divergence is very high: 100.3038 [2023-12-26 17:36:40,291][105692] Updated weights for policy 0, policy_version 308714 (0.0009) [2023-12-26 17:36:40,291][105585] KL-divergence is very high: 127.6485 [2023-12-26 17:36:40,357][105692] Updated weights for policy 0, policy_version 308724 (0.0008) [2023-12-26 17:36:40,428][105692] Updated weights for policy 0, policy_version 308734 (0.0006) [2023-12-26 17:36:40,498][105692] Updated weights for policy 0, policy_version 308744 (0.0006) [2023-12-26 17:36:40,524][105620] Updated weights for policy 1, policy_version 308906 (0.0010) [2023-12-26 17:36:40,573][105620] Updated weights for policy 1, policy_version 308916 (0.0010) [2023-12-26 17:36:40,639][105620] Updated weights for policy 1, policy_version 308926 (0.0010) [2023-12-26 17:36:40,705][105620] Updated weights for policy 1, policy_version 308936 (0.0010) [2023-12-26 17:36:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 158146560. Throughput: 0: 9838.3, 1: 9880.8. Samples: 158156584. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 17:36:41,062][104569] Avg episode reward: [(0, '1800.186'), (1, '8636.936')] [2023-12-26 17:36:41,094][105692] Updated weights for policy 0, policy_version 308754 (0.0007) [2023-12-26 17:36:41,165][105692] Updated weights for policy 0, policy_version 308764 (0.0008) [2023-12-26 17:36:41,232][105692] Updated weights for policy 0, policy_version 308774 (0.0006) [2023-12-26 17:36:41,457][105620] Updated weights for policy 1, policy_version 308946 (0.0010) [2023-12-26 17:36:41,522][105620] Updated weights for policy 1, policy_version 308956 (0.0010) [2023-12-26 17:36:41,582][105620] Updated weights for policy 1, policy_version 308966 (0.0010) [2023-12-26 17:36:41,955][105692] Updated weights for policy 0, policy_version 308784 (0.0008) [2023-12-26 17:36:42,015][105692] Updated weights for policy 0, policy_version 308794 (0.0009) [2023-12-26 17:36:42,081][105692] Updated weights for policy 0, policy_version 308804 (0.0008) [2023-12-26 17:36:42,359][105620] Updated weights for policy 1, policy_version 308976 (0.0009) [2023-12-26 17:36:42,463][105620] Updated weights for policy 1, policy_version 308986 (0.0008) [2023-12-26 17:36:42,522][105620] Updated weights for policy 1, policy_version 308996 (0.0009) [2023-12-26 17:36:42,743][105692] Updated weights for policy 0, policy_version 308814 (0.0007) [2023-12-26 17:36:42,810][105692] Updated weights for policy 0, policy_version 308824 (0.0008) [2023-12-26 17:36:42,868][105692] Updated weights for policy 0, policy_version 308834 (0.0009) [2023-12-26 17:36:43,229][105620] Updated weights for policy 1, policy_version 309006 (0.0010) [2023-12-26 17:36:43,288][105620] Updated weights for policy 1, policy_version 309016 (0.0010) [2023-12-26 17:36:43,336][105620] Updated weights for policy 1, policy_version 309026 (0.0010) [2023-12-26 17:36:43,519][105692] Updated weights for policy 0, policy_version 308844 (0.0009) [2023-12-26 17:36:43,580][105692] Updated weights for policy 0, policy_version 308854 (0.0008) [2023-12-26 17:36:43,635][105692] Updated weights for policy 0, policy_version 308864 (0.0006) [2023-12-26 17:36:44,090][105620] Updated weights for policy 1, policy_version 309036 (0.0010) [2023-12-26 17:36:44,146][105620] Updated weights for policy 1, policy_version 309046 (0.0005) [2023-12-26 17:36:44,198][105620] Updated weights for policy 1, policy_version 309056 (0.0005) [2023-12-26 17:36:44,331][105692] Updated weights for policy 0, policy_version 308874 (0.0007) [2023-12-26 17:36:44,383][105692] Updated weights for policy 0, policy_version 308884 (0.0010) [2023-12-26 17:36:44,436][105692] Updated weights for policy 0, policy_version 308894 (0.0011) [2023-12-26 17:36:44,488][105692] Updated weights for policy 0, policy_version 308904 (0.0010) [2023-12-26 17:36:44,965][105620] Updated weights for policy 1, policy_version 309066 (0.0008) [2023-12-26 17:36:45,028][105620] Updated weights for policy 1, policy_version 309076 (0.0010) [2023-12-26 17:36:45,087][105620] Updated weights for policy 1, policy_version 309086 (0.0010) [2023-12-26 17:36:45,144][105620] Updated weights for policy 1, policy_version 309096 (0.0010) [2023-12-26 17:36:45,162][105692] Updated weights for policy 0, policy_version 308914 (0.0011) [2023-12-26 17:36:45,228][105692] Updated weights for policy 0, policy_version 308924 (0.0011) [2023-12-26 17:36:45,282][105692] Updated weights for policy 0, policy_version 308934 (0.0011) [2023-12-26 17:36:45,889][105620] Updated weights for policy 1, policy_version 309106 (0.0008) [2023-12-26 17:36:45,941][105620] Updated weights for policy 1, policy_version 309116 (0.0008) [2023-12-26 17:36:46,006][105620] Updated weights for policy 1, policy_version 309126 (0.0009) [2023-12-26 17:36:46,027][105692] Updated weights for policy 0, policy_version 308944 (0.0010) [2023-12-26 17:36:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 158244864. Throughput: 0: 9817.7, 1: 9775.6. Samples: 158214480. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 17:36:46,063][104569] Avg episode reward: [(0, '1879.632'), (1, '8438.627')] [2023-12-26 17:36:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000309128_79142912.pth... [2023-12-26 17:36:46,075][105692] Updated weights for policy 0, policy_version 308954 (0.0010) [2023-12-26 17:36:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000307976_78848000.pth [2023-12-26 17:36:46,127][105692] Updated weights for policy 0, policy_version 308964 (0.0008) [2023-12-26 17:36:46,149][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000308968_79110144.pth... [2023-12-26 17:36:46,154][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000307784_78807040.pth [2023-12-26 17:36:46,668][105620] Updated weights for policy 1, policy_version 309136 (0.0006) [2023-12-26 17:36:46,720][105692] Updated weights for policy 0, policy_version 308974 (0.0008) [2023-12-26 17:36:46,732][105620] Updated weights for policy 1, policy_version 309146 (0.0009) [2023-12-26 17:36:46,779][105692] Updated weights for policy 0, policy_version 308984 (0.0007) [2023-12-26 17:36:46,785][105585] KL-divergence is very high: 114.1192 [2023-12-26 17:36:46,790][105620] Updated weights for policy 1, policy_version 309156 (0.0010) [2023-12-26 17:36:46,802][105585] KL-divergence is very high: 273.5755 [2023-12-26 17:36:46,808][105585] KL-divergence is very high: 184.2696 [2023-12-26 17:36:46,815][105585] KL-divergence is very high: 261.0760 [2023-12-26 17:36:46,821][105585] KL-divergence is very high: 234.4518 [2023-12-26 17:36:46,835][105585] KL-divergence is very high: 127.9085 [2023-12-26 17:36:46,840][105692] Updated weights for policy 0, policy_version 308994 (0.0005) [2023-12-26 17:36:46,853][105585] KL-divergence is very high: 319.9377 [2023-12-26 17:36:46,865][105585] KL-divergence is very high: 230.3038 [2023-12-26 17:36:46,870][105585] KL-divergence is very high: 177.2143 [2023-12-26 17:36:47,424][105620] Updated weights for policy 1, policy_version 309166 (0.0007) [2023-12-26 17:36:47,463][105585] KL-divergence is very high: 297.5410 [2023-12-26 17:36:47,471][105620] Updated weights for policy 1, policy_version 309176 (0.0005) [2023-12-26 17:36:47,482][105692] Updated weights for policy 0, policy_version 309004 (0.0006) [2023-12-26 17:36:47,512][105585] KL-divergence is very high: 171.5834 [2023-12-26 17:36:47,526][105620] Updated weights for policy 1, policy_version 309186 (0.0005) [2023-12-26 17:36:47,543][105692] Updated weights for policy 0, policy_version 309014 (0.0009) [2023-12-26 17:36:47,561][105585] KL-divergence is very high: 118.7110 [2023-12-26 17:36:47,603][105692] Updated weights for policy 0, policy_version 309024 (0.0010) [2023-12-26 17:36:47,610][105585] KL-divergence is very high: 136.5544 [2023-12-26 17:36:48,167][105692] Updated weights for policy 0, policy_version 309034 (0.0005) [2023-12-26 17:36:48,233][105692] Updated weights for policy 0, policy_version 309044 (0.0005) [2023-12-26 17:36:48,302][105692] Updated weights for policy 0, policy_version 309054 (0.0006) [2023-12-26 17:36:48,339][105620] Updated weights for policy 1, policy_version 309196 (0.0008) [2023-12-26 17:36:48,371][105692] Updated weights for policy 0, policy_version 309064 (0.0006) [2023-12-26 17:36:48,402][105620] Updated weights for policy 1, policy_version 309206 (0.0009) [2023-12-26 17:36:48,464][105620] Updated weights for policy 1, policy_version 309216 (0.0009) [2023-12-26 17:36:49,040][105692] Updated weights for policy 0, policy_version 309074 (0.0009) [2023-12-26 17:36:49,102][105692] Updated weights for policy 0, policy_version 309084 (0.0009) [2023-12-26 17:36:49,164][105692] Updated weights for policy 0, policy_version 309094 (0.0009) [2023-12-26 17:36:49,206][105620] Updated weights for policy 1, policy_version 309226 (0.0009) [2023-12-26 17:36:49,271][105620] Updated weights for policy 1, policy_version 309236 (0.0008) [2023-12-26 17:36:49,331][105620] Updated weights for policy 1, policy_version 309246 (0.0009) [2023-12-26 17:36:49,394][105620] Updated weights for policy 1, policy_version 309256 (0.0008) [2023-12-26 17:36:49,924][105692] Updated weights for policy 0, policy_version 309104 (0.0010) [2023-12-26 17:36:49,986][105692] Updated weights for policy 0, policy_version 309114 (0.0008) [2023-12-26 17:36:50,046][105692] Updated weights for policy 0, policy_version 309124 (0.0008) [2023-12-26 17:36:50,164][105620] Updated weights for policy 1, policy_version 309266 (0.0008) [2023-12-26 17:36:50,215][105620] Updated weights for policy 1, policy_version 309276 (0.0008) [2023-12-26 17:36:50,261][105620] Updated weights for policy 1, policy_version 309286 (0.0008) [2023-12-26 17:36:50,827][105692] Updated weights for policy 0, policy_version 309134 (0.0008) [2023-12-26 17:36:50,889][105692] Updated weights for policy 0, policy_version 309144 (0.0009) [2023-12-26 17:36:50,947][105692] Updated weights for policy 0, policy_version 309154 (0.0009) [2023-12-26 17:36:50,999][105620] Updated weights for policy 1, policy_version 309296 (0.0008) [2023-12-26 17:36:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 158343168. Throughput: 0: 9853.1, 1: 9813.3. Samples: 158333556. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 17:36:51,062][104569] Avg episode reward: [(0, '3285.909'), (1, '8714.107')] [2023-12-26 17:36:51,063][105620] Updated weights for policy 1, policy_version 309306 (0.0009) [2023-12-26 17:36:51,116][105620] Updated weights for policy 1, policy_version 309316 (0.0008) [2023-12-26 17:36:51,712][105692] Updated weights for policy 0, policy_version 309164 (0.0008) [2023-12-26 17:36:51,782][105692] Updated weights for policy 0, policy_version 309174 (0.0008) [2023-12-26 17:36:51,840][105692] Updated weights for policy 0, policy_version 309184 (0.0010) [2023-12-26 17:36:51,937][105620] Updated weights for policy 1, policy_version 309326 (0.0007) [2023-12-26 17:36:51,997][105620] Updated weights for policy 1, policy_version 309336 (0.0006) [2023-12-26 17:36:52,048][105620] Updated weights for policy 1, policy_version 309346 (0.0008) [2023-12-26 17:36:52,610][105692] Updated weights for policy 0, policy_version 309194 (0.0010) [2023-12-26 17:36:52,668][105692] Updated weights for policy 0, policy_version 309204 (0.0009) [2023-12-26 17:36:52,715][105692] Updated weights for policy 0, policy_version 309214 (0.0008) [2023-12-26 17:36:52,756][105620] Updated weights for policy 1, policy_version 309356 (0.0009) [2023-12-26 17:36:52,766][105692] Updated weights for policy 0, policy_version 309224 (0.0008) [2023-12-26 17:36:52,815][105620] Updated weights for policy 1, policy_version 309366 (0.0008) [2023-12-26 17:36:52,878][105620] Updated weights for policy 1, policy_version 309376 (0.0009) [2023-12-26 17:36:53,389][105692] Updated weights for policy 0, policy_version 309234 (0.0010) [2023-12-26 17:36:53,441][105692] Updated weights for policy 0, policy_version 309244 (0.0009) [2023-12-26 17:36:53,503][105692] Updated weights for policy 0, policy_version 309254 (0.0009) [2023-12-26 17:36:53,531][105620] Updated weights for policy 1, policy_version 309386 (0.0008) [2023-12-26 17:36:53,586][105620] Updated weights for policy 1, policy_version 309396 (0.0009) [2023-12-26 17:36:53,653][105620] Updated weights for policy 1, policy_version 309406 (0.0006) [2023-12-26 17:36:53,720][105620] Updated weights for policy 1, policy_version 309416 (0.0005) [2023-12-26 17:36:54,311][105692] Updated weights for policy 0, policy_version 309264 (0.0009) [2023-12-26 17:36:54,361][105692] Updated weights for policy 0, policy_version 309274 (0.0007) [2023-12-26 17:36:54,367][105620] Updated weights for policy 1, policy_version 309426 (0.0008) [2023-12-26 17:36:54,414][105692] Updated weights for policy 0, policy_version 309284 (0.0007) [2023-12-26 17:36:54,417][105620] Updated weights for policy 1, policy_version 309436 (0.0007) [2023-12-26 17:36:54,472][105620] Updated weights for policy 1, policy_version 309446 (0.0006) [2023-12-26 17:36:55,160][105692] Updated weights for policy 0, policy_version 309294 (0.0008) [2023-12-26 17:36:55,214][105692] Updated weights for policy 0, policy_version 309304 (0.0007) [2023-12-26 17:36:55,223][105620] Updated weights for policy 1, policy_version 309456 (0.0007) [2023-12-26 17:36:55,259][105692] Updated weights for policy 0, policy_version 309314 (0.0008) [2023-12-26 17:36:55,282][105620] Updated weights for policy 1, policy_version 309466 (0.0008) [2023-12-26 17:36:55,334][105620] Updated weights for policy 1, policy_version 309476 (0.0008) [2023-12-26 17:36:56,040][105620] Updated weights for policy 1, policy_version 309486 (0.0009) [2023-12-26 17:36:56,061][105692] Updated weights for policy 0, policy_version 309324 (0.0007) [2023-12-26 17:36:56,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 158433280. Throughput: 0: 9766.8, 1: 9822.8. Samples: 158448212. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 17:36:56,062][104569] Avg episode reward: [(0, '4551.432'), (1, '8990.289')] [2023-12-26 17:36:56,094][105620] Updated weights for policy 1, policy_version 309496 (0.0006) [2023-12-26 17:36:56,117][105692] Updated weights for policy 0, policy_version 309334 (0.0008) [2023-12-26 17:36:56,152][105620] Updated weights for policy 1, policy_version 309506 (0.0007) [2023-12-26 17:36:56,170][105692] Updated weights for policy 0, policy_version 309344 (0.0006) [2023-12-26 17:36:56,892][105620] Updated weights for policy 1, policy_version 309516 (0.0008) [2023-12-26 17:36:56,898][105692] Updated weights for policy 0, policy_version 309354 (0.0008) [2023-12-26 17:36:56,936][105620] Updated weights for policy 1, policy_version 309526 (0.0005) [2023-12-26 17:36:56,953][105692] Updated weights for policy 0, policy_version 309364 (0.0010) [2023-12-26 17:36:56,986][105620] Updated weights for policy 1, policy_version 309536 (0.0007) [2023-12-26 17:36:57,010][105692] Updated weights for policy 0, policy_version 309374 (0.0007) [2023-12-26 17:36:57,061][105692] Updated weights for policy 0, policy_version 309384 (0.0008) [2023-12-26 17:36:57,670][105620] Updated weights for policy 1, policy_version 309546 (0.0008) [2023-12-26 17:36:57,696][105692] Updated weights for policy 0, policy_version 309394 (0.0007) [2023-12-26 17:36:57,729][105620] Updated weights for policy 1, policy_version 309556 (0.0007) [2023-12-26 17:36:57,751][105692] Updated weights for policy 0, policy_version 309404 (0.0006) [2023-12-26 17:36:57,779][105586] KL-divergence is very high: 111.3528 [2023-12-26 17:36:57,781][105620] Updated weights for policy 1, policy_version 309566 (0.0006) [2023-12-26 17:36:57,803][105692] Updated weights for policy 0, policy_version 309414 (0.0009) [2023-12-26 17:36:57,823][105586] KL-divergence is very high: 165.0895 [2023-12-26 17:36:57,837][105620] Updated weights for policy 1, policy_version 309576 (0.0006) [2023-12-26 17:36:58,461][105585] KL-divergence is very high: 100.4782 [2023-12-26 17:36:58,462][105692] Updated weights for policy 0, policy_version 309424 (0.0010) [2023-12-26 17:36:58,519][105585] KL-divergence is very high: 108.2616 [2023-12-26 17:36:58,534][105692] Updated weights for policy 0, policy_version 309434 (0.0011) [2023-12-26 17:36:58,599][105692] Updated weights for policy 0, policy_version 309444 (0.0009) [2023-12-26 17:36:58,632][105620] Updated weights for policy 1, policy_version 309586 (0.0010) [2023-12-26 17:36:58,698][105620] Updated weights for policy 1, policy_version 309596 (0.0011) [2023-12-26 17:36:58,778][105620] Updated weights for policy 1, policy_version 309607 (0.0009) [2023-12-26 17:36:59,405][105692] Updated weights for policy 0, policy_version 309454 (0.0009) [2023-12-26 17:36:59,453][105692] Updated weights for policy 0, policy_version 309464 (0.0005) [2023-12-26 17:36:59,508][105692] Updated weights for policy 0, policy_version 309474 (0.0008) [2023-12-26 17:36:59,631][105620] Updated weights for policy 1, policy_version 309617 (0.0009) [2023-12-26 17:36:59,684][105620] Updated weights for policy 1, policy_version 309627 (0.0008) [2023-12-26 17:36:59,734][105620] Updated weights for policy 1, policy_version 309637 (0.0005) [2023-12-26 17:37:00,294][105692] Updated weights for policy 0, policy_version 309484 (0.0008) [2023-12-26 17:37:00,358][105692] Updated weights for policy 0, policy_version 309494 (0.0006) [2023-12-26 17:37:00,378][105620] Updated weights for policy 1, policy_version 309647 (0.0008) [2023-12-26 17:37:00,418][105692] Updated weights for policy 0, policy_version 309504 (0.0005) [2023-12-26 17:37:00,433][105620] Updated weights for policy 1, policy_version 309657 (0.0009) [2023-12-26 17:37:00,457][105585] KL-divergence is very high: 100.1989 [2023-12-26 17:37:00,490][105620] Updated weights for policy 1, policy_version 309667 (0.0008) [2023-12-26 17:37:01,041][105620] Updated weights for policy 1, policy_version 309677 (0.0007) [2023-12-26 17:37:01,054][105692] Updated weights for policy 0, policy_version 309514 (0.0009) [2023-12-26 17:37:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 158531584. Throughput: 0: 9867.2, 1: 9752.6. Samples: 158505824. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 17:37:01,063][104569] Avg episode reward: [(0, '2205.603'), (1, '8805.184')] [2023-12-26 17:37:01,101][105620] Updated weights for policy 1, policy_version 309687 (0.0011) [2023-12-26 17:37:01,108][105692] Updated weights for policy 0, policy_version 309524 (0.0009) [2023-12-26 17:37:01,164][105620] Updated weights for policy 1, policy_version 309697 (0.0009) [2023-12-26 17:37:01,175][105692] Updated weights for policy 0, policy_version 309534 (0.0010) [2023-12-26 17:37:01,210][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000309704_79290368.pth... [2023-12-26 17:37:01,217][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000308552_78995456.pth [2023-12-26 17:37:01,241][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000309544_79257600.pth... [2023-12-26 17:37:01,243][105692] Updated weights for policy 0, policy_version 309544 (0.0006) [2023-12-26 17:37:01,247][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000308392_78962688.pth [2023-12-26 17:37:01,914][105692] Updated weights for policy 0, policy_version 309554 (0.0011) [2023-12-26 17:37:01,933][105620] Updated weights for policy 1, policy_version 309707 (0.0007) [2023-12-26 17:37:01,969][105692] Updated weights for policy 0, policy_version 309564 (0.0010) [2023-12-26 17:37:01,988][105620] Updated weights for policy 1, policy_version 309717 (0.0006) [2023-12-26 17:37:02,028][105692] Updated weights for policy 0, policy_version 309574 (0.0010) [2023-12-26 17:37:02,047][105620] Updated weights for policy 1, policy_version 309727 (0.0007) [2023-12-26 17:37:02,774][105620] Updated weights for policy 1, policy_version 309737 (0.0008) [2023-12-26 17:37:02,779][105692] Updated weights for policy 0, policy_version 309584 (0.0010) [2023-12-26 17:37:02,824][105620] Updated weights for policy 1, policy_version 309747 (0.0006) [2023-12-26 17:37:02,845][105692] Updated weights for policy 0, policy_version 309594 (0.0011) [2023-12-26 17:37:02,879][105620] Updated weights for policy 1, policy_version 309757 (0.0007) [2023-12-26 17:37:02,910][105692] Updated weights for policy 0, policy_version 309604 (0.0009) [2023-12-26 17:37:02,937][105620] Updated weights for policy 1, policy_version 309767 (0.0006) [2023-12-26 17:37:03,514][105620] Updated weights for policy 1, policy_version 309777 (0.0009) [2023-12-26 17:37:03,572][105620] Updated weights for policy 1, policy_version 309787 (0.0008) [2023-12-26 17:37:03,633][105620] Updated weights for policy 1, policy_version 309797 (0.0009) [2023-12-26 17:37:03,680][105692] Updated weights for policy 0, policy_version 309614 (0.0008) [2023-12-26 17:37:03,737][105692] Updated weights for policy 0, policy_version 309624 (0.0009) [2023-12-26 17:37:03,787][105692] Updated weights for policy 0, policy_version 309634 (0.0009) [2023-12-26 17:37:04,399][105620] Updated weights for policy 1, policy_version 309807 (0.0009) [2023-12-26 17:37:04,461][105620] Updated weights for policy 1, policy_version 309817 (0.0009) [2023-12-26 17:37:04,514][105620] Updated weights for policy 1, policy_version 309827 (0.0011) [2023-12-26 17:37:04,580][105692] Updated weights for policy 0, policy_version 309644 (0.0008) [2023-12-26 17:37:04,636][105692] Updated weights for policy 0, policy_version 309654 (0.0008) [2023-12-26 17:37:04,700][105692] Updated weights for policy 0, policy_version 309664 (0.0009) [2023-12-26 17:37:05,264][105620] Updated weights for policy 1, policy_version 309837 (0.0008) [2023-12-26 17:37:05,332][105692] Updated weights for policy 0, policy_version 309674 (0.0010) [2023-12-26 17:37:05,333][105620] Updated weights for policy 1, policy_version 309847 (0.0006) [2023-12-26 17:37:05,389][105692] Updated weights for policy 0, policy_version 309684 (0.0005) [2023-12-26 17:37:05,390][105620] Updated weights for policy 1, policy_version 309857 (0.0006) [2023-12-26 17:37:05,447][105692] Updated weights for policy 0, policy_version 309694 (0.0006) [2023-12-26 17:37:05,504][105692] Updated weights for policy 0, policy_version 309704 (0.0005) [2023-12-26 17:37:06,044][105692] Updated weights for policy 0, policy_version 309714 (0.0005) [2023-12-26 17:37:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 158629888. Throughput: 0: 9781.6, 1: 9801.6. Samples: 158622616. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 17:37:06,063][104569] Avg episode reward: [(0, '3561.882'), (1, '9081.756')] [2023-12-26 17:37:06,099][105692] Updated weights for policy 0, policy_version 309724 (0.0008) [2023-12-26 17:37:06,129][105620] Updated weights for policy 1, policy_version 309867 (0.0007) [2023-12-26 17:37:06,168][105692] Updated weights for policy 0, policy_version 309734 (0.0008) [2023-12-26 17:37:06,190][105620] Updated weights for policy 1, policy_version 309877 (0.0006) [2023-12-26 17:37:06,251][105620] Updated weights for policy 1, policy_version 309887 (0.0006) [2023-12-26 17:37:06,854][105620] Updated weights for policy 1, policy_version 309897 (0.0009) [2023-12-26 17:37:06,918][105620] Updated weights for policy 1, policy_version 309907 (0.0009) [2023-12-26 17:37:06,951][105692] Updated weights for policy 0, policy_version 309744 (0.0008) [2023-12-26 17:37:06,978][105620] Updated weights for policy 1, policy_version 309917 (0.0006) [2023-12-26 17:37:07,011][105692] Updated weights for policy 0, policy_version 309754 (0.0008) [2023-12-26 17:37:07,046][105620] Updated weights for policy 1, policy_version 309927 (0.0005) [2023-12-26 17:37:07,080][105692] Updated weights for policy 0, policy_version 309764 (0.0010) [2023-12-26 17:37:07,614][105620] Updated weights for policy 1, policy_version 309937 (0.0006) [2023-12-26 17:37:07,670][105620] Updated weights for policy 1, policy_version 309947 (0.0005) [2023-12-26 17:37:07,732][105620] Updated weights for policy 1, policy_version 309957 (0.0006) [2023-12-26 17:37:07,792][105692] Updated weights for policy 0, policy_version 309774 (0.0008) [2023-12-26 17:37:07,846][105692] Updated weights for policy 0, policy_version 309784 (0.0008) [2023-12-26 17:37:07,909][105692] Updated weights for policy 0, policy_version 309794 (0.0005) [2023-12-26 17:37:08,258][105620] Updated weights for policy 1, policy_version 309967 (0.0009) [2023-12-26 17:37:08,317][105620] Updated weights for policy 1, policy_version 309977 (0.0010) [2023-12-26 17:37:08,379][105620] Updated weights for policy 1, policy_version 309987 (0.0010) [2023-12-26 17:37:08,647][105692] Updated weights for policy 0, policy_version 309804 (0.0011) [2023-12-26 17:37:08,707][105692] Updated weights for policy 0, policy_version 309814 (0.0011) [2023-12-26 17:37:08,763][105692] Updated weights for policy 0, policy_version 309824 (0.0010) [2023-12-26 17:37:08,977][105620] Updated weights for policy 1, policy_version 309997 (0.0006) [2023-12-26 17:37:09,030][105620] Updated weights for policy 1, policy_version 310007 (0.0005) [2023-12-26 17:37:09,087][105620] Updated weights for policy 1, policy_version 310017 (0.0009) [2023-12-26 17:37:09,554][105692] Updated weights for policy 0, policy_version 309834 (0.0010) [2023-12-26 17:37:09,613][105692] Updated weights for policy 0, policy_version 309844 (0.0011) [2023-12-26 17:37:09,673][105692] Updated weights for policy 0, policy_version 309854 (0.0011) [2023-12-26 17:37:09,736][105692] Updated weights for policy 0, policy_version 309864 (0.0011) [2023-12-26 17:37:09,758][105620] Updated weights for policy 1, policy_version 310027 (0.0007) [2023-12-26 17:37:09,814][105620] Updated weights for policy 1, policy_version 310037 (0.0011) [2023-12-26 17:37:09,881][105620] Updated weights for policy 1, policy_version 310047 (0.0011) [2023-12-26 17:37:10,398][105692] Updated weights for policy 0, policy_version 309874 (0.0008) [2023-12-26 17:37:10,447][105692] Updated weights for policy 0, policy_version 309884 (0.0008) [2023-12-26 17:37:10,497][105692] Updated weights for policy 0, policy_version 309894 (0.0008) [2023-12-26 17:37:10,630][105620] Updated weights for policy 1, policy_version 310057 (0.0009) [2023-12-26 17:37:10,692][105620] Updated weights for policy 1, policy_version 310067 (0.0010) [2023-12-26 17:37:10,743][105620] Updated weights for policy 1, policy_version 310077 (0.0010) [2023-12-26 17:37:10,795][105620] Updated weights for policy 1, policy_version 310087 (0.0009) [2023-12-26 17:37:11,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 158736384. Throughput: 0: 9714.3, 1: 9932.0. Samples: 158744548. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 17:37:11,062][104569] Avg episode reward: [(0, '2088.328'), (1, '7818.221')] [2023-12-26 17:37:11,245][105692] Updated weights for policy 0, policy_version 309904 (0.0011) [2023-12-26 17:37:11,304][105692] Updated weights for policy 0, policy_version 309914 (0.0011) [2023-12-26 17:37:11,364][105692] Updated weights for policy 0, policy_version 309924 (0.0012) [2023-12-26 17:37:11,572][105620] Updated weights for policy 1, policy_version 310097 (0.0010) [2023-12-26 17:37:11,660][105620] Updated weights for policy 1, policy_version 310107 (0.0010) [2023-12-26 17:37:11,731][105620] Updated weights for policy 1, policy_version 310117 (0.0008) [2023-12-26 17:37:12,119][105692] Updated weights for policy 0, policy_version 309934 (0.0009) [2023-12-26 17:37:12,181][105692] Updated weights for policy 0, policy_version 309944 (0.0009) [2023-12-26 17:37:12,243][105692] Updated weights for policy 0, policy_version 309954 (0.0009) [2023-12-26 17:37:12,361][105620] Updated weights for policy 1, policy_version 310127 (0.0008) [2023-12-26 17:37:12,423][105620] Updated weights for policy 1, policy_version 310137 (0.0009) [2023-12-26 17:37:12,484][105620] Updated weights for policy 1, policy_version 310147 (0.0009) [2023-12-26 17:37:13,006][105692] Updated weights for policy 0, policy_version 309964 (0.0010) [2023-12-26 17:37:13,076][105692] Updated weights for policy 0, policy_version 309974 (0.0010) [2023-12-26 17:37:13,143][105692] Updated weights for policy 0, policy_version 309984 (0.0010) [2023-12-26 17:37:13,156][105620] Updated weights for policy 1, policy_version 310157 (0.0007) [2023-12-26 17:37:13,209][105620] Updated weights for policy 1, policy_version 310167 (0.0005) [2023-12-26 17:37:13,255][105620] Updated weights for policy 1, policy_version 310177 (0.0006) [2023-12-26 17:37:13,759][105692] Updated weights for policy 0, policy_version 309994 (0.0008) [2023-12-26 17:37:13,818][105692] Updated weights for policy 0, policy_version 310004 (0.0005) [2023-12-26 17:37:13,879][105692] Updated weights for policy 0, policy_version 310014 (0.0005) [2023-12-26 17:37:13,939][105692] Updated weights for policy 0, policy_version 310024 (0.0005) [2023-12-26 17:37:14,038][105620] Updated weights for policy 1, policy_version 310187 (0.0008) [2023-12-26 17:37:14,110][105620] Updated weights for policy 1, policy_version 310197 (0.0005) [2023-12-26 17:37:14,178][105620] Updated weights for policy 1, policy_version 310207 (0.0005) [2023-12-26 17:37:14,519][105692] Updated weights for policy 0, policy_version 310034 (0.0005) [2023-12-26 17:37:14,569][105692] Updated weights for policy 0, policy_version 310044 (0.0005) [2023-12-26 17:37:14,626][105692] Updated weights for policy 0, policy_version 310054 (0.0010) [2023-12-26 17:37:14,791][105620] Updated weights for policy 1, policy_version 310217 (0.0006) [2023-12-26 17:37:14,850][105620] Updated weights for policy 1, policy_version 310227 (0.0008) [2023-12-26 17:37:14,919][105620] Updated weights for policy 1, policy_version 310237 (0.0009) [2023-12-26 17:37:14,983][105620] Updated weights for policy 1, policy_version 310247 (0.0006) [2023-12-26 17:37:15,381][105692] Updated weights for policy 0, policy_version 310064 (0.0011) [2023-12-26 17:37:15,443][105692] Updated weights for policy 0, policy_version 310074 (0.0010) [2023-12-26 17:37:15,494][105692] Updated weights for policy 0, policy_version 310084 (0.0009) [2023-12-26 17:37:15,729][105620] Updated weights for policy 1, policy_version 310257 (0.0007) [2023-12-26 17:37:15,792][105620] Updated weights for policy 1, policy_version 310267 (0.0005) [2023-12-26 17:37:15,857][105620] Updated weights for policy 1, policy_version 310277 (0.0007) [2023-12-26 17:37:16,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19797.3, 300 sec: 19521.9). Total num frames: 158834688. Throughput: 0: 9641.2, 1: 9893.4. Samples: 158802152. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 17:37:16,063][104569] Avg episode reward: [(0, '791.717'), (1, '8261.341')] [2023-12-26 17:37:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000310088_79396864.pth... [2023-12-26 17:37:16,074][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000310280_79437824.pth... [2023-12-26 17:37:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000308968_79110144.pth [2023-12-26 17:37:16,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000309128_79142912.pth [2023-12-26 17:37:16,149][105692] Updated weights for policy 0, policy_version 310094 (0.0007) [2023-12-26 17:37:16,202][105692] Updated weights for policy 0, policy_version 310104 (0.0005) [2023-12-26 17:37:16,251][105692] Updated weights for policy 0, policy_version 310114 (0.0005) [2023-12-26 17:37:16,382][105620] Updated weights for policy 1, policy_version 310287 (0.0007) [2023-12-26 17:37:16,435][105620] Updated weights for policy 1, policy_version 310297 (0.0005) [2023-12-26 17:37:16,489][105620] Updated weights for policy 1, policy_version 310307 (0.0005) [2023-12-26 17:37:16,947][105692] Updated weights for policy 0, policy_version 310124 (0.0007) [2023-12-26 17:37:16,998][105692] Updated weights for policy 0, policy_version 310134 (0.0008) [2023-12-26 17:37:17,050][105692] Updated weights for policy 0, policy_version 310144 (0.0008) [2023-12-26 17:37:17,105][105620] Updated weights for policy 1, policy_version 310317 (0.0008) [2023-12-26 17:37:17,159][105620] Updated weights for policy 1, policy_version 310327 (0.0010) [2023-12-26 17:37:17,204][105620] Updated weights for policy 1, policy_version 310337 (0.0010) [2023-12-26 17:37:17,846][105692] Updated weights for policy 0, policy_version 310154 (0.0008) [2023-12-26 17:37:17,881][105620] Updated weights for policy 1, policy_version 310347 (0.0009) [2023-12-26 17:37:17,907][105692] Updated weights for policy 0, policy_version 310164 (0.0009) [2023-12-26 17:37:17,941][105620] Updated weights for policy 1, policy_version 310357 (0.0005) [2023-12-26 17:37:17,967][105692] Updated weights for policy 0, policy_version 310174 (0.0007) [2023-12-26 17:37:17,990][105620] Updated weights for policy 1, policy_version 310367 (0.0006) [2023-12-26 17:37:18,031][105692] Updated weights for policy 0, policy_version 310184 (0.0008) [2023-12-26 17:37:18,597][105620] Updated weights for policy 1, policy_version 310377 (0.0007) [2023-12-26 17:37:18,665][105620] Updated weights for policy 1, policy_version 310387 (0.0007) [2023-12-26 17:37:18,685][105586] KL-divergence is very high: 174.2295 [2023-12-26 17:37:18,723][105620] Updated weights for policy 1, policy_version 310397 (0.0009) [2023-12-26 17:37:18,735][105586] KL-divergence is very high: 304.8998 [2023-12-26 17:37:18,786][105620] Updated weights for policy 1, policy_version 310407 (0.0009) [2023-12-26 17:37:18,786][105586] KL-divergence is very high: 300.6652 [2023-12-26 17:37:18,853][105692] Updated weights for policy 0, policy_version 310194 (0.0010) [2023-12-26 17:37:18,910][105692] Updated weights for policy 0, policy_version 310204 (0.0010) [2023-12-26 17:37:18,964][105692] Updated weights for policy 0, policy_version 310214 (0.0009) [2023-12-26 17:37:19,379][105586] KL-divergence is very high: 218.5841 [2023-12-26 17:37:19,429][105620] Updated weights for policy 1, policy_version 310417 (0.0006) [2023-12-26 17:37:19,431][105586] KL-divergence is very high: 146.7175 [2023-12-26 17:37:19,483][105586] KL-divergence is very high: 122.0244 [2023-12-26 17:37:19,497][105620] Updated weights for policy 1, policy_version 310427 (0.0006) [2023-12-26 17:37:19,560][105620] Updated weights for policy 1, policy_version 310437 (0.0006) [2023-12-26 17:37:19,647][105692] Updated weights for policy 0, policy_version 310224 (0.0009) [2023-12-26 17:37:19,718][105692] Updated weights for policy 0, policy_version 310234 (0.0009) [2023-12-26 17:37:19,787][105692] Updated weights for policy 0, policy_version 310244 (0.0008) [2023-12-26 17:37:20,210][105620] Updated weights for policy 1, policy_version 310447 (0.0007) [2023-12-26 17:37:20,274][105620] Updated weights for policy 1, policy_version 310457 (0.0009) [2023-12-26 17:37:20,333][105620] Updated weights for policy 1, policy_version 310467 (0.0009) [2023-12-26 17:37:20,560][105692] Updated weights for policy 0, policy_version 310254 (0.0009) [2023-12-26 17:37:20,629][105692] Updated weights for policy 0, policy_version 310264 (0.0009) [2023-12-26 17:37:20,693][105692] Updated weights for policy 0, policy_version 310274 (0.0010) [2023-12-26 17:37:20,996][105620] Updated weights for policy 1, policy_version 310477 (0.0007) [2023-12-26 17:37:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 158932992. Throughput: 0: 9688.4, 1: 9915.2. Samples: 158925868. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 17:37:21,062][104569] Avg episode reward: [(0, '1497.699'), (1, '8990.352')] [2023-12-26 17:37:21,066][105620] Updated weights for policy 1, policy_version 310487 (0.0007) [2023-12-26 17:37:21,139][105620] Updated weights for policy 1, policy_version 310497 (0.0009) [2023-12-26 17:37:21,465][105692] Updated weights for policy 0, policy_version 310284 (0.0009) [2023-12-26 17:37:21,525][105692] Updated weights for policy 0, policy_version 310294 (0.0008) [2023-12-26 17:37:21,581][105692] Updated weights for policy 0, policy_version 310304 (0.0008) [2023-12-26 17:37:21,882][105620] Updated weights for policy 1, policy_version 310507 (0.0008) [2023-12-26 17:37:21,926][105620] Updated weights for policy 1, policy_version 310517 (0.0008) [2023-12-26 17:37:21,974][105620] Updated weights for policy 1, policy_version 310527 (0.0008) [2023-12-26 17:37:22,408][105692] Updated weights for policy 0, policy_version 310314 (0.0008) [2023-12-26 17:37:22,476][105692] Updated weights for policy 0, policy_version 310324 (0.0009) [2023-12-26 17:37:22,530][105692] Updated weights for policy 0, policy_version 310334 (0.0009) [2023-12-26 17:37:22,580][105692] Updated weights for policy 0, policy_version 310344 (0.0009) [2023-12-26 17:37:22,647][105620] Updated weights for policy 1, policy_version 310537 (0.0008) [2023-12-26 17:37:22,706][105620] Updated weights for policy 1, policy_version 310547 (0.0009) [2023-12-26 17:37:22,761][105620] Updated weights for policy 1, policy_version 310557 (0.0009) [2023-12-26 17:37:22,817][105620] Updated weights for policy 1, policy_version 310567 (0.0009) [2023-12-26 17:37:23,347][105692] Updated weights for policy 0, policy_version 310354 (0.0009) [2023-12-26 17:37:23,411][105692] Updated weights for policy 0, policy_version 310364 (0.0010) [2023-12-26 17:37:23,473][105692] Updated weights for policy 0, policy_version 310374 (0.0009) [2023-12-26 17:37:23,539][105620] Updated weights for policy 1, policy_version 310577 (0.0006) [2023-12-26 17:37:23,590][105620] Updated weights for policy 1, policy_version 310587 (0.0005) [2023-12-26 17:37:23,660][105620] Updated weights for policy 1, policy_version 310597 (0.0005) [2023-12-26 17:37:24,175][105620] Updated weights for policy 1, policy_version 310607 (0.0009) [2023-12-26 17:37:24,227][105620] Updated weights for policy 1, policy_version 310617 (0.0010) [2023-12-26 17:37:24,279][105620] Updated weights for policy 1, policy_version 310627 (0.0010) [2023-12-26 17:37:24,319][105692] Updated weights for policy 0, policy_version 310384 (0.0007) [2023-12-26 17:37:24,371][105692] Updated weights for policy 0, policy_version 310394 (0.0008) [2023-12-26 17:37:24,426][105692] Updated weights for policy 0, policy_version 310404 (0.0008) [2023-12-26 17:37:24,913][105620] Updated weights for policy 1, policy_version 310637 (0.0008) [2023-12-26 17:37:24,965][105620] Updated weights for policy 1, policy_version 310647 (0.0005) [2023-12-26 17:37:25,019][105620] Updated weights for policy 1, policy_version 310657 (0.0007) [2023-12-26 17:37:25,316][105692] Updated weights for policy 0, policy_version 310414 (0.0009) [2023-12-26 17:37:25,374][105692] Updated weights for policy 0, policy_version 310424 (0.0009) [2023-12-26 17:37:25,444][105692] Updated weights for policy 0, policy_version 310434 (0.0010) [2023-12-26 17:37:25,570][105620] Updated weights for policy 1, policy_version 310667 (0.0008) [2023-12-26 17:37:25,621][105620] Updated weights for policy 1, policy_version 310677 (0.0005) [2023-12-26 17:37:25,669][105620] Updated weights for policy 1, policy_version 310687 (0.0005) [2023-12-26 17:37:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 159031296. Throughput: 0: 9620.3, 1: 10019.9. Samples: 159040396. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 17:37:26,062][104569] Avg episode reward: [(0, '4004.805'), (1, '9082.216')] [2023-12-26 17:37:26,245][105692] Updated weights for policy 0, policy_version 310444 (0.0008) [2023-12-26 17:37:26,297][105692] Updated weights for policy 0, policy_version 310454 (0.0008) [2023-12-26 17:37:26,345][105692] Updated weights for policy 0, policy_version 310464 (0.0008) [2023-12-26 17:37:26,359][105620] Updated weights for policy 1, policy_version 310697 (0.0006) [2023-12-26 17:37:26,404][105620] Updated weights for policy 1, policy_version 310707 (0.0010) [2023-12-26 17:37:26,452][105620] Updated weights for policy 1, policy_version 310717 (0.0010) [2023-12-26 17:37:26,510][105620] Updated weights for policy 1, policy_version 310727 (0.0010) [2023-12-26 17:37:27,116][105692] Updated weights for policy 0, policy_version 310474 (0.0006) [2023-12-26 17:37:27,170][105692] Updated weights for policy 0, policy_version 310484 (0.0008) [2023-12-26 17:37:27,226][105692] Updated weights for policy 0, policy_version 310494 (0.0008) [2023-12-26 17:37:27,262][105620] Updated weights for policy 1, policy_version 310737 (0.0007) [2023-12-26 17:37:27,287][105692] Updated weights for policy 0, policy_version 310504 (0.0008) [2023-12-26 17:37:27,315][105620] Updated weights for policy 1, policy_version 310747 (0.0008) [2023-12-26 17:37:27,375][105620] Updated weights for policy 1, policy_version 310757 (0.0009) [2023-12-26 17:37:28,046][105692] Updated weights for policy 0, policy_version 310514 (0.0008) [2023-12-26 17:37:28,060][105620] Updated weights for policy 1, policy_version 310767 (0.0010) [2023-12-26 17:37:28,101][105692] Updated weights for policy 0, policy_version 310524 (0.0006) [2023-12-26 17:37:28,108][105620] Updated weights for policy 1, policy_version 310777 (0.0010) [2023-12-26 17:37:28,155][105692] Updated weights for policy 0, policy_version 310534 (0.0006) [2023-12-26 17:37:28,160][105620] Updated weights for policy 1, policy_version 310787 (0.0007) [2023-12-26 17:37:28,758][105620] Updated weights for policy 1, policy_version 310797 (0.0005) [2023-12-26 17:37:28,813][105620] Updated weights for policy 1, policy_version 310807 (0.0007) [2023-12-26 17:37:28,870][105620] Updated weights for policy 1, policy_version 310817 (0.0007) [2023-12-26 17:37:29,014][105692] Updated weights for policy 0, policy_version 310544 (0.0010) [2023-12-26 17:37:29,071][105692] Updated weights for policy 0, policy_version 310555 (0.0010) [2023-12-26 17:37:29,119][105692] Updated weights for policy 0, policy_version 310565 (0.0008) [2023-12-26 17:37:29,501][105620] Updated weights for policy 1, policy_version 310827 (0.0009) [2023-12-26 17:37:29,559][105620] Updated weights for policy 1, policy_version 310837 (0.0010) [2023-12-26 17:37:29,616][105620] Updated weights for policy 1, policy_version 310847 (0.0008) [2023-12-26 17:37:29,983][105692] Updated weights for policy 0, policy_version 310575 (0.0008) [2023-12-26 17:37:30,051][105692] Updated weights for policy 0, policy_version 310585 (0.0008) [2023-12-26 17:37:30,118][105692] Updated weights for policy 0, policy_version 310595 (0.0008) [2023-12-26 17:37:30,349][105620] Updated weights for policy 1, policy_version 310857 (0.0006) [2023-12-26 17:37:30,407][105620] Updated weights for policy 1, policy_version 310867 (0.0009) [2023-12-26 17:37:30,456][105620] Updated weights for policy 1, policy_version 310877 (0.0009) [2023-12-26 17:37:30,514][105620] Updated weights for policy 1, policy_version 310887 (0.0010) [2023-12-26 17:37:30,768][105692] Updated weights for policy 0, policy_version 310605 (0.0006) [2023-12-26 17:37:30,826][105692] Updated weights for policy 0, policy_version 310615 (0.0006) [2023-12-26 17:37:30,888][105692] Updated weights for policy 0, policy_version 310625 (0.0005) [2023-12-26 17:37:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 159129600. Throughput: 0: 9552.6, 1: 10086.3. Samples: 159098232. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 17:37:31,063][104569] Avg episode reward: [(0, '6747.421'), (1, '9174.394')] [2023-12-26 17:37:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000310632_79536128.pth... [2023-12-26 17:37:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000310888_79593472.pth... [2023-12-26 17:37:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000309704_79290368.pth [2023-12-26 17:37:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000309544_79257600.pth [2023-12-26 17:37:31,163][105620] Updated weights for policy 1, policy_version 310897 (0.0008) [2023-12-26 17:37:31,225][105620] Updated weights for policy 1, policy_version 310907 (0.0008) [2023-12-26 17:37:31,286][105620] Updated weights for policy 1, policy_version 310917 (0.0008) [2023-12-26 17:37:31,536][105692] Updated weights for policy 0, policy_version 310635 (0.0007) [2023-12-26 17:37:31,595][105692] Updated weights for policy 0, policy_version 310645 (0.0010) [2023-12-26 17:37:31,657][105692] Updated weights for policy 0, policy_version 310655 (0.0011) [2023-12-26 17:37:31,939][105620] Updated weights for policy 1, policy_version 310927 (0.0010) [2023-12-26 17:37:31,956][105586] KL-divergence is very high: 108.3730 [2023-12-26 17:37:31,989][105586] KL-divergence is very high: 124.6667 [2023-12-26 17:37:31,994][105620] Updated weights for policy 1, policy_version 310937 (0.0010) [2023-12-26 17:37:32,001][105586] KL-divergence is very high: 155.8316 [2023-12-26 17:37:32,045][105586] KL-divergence is very high: 125.7970 [2023-12-26 17:37:32,052][105620] Updated weights for policy 1, policy_version 310947 (0.0010) [2023-12-26 17:37:32,456][105692] Updated weights for policy 0, policy_version 310665 (0.0011) [2023-12-26 17:37:32,512][105692] Updated weights for policy 0, policy_version 310675 (0.0009) [2023-12-26 17:37:32,578][105692] Updated weights for policy 0, policy_version 310685 (0.0010) [2023-12-26 17:37:32,633][105692] Updated weights for policy 0, policy_version 310695 (0.0008) [2023-12-26 17:37:32,720][105620] Updated weights for policy 1, policy_version 310957 (0.0010) [2023-12-26 17:37:32,775][105620] Updated weights for policy 1, policy_version 310967 (0.0010) [2023-12-26 17:37:32,823][105620] Updated weights for policy 1, policy_version 310977 (0.0010) [2023-12-26 17:37:33,428][105692] Updated weights for policy 0, policy_version 310705 (0.0005) [2023-12-26 17:37:33,484][105692] Updated weights for policy 0, policy_version 310715 (0.0007) [2023-12-26 17:37:33,541][105692] Updated weights for policy 0, policy_version 310725 (0.0008) [2023-12-26 17:37:33,583][105620] Updated weights for policy 1, policy_version 310987 (0.0010) [2023-12-26 17:37:33,635][105620] Updated weights for policy 1, policy_version 310997 (0.0010) [2023-12-26 17:37:33,679][105620] Updated weights for policy 1, policy_version 311007 (0.0010) [2023-12-26 17:37:34,265][105692] Updated weights for policy 0, policy_version 310735 (0.0006) [2023-12-26 17:37:34,325][105692] Updated weights for policy 0, policy_version 310745 (0.0009) [2023-12-26 17:37:34,388][105692] Updated weights for policy 0, policy_version 310755 (0.0011) [2023-12-26 17:37:34,446][105620] Updated weights for policy 1, policy_version 311017 (0.0010) [2023-12-26 17:37:34,517][105620] Updated weights for policy 1, policy_version 311027 (0.0009) [2023-12-26 17:37:34,583][105620] Updated weights for policy 1, policy_version 311037 (0.0011) [2023-12-26 17:37:34,640][105620] Updated weights for policy 1, policy_version 311047 (0.0008) [2023-12-26 17:37:35,118][105692] Updated weights for policy 0, policy_version 310765 (0.0008) [2023-12-26 17:37:35,178][105692] Updated weights for policy 0, policy_version 310775 (0.0005) [2023-12-26 17:37:35,234][105692] Updated weights for policy 0, policy_version 310785 (0.0005) [2023-12-26 17:37:35,354][105620] Updated weights for policy 1, policy_version 311057 (0.0011) [2023-12-26 17:37:35,407][105620] Updated weights for policy 1, policy_version 311067 (0.0011) [2023-12-26 17:37:35,451][105620] Updated weights for policy 1, policy_version 311077 (0.0010) [2023-12-26 17:37:35,807][105692] Updated weights for policy 0, policy_version 310795 (0.0007) [2023-12-26 17:37:35,857][105692] Updated weights for policy 0, policy_version 310805 (0.0010) [2023-12-26 17:37:35,906][105692] Updated weights for policy 0, policy_version 310815 (0.0010) [2023-12-26 17:37:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 159227904. Throughput: 0: 9411.8, 1: 10153.9. Samples: 159214016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:37:36,062][104569] Avg episode reward: [(0, '8635.319'), (1, '8896.246')] [2023-12-26 17:37:36,208][105620] Updated weights for policy 1, policy_version 311087 (0.0011) [2023-12-26 17:37:36,272][105620] Updated weights for policy 1, policy_version 311097 (0.0011) [2023-12-26 17:37:36,342][105620] Updated weights for policy 1, policy_version 311107 (0.0011) [2023-12-26 17:37:36,676][105692] Updated weights for policy 0, policy_version 310825 (0.0010) [2023-12-26 17:37:36,742][105692] Updated weights for policy 0, policy_version 310835 (0.0011) [2023-12-26 17:37:36,805][105692] Updated weights for policy 0, policy_version 310845 (0.0011) [2023-12-26 17:37:36,874][105692] Updated weights for policy 0, policy_version 310855 (0.0010) [2023-12-26 17:37:37,101][105620] Updated weights for policy 1, policy_version 311117 (0.0011) [2023-12-26 17:37:37,163][105620] Updated weights for policy 1, policy_version 311127 (0.0011) [2023-12-26 17:37:37,222][105620] Updated weights for policy 1, policy_version 311137 (0.0011) [2023-12-26 17:37:37,584][105692] Updated weights for policy 0, policy_version 310865 (0.0008) [2023-12-26 17:37:37,633][105692] Updated weights for policy 0, policy_version 310875 (0.0005) [2023-12-26 17:37:37,684][105692] Updated weights for policy 0, policy_version 310885 (0.0005) [2023-12-26 17:37:37,944][105620] Updated weights for policy 1, policy_version 311147 (0.0011) [2023-12-26 17:37:37,992][105620] Updated weights for policy 1, policy_version 311157 (0.0010) [2023-12-26 17:37:38,043][105620] Updated weights for policy 1, policy_version 311167 (0.0010) [2023-12-26 17:37:38,412][105692] Updated weights for policy 0, policy_version 310895 (0.0008) [2023-12-26 17:37:38,474][105692] Updated weights for policy 0, policy_version 310905 (0.0009) [2023-12-26 17:37:38,542][105692] Updated weights for policy 0, policy_version 310915 (0.0009) [2023-12-26 17:37:38,796][105620] Updated weights for policy 1, policy_version 311177 (0.0010) [2023-12-26 17:37:38,859][105620] Updated weights for policy 1, policy_version 311187 (0.0006) [2023-12-26 17:37:38,913][105620] Updated weights for policy 1, policy_version 311197 (0.0005) [2023-12-26 17:37:38,967][105620] Updated weights for policy 1, policy_version 311207 (0.0008) [2023-12-26 17:37:39,362][105692] Updated weights for policy 0, policy_version 310925 (0.0009) [2023-12-26 17:37:39,420][105692] Updated weights for policy 0, policy_version 310935 (0.0008) [2023-12-26 17:37:39,483][105692] Updated weights for policy 0, policy_version 310945 (0.0007) [2023-12-26 17:37:39,705][105620] Updated weights for policy 1, policy_version 311217 (0.0011) [2023-12-26 17:37:39,768][105620] Updated weights for policy 1, policy_version 311227 (0.0011) [2023-12-26 17:37:39,840][105620] Updated weights for policy 1, policy_version 311237 (0.0009) [2023-12-26 17:37:40,249][105692] Updated weights for policy 0, policy_version 310955 (0.0008) [2023-12-26 17:37:40,300][105692] Updated weights for policy 0, policy_version 310965 (0.0008) [2023-12-26 17:37:40,353][105692] Updated weights for policy 0, policy_version 310975 (0.0009) [2023-12-26 17:37:40,549][105620] Updated weights for policy 1, policy_version 311247 (0.0010) [2023-12-26 17:37:40,604][105620] Updated weights for policy 1, policy_version 311257 (0.0005) [2023-12-26 17:37:40,666][105620] Updated weights for policy 1, policy_version 311267 (0.0007) [2023-12-26 17:37:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19549.8). Total num frames: 159318016. Throughput: 0: 9425.2, 1: 10122.1. Samples: 159327836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:37:41,062][104569] Avg episode reward: [(0, '8726.938'), (1, '8894.879')] [2023-12-26 17:37:41,088][105692] Updated weights for policy 0, policy_version 310985 (0.0008) [2023-12-26 17:37:41,160][105692] Updated weights for policy 0, policy_version 310995 (0.0009) [2023-12-26 17:37:41,223][105692] Updated weights for policy 0, policy_version 311005 (0.0007) [2023-12-26 17:37:41,280][105692] Updated weights for policy 0, policy_version 311015 (0.0008) [2023-12-26 17:37:41,432][105620] Updated weights for policy 1, policy_version 311277 (0.0009) [2023-12-26 17:37:41,498][105620] Updated weights for policy 1, policy_version 311287 (0.0009) [2023-12-26 17:37:41,560][105620] Updated weights for policy 1, policy_version 311297 (0.0009) [2023-12-26 17:37:42,010][105692] Updated weights for policy 0, policy_version 311025 (0.0010) [2023-12-26 17:37:42,062][105692] Updated weights for policy 0, policy_version 311035 (0.0009) [2023-12-26 17:37:42,116][105692] Updated weights for policy 0, policy_version 311045 (0.0009) [2023-12-26 17:37:42,327][105620] Updated weights for policy 1, policy_version 311307 (0.0008) [2023-12-26 17:37:42,391][105620] Updated weights for policy 1, policy_version 311317 (0.0008) [2023-12-26 17:37:42,454][105620] Updated weights for policy 1, policy_version 311327 (0.0008) [2023-12-26 17:37:42,944][105692] Updated weights for policy 0, policy_version 311055 (0.0009) [2023-12-26 17:37:42,990][105692] Updated weights for policy 0, policy_version 311065 (0.0008) [2023-12-26 17:37:43,042][105692] Updated weights for policy 0, policy_version 311075 (0.0009) [2023-12-26 17:37:43,095][105620] Updated weights for policy 1, policy_version 311337 (0.0006) [2023-12-26 17:37:43,156][105620] Updated weights for policy 1, policy_version 311347 (0.0009) [2023-12-26 17:37:43,211][105620] Updated weights for policy 1, policy_version 311357 (0.0009) [2023-12-26 17:37:43,268][105620] Updated weights for policy 1, policy_version 311367 (0.0008) [2023-12-26 17:37:43,847][105692] Updated weights for policy 0, policy_version 311085 (0.0007) [2023-12-26 17:37:43,902][105692] Updated weights for policy 0, policy_version 311095 (0.0006) [2023-12-26 17:37:43,920][105620] Updated weights for policy 1, policy_version 311377 (0.0006) [2023-12-26 17:37:43,962][105692] Updated weights for policy 0, policy_version 311105 (0.0011) [2023-12-26 17:37:43,984][105620] Updated weights for policy 1, policy_version 311387 (0.0005) [2023-12-26 17:37:44,043][105620] Updated weights for policy 1, policy_version 311397 (0.0005) [2023-12-26 17:37:44,611][105692] Updated weights for policy 0, policy_version 311115 (0.0010) [2023-12-26 17:37:44,661][105692] Updated weights for policy 0, policy_version 311125 (0.0009) [2023-12-26 17:37:44,708][105620] Updated weights for policy 1, policy_version 311407 (0.0007) [2023-12-26 17:37:44,710][105692] Updated weights for policy 0, policy_version 311135 (0.0006) [2023-12-26 17:37:44,754][105620] Updated weights for policy 1, policy_version 311417 (0.0007) [2023-12-26 17:37:44,811][105620] Updated weights for policy 1, policy_version 311427 (0.0008) [2023-12-26 17:37:45,398][105692] Updated weights for policy 0, policy_version 311145 (0.0007) [2023-12-26 17:37:45,447][105692] Updated weights for policy 0, policy_version 311155 (0.0011) [2023-12-26 17:37:45,495][105692] Updated weights for policy 0, policy_version 311165 (0.0005) [2023-12-26 17:37:45,543][105692] Updated weights for policy 0, policy_version 311175 (0.0006) [2023-12-26 17:37:45,625][105620] Updated weights for policy 1, policy_version 311437 (0.0008) [2023-12-26 17:37:45,679][105620] Updated weights for policy 1, policy_version 311447 (0.0009) [2023-12-26 17:37:45,737][105620] Updated weights for policy 1, policy_version 311457 (0.0010) [2023-12-26 17:37:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 159416320. Throughput: 0: 9370.6, 1: 10164.7. Samples: 159384912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:37:46,062][104569] Avg episode reward: [(0, '9082.430'), (1, '8709.723')] [2023-12-26 17:37:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000311464_79740928.pth... [2023-12-26 17:37:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000311176_79675392.pth... [2023-12-26 17:37:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000310280_79437824.pth [2023-12-26 17:37:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000310088_79396864.pth [2023-12-26 17:37:46,186][105692] Updated weights for policy 0, policy_version 311185 (0.0010) [2023-12-26 17:37:46,251][105692] Updated weights for policy 0, policy_version 311195 (0.0010) [2023-12-26 17:37:46,302][105692] Updated weights for policy 0, policy_version 311205 (0.0010) [2023-12-26 17:37:46,488][105620] Updated weights for policy 1, policy_version 311467 (0.0009) [2023-12-26 17:37:46,536][105620] Updated weights for policy 1, policy_version 311477 (0.0008) [2023-12-26 17:37:46,588][105620] Updated weights for policy 1, policy_version 311487 (0.0008) [2023-12-26 17:37:47,059][105692] Updated weights for policy 0, policy_version 311215 (0.0010) [2023-12-26 17:37:47,122][105692] Updated weights for policy 0, policy_version 311225 (0.0006) [2023-12-26 17:37:47,187][105692] Updated weights for policy 0, policy_version 311235 (0.0005) [2023-12-26 17:37:47,365][105620] Updated weights for policy 1, policy_version 311497 (0.0008) [2023-12-26 17:37:47,423][105620] Updated weights for policy 1, policy_version 311507 (0.0008) [2023-12-26 17:37:47,488][105620] Updated weights for policy 1, policy_version 311517 (0.0008) [2023-12-26 17:37:47,553][105620] Updated weights for policy 1, policy_version 311527 (0.0008) [2023-12-26 17:37:47,814][105692] Updated weights for policy 0, policy_version 311245 (0.0008) [2023-12-26 17:37:47,866][105692] Updated weights for policy 0, policy_version 311255 (0.0010) [2023-12-26 17:37:47,925][105692] Updated weights for policy 0, policy_version 311265 (0.0011) [2023-12-26 17:37:48,188][105620] Updated weights for policy 1, policy_version 311537 (0.0009) [2023-12-26 17:37:48,235][105620] Updated weights for policy 1, policy_version 311547 (0.0008) [2023-12-26 17:37:48,286][105620] Updated weights for policy 1, policy_version 311557 (0.0007) [2023-12-26 17:37:48,600][105692] Updated weights for policy 0, policy_version 311275 (0.0010) [2023-12-26 17:37:48,662][105692] Updated weights for policy 0, policy_version 311285 (0.0010) [2023-12-26 17:37:48,724][105692] Updated weights for policy 0, policy_version 311295 (0.0010) [2023-12-26 17:37:49,034][105620] Updated weights for policy 1, policy_version 311567 (0.0010) [2023-12-26 17:37:49,089][105620] Updated weights for policy 1, policy_version 311577 (0.0010) [2023-12-26 17:37:49,144][105620] Updated weights for policy 1, policy_version 311587 (0.0010) [2023-12-26 17:37:49,443][105692] Updated weights for policy 0, policy_version 311305 (0.0011) [2023-12-26 17:37:49,501][105692] Updated weights for policy 0, policy_version 311315 (0.0011) [2023-12-26 17:37:49,560][105692] Updated weights for policy 0, policy_version 311325 (0.0010) [2023-12-26 17:37:49,613][105692] Updated weights for policy 0, policy_version 311335 (0.0010) [2023-12-26 17:37:49,882][105620] Updated weights for policy 1, policy_version 311597 (0.0010) [2023-12-26 17:37:49,948][105620] Updated weights for policy 1, policy_version 311607 (0.0010) [2023-12-26 17:37:50,002][105620] Updated weights for policy 1, policy_version 311617 (0.0007) [2023-12-26 17:37:50,377][105692] Updated weights for policy 0, policy_version 311345 (0.0010) [2023-12-26 17:37:50,436][105692] Updated weights for policy 0, policy_version 311355 (0.0008) [2023-12-26 17:37:50,490][105692] Updated weights for policy 0, policy_version 311365 (0.0008) [2023-12-26 17:37:50,724][105620] Updated weights for policy 1, policy_version 311627 (0.0009) [2023-12-26 17:37:50,788][105620] Updated weights for policy 1, policy_version 311637 (0.0009) [2023-12-26 17:37:50,854][105620] Updated weights for policy 1, policy_version 311647 (0.0010) [2023-12-26 17:37:51,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 159514624. Throughput: 0: 9468.2, 1: 10093.9. Samples: 159502908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:37:51,063][104569] Avg episode reward: [(0, '8899.699'), (1, '8525.176')] [2023-12-26 17:37:51,235][105692] Updated weights for policy 0, policy_version 311375 (0.0007) [2023-12-26 17:37:51,293][105692] Updated weights for policy 0, policy_version 311385 (0.0008) [2023-12-26 17:37:51,360][105692] Updated weights for policy 0, policy_version 311395 (0.0006) [2023-12-26 17:37:51,567][105620] Updated weights for policy 1, policy_version 311657 (0.0011) [2023-12-26 17:37:51,639][105620] Updated weights for policy 1, policy_version 311667 (0.0011) [2023-12-26 17:37:51,703][105620] Updated weights for policy 1, policy_version 311677 (0.0011) [2023-12-26 17:37:51,770][105620] Updated weights for policy 1, policy_version 311687 (0.0009) [2023-12-26 17:37:52,104][105692] Updated weights for policy 0, policy_version 311405 (0.0008) [2023-12-26 17:37:52,167][105692] Updated weights for policy 0, policy_version 311415 (0.0008) [2023-12-26 17:37:52,230][105692] Updated weights for policy 0, policy_version 311425 (0.0008) [2023-12-26 17:37:52,535][105620] Updated weights for policy 1, policy_version 311697 (0.0009) [2023-12-26 17:37:52,584][105620] Updated weights for policy 1, policy_version 311707 (0.0010) [2023-12-26 17:37:52,636][105620] Updated weights for policy 1, policy_version 311717 (0.0010) [2023-12-26 17:37:52,951][105692] Updated weights for policy 0, policy_version 311435 (0.0008) [2023-12-26 17:37:53,010][105692] Updated weights for policy 0, policy_version 311445 (0.0009) [2023-12-26 17:37:53,076][105692] Updated weights for policy 0, policy_version 311455 (0.0006) [2023-12-26 17:37:53,440][105620] Updated weights for policy 1, policy_version 311727 (0.0009) [2023-12-26 17:37:53,496][105620] Updated weights for policy 1, policy_version 311737 (0.0008) [2023-12-26 17:37:53,550][105620] Updated weights for policy 1, policy_version 311747 (0.0008) [2023-12-26 17:37:53,734][105692] Updated weights for policy 0, policy_version 311465 (0.0006) [2023-12-26 17:37:53,794][105692] Updated weights for policy 0, policy_version 311475 (0.0009) [2023-12-26 17:37:53,844][105692] Updated weights for policy 0, policy_version 311485 (0.0008) [2023-12-26 17:37:53,901][105692] Updated weights for policy 0, policy_version 311495 (0.0009) [2023-12-26 17:37:54,264][105620] Updated weights for policy 1, policy_version 311757 (0.0010) [2023-12-26 17:37:54,322][105620] Updated weights for policy 1, policy_version 311767 (0.0009) [2023-12-26 17:37:54,376][105620] Updated weights for policy 1, policy_version 311777 (0.0010) [2023-12-26 17:37:54,512][105692] Updated weights for policy 0, policy_version 311505 (0.0007) [2023-12-26 17:37:54,564][105585] KL-divergence is very high: 102.7073 [2023-12-26 17:37:54,575][105692] Updated weights for policy 0, policy_version 311515 (0.0008) [2023-12-26 17:37:54,645][105692] Updated weights for policy 0, policy_version 311525 (0.0009) [2023-12-26 17:37:55,001][105620] Updated weights for policy 1, policy_version 311787 (0.0008) [2023-12-26 17:37:55,061][105620] Updated weights for policy 1, policy_version 311797 (0.0006) [2023-12-26 17:37:55,131][105620] Updated weights for policy 1, policy_version 311807 (0.0006) [2023-12-26 17:37:55,426][105692] Updated weights for policy 0, policy_version 311535 (0.0009) [2023-12-26 17:37:55,488][105692] Updated weights for policy 0, policy_version 311545 (0.0008) [2023-12-26 17:37:55,546][105692] Updated weights for policy 0, policy_version 311555 (0.0008) [2023-12-26 17:37:55,817][105620] Updated weights for policy 1, policy_version 311817 (0.0009) [2023-12-26 17:37:55,882][105620] Updated weights for policy 1, policy_version 311827 (0.0010) [2023-12-26 17:37:55,941][105620] Updated weights for policy 1, policy_version 311837 (0.0010) [2023-12-26 17:37:55,999][105620] Updated weights for policy 1, policy_version 311847 (0.0010) [2023-12-26 17:37:56,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 159612928. Throughput: 0: 9441.7, 1: 9987.0. Samples: 159618844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:37:56,063][104569] Avg episode reward: [(0, '8808.819'), (1, '8617.514')] [2023-12-26 17:37:56,151][105692] Updated weights for policy 0, policy_version 311566 (0.0007) [2023-12-26 17:37:56,199][105692] Updated weights for policy 0, policy_version 311576 (0.0005) [2023-12-26 17:37:56,255][105692] Updated weights for policy 0, policy_version 311586 (0.0005) [2023-12-26 17:37:56,734][105620] Updated weights for policy 1, policy_version 311857 (0.0010) [2023-12-26 17:37:56,801][105620] Updated weights for policy 1, policy_version 311867 (0.0010) [2023-12-26 17:37:56,842][105692] Updated weights for policy 0, policy_version 311596 (0.0007) [2023-12-26 17:37:56,868][105620] Updated weights for policy 1, policy_version 311877 (0.0010) [2023-12-26 17:37:56,897][105692] Updated weights for policy 0, policy_version 311606 (0.0005) [2023-12-26 17:37:56,942][105692] Updated weights for policy 0, policy_version 311616 (0.0005) [2023-12-26 17:37:57,533][105692] Updated weights for policy 0, policy_version 311626 (0.0006) [2023-12-26 17:37:57,559][105620] Updated weights for policy 1, policy_version 311887 (0.0010) [2023-12-26 17:37:57,581][105692] Updated weights for policy 0, policy_version 311636 (0.0006) [2023-12-26 17:37:57,617][105620] Updated weights for policy 1, policy_version 311897 (0.0009) [2023-12-26 17:37:57,631][105692] Updated weights for policy 0, policy_version 311646 (0.0007) [2023-12-26 17:37:57,680][105620] Updated weights for policy 1, policy_version 311907 (0.0005) [2023-12-26 17:37:57,696][105692] Updated weights for policy 0, policy_version 311656 (0.0006) [2023-12-26 17:37:58,247][105620] Updated weights for policy 1, policy_version 311917 (0.0005) [2023-12-26 17:37:58,300][105620] Updated weights for policy 1, policy_version 311927 (0.0006) [2023-12-26 17:37:58,372][105620] Updated weights for policy 1, policy_version 311937 (0.0010) [2023-12-26 17:37:58,542][105692] Updated weights for policy 0, policy_version 311666 (0.0011) [2023-12-26 17:37:58,606][105692] Updated weights for policy 0, policy_version 311676 (0.0010) [2023-12-26 17:37:58,671][105692] Updated weights for policy 0, policy_version 311686 (0.0009) [2023-12-26 17:37:59,130][105620] Updated weights for policy 1, policy_version 311947 (0.0010) [2023-12-26 17:37:59,188][105620] Updated weights for policy 1, policy_version 311957 (0.0009) [2023-12-26 17:37:59,265][105620] Updated weights for policy 1, policy_version 311967 (0.0009) [2023-12-26 17:37:59,484][105692] Updated weights for policy 0, policy_version 311696 (0.0010) [2023-12-26 17:37:59,543][105692] Updated weights for policy 0, policy_version 311706 (0.0009) [2023-12-26 17:37:59,580][105585] KL-divergence is very high: 128.3545 [2023-12-26 17:37:59,609][105692] Updated weights for policy 0, policy_version 311716 (0.0005) [2023-12-26 17:37:59,628][105585] KL-divergence is very high: 226.8657 [2023-12-26 17:37:59,854][105620] Updated weights for policy 1, policy_version 311977 (0.0010) [2023-12-26 17:37:59,918][105620] Updated weights for policy 1, policy_version 311987 (0.0006) [2023-12-26 17:37:59,983][105620] Updated weights for policy 1, policy_version 311997 (0.0007) [2023-12-26 17:38:00,052][105620] Updated weights for policy 1, policy_version 312007 (0.0006) [2023-12-26 17:38:00,235][105692] Updated weights for policy 0, policy_version 311726 (0.0006) [2023-12-26 17:38:00,297][105692] Updated weights for policy 0, policy_version 311736 (0.0005) [2023-12-26 17:38:00,363][105692] Updated weights for policy 0, policy_version 311746 (0.0008) [2023-12-26 17:38:00,675][105620] Updated weights for policy 1, policy_version 312017 (0.0007) [2023-12-26 17:38:00,740][105620] Updated weights for policy 1, policy_version 312027 (0.0005) [2023-12-26 17:38:00,801][105620] Updated weights for policy 1, policy_version 312037 (0.0005) [2023-12-26 17:38:01,053][105692] Updated weights for policy 0, policy_version 311756 (0.0011) [2023-12-26 17:38:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 159711232. Throughput: 0: 9528.6, 1: 9995.9. Samples: 159680756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:38:01,063][104569] Avg episode reward: [(0, '8720.659'), (1, '8709.527')] [2023-12-26 17:38:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000312040_79888384.pth... [2023-12-26 17:38:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000310888_79593472.pth [2023-12-26 17:38:01,115][105692] Updated weights for policy 0, policy_version 311766 (0.0011) [2023-12-26 17:38:01,180][105692] Updated weights for policy 0, policy_version 311776 (0.0010) [2023-12-26 17:38:01,219][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000311784_79831040.pth... [2023-12-26 17:38:01,222][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000310632_79536128.pth [2023-12-26 17:38:01,395][105620] Updated weights for policy 1, policy_version 312047 (0.0008) [2023-12-26 17:38:01,463][105620] Updated weights for policy 1, policy_version 312057 (0.0007) [2023-12-26 17:38:01,530][105620] Updated weights for policy 1, policy_version 312067 (0.0008) [2023-12-26 17:38:01,950][105692] Updated weights for policy 0, policy_version 311786 (0.0010) [2023-12-26 17:38:02,004][105692] Updated weights for policy 0, policy_version 311796 (0.0009) [2023-12-26 17:38:02,058][105692] Updated weights for policy 0, policy_version 311806 (0.0007) [2023-12-26 17:38:02,111][105692] Updated weights for policy 0, policy_version 311816 (0.0006) [2023-12-26 17:38:02,165][105620] Updated weights for policy 1, policy_version 312077 (0.0006) [2023-12-26 17:38:02,228][105620] Updated weights for policy 1, policy_version 312087 (0.0008) [2023-12-26 17:38:02,299][105620] Updated weights for policy 1, policy_version 312097 (0.0008) [2023-12-26 17:38:02,760][105692] Updated weights for policy 0, policy_version 311826 (0.0007) [2023-12-26 17:38:02,811][105692] Updated weights for policy 0, policy_version 311836 (0.0010) [2023-12-26 17:38:02,856][105692] Updated weights for policy 0, policy_version 311846 (0.0010) [2023-12-26 17:38:02,907][105620] Updated weights for policy 1, policy_version 312107 (0.0008) [2023-12-26 17:38:02,971][105620] Updated weights for policy 1, policy_version 312117 (0.0005) [2023-12-26 17:38:03,026][105620] Updated weights for policy 1, policy_version 312127 (0.0005) [2023-12-26 17:38:03,552][105692] Updated weights for policy 0, policy_version 311856 (0.0010) [2023-12-26 17:38:03,603][105692] Updated weights for policy 0, policy_version 311866 (0.0010) [2023-12-26 17:38:03,660][105692] Updated weights for policy 0, policy_version 311876 (0.0010) [2023-12-26 17:38:03,712][105620] Updated weights for policy 1, policy_version 312137 (0.0010) [2023-12-26 17:38:03,759][105620] Updated weights for policy 1, policy_version 312147 (0.0008) [2023-12-26 17:38:03,802][105620] Updated weights for policy 1, policy_version 312157 (0.0008) [2023-12-26 17:38:03,861][105620] Updated weights for policy 1, policy_version 312167 (0.0008) [2023-12-26 17:38:04,436][105692] Updated weights for policy 0, policy_version 311886 (0.0008) [2023-12-26 17:38:04,499][105692] Updated weights for policy 0, policy_version 311896 (0.0006) [2023-12-26 17:38:04,518][105620] Updated weights for policy 1, policy_version 312177 (0.0005) [2023-12-26 17:38:04,564][105692] Updated weights for policy 0, policy_version 311906 (0.0006) [2023-12-26 17:38:04,576][105620] Updated weights for policy 1, policy_version 312187 (0.0005) [2023-12-26 17:38:04,642][105620] Updated weights for policy 1, policy_version 312197 (0.0007) [2023-12-26 17:38:05,162][105692] Updated weights for policy 0, policy_version 311916 (0.0008) [2023-12-26 17:38:05,218][105692] Updated weights for policy 0, policy_version 311926 (0.0011) [2023-12-26 17:38:05,248][105620] Updated weights for policy 1, policy_version 312207 (0.0006) [2023-12-26 17:38:05,271][105692] Updated weights for policy 0, policy_version 311936 (0.0011) [2023-12-26 17:38:05,309][105620] Updated weights for policy 1, policy_version 312217 (0.0006) [2023-12-26 17:38:05,381][105620] Updated weights for policy 1, policy_version 312227 (0.0010) [2023-12-26 17:38:06,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 159809536. Throughput: 0: 9508.8, 1: 9987.6. Samples: 159803204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:38:06,062][104569] Avg episode reward: [(0, '8540.729'), (1, '8803.437')] [2023-12-26 17:38:06,079][105620] Updated weights for policy 1, policy_version 312237 (0.0007) [2023-12-26 17:38:06,094][105692] Updated weights for policy 0, policy_version 311946 (0.0008) [2023-12-26 17:38:06,141][105620] Updated weights for policy 1, policy_version 312247 (0.0008) [2023-12-26 17:38:06,159][105692] Updated weights for policy 0, policy_version 311956 (0.0007) [2023-12-26 17:38:06,209][105620] Updated weights for policy 1, policy_version 312257 (0.0009) [2023-12-26 17:38:06,226][105692] Updated weights for policy 0, policy_version 311966 (0.0005) [2023-12-26 17:38:06,233][105585] KL-divergence is very high: 111.7853 [2023-12-26 17:38:06,288][105692] Updated weights for policy 0, policy_version 311976 (0.0006) [2023-12-26 17:38:06,855][105692] Updated weights for policy 0, policy_version 311986 (0.0011) [2023-12-26 17:38:06,910][105692] Updated weights for policy 0, policy_version 311996 (0.0010) [2023-12-26 17:38:06,966][105692] Updated weights for policy 0, policy_version 312006 (0.0011) [2023-12-26 17:38:06,999][105620] Updated weights for policy 1, policy_version 312267 (0.0007) [2023-12-26 17:38:07,057][105620] Updated weights for policy 1, policy_version 312277 (0.0006) [2023-12-26 17:38:07,119][105620] Updated weights for policy 1, policy_version 312287 (0.0008) [2023-12-26 17:38:07,721][105692] Updated weights for policy 0, policy_version 312016 (0.0011) [2023-12-26 17:38:07,742][105585] KL-divergence is very high: 189.9544 [2023-12-26 17:38:07,774][105692] Updated weights for policy 0, policy_version 312026 (0.0011) [2023-12-26 17:38:07,783][105585] KL-divergence is very high: 326.7631 [2023-12-26 17:38:07,821][105585] KL-divergence is very high: 343.9531 [2023-12-26 17:38:07,823][105692] Updated weights for policy 0, policy_version 312036 (0.0010) [2023-12-26 17:38:07,869][105620] Updated weights for policy 1, policy_version 312297 (0.0008) [2023-12-26 17:38:07,918][105620] Updated weights for policy 1, policy_version 312307 (0.0008) [2023-12-26 17:38:07,976][105620] Updated weights for policy 1, policy_version 312317 (0.0008) [2023-12-26 17:38:08,038][105620] Updated weights for policy 1, policy_version 312327 (0.0008) [2023-12-26 17:38:08,567][105692] Updated weights for policy 0, policy_version 312046 (0.0008) [2023-12-26 17:38:08,620][105692] Updated weights for policy 0, policy_version 312056 (0.0006) [2023-12-26 17:38:08,679][105692] Updated weights for policy 0, policy_version 312066 (0.0005) [2023-12-26 17:38:08,764][105620] Updated weights for policy 1, policy_version 312337 (0.0009) [2023-12-26 17:38:08,817][105620] Updated weights for policy 1, policy_version 312347 (0.0010) [2023-12-26 17:38:08,878][105620] Updated weights for policy 1, policy_version 312357 (0.0009) [2023-12-26 17:38:09,187][105692] Updated weights for policy 0, policy_version 312076 (0.0005) [2023-12-26 17:38:09,249][105692] Updated weights for policy 0, policy_version 312086 (0.0007) [2023-12-26 17:38:09,314][105692] Updated weights for policy 0, policy_version 312096 (0.0011) [2023-12-26 17:38:09,802][105620] Updated weights for policy 1, policy_version 312367 (0.0009) [2023-12-26 17:38:09,868][105620] Updated weights for policy 1, policy_version 312377 (0.0009) [2023-12-26 17:38:09,929][105692] Updated weights for policy 0, policy_version 312106 (0.0011) [2023-12-26 17:38:09,944][105620] Updated weights for policy 1, policy_version 312388 (0.0009) [2023-12-26 17:38:09,994][105692] Updated weights for policy 0, policy_version 312116 (0.0008) [2023-12-26 17:38:10,056][105692] Updated weights for policy 0, policy_version 312126 (0.0009) [2023-12-26 17:38:10,108][105692] Updated weights for policy 0, policy_version 312136 (0.0009) [2023-12-26 17:38:10,625][105620] Updated weights for policy 1, policy_version 312398 (0.0006) [2023-12-26 17:38:10,675][105620] Updated weights for policy 1, policy_version 312408 (0.0008) [2023-12-26 17:38:10,727][105620] Updated weights for policy 1, policy_version 312418 (0.0009) [2023-12-26 17:38:10,907][105692] Updated weights for policy 0, policy_version 312146 (0.0009) [2023-12-26 17:38:10,961][105692] Updated weights for policy 0, policy_version 312156 (0.0009) [2023-12-26 17:38:11,013][105692] Updated weights for policy 0, policy_version 312166 (0.0009) [2023-12-26 17:38:11,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 159916032. Throughput: 0: 9726.7, 1: 9803.8. Samples: 159919264. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:38:11,063][104569] Avg episode reward: [(0, '8175.340'), (1, '9082.103')] [2023-12-26 17:38:11,489][105620] Updated weights for policy 1, policy_version 312428 (0.0009) [2023-12-26 17:38:11,551][105620] Updated weights for policy 1, policy_version 312438 (0.0007) [2023-12-26 17:38:11,618][105620] Updated weights for policy 1, policy_version 312448 (0.0007) [2023-12-26 17:38:11,827][105692] Updated weights for policy 0, policy_version 312176 (0.0006) [2023-12-26 17:38:11,887][105692] Updated weights for policy 0, policy_version 312186 (0.0005) [2023-12-26 17:38:11,950][105692] Updated weights for policy 0, policy_version 312196 (0.0006) [2023-12-26 17:38:12,457][105620] Updated weights for policy 1, policy_version 312458 (0.0008) [2023-12-26 17:38:12,516][105620] Updated weights for policy 1, policy_version 312468 (0.0009) [2023-12-26 17:38:12,566][105692] Updated weights for policy 0, policy_version 312206 (0.0006) [2023-12-26 17:38:12,573][105620] Updated weights for policy 1, policy_version 312478 (0.0009) [2023-12-26 17:38:12,621][105692] Updated weights for policy 0, policy_version 312216 (0.0006) [2023-12-26 17:38:12,634][105620] Updated weights for policy 1, policy_version 312488 (0.0008) [2023-12-26 17:38:12,675][105692] Updated weights for policy 0, policy_version 312226 (0.0008) [2023-12-26 17:38:13,306][105692] Updated weights for policy 0, policy_version 312236 (0.0009) [2023-12-26 17:38:13,371][105692] Updated weights for policy 0, policy_version 312246 (0.0009) [2023-12-26 17:38:13,432][105692] Updated weights for policy 0, policy_version 312256 (0.0010) [2023-12-26 17:38:13,446][105620] Updated weights for policy 1, policy_version 312498 (0.0008) [2023-12-26 17:38:13,499][105620] Updated weights for policy 1, policy_version 312508 (0.0007) [2023-12-26 17:38:13,546][105620] Updated weights for policy 1, policy_version 312519 (0.0008) [2023-12-26 17:38:13,999][105692] Updated weights for policy 0, policy_version 312266 (0.0010) [2023-12-26 17:38:14,050][105692] Updated weights for policy 0, policy_version 312276 (0.0005) [2023-12-26 17:38:14,113][105692] Updated weights for policy 0, policy_version 312286 (0.0005) [2023-12-26 17:38:14,138][105620] Updated weights for policy 1, policy_version 312529 (0.0008) [2023-12-26 17:38:14,173][105692] Updated weights for policy 0, policy_version 312296 (0.0005) [2023-12-26 17:38:14,187][105620] Updated weights for policy 1, policy_version 312539 (0.0009) [2023-12-26 17:38:14,239][105620] Updated weights for policy 1, policy_version 312549 (0.0010) [2023-12-26 17:38:14,678][105692] Updated weights for policy 0, policy_version 312306 (0.0007) [2023-12-26 17:38:14,739][105692] Updated weights for policy 0, policy_version 312316 (0.0010) [2023-12-26 17:38:14,765][105585] KL-divergence is very high: 111.0293 [2023-12-26 17:38:14,793][105692] Updated weights for policy 0, policy_version 312326 (0.0009) [2023-12-26 17:38:15,113][105620] Updated weights for policy 1, policy_version 312559 (0.0010) [2023-12-26 17:38:15,177][105620] Updated weights for policy 1, policy_version 312569 (0.0008) [2023-12-26 17:38:15,236][105620] Updated weights for policy 1, policy_version 312579 (0.0009) [2023-12-26 17:38:15,535][105692] Updated weights for policy 0, policy_version 312336 (0.0009) [2023-12-26 17:38:15,600][105692] Updated weights for policy 0, policy_version 312346 (0.0007) [2023-12-26 17:38:15,665][105692] Updated weights for policy 0, policy_version 312356 (0.0005) [2023-12-26 17:38:15,981][105620] Updated weights for policy 1, policy_version 312589 (0.0009) [2023-12-26 17:38:16,001][105586] KL-divergence is very high: 111.3060 [2023-12-26 17:38:16,007][105586] KL-divergence is very high: 126.8544 [2023-12-26 17:38:16,040][105620] Updated weights for policy 1, policy_version 312599 (0.0009) [2023-12-26 17:38:16,052][105586] KL-divergence is very high: 227.4178 [2023-12-26 17:38:16,057][105586] KL-divergence is very high: 229.7126 [2023-12-26 17:38:16,062][105586] KL-divergence is very high: 136.7033 [2023-12-26 17:38:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 160006144. Throughput: 0: 9786.2, 1: 9745.7. Samples: 159977168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:38:16,063][104569] Avg episode reward: [(0, '8259.615'), (1, '8896.126')] [2023-12-26 17:38:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000312360_79978496.pth... [2023-12-26 17:38:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000311176_79675392.pth [2023-12-26 17:38:16,099][105620] Updated weights for policy 1, policy_version 312609 (0.0009) [2023-12-26 17:38:16,100][105586] KL-divergence is very high: 238.9761 [2023-12-26 17:38:16,106][105586] KL-divergence is very high: 236.7020 [2023-12-26 17:38:16,112][105586] KL-divergence is very high: 127.5261 [2023-12-26 17:38:16,135][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000312616_80035840.pth... [2023-12-26 17:38:16,138][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000311464_79740928.pth [2023-12-26 17:38:16,294][105692] Updated weights for policy 0, policy_version 312366 (0.0008) [2023-12-26 17:38:16,341][105692] Updated weights for policy 0, policy_version 312376 (0.0009) [2023-12-26 17:38:16,389][105692] Updated weights for policy 0, policy_version 312386 (0.0009) [2023-12-26 17:38:16,821][105620] Updated weights for policy 1, policy_version 312619 (0.0008) [2023-12-26 17:38:16,884][105620] Updated weights for policy 1, policy_version 312629 (0.0009) [2023-12-26 17:38:16,948][105620] Updated weights for policy 1, policy_version 312639 (0.0008) [2023-12-26 17:38:17,128][105692] Updated weights for policy 0, policy_version 312396 (0.0010) [2023-12-26 17:38:17,193][105692] Updated weights for policy 0, policy_version 312406 (0.0011) [2023-12-26 17:38:17,255][105692] Updated weights for policy 0, policy_version 312416 (0.0010) [2023-12-26 17:38:17,685][105620] Updated weights for policy 1, policy_version 312649 (0.0009) [2023-12-26 17:38:17,738][105620] Updated weights for policy 1, policy_version 312659 (0.0008) [2023-12-26 17:38:17,783][105620] Updated weights for policy 1, policy_version 312669 (0.0009) [2023-12-26 17:38:17,851][105620] Updated weights for policy 1, policy_version 312679 (0.0009) [2023-12-26 17:38:17,963][105692] Updated weights for policy 0, policy_version 312426 (0.0008) [2023-12-26 17:38:18,017][105692] Updated weights for policy 0, policy_version 312436 (0.0005) [2023-12-26 17:38:18,065][105692] Updated weights for policy 0, policy_version 312446 (0.0008) [2023-12-26 17:38:18,112][105692] Updated weights for policy 0, policy_version 312456 (0.0010) [2023-12-26 17:38:18,645][105620] Updated weights for policy 1, policy_version 312689 (0.0008) [2023-12-26 17:38:18,697][105620] Updated weights for policy 1, policy_version 312699 (0.0008) [2023-12-26 17:38:18,749][105620] Updated weights for policy 1, policy_version 312709 (0.0007) [2023-12-26 17:38:18,869][105692] Updated weights for policy 0, policy_version 312466 (0.0011) [2023-12-26 17:38:18,932][105692] Updated weights for policy 0, policy_version 312476 (0.0011) [2023-12-26 17:38:18,991][105692] Updated weights for policy 0, policy_version 312486 (0.0011) [2023-12-26 17:38:19,564][105620] Updated weights for policy 1, policy_version 312719 (0.0010) [2023-12-26 17:38:19,623][105620] Updated weights for policy 1, policy_version 312729 (0.0010) [2023-12-26 17:38:19,686][105620] Updated weights for policy 1, policy_version 312739 (0.0011) [2023-12-26 17:38:19,758][105692] Updated weights for policy 0, policy_version 312496 (0.0011) [2023-12-26 17:38:19,806][105692] Updated weights for policy 0, policy_version 312506 (0.0011) [2023-12-26 17:38:19,879][105692] Updated weights for policy 0, policy_version 312516 (0.0011) [2023-12-26 17:38:20,379][105620] Updated weights for policy 1, policy_version 312749 (0.0009) [2023-12-26 17:38:20,451][105620] Updated weights for policy 1, policy_version 312759 (0.0008) [2023-12-26 17:38:20,519][105620] Updated weights for policy 1, policy_version 312769 (0.0009) [2023-12-26 17:38:20,628][105692] Updated weights for policy 0, policy_version 312526 (0.0011) [2023-12-26 17:38:20,695][105692] Updated weights for policy 0, policy_version 312536 (0.0010) [2023-12-26 17:38:20,759][105692] Updated weights for policy 0, policy_version 312546 (0.0010) [2023-12-26 17:38:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 160104448. Throughput: 0: 9918.5, 1: 9629.2. Samples: 160093660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:38:21,063][104569] Avg episode reward: [(0, '8991.446'), (1, '9079.404')] [2023-12-26 17:38:21,243][105620] Updated weights for policy 1, policy_version 312780 (0.0010) [2023-12-26 17:38:21,307][105620] Updated weights for policy 1, policy_version 312790 (0.0009) [2023-12-26 17:38:21,377][105620] Updated weights for policy 1, policy_version 312800 (0.0008) [2023-12-26 17:38:21,440][105692] Updated weights for policy 0, policy_version 312556 (0.0009) [2023-12-26 17:38:21,507][105692] Updated weights for policy 0, policy_version 312566 (0.0010) [2023-12-26 17:38:21,579][105692] Updated weights for policy 0, policy_version 312576 (0.0009) [2023-12-26 17:38:22,137][105620] Updated weights for policy 1, policy_version 312810 (0.0009) [2023-12-26 17:38:22,189][105620] Updated weights for policy 1, policy_version 312820 (0.0008) [2023-12-26 17:38:22,244][105620] Updated weights for policy 1, policy_version 312830 (0.0007) [2023-12-26 17:38:22,303][105620] Updated weights for policy 1, policy_version 312840 (0.0008) [2023-12-26 17:38:22,333][105692] Updated weights for policy 0, policy_version 312586 (0.0009) [2023-12-26 17:38:22,394][105692] Updated weights for policy 0, policy_version 312596 (0.0011) [2023-12-26 17:38:22,413][105585] KL-divergence is very high: 105.0289 [2023-12-26 17:38:22,457][105692] Updated weights for policy 0, policy_version 312606 (0.0011) [2023-12-26 17:38:22,466][105585] KL-divergence is very high: 170.2177 [2023-12-26 17:38:22,513][105585] KL-divergence is very high: 149.6516 [2023-12-26 17:38:22,520][105692] Updated weights for policy 0, policy_version 312616 (0.0011) [2023-12-26 17:38:23,084][105620] Updated weights for policy 1, policy_version 312850 (0.0008) [2023-12-26 17:38:23,150][105620] Updated weights for policy 1, policy_version 312860 (0.0008) [2023-12-26 17:38:23,207][105620] Updated weights for policy 1, policy_version 312870 (0.0008) [2023-12-26 17:38:23,321][105692] Updated weights for policy 0, policy_version 312626 (0.0011) [2023-12-26 17:38:23,376][105692] Updated weights for policy 0, policy_version 312636 (0.0010) [2023-12-26 17:38:23,434][105692] Updated weights for policy 0, policy_version 312646 (0.0010) [2023-12-26 17:38:23,964][105620] Updated weights for policy 1, policy_version 312880 (0.0008) [2023-12-26 17:38:24,011][105620] Updated weights for policy 1, policy_version 312890 (0.0007) [2023-12-26 17:38:24,059][105620] Updated weights for policy 1, policy_version 312900 (0.0008) [2023-12-26 17:38:24,163][105692] Updated weights for policy 0, policy_version 312656 (0.0010) [2023-12-26 17:38:24,227][105692] Updated weights for policy 0, policy_version 312666 (0.0011) [2023-12-26 17:38:24,286][105692] Updated weights for policy 0, policy_version 312676 (0.0011) [2023-12-26 17:38:24,852][105620] Updated weights for policy 1, policy_version 312910 (0.0008) [2023-12-26 17:38:24,903][105620] Updated weights for policy 1, policy_version 312920 (0.0008) [2023-12-26 17:38:24,967][105620] Updated weights for policy 1, policy_version 312930 (0.0008) [2023-12-26 17:38:25,027][105692] Updated weights for policy 0, policy_version 312686 (0.0011) [2023-12-26 17:38:25,084][105692] Updated weights for policy 0, policy_version 312696 (0.0010) [2023-12-26 17:38:25,142][105692] Updated weights for policy 0, policy_version 312706 (0.0010) [2023-12-26 17:38:25,704][105620] Updated weights for policy 1, policy_version 312940 (0.0008) [2023-12-26 17:38:25,748][105620] Updated weights for policy 1, policy_version 312950 (0.0008) [2023-12-26 17:38:25,802][105620] Updated weights for policy 1, policy_version 312960 (0.0008) [2023-12-26 17:38:25,875][105692] Updated weights for policy 0, policy_version 312716 (0.0010) [2023-12-26 17:38:25,927][105692] Updated weights for policy 0, policy_version 312726 (0.0010) [2023-12-26 17:38:25,981][105692] Updated weights for policy 0, policy_version 312736 (0.0010) [2023-12-26 17:38:26,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 160202752. Throughput: 0: 9899.5, 1: 9605.5. Samples: 160205564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:38:26,063][104569] Avg episode reward: [(0, '8535.827'), (1, '9356.681')] [2023-12-26 17:38:26,558][105620] Updated weights for policy 1, policy_version 312970 (0.0008) [2023-12-26 17:38:26,612][105620] Updated weights for policy 1, policy_version 312980 (0.0008) [2023-12-26 17:38:26,659][105620] Updated weights for policy 1, policy_version 312990 (0.0007) [2023-12-26 17:38:26,707][105620] Updated weights for policy 1, policy_version 313000 (0.0008) [2023-12-26 17:38:26,742][105692] Updated weights for policy 0, policy_version 312746 (0.0010) [2023-12-26 17:38:26,800][105692] Updated weights for policy 0, policy_version 312756 (0.0010) [2023-12-26 17:38:26,855][105692] Updated weights for policy 0, policy_version 312766 (0.0010) [2023-12-26 17:38:26,903][105692] Updated weights for policy 0, policy_version 312776 (0.0010) [2023-12-26 17:38:27,480][105620] Updated weights for policy 1, policy_version 313010 (0.0008) [2023-12-26 17:38:27,530][105620] Updated weights for policy 1, policy_version 313020 (0.0007) [2023-12-26 17:38:27,582][105620] Updated weights for policy 1, policy_version 313030 (0.0008) [2023-12-26 17:38:27,654][105692] Updated weights for policy 0, policy_version 312786 (0.0010) [2023-12-26 17:38:27,705][105692] Updated weights for policy 0, policy_version 312796 (0.0010) [2023-12-26 17:38:27,756][105692] Updated weights for policy 0, policy_version 312806 (0.0010) [2023-12-26 17:38:28,272][105620] Updated weights for policy 1, policy_version 313040 (0.0005) [2023-12-26 17:38:28,350][105620] Updated weights for policy 1, policy_version 313050 (0.0008) [2023-12-26 17:38:28,362][105692] Updated weights for policy 0, policy_version 312816 (0.0008) [2023-12-26 17:38:28,412][105620] Updated weights for policy 1, policy_version 313060 (0.0010) [2023-12-26 17:38:28,421][105692] Updated weights for policy 0, policy_version 312826 (0.0010) [2023-12-26 17:38:28,476][105692] Updated weights for policy 0, policy_version 312836 (0.0011) [2023-12-26 17:38:29,027][105620] Updated weights for policy 1, policy_version 313070 (0.0008) [2023-12-26 17:38:29,071][105620] Updated weights for policy 1, policy_version 313080 (0.0007) [2023-12-26 17:38:29,119][105620] Updated weights for policy 1, policy_version 313090 (0.0007) [2023-12-26 17:38:29,140][105692] Updated weights for policy 0, policy_version 312846 (0.0010) [2023-12-26 17:38:29,197][105692] Updated weights for policy 0, policy_version 312856 (0.0010) [2023-12-26 17:38:29,259][105692] Updated weights for policy 0, policy_version 312866 (0.0010) [2023-12-26 17:38:29,905][105620] Updated weights for policy 1, policy_version 313100 (0.0008) [2023-12-26 17:38:29,971][105620] Updated weights for policy 1, policy_version 313110 (0.0009) [2023-12-26 17:38:30,009][105692] Updated weights for policy 0, policy_version 312876 (0.0009) [2023-12-26 17:38:30,032][105620] Updated weights for policy 1, policy_version 313120 (0.0008) [2023-12-26 17:38:30,061][105692] Updated weights for policy 0, policy_version 312886 (0.0008) [2023-12-26 17:38:30,112][105692] Updated weights for policy 0, policy_version 312896 (0.0008) [2023-12-26 17:38:30,711][105620] Updated weights for policy 1, policy_version 313130 (0.0007) [2023-12-26 17:38:30,766][105620] Updated weights for policy 1, policy_version 313140 (0.0010) [2023-12-26 17:38:30,791][105692] Updated weights for policy 0, policy_version 312906 (0.0008) [2023-12-26 17:38:30,827][105620] Updated weights for policy 1, policy_version 313150 (0.0010) [2023-12-26 17:38:30,844][105692] Updated weights for policy 0, policy_version 312916 (0.0005) [2023-12-26 17:38:30,878][105620] Updated weights for policy 1, policy_version 313160 (0.0010) [2023-12-26 17:38:30,907][105692] Updated weights for policy 0, policy_version 312926 (0.0005) [2023-12-26 17:38:30,960][105692] Updated weights for policy 0, policy_version 312936 (0.0005) [2023-12-26 17:38:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 160301056. Throughput: 0: 9953.3, 1: 9607.1. Samples: 160265132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:38:31,063][104569] Avg episode reward: [(0, '8353.060'), (1, '9264.438')] [2023-12-26 17:38:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000312936_80125952.pth... [2023-12-26 17:38:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000313160_80175104.pth... [2023-12-26 17:38:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000312040_79888384.pth [2023-12-26 17:38:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000311784_79831040.pth [2023-12-26 17:38:31,548][105692] Updated weights for policy 0, policy_version 312946 (0.0007) [2023-12-26 17:38:31,581][105620] Updated weights for policy 1, policy_version 313170 (0.0008) [2023-12-26 17:38:31,610][105692] Updated weights for policy 0, policy_version 312956 (0.0010) [2023-12-26 17:38:31,644][105620] Updated weights for policy 1, policy_version 313180 (0.0008) [2023-12-26 17:38:31,664][105692] Updated weights for policy 0, policy_version 312966 (0.0007) [2023-12-26 17:38:31,707][105620] Updated weights for policy 1, policy_version 313190 (0.0008) [2023-12-26 17:38:32,466][105692] Updated weights for policy 0, policy_version 312976 (0.0007) [2023-12-26 17:38:32,476][105620] Updated weights for policy 1, policy_version 313200 (0.0008) [2023-12-26 17:38:32,524][105692] Updated weights for policy 0, policy_version 312986 (0.0005) [2023-12-26 17:38:32,526][105620] Updated weights for policy 1, policy_version 313210 (0.0007) [2023-12-26 17:38:32,582][105692] Updated weights for policy 0, policy_version 312996 (0.0007) [2023-12-26 17:38:32,587][105620] Updated weights for policy 1, policy_version 313220 (0.0007) [2023-12-26 17:38:33,342][105692] Updated weights for policy 0, policy_version 313006 (0.0006) [2023-12-26 17:38:33,353][105620] Updated weights for policy 1, policy_version 313230 (0.0009) [2023-12-26 17:38:33,407][105692] Updated weights for policy 0, policy_version 313016 (0.0008) [2023-12-26 17:38:33,407][105620] Updated weights for policy 1, policy_version 313240 (0.0010) [2023-12-26 17:38:33,463][105692] Updated weights for policy 0, policy_version 313026 (0.0010) [2023-12-26 17:38:33,465][105620] Updated weights for policy 1, policy_version 313250 (0.0006) [2023-12-26 17:38:34,023][105620] Updated weights for policy 1, policy_version 313260 (0.0005) [2023-12-26 17:38:34,073][105620] Updated weights for policy 1, policy_version 313270 (0.0005) [2023-12-26 17:38:34,133][105620] Updated weights for policy 1, policy_version 313280 (0.0005) [2023-12-26 17:38:34,175][105692] Updated weights for policy 0, policy_version 313036 (0.0010) [2023-12-26 17:38:34,224][105692] Updated weights for policy 0, policy_version 313046 (0.0010) [2023-12-26 17:38:34,279][105692] Updated weights for policy 0, policy_version 313056 (0.0010) [2023-12-26 17:38:34,789][105620] Updated weights for policy 1, policy_version 313290 (0.0008) [2023-12-26 17:38:34,836][105620] Updated weights for policy 1, policy_version 313300 (0.0008) [2023-12-26 17:38:34,881][105620] Updated weights for policy 1, policy_version 313310 (0.0008) [2023-12-26 17:38:34,933][105620] Updated weights for policy 1, policy_version 313320 (0.0008) [2023-12-26 17:38:35,060][105692] Updated weights for policy 0, policy_version 313066 (0.0011) [2023-12-26 17:38:35,119][105692] Updated weights for policy 0, policy_version 313076 (0.0010) [2023-12-26 17:38:35,170][105692] Updated weights for policy 0, policy_version 313086 (0.0010) [2023-12-26 17:38:35,230][105692] Updated weights for policy 0, policy_version 313096 (0.0011) [2023-12-26 17:38:35,695][105620] Updated weights for policy 1, policy_version 313330 (0.0007) [2023-12-26 17:38:35,759][105620] Updated weights for policy 1, policy_version 313340 (0.0008) [2023-12-26 17:38:35,827][105620] Updated weights for policy 1, policy_version 313350 (0.0008) [2023-12-26 17:38:35,996][105692] Updated weights for policy 0, policy_version 313106 (0.0010) [2023-12-26 17:38:36,061][105692] Updated weights for policy 0, policy_version 313116 (0.0006) [2023-12-26 17:38:36,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 160391168. Throughput: 0: 9907.7, 1: 9661.0. Samples: 160383496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:38:36,062][104569] Avg episode reward: [(0, '8722.433'), (1, '9171.960')] [2023-12-26 17:38:36,130][105692] Updated weights for policy 0, policy_version 313126 (0.0007) [2023-12-26 17:38:36,531][105620] Updated weights for policy 1, policy_version 313360 (0.0008) [2023-12-26 17:38:36,587][105620] Updated weights for policy 1, policy_version 313370 (0.0008) [2023-12-26 17:38:36,646][105620] Updated weights for policy 1, policy_version 313380 (0.0008) [2023-12-26 17:38:36,844][105692] Updated weights for policy 0, policy_version 313136 (0.0010) [2023-12-26 17:38:36,906][105692] Updated weights for policy 0, policy_version 313146 (0.0011) [2023-12-26 17:38:36,972][105692] Updated weights for policy 0, policy_version 313156 (0.0010) [2023-12-26 17:38:37,400][105620] Updated weights for policy 1, policy_version 313390 (0.0006) [2023-12-26 17:38:37,446][105620] Updated weights for policy 1, policy_version 313400 (0.0005) [2023-12-26 17:38:37,502][105620] Updated weights for policy 1, policy_version 313410 (0.0005) [2023-12-26 17:38:37,686][105692] Updated weights for policy 0, policy_version 313166 (0.0008) [2023-12-26 17:38:37,748][105692] Updated weights for policy 0, policy_version 313176 (0.0007) [2023-12-26 17:38:37,808][105692] Updated weights for policy 0, policy_version 313186 (0.0008) [2023-12-26 17:38:38,199][105620] Updated weights for policy 1, policy_version 313420 (0.0008) [2023-12-26 17:38:38,243][105620] Updated weights for policy 1, policy_version 313430 (0.0010) [2023-12-26 17:38:38,289][105620] Updated weights for policy 1, policy_version 313440 (0.0010) [2023-12-26 17:38:38,502][105692] Updated weights for policy 0, policy_version 313196 (0.0007) [2023-12-26 17:38:38,551][105692] Updated weights for policy 0, policy_version 313206 (0.0006) [2023-12-26 17:38:38,601][105692] Updated weights for policy 0, policy_version 313216 (0.0006) [2023-12-26 17:38:38,998][105620] Updated weights for policy 1, policy_version 313450 (0.0010) [2023-12-26 17:38:39,058][105620] Updated weights for policy 1, policy_version 313460 (0.0010) [2023-12-26 17:38:39,111][105620] Updated weights for policy 1, policy_version 313470 (0.0009) [2023-12-26 17:38:39,177][105620] Updated weights for policy 1, policy_version 313480 (0.0010) [2023-12-26 17:38:39,211][105692] Updated weights for policy 0, policy_version 313226 (0.0007) [2023-12-26 17:38:39,276][105692] Updated weights for policy 0, policy_version 313236 (0.0009) [2023-12-26 17:38:39,335][105692] Updated weights for policy 0, policy_version 313246 (0.0009) [2023-12-26 17:38:39,401][105692] Updated weights for policy 0, policy_version 313256 (0.0008) [2023-12-26 17:38:39,973][105620] Updated weights for policy 1, policy_version 313490 (0.0008) [2023-12-26 17:38:40,031][105620] Updated weights for policy 1, policy_version 313500 (0.0008) [2023-12-26 17:38:40,092][105620] Updated weights for policy 1, policy_version 313510 (0.0008) [2023-12-26 17:38:40,197][105692] Updated weights for policy 0, policy_version 313266 (0.0010) [2023-12-26 17:38:40,249][105692] Updated weights for policy 0, policy_version 313276 (0.0009) [2023-12-26 17:38:40,311][105692] Updated weights for policy 0, policy_version 313286 (0.0009) [2023-12-26 17:38:40,740][105620] Updated weights for policy 1, policy_version 313520 (0.0010) [2023-12-26 17:38:40,799][105620] Updated weights for policy 1, policy_version 313530 (0.0011) [2023-12-26 17:38:40,862][105620] Updated weights for policy 1, policy_version 313540 (0.0011) [2023-12-26 17:38:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 160489472. Throughput: 0: 9890.4, 1: 9668.3. Samples: 160498984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:38:41,063][104569] Avg episode reward: [(0, '8993.522'), (1, '9263.867')] [2023-12-26 17:38:41,083][105692] Updated weights for policy 0, policy_version 313296 (0.0009) [2023-12-26 17:38:41,145][105692] Updated weights for policy 0, policy_version 313306 (0.0009) [2023-12-26 17:38:41,208][105692] Updated weights for policy 0, policy_version 313316 (0.0008) [2023-12-26 17:38:41,626][105620] Updated weights for policy 1, policy_version 313550 (0.0010) [2023-12-26 17:38:41,686][105620] Updated weights for policy 1, policy_version 313560 (0.0011) [2023-12-26 17:38:41,748][105620] Updated weights for policy 1, policy_version 313570 (0.0009) [2023-12-26 17:38:41,953][105692] Updated weights for policy 0, policy_version 313326 (0.0010) [2023-12-26 17:38:41,998][105692] Updated weights for policy 0, policy_version 313336 (0.0011) [2023-12-26 17:38:42,059][105692] Updated weights for policy 0, policy_version 313346 (0.0010) [2023-12-26 17:38:42,357][105620] Updated weights for policy 1, policy_version 313580 (0.0007) [2023-12-26 17:38:42,414][105620] Updated weights for policy 1, policy_version 313590 (0.0009) [2023-12-26 17:38:42,472][105620] Updated weights for policy 1, policy_version 313600 (0.0008) [2023-12-26 17:38:42,839][105692] Updated weights for policy 0, policy_version 313356 (0.0008) [2023-12-26 17:38:42,898][105692] Updated weights for policy 0, policy_version 313366 (0.0009) [2023-12-26 17:38:42,946][105692] Updated weights for policy 0, policy_version 313376 (0.0009) [2023-12-26 17:38:43,245][105620] Updated weights for policy 1, policy_version 313610 (0.0009) [2023-12-26 17:38:43,298][105620] Updated weights for policy 1, policy_version 313620 (0.0010) [2023-12-26 17:38:43,351][105620] Updated weights for policy 1, policy_version 313631 (0.0010) [2023-12-26 17:38:43,601][105692] Updated weights for policy 0, policy_version 313386 (0.0009) [2023-12-26 17:38:43,650][105692] Updated weights for policy 0, policy_version 313397 (0.0009) [2023-12-26 17:38:43,707][105692] Updated weights for policy 0, policy_version 313407 (0.0009) [2023-12-26 17:38:44,153][105620] Updated weights for policy 1, policy_version 313642 (0.0010) [2023-12-26 17:38:44,219][105620] Updated weights for policy 1, policy_version 313652 (0.0010) [2023-12-26 17:38:44,280][105692] Updated weights for policy 0, policy_version 313417 (0.0006) [2023-12-26 17:38:44,288][105620] Updated weights for policy 1, policy_version 313662 (0.0009) [2023-12-26 17:38:44,332][105692] Updated weights for policy 0, policy_version 313427 (0.0008) [2023-12-26 17:38:44,350][105620] Updated weights for policy 1, policy_version 313672 (0.0009) [2023-12-26 17:38:44,391][105692] Updated weights for policy 0, policy_version 313437 (0.0008) [2023-12-26 17:38:44,438][105692] Updated weights for policy 0, policy_version 313447 (0.0005) [2023-12-26 17:38:45,102][105692] Updated weights for policy 0, policy_version 313457 (0.0008) [2023-12-26 17:38:45,153][105620] Updated weights for policy 1, policy_version 313682 (0.0008) [2023-12-26 17:38:45,155][105692] Updated weights for policy 0, policy_version 313467 (0.0006) [2023-12-26 17:38:45,214][105620] Updated weights for policy 1, policy_version 313692 (0.0007) [2023-12-26 17:38:45,221][105692] Updated weights for policy 0, policy_version 313477 (0.0007) [2023-12-26 17:38:45,264][105620] Updated weights for policy 1, policy_version 313702 (0.0009) [2023-12-26 17:38:45,778][105692] Updated weights for policy 0, policy_version 313487 (0.0005) [2023-12-26 17:38:45,829][105692] Updated weights for policy 0, policy_version 313497 (0.0005) [2023-12-26 17:38:45,890][105692] Updated weights for policy 0, policy_version 313507 (0.0007) [2023-12-26 17:38:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 160587776. Throughput: 0: 9816.6, 1: 9652.8. Samples: 160556880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:38:46,063][104569] Avg episode reward: [(0, '8720.733'), (1, '9355.917')] [2023-12-26 17:38:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000313512_80273408.pth... [2023-12-26 17:38:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000313704_80314368.pth... [2023-12-26 17:38:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000312360_79978496.pth [2023-12-26 17:38:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000312616_80035840.pth [2023-12-26 17:38:46,158][105620] Updated weights for policy 1, policy_version 313712 (0.0010) [2023-12-26 17:38:46,212][105620] Updated weights for policy 1, policy_version 313723 (0.0010) [2023-12-26 17:38:46,264][105620] Updated weights for policy 1, policy_version 313735 (0.0009) [2023-12-26 17:38:46,445][105692] Updated weights for policy 0, policy_version 313517 (0.0007) [2023-12-26 17:38:46,491][105692] Updated weights for policy 0, policy_version 313527 (0.0005) [2023-12-26 17:38:46,543][105692] Updated weights for policy 0, policy_version 313537 (0.0005) [2023-12-26 17:38:47,003][105620] Updated weights for policy 1, policy_version 313745 (0.0006) [2023-12-26 17:38:47,062][105620] Updated weights for policy 1, policy_version 313755 (0.0007) [2023-12-26 17:38:47,075][105692] Updated weights for policy 0, policy_version 313547 (0.0005) [2023-12-26 17:38:47,114][105620] Updated weights for policy 1, policy_version 313765 (0.0008) [2023-12-26 17:38:47,127][105692] Updated weights for policy 0, policy_version 313557 (0.0005) [2023-12-26 17:38:47,181][105692] Updated weights for policy 0, policy_version 313567 (0.0009) [2023-12-26 17:38:47,801][105620] Updated weights for policy 1, policy_version 313775 (0.0007) [2023-12-26 17:38:47,872][105620] Updated weights for policy 1, policy_version 313785 (0.0006) [2023-12-26 17:38:47,892][105692] Updated weights for policy 0, policy_version 313577 (0.0010) [2023-12-26 17:38:47,933][105620] Updated weights for policy 1, policy_version 313795 (0.0006) [2023-12-26 17:38:47,953][105692] Updated weights for policy 0, policy_version 313587 (0.0010) [2023-12-26 17:38:48,018][105692] Updated weights for policy 0, policy_version 313597 (0.0008) [2023-12-26 17:38:48,078][105692] Updated weights for policy 0, policy_version 313607 (0.0010) [2023-12-26 17:38:48,455][105620] Updated weights for policy 1, policy_version 313805 (0.0006) [2023-12-26 17:38:48,509][105620] Updated weights for policy 1, policy_version 313815 (0.0005) [2023-12-26 17:38:48,564][105620] Updated weights for policy 1, policy_version 313825 (0.0005) [2023-12-26 17:38:48,781][105692] Updated weights for policy 0, policy_version 313617 (0.0010) [2023-12-26 17:38:48,845][105692] Updated weights for policy 0, policy_version 313627 (0.0008) [2023-12-26 17:38:48,901][105692] Updated weights for policy 0, policy_version 313637 (0.0006) [2023-12-26 17:38:49,115][105620] Updated weights for policy 1, policy_version 313835 (0.0006) [2023-12-26 17:38:49,184][105620] Updated weights for policy 1, policy_version 313845 (0.0005) [2023-12-26 17:38:49,252][105620] Updated weights for policy 1, policy_version 313855 (0.0006) [2023-12-26 17:38:49,559][105692] Updated weights for policy 0, policy_version 313647 (0.0005) [2023-12-26 17:38:49,618][105692] Updated weights for policy 0, policy_version 313657 (0.0007) [2023-12-26 17:38:49,679][105692] Updated weights for policy 0, policy_version 313667 (0.0007) [2023-12-26 17:38:49,789][105620] Updated weights for policy 1, policy_version 313865 (0.0006) [2023-12-26 17:38:49,855][105620] Updated weights for policy 1, policy_version 313875 (0.0007) [2023-12-26 17:38:49,918][105620] Updated weights for policy 1, policy_version 313885 (0.0007) [2023-12-26 17:38:49,986][105620] Updated weights for policy 1, policy_version 313895 (0.0006) [2023-12-26 17:38:50,328][105692] Updated weights for policy 0, policy_version 313677 (0.0008) [2023-12-26 17:38:50,386][105692] Updated weights for policy 0, policy_version 313687 (0.0009) [2023-12-26 17:38:50,445][105692] Updated weights for policy 0, policy_version 313697 (0.0010) [2023-12-26 17:38:50,733][105620] Updated weights for policy 1, policy_version 313905 (0.0008) [2023-12-26 17:38:50,784][105620] Updated weights for policy 1, policy_version 313915 (0.0008) [2023-12-26 17:38:50,831][105620] Updated weights for policy 1, policy_version 313925 (0.0009) [2023-12-26 17:38:51,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 160694272. Throughput: 0: 9996.5, 1: 9566.4. Samples: 160683536. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 17:38:51,062][104569] Avg episode reward: [(0, '8811.634'), (1, '9355.912')] [2023-12-26 17:38:51,209][105692] Updated weights for policy 0, policy_version 313707 (0.0009) [2023-12-26 17:38:51,272][105692] Updated weights for policy 0, policy_version 313717 (0.0009) [2023-12-26 17:38:51,332][105692] Updated weights for policy 0, policy_version 313727 (0.0009) [2023-12-26 17:38:51,623][105620] Updated weights for policy 1, policy_version 313935 (0.0009) [2023-12-26 17:38:51,680][105620] Updated weights for policy 1, policy_version 313945 (0.0009) [2023-12-26 17:38:51,747][105620] Updated weights for policy 1, policy_version 313955 (0.0008) [2023-12-26 17:38:52,180][105692] Updated weights for policy 0, policy_version 313737 (0.0009) [2023-12-26 17:38:52,234][105692] Updated weights for policy 0, policy_version 313747 (0.0010) [2023-12-26 17:38:52,299][105692] Updated weights for policy 0, policy_version 313757 (0.0011) [2023-12-26 17:38:52,354][105692] Updated weights for policy 0, policy_version 313767 (0.0009) [2023-12-26 17:38:52,368][105620] Updated weights for policy 1, policy_version 313965 (0.0007) [2023-12-26 17:38:52,425][105620] Updated weights for policy 1, policy_version 313975 (0.0009) [2023-12-26 17:38:52,474][105620] Updated weights for policy 1, policy_version 313985 (0.0009) [2023-12-26 17:38:53,113][105692] Updated weights for policy 0, policy_version 313777 (0.0009) [2023-12-26 17:38:53,166][105692] Updated weights for policy 0, policy_version 313787 (0.0008) [2023-12-26 17:38:53,223][105692] Updated weights for policy 0, policy_version 313797 (0.0010) [2023-12-26 17:38:53,241][105620] Updated weights for policy 1, policy_version 313995 (0.0009) [2023-12-26 17:38:53,301][105620] Updated weights for policy 1, policy_version 314005 (0.0006) [2023-12-26 17:38:53,359][105620] Updated weights for policy 1, policy_version 314015 (0.0005) [2023-12-26 17:38:53,868][105620] Updated weights for policy 1, policy_version 314025 (0.0006) [2023-12-26 17:38:53,926][105620] Updated weights for policy 1, policy_version 314035 (0.0009) [2023-12-26 17:38:53,979][105620] Updated weights for policy 1, policy_version 314045 (0.0009) [2023-12-26 17:38:54,024][105620] Updated weights for policy 1, policy_version 314055 (0.0008) [2023-12-26 17:38:54,054][105692] Updated weights for policy 0, policy_version 313807 (0.0008) [2023-12-26 17:38:54,119][105692] Updated weights for policy 0, policy_version 313817 (0.0009) [2023-12-26 17:38:54,178][105692] Updated weights for policy 0, policy_version 313827 (0.0009) [2023-12-26 17:38:54,831][105620] Updated weights for policy 1, policy_version 314065 (0.0010) [2023-12-26 17:38:54,868][105692] Updated weights for policy 0, policy_version 313837 (0.0007) [2023-12-26 17:38:54,880][105620] Updated weights for policy 1, policy_version 314075 (0.0008) [2023-12-26 17:38:54,915][105692] Updated weights for policy 0, policy_version 313847 (0.0005) [2023-12-26 17:38:54,932][105620] Updated weights for policy 1, policy_version 314085 (0.0008) [2023-12-26 17:38:54,967][105692] Updated weights for policy 0, policy_version 313857 (0.0006) [2023-12-26 17:38:55,578][105692] Updated weights for policy 0, policy_version 313867 (0.0007) [2023-12-26 17:38:55,624][105692] Updated weights for policy 0, policy_version 313877 (0.0007) [2023-12-26 17:38:55,686][105692] Updated weights for policy 0, policy_version 313887 (0.0010) [2023-12-26 17:38:55,797][105620] Updated weights for policy 1, policy_version 314095 (0.0006) [2023-12-26 17:38:55,855][105620] Updated weights for policy 1, policy_version 314105 (0.0005) [2023-12-26 17:38:55,913][105620] Updated weights for policy 1, policy_version 314115 (0.0005) [2023-12-26 17:38:56,062][104569] Fps is (10 sec: 20480.6, 60 sec: 19660.9, 300 sec: 19633.0). Total num frames: 160792576. Throughput: 0: 9897.9, 1: 9642.6. Samples: 160798584. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 17:38:56,062][104569] Avg episode reward: [(0, '8993.949'), (1, '9171.097')] [2023-12-26 17:38:56,334][105692] Updated weights for policy 0, policy_version 313897 (0.0010) [2023-12-26 17:38:56,389][105692] Updated weights for policy 0, policy_version 313907 (0.0005) [2023-12-26 17:38:56,439][105692] Updated weights for policy 0, policy_version 313917 (0.0005) [2023-12-26 17:38:56,504][105692] Updated weights for policy 0, policy_version 313927 (0.0008) [2023-12-26 17:38:56,552][105620] Updated weights for policy 1, policy_version 314125 (0.0007) [2023-12-26 17:38:56,616][105620] Updated weights for policy 1, policy_version 314135 (0.0008) [2023-12-26 17:38:56,680][105620] Updated weights for policy 1, policy_version 314145 (0.0007) [2023-12-26 17:38:57,060][105692] Updated weights for policy 0, policy_version 313937 (0.0007) [2023-12-26 17:38:57,123][105692] Updated weights for policy 0, policy_version 313947 (0.0008) [2023-12-26 17:38:57,188][105692] Updated weights for policy 0, policy_version 313957 (0.0008) [2023-12-26 17:38:57,300][105620] Updated weights for policy 1, policy_version 314155 (0.0007) [2023-12-26 17:38:57,355][105620] Updated weights for policy 1, policy_version 314165 (0.0006) [2023-12-26 17:38:57,411][105620] Updated weights for policy 1, policy_version 314175 (0.0006) [2023-12-26 17:38:57,782][105692] Updated weights for policy 0, policy_version 313967 (0.0009) [2023-12-26 17:38:57,799][105585] KL-divergence is very high: 107.3813 [2023-12-26 17:38:57,839][105692] Updated weights for policy 0, policy_version 313978 (0.0010) [2023-12-26 17:38:57,845][105585] KL-divergence is very high: 156.4504 [2023-12-26 17:38:57,887][105585] KL-divergence is very high: 140.8028 [2023-12-26 17:38:57,895][105692] Updated weights for policy 0, policy_version 313988 (0.0009) [2023-12-26 17:38:57,982][105620] Updated weights for policy 1, policy_version 314185 (0.0010) [2023-12-26 17:38:58,041][105620] Updated weights for policy 1, policy_version 314195 (0.0009) [2023-12-26 17:38:58,089][105620] Updated weights for policy 1, policy_version 314205 (0.0010) [2023-12-26 17:38:58,155][105620] Updated weights for policy 1, policy_version 314215 (0.0009) [2023-12-26 17:38:58,670][105692] Updated weights for policy 0, policy_version 313999 (0.0009) [2023-12-26 17:38:58,734][105692] Updated weights for policy 0, policy_version 314009 (0.0008) [2023-12-26 17:38:58,794][105692] Updated weights for policy 0, policy_version 314019 (0.0008) [2023-12-26 17:38:58,884][105620] Updated weights for policy 1, policy_version 314225 (0.0010) [2023-12-26 17:38:58,943][105620] Updated weights for policy 1, policy_version 314235 (0.0010) [2023-12-26 17:38:59,005][105620] Updated weights for policy 1, policy_version 314245 (0.0010) [2023-12-26 17:38:59,514][105692] Updated weights for policy 0, policy_version 314029 (0.0009) [2023-12-26 17:38:59,573][105692] Updated weights for policy 0, policy_version 314039 (0.0009) [2023-12-26 17:38:59,626][105692] Updated weights for policy 0, policy_version 314049 (0.0009) [2023-12-26 17:38:59,690][105620] Updated weights for policy 1, policy_version 314255 (0.0007) [2023-12-26 17:38:59,755][105620] Updated weights for policy 1, policy_version 314265 (0.0006) [2023-12-26 17:38:59,809][105620] Updated weights for policy 1, policy_version 314275 (0.0006) [2023-12-26 17:39:00,338][105692] Updated weights for policy 0, policy_version 314060 (0.0010) [2023-12-26 17:39:00,403][105692] Updated weights for policy 0, policy_version 314070 (0.0010) [2023-12-26 17:39:00,458][105692] Updated weights for policy 0, policy_version 314080 (0.0009) [2023-12-26 17:39:00,570][105620] Updated weights for policy 1, policy_version 314285 (0.0009) [2023-12-26 17:39:00,628][105620] Updated weights for policy 1, policy_version 314295 (0.0010) [2023-12-26 17:39:00,686][105620] Updated weights for policy 1, policy_version 314305 (0.0007) [2023-12-26 17:39:01,029][105692] Updated weights for policy 0, policy_version 314090 (0.0010) [2023-12-26 17:39:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 160890880. Throughput: 0: 9946.9, 1: 9716.5. Samples: 160862024. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 17:39:01,062][104569] Avg episode reward: [(0, '8902.824'), (1, '3503.059')] [2023-12-26 17:39:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000314312_80470016.pth... [2023-12-26 17:39:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000313160_80175104.pth [2023-12-26 17:39:01,088][105692] Updated weights for policy 0, policy_version 314100 (0.0007) [2023-12-26 17:39:01,140][105692] Updated weights for policy 0, policy_version 314110 (0.0006) [2023-12-26 17:39:01,198][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000314120_80429056.pth... [2023-12-26 17:39:01,200][105692] Updated weights for policy 0, policy_version 314120 (0.0008) [2023-12-26 17:39:01,202][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000312936_80125952.pth [2023-12-26 17:39:01,326][105620] Updated weights for policy 1, policy_version 314315 (0.0006) [2023-12-26 17:39:01,388][105620] Updated weights for policy 1, policy_version 314325 (0.0009) [2023-12-26 17:39:01,448][105620] Updated weights for policy 1, policy_version 314335 (0.0009) [2023-12-26 17:39:01,833][105692] Updated weights for policy 0, policy_version 314130 (0.0009) [2023-12-26 17:39:01,882][105692] Updated weights for policy 0, policy_version 314140 (0.0009) [2023-12-26 17:39:01,941][105692] Updated weights for policy 0, policy_version 314150 (0.0009) [2023-12-26 17:39:02,281][105620] Updated weights for policy 1, policy_version 314345 (0.0010) [2023-12-26 17:39:02,341][105620] Updated weights for policy 1, policy_version 314355 (0.0009) [2023-12-26 17:39:02,408][105620] Updated weights for policy 1, policy_version 314365 (0.0009) [2023-12-26 17:39:02,468][105620] Updated weights for policy 1, policy_version 314375 (0.0005) [2023-12-26 17:39:02,740][105692] Updated weights for policy 0, policy_version 314160 (0.0008) [2023-12-26 17:39:02,791][105692] Updated weights for policy 0, policy_version 314170 (0.0008) [2023-12-26 17:39:02,810][105585] KL-divergence is very high: 118.3187 [2023-12-26 17:39:02,843][105692] Updated weights for policy 0, policy_version 314180 (0.0008) [2023-12-26 17:39:02,848][105585] KL-divergence is very high: 125.5651 [2023-12-26 17:39:03,159][105620] Updated weights for policy 1, policy_version 314385 (0.0010) [2023-12-26 17:39:03,215][105620] Updated weights for policy 1, policy_version 314395 (0.0010) [2023-12-26 17:39:03,280][105620] Updated weights for policy 1, policy_version 314405 (0.0010) [2023-12-26 17:39:03,646][105692] Updated weights for policy 0, policy_version 314190 (0.0010) [2023-12-26 17:39:03,702][105692] Updated weights for policy 0, policy_version 314200 (0.0008) [2023-12-26 17:39:03,757][105692] Updated weights for policy 0, policy_version 314210 (0.0010) [2023-12-26 17:39:03,886][105620] Updated weights for policy 1, policy_version 314415 (0.0007) [2023-12-26 17:39:03,948][105620] Updated weights for policy 1, policy_version 314425 (0.0009) [2023-12-26 17:39:04,015][105620] Updated weights for policy 1, policy_version 314435 (0.0007) [2023-12-26 17:39:04,476][105692] Updated weights for policy 0, policy_version 314220 (0.0010) [2023-12-26 17:39:04,536][105692] Updated weights for policy 0, policy_version 314230 (0.0010) [2023-12-26 17:39:04,592][105692] Updated weights for policy 0, policy_version 314240 (0.0010) [2023-12-26 17:39:04,671][105620] Updated weights for policy 1, policy_version 314445 (0.0007) [2023-12-26 17:39:04,730][105620] Updated weights for policy 1, policy_version 314455 (0.0005) [2023-12-26 17:39:04,788][105620] Updated weights for policy 1, policy_version 314465 (0.0010) [2023-12-26 17:39:05,344][105692] Updated weights for policy 0, policy_version 314250 (0.0010) [2023-12-26 17:39:05,388][105585] KL-divergence is very high: 184.0894 [2023-12-26 17:39:05,392][105692] Updated weights for policy 0, policy_version 314260 (0.0010) [2023-12-26 17:39:05,415][105585] KL-divergence is very high: 131.3485 [2023-12-26 17:39:05,425][105585] KL-divergence is very high: 337.8721 [2023-12-26 17:39:05,440][105692] Updated weights for policy 0, policy_version 314270 (0.0010) [2023-12-26 17:39:05,452][105585] KL-divergence is very high: 110.3146 [2023-12-26 17:39:05,461][105585] KL-divergence is very high: 338.8993 [2023-12-26 17:39:05,487][105692] Updated weights for policy 0, policy_version 314280 (0.0010) [2023-12-26 17:39:05,501][105620] Updated weights for policy 1, policy_version 314475 (0.0010) [2023-12-26 17:39:05,565][105620] Updated weights for policy 1, policy_version 314485 (0.0009) [2023-12-26 17:39:05,621][105620] Updated weights for policy 1, policy_version 314495 (0.0008) [2023-12-26 17:39:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 160989184. Throughput: 0: 9886.4, 1: 9809.2. Samples: 160979960. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 17:39:06,063][104569] Avg episode reward: [(0, '8809.615'), (1, '807.691')] [2023-12-26 17:39:06,274][105692] Updated weights for policy 0, policy_version 314290 (0.0011) [2023-12-26 17:39:06,331][105692] Updated weights for policy 0, policy_version 314300 (0.0008) [2023-12-26 17:39:06,398][105692] Updated weights for policy 0, policy_version 314310 (0.0011) [2023-12-26 17:39:06,431][105620] Updated weights for policy 1, policy_version 314505 (0.0009) [2023-12-26 17:39:06,491][105620] Updated weights for policy 1, policy_version 314515 (0.0008) [2023-12-26 17:39:06,551][105620] Updated weights for policy 1, policy_version 314525 (0.0009) [2023-12-26 17:39:06,601][105620] Updated weights for policy 1, policy_version 314535 (0.0008) [2023-12-26 17:39:07,132][105692] Updated weights for policy 0, policy_version 314320 (0.0010) [2023-12-26 17:39:07,185][105692] Updated weights for policy 0, policy_version 314330 (0.0010) [2023-12-26 17:39:07,236][105692] Updated weights for policy 0, policy_version 314340 (0.0010) [2023-12-26 17:39:07,380][105620] Updated weights for policy 1, policy_version 314545 (0.0008) [2023-12-26 17:39:07,435][105620] Updated weights for policy 1, policy_version 314555 (0.0008) [2023-12-26 17:39:07,484][105620] Updated weights for policy 1, policy_version 314565 (0.0008) [2023-12-26 17:39:07,974][105692] Updated weights for policy 0, policy_version 314350 (0.0007) [2023-12-26 17:39:08,027][105692] Updated weights for policy 0, policy_version 314360 (0.0005) [2023-12-26 17:39:08,072][105692] Updated weights for policy 0, policy_version 314370 (0.0005) [2023-12-26 17:39:08,298][105620] Updated weights for policy 1, policy_version 314575 (0.0009) [2023-12-26 17:39:08,365][105620] Updated weights for policy 1, policy_version 314585 (0.0008) [2023-12-26 17:39:08,427][105620] Updated weights for policy 1, policy_version 314595 (0.0005) [2023-12-26 17:39:08,687][105692] Updated weights for policy 0, policy_version 314380 (0.0007) [2023-12-26 17:39:08,751][105692] Updated weights for policy 0, policy_version 314390 (0.0008) [2023-12-26 17:39:08,817][105692] Updated weights for policy 0, policy_version 314400 (0.0011) [2023-12-26 17:39:09,159][105620] Updated weights for policy 1, policy_version 314606 (0.0008) [2023-12-26 17:39:09,211][105620] Updated weights for policy 1, policy_version 314616 (0.0008) [2023-12-26 17:39:09,280][105620] Updated weights for policy 1, policy_version 314626 (0.0009) [2023-12-26 17:39:09,472][105692] Updated weights for policy 0, policy_version 314410 (0.0010) [2023-12-26 17:39:09,538][105692] Updated weights for policy 0, policy_version 314420 (0.0006) [2023-12-26 17:39:09,604][105692] Updated weights for policy 0, policy_version 314430 (0.0006) [2023-12-26 17:39:09,666][105692] Updated weights for policy 0, policy_version 314440 (0.0006) [2023-12-26 17:39:10,095][105620] Updated weights for policy 1, policy_version 314636 (0.0009) [2023-12-26 17:39:10,159][105620] Updated weights for policy 1, policy_version 314646 (0.0008) [2023-12-26 17:39:10,222][105620] Updated weights for policy 1, policy_version 314656 (0.0008) [2023-12-26 17:39:10,357][105692] Updated weights for policy 0, policy_version 314450 (0.0009) [2023-12-26 17:39:10,417][105692] Updated weights for policy 0, policy_version 314460 (0.0009) [2023-12-26 17:39:10,474][105692] Updated weights for policy 0, policy_version 314470 (0.0009) [2023-12-26 17:39:10,948][105620] Updated weights for policy 1, policy_version 314666 (0.0008) [2023-12-26 17:39:11,017][105620] Updated weights for policy 1, policy_version 314676 (0.0010) [2023-12-26 17:39:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 161079296. Throughput: 0: 9933.2, 1: 9782.4. Samples: 161092760. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 17:39:11,062][104569] Avg episode reward: [(0, '8628.709'), (1, '969.908')] [2023-12-26 17:39:11,084][105620] Updated weights for policy 1, policy_version 314686 (0.0008) [2023-12-26 17:39:11,158][105620] Updated weights for policy 1, policy_version 314696 (0.0007) [2023-12-26 17:39:11,297][105692] Updated weights for policy 0, policy_version 314480 (0.0007) [2023-12-26 17:39:11,358][105585] KL-divergence is very high: 100.1528 [2023-12-26 17:39:11,359][105692] Updated weights for policy 0, policy_version 314490 (0.0008) [2023-12-26 17:39:11,372][105585] KL-divergence is very high: 115.0724 [2023-12-26 17:39:11,411][105585] KL-divergence is very high: 110.9944 [2023-12-26 17:39:11,423][105692] Updated weights for policy 0, policy_version 314500 (0.0008) [2023-12-26 17:39:11,953][105620] Updated weights for policy 1, policy_version 314706 (0.0010) [2023-12-26 17:39:12,016][105620] Updated weights for policy 1, policy_version 314716 (0.0008) [2023-12-26 17:39:12,075][105620] Updated weights for policy 1, policy_version 314726 (0.0008) [2023-12-26 17:39:12,121][105692] Updated weights for policy 0, policy_version 314510 (0.0009) [2023-12-26 17:39:12,133][105585] KL-divergence is very high: 226.4752 [2023-12-26 17:39:12,185][105585] KL-divergence is very high: 288.4177 [2023-12-26 17:39:12,186][105692] Updated weights for policy 0, policy_version 314520 (0.0010) [2023-12-26 17:39:12,236][105585] KL-divergence is very high: 216.4021 [2023-12-26 17:39:12,253][105692] Updated weights for policy 0, policy_version 314530 (0.0009) [2023-12-26 17:39:12,892][105620] Updated weights for policy 1, policy_version 314736 (0.0008) [2023-12-26 17:39:12,939][105692] Updated weights for policy 0, policy_version 314540 (0.0009) [2023-12-26 17:39:12,949][105620] Updated weights for policy 1, policy_version 314746 (0.0007) [2023-12-26 17:39:12,993][105692] Updated weights for policy 0, policy_version 314550 (0.0010) [2023-12-26 17:39:13,008][105620] Updated weights for policy 1, policy_version 314756 (0.0006) [2023-12-26 17:39:13,054][105692] Updated weights for policy 0, policy_version 314560 (0.0010) [2023-12-26 17:39:13,657][105692] Updated weights for policy 0, policy_version 314570 (0.0009) [2023-12-26 17:39:13,674][105620] Updated weights for policy 1, policy_version 314766 (0.0008) [2023-12-26 17:39:13,717][105692] Updated weights for policy 0, policy_version 314580 (0.0007) [2023-12-26 17:39:13,720][105620] Updated weights for policy 1, policy_version 314776 (0.0010) [2023-12-26 17:39:13,762][105692] Updated weights for policy 0, policy_version 314590 (0.0006) [2023-12-26 17:39:13,769][105620] Updated weights for policy 1, policy_version 314786 (0.0009) [2023-12-26 17:39:13,822][105692] Updated weights for policy 0, policy_version 314600 (0.0009) [2023-12-26 17:39:14,409][105620] Updated weights for policy 1, policy_version 314796 (0.0006) [2023-12-26 17:39:14,466][105620] Updated weights for policy 1, policy_version 314806 (0.0006) [2023-12-26 17:39:14,526][105620] Updated weights for policy 1, policy_version 314816 (0.0006) [2023-12-26 17:39:14,604][105692] Updated weights for policy 0, policy_version 314610 (0.0008) [2023-12-26 17:39:14,659][105692] Updated weights for policy 0, policy_version 314620 (0.0008) [2023-12-26 17:39:14,710][105692] Updated weights for policy 0, policy_version 314630 (0.0008) [2023-12-26 17:39:15,207][105620] Updated weights for policy 1, policy_version 314826 (0.0007) [2023-12-26 17:39:15,266][105620] Updated weights for policy 1, policy_version 314836 (0.0011) [2023-12-26 17:39:15,332][105620] Updated weights for policy 1, policy_version 314846 (0.0007) [2023-12-26 17:39:15,406][105620] Updated weights for policy 1, policy_version 314856 (0.0006) [2023-12-26 17:39:15,554][105692] Updated weights for policy 0, policy_version 314640 (0.0007) [2023-12-26 17:39:15,613][105692] Updated weights for policy 0, policy_version 314650 (0.0008) [2023-12-26 17:39:15,666][105692] Updated weights for policy 0, policy_version 314660 (0.0006) [2023-12-26 17:39:16,021][105620] Updated weights for policy 1, policy_version 314866 (0.0005) [2023-12-26 17:39:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 161177600. Throughput: 0: 9931.7, 1: 9747.4. Samples: 161150688. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 17:39:16,062][104569] Avg episode reward: [(0, '8627.673'), (1, '2214.969')] [2023-12-26 17:39:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000314664_80568320.pth... [2023-12-26 17:39:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000313512_80273408.pth [2023-12-26 17:39:16,086][105620] Updated weights for policy 1, policy_version 314876 (0.0010) [2023-12-26 17:39:16,155][105620] Updated weights for policy 1, policy_version 314886 (0.0008) [2023-12-26 17:39:16,167][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000314888_80617472.pth... [2023-12-26 17:39:16,170][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000313704_80314368.pth [2023-12-26 17:39:16,491][105692] Updated weights for policy 0, policy_version 314670 (0.0007) [2023-12-26 17:39:16,549][105692] Updated weights for policy 0, policy_version 314680 (0.0008) [2023-12-26 17:39:16,609][105692] Updated weights for policy 0, policy_version 314690 (0.0009) [2023-12-26 17:39:16,719][105620] Updated weights for policy 1, policy_version 314896 (0.0009) [2023-12-26 17:39:16,783][105620] Updated weights for policy 1, policy_version 314906 (0.0009) [2023-12-26 17:39:16,838][105620] Updated weights for policy 1, policy_version 314916 (0.0010) [2023-12-26 17:39:17,381][105692] Updated weights for policy 0, policy_version 314700 (0.0008) [2023-12-26 17:39:17,440][105692] Updated weights for policy 0, policy_version 314710 (0.0009) [2023-12-26 17:39:17,494][105692] Updated weights for policy 0, policy_version 314720 (0.0005) [2023-12-26 17:39:17,580][105620] Updated weights for policy 1, policy_version 314926 (0.0010) [2023-12-26 17:39:17,634][105620] Updated weights for policy 1, policy_version 314936 (0.0010) [2023-12-26 17:39:17,697][105620] Updated weights for policy 1, policy_version 314946 (0.0011) [2023-12-26 17:39:18,250][105692] Updated weights for policy 0, policy_version 314730 (0.0008) [2023-12-26 17:39:18,310][105692] Updated weights for policy 0, policy_version 314740 (0.0009) [2023-12-26 17:39:18,332][105620] Updated weights for policy 1, policy_version 314956 (0.0010) [2023-12-26 17:39:18,374][105692] Updated weights for policy 0, policy_version 314750 (0.0006) [2023-12-26 17:39:18,391][105620] Updated weights for policy 1, policy_version 314966 (0.0009) [2023-12-26 17:39:18,421][105586] KL-divergence is very high: 118.0987 [2023-12-26 17:39:18,423][105692] Updated weights for policy 0, policy_version 314760 (0.0007) [2023-12-26 17:39:18,453][105620] Updated weights for policy 1, policy_version 314976 (0.0006) [2023-12-26 17:39:19,067][105620] Updated weights for policy 1, policy_version 314986 (0.0006) [2023-12-26 17:39:19,121][105620] Updated weights for policy 1, policy_version 314996 (0.0005) [2023-12-26 17:39:19,175][105620] Updated weights for policy 1, policy_version 315006 (0.0005) [2023-12-26 17:39:19,242][105620] Updated weights for policy 1, policy_version 315016 (0.0006) [2023-12-26 17:39:19,242][105692] Updated weights for policy 0, policy_version 314770 (0.0008) [2023-12-26 17:39:19,313][105692] Updated weights for policy 0, policy_version 314780 (0.0009) [2023-12-26 17:39:19,384][105692] Updated weights for policy 0, policy_version 314790 (0.0010) [2023-12-26 17:39:19,813][105620] Updated weights for policy 1, policy_version 315026 (0.0005) [2023-12-26 17:39:19,872][105620] Updated weights for policy 1, policy_version 315036 (0.0008) [2023-12-26 17:39:19,935][105620] Updated weights for policy 1, policy_version 315046 (0.0008) [2023-12-26 17:39:20,252][105692] Updated weights for policy 0, policy_version 314800 (0.0009) [2023-12-26 17:39:20,310][105692] Updated weights for policy 0, policy_version 314810 (0.0009) [2023-12-26 17:39:20,368][105692] Updated weights for policy 0, policy_version 314820 (0.0009) [2023-12-26 17:39:20,596][105620] Updated weights for policy 1, policy_version 315056 (0.0007) [2023-12-26 17:39:20,650][105620] Updated weights for policy 1, policy_version 315066 (0.0007) [2023-12-26 17:39:20,715][105620] Updated weights for policy 1, policy_version 315076 (0.0008) [2023-12-26 17:39:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 161275904. Throughput: 0: 9804.3, 1: 9837.5. Samples: 161267380. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 17:39:21,063][104569] Avg episode reward: [(0, '8262.709'), (1, '6366.721')] [2023-12-26 17:39:21,186][105692] Updated weights for policy 0, policy_version 314830 (0.0008) [2023-12-26 17:39:21,247][105692] Updated weights for policy 0, policy_version 314840 (0.0006) [2023-12-26 17:39:21,303][105692] Updated weights for policy 0, policy_version 314850 (0.0010) [2023-12-26 17:39:21,455][105620] Updated weights for policy 1, policy_version 315086 (0.0007) [2023-12-26 17:39:21,521][105620] Updated weights for policy 1, policy_version 315096 (0.0008) [2023-12-26 17:39:21,587][105620] Updated weights for policy 1, policy_version 315106 (0.0008) [2023-12-26 17:39:21,956][105692] Updated weights for policy 0, policy_version 314860 (0.0008) [2023-12-26 17:39:22,020][105692] Updated weights for policy 0, policy_version 314870 (0.0005) [2023-12-26 17:39:22,079][105692] Updated weights for policy 0, policy_version 314880 (0.0005) [2023-12-26 17:39:22,386][105620] Updated weights for policy 1, policy_version 315116 (0.0008) [2023-12-26 17:39:22,449][105620] Updated weights for policy 1, policy_version 315126 (0.0010) [2023-12-26 17:39:22,502][105620] Updated weights for policy 1, policy_version 315136 (0.0010) [2023-12-26 17:39:22,690][105692] Updated weights for policy 0, policy_version 314890 (0.0007) [2023-12-26 17:39:22,750][105692] Updated weights for policy 0, policy_version 314900 (0.0010) [2023-12-26 17:39:22,809][105692] Updated weights for policy 0, policy_version 314910 (0.0009) [2023-12-26 17:39:22,865][105692] Updated weights for policy 0, policy_version 314920 (0.0009) [2023-12-26 17:39:23,283][105620] Updated weights for policy 1, policy_version 315146 (0.0010) [2023-12-26 17:39:23,344][105620] Updated weights for policy 1, policy_version 315156 (0.0009) [2023-12-26 17:39:23,402][105620] Updated weights for policy 1, policy_version 315166 (0.0009) [2023-12-26 17:39:23,459][105620] Updated weights for policy 1, policy_version 315176 (0.0008) [2023-12-26 17:39:23,603][105692] Updated weights for policy 0, policy_version 314930 (0.0009) [2023-12-26 17:39:23,664][105692] Updated weights for policy 0, policy_version 314940 (0.0009) [2023-12-26 17:39:23,722][105692] Updated weights for policy 0, policy_version 314950 (0.0009) [2023-12-26 17:39:24,112][105620] Updated weights for policy 1, policy_version 315186 (0.0009) [2023-12-26 17:39:24,167][105620] Updated weights for policy 1, policy_version 315196 (0.0009) [2023-12-26 17:39:24,226][105620] Updated weights for policy 1, policy_version 315206 (0.0008) [2023-12-26 17:39:24,417][105692] Updated weights for policy 0, policy_version 314960 (0.0006) [2023-12-26 17:39:24,465][105692] Updated weights for policy 0, policy_version 314970 (0.0005) [2023-12-26 17:39:24,516][105692] Updated weights for policy 0, policy_version 314980 (0.0005) [2023-12-26 17:39:25,045][105620] Updated weights for policy 1, policy_version 315216 (0.0010) [2023-12-26 17:39:25,104][105620] Updated weights for policy 1, policy_version 315226 (0.0010) [2023-12-26 17:39:25,164][105620] Updated weights for policy 1, policy_version 315236 (0.0008) [2023-12-26 17:39:25,180][105692] Updated weights for policy 0, policy_version 314990 (0.0008) [2023-12-26 17:39:25,239][105692] Updated weights for policy 0, policy_version 315000 (0.0007) [2023-12-26 17:39:25,301][105692] Updated weights for policy 0, policy_version 315010 (0.0005) [2023-12-26 17:39:25,848][105692] Updated weights for policy 0, policy_version 315020 (0.0005) [2023-12-26 17:39:25,853][105620] Updated weights for policy 1, policy_version 315246 (0.0006) [2023-12-26 17:39:25,905][105692] Updated weights for policy 0, policy_version 315030 (0.0006) [2023-12-26 17:39:25,907][105620] Updated weights for policy 1, policy_version 315256 (0.0010) [2023-12-26 17:39:25,958][105620] Updated weights for policy 1, policy_version 315266 (0.0008) [2023-12-26 17:39:25,963][105692] Updated weights for policy 0, policy_version 315040 (0.0006) [2023-12-26 17:39:26,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 161382400. Throughput: 0: 9857.5, 1: 9792.4. Samples: 161383232. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 17:39:26,063][104569] Avg episode reward: [(0, '8173.462'), (1, '7205.891')] [2023-12-26 17:39:26,489][105692] Updated weights for policy 0, policy_version 315050 (0.0007) [2023-12-26 17:39:26,516][105620] Updated weights for policy 1, policy_version 315276 (0.0006) [2023-12-26 17:39:26,545][105692] Updated weights for policy 0, policy_version 315060 (0.0005) [2023-12-26 17:39:26,578][105620] Updated weights for policy 1, policy_version 315286 (0.0007) [2023-12-26 17:39:26,604][105692] Updated weights for policy 0, policy_version 315070 (0.0007) [2023-12-26 17:39:26,637][105620] Updated weights for policy 1, policy_version 315296 (0.0011) [2023-12-26 17:39:26,656][105692] Updated weights for policy 0, policy_version 315080 (0.0008) [2023-12-26 17:39:27,287][105692] Updated weights for policy 0, policy_version 315090 (0.0007) [2023-12-26 17:39:27,287][105620] Updated weights for policy 1, policy_version 315306 (0.0009) [2023-12-26 17:39:27,340][105692] Updated weights for policy 0, policy_version 315100 (0.0006) [2023-12-26 17:39:27,347][105620] Updated weights for policy 1, policy_version 315316 (0.0010) [2023-12-26 17:39:27,388][105692] Updated weights for policy 0, policy_version 315110 (0.0010) [2023-12-26 17:39:27,394][105620] Updated weights for policy 1, policy_version 315326 (0.0010) [2023-12-26 17:39:27,448][105620] Updated weights for policy 1, policy_version 315336 (0.0010) [2023-12-26 17:39:27,954][105692] Updated weights for policy 0, policy_version 315120 (0.0006) [2023-12-26 17:39:28,018][105692] Updated weights for policy 0, policy_version 315130 (0.0005) [2023-12-26 17:39:28,079][105692] Updated weights for policy 0, policy_version 315140 (0.0007) [2023-12-26 17:39:28,121][105620] Updated weights for policy 1, policy_version 315346 (0.0009) [2023-12-26 17:39:28,185][105620] Updated weights for policy 1, policy_version 315356 (0.0008) [2023-12-26 17:39:28,245][105620] Updated weights for policy 1, policy_version 315366 (0.0008) [2023-12-26 17:39:28,593][105692] Updated weights for policy 0, policy_version 315150 (0.0007) [2023-12-26 17:39:28,644][105692] Updated weights for policy 0, policy_version 315160 (0.0005) [2023-12-26 17:39:28,696][105692] Updated weights for policy 0, policy_version 315170 (0.0005) [2023-12-26 17:39:28,877][105620] Updated weights for policy 1, policy_version 315376 (0.0006) [2023-12-26 17:39:28,939][105620] Updated weights for policy 1, policy_version 315386 (0.0008) [2023-12-26 17:39:29,002][105620] Updated weights for policy 1, policy_version 315396 (0.0008) [2023-12-26 17:39:29,243][105692] Updated weights for policy 0, policy_version 315180 (0.0006) [2023-12-26 17:39:29,300][105692] Updated weights for policy 0, policy_version 315190 (0.0008) [2023-12-26 17:39:29,360][105692] Updated weights for policy 0, policy_version 315200 (0.0008) [2023-12-26 17:39:29,681][105620] Updated weights for policy 1, policy_version 315406 (0.0007) [2023-12-26 17:39:29,730][105620] Updated weights for policy 1, policy_version 315416 (0.0007) [2023-12-26 17:39:29,781][105620] Updated weights for policy 1, policy_version 315426 (0.0008) [2023-12-26 17:39:29,972][105692] Updated weights for policy 0, policy_version 315210 (0.0007) [2023-12-26 17:39:30,031][105692] Updated weights for policy 0, policy_version 315220 (0.0008) [2023-12-26 17:39:30,099][105692] Updated weights for policy 0, policy_version 315230 (0.0006) [2023-12-26 17:39:30,156][105692] Updated weights for policy 0, policy_version 315240 (0.0008) [2023-12-26 17:39:30,538][105620] Updated weights for policy 1, policy_version 315436 (0.0009) [2023-12-26 17:39:30,604][105620] Updated weights for policy 1, policy_version 315446 (0.0011) [2023-12-26 17:39:30,669][105620] Updated weights for policy 1, policy_version 315456 (0.0011) [2023-12-26 17:39:30,828][105692] Updated weights for policy 0, policy_version 315250 (0.0005) [2023-12-26 17:39:30,877][105585] KL-divergence is very high: 100.3273 [2023-12-26 17:39:30,898][105692] Updated weights for policy 0, policy_version 315260 (0.0005) [2023-12-26 17:39:30,967][105692] Updated weights for policy 0, policy_version 315270 (0.0005) [2023-12-26 17:39:31,062][104569] Fps is (10 sec: 21299.4, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 161488896. Throughput: 0: 10012.7, 1: 9878.5. Samples: 161451980. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 17:39:31,062][104569] Avg episode reward: [(0, '8080.498'), (1, '8825.445')] [2023-12-26 17:39:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000315272_80723968.pth... [2023-12-26 17:39:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000315464_80764928.pth... [2023-12-26 17:39:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000314312_80470016.pth [2023-12-26 17:39:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000314120_80429056.pth [2023-12-26 17:39:31,321][105620] Updated weights for policy 1, policy_version 315466 (0.0010) [2023-12-26 17:39:31,391][105620] Updated weights for policy 1, policy_version 315476 (0.0008) [2023-12-26 17:39:31,450][105620] Updated weights for policy 1, policy_version 315486 (0.0005) [2023-12-26 17:39:31,509][105692] Updated weights for policy 0, policy_version 315280 (0.0008) [2023-12-26 17:39:31,509][105620] Updated weights for policy 1, policy_version 315496 (0.0005) [2023-12-26 17:39:31,511][105585] KL-divergence is very high: 116.4457 [2023-12-26 17:39:31,517][105585] KL-divergence is very high: 140.4107 [2023-12-26 17:39:31,560][105585] KL-divergence is very high: 117.3032 [2023-12-26 17:39:31,566][105585] KL-divergence is very high: 119.4392 [2023-12-26 17:39:31,570][105692] Updated weights for policy 0, policy_version 315290 (0.0005) [2023-12-26 17:39:31,633][105692] Updated weights for policy 0, policy_version 315300 (0.0007) [2023-12-26 17:39:32,125][105620] Updated weights for policy 1, policy_version 315506 (0.0011) [2023-12-26 17:39:32,173][105620] Updated weights for policy 1, policy_version 315516 (0.0010) [2023-12-26 17:39:32,228][105620] Updated weights for policy 1, policy_version 315526 (0.0010) [2023-12-26 17:39:32,268][105692] Updated weights for policy 0, policy_version 315310 (0.0008) [2023-12-26 17:39:32,330][105692] Updated weights for policy 0, policy_version 315320 (0.0008) [2023-12-26 17:39:32,393][105692] Updated weights for policy 0, policy_version 315330 (0.0008) [2023-12-26 17:39:32,860][105620] Updated weights for policy 1, policy_version 315536 (0.0006) [2023-12-26 17:39:32,914][105620] Updated weights for policy 1, policy_version 315546 (0.0005) [2023-12-26 17:39:32,968][105620] Updated weights for policy 1, policy_version 315556 (0.0005) [2023-12-26 17:39:33,035][105692] Updated weights for policy 0, policy_version 315340 (0.0007) [2023-12-26 17:39:33,091][105692] Updated weights for policy 0, policy_version 315351 (0.0010) [2023-12-26 17:39:33,142][105692] Updated weights for policy 0, policy_version 315361 (0.0009) [2023-12-26 17:39:33,526][105620] Updated weights for policy 1, policy_version 315566 (0.0008) [2023-12-26 17:39:33,579][105620] Updated weights for policy 1, policy_version 315576 (0.0007) [2023-12-26 17:39:33,634][105620] Updated weights for policy 1, policy_version 315586 (0.0005) [2023-12-26 17:39:33,917][105692] Updated weights for policy 0, policy_version 315371 (0.0010) [2023-12-26 17:39:33,961][105692] Updated weights for policy 0, policy_version 315381 (0.0010) [2023-12-26 17:39:34,018][105692] Updated weights for policy 0, policy_version 315391 (0.0010) [2023-12-26 17:39:34,343][105620] Updated weights for policy 1, policy_version 315596 (0.0006) [2023-12-26 17:39:34,414][105620] Updated weights for policy 1, policy_version 315606 (0.0007) [2023-12-26 17:39:34,474][105620] Updated weights for policy 1, policy_version 315616 (0.0011) [2023-12-26 17:39:34,786][105692] Updated weights for policy 0, policy_version 315401 (0.0010) [2023-12-26 17:39:34,840][105692] Updated weights for policy 0, policy_version 315411 (0.0010) [2023-12-26 17:39:34,897][105692] Updated weights for policy 0, policy_version 315421 (0.0008) [2023-12-26 17:39:34,957][105692] Updated weights for policy 0, policy_version 315431 (0.0008) [2023-12-26 17:39:35,194][105620] Updated weights for policy 1, policy_version 315626 (0.0011) [2023-12-26 17:39:35,258][105620] Updated weights for policy 1, policy_version 315636 (0.0011) [2023-12-26 17:39:35,341][105620] Updated weights for policy 1, policy_version 315646 (0.0011) [2023-12-26 17:39:35,404][105620] Updated weights for policy 1, policy_version 315656 (0.0011) [2023-12-26 17:39:35,714][105692] Updated weights for policy 0, policy_version 315441 (0.0010) [2023-12-26 17:39:35,778][105692] Updated weights for policy 0, policy_version 315451 (0.0011) [2023-12-26 17:39:35,840][105692] Updated weights for policy 0, policy_version 315461 (0.0010) [2023-12-26 17:39:36,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 161587200. Throughput: 0: 9946.2, 1: 9916.4. Samples: 161577352. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 17:39:36,062][104569] Avg episode reward: [(0, '8173.500'), (1, '8912.834')] [2023-12-26 17:39:36,109][105620] Updated weights for policy 1, policy_version 315666 (0.0007) [2023-12-26 17:39:36,177][105620] Updated weights for policy 1, policy_version 315676 (0.0008) [2023-12-26 17:39:36,242][105620] Updated weights for policy 1, policy_version 315686 (0.0009) [2023-12-26 17:39:36,593][105692] Updated weights for policy 0, policy_version 315471 (0.0010) [2023-12-26 17:39:36,661][105692] Updated weights for policy 0, policy_version 315481 (0.0010) [2023-12-26 17:39:36,728][105692] Updated weights for policy 0, policy_version 315491 (0.0009) [2023-12-26 17:39:36,984][105620] Updated weights for policy 1, policy_version 315696 (0.0009) [2023-12-26 17:39:37,036][105620] Updated weights for policy 1, policy_version 315706 (0.0009) [2023-12-26 17:39:37,091][105620] Updated weights for policy 1, policy_version 315716 (0.0008) [2023-12-26 17:39:37,442][105692] Updated weights for policy 0, policy_version 315501 (0.0009) [2023-12-26 17:39:37,503][105692] Updated weights for policy 0, policy_version 315511 (0.0006) [2023-12-26 17:39:37,572][105692] Updated weights for policy 0, policy_version 315521 (0.0008) [2023-12-26 17:39:37,957][105620] Updated weights for policy 1, policy_version 315727 (0.0009) [2023-12-26 17:39:38,014][105620] Updated weights for policy 1, policy_version 315737 (0.0008) [2023-12-26 17:39:38,070][105620] Updated weights for policy 1, policy_version 315747 (0.0008) [2023-12-26 17:39:38,155][105692] Updated weights for policy 0, policy_version 315531 (0.0007) [2023-12-26 17:39:38,218][105692] Updated weights for policy 0, policy_version 315541 (0.0008) [2023-12-26 17:39:38,279][105692] Updated weights for policy 0, policy_version 315551 (0.0010) [2023-12-26 17:39:38,842][105692] Updated weights for policy 0, policy_version 315561 (0.0010) [2023-12-26 17:39:38,904][105692] Updated weights for policy 0, policy_version 315571 (0.0009) [2023-12-26 17:39:38,922][105620] Updated weights for policy 1, policy_version 315757 (0.0007) [2023-12-26 17:39:38,960][105692] Updated weights for policy 0, policy_version 315581 (0.0010) [2023-12-26 17:39:38,978][105620] Updated weights for policy 1, policy_version 315767 (0.0005) [2023-12-26 17:39:39,022][105692] Updated weights for policy 0, policy_version 315591 (0.0009) [2023-12-26 17:39:39,033][105620] Updated weights for policy 1, policy_version 315777 (0.0008) [2023-12-26 17:39:39,723][105692] Updated weights for policy 0, policy_version 315601 (0.0010) [2023-12-26 17:39:39,783][105692] Updated weights for policy 0, policy_version 315611 (0.0011) [2023-12-26 17:39:39,786][105620] Updated weights for policy 1, policy_version 315787 (0.0007) [2023-12-26 17:39:39,848][105692] Updated weights for policy 0, policy_version 315621 (0.0011) [2023-12-26 17:39:39,848][105620] Updated weights for policy 1, policy_version 315797 (0.0008) [2023-12-26 17:39:39,907][105620] Updated weights for policy 1, policy_version 315807 (0.0009) [2023-12-26 17:39:40,604][105620] Updated weights for policy 1, policy_version 315817 (0.0007) [2023-12-26 17:39:40,612][105692] Updated weights for policy 0, policy_version 315631 (0.0011) [2023-12-26 17:39:40,663][105620] Updated weights for policy 1, policy_version 315827 (0.0006) [2023-12-26 17:39:40,673][105692] Updated weights for policy 0, policy_version 315641 (0.0010) [2023-12-26 17:39:40,727][105620] Updated weights for policy 1, policy_version 315837 (0.0007) [2023-12-26 17:39:40,729][105692] Updated weights for policy 0, policy_version 315651 (0.0008) [2023-12-26 17:39:40,796][105620] Updated weights for policy 1, policy_version 315847 (0.0009) [2023-12-26 17:39:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 161685504. Throughput: 0: 10005.1, 1: 9828.9. Samples: 161691112. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 17:39:41,062][104569] Avg episode reward: [(0, '8265.214'), (1, '9001.753')] [2023-12-26 17:39:41,484][105620] Updated weights for policy 1, policy_version 315857 (0.0006) [2023-12-26 17:39:41,547][105692] Updated weights for policy 0, policy_version 315661 (0.0006) [2023-12-26 17:39:41,548][105620] Updated weights for policy 1, policy_version 315867 (0.0005) [2023-12-26 17:39:41,613][105692] Updated weights for policy 0, policy_version 315671 (0.0007) [2023-12-26 17:39:41,618][105620] Updated weights for policy 1, policy_version 315877 (0.0007) [2023-12-26 17:39:41,680][105692] Updated weights for policy 0, policy_version 315681 (0.0008) [2023-12-26 17:39:42,264][105620] Updated weights for policy 1, policy_version 315887 (0.0009) [2023-12-26 17:39:42,328][105620] Updated weights for policy 1, policy_version 315897 (0.0009) [2023-12-26 17:39:42,396][105620] Updated weights for policy 1, policy_version 315907 (0.0008) [2023-12-26 17:39:42,443][105692] Updated weights for policy 0, policy_version 315691 (0.0009) [2023-12-26 17:39:42,511][105692] Updated weights for policy 0, policy_version 315701 (0.0011) [2023-12-26 17:39:42,573][105692] Updated weights for policy 0, policy_version 315711 (0.0011) [2023-12-26 17:39:43,045][105620] Updated weights for policy 1, policy_version 315917 (0.0007) [2023-12-26 17:39:43,112][105620] Updated weights for policy 1, policy_version 315927 (0.0005) [2023-12-26 17:39:43,171][105620] Updated weights for policy 1, policy_version 315937 (0.0007) [2023-12-26 17:39:43,272][105692] Updated weights for policy 0, policy_version 315721 (0.0011) [2023-12-26 17:39:43,330][105692] Updated weights for policy 0, policy_version 315731 (0.0010) [2023-12-26 17:39:43,384][105692] Updated weights for policy 0, policy_version 315741 (0.0009) [2023-12-26 17:39:43,435][105692] Updated weights for policy 0, policy_version 315751 (0.0009) [2023-12-26 17:39:43,867][105620] Updated weights for policy 1, policy_version 315948 (0.0008) [2023-12-26 17:39:43,925][105620] Updated weights for policy 1, policy_version 315958 (0.0010) [2023-12-26 17:39:43,978][105620] Updated weights for policy 1, policy_version 315968 (0.0010) [2023-12-26 17:39:44,070][105692] Updated weights for policy 0, policy_version 315761 (0.0008) [2023-12-26 17:39:44,126][105692] Updated weights for policy 0, policy_version 315771 (0.0005) [2023-12-26 17:39:44,175][105692] Updated weights for policy 0, policy_version 315781 (0.0005) [2023-12-26 17:39:44,779][105620] Updated weights for policy 1, policy_version 315978 (0.0009) [2023-12-26 17:39:44,786][105692] Updated weights for policy 0, policy_version 315791 (0.0007) [2023-12-26 17:39:44,835][105620] Updated weights for policy 1, policy_version 315988 (0.0009) [2023-12-26 17:39:44,847][105692] Updated weights for policy 0, policy_version 315801 (0.0007) [2023-12-26 17:39:44,885][105620] Updated weights for policy 1, policy_version 315998 (0.0008) [2023-12-26 17:39:44,896][105692] Updated weights for policy 0, policy_version 315811 (0.0006) [2023-12-26 17:39:44,934][105620] Updated weights for policy 1, policy_version 316008 (0.0009) [2023-12-26 17:39:45,611][105692] Updated weights for policy 0, policy_version 315821 (0.0005) [2023-12-26 17:39:45,641][105585] KL-divergence is very high: 171.2708 [2023-12-26 17:39:45,663][105692] Updated weights for policy 0, policy_version 315831 (0.0008) [2023-12-26 17:39:45,681][105585] KL-divergence is very high: 290.0408 [2023-12-26 17:39:45,718][105692] Updated weights for policy 0, policy_version 315841 (0.0010) [2023-12-26 17:39:45,725][105620] Updated weights for policy 1, policy_version 316018 (0.0010) [2023-12-26 17:39:45,729][105585] KL-divergence is very high: 285.6149 [2023-12-26 17:39:45,784][105620] Updated weights for policy 1, policy_version 316028 (0.0007) [2023-12-26 17:39:45,838][105620] Updated weights for policy 1, policy_version 316038 (0.0008) [2023-12-26 17:39:46,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 161783808. Throughput: 0: 9894.9, 1: 9817.3. Samples: 161749076. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 17:39:46,063][104569] Avg episode reward: [(0, '8627.102'), (1, '9180.437')] [2023-12-26 17:39:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000315848_80871424.pth... [2023-12-26 17:39:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000316040_80912384.pth... [2023-12-26 17:39:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000314664_80568320.pth [2023-12-26 17:39:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000314888_80617472.pth [2023-12-26 17:39:46,414][105692] Updated weights for policy 0, policy_version 315851 (0.0009) [2023-12-26 17:39:46,464][105692] Updated weights for policy 0, policy_version 315861 (0.0010) [2023-12-26 17:39:46,511][105692] Updated weights for policy 0, policy_version 315871 (0.0008) [2023-12-26 17:39:46,641][105620] Updated weights for policy 1, policy_version 316048 (0.0009) [2023-12-26 17:39:46,692][105620] Updated weights for policy 1, policy_version 316058 (0.0009) [2023-12-26 17:39:46,749][105620] Updated weights for policy 1, policy_version 316068 (0.0009) [2023-12-26 17:39:47,246][105692] Updated weights for policy 0, policy_version 315881 (0.0008) [2023-12-26 17:39:47,300][105692] Updated weights for policy 0, policy_version 315891 (0.0005) [2023-12-26 17:39:47,358][105692] Updated weights for policy 0, policy_version 315901 (0.0007) [2023-12-26 17:39:47,421][105692] Updated weights for policy 0, policy_version 315911 (0.0005) [2023-12-26 17:39:47,479][105620] Updated weights for policy 1, policy_version 316078 (0.0009) [2023-12-26 17:39:47,533][105620] Updated weights for policy 1, policy_version 316088 (0.0012) [2023-12-26 17:39:47,580][105620] Updated weights for policy 1, policy_version 316098 (0.0009) [2023-12-26 17:39:48,051][105692] Updated weights for policy 0, policy_version 315921 (0.0009) [2023-12-26 17:39:48,098][105692] Updated weights for policy 0, policy_version 315931 (0.0008) [2023-12-26 17:39:48,101][105585] KL-divergence is very high: 104.3510 [2023-12-26 17:39:48,138][105585] KL-divergence is very high: 118.0211 [2023-12-26 17:39:48,145][105692] Updated weights for policy 0, policy_version 315941 (0.0009) [2023-12-26 17:39:48,338][105620] Updated weights for policy 1, policy_version 316108 (0.0008) [2023-12-26 17:39:48,398][105620] Updated weights for policy 1, policy_version 316118 (0.0006) [2023-12-26 17:39:48,459][105620] Updated weights for policy 1, policy_version 316128 (0.0006) [2023-12-26 17:39:48,894][105692] Updated weights for policy 0, policy_version 315951 (0.0009) [2023-12-26 17:39:48,913][105585] KL-divergence is very high: 121.2200 [2023-12-26 17:39:48,947][105692] Updated weights for policy 0, policy_version 315961 (0.0008) [2023-12-26 17:39:48,961][105585] KL-divergence is very high: 260.7780 [2023-12-26 17:39:49,005][105585] KL-divergence is very high: 1760.5852 [2023-12-26 17:39:49,008][105692] Updated weights for policy 0, policy_version 315971 (0.0007) [2023-12-26 17:39:49,021][105585] KL-divergence is very high: 156.6658 [2023-12-26 17:39:49,215][105620] Updated weights for policy 1, policy_version 316138 (0.0008) [2023-12-26 17:39:49,282][105620] Updated weights for policy 1, policy_version 316148 (0.0010) [2023-12-26 17:39:49,333][105620] Updated weights for policy 1, policy_version 316158 (0.0009) [2023-12-26 17:39:49,401][105620] Updated weights for policy 1, policy_version 316168 (0.0008) [2023-12-26 17:39:49,678][105585] KL-divergence is very high: 128.9221 [2023-12-26 17:39:49,688][105692] Updated weights for policy 0, policy_version 315981 (0.0009) [2023-12-26 17:39:49,696][105585] KL-divergence is very high: 548.3014 [2023-12-26 17:39:49,724][105585] KL-divergence is very high: 115.4662 [2023-12-26 17:39:49,743][105585] KL-divergence is very high: 533.3132 [2023-12-26 17:39:49,749][105692] Updated weights for policy 0, policy_version 315991 (0.0009) [2023-12-26 17:39:49,791][105585] KL-divergence is very high: 517.7590 [2023-12-26 17:39:49,811][105692] Updated weights for policy 0, policy_version 316001 (0.0009) [2023-12-26 17:39:49,843][105585] KL-divergence is very high: 491.5963 [2023-12-26 17:39:50,210][105620] Updated weights for policy 1, policy_version 316178 (0.0011) [2023-12-26 17:39:50,273][105620] Updated weights for policy 1, policy_version 316188 (0.0011) [2023-12-26 17:39:50,330][105620] Updated weights for policy 1, policy_version 316198 (0.0011) [2023-12-26 17:39:50,605][105692] Updated weights for policy 0, policy_version 316011 (0.0008) [2023-12-26 17:39:50,660][105692] Updated weights for policy 0, policy_version 316021 (0.0009) [2023-12-26 17:39:50,719][105692] Updated weights for policy 0, policy_version 316031 (0.0006) [2023-12-26 17:39:51,004][105620] Updated weights for policy 1, policy_version 316208 (0.0009) [2023-12-26 17:39:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 161873920. Throughput: 0: 9967.8, 1: 9721.9. Samples: 161865996. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 17:39:51,062][104569] Avg episode reward: [(0, '7991.999'), (1, '9096.995')] [2023-12-26 17:39:51,069][105620] Updated weights for policy 1, policy_version 316218 (0.0008) [2023-12-26 17:39:51,127][105620] Updated weights for policy 1, policy_version 316228 (0.0008) [2023-12-26 17:39:51,511][105692] Updated weights for policy 0, policy_version 316041 (0.0007) [2023-12-26 17:39:51,562][105692] Updated weights for policy 0, policy_version 316051 (0.0009) [2023-12-26 17:39:51,615][105692] Updated weights for policy 0, policy_version 316061 (0.0009) [2023-12-26 17:39:51,674][105692] Updated weights for policy 0, policy_version 316071 (0.0008) [2023-12-26 17:39:51,869][105620] Updated weights for policy 1, policy_version 316238 (0.0009) [2023-12-26 17:39:51,934][105620] Updated weights for policy 1, policy_version 316248 (0.0009) [2023-12-26 17:39:52,002][105620] Updated weights for policy 1, policy_version 316258 (0.0009) [2023-12-26 17:39:52,405][105692] Updated weights for policy 0, policy_version 316081 (0.0009) [2023-12-26 17:39:52,453][105692] Updated weights for policy 0, policy_version 316091 (0.0009) [2023-12-26 17:39:52,506][105585] KL-divergence is very high: 108.3789 [2023-12-26 17:39:52,507][105692] Updated weights for policy 0, policy_version 316101 (0.0009) [2023-12-26 17:39:52,694][105620] Updated weights for policy 1, policy_version 316268 (0.0008) [2023-12-26 17:39:52,749][105620] Updated weights for policy 1, policy_version 316278 (0.0005) [2023-12-26 17:39:52,818][105620] Updated weights for policy 1, policy_version 316288 (0.0006) [2023-12-26 17:39:53,294][105692] Updated weights for policy 0, policy_version 316111 (0.0009) [2023-12-26 17:39:53,354][105692] Updated weights for policy 0, policy_version 316121 (0.0008) [2023-12-26 17:39:53,421][105692] Updated weights for policy 0, policy_version 316131 (0.0006) [2023-12-26 17:39:53,422][105620] Updated weights for policy 1, policy_version 316298 (0.0010) [2023-12-26 17:39:53,481][105620] Updated weights for policy 1, policy_version 316308 (0.0011) [2023-12-26 17:39:53,529][105620] Updated weights for policy 1, policy_version 316318 (0.0010) [2023-12-26 17:39:53,581][105620] Updated weights for policy 1, policy_version 316328 (0.0010) [2023-12-26 17:39:53,995][105692] Updated weights for policy 0, policy_version 316141 (0.0006) [2023-12-26 17:39:54,044][105692] Updated weights for policy 0, policy_version 316151 (0.0009) [2023-12-26 17:39:54,091][105692] Updated weights for policy 0, policy_version 316161 (0.0008) [2023-12-26 17:39:54,222][105620] Updated weights for policy 1, policy_version 316338 (0.0005) [2023-12-26 17:39:54,286][105620] Updated weights for policy 1, policy_version 316348 (0.0005) [2023-12-26 17:39:54,343][105620] Updated weights for policy 1, policy_version 316358 (0.0006) [2023-12-26 17:39:54,867][105620] Updated weights for policy 1, policy_version 316368 (0.0008) [2023-12-26 17:39:54,924][105620] Updated weights for policy 1, policy_version 316378 (0.0008) [2023-12-26 17:39:54,938][105692] Updated weights for policy 0, policy_version 316171 (0.0008) [2023-12-26 17:39:54,986][105620] Updated weights for policy 1, policy_version 316388 (0.0007) [2023-12-26 17:39:55,011][105692] Updated weights for policy 0, policy_version 316181 (0.0007) [2023-12-26 17:39:55,067][105692] Updated weights for policy 0, policy_version 316191 (0.0008) [2023-12-26 17:39:55,727][105620] Updated weights for policy 1, policy_version 316398 (0.0010) [2023-12-26 17:39:55,746][105692] Updated weights for policy 0, policy_version 316201 (0.0005) [2023-12-26 17:39:55,789][105620] Updated weights for policy 1, policy_version 316408 (0.0010) [2023-12-26 17:39:55,800][105692] Updated weights for policy 0, policy_version 316211 (0.0008) [2023-12-26 17:39:55,847][105620] Updated weights for policy 1, policy_version 316418 (0.0010) [2023-12-26 17:39:55,857][105692] Updated weights for policy 0, policy_version 316221 (0.0006) [2023-12-26 17:39:55,918][105692] Updated weights for policy 0, policy_version 316231 (0.0007) [2023-12-26 17:39:56,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 161980416. Throughput: 0: 9924.0, 1: 9871.1. Samples: 161983540. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 17:39:56,062][104569] Avg episode reward: [(0, '7994.600'), (1, '9098.505')] [2023-12-26 17:39:56,518][105620] Updated weights for policy 1, policy_version 316428 (0.0009) [2023-12-26 17:39:56,579][105620] Updated weights for policy 1, policy_version 316438 (0.0005) [2023-12-26 17:39:56,635][105620] Updated weights for policy 1, policy_version 316448 (0.0006) [2023-12-26 17:39:56,732][105692] Updated weights for policy 0, policy_version 316241 (0.0009) [2023-12-26 17:39:56,780][105692] Updated weights for policy 0, policy_version 316251 (0.0009) [2023-12-26 17:39:56,835][105692] Updated weights for policy 0, policy_version 316261 (0.0009) [2023-12-26 17:39:57,249][105620] Updated weights for policy 1, policy_version 316458 (0.0006) [2023-12-26 17:39:57,302][105620] Updated weights for policy 1, policy_version 316468 (0.0009) [2023-12-26 17:39:57,356][105620] Updated weights for policy 1, policy_version 316478 (0.0009) [2023-12-26 17:39:57,530][105692] Updated weights for policy 0, policy_version 316271 (0.0007) [2023-12-26 17:39:57,579][105692] Updated weights for policy 0, policy_version 316281 (0.0007) [2023-12-26 17:39:57,632][105692] Updated weights for policy 0, policy_version 316293 (0.0010) [2023-12-26 17:39:58,011][105620] Updated weights for policy 1, policy_version 316489 (0.0009) [2023-12-26 17:39:58,072][105620] Updated weights for policy 1, policy_version 316499 (0.0006) [2023-12-26 17:39:58,125][105620] Updated weights for policy 1, policy_version 316509 (0.0008) [2023-12-26 17:39:58,177][105620] Updated weights for policy 1, policy_version 316519 (0.0007) [2023-12-26 17:39:58,475][105692] Updated weights for policy 0, policy_version 316303 (0.0009) [2023-12-26 17:39:58,536][105692] Updated weights for policy 0, policy_version 316313 (0.0009) [2023-12-26 17:39:58,588][105692] Updated weights for policy 0, policy_version 316323 (0.0009) [2023-12-26 17:39:59,016][105620] Updated weights for policy 1, policy_version 316529 (0.0009) [2023-12-26 17:39:59,076][105620] Updated weights for policy 1, policy_version 316539 (0.0009) [2023-12-26 17:39:59,138][105620] Updated weights for policy 1, policy_version 316549 (0.0008) [2023-12-26 17:39:59,431][105692] Updated weights for policy 0, policy_version 316333 (0.0008) [2023-12-26 17:39:59,478][105692] Updated weights for policy 0, policy_version 316343 (0.0008) [2023-12-26 17:39:59,522][105692] Updated weights for policy 0, policy_version 316353 (0.0008) [2023-12-26 17:39:59,782][105620] Updated weights for policy 1, policy_version 316559 (0.0007) [2023-12-26 17:39:59,844][105620] Updated weights for policy 1, policy_version 316569 (0.0007) [2023-12-26 17:39:59,906][105620] Updated weights for policy 1, policy_version 316579 (0.0008) [2023-12-26 17:40:00,245][105692] Updated weights for policy 0, policy_version 316363 (0.0008) [2023-12-26 17:40:00,300][105692] Updated weights for policy 0, policy_version 316373 (0.0009) [2023-12-26 17:40:00,362][105692] Updated weights for policy 0, policy_version 316383 (0.0006) [2023-12-26 17:40:00,596][105620] Updated weights for policy 1, policy_version 316589 (0.0009) [2023-12-26 17:40:00,649][105620] Updated weights for policy 1, policy_version 316599 (0.0010) [2023-12-26 17:40:00,703][105620] Updated weights for policy 1, policy_version 316609 (0.0010) [2023-12-26 17:40:00,933][105692] Updated weights for policy 0, policy_version 316393 (0.0006) [2023-12-26 17:40:00,985][105692] Updated weights for policy 0, policy_version 316403 (0.0005) [2023-12-26 17:40:01,043][105692] Updated weights for policy 0, policy_version 316413 (0.0007) [2023-12-26 17:40:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 162070528. Throughput: 0: 9875.9, 1: 9924.5. Samples: 162041708. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 17:40:01,062][104569] Avg episode reward: [(0, '8354.912'), (1, '9188.756')] [2023-12-26 17:40:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000316616_81059840.pth... [2023-12-26 17:40:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000315464_80764928.pth [2023-12-26 17:40:01,102][105692] Updated weights for policy 0, policy_version 316423 (0.0008) [2023-12-26 17:40:01,106][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000316424_81018880.pth... [2023-12-26 17:40:01,111][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000315272_80723968.pth [2023-12-26 17:40:01,438][105620] Updated weights for policy 1, policy_version 316619 (0.0009) [2023-12-26 17:40:01,492][105620] Updated weights for policy 1, policy_version 316629 (0.0005) [2023-12-26 17:40:01,547][105620] Updated weights for policy 1, policy_version 316639 (0.0005) [2023-12-26 17:40:01,752][105692] Updated weights for policy 0, policy_version 316433 (0.0008) [2023-12-26 17:40:01,804][105692] Updated weights for policy 0, policy_version 316443 (0.0008) [2023-12-26 17:40:01,861][105692] Updated weights for policy 0, policy_version 316453 (0.0008) [2023-12-26 17:40:02,182][105620] Updated weights for policy 1, policy_version 316649 (0.0006) [2023-12-26 17:40:02,228][105620] Updated weights for policy 1, policy_version 316659 (0.0007) [2023-12-26 17:40:02,280][105620] Updated weights for policy 1, policy_version 316669 (0.0009) [2023-12-26 17:40:02,331][105620] Updated weights for policy 1, policy_version 316679 (0.0008) [2023-12-26 17:40:02,617][105692] Updated weights for policy 0, policy_version 316463 (0.0010) [2023-12-26 17:40:02,673][105585] KL-divergence is very high: 165.7930 [2023-12-26 17:40:02,678][105692] Updated weights for policy 0, policy_version 316473 (0.0010) [2023-12-26 17:40:02,692][105585] KL-divergence is very high: 158.1814 [2023-12-26 17:40:02,722][105585] KL-divergence is very high: 166.7913 [2023-12-26 17:40:02,739][105692] Updated weights for policy 0, policy_version 316483 (0.0010) [2023-12-26 17:40:03,106][105620] Updated weights for policy 1, policy_version 316689 (0.0008) [2023-12-26 17:40:03,164][105620] Updated weights for policy 1, policy_version 316699 (0.0008) [2023-12-26 17:40:03,225][105620] Updated weights for policy 1, policy_version 316709 (0.0008) [2023-12-26 17:40:03,476][105692] Updated weights for policy 0, policy_version 316493 (0.0010) [2023-12-26 17:40:03,537][105692] Updated weights for policy 0, policy_version 316503 (0.0010) [2023-12-26 17:40:03,594][105692] Updated weights for policy 0, policy_version 316513 (0.0010) [2023-12-26 17:40:03,982][105620] Updated weights for policy 1, policy_version 316719 (0.0008) [2023-12-26 17:40:04,050][105620] Updated weights for policy 1, policy_version 316729 (0.0009) [2023-12-26 17:40:04,117][105620] Updated weights for policy 1, policy_version 316739 (0.0008) [2023-12-26 17:40:04,330][105692] Updated weights for policy 0, policy_version 316523 (0.0010) [2023-12-26 17:40:04,391][105692] Updated weights for policy 0, policy_version 316533 (0.0010) [2023-12-26 17:40:04,453][105692] Updated weights for policy 0, policy_version 316543 (0.0010) [2023-12-26 17:40:04,864][105620] Updated weights for policy 1, policy_version 316749 (0.0008) [2023-12-26 17:40:04,912][105620] Updated weights for policy 1, policy_version 316759 (0.0008) [2023-12-26 17:40:04,958][105620] Updated weights for policy 1, policy_version 316769 (0.0007) [2023-12-26 17:40:05,164][105692] Updated weights for policy 0, policy_version 316553 (0.0010) [2023-12-26 17:40:05,217][105692] Updated weights for policy 0, policy_version 316563 (0.0010) [2023-12-26 17:40:05,269][105692] Updated weights for policy 0, policy_version 316573 (0.0010) [2023-12-26 17:40:05,327][105692] Updated weights for policy 0, policy_version 316583 (0.0010) [2023-12-26 17:40:05,600][105620] Updated weights for policy 1, policy_version 316779 (0.0006) [2023-12-26 17:40:05,649][105620] Updated weights for policy 1, policy_version 316789 (0.0008) [2023-12-26 17:40:05,707][105620] Updated weights for policy 1, policy_version 316799 (0.0007) [2023-12-26 17:40:06,049][105692] Updated weights for policy 0, policy_version 316593 (0.0011) [2023-12-26 17:40:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 162168832. Throughput: 0: 10007.7, 1: 9803.7. Samples: 162158892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:40:06,063][104569] Avg episode reward: [(0, '7993.736'), (1, '9272.133')] [2023-12-26 17:40:06,108][105692] Updated weights for policy 0, policy_version 316603 (0.0011) [2023-12-26 17:40:06,179][105692] Updated weights for policy 0, policy_version 316613 (0.0008) [2023-12-26 17:40:06,471][105620] Updated weights for policy 1, policy_version 316809 (0.0009) [2023-12-26 17:40:06,534][105620] Updated weights for policy 1, policy_version 316819 (0.0011) [2023-12-26 17:40:06,597][105620] Updated weights for policy 1, policy_version 316829 (0.0011) [2023-12-26 17:40:06,658][105620] Updated weights for policy 1, policy_version 316839 (0.0011) [2023-12-26 17:40:06,932][105692] Updated weights for policy 0, policy_version 316623 (0.0010) [2023-12-26 17:40:06,994][105692] Updated weights for policy 0, policy_version 316634 (0.0010) [2023-12-26 17:40:07,058][105692] Updated weights for policy 0, policy_version 316644 (0.0009) [2023-12-26 17:40:07,289][105620] Updated weights for policy 1, policy_version 316849 (0.0006) [2023-12-26 17:40:07,349][105620] Updated weights for policy 1, policy_version 316859 (0.0005) [2023-12-26 17:40:07,411][105620] Updated weights for policy 1, policy_version 316869 (0.0010) [2023-12-26 17:40:07,891][105692] Updated weights for policy 0, policy_version 316654 (0.0008) [2023-12-26 17:40:07,957][105692] Updated weights for policy 0, policy_version 316664 (0.0008) [2023-12-26 17:40:08,018][105692] Updated weights for policy 0, policy_version 316674 (0.0008) [2023-12-26 17:40:08,088][105620] Updated weights for policy 1, policy_version 316879 (0.0011) [2023-12-26 17:40:08,147][105620] Updated weights for policy 1, policy_version 316889 (0.0011) [2023-12-26 17:40:08,206][105620] Updated weights for policy 1, policy_version 316899 (0.0011) [2023-12-26 17:40:08,706][105692] Updated weights for policy 0, policy_version 316684 (0.0007) [2023-12-26 17:40:08,758][105692] Updated weights for policy 0, policy_version 316694 (0.0008) [2023-12-26 17:40:08,815][105692] Updated weights for policy 0, policy_version 316704 (0.0008) [2023-12-26 17:40:08,940][105620] Updated weights for policy 1, policy_version 316909 (0.0011) [2023-12-26 17:40:08,998][105620] Updated weights for policy 1, policy_version 316919 (0.0010) [2023-12-26 17:40:09,060][105620] Updated weights for policy 1, policy_version 316929 (0.0010) [2023-12-26 17:40:09,516][105692] Updated weights for policy 0, policy_version 316714 (0.0007) [2023-12-26 17:40:09,581][105692] Updated weights for policy 0, policy_version 316724 (0.0009) [2023-12-26 17:40:09,641][105692] Updated weights for policy 0, policy_version 316734 (0.0009) [2023-12-26 17:40:09,701][105692] Updated weights for policy 0, policy_version 316744 (0.0009) [2023-12-26 17:40:09,791][105620] Updated weights for policy 1, policy_version 316939 (0.0010) [2023-12-26 17:40:09,850][105620] Updated weights for policy 1, policy_version 316949 (0.0009) [2023-12-26 17:40:09,909][105620] Updated weights for policy 1, policy_version 316959 (0.0009) [2023-12-26 17:40:10,483][105692] Updated weights for policy 0, policy_version 316754 (0.0008) [2023-12-26 17:40:10,549][105692] Updated weights for policy 0, policy_version 316764 (0.0010) [2023-12-26 17:40:10,617][105692] Updated weights for policy 0, policy_version 316774 (0.0010) [2023-12-26 17:40:10,647][105620] Updated weights for policy 1, policy_version 316969 (0.0008) [2023-12-26 17:40:10,711][105620] Updated weights for policy 1, policy_version 316979 (0.0005) [2023-12-26 17:40:10,762][105620] Updated weights for policy 1, policy_version 316989 (0.0005) [2023-12-26 17:40:10,819][105620] Updated weights for policy 1, policy_version 316999 (0.0005) [2023-12-26 17:40:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 162267136. Throughput: 0: 9928.7, 1: 9876.2. Samples: 162274452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:40:11,062][104569] Avg episode reward: [(0, '8001.594'), (1, '7635.264')] [2023-12-26 17:40:11,456][105620] Updated weights for policy 1, policy_version 317009 (0.0008) [2023-12-26 17:40:11,487][105692] Updated weights for policy 0, policy_version 316784 (0.0006) [2023-12-26 17:40:11,529][105620] Updated weights for policy 1, policy_version 317019 (0.0008) [2023-12-26 17:40:11,554][105692] Updated weights for policy 0, policy_version 316794 (0.0006) [2023-12-26 17:40:11,598][105620] Updated weights for policy 1, policy_version 317029 (0.0008) [2023-12-26 17:40:11,614][105692] Updated weights for policy 0, policy_version 316804 (0.0008) [2023-12-26 17:40:12,299][105692] Updated weights for policy 0, policy_version 316814 (0.0008) [2023-12-26 17:40:12,369][105692] Updated weights for policy 0, policy_version 316824 (0.0008) [2023-12-26 17:40:12,377][105620] Updated weights for policy 1, policy_version 317039 (0.0008) [2023-12-26 17:40:12,432][105692] Updated weights for policy 0, policy_version 316834 (0.0008) [2023-12-26 17:40:12,444][105620] Updated weights for policy 1, policy_version 317049 (0.0006) [2023-12-26 17:40:12,512][105620] Updated weights for policy 1, policy_version 317059 (0.0005) [2023-12-26 17:40:13,128][105620] Updated weights for policy 1, policy_version 317069 (0.0005) [2023-12-26 17:40:13,149][105692] Updated weights for policy 0, policy_version 316844 (0.0009) [2023-12-26 17:40:13,196][105620] Updated weights for policy 1, policy_version 317079 (0.0005) [2023-12-26 17:40:13,196][105692] Updated weights for policy 0, policy_version 316854 (0.0009) [2023-12-26 17:40:13,244][105692] Updated weights for policy 0, policy_version 316864 (0.0010) [2023-12-26 17:40:13,261][105620] Updated weights for policy 1, policy_version 317089 (0.0005) [2023-12-26 17:40:13,818][105620] Updated weights for policy 1, policy_version 317099 (0.0006) [2023-12-26 17:40:13,876][105620] Updated weights for policy 1, policy_version 317109 (0.0007) [2023-12-26 17:40:13,926][105620] Updated weights for policy 1, policy_version 317119 (0.0008) [2023-12-26 17:40:13,938][105692] Updated weights for policy 0, policy_version 316874 (0.0009) [2023-12-26 17:40:13,991][105692] Updated weights for policy 0, policy_version 316885 (0.0009) [2023-12-26 17:40:14,043][105692] Updated weights for policy 0, policy_version 316896 (0.0010) [2023-12-26 17:40:14,571][105620] Updated weights for policy 1, policy_version 317129 (0.0007) [2023-12-26 17:40:14,628][105620] Updated weights for policy 1, policy_version 317139 (0.0009) [2023-12-26 17:40:14,685][105620] Updated weights for policy 1, policy_version 317149 (0.0008) [2023-12-26 17:40:14,736][105620] Updated weights for policy 1, policy_version 317159 (0.0007) [2023-12-26 17:40:14,833][105692] Updated weights for policy 0, policy_version 316906 (0.0009) [2023-12-26 17:40:14,889][105692] Updated weights for policy 0, policy_version 316916 (0.0008) [2023-12-26 17:40:14,945][105692] Updated weights for policy 0, policy_version 316926 (0.0008) [2023-12-26 17:40:14,997][105692] Updated weights for policy 0, policy_version 316936 (0.0008) [2023-12-26 17:40:15,493][105620] Updated weights for policy 1, policy_version 317169 (0.0010) [2023-12-26 17:40:15,538][105620] Updated weights for policy 1, policy_version 317179 (0.0010) [2023-12-26 17:40:15,592][105620] Updated weights for policy 1, policy_version 317189 (0.0008) [2023-12-26 17:40:15,752][105692] Updated weights for policy 0, policy_version 316946 (0.0010) [2023-12-26 17:40:15,805][105692] Updated weights for policy 0, policy_version 316957 (0.0010) [2023-12-26 17:40:15,858][105692] Updated weights for policy 0, policy_version 316967 (0.0009) [2023-12-26 17:40:16,067][104569] Fps is (10 sec: 19650.0, 60 sec: 19795.5, 300 sec: 19632.7). Total num frames: 162365440. Throughput: 0: 9733.9, 1: 9832.0. Samples: 162332556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:40:16,070][104569] Avg episode reward: [(0, '8544.996'), (1, '6671.303')] [2023-12-26 17:40:16,078][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000316968_81158144.pth... [2023-12-26 17:40:16,078][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000317192_81207296.pth... [2023-12-26 17:40:16,083][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000315848_80871424.pth [2023-12-26 17:40:16,083][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000316040_80912384.pth [2023-12-26 17:40:16,239][105620] Updated weights for policy 1, policy_version 317199 (0.0009) [2023-12-26 17:40:16,297][105620] Updated weights for policy 1, policy_version 317209 (0.0010) [2023-12-26 17:40:16,367][105620] Updated weights for policy 1, policy_version 317219 (0.0005) [2023-12-26 17:40:16,572][105692] Updated weights for policy 0, policy_version 316977 (0.0005) [2023-12-26 17:40:16,638][105692] Updated weights for policy 0, policy_version 316987 (0.0005) [2023-12-26 17:40:16,702][105692] Updated weights for policy 0, policy_version 316997 (0.0006) [2023-12-26 17:40:17,059][105620] Updated weights for policy 1, policy_version 317229 (0.0008) [2023-12-26 17:40:17,104][105620] Updated weights for policy 1, policy_version 317239 (0.0010) [2023-12-26 17:40:17,163][105620] Updated weights for policy 1, policy_version 317249 (0.0010) [2023-12-26 17:40:17,183][105692] Updated weights for policy 0, policy_version 317007 (0.0005) [2023-12-26 17:40:17,245][105692] Updated weights for policy 0, policy_version 317017 (0.0006) [2023-12-26 17:40:17,310][105692] Updated weights for policy 0, policy_version 317027 (0.0010) [2023-12-26 17:40:17,845][105620] Updated weights for policy 1, policy_version 317259 (0.0011) [2023-12-26 17:40:17,902][105692] Updated weights for policy 0, policy_version 317037 (0.0008) [2023-12-26 17:40:17,908][105620] Updated weights for policy 1, policy_version 317269 (0.0010) [2023-12-26 17:40:17,961][105692] Updated weights for policy 0, policy_version 317047 (0.0007) [2023-12-26 17:40:17,968][105620] Updated weights for policy 1, policy_version 317279 (0.0006) [2023-12-26 17:40:18,016][105692] Updated weights for policy 0, policy_version 317057 (0.0009) [2023-12-26 17:40:18,592][105620] Updated weights for policy 1, policy_version 317289 (0.0007) [2023-12-26 17:40:18,615][105692] Updated weights for policy 0, policy_version 317067 (0.0007) [2023-12-26 17:40:18,644][105620] Updated weights for policy 1, policy_version 317299 (0.0006) [2023-12-26 17:40:18,676][105692] Updated weights for policy 0, policy_version 317077 (0.0010) [2023-12-26 17:40:18,693][105620] Updated weights for policy 1, policy_version 317309 (0.0010) [2023-12-26 17:40:18,728][105692] Updated weights for policy 0, policy_version 317087 (0.0009) [2023-12-26 17:40:18,746][105620] Updated weights for policy 1, policy_version 317319 (0.0011) [2023-12-26 17:40:19,401][105692] Updated weights for policy 0, policy_version 317097 (0.0009) [2023-12-26 17:40:19,461][105692] Updated weights for policy 0, policy_version 317107 (0.0011) [2023-12-26 17:40:19,508][105620] Updated weights for policy 1, policy_version 317329 (0.0011) [2023-12-26 17:40:19,525][105692] Updated weights for policy 0, policy_version 317117 (0.0010) [2023-12-26 17:40:19,553][105620] Updated weights for policy 1, policy_version 317339 (0.0009) [2023-12-26 17:40:19,582][105692] Updated weights for policy 0, policy_version 317127 (0.0010) [2023-12-26 17:40:19,606][105620] Updated weights for policy 1, policy_version 317349 (0.0011) [2023-12-26 17:40:20,302][105692] Updated weights for policy 0, policy_version 317137 (0.0011) [2023-12-26 17:40:20,364][105692] Updated weights for policy 0, policy_version 317147 (0.0010) [2023-12-26 17:40:20,385][105620] Updated weights for policy 1, policy_version 317359 (0.0011) [2023-12-26 17:40:20,425][105692] Updated weights for policy 0, policy_version 317157 (0.0006) [2023-12-26 17:40:20,447][105620] Updated weights for policy 1, policy_version 317369 (0.0009) [2023-12-26 17:40:20,496][105620] Updated weights for policy 1, policy_version 317379 (0.0008) [2023-12-26 17:40:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 162463744. Throughput: 0: 9722.7, 1: 9801.6. Samples: 162455944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:40:21,062][104569] Avg episode reward: [(0, '8904.004'), (1, '7660.402')] [2023-12-26 17:40:21,147][105692] Updated weights for policy 0, policy_version 317167 (0.0009) [2023-12-26 17:40:21,214][105692] Updated weights for policy 0, policy_version 317177 (0.0008) [2023-12-26 17:40:21,246][105620] Updated weights for policy 1, policy_version 317389 (0.0007) [2023-12-26 17:40:21,273][105692] Updated weights for policy 0, policy_version 317187 (0.0008) [2023-12-26 17:40:21,308][105620] Updated weights for policy 1, policy_version 317399 (0.0008) [2023-12-26 17:40:21,378][105620] Updated weights for policy 1, policy_version 317409 (0.0008) [2023-12-26 17:40:22,086][105620] Updated weights for policy 1, policy_version 317419 (0.0008) [2023-12-26 17:40:22,088][105692] Updated weights for policy 0, policy_version 317197 (0.0008) [2023-12-26 17:40:22,140][105620] Updated weights for policy 1, policy_version 317429 (0.0008) [2023-12-26 17:40:22,150][105692] Updated weights for policy 0, policy_version 317207 (0.0007) [2023-12-26 17:40:22,189][105620] Updated weights for policy 1, policy_version 317439 (0.0006) [2023-12-26 17:40:22,203][105692] Updated weights for policy 0, policy_version 317217 (0.0008) [2023-12-26 17:40:22,978][105692] Updated weights for policy 0, policy_version 317227 (0.0008) [2023-12-26 17:40:22,991][105620] Updated weights for policy 1, policy_version 317449 (0.0007) [2023-12-26 17:40:23,033][105692] Updated weights for policy 0, policy_version 317237 (0.0006) [2023-12-26 17:40:23,052][105620] Updated weights for policy 1, policy_version 317459 (0.0009) [2023-12-26 17:40:23,095][105692] Updated weights for policy 0, policy_version 317247 (0.0005) [2023-12-26 17:40:23,110][105620] Updated weights for policy 1, policy_version 317469 (0.0008) [2023-12-26 17:40:23,168][105620] Updated weights for policy 1, policy_version 317479 (0.0009) [2023-12-26 17:40:23,765][105692] Updated weights for policy 0, policy_version 317257 (0.0008) [2023-12-26 17:40:23,836][105692] Updated weights for policy 0, policy_version 317267 (0.0005) [2023-12-26 17:40:23,845][105586] KL-divergence is very high: 125.4228 [2023-12-26 17:40:23,850][105620] Updated weights for policy 1, policy_version 317489 (0.0006) [2023-12-26 17:40:23,898][105692] Updated weights for policy 0, policy_version 317277 (0.0008) [2023-12-26 17:40:23,902][105620] Updated weights for policy 1, policy_version 317499 (0.0005) [2023-12-26 17:40:23,917][105586] KL-divergence is very high: 216.4561 [2023-12-26 17:40:23,922][105586] KL-divergence is very high: 191.0447 [2023-12-26 17:40:23,927][105586] KL-divergence is very high: 147.4726 [2023-12-26 17:40:23,933][105586] KL-divergence is very high: 140.8657 [2023-12-26 17:40:23,939][105586] KL-divergence is very high: 262.3891 [2023-12-26 17:40:23,950][105692] Updated weights for policy 0, policy_version 317287 (0.0009) [2023-12-26 17:40:23,953][105620] Updated weights for policy 1, policy_version 317509 (0.0005) [2023-12-26 17:40:23,953][105586] KL-divergence is very high: 110.9305 [2023-12-26 17:40:23,959][105586] KL-divergence is very high: 248.7116 [2023-12-26 17:40:23,965][105586] KL-divergence is very high: 151.0169 [2023-12-26 17:40:24,532][105586] KL-divergence is very high: 103.4808 [2023-12-26 17:40:24,545][105586] KL-divergence is very high: 103.7214 [2023-12-26 17:40:24,577][105620] Updated weights for policy 1, policy_version 317519 (0.0006) [2023-12-26 17:40:24,609][105586] KL-divergence is very high: 122.7587 [2023-12-26 17:40:24,638][105620] Updated weights for policy 1, policy_version 317529 (0.0009) [2023-12-26 17:40:24,686][105586] KL-divergence is very high: 104.2606 [2023-12-26 17:40:24,690][105620] Updated weights for policy 1, policy_version 317539 (0.0006) [2023-12-26 17:40:24,693][105692] Updated weights for policy 0, policy_version 317297 (0.0008) [2023-12-26 17:40:24,748][105692] Updated weights for policy 0, policy_version 317307 (0.0009) [2023-12-26 17:40:24,798][105692] Updated weights for policy 0, policy_version 317318 (0.0009) [2023-12-26 17:40:25,287][105620] Updated weights for policy 1, policy_version 317549 (0.0005) [2023-12-26 17:40:25,337][105620] Updated weights for policy 1, policy_version 317559 (0.0007) [2023-12-26 17:40:25,343][105586] KL-divergence is very high: 107.2979 [2023-12-26 17:40:25,398][105620] Updated weights for policy 1, policy_version 317569 (0.0010) [2023-12-26 17:40:25,636][105692] Updated weights for policy 0, policy_version 317328 (0.0008) [2023-12-26 17:40:25,687][105692] Updated weights for policy 0, policy_version 317338 (0.0008) [2023-12-26 17:40:25,732][105692] Updated weights for policy 0, policy_version 317348 (0.0008) [2023-12-26 17:40:26,062][104569] Fps is (10 sec: 19671.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 162562048. Throughput: 0: 9647.5, 1: 9920.6. Samples: 162571676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:40:26,062][104569] Avg episode reward: [(0, '8542.857'), (1, '1959.017')] [2023-12-26 17:40:26,109][105620] Updated weights for policy 1, policy_version 317579 (0.0010) [2023-12-26 17:40:26,131][105586] KL-divergence is very high: 103.6610 [2023-12-26 17:40:26,136][105586] KL-divergence is very high: 129.0559 [2023-12-26 17:40:26,142][105586] KL-divergence is very high: 121.0320 [2023-12-26 17:40:26,158][105586] KL-divergence is very high: 104.1316 [2023-12-26 17:40:26,164][105620] Updated weights for policy 1, policy_version 317589 (0.0010) [2023-12-26 17:40:26,216][105620] Updated weights for policy 1, policy_version 317599 (0.0007) [2023-12-26 17:40:26,235][105586] KL-divergence is very high: 122.3219 [2023-12-26 17:40:26,240][105586] KL-divergence is very high: 119.7927 [2023-12-26 17:40:26,245][105586] KL-divergence is very high: 108.7793 [2023-12-26 17:40:26,250][105586] KL-divergence is very high: 105.6400 [2023-12-26 17:40:26,586][105692] Updated weights for policy 0, policy_version 317358 (0.0008) [2023-12-26 17:40:26,637][105692] Updated weights for policy 0, policy_version 317368 (0.0007) [2023-12-26 17:40:26,695][105692] Updated weights for policy 0, policy_version 317378 (0.0008) [2023-12-26 17:40:26,819][105620] Updated weights for policy 1, policy_version 317609 (0.0005) [2023-12-26 17:40:26,877][105620] Updated weights for policy 1, policy_version 317619 (0.0005) [2023-12-26 17:40:26,937][105620] Updated weights for policy 1, policy_version 317629 (0.0005) [2023-12-26 17:40:26,988][105620] Updated weights for policy 1, policy_version 317639 (0.0005) [2023-12-26 17:40:27,468][105692] Updated weights for policy 0, policy_version 317388 (0.0007) [2023-12-26 17:40:27,506][105586] KL-divergence is very high: 101.6986 [2023-12-26 17:40:27,511][105620] Updated weights for policy 1, policy_version 317649 (0.0010) [2023-12-26 17:40:27,512][105586] KL-divergence is very high: 162.8537 [2023-12-26 17:40:27,524][105586] KL-divergence is very high: 105.7846 [2023-12-26 17:40:27,526][105692] Updated weights for policy 0, policy_version 317398 (0.0009) [2023-12-26 17:40:27,536][105586] KL-divergence is very high: 107.3589 [2023-12-26 17:40:27,542][105586] KL-divergence is very high: 104.8539 [2023-12-26 17:40:27,554][105586] KL-divergence is very high: 135.2527 [2023-12-26 17:40:27,559][105586] KL-divergence is very high: 162.6089 [2023-12-26 17:40:27,572][105620] Updated weights for policy 1, policy_version 317659 (0.0009) [2023-12-26 17:40:27,586][105692] Updated weights for policy 0, policy_version 317408 (0.0006) [2023-12-26 17:40:27,623][105620] Updated weights for policy 1, policy_version 317669 (0.0010) [2023-12-26 17:40:28,295][105692] Updated weights for policy 0, policy_version 317418 (0.0006) [2023-12-26 17:40:28,349][105692] Updated weights for policy 0, policy_version 317428 (0.0007) [2023-12-26 17:40:28,362][105620] Updated weights for policy 1, policy_version 317679 (0.0011) [2023-12-26 17:40:28,412][105692] Updated weights for policy 0, policy_version 317438 (0.0008) [2023-12-26 17:40:28,423][105620] Updated weights for policy 1, policy_version 317689 (0.0009) [2023-12-26 17:40:28,470][105692] Updated weights for policy 0, policy_version 317448 (0.0008) [2023-12-26 17:40:28,480][105620] Updated weights for policy 1, policy_version 317699 (0.0005) [2023-12-26 17:40:29,130][105620] Updated weights for policy 1, policy_version 317709 (0.0008) [2023-12-26 17:40:29,174][105692] Updated weights for policy 0, policy_version 317458 (0.0010) [2023-12-26 17:40:29,174][105620] Updated weights for policy 1, policy_version 317719 (0.0010) [2023-12-26 17:40:29,222][105620] Updated weights for policy 1, policy_version 317729 (0.0010) [2023-12-26 17:40:29,236][105692] Updated weights for policy 0, policy_version 317468 (0.0008) [2023-12-26 17:40:29,303][105692] Updated weights for policy 0, policy_version 317478 (0.0011) [2023-12-26 17:40:29,985][105692] Updated weights for policy 0, policy_version 317488 (0.0011) [2023-12-26 17:40:30,005][105620] Updated weights for policy 1, policy_version 317739 (0.0009) [2023-12-26 17:40:30,042][105692] Updated weights for policy 0, policy_version 317498 (0.0011) [2023-12-26 17:40:30,064][105620] Updated weights for policy 1, policy_version 317749 (0.0010) [2023-12-26 17:40:30,096][105692] Updated weights for policy 0, policy_version 317508 (0.0011) [2023-12-26 17:40:30,122][105620] Updated weights for policy 1, policy_version 317759 (0.0010) [2023-12-26 17:40:30,758][105692] Updated weights for policy 0, policy_version 317518 (0.0007) [2023-12-26 17:40:30,808][105692] Updated weights for policy 0, policy_version 317528 (0.0005) [2023-12-26 17:40:30,867][105692] Updated weights for policy 0, policy_version 317538 (0.0005) [2023-12-26 17:40:30,868][105620] Updated weights for policy 1, policy_version 317769 (0.0010) [2023-12-26 17:40:30,928][105620] Updated weights for policy 1, policy_version 317779 (0.0010) [2023-12-26 17:40:30,996][105620] Updated weights for policy 1, policy_version 317789 (0.0010) [2023-12-26 17:40:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 162660352. Throughput: 0: 9645.0, 1: 9961.8. Samples: 162631380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:40:31,063][104569] Avg episode reward: [(0, '7819.596'), (1, '2643.893')] [2023-12-26 17:40:31,064][105620] Updated weights for policy 1, policy_version 317799 (0.0010) [2023-12-26 17:40:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000317544_81305600.pth... [2023-12-26 17:40:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000317800_81362944.pth... [2023-12-26 17:40:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000316616_81059840.pth [2023-12-26 17:40:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000316424_81018880.pth [2023-12-26 17:40:31,465][105692] Updated weights for policy 0, policy_version 317548 (0.0007) [2023-12-26 17:40:31,525][105692] Updated weights for policy 0, policy_version 317558 (0.0009) [2023-12-26 17:40:31,593][105692] Updated weights for policy 0, policy_version 317568 (0.0010) [2023-12-26 17:40:31,737][105620] Updated weights for policy 1, policy_version 317809 (0.0008) [2023-12-26 17:40:31,812][105620] Updated weights for policy 1, policy_version 317819 (0.0009) [2023-12-26 17:40:31,886][105620] Updated weights for policy 1, policy_version 317829 (0.0010) [2023-12-26 17:40:32,360][105692] Updated weights for policy 0, policy_version 317578 (0.0009) [2023-12-26 17:40:32,424][105692] Updated weights for policy 0, policy_version 317588 (0.0007) [2023-12-26 17:40:32,483][105692] Updated weights for policy 0, policy_version 317598 (0.0008) [2023-12-26 17:40:32,546][105692] Updated weights for policy 0, policy_version 317608 (0.0008) [2023-12-26 17:40:32,633][105620] Updated weights for policy 1, policy_version 317839 (0.0007) [2023-12-26 17:40:32,694][105620] Updated weights for policy 1, policy_version 317849 (0.0009) [2023-12-26 17:40:32,747][105620] Updated weights for policy 1, policy_version 317859 (0.0009) [2023-12-26 17:40:33,319][105692] Updated weights for policy 0, policy_version 317618 (0.0010) [2023-12-26 17:40:33,370][105692] Updated weights for policy 0, policy_version 317628 (0.0010) [2023-12-26 17:40:33,400][105620] Updated weights for policy 1, policy_version 317869 (0.0007) [2023-12-26 17:40:33,425][105692] Updated weights for policy 0, policy_version 317638 (0.0010) [2023-12-26 17:40:33,452][105620] Updated weights for policy 1, policy_version 317879 (0.0006) [2023-12-26 17:40:33,508][105620] Updated weights for policy 1, policy_version 317889 (0.0008) [2023-12-26 17:40:34,066][105692] Updated weights for policy 0, policy_version 317648 (0.0006) [2023-12-26 17:40:34,110][105692] Updated weights for policy 0, policy_version 317658 (0.0006) [2023-12-26 17:40:34,161][105692] Updated weights for policy 0, policy_version 317668 (0.0008) [2023-12-26 17:40:34,168][105620] Updated weights for policy 1, policy_version 317899 (0.0007) [2023-12-26 17:40:34,237][105620] Updated weights for policy 1, policy_version 317909 (0.0008) [2023-12-26 17:40:34,297][105620] Updated weights for policy 1, policy_version 317919 (0.0007) [2023-12-26 17:40:34,901][105692] Updated weights for policy 0, policy_version 317678 (0.0008) [2023-12-26 17:40:34,946][105692] Updated weights for policy 0, policy_version 317688 (0.0008) [2023-12-26 17:40:35,000][105692] Updated weights for policy 0, policy_version 317698 (0.0009) [2023-12-26 17:40:35,008][105620] Updated weights for policy 1, policy_version 317929 (0.0005) [2023-12-26 17:40:35,061][105620] Updated weights for policy 1, policy_version 317939 (0.0009) [2023-12-26 17:40:35,109][105620] Updated weights for policy 1, policy_version 317949 (0.0009) [2023-12-26 17:40:35,159][105620] Updated weights for policy 1, policy_version 317959 (0.0007) [2023-12-26 17:40:35,786][105620] Updated weights for policy 1, policy_version 317969 (0.0009) [2023-12-26 17:40:35,816][105692] Updated weights for policy 0, policy_version 317708 (0.0007) [2023-12-26 17:40:35,845][105620] Updated weights for policy 1, policy_version 317979 (0.0010) [2023-12-26 17:40:35,867][105692] Updated weights for policy 0, policy_version 317718 (0.0006) [2023-12-26 17:40:35,899][105620] Updated weights for policy 1, policy_version 317989 (0.0010) [2023-12-26 17:40:35,922][105692] Updated weights for policy 0, policy_version 317728 (0.0006) [2023-12-26 17:40:36,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.7, 300 sec: 19633.0). Total num frames: 162766848. Throughput: 0: 9612.9, 1: 10050.4. Samples: 162750848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:40:36,063][104569] Avg episode reward: [(0, '7829.124'), (1, '6955.544')] [2023-12-26 17:40:36,603][105620] Updated weights for policy 1, policy_version 317999 (0.0007) [2023-12-26 17:40:36,658][105620] Updated weights for policy 1, policy_version 318009 (0.0006) [2023-12-26 17:40:36,709][105620] Updated weights for policy 1, policy_version 318019 (0.0006) [2023-12-26 17:40:36,727][105692] Updated weights for policy 0, policy_version 317738 (0.0009) [2023-12-26 17:40:36,787][105692] Updated weights for policy 0, policy_version 317748 (0.0011) [2023-12-26 17:40:36,858][105692] Updated weights for policy 0, policy_version 317758 (0.0011) [2023-12-26 17:40:36,935][105692] Updated weights for policy 0, policy_version 317768 (0.0011) [2023-12-26 17:40:37,394][105620] Updated weights for policy 1, policy_version 318029 (0.0009) [2023-12-26 17:40:37,462][105620] Updated weights for policy 1, policy_version 318039 (0.0010) [2023-12-26 17:40:37,509][105692] Updated weights for policy 0, policy_version 317778 (0.0007) [2023-12-26 17:40:37,532][105620] Updated weights for policy 1, policy_version 318049 (0.0010) [2023-12-26 17:40:37,569][105692] Updated weights for policy 0, policy_version 317788 (0.0010) [2023-12-26 17:40:37,633][105692] Updated weights for policy 0, policy_version 317798 (0.0008) [2023-12-26 17:40:38,261][105692] Updated weights for policy 0, policy_version 317808 (0.0008) [2023-12-26 17:40:38,269][105620] Updated weights for policy 1, policy_version 318059 (0.0010) [2023-12-26 17:40:38,319][105620] Updated weights for policy 1, policy_version 318069 (0.0010) [2023-12-26 17:40:38,323][105692] Updated weights for policy 0, policy_version 317818 (0.0010) [2023-12-26 17:40:38,382][105620] Updated weights for policy 1, policy_version 318079 (0.0011) [2023-12-26 17:40:38,386][105692] Updated weights for policy 0, policy_version 317828 (0.0008) [2023-12-26 17:40:39,125][105620] Updated weights for policy 1, policy_version 318089 (0.0011) [2023-12-26 17:40:39,165][105692] Updated weights for policy 0, policy_version 317838 (0.0010) [2023-12-26 17:40:39,187][105620] Updated weights for policy 1, policy_version 318099 (0.0010) [2023-12-26 17:40:39,228][105692] Updated weights for policy 0, policy_version 317848 (0.0009) [2023-12-26 17:40:39,250][105620] Updated weights for policy 1, policy_version 318109 (0.0009) [2023-12-26 17:40:39,295][105692] Updated weights for policy 0, policy_version 317858 (0.0011) [2023-12-26 17:40:39,311][105620] Updated weights for policy 1, policy_version 318119 (0.0010) [2023-12-26 17:40:39,963][105692] Updated weights for policy 0, policy_version 317868 (0.0010) [2023-12-26 17:40:39,991][105620] Updated weights for policy 1, policy_version 318129 (0.0008) [2023-12-26 17:40:40,026][105692] Updated weights for policy 0, policy_version 317878 (0.0010) [2023-12-26 17:40:40,056][105620] Updated weights for policy 1, policy_version 318139 (0.0006) [2023-12-26 17:40:40,089][105692] Updated weights for policy 0, policy_version 317888 (0.0010) [2023-12-26 17:40:40,114][105620] Updated weights for policy 1, policy_version 318149 (0.0009) [2023-12-26 17:40:40,769][105692] Updated weights for policy 0, policy_version 317898 (0.0010) [2023-12-26 17:40:40,825][105620] Updated weights for policy 1, policy_version 318159 (0.0009) [2023-12-26 17:40:40,827][105692] Updated weights for policy 0, policy_version 317908 (0.0005) [2023-12-26 17:40:40,871][105620] Updated weights for policy 1, policy_version 318169 (0.0006) [2023-12-26 17:40:40,876][105692] Updated weights for policy 0, policy_version 317918 (0.0010) [2023-12-26 17:40:40,922][105620] Updated weights for policy 1, policy_version 318179 (0.0006) [2023-12-26 17:40:40,928][105692] Updated weights for policy 0, policy_version 317928 (0.0010) [2023-12-26 17:40:41,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 162865152. Throughput: 0: 9645.2, 1: 10007.1. Samples: 162867896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:40:41,062][104569] Avg episode reward: [(0, '7562.696'), (1, '9280.563')] [2023-12-26 17:40:41,652][105620] Updated weights for policy 1, policy_version 318189 (0.0006) [2023-12-26 17:40:41,701][105692] Updated weights for policy 0, policy_version 317938 (0.0010) [2023-12-26 17:40:41,720][105620] Updated weights for policy 1, policy_version 318199 (0.0007) [2023-12-26 17:40:41,771][105692] Updated weights for policy 0, policy_version 317948 (0.0010) [2023-12-26 17:40:41,787][105620] Updated weights for policy 1, policy_version 318209 (0.0009) [2023-12-26 17:40:41,834][105692] Updated weights for policy 0, policy_version 317958 (0.0010) [2023-12-26 17:40:42,519][105620] Updated weights for policy 1, policy_version 318219 (0.0009) [2023-12-26 17:40:42,561][105692] Updated weights for policy 0, policy_version 317968 (0.0010) [2023-12-26 17:40:42,569][105620] Updated weights for policy 1, policy_version 318229 (0.0009) [2023-12-26 17:40:42,610][105692] Updated weights for policy 0, policy_version 317978 (0.0010) [2023-12-26 17:40:42,632][105620] Updated weights for policy 1, policy_version 318239 (0.0006) [2023-12-26 17:40:42,665][105692] Updated weights for policy 0, policy_version 317988 (0.0010) [2023-12-26 17:40:43,383][105620] Updated weights for policy 1, policy_version 318249 (0.0007) [2023-12-26 17:40:43,432][105620] Updated weights for policy 1, policy_version 318259 (0.0005) [2023-12-26 17:40:43,439][105692] Updated weights for policy 0, policy_version 317998 (0.0011) [2023-12-26 17:40:43,483][105620] Updated weights for policy 1, policy_version 318269 (0.0005) [2023-12-26 17:40:43,497][105692] Updated weights for policy 0, policy_version 318008 (0.0010) [2023-12-26 17:40:43,548][105620] Updated weights for policy 1, policy_version 318279 (0.0008) [2023-12-26 17:40:43,556][105692] Updated weights for policy 0, policy_version 318018 (0.0010) [2023-12-26 17:40:44,168][105620] Updated weights for policy 1, policy_version 318289 (0.0006) [2023-12-26 17:40:44,229][105620] Updated weights for policy 1, policy_version 318299 (0.0010) [2023-12-26 17:40:44,254][105692] Updated weights for policy 0, policy_version 318028 (0.0008) [2023-12-26 17:40:44,283][105620] Updated weights for policy 1, policy_version 318309 (0.0010) [2023-12-26 17:40:44,306][105692] Updated weights for policy 0, policy_version 318038 (0.0006) [2023-12-26 17:40:44,341][105585] KL-divergence is very high: 142.7903 [2023-12-26 17:40:44,368][105692] Updated weights for policy 0, policy_version 318048 (0.0007) [2023-12-26 17:40:44,399][105585] KL-divergence is very high: 154.3584 [2023-12-26 17:40:44,931][105620] Updated weights for policy 1, policy_version 318319 (0.0007) [2023-12-26 17:40:44,944][105692] Updated weights for policy 0, policy_version 318058 (0.0005) [2023-12-26 17:40:44,998][105620] Updated weights for policy 1, policy_version 318329 (0.0006) [2023-12-26 17:40:45,012][105692] Updated weights for policy 0, policy_version 318068 (0.0006) [2023-12-26 17:40:45,066][105620] Updated weights for policy 1, policy_version 318339 (0.0011) [2023-12-26 17:40:45,073][105692] Updated weights for policy 0, policy_version 318078 (0.0006) [2023-12-26 17:40:45,131][105692] Updated weights for policy 0, policy_version 318088 (0.0009) [2023-12-26 17:40:45,602][105620] Updated weights for policy 1, policy_version 318349 (0.0009) [2023-12-26 17:40:45,660][105620] Updated weights for policy 1, policy_version 318359 (0.0005) [2023-12-26 17:40:45,720][105620] Updated weights for policy 1, policy_version 318369 (0.0005) [2023-12-26 17:40:45,814][105692] Updated weights for policy 0, policy_version 318098 (0.0008) [2023-12-26 17:40:45,874][105692] Updated weights for policy 0, policy_version 318108 (0.0009) [2023-12-26 17:40:45,928][105692] Updated weights for policy 0, policy_version 318119 (0.0011) [2023-12-26 17:40:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.9, 300 sec: 19633.0). Total num frames: 162963456. Throughput: 0: 9666.4, 1: 9984.8. Samples: 162926012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:40:46,063][104569] Avg episode reward: [(0, '8365.737'), (1, '9357.564')] [2023-12-26 17:40:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000318120_81453056.pth... [2023-12-26 17:40:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000318376_81510400.pth... [2023-12-26 17:40:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000316968_81158144.pth [2023-12-26 17:40:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000317192_81207296.pth [2023-12-26 17:40:46,266][105620] Updated weights for policy 1, policy_version 318379 (0.0005) [2023-12-26 17:40:46,322][105620] Updated weights for policy 1, policy_version 318389 (0.0005) [2023-12-26 17:40:46,376][105620] Updated weights for policy 1, policy_version 318399 (0.0005) [2023-12-26 17:40:46,785][105692] Updated weights for policy 0, policy_version 318129 (0.0007) [2023-12-26 17:40:46,836][105692] Updated weights for policy 0, policy_version 318139 (0.0006) [2023-12-26 17:40:46,897][105692] Updated weights for policy 0, policy_version 318149 (0.0005) [2023-12-26 17:40:47,014][105620] Updated weights for policy 1, policy_version 318409 (0.0006) [2023-12-26 17:40:47,068][105620] Updated weights for policy 1, policy_version 318419 (0.0009) [2023-12-26 17:40:47,119][105620] Updated weights for policy 1, policy_version 318429 (0.0008) [2023-12-26 17:40:47,168][105620] Updated weights for policy 1, policy_version 318439 (0.0009) [2023-12-26 17:40:47,505][105692] Updated weights for policy 0, policy_version 318159 (0.0008) [2023-12-26 17:40:47,552][105692] Updated weights for policy 0, policy_version 318169 (0.0005) [2023-12-26 17:40:47,605][105692] Updated weights for policy 0, policy_version 318179 (0.0005) [2023-12-26 17:40:48,043][105620] Updated weights for policy 1, policy_version 318449 (0.0006) [2023-12-26 17:40:48,092][105620] Updated weights for policy 1, policy_version 318459 (0.0005) [2023-12-26 17:40:48,140][105620] Updated weights for policy 1, policy_version 318469 (0.0008) [2023-12-26 17:40:48,193][105692] Updated weights for policy 0, policy_version 318189 (0.0008) [2023-12-26 17:40:48,261][105692] Updated weights for policy 0, policy_version 318199 (0.0009) [2023-12-26 17:40:48,323][105692] Updated weights for policy 0, policy_version 318209 (0.0006) [2023-12-26 17:40:48,789][105620] Updated weights for policy 1, policy_version 318479 (0.0006) [2023-12-26 17:40:48,837][105620] Updated weights for policy 1, policy_version 318489 (0.0006) [2023-12-26 17:40:48,898][105620] Updated weights for policy 1, policy_version 318499 (0.0005) [2023-12-26 17:40:49,014][105692] Updated weights for policy 0, policy_version 318219 (0.0007) [2023-12-26 17:40:49,085][105692] Updated weights for policy 0, policy_version 318229 (0.0005) [2023-12-26 17:40:49,151][105692] Updated weights for policy 0, policy_version 318239 (0.0006) [2023-12-26 17:40:49,624][105620] Updated weights for policy 1, policy_version 318509 (0.0007) [2023-12-26 17:40:49,681][105620] Updated weights for policy 1, policy_version 318519 (0.0009) [2023-12-26 17:40:49,739][105620] Updated weights for policy 1, policy_version 318529 (0.0009) [2023-12-26 17:40:49,817][105692] Updated weights for policy 0, policy_version 318249 (0.0006) [2023-12-26 17:40:49,879][105692] Updated weights for policy 0, policy_version 318259 (0.0009) [2023-12-26 17:40:49,945][105692] Updated weights for policy 0, policy_version 318269 (0.0009) [2023-12-26 17:40:50,004][105692] Updated weights for policy 0, policy_version 318279 (0.0008) [2023-12-26 17:40:50,550][105620] Updated weights for policy 1, policy_version 318539 (0.0008) [2023-12-26 17:40:50,616][105620] Updated weights for policy 1, policy_version 318549 (0.0006) [2023-12-26 17:40:50,682][105620] Updated weights for policy 1, policy_version 318559 (0.0008) [2023-12-26 17:40:50,781][105692] Updated weights for policy 0, policy_version 318289 (0.0009) [2023-12-26 17:40:50,849][105692] Updated weights for policy 0, policy_version 318300 (0.0010) [2023-12-26 17:40:50,912][105692] Updated weights for policy 0, policy_version 318310 (0.0009) [2023-12-26 17:40:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 163061760. Throughput: 0: 9725.4, 1: 10062.8. Samples: 163049360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:40:51,062][104569] Avg episode reward: [(0, '8774.297'), (1, '9357.390')] [2023-12-26 17:40:51,378][105620] Updated weights for policy 1, policy_version 318569 (0.0009) [2023-12-26 17:40:51,441][105620] Updated weights for policy 1, policy_version 318579 (0.0009) [2023-12-26 17:40:51,505][105620] Updated weights for policy 1, policy_version 318589 (0.0008) [2023-12-26 17:40:51,566][105620] Updated weights for policy 1, policy_version 318599 (0.0009) [2023-12-26 17:40:51,673][105692] Updated weights for policy 0, policy_version 318320 (0.0009) [2023-12-26 17:40:51,740][105692] Updated weights for policy 0, policy_version 318330 (0.0008) [2023-12-26 17:40:51,803][105692] Updated weights for policy 0, policy_version 318340 (0.0007) [2023-12-26 17:40:52,371][105620] Updated weights for policy 1, policy_version 318609 (0.0010) [2023-12-26 17:40:52,429][105620] Updated weights for policy 1, policy_version 318619 (0.0008) [2023-12-26 17:40:52,491][105620] Updated weights for policy 1, policy_version 318629 (0.0007) [2023-12-26 17:40:52,493][105692] Updated weights for policy 0, policy_version 318350 (0.0008) [2023-12-26 17:40:52,553][105692] Updated weights for policy 0, policy_version 318360 (0.0008) [2023-12-26 17:40:52,618][105692] Updated weights for policy 0, policy_version 318370 (0.0008) [2023-12-26 17:40:53,286][105620] Updated weights for policy 1, policy_version 318639 (0.0008) [2023-12-26 17:40:53,288][105692] Updated weights for policy 0, policy_version 318380 (0.0007) [2023-12-26 17:40:53,342][105692] Updated weights for policy 0, policy_version 318390 (0.0005) [2023-12-26 17:40:53,342][105620] Updated weights for policy 1, policy_version 318649 (0.0008) [2023-12-26 17:40:53,387][105620] Updated weights for policy 1, policy_version 318659 (0.0008) [2023-12-26 17:40:53,392][105692] Updated weights for policy 0, policy_version 318400 (0.0007) [2023-12-26 17:40:54,086][105692] Updated weights for policy 0, policy_version 318410 (0.0007) [2023-12-26 17:40:54,128][105585] KL-divergence is very high: 125.1261 [2023-12-26 17:40:54,139][105692] Updated weights for policy 0, policy_version 318420 (0.0009) [2023-12-26 17:40:54,175][105620] Updated weights for policy 1, policy_version 318669 (0.0006) [2023-12-26 17:40:54,176][105585] KL-divergence is very high: 140.0446 [2023-12-26 17:40:54,197][105692] Updated weights for policy 0, policy_version 318430 (0.0008) [2023-12-26 17:40:54,231][105620] Updated weights for policy 1, policy_version 318679 (0.0008) [2023-12-26 17:40:54,253][105692] Updated weights for policy 0, policy_version 318440 (0.0006) [2023-12-26 17:40:54,282][105620] Updated weights for policy 1, policy_version 318689 (0.0007) [2023-12-26 17:40:54,950][105692] Updated weights for policy 0, policy_version 318450 (0.0007) [2023-12-26 17:40:55,004][105692] Updated weights for policy 0, policy_version 318460 (0.0006) [2023-12-26 17:40:55,055][105692] Updated weights for policy 0, policy_version 318470 (0.0005) [2023-12-26 17:40:55,088][105620] Updated weights for policy 1, policy_version 318699 (0.0009) [2023-12-26 17:40:55,153][105620] Updated weights for policy 1, policy_version 318709 (0.0010) [2023-12-26 17:40:55,223][105620] Updated weights for policy 1, policy_version 318719 (0.0010) [2023-12-26 17:40:55,597][105692] Updated weights for policy 0, policy_version 318480 (0.0006) [2023-12-26 17:40:55,646][105692] Updated weights for policy 0, policy_version 318490 (0.0007) [2023-12-26 17:40:55,691][105692] Updated weights for policy 0, policy_version 318500 (0.0010) [2023-12-26 17:40:56,001][105620] Updated weights for policy 1, policy_version 318729 (0.0010) [2023-12-26 17:40:56,056][105620] Updated weights for policy 1, policy_version 318739 (0.0008) [2023-12-26 17:40:56,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 163151872. Throughput: 0: 9818.0, 1: 9934.7. Samples: 163163320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:40:56,062][104569] Avg episode reward: [(0, '9088.695'), (1, '9357.499')] [2023-12-26 17:40:56,114][105620] Updated weights for policy 1, policy_version 318749 (0.0006) [2023-12-26 17:40:56,178][105620] Updated weights for policy 1, policy_version 318759 (0.0006) [2023-12-26 17:40:56,407][105692] Updated weights for policy 0, policy_version 318510 (0.0008) [2023-12-26 17:40:56,462][105692] Updated weights for policy 0, policy_version 318520 (0.0011) [2023-12-26 17:40:56,528][105692] Updated weights for policy 0, policy_version 318530 (0.0011) [2023-12-26 17:40:56,793][105620] Updated weights for policy 1, policy_version 318769 (0.0006) [2023-12-26 17:40:56,862][105620] Updated weights for policy 1, policy_version 318779 (0.0007) [2023-12-26 17:40:56,912][105620] Updated weights for policy 1, policy_version 318789 (0.0010) [2023-12-26 17:40:57,170][105692] Updated weights for policy 0, policy_version 318540 (0.0008) [2023-12-26 17:40:57,215][105692] Updated weights for policy 0, policy_version 318550 (0.0005) [2023-12-26 17:40:57,261][105692] Updated weights for policy 0, policy_version 318560 (0.0005) [2023-12-26 17:40:57,537][105620] Updated weights for policy 1, policy_version 318799 (0.0007) [2023-12-26 17:40:57,603][105620] Updated weights for policy 1, policy_version 318809 (0.0008) [2023-12-26 17:40:57,668][105620] Updated weights for policy 1, policy_version 318819 (0.0008) [2023-12-26 17:40:57,940][105692] Updated weights for policy 0, policy_version 318570 (0.0006) [2023-12-26 17:40:57,987][105692] Updated weights for policy 0, policy_version 318580 (0.0010) [2023-12-26 17:40:58,025][105585] KL-divergence is very high: 110.3231 [2023-12-26 17:40:58,041][105692] Updated weights for policy 0, policy_version 318590 (0.0010) [2023-12-26 17:40:58,091][105585] KL-divergence is very high: 180.5482 [2023-12-26 17:40:58,126][105692] Updated weights for policy 0, policy_version 318600 (0.0008) [2023-12-26 17:40:58,381][105620] Updated weights for policy 1, policy_version 318829 (0.0009) [2023-12-26 17:40:58,447][105620] Updated weights for policy 1, policy_version 318839 (0.0008) [2023-12-26 17:40:58,512][105620] Updated weights for policy 1, policy_version 318849 (0.0009) [2023-12-26 17:40:58,910][105692] Updated weights for policy 0, policy_version 318610 (0.0007) [2023-12-26 17:40:58,973][105692] Updated weights for policy 0, policy_version 318620 (0.0008) [2023-12-26 17:40:59,036][105692] Updated weights for policy 0, policy_version 318630 (0.0008) [2023-12-26 17:40:59,347][105620] Updated weights for policy 1, policy_version 318859 (0.0009) [2023-12-26 17:40:59,413][105620] Updated weights for policy 1, policy_version 318869 (0.0011) [2023-12-26 17:40:59,476][105620] Updated weights for policy 1, policy_version 318879 (0.0007) [2023-12-26 17:40:59,820][105692] Updated weights for policy 0, policy_version 318640 (0.0006) [2023-12-26 17:40:59,885][105692] Updated weights for policy 0, policy_version 318650 (0.0009) [2023-12-26 17:40:59,945][105692] Updated weights for policy 0, policy_version 318660 (0.0009) [2023-12-26 17:41:00,082][105620] Updated weights for policy 1, policy_version 318889 (0.0006) [2023-12-26 17:41:00,149][105620] Updated weights for policy 1, policy_version 318899 (0.0011) [2023-12-26 17:41:00,210][105620] Updated weights for policy 1, policy_version 318909 (0.0010) [2023-12-26 17:41:00,264][105620] Updated weights for policy 1, policy_version 318919 (0.0010) [2023-12-26 17:41:00,646][105692] Updated weights for policy 0, policy_version 318670 (0.0007) [2023-12-26 17:41:00,697][105692] Updated weights for policy 0, policy_version 318680 (0.0007) [2023-12-26 17:41:00,750][105692] Updated weights for policy 0, policy_version 318690 (0.0005) [2023-12-26 17:41:00,870][105620] Updated weights for policy 1, policy_version 318929 (0.0010) [2023-12-26 17:41:00,931][105620] Updated weights for policy 1, policy_version 318939 (0.0008) [2023-12-26 17:41:00,995][105620] Updated weights for policy 1, policy_version 318949 (0.0010) [2023-12-26 17:41:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 163258368. Throughput: 0: 9890.3, 1: 9902.5. Samples: 163223128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:41:01,063][104569] Avg episode reward: [(0, '9178.243'), (1, '9270.655')] [2023-12-26 17:41:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000318696_81600512.pth... [2023-12-26 17:41:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000318952_81657856.pth... [2023-12-26 17:41:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000317800_81362944.pth [2023-12-26 17:41:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000317544_81305600.pth [2023-12-26 17:41:01,385][105692] Updated weights for policy 0, policy_version 318700 (0.0007) [2023-12-26 17:41:01,446][105692] Updated weights for policy 0, policy_version 318710 (0.0010) [2023-12-26 17:41:01,509][105692] Updated weights for policy 0, policy_version 318720 (0.0010) [2023-12-26 17:41:01,656][105620] Updated weights for policy 1, policy_version 318959 (0.0010) [2023-12-26 17:41:01,713][105620] Updated weights for policy 1, policy_version 318969 (0.0008) [2023-12-26 17:41:01,778][105620] Updated weights for policy 1, policy_version 318979 (0.0009) [2023-12-26 17:41:02,369][105692] Updated weights for policy 0, policy_version 318730 (0.0010) [2023-12-26 17:41:02,392][105620] Updated weights for policy 1, policy_version 318989 (0.0007) [2023-12-26 17:41:02,430][105692] Updated weights for policy 0, policy_version 318740 (0.0009) [2023-12-26 17:41:02,458][105620] Updated weights for policy 1, policy_version 318999 (0.0006) [2023-12-26 17:41:02,482][105692] Updated weights for policy 0, policy_version 318750 (0.0008) [2023-12-26 17:41:02,517][105620] Updated weights for policy 1, policy_version 319009 (0.0006) [2023-12-26 17:41:02,533][105692] Updated weights for policy 0, policy_version 318760 (0.0009) [2023-12-26 17:41:03,195][105620] Updated weights for policy 1, policy_version 319019 (0.0007) [2023-12-26 17:41:03,215][105692] Updated weights for policy 0, policy_version 318770 (0.0005) [2023-12-26 17:41:03,252][105620] Updated weights for policy 1, policy_version 319029 (0.0010) [2023-12-26 17:41:03,278][105692] Updated weights for policy 0, policy_version 318780 (0.0006) [2023-12-26 17:41:03,307][105620] Updated weights for policy 1, policy_version 319039 (0.0010) [2023-12-26 17:41:03,337][105692] Updated weights for policy 0, policy_version 318790 (0.0006) [2023-12-26 17:41:03,973][105692] Updated weights for policy 0, policy_version 318800 (0.0005) [2023-12-26 17:41:04,012][105620] Updated weights for policy 1, policy_version 319049 (0.0010) [2023-12-26 17:41:04,034][105692] Updated weights for policy 0, policy_version 318810 (0.0007) [2023-12-26 17:41:04,071][105620] Updated weights for policy 1, policy_version 319059 (0.0010) [2023-12-26 17:41:04,094][105692] Updated weights for policy 0, policy_version 318820 (0.0007) [2023-12-26 17:41:04,129][105620] Updated weights for policy 1, policy_version 319069 (0.0010) [2023-12-26 17:41:04,199][105620] Updated weights for policy 1, policy_version 319079 (0.0010) [2023-12-26 17:41:04,782][105692] Updated weights for policy 0, policy_version 318830 (0.0008) [2023-12-26 17:41:04,849][105692] Updated weights for policy 0, policy_version 318840 (0.0008) [2023-12-26 17:41:04,910][105692] Updated weights for policy 0, policy_version 318850 (0.0007) [2023-12-26 17:41:04,932][105620] Updated weights for policy 1, policy_version 319089 (0.0010) [2023-12-26 17:41:04,993][105620] Updated weights for policy 1, policy_version 319099 (0.0010) [2023-12-26 17:41:05,057][105620] Updated weights for policy 1, policy_version 319109 (0.0010) [2023-12-26 17:41:05,663][105692] Updated weights for policy 0, policy_version 318860 (0.0007) [2023-12-26 17:41:05,714][105692] Updated weights for policy 0, policy_version 318870 (0.0010) [2023-12-26 17:41:05,738][105620] Updated weights for policy 1, policy_version 319119 (0.0007) [2023-12-26 17:41:05,768][105692] Updated weights for policy 0, policy_version 318880 (0.0009) [2023-12-26 17:41:05,794][105620] Updated weights for policy 1, policy_version 319129 (0.0005) [2023-12-26 17:41:05,853][105620] Updated weights for policy 1, policy_version 319139 (0.0006) [2023-12-26 17:41:06,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 163356672. Throughput: 0: 9773.2, 1: 9923.5. Samples: 163342292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:41:06,062][104569] Avg episode reward: [(0, '9177.411'), (1, '9085.850')] [2023-12-26 17:41:06,540][105620] Updated weights for policy 1, policy_version 319149 (0.0010) [2023-12-26 17:41:06,574][105692] Updated weights for policy 0, policy_version 318890 (0.0008) [2023-12-26 17:41:06,603][105620] Updated weights for policy 1, policy_version 319159 (0.0011) [2023-12-26 17:41:06,634][105692] Updated weights for policy 0, policy_version 318900 (0.0006) [2023-12-26 17:41:06,653][105620] Updated weights for policy 1, policy_version 319169 (0.0011) [2023-12-26 17:41:06,691][105692] Updated weights for policy 0, policy_version 318910 (0.0006) [2023-12-26 17:41:06,746][105692] Updated weights for policy 0, policy_version 318920 (0.0008) [2023-12-26 17:41:07,414][105620] Updated weights for policy 1, policy_version 319179 (0.0010) [2023-12-26 17:41:07,468][105620] Updated weights for policy 1, policy_version 319189 (0.0010) [2023-12-26 17:41:07,491][105692] Updated weights for policy 0, policy_version 318930 (0.0005) [2023-12-26 17:41:07,530][105620] Updated weights for policy 1, policy_version 319199 (0.0010) [2023-12-26 17:41:07,540][105692] Updated weights for policy 0, policy_version 318940 (0.0007) [2023-12-26 17:41:07,589][105692] Updated weights for policy 0, policy_version 318950 (0.0006) [2023-12-26 17:41:08,280][105620] Updated weights for policy 1, policy_version 319209 (0.0010) [2023-12-26 17:41:08,360][105620] Updated weights for policy 1, policy_version 319219 (0.0011) [2023-12-26 17:41:08,370][105692] Updated weights for policy 0, policy_version 318960 (0.0009) [2023-12-26 17:41:08,425][105692] Updated weights for policy 0, policy_version 318970 (0.0008) [2023-12-26 17:41:08,426][105620] Updated weights for policy 1, policy_version 319229 (0.0010) [2023-12-26 17:41:08,476][105692] Updated weights for policy 0, policy_version 318980 (0.0007) [2023-12-26 17:41:08,485][105620] Updated weights for policy 1, policy_version 319239 (0.0011) [2023-12-26 17:41:09,218][105620] Updated weights for policy 1, policy_version 319249 (0.0010) [2023-12-26 17:41:09,259][105692] Updated weights for policy 0, policy_version 318990 (0.0007) [2023-12-26 17:41:09,286][105620] Updated weights for policy 1, policy_version 319259 (0.0011) [2023-12-26 17:41:09,325][105692] Updated weights for policy 0, policy_version 319000 (0.0008) [2023-12-26 17:41:09,355][105620] Updated weights for policy 1, policy_version 319269 (0.0010) [2023-12-26 17:41:09,395][105692] Updated weights for policy 0, policy_version 319010 (0.0009) [2023-12-26 17:41:10,101][105620] Updated weights for policy 1, policy_version 319279 (0.0008) [2023-12-26 17:41:10,164][105620] Updated weights for policy 1, policy_version 319289 (0.0009) [2023-12-26 17:41:10,198][105692] Updated weights for policy 0, policy_version 319020 (0.0007) [2023-12-26 17:41:10,220][105620] Updated weights for policy 1, policy_version 319299 (0.0008) [2023-12-26 17:41:10,260][105692] Updated weights for policy 0, policy_version 319030 (0.0007) [2023-12-26 17:41:10,322][105692] Updated weights for policy 0, policy_version 319040 (0.0009) [2023-12-26 17:41:10,863][105620] Updated weights for policy 1, policy_version 319309 (0.0007) [2023-12-26 17:41:10,920][105620] Updated weights for policy 1, policy_version 319319 (0.0005) [2023-12-26 17:41:10,980][105620] Updated weights for policy 1, policy_version 319329 (0.0005) [2023-12-26 17:41:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 163446784. Throughput: 0: 9722.5, 1: 9894.5. Samples: 163454440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:41:11,063][104569] Avg episode reward: [(0, '9177.097'), (1, '9172.918')] [2023-12-26 17:41:11,205][105692] Updated weights for policy 0, policy_version 319050 (0.0008) [2023-12-26 17:41:11,268][105692] Updated weights for policy 0, policy_version 319060 (0.0009) [2023-12-26 17:41:11,331][105692] Updated weights for policy 0, policy_version 319070 (0.0008) [2023-12-26 17:41:11,399][105692] Updated weights for policy 0, policy_version 319080 (0.0009) [2023-12-26 17:41:11,631][105620] Updated weights for policy 1, policy_version 319339 (0.0011) [2023-12-26 17:41:11,693][105620] Updated weights for policy 1, policy_version 319349 (0.0007) [2023-12-26 17:41:11,758][105620] Updated weights for policy 1, policy_version 319359 (0.0009) [2023-12-26 17:41:12,232][105692] Updated weights for policy 0, policy_version 319090 (0.0008) [2023-12-26 17:41:12,297][105692] Updated weights for policy 0, policy_version 319100 (0.0008) [2023-12-26 17:41:12,365][105692] Updated weights for policy 0, policy_version 319110 (0.0009) [2023-12-26 17:41:12,476][105620] Updated weights for policy 1, policy_version 319369 (0.0010) [2023-12-26 17:41:12,542][105620] Updated weights for policy 1, policy_version 319379 (0.0007) [2023-12-26 17:41:12,605][105620] Updated weights for policy 1, policy_version 319389 (0.0006) [2023-12-26 17:41:12,661][105620] Updated weights for policy 1, policy_version 319399 (0.0006) [2023-12-26 17:41:13,133][105692] Updated weights for policy 0, policy_version 319120 (0.0008) [2023-12-26 17:41:13,194][105692] Updated weights for policy 0, policy_version 319130 (0.0006) [2023-12-26 17:41:13,236][105692] Updated weights for policy 0, policy_version 319140 (0.0005) [2023-12-26 17:41:13,353][105620] Updated weights for policy 1, policy_version 319409 (0.0010) [2023-12-26 17:41:13,390][105586] KL-divergence is very high: 163.2640 [2023-12-26 17:41:13,400][105586] KL-divergence is very high: 118.4083 [2023-12-26 17:41:13,414][105620] Updated weights for policy 1, policy_version 319419 (0.0010) [2023-12-26 17:41:13,430][105586] KL-divergence is very high: 242.7296 [2023-12-26 17:41:13,441][105586] KL-divergence is very high: 132.5993 [2023-12-26 17:41:13,467][105620] Updated weights for policy 1, policy_version 319429 (0.0010) [2023-12-26 17:41:13,471][105586] KL-divergence is very high: 256.8977 [2023-12-26 17:41:13,819][105692] Updated weights for policy 0, policy_version 319150 (0.0006) [2023-12-26 17:41:13,878][105692] Updated weights for policy 0, policy_version 319160 (0.0009) [2023-12-26 17:41:13,938][105692] Updated weights for policy 0, policy_version 319170 (0.0005) [2023-12-26 17:41:14,212][105620] Updated weights for policy 1, policy_version 319439 (0.0010) [2023-12-26 17:41:14,274][105620] Updated weights for policy 1, policy_version 319449 (0.0010) [2023-12-26 17:41:14,344][105620] Updated weights for policy 1, policy_version 319459 (0.0010) [2023-12-26 17:41:14,516][105692] Updated weights for policy 0, policy_version 319180 (0.0007) [2023-12-26 17:41:14,574][105692] Updated weights for policy 0, policy_version 319190 (0.0008) [2023-12-26 17:41:14,625][105692] Updated weights for policy 0, policy_version 319200 (0.0007) [2023-12-26 17:41:15,069][105620] Updated weights for policy 1, policy_version 319469 (0.0011) [2023-12-26 17:41:15,134][105620] Updated weights for policy 1, policy_version 319479 (0.0009) [2023-12-26 17:41:15,201][105620] Updated weights for policy 1, policy_version 319489 (0.0005) [2023-12-26 17:41:15,420][105692] Updated weights for policy 0, policy_version 319210 (0.0007) [2023-12-26 17:41:15,480][105692] Updated weights for policy 0, policy_version 319220 (0.0008) [2023-12-26 17:41:15,531][105692] Updated weights for policy 0, policy_version 319230 (0.0009) [2023-12-26 17:41:15,783][105620] Updated weights for policy 1, policy_version 319499 (0.0010) [2023-12-26 17:41:15,846][105620] Updated weights for policy 1, policy_version 319509 (0.0011) [2023-12-26 17:41:15,901][105620] Updated weights for policy 1, policy_version 319519 (0.0011) [2023-12-26 17:41:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19662.6, 300 sec: 19660.8). Total num frames: 163545088. Throughput: 0: 9726.5, 1: 9819.5. Samples: 163510948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:41:16,062][104569] Avg episode reward: [(0, '2801.046'), (1, '9081.318')] [2023-12-26 17:41:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000319240_81739776.pth... [2023-12-26 17:41:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000319528_81805312.pth... [2023-12-26 17:41:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000318376_81510400.pth [2023-12-26 17:41:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000318120_81453056.pth [2023-12-26 17:41:16,280][105692] Updated weights for policy 0, policy_version 319241 (0.0010) [2023-12-26 17:41:16,335][105692] Updated weights for policy 0, policy_version 319251 (0.0005) [2023-12-26 17:41:16,395][105692] Updated weights for policy 0, policy_version 319261 (0.0005) [2023-12-26 17:41:16,461][105692] Updated weights for policy 0, policy_version 319271 (0.0005) [2023-12-26 17:41:16,637][105620] Updated weights for policy 1, policy_version 319529 (0.0010) [2023-12-26 17:41:16,685][105620] Updated weights for policy 1, policy_version 319539 (0.0005) [2023-12-26 17:41:16,746][105620] Updated weights for policy 1, policy_version 319549 (0.0005) [2023-12-26 17:41:16,814][105620] Updated weights for policy 1, policy_version 319559 (0.0005) [2023-12-26 17:41:16,963][105692] Updated weights for policy 0, policy_version 319281 (0.0005) [2023-12-26 17:41:17,009][105692] Updated weights for policy 0, policy_version 319291 (0.0005) [2023-12-26 17:41:17,052][105692] Updated weights for policy 0, policy_version 319301 (0.0005) [2023-12-26 17:41:17,321][105620] Updated weights for policy 1, policy_version 319569 (0.0005) [2023-12-26 17:41:17,370][105620] Updated weights for policy 1, policy_version 319579 (0.0005) [2023-12-26 17:41:17,424][105620] Updated weights for policy 1, policy_version 319589 (0.0005) [2023-12-26 17:41:17,613][105692] Updated weights for policy 0, policy_version 319311 (0.0006) [2023-12-26 17:41:17,678][105692] Updated weights for policy 0, policy_version 319321 (0.0011) [2023-12-26 17:41:17,738][105692] Updated weights for policy 0, policy_version 319331 (0.0011) [2023-12-26 17:41:18,070][105620] Updated weights for policy 1, policy_version 319599 (0.0008) [2023-12-26 17:41:18,139][105620] Updated weights for policy 1, policy_version 319609 (0.0009) [2023-12-26 17:41:18,188][105620] Updated weights for policy 1, policy_version 319619 (0.0005) [2023-12-26 17:41:18,325][105692] Updated weights for policy 0, policy_version 319341 (0.0009) [2023-12-26 17:41:18,384][105692] Updated weights for policy 0, policy_version 319351 (0.0008) [2023-12-26 17:41:18,440][105692] Updated weights for policy 0, policy_version 319361 (0.0009) [2023-12-26 17:41:18,860][105620] Updated weights for policy 1, policy_version 319629 (0.0008) [2023-12-26 17:41:18,922][105620] Updated weights for policy 1, policy_version 319639 (0.0008) [2023-12-26 17:41:18,983][105620] Updated weights for policy 1, policy_version 319649 (0.0009) [2023-12-26 17:41:19,216][105692] Updated weights for policy 0, policy_version 319371 (0.0009) [2023-12-26 17:41:19,275][105692] Updated weights for policy 0, policy_version 319381 (0.0008) [2023-12-26 17:41:19,323][105692] Updated weights for policy 0, policy_version 319391 (0.0008) [2023-12-26 17:41:19,691][105620] Updated weights for policy 1, policy_version 319659 (0.0008) [2023-12-26 17:41:19,762][105620] Updated weights for policy 1, policy_version 319669 (0.0006) [2023-12-26 17:41:19,832][105620] Updated weights for policy 1, policy_version 319679 (0.0006) [2023-12-26 17:41:20,133][105692] Updated weights for policy 0, policy_version 319401 (0.0008) [2023-12-26 17:41:20,188][105692] Updated weights for policy 0, policy_version 319411 (0.0009) [2023-12-26 17:41:20,237][105692] Updated weights for policy 0, policy_version 319421 (0.0009) [2023-12-26 17:41:20,299][105692] Updated weights for policy 0, policy_version 319431 (0.0009) [2023-12-26 17:41:20,494][105620] Updated weights for policy 1, policy_version 319689 (0.0009) [2023-12-26 17:41:20,551][105620] Updated weights for policy 1, policy_version 319699 (0.0007) [2023-12-26 17:41:20,620][105620] Updated weights for policy 1, policy_version 319709 (0.0009) [2023-12-26 17:41:20,677][105620] Updated weights for policy 1, policy_version 319719 (0.0010) [2023-12-26 17:41:21,035][105692] Updated weights for policy 0, policy_version 319441 (0.0008) [2023-12-26 17:41:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 163643392. Throughput: 0: 9782.8, 1: 9898.4. Samples: 163636500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:41:21,062][104569] Avg episode reward: [(0, '717.366'), (1, '9173.262')] [2023-12-26 17:41:21,093][105692] Updated weights for policy 0, policy_version 319451 (0.0010) [2023-12-26 17:41:21,165][105692] Updated weights for policy 0, policy_version 319461 (0.0008) [2023-12-26 17:41:21,506][105620] Updated weights for policy 1, policy_version 319729 (0.0007) [2023-12-26 17:41:21,571][105620] Updated weights for policy 1, policy_version 319739 (0.0008) [2023-12-26 17:41:21,641][105620] Updated weights for policy 1, policy_version 319749 (0.0007) [2023-12-26 17:41:21,900][105692] Updated weights for policy 0, policy_version 319471 (0.0010) [2023-12-26 17:41:21,960][105692] Updated weights for policy 0, policy_version 319481 (0.0010) [2023-12-26 17:41:22,014][105692] Updated weights for policy 0, policy_version 319491 (0.0010) [2023-12-26 17:41:22,246][105620] Updated weights for policy 1, policy_version 319759 (0.0006) [2023-12-26 17:41:22,312][105620] Updated weights for policy 1, policy_version 319769 (0.0006) [2023-12-26 17:41:22,382][105620] Updated weights for policy 1, policy_version 319779 (0.0006) [2023-12-26 17:41:22,793][105692] Updated weights for policy 0, policy_version 319501 (0.0008) [2023-12-26 17:41:22,856][105692] Updated weights for policy 0, policy_version 319511 (0.0005) [2023-12-26 17:41:22,911][105692] Updated weights for policy 0, policy_version 319521 (0.0006) [2023-12-26 17:41:23,108][105620] Updated weights for policy 1, policy_version 319789 (0.0007) [2023-12-26 17:41:23,172][105620] Updated weights for policy 1, policy_version 319799 (0.0008) [2023-12-26 17:41:23,231][105620] Updated weights for policy 1, policy_version 319809 (0.0009) [2023-12-26 17:41:23,547][105692] Updated weights for policy 0, policy_version 319531 (0.0007) [2023-12-26 17:41:23,605][105692] Updated weights for policy 0, policy_version 319541 (0.0009) [2023-12-26 17:41:23,666][105692] Updated weights for policy 0, policy_version 319551 (0.0009) [2023-12-26 17:41:23,979][105620] Updated weights for policy 1, policy_version 319819 (0.0008) [2023-12-26 17:41:24,026][105620] Updated weights for policy 1, policy_version 319829 (0.0009) [2023-12-26 17:41:24,072][105620] Updated weights for policy 1, policy_version 319839 (0.0008) [2023-12-26 17:41:24,383][105692] Updated weights for policy 0, policy_version 319561 (0.0007) [2023-12-26 17:41:24,447][105692] Updated weights for policy 0, policy_version 319571 (0.0009) [2023-12-26 17:41:24,495][105692] Updated weights for policy 0, policy_version 319581 (0.0009) [2023-12-26 17:41:24,546][105692] Updated weights for policy 0, policy_version 319591 (0.0009) [2023-12-26 17:41:24,855][105620] Updated weights for policy 1, policy_version 319849 (0.0010) [2023-12-26 17:41:24,915][105620] Updated weights for policy 1, policy_version 319859 (0.0009) [2023-12-26 17:41:24,962][105620] Updated weights for policy 1, policy_version 319869 (0.0009) [2023-12-26 17:41:25,010][105620] Updated weights for policy 1, policy_version 319879 (0.0009) [2023-12-26 17:41:25,261][105692] Updated weights for policy 0, policy_version 319601 (0.0009) [2023-12-26 17:41:25,307][105692] Updated weights for policy 0, policy_version 319611 (0.0008) [2023-12-26 17:41:25,353][105692] Updated weights for policy 0, policy_version 319621 (0.0007) [2023-12-26 17:41:25,849][105620] Updated weights for policy 1, policy_version 319889 (0.0009) [2023-12-26 17:41:25,907][105620] Updated weights for policy 1, policy_version 319899 (0.0009) [2023-12-26 17:41:25,953][105620] Updated weights for policy 1, policy_version 319909 (0.0009) [2023-12-26 17:41:26,003][105692] Updated weights for policy 0, policy_version 319631 (0.0009) [2023-12-26 17:41:26,056][105692] Updated weights for policy 0, policy_version 319641 (0.0005) [2023-12-26 17:41:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 163741696. Throughput: 0: 9787.3, 1: 9817.9. Samples: 163750128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:41:26,062][104569] Avg episode reward: [(0, '975.745'), (1, '9266.635')] [2023-12-26 17:41:26,119][105692] Updated weights for policy 0, policy_version 319651 (0.0005) [2023-12-26 17:41:26,650][105692] Updated weights for policy 0, policy_version 319661 (0.0005) [2023-12-26 17:41:26,706][105692] Updated weights for policy 0, policy_version 319671 (0.0005) [2023-12-26 17:41:26,712][105586] KL-divergence is very high: 109.2402 [2023-12-26 17:41:26,748][105620] Updated weights for policy 1, policy_version 319919 (0.0006) [2023-12-26 17:41:26,762][105586] KL-divergence is very high: 465.6527 [2023-12-26 17:41:26,769][105692] Updated weights for policy 0, policy_version 319681 (0.0009) [2023-12-26 17:41:26,811][105620] Updated weights for policy 1, policy_version 319929 (0.0007) [2023-12-26 17:41:26,812][105586] KL-divergence is very high: 604.8917 [2023-12-26 17:41:26,862][105586] KL-divergence is very high: 607.9189 [2023-12-26 17:41:26,874][105620] Updated weights for policy 1, policy_version 319939 (0.0005) [2023-12-26 17:41:27,387][105692] Updated weights for policy 0, policy_version 319691 (0.0010) [2023-12-26 17:41:27,444][105692] Updated weights for policy 0, policy_version 319701 (0.0010) [2023-12-26 17:41:27,491][105692] Updated weights for policy 0, policy_version 319711 (0.0009) [2023-12-26 17:41:27,506][105620] Updated weights for policy 1, policy_version 319949 (0.0006) [2023-12-26 17:41:27,553][105620] Updated weights for policy 1, policy_version 319959 (0.0006) [2023-12-26 17:41:27,607][105620] Updated weights for policy 1, policy_version 319969 (0.0010) [2023-12-26 17:41:28,083][105692] Updated weights for policy 0, policy_version 319721 (0.0010) [2023-12-26 17:41:28,138][105692] Updated weights for policy 0, policy_version 319731 (0.0010) [2023-12-26 17:41:28,160][105620] Updated weights for policy 1, policy_version 319979 (0.0008) [2023-12-26 17:41:28,189][105692] Updated weights for policy 0, policy_version 319741 (0.0010) [2023-12-26 17:41:28,211][105620] Updated weights for policy 1, policy_version 319989 (0.0010) [2023-12-26 17:41:28,252][105692] Updated weights for policy 0, policy_version 319751 (0.0006) [2023-12-26 17:41:28,259][105620] Updated weights for policy 1, policy_version 319999 (0.0010) [2023-12-26 17:41:28,795][105692] Updated weights for policy 0, policy_version 319761 (0.0005) [2023-12-26 17:41:28,863][105692] Updated weights for policy 0, policy_version 319771 (0.0005) [2023-12-26 17:41:28,934][105692] Updated weights for policy 0, policy_version 319781 (0.0006) [2023-12-26 17:41:28,970][105620] Updated weights for policy 1, policy_version 320009 (0.0010) [2023-12-26 17:41:29,022][105620] Updated weights for policy 1, policy_version 320019 (0.0005) [2023-12-26 17:41:29,080][105620] Updated weights for policy 1, policy_version 320029 (0.0005) [2023-12-26 17:41:29,139][105620] Updated weights for policy 1, policy_version 320039 (0.0005) [2023-12-26 17:41:29,600][105692] Updated weights for policy 0, policy_version 319791 (0.0009) [2023-12-26 17:41:29,624][105585] KL-divergence is very high: 180.1488 [2023-12-26 17:41:29,659][105692] Updated weights for policy 0, policy_version 319801 (0.0010) [2023-12-26 17:41:29,671][105585] KL-divergence is very high: 252.0368 [2023-12-26 17:41:29,717][105692] Updated weights for policy 0, policy_version 319811 (0.0011) [2023-12-26 17:41:29,719][105585] KL-divergence is very high: 241.6716 [2023-12-26 17:41:29,807][105620] Updated weights for policy 1, policy_version 320049 (0.0010) [2023-12-26 17:41:29,871][105620] Updated weights for policy 1, policy_version 320059 (0.0011) [2023-12-26 17:41:29,920][105620] Updated weights for policy 1, policy_version 320069 (0.0010) [2023-12-26 17:41:30,440][105692] Updated weights for policy 0, policy_version 319821 (0.0010) [2023-12-26 17:41:30,494][105692] Updated weights for policy 0, policy_version 319831 (0.0005) [2023-12-26 17:41:30,544][105692] Updated weights for policy 0, policy_version 319841 (0.0005) [2023-12-26 17:41:30,619][105620] Updated weights for policy 1, policy_version 320079 (0.0010) [2023-12-26 17:41:30,667][105620] Updated weights for policy 1, policy_version 320089 (0.0010) [2023-12-26 17:41:30,711][105620] Updated weights for policy 1, policy_version 320099 (0.0010) [2023-12-26 17:41:31,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 163848192. Throughput: 0: 9922.5, 1: 9858.8. Samples: 163816172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:41:31,063][104569] Avg episode reward: [(0, '1941.339'), (1, '8021.067')] [2023-12-26 17:41:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000319848_81895424.pth... [2023-12-26 17:41:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000320104_81952768.pth... [2023-12-26 17:41:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000318696_81600512.pth [2023-12-26 17:41:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000318952_81657856.pth [2023-12-26 17:41:31,221][105692] Updated weights for policy 0, policy_version 319851 (0.0009) [2023-12-26 17:41:31,283][105692] Updated weights for policy 0, policy_version 319861 (0.0010) [2023-12-26 17:41:31,343][105692] Updated weights for policy 0, policy_version 319871 (0.0011) [2023-12-26 17:41:31,470][105620] Updated weights for policy 1, policy_version 320109 (0.0010) [2023-12-26 17:41:31,528][105620] Updated weights for policy 1, policy_version 320119 (0.0010) [2023-12-26 17:41:31,582][105620] Updated weights for policy 1, policy_version 320129 (0.0010) [2023-12-26 17:41:32,034][105692] Updated weights for policy 0, policy_version 319881 (0.0009) [2023-12-26 17:41:32,086][105692] Updated weights for policy 0, policy_version 319891 (0.0009) [2023-12-26 17:41:32,148][105692] Updated weights for policy 0, policy_version 319901 (0.0009) [2023-12-26 17:41:32,202][105692] Updated weights for policy 0, policy_version 319911 (0.0010) [2023-12-26 17:41:32,297][105620] Updated weights for policy 1, policy_version 320139 (0.0009) [2023-12-26 17:41:32,360][105620] Updated weights for policy 1, policy_version 320149 (0.0007) [2023-12-26 17:41:32,413][105620] Updated weights for policy 1, policy_version 320159 (0.0010) [2023-12-26 17:41:32,921][105692] Updated weights for policy 0, policy_version 319921 (0.0008) [2023-12-26 17:41:32,971][105692] Updated weights for policy 0, policy_version 319931 (0.0006) [2023-12-26 17:41:33,020][105692] Updated weights for policy 0, policy_version 319941 (0.0005) [2023-12-26 17:41:33,074][105620] Updated weights for policy 1, policy_version 320169 (0.0008) [2023-12-26 17:41:33,133][105620] Updated weights for policy 1, policy_version 320179 (0.0010) [2023-12-26 17:41:33,190][105620] Updated weights for policy 1, policy_version 320189 (0.0010) [2023-12-26 17:41:33,244][105620] Updated weights for policy 1, policy_version 320199 (0.0010) [2023-12-26 17:41:33,669][105692] Updated weights for policy 0, policy_version 319951 (0.0008) [2023-12-26 17:41:33,726][105692] Updated weights for policy 0, policy_version 319962 (0.0010) [2023-12-26 17:41:33,773][105692] Updated weights for policy 0, policy_version 319972 (0.0008) [2023-12-26 17:41:33,859][105620] Updated weights for policy 1, policy_version 320209 (0.0010) [2023-12-26 17:41:33,903][105620] Updated weights for policy 1, policy_version 320219 (0.0010) [2023-12-26 17:41:33,951][105620] Updated weights for policy 1, policy_version 320229 (0.0010) [2023-12-26 17:41:34,575][105692] Updated weights for policy 0, policy_version 319982 (0.0010) [2023-12-26 17:41:34,631][105692] Updated weights for policy 0, policy_version 319992 (0.0010) [2023-12-26 17:41:34,697][105692] Updated weights for policy 0, policy_version 320002 (0.0008) [2023-12-26 17:41:34,698][105620] Updated weights for policy 1, policy_version 320239 (0.0007) [2023-12-26 17:41:34,761][105620] Updated weights for policy 1, policy_version 320249 (0.0006) [2023-12-26 17:41:34,828][105620] Updated weights for policy 1, policy_version 320259 (0.0005) [2023-12-26 17:41:35,327][105692] Updated weights for policy 0, policy_version 320012 (0.0008) [2023-12-26 17:41:35,386][105692] Updated weights for policy 0, policy_version 320022 (0.0008) [2023-12-26 17:41:35,445][105692] Updated weights for policy 0, policy_version 320032 (0.0008) [2023-12-26 17:41:35,477][105620] Updated weights for policy 1, policy_version 320269 (0.0008) [2023-12-26 17:41:35,542][105620] Updated weights for policy 1, policy_version 320279 (0.0010) [2023-12-26 17:41:35,586][105620] Updated weights for policy 1, policy_version 320289 (0.0010) [2023-12-26 17:41:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 163946496. Throughput: 0: 9892.8, 1: 9836.5. Samples: 163937180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:41:36,062][104569] Avg episode reward: [(0, '5787.518'), (1, '7699.627')] [2023-12-26 17:41:36,112][105692] Updated weights for policy 0, policy_version 320042 (0.0007) [2023-12-26 17:41:36,172][105692] Updated weights for policy 0, policy_version 320052 (0.0007) [2023-12-26 17:41:36,226][105692] Updated weights for policy 0, policy_version 320062 (0.0006) [2023-12-26 17:41:36,276][105692] Updated weights for policy 0, policy_version 320072 (0.0005) [2023-12-26 17:41:36,340][105620] Updated weights for policy 1, policy_version 320299 (0.0010) [2023-12-26 17:41:36,399][105620] Updated weights for policy 1, policy_version 320309 (0.0010) [2023-12-26 17:41:36,461][105620] Updated weights for policy 1, policy_version 320319 (0.0010) [2023-12-26 17:41:36,999][105692] Updated weights for policy 0, policy_version 320082 (0.0011) [2023-12-26 17:41:37,066][105692] Updated weights for policy 0, policy_version 320092 (0.0011) [2023-12-26 17:41:37,127][105620] Updated weights for policy 1, policy_version 320329 (0.0009) [2023-12-26 17:41:37,134][105692] Updated weights for policy 0, policy_version 320102 (0.0011) [2023-12-26 17:41:37,182][105620] Updated weights for policy 1, policy_version 320339 (0.0008) [2023-12-26 17:41:37,234][105620] Updated weights for policy 1, policy_version 320349 (0.0008) [2023-12-26 17:41:37,286][105620] Updated weights for policy 1, policy_version 320359 (0.0008) [2023-12-26 17:41:37,870][105692] Updated weights for policy 0, policy_version 320112 (0.0010) [2023-12-26 17:41:37,935][105692] Updated weights for policy 0, policy_version 320122 (0.0010) [2023-12-26 17:41:37,997][105692] Updated weights for policy 0, policy_version 320132 (0.0011) [2023-12-26 17:41:38,023][105620] Updated weights for policy 1, policy_version 320369 (0.0006) [2023-12-26 17:41:38,089][105620] Updated weights for policy 1, policy_version 320379 (0.0006) [2023-12-26 17:41:38,137][105620] Updated weights for policy 1, policy_version 320389 (0.0010) [2023-12-26 17:41:38,722][105692] Updated weights for policy 0, policy_version 320142 (0.0007) [2023-12-26 17:41:38,745][105620] Updated weights for policy 1, policy_version 320399 (0.0011) [2023-12-26 17:41:38,779][105692] Updated weights for policy 0, policy_version 320152 (0.0005) [2023-12-26 17:41:38,804][105620] Updated weights for policy 1, policy_version 320409 (0.0007) [2023-12-26 17:41:38,844][105692] Updated weights for policy 0, policy_version 320162 (0.0009) [2023-12-26 17:41:38,867][105620] Updated weights for policy 1, policy_version 320419 (0.0006) [2023-12-26 17:41:39,499][105692] Updated weights for policy 0, policy_version 320172 (0.0010) [2023-12-26 17:41:39,568][105692] Updated weights for policy 0, policy_version 320182 (0.0006) [2023-12-26 17:41:39,587][105620] Updated weights for policy 1, policy_version 320429 (0.0007) [2023-12-26 17:41:39,631][105692] Updated weights for policy 0, policy_version 320192 (0.0008) [2023-12-26 17:41:39,653][105620] Updated weights for policy 1, policy_version 320439 (0.0007) [2023-12-26 17:41:39,714][105620] Updated weights for policy 1, policy_version 320449 (0.0006) [2023-12-26 17:41:40,365][105692] Updated weights for policy 0, policy_version 320202 (0.0010) [2023-12-26 17:41:40,429][105620] Updated weights for policy 1, policy_version 320459 (0.0007) [2023-12-26 17:41:40,435][105692] Updated weights for policy 0, policy_version 320212 (0.0007) [2023-12-26 17:41:40,499][105692] Updated weights for policy 0, policy_version 320222 (0.0007) [2023-12-26 17:41:40,500][105620] Updated weights for policy 1, policy_version 320469 (0.0007) [2023-12-26 17:41:40,551][105620] Updated weights for policy 1, policy_version 320479 (0.0006) [2023-12-26 17:41:40,561][105692] Updated weights for policy 0, policy_version 320232 (0.0008) [2023-12-26 17:41:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 164044800. Throughput: 0: 9870.4, 1: 9946.0. Samples: 164055060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:41:41,063][104569] Avg episode reward: [(0, '8433.994'), (1, '8022.512')] [2023-12-26 17:41:41,271][105692] Updated weights for policy 0, policy_version 320242 (0.0009) [2023-12-26 17:41:41,292][105620] Updated weights for policy 1, policy_version 320489 (0.0009) [2023-12-26 17:41:41,327][105692] Updated weights for policy 0, policy_version 320252 (0.0010) [2023-12-26 17:41:41,360][105620] Updated weights for policy 1, policy_version 320499 (0.0007) [2023-12-26 17:41:41,399][105692] Updated weights for policy 0, policy_version 320262 (0.0009) [2023-12-26 17:41:41,419][105620] Updated weights for policy 1, policy_version 320509 (0.0007) [2023-12-26 17:41:41,467][105620] Updated weights for policy 1, policy_version 320519 (0.0009) [2023-12-26 17:41:42,055][105585] KL-divergence is very high: 118.5497 [2023-12-26 17:41:42,069][105585] KL-divergence is very high: 275.2712 [2023-12-26 17:41:42,095][105692] Updated weights for policy 0, policy_version 320272 (0.0008) [2023-12-26 17:41:42,110][105585] KL-divergence is very high: 121.2230 [2023-12-26 17:41:42,125][105585] KL-divergence is very high: 248.9805 [2023-12-26 17:41:42,165][105692] Updated weights for policy 0, policy_version 320282 (0.0005) [2023-12-26 17:41:42,166][105585] KL-divergence is very high: 113.0272 [2023-12-26 17:41:42,179][105585] KL-divergence is very high: 197.5244 [2023-12-26 17:41:42,218][105585] KL-divergence is very high: 101.9250 [2023-12-26 17:41:42,230][105692] Updated weights for policy 0, policy_version 320292 (0.0005) [2023-12-26 17:41:42,232][105585] KL-divergence is very high: 154.0408 [2023-12-26 17:41:42,333][105620] Updated weights for policy 1, policy_version 320529 (0.0009) [2023-12-26 17:41:42,403][105620] Updated weights for policy 1, policy_version 320539 (0.0009) [2023-12-26 17:41:42,466][105620] Updated weights for policy 1, policy_version 320549 (0.0007) [2023-12-26 17:41:42,968][105692] Updated weights for policy 0, policy_version 320302 (0.0009) [2023-12-26 17:41:43,034][105692] Updated weights for policy 0, policy_version 320312 (0.0009) [2023-12-26 17:41:43,036][105585] KL-divergence is very high: 130.6939 [2023-12-26 17:41:43,068][105620] Updated weights for policy 1, policy_version 320559 (0.0008) [2023-12-26 17:41:43,081][105585] KL-divergence is very high: 132.0174 [2023-12-26 17:41:43,093][105692] Updated weights for policy 0, policy_version 320322 (0.0006) [2023-12-26 17:41:43,116][105620] Updated weights for policy 1, policy_version 320569 (0.0005) [2023-12-26 17:41:43,170][105620] Updated weights for policy 1, policy_version 320579 (0.0006) [2023-12-26 17:41:43,803][105692] Updated weights for policy 0, policy_version 320332 (0.0007) [2023-12-26 17:41:43,866][105692] Updated weights for policy 0, policy_version 320342 (0.0005) [2023-12-26 17:41:43,892][105620] Updated weights for policy 1, policy_version 320589 (0.0007) [2023-12-26 17:41:43,923][105692] Updated weights for policy 0, policy_version 320352 (0.0009) [2023-12-26 17:41:43,942][105620] Updated weights for policy 1, policy_version 320599 (0.0007) [2023-12-26 17:41:43,993][105620] Updated weights for policy 1, policy_version 320609 (0.0009) [2023-12-26 17:41:44,615][105620] Updated weights for policy 1, policy_version 320619 (0.0009) [2023-12-26 17:41:44,671][105692] Updated weights for policy 0, policy_version 320362 (0.0007) [2023-12-26 17:41:44,673][105620] Updated weights for policy 1, policy_version 320629 (0.0009) [2023-12-26 17:41:44,720][105692] Updated weights for policy 0, policy_version 320372 (0.0006) [2023-12-26 17:41:44,730][105620] Updated weights for policy 1, policy_version 320639 (0.0007) [2023-12-26 17:41:44,773][105692] Updated weights for policy 0, policy_version 320382 (0.0006) [2023-12-26 17:41:44,829][105692] Updated weights for policy 0, policy_version 320392 (0.0008) [2023-12-26 17:41:45,460][105620] Updated weights for policy 1, policy_version 320649 (0.0007) [2023-12-26 17:41:45,525][105620] Updated weights for policy 1, policy_version 320659 (0.0009) [2023-12-26 17:41:45,590][105620] Updated weights for policy 1, policy_version 320669 (0.0009) [2023-12-26 17:41:45,628][105692] Updated weights for policy 0, policy_version 320402 (0.0008) [2023-12-26 17:41:45,654][105620] Updated weights for policy 1, policy_version 320679 (0.0007) [2023-12-26 17:41:45,689][105692] Updated weights for policy 0, policy_version 320412 (0.0009) [2023-12-26 17:41:45,742][105692] Updated weights for policy 0, policy_version 320422 (0.0010) [2023-12-26 17:41:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 164143104. Throughput: 0: 9822.3, 1: 9922.6. Samples: 164111652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:41:46,063][104569] Avg episode reward: [(0, '8633.983'), (1, '8814.680')] [2023-12-26 17:41:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000320424_82042880.pth... [2023-12-26 17:41:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000320680_82100224.pth... [2023-12-26 17:41:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000319240_81739776.pth [2023-12-26 17:41:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000319528_81805312.pth [2023-12-26 17:41:46,073][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_000320424_82042880.pth [2023-12-26 17:41:46,074][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_000320680_82100224.pth [2023-12-26 17:41:46,243][105620] Updated weights for policy 1, policy_version 320690 (0.0009) [2023-12-26 17:41:46,289][105620] Updated weights for policy 1, policy_version 320700 (0.0008) [2023-12-26 17:41:46,339][105620] Updated weights for policy 1, policy_version 320710 (0.0009) [2023-12-26 17:41:46,466][105692] Updated weights for policy 0, policy_version 320432 (0.0006) [2023-12-26 17:41:46,527][105692] Updated weights for policy 0, policy_version 320442 (0.0005) [2023-12-26 17:41:46,586][105692] Updated weights for policy 0, policy_version 320452 (0.0005) [2023-12-26 17:41:47,103][105692] Updated weights for policy 0, policy_version 320462 (0.0005) [2023-12-26 17:41:47,166][105692] Updated weights for policy 0, policy_version 320472 (0.0010) [2023-12-26 17:41:47,180][105620] Updated weights for policy 1, policy_version 320720 (0.0008) [2023-12-26 17:41:47,225][105692] Updated weights for policy 0, policy_version 320482 (0.0010) [2023-12-26 17:41:47,238][105620] Updated weights for policy 1, policy_version 320730 (0.0005) [2023-12-26 17:41:47,295][105620] Updated weights for policy 1, policy_version 320740 (0.0007) [2023-12-26 17:41:47,885][105692] Updated weights for policy 0, policy_version 320492 (0.0008) [2023-12-26 17:41:47,934][105692] Updated weights for policy 0, policy_version 320502 (0.0005) [2023-12-26 17:41:47,957][105620] Updated weights for policy 1, policy_version 320750 (0.0008) [2023-12-26 17:41:47,980][105692] Updated weights for policy 0, policy_version 320512 (0.0005) [2023-12-26 17:41:48,015][105620] Updated weights for policy 1, policy_version 320760 (0.0008) [2023-12-26 17:41:48,077][105620] Updated weights for policy 1, policy_version 320770 (0.0008) [2023-12-26 17:41:48,591][105692] Updated weights for policy 0, policy_version 320522 (0.0006) [2023-12-26 17:41:48,656][105692] Updated weights for policy 0, policy_version 320532 (0.0006) [2023-12-26 17:41:48,718][105692] Updated weights for policy 0, policy_version 320542 (0.0010) [2023-12-26 17:41:48,770][105620] Updated weights for policy 1, policy_version 320780 (0.0008) [2023-12-26 17:41:48,777][105692] Updated weights for policy 0, policy_version 320552 (0.0010) [2023-12-26 17:41:48,829][105620] Updated weights for policy 1, policy_version 320790 (0.0007) [2023-12-26 17:41:48,887][105620] Updated weights for policy 1, policy_version 320800 (0.0006) [2023-12-26 17:41:49,502][105692] Updated weights for policy 0, policy_version 320562 (0.0009) [2023-12-26 17:41:49,558][105692] Updated weights for policy 0, policy_version 320572 (0.0010) [2023-12-26 17:41:49,617][105692] Updated weights for policy 0, policy_version 320582 (0.0010) [2023-12-26 17:41:49,638][105620] Updated weights for policy 1, policy_version 320810 (0.0007) [2023-12-26 17:41:49,700][105620] Updated weights for policy 1, policy_version 320820 (0.0009) [2023-12-26 17:41:49,760][105620] Updated weights for policy 1, policy_version 320830 (0.0010) [2023-12-26 17:41:49,817][105620] Updated weights for policy 1, policy_version 320840 (0.0010) [2023-12-26 17:41:50,304][105692] Updated weights for policy 0, policy_version 320592 (0.0010) [2023-12-26 17:41:50,356][105692] Updated weights for policy 0, policy_version 320602 (0.0008) [2023-12-26 17:41:50,408][105692] Updated weights for policy 0, policy_version 320612 (0.0007) [2023-12-26 17:41:50,557][105620] Updated weights for policy 1, policy_version 320850 (0.0010) [2023-12-26 17:41:50,625][105620] Updated weights for policy 1, policy_version 320860 (0.0011) [2023-12-26 17:41:50,685][105620] Updated weights for policy 1, policy_version 320870 (0.0011) [2023-12-26 17:41:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 164241408. Throughput: 0: 9900.7, 1: 9868.0. Samples: 164231884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:41:51,063][104569] Avg episode reward: [(0, '9084.103'), (1, '9087.238')] [2023-12-26 17:41:51,169][105692] Updated weights for policy 0, policy_version 320622 (0.0009) [2023-12-26 17:41:51,235][105692] Updated weights for policy 0, policy_version 320632 (0.0006) [2023-12-26 17:41:51,300][105692] Updated weights for policy 0, policy_version 320642 (0.0006) [2023-12-26 17:41:51,526][105620] Updated weights for policy 1, policy_version 320880 (0.0008) [2023-12-26 17:41:51,577][105620] Updated weights for policy 1, policy_version 320890 (0.0007) [2023-12-26 17:41:51,643][105620] Updated weights for policy 1, policy_version 320900 (0.0008) [2023-12-26 17:41:52,010][105692] Updated weights for policy 0, policy_version 320652 (0.0008) [2023-12-26 17:41:52,074][105692] Updated weights for policy 0, policy_version 320662 (0.0009) [2023-12-26 17:41:52,144][105692] Updated weights for policy 0, policy_version 320672 (0.0010) [2023-12-26 17:41:52,342][105620] Updated weights for policy 1, policy_version 320910 (0.0009) [2023-12-26 17:41:52,413][105620] Updated weights for policy 1, policy_version 320920 (0.0009) [2023-12-26 17:41:52,466][105620] Updated weights for policy 1, policy_version 320930 (0.0009) [2023-12-26 17:41:52,934][105692] Updated weights for policy 0, policy_version 320682 (0.0010) [2023-12-26 17:41:52,997][105692] Updated weights for policy 0, policy_version 320692 (0.0009) [2023-12-26 17:41:53,053][105692] Updated weights for policy 0, policy_version 320702 (0.0008) [2023-12-26 17:41:53,115][105692] Updated weights for policy 0, policy_version 320712 (0.0010) [2023-12-26 17:41:53,174][105620] Updated weights for policy 1, policy_version 320940 (0.0008) [2023-12-26 17:41:53,238][105620] Updated weights for policy 1, policy_version 320950 (0.0009) [2023-12-26 17:41:53,283][105620] Updated weights for policy 1, policy_version 320960 (0.0009) [2023-12-26 17:41:53,881][105692] Updated weights for policy 0, policy_version 320722 (0.0009) [2023-12-26 17:41:53,936][105692] Updated weights for policy 0, policy_version 320732 (0.0009) [2023-12-26 17:41:53,999][105692] Updated weights for policy 0, policy_version 320742 (0.0010) [2023-12-26 17:41:54,025][105620] Updated weights for policy 1, policy_version 320970 (0.0007) [2023-12-26 17:41:54,088][105620] Updated weights for policy 1, policy_version 320980 (0.0005) [2023-12-26 17:41:54,155][105620] Updated weights for policy 1, policy_version 320990 (0.0007) [2023-12-26 17:41:54,212][105620] Updated weights for policy 1, policy_version 321000 (0.0009) [2023-12-26 17:41:54,772][105620] Updated weights for policy 1, policy_version 321010 (0.0005) [2023-12-26 17:41:54,823][105620] Updated weights for policy 1, policy_version 321020 (0.0007) [2023-12-26 17:41:54,845][105692] Updated weights for policy 0, policy_version 320752 (0.0007) [2023-12-26 17:41:54,879][105620] Updated weights for policy 1, policy_version 321030 (0.0007) [2023-12-26 17:41:54,898][105692] Updated weights for policy 0, policy_version 320762 (0.0007) [2023-12-26 17:41:54,961][105692] Updated weights for policy 0, policy_version 320772 (0.0009) [2023-12-26 17:41:55,563][105620] Updated weights for policy 1, policy_version 321040 (0.0006) [2023-12-26 17:41:55,621][105620] Updated weights for policy 1, policy_version 321050 (0.0008) [2023-12-26 17:41:55,682][105620] Updated weights for policy 1, policy_version 321060 (0.0009) [2023-12-26 17:41:55,763][105692] Updated weights for policy 0, policy_version 320782 (0.0009) [2023-12-26 17:41:55,810][105692] Updated weights for policy 0, policy_version 320792 (0.0009) [2023-12-26 17:41:55,872][105692] Updated weights for policy 0, policy_version 320802 (0.0009) [2023-12-26 17:41:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 164339712. Throughput: 0: 9915.1, 1: 9900.4. Samples: 164346144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:41:56,063][104569] Avg episode reward: [(0, '3812.620'), (1, '9087.231')] [2023-12-26 17:41:56,406][105620] Updated weights for policy 1, policy_version 321070 (0.0009) [2023-12-26 17:41:56,458][105620] Updated weights for policy 1, policy_version 321080 (0.0008) [2023-12-26 17:41:56,503][105620] Updated weights for policy 1, policy_version 321090 (0.0009) [2023-12-26 17:41:56,628][105692] Updated weights for policy 0, policy_version 320812 (0.0009) [2023-12-26 17:41:56,688][105692] Updated weights for policy 0, policy_version 320822 (0.0009) [2023-12-26 17:41:56,750][105692] Updated weights for policy 0, policy_version 320832 (0.0008) [2023-12-26 17:41:57,129][105620] Updated weights for policy 1, policy_version 321100 (0.0007) [2023-12-26 17:41:57,179][105620] Updated weights for policy 1, policy_version 321110 (0.0005) [2023-12-26 17:41:57,240][105620] Updated weights for policy 1, policy_version 321120 (0.0005) [2023-12-26 17:41:57,571][105692] Updated weights for policy 0, policy_version 320842 (0.0008) [2023-12-26 17:41:57,639][105692] Updated weights for policy 0, policy_version 320852 (0.0005) [2023-12-26 17:41:57,703][105692] Updated weights for policy 0, policy_version 320862 (0.0005) [2023-12-26 17:41:57,771][105692] Updated weights for policy 0, policy_version 320872 (0.0005) [2023-12-26 17:41:57,810][105620] Updated weights for policy 1, policy_version 321130 (0.0006) [2023-12-26 17:41:57,863][105620] Updated weights for policy 1, policy_version 321140 (0.0005) [2023-12-26 17:41:57,915][105620] Updated weights for policy 1, policy_version 321150 (0.0005) [2023-12-26 17:41:57,965][105620] Updated weights for policy 1, policy_version 321160 (0.0007) [2023-12-26 17:41:58,401][105692] Updated weights for policy 0, policy_version 320882 (0.0010) [2023-12-26 17:41:58,465][105692] Updated weights for policy 0, policy_version 320892 (0.0009) [2023-12-26 17:41:58,530][105692] Updated weights for policy 0, policy_version 320902 (0.0009) [2023-12-26 17:41:58,719][105620] Updated weights for policy 1, policy_version 321170 (0.0007) [2023-12-26 17:41:58,791][105620] Updated weights for policy 1, policy_version 321180 (0.0008) [2023-12-26 17:41:58,860][105620] Updated weights for policy 1, policy_version 321190 (0.0008) [2023-12-26 17:41:59,364][105692] Updated weights for policy 0, policy_version 320912 (0.0007) [2023-12-26 17:41:59,421][105692] Updated weights for policy 0, policy_version 320922 (0.0006) [2023-12-26 17:41:59,472][105692] Updated weights for policy 0, policy_version 320932 (0.0005) [2023-12-26 17:41:59,545][105620] Updated weights for policy 1, policy_version 321200 (0.0009) [2023-12-26 17:41:59,608][105620] Updated weights for policy 1, policy_version 321210 (0.0009) [2023-12-26 17:41:59,677][105620] Updated weights for policy 1, policy_version 321220 (0.0009) [2023-12-26 17:42:00,079][105692] Updated weights for policy 0, policy_version 320942 (0.0008) [2023-12-26 17:42:00,129][105692] Updated weights for policy 0, policy_version 320952 (0.0008) [2023-12-26 17:42:00,188][105692] Updated weights for policy 0, policy_version 320962 (0.0008) [2023-12-26 17:42:00,314][105620] Updated weights for policy 1, policy_version 321230 (0.0008) [2023-12-26 17:42:00,367][105620] Updated weights for policy 1, policy_version 321240 (0.0005) [2023-12-26 17:42:00,434][105620] Updated weights for policy 1, policy_version 321250 (0.0006) [2023-12-26 17:42:00,927][105692] Updated weights for policy 0, policy_version 320972 (0.0007) [2023-12-26 17:42:00,983][105692] Updated weights for policy 0, policy_version 320982 (0.0009) [2023-12-26 17:42:01,024][105620] Updated weights for policy 1, policy_version 321260 (0.0006) [2023-12-26 17:42:01,038][105692] Updated weights for policy 0, policy_version 320992 (0.0008) [2023-12-26 17:42:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 164429824. Throughput: 0: 9928.1, 1: 9931.9. Samples: 164404648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:42:01,063][104569] Avg episode reward: [(0, '4927.968'), (1, '9175.782')] [2023-12-26 17:42:01,084][105620] Updated weights for policy 1, policy_version 321270 (0.0008) [2023-12-26 17:42:01,084][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000321000_82190336.pth... [2023-12-26 17:42:01,089][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000319848_81895424.pth [2023-12-26 17:42:01,142][105620] Updated weights for policy 1, policy_version 321280 (0.0008) [2023-12-26 17:42:01,195][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000321288_82255872.pth... [2023-12-26 17:42:01,201][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000320104_81952768.pth [2023-12-26 17:42:01,789][105692] Updated weights for policy 0, policy_version 321002 (0.0008) [2023-12-26 17:42:01,838][105620] Updated weights for policy 1, policy_version 321290 (0.0006) [2023-12-26 17:42:01,843][105692] Updated weights for policy 0, policy_version 321012 (0.0008) [2023-12-26 17:42:01,893][105692] Updated weights for policy 0, policy_version 321022 (0.0007) [2023-12-26 17:42:01,894][105620] Updated weights for policy 1, policy_version 321300 (0.0006) [2023-12-26 17:42:01,942][105692] Updated weights for policy 0, policy_version 321032 (0.0007) [2023-12-26 17:42:01,952][105620] Updated weights for policy 1, policy_version 321310 (0.0006) [2023-12-26 17:42:02,021][105620] Updated weights for policy 1, policy_version 321320 (0.0005) [2023-12-26 17:42:02,674][105692] Updated weights for policy 0, policy_version 321042 (0.0005) [2023-12-26 17:42:02,696][105620] Updated weights for policy 1, policy_version 321330 (0.0010) [2023-12-26 17:42:02,728][105692] Updated weights for policy 0, policy_version 321052 (0.0007) [2023-12-26 17:42:02,750][105620] Updated weights for policy 1, policy_version 321340 (0.0010) [2023-12-26 17:42:02,784][105692] Updated weights for policy 0, policy_version 321062 (0.0005) [2023-12-26 17:42:02,812][105620] Updated weights for policy 1, policy_version 321350 (0.0010) [2023-12-26 17:42:03,502][105692] Updated weights for policy 0, policy_version 321072 (0.0007) [2023-12-26 17:42:03,518][105620] Updated weights for policy 1, policy_version 321360 (0.0008) [2023-12-26 17:42:03,564][105692] Updated weights for policy 0, policy_version 321082 (0.0007) [2023-12-26 17:42:03,568][105620] Updated weights for policy 1, policy_version 321370 (0.0008) [2023-12-26 17:42:03,614][105692] Updated weights for policy 0, policy_version 321092 (0.0008) [2023-12-26 17:42:03,620][105620] Updated weights for policy 1, policy_version 321380 (0.0010) [2023-12-26 17:42:04,308][105620] Updated weights for policy 1, policy_version 321390 (0.0008) [2023-12-26 17:42:04,336][105692] Updated weights for policy 0, policy_version 321102 (0.0009) [2023-12-26 17:42:04,371][105620] Updated weights for policy 1, policy_version 321400 (0.0007) [2023-12-26 17:42:04,399][105692] Updated weights for policy 0, policy_version 321112 (0.0008) [2023-12-26 17:42:04,434][105620] Updated weights for policy 1, policy_version 321410 (0.0008) [2023-12-26 17:42:04,463][105692] Updated weights for policy 0, policy_version 321122 (0.0008) [2023-12-26 17:42:05,058][105620] Updated weights for policy 1, policy_version 321420 (0.0008) [2023-12-26 17:42:05,119][105692] Updated weights for policy 0, policy_version 321132 (0.0006) [2023-12-26 17:42:05,124][105620] Updated weights for policy 1, policy_version 321430 (0.0006) [2023-12-26 17:42:05,171][105692] Updated weights for policy 0, policy_version 321142 (0.0006) [2023-12-26 17:42:05,194][105620] Updated weights for policy 1, policy_version 321440 (0.0005) [2023-12-26 17:42:05,218][105692] Updated weights for policy 0, policy_version 321152 (0.0005) [2023-12-26 17:42:05,700][105620] Updated weights for policy 1, policy_version 321450 (0.0005) [2023-12-26 17:42:05,749][105620] Updated weights for policy 1, policy_version 321460 (0.0005) [2023-12-26 17:42:05,799][105620] Updated weights for policy 1, policy_version 321470 (0.0006) [2023-12-26 17:42:05,849][105620] Updated weights for policy 1, policy_version 321480 (0.0009) [2023-12-26 17:42:05,897][105692] Updated weights for policy 0, policy_version 321162 (0.0006) [2023-12-26 17:42:05,958][105692] Updated weights for policy 0, policy_version 321172 (0.0008) [2023-12-26 17:42:06,016][105692] Updated weights for policy 0, policy_version 321182 (0.0007) [2023-12-26 17:42:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.7, 300 sec: 19660.8). Total num frames: 164536320. Throughput: 0: 9825.9, 1: 9920.1. Samples: 164525076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:42:06,063][104569] Avg episode reward: [(0, '501.449'), (1, '9087.583')] [2023-12-26 17:42:06,069][105692] Updated weights for policy 0, policy_version 321192 (0.0010) [2023-12-26 17:42:06,460][105620] Updated weights for policy 1, policy_version 321490 (0.0008) [2023-12-26 17:42:06,523][105620] Updated weights for policy 1, policy_version 321500 (0.0009) [2023-12-26 17:42:06,577][105620] Updated weights for policy 1, policy_version 321510 (0.0008) [2023-12-26 17:42:06,799][105692] Updated weights for policy 0, policy_version 321202 (0.0006) [2023-12-26 17:42:06,857][105692] Updated weights for policy 0, policy_version 321212 (0.0009) [2023-12-26 17:42:06,918][105692] Updated weights for policy 0, policy_version 321222 (0.0009) [2023-12-26 17:42:07,331][105620] Updated weights for policy 1, policy_version 321520 (0.0010) [2023-12-26 17:42:07,400][105620] Updated weights for policy 1, policy_version 321530 (0.0010) [2023-12-26 17:42:07,462][105620] Updated weights for policy 1, policy_version 321540 (0.0007) [2023-12-26 17:42:07,608][105692] Updated weights for policy 0, policy_version 321232 (0.0009) [2023-12-26 17:42:07,656][105692] Updated weights for policy 0, policy_version 321242 (0.0009) [2023-12-26 17:42:07,703][105692] Updated weights for policy 0, policy_version 321252 (0.0009) [2023-12-26 17:42:08,112][105620] Updated weights for policy 1, policy_version 321550 (0.0005) [2023-12-26 17:42:08,167][105620] Updated weights for policy 1, policy_version 321560 (0.0010) [2023-12-26 17:42:08,219][105620] Updated weights for policy 1, policy_version 321570 (0.0008) [2023-12-26 17:42:08,510][105692] Updated weights for policy 0, policy_version 321262 (0.0007) [2023-12-26 17:42:08,568][105692] Updated weights for policy 0, policy_version 321272 (0.0009) [2023-12-26 17:42:08,629][105692] Updated weights for policy 0, policy_version 321282 (0.0007) [2023-12-26 17:42:08,927][105620] Updated weights for policy 1, policy_version 321580 (0.0006) [2023-12-26 17:42:09,022][105620] Updated weights for policy 1, policy_version 321590 (0.0007) [2023-12-26 17:42:09,085][105620] Updated weights for policy 1, policy_version 321600 (0.0010) [2023-12-26 17:42:09,263][105692] Updated weights for policy 0, policy_version 321292 (0.0008) [2023-12-26 17:42:09,320][105692] Updated weights for policy 0, policy_version 321302 (0.0010) [2023-12-26 17:42:09,381][105692] Updated weights for policy 0, policy_version 321312 (0.0007) [2023-12-26 17:42:09,741][105620] Updated weights for policy 1, policy_version 321610 (0.0009) [2023-12-26 17:42:09,814][105620] Updated weights for policy 1, policy_version 321620 (0.0006) [2023-12-26 17:42:09,879][105620] Updated weights for policy 1, policy_version 321630 (0.0012) [2023-12-26 17:42:09,934][105620] Updated weights for policy 1, policy_version 321640 (0.0010) [2023-12-26 17:42:10,167][105692] Updated weights for policy 0, policy_version 321322 (0.0009) [2023-12-26 17:42:10,237][105692] Updated weights for policy 0, policy_version 321332 (0.0011) [2023-12-26 17:42:10,301][105692] Updated weights for policy 0, policy_version 321342 (0.0011) [2023-12-26 17:42:10,373][105692] Updated weights for policy 0, policy_version 321352 (0.0011) [2023-12-26 17:42:10,574][105620] Updated weights for policy 1, policy_version 321650 (0.0009) [2023-12-26 17:42:10,638][105620] Updated weights for policy 1, policy_version 321660 (0.0011) [2023-12-26 17:42:10,706][105620] Updated weights for policy 1, policy_version 321670 (0.0010) [2023-12-26 17:42:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 164634624. Throughput: 0: 9835.6, 1: 10088.0. Samples: 164646692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:42:11,063][104569] Avg episode reward: [(0, '520.080'), (1, '9177.533')] [2023-12-26 17:42:11,123][105692] Updated weights for policy 0, policy_version 321362 (0.0009) [2023-12-26 17:42:11,200][105692] Updated weights for policy 0, policy_version 321372 (0.0010) [2023-12-26 17:42:11,256][105692] Updated weights for policy 0, policy_version 321382 (0.0011) [2023-12-26 17:42:11,376][105620] Updated weights for policy 1, policy_version 321680 (0.0008) [2023-12-26 17:42:11,442][105620] Updated weights for policy 1, policy_version 321690 (0.0009) [2023-12-26 17:42:11,511][105620] Updated weights for policy 1, policy_version 321700 (0.0008) [2023-12-26 17:42:12,058][105692] Updated weights for policy 0, policy_version 321392 (0.0011) [2023-12-26 17:42:12,122][105692] Updated weights for policy 0, policy_version 321402 (0.0010) [2023-12-26 17:42:12,176][105692] Updated weights for policy 0, policy_version 321412 (0.0006) [2023-12-26 17:42:12,301][105620] Updated weights for policy 1, policy_version 321710 (0.0008) [2023-12-26 17:42:12,365][105620] Updated weights for policy 1, policy_version 321720 (0.0008) [2023-12-26 17:42:12,402][105586] KL-divergence is very high: 158.0919 [2023-12-26 17:42:12,427][105620] Updated weights for policy 1, policy_version 321730 (0.0008) [2023-12-26 17:42:12,450][105586] KL-divergence is very high: 169.7281 [2023-12-26 17:42:12,813][105692] Updated weights for policy 0, policy_version 321422 (0.0011) [2023-12-26 17:42:12,878][105692] Updated weights for policy 0, policy_version 321432 (0.0010) [2023-12-26 17:42:12,937][105692] Updated weights for policy 0, policy_version 321442 (0.0011) [2023-12-26 17:42:13,212][105620] Updated weights for policy 1, policy_version 321740 (0.0009) [2023-12-26 17:42:13,266][105620] Updated weights for policy 1, policy_version 321750 (0.0008) [2023-12-26 17:42:13,321][105620] Updated weights for policy 1, policy_version 321760 (0.0008) [2023-12-26 17:42:13,666][105692] Updated weights for policy 0, policy_version 321452 (0.0010) [2023-12-26 17:42:13,723][105692] Updated weights for policy 0, policy_version 321462 (0.0008) [2023-12-26 17:42:13,784][105692] Updated weights for policy 0, policy_version 321472 (0.0009) [2023-12-26 17:42:13,996][105620] Updated weights for policy 1, policy_version 321770 (0.0008) [2023-12-26 17:42:14,047][105620] Updated weights for policy 1, policy_version 321780 (0.0008) [2023-12-26 17:42:14,101][105620] Updated weights for policy 1, policy_version 321790 (0.0009) [2023-12-26 17:42:14,173][105620] Updated weights for policy 1, policy_version 321800 (0.0009) [2023-12-26 17:42:14,581][105692] Updated weights for policy 0, policy_version 321482 (0.0010) [2023-12-26 17:42:14,637][105692] Updated weights for policy 0, policy_version 321492 (0.0006) [2023-12-26 17:42:14,695][105692] Updated weights for policy 0, policy_version 321502 (0.0010) [2023-12-26 17:42:14,753][105692] Updated weights for policy 0, policy_version 321512 (0.0010) [2023-12-26 17:42:14,832][105620] Updated weights for policy 1, policy_version 321810 (0.0008) [2023-12-26 17:42:14,894][105620] Updated weights for policy 1, policy_version 321820 (0.0006) [2023-12-26 17:42:14,953][105620] Updated weights for policy 1, policy_version 321830 (0.0006) [2023-12-26 17:42:15,500][105692] Updated weights for policy 0, policy_version 321522 (0.0011) [2023-12-26 17:42:15,519][105620] Updated weights for policy 1, policy_version 321840 (0.0006) [2023-12-26 17:42:15,553][105692] Updated weights for policy 0, policy_version 321532 (0.0011) [2023-12-26 17:42:15,585][105620] Updated weights for policy 1, policy_version 321850 (0.0006) [2023-12-26 17:42:15,606][105692] Updated weights for policy 0, policy_version 321542 (0.0011) [2023-12-26 17:42:15,650][105620] Updated weights for policy 1, policy_version 321860 (0.0006) [2023-12-26 17:42:16,062][104569] Fps is (10 sec: 19659.9, 60 sec: 19797.1, 300 sec: 19660.8). Total num frames: 164732928. Throughput: 0: 9687.5, 1: 10010.2. Samples: 164702576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:42:16,064][104569] Avg episode reward: [(0, '740.624'), (1, '9174.340')] [2023-12-26 17:42:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000321544_82329600.pth... [2023-12-26 17:42:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000321864_82403328.pth... [2023-12-26 17:42:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000320424_82042880.pth [2023-12-26 17:42:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000320680_82100224.pth [2023-12-26 17:42:16,243][105620] Updated weights for policy 1, policy_version 321870 (0.0009) [2023-12-26 17:42:16,295][105620] Updated weights for policy 1, policy_version 321880 (0.0010) [2023-12-26 17:42:16,344][105620] Updated weights for policy 1, policy_version 321890 (0.0010) [2023-12-26 17:42:16,355][105586] KL-divergence is very high: 210.6360 [2023-12-26 17:42:16,377][105692] Updated weights for policy 0, policy_version 321552 (0.0007) [2023-12-26 17:42:16,439][105692] Updated weights for policy 0, policy_version 321562 (0.0007) [2023-12-26 17:42:16,487][105692] Updated weights for policy 0, policy_version 321572 (0.0010) [2023-12-26 17:42:16,996][105620] Updated weights for policy 1, policy_version 321900 (0.0009) [2023-12-26 17:42:17,068][105620] Updated weights for policy 1, policy_version 321910 (0.0005) [2023-12-26 17:42:17,111][105692] Updated weights for policy 0, policy_version 321582 (0.0009) [2023-12-26 17:42:17,136][105620] Updated weights for policy 1, policy_version 321920 (0.0010) [2023-12-26 17:42:17,163][105692] Updated weights for policy 0, policy_version 321592 (0.0008) [2023-12-26 17:42:17,212][105692] Updated weights for policy 0, policy_version 321602 (0.0010) [2023-12-26 17:42:17,712][105620] Updated weights for policy 1, policy_version 321930 (0.0010) [2023-12-26 17:42:17,769][105620] Updated weights for policy 1, policy_version 321940 (0.0005) [2023-12-26 17:42:17,826][105620] Updated weights for policy 1, policy_version 321950 (0.0005) [2023-12-26 17:42:17,888][105620] Updated weights for policy 1, policy_version 321960 (0.0005) [2023-12-26 17:42:17,936][105692] Updated weights for policy 0, policy_version 321612 (0.0010) [2023-12-26 17:42:18,003][105692] Updated weights for policy 0, policy_version 321622 (0.0010) [2023-12-26 17:42:18,062][105692] Updated weights for policy 0, policy_version 321632 (0.0010) [2023-12-26 17:42:18,489][105620] Updated weights for policy 1, policy_version 321970 (0.0008) [2023-12-26 17:42:18,534][105620] Updated weights for policy 1, policy_version 321980 (0.0006) [2023-12-26 17:42:18,586][105620] Updated weights for policy 1, policy_version 321990 (0.0007) [2023-12-26 17:42:18,802][105692] Updated weights for policy 0, policy_version 321642 (0.0010) [2023-12-26 17:42:18,850][105692] Updated weights for policy 0, policy_version 321652 (0.0010) [2023-12-26 17:42:18,901][105692] Updated weights for policy 0, policy_version 321662 (0.0010) [2023-12-26 17:42:18,960][105692] Updated weights for policy 0, policy_version 321672 (0.0010) [2023-12-26 17:42:19,275][105620] Updated weights for policy 1, policy_version 322000 (0.0008) [2023-12-26 17:42:19,344][105620] Updated weights for policy 1, policy_version 322010 (0.0007) [2023-12-26 17:42:19,410][105620] Updated weights for policy 1, policy_version 322020 (0.0009) [2023-12-26 17:42:19,772][105692] Updated weights for policy 0, policy_version 321682 (0.0010) [2023-12-26 17:42:19,837][105692] Updated weights for policy 0, policy_version 321692 (0.0010) [2023-12-26 17:42:19,908][105692] Updated weights for policy 0, policy_version 321702 (0.0010) [2023-12-26 17:42:20,084][105620] Updated weights for policy 1, policy_version 322030 (0.0007) [2023-12-26 17:42:20,131][105620] Updated weights for policy 1, policy_version 322040 (0.0006) [2023-12-26 17:42:20,181][105620] Updated weights for policy 1, policy_version 322050 (0.0005) [2023-12-26 17:42:20,594][105692] Updated weights for policy 0, policy_version 321712 (0.0007) [2023-12-26 17:42:20,651][105692] Updated weights for policy 0, policy_version 321722 (0.0006) [2023-12-26 17:42:20,719][105692] Updated weights for policy 0, policy_version 321732 (0.0008) [2023-12-26 17:42:20,940][105620] Updated weights for policy 1, policy_version 322060 (0.0009) [2023-12-26 17:42:21,007][105620] Updated weights for policy 1, policy_version 322070 (0.0010) [2023-12-26 17:42:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 164831232. Throughput: 0: 9612.5, 1: 10106.8. Samples: 164824548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:42:21,063][104569] Avg episode reward: [(0, '5899.039'), (1, '9174.334')] [2023-12-26 17:42:21,076][105620] Updated weights for policy 1, policy_version 322080 (0.0008) [2023-12-26 17:42:21,459][105692] Updated weights for policy 0, policy_version 321742 (0.0008) [2023-12-26 17:42:21,501][105585] KL-divergence is very high: 123.5077 [2023-12-26 17:42:21,506][105585] KL-divergence is very high: 128.2291 [2023-12-26 17:42:21,518][105692] Updated weights for policy 0, policy_version 321752 (0.0009) [2023-12-26 17:42:21,518][105585] KL-divergence is very high: 128.1585 [2023-12-26 17:42:21,577][105692] Updated weights for policy 0, policy_version 321762 (0.0008) [2023-12-26 17:42:21,743][105620] Updated weights for policy 1, policy_version 322090 (0.0007) [2023-12-26 17:42:21,806][105620] Updated weights for policy 1, policy_version 322100 (0.0009) [2023-12-26 17:42:21,871][105620] Updated weights for policy 1, policy_version 322110 (0.0006) [2023-12-26 17:42:21,933][105620] Updated weights for policy 1, policy_version 322120 (0.0006) [2023-12-26 17:42:22,380][105692] Updated weights for policy 0, policy_version 321772 (0.0008) [2023-12-26 17:42:22,440][105692] Updated weights for policy 0, policy_version 321782 (0.0010) [2023-12-26 17:42:22,506][105692] Updated weights for policy 0, policy_version 321792 (0.0009) [2023-12-26 17:42:22,620][105620] Updated weights for policy 1, policy_version 322130 (0.0006) [2023-12-26 17:42:22,683][105620] Updated weights for policy 1, policy_version 322140 (0.0007) [2023-12-26 17:42:22,750][105620] Updated weights for policy 1, policy_version 322150 (0.0007) [2023-12-26 17:42:23,215][105692] Updated weights for policy 0, policy_version 321802 (0.0010) [2023-12-26 17:42:23,277][105692] Updated weights for policy 0, policy_version 321812 (0.0005) [2023-12-26 17:42:23,337][105692] Updated weights for policy 0, policy_version 321822 (0.0005) [2023-12-26 17:42:23,394][105692] Updated weights for policy 0, policy_version 321832 (0.0007) [2023-12-26 17:42:23,530][105620] Updated weights for policy 1, policy_version 322160 (0.0009) [2023-12-26 17:42:23,588][105620] Updated weights for policy 1, policy_version 322170 (0.0009) [2023-12-26 17:42:23,654][105620] Updated weights for policy 1, policy_version 322180 (0.0007) [2023-12-26 17:42:24,168][105692] Updated weights for policy 0, policy_version 321842 (0.0007) [2023-12-26 17:42:24,213][105620] Updated weights for policy 1, policy_version 322190 (0.0006) [2023-12-26 17:42:24,236][105692] Updated weights for policy 0, policy_version 321852 (0.0007) [2023-12-26 17:42:24,272][105620] Updated weights for policy 1, policy_version 322200 (0.0006) [2023-12-26 17:42:24,306][105692] Updated weights for policy 0, policy_version 321862 (0.0005) [2023-12-26 17:42:24,329][105620] Updated weights for policy 1, policy_version 322210 (0.0006) [2023-12-26 17:42:24,857][105692] Updated weights for policy 0, policy_version 321872 (0.0007) [2023-12-26 17:42:24,914][105692] Updated weights for policy 0, policy_version 321882 (0.0009) [2023-12-26 17:42:24,971][105692] Updated weights for policy 0, policy_version 321892 (0.0006) [2023-12-26 17:42:25,087][105620] Updated weights for policy 1, policy_version 322220 (0.0007) [2023-12-26 17:42:25,140][105620] Updated weights for policy 1, policy_version 322230 (0.0009) [2023-12-26 17:42:25,207][105620] Updated weights for policy 1, policy_version 322240 (0.0010) [2023-12-26 17:42:25,544][105692] Updated weights for policy 0, policy_version 321902 (0.0007) [2023-12-26 17:42:25,593][105692] Updated weights for policy 0, policy_version 321912 (0.0009) [2023-12-26 17:42:25,654][105692] Updated weights for policy 0, policy_version 321922 (0.0005) [2023-12-26 17:42:26,030][105620] Updated weights for policy 1, policy_version 322250 (0.0009) [2023-12-26 17:42:26,062][104569] Fps is (10 sec: 19661.9, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 164929536. Throughput: 0: 9608.5, 1: 10068.7. Samples: 164940532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:42:26,062][104569] Avg episode reward: [(0, '3170.266'), (1, '9174.432')] [2023-12-26 17:42:26,086][105620] Updated weights for policy 1, policy_version 322260 (0.0009) [2023-12-26 17:42:26,141][105620] Updated weights for policy 1, policy_version 322270 (0.0009) [2023-12-26 17:42:26,203][105620] Updated weights for policy 1, policy_version 322280 (0.0008) [2023-12-26 17:42:26,204][105692] Updated weights for policy 0, policy_version 321932 (0.0007) [2023-12-26 17:42:26,266][105692] Updated weights for policy 0, policy_version 321942 (0.0009) [2023-12-26 17:42:26,320][105692] Updated weights for policy 0, policy_version 321952 (0.0009) [2023-12-26 17:42:26,899][105620] Updated weights for policy 1, policy_version 322290 (0.0005) [2023-12-26 17:42:26,955][105620] Updated weights for policy 1, policy_version 322300 (0.0005) [2023-12-26 17:42:27,011][105620] Updated weights for policy 1, policy_version 322310 (0.0005) [2023-12-26 17:42:27,034][105692] Updated weights for policy 0, policy_version 321962 (0.0008) [2023-12-26 17:42:27,082][105692] Updated weights for policy 0, policy_version 321972 (0.0008) [2023-12-26 17:42:27,136][105692] Updated weights for policy 0, policy_version 321982 (0.0009) [2023-12-26 17:42:27,190][105692] Updated weights for policy 0, policy_version 321992 (0.0009) [2023-12-26 17:42:27,584][105620] Updated weights for policy 1, policy_version 322320 (0.0009) [2023-12-26 17:42:27,645][105620] Updated weights for policy 1, policy_version 322330 (0.0009) [2023-12-26 17:42:27,702][105620] Updated weights for policy 1, policy_version 322340 (0.0009) [2023-12-26 17:42:27,827][105692] Updated weights for policy 0, policy_version 322002 (0.0007) [2023-12-26 17:42:27,875][105692] Updated weights for policy 0, policy_version 322012 (0.0008) [2023-12-26 17:42:27,933][105692] Updated weights for policy 0, policy_version 322022 (0.0009) [2023-12-26 17:42:28,401][105620] Updated weights for policy 1, policy_version 322350 (0.0007) [2023-12-26 17:42:28,460][105620] Updated weights for policy 1, policy_version 322360 (0.0009) [2023-12-26 17:42:28,510][105620] Updated weights for policy 1, policy_version 322370 (0.0008) [2023-12-26 17:42:28,685][105692] Updated weights for policy 0, policy_version 322032 (0.0010) [2023-12-26 17:42:28,746][105692] Updated weights for policy 0, policy_version 322042 (0.0011) [2023-12-26 17:42:28,790][105692] Updated weights for policy 0, policy_version 322052 (0.0010) [2023-12-26 17:42:29,335][105620] Updated weights for policy 1, policy_version 322380 (0.0009) [2023-12-26 17:42:29,377][105692] Updated weights for policy 0, policy_version 322062 (0.0009) [2023-12-26 17:42:29,393][105620] Updated weights for policy 1, policy_version 322390 (0.0008) [2023-12-26 17:42:29,439][105692] Updated weights for policy 0, policy_version 322072 (0.0007) [2023-12-26 17:42:29,451][105620] Updated weights for policy 1, policy_version 322400 (0.0007) [2023-12-26 17:42:29,504][105692] Updated weights for policy 0, policy_version 322082 (0.0005) [2023-12-26 17:42:30,139][105692] Updated weights for policy 0, policy_version 322092 (0.0006) [2023-12-26 17:42:30,194][105692] Updated weights for policy 0, policy_version 322102 (0.0009) [2023-12-26 17:42:30,248][105692] Updated weights for policy 0, policy_version 322112 (0.0009) [2023-12-26 17:42:30,261][105620] Updated weights for policy 1, policy_version 322410 (0.0010) [2023-12-26 17:42:30,316][105620] Updated weights for policy 1, policy_version 322420 (0.0007) [2023-12-26 17:42:30,381][105620] Updated weights for policy 1, policy_version 322430 (0.0009) [2023-12-26 17:42:30,438][105620] Updated weights for policy 1, policy_version 322440 (0.0008) [2023-12-26 17:42:30,987][105692] Updated weights for policy 0, policy_version 322122 (0.0007) [2023-12-26 17:42:31,043][105692] Updated weights for policy 0, policy_version 322132 (0.0009) [2023-12-26 17:42:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 165027840. Throughput: 0: 9680.1, 1: 10123.4. Samples: 165002808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:42:31,062][104569] Avg episode reward: [(0, '6074.841'), (1, '9356.715')] [2023-12-26 17:42:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000322440_82550784.pth... [2023-12-26 17:42:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000321288_82255872.pth [2023-12-26 17:42:31,094][105692] Updated weights for policy 0, policy_version 322142 (0.0008) [2023-12-26 17:42:31,161][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000322152_82485248.pth... [2023-12-26 17:42:31,162][105692] Updated weights for policy 0, policy_version 322152 (0.0009) [2023-12-26 17:42:31,165][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000321000_82190336.pth [2023-12-26 17:42:31,184][105620] Updated weights for policy 1, policy_version 322450 (0.0006) [2023-12-26 17:42:31,247][105620] Updated weights for policy 1, policy_version 322460 (0.0006) [2023-12-26 17:42:31,316][105620] Updated weights for policy 1, policy_version 322470 (0.0006) [2023-12-26 17:42:31,913][105692] Updated weights for policy 0, policy_version 322162 (0.0006) [2023-12-26 17:42:31,969][105692] Updated weights for policy 0, policy_version 322172 (0.0006) [2023-12-26 17:42:32,004][105620] Updated weights for policy 1, policy_version 322480 (0.0007) [2023-12-26 17:42:32,022][105692] Updated weights for policy 0, policy_version 322182 (0.0006) [2023-12-26 17:42:32,069][105620] Updated weights for policy 1, policy_version 322490 (0.0005) [2023-12-26 17:42:32,140][105620] Updated weights for policy 1, policy_version 322500 (0.0007) [2023-12-26 17:42:32,619][105692] Updated weights for policy 0, policy_version 322192 (0.0009) [2023-12-26 17:42:32,676][105692] Updated weights for policy 0, policy_version 322202 (0.0010) [2023-12-26 17:42:32,727][105692] Updated weights for policy 0, policy_version 322212 (0.0009) [2023-12-26 17:42:32,789][105620] Updated weights for policy 1, policy_version 322510 (0.0009) [2023-12-26 17:42:32,846][105620] Updated weights for policy 1, policy_version 322520 (0.0008) [2023-12-26 17:42:32,901][105620] Updated weights for policy 1, policy_version 322530 (0.0009) [2023-12-26 17:42:33,496][105692] Updated weights for policy 0, policy_version 322222 (0.0006) [2023-12-26 17:42:33,542][105692] Updated weights for policy 0, policy_version 322232 (0.0005) [2023-12-26 17:42:33,591][105692] Updated weights for policy 0, policy_version 322242 (0.0005) [2023-12-26 17:42:33,634][105620] Updated weights for policy 1, policy_version 322540 (0.0008) [2023-12-26 17:42:33,692][105620] Updated weights for policy 1, policy_version 322550 (0.0007) [2023-12-26 17:42:33,750][105620] Updated weights for policy 1, policy_version 322560 (0.0005) [2023-12-26 17:42:34,120][105692] Updated weights for policy 0, policy_version 322252 (0.0005) [2023-12-26 17:42:34,186][105692] Updated weights for policy 0, policy_version 322262 (0.0008) [2023-12-26 17:42:34,244][105692] Updated weights for policy 0, policy_version 322272 (0.0009) [2023-12-26 17:42:34,521][105620] Updated weights for policy 1, policy_version 322570 (0.0006) [2023-12-26 17:42:34,572][105620] Updated weights for policy 1, policy_version 322580 (0.0010) [2023-12-26 17:42:34,625][105620] Updated weights for policy 1, policy_version 322591 (0.0009) [2023-12-26 17:42:34,861][105692] Updated weights for policy 0, policy_version 322282 (0.0007) [2023-12-26 17:42:34,919][105692] Updated weights for policy 0, policy_version 322292 (0.0010) [2023-12-26 17:42:34,982][105692] Updated weights for policy 0, policy_version 322302 (0.0010) [2023-12-26 17:42:35,040][105692] Updated weights for policy 0, policy_version 322312 (0.0010) [2023-12-26 17:42:35,407][105620] Updated weights for policy 1, policy_version 322601 (0.0008) [2023-12-26 17:42:35,452][105620] Updated weights for policy 1, policy_version 322611 (0.0008) [2023-12-26 17:42:35,507][105620] Updated weights for policy 1, policy_version 322623 (0.0010) [2023-12-26 17:42:35,533][105586] KL-divergence is very high: 104.4142 [2023-12-26 17:42:35,542][105586] KL-divergence is very high: 121.6429 [2023-12-26 17:42:35,738][105692] Updated weights for policy 0, policy_version 322322 (0.0007) [2023-12-26 17:42:35,785][105692] Updated weights for policy 0, policy_version 322332 (0.0010) [2023-12-26 17:42:35,833][105692] Updated weights for policy 0, policy_version 322342 (0.0010) [2023-12-26 17:42:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19716.3). Total num frames: 165134336. Throughput: 0: 9725.6, 1: 10053.2. Samples: 165121928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:42:36,062][104569] Avg episode reward: [(0, '8010.178'), (1, '8996.428')] [2023-12-26 17:42:36,393][105620] Updated weights for policy 1, policy_version 322634 (0.0010) [2023-12-26 17:42:36,453][105620] Updated weights for policy 1, policy_version 322644 (0.0009) [2023-12-26 17:42:36,506][105692] Updated weights for policy 0, policy_version 322352 (0.0010) [2023-12-26 17:42:36,518][105620] Updated weights for policy 1, policy_version 322654 (0.0010) [2023-12-26 17:42:36,570][105692] Updated weights for policy 0, policy_version 322362 (0.0007) [2023-12-26 17:42:36,580][105620] Updated weights for policy 1, policy_version 322664 (0.0009) [2023-12-26 17:42:36,631][105692] Updated weights for policy 0, policy_version 322372 (0.0009) [2023-12-26 17:42:37,285][105692] Updated weights for policy 0, policy_version 322382 (0.0007) [2023-12-26 17:42:37,351][105692] Updated weights for policy 0, policy_version 322392 (0.0008) [2023-12-26 17:42:37,384][105620] Updated weights for policy 1, policy_version 322674 (0.0007) [2023-12-26 17:42:37,410][105692] Updated weights for policy 0, policy_version 322402 (0.0007) [2023-12-26 17:42:37,446][105620] Updated weights for policy 1, policy_version 322684 (0.0007) [2023-12-26 17:42:37,512][105620] Updated weights for policy 1, policy_version 322694 (0.0010) [2023-12-26 17:42:38,154][105692] Updated weights for policy 0, policy_version 322412 (0.0008) [2023-12-26 17:42:38,208][105692] Updated weights for policy 0, policy_version 322422 (0.0009) [2023-12-26 17:42:38,236][105620] Updated weights for policy 1, policy_version 322704 (0.0006) [2023-12-26 17:42:38,263][105692] Updated weights for policy 0, policy_version 322432 (0.0008) [2023-12-26 17:42:38,288][105620] Updated weights for policy 1, policy_version 322714 (0.0005) [2023-12-26 17:42:38,350][105620] Updated weights for policy 1, policy_version 322724 (0.0009) [2023-12-26 17:42:39,048][105692] Updated weights for policy 0, policy_version 322442 (0.0007) [2023-12-26 17:42:39,070][105620] Updated weights for policy 1, policy_version 322734 (0.0008) [2023-12-26 17:42:39,108][105692] Updated weights for policy 0, policy_version 322452 (0.0008) [2023-12-26 17:42:39,122][105620] Updated weights for policy 1, policy_version 322744 (0.0008) [2023-12-26 17:42:39,161][105692] Updated weights for policy 0, policy_version 322462 (0.0007) [2023-12-26 17:42:39,175][105620] Updated weights for policy 1, policy_version 322754 (0.0007) [2023-12-26 17:42:39,219][105692] Updated weights for policy 0, policy_version 322472 (0.0007) [2023-12-26 17:42:39,944][105620] Updated weights for policy 1, policy_version 322764 (0.0008) [2023-12-26 17:42:39,967][105692] Updated weights for policy 0, policy_version 322482 (0.0007) [2023-12-26 17:42:39,997][105620] Updated weights for policy 1, policy_version 322774 (0.0010) [2023-12-26 17:42:40,028][105692] Updated weights for policy 0, policy_version 322492 (0.0006) [2023-12-26 17:42:40,057][105620] Updated weights for policy 1, policy_version 322784 (0.0011) [2023-12-26 17:42:40,091][105692] Updated weights for policy 0, policy_version 322502 (0.0006) [2023-12-26 17:42:40,787][105620] Updated weights for policy 1, policy_version 322794 (0.0009) [2023-12-26 17:42:40,856][105620] Updated weights for policy 1, policy_version 322804 (0.0009) [2023-12-26 17:42:40,878][105692] Updated weights for policy 0, policy_version 322512 (0.0007) [2023-12-26 17:42:40,912][105620] Updated weights for policy 1, policy_version 322814 (0.0010) [2023-12-26 17:42:40,930][105692] Updated weights for policy 0, policy_version 322522 (0.0006) [2023-12-26 17:42:40,967][105620] Updated weights for policy 1, policy_version 322824 (0.0010) [2023-12-26 17:42:40,987][105692] Updated weights for policy 0, policy_version 322532 (0.0006) [2023-12-26 17:42:41,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.4, 300 sec: 19716.3). Total num frames: 165232640. Throughput: 0: 9809.5, 1: 9959.3. Samples: 165235732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:42:41,062][104569] Avg episode reward: [(0, '8007.687'), (1, '8996.429')] [2023-12-26 17:42:41,686][105620] Updated weights for policy 1, policy_version 322834 (0.0008) [2023-12-26 17:42:41,755][105620] Updated weights for policy 1, policy_version 322844 (0.0009) [2023-12-26 17:42:41,795][105692] Updated weights for policy 0, policy_version 322542 (0.0010) [2023-12-26 17:42:41,814][105620] Updated weights for policy 1, policy_version 322854 (0.0009) [2023-12-26 17:42:41,855][105692] Updated weights for policy 0, policy_version 322552 (0.0011) [2023-12-26 17:42:41,923][105692] Updated weights for policy 0, policy_version 322562 (0.0011) [2023-12-26 17:42:42,630][105620] Updated weights for policy 1, policy_version 322864 (0.0008) [2023-12-26 17:42:42,676][105692] Updated weights for policy 0, policy_version 322572 (0.0010) [2023-12-26 17:42:42,698][105620] Updated weights for policy 1, policy_version 322874 (0.0009) [2023-12-26 17:42:42,736][105692] Updated weights for policy 0, policy_version 322582 (0.0006) [2023-12-26 17:42:42,764][105620] Updated weights for policy 1, policy_version 322884 (0.0008) [2023-12-26 17:42:42,798][105692] Updated weights for policy 0, policy_version 322592 (0.0006) [2023-12-26 17:42:43,468][105692] Updated weights for policy 0, policy_version 322602 (0.0008) [2023-12-26 17:42:43,483][105620] Updated weights for policy 1, policy_version 322894 (0.0010) [2023-12-26 17:42:43,532][105692] Updated weights for policy 0, policy_version 322612 (0.0010) [2023-12-26 17:42:43,539][105620] Updated weights for policy 1, policy_version 322904 (0.0006) [2023-12-26 17:42:43,591][105692] Updated weights for policy 0, policy_version 322622 (0.0008) [2023-12-26 17:42:43,592][105620] Updated weights for policy 1, policy_version 322914 (0.0006) [2023-12-26 17:42:43,645][105692] Updated weights for policy 0, policy_version 322632 (0.0009) [2023-12-26 17:42:44,141][105620] Updated weights for policy 1, policy_version 322924 (0.0005) [2023-12-26 17:42:44,202][105620] Updated weights for policy 1, policy_version 322934 (0.0006) [2023-12-26 17:42:44,264][105620] Updated weights for policy 1, policy_version 322944 (0.0010) [2023-12-26 17:42:44,314][105692] Updated weights for policy 0, policy_version 322642 (0.0006) [2023-12-26 17:42:44,362][105692] Updated weights for policy 0, policy_version 322652 (0.0006) [2023-12-26 17:42:44,408][105692] Updated weights for policy 0, policy_version 322662 (0.0005) [2023-12-26 17:42:44,925][105620] Updated weights for policy 1, policy_version 322954 (0.0010) [2023-12-26 17:42:44,980][105620] Updated weights for policy 1, policy_version 322964 (0.0009) [2023-12-26 17:42:45,040][105620] Updated weights for policy 1, policy_version 322974 (0.0009) [2023-12-26 17:42:45,103][105620] Updated weights for policy 1, policy_version 322984 (0.0007) [2023-12-26 17:42:45,118][105692] Updated weights for policy 0, policy_version 322672 (0.0007) [2023-12-26 17:42:45,184][105692] Updated weights for policy 0, policy_version 322682 (0.0010) [2023-12-26 17:42:45,248][105692] Updated weights for policy 0, policy_version 322692 (0.0009) [2023-12-26 17:42:45,749][105620] Updated weights for policy 1, policy_version 322994 (0.0006) [2023-12-26 17:42:45,809][105620] Updated weights for policy 1, policy_version 323004 (0.0010) [2023-12-26 17:42:45,867][105620] Updated weights for policy 1, policy_version 323014 (0.0009) [2023-12-26 17:42:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.9, 300 sec: 19688.6). Total num frames: 165322752. Throughput: 0: 9809.8, 1: 9910.2. Samples: 165292048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:42:46,062][104569] Avg episode reward: [(0, '7756.512'), (1, '9087.403')] [2023-12-26 17:42:46,067][105692] Updated weights for policy 0, policy_version 322702 (0.0009) [2023-12-26 17:42:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000323016_82698240.pth... [2023-12-26 17:42:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000321864_82403328.pth [2023-12-26 17:42:46,117][105692] Updated weights for policy 0, policy_version 322712 (0.0005) [2023-12-26 17:42:46,163][105692] Updated weights for policy 0, policy_version 322722 (0.0006) [2023-12-26 17:42:46,189][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000322728_82632704.pth... [2023-12-26 17:42:46,192][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000321544_82329600.pth [2023-12-26 17:42:46,533][105620] Updated weights for policy 1, policy_version 323024 (0.0007) [2023-12-26 17:42:46,586][105620] Updated weights for policy 1, policy_version 323034 (0.0008) [2023-12-26 17:42:46,639][105620] Updated weights for policy 1, policy_version 323044 (0.0006) [2023-12-26 17:42:46,796][105692] Updated weights for policy 0, policy_version 322732 (0.0005) [2023-12-26 17:42:46,852][105692] Updated weights for policy 0, policy_version 322742 (0.0005) [2023-12-26 17:42:46,900][105692] Updated weights for policy 0, policy_version 322752 (0.0005) [2023-12-26 17:42:47,178][105620] Updated weights for policy 1, policy_version 323054 (0.0005) [2023-12-26 17:42:47,242][105620] Updated weights for policy 1, policy_version 323064 (0.0005) [2023-12-26 17:42:47,310][105620] Updated weights for policy 1, policy_version 323074 (0.0005) [2023-12-26 17:42:47,622][105692] Updated weights for policy 0, policy_version 322763 (0.0007) [2023-12-26 17:42:47,675][105692] Updated weights for policy 0, policy_version 322773 (0.0010) [2023-12-26 17:42:47,728][105692] Updated weights for policy 0, policy_version 322785 (0.0010) [2023-12-26 17:42:47,849][105620] Updated weights for policy 1, policy_version 323084 (0.0005) [2023-12-26 17:42:47,909][105620] Updated weights for policy 1, policy_version 323094 (0.0005) [2023-12-26 17:42:47,962][105620] Updated weights for policy 1, policy_version 323104 (0.0005) [2023-12-26 17:42:48,494][105620] Updated weights for policy 1, policy_version 323114 (0.0005) [2023-12-26 17:42:48,522][105692] Updated weights for policy 0, policy_version 322796 (0.0009) [2023-12-26 17:42:48,558][105620] Updated weights for policy 1, policy_version 323124 (0.0006) [2023-12-26 17:42:48,581][105692] Updated weights for policy 0, policy_version 322806 (0.0008) [2023-12-26 17:42:48,617][105620] Updated weights for policy 1, policy_version 323134 (0.0007) [2023-12-26 17:42:48,634][105692] Updated weights for policy 0, policy_version 322816 (0.0006) [2023-12-26 17:42:48,674][105620] Updated weights for policy 1, policy_version 323144 (0.0009) [2023-12-26 17:42:49,368][105692] Updated weights for policy 0, policy_version 322826 (0.0006) [2023-12-26 17:42:49,379][105620] Updated weights for policy 1, policy_version 323154 (0.0008) [2023-12-26 17:42:49,429][105692] Updated weights for policy 0, policy_version 322836 (0.0010) [2023-12-26 17:42:49,440][105620] Updated weights for policy 1, policy_version 323164 (0.0007) [2023-12-26 17:42:49,486][105692] Updated weights for policy 0, policy_version 322846 (0.0008) [2023-12-26 17:42:49,500][105620] Updated weights for policy 1, policy_version 323174 (0.0007) [2023-12-26 17:42:49,535][105692] Updated weights for policy 0, policy_version 322856 (0.0009) [2023-12-26 17:42:50,191][105620] Updated weights for policy 1, policy_version 323184 (0.0006) [2023-12-26 17:42:50,242][105620] Updated weights for policy 1, policy_version 323194 (0.0005) [2023-12-26 17:42:50,296][105620] Updated weights for policy 1, policy_version 323204 (0.0005) [2023-12-26 17:42:50,384][105692] Updated weights for policy 0, policy_version 322866 (0.0008) [2023-12-26 17:42:50,442][105692] Updated weights for policy 0, policy_version 322876 (0.0009) [2023-12-26 17:42:50,506][105692] Updated weights for policy 0, policy_version 322886 (0.0009) [2023-12-26 17:42:50,956][105620] Updated weights for policy 1, policy_version 323214 (0.0006) [2023-12-26 17:42:51,024][105620] Updated weights for policy 1, policy_version 323224 (0.0006) [2023-12-26 17:42:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 165421056. Throughput: 0: 9801.3, 1: 9996.8. Samples: 165415988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:42:51,062][104569] Avg episode reward: [(0, '8168.575'), (1, '9356.853')] [2023-12-26 17:42:51,091][105620] Updated weights for policy 1, policy_version 323234 (0.0007) [2023-12-26 17:42:51,151][105692] Updated weights for policy 0, policy_version 322896 (0.0010) [2023-12-26 17:42:51,206][105692] Updated weights for policy 0, policy_version 322906 (0.0008) [2023-12-26 17:42:51,261][105692] Updated weights for policy 0, policy_version 322916 (0.0008) [2023-12-26 17:42:51,769][105620] Updated weights for policy 1, policy_version 323244 (0.0009) [2023-12-26 17:42:51,827][105620] Updated weights for policy 1, policy_version 323254 (0.0010) [2023-12-26 17:42:51,887][105620] Updated weights for policy 1, policy_version 323264 (0.0010) [2023-12-26 17:42:51,970][105692] Updated weights for policy 0, policy_version 322926 (0.0009) [2023-12-26 17:42:52,025][105692] Updated weights for policy 0, policy_version 322936 (0.0009) [2023-12-26 17:42:52,081][105692] Updated weights for policy 0, policy_version 322946 (0.0009) [2023-12-26 17:42:52,633][105620] Updated weights for policy 1, policy_version 323274 (0.0010) [2023-12-26 17:42:52,682][105620] Updated weights for policy 1, policy_version 323284 (0.0010) [2023-12-26 17:42:52,738][105620] Updated weights for policy 1, policy_version 323294 (0.0010) [2023-12-26 17:42:52,745][105692] Updated weights for policy 0, policy_version 322956 (0.0008) [2023-12-26 17:42:52,796][105620] Updated weights for policy 1, policy_version 323304 (0.0010) [2023-12-26 17:42:52,800][105692] Updated weights for policy 0, policy_version 322966 (0.0007) [2023-12-26 17:42:52,849][105692] Updated weights for policy 0, policy_version 322976 (0.0008) [2023-12-26 17:42:53,546][105620] Updated weights for policy 1, policy_version 323314 (0.0010) [2023-12-26 17:42:53,553][105692] Updated weights for policy 0, policy_version 322986 (0.0007) [2023-12-26 17:42:53,602][105620] Updated weights for policy 1, policy_version 323324 (0.0010) [2023-12-26 17:42:53,602][105692] Updated weights for policy 0, policy_version 322996 (0.0005) [2023-12-26 17:42:53,653][105620] Updated weights for policy 1, policy_version 323334 (0.0010) [2023-12-26 17:42:53,653][105692] Updated weights for policy 0, policy_version 323006 (0.0005) [2023-12-26 17:42:53,708][105692] Updated weights for policy 0, policy_version 323016 (0.0005) [2023-12-26 17:42:54,257][105692] Updated weights for policy 0, policy_version 323026 (0.0006) [2023-12-26 17:42:54,321][105692] Updated weights for policy 0, policy_version 323036 (0.0010) [2023-12-26 17:42:54,380][105692] Updated weights for policy 0, policy_version 323046 (0.0010) [2023-12-26 17:42:54,392][105620] Updated weights for policy 1, policy_version 323344 (0.0009) [2023-12-26 17:42:54,448][105620] Updated weights for policy 1, policy_version 323354 (0.0006) [2023-12-26 17:42:54,497][105620] Updated weights for policy 1, policy_version 323364 (0.0005) [2023-12-26 17:42:54,933][105692] Updated weights for policy 0, policy_version 323056 (0.0006) [2023-12-26 17:42:54,996][105692] Updated weights for policy 0, policy_version 323066 (0.0006) [2023-12-26 17:42:55,048][105620] Updated weights for policy 1, policy_version 323374 (0.0005) [2023-12-26 17:42:55,061][105692] Updated weights for policy 0, policy_version 323076 (0.0006) [2023-12-26 17:42:55,119][105586] KL-divergence is very high: 130.5546 [2023-12-26 17:42:55,119][105620] Updated weights for policy 1, policy_version 323384 (0.0009) [2023-12-26 17:42:55,132][105586] KL-divergence is very high: 107.8199 [2023-12-26 17:42:55,177][105620] Updated weights for policy 1, policy_version 323394 (0.0009) [2023-12-26 17:42:55,658][105692] Updated weights for policy 0, policy_version 323086 (0.0008) [2023-12-26 17:42:55,713][105692] Updated weights for policy 0, policy_version 323096 (0.0010) [2023-12-26 17:42:55,772][105692] Updated weights for policy 0, policy_version 323106 (0.0010) [2023-12-26 17:42:55,802][105620] Updated weights for policy 1, policy_version 323404 (0.0006) [2023-12-26 17:42:55,853][105620] Updated weights for policy 1, policy_version 323414 (0.0007) [2023-12-26 17:42:55,898][105620] Updated weights for policy 1, policy_version 323424 (0.0008) [2023-12-26 17:42:56,062][104569] Fps is (10 sec: 21298.8, 60 sec: 19933.9, 300 sec: 19744.1). Total num frames: 165535744. Throughput: 0: 9911.7, 1: 9942.2. Samples: 165540120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:42:56,063][104569] Avg episode reward: [(0, '8360.856'), (1, '9356.938')] [2023-12-26 17:42:56,498][105692] Updated weights for policy 0, policy_version 323116 (0.0011) [2023-12-26 17:42:56,563][105692] Updated weights for policy 0, policy_version 323126 (0.0011) [2023-12-26 17:42:56,638][105692] Updated weights for policy 0, policy_version 323136 (0.0011) [2023-12-26 17:42:56,656][105620] Updated weights for policy 1, policy_version 323434 (0.0005) [2023-12-26 17:42:56,709][105620] Updated weights for policy 1, policy_version 323444 (0.0005) [2023-12-26 17:42:56,763][105620] Updated weights for policy 1, policy_version 323454 (0.0005) [2023-12-26 17:42:56,811][105620] Updated weights for policy 1, policy_version 323464 (0.0005) [2023-12-26 17:42:57,306][105692] Updated weights for policy 0, policy_version 323146 (0.0009) [2023-12-26 17:42:57,359][105620] Updated weights for policy 1, policy_version 323474 (0.0006) [2023-12-26 17:42:57,361][105692] Updated weights for policy 0, policy_version 323156 (0.0010) [2023-12-26 17:42:57,405][105692] Updated weights for policy 0, policy_version 323166 (0.0010) [2023-12-26 17:42:57,415][105620] Updated weights for policy 1, policy_version 323484 (0.0006) [2023-12-26 17:42:57,449][105692] Updated weights for policy 0, policy_version 323176 (0.0010) [2023-12-26 17:42:57,467][105620] Updated weights for policy 1, policy_version 323494 (0.0006) [2023-12-26 17:42:58,056][105692] Updated weights for policy 0, policy_version 323186 (0.0005) [2023-12-26 17:42:58,115][105692] Updated weights for policy 0, policy_version 323196 (0.0006) [2023-12-26 17:42:58,174][105692] Updated weights for policy 0, policy_version 323206 (0.0008) [2023-12-26 17:42:58,249][105620] Updated weights for policy 1, policy_version 323504 (0.0008) [2023-12-26 17:42:58,314][105620] Updated weights for policy 1, policy_version 323514 (0.0009) [2023-12-26 17:42:58,390][105620] Updated weights for policy 1, policy_version 323524 (0.0010) [2023-12-26 17:42:58,860][105692] Updated weights for policy 0, policy_version 323217 (0.0008) [2023-12-26 17:42:58,919][105692] Updated weights for policy 0, policy_version 323227 (0.0009) [2023-12-26 17:42:58,981][105692] Updated weights for policy 0, policy_version 323237 (0.0009) [2023-12-26 17:42:59,253][105620] Updated weights for policy 1, policy_version 323534 (0.0008) [2023-12-26 17:42:59,316][105620] Updated weights for policy 1, policy_version 323544 (0.0008) [2023-12-26 17:42:59,380][105620] Updated weights for policy 1, policy_version 323554 (0.0009) [2023-12-26 17:42:59,811][105692] Updated weights for policy 0, policy_version 323247 (0.0009) [2023-12-26 17:42:59,878][105692] Updated weights for policy 0, policy_version 323257 (0.0007) [2023-12-26 17:42:59,944][105692] Updated weights for policy 0, policy_version 323267 (0.0008) [2023-12-26 17:43:00,131][105620] Updated weights for policy 1, policy_version 323564 (0.0007) [2023-12-26 17:43:00,185][105620] Updated weights for policy 1, policy_version 323574 (0.0009) [2023-12-26 17:43:00,247][105620] Updated weights for policy 1, policy_version 323584 (0.0010) [2023-12-26 17:43:00,639][105692] Updated weights for policy 0, policy_version 323277 (0.0009) [2023-12-26 17:43:00,693][105692] Updated weights for policy 0, policy_version 323287 (0.0009) [2023-12-26 17:43:00,737][105692] Updated weights for policy 0, policy_version 323297 (0.0008) [2023-12-26 17:43:01,008][105620] Updated weights for policy 1, policy_version 323594 (0.0009) [2023-12-26 17:43:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19933.9, 300 sec: 19716.3). Total num frames: 165625856. Throughput: 0: 9980.6, 1: 9968.5. Samples: 165600276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:43:01,062][104569] Avg episode reward: [(0, '8279.765'), (1, '9356.716')] [2023-12-26 17:43:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000323304_82780160.pth... [2023-12-26 17:43:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000322152_82485248.pth [2023-12-26 17:43:01,075][105620] Updated weights for policy 1, policy_version 323604 (0.0009) [2023-12-26 17:43:01,147][105620] Updated weights for policy 1, policy_version 323614 (0.0009) [2023-12-26 17:43:01,201][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000323624_82853888.pth... [2023-12-26 17:43:01,203][105620] Updated weights for policy 1, policy_version 323624 (0.0009) [2023-12-26 17:43:01,205][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000322440_82550784.pth [2023-12-26 17:43:01,514][105692] Updated weights for policy 0, policy_version 323307 (0.0008) [2023-12-26 17:43:01,576][105692] Updated weights for policy 0, policy_version 323317 (0.0008) [2023-12-26 17:43:01,641][105692] Updated weights for policy 0, policy_version 323327 (0.0009) [2023-12-26 17:43:01,935][105620] Updated weights for policy 1, policy_version 323634 (0.0009) [2023-12-26 17:43:01,995][105620] Updated weights for policy 1, policy_version 323644 (0.0009) [2023-12-26 17:43:02,053][105620] Updated weights for policy 1, policy_version 323654 (0.0009) [2023-12-26 17:43:02,395][105692] Updated weights for policy 0, policy_version 323337 (0.0009) [2023-12-26 17:43:02,443][105692] Updated weights for policy 0, policy_version 323347 (0.0009) [2023-12-26 17:43:02,495][105692] Updated weights for policy 0, policy_version 323357 (0.0008) [2023-12-26 17:43:02,546][105692] Updated weights for policy 0, policy_version 323367 (0.0005) [2023-12-26 17:43:02,808][105620] Updated weights for policy 1, policy_version 323664 (0.0010) [2023-12-26 17:43:02,872][105620] Updated weights for policy 1, policy_version 323674 (0.0008) [2023-12-26 17:43:02,937][105620] Updated weights for policy 1, policy_version 323684 (0.0007) [2023-12-26 17:43:03,249][105692] Updated weights for policy 0, policy_version 323377 (0.0009) [2023-12-26 17:43:03,313][105692] Updated weights for policy 0, policy_version 323387 (0.0009) [2023-12-26 17:43:03,372][105692] Updated weights for policy 0, policy_version 323397 (0.0008) [2023-12-26 17:43:03,693][105620] Updated weights for policy 1, policy_version 323694 (0.0008) [2023-12-26 17:43:03,749][105620] Updated weights for policy 1, policy_version 323704 (0.0009) [2023-12-26 17:43:03,797][105620] Updated weights for policy 1, policy_version 323714 (0.0008) [2023-12-26 17:43:03,985][105692] Updated weights for policy 0, policy_version 323407 (0.0007) [2023-12-26 17:43:04,052][105692] Updated weights for policy 0, policy_version 323417 (0.0009) [2023-12-26 17:43:04,110][105692] Updated weights for policy 0, policy_version 323427 (0.0009) [2023-12-26 17:43:04,606][105620] Updated weights for policy 1, policy_version 323724 (0.0009) [2023-12-26 17:43:04,656][105620] Updated weights for policy 1, policy_version 323734 (0.0008) [2023-12-26 17:43:04,707][105620] Updated weights for policy 1, policy_version 323744 (0.0009) [2023-12-26 17:43:04,812][105692] Updated weights for policy 0, policy_version 323437 (0.0009) [2023-12-26 17:43:04,866][105692] Updated weights for policy 0, policy_version 323447 (0.0009) [2023-12-26 17:43:04,920][105692] Updated weights for policy 0, policy_version 323457 (0.0008) [2023-12-26 17:43:05,361][105620] Updated weights for policy 1, policy_version 323754 (0.0009) [2023-12-26 17:43:05,420][105620] Updated weights for policy 1, policy_version 323764 (0.0010) [2023-12-26 17:43:05,482][105620] Updated weights for policy 1, policy_version 323774 (0.0011) [2023-12-26 17:43:05,536][105620] Updated weights for policy 1, policy_version 323784 (0.0009) [2023-12-26 17:43:05,590][105692] Updated weights for policy 0, policy_version 323467 (0.0005) [2023-12-26 17:43:05,644][105692] Updated weights for policy 0, policy_version 323477 (0.0008) [2023-12-26 17:43:05,703][105692] Updated weights for policy 0, policy_version 323487 (0.0008) [2023-12-26 17:43:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 165724160. Throughput: 0: 9996.9, 1: 9736.0. Samples: 165712528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:43:06,062][104569] Avg episode reward: [(0, '8729.745'), (1, '9356.663')] [2023-12-26 17:43:06,290][105620] Updated weights for policy 1, policy_version 323794 (0.0009) [2023-12-26 17:43:06,358][105620] Updated weights for policy 1, policy_version 323804 (0.0009) [2023-12-26 17:43:06,420][105620] Updated weights for policy 1, policy_version 323814 (0.0009) [2023-12-26 17:43:06,446][105692] Updated weights for policy 0, policy_version 323497 (0.0007) [2023-12-26 17:43:06,508][105692] Updated weights for policy 0, policy_version 323507 (0.0009) [2023-12-26 17:43:06,574][105692] Updated weights for policy 0, policy_version 323517 (0.0010) [2023-12-26 17:43:06,634][105692] Updated weights for policy 0, policy_version 323527 (0.0009) [2023-12-26 17:43:07,158][105620] Updated weights for policy 1, policy_version 323824 (0.0009) [2023-12-26 17:43:07,214][105620] Updated weights for policy 1, policy_version 323834 (0.0009) [2023-12-26 17:43:07,274][105620] Updated weights for policy 1, policy_version 323844 (0.0008) [2023-12-26 17:43:07,391][105692] Updated weights for policy 0, policy_version 323537 (0.0008) [2023-12-26 17:43:07,455][105692] Updated weights for policy 0, policy_version 323547 (0.0009) [2023-12-26 17:43:07,521][105692] Updated weights for policy 0, policy_version 323557 (0.0009) [2023-12-26 17:43:07,993][105620] Updated weights for policy 1, policy_version 323854 (0.0010) [2023-12-26 17:43:08,052][105620] Updated weights for policy 1, policy_version 323864 (0.0010) [2023-12-26 17:43:08,113][105620] Updated weights for policy 1, policy_version 323874 (0.0010) [2023-12-26 17:43:08,317][105692] Updated weights for policy 0, policy_version 323567 (0.0008) [2023-12-26 17:43:08,377][105692] Updated weights for policy 0, policy_version 323577 (0.0008) [2023-12-26 17:43:08,426][105692] Updated weights for policy 0, policy_version 323587 (0.0008) [2023-12-26 17:43:08,800][105620] Updated weights for policy 1, policy_version 323884 (0.0008) [2023-12-26 17:43:08,858][105620] Updated weights for policy 1, policy_version 323894 (0.0010) [2023-12-26 17:43:08,907][105620] Updated weights for policy 1, policy_version 323904 (0.0010) [2023-12-26 17:43:09,224][105692] Updated weights for policy 0, policy_version 323597 (0.0007) [2023-12-26 17:43:09,291][105692] Updated weights for policy 0, policy_version 323607 (0.0007) [2023-12-26 17:43:09,352][105692] Updated weights for policy 0, policy_version 323617 (0.0006) [2023-12-26 17:43:09,664][105620] Updated weights for policy 1, policy_version 323914 (0.0010) [2023-12-26 17:43:09,730][105620] Updated weights for policy 1, policy_version 323924 (0.0010) [2023-12-26 17:43:09,797][105620] Updated weights for policy 1, policy_version 323934 (0.0011) [2023-12-26 17:43:09,867][105620] Updated weights for policy 1, policy_version 323944 (0.0010) [2023-12-26 17:43:09,952][105692] Updated weights for policy 0, policy_version 323627 (0.0008) [2023-12-26 17:43:10,009][105692] Updated weights for policy 0, policy_version 323637 (0.0008) [2023-12-26 17:43:10,073][105692] Updated weights for policy 0, policy_version 323647 (0.0008) [2023-12-26 17:43:10,531][105620] Updated weights for policy 1, policy_version 323954 (0.0005) [2023-12-26 17:43:10,583][105620] Updated weights for policy 1, policy_version 323964 (0.0005) [2023-12-26 17:43:10,632][105620] Updated weights for policy 1, policy_version 323974 (0.0005) [2023-12-26 17:43:10,915][105692] Updated weights for policy 0, policy_version 323657 (0.0008) [2023-12-26 17:43:10,977][105692] Updated weights for policy 0, policy_version 323667 (0.0010) [2023-12-26 17:43:11,039][105692] Updated weights for policy 0, policy_version 323677 (0.0010) [2023-12-26 17:43:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 165814272. Throughput: 0: 9949.5, 1: 9796.4. Samples: 165829100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:43:11,063][104569] Avg episode reward: [(0, '8820.015'), (1, '9356.426')] [2023-12-26 17:43:11,108][105692] Updated weights for policy 0, policy_version 323687 (0.0009) [2023-12-26 17:43:11,184][105620] Updated weights for policy 1, policy_version 323984 (0.0010) [2023-12-26 17:43:11,235][105620] Updated weights for policy 1, policy_version 323994 (0.0009) [2023-12-26 17:43:11,300][105620] Updated weights for policy 1, policy_version 324004 (0.0008) [2023-12-26 17:43:11,913][105692] Updated weights for policy 0, policy_version 323697 (0.0008) [2023-12-26 17:43:11,974][105692] Updated weights for policy 0, policy_version 323707 (0.0009) [2023-12-26 17:43:12,036][105692] Updated weights for policy 0, policy_version 323717 (0.0008) [2023-12-26 17:43:12,125][105620] Updated weights for policy 1, policy_version 324014 (0.0009) [2023-12-26 17:43:12,182][105620] Updated weights for policy 1, policy_version 324024 (0.0011) [2023-12-26 17:43:12,248][105620] Updated weights for policy 1, policy_version 324034 (0.0010) [2023-12-26 17:43:12,853][105692] Updated weights for policy 0, policy_version 323727 (0.0008) [2023-12-26 17:43:12,913][105692] Updated weights for policy 0, policy_version 323737 (0.0008) [2023-12-26 17:43:12,966][105692] Updated weights for policy 0, policy_version 323747 (0.0008) [2023-12-26 17:43:13,018][105620] Updated weights for policy 1, policy_version 324044 (0.0011) [2023-12-26 17:43:13,076][105620] Updated weights for policy 1, policy_version 324054 (0.0010) [2023-12-26 17:43:13,127][105620] Updated weights for policy 1, policy_version 324064 (0.0010) [2023-12-26 17:43:13,738][105692] Updated weights for policy 0, policy_version 323757 (0.0007) [2023-12-26 17:43:13,800][105692] Updated weights for policy 0, policy_version 323767 (0.0005) [2023-12-26 17:43:13,850][105620] Updated weights for policy 1, policy_version 324074 (0.0009) [2023-12-26 17:43:13,861][105692] Updated weights for policy 0, policy_version 323777 (0.0005) [2023-12-26 17:43:13,907][105620] Updated weights for policy 1, policy_version 324084 (0.0009) [2023-12-26 17:43:13,967][105620] Updated weights for policy 1, policy_version 324094 (0.0006) [2023-12-26 17:43:14,027][105620] Updated weights for policy 1, policy_version 324104 (0.0005) [2023-12-26 17:43:14,504][105692] Updated weights for policy 0, policy_version 323787 (0.0007) [2023-12-26 17:43:14,567][105692] Updated weights for policy 0, policy_version 323797 (0.0006) [2023-12-26 17:43:14,570][105620] Updated weights for policy 1, policy_version 324114 (0.0010) [2023-12-26 17:43:14,622][105692] Updated weights for policy 0, policy_version 323807 (0.0005) [2023-12-26 17:43:14,626][105620] Updated weights for policy 1, policy_version 324124 (0.0008) [2023-12-26 17:43:14,687][105620] Updated weights for policy 1, policy_version 324134 (0.0008) [2023-12-26 17:43:15,353][105692] Updated weights for policy 0, policy_version 323817 (0.0006) [2023-12-26 17:43:15,401][105692] Updated weights for policy 0, policy_version 323827 (0.0007) [2023-12-26 17:43:15,411][105620] Updated weights for policy 1, policy_version 324144 (0.0006) [2023-12-26 17:43:15,451][105692] Updated weights for policy 0, policy_version 323837 (0.0007) [2023-12-26 17:43:15,461][105620] Updated weights for policy 1, policy_version 324154 (0.0006) [2023-12-26 17:43:15,513][105692] Updated weights for policy 0, policy_version 323847 (0.0009) [2023-12-26 17:43:15,514][105620] Updated weights for policy 1, policy_version 324164 (0.0007) [2023-12-26 17:43:16,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19661.0, 300 sec: 19688.6). Total num frames: 165912576. Throughput: 0: 9823.6, 1: 9742.3. Samples: 165883272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:43:16,062][104569] Avg episode reward: [(0, '8729.234'), (1, '9356.171')] [2023-12-26 17:43:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000323848_82919424.pth... [2023-12-26 17:43:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000324168_82993152.pth... [2023-12-26 17:43:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000322728_82632704.pth [2023-12-26 17:43:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000323016_82698240.pth [2023-12-26 17:43:16,189][105620] Updated weights for policy 1, policy_version 324174 (0.0009) [2023-12-26 17:43:16,236][105620] Updated weights for policy 1, policy_version 324184 (0.0008) [2023-12-26 17:43:16,282][105620] Updated weights for policy 1, policy_version 324194 (0.0009) [2023-12-26 17:43:16,309][105692] Updated weights for policy 0, policy_version 323857 (0.0007) [2023-12-26 17:43:16,354][105692] Updated weights for policy 0, policy_version 323867 (0.0007) [2023-12-26 17:43:16,405][105692] Updated weights for policy 0, policy_version 323877 (0.0009) [2023-12-26 17:43:16,953][105620] Updated weights for policy 1, policy_version 324204 (0.0008) [2023-12-26 17:43:17,014][105620] Updated weights for policy 1, policy_version 324214 (0.0009) [2023-12-26 17:43:17,081][105620] Updated weights for policy 1, policy_version 324224 (0.0007) [2023-12-26 17:43:17,244][105692] Updated weights for policy 0, policy_version 323887 (0.0009) [2023-12-26 17:43:17,295][105692] Updated weights for policy 0, policy_version 323897 (0.0009) [2023-12-26 17:43:17,348][105692] Updated weights for policy 0, policy_version 323907 (0.0008) [2023-12-26 17:43:17,709][105620] Updated weights for policy 1, policy_version 324234 (0.0006) [2023-12-26 17:43:17,764][105620] Updated weights for policy 1, policy_version 324244 (0.0009) [2023-12-26 17:43:17,818][105620] Updated weights for policy 1, policy_version 324254 (0.0005) [2023-12-26 17:43:17,877][105620] Updated weights for policy 1, policy_version 324264 (0.0007) [2023-12-26 17:43:18,138][105692] Updated weights for policy 0, policy_version 323917 (0.0009) [2023-12-26 17:43:18,186][105692] Updated weights for policy 0, policy_version 323927 (0.0009) [2023-12-26 17:43:18,252][105692] Updated weights for policy 0, policy_version 323937 (0.0010) [2023-12-26 17:43:18,616][105620] Updated weights for policy 1, policy_version 324274 (0.0009) [2023-12-26 17:43:18,667][105620] Updated weights for policy 1, policy_version 324284 (0.0008) [2023-12-26 17:43:18,717][105620] Updated weights for policy 1, policy_version 324294 (0.0009) [2023-12-26 17:43:19,013][105692] Updated weights for policy 0, policy_version 323947 (0.0010) [2023-12-26 17:43:19,060][105692] Updated weights for policy 0, policy_version 323957 (0.0009) [2023-12-26 17:43:19,110][105692] Updated weights for policy 0, policy_version 323967 (0.0009) [2023-12-26 17:43:19,494][105620] Updated weights for policy 1, policy_version 324304 (0.0010) [2023-12-26 17:43:19,554][105620] Updated weights for policy 1, policy_version 324314 (0.0009) [2023-12-26 17:43:19,616][105620] Updated weights for policy 1, policy_version 324324 (0.0009) [2023-12-26 17:43:19,885][105692] Updated weights for policy 0, policy_version 323977 (0.0009) [2023-12-26 17:43:19,954][105692] Updated weights for policy 0, policy_version 323987 (0.0009) [2023-12-26 17:43:20,013][105692] Updated weights for policy 0, policy_version 323997 (0.0009) [2023-12-26 17:43:20,079][105692] Updated weights for policy 0, policy_version 324007 (0.0009) [2023-12-26 17:43:20,402][105620] Updated weights for policy 1, policy_version 324334 (0.0008) [2023-12-26 17:43:20,457][105620] Updated weights for policy 1, policy_version 324344 (0.0009) [2023-12-26 17:43:20,530][105620] Updated weights for policy 1, policy_version 324354 (0.0008) [2023-12-26 17:43:20,850][105692] Updated weights for policy 0, policy_version 324017 (0.0006) [2023-12-26 17:43:20,901][105692] Updated weights for policy 0, policy_version 324027 (0.0006) [2023-12-26 17:43:20,957][105692] Updated weights for policy 0, policy_version 324037 (0.0010) [2023-12-26 17:43:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 166010880. Throughput: 0: 9655.2, 1: 9845.1. Samples: 165999444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:43:21,063][104569] Avg episode reward: [(0, '8635.928'), (1, '9266.017')] [2023-12-26 17:43:21,290][105620] Updated weights for policy 1, policy_version 324364 (0.0009) [2023-12-26 17:43:21,356][105620] Updated weights for policy 1, policy_version 324374 (0.0008) [2023-12-26 17:43:21,414][105620] Updated weights for policy 1, policy_version 324384 (0.0008) [2023-12-26 17:43:21,721][105692] Updated weights for policy 0, policy_version 324047 (0.0009) [2023-12-26 17:43:21,789][105692] Updated weights for policy 0, policy_version 324057 (0.0009) [2023-12-26 17:43:21,848][105692] Updated weights for policy 0, policy_version 324067 (0.0006) [2023-12-26 17:43:22,233][105620] Updated weights for policy 1, policy_version 324394 (0.0009) [2023-12-26 17:43:22,301][105620] Updated weights for policy 1, policy_version 324404 (0.0009) [2023-12-26 17:43:22,365][105620] Updated weights for policy 1, policy_version 324414 (0.0010) [2023-12-26 17:43:22,430][105620] Updated weights for policy 1, policy_version 324424 (0.0008) [2023-12-26 17:43:22,477][105692] Updated weights for policy 0, policy_version 324077 (0.0007) [2023-12-26 17:43:22,536][105692] Updated weights for policy 0, policy_version 324087 (0.0008) [2023-12-26 17:43:22,607][105692] Updated weights for policy 0, policy_version 324097 (0.0008) [2023-12-26 17:43:23,120][105620] Updated weights for policy 1, policy_version 324434 (0.0009) [2023-12-26 17:43:23,176][105620] Updated weights for policy 1, policy_version 324444 (0.0009) [2023-12-26 17:43:23,235][105620] Updated weights for policy 1, policy_version 324454 (0.0009) [2023-12-26 17:43:23,397][105692] Updated weights for policy 0, policy_version 324107 (0.0008) [2023-12-26 17:43:23,443][105692] Updated weights for policy 0, policy_version 324117 (0.0008) [2023-12-26 17:43:23,505][105692] Updated weights for policy 0, policy_version 324127 (0.0009) [2023-12-26 17:43:23,929][105620] Updated weights for policy 1, policy_version 324464 (0.0010) [2023-12-26 17:43:23,989][105620] Updated weights for policy 1, policy_version 324474 (0.0009) [2023-12-26 17:43:24,042][105620] Updated weights for policy 1, policy_version 324484 (0.0009) [2023-12-26 17:43:24,291][105692] Updated weights for policy 0, policy_version 324137 (0.0009) [2023-12-26 17:43:24,348][105692] Updated weights for policy 0, policy_version 324147 (0.0005) [2023-12-26 17:43:24,405][105692] Updated weights for policy 0, policy_version 324157 (0.0005) [2023-12-26 17:43:24,465][105692] Updated weights for policy 0, policy_version 324167 (0.0005) [2023-12-26 17:43:24,871][105620] Updated weights for policy 1, policy_version 324494 (0.0009) [2023-12-26 17:43:24,933][105620] Updated weights for policy 1, policy_version 324504 (0.0009) [2023-12-26 17:43:24,988][105620] Updated weights for policy 1, policy_version 324514 (0.0011) [2023-12-26 17:43:25,041][105692] Updated weights for policy 0, policy_version 324177 (0.0005) [2023-12-26 17:43:25,111][105692] Updated weights for policy 0, policy_version 324187 (0.0005) [2023-12-26 17:43:25,171][105692] Updated weights for policy 0, policy_version 324197 (0.0008) [2023-12-26 17:43:25,710][105620] Updated weights for policy 1, policy_version 324524 (0.0010) [2023-12-26 17:43:25,740][105692] Updated weights for policy 0, policy_version 324207 (0.0007) [2023-12-26 17:43:25,759][105620] Updated weights for policy 1, policy_version 324534 (0.0010) [2023-12-26 17:43:25,795][105692] Updated weights for policy 0, policy_version 324217 (0.0010) [2023-12-26 17:43:25,807][105620] Updated weights for policy 1, policy_version 324544 (0.0010) [2023-12-26 17:43:25,841][105692] Updated weights for policy 0, policy_version 324227 (0.0005) [2023-12-26 17:43:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 166109184. Throughput: 0: 9655.5, 1: 9846.8. Samples: 166113340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:43:26,062][104569] Avg episode reward: [(0, '8813.270'), (1, '9266.011')] [2023-12-26 17:43:26,473][105692] Updated weights for policy 0, policy_version 324237 (0.0008) [2023-12-26 17:43:26,520][105692] Updated weights for policy 0, policy_version 324247 (0.0010) [2023-12-26 17:43:26,559][105620] Updated weights for policy 1, policy_version 324554 (0.0010) [2023-12-26 17:43:26,568][105692] Updated weights for policy 0, policy_version 324257 (0.0010) [2023-12-26 17:43:26,607][105620] Updated weights for policy 1, policy_version 324564 (0.0010) [2023-12-26 17:43:26,656][105620] Updated weights for policy 1, policy_version 324574 (0.0010) [2023-12-26 17:43:26,720][105620] Updated weights for policy 1, policy_version 324584 (0.0010) [2023-12-26 17:43:27,311][105692] Updated weights for policy 0, policy_version 324267 (0.0010) [2023-12-26 17:43:27,361][105620] Updated weights for policy 1, policy_version 324594 (0.0011) [2023-12-26 17:43:27,365][105692] Updated weights for policy 0, policy_version 324277 (0.0009) [2023-12-26 17:43:27,413][105692] Updated weights for policy 0, policy_version 324287 (0.0005) [2023-12-26 17:43:27,420][105620] Updated weights for policy 1, policy_version 324604 (0.0010) [2023-12-26 17:43:27,472][105620] Updated weights for policy 1, policy_version 324614 (0.0010) [2023-12-26 17:43:27,975][105692] Updated weights for policy 0, policy_version 324297 (0.0006) [2023-12-26 17:43:28,029][105692] Updated weights for policy 0, policy_version 324307 (0.0010) [2023-12-26 17:43:28,077][105692] Updated weights for policy 0, policy_version 324317 (0.0008) [2023-12-26 17:43:28,134][105692] Updated weights for policy 0, policy_version 324327 (0.0006) [2023-12-26 17:43:28,228][105620] Updated weights for policy 1, policy_version 324624 (0.0008) [2023-12-26 17:43:28,285][105620] Updated weights for policy 1, policy_version 324634 (0.0005) [2023-12-26 17:43:28,344][105620] Updated weights for policy 1, policy_version 324644 (0.0009) [2023-12-26 17:43:28,803][105692] Updated weights for policy 0, policy_version 324337 (0.0010) [2023-12-26 17:43:28,858][105692] Updated weights for policy 0, policy_version 324347 (0.0010) [2023-12-26 17:43:28,920][105692] Updated weights for policy 0, policy_version 324357 (0.0010) [2023-12-26 17:43:29,005][105620] Updated weights for policy 1, policy_version 324654 (0.0010) [2023-12-26 17:43:29,059][105620] Updated weights for policy 1, policy_version 324664 (0.0010) [2023-12-26 17:43:29,111][105620] Updated weights for policy 1, policy_version 324674 (0.0010) [2023-12-26 17:43:29,669][105692] Updated weights for policy 0, policy_version 324367 (0.0010) [2023-12-26 17:43:29,729][105692] Updated weights for policy 0, policy_version 324377 (0.0010) [2023-12-26 17:43:29,783][105692] Updated weights for policy 0, policy_version 324387 (0.0010) [2023-12-26 17:43:29,839][105620] Updated weights for policy 1, policy_version 324684 (0.0010) [2023-12-26 17:43:29,896][105620] Updated weights for policy 1, policy_version 324694 (0.0011) [2023-12-26 17:43:29,953][105620] Updated weights for policy 1, policy_version 324704 (0.0011) [2023-12-26 17:43:30,525][105692] Updated weights for policy 0, policy_version 324397 (0.0009) [2023-12-26 17:43:30,571][105692] Updated weights for policy 0, policy_version 324407 (0.0007) [2023-12-26 17:43:30,618][105692] Updated weights for policy 0, policy_version 324417 (0.0008) [2023-12-26 17:43:30,636][105620] Updated weights for policy 1, policy_version 324714 (0.0010) [2023-12-26 17:43:30,684][105620] Updated weights for policy 1, policy_version 324724 (0.0010) [2023-12-26 17:43:30,738][105620] Updated weights for policy 1, policy_version 324734 (0.0010) [2023-12-26 17:43:30,791][105620] Updated weights for policy 1, policy_version 324744 (0.0010) [2023-12-26 17:43:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19716.3). Total num frames: 166207488. Throughput: 0: 9759.4, 1: 9886.9. Samples: 166176136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:43:31,063][104569] Avg episode reward: [(0, '8814.888'), (1, '9176.437')] [2023-12-26 17:43:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000324424_83066880.pth... [2023-12-26 17:43:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000324744_83140608.pth... [2023-12-26 17:43:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000323304_82780160.pth [2023-12-26 17:43:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000323624_82853888.pth [2023-12-26 17:43:31,410][105692] Updated weights for policy 0, policy_version 324427 (0.0008) [2023-12-26 17:43:31,458][105692] Updated weights for policy 0, policy_version 324437 (0.0008) [2023-12-26 17:43:31,506][105620] Updated weights for policy 1, policy_version 324754 (0.0010) [2023-12-26 17:43:31,507][105692] Updated weights for policy 0, policy_version 324447 (0.0009) [2023-12-26 17:43:31,553][105620] Updated weights for policy 1, policy_version 324764 (0.0010) [2023-12-26 17:43:31,601][105620] Updated weights for policy 1, policy_version 324774 (0.0010) [2023-12-26 17:43:32,249][105692] Updated weights for policy 0, policy_version 324457 (0.0006) [2023-12-26 17:43:32,305][105692] Updated weights for policy 0, policy_version 324467 (0.0008) [2023-12-26 17:43:32,335][105620] Updated weights for policy 1, policy_version 324784 (0.0011) [2023-12-26 17:43:32,362][105692] Updated weights for policy 0, policy_version 324477 (0.0006) [2023-12-26 17:43:32,390][105620] Updated weights for policy 1, policy_version 324794 (0.0011) [2023-12-26 17:43:32,421][105692] Updated weights for policy 0, policy_version 324487 (0.0006) [2023-12-26 17:43:32,444][105620] Updated weights for policy 1, policy_version 324804 (0.0005) [2023-12-26 17:43:33,140][105692] Updated weights for policy 0, policy_version 324497 (0.0009) [2023-12-26 17:43:33,166][105620] Updated weights for policy 1, policy_version 324814 (0.0010) [2023-12-26 17:43:33,187][105692] Updated weights for policy 0, policy_version 324507 (0.0010) [2023-12-26 17:43:33,220][105620] Updated weights for policy 1, policy_version 324824 (0.0010) [2023-12-26 17:43:33,241][105692] Updated weights for policy 0, policy_version 324517 (0.0010) [2023-12-26 17:43:33,278][105620] Updated weights for policy 1, policy_version 324834 (0.0010) [2023-12-26 17:43:33,928][105692] Updated weights for policy 0, policy_version 324527 (0.0007) [2023-12-26 17:43:33,937][105620] Updated weights for policy 1, policy_version 324844 (0.0010) [2023-12-26 17:43:33,994][105620] Updated weights for policy 1, policy_version 324854 (0.0010) [2023-12-26 17:43:34,006][105692] Updated weights for policy 0, policy_version 324537 (0.0006) [2023-12-26 17:43:34,044][105620] Updated weights for policy 1, policy_version 324864 (0.0009) [2023-12-26 17:43:34,065][105692] Updated weights for policy 0, policy_version 324547 (0.0005) [2023-12-26 17:43:34,592][105692] Updated weights for policy 0, policy_version 324557 (0.0008) [2023-12-26 17:43:34,651][105692] Updated weights for policy 0, policy_version 324567 (0.0010) [2023-12-26 17:43:34,717][105692] Updated weights for policy 0, policy_version 324577 (0.0010) [2023-12-26 17:43:34,760][105620] Updated weights for policy 1, policy_version 324874 (0.0006) [2023-12-26 17:43:34,808][105620] Updated weights for policy 1, policy_version 324884 (0.0010) [2023-12-26 17:43:34,860][105620] Updated weights for policy 1, policy_version 324894 (0.0010) [2023-12-26 17:43:34,909][105620] Updated weights for policy 1, policy_version 324904 (0.0010) [2023-12-26 17:43:35,273][105692] Updated weights for policy 0, policy_version 324587 (0.0007) [2023-12-26 17:43:35,330][105692] Updated weights for policy 0, policy_version 324597 (0.0005) [2023-12-26 17:43:35,394][105692] Updated weights for policy 0, policy_version 324607 (0.0008) [2023-12-26 17:43:35,645][105620] Updated weights for policy 1, policy_version 324914 (0.0010) [2023-12-26 17:43:35,693][105620] Updated weights for policy 1, policy_version 324924 (0.0005) [2023-12-26 17:43:35,739][105620] Updated weights for policy 1, policy_version 324934 (0.0005) [2023-12-26 17:43:35,990][105692] Updated weights for policy 0, policy_version 324617 (0.0010) [2023-12-26 17:43:36,055][105692] Updated weights for policy 0, policy_version 324627 (0.0010) [2023-12-26 17:43:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19716.3). Total num frames: 166305792. Throughput: 0: 9787.7, 1: 9736.6. Samples: 166294580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:43:36,062][104569] Avg episode reward: [(0, '8814.652'), (1, '9266.579')] [2023-12-26 17:43:36,118][105692] Updated weights for policy 0, policy_version 324637 (0.0010) [2023-12-26 17:43:36,182][105692] Updated weights for policy 0, policy_version 324647 (0.0010) [2023-12-26 17:43:36,444][105620] Updated weights for policy 1, policy_version 324944 (0.0008) [2023-12-26 17:43:36,496][105620] Updated weights for policy 1, policy_version 324954 (0.0009) [2023-12-26 17:43:36,551][105620] Updated weights for policy 1, policy_version 324964 (0.0009) [2023-12-26 17:43:36,929][105692] Updated weights for policy 0, policy_version 324657 (0.0011) [2023-12-26 17:43:36,988][105692] Updated weights for policy 0, policy_version 324667 (0.0010) [2023-12-26 17:43:37,046][105692] Updated weights for policy 0, policy_version 324677 (0.0010) [2023-12-26 17:43:37,226][105620] Updated weights for policy 1, policy_version 324974 (0.0009) [2023-12-26 17:43:37,282][105620] Updated weights for policy 1, policy_version 324984 (0.0006) [2023-12-26 17:43:37,333][105620] Updated weights for policy 1, policy_version 324994 (0.0007) [2023-12-26 17:43:37,765][105692] Updated weights for policy 0, policy_version 324687 (0.0008) [2023-12-26 17:43:37,823][105692] Updated weights for policy 0, policy_version 324697 (0.0008) [2023-12-26 17:43:37,887][105692] Updated weights for policy 0, policy_version 324707 (0.0006) [2023-12-26 17:43:38,044][105620] Updated weights for policy 1, policy_version 325004 (0.0011) [2023-12-26 17:43:38,100][105620] Updated weights for policy 1, policy_version 325014 (0.0010) [2023-12-26 17:43:38,161][105620] Updated weights for policy 1, policy_version 325024 (0.0011) [2023-12-26 17:43:38,641][105692] Updated weights for policy 0, policy_version 324717 (0.0008) [2023-12-26 17:43:38,702][105692] Updated weights for policy 0, policy_version 324727 (0.0006) [2023-12-26 17:43:38,762][105692] Updated weights for policy 0, policy_version 324737 (0.0005) [2023-12-26 17:43:38,908][105620] Updated weights for policy 1, policy_version 325034 (0.0010) [2023-12-26 17:43:38,959][105620] Updated weights for policy 1, policy_version 325044 (0.0005) [2023-12-26 17:43:39,017][105620] Updated weights for policy 1, policy_version 325054 (0.0009) [2023-12-26 17:43:39,068][105620] Updated weights for policy 1, policy_version 325064 (0.0010) [2023-12-26 17:43:39,489][105692] Updated weights for policy 0, policy_version 324747 (0.0009) [2023-12-26 17:43:39,550][105692] Updated weights for policy 0, policy_version 324757 (0.0008) [2023-12-26 17:43:39,609][105692] Updated weights for policy 0, policy_version 324767 (0.0008) [2023-12-26 17:43:39,754][105620] Updated weights for policy 1, policy_version 325074 (0.0011) [2023-12-26 17:43:39,787][105586] KL-divergence is very high: 302.3884 [2023-12-26 17:43:39,820][105620] Updated weights for policy 1, policy_version 325084 (0.0011) [2023-12-26 17:43:39,842][105586] KL-divergence is very high: 509.3225 [2023-12-26 17:43:39,887][105620] Updated weights for policy 1, policy_version 325094 (0.0011) [2023-12-26 17:43:39,896][105586] KL-divergence is very high: 542.9682 [2023-12-26 17:43:40,320][105692] Updated weights for policy 0, policy_version 324777 (0.0008) [2023-12-26 17:43:40,379][105692] Updated weights for policy 0, policy_version 324787 (0.0006) [2023-12-26 17:43:40,433][105692] Updated weights for policy 0, policy_version 324797 (0.0006) [2023-12-26 17:43:40,499][105692] Updated weights for policy 0, policy_version 324807 (0.0006) [2023-12-26 17:43:40,526][105586] KL-divergence is very high: 551.8113 [2023-12-26 17:43:40,571][105620] Updated weights for policy 1, policy_version 325104 (0.0010) [2023-12-26 17:43:40,577][105586] KL-divergence is very high: 626.8721 [2023-12-26 17:43:40,620][105586] KL-divergence is very high: 519.6548 [2023-12-26 17:43:40,626][105620] Updated weights for policy 1, policy_version 325114 (0.0010) [2023-12-26 17:43:40,670][105586] KL-divergence is very high: 589.0897 [2023-12-26 17:43:40,688][105620] Updated weights for policy 1, policy_version 325124 (0.0010) [2023-12-26 17:43:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19716.4). Total num frames: 166404096. Throughput: 0: 9716.3, 1: 9702.3. Samples: 166413956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:43:41,062][104569] Avg episode reward: [(0, '8907.396'), (1, '9264.972')] [2023-12-26 17:43:41,170][105692] Updated weights for policy 0, policy_version 324817 (0.0008) [2023-12-26 17:43:41,222][105692] Updated weights for policy 0, policy_version 324827 (0.0008) [2023-12-26 17:43:41,282][105692] Updated weights for policy 0, policy_version 324837 (0.0008) [2023-12-26 17:43:41,477][105620] Updated weights for policy 1, policy_version 325134 (0.0008) [2023-12-26 17:43:41,537][105620] Updated weights for policy 1, policy_version 325144 (0.0005) [2023-12-26 17:43:41,594][105620] Updated weights for policy 1, policy_version 325154 (0.0006) [2023-12-26 17:43:42,077][105692] Updated weights for policy 0, policy_version 324847 (0.0010) [2023-12-26 17:43:42,129][105692] Updated weights for policy 0, policy_version 324857 (0.0010) [2023-12-26 17:43:42,183][105692] Updated weights for policy 0, policy_version 324867 (0.0011) [2023-12-26 17:43:42,301][105620] Updated weights for policy 1, policy_version 325164 (0.0009) [2023-12-26 17:43:42,377][105620] Updated weights for policy 1, policy_version 325174 (0.0008) [2023-12-26 17:43:42,435][105620] Updated weights for policy 1, policy_version 325184 (0.0011) [2023-12-26 17:43:42,961][105692] Updated weights for policy 0, policy_version 324877 (0.0011) [2023-12-26 17:43:43,028][105692] Updated weights for policy 0, policy_version 324887 (0.0011) [2023-12-26 17:43:43,084][105692] Updated weights for policy 0, policy_version 324897 (0.0010) [2023-12-26 17:43:43,100][105620] Updated weights for policy 1, policy_version 325194 (0.0007) [2023-12-26 17:43:43,160][105620] Updated weights for policy 1, policy_version 325204 (0.0011) [2023-12-26 17:43:43,225][105620] Updated weights for policy 1, policy_version 325214 (0.0011) [2023-12-26 17:43:43,292][105620] Updated weights for policy 1, policy_version 325224 (0.0011) [2023-12-26 17:43:43,766][105692] Updated weights for policy 0, policy_version 324907 (0.0009) [2023-12-26 17:43:43,824][105692] Updated weights for policy 0, policy_version 324917 (0.0006) [2023-12-26 17:43:43,885][105692] Updated weights for policy 0, policy_version 324927 (0.0008) [2023-12-26 17:43:44,019][105620] Updated weights for policy 1, policy_version 325234 (0.0010) [2023-12-26 17:43:44,071][105620] Updated weights for policy 1, policy_version 325244 (0.0007) [2023-12-26 17:43:44,129][105620] Updated weights for policy 1, policy_version 325254 (0.0009) [2023-12-26 17:43:44,618][105692] Updated weights for policy 0, policy_version 324937 (0.0009) [2023-12-26 17:43:44,669][105692] Updated weights for policy 0, policy_version 324947 (0.0008) [2023-12-26 17:43:44,726][105692] Updated weights for policy 0, policy_version 324957 (0.0006) [2023-12-26 17:43:44,789][105692] Updated weights for policy 0, policy_version 324967 (0.0007) [2023-12-26 17:43:44,837][105620] Updated weights for policy 1, policy_version 325264 (0.0009) [2023-12-26 17:43:44,901][105620] Updated weights for policy 1, policy_version 325274 (0.0006) [2023-12-26 17:43:44,967][105620] Updated weights for policy 1, policy_version 325284 (0.0005) [2023-12-26 17:43:45,543][105692] Updated weights for policy 0, policy_version 324977 (0.0009) [2023-12-26 17:43:45,598][105692] Updated weights for policy 0, policy_version 324987 (0.0008) [2023-12-26 17:43:45,651][105692] Updated weights for policy 0, policy_version 324997 (0.0010) [2023-12-26 17:43:45,688][105620] Updated weights for policy 1, policy_version 325294 (0.0006) [2023-12-26 17:43:45,739][105620] Updated weights for policy 1, policy_version 325304 (0.0005) [2023-12-26 17:43:45,792][105620] Updated weights for policy 1, policy_version 325314 (0.0008) [2023-12-26 17:43:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 166502400. Throughput: 0: 9651.3, 1: 9699.1. Samples: 166471044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:43:46,063][104569] Avg episode reward: [(0, '8454.091'), (1, '9264.962')] [2023-12-26 17:43:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000325000_83214336.pth... [2023-12-26 17:43:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000325320_83288064.pth... [2023-12-26 17:43:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000323848_82919424.pth [2023-12-26 17:43:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000324168_82993152.pth [2023-12-26 17:43:46,408][105692] Updated weights for policy 0, policy_version 325007 (0.0008) [2023-12-26 17:43:46,468][105692] Updated weights for policy 0, policy_version 325017 (0.0008) [2023-12-26 17:43:46,489][105620] Updated weights for policy 1, policy_version 325324 (0.0009) [2023-12-26 17:43:46,518][105586] KL-divergence is very high: 150.2852 [2023-12-26 17:43:46,522][105692] Updated weights for policy 0, policy_version 325027 (0.0008) [2023-12-26 17:43:46,533][105620] Updated weights for policy 1, policy_version 325334 (0.0006) [2023-12-26 17:43:46,537][105586] KL-divergence is very high: 144.5372 [2023-12-26 17:43:46,555][105586] KL-divergence is very high: 228.4524 [2023-12-26 17:43:46,573][105586] KL-divergence is very high: 155.8036 [2023-12-26 17:43:46,577][105620] Updated weights for policy 1, policy_version 325344 (0.0005) [2023-12-26 17:43:46,592][105586] KL-divergence is very high: 197.0667 [2023-12-26 17:43:46,612][105586] KL-divergence is very high: 118.1944 [2023-12-26 17:43:47,162][105692] Updated weights for policy 0, policy_version 325037 (0.0008) [2023-12-26 17:43:47,219][105692] Updated weights for policy 0, policy_version 325047 (0.0006) [2023-12-26 17:43:47,253][105620] Updated weights for policy 1, policy_version 325354 (0.0006) [2023-12-26 17:43:47,275][105692] Updated weights for policy 0, policy_version 325057 (0.0005) [2023-12-26 17:43:47,305][105620] Updated weights for policy 1, policy_version 325364 (0.0010) [2023-12-26 17:43:47,364][105620] Updated weights for policy 1, policy_version 325374 (0.0010) [2023-12-26 17:43:47,412][105620] Updated weights for policy 1, policy_version 325384 (0.0010) [2023-12-26 17:43:47,881][105692] Updated weights for policy 0, policy_version 325067 (0.0006) [2023-12-26 17:43:47,936][105692] Updated weights for policy 0, policy_version 325077 (0.0008) [2023-12-26 17:43:47,984][105692] Updated weights for policy 0, policy_version 325087 (0.0008) [2023-12-26 17:43:48,175][105620] Updated weights for policy 1, policy_version 325394 (0.0010) [2023-12-26 17:43:48,236][105620] Updated weights for policy 1, policy_version 325404 (0.0010) [2023-12-26 17:43:48,298][105620] Updated weights for policy 1, policy_version 325414 (0.0010) [2023-12-26 17:43:48,638][105692] Updated weights for policy 0, policy_version 325097 (0.0009) [2023-12-26 17:43:48,704][105692] Updated weights for policy 0, policy_version 325107 (0.0008) [2023-12-26 17:43:48,768][105692] Updated weights for policy 0, policy_version 325117 (0.0008) [2023-12-26 17:43:48,815][105692] Updated weights for policy 0, policy_version 325127 (0.0009) [2023-12-26 17:43:49,017][105620] Updated weights for policy 1, policy_version 325424 (0.0008) [2023-12-26 17:43:49,069][105620] Updated weights for policy 1, policy_version 325434 (0.0010) [2023-12-26 17:43:49,124][105620] Updated weights for policy 1, policy_version 325444 (0.0010) [2023-12-26 17:43:49,551][105692] Updated weights for policy 0, policy_version 325137 (0.0009) [2023-12-26 17:43:49,621][105692] Updated weights for policy 0, policy_version 325147 (0.0006) [2023-12-26 17:43:49,686][105692] Updated weights for policy 0, policy_version 325157 (0.0006) [2023-12-26 17:43:49,765][105620] Updated weights for policy 1, policy_version 325454 (0.0010) [2023-12-26 17:43:49,826][105620] Updated weights for policy 1, policy_version 325464 (0.0010) [2023-12-26 17:43:49,889][105620] Updated weights for policy 1, policy_version 325474 (0.0010) [2023-12-26 17:43:50,444][105692] Updated weights for policy 0, policy_version 325167 (0.0008) [2023-12-26 17:43:50,494][105620] Updated weights for policy 1, policy_version 325484 (0.0010) [2023-12-26 17:43:50,505][105692] Updated weights for policy 0, policy_version 325177 (0.0008) [2023-12-26 17:43:50,563][105620] Updated weights for policy 1, policy_version 325494 (0.0008) [2023-12-26 17:43:50,567][105692] Updated weights for policy 0, policy_version 325187 (0.0009) [2023-12-26 17:43:50,624][105620] Updated weights for policy 1, policy_version 325504 (0.0009) [2023-12-26 17:43:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 166600704. Throughput: 0: 9712.8, 1: 9800.3. Samples: 166590616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:43:51,063][104569] Avg episode reward: [(0, '5247.394'), (1, '9265.025')] [2023-12-26 17:43:51,290][105692] Updated weights for policy 0, policy_version 325197 (0.0008) [2023-12-26 17:43:51,348][105620] Updated weights for policy 1, policy_version 325514 (0.0006) [2023-12-26 17:43:51,351][105692] Updated weights for policy 0, policy_version 325207 (0.0008) [2023-12-26 17:43:51,412][105620] Updated weights for policy 1, policy_version 325524 (0.0010) [2023-12-26 17:43:51,419][105692] Updated weights for policy 0, policy_version 325217 (0.0008) [2023-12-26 17:43:51,462][105620] Updated weights for policy 1, policy_version 325534 (0.0006) [2023-12-26 17:43:51,527][105620] Updated weights for policy 1, policy_version 325544 (0.0009) [2023-12-26 17:43:52,123][105692] Updated weights for policy 0, policy_version 325227 (0.0008) [2023-12-26 17:43:52,170][105585] KL-divergence is very high: 111.0541 [2023-12-26 17:43:52,185][105585] KL-divergence is very high: 107.8285 [2023-12-26 17:43:52,191][105585] KL-divergence is very high: 142.0568 [2023-12-26 17:43:52,192][105692] Updated weights for policy 0, policy_version 325237 (0.0009) [2023-12-26 17:43:52,209][105585] KL-divergence is very high: 109.7145 [2023-12-26 17:43:52,253][105692] Updated weights for policy 0, policy_version 325247 (0.0009) [2023-12-26 17:43:52,295][105620] Updated weights for policy 1, policy_version 325554 (0.0010) [2023-12-26 17:43:52,351][105620] Updated weights for policy 1, policy_version 325564 (0.0008) [2023-12-26 17:43:52,410][105620] Updated weights for policy 1, policy_version 325574 (0.0009) [2023-12-26 17:43:52,982][105692] Updated weights for policy 0, policy_version 325257 (0.0008) [2023-12-26 17:43:52,988][105585] KL-divergence is very high: 123.1207 [2023-12-26 17:43:53,037][105585] KL-divergence is very high: 146.8314 [2023-12-26 17:43:53,043][105692] Updated weights for policy 0, policy_version 325267 (0.0006) [2023-12-26 17:43:53,084][105585] KL-divergence is very high: 123.1956 [2023-12-26 17:43:53,100][105692] Updated weights for policy 0, policy_version 325277 (0.0008) [2023-12-26 17:43:53,127][105585] KL-divergence is very high: 110.3545 [2023-12-26 17:43:53,153][105692] Updated weights for policy 0, policy_version 325287 (0.0007) [2023-12-26 17:43:53,191][105620] Updated weights for policy 1, policy_version 325584 (0.0010) [2023-12-26 17:43:53,245][105620] Updated weights for policy 1, policy_version 325594 (0.0010) [2023-12-26 17:43:53,302][105586] KL-divergence is very high: 112.0046 [2023-12-26 17:43:53,307][105620] Updated weights for policy 1, policy_version 325604 (0.0010) [2023-12-26 17:43:53,892][105692] Updated weights for policy 0, policy_version 325297 (0.0009) [2023-12-26 17:43:53,939][105692] Updated weights for policy 0, policy_version 325307 (0.0008) [2023-12-26 17:43:53,982][105586] KL-divergence is very high: 112.1039 [2023-12-26 17:43:53,993][105692] Updated weights for policy 0, policy_version 325317 (0.0009) [2023-12-26 17:43:54,017][105620] Updated weights for policy 1, policy_version 325614 (0.0007) [2023-12-26 17:43:54,070][105620] Updated weights for policy 1, policy_version 325624 (0.0005) [2023-12-26 17:43:54,131][105620] Updated weights for policy 1, policy_version 325634 (0.0006) [2023-12-26 17:43:54,753][105620] Updated weights for policy 1, policy_version 325644 (0.0009) [2023-12-26 17:43:54,812][105620] Updated weights for policy 1, policy_version 325654 (0.0009) [2023-12-26 17:43:54,814][105692] Updated weights for policy 0, policy_version 325327 (0.0007) [2023-12-26 17:43:54,870][105620] Updated weights for policy 1, policy_version 325664 (0.0007) [2023-12-26 17:43:54,879][105692] Updated weights for policy 0, policy_version 325337 (0.0006) [2023-12-26 17:43:54,938][105692] Updated weights for policy 0, policy_version 325347 (0.0009) [2023-12-26 17:43:55,601][105620] Updated weights for policy 1, policy_version 325674 (0.0009) [2023-12-26 17:43:55,665][105620] Updated weights for policy 1, policy_version 325684 (0.0006) [2023-12-26 17:43:55,698][105692] Updated weights for policy 0, policy_version 325357 (0.0008) [2023-12-26 17:43:55,730][105620] Updated weights for policy 1, policy_version 325694 (0.0010) [2023-12-26 17:43:55,757][105692] Updated weights for policy 0, policy_version 325367 (0.0009) [2023-12-26 17:43:55,780][105620] Updated weights for policy 1, policy_version 325704 (0.0010) [2023-12-26 17:43:55,806][105692] Updated weights for policy 0, policy_version 325377 (0.0009) [2023-12-26 17:43:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.8, 300 sec: 19688.6). Total num frames: 166699008. Throughput: 0: 9696.8, 1: 9783.5. Samples: 166705712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:43:56,062][104569] Avg episode reward: [(0, '4562.556'), (1, '9356.188')] [2023-12-26 17:43:56,430][105620] Updated weights for policy 1, policy_version 325714 (0.0008) [2023-12-26 17:43:56,486][105620] Updated weights for policy 1, policy_version 325724 (0.0005) [2023-12-26 17:43:56,539][105620] Updated weights for policy 1, policy_version 325734 (0.0008) [2023-12-26 17:43:56,547][105692] Updated weights for policy 0, policy_version 325387 (0.0009) [2023-12-26 17:43:56,591][105692] Updated weights for policy 0, policy_version 325397 (0.0008) [2023-12-26 17:43:56,643][105692] Updated weights for policy 0, policy_version 325407 (0.0008) [2023-12-26 17:43:57,194][105620] Updated weights for policy 1, policy_version 325744 (0.0006) [2023-12-26 17:43:57,244][105620] Updated weights for policy 1, policy_version 325754 (0.0005) [2023-12-26 17:43:57,246][105692] Updated weights for policy 0, policy_version 325417 (0.0008) [2023-12-26 17:43:57,297][105692] Updated weights for policy 0, policy_version 325427 (0.0006) [2023-12-26 17:43:57,303][105620] Updated weights for policy 1, policy_version 325764 (0.0005) [2023-12-26 17:43:57,355][105692] Updated weights for policy 0, policy_version 325437 (0.0006) [2023-12-26 17:43:57,414][105692] Updated weights for policy 0, policy_version 325447 (0.0005) [2023-12-26 17:43:57,829][105620] Updated weights for policy 1, policy_version 325774 (0.0008) [2023-12-26 17:43:57,887][105620] Updated weights for policy 1, policy_version 325784 (0.0010) [2023-12-26 17:43:57,938][105620] Updated weights for policy 1, policy_version 325794 (0.0010) [2023-12-26 17:43:57,939][105692] Updated weights for policy 0, policy_version 325457 (0.0005) [2023-12-26 17:43:57,996][105692] Updated weights for policy 0, policy_version 325467 (0.0005) [2023-12-26 17:43:58,049][105692] Updated weights for policy 0, policy_version 325477 (0.0005) [2023-12-26 17:43:58,735][105620] Updated weights for policy 1, policy_version 325804 (0.0011) [2023-12-26 17:43:58,805][105620] Updated weights for policy 1, policy_version 325814 (0.0008) [2023-12-26 17:43:58,824][105692] Updated weights for policy 0, policy_version 325487 (0.0008) [2023-12-26 17:43:58,871][105620] Updated weights for policy 1, policy_version 325824 (0.0010) [2023-12-26 17:43:58,886][105692] Updated weights for policy 0, policy_version 325497 (0.0007) [2023-12-26 17:43:58,948][105692] Updated weights for policy 0, policy_version 325507 (0.0008) [2023-12-26 17:43:59,682][105620] Updated weights for policy 1, policy_version 325834 (0.0010) [2023-12-26 17:43:59,733][105620] Updated weights for policy 1, policy_version 325844 (0.0010) [2023-12-26 17:43:59,736][105692] Updated weights for policy 0, policy_version 325517 (0.0007) [2023-12-26 17:43:59,788][105620] Updated weights for policy 1, policy_version 325854 (0.0010) [2023-12-26 17:43:59,794][105692] Updated weights for policy 0, policy_version 325527 (0.0006) [2023-12-26 17:43:59,853][105620] Updated weights for policy 1, policy_version 325864 (0.0009) [2023-12-26 17:43:59,859][105692] Updated weights for policy 0, policy_version 325537 (0.0008) [2023-12-26 17:44:00,504][105692] Updated weights for policy 0, policy_version 325547 (0.0009) [2023-12-26 17:44:00,563][105692] Updated weights for policy 0, policy_version 325557 (0.0008) [2023-12-26 17:44:00,590][105620] Updated weights for policy 1, policy_version 325874 (0.0010) [2023-12-26 17:44:00,615][105692] Updated weights for policy 0, policy_version 325567 (0.0006) [2023-12-26 17:44:00,637][105620] Updated weights for policy 1, policy_version 325884 (0.0010) [2023-12-26 17:44:00,688][105620] Updated weights for policy 1, policy_version 325894 (0.0009) [2023-12-26 17:44:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 166797312. Throughput: 0: 9837.7, 1: 9841.5. Samples: 166768836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:44:01,062][104569] Avg episode reward: [(0, '1059.479'), (1, '9355.999')] [2023-12-26 17:44:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000325576_83361792.pth... [2023-12-26 17:44:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000325896_83435520.pth... [2023-12-26 17:44:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000324744_83140608.pth [2023-12-26 17:44:01,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000324424_83066880.pth [2023-12-26 17:44:01,382][105692] Updated weights for policy 0, policy_version 325577 (0.0006) [2023-12-26 17:44:01,417][105620] Updated weights for policy 1, policy_version 325904 (0.0009) [2023-12-26 17:44:01,443][105692] Updated weights for policy 0, policy_version 325587 (0.0005) [2023-12-26 17:44:01,480][105620] Updated weights for policy 1, policy_version 325914 (0.0009) [2023-12-26 17:44:01,498][105692] Updated weights for policy 0, policy_version 325597 (0.0006) [2023-12-26 17:44:01,538][105620] Updated weights for policy 1, policy_version 325924 (0.0008) [2023-12-26 17:44:01,557][105692] Updated weights for policy 0, policy_version 325607 (0.0005) [2023-12-26 17:44:02,182][105620] Updated weights for policy 1, policy_version 325934 (0.0007) [2023-12-26 17:44:02,225][105692] Updated weights for policy 0, policy_version 325617 (0.0005) [2023-12-26 17:44:02,240][105620] Updated weights for policy 1, policy_version 325944 (0.0006) [2023-12-26 17:44:02,282][105692] Updated weights for policy 0, policy_version 325627 (0.0007) [2023-12-26 17:44:02,293][105620] Updated weights for policy 1, policy_version 325954 (0.0008) [2023-12-26 17:44:02,342][105692] Updated weights for policy 0, policy_version 325637 (0.0010) [2023-12-26 17:44:02,945][105692] Updated weights for policy 0, policy_version 325647 (0.0010) [2023-12-26 17:44:02,989][105692] Updated weights for policy 0, policy_version 325657 (0.0005) [2023-12-26 17:44:03,037][105692] Updated weights for policy 0, policy_version 325667 (0.0006) [2023-12-26 17:44:03,050][105620] Updated weights for policy 1, policy_version 325964 (0.0007) [2023-12-26 17:44:03,099][105620] Updated weights for policy 1, policy_version 325974 (0.0005) [2023-12-26 17:44:03,148][105620] Updated weights for policy 1, policy_version 325984 (0.0005) [2023-12-26 17:44:03,588][105692] Updated weights for policy 0, policy_version 325677 (0.0005) [2023-12-26 17:44:03,637][105692] Updated weights for policy 0, policy_version 325687 (0.0005) [2023-12-26 17:44:03,691][105692] Updated weights for policy 0, policy_version 325697 (0.0005) [2023-12-26 17:44:03,952][105620] Updated weights for policy 1, policy_version 325994 (0.0006) [2023-12-26 17:44:04,009][105620] Updated weights for policy 1, policy_version 326004 (0.0009) [2023-12-26 17:44:04,077][105620] Updated weights for policy 1, policy_version 326014 (0.0009) [2023-12-26 17:44:04,145][105620] Updated weights for policy 1, policy_version 326024 (0.0007) [2023-12-26 17:44:04,290][105692] Updated weights for policy 0, policy_version 325707 (0.0005) [2023-12-26 17:44:04,353][105692] Updated weights for policy 0, policy_version 325717 (0.0006) [2023-12-26 17:44:04,412][105692] Updated weights for policy 0, policy_version 325727 (0.0005) [2023-12-26 17:44:04,824][105620] Updated weights for policy 1, policy_version 326034 (0.0005) [2023-12-26 17:44:04,876][105620] Updated weights for policy 1, policy_version 326044 (0.0006) [2023-12-26 17:44:04,929][105620] Updated weights for policy 1, policy_version 326054 (0.0010) [2023-12-26 17:44:05,101][105692] Updated weights for policy 0, policy_version 325737 (0.0008) [2023-12-26 17:44:05,159][105692] Updated weights for policy 0, policy_version 325747 (0.0007) [2023-12-26 17:44:05,218][105692] Updated weights for policy 0, policy_version 325757 (0.0007) [2023-12-26 17:44:05,274][105692] Updated weights for policy 0, policy_version 325767 (0.0010) [2023-12-26 17:44:05,492][105620] Updated weights for policy 1, policy_version 326064 (0.0007) [2023-12-26 17:44:05,544][105620] Updated weights for policy 1, policy_version 326074 (0.0006) [2023-12-26 17:44:05,587][105620] Updated weights for policy 1, policy_version 326084 (0.0005) [2023-12-26 17:44:05,921][105692] Updated weights for policy 0, policy_version 325777 (0.0006) [2023-12-26 17:44:05,994][105692] Updated weights for policy 0, policy_version 325787 (0.0006) [2023-12-26 17:44:06,049][105692] Updated weights for policy 0, policy_version 325797 (0.0006) [2023-12-26 17:44:06,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19744.1). Total num frames: 166903808. Throughput: 0: 9968.2, 1: 9782.7. Samples: 166888232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:44:06,063][104569] Avg episode reward: [(0, '5124.739'), (1, '9355.876')] [2023-12-26 17:44:06,251][105620] Updated weights for policy 1, policy_version 326094 (0.0008) [2023-12-26 17:44:06,303][105620] Updated weights for policy 1, policy_version 326104 (0.0009) [2023-12-26 17:44:06,359][105620] Updated weights for policy 1, policy_version 326114 (0.0009) [2023-12-26 17:44:06,600][105692] Updated weights for policy 0, policy_version 325807 (0.0008) [2023-12-26 17:44:06,650][105692] Updated weights for policy 0, policy_version 325817 (0.0011) [2023-12-26 17:44:06,713][105692] Updated weights for policy 0, policy_version 325827 (0.0007) [2023-12-26 17:44:07,220][105620] Updated weights for policy 1, policy_version 326124 (0.0010) [2023-12-26 17:44:07,257][105692] Updated weights for policy 0, policy_version 325837 (0.0006) [2023-12-26 17:44:07,281][105620] Updated weights for policy 1, policy_version 326134 (0.0010) [2023-12-26 17:44:07,317][105692] Updated weights for policy 0, policy_version 325847 (0.0010) [2023-12-26 17:44:07,338][105620] Updated weights for policy 1, policy_version 326144 (0.0010) [2023-12-26 17:44:07,377][105692] Updated weights for policy 0, policy_version 325857 (0.0006) [2023-12-26 17:44:08,046][105620] Updated weights for policy 1, policy_version 326154 (0.0010) [2023-12-26 17:44:08,049][105692] Updated weights for policy 0, policy_version 325867 (0.0008) [2023-12-26 17:44:08,102][105620] Updated weights for policy 1, policy_version 326164 (0.0007) [2023-12-26 17:44:08,106][105692] Updated weights for policy 0, policy_version 325877 (0.0006) [2023-12-26 17:44:08,163][105692] Updated weights for policy 0, policy_version 325887 (0.0008) [2023-12-26 17:44:08,165][105620] Updated weights for policy 1, policy_version 326174 (0.0008) [2023-12-26 17:44:08,226][105620] Updated weights for policy 1, policy_version 326184 (0.0007) [2023-12-26 17:44:08,860][105692] Updated weights for policy 0, policy_version 325897 (0.0007) [2023-12-26 17:44:08,923][105692] Updated weights for policy 0, policy_version 325907 (0.0006) [2023-12-26 17:44:08,972][105620] Updated weights for policy 1, policy_version 326194 (0.0008) [2023-12-26 17:44:08,981][105692] Updated weights for policy 0, policy_version 325917 (0.0005) [2023-12-26 17:44:09,037][105620] Updated weights for policy 1, policy_version 326204 (0.0006) [2023-12-26 17:44:09,050][105692] Updated weights for policy 0, policy_version 325927 (0.0008) [2023-12-26 17:44:09,101][105620] Updated weights for policy 1, policy_version 326214 (0.0006) [2023-12-26 17:44:09,745][105692] Updated weights for policy 0, policy_version 325937 (0.0009) [2023-12-26 17:44:09,811][105692] Updated weights for policy 0, policy_version 325947 (0.0008) [2023-12-26 17:44:09,857][105620] Updated weights for policy 1, policy_version 326224 (0.0006) [2023-12-26 17:44:09,876][105692] Updated weights for policy 0, policy_version 325957 (0.0006) [2023-12-26 17:44:09,919][105620] Updated weights for policy 1, policy_version 326234 (0.0007) [2023-12-26 17:44:09,989][105620] Updated weights for policy 1, policy_version 326244 (0.0009) [2023-12-26 17:44:10,577][105692] Updated weights for policy 0, policy_version 325967 (0.0007) [2023-12-26 17:44:10,636][105692] Updated weights for policy 0, policy_version 325977 (0.0007) [2023-12-26 17:44:10,704][105692] Updated weights for policy 0, policy_version 325987 (0.0006) [2023-12-26 17:44:10,706][105620] Updated weights for policy 1, policy_version 326254 (0.0008) [2023-12-26 17:44:10,761][105620] Updated weights for policy 1, policy_version 326264 (0.0009) [2023-12-26 17:44:10,819][105620] Updated weights for policy 1, policy_version 326274 (0.0009) [2023-12-26 17:44:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19744.1). Total num frames: 167002112. Throughput: 0: 10068.4, 1: 9819.4. Samples: 167008292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:44:11,062][104569] Avg episode reward: [(0, '7466.963'), (1, '9355.876')] [2023-12-26 17:44:11,324][105692] Updated weights for policy 0, policy_version 325997 (0.0008) [2023-12-26 17:44:11,392][105692] Updated weights for policy 0, policy_version 326007 (0.0010) [2023-12-26 17:44:11,456][105692] Updated weights for policy 0, policy_version 326017 (0.0006) [2023-12-26 17:44:11,696][105620] Updated weights for policy 1, policy_version 326284 (0.0009) [2023-12-26 17:44:11,757][105620] Updated weights for policy 1, policy_version 326294 (0.0009) [2023-12-26 17:44:11,815][105620] Updated weights for policy 1, policy_version 326304 (0.0008) [2023-12-26 17:44:12,178][105692] Updated weights for policy 0, policy_version 326027 (0.0008) [2023-12-26 17:44:12,233][105692] Updated weights for policy 0, policy_version 326037 (0.0009) [2023-12-26 17:44:12,296][105692] Updated weights for policy 0, policy_version 326047 (0.0010) [2023-12-26 17:44:12,552][105620] Updated weights for policy 1, policy_version 326314 (0.0006) [2023-12-26 17:44:12,615][105620] Updated weights for policy 1, policy_version 326324 (0.0008) [2023-12-26 17:44:12,677][105620] Updated weights for policy 1, policy_version 326334 (0.0007) [2023-12-26 17:44:12,740][105620] Updated weights for policy 1, policy_version 326344 (0.0005) [2023-12-26 17:44:13,074][105692] Updated weights for policy 0, policy_version 326057 (0.0010) [2023-12-26 17:44:13,138][105692] Updated weights for policy 0, policy_version 326067 (0.0005) [2023-12-26 17:44:13,209][105692] Updated weights for policy 0, policy_version 326077 (0.0005) [2023-12-26 17:44:13,280][105692] Updated weights for policy 0, policy_version 326087 (0.0008) [2023-12-26 17:44:13,406][105620] Updated weights for policy 1, policy_version 326354 (0.0010) [2023-12-26 17:44:13,457][105620] Updated weights for policy 1, policy_version 326364 (0.0009) [2023-12-26 17:44:13,504][105620] Updated weights for policy 1, policy_version 326374 (0.0009) [2023-12-26 17:44:13,886][105692] Updated weights for policy 0, policy_version 326097 (0.0009) [2023-12-26 17:44:13,941][105692] Updated weights for policy 0, policy_version 326107 (0.0010) [2023-12-26 17:44:13,995][105692] Updated weights for policy 0, policy_version 326117 (0.0010) [2023-12-26 17:44:14,181][105620] Updated weights for policy 1, policy_version 326384 (0.0009) [2023-12-26 17:44:14,240][105620] Updated weights for policy 1, policy_version 326394 (0.0009) [2023-12-26 17:44:14,302][105620] Updated weights for policy 1, policy_version 326404 (0.0009) [2023-12-26 17:44:14,767][105692] Updated weights for policy 0, policy_version 326127 (0.0007) [2023-12-26 17:44:14,831][105692] Updated weights for policy 0, policy_version 326137 (0.0008) [2023-12-26 17:44:14,883][105692] Updated weights for policy 0, policy_version 326147 (0.0008) [2023-12-26 17:44:15,090][105620] Updated weights for policy 1, policy_version 326414 (0.0008) [2023-12-26 17:44:15,151][105620] Updated weights for policy 1, policy_version 326424 (0.0009) [2023-12-26 17:44:15,202][105620] Updated weights for policy 1, policy_version 326434 (0.0008) [2023-12-26 17:44:15,624][105692] Updated weights for policy 0, policy_version 326157 (0.0009) [2023-12-26 17:44:15,681][105692] Updated weights for policy 0, policy_version 326167 (0.0010) [2023-12-26 17:44:15,733][105692] Updated weights for policy 0, policy_version 326177 (0.0006) [2023-12-26 17:44:16,000][105620] Updated weights for policy 1, policy_version 326444 (0.0009) [2023-12-26 17:44:16,054][105620] Updated weights for policy 1, policy_version 326455 (0.0010) [2023-12-26 17:44:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19716.3). Total num frames: 167092224. Throughput: 0: 10004.5, 1: 9779.4. Samples: 167066412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:44:16,062][104569] Avg episode reward: [(0, '8543.337'), (1, '9355.915')] [2023-12-26 17:44:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000326184_83517440.pth... [2023-12-26 17:44:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000325000_83214336.pth [2023-12-26 17:44:16,104][105620] Updated weights for policy 1, policy_version 326465 (0.0009) [2023-12-26 17:44:16,143][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000326472_83582976.pth... [2023-12-26 17:44:16,146][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000325320_83288064.pth [2023-12-26 17:44:16,320][105692] Updated weights for policy 0, policy_version 326187 (0.0008) [2023-12-26 17:44:16,376][105692] Updated weights for policy 0, policy_version 326197 (0.0005) [2023-12-26 17:44:16,433][105692] Updated weights for policy 0, policy_version 326207 (0.0006) [2023-12-26 17:44:16,987][105620] Updated weights for policy 1, policy_version 326475 (0.0009) [2023-12-26 17:44:17,041][105620] Updated weights for policy 1, policy_version 326485 (0.0008) [2023-12-26 17:44:17,043][105692] Updated weights for policy 0, policy_version 326217 (0.0006) [2023-12-26 17:44:17,100][105620] Updated weights for policy 1, policy_version 326495 (0.0005) [2023-12-26 17:44:17,101][105692] Updated weights for policy 0, policy_version 326227 (0.0007) [2023-12-26 17:44:17,155][105692] Updated weights for policy 0, policy_version 326237 (0.0006) [2023-12-26 17:44:17,203][105692] Updated weights for policy 0, policy_version 326247 (0.0005) [2023-12-26 17:44:17,782][105620] Updated weights for policy 1, policy_version 326505 (0.0006) [2023-12-26 17:44:17,837][105620] Updated weights for policy 1, policy_version 326515 (0.0007) [2023-12-26 17:44:17,847][105692] Updated weights for policy 0, policy_version 326257 (0.0008) [2023-12-26 17:44:17,889][105620] Updated weights for policy 1, policy_version 326525 (0.0007) [2023-12-26 17:44:17,899][105692] Updated weights for policy 0, policy_version 326267 (0.0006) [2023-12-26 17:44:17,937][105620] Updated weights for policy 1, policy_version 326535 (0.0007) [2023-12-26 17:44:17,955][105692] Updated weights for policy 0, policy_version 326277 (0.0006) [2023-12-26 17:44:18,649][105692] Updated weights for policy 0, policy_version 326287 (0.0006) [2023-12-26 17:44:18,706][105692] Updated weights for policy 0, policy_version 326297 (0.0005) [2023-12-26 17:44:18,758][105620] Updated weights for policy 1, policy_version 326545 (0.0007) [2023-12-26 17:44:18,762][105692] Updated weights for policy 0, policy_version 326307 (0.0008) [2023-12-26 17:44:18,828][105620] Updated weights for policy 1, policy_version 326555 (0.0008) [2023-12-26 17:44:18,888][105620] Updated weights for policy 1, policy_version 326565 (0.0009) [2023-12-26 17:44:19,383][105692] Updated weights for policy 0, policy_version 326317 (0.0007) [2023-12-26 17:44:19,438][105692] Updated weights for policy 0, policy_version 326327 (0.0009) [2023-12-26 17:44:19,495][105692] Updated weights for policy 0, policy_version 326337 (0.0009) [2023-12-26 17:44:19,665][105620] Updated weights for policy 1, policy_version 326575 (0.0009) [2023-12-26 17:44:19,720][105620] Updated weights for policy 1, policy_version 326585 (0.0009) [2023-12-26 17:44:19,778][105620] Updated weights for policy 1, policy_version 326595 (0.0006) [2023-12-26 17:44:20,241][105692] Updated weights for policy 0, policy_version 326347 (0.0008) [2023-12-26 17:44:20,293][105692] Updated weights for policy 0, policy_version 326357 (0.0009) [2023-12-26 17:44:20,345][105692] Updated weights for policy 0, policy_version 326367 (0.0008) [2023-12-26 17:44:20,514][105620] Updated weights for policy 1, policy_version 326605 (0.0007) [2023-12-26 17:44:20,586][105620] Updated weights for policy 1, policy_version 326615 (0.0008) [2023-12-26 17:44:20,623][105586] KL-divergence is very high: 115.7117 [2023-12-26 17:44:20,636][105586] KL-divergence is very high: 155.5188 [2023-12-26 17:44:20,650][105586] KL-divergence is very high: 189.3956 [2023-12-26 17:44:20,651][105620] Updated weights for policy 1, policy_version 326625 (0.0008) [2023-12-26 17:44:20,658][105586] KL-divergence is very high: 212.7844 [2023-12-26 17:44:20,670][105586] KL-divergence is very high: 154.4261 [2023-12-26 17:44:20,677][105586] KL-divergence is very high: 224.3706 [2023-12-26 17:44:20,691][105586] KL-divergence is very high: 209.9265 [2023-12-26 17:44:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 167190528. Throughput: 0: 10056.3, 1: 9675.2. Samples: 167182500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:44:21,063][104569] Avg episode reward: [(0, '8722.843'), (1, '9355.951')] [2023-12-26 17:44:21,148][105692] Updated weights for policy 0, policy_version 326377 (0.0007) [2023-12-26 17:44:21,215][105692] Updated weights for policy 0, policy_version 326387 (0.0010) [2023-12-26 17:44:21,276][105692] Updated weights for policy 0, policy_version 326397 (0.0009) [2023-12-26 17:44:21,336][105692] Updated weights for policy 0, policy_version 326407 (0.0009) [2023-12-26 17:44:21,397][105620] Updated weights for policy 1, policy_version 326635 (0.0009) [2023-12-26 17:44:21,456][105620] Updated weights for policy 1, policy_version 326645 (0.0008) [2023-12-26 17:44:21,522][105620] Updated weights for policy 1, policy_version 326655 (0.0009) [2023-12-26 17:44:22,139][105692] Updated weights for policy 0, policy_version 326417 (0.0008) [2023-12-26 17:44:22,198][105692] Updated weights for policy 0, policy_version 326427 (0.0009) [2023-12-26 17:44:22,250][105692] Updated weights for policy 0, policy_version 326437 (0.0009) [2023-12-26 17:44:22,288][105620] Updated weights for policy 1, policy_version 326665 (0.0009) [2023-12-26 17:44:22,353][105620] Updated weights for policy 1, policy_version 326675 (0.0008) [2023-12-26 17:44:22,410][105620] Updated weights for policy 1, policy_version 326685 (0.0009) [2023-12-26 17:44:22,474][105620] Updated weights for policy 1, policy_version 326695 (0.0009) [2023-12-26 17:44:23,055][105692] Updated weights for policy 0, policy_version 326447 (0.0009) [2023-12-26 17:44:23,123][105692] Updated weights for policy 0, policy_version 326457 (0.0010) [2023-12-26 17:44:23,180][105692] Updated weights for policy 0, policy_version 326467 (0.0007) [2023-12-26 17:44:23,203][105620] Updated weights for policy 1, policy_version 326705 (0.0008) [2023-12-26 17:44:23,254][105620] Updated weights for policy 1, policy_version 326715 (0.0009) [2023-12-26 17:44:23,300][105620] Updated weights for policy 1, policy_version 326725 (0.0008) [2023-12-26 17:44:23,973][105692] Updated weights for policy 0, policy_version 326477 (0.0007) [2023-12-26 17:44:24,033][105692] Updated weights for policy 0, policy_version 326487 (0.0008) [2023-12-26 17:44:24,077][105692] Updated weights for policy 0, policy_version 326497 (0.0008) [2023-12-26 17:44:24,085][105620] Updated weights for policy 1, policy_version 326735 (0.0010) [2023-12-26 17:44:24,144][105620] Updated weights for policy 1, policy_version 326745 (0.0010) [2023-12-26 17:44:24,192][105620] Updated weights for policy 1, policy_version 326755 (0.0010) [2023-12-26 17:44:24,691][105692] Updated weights for policy 0, policy_version 326507 (0.0008) [2023-12-26 17:44:24,753][105692] Updated weights for policy 0, policy_version 326517 (0.0005) [2023-12-26 17:44:24,813][105692] Updated weights for policy 0, policy_version 326527 (0.0005) [2023-12-26 17:44:24,901][105620] Updated weights for policy 1, policy_version 326765 (0.0010) [2023-12-26 17:44:24,963][105620] Updated weights for policy 1, policy_version 326775 (0.0010) [2023-12-26 17:44:25,028][105620] Updated weights for policy 1, policy_version 326785 (0.0010) [2023-12-26 17:44:25,316][105692] Updated weights for policy 0, policy_version 326537 (0.0005) [2023-12-26 17:44:25,373][105692] Updated weights for policy 0, policy_version 326547 (0.0005) [2023-12-26 17:44:25,423][105692] Updated weights for policy 0, policy_version 326557 (0.0005) [2023-12-26 17:44:25,493][105692] Updated weights for policy 0, policy_version 326567 (0.0008) [2023-12-26 17:44:25,674][105620] Updated weights for policy 1, policy_version 326795 (0.0009) [2023-12-26 17:44:25,731][105620] Updated weights for policy 1, policy_version 326805 (0.0010) [2023-12-26 17:44:25,788][105620] Updated weights for policy 1, policy_version 326815 (0.0011) [2023-12-26 17:44:26,023][105692] Updated weights for policy 0, policy_version 326577 (0.0005) [2023-12-26 17:44:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 167288832. Throughput: 0: 10026.0, 1: 9635.3. Samples: 167298712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:44:26,063][104569] Avg episode reward: [(0, '8453.444'), (1, '9355.965')] [2023-12-26 17:44:26,069][105692] Updated weights for policy 0, policy_version 326587 (0.0005) [2023-12-26 17:44:26,128][105692] Updated weights for policy 0, policy_version 326597 (0.0005) [2023-12-26 17:44:26,378][105620] Updated weights for policy 1, policy_version 326825 (0.0007) [2023-12-26 17:44:26,438][105620] Updated weights for policy 1, policy_version 326835 (0.0011) [2023-12-26 17:44:26,497][105620] Updated weights for policy 1, policy_version 326845 (0.0011) [2023-12-26 17:44:26,563][105620] Updated weights for policy 1, policy_version 326855 (0.0010) [2023-12-26 17:44:26,830][105692] Updated weights for policy 0, policy_version 326607 (0.0009) [2023-12-26 17:44:26,889][105692] Updated weights for policy 0, policy_version 326617 (0.0011) [2023-12-26 17:44:26,941][105692] Updated weights for policy 0, policy_version 326627 (0.0010) [2023-12-26 17:44:27,150][105620] Updated weights for policy 1, policy_version 326865 (0.0007) [2023-12-26 17:44:27,162][105586] KL-divergence is very high: 133.6393 [2023-12-26 17:44:27,202][105586] KL-divergence is very high: 204.6982 [2023-12-26 17:44:27,203][105620] Updated weights for policy 1, policy_version 326875 (0.0006) [2023-12-26 17:44:27,249][105586] KL-divergence is very high: 179.1478 [2023-12-26 17:44:27,257][105620] Updated weights for policy 1, policy_version 326885 (0.0005) [2023-12-26 17:44:27,590][105692] Updated weights for policy 0, policy_version 326637 (0.0008) [2023-12-26 17:44:27,642][105692] Updated weights for policy 0, policy_version 326647 (0.0005) [2023-12-26 17:44:27,687][105692] Updated weights for policy 0, policy_version 326657 (0.0005) [2023-12-26 17:44:27,851][105620] Updated weights for policy 1, policy_version 326895 (0.0005) [2023-12-26 17:44:27,918][105620] Updated weights for policy 1, policy_version 326905 (0.0005) [2023-12-26 17:44:27,972][105620] Updated weights for policy 1, policy_version 326915 (0.0006) [2023-12-26 17:44:28,226][105692] Updated weights for policy 0, policy_version 326667 (0.0007) [2023-12-26 17:44:28,273][105692] Updated weights for policy 0, policy_version 326677 (0.0010) [2023-12-26 17:44:28,339][105692] Updated weights for policy 0, policy_version 326687 (0.0010) [2023-12-26 17:44:28,524][105620] Updated weights for policy 1, policy_version 326925 (0.0005) [2023-12-26 17:44:28,573][105620] Updated weights for policy 1, policy_version 326935 (0.0005) [2023-12-26 17:44:28,623][105620] Updated weights for policy 1, policy_version 326945 (0.0005) [2023-12-26 17:44:29,084][105692] Updated weights for policy 0, policy_version 326697 (0.0011) [2023-12-26 17:44:29,139][105692] Updated weights for policy 0, policy_version 326707 (0.0010) [2023-12-26 17:44:29,196][105692] Updated weights for policy 0, policy_version 326717 (0.0010) [2023-12-26 17:44:29,226][105620] Updated weights for policy 1, policy_version 326955 (0.0006) [2023-12-26 17:44:29,262][105692] Updated weights for policy 0, policy_version 326727 (0.0011) [2023-12-26 17:44:29,293][105620] Updated weights for policy 1, policy_version 326965 (0.0006) [2023-12-26 17:44:29,353][105620] Updated weights for policy 1, policy_version 326975 (0.0010) [2023-12-26 17:44:29,993][105692] Updated weights for policy 0, policy_version 326737 (0.0009) [2023-12-26 17:44:30,047][105620] Updated weights for policy 1, policy_version 326985 (0.0011) [2023-12-26 17:44:30,055][105692] Updated weights for policy 0, policy_version 326747 (0.0008) [2023-12-26 17:44:30,106][105620] Updated weights for policy 1, policy_version 326995 (0.0011) [2023-12-26 17:44:30,113][105692] Updated weights for policy 0, policy_version 326757 (0.0007) [2023-12-26 17:44:30,159][105620] Updated weights for policy 1, policy_version 327005 (0.0011) [2023-12-26 17:44:30,207][105620] Updated weights for policy 1, policy_version 327015 (0.0010) [2023-12-26 17:44:30,755][105692] Updated weights for policy 0, policy_version 326767 (0.0006) [2023-12-26 17:44:30,776][105620] Updated weights for policy 1, policy_version 327025 (0.0006) [2023-12-26 17:44:30,804][105692] Updated weights for policy 0, policy_version 326777 (0.0007) [2023-12-26 17:44:30,827][105620] Updated weights for policy 1, policy_version 327035 (0.0005) [2023-12-26 17:44:30,863][105692] Updated weights for policy 0, policy_version 326787 (0.0007) [2023-12-26 17:44:30,885][105620] Updated weights for policy 1, policy_version 327045 (0.0011) [2023-12-26 17:44:31,062][104569] Fps is (10 sec: 21299.4, 60 sec: 19933.9, 300 sec: 19716.3). Total num frames: 167403520. Throughput: 0: 10131.9, 1: 9781.1. Samples: 167367128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:44:31,062][104569] Avg episode reward: [(0, '8724.027'), (1, '9355.954')] [2023-12-26 17:44:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000326792_83673088.pth... [2023-12-26 17:44:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000327048_83730432.pth... [2023-12-26 17:44:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000325576_83361792.pth [2023-12-26 17:44:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000325896_83435520.pth [2023-12-26 17:44:31,590][105692] Updated weights for policy 0, policy_version 326797 (0.0005) [2023-12-26 17:44:31,592][105620] Updated weights for policy 1, policy_version 327055 (0.0010) [2023-12-26 17:44:31,655][105692] Updated weights for policy 0, policy_version 326807 (0.0008) [2023-12-26 17:44:31,659][105620] Updated weights for policy 1, policy_version 327065 (0.0008) [2023-12-26 17:44:31,715][105692] Updated weights for policy 0, policy_version 326817 (0.0007) [2023-12-26 17:44:31,725][105620] Updated weights for policy 1, policy_version 327075 (0.0008) [2023-12-26 17:44:32,390][105620] Updated weights for policy 1, policy_version 327085 (0.0008) [2023-12-26 17:44:32,446][105620] Updated weights for policy 1, policy_version 327095 (0.0009) [2023-12-26 17:44:32,493][105620] Updated weights for policy 1, policy_version 327105 (0.0009) [2023-12-26 17:44:32,517][105692] Updated weights for policy 0, policy_version 326827 (0.0007) [2023-12-26 17:44:32,550][105585] KL-divergence is very high: 474.9048 [2023-12-26 17:44:32,574][105692] Updated weights for policy 0, policy_version 326837 (0.0008) [2023-12-26 17:44:32,595][105585] KL-divergence is very high: 678.8147 [2023-12-26 17:44:32,630][105692] Updated weights for policy 0, policy_version 326847 (0.0008) [2023-12-26 17:44:32,639][105585] KL-divergence is very high: 569.1049 [2023-12-26 17:44:33,138][105620] Updated weights for policy 1, policy_version 327115 (0.0007) [2023-12-26 17:44:33,183][105620] Updated weights for policy 1, policy_version 327125 (0.0005) [2023-12-26 17:44:33,229][105620] Updated weights for policy 1, policy_version 327135 (0.0005) [2023-12-26 17:44:33,457][105692] Updated weights for policy 0, policy_version 326857 (0.0009) [2023-12-26 17:44:33,522][105692] Updated weights for policy 0, policy_version 326868 (0.0011) [2023-12-26 17:44:33,580][105692] Updated weights for policy 0, policy_version 326879 (0.0011) [2023-12-26 17:44:33,791][105620] Updated weights for policy 1, policy_version 327145 (0.0005) [2023-12-26 17:44:33,847][105620] Updated weights for policy 1, policy_version 327155 (0.0005) [2023-12-26 17:44:33,908][105620] Updated weights for policy 1, policy_version 327165 (0.0005) [2023-12-26 17:44:33,957][105620] Updated weights for policy 1, policy_version 327175 (0.0006) [2023-12-26 17:44:34,348][105692] Updated weights for policy 0, policy_version 326890 (0.0010) [2023-12-26 17:44:34,403][105692] Updated weights for policy 0, policy_version 326900 (0.0009) [2023-12-26 17:44:34,464][105692] Updated weights for policy 0, policy_version 326910 (0.0009) [2023-12-26 17:44:34,528][105692] Updated weights for policy 0, policy_version 326920 (0.0009) [2023-12-26 17:44:34,639][105620] Updated weights for policy 1, policy_version 327185 (0.0006) [2023-12-26 17:44:34,699][105620] Updated weights for policy 1, policy_version 327195 (0.0005) [2023-12-26 17:44:34,758][105620] Updated weights for policy 1, policy_version 327205 (0.0007) [2023-12-26 17:44:35,304][105692] Updated weights for policy 0, policy_version 326930 (0.0009) [2023-12-26 17:44:35,357][105692] Updated weights for policy 0, policy_version 326940 (0.0008) [2023-12-26 17:44:35,410][105692] Updated weights for policy 0, policy_version 326950 (0.0008) [2023-12-26 17:44:35,458][105620] Updated weights for policy 1, policy_version 327215 (0.0009) [2023-12-26 17:44:35,506][105620] Updated weights for policy 1, policy_version 327225 (0.0009) [2023-12-26 17:44:35,554][105620] Updated weights for policy 1, policy_version 327235 (0.0008) [2023-12-26 17:44:36,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.3, 300 sec: 19688.5). Total num frames: 167493632. Throughput: 0: 10030.0, 1: 9877.1. Samples: 167486440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:44:36,063][104569] Avg episode reward: [(0, '8632.810'), (1, '9355.923')] [2023-12-26 17:44:36,177][105620] Updated weights for policy 1, policy_version 327245 (0.0008) [2023-12-26 17:44:36,204][105692] Updated weights for policy 0, policy_version 326960 (0.0007) [2023-12-26 17:44:36,244][105620] Updated weights for policy 1, policy_version 327255 (0.0006) [2023-12-26 17:44:36,274][105692] Updated weights for policy 0, policy_version 326970 (0.0009) [2023-12-26 17:44:36,311][105620] Updated weights for policy 1, policy_version 327265 (0.0006) [2023-12-26 17:44:36,335][105692] Updated weights for policy 0, policy_version 326980 (0.0009) [2023-12-26 17:44:36,889][105620] Updated weights for policy 1, policy_version 327275 (0.0006) [2023-12-26 17:44:36,952][105620] Updated weights for policy 1, policy_version 327285 (0.0009) [2023-12-26 17:44:37,012][105620] Updated weights for policy 1, policy_version 327295 (0.0009) [2023-12-26 17:44:37,156][105692] Updated weights for policy 0, policy_version 326990 (0.0008) [2023-12-26 17:44:37,224][105692] Updated weights for policy 0, policy_version 327000 (0.0009) [2023-12-26 17:44:37,289][105692] Updated weights for policy 0, policy_version 327010 (0.0009) [2023-12-26 17:44:37,712][105620] Updated weights for policy 1, policy_version 327305 (0.0006) [2023-12-26 17:44:37,781][105620] Updated weights for policy 1, policy_version 327315 (0.0009) [2023-12-26 17:44:37,850][105620] Updated weights for policy 1, policy_version 327325 (0.0010) [2023-12-26 17:44:37,918][105620] Updated weights for policy 1, policy_version 327335 (0.0010) [2023-12-26 17:44:37,968][105692] Updated weights for policy 0, policy_version 327020 (0.0009) [2023-12-26 17:44:38,020][105692] Updated weights for policy 0, policy_version 327030 (0.0009) [2023-12-26 17:44:38,073][105692] Updated weights for policy 0, policy_version 327040 (0.0008) [2023-12-26 17:44:38,606][105620] Updated weights for policy 1, policy_version 327345 (0.0006) [2023-12-26 17:44:38,657][105620] Updated weights for policy 1, policy_version 327355 (0.0005) [2023-12-26 17:44:38,725][105620] Updated weights for policy 1, policy_version 327365 (0.0005) [2023-12-26 17:44:38,770][105692] Updated weights for policy 0, policy_version 327050 (0.0008) [2023-12-26 17:44:38,832][105692] Updated weights for policy 0, policy_version 327060 (0.0005) [2023-12-26 17:44:38,904][105692] Updated weights for policy 0, policy_version 327070 (0.0005) [2023-12-26 17:44:38,980][105692] Updated weights for policy 0, policy_version 327080 (0.0009) [2023-12-26 17:44:39,376][105620] Updated weights for policy 1, policy_version 327375 (0.0007) [2023-12-26 17:44:39,443][105620] Updated weights for policy 1, policy_version 327385 (0.0008) [2023-12-26 17:44:39,494][105620] Updated weights for policy 1, policy_version 327395 (0.0008) [2023-12-26 17:44:39,622][105692] Updated weights for policy 0, policy_version 327090 (0.0007) [2023-12-26 17:44:39,689][105692] Updated weights for policy 0, policy_version 327100 (0.0007) [2023-12-26 17:44:39,755][105692] Updated weights for policy 0, policy_version 327110 (0.0009) [2023-12-26 17:44:40,239][105620] Updated weights for policy 1, policy_version 327405 (0.0008) [2023-12-26 17:44:40,301][105620] Updated weights for policy 1, policy_version 327415 (0.0009) [2023-12-26 17:44:40,371][105620] Updated weights for policy 1, policy_version 327425 (0.0010) [2023-12-26 17:44:40,501][105692] Updated weights for policy 0, policy_version 327120 (0.0009) [2023-12-26 17:44:40,554][105692] Updated weights for policy 0, policy_version 327130 (0.0010) [2023-12-26 17:44:40,608][105692] Updated weights for policy 0, policy_version 327140 (0.0009) [2023-12-26 17:44:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 167591936. Throughput: 0: 10048.5, 1: 9908.3. Samples: 167603768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:44:41,062][104569] Avg episode reward: [(0, '8994.771'), (1, '9265.177')] [2023-12-26 17:44:41,084][105620] Updated weights for policy 1, policy_version 327435 (0.0010) [2023-12-26 17:44:41,148][105620] Updated weights for policy 1, policy_version 327445 (0.0010) [2023-12-26 17:44:41,215][105620] Updated weights for policy 1, policy_version 327455 (0.0007) [2023-12-26 17:44:41,378][105692] Updated weights for policy 0, policy_version 327150 (0.0008) [2023-12-26 17:44:41,449][105692] Updated weights for policy 0, policy_version 327160 (0.0008) [2023-12-26 17:44:41,508][105692] Updated weights for policy 0, policy_version 327170 (0.0010) [2023-12-26 17:44:41,895][105620] Updated weights for policy 1, policy_version 327465 (0.0008) [2023-12-26 17:44:41,966][105620] Updated weights for policy 1, policy_version 327475 (0.0008) [2023-12-26 17:44:42,033][105620] Updated weights for policy 1, policy_version 327485 (0.0008) [2023-12-26 17:44:42,098][105620] Updated weights for policy 1, policy_version 327495 (0.0008) [2023-12-26 17:44:42,274][105692] Updated weights for policy 0, policy_version 327180 (0.0009) [2023-12-26 17:44:42,335][105692] Updated weights for policy 0, policy_version 327190 (0.0011) [2023-12-26 17:44:42,407][105692] Updated weights for policy 0, policy_version 327200 (0.0010) [2023-12-26 17:44:42,750][105620] Updated weights for policy 1, policy_version 327505 (0.0006) [2023-12-26 17:44:42,816][105620] Updated weights for policy 1, policy_version 327515 (0.0005) [2023-12-26 17:44:42,876][105620] Updated weights for policy 1, policy_version 327525 (0.0005) [2023-12-26 17:44:43,020][105692] Updated weights for policy 0, policy_version 327210 (0.0010) [2023-12-26 17:44:43,073][105692] Updated weights for policy 0, policy_version 327220 (0.0007) [2023-12-26 17:44:43,130][105692] Updated weights for policy 0, policy_version 327230 (0.0009) [2023-12-26 17:44:43,185][105692] Updated weights for policy 0, policy_version 327240 (0.0010) [2023-12-26 17:44:43,479][105620] Updated weights for policy 1, policy_version 327535 (0.0009) [2023-12-26 17:44:43,534][105620] Updated weights for policy 1, policy_version 327545 (0.0010) [2023-12-26 17:44:43,596][105620] Updated weights for policy 1, policy_version 327555 (0.0010) [2023-12-26 17:44:43,945][105692] Updated weights for policy 0, policy_version 327250 (0.0011) [2023-12-26 17:44:44,009][105692] Updated weights for policy 0, policy_version 327260 (0.0011) [2023-12-26 17:44:44,065][105692] Updated weights for policy 0, policy_version 327270 (0.0010) [2023-12-26 17:44:44,294][105620] Updated weights for policy 1, policy_version 327565 (0.0008) [2023-12-26 17:44:44,360][105620] Updated weights for policy 1, policy_version 327575 (0.0008) [2023-12-26 17:44:44,428][105620] Updated weights for policy 1, policy_version 327585 (0.0008) [2023-12-26 17:44:44,756][105692] Updated weights for policy 0, policy_version 327280 (0.0007) [2023-12-26 17:44:44,824][105692] Updated weights for policy 0, policy_version 327290 (0.0008) [2023-12-26 17:44:44,890][105692] Updated weights for policy 0, policy_version 327300 (0.0007) [2023-12-26 17:44:45,175][105620] Updated weights for policy 1, policy_version 327595 (0.0007) [2023-12-26 17:44:45,239][105620] Updated weights for policy 1, policy_version 327605 (0.0009) [2023-12-26 17:44:45,240][105586] KL-divergence is very high: 116.0869 [2023-12-26 17:44:45,289][105586] KL-divergence is very high: 141.3784 [2023-12-26 17:44:45,302][105620] Updated weights for policy 1, policy_version 327615 (0.0009) [2023-12-26 17:44:45,339][105586] KL-divergence is very high: 107.1100 [2023-12-26 17:44:45,591][105692] Updated weights for policy 0, policy_version 327310 (0.0007) [2023-12-26 17:44:45,639][105692] Updated weights for policy 0, policy_version 327320 (0.0009) [2023-12-26 17:44:45,697][105692] Updated weights for policy 0, policy_version 327330 (0.0009) [2023-12-26 17:44:46,038][105620] Updated weights for policy 1, policy_version 327625 (0.0009) [2023-12-26 17:44:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19716.3). Total num frames: 167690240. Throughput: 0: 9961.5, 1: 9908.5. Samples: 167662988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:44:46,063][104569] Avg episode reward: [(0, '9085.957'), (1, '8914.795')] [2023-12-26 17:44:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000327336_83812352.pth... [2023-12-26 17:44:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000326184_83517440.pth [2023-12-26 17:44:46,091][105586] KL-divergence is very high: 127.6384 [2023-12-26 17:44:46,098][105620] Updated weights for policy 1, policy_version 327635 (0.0008) [2023-12-26 17:44:46,133][105586] KL-divergence is very high: 160.5738 [2023-12-26 17:44:46,149][105620] Updated weights for policy 1, policy_version 327645 (0.0007) [2023-12-26 17:44:46,204][105620] Updated weights for policy 1, policy_version 327655 (0.0008) [2023-12-26 17:44:46,210][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000327656_83886080.pth... [2023-12-26 17:44:46,214][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000326472_83582976.pth [2023-12-26 17:44:46,474][105692] Updated weights for policy 0, policy_version 327340 (0.0009) [2023-12-26 17:44:46,527][105692] Updated weights for policy 0, policy_version 327350 (0.0009) [2023-12-26 17:44:46,580][105692] Updated weights for policy 0, policy_version 327360 (0.0009) [2023-12-26 17:44:46,978][105620] Updated weights for policy 1, policy_version 327665 (0.0009) [2023-12-26 17:44:47,036][105620] Updated weights for policy 1, policy_version 327675 (0.0009) [2023-12-26 17:44:47,089][105620] Updated weights for policy 1, policy_version 327685 (0.0009) [2023-12-26 17:44:47,243][105692] Updated weights for policy 0, policy_version 327370 (0.0006) [2023-12-26 17:44:47,296][105692] Updated weights for policy 0, policy_version 327380 (0.0007) [2023-12-26 17:44:47,350][105692] Updated weights for policy 0, policy_version 327390 (0.0007) [2023-12-26 17:44:47,409][105692] Updated weights for policy 0, policy_version 327400 (0.0007) [2023-12-26 17:44:47,909][105620] Updated weights for policy 1, policy_version 327695 (0.0010) [2023-12-26 17:44:47,966][105620] Updated weights for policy 1, policy_version 327705 (0.0009) [2023-12-26 17:44:48,016][105620] Updated weights for policy 1, policy_version 327715 (0.0009) [2023-12-26 17:44:48,066][105692] Updated weights for policy 0, policy_version 327410 (0.0007) [2023-12-26 17:44:48,119][105692] Updated weights for policy 0, policy_version 327420 (0.0009) [2023-12-26 17:44:48,175][105692] Updated weights for policy 0, policy_version 327430 (0.0006) [2023-12-26 17:44:48,817][105620] Updated weights for policy 1, policy_version 327725 (0.0008) [2023-12-26 17:44:48,875][105620] Updated weights for policy 1, policy_version 327735 (0.0009) [2023-12-26 17:44:48,933][105620] Updated weights for policy 1, policy_version 327745 (0.0008) [2023-12-26 17:44:48,940][105692] Updated weights for policy 0, policy_version 327440 (0.0006) [2023-12-26 17:44:48,987][105692] Updated weights for policy 0, policy_version 327450 (0.0008) [2023-12-26 17:44:49,035][105692] Updated weights for policy 0, policy_version 327460 (0.0009) [2023-12-26 17:44:49,723][105620] Updated weights for policy 1, policy_version 327755 (0.0009) [2023-12-26 17:44:49,774][105692] Updated weights for policy 0, policy_version 327470 (0.0008) [2023-12-26 17:44:49,783][105620] Updated weights for policy 1, policy_version 327765 (0.0006) [2023-12-26 17:44:49,832][105692] Updated weights for policy 0, policy_version 327480 (0.0008) [2023-12-26 17:44:49,848][105620] Updated weights for policy 1, policy_version 327775 (0.0009) [2023-12-26 17:44:49,896][105692] Updated weights for policy 0, policy_version 327490 (0.0007) [2023-12-26 17:44:50,543][105620] Updated weights for policy 1, policy_version 327785 (0.0007) [2023-12-26 17:44:50,605][105620] Updated weights for policy 1, policy_version 327795 (0.0007) [2023-12-26 17:44:50,670][105620] Updated weights for policy 1, policy_version 327805 (0.0006) [2023-12-26 17:44:50,683][105692] Updated weights for policy 0, policy_version 327500 (0.0010) [2023-12-26 17:44:50,731][105620] Updated weights for policy 1, policy_version 327815 (0.0006) [2023-12-26 17:44:50,750][105692] Updated weights for policy 0, policy_version 327510 (0.0009) [2023-12-26 17:44:50,809][105692] Updated weights for policy 0, policy_version 327520 (0.0010) [2023-12-26 17:44:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19688.6). Total num frames: 167788544. Throughput: 0: 9896.5, 1: 9832.4. Samples: 167776032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:44:51,062][104569] Avg episode reward: [(0, '9268.533'), (1, '8914.741')] [2023-12-26 17:44:51,434][105620] Updated weights for policy 1, policy_version 327825 (0.0008) [2023-12-26 17:44:51,485][105620] Updated weights for policy 1, policy_version 327835 (0.0009) [2023-12-26 17:44:51,488][105692] Updated weights for policy 0, policy_version 327530 (0.0010) [2023-12-26 17:44:51,535][105620] Updated weights for policy 1, policy_version 327845 (0.0008) [2023-12-26 17:44:51,549][105692] Updated weights for policy 0, policy_version 327540 (0.0009) [2023-12-26 17:44:51,608][105692] Updated weights for policy 0, policy_version 327550 (0.0011) [2023-12-26 17:44:51,671][105692] Updated weights for policy 0, policy_version 327560 (0.0007) [2023-12-26 17:44:52,308][105620] Updated weights for policy 1, policy_version 327855 (0.0006) [2023-12-26 17:44:52,308][105692] Updated weights for policy 0, policy_version 327570 (0.0008) [2023-12-26 17:44:52,368][105692] Updated weights for policy 0, policy_version 327580 (0.0008) [2023-12-26 17:44:52,370][105620] Updated weights for policy 1, policy_version 327865 (0.0007) [2023-12-26 17:44:52,418][105692] Updated weights for policy 0, policy_version 327590 (0.0008) [2023-12-26 17:44:52,423][105620] Updated weights for policy 1, policy_version 327875 (0.0007) [2023-12-26 17:44:52,999][105692] Updated weights for policy 0, policy_version 327600 (0.0010) [2023-12-26 17:44:53,058][105692] Updated weights for policy 0, policy_version 327610 (0.0009) [2023-12-26 17:44:53,112][105692] Updated weights for policy 0, policy_version 327620 (0.0010) [2023-12-26 17:44:53,241][105620] Updated weights for policy 1, policy_version 327885 (0.0007) [2023-12-26 17:44:53,304][105620] Updated weights for policy 1, policy_version 327895 (0.0008) [2023-12-26 17:44:53,328][105586] KL-divergence is very high: 133.2554 [2023-12-26 17:44:53,365][105620] Updated weights for policy 1, policy_version 327905 (0.0009) [2023-12-26 17:44:53,381][105586] KL-divergence is very high: 115.2898 [2023-12-26 17:44:53,839][105692] Updated weights for policy 0, policy_version 327630 (0.0011) [2023-12-26 17:44:53,900][105692] Updated weights for policy 0, policy_version 327640 (0.0009) [2023-12-26 17:44:53,952][105692] Updated weights for policy 0, policy_version 327650 (0.0011) [2023-12-26 17:44:54,092][105620] Updated weights for policy 1, policy_version 327915 (0.0007) [2023-12-26 17:44:54,151][105620] Updated weights for policy 1, policy_version 327925 (0.0008) [2023-12-26 17:44:54,205][105620] Updated weights for policy 1, policy_version 327935 (0.0005) [2023-12-26 17:44:54,702][105692] Updated weights for policy 0, policy_version 327660 (0.0011) [2023-12-26 17:44:54,769][105692] Updated weights for policy 0, policy_version 327670 (0.0010) [2023-12-26 17:44:54,824][105692] Updated weights for policy 0, policy_version 327680 (0.0010) [2023-12-26 17:44:54,880][105620] Updated weights for policy 1, policy_version 327945 (0.0006) [2023-12-26 17:44:54,928][105620] Updated weights for policy 1, policy_version 327955 (0.0008) [2023-12-26 17:44:54,990][105620] Updated weights for policy 1, policy_version 327965 (0.0010) [2023-12-26 17:44:55,051][105620] Updated weights for policy 1, policy_version 327975 (0.0008) [2023-12-26 17:44:55,523][105692] Updated weights for policy 0, policy_version 327690 (0.0010) [2023-12-26 17:44:55,585][105692] Updated weights for policy 0, policy_version 327700 (0.0010) [2023-12-26 17:44:55,643][105692] Updated weights for policy 0, policy_version 327710 (0.0010) [2023-12-26 17:44:55,700][105692] Updated weights for policy 0, policy_version 327720 (0.0010) [2023-12-26 17:44:55,819][105620] Updated weights for policy 1, policy_version 327985 (0.0008) [2023-12-26 17:44:55,866][105620] Updated weights for policy 1, policy_version 327995 (0.0007) [2023-12-26 17:44:55,912][105620] Updated weights for policy 1, policy_version 328005 (0.0007) [2023-12-26 17:44:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19716.3). Total num frames: 167886848. Throughput: 0: 9823.0, 1: 9816.5. Samples: 167892072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:44:56,063][104569] Avg episode reward: [(0, '9268.596'), (1, '8738.348')] [2023-12-26 17:44:56,445][105692] Updated weights for policy 0, policy_version 327730 (0.0011) [2023-12-26 17:44:56,508][105692] Updated weights for policy 0, policy_version 327740 (0.0011) [2023-12-26 17:44:56,564][105692] Updated weights for policy 0, policy_version 327750 (0.0010) [2023-12-26 17:44:56,706][105620] Updated weights for policy 1, policy_version 328015 (0.0010) [2023-12-26 17:44:56,765][105620] Updated weights for policy 1, policy_version 328025 (0.0010) [2023-12-26 17:44:56,817][105620] Updated weights for policy 1, policy_version 328035 (0.0010) [2023-12-26 17:44:57,303][105692] Updated weights for policy 0, policy_version 327760 (0.0010) [2023-12-26 17:44:57,357][105692] Updated weights for policy 0, policy_version 327770 (0.0010) [2023-12-26 17:44:57,411][105692] Updated weights for policy 0, policy_version 327780 (0.0010) [2023-12-26 17:44:57,530][105620] Updated weights for policy 1, policy_version 328045 (0.0008) [2023-12-26 17:44:57,582][105620] Updated weights for policy 1, policy_version 328055 (0.0005) [2023-12-26 17:44:57,630][105620] Updated weights for policy 1, policy_version 328065 (0.0005) [2023-12-26 17:44:57,991][105692] Updated weights for policy 0, policy_version 327790 (0.0007) [2023-12-26 17:44:58,054][105692] Updated weights for policy 0, policy_version 327800 (0.0005) [2023-12-26 17:44:58,122][105692] Updated weights for policy 0, policy_version 327810 (0.0010) [2023-12-26 17:44:58,334][105620] Updated weights for policy 1, policy_version 328075 (0.0010) [2023-12-26 17:44:58,404][105620] Updated weights for policy 1, policy_version 328085 (0.0010) [2023-12-26 17:44:58,466][105620] Updated weights for policy 1, policy_version 328095 (0.0008) [2023-12-26 17:44:58,908][105692] Updated weights for policy 0, policy_version 327820 (0.0009) [2023-12-26 17:44:58,962][105692] Updated weights for policy 0, policy_version 327830 (0.0008) [2023-12-26 17:44:59,027][105692] Updated weights for policy 0, policy_version 327840 (0.0009) [2023-12-26 17:44:59,192][105620] Updated weights for policy 1, policy_version 328105 (0.0009) [2023-12-26 17:44:59,258][105620] Updated weights for policy 1, policy_version 328115 (0.0009) [2023-12-26 17:44:59,321][105620] Updated weights for policy 1, policy_version 328125 (0.0006) [2023-12-26 17:44:59,387][105620] Updated weights for policy 1, policy_version 328135 (0.0008) [2023-12-26 17:44:59,742][105692] Updated weights for policy 0, policy_version 327850 (0.0009) [2023-12-26 17:44:59,792][105692] Updated weights for policy 0, policy_version 327860 (0.0009) [2023-12-26 17:44:59,859][105692] Updated weights for policy 0, policy_version 327870 (0.0011) [2023-12-26 17:44:59,907][105692] Updated weights for policy 0, policy_version 327880 (0.0010) [2023-12-26 17:45:00,012][105620] Updated weights for policy 1, policy_version 328145 (0.0008) [2023-12-26 17:45:00,065][105620] Updated weights for policy 1, policy_version 328155 (0.0008) [2023-12-26 17:45:00,116][105620] Updated weights for policy 1, policy_version 328165 (0.0008) [2023-12-26 17:45:00,600][105692] Updated weights for policy 0, policy_version 327890 (0.0009) [2023-12-26 17:45:00,662][105692] Updated weights for policy 0, policy_version 327900 (0.0010) [2023-12-26 17:45:00,724][105692] Updated weights for policy 0, policy_version 327910 (0.0008) [2023-12-26 17:45:00,881][105620] Updated weights for policy 1, policy_version 328175 (0.0007) [2023-12-26 17:45:00,925][105620] Updated weights for policy 1, policy_version 328185 (0.0008) [2023-12-26 17:45:00,983][105620] Updated weights for policy 1, policy_version 328195 (0.0008) [2023-12-26 17:45:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19716.3). Total num frames: 167985152. Throughput: 0: 9832.5, 1: 9827.3. Samples: 167951104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:45:01,062][104569] Avg episode reward: [(0, '9268.475'), (1, '8818.895')] [2023-12-26 17:45:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000327912_83959808.pth... [2023-12-26 17:45:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000328200_84025344.pth... [2023-12-26 17:45:01,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000327048_83730432.pth [2023-12-26 17:45:01,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000326792_83673088.pth [2023-12-26 17:45:01,418][105692] Updated weights for policy 0, policy_version 327920 (0.0009) [2023-12-26 17:45:01,470][105692] Updated weights for policy 0, policy_version 327930 (0.0010) [2023-12-26 17:45:01,521][105692] Updated weights for policy 0, policy_version 327940 (0.0010) [2023-12-26 17:45:01,792][105620] Updated weights for policy 1, policy_version 328205 (0.0007) [2023-12-26 17:45:01,858][105620] Updated weights for policy 1, policy_version 328215 (0.0005) [2023-12-26 17:45:01,920][105620] Updated weights for policy 1, policy_version 328225 (0.0006) [2023-12-26 17:45:02,287][105692] Updated weights for policy 0, policy_version 327950 (0.0010) [2023-12-26 17:45:02,340][105692] Updated weights for policy 0, policy_version 327960 (0.0010) [2023-12-26 17:45:02,404][105692] Updated weights for policy 0, policy_version 327970 (0.0010) [2023-12-26 17:45:02,455][105620] Updated weights for policy 1, policy_version 328235 (0.0006) [2023-12-26 17:45:02,517][105620] Updated weights for policy 1, policy_version 328245 (0.0008) [2023-12-26 17:45:02,583][105620] Updated weights for policy 1, policy_version 328255 (0.0008) [2023-12-26 17:45:03,103][105692] Updated weights for policy 0, policy_version 327980 (0.0010) [2023-12-26 17:45:03,156][105692] Updated weights for policy 0, policy_version 327990 (0.0008) [2023-12-26 17:45:03,214][105692] Updated weights for policy 0, policy_version 328001 (0.0011) [2023-12-26 17:45:03,258][105620] Updated weights for policy 1, policy_version 328265 (0.0007) [2023-12-26 17:45:03,316][105620] Updated weights for policy 1, policy_version 328275 (0.0005) [2023-12-26 17:45:03,373][105620] Updated weights for policy 1, policy_version 328285 (0.0005) [2023-12-26 17:45:03,422][105620] Updated weights for policy 1, policy_version 328295 (0.0005) [2023-12-26 17:45:03,969][105620] Updated weights for policy 1, policy_version 328305 (0.0005) [2023-12-26 17:45:04,021][105620] Updated weights for policy 1, policy_version 328315 (0.0005) [2023-12-26 17:45:04,074][105620] Updated weights for policy 1, policy_version 328325 (0.0005) [2023-12-26 17:45:04,083][105692] Updated weights for policy 0, policy_version 328011 (0.0006) [2023-12-26 17:45:04,148][105692] Updated weights for policy 0, policy_version 328021 (0.0010) [2023-12-26 17:45:04,212][105692] Updated weights for policy 0, policy_version 328031 (0.0010) [2023-12-26 17:45:04,692][105620] Updated weights for policy 1, policy_version 328335 (0.0008) [2023-12-26 17:45:04,753][105620] Updated weights for policy 1, policy_version 328345 (0.0009) [2023-12-26 17:45:04,811][105620] Updated weights for policy 1, policy_version 328355 (0.0009) [2023-12-26 17:45:04,994][105692] Updated weights for policy 0, policy_version 328041 (0.0010) [2023-12-26 17:45:05,056][105692] Updated weights for policy 0, policy_version 328051 (0.0010) [2023-12-26 17:45:05,103][105692] Updated weights for policy 0, policy_version 328061 (0.0006) [2023-12-26 17:45:05,172][105692] Updated weights for policy 0, policy_version 328071 (0.0005) [2023-12-26 17:45:05,582][105620] Updated weights for policy 1, policy_version 328365 (0.0007) [2023-12-26 17:45:05,642][105620] Updated weights for policy 1, policy_version 328375 (0.0005) [2023-12-26 17:45:05,691][105620] Updated weights for policy 1, policy_version 328385 (0.0008) [2023-12-26 17:45:05,697][105692] Updated weights for policy 0, policy_version 328081 (0.0010) [2023-12-26 17:45:05,755][105692] Updated weights for policy 0, policy_version 328091 (0.0010) [2023-12-26 17:45:05,806][105692] Updated weights for policy 0, policy_version 328101 (0.0010) [2023-12-26 17:45:06,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19716.3). Total num frames: 168083456. Throughput: 0: 9700.9, 1: 10000.2. Samples: 168069048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:45:06,062][104569] Avg episode reward: [(0, '9177.848'), (1, '8821.369')] [2023-12-26 17:45:06,431][105620] Updated weights for policy 1, policy_version 328395 (0.0007) [2023-12-26 17:45:06,486][105620] Updated weights for policy 1, policy_version 328405 (0.0009) [2023-12-26 17:45:06,512][105692] Updated weights for policy 0, policy_version 328111 (0.0007) [2023-12-26 17:45:06,545][105620] Updated weights for policy 1, policy_version 328415 (0.0007) [2023-12-26 17:45:06,578][105692] Updated weights for policy 0, policy_version 328121 (0.0009) [2023-12-26 17:45:06,640][105692] Updated weights for policy 0, policy_version 328131 (0.0010) [2023-12-26 17:45:07,291][105620] Updated weights for policy 1, policy_version 328425 (0.0007) [2023-12-26 17:45:07,353][105620] Updated weights for policy 1, policy_version 328435 (0.0008) [2023-12-26 17:45:07,386][105692] Updated weights for policy 0, policy_version 328141 (0.0010) [2023-12-26 17:45:07,416][105620] Updated weights for policy 1, policy_version 328445 (0.0005) [2023-12-26 17:45:07,441][105692] Updated weights for policy 0, policy_version 328151 (0.0011) [2023-12-26 17:45:07,476][105620] Updated weights for policy 1, policy_version 328455 (0.0007) [2023-12-26 17:45:07,505][105692] Updated weights for policy 0, policy_version 328161 (0.0005) [2023-12-26 17:45:08,054][105692] Updated weights for policy 0, policy_version 328171 (0.0006) [2023-12-26 17:45:08,103][105692] Updated weights for policy 0, policy_version 328181 (0.0005) [2023-12-26 17:45:08,156][105692] Updated weights for policy 0, policy_version 328191 (0.0005) [2023-12-26 17:45:08,331][105620] Updated weights for policy 1, policy_version 328465 (0.0008) [2023-12-26 17:45:08,402][105620] Updated weights for policy 1, policy_version 328475 (0.0008) [2023-12-26 17:45:08,468][105620] Updated weights for policy 1, policy_version 328485 (0.0009) [2023-12-26 17:45:08,863][105692] Updated weights for policy 0, policy_version 328201 (0.0007) [2023-12-26 17:45:08,921][105692] Updated weights for policy 0, policy_version 328211 (0.0009) [2023-12-26 17:45:08,978][105692] Updated weights for policy 0, policy_version 328221 (0.0006) [2023-12-26 17:45:09,027][105692] Updated weights for policy 0, policy_version 328231 (0.0005) [2023-12-26 17:45:09,251][105620] Updated weights for policy 1, policy_version 328495 (0.0008) [2023-12-26 17:45:09,313][105620] Updated weights for policy 1, policy_version 328505 (0.0007) [2023-12-26 17:45:09,379][105620] Updated weights for policy 1, policy_version 328515 (0.0009) [2023-12-26 17:45:09,715][105692] Updated weights for policy 0, policy_version 328241 (0.0009) [2023-12-26 17:45:09,762][105692] Updated weights for policy 0, policy_version 328251 (0.0009) [2023-12-26 17:45:09,814][105692] Updated weights for policy 0, policy_version 328261 (0.0009) [2023-12-26 17:45:10,125][105620] Updated weights for policy 1, policy_version 328525 (0.0008) [2023-12-26 17:45:10,179][105620] Updated weights for policy 1, policy_version 328535 (0.0008) [2023-12-26 17:45:10,239][105620] Updated weights for policy 1, policy_version 328545 (0.0008) [2023-12-26 17:45:10,627][105692] Updated weights for policy 0, policy_version 328271 (0.0007) [2023-12-26 17:45:10,671][105692] Updated weights for policy 0, policy_version 328281 (0.0005) [2023-12-26 17:45:10,720][105692] Updated weights for policy 0, policy_version 328291 (0.0005) [2023-12-26 17:45:11,058][105620] Updated weights for policy 1, policy_version 328555 (0.0009) [2023-12-26 17:45:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19688.9). Total num frames: 168173568. Throughput: 0: 9761.7, 1: 9914.1. Samples: 168184120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:45:11,062][104569] Avg episode reward: [(0, '9177.777'), (1, '8821.358')] [2023-12-26 17:45:11,118][105620] Updated weights for policy 1, policy_version 328565 (0.0009) [2023-12-26 17:45:11,183][105620] Updated weights for policy 1, policy_version 328575 (0.0009) [2023-12-26 17:45:11,391][105692] Updated weights for policy 0, policy_version 328301 (0.0006) [2023-12-26 17:45:11,441][105692] Updated weights for policy 0, policy_version 328311 (0.0008) [2023-12-26 17:45:11,492][105692] Updated weights for policy 0, policy_version 328321 (0.0009) [2023-12-26 17:45:11,964][105620] Updated weights for policy 1, policy_version 328585 (0.0009) [2023-12-26 17:45:12,030][105620] Updated weights for policy 1, policy_version 328595 (0.0011) [2023-12-26 17:45:12,093][105620] Updated weights for policy 1, policy_version 328605 (0.0010) [2023-12-26 17:45:12,160][105620] Updated weights for policy 1, policy_version 328615 (0.0007) [2023-12-26 17:45:12,244][105692] Updated weights for policy 0, policy_version 328331 (0.0008) [2023-12-26 17:45:12,308][105692] Updated weights for policy 0, policy_version 328341 (0.0008) [2023-12-26 17:45:12,374][105692] Updated weights for policy 0, policy_version 328351 (0.0008) [2023-12-26 17:45:12,857][105620] Updated weights for policy 1, policy_version 328625 (0.0006) [2023-12-26 17:45:12,913][105620] Updated weights for policy 1, policy_version 328635 (0.0008) [2023-12-26 17:45:12,918][105586] KL-divergence is very high: 197.9891 [2023-12-26 17:45:12,955][105692] Updated weights for policy 0, policy_version 328361 (0.0005) [2023-12-26 17:45:12,958][105586] KL-divergence is very high: 343.1136 [2023-12-26 17:45:12,963][105620] Updated weights for policy 1, policy_version 328645 (0.0009) [2023-12-26 17:45:13,020][105692] Updated weights for policy 0, policy_version 328371 (0.0010) [2023-12-26 17:45:13,067][105692] Updated weights for policy 0, policy_version 328381 (0.0009) [2023-12-26 17:45:13,118][105692] Updated weights for policy 0, policy_version 328391 (0.0009) [2023-12-26 17:45:13,601][105620] Updated weights for policy 1, policy_version 328655 (0.0006) [2023-12-26 17:45:13,655][105620] Updated weights for policy 1, policy_version 328665 (0.0009) [2023-12-26 17:45:13,723][105620] Updated weights for policy 1, policy_version 328675 (0.0010) [2023-12-26 17:45:13,860][105692] Updated weights for policy 0, policy_version 328401 (0.0010) [2023-12-26 17:45:13,911][105692] Updated weights for policy 0, policy_version 328411 (0.0010) [2023-12-26 17:45:13,966][105692] Updated weights for policy 0, policy_version 328421 (0.0011) [2023-12-26 17:45:14,400][105620] Updated weights for policy 1, policy_version 328685 (0.0008) [2023-12-26 17:45:14,451][105620] Updated weights for policy 1, policy_version 328695 (0.0005) [2023-12-26 17:45:14,514][105620] Updated weights for policy 1, policy_version 328705 (0.0007) [2023-12-26 17:45:14,624][105692] Updated weights for policy 0, policy_version 328431 (0.0007) [2023-12-26 17:45:14,691][105692] Updated weights for policy 0, policy_version 328441 (0.0005) [2023-12-26 17:45:14,760][105692] Updated weights for policy 0, policy_version 328451 (0.0006) [2023-12-26 17:45:15,249][105620] Updated weights for policy 1, policy_version 328716 (0.0010) [2023-12-26 17:45:15,311][105620] Updated weights for policy 1, policy_version 328726 (0.0008) [2023-12-26 17:45:15,365][105620] Updated weights for policy 1, policy_version 328736 (0.0008) [2023-12-26 17:45:15,434][105692] Updated weights for policy 0, policy_version 328461 (0.0009) [2023-12-26 17:45:15,498][105692] Updated weights for policy 0, policy_version 328471 (0.0011) [2023-12-26 17:45:15,557][105692] Updated weights for policy 0, policy_version 328481 (0.0010) [2023-12-26 17:45:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 168271872. Throughput: 0: 9697.7, 1: 9771.7. Samples: 168243252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:45:16,062][104569] Avg episode reward: [(0, '9268.633'), (1, '8821.753')] [2023-12-26 17:45:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000328488_84107264.pth... [2023-12-26 17:45:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000328744_84164608.pth... [2023-12-26 17:45:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000327336_83812352.pth [2023-12-26 17:45:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000327656_83886080.pth [2023-12-26 17:45:16,123][105620] Updated weights for policy 1, policy_version 328746 (0.0008) [2023-12-26 17:45:16,177][105620] Updated weights for policy 1, policy_version 328756 (0.0009) [2023-12-26 17:45:16,196][105692] Updated weights for policy 0, policy_version 328491 (0.0008) [2023-12-26 17:45:16,233][105620] Updated weights for policy 1, policy_version 328766 (0.0009) [2023-12-26 17:45:16,243][105692] Updated weights for policy 0, policy_version 328501 (0.0006) [2023-12-26 17:45:16,283][105620] Updated weights for policy 1, policy_version 328776 (0.0005) [2023-12-26 17:45:16,293][105692] Updated weights for policy 0, policy_version 328511 (0.0007) [2023-12-26 17:45:16,944][105620] Updated weights for policy 1, policy_version 328786 (0.0005) [2023-12-26 17:45:17,010][105620] Updated weights for policy 1, policy_version 328796 (0.0009) [2023-12-26 17:45:17,037][105692] Updated weights for policy 0, policy_version 328521 (0.0009) [2023-12-26 17:45:17,064][105620] Updated weights for policy 1, policy_version 328806 (0.0007) [2023-12-26 17:45:17,086][105692] Updated weights for policy 0, policy_version 328531 (0.0008) [2023-12-26 17:45:17,139][105692] Updated weights for policy 0, policy_version 328541 (0.0008) [2023-12-26 17:45:17,203][105692] Updated weights for policy 0, policy_version 328551 (0.0009) [2023-12-26 17:45:17,625][105620] Updated weights for policy 1, policy_version 328816 (0.0007) [2023-12-26 17:45:17,680][105620] Updated weights for policy 1, policy_version 328826 (0.0009) [2023-12-26 17:45:17,727][105620] Updated weights for policy 1, policy_version 328836 (0.0009) [2023-12-26 17:45:18,023][105692] Updated weights for policy 0, policy_version 328561 (0.0009) [2023-12-26 17:45:18,082][105692] Updated weights for policy 0, policy_version 328571 (0.0009) [2023-12-26 17:45:18,132][105692] Updated weights for policy 0, policy_version 328581 (0.0010) [2023-12-26 17:45:18,440][105620] Updated weights for policy 1, policy_version 328846 (0.0009) [2023-12-26 17:45:18,499][105620] Updated weights for policy 1, policy_version 328856 (0.0009) [2023-12-26 17:45:18,554][105620] Updated weights for policy 1, policy_version 328866 (0.0009) [2023-12-26 17:45:18,923][105692] Updated weights for policy 0, policy_version 328592 (0.0009) [2023-12-26 17:45:18,989][105692] Updated weights for policy 0, policy_version 328602 (0.0009) [2023-12-26 17:45:19,055][105692] Updated weights for policy 0, policy_version 328612 (0.0009) [2023-12-26 17:45:19,276][105620] Updated weights for policy 1, policy_version 328876 (0.0008) [2023-12-26 17:45:19,342][105620] Updated weights for policy 1, policy_version 328886 (0.0009) [2023-12-26 17:45:19,411][105620] Updated weights for policy 1, policy_version 328896 (0.0009) [2023-12-26 17:45:19,815][105692] Updated weights for policy 0, policy_version 328622 (0.0009) [2023-12-26 17:45:19,882][105692] Updated weights for policy 0, policy_version 328632 (0.0010) [2023-12-26 17:45:19,952][105692] Updated weights for policy 0, policy_version 328642 (0.0008) [2023-12-26 17:45:20,181][105620] Updated weights for policy 1, policy_version 328906 (0.0010) [2023-12-26 17:45:20,244][105620] Updated weights for policy 1, policy_version 328916 (0.0009) [2023-12-26 17:45:20,310][105620] Updated weights for policy 1, policy_version 328926 (0.0009) [2023-12-26 17:45:20,373][105620] Updated weights for policy 1, policy_version 328936 (0.0009) [2023-12-26 17:45:20,670][105692] Updated weights for policy 0, policy_version 328652 (0.0008) [2023-12-26 17:45:20,726][105692] Updated weights for policy 0, policy_version 328662 (0.0009) [2023-12-26 17:45:20,788][105692] Updated weights for policy 0, policy_version 328672 (0.0008) [2023-12-26 17:45:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 168370176. Throughput: 0: 9755.4, 1: 9686.9. Samples: 168361340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:45:21,063][104569] Avg episode reward: [(0, '9176.947'), (1, '8821.418')] [2023-12-26 17:45:21,178][105620] Updated weights for policy 1, policy_version 328946 (0.0009) [2023-12-26 17:45:21,236][105620] Updated weights for policy 1, policy_version 328956 (0.0009) [2023-12-26 17:45:21,302][105620] Updated weights for policy 1, policy_version 328966 (0.0009) [2023-12-26 17:45:21,564][105692] Updated weights for policy 0, policy_version 328682 (0.0009) [2023-12-26 17:45:21,632][105692] Updated weights for policy 0, policy_version 328692 (0.0009) [2023-12-26 17:45:21,695][105692] Updated weights for policy 0, policy_version 328702 (0.0008) [2023-12-26 17:45:21,766][105692] Updated weights for policy 0, policy_version 328712 (0.0008) [2023-12-26 17:45:22,007][105620] Updated weights for policy 1, policy_version 328976 (0.0010) [2023-12-26 17:45:22,065][105620] Updated weights for policy 1, policy_version 328986 (0.0008) [2023-12-26 17:45:22,124][105620] Updated weights for policy 1, policy_version 328996 (0.0007) [2023-12-26 17:45:22,403][105692] Updated weights for policy 0, policy_version 328722 (0.0009) [2023-12-26 17:45:22,471][105692] Updated weights for policy 0, policy_version 328732 (0.0008) [2023-12-26 17:45:22,534][105692] Updated weights for policy 0, policy_version 328742 (0.0008) [2023-12-26 17:45:22,900][105620] Updated weights for policy 1, policy_version 329006 (0.0011) [2023-12-26 17:45:22,958][105620] Updated weights for policy 1, policy_version 329016 (0.0010) [2023-12-26 17:45:23,004][105620] Updated weights for policy 1, policy_version 329026 (0.0010) [2023-12-26 17:45:23,323][105692] Updated weights for policy 0, policy_version 328752 (0.0008) [2023-12-26 17:45:23,381][105692] Updated weights for policy 0, policy_version 328762 (0.0008) [2023-12-26 17:45:23,440][105692] Updated weights for policy 0, policy_version 328772 (0.0008) [2023-12-26 17:45:23,780][105620] Updated weights for policy 1, policy_version 329036 (0.0009) [2023-12-26 17:45:23,843][105620] Updated weights for policy 1, policy_version 329046 (0.0008) [2023-12-26 17:45:23,894][105620] Updated weights for policy 1, policy_version 329056 (0.0007) [2023-12-26 17:45:24,165][105692] Updated weights for policy 0, policy_version 328782 (0.0008) [2023-12-26 17:45:24,227][105692] Updated weights for policy 0, policy_version 328792 (0.0009) [2023-12-26 17:45:24,292][105692] Updated weights for policy 0, policy_version 328802 (0.0009) [2023-12-26 17:45:24,642][105620] Updated weights for policy 1, policy_version 329066 (0.0008) [2023-12-26 17:45:24,696][105620] Updated weights for policy 1, policy_version 329076 (0.0009) [2023-12-26 17:45:24,747][105620] Updated weights for policy 1, policy_version 329086 (0.0009) [2023-12-26 17:45:24,794][105620] Updated weights for policy 1, policy_version 329096 (0.0007) [2023-12-26 17:45:25,029][105692] Updated weights for policy 0, policy_version 328812 (0.0009) [2023-12-26 17:45:25,082][105692] Updated weights for policy 0, policy_version 328822 (0.0007) [2023-12-26 17:45:25,136][105692] Updated weights for policy 0, policy_version 328832 (0.0009) [2023-12-26 17:45:25,526][105620] Updated weights for policy 1, policy_version 329106 (0.0008) [2023-12-26 17:45:25,583][105620] Updated weights for policy 1, policy_version 329116 (0.0009) [2023-12-26 17:45:25,643][105620] Updated weights for policy 1, policy_version 329126 (0.0009) [2023-12-26 17:45:25,889][105692] Updated weights for policy 0, policy_version 328842 (0.0009) [2023-12-26 17:45:25,954][105692] Updated weights for policy 0, policy_version 328852 (0.0009) [2023-12-26 17:45:26,013][105692] Updated weights for policy 0, policy_version 328862 (0.0010) [2023-12-26 17:45:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 168468480. Throughput: 0: 9744.8, 1: 9573.2. Samples: 168473076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:45:26,062][104569] Avg episode reward: [(0, '9359.740'), (1, '9177.740')] [2023-12-26 17:45:26,063][105585] Saving new best policy, reward=9359.740! [2023-12-26 17:45:26,064][105692] Updated weights for policy 0, policy_version 328872 (0.0009) [2023-12-26 17:45:26,361][105620] Updated weights for policy 1, policy_version 329136 (0.0009) [2023-12-26 17:45:26,419][105620] Updated weights for policy 1, policy_version 329146 (0.0007) [2023-12-26 17:45:26,477][105620] Updated weights for policy 1, policy_version 329156 (0.0009) [2023-12-26 17:45:26,850][105692] Updated weights for policy 0, policy_version 328882 (0.0011) [2023-12-26 17:45:26,909][105692] Updated weights for policy 0, policy_version 328893 (0.0010) [2023-12-26 17:45:26,962][105692] Updated weights for policy 0, policy_version 328903 (0.0010) [2023-12-26 17:45:27,067][105620] Updated weights for policy 1, policy_version 329166 (0.0007) [2023-12-26 17:45:27,131][105620] Updated weights for policy 1, policy_version 329176 (0.0008) [2023-12-26 17:45:27,196][105620] Updated weights for policy 1, policy_version 329186 (0.0009) [2023-12-26 17:45:27,761][105692] Updated weights for policy 0, policy_version 328913 (0.0009) [2023-12-26 17:45:27,807][105692] Updated weights for policy 0, policy_version 328923 (0.0008) [2023-12-26 17:45:27,853][105692] Updated weights for policy 0, policy_version 328933 (0.0008) [2023-12-26 17:45:27,873][105620] Updated weights for policy 1, policy_version 329196 (0.0009) [2023-12-26 17:45:27,926][105620] Updated weights for policy 1, policy_version 329206 (0.0009) [2023-12-26 17:45:27,978][105620] Updated weights for policy 1, policy_version 329216 (0.0006) [2023-12-26 17:45:28,655][105692] Updated weights for policy 0, policy_version 328943 (0.0010) [2023-12-26 17:45:28,666][105620] Updated weights for policy 1, policy_version 329226 (0.0007) [2023-12-26 17:45:28,711][105692] Updated weights for policy 0, policy_version 328953 (0.0008) [2023-12-26 17:45:28,720][105620] Updated weights for policy 1, policy_version 329236 (0.0005) [2023-12-26 17:45:28,773][105692] Updated weights for policy 0, policy_version 328963 (0.0008) [2023-12-26 17:45:28,776][105620] Updated weights for policy 1, policy_version 329246 (0.0006) [2023-12-26 17:45:28,827][105620] Updated weights for policy 1, policy_version 329256 (0.0008) [2023-12-26 17:45:29,499][105620] Updated weights for policy 1, policy_version 329266 (0.0009) [2023-12-26 17:45:29,552][105620] Updated weights for policy 1, policy_version 329276 (0.0008) [2023-12-26 17:45:29,565][105692] Updated weights for policy 0, policy_version 328973 (0.0009) [2023-12-26 17:45:29,601][105620] Updated weights for policy 1, policy_version 329286 (0.0007) [2023-12-26 17:45:29,610][105692] Updated weights for policy 0, policy_version 328983 (0.0006) [2023-12-26 17:45:29,660][105692] Updated weights for policy 0, policy_version 328993 (0.0008) [2023-12-26 17:45:30,312][105620] Updated weights for policy 1, policy_version 329296 (0.0006) [2023-12-26 17:45:30,376][105620] Updated weights for policy 1, policy_version 329306 (0.0006) [2023-12-26 17:45:30,434][105620] Updated weights for policy 1, policy_version 329316 (0.0005) [2023-12-26 17:45:30,540][105692] Updated weights for policy 0, policy_version 329003 (0.0009) [2023-12-26 17:45:30,598][105692] Updated weights for policy 0, policy_version 329013 (0.0005) [2023-12-26 17:45:30,651][105692] Updated weights for policy 0, policy_version 329023 (0.0005) [2023-12-26 17:45:30,933][105620] Updated weights for policy 1, policy_version 329326 (0.0005) [2023-12-26 17:45:30,986][105620] Updated weights for policy 1, policy_version 329336 (0.0005) [2023-12-26 17:45:31,046][105620] Updated weights for policy 1, policy_version 329346 (0.0006) [2023-12-26 17:45:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19633.0). Total num frames: 168558592. Throughput: 0: 9723.1, 1: 9582.4. Samples: 168531732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:45:31,062][104569] Avg episode reward: [(0, '9359.754'), (1, '9355.954')] [2023-12-26 17:45:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000329032_84246528.pth... [2023-12-26 17:45:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000327912_83959808.pth [2023-12-26 17:45:31,073][105585] Saving new best policy, reward=9359.754! [2023-12-26 17:45:31,081][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000329352_84320256.pth... [2023-12-26 17:45:31,084][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000328200_84025344.pth [2023-12-26 17:45:31,474][105692] Updated weights for policy 0, policy_version 329033 (0.0006) [2023-12-26 17:45:31,534][105692] Updated weights for policy 0, policy_version 329043 (0.0009) [2023-12-26 17:45:31,595][105692] Updated weights for policy 0, policy_version 329055 (0.0009) [2023-12-26 17:45:31,664][105620] Updated weights for policy 1, policy_version 329356 (0.0007) [2023-12-26 17:45:31,735][105620] Updated weights for policy 1, policy_version 329366 (0.0008) [2023-12-26 17:45:31,788][105620] Updated weights for policy 1, policy_version 329376 (0.0008) [2023-12-26 17:45:32,328][105692] Updated weights for policy 0, policy_version 329065 (0.0009) [2023-12-26 17:45:32,389][105692] Updated weights for policy 0, policy_version 329075 (0.0008) [2023-12-26 17:45:32,445][105692] Updated weights for policy 0, policy_version 329085 (0.0008) [2023-12-26 17:45:32,508][105692] Updated weights for policy 0, policy_version 329095 (0.0006) [2023-12-26 17:45:32,522][105620] Updated weights for policy 1, policy_version 329386 (0.0009) [2023-12-26 17:45:32,581][105620] Updated weights for policy 1, policy_version 329396 (0.0007) [2023-12-26 17:45:32,648][105620] Updated weights for policy 1, policy_version 329406 (0.0007) [2023-12-26 17:45:32,709][105620] Updated weights for policy 1, policy_version 329416 (0.0010) [2023-12-26 17:45:33,102][105692] Updated weights for policy 0, policy_version 329105 (0.0005) [2023-12-26 17:45:33,158][105692] Updated weights for policy 0, policy_version 329115 (0.0005) [2023-12-26 17:45:33,213][105692] Updated weights for policy 0, policy_version 329125 (0.0005) [2023-12-26 17:45:33,481][105620] Updated weights for policy 1, policy_version 329426 (0.0010) [2023-12-26 17:45:33,550][105620] Updated weights for policy 1, policy_version 329436 (0.0009) [2023-12-26 17:45:33,610][105620] Updated weights for policy 1, policy_version 329446 (0.0008) [2023-12-26 17:45:33,811][105692] Updated weights for policy 0, policy_version 329135 (0.0008) [2023-12-26 17:45:33,864][105692] Updated weights for policy 0, policy_version 329145 (0.0009) [2023-12-26 17:45:33,908][105692] Updated weights for policy 0, policy_version 329155 (0.0007) [2023-12-26 17:45:34,372][105620] Updated weights for policy 1, policy_version 329456 (0.0009) [2023-12-26 17:45:34,436][105620] Updated weights for policy 1, policy_version 329466 (0.0009) [2023-12-26 17:45:34,498][105620] Updated weights for policy 1, policy_version 329476 (0.0009) [2023-12-26 17:45:34,606][105692] Updated weights for policy 0, policy_version 329165 (0.0007) [2023-12-26 17:45:34,661][105692] Updated weights for policy 0, policy_version 329175 (0.0009) [2023-12-26 17:45:34,718][105692] Updated weights for policy 0, policy_version 329185 (0.0009) [2023-12-26 17:45:35,240][105620] Updated weights for policy 1, policy_version 329486 (0.0009) [2023-12-26 17:45:35,299][105620] Updated weights for policy 1, policy_version 329496 (0.0009) [2023-12-26 17:45:35,354][105620] Updated weights for policy 1, policy_version 329506 (0.0009) [2023-12-26 17:45:35,498][105692] Updated weights for policy 0, policy_version 329195 (0.0009) [2023-12-26 17:45:35,559][105692] Updated weights for policy 0, policy_version 329205 (0.0009) [2023-12-26 17:45:35,621][105692] Updated weights for policy 0, policy_version 329215 (0.0009) [2023-12-26 17:45:36,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 168656896. Throughput: 0: 9700.0, 1: 9720.6. Samples: 168649960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:45:36,063][104569] Avg episode reward: [(0, '9269.502'), (1, '9356.008')] [2023-12-26 17:45:36,106][105620] Updated weights for policy 1, policy_version 329516 (0.0009) [2023-12-26 17:45:36,175][105620] Updated weights for policy 1, policy_version 329526 (0.0009) [2023-12-26 17:45:36,235][105620] Updated weights for policy 1, policy_version 329536 (0.0010) [2023-12-26 17:45:36,318][105692] Updated weights for policy 0, policy_version 329225 (0.0009) [2023-12-26 17:45:36,383][105692] Updated weights for policy 0, policy_version 329235 (0.0007) [2023-12-26 17:45:36,442][105692] Updated weights for policy 0, policy_version 329245 (0.0008) [2023-12-26 17:45:36,501][105692] Updated weights for policy 0, policy_version 329255 (0.0009) [2023-12-26 17:45:37,018][105620] Updated weights for policy 1, policy_version 329546 (0.0010) [2023-12-26 17:45:37,077][105620] Updated weights for policy 1, policy_version 329556 (0.0009) [2023-12-26 17:45:37,132][105620] Updated weights for policy 1, policy_version 329566 (0.0009) [2023-12-26 17:45:37,188][105620] Updated weights for policy 1, policy_version 329576 (0.0009) [2023-12-26 17:45:37,245][105692] Updated weights for policy 0, policy_version 329265 (0.0009) [2023-12-26 17:45:37,308][105692] Updated weights for policy 0, policy_version 329275 (0.0009) [2023-12-26 17:45:37,364][105692] Updated weights for policy 0, policy_version 329285 (0.0010) [2023-12-26 17:45:37,841][105620] Updated weights for policy 1, policy_version 329586 (0.0009) [2023-12-26 17:45:37,892][105620] Updated weights for policy 1, policy_version 329596 (0.0009) [2023-12-26 17:45:37,941][105620] Updated weights for policy 1, policy_version 329606 (0.0009) [2023-12-26 17:45:38,176][105692] Updated weights for policy 0, policy_version 329295 (0.0009) [2023-12-26 17:45:38,238][105692] Updated weights for policy 0, policy_version 329305 (0.0008) [2023-12-26 17:45:38,307][105692] Updated weights for policy 0, policy_version 329315 (0.0007) [2023-12-26 17:45:38,780][105620] Updated weights for policy 1, policy_version 329616 (0.0010) [2023-12-26 17:45:38,849][105620] Updated weights for policy 1, policy_version 329626 (0.0010) [2023-12-26 17:45:38,905][105620] Updated weights for policy 1, policy_version 329636 (0.0011) [2023-12-26 17:45:39,053][105692] Updated weights for policy 0, policy_version 329325 (0.0010) [2023-12-26 17:45:39,109][105692] Updated weights for policy 0, policy_version 329335 (0.0010) [2023-12-26 17:45:39,166][105692] Updated weights for policy 0, policy_version 329345 (0.0011) [2023-12-26 17:45:39,642][105620] Updated weights for policy 1, policy_version 329646 (0.0009) [2023-12-26 17:45:39,710][105620] Updated weights for policy 1, policy_version 329656 (0.0011) [2023-12-26 17:45:39,780][105620] Updated weights for policy 1, policy_version 329666 (0.0010) [2023-12-26 17:45:39,933][105692] Updated weights for policy 0, policy_version 329355 (0.0010) [2023-12-26 17:45:39,989][105692] Updated weights for policy 0, policy_version 329365 (0.0008) [2023-12-26 17:45:40,052][105692] Updated weights for policy 0, policy_version 329375 (0.0008) [2023-12-26 17:45:40,544][105620] Updated weights for policy 1, policy_version 329676 (0.0008) [2023-12-26 17:45:40,615][105620] Updated weights for policy 1, policy_version 329686 (0.0008) [2023-12-26 17:45:40,683][105620] Updated weights for policy 1, policy_version 329696 (0.0008) [2023-12-26 17:45:40,862][105692] Updated weights for policy 0, policy_version 329385 (0.0008) [2023-12-26 17:45:40,924][105692] Updated weights for policy 0, policy_version 329395 (0.0008) [2023-12-26 17:45:40,985][105692] Updated weights for policy 0, policy_version 329405 (0.0008) [2023-12-26 17:45:41,048][105692] Updated weights for policy 0, policy_version 329415 (0.0008) [2023-12-26 17:45:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 168755200. Throughput: 0: 9600.0, 1: 9682.8. Samples: 168759796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:45:41,062][104569] Avg episode reward: [(0, '9269.502'), (1, '9356.030')] [2023-12-26 17:45:41,472][105620] Updated weights for policy 1, policy_version 329706 (0.0009) [2023-12-26 17:45:41,536][105620] Updated weights for policy 1, policy_version 329716 (0.0008) [2023-12-26 17:45:41,606][105620] Updated weights for policy 1, policy_version 329726 (0.0008) [2023-12-26 17:45:41,670][105620] Updated weights for policy 1, policy_version 329736 (0.0009) [2023-12-26 17:45:41,836][105692] Updated weights for policy 0, policy_version 329425 (0.0010) [2023-12-26 17:45:41,898][105692] Updated weights for policy 0, policy_version 329435 (0.0010) [2023-12-26 17:45:41,961][105692] Updated weights for policy 0, policy_version 329445 (0.0011) [2023-12-26 17:45:42,448][105620] Updated weights for policy 1, policy_version 329746 (0.0009) [2023-12-26 17:45:42,514][105620] Updated weights for policy 1, policy_version 329756 (0.0010) [2023-12-26 17:45:42,574][105620] Updated weights for policy 1, policy_version 329766 (0.0007) [2023-12-26 17:45:42,677][105692] Updated weights for policy 0, policy_version 329455 (0.0009) [2023-12-26 17:45:42,741][105692] Updated weights for policy 0, policy_version 329465 (0.0009) [2023-12-26 17:45:42,805][105692] Updated weights for policy 0, policy_version 329475 (0.0009) [2023-12-26 17:45:43,219][105620] Updated weights for policy 1, policy_version 329776 (0.0008) [2023-12-26 17:45:43,272][105620] Updated weights for policy 1, policy_version 329786 (0.0010) [2023-12-26 17:45:43,338][105620] Updated weights for policy 1, policy_version 329796 (0.0008) [2023-12-26 17:45:43,542][105692] Updated weights for policy 0, policy_version 329485 (0.0009) [2023-12-26 17:45:43,598][105692] Updated weights for policy 0, policy_version 329495 (0.0009) [2023-12-26 17:45:43,651][105692] Updated weights for policy 0, policy_version 329505 (0.0010) [2023-12-26 17:45:44,008][105620] Updated weights for policy 1, policy_version 329806 (0.0008) [2023-12-26 17:45:44,055][105620] Updated weights for policy 1, policy_version 329816 (0.0008) [2023-12-26 17:45:44,109][105620] Updated weights for policy 1, policy_version 329826 (0.0009) [2023-12-26 17:45:44,401][105692] Updated weights for policy 0, policy_version 329515 (0.0010) [2023-12-26 17:45:44,460][105692] Updated weights for policy 0, policy_version 329525 (0.0009) [2023-12-26 17:45:44,515][105692] Updated weights for policy 0, policy_version 329535 (0.0009) [2023-12-26 17:45:44,779][105620] Updated weights for policy 1, policy_version 329836 (0.0009) [2023-12-26 17:45:44,837][105620] Updated weights for policy 1, policy_version 329846 (0.0009) [2023-12-26 17:45:44,901][105620] Updated weights for policy 1, policy_version 329856 (0.0008) [2023-12-26 17:45:45,314][105692] Updated weights for policy 0, policy_version 329545 (0.0009) [2023-12-26 17:45:45,379][105692] Updated weights for policy 0, policy_version 329555 (0.0011) [2023-12-26 17:45:45,443][105692] Updated weights for policy 0, policy_version 329565 (0.0011) [2023-12-26 17:45:45,514][105692] Updated weights for policy 0, policy_version 329575 (0.0011) [2023-12-26 17:45:45,680][105620] Updated weights for policy 1, policy_version 329866 (0.0009) [2023-12-26 17:45:45,741][105620] Updated weights for policy 1, policy_version 329876 (0.0011) [2023-12-26 17:45:45,801][105620] Updated weights for policy 1, policy_version 329886 (0.0009) [2023-12-26 17:45:45,870][105620] Updated weights for policy 1, policy_version 329896 (0.0009) [2023-12-26 17:45:46,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.3, 300 sec: 19605.3). Total num frames: 168845312. Throughput: 0: 9537.2, 1: 9674.0. Samples: 168815604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:45:46,062][104569] Avg episode reward: [(0, '9359.668'), (1, '9266.709')] [2023-12-26 17:45:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000329576_84385792.pth... [2023-12-26 17:45:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000329896_84459520.pth... [2023-12-26 17:45:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000328744_84164608.pth [2023-12-26 17:45:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000328488_84107264.pth [2023-12-26 17:45:46,167][105692] Updated weights for policy 0, policy_version 329585 (0.0009) [2023-12-26 17:45:46,231][105692] Updated weights for policy 0, policy_version 329595 (0.0009) [2023-12-26 17:45:46,297][105692] Updated weights for policy 0, policy_version 329605 (0.0009) [2023-12-26 17:45:46,567][105620] Updated weights for policy 1, policy_version 329906 (0.0009) [2023-12-26 17:45:46,627][105620] Updated weights for policy 1, policy_version 329916 (0.0009) [2023-12-26 17:45:46,676][105620] Updated weights for policy 1, policy_version 329926 (0.0009) [2023-12-26 17:45:46,959][105692] Updated weights for policy 0, policy_version 329615 (0.0008) [2023-12-26 17:45:47,030][105692] Updated weights for policy 0, policy_version 329625 (0.0010) [2023-12-26 17:45:47,088][105692] Updated weights for policy 0, policy_version 329635 (0.0007) [2023-12-26 17:45:47,419][105620] Updated weights for policy 1, policy_version 329936 (0.0006) [2023-12-26 17:45:47,478][105620] Updated weights for policy 1, policy_version 329946 (0.0005) [2023-12-26 17:45:47,527][105620] Updated weights for policy 1, policy_version 329956 (0.0005) [2023-12-26 17:45:47,675][105692] Updated weights for policy 0, policy_version 329645 (0.0007) [2023-12-26 17:45:47,732][105692] Updated weights for policy 0, policy_version 329655 (0.0006) [2023-12-26 17:45:47,805][105692] Updated weights for policy 0, policy_version 329665 (0.0005) [2023-12-26 17:45:48,140][105620] Updated weights for policy 1, policy_version 329966 (0.0007) [2023-12-26 17:45:48,197][105620] Updated weights for policy 1, policy_version 329976 (0.0009) [2023-12-26 17:45:48,255][105620] Updated weights for policy 1, policy_version 329987 (0.0010) [2023-12-26 17:45:48,337][105692] Updated weights for policy 0, policy_version 329675 (0.0007) [2023-12-26 17:45:48,401][105692] Updated weights for policy 0, policy_version 329685 (0.0008) [2023-12-26 17:45:48,458][105692] Updated weights for policy 0, policy_version 329695 (0.0009) [2023-12-26 17:45:48,997][105620] Updated weights for policy 1, policy_version 329997 (0.0009) [2023-12-26 17:45:49,055][105620] Updated weights for policy 1, policy_version 330007 (0.0009) [2023-12-26 17:45:49,103][105692] Updated weights for policy 0, policy_version 329705 (0.0009) [2023-12-26 17:45:49,109][105620] Updated weights for policy 1, policy_version 330017 (0.0007) [2023-12-26 17:45:49,161][105692] Updated weights for policy 0, policy_version 329715 (0.0009) [2023-12-26 17:45:49,230][105692] Updated weights for policy 0, policy_version 329725 (0.0009) [2023-12-26 17:45:49,294][105692] Updated weights for policy 0, policy_version 329735 (0.0009) [2023-12-26 17:45:49,875][105620] Updated weights for policy 1, policy_version 330027 (0.0007) [2023-12-26 17:45:49,937][105620] Updated weights for policy 1, policy_version 330037 (0.0007) [2023-12-26 17:45:50,007][105620] Updated weights for policy 1, policy_version 330047 (0.0009) [2023-12-26 17:45:50,023][105692] Updated weights for policy 0, policy_version 329745 (0.0007) [2023-12-26 17:45:50,082][105692] Updated weights for policy 0, policy_version 329755 (0.0009) [2023-12-26 17:45:50,130][105692] Updated weights for policy 0, policy_version 329765 (0.0008) [2023-12-26 17:45:50,725][105620] Updated weights for policy 1, policy_version 330057 (0.0011) [2023-12-26 17:45:50,732][105692] Updated weights for policy 0, policy_version 329775 (0.0006) [2023-12-26 17:45:50,789][105620] Updated weights for policy 1, policy_version 330067 (0.0011) [2023-12-26 17:45:50,797][105692] Updated weights for policy 0, policy_version 329785 (0.0007) [2023-12-26 17:45:50,847][105620] Updated weights for policy 1, policy_version 330077 (0.0011) [2023-12-26 17:45:50,863][105692] Updated weights for policy 0, policy_version 329795 (0.0010) [2023-12-26 17:45:50,903][105620] Updated weights for policy 1, policy_version 330087 (0.0011) [2023-12-26 17:45:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 168951808. Throughput: 0: 9652.4, 1: 9596.6. Samples: 168935252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:45:51,062][104569] Avg episode reward: [(0, '9359.595'), (1, '9266.720')] [2023-12-26 17:45:51,559][105692] Updated weights for policy 0, policy_version 329805 (0.0008) [2023-12-26 17:45:51,628][105692] Updated weights for policy 0, policy_version 329815 (0.0008) [2023-12-26 17:45:51,685][105692] Updated weights for policy 0, policy_version 329825 (0.0006) [2023-12-26 17:45:51,687][105620] Updated weights for policy 1, policy_version 330097 (0.0010) [2023-12-26 17:45:51,753][105620] Updated weights for policy 1, policy_version 330107 (0.0010) [2023-12-26 17:45:51,800][105620] Updated weights for policy 1, policy_version 330117 (0.0009) [2023-12-26 17:45:52,267][105692] Updated weights for policy 0, policy_version 329835 (0.0007) [2023-12-26 17:45:52,333][105692] Updated weights for policy 0, policy_version 329845 (0.0008) [2023-12-26 17:45:52,392][105692] Updated weights for policy 0, policy_version 329855 (0.0009) [2023-12-26 17:45:52,552][105620] Updated weights for policy 1, policy_version 330127 (0.0008) [2023-12-26 17:45:52,556][105586] KL-divergence is very high: 256.7950 [2023-12-26 17:45:52,603][105586] KL-divergence is very high: 288.1556 [2023-12-26 17:45:52,608][105620] Updated weights for policy 1, policy_version 330137 (0.0008) [2023-12-26 17:45:52,654][105586] KL-divergence is very high: 200.0872 [2023-12-26 17:45:52,671][105620] Updated weights for policy 1, policy_version 330147 (0.0008) [2023-12-26 17:45:53,169][105692] Updated weights for policy 0, policy_version 329865 (0.0008) [2023-12-26 17:45:53,216][105692] Updated weights for policy 0, policy_version 329875 (0.0009) [2023-12-26 17:45:53,263][105692] Updated weights for policy 0, policy_version 329885 (0.0008) [2023-12-26 17:45:53,318][105692] Updated weights for policy 0, policy_version 329895 (0.0009) [2023-12-26 17:45:53,366][105620] Updated weights for policy 1, policy_version 330157 (0.0009) [2023-12-26 17:45:53,412][105620] Updated weights for policy 1, policy_version 330167 (0.0006) [2023-12-26 17:45:53,460][105620] Updated weights for policy 1, policy_version 330177 (0.0005) [2023-12-26 17:45:54,073][105620] Updated weights for policy 1, policy_version 330187 (0.0006) [2023-12-26 17:45:54,138][105620] Updated weights for policy 1, policy_version 330197 (0.0009) [2023-12-26 17:45:54,177][105692] Updated weights for policy 0, policy_version 329905 (0.0006) [2023-12-26 17:45:54,203][105620] Updated weights for policy 1, policy_version 330207 (0.0008) [2023-12-26 17:45:54,235][105692] Updated weights for policy 0, policy_version 329915 (0.0007) [2023-12-26 17:45:54,296][105692] Updated weights for policy 0, policy_version 329925 (0.0010) [2023-12-26 17:45:54,836][105620] Updated weights for policy 1, policy_version 330217 (0.0009) [2023-12-26 17:45:54,887][105620] Updated weights for policy 1, policy_version 330227 (0.0005) [2023-12-26 17:45:54,955][105620] Updated weights for policy 1, policy_version 330237 (0.0007) [2023-12-26 17:45:54,965][105692] Updated weights for policy 0, policy_version 329935 (0.0007) [2023-12-26 17:45:55,014][105620] Updated weights for policy 1, policy_version 330247 (0.0011) [2023-12-26 17:45:55,022][105692] Updated weights for policy 0, policy_version 329945 (0.0005) [2023-12-26 17:45:55,073][105692] Updated weights for policy 0, policy_version 329955 (0.0006) [2023-12-26 17:45:55,705][105620] Updated weights for policy 1, policy_version 330257 (0.0011) [2023-12-26 17:45:55,727][105692] Updated weights for policy 0, policy_version 329965 (0.0010) [2023-12-26 17:45:55,750][105620] Updated weights for policy 1, policy_version 330267 (0.0010) [2023-12-26 17:45:55,781][105692] Updated weights for policy 0, policy_version 329975 (0.0010) [2023-12-26 17:45:55,794][105620] Updated weights for policy 1, policy_version 330277 (0.0010) [2023-12-26 17:45:55,839][105692] Updated weights for policy 0, policy_version 329985 (0.0010) [2023-12-26 17:45:56,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 169050112. Throughput: 0: 9595.9, 1: 9716.2. Samples: 169053164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:45:56,063][104569] Avg episode reward: [(0, '9359.599'), (1, '9175.934')] [2023-12-26 17:45:56,441][105692] Updated weights for policy 0, policy_version 329995 (0.0009) [2023-12-26 17:45:56,496][105692] Updated weights for policy 0, policy_version 330005 (0.0006) [2023-12-26 17:45:56,518][105620] Updated weights for policy 1, policy_version 330287 (0.0010) [2023-12-26 17:45:56,548][105692] Updated weights for policy 0, policy_version 330015 (0.0005) [2023-12-26 17:45:56,576][105620] Updated weights for policy 1, policy_version 330297 (0.0010) [2023-12-26 17:45:56,627][105620] Updated weights for policy 1, policy_version 330307 (0.0010) [2023-12-26 17:45:57,118][105692] Updated weights for policy 0, policy_version 330025 (0.0006) [2023-12-26 17:45:57,169][105692] Updated weights for policy 0, policy_version 330035 (0.0005) [2023-12-26 17:45:57,219][105692] Updated weights for policy 0, policy_version 330045 (0.0005) [2023-12-26 17:45:57,272][105692] Updated weights for policy 0, policy_version 330055 (0.0005) [2023-12-26 17:45:57,377][105620] Updated weights for policy 1, policy_version 330317 (0.0010) [2023-12-26 17:45:57,445][105620] Updated weights for policy 1, policy_version 330327 (0.0010) [2023-12-26 17:45:57,511][105620] Updated weights for policy 1, policy_version 330337 (0.0010) [2023-12-26 17:45:57,825][105692] Updated weights for policy 0, policy_version 330065 (0.0005) [2023-12-26 17:45:57,868][105692] Updated weights for policy 0, policy_version 330075 (0.0005) [2023-12-26 17:45:57,920][105692] Updated weights for policy 0, policy_version 330085 (0.0005) [2023-12-26 17:45:58,219][105620] Updated weights for policy 1, policy_version 330347 (0.0010) [2023-12-26 17:45:58,277][105620] Updated weights for policy 1, policy_version 330357 (0.0010) [2023-12-26 17:45:58,345][105620] Updated weights for policy 1, policy_version 330367 (0.0011) [2023-12-26 17:45:58,647][105692] Updated weights for policy 0, policy_version 330095 (0.0007) [2023-12-26 17:45:58,716][105692] Updated weights for policy 0, policy_version 330105 (0.0008) [2023-12-26 17:45:58,788][105692] Updated weights for policy 0, policy_version 330115 (0.0008) [2023-12-26 17:45:59,139][105620] Updated weights for policy 1, policy_version 330377 (0.0010) [2023-12-26 17:45:59,196][105620] Updated weights for policy 1, policy_version 330387 (0.0010) [2023-12-26 17:45:59,256][105620] Updated weights for policy 1, policy_version 330397 (0.0012) [2023-12-26 17:45:59,310][105620] Updated weights for policy 1, policy_version 330407 (0.0007) [2023-12-26 17:45:59,589][105692] Updated weights for policy 0, policy_version 330125 (0.0008) [2023-12-26 17:45:59,644][105692] Updated weights for policy 0, policy_version 330135 (0.0009) [2023-12-26 17:45:59,697][105692] Updated weights for policy 0, policy_version 330145 (0.0009) [2023-12-26 17:46:00,032][105620] Updated weights for policy 1, policy_version 330417 (0.0009) [2023-12-26 17:46:00,094][105620] Updated weights for policy 1, policy_version 330427 (0.0008) [2023-12-26 17:46:00,156][105620] Updated weights for policy 1, policy_version 330437 (0.0006) [2023-12-26 17:46:00,510][105692] Updated weights for policy 0, policy_version 330155 (0.0008) [2023-12-26 17:46:00,559][105692] Updated weights for policy 0, policy_version 330165 (0.0008) [2023-12-26 17:46:00,613][105692] Updated weights for policy 0, policy_version 330175 (0.0009) [2023-12-26 17:46:00,817][105620] Updated weights for policy 1, policy_version 330447 (0.0007) [2023-12-26 17:46:00,867][105620] Updated weights for policy 1, policy_version 330457 (0.0009) [2023-12-26 17:46:00,916][105620] Updated weights for policy 1, policy_version 330467 (0.0009) [2023-12-26 17:46:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 169148416. Throughput: 0: 9696.3, 1: 9701.5. Samples: 169116156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:46:01,063][104569] Avg episode reward: [(0, '9359.573'), (1, '9175.110')] [2023-12-26 17:46:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000330184_84541440.pth... [2023-12-26 17:46:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000330472_84606976.pth... [2023-12-26 17:46:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000329032_84246528.pth [2023-12-26 17:46:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000329352_84320256.pth [2023-12-26 17:46:01,358][105692] Updated weights for policy 0, policy_version 330185 (0.0009) [2023-12-26 17:46:01,420][105692] Updated weights for policy 0, policy_version 330195 (0.0010) [2023-12-26 17:46:01,486][105692] Updated weights for policy 0, policy_version 330205 (0.0010) [2023-12-26 17:46:01,549][105692] Updated weights for policy 0, policy_version 330215 (0.0009) [2023-12-26 17:46:01,663][105620] Updated weights for policy 1, policy_version 330477 (0.0008) [2023-12-26 17:46:01,723][105620] Updated weights for policy 1, policy_version 330487 (0.0007) [2023-12-26 17:46:01,788][105620] Updated weights for policy 1, policy_version 330497 (0.0008) [2023-12-26 17:46:02,291][105692] Updated weights for policy 0, policy_version 330225 (0.0009) [2023-12-26 17:46:02,345][105692] Updated weights for policy 0, policy_version 330235 (0.0008) [2023-12-26 17:46:02,410][105692] Updated weights for policy 0, policy_version 330245 (0.0009) [2023-12-26 17:46:02,543][105620] Updated weights for policy 1, policy_version 330507 (0.0010) [2023-12-26 17:46:02,597][105620] Updated weights for policy 1, policy_version 330517 (0.0009) [2023-12-26 17:46:02,652][105620] Updated weights for policy 1, policy_version 330527 (0.0008) [2023-12-26 17:46:03,070][105692] Updated weights for policy 0, policy_version 330255 (0.0005) [2023-12-26 17:46:03,123][105692] Updated weights for policy 0, policy_version 330265 (0.0005) [2023-12-26 17:46:03,175][105692] Updated weights for policy 0, policy_version 330275 (0.0006) [2023-12-26 17:46:03,441][105620] Updated weights for policy 1, policy_version 330537 (0.0008) [2023-12-26 17:46:03,496][105620] Updated weights for policy 1, policy_version 330547 (0.0009) [2023-12-26 17:46:03,547][105620] Updated weights for policy 1, policy_version 330557 (0.0010) [2023-12-26 17:46:03,591][105620] Updated weights for policy 1, policy_version 330567 (0.0010) [2023-12-26 17:46:03,778][105692] Updated weights for policy 0, policy_version 330285 (0.0007) [2023-12-26 17:46:03,833][105692] Updated weights for policy 0, policy_version 330295 (0.0008) [2023-12-26 17:46:03,897][105692] Updated weights for policy 0, policy_version 330305 (0.0008) [2023-12-26 17:46:04,367][105620] Updated weights for policy 1, policy_version 330577 (0.0011) [2023-12-26 17:46:04,424][105620] Updated weights for policy 1, policy_version 330587 (0.0011) [2023-12-26 17:46:04,491][105620] Updated weights for policy 1, policy_version 330597 (0.0011) [2023-12-26 17:46:04,679][105692] Updated weights for policy 0, policy_version 330315 (0.0008) [2023-12-26 17:46:04,741][105692] Updated weights for policy 0, policy_version 330325 (0.0007) [2023-12-26 17:46:04,797][105692] Updated weights for policy 0, policy_version 330335 (0.0008) [2023-12-26 17:46:05,246][105620] Updated weights for policy 1, policy_version 330607 (0.0010) [2023-12-26 17:46:05,304][105620] Updated weights for policy 1, policy_version 330617 (0.0010) [2023-12-26 17:46:05,362][105620] Updated weights for policy 1, policy_version 330627 (0.0010) [2023-12-26 17:46:05,547][105692] Updated weights for policy 0, policy_version 330345 (0.0008) [2023-12-26 17:46:05,612][105692] Updated weights for policy 0, policy_version 330355 (0.0008) [2023-12-26 17:46:05,669][105692] Updated weights for policy 0, policy_version 330365 (0.0008) [2023-12-26 17:46:05,719][105692] Updated weights for policy 0, policy_version 330375 (0.0010) [2023-12-26 17:46:05,994][105620] Updated weights for policy 1, policy_version 330637 (0.0008) [2023-12-26 17:46:06,054][105620] Updated weights for policy 1, policy_version 330647 (0.0006) [2023-12-26 17:46:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19633.0). Total num frames: 169238528. Throughput: 0: 9677.3, 1: 9625.5. Samples: 169229964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:46:06,062][104569] Avg episode reward: [(0, '9359.523'), (1, '9175.082')] [2023-12-26 17:46:06,124][105620] Updated weights for policy 1, policy_version 330657 (0.0008) [2023-12-26 17:46:06,447][105692] Updated weights for policy 0, policy_version 330385 (0.0009) [2023-12-26 17:46:06,509][105692] Updated weights for policy 0, policy_version 330395 (0.0009) [2023-12-26 17:46:06,574][105692] Updated weights for policy 0, policy_version 330405 (0.0008) [2023-12-26 17:46:06,827][105620] Updated weights for policy 1, policy_version 330667 (0.0008) [2023-12-26 17:46:06,886][105620] Updated weights for policy 1, policy_version 330677 (0.0009) [2023-12-26 17:46:06,948][105620] Updated weights for policy 1, policy_version 330687 (0.0007) [2023-12-26 17:46:07,316][105692] Updated weights for policy 0, policy_version 330415 (0.0008) [2023-12-26 17:46:07,364][105692] Updated weights for policy 0, policy_version 330425 (0.0009) [2023-12-26 17:46:07,422][105692] Updated weights for policy 0, policy_version 330435 (0.0009) [2023-12-26 17:46:07,683][105620] Updated weights for policy 1, policy_version 330697 (0.0010) [2023-12-26 17:46:07,741][105620] Updated weights for policy 1, policy_version 330707 (0.0010) [2023-12-26 17:46:07,793][105620] Updated weights for policy 1, policy_version 330717 (0.0010) [2023-12-26 17:46:07,850][105620] Updated weights for policy 1, policy_version 330727 (0.0010) [2023-12-26 17:46:08,217][105692] Updated weights for policy 0, policy_version 330445 (0.0008) [2023-12-26 17:46:08,261][105692] Updated weights for policy 0, policy_version 330455 (0.0008) [2023-12-26 17:46:08,316][105692] Updated weights for policy 0, policy_version 330465 (0.0008) [2023-12-26 17:46:08,624][105620] Updated weights for policy 1, policy_version 330737 (0.0010) [2023-12-26 17:46:08,685][105620] Updated weights for policy 1, policy_version 330747 (0.0010) [2023-12-26 17:46:08,748][105620] Updated weights for policy 1, policy_version 330757 (0.0010) [2023-12-26 17:46:09,026][105692] Updated weights for policy 0, policy_version 330475 (0.0007) [2023-12-26 17:46:09,113][105692] Updated weights for policy 0, policy_version 330485 (0.0010) [2023-12-26 17:46:09,178][105692] Updated weights for policy 0, policy_version 330495 (0.0010) [2023-12-26 17:46:09,464][105620] Updated weights for policy 1, policy_version 330767 (0.0010) [2023-12-26 17:46:09,525][105620] Updated weights for policy 1, policy_version 330777 (0.0011) [2023-12-26 17:46:09,584][105620] Updated weights for policy 1, policy_version 330787 (0.0010) [2023-12-26 17:46:09,891][105692] Updated weights for policy 0, policy_version 330505 (0.0010) [2023-12-26 17:46:09,956][105692] Updated weights for policy 0, policy_version 330515 (0.0010) [2023-12-26 17:46:10,008][105692] Updated weights for policy 0, policy_version 330525 (0.0010) [2023-12-26 17:46:10,012][105585] KL-divergence is very high: 139.3968 [2023-12-26 17:46:10,067][105692] Updated weights for policy 0, policy_version 330535 (0.0010) [2023-12-26 17:46:10,268][105620] Updated weights for policy 1, policy_version 330797 (0.0011) [2023-12-26 17:46:10,317][105620] Updated weights for policy 1, policy_version 330807 (0.0010) [2023-12-26 17:46:10,370][105620] Updated weights for policy 1, policy_version 330817 (0.0011) [2023-12-26 17:46:10,811][105692] Updated weights for policy 0, policy_version 330545 (0.0011) [2023-12-26 17:46:10,859][105692] Updated weights for policy 0, policy_version 330555 (0.0007) [2023-12-26 17:46:10,914][105692] Updated weights for policy 0, policy_version 330565 (0.0005) [2023-12-26 17:46:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 169336832. Throughput: 0: 9672.1, 1: 9694.6. Samples: 169344580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:46:11,062][104569] Avg episode reward: [(0, '9359.489'), (1, '9265.069')] [2023-12-26 17:46:11,150][105620] Updated weights for policy 1, policy_version 330827 (0.0011) [2023-12-26 17:46:11,205][105620] Updated weights for policy 1, policy_version 330837 (0.0010) [2023-12-26 17:46:11,269][105620] Updated weights for policy 1, policy_version 330847 (0.0010) [2023-12-26 17:46:11,600][105692] Updated weights for policy 0, policy_version 330575 (0.0007) [2023-12-26 17:46:11,666][105692] Updated weights for policy 0, policy_version 330585 (0.0007) [2023-12-26 17:46:11,730][105692] Updated weights for policy 0, policy_version 330595 (0.0009) [2023-12-26 17:46:12,032][105620] Updated weights for policy 1, policy_version 330857 (0.0011) [2023-12-26 17:46:12,084][105620] Updated weights for policy 1, policy_version 330867 (0.0010) [2023-12-26 17:46:12,139][105620] Updated weights for policy 1, policy_version 330877 (0.0010) [2023-12-26 17:46:12,196][105620] Updated weights for policy 1, policy_version 330887 (0.0010) [2023-12-26 17:46:12,453][105692] Updated weights for policy 0, policy_version 330605 (0.0008) [2023-12-26 17:46:12,511][105692] Updated weights for policy 0, policy_version 330615 (0.0008) [2023-12-26 17:46:12,568][105692] Updated weights for policy 0, policy_version 330625 (0.0008) [2023-12-26 17:46:12,981][105620] Updated weights for policy 1, policy_version 330897 (0.0008) [2023-12-26 17:46:13,033][105620] Updated weights for policy 1, policy_version 330907 (0.0010) [2023-12-26 17:46:13,084][105620] Updated weights for policy 1, policy_version 330917 (0.0010) [2023-12-26 17:46:13,376][105692] Updated weights for policy 0, policy_version 330635 (0.0008) [2023-12-26 17:46:13,430][105692] Updated weights for policy 0, policy_version 330645 (0.0010) [2023-12-26 17:46:13,484][105692] Updated weights for policy 0, policy_version 330655 (0.0010) [2023-12-26 17:46:13,700][105620] Updated weights for policy 1, policy_version 330927 (0.0007) [2023-12-26 17:46:13,755][105620] Updated weights for policy 1, policy_version 330937 (0.0005) [2023-12-26 17:46:13,808][105620] Updated weights for policy 1, policy_version 330947 (0.0005) [2023-12-26 17:46:14,206][105692] Updated weights for policy 0, policy_version 330665 (0.0008) [2023-12-26 17:46:14,262][105692] Updated weights for policy 0, policy_version 330675 (0.0005) [2023-12-26 17:46:14,323][105692] Updated weights for policy 0, policy_version 330685 (0.0005) [2023-12-26 17:46:14,390][105692] Updated weights for policy 0, policy_version 330695 (0.0005) [2023-12-26 17:46:14,445][105620] Updated weights for policy 1, policy_version 330957 (0.0005) [2023-12-26 17:46:14,501][105620] Updated weights for policy 1, policy_version 330967 (0.0006) [2023-12-26 17:46:14,577][105620] Updated weights for policy 1, policy_version 330977 (0.0006) [2023-12-26 17:46:14,989][105692] Updated weights for policy 0, policy_version 330705 (0.0009) [2023-12-26 17:46:15,051][105692] Updated weights for policy 0, policy_version 330716 (0.0010) [2023-12-26 17:46:15,117][105692] Updated weights for policy 0, policy_version 330726 (0.0006) [2023-12-26 17:46:15,240][105620] Updated weights for policy 1, policy_version 330987 (0.0007) [2023-12-26 17:46:15,303][105620] Updated weights for policy 1, policy_version 330997 (0.0007) [2023-12-26 17:46:15,361][105620] Updated weights for policy 1, policy_version 331007 (0.0006) [2023-12-26 17:46:15,704][105692] Updated weights for policy 0, policy_version 330736 (0.0006) [2023-12-26 17:46:15,758][105692] Updated weights for policy 0, policy_version 330746 (0.0005) [2023-12-26 17:46:15,818][105692] Updated weights for policy 0, policy_version 330756 (0.0006) [2023-12-26 17:46:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19387.6, 300 sec: 19633.0). Total num frames: 169435136. Throughput: 0: 9680.1, 1: 9654.1. Samples: 169401776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:46:16,063][104569] Avg episode reward: [(0, '9358.322'), (1, '9085.002')] [2023-12-26 17:46:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000331016_84746240.pth... [2023-12-26 17:46:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000330760_84688896.pth... [2023-12-26 17:46:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000329576_84385792.pth [2023-12-26 17:46:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000329896_84459520.pth [2023-12-26 17:46:16,111][105620] Updated weights for policy 1, policy_version 331017 (0.0006) [2023-12-26 17:46:16,169][105620] Updated weights for policy 1, policy_version 331027 (0.0010) [2023-12-26 17:46:16,231][105620] Updated weights for policy 1, policy_version 331037 (0.0009) [2023-12-26 17:46:16,290][105620] Updated weights for policy 1, policy_version 331047 (0.0009) [2023-12-26 17:46:16,415][105692] Updated weights for policy 0, policy_version 330766 (0.0008) [2023-12-26 17:46:16,481][105692] Updated weights for policy 0, policy_version 330776 (0.0010) [2023-12-26 17:46:16,528][105692] Updated weights for policy 0, policy_version 330786 (0.0009) [2023-12-26 17:46:17,058][105620] Updated weights for policy 1, policy_version 331057 (0.0009) [2023-12-26 17:46:17,109][105620] Updated weights for policy 1, policy_version 331067 (0.0009) [2023-12-26 17:46:17,161][105620] Updated weights for policy 1, policy_version 331077 (0.0008) [2023-12-26 17:46:17,253][105692] Updated weights for policy 0, policy_version 330796 (0.0009) [2023-12-26 17:46:17,311][105692] Updated weights for policy 0, policy_version 330806 (0.0009) [2023-12-26 17:46:17,376][105692] Updated weights for policy 0, policy_version 330816 (0.0009) [2023-12-26 17:46:17,972][105620] Updated weights for policy 1, policy_version 331087 (0.0010) [2023-12-26 17:46:17,993][105692] Updated weights for policy 0, policy_version 330826 (0.0009) [2023-12-26 17:46:18,025][105620] Updated weights for policy 1, policy_version 331097 (0.0010) [2023-12-26 17:46:18,043][105692] Updated weights for policy 0, policy_version 330836 (0.0007) [2023-12-26 17:46:18,076][105620] Updated weights for policy 1, policy_version 331107 (0.0010) [2023-12-26 17:46:18,089][105692] Updated weights for policy 0, policy_version 330846 (0.0009) [2023-12-26 17:46:18,145][105692] Updated weights for policy 0, policy_version 330856 (0.0009) [2023-12-26 17:46:18,828][105620] Updated weights for policy 1, policy_version 331117 (0.0010) [2023-12-26 17:46:18,894][105620] Updated weights for policy 1, policy_version 331127 (0.0011) [2023-12-26 17:46:18,913][105692] Updated weights for policy 0, policy_version 330866 (0.0007) [2023-12-26 17:46:18,927][105586] KL-divergence is very high: 120.9365 [2023-12-26 17:46:18,933][105586] KL-divergence is very high: 132.5387 [2023-12-26 17:46:18,938][105586] KL-divergence is very high: 157.6968 [2023-12-26 17:46:18,943][105586] KL-divergence is very high: 156.3155 [2023-12-26 17:46:18,943][105620] Updated weights for policy 1, policy_version 331137 (0.0010) [2023-12-26 17:46:18,949][105586] KL-divergence is very high: 209.0347 [2023-12-26 17:46:18,955][105586] KL-divergence is very high: 198.7279 [2023-12-26 17:46:18,966][105692] Updated weights for policy 0, policy_version 330876 (0.0005) [2023-12-26 17:46:18,966][105586] KL-divergence is very high: 136.4121 [2023-12-26 17:46:18,971][105586] KL-divergence is very high: 126.8222 [2023-12-26 17:46:19,029][105692] Updated weights for policy 0, policy_version 330886 (0.0008) [2023-12-26 17:46:19,693][105586] KL-divergence is very high: 129.2218 [2023-12-26 17:46:19,705][105620] Updated weights for policy 1, policy_version 331147 (0.0009) [2023-12-26 17:46:19,776][105620] Updated weights for policy 1, policy_version 331157 (0.0006) [2023-12-26 17:46:19,797][105586] KL-divergence is very high: 100.6374 [2023-12-26 17:46:19,805][105586] KL-divergence is very high: 180.9873 [2023-12-26 17:46:19,812][105586] KL-divergence is very high: 140.8915 [2023-12-26 17:46:19,815][105692] Updated weights for policy 0, policy_version 330896 (0.0007) [2023-12-26 17:46:19,848][105620] Updated weights for policy 1, policy_version 331167 (0.0006) [2023-12-26 17:46:19,861][105586] KL-divergence is very high: 100.9324 [2023-12-26 17:46:19,883][105692] Updated weights for policy 0, policy_version 330906 (0.0007) [2023-12-26 17:46:19,955][105692] Updated weights for policy 0, policy_version 330916 (0.0008) [2023-12-26 17:46:20,591][105620] Updated weights for policy 1, policy_version 331177 (0.0008) [2023-12-26 17:46:20,648][105620] Updated weights for policy 1, policy_version 331187 (0.0009) [2023-12-26 17:46:20,703][105692] Updated weights for policy 0, policy_version 330926 (0.0007) [2023-12-26 17:46:20,709][105620] Updated weights for policy 1, policy_version 331197 (0.0008) [2023-12-26 17:46:20,767][105692] Updated weights for policy 0, policy_version 330936 (0.0006) [2023-12-26 17:46:20,771][105620] Updated weights for policy 1, policy_version 331207 (0.0008) [2023-12-26 17:46:20,827][105692] Updated weights for policy 0, policy_version 330946 (0.0009) [2023-12-26 17:46:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 169533440. Throughput: 0: 9780.1, 1: 9564.1. Samples: 169520448. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 17:46:21,063][104569] Avg episode reward: [(0, '9356.923'), (1, '1955.810')] [2023-12-26 17:46:21,531][105620] Updated weights for policy 1, policy_version 331217 (0.0009) [2023-12-26 17:46:21,563][105692] Updated weights for policy 0, policy_version 330956 (0.0009) [2023-12-26 17:46:21,591][105620] Updated weights for policy 1, policy_version 331227 (0.0007) [2023-12-26 17:46:21,625][105692] Updated weights for policy 0, policy_version 330966 (0.0008) [2023-12-26 17:46:21,656][105620] Updated weights for policy 1, policy_version 331237 (0.0009) [2023-12-26 17:46:21,693][105692] Updated weights for policy 0, policy_version 330976 (0.0008) [2023-12-26 17:46:22,457][105620] Updated weights for policy 1, policy_version 331247 (0.0009) [2023-12-26 17:46:22,463][105692] Updated weights for policy 0, policy_version 330986 (0.0008) [2023-12-26 17:46:22,513][105692] Updated weights for policy 0, policy_version 330996 (0.0006) [2023-12-26 17:46:22,519][105620] Updated weights for policy 1, policy_version 331257 (0.0009) [2023-12-26 17:46:22,565][105692] Updated weights for policy 0, policy_version 331006 (0.0007) [2023-12-26 17:46:22,580][105620] Updated weights for policy 1, policy_version 331267 (0.0007) [2023-12-26 17:46:22,616][105692] Updated weights for policy 0, policy_version 331016 (0.0008) [2023-12-26 17:46:23,275][105620] Updated weights for policy 1, policy_version 331277 (0.0007) [2023-12-26 17:46:23,337][105620] Updated weights for policy 1, policy_version 331287 (0.0009) [2023-12-26 17:46:23,393][105620] Updated weights for policy 1, policy_version 331297 (0.0009) [2023-12-26 17:46:23,425][105692] Updated weights for policy 0, policy_version 331026 (0.0009) [2023-12-26 17:46:23,474][105692] Updated weights for policy 0, policy_version 331036 (0.0008) [2023-12-26 17:46:23,522][105692] Updated weights for policy 0, policy_version 331046 (0.0010) [2023-12-26 17:46:24,096][105620] Updated weights for policy 1, policy_version 331307 (0.0009) [2023-12-26 17:46:24,167][105620] Updated weights for policy 1, policy_version 331317 (0.0009) [2023-12-26 17:46:24,228][105620] Updated weights for policy 1, policy_version 331327 (0.0008) [2023-12-26 17:46:24,298][105692] Updated weights for policy 0, policy_version 331056 (0.0010) [2023-12-26 17:46:24,357][105692] Updated weights for policy 0, policy_version 331066 (0.0011) [2023-12-26 17:46:24,420][105692] Updated weights for policy 0, policy_version 331076 (0.0011) [2023-12-26 17:46:24,963][105620] Updated weights for policy 1, policy_version 331337 (0.0007) [2023-12-26 17:46:25,019][105620] Updated weights for policy 1, policy_version 331347 (0.0011) [2023-12-26 17:46:25,075][105692] Updated weights for policy 0, policy_version 331086 (0.0007) [2023-12-26 17:46:25,081][105620] Updated weights for policy 1, policy_version 331357 (0.0009) [2023-12-26 17:46:25,132][105620] Updated weights for policy 1, policy_version 331367 (0.0010) [2023-12-26 17:46:25,139][105692] Updated weights for policy 0, policy_version 331096 (0.0005) [2023-12-26 17:46:25,188][105692] Updated weights for policy 0, policy_version 331106 (0.0006) [2023-12-26 17:46:25,776][105692] Updated weights for policy 0, policy_version 331116 (0.0005) [2023-12-26 17:46:25,831][105692] Updated weights for policy 0, policy_version 331126 (0.0005) [2023-12-26 17:46:25,867][105620] Updated weights for policy 1, policy_version 331377 (0.0010) [2023-12-26 17:46:25,887][105692] Updated weights for policy 0, policy_version 331136 (0.0009) [2023-12-26 17:46:25,926][105620] Updated weights for policy 1, policy_version 331387 (0.0010) [2023-12-26 17:46:25,984][105620] Updated weights for policy 1, policy_version 331397 (0.0010) [2023-12-26 17:46:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 169631744. Throughput: 0: 9823.0, 1: 9593.2. Samples: 169633524. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 17:46:26,062][104569] Avg episode reward: [(0, '9355.670'), (1, '2507.489')] [2023-12-26 17:46:26,605][105692] Updated weights for policy 0, policy_version 331146 (0.0009) [2023-12-26 17:46:26,655][105692] Updated weights for policy 0, policy_version 331156 (0.0005) [2023-12-26 17:46:26,708][105692] Updated weights for policy 0, policy_version 331166 (0.0005) [2023-12-26 17:46:26,721][105620] Updated weights for policy 1, policy_version 331407 (0.0010) [2023-12-26 17:46:26,764][105692] Updated weights for policy 0, policy_version 331176 (0.0005) [2023-12-26 17:46:26,779][105620] Updated weights for policy 1, policy_version 331417 (0.0010) [2023-12-26 17:46:26,832][105620] Updated weights for policy 1, policy_version 331427 (0.0010) [2023-12-26 17:46:27,406][105692] Updated weights for policy 0, policy_version 331186 (0.0010) [2023-12-26 17:46:27,470][105692] Updated weights for policy 0, policy_version 331196 (0.0010) [2023-12-26 17:46:27,520][105692] Updated weights for policy 0, policy_version 331206 (0.0010) [2023-12-26 17:46:27,531][105620] Updated weights for policy 1, policy_version 331437 (0.0008) [2023-12-26 17:46:27,593][105620] Updated weights for policy 1, policy_version 331447 (0.0009) [2023-12-26 17:46:27,648][105620] Updated weights for policy 1, policy_version 331457 (0.0008) [2023-12-26 17:46:28,156][105692] Updated weights for policy 0, policy_version 331216 (0.0006) [2023-12-26 17:46:28,221][105692] Updated weights for policy 0, policy_version 331226 (0.0005) [2023-12-26 17:46:28,275][105692] Updated weights for policy 0, policy_version 331236 (0.0005) [2023-12-26 17:46:28,298][105620] Updated weights for policy 1, policy_version 331467 (0.0009) [2023-12-26 17:46:28,364][105620] Updated weights for policy 1, policy_version 331477 (0.0009) [2023-12-26 17:46:28,405][105586] KL-divergence is very high: 111.9116 [2023-12-26 17:46:28,426][105620] Updated weights for policy 1, policy_version 331487 (0.0007) [2023-12-26 17:46:28,426][105586] KL-divergence is very high: 109.0182 [2023-12-26 17:46:28,852][105692] Updated weights for policy 0, policy_version 331246 (0.0005) [2023-12-26 17:46:28,900][105692] Updated weights for policy 0, policy_version 331256 (0.0005) [2023-12-26 17:46:28,954][105692] Updated weights for policy 0, policy_version 331266 (0.0006) [2023-12-26 17:46:29,060][105620] Updated weights for policy 1, policy_version 331497 (0.0006) [2023-12-26 17:46:29,125][105620] Updated weights for policy 1, policy_version 331507 (0.0007) [2023-12-26 17:46:29,185][105620] Updated weights for policy 1, policy_version 331517 (0.0005) [2023-12-26 17:46:29,246][105620] Updated weights for policy 1, policy_version 331527 (0.0006) [2023-12-26 17:46:29,647][105692] Updated weights for policy 0, policy_version 331276 (0.0010) [2023-12-26 17:46:29,705][105692] Updated weights for policy 0, policy_version 331286 (0.0010) [2023-12-26 17:46:29,759][105692] Updated weights for policy 0, policy_version 331296 (0.0010) [2023-12-26 17:46:29,867][105620] Updated weights for policy 1, policy_version 331537 (0.0007) [2023-12-26 17:46:29,918][105620] Updated weights for policy 1, policy_version 331547 (0.0008) [2023-12-26 17:46:29,983][105620] Updated weights for policy 1, policy_version 331557 (0.0008) [2023-12-26 17:46:30,517][105692] Updated weights for policy 0, policy_version 331306 (0.0010) [2023-12-26 17:46:30,565][105692] Updated weights for policy 0, policy_version 331316 (0.0010) [2023-12-26 17:46:30,625][105692] Updated weights for policy 0, policy_version 331326 (0.0010) [2023-12-26 17:46:30,688][105692] Updated weights for policy 0, policy_version 331336 (0.0010) [2023-12-26 17:46:30,736][105620] Updated weights for policy 1, policy_version 331567 (0.0007) [2023-12-26 17:46:30,790][105620] Updated weights for policy 1, policy_version 331577 (0.0008) [2023-12-26 17:46:30,838][105620] Updated weights for policy 1, policy_version 331587 (0.0010) [2023-12-26 17:46:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 169730048. Throughput: 0: 9930.5, 1: 9630.5. Samples: 169695852. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 17:46:31,063][104569] Avg episode reward: [(0, '9355.988'), (1, '3503.769')] [2023-12-26 17:46:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000331336_84836352.pth... [2023-12-26 17:46:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000331592_84893696.pth... [2023-12-26 17:46:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000330184_84541440.pth [2023-12-26 17:46:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000330472_84606976.pth [2023-12-26 17:46:31,427][105692] Updated weights for policy 0, policy_version 331346 (0.0011) [2023-12-26 17:46:31,481][105585] KL-divergence is very high: 100.8711 [2023-12-26 17:46:31,495][105692] Updated weights for policy 0, policy_version 331356 (0.0010) [2023-12-26 17:46:31,544][105692] Updated weights for policy 0, policy_version 331366 (0.0010) [2023-12-26 17:46:31,561][105620] Updated weights for policy 1, policy_version 331597 (0.0007) [2023-12-26 17:46:31,622][105620] Updated weights for policy 1, policy_version 331607 (0.0006) [2023-12-26 17:46:31,676][105620] Updated weights for policy 1, policy_version 331617 (0.0006) [2023-12-26 17:46:32,313][105692] Updated weights for policy 0, policy_version 331376 (0.0010) [2023-12-26 17:46:32,373][105692] Updated weights for policy 0, policy_version 331386 (0.0011) [2023-12-26 17:46:32,377][105620] Updated weights for policy 1, policy_version 331627 (0.0008) [2023-12-26 17:46:32,431][105692] Updated weights for policy 0, policy_version 331396 (0.0010) [2023-12-26 17:46:32,433][105620] Updated weights for policy 1, policy_version 331637 (0.0006) [2023-12-26 17:46:32,488][105620] Updated weights for policy 1, policy_version 331647 (0.0008) [2023-12-26 17:46:33,016][105692] Updated weights for policy 0, policy_version 331406 (0.0010) [2023-12-26 17:46:33,070][105692] Updated weights for policy 0, policy_version 331416 (0.0009) [2023-12-26 17:46:33,122][105692] Updated weights for policy 0, policy_version 331426 (0.0006) [2023-12-26 17:46:33,198][105620] Updated weights for policy 1, policy_version 331657 (0.0009) [2023-12-26 17:46:33,266][105620] Updated weights for policy 1, policy_version 331667 (0.0005) [2023-12-26 17:46:33,325][105620] Updated weights for policy 1, policy_version 331677 (0.0005) [2023-12-26 17:46:33,395][105620] Updated weights for policy 1, policy_version 331687 (0.0005) [2023-12-26 17:46:33,931][105692] Updated weights for policy 0, policy_version 331436 (0.0008) [2023-12-26 17:46:33,982][105692] Updated weights for policy 0, policy_version 331446 (0.0005) [2023-12-26 17:46:34,027][105692] Updated weights for policy 0, policy_version 331456 (0.0005) [2023-12-26 17:46:34,030][105620] Updated weights for policy 1, policy_version 331697 (0.0006) [2023-12-26 17:46:34,079][105620] Updated weights for policy 1, policy_version 331707 (0.0005) [2023-12-26 17:46:34,144][105620] Updated weights for policy 1, policy_version 331717 (0.0006) [2023-12-26 17:46:34,681][105692] Updated weights for policy 0, policy_version 331466 (0.0005) [2023-12-26 17:46:34,750][105692] Updated weights for policy 0, policy_version 331476 (0.0010) [2023-12-26 17:46:34,813][105692] Updated weights for policy 0, policy_version 331486 (0.0009) [2023-12-26 17:46:34,871][105692] Updated weights for policy 0, policy_version 331496 (0.0007) [2023-12-26 17:46:34,881][105620] Updated weights for policy 1, policy_version 331727 (0.0008) [2023-12-26 17:46:34,939][105620] Updated weights for policy 1, policy_version 331737 (0.0005) [2023-12-26 17:46:35,008][105620] Updated weights for policy 1, policy_version 331747 (0.0006) [2023-12-26 17:46:35,593][105692] Updated weights for policy 0, policy_version 331506 (0.0010) [2023-12-26 17:46:35,646][105620] Updated weights for policy 1, policy_version 331757 (0.0005) [2023-12-26 17:46:35,656][105692] Updated weights for policy 0, policy_version 331516 (0.0008) [2023-12-26 17:46:35,704][105620] Updated weights for policy 1, policy_version 331767 (0.0005) [2023-12-26 17:46:35,714][105692] Updated weights for policy 0, policy_version 331526 (0.0010) [2023-12-26 17:46:35,752][105620] Updated weights for policy 1, policy_version 331777 (0.0005) [2023-12-26 17:46:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 169828352. Throughput: 0: 9883.7, 1: 9661.2. Samples: 169814768. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 17:46:36,062][104569] Avg episode reward: [(0, '9356.068'), (1, '7112.456')] [2023-12-26 17:46:36,475][105692] Updated weights for policy 0, policy_version 331536 (0.0010) [2023-12-26 17:46:36,481][105620] Updated weights for policy 1, policy_version 331787 (0.0008) [2023-12-26 17:46:36,533][105692] Updated weights for policy 0, policy_version 331546 (0.0006) [2023-12-26 17:46:36,543][105620] Updated weights for policy 1, policy_version 331797 (0.0008) [2023-12-26 17:46:36,590][105692] Updated weights for policy 0, policy_version 331556 (0.0007) [2023-12-26 17:46:36,603][105620] Updated weights for policy 1, policy_version 331807 (0.0010) [2023-12-26 17:46:37,294][105620] Updated weights for policy 1, policy_version 331817 (0.0009) [2023-12-26 17:46:37,356][105620] Updated weights for policy 1, policy_version 331827 (0.0008) [2023-12-26 17:46:37,382][105692] Updated weights for policy 0, policy_version 331566 (0.0009) [2023-12-26 17:46:37,412][105620] Updated weights for policy 1, policy_version 331837 (0.0009) [2023-12-26 17:46:37,438][105692] Updated weights for policy 0, policy_version 331576 (0.0007) [2023-12-26 17:46:37,468][105620] Updated weights for policy 1, policy_version 331847 (0.0007) [2023-12-26 17:46:37,496][105692] Updated weights for policy 0, policy_version 331586 (0.0008) [2023-12-26 17:46:38,178][105620] Updated weights for policy 1, policy_version 331857 (0.0009) [2023-12-26 17:46:38,224][105692] Updated weights for policy 0, policy_version 331596 (0.0009) [2023-12-26 17:46:38,240][105620] Updated weights for policy 1, policy_version 331867 (0.0010) [2023-12-26 17:46:38,273][105692] Updated weights for policy 0, policy_version 331606 (0.0008) [2023-12-26 17:46:38,294][105620] Updated weights for policy 1, policy_version 331877 (0.0010) [2023-12-26 17:46:38,341][105692] Updated weights for policy 0, policy_version 331616 (0.0006) [2023-12-26 17:46:38,985][105620] Updated weights for policy 1, policy_version 331887 (0.0007) [2023-12-26 17:46:39,044][105620] Updated weights for policy 1, policy_version 331897 (0.0005) [2023-12-26 17:46:39,099][105620] Updated weights for policy 1, policy_version 331907 (0.0005) [2023-12-26 17:46:39,164][105692] Updated weights for policy 0, policy_version 331626 (0.0008) [2023-12-26 17:46:39,221][105692] Updated weights for policy 0, policy_version 331637 (0.0009) [2023-12-26 17:46:39,288][105692] Updated weights for policy 0, policy_version 331647 (0.0008) [2023-12-26 17:46:39,762][105620] Updated weights for policy 1, policy_version 331917 (0.0008) [2023-12-26 17:46:39,826][105620] Updated weights for policy 1, policy_version 331927 (0.0011) [2023-12-26 17:46:39,892][105620] Updated weights for policy 1, policy_version 331937 (0.0010) [2023-12-26 17:46:40,070][105692] Updated weights for policy 0, policy_version 331657 (0.0009) [2023-12-26 17:46:40,134][105692] Updated weights for policy 0, policy_version 331667 (0.0008) [2023-12-26 17:46:40,196][105692] Updated weights for policy 0, policy_version 331677 (0.0007) [2023-12-26 17:46:40,255][105692] Updated weights for policy 0, policy_version 331687 (0.0006) [2023-12-26 17:46:40,693][105620] Updated weights for policy 1, policy_version 331947 (0.0007) [2023-12-26 17:46:40,754][105620] Updated weights for policy 1, policy_version 331957 (0.0007) [2023-12-26 17:46:40,813][105620] Updated weights for policy 1, policy_version 331967 (0.0010) [2023-12-26 17:46:40,997][105692] Updated weights for policy 0, policy_version 331697 (0.0010) [2023-12-26 17:46:41,059][105692] Updated weights for policy 0, policy_version 331708 (0.0010) [2023-12-26 17:46:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 169918464. Throughput: 0: 9792.3, 1: 9661.7. Samples: 169928592. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 17:46:41,062][104569] Avg episode reward: [(0, '9356.100'), (1, '9203.861')] [2023-12-26 17:46:41,123][105692] Updated weights for policy 0, policy_version 331718 (0.0009) [2023-12-26 17:46:41,519][105620] Updated weights for policy 1, policy_version 331977 (0.0007) [2023-12-26 17:46:41,584][105620] Updated weights for policy 1, policy_version 331987 (0.0010) [2023-12-26 17:46:41,650][105620] Updated weights for policy 1, policy_version 331997 (0.0011) [2023-12-26 17:46:41,716][105620] Updated weights for policy 1, policy_version 332007 (0.0011) [2023-12-26 17:46:41,964][105692] Updated weights for policy 0, policy_version 331728 (0.0008) [2023-12-26 17:46:42,028][105692] Updated weights for policy 0, policy_version 331738 (0.0008) [2023-12-26 17:46:42,085][105692] Updated weights for policy 0, policy_version 331748 (0.0008) [2023-12-26 17:46:42,491][105620] Updated weights for policy 1, policy_version 332017 (0.0006) [2023-12-26 17:46:42,561][105620] Updated weights for policy 1, policy_version 332027 (0.0006) [2023-12-26 17:46:42,626][105620] Updated weights for policy 1, policy_version 332037 (0.0007) [2023-12-26 17:46:42,913][105692] Updated weights for policy 0, policy_version 331758 (0.0009) [2023-12-26 17:46:42,964][105692] Updated weights for policy 0, policy_version 331768 (0.0009) [2023-12-26 17:46:43,019][105692] Updated weights for policy 0, policy_version 331778 (0.0009) [2023-12-26 17:46:43,208][105620] Updated weights for policy 1, policy_version 332047 (0.0009) [2023-12-26 17:46:43,257][105620] Updated weights for policy 1, policy_version 332057 (0.0008) [2023-12-26 17:46:43,308][105620] Updated weights for policy 1, policy_version 332067 (0.0009) [2023-12-26 17:46:43,898][105692] Updated weights for policy 0, policy_version 331788 (0.0009) [2023-12-26 17:46:43,916][105620] Updated weights for policy 1, policy_version 332077 (0.0009) [2023-12-26 17:46:43,954][105692] Updated weights for policy 0, policy_version 331798 (0.0007) [2023-12-26 17:46:43,969][105620] Updated weights for policy 1, policy_version 332087 (0.0007) [2023-12-26 17:46:44,015][105692] Updated weights for policy 0, policy_version 331808 (0.0008) [2023-12-26 17:46:44,028][105620] Updated weights for policy 1, policy_version 332097 (0.0009) [2023-12-26 17:46:44,673][105620] Updated weights for policy 1, policy_version 332107 (0.0009) [2023-12-26 17:46:44,741][105620] Updated weights for policy 1, policy_version 332117 (0.0008) [2023-12-26 17:46:44,807][105620] Updated weights for policy 1, policy_version 332127 (0.0009) [2023-12-26 17:46:44,891][105692] Updated weights for policy 0, policy_version 331818 (0.0007) [2023-12-26 17:46:44,944][105692] Updated weights for policy 0, policy_version 331828 (0.0007) [2023-12-26 17:46:44,999][105692] Updated weights for policy 0, policy_version 331838 (0.0007) [2023-12-26 17:46:45,055][105692] Updated weights for policy 0, policy_version 331848 (0.0009) [2023-12-26 17:46:45,520][105620] Updated weights for policy 1, policy_version 332137 (0.0011) [2023-12-26 17:46:45,584][105620] Updated weights for policy 1, policy_version 332147 (0.0009) [2023-12-26 17:46:45,638][105620] Updated weights for policy 1, policy_version 332157 (0.0009) [2023-12-26 17:46:45,699][105620] Updated weights for policy 1, policy_version 332167 (0.0009) [2023-12-26 17:46:45,802][105692] Updated weights for policy 0, policy_version 331858 (0.0009) [2023-12-26 17:46:45,853][105692] Updated weights for policy 0, policy_version 331868 (0.0009) [2023-12-26 17:46:45,912][105692] Updated weights for policy 0, policy_version 331878 (0.0009) [2023-12-26 17:46:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 170016768. Throughput: 0: 9593.2, 1: 9728.3. Samples: 169985620. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 17:46:46,062][104569] Avg episode reward: [(0, '9356.678'), (1, '9295.276')] [2023-12-26 17:46:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000331880_84975616.pth... [2023-12-26 17:46:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000332168_85041152.pth... [2023-12-26 17:46:46,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000330760_84688896.pth [2023-12-26 17:46:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000331016_84746240.pth [2023-12-26 17:46:46,457][105620] Updated weights for policy 1, policy_version 332177 (0.0009) [2023-12-26 17:46:46,512][105620] Updated weights for policy 1, policy_version 332187 (0.0009) [2023-12-26 17:46:46,569][105620] Updated weights for policy 1, policy_version 332197 (0.0009) [2023-12-26 17:46:46,675][105692] Updated weights for policy 0, policy_version 331888 (0.0009) [2023-12-26 17:46:46,727][105692] Updated weights for policy 0, policy_version 331898 (0.0009) [2023-12-26 17:46:46,794][105692] Updated weights for policy 0, policy_version 331908 (0.0010) [2023-12-26 17:46:47,285][105620] Updated weights for policy 1, policy_version 332207 (0.0007) [2023-12-26 17:46:47,340][105620] Updated weights for policy 1, policy_version 332217 (0.0009) [2023-12-26 17:46:47,394][105620] Updated weights for policy 1, policy_version 332227 (0.0008) [2023-12-26 17:46:47,574][105692] Updated weights for policy 0, policy_version 331918 (0.0009) [2023-12-26 17:46:47,625][105692] Updated weights for policy 0, policy_version 331928 (0.0009) [2023-12-26 17:46:47,677][105692] Updated weights for policy 0, policy_version 331938 (0.0009) [2023-12-26 17:46:48,146][105620] Updated weights for policy 1, policy_version 332237 (0.0009) [2023-12-26 17:46:48,205][105620] Updated weights for policy 1, policy_version 332247 (0.0009) [2023-12-26 17:46:48,265][105620] Updated weights for policy 1, policy_version 332257 (0.0008) [2023-12-26 17:46:48,467][105692] Updated weights for policy 0, policy_version 331948 (0.0009) [2023-12-26 17:46:48,519][105692] Updated weights for policy 0, policy_version 331958 (0.0009) [2023-12-26 17:46:48,567][105692] Updated weights for policy 0, policy_version 331968 (0.0009) [2023-12-26 17:46:49,008][105620] Updated weights for policy 1, policy_version 332267 (0.0008) [2023-12-26 17:46:49,066][105620] Updated weights for policy 1, policy_version 332277 (0.0006) [2023-12-26 17:46:49,132][105620] Updated weights for policy 1, policy_version 332287 (0.0009) [2023-12-26 17:46:49,372][105692] Updated weights for policy 0, policy_version 331978 (0.0009) [2023-12-26 17:46:49,435][105692] Updated weights for policy 0, policy_version 331988 (0.0006) [2023-12-26 17:46:49,487][105692] Updated weights for policy 0, policy_version 331998 (0.0007) [2023-12-26 17:46:49,543][105692] Updated weights for policy 0, policy_version 332008 (0.0009) [2023-12-26 17:46:49,898][105620] Updated weights for policy 1, policy_version 332297 (0.0009) [2023-12-26 17:46:49,954][105620] Updated weights for policy 1, policy_version 332307 (0.0009) [2023-12-26 17:46:50,004][105620] Updated weights for policy 1, policy_version 332317 (0.0009) [2023-12-26 17:46:50,055][105620] Updated weights for policy 1, policy_version 332327 (0.0009) [2023-12-26 17:46:50,288][105692] Updated weights for policy 0, policy_version 332018 (0.0009) [2023-12-26 17:46:50,340][105692] Updated weights for policy 0, policy_version 332028 (0.0009) [2023-12-26 17:46:50,401][105692] Updated weights for policy 0, policy_version 332038 (0.0009) [2023-12-26 17:46:50,839][105620] Updated weights for policy 1, policy_version 332337 (0.0009) [2023-12-26 17:46:50,903][105620] Updated weights for policy 1, policy_version 332347 (0.0009) [2023-12-26 17:46:50,968][105620] Updated weights for policy 1, policy_version 332357 (0.0009) [2023-12-26 17:46:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 170106880. Throughput: 0: 9510.7, 1: 9748.0. Samples: 170096604. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 17:46:51,063][104569] Avg episode reward: [(0, '9357.180'), (1, '9175.435')] [2023-12-26 17:46:51,139][105692] Updated weights for policy 0, policy_version 332048 (0.0009) [2023-12-26 17:46:51,184][105692] Updated weights for policy 0, policy_version 332058 (0.0008) [2023-12-26 17:46:51,233][105692] Updated weights for policy 0, policy_version 332068 (0.0008) [2023-12-26 17:46:51,754][105620] Updated weights for policy 1, policy_version 332367 (0.0009) [2023-12-26 17:46:51,816][105620] Updated weights for policy 1, policy_version 332377 (0.0005) [2023-12-26 17:46:51,874][105620] Updated weights for policy 1, policy_version 332387 (0.0008) [2023-12-26 17:46:51,994][105692] Updated weights for policy 0, policy_version 332078 (0.0009) [2023-12-26 17:46:52,060][105692] Updated weights for policy 0, policy_version 332088 (0.0011) [2023-12-26 17:46:52,120][105692] Updated weights for policy 0, policy_version 332098 (0.0011) [2023-12-26 17:46:52,594][105620] Updated weights for policy 1, policy_version 332397 (0.0008) [2023-12-26 17:46:52,643][105620] Updated weights for policy 1, policy_version 332407 (0.0008) [2023-12-26 17:46:52,699][105620] Updated weights for policy 1, policy_version 332417 (0.0008) [2023-12-26 17:46:52,869][105692] Updated weights for policy 0, policy_version 332108 (0.0011) [2023-12-26 17:46:52,928][105692] Updated weights for policy 0, policy_version 332118 (0.0011) [2023-12-26 17:46:52,988][105692] Updated weights for policy 0, policy_version 332128 (0.0011) [2023-12-26 17:46:53,437][105620] Updated weights for policy 1, policy_version 332427 (0.0008) [2023-12-26 17:46:53,500][105620] Updated weights for policy 1, policy_version 332437 (0.0008) [2023-12-26 17:46:53,556][105620] Updated weights for policy 1, policy_version 332447 (0.0008) [2023-12-26 17:46:53,742][105692] Updated weights for policy 0, policy_version 332138 (0.0010) [2023-12-26 17:46:53,807][105692] Updated weights for policy 0, policy_version 332148 (0.0010) [2023-12-26 17:46:53,866][105692] Updated weights for policy 0, policy_version 332158 (0.0010) [2023-12-26 17:46:53,925][105692] Updated weights for policy 0, policy_version 332168 (0.0011) [2023-12-26 17:46:54,228][105620] Updated weights for policy 1, policy_version 332457 (0.0007) [2023-12-26 17:46:54,276][105620] Updated weights for policy 1, policy_version 332467 (0.0005) [2023-12-26 17:46:54,327][105620] Updated weights for policy 1, policy_version 332477 (0.0010) [2023-12-26 17:46:54,390][105620] Updated weights for policy 1, policy_version 332487 (0.0010) [2023-12-26 17:46:54,674][105692] Updated weights for policy 0, policy_version 332178 (0.0011) [2023-12-26 17:46:54,740][105692] Updated weights for policy 0, policy_version 332188 (0.0011) [2023-12-26 17:46:54,809][105692] Updated weights for policy 0, policy_version 332198 (0.0011) [2023-12-26 17:46:55,146][105620] Updated weights for policy 1, policy_version 332497 (0.0008) [2023-12-26 17:46:55,210][105620] Updated weights for policy 1, policy_version 332507 (0.0008) [2023-12-26 17:46:55,270][105620] Updated weights for policy 1, policy_version 332517 (0.0008) [2023-12-26 17:46:55,544][105692] Updated weights for policy 0, policy_version 332208 (0.0010) [2023-12-26 17:46:55,596][105692] Updated weights for policy 0, policy_version 332218 (0.0010) [2023-12-26 17:46:55,660][105692] Updated weights for policy 0, policy_version 332228 (0.0011) [2023-12-26 17:46:55,993][105620] Updated weights for policy 1, policy_version 332527 (0.0010) [2023-12-26 17:46:56,046][105620] Updated weights for policy 1, policy_version 332537 (0.0010) [2023-12-26 17:46:56,062][104569] Fps is (10 sec: 18022.1, 60 sec: 19114.6, 300 sec: 19549.7). Total num frames: 170196992. Throughput: 0: 9509.0, 1: 9707.8. Samples: 170209340. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 17:46:56,063][104569] Avg episode reward: [(0, '9357.188'), (1, '8995.018')] [2023-12-26 17:46:56,098][105620] Updated weights for policy 1, policy_version 332547 (0.0009) [2023-12-26 17:46:56,421][105692] Updated weights for policy 0, policy_version 332238 (0.0009) [2023-12-26 17:46:56,485][105692] Updated weights for policy 0, policy_version 332248 (0.0009) [2023-12-26 17:46:56,549][105692] Updated weights for policy 0, policy_version 332258 (0.0009) [2023-12-26 17:46:56,746][105620] Updated weights for policy 1, policy_version 332557 (0.0007) [2023-12-26 17:46:56,815][105620] Updated weights for policy 1, policy_version 332567 (0.0007) [2023-12-26 17:46:56,871][105620] Updated weights for policy 1, policy_version 332577 (0.0005) [2023-12-26 17:46:57,289][105692] Updated weights for policy 0, policy_version 332268 (0.0009) [2023-12-26 17:46:57,340][105692] Updated weights for policy 0, policy_version 332278 (0.0009) [2023-12-26 17:46:57,398][105692] Updated weights for policy 0, policy_version 332288 (0.0008) [2023-12-26 17:46:57,427][105620] Updated weights for policy 1, policy_version 332587 (0.0006) [2023-12-26 17:46:57,493][105620] Updated weights for policy 1, policy_version 332597 (0.0005) [2023-12-26 17:46:57,554][105620] Updated weights for policy 1, policy_version 332607 (0.0007) [2023-12-26 17:46:58,140][105692] Updated weights for policy 0, policy_version 332298 (0.0008) [2023-12-26 17:46:58,197][105692] Updated weights for policy 0, policy_version 332308 (0.0008) [2023-12-26 17:46:58,224][105620] Updated weights for policy 1, policy_version 332617 (0.0007) [2023-12-26 17:46:58,256][105692] Updated weights for policy 0, policy_version 332318 (0.0006) [2023-12-26 17:46:58,286][105620] Updated weights for policy 1, policy_version 332627 (0.0010) [2023-12-26 17:46:58,307][105692] Updated weights for policy 0, policy_version 332328 (0.0006) [2023-12-26 17:46:58,350][105620] Updated weights for policy 1, policy_version 332637 (0.0009) [2023-12-26 17:46:58,418][105620] Updated weights for policy 1, policy_version 332647 (0.0009) [2023-12-26 17:46:59,093][105692] Updated weights for policy 0, policy_version 332338 (0.0011) [2023-12-26 17:46:59,149][105692] Updated weights for policy 0, policy_version 332348 (0.0010) [2023-12-26 17:46:59,204][105620] Updated weights for policy 1, policy_version 332657 (0.0008) [2023-12-26 17:46:59,204][105692] Updated weights for policy 0, policy_version 332358 (0.0008) [2023-12-26 17:46:59,268][105620] Updated weights for policy 1, policy_version 332667 (0.0008) [2023-12-26 17:46:59,330][105620] Updated weights for policy 1, policy_version 332677 (0.0007) [2023-12-26 17:46:59,858][105692] Updated weights for policy 0, policy_version 332368 (0.0010) [2023-12-26 17:46:59,919][105692] Updated weights for policy 0, policy_version 332378 (0.0010) [2023-12-26 17:46:59,978][105692] Updated weights for policy 0, policy_version 332388 (0.0011) [2023-12-26 17:47:00,001][105620] Updated weights for policy 1, policy_version 332687 (0.0008) [2023-12-26 17:47:00,058][105620] Updated weights for policy 1, policy_version 332697 (0.0009) [2023-12-26 17:47:00,110][105620] Updated weights for policy 1, policy_version 332707 (0.0009) [2023-12-26 17:47:00,644][105692] Updated weights for policy 0, policy_version 332398 (0.0007) [2023-12-26 17:47:00,695][105692] Updated weights for policy 0, policy_version 332408 (0.0005) [2023-12-26 17:47:00,747][105692] Updated weights for policy 0, policy_version 332418 (0.0005) [2023-12-26 17:47:00,793][105620] Updated weights for policy 1, policy_version 332717 (0.0008) [2023-12-26 17:47:00,843][105620] Updated weights for policy 1, policy_version 332727 (0.0005) [2023-12-26 17:47:00,896][105620] Updated weights for policy 1, policy_version 332737 (0.0005) [2023-12-26 17:47:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 170303488. Throughput: 0: 9516.0, 1: 9730.0. Samples: 170267844. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 17:47:01,063][104569] Avg episode reward: [(0, '9356.729'), (1, '9264.529')] [2023-12-26 17:47:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000332424_85114880.pth... [2023-12-26 17:47:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000332744_85188608.pth... [2023-12-26 17:47:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000331336_84836352.pth [2023-12-26 17:47:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000331592_84893696.pth [2023-12-26 17:47:01,390][105692] Updated weights for policy 0, policy_version 332428 (0.0008) [2023-12-26 17:47:01,452][105692] Updated weights for policy 0, policy_version 332438 (0.0010) [2023-12-26 17:47:01,517][105692] Updated weights for policy 0, policy_version 332448 (0.0010) [2023-12-26 17:47:01,561][105620] Updated weights for policy 1, policy_version 332747 (0.0005) [2023-12-26 17:47:01,618][105620] Updated weights for policy 1, policy_version 332757 (0.0008) [2023-12-26 17:47:01,680][105620] Updated weights for policy 1, policy_version 332767 (0.0008) [2023-12-26 17:47:02,119][105692] Updated weights for policy 0, policy_version 332458 (0.0010) [2023-12-26 17:47:02,180][105692] Updated weights for policy 0, policy_version 332468 (0.0005) [2023-12-26 17:47:02,231][105692] Updated weights for policy 0, policy_version 332478 (0.0005) [2023-12-26 17:47:02,290][105692] Updated weights for policy 0, policy_version 332488 (0.0006) [2023-12-26 17:47:02,545][105620] Updated weights for policy 1, policy_version 332777 (0.0009) [2023-12-26 17:47:02,607][105620] Updated weights for policy 1, policy_version 332787 (0.0009) [2023-12-26 17:47:02,654][105620] Updated weights for policy 1, policy_version 332797 (0.0009) [2023-12-26 17:47:02,705][105620] Updated weights for policy 1, policy_version 332807 (0.0009) [2023-12-26 17:47:02,899][105692] Updated weights for policy 0, policy_version 332498 (0.0006) [2023-12-26 17:47:02,953][105692] Updated weights for policy 0, policy_version 332508 (0.0005) [2023-12-26 17:47:02,995][105692] Updated weights for policy 0, policy_version 332518 (0.0005) [2023-12-26 17:47:03,550][105692] Updated weights for policy 0, policy_version 332528 (0.0006) [2023-12-26 17:47:03,563][105620] Updated weights for policy 1, policy_version 332817 (0.0009) [2023-12-26 17:47:03,598][105692] Updated weights for policy 0, policy_version 332538 (0.0005) [2023-12-26 17:47:03,615][105620] Updated weights for policy 1, policy_version 332827 (0.0009) [2023-12-26 17:47:03,647][105692] Updated weights for policy 0, policy_version 332548 (0.0006) [2023-12-26 17:47:03,663][105620] Updated weights for policy 1, policy_version 332837 (0.0006) [2023-12-26 17:47:04,389][105620] Updated weights for policy 1, policy_version 332847 (0.0008) [2023-12-26 17:47:04,411][105692] Updated weights for policy 0, policy_version 332558 (0.0009) [2023-12-26 17:47:04,442][105620] Updated weights for policy 1, policy_version 332857 (0.0007) [2023-12-26 17:47:04,474][105692] Updated weights for policy 0, policy_version 332568 (0.0008) [2023-12-26 17:47:04,491][105620] Updated weights for policy 1, policy_version 332867 (0.0006) [2023-12-26 17:47:04,537][105692] Updated weights for policy 0, policy_version 332578 (0.0009) [2023-12-26 17:47:05,268][105620] Updated weights for policy 1, policy_version 332877 (0.0008) [2023-12-26 17:47:05,268][105692] Updated weights for policy 0, policy_version 332588 (0.0007) [2023-12-26 17:47:05,316][105620] Updated weights for policy 1, policy_version 332887 (0.0010) [2023-12-26 17:47:05,326][105692] Updated weights for policy 0, policy_version 332598 (0.0006) [2023-12-26 17:47:05,365][105620] Updated weights for policy 1, policy_version 332897 (0.0010) [2023-12-26 17:47:05,382][105692] Updated weights for policy 0, policy_version 332608 (0.0006) [2023-12-26 17:47:06,032][105620] Updated weights for policy 1, policy_version 332907 (0.0009) [2023-12-26 17:47:06,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 170393600. Throughput: 0: 9532.3, 1: 9738.7. Samples: 170387640. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 17:47:06,062][104569] Avg episode reward: [(0, '9356.468'), (1, '9356.404')] [2023-12-26 17:47:06,092][105620] Updated weights for policy 1, policy_version 332917 (0.0005) [2023-12-26 17:47:06,159][105620] Updated weights for policy 1, policy_version 332927 (0.0008) [2023-12-26 17:47:06,189][105692] Updated weights for policy 0, policy_version 332618 (0.0009) [2023-12-26 17:47:06,246][105692] Updated weights for policy 0, policy_version 332628 (0.0008) [2023-12-26 17:47:06,306][105692] Updated weights for policy 0, policy_version 332638 (0.0009) [2023-12-26 17:47:06,366][105692] Updated weights for policy 0, policy_version 332648 (0.0010) [2023-12-26 17:47:06,819][105620] Updated weights for policy 1, policy_version 332937 (0.0008) [2023-12-26 17:47:06,869][105620] Updated weights for policy 1, policy_version 332947 (0.0008) [2023-12-26 17:47:06,931][105620] Updated weights for policy 1, policy_version 332957 (0.0009) [2023-12-26 17:47:06,987][105620] Updated weights for policy 1, policy_version 332967 (0.0007) [2023-12-26 17:47:07,173][105692] Updated weights for policy 0, policy_version 332658 (0.0009) [2023-12-26 17:47:07,233][105692] Updated weights for policy 0, policy_version 332668 (0.0009) [2023-12-26 17:47:07,299][105692] Updated weights for policy 0, policy_version 332678 (0.0009) [2023-12-26 17:47:07,698][105620] Updated weights for policy 1, policy_version 332977 (0.0008) [2023-12-26 17:47:07,747][105620] Updated weights for policy 1, policy_version 332987 (0.0008) [2023-12-26 17:47:07,801][105620] Updated weights for policy 1, policy_version 332997 (0.0008) [2023-12-26 17:47:08,012][105692] Updated weights for policy 0, policy_version 332688 (0.0006) [2023-12-26 17:47:08,060][105692] Updated weights for policy 0, policy_version 332698 (0.0005) [2023-12-26 17:47:08,116][105692] Updated weights for policy 0, policy_version 332708 (0.0008) [2023-12-26 17:47:08,600][105620] Updated weights for policy 1, policy_version 333007 (0.0008) [2023-12-26 17:47:08,660][105620] Updated weights for policy 1, policy_version 333017 (0.0008) [2023-12-26 17:47:08,708][105620] Updated weights for policy 1, policy_version 333027 (0.0008) [2023-12-26 17:47:08,834][105692] Updated weights for policy 0, policy_version 332718 (0.0009) [2023-12-26 17:47:08,901][105692] Updated weights for policy 0, policy_version 332728 (0.0008) [2023-12-26 17:47:08,957][105692] Updated weights for policy 0, policy_version 332738 (0.0011) [2023-12-26 17:47:09,462][105620] Updated weights for policy 1, policy_version 333037 (0.0007) [2023-12-26 17:47:09,524][105620] Updated weights for policy 1, policy_version 333047 (0.0008) [2023-12-26 17:47:09,577][105620] Updated weights for policy 1, policy_version 333057 (0.0006) [2023-12-26 17:47:09,727][105692] Updated weights for policy 0, policy_version 332748 (0.0010) [2023-12-26 17:47:09,786][105692] Updated weights for policy 0, policy_version 332758 (0.0011) [2023-12-26 17:47:09,852][105692] Updated weights for policy 0, policy_version 332768 (0.0010) [2023-12-26 17:47:10,324][105620] Updated weights for policy 1, policy_version 333067 (0.0011) [2023-12-26 17:47:10,388][105620] Updated weights for policy 1, policy_version 333077 (0.0009) [2023-12-26 17:47:10,451][105620] Updated weights for policy 1, policy_version 333087 (0.0011) [2023-12-26 17:47:10,598][105692] Updated weights for policy 0, policy_version 332778 (0.0011) [2023-12-26 17:47:10,660][105692] Updated weights for policy 0, policy_version 332788 (0.0010) [2023-12-26 17:47:10,721][105692] Updated weights for policy 0, policy_version 332798 (0.0010) [2023-12-26 17:47:10,780][105692] Updated weights for policy 0, policy_version 332808 (0.0010) [2023-12-26 17:47:11,036][105620] Updated weights for policy 1, policy_version 333097 (0.0010) [2023-12-26 17:47:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 170491904. Throughput: 0: 9512.7, 1: 9796.9. Samples: 170502456. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 17:47:11,062][104569] Avg episode reward: [(0, '9175.526'), (1, '9265.392')] [2023-12-26 17:47:11,096][105620] Updated weights for policy 1, policy_version 333107 (0.0007) [2023-12-26 17:47:11,170][105620] Updated weights for policy 1, policy_version 333117 (0.0008) [2023-12-26 17:47:11,237][105620] Updated weights for policy 1, policy_version 333127 (0.0010) [2023-12-26 17:47:11,546][105692] Updated weights for policy 0, policy_version 332818 (0.0008) [2023-12-26 17:47:11,608][105692] Updated weights for policy 0, policy_version 332828 (0.0010) [2023-12-26 17:47:11,679][105692] Updated weights for policy 0, policy_version 332838 (0.0010) [2023-12-26 17:47:11,982][105620] Updated weights for policy 1, policy_version 333137 (0.0011) [2023-12-26 17:47:12,045][105620] Updated weights for policy 1, policy_version 333147 (0.0011) [2023-12-26 17:47:12,110][105620] Updated weights for policy 1, policy_version 333157 (0.0010) [2023-12-26 17:47:12,414][105692] Updated weights for policy 0, policy_version 332848 (0.0011) [2023-12-26 17:47:12,472][105692] Updated weights for policy 0, policy_version 332858 (0.0010) [2023-12-26 17:47:12,527][105692] Updated weights for policy 0, policy_version 332868 (0.0010) [2023-12-26 17:47:12,886][105620] Updated weights for policy 1, policy_version 333167 (0.0009) [2023-12-26 17:47:12,945][105620] Updated weights for policy 1, policy_version 333177 (0.0010) [2023-12-26 17:47:13,000][105620] Updated weights for policy 1, policy_version 333187 (0.0010) [2023-12-26 17:47:13,272][105692] Updated weights for policy 0, policy_version 332878 (0.0010) [2023-12-26 17:47:13,330][105692] Updated weights for policy 0, policy_version 332888 (0.0010) [2023-12-26 17:47:13,384][105692] Updated weights for policy 0, policy_version 332898 (0.0010) [2023-12-26 17:47:13,731][105620] Updated weights for policy 1, policy_version 333197 (0.0010) [2023-12-26 17:47:13,787][105620] Updated weights for policy 1, policy_version 333207 (0.0009) [2023-12-26 17:47:13,844][105620] Updated weights for policy 1, policy_version 333218 (0.0010) [2023-12-26 17:47:14,057][105692] Updated weights for policy 0, policy_version 332908 (0.0010) [2023-12-26 17:47:14,127][105692] Updated weights for policy 0, policy_version 332918 (0.0009) [2023-12-26 17:47:14,192][105692] Updated weights for policy 0, policy_version 332928 (0.0007) [2023-12-26 17:47:14,475][105620] Updated weights for policy 1, policy_version 333228 (0.0006) [2023-12-26 17:47:14,532][105620] Updated weights for policy 1, policy_version 333238 (0.0006) [2023-12-26 17:47:14,590][105620] Updated weights for policy 1, policy_version 333248 (0.0009) [2023-12-26 17:47:14,842][105692] Updated weights for policy 0, policy_version 332938 (0.0006) [2023-12-26 17:47:14,909][105692] Updated weights for policy 0, policy_version 332948 (0.0011) [2023-12-26 17:47:14,969][105692] Updated weights for policy 0, policy_version 332958 (0.0010) [2023-12-26 17:47:15,029][105692] Updated weights for policy 0, policy_version 332968 (0.0010) [2023-12-26 17:47:15,182][105620] Updated weights for policy 1, policy_version 333258 (0.0006) [2023-12-26 17:47:15,231][105620] Updated weights for policy 1, policy_version 333268 (0.0008) [2023-12-26 17:47:15,288][105620] Updated weights for policy 1, policy_version 333278 (0.0008) [2023-12-26 17:47:15,348][105620] Updated weights for policy 1, policy_version 333288 (0.0008) [2023-12-26 17:47:15,757][105692] Updated weights for policy 0, policy_version 332978 (0.0011) [2023-12-26 17:47:15,822][105692] Updated weights for policy 0, policy_version 332988 (0.0010) [2023-12-26 17:47:15,886][105692] Updated weights for policy 0, policy_version 332998 (0.0010) [2023-12-26 17:47:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.3, 300 sec: 19522.0). Total num frames: 170590208. Throughput: 0: 9417.0, 1: 9747.8. Samples: 170558264. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 17:47:16,062][104569] Avg episode reward: [(0, '9175.382'), (1, '9178.637')] [2023-12-26 17:47:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000333288_85327872.pth... [2023-12-26 17:47:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000333000_85262336.pth... [2023-12-26 17:47:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000332168_85041152.pth [2023-12-26 17:47:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000331880_84975616.pth [2023-12-26 17:47:16,163][105620] Updated weights for policy 1, policy_version 333298 (0.0010) [2023-12-26 17:47:16,218][105620] Updated weights for policy 1, policy_version 333308 (0.0012) [2023-12-26 17:47:16,275][105620] Updated weights for policy 1, policy_version 333318 (0.0012) [2023-12-26 17:47:16,436][105692] Updated weights for policy 0, policy_version 333008 (0.0006) [2023-12-26 17:47:16,491][105692] Updated weights for policy 0, policy_version 333018 (0.0005) [2023-12-26 17:47:16,537][105692] Updated weights for policy 0, policy_version 333028 (0.0005) [2023-12-26 17:47:17,049][105692] Updated weights for policy 0, policy_version 333038 (0.0005) [2023-12-26 17:47:17,111][105692] Updated weights for policy 0, policy_version 333048 (0.0006) [2023-12-26 17:47:17,176][105692] Updated weights for policy 0, policy_version 333058 (0.0005) [2023-12-26 17:47:17,216][105620] Updated weights for policy 1, policy_version 333328 (0.0010) [2023-12-26 17:47:17,278][105620] Updated weights for policy 1, policy_version 333338 (0.0009) [2023-12-26 17:47:17,345][105620] Updated weights for policy 1, policy_version 333348 (0.0010) [2023-12-26 17:47:17,726][105692] Updated weights for policy 0, policy_version 333068 (0.0007) [2023-12-26 17:47:17,774][105692] Updated weights for policy 0, policy_version 333078 (0.0010) [2023-12-26 17:47:17,837][105692] Updated weights for policy 0, policy_version 333088 (0.0011) [2023-12-26 17:47:18,156][105620] Updated weights for policy 1, policy_version 333358 (0.0009) [2023-12-26 17:47:18,215][105620] Updated weights for policy 1, policy_version 333368 (0.0008) [2023-12-26 17:47:18,268][105620] Updated weights for policy 1, policy_version 333378 (0.0008) [2023-12-26 17:47:18,583][105692] Updated weights for policy 0, policy_version 333098 (0.0010) [2023-12-26 17:47:18,638][105692] Updated weights for policy 0, policy_version 333108 (0.0010) [2023-12-26 17:47:18,703][105692] Updated weights for policy 0, policy_version 333118 (0.0010) [2023-12-26 17:47:18,765][105692] Updated weights for policy 0, policy_version 333128 (0.0010) [2023-12-26 17:47:19,042][105620] Updated weights for policy 1, policy_version 333388 (0.0009) [2023-12-26 17:47:19,108][105620] Updated weights for policy 1, policy_version 333398 (0.0008) [2023-12-26 17:47:19,171][105620] Updated weights for policy 1, policy_version 333408 (0.0008) [2023-12-26 17:47:19,526][105692] Updated weights for policy 0, policy_version 333138 (0.0011) [2023-12-26 17:47:19,590][105692] Updated weights for policy 0, policy_version 333148 (0.0009) [2023-12-26 17:47:19,645][105692] Updated weights for policy 0, policy_version 333158 (0.0010) [2023-12-26 17:47:19,937][105620] Updated weights for policy 1, policy_version 333418 (0.0008) [2023-12-26 17:47:20,001][105620] Updated weights for policy 1, policy_version 333428 (0.0008) [2023-12-26 17:47:20,061][105620] Updated weights for policy 1, policy_version 333438 (0.0008) [2023-12-26 17:47:20,110][105620] Updated weights for policy 1, policy_version 333448 (0.0009) [2023-12-26 17:47:20,306][105692] Updated weights for policy 0, policy_version 333168 (0.0008) [2023-12-26 17:47:20,367][105692] Updated weights for policy 0, policy_version 333178 (0.0008) [2023-12-26 17:47:20,422][105692] Updated weights for policy 0, policy_version 333188 (0.0009) [2023-12-26 17:47:20,925][105620] Updated weights for policy 1, policy_version 333458 (0.0011) [2023-12-26 17:47:21,016][105620] Updated weights for policy 1, policy_version 333468 (0.0011) [2023-12-26 17:47:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19494.2). Total num frames: 170680320. Throughput: 0: 9515.9, 1: 9653.2. Samples: 170677376. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 17:47:21,062][104569] Avg episode reward: [(0, '9265.899'), (1, '9269.957')] [2023-12-26 17:47:21,083][105620] Updated weights for policy 1, policy_version 333478 (0.0011) [2023-12-26 17:47:21,197][105692] Updated weights for policy 0, policy_version 333198 (0.0008) [2023-12-26 17:47:21,271][105692] Updated weights for policy 0, policy_version 333208 (0.0008) [2023-12-26 17:47:21,338][105692] Updated weights for policy 0, policy_version 333218 (0.0007) [2023-12-26 17:47:21,756][105620] Updated weights for policy 1, policy_version 333488 (0.0009) [2023-12-26 17:47:21,808][105620] Updated weights for policy 1, policy_version 333498 (0.0006) [2023-12-26 17:47:21,869][105620] Updated weights for policy 1, policy_version 333508 (0.0005) [2023-12-26 17:47:22,020][105692] Updated weights for policy 0, policy_version 333228 (0.0007) [2023-12-26 17:47:22,080][105692] Updated weights for policy 0, policy_version 333238 (0.0006) [2023-12-26 17:47:22,147][105692] Updated weights for policy 0, policy_version 333248 (0.0009) [2023-12-26 17:47:22,577][105620] Updated weights for policy 1, policy_version 333518 (0.0007) [2023-12-26 17:47:22,641][105620] Updated weights for policy 1, policy_version 333528 (0.0009) [2023-12-26 17:47:22,711][105620] Updated weights for policy 1, policy_version 333538 (0.0010) [2023-12-26 17:47:22,862][105692] Updated weights for policy 0, policy_version 333258 (0.0009) [2023-12-26 17:47:22,913][105692] Updated weights for policy 0, policy_version 333268 (0.0009) [2023-12-26 17:47:22,964][105692] Updated weights for policy 0, policy_version 333278 (0.0008) [2023-12-26 17:47:23,015][105692] Updated weights for policy 0, policy_version 333288 (0.0009) [2023-12-26 17:47:23,487][105620] Updated weights for policy 1, policy_version 333548 (0.0009) [2023-12-26 17:47:23,548][105620] Updated weights for policy 1, policy_version 333558 (0.0009) [2023-12-26 17:47:23,605][105620] Updated weights for policy 1, policy_version 333568 (0.0009) [2023-12-26 17:47:23,772][105692] Updated weights for policy 0, policy_version 333298 (0.0009) [2023-12-26 17:47:23,826][105692] Updated weights for policy 0, policy_version 333308 (0.0009) [2023-12-26 17:47:23,877][105692] Updated weights for policy 0, policy_version 333318 (0.0009) [2023-12-26 17:47:24,280][105620] Updated weights for policy 1, policy_version 333578 (0.0008) [2023-12-26 17:47:24,352][105620] Updated weights for policy 1, policy_version 333588 (0.0005) [2023-12-26 17:47:24,418][105620] Updated weights for policy 1, policy_version 333598 (0.0006) [2023-12-26 17:47:24,482][105620] Updated weights for policy 1, policy_version 333608 (0.0008) [2023-12-26 17:47:24,678][105692] Updated weights for policy 0, policy_version 333328 (0.0009) [2023-12-26 17:47:24,746][105692] Updated weights for policy 0, policy_version 333338 (0.0009) [2023-12-26 17:47:24,802][105692] Updated weights for policy 0, policy_version 333348 (0.0009) [2023-12-26 17:47:25,144][105620] Updated weights for policy 1, policy_version 333618 (0.0008) [2023-12-26 17:47:25,209][105620] Updated weights for policy 1, policy_version 333628 (0.0008) [2023-12-26 17:47:25,271][105620] Updated weights for policy 1, policy_version 333638 (0.0009) [2023-12-26 17:47:25,576][105692] Updated weights for policy 0, policy_version 333358 (0.0009) [2023-12-26 17:47:25,634][105692] Updated weights for policy 0, policy_version 333368 (0.0009) [2023-12-26 17:47:25,683][105692] Updated weights for policy 0, policy_version 333378 (0.0008) [2023-12-26 17:47:26,003][105620] Updated weights for policy 1, policy_version 333648 (0.0009) [2023-12-26 17:47:26,059][105620] Updated weights for policy 1, policy_version 333658 (0.0008) [2023-12-26 17:47:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.6, 300 sec: 19494.2). Total num frames: 170778624. Throughput: 0: 9553.8, 1: 9603.9. Samples: 170790688. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 17:47:26,063][104569] Avg episode reward: [(0, '9355.521'), (1, '9356.862')] [2023-12-26 17:47:26,106][105620] Updated weights for policy 1, policy_version 333668 (0.0009) [2023-12-26 17:47:26,457][105692] Updated weights for policy 0, policy_version 333388 (0.0009) [2023-12-26 17:47:26,511][105692] Updated weights for policy 0, policy_version 333398 (0.0009) [2023-12-26 17:47:26,568][105692] Updated weights for policy 0, policy_version 333408 (0.0009) [2023-12-26 17:47:26,786][105620] Updated weights for policy 1, policy_version 333678 (0.0006) [2023-12-26 17:47:26,832][105620] Updated weights for policy 1, policy_version 333688 (0.0006) [2023-12-26 17:47:26,884][105620] Updated weights for policy 1, policy_version 333698 (0.0009) [2023-12-26 17:47:27,471][105620] Updated weights for policy 1, policy_version 333708 (0.0007) [2023-12-26 17:47:27,472][105692] Updated weights for policy 0, policy_version 333418 (0.0012) [2023-12-26 17:47:27,526][105692] Updated weights for policy 0, policy_version 333428 (0.0008) [2023-12-26 17:47:27,536][105620] Updated weights for policy 1, policy_version 333718 (0.0006) [2023-12-26 17:47:27,573][105692] Updated weights for policy 0, policy_version 333438 (0.0008) [2023-12-26 17:47:27,587][105620] Updated weights for policy 1, policy_version 333728 (0.0010) [2023-12-26 17:47:27,613][105692] Updated weights for policy 0, policy_version 333448 (0.0005) [2023-12-26 17:47:28,195][105620] Updated weights for policy 1, policy_version 333738 (0.0009) [2023-12-26 17:47:28,246][105620] Updated weights for policy 1, policy_version 333748 (0.0005) [2023-12-26 17:47:28,299][105620] Updated weights for policy 1, policy_version 333758 (0.0005) [2023-12-26 17:47:28,358][105692] Updated weights for policy 0, policy_version 333458 (0.0007) [2023-12-26 17:47:28,365][105620] Updated weights for policy 1, policy_version 333768 (0.0008) [2023-12-26 17:47:28,413][105692] Updated weights for policy 0, policy_version 333468 (0.0010) [2023-12-26 17:47:28,476][105692] Updated weights for policy 0, policy_version 333478 (0.0009) [2023-12-26 17:47:28,916][105620] Updated weights for policy 1, policy_version 333778 (0.0005) [2023-12-26 17:47:28,968][105620] Updated weights for policy 1, policy_version 333788 (0.0007) [2023-12-26 17:47:29,028][105620] Updated weights for policy 1, policy_version 333798 (0.0005) [2023-12-26 17:47:29,383][105692] Updated weights for policy 0, policy_version 333488 (0.0009) [2023-12-26 17:47:29,439][105692] Updated weights for policy 0, policy_version 333498 (0.0009) [2023-12-26 17:47:29,499][105692] Updated weights for policy 0, policy_version 333508 (0.0009) [2023-12-26 17:47:29,672][105620] Updated weights for policy 1, policy_version 333808 (0.0008) [2023-12-26 17:47:29,726][105620] Updated weights for policy 1, policy_version 333818 (0.0009) [2023-12-26 17:47:29,784][105620] Updated weights for policy 1, policy_version 333828 (0.0009) [2023-12-26 17:47:30,264][105692] Updated weights for policy 0, policy_version 333518 (0.0009) [2023-12-26 17:47:30,322][105692] Updated weights for policy 0, policy_version 333528 (0.0006) [2023-12-26 17:47:30,384][105692] Updated weights for policy 0, policy_version 333538 (0.0009) [2023-12-26 17:47:30,520][105620] Updated weights for policy 1, policy_version 333838 (0.0009) [2023-12-26 17:47:30,566][105620] Updated weights for policy 1, policy_version 333848 (0.0008) [2023-12-26 17:47:30,616][105620] Updated weights for policy 1, policy_version 333858 (0.0009) [2023-12-26 17:47:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 170876928. Throughput: 0: 9569.5, 1: 9657.4. Samples: 170850832. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 17:47:31,062][104569] Avg episode reward: [(0, '9087.565'), (1, '9356.631')] [2023-12-26 17:47:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000333544_85401600.pth... [2023-12-26 17:47:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000333864_85475328.pth... [2023-12-26 17:47:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000332744_85188608.pth [2023-12-26 17:47:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000332424_85114880.pth [2023-12-26 17:47:31,148][105692] Updated weights for policy 0, policy_version 333548 (0.0007) [2023-12-26 17:47:31,200][105692] Updated weights for policy 0, policy_version 333558 (0.0009) [2023-12-26 17:47:31,255][105692] Updated weights for policy 0, policy_version 333568 (0.0009) [2023-12-26 17:47:31,298][105620] Updated weights for policy 1, policy_version 333868 (0.0008) [2023-12-26 17:47:31,357][105620] Updated weights for policy 1, policy_version 333878 (0.0008) [2023-12-26 17:47:31,418][105620] Updated weights for policy 1, policy_version 333888 (0.0008) [2023-12-26 17:47:32,057][105620] Updated weights for policy 1, policy_version 333898 (0.0009) [2023-12-26 17:47:32,086][105692] Updated weights for policy 0, policy_version 333578 (0.0008) [2023-12-26 17:47:32,112][105620] Updated weights for policy 1, policy_version 333908 (0.0007) [2023-12-26 17:47:32,144][105692] Updated weights for policy 0, policy_version 333588 (0.0007) [2023-12-26 17:47:32,175][105620] Updated weights for policy 1, policy_version 333918 (0.0008) [2023-12-26 17:47:32,192][105692] Updated weights for policy 0, policy_version 333598 (0.0007) [2023-12-26 17:47:32,241][105620] Updated weights for policy 1, policy_version 333928 (0.0006) [2023-12-26 17:47:32,244][105692] Updated weights for policy 0, policy_version 333608 (0.0006) [2023-12-26 17:47:32,963][105620] Updated weights for policy 1, policy_version 333938 (0.0008) [2023-12-26 17:47:33,005][105692] Updated weights for policy 0, policy_version 333618 (0.0007) [2023-12-26 17:47:33,018][105620] Updated weights for policy 1, policy_version 333948 (0.0008) [2023-12-26 17:47:33,065][105692] Updated weights for policy 0, policy_version 333628 (0.0006) [2023-12-26 17:47:33,078][105620] Updated weights for policy 1, policy_version 333958 (0.0008) [2023-12-26 17:47:33,124][105692] Updated weights for policy 0, policy_version 333638 (0.0008) [2023-12-26 17:47:33,698][105620] Updated weights for policy 1, policy_version 333968 (0.0006) [2023-12-26 17:47:33,760][105620] Updated weights for policy 1, policy_version 333978 (0.0008) [2023-12-26 17:47:33,818][105620] Updated weights for policy 1, policy_version 333988 (0.0008) [2023-12-26 17:47:33,939][105692] Updated weights for policy 0, policy_version 333648 (0.0010) [2023-12-26 17:47:33,985][105692] Updated weights for policy 0, policy_version 333658 (0.0008) [2023-12-26 17:47:34,039][105692] Updated weights for policy 0, policy_version 333668 (0.0009) [2023-12-26 17:47:34,537][105620] Updated weights for policy 1, policy_version 333998 (0.0009) [2023-12-26 17:47:34,590][105620] Updated weights for policy 1, policy_version 334008 (0.0008) [2023-12-26 17:47:34,645][105620] Updated weights for policy 1, policy_version 334018 (0.0009) [2023-12-26 17:47:34,813][105692] Updated weights for policy 0, policy_version 333678 (0.0008) [2023-12-26 17:47:34,872][105692] Updated weights for policy 0, policy_version 333688 (0.0009) [2023-12-26 17:47:34,930][105692] Updated weights for policy 0, policy_version 333698 (0.0009) [2023-12-26 17:47:35,434][105620] Updated weights for policy 1, policy_version 334028 (0.0008) [2023-12-26 17:47:35,488][105620] Updated weights for policy 1, policy_version 334038 (0.0009) [2023-12-26 17:47:35,545][105620] Updated weights for policy 1, policy_version 334048 (0.0009) [2023-12-26 17:47:35,670][105692] Updated weights for policy 0, policy_version 333708 (0.0009) [2023-12-26 17:47:35,725][105692] Updated weights for policy 0, policy_version 333718 (0.0009) [2023-12-26 17:47:35,784][105692] Updated weights for policy 0, policy_version 333728 (0.0009) [2023-12-26 17:47:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 170975232. Throughput: 0: 9561.6, 1: 9724.7. Samples: 170964488. Policy #0 lag: (min: 1.0, avg: 19.3, max: 33.0) [2023-12-26 17:47:36,062][104569] Avg episode reward: [(0, '9088.153'), (1, '9357.215')] [2023-12-26 17:47:36,307][105620] Updated weights for policy 1, policy_version 334058 (0.0008) [2023-12-26 17:47:36,369][105620] Updated weights for policy 1, policy_version 334068 (0.0009) [2023-12-26 17:47:36,431][105620] Updated weights for policy 1, policy_version 334078 (0.0009) [2023-12-26 17:47:36,490][105620] Updated weights for policy 1, policy_version 334088 (0.0009) [2023-12-26 17:47:36,555][105692] Updated weights for policy 0, policy_version 333738 (0.0009) [2023-12-26 17:47:36,603][105692] Updated weights for policy 0, policy_version 333748 (0.0009) [2023-12-26 17:47:36,651][105692] Updated weights for policy 0, policy_version 333758 (0.0008) [2023-12-26 17:47:36,700][105692] Updated weights for policy 0, policy_version 333768 (0.0009) [2023-12-26 17:47:37,246][105620] Updated weights for policy 1, policy_version 334098 (0.0009) [2023-12-26 17:47:37,302][105620] Updated weights for policy 1, policy_version 334108 (0.0009) [2023-12-26 17:47:37,363][105620] Updated weights for policy 1, policy_version 334118 (0.0009) [2023-12-26 17:47:37,461][105692] Updated weights for policy 0, policy_version 333778 (0.0005) [2023-12-26 17:47:37,524][105692] Updated weights for policy 0, policy_version 333788 (0.0007) [2023-12-26 17:47:37,584][105692] Updated weights for policy 0, policy_version 333798 (0.0008) [2023-12-26 17:47:38,181][105620] Updated weights for policy 1, policy_version 334128 (0.0009) [2023-12-26 17:47:38,218][105692] Updated weights for policy 0, policy_version 333808 (0.0006) [2023-12-26 17:47:38,229][105620] Updated weights for policy 1, policy_version 334138 (0.0008) [2023-12-26 17:47:38,261][105692] Updated weights for policy 0, policy_version 333818 (0.0006) [2023-12-26 17:47:38,278][105620] Updated weights for policy 1, policy_version 334148 (0.0009) [2023-12-26 17:47:38,311][105692] Updated weights for policy 0, policy_version 333828 (0.0009) [2023-12-26 17:47:39,039][105620] Updated weights for policy 1, policy_version 334158 (0.0006) [2023-12-26 17:47:39,054][105692] Updated weights for policy 0, policy_version 333838 (0.0008) [2023-12-26 17:47:39,095][105620] Updated weights for policy 1, policy_version 334168 (0.0006) [2023-12-26 17:47:39,114][105692] Updated weights for policy 0, policy_version 333848 (0.0007) [2023-12-26 17:47:39,145][105620] Updated weights for policy 1, policy_version 334178 (0.0007) [2023-12-26 17:47:39,172][105692] Updated weights for policy 0, policy_version 333858 (0.0006) [2023-12-26 17:47:39,876][105692] Updated weights for policy 0, policy_version 333868 (0.0006) [2023-12-26 17:47:39,924][105620] Updated weights for policy 1, policy_version 334188 (0.0008) [2023-12-26 17:47:39,942][105692] Updated weights for policy 0, policy_version 333878 (0.0008) [2023-12-26 17:47:39,992][105620] Updated weights for policy 1, policy_version 334198 (0.0006) [2023-12-26 17:47:40,002][105692] Updated weights for policy 0, policy_version 333888 (0.0009) [2023-12-26 17:47:40,064][105620] Updated weights for policy 1, policy_version 334208 (0.0007) [2023-12-26 17:47:40,636][105620] Updated weights for policy 1, policy_version 334218 (0.0008) [2023-12-26 17:47:40,703][105620] Updated weights for policy 1, policy_version 334228 (0.0007) [2023-12-26 17:47:40,758][105620] Updated weights for policy 1, policy_version 334238 (0.0008) [2023-12-26 17:47:40,813][105620] Updated weights for policy 1, policy_version 334248 (0.0006) [2023-12-26 17:47:40,848][105692] Updated weights for policy 0, policy_version 333898 (0.0006) [2023-12-26 17:47:40,898][105692] Updated weights for policy 0, policy_version 333908 (0.0005) [2023-12-26 17:47:40,959][105692] Updated weights for policy 0, policy_version 333918 (0.0007) [2023-12-26 17:47:41,006][105692] Updated weights for policy 0, policy_version 333928 (0.0007) [2023-12-26 17:47:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 171073536. Throughput: 0: 9584.5, 1: 9728.1. Samples: 171078404. Policy #0 lag: (min: 1.0, avg: 19.3, max: 33.0) [2023-12-26 17:47:41,062][104569] Avg episode reward: [(0, '8997.665'), (1, '9278.493')] [2023-12-26 17:47:41,519][105620] Updated weights for policy 1, policy_version 334258 (0.0009) [2023-12-26 17:47:41,551][105586] KL-divergence is very high: 107.0754 [2023-12-26 17:47:41,586][105620] Updated weights for policy 1, policy_version 334268 (0.0008) [2023-12-26 17:47:41,655][105620] Updated weights for policy 1, policy_version 334278 (0.0007) [2023-12-26 17:47:41,771][105692] Updated weights for policy 0, policy_version 333938 (0.0009) [2023-12-26 17:47:41,824][105692] Updated weights for policy 0, policy_version 333948 (0.0009) [2023-12-26 17:47:41,885][105692] Updated weights for policy 0, policy_version 333958 (0.0009) [2023-12-26 17:47:42,393][105620] Updated weights for policy 1, policy_version 334288 (0.0008) [2023-12-26 17:47:42,456][105620] Updated weights for policy 1, policy_version 334298 (0.0009) [2023-12-26 17:47:42,517][105620] Updated weights for policy 1, policy_version 334308 (0.0009) [2023-12-26 17:47:42,727][105692] Updated weights for policy 0, policy_version 333968 (0.0009) [2023-12-26 17:47:42,785][105692] Updated weights for policy 0, policy_version 333978 (0.0011) [2023-12-26 17:47:42,844][105692] Updated weights for policy 0, policy_version 333988 (0.0010) [2023-12-26 17:47:43,304][105620] Updated weights for policy 1, policy_version 334318 (0.0008) [2023-12-26 17:47:43,360][105586] KL-divergence is very high: 121.2659 [2023-12-26 17:47:43,361][105620] Updated weights for policy 1, policy_version 334328 (0.0008) [2023-12-26 17:47:43,417][105620] Updated weights for policy 1, policy_version 334338 (0.0008) [2023-12-26 17:47:43,496][105692] Updated weights for policy 0, policy_version 333998 (0.0011) [2023-12-26 17:47:43,555][105692] Updated weights for policy 0, policy_version 334008 (0.0010) [2023-12-26 17:47:43,611][105692] Updated weights for policy 0, policy_version 334018 (0.0010) [2023-12-26 17:47:44,016][105620] Updated weights for policy 1, policy_version 334348 (0.0007) [2023-12-26 17:47:44,080][105620] Updated weights for policy 1, policy_version 334358 (0.0006) [2023-12-26 17:47:44,140][105620] Updated weights for policy 1, policy_version 334368 (0.0008) [2023-12-26 17:47:44,140][105586] KL-divergence is very high: 102.1857 [2023-12-26 17:47:44,287][105692] Updated weights for policy 0, policy_version 334028 (0.0010) [2023-12-26 17:47:44,334][105692] Updated weights for policy 0, policy_version 334038 (0.0010) [2023-12-26 17:47:44,381][105692] Updated weights for policy 0, policy_version 334048 (0.0010) [2023-12-26 17:47:44,756][105620] Updated weights for policy 1, policy_version 334378 (0.0009) [2023-12-26 17:47:44,817][105620] Updated weights for policy 1, policy_version 334388 (0.0008) [2023-12-26 17:47:44,878][105620] Updated weights for policy 1, policy_version 334398 (0.0008) [2023-12-26 17:47:44,938][105620] Updated weights for policy 1, policy_version 334408 (0.0011) [2023-12-26 17:47:45,157][105692] Updated weights for policy 0, policy_version 334058 (0.0010) [2023-12-26 17:47:45,220][105692] Updated weights for policy 0, policy_version 334068 (0.0010) [2023-12-26 17:47:45,289][105692] Updated weights for policy 0, policy_version 334078 (0.0006) [2023-12-26 17:47:45,355][105692] Updated weights for policy 0, policy_version 334088 (0.0007) [2023-12-26 17:47:45,675][105620] Updated weights for policy 1, policy_version 334418 (0.0009) [2023-12-26 17:47:45,734][105620] Updated weights for policy 1, policy_version 334428 (0.0011) [2023-12-26 17:47:45,793][105620] Updated weights for policy 1, policy_version 334438 (0.0010) [2023-12-26 17:47:45,799][105586] KL-divergence is very high: 103.2077 [2023-12-26 17:47:46,018][105692] Updated weights for policy 0, policy_version 334098 (0.0005) [2023-12-26 17:47:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19114.6, 300 sec: 19466.4). Total num frames: 171163648. Throughput: 0: 9586.2, 1: 9687.2. Samples: 171135148. Policy #0 lag: (min: 1.0, avg: 19.3, max: 33.0) [2023-12-26 17:47:46,062][104569] Avg episode reward: [(0, '8638.941'), (1, '1412.563')] [2023-12-26 17:47:46,068][105692] Updated weights for policy 0, policy_version 334108 (0.0008) [2023-12-26 17:47:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000334440_85622784.pth... [2023-12-26 17:47:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000333288_85327872.pth [2023-12-26 17:47:46,123][105692] Updated weights for policy 0, policy_version 334118 (0.0008) [2023-12-26 17:47:46,130][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000334120_85549056.pth... [2023-12-26 17:47:46,132][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000333000_85262336.pth [2023-12-26 17:47:46,496][105620] Updated weights for policy 1, policy_version 334448 (0.0010) [2023-12-26 17:47:46,551][105620] Updated weights for policy 1, policy_version 334458 (0.0010) [2023-12-26 17:47:46,605][105620] Updated weights for policy 1, policy_version 334468 (0.0010) [2023-12-26 17:47:46,776][105692] Updated weights for policy 0, policy_version 334128 (0.0006) [2023-12-26 17:47:46,840][105692] Updated weights for policy 0, policy_version 334138 (0.0005) [2023-12-26 17:47:46,904][105692] Updated weights for policy 0, policy_version 334148 (0.0006) [2023-12-26 17:47:47,213][105620] Updated weights for policy 1, policy_version 334478 (0.0007) [2023-12-26 17:47:47,270][105620] Updated weights for policy 1, policy_version 334488 (0.0006) [2023-12-26 17:47:47,326][105620] Updated weights for policy 1, policy_version 334498 (0.0010) [2023-12-26 17:47:47,356][105586] KL-divergence is very high: 121.9220 [2023-12-26 17:47:47,573][105692] Updated weights for policy 0, policy_version 334158 (0.0011) [2023-12-26 17:47:47,635][105692] Updated weights for policy 0, policy_version 334168 (0.0011) [2023-12-26 17:47:47,701][105692] Updated weights for policy 0, policy_version 334178 (0.0011) [2023-12-26 17:47:47,963][105586] KL-divergence is very high: 147.5997 [2023-12-26 17:47:47,969][105586] KL-divergence is very high: 136.0225 [2023-12-26 17:47:47,983][105620] Updated weights for policy 1, policy_version 334508 (0.0010) [2023-12-26 17:47:48,041][105620] Updated weights for policy 1, policy_version 334519 (0.0010) [2023-12-26 17:47:48,100][105620] Updated weights for policy 1, policy_version 334529 (0.0008) [2023-12-26 17:47:48,411][105692] Updated weights for policy 0, policy_version 334188 (0.0010) [2023-12-26 17:47:48,477][105692] Updated weights for policy 0, policy_version 334198 (0.0011) [2023-12-26 17:47:48,534][105692] Updated weights for policy 0, policy_version 334208 (0.0011) [2023-12-26 17:47:48,802][105620] Updated weights for policy 1, policy_version 334539 (0.0008) [2023-12-26 17:47:48,857][105620] Updated weights for policy 1, policy_version 334549 (0.0008) [2023-12-26 17:47:48,914][105620] Updated weights for policy 1, policy_version 334559 (0.0008) [2023-12-26 17:47:49,285][105692] Updated weights for policy 0, policy_version 334218 (0.0010) [2023-12-26 17:47:49,351][105692] Updated weights for policy 0, policy_version 334228 (0.0008) [2023-12-26 17:47:49,413][105692] Updated weights for policy 0, policy_version 334238 (0.0007) [2023-12-26 17:47:49,466][105692] Updated weights for policy 0, policy_version 334248 (0.0009) [2023-12-26 17:47:49,588][105620] Updated weights for policy 1, policy_version 334569 (0.0007) [2023-12-26 17:47:49,632][105586] KL-divergence is very high: 117.2183 [2023-12-26 17:47:49,637][105620] Updated weights for policy 1, policy_version 334579 (0.0008) [2023-12-26 17:47:49,690][105620] Updated weights for policy 1, policy_version 334589 (0.0009) [2023-12-26 17:47:49,744][105620] Updated weights for policy 1, policy_version 334599 (0.0009) [2023-12-26 17:47:50,131][105692] Updated weights for policy 0, policy_version 334258 (0.0005) [2023-12-26 17:47:50,183][105692] Updated weights for policy 0, policy_version 334268 (0.0005) [2023-12-26 17:47:50,241][105692] Updated weights for policy 0, policy_version 334278 (0.0005) [2023-12-26 17:47:50,577][105620] Updated weights for policy 1, policy_version 334609 (0.0011) [2023-12-26 17:47:50,612][105586] KL-divergence is very high: 103.7312 [2023-12-26 17:47:50,644][105620] Updated weights for policy 1, policy_version 334619 (0.0010) [2023-12-26 17:47:50,706][105620] Updated weights for policy 1, policy_version 334629 (0.0007) [2023-12-26 17:47:50,819][105692] Updated weights for policy 0, policy_version 334288 (0.0008) [2023-12-26 17:47:50,893][105692] Updated weights for policy 0, policy_version 334298 (0.0008) [2023-12-26 17:47:50,961][105692] Updated weights for policy 0, policy_version 334308 (0.0009) [2023-12-26 17:47:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19438.7). Total num frames: 171270144. Throughput: 0: 9507.5, 1: 9774.3. Samples: 171255324. Policy #0 lag: (min: 1.0, avg: 19.3, max: 33.0) [2023-12-26 17:47:51,063][104569] Avg episode reward: [(0, '8547.656'), (1, '2178.913')] [2023-12-26 17:47:51,343][105620] Updated weights for policy 1, policy_version 334639 (0.0009) [2023-12-26 17:47:51,418][105620] Updated weights for policy 1, policy_version 334649 (0.0009) [2023-12-26 17:47:51,479][105620] Updated weights for policy 1, policy_version 334659 (0.0010) [2023-12-26 17:47:51,651][105692] Updated weights for policy 0, policy_version 334318 (0.0009) [2023-12-26 17:47:51,715][105692] Updated weights for policy 0, policy_version 334328 (0.0009) [2023-12-26 17:47:51,783][105692] Updated weights for policy 0, policy_version 334338 (0.0009) [2023-12-26 17:47:52,230][105620] Updated weights for policy 1, policy_version 334669 (0.0008) [2023-12-26 17:47:52,297][105620] Updated weights for policy 1, policy_version 334679 (0.0006) [2023-12-26 17:47:52,368][105620] Updated weights for policy 1, policy_version 334689 (0.0007) [2023-12-26 17:47:52,407][105692] Updated weights for policy 0, policy_version 334348 (0.0009) [2023-12-26 17:47:52,459][105692] Updated weights for policy 0, policy_version 334358 (0.0010) [2023-12-26 17:47:52,511][105692] Updated weights for policy 0, policy_version 334368 (0.0009) [2023-12-26 17:47:52,957][105620] Updated weights for policy 1, policy_version 334699 (0.0006) [2023-12-26 17:47:53,019][105620] Updated weights for policy 1, policy_version 334709 (0.0005) [2023-12-26 17:47:53,076][105620] Updated weights for policy 1, policy_version 334719 (0.0006) [2023-12-26 17:47:53,261][105692] Updated weights for policy 0, policy_version 334378 (0.0008) [2023-12-26 17:47:53,327][105692] Updated weights for policy 0, policy_version 334388 (0.0005) [2023-12-26 17:47:53,388][105692] Updated weights for policy 0, policy_version 334398 (0.0005) [2023-12-26 17:47:53,457][105692] Updated weights for policy 0, policy_version 334408 (0.0005) [2023-12-26 17:47:53,582][105620] Updated weights for policy 1, policy_version 334729 (0.0005) [2023-12-26 17:47:53,633][105620] Updated weights for policy 1, policy_version 334739 (0.0005) [2023-12-26 17:47:53,685][105620] Updated weights for policy 1, policy_version 334749 (0.0005) [2023-12-26 17:47:53,740][105620] Updated weights for policy 1, policy_version 334759 (0.0005) [2023-12-26 17:47:53,995][105692] Updated weights for policy 0, policy_version 334418 (0.0005) [2023-12-26 17:47:54,044][105692] Updated weights for policy 0, policy_version 334428 (0.0005) [2023-12-26 17:47:54,099][105692] Updated weights for policy 0, policy_version 334438 (0.0005) [2023-12-26 17:47:54,287][105620] Updated weights for policy 1, policy_version 334769 (0.0009) [2023-12-26 17:47:54,351][105620] Updated weights for policy 1, policy_version 334779 (0.0009) [2023-12-26 17:47:54,411][105620] Updated weights for policy 1, policy_version 334789 (0.0007) [2023-12-26 17:47:54,742][105692] Updated weights for policy 0, policy_version 334448 (0.0005) [2023-12-26 17:47:54,790][105692] Updated weights for policy 0, policy_version 334458 (0.0006) [2023-12-26 17:47:54,847][105692] Updated weights for policy 0, policy_version 334468 (0.0008) [2023-12-26 17:47:54,985][105620] Updated weights for policy 1, policy_version 334799 (0.0006) [2023-12-26 17:47:55,054][105620] Updated weights for policy 1, policy_version 334809 (0.0010) [2023-12-26 17:47:55,120][105620] Updated weights for policy 1, policy_version 334819 (0.0011) [2023-12-26 17:47:55,535][105692] Updated weights for policy 0, policy_version 334479 (0.0009) [2023-12-26 17:47:55,540][105585] KL-divergence is very high: 100.0542 [2023-12-26 17:47:55,549][105585] KL-divergence is very high: 140.2039 [2023-12-26 17:47:55,591][105692] Updated weights for policy 0, policy_version 334490 (0.0010) [2023-12-26 17:47:55,592][105585] KL-divergence is very high: 193.9230 [2023-12-26 17:47:55,630][105585] KL-divergence is very high: 265.4726 [2023-12-26 17:47:55,636][105585] KL-divergence is very high: 127.9277 [2023-12-26 17:47:55,643][105585] KL-divergence is very high: 137.0976 [2023-12-26 17:47:55,644][105692] Updated weights for policy 0, policy_version 334501 (0.0011) [2023-12-26 17:47:55,717][105620] Updated weights for policy 1, policy_version 334829 (0.0008) [2023-12-26 17:47:55,782][105620] Updated weights for policy 1, policy_version 334839 (0.0005) [2023-12-26 17:47:55,844][105620] Updated weights for policy 1, policy_version 334849 (0.0008) [2023-12-26 17:47:56,062][104569] Fps is (10 sec: 21299.5, 60 sec: 19660.9, 300 sec: 19494.2). Total num frames: 171376640. Throughput: 0: 9675.1, 1: 9933.8. Samples: 171384856. Policy #0 lag: (min: 1.0, avg: 19.3, max: 33.0) [2023-12-26 17:47:56,062][104569] Avg episode reward: [(0, '8759.509'), (1, '3504.011')] [2023-12-26 17:47:56,374][105692] Updated weights for policy 0, policy_version 334511 (0.0009) [2023-12-26 17:47:56,441][105692] Updated weights for policy 0, policy_version 334521 (0.0006) [2023-12-26 17:47:56,483][105585] KL-divergence is very high: 149.2939 [2023-12-26 17:47:56,510][105692] Updated weights for policy 0, policy_version 334531 (0.0005) [2023-12-26 17:47:56,520][105620] Updated weights for policy 1, policy_version 334859 (0.0009) [2023-12-26 17:47:56,538][105585] KL-divergence is very high: 138.2216 [2023-12-26 17:47:56,586][105620] Updated weights for policy 1, policy_version 334869 (0.0008) [2023-12-26 17:47:56,645][105620] Updated weights for policy 1, policy_version 334879 (0.0010) [2023-12-26 17:47:57,092][105692] Updated weights for policy 0, policy_version 334541 (0.0008) [2023-12-26 17:47:57,149][105692] Updated weights for policy 0, policy_version 334551 (0.0010) [2023-12-26 17:47:57,200][105692] Updated weights for policy 0, policy_version 334561 (0.0010) [2023-12-26 17:47:57,397][105620] Updated weights for policy 1, policy_version 334889 (0.0010) [2023-12-26 17:47:57,453][105620] Updated weights for policy 1, policy_version 334899 (0.0008) [2023-12-26 17:47:57,504][105620] Updated weights for policy 1, policy_version 334909 (0.0008) [2023-12-26 17:47:57,557][105620] Updated weights for policy 1, policy_version 334919 (0.0005) [2023-12-26 17:47:57,920][105692] Updated weights for policy 0, policy_version 334571 (0.0010) [2023-12-26 17:47:57,982][105692] Updated weights for policy 0, policy_version 334581 (0.0009) [2023-12-26 17:47:58,036][105692] Updated weights for policy 0, policy_version 334591 (0.0007) [2023-12-26 17:47:58,282][105620] Updated weights for policy 1, policy_version 334929 (0.0007) [2023-12-26 17:47:58,357][105620] Updated weights for policy 1, policy_version 334939 (0.0008) [2023-12-26 17:47:58,422][105620] Updated weights for policy 1, policy_version 334949 (0.0009) [2023-12-26 17:47:58,868][105692] Updated weights for policy 0, policy_version 334601 (0.0005) [2023-12-26 17:47:58,934][105692] Updated weights for policy 0, policy_version 334611 (0.0007) [2023-12-26 17:47:59,000][105692] Updated weights for policy 0, policy_version 334621 (0.0008) [2023-12-26 17:47:59,066][105692] Updated weights for policy 0, policy_version 334631 (0.0008) [2023-12-26 17:47:59,188][105620] Updated weights for policy 1, policy_version 334959 (0.0007) [2023-12-26 17:47:59,251][105620] Updated weights for policy 1, policy_version 334969 (0.0007) [2023-12-26 17:47:59,304][105620] Updated weights for policy 1, policy_version 334979 (0.0008) [2023-12-26 17:47:59,782][105692] Updated weights for policy 0, policy_version 334641 (0.0010) [2023-12-26 17:47:59,844][105692] Updated weights for policy 0, policy_version 334651 (0.0009) [2023-12-26 17:47:59,905][105692] Updated weights for policy 0, policy_version 334661 (0.0007) [2023-12-26 17:48:00,065][105620] Updated weights for policy 1, policy_version 334989 (0.0007) [2023-12-26 17:48:00,128][105620] Updated weights for policy 1, policy_version 334999 (0.0006) [2023-12-26 17:48:00,190][105620] Updated weights for policy 1, policy_version 335009 (0.0006) [2023-12-26 17:48:00,689][105692] Updated weights for policy 0, policy_version 334671 (0.0009) [2023-12-26 17:48:00,739][105692] Updated weights for policy 0, policy_version 334681 (0.0008) [2023-12-26 17:48:00,803][105692] Updated weights for policy 0, policy_version 334691 (0.0010) [2023-12-26 17:48:00,817][105620] Updated weights for policy 1, policy_version 335019 (0.0006) [2023-12-26 17:48:00,879][105620] Updated weights for policy 1, policy_version 335029 (0.0005) [2023-12-26 17:48:00,938][105620] Updated weights for policy 1, policy_version 335039 (0.0005) [2023-12-26 17:48:01,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 171474944. Throughput: 0: 9722.8, 1: 9926.7. Samples: 171442492. Policy #0 lag: (min: 1.0, avg: 19.3, max: 33.0) [2023-12-26 17:48:01,062][104569] Avg episode reward: [(0, '8412.150'), (1, '7236.071')] [2023-12-26 17:48:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000334696_85696512.pth... [2023-12-26 17:48:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000335048_85778432.pth... [2023-12-26 17:48:01,069][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000333864_85475328.pth [2023-12-26 17:48:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000333544_85401600.pth [2023-12-26 17:48:01,597][105692] Updated weights for policy 0, policy_version 334701 (0.0009) [2023-12-26 17:48:01,614][105620] Updated weights for policy 1, policy_version 335049 (0.0006) [2023-12-26 17:48:01,662][105692] Updated weights for policy 0, policy_version 334711 (0.0008) [2023-12-26 17:48:01,680][105620] Updated weights for policy 1, policy_version 335059 (0.0008) [2023-12-26 17:48:01,715][105692] Updated weights for policy 0, policy_version 334721 (0.0006) [2023-12-26 17:48:01,742][105620] Updated weights for policy 1, policy_version 335069 (0.0008) [2023-12-26 17:48:01,791][105620] Updated weights for policy 1, policy_version 335079 (0.0009) [2023-12-26 17:48:02,499][105692] Updated weights for policy 0, policy_version 334731 (0.0008) [2023-12-26 17:48:02,547][105620] Updated weights for policy 1, policy_version 335089 (0.0008) [2023-12-26 17:48:02,552][105692] Updated weights for policy 0, policy_version 334741 (0.0009) [2023-12-26 17:48:02,606][105692] Updated weights for policy 0, policy_version 334751 (0.0005) [2023-12-26 17:48:02,610][105620] Updated weights for policy 1, policy_version 335099 (0.0008) [2023-12-26 17:48:02,664][105620] Updated weights for policy 1, policy_version 335109 (0.0008) [2023-12-26 17:48:03,308][105692] Updated weights for policy 0, policy_version 334761 (0.0005) [2023-12-26 17:48:03,320][105620] Updated weights for policy 1, policy_version 335119 (0.0007) [2023-12-26 17:48:03,368][105620] Updated weights for policy 1, policy_version 335129 (0.0005) [2023-12-26 17:48:03,369][105692] Updated weights for policy 0, policy_version 334771 (0.0005) [2023-12-26 17:48:03,419][105620] Updated weights for policy 1, policy_version 335139 (0.0005) [2023-12-26 17:48:03,427][105692] Updated weights for policy 0, policy_version 334781 (0.0007) [2023-12-26 17:48:03,476][105692] Updated weights for policy 0, policy_version 334791 (0.0009) [2023-12-26 17:48:04,077][105620] Updated weights for policy 1, policy_version 335149 (0.0007) [2023-12-26 17:48:04,139][105620] Updated weights for policy 1, policy_version 335159 (0.0009) [2023-12-26 17:48:04,197][105692] Updated weights for policy 0, policy_version 334801 (0.0011) [2023-12-26 17:48:04,199][105620] Updated weights for policy 1, policy_version 335169 (0.0006) [2023-12-26 17:48:04,250][105692] Updated weights for policy 0, policy_version 334811 (0.0010) [2023-12-26 17:48:04,306][105692] Updated weights for policy 0, policy_version 334821 (0.0011) [2023-12-26 17:48:04,900][105620] Updated weights for policy 1, policy_version 335179 (0.0007) [2023-12-26 17:48:04,960][105620] Updated weights for policy 1, policy_version 335189 (0.0008) [2023-12-26 17:48:05,020][105620] Updated weights for policy 1, policy_version 335199 (0.0006) [2023-12-26 17:48:05,029][105692] Updated weights for policy 0, policy_version 334831 (0.0010) [2023-12-26 17:48:05,094][105692] Updated weights for policy 0, policy_version 334841 (0.0010) [2023-12-26 17:48:05,160][105692] Updated weights for policy 0, policy_version 334851 (0.0011) [2023-12-26 17:48:05,623][105620] Updated weights for policy 1, policy_version 335209 (0.0007) [2023-12-26 17:48:05,673][105620] Updated weights for policy 1, policy_version 335219 (0.0010) [2023-12-26 17:48:05,721][105620] Updated weights for policy 1, policy_version 335229 (0.0010) [2023-12-26 17:48:05,768][105620] Updated weights for policy 1, policy_version 335239 (0.0010) [2023-12-26 17:48:05,883][105692] Updated weights for policy 0, policy_version 334861 (0.0011) [2023-12-26 17:48:05,947][105692] Updated weights for policy 0, policy_version 334871 (0.0009) [2023-12-26 17:48:06,009][105692] Updated weights for policy 0, policy_version 334881 (0.0010) [2023-12-26 17:48:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 171573248. Throughput: 0: 9556.3, 1: 10026.6. Samples: 171558612. Policy #0 lag: (min: 1.0, avg: 19.3, max: 33.0) [2023-12-26 17:48:06,062][104569] Avg episode reward: [(0, '8649.071'), (1, '9021.064')] [2023-12-26 17:48:06,384][105620] Updated weights for policy 1, policy_version 335249 (0.0006) [2023-12-26 17:48:06,437][105620] Updated weights for policy 1, policy_version 335259 (0.0005) [2023-12-26 17:48:06,490][105620] Updated weights for policy 1, policy_version 335269 (0.0005) [2023-12-26 17:48:06,752][105692] Updated weights for policy 0, policy_version 334891 (0.0010) [2023-12-26 17:48:06,808][105692] Updated weights for policy 0, policy_version 334901 (0.0011) [2023-12-26 17:48:06,871][105692] Updated weights for policy 0, policy_version 334911 (0.0011) [2023-12-26 17:48:07,125][105620] Updated weights for policy 1, policy_version 335279 (0.0006) [2023-12-26 17:48:07,186][105620] Updated weights for policy 1, policy_version 335289 (0.0006) [2023-12-26 17:48:07,246][105620] Updated weights for policy 1, policy_version 335299 (0.0008) [2023-12-26 17:48:07,624][105692] Updated weights for policy 0, policy_version 334921 (0.0010) [2023-12-26 17:48:07,672][105692] Updated weights for policy 0, policy_version 334931 (0.0010) [2023-12-26 17:48:07,720][105692] Updated weights for policy 0, policy_version 334941 (0.0010) [2023-12-26 17:48:07,767][105692] Updated weights for policy 0, policy_version 334951 (0.0010) [2023-12-26 17:48:07,953][105620] Updated weights for policy 1, policy_version 335309 (0.0008) [2023-12-26 17:48:08,016][105620] Updated weights for policy 1, policy_version 335319 (0.0008) [2023-12-26 17:48:08,072][105620] Updated weights for policy 1, policy_version 335329 (0.0007) [2023-12-26 17:48:08,534][105692] Updated weights for policy 0, policy_version 334961 (0.0010) [2023-12-26 17:48:08,585][105692] Updated weights for policy 0, policy_version 334971 (0.0010) [2023-12-26 17:48:08,640][105692] Updated weights for policy 0, policy_version 334981 (0.0008) [2023-12-26 17:48:08,815][105620] Updated weights for policy 1, policy_version 335339 (0.0007) [2023-12-26 17:48:08,869][105620] Updated weights for policy 1, policy_version 335349 (0.0009) [2023-12-26 17:48:08,923][105620] Updated weights for policy 1, policy_version 335359 (0.0008) [2023-12-26 17:48:09,442][105692] Updated weights for policy 0, policy_version 334991 (0.0009) [2023-12-26 17:48:09,501][105692] Updated weights for policy 0, policy_version 335001 (0.0009) [2023-12-26 17:48:09,555][105692] Updated weights for policy 0, policy_version 335011 (0.0009) [2023-12-26 17:48:09,688][105620] Updated weights for policy 1, policy_version 335369 (0.0006) [2023-12-26 17:48:09,753][105620] Updated weights for policy 1, policy_version 335379 (0.0009) [2023-12-26 17:48:09,824][105620] Updated weights for policy 1, policy_version 335389 (0.0009) [2023-12-26 17:48:09,889][105620] Updated weights for policy 1, policy_version 335399 (0.0008) [2023-12-26 17:48:10,338][105692] Updated weights for policy 0, policy_version 335021 (0.0009) [2023-12-26 17:48:10,401][105692] Updated weights for policy 0, policy_version 335031 (0.0009) [2023-12-26 17:48:10,453][105692] Updated weights for policy 0, policy_version 335041 (0.0009) [2023-12-26 17:48:10,652][105620] Updated weights for policy 1, policy_version 335409 (0.0009) [2023-12-26 17:48:10,712][105620] Updated weights for policy 1, policy_version 335419 (0.0006) [2023-12-26 17:48:10,774][105620] Updated weights for policy 1, policy_version 335429 (0.0006) [2023-12-26 17:48:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 171663360. Throughput: 0: 9538.2, 1: 10093.2. Samples: 171674100. Policy #0 lag: (min: 1.0, avg: 19.3, max: 33.0) [2023-12-26 17:48:11,062][104569] Avg episode reward: [(0, '9175.529'), (1, '9310.153')] [2023-12-26 17:48:11,269][105692] Updated weights for policy 0, policy_version 335051 (0.0009) [2023-12-26 17:48:11,343][105692] Updated weights for policy 0, policy_version 335061 (0.0009) [2023-12-26 17:48:11,410][105692] Updated weights for policy 0, policy_version 335071 (0.0009) [2023-12-26 17:48:11,467][105620] Updated weights for policy 1, policy_version 335439 (0.0008) [2023-12-26 17:48:11,529][105620] Updated weights for policy 1, policy_version 335449 (0.0007) [2023-12-26 17:48:11,586][105620] Updated weights for policy 1, policy_version 335459 (0.0008) [2023-12-26 17:48:12,201][105692] Updated weights for policy 0, policy_version 335081 (0.0007) [2023-12-26 17:48:12,253][105692] Updated weights for policy 0, policy_version 335091 (0.0009) [2023-12-26 17:48:12,314][105692] Updated weights for policy 0, policy_version 335101 (0.0009) [2023-12-26 17:48:12,334][105620] Updated weights for policy 1, policy_version 335469 (0.0008) [2023-12-26 17:48:12,382][105692] Updated weights for policy 0, policy_version 335111 (0.0010) [2023-12-26 17:48:12,398][105620] Updated weights for policy 1, policy_version 335479 (0.0009) [2023-12-26 17:48:12,459][105620] Updated weights for policy 1, policy_version 335489 (0.0010) [2023-12-26 17:48:13,141][105692] Updated weights for policy 0, policy_version 335121 (0.0007) [2023-12-26 17:48:13,160][105620] Updated weights for policy 1, policy_version 335499 (0.0010) [2023-12-26 17:48:13,189][105692] Updated weights for policy 0, policy_version 335131 (0.0007) [2023-12-26 17:48:13,218][105620] Updated weights for policy 1, policy_version 335509 (0.0008) [2023-12-26 17:48:13,236][105692] Updated weights for policy 0, policy_version 335141 (0.0008) [2023-12-26 17:48:13,278][105620] Updated weights for policy 1, policy_version 335519 (0.0007) [2023-12-26 17:48:13,934][105692] Updated weights for policy 0, policy_version 335151 (0.0009) [2023-12-26 17:48:13,969][105620] Updated weights for policy 1, policy_version 335529 (0.0007) [2023-12-26 17:48:13,990][105692] Updated weights for policy 0, policy_version 335161 (0.0009) [2023-12-26 17:48:14,024][105620] Updated weights for policy 1, policy_version 335539 (0.0007) [2023-12-26 17:48:14,043][105692] Updated weights for policy 0, policy_version 335171 (0.0006) [2023-12-26 17:48:14,075][105620] Updated weights for policy 1, policy_version 335549 (0.0007) [2023-12-26 17:48:14,122][105620] Updated weights for policy 1, policy_version 335559 (0.0008) [2023-12-26 17:48:14,796][105692] Updated weights for policy 0, policy_version 335181 (0.0008) [2023-12-26 17:48:14,861][105692] Updated weights for policy 0, policy_version 335191 (0.0008) [2023-12-26 17:48:14,912][105620] Updated weights for policy 1, policy_version 335569 (0.0007) [2023-12-26 17:48:14,921][105692] Updated weights for policy 0, policy_version 335201 (0.0007) [2023-12-26 17:48:14,971][105620] Updated weights for policy 1, policy_version 335579 (0.0010) [2023-12-26 17:48:15,033][105620] Updated weights for policy 1, policy_version 335589 (0.0009) [2023-12-26 17:48:15,632][105692] Updated weights for policy 0, policy_version 335211 (0.0008) [2023-12-26 17:48:15,699][105692] Updated weights for policy 0, policy_version 335221 (0.0010) [2023-12-26 17:48:15,761][105692] Updated weights for policy 0, policy_version 335231 (0.0010) [2023-12-26 17:48:15,804][105620] Updated weights for policy 1, policy_version 335599 (0.0006) [2023-12-26 17:48:15,854][105620] Updated weights for policy 1, policy_version 335609 (0.0005) [2023-12-26 17:48:15,920][105620] Updated weights for policy 1, policy_version 335619 (0.0005) [2023-12-26 17:48:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 171761664. Throughput: 0: 9548.3, 1: 9992.3. Samples: 171730164. Policy #0 lag: (min: 1.0, avg: 19.3, max: 33.0) [2023-12-26 17:48:16,063][104569] Avg episode reward: [(0, '9267.372'), (1, '8348.635')] [2023-12-26 17:48:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000335240_85835776.pth... [2023-12-26 17:48:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000335624_85925888.pth... [2023-12-26 17:48:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000334440_85622784.pth [2023-12-26 17:48:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000334120_85549056.pth [2023-12-26 17:48:16,426][105692] Updated weights for policy 0, policy_version 335241 (0.0008) [2023-12-26 17:48:16,481][105692] Updated weights for policy 0, policy_version 335251 (0.0008) [2023-12-26 17:48:16,486][105620] Updated weights for policy 1, policy_version 335629 (0.0007) [2023-12-26 17:48:16,539][105620] Updated weights for policy 1, policy_version 335639 (0.0008) [2023-12-26 17:48:16,545][105692] Updated weights for policy 0, policy_version 335261 (0.0005) [2023-12-26 17:48:16,594][105620] Updated weights for policy 1, policy_version 335649 (0.0008) [2023-12-26 17:48:16,611][105692] Updated weights for policy 0, policy_version 335271 (0.0005) [2023-12-26 17:48:17,197][105692] Updated weights for policy 0, policy_version 335281 (0.0008) [2023-12-26 17:48:17,253][105692] Updated weights for policy 0, policy_version 335291 (0.0008) [2023-12-26 17:48:17,321][105692] Updated weights for policy 0, policy_version 335301 (0.0005) [2023-12-26 17:48:17,428][105620] Updated weights for policy 1, policy_version 335659 (0.0009) [2023-12-26 17:48:17,487][105620] Updated weights for policy 1, policy_version 335669 (0.0008) [2023-12-26 17:48:17,550][105620] Updated weights for policy 1, policy_version 335679 (0.0008) [2023-12-26 17:48:17,926][105692] Updated weights for policy 0, policy_version 335311 (0.0005) [2023-12-26 17:48:17,978][105692] Updated weights for policy 0, policy_version 335321 (0.0005) [2023-12-26 17:48:18,029][105692] Updated weights for policy 0, policy_version 335331 (0.0006) [2023-12-26 17:48:18,340][105620] Updated weights for policy 1, policy_version 335689 (0.0009) [2023-12-26 17:48:18,410][105620] Updated weights for policy 1, policy_version 335699 (0.0009) [2023-12-26 17:48:18,471][105620] Updated weights for policy 1, policy_version 335709 (0.0009) [2023-12-26 17:48:18,522][105620] Updated weights for policy 1, policy_version 335719 (0.0008) [2023-12-26 17:48:18,744][105692] Updated weights for policy 0, policy_version 335341 (0.0007) [2023-12-26 17:48:18,811][105692] Updated weights for policy 0, policy_version 335351 (0.0009) [2023-12-26 17:48:18,875][105692] Updated weights for policy 0, policy_version 335361 (0.0008) [2023-12-26 17:48:19,306][105620] Updated weights for policy 1, policy_version 335729 (0.0009) [2023-12-26 17:48:19,369][105620] Updated weights for policy 1, policy_version 335739 (0.0009) [2023-12-26 17:48:19,436][105620] Updated weights for policy 1, policy_version 335749 (0.0006) [2023-12-26 17:48:19,596][105692] Updated weights for policy 0, policy_version 335371 (0.0006) [2023-12-26 17:48:19,667][105692] Updated weights for policy 0, policy_version 335381 (0.0009) [2023-12-26 17:48:19,726][105692] Updated weights for policy 0, policy_version 335391 (0.0009) [2023-12-26 17:48:20,203][105620] Updated weights for policy 1, policy_version 335759 (0.0007) [2023-12-26 17:48:20,262][105620] Updated weights for policy 1, policy_version 335769 (0.0008) [2023-12-26 17:48:20,322][105620] Updated weights for policy 1, policy_version 335779 (0.0009) [2023-12-26 17:48:20,446][105692] Updated weights for policy 0, policy_version 335401 (0.0009) [2023-12-26 17:48:20,509][105692] Updated weights for policy 0, policy_version 335411 (0.0009) [2023-12-26 17:48:20,575][105692] Updated weights for policy 0, policy_version 335421 (0.0009) [2023-12-26 17:48:20,633][105692] Updated weights for policy 0, policy_version 335431 (0.0006) [2023-12-26 17:48:21,047][105620] Updated weights for policy 1, policy_version 335789 (0.0010) [2023-12-26 17:48:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 171851776. Throughput: 0: 9722.9, 1: 9890.0. Samples: 171847068. Policy #0 lag: (min: 1.0, avg: 19.3, max: 33.0) [2023-12-26 17:48:21,062][104569] Avg episode reward: [(0, '9267.893'), (1, '8167.382')] [2023-12-26 17:48:21,115][105620] Updated weights for policy 1, policy_version 335799 (0.0010) [2023-12-26 17:48:21,173][105620] Updated weights for policy 1, policy_version 335809 (0.0009) [2023-12-26 17:48:21,346][105692] Updated weights for policy 0, policy_version 335441 (0.0008) [2023-12-26 17:48:21,413][105692] Updated weights for policy 0, policy_version 335451 (0.0008) [2023-12-26 17:48:21,480][105692] Updated weights for policy 0, policy_version 335461 (0.0008) [2023-12-26 17:48:21,907][105620] Updated weights for policy 1, policy_version 335819 (0.0008) [2023-12-26 17:48:21,975][105620] Updated weights for policy 1, policy_version 335829 (0.0007) [2023-12-26 17:48:22,037][105620] Updated weights for policy 1, policy_version 335839 (0.0006) [2023-12-26 17:48:22,346][105692] Updated weights for policy 0, policy_version 335471 (0.0008) [2023-12-26 17:48:22,412][105692] Updated weights for policy 0, policy_version 335481 (0.0008) [2023-12-26 17:48:22,464][105692] Updated weights for policy 0, policy_version 335491 (0.0008) [2023-12-26 17:48:22,696][105620] Updated weights for policy 1, policy_version 335849 (0.0006) [2023-12-26 17:48:22,763][105620] Updated weights for policy 1, policy_version 335859 (0.0009) [2023-12-26 17:48:22,827][105620] Updated weights for policy 1, policy_version 335869 (0.0010) [2023-12-26 17:48:22,892][105620] Updated weights for policy 1, policy_version 335879 (0.0011) [2023-12-26 17:48:23,313][105692] Updated weights for policy 0, policy_version 335501 (0.0008) [2023-12-26 17:48:23,368][105692] Updated weights for policy 0, policy_version 335511 (0.0005) [2023-12-26 17:48:23,411][105692] Updated weights for policy 0, policy_version 335521 (0.0005) [2023-12-26 17:48:23,509][105620] Updated weights for policy 1, policy_version 335889 (0.0008) [2023-12-26 17:48:23,567][105620] Updated weights for policy 1, policy_version 335899 (0.0010) [2023-12-26 17:48:23,615][105620] Updated weights for policy 1, policy_version 335909 (0.0010) [2023-12-26 17:48:23,992][105692] Updated weights for policy 0, policy_version 335531 (0.0005) [2023-12-26 17:48:24,045][105692] Updated weights for policy 0, policy_version 335541 (0.0005) [2023-12-26 17:48:24,093][105692] Updated weights for policy 0, policy_version 335551 (0.0006) [2023-12-26 17:48:24,176][105620] Updated weights for policy 1, policy_version 335919 (0.0009) [2023-12-26 17:48:24,240][105620] Updated weights for policy 1, policy_version 335929 (0.0009) [2023-12-26 17:48:24,301][105620] Updated weights for policy 1, policy_version 335939 (0.0008) [2023-12-26 17:48:24,686][105692] Updated weights for policy 0, policy_version 335561 (0.0007) [2023-12-26 17:48:24,744][105692] Updated weights for policy 0, policy_version 335571 (0.0009) [2023-12-26 17:48:24,791][105692] Updated weights for policy 0, policy_version 335581 (0.0008) [2023-12-26 17:48:24,843][105692] Updated weights for policy 0, policy_version 335591 (0.0007) [2023-12-26 17:48:25,165][105620] Updated weights for policy 1, policy_version 335949 (0.0009) [2023-12-26 17:48:25,221][105620] Updated weights for policy 1, policy_version 335959 (0.0009) [2023-12-26 17:48:25,276][105620] Updated weights for policy 1, policy_version 335969 (0.0009) [2023-12-26 17:48:25,471][105692] Updated weights for policy 0, policy_version 335601 (0.0008) [2023-12-26 17:48:25,537][105692] Updated weights for policy 0, policy_version 335611 (0.0009) [2023-12-26 17:48:25,601][105692] Updated weights for policy 0, policy_version 335621 (0.0010) [2023-12-26 17:48:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 171950080. Throughput: 0: 9760.0, 1: 9920.9. Samples: 171964044. Policy #0 lag: (min: 1.0, avg: 19.3, max: 33.0) [2023-12-26 17:48:26,063][104569] Avg episode reward: [(0, '9087.231'), (1, '8743.562')] [2023-12-26 17:48:26,097][105620] Updated weights for policy 1, policy_version 335979 (0.0009) [2023-12-26 17:48:26,146][105620] Updated weights for policy 1, policy_version 335989 (0.0006) [2023-12-26 17:48:26,199][105620] Updated weights for policy 1, policy_version 335999 (0.0005) [2023-12-26 17:48:26,237][105692] Updated weights for policy 0, policy_version 335631 (0.0006) [2023-12-26 17:48:26,306][105692] Updated weights for policy 0, policy_version 335641 (0.0006) [2023-12-26 17:48:26,363][105692] Updated weights for policy 0, policy_version 335651 (0.0005) [2023-12-26 17:48:26,743][105620] Updated weights for policy 1, policy_version 336009 (0.0005) [2023-12-26 17:48:26,791][105620] Updated weights for policy 1, policy_version 336019 (0.0005) [2023-12-26 17:48:26,851][105620] Updated weights for policy 1, policy_version 336029 (0.0006) [2023-12-26 17:48:26,901][105620] Updated weights for policy 1, policy_version 336039 (0.0009) [2023-12-26 17:48:27,025][105692] Updated weights for policy 0, policy_version 335661 (0.0007) [2023-12-26 17:48:27,074][105692] Updated weights for policy 0, policy_version 335671 (0.0009) [2023-12-26 17:48:27,126][105692] Updated weights for policy 0, policy_version 335681 (0.0010) [2023-12-26 17:48:27,561][105620] Updated weights for policy 1, policy_version 336049 (0.0009) [2023-12-26 17:48:27,608][105620] Updated weights for policy 1, policy_version 336059 (0.0009) [2023-12-26 17:48:27,655][105620] Updated weights for policy 1, policy_version 336069 (0.0009) [2023-12-26 17:48:27,883][105692] Updated weights for policy 0, policy_version 335692 (0.0009) [2023-12-26 17:48:27,933][105692] Updated weights for policy 0, policy_version 335702 (0.0009) [2023-12-26 17:48:27,984][105692] Updated weights for policy 0, policy_version 335712 (0.0009) [2023-12-26 17:48:28,420][105620] Updated weights for policy 1, policy_version 336079 (0.0009) [2023-12-26 17:48:28,479][105620] Updated weights for policy 1, policy_version 336089 (0.0009) [2023-12-26 17:48:28,526][105620] Updated weights for policy 1, policy_version 336099 (0.0009) [2023-12-26 17:48:28,747][105692] Updated weights for policy 0, policy_version 335722 (0.0009) [2023-12-26 17:48:28,812][105692] Updated weights for policy 0, policy_version 335732 (0.0009) [2023-12-26 17:48:28,867][105692] Updated weights for policy 0, policy_version 335742 (0.0009) [2023-12-26 17:48:28,932][105692] Updated weights for policy 0, policy_version 335752 (0.0009) [2023-12-26 17:48:29,299][105620] Updated weights for policy 1, policy_version 336109 (0.0009) [2023-12-26 17:48:29,367][105620] Updated weights for policy 1, policy_version 336119 (0.0010) [2023-12-26 17:48:29,434][105620] Updated weights for policy 1, policy_version 336129 (0.0006) [2023-12-26 17:48:29,670][105692] Updated weights for policy 0, policy_version 335762 (0.0005) [2023-12-26 17:48:29,743][105692] Updated weights for policy 0, policy_version 335772 (0.0005) [2023-12-26 17:48:29,811][105692] Updated weights for policy 0, policy_version 335782 (0.0005) [2023-12-26 17:48:30,167][105620] Updated weights for policy 1, policy_version 336139 (0.0007) [2023-12-26 17:48:30,226][105620] Updated weights for policy 1, policy_version 336149 (0.0009) [2023-12-26 17:48:30,280][105620] Updated weights for policy 1, policy_version 336159 (0.0005) [2023-12-26 17:48:30,454][105692] Updated weights for policy 0, policy_version 335793 (0.0008) [2023-12-26 17:48:30,507][105692] Updated weights for policy 0, policy_version 335803 (0.0010) [2023-12-26 17:48:30,565][105692] Updated weights for policy 0, policy_version 335814 (0.0008) [2023-12-26 17:48:30,871][105620] Updated weights for policy 1, policy_version 336169 (0.0006) [2023-12-26 17:48:30,935][105620] Updated weights for policy 1, policy_version 336179 (0.0005) [2023-12-26 17:48:31,011][105620] Updated weights for policy 1, policy_version 336189 (0.0007) [2023-12-26 17:48:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 172048384. Throughput: 0: 9802.4, 1: 9967.7. Samples: 172024804. Policy #0 lag: (min: 1.0, avg: 19.3, max: 33.0) [2023-12-26 17:48:31,062][104569] Avg episode reward: [(0, '9177.658'), (1, '9265.506')] [2023-12-26 17:48:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000335816_85983232.pth... [2023-12-26 17:48:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000334696_85696512.pth [2023-12-26 17:48:31,077][105620] Updated weights for policy 1, policy_version 336199 (0.0008) [2023-12-26 17:48:31,084][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000336200_86073344.pth... [2023-12-26 17:48:31,090][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000335048_85778432.pth [2023-12-26 17:48:31,164][105692] Updated weights for policy 0, policy_version 335824 (0.0008) [2023-12-26 17:48:31,215][105692] Updated weights for policy 0, policy_version 335834 (0.0010) [2023-12-26 17:48:31,271][105692] Updated weights for policy 0, policy_version 335844 (0.0011) [2023-12-26 17:48:31,733][105620] Updated weights for policy 1, policy_version 336209 (0.0006) [2023-12-26 17:48:31,799][105620] Updated weights for policy 1, policy_version 336219 (0.0007) [2023-12-26 17:48:31,858][105620] Updated weights for policy 1, policy_version 336229 (0.0011) [2023-12-26 17:48:32,028][105692] Updated weights for policy 0, policy_version 335854 (0.0011) [2023-12-26 17:48:32,093][105692] Updated weights for policy 0, policy_version 335864 (0.0010) [2023-12-26 17:48:32,157][105692] Updated weights for policy 0, policy_version 335874 (0.0011) [2023-12-26 17:48:32,487][105620] Updated weights for policy 1, policy_version 336239 (0.0008) [2023-12-26 17:48:32,540][105620] Updated weights for policy 1, policy_version 336249 (0.0006) [2023-12-26 17:48:32,588][105620] Updated weights for policy 1, policy_version 336259 (0.0008) [2023-12-26 17:48:32,847][105692] Updated weights for policy 0, policy_version 335884 (0.0011) [2023-12-26 17:48:32,908][105692] Updated weights for policy 0, policy_version 335894 (0.0010) [2023-12-26 17:48:32,977][105692] Updated weights for policy 0, policy_version 335904 (0.0010) [2023-12-26 17:48:33,347][105620] Updated weights for policy 1, policy_version 336269 (0.0010) [2023-12-26 17:48:33,408][105620] Updated weights for policy 1, policy_version 336279 (0.0010) [2023-12-26 17:48:33,459][105620] Updated weights for policy 1, policy_version 336289 (0.0010) [2023-12-26 17:48:33,686][105692] Updated weights for policy 0, policy_version 335914 (0.0007) [2023-12-26 17:48:33,740][105692] Updated weights for policy 0, policy_version 335924 (0.0010) [2023-12-26 17:48:33,794][105692] Updated weights for policy 0, policy_version 335934 (0.0010) [2023-12-26 17:48:33,847][105692] Updated weights for policy 0, policy_version 335944 (0.0010) [2023-12-26 17:48:34,176][105620] Updated weights for policy 1, policy_version 336299 (0.0010) [2023-12-26 17:48:34,235][105620] Updated weights for policy 1, policy_version 336309 (0.0010) [2023-12-26 17:48:34,295][105620] Updated weights for policy 1, policy_version 336319 (0.0011) [2023-12-26 17:48:34,548][105692] Updated weights for policy 0, policy_version 335954 (0.0010) [2023-12-26 17:48:34,597][105692] Updated weights for policy 0, policy_version 335964 (0.0010) [2023-12-26 17:48:34,650][105692] Updated weights for policy 0, policy_version 335974 (0.0011) [2023-12-26 17:48:35,054][105620] Updated weights for policy 1, policy_version 336329 (0.0010) [2023-12-26 17:48:35,113][105620] Updated weights for policy 1, policy_version 336339 (0.0010) [2023-12-26 17:48:35,157][105620] Updated weights for policy 1, policy_version 336349 (0.0010) [2023-12-26 17:48:35,202][105620] Updated weights for policy 1, policy_version 336359 (0.0010) [2023-12-26 17:48:35,314][105692] Updated weights for policy 0, policy_version 335984 (0.0006) [2023-12-26 17:48:35,362][105692] Updated weights for policy 0, policy_version 335994 (0.0005) [2023-12-26 17:48:35,421][105692] Updated weights for policy 0, policy_version 336004 (0.0005) [2023-12-26 17:48:35,921][105692] Updated weights for policy 0, policy_version 336014 (0.0005) [2023-12-26 17:48:35,977][105692] Updated weights for policy 0, policy_version 336024 (0.0006) [2023-12-26 17:48:35,978][105620] Updated weights for policy 1, policy_version 336369 (0.0011) [2023-12-26 17:48:36,006][105586] KL-divergence is very high: 169.7862 [2023-12-26 17:48:36,011][105586] KL-divergence is very high: 436.1116 [2023-12-26 17:48:36,022][105586] KL-divergence is very high: 295.1536 [2023-12-26 17:48:36,030][105692] Updated weights for policy 0, policy_version 336034 (0.0007) [2023-12-26 17:48:36,035][105620] Updated weights for policy 1, policy_version 336379 (0.0010) [2023-12-26 17:48:36,048][105586] KL-divergence is very high: 225.8262 [2023-12-26 17:48:36,055][105586] KL-divergence is very high: 330.6859 [2023-12-26 17:48:36,061][105586] KL-divergence is very high: 775.3588 [2023-12-26 17:48:36,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 172154880. Throughput: 0: 9807.0, 1: 9938.3. Samples: 172143860. Policy #0 lag: (min: 1.0, avg: 19.3, max: 33.0) [2023-12-26 17:48:36,062][104569] Avg episode reward: [(0, '9268.676'), (1, '9172.925')] [2023-12-26 17:48:36,072][105586] KL-divergence is very high: 387.8055 [2023-12-26 17:48:36,091][105586] KL-divergence is very high: 233.3317 [2023-12-26 17:48:36,094][105620] Updated weights for policy 1, policy_version 336389 (0.0011) [2023-12-26 17:48:36,097][105586] KL-divergence is very high: 332.3782 [2023-12-26 17:48:36,106][105586] KL-divergence is very high: 820.2115 [2023-12-26 17:48:36,665][105692] Updated weights for policy 0, policy_version 336044 (0.0009) [2023-12-26 17:48:36,717][105692] Updated weights for policy 0, policy_version 336054 (0.0011) [2023-12-26 17:48:36,766][105692] Updated weights for policy 0, policy_version 336064 (0.0011) [2023-12-26 17:48:36,867][105620] Updated weights for policy 1, policy_version 336399 (0.0011) [2023-12-26 17:48:36,915][105620] Updated weights for policy 1, policy_version 336409 (0.0010) [2023-12-26 17:48:36,974][105620] Updated weights for policy 1, policy_version 336419 (0.0011) [2023-12-26 17:48:37,437][105692] Updated weights for policy 0, policy_version 336074 (0.0010) [2023-12-26 17:48:37,491][105692] Updated weights for policy 0, policy_version 336084 (0.0005) [2023-12-26 17:48:37,543][105692] Updated weights for policy 0, policy_version 336094 (0.0007) [2023-12-26 17:48:37,593][105692] Updated weights for policy 0, policy_version 336104 (0.0008) [2023-12-26 17:48:37,739][105620] Updated weights for policy 1, policy_version 336429 (0.0011) [2023-12-26 17:48:37,805][105620] Updated weights for policy 1, policy_version 336439 (0.0011) [2023-12-26 17:48:37,868][105620] Updated weights for policy 1, policy_version 336449 (0.0010) [2023-12-26 17:48:38,291][105692] Updated weights for policy 0, policy_version 336114 (0.0008) [2023-12-26 17:48:38,350][105692] Updated weights for policy 0, policy_version 336124 (0.0008) [2023-12-26 17:48:38,413][105692] Updated weights for policy 0, policy_version 336134 (0.0008) [2023-12-26 17:48:38,623][105620] Updated weights for policy 1, policy_version 336459 (0.0010) [2023-12-26 17:48:38,689][105620] Updated weights for policy 1, policy_version 336469 (0.0011) [2023-12-26 17:48:38,755][105620] Updated weights for policy 1, policy_version 336479 (0.0011) [2023-12-26 17:48:39,062][105692] Updated weights for policy 0, policy_version 336144 (0.0005) [2023-12-26 17:48:39,116][105692] Updated weights for policy 0, policy_version 336154 (0.0008) [2023-12-26 17:48:39,169][105692] Updated weights for policy 0, policy_version 336164 (0.0006) [2023-12-26 17:48:39,516][105620] Updated weights for policy 1, policy_version 336489 (0.0011) [2023-12-26 17:48:39,576][105620] Updated weights for policy 1, policy_version 336499 (0.0011) [2023-12-26 17:48:39,632][105620] Updated weights for policy 1, policy_version 336509 (0.0010) [2023-12-26 17:48:39,685][105620] Updated weights for policy 1, policy_version 336519 (0.0010) [2023-12-26 17:48:39,951][105692] Updated weights for policy 0, policy_version 336174 (0.0010) [2023-12-26 17:48:40,015][105692] Updated weights for policy 0, policy_version 336184 (0.0011) [2023-12-26 17:48:40,080][105692] Updated weights for policy 0, policy_version 336194 (0.0010) [2023-12-26 17:48:40,440][105620] Updated weights for policy 1, policy_version 336529 (0.0010) [2023-12-26 17:48:40,500][105620] Updated weights for policy 1, policy_version 336539 (0.0011) [2023-12-26 17:48:40,563][105620] Updated weights for policy 1, policy_version 336549 (0.0011) [2023-12-26 17:48:40,836][105692] Updated weights for policy 0, policy_version 336204 (0.0010) [2023-12-26 17:48:40,902][105692] Updated weights for policy 0, policy_version 336214 (0.0011) [2023-12-26 17:48:40,961][105692] Updated weights for policy 0, policy_version 336224 (0.0011) [2023-12-26 17:48:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 172253184. Throughput: 0: 9776.5, 1: 9695.4. Samples: 172261092. Policy #0 lag: (min: 1.0, avg: 19.3, max: 33.0) [2023-12-26 17:48:41,062][104569] Avg episode reward: [(0, '9268.646'), (1, '9172.987')] [2023-12-26 17:48:41,331][105620] Updated weights for policy 1, policy_version 336559 (0.0008) [2023-12-26 17:48:41,400][105620] Updated weights for policy 1, policy_version 336569 (0.0008) [2023-12-26 17:48:41,459][105620] Updated weights for policy 1, policy_version 336579 (0.0008) [2023-12-26 17:48:41,725][105692] Updated weights for policy 0, policy_version 336234 (0.0008) [2023-12-26 17:48:41,799][105692] Updated weights for policy 0, policy_version 336244 (0.0010) [2023-12-26 17:48:41,858][105692] Updated weights for policy 0, policy_version 336254 (0.0009) [2023-12-26 17:48:41,928][105692] Updated weights for policy 0, policy_version 336264 (0.0011) [2023-12-26 17:48:42,111][105620] Updated weights for policy 1, policy_version 336589 (0.0008) [2023-12-26 17:48:42,175][105620] Updated weights for policy 1, policy_version 336599 (0.0007) [2023-12-26 17:48:42,241][105620] Updated weights for policy 1, policy_version 336609 (0.0007) [2023-12-26 17:48:42,696][105692] Updated weights for policy 0, policy_version 336274 (0.0009) [2023-12-26 17:48:42,759][105692] Updated weights for policy 0, policy_version 336284 (0.0009) [2023-12-26 17:48:42,813][105692] Updated weights for policy 0, policy_version 336294 (0.0008) [2023-12-26 17:48:42,890][105620] Updated weights for policy 1, policy_version 336619 (0.0008) [2023-12-26 17:48:42,954][105620] Updated weights for policy 1, policy_version 336629 (0.0009) [2023-12-26 17:48:43,008][105620] Updated weights for policy 1, policy_version 336639 (0.0009) [2023-12-26 17:48:43,565][105692] Updated weights for policy 0, policy_version 336304 (0.0009) [2023-12-26 17:48:43,619][105692] Updated weights for policy 0, policy_version 336314 (0.0008) [2023-12-26 17:48:43,677][105692] Updated weights for policy 0, policy_version 336324 (0.0005) [2023-12-26 17:48:43,774][105620] Updated weights for policy 1, policy_version 336649 (0.0009) [2023-12-26 17:48:43,830][105620] Updated weights for policy 1, policy_version 336659 (0.0009) [2023-12-26 17:48:43,882][105620] Updated weights for policy 1, policy_version 336669 (0.0011) [2023-12-26 17:48:43,932][105620] Updated weights for policy 1, policy_version 336679 (0.0008) [2023-12-26 17:48:44,341][105692] Updated weights for policy 0, policy_version 336334 (0.0007) [2023-12-26 17:48:44,404][105692] Updated weights for policy 0, policy_version 336344 (0.0008) [2023-12-26 17:48:44,455][105692] Updated weights for policy 0, policy_version 336354 (0.0008) [2023-12-26 17:48:44,712][105620] Updated weights for policy 1, policy_version 336689 (0.0010) [2023-12-26 17:48:44,760][105620] Updated weights for policy 1, policy_version 336699 (0.0010) [2023-12-26 17:48:44,819][105620] Updated weights for policy 1, policy_version 336709 (0.0009) [2023-12-26 17:48:45,280][105692] Updated weights for policy 0, policy_version 336364 (0.0009) [2023-12-26 17:48:45,341][105692] Updated weights for policy 0, policy_version 336374 (0.0008) [2023-12-26 17:48:45,398][105692] Updated weights for policy 0, policy_version 336384 (0.0008) [2023-12-26 17:48:45,534][105620] Updated weights for policy 1, policy_version 336719 (0.0009) [2023-12-26 17:48:45,597][105620] Updated weights for policy 1, policy_version 336729 (0.0011) [2023-12-26 17:48:45,659][105620] Updated weights for policy 1, policy_version 336739 (0.0010) [2023-12-26 17:48:46,062][104569] Fps is (10 sec: 18840.8, 60 sec: 19660.7, 300 sec: 19466.4). Total num frames: 172343296. Throughput: 0: 9722.7, 1: 9728.3. Samples: 172317792. Policy #0 lag: (min: 1.0, avg: 19.3, max: 33.0) [2023-12-26 17:48:46,063][104569] Avg episode reward: [(0, '9268.752'), (1, '9265.864')] [2023-12-26 17:48:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000336392_86130688.pth... [2023-12-26 17:48:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000336744_86212608.pth... [2023-12-26 17:48:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000335240_85835776.pth [2023-12-26 17:48:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000335624_85925888.pth [2023-12-26 17:48:46,146][105692] Updated weights for policy 0, policy_version 336394 (0.0009) [2023-12-26 17:48:46,194][105692] Updated weights for policy 0, policy_version 336404 (0.0008) [2023-12-26 17:48:46,242][105692] Updated weights for policy 0, policy_version 336414 (0.0008) [2023-12-26 17:48:46,294][105692] Updated weights for policy 0, policy_version 336424 (0.0008) [2023-12-26 17:48:46,388][105620] Updated weights for policy 1, policy_version 336749 (0.0010) [2023-12-26 17:48:46,445][105620] Updated weights for policy 1, policy_version 336759 (0.0010) [2023-12-26 17:48:46,503][105620] Updated weights for policy 1, policy_version 336769 (0.0010) [2023-12-26 17:48:47,069][105692] Updated weights for policy 0, policy_version 336434 (0.0008) [2023-12-26 17:48:47,131][105692] Updated weights for policy 0, policy_version 336444 (0.0008) [2023-12-26 17:48:47,156][105620] Updated weights for policy 1, policy_version 336779 (0.0010) [2023-12-26 17:48:47,186][105692] Updated weights for policy 0, policy_version 336454 (0.0007) [2023-12-26 17:48:47,214][105620] Updated weights for policy 1, policy_version 336789 (0.0010) [2023-12-26 17:48:47,275][105620] Updated weights for policy 1, policy_version 336799 (0.0010) [2023-12-26 17:48:47,936][105692] Updated weights for policy 0, policy_version 336464 (0.0010) [2023-12-26 17:48:47,949][105585] KL-divergence is very high: 135.3168 [2023-12-26 17:48:47,978][105585] KL-divergence is very high: 105.1256 [2023-12-26 17:48:47,994][105692] Updated weights for policy 0, policy_version 336474 (0.0010) [2023-12-26 17:48:48,009][105620] Updated weights for policy 1, policy_version 336809 (0.0010) [2023-12-26 17:48:48,017][105585] KL-divergence is very high: 105.6916 [2023-12-26 17:48:48,023][105585] KL-divergence is very high: 113.8575 [2023-12-26 17:48:48,038][105585] KL-divergence is very high: 120.9820 [2023-12-26 17:48:48,048][105692] Updated weights for policy 0, policy_version 336484 (0.0005) [2023-12-26 17:48:48,061][105585] KL-divergence is very high: 111.6590 [2023-12-26 17:48:48,064][105620] Updated weights for policy 1, policy_version 336819 (0.0010) [2023-12-26 17:48:48,122][105620] Updated weights for policy 1, policy_version 336829 (0.0010) [2023-12-26 17:48:48,179][105620] Updated weights for policy 1, policy_version 336839 (0.0010) [2023-12-26 17:48:48,741][105585] KL-divergence is very high: 138.9802 [2023-12-26 17:48:48,771][105692] Updated weights for policy 0, policy_version 336494 (0.0008) [2023-12-26 17:48:48,834][105585] KL-divergence is very high: 103.9896 [2023-12-26 17:48:48,840][105692] Updated weights for policy 0, policy_version 336504 (0.0010) [2023-12-26 17:48:48,849][105585] KL-divergence is very high: 105.8368 [2023-12-26 17:48:48,887][105620] Updated weights for policy 1, policy_version 336849 (0.0010) [2023-12-26 17:48:48,900][105692] Updated weights for policy 0, policy_version 336514 (0.0010) [2023-12-26 17:48:48,953][105620] Updated weights for policy 1, policy_version 336859 (0.0011) [2023-12-26 17:48:49,016][105620] Updated weights for policy 1, policy_version 336869 (0.0010) [2023-12-26 17:48:49,610][105692] Updated weights for policy 0, policy_version 336524 (0.0010) [2023-12-26 17:48:49,678][105692] Updated weights for policy 0, policy_version 336534 (0.0008) [2023-12-26 17:48:49,693][105585] KL-divergence is very high: 404.3261 [2023-12-26 17:48:49,740][105620] Updated weights for policy 1, policy_version 336879 (0.0008) [2023-12-26 17:48:49,748][105585] KL-divergence is very high: 552.2351 [2023-12-26 17:48:49,749][105692] Updated weights for policy 0, policy_version 336544 (0.0008) [2023-12-26 17:48:49,800][105620] Updated weights for policy 1, policy_version 336889 (0.0006) [2023-12-26 17:48:49,869][105620] Updated weights for policy 1, policy_version 336899 (0.0009) [2023-12-26 17:48:50,363][105692] Updated weights for policy 0, policy_version 336554 (0.0009) [2023-12-26 17:48:50,426][105692] Updated weights for policy 0, policy_version 336564 (0.0011) [2023-12-26 17:48:50,481][105692] Updated weights for policy 0, policy_version 336574 (0.0010) [2023-12-26 17:48:50,541][105692] Updated weights for policy 0, policy_version 336584 (0.0011) [2023-12-26 17:48:50,668][105620] Updated weights for policy 1, policy_version 336909 (0.0009) [2023-12-26 17:48:50,725][105620] Updated weights for policy 1, policy_version 336919 (0.0010) [2023-12-26 17:48:50,781][105620] Updated weights for policy 1, policy_version 336929 (0.0011) [2023-12-26 17:48:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 172441600. Throughput: 0: 9735.7, 1: 9678.2. Samples: 172432232. Policy #0 lag: (min: 1.0, avg: 19.3, max: 33.0) [2023-12-26 17:48:51,062][104569] Avg episode reward: [(0, '5220.932'), (1, '9357.786')] [2023-12-26 17:48:51,322][105692] Updated weights for policy 0, policy_version 336594 (0.0007) [2023-12-26 17:48:51,387][105692] Updated weights for policy 0, policy_version 336604 (0.0008) [2023-12-26 17:48:51,443][105692] Updated weights for policy 0, policy_version 336614 (0.0007) [2023-12-26 17:48:51,499][105620] Updated weights for policy 1, policy_version 336939 (0.0011) [2023-12-26 17:48:51,558][105620] Updated weights for policy 1, policy_version 336949 (0.0011) [2023-12-26 17:48:51,621][105620] Updated weights for policy 1, policy_version 336959 (0.0011) [2023-12-26 17:48:52,207][105692] Updated weights for policy 0, policy_version 336624 (0.0010) [2023-12-26 17:48:52,269][105692] Updated weights for policy 0, policy_version 336634 (0.0010) [2023-12-26 17:48:52,314][105620] Updated weights for policy 1, policy_version 336969 (0.0011) [2023-12-26 17:48:52,334][105692] Updated weights for policy 0, policy_version 336644 (0.0007) [2023-12-26 17:48:52,388][105620] Updated weights for policy 1, policy_version 336979 (0.0009) [2023-12-26 17:48:52,454][105620] Updated weights for policy 1, policy_version 336989 (0.0008) [2023-12-26 17:48:52,518][105620] Updated weights for policy 1, policy_version 336999 (0.0008) [2023-12-26 17:48:53,031][105692] Updated weights for policy 0, policy_version 336654 (0.0007) [2023-12-26 17:48:53,080][105692] Updated weights for policy 0, policy_version 336664 (0.0008) [2023-12-26 17:48:53,135][105692] Updated weights for policy 0, policy_version 336674 (0.0008) [2023-12-26 17:48:53,228][105620] Updated weights for policy 1, policy_version 337009 (0.0010) [2023-12-26 17:48:53,276][105620] Updated weights for policy 1, policy_version 337019 (0.0010) [2023-12-26 17:48:53,324][105620] Updated weights for policy 1, policy_version 337029 (0.0010) [2023-12-26 17:48:53,899][105692] Updated weights for policy 0, policy_version 336684 (0.0009) [2023-12-26 17:48:53,938][105585] KL-divergence is very high: 139.8457 [2023-12-26 17:48:53,964][105692] Updated weights for policy 0, policy_version 336694 (0.0010) [2023-12-26 17:48:53,987][105585] KL-divergence is very high: 110.6771 [2023-12-26 17:48:54,022][105692] Updated weights for policy 0, policy_version 336704 (0.0010) [2023-12-26 17:48:54,087][105620] Updated weights for policy 1, policy_version 337039 (0.0009) [2023-12-26 17:48:54,143][105620] Updated weights for policy 1, policy_version 337049 (0.0008) [2023-12-26 17:48:54,205][105620] Updated weights for policy 1, policy_version 337059 (0.0008) [2023-12-26 17:48:54,758][105692] Updated weights for policy 0, policy_version 336714 (0.0011) [2023-12-26 17:48:54,813][105692] Updated weights for policy 0, policy_version 336724 (0.0010) [2023-12-26 17:48:54,868][105692] Updated weights for policy 0, policy_version 336734 (0.0010) [2023-12-26 17:48:54,922][105692] Updated weights for policy 0, policy_version 336744 (0.0010) [2023-12-26 17:48:54,979][105620] Updated weights for policy 1, policy_version 337069 (0.0009) [2023-12-26 17:48:55,035][105620] Updated weights for policy 1, policy_version 337079 (0.0008) [2023-12-26 17:48:55,082][105620] Updated weights for policy 1, policy_version 337089 (0.0007) [2023-12-26 17:48:55,672][105692] Updated weights for policy 0, policy_version 336754 (0.0010) [2023-12-26 17:48:55,721][105692] Updated weights for policy 0, policy_version 336764 (0.0010) [2023-12-26 17:48:55,766][105692] Updated weights for policy 0, policy_version 336774 (0.0010) [2023-12-26 17:48:55,849][105620] Updated weights for policy 1, policy_version 337099 (0.0008) [2023-12-26 17:48:55,914][105620] Updated weights for policy 1, policy_version 337109 (0.0008) [2023-12-26 17:48:55,974][105620] Updated weights for policy 1, policy_version 337119 (0.0008) [2023-12-26 17:48:56,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 172539904. Throughput: 0: 9765.5, 1: 9594.6. Samples: 172545304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:48:56,063][104569] Avg episode reward: [(0, '5276.730'), (1, '9356.745')] [2023-12-26 17:48:56,534][105692] Updated weights for policy 0, policy_version 336784 (0.0010) [2023-12-26 17:48:56,582][105692] Updated weights for policy 0, policy_version 336794 (0.0010) [2023-12-26 17:48:56,627][105692] Updated weights for policy 0, policy_version 336804 (0.0010) [2023-12-26 17:48:56,696][105620] Updated weights for policy 1, policy_version 337129 (0.0008) [2023-12-26 17:48:56,754][105620] Updated weights for policy 1, policy_version 337139 (0.0008) [2023-12-26 17:48:56,801][105620] Updated weights for policy 1, policy_version 337149 (0.0008) [2023-12-26 17:48:56,844][105620] Updated weights for policy 1, policy_version 337159 (0.0008) [2023-12-26 17:48:57,396][105692] Updated weights for policy 0, policy_version 336814 (0.0007) [2023-12-26 17:48:57,456][105692] Updated weights for policy 0, policy_version 336824 (0.0009) [2023-12-26 17:48:57,500][105692] Updated weights for policy 0, policy_version 336834 (0.0010) [2023-12-26 17:48:57,554][105620] Updated weights for policy 1, policy_version 337169 (0.0007) [2023-12-26 17:48:57,605][105620] Updated weights for policy 1, policy_version 337179 (0.0008) [2023-12-26 17:48:57,648][105620] Updated weights for policy 1, policy_version 337189 (0.0008) [2023-12-26 17:48:58,208][105692] Updated weights for policy 0, policy_version 336844 (0.0009) [2023-12-26 17:48:58,270][105692] Updated weights for policy 0, policy_version 336854 (0.0008) [2023-12-26 17:48:58,347][105692] Updated weights for policy 0, policy_version 336864 (0.0009) [2023-12-26 17:48:58,431][105620] Updated weights for policy 1, policy_version 337199 (0.0008) [2023-12-26 17:48:58,494][105620] Updated weights for policy 1, policy_version 337209 (0.0008) [2023-12-26 17:48:58,557][105620] Updated weights for policy 1, policy_version 337219 (0.0007) [2023-12-26 17:48:59,089][105692] Updated weights for policy 0, policy_version 336874 (0.0009) [2023-12-26 17:48:59,138][105692] Updated weights for policy 0, policy_version 336884 (0.0010) [2023-12-26 17:48:59,193][105692] Updated weights for policy 0, policy_version 336894 (0.0010) [2023-12-26 17:48:59,233][105620] Updated weights for policy 1, policy_version 337229 (0.0008) [2023-12-26 17:48:59,250][105692] Updated weights for policy 0, policy_version 336904 (0.0009) [2023-12-26 17:48:59,292][105620] Updated weights for policy 1, policy_version 337239 (0.0007) [2023-12-26 17:48:59,361][105620] Updated weights for policy 1, policy_version 337249 (0.0008) [2023-12-26 17:48:59,997][105692] Updated weights for policy 0, policy_version 336914 (0.0011) [2023-12-26 17:49:00,014][105585] KL-divergence is very high: 102.8023 [2023-12-26 17:49:00,054][105692] Updated weights for policy 0, policy_version 336924 (0.0011) [2023-12-26 17:49:00,105][105620] Updated weights for policy 1, policy_version 337259 (0.0008) [2023-12-26 17:49:00,113][105692] Updated weights for policy 0, policy_version 336934 (0.0010) [2023-12-26 17:49:00,160][105620] Updated weights for policy 1, policy_version 337269 (0.0008) [2023-12-26 17:49:00,211][105620] Updated weights for policy 1, policy_version 337279 (0.0008) [2023-12-26 17:49:00,849][105692] Updated weights for policy 0, policy_version 336944 (0.0011) [2023-12-26 17:49:00,917][105692] Updated weights for policy 0, policy_version 336954 (0.0011) [2023-12-26 17:49:00,976][105620] Updated weights for policy 1, policy_version 337289 (0.0008) [2023-12-26 17:49:00,977][105692] Updated weights for policy 0, policy_version 336964 (0.0011) [2023-12-26 17:49:01,030][105620] Updated weights for policy 1, policy_version 337299 (0.0008) [2023-12-26 17:49:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 172630016. Throughput: 0: 9797.9, 1: 9575.7. Samples: 172601972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:49:01,062][104569] Avg episode reward: [(0, '6623.254'), (1, '2796.171')] [2023-12-26 17:49:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000336968_86278144.pth... [2023-12-26 17:49:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000335816_85983232.pth [2023-12-26 17:49:01,097][105620] Updated weights for policy 1, policy_version 337309 (0.0008) [2023-12-26 17:49:01,163][105620] Updated weights for policy 1, policy_version 337319 (0.0008) [2023-12-26 17:49:01,168][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000337320_86360064.pth... [2023-12-26 17:49:01,175][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000336200_86073344.pth [2023-12-26 17:49:01,747][105692] Updated weights for policy 0, policy_version 336974 (0.0010) [2023-12-26 17:49:01,801][105692] Updated weights for policy 0, policy_version 336984 (0.0009) [2023-12-26 17:49:01,815][105585] KL-divergence is very high: 105.7648 [2023-12-26 17:49:01,822][105585] KL-divergence is very high: 108.7493 [2023-12-26 17:49:01,847][105585] KL-divergence is very high: 168.0682 [2023-12-26 17:49:01,859][105692] Updated weights for policy 0, policy_version 336994 (0.0008) [2023-12-26 17:49:01,865][105585] KL-divergence is very high: 130.0363 [2023-12-26 17:49:01,872][105585] KL-divergence is very high: 128.2997 [2023-12-26 17:49:01,881][105620] Updated weights for policy 1, policy_version 337329 (0.0005) [2023-12-26 17:49:01,939][105620] Updated weights for policy 1, policy_version 337339 (0.0006) [2023-12-26 17:49:01,996][105620] Updated weights for policy 1, policy_version 337349 (0.0005) [2023-12-26 17:49:02,641][105620] Updated weights for policy 1, policy_version 337359 (0.0008) [2023-12-26 17:49:02,661][105585] KL-divergence is very high: 100.1815 [2023-12-26 17:49:02,684][105692] Updated weights for policy 0, policy_version 337004 (0.0008) [2023-12-26 17:49:02,698][105620] Updated weights for policy 1, policy_version 337369 (0.0007) [2023-12-26 17:49:02,732][105692] Updated weights for policy 0, policy_version 337014 (0.0006) [2023-12-26 17:49:02,754][105620] Updated weights for policy 1, policy_version 337379 (0.0008) [2023-12-26 17:49:02,784][105692] Updated weights for policy 0, policy_version 337024 (0.0006) [2023-12-26 17:49:03,415][105620] Updated weights for policy 1, policy_version 337389 (0.0006) [2023-12-26 17:49:03,466][105620] Updated weights for policy 1, policy_version 337399 (0.0005) [2023-12-26 17:49:03,515][105620] Updated weights for policy 1, policy_version 337409 (0.0005) [2023-12-26 17:49:03,590][105692] Updated weights for policy 0, policy_version 337034 (0.0008) [2023-12-26 17:49:03,638][105692] Updated weights for policy 0, policy_version 337044 (0.0008) [2023-12-26 17:49:03,685][105692] Updated weights for policy 0, policy_version 337054 (0.0009) [2023-12-26 17:49:03,730][105692] Updated weights for policy 0, policy_version 337064 (0.0008) [2023-12-26 17:49:04,187][105620] Updated weights for policy 1, policy_version 337419 (0.0007) [2023-12-26 17:49:04,249][105620] Updated weights for policy 1, policy_version 337429 (0.0009) [2023-12-26 17:49:04,298][105620] Updated weights for policy 1, policy_version 337439 (0.0009) [2023-12-26 17:49:04,468][105692] Updated weights for policy 0, policy_version 337074 (0.0005) [2023-12-26 17:49:04,540][105692] Updated weights for policy 0, policy_version 337084 (0.0005) [2023-12-26 17:49:04,588][105692] Updated weights for policy 0, policy_version 337094 (0.0008) [2023-12-26 17:49:05,141][105620] Updated weights for policy 1, policy_version 337449 (0.0009) [2023-12-26 17:49:05,169][105692] Updated weights for policy 0, policy_version 337104 (0.0006) [2023-12-26 17:49:05,197][105620] Updated weights for policy 1, policy_version 337459 (0.0009) [2023-12-26 17:49:05,222][105692] Updated weights for policy 0, policy_version 337114 (0.0005) [2023-12-26 17:49:05,249][105620] Updated weights for policy 1, policy_version 337469 (0.0008) [2023-12-26 17:49:05,285][105692] Updated weights for policy 0, policy_version 337124 (0.0005) [2023-12-26 17:49:05,302][105620] Updated weights for policy 1, policy_version 337479 (0.0009) [2023-12-26 17:49:05,936][105692] Updated weights for policy 0, policy_version 337134 (0.0008) [2023-12-26 17:49:05,992][105692] Updated weights for policy 0, policy_version 337144 (0.0008) [2023-12-26 17:49:06,046][105692] Updated weights for policy 0, policy_version 337154 (0.0008) [2023-12-26 17:49:06,047][105620] Updated weights for policy 1, policy_version 337489 (0.0007) [2023-12-26 17:49:06,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19114.7, 300 sec: 19383.1). Total num frames: 172720128. Throughput: 0: 9676.9, 1: 9642.9. Samples: 172716460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:49:06,063][104569] Avg episode reward: [(0, '7165.786'), (1, '3364.714')] [2023-12-26 17:49:06,108][105620] Updated weights for policy 1, policy_version 337499 (0.0008) [2023-12-26 17:49:06,168][105620] Updated weights for policy 1, policy_version 337509 (0.0009) [2023-12-26 17:49:06,797][105692] Updated weights for policy 0, policy_version 337164 (0.0005) [2023-12-26 17:49:06,863][105692] Updated weights for policy 0, policy_version 337174 (0.0007) [2023-12-26 17:49:06,925][105692] Updated weights for policy 0, policy_version 337184 (0.0009) [2023-12-26 17:49:06,968][105620] Updated weights for policy 1, policy_version 337519 (0.0007) [2023-12-26 17:49:07,030][105620] Updated weights for policy 1, policy_version 337529 (0.0009) [2023-12-26 17:49:07,099][105620] Updated weights for policy 1, policy_version 337539 (0.0009) [2023-12-26 17:49:07,518][105692] Updated weights for policy 0, policy_version 337194 (0.0007) [2023-12-26 17:49:07,567][105692] Updated weights for policy 0, policy_version 337204 (0.0009) [2023-12-26 17:49:07,623][105692] Updated weights for policy 0, policy_version 337214 (0.0009) [2023-12-26 17:49:07,672][105692] Updated weights for policy 0, policy_version 337224 (0.0008) [2023-12-26 17:49:07,879][105620] Updated weights for policy 1, policy_version 337549 (0.0010) [2023-12-26 17:49:07,930][105620] Updated weights for policy 1, policy_version 337559 (0.0008) [2023-12-26 17:49:07,985][105620] Updated weights for policy 1, policy_version 337569 (0.0009) [2023-12-26 17:49:08,436][105692] Updated weights for policy 0, policy_version 337234 (0.0006) [2023-12-26 17:49:08,490][105692] Updated weights for policy 0, policy_version 337244 (0.0005) [2023-12-26 17:49:08,540][105692] Updated weights for policy 0, policy_version 337254 (0.0005) [2023-12-26 17:49:08,726][105620] Updated weights for policy 1, policy_version 337579 (0.0009) [2023-12-26 17:49:08,781][105620] Updated weights for policy 1, policy_version 337589 (0.0008) [2023-12-26 17:49:08,846][105620] Updated weights for policy 1, policy_version 337599 (0.0008) [2023-12-26 17:49:09,213][105692] Updated weights for policy 0, policy_version 337264 (0.0009) [2023-12-26 17:49:09,280][105692] Updated weights for policy 0, policy_version 337274 (0.0007) [2023-12-26 17:49:09,348][105692] Updated weights for policy 0, policy_version 337284 (0.0007) [2023-12-26 17:49:09,599][105620] Updated weights for policy 1, policy_version 337609 (0.0009) [2023-12-26 17:49:09,650][105620] Updated weights for policy 1, policy_version 337619 (0.0008) [2023-12-26 17:49:09,711][105620] Updated weights for policy 1, policy_version 337629 (0.0009) [2023-12-26 17:49:09,774][105620] Updated weights for policy 1, policy_version 337639 (0.0009) [2023-12-26 17:49:10,091][105692] Updated weights for policy 0, policy_version 337294 (0.0009) [2023-12-26 17:49:10,151][105692] Updated weights for policy 0, policy_version 337304 (0.0008) [2023-12-26 17:49:10,214][105692] Updated weights for policy 0, policy_version 337314 (0.0008) [2023-12-26 17:49:10,525][105620] Updated weights for policy 1, policy_version 337649 (0.0008) [2023-12-26 17:49:10,590][105620] Updated weights for policy 1, policy_version 337659 (0.0007) [2023-12-26 17:49:10,652][105620] Updated weights for policy 1, policy_version 337669 (0.0008) [2023-12-26 17:49:10,948][105692] Updated weights for policy 0, policy_version 337324 (0.0009) [2023-12-26 17:49:11,014][105692] Updated weights for policy 0, policy_version 337334 (0.0011) [2023-12-26 17:49:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 172818432. Throughput: 0: 9695.8, 1: 9562.6. Samples: 172830668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:49:11,062][104569] Avg episode reward: [(0, '9001.318'), (1, '7451.481')] [2023-12-26 17:49:11,077][105692] Updated weights for policy 0, policy_version 337344 (0.0010) [2023-12-26 17:49:11,442][105620] Updated weights for policy 1, policy_version 337679 (0.0007) [2023-12-26 17:49:11,514][105620] Updated weights for policy 1, policy_version 337689 (0.0005) [2023-12-26 17:49:11,575][105620] Updated weights for policy 1, policy_version 337699 (0.0006) [2023-12-26 17:49:11,819][105692] Updated weights for policy 0, policy_version 337354 (0.0009) [2023-12-26 17:49:11,886][105692] Updated weights for policy 0, policy_version 337364 (0.0008) [2023-12-26 17:49:11,942][105692] Updated weights for policy 0, policy_version 337374 (0.0008) [2023-12-26 17:49:12,002][105692] Updated weights for policy 0, policy_version 337384 (0.0008) [2023-12-26 17:49:12,237][105620] Updated weights for policy 1, policy_version 337709 (0.0007) [2023-12-26 17:49:12,299][105620] Updated weights for policy 1, policy_version 337719 (0.0007) [2023-12-26 17:49:12,363][105620] Updated weights for policy 1, policy_version 337729 (0.0007) [2023-12-26 17:49:12,718][105692] Updated weights for policy 0, policy_version 337394 (0.0010) [2023-12-26 17:49:12,780][105692] Updated weights for policy 0, policy_version 337404 (0.0010) [2023-12-26 17:49:12,837][105692] Updated weights for policy 0, policy_version 337414 (0.0010) [2023-12-26 17:49:12,987][105620] Updated weights for policy 1, policy_version 337739 (0.0007) [2023-12-26 17:49:13,044][105620] Updated weights for policy 1, policy_version 337749 (0.0005) [2023-12-26 17:49:13,099][105620] Updated weights for policy 1, policy_version 337759 (0.0005) [2023-12-26 17:49:13,637][105692] Updated weights for policy 0, policy_version 337424 (0.0006) [2023-12-26 17:49:13,683][105692] Updated weights for policy 0, policy_version 337434 (0.0006) [2023-12-26 17:49:13,730][105692] Updated weights for policy 0, policy_version 337444 (0.0009) [2023-12-26 17:49:13,747][105620] Updated weights for policy 1, policy_version 337769 (0.0005) [2023-12-26 17:49:13,792][105620] Updated weights for policy 1, policy_version 337779 (0.0005) [2023-12-26 17:49:13,847][105620] Updated weights for policy 1, policy_version 337789 (0.0005) [2023-12-26 17:49:13,899][105620] Updated weights for policy 1, policy_version 337799 (0.0005) [2023-12-26 17:49:14,353][105692] Updated weights for policy 0, policy_version 337454 (0.0008) [2023-12-26 17:49:14,408][105692] Updated weights for policy 0, policy_version 337464 (0.0009) [2023-12-26 17:49:14,459][105692] Updated weights for policy 0, policy_version 337474 (0.0009) [2023-12-26 17:49:14,554][105620] Updated weights for policy 1, policy_version 337809 (0.0009) [2023-12-26 17:49:14,612][105620] Updated weights for policy 1, policy_version 337819 (0.0009) [2023-12-26 17:49:14,677][105620] Updated weights for policy 1, policy_version 337829 (0.0009) [2023-12-26 17:49:15,176][105692] Updated weights for policy 0, policy_version 337484 (0.0009) [2023-12-26 17:49:15,231][105692] Updated weights for policy 0, policy_version 337494 (0.0010) [2023-12-26 17:49:15,290][105692] Updated weights for policy 0, policy_version 337504 (0.0008) [2023-12-26 17:49:15,442][105620] Updated weights for policy 1, policy_version 337839 (0.0009) [2023-12-26 17:49:15,511][105620] Updated weights for policy 1, policy_version 337849 (0.0010) [2023-12-26 17:49:15,570][105620] Updated weights for policy 1, policy_version 337859 (0.0010) [2023-12-26 17:49:15,966][105692] Updated weights for policy 0, policy_version 337514 (0.0006) [2023-12-26 17:49:16,014][105692] Updated weights for policy 0, policy_version 337524 (0.0010) [2023-12-26 17:49:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 172916736. Throughput: 0: 9652.4, 1: 9574.5. Samples: 172890020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:49:16,063][104569] Avg episode reward: [(0, '9182.067'), (1, '9263.961')] [2023-12-26 17:49:16,070][105692] Updated weights for policy 0, policy_version 337534 (0.0005) [2023-12-26 17:49:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000337864_86499328.pth... [2023-12-26 17:49:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000336744_86212608.pth [2023-12-26 17:49:16,130][105692] Updated weights for policy 0, policy_version 337544 (0.0006) [2023-12-26 17:49:16,130][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000337544_86425600.pth... [2023-12-26 17:49:16,133][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000336392_86130688.pth [2023-12-26 17:49:16,338][105620] Updated weights for policy 1, policy_version 337869 (0.0010) [2023-12-26 17:49:16,390][105620] Updated weights for policy 1, policy_version 337879 (0.0010) [2023-12-26 17:49:16,456][105620] Updated weights for policy 1, policy_version 337889 (0.0010) [2023-12-26 17:49:16,680][105692] Updated weights for policy 0, policy_version 337554 (0.0005) [2023-12-26 17:49:16,732][105692] Updated weights for policy 0, policy_version 337564 (0.0005) [2023-12-26 17:49:16,781][105692] Updated weights for policy 0, policy_version 337574 (0.0006) [2023-12-26 17:49:17,070][105620] Updated weights for policy 1, policy_version 337899 (0.0010) [2023-12-26 17:49:17,142][105620] Updated weights for policy 1, policy_version 337909 (0.0010) [2023-12-26 17:49:17,204][105620] Updated weights for policy 1, policy_version 337919 (0.0009) [2023-12-26 17:49:17,386][105692] Updated weights for policy 0, policy_version 337584 (0.0008) [2023-12-26 17:49:17,433][105692] Updated weights for policy 0, policy_version 337594 (0.0009) [2023-12-26 17:49:17,487][105692] Updated weights for policy 0, policy_version 337604 (0.0009) [2023-12-26 17:49:17,897][105620] Updated weights for policy 1, policy_version 337929 (0.0006) [2023-12-26 17:49:17,947][105620] Updated weights for policy 1, policy_version 337939 (0.0009) [2023-12-26 17:49:17,993][105620] Updated weights for policy 1, policy_version 337949 (0.0008) [2023-12-26 17:49:18,040][105620] Updated weights for policy 1, policy_version 337959 (0.0009) [2023-12-26 17:49:18,252][105692] Updated weights for policy 0, policy_version 337614 (0.0009) [2023-12-26 17:49:18,299][105692] Updated weights for policy 0, policy_version 337624 (0.0009) [2023-12-26 17:49:18,357][105692] Updated weights for policy 0, policy_version 337634 (0.0009) [2023-12-26 17:49:18,851][105620] Updated weights for policy 1, policy_version 337969 (0.0006) [2023-12-26 17:49:18,915][105620] Updated weights for policy 1, policy_version 337979 (0.0005) [2023-12-26 17:49:18,976][105620] Updated weights for policy 1, policy_version 337989 (0.0008) [2023-12-26 17:49:19,128][105692] Updated weights for policy 0, policy_version 337644 (0.0010) [2023-12-26 17:49:19,190][105692] Updated weights for policy 0, policy_version 337654 (0.0009) [2023-12-26 17:49:19,253][105692] Updated weights for policy 0, policy_version 337664 (0.0009) [2023-12-26 17:49:19,631][105620] Updated weights for policy 1, policy_version 337999 (0.0008) [2023-12-26 17:49:19,697][105620] Updated weights for policy 1, policy_version 338009 (0.0009) [2023-12-26 17:49:19,760][105620] Updated weights for policy 1, policy_version 338019 (0.0009) [2023-12-26 17:49:20,021][105692] Updated weights for policy 0, policy_version 337674 (0.0009) [2023-12-26 17:49:20,080][105692] Updated weights for policy 0, policy_version 337684 (0.0009) [2023-12-26 17:49:20,134][105692] Updated weights for policy 0, policy_version 337694 (0.0009) [2023-12-26 17:49:20,193][105692] Updated weights for policy 0, policy_version 337704 (0.0009) [2023-12-26 17:49:20,522][105620] Updated weights for policy 1, policy_version 338029 (0.0010) [2023-12-26 17:49:20,583][105620] Updated weights for policy 1, policy_version 338039 (0.0008) [2023-12-26 17:49:20,649][105620] Updated weights for policy 1, policy_version 338049 (0.0009) [2023-12-26 17:49:20,914][105692] Updated weights for policy 0, policy_version 337714 (0.0009) [2023-12-26 17:49:20,972][105692] Updated weights for policy 0, policy_version 337724 (0.0009) [2023-12-26 17:49:21,040][105692] Updated weights for policy 0, policy_version 337734 (0.0009) [2023-12-26 17:49:21,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 173023232. Throughput: 0: 9691.8, 1: 9538.0. Samples: 173009204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:49:21,063][104569] Avg episode reward: [(0, '9182.341'), (1, '3542.089')] [2023-12-26 17:49:21,391][105620] Updated weights for policy 1, policy_version 338059 (0.0008) [2023-12-26 17:49:21,463][105620] Updated weights for policy 1, policy_version 338069 (0.0008) [2023-12-26 17:49:21,530][105620] Updated weights for policy 1, policy_version 338079 (0.0009) [2023-12-26 17:49:21,819][105692] Updated weights for policy 0, policy_version 337744 (0.0006) [2023-12-26 17:49:21,888][105692] Updated weights for policy 0, policy_version 337754 (0.0008) [2023-12-26 17:49:21,951][105692] Updated weights for policy 0, policy_version 337764 (0.0009) [2023-12-26 17:49:22,279][105620] Updated weights for policy 1, policy_version 338089 (0.0008) [2023-12-26 17:49:22,339][105620] Updated weights for policy 1, policy_version 338099 (0.0010) [2023-12-26 17:49:22,410][105620] Updated weights for policy 1, policy_version 338109 (0.0009) [2023-12-26 17:49:22,469][105620] Updated weights for policy 1, policy_version 338119 (0.0008) [2023-12-26 17:49:22,647][105692] Updated weights for policy 0, policy_version 337774 (0.0008) [2023-12-26 17:49:22,717][105692] Updated weights for policy 0, policy_version 337784 (0.0008) [2023-12-26 17:49:22,784][105692] Updated weights for policy 0, policy_version 337794 (0.0008) [2023-12-26 17:49:23,173][105620] Updated weights for policy 1, policy_version 338129 (0.0009) [2023-12-26 17:49:23,235][105620] Updated weights for policy 1, policy_version 338139 (0.0010) [2023-12-26 17:49:23,287][105620] Updated weights for policy 1, policy_version 338149 (0.0009) [2023-12-26 17:49:23,451][105692] Updated weights for policy 0, policy_version 337804 (0.0008) [2023-12-26 17:49:23,508][105692] Updated weights for policy 0, policy_version 337814 (0.0008) [2023-12-26 17:49:23,568][105692] Updated weights for policy 0, policy_version 337824 (0.0009) [2023-12-26 17:49:23,935][105620] Updated weights for policy 1, policy_version 338159 (0.0009) [2023-12-26 17:49:23,987][105620] Updated weights for policy 1, policy_version 338169 (0.0009) [2023-12-26 17:49:24,040][105620] Updated weights for policy 1, policy_version 338179 (0.0010) [2023-12-26 17:49:24,323][105692] Updated weights for policy 0, policy_version 337834 (0.0009) [2023-12-26 17:49:24,373][105692] Updated weights for policy 0, policy_version 337844 (0.0009) [2023-12-26 17:49:24,428][105692] Updated weights for policy 0, policy_version 337854 (0.0009) [2023-12-26 17:49:24,490][105692] Updated weights for policy 0, policy_version 337864 (0.0009) [2023-12-26 17:49:24,811][105620] Updated weights for policy 1, policy_version 338189 (0.0009) [2023-12-26 17:49:24,857][105620] Updated weights for policy 1, policy_version 338199 (0.0008) [2023-12-26 17:49:24,904][105620] Updated weights for policy 1, policy_version 338209 (0.0009) [2023-12-26 17:49:25,253][105692] Updated weights for policy 0, policy_version 337874 (0.0008) [2023-12-26 17:49:25,314][105692] Updated weights for policy 0, policy_version 337884 (0.0009) [2023-12-26 17:49:25,375][105692] Updated weights for policy 0, policy_version 337894 (0.0009) [2023-12-26 17:49:25,672][105620] Updated weights for policy 1, policy_version 338219 (0.0009) [2023-12-26 17:49:25,738][105620] Updated weights for policy 1, policy_version 338229 (0.0008) [2023-12-26 17:49:25,808][105620] Updated weights for policy 1, policy_version 338239 (0.0007) [2023-12-26 17:49:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 173113344. Throughput: 0: 9556.4, 1: 9571.6. Samples: 173121856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:49:26,063][104569] Avg episode reward: [(0, '9358.847'), (1, '3284.814')] [2023-12-26 17:49:26,159][105692] Updated weights for policy 0, policy_version 337904 (0.0009) [2023-12-26 17:49:26,217][105692] Updated weights for policy 0, policy_version 337914 (0.0009) [2023-12-26 17:49:26,279][105692] Updated weights for policy 0, policy_version 337924 (0.0009) [2023-12-26 17:49:26,524][105620] Updated weights for policy 1, policy_version 338249 (0.0008) [2023-12-26 17:49:26,582][105620] Updated weights for policy 1, policy_version 338259 (0.0008) [2023-12-26 17:49:26,638][105620] Updated weights for policy 1, policy_version 338269 (0.0008) [2023-12-26 17:49:26,688][105620] Updated weights for policy 1, policy_version 338279 (0.0007) [2023-12-26 17:49:26,917][105692] Updated weights for policy 0, policy_version 337934 (0.0010) [2023-12-26 17:49:26,964][105692] Updated weights for policy 0, policy_version 337944 (0.0010) [2023-12-26 17:49:27,018][105692] Updated weights for policy 0, policy_version 337954 (0.0010) [2023-12-26 17:49:27,345][105620] Updated weights for policy 1, policy_version 338289 (0.0008) [2023-12-26 17:49:27,404][105620] Updated weights for policy 1, policy_version 338299 (0.0008) [2023-12-26 17:49:27,465][105620] Updated weights for policy 1, policy_version 338309 (0.0006) [2023-12-26 17:49:27,764][105692] Updated weights for policy 0, policy_version 337964 (0.0010) [2023-12-26 17:49:27,825][105692] Updated weights for policy 0, policy_version 337974 (0.0007) [2023-12-26 17:49:27,895][105692] Updated weights for policy 0, policy_version 337984 (0.0005) [2023-12-26 17:49:28,111][105620] Updated weights for policy 1, policy_version 338319 (0.0006) [2023-12-26 17:49:28,174][105620] Updated weights for policy 1, policy_version 338329 (0.0005) [2023-12-26 17:49:28,220][105620] Updated weights for policy 1, policy_version 338339 (0.0005) [2023-12-26 17:49:28,516][105692] Updated weights for policy 0, policy_version 337994 (0.0006) [2023-12-26 17:49:28,566][105692] Updated weights for policy 0, policy_version 338004 (0.0007) [2023-12-26 17:49:28,618][105692] Updated weights for policy 0, policy_version 338014 (0.0005) [2023-12-26 17:49:28,671][105692] Updated weights for policy 0, policy_version 338024 (0.0005) [2023-12-26 17:49:28,890][105620] Updated weights for policy 1, policy_version 338349 (0.0007) [2023-12-26 17:49:28,949][105620] Updated weights for policy 1, policy_version 338359 (0.0008) [2023-12-26 17:49:29,007][105620] Updated weights for policy 1, policy_version 338369 (0.0008) [2023-12-26 17:49:29,318][105692] Updated weights for policy 0, policy_version 338034 (0.0011) [2023-12-26 17:49:29,382][105692] Updated weights for policy 0, policy_version 338044 (0.0011) [2023-12-26 17:49:29,443][105692] Updated weights for policy 0, policy_version 338054 (0.0010) [2023-12-26 17:49:29,784][105620] Updated weights for policy 1, policy_version 338379 (0.0009) [2023-12-26 17:49:29,853][105620] Updated weights for policy 1, policy_version 338389 (0.0008) [2023-12-26 17:49:29,911][105620] Updated weights for policy 1, policy_version 338399 (0.0007) [2023-12-26 17:49:30,200][105692] Updated weights for policy 0, policy_version 338064 (0.0010) [2023-12-26 17:49:30,244][105692] Updated weights for policy 0, policy_version 338074 (0.0010) [2023-12-26 17:49:30,292][105692] Updated weights for policy 0, policy_version 338084 (0.0010) [2023-12-26 17:49:30,694][105620] Updated weights for policy 1, policy_version 338409 (0.0008) [2023-12-26 17:49:30,743][105620] Updated weights for policy 1, policy_version 338419 (0.0005) [2023-12-26 17:49:30,805][105620] Updated weights for policy 1, policy_version 338429 (0.0007) [2023-12-26 17:49:30,861][105620] Updated weights for policy 1, policy_version 338439 (0.0005) [2023-12-26 17:49:31,008][105692] Updated weights for policy 0, policy_version 338094 (0.0010) [2023-12-26 17:49:31,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 173211648. Throughput: 0: 9616.3, 1: 9612.0. Samples: 173183056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:49:31,062][104569] Avg episode reward: [(0, '9177.122'), (1, '7261.641')] [2023-12-26 17:49:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000338440_86646784.pth... [2023-12-26 17:49:31,072][105692] Updated weights for policy 0, policy_version 338104 (0.0009) [2023-12-26 17:49:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000337320_86360064.pth [2023-12-26 17:49:31,138][105692] Updated weights for policy 0, policy_version 338114 (0.0012) [2023-12-26 17:49:31,175][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000338120_86573056.pth... [2023-12-26 17:49:31,181][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000336968_86278144.pth [2023-12-26 17:49:31,628][105620] Updated weights for policy 1, policy_version 338449 (0.0009) [2023-12-26 17:49:31,688][105620] Updated weights for policy 1, policy_version 338459 (0.0009) [2023-12-26 17:49:31,755][105620] Updated weights for policy 1, policy_version 338469 (0.0008) [2023-12-26 17:49:31,854][105692] Updated weights for policy 0, policy_version 338124 (0.0008) [2023-12-26 17:49:31,907][105692] Updated weights for policy 0, policy_version 338134 (0.0009) [2023-12-26 17:49:31,966][105692] Updated weights for policy 0, policy_version 338144 (0.0009) [2023-12-26 17:49:32,409][105620] Updated weights for policy 1, policy_version 338479 (0.0008) [2023-12-26 17:49:32,460][105620] Updated weights for policy 1, policy_version 338489 (0.0009) [2023-12-26 17:49:32,507][105620] Updated weights for policy 1, policy_version 338499 (0.0009) [2023-12-26 17:49:32,784][105692] Updated weights for policy 0, policy_version 338154 (0.0009) [2023-12-26 17:49:32,831][105692] Updated weights for policy 0, policy_version 338164 (0.0008) [2023-12-26 17:49:32,878][105692] Updated weights for policy 0, policy_version 338174 (0.0008) [2023-12-26 17:49:32,932][105692] Updated weights for policy 0, policy_version 338184 (0.0009) [2023-12-26 17:49:33,293][105620] Updated weights for policy 1, policy_version 338509 (0.0009) [2023-12-26 17:49:33,355][105620] Updated weights for policy 1, policy_version 338519 (0.0008) [2023-12-26 17:49:33,411][105620] Updated weights for policy 1, policy_version 338529 (0.0005) [2023-12-26 17:49:33,649][105692] Updated weights for policy 0, policy_version 338194 (0.0006) [2023-12-26 17:49:33,708][105692] Updated weights for policy 0, policy_version 338204 (0.0005) [2023-12-26 17:49:33,759][105692] Updated weights for policy 0, policy_version 338214 (0.0010) [2023-12-26 17:49:33,955][105620] Updated weights for policy 1, policy_version 338539 (0.0005) [2023-12-26 17:49:34,011][105620] Updated weights for policy 1, policy_version 338549 (0.0006) [2023-12-26 17:49:34,064][105620] Updated weights for policy 1, policy_version 338559 (0.0008) [2023-12-26 17:49:34,329][105692] Updated weights for policy 0, policy_version 338224 (0.0008) [2023-12-26 17:49:34,390][105692] Updated weights for policy 0, policy_version 338234 (0.0008) [2023-12-26 17:49:34,452][105692] Updated weights for policy 0, policy_version 338244 (0.0009) [2023-12-26 17:49:34,756][105620] Updated weights for policy 1, policy_version 338569 (0.0006) [2023-12-26 17:49:34,809][105620] Updated weights for policy 1, policy_version 338579 (0.0010) [2023-12-26 17:49:34,865][105620] Updated weights for policy 1, policy_version 338589 (0.0009) [2023-12-26 17:49:34,913][105620] Updated weights for policy 1, policy_version 338599 (0.0009) [2023-12-26 17:49:35,183][105692] Updated weights for policy 0, policy_version 338254 (0.0009) [2023-12-26 17:49:35,241][105692] Updated weights for policy 0, policy_version 338264 (0.0007) [2023-12-26 17:49:35,305][105692] Updated weights for policy 0, policy_version 338274 (0.0006) [2023-12-26 17:49:35,640][105620] Updated weights for policy 1, policy_version 338609 (0.0006) [2023-12-26 17:49:35,698][105620] Updated weights for policy 1, policy_version 338619 (0.0005) [2023-12-26 17:49:35,744][105620] Updated weights for policy 1, policy_version 338629 (0.0005) [2023-12-26 17:49:35,936][105692] Updated weights for policy 0, policy_version 338284 (0.0008) [2023-12-26 17:49:35,994][105692] Updated weights for policy 0, policy_version 338294 (0.0010) [2023-12-26 17:49:36,049][105692] Updated weights for policy 0, policy_version 338304 (0.0010) [2023-12-26 17:49:36,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 173309952. Throughput: 0: 9673.4, 1: 9615.1. Samples: 173300216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:49:36,062][104569] Avg episode reward: [(0, '9177.088'), (1, '9356.772')] [2023-12-26 17:49:36,404][105620] Updated weights for policy 1, policy_version 338639 (0.0006) [2023-12-26 17:49:36,456][105620] Updated weights for policy 1, policy_version 338649 (0.0005) [2023-12-26 17:49:36,516][105620] Updated weights for policy 1, policy_version 338659 (0.0006) [2023-12-26 17:49:36,805][105692] Updated weights for policy 0, policy_version 338314 (0.0010) [2023-12-26 17:49:36,854][105692] Updated weights for policy 0, policy_version 338324 (0.0010) [2023-12-26 17:49:36,903][105692] Updated weights for policy 0, policy_version 338334 (0.0010) [2023-12-26 17:49:36,958][105692] Updated weights for policy 0, policy_version 338344 (0.0007) [2023-12-26 17:49:37,173][105620] Updated weights for policy 1, policy_version 338669 (0.0007) [2023-12-26 17:49:37,235][105620] Updated weights for policy 1, policy_version 338679 (0.0009) [2023-12-26 17:49:37,302][105620] Updated weights for policy 1, policy_version 338689 (0.0008) [2023-12-26 17:49:37,729][105692] Updated weights for policy 0, policy_version 338354 (0.0010) [2023-12-26 17:49:37,787][105692] Updated weights for policy 0, policy_version 338364 (0.0010) [2023-12-26 17:49:37,845][105692] Updated weights for policy 0, policy_version 338374 (0.0010) [2023-12-26 17:49:38,089][105620] Updated weights for policy 1, policy_version 338699 (0.0008) [2023-12-26 17:49:38,150][105620] Updated weights for policy 1, policy_version 338709 (0.0009) [2023-12-26 17:49:38,203][105620] Updated weights for policy 1, policy_version 338719 (0.0008) [2023-12-26 17:49:38,479][105692] Updated weights for policy 0, policy_version 338384 (0.0008) [2023-12-26 17:49:38,546][105692] Updated weights for policy 0, policy_version 338394 (0.0006) [2023-12-26 17:49:38,607][105692] Updated weights for policy 0, policy_version 338404 (0.0008) [2023-12-26 17:49:38,965][105620] Updated weights for policy 1, policy_version 338729 (0.0007) [2023-12-26 17:49:39,013][105620] Updated weights for policy 1, policy_version 338739 (0.0008) [2023-12-26 17:49:39,070][105620] Updated weights for policy 1, policy_version 338749 (0.0009) [2023-12-26 17:49:39,130][105620] Updated weights for policy 1, policy_version 338759 (0.0008) [2023-12-26 17:49:39,297][105692] Updated weights for policy 0, policy_version 338414 (0.0008) [2023-12-26 17:49:39,360][105692] Updated weights for policy 0, policy_version 338424 (0.0008) [2023-12-26 17:49:39,431][105692] Updated weights for policy 0, policy_version 338434 (0.0008) [2023-12-26 17:49:40,000][105620] Updated weights for policy 1, policy_version 338769 (0.0008) [2023-12-26 17:49:40,060][105620] Updated weights for policy 1, policy_version 338779 (0.0007) [2023-12-26 17:49:40,067][105692] Updated weights for policy 0, policy_version 338444 (0.0008) [2023-12-26 17:49:40,122][105620] Updated weights for policy 1, policy_version 338789 (0.0007) [2023-12-26 17:49:40,128][105692] Updated weights for policy 0, policy_version 338454 (0.0007) [2023-12-26 17:49:40,187][105692] Updated weights for policy 0, policy_version 338464 (0.0008) [2023-12-26 17:49:40,888][105620] Updated weights for policy 1, policy_version 338799 (0.0009) [2023-12-26 17:49:40,924][105692] Updated weights for policy 0, policy_version 338474 (0.0008) [2023-12-26 17:49:40,951][105620] Updated weights for policy 1, policy_version 338809 (0.0009) [2023-12-26 17:49:40,979][105692] Updated weights for policy 0, policy_version 338484 (0.0006) [2023-12-26 17:49:41,004][105620] Updated weights for policy 1, policy_version 338819 (0.0009) [2023-12-26 17:49:41,039][105692] Updated weights for policy 0, policy_version 338494 (0.0006) [2023-12-26 17:49:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 173408256. Throughput: 0: 9742.7, 1: 9626.1. Samples: 173416900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:49:41,062][104569] Avg episode reward: [(0, '9358.729'), (1, '9356.935')] [2023-12-26 17:49:41,101][105692] Updated weights for policy 0, policy_version 338504 (0.0007) [2023-12-26 17:49:41,798][105620] Updated weights for policy 1, policy_version 338829 (0.0007) [2023-12-26 17:49:41,799][105692] Updated weights for policy 0, policy_version 338514 (0.0010) [2023-12-26 17:49:41,849][105692] Updated weights for policy 0, policy_version 338524 (0.0010) [2023-12-26 17:49:41,856][105620] Updated weights for policy 1, policy_version 338839 (0.0007) [2023-12-26 17:49:41,909][105692] Updated weights for policy 0, policy_version 338534 (0.0008) [2023-12-26 17:49:41,911][105620] Updated weights for policy 1, policy_version 338849 (0.0008) [2023-12-26 17:49:42,605][105692] Updated weights for policy 0, policy_version 338544 (0.0009) [2023-12-26 17:49:42,663][105692] Updated weights for policy 0, policy_version 338554 (0.0009) [2023-12-26 17:49:42,718][105620] Updated weights for policy 1, policy_version 338859 (0.0007) [2023-12-26 17:49:42,720][105692] Updated weights for policy 0, policy_version 338564 (0.0008) [2023-12-26 17:49:42,778][105620] Updated weights for policy 1, policy_version 338869 (0.0007) [2023-12-26 17:49:42,837][105620] Updated weights for policy 1, policy_version 338879 (0.0009) [2023-12-26 17:49:43,494][105692] Updated weights for policy 0, policy_version 338574 (0.0008) [2023-12-26 17:49:43,519][105620] Updated weights for policy 1, policy_version 338889 (0.0009) [2023-12-26 17:49:43,551][105692] Updated weights for policy 0, policy_version 338584 (0.0007) [2023-12-26 17:49:43,579][105620] Updated weights for policy 1, policy_version 338899 (0.0006) [2023-12-26 17:49:43,598][105692] Updated weights for policy 0, policy_version 338594 (0.0007) [2023-12-26 17:49:43,633][105620] Updated weights for policy 1, policy_version 338909 (0.0006) [2023-12-26 17:49:43,682][105620] Updated weights for policy 1, policy_version 338919 (0.0009) [2023-12-26 17:49:44,293][105692] Updated weights for policy 0, policy_version 338605 (0.0008) [2023-12-26 17:49:44,354][105692] Updated weights for policy 0, policy_version 338615 (0.0008) [2023-12-26 17:49:44,397][105620] Updated weights for policy 1, policy_version 338929 (0.0006) [2023-12-26 17:49:44,414][105692] Updated weights for policy 0, policy_version 338625 (0.0009) [2023-12-26 17:49:44,459][105620] Updated weights for policy 1, policy_version 338939 (0.0006) [2023-12-26 17:49:44,522][105620] Updated weights for policy 1, policy_version 338949 (0.0005) [2023-12-26 17:49:45,120][105620] Updated weights for policy 1, policy_version 338959 (0.0008) [2023-12-26 17:49:45,188][105620] Updated weights for policy 1, policy_version 338969 (0.0008) [2023-12-26 17:49:45,241][105692] Updated weights for policy 0, policy_version 338635 (0.0009) [2023-12-26 17:49:45,251][105620] Updated weights for policy 1, policy_version 338979 (0.0008) [2023-12-26 17:49:45,300][105692] Updated weights for policy 0, policy_version 338645 (0.0011) [2023-12-26 17:49:45,359][105692] Updated weights for policy 0, policy_version 338655 (0.0010) [2023-12-26 17:49:46,020][105620] Updated weights for policy 1, policy_version 338989 (0.0006) [2023-12-26 17:49:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.3, 300 sec: 19355.3). Total num frames: 173498368. Throughput: 0: 9752.4, 1: 9606.0. Samples: 173473104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:49:46,062][104569] Avg episode reward: [(0, '9268.957'), (1, '9357.220')] [2023-12-26 17:49:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000338664_86712320.pth... [2023-12-26 17:49:46,068][105620] Updated weights for policy 1, policy_version 338999 (0.0008) [2023-12-26 17:49:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000337544_86425600.pth [2023-12-26 17:49:46,094][105692] Updated weights for policy 0, policy_version 338665 (0.0010) [2023-12-26 17:49:46,115][105620] Updated weights for policy 1, policy_version 339009 (0.0007) [2023-12-26 17:49:46,145][105692] Updated weights for policy 0, policy_version 338675 (0.0010) [2023-12-26 17:49:46,151][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000339016_86794240.pth... [2023-12-26 17:49:46,154][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000337864_86499328.pth [2023-12-26 17:49:46,193][105692] Updated weights for policy 0, policy_version 338685 (0.0010) [2023-12-26 17:49:46,251][105692] Updated weights for policy 0, policy_version 338695 (0.0010) [2023-12-26 17:49:46,822][105620] Updated weights for policy 1, policy_version 339019 (0.0006) [2023-12-26 17:49:46,869][105620] Updated weights for policy 1, policy_version 339029 (0.0005) [2023-12-26 17:49:46,922][105620] Updated weights for policy 1, policy_version 339039 (0.0005) [2023-12-26 17:49:46,997][105692] Updated weights for policy 0, policy_version 338705 (0.0008) [2023-12-26 17:49:47,058][105692] Updated weights for policy 0, policy_version 338715 (0.0005) [2023-12-26 17:49:47,109][105692] Updated weights for policy 0, policy_version 338725 (0.0008) [2023-12-26 17:49:47,459][105620] Updated weights for policy 1, policy_version 339049 (0.0005) [2023-12-26 17:49:47,519][105620] Updated weights for policy 1, policy_version 339059 (0.0006) [2023-12-26 17:49:47,570][105620] Updated weights for policy 1, policy_version 339069 (0.0010) [2023-12-26 17:49:47,620][105620] Updated weights for policy 1, policy_version 339079 (0.0010) [2023-12-26 17:49:47,851][105692] Updated weights for policy 0, policy_version 338735 (0.0007) [2023-12-26 17:49:47,916][105692] Updated weights for policy 0, policy_version 338745 (0.0005) [2023-12-26 17:49:47,972][105692] Updated weights for policy 0, policy_version 338755 (0.0005) [2023-12-26 17:49:48,363][105620] Updated weights for policy 1, policy_version 339089 (0.0010) [2023-12-26 17:49:48,421][105620] Updated weights for policy 1, policy_version 339099 (0.0010) [2023-12-26 17:49:48,476][105620] Updated weights for policy 1, policy_version 339109 (0.0005) [2023-12-26 17:49:48,481][105692] Updated weights for policy 0, policy_version 338765 (0.0006) [2023-12-26 17:49:48,537][105692] Updated weights for policy 0, policy_version 338775 (0.0006) [2023-12-26 17:49:48,600][105692] Updated weights for policy 0, policy_version 338785 (0.0008) [2023-12-26 17:49:49,117][105620] Updated weights for policy 1, policy_version 339119 (0.0009) [2023-12-26 17:49:49,186][105620] Updated weights for policy 1, policy_version 339129 (0.0009) [2023-12-26 17:49:49,253][105620] Updated weights for policy 1, policy_version 339139 (0.0008) [2023-12-26 17:49:49,331][105692] Updated weights for policy 0, policy_version 338795 (0.0009) [2023-12-26 17:49:49,395][105692] Updated weights for policy 0, policy_version 338805 (0.0008) [2023-12-26 17:49:49,448][105692] Updated weights for policy 0, policy_version 338815 (0.0008) [2023-12-26 17:49:49,899][105620] Updated weights for policy 1, policy_version 339149 (0.0007) [2023-12-26 17:49:49,966][105620] Updated weights for policy 1, policy_version 339159 (0.0007) [2023-12-26 17:49:50,036][105620] Updated weights for policy 1, policy_version 339169 (0.0007) [2023-12-26 17:49:50,102][105692] Updated weights for policy 0, policy_version 338825 (0.0008) [2023-12-26 17:49:50,165][105692] Updated weights for policy 0, policy_version 338835 (0.0010) [2023-12-26 17:49:50,227][105692] Updated weights for policy 0, policy_version 338845 (0.0009) [2023-12-26 17:49:50,284][105692] Updated weights for policy 0, policy_version 338855 (0.0010) [2023-12-26 17:49:50,669][105620] Updated weights for policy 1, policy_version 339179 (0.0007) [2023-12-26 17:49:50,738][105620] Updated weights for policy 1, policy_version 339189 (0.0009) [2023-12-26 17:49:50,806][105620] Updated weights for policy 1, policy_version 339199 (0.0009) [2023-12-26 17:49:51,046][105692] Updated weights for policy 0, policy_version 338865 (0.0009) [2023-12-26 17:49:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 173604864. Throughput: 0: 9808.6, 1: 9695.0. Samples: 173594120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:49:51,062][104569] Avg episode reward: [(0, '9090.916'), (1, '9178.900')] [2023-12-26 17:49:51,110][105692] Updated weights for policy 0, policy_version 338875 (0.0008) [2023-12-26 17:49:51,178][105692] Updated weights for policy 0, policy_version 338885 (0.0010) [2023-12-26 17:49:51,479][105620] Updated weights for policy 1, policy_version 339209 (0.0008) [2023-12-26 17:49:51,538][105620] Updated weights for policy 1, policy_version 339219 (0.0010) [2023-12-26 17:49:51,600][105620] Updated weights for policy 1, policy_version 339229 (0.0010) [2023-12-26 17:49:51,665][105620] Updated weights for policy 1, policy_version 339239 (0.0007) [2023-12-26 17:49:52,046][105692] Updated weights for policy 0, policy_version 338895 (0.0009) [2023-12-26 17:49:52,113][105692] Updated weights for policy 0, policy_version 338905 (0.0009) [2023-12-26 17:49:52,172][105692] Updated weights for policy 0, policy_version 338915 (0.0009) [2023-12-26 17:49:52,312][105620] Updated weights for policy 1, policy_version 339249 (0.0009) [2023-12-26 17:49:52,371][105620] Updated weights for policy 1, policy_version 339259 (0.0010) [2023-12-26 17:49:52,428][105620] Updated weights for policy 1, policy_version 339269 (0.0008) [2023-12-26 17:49:52,921][105692] Updated weights for policy 0, policy_version 338925 (0.0008) [2023-12-26 17:49:52,968][105692] Updated weights for policy 0, policy_version 338935 (0.0009) [2023-12-26 17:49:53,021][105692] Updated weights for policy 0, policy_version 338945 (0.0009) [2023-12-26 17:49:53,152][105620] Updated weights for policy 1, policy_version 339279 (0.0006) [2023-12-26 17:49:53,215][105620] Updated weights for policy 1, policy_version 339289 (0.0005) [2023-12-26 17:49:53,284][105620] Updated weights for policy 1, policy_version 339299 (0.0005) [2023-12-26 17:49:53,815][105692] Updated weights for policy 0, policy_version 338955 (0.0009) [2023-12-26 17:49:53,866][105692] Updated weights for policy 0, policy_version 338965 (0.0008) [2023-12-26 17:49:53,929][105692] Updated weights for policy 0, policy_version 338975 (0.0008) [2023-12-26 17:49:53,951][105620] Updated weights for policy 1, policy_version 339309 (0.0010) [2023-12-26 17:49:54,016][105620] Updated weights for policy 1, policy_version 339319 (0.0010) [2023-12-26 17:49:54,072][105620] Updated weights for policy 1, policy_version 339329 (0.0010) [2023-12-26 17:49:54,681][105692] Updated weights for policy 0, policy_version 338985 (0.0006) [2023-12-26 17:49:54,724][105620] Updated weights for policy 1, policy_version 339339 (0.0010) [2023-12-26 17:49:54,739][105692] Updated weights for policy 0, policy_version 338995 (0.0006) [2023-12-26 17:49:54,770][105620] Updated weights for policy 1, policy_version 339349 (0.0005) [2023-12-26 17:49:54,797][105692] Updated weights for policy 0, policy_version 339005 (0.0008) [2023-12-26 17:49:54,818][105620] Updated weights for policy 1, policy_version 339359 (0.0005) [2023-12-26 17:49:54,854][105692] Updated weights for policy 0, policy_version 339015 (0.0006) [2023-12-26 17:49:55,471][105692] Updated weights for policy 0, policy_version 339025 (0.0008) [2023-12-26 17:49:55,534][105692] Updated weights for policy 0, policy_version 339035 (0.0008) [2023-12-26 17:49:55,602][105692] Updated weights for policy 0, policy_version 339045 (0.0007) [2023-12-26 17:49:55,625][105620] Updated weights for policy 1, policy_version 339369 (0.0008) [2023-12-26 17:49:55,695][105620] Updated weights for policy 1, policy_version 339379 (0.0010) [2023-12-26 17:49:55,753][105620] Updated weights for policy 1, policy_version 339389 (0.0008) [2023-12-26 17:49:55,807][105620] Updated weights for policy 1, policy_version 339399 (0.0005) [2023-12-26 17:49:56,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 173703168. Throughput: 0: 9743.4, 1: 9807.6. Samples: 173710464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:49:56,062][104569] Avg episode reward: [(0, '9180.794'), (1, '9178.566')] [2023-12-26 17:49:56,354][105692] Updated weights for policy 0, policy_version 339055 (0.0009) [2023-12-26 17:49:56,407][105692] Updated weights for policy 0, policy_version 339065 (0.0009) [2023-12-26 17:49:56,470][105692] Updated weights for policy 0, policy_version 339076 (0.0011) [2023-12-26 17:49:56,516][105620] Updated weights for policy 1, policy_version 339409 (0.0005) [2023-12-26 17:49:56,578][105620] Updated weights for policy 1, policy_version 339419 (0.0006) [2023-12-26 17:49:56,636][105620] Updated weights for policy 1, policy_version 339429 (0.0006) [2023-12-26 17:49:57,204][105692] Updated weights for policy 0, policy_version 339086 (0.0008) [2023-12-26 17:49:57,243][105620] Updated weights for policy 1, policy_version 339439 (0.0010) [2023-12-26 17:49:57,249][105692] Updated weights for policy 0, policy_version 339096 (0.0006) [2023-12-26 17:49:57,294][105620] Updated weights for policy 1, policy_version 339449 (0.0010) [2023-12-26 17:49:57,304][105692] Updated weights for policy 0, policy_version 339106 (0.0006) [2023-12-26 17:49:57,355][105620] Updated weights for policy 1, policy_version 339459 (0.0010) [2023-12-26 17:49:57,934][105692] Updated weights for policy 0, policy_version 339116 (0.0008) [2023-12-26 17:49:57,984][105692] Updated weights for policy 0, policy_version 339126 (0.0005) [2023-12-26 17:49:58,035][105692] Updated weights for policy 0, policy_version 339136 (0.0005) [2023-12-26 17:49:58,071][105620] Updated weights for policy 1, policy_version 339469 (0.0010) [2023-12-26 17:49:58,136][105620] Updated weights for policy 1, policy_version 339479 (0.0007) [2023-12-26 17:49:58,201][105620] Updated weights for policy 1, policy_version 339489 (0.0006) [2023-12-26 17:49:58,848][105692] Updated weights for policy 0, policy_version 339146 (0.0006) [2023-12-26 17:49:58,904][105692] Updated weights for policy 0, policy_version 339156 (0.0008) [2023-12-26 17:49:58,957][105620] Updated weights for policy 1, policy_version 339499 (0.0006) [2023-12-26 17:49:58,969][105692] Updated weights for policy 0, policy_version 339166 (0.0008) [2023-12-26 17:49:59,012][105620] Updated weights for policy 1, policy_version 339509 (0.0008) [2023-12-26 17:49:59,027][105692] Updated weights for policy 0, policy_version 339176 (0.0007) [2023-12-26 17:49:59,077][105620] Updated weights for policy 1, policy_version 339519 (0.0007) [2023-12-26 17:49:59,723][105620] Updated weights for policy 1, policy_version 339529 (0.0008) [2023-12-26 17:49:59,784][105620] Updated weights for policy 1, policy_version 339539 (0.0009) [2023-12-26 17:49:59,849][105620] Updated weights for policy 1, policy_version 339549 (0.0008) [2023-12-26 17:49:59,889][105692] Updated weights for policy 0, policy_version 339186 (0.0007) [2023-12-26 17:49:59,910][105620] Updated weights for policy 1, policy_version 339559 (0.0006) [2023-12-26 17:49:59,948][105692] Updated weights for policy 0, policy_version 339196 (0.0009) [2023-12-26 17:50:00,007][105692] Updated weights for policy 0, policy_version 339206 (0.0009) [2023-12-26 17:50:00,510][105620] Updated weights for policy 1, policy_version 339569 (0.0009) [2023-12-26 17:50:00,564][105620] Updated weights for policy 1, policy_version 339581 (0.0011) [2023-12-26 17:50:00,620][105620] Updated weights for policy 1, policy_version 339591 (0.0009) [2023-12-26 17:50:00,741][105692] Updated weights for policy 0, policy_version 339216 (0.0006) [2023-12-26 17:50:00,796][105692] Updated weights for policy 0, policy_version 339226 (0.0007) [2023-12-26 17:50:00,850][105692] Updated weights for policy 0, policy_version 339236 (0.0006) [2023-12-26 17:50:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 173801472. Throughput: 0: 9778.6, 1: 9768.2. Samples: 173769624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:50:01,062][104569] Avg episode reward: [(0, '9267.646'), (1, '9266.746')] [2023-12-26 17:50:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000339240_86859776.pth... [2023-12-26 17:50:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000339592_86941696.pth... [2023-12-26 17:50:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000338440_86646784.pth [2023-12-26 17:50:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000338120_86573056.pth [2023-12-26 17:50:01,471][105620] Updated weights for policy 1, policy_version 339601 (0.0008) [2023-12-26 17:50:01,537][105620] Updated weights for policy 1, policy_version 339611 (0.0009) [2023-12-26 17:50:01,588][105692] Updated weights for policy 0, policy_version 339246 (0.0007) [2023-12-26 17:50:01,594][105620] Updated weights for policy 1, policy_version 339621 (0.0008) [2023-12-26 17:50:01,648][105692] Updated weights for policy 0, policy_version 339256 (0.0007) [2023-12-26 17:50:01,708][105692] Updated weights for policy 0, policy_version 339266 (0.0006) [2023-12-26 17:50:02,287][105692] Updated weights for policy 0, policy_version 339276 (0.0007) [2023-12-26 17:50:02,351][105692] Updated weights for policy 0, policy_version 339286 (0.0009) [2023-12-26 17:50:02,411][105692] Updated weights for policy 0, policy_version 339296 (0.0008) [2023-12-26 17:50:02,453][105620] Updated weights for policy 1, policy_version 339631 (0.0008) [2023-12-26 17:50:02,521][105620] Updated weights for policy 1, policy_version 339641 (0.0008) [2023-12-26 17:50:02,585][105620] Updated weights for policy 1, policy_version 339651 (0.0008) [2023-12-26 17:50:03,082][105692] Updated weights for policy 0, policy_version 339306 (0.0006) [2023-12-26 17:50:03,133][105692] Updated weights for policy 0, policy_version 339316 (0.0009) [2023-12-26 17:50:03,186][105692] Updated weights for policy 0, policy_version 339326 (0.0009) [2023-12-26 17:50:03,239][105692] Updated weights for policy 0, policy_version 339336 (0.0009) [2023-12-26 17:50:03,347][105620] Updated weights for policy 1, policy_version 339661 (0.0009) [2023-12-26 17:50:03,396][105620] Updated weights for policy 1, policy_version 339671 (0.0005) [2023-12-26 17:50:03,448][105620] Updated weights for policy 1, policy_version 339681 (0.0008) [2023-12-26 17:50:03,984][105692] Updated weights for policy 0, policy_version 339346 (0.0009) [2023-12-26 17:50:04,046][105692] Updated weights for policy 0, policy_version 339356 (0.0009) [2023-12-26 17:50:04,118][105692] Updated weights for policy 0, policy_version 339366 (0.0009) [2023-12-26 17:50:04,210][105620] Updated weights for policy 1, policy_version 339691 (0.0009) [2023-12-26 17:50:04,262][105620] Updated weights for policy 1, policy_version 339701 (0.0009) [2023-12-26 17:50:04,326][105620] Updated weights for policy 1, policy_version 339711 (0.0009) [2023-12-26 17:50:04,859][105692] Updated weights for policy 0, policy_version 339376 (0.0009) [2023-12-26 17:50:04,917][105692] Updated weights for policy 0, policy_version 339386 (0.0009) [2023-12-26 17:50:04,968][105692] Updated weights for policy 0, policy_version 339396 (0.0009) [2023-12-26 17:50:05,066][105620] Updated weights for policy 1, policy_version 339721 (0.0010) [2023-12-26 17:50:05,128][105620] Updated weights for policy 1, policy_version 339731 (0.0009) [2023-12-26 17:50:05,181][105620] Updated weights for policy 1, policy_version 339741 (0.0008) [2023-12-26 17:50:05,242][105620] Updated weights for policy 1, policy_version 339751 (0.0009) [2023-12-26 17:50:05,718][105692] Updated weights for policy 0, policy_version 339406 (0.0009) [2023-12-26 17:50:05,764][105692] Updated weights for policy 0, policy_version 339416 (0.0008) [2023-12-26 17:50:05,811][105692] Updated weights for policy 0, policy_version 339427 (0.0009) [2023-12-26 17:50:05,975][105620] Updated weights for policy 1, policy_version 339761 (0.0009) [2023-12-26 17:50:06,029][105620] Updated weights for policy 1, policy_version 339771 (0.0009) [2023-12-26 17:50:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 173891584. Throughput: 0: 9698.9, 1: 9730.9. Samples: 173883544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 17:50:06,063][104569] Avg episode reward: [(0, '9086.747'), (1, '9266.727')] [2023-12-26 17:50:06,076][105620] Updated weights for policy 1, policy_version 339781 (0.0008) [2023-12-26 17:50:06,603][105692] Updated weights for policy 0, policy_version 339437 (0.0009) [2023-12-26 17:50:06,666][105692] Updated weights for policy 0, policy_version 339447 (0.0009) [2023-12-26 17:50:06,728][105692] Updated weights for policy 0, policy_version 339457 (0.0009) [2023-12-26 17:50:06,854][105620] Updated weights for policy 1, policy_version 339791 (0.0009) [2023-12-26 17:50:06,909][105620] Updated weights for policy 1, policy_version 339801 (0.0009) [2023-12-26 17:50:06,966][105620] Updated weights for policy 1, policy_version 339811 (0.0009) [2023-12-26 17:50:07,476][105692] Updated weights for policy 0, policy_version 339467 (0.0009) [2023-12-26 17:50:07,527][105692] Updated weights for policy 0, policy_version 339477 (0.0009) [2023-12-26 17:50:07,575][105692] Updated weights for policy 0, policy_version 339487 (0.0009) [2023-12-26 17:50:07,723][105620] Updated weights for policy 1, policy_version 339821 (0.0009) [2023-12-26 17:50:07,769][105620] Updated weights for policy 1, policy_version 339831 (0.0008) [2023-12-26 17:50:07,816][105620] Updated weights for policy 1, policy_version 339841 (0.0009) [2023-12-26 17:50:08,340][105692] Updated weights for policy 0, policy_version 339497 (0.0009) [2023-12-26 17:50:08,399][105692] Updated weights for policy 0, policy_version 339507 (0.0009) [2023-12-26 17:50:08,464][105692] Updated weights for policy 0, policy_version 339517 (0.0009) [2023-12-26 17:50:08,535][105692] Updated weights for policy 0, policy_version 339527 (0.0009) [2023-12-26 17:50:08,554][105620] Updated weights for policy 1, policy_version 339851 (0.0008) [2023-12-26 17:50:08,619][105620] Updated weights for policy 1, policy_version 339861 (0.0008) [2023-12-26 17:50:08,681][105620] Updated weights for policy 1, policy_version 339871 (0.0007) [2023-12-26 17:50:09,252][105692] Updated weights for policy 0, policy_version 339537 (0.0009) [2023-12-26 17:50:09,314][105692] Updated weights for policy 0, policy_version 339547 (0.0009) [2023-12-26 17:50:09,381][105692] Updated weights for policy 0, policy_version 339557 (0.0008) [2023-12-26 17:50:09,429][105620] Updated weights for policy 1, policy_version 339881 (0.0009) [2023-12-26 17:50:09,499][105620] Updated weights for policy 1, policy_version 339891 (0.0009) [2023-12-26 17:50:09,552][105620] Updated weights for policy 1, policy_version 339901 (0.0009) [2023-12-26 17:50:09,610][105620] Updated weights for policy 1, policy_version 339911 (0.0006) [2023-12-26 17:50:10,174][105692] Updated weights for policy 0, policy_version 339567 (0.0006) [2023-12-26 17:50:10,232][105692] Updated weights for policy 0, policy_version 339577 (0.0006) [2023-12-26 17:50:10,292][105692] Updated weights for policy 0, policy_version 339587 (0.0006) [2023-12-26 17:50:10,316][105620] Updated weights for policy 1, policy_version 339921 (0.0009) [2023-12-26 17:50:10,381][105620] Updated weights for policy 1, policy_version 339931 (0.0009) [2023-12-26 17:50:10,435][105620] Updated weights for policy 1, policy_version 339941 (0.0009) [2023-12-26 17:50:10,981][105692] Updated weights for policy 0, policy_version 339597 (0.0007) [2023-12-26 17:50:11,035][105692] Updated weights for policy 0, policy_version 339607 (0.0008) [2023-12-26 17:50:11,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 173981696. Throughput: 0: 9685.7, 1: 9716.6. Samples: 173994952. Policy #0 lag: (min: 25.0, avg: 48.8, max: 57.0) [2023-12-26 17:50:11,063][104569] Avg episode reward: [(0, '9177.871'), (1, '9267.025')] [2023-12-26 17:50:11,104][105692] Updated weights for policy 0, policy_version 339617 (0.0009) [2023-12-26 17:50:11,237][105620] Updated weights for policy 1, policy_version 339951 (0.0009) [2023-12-26 17:50:11,303][105620] Updated weights for policy 1, policy_version 339961 (0.0009) [2023-12-26 17:50:11,373][105620] Updated weights for policy 1, policy_version 339971 (0.0009) [2023-12-26 17:50:11,895][105692] Updated weights for policy 0, policy_version 339627 (0.0008) [2023-12-26 17:50:11,958][105692] Updated weights for policy 0, policy_version 339637 (0.0009) [2023-12-26 17:50:12,019][105692] Updated weights for policy 0, policy_version 339647 (0.0008) [2023-12-26 17:50:12,156][105620] Updated weights for policy 1, policy_version 339981 (0.0009) [2023-12-26 17:50:12,215][105620] Updated weights for policy 1, policy_version 339991 (0.0009) [2023-12-26 17:50:12,282][105620] Updated weights for policy 1, policy_version 340001 (0.0009) [2023-12-26 17:50:12,822][105692] Updated weights for policy 0, policy_version 339657 (0.0009) [2023-12-26 17:50:12,883][105692] Updated weights for policy 0, policy_version 339667 (0.0010) [2023-12-26 17:50:12,923][105620] Updated weights for policy 1, policy_version 340011 (0.0007) [2023-12-26 17:50:12,936][105692] Updated weights for policy 0, policy_version 339677 (0.0009) [2023-12-26 17:50:12,985][105620] Updated weights for policy 1, policy_version 340021 (0.0007) [2023-12-26 17:50:12,987][105692] Updated weights for policy 0, policy_version 339687 (0.0008) [2023-12-26 17:50:13,046][105620] Updated weights for policy 1, policy_version 340031 (0.0008) [2023-12-26 17:50:13,738][105620] Updated weights for policy 1, policy_version 340041 (0.0007) [2023-12-26 17:50:13,775][105692] Updated weights for policy 0, policy_version 339697 (0.0009) [2023-12-26 17:50:13,789][105620] Updated weights for policy 1, policy_version 340051 (0.0006) [2023-12-26 17:50:13,825][105692] Updated weights for policy 0, policy_version 339707 (0.0007) [2023-12-26 17:50:13,848][105620] Updated weights for policy 1, policy_version 340061 (0.0007) [2023-12-26 17:50:13,878][105692] Updated weights for policy 0, policy_version 339717 (0.0007) [2023-12-26 17:50:13,903][105620] Updated weights for policy 1, policy_version 340071 (0.0008) [2023-12-26 17:50:14,551][105692] Updated weights for policy 0, policy_version 339727 (0.0008) [2023-12-26 17:50:14,551][105620] Updated weights for policy 1, policy_version 340081 (0.0006) [2023-12-26 17:50:14,608][105692] Updated weights for policy 0, policy_version 339737 (0.0009) [2023-12-26 17:50:14,609][105620] Updated weights for policy 1, policy_version 340091 (0.0007) [2023-12-26 17:50:14,659][105692] Updated weights for policy 0, policy_version 339747 (0.0010) [2023-12-26 17:50:14,663][105620] Updated weights for policy 1, policy_version 340101 (0.0009) [2023-12-26 17:50:15,362][105620] Updated weights for policy 1, policy_version 340111 (0.0011) [2023-12-26 17:50:15,368][105692] Updated weights for policy 0, policy_version 339757 (0.0008) [2023-12-26 17:50:15,425][105692] Updated weights for policy 0, policy_version 339767 (0.0006) [2023-12-26 17:50:15,426][105620] Updated weights for policy 1, policy_version 340121 (0.0011) [2023-12-26 17:50:15,478][105692] Updated weights for policy 0, policy_version 339777 (0.0007) [2023-12-26 17:50:15,489][105620] Updated weights for policy 1, policy_version 340131 (0.0011) [2023-12-26 17:50:16,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 174080000. Throughput: 0: 9612.1, 1: 9669.3. Samples: 174050724. Policy #0 lag: (min: 25.0, avg: 48.8, max: 57.0) [2023-12-26 17:50:16,063][104569] Avg episode reward: [(0, '9177.471'), (1, '9356.745')] [2023-12-26 17:50:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000339784_86999040.pth... [2023-12-26 17:50:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000340136_87080960.pth... [2023-12-26 17:50:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000338664_86712320.pth [2023-12-26 17:50:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000339016_86794240.pth [2023-12-26 17:50:16,131][105620] Updated weights for policy 1, policy_version 340141 (0.0011) [2023-12-26 17:50:16,149][105692] Updated weights for policy 0, policy_version 339787 (0.0009) [2023-12-26 17:50:16,182][105620] Updated weights for policy 1, policy_version 340151 (0.0010) [2023-12-26 17:50:16,194][105692] Updated weights for policy 0, policy_version 339797 (0.0008) [2023-12-26 17:50:16,234][105620] Updated weights for policy 1, policy_version 340161 (0.0010) [2023-12-26 17:50:16,245][105692] Updated weights for policy 0, policy_version 339807 (0.0010) [2023-12-26 17:50:16,853][105692] Updated weights for policy 0, policy_version 339817 (0.0007) [2023-12-26 17:50:16,905][105692] Updated weights for policy 0, policy_version 339827 (0.0008) [2023-12-26 17:50:16,957][105692] Updated weights for policy 0, policy_version 339837 (0.0011) [2023-12-26 17:50:16,989][105620] Updated weights for policy 1, policy_version 340171 (0.0011) [2023-12-26 17:50:17,012][105692] Updated weights for policy 0, policy_version 339847 (0.0009) [2023-12-26 17:50:17,050][105620] Updated weights for policy 1, policy_version 340181 (0.0010) [2023-12-26 17:50:17,111][105620] Updated weights for policy 1, policy_version 340191 (0.0010) [2023-12-26 17:50:17,708][105692] Updated weights for policy 0, policy_version 339857 (0.0006) [2023-12-26 17:50:17,765][105692] Updated weights for policy 0, policy_version 339867 (0.0010) [2023-12-26 17:50:17,825][105692] Updated weights for policy 0, policy_version 339877 (0.0007) [2023-12-26 17:50:17,842][105620] Updated weights for policy 1, policy_version 340201 (0.0010) [2023-12-26 17:50:17,910][105620] Updated weights for policy 1, policy_version 340211 (0.0010) [2023-12-26 17:50:17,975][105620] Updated weights for policy 1, policy_version 340221 (0.0010) [2023-12-26 17:50:18,047][105620] Updated weights for policy 1, policy_version 340231 (0.0011) [2023-12-26 17:50:18,372][105692] Updated weights for policy 0, policy_version 339887 (0.0006) [2023-12-26 17:50:18,440][105692] Updated weights for policy 0, policy_version 339897 (0.0005) [2023-12-26 17:50:18,507][105692] Updated weights for policy 0, policy_version 339907 (0.0007) [2023-12-26 17:50:18,755][105620] Updated weights for policy 1, policy_version 340241 (0.0010) [2023-12-26 17:50:18,818][105620] Updated weights for policy 1, policy_version 340251 (0.0011) [2023-12-26 17:50:18,880][105620] Updated weights for policy 1, policy_version 340261 (0.0010) [2023-12-26 17:50:19,041][105692] Updated weights for policy 0, policy_version 339917 (0.0007) [2023-12-26 17:50:19,099][105692] Updated weights for policy 0, policy_version 339927 (0.0007) [2023-12-26 17:50:19,147][105692] Updated weights for policy 0, policy_version 339937 (0.0010) [2023-12-26 17:50:19,649][105620] Updated weights for policy 1, policy_version 340271 (0.0010) [2023-12-26 17:50:19,710][105620] Updated weights for policy 1, policy_version 340281 (0.0010) [2023-12-26 17:50:19,777][105620] Updated weights for policy 1, policy_version 340291 (0.0010) [2023-12-26 17:50:19,880][105692] Updated weights for policy 0, policy_version 339947 (0.0010) [2023-12-26 17:50:19,940][105692] Updated weights for policy 0, policy_version 339957 (0.0011) [2023-12-26 17:50:20,002][105692] Updated weights for policy 0, policy_version 339967 (0.0011) [2023-12-26 17:50:20,533][105620] Updated weights for policy 1, policy_version 340301 (0.0011) [2023-12-26 17:50:20,600][105620] Updated weights for policy 1, policy_version 340311 (0.0011) [2023-12-26 17:50:20,671][105620] Updated weights for policy 1, policy_version 340321 (0.0010) [2023-12-26 17:50:20,780][105692] Updated weights for policy 0, policy_version 339977 (0.0011) [2023-12-26 17:50:20,848][105692] Updated weights for policy 0, policy_version 339987 (0.0010) [2023-12-26 17:50:20,923][105692] Updated weights for policy 0, policy_version 339997 (0.0011) [2023-12-26 17:50:20,997][105692] Updated weights for policy 0, policy_version 340007 (0.0011) [2023-12-26 17:50:21,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 174186496. Throughput: 0: 9729.2, 1: 9673.3. Samples: 174173324. Policy #0 lag: (min: 25.0, avg: 48.8, max: 57.0) [2023-12-26 17:50:21,063][104569] Avg episode reward: [(0, '9177.733'), (1, '9356.742')] [2023-12-26 17:50:21,342][105620] Updated weights for policy 1, policy_version 340331 (0.0007) [2023-12-26 17:50:21,410][105620] Updated weights for policy 1, policy_version 340341 (0.0010) [2023-12-26 17:50:21,470][105620] Updated weights for policy 1, policy_version 340351 (0.0011) [2023-12-26 17:50:21,729][105692] Updated weights for policy 0, policy_version 340017 (0.0010) [2023-12-26 17:50:21,800][105692] Updated weights for policy 0, policy_version 340027 (0.0006) [2023-12-26 17:50:21,865][105692] Updated weights for policy 0, policy_version 340037 (0.0006) [2023-12-26 17:50:22,322][105620] Updated weights for policy 1, policy_version 340361 (0.0010) [2023-12-26 17:50:22,391][105620] Updated weights for policy 1, policy_version 340371 (0.0009) [2023-12-26 17:50:22,452][105620] Updated weights for policy 1, policy_version 340381 (0.0008) [2023-12-26 17:50:22,503][105692] Updated weights for policy 0, policy_version 340047 (0.0007) [2023-12-26 17:50:22,516][105620] Updated weights for policy 1, policy_version 340391 (0.0005) [2023-12-26 17:50:22,555][105692] Updated weights for policy 0, policy_version 340057 (0.0009) [2023-12-26 17:50:22,610][105692] Updated weights for policy 0, policy_version 340067 (0.0007) [2023-12-26 17:50:23,224][105620] Updated weights for policy 1, policy_version 340401 (0.0010) [2023-12-26 17:50:23,273][105620] Updated weights for policy 1, policy_version 340411 (0.0010) [2023-12-26 17:50:23,326][105620] Updated weights for policy 1, policy_version 340421 (0.0009) [2023-12-26 17:50:23,367][105692] Updated weights for policy 0, policy_version 340077 (0.0011) [2023-12-26 17:50:23,426][105692] Updated weights for policy 0, policy_version 340087 (0.0010) [2023-12-26 17:50:23,475][105692] Updated weights for policy 0, policy_version 340097 (0.0009) [2023-12-26 17:50:24,026][105620] Updated weights for policy 1, policy_version 340431 (0.0007) [2023-12-26 17:50:24,078][105620] Updated weights for policy 1, policy_version 340441 (0.0005) [2023-12-26 17:50:24,136][105620] Updated weights for policy 1, policy_version 340451 (0.0006) [2023-12-26 17:50:24,171][105692] Updated weights for policy 0, policy_version 340107 (0.0007) [2023-12-26 17:50:24,226][105692] Updated weights for policy 0, policy_version 340117 (0.0010) [2023-12-26 17:50:24,287][105692] Updated weights for policy 0, policy_version 340127 (0.0010) [2023-12-26 17:50:24,856][105620] Updated weights for policy 1, policy_version 340461 (0.0006) [2023-12-26 17:50:24,899][105692] Updated weights for policy 0, policy_version 340137 (0.0010) [2023-12-26 17:50:24,908][105620] Updated weights for policy 1, policy_version 340471 (0.0008) [2023-12-26 17:50:24,951][105692] Updated weights for policy 0, policy_version 340147 (0.0006) [2023-12-26 17:50:24,963][105620] Updated weights for policy 1, policy_version 340481 (0.0008) [2023-12-26 17:50:25,005][105692] Updated weights for policy 0, policy_version 340157 (0.0009) [2023-12-26 17:50:25,059][105692] Updated weights for policy 0, policy_version 340167 (0.0010) [2023-12-26 17:50:25,573][105620] Updated weights for policy 1, policy_version 340491 (0.0007) [2023-12-26 17:50:25,627][105620] Updated weights for policy 1, policy_version 340501 (0.0009) [2023-12-26 17:50:25,670][105620] Updated weights for policy 1, policy_version 340511 (0.0008) [2023-12-26 17:50:25,692][105692] Updated weights for policy 0, policy_version 340177 (0.0008) [2023-12-26 17:50:25,741][105692] Updated weights for policy 0, policy_version 340187 (0.0007) [2023-12-26 17:50:25,799][105692] Updated weights for policy 0, policy_version 340197 (0.0009) [2023-12-26 17:50:26,062][104569] Fps is (10 sec: 20480.6, 60 sec: 19524.4, 300 sec: 19410.9). Total num frames: 174284800. Throughput: 0: 9705.0, 1: 9699.7. Samples: 174290108. Policy #0 lag: (min: 25.0, avg: 48.8, max: 57.0) [2023-12-26 17:50:26,062][104569] Avg episode reward: [(0, '8915.324'), (1, '9356.834')] [2023-12-26 17:50:26,474][105692] Updated weights for policy 0, policy_version 340207 (0.0006) [2023-12-26 17:50:26,484][105620] Updated weights for policy 1, policy_version 340521 (0.0006) [2023-12-26 17:50:26,535][105692] Updated weights for policy 0, policy_version 340217 (0.0007) [2023-12-26 17:50:26,543][105620] Updated weights for policy 1, policy_version 340531 (0.0006) [2023-12-26 17:50:26,594][105692] Updated weights for policy 0, policy_version 340227 (0.0007) [2023-12-26 17:50:26,600][105620] Updated weights for policy 1, policy_version 340541 (0.0009) [2023-12-26 17:50:26,659][105620] Updated weights for policy 1, policy_version 340551 (0.0007) [2023-12-26 17:50:27,319][105692] Updated weights for policy 0, policy_version 340237 (0.0007) [2023-12-26 17:50:27,371][105692] Updated weights for policy 0, policy_version 340247 (0.0008) [2023-12-26 17:50:27,402][105620] Updated weights for policy 1, policy_version 340561 (0.0008) [2023-12-26 17:50:27,420][105692] Updated weights for policy 0, policy_version 340257 (0.0008) [2023-12-26 17:50:27,459][105620] Updated weights for policy 1, policy_version 340571 (0.0008) [2023-12-26 17:50:27,518][105620] Updated weights for policy 1, policy_version 340581 (0.0008) [2023-12-26 17:50:28,186][105620] Updated weights for policy 1, policy_version 340591 (0.0009) [2023-12-26 17:50:28,193][105692] Updated weights for policy 0, policy_version 340267 (0.0009) [2023-12-26 17:50:28,238][105620] Updated weights for policy 1, policy_version 340601 (0.0006) [2023-12-26 17:50:28,247][105692] Updated weights for policy 0, policy_version 340277 (0.0010) [2023-12-26 17:50:28,296][105620] Updated weights for policy 1, policy_version 340611 (0.0008) [2023-12-26 17:50:28,301][105692] Updated weights for policy 0, policy_version 340287 (0.0010) [2023-12-26 17:50:28,955][105620] Updated weights for policy 1, policy_version 340621 (0.0009) [2023-12-26 17:50:28,982][105586] KL-divergence is very high: 102.0603 [2023-12-26 17:50:29,006][105620] Updated weights for policy 1, policy_version 340631 (0.0008) [2023-12-26 17:50:29,017][105586] KL-divergence is very high: 107.8567 [2023-12-26 17:50:29,038][105692] Updated weights for policy 0, policy_version 340297 (0.0009) [2023-12-26 17:50:29,048][105620] Updated weights for policy 1, policy_version 340641 (0.0007) [2023-12-26 17:50:29,095][105692] Updated weights for policy 0, policy_version 340307 (0.0010) [2023-12-26 17:50:29,153][105692] Updated weights for policy 0, policy_version 340317 (0.0010) [2023-12-26 17:50:29,207][105692] Updated weights for policy 0, policy_version 340327 (0.0007) [2023-12-26 17:50:29,782][105692] Updated weights for policy 0, policy_version 340337 (0.0005) [2023-12-26 17:50:29,846][105692] Updated weights for policy 0, policy_version 340347 (0.0007) [2023-12-26 17:50:29,902][105692] Updated weights for policy 0, policy_version 340357 (0.0008) [2023-12-26 17:50:29,925][105620] Updated weights for policy 1, policy_version 340651 (0.0009) [2023-12-26 17:50:29,976][105620] Updated weights for policy 1, policy_version 340661 (0.0008) [2023-12-26 17:50:30,031][105620] Updated weights for policy 1, policy_version 340671 (0.0008) [2023-12-26 17:50:30,637][105692] Updated weights for policy 0, policy_version 340367 (0.0010) [2023-12-26 17:50:30,707][105692] Updated weights for policy 0, policy_version 340377 (0.0010) [2023-12-26 17:50:30,752][105620] Updated weights for policy 1, policy_version 340681 (0.0008) [2023-12-26 17:50:30,761][105692] Updated weights for policy 0, policy_version 340387 (0.0010) [2023-12-26 17:50:30,803][105620] Updated weights for policy 1, policy_version 340691 (0.0005) [2023-12-26 17:50:30,867][105620] Updated weights for policy 1, policy_version 340701 (0.0008) [2023-12-26 17:50:30,928][105620] Updated weights for policy 1, policy_version 340711 (0.0008) [2023-12-26 17:50:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 174383104. Throughput: 0: 9705.1, 1: 9743.2. Samples: 174348276. Policy #0 lag: (min: 25.0, avg: 48.8, max: 57.0) [2023-12-26 17:50:31,062][104569] Avg episode reward: [(0, '8202.923'), (1, '8816.058')] [2023-12-26 17:50:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000340392_87154688.pth... [2023-12-26 17:50:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000340712_87228416.pth... [2023-12-26 17:50:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000339240_86859776.pth [2023-12-26 17:50:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000339592_86941696.pth [2023-12-26 17:50:31,524][105692] Updated weights for policy 0, policy_version 340397 (0.0010) [2023-12-26 17:50:31,584][105692] Updated weights for policy 0, policy_version 340407 (0.0009) [2023-12-26 17:50:31,646][105692] Updated weights for policy 0, policy_version 340417 (0.0010) [2023-12-26 17:50:31,665][105620] Updated weights for policy 1, policy_version 340721 (0.0007) [2023-12-26 17:50:31,726][105620] Updated weights for policy 1, policy_version 340731 (0.0007) [2023-12-26 17:50:31,795][105620] Updated weights for policy 1, policy_version 340741 (0.0011) [2023-12-26 17:50:32,325][105692] Updated weights for policy 0, policy_version 340427 (0.0011) [2023-12-26 17:50:32,390][105692] Updated weights for policy 0, policy_version 340437 (0.0011) [2023-12-26 17:50:32,459][105692] Updated weights for policy 0, policy_version 340447 (0.0011) [2023-12-26 17:50:32,481][105620] Updated weights for policy 1, policy_version 340751 (0.0007) [2023-12-26 17:50:32,538][105620] Updated weights for policy 1, policy_version 340761 (0.0007) [2023-12-26 17:50:32,598][105620] Updated weights for policy 1, policy_version 340771 (0.0008) [2023-12-26 17:50:33,202][105692] Updated weights for policy 0, policy_version 340457 (0.0011) [2023-12-26 17:50:33,265][105692] Updated weights for policy 0, policy_version 340467 (0.0010) [2023-12-26 17:50:33,301][105620] Updated weights for policy 1, policy_version 340781 (0.0008) [2023-12-26 17:50:33,326][105692] Updated weights for policy 0, policy_version 340477 (0.0009) [2023-12-26 17:50:33,354][105620] Updated weights for policy 1, policy_version 340791 (0.0010) [2023-12-26 17:50:33,386][105692] Updated weights for policy 0, policy_version 340487 (0.0011) [2023-12-26 17:50:33,404][105620] Updated weights for policy 1, policy_version 340801 (0.0010) [2023-12-26 17:50:34,107][105620] Updated weights for policy 1, policy_version 340811 (0.0009) [2023-12-26 17:50:34,119][105692] Updated weights for policy 0, policy_version 340497 (0.0006) [2023-12-26 17:50:34,176][105620] Updated weights for policy 1, policy_version 340821 (0.0008) [2023-12-26 17:50:34,183][105692] Updated weights for policy 0, policy_version 340507 (0.0011) [2023-12-26 17:50:34,233][105620] Updated weights for policy 1, policy_version 340831 (0.0006) [2023-12-26 17:50:34,242][105692] Updated weights for policy 0, policy_version 340517 (0.0011) [2023-12-26 17:50:34,862][105692] Updated weights for policy 0, policy_version 340527 (0.0009) [2023-12-26 17:50:34,910][105692] Updated weights for policy 0, policy_version 340537 (0.0006) [2023-12-26 17:50:34,962][105692] Updated weights for policy 0, policy_version 340547 (0.0008) [2023-12-26 17:50:35,035][105620] Updated weights for policy 1, policy_version 340841 (0.0007) [2023-12-26 17:50:35,087][105620] Updated weights for policy 1, policy_version 340851 (0.0009) [2023-12-26 17:50:35,151][105620] Updated weights for policy 1, policy_version 340861 (0.0009) [2023-12-26 17:50:35,206][105620] Updated weights for policy 1, policy_version 340871 (0.0009) [2023-12-26 17:50:35,710][105692] Updated weights for policy 0, policy_version 340557 (0.0008) [2023-12-26 17:50:35,768][105692] Updated weights for policy 0, policy_version 340567 (0.0008) [2023-12-26 17:50:35,829][105692] Updated weights for policy 0, policy_version 340577 (0.0008) [2023-12-26 17:50:35,983][105620] Updated weights for policy 1, policy_version 340881 (0.0010) [2023-12-26 17:50:36,038][105620] Updated weights for policy 1, policy_version 340891 (0.0010) [2023-12-26 17:50:36,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 174473216. Throughput: 0: 9723.6, 1: 9600.2. Samples: 174463692. Policy #0 lag: (min: 25.0, avg: 48.8, max: 57.0) [2023-12-26 17:50:36,063][104569] Avg episode reward: [(0, '8372.794'), (1, '8995.275')] [2023-12-26 17:50:36,090][105620] Updated weights for policy 1, policy_version 340901 (0.0010) [2023-12-26 17:50:36,578][105692] Updated weights for policy 0, policy_version 340587 (0.0008) [2023-12-26 17:50:36,639][105692] Updated weights for policy 0, policy_version 340597 (0.0008) [2023-12-26 17:50:36,666][105585] KL-divergence is very high: 152.6239 [2023-12-26 17:50:36,691][105692] Updated weights for policy 0, policy_version 340607 (0.0008) [2023-12-26 17:50:36,711][105585] KL-divergence is very high: 168.2466 [2023-12-26 17:50:36,845][105620] Updated weights for policy 1, policy_version 340911 (0.0010) [2023-12-26 17:50:36,894][105620] Updated weights for policy 1, policy_version 340921 (0.0011) [2023-12-26 17:50:36,947][105620] Updated weights for policy 1, policy_version 340931 (0.0010) [2023-12-26 17:50:37,399][105692] Updated weights for policy 0, policy_version 340617 (0.0008) [2023-12-26 17:50:37,458][105692] Updated weights for policy 0, policy_version 340627 (0.0005) [2023-12-26 17:50:37,528][105692] Updated weights for policy 0, policy_version 340637 (0.0005) [2023-12-26 17:50:37,588][105692] Updated weights for policy 0, policy_version 340647 (0.0005) [2023-12-26 17:50:37,657][105620] Updated weights for policy 1, policy_version 340941 (0.0008) [2023-12-26 17:50:37,711][105620] Updated weights for policy 1, policy_version 340951 (0.0005) [2023-12-26 17:50:37,761][105620] Updated weights for policy 1, policy_version 340961 (0.0005) [2023-12-26 17:50:38,099][105692] Updated weights for policy 0, policy_version 340657 (0.0005) [2023-12-26 17:50:38,158][105692] Updated weights for policy 0, policy_version 340667 (0.0005) [2023-12-26 17:50:38,228][105692] Updated weights for policy 0, policy_version 340677 (0.0009) [2023-12-26 17:50:38,410][105620] Updated weights for policy 1, policy_version 340971 (0.0006) [2023-12-26 17:50:38,459][105620] Updated weights for policy 1, policy_version 340981 (0.0007) [2023-12-26 17:50:38,508][105620] Updated weights for policy 1, policy_version 340991 (0.0009) [2023-12-26 17:50:38,828][105692] Updated weights for policy 0, policy_version 340687 (0.0010) [2023-12-26 17:50:38,880][105692] Updated weights for policy 0, policy_version 340698 (0.0010) [2023-12-26 17:50:38,936][105692] Updated weights for policy 0, policy_version 340709 (0.0009) [2023-12-26 17:50:39,170][105620] Updated weights for policy 1, policy_version 341002 (0.0010) [2023-12-26 17:50:39,229][105620] Updated weights for policy 1, policy_version 341012 (0.0008) [2023-12-26 17:50:39,295][105620] Updated weights for policy 1, policy_version 341022 (0.0009) [2023-12-26 17:50:39,370][105620] Updated weights for policy 1, policy_version 341032 (0.0010) [2023-12-26 17:50:39,742][105692] Updated weights for policy 0, policy_version 340719 (0.0008) [2023-12-26 17:50:39,798][105692] Updated weights for policy 0, policy_version 340729 (0.0008) [2023-12-26 17:50:39,867][105692] Updated weights for policy 0, policy_version 340739 (0.0009) [2023-12-26 17:50:40,129][105620] Updated weights for policy 1, policy_version 341042 (0.0011) [2023-12-26 17:50:40,179][105620] Updated weights for policy 1, policy_version 341052 (0.0011) [2023-12-26 17:50:40,232][105620] Updated weights for policy 1, policy_version 341062 (0.0011) [2023-12-26 17:50:40,635][105692] Updated weights for policy 0, policy_version 340749 (0.0009) [2023-12-26 17:50:40,656][105585] KL-divergence is very high: 161.1532 [2023-12-26 17:50:40,700][105692] Updated weights for policy 0, policy_version 340759 (0.0009) [2023-12-26 17:50:40,707][105585] KL-divergence is very high: 163.8802 [2023-12-26 17:50:40,762][105692] Updated weights for policy 0, policy_version 340769 (0.0008) [2023-12-26 17:50:41,004][105620] Updated weights for policy 1, policy_version 341072 (0.0009) [2023-12-26 17:50:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 174571520. Throughput: 0: 9805.4, 1: 9559.0. Samples: 174581864. Policy #0 lag: (min: 25.0, avg: 48.8, max: 57.0) [2023-12-26 17:50:41,062][104569] Avg episode reward: [(0, '8904.237'), (1, '8994.293')] [2023-12-26 17:50:41,071][105620] Updated weights for policy 1, policy_version 341082 (0.0010) [2023-12-26 17:50:41,142][105620] Updated weights for policy 1, policy_version 341092 (0.0009) [2023-12-26 17:50:41,487][105692] Updated weights for policy 0, policy_version 340779 (0.0010) [2023-12-26 17:50:41,550][105692] Updated weights for policy 0, policy_version 340789 (0.0011) [2023-12-26 17:50:41,606][105692] Updated weights for policy 0, policy_version 340799 (0.0011) [2023-12-26 17:50:41,938][105620] Updated weights for policy 1, policy_version 341102 (0.0006) [2023-12-26 17:50:42,001][105620] Updated weights for policy 1, policy_version 341112 (0.0005) [2023-12-26 17:50:42,067][105620] Updated weights for policy 1, policy_version 341122 (0.0006) [2023-12-26 17:50:42,236][105692] Updated weights for policy 0, policy_version 340809 (0.0011) [2023-12-26 17:50:42,299][105692] Updated weights for policy 0, policy_version 340819 (0.0011) [2023-12-26 17:50:42,370][105692] Updated weights for policy 0, policy_version 340829 (0.0011) [2023-12-26 17:50:42,434][105692] Updated weights for policy 0, policy_version 340839 (0.0011) [2023-12-26 17:50:42,698][105620] Updated weights for policy 1, policy_version 341132 (0.0007) [2023-12-26 17:50:42,764][105620] Updated weights for policy 1, policy_version 341142 (0.0007) [2023-12-26 17:50:42,828][105620] Updated weights for policy 1, policy_version 341152 (0.0008) [2023-12-26 17:50:43,084][105692] Updated weights for policy 0, policy_version 340849 (0.0006) [2023-12-26 17:50:43,140][105692] Updated weights for policy 0, policy_version 340859 (0.0005) [2023-12-26 17:50:43,190][105692] Updated weights for policy 0, policy_version 340869 (0.0006) [2023-12-26 17:50:43,359][105620] Updated weights for policy 1, policy_version 341162 (0.0007) [2023-12-26 17:50:43,406][105620] Updated weights for policy 1, policy_version 341172 (0.0005) [2023-12-26 17:50:43,459][105620] Updated weights for policy 1, policy_version 341182 (0.0005) [2023-12-26 17:50:43,517][105620] Updated weights for policy 1, policy_version 341192 (0.0007) [2023-12-26 17:50:43,870][105692] Updated weights for policy 0, policy_version 340879 (0.0010) [2023-12-26 17:50:43,921][105692] Updated weights for policy 0, policy_version 340889 (0.0010) [2023-12-26 17:50:43,965][105692] Updated weights for policy 0, policy_version 340899 (0.0010) [2023-12-26 17:50:44,094][105620] Updated weights for policy 1, policy_version 341202 (0.0006) [2023-12-26 17:50:44,140][105620] Updated weights for policy 1, policy_version 341212 (0.0008) [2023-12-26 17:50:44,205][105620] Updated weights for policy 1, policy_version 341222 (0.0007) [2023-12-26 17:50:44,723][105692] Updated weights for policy 0, policy_version 340909 (0.0010) [2023-12-26 17:50:44,780][105692] Updated weights for policy 0, policy_version 340919 (0.0010) [2023-12-26 17:50:44,807][105620] Updated weights for policy 1, policy_version 341232 (0.0007) [2023-12-26 17:50:44,845][105692] Updated weights for policy 0, policy_version 340929 (0.0010) [2023-12-26 17:50:44,866][105620] Updated weights for policy 1, policy_version 341242 (0.0010) [2023-12-26 17:50:44,919][105620] Updated weights for policy 1, policy_version 341252 (0.0011) [2023-12-26 17:50:45,544][105620] Updated weights for policy 1, policy_version 341262 (0.0008) [2023-12-26 17:50:45,600][105692] Updated weights for policy 0, policy_version 340939 (0.0011) [2023-12-26 17:50:45,603][105620] Updated weights for policy 1, policy_version 341272 (0.0006) [2023-12-26 17:50:45,656][105692] Updated weights for policy 0, policy_version 340949 (0.0010) [2023-12-26 17:50:45,664][105620] Updated weights for policy 1, policy_version 341282 (0.0006) [2023-12-26 17:50:45,705][105692] Updated weights for policy 0, policy_version 340959 (0.0010) [2023-12-26 17:50:46,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 174678016. Throughput: 0: 9823.5, 1: 9607.8. Samples: 174644036. Policy #0 lag: (min: 25.0, avg: 48.8, max: 57.0) [2023-12-26 17:50:46,062][104569] Avg episode reward: [(0, '8812.793'), (1, '8636.134')] [2023-12-26 17:50:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000340968_87302144.pth... [2023-12-26 17:50:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000341288_87375872.pth... [2023-12-26 17:50:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000339784_86999040.pth [2023-12-26 17:50:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000340136_87080960.pth [2023-12-26 17:50:46,182][105620] Updated weights for policy 1, policy_version 341292 (0.0006) [2023-12-26 17:50:46,249][105620] Updated weights for policy 1, policy_version 341302 (0.0005) [2023-12-26 17:50:46,309][105620] Updated weights for policy 1, policy_version 341312 (0.0005) [2023-12-26 17:50:46,376][105692] Updated weights for policy 0, policy_version 340969 (0.0010) [2023-12-26 17:50:46,439][105692] Updated weights for policy 0, policy_version 340979 (0.0008) [2023-12-26 17:50:46,492][105692] Updated weights for policy 0, policy_version 340989 (0.0010) [2023-12-26 17:50:46,544][105692] Updated weights for policy 0, policy_version 340999 (0.0007) [2023-12-26 17:50:46,840][105620] Updated weights for policy 1, policy_version 341322 (0.0006) [2023-12-26 17:50:46,884][105620] Updated weights for policy 1, policy_version 341332 (0.0010) [2023-12-26 17:50:46,932][105620] Updated weights for policy 1, policy_version 341342 (0.0008) [2023-12-26 17:50:46,983][105620] Updated weights for policy 1, policy_version 341352 (0.0009) [2023-12-26 17:50:47,140][105692] Updated weights for policy 0, policy_version 341009 (0.0005) [2023-12-26 17:50:47,189][105692] Updated weights for policy 0, policy_version 341019 (0.0005) [2023-12-26 17:50:47,234][105692] Updated weights for policy 0, policy_version 341029 (0.0005) [2023-12-26 17:50:47,714][105620] Updated weights for policy 1, policy_version 341362 (0.0005) [2023-12-26 17:50:47,765][105620] Updated weights for policy 1, policy_version 341372 (0.0005) [2023-12-26 17:50:47,811][105620] Updated weights for policy 1, policy_version 341382 (0.0005) [2023-12-26 17:50:47,888][105692] Updated weights for policy 0, policy_version 341039 (0.0009) [2023-12-26 17:50:47,946][105692] Updated weights for policy 0, policy_version 341049 (0.0009) [2023-12-26 17:50:48,008][105692] Updated weights for policy 0, policy_version 341059 (0.0009) [2023-12-26 17:50:48,516][105620] Updated weights for policy 1, policy_version 341392 (0.0009) [2023-12-26 17:50:48,584][105620] Updated weights for policy 1, policy_version 341402 (0.0005) [2023-12-26 17:50:48,647][105620] Updated weights for policy 1, policy_version 341412 (0.0008) [2023-12-26 17:50:48,714][105692] Updated weights for policy 0, policy_version 341069 (0.0007) [2023-12-26 17:50:48,774][105692] Updated weights for policy 0, policy_version 341079 (0.0009) [2023-12-26 17:50:48,826][105692] Updated weights for policy 0, policy_version 341089 (0.0009) [2023-12-26 17:50:49,250][105620] Updated weights for policy 1, policy_version 341422 (0.0010) [2023-12-26 17:50:49,302][105620] Updated weights for policy 1, policy_version 341432 (0.0011) [2023-12-26 17:50:49,367][105620] Updated weights for policy 1, policy_version 341442 (0.0010) [2023-12-26 17:50:49,510][105692] Updated weights for policy 0, policy_version 341099 (0.0007) [2023-12-26 17:50:49,573][105692] Updated weights for policy 0, policy_version 341109 (0.0005) [2023-12-26 17:50:49,630][105692] Updated weights for policy 0, policy_version 341119 (0.0006) [2023-12-26 17:50:50,131][105620] Updated weights for policy 1, policy_version 341452 (0.0008) [2023-12-26 17:50:50,183][105620] Updated weights for policy 1, policy_version 341462 (0.0010) [2023-12-26 17:50:50,227][105620] Updated weights for policy 1, policy_version 341472 (0.0010) [2023-12-26 17:50:50,349][105692] Updated weights for policy 0, policy_version 341129 (0.0009) [2023-12-26 17:50:50,394][105692] Updated weights for policy 0, policy_version 341139 (0.0008) [2023-12-26 17:50:50,442][105692] Updated weights for policy 0, policy_version 341149 (0.0008) [2023-12-26 17:50:50,502][105692] Updated weights for policy 0, policy_version 341159 (0.0008) [2023-12-26 17:50:50,960][105620] Updated weights for policy 1, policy_version 341482 (0.0010) [2023-12-26 17:50:51,023][105620] Updated weights for policy 1, policy_version 341492 (0.0011) [2023-12-26 17:50:51,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 174776320. Throughput: 0: 9889.7, 1: 9808.6. Samples: 174769968. Policy #0 lag: (min: 25.0, avg: 48.8, max: 57.0) [2023-12-26 17:50:51,062][104569] Avg episode reward: [(0, '8997.778'), (1, '8562.586')] [2023-12-26 17:50:51,078][105620] Updated weights for policy 1, policy_version 341502 (0.0009) [2023-12-26 17:50:51,133][105620] Updated weights for policy 1, policy_version 341512 (0.0006) [2023-12-26 17:50:51,310][105692] Updated weights for policy 0, policy_version 341169 (0.0008) [2023-12-26 17:50:51,384][105692] Updated weights for policy 0, policy_version 341179 (0.0008) [2023-12-26 17:50:51,440][105692] Updated weights for policy 0, policy_version 341189 (0.0008) [2023-12-26 17:50:51,947][105620] Updated weights for policy 1, policy_version 341522 (0.0009) [2023-12-26 17:50:52,010][105620] Updated weights for policy 1, policy_version 341532 (0.0008) [2023-12-26 17:50:52,072][105620] Updated weights for policy 1, policy_version 341542 (0.0009) [2023-12-26 17:50:52,123][105692] Updated weights for policy 0, policy_version 341199 (0.0008) [2023-12-26 17:50:52,186][105692] Updated weights for policy 0, policy_version 341209 (0.0009) [2023-12-26 17:50:52,245][105692] Updated weights for policy 0, policy_version 341219 (0.0009) [2023-12-26 17:50:52,834][105620] Updated weights for policy 1, policy_version 341552 (0.0007) [2023-12-26 17:50:52,901][105620] Updated weights for policy 1, policy_version 341562 (0.0007) [2023-12-26 17:50:52,971][105620] Updated weights for policy 1, policy_version 341572 (0.0009) [2023-12-26 17:50:52,980][105692] Updated weights for policy 0, policy_version 341229 (0.0007) [2023-12-26 17:50:53,036][105692] Updated weights for policy 0, policy_version 341239 (0.0005) [2023-12-26 17:50:53,092][105692] Updated weights for policy 0, policy_version 341249 (0.0005) [2023-12-26 17:50:53,604][105692] Updated weights for policy 0, policy_version 341259 (0.0006) [2023-12-26 17:50:53,610][105620] Updated weights for policy 1, policy_version 341582 (0.0006) [2023-12-26 17:50:53,663][105692] Updated weights for policy 0, policy_version 341269 (0.0006) [2023-12-26 17:50:53,683][105620] Updated weights for policy 1, policy_version 341592 (0.0005) [2023-12-26 17:50:53,719][105692] Updated weights for policy 0, policy_version 341279 (0.0005) [2023-12-26 17:50:53,754][105620] Updated weights for policy 1, policy_version 341602 (0.0005) [2023-12-26 17:50:54,251][105692] Updated weights for policy 0, policy_version 341289 (0.0005) [2023-12-26 17:50:54,310][105692] Updated weights for policy 0, policy_version 341299 (0.0005) [2023-12-26 17:50:54,322][105620] Updated weights for policy 1, policy_version 341612 (0.0008) [2023-12-26 17:50:54,365][105692] Updated weights for policy 0, policy_version 341309 (0.0005) [2023-12-26 17:50:54,378][105620] Updated weights for policy 1, policy_version 341622 (0.0011) [2023-12-26 17:50:54,415][105692] Updated weights for policy 0, policy_version 341319 (0.0005) [2023-12-26 17:50:54,434][105620] Updated weights for policy 1, policy_version 341632 (0.0011) [2023-12-26 17:50:55,065][105692] Updated weights for policy 0, policy_version 341329 (0.0010) [2023-12-26 17:50:55,123][105692] Updated weights for policy 0, policy_version 341339 (0.0010) [2023-12-26 17:50:55,180][105692] Updated weights for policy 0, policy_version 341349 (0.0009) [2023-12-26 17:50:55,189][105620] Updated weights for policy 1, policy_version 341642 (0.0010) [2023-12-26 17:50:55,233][105620] Updated weights for policy 1, policy_version 341652 (0.0010) [2023-12-26 17:50:55,277][105620] Updated weights for policy 1, policy_version 341662 (0.0010) [2023-12-26 17:50:55,325][105620] Updated weights for policy 1, policy_version 341672 (0.0010) [2023-12-26 17:50:55,889][105692] Updated weights for policy 0, policy_version 341359 (0.0010) [2023-12-26 17:50:55,934][105692] Updated weights for policy 0, policy_version 341369 (0.0010) [2023-12-26 17:50:55,978][105692] Updated weights for policy 0, policy_version 341379 (0.0010) [2023-12-26 17:50:55,996][105620] Updated weights for policy 1, policy_version 341682 (0.0005) [2023-12-26 17:50:56,050][105620] Updated weights for policy 1, policy_version 341692 (0.0005) [2023-12-26 17:50:56,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 174882816. Throughput: 0: 10033.3, 1: 9883.8. Samples: 174891224. Policy #0 lag: (min: 25.0, avg: 48.8, max: 57.0) [2023-12-26 17:50:56,062][104569] Avg episode reward: [(0, '8912.075'), (1, '8739.577')] [2023-12-26 17:50:56,114][105620] Updated weights for policy 1, policy_version 341702 (0.0005) [2023-12-26 17:50:56,607][105692] Updated weights for policy 0, policy_version 341389 (0.0008) [2023-12-26 17:50:56,664][105692] Updated weights for policy 0, policy_version 341399 (0.0006) [2023-12-26 17:50:56,718][105692] Updated weights for policy 0, policy_version 341409 (0.0010) [2023-12-26 17:50:56,828][105620] Updated weights for policy 1, policy_version 341712 (0.0010) [2023-12-26 17:50:56,880][105620] Updated weights for policy 1, policy_version 341722 (0.0010) [2023-12-26 17:50:56,928][105620] Updated weights for policy 1, policy_version 341732 (0.0010) [2023-12-26 17:50:57,371][105692] Updated weights for policy 0, policy_version 341419 (0.0010) [2023-12-26 17:50:57,422][105692] Updated weights for policy 0, policy_version 341429 (0.0010) [2023-12-26 17:50:57,470][105692] Updated weights for policy 0, policy_version 341439 (0.0010) [2023-12-26 17:50:57,623][105620] Updated weights for policy 1, policy_version 341742 (0.0010) [2023-12-26 17:50:57,677][105620] Updated weights for policy 1, policy_version 341752 (0.0007) [2023-12-26 17:50:57,729][105620] Updated weights for policy 1, policy_version 341762 (0.0005) [2023-12-26 17:50:58,124][105692] Updated weights for policy 0, policy_version 341449 (0.0010) [2023-12-26 17:50:58,190][105692] Updated weights for policy 0, policy_version 341459 (0.0007) [2023-12-26 17:50:58,248][105692] Updated weights for policy 0, policy_version 341469 (0.0009) [2023-12-26 17:50:58,306][105692] Updated weights for policy 0, policy_version 341479 (0.0009) [2023-12-26 17:50:58,469][105620] Updated weights for policy 1, policy_version 341772 (0.0008) [2023-12-26 17:50:58,530][105620] Updated weights for policy 1, policy_version 341782 (0.0009) [2023-12-26 17:50:58,590][105620] Updated weights for policy 1, policy_version 341792 (0.0011) [2023-12-26 17:50:59,146][105692] Updated weights for policy 0, policy_version 341489 (0.0008) [2023-12-26 17:50:59,199][105692] Updated weights for policy 0, policy_version 341499 (0.0007) [2023-12-26 17:50:59,263][105692] Updated weights for policy 0, policy_version 341509 (0.0008) [2023-12-26 17:50:59,385][105620] Updated weights for policy 1, policy_version 341802 (0.0011) [2023-12-26 17:50:59,433][105620] Updated weights for policy 1, policy_version 341812 (0.0010) [2023-12-26 17:50:59,478][105620] Updated weights for policy 1, policy_version 341822 (0.0010) [2023-12-26 17:50:59,540][105620] Updated weights for policy 1, policy_version 341832 (0.0010) [2023-12-26 17:51:00,066][105692] Updated weights for policy 0, policy_version 341519 (0.0008) [2023-12-26 17:51:00,122][105692] Updated weights for policy 0, policy_version 341529 (0.0008) [2023-12-26 17:51:00,178][105692] Updated weights for policy 0, policy_version 341539 (0.0008) [2023-12-26 17:51:00,310][105620] Updated weights for policy 1, policy_version 341842 (0.0010) [2023-12-26 17:51:00,372][105620] Updated weights for policy 1, policy_version 341852 (0.0010) [2023-12-26 17:51:00,427][105620] Updated weights for policy 1, policy_version 341862 (0.0010) [2023-12-26 17:51:00,984][105692] Updated weights for policy 0, policy_version 341549 (0.0008) [2023-12-26 17:51:01,033][105692] Updated weights for policy 0, policy_version 341559 (0.0009) [2023-12-26 17:51:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 174972928. Throughput: 0: 10130.2, 1: 9884.5. Samples: 174951380. Policy #0 lag: (min: 25.0, avg: 48.8, max: 57.0) [2023-12-26 17:51:01,062][104569] Avg episode reward: [(0, '9091.948'), (1, '9268.481')] [2023-12-26 17:51:01,097][105620] Updated weights for policy 1, policy_version 341872 (0.0007) [2023-12-26 17:51:01,097][105692] Updated weights for policy 0, policy_version 341569 (0.0008) [2023-12-26 17:51:01,141][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000341576_87457792.pth... [2023-12-26 17:51:01,144][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000340392_87154688.pth [2023-12-26 17:51:01,154][105620] Updated weights for policy 1, policy_version 341882 (0.0007) [2023-12-26 17:51:01,202][105620] Updated weights for policy 1, policy_version 341892 (0.0008) [2023-12-26 17:51:01,220][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000341896_87531520.pth... [2023-12-26 17:51:01,227][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000340712_87228416.pth [2023-12-26 17:51:01,788][105692] Updated weights for policy 0, policy_version 341579 (0.0009) [2023-12-26 17:51:01,848][105692] Updated weights for policy 0, policy_version 341589 (0.0009) [2023-12-26 17:51:01,906][105692] Updated weights for policy 0, policy_version 341599 (0.0008) [2023-12-26 17:51:01,993][105620] Updated weights for policy 1, policy_version 341902 (0.0009) [2023-12-26 17:51:02,059][105620] Updated weights for policy 1, policy_version 341912 (0.0009) [2023-12-26 17:51:02,128][105620] Updated weights for policy 1, policy_version 341922 (0.0010) [2023-12-26 17:51:02,600][105692] Updated weights for policy 0, policy_version 341609 (0.0009) [2023-12-26 17:51:02,649][105692] Updated weights for policy 0, policy_version 341619 (0.0008) [2023-12-26 17:51:02,706][105692] Updated weights for policy 0, policy_version 341629 (0.0007) [2023-12-26 17:51:02,753][105692] Updated weights for policy 0, policy_version 341639 (0.0008) [2023-12-26 17:51:02,880][105620] Updated weights for policy 1, policy_version 341932 (0.0009) [2023-12-26 17:51:02,929][105620] Updated weights for policy 1, policy_version 341942 (0.0010) [2023-12-26 17:51:02,983][105620] Updated weights for policy 1, policy_version 341953 (0.0010) [2023-12-26 17:51:03,450][105692] Updated weights for policy 0, policy_version 341649 (0.0006) [2023-12-26 17:51:03,503][105692] Updated weights for policy 0, policy_version 341659 (0.0005) [2023-12-26 17:51:03,549][105692] Updated weights for policy 0, policy_version 341669 (0.0005) [2023-12-26 17:51:03,774][105620] Updated weights for policy 1, policy_version 341963 (0.0010) [2023-12-26 17:51:03,822][105620] Updated weights for policy 1, policy_version 341973 (0.0009) [2023-12-26 17:51:03,896][105620] Updated weights for policy 1, policy_version 341983 (0.0009) [2023-12-26 17:51:04,211][105692] Updated weights for policy 0, policy_version 341679 (0.0007) [2023-12-26 17:51:04,283][105692] Updated weights for policy 0, policy_version 341689 (0.0006) [2023-12-26 17:51:04,354][105692] Updated weights for policy 0, policy_version 341699 (0.0006) [2023-12-26 17:51:04,622][105620] Updated weights for policy 1, policy_version 341993 (0.0009) [2023-12-26 17:51:04,684][105620] Updated weights for policy 1, policy_version 342003 (0.0009) [2023-12-26 17:51:04,750][105620] Updated weights for policy 1, policy_version 342013 (0.0009) [2023-12-26 17:51:04,812][105620] Updated weights for policy 1, policy_version 342023 (0.0009) [2023-12-26 17:51:05,054][105692] Updated weights for policy 0, policy_version 341709 (0.0008) [2023-12-26 17:51:05,108][105692] Updated weights for policy 0, policy_version 341719 (0.0008) [2023-12-26 17:51:05,168][105692] Updated weights for policy 0, policy_version 341729 (0.0008) [2023-12-26 17:51:05,570][105620] Updated weights for policy 1, policy_version 342033 (0.0009) [2023-12-26 17:51:05,616][105620] Updated weights for policy 1, policy_version 342043 (0.0008) [2023-12-26 17:51:05,666][105620] Updated weights for policy 1, policy_version 342053 (0.0009) [2023-12-26 17:51:05,910][105692] Updated weights for policy 0, policy_version 341739 (0.0009) [2023-12-26 17:51:05,956][105692] Updated weights for policy 0, policy_version 341749 (0.0008) [2023-12-26 17:51:06,018][105692] Updated weights for policy 0, policy_version 341759 (0.0009) [2023-12-26 17:51:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 175079424. Throughput: 0: 9962.8, 1: 9825.4. Samples: 175063796. Policy #0 lag: (min: 25.0, avg: 48.8, max: 57.0) [2023-12-26 17:51:06,063][104569] Avg episode reward: [(0, '8909.301'), (1, '9356.610')] [2023-12-26 17:51:06,519][105620] Updated weights for policy 1, policy_version 342063 (0.0009) [2023-12-26 17:51:06,577][105620] Updated weights for policy 1, policy_version 342073 (0.0010) [2023-12-26 17:51:06,632][105620] Updated weights for policy 1, policy_version 342083 (0.0009) [2023-12-26 17:51:06,632][105692] Updated weights for policy 0, policy_version 341769 (0.0009) [2023-12-26 17:51:06,691][105692] Updated weights for policy 0, policy_version 341779 (0.0008) [2023-12-26 17:51:06,757][105692] Updated weights for policy 0, policy_version 341789 (0.0008) [2023-12-26 17:51:06,823][105692] Updated weights for policy 0, policy_version 341799 (0.0009) [2023-12-26 17:51:07,447][105692] Updated weights for policy 0, policy_version 341809 (0.0007) [2023-12-26 17:51:07,496][105620] Updated weights for policy 1, policy_version 342093 (0.0006) [2023-12-26 17:51:07,505][105692] Updated weights for policy 0, policy_version 341819 (0.0007) [2023-12-26 17:51:07,549][105620] Updated weights for policy 1, policy_version 342103 (0.0006) [2023-12-26 17:51:07,569][105692] Updated weights for policy 0, policy_version 341829 (0.0008) [2023-12-26 17:51:07,601][105620] Updated weights for policy 1, policy_version 342113 (0.0007) [2023-12-26 17:51:08,157][105692] Updated weights for policy 0, policy_version 341839 (0.0009) [2023-12-26 17:51:08,216][105692] Updated weights for policy 0, policy_version 341849 (0.0009) [2023-12-26 17:51:08,276][105692] Updated weights for policy 0, policy_version 341859 (0.0009) [2023-12-26 17:51:08,396][105620] Updated weights for policy 1, policy_version 342123 (0.0008) [2023-12-26 17:51:08,461][105620] Updated weights for policy 1, policy_version 342133 (0.0006) [2023-12-26 17:51:08,530][105620] Updated weights for policy 1, policy_version 342143 (0.0006) [2023-12-26 17:51:09,080][105692] Updated weights for policy 0, policy_version 341869 (0.0009) [2023-12-26 17:51:09,140][105692] Updated weights for policy 0, policy_version 341879 (0.0009) [2023-12-26 17:51:09,202][105692] Updated weights for policy 0, policy_version 341889 (0.0009) [2023-12-26 17:51:09,237][105620] Updated weights for policy 1, policy_version 342153 (0.0010) [2023-12-26 17:51:09,298][105620] Updated weights for policy 1, policy_version 342163 (0.0006) [2023-12-26 17:51:09,360][105620] Updated weights for policy 1, policy_version 342173 (0.0007) [2023-12-26 17:51:09,431][105620] Updated weights for policy 1, policy_version 342183 (0.0008) [2023-12-26 17:51:10,014][105692] Updated weights for policy 0, policy_version 341899 (0.0009) [2023-12-26 17:51:10,076][105692] Updated weights for policy 0, policy_version 341909 (0.0008) [2023-12-26 17:51:10,109][105620] Updated weights for policy 1, policy_version 342193 (0.0008) [2023-12-26 17:51:10,137][105692] Updated weights for policy 0, policy_version 341919 (0.0006) [2023-12-26 17:51:10,175][105620] Updated weights for policy 1, policy_version 342203 (0.0008) [2023-12-26 17:51:10,237][105620] Updated weights for policy 1, policy_version 342213 (0.0007) [2023-12-26 17:51:10,880][105620] Updated weights for policy 1, policy_version 342223 (0.0008) [2023-12-26 17:51:10,937][105620] Updated weights for policy 1, policy_version 342234 (0.0009) [2023-12-26 17:51:10,943][105692] Updated weights for policy 0, policy_version 341929 (0.0007) [2023-12-26 17:51:10,994][105620] Updated weights for policy 1, policy_version 342244 (0.0007) [2023-12-26 17:51:11,000][105692] Updated weights for policy 0, policy_version 341939 (0.0007) [2023-12-26 17:51:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19438.7). Total num frames: 175169536. Throughput: 0: 9950.6, 1: 9776.1. Samples: 175177808. Policy #0 lag: (min: 25.0, avg: 48.8, max: 57.0) [2023-12-26 17:51:11,063][104569] Avg episode reward: [(0, '8638.402'), (1, '9266.296')] [2023-12-26 17:51:11,063][105692] Updated weights for policy 0, policy_version 341949 (0.0007) [2023-12-26 17:51:11,137][105692] Updated weights for policy 0, policy_version 341959 (0.0007) [2023-12-26 17:51:11,762][105620] Updated weights for policy 1, policy_version 342254 (0.0007) [2023-12-26 17:51:11,835][105620] Updated weights for policy 1, policy_version 342264 (0.0006) [2023-12-26 17:51:11,897][105620] Updated weights for policy 1, policy_version 342274 (0.0009) [2023-12-26 17:51:11,931][105692] Updated weights for policy 0, policy_version 341969 (0.0006) [2023-12-26 17:51:12,001][105692] Updated weights for policy 0, policy_version 341979 (0.0005) [2023-12-26 17:51:12,061][105692] Updated weights for policy 0, policy_version 341989 (0.0009) [2023-12-26 17:51:12,550][105620] Updated weights for policy 1, policy_version 342284 (0.0007) [2023-12-26 17:51:12,607][105620] Updated weights for policy 1, policy_version 342294 (0.0006) [2023-12-26 17:51:12,663][105620] Updated weights for policy 1, policy_version 342304 (0.0010) [2023-12-26 17:51:12,751][105692] Updated weights for policy 0, policy_version 341999 (0.0008) [2023-12-26 17:51:12,803][105692] Updated weights for policy 0, policy_version 342009 (0.0008) [2023-12-26 17:51:12,861][105692] Updated weights for policy 0, policy_version 342019 (0.0006) [2023-12-26 17:51:13,278][105620] Updated weights for policy 1, policy_version 342314 (0.0009) [2023-12-26 17:51:13,332][105620] Updated weights for policy 1, policy_version 342324 (0.0006) [2023-12-26 17:51:13,379][105620] Updated weights for policy 1, policy_version 342334 (0.0005) [2023-12-26 17:51:13,426][105620] Updated weights for policy 1, policy_version 342344 (0.0005) [2023-12-26 17:51:13,456][105692] Updated weights for policy 0, policy_version 342029 (0.0008) [2023-12-26 17:51:13,515][105692] Updated weights for policy 0, policy_version 342039 (0.0008) [2023-12-26 17:51:13,578][105692] Updated weights for policy 0, policy_version 342049 (0.0011) [2023-12-26 17:51:14,019][105620] Updated weights for policy 1, policy_version 342354 (0.0005) [2023-12-26 17:51:14,075][105620] Updated weights for policy 1, policy_version 342364 (0.0005) [2023-12-26 17:51:14,124][105620] Updated weights for policy 1, policy_version 342374 (0.0009) [2023-12-26 17:51:14,293][105692] Updated weights for policy 0, policy_version 342059 (0.0010) [2023-12-26 17:51:14,349][105692] Updated weights for policy 0, policy_version 342069 (0.0008) [2023-12-26 17:51:14,407][105692] Updated weights for policy 0, policy_version 342079 (0.0008) [2023-12-26 17:51:14,824][105620] Updated weights for policy 1, policy_version 342384 (0.0010) [2023-12-26 17:51:14,894][105620] Updated weights for policy 1, policy_version 342394 (0.0011) [2023-12-26 17:51:14,957][105620] Updated weights for policy 1, policy_version 342404 (0.0011) [2023-12-26 17:51:15,110][105692] Updated weights for policy 0, policy_version 342089 (0.0009) [2023-12-26 17:51:15,173][105692] Updated weights for policy 0, policy_version 342099 (0.0006) [2023-12-26 17:51:15,240][105692] Updated weights for policy 0, policy_version 342109 (0.0005) [2023-12-26 17:51:15,307][105692] Updated weights for policy 0, policy_version 342119 (0.0006) [2023-12-26 17:51:15,706][105620] Updated weights for policy 1, policy_version 342414 (0.0011) [2023-12-26 17:51:15,757][105620] Updated weights for policy 1, policy_version 342424 (0.0010) [2023-12-26 17:51:15,809][105620] Updated weights for policy 1, policy_version 342434 (0.0010) [2023-12-26 17:51:15,874][105692] Updated weights for policy 0, policy_version 342129 (0.0005) [2023-12-26 17:51:15,939][105692] Updated weights for policy 0, policy_version 342139 (0.0005) [2023-12-26 17:51:15,998][105692] Updated weights for policy 0, policy_version 342149 (0.0008) [2023-12-26 17:51:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19933.9, 300 sec: 19466.4). Total num frames: 175276032. Throughput: 0: 9958.9, 1: 9823.9. Samples: 175238508. Policy #0 lag: (min: 25.0, avg: 48.8, max: 57.0) [2023-12-26 17:51:16,063][104569] Avg episode reward: [(0, '8907.418'), (1, '8995.421')] [2023-12-26 17:51:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000342152_87605248.pth... [2023-12-26 17:51:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000342440_87670784.pth... [2023-12-26 17:51:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000340968_87302144.pth [2023-12-26 17:51:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000341288_87375872.pth [2023-12-26 17:51:16,420][105620] Updated weights for policy 1, policy_version 342444 (0.0008) [2023-12-26 17:51:16,468][105620] Updated weights for policy 1, policy_version 342454 (0.0006) [2023-12-26 17:51:16,536][105620] Updated weights for policy 1, policy_version 342464 (0.0006) [2023-12-26 17:51:16,770][105692] Updated weights for policy 0, policy_version 342159 (0.0007) [2023-12-26 17:51:16,832][105692] Updated weights for policy 0, policy_version 342169 (0.0008) [2023-12-26 17:51:16,889][105692] Updated weights for policy 0, policy_version 342179 (0.0010) [2023-12-26 17:51:17,087][105620] Updated weights for policy 1, policy_version 342474 (0.0006) [2023-12-26 17:51:17,153][105620] Updated weights for policy 1, policy_version 342484 (0.0007) [2023-12-26 17:51:17,218][105620] Updated weights for policy 1, policy_version 342494 (0.0005) [2023-12-26 17:51:17,283][105620] Updated weights for policy 1, policy_version 342504 (0.0005) [2023-12-26 17:51:17,606][105692] Updated weights for policy 0, policy_version 342189 (0.0009) [2023-12-26 17:51:17,662][105692] Updated weights for policy 0, policy_version 342199 (0.0010) [2023-12-26 17:51:17,714][105692] Updated weights for policy 0, policy_version 342209 (0.0008) [2023-12-26 17:51:17,791][105620] Updated weights for policy 1, policy_version 342514 (0.0005) [2023-12-26 17:51:17,839][105620] Updated weights for policy 1, policy_version 342524 (0.0005) [2023-12-26 17:51:17,890][105620] Updated weights for policy 1, policy_version 342534 (0.0005) [2023-12-26 17:51:18,387][105692] Updated weights for policy 0, policy_version 342219 (0.0007) [2023-12-26 17:51:18,450][105692] Updated weights for policy 0, policy_version 342229 (0.0011) [2023-12-26 17:51:18,502][105620] Updated weights for policy 1, policy_version 342544 (0.0009) [2023-12-26 17:51:18,512][105692] Updated weights for policy 0, policy_version 342239 (0.0010) [2023-12-26 17:51:18,562][105620] Updated weights for policy 1, policy_version 342554 (0.0010) [2023-12-26 17:51:18,609][105620] Updated weights for policy 1, policy_version 342564 (0.0005) [2023-12-26 17:51:19,193][105692] Updated weights for policy 0, policy_version 342249 (0.0010) [2023-12-26 17:51:19,254][105692] Updated weights for policy 0, policy_version 342259 (0.0009) [2023-12-26 17:51:19,313][105620] Updated weights for policy 1, policy_version 342574 (0.0007) [2023-12-26 17:51:19,314][105692] Updated weights for policy 0, policy_version 342269 (0.0011) [2023-12-26 17:51:19,380][105620] Updated weights for policy 1, policy_version 342584 (0.0007) [2023-12-26 17:51:19,383][105692] Updated weights for policy 0, policy_version 342279 (0.0007) [2023-12-26 17:51:19,445][105620] Updated weights for policy 1, policy_version 342594 (0.0007) [2023-12-26 17:51:20,063][105692] Updated weights for policy 0, policy_version 342289 (0.0008) [2023-12-26 17:51:20,116][105692] Updated weights for policy 0, policy_version 342299 (0.0009) [2023-12-26 17:51:20,177][105692] Updated weights for policy 0, policy_version 342309 (0.0009) [2023-12-26 17:51:20,186][105620] Updated weights for policy 1, policy_version 342604 (0.0010) [2023-12-26 17:51:20,248][105620] Updated weights for policy 1, policy_version 342614 (0.0009) [2023-12-26 17:51:20,305][105620] Updated weights for policy 1, policy_version 342624 (0.0009) [2023-12-26 17:51:21,000][105692] Updated weights for policy 0, policy_version 342319 (0.0009) [2023-12-26 17:51:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 175366144. Throughput: 0: 9977.0, 1: 9999.7. Samples: 175362644. Policy #0 lag: (min: 25.0, avg: 48.8, max: 57.0) [2023-12-26 17:51:21,062][104569] Avg episode reward: [(0, '8997.471'), (1, '9176.903')] [2023-12-26 17:51:21,063][105692] Updated weights for policy 0, policy_version 342329 (0.0009) [2023-12-26 17:51:21,128][105692] Updated weights for policy 0, policy_version 342339 (0.0008) [2023-12-26 17:51:21,137][105620] Updated weights for policy 1, policy_version 342634 (0.0009) [2023-12-26 17:51:21,200][105620] Updated weights for policy 1, policy_version 342644 (0.0010) [2023-12-26 17:51:21,264][105620] Updated weights for policy 1, policy_version 342654 (0.0010) [2023-12-26 17:51:21,329][105620] Updated weights for policy 1, policy_version 342664 (0.0011) [2023-12-26 17:51:21,868][105692] Updated weights for policy 0, policy_version 342349 (0.0007) [2023-12-26 17:51:21,924][105692] Updated weights for policy 0, policy_version 342359 (0.0009) [2023-12-26 17:51:21,983][105692] Updated weights for policy 0, policy_version 342369 (0.0009) [2023-12-26 17:51:22,087][105620] Updated weights for policy 1, policy_version 342674 (0.0011) [2023-12-26 17:51:22,148][105620] Updated weights for policy 1, policy_version 342684 (0.0011) [2023-12-26 17:51:22,201][105620] Updated weights for policy 1, policy_version 342694 (0.0010) [2023-12-26 17:51:22,807][105692] Updated weights for policy 0, policy_version 342379 (0.0008) [2023-12-26 17:51:22,868][105692] Updated weights for policy 0, policy_version 342389 (0.0007) [2023-12-26 17:51:22,931][105692] Updated weights for policy 0, policy_version 342399 (0.0009) [2023-12-26 17:51:22,966][105620] Updated weights for policy 1, policy_version 342704 (0.0009) [2023-12-26 17:51:23,031][105620] Updated weights for policy 1, policy_version 342714 (0.0007) [2023-12-26 17:51:23,093][105620] Updated weights for policy 1, policy_version 342724 (0.0009) [2023-12-26 17:51:23,644][105620] Updated weights for policy 1, policy_version 342734 (0.0007) [2023-12-26 17:51:23,675][105692] Updated weights for policy 0, policy_version 342409 (0.0006) [2023-12-26 17:51:23,700][105620] Updated weights for policy 1, policy_version 342744 (0.0005) [2023-12-26 17:51:23,729][105692] Updated weights for policy 0, policy_version 342419 (0.0005) [2023-12-26 17:51:23,764][105620] Updated weights for policy 1, policy_version 342754 (0.0005) [2023-12-26 17:51:23,793][105692] Updated weights for policy 0, policy_version 342429 (0.0005) [2023-12-26 17:51:23,842][105692] Updated weights for policy 0, policy_version 342439 (0.0005) [2023-12-26 17:51:24,443][105620] Updated weights for policy 1, policy_version 342765 (0.0007) [2023-12-26 17:51:24,495][105620] Updated weights for policy 1, policy_version 342775 (0.0008) [2023-12-26 17:51:24,510][105692] Updated weights for policy 0, policy_version 342449 (0.0006) [2023-12-26 17:51:24,551][105620] Updated weights for policy 1, policy_version 342785 (0.0007) [2023-12-26 17:51:24,569][105692] Updated weights for policy 0, policy_version 342459 (0.0005) [2023-12-26 17:51:24,628][105692] Updated weights for policy 0, policy_version 342469 (0.0006) [2023-12-26 17:51:25,224][105692] Updated weights for policy 0, policy_version 342479 (0.0005) [2023-12-26 17:51:25,237][105620] Updated weights for policy 1, policy_version 342795 (0.0009) [2023-12-26 17:51:25,287][105692] Updated weights for policy 0, policy_version 342489 (0.0008) [2023-12-26 17:51:25,288][105620] Updated weights for policy 1, policy_version 342805 (0.0007) [2023-12-26 17:51:25,340][105620] Updated weights for policy 1, policy_version 342815 (0.0009) [2023-12-26 17:51:25,349][105692] Updated weights for policy 0, policy_version 342499 (0.0005) [2023-12-26 17:51:25,942][105692] Updated weights for policy 0, policy_version 342509 (0.0008) [2023-12-26 17:51:25,985][105620] Updated weights for policy 1, policy_version 342825 (0.0008) [2023-12-26 17:51:26,004][105692] Updated weights for policy 0, policy_version 342519 (0.0010) [2023-12-26 17:51:26,036][105620] Updated weights for policy 1, policy_version 342835 (0.0010) [2023-12-26 17:51:26,062][104569] Fps is (10 sec: 18842.3, 60 sec: 19660.8, 300 sec: 19438.7). Total num frames: 175464448. Throughput: 0: 9932.6, 1: 10024.1. Samples: 175479916. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-26 17:51:26,062][104569] Avg episode reward: [(0, '9087.840'), (1, '9087.937')] [2023-12-26 17:51:26,063][105692] Updated weights for policy 0, policy_version 342529 (0.0010) [2023-12-26 17:51:26,090][105620] Updated weights for policy 1, policy_version 342845 (0.0010) [2023-12-26 17:51:26,145][105620] Updated weights for policy 1, policy_version 342855 (0.0010) [2023-12-26 17:51:26,793][105692] Updated weights for policy 0, policy_version 342539 (0.0010) [2023-12-26 17:51:26,853][105692] Updated weights for policy 0, policy_version 342549 (0.0010) [2023-12-26 17:51:26,899][105620] Updated weights for policy 1, policy_version 342865 (0.0007) [2023-12-26 17:51:26,909][105692] Updated weights for policy 0, policy_version 342559 (0.0011) [2023-12-26 17:51:26,963][105620] Updated weights for policy 1, policy_version 342875 (0.0006) [2023-12-26 17:51:27,022][105620] Updated weights for policy 1, policy_version 342885 (0.0008) [2023-12-26 17:51:27,660][105692] Updated weights for policy 0, policy_version 342569 (0.0011) [2023-12-26 17:51:27,714][105692] Updated weights for policy 0, policy_version 342579 (0.0010) [2023-12-26 17:51:27,765][105692] Updated weights for policy 0, policy_version 342589 (0.0010) [2023-12-26 17:51:27,772][105620] Updated weights for policy 1, policy_version 342895 (0.0006) [2023-12-26 17:51:27,816][105692] Updated weights for policy 0, policy_version 342599 (0.0010) [2023-12-26 17:51:27,819][105620] Updated weights for policy 1, policy_version 342905 (0.0007) [2023-12-26 17:51:27,866][105620] Updated weights for policy 1, policy_version 342915 (0.0008) [2023-12-26 17:51:28,598][105692] Updated weights for policy 0, policy_version 342609 (0.0010) [2023-12-26 17:51:28,632][105620] Updated weights for policy 1, policy_version 342925 (0.0006) [2023-12-26 17:51:28,653][105692] Updated weights for policy 0, policy_version 342619 (0.0010) [2023-12-26 17:51:28,678][105620] Updated weights for policy 1, policy_version 342935 (0.0009) [2023-12-26 17:51:28,711][105692] Updated weights for policy 0, policy_version 342629 (0.0010) [2023-12-26 17:51:28,726][105620] Updated weights for policy 1, policy_version 342945 (0.0006) [2023-12-26 17:51:29,474][105692] Updated weights for policy 0, policy_version 342639 (0.0010) [2023-12-26 17:51:29,509][105620] Updated weights for policy 1, policy_version 342955 (0.0009) [2023-12-26 17:51:29,537][105692] Updated weights for policy 0, policy_version 342649 (0.0009) [2023-12-26 17:51:29,565][105620] Updated weights for policy 1, policy_version 342965 (0.0009) [2023-12-26 17:51:29,590][105692] Updated weights for policy 0, policy_version 342659 (0.0009) [2023-12-26 17:51:29,618][105620] Updated weights for policy 1, policy_version 342975 (0.0009) [2023-12-26 17:51:30,218][105620] Updated weights for policy 1, policy_version 342985 (0.0007) [2023-12-26 17:51:30,272][105620] Updated weights for policy 1, policy_version 342995 (0.0005) [2023-12-26 17:51:30,327][105620] Updated weights for policy 1, policy_version 343005 (0.0006) [2023-12-26 17:51:30,376][105620] Updated weights for policy 1, policy_version 343015 (0.0009) [2023-12-26 17:51:30,425][105692] Updated weights for policy 0, policy_version 342669 (0.0009) [2023-12-26 17:51:30,487][105692] Updated weights for policy 0, policy_version 342679 (0.0009) [2023-12-26 17:51:30,541][105692] Updated weights for policy 0, policy_version 342689 (0.0009) [2023-12-26 17:51:30,988][105620] Updated weights for policy 1, policy_version 343025 (0.0006) [2023-12-26 17:51:31,050][105620] Updated weights for policy 1, policy_version 343035 (0.0006) [2023-12-26 17:51:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 175562752. Throughput: 0: 9894.4, 1: 9935.0. Samples: 175536360. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-26 17:51:31,063][104569] Avg episode reward: [(0, '9177.934'), (1, '9267.922')] [2023-12-26 17:51:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000342696_87744512.pth... [2023-12-26 17:51:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000341576_87457792.pth [2023-12-26 17:51:31,126][105620] Updated weights for policy 1, policy_version 343045 (0.0006) [2023-12-26 17:51:31,147][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000343048_87826432.pth... [2023-12-26 17:51:31,152][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000341896_87531520.pth [2023-12-26 17:51:31,352][105692] Updated weights for policy 0, policy_version 342699 (0.0008) [2023-12-26 17:51:31,418][105692] Updated weights for policy 0, policy_version 342709 (0.0008) [2023-12-26 17:51:31,471][105692] Updated weights for policy 0, policy_version 342719 (0.0008) [2023-12-26 17:51:31,816][105620] Updated weights for policy 1, policy_version 343055 (0.0009) [2023-12-26 17:51:31,879][105620] Updated weights for policy 1, policy_version 343065 (0.0008) [2023-12-26 17:51:31,943][105620] Updated weights for policy 1, policy_version 343075 (0.0006) [2023-12-26 17:51:32,129][105692] Updated weights for policy 0, policy_version 342729 (0.0006) [2023-12-26 17:51:32,176][105692] Updated weights for policy 0, policy_version 342739 (0.0008) [2023-12-26 17:51:32,229][105692] Updated weights for policy 0, policy_version 342749 (0.0008) [2023-12-26 17:51:32,297][105692] Updated weights for policy 0, policy_version 342759 (0.0006) [2023-12-26 17:51:32,577][105620] Updated weights for policy 1, policy_version 343085 (0.0005) [2023-12-26 17:51:32,641][105620] Updated weights for policy 1, policy_version 343095 (0.0008) [2023-12-26 17:51:32,693][105620] Updated weights for policy 1, policy_version 343105 (0.0010) [2023-12-26 17:51:32,907][105692] Updated weights for policy 0, policy_version 342769 (0.0005) [2023-12-26 17:51:32,955][105692] Updated weights for policy 0, policy_version 342779 (0.0005) [2023-12-26 17:51:33,009][105692] Updated weights for policy 0, policy_version 342789 (0.0007) [2023-12-26 17:51:33,426][105620] Updated weights for policy 1, policy_version 343115 (0.0008) [2023-12-26 17:51:33,475][105620] Updated weights for policy 1, policy_version 343125 (0.0008) [2023-12-26 17:51:33,536][105620] Updated weights for policy 1, policy_version 343135 (0.0007) [2023-12-26 17:51:33,706][105692] Updated weights for policy 0, policy_version 342799 (0.0009) [2023-12-26 17:51:33,757][105692] Updated weights for policy 0, policy_version 342809 (0.0008) [2023-12-26 17:51:33,814][105692] Updated weights for policy 0, policy_version 342819 (0.0005) [2023-12-26 17:51:34,284][105620] Updated weights for policy 1, policy_version 343145 (0.0009) [2023-12-26 17:51:34,337][105620] Updated weights for policy 1, policy_version 343155 (0.0009) [2023-12-26 17:51:34,391][105620] Updated weights for policy 1, policy_version 343165 (0.0009) [2023-12-26 17:51:34,450][105620] Updated weights for policy 1, policy_version 343175 (0.0009) [2023-12-26 17:51:34,535][105692] Updated weights for policy 0, policy_version 342829 (0.0008) [2023-12-26 17:51:34,596][105692] Updated weights for policy 0, policy_version 342839 (0.0009) [2023-12-26 17:51:34,657][105692] Updated weights for policy 0, policy_version 342849 (0.0009) [2023-12-26 17:51:35,284][105692] Updated weights for policy 0, policy_version 342859 (0.0006) [2023-12-26 17:51:35,296][105620] Updated weights for policy 1, policy_version 343185 (0.0009) [2023-12-26 17:51:35,345][105692] Updated weights for policy 0, policy_version 342869 (0.0008) [2023-12-26 17:51:35,351][105620] Updated weights for policy 1, policy_version 343195 (0.0007) [2023-12-26 17:51:35,405][105692] Updated weights for policy 0, policy_version 342879 (0.0007) [2023-12-26 17:51:35,407][105620] Updated weights for policy 1, policy_version 343205 (0.0006) [2023-12-26 17:51:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 175661056. Throughput: 0: 9824.3, 1: 9825.6. Samples: 175654212. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-26 17:51:36,063][104569] Avg episode reward: [(0, '9087.218'), (1, '9177.508')] [2023-12-26 17:51:36,066][105620] Updated weights for policy 1, policy_version 343215 (0.0009) [2023-12-26 17:51:36,129][105620] Updated weights for policy 1, policy_version 343225 (0.0011) [2023-12-26 17:51:36,179][105692] Updated weights for policy 0, policy_version 342889 (0.0008) [2023-12-26 17:51:36,195][105620] Updated weights for policy 1, policy_version 343235 (0.0011) [2023-12-26 17:51:36,234][105692] Updated weights for policy 0, policy_version 342899 (0.0007) [2023-12-26 17:51:36,303][105692] Updated weights for policy 0, policy_version 342909 (0.0008) [2023-12-26 17:51:36,372][105692] Updated weights for policy 0, policy_version 342919 (0.0006) [2023-12-26 17:51:36,958][105620] Updated weights for policy 1, policy_version 343245 (0.0011) [2023-12-26 17:51:37,021][105620] Updated weights for policy 1, policy_version 343255 (0.0011) [2023-12-26 17:51:37,051][105692] Updated weights for policy 0, policy_version 342929 (0.0006) [2023-12-26 17:51:37,081][105620] Updated weights for policy 1, policy_version 343265 (0.0011) [2023-12-26 17:51:37,115][105692] Updated weights for policy 0, policy_version 342939 (0.0006) [2023-12-26 17:51:37,181][105692] Updated weights for policy 0, policy_version 342949 (0.0007) [2023-12-26 17:51:37,732][105620] Updated weights for policy 1, policy_version 343275 (0.0010) [2023-12-26 17:51:37,787][105620] Updated weights for policy 1, policy_version 343285 (0.0010) [2023-12-26 17:51:37,836][105620] Updated weights for policy 1, policy_version 343295 (0.0010) [2023-12-26 17:51:37,940][105692] Updated weights for policy 0, policy_version 342959 (0.0008) [2023-12-26 17:51:37,984][105692] Updated weights for policy 0, policy_version 342969 (0.0008) [2023-12-26 17:51:38,036][105692] Updated weights for policy 0, policy_version 342979 (0.0008) [2023-12-26 17:51:38,588][105620] Updated weights for policy 1, policy_version 343305 (0.0010) [2023-12-26 17:51:38,651][105620] Updated weights for policy 1, policy_version 343315 (0.0010) [2023-12-26 17:51:38,705][105620] Updated weights for policy 1, policy_version 343325 (0.0010) [2023-12-26 17:51:38,748][105692] Updated weights for policy 0, policy_version 342989 (0.0007) [2023-12-26 17:51:38,761][105620] Updated weights for policy 1, policy_version 343335 (0.0010) [2023-12-26 17:51:38,805][105692] Updated weights for policy 0, policy_version 342999 (0.0005) [2023-12-26 17:51:38,861][105692] Updated weights for policy 0, policy_version 343009 (0.0005) [2023-12-26 17:51:39,514][105620] Updated weights for policy 1, policy_version 343345 (0.0011) [2023-12-26 17:51:39,575][105692] Updated weights for policy 0, policy_version 343019 (0.0005) [2023-12-26 17:51:39,577][105620] Updated weights for policy 1, policy_version 343355 (0.0011) [2023-12-26 17:51:39,633][105620] Updated weights for policy 1, policy_version 343365 (0.0010) [2023-12-26 17:51:39,635][105692] Updated weights for policy 0, policy_version 343029 (0.0005) [2023-12-26 17:51:39,698][105692] Updated weights for policy 0, policy_version 343039 (0.0007) [2023-12-26 17:51:40,384][105620] Updated weights for policy 1, policy_version 343375 (0.0007) [2023-12-26 17:51:40,440][105620] Updated weights for policy 1, policy_version 343385 (0.0005) [2023-12-26 17:51:40,493][105620] Updated weights for policy 1, policy_version 343395 (0.0005) [2023-12-26 17:51:40,505][105692] Updated weights for policy 0, policy_version 343049 (0.0009) [2023-12-26 17:51:40,559][105692] Updated weights for policy 0, policy_version 343060 (0.0010) [2023-12-26 17:51:40,613][105692] Updated weights for policy 0, policy_version 343070 (0.0010) [2023-12-26 17:51:40,670][105692] Updated weights for policy 0, policy_version 343080 (0.0009) [2023-12-26 17:51:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 175759360. Throughput: 0: 9719.9, 1: 9800.0. Samples: 175769620. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-26 17:51:41,063][104569] Avg episode reward: [(0, '9268.218'), (1, '9176.496')] [2023-12-26 17:51:41,106][105620] Updated weights for policy 1, policy_version 343405 (0.0007) [2023-12-26 17:51:41,173][105620] Updated weights for policy 1, policy_version 343415 (0.0008) [2023-12-26 17:51:41,232][105620] Updated weights for policy 1, policy_version 343425 (0.0009) [2023-12-26 17:51:41,454][105692] Updated weights for policy 0, policy_version 343090 (0.0007) [2023-12-26 17:51:41,514][105692] Updated weights for policy 0, policy_version 343100 (0.0006) [2023-12-26 17:51:41,576][105692] Updated weights for policy 0, policy_version 343110 (0.0006) [2023-12-26 17:51:42,079][105620] Updated weights for policy 1, policy_version 343435 (0.0010) [2023-12-26 17:51:42,126][105620] Updated weights for policy 1, policy_version 343445 (0.0009) [2023-12-26 17:51:42,180][105620] Updated weights for policy 1, policy_version 343455 (0.0009) [2023-12-26 17:51:42,233][105692] Updated weights for policy 0, policy_version 343120 (0.0007) [2023-12-26 17:51:42,300][105692] Updated weights for policy 0, policy_version 343130 (0.0009) [2023-12-26 17:51:42,373][105692] Updated weights for policy 0, policy_version 343140 (0.0009) [2023-12-26 17:51:42,985][105620] Updated weights for policy 1, policy_version 343465 (0.0008) [2023-12-26 17:51:43,042][105620] Updated weights for policy 1, policy_version 343475 (0.0009) [2023-12-26 17:51:43,060][105692] Updated weights for policy 0, policy_version 343150 (0.0008) [2023-12-26 17:51:43,087][105620] Updated weights for policy 1, policy_version 343485 (0.0006) [2023-12-26 17:51:43,102][105692] Updated weights for policy 0, policy_version 343160 (0.0006) [2023-12-26 17:51:43,132][105620] Updated weights for policy 1, policy_version 343495 (0.0007) [2023-12-26 17:51:43,148][105692] Updated weights for policy 0, policy_version 343170 (0.0008) [2023-12-26 17:51:43,879][105692] Updated weights for policy 0, policy_version 343180 (0.0008) [2023-12-26 17:51:43,930][105620] Updated weights for policy 1, policy_version 343505 (0.0007) [2023-12-26 17:51:43,940][105692] Updated weights for policy 0, policy_version 343190 (0.0009) [2023-12-26 17:51:43,989][105620] Updated weights for policy 1, policy_version 343515 (0.0008) [2023-12-26 17:51:43,999][105692] Updated weights for policy 0, policy_version 343200 (0.0005) [2023-12-26 17:51:44,050][105620] Updated weights for policy 1, policy_version 343525 (0.0010) [2023-12-26 17:51:44,713][105692] Updated weights for policy 0, policy_version 343210 (0.0010) [2023-12-26 17:51:44,776][105692] Updated weights for policy 0, policy_version 343220 (0.0009) [2023-12-26 17:51:44,831][105620] Updated weights for policy 1, policy_version 343535 (0.0009) [2023-12-26 17:51:44,838][105692] Updated weights for policy 0, policy_version 343230 (0.0008) [2023-12-26 17:51:44,890][105620] Updated weights for policy 1, policy_version 343545 (0.0009) [2023-12-26 17:51:44,906][105692] Updated weights for policy 0, policy_version 343240 (0.0007) [2023-12-26 17:51:44,940][105620] Updated weights for policy 1, policy_version 343555 (0.0008) [2023-12-26 17:51:45,648][105692] Updated weights for policy 0, policy_version 343250 (0.0009) [2023-12-26 17:51:45,684][105620] Updated weights for policy 1, policy_version 343565 (0.0007) [2023-12-26 17:51:45,705][105692] Updated weights for policy 0, policy_version 343260 (0.0007) [2023-12-26 17:51:45,746][105620] Updated weights for policy 1, policy_version 343575 (0.0005) [2023-12-26 17:51:45,768][105692] Updated weights for policy 0, policy_version 343270 (0.0008) [2023-12-26 17:51:45,807][105620] Updated weights for policy 1, policy_version 343585 (0.0006) [2023-12-26 17:51:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 175857664. Throughput: 0: 9667.4, 1: 9744.9. Samples: 175824932. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-26 17:51:46,062][104569] Avg episode reward: [(0, '9268.040'), (1, '9086.993')] [2023-12-26 17:51:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000343272_87891968.pth... [2023-12-26 17:51:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000343592_87965696.pth... [2023-12-26 17:51:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000342152_87605248.pth [2023-12-26 17:51:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000342440_87670784.pth [2023-12-26 17:51:46,426][105620] Updated weights for policy 1, policy_version 343595 (0.0007) [2023-12-26 17:51:46,470][105620] Updated weights for policy 1, policy_version 343605 (0.0010) [2023-12-26 17:51:46,518][105620] Updated weights for policy 1, policy_version 343615 (0.0010) [2023-12-26 17:51:46,590][105692] Updated weights for policy 0, policy_version 343280 (0.0009) [2023-12-26 17:51:46,647][105692] Updated weights for policy 0, policy_version 343290 (0.0008) [2023-12-26 17:51:46,707][105692] Updated weights for policy 0, policy_version 343300 (0.0007) [2023-12-26 17:51:47,275][105620] Updated weights for policy 1, policy_version 343625 (0.0008) [2023-12-26 17:51:47,340][105620] Updated weights for policy 1, policy_version 343635 (0.0011) [2023-12-26 17:51:47,341][105692] Updated weights for policy 0, policy_version 343310 (0.0009) [2023-12-26 17:51:47,396][105692] Updated weights for policy 0, policy_version 343320 (0.0006) [2023-12-26 17:51:47,402][105620] Updated weights for policy 1, policy_version 343645 (0.0010) [2023-12-26 17:51:47,462][105692] Updated weights for policy 0, policy_version 343330 (0.0005) [2023-12-26 17:51:47,471][105620] Updated weights for policy 1, policy_version 343655 (0.0010) [2023-12-26 17:51:48,098][105692] Updated weights for policy 0, policy_version 343340 (0.0007) [2023-12-26 17:51:48,148][105692] Updated weights for policy 0, policy_version 343350 (0.0008) [2023-12-26 17:51:48,180][105620] Updated weights for policy 1, policy_version 343665 (0.0010) [2023-12-26 17:51:48,206][105692] Updated weights for policy 0, policy_version 343360 (0.0006) [2023-12-26 17:51:48,235][105620] Updated weights for policy 1, policy_version 343675 (0.0010) [2023-12-26 17:51:48,279][105620] Updated weights for policy 1, policy_version 343685 (0.0010) [2023-12-26 17:51:48,985][105692] Updated weights for policy 0, policy_version 343370 (0.0006) [2023-12-26 17:51:49,032][105620] Updated weights for policy 1, policy_version 343695 (0.0010) [2023-12-26 17:51:49,038][105692] Updated weights for policy 0, policy_version 343380 (0.0007) [2023-12-26 17:51:49,083][105620] Updated weights for policy 1, policy_version 343705 (0.0010) [2023-12-26 17:51:49,085][105692] Updated weights for policy 0, policy_version 343390 (0.0005) [2023-12-26 17:51:49,128][105620] Updated weights for policy 1, policy_version 343715 (0.0010) [2023-12-26 17:51:49,134][105692] Updated weights for policy 0, policy_version 343400 (0.0005) [2023-12-26 17:51:49,866][105620] Updated weights for policy 1, policy_version 343725 (0.0011) [2023-12-26 17:51:49,938][105620] Updated weights for policy 1, policy_version 343735 (0.0008) [2023-12-26 17:51:49,951][105692] Updated weights for policy 0, policy_version 343410 (0.0008) [2023-12-26 17:51:49,990][105620] Updated weights for policy 1, policy_version 343745 (0.0008) [2023-12-26 17:51:50,009][105692] Updated weights for policy 0, policy_version 343420 (0.0007) [2023-12-26 17:51:50,061][105692] Updated weights for policy 0, policy_version 343430 (0.0007) [2023-12-26 17:51:50,723][105620] Updated weights for policy 1, policy_version 343755 (0.0008) [2023-12-26 17:51:50,785][105620] Updated weights for policy 1, policy_version 343765 (0.0009) [2023-12-26 17:51:50,842][105620] Updated weights for policy 1, policy_version 343775 (0.0008) [2023-12-26 17:51:50,874][105692] Updated weights for policy 0, policy_version 343440 (0.0008) [2023-12-26 17:51:50,930][105692] Updated weights for policy 0, policy_version 343450 (0.0008) [2023-12-26 17:51:50,981][105692] Updated weights for policy 0, policy_version 343460 (0.0007) [2023-12-26 17:51:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 175955968. Throughput: 0: 9681.2, 1: 9791.8. Samples: 175940080. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-26 17:51:51,067][104569] Avg episode reward: [(0, '9268.006'), (1, '8997.194')] [2023-12-26 17:51:51,647][105620] Updated weights for policy 1, policy_version 343785 (0.0007) [2023-12-26 17:51:51,679][105692] Updated weights for policy 0, policy_version 343470 (0.0006) [2023-12-26 17:51:51,716][105620] Updated weights for policy 1, policy_version 343795 (0.0008) [2023-12-26 17:51:51,749][105692] Updated weights for policy 0, policy_version 343480 (0.0008) [2023-12-26 17:51:51,781][105620] Updated weights for policy 1, policy_version 343805 (0.0006) [2023-12-26 17:51:51,816][105692] Updated weights for policy 0, policy_version 343490 (0.0008) [2023-12-26 17:51:51,844][105620] Updated weights for policy 1, policy_version 343815 (0.0007) [2023-12-26 17:51:52,475][105620] Updated weights for policy 1, policy_version 343825 (0.0009) [2023-12-26 17:51:52,521][105692] Updated weights for policy 0, policy_version 343500 (0.0007) [2023-12-26 17:51:52,524][105620] Updated weights for policy 1, policy_version 343835 (0.0008) [2023-12-26 17:51:52,575][105692] Updated weights for policy 0, policy_version 343510 (0.0008) [2023-12-26 17:51:52,577][105620] Updated weights for policy 1, policy_version 343845 (0.0007) [2023-12-26 17:51:52,633][105692] Updated weights for policy 0, policy_version 343520 (0.0007) [2023-12-26 17:51:53,272][105692] Updated weights for policy 0, policy_version 343530 (0.0008) [2023-12-26 17:51:53,329][105692] Updated weights for policy 0, policy_version 343540 (0.0009) [2023-12-26 17:51:53,381][105692] Updated weights for policy 0, policy_version 343550 (0.0010) [2023-12-26 17:51:53,412][105620] Updated weights for policy 1, policy_version 343855 (0.0006) [2023-12-26 17:51:53,429][105692] Updated weights for policy 0, policy_version 343560 (0.0008) [2023-12-26 17:51:53,479][105620] Updated weights for policy 1, policy_version 343865 (0.0005) [2023-12-26 17:51:53,547][105620] Updated weights for policy 1, policy_version 343875 (0.0006) [2023-12-26 17:51:54,123][105692] Updated weights for policy 0, policy_version 343570 (0.0007) [2023-12-26 17:51:54,193][105692] Updated weights for policy 0, policy_version 343580 (0.0005) [2023-12-26 17:51:54,194][105620] Updated weights for policy 1, policy_version 343885 (0.0006) [2023-12-26 17:51:54,241][105620] Updated weights for policy 1, policy_version 343895 (0.0007) [2023-12-26 17:51:54,250][105692] Updated weights for policy 0, policy_version 343590 (0.0005) [2023-12-26 17:51:54,292][105620] Updated weights for policy 1, policy_version 343905 (0.0009) [2023-12-26 17:51:54,859][105692] Updated weights for policy 0, policy_version 343600 (0.0005) [2023-12-26 17:51:54,914][105692] Updated weights for policy 0, policy_version 343610 (0.0005) [2023-12-26 17:51:54,967][105692] Updated weights for policy 0, policy_version 343620 (0.0006) [2023-12-26 17:51:55,113][105620] Updated weights for policy 1, policy_version 343915 (0.0010) [2023-12-26 17:51:55,168][105620] Updated weights for policy 1, policy_version 343925 (0.0008) [2023-12-26 17:51:55,223][105620] Updated weights for policy 1, policy_version 343935 (0.0008) [2023-12-26 17:51:55,647][105692] Updated weights for policy 0, policy_version 343630 (0.0008) [2023-12-26 17:51:55,706][105692] Updated weights for policy 0, policy_version 343640 (0.0006) [2023-12-26 17:51:55,766][105692] Updated weights for policy 0, policy_version 343650 (0.0005) [2023-12-26 17:51:56,020][105620] Updated weights for policy 1, policy_version 343945 (0.0008) [2023-12-26 17:51:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 176046080. Throughput: 0: 9718.9, 1: 9790.4. Samples: 176055724. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-26 17:51:56,062][104569] Avg episode reward: [(0, '9086.044'), (1, '9266.512')] [2023-12-26 17:51:56,071][105620] Updated weights for policy 1, policy_version 343955 (0.0008) [2023-12-26 17:51:56,130][105620] Updated weights for policy 1, policy_version 343965 (0.0008) [2023-12-26 17:51:56,181][105620] Updated weights for policy 1, policy_version 343975 (0.0007) [2023-12-26 17:51:56,459][105692] Updated weights for policy 0, policy_version 343660 (0.0007) [2023-12-26 17:51:56,511][105692] Updated weights for policy 0, policy_version 343670 (0.0010) [2023-12-26 17:51:56,569][105692] Updated weights for policy 0, policy_version 343680 (0.0010) [2023-12-26 17:51:56,936][105620] Updated weights for policy 1, policy_version 343985 (0.0006) [2023-12-26 17:51:56,996][105620] Updated weights for policy 1, policy_version 343995 (0.0006) [2023-12-26 17:51:57,053][105620] Updated weights for policy 1, policy_version 344005 (0.0009) [2023-12-26 17:51:57,300][105692] Updated weights for policy 0, policy_version 343690 (0.0009) [2023-12-26 17:51:57,364][105692] Updated weights for policy 0, policy_version 343700 (0.0008) [2023-12-26 17:51:57,418][105692] Updated weights for policy 0, policy_version 343710 (0.0010) [2023-12-26 17:51:57,481][105692] Updated weights for policy 0, policy_version 343720 (0.0010) [2023-12-26 17:51:57,667][105620] Updated weights for policy 1, policy_version 344015 (0.0010) [2023-12-26 17:51:57,732][105620] Updated weights for policy 1, policy_version 344025 (0.0010) [2023-12-26 17:51:57,779][105620] Updated weights for policy 1, policy_version 344035 (0.0010) [2023-12-26 17:51:58,101][105692] Updated weights for policy 0, policy_version 343730 (0.0006) [2023-12-26 17:51:58,165][105692] Updated weights for policy 0, policy_version 343740 (0.0010) [2023-12-26 17:51:58,223][105692] Updated weights for policy 0, policy_version 343750 (0.0010) [2023-12-26 17:51:58,561][105620] Updated weights for policy 1, policy_version 344045 (0.0010) [2023-12-26 17:51:58,621][105620] Updated weights for policy 1, policy_version 344055 (0.0011) [2023-12-26 17:51:58,679][105620] Updated weights for policy 1, policy_version 344065 (0.0010) [2023-12-26 17:51:58,967][105692] Updated weights for policy 0, policy_version 343760 (0.0008) [2023-12-26 17:51:59,030][105692] Updated weights for policy 0, policy_version 343770 (0.0009) [2023-12-26 17:51:59,094][105692] Updated weights for policy 0, policy_version 343780 (0.0009) [2023-12-26 17:51:59,477][105620] Updated weights for policy 1, policy_version 344075 (0.0009) [2023-12-26 17:51:59,535][105620] Updated weights for policy 1, policy_version 344085 (0.0009) [2023-12-26 17:51:59,592][105620] Updated weights for policy 1, policy_version 344095 (0.0009) [2023-12-26 17:51:59,916][105692] Updated weights for policy 0, policy_version 343790 (0.0009) [2023-12-26 17:51:59,980][105692] Updated weights for policy 0, policy_version 343800 (0.0009) [2023-12-26 17:52:00,044][105692] Updated weights for policy 0, policy_version 343810 (0.0009) [2023-12-26 17:52:00,266][105620] Updated weights for policy 1, policy_version 344105 (0.0008) [2023-12-26 17:52:00,319][105620] Updated weights for policy 1, policy_version 344115 (0.0006) [2023-12-26 17:52:00,385][105620] Updated weights for policy 1, policy_version 344125 (0.0006) [2023-12-26 17:52:00,442][105620] Updated weights for policy 1, policy_version 344135 (0.0006) [2023-12-26 17:52:00,770][105692] Updated weights for policy 0, policy_version 343820 (0.0008) [2023-12-26 17:52:00,821][105692] Updated weights for policy 0, policy_version 343830 (0.0005) [2023-12-26 17:52:00,858][105585] KL-divergence is very high: 159.2885 [2023-12-26 17:52:00,872][105692] Updated weights for policy 0, policy_version 343840 (0.0005) [2023-12-26 17:52:00,900][105585] KL-divergence is very high: 160.1335 [2023-12-26 17:52:00,961][105620] Updated weights for policy 1, policy_version 344145 (0.0006) [2023-12-26 17:52:01,015][105620] Updated weights for policy 1, policy_version 344155 (0.0007) [2023-12-26 17:52:01,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 176144384. Throughput: 0: 9736.9, 1: 9720.3. Samples: 176114076. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-26 17:52:01,062][104569] Avg episode reward: [(0, '8994.166'), (1, '9266.055')] [2023-12-26 17:52:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000343848_88039424.pth... [2023-12-26 17:52:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000342696_87744512.pth [2023-12-26 17:52:01,078][105620] Updated weights for policy 1, policy_version 344165 (0.0008) [2023-12-26 17:52:01,094][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000344168_88113152.pth... [2023-12-26 17:52:01,099][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000343048_87826432.pth [2023-12-26 17:52:01,608][105692] Updated weights for policy 0, policy_version 343850 (0.0009) [2023-12-26 17:52:01,665][105692] Updated weights for policy 0, policy_version 343860 (0.0008) [2023-12-26 17:52:01,727][105692] Updated weights for policy 0, policy_version 343870 (0.0010) [2023-12-26 17:52:01,747][105620] Updated weights for policy 1, policy_version 344175 (0.0009) [2023-12-26 17:52:01,786][105692] Updated weights for policy 0, policy_version 343880 (0.0007) [2023-12-26 17:52:01,798][105620] Updated weights for policy 1, policy_version 344185 (0.0008) [2023-12-26 17:52:01,856][105620] Updated weights for policy 1, policy_version 344195 (0.0007) [2023-12-26 17:52:02,423][105620] Updated weights for policy 1, policy_version 344205 (0.0007) [2023-12-26 17:52:02,473][105620] Updated weights for policy 1, policy_version 344215 (0.0008) [2023-12-26 17:52:02,518][105620] Updated weights for policy 1, policy_version 344225 (0.0008) [2023-12-26 17:52:02,536][105692] Updated weights for policy 0, policy_version 343890 (0.0008) [2023-12-26 17:52:02,587][105692] Updated weights for policy 0, policy_version 343900 (0.0009) [2023-12-26 17:52:02,641][105692] Updated weights for policy 0, policy_version 343910 (0.0009) [2023-12-26 17:52:03,217][105620] Updated weights for policy 1, policy_version 344235 (0.0007) [2023-12-26 17:52:03,273][105620] Updated weights for policy 1, policy_version 344245 (0.0009) [2023-12-26 17:52:03,286][105692] Updated weights for policy 0, policy_version 343920 (0.0006) [2023-12-26 17:52:03,330][105620] Updated weights for policy 1, policy_version 344255 (0.0008) [2023-12-26 17:52:03,344][105692] Updated weights for policy 0, policy_version 343930 (0.0005) [2023-12-26 17:52:03,389][105692] Updated weights for policy 0, policy_version 343940 (0.0005) [2023-12-26 17:52:04,016][105620] Updated weights for policy 1, policy_version 344265 (0.0008) [2023-12-26 17:52:04,037][105692] Updated weights for policy 0, policy_version 343950 (0.0007) [2023-12-26 17:52:04,084][105620] Updated weights for policy 1, policy_version 344275 (0.0006) [2023-12-26 17:52:04,095][105692] Updated weights for policy 0, policy_version 343960 (0.0008) [2023-12-26 17:52:04,154][105620] Updated weights for policy 1, policy_version 344285 (0.0006) [2023-12-26 17:52:04,158][105692] Updated weights for policy 0, policy_version 343970 (0.0010) [2023-12-26 17:52:04,216][105620] Updated weights for policy 1, policy_version 344295 (0.0006) [2023-12-26 17:52:04,902][105620] Updated weights for policy 1, policy_version 344305 (0.0006) [2023-12-26 17:52:04,909][105692] Updated weights for policy 0, policy_version 343980 (0.0010) [2023-12-26 17:52:04,959][105620] Updated weights for policy 1, policy_version 344315 (0.0008) [2023-12-26 17:52:04,965][105692] Updated weights for policy 0, policy_version 343990 (0.0007) [2023-12-26 17:52:05,019][105620] Updated weights for policy 1, policy_version 344325 (0.0007) [2023-12-26 17:52:05,030][105692] Updated weights for policy 0, policy_version 344000 (0.0010) [2023-12-26 17:52:05,682][105620] Updated weights for policy 1, policy_version 344335 (0.0006) [2023-12-26 17:52:05,728][105620] Updated weights for policy 1, policy_version 344345 (0.0006) [2023-12-26 17:52:05,767][105692] Updated weights for policy 0, policy_version 344010 (0.0010) [2023-12-26 17:52:05,774][105620] Updated weights for policy 1, policy_version 344355 (0.0008) [2023-12-26 17:52:05,825][105692] Updated weights for policy 0, policy_version 344020 (0.0008) [2023-12-26 17:52:05,884][105692] Updated weights for policy 0, policy_version 344030 (0.0007) [2023-12-26 17:52:05,940][105692] Updated weights for policy 0, policy_version 344040 (0.0005) [2023-12-26 17:52:06,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 176250880. Throughput: 0: 9703.9, 1: 9702.4. Samples: 176235928. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-26 17:52:06,063][104569] Avg episode reward: [(0, '9085.738'), (1, '9265.945')] [2023-12-26 17:52:06,479][105620] Updated weights for policy 1, policy_version 344365 (0.0010) [2023-12-26 17:52:06,538][105620] Updated weights for policy 1, policy_version 344375 (0.0010) [2023-12-26 17:52:06,597][105620] Updated weights for policy 1, policy_version 344385 (0.0009) [2023-12-26 17:52:06,659][105692] Updated weights for policy 0, policy_version 344050 (0.0006) [2023-12-26 17:52:06,726][105692] Updated weights for policy 0, policy_version 344060 (0.0009) [2023-12-26 17:52:06,784][105692] Updated weights for policy 0, policy_version 344070 (0.0010) [2023-12-26 17:52:07,257][105620] Updated weights for policy 1, policy_version 344395 (0.0009) [2023-12-26 17:52:07,323][105620] Updated weights for policy 1, policy_version 344405 (0.0009) [2023-12-26 17:52:07,389][105620] Updated weights for policy 1, policy_version 344415 (0.0009) [2023-12-26 17:52:07,595][105692] Updated weights for policy 0, policy_version 344080 (0.0007) [2023-12-26 17:52:07,660][105692] Updated weights for policy 0, policy_version 344090 (0.0009) [2023-12-26 17:52:07,722][105692] Updated weights for policy 0, policy_version 344100 (0.0009) [2023-12-26 17:52:08,014][105620] Updated weights for policy 1, policy_version 344425 (0.0009) [2023-12-26 17:52:08,070][105620] Updated weights for policy 1, policy_version 344435 (0.0005) [2023-12-26 17:52:08,130][105620] Updated weights for policy 1, policy_version 344445 (0.0005) [2023-12-26 17:52:08,191][105620] Updated weights for policy 1, policy_version 344455 (0.0006) [2023-12-26 17:52:08,494][105692] Updated weights for policy 0, policy_version 344110 (0.0010) [2023-12-26 17:52:08,555][105692] Updated weights for policy 0, policy_version 344120 (0.0009) [2023-12-26 17:52:08,619][105692] Updated weights for policy 0, policy_version 344130 (0.0009) [2023-12-26 17:52:08,799][105620] Updated weights for policy 1, policy_version 344465 (0.0006) [2023-12-26 17:52:08,858][105620] Updated weights for policy 1, policy_version 344475 (0.0008) [2023-12-26 17:52:08,919][105620] Updated weights for policy 1, policy_version 344485 (0.0009) [2023-12-26 17:52:09,413][105692] Updated weights for policy 0, policy_version 344140 (0.0009) [2023-12-26 17:52:09,461][105692] Updated weights for policy 0, policy_version 344150 (0.0008) [2023-12-26 17:52:09,525][105692] Updated weights for policy 0, policy_version 344160 (0.0008) [2023-12-26 17:52:09,669][105620] Updated weights for policy 1, policy_version 344495 (0.0009) [2023-12-26 17:52:09,728][105620] Updated weights for policy 1, policy_version 344505 (0.0009) [2023-12-26 17:52:09,794][105620] Updated weights for policy 1, policy_version 344515 (0.0010) [2023-12-26 17:52:10,269][105692] Updated weights for policy 0, policy_version 344170 (0.0008) [2023-12-26 17:52:10,321][105692] Updated weights for policy 0, policy_version 344180 (0.0009) [2023-12-26 17:52:10,388][105692] Updated weights for policy 0, policy_version 344190 (0.0008) [2023-12-26 17:52:10,449][105692] Updated weights for policy 0, policy_version 344200 (0.0008) [2023-12-26 17:52:10,592][105620] Updated weights for policy 1, policy_version 344525 (0.0009) [2023-12-26 17:52:10,657][105620] Updated weights for policy 1, policy_version 344535 (0.0009) [2023-12-26 17:52:10,713][105620] Updated weights for policy 1, policy_version 344545 (0.0009) [2023-12-26 17:52:11,059][105692] Updated weights for policy 0, policy_version 344210 (0.0009) [2023-12-26 17:52:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 176340992. Throughput: 0: 9624.0, 1: 9700.0. Samples: 176349496. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-26 17:52:11,062][104569] Avg episode reward: [(0, '9087.908'), (1, '9175.195')] [2023-12-26 17:52:11,128][105692] Updated weights for policy 0, policy_version 344220 (0.0012) [2023-12-26 17:52:11,192][105692] Updated weights for policy 0, policy_version 344230 (0.0010) [2023-12-26 17:52:11,518][105620] Updated weights for policy 1, policy_version 344555 (0.0010) [2023-12-26 17:52:11,578][105620] Updated weights for policy 1, policy_version 344565 (0.0011) [2023-12-26 17:52:11,643][105620] Updated weights for policy 1, policy_version 344575 (0.0011) [2023-12-26 17:52:12,000][105692] Updated weights for policy 0, policy_version 344240 (0.0010) [2023-12-26 17:52:12,051][105692] Updated weights for policy 0, policy_version 344250 (0.0009) [2023-12-26 17:52:12,113][105692] Updated weights for policy 0, policy_version 344260 (0.0009) [2023-12-26 17:52:12,352][105620] Updated weights for policy 1, policy_version 344585 (0.0006) [2023-12-26 17:52:12,414][105620] Updated weights for policy 1, policy_version 344595 (0.0008) [2023-12-26 17:52:12,465][105620] Updated weights for policy 1, policy_version 344605 (0.0009) [2023-12-26 17:52:12,520][105620] Updated weights for policy 1, policy_version 344615 (0.0009) [2023-12-26 17:52:12,881][105692] Updated weights for policy 0, policy_version 344270 (0.0009) [2023-12-26 17:52:12,929][105692] Updated weights for policy 0, policy_version 344280 (0.0009) [2023-12-26 17:52:12,978][105692] Updated weights for policy 0, policy_version 344290 (0.0008) [2023-12-26 17:52:13,314][105620] Updated weights for policy 1, policy_version 344625 (0.0009) [2023-12-26 17:52:13,368][105620] Updated weights for policy 1, policy_version 344635 (0.0009) [2023-12-26 17:52:13,425][105620] Updated weights for policy 1, policy_version 344645 (0.0009) [2023-12-26 17:52:13,633][105692] Updated weights for policy 0, policy_version 344300 (0.0005) [2023-12-26 17:52:13,686][105692] Updated weights for policy 0, policy_version 344310 (0.0005) [2023-12-26 17:52:13,732][105692] Updated weights for policy 0, policy_version 344320 (0.0005) [2023-12-26 17:52:14,285][105620] Updated weights for policy 1, policy_version 344655 (0.0010) [2023-12-26 17:52:14,321][105692] Updated weights for policy 0, policy_version 344330 (0.0006) [2023-12-26 17:52:14,344][105620] Updated weights for policy 1, policy_version 344665 (0.0010) [2023-12-26 17:52:14,368][105692] Updated weights for policy 0, policy_version 344340 (0.0007) [2023-12-26 17:52:14,402][105620] Updated weights for policy 1, policy_version 344675 (0.0010) [2023-12-26 17:52:14,430][105692] Updated weights for policy 0, policy_version 344350 (0.0008) [2023-12-26 17:52:14,491][105692] Updated weights for policy 0, policy_version 344360 (0.0010) [2023-12-26 17:52:15,111][105620] Updated weights for policy 1, policy_version 344685 (0.0008) [2023-12-26 17:52:15,174][105620] Updated weights for policy 1, policy_version 344695 (0.0011) [2023-12-26 17:52:15,177][105692] Updated weights for policy 0, policy_version 344370 (0.0006) [2023-12-26 17:52:15,234][105620] Updated weights for policy 1, policy_version 344705 (0.0011) [2023-12-26 17:52:15,243][105692] Updated weights for policy 0, policy_version 344380 (0.0006) [2023-12-26 17:52:15,310][105692] Updated weights for policy 0, policy_version 344390 (0.0005) [2023-12-26 17:52:15,941][105692] Updated weights for policy 0, policy_version 344400 (0.0008) [2023-12-26 17:52:15,978][105620] Updated weights for policy 1, policy_version 344715 (0.0011) [2023-12-26 17:52:15,997][105692] Updated weights for policy 0, policy_version 344410 (0.0008) [2023-12-26 17:52:16,036][105620] Updated weights for policy 1, policy_version 344725 (0.0010) [2023-12-26 17:52:16,047][105692] Updated weights for policy 0, policy_version 344420 (0.0007) [2023-12-26 17:52:16,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19251.3, 300 sec: 19494.2). Total num frames: 176431104. Throughput: 0: 9635.7, 1: 9696.4. Samples: 176406300. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-26 17:52:16,062][104569] Avg episode reward: [(0, '8905.048'), (1, '9265.438')] [2023-12-26 17:52:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000344424_88186880.pth... [2023-12-26 17:52:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000343272_87891968.pth [2023-12-26 17:52:16,099][105620] Updated weights for policy 1, policy_version 344735 (0.0010) [2023-12-26 17:52:16,151][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000344744_88260608.pth... [2023-12-26 17:52:16,156][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000343592_87965696.pth [2023-12-26 17:52:16,644][105692] Updated weights for policy 0, policy_version 344430 (0.0006) [2023-12-26 17:52:16,689][105692] Updated weights for policy 0, policy_version 344440 (0.0005) [2023-12-26 17:52:16,741][105692] Updated weights for policy 0, policy_version 344450 (0.0005) [2023-12-26 17:52:16,838][105620] Updated weights for policy 1, policy_version 344745 (0.0010) [2023-12-26 17:52:16,891][105620] Updated weights for policy 1, policy_version 344755 (0.0011) [2023-12-26 17:52:16,951][105620] Updated weights for policy 1, policy_version 344765 (0.0010) [2023-12-26 17:52:17,013][105620] Updated weights for policy 1, policy_version 344775 (0.0010) [2023-12-26 17:52:17,418][105692] Updated weights for policy 0, policy_version 344460 (0.0007) [2023-12-26 17:52:17,479][105692] Updated weights for policy 0, policy_version 344470 (0.0005) [2023-12-26 17:52:17,525][105692] Updated weights for policy 0, policy_version 344480 (0.0005) [2023-12-26 17:52:17,644][105620] Updated weights for policy 1, policy_version 344785 (0.0009) [2023-12-26 17:52:17,699][105620] Updated weights for policy 1, policy_version 344795 (0.0010) [2023-12-26 17:52:17,760][105620] Updated weights for policy 1, policy_version 344805 (0.0010) [2023-12-26 17:52:18,117][105692] Updated weights for policy 0, policy_version 344490 (0.0005) [2023-12-26 17:52:18,181][105692] Updated weights for policy 0, policy_version 344500 (0.0009) [2023-12-26 17:52:18,240][105692] Updated weights for policy 0, policy_version 344510 (0.0008) [2023-12-26 17:52:18,306][105692] Updated weights for policy 0, policy_version 344520 (0.0005) [2023-12-26 17:52:18,473][105620] Updated weights for policy 1, policy_version 344815 (0.0009) [2023-12-26 17:52:18,521][105620] Updated weights for policy 1, policy_version 344825 (0.0008) [2023-12-26 17:52:18,570][105620] Updated weights for policy 1, policy_version 344835 (0.0009) [2023-12-26 17:52:19,056][105692] Updated weights for policy 0, policy_version 344530 (0.0009) [2023-12-26 17:52:19,118][105692] Updated weights for policy 0, policy_version 344540 (0.0009) [2023-12-26 17:52:19,178][105692] Updated weights for policy 0, policy_version 344550 (0.0008) [2023-12-26 17:52:19,326][105620] Updated weights for policy 1, policy_version 344845 (0.0008) [2023-12-26 17:52:19,393][105620] Updated weights for policy 1, policy_version 344855 (0.0009) [2023-12-26 17:52:19,452][105620] Updated weights for policy 1, policy_version 344865 (0.0010) [2023-12-26 17:52:19,846][105692] Updated weights for policy 0, policy_version 344560 (0.0007) [2023-12-26 17:52:19,903][105692] Updated weights for policy 0, policy_version 344570 (0.0008) [2023-12-26 17:52:19,961][105692] Updated weights for policy 0, policy_version 344580 (0.0009) [2023-12-26 17:52:20,260][105620] Updated weights for policy 1, policy_version 344875 (0.0009) [2023-12-26 17:52:20,320][105620] Updated weights for policy 1, policy_version 344885 (0.0009) [2023-12-26 17:52:20,384][105620] Updated weights for policy 1, policy_version 344895 (0.0008) [2023-12-26 17:52:20,686][105692] Updated weights for policy 0, policy_version 344590 (0.0009) [2023-12-26 17:52:20,739][105692] Updated weights for policy 0, policy_version 344600 (0.0010) [2023-12-26 17:52:20,792][105692] Updated weights for policy 0, policy_version 344610 (0.0010) [2023-12-26 17:52:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 176537600. Throughput: 0: 9762.0, 1: 9626.9. Samples: 176526712. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-26 17:52:21,062][104569] Avg episode reward: [(0, '8724.597'), (1, '9355.981')] [2023-12-26 17:52:21,148][105620] Updated weights for policy 1, policy_version 344905 (0.0009) [2023-12-26 17:52:21,207][105620] Updated weights for policy 1, policy_version 344915 (0.0010) [2023-12-26 17:52:21,275][105620] Updated weights for policy 1, policy_version 344925 (0.0009) [2023-12-26 17:52:21,342][105620] Updated weights for policy 1, policy_version 344935 (0.0011) [2023-12-26 17:52:21,493][105692] Updated weights for policy 0, policy_version 344620 (0.0009) [2023-12-26 17:52:21,559][105692] Updated weights for policy 0, policy_version 344630 (0.0005) [2023-12-26 17:52:21,629][105692] Updated weights for policy 0, policy_version 344640 (0.0007) [2023-12-26 17:52:22,092][105620] Updated weights for policy 1, policy_version 344945 (0.0008) [2023-12-26 17:52:22,163][105620] Updated weights for policy 1, policy_version 344955 (0.0006) [2023-12-26 17:52:22,227][105620] Updated weights for policy 1, policy_version 344965 (0.0006) [2023-12-26 17:52:22,349][105692] Updated weights for policy 0, policy_version 344650 (0.0010) [2023-12-26 17:52:22,414][105692] Updated weights for policy 0, policy_version 344660 (0.0010) [2023-12-26 17:52:22,480][105692] Updated weights for policy 0, policy_version 344670 (0.0011) [2023-12-26 17:52:22,545][105692] Updated weights for policy 0, policy_version 344680 (0.0011) [2023-12-26 17:52:22,880][105620] Updated weights for policy 1, policy_version 344975 (0.0006) [2023-12-26 17:52:22,934][105620] Updated weights for policy 1, policy_version 344985 (0.0008) [2023-12-26 17:52:22,995][105620] Updated weights for policy 1, policy_version 344995 (0.0008) [2023-12-26 17:52:23,235][105692] Updated weights for policy 0, policy_version 344690 (0.0010) [2023-12-26 17:52:23,294][105692] Updated weights for policy 0, policy_version 344700 (0.0010) [2023-12-26 17:52:23,363][105692] Updated weights for policy 0, policy_version 344710 (0.0010) [2023-12-26 17:52:23,615][105620] Updated weights for policy 1, policy_version 345005 (0.0009) [2023-12-26 17:52:23,665][105620] Updated weights for policy 1, policy_version 345015 (0.0008) [2023-12-26 17:52:23,718][105620] Updated weights for policy 1, policy_version 345025 (0.0010) [2023-12-26 17:52:23,966][105692] Updated weights for policy 0, policy_version 344720 (0.0010) [2023-12-26 17:52:24,024][105692] Updated weights for policy 0, policy_version 344730 (0.0010) [2023-12-26 17:52:24,076][105692] Updated weights for policy 0, policy_version 344740 (0.0010) [2023-12-26 17:52:24,542][105620] Updated weights for policy 1, policy_version 345035 (0.0009) [2023-12-26 17:52:24,595][105620] Updated weights for policy 1, policy_version 345045 (0.0008) [2023-12-26 17:52:24,644][105620] Updated weights for policy 1, policy_version 345055 (0.0008) [2023-12-26 17:52:24,805][105692] Updated weights for policy 0, policy_version 344750 (0.0010) [2023-12-26 17:52:24,866][105692] Updated weights for policy 0, policy_version 344760 (0.0010) [2023-12-26 17:52:24,928][105692] Updated weights for policy 0, policy_version 344770 (0.0010) [2023-12-26 17:52:25,414][105620] Updated weights for policy 1, policy_version 345065 (0.0008) [2023-12-26 17:52:25,470][105620] Updated weights for policy 1, policy_version 345075 (0.0010) [2023-12-26 17:52:25,525][105620] Updated weights for policy 1, policy_version 345085 (0.0010) [2023-12-26 17:52:25,567][105692] Updated weights for policy 0, policy_version 344780 (0.0010) [2023-12-26 17:52:25,583][105620] Updated weights for policy 1, policy_version 345095 (0.0010) [2023-12-26 17:52:25,614][105692] Updated weights for policy 0, policy_version 344790 (0.0010) [2023-12-26 17:52:25,662][105692] Updated weights for policy 0, policy_version 344800 (0.0009) [2023-12-26 17:52:26,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 176635904. Throughput: 0: 9833.8, 1: 9608.9. Samples: 176644540. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-26 17:52:26,063][104569] Avg episode reward: [(0, '8367.084'), (1, '9355.980')] [2023-12-26 17:52:26,224][105620] Updated weights for policy 1, policy_version 345105 (0.0007) [2023-12-26 17:52:26,271][105620] Updated weights for policy 1, policy_version 345115 (0.0010) [2023-12-26 17:52:26,325][105620] Updated weights for policy 1, policy_version 345125 (0.0010) [2023-12-26 17:52:26,358][105692] Updated weights for policy 0, policy_version 344810 (0.0008) [2023-12-26 17:52:26,409][105692] Updated weights for policy 0, policy_version 344820 (0.0008) [2023-12-26 17:52:26,469][105692] Updated weights for policy 0, policy_version 344830 (0.0008) [2023-12-26 17:52:26,540][105692] Updated weights for policy 0, policy_version 344840 (0.0009) [2023-12-26 17:52:26,941][105620] Updated weights for policy 1, policy_version 345135 (0.0007) [2023-12-26 17:52:26,990][105620] Updated weights for policy 1, policy_version 345145 (0.0005) [2023-12-26 17:52:27,041][105620] Updated weights for policy 1, policy_version 345155 (0.0005) [2023-12-26 17:52:27,399][105692] Updated weights for policy 0, policy_version 344850 (0.0009) [2023-12-26 17:52:27,463][105692] Updated weights for policy 0, policy_version 344860 (0.0010) [2023-12-26 17:52:27,522][105692] Updated weights for policy 0, policy_version 344870 (0.0011) [2023-12-26 17:52:27,667][105620] Updated weights for policy 1, policy_version 345165 (0.0005) [2023-12-26 17:52:27,729][105620] Updated weights for policy 1, policy_version 345175 (0.0005) [2023-12-26 17:52:27,793][105620] Updated weights for policy 1, policy_version 345185 (0.0007) [2023-12-26 17:52:28,174][105692] Updated weights for policy 0, policy_version 344880 (0.0006) [2023-12-26 17:52:28,229][105692] Updated weights for policy 0, policy_version 344890 (0.0005) [2023-12-26 17:52:28,285][105692] Updated weights for policy 0, policy_version 344900 (0.0005) [2023-12-26 17:52:28,315][105620] Updated weights for policy 1, policy_version 345195 (0.0009) [2023-12-26 17:52:28,378][105620] Updated weights for policy 1, policy_version 345205 (0.0008) [2023-12-26 17:52:28,435][105620] Updated weights for policy 1, policy_version 345215 (0.0010) [2023-12-26 17:52:28,947][105692] Updated weights for policy 0, policy_version 344910 (0.0007) [2023-12-26 17:52:29,001][105692] Updated weights for policy 0, policy_version 344920 (0.0010) [2023-12-26 17:52:29,053][105620] Updated weights for policy 1, policy_version 345225 (0.0007) [2023-12-26 17:52:29,055][105692] Updated weights for policy 0, policy_version 344930 (0.0010) [2023-12-26 17:52:29,111][105620] Updated weights for policy 1, policy_version 345235 (0.0005) [2023-12-26 17:52:29,158][105620] Updated weights for policy 1, policy_version 345245 (0.0005) [2023-12-26 17:52:29,207][105620] Updated weights for policy 1, policy_version 345255 (0.0005) [2023-12-26 17:52:29,796][105692] Updated weights for policy 0, policy_version 344940 (0.0010) [2023-12-26 17:52:29,860][105692] Updated weights for policy 0, policy_version 344950 (0.0008) [2023-12-26 17:52:29,891][105620] Updated weights for policy 1, policy_version 345265 (0.0007) [2023-12-26 17:52:29,915][105692] Updated weights for policy 0, policy_version 344960 (0.0008) [2023-12-26 17:52:29,960][105620] Updated weights for policy 1, policy_version 345275 (0.0008) [2023-12-26 17:52:30,022][105620] Updated weights for policy 1, policy_version 345285 (0.0008) [2023-12-26 17:52:30,615][105692] Updated weights for policy 0, policy_version 344970 (0.0007) [2023-12-26 17:52:30,672][105692] Updated weights for policy 0, policy_version 344980 (0.0008) [2023-12-26 17:52:30,722][105692] Updated weights for policy 0, policy_version 344990 (0.0007) [2023-12-26 17:52:30,776][105692] Updated weights for policy 0, policy_version 345000 (0.0005) [2023-12-26 17:52:30,778][105620] Updated weights for policy 1, policy_version 345295 (0.0009) [2023-12-26 17:52:30,843][105620] Updated weights for policy 1, policy_version 345305 (0.0008) [2023-12-26 17:52:30,900][105620] Updated weights for policy 1, policy_version 345315 (0.0008) [2023-12-26 17:52:31,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 176742400. Throughput: 0: 9837.3, 1: 9790.2. Samples: 176708172. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-26 17:52:31,063][104569] Avg episode reward: [(0, '8636.034'), (1, '9355.918')] [2023-12-26 17:52:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000345000_88334336.pth... [2023-12-26 17:52:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000345320_88408064.pth... [2023-12-26 17:52:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000343848_88039424.pth [2023-12-26 17:52:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000344168_88113152.pth [2023-12-26 17:52:31,514][105692] Updated weights for policy 0, policy_version 345010 (0.0010) [2023-12-26 17:52:31,569][105692] Updated weights for policy 0, policy_version 345020 (0.0010) [2023-12-26 17:52:31,627][105620] Updated weights for policy 1, policy_version 345325 (0.0008) [2023-12-26 17:52:31,631][105692] Updated weights for policy 0, policy_version 345030 (0.0009) [2023-12-26 17:52:31,691][105620] Updated weights for policy 1, policy_version 345335 (0.0009) [2023-12-26 17:52:31,780][105620] Updated weights for policy 1, policy_version 345345 (0.0007) [2023-12-26 17:52:32,416][105692] Updated weights for policy 0, policy_version 345040 (0.0009) [2023-12-26 17:52:32,463][105692] Updated weights for policy 0, policy_version 345050 (0.0009) [2023-12-26 17:52:32,472][105620] Updated weights for policy 1, policy_version 345355 (0.0008) [2023-12-26 17:52:32,520][105692] Updated weights for policy 0, policy_version 345060 (0.0008) [2023-12-26 17:52:32,533][105620] Updated weights for policy 1, policy_version 345365 (0.0008) [2023-12-26 17:52:32,586][105620] Updated weights for policy 1, policy_version 345375 (0.0008) [2023-12-26 17:52:33,271][105620] Updated weights for policy 1, policy_version 345385 (0.0009) [2023-12-26 17:52:33,318][105692] Updated weights for policy 0, policy_version 345070 (0.0008) [2023-12-26 17:52:33,327][105620] Updated weights for policy 1, policy_version 345395 (0.0007) [2023-12-26 17:52:33,360][105692] Updated weights for policy 0, policy_version 345080 (0.0009) [2023-12-26 17:52:33,374][105620] Updated weights for policy 1, policy_version 345405 (0.0007) [2023-12-26 17:52:33,401][105692] Updated weights for policy 0, policy_version 345090 (0.0005) [2023-12-26 17:52:33,425][105620] Updated weights for policy 1, policy_version 345415 (0.0008) [2023-12-26 17:52:34,007][105692] Updated weights for policy 0, policy_version 345100 (0.0010) [2023-12-26 17:52:34,055][105692] Updated weights for policy 0, policy_version 345110 (0.0010) [2023-12-26 17:52:34,120][105692] Updated weights for policy 0, policy_version 345120 (0.0010) [2023-12-26 17:52:34,126][105585] KL-divergence is very high: 146.2094 [2023-12-26 17:52:34,150][105620] Updated weights for policy 1, policy_version 345425 (0.0007) [2023-12-26 17:52:34,209][105620] Updated weights for policy 1, policy_version 345435 (0.0008) [2023-12-26 17:52:34,269][105620] Updated weights for policy 1, policy_version 345445 (0.0011) [2023-12-26 17:52:34,849][105692] Updated weights for policy 0, policy_version 345130 (0.0009) [2023-12-26 17:52:34,860][105620] Updated weights for policy 1, policy_version 345455 (0.0010) [2023-12-26 17:52:34,904][105692] Updated weights for policy 0, policy_version 345140 (0.0008) [2023-12-26 17:52:34,926][105620] Updated weights for policy 1, policy_version 345465 (0.0010) [2023-12-26 17:52:34,953][105692] Updated weights for policy 0, policy_version 345150 (0.0010) [2023-12-26 17:52:34,985][105620] Updated weights for policy 1, policy_version 345475 (0.0010) [2023-12-26 17:52:35,009][105692] Updated weights for policy 0, policy_version 345160 (0.0010) [2023-12-26 17:52:35,644][105620] Updated weights for policy 1, policy_version 345485 (0.0006) [2023-12-26 17:52:35,706][105620] Updated weights for policy 1, policy_version 345495 (0.0006) [2023-12-26 17:52:35,745][105692] Updated weights for policy 0, policy_version 345170 (0.0009) [2023-12-26 17:52:35,769][105620] Updated weights for policy 1, policy_version 345505 (0.0008) [2023-12-26 17:52:35,810][105692] Updated weights for policy 0, policy_version 345180 (0.0008) [2023-12-26 17:52:35,869][105692] Updated weights for policy 0, policy_version 345190 (0.0010) [2023-12-26 17:52:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 176840704. Throughput: 0: 9855.0, 1: 9844.8. Samples: 176826572. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-26 17:52:36,062][104569] Avg episode reward: [(0, '8722.104'), (1, '9355.888')] [2023-12-26 17:52:36,470][105620] Updated weights for policy 1, policy_version 345515 (0.0011) [2023-12-26 17:52:36,530][105620] Updated weights for policy 1, policy_version 345525 (0.0010) [2023-12-26 17:52:36,599][105620] Updated weights for policy 1, policy_version 345535 (0.0007) [2023-12-26 17:52:36,632][105692] Updated weights for policy 0, policy_version 345200 (0.0010) [2023-12-26 17:52:36,691][105692] Updated weights for policy 0, policy_version 345210 (0.0010) [2023-12-26 17:52:36,743][105692] Updated weights for policy 0, policy_version 345220 (0.0010) [2023-12-26 17:52:37,327][105620] Updated weights for policy 1, policy_version 345545 (0.0011) [2023-12-26 17:52:37,360][105692] Updated weights for policy 0, policy_version 345230 (0.0008) [2023-12-26 17:52:37,390][105620] Updated weights for policy 1, policy_version 345555 (0.0011) [2023-12-26 17:52:37,417][105692] Updated weights for policy 0, policy_version 345240 (0.0006) [2023-12-26 17:52:37,442][105620] Updated weights for policy 1, policy_version 345565 (0.0010) [2023-12-26 17:52:37,469][105692] Updated weights for policy 0, policy_version 345250 (0.0005) [2023-12-26 17:52:37,497][105620] Updated weights for policy 1, policy_version 345575 (0.0010) [2023-12-26 17:52:38,077][105692] Updated weights for policy 0, policy_version 345260 (0.0008) [2023-12-26 17:52:38,137][105692] Updated weights for policy 0, policy_version 345270 (0.0010) [2023-12-26 17:52:38,195][105692] Updated weights for policy 0, policy_version 345280 (0.0010) [2023-12-26 17:52:38,246][105620] Updated weights for policy 1, policy_version 345585 (0.0010) [2023-12-26 17:52:38,291][105620] Updated weights for policy 1, policy_version 345595 (0.0010) [2023-12-26 17:52:38,351][105620] Updated weights for policy 1, policy_version 345605 (0.0009) [2023-12-26 17:52:38,931][105692] Updated weights for policy 0, policy_version 345290 (0.0010) [2023-12-26 17:52:38,996][105692] Updated weights for policy 0, policy_version 345300 (0.0010) [2023-12-26 17:52:39,031][105620] Updated weights for policy 1, policy_version 345615 (0.0006) [2023-12-26 17:52:39,062][105692] Updated weights for policy 0, policy_version 345310 (0.0011) [2023-12-26 17:52:39,087][105620] Updated weights for policy 1, policy_version 345625 (0.0009) [2023-12-26 17:52:39,127][105692] Updated weights for policy 0, policy_version 345320 (0.0010) [2023-12-26 17:52:39,148][105620] Updated weights for policy 1, policy_version 345635 (0.0009) [2023-12-26 17:52:39,788][105620] Updated weights for policy 1, policy_version 345645 (0.0006) [2023-12-26 17:52:39,836][105692] Updated weights for policy 0, policy_version 345330 (0.0009) [2023-12-26 17:52:39,851][105620] Updated weights for policy 1, policy_version 345655 (0.0007) [2023-12-26 17:52:39,896][105692] Updated weights for policy 0, policy_version 345340 (0.0007) [2023-12-26 17:52:39,911][105620] Updated weights for policy 1, policy_version 345665 (0.0008) [2023-12-26 17:52:39,962][105692] Updated weights for policy 0, policy_version 345350 (0.0007) [2023-12-26 17:52:40,617][105620] Updated weights for policy 1, policy_version 345675 (0.0008) [2023-12-26 17:52:40,672][105620] Updated weights for policy 1, policy_version 345685 (0.0005) [2023-12-26 17:52:40,713][105692] Updated weights for policy 0, policy_version 345360 (0.0008) [2023-12-26 17:52:40,733][105620] Updated weights for policy 1, policy_version 345695 (0.0006) [2023-12-26 17:52:40,735][105585] KL-divergence is very high: 114.2680 [2023-12-26 17:52:40,746][105585] KL-divergence is very high: 184.1503 [2023-12-26 17:52:40,771][105692] Updated weights for policy 0, policy_version 345370 (0.0009) [2023-12-26 17:52:40,811][105585] KL-divergence is very high: 126.6601 [2023-12-26 17:52:40,827][105692] Updated weights for policy 0, policy_version 345380 (0.0010) [2023-12-26 17:52:41,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 176939008. Throughput: 0: 9833.5, 1: 9920.7. Samples: 176944664. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-26 17:52:41,062][104569] Avg episode reward: [(0, '5469.495'), (1, '9356.003')] [2023-12-26 17:52:41,470][105620] Updated weights for policy 1, policy_version 345705 (0.0007) [2023-12-26 17:52:41,529][105620] Updated weights for policy 1, policy_version 345715 (0.0008) [2023-12-26 17:52:41,588][105692] Updated weights for policy 0, policy_version 345390 (0.0009) [2023-12-26 17:52:41,592][105620] Updated weights for policy 1, policy_version 345725 (0.0008) [2023-12-26 17:52:41,607][105585] KL-divergence is very high: 545.3754 [2023-12-26 17:52:41,616][105585] KL-divergence is very high: 586.0101 [2023-12-26 17:52:41,655][105620] Updated weights for policy 1, policy_version 345735 (0.0008) [2023-12-26 17:52:41,659][105692] Updated weights for policy 0, policy_version 345400 (0.0011) [2023-12-26 17:52:41,663][105585] KL-divergence is very high: 883.8491 [2023-12-26 17:52:41,670][105585] KL-divergence is very high: 767.2028 [2023-12-26 17:52:41,714][105585] KL-divergence is very high: 799.1711 [2023-12-26 17:52:41,722][105692] Updated weights for policy 0, policy_version 345410 (0.0010) [2023-12-26 17:52:41,722][105585] KL-divergence is very high: 688.4351 [2023-12-26 17:52:42,477][105692] Updated weights for policy 0, policy_version 345420 (0.0008) [2023-12-26 17:52:42,512][105620] Updated weights for policy 1, policy_version 345745 (0.0007) [2023-12-26 17:52:42,539][105692] Updated weights for policy 0, policy_version 345430 (0.0007) [2023-12-26 17:52:42,563][105620] Updated weights for policy 1, policy_version 345755 (0.0006) [2023-12-26 17:52:42,598][105692] Updated weights for policy 0, policy_version 345440 (0.0007) [2023-12-26 17:52:42,613][105620] Updated weights for policy 1, policy_version 345765 (0.0007) [2023-12-26 17:52:43,228][105692] Updated weights for policy 0, policy_version 345450 (0.0008) [2023-12-26 17:52:43,286][105692] Updated weights for policy 0, policy_version 345460 (0.0007) [2023-12-26 17:52:43,353][105692] Updated weights for policy 0, policy_version 345470 (0.0006) [2023-12-26 17:52:43,356][105620] Updated weights for policy 1, policy_version 345775 (0.0010) [2023-12-26 17:52:43,404][105620] Updated weights for policy 1, policy_version 345785 (0.0010) [2023-12-26 17:52:43,414][105692] Updated weights for policy 0, policy_version 345480 (0.0007) [2023-12-26 17:52:43,448][105620] Updated weights for policy 1, policy_version 345795 (0.0010) [2023-12-26 17:52:44,147][105692] Updated weights for policy 0, policy_version 345490 (0.0006) [2023-12-26 17:52:44,151][105620] Updated weights for policy 1, policy_version 345805 (0.0008) [2023-12-26 17:52:44,201][105692] Updated weights for policy 0, policy_version 345500 (0.0005) [2023-12-26 17:52:44,215][105620] Updated weights for policy 1, policy_version 345815 (0.0008) [2023-12-26 17:52:44,252][105692] Updated weights for policy 0, policy_version 345510 (0.0005) [2023-12-26 17:52:44,270][105620] Updated weights for policy 1, policy_version 345825 (0.0010) [2023-12-26 17:52:44,858][105620] Updated weights for policy 1, policy_version 345835 (0.0009) [2023-12-26 17:52:44,922][105620] Updated weights for policy 1, policy_version 345845 (0.0008) [2023-12-26 17:52:44,965][105692] Updated weights for policy 0, policy_version 345520 (0.0006) [2023-12-26 17:52:44,978][105620] Updated weights for policy 1, policy_version 345855 (0.0011) [2023-12-26 17:52:44,997][105585] KL-divergence is very high: 100.2093 [2023-12-26 17:52:45,019][105692] Updated weights for policy 0, policy_version 345530 (0.0006) [2023-12-26 17:52:45,043][105585] KL-divergence is very high: 151.0170 [2023-12-26 17:52:45,080][105692] Updated weights for policy 0, policy_version 345540 (0.0005) [2023-12-26 17:52:45,092][105585] KL-divergence is very high: 134.5251 [2023-12-26 17:52:45,605][105620] Updated weights for policy 1, policy_version 345865 (0.0010) [2023-12-26 17:52:45,659][105692] Updated weights for policy 0, policy_version 345550 (0.0006) [2023-12-26 17:52:45,659][105620] Updated weights for policy 1, policy_version 345875 (0.0010) [2023-12-26 17:52:45,721][105620] Updated weights for policy 1, policy_version 345885 (0.0010) [2023-12-26 17:52:45,724][105692] Updated weights for policy 0, policy_version 345560 (0.0006) [2023-12-26 17:52:45,776][105620] Updated weights for policy 1, policy_version 345895 (0.0010) [2023-12-26 17:52:45,781][105692] Updated weights for policy 0, policy_version 345570 (0.0005) [2023-12-26 17:52:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 177037312. Throughput: 0: 9807.8, 1: 9902.7. Samples: 177001048. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 17:52:46,062][104569] Avg episode reward: [(0, '3350.758'), (1, '9355.970')] [2023-12-26 17:52:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000345896_88555520.pth... [2023-12-26 17:52:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000345576_88481792.pth... [2023-12-26 17:52:46,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000344424_88186880.pth [2023-12-26 17:52:46,069][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000344744_88260608.pth [2023-12-26 17:52:46,429][105620] Updated weights for policy 1, policy_version 345905 (0.0010) [2023-12-26 17:52:46,490][105620] Updated weights for policy 1, policy_version 345915 (0.0010) [2023-12-26 17:52:46,496][105692] Updated weights for policy 0, policy_version 345580 (0.0006) [2023-12-26 17:52:46,553][105620] Updated weights for policy 1, policy_version 345925 (0.0011) [2023-12-26 17:52:46,559][105692] Updated weights for policy 0, policy_version 345590 (0.0007) [2023-12-26 17:52:46,618][105692] Updated weights for policy 0, policy_version 345600 (0.0008) [2023-12-26 17:52:47,288][105620] Updated weights for policy 1, policy_version 345935 (0.0010) [2023-12-26 17:52:47,342][105620] Updated weights for policy 1, policy_version 345945 (0.0010) [2023-12-26 17:52:47,387][105692] Updated weights for policy 0, policy_version 345610 (0.0008) [2023-12-26 17:52:47,407][105620] Updated weights for policy 1, policy_version 345955 (0.0010) [2023-12-26 17:52:47,437][105692] Updated weights for policy 0, policy_version 345620 (0.0008) [2023-12-26 17:52:47,488][105692] Updated weights for policy 0, policy_version 345630 (0.0008) [2023-12-26 17:52:47,534][105692] Updated weights for policy 0, policy_version 345640 (0.0008) [2023-12-26 17:52:48,014][105620] Updated weights for policy 1, policy_version 345965 (0.0008) [2023-12-26 17:52:48,068][105620] Updated weights for policy 1, policy_version 345975 (0.0005) [2023-12-26 17:52:48,121][105620] Updated weights for policy 1, policy_version 345985 (0.0005) [2023-12-26 17:52:48,219][105692] Updated weights for policy 0, policy_version 345650 (0.0010) [2023-12-26 17:52:48,270][105692] Updated weights for policy 0, policy_version 345660 (0.0008) [2023-12-26 17:52:48,331][105692] Updated weights for policy 0, policy_version 345670 (0.0008) [2023-12-26 17:52:48,794][105620] Updated weights for policy 1, policy_version 345995 (0.0007) [2023-12-26 17:52:48,852][105620] Updated weights for policy 1, policy_version 346005 (0.0010) [2023-12-26 17:52:48,913][105620] Updated weights for policy 1, policy_version 346015 (0.0010) [2023-12-26 17:52:49,106][105692] Updated weights for policy 0, policy_version 345680 (0.0008) [2023-12-26 17:52:49,160][105692] Updated weights for policy 0, policy_version 345690 (0.0008) [2023-12-26 17:52:49,211][105692] Updated weights for policy 0, policy_version 345700 (0.0009) [2023-12-26 17:52:49,635][105620] Updated weights for policy 1, policy_version 346025 (0.0010) [2023-12-26 17:52:49,692][105620] Updated weights for policy 1, policy_version 346035 (0.0006) [2023-12-26 17:52:49,749][105620] Updated weights for policy 1, policy_version 346045 (0.0005) [2023-12-26 17:52:49,812][105620] Updated weights for policy 1, policy_version 346055 (0.0007) [2023-12-26 17:52:50,039][105692] Updated weights for policy 0, policy_version 345710 (0.0008) [2023-12-26 17:52:50,097][105692] Updated weights for policy 0, policy_version 345720 (0.0008) [2023-12-26 17:52:50,155][105692] Updated weights for policy 0, policy_version 345730 (0.0008) [2023-12-26 17:52:50,522][105620] Updated weights for policy 1, policy_version 346065 (0.0010) [2023-12-26 17:52:50,578][105620] Updated weights for policy 1, policy_version 346075 (0.0010) [2023-12-26 17:52:50,636][105620] Updated weights for policy 1, policy_version 346085 (0.0010) [2023-12-26 17:52:50,894][105692] Updated weights for policy 0, policy_version 345740 (0.0008) [2023-12-26 17:52:50,953][105692] Updated weights for policy 0, policy_version 345750 (0.0009) [2023-12-26 17:52:51,021][105692] Updated weights for policy 0, policy_version 345760 (0.0009) [2023-12-26 17:52:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 177127424. Throughput: 0: 9800.5, 1: 9886.7. Samples: 177121852. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 17:52:51,063][104569] Avg episode reward: [(0, '7012.957'), (1, '9355.920')] [2023-12-26 17:52:51,383][105620] Updated weights for policy 1, policy_version 346095 (0.0009) [2023-12-26 17:52:51,434][105620] Updated weights for policy 1, policy_version 346105 (0.0007) [2023-12-26 17:52:51,494][105620] Updated weights for policy 1, policy_version 346115 (0.0009) [2023-12-26 17:52:51,728][105692] Updated weights for policy 0, policy_version 345770 (0.0009) [2023-12-26 17:52:51,790][105692] Updated weights for policy 0, policy_version 345780 (0.0006) [2023-12-26 17:52:51,840][105692] Updated weights for policy 0, policy_version 345790 (0.0008) [2023-12-26 17:52:51,885][105692] Updated weights for policy 0, policy_version 345800 (0.0008) [2023-12-26 17:52:52,262][105620] Updated weights for policy 1, policy_version 346125 (0.0009) [2023-12-26 17:52:52,322][105620] Updated weights for policy 1, policy_version 346135 (0.0008) [2023-12-26 17:52:52,386][105620] Updated weights for policy 1, policy_version 346145 (0.0007) [2023-12-26 17:52:52,665][105692] Updated weights for policy 0, policy_version 345810 (0.0006) [2023-12-26 17:52:52,731][105692] Updated weights for policy 0, policy_version 345820 (0.0005) [2023-12-26 17:52:52,792][105692] Updated weights for policy 0, policy_version 345830 (0.0006) [2023-12-26 17:52:53,123][105620] Updated weights for policy 1, policy_version 346155 (0.0005) [2023-12-26 17:52:53,190][105620] Updated weights for policy 1, policy_version 346165 (0.0006) [2023-12-26 17:52:53,231][105586] KL-divergence is very high: 113.6345 [2023-12-26 17:52:53,254][105620] Updated weights for policy 1, policy_version 346175 (0.0008) [2023-12-26 17:52:53,271][105586] KL-divergence is very high: 115.3613 [2023-12-26 17:52:53,367][105692] Updated weights for policy 0, policy_version 345840 (0.0008) [2023-12-26 17:52:53,424][105692] Updated weights for policy 0, policy_version 345850 (0.0010) [2023-12-26 17:52:53,474][105692] Updated weights for policy 0, policy_version 345860 (0.0009) [2023-12-26 17:52:53,876][105620] Updated weights for policy 1, policy_version 346186 (0.0010) [2023-12-26 17:52:53,932][105620] Updated weights for policy 1, policy_version 346196 (0.0006) [2023-12-26 17:52:53,994][105620] Updated weights for policy 1, policy_version 346206 (0.0007) [2023-12-26 17:52:54,061][105620] Updated weights for policy 1, policy_version 346216 (0.0010) [2023-12-26 17:52:54,224][105692] Updated weights for policy 0, policy_version 345870 (0.0008) [2023-12-26 17:52:54,283][105692] Updated weights for policy 0, policy_version 345880 (0.0008) [2023-12-26 17:52:54,339][105692] Updated weights for policy 0, policy_version 345890 (0.0008) [2023-12-26 17:52:54,782][105620] Updated weights for policy 1, policy_version 346226 (0.0011) [2023-12-26 17:52:54,844][105620] Updated weights for policy 1, policy_version 346236 (0.0010) [2023-12-26 17:52:54,906][105620] Updated weights for policy 1, policy_version 346246 (0.0010) [2023-12-26 17:52:55,075][105692] Updated weights for policy 0, policy_version 345900 (0.0008) [2023-12-26 17:52:55,133][105692] Updated weights for policy 0, policy_version 345910 (0.0008) [2023-12-26 17:52:55,178][105692] Updated weights for policy 0, policy_version 345920 (0.0008) [2023-12-26 17:52:55,638][105620] Updated weights for policy 1, policy_version 346256 (0.0010) [2023-12-26 17:52:55,685][105620] Updated weights for policy 1, policy_version 346266 (0.0010) [2023-12-26 17:52:55,730][105620] Updated weights for policy 1, policy_version 346276 (0.0010) [2023-12-26 17:52:55,957][105692] Updated weights for policy 0, policy_version 345930 (0.0008) [2023-12-26 17:52:56,012][105692] Updated weights for policy 0, policy_version 345940 (0.0008) [2023-12-26 17:52:56,060][105692] Updated weights for policy 0, policy_version 345950 (0.0008) [2023-12-26 17:52:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 177225728. Throughput: 0: 9862.0, 1: 9856.6. Samples: 177236836. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 17:52:56,062][104569] Avg episode reward: [(0, '8903.832'), (1, '9265.695')] [2023-12-26 17:52:56,108][105692] Updated weights for policy 0, policy_version 345960 (0.0008) [2023-12-26 17:52:56,498][105620] Updated weights for policy 1, policy_version 346286 (0.0010) [2023-12-26 17:52:56,546][105620] Updated weights for policy 1, policy_version 346296 (0.0010) [2023-12-26 17:52:56,594][105620] Updated weights for policy 1, policy_version 346306 (0.0010) [2023-12-26 17:52:56,877][105692] Updated weights for policy 0, policy_version 345970 (0.0007) [2023-12-26 17:52:56,935][105692] Updated weights for policy 0, policy_version 345980 (0.0008) [2023-12-26 17:52:56,981][105692] Updated weights for policy 0, policy_version 345990 (0.0008) [2023-12-26 17:52:57,349][105620] Updated weights for policy 1, policy_version 346316 (0.0010) [2023-12-26 17:52:57,413][105620] Updated weights for policy 1, policy_version 346326 (0.0010) [2023-12-26 17:52:57,467][105620] Updated weights for policy 1, policy_version 346336 (0.0010) [2023-12-26 17:52:57,749][105692] Updated weights for policy 0, policy_version 346000 (0.0008) [2023-12-26 17:52:57,799][105692] Updated weights for policy 0, policy_version 346010 (0.0008) [2023-12-26 17:52:57,859][105692] Updated weights for policy 0, policy_version 346020 (0.0009) [2023-12-26 17:52:58,206][105620] Updated weights for policy 1, policy_version 346346 (0.0010) [2023-12-26 17:52:58,272][105620] Updated weights for policy 1, policy_version 346356 (0.0009) [2023-12-26 17:52:58,333][105620] Updated weights for policy 1, policy_version 346366 (0.0009) [2023-12-26 17:52:58,399][105620] Updated weights for policy 1, policy_version 346376 (0.0008) [2023-12-26 17:52:58,662][105692] Updated weights for policy 0, policy_version 346030 (0.0009) [2023-12-26 17:52:58,732][105692] Updated weights for policy 0, policy_version 346040 (0.0009) [2023-12-26 17:52:58,806][105692] Updated weights for policy 0, policy_version 346051 (0.0008) [2023-12-26 17:52:59,210][105620] Updated weights for policy 1, policy_version 346386 (0.0011) [2023-12-26 17:52:59,275][105620] Updated weights for policy 1, policy_version 346396 (0.0007) [2023-12-26 17:52:59,337][105620] Updated weights for policy 1, policy_version 346406 (0.0006) [2023-12-26 17:52:59,599][105692] Updated weights for policy 0, policy_version 346061 (0.0009) [2023-12-26 17:52:59,650][105692] Updated weights for policy 0, policy_version 346071 (0.0009) [2023-12-26 17:52:59,697][105692] Updated weights for policy 0, policy_version 346081 (0.0009) [2023-12-26 17:53:00,012][105620] Updated weights for policy 1, policy_version 346416 (0.0009) [2023-12-26 17:53:00,066][105620] Updated weights for policy 1, policy_version 346426 (0.0008) [2023-12-26 17:53:00,120][105620] Updated weights for policy 1, policy_version 346436 (0.0009) [2023-12-26 17:53:00,480][105692] Updated weights for policy 0, policy_version 346091 (0.0009) [2023-12-26 17:53:00,547][105692] Updated weights for policy 0, policy_version 346101 (0.0010) [2023-12-26 17:53:00,603][105692] Updated weights for policy 0, policy_version 346111 (0.0013) [2023-12-26 17:53:00,754][105620] Updated weights for policy 1, policy_version 346446 (0.0006) [2023-12-26 17:53:00,817][105620] Updated weights for policy 1, policy_version 346456 (0.0007) [2023-12-26 17:53:00,878][105620] Updated weights for policy 1, policy_version 346466 (0.0008) [2023-12-26 17:53:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 177324032. Throughput: 0: 9826.3, 1: 9861.1. Samples: 177292232. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 17:53:01,062][104569] Avg episode reward: [(0, '8812.907'), (1, '9266.115')] [2023-12-26 17:53:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000346120_88621056.pth... [2023-12-26 17:53:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000346472_88702976.pth... [2023-12-26 17:53:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000345000_88334336.pth [2023-12-26 17:53:01,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000345320_88408064.pth [2023-12-26 17:53:01,362][105692] Updated weights for policy 0, policy_version 346121 (0.0010) [2023-12-26 17:53:01,425][105692] Updated weights for policy 0, policy_version 346131 (0.0009) [2023-12-26 17:53:01,484][105692] Updated weights for policy 0, policy_version 346141 (0.0009) [2023-12-26 17:53:01,542][105692] Updated weights for policy 0, policy_version 346151 (0.0009) [2023-12-26 17:53:01,647][105620] Updated weights for policy 1, policy_version 346476 (0.0008) [2023-12-26 17:53:01,701][105620] Updated weights for policy 1, policy_version 346486 (0.0009) [2023-12-26 17:53:01,757][105620] Updated weights for policy 1, policy_version 346496 (0.0010) [2023-12-26 17:53:02,342][105692] Updated weights for policy 0, policy_version 346161 (0.0009) [2023-12-26 17:53:02,408][105692] Updated weights for policy 0, policy_version 346171 (0.0010) [2023-12-26 17:53:02,467][105692] Updated weights for policy 0, policy_version 346181 (0.0009) [2023-12-26 17:53:02,471][105620] Updated weights for policy 1, policy_version 346506 (0.0009) [2023-12-26 17:53:02,518][105620] Updated weights for policy 1, policy_version 346516 (0.0008) [2023-12-26 17:53:02,565][105620] Updated weights for policy 1, policy_version 346526 (0.0009) [2023-12-26 17:53:02,618][105620] Updated weights for policy 1, policy_version 346536 (0.0009) [2023-12-26 17:53:03,239][105692] Updated weights for policy 0, policy_version 346191 (0.0006) [2023-12-26 17:53:03,245][105620] Updated weights for policy 1, policy_version 346546 (0.0009) [2023-12-26 17:53:03,289][105692] Updated weights for policy 0, policy_version 346201 (0.0006) [2023-12-26 17:53:03,298][105620] Updated weights for policy 1, policy_version 346556 (0.0009) [2023-12-26 17:53:03,340][105692] Updated weights for policy 0, policy_version 346211 (0.0006) [2023-12-26 17:53:03,354][105620] Updated weights for policy 1, policy_version 346566 (0.0009) [2023-12-26 17:53:04,077][105620] Updated weights for policy 1, policy_version 346576 (0.0009) [2023-12-26 17:53:04,102][105692] Updated weights for policy 0, policy_version 346221 (0.0008) [2023-12-26 17:53:04,136][105620] Updated weights for policy 1, policy_version 346586 (0.0008) [2023-12-26 17:53:04,167][105692] Updated weights for policy 0, policy_version 346231 (0.0008) [2023-12-26 17:53:04,186][105620] Updated weights for policy 1, policy_version 346596 (0.0006) [2023-12-26 17:53:04,227][105692] Updated weights for policy 0, policy_version 346241 (0.0009) [2023-12-26 17:53:04,936][105620] Updated weights for policy 1, policy_version 346606 (0.0008) [2023-12-26 17:53:04,983][105692] Updated weights for policy 0, policy_version 346251 (0.0009) [2023-12-26 17:53:04,989][105620] Updated weights for policy 1, policy_version 346616 (0.0008) [2023-12-26 17:53:05,040][105692] Updated weights for policy 0, policy_version 346261 (0.0006) [2023-12-26 17:53:05,058][105620] Updated weights for policy 1, policy_version 346626 (0.0009) [2023-12-26 17:53:05,100][105692] Updated weights for policy 0, policy_version 346271 (0.0008) [2023-12-26 17:53:05,786][105620] Updated weights for policy 1, policy_version 346636 (0.0007) [2023-12-26 17:53:05,833][105692] Updated weights for policy 0, policy_version 346281 (0.0009) [2023-12-26 17:53:05,850][105620] Updated weights for policy 1, policy_version 346646 (0.0008) [2023-12-26 17:53:05,879][105692] Updated weights for policy 0, policy_version 346291 (0.0005) [2023-12-26 17:53:05,897][105620] Updated weights for policy 1, policy_version 346656 (0.0008) [2023-12-26 17:53:05,930][105692] Updated weights for policy 0, policy_version 346301 (0.0005) [2023-12-26 17:53:05,975][105692] Updated weights for policy 0, policy_version 346311 (0.0005) [2023-12-26 17:53:06,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 177422336. Throughput: 0: 9619.9, 1: 9920.8. Samples: 177406048. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 17:53:06,063][104569] Avg episode reward: [(0, '9177.582'), (1, '9356.718')] [2023-12-26 17:53:06,629][105620] Updated weights for policy 1, policy_version 346666 (0.0008) [2023-12-26 17:53:06,695][105620] Updated weights for policy 1, policy_version 346676 (0.0009) [2023-12-26 17:53:06,744][105692] Updated weights for policy 0, policy_version 346321 (0.0007) [2023-12-26 17:53:06,750][105620] Updated weights for policy 1, policy_version 346686 (0.0011) [2023-12-26 17:53:06,800][105692] Updated weights for policy 0, policy_version 346331 (0.0006) [2023-12-26 17:53:06,808][105620] Updated weights for policy 1, policy_version 346696 (0.0011) [2023-12-26 17:53:06,852][105692] Updated weights for policy 0, policy_version 346341 (0.0008) [2023-12-26 17:53:07,561][105620] Updated weights for policy 1, policy_version 346706 (0.0010) [2023-12-26 17:53:07,609][105620] Updated weights for policy 1, policy_version 346716 (0.0010) [2023-12-26 17:53:07,631][105692] Updated weights for policy 0, policy_version 346351 (0.0006) [2023-12-26 17:53:07,654][105620] Updated weights for policy 1, policy_version 346726 (0.0010) [2023-12-26 17:53:07,679][105692] Updated weights for policy 0, policy_version 346361 (0.0006) [2023-12-26 17:53:07,729][105692] Updated weights for policy 0, policy_version 346371 (0.0008) [2023-12-26 17:53:08,318][105620] Updated weights for policy 1, policy_version 346736 (0.0009) [2023-12-26 17:53:08,387][105620] Updated weights for policy 1, policy_version 346746 (0.0007) [2023-12-26 17:53:08,453][105620] Updated weights for policy 1, policy_version 346756 (0.0010) [2023-12-26 17:53:08,542][105692] Updated weights for policy 0, policy_version 346382 (0.0007) [2023-12-26 17:53:08,591][105692] Updated weights for policy 0, policy_version 346392 (0.0005) [2023-12-26 17:53:08,649][105692] Updated weights for policy 0, policy_version 346402 (0.0005) [2023-12-26 17:53:09,149][105620] Updated weights for policy 1, policy_version 346766 (0.0009) [2023-12-26 17:53:09,205][105620] Updated weights for policy 1, policy_version 346776 (0.0006) [2023-12-26 17:53:09,270][105620] Updated weights for policy 1, policy_version 346786 (0.0008) [2023-12-26 17:53:09,387][105692] Updated weights for policy 0, policy_version 346412 (0.0009) [2023-12-26 17:53:09,455][105692] Updated weights for policy 0, policy_version 346422 (0.0008) [2023-12-26 17:53:09,522][105692] Updated weights for policy 0, policy_version 346432 (0.0009) [2023-12-26 17:53:10,160][105620] Updated weights for policy 1, policy_version 346796 (0.0009) [2023-12-26 17:53:10,177][105692] Updated weights for policy 0, policy_version 346442 (0.0007) [2023-12-26 17:53:10,215][105620] Updated weights for policy 1, policy_version 346806 (0.0009) [2023-12-26 17:53:10,243][105692] Updated weights for policy 0, policy_version 346452 (0.0008) [2023-12-26 17:53:10,271][105620] Updated weights for policy 1, policy_version 346816 (0.0009) [2023-12-26 17:53:10,296][105692] Updated weights for policy 0, policy_version 346462 (0.0007) [2023-12-26 17:53:10,350][105692] Updated weights for policy 0, policy_version 346472 (0.0006) [2023-12-26 17:53:11,011][105620] Updated weights for policy 1, policy_version 346826 (0.0009) [2023-12-26 17:53:11,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 177504256. Throughput: 0: 9540.1, 1: 9894.8. Samples: 177519108. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 17:53:11,063][104569] Avg episode reward: [(0, '9266.822'), (1, '9271.430')] [2023-12-26 17:53:11,077][105620] Updated weights for policy 1, policy_version 346836 (0.0007) [2023-12-26 17:53:11,081][105692] Updated weights for policy 0, policy_version 346482 (0.0008) [2023-12-26 17:53:11,147][105620] Updated weights for policy 1, policy_version 346846 (0.0007) [2023-12-26 17:53:11,149][105692] Updated weights for policy 0, policy_version 346492 (0.0007) [2023-12-26 17:53:11,208][105692] Updated weights for policy 0, policy_version 346502 (0.0008) [2023-12-26 17:53:11,213][105620] Updated weights for policy 1, policy_version 346856 (0.0006) [2023-12-26 17:53:11,915][105620] Updated weights for policy 1, policy_version 346866 (0.0008) [2023-12-26 17:53:11,988][105620] Updated weights for policy 1, policy_version 346876 (0.0009) [2023-12-26 17:53:12,043][105692] Updated weights for policy 0, policy_version 346512 (0.0009) [2023-12-26 17:53:12,046][105620] Updated weights for policy 1, policy_version 346886 (0.0007) [2023-12-26 17:53:12,105][105692] Updated weights for policy 0, policy_version 346522 (0.0007) [2023-12-26 17:53:12,167][105692] Updated weights for policy 0, policy_version 346532 (0.0010) [2023-12-26 17:53:12,761][105620] Updated weights for policy 1, policy_version 346896 (0.0006) [2023-12-26 17:53:12,826][105620] Updated weights for policy 1, policy_version 346906 (0.0007) [2023-12-26 17:53:12,836][105692] Updated weights for policy 0, policy_version 346542 (0.0009) [2023-12-26 17:53:12,884][105620] Updated weights for policy 1, policy_version 346916 (0.0007) [2023-12-26 17:53:12,902][105692] Updated weights for policy 0, policy_version 346552 (0.0008) [2023-12-26 17:53:12,966][105692] Updated weights for policy 0, policy_version 346562 (0.0008) [2023-12-26 17:53:13,453][105620] Updated weights for policy 1, policy_version 346926 (0.0007) [2023-12-26 17:53:13,499][105620] Updated weights for policy 1, policy_version 346936 (0.0005) [2023-12-26 17:53:13,554][105620] Updated weights for policy 1, policy_version 346946 (0.0005) [2023-12-26 17:53:13,810][105692] Updated weights for policy 0, policy_version 346572 (0.0008) [2023-12-26 17:53:13,879][105692] Updated weights for policy 0, policy_version 346582 (0.0010) [2023-12-26 17:53:13,942][105692] Updated weights for policy 0, policy_version 346592 (0.0009) [2023-12-26 17:53:14,137][105620] Updated weights for policy 1, policy_version 346956 (0.0007) [2023-12-26 17:53:14,188][105620] Updated weights for policy 1, policy_version 346966 (0.0010) [2023-12-26 17:53:14,238][105620] Updated weights for policy 1, policy_version 346976 (0.0007) [2023-12-26 17:53:14,740][105692] Updated weights for policy 0, policy_version 346602 (0.0009) [2023-12-26 17:53:14,811][105692] Updated weights for policy 0, policy_version 346612 (0.0009) [2023-12-26 17:53:14,881][105692] Updated weights for policy 0, policy_version 346622 (0.0007) [2023-12-26 17:53:14,934][105620] Updated weights for policy 1, policy_version 346986 (0.0007) [2023-12-26 17:53:14,945][105692] Updated weights for policy 0, policy_version 346632 (0.0007) [2023-12-26 17:53:14,994][105620] Updated weights for policy 1, policy_version 346996 (0.0009) [2023-12-26 17:53:15,061][105620] Updated weights for policy 1, policy_version 347006 (0.0008) [2023-12-26 17:53:15,122][105620] Updated weights for policy 1, policy_version 347016 (0.0009) [2023-12-26 17:53:15,631][105692] Updated weights for policy 0, policy_version 346642 (0.0010) [2023-12-26 17:53:15,684][105692] Updated weights for policy 0, policy_version 346652 (0.0010) [2023-12-26 17:53:15,734][105692] Updated weights for policy 0, policy_version 346662 (0.0006) [2023-12-26 17:53:15,743][105620] Updated weights for policy 1, policy_version 347026 (0.0009) [2023-12-26 17:53:15,800][105620] Updated weights for policy 1, policy_version 347036 (0.0008) [2023-12-26 17:53:15,856][105620] Updated weights for policy 1, policy_version 347046 (0.0009) [2023-12-26 17:53:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 177610752. Throughput: 0: 9496.5, 1: 9817.0. Samples: 177577280. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 17:53:16,063][104569] Avg episode reward: [(0, '9266.911'), (1, '9271.416')] [2023-12-26 17:53:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000346664_88760320.pth... [2023-12-26 17:53:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000347048_88850432.pth... [2023-12-26 17:53:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000345576_88481792.pth [2023-12-26 17:53:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000345896_88555520.pth [2023-12-26 17:53:16,527][105692] Updated weights for policy 0, policy_version 346672 (0.0008) [2023-12-26 17:53:16,550][105620] Updated weights for policy 1, policy_version 347056 (0.0007) [2023-12-26 17:53:16,577][105692] Updated weights for policy 0, policy_version 346682 (0.0006) [2023-12-26 17:53:16,599][105620] Updated weights for policy 1, policy_version 347066 (0.0006) [2023-12-26 17:53:16,630][105692] Updated weights for policy 0, policy_version 346692 (0.0008) [2023-12-26 17:53:16,644][105620] Updated weights for policy 1, policy_version 347076 (0.0006) [2023-12-26 17:53:17,305][105620] Updated weights for policy 1, policy_version 347086 (0.0005) [2023-12-26 17:53:17,366][105692] Updated weights for policy 0, policy_version 346702 (0.0007) [2023-12-26 17:53:17,372][105620] Updated weights for policy 1, policy_version 347096 (0.0007) [2023-12-26 17:53:17,431][105692] Updated weights for policy 0, policy_version 346712 (0.0005) [2023-12-26 17:53:17,438][105620] Updated weights for policy 1, policy_version 347106 (0.0007) [2023-12-26 17:53:17,503][105692] Updated weights for policy 0, policy_version 346722 (0.0007) [2023-12-26 17:53:18,031][105620] Updated weights for policy 1, policy_version 347116 (0.0007) [2023-12-26 17:53:18,082][105620] Updated weights for policy 1, policy_version 347126 (0.0009) [2023-12-26 17:53:18,136][105620] Updated weights for policy 1, policy_version 347136 (0.0010) [2023-12-26 17:53:18,216][105692] Updated weights for policy 0, policy_version 346732 (0.0008) [2023-12-26 17:53:18,271][105692] Updated weights for policy 0, policy_version 346742 (0.0008) [2023-12-26 17:53:18,315][105692] Updated weights for policy 0, policy_version 346752 (0.0008) [2023-12-26 17:53:18,859][105620] Updated weights for policy 1, policy_version 347146 (0.0010) [2023-12-26 17:53:18,907][105620] Updated weights for policy 1, policy_version 347156 (0.0010) [2023-12-26 17:53:18,957][105620] Updated weights for policy 1, policy_version 347166 (0.0010) [2023-12-26 17:53:19,019][105620] Updated weights for policy 1, policy_version 347176 (0.0010) [2023-12-26 17:53:19,101][105692] Updated weights for policy 0, policy_version 346762 (0.0008) [2023-12-26 17:53:19,159][105692] Updated weights for policy 0, policy_version 346772 (0.0007) [2023-12-26 17:53:19,221][105692] Updated weights for policy 0, policy_version 346782 (0.0008) [2023-12-26 17:53:19,285][105692] Updated weights for policy 0, policy_version 346792 (0.0008) [2023-12-26 17:53:19,767][105620] Updated weights for policy 1, policy_version 347186 (0.0009) [2023-12-26 17:53:19,830][105620] Updated weights for policy 1, policy_version 347196 (0.0009) [2023-12-26 17:53:19,889][105620] Updated weights for policy 1, policy_version 347206 (0.0008) [2023-12-26 17:53:20,072][105692] Updated weights for policy 0, policy_version 346802 (0.0009) [2023-12-26 17:53:20,128][105692] Updated weights for policy 0, policy_version 346812 (0.0009) [2023-12-26 17:53:20,188][105692] Updated weights for policy 0, policy_version 346822 (0.0009) [2023-12-26 17:53:20,631][105620] Updated weights for policy 1, policy_version 347216 (0.0007) [2023-12-26 17:53:20,690][105620] Updated weights for policy 1, policy_version 347226 (0.0006) [2023-12-26 17:53:20,755][105620] Updated weights for policy 1, policy_version 347236 (0.0006) [2023-12-26 17:53:21,019][105692] Updated weights for policy 0, policy_version 346832 (0.0009) [2023-12-26 17:53:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 177700864. Throughput: 0: 9426.7, 1: 9844.5. Samples: 177693776. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 17:53:21,063][104569] Avg episode reward: [(0, '9268.099'), (1, '9287.795')] [2023-12-26 17:53:21,082][105692] Updated weights for policy 0, policy_version 346842 (0.0009) [2023-12-26 17:53:21,145][105692] Updated weights for policy 0, policy_version 346852 (0.0009) [2023-12-26 17:53:21,460][105620] Updated weights for policy 1, policy_version 347246 (0.0009) [2023-12-26 17:53:21,518][105620] Updated weights for policy 1, policy_version 347256 (0.0008) [2023-12-26 17:53:21,574][105620] Updated weights for policy 1, policy_version 347266 (0.0011) [2023-12-26 17:53:21,863][105692] Updated weights for policy 0, policy_version 346862 (0.0008) [2023-12-26 17:53:21,930][105692] Updated weights for policy 0, policy_version 346872 (0.0006) [2023-12-26 17:53:21,999][105692] Updated weights for policy 0, policy_version 346882 (0.0006) [2023-12-26 17:53:22,364][105620] Updated weights for policy 1, policy_version 347276 (0.0010) [2023-12-26 17:53:22,421][105620] Updated weights for policy 1, policy_version 347286 (0.0008) [2023-12-26 17:53:22,481][105620] Updated weights for policy 1, policy_version 347296 (0.0009) [2023-12-26 17:53:22,675][105692] Updated weights for policy 0, policy_version 346892 (0.0006) [2023-12-26 17:53:22,738][105692] Updated weights for policy 0, policy_version 346902 (0.0006) [2023-12-26 17:53:22,797][105692] Updated weights for policy 0, policy_version 346912 (0.0006) [2023-12-26 17:53:23,265][105620] Updated weights for policy 1, policy_version 347306 (0.0008) [2023-12-26 17:53:23,317][105620] Updated weights for policy 1, policy_version 347316 (0.0008) [2023-12-26 17:53:23,355][105692] Updated weights for policy 0, policy_version 346922 (0.0009) [2023-12-26 17:53:23,374][105620] Updated weights for policy 1, policy_version 347326 (0.0007) [2023-12-26 17:53:23,399][105692] Updated weights for policy 0, policy_version 346932 (0.0010) [2023-12-26 17:53:23,421][105620] Updated weights for policy 1, policy_version 347336 (0.0005) [2023-12-26 17:53:23,447][105692] Updated weights for policy 0, policy_version 346942 (0.0010) [2023-12-26 17:53:23,495][105692] Updated weights for policy 0, policy_version 346952 (0.0010) [2023-12-26 17:53:24,072][105620] Updated weights for policy 1, policy_version 347346 (0.0005) [2023-12-26 17:53:24,124][105620] Updated weights for policy 1, policy_version 347356 (0.0005) [2023-12-26 17:53:24,139][105692] Updated weights for policy 0, policy_version 346962 (0.0006) [2023-12-26 17:53:24,184][105620] Updated weights for policy 1, policy_version 347366 (0.0006) [2023-12-26 17:53:24,201][105692] Updated weights for policy 0, policy_version 346972 (0.0009) [2023-12-26 17:53:24,258][105692] Updated weights for policy 0, policy_version 346982 (0.0010) [2023-12-26 17:53:24,877][105692] Updated weights for policy 0, policy_version 346992 (0.0006) [2023-12-26 17:53:24,898][105620] Updated weights for policy 1, policy_version 347376 (0.0008) [2023-12-26 17:53:24,946][105692] Updated weights for policy 0, policy_version 347002 (0.0006) [2023-12-26 17:53:24,960][105620] Updated weights for policy 1, policy_version 347386 (0.0009) [2023-12-26 17:53:25,003][105692] Updated weights for policy 0, policy_version 347012 (0.0009) [2023-12-26 17:53:25,018][105620] Updated weights for policy 1, policy_version 347396 (0.0007) [2023-12-26 17:53:25,576][105692] Updated weights for policy 0, policy_version 347022 (0.0008) [2023-12-26 17:53:25,640][105692] Updated weights for policy 0, policy_version 347032 (0.0011) [2023-12-26 17:53:25,700][105692] Updated weights for policy 0, policy_version 347042 (0.0010) [2023-12-26 17:53:25,788][105620] Updated weights for policy 1, policy_version 347406 (0.0008) [2023-12-26 17:53:25,848][105620] Updated weights for policy 1, policy_version 347416 (0.0008) [2023-12-26 17:53:25,901][105620] Updated weights for policy 1, policy_version 347426 (0.0008) [2023-12-26 17:53:26,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 177807360. Throughput: 0: 9485.9, 1: 9798.1. Samples: 177812444. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 17:53:26,062][104569] Avg episode reward: [(0, '8996.363'), (1, '9266.263')] [2023-12-26 17:53:26,324][105692] Updated weights for policy 0, policy_version 347052 (0.0005) [2023-12-26 17:53:26,386][105692] Updated weights for policy 0, policy_version 347062 (0.0005) [2023-12-26 17:53:26,449][105692] Updated weights for policy 0, policy_version 347072 (0.0005) [2023-12-26 17:53:26,747][105620] Updated weights for policy 1, policy_version 347436 (0.0008) [2023-12-26 17:53:26,801][105620] Updated weights for policy 1, policy_version 347446 (0.0008) [2023-12-26 17:53:26,853][105620] Updated weights for policy 1, policy_version 347456 (0.0008) [2023-12-26 17:53:27,007][105692] Updated weights for policy 0, policy_version 347082 (0.0007) [2023-12-26 17:53:27,061][105692] Updated weights for policy 0, policy_version 347092 (0.0010) [2023-12-26 17:53:27,115][105692] Updated weights for policy 0, policy_version 347102 (0.0010) [2023-12-26 17:53:27,173][105692] Updated weights for policy 0, policy_version 347112 (0.0008) [2023-12-26 17:53:27,575][105620] Updated weights for policy 1, policy_version 347466 (0.0007) [2023-12-26 17:53:27,621][105620] Updated weights for policy 1, policy_version 347476 (0.0005) [2023-12-26 17:53:27,671][105620] Updated weights for policy 1, policy_version 347486 (0.0005) [2023-12-26 17:53:27,719][105620] Updated weights for policy 1, policy_version 347496 (0.0005) [2023-12-26 17:53:27,859][105692] Updated weights for policy 0, policy_version 347122 (0.0007) [2023-12-26 17:53:27,923][105692] Updated weights for policy 0, policy_version 347132 (0.0007) [2023-12-26 17:53:27,982][105692] Updated weights for policy 0, policy_version 347142 (0.0009) [2023-12-26 17:53:28,386][105620] Updated weights for policy 1, policy_version 347506 (0.0009) [2023-12-26 17:53:28,441][105620] Updated weights for policy 1, policy_version 347516 (0.0009) [2023-12-26 17:53:28,492][105620] Updated weights for policy 1, policy_version 347526 (0.0009) [2023-12-26 17:53:28,685][105692] Updated weights for policy 0, policy_version 347152 (0.0008) [2023-12-26 17:53:28,745][105692] Updated weights for policy 0, policy_version 347162 (0.0007) [2023-12-26 17:53:28,813][105692] Updated weights for policy 0, policy_version 347172 (0.0005) [2023-12-26 17:53:29,280][105620] Updated weights for policy 1, policy_version 347536 (0.0009) [2023-12-26 17:53:29,334][105620] Updated weights for policy 1, policy_version 347546 (0.0011) [2023-12-26 17:53:29,394][105620] Updated weights for policy 1, policy_version 347556 (0.0011) [2023-12-26 17:53:29,497][105692] Updated weights for policy 0, policy_version 347182 (0.0007) [2023-12-26 17:53:29,553][105692] Updated weights for policy 0, policy_version 347192 (0.0008) [2023-12-26 17:53:29,612][105692] Updated weights for policy 0, policy_version 347202 (0.0010) [2023-12-26 17:53:30,005][105620] Updated weights for policy 1, policy_version 347566 (0.0007) [2023-12-26 17:53:30,058][105620] Updated weights for policy 1, policy_version 347576 (0.0007) [2023-12-26 17:53:30,103][105620] Updated weights for policy 1, policy_version 347586 (0.0010) [2023-12-26 17:53:30,403][105692] Updated weights for policy 0, policy_version 347212 (0.0009) [2023-12-26 17:53:30,456][105692] Updated weights for policy 0, policy_version 347222 (0.0006) [2023-12-26 17:53:30,521][105692] Updated weights for policy 0, policy_version 347232 (0.0005) [2023-12-26 17:53:30,846][105620] Updated weights for policy 1, policy_version 347596 (0.0010) [2023-12-26 17:53:30,897][105620] Updated weights for policy 1, policy_version 347606 (0.0010) [2023-12-26 17:53:30,951][105620] Updated weights for policy 1, policy_version 347616 (0.0010) [2023-12-26 17:53:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 177905664. Throughput: 0: 9550.2, 1: 9816.6. Samples: 177872556. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 17:53:31,063][104569] Avg episode reward: [(0, '8904.446'), (1, '9178.309')] [2023-12-26 17:53:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000347240_88907776.pth... [2023-12-26 17:53:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000347624_88997888.pth... [2023-12-26 17:53:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000346120_88621056.pth [2023-12-26 17:53:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000346472_88702976.pth [2023-12-26 17:53:31,245][105692] Updated weights for policy 0, policy_version 347242 (0.0008) [2023-12-26 17:53:31,310][105692] Updated weights for policy 0, policy_version 347252 (0.0010) [2023-12-26 17:53:31,364][105692] Updated weights for policy 0, policy_version 347262 (0.0011) [2023-12-26 17:53:31,431][105692] Updated weights for policy 0, policy_version 347272 (0.0011) [2023-12-26 17:53:31,726][105620] Updated weights for policy 1, policy_version 347626 (0.0010) [2023-12-26 17:53:31,783][105620] Updated weights for policy 1, policy_version 347636 (0.0008) [2023-12-26 17:53:31,838][105620] Updated weights for policy 1, policy_version 347646 (0.0008) [2023-12-26 17:53:31,892][105620] Updated weights for policy 1, policy_version 347656 (0.0010) [2023-12-26 17:53:32,127][105692] Updated weights for policy 0, policy_version 347282 (0.0008) [2023-12-26 17:53:32,186][105692] Updated weights for policy 0, policy_version 347292 (0.0006) [2023-12-26 17:53:32,255][105692] Updated weights for policy 0, policy_version 347302 (0.0006) [2023-12-26 17:53:32,659][105620] Updated weights for policy 1, policy_version 347666 (0.0010) [2023-12-26 17:53:32,704][105620] Updated weights for policy 1, policy_version 347676 (0.0010) [2023-12-26 17:53:32,749][105620] Updated weights for policy 1, policy_version 347686 (0.0009) [2023-12-26 17:53:32,852][105692] Updated weights for policy 0, policy_version 347312 (0.0010) [2023-12-26 17:53:32,897][105692] Updated weights for policy 0, policy_version 347322 (0.0010) [2023-12-26 17:53:32,945][105692] Updated weights for policy 0, policy_version 347332 (0.0010) [2023-12-26 17:53:33,411][105620] Updated weights for policy 1, policy_version 347696 (0.0005) [2023-12-26 17:53:33,466][105620] Updated weights for policy 1, policy_version 347706 (0.0005) [2023-12-26 17:53:33,521][105620] Updated weights for policy 1, policy_version 347716 (0.0005) [2023-12-26 17:53:33,621][105692] Updated weights for policy 0, policy_version 347342 (0.0007) [2023-12-26 17:53:33,669][105692] Updated weights for policy 0, policy_version 347352 (0.0005) [2023-12-26 17:53:33,712][105692] Updated weights for policy 0, policy_version 347362 (0.0005) [2023-12-26 17:53:34,126][105620] Updated weights for policy 1, policy_version 347726 (0.0005) [2023-12-26 17:53:34,202][105620] Updated weights for policy 1, policy_version 347736 (0.0006) [2023-12-26 17:53:34,268][105620] Updated weights for policy 1, policy_version 347746 (0.0005) [2023-12-26 17:53:34,387][105692] Updated weights for policy 0, policy_version 347372 (0.0010) [2023-12-26 17:53:34,453][105692] Updated weights for policy 0, policy_version 347382 (0.0011) [2023-12-26 17:53:34,506][105692] Updated weights for policy 0, policy_version 347392 (0.0009) [2023-12-26 17:53:34,874][105620] Updated weights for policy 1, policy_version 347756 (0.0007) [2023-12-26 17:53:34,936][105620] Updated weights for policy 1, policy_version 347766 (0.0010) [2023-12-26 17:53:34,987][105620] Updated weights for policy 1, policy_version 347776 (0.0010) [2023-12-26 17:53:35,259][105692] Updated weights for policy 0, policy_version 347402 (0.0008) [2023-12-26 17:53:35,317][105692] Updated weights for policy 0, policy_version 347412 (0.0008) [2023-12-26 17:53:35,384][105692] Updated weights for policy 0, policy_version 347422 (0.0009) [2023-12-26 17:53:35,436][105692] Updated weights for policy 0, policy_version 347432 (0.0008) [2023-12-26 17:53:35,735][105620] Updated weights for policy 1, policy_version 347786 (0.0010) [2023-12-26 17:53:35,786][105620] Updated weights for policy 1, policy_version 347796 (0.0010) [2023-12-26 17:53:35,837][105620] Updated weights for policy 1, policy_version 347806 (0.0010) [2023-12-26 17:53:35,899][105620] Updated weights for policy 1, policy_version 347816 (0.0010) [2023-12-26 17:53:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 178003968. Throughput: 0: 9581.3, 1: 9780.1. Samples: 177993116. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 17:53:36,063][104569] Avg episode reward: [(0, '8816.616'), (1, '9098.021')] [2023-12-26 17:53:36,207][105692] Updated weights for policy 0, policy_version 347442 (0.0008) [2023-12-26 17:53:36,267][105692] Updated weights for policy 0, policy_version 347452 (0.0008) [2023-12-26 17:53:36,323][105692] Updated weights for policy 0, policy_version 347462 (0.0008) [2023-12-26 17:53:36,679][105620] Updated weights for policy 1, policy_version 347826 (0.0011) [2023-12-26 17:53:36,741][105620] Updated weights for policy 1, policy_version 347836 (0.0010) [2023-12-26 17:53:36,796][105620] Updated weights for policy 1, policy_version 347846 (0.0010) [2023-12-26 17:53:37,094][105692] Updated weights for policy 0, policy_version 347472 (0.0008) [2023-12-26 17:53:37,144][105692] Updated weights for policy 0, policy_version 347482 (0.0008) [2023-12-26 17:53:37,200][105692] Updated weights for policy 0, policy_version 347492 (0.0008) [2023-12-26 17:53:37,549][105620] Updated weights for policy 1, policy_version 347856 (0.0011) [2023-12-26 17:53:37,609][105620] Updated weights for policy 1, policy_version 347866 (0.0010) [2023-12-26 17:53:37,664][105620] Updated weights for policy 1, policy_version 347876 (0.0010) [2023-12-26 17:53:37,994][105692] Updated weights for policy 0, policy_version 347502 (0.0008) [2023-12-26 17:53:38,058][105692] Updated weights for policy 0, policy_version 347512 (0.0008) [2023-12-26 17:53:38,116][105692] Updated weights for policy 0, policy_version 347522 (0.0008) [2023-12-26 17:53:38,434][105620] Updated weights for policy 1, policy_version 347886 (0.0010) [2023-12-26 17:53:38,496][105620] Updated weights for policy 1, policy_version 347896 (0.0010) [2023-12-26 17:53:38,558][105620] Updated weights for policy 1, policy_version 347906 (0.0010) [2023-12-26 17:53:38,872][105692] Updated weights for policy 0, policy_version 347532 (0.0009) [2023-12-26 17:53:38,940][105692] Updated weights for policy 0, policy_version 347542 (0.0010) [2023-12-26 17:53:39,002][105692] Updated weights for policy 0, policy_version 347552 (0.0009) [2023-12-26 17:53:39,298][105620] Updated weights for policy 1, policy_version 347916 (0.0010) [2023-12-26 17:53:39,370][105620] Updated weights for policy 1, policy_version 347926 (0.0008) [2023-12-26 17:53:39,433][105620] Updated weights for policy 1, policy_version 347936 (0.0010) [2023-12-26 17:53:39,781][105692] Updated weights for policy 0, policy_version 347562 (0.0008) [2023-12-26 17:53:39,837][105692] Updated weights for policy 0, policy_version 347572 (0.0008) [2023-12-26 17:53:39,898][105692] Updated weights for policy 0, policy_version 347582 (0.0008) [2023-12-26 17:53:39,960][105692] Updated weights for policy 0, policy_version 347592 (0.0008) [2023-12-26 17:53:40,178][105620] Updated weights for policy 1, policy_version 347946 (0.0010) [2023-12-26 17:53:40,229][105620] Updated weights for policy 1, policy_version 347956 (0.0009) [2023-12-26 17:53:40,283][105620] Updated weights for policy 1, policy_version 347966 (0.0009) [2023-12-26 17:53:40,354][105620] Updated weights for policy 1, policy_version 347976 (0.0009) [2023-12-26 17:53:40,723][105692] Updated weights for policy 0, policy_version 347602 (0.0009) [2023-12-26 17:53:40,786][105692] Updated weights for policy 0, policy_version 347612 (0.0009) [2023-12-26 17:53:40,848][105692] Updated weights for policy 0, policy_version 347622 (0.0010) [2023-12-26 17:53:41,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 178094080. Throughput: 0: 9524.7, 1: 9737.7. Samples: 178103644. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 17:53:41,062][104569] Avg episode reward: [(0, '8726.190'), (1, '9185.968')] [2023-12-26 17:53:41,073][105620] Updated weights for policy 1, policy_version 347986 (0.0009) [2023-12-26 17:53:41,140][105620] Updated weights for policy 1, policy_version 347996 (0.0009) [2023-12-26 17:53:41,203][105620] Updated weights for policy 1, policy_version 348006 (0.0009) [2023-12-26 17:53:41,719][105692] Updated weights for policy 0, policy_version 347632 (0.0009) [2023-12-26 17:53:41,783][105692] Updated weights for policy 0, policy_version 347642 (0.0006) [2023-12-26 17:53:41,844][105692] Updated weights for policy 0, policy_version 347652 (0.0008) [2023-12-26 17:53:42,005][105620] Updated weights for policy 1, policy_version 348016 (0.0009) [2023-12-26 17:53:42,068][105620] Updated weights for policy 1, policy_version 348026 (0.0009) [2023-12-26 17:53:42,116][105620] Updated weights for policy 1, policy_version 348036 (0.0009) [2023-12-26 17:53:42,587][105692] Updated weights for policy 0, policy_version 347662 (0.0010) [2023-12-26 17:53:42,646][105692] Updated weights for policy 0, policy_version 347672 (0.0010) [2023-12-26 17:53:42,701][105692] Updated weights for policy 0, policy_version 347682 (0.0010) [2023-12-26 17:53:42,824][105620] Updated weights for policy 1, policy_version 348046 (0.0009) [2023-12-26 17:53:42,886][105620] Updated weights for policy 1, policy_version 348056 (0.0009) [2023-12-26 17:53:42,938][105620] Updated weights for policy 1, policy_version 348066 (0.0009) [2023-12-26 17:53:43,472][105692] Updated weights for policy 0, policy_version 347692 (0.0010) [2023-12-26 17:53:43,533][105692] Updated weights for policy 0, policy_version 347702 (0.0008) [2023-12-26 17:53:43,585][105692] Updated weights for policy 0, policy_version 347712 (0.0009) [2023-12-26 17:53:43,734][105620] Updated weights for policy 1, policy_version 348076 (0.0009) [2023-12-26 17:53:43,797][105620] Updated weights for policy 1, policy_version 348086 (0.0010) [2023-12-26 17:53:43,859][105620] Updated weights for policy 1, policy_version 348096 (0.0010) [2023-12-26 17:53:44,163][105692] Updated weights for policy 0, policy_version 347722 (0.0007) [2023-12-26 17:53:44,215][105692] Updated weights for policy 0, policy_version 347732 (0.0005) [2023-12-26 17:53:44,264][105692] Updated weights for policy 0, policy_version 347742 (0.0005) [2023-12-26 17:53:44,313][105692] Updated weights for policy 0, policy_version 347752 (0.0008) [2023-12-26 17:53:44,534][105620] Updated weights for policy 1, policy_version 348106 (0.0009) [2023-12-26 17:53:44,585][105620] Updated weights for policy 1, policy_version 348116 (0.0009) [2023-12-26 17:53:44,631][105620] Updated weights for policy 1, policy_version 348126 (0.0007) [2023-12-26 17:53:44,679][105620] Updated weights for policy 1, policy_version 348136 (0.0009) [2023-12-26 17:53:45,069][105692] Updated weights for policy 0, policy_version 347762 (0.0011) [2023-12-26 17:53:45,130][105692] Updated weights for policy 0, policy_version 347772 (0.0011) [2023-12-26 17:53:45,192][105692] Updated weights for policy 0, policy_version 347782 (0.0006) [2023-12-26 17:53:45,462][105620] Updated weights for policy 1, policy_version 348146 (0.0007) [2023-12-26 17:53:45,528][105620] Updated weights for policy 1, policy_version 348156 (0.0011) [2023-12-26 17:53:45,584][105620] Updated weights for policy 1, policy_version 348166 (0.0011) [2023-12-26 17:53:45,859][105692] Updated weights for policy 0, policy_version 347792 (0.0005) [2023-12-26 17:53:45,909][105692] Updated weights for policy 0, policy_version 347802 (0.0005) [2023-12-26 17:53:45,980][105692] Updated weights for policy 0, policy_version 347812 (0.0005) [2023-12-26 17:53:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 178192384. Throughput: 0: 9504.9, 1: 9718.6. Samples: 178157292. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 17:53:46,063][104569] Avg episode reward: [(0, '8548.285'), (1, '9265.966')] [2023-12-26 17:53:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000347816_89055232.pth... [2023-12-26 17:53:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000348168_89137152.pth... [2023-12-26 17:53:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000347048_88850432.pth [2023-12-26 17:53:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000346664_88760320.pth [2023-12-26 17:53:46,193][105620] Updated weights for policy 1, policy_version 348176 (0.0009) [2023-12-26 17:53:46,254][105620] Updated weights for policy 1, policy_version 348186 (0.0010) [2023-12-26 17:53:46,308][105620] Updated weights for policy 1, policy_version 348196 (0.0010) [2023-12-26 17:53:46,564][105692] Updated weights for policy 0, policy_version 347822 (0.0008) [2023-12-26 17:53:46,614][105692] Updated weights for policy 0, policy_version 347832 (0.0010) [2023-12-26 17:53:46,670][105692] Updated weights for policy 0, policy_version 347842 (0.0010) [2023-12-26 17:53:46,994][105620] Updated weights for policy 1, policy_version 348206 (0.0010) [2023-12-26 17:53:47,049][105620] Updated weights for policy 1, policy_version 348216 (0.0010) [2023-12-26 17:53:47,100][105620] Updated weights for policy 1, policy_version 348226 (0.0010) [2023-12-26 17:53:47,440][105692] Updated weights for policy 0, policy_version 347852 (0.0010) [2023-12-26 17:53:47,492][105692] Updated weights for policy 0, policy_version 347862 (0.0010) [2023-12-26 17:53:47,545][105692] Updated weights for policy 0, policy_version 347872 (0.0006) [2023-12-26 17:53:47,716][105620] Updated weights for policy 1, policy_version 348236 (0.0009) [2023-12-26 17:53:47,774][105620] Updated weights for policy 1, policy_version 348246 (0.0010) [2023-12-26 17:53:47,825][105620] Updated weights for policy 1, policy_version 348256 (0.0010) [2023-12-26 17:53:48,223][105692] Updated weights for policy 0, policy_version 347882 (0.0008) [2023-12-26 17:53:48,286][105692] Updated weights for policy 0, policy_version 347892 (0.0008) [2023-12-26 17:53:48,354][105692] Updated weights for policy 0, policy_version 347902 (0.0007) [2023-12-26 17:53:48,417][105692] Updated weights for policy 0, policy_version 347912 (0.0010) [2023-12-26 17:53:48,557][105620] Updated weights for policy 1, policy_version 348266 (0.0010) [2023-12-26 17:53:48,622][105620] Updated weights for policy 1, policy_version 348276 (0.0011) [2023-12-26 17:53:48,692][105620] Updated weights for policy 1, policy_version 348286 (0.0010) [2023-12-26 17:53:48,758][105620] Updated weights for policy 1, policy_version 348296 (0.0011) [2023-12-26 17:53:49,006][105692] Updated weights for policy 0, policy_version 347922 (0.0007) [2023-12-26 17:53:49,068][105692] Updated weights for policy 0, policy_version 347932 (0.0010) [2023-12-26 17:53:49,134][105692] Updated weights for policy 0, policy_version 347942 (0.0010) [2023-12-26 17:53:49,494][105620] Updated weights for policy 1, policy_version 348306 (0.0010) [2023-12-26 17:53:49,554][105620] Updated weights for policy 1, policy_version 348316 (0.0010) [2023-12-26 17:53:49,608][105620] Updated weights for policy 1, policy_version 348326 (0.0007) [2023-12-26 17:53:49,842][105692] Updated weights for policy 0, policy_version 347952 (0.0011) [2023-12-26 17:53:49,894][105692] Updated weights for policy 0, policy_version 347962 (0.0011) [2023-12-26 17:53:49,954][105692] Updated weights for policy 0, policy_version 347972 (0.0007) [2023-12-26 17:53:50,292][105620] Updated weights for policy 1, policy_version 348336 (0.0006) [2023-12-26 17:53:50,340][105620] Updated weights for policy 1, policy_version 348346 (0.0006) [2023-12-26 17:53:50,398][105620] Updated weights for policy 1, policy_version 348356 (0.0010) [2023-12-26 17:53:50,718][105692] Updated weights for policy 0, policy_version 347982 (0.0010) [2023-12-26 17:53:50,781][105692] Updated weights for policy 0, policy_version 347992 (0.0011) [2023-12-26 17:53:50,842][105692] Updated weights for policy 0, policy_version 348002 (0.0010) [2023-12-26 17:53:51,035][105620] Updated weights for policy 1, policy_version 348366 (0.0009) [2023-12-26 17:53:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 178290688. Throughput: 0: 9692.3, 1: 9729.3. Samples: 178280016. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 17:53:51,063][104569] Avg episode reward: [(0, '8993.786'), (1, '9355.320')] [2023-12-26 17:53:51,097][105620] Updated weights for policy 1, policy_version 348376 (0.0007) [2023-12-26 17:53:51,165][105620] Updated weights for policy 1, policy_version 348386 (0.0009) [2023-12-26 17:53:51,565][105692] Updated weights for policy 0, policy_version 348012 (0.0010) [2023-12-26 17:53:51,625][105692] Updated weights for policy 0, policy_version 348022 (0.0008) [2023-12-26 17:53:51,684][105692] Updated weights for policy 0, policy_version 348032 (0.0009) [2023-12-26 17:53:51,900][105620] Updated weights for policy 1, policy_version 348396 (0.0008) [2023-12-26 17:53:51,951][105620] Updated weights for policy 1, policy_version 348406 (0.0009) [2023-12-26 17:53:52,013][105620] Updated weights for policy 1, policy_version 348416 (0.0009) [2023-12-26 17:53:52,475][105692] Updated weights for policy 0, policy_version 348042 (0.0009) [2023-12-26 17:53:52,531][105692] Updated weights for policy 0, policy_version 348052 (0.0009) [2023-12-26 17:53:52,590][105692] Updated weights for policy 0, policy_version 348062 (0.0009) [2023-12-26 17:53:52,652][105692] Updated weights for policy 0, policy_version 348072 (0.0009) [2023-12-26 17:53:52,682][105620] Updated weights for policy 1, policy_version 348426 (0.0008) [2023-12-26 17:53:52,729][105620] Updated weights for policy 1, policy_version 348436 (0.0009) [2023-12-26 17:53:52,775][105620] Updated weights for policy 1, policy_version 348446 (0.0008) [2023-12-26 17:53:52,832][105620] Updated weights for policy 1, policy_version 348456 (0.0010) [2023-12-26 17:53:53,256][105692] Updated weights for policy 0, policy_version 348082 (0.0005) [2023-12-26 17:53:53,315][105692] Updated weights for policy 0, policy_version 348092 (0.0005) [2023-12-26 17:53:53,371][105692] Updated weights for policy 0, policy_version 348102 (0.0005) [2023-12-26 17:53:53,770][105620] Updated weights for policy 1, policy_version 348466 (0.0010) [2023-12-26 17:53:53,820][105620] Updated weights for policy 1, policy_version 348476 (0.0009) [2023-12-26 17:53:53,864][105620] Updated weights for policy 1, policy_version 348486 (0.0007) [2023-12-26 17:53:53,881][105692] Updated weights for policy 0, policy_version 348112 (0.0008) [2023-12-26 17:53:53,932][105692] Updated weights for policy 0, policy_version 348122 (0.0009) [2023-12-26 17:53:53,943][105585] KL-divergence is very high: 115.2347 [2023-12-26 17:53:53,982][105692] Updated weights for policy 0, policy_version 348132 (0.0009) [2023-12-26 17:53:54,608][105620] Updated weights for policy 1, policy_version 348496 (0.0010) [2023-12-26 17:53:54,647][105692] Updated weights for policy 0, policy_version 348142 (0.0008) [2023-12-26 17:53:54,668][105620] Updated weights for policy 1, policy_version 348506 (0.0008) [2023-12-26 17:53:54,702][105692] Updated weights for policy 0, policy_version 348152 (0.0009) [2023-12-26 17:53:54,727][105620] Updated weights for policy 1, policy_version 348516 (0.0009) [2023-12-26 17:53:54,761][105692] Updated weights for policy 0, policy_version 348162 (0.0007) [2023-12-26 17:53:55,447][105585] KL-divergence is very high: 125.8667 [2023-12-26 17:53:55,453][105692] Updated weights for policy 0, policy_version 348172 (0.0008) [2023-12-26 17:53:55,477][105620] Updated weights for policy 1, policy_version 348526 (0.0006) [2023-12-26 17:53:55,504][105692] Updated weights for policy 0, policy_version 348182 (0.0006) [2023-12-26 17:53:55,531][105620] Updated weights for policy 1, policy_version 348536 (0.0005) [2023-12-26 17:53:55,553][105692] Updated weights for policy 0, policy_version 348192 (0.0006) [2023-12-26 17:53:55,586][105620] Updated weights for policy 1, policy_version 348546 (0.0005) [2023-12-26 17:53:56,063][104569] Fps is (10 sec: 19659.3, 60 sec: 19387.4, 300 sec: 19521.9). Total num frames: 178388992. Throughput: 0: 9785.8, 1: 9781.6. Samples: 178399656. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 17:53:56,063][104569] Avg episode reward: [(0, '8632.416'), (1, '9355.199')] [2023-12-26 17:53:56,123][105692] Updated weights for policy 0, policy_version 348202 (0.0006) [2023-12-26 17:53:56,182][105692] Updated weights for policy 0, policy_version 348212 (0.0009) [2023-12-26 17:53:56,201][105620] Updated weights for policy 1, policy_version 348556 (0.0006) [2023-12-26 17:53:56,244][105692] Updated weights for policy 0, policy_version 348222 (0.0007) [2023-12-26 17:53:56,258][105620] Updated weights for policy 1, policy_version 348566 (0.0007) [2023-12-26 17:53:56,299][105692] Updated weights for policy 0, policy_version 348232 (0.0007) [2023-12-26 17:53:56,314][105620] Updated weights for policy 1, policy_version 348576 (0.0007) [2023-12-26 17:53:57,042][105692] Updated weights for policy 0, policy_version 348242 (0.0010) [2023-12-26 17:53:57,042][105620] Updated weights for policy 1, policy_version 348586 (0.0009) [2023-12-26 17:53:57,095][105692] Updated weights for policy 0, policy_version 348252 (0.0008) [2023-12-26 17:53:57,100][105586] KL-divergence is very high: 246.3331 [2023-12-26 17:53:57,105][105620] Updated weights for policy 1, policy_version 348596 (0.0005) [2023-12-26 17:53:57,140][105692] Updated weights for policy 0, policy_version 348262 (0.0009) [2023-12-26 17:53:57,141][105586] KL-divergence is very high: 385.7829 [2023-12-26 17:53:57,159][105620] Updated weights for policy 1, policy_version 348606 (0.0007) [2023-12-26 17:53:57,189][105586] KL-divergence is very high: 347.8644 [2023-12-26 17:53:57,219][105620] Updated weights for policy 1, policy_version 348616 (0.0008) [2023-12-26 17:53:57,796][105620] Updated weights for policy 1, policy_version 348626 (0.0006) [2023-12-26 17:53:57,857][105620] Updated weights for policy 1, policy_version 348636 (0.0010) [2023-12-26 17:53:57,907][105620] Updated weights for policy 1, policy_version 348646 (0.0010) [2023-12-26 17:53:57,995][105692] Updated weights for policy 0, policy_version 348272 (0.0008) [2023-12-26 17:53:58,042][105692] Updated weights for policy 0, policy_version 348282 (0.0007) [2023-12-26 17:53:58,098][105692] Updated weights for policy 0, policy_version 348292 (0.0008) [2023-12-26 17:53:58,681][105620] Updated weights for policy 1, policy_version 348656 (0.0011) [2023-12-26 17:53:58,748][105620] Updated weights for policy 1, policy_version 348666 (0.0011) [2023-12-26 17:53:58,819][105620] Updated weights for policy 1, policy_version 348676 (0.0010) [2023-12-26 17:53:58,935][105692] Updated weights for policy 0, policy_version 348302 (0.0008) [2023-12-26 17:53:59,001][105692] Updated weights for policy 0, policy_version 348312 (0.0008) [2023-12-26 17:53:59,059][105692] Updated weights for policy 0, policy_version 348322 (0.0007) [2023-12-26 17:53:59,552][105620] Updated weights for policy 1, policy_version 348686 (0.0009) [2023-12-26 17:53:59,606][105620] Updated weights for policy 1, policy_version 348696 (0.0010) [2023-12-26 17:53:59,674][105620] Updated weights for policy 1, policy_version 348706 (0.0010) [2023-12-26 17:53:59,787][105692] Updated weights for policy 0, policy_version 348332 (0.0008) [2023-12-26 17:53:59,851][105692] Updated weights for policy 0, policy_version 348342 (0.0009) [2023-12-26 17:53:59,908][105692] Updated weights for policy 0, policy_version 348352 (0.0009) [2023-12-26 17:54:00,387][105620] Updated weights for policy 1, policy_version 348716 (0.0009) [2023-12-26 17:54:00,448][105620] Updated weights for policy 1, policy_version 348726 (0.0009) [2023-12-26 17:54:00,503][105620] Updated weights for policy 1, policy_version 348736 (0.0009) [2023-12-26 17:54:00,684][105692] Updated weights for policy 0, policy_version 348362 (0.0009) [2023-12-26 17:54:00,745][105692] Updated weights for policy 0, policy_version 348372 (0.0009) [2023-12-26 17:54:00,805][105692] Updated weights for policy 0, policy_version 348382 (0.0009) [2023-12-26 17:54:00,865][105692] Updated weights for policy 0, policy_version 348392 (0.0009) [2023-12-26 17:54:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 178487296. Throughput: 0: 9809.4, 1: 9743.5. Samples: 178457160. Policy #0 lag: (min: 1.0, avg: 20.9, max: 33.0) [2023-12-26 17:54:01,063][104569] Avg episode reward: [(0, '8356.780'), (1, '9263.075')] [2023-12-26 17:54:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000348392_89202688.pth... [2023-12-26 17:54:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000348744_89284608.pth... [2023-12-26 17:54:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000347240_88907776.pth [2023-12-26 17:54:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000347624_88997888.pth [2023-12-26 17:54:01,293][105620] Updated weights for policy 1, policy_version 348746 (0.0009) [2023-12-26 17:54:01,348][105620] Updated weights for policy 1, policy_version 348756 (0.0011) [2023-12-26 17:54:01,411][105620] Updated weights for policy 1, policy_version 348766 (0.0008) [2023-12-26 17:54:01,477][105620] Updated weights for policy 1, policy_version 348776 (0.0005) [2023-12-26 17:54:01,638][105692] Updated weights for policy 0, policy_version 348402 (0.0009) [2023-12-26 17:54:01,706][105692] Updated weights for policy 0, policy_version 348412 (0.0010) [2023-12-26 17:54:01,769][105692] Updated weights for policy 0, policy_version 348422 (0.0009) [2023-12-26 17:54:02,168][105620] Updated weights for policy 1, policy_version 348786 (0.0007) [2023-12-26 17:54:02,233][105620] Updated weights for policy 1, policy_version 348796 (0.0009) [2023-12-26 17:54:02,291][105620] Updated weights for policy 1, policy_version 348806 (0.0008) [2023-12-26 17:54:02,610][105692] Updated weights for policy 0, policy_version 348432 (0.0009) [2023-12-26 17:54:02,661][105692] Updated weights for policy 0, policy_version 348442 (0.0008) [2023-12-26 17:54:02,718][105692] Updated weights for policy 0, policy_version 348452 (0.0009) [2023-12-26 17:54:02,954][105620] Updated weights for policy 1, policy_version 348816 (0.0005) [2023-12-26 17:54:03,006][105620] Updated weights for policy 1, policy_version 348826 (0.0005) [2023-12-26 17:54:03,062][105620] Updated weights for policy 1, policy_version 348836 (0.0006) [2023-12-26 17:54:03,475][105692] Updated weights for policy 0, policy_version 348462 (0.0007) [2023-12-26 17:54:03,527][105692] Updated weights for policy 0, policy_version 348472 (0.0005) [2023-12-26 17:54:03,582][105692] Updated weights for policy 0, policy_version 348482 (0.0005) [2023-12-26 17:54:03,733][105620] Updated weights for policy 1, policy_version 348846 (0.0010) [2023-12-26 17:54:03,785][105620] Updated weights for policy 1, policy_version 348858 (0.0010) [2023-12-26 17:54:03,837][105620] Updated weights for policy 1, policy_version 348868 (0.0009) [2023-12-26 17:54:04,176][105692] Updated weights for policy 0, policy_version 348492 (0.0007) [2023-12-26 17:54:04,238][105692] Updated weights for policy 0, policy_version 348502 (0.0010) [2023-12-26 17:54:04,303][105692] Updated weights for policy 0, policy_version 348512 (0.0010) [2023-12-26 17:54:04,581][105620] Updated weights for policy 1, policy_version 348878 (0.0007) [2023-12-26 17:54:04,644][105620] Updated weights for policy 1, policy_version 348888 (0.0008) [2023-12-26 17:54:04,695][105620] Updated weights for policy 1, policy_version 348898 (0.0009) [2023-12-26 17:54:05,072][105692] Updated weights for policy 0, policy_version 348522 (0.0009) [2023-12-26 17:54:05,132][105692] Updated weights for policy 0, policy_version 348532 (0.0010) [2023-12-26 17:54:05,190][105692] Updated weights for policy 0, policy_version 348542 (0.0009) [2023-12-26 17:54:05,247][105692] Updated weights for policy 0, policy_version 348552 (0.0010) [2023-12-26 17:54:05,360][105620] Updated weights for policy 1, policy_version 348908 (0.0007) [2023-12-26 17:54:05,422][105620] Updated weights for policy 1, policy_version 348918 (0.0005) [2023-12-26 17:54:05,485][105620] Updated weights for policy 1, policy_version 348928 (0.0005) [2023-12-26 17:54:06,053][105692] Updated weights for policy 0, policy_version 348562 (0.0009) [2023-12-26 17:54:06,062][104569] Fps is (10 sec: 18843.0, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 178577408. Throughput: 0: 9839.0, 1: 9682.1. Samples: 178572228. Policy #0 lag: (min: 1.0, avg: 20.9, max: 33.0) [2023-12-26 17:54:06,063][104569] Avg episode reward: [(0, '8721.336'), (1, '9355.984')] [2023-12-26 17:54:06,113][105620] Updated weights for policy 1, policy_version 348938 (0.0006) [2023-12-26 17:54:06,114][105692] Updated weights for policy 0, policy_version 348572 (0.0009) [2023-12-26 17:54:06,174][105692] Updated weights for policy 0, policy_version 348582 (0.0008) [2023-12-26 17:54:06,181][105620] Updated weights for policy 1, policy_version 348948 (0.0007) [2023-12-26 17:54:06,242][105620] Updated weights for policy 1, policy_version 348958 (0.0005) [2023-12-26 17:54:06,303][105620] Updated weights for policy 1, policy_version 348968 (0.0009) [2023-12-26 17:54:06,914][105620] Updated weights for policy 1, policy_version 348978 (0.0008) [2023-12-26 17:54:06,970][105620] Updated weights for policy 1, policy_version 348988 (0.0007) [2023-12-26 17:54:06,991][105692] Updated weights for policy 0, policy_version 348592 (0.0008) [2023-12-26 17:54:07,022][105620] Updated weights for policy 1, policy_version 348998 (0.0006) [2023-12-26 17:54:07,053][105692] Updated weights for policy 0, policy_version 348602 (0.0008) [2023-12-26 17:54:07,104][105692] Updated weights for policy 0, policy_version 348612 (0.0009) [2023-12-26 17:54:07,689][105620] Updated weights for policy 1, policy_version 349008 (0.0008) [2023-12-26 17:54:07,746][105620] Updated weights for policy 1, policy_version 349018 (0.0007) [2023-12-26 17:54:07,795][105620] Updated weights for policy 1, policy_version 349028 (0.0010) [2023-12-26 17:54:07,928][105692] Updated weights for policy 0, policy_version 348622 (0.0008) [2023-12-26 17:54:07,989][105692] Updated weights for policy 0, policy_version 348632 (0.0008) [2023-12-26 17:54:08,038][105585] KL-divergence is very high: 117.2817 [2023-12-26 17:54:08,048][105692] Updated weights for policy 0, policy_version 348642 (0.0008) [2023-12-26 17:54:08,058][105585] KL-divergence is very high: 157.8461 [2023-12-26 17:54:08,474][105620] Updated weights for policy 1, policy_version 349038 (0.0007) [2023-12-26 17:54:08,523][105620] Updated weights for policy 1, policy_version 349048 (0.0009) [2023-12-26 17:54:08,571][105620] Updated weights for policy 1, policy_version 349058 (0.0010) [2023-12-26 17:54:08,812][105692] Updated weights for policy 0, policy_version 348652 (0.0009) [2023-12-26 17:54:08,841][105585] KL-divergence is very high: 197.0291 [2023-12-26 17:54:08,876][105692] Updated weights for policy 0, policy_version 348662 (0.0008) [2023-12-26 17:54:08,891][105585] KL-divergence is very high: 134.0985 [2023-12-26 17:54:08,940][105692] Updated weights for policy 0, policy_version 348672 (0.0008) [2023-12-26 17:54:09,264][105620] Updated weights for policy 1, policy_version 349068 (0.0010) [2023-12-26 17:54:09,325][105620] Updated weights for policy 1, policy_version 349078 (0.0011) [2023-12-26 17:54:09,395][105620] Updated weights for policy 1, policy_version 349088 (0.0010) [2023-12-26 17:54:09,740][105692] Updated weights for policy 0, policy_version 348682 (0.0010) [2023-12-26 17:54:09,800][105692] Updated weights for policy 0, policy_version 348692 (0.0009) [2023-12-26 17:54:09,836][105585] KL-divergence is very high: 135.2478 [2023-12-26 17:54:09,869][105692] Updated weights for policy 0, policy_version 348702 (0.0008) [2023-12-26 17:54:09,935][105692] Updated weights for policy 0, policy_version 348712 (0.0009) [2023-12-26 17:54:10,198][105620] Updated weights for policy 1, policy_version 349098 (0.0010) [2023-12-26 17:54:10,245][105620] Updated weights for policy 1, policy_version 349108 (0.0008) [2023-12-26 17:54:10,292][105620] Updated weights for policy 1, policy_version 349118 (0.0009) [2023-12-26 17:54:10,343][105620] Updated weights for policy 1, policy_version 349128 (0.0009) [2023-12-26 17:54:10,680][105585] KL-divergence is very high: 133.5028 [2023-12-26 17:54:10,727][105692] Updated weights for policy 0, policy_version 348722 (0.0010) [2023-12-26 17:54:10,776][105692] Updated weights for policy 0, policy_version 348732 (0.0008) [2023-12-26 17:54:10,830][105692] Updated weights for policy 0, policy_version 348742 (0.0010) [2023-12-26 17:54:10,973][105620] Updated weights for policy 1, policy_version 349138 (0.0005) [2023-12-26 17:54:11,041][105620] Updated weights for policy 1, policy_version 349148 (0.0007) [2023-12-26 17:54:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 178675712. Throughput: 0: 9643.5, 1: 9811.1. Samples: 178687900. Policy #0 lag: (min: 1.0, avg: 20.9, max: 33.0) [2023-12-26 17:54:11,062][104569] Avg episode reward: [(0, '3306.935'), (1, '5863.631')] [2023-12-26 17:54:11,096][105620] Updated weights for policy 1, policy_version 349158 (0.0008) [2023-12-26 17:54:11,669][105692] Updated weights for policy 0, policy_version 348752 (0.0009) [2023-12-26 17:54:11,740][105692] Updated weights for policy 0, policy_version 348762 (0.0009) [2023-12-26 17:54:11,805][105692] Updated weights for policy 0, policy_version 348772 (0.0008) [2023-12-26 17:54:11,867][105620] Updated weights for policy 1, policy_version 349168 (0.0010) [2023-12-26 17:54:11,937][105620] Updated weights for policy 1, policy_version 349178 (0.0010) [2023-12-26 17:54:11,993][105620] Updated weights for policy 1, policy_version 349188 (0.0010) [2023-12-26 17:54:12,562][105692] Updated weights for policy 0, policy_version 348782 (0.0008) [2023-12-26 17:54:12,620][105692] Updated weights for policy 0, policy_version 348792 (0.0010) [2023-12-26 17:54:12,688][105692] Updated weights for policy 0, policy_version 348802 (0.0009) [2023-12-26 17:54:12,698][105620] Updated weights for policy 1, policy_version 349198 (0.0009) [2023-12-26 17:54:12,764][105620] Updated weights for policy 1, policy_version 349208 (0.0011) [2023-12-26 17:54:12,829][105620] Updated weights for policy 1, policy_version 349218 (0.0010) [2023-12-26 17:54:13,454][105620] Updated weights for policy 1, policy_version 349228 (0.0008) [2023-12-26 17:54:13,515][105620] Updated weights for policy 1, policy_version 349238 (0.0010) [2023-12-26 17:54:13,530][105692] Updated weights for policy 0, policy_version 348812 (0.0006) [2023-12-26 17:54:13,578][105620] Updated weights for policy 1, policy_version 349248 (0.0010) [2023-12-26 17:54:13,588][105692] Updated weights for policy 0, policy_version 348822 (0.0006) [2023-12-26 17:54:13,649][105692] Updated weights for policy 0, policy_version 348832 (0.0008) [2023-12-26 17:54:14,307][105620] Updated weights for policy 1, policy_version 349258 (0.0010) [2023-12-26 17:54:14,358][105620] Updated weights for policy 1, policy_version 349268 (0.0010) [2023-12-26 17:54:14,416][105620] Updated weights for policy 1, policy_version 349278 (0.0010) [2023-12-26 17:54:14,423][105692] Updated weights for policy 0, policy_version 348842 (0.0008) [2023-12-26 17:54:14,473][105620] Updated weights for policy 1, policy_version 349288 (0.0010) [2023-12-26 17:54:14,480][105692] Updated weights for policy 0, policy_version 348852 (0.0006) [2023-12-26 17:54:14,525][105692] Updated weights for policy 0, policy_version 348862 (0.0007) [2023-12-26 17:54:14,574][105692] Updated weights for policy 0, policy_version 348872 (0.0008) [2023-12-26 17:54:15,134][105620] Updated weights for policy 1, policy_version 349298 (0.0010) [2023-12-26 17:54:15,186][105620] Updated weights for policy 1, policy_version 349308 (0.0010) [2023-12-26 17:54:15,245][105620] Updated weights for policy 1, policy_version 349318 (0.0010) [2023-12-26 17:54:15,409][105692] Updated weights for policy 0, policy_version 348882 (0.0008) [2023-12-26 17:54:15,472][105692] Updated weights for policy 0, policy_version 348892 (0.0008) [2023-12-26 17:54:15,532][105692] Updated weights for policy 0, policy_version 348902 (0.0008) [2023-12-26 17:54:15,903][105620] Updated weights for policy 1, policy_version 349328 (0.0010) [2023-12-26 17:54:15,947][105620] Updated weights for policy 1, policy_version 349338 (0.0010) [2023-12-26 17:54:16,012][105620] Updated weights for policy 1, policy_version 349348 (0.0010) [2023-12-26 17:54:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 178774016. Throughput: 0: 9511.5, 1: 9821.1. Samples: 178742520. Policy #0 lag: (min: 1.0, avg: 20.9, max: 33.0) [2023-12-26 17:54:16,062][104569] Avg episode reward: [(0, '2089.841'), (1, '2983.381')] [2023-12-26 17:54:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000349352_89440256.pth... [2023-12-26 17:54:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000348904_89333760.pth... [2023-12-26 17:54:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000347816_89055232.pth [2023-12-26 17:54:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000348168_89137152.pth [2023-12-26 17:54:16,322][105692] Updated weights for policy 0, policy_version 348912 (0.0009) [2023-12-26 17:54:16,380][105692] Updated weights for policy 0, policy_version 348922 (0.0009) [2023-12-26 17:54:16,427][105692] Updated weights for policy 0, policy_version 348932 (0.0009) [2023-12-26 17:54:16,693][105620] Updated weights for policy 1, policy_version 349358 (0.0007) [2023-12-26 17:54:16,752][105620] Updated weights for policy 1, policy_version 349368 (0.0005) [2023-12-26 17:54:16,808][105620] Updated weights for policy 1, policy_version 349378 (0.0008) [2023-12-26 17:54:17,261][105692] Updated weights for policy 0, policy_version 348942 (0.0009) [2023-12-26 17:54:17,315][105692] Updated weights for policy 0, policy_version 348952 (0.0009) [2023-12-26 17:54:17,367][105692] Updated weights for policy 0, policy_version 348963 (0.0009) [2023-12-26 17:54:17,417][105620] Updated weights for policy 1, policy_version 349388 (0.0008) [2023-12-26 17:54:17,466][105620] Updated weights for policy 1, policy_version 349398 (0.0007) [2023-12-26 17:54:17,527][105620] Updated weights for policy 1, policy_version 349408 (0.0006) [2023-12-26 17:54:18,178][105620] Updated weights for policy 1, policy_version 349418 (0.0006) [2023-12-26 17:54:18,221][105692] Updated weights for policy 0, policy_version 348973 (0.0010) [2023-12-26 17:54:18,242][105620] Updated weights for policy 1, policy_version 349428 (0.0006) [2023-12-26 17:54:18,277][105692] Updated weights for policy 0, policy_version 348983 (0.0008) [2023-12-26 17:54:18,303][105620] Updated weights for policy 1, policy_version 349438 (0.0009) [2023-12-26 17:54:18,335][105692] Updated weights for policy 0, policy_version 348993 (0.0007) [2023-12-26 17:54:18,366][105620] Updated weights for policy 1, policy_version 349448 (0.0009) [2023-12-26 17:54:19,112][105620] Updated weights for policy 1, policy_version 349458 (0.0008) [2023-12-26 17:54:19,114][105692] Updated weights for policy 0, policy_version 349003 (0.0009) [2023-12-26 17:54:19,162][105692] Updated weights for policy 0, policy_version 349013 (0.0006) [2023-12-26 17:54:19,171][105620] Updated weights for policy 1, policy_version 349468 (0.0008) [2023-12-26 17:54:19,223][105692] Updated weights for policy 0, policy_version 349023 (0.0007) [2023-12-26 17:54:19,235][105620] Updated weights for policy 1, policy_version 349478 (0.0009) [2023-12-26 17:54:19,970][105692] Updated weights for policy 0, policy_version 349033 (0.0008) [2023-12-26 17:54:19,973][105620] Updated weights for policy 1, policy_version 349488 (0.0008) [2023-12-26 17:54:20,033][105692] Updated weights for policy 0, policy_version 349043 (0.0008) [2023-12-26 17:54:20,041][105620] Updated weights for policy 1, policy_version 349498 (0.0009) [2023-12-26 17:54:20,095][105692] Updated weights for policy 0, policy_version 349053 (0.0008) [2023-12-26 17:54:20,105][105620] Updated weights for policy 1, policy_version 349508 (0.0006) [2023-12-26 17:54:20,151][105692] Updated weights for policy 0, policy_version 349063 (0.0010) [2023-12-26 17:54:20,845][105692] Updated weights for policy 0, policy_version 349073 (0.0009) [2023-12-26 17:54:20,865][105620] Updated weights for policy 1, policy_version 349518 (0.0007) [2023-12-26 17:54:20,907][105692] Updated weights for policy 0, policy_version 349083 (0.0006) [2023-12-26 17:54:20,928][105620] Updated weights for policy 1, policy_version 349528 (0.0006) [2023-12-26 17:54:20,956][105692] Updated weights for policy 0, policy_version 349093 (0.0008) [2023-12-26 17:54:20,982][105586] KL-divergence is very high: 146.0574 [2023-12-26 17:54:20,987][105620] Updated weights for policy 1, policy_version 349538 (0.0006) [2023-12-26 17:54:20,989][105586] KL-divergence is very high: 142.0338 [2023-12-26 17:54:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 178872320. Throughput: 0: 9353.9, 1: 9819.7. Samples: 178855928. Policy #0 lag: (min: 1.0, avg: 20.9, max: 33.0) [2023-12-26 17:54:21,062][104569] Avg episode reward: [(0, '4220.089'), (1, '6930.471')] [2023-12-26 17:54:21,630][105620] Updated weights for policy 1, policy_version 349548 (0.0007) [2023-12-26 17:54:21,654][105692] Updated weights for policy 0, policy_version 349103 (0.0008) [2023-12-26 17:54:21,668][105586] KL-divergence is very high: 144.6355 [2023-12-26 17:54:21,673][105586] KL-divergence is very high: 184.7726 [2023-12-26 17:54:21,680][105586] KL-divergence is very high: 123.6398 [2023-12-26 17:54:21,688][105586] KL-divergence is very high: 237.2014 [2023-12-26 17:54:21,694][105586] KL-divergence is very high: 248.8256 [2023-12-26 17:54:21,696][105620] Updated weights for policy 1, policy_version 349558 (0.0009) [2023-12-26 17:54:21,705][105586] KL-divergence is very high: 257.3444 [2023-12-26 17:54:21,712][105586] KL-divergence is very high: 127.9467 [2023-12-26 17:54:21,721][105692] Updated weights for policy 0, policy_version 349113 (0.0008) [2023-12-26 17:54:21,725][105586] KL-divergence is very high: 132.6960 [2023-12-26 17:54:21,733][105586] KL-divergence is very high: 157.0816 [2023-12-26 17:54:21,741][105586] KL-divergence is very high: 124.0319 [2023-12-26 17:54:21,772][105620] Updated weights for policy 1, policy_version 349568 (0.0008) [2023-12-26 17:54:21,789][105692] Updated weights for policy 0, policy_version 349123 (0.0007) [2023-12-26 17:54:21,791][105586] KL-divergence is very high: 106.5670 [2023-12-26 17:54:22,491][105692] Updated weights for policy 0, policy_version 349133 (0.0007) [2023-12-26 17:54:22,531][105586] KL-divergence is very high: 431.3377 [2023-12-26 17:54:22,538][105586] KL-divergence is very high: 438.2763 [2023-12-26 17:54:22,543][105692] Updated weights for policy 0, policy_version 349143 (0.0008) [2023-12-26 17:54:22,544][105620] Updated weights for policy 1, policy_version 349578 (0.0008) [2023-12-26 17:54:22,545][105586] KL-divergence is very high: 334.9525 [2023-12-26 17:54:22,551][105586] KL-divergence is very high: 409.5154 [2023-12-26 17:54:22,557][105586] KL-divergence is very high: 346.4993 [2023-12-26 17:54:22,583][105586] KL-divergence is very high: 293.5564 [2023-12-26 17:54:22,589][105586] KL-divergence is very high: 291.6699 [2023-12-26 17:54:22,592][105692] Updated weights for policy 0, policy_version 349153 (0.0008) [2023-12-26 17:54:22,596][105586] KL-divergence is very high: 203.9748 [2023-12-26 17:54:22,602][105586] KL-divergence is very high: 275.9341 [2023-12-26 17:54:22,608][105620] Updated weights for policy 1, policy_version 349588 (0.0007) [2023-12-26 17:54:22,609][105586] KL-divergence is very high: 259.1111 [2023-12-26 17:54:22,637][105586] KL-divergence is very high: 235.1440 [2023-12-26 17:54:22,643][105586] KL-divergence is very high: 236.8633 [2023-12-26 17:54:22,648][105586] KL-divergence is very high: 150.8624 [2023-12-26 17:54:22,653][105586] KL-divergence is very high: 221.2088 [2023-12-26 17:54:22,659][105586] KL-divergence is very high: 206.4021 [2023-12-26 17:54:22,672][105620] Updated weights for policy 1, policy_version 349598 (0.0010) [2023-12-26 17:54:22,683][105586] KL-divergence is very high: 173.3493 [2023-12-26 17:54:22,690][105586] KL-divergence is very high: 182.2748 [2023-12-26 17:54:22,695][105586] KL-divergence is very high: 106.7768 [2023-12-26 17:54:22,703][105586] KL-divergence is very high: 171.8677 [2023-12-26 17:54:22,708][105586] KL-divergence is very high: 158.9062 [2023-12-26 17:54:22,733][105620] Updated weights for policy 1, policy_version 349608 (0.0010) [2023-12-26 17:54:23,282][105692] Updated weights for policy 0, policy_version 349163 (0.0008) [2023-12-26 17:54:23,333][105692] Updated weights for policy 0, policy_version 349173 (0.0009) [2023-12-26 17:54:23,389][105692] Updated weights for policy 0, policy_version 349183 (0.0006) [2023-12-26 17:54:23,508][105620] Updated weights for policy 1, policy_version 349618 (0.0006) [2023-12-26 17:54:23,554][105620] Updated weights for policy 1, policy_version 349628 (0.0005) [2023-12-26 17:54:23,599][105620] Updated weights for policy 1, policy_version 349638 (0.0005) [2023-12-26 17:54:24,207][105692] Updated weights for policy 0, policy_version 349193 (0.0009) [2023-12-26 17:54:24,226][105620] Updated weights for policy 1, policy_version 349648 (0.0009) [2023-12-26 17:54:24,263][105692] Updated weights for policy 0, policy_version 349203 (0.0005) [2023-12-26 17:54:24,281][105620] Updated weights for policy 1, policy_version 349658 (0.0010) [2023-12-26 17:54:24,322][105692] Updated weights for policy 0, policy_version 349213 (0.0007) [2023-12-26 17:54:24,339][105620] Updated weights for policy 1, policy_version 349668 (0.0007) [2023-12-26 17:54:24,368][105692] Updated weights for policy 0, policy_version 349223 (0.0008) [2023-12-26 17:54:24,953][105620] Updated weights for policy 1, policy_version 349678 (0.0008) [2023-12-26 17:54:25,001][105620] Updated weights for policy 1, policy_version 349688 (0.0009) [2023-12-26 17:54:25,052][105620] Updated weights for policy 1, policy_version 349698 (0.0008) [2023-12-26 17:54:25,206][105692] Updated weights for policy 0, policy_version 349235 (0.0011) [2023-12-26 17:54:25,259][105692] Updated weights for policy 0, policy_version 349246 (0.0010) [2023-12-26 17:54:25,644][105620] Updated weights for policy 1, policy_version 349708 (0.0006) [2023-12-26 17:54:25,711][105620] Updated weights for policy 1, policy_version 349718 (0.0005) [2023-12-26 17:54:25,779][105620] Updated weights for policy 1, policy_version 349728 (0.0005) [2023-12-26 17:54:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 178962432. Throughput: 0: 9397.6, 1: 9941.4. Samples: 178973900. Policy #0 lag: (min: 1.0, avg: 20.9, max: 33.0) [2023-12-26 17:54:26,062][104569] Avg episode reward: [(0, '7501.655'), (1, '1162.519')] [2023-12-26 17:54:26,177][105692] Updated weights for policy 0, policy_version 349258 (0.0010) [2023-12-26 17:54:26,234][105692] Updated weights for policy 0, policy_version 349268 (0.0010) [2023-12-26 17:54:26,286][105692] Updated weights for policy 0, policy_version 349278 (0.0010) [2023-12-26 17:54:26,293][105620] Updated weights for policy 1, policy_version 349738 (0.0005) [2023-12-26 17:54:26,334][105692] Updated weights for policy 0, policy_version 349288 (0.0008) [2023-12-26 17:54:26,350][105620] Updated weights for policy 1, policy_version 349748 (0.0006) [2023-12-26 17:54:26,404][105620] Updated weights for policy 1, policy_version 349758 (0.0009) [2023-12-26 17:54:26,463][105620] Updated weights for policy 1, policy_version 349768 (0.0010) [2023-12-26 17:54:27,120][105620] Updated weights for policy 1, policy_version 349778 (0.0010) [2023-12-26 17:54:27,170][105692] Updated weights for policy 0, policy_version 349298 (0.0007) [2023-12-26 17:54:27,179][105620] Updated weights for policy 1, policy_version 349788 (0.0010) [2023-12-26 17:54:27,225][105692] Updated weights for policy 0, policy_version 349308 (0.0009) [2023-12-26 17:54:27,245][105620] Updated weights for policy 1, policy_version 349798 (0.0011) [2023-12-26 17:54:27,278][105692] Updated weights for policy 0, policy_version 349318 (0.0007) [2023-12-26 17:54:27,976][105620] Updated weights for policy 1, policy_version 349808 (0.0010) [2023-12-26 17:54:28,002][105692] Updated weights for policy 0, policy_version 349328 (0.0006) [2023-12-26 17:54:28,034][105620] Updated weights for policy 1, policy_version 349818 (0.0010) [2023-12-26 17:54:28,049][105692] Updated weights for policy 0, policy_version 349338 (0.0006) [2023-12-26 17:54:28,088][105620] Updated weights for policy 1, policy_version 349828 (0.0010) [2023-12-26 17:54:28,105][105692] Updated weights for policy 0, policy_version 349348 (0.0005) [2023-12-26 17:54:28,785][105692] Updated weights for policy 0, policy_version 349358 (0.0007) [2023-12-26 17:54:28,830][105620] Updated weights for policy 1, policy_version 349838 (0.0008) [2023-12-26 17:54:28,838][105692] Updated weights for policy 0, policy_version 349368 (0.0008) [2023-12-26 17:54:28,880][105620] Updated weights for policy 1, policy_version 349848 (0.0005) [2023-12-26 17:54:28,899][105692] Updated weights for policy 0, policy_version 349378 (0.0008) [2023-12-26 17:54:28,941][105620] Updated weights for policy 1, policy_version 349858 (0.0009) [2023-12-26 17:54:29,665][105620] Updated weights for policy 1, policy_version 349868 (0.0010) [2023-12-26 17:54:29,690][105692] Updated weights for policy 0, policy_version 349388 (0.0007) [2023-12-26 17:54:29,716][105620] Updated weights for policy 1, policy_version 349878 (0.0010) [2023-12-26 17:54:29,743][105692] Updated weights for policy 0, policy_version 349398 (0.0006) [2023-12-26 17:54:29,764][105620] Updated weights for policy 1, policy_version 349888 (0.0010) [2023-12-26 17:54:29,802][105692] Updated weights for policy 0, policy_version 349408 (0.0005) [2023-12-26 17:54:30,387][105620] Updated weights for policy 1, policy_version 349898 (0.0009) [2023-12-26 17:54:30,443][105620] Updated weights for policy 1, policy_version 349908 (0.0005) [2023-12-26 17:54:30,510][105620] Updated weights for policy 1, policy_version 349918 (0.0006) [2023-12-26 17:54:30,577][105620] Updated weights for policy 1, policy_version 349928 (0.0006) [2023-12-26 17:54:30,583][105692] Updated weights for policy 0, policy_version 349418 (0.0007) [2023-12-26 17:54:30,646][105692] Updated weights for policy 0, policy_version 349428 (0.0008) [2023-12-26 17:54:30,691][105692] Updated weights for policy 0, policy_version 349438 (0.0008) [2023-12-26 17:54:30,746][105692] Updated weights for policy 0, policy_version 349448 (0.0008) [2023-12-26 17:54:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 179060736. Throughput: 0: 9421.4, 1: 10013.2. Samples: 179031844. Policy #0 lag: (min: 1.0, avg: 20.9, max: 33.0) [2023-12-26 17:54:31,062][104569] Avg episode reward: [(0, '8450.136'), (1, '1525.634')] [2023-12-26 17:54:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000349448_89473024.pth... [2023-12-26 17:54:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000349928_89587712.pth... [2023-12-26 17:54:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000348392_89202688.pth [2023-12-26 17:54:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000348744_89284608.pth [2023-12-26 17:54:31,226][105620] Updated weights for policy 1, policy_version 349938 (0.0006) [2023-12-26 17:54:31,289][105620] Updated weights for policy 1, policy_version 349948 (0.0010) [2023-12-26 17:54:31,348][105620] Updated weights for policy 1, policy_version 349958 (0.0010) [2023-12-26 17:54:31,422][105585] KL-divergence is very high: 127.0057 [2023-12-26 17:54:31,447][105585] KL-divergence is very high: 120.5991 [2023-12-26 17:54:31,470][105692] Updated weights for policy 0, policy_version 349458 (0.0005) [2023-12-26 17:54:31,503][105585] KL-divergence is very high: 389.4119 [2023-12-26 17:54:31,534][105692] Updated weights for policy 0, policy_version 349468 (0.0006) [2023-12-26 17:54:31,549][105585] KL-divergence is very high: 268.4454 [2023-12-26 17:54:31,559][105585] KL-divergence is very high: 930.1730 [2023-12-26 17:54:31,582][105585] KL-divergence is very high: 123.3046 [2023-12-26 17:54:31,607][105692] Updated weights for policy 0, policy_version 349478 (0.0006) [2023-12-26 17:54:31,607][105585] KL-divergence is very high: 379.2702 [2023-12-26 17:54:31,613][105585] KL-divergence is very high: 1078.6145 [2023-12-26 17:54:32,107][105620] Updated weights for policy 1, policy_version 349968 (0.0009) [2023-12-26 17:54:32,168][105620] Updated weights for policy 1, policy_version 349978 (0.0009) [2023-12-26 17:54:32,213][105692] Updated weights for policy 0, policy_version 349488 (0.0009) [2023-12-26 17:54:32,224][105620] Updated weights for policy 1, policy_version 349988 (0.0007) [2023-12-26 17:54:32,272][105692] Updated weights for policy 0, policy_version 349498 (0.0008) [2023-12-26 17:54:32,320][105692] Updated weights for policy 0, policy_version 349508 (0.0009) [2023-12-26 17:54:32,955][105620] Updated weights for policy 1, policy_version 349998 (0.0009) [2023-12-26 17:54:33,010][105620] Updated weights for policy 1, policy_version 350008 (0.0010) [2023-12-26 17:54:33,068][105692] Updated weights for policy 0, policy_version 349518 (0.0008) [2023-12-26 17:54:33,068][105620] Updated weights for policy 1, policy_version 350018 (0.0007) [2023-12-26 17:54:33,128][105692] Updated weights for policy 0, policy_version 349528 (0.0009) [2023-12-26 17:54:33,193][105692] Updated weights for policy 0, policy_version 349538 (0.0009) [2023-12-26 17:54:33,682][105620] Updated weights for policy 1, policy_version 350028 (0.0005) [2023-12-26 17:54:33,737][105620] Updated weights for policy 1, policy_version 350038 (0.0005) [2023-12-26 17:54:33,793][105620] Updated weights for policy 1, policy_version 350048 (0.0007) [2023-12-26 17:54:33,939][105692] Updated weights for policy 0, policy_version 349548 (0.0008) [2023-12-26 17:54:33,999][105692] Updated weights for policy 0, policy_version 349558 (0.0005) [2023-12-26 17:54:34,063][105692] Updated weights for policy 0, policy_version 349568 (0.0005) [2023-12-26 17:54:34,501][105620] Updated weights for policy 1, policy_version 350058 (0.0010) [2023-12-26 17:54:34,563][105620] Updated weights for policy 1, policy_version 350068 (0.0011) [2023-12-26 17:54:34,622][105620] Updated weights for policy 1, policy_version 350078 (0.0008) [2023-12-26 17:54:34,682][105620] Updated weights for policy 1, policy_version 350088 (0.0008) [2023-12-26 17:54:34,742][105692] Updated weights for policy 0, policy_version 349578 (0.0010) [2023-12-26 17:54:34,787][105692] Updated weights for policy 0, policy_version 349588 (0.0005) [2023-12-26 17:54:34,836][105692] Updated weights for policy 0, policy_version 349598 (0.0005) [2023-12-26 17:54:34,892][105692] Updated weights for policy 0, policy_version 349608 (0.0010) [2023-12-26 17:54:35,426][105620] Updated weights for policy 1, policy_version 350098 (0.0011) [2023-12-26 17:54:35,474][105620] Updated weights for policy 1, policy_version 350108 (0.0010) [2023-12-26 17:54:35,529][105620] Updated weights for policy 1, policy_version 350118 (0.0010) [2023-12-26 17:54:35,603][105692] Updated weights for policy 0, policy_version 349618 (0.0011) [2023-12-26 17:54:35,661][105692] Updated weights for policy 0, policy_version 349628 (0.0010) [2023-12-26 17:54:35,678][105585] KL-divergence is very high: 134.4979 [2023-12-26 17:54:35,685][105585] KL-divergence is very high: 115.1450 [2023-12-26 17:54:35,712][105585] KL-divergence is very high: 120.5468 [2023-12-26 17:54:35,718][105585] KL-divergence is very high: 122.8046 [2023-12-26 17:54:35,719][105692] Updated weights for policy 0, policy_version 349638 (0.0010) [2023-12-26 17:54:35,725][105585] KL-divergence is very high: 205.2680 [2023-12-26 17:54:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 179159040. Throughput: 0: 9308.3, 1: 10012.9. Samples: 179149472. Policy #0 lag: (min: 1.0, avg: 20.9, max: 33.0) [2023-12-26 17:54:36,063][104569] Avg episode reward: [(0, '7410.483'), (1, '1410.419')] [2023-12-26 17:54:36,200][105620] Updated weights for policy 1, policy_version 350128 (0.0008) [2023-12-26 17:54:36,265][105620] Updated weights for policy 1, policy_version 350138 (0.0006) [2023-12-26 17:54:36,336][105620] Updated weights for policy 1, policy_version 350148 (0.0006) [2023-12-26 17:54:36,386][105585] KL-divergence is very high: 245.2111 [2023-12-26 17:54:36,393][105585] KL-divergence is very high: 207.0496 [2023-12-26 17:54:36,400][105585] KL-divergence is very high: 166.5278 [2023-12-26 17:54:36,406][105585] KL-divergence is very high: 305.2704 [2023-12-26 17:54:36,412][105585] KL-divergence is very high: 201.7090 [2023-12-26 17:54:36,435][105692] Updated weights for policy 0, policy_version 349648 (0.0007) [2023-12-26 17:54:36,437][105585] KL-divergence is very high: 183.7927 [2023-12-26 17:54:36,443][105585] KL-divergence is very high: 125.7435 [2023-12-26 17:54:36,455][105585] KL-divergence is very high: 205.4164 [2023-12-26 17:54:36,461][105585] KL-divergence is very high: 118.6469 [2023-12-26 17:54:36,486][105585] KL-divergence is very high: 102.1204 [2023-12-26 17:54:36,499][105692] Updated weights for policy 0, policy_version 349658 (0.0006) [2023-12-26 17:54:36,505][105585] KL-divergence is very high: 122.7843 [2023-12-26 17:54:36,558][105585] KL-divergence is very high: 156.6851 [2023-12-26 17:54:36,566][105692] Updated weights for policy 0, policy_version 349668 (0.0011) [2023-12-26 17:54:36,895][105620] Updated weights for policy 1, policy_version 350158 (0.0007) [2023-12-26 17:54:36,952][105620] Updated weights for policy 1, policy_version 350168 (0.0008) [2023-12-26 17:54:37,007][105620] Updated weights for policy 1, policy_version 350178 (0.0008) [2023-12-26 17:54:37,258][105692] Updated weights for policy 0, policy_version 349678 (0.0011) [2023-12-26 17:54:37,320][105692] Updated weights for policy 0, policy_version 349688 (0.0010) [2023-12-26 17:54:37,378][105692] Updated weights for policy 0, policy_version 349698 (0.0010) [2023-12-26 17:54:37,639][105620] Updated weights for policy 1, policy_version 350188 (0.0009) [2023-12-26 17:54:37,698][105620] Updated weights for policy 1, policy_version 350198 (0.0010) [2023-12-26 17:54:37,760][105620] Updated weights for policy 1, policy_version 350208 (0.0010) [2023-12-26 17:54:38,130][105692] Updated weights for policy 0, policy_version 349708 (0.0009) [2023-12-26 17:54:38,182][105692] Updated weights for policy 0, policy_version 349718 (0.0008) [2023-12-26 17:54:38,230][105692] Updated weights for policy 0, policy_version 349728 (0.0008) [2023-12-26 17:54:38,522][105620] Updated weights for policy 1, policy_version 350218 (0.0009) [2023-12-26 17:54:38,582][105620] Updated weights for policy 1, policy_version 350228 (0.0010) [2023-12-26 17:54:38,644][105620] Updated weights for policy 1, policy_version 350238 (0.0010) [2023-12-26 17:54:38,703][105620] Updated weights for policy 1, policy_version 350248 (0.0010) [2023-12-26 17:54:39,026][105692] Updated weights for policy 0, policy_version 349738 (0.0008) [2023-12-26 17:54:39,077][105692] Updated weights for policy 0, policy_version 349748 (0.0008) [2023-12-26 17:54:39,143][105692] Updated weights for policy 0, policy_version 349758 (0.0008) [2023-12-26 17:54:39,204][105692] Updated weights for policy 0, policy_version 349768 (0.0008) [2023-12-26 17:54:39,493][105620] Updated weights for policy 1, policy_version 350258 (0.0006) [2023-12-26 17:54:39,535][105586] KL-divergence is very high: 103.5948 [2023-12-26 17:54:39,542][105586] KL-divergence is very high: 113.3639 [2023-12-26 17:54:39,549][105586] KL-divergence is very high: 103.4052 [2023-12-26 17:54:39,563][105620] Updated weights for policy 1, policy_version 350268 (0.0010) [2023-12-26 17:54:39,589][105586] KL-divergence is very high: 108.1169 [2023-12-26 17:54:39,595][105586] KL-divergence is very high: 112.1819 [2023-12-26 17:54:39,624][105620] Updated weights for policy 1, policy_version 350278 (0.0006) [2023-12-26 17:54:40,012][105692] Updated weights for policy 0, policy_version 349778 (0.0008) [2023-12-26 17:54:40,082][105692] Updated weights for policy 0, policy_version 349788 (0.0008) [2023-12-26 17:54:40,088][105585] KL-divergence is very high: 116.0476 [2023-12-26 17:54:40,148][105692] Updated weights for policy 0, policy_version 349798 (0.0008) [2023-12-26 17:54:40,296][105620] Updated weights for policy 1, policy_version 350288 (0.0008) [2023-12-26 17:54:40,365][105620] Updated weights for policy 1, policy_version 350298 (0.0009) [2023-12-26 17:54:40,425][105620] Updated weights for policy 1, policy_version 350308 (0.0008) [2023-12-26 17:54:40,910][105692] Updated weights for policy 0, policy_version 349808 (0.0009) [2023-12-26 17:54:40,973][105692] Updated weights for policy 0, policy_version 349818 (0.0009) [2023-12-26 17:54:41,035][105692] Updated weights for policy 0, policy_version 349828 (0.0009) [2023-12-26 17:54:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 179257344. Throughput: 0: 9198.7, 1: 10051.0. Samples: 179265872. Policy #0 lag: (min: 1.0, avg: 20.9, max: 33.0) [2023-12-26 17:54:41,062][104569] Avg episode reward: [(0, '3547.599'), (1, '1066.232')] [2023-12-26 17:54:41,096][105620] Updated weights for policy 1, policy_version 350318 (0.0008) [2023-12-26 17:54:41,166][105620] Updated weights for policy 1, policy_version 350328 (0.0008) [2023-12-26 17:54:41,204][105586] KL-divergence is very high: 112.3719 [2023-12-26 17:54:41,211][105586] KL-divergence is very high: 110.0005 [2023-12-26 17:54:41,230][105620] Updated weights for policy 1, policy_version 350338 (0.0006) [2023-12-26 17:54:41,754][105692] Updated weights for policy 0, policy_version 349838 (0.0010) [2023-12-26 17:54:41,810][105692] Updated weights for policy 0, policy_version 349848 (0.0009) [2023-12-26 17:54:41,869][105692] Updated weights for policy 0, policy_version 349858 (0.0009) [2023-12-26 17:54:41,955][105620] Updated weights for policy 1, policy_version 350348 (0.0008) [2023-12-26 17:54:42,018][105620] Updated weights for policy 1, policy_version 350358 (0.0009) [2023-12-26 17:54:42,081][105620] Updated weights for policy 1, policy_version 350368 (0.0009) [2023-12-26 17:54:42,628][105692] Updated weights for policy 0, policy_version 349868 (0.0009) [2023-12-26 17:54:42,680][105692] Updated weights for policy 0, policy_version 349878 (0.0010) [2023-12-26 17:54:42,732][105692] Updated weights for policy 0, policy_version 349888 (0.0010) [2023-12-26 17:54:42,859][105620] Updated weights for policy 1, policy_version 350378 (0.0009) [2023-12-26 17:54:42,915][105620] Updated weights for policy 1, policy_version 350388 (0.0008) [2023-12-26 17:54:42,959][105620] Updated weights for policy 1, policy_version 350398 (0.0007) [2023-12-26 17:54:43,015][105620] Updated weights for policy 1, policy_version 350408 (0.0008) [2023-12-26 17:54:43,406][105692] Updated weights for policy 0, policy_version 349898 (0.0009) [2023-12-26 17:54:43,462][105692] Updated weights for policy 0, policy_version 349908 (0.0005) [2023-12-26 17:54:43,526][105692] Updated weights for policy 0, policy_version 349918 (0.0009) [2023-12-26 17:54:43,586][105692] Updated weights for policy 0, policy_version 349928 (0.0011) [2023-12-26 17:54:43,687][105620] Updated weights for policy 1, policy_version 350418 (0.0008) [2023-12-26 17:54:43,751][105620] Updated weights for policy 1, policy_version 350428 (0.0008) [2023-12-26 17:54:43,810][105620] Updated weights for policy 1, policy_version 350438 (0.0008) [2023-12-26 17:54:44,292][105692] Updated weights for policy 0, policy_version 349938 (0.0008) [2023-12-26 17:54:44,348][105692] Updated weights for policy 0, policy_version 349948 (0.0005) [2023-12-26 17:54:44,406][105692] Updated weights for policy 0, policy_version 349958 (0.0005) [2023-12-26 17:54:44,523][105620] Updated weights for policy 1, policy_version 350448 (0.0006) [2023-12-26 17:54:44,587][105620] Updated weights for policy 1, policy_version 350458 (0.0007) [2023-12-26 17:54:44,653][105620] Updated weights for policy 1, policy_version 350468 (0.0010) [2023-12-26 17:54:45,076][105692] Updated weights for policy 0, policy_version 349968 (0.0006) [2023-12-26 17:54:45,138][105692] Updated weights for policy 0, policy_version 349978 (0.0009) [2023-12-26 17:54:45,195][105692] Updated weights for policy 0, policy_version 349988 (0.0009) [2023-12-26 17:54:45,376][105620] Updated weights for policy 1, policy_version 350478 (0.0010) [2023-12-26 17:54:45,438][105620] Updated weights for policy 1, policy_version 350488 (0.0009) [2023-12-26 17:54:45,500][105620] Updated weights for policy 1, policy_version 350498 (0.0010) [2023-12-26 17:54:45,794][105692] Updated weights for policy 0, policy_version 349998 (0.0005) [2023-12-26 17:54:45,845][105692] Updated weights for policy 0, policy_version 350008 (0.0005) [2023-12-26 17:54:45,911][105692] Updated weights for policy 0, policy_version 350018 (0.0005) [2023-12-26 17:54:46,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 179355648. Throughput: 0: 9221.6, 1: 10027.3. Samples: 179323360. Policy #0 lag: (min: 1.0, avg: 20.9, max: 33.0) [2023-12-26 17:54:46,062][104569] Avg episode reward: [(0, '6412.377'), (1, '1329.139')] [2023-12-26 17:54:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000350024_89620480.pth... [2023-12-26 17:54:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000350504_89735168.pth... [2023-12-26 17:54:46,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000348904_89333760.pth [2023-12-26 17:54:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000349352_89440256.pth [2023-12-26 17:54:46,278][105620] Updated weights for policy 1, policy_version 350508 (0.0010) [2023-12-26 17:54:46,332][105620] Updated weights for policy 1, policy_version 350518 (0.0010) [2023-12-26 17:54:46,383][105620] Updated weights for policy 1, policy_version 350529 (0.0009) [2023-12-26 17:54:46,491][105692] Updated weights for policy 0, policy_version 350028 (0.0007) [2023-12-26 17:54:46,541][105692] Updated weights for policy 0, policy_version 350038 (0.0006) [2023-12-26 17:54:46,587][105692] Updated weights for policy 0, policy_version 350048 (0.0005) [2023-12-26 17:54:47,096][105620] Updated weights for policy 1, policy_version 350539 (0.0008) [2023-12-26 17:54:47,148][105620] Updated weights for policy 1, policy_version 350549 (0.0008) [2023-12-26 17:54:47,203][105620] Updated weights for policy 1, policy_version 350559 (0.0008) [2023-12-26 17:54:47,271][105692] Updated weights for policy 0, policy_version 350058 (0.0006) [2023-12-26 17:54:47,328][105692] Updated weights for policy 0, policy_version 350068 (0.0010) [2023-12-26 17:54:47,386][105692] Updated weights for policy 0, policy_version 350078 (0.0010) [2023-12-26 17:54:47,438][105692] Updated weights for policy 0, policy_version 350088 (0.0010) [2023-12-26 17:54:47,915][105620] Updated weights for policy 1, policy_version 350569 (0.0008) [2023-12-26 17:54:47,973][105620] Updated weights for policy 1, policy_version 350579 (0.0010) [2023-12-26 17:54:48,022][105620] Updated weights for policy 1, policy_version 350589 (0.0009) [2023-12-26 17:54:48,063][105692] Updated weights for policy 0, policy_version 350098 (0.0005) [2023-12-26 17:54:48,092][105620] Updated weights for policy 1, policy_version 350599 (0.0010) [2023-12-26 17:54:48,125][105692] Updated weights for policy 0, policy_version 350108 (0.0007) [2023-12-26 17:54:48,177][105692] Updated weights for policy 0, policy_version 350118 (0.0006) [2023-12-26 17:54:48,780][105692] Updated weights for policy 0, policy_version 350128 (0.0008) [2023-12-26 17:54:48,828][105692] Updated weights for policy 0, policy_version 350138 (0.0009) [2023-12-26 17:54:48,881][105692] Updated weights for policy 0, policy_version 350148 (0.0006) [2023-12-26 17:54:48,887][105620] Updated weights for policy 1, policy_version 350609 (0.0007) [2023-12-26 17:54:48,936][105620] Updated weights for policy 1, policy_version 350619 (0.0007) [2023-12-26 17:54:48,986][105620] Updated weights for policy 1, policy_version 350629 (0.0009) [2023-12-26 17:54:49,672][105692] Updated weights for policy 0, policy_version 350158 (0.0008) [2023-12-26 17:54:49,729][105692] Updated weights for policy 0, policy_version 350168 (0.0010) [2023-12-26 17:54:49,763][105620] Updated weights for policy 1, policy_version 350639 (0.0008) [2023-12-26 17:54:49,786][105692] Updated weights for policy 0, policy_version 350178 (0.0009) [2023-12-26 17:54:49,829][105620] Updated weights for policy 1, policy_version 350649 (0.0007) [2023-12-26 17:54:49,895][105620] Updated weights for policy 1, policy_version 350659 (0.0008) [2023-12-26 17:54:50,563][105692] Updated weights for policy 0, policy_version 350188 (0.0009) [2023-12-26 17:54:50,630][105692] Updated weights for policy 0, policy_version 350198 (0.0009) [2023-12-26 17:54:50,641][105620] Updated weights for policy 1, policy_version 350669 (0.0008) [2023-12-26 17:54:50,697][105692] Updated weights for policy 0, policy_version 350208 (0.0008) [2023-12-26 17:54:50,708][105620] Updated weights for policy 1, policy_version 350679 (0.0006) [2023-12-26 17:54:50,774][105620] Updated weights for policy 1, policy_version 350689 (0.0007) [2023-12-26 17:54:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 179453952. Throughput: 0: 9363.9, 1: 9988.1. Samples: 179443064. Policy #0 lag: (min: 1.0, avg: 20.9, max: 33.0) [2023-12-26 17:54:51,062][104569] Avg episode reward: [(0, '7373.058'), (1, '5378.062')] [2023-12-26 17:54:51,451][105692] Updated weights for policy 0, policy_version 350218 (0.0008) [2023-12-26 17:54:51,504][105620] Updated weights for policy 1, policy_version 350699 (0.0009) [2023-12-26 17:54:51,515][105692] Updated weights for policy 0, policy_version 350228 (0.0006) [2023-12-26 17:54:51,565][105620] Updated weights for policy 1, policy_version 350709 (0.0008) [2023-12-26 17:54:51,580][105692] Updated weights for policy 0, policy_version 350238 (0.0008) [2023-12-26 17:54:51,633][105620] Updated weights for policy 1, policy_version 350719 (0.0007) [2023-12-26 17:54:51,646][105692] Updated weights for policy 0, policy_version 350248 (0.0008) [2023-12-26 17:54:52,321][105692] Updated weights for policy 0, policy_version 350258 (0.0009) [2023-12-26 17:54:52,358][105620] Updated weights for policy 1, policy_version 350729 (0.0009) [2023-12-26 17:54:52,385][105692] Updated weights for policy 0, policy_version 350268 (0.0008) [2023-12-26 17:54:52,426][105620] Updated weights for policy 1, policy_version 350739 (0.0007) [2023-12-26 17:54:52,440][105692] Updated weights for policy 0, policy_version 350278 (0.0008) [2023-12-26 17:54:52,487][105620] Updated weights for policy 1, policy_version 350749 (0.0006) [2023-12-26 17:54:52,548][105620] Updated weights for policy 1, policy_version 350759 (0.0005) [2023-12-26 17:54:53,193][105692] Updated weights for policy 0, policy_version 350288 (0.0006) [2023-12-26 17:54:53,253][105692] Updated weights for policy 0, policy_version 350298 (0.0008) [2023-12-26 17:54:53,275][105620] Updated weights for policy 1, policy_version 350769 (0.0006) [2023-12-26 17:54:53,318][105692] Updated weights for policy 0, policy_version 350308 (0.0008) [2023-12-26 17:54:53,324][105620] Updated weights for policy 1, policy_version 350779 (0.0007) [2023-12-26 17:54:53,380][105620] Updated weights for policy 1, policy_version 350789 (0.0008) [2023-12-26 17:54:53,848][105692] Updated weights for policy 0, policy_version 350318 (0.0006) [2023-12-26 17:54:53,904][105692] Updated weights for policy 0, policy_version 350328 (0.0005) [2023-12-26 17:54:53,959][105692] Updated weights for policy 0, policy_version 350338 (0.0005) [2023-12-26 17:54:54,167][105620] Updated weights for policy 1, policy_version 350799 (0.0010) [2023-12-26 17:54:54,220][105620] Updated weights for policy 1, policy_version 350809 (0.0009) [2023-12-26 17:54:54,271][105620] Updated weights for policy 1, policy_version 350820 (0.0010) [2023-12-26 17:54:54,472][105692] Updated weights for policy 0, policy_version 350348 (0.0005) [2023-12-26 17:54:54,532][105692] Updated weights for policy 0, policy_version 350358 (0.0005) [2023-12-26 17:54:54,583][105692] Updated weights for policy 0, policy_version 350368 (0.0005) [2023-12-26 17:54:54,989][105620] Updated weights for policy 1, policy_version 350830 (0.0009) [2023-12-26 17:54:55,048][105620] Updated weights for policy 1, policy_version 350840 (0.0008) [2023-12-26 17:54:55,100][105620] Updated weights for policy 1, policy_version 350850 (0.0008) [2023-12-26 17:54:55,279][105692] Updated weights for policy 0, policy_version 350378 (0.0007) [2023-12-26 17:54:55,330][105692] Updated weights for policy 0, policy_version 350388 (0.0010) [2023-12-26 17:54:55,375][105692] Updated weights for policy 0, policy_version 350398 (0.0010) [2023-12-26 17:54:55,433][105692] Updated weights for policy 0, policy_version 350408 (0.0010) [2023-12-26 17:54:55,881][105620] Updated weights for policy 1, policy_version 350860 (0.0008) [2023-12-26 17:54:55,927][105620] Updated weights for policy 1, policy_version 350870 (0.0008) [2023-12-26 17:54:55,974][105620] Updated weights for policy 1, policy_version 350880 (0.0008) [2023-12-26 17:54:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19388.0, 300 sec: 19494.2). Total num frames: 179552256. Throughput: 0: 9535.0, 1: 9848.1. Samples: 179560140. Policy #0 lag: (min: 1.0, avg: 20.9, max: 33.0) [2023-12-26 17:54:56,063][104569] Avg episode reward: [(0, '7570.748'), (1, '3774.450')] [2023-12-26 17:54:56,193][105692] Updated weights for policy 0, policy_version 350418 (0.0010) [2023-12-26 17:54:56,250][105692] Updated weights for policy 0, policy_version 350428 (0.0010) [2023-12-26 17:54:56,304][105692] Updated weights for policy 0, policy_version 350438 (0.0010) [2023-12-26 17:54:56,711][105620] Updated weights for policy 1, policy_version 350890 (0.0008) [2023-12-26 17:54:56,755][105620] Updated weights for policy 1, policy_version 350900 (0.0010) [2023-12-26 17:54:56,806][105620] Updated weights for policy 1, policy_version 350910 (0.0010) [2023-12-26 17:54:56,857][105620] Updated weights for policy 1, policy_version 350920 (0.0010) [2023-12-26 17:54:57,042][105692] Updated weights for policy 0, policy_version 350448 (0.0006) [2023-12-26 17:54:57,096][105692] Updated weights for policy 0, policy_version 350458 (0.0005) [2023-12-26 17:54:57,147][105692] Updated weights for policy 0, policy_version 350468 (0.0005) [2023-12-26 17:54:57,506][105620] Updated weights for policy 1, policy_version 350930 (0.0010) [2023-12-26 17:54:57,570][105620] Updated weights for policy 1, policy_version 350940 (0.0010) [2023-12-26 17:54:57,628][105620] Updated weights for policy 1, policy_version 350950 (0.0010) [2023-12-26 17:54:57,828][105692] Updated weights for policy 0, policy_version 350478 (0.0007) [2023-12-26 17:54:57,892][105692] Updated weights for policy 0, policy_version 350488 (0.0008) [2023-12-26 17:54:57,951][105692] Updated weights for policy 0, policy_version 350498 (0.0009) [2023-12-26 17:54:58,307][105620] Updated weights for policy 1, policy_version 350960 (0.0009) [2023-12-26 17:54:58,376][105620] Updated weights for policy 1, policy_version 350970 (0.0009) [2023-12-26 17:54:58,445][105620] Updated weights for policy 1, policy_version 350980 (0.0009) [2023-12-26 17:54:58,790][105692] Updated weights for policy 0, policy_version 350508 (0.0008) [2023-12-26 17:54:58,848][105692] Updated weights for policy 0, policy_version 350518 (0.0008) [2023-12-26 17:54:58,909][105692] Updated weights for policy 0, policy_version 350528 (0.0008) [2023-12-26 17:54:59,149][105620] Updated weights for policy 1, policy_version 350990 (0.0007) [2023-12-26 17:54:59,197][105620] Updated weights for policy 1, policy_version 351001 (0.0007) [2023-12-26 17:54:59,254][105620] Updated weights for policy 1, policy_version 351011 (0.0008) [2023-12-26 17:54:59,503][105692] Updated weights for policy 0, policy_version 350538 (0.0006) [2023-12-26 17:54:59,554][105692] Updated weights for policy 0, policy_version 350548 (0.0007) [2023-12-26 17:54:59,573][105585] KL-divergence is very high: 156.7300 [2023-12-26 17:54:59,595][105585] KL-divergence is very high: 277.9612 [2023-12-26 17:54:59,607][105692] Updated weights for policy 0, policy_version 350558 (0.0005) [2023-12-26 17:54:59,618][105585] KL-divergence is very high: 307.9783 [2023-12-26 17:54:59,641][105585] KL-divergence is very high: 359.8143 [2023-12-26 17:54:59,664][105692] Updated weights for policy 0, policy_version 350568 (0.0005) [2023-12-26 17:54:59,970][105620] Updated weights for policy 1, policy_version 351021 (0.0007) [2023-12-26 17:55:00,021][105620] Updated weights for policy 1, policy_version 351031 (0.0009) [2023-12-26 17:55:00,069][105620] Updated weights for policy 1, policy_version 351041 (0.0009) [2023-12-26 17:55:00,346][105692] Updated weights for policy 0, policy_version 350578 (0.0010) [2023-12-26 17:55:00,400][105692] Updated weights for policy 0, policy_version 350588 (0.0010) [2023-12-26 17:55:00,453][105692] Updated weights for policy 0, policy_version 350599 (0.0010) [2023-12-26 17:55:00,806][105620] Updated weights for policy 1, policy_version 351052 (0.0009) [2023-12-26 17:55:00,858][105620] Updated weights for policy 1, policy_version 351062 (0.0008) [2023-12-26 17:55:00,913][105620] Updated weights for policy 1, policy_version 351072 (0.0008) [2023-12-26 17:55:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 179650560. Throughput: 0: 9594.4, 1: 9865.4. Samples: 179618212. Policy #0 lag: (min: 1.0, avg: 20.9, max: 33.0) [2023-12-26 17:55:01,062][104569] Avg episode reward: [(0, '8280.119'), (1, '6529.628')] [2023-12-26 17:55:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000350600_89767936.pth... [2023-12-26 17:55:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000351080_89882624.pth... [2023-12-26 17:55:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000349448_89473024.pth [2023-12-26 17:55:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000349928_89587712.pth [2023-12-26 17:55:01,253][105692] Updated weights for policy 0, policy_version 350609 (0.0011) [2023-12-26 17:55:01,315][105692] Updated weights for policy 0, policy_version 350619 (0.0010) [2023-12-26 17:55:01,382][105692] Updated weights for policy 0, policy_version 350629 (0.0010) [2023-12-26 17:55:01,696][105620] Updated weights for policy 1, policy_version 351082 (0.0008) [2023-12-26 17:55:01,754][105620] Updated weights for policy 1, policy_version 351092 (0.0011) [2023-12-26 17:55:01,816][105620] Updated weights for policy 1, policy_version 351102 (0.0010) [2023-12-26 17:55:01,864][105620] Updated weights for policy 1, policy_version 351112 (0.0010) [2023-12-26 17:55:02,004][105692] Updated weights for policy 0, policy_version 350639 (0.0007) [2023-12-26 17:55:02,064][105692] Updated weights for policy 0, policy_version 350649 (0.0005) [2023-12-26 17:55:02,121][105692] Updated weights for policy 0, policy_version 350659 (0.0006) [2023-12-26 17:55:02,583][105620] Updated weights for policy 1, policy_version 351122 (0.0006) [2023-12-26 17:55:02,640][105620] Updated weights for policy 1, policy_version 351132 (0.0006) [2023-12-26 17:55:02,702][105620] Updated weights for policy 1, policy_version 351142 (0.0008) [2023-12-26 17:55:02,715][105692] Updated weights for policy 0, policy_version 350669 (0.0008) [2023-12-26 17:55:02,777][105692] Updated weights for policy 0, policy_version 350679 (0.0005) [2023-12-26 17:55:02,831][105692] Updated weights for policy 0, policy_version 350689 (0.0005) [2023-12-26 17:55:03,346][105692] Updated weights for policy 0, policy_version 350699 (0.0005) [2023-12-26 17:55:03,416][105692] Updated weights for policy 0, policy_version 350709 (0.0006) [2023-12-26 17:55:03,469][105692] Updated weights for policy 0, policy_version 350719 (0.0005) [2023-12-26 17:55:03,515][105620] Updated weights for policy 1, policy_version 351152 (0.0007) [2023-12-26 17:55:03,580][105620] Updated weights for policy 1, policy_version 351162 (0.0005) [2023-12-26 17:55:03,648][105620] Updated weights for policy 1, policy_version 351172 (0.0005) [2023-12-26 17:55:03,999][105692] Updated weights for policy 0, policy_version 350729 (0.0006) [2023-12-26 17:55:04,044][105692] Updated weights for policy 0, policy_version 350739 (0.0011) [2023-12-26 17:55:04,090][105692] Updated weights for policy 0, policy_version 350749 (0.0010) [2023-12-26 17:55:04,140][105692] Updated weights for policy 0, policy_version 350759 (0.0010) [2023-12-26 17:55:04,216][105620] Updated weights for policy 1, policy_version 351182 (0.0005) [2023-12-26 17:55:04,283][105620] Updated weights for policy 1, policy_version 351192 (0.0008) [2023-12-26 17:55:04,350][105620] Updated weights for policy 1, policy_version 351202 (0.0010) [2023-12-26 17:55:04,941][105692] Updated weights for policy 0, policy_version 350769 (0.0010) [2023-12-26 17:55:04,986][105620] Updated weights for policy 1, policy_version 351212 (0.0008) [2023-12-26 17:55:04,999][105692] Updated weights for policy 0, policy_version 350779 (0.0010) [2023-12-26 17:55:05,043][105620] Updated weights for policy 1, policy_version 351222 (0.0006) [2023-12-26 17:55:05,055][105692] Updated weights for policy 0, policy_version 350789 (0.0011) [2023-12-26 17:55:05,091][105620] Updated weights for policy 1, policy_version 351232 (0.0005) [2023-12-26 17:55:05,775][105620] Updated weights for policy 1, policy_version 351242 (0.0006) [2023-12-26 17:55:05,802][105692] Updated weights for policy 0, policy_version 350799 (0.0010) [2023-12-26 17:55:05,827][105620] Updated weights for policy 1, policy_version 351252 (0.0007) [2023-12-26 17:55:05,867][105692] Updated weights for policy 0, policy_version 350809 (0.0006) [2023-12-26 17:55:05,886][105620] Updated weights for policy 1, policy_version 351262 (0.0010) [2023-12-26 17:55:05,919][105692] Updated weights for policy 0, policy_version 350819 (0.0006) [2023-12-26 17:55:05,935][105620] Updated weights for policy 1, policy_version 351272 (0.0010) [2023-12-26 17:55:06,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 179757056. Throughput: 0: 9854.9, 1: 9822.1. Samples: 179741392. Policy #0 lag: (min: 1.0, avg: 20.9, max: 33.0) [2023-12-26 17:55:06,062][104569] Avg episode reward: [(0, '8904.412'), (1, '8810.510')] [2023-12-26 17:55:06,587][105620] Updated weights for policy 1, policy_version 351282 (0.0006) [2023-12-26 17:55:06,636][105692] Updated weights for policy 0, policy_version 350829 (0.0011) [2023-12-26 17:55:06,656][105620] Updated weights for policy 1, policy_version 351292 (0.0008) [2023-12-26 17:55:06,703][105692] Updated weights for policy 0, policy_version 350839 (0.0011) [2023-12-26 17:55:06,715][105620] Updated weights for policy 1, policy_version 351302 (0.0007) [2023-12-26 17:55:06,770][105692] Updated weights for policy 0, policy_version 350849 (0.0011) [2023-12-26 17:55:07,396][105620] Updated weights for policy 1, policy_version 351312 (0.0007) [2023-12-26 17:55:07,446][105620] Updated weights for policy 1, policy_version 351322 (0.0005) [2023-12-26 17:55:07,493][105620] Updated weights for policy 1, policy_version 351332 (0.0005) [2023-12-26 17:55:07,500][105692] Updated weights for policy 0, policy_version 350859 (0.0011) [2023-12-26 17:55:07,548][105692] Updated weights for policy 0, policy_version 350869 (0.0010) [2023-12-26 17:55:07,597][105692] Updated weights for policy 0, policy_version 350879 (0.0010) [2023-12-26 17:55:08,177][105620] Updated weights for policy 1, policy_version 351342 (0.0008) [2023-12-26 17:55:08,225][105620] Updated weights for policy 1, policy_version 351352 (0.0010) [2023-12-26 17:55:08,272][105620] Updated weights for policy 1, policy_version 351362 (0.0010) [2023-12-26 17:55:08,380][105692] Updated weights for policy 0, policy_version 350889 (0.0010) [2023-12-26 17:55:08,439][105692] Updated weights for policy 0, policy_version 350899 (0.0011) [2023-12-26 17:55:08,494][105692] Updated weights for policy 0, policy_version 350909 (0.0010) [2023-12-26 17:55:08,556][105692] Updated weights for policy 0, policy_version 350919 (0.0010) [2023-12-26 17:55:09,026][105620] Updated weights for policy 1, policy_version 351372 (0.0008) [2023-12-26 17:55:09,084][105620] Updated weights for policy 1, policy_version 351382 (0.0008) [2023-12-26 17:55:09,143][105620] Updated weights for policy 1, policy_version 351392 (0.0008) [2023-12-26 17:55:09,310][105692] Updated weights for policy 0, policy_version 350929 (0.0011) [2023-12-26 17:55:09,377][105692] Updated weights for policy 0, policy_version 350939 (0.0009) [2023-12-26 17:55:09,445][105692] Updated weights for policy 0, policy_version 350949 (0.0007) [2023-12-26 17:55:09,880][105620] Updated weights for policy 1, policy_version 351402 (0.0008) [2023-12-26 17:55:09,945][105620] Updated weights for policy 1, policy_version 351412 (0.0009) [2023-12-26 17:55:10,004][105620] Updated weights for policy 1, policy_version 351422 (0.0010) [2023-12-26 17:55:10,060][105620] Updated weights for policy 1, policy_version 351432 (0.0010) [2023-12-26 17:55:10,223][105692] Updated weights for policy 0, policy_version 350959 (0.0006) [2023-12-26 17:55:10,291][105692] Updated weights for policy 0, policy_version 350969 (0.0006) [2023-12-26 17:55:10,364][105692] Updated weights for policy 0, policy_version 350979 (0.0008) [2023-12-26 17:55:10,802][105620] Updated weights for policy 1, policy_version 351442 (0.0010) [2023-12-26 17:55:10,862][105620] Updated weights for policy 1, policy_version 351452 (0.0010) [2023-12-26 17:55:10,918][105620] Updated weights for policy 1, policy_version 351462 (0.0007) [2023-12-26 17:55:11,034][105692] Updated weights for policy 0, policy_version 350989 (0.0009) [2023-12-26 17:55:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 179847168. Throughput: 0: 9832.5, 1: 9781.5. Samples: 179856528. Policy #0 lag: (min: 1.0, avg: 20.9, max: 33.0) [2023-12-26 17:55:11,062][104569] Avg episode reward: [(0, '8993.845'), (1, '9046.878')] [2023-12-26 17:55:11,105][105692] Updated weights for policy 0, policy_version 350999 (0.0009) [2023-12-26 17:55:11,177][105692] Updated weights for policy 0, policy_version 351009 (0.0009) [2023-12-26 17:55:11,671][105620] Updated weights for policy 1, policy_version 351472 (0.0009) [2023-12-26 17:55:11,732][105620] Updated weights for policy 1, policy_version 351482 (0.0007) [2023-12-26 17:55:11,795][105620] Updated weights for policy 1, policy_version 351492 (0.0008) [2023-12-26 17:55:11,921][105692] Updated weights for policy 0, policy_version 351019 (0.0008) [2023-12-26 17:55:11,979][105692] Updated weights for policy 0, policy_version 351029 (0.0010) [2023-12-26 17:55:12,045][105692] Updated weights for policy 0, policy_version 351039 (0.0010) [2023-12-26 17:55:12,501][105620] Updated weights for policy 1, policy_version 351502 (0.0007) [2023-12-26 17:55:12,562][105620] Updated weights for policy 1, policy_version 351512 (0.0008) [2023-12-26 17:55:12,622][105620] Updated weights for policy 1, policy_version 351522 (0.0009) [2023-12-26 17:55:12,809][105692] Updated weights for policy 0, policy_version 351049 (0.0010) [2023-12-26 17:55:12,867][105692] Updated weights for policy 0, policy_version 351059 (0.0009) [2023-12-26 17:55:12,925][105692] Updated weights for policy 0, policy_version 351069 (0.0009) [2023-12-26 17:55:12,980][105692] Updated weights for policy 0, policy_version 351079 (0.0009) [2023-12-26 17:55:13,361][105620] Updated weights for policy 1, policy_version 351532 (0.0009) [2023-12-26 17:55:13,418][105620] Updated weights for policy 1, policy_version 351542 (0.0009) [2023-12-26 17:55:13,477][105620] Updated weights for policy 1, policy_version 351552 (0.0010) [2023-12-26 17:55:13,659][105692] Updated weights for policy 0, policy_version 351089 (0.0007) [2023-12-26 17:55:13,722][105692] Updated weights for policy 0, policy_version 351099 (0.0009) [2023-12-26 17:55:13,772][105692] Updated weights for policy 0, policy_version 351109 (0.0010) [2023-12-26 17:55:14,331][105620] Updated weights for policy 1, policy_version 351562 (0.0010) [2023-12-26 17:55:14,385][105620] Updated weights for policy 1, policy_version 351572 (0.0010) [2023-12-26 17:55:14,439][105620] Updated weights for policy 1, policy_version 351582 (0.0009) [2023-12-26 17:55:14,450][105692] Updated weights for policy 0, policy_version 351119 (0.0008) [2023-12-26 17:55:14,484][105620] Updated weights for policy 1, policy_version 351592 (0.0006) [2023-12-26 17:55:14,503][105692] Updated weights for policy 0, policy_version 351129 (0.0010) [2023-12-26 17:55:14,553][105692] Updated weights for policy 0, policy_version 351139 (0.0007) [2023-12-26 17:55:15,150][105620] Updated weights for policy 1, policy_version 351602 (0.0009) [2023-12-26 17:55:15,214][105620] Updated weights for policy 1, policy_version 351612 (0.0011) [2023-12-26 17:55:15,242][105692] Updated weights for policy 0, policy_version 351149 (0.0009) [2023-12-26 17:55:15,271][105620] Updated weights for policy 1, policy_version 351622 (0.0011) [2023-12-26 17:55:15,295][105692] Updated weights for policy 0, policy_version 351159 (0.0009) [2023-12-26 17:55:15,348][105692] Updated weights for policy 0, policy_version 351169 (0.0006) [2023-12-26 17:55:16,019][105620] Updated weights for policy 1, policy_version 351632 (0.0011) [2023-12-26 17:55:16,038][105692] Updated weights for policy 0, policy_version 351179 (0.0005) [2023-12-26 17:55:16,062][104569] Fps is (10 sec: 18021.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 179937280. Throughput: 0: 9849.1, 1: 9717.9. Samples: 179912364. Policy #0 lag: (min: 1.0, avg: 20.9, max: 33.0) [2023-12-26 17:55:16,063][104569] Avg episode reward: [(0, '9175.194'), (1, '9261.103')] [2023-12-26 17:55:16,072][105620] Updated weights for policy 1, policy_version 351642 (0.0010) [2023-12-26 17:55:16,095][105692] Updated weights for policy 0, policy_version 351189 (0.0005) [2023-12-26 17:55:16,128][105620] Updated weights for policy 1, policy_version 351652 (0.0011) [2023-12-26 17:55:16,150][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000351656_90030080.pth... [2023-12-26 17:55:16,153][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000350504_89735168.pth [2023-12-26 17:55:16,159][105692] Updated weights for policy 0, policy_version 351199 (0.0005) [2023-12-26 17:55:16,212][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000351208_89923584.pth... [2023-12-26 17:55:16,217][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000350024_89620480.pth [2023-12-26 17:55:16,665][105692] Updated weights for policy 0, policy_version 351209 (0.0006) [2023-12-26 17:55:16,721][105692] Updated weights for policy 0, policy_version 351219 (0.0005) [2023-12-26 17:55:16,773][105692] Updated weights for policy 0, policy_version 351229 (0.0009) [2023-12-26 17:55:16,821][105692] Updated weights for policy 0, policy_version 351239 (0.0010) [2023-12-26 17:55:16,841][105620] Updated weights for policy 1, policy_version 351662 (0.0006) [2023-12-26 17:55:16,905][105620] Updated weights for policy 1, policy_version 351672 (0.0005) [2023-12-26 17:55:16,960][105620] Updated weights for policy 1, policy_version 351682 (0.0005) [2023-12-26 17:55:17,538][105692] Updated weights for policy 0, policy_version 351249 (0.0010) [2023-12-26 17:55:17,552][105620] Updated weights for policy 1, policy_version 351692 (0.0007) [2023-12-26 17:55:17,596][105692] Updated weights for policy 0, policy_version 351259 (0.0010) [2023-12-26 17:55:17,614][105620] Updated weights for policy 1, policy_version 351702 (0.0006) [2023-12-26 17:55:17,651][105692] Updated weights for policy 0, policy_version 351269 (0.0010) [2023-12-26 17:55:17,670][105620] Updated weights for policy 1, policy_version 351712 (0.0006) [2023-12-26 17:55:18,221][105620] Updated weights for policy 1, policy_version 351722 (0.0005) [2023-12-26 17:55:18,281][105620] Updated weights for policy 1, policy_version 351732 (0.0005) [2023-12-26 17:55:18,336][105620] Updated weights for policy 1, policy_version 351742 (0.0007) [2023-12-26 17:55:18,396][105620] Updated weights for policy 1, policy_version 351752 (0.0008) [2023-12-26 17:55:18,407][105692] Updated weights for policy 0, policy_version 351279 (0.0010) [2023-12-26 17:55:18,469][105692] Updated weights for policy 0, policy_version 351289 (0.0010) [2023-12-26 17:55:18,532][105692] Updated weights for policy 0, policy_version 351299 (0.0010) [2023-12-26 17:55:19,102][105620] Updated weights for policy 1, policy_version 351762 (0.0010) [2023-12-26 17:55:19,164][105620] Updated weights for policy 1, policy_version 351772 (0.0007) [2023-12-26 17:55:19,224][105620] Updated weights for policy 1, policy_version 351782 (0.0006) [2023-12-26 17:55:19,269][105692] Updated weights for policy 0, policy_version 351309 (0.0008) [2023-12-26 17:55:19,341][105692] Updated weights for policy 0, policy_version 351319 (0.0006) [2023-12-26 17:55:19,405][105692] Updated weights for policy 0, policy_version 351329 (0.0010) [2023-12-26 17:55:19,902][105620] Updated weights for policy 1, policy_version 351792 (0.0010) [2023-12-26 17:55:19,966][105620] Updated weights for policy 1, policy_version 351802 (0.0011) [2023-12-26 17:55:20,036][105620] Updated weights for policy 1, policy_version 351812 (0.0011) [2023-12-26 17:55:20,136][105692] Updated weights for policy 0, policy_version 351339 (0.0010) [2023-12-26 17:55:20,195][105692] Updated weights for policy 0, policy_version 351349 (0.0010) [2023-12-26 17:55:20,262][105692] Updated weights for policy 0, policy_version 351359 (0.0010) [2023-12-26 17:55:20,620][105620] Updated weights for policy 1, policy_version 351822 (0.0010) [2023-12-26 17:55:20,681][105620] Updated weights for policy 1, policy_version 351832 (0.0009) [2023-12-26 17:55:20,741][105620] Updated weights for policy 1, policy_version 351842 (0.0011) [2023-12-26 17:55:20,944][105692] Updated weights for policy 0, policy_version 351369 (0.0010) [2023-12-26 17:55:21,002][105692] Updated weights for policy 0, policy_version 351379 (0.0005) [2023-12-26 17:55:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 180043776. Throughput: 0: 9925.6, 1: 9742.7. Samples: 180034544. Policy #0 lag: (min: 14.0, avg: 14.3, max: 22.0) [2023-12-26 17:55:21,062][104569] Avg episode reward: [(0, '8994.670'), (1, '9352.319')] [2023-12-26 17:55:21,069][105692] Updated weights for policy 0, policy_version 351389 (0.0011) [2023-12-26 17:55:21,130][105692] Updated weights for policy 0, policy_version 351399 (0.0008) [2023-12-26 17:55:21,542][105620] Updated weights for policy 1, policy_version 351852 (0.0010) [2023-12-26 17:55:21,594][105620] Updated weights for policy 1, policy_version 351862 (0.0009) [2023-12-26 17:55:21,663][105620] Updated weights for policy 1, policy_version 351872 (0.0009) [2023-12-26 17:55:21,833][105692] Updated weights for policy 0, policy_version 351409 (0.0011) [2023-12-26 17:55:21,890][105692] Updated weights for policy 0, policy_version 351419 (0.0010) [2023-12-26 17:55:21,939][105692] Updated weights for policy 0, policy_version 351429 (0.0010) [2023-12-26 17:55:22,467][105620] Updated weights for policy 1, policy_version 351882 (0.0009) [2023-12-26 17:55:22,533][105620] Updated weights for policy 1, policy_version 351892 (0.0008) [2023-12-26 17:55:22,591][105620] Updated weights for policy 1, policy_version 351902 (0.0009) [2023-12-26 17:55:22,657][105620] Updated weights for policy 1, policy_version 351912 (0.0008) [2023-12-26 17:55:22,712][105692] Updated weights for policy 0, policy_version 351439 (0.0011) [2023-12-26 17:55:22,772][105692] Updated weights for policy 0, policy_version 351449 (0.0008) [2023-12-26 17:55:22,832][105692] Updated weights for policy 0, policy_version 351459 (0.0006) [2023-12-26 17:55:23,450][105620] Updated weights for policy 1, policy_version 351922 (0.0011) [2023-12-26 17:55:23,453][105692] Updated weights for policy 0, policy_version 351469 (0.0008) [2023-12-26 17:55:23,500][105692] Updated weights for policy 0, policy_version 351479 (0.0010) [2023-12-26 17:55:23,515][105620] Updated weights for policy 1, policy_version 351932 (0.0010) [2023-12-26 17:55:23,548][105692] Updated weights for policy 0, policy_version 351489 (0.0010) [2023-12-26 17:55:23,574][105620] Updated weights for policy 1, policy_version 351942 (0.0008) [2023-12-26 17:55:24,188][105620] Updated weights for policy 1, policy_version 351952 (0.0008) [2023-12-26 17:55:24,248][105620] Updated weights for policy 1, policy_version 351962 (0.0007) [2023-12-26 17:55:24,272][105692] Updated weights for policy 0, policy_version 351499 (0.0010) [2023-12-26 17:55:24,307][105620] Updated weights for policy 1, policy_version 351972 (0.0009) [2023-12-26 17:55:24,323][105692] Updated weights for policy 0, policy_version 351509 (0.0010) [2023-12-26 17:55:24,388][105692] Updated weights for policy 0, policy_version 351519 (0.0010) [2023-12-26 17:55:24,949][105692] Updated weights for policy 0, policy_version 351529 (0.0008) [2023-12-26 17:55:25,006][105692] Updated weights for policy 0, policy_version 351539 (0.0006) [2023-12-26 17:55:25,050][105692] Updated weights for policy 0, policy_version 351549 (0.0010) [2023-12-26 17:55:25,098][105692] Updated weights for policy 0, policy_version 351559 (0.0010) [2023-12-26 17:55:25,124][105620] Updated weights for policy 1, policy_version 351982 (0.0007) [2023-12-26 17:55:25,181][105620] Updated weights for policy 1, policy_version 351992 (0.0007) [2023-12-26 17:55:25,258][105620] Updated weights for policy 1, policy_version 352002 (0.0005) [2023-12-26 17:55:25,772][105620] Updated weights for policy 1, policy_version 352012 (0.0006) [2023-12-26 17:55:25,794][105692] Updated weights for policy 0, policy_version 351569 (0.0006) [2023-12-26 17:55:25,838][105620] Updated weights for policy 1, policy_version 352022 (0.0005) [2023-12-26 17:55:25,853][105692] Updated weights for policy 0, policy_version 351579 (0.0006) [2023-12-26 17:55:25,893][105620] Updated weights for policy 1, policy_version 352032 (0.0008) [2023-12-26 17:55:25,906][105692] Updated weights for policy 0, policy_version 351589 (0.0005) [2023-12-26 17:55:26,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 180150272. Throughput: 0: 10016.3, 1: 9706.7. Samples: 180153412. Policy #0 lag: (min: 14.0, avg: 14.3, max: 22.0) [2023-12-26 17:55:26,063][104569] Avg episode reward: [(0, '8813.806'), (1, '9167.470')] [2023-12-26 17:55:26,555][105692] Updated weights for policy 0, policy_version 351599 (0.0009) [2023-12-26 17:55:26,612][105692] Updated weights for policy 0, policy_version 351609 (0.0010) [2023-12-26 17:55:26,631][105620] Updated weights for policy 1, policy_version 352043 (0.0008) [2023-12-26 17:55:26,673][105692] Updated weights for policy 0, policy_version 351619 (0.0010) [2023-12-26 17:55:26,683][105620] Updated weights for policy 1, policy_version 352053 (0.0005) [2023-12-26 17:55:26,736][105620] Updated weights for policy 1, policy_version 352063 (0.0007) [2023-12-26 17:55:27,323][105620] Updated weights for policy 1, policy_version 352073 (0.0008) [2023-12-26 17:55:27,378][105620] Updated weights for policy 1, policy_version 352083 (0.0006) [2023-12-26 17:55:27,408][105692] Updated weights for policy 0, policy_version 351629 (0.0010) [2023-12-26 17:55:27,434][105620] Updated weights for policy 1, policy_version 352093 (0.0007) [2023-12-26 17:55:27,459][105692] Updated weights for policy 0, policy_version 351639 (0.0010) [2023-12-26 17:55:27,485][105620] Updated weights for policy 1, policy_version 352103 (0.0006) [2023-12-26 17:55:27,516][105692] Updated weights for policy 0, policy_version 351649 (0.0010) [2023-12-26 17:55:28,150][105692] Updated weights for policy 0, policy_version 351659 (0.0010) [2023-12-26 17:55:28,176][105620] Updated weights for policy 1, policy_version 352113 (0.0007) [2023-12-26 17:55:28,208][105692] Updated weights for policy 0, policy_version 351669 (0.0010) [2023-12-26 17:55:28,232][105620] Updated weights for policy 1, policy_version 352123 (0.0008) [2023-12-26 17:55:28,268][105692] Updated weights for policy 0, policy_version 351679 (0.0011) [2023-12-26 17:55:28,289][105620] Updated weights for policy 1, policy_version 352133 (0.0009) [2023-12-26 17:55:29,020][105620] Updated weights for policy 1, policy_version 352143 (0.0007) [2023-12-26 17:55:29,038][105692] Updated weights for policy 0, policy_version 351689 (0.0011) [2023-12-26 17:55:29,094][105692] Updated weights for policy 0, policy_version 351699 (0.0011) [2023-12-26 17:55:29,096][105620] Updated weights for policy 1, policy_version 352153 (0.0007) [2023-12-26 17:55:29,154][105692] Updated weights for policy 0, policy_version 351709 (0.0011) [2023-12-26 17:55:29,156][105620] Updated weights for policy 1, policy_version 352163 (0.0005) [2023-12-26 17:55:29,208][105692] Updated weights for policy 0, policy_version 351719 (0.0009) [2023-12-26 17:55:29,913][105620] Updated weights for policy 1, policy_version 352173 (0.0007) [2023-12-26 17:55:29,969][105692] Updated weights for policy 0, policy_version 351729 (0.0010) [2023-12-26 17:55:29,975][105620] Updated weights for policy 1, policy_version 352183 (0.0007) [2023-12-26 17:55:30,028][105692] Updated weights for policy 0, policy_version 351739 (0.0011) [2023-12-26 17:55:30,030][105620] Updated weights for policy 1, policy_version 352193 (0.0006) [2023-12-26 17:55:30,089][105692] Updated weights for policy 0, policy_version 351749 (0.0010) [2023-12-26 17:55:30,721][105692] Updated weights for policy 0, policy_version 351759 (0.0008) [2023-12-26 17:55:30,783][105692] Updated weights for policy 0, policy_version 351769 (0.0009) [2023-12-26 17:55:30,845][105692] Updated weights for policy 0, policy_version 351779 (0.0006) [2023-12-26 17:55:30,853][105620] Updated weights for policy 1, policy_version 352203 (0.0009) [2023-12-26 17:55:30,904][105620] Updated weights for policy 1, policy_version 352213 (0.0009) [2023-12-26 17:55:30,958][105620] Updated weights for policy 1, policy_version 352224 (0.0010) [2023-12-26 17:55:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 180248576. Throughput: 0: 10044.3, 1: 9757.6. Samples: 180214444. Policy #0 lag: (min: 14.0, avg: 14.3, max: 22.0) [2023-12-26 17:55:31,062][104569] Avg episode reward: [(0, '8994.649'), (1, '1598.478')] [2023-12-26 17:55:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000351784_90071040.pth... [2023-12-26 17:55:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000352232_90177536.pth... [2023-12-26 17:55:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000350600_89767936.pth [2023-12-26 17:55:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000351080_89882624.pth [2023-12-26 17:55:31,487][105692] Updated weights for policy 0, policy_version 351789 (0.0008) [2023-12-26 17:55:31,541][105692] Updated weights for policy 0, policy_version 351799 (0.0008) [2023-12-26 17:55:31,600][105692] Updated weights for policy 0, policy_version 351809 (0.0008) [2023-12-26 17:55:31,787][105620] Updated weights for policy 1, policy_version 352235 (0.0009) [2023-12-26 17:55:31,848][105620] Updated weights for policy 1, policy_version 352245 (0.0009) [2023-12-26 17:55:31,903][105620] Updated weights for policy 1, policy_version 352255 (0.0009) [2023-12-26 17:55:32,362][105692] Updated weights for policy 0, policy_version 351819 (0.0008) [2023-12-26 17:55:32,424][105692] Updated weights for policy 0, policy_version 351829 (0.0008) [2023-12-26 17:55:32,474][105692] Updated weights for policy 0, policy_version 351839 (0.0009) [2023-12-26 17:55:32,655][105620] Updated weights for policy 1, policy_version 352265 (0.0009) [2023-12-26 17:55:32,705][105620] Updated weights for policy 1, policy_version 352275 (0.0006) [2023-12-26 17:55:32,765][105620] Updated weights for policy 1, policy_version 352285 (0.0007) [2023-12-26 17:55:32,826][105620] Updated weights for policy 1, policy_version 352295 (0.0009) [2023-12-26 17:55:33,252][105692] Updated weights for policy 0, policy_version 351849 (0.0008) [2023-12-26 17:55:33,298][105692] Updated weights for policy 0, policy_version 351859 (0.0009) [2023-12-26 17:55:33,345][105692] Updated weights for policy 0, policy_version 351869 (0.0009) [2023-12-26 17:55:33,391][105692] Updated weights for policy 0, policy_version 351879 (0.0009) [2023-12-26 17:55:33,524][105620] Updated weights for policy 1, policy_version 352305 (0.0009) [2023-12-26 17:55:33,571][105620] Updated weights for policy 1, policy_version 352315 (0.0009) [2023-12-26 17:55:33,626][105620] Updated weights for policy 1, policy_version 352325 (0.0009) [2023-12-26 17:55:34,093][105692] Updated weights for policy 0, policy_version 351889 (0.0007) [2023-12-26 17:55:34,152][105692] Updated weights for policy 0, policy_version 351899 (0.0008) [2023-12-26 17:55:34,220][105692] Updated weights for policy 0, policy_version 351909 (0.0007) [2023-12-26 17:55:34,423][105620] Updated weights for policy 1, policy_version 352335 (0.0010) [2023-12-26 17:55:34,487][105620] Updated weights for policy 1, policy_version 352345 (0.0007) [2023-12-26 17:55:34,552][105620] Updated weights for policy 1, policy_version 352355 (0.0006) [2023-12-26 17:55:34,979][105692] Updated weights for policy 0, policy_version 351919 (0.0009) [2023-12-26 17:55:35,040][105692] Updated weights for policy 0, policy_version 351929 (0.0009) [2023-12-26 17:55:35,086][105692] Updated weights for policy 0, policy_version 351939 (0.0009) [2023-12-26 17:55:35,224][105620] Updated weights for policy 1, policy_version 352365 (0.0007) [2023-12-26 17:55:35,271][105620] Updated weights for policy 1, policy_version 352375 (0.0009) [2023-12-26 17:55:35,333][105620] Updated weights for policy 1, policy_version 352385 (0.0008) [2023-12-26 17:55:35,833][105692] Updated weights for policy 0, policy_version 351949 (0.0009) [2023-12-26 17:55:35,880][105692] Updated weights for policy 0, policy_version 351959 (0.0009) [2023-12-26 17:55:35,938][105692] Updated weights for policy 0, policy_version 351969 (0.0009) [2023-12-26 17:55:36,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 180338688. Throughput: 0: 9943.5, 1: 9718.8. Samples: 180327868. Policy #0 lag: (min: 14.0, avg: 14.3, max: 22.0) [2023-12-26 17:55:36,062][104569] Avg episode reward: [(0, '8994.700'), (1, '2578.338')] [2023-12-26 17:55:36,093][105620] Updated weights for policy 1, policy_version 352395 (0.0009) [2023-12-26 17:55:36,157][105620] Updated weights for policy 1, policy_version 352405 (0.0008) [2023-12-26 17:55:36,209][105620] Updated weights for policy 1, policy_version 352415 (0.0009) [2023-12-26 17:55:36,642][105692] Updated weights for policy 0, policy_version 351979 (0.0009) [2023-12-26 17:55:36,704][105692] Updated weights for policy 0, policy_version 351989 (0.0009) [2023-12-26 17:55:36,765][105692] Updated weights for policy 0, policy_version 351999 (0.0008) [2023-12-26 17:55:37,031][105620] Updated weights for policy 1, policy_version 352425 (0.0009) [2023-12-26 17:55:37,086][105620] Updated weights for policy 1, policy_version 352435 (0.0009) [2023-12-26 17:55:37,124][105586] KL-divergence is very high: 126.0010 [2023-12-26 17:55:37,130][105586] KL-divergence is very high: 138.1520 [2023-12-26 17:55:37,137][105586] KL-divergence is very high: 152.8810 [2023-12-26 17:55:37,143][105586] KL-divergence is very high: 143.0221 [2023-12-26 17:55:37,148][105620] Updated weights for policy 1, policy_version 352445 (0.0009) [2023-12-26 17:55:37,173][105586] KL-divergence is very high: 160.0623 [2023-12-26 17:55:37,179][105586] KL-divergence is very high: 157.8444 [2023-12-26 17:55:37,188][105586] KL-divergence is very high: 163.4983 [2023-12-26 17:55:37,196][105586] KL-divergence is very high: 136.5137 [2023-12-26 17:55:37,213][105620] Updated weights for policy 1, policy_version 352455 (0.0009) [2023-12-26 17:55:37,417][105692] Updated weights for policy 0, policy_version 352009 (0.0006) [2023-12-26 17:55:37,472][105692] Updated weights for policy 0, policy_version 352019 (0.0009) [2023-12-26 17:55:37,531][105692] Updated weights for policy 0, policy_version 352029 (0.0009) [2023-12-26 17:55:37,588][105692] Updated weights for policy 0, policy_version 352039 (0.0009) [2023-12-26 17:55:38,016][105620] Updated weights for policy 1, policy_version 352465 (0.0009) [2023-12-26 17:55:38,071][105620] Updated weights for policy 1, policy_version 352475 (0.0008) [2023-12-26 17:55:38,126][105620] Updated weights for policy 1, policy_version 352485 (0.0010) [2023-12-26 17:55:38,242][105692] Updated weights for policy 0, policy_version 352049 (0.0008) [2023-12-26 17:55:38,296][105692] Updated weights for policy 0, policy_version 352059 (0.0010) [2023-12-26 17:55:38,356][105692] Updated weights for policy 0, policy_version 352069 (0.0010) [2023-12-26 17:55:38,986][105620] Updated weights for policy 1, policy_version 352495 (0.0010) [2023-12-26 17:55:38,992][105692] Updated weights for policy 0, policy_version 352079 (0.0010) [2023-12-26 17:55:39,045][105620] Updated weights for policy 1, policy_version 352505 (0.0006) [2023-12-26 17:55:39,054][105692] Updated weights for policy 0, policy_version 352089 (0.0011) [2023-12-26 17:55:39,092][105620] Updated weights for policy 1, policy_version 352515 (0.0006) [2023-12-26 17:55:39,106][105692] Updated weights for policy 0, policy_version 352099 (0.0011) [2023-12-26 17:55:39,824][105692] Updated weights for policy 0, policy_version 352109 (0.0008) [2023-12-26 17:55:39,836][105620] Updated weights for policy 1, policy_version 352525 (0.0007) [2023-12-26 17:55:39,888][105692] Updated weights for policy 0, policy_version 352119 (0.0007) [2023-12-26 17:55:39,895][105620] Updated weights for policy 1, policy_version 352535 (0.0009) [2023-12-26 17:55:39,950][105692] Updated weights for policy 0, policy_version 352129 (0.0008) [2023-12-26 17:55:39,956][105620] Updated weights for policy 1, policy_version 352545 (0.0008) [2023-12-26 17:55:40,703][105692] Updated weights for policy 0, policy_version 352139 (0.0006) [2023-12-26 17:55:40,763][105692] Updated weights for policy 0, policy_version 352149 (0.0005) [2023-12-26 17:55:40,787][105620] Updated weights for policy 1, policy_version 352555 (0.0008) [2023-12-26 17:55:40,824][105692] Updated weights for policy 0, policy_version 352159 (0.0006) [2023-12-26 17:55:40,842][105620] Updated weights for policy 1, policy_version 352565 (0.0008) [2023-12-26 17:55:40,905][105620] Updated weights for policy 1, policy_version 352575 (0.0009) [2023-12-26 17:55:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 180436992. Throughput: 0: 9913.0, 1: 9649.3. Samples: 180440444. Policy #0 lag: (min: 14.0, avg: 14.3, max: 22.0) [2023-12-26 17:55:41,063][104569] Avg episode reward: [(0, '8722.711'), (1, '2977.180')] [2023-12-26 17:55:41,493][105692] Updated weights for policy 0, policy_version 352169 (0.0007) [2023-12-26 17:55:41,541][105692] Updated weights for policy 0, policy_version 352179 (0.0009) [2023-12-26 17:55:41,594][105692] Updated weights for policy 0, policy_version 352189 (0.0009) [2023-12-26 17:55:41,660][105692] Updated weights for policy 0, policy_version 352199 (0.0009) [2023-12-26 17:55:41,697][105620] Updated weights for policy 1, policy_version 352585 (0.0010) [2023-12-26 17:55:41,761][105620] Updated weights for policy 1, policy_version 352595 (0.0009) [2023-12-26 17:55:41,813][105620] Updated weights for policy 1, policy_version 352605 (0.0009) [2023-12-26 17:55:41,864][105620] Updated weights for policy 1, policy_version 352615 (0.0008) [2023-12-26 17:55:42,440][105692] Updated weights for policy 0, policy_version 352209 (0.0008) [2023-12-26 17:55:42,505][105692] Updated weights for policy 0, policy_version 352219 (0.0009) [2023-12-26 17:55:42,561][105692] Updated weights for policy 0, policy_version 352229 (0.0009) [2023-12-26 17:55:42,667][105620] Updated weights for policy 1, policy_version 352625 (0.0009) [2023-12-26 17:55:42,729][105620] Updated weights for policy 1, policy_version 352635 (0.0009) [2023-12-26 17:55:42,791][105620] Updated weights for policy 1, policy_version 352645 (0.0008) [2023-12-26 17:55:43,330][105692] Updated weights for policy 0, policy_version 352239 (0.0009) [2023-12-26 17:55:43,391][105692] Updated weights for policy 0, policy_version 352249 (0.0009) [2023-12-26 17:55:43,450][105692] Updated weights for policy 0, policy_version 352259 (0.0008) [2023-12-26 17:55:43,472][105620] Updated weights for policy 1, policy_version 352655 (0.0009) [2023-12-26 17:55:43,536][105620] Updated weights for policy 1, policy_version 352665 (0.0009) [2023-12-26 17:55:43,583][105620] Updated weights for policy 1, policy_version 352675 (0.0008) [2023-12-26 17:55:44,237][105692] Updated weights for policy 0, policy_version 352269 (0.0007) [2023-12-26 17:55:44,292][105620] Updated weights for policy 1, policy_version 352685 (0.0008) [2023-12-26 17:55:44,294][105692] Updated weights for policy 0, policy_version 352279 (0.0010) [2023-12-26 17:55:44,349][105620] Updated weights for policy 1, policy_version 352695 (0.0007) [2023-12-26 17:55:44,351][105692] Updated weights for policy 0, policy_version 352289 (0.0007) [2023-12-26 17:55:44,407][105620] Updated weights for policy 1, policy_version 352705 (0.0008) [2023-12-26 17:55:45,141][105620] Updated weights for policy 1, policy_version 352715 (0.0008) [2023-12-26 17:55:45,141][105692] Updated weights for policy 0, policy_version 352299 (0.0008) [2023-12-26 17:55:45,208][105620] Updated weights for policy 1, policy_version 352725 (0.0007) [2023-12-26 17:55:45,208][105692] Updated weights for policy 0, policy_version 352309 (0.0011) [2023-12-26 17:55:45,267][105620] Updated weights for policy 1, policy_version 352735 (0.0010) [2023-12-26 17:55:45,268][105692] Updated weights for policy 0, policy_version 352319 (0.0011) [2023-12-26 17:55:45,964][105620] Updated weights for policy 1, policy_version 352745 (0.0009) [2023-12-26 17:55:46,012][105692] Updated weights for policy 0, policy_version 352329 (0.0011) [2023-12-26 17:55:46,026][105620] Updated weights for policy 1, policy_version 352755 (0.0007) [2023-12-26 17:55:46,059][105692] Updated weights for policy 0, policy_version 352339 (0.0010) [2023-12-26 17:55:46,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 180518912. Throughput: 0: 9904.4, 1: 9617.7. Samples: 180496704. Policy #0 lag: (min: 14.0, avg: 14.3, max: 22.0) [2023-12-26 17:55:46,062][104569] Avg episode reward: [(0, '8065.248'), (1, '6797.734')] [2023-12-26 17:55:46,070][105620] Updated weights for policy 1, policy_version 352765 (0.0006) [2023-12-26 17:55:46,114][105620] Updated weights for policy 1, policy_version 352775 (0.0006) [2023-12-26 17:55:46,117][105692] Updated weights for policy 0, policy_version 352349 (0.0010) [2023-12-26 17:55:46,117][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000352776_90316800.pth... [2023-12-26 17:55:46,120][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000351656_90030080.pth [2023-12-26 17:55:46,176][105692] Updated weights for policy 0, policy_version 352359 (0.0010) [2023-12-26 17:55:46,180][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000352360_90218496.pth... [2023-12-26 17:55:46,184][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000351208_89923584.pth [2023-12-26 17:55:46,881][105620] Updated weights for policy 1, policy_version 352785 (0.0008) [2023-12-26 17:55:46,939][105620] Updated weights for policy 1, policy_version 352795 (0.0008) [2023-12-26 17:55:46,951][105692] Updated weights for policy 0, policy_version 352369 (0.0006) [2023-12-26 17:55:47,000][105620] Updated weights for policy 1, policy_version 352805 (0.0009) [2023-12-26 17:55:47,012][105692] Updated weights for policy 0, policy_version 352379 (0.0005) [2023-12-26 17:55:47,069][105692] Updated weights for policy 0, policy_version 352389 (0.0005) [2023-12-26 17:55:47,600][105692] Updated weights for policy 0, policy_version 352399 (0.0008) [2023-12-26 17:55:47,645][105620] Updated weights for policy 1, policy_version 352815 (0.0006) [2023-12-26 17:55:47,648][105692] Updated weights for policy 0, policy_version 352409 (0.0009) [2023-12-26 17:55:47,697][105692] Updated weights for policy 0, policy_version 352419 (0.0008) [2023-12-26 17:55:47,702][105620] Updated weights for policy 1, policy_version 352825 (0.0005) [2023-12-26 17:55:47,760][105620] Updated weights for policy 1, policy_version 352835 (0.0006) [2023-12-26 17:55:48,391][105692] Updated weights for policy 0, policy_version 352429 (0.0006) [2023-12-26 17:55:48,451][105692] Updated weights for policy 0, policy_version 352439 (0.0005) [2023-12-26 17:55:48,467][105620] Updated weights for policy 1, policy_version 352845 (0.0008) [2023-12-26 17:55:48,517][105692] Updated weights for policy 0, policy_version 352449 (0.0005) [2023-12-26 17:55:48,534][105620] Updated weights for policy 1, policy_version 352855 (0.0009) [2023-12-26 17:55:48,603][105620] Updated weights for policy 1, policy_version 352865 (0.0009) [2023-12-26 17:55:49,206][105692] Updated weights for policy 0, policy_version 352459 (0.0006) [2023-12-26 17:55:49,278][105692] Updated weights for policy 0, policy_version 352469 (0.0010) [2023-12-26 17:55:49,291][105620] Updated weights for policy 1, policy_version 352875 (0.0006) [2023-12-26 17:55:49,339][105692] Updated weights for policy 0, policy_version 352479 (0.0008) [2023-12-26 17:55:49,357][105620] Updated weights for policy 1, policy_version 352885 (0.0008) [2023-12-26 17:55:49,419][105620] Updated weights for policy 1, policy_version 352895 (0.0008) [2023-12-26 17:55:50,058][105692] Updated weights for policy 0, policy_version 352489 (0.0007) [2023-12-26 17:55:50,122][105692] Updated weights for policy 0, policy_version 352499 (0.0005) [2023-12-26 17:55:50,176][105620] Updated weights for policy 1, policy_version 352905 (0.0008) [2023-12-26 17:55:50,184][105692] Updated weights for policy 0, policy_version 352509 (0.0005) [2023-12-26 17:55:50,236][105620] Updated weights for policy 1, policy_version 352915 (0.0011) [2023-12-26 17:55:50,238][105692] Updated weights for policy 0, policy_version 352519 (0.0007) [2023-12-26 17:55:50,291][105620] Updated weights for policy 1, policy_version 352925 (0.0011) [2023-12-26 17:55:50,344][105620] Updated weights for policy 1, policy_version 352935 (0.0010) [2023-12-26 17:55:50,901][105692] Updated weights for policy 0, policy_version 352529 (0.0008) [2023-12-26 17:55:50,968][105692] Updated weights for policy 0, policy_version 352539 (0.0008) [2023-12-26 17:55:51,033][105692] Updated weights for policy 0, policy_version 352549 (0.0007) [2023-12-26 17:55:51,054][105620] Updated weights for policy 1, policy_version 352945 (0.0011) [2023-12-26 17:55:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 180625408. Throughput: 0: 9783.7, 1: 9600.3. Samples: 180613676. Policy #0 lag: (min: 14.0, avg: 14.3, max: 22.0) [2023-12-26 17:55:51,063][104569] Avg episode reward: [(0, '7252.947'), (1, '6597.309')] [2023-12-26 17:55:51,119][105620] Updated weights for policy 1, policy_version 352955 (0.0006) [2023-12-26 17:55:51,191][105620] Updated weights for policy 1, policy_version 352965 (0.0010) [2023-12-26 17:55:51,784][105692] Updated weights for policy 0, policy_version 352559 (0.0006) [2023-12-26 17:55:51,848][105692] Updated weights for policy 0, policy_version 352569 (0.0006) [2023-12-26 17:55:51,913][105692] Updated weights for policy 0, policy_version 352579 (0.0005) [2023-12-26 17:55:51,953][105620] Updated weights for policy 1, policy_version 352975 (0.0011) [2023-12-26 17:55:52,019][105620] Updated weights for policy 1, policy_version 352985 (0.0010) [2023-12-26 17:55:52,089][105620] Updated weights for policy 1, policy_version 352995 (0.0005) [2023-12-26 17:55:52,557][105692] Updated weights for policy 0, policy_version 352589 (0.0007) [2023-12-26 17:55:52,617][105692] Updated weights for policy 0, policy_version 352599 (0.0008) [2023-12-26 17:55:52,677][105692] Updated weights for policy 0, policy_version 352609 (0.0008) [2023-12-26 17:55:52,797][105620] Updated weights for policy 1, policy_version 353005 (0.0009) [2023-12-26 17:55:52,856][105620] Updated weights for policy 1, policy_version 353015 (0.0010) [2023-12-26 17:55:52,915][105620] Updated weights for policy 1, policy_version 353025 (0.0010) [2023-12-26 17:55:53,450][105692] Updated weights for policy 0, policy_version 352619 (0.0008) [2023-12-26 17:55:53,508][105692] Updated weights for policy 0, policy_version 352629 (0.0010) [2023-12-26 17:55:53,566][105692] Updated weights for policy 0, policy_version 352639 (0.0009) [2023-12-26 17:55:53,583][105620] Updated weights for policy 1, policy_version 353035 (0.0010) [2023-12-26 17:55:53,641][105620] Updated weights for policy 1, policy_version 353045 (0.0010) [2023-12-26 17:55:53,706][105620] Updated weights for policy 1, policy_version 353055 (0.0010) [2023-12-26 17:55:54,166][105692] Updated weights for policy 0, policy_version 352649 (0.0010) [2023-12-26 17:55:54,225][105692] Updated weights for policy 0, policy_version 352659 (0.0010) [2023-12-26 17:55:54,284][105692] Updated weights for policy 0, policy_version 352669 (0.0010) [2023-12-26 17:55:54,343][105692] Updated weights for policy 0, policy_version 352679 (0.0008) [2023-12-26 17:55:54,441][105620] Updated weights for policy 1, policy_version 353065 (0.0010) [2023-12-26 17:55:54,486][105620] Updated weights for policy 1, policy_version 353075 (0.0008) [2023-12-26 17:55:54,536][105620] Updated weights for policy 1, policy_version 353085 (0.0005) [2023-12-26 17:55:54,593][105620] Updated weights for policy 1, policy_version 353095 (0.0010) [2023-12-26 17:55:54,993][105692] Updated weights for policy 0, policy_version 352689 (0.0010) [2023-12-26 17:55:55,042][105692] Updated weights for policy 0, policy_version 352699 (0.0010) [2023-12-26 17:55:55,089][105692] Updated weights for policy 0, policy_version 352709 (0.0010) [2023-12-26 17:55:55,211][105620] Updated weights for policy 1, policy_version 353105 (0.0010) [2023-12-26 17:55:55,266][105620] Updated weights for policy 1, policy_version 353115 (0.0010) [2023-12-26 17:55:55,330][105620] Updated weights for policy 1, policy_version 353125 (0.0010) [2023-12-26 17:55:55,779][105692] Updated weights for policy 0, policy_version 352719 (0.0010) [2023-12-26 17:55:55,833][105692] Updated weights for policy 0, policy_version 352729 (0.0010) [2023-12-26 17:55:55,888][105692] Updated weights for policy 0, policy_version 352739 (0.0010) [2023-12-26 17:55:55,911][105620] Updated weights for policy 1, policy_version 353135 (0.0008) [2023-12-26 17:55:55,964][105620] Updated weights for policy 1, policy_version 353145 (0.0005) [2023-12-26 17:55:56,020][105620] Updated weights for policy 1, policy_version 353155 (0.0005) [2023-12-26 17:55:56,062][104569] Fps is (10 sec: 21299.3, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 180731904. Throughput: 0: 9885.4, 1: 9621.1. Samples: 180734324. Policy #0 lag: (min: 14.0, avg: 14.3, max: 22.0) [2023-12-26 17:55:56,062][104569] Avg episode reward: [(0, '7660.314'), (1, '6234.901')] [2023-12-26 17:55:56,614][105620] Updated weights for policy 1, policy_version 353165 (0.0006) [2023-12-26 17:55:56,639][105692] Updated weights for policy 0, policy_version 352749 (0.0010) [2023-12-26 17:55:56,669][105620] Updated weights for policy 1, policy_version 353175 (0.0006) [2023-12-26 17:55:56,684][105692] Updated weights for policy 0, policy_version 352759 (0.0010) [2023-12-26 17:55:56,720][105620] Updated weights for policy 1, policy_version 353185 (0.0010) [2023-12-26 17:55:56,728][105692] Updated weights for policy 0, policy_version 352769 (0.0010) [2023-12-26 17:55:57,296][105620] Updated weights for policy 1, policy_version 353195 (0.0010) [2023-12-26 17:55:57,357][105620] Updated weights for policy 1, policy_version 353205 (0.0008) [2023-12-26 17:55:57,418][105620] Updated weights for policy 1, policy_version 353215 (0.0008) [2023-12-26 17:55:57,501][105692] Updated weights for policy 0, policy_version 352779 (0.0010) [2023-12-26 17:55:57,561][105692] Updated weights for policy 0, policy_version 352789 (0.0008) [2023-12-26 17:55:57,615][105692] Updated weights for policy 0, policy_version 352799 (0.0010) [2023-12-26 17:55:58,182][105620] Updated weights for policy 1, policy_version 353225 (0.0008) [2023-12-26 17:55:58,246][105620] Updated weights for policy 1, policy_version 353235 (0.0010) [2023-12-26 17:55:58,275][105692] Updated weights for policy 0, policy_version 352809 (0.0010) [2023-12-26 17:55:58,305][105620] Updated weights for policy 1, policy_version 353245 (0.0009) [2023-12-26 17:55:58,346][105692] Updated weights for policy 0, policy_version 352819 (0.0007) [2023-12-26 17:55:58,386][105620] Updated weights for policy 1, policy_version 353255 (0.0007) [2023-12-26 17:55:58,416][105692] Updated weights for policy 0, policy_version 352829 (0.0010) [2023-12-26 17:55:58,479][105692] Updated weights for policy 0, policy_version 352839 (0.0011) [2023-12-26 17:55:59,179][105620] Updated weights for policy 1, policy_version 353265 (0.0006) [2023-12-26 17:55:59,185][105692] Updated weights for policy 0, policy_version 352849 (0.0008) [2023-12-26 17:55:59,243][105620] Updated weights for policy 1, policy_version 353275 (0.0008) [2023-12-26 17:55:59,244][105692] Updated weights for policy 0, policy_version 352859 (0.0008) [2023-12-26 17:55:59,311][105620] Updated weights for policy 1, policy_version 353285 (0.0009) [2023-12-26 17:55:59,314][105692] Updated weights for policy 0, policy_version 352869 (0.0007) [2023-12-26 17:56:00,068][105692] Updated weights for policy 0, policy_version 352879 (0.0008) [2023-12-26 17:56:00,074][105620] Updated weights for policy 1, policy_version 353295 (0.0008) [2023-12-26 17:56:00,115][105692] Updated weights for policy 0, policy_version 352889 (0.0007) [2023-12-26 17:56:00,125][105620] Updated weights for policy 1, policy_version 353305 (0.0007) [2023-12-26 17:56:00,166][105692] Updated weights for policy 0, policy_version 352899 (0.0007) [2023-12-26 17:56:00,183][105620] Updated weights for policy 1, policy_version 353315 (0.0007) [2023-12-26 17:56:00,917][105692] Updated weights for policy 0, policy_version 352909 (0.0008) [2023-12-26 17:56:00,960][105620] Updated weights for policy 1, policy_version 353325 (0.0007) [2023-12-26 17:56:00,970][105692] Updated weights for policy 0, policy_version 352919 (0.0008) [2023-12-26 17:56:01,019][105620] Updated weights for policy 1, policy_version 353335 (0.0007) [2023-12-26 17:56:01,026][105692] Updated weights for policy 0, policy_version 352929 (0.0006) [2023-12-26 17:56:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 180813824. Throughput: 0: 9891.0, 1: 9685.8. Samples: 180793316. Policy #0 lag: (min: 14.0, avg: 14.3, max: 22.0) [2023-12-26 17:56:01,062][104569] Avg episode reward: [(0, '8721.760'), (1, '4893.990')] [2023-12-26 17:56:01,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000352936_90365952.pth... [2023-12-26 17:56:01,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000351784_90071040.pth [2023-12-26 17:56:01,079][105620] Updated weights for policy 1, policy_version 353345 (0.0008) [2023-12-26 17:56:01,127][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000353352_90464256.pth... [2023-12-26 17:56:01,132][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000352232_90177536.pth [2023-12-26 17:56:01,781][105692] Updated weights for policy 0, policy_version 352939 (0.0007) [2023-12-26 17:56:01,838][105692] Updated weights for policy 0, policy_version 352949 (0.0007) [2023-12-26 17:56:01,869][105620] Updated weights for policy 1, policy_version 353355 (0.0009) [2023-12-26 17:56:01,899][105692] Updated weights for policy 0, policy_version 352959 (0.0005) [2023-12-26 17:56:01,925][105620] Updated weights for policy 1, policy_version 353365 (0.0010) [2023-12-26 17:56:01,983][105620] Updated weights for policy 1, policy_version 353375 (0.0011) [2023-12-26 17:56:02,613][105692] Updated weights for policy 0, policy_version 352969 (0.0006) [2023-12-26 17:56:02,667][105692] Updated weights for policy 0, policy_version 352979 (0.0010) [2023-12-26 17:56:02,684][105620] Updated weights for policy 1, policy_version 353385 (0.0010) [2023-12-26 17:56:02,719][105692] Updated weights for policy 0, policy_version 352989 (0.0008) [2023-12-26 17:56:02,748][105620] Updated weights for policy 1, policy_version 353395 (0.0006) [2023-12-26 17:56:02,767][105692] Updated weights for policy 0, policy_version 352999 (0.0008) [2023-12-26 17:56:02,807][105620] Updated weights for policy 1, policy_version 353405 (0.0008) [2023-12-26 17:56:02,858][105620] Updated weights for policy 1, policy_version 353415 (0.0006) [2023-12-26 17:56:03,392][105692] Updated weights for policy 0, policy_version 353009 (0.0006) [2023-12-26 17:56:03,459][105692] Updated weights for policy 0, policy_version 353019 (0.0007) [2023-12-26 17:56:03,520][105692] Updated weights for policy 0, policy_version 353029 (0.0007) [2023-12-26 17:56:03,550][105620] Updated weights for policy 1, policy_version 353425 (0.0009) [2023-12-26 17:56:03,598][105620] Updated weights for policy 1, policy_version 353435 (0.0007) [2023-12-26 17:56:03,644][105620] Updated weights for policy 1, policy_version 353445 (0.0007) [2023-12-26 17:56:04,172][105692] Updated weights for policy 0, policy_version 353039 (0.0007) [2023-12-26 17:56:04,237][105692] Updated weights for policy 0, policy_version 353049 (0.0008) [2023-12-26 17:56:04,309][105692] Updated weights for policy 0, policy_version 353059 (0.0008) [2023-12-26 17:56:04,350][105620] Updated weights for policy 1, policy_version 353455 (0.0008) [2023-12-26 17:56:04,415][105620] Updated weights for policy 1, policy_version 353465 (0.0006) [2023-12-26 17:56:04,473][105620] Updated weights for policy 1, policy_version 353475 (0.0006) [2023-12-26 17:56:05,071][105692] Updated weights for policy 0, policy_version 353069 (0.0007) [2023-12-26 17:56:05,127][105692] Updated weights for policy 0, policy_version 353079 (0.0008) [2023-12-26 17:56:05,171][105692] Updated weights for policy 0, policy_version 353089 (0.0007) [2023-12-26 17:56:05,202][105620] Updated weights for policy 1, policy_version 353485 (0.0010) [2023-12-26 17:56:05,250][105620] Updated weights for policy 1, policy_version 353495 (0.0010) [2023-12-26 17:56:05,305][105620] Updated weights for policy 1, policy_version 353505 (0.0010) [2023-12-26 17:56:05,945][105692] Updated weights for policy 0, policy_version 353099 (0.0006) [2023-12-26 17:56:06,003][105692] Updated weights for policy 0, policy_version 353109 (0.0005) [2023-12-26 17:56:06,031][105620] Updated weights for policy 1, policy_version 353515 (0.0010) [2023-12-26 17:56:06,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 180912128. Throughput: 0: 9838.7, 1: 9593.3. Samples: 180908984. Policy #0 lag: (min: 14.0, avg: 14.3, max: 22.0) [2023-12-26 17:56:06,062][104569] Avg episode reward: [(0, '8813.038'), (1, '6805.999')] [2023-12-26 17:56:06,066][105692] Updated weights for policy 0, policy_version 353119 (0.0005) [2023-12-26 17:56:06,081][105620] Updated weights for policy 1, policy_version 353525 (0.0009) [2023-12-26 17:56:06,146][105620] Updated weights for policy 1, policy_version 353535 (0.0007) [2023-12-26 17:56:06,794][105692] Updated weights for policy 0, policy_version 353129 (0.0007) [2023-12-26 17:56:06,852][105692] Updated weights for policy 0, policy_version 353139 (0.0008) [2023-12-26 17:56:06,876][105620] Updated weights for policy 1, policy_version 353545 (0.0007) [2023-12-26 17:56:06,902][105692] Updated weights for policy 0, policy_version 353149 (0.0007) [2023-12-26 17:56:06,939][105620] Updated weights for policy 1, policy_version 353555 (0.0011) [2023-12-26 17:56:06,957][105692] Updated weights for policy 0, policy_version 353159 (0.0008) [2023-12-26 17:56:06,994][105620] Updated weights for policy 1, policy_version 353565 (0.0011) [2023-12-26 17:56:07,052][105620] Updated weights for policy 1, policy_version 353575 (0.0010) [2023-12-26 17:56:07,638][105620] Updated weights for policy 1, policy_version 353585 (0.0006) [2023-12-26 17:56:07,702][105620] Updated weights for policy 1, policy_version 353595 (0.0005) [2023-12-26 17:56:07,761][105620] Updated weights for policy 1, policy_version 353605 (0.0006) [2023-12-26 17:56:07,813][105692] Updated weights for policy 0, policy_version 353169 (0.0009) [2023-12-26 17:56:07,872][105692] Updated weights for policy 0, policy_version 353180 (0.0011) [2023-12-26 17:56:07,924][105692] Updated weights for policy 0, policy_version 353190 (0.0010) [2023-12-26 17:56:08,321][105620] Updated weights for policy 1, policy_version 353615 (0.0008) [2023-12-26 17:56:08,385][105620] Updated weights for policy 1, policy_version 353625 (0.0008) [2023-12-26 17:56:08,454][105620] Updated weights for policy 1, policy_version 353635 (0.0008) [2023-12-26 17:56:08,670][105692] Updated weights for policy 0, policy_version 353201 (0.0009) [2023-12-26 17:56:08,732][105692] Updated weights for policy 0, policy_version 353211 (0.0009) [2023-12-26 17:56:08,791][105692] Updated weights for policy 0, policy_version 353221 (0.0009) [2023-12-26 17:56:09,183][105620] Updated weights for policy 1, policy_version 353645 (0.0009) [2023-12-26 17:56:09,252][105620] Updated weights for policy 1, policy_version 353655 (0.0009) [2023-12-26 17:56:09,312][105620] Updated weights for policy 1, policy_version 353665 (0.0009) [2023-12-26 17:56:09,558][105692] Updated weights for policy 0, policy_version 353231 (0.0008) [2023-12-26 17:56:09,623][105692] Updated weights for policy 0, policy_version 353241 (0.0009) [2023-12-26 17:56:09,680][105692] Updated weights for policy 0, policy_version 353251 (0.0009) [2023-12-26 17:56:10,089][105620] Updated weights for policy 1, policy_version 353675 (0.0009) [2023-12-26 17:56:10,149][105620] Updated weights for policy 1, policy_version 353685 (0.0008) [2023-12-26 17:56:10,208][105620] Updated weights for policy 1, policy_version 353695 (0.0009) [2023-12-26 17:56:10,454][105692] Updated weights for policy 0, policy_version 353261 (0.0009) [2023-12-26 17:56:10,516][105692] Updated weights for policy 0, policy_version 353271 (0.0009) [2023-12-26 17:56:10,575][105692] Updated weights for policy 0, policy_version 353281 (0.0009) [2023-12-26 17:56:10,965][105620] Updated weights for policy 1, policy_version 353705 (0.0008) [2023-12-26 17:56:11,027][105620] Updated weights for policy 1, policy_version 353715 (0.0009) [2023-12-26 17:56:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19438.7). Total num frames: 181010432. Throughput: 0: 9722.6, 1: 9609.0. Samples: 181023332. Policy #0 lag: (min: 14.0, avg: 14.3, max: 22.0) [2023-12-26 17:56:11,062][104569] Avg episode reward: [(0, '8813.474'), (1, '3666.354')] [2023-12-26 17:56:11,089][105620] Updated weights for policy 1, policy_version 353725 (0.0009) [2023-12-26 17:56:11,155][105620] Updated weights for policy 1, policy_version 353735 (0.0008) [2023-12-26 17:56:11,326][105692] Updated weights for policy 0, policy_version 353291 (0.0009) [2023-12-26 17:56:11,407][105692] Updated weights for policy 0, policy_version 353301 (0.0010) [2023-12-26 17:56:11,475][105692] Updated weights for policy 0, policy_version 353311 (0.0008) [2023-12-26 17:56:11,990][105620] Updated weights for policy 1, policy_version 353745 (0.0011) [2023-12-26 17:56:12,049][105620] Updated weights for policy 1, policy_version 353755 (0.0011) [2023-12-26 17:56:12,102][105620] Updated weights for policy 1, policy_version 353765 (0.0010) [2023-12-26 17:56:12,242][105692] Updated weights for policy 0, policy_version 353321 (0.0009) [2023-12-26 17:56:12,304][105692] Updated weights for policy 0, policy_version 353331 (0.0008) [2023-12-26 17:56:12,371][105692] Updated weights for policy 0, policy_version 353341 (0.0009) [2023-12-26 17:56:12,438][105692] Updated weights for policy 0, policy_version 353351 (0.0008) [2023-12-26 17:56:12,895][105620] Updated weights for policy 1, policy_version 353775 (0.0010) [2023-12-26 17:56:12,941][105620] Updated weights for policy 1, policy_version 353785 (0.0010) [2023-12-26 17:56:12,997][105620] Updated weights for policy 1, policy_version 353795 (0.0010) [2023-12-26 17:56:13,205][105692] Updated weights for policy 0, policy_version 353361 (0.0009) [2023-12-26 17:56:13,259][105692] Updated weights for policy 0, policy_version 353371 (0.0008) [2023-12-26 17:56:13,315][105692] Updated weights for policy 0, policy_version 353381 (0.0008) [2023-12-26 17:56:13,661][105620] Updated weights for policy 1, policy_version 353805 (0.0008) [2023-12-26 17:56:13,710][105620] Updated weights for policy 1, policy_version 353815 (0.0010) [2023-12-26 17:56:13,765][105620] Updated weights for policy 1, policy_version 353826 (0.0009) [2023-12-26 17:56:14,167][105692] Updated weights for policy 0, policy_version 353391 (0.0010) [2023-12-26 17:56:14,222][105692] Updated weights for policy 0, policy_version 353401 (0.0008) [2023-12-26 17:56:14,283][105692] Updated weights for policy 0, policy_version 353411 (0.0008) [2023-12-26 17:56:14,389][105620] Updated weights for policy 1, policy_version 353836 (0.0008) [2023-12-26 17:56:14,438][105620] Updated weights for policy 1, policy_version 353846 (0.0011) [2023-12-26 17:56:14,495][105620] Updated weights for policy 1, policy_version 353856 (0.0010) [2023-12-26 17:56:15,055][105692] Updated weights for policy 0, policy_version 353421 (0.0008) [2023-12-26 17:56:15,123][105692] Updated weights for policy 0, policy_version 353431 (0.0008) [2023-12-26 17:56:15,186][105692] Updated weights for policy 0, policy_version 353441 (0.0008) [2023-12-26 17:56:15,271][105620] Updated weights for policy 1, policy_version 353866 (0.0011) [2023-12-26 17:56:15,337][105620] Updated weights for policy 1, policy_version 353876 (0.0011) [2023-12-26 17:56:15,392][105620] Updated weights for policy 1, policy_version 353886 (0.0010) [2023-12-26 17:56:15,457][105620] Updated weights for policy 1, policy_version 353896 (0.0010) [2023-12-26 17:56:15,941][105692] Updated weights for policy 0, policy_version 353451 (0.0008) [2023-12-26 17:56:16,004][105692] Updated weights for policy 0, policy_version 353461 (0.0008) [2023-12-26 17:56:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 181100544. Throughput: 0: 9638.6, 1: 9559.3. Samples: 181078348. Policy #0 lag: (min: 14.0, avg: 14.3, max: 22.0) [2023-12-26 17:56:16,062][104569] Avg episode reward: [(0, '8906.929'), (1, '4677.904')] [2023-12-26 17:56:16,065][105585] KL-divergence is very high: 140.7043 [2023-12-26 17:56:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000353896_90603520.pth... [2023-12-26 17:56:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000352776_90316800.pth [2023-12-26 17:56:16,071][105692] Updated weights for policy 0, policy_version 353471 (0.0008) [2023-12-26 17:56:16,112][105585] KL-divergence is very high: 202.1073 [2023-12-26 17:56:16,122][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000353480_90505216.pth... [2023-12-26 17:56:16,126][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000352360_90218496.pth [2023-12-26 17:56:16,191][105620] Updated weights for policy 1, policy_version 353906 (0.0011) [2023-12-26 17:56:16,252][105620] Updated weights for policy 1, policy_version 353916 (0.0010) [2023-12-26 17:56:16,307][105620] Updated weights for policy 1, policy_version 353926 (0.0010) [2023-12-26 17:56:16,829][105692] Updated weights for policy 0, policy_version 353481 (0.0008) [2023-12-26 17:56:16,883][105692] Updated weights for policy 0, policy_version 353491 (0.0008) [2023-12-26 17:56:16,937][105692] Updated weights for policy 0, policy_version 353501 (0.0008) [2023-12-26 17:56:16,988][105692] Updated weights for policy 0, policy_version 353511 (0.0008) [2023-12-26 17:56:17,030][105620] Updated weights for policy 1, policy_version 353936 (0.0010) [2023-12-26 17:56:17,087][105620] Updated weights for policy 1, policy_version 353946 (0.0010) [2023-12-26 17:56:17,145][105620] Updated weights for policy 1, policy_version 353956 (0.0011) [2023-12-26 17:56:17,773][105692] Updated weights for policy 0, policy_version 353521 (0.0009) [2023-12-26 17:56:17,837][105692] Updated weights for policy 0, policy_version 353531 (0.0008) [2023-12-26 17:56:17,838][105620] Updated weights for policy 1, policy_version 353966 (0.0010) [2023-12-26 17:56:17,891][105692] Updated weights for policy 0, policy_version 353541 (0.0006) [2023-12-26 17:56:17,903][105620] Updated weights for policy 1, policy_version 353976 (0.0010) [2023-12-26 17:56:17,972][105620] Updated weights for policy 1, policy_version 353986 (0.0010) [2023-12-26 17:56:18,634][105620] Updated weights for policy 1, policy_version 353996 (0.0008) [2023-12-26 17:56:18,692][105692] Updated weights for policy 0, policy_version 353551 (0.0007) [2023-12-26 17:56:18,697][105620] Updated weights for policy 1, policy_version 354006 (0.0010) [2023-12-26 17:56:18,747][105620] Updated weights for policy 1, policy_version 354016 (0.0011) [2023-12-26 17:56:18,753][105692] Updated weights for policy 0, policy_version 353561 (0.0006) [2023-12-26 17:56:18,817][105692] Updated weights for policy 0, policy_version 353571 (0.0006) [2023-12-26 17:56:19,439][105620] Updated weights for policy 1, policy_version 354026 (0.0011) [2023-12-26 17:56:19,503][105620] Updated weights for policy 1, policy_version 354036 (0.0008) [2023-12-26 17:56:19,562][105620] Updated weights for policy 1, policy_version 354046 (0.0010) [2023-12-26 17:56:19,565][105692] Updated weights for policy 0, policy_version 353581 (0.0007) [2023-12-26 17:56:19,619][105692] Updated weights for policy 0, policy_version 353591 (0.0005) [2023-12-26 17:56:19,620][105620] Updated weights for policy 1, policy_version 354056 (0.0009) [2023-12-26 17:56:19,683][105692] Updated weights for policy 0, policy_version 353601 (0.0007) [2023-12-26 17:56:20,342][105620] Updated weights for policy 1, policy_version 354066 (0.0006) [2023-12-26 17:56:20,409][105620] Updated weights for policy 1, policy_version 354076 (0.0008) [2023-12-26 17:56:20,432][105692] Updated weights for policy 0, policy_version 353611 (0.0011) [2023-12-26 17:56:20,463][105620] Updated weights for policy 1, policy_version 354086 (0.0008) [2023-12-26 17:56:20,487][105692] Updated weights for policy 0, policy_version 353621 (0.0008) [2023-12-26 17:56:20,538][105692] Updated weights for policy 0, policy_version 353631 (0.0009) [2023-12-26 17:56:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 181198848. Throughput: 0: 9551.9, 1: 9625.9. Samples: 181190868. Policy #0 lag: (min: 14.0, avg: 14.3, max: 22.0) [2023-12-26 17:56:21,063][104569] Avg episode reward: [(0, '9129.080'), (1, '6648.802')] [2023-12-26 17:56:21,170][105620] Updated weights for policy 1, policy_version 354096 (0.0008) [2023-12-26 17:56:21,235][105620] Updated weights for policy 1, policy_version 354106 (0.0009) [2023-12-26 17:56:21,294][105620] Updated weights for policy 1, policy_version 354116 (0.0009) [2023-12-26 17:56:21,316][105692] Updated weights for policy 0, policy_version 353641 (0.0008) [2023-12-26 17:56:21,392][105692] Updated weights for policy 0, policy_version 353651 (0.0010) [2023-12-26 17:56:21,435][105585] KL-divergence is very high: 159.9282 [2023-12-26 17:56:21,449][105692] Updated weights for policy 0, policy_version 353661 (0.0008) [2023-12-26 17:56:21,478][105585] KL-divergence is very high: 101.6484 [2023-12-26 17:56:21,496][105692] Updated weights for policy 0, policy_version 353671 (0.0008) [2023-12-26 17:56:22,075][105620] Updated weights for policy 1, policy_version 354126 (0.0009) [2023-12-26 17:56:22,125][105620] Updated weights for policy 1, policy_version 354136 (0.0008) [2023-12-26 17:56:22,180][105620] Updated weights for policy 1, policy_version 354146 (0.0009) [2023-12-26 17:56:22,262][105692] Updated weights for policy 0, policy_version 353681 (0.0008) [2023-12-26 17:56:22,320][105692] Updated weights for policy 0, policy_version 353691 (0.0009) [2023-12-26 17:56:22,417][105692] Updated weights for policy 0, policy_version 353701 (0.0008) [2023-12-26 17:56:22,998][105692] Updated weights for policy 0, policy_version 353711 (0.0009) [2023-12-26 17:56:23,042][105620] Updated weights for policy 1, policy_version 354156 (0.0009) [2023-12-26 17:56:23,055][105692] Updated weights for policy 0, policy_version 353721 (0.0007) [2023-12-26 17:56:23,094][105620] Updated weights for policy 1, policy_version 354166 (0.0006) [2023-12-26 17:56:23,100][105692] Updated weights for policy 0, policy_version 353731 (0.0006) [2023-12-26 17:56:23,148][105620] Updated weights for policy 1, policy_version 354176 (0.0008) [2023-12-26 17:56:23,857][105692] Updated weights for policy 0, policy_version 353741 (0.0007) [2023-12-26 17:56:23,902][105620] Updated weights for policy 1, policy_version 354186 (0.0009) [2023-12-26 17:56:23,913][105692] Updated weights for policy 0, policy_version 353751 (0.0009) [2023-12-26 17:56:23,961][105620] Updated weights for policy 1, policy_version 354196 (0.0010) [2023-12-26 17:56:23,968][105692] Updated weights for policy 0, policy_version 353761 (0.0008) [2023-12-26 17:56:24,015][105620] Updated weights for policy 1, policy_version 354206 (0.0009) [2023-12-26 17:56:24,066][105620] Updated weights for policy 1, policy_version 354216 (0.0009) [2023-12-26 17:56:24,716][105692] Updated weights for policy 0, policy_version 353771 (0.0009) [2023-12-26 17:56:24,762][105620] Updated weights for policy 1, policy_version 354226 (0.0006) [2023-12-26 17:56:24,764][105692] Updated weights for policy 0, policy_version 353781 (0.0010) [2023-12-26 17:56:24,809][105692] Updated weights for policy 0, policy_version 353791 (0.0010) [2023-12-26 17:56:24,819][105620] Updated weights for policy 1, policy_version 354236 (0.0007) [2023-12-26 17:56:24,875][105620] Updated weights for policy 1, policy_version 354246 (0.0009) [2023-12-26 17:56:25,566][105620] Updated weights for policy 1, policy_version 354256 (0.0009) [2023-12-26 17:56:25,576][105692] Updated weights for policy 0, policy_version 353801 (0.0010) [2023-12-26 17:56:25,626][105620] Updated weights for policy 1, policy_version 354266 (0.0008) [2023-12-26 17:56:25,629][105692] Updated weights for policy 0, policy_version 353811 (0.0005) [2023-12-26 17:56:25,682][105620] Updated weights for policy 1, policy_version 354276 (0.0009) [2023-12-26 17:56:25,684][105692] Updated weights for policy 0, policy_version 353821 (0.0005) [2023-12-26 17:56:25,745][105692] Updated weights for policy 0, policy_version 353831 (0.0005) [2023-12-26 17:56:26,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19114.7, 300 sec: 19438.6). Total num frames: 181297152. Throughput: 0: 9498.9, 1: 9693.5. Samples: 181304108. Policy #0 lag: (min: 14.0, avg: 14.3, max: 22.0) [2023-12-26 17:56:26,063][104569] Avg episode reward: [(0, '9215.855'), (1, '9265.733')] [2023-12-26 17:56:26,332][105692] Updated weights for policy 0, policy_version 353841 (0.0007) [2023-12-26 17:56:26,380][105692] Updated weights for policy 0, policy_version 353851 (0.0007) [2023-12-26 17:56:26,433][105692] Updated weights for policy 0, policy_version 353861 (0.0008) [2023-12-26 17:56:26,523][105620] Updated weights for policy 1, policy_version 354286 (0.0009) [2023-12-26 17:56:26,584][105620] Updated weights for policy 1, policy_version 354296 (0.0010) [2023-12-26 17:56:26,628][105620] Updated weights for policy 1, policy_version 354306 (0.0010) [2023-12-26 17:56:26,984][105692] Updated weights for policy 0, policy_version 353871 (0.0005) [2023-12-26 17:56:27,037][105692] Updated weights for policy 0, policy_version 353881 (0.0005) [2023-12-26 17:56:27,090][105692] Updated weights for policy 0, policy_version 353891 (0.0005) [2023-12-26 17:56:27,207][105620] Updated weights for policy 1, policy_version 354316 (0.0008) [2023-12-26 17:56:27,253][105620] Updated weights for policy 1, policy_version 354326 (0.0005) [2023-12-26 17:56:27,296][105620] Updated weights for policy 1, policy_version 354336 (0.0005) [2023-12-26 17:56:27,646][105692] Updated weights for policy 0, policy_version 353901 (0.0006) [2023-12-26 17:56:27,700][105692] Updated weights for policy 0, policy_version 353911 (0.0006) [2023-12-26 17:56:27,763][105692] Updated weights for policy 0, policy_version 353921 (0.0006) [2023-12-26 17:56:28,050][105620] Updated weights for policy 1, policy_version 354346 (0.0007) [2023-12-26 17:56:28,111][105620] Updated weights for policy 1, policy_version 354356 (0.0009) [2023-12-26 17:56:28,165][105620] Updated weights for policy 1, policy_version 354367 (0.0010) [2023-12-26 17:56:28,348][105692] Updated weights for policy 0, policy_version 353931 (0.0007) [2023-12-26 17:56:28,409][105692] Updated weights for policy 0, policy_version 353941 (0.0007) [2023-12-26 17:56:28,461][105692] Updated weights for policy 0, policy_version 353951 (0.0007) [2023-12-26 17:56:29,043][105620] Updated weights for policy 1, policy_version 354377 (0.0009) [2023-12-26 17:56:29,044][105692] Updated weights for policy 0, policy_version 353961 (0.0008) [2023-12-26 17:56:29,092][105692] Updated weights for policy 0, policy_version 353971 (0.0005) [2023-12-26 17:56:29,094][105620] Updated weights for policy 1, policy_version 354387 (0.0008) [2023-12-26 17:56:29,096][105585] KL-divergence is very high: 165.2361 [2023-12-26 17:56:29,139][105585] KL-divergence is very high: 271.1610 [2023-12-26 17:56:29,145][105692] Updated weights for policy 0, policy_version 353981 (0.0005) [2023-12-26 17:56:29,145][105620] Updated weights for policy 1, policy_version 354397 (0.0008) [2023-12-26 17:56:29,183][105585] KL-divergence is very high: 287.6730 [2023-12-26 17:56:29,201][105692] Updated weights for policy 0, policy_version 353991 (0.0005) [2023-12-26 17:56:29,204][105620] Updated weights for policy 1, policy_version 354407 (0.0009) [2023-12-26 17:56:29,870][105692] Updated weights for policy 0, policy_version 354001 (0.0008) [2023-12-26 17:56:29,937][105692] Updated weights for policy 0, policy_version 354011 (0.0009) [2023-12-26 17:56:29,996][105692] Updated weights for policy 0, policy_version 354021 (0.0007) [2023-12-26 17:56:30,010][105620] Updated weights for policy 1, policy_version 354417 (0.0008) [2023-12-26 17:56:30,059][105620] Updated weights for policy 1, policy_version 354427 (0.0008) [2023-12-26 17:56:30,107][105620] Updated weights for policy 1, policy_version 354437 (0.0009) [2023-12-26 17:56:30,733][105692] Updated weights for policy 0, policy_version 354031 (0.0008) [2023-12-26 17:56:30,791][105692] Updated weights for policy 0, policy_version 354041 (0.0009) [2023-12-26 17:56:30,838][105692] Updated weights for policy 0, policy_version 354051 (0.0010) [2023-12-26 17:56:30,885][105620] Updated weights for policy 1, policy_version 354447 (0.0007) [2023-12-26 17:56:30,948][105620] Updated weights for policy 1, policy_version 354457 (0.0008) [2023-12-26 17:56:31,004][105620] Updated weights for policy 1, policy_version 354467 (0.0010) [2023-12-26 17:56:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 181403648. Throughput: 0: 9654.6, 1: 9704.9. Samples: 181367880. Policy #0 lag: (min: 14.0, avg: 14.3, max: 22.0) [2023-12-26 17:56:31,062][104569] Avg episode reward: [(0, '9005.202'), (1, '9353.298')] [2023-12-26 17:56:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000354472_90750976.pth... [2023-12-26 17:56:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000354056_90652672.pth... [2023-12-26 17:56:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000352936_90365952.pth [2023-12-26 17:56:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000353352_90464256.pth [2023-12-26 17:56:31,596][105692] Updated weights for policy 0, policy_version 354061 (0.0010) [2023-12-26 17:56:31,661][105692] Updated weights for policy 0, policy_version 354071 (0.0009) [2023-12-26 17:56:31,718][105692] Updated weights for policy 0, policy_version 354081 (0.0008) [2023-12-26 17:56:31,738][105620] Updated weights for policy 1, policy_version 354477 (0.0006) [2023-12-26 17:56:31,795][105620] Updated weights for policy 1, policy_version 354487 (0.0005) [2023-12-26 17:56:31,849][105620] Updated weights for policy 1, policy_version 354497 (0.0005) [2023-12-26 17:56:32,403][105620] Updated weights for policy 1, policy_version 354507 (0.0006) [2023-12-26 17:56:32,457][105620] Updated weights for policy 1, policy_version 354517 (0.0008) [2023-12-26 17:56:32,504][105620] Updated weights for policy 1, policy_version 354527 (0.0010) [2023-12-26 17:56:32,567][105692] Updated weights for policy 0, policy_version 354091 (0.0009) [2023-12-26 17:56:32,625][105692] Updated weights for policy 0, policy_version 354101 (0.0009) [2023-12-26 17:56:32,683][105692] Updated weights for policy 0, policy_version 354111 (0.0009) [2023-12-26 17:56:33,275][105620] Updated weights for policy 1, policy_version 354537 (0.0009) [2023-12-26 17:56:33,327][105620] Updated weights for policy 1, policy_version 354547 (0.0007) [2023-12-26 17:56:33,377][105620] Updated weights for policy 1, policy_version 354557 (0.0008) [2023-12-26 17:56:33,431][105620] Updated weights for policy 1, policy_version 354567 (0.0009) [2023-12-26 17:56:33,433][105692] Updated weights for policy 0, policy_version 354121 (0.0008) [2023-12-26 17:56:33,483][105692] Updated weights for policy 0, policy_version 354131 (0.0008) [2023-12-26 17:56:33,530][105692] Updated weights for policy 0, policy_version 354141 (0.0009) [2023-12-26 17:56:33,576][105692] Updated weights for policy 0, policy_version 354151 (0.0009) [2023-12-26 17:56:34,250][105692] Updated weights for policy 0, policy_version 354161 (0.0009) [2023-12-26 17:56:34,256][105620] Updated weights for policy 1, policy_version 354577 (0.0006) [2023-12-26 17:56:34,301][105620] Updated weights for policy 1, policy_version 354587 (0.0007) [2023-12-26 17:56:34,311][105692] Updated weights for policy 0, policy_version 354171 (0.0008) [2023-12-26 17:56:34,358][105620] Updated weights for policy 1, policy_version 354597 (0.0006) [2023-12-26 17:56:34,373][105692] Updated weights for policy 0, policy_version 354181 (0.0007) [2023-12-26 17:56:34,992][105620] Updated weights for policy 1, policy_version 354607 (0.0005) [2023-12-26 17:56:35,057][105620] Updated weights for policy 1, policy_version 354617 (0.0008) [2023-12-26 17:56:35,103][105620] Updated weights for policy 1, policy_version 354627 (0.0009) [2023-12-26 17:56:35,192][105692] Updated weights for policy 0, policy_version 354191 (0.0009) [2023-12-26 17:56:35,243][105692] Updated weights for policy 0, policy_version 354201 (0.0009) [2023-12-26 17:56:35,297][105692] Updated weights for policy 0, policy_version 354211 (0.0009) [2023-12-26 17:56:35,807][105620] Updated weights for policy 1, policy_version 354637 (0.0007) [2023-12-26 17:56:35,855][105620] Updated weights for policy 1, policy_version 354647 (0.0005) [2023-12-26 17:56:35,916][105620] Updated weights for policy 1, policy_version 354657 (0.0005) [2023-12-26 17:56:35,939][105692] Updated weights for policy 0, policy_version 354221 (0.0009) [2023-12-26 17:56:35,987][105585] KL-divergence is very high: 111.5702 [2023-12-26 17:56:36,008][105692] Updated weights for policy 0, policy_version 354231 (0.0007) [2023-12-26 17:56:36,055][105692] Updated weights for policy 0, policy_version 354241 (0.0008) [2023-12-26 17:56:36,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19251.2, 300 sec: 19438.7). Total num frames: 181493760. Throughput: 0: 9640.6, 1: 9698.1. Samples: 181483912. Policy #0 lag: (min: 30.0, avg: 31.6, max: 62.0) [2023-12-26 17:56:36,062][104569] Avg episode reward: [(0, '3674.402'), (1, '9353.047')] [2023-12-26 17:56:36,662][105620] Updated weights for policy 1, policy_version 354667 (0.0006) [2023-12-26 17:56:36,665][105692] Updated weights for policy 0, policy_version 354251 (0.0007) [2023-12-26 17:56:36,721][105692] Updated weights for policy 0, policy_version 354261 (0.0006) [2023-12-26 17:56:36,731][105620] Updated weights for policy 1, policy_version 354677 (0.0008) [2023-12-26 17:56:36,770][105692] Updated weights for policy 0, policy_version 354271 (0.0005) [2023-12-26 17:56:36,798][105620] Updated weights for policy 1, policy_version 354687 (0.0009) [2023-12-26 17:56:37,455][105692] Updated weights for policy 0, policy_version 354281 (0.0006) [2023-12-26 17:56:37,506][105692] Updated weights for policy 0, policy_version 354291 (0.0009) [2023-12-26 17:56:37,567][105692] Updated weights for policy 0, policy_version 354301 (0.0009) [2023-12-26 17:56:37,574][105620] Updated weights for policy 1, policy_version 354697 (0.0008) [2023-12-26 17:56:37,625][105692] Updated weights for policy 0, policy_version 354311 (0.0007) [2023-12-26 17:56:37,627][105620] Updated weights for policy 1, policy_version 354707 (0.0006) [2023-12-26 17:56:37,674][105620] Updated weights for policy 1, policy_version 354717 (0.0009) [2023-12-26 17:56:37,735][105620] Updated weights for policy 1, policy_version 354727 (0.0008) [2023-12-26 17:56:38,355][105692] Updated weights for policy 0, policy_version 354321 (0.0008) [2023-12-26 17:56:38,420][105692] Updated weights for policy 0, policy_version 354331 (0.0008) [2023-12-26 17:56:38,476][105692] Updated weights for policy 0, policy_version 354341 (0.0009) [2023-12-26 17:56:38,507][105620] Updated weights for policy 1, policy_version 354737 (0.0009) [2023-12-26 17:56:38,571][105620] Updated weights for policy 1, policy_version 354747 (0.0009) [2023-12-26 17:56:38,631][105620] Updated weights for policy 1, policy_version 354757 (0.0009) [2023-12-26 17:56:39,236][105692] Updated weights for policy 0, policy_version 354351 (0.0008) [2023-12-26 17:56:39,292][105692] Updated weights for policy 0, policy_version 354361 (0.0011) [2023-12-26 17:56:39,360][105620] Updated weights for policy 1, policy_version 354767 (0.0008) [2023-12-26 17:56:39,362][105692] Updated weights for policy 0, policy_version 354372 (0.0009) [2023-12-26 17:56:39,433][105620] Updated weights for policy 1, policy_version 354777 (0.0009) [2023-12-26 17:56:39,488][105620] Updated weights for policy 1, policy_version 354787 (0.0009) [2023-12-26 17:56:40,060][105692] Updated weights for policy 0, policy_version 354382 (0.0009) [2023-12-26 17:56:40,119][105692] Updated weights for policy 0, policy_version 354392 (0.0009) [2023-12-26 17:56:40,178][105692] Updated weights for policy 0, policy_version 354402 (0.0009) [2023-12-26 17:56:40,224][105620] Updated weights for policy 1, policy_version 354797 (0.0007) [2023-12-26 17:56:40,294][105620] Updated weights for policy 1, policy_version 354807 (0.0006) [2023-12-26 17:56:40,362][105620] Updated weights for policy 1, policy_version 354817 (0.0006) [2023-12-26 17:56:40,922][105692] Updated weights for policy 0, policy_version 354412 (0.0010) [2023-12-26 17:56:40,988][105692] Updated weights for policy 0, policy_version 354422 (0.0011) [2023-12-26 17:56:41,018][105620] Updated weights for policy 1, policy_version 354827 (0.0009) [2023-12-26 17:56:41,052][105692] Updated weights for policy 0, policy_version 354432 (0.0009) [2023-12-26 17:56:41,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19114.7, 300 sec: 19410.9). Total num frames: 181583872. Throughput: 0: 9607.6, 1: 9618.3. Samples: 181599488. Policy #0 lag: (min: 30.0, avg: 31.6, max: 62.0) [2023-12-26 17:56:41,062][104569] Avg episode reward: [(0, '5253.930'), (1, '9263.987')] [2023-12-26 17:56:41,080][105620] Updated weights for policy 1, policy_version 354837 (0.0006) [2023-12-26 17:56:41,147][105620] Updated weights for policy 1, policy_version 354847 (0.0009) [2023-12-26 17:56:41,886][105692] Updated weights for policy 0, policy_version 354442 (0.0009) [2023-12-26 17:56:41,927][105620] Updated weights for policy 1, policy_version 354857 (0.0008) [2023-12-26 17:56:41,948][105692] Updated weights for policy 0, policy_version 354452 (0.0006) [2023-12-26 17:56:41,994][105620] Updated weights for policy 1, policy_version 354867 (0.0009) [2023-12-26 17:56:42,018][105692] Updated weights for policy 0, policy_version 354462 (0.0007) [2023-12-26 17:56:42,055][105620] Updated weights for policy 1, policy_version 354877 (0.0008) [2023-12-26 17:56:42,081][105692] Updated weights for policy 0, policy_version 354472 (0.0007) [2023-12-26 17:56:42,118][105620] Updated weights for policy 1, policy_version 354887 (0.0007) [2023-12-26 17:56:42,685][105692] Updated weights for policy 0, policy_version 354482 (0.0009) [2023-12-26 17:56:42,738][105692] Updated weights for policy 0, policy_version 354492 (0.0007) [2023-12-26 17:56:42,804][105692] Updated weights for policy 0, policy_version 354502 (0.0006) [2023-12-26 17:56:42,881][105620] Updated weights for policy 1, policy_version 354897 (0.0006) [2023-12-26 17:56:42,942][105620] Updated weights for policy 1, policy_version 354907 (0.0006) [2023-12-26 17:56:42,996][105620] Updated weights for policy 1, policy_version 354917 (0.0009) [2023-12-26 17:56:43,410][105692] Updated weights for policy 0, policy_version 354512 (0.0005) [2023-12-26 17:56:43,457][105692] Updated weights for policy 0, policy_version 354522 (0.0005) [2023-12-26 17:56:43,463][105585] KL-divergence is very high: 155.1519 [2023-12-26 17:56:43,472][105585] KL-divergence is very high: 158.2739 [2023-12-26 17:56:43,502][105585] KL-divergence is very high: 146.1083 [2023-12-26 17:56:43,513][105585] KL-divergence is very high: 120.7461 [2023-12-26 17:56:43,515][105692] Updated weights for policy 0, policy_version 354533 (0.0008) [2023-12-26 17:56:43,716][105620] Updated weights for policy 1, policy_version 354927 (0.0009) [2023-12-26 17:56:43,770][105620] Updated weights for policy 1, policy_version 354938 (0.0010) [2023-12-26 17:56:43,822][105620] Updated weights for policy 1, policy_version 354948 (0.0010) [2023-12-26 17:56:44,098][105692] Updated weights for policy 0, policy_version 354543 (0.0006) [2023-12-26 17:56:44,146][105692] Updated weights for policy 0, policy_version 354553 (0.0005) [2023-12-26 17:56:44,192][105692] Updated weights for policy 0, policy_version 354563 (0.0005) [2023-12-26 17:56:44,725][105620] Updated weights for policy 1, policy_version 354959 (0.0007) [2023-12-26 17:56:44,792][105620] Updated weights for policy 1, policy_version 354969 (0.0009) [2023-12-26 17:56:44,835][105692] Updated weights for policy 0, policy_version 354573 (0.0008) [2023-12-26 17:56:44,842][105620] Updated weights for policy 1, policy_version 354979 (0.0006) [2023-12-26 17:56:44,863][105585] KL-divergence is very high: 131.5665 [2023-12-26 17:56:44,902][105692] Updated weights for policy 0, policy_version 354583 (0.0011) [2023-12-26 17:56:44,913][105585] KL-divergence is very high: 153.2557 [2023-12-26 17:56:44,965][105692] Updated weights for policy 0, policy_version 354593 (0.0010) [2023-12-26 17:56:44,966][105585] KL-divergence is very high: 121.3615 [2023-12-26 17:56:45,521][105620] Updated weights for policy 1, policy_version 354989 (0.0007) [2023-12-26 17:56:45,586][105620] Updated weights for policy 1, policy_version 354999 (0.0008) [2023-12-26 17:56:45,641][105620] Updated weights for policy 1, policy_version 355009 (0.0008) [2023-12-26 17:56:45,673][105692] Updated weights for policy 0, policy_version 354603 (0.0007) [2023-12-26 17:56:45,733][105692] Updated weights for policy 0, policy_version 354613 (0.0010) [2023-12-26 17:56:45,794][105692] Updated weights for policy 0, policy_version 354623 (0.0010) [2023-12-26 17:56:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 181690368. Throughput: 0: 9640.5, 1: 9554.1. Samples: 181657068. Policy #0 lag: (min: 30.0, avg: 31.6, max: 62.0) [2023-12-26 17:56:46,062][104569] Avg episode reward: [(0, '6303.607'), (1, '2167.944')] [2023-12-26 17:56:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000355016_90890240.pth... [2023-12-26 17:56:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000354632_90800128.pth... [2023-12-26 17:56:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000353480_90505216.pth [2023-12-26 17:56:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000353896_90603520.pth [2023-12-26 17:56:46,389][105620] Updated weights for policy 1, policy_version 355019 (0.0008) [2023-12-26 17:56:46,441][105620] Updated weights for policy 1, policy_version 355029 (0.0008) [2023-12-26 17:56:46,492][105620] Updated weights for policy 1, policy_version 355039 (0.0008) [2023-12-26 17:56:46,523][105692] Updated weights for policy 0, policy_version 354633 (0.0008) [2023-12-26 17:56:46,585][105692] Updated weights for policy 0, policy_version 354643 (0.0009) [2023-12-26 17:56:46,644][105692] Updated weights for policy 0, policy_version 354653 (0.0009) [2023-12-26 17:56:46,693][105692] Updated weights for policy 0, policy_version 354663 (0.0005) [2023-12-26 17:56:47,246][105620] Updated weights for policy 1, policy_version 355049 (0.0008) [2023-12-26 17:56:47,298][105620] Updated weights for policy 1, policy_version 355059 (0.0008) [2023-12-26 17:56:47,336][105692] Updated weights for policy 0, policy_version 354673 (0.0006) [2023-12-26 17:56:47,357][105620] Updated weights for policy 1, policy_version 355069 (0.0009) [2023-12-26 17:56:47,394][105692] Updated weights for policy 0, policy_version 354683 (0.0005) [2023-12-26 17:56:47,416][105620] Updated weights for policy 1, policy_version 355079 (0.0008) [2023-12-26 17:56:47,448][105692] Updated weights for policy 0, policy_version 354693 (0.0005) [2023-12-26 17:56:47,976][105692] Updated weights for policy 0, policy_version 354703 (0.0005) [2023-12-26 17:56:48,039][105692] Updated weights for policy 0, policy_version 354713 (0.0005) [2023-12-26 17:56:48,107][105692] Updated weights for policy 0, policy_version 354723 (0.0006) [2023-12-26 17:56:48,267][105620] Updated weights for policy 1, policy_version 355089 (0.0009) [2023-12-26 17:56:48,321][105620] Updated weights for policy 1, policy_version 355099 (0.0009) [2023-12-26 17:56:48,382][105620] Updated weights for policy 1, policy_version 355109 (0.0008) [2023-12-26 17:56:48,768][105692] Updated weights for policy 0, policy_version 354733 (0.0008) [2023-12-26 17:56:48,825][105692] Updated weights for policy 0, policy_version 354743 (0.0005) [2023-12-26 17:56:48,888][105692] Updated weights for policy 0, policy_version 354753 (0.0007) [2023-12-26 17:56:49,188][105620] Updated weights for policy 1, policy_version 355119 (0.0009) [2023-12-26 17:56:49,250][105620] Updated weights for policy 1, policy_version 355129 (0.0009) [2023-12-26 17:56:49,306][105620] Updated weights for policy 1, policy_version 355139 (0.0008) [2023-12-26 17:56:49,573][105692] Updated weights for policy 0, policy_version 354763 (0.0009) [2023-12-26 17:56:49,628][105692] Updated weights for policy 0, policy_version 354773 (0.0009) [2023-12-26 17:56:49,676][105692] Updated weights for policy 0, policy_version 354783 (0.0009) [2023-12-26 17:56:50,069][105620] Updated weights for policy 1, policy_version 355149 (0.0009) [2023-12-26 17:56:50,139][105620] Updated weights for policy 1, policy_version 355159 (0.0010) [2023-12-26 17:56:50,199][105620] Updated weights for policy 1, policy_version 355169 (0.0010) [2023-12-26 17:56:50,386][105692] Updated weights for policy 0, policy_version 354793 (0.0010) [2023-12-26 17:56:50,444][105692] Updated weights for policy 0, policy_version 354803 (0.0009) [2023-12-26 17:56:50,499][105692] Updated weights for policy 0, policy_version 354813 (0.0009) [2023-12-26 17:56:50,556][105692] Updated weights for policy 0, policy_version 354823 (0.0009) [2023-12-26 17:56:50,886][105620] Updated weights for policy 1, policy_version 355179 (0.0007) [2023-12-26 17:56:50,937][105620] Updated weights for policy 1, policy_version 355189 (0.0005) [2023-12-26 17:56:50,994][105620] Updated weights for policy 1, policy_version 355199 (0.0005) [2023-12-26 17:56:51,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 181788672. Throughput: 0: 9760.2, 1: 9494.0. Samples: 181775424. Policy #0 lag: (min: 30.0, avg: 31.6, max: 62.0) [2023-12-26 17:56:51,063][104569] Avg episode reward: [(0, '7564.462'), (1, '1059.055')] [2023-12-26 17:56:51,299][105692] Updated weights for policy 0, policy_version 354833 (0.0009) [2023-12-26 17:56:51,362][105692] Updated weights for policy 0, policy_version 354843 (0.0008) [2023-12-26 17:56:51,431][105692] Updated weights for policy 0, policy_version 354853 (0.0009) [2023-12-26 17:56:51,683][105620] Updated weights for policy 1, policy_version 355209 (0.0009) [2023-12-26 17:56:51,754][105620] Updated weights for policy 1, policy_version 355219 (0.0010) [2023-12-26 17:56:51,821][105620] Updated weights for policy 1, policy_version 355229 (0.0011) [2023-12-26 17:56:51,876][105620] Updated weights for policy 1, policy_version 355239 (0.0010) [2023-12-26 17:56:52,127][105692] Updated weights for policy 0, policy_version 354863 (0.0007) [2023-12-26 17:56:52,186][105692] Updated weights for policy 0, policy_version 354873 (0.0006) [2023-12-26 17:56:52,245][105692] Updated weights for policy 0, policy_version 354883 (0.0006) [2023-12-26 17:56:52,528][105620] Updated weights for policy 1, policy_version 355249 (0.0008) [2023-12-26 17:56:52,586][105620] Updated weights for policy 1, policy_version 355259 (0.0005) [2023-12-26 17:56:52,652][105620] Updated weights for policy 1, policy_version 355269 (0.0011) [2023-12-26 17:56:52,809][105692] Updated weights for policy 0, policy_version 354893 (0.0008) [2023-12-26 17:56:52,857][105692] Updated weights for policy 0, policy_version 354903 (0.0005) [2023-12-26 17:56:52,911][105692] Updated weights for policy 0, policy_version 354913 (0.0005) [2023-12-26 17:56:53,265][105620] Updated weights for policy 1, policy_version 355279 (0.0009) [2023-12-26 17:56:53,323][105620] Updated weights for policy 1, policy_version 355289 (0.0010) [2023-12-26 17:56:53,380][105620] Updated weights for policy 1, policy_version 355299 (0.0010) [2023-12-26 17:56:53,533][105692] Updated weights for policy 0, policy_version 354923 (0.0007) [2023-12-26 17:56:53,585][105692] Updated weights for policy 0, policy_version 354933 (0.0010) [2023-12-26 17:56:53,629][105692] Updated weights for policy 0, policy_version 354943 (0.0010) [2023-12-26 17:56:54,102][105620] Updated weights for policy 1, policy_version 355309 (0.0010) [2023-12-26 17:56:54,165][105620] Updated weights for policy 1, policy_version 355319 (0.0010) [2023-12-26 17:56:54,223][105620] Updated weights for policy 1, policy_version 355329 (0.0010) [2023-12-26 17:56:54,388][105692] Updated weights for policy 0, policy_version 354953 (0.0010) [2023-12-26 17:56:54,440][105692] Updated weights for policy 0, policy_version 354963 (0.0010) [2023-12-26 17:56:54,491][105692] Updated weights for policy 0, policy_version 354973 (0.0010) [2023-12-26 17:56:54,553][105692] Updated weights for policy 0, policy_version 354983 (0.0010) [2023-12-26 17:56:54,961][105620] Updated weights for policy 1, policy_version 355339 (0.0009) [2023-12-26 17:56:55,021][105620] Updated weights for policy 1, policy_version 355349 (0.0011) [2023-12-26 17:56:55,087][105620] Updated weights for policy 1, policy_version 355359 (0.0010) [2023-12-26 17:56:55,269][105692] Updated weights for policy 0, policy_version 354993 (0.0011) [2023-12-26 17:56:55,326][105692] Updated weights for policy 0, policy_version 355003 (0.0010) [2023-12-26 17:56:55,381][105692] Updated weights for policy 0, policy_version 355013 (0.0010) [2023-12-26 17:56:55,830][105620] Updated weights for policy 1, policy_version 355369 (0.0011) [2023-12-26 17:56:55,885][105620] Updated weights for policy 1, policy_version 355379 (0.0010) [2023-12-26 17:56:55,936][105692] Updated weights for policy 0, policy_version 355023 (0.0005) [2023-12-26 17:56:55,941][105620] Updated weights for policy 1, policy_version 355389 (0.0010) [2023-12-26 17:56:55,991][105692] Updated weights for policy 0, policy_version 355033 (0.0005) [2023-12-26 17:56:55,993][105620] Updated weights for policy 1, policy_version 355399 (0.0010) [2023-12-26 17:56:56,056][105692] Updated weights for policy 0, policy_version 355043 (0.0005) [2023-12-26 17:56:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 181886976. Throughput: 0: 9892.5, 1: 9487.6. Samples: 181895440. Policy #0 lag: (min: 30.0, avg: 31.6, max: 62.0) [2023-12-26 17:56:56,062][104569] Avg episode reward: [(0, '8492.672'), (1, '6386.600')] [2023-12-26 17:56:56,637][105692] Updated weights for policy 0, policy_version 355053 (0.0008) [2023-12-26 17:56:56,690][105692] Updated weights for policy 0, policy_version 355063 (0.0010) [2023-12-26 17:56:56,692][105620] Updated weights for policy 1, policy_version 355409 (0.0010) [2023-12-26 17:56:56,740][105620] Updated weights for policy 1, policy_version 355419 (0.0010) [2023-12-26 17:56:56,747][105692] Updated weights for policy 0, policy_version 355073 (0.0010) [2023-12-26 17:56:56,788][105620] Updated weights for policy 1, policy_version 355429 (0.0010) [2023-12-26 17:56:57,429][105692] Updated weights for policy 0, policy_version 355083 (0.0009) [2023-12-26 17:56:57,486][105692] Updated weights for policy 0, policy_version 355093 (0.0005) [2023-12-26 17:56:57,544][105692] Updated weights for policy 0, policy_version 355103 (0.0007) [2023-12-26 17:56:57,552][105620] Updated weights for policy 1, policy_version 355439 (0.0010) [2023-12-26 17:56:57,620][105620] Updated weights for policy 1, policy_version 355449 (0.0010) [2023-12-26 17:56:57,683][105620] Updated weights for policy 1, policy_version 355459 (0.0010) [2023-12-26 17:56:58,200][105692] Updated weights for policy 0, policy_version 355113 (0.0007) [2023-12-26 17:56:58,258][105692] Updated weights for policy 0, policy_version 355123 (0.0010) [2023-12-26 17:56:58,317][105692] Updated weights for policy 0, policy_version 355133 (0.0010) [2023-12-26 17:56:58,394][105692] Updated weights for policy 0, policy_version 355143 (0.0012) [2023-12-26 17:56:58,434][105620] Updated weights for policy 1, policy_version 355469 (0.0009) [2023-12-26 17:56:58,499][105620] Updated weights for policy 1, policy_version 355479 (0.0008) [2023-12-26 17:56:58,567][105620] Updated weights for policy 1, policy_version 355489 (0.0006) [2023-12-26 17:56:59,195][105692] Updated weights for policy 0, policy_version 355153 (0.0006) [2023-12-26 17:56:59,258][105620] Updated weights for policy 1, policy_version 355499 (0.0008) [2023-12-26 17:56:59,262][105692] Updated weights for policy 0, policy_version 355163 (0.0010) [2023-12-26 17:56:59,320][105692] Updated weights for policy 0, policy_version 355173 (0.0012) [2023-12-26 17:56:59,321][105620] Updated weights for policy 1, policy_version 355509 (0.0011) [2023-12-26 17:56:59,387][105620] Updated weights for policy 1, policy_version 355519 (0.0008) [2023-12-26 17:57:00,042][105620] Updated weights for policy 1, policy_version 355529 (0.0006) [2023-12-26 17:57:00,095][105620] Updated weights for policy 1, policy_version 355539 (0.0007) [2023-12-26 17:57:00,116][105692] Updated weights for policy 0, policy_version 355183 (0.0010) [2023-12-26 17:57:00,152][105620] Updated weights for policy 1, policy_version 355549 (0.0010) [2023-12-26 17:57:00,176][105692] Updated weights for policy 0, policy_version 355193 (0.0009) [2023-12-26 17:57:00,211][105620] Updated weights for policy 1, policy_version 355559 (0.0010) [2023-12-26 17:57:00,239][105692] Updated weights for policy 0, policy_version 355203 (0.0005) [2023-12-26 17:57:00,888][105620] Updated weights for policy 1, policy_version 355569 (0.0006) [2023-12-26 17:57:00,953][105620] Updated weights for policy 1, policy_version 355579 (0.0006) [2023-12-26 17:57:00,983][105692] Updated weights for policy 0, policy_version 355213 (0.0009) [2023-12-26 17:57:01,008][105620] Updated weights for policy 1, policy_version 355589 (0.0006) [2023-12-26 17:57:01,046][105692] Updated weights for policy 0, policy_version 355223 (0.0011) [2023-12-26 17:57:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 181985280. Throughput: 0: 10007.4, 1: 9497.6. Samples: 181956072. Policy #0 lag: (min: 30.0, avg: 31.6, max: 62.0) [2023-12-26 17:57:01,062][104569] Avg episode reward: [(0, '8899.815'), (1, '2614.640')] [2023-12-26 17:57:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000355592_91037696.pth... [2023-12-26 17:57:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000354472_90750976.pth [2023-12-26 17:57:01,101][105692] Updated weights for policy 0, policy_version 355233 (0.0010) [2023-12-26 17:57:01,145][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000355240_90955776.pth... [2023-12-26 17:57:01,148][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000354056_90652672.pth [2023-12-26 17:57:01,800][105620] Updated weights for policy 1, policy_version 355599 (0.0009) [2023-12-26 17:57:01,827][105692] Updated weights for policy 0, policy_version 355243 (0.0011) [2023-12-26 17:57:01,865][105620] Updated weights for policy 1, policy_version 355609 (0.0008) [2023-12-26 17:57:01,878][105692] Updated weights for policy 0, policy_version 355253 (0.0009) [2023-12-26 17:57:01,924][105620] Updated weights for policy 1, policy_version 355619 (0.0008) [2023-12-26 17:57:01,939][105692] Updated weights for policy 0, policy_version 355263 (0.0008) [2023-12-26 17:57:02,631][105692] Updated weights for policy 0, policy_version 355273 (0.0008) [2023-12-26 17:57:02,693][105620] Updated weights for policy 1, policy_version 355629 (0.0007) [2023-12-26 17:57:02,699][105692] Updated weights for policy 0, policy_version 355283 (0.0007) [2023-12-26 17:57:02,750][105620] Updated weights for policy 1, policy_version 355639 (0.0007) [2023-12-26 17:57:02,769][105692] Updated weights for policy 0, policy_version 355293 (0.0006) [2023-12-26 17:57:02,808][105620] Updated weights for policy 1, policy_version 355649 (0.0008) [2023-12-26 17:57:02,835][105692] Updated weights for policy 0, policy_version 355303 (0.0006) [2023-12-26 17:57:03,382][105692] Updated weights for policy 0, policy_version 355313 (0.0005) [2023-12-26 17:57:03,421][105620] Updated weights for policy 1, policy_version 355659 (0.0006) [2023-12-26 17:57:03,438][105692] Updated weights for policy 0, policy_version 355323 (0.0005) [2023-12-26 17:57:03,481][105620] Updated weights for policy 1, policy_version 355669 (0.0005) [2023-12-26 17:57:03,505][105692] Updated weights for policy 0, policy_version 355333 (0.0007) [2023-12-26 17:57:03,528][105620] Updated weights for policy 1, policy_version 355679 (0.0006) [2023-12-26 17:57:04,203][105692] Updated weights for policy 0, policy_version 355343 (0.0008) [2023-12-26 17:57:04,209][105620] Updated weights for policy 1, policy_version 355689 (0.0010) [2023-12-26 17:57:04,255][105692] Updated weights for policy 0, policy_version 355353 (0.0006) [2023-12-26 17:57:04,261][105620] Updated weights for policy 1, policy_version 355699 (0.0010) [2023-12-26 17:57:04,311][105692] Updated weights for policy 0, policy_version 355363 (0.0008) [2023-12-26 17:57:04,323][105620] Updated weights for policy 1, policy_version 355709 (0.0010) [2023-12-26 17:57:04,376][105620] Updated weights for policy 1, policy_version 355719 (0.0010) [2023-12-26 17:57:05,074][105692] Updated weights for policy 0, policy_version 355373 (0.0009) [2023-12-26 17:57:05,124][105692] Updated weights for policy 0, policy_version 355383 (0.0008) [2023-12-26 17:57:05,130][105620] Updated weights for policy 1, policy_version 355729 (0.0010) [2023-12-26 17:57:05,176][105692] Updated weights for policy 0, policy_version 355393 (0.0006) [2023-12-26 17:57:05,189][105620] Updated weights for policy 1, policy_version 355739 (0.0010) [2023-12-26 17:57:05,236][105620] Updated weights for policy 1, policy_version 355749 (0.0010) [2023-12-26 17:57:05,860][105692] Updated weights for policy 0, policy_version 355403 (0.0006) [2023-12-26 17:57:05,910][105692] Updated weights for policy 0, policy_version 355413 (0.0005) [2023-12-26 17:57:05,957][105692] Updated weights for policy 0, policy_version 355423 (0.0007) [2023-12-26 17:57:05,989][105620] Updated weights for policy 1, policy_version 355759 (0.0010) [2023-12-26 17:57:06,047][105620] Updated weights for policy 1, policy_version 355769 (0.0010) [2023-12-26 17:57:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 182083584. Throughput: 0: 10079.0, 1: 9513.0. Samples: 182072512. Policy #0 lag: (min: 30.0, avg: 31.6, max: 62.0) [2023-12-26 17:57:06,062][104569] Avg episode reward: [(0, '8903.222'), (1, '3450.810')] [2023-12-26 17:57:06,116][105620] Updated weights for policy 1, policy_version 355779 (0.0011) [2023-12-26 17:57:06,708][105692] Updated weights for policy 0, policy_version 355433 (0.0008) [2023-12-26 17:57:06,772][105692] Updated weights for policy 0, policy_version 355443 (0.0008) [2023-12-26 17:57:06,836][105692] Updated weights for policy 0, policy_version 355453 (0.0010) [2023-12-26 17:57:06,852][105620] Updated weights for policy 1, policy_version 355789 (0.0011) [2023-12-26 17:57:06,891][105692] Updated weights for policy 0, policy_version 355463 (0.0010) [2023-12-26 17:57:06,921][105620] Updated weights for policy 1, policy_version 355799 (0.0009) [2023-12-26 17:57:06,970][105620] Updated weights for policy 1, policy_version 355809 (0.0010) [2023-12-26 17:57:07,543][105692] Updated weights for policy 0, policy_version 355473 (0.0006) [2023-12-26 17:57:07,594][105692] Updated weights for policy 0, policy_version 355483 (0.0006) [2023-12-26 17:57:07,639][105620] Updated weights for policy 1, policy_version 355819 (0.0007) [2023-12-26 17:57:07,644][105692] Updated weights for policy 0, policy_version 355493 (0.0005) [2023-12-26 17:57:07,695][105620] Updated weights for policy 1, policy_version 355829 (0.0005) [2023-12-26 17:57:07,757][105620] Updated weights for policy 1, policy_version 355839 (0.0010) [2023-12-26 17:57:08,223][105692] Updated weights for policy 0, policy_version 355503 (0.0005) [2023-12-26 17:57:08,272][105692] Updated weights for policy 0, policy_version 355513 (0.0009) [2023-12-26 17:57:08,318][105692] Updated weights for policy 0, policy_version 355523 (0.0010) [2023-12-26 17:57:08,321][105620] Updated weights for policy 1, policy_version 355849 (0.0010) [2023-12-26 17:57:08,383][105620] Updated weights for policy 1, policy_version 355859 (0.0011) [2023-12-26 17:57:08,435][105620] Updated weights for policy 1, policy_version 355869 (0.0010) [2023-12-26 17:57:08,488][105620] Updated weights for policy 1, policy_version 355879 (0.0010) [2023-12-26 17:57:09,033][105692] Updated weights for policy 0, policy_version 355533 (0.0007) [2023-12-26 17:57:09,093][105692] Updated weights for policy 0, policy_version 355543 (0.0006) [2023-12-26 17:57:09,149][105692] Updated weights for policy 0, policy_version 355553 (0.0007) [2023-12-26 17:57:09,168][105620] Updated weights for policy 1, policy_version 355889 (0.0006) [2023-12-26 17:57:09,237][105620] Updated weights for policy 1, policy_version 355899 (0.0006) [2023-12-26 17:57:09,296][105620] Updated weights for policy 1, policy_version 355909 (0.0010) [2023-12-26 17:57:09,776][105692] Updated weights for policy 0, policy_version 355563 (0.0007) [2023-12-26 17:57:09,843][105692] Updated weights for policy 0, policy_version 355573 (0.0009) [2023-12-26 17:57:09,898][105692] Updated weights for policy 0, policy_version 355583 (0.0008) [2023-12-26 17:57:10,017][105620] Updated weights for policy 1, policy_version 355919 (0.0009) [2023-12-26 17:57:10,082][105620] Updated weights for policy 1, policy_version 355929 (0.0008) [2023-12-26 17:57:10,139][105620] Updated weights for policy 1, policy_version 355939 (0.0006) [2023-12-26 17:57:10,726][105692] Updated weights for policy 0, policy_version 355593 (0.0008) [2023-12-26 17:57:10,781][105692] Updated weights for policy 0, policy_version 355603 (0.0010) [2023-12-26 17:57:10,796][105620] Updated weights for policy 1, policy_version 355949 (0.0007) [2023-12-26 17:57:10,843][105692] Updated weights for policy 0, policy_version 355613 (0.0008) [2023-12-26 17:57:10,852][105620] Updated weights for policy 1, policy_version 355959 (0.0006) [2023-12-26 17:57:10,904][105692] Updated weights for policy 0, policy_version 355623 (0.0007) [2023-12-26 17:57:10,915][105620] Updated weights for policy 1, policy_version 355969 (0.0006) [2023-12-26 17:57:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 182190080. Throughput: 0: 10173.2, 1: 9607.8. Samples: 182194248. Policy #0 lag: (min: 30.0, avg: 31.6, max: 62.0) [2023-12-26 17:57:11,062][104569] Avg episode reward: [(0, '7927.004'), (1, '6758.191')] [2023-12-26 17:57:11,607][105692] Updated weights for policy 0, policy_version 355633 (0.0006) [2023-12-26 17:57:11,622][105585] KL-divergence is very high: 110.8390 [2023-12-26 17:57:11,678][105585] KL-divergence is very high: 168.6510 [2023-12-26 17:57:11,679][105620] Updated weights for policy 1, policy_version 355979 (0.0008) [2023-12-26 17:57:11,680][105692] Updated weights for policy 0, policy_version 355643 (0.0008) [2023-12-26 17:57:11,747][105620] Updated weights for policy 1, policy_version 355989 (0.0006) [2023-12-26 17:57:11,751][105585] KL-divergence is very high: 134.4628 [2023-12-26 17:57:11,764][105692] Updated weights for policy 0, policy_version 355653 (0.0007) [2023-12-26 17:57:11,817][105620] Updated weights for policy 1, policy_version 355999 (0.0006) [2023-12-26 17:57:12,378][105692] Updated weights for policy 0, policy_version 355663 (0.0007) [2023-12-26 17:57:12,389][105620] Updated weights for policy 1, policy_version 356009 (0.0006) [2023-12-26 17:57:12,440][105692] Updated weights for policy 0, policy_version 355673 (0.0008) [2023-12-26 17:57:12,457][105620] Updated weights for policy 1, policy_version 356019 (0.0008) [2023-12-26 17:57:12,498][105692] Updated weights for policy 0, policy_version 355683 (0.0008) [2023-12-26 17:57:12,516][105620] Updated weights for policy 1, policy_version 356029 (0.0008) [2023-12-26 17:57:12,578][105620] Updated weights for policy 1, policy_version 356039 (0.0006) [2023-12-26 17:57:13,221][105620] Updated weights for policy 1, policy_version 356049 (0.0008) [2023-12-26 17:57:13,280][105620] Updated weights for policy 1, policy_version 356059 (0.0009) [2023-12-26 17:57:13,287][105692] Updated weights for policy 0, policy_version 355693 (0.0008) [2023-12-26 17:57:13,334][105620] Updated weights for policy 1, policy_version 356069 (0.0008) [2023-12-26 17:57:13,340][105692] Updated weights for policy 0, policy_version 355703 (0.0006) [2023-12-26 17:57:13,393][105692] Updated weights for policy 0, policy_version 355713 (0.0009) [2023-12-26 17:57:13,996][105620] Updated weights for policy 1, policy_version 356079 (0.0008) [2023-12-26 17:57:14,045][105620] Updated weights for policy 1, policy_version 356089 (0.0008) [2023-12-26 17:57:14,092][105620] Updated weights for policy 1, policy_version 356099 (0.0009) [2023-12-26 17:57:14,196][105692] Updated weights for policy 0, policy_version 355723 (0.0009) [2023-12-26 17:57:14,250][105692] Updated weights for policy 0, policy_version 355734 (0.0010) [2023-12-26 17:57:14,304][105692] Updated weights for policy 0, policy_version 355744 (0.0009) [2023-12-26 17:57:14,734][105620] Updated weights for policy 1, policy_version 356109 (0.0007) [2023-12-26 17:57:14,798][105620] Updated weights for policy 1, policy_version 356119 (0.0006) [2023-12-26 17:57:14,846][105620] Updated weights for policy 1, policy_version 356129 (0.0008) [2023-12-26 17:57:15,094][105692] Updated weights for policy 0, policy_version 355754 (0.0009) [2023-12-26 17:57:15,156][105692] Updated weights for policy 0, policy_version 355764 (0.0009) [2023-12-26 17:57:15,221][105692] Updated weights for policy 0, policy_version 355774 (0.0010) [2023-12-26 17:57:15,284][105692] Updated weights for policy 0, policy_version 355784 (0.0010) [2023-12-26 17:57:15,467][105620] Updated weights for policy 1, policy_version 356139 (0.0008) [2023-12-26 17:57:15,535][105620] Updated weights for policy 1, policy_version 356149 (0.0007) [2023-12-26 17:57:15,600][105620] Updated weights for policy 1, policy_version 356159 (0.0010) [2023-12-26 17:57:15,934][105692] Updated weights for policy 0, policy_version 355794 (0.0005) [2023-12-26 17:57:16,001][105692] Updated weights for policy 0, policy_version 355804 (0.0005) [2023-12-26 17:57:16,053][105692] Updated weights for policy 0, policy_version 355814 (0.0008) [2023-12-26 17:57:16,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 182288384. Throughput: 0: 10021.6, 1: 9654.1. Samples: 182253288. Policy #0 lag: (min: 30.0, avg: 31.6, max: 62.0) [2023-12-26 17:57:16,062][104569] Avg episode reward: [(0, '7547.747'), (1, '9351.300')] [2023-12-26 17:57:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000355816_91103232.pth... [2023-12-26 17:57:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000356168_91185152.pth... [2023-12-26 17:57:16,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000354632_90800128.pth [2023-12-26 17:57:16,069][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000355016_90890240.pth [2023-12-26 17:57:16,302][105620] Updated weights for policy 1, policy_version 356169 (0.0006) [2023-12-26 17:57:16,350][105620] Updated weights for policy 1, policy_version 356179 (0.0005) [2023-12-26 17:57:16,413][105620] Updated weights for policy 1, policy_version 356189 (0.0007) [2023-12-26 17:57:16,466][105620] Updated weights for policy 1, policy_version 356199 (0.0009) [2023-12-26 17:57:16,690][105692] Updated weights for policy 0, policy_version 355824 (0.0009) [2023-12-26 17:57:16,740][105692] Updated weights for policy 0, policy_version 355834 (0.0008) [2023-12-26 17:57:16,795][105692] Updated weights for policy 0, policy_version 355844 (0.0009) [2023-12-26 17:57:17,185][105620] Updated weights for policy 1, policy_version 356209 (0.0009) [2023-12-26 17:57:17,246][105620] Updated weights for policy 1, policy_version 356219 (0.0009) [2023-12-26 17:57:17,303][105620] Updated weights for policy 1, policy_version 356229 (0.0009) [2023-12-26 17:57:17,584][105692] Updated weights for policy 0, policy_version 355854 (0.0009) [2023-12-26 17:57:17,641][105692] Updated weights for policy 0, policy_version 355864 (0.0009) [2023-12-26 17:57:17,699][105692] Updated weights for policy 0, policy_version 355874 (0.0009) [2023-12-26 17:57:18,000][105620] Updated weights for policy 1, policy_version 356239 (0.0008) [2023-12-26 17:57:18,062][105620] Updated weights for policy 1, policy_version 356249 (0.0009) [2023-12-26 17:57:18,124][105620] Updated weights for policy 1, policy_version 356259 (0.0009) [2023-12-26 17:57:18,445][105692] Updated weights for policy 0, policy_version 355884 (0.0009) [2023-12-26 17:57:18,492][105692] Updated weights for policy 0, policy_version 355894 (0.0009) [2023-12-26 17:57:18,543][105692] Updated weights for policy 0, policy_version 355904 (0.0009) [2023-12-26 17:57:18,892][105620] Updated weights for policy 1, policy_version 356269 (0.0009) [2023-12-26 17:57:18,940][105620] Updated weights for policy 1, policy_version 356279 (0.0008) [2023-12-26 17:57:18,986][105620] Updated weights for policy 1, policy_version 356289 (0.0005) [2023-12-26 17:57:19,371][105692] Updated weights for policy 0, policy_version 355914 (0.0008) [2023-12-26 17:57:19,430][105692] Updated weights for policy 0, policy_version 355924 (0.0008) [2023-12-26 17:57:19,480][105692] Updated weights for policy 0, policy_version 355934 (0.0008) [2023-12-26 17:57:19,539][105692] Updated weights for policy 0, policy_version 355944 (0.0008) [2023-12-26 17:57:19,737][105620] Updated weights for policy 1, policy_version 356299 (0.0008) [2023-12-26 17:57:19,796][105620] Updated weights for policy 1, policy_version 356309 (0.0011) [2023-12-26 17:57:19,860][105620] Updated weights for policy 1, policy_version 356319 (0.0011) [2023-12-26 17:57:20,290][105692] Updated weights for policy 0, policy_version 355954 (0.0007) [2023-12-26 17:57:20,347][105692] Updated weights for policy 0, policy_version 355964 (0.0009) [2023-12-26 17:57:20,416][105692] Updated weights for policy 0, policy_version 355974 (0.0010) [2023-12-26 17:57:20,636][105620] Updated weights for policy 1, policy_version 356329 (0.0009) [2023-12-26 17:57:20,702][105620] Updated weights for policy 1, policy_version 356339 (0.0009) [2023-12-26 17:57:20,757][105620] Updated weights for policy 1, policy_version 356349 (0.0008) [2023-12-26 17:57:20,815][105620] Updated weights for policy 1, policy_version 356359 (0.0009) [2023-12-26 17:57:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 182378496. Throughput: 0: 9991.2, 1: 9682.4. Samples: 182369224. Policy #0 lag: (min: 30.0, avg: 31.6, max: 62.0) [2023-12-26 17:57:21,062][104569] Avg episode reward: [(0, '7906.609'), (1, '9257.438')] [2023-12-26 17:57:21,114][105692] Updated weights for policy 0, policy_version 355984 (0.0007) [2023-12-26 17:57:21,178][105692] Updated weights for policy 0, policy_version 355994 (0.0007) [2023-12-26 17:57:21,237][105692] Updated weights for policy 0, policy_version 356004 (0.0006) [2023-12-26 17:57:21,649][105620] Updated weights for policy 1, policy_version 356369 (0.0009) [2023-12-26 17:57:21,713][105620] Updated weights for policy 1, policy_version 356379 (0.0008) [2023-12-26 17:57:21,779][105620] Updated weights for policy 1, policy_version 356389 (0.0008) [2023-12-26 17:57:21,958][105692] Updated weights for policy 0, policy_version 356014 (0.0008) [2023-12-26 17:57:22,006][105692] Updated weights for policy 0, policy_version 356025 (0.0009) [2023-12-26 17:57:22,064][105692] Updated weights for policy 0, policy_version 356035 (0.0010) [2023-12-26 17:57:22,550][105620] Updated weights for policy 1, policy_version 356399 (0.0007) [2023-12-26 17:57:22,609][105620] Updated weights for policy 1, policy_version 356409 (0.0009) [2023-12-26 17:57:22,667][105620] Updated weights for policy 1, policy_version 356419 (0.0010) [2023-12-26 17:57:22,863][105692] Updated weights for policy 0, policy_version 356045 (0.0011) [2023-12-26 17:57:22,919][105692] Updated weights for policy 0, policy_version 356055 (0.0011) [2023-12-26 17:57:22,972][105692] Updated weights for policy 0, policy_version 356065 (0.0011) [2023-12-26 17:57:23,304][105620] Updated weights for policy 1, policy_version 356429 (0.0007) [2023-12-26 17:57:23,363][105620] Updated weights for policy 1, policy_version 356439 (0.0005) [2023-12-26 17:57:23,416][105620] Updated weights for policy 1, policy_version 356449 (0.0005) [2023-12-26 17:57:23,734][105692] Updated weights for policy 0, policy_version 356075 (0.0010) [2023-12-26 17:57:23,787][105692] Updated weights for policy 0, policy_version 356085 (0.0010) [2023-12-26 17:57:23,840][105692] Updated weights for policy 0, policy_version 356095 (0.0010) [2023-12-26 17:57:23,943][105620] Updated weights for policy 1, policy_version 356459 (0.0006) [2023-12-26 17:57:23,992][105620] Updated weights for policy 1, policy_version 356469 (0.0007) [2023-12-26 17:57:24,038][105620] Updated weights for policy 1, policy_version 356479 (0.0010) [2023-12-26 17:57:24,528][105692] Updated weights for policy 0, policy_version 356106 (0.0008) [2023-12-26 17:57:24,596][105692] Updated weights for policy 0, policy_version 356116 (0.0008) [2023-12-26 17:57:24,657][105692] Updated weights for policy 0, policy_version 356126 (0.0008) [2023-12-26 17:57:24,702][105620] Updated weights for policy 1, policy_version 356489 (0.0010) [2023-12-26 17:57:24,713][105692] Updated weights for policy 0, policy_version 356136 (0.0008) [2023-12-26 17:57:24,753][105620] Updated weights for policy 1, policy_version 356499 (0.0010) [2023-12-26 17:57:24,801][105620] Updated weights for policy 1, policy_version 356509 (0.0010) [2023-12-26 17:57:24,846][105620] Updated weights for policy 1, policy_version 356519 (0.0010) [2023-12-26 17:57:25,390][105692] Updated weights for policy 0, policy_version 356146 (0.0009) [2023-12-26 17:57:25,445][105692] Updated weights for policy 0, policy_version 356156 (0.0009) [2023-12-26 17:57:25,495][105692] Updated weights for policy 0, policy_version 356167 (0.0009) [2023-12-26 17:57:25,526][105620] Updated weights for policy 1, policy_version 356529 (0.0006) [2023-12-26 17:57:25,586][105620] Updated weights for policy 1, policy_version 356539 (0.0005) [2023-12-26 17:57:25,651][105620] Updated weights for policy 1, policy_version 356549 (0.0006) [2023-12-26 17:57:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.9, 300 sec: 19438.7). Total num frames: 182476800. Throughput: 0: 9957.3, 1: 9771.2. Samples: 182487272. Policy #0 lag: (min: 30.0, avg: 31.6, max: 62.0) [2023-12-26 17:57:26,062][104569] Avg episode reward: [(0, '8620.102'), (1, '9262.113')] [2023-12-26 17:57:26,184][105692] Updated weights for policy 0, policy_version 356177 (0.0008) [2023-12-26 17:57:26,234][105692] Updated weights for policy 0, policy_version 356187 (0.0009) [2023-12-26 17:57:26,263][105620] Updated weights for policy 1, policy_version 356559 (0.0007) [2023-12-26 17:57:26,285][105692] Updated weights for policy 0, policy_version 356197 (0.0008) [2023-12-26 17:57:26,311][105620] Updated weights for policy 1, policy_version 356569 (0.0005) [2023-12-26 17:57:26,365][105620] Updated weights for policy 1, policy_version 356579 (0.0010) [2023-12-26 17:57:27,020][105620] Updated weights for policy 1, policy_version 356589 (0.0010) [2023-12-26 17:57:27,086][105620] Updated weights for policy 1, policy_version 356599 (0.0010) [2023-12-26 17:57:27,120][105692] Updated weights for policy 0, policy_version 356207 (0.0007) [2023-12-26 17:57:27,140][105620] Updated weights for policy 1, policy_version 356609 (0.0010) [2023-12-26 17:57:27,174][105692] Updated weights for policy 0, policy_version 356217 (0.0006) [2023-12-26 17:57:27,228][105692] Updated weights for policy 0, policy_version 356227 (0.0008) [2023-12-26 17:57:27,792][105620] Updated weights for policy 1, policy_version 356619 (0.0010) [2023-12-26 17:57:27,839][105620] Updated weights for policy 1, policy_version 356629 (0.0010) [2023-12-26 17:57:27,882][105620] Updated weights for policy 1, policy_version 356639 (0.0009) [2023-12-26 17:57:28,036][105692] Updated weights for policy 0, policy_version 356237 (0.0009) [2023-12-26 17:57:28,089][105692] Updated weights for policy 0, policy_version 356247 (0.0009) [2023-12-26 17:57:28,157][105692] Updated weights for policy 0, policy_version 356257 (0.0009) [2023-12-26 17:57:28,447][105620] Updated weights for policy 1, policy_version 356649 (0.0006) [2023-12-26 17:57:28,507][105620] Updated weights for policy 1, policy_version 356659 (0.0008) [2023-12-26 17:57:28,575][105620] Updated weights for policy 1, policy_version 356669 (0.0005) [2023-12-26 17:57:28,640][105620] Updated weights for policy 1, policy_version 356679 (0.0005) [2023-12-26 17:57:28,960][105692] Updated weights for policy 0, policy_version 356267 (0.0010) [2023-12-26 17:57:29,018][105692] Updated weights for policy 0, policy_version 356277 (0.0010) [2023-12-26 17:57:29,075][105692] Updated weights for policy 0, policy_version 356287 (0.0010) [2023-12-26 17:57:29,159][105620] Updated weights for policy 1, policy_version 356689 (0.0005) [2023-12-26 17:57:29,213][105620] Updated weights for policy 1, policy_version 356699 (0.0006) [2023-12-26 17:57:29,278][105620] Updated weights for policy 1, policy_version 356709 (0.0007) [2023-12-26 17:57:29,781][105692] Updated weights for policy 0, policy_version 356297 (0.0010) [2023-12-26 17:57:29,848][105692] Updated weights for policy 0, policy_version 356307 (0.0008) [2023-12-26 17:57:29,881][105620] Updated weights for policy 1, policy_version 356719 (0.0010) [2023-12-26 17:57:29,903][105692] Updated weights for policy 0, policy_version 356317 (0.0005) [2023-12-26 17:57:29,938][105620] Updated weights for policy 1, policy_version 356729 (0.0009) [2023-12-26 17:57:29,968][105692] Updated weights for policy 0, policy_version 356327 (0.0006) [2023-12-26 17:57:30,000][105620] Updated weights for policy 1, policy_version 356739 (0.0010) [2023-12-26 17:57:30,696][105692] Updated weights for policy 0, policy_version 356337 (0.0008) [2023-12-26 17:57:30,742][105692] Updated weights for policy 0, policy_version 356347 (0.0008) [2023-12-26 17:57:30,749][105620] Updated weights for policy 1, policy_version 356749 (0.0010) [2023-12-26 17:57:30,793][105620] Updated weights for policy 1, policy_version 356759 (0.0010) [2023-12-26 17:57:30,794][105692] Updated weights for policy 0, policy_version 356357 (0.0005) [2023-12-26 17:57:30,840][105620] Updated weights for policy 1, policy_version 356769 (0.0010) [2023-12-26 17:57:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 182583296. Throughput: 0: 9895.7, 1: 9914.7. Samples: 182548536. Policy #0 lag: (min: 30.0, avg: 31.6, max: 62.0) [2023-12-26 17:57:31,062][104569] Avg episode reward: [(0, '8663.114'), (1, '9261.716')] [2023-12-26 17:57:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000356360_91242496.pth... [2023-12-26 17:57:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000356776_91340800.pth... [2023-12-26 17:57:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000355240_90955776.pth [2023-12-26 17:57:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000355592_91037696.pth [2023-12-26 17:57:31,536][105692] Updated weights for policy 0, policy_version 356367 (0.0008) [2023-12-26 17:57:31,601][105692] Updated weights for policy 0, policy_version 356377 (0.0010) [2023-12-26 17:57:31,655][105620] Updated weights for policy 1, policy_version 356779 (0.0008) [2023-12-26 17:57:31,664][105692] Updated weights for policy 0, policy_version 356387 (0.0010) [2023-12-26 17:57:31,711][105620] Updated weights for policy 1, policy_version 356789 (0.0008) [2023-12-26 17:57:31,767][105620] Updated weights for policy 1, policy_version 356799 (0.0008) [2023-12-26 17:57:32,397][105692] Updated weights for policy 0, policy_version 356397 (0.0009) [2023-12-26 17:57:32,459][105692] Updated weights for policy 0, policy_version 356407 (0.0008) [2023-12-26 17:57:32,472][105620] Updated weights for policy 1, policy_version 356809 (0.0008) [2023-12-26 17:57:32,518][105692] Updated weights for policy 0, policy_version 356417 (0.0008) [2023-12-26 17:57:32,530][105620] Updated weights for policy 1, policy_version 356819 (0.0010) [2023-12-26 17:57:32,586][105620] Updated weights for policy 1, policy_version 356829 (0.0010) [2023-12-26 17:57:32,648][105620] Updated weights for policy 1, policy_version 356839 (0.0010) [2023-12-26 17:57:33,261][105692] Updated weights for policy 0, policy_version 356427 (0.0006) [2023-12-26 17:57:33,310][105692] Updated weights for policy 0, policy_version 356437 (0.0005) [2023-12-26 17:57:33,362][105692] Updated weights for policy 0, policy_version 356447 (0.0010) [2023-12-26 17:57:33,387][105620] Updated weights for policy 1, policy_version 356849 (0.0006) [2023-12-26 17:57:33,451][105620] Updated weights for policy 1, policy_version 356859 (0.0008) [2023-12-26 17:57:33,515][105620] Updated weights for policy 1, policy_version 356869 (0.0007) [2023-12-26 17:57:33,962][105692] Updated weights for policy 0, policy_version 356457 (0.0010) [2023-12-26 17:57:34,010][105692] Updated weights for policy 0, policy_version 356467 (0.0005) [2023-12-26 17:57:34,053][105692] Updated weights for policy 0, policy_version 356477 (0.0005) [2023-12-26 17:57:34,102][105692] Updated weights for policy 0, policy_version 356487 (0.0005) [2023-12-26 17:57:34,283][105620] Updated weights for policy 1, policy_version 356879 (0.0008) [2023-12-26 17:57:34,340][105620] Updated weights for policy 1, policy_version 356889 (0.0011) [2023-12-26 17:57:34,396][105620] Updated weights for policy 1, policy_version 356899 (0.0010) [2023-12-26 17:57:34,751][105692] Updated weights for policy 0, policy_version 356497 (0.0008) [2023-12-26 17:57:34,807][105692] Updated weights for policy 0, policy_version 356507 (0.0010) [2023-12-26 17:57:34,859][105692] Updated weights for policy 0, policy_version 356517 (0.0010) [2023-12-26 17:57:35,182][105620] Updated weights for policy 1, policy_version 356909 (0.0010) [2023-12-26 17:57:35,236][105620] Updated weights for policy 1, policy_version 356919 (0.0010) [2023-12-26 17:57:35,294][105620] Updated weights for policy 1, policy_version 356929 (0.0009) [2023-12-26 17:57:35,508][105692] Updated weights for policy 0, policy_version 356527 (0.0010) [2023-12-26 17:57:35,555][105692] Updated weights for policy 0, policy_version 356537 (0.0009) [2023-12-26 17:57:35,616][105692] Updated weights for policy 0, policy_version 356547 (0.0009) [2023-12-26 17:57:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 182673408. Throughput: 0: 9796.2, 1: 9987.6. Samples: 182665692. Policy #0 lag: (min: 30.0, avg: 31.6, max: 62.0) [2023-12-26 17:57:36,063][104569] Avg episode reward: [(0, '9025.983'), (1, '9256.130')] [2023-12-26 17:57:36,079][105620] Updated weights for policy 1, policy_version 356939 (0.0009) [2023-12-26 17:57:36,139][105620] Updated weights for policy 1, policy_version 356949 (0.0007) [2023-12-26 17:57:36,193][105620] Updated weights for policy 1, policy_version 356959 (0.0006) [2023-12-26 17:57:36,356][105692] Updated weights for policy 0, policy_version 356557 (0.0009) [2023-12-26 17:57:36,418][105692] Updated weights for policy 0, policy_version 356567 (0.0009) [2023-12-26 17:57:36,486][105692] Updated weights for policy 0, policy_version 356577 (0.0009) [2023-12-26 17:57:36,881][105620] Updated weights for policy 1, policy_version 356969 (0.0008) [2023-12-26 17:57:36,946][105620] Updated weights for policy 1, policy_version 356979 (0.0009) [2023-12-26 17:57:37,007][105620] Updated weights for policy 1, policy_version 356989 (0.0008) [2023-12-26 17:57:37,068][105620] Updated weights for policy 1, policy_version 356999 (0.0009) [2023-12-26 17:57:37,258][105692] Updated weights for policy 0, policy_version 356587 (0.0009) [2023-12-26 17:57:37,320][105692] Updated weights for policy 0, policy_version 356597 (0.0006) [2023-12-26 17:57:37,379][105692] Updated weights for policy 0, policy_version 356607 (0.0005) [2023-12-26 17:57:37,746][105620] Updated weights for policy 1, policy_version 357009 (0.0010) [2023-12-26 17:57:37,796][105620] Updated weights for policy 1, policy_version 357019 (0.0010) [2023-12-26 17:57:37,853][105620] Updated weights for policy 1, policy_version 357029 (0.0010) [2023-12-26 17:57:38,115][105692] Updated weights for policy 0, policy_version 356617 (0.0009) [2023-12-26 17:57:38,158][105692] Updated weights for policy 0, policy_version 356627 (0.0008) [2023-12-26 17:57:38,206][105692] Updated weights for policy 0, policy_version 356637 (0.0008) [2023-12-26 17:57:38,250][105692] Updated weights for policy 0, policy_version 356647 (0.0007) [2023-12-26 17:57:38,589][105620] Updated weights for policy 1, policy_version 357039 (0.0010) [2023-12-26 17:57:38,644][105620] Updated weights for policy 1, policy_version 357049 (0.0010) [2023-12-26 17:57:38,702][105620] Updated weights for policy 1, policy_version 357059 (0.0010) [2023-12-26 17:57:39,048][105692] Updated weights for policy 0, policy_version 356657 (0.0008) [2023-12-26 17:57:39,105][105692] Updated weights for policy 0, policy_version 356667 (0.0008) [2023-12-26 17:57:39,169][105692] Updated weights for policy 0, policy_version 356677 (0.0009) [2023-12-26 17:57:39,454][105620] Updated weights for policy 1, policy_version 357069 (0.0009) [2023-12-26 17:57:39,516][105620] Updated weights for policy 1, policy_version 357079 (0.0009) [2023-12-26 17:57:39,573][105620] Updated weights for policy 1, policy_version 357089 (0.0011) [2023-12-26 17:57:39,898][105692] Updated weights for policy 0, policy_version 356687 (0.0010) [2023-12-26 17:57:39,962][105692] Updated weights for policy 0, policy_version 356697 (0.0008) [2023-12-26 17:57:40,017][105692] Updated weights for policy 0, policy_version 356707 (0.0009) [2023-12-26 17:57:40,190][105620] Updated weights for policy 1, policy_version 357099 (0.0009) [2023-12-26 17:57:40,252][105620] Updated weights for policy 1, policy_version 357109 (0.0009) [2023-12-26 17:57:40,308][105620] Updated weights for policy 1, policy_version 357119 (0.0009) [2023-12-26 17:57:40,873][105692] Updated weights for policy 0, policy_version 356717 (0.0010) [2023-12-26 17:57:40,926][105692] Updated weights for policy 0, policy_version 356727 (0.0009) [2023-12-26 17:57:40,945][105620] Updated weights for policy 1, policy_version 357129 (0.0008) [2023-12-26 17:57:40,985][105692] Updated weights for policy 0, policy_version 356737 (0.0008) [2023-12-26 17:57:40,992][105620] Updated weights for policy 1, policy_version 357139 (0.0006) [2023-12-26 17:57:41,058][105620] Updated weights for policy 1, policy_version 357149 (0.0007) [2023-12-26 17:57:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19797.3, 300 sec: 19438.6). Total num frames: 182771712. Throughput: 0: 9695.5, 1: 9992.2. Samples: 182781384. Policy #0 lag: (min: 30.0, avg: 31.6, max: 62.0) [2023-12-26 17:57:41,063][104569] Avg episode reward: [(0, '9083.612'), (1, '9347.546')] [2023-12-26 17:57:41,122][105620] Updated weights for policy 1, policy_version 357159 (0.0008) [2023-12-26 17:57:41,795][105692] Updated weights for policy 0, policy_version 356747 (0.0007) [2023-12-26 17:57:41,818][105620] Updated weights for policy 1, policy_version 357169 (0.0008) [2023-12-26 17:57:41,862][105692] Updated weights for policy 0, policy_version 356757 (0.0006) [2023-12-26 17:57:41,879][105620] Updated weights for policy 1, policy_version 357179 (0.0008) [2023-12-26 17:57:41,929][105692] Updated weights for policy 0, policy_version 356767 (0.0006) [2023-12-26 17:57:41,943][105620] Updated weights for policy 1, policy_version 357189 (0.0006) [2023-12-26 17:57:42,609][105692] Updated weights for policy 0, policy_version 356777 (0.0006) [2023-12-26 17:57:42,669][105692] Updated weights for policy 0, policy_version 356787 (0.0009) [2023-12-26 17:57:42,691][105620] Updated weights for policy 1, policy_version 357199 (0.0008) [2023-12-26 17:57:42,731][105692] Updated weights for policy 0, policy_version 356797 (0.0007) [2023-12-26 17:57:42,745][105620] Updated weights for policy 1, policy_version 357209 (0.0008) [2023-12-26 17:57:42,793][105692] Updated weights for policy 0, policy_version 356807 (0.0007) [2023-12-26 17:57:42,799][105620] Updated weights for policy 1, policy_version 357219 (0.0007) [2023-12-26 17:57:43,542][105620] Updated weights for policy 1, policy_version 357229 (0.0007) [2023-12-26 17:57:43,554][105692] Updated weights for policy 0, policy_version 356817 (0.0009) [2023-12-26 17:57:43,601][105620] Updated weights for policy 1, policy_version 357239 (0.0008) [2023-12-26 17:57:43,611][105692] Updated weights for policy 0, policy_version 356827 (0.0007) [2023-12-26 17:57:43,661][105620] Updated weights for policy 1, policy_version 357249 (0.0010) [2023-12-26 17:57:43,671][105692] Updated weights for policy 0, policy_version 356837 (0.0006) [2023-12-26 17:57:44,254][105620] Updated weights for policy 1, policy_version 357259 (0.0008) [2023-12-26 17:57:44,315][105620] Updated weights for policy 1, policy_version 357269 (0.0009) [2023-12-26 17:57:44,373][105620] Updated weights for policy 1, policy_version 357279 (0.0011) [2023-12-26 17:57:44,489][105692] Updated weights for policy 0, policy_version 356847 (0.0008) [2023-12-26 17:57:44,547][105692] Updated weights for policy 0, policy_version 356857 (0.0009) [2023-12-26 17:57:44,607][105692] Updated weights for policy 0, policy_version 356867 (0.0009) [2023-12-26 17:57:45,068][105620] Updated weights for policy 1, policy_version 357289 (0.0010) [2023-12-26 17:57:45,129][105620] Updated weights for policy 1, policy_version 357299 (0.0010) [2023-12-26 17:57:45,179][105620] Updated weights for policy 1, policy_version 357309 (0.0010) [2023-12-26 17:57:45,227][105620] Updated weights for policy 1, policy_version 357319 (0.0010) [2023-12-26 17:57:45,406][105692] Updated weights for policy 0, policy_version 356877 (0.0009) [2023-12-26 17:57:45,462][105692] Updated weights for policy 0, policy_version 356887 (0.0008) [2023-12-26 17:57:45,522][105692] Updated weights for policy 0, policy_version 356897 (0.0008) [2023-12-26 17:57:46,006][105620] Updated weights for policy 1, policy_version 357329 (0.0011) [2023-12-26 17:57:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 182861824. Throughput: 0: 9596.4, 1: 9991.3. Samples: 182837520. Policy #0 lag: (min: 30.0, avg: 31.6, max: 62.0) [2023-12-26 17:57:46,062][104569] Avg episode reward: [(0, '9083.600'), (1, '1336.685')] [2023-12-26 17:57:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000356904_91381760.pth... [2023-12-26 17:57:46,071][105620] Updated weights for policy 1, policy_version 357339 (0.0010) [2023-12-26 17:57:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000355816_91103232.pth [2023-12-26 17:57:46,119][105620] Updated weights for policy 1, policy_version 357349 (0.0010) [2023-12-26 17:57:46,132][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000357352_91488256.pth... [2023-12-26 17:57:46,136][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000356168_91185152.pth [2023-12-26 17:57:46,250][105692] Updated weights for policy 0, policy_version 356907 (0.0007) [2023-12-26 17:57:46,305][105692] Updated weights for policy 0, policy_version 356917 (0.0005) [2023-12-26 17:57:46,353][105692] Updated weights for policy 0, policy_version 356927 (0.0006) [2023-12-26 17:57:46,820][105620] Updated weights for policy 1, policy_version 357359 (0.0010) [2023-12-26 17:57:46,883][105586] KL-divergence is very high: 147.1264 [2023-12-26 17:57:46,885][105620] Updated weights for policy 1, policy_version 357370 (0.0009) [2023-12-26 17:57:46,888][105586] KL-divergence is very high: 151.6081 [2023-12-26 17:57:46,899][105586] KL-divergence is very high: 171.3652 [2023-12-26 17:57:46,905][105586] KL-divergence is very high: 145.6197 [2023-12-26 17:57:46,911][105586] KL-divergence is very high: 152.8307 [2023-12-26 17:57:46,916][105586] KL-divergence is very high: 107.2768 [2023-12-26 17:57:46,922][105586] KL-divergence is very high: 108.8958 [2023-12-26 17:57:46,928][105586] KL-divergence is very high: 123.4977 [2023-12-26 17:57:46,934][105586] KL-divergence is very high: 110.3498 [2023-12-26 17:57:46,939][105620] Updated weights for policy 1, policy_version 357380 (0.0010) [2023-12-26 17:57:47,007][105692] Updated weights for policy 0, policy_version 356937 (0.0008) [2023-12-26 17:57:47,068][105692] Updated weights for policy 0, policy_version 356947 (0.0009) [2023-12-26 17:57:47,128][105692] Updated weights for policy 0, policy_version 356957 (0.0009) [2023-12-26 17:57:47,186][105692] Updated weights for policy 0, policy_version 356967 (0.0009) [2023-12-26 17:57:47,708][105620] Updated weights for policy 1, policy_version 357390 (0.0009) [2023-12-26 17:57:47,772][105620] Updated weights for policy 1, policy_version 357400 (0.0009) [2023-12-26 17:57:47,819][105620] Updated weights for policy 1, policy_version 357410 (0.0008) [2023-12-26 17:57:47,841][105692] Updated weights for policy 0, policy_version 356977 (0.0008) [2023-12-26 17:57:47,894][105692] Updated weights for policy 0, policy_version 356987 (0.0009) [2023-12-26 17:57:47,956][105692] Updated weights for policy 0, policy_version 356997 (0.0009) [2023-12-26 17:57:48,575][105620] Updated weights for policy 1, policy_version 357420 (0.0007) [2023-12-26 17:57:48,636][105620] Updated weights for policy 1, policy_version 357430 (0.0008) [2023-12-26 17:57:48,696][105620] Updated weights for policy 1, policy_version 357440 (0.0008) [2023-12-26 17:57:48,717][105692] Updated weights for policy 0, policy_version 357007 (0.0010) [2023-12-26 17:57:48,775][105692] Updated weights for policy 0, policy_version 357017 (0.0010) [2023-12-26 17:57:48,831][105692] Updated weights for policy 0, policy_version 357027 (0.0010) [2023-12-26 17:57:49,457][105620] Updated weights for policy 1, policy_version 357450 (0.0006) [2023-12-26 17:57:49,523][105620] Updated weights for policy 1, policy_version 357460 (0.0010) [2023-12-26 17:57:49,588][105692] Updated weights for policy 0, policy_version 357037 (0.0010) [2023-12-26 17:57:49,590][105620] Updated weights for policy 1, policy_version 357470 (0.0011) [2023-12-26 17:57:49,644][105620] Updated weights for policy 1, policy_version 357480 (0.0010) [2023-12-26 17:57:49,646][105692] Updated weights for policy 0, policy_version 357047 (0.0006) [2023-12-26 17:57:49,712][105692] Updated weights for policy 0, policy_version 357057 (0.0010) [2023-12-26 17:57:50,263][105620] Updated weights for policy 1, policy_version 357490 (0.0006) [2023-12-26 17:57:50,295][105586] KL-divergence is very high: 107.9372 [2023-12-26 17:57:50,318][105620] Updated weights for policy 1, policy_version 357500 (0.0008) [2023-12-26 17:57:50,328][105586] KL-divergence is very high: 166.1548 [2023-12-26 17:57:50,340][105586] KL-divergence is very high: 174.9837 [2023-12-26 17:57:50,362][105586] KL-divergence is very high: 108.8546 [2023-12-26 17:57:50,372][105586] KL-divergence is very high: 179.3288 [2023-12-26 17:57:50,373][105620] Updated weights for policy 1, policy_version 357510 (0.0006) [2023-12-26 17:57:50,493][105692] Updated weights for policy 0, policy_version 357067 (0.0009) [2023-12-26 17:57:50,557][105692] Updated weights for policy 0, policy_version 357077 (0.0008) [2023-12-26 17:57:50,619][105692] Updated weights for policy 0, policy_version 357087 (0.0009) [2023-12-26 17:57:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 182960128. Throughput: 0: 9580.4, 1: 9974.1. Samples: 182952460. Policy #0 lag: (min: 30.0, avg: 31.6, max: 62.0) [2023-12-26 17:57:51,062][104569] Avg episode reward: [(0, '9265.223'), (1, '1752.784')] [2023-12-26 17:57:51,077][105620] Updated weights for policy 1, policy_version 357520 (0.0011) [2023-12-26 17:57:51,145][105620] Updated weights for policy 1, policy_version 357530 (0.0010) [2023-12-26 17:57:51,190][105620] Updated weights for policy 1, policy_version 357540 (0.0010) [2023-12-26 17:57:51,378][105692] Updated weights for policy 0, policy_version 357097 (0.0007) [2023-12-26 17:57:51,427][105692] Updated weights for policy 0, policy_version 357107 (0.0007) [2023-12-26 17:57:51,476][105692] Updated weights for policy 0, policy_version 357117 (0.0005) [2023-12-26 17:57:51,529][105692] Updated weights for policy 0, policy_version 357127 (0.0008) [2023-12-26 17:57:51,962][105620] Updated weights for policy 1, policy_version 357550 (0.0009) [2023-12-26 17:57:52,018][105620] Updated weights for policy 1, policy_version 357560 (0.0008) [2023-12-26 17:57:52,080][105620] Updated weights for policy 1, policy_version 357570 (0.0005) [2023-12-26 17:57:52,296][105692] Updated weights for policy 0, policy_version 357137 (0.0010) [2023-12-26 17:57:52,356][105692] Updated weights for policy 0, policy_version 357147 (0.0011) [2023-12-26 17:57:52,425][105692] Updated weights for policy 0, policy_version 357157 (0.0008) [2023-12-26 17:57:52,699][105620] Updated weights for policy 1, policy_version 357580 (0.0008) [2023-12-26 17:57:52,749][105620] Updated weights for policy 1, policy_version 357590 (0.0010) [2023-12-26 17:57:52,805][105620] Updated weights for policy 1, policy_version 357600 (0.0010) [2023-12-26 17:57:53,138][105692] Updated weights for policy 0, policy_version 357167 (0.0006) [2023-12-26 17:57:53,193][105692] Updated weights for policy 0, policy_version 357177 (0.0009) [2023-12-26 17:57:53,251][105692] Updated weights for policy 0, policy_version 357187 (0.0005) [2023-12-26 17:57:53,476][105620] Updated weights for policy 1, policy_version 357610 (0.0009) [2023-12-26 17:57:53,528][105620] Updated weights for policy 1, policy_version 357620 (0.0005) [2023-12-26 17:57:53,581][105620] Updated weights for policy 1, policy_version 357630 (0.0005) [2023-12-26 17:57:53,642][105620] Updated weights for policy 1, policy_version 357640 (0.0005) [2023-12-26 17:57:53,902][105692] Updated weights for policy 0, policy_version 357197 (0.0007) [2023-12-26 17:57:53,959][105692] Updated weights for policy 0, policy_version 357207 (0.0010) [2023-12-26 17:57:54,021][105692] Updated weights for policy 0, policy_version 357217 (0.0010) [2023-12-26 17:57:54,251][105620] Updated weights for policy 1, policy_version 357650 (0.0010) [2023-12-26 17:57:54,306][105620] Updated weights for policy 1, policy_version 357660 (0.0010) [2023-12-26 17:57:54,362][105620] Updated weights for policy 1, policy_version 357670 (0.0010) [2023-12-26 17:57:54,735][105692] Updated weights for policy 0, policy_version 357227 (0.0010) [2023-12-26 17:57:54,799][105692] Updated weights for policy 0, policy_version 357237 (0.0011) [2023-12-26 17:57:54,851][105692] Updated weights for policy 0, policy_version 357247 (0.0010) [2023-12-26 17:57:55,030][105620] Updated weights for policy 1, policy_version 357680 (0.0007) [2023-12-26 17:57:55,086][105620] Updated weights for policy 1, policy_version 357690 (0.0008) [2023-12-26 17:57:55,136][105620] Updated weights for policy 1, policy_version 357700 (0.0005) [2023-12-26 17:57:55,603][105692] Updated weights for policy 0, policy_version 357257 (0.0010) [2023-12-26 17:57:55,664][105692] Updated weights for policy 0, policy_version 357267 (0.0010) [2023-12-26 17:57:55,725][105692] Updated weights for policy 0, policy_version 357277 (0.0010) [2023-12-26 17:57:55,748][105620] Updated weights for policy 1, policy_version 357710 (0.0007) [2023-12-26 17:57:55,783][105692] Updated weights for policy 0, policy_version 357287 (0.0010) [2023-12-26 17:57:55,803][105620] Updated weights for policy 1, policy_version 357720 (0.0009) [2023-12-26 17:57:55,854][105620] Updated weights for policy 1, policy_version 357730 (0.0006) [2023-12-26 17:57:56,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 183066624. Throughput: 0: 9481.4, 1: 10020.0. Samples: 183071812. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 17:57:56,063][104569] Avg episode reward: [(0, '9174.908'), (1, '6414.542')] [2023-12-26 17:57:56,460][105620] Updated weights for policy 1, policy_version 357740 (0.0007) [2023-12-26 17:57:56,511][105692] Updated weights for policy 0, policy_version 357297 (0.0010) [2023-12-26 17:57:56,515][105620] Updated weights for policy 1, policy_version 357750 (0.0009) [2023-12-26 17:57:56,568][105692] Updated weights for policy 0, policy_version 357307 (0.0010) [2023-12-26 17:57:56,570][105620] Updated weights for policy 1, policy_version 357760 (0.0006) [2023-12-26 17:57:56,619][105692] Updated weights for policy 0, policy_version 357317 (0.0010) [2023-12-26 17:57:57,291][105620] Updated weights for policy 1, policy_version 357770 (0.0006) [2023-12-26 17:57:57,352][105620] Updated weights for policy 1, policy_version 357780 (0.0007) [2023-12-26 17:57:57,352][105692] Updated weights for policy 0, policy_version 357327 (0.0009) [2023-12-26 17:57:57,410][105620] Updated weights for policy 1, policy_version 357790 (0.0009) [2023-12-26 17:57:57,417][105692] Updated weights for policy 0, policy_version 357337 (0.0010) [2023-12-26 17:57:57,464][105692] Updated weights for policy 0, policy_version 357347 (0.0010) [2023-12-26 17:57:57,466][105620] Updated weights for policy 1, policy_version 357800 (0.0005) [2023-12-26 17:57:58,100][105620] Updated weights for policy 1, policy_version 357810 (0.0006) [2023-12-26 17:57:58,158][105620] Updated weights for policy 1, policy_version 357820 (0.0008) [2023-12-26 17:57:58,170][105692] Updated weights for policy 0, policy_version 357357 (0.0011) [2023-12-26 17:57:58,218][105620] Updated weights for policy 1, policy_version 357830 (0.0008) [2023-12-26 17:57:58,225][105692] Updated weights for policy 0, policy_version 357367 (0.0010) [2023-12-26 17:57:58,286][105692] Updated weights for policy 0, policy_version 357377 (0.0010) [2023-12-26 17:57:58,956][105620] Updated weights for policy 1, policy_version 357840 (0.0008) [2023-12-26 17:57:59,007][105620] Updated weights for policy 1, policy_version 357850 (0.0008) [2023-12-26 17:57:59,051][105692] Updated weights for policy 0, policy_version 357387 (0.0010) [2023-12-26 17:57:59,057][105620] Updated weights for policy 1, policy_version 357860 (0.0007) [2023-12-26 17:57:59,109][105692] Updated weights for policy 0, policy_version 357397 (0.0010) [2023-12-26 17:57:59,160][105692] Updated weights for policy 0, policy_version 357407 (0.0010) [2023-12-26 17:57:59,858][105620] Updated weights for policy 1, policy_version 357870 (0.0007) [2023-12-26 17:57:59,904][105692] Updated weights for policy 0, policy_version 357417 (0.0010) [2023-12-26 17:57:59,910][105620] Updated weights for policy 1, policy_version 357880 (0.0008) [2023-12-26 17:57:59,966][105692] Updated weights for policy 0, policy_version 357427 (0.0010) [2023-12-26 17:57:59,968][105620] Updated weights for policy 1, policy_version 357890 (0.0007) [2023-12-26 17:58:00,026][105692] Updated weights for policy 0, policy_version 357437 (0.0011) [2023-12-26 17:58:00,085][105692] Updated weights for policy 0, policy_version 357447 (0.0011) [2023-12-26 17:58:00,740][105620] Updated weights for policy 1, policy_version 357900 (0.0008) [2023-12-26 17:58:00,806][105620] Updated weights for policy 1, policy_version 357910 (0.0006) [2023-12-26 17:58:00,835][105692] Updated weights for policy 0, policy_version 357457 (0.0010) [2023-12-26 17:58:00,856][105620] Updated weights for policy 1, policy_version 357920 (0.0006) [2023-12-26 17:58:00,889][105692] Updated weights for policy 0, policy_version 357467 (0.0010) [2023-12-26 17:58:00,939][105692] Updated weights for policy 0, policy_version 357477 (0.0010) [2023-12-26 17:58:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 183164928. Throughput: 0: 9488.4, 1: 10016.7. Samples: 183131020. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 17:58:01,062][104569] Avg episode reward: [(0, '9267.057'), (1, '7174.550')] [2023-12-26 17:58:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000357480_91529216.pth... [2023-12-26 17:58:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000357928_91635712.pth... [2023-12-26 17:58:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000356360_91242496.pth [2023-12-26 17:58:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000356776_91340800.pth [2023-12-26 17:58:01,564][105620] Updated weights for policy 1, policy_version 357930 (0.0006) [2023-12-26 17:58:01,625][105620] Updated weights for policy 1, policy_version 357940 (0.0006) [2023-12-26 17:58:01,690][105620] Updated weights for policy 1, policy_version 357950 (0.0006) [2023-12-26 17:58:01,717][105692] Updated weights for policy 0, policy_version 357487 (0.0010) [2023-12-26 17:58:01,759][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000009 [2023-12-26 17:58:01,760][105620] Updated weights for policy 1, policy_version 357960 (0.0007) [2023-12-26 17:58:01,781][105692] Updated weights for policy 0, policy_version 357497 (0.0006) [2023-12-26 17:58:01,837][105692] Updated weights for policy 0, policy_version 357507 (0.0005) [2023-12-26 17:58:02,341][105620] Updated weights for policy 1, policy_version 357970 (0.0006) [2023-12-26 17:58:02,408][105620] Updated weights for policy 1, policy_version 357980 (0.0008) [2023-12-26 17:58:02,474][105620] Updated weights for policy 1, policy_version 357990 (0.0007) [2023-12-26 17:58:02,488][105692] Updated weights for policy 0, policy_version 357517 (0.0008) [2023-12-26 17:58:02,549][105692] Updated weights for policy 0, policy_version 357527 (0.0010) [2023-12-26 17:58:02,615][105692] Updated weights for policy 0, policy_version 357537 (0.0010) [2023-12-26 17:58:03,192][105620] Updated weights for policy 1, policy_version 358000 (0.0008) [2023-12-26 17:58:03,236][105620] Updated weights for policy 1, policy_version 358010 (0.0008) [2023-12-26 17:58:03,287][105620] Updated weights for policy 1, policy_version 358020 (0.0008) [2023-12-26 17:58:03,323][105692] Updated weights for policy 0, policy_version 357547 (0.0010) [2023-12-26 17:58:03,374][105692] Updated weights for policy 0, policy_version 357557 (0.0008) [2023-12-26 17:58:03,421][105692] Updated weights for policy 0, policy_version 357567 (0.0005) [2023-12-26 17:58:04,099][105620] Updated weights for policy 1, policy_version 358030 (0.0008) [2023-12-26 17:58:04,139][105692] Updated weights for policy 0, policy_version 357577 (0.0006) [2023-12-26 17:58:04,161][105620] Updated weights for policy 1, policy_version 358040 (0.0007) [2023-12-26 17:58:04,198][105692] Updated weights for policy 0, policy_version 357587 (0.0010) [2023-12-26 17:58:04,224][105620] Updated weights for policy 1, policy_version 358050 (0.0006) [2023-12-26 17:58:04,257][105692] Updated weights for policy 0, policy_version 357597 (0.0007) [2023-12-26 17:58:04,321][105692] Updated weights for policy 0, policy_version 357607 (0.0009) [2023-12-26 17:58:04,996][105620] Updated weights for policy 1, policy_version 358060 (0.0008) [2023-12-26 17:58:05,045][105692] Updated weights for policy 0, policy_version 357617 (0.0010) [2023-12-26 17:58:05,048][105620] Updated weights for policy 1, policy_version 358070 (0.0007) [2023-12-26 17:58:05,095][105620] Updated weights for policy 1, policy_version 358080 (0.0007) [2023-12-26 17:58:05,096][105692] Updated weights for policy 0, policy_version 357627 (0.0010) [2023-12-26 17:58:05,154][105692] Updated weights for policy 0, policy_version 357637 (0.0010) [2023-12-26 17:58:05,809][105620] Updated weights for policy 1, policy_version 358090 (0.0005) [2023-12-26 17:58:05,864][105620] Updated weights for policy 1, policy_version 358100 (0.0007) [2023-12-26 17:58:05,901][105692] Updated weights for policy 0, policy_version 357647 (0.0010) [2023-12-26 17:58:05,918][105620] Updated weights for policy 1, policy_version 358110 (0.0005) [2023-12-26 17:58:05,963][105692] Updated weights for policy 0, policy_version 357657 (0.0010) [2023-12-26 17:58:05,977][105620] Updated weights for policy 1, policy_version 358120 (0.0007) [2023-12-26 17:58:06,021][105692] Updated weights for policy 0, policy_version 357667 (0.0010) [2023-12-26 17:58:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 183263232. Throughput: 0: 9506.5, 1: 9969.5. Samples: 183245652. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 17:58:06,063][104569] Avg episode reward: [(0, '9266.934'), (1, '9265.218')] [2023-12-26 17:58:06,657][105692] Updated weights for policy 0, policy_version 357677 (0.0009) [2023-12-26 17:58:06,718][105692] Updated weights for policy 0, policy_version 357687 (0.0008) [2023-12-26 17:58:06,771][105620] Updated weights for policy 1, policy_version 358130 (0.0006) [2023-12-26 17:58:06,777][105692] Updated weights for policy 0, policy_version 357697 (0.0008) [2023-12-26 17:58:06,817][105620] Updated weights for policy 1, policy_version 358140 (0.0006) [2023-12-26 17:58:06,875][105620] Updated weights for policy 1, policy_version 358150 (0.0008) [2023-12-26 17:58:07,489][105620] Updated weights for policy 1, policy_version 358160 (0.0005) [2023-12-26 17:58:07,552][105620] Updated weights for policy 1, policy_version 358170 (0.0005) [2023-12-26 17:58:07,585][105692] Updated weights for policy 0, policy_version 357707 (0.0009) [2023-12-26 17:58:07,612][105620] Updated weights for policy 1, policy_version 358180 (0.0005) [2023-12-26 17:58:07,649][105692] Updated weights for policy 0, policy_version 357717 (0.0009) [2023-12-26 17:58:07,716][105692] Updated weights for policy 0, policy_version 357727 (0.0009) [2023-12-26 17:58:08,225][105620] Updated weights for policy 1, policy_version 358190 (0.0007) [2023-12-26 17:58:08,283][105620] Updated weights for policy 1, policy_version 358200 (0.0009) [2023-12-26 17:58:08,344][105620] Updated weights for policy 1, policy_version 358210 (0.0007) [2023-12-26 17:58:08,493][105692] Updated weights for policy 0, policy_version 357737 (0.0009) [2023-12-26 17:58:08,542][105692] Updated weights for policy 0, policy_version 357747 (0.0010) [2023-12-26 17:58:08,598][105692] Updated weights for policy 0, policy_version 357757 (0.0011) [2023-12-26 17:58:08,656][105692] Updated weights for policy 0, policy_version 357767 (0.0011) [2023-12-26 17:58:09,101][105620] Updated weights for policy 1, policy_version 358220 (0.0009) [2023-12-26 17:58:09,159][105620] Updated weights for policy 1, policy_version 358230 (0.0010) [2023-12-26 17:58:09,212][105620] Updated weights for policy 1, policy_version 358240 (0.0010) [2023-12-26 17:58:09,328][105692] Updated weights for policy 0, policy_version 357777 (0.0009) [2023-12-26 17:58:09,398][105692] Updated weights for policy 0, policy_version 357787 (0.0007) [2023-12-26 17:58:09,458][105692] Updated weights for policy 0, policy_version 357797 (0.0009) [2023-12-26 17:58:09,978][105620] Updated weights for policy 1, policy_version 358250 (0.0008) [2023-12-26 17:58:10,043][105620] Updated weights for policy 1, policy_version 358260 (0.0010) [2023-12-26 17:58:10,104][105620] Updated weights for policy 1, policy_version 358270 (0.0008) [2023-12-26 17:58:10,170][105620] Updated weights for policy 1, policy_version 358280 (0.0009) [2023-12-26 17:58:10,249][105692] Updated weights for policy 0, policy_version 357807 (0.0007) [2023-12-26 17:58:10,306][105692] Updated weights for policy 0, policy_version 357817 (0.0006) [2023-12-26 17:58:10,369][105692] Updated weights for policy 0, policy_version 357827 (0.0007) [2023-12-26 17:58:10,937][105620] Updated weights for policy 1, policy_version 358290 (0.0009) [2023-12-26 17:58:10,987][105620] Updated weights for policy 1, policy_version 358300 (0.0009) [2023-12-26 17:58:11,047][105620] Updated weights for policy 1, policy_version 358310 (0.0008) [2023-12-26 17:58:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 183353344. Throughput: 0: 9493.8, 1: 9910.7. Samples: 183360472. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 17:58:11,062][104569] Avg episode reward: [(0, '9084.931'), (1, '9265.895')] [2023-12-26 17:58:11,070][105692] Updated weights for policy 0, policy_version 357837 (0.0008) [2023-12-26 17:58:11,129][105692] Updated weights for policy 0, policy_version 357847 (0.0008) [2023-12-26 17:58:11,200][105692] Updated weights for policy 0, policy_version 357857 (0.0010) [2023-12-26 17:58:11,760][105620] Updated weights for policy 1, policy_version 358320 (0.0008) [2023-12-26 17:58:11,833][105620] Updated weights for policy 1, policy_version 358330 (0.0007) [2023-12-26 17:58:11,907][105620] Updated weights for policy 1, policy_version 358340 (0.0010) [2023-12-26 17:58:11,956][105692] Updated weights for policy 0, policy_version 357867 (0.0008) [2023-12-26 17:58:12,021][105692] Updated weights for policy 0, policy_version 357877 (0.0006) [2023-12-26 17:58:12,090][105692] Updated weights for policy 0, policy_version 357887 (0.0006) [2023-12-26 17:58:12,658][105620] Updated weights for policy 1, policy_version 358350 (0.0006) [2023-12-26 17:58:12,696][105692] Updated weights for policy 0, policy_version 357897 (0.0009) [2023-12-26 17:58:12,714][105620] Updated weights for policy 1, policy_version 358360 (0.0006) [2023-12-26 17:58:12,748][105692] Updated weights for policy 0, policy_version 357907 (0.0011) [2023-12-26 17:58:12,774][105620] Updated weights for policy 1, policy_version 358370 (0.0006) [2023-12-26 17:58:12,801][105692] Updated weights for policy 0, policy_version 357917 (0.0011) [2023-12-26 17:58:12,851][105692] Updated weights for policy 0, policy_version 357927 (0.0011) [2023-12-26 17:58:13,502][105620] Updated weights for policy 1, policy_version 358380 (0.0006) [2023-12-26 17:58:13,552][105620] Updated weights for policy 1, policy_version 358390 (0.0007) [2023-12-26 17:58:13,599][105692] Updated weights for policy 0, policy_version 357937 (0.0010) [2023-12-26 17:58:13,601][105620] Updated weights for policy 1, policy_version 358400 (0.0006) [2023-12-26 17:58:13,656][105692] Updated weights for policy 0, policy_version 357947 (0.0009) [2023-12-26 17:58:13,714][105692] Updated weights for policy 0, policy_version 357957 (0.0008) [2023-12-26 17:58:14,234][105620] Updated weights for policy 1, policy_version 358410 (0.0007) [2023-12-26 17:58:14,296][105620] Updated weights for policy 1, policy_version 358420 (0.0008) [2023-12-26 17:58:14,345][105620] Updated weights for policy 1, policy_version 358430 (0.0010) [2023-12-26 17:58:14,405][105620] Updated weights for policy 1, policy_version 358440 (0.0011) [2023-12-26 17:58:14,434][105692] Updated weights for policy 0, policy_version 357967 (0.0008) [2023-12-26 17:58:14,491][105692] Updated weights for policy 0, policy_version 357977 (0.0010) [2023-12-26 17:58:14,543][105692] Updated weights for policy 0, policy_version 357987 (0.0008) [2023-12-26 17:58:15,130][105620] Updated weights for policy 1, policy_version 358450 (0.0011) [2023-12-26 17:58:15,193][105620] Updated weights for policy 1, policy_version 358460 (0.0011) [2023-12-26 17:58:15,264][105620] Updated weights for policy 1, policy_version 358470 (0.0011) [2023-12-26 17:58:15,295][105692] Updated weights for policy 0, policy_version 357997 (0.0008) [2023-12-26 17:58:15,354][105692] Updated weights for policy 0, policy_version 358007 (0.0007) [2023-12-26 17:58:15,420][105692] Updated weights for policy 0, policy_version 358017 (0.0006) [2023-12-26 17:58:15,927][105620] Updated weights for policy 1, policy_version 358480 (0.0010) [2023-12-26 17:58:15,946][105692] Updated weights for policy 0, policy_version 358027 (0.0007) [2023-12-26 17:58:15,988][105620] Updated weights for policy 1, policy_version 358490 (0.0011) [2023-12-26 17:58:15,997][105692] Updated weights for policy 0, policy_version 358037 (0.0006) [2023-12-26 17:58:16,047][105692] Updated weights for policy 0, policy_version 358047 (0.0005) [2023-12-26 17:58:16,048][105620] Updated weights for policy 1, policy_version 358500 (0.0010) [2023-12-26 17:58:16,062][104569] Fps is (10 sec: 18022.6, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 183443456. Throughput: 0: 9546.2, 1: 9800.7. Samples: 183419148. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 17:58:16,063][104569] Avg episode reward: [(0, '9083.490'), (1, '9358.340')] [2023-12-26 17:58:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000358504_91783168.pth... [2023-12-26 17:58:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000357352_91488256.pth [2023-12-26 17:58:16,097][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000358056_91676672.pth... [2023-12-26 17:58:16,101][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000356904_91381760.pth [2023-12-26 17:58:16,579][105692] Updated weights for policy 0, policy_version 358057 (0.0006) [2023-12-26 17:58:16,633][105692] Updated weights for policy 0, policy_version 358067 (0.0005) [2023-12-26 17:58:16,687][105692] Updated weights for policy 0, policy_version 358077 (0.0007) [2023-12-26 17:58:16,746][105692] Updated weights for policy 0, policy_version 358087 (0.0009) [2023-12-26 17:58:16,748][105620] Updated weights for policy 1, policy_version 358510 (0.0007) [2023-12-26 17:58:16,802][105620] Updated weights for policy 1, policy_version 358520 (0.0008) [2023-12-26 17:58:16,858][105620] Updated weights for policy 1, policy_version 358530 (0.0008) [2023-12-26 17:58:17,453][105620] Updated weights for policy 1, policy_version 358540 (0.0008) [2023-12-26 17:58:17,504][105620] Updated weights for policy 1, policy_version 358550 (0.0008) [2023-12-26 17:58:17,534][105692] Updated weights for policy 0, policy_version 358097 (0.0005) [2023-12-26 17:58:17,551][105620] Updated weights for policy 1, policy_version 358560 (0.0010) [2023-12-26 17:58:17,582][105692] Updated weights for policy 0, policy_version 358107 (0.0007) [2023-12-26 17:58:17,638][105692] Updated weights for policy 0, policy_version 358117 (0.0009) [2023-12-26 17:58:18,102][105620] Updated weights for policy 1, policy_version 358570 (0.0005) [2023-12-26 17:58:18,170][105620] Updated weights for policy 1, policy_version 358580 (0.0005) [2023-12-26 17:58:18,239][105620] Updated weights for policy 1, policy_version 358590 (0.0008) [2023-12-26 17:58:18,304][105620] Updated weights for policy 1, policy_version 358600 (0.0008) [2023-12-26 17:58:18,525][105692] Updated weights for policy 0, policy_version 358127 (0.0010) [2023-12-26 17:58:18,583][105692] Updated weights for policy 0, policy_version 358137 (0.0011) [2023-12-26 17:58:18,645][105692] Updated weights for policy 0, policy_version 358147 (0.0009) [2023-12-26 17:58:18,997][105620] Updated weights for policy 1, policy_version 358610 (0.0008) [2023-12-26 17:58:19,063][105620] Updated weights for policy 1, policy_version 358620 (0.0008) [2023-12-26 17:58:19,110][105620] Updated weights for policy 1, policy_version 358630 (0.0008) [2023-12-26 17:58:19,383][105692] Updated weights for policy 0, policy_version 358157 (0.0009) [2023-12-26 17:58:19,449][105692] Updated weights for policy 0, policy_version 358167 (0.0005) [2023-12-26 17:58:19,518][105692] Updated weights for policy 0, policy_version 358177 (0.0009) [2023-12-26 17:58:19,769][105620] Updated weights for policy 1, policy_version 358640 (0.0006) [2023-12-26 17:58:19,828][105620] Updated weights for policy 1, policy_version 358650 (0.0006) [2023-12-26 17:58:19,881][105620] Updated weights for policy 1, policy_version 358660 (0.0008) [2023-12-26 17:58:20,229][105692] Updated weights for policy 0, policy_version 358187 (0.0009) [2023-12-26 17:58:20,297][105692] Updated weights for policy 0, policy_version 358197 (0.0009) [2023-12-26 17:58:20,360][105692] Updated weights for policy 0, policy_version 358207 (0.0009) [2023-12-26 17:58:20,610][105620] Updated weights for policy 1, policy_version 358670 (0.0008) [2023-12-26 17:58:20,668][105620] Updated weights for policy 1, policy_version 358680 (0.0006) [2023-12-26 17:58:20,725][105620] Updated weights for policy 1, policy_version 358690 (0.0006) [2023-12-26 17:58:21,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 183549952. Throughput: 0: 9535.0, 1: 9913.0. Samples: 183540856. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 17:58:21,063][104569] Avg episode reward: [(0, '9265.562'), (1, '9358.424')] [2023-12-26 17:58:21,184][105692] Updated weights for policy 0, policy_version 358217 (0.0009) [2023-12-26 17:58:21,246][105692] Updated weights for policy 0, policy_version 358227 (0.0008) [2023-12-26 17:58:21,313][105692] Updated weights for policy 0, policy_version 358237 (0.0008) [2023-12-26 17:58:21,381][105692] Updated weights for policy 0, policy_version 358247 (0.0009) [2023-12-26 17:58:21,437][105620] Updated weights for policy 1, policy_version 358700 (0.0007) [2023-12-26 17:58:21,502][105620] Updated weights for policy 1, policy_version 358710 (0.0009) [2023-12-26 17:58:21,561][105620] Updated weights for policy 1, policy_version 358720 (0.0009) [2023-12-26 17:58:22,085][105692] Updated weights for policy 0, policy_version 358257 (0.0006) [2023-12-26 17:58:22,141][105692] Updated weights for policy 0, policy_version 358267 (0.0009) [2023-12-26 17:58:22,195][105692] Updated weights for policy 0, policy_version 358277 (0.0007) [2023-12-26 17:58:22,389][105620] Updated weights for policy 1, policy_version 358730 (0.0008) [2023-12-26 17:58:22,450][105620] Updated weights for policy 1, policy_version 358740 (0.0009) [2023-12-26 17:58:22,503][105620] Updated weights for policy 1, policy_version 358750 (0.0011) [2023-12-26 17:58:22,564][105620] Updated weights for policy 1, policy_version 358760 (0.0010) [2023-12-26 17:58:22,908][105692] Updated weights for policy 0, policy_version 358287 (0.0009) [2023-12-26 17:58:22,970][105692] Updated weights for policy 0, policy_version 358297 (0.0009) [2023-12-26 17:58:23,028][105692] Updated weights for policy 0, policy_version 358307 (0.0010) [2023-12-26 17:58:23,311][105620] Updated weights for policy 1, policy_version 358770 (0.0010) [2023-12-26 17:58:23,356][105620] Updated weights for policy 1, policy_version 358780 (0.0010) [2023-12-26 17:58:23,408][105620] Updated weights for policy 1, policy_version 358790 (0.0010) [2023-12-26 17:58:23,774][105692] Updated weights for policy 0, policy_version 358317 (0.0010) [2023-12-26 17:58:23,832][105692] Updated weights for policy 0, policy_version 358327 (0.0010) [2023-12-26 17:58:23,897][105692] Updated weights for policy 0, policy_version 358337 (0.0011) [2023-12-26 17:58:24,184][105620] Updated weights for policy 1, policy_version 358800 (0.0010) [2023-12-26 17:58:24,244][105620] Updated weights for policy 1, policy_version 358810 (0.0010) [2023-12-26 17:58:24,298][105620] Updated weights for policy 1, policy_version 358820 (0.0010) [2023-12-26 17:58:24,627][105692] Updated weights for policy 0, policy_version 358347 (0.0010) [2023-12-26 17:58:24,678][105692] Updated weights for policy 0, policy_version 358357 (0.0010) [2023-12-26 17:58:24,733][105692] Updated weights for policy 0, policy_version 358367 (0.0010) [2023-12-26 17:58:24,882][105620] Updated weights for policy 1, policy_version 358830 (0.0007) [2023-12-26 17:58:24,932][105620] Updated weights for policy 1, policy_version 358840 (0.0005) [2023-12-26 17:58:24,991][105620] Updated weights for policy 1, policy_version 358850 (0.0005) [2023-12-26 17:58:25,502][105620] Updated weights for policy 1, policy_version 358860 (0.0007) [2023-12-26 17:58:25,503][105692] Updated weights for policy 0, policy_version 358377 (0.0010) [2023-12-26 17:58:25,552][105692] Updated weights for policy 0, policy_version 358387 (0.0005) [2023-12-26 17:58:25,556][105620] Updated weights for policy 1, policy_version 358870 (0.0010) [2023-12-26 17:58:25,605][105620] Updated weights for policy 1, policy_version 358880 (0.0009) [2023-12-26 17:58:25,606][105692] Updated weights for policy 0, policy_version 358397 (0.0008) [2023-12-26 17:58:25,661][105692] Updated weights for policy 0, policy_version 358407 (0.0009) [2023-12-26 17:58:26,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 183648256. Throughput: 0: 9529.2, 1: 9945.6. Samples: 183657752. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 17:58:26,063][104569] Avg episode reward: [(0, '9174.591'), (1, '9358.515')] [2023-12-26 17:58:26,164][105620] Updated weights for policy 1, policy_version 358890 (0.0006) [2023-12-26 17:58:26,211][105620] Updated weights for policy 1, policy_version 358900 (0.0005) [2023-12-26 17:58:26,260][105620] Updated weights for policy 1, policy_version 358910 (0.0005) [2023-12-26 17:58:26,311][105620] Updated weights for policy 1, policy_version 358920 (0.0007) [2023-12-26 17:58:26,459][105692] Updated weights for policy 0, policy_version 358417 (0.0008) [2023-12-26 17:58:26,515][105692] Updated weights for policy 0, policy_version 358427 (0.0009) [2023-12-26 17:58:26,564][105692] Updated weights for policy 0, policy_version 358437 (0.0008) [2023-12-26 17:58:27,027][105620] Updated weights for policy 1, policy_version 358930 (0.0010) [2023-12-26 17:58:27,082][105620] Updated weights for policy 1, policy_version 358940 (0.0010) [2023-12-26 17:58:27,136][105620] Updated weights for policy 1, policy_version 358950 (0.0010) [2023-12-26 17:58:27,330][105692] Updated weights for policy 0, policy_version 358447 (0.0008) [2023-12-26 17:58:27,379][105692] Updated weights for policy 0, policy_version 358457 (0.0008) [2023-12-26 17:58:27,427][105692] Updated weights for policy 0, policy_version 358467 (0.0008) [2023-12-26 17:58:27,889][105620] Updated weights for policy 1, policy_version 358960 (0.0010) [2023-12-26 17:58:27,946][105620] Updated weights for policy 1, policy_version 358970 (0.0010) [2023-12-26 17:58:28,017][105620] Updated weights for policy 1, policy_version 358980 (0.0010) [2023-12-26 17:58:28,196][105692] Updated weights for policy 0, policy_version 358477 (0.0008) [2023-12-26 17:58:28,251][105692] Updated weights for policy 0, policy_version 358487 (0.0008) [2023-12-26 17:58:28,310][105692] Updated weights for policy 0, policy_version 358497 (0.0008) [2023-12-26 17:58:28,753][105620] Updated weights for policy 1, policy_version 358990 (0.0010) [2023-12-26 17:58:28,807][105620] Updated weights for policy 1, policy_version 359000 (0.0010) [2023-12-26 17:58:28,862][105620] Updated weights for policy 1, policy_version 359010 (0.0010) [2023-12-26 17:58:29,089][105692] Updated weights for policy 0, policy_version 358507 (0.0009) [2023-12-26 17:58:29,137][105692] Updated weights for policy 0, policy_version 358517 (0.0008) [2023-12-26 17:58:29,184][105692] Updated weights for policy 0, policy_version 358527 (0.0008) [2023-12-26 17:58:29,228][105585] KL-divergence is very high: 102.6667 [2023-12-26 17:58:29,616][105620] Updated weights for policy 1, policy_version 359020 (0.0010) [2023-12-26 17:58:29,671][105620] Updated weights for policy 1, policy_version 359030 (0.0010) [2023-12-26 17:58:29,736][105620] Updated weights for policy 1, policy_version 359040 (0.0010) [2023-12-26 17:58:29,975][105692] Updated weights for policy 0, policy_version 358537 (0.0008) [2023-12-26 17:58:30,038][105692] Updated weights for policy 0, policy_version 358547 (0.0008) [2023-12-26 17:58:30,102][105692] Updated weights for policy 0, policy_version 358557 (0.0008) [2023-12-26 17:58:30,154][105692] Updated weights for policy 0, policy_version 358567 (0.0008) [2023-12-26 17:58:30,505][105620] Updated weights for policy 1, policy_version 359050 (0.0010) [2023-12-26 17:58:30,563][105620] Updated weights for policy 1, policy_version 359060 (0.0010) [2023-12-26 17:58:30,618][105620] Updated weights for policy 1, policy_version 359070 (0.0010) [2023-12-26 17:58:30,674][105620] Updated weights for policy 1, policy_version 359080 (0.0010) [2023-12-26 17:58:30,928][105692] Updated weights for policy 0, policy_version 358577 (0.0009) [2023-12-26 17:58:30,987][105692] Updated weights for policy 0, policy_version 358587 (0.0008) [2023-12-26 17:58:31,049][105692] Updated weights for policy 0, policy_version 358597 (0.0009) [2023-12-26 17:58:31,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 183738368. Throughput: 0: 9526.2, 1: 9959.1. Samples: 183714360. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 17:58:31,062][104569] Avg episode reward: [(0, '9174.537'), (1, '9175.880')] [2023-12-26 17:58:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000358600_91815936.pth... [2023-12-26 17:58:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000359080_91930624.pth... [2023-12-26 17:58:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000357480_91529216.pth [2023-12-26 17:58:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000357928_91635712.pth [2023-12-26 17:58:31,432][105620] Updated weights for policy 1, policy_version 359090 (0.0010) [2023-12-26 17:58:31,483][105620] Updated weights for policy 1, policy_version 359100 (0.0010) [2023-12-26 17:58:31,538][105620] Updated weights for policy 1, policy_version 359110 (0.0010) [2023-12-26 17:58:31,820][105692] Updated weights for policy 0, policy_version 358607 (0.0009) [2023-12-26 17:58:31,878][105692] Updated weights for policy 0, policy_version 358617 (0.0007) [2023-12-26 17:58:31,943][105692] Updated weights for policy 0, policy_version 358627 (0.0008) [2023-12-26 17:58:32,267][105620] Updated weights for policy 1, policy_version 359120 (0.0007) [2023-12-26 17:58:32,318][105620] Updated weights for policy 1, policy_version 359130 (0.0009) [2023-12-26 17:58:32,381][105620] Updated weights for policy 1, policy_version 359140 (0.0009) [2023-12-26 17:58:32,617][105692] Updated weights for policy 0, policy_version 358637 (0.0008) [2023-12-26 17:58:32,676][105692] Updated weights for policy 0, policy_version 358647 (0.0006) [2023-12-26 17:58:32,725][105692] Updated weights for policy 0, policy_version 358657 (0.0005) [2023-12-26 17:58:33,187][105620] Updated weights for policy 1, policy_version 359150 (0.0009) [2023-12-26 17:58:33,244][105620] Updated weights for policy 1, policy_version 359160 (0.0010) [2023-12-26 17:58:33,308][105620] Updated weights for policy 1, policy_version 359171 (0.0009) [2023-12-26 17:58:33,346][105692] Updated weights for policy 0, policy_version 358667 (0.0006) [2023-12-26 17:58:33,399][105692] Updated weights for policy 0, policy_version 358677 (0.0008) [2023-12-26 17:58:33,456][105692] Updated weights for policy 0, policy_version 358687 (0.0010) [2023-12-26 17:58:34,081][105620] Updated weights for policy 1, policy_version 359181 (0.0009) [2023-12-26 17:58:34,138][105620] Updated weights for policy 1, policy_version 359191 (0.0011) [2023-12-26 17:58:34,188][105692] Updated weights for policy 0, policy_version 358697 (0.0007) [2023-12-26 17:58:34,201][105620] Updated weights for policy 1, policy_version 359201 (0.0011) [2023-12-26 17:58:34,247][105692] Updated weights for policy 0, policy_version 358707 (0.0010) [2023-12-26 17:58:34,310][105692] Updated weights for policy 0, policy_version 358717 (0.0011) [2023-12-26 17:58:34,372][105692] Updated weights for policy 0, policy_version 358727 (0.0010) [2023-12-26 17:58:34,915][105620] Updated weights for policy 1, policy_version 359211 (0.0009) [2023-12-26 17:58:34,971][105620] Updated weights for policy 1, policy_version 359221 (0.0006) [2023-12-26 17:58:35,031][105620] Updated weights for policy 1, policy_version 359231 (0.0009) [2023-12-26 17:58:35,107][105692] Updated weights for policy 0, policy_version 358737 (0.0006) [2023-12-26 17:58:35,163][105692] Updated weights for policy 0, policy_version 358747 (0.0005) [2023-12-26 17:58:35,222][105692] Updated weights for policy 0, policy_version 358757 (0.0005) [2023-12-26 17:58:35,564][105620] Updated weights for policy 1, policy_version 359241 (0.0005) [2023-12-26 17:58:35,622][105620] Updated weights for policy 1, policy_version 359251 (0.0006) [2023-12-26 17:58:35,670][105620] Updated weights for policy 1, policy_version 359261 (0.0010) [2023-12-26 17:58:35,731][105620] Updated weights for policy 1, policy_version 359271 (0.0008) [2023-12-26 17:58:35,763][105692] Updated weights for policy 0, policy_version 358767 (0.0008) [2023-12-26 17:58:35,820][105692] Updated weights for policy 0, policy_version 358777 (0.0010) [2023-12-26 17:58:35,879][105692] Updated weights for policy 0, policy_version 358788 (0.0010) [2023-12-26 17:58:36,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 183844864. Throughput: 0: 9530.9, 1: 9911.0. Samples: 183827344. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 17:58:36,062][104569] Avg episode reward: [(0, '9175.244'), (1, '8990.420')] [2023-12-26 17:58:36,285][105620] Updated weights for policy 1, policy_version 359281 (0.0007) [2023-12-26 17:58:36,350][105620] Updated weights for policy 1, policy_version 359291 (0.0008) [2023-12-26 17:58:36,418][105620] Updated weights for policy 1, policy_version 359301 (0.0007) [2023-12-26 17:58:36,797][105692] Updated weights for policy 0, policy_version 358798 (0.0009) [2023-12-26 17:58:36,855][105692] Updated weights for policy 0, policy_version 358808 (0.0008) [2023-12-26 17:58:36,913][105692] Updated weights for policy 0, policy_version 358818 (0.0007) [2023-12-26 17:58:36,986][105620] Updated weights for policy 1, policy_version 359311 (0.0007) [2023-12-26 17:58:37,039][105620] Updated weights for policy 1, policy_version 359321 (0.0009) [2023-12-26 17:58:37,091][105620] Updated weights for policy 1, policy_version 359331 (0.0006) [2023-12-26 17:58:37,734][105692] Updated weights for policy 0, policy_version 358828 (0.0008) [2023-12-26 17:58:37,752][105620] Updated weights for policy 1, policy_version 359341 (0.0007) [2023-12-26 17:58:37,787][105692] Updated weights for policy 0, policy_version 358838 (0.0007) [2023-12-26 17:58:37,801][105620] Updated weights for policy 1, policy_version 359351 (0.0006) [2023-12-26 17:58:37,832][105692] Updated weights for policy 0, policy_version 358848 (0.0006) [2023-12-26 17:58:37,850][105620] Updated weights for policy 1, policy_version 359361 (0.0006) [2023-12-26 17:58:38,482][105620] Updated weights for policy 1, policy_version 359371 (0.0006) [2023-12-26 17:58:38,546][105620] Updated weights for policy 1, policy_version 359381 (0.0007) [2023-12-26 17:58:38,609][105620] Updated weights for policy 1, policy_version 359391 (0.0009) [2023-12-26 17:58:38,650][105692] Updated weights for policy 0, policy_version 358858 (0.0008) [2023-12-26 17:58:38,709][105692] Updated weights for policy 0, policy_version 358868 (0.0008) [2023-12-26 17:58:38,772][105692] Updated weights for policy 0, policy_version 358878 (0.0008) [2023-12-26 17:58:38,830][105692] Updated weights for policy 0, policy_version 358888 (0.0009) [2023-12-26 17:58:39,230][105620] Updated weights for policy 1, policy_version 359401 (0.0008) [2023-12-26 17:58:39,293][105620] Updated weights for policy 1, policy_version 359411 (0.0007) [2023-12-26 17:58:39,363][105620] Updated weights for policy 1, policy_version 359421 (0.0007) [2023-12-26 17:58:39,431][105620] Updated weights for policy 1, policy_version 359431 (0.0008) [2023-12-26 17:58:39,635][105692] Updated weights for policy 0, policy_version 358898 (0.0009) [2023-12-26 17:58:39,701][105692] Updated weights for policy 0, policy_version 358908 (0.0008) [2023-12-26 17:58:39,764][105692] Updated weights for policy 0, policy_version 358918 (0.0009) [2023-12-26 17:58:40,149][105620] Updated weights for policy 1, policy_version 359441 (0.0008) [2023-12-26 17:58:40,207][105620] Updated weights for policy 1, policy_version 359451 (0.0009) [2023-12-26 17:58:40,268][105620] Updated weights for policy 1, policy_version 359461 (0.0009) [2023-12-26 17:58:40,527][105692] Updated weights for policy 0, policy_version 358928 (0.0009) [2023-12-26 17:58:40,582][105692] Updated weights for policy 0, policy_version 358938 (0.0005) [2023-12-26 17:58:40,647][105692] Updated weights for policy 0, policy_version 358948 (0.0005) [2023-12-26 17:58:40,950][105620] Updated weights for policy 1, policy_version 359471 (0.0007) [2023-12-26 17:58:41,001][105620] Updated weights for policy 1, policy_version 359481 (0.0005) [2023-12-26 17:58:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 183934976. Throughput: 0: 9479.5, 1: 9976.9. Samples: 183947348. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 17:58:41,063][104569] Avg episode reward: [(0, '9176.022'), (1, '9080.800')] [2023-12-26 17:58:41,063][105620] Updated weights for policy 1, policy_version 359491 (0.0007) [2023-12-26 17:58:41,326][105692] Updated weights for policy 0, policy_version 358958 (0.0008) [2023-12-26 17:58:41,398][105692] Updated weights for policy 0, policy_version 358968 (0.0008) [2023-12-26 17:58:41,462][105692] Updated weights for policy 0, policy_version 358978 (0.0007) [2023-12-26 17:58:41,786][105620] Updated weights for policy 1, policy_version 359501 (0.0009) [2023-12-26 17:58:41,856][105620] Updated weights for policy 1, policy_version 359511 (0.0008) [2023-12-26 17:58:41,921][105620] Updated weights for policy 1, policy_version 359521 (0.0008) [2023-12-26 17:58:42,143][105692] Updated weights for policy 0, policy_version 358988 (0.0009) [2023-12-26 17:58:42,199][105692] Updated weights for policy 0, policy_version 358998 (0.0009) [2023-12-26 17:58:42,257][105692] Updated weights for policy 0, policy_version 359008 (0.0009) [2023-12-26 17:58:42,668][105620] Updated weights for policy 1, policy_version 359531 (0.0008) [2023-12-26 17:58:42,725][105620] Updated weights for policy 1, policy_version 359541 (0.0008) [2023-12-26 17:58:42,787][105620] Updated weights for policy 1, policy_version 359551 (0.0009) [2023-12-26 17:58:42,948][105692] Updated weights for policy 0, policy_version 359018 (0.0008) [2023-12-26 17:58:43,014][105692] Updated weights for policy 0, policy_version 359028 (0.0009) [2023-12-26 17:58:43,073][105692] Updated weights for policy 0, policy_version 359038 (0.0009) [2023-12-26 17:58:43,135][105692] Updated weights for policy 0, policy_version 359048 (0.0009) [2023-12-26 17:58:43,553][105620] Updated weights for policy 1, policy_version 359561 (0.0009) [2023-12-26 17:58:43,624][105620] Updated weights for policy 1, policy_version 359571 (0.0007) [2023-12-26 17:58:43,688][105620] Updated weights for policy 1, policy_version 359581 (0.0008) [2023-12-26 17:58:43,754][105620] Updated weights for policy 1, policy_version 359591 (0.0007) [2023-12-26 17:58:43,873][105692] Updated weights for policy 0, policy_version 359058 (0.0009) [2023-12-26 17:58:43,927][105692] Updated weights for policy 0, policy_version 359068 (0.0009) [2023-12-26 17:58:43,993][105692] Updated weights for policy 0, policy_version 359078 (0.0009) [2023-12-26 17:58:44,347][105620] Updated weights for policy 1, policy_version 359601 (0.0005) [2023-12-26 17:58:44,391][105620] Updated weights for policy 1, policy_version 359611 (0.0005) [2023-12-26 17:58:44,449][105620] Updated weights for policy 1, policy_version 359621 (0.0008) [2023-12-26 17:58:44,831][105692] Updated weights for policy 0, policy_version 359088 (0.0010) [2023-12-26 17:58:44,882][105692] Updated weights for policy 0, policy_version 359098 (0.0009) [2023-12-26 17:58:44,930][105692] Updated weights for policy 0, policy_version 359108 (0.0009) [2023-12-26 17:58:45,166][105620] Updated weights for policy 1, policy_version 359631 (0.0009) [2023-12-26 17:58:45,229][105620] Updated weights for policy 1, policy_version 359641 (0.0009) [2023-12-26 17:58:45,293][105620] Updated weights for policy 1, policy_version 359651 (0.0009) [2023-12-26 17:58:45,721][105692] Updated weights for policy 0, policy_version 359118 (0.0007) [2023-12-26 17:58:45,783][105692] Updated weights for policy 0, policy_version 359128 (0.0008) [2023-12-26 17:58:45,850][105692] Updated weights for policy 0, policy_version 359138 (0.0010) [2023-12-26 17:58:45,998][105620] Updated weights for policy 1, policy_version 359661 (0.0007) [2023-12-26 17:58:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 184033280. Throughput: 0: 9503.3, 1: 9907.5. Samples: 184004504. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 17:58:46,062][104569] Avg episode reward: [(0, '8905.161'), (1, '9173.570')] [2023-12-26 17:58:46,067][105620] Updated weights for policy 1, policy_version 359671 (0.0005) [2023-12-26 17:58:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000359144_91955200.pth... [2023-12-26 17:58:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000358056_91676672.pth [2023-12-26 17:58:46,125][105620] Updated weights for policy 1, policy_version 359681 (0.0008) [2023-12-26 17:58:46,157][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000359688_92086272.pth... [2023-12-26 17:58:46,160][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000358504_91783168.pth [2023-12-26 17:58:46,625][105692] Updated weights for policy 0, policy_version 359148 (0.0008) [2023-12-26 17:58:46,682][105692] Updated weights for policy 0, policy_version 359158 (0.0009) [2023-12-26 17:58:46,739][105692] Updated weights for policy 0, policy_version 359168 (0.0008) [2023-12-26 17:58:46,807][105620] Updated weights for policy 1, policy_version 359691 (0.0009) [2023-12-26 17:58:46,861][105620] Updated weights for policy 1, policy_version 359701 (0.0009) [2023-12-26 17:58:46,912][105620] Updated weights for policy 1, policy_version 359711 (0.0008) [2023-12-26 17:58:47,470][105692] Updated weights for policy 0, policy_version 359178 (0.0008) [2023-12-26 17:58:47,528][105692] Updated weights for policy 0, policy_version 359188 (0.0006) [2023-12-26 17:58:47,595][105692] Updated weights for policy 0, policy_version 359198 (0.0007) [2023-12-26 17:58:47,654][105692] Updated weights for policy 0, policy_version 359208 (0.0009) [2023-12-26 17:58:47,664][105620] Updated weights for policy 1, policy_version 359721 (0.0009) [2023-12-26 17:58:47,714][105620] Updated weights for policy 1, policy_version 359731 (0.0009) [2023-12-26 17:58:47,762][105620] Updated weights for policy 1, policy_version 359741 (0.0009) [2023-12-26 17:58:47,816][105620] Updated weights for policy 1, policy_version 359751 (0.0009) [2023-12-26 17:58:48,404][105620] Updated weights for policy 1, policy_version 359761 (0.0006) [2023-12-26 17:58:48,446][105692] Updated weights for policy 0, policy_version 359218 (0.0008) [2023-12-26 17:58:48,464][105620] Updated weights for policy 1, policy_version 359771 (0.0007) [2023-12-26 17:58:48,495][105692] Updated weights for policy 0, policy_version 359228 (0.0006) [2023-12-26 17:58:48,521][105620] Updated weights for policy 1, policy_version 359781 (0.0008) [2023-12-26 17:58:48,549][105692] Updated weights for policy 0, policy_version 359238 (0.0007) [2023-12-26 17:58:49,207][105620] Updated weights for policy 1, policy_version 359791 (0.0008) [2023-12-26 17:58:49,278][105620] Updated weights for policy 1, policy_version 359801 (0.0008) [2023-12-26 17:58:49,339][105620] Updated weights for policy 1, policy_version 359811 (0.0008) [2023-12-26 17:58:49,367][105692] Updated weights for policy 0, policy_version 359248 (0.0008) [2023-12-26 17:58:49,431][105692] Updated weights for policy 0, policy_version 359258 (0.0008) [2023-12-26 17:58:49,473][105585] KL-divergence is very high: 139.6959 [2023-12-26 17:58:49,478][105692] Updated weights for policy 0, policy_version 359268 (0.0005) [2023-12-26 17:58:50,092][105620] Updated weights for policy 1, policy_version 359821 (0.0008) [2023-12-26 17:58:50,141][105585] KL-divergence is very high: 135.6738 [2023-12-26 17:58:50,154][105620] Updated weights for policy 1, policy_version 359831 (0.0007) [2023-12-26 17:58:50,155][105692] Updated weights for policy 0, policy_version 359278 (0.0009) [2023-12-26 17:58:50,187][105585] KL-divergence is very high: 136.3540 [2023-12-26 17:58:50,210][105692] Updated weights for policy 0, policy_version 359288 (0.0011) [2023-12-26 17:58:50,222][105620] Updated weights for policy 1, policy_version 359841 (0.0005) [2023-12-26 17:58:50,236][105585] KL-divergence is very high: 202.1093 [2023-12-26 17:58:50,277][105692] Updated weights for policy 0, policy_version 359298 (0.0011) [2023-12-26 17:58:50,287][105585] KL-divergence is very high: 139.0347 [2023-12-26 17:58:50,856][105620] Updated weights for policy 1, policy_version 359851 (0.0006) [2023-12-26 17:58:50,918][105620] Updated weights for policy 1, policy_version 359861 (0.0010) [2023-12-26 17:58:50,946][105692] Updated weights for policy 0, policy_version 359308 (0.0008) [2023-12-26 17:58:50,976][105620] Updated weights for policy 1, policy_version 359871 (0.0008) [2023-12-26 17:58:50,995][105585] KL-divergence is very high: 117.0236 [2023-12-26 17:58:51,016][105692] Updated weights for policy 0, policy_version 359318 (0.0006) [2023-12-26 17:58:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.5). Total num frames: 184131584. Throughput: 0: 9424.0, 1: 9986.0. Samples: 184119096. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 17:58:51,062][104569] Avg episode reward: [(0, '2744.314'), (1, '9081.097')] [2023-12-26 17:58:51,082][105692] Updated weights for policy 0, policy_version 359328 (0.0008) [2023-12-26 17:58:51,712][105692] Updated weights for policy 0, policy_version 359338 (0.0011) [2023-12-26 17:58:51,774][105692] Updated weights for policy 0, policy_version 359348 (0.0009) [2023-12-26 17:58:51,838][105620] Updated weights for policy 1, policy_version 359881 (0.0008) [2023-12-26 17:58:51,839][105692] Updated weights for policy 0, policy_version 359358 (0.0011) [2023-12-26 17:58:51,900][105692] Updated weights for policy 0, policy_version 359368 (0.0011) [2023-12-26 17:58:51,900][105620] Updated weights for policy 1, policy_version 359891 (0.0007) [2023-12-26 17:58:51,948][105620] Updated weights for policy 1, policy_version 359901 (0.0008) [2023-12-26 17:58:52,000][105620] Updated weights for policy 1, policy_version 359911 (0.0008) [2023-12-26 17:58:52,621][105692] Updated weights for policy 0, policy_version 359378 (0.0010) [2023-12-26 17:58:52,680][105692] Updated weights for policy 0, policy_version 359388 (0.0011) [2023-12-26 17:58:52,749][105692] Updated weights for policy 0, policy_version 359398 (0.0011) [2023-12-26 17:58:52,786][105620] Updated weights for policy 1, policy_version 359921 (0.0008) [2023-12-26 17:58:52,839][105620] Updated weights for policy 1, policy_version 359931 (0.0007) [2023-12-26 17:58:52,894][105620] Updated weights for policy 1, policy_version 359941 (0.0005) [2023-12-26 17:58:53,416][105692] Updated weights for policy 0, policy_version 359408 (0.0009) [2023-12-26 17:58:53,467][105692] Updated weights for policy 0, policy_version 359418 (0.0009) [2023-12-26 17:58:53,514][105692] Updated weights for policy 0, policy_version 359428 (0.0009) [2023-12-26 17:58:53,613][105620] Updated weights for policy 1, policy_version 359951 (0.0008) [2023-12-26 17:58:53,668][105620] Updated weights for policy 1, policy_version 359961 (0.0009) [2023-12-26 17:58:53,722][105620] Updated weights for policy 1, policy_version 359971 (0.0009) [2023-12-26 17:58:54,278][105692] Updated weights for policy 0, policy_version 359438 (0.0009) [2023-12-26 17:58:54,340][105692] Updated weights for policy 0, policy_version 359448 (0.0009) [2023-12-26 17:58:54,407][105692] Updated weights for policy 0, policy_version 359458 (0.0009) [2023-12-26 17:58:54,478][105620] Updated weights for policy 1, policy_version 359981 (0.0009) [2023-12-26 17:58:54,535][105620] Updated weights for policy 1, policy_version 359991 (0.0008) [2023-12-26 17:58:54,543][105586] KL-divergence is very high: 143.1939 [2023-12-26 17:58:54,548][105586] KL-divergence is very high: 137.3037 [2023-12-26 17:58:54,581][105586] KL-divergence is very high: 134.9985 [2023-12-26 17:58:54,582][105620] Updated weights for policy 1, policy_version 360001 (0.0008) [2023-12-26 17:58:54,586][105586] KL-divergence is very high: 122.4699 [2023-12-26 17:58:55,148][105692] Updated weights for policy 0, policy_version 359468 (0.0010) [2023-12-26 17:58:55,206][105692] Updated weights for policy 0, policy_version 359478 (0.0009) [2023-12-26 17:58:55,253][105692] Updated weights for policy 0, policy_version 359488 (0.0009) [2023-12-26 17:58:55,349][105620] Updated weights for policy 1, policy_version 360011 (0.0009) [2023-12-26 17:58:55,394][105620] Updated weights for policy 1, policy_version 360021 (0.0007) [2023-12-26 17:58:55,446][105620] Updated weights for policy 1, policy_version 360031 (0.0005) [2023-12-26 17:58:56,045][105692] Updated weights for policy 0, policy_version 359498 (0.0009) [2023-12-26 17:58:56,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.3, 300 sec: 19438.7). Total num frames: 184221696. Throughput: 0: 9457.3, 1: 9962.0. Samples: 184234340. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 17:58:56,062][104569] Avg episode reward: [(0, '2432.592'), (1, '8988.871')] [2023-12-26 17:58:56,096][105692] Updated weights for policy 0, policy_version 359508 (0.0009) [2023-12-26 17:58:56,143][105692] Updated weights for policy 0, policy_version 359518 (0.0009) [2023-12-26 17:58:56,150][105620] Updated weights for policy 1, policy_version 360041 (0.0005) [2023-12-26 17:58:56,192][105692] Updated weights for policy 0, policy_version 359528 (0.0006) [2023-12-26 17:58:56,206][105620] Updated weights for policy 1, policy_version 360051 (0.0008) [2023-12-26 17:58:56,253][105620] Updated weights for policy 1, policy_version 360061 (0.0009) [2023-12-26 17:58:56,314][105620] Updated weights for policy 1, policy_version 360071 (0.0009) [2023-12-26 17:58:56,865][105692] Updated weights for policy 0, policy_version 359538 (0.0005) [2023-12-26 17:58:56,920][105692] Updated weights for policy 0, policy_version 359548 (0.0005) [2023-12-26 17:58:56,942][105620] Updated weights for policy 1, policy_version 360081 (0.0009) [2023-12-26 17:58:56,968][105692] Updated weights for policy 0, policy_version 359558 (0.0006) [2023-12-26 17:58:56,994][105620] Updated weights for policy 1, policy_version 360091 (0.0007) [2023-12-26 17:58:57,046][105620] Updated weights for policy 1, policy_version 360101 (0.0009) [2023-12-26 17:58:57,535][105692] Updated weights for policy 0, policy_version 359568 (0.0005) [2023-12-26 17:58:57,592][105692] Updated weights for policy 0, policy_version 359578 (0.0005) [2023-12-26 17:58:57,629][105585] KL-divergence is very high: 108.7172 [2023-12-26 17:58:57,635][105585] KL-divergence is very high: 114.8891 [2023-12-26 17:58:57,653][105692] Updated weights for policy 0, policy_version 359588 (0.0008) [2023-12-26 17:58:57,665][105585] KL-divergence is very high: 104.6599 [2023-12-26 17:58:57,907][105620] Updated weights for policy 1, policy_version 360111 (0.0010) [2023-12-26 17:58:57,958][105620] Updated weights for policy 1, policy_version 360121 (0.0009) [2023-12-26 17:58:58,009][105620] Updated weights for policy 1, policy_version 360131 (0.0009) [2023-12-26 17:58:58,244][105692] Updated weights for policy 0, policy_version 359598 (0.0010) [2023-12-26 17:58:58,304][105692] Updated weights for policy 0, policy_version 359608 (0.0010) [2023-12-26 17:58:58,382][105692] Updated weights for policy 0, policy_version 359618 (0.0008) [2023-12-26 17:58:58,867][105620] Updated weights for policy 1, policy_version 360141 (0.0009) [2023-12-26 17:58:58,926][105620] Updated weights for policy 1, policy_version 360151 (0.0009) [2023-12-26 17:58:58,983][105620] Updated weights for policy 1, policy_version 360161 (0.0009) [2023-12-26 17:58:59,131][105692] Updated weights for policy 0, policy_version 359628 (0.0010) [2023-12-26 17:58:59,190][105692] Updated weights for policy 0, policy_version 359638 (0.0008) [2023-12-26 17:58:59,283][105692] Updated weights for policy 0, policy_version 359648 (0.0007) [2023-12-26 17:58:59,781][105620] Updated weights for policy 1, policy_version 360171 (0.0009) [2023-12-26 17:58:59,835][105620] Updated weights for policy 1, policy_version 360181 (0.0009) [2023-12-26 17:58:59,894][105620] Updated weights for policy 1, policy_version 360191 (0.0009) [2023-12-26 17:58:59,982][105692] Updated weights for policy 0, policy_version 359658 (0.0007) [2023-12-26 17:59:00,045][105692] Updated weights for policy 0, policy_version 359668 (0.0005) [2023-12-26 17:59:00,101][105692] Updated weights for policy 0, policy_version 359678 (0.0005) [2023-12-26 17:59:00,156][105692] Updated weights for policy 0, policy_version 359688 (0.0008) [2023-12-26 17:59:00,585][105620] Updated weights for policy 1, policy_version 360201 (0.0008) [2023-12-26 17:59:00,652][105620] Updated weights for policy 1, policy_version 360211 (0.0005) [2023-12-26 17:59:00,716][105620] Updated weights for policy 1, policy_version 360221 (0.0006) [2023-12-26 17:59:00,771][105620] Updated weights for policy 1, policy_version 360231 (0.0007) [2023-12-26 17:59:00,918][105692] Updated weights for policy 0, policy_version 359698 (0.0008) [2023-12-26 17:59:00,979][105692] Updated weights for policy 0, policy_version 359708 (0.0008) [2023-12-26 17:59:01,045][105692] Updated weights for policy 0, policy_version 359718 (0.0009) [2023-12-26 17:59:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 184328192. Throughput: 0: 9507.1, 1: 9924.4. Samples: 184293564. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 17:59:01,062][104569] Avg episode reward: [(0, '922.017'), (1, '8988.881')] [2023-12-26 17:59:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000359720_92102656.pth... [2023-12-26 17:59:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000360232_92225536.pth... [2023-12-26 17:59:01,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000359080_91930624.pth [2023-12-26 17:59:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000358600_91815936.pth [2023-12-26 17:59:01,449][105620] Updated weights for policy 1, policy_version 360241 (0.0009) [2023-12-26 17:59:01,505][105620] Updated weights for policy 1, policy_version 360251 (0.0008) [2023-12-26 17:59:01,557][105620] Updated weights for policy 1, policy_version 360261 (0.0008) [2023-12-26 17:59:01,832][105692] Updated weights for policy 0, policy_version 359728 (0.0010) [2023-12-26 17:59:01,892][105692] Updated weights for policy 0, policy_version 359738 (0.0007) [2023-12-26 17:59:01,951][105692] Updated weights for policy 0, policy_version 359748 (0.0009) [2023-12-26 17:59:02,262][105620] Updated weights for policy 1, policy_version 360271 (0.0009) [2023-12-26 17:59:02,315][105620] Updated weights for policy 1, policy_version 360281 (0.0008) [2023-12-26 17:59:02,375][105620] Updated weights for policy 1, policy_version 360291 (0.0009) [2023-12-26 17:59:02,629][105692] Updated weights for policy 0, policy_version 359758 (0.0008) [2023-12-26 17:59:02,676][105692] Updated weights for policy 0, policy_version 359768 (0.0009) [2023-12-26 17:59:02,728][105692] Updated weights for policy 0, policy_version 359778 (0.0009) [2023-12-26 17:59:03,076][105620] Updated weights for policy 1, policy_version 360301 (0.0007) [2023-12-26 17:59:03,133][105620] Updated weights for policy 1, policy_version 360311 (0.0006) [2023-12-26 17:59:03,190][105620] Updated weights for policy 1, policy_version 360321 (0.0010) [2023-12-26 17:59:03,476][105692] Updated weights for policy 0, policy_version 359788 (0.0009) [2023-12-26 17:59:03,529][105692] Updated weights for policy 0, policy_version 359798 (0.0010) [2023-12-26 17:59:03,588][105692] Updated weights for policy 0, policy_version 359808 (0.0010) [2023-12-26 17:59:03,816][105620] Updated weights for policy 1, policy_version 360331 (0.0007) [2023-12-26 17:59:03,883][105620] Updated weights for policy 1, policy_version 360341 (0.0008) [2023-12-26 17:59:03,939][105620] Updated weights for policy 1, policy_version 360351 (0.0009) [2023-12-26 17:59:04,391][105692] Updated weights for policy 0, policy_version 359818 (0.0009) [2023-12-26 17:59:04,455][105692] Updated weights for policy 0, policy_version 359828 (0.0009) [2023-12-26 17:59:04,519][105692] Updated weights for policy 0, policy_version 359838 (0.0009) [2023-12-26 17:59:04,655][105620] Updated weights for policy 1, policy_version 360361 (0.0009) [2023-12-26 17:59:04,707][105620] Updated weights for policy 1, policy_version 360371 (0.0009) [2023-12-26 17:59:04,762][105620] Updated weights for policy 1, policy_version 360381 (0.0009) [2023-12-26 17:59:04,813][105620] Updated weights for policy 1, policy_version 360391 (0.0009) [2023-12-26 17:59:05,339][105692] Updated weights for policy 0, policy_version 359849 (0.0010) [2023-12-26 17:59:05,392][105692] Updated weights for policy 0, policy_version 359859 (0.0006) [2023-12-26 17:59:05,435][105692] Updated weights for policy 0, policy_version 359869 (0.0005) [2023-12-26 17:59:05,481][105692] Updated weights for policy 0, policy_version 359879 (0.0005) [2023-12-26 17:59:05,497][105620] Updated weights for policy 1, policy_version 360401 (0.0009) [2023-12-26 17:59:05,546][105620] Updated weights for policy 1, policy_version 360411 (0.0009) [2023-12-26 17:59:05,599][105620] Updated weights for policy 1, policy_version 360422 (0.0009) [2023-12-26 17:59:06,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 184418304. Throughput: 0: 9435.5, 1: 9835.7. Samples: 184408060. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 17:59:06,063][104569] Avg episode reward: [(0, '6071.581'), (1, '8988.888')] [2023-12-26 17:59:06,208][105692] Updated weights for policy 0, policy_version 359889 (0.0009) [2023-12-26 17:59:06,270][105692] Updated weights for policy 0, policy_version 359899 (0.0009) [2023-12-26 17:59:06,335][105692] Updated weights for policy 0, policy_version 359909 (0.0009) [2023-12-26 17:59:06,375][105620] Updated weights for policy 1, policy_version 360432 (0.0008) [2023-12-26 17:59:06,439][105620] Updated weights for policy 1, policy_version 360442 (0.0008) [2023-12-26 17:59:06,509][105620] Updated weights for policy 1, policy_version 360452 (0.0005) [2023-12-26 17:59:07,086][105692] Updated weights for policy 0, policy_version 359919 (0.0008) [2023-12-26 17:59:07,138][105692] Updated weights for policy 0, policy_version 359929 (0.0009) [2023-12-26 17:59:07,199][105692] Updated weights for policy 0, policy_version 359939 (0.0009) [2023-12-26 17:59:07,210][105620] Updated weights for policy 1, policy_version 360462 (0.0006) [2023-12-26 17:59:07,263][105620] Updated weights for policy 1, policy_version 360472 (0.0007) [2023-12-26 17:59:07,317][105620] Updated weights for policy 1, policy_version 360482 (0.0010) [2023-12-26 17:59:07,977][105692] Updated weights for policy 0, policy_version 359949 (0.0009) [2023-12-26 17:59:08,027][105692] Updated weights for policy 0, policy_version 359959 (0.0008) [2023-12-26 17:59:08,056][105620] Updated weights for policy 1, policy_version 360492 (0.0007) [2023-12-26 17:59:08,080][105692] Updated weights for policy 0, policy_version 359969 (0.0008) [2023-12-26 17:59:08,101][105620] Updated weights for policy 1, policy_version 360502 (0.0008) [2023-12-26 17:59:08,150][105620] Updated weights for policy 1, policy_version 360512 (0.0006) [2023-12-26 17:59:08,866][105620] Updated weights for policy 1, policy_version 360522 (0.0008) [2023-12-26 17:59:08,876][105692] Updated weights for policy 0, policy_version 359979 (0.0008) [2023-12-26 17:59:08,928][105620] Updated weights for policy 1, policy_version 360532 (0.0006) [2023-12-26 17:59:08,932][105692] Updated weights for policy 0, policy_version 359989 (0.0009) [2023-12-26 17:59:08,986][105620] Updated weights for policy 1, policy_version 360542 (0.0008) [2023-12-26 17:59:08,992][105692] Updated weights for policy 0, policy_version 359999 (0.0007) [2023-12-26 17:59:09,046][105620] Updated weights for policy 1, policy_version 360552 (0.0011) [2023-12-26 17:59:09,759][105620] Updated weights for policy 1, policy_version 360562 (0.0011) [2023-12-26 17:59:09,800][105692] Updated weights for policy 0, policy_version 360009 (0.0005) [2023-12-26 17:59:09,829][105620] Updated weights for policy 1, policy_version 360572 (0.0008) [2023-12-26 17:59:09,863][105692] Updated weights for policy 0, policy_version 360019 (0.0008) [2023-12-26 17:59:09,893][105620] Updated weights for policy 1, policy_version 360582 (0.0010) [2023-12-26 17:59:09,926][105692] Updated weights for policy 0, policy_version 360029 (0.0008) [2023-12-26 17:59:09,984][105692] Updated weights for policy 0, policy_version 360039 (0.0009) [2023-12-26 17:59:10,585][105620] Updated weights for policy 1, policy_version 360592 (0.0010) [2023-12-26 17:59:10,637][105620] Updated weights for policy 1, policy_version 360602 (0.0010) [2023-12-26 17:59:10,682][105620] Updated weights for policy 1, policy_version 360612 (0.0010) [2023-12-26 17:59:10,772][105692] Updated weights for policy 0, policy_version 360049 (0.0008) [2023-12-26 17:59:10,828][105692] Updated weights for policy 0, policy_version 360059 (0.0007) [2023-12-26 17:59:10,888][105692] Updated weights for policy 0, policy_version 360069 (0.0008) [2023-12-26 17:59:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 184516608. Throughput: 0: 9393.0, 1: 9794.4. Samples: 184521184. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-12-26 17:59:11,063][104569] Avg episode reward: [(0, '7591.354'), (1, '9173.254')] [2023-12-26 17:59:11,508][105620] Updated weights for policy 1, policy_version 360622 (0.0009) [2023-12-26 17:59:11,568][105620] Updated weights for policy 1, policy_version 360632 (0.0009) [2023-12-26 17:59:11,630][105620] Updated weights for policy 1, policy_version 360642 (0.0008) [2023-12-26 17:59:11,714][105692] Updated weights for policy 0, policy_version 360079 (0.0008) [2023-12-26 17:59:11,784][105692] Updated weights for policy 0, policy_version 360089 (0.0008) [2023-12-26 17:59:11,843][105692] Updated weights for policy 0, policy_version 360099 (0.0008) [2023-12-26 17:59:12,397][105620] Updated weights for policy 1, policy_version 360652 (0.0008) [2023-12-26 17:59:12,459][105620] Updated weights for policy 1, policy_version 360662 (0.0008) [2023-12-26 17:59:12,518][105692] Updated weights for policy 0, policy_version 360109 (0.0007) [2023-12-26 17:59:12,522][105620] Updated weights for policy 1, policy_version 360672 (0.0008) [2023-12-26 17:59:12,570][105692] Updated weights for policy 0, policy_version 360119 (0.0008) [2023-12-26 17:59:12,624][105692] Updated weights for policy 0, policy_version 360129 (0.0009) [2023-12-26 17:59:13,170][105620] Updated weights for policy 1, policy_version 360682 (0.0007) [2023-12-26 17:59:13,220][105620] Updated weights for policy 1, policy_version 360692 (0.0009) [2023-12-26 17:59:13,274][105620] Updated weights for policy 1, policy_version 360702 (0.0008) [2023-12-26 17:59:13,337][105620] Updated weights for policy 1, policy_version 360712 (0.0008) [2023-12-26 17:59:13,406][105692] Updated weights for policy 0, policy_version 360139 (0.0009) [2023-12-26 17:59:13,454][105692] Updated weights for policy 0, policy_version 360149 (0.0009) [2023-12-26 17:59:13,502][105692] Updated weights for policy 0, policy_version 360159 (0.0009) [2023-12-26 17:59:14,096][105620] Updated weights for policy 1, policy_version 360722 (0.0005) [2023-12-26 17:59:14,121][105586] KL-divergence is very high: 169.9741 [2023-12-26 17:59:14,155][105620] Updated weights for policy 1, policy_version 360732 (0.0006) [2023-12-26 17:59:14,160][105692] Updated weights for policy 0, policy_version 360169 (0.0008) [2023-12-26 17:59:14,165][105586] KL-divergence is very high: 261.5928 [2023-12-26 17:59:14,208][105586] KL-divergence is very high: 242.4374 [2023-12-26 17:59:14,208][105620] Updated weights for policy 1, policy_version 360742 (0.0005) [2023-12-26 17:59:14,225][105692] Updated weights for policy 0, policy_version 360179 (0.0005) [2023-12-26 17:59:14,273][105692] Updated weights for policy 0, policy_version 360189 (0.0005) [2023-12-26 17:59:14,327][105692] Updated weights for policy 0, policy_version 360199 (0.0006) [2023-12-26 17:59:14,906][105620] Updated weights for policy 1, policy_version 360752 (0.0007) [2023-12-26 17:59:14,974][105620] Updated weights for policy 1, policy_version 360762 (0.0009) [2023-12-26 17:59:15,032][105692] Updated weights for policy 0, policy_version 360209 (0.0007) [2023-12-26 17:59:15,038][105620] Updated weights for policy 1, policy_version 360772 (0.0008) [2023-12-26 17:59:15,092][105692] Updated weights for policy 0, policy_version 360219 (0.0008) [2023-12-26 17:59:15,147][105692] Updated weights for policy 0, policy_version 360229 (0.0008) [2023-12-26 17:59:15,738][105620] Updated weights for policy 1, policy_version 360782 (0.0006) [2023-12-26 17:59:15,806][105620] Updated weights for policy 1, policy_version 360792 (0.0006) [2023-12-26 17:59:15,813][105692] Updated weights for policy 0, policy_version 360239 (0.0008) [2023-12-26 17:59:15,862][105620] Updated weights for policy 1, policy_version 360802 (0.0007) [2023-12-26 17:59:15,876][105692] Updated weights for policy 0, policy_version 360249 (0.0009) [2023-12-26 17:59:15,937][105692] Updated weights for policy 0, policy_version 360259 (0.0008) [2023-12-26 17:59:16,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 184614912. Throughput: 0: 9396.0, 1: 9764.9. Samples: 184576600. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-12-26 17:59:16,062][104569] Avg episode reward: [(0, '8812.643'), (1, '8360.028')] [2023-12-26 17:59:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000360808_92372992.pth... [2023-12-26 17:59:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000360264_92241920.pth... [2023-12-26 17:59:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000359144_91955200.pth [2023-12-26 17:59:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000359688_92086272.pth [2023-12-26 17:59:16,613][105620] Updated weights for policy 1, policy_version 360812 (0.0007) [2023-12-26 17:59:16,633][105692] Updated weights for policy 0, policy_version 360269 (0.0010) [2023-12-26 17:59:16,638][105586] KL-divergence is very high: 794.6939 [2023-12-26 17:59:16,675][105620] Updated weights for policy 1, policy_version 360822 (0.0006) [2023-12-26 17:59:16,678][105692] Updated weights for policy 0, policy_version 360279 (0.0010) [2023-12-26 17:59:16,686][105586] KL-divergence is very high: 1516.9636 [2023-12-26 17:59:16,730][105692] Updated weights for policy 0, policy_version 360289 (0.0005) [2023-12-26 17:59:16,740][105620] Updated weights for policy 1, policy_version 360832 (0.0009) [2023-12-26 17:59:16,741][105586] KL-divergence is very high: 1717.1925 [2023-12-26 17:59:17,288][105692] Updated weights for policy 0, policy_version 360299 (0.0005) [2023-12-26 17:59:17,338][105692] Updated weights for policy 0, policy_version 360309 (0.0008) [2023-12-26 17:59:17,376][105620] Updated weights for policy 1, policy_version 360842 (0.0008) [2023-12-26 17:59:17,394][105692] Updated weights for policy 0, policy_version 360319 (0.0008) [2023-12-26 17:59:17,431][105620] Updated weights for policy 1, policy_version 360852 (0.0010) [2023-12-26 17:59:17,482][105620] Updated weights for policy 1, policy_version 360862 (0.0010) [2023-12-26 17:59:17,537][105620] Updated weights for policy 1, policy_version 360872 (0.0010) [2023-12-26 17:59:18,126][105692] Updated weights for policy 0, policy_version 360329 (0.0009) [2023-12-26 17:59:18,184][105692] Updated weights for policy 0, policy_version 360339 (0.0008) [2023-12-26 17:59:18,242][105692] Updated weights for policy 0, policy_version 360349 (0.0008) [2023-12-26 17:59:18,291][105692] Updated weights for policy 0, policy_version 360359 (0.0005) [2023-12-26 17:59:18,293][105620] Updated weights for policy 1, policy_version 360882 (0.0010) [2023-12-26 17:59:18,343][105620] Updated weights for policy 1, policy_version 360892 (0.0011) [2023-12-26 17:59:18,409][105620] Updated weights for policy 1, policy_version 360902 (0.0010) [2023-12-26 17:59:19,054][105692] Updated weights for policy 0, policy_version 360369 (0.0008) [2023-12-26 17:59:19,109][105692] Updated weights for policy 0, policy_version 360379 (0.0008) [2023-12-26 17:59:19,160][105692] Updated weights for policy 0, policy_version 360389 (0.0007) [2023-12-26 17:59:19,166][105620] Updated weights for policy 1, policy_version 360912 (0.0011) [2023-12-26 17:59:19,229][105620] Updated weights for policy 1, policy_version 360922 (0.0011) [2023-12-26 17:59:19,299][105620] Updated weights for policy 1, policy_version 360932 (0.0011) [2023-12-26 17:59:19,982][105620] Updated weights for policy 1, policy_version 360942 (0.0009) [2023-12-26 17:59:20,039][105620] Updated weights for policy 1, policy_version 360952 (0.0008) [2023-12-26 17:59:20,082][105692] Updated weights for policy 0, policy_version 360399 (0.0009) [2023-12-26 17:59:20,096][105620] Updated weights for policy 1, policy_version 360962 (0.0009) [2023-12-26 17:59:20,138][105692] Updated weights for policy 0, policy_version 360409 (0.0008) [2023-12-26 17:59:20,198][105692] Updated weights for policy 0, policy_version 360419 (0.0009) [2023-12-26 17:59:20,829][105620] Updated weights for policy 1, policy_version 360972 (0.0007) [2023-12-26 17:59:20,876][105620] Updated weights for policy 1, policy_version 360982 (0.0008) [2023-12-26 17:59:20,931][105620] Updated weights for policy 1, policy_version 360992 (0.0010) [2023-12-26 17:59:20,997][105692] Updated weights for policy 0, policy_version 360429 (0.0009) [2023-12-26 17:59:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 184705024. Throughput: 0: 9479.6, 1: 9822.9. Samples: 184695956. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-12-26 17:59:21,062][104569] Avg episode reward: [(0, '8812.036'), (1, '8182.062')] [2023-12-26 17:59:21,063][105692] Updated weights for policy 0, policy_version 360439 (0.0008) [2023-12-26 17:59:21,133][105692] Updated weights for policy 0, policy_version 360449 (0.0007) [2023-12-26 17:59:21,678][105620] Updated weights for policy 1, policy_version 361002 (0.0008) [2023-12-26 17:59:21,745][105620] Updated weights for policy 1, policy_version 361012 (0.0010) [2023-12-26 17:59:21,798][105586] KL-divergence is very high: 140.5164 [2023-12-26 17:59:21,805][105620] Updated weights for policy 1, policy_version 361022 (0.0011) [2023-12-26 17:59:21,849][105586] KL-divergence is very high: 145.5378 [2023-12-26 17:59:21,875][105620] Updated weights for policy 1, policy_version 361032 (0.0010) [2023-12-26 17:59:21,905][105692] Updated weights for policy 0, policy_version 360459 (0.0009) [2023-12-26 17:59:21,955][105692] Updated weights for policy 0, policy_version 360469 (0.0008) [2023-12-26 17:59:22,003][105692] Updated weights for policy 0, policy_version 360479 (0.0007) [2023-12-26 17:59:22,601][105620] Updated weights for policy 1, policy_version 361042 (0.0011) [2023-12-26 17:59:22,665][105620] Updated weights for policy 1, policy_version 361052 (0.0011) [2023-12-26 17:59:22,731][105620] Updated weights for policy 1, policy_version 361062 (0.0011) [2023-12-26 17:59:22,775][105692] Updated weights for policy 0, policy_version 360489 (0.0008) [2023-12-26 17:59:22,828][105692] Updated weights for policy 0, policy_version 360499 (0.0008) [2023-12-26 17:59:22,883][105692] Updated weights for policy 0, policy_version 360509 (0.0009) [2023-12-26 17:59:22,946][105692] Updated weights for policy 0, policy_version 360519 (0.0010) [2023-12-26 17:59:23,397][105620] Updated weights for policy 1, policy_version 361072 (0.0010) [2023-12-26 17:59:23,462][105620] Updated weights for policy 1, policy_version 361082 (0.0011) [2023-12-26 17:59:23,510][105620] Updated weights for policy 1, policy_version 361092 (0.0010) [2023-12-26 17:59:23,628][105692] Updated weights for policy 0, policy_version 360529 (0.0007) [2023-12-26 17:59:23,686][105692] Updated weights for policy 0, policy_version 360539 (0.0009) [2023-12-26 17:59:23,742][105692] Updated weights for policy 0, policy_version 360549 (0.0010) [2023-12-26 17:59:24,104][105620] Updated weights for policy 1, policy_version 361102 (0.0007) [2023-12-26 17:59:24,167][105620] Updated weights for policy 1, policy_version 361112 (0.0008) [2023-12-26 17:59:24,233][105620] Updated weights for policy 1, policy_version 361122 (0.0008) [2023-12-26 17:59:24,440][105692] Updated weights for policy 0, policy_version 360559 (0.0009) [2023-12-26 17:59:24,503][105692] Updated weights for policy 0, policy_version 360569 (0.0009) [2023-12-26 17:59:24,557][105692] Updated weights for policy 0, policy_version 360579 (0.0005) [2023-12-26 17:59:25,012][105620] Updated weights for policy 1, policy_version 361132 (0.0009) [2023-12-26 17:59:25,063][105620] Updated weights for policy 1, policy_version 361142 (0.0009) [2023-12-26 17:59:25,112][105620] Updated weights for policy 1, policy_version 361152 (0.0005) [2023-12-26 17:59:25,209][105692] Updated weights for policy 0, policy_version 360589 (0.0007) [2023-12-26 17:59:25,263][105692] Updated weights for policy 0, policy_version 360599 (0.0009) [2023-12-26 17:59:25,323][105692] Updated weights for policy 0, policy_version 360609 (0.0009) [2023-12-26 17:59:25,821][105620] Updated weights for policy 1, policy_version 361162 (0.0007) [2023-12-26 17:59:25,886][105620] Updated weights for policy 1, policy_version 361172 (0.0010) [2023-12-26 17:59:25,938][105620] Updated weights for policy 1, policy_version 361182 (0.0010) [2023-12-26 17:59:25,993][105620] Updated weights for policy 1, policy_version 361192 (0.0010) [2023-12-26 17:59:26,059][105692] Updated weights for policy 0, policy_version 360619 (0.0008) [2023-12-26 17:59:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.3, 300 sec: 19466.4). Total num frames: 184803328. Throughput: 0: 9529.9, 1: 9661.9. Samples: 184810976. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-12-26 17:59:26,062][104569] Avg episode reward: [(0, '9175.514'), (1, '8096.125')] [2023-12-26 17:59:26,116][105692] Updated weights for policy 0, policy_version 360629 (0.0008) [2023-12-26 17:59:26,166][105692] Updated weights for policy 0, policy_version 360639 (0.0008) [2023-12-26 17:59:26,758][105620] Updated weights for policy 1, policy_version 361202 (0.0010) [2023-12-26 17:59:26,806][105620] Updated weights for policy 1, policy_version 361212 (0.0010) [2023-12-26 17:59:26,867][105620] Updated weights for policy 1, policy_version 361222 (0.0010) [2023-12-26 17:59:26,929][105692] Updated weights for policy 0, policy_version 360649 (0.0008) [2023-12-26 17:59:26,978][105692] Updated weights for policy 0, policy_version 360659 (0.0010) [2023-12-26 17:59:27,035][105692] Updated weights for policy 0, policy_version 360669 (0.0010) [2023-12-26 17:59:27,082][105692] Updated weights for policy 0, policy_version 360679 (0.0010) [2023-12-26 17:59:27,482][105620] Updated weights for policy 1, policy_version 361232 (0.0009) [2023-12-26 17:59:27,541][105620] Updated weights for policy 1, policy_version 361242 (0.0009) [2023-12-26 17:59:27,592][105620] Updated weights for policy 1, policy_version 361252 (0.0009) [2023-12-26 17:59:27,774][105692] Updated weights for policy 0, policy_version 360689 (0.0009) [2023-12-26 17:59:27,821][105692] Updated weights for policy 0, policy_version 360699 (0.0009) [2023-12-26 17:59:27,876][105692] Updated weights for policy 0, policy_version 360709 (0.0009) [2023-12-26 17:59:28,282][105620] Updated weights for policy 1, policy_version 361262 (0.0008) [2023-12-26 17:59:28,347][105620] Updated weights for policy 1, policy_version 361272 (0.0007) [2023-12-26 17:59:28,404][105620] Updated weights for policy 1, policy_version 361282 (0.0008) [2023-12-26 17:59:28,677][105692] Updated weights for policy 0, policy_version 360719 (0.0007) [2023-12-26 17:59:28,732][105692] Updated weights for policy 0, policy_version 360729 (0.0006) [2023-12-26 17:59:28,786][105692] Updated weights for policy 0, policy_version 360739 (0.0010) [2023-12-26 17:59:29,085][105620] Updated weights for policy 1, policy_version 361292 (0.0008) [2023-12-26 17:59:29,132][105620] Updated weights for policy 1, policy_version 361302 (0.0005) [2023-12-26 17:59:29,178][105620] Updated weights for policy 1, policy_version 361312 (0.0005) [2023-12-26 17:59:29,569][105692] Updated weights for policy 0, policy_version 360749 (0.0009) [2023-12-26 17:59:29,628][105692] Updated weights for policy 0, policy_version 360759 (0.0009) [2023-12-26 17:59:29,683][105692] Updated weights for policy 0, policy_version 360769 (0.0009) [2023-12-26 17:59:29,845][105620] Updated weights for policy 1, policy_version 361322 (0.0006) [2023-12-26 17:59:29,899][105620] Updated weights for policy 1, policy_version 361332 (0.0009) [2023-12-26 17:59:29,961][105620] Updated weights for policy 1, policy_version 361342 (0.0009) [2023-12-26 17:59:30,022][105620] Updated weights for policy 1, policy_version 361352 (0.0006) [2023-12-26 17:59:30,492][105692] Updated weights for policy 0, policy_version 360779 (0.0010) [2023-12-26 17:59:30,548][105692] Updated weights for policy 0, policy_version 360789 (0.0009) [2023-12-26 17:59:30,600][105692] Updated weights for policy 0, policy_version 360799 (0.0009) [2023-12-26 17:59:30,708][105620] Updated weights for policy 1, policy_version 361362 (0.0006) [2023-12-26 17:59:30,757][105620] Updated weights for policy 1, policy_version 361372 (0.0006) [2023-12-26 17:59:30,825][105620] Updated weights for policy 1, policy_version 361382 (0.0006) [2023-12-26 17:59:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 184901632. Throughput: 0: 9502.8, 1: 9725.2. Samples: 184869760. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-12-26 17:59:31,063][104569] Avg episode reward: [(0, '8977.790'), (1, '8837.139')] [2023-12-26 17:59:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000360808_92381184.pth... [2023-12-26 17:59:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000361384_92520448.pth... [2023-12-26 17:59:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000359720_92102656.pth [2023-12-26 17:59:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000360232_92225536.pth [2023-12-26 17:59:31,455][105620] Updated weights for policy 1, policy_version 361392 (0.0006) [2023-12-26 17:59:31,482][105692] Updated weights for policy 0, policy_version 360809 (0.0009) [2023-12-26 17:59:31,513][105620] Updated weights for policy 1, policy_version 361402 (0.0008) [2023-12-26 17:59:31,527][105692] Updated weights for policy 0, policy_version 360819 (0.0005) [2023-12-26 17:59:31,573][105620] Updated weights for policy 1, policy_version 361412 (0.0008) [2023-12-26 17:59:31,573][105692] Updated weights for policy 0, policy_version 360829 (0.0009) [2023-12-26 17:59:31,632][105692] Updated weights for policy 0, policy_version 360839 (0.0010) [2023-12-26 17:59:32,316][105692] Updated weights for policy 0, policy_version 360849 (0.0007) [2023-12-26 17:59:32,325][105585] KL-divergence is very high: 100.2185 [2023-12-26 17:59:32,366][105620] Updated weights for policy 1, policy_version 361422 (0.0007) [2023-12-26 17:59:32,385][105585] KL-divergence is very high: 134.6950 [2023-12-26 17:59:32,386][105692] Updated weights for policy 0, policy_version 360859 (0.0007) [2023-12-26 17:59:32,397][105585] KL-divergence is very high: 121.2148 [2023-12-26 17:59:32,438][105620] Updated weights for policy 1, policy_version 361433 (0.0009) [2023-12-26 17:59:32,445][105692] Updated weights for policy 0, policy_version 360869 (0.0006) [2023-12-26 17:59:32,496][105620] Updated weights for policy 1, policy_version 361443 (0.0009) [2023-12-26 17:59:33,116][105620] Updated weights for policy 1, policy_version 361453 (0.0008) [2023-12-26 17:59:33,173][105620] Updated weights for policy 1, policy_version 361463 (0.0005) [2023-12-26 17:59:33,202][105692] Updated weights for policy 0, policy_version 360879 (0.0008) [2023-12-26 17:59:33,205][105585] KL-divergence is very high: 147.1122 [2023-12-26 17:59:33,229][105620] Updated weights for policy 1, policy_version 361473 (0.0005) [2023-12-26 17:59:33,251][105692] Updated weights for policy 0, policy_version 360889 (0.0008) [2023-12-26 17:59:33,306][105692] Updated weights for policy 0, policy_version 360900 (0.0009) [2023-12-26 17:59:33,789][105620] Updated weights for policy 1, policy_version 361483 (0.0007) [2023-12-26 17:59:33,850][105620] Updated weights for policy 1, policy_version 361493 (0.0010) [2023-12-26 17:59:33,911][105620] Updated weights for policy 1, policy_version 361503 (0.0010) [2023-12-26 17:59:34,148][105692] Updated weights for policy 0, policy_version 360911 (0.0010) [2023-12-26 17:59:34,216][105692] Updated weights for policy 0, policy_version 360921 (0.0009) [2023-12-26 17:59:34,282][105692] Updated weights for policy 0, policy_version 360931 (0.0008) [2023-12-26 17:59:34,627][105620] Updated weights for policy 1, policy_version 361513 (0.0010) [2023-12-26 17:59:34,688][105620] Updated weights for policy 1, policy_version 361523 (0.0011) [2023-12-26 17:59:34,741][105620] Updated weights for policy 1, policy_version 361533 (0.0010) [2023-12-26 17:59:34,802][105620] Updated weights for policy 1, policy_version 361543 (0.0011) [2023-12-26 17:59:34,991][105692] Updated weights for policy 0, policy_version 360941 (0.0009) [2023-12-26 17:59:35,056][105692] Updated weights for policy 0, policy_version 360951 (0.0011) [2023-12-26 17:59:35,118][105692] Updated weights for policy 0, policy_version 360961 (0.0011) [2023-12-26 17:59:35,566][105620] Updated weights for policy 1, policy_version 361553 (0.0009) [2023-12-26 17:59:35,617][105620] Updated weights for policy 1, policy_version 361563 (0.0009) [2023-12-26 17:59:35,677][105620] Updated weights for policy 1, policy_version 361574 (0.0010) [2023-12-26 17:59:35,807][105692] Updated weights for policy 0, policy_version 360971 (0.0011) [2023-12-26 17:59:35,859][105692] Updated weights for policy 0, policy_version 360981 (0.0010) [2023-12-26 17:59:35,916][105692] Updated weights for policy 0, policy_version 360991 (0.0010) [2023-12-26 17:59:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 184999936. Throughput: 0: 9514.6, 1: 9737.3. Samples: 184985428. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-12-26 17:59:36,062][104569] Avg episode reward: [(0, '1054.692'), (1, '8500.418')] [2023-12-26 17:59:36,337][105620] Updated weights for policy 1, policy_version 361584 (0.0007) [2023-12-26 17:59:36,390][105620] Updated weights for policy 1, policy_version 361594 (0.0005) [2023-12-26 17:59:36,448][105620] Updated weights for policy 1, policy_version 361604 (0.0005) [2023-12-26 17:59:36,674][105692] Updated weights for policy 0, policy_version 361001 (0.0010) [2023-12-26 17:59:36,727][105692] Updated weights for policy 0, policy_version 361011 (0.0011) [2023-12-26 17:59:36,783][105692] Updated weights for policy 0, policy_version 361021 (0.0007) [2023-12-26 17:59:36,837][105692] Updated weights for policy 0, policy_version 361031 (0.0009) [2023-12-26 17:59:37,072][105620] Updated weights for policy 1, policy_version 361614 (0.0007) [2023-12-26 17:59:37,124][105620] Updated weights for policy 1, policy_version 361624 (0.0008) [2023-12-26 17:59:37,176][105620] Updated weights for policy 1, policy_version 361634 (0.0008) [2023-12-26 17:59:37,583][105692] Updated weights for policy 0, policy_version 361041 (0.0010) [2023-12-26 17:59:37,635][105692] Updated weights for policy 0, policy_version 361051 (0.0010) [2023-12-26 17:59:37,694][105692] Updated weights for policy 0, policy_version 361061 (0.0010) [2023-12-26 17:59:37,937][105620] Updated weights for policy 1, policy_version 361644 (0.0008) [2023-12-26 17:59:37,986][105620] Updated weights for policy 1, policy_version 361654 (0.0008) [2023-12-26 17:59:38,040][105620] Updated weights for policy 1, policy_version 361664 (0.0008) [2023-12-26 17:59:38,444][105692] Updated weights for policy 0, policy_version 361071 (0.0010) [2023-12-26 17:59:38,506][105692] Updated weights for policy 0, policy_version 361081 (0.0011) [2023-12-26 17:59:38,568][105692] Updated weights for policy 0, policy_version 361091 (0.0010) [2023-12-26 17:59:38,804][105620] Updated weights for policy 1, policy_version 361674 (0.0008) [2023-12-26 17:59:38,863][105620] Updated weights for policy 1, policy_version 361684 (0.0008) [2023-12-26 17:59:38,921][105620] Updated weights for policy 1, policy_version 361694 (0.0008) [2023-12-26 17:59:38,984][105620] Updated weights for policy 1, policy_version 361704 (0.0007) [2023-12-26 17:59:39,308][105692] Updated weights for policy 0, policy_version 361101 (0.0010) [2023-12-26 17:59:39,380][105692] Updated weights for policy 0, policy_version 361111 (0.0012) [2023-12-26 17:59:39,449][105692] Updated weights for policy 0, policy_version 361121 (0.0010) [2023-12-26 17:59:39,631][105620] Updated weights for policy 1, policy_version 361714 (0.0008) [2023-12-26 17:59:39,685][105620] Updated weights for policy 1, policy_version 361724 (0.0008) [2023-12-26 17:59:39,740][105620] Updated weights for policy 1, policy_version 361734 (0.0008) [2023-12-26 17:59:40,211][105692] Updated weights for policy 0, policy_version 361131 (0.0010) [2023-12-26 17:59:40,279][105692] Updated weights for policy 0, policy_version 361141 (0.0010) [2023-12-26 17:59:40,338][105692] Updated weights for policy 0, policy_version 361151 (0.0008) [2023-12-26 17:59:40,523][105620] Updated weights for policy 1, policy_version 361744 (0.0010) [2023-12-26 17:59:40,575][105620] Updated weights for policy 1, policy_version 361754 (0.0009) [2023-12-26 17:59:40,627][105620] Updated weights for policy 1, policy_version 361764 (0.0009) [2023-12-26 17:59:40,932][105692] Updated weights for policy 0, policy_version 361161 (0.0008) [2023-12-26 17:59:40,991][105692] Updated weights for policy 0, policy_version 361171 (0.0010) [2023-12-26 17:59:41,054][105692] Updated weights for policy 0, policy_version 361181 (0.0010) [2023-12-26 17:59:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 185090048. Throughput: 0: 9467.6, 1: 9769.7. Samples: 185100020. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-12-26 17:59:41,063][104569] Avg episode reward: [(0, '1680.208'), (1, '8087.144')] [2023-12-26 17:59:41,124][105692] Updated weights for policy 0, policy_version 361191 (0.0011) [2023-12-26 17:59:41,487][105620] Updated weights for policy 1, policy_version 361774 (0.0007) [2023-12-26 17:59:41,550][105620] Updated weights for policy 1, policy_version 361784 (0.0006) [2023-12-26 17:59:41,611][105620] Updated weights for policy 1, policy_version 361794 (0.0008) [2023-12-26 17:59:41,829][105692] Updated weights for policy 0, policy_version 361201 (0.0006) [2023-12-26 17:59:41,889][105692] Updated weights for policy 0, policy_version 361211 (0.0005) [2023-12-26 17:59:41,951][105692] Updated weights for policy 0, policy_version 361221 (0.0005) [2023-12-26 17:59:42,278][105620] Updated weights for policy 1, policy_version 361804 (0.0007) [2023-12-26 17:59:42,342][105620] Updated weights for policy 1, policy_version 361814 (0.0009) [2023-12-26 17:59:42,412][105620] Updated weights for policy 1, policy_version 361824 (0.0009) [2023-12-26 17:59:42,537][105692] Updated weights for policy 0, policy_version 361231 (0.0008) [2023-12-26 17:59:42,590][105692] Updated weights for policy 0, policy_version 361241 (0.0009) [2023-12-26 17:59:42,644][105692] Updated weights for policy 0, policy_version 361251 (0.0009) [2023-12-26 17:59:43,138][105620] Updated weights for policy 1, policy_version 361834 (0.0008) [2023-12-26 17:59:43,185][105620] Updated weights for policy 1, policy_version 361844 (0.0009) [2023-12-26 17:59:43,243][105620] Updated weights for policy 1, policy_version 361854 (0.0009) [2023-12-26 17:59:43,297][105620] Updated weights for policy 1, policy_version 361864 (0.0009) [2023-12-26 17:59:43,425][105692] Updated weights for policy 0, policy_version 361261 (0.0009) [2023-12-26 17:59:43,481][105692] Updated weights for policy 0, policy_version 361271 (0.0009) [2023-12-26 17:59:43,543][105692] Updated weights for policy 0, policy_version 361281 (0.0008) [2023-12-26 17:59:44,056][105620] Updated weights for policy 1, policy_version 361874 (0.0010) [2023-12-26 17:59:44,109][105620] Updated weights for policy 1, policy_version 361884 (0.0006) [2023-12-26 17:59:44,170][105620] Updated weights for policy 1, policy_version 361894 (0.0005) [2023-12-26 17:59:44,289][105692] Updated weights for policy 0, policy_version 361291 (0.0008) [2023-12-26 17:59:44,344][105692] Updated weights for policy 0, policy_version 361301 (0.0008) [2023-12-26 17:59:44,389][105692] Updated weights for policy 0, policy_version 361311 (0.0008) [2023-12-26 17:59:44,857][105620] Updated weights for policy 1, policy_version 361904 (0.0010) [2023-12-26 17:59:44,924][105620] Updated weights for policy 1, policy_version 361914 (0.0011) [2023-12-26 17:59:44,982][105620] Updated weights for policy 1, policy_version 361924 (0.0010) [2023-12-26 17:59:45,166][105692] Updated weights for policy 0, policy_version 361321 (0.0008) [2023-12-26 17:59:45,215][105692] Updated weights for policy 0, policy_version 361331 (0.0008) [2023-12-26 17:59:45,275][105692] Updated weights for policy 0, policy_version 361341 (0.0008) [2023-12-26 17:59:45,339][105692] Updated weights for policy 0, policy_version 361351 (0.0008) [2023-12-26 17:59:45,738][105620] Updated weights for policy 1, policy_version 361934 (0.0011) [2023-12-26 17:59:45,797][105620] Updated weights for policy 1, policy_version 361944 (0.0011) [2023-12-26 17:59:45,860][105620] Updated weights for policy 1, policy_version 361954 (0.0011) [2023-12-26 17:59:45,871][105586] KL-divergence is very high: 148.1615 [2023-12-26 17:59:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 185188352. Throughput: 0: 9435.9, 1: 9802.2. Samples: 185159280. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-12-26 17:59:46,063][104569] Avg episode reward: [(0, '6350.099'), (1, '8430.335')] [2023-12-26 17:59:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000361960_92667904.pth... [2023-12-26 17:59:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000360808_92372992.pth [2023-12-26 17:59:46,123][105692] Updated weights for policy 0, policy_version 361361 (0.0008) [2023-12-26 17:59:46,184][105692] Updated weights for policy 0, policy_version 361371 (0.0008) [2023-12-26 17:59:46,232][105692] Updated weights for policy 0, policy_version 361381 (0.0008) [2023-12-26 17:59:46,247][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000361384_92528640.pth... [2023-12-26 17:59:46,251][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000360264_92241920.pth [2023-12-26 17:59:46,603][105620] Updated weights for policy 1, policy_version 361964 (0.0010) [2023-12-26 17:59:46,671][105620] Updated weights for policy 1, policy_version 361974 (0.0010) [2023-12-26 17:59:46,743][105620] Updated weights for policy 1, policy_version 361984 (0.0009) [2023-12-26 17:59:46,963][105692] Updated weights for policy 0, policy_version 361391 (0.0009) [2023-12-26 17:59:47,030][105692] Updated weights for policy 0, policy_version 361401 (0.0011) [2023-12-26 17:59:47,078][105692] Updated weights for policy 0, policy_version 361411 (0.0010) [2023-12-26 17:59:47,448][105620] Updated weights for policy 1, policy_version 361994 (0.0010) [2023-12-26 17:59:47,493][105620] Updated weights for policy 1, policy_version 362004 (0.0008) [2023-12-26 17:59:47,543][105620] Updated weights for policy 1, policy_version 362014 (0.0007) [2023-12-26 17:59:47,595][105620] Updated weights for policy 1, policy_version 362024 (0.0008) [2023-12-26 17:59:47,707][105692] Updated weights for policy 0, policy_version 361421 (0.0010) [2023-12-26 17:59:47,758][105692] Updated weights for policy 0, policy_version 361431 (0.0011) [2023-12-26 17:59:47,806][105692] Updated weights for policy 0, policy_version 361441 (0.0010) [2023-12-26 17:59:48,344][105620] Updated weights for policy 1, policy_version 362034 (0.0008) [2023-12-26 17:59:48,407][105620] Updated weights for policy 1, policy_version 362044 (0.0008) [2023-12-26 17:59:48,469][105620] Updated weights for policy 1, policy_version 362054 (0.0008) [2023-12-26 17:59:48,582][105692] Updated weights for policy 0, policy_version 361451 (0.0011) [2023-12-26 17:59:48,640][105692] Updated weights for policy 0, policy_version 361461 (0.0010) [2023-12-26 17:59:48,697][105692] Updated weights for policy 0, policy_version 361471 (0.0007) [2023-12-26 17:59:49,249][105692] Updated weights for policy 0, policy_version 361481 (0.0006) [2023-12-26 17:59:49,250][105620] Updated weights for policy 1, policy_version 362064 (0.0010) [2023-12-26 17:59:49,298][105692] Updated weights for policy 0, policy_version 361491 (0.0011) [2023-12-26 17:59:49,305][105620] Updated weights for policy 1, policy_version 362074 (0.0011) [2023-12-26 17:59:49,365][105692] Updated weights for policy 0, policy_version 361501 (0.0011) [2023-12-26 17:59:49,368][105620] Updated weights for policy 1, policy_version 362084 (0.0010) [2023-12-26 17:59:49,434][105692] Updated weights for policy 0, policy_version 361511 (0.0011) [2023-12-26 17:59:50,120][105620] Updated weights for policy 1, policy_version 362094 (0.0009) [2023-12-26 17:59:50,179][105620] Updated weights for policy 1, policy_version 362104 (0.0011) [2023-12-26 17:59:50,183][105692] Updated weights for policy 0, policy_version 361521 (0.0011) [2023-12-26 17:59:50,239][105620] Updated weights for policy 1, policy_version 362114 (0.0010) [2023-12-26 17:59:50,245][105692] Updated weights for policy 0, policy_version 361531 (0.0009) [2023-12-26 17:59:50,298][105692] Updated weights for policy 0, policy_version 361541 (0.0007) [2023-12-26 17:59:50,983][105620] Updated weights for policy 1, policy_version 362124 (0.0011) [2023-12-26 17:59:51,043][105620] Updated weights for policy 1, policy_version 362134 (0.0011) [2023-12-26 17:59:51,055][105692] Updated weights for policy 0, policy_version 361551 (0.0008) [2023-12-26 17:59:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.6, 300 sec: 19410.9). Total num frames: 185278464. Throughput: 0: 9509.5, 1: 9748.6. Samples: 185274676. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-12-26 17:59:51,062][104569] Avg episode reward: [(0, '9185.968'), (1, '8363.583')] [2023-12-26 17:59:51,110][105620] Updated weights for policy 1, policy_version 362144 (0.0011) [2023-12-26 17:59:51,114][105692] Updated weights for policy 0, policy_version 361561 (0.0008) [2023-12-26 17:59:51,180][105692] Updated weights for policy 0, policy_version 361571 (0.0008) [2023-12-26 17:59:51,879][105620] Updated weights for policy 1, policy_version 362154 (0.0011) [2023-12-26 17:59:51,918][105692] Updated weights for policy 0, policy_version 361581 (0.0007) [2023-12-26 17:59:51,939][105620] Updated weights for policy 1, policy_version 362164 (0.0011) [2023-12-26 17:59:51,968][105692] Updated weights for policy 0, policy_version 361591 (0.0007) [2023-12-26 17:59:51,997][105620] Updated weights for policy 1, policy_version 362174 (0.0011) [2023-12-26 17:59:52,027][105692] Updated weights for policy 0, policy_version 361601 (0.0006) [2023-12-26 17:59:52,053][105620] Updated weights for policy 1, policy_version 362184 (0.0010) [2023-12-26 17:59:52,765][105692] Updated weights for policy 0, policy_version 361611 (0.0005) [2023-12-26 17:59:52,772][105620] Updated weights for policy 1, policy_version 362194 (0.0008) [2023-12-26 17:59:52,823][105692] Updated weights for policy 0, policy_version 361621 (0.0006) [2023-12-26 17:59:52,839][105620] Updated weights for policy 1, policy_version 362204 (0.0007) [2023-12-26 17:59:52,873][105692] Updated weights for policy 0, policy_version 361631 (0.0005) [2023-12-26 17:59:52,891][105620] Updated weights for policy 1, policy_version 362214 (0.0008) [2023-12-26 17:59:53,541][105620] Updated weights for policy 1, policy_version 362224 (0.0005) [2023-12-26 17:59:53,603][105620] Updated weights for policy 1, policy_version 362234 (0.0007) [2023-12-26 17:59:53,640][105692] Updated weights for policy 0, policy_version 361641 (0.0008) [2023-12-26 17:59:53,653][105620] Updated weights for policy 1, policy_version 362244 (0.0009) [2023-12-26 17:59:53,689][105692] Updated weights for policy 0, policy_version 361651 (0.0007) [2023-12-26 17:59:53,744][105692] Updated weights for policy 0, policy_version 361661 (0.0009) [2023-12-26 17:59:53,791][105692] Updated weights for policy 0, policy_version 361671 (0.0009) [2023-12-26 17:59:54,235][105620] Updated weights for policy 1, policy_version 362254 (0.0007) [2023-12-26 17:59:54,280][105620] Updated weights for policy 1, policy_version 362264 (0.0008) [2023-12-26 17:59:54,327][105620] Updated weights for policy 1, policy_version 362274 (0.0009) [2023-12-26 17:59:54,602][105692] Updated weights for policy 0, policy_version 361681 (0.0009) [2023-12-26 17:59:54,653][105692] Updated weights for policy 0, policy_version 361691 (0.0009) [2023-12-26 17:59:54,702][105692] Updated weights for policy 0, policy_version 361701 (0.0009) [2023-12-26 17:59:55,089][105620] Updated weights for policy 1, policy_version 362284 (0.0009) [2023-12-26 17:59:55,151][105620] Updated weights for policy 1, policy_version 362294 (0.0010) [2023-12-26 17:59:55,214][105620] Updated weights for policy 1, policy_version 362304 (0.0010) [2023-12-26 17:59:55,487][105692] Updated weights for policy 0, policy_version 361711 (0.0010) [2023-12-26 17:59:55,545][105692] Updated weights for policy 0, policy_version 361721 (0.0010) [2023-12-26 17:59:55,601][105692] Updated weights for policy 0, policy_version 361731 (0.0010) [2023-12-26 17:59:55,771][105620] Updated weights for policy 1, policy_version 362314 (0.0010) [2023-12-26 17:59:55,818][105620] Updated weights for policy 1, policy_version 362324 (0.0008) [2023-12-26 17:59:55,871][105620] Updated weights for policy 1, policy_version 362334 (0.0010) [2023-12-26 17:59:55,924][105620] Updated weights for policy 1, policy_version 362344 (0.0009) [2023-12-26 17:59:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 185384960. Throughput: 0: 9542.4, 1: 9771.3. Samples: 185390300. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-12-26 17:59:56,062][104569] Avg episode reward: [(0, '9266.969'), (1, '8841.996')] [2023-12-26 17:59:56,288][105692] Updated weights for policy 0, policy_version 361741 (0.0010) [2023-12-26 17:59:56,343][105692] Updated weights for policy 0, policy_version 361751 (0.0010) [2023-12-26 17:59:56,406][105692] Updated weights for policy 0, policy_version 361761 (0.0011) [2023-12-26 17:59:56,645][105620] Updated weights for policy 1, policy_version 362354 (0.0005) [2023-12-26 17:59:56,693][105620] Updated weights for policy 1, policy_version 362364 (0.0005) [2023-12-26 17:59:56,741][105620] Updated weights for policy 1, policy_version 362374 (0.0005) [2023-12-26 17:59:57,113][105692] Updated weights for policy 0, policy_version 361771 (0.0010) [2023-12-26 17:59:57,170][105692] Updated weights for policy 0, policy_version 361781 (0.0010) [2023-12-26 17:59:57,231][105692] Updated weights for policy 0, policy_version 361791 (0.0010) [2023-12-26 17:59:57,435][105620] Updated weights for policy 1, policy_version 362384 (0.0009) [2023-12-26 17:59:57,484][105620] Updated weights for policy 1, policy_version 362394 (0.0009) [2023-12-26 17:59:57,537][105620] Updated weights for policy 1, policy_version 362404 (0.0008) [2023-12-26 17:59:57,942][105692] Updated weights for policy 0, policy_version 361801 (0.0010) [2023-12-26 17:59:57,999][105692] Updated weights for policy 0, policy_version 361811 (0.0008) [2023-12-26 17:59:58,063][105692] Updated weights for policy 0, policy_version 361821 (0.0005) [2023-12-26 17:59:58,127][105692] Updated weights for policy 0, policy_version 361831 (0.0008) [2023-12-26 17:59:58,231][105620] Updated weights for policy 1, policy_version 362414 (0.0009) [2023-12-26 17:59:58,297][105620] Updated weights for policy 1, policy_version 362424 (0.0008) [2023-12-26 17:59:58,360][105620] Updated weights for policy 1, policy_version 362434 (0.0008) [2023-12-26 17:59:58,909][105692] Updated weights for policy 0, policy_version 361841 (0.0010) [2023-12-26 17:59:58,969][105692] Updated weights for policy 0, policy_version 361851 (0.0011) [2023-12-26 17:59:59,029][105692] Updated weights for policy 0, policy_version 361861 (0.0011) [2023-12-26 17:59:59,128][105620] Updated weights for policy 1, policy_version 362444 (0.0011) [2023-12-26 17:59:59,186][105620] Updated weights for policy 1, policy_version 362454 (0.0010) [2023-12-26 17:59:59,261][105620] Updated weights for policy 1, policy_version 362464 (0.0009) [2023-12-26 17:59:59,765][105692] Updated weights for policy 0, policy_version 361871 (0.0007) [2023-12-26 17:59:59,833][105692] Updated weights for policy 0, policy_version 361881 (0.0006) [2023-12-26 17:59:59,887][105692] Updated weights for policy 0, policy_version 361891 (0.0006) [2023-12-26 17:59:59,961][105620] Updated weights for policy 1, policy_version 362474 (0.0010) [2023-12-26 18:00:00,025][105620] Updated weights for policy 1, policy_version 362484 (0.0011) [2023-12-26 18:00:00,094][105620] Updated weights for policy 1, policy_version 362494 (0.0011) [2023-12-26 18:00:00,159][105620] Updated weights for policy 1, policy_version 362504 (0.0009) [2023-12-26 18:00:00,437][105692] Updated weights for policy 0, policy_version 361901 (0.0007) [2023-12-26 18:00:00,485][105692] Updated weights for policy 0, policy_version 361911 (0.0009) [2023-12-26 18:00:00,536][105692] Updated weights for policy 0, policy_version 361921 (0.0009) [2023-12-26 18:00:00,943][105620] Updated weights for policy 1, policy_version 362514 (0.0009) [2023-12-26 18:00:01,001][105620] Updated weights for policy 1, policy_version 362524 (0.0009) [2023-12-26 18:00:01,058][105620] Updated weights for policy 1, policy_version 362534 (0.0008) [2023-12-26 18:00:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19114.6, 300 sec: 19383.1). Total num frames: 185475072. Throughput: 0: 9581.8, 1: 9814.6. Samples: 185449436. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-12-26 18:00:01,063][104569] Avg episode reward: [(0, '9266.873'), (1, '9174.004')] [2023-12-26 18:00:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000361928_92667904.pth... [2023-12-26 18:00:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000362536_92815360.pth... [2023-12-26 18:00:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000360808_92381184.pth [2023-12-26 18:00:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000361384_92520448.pth [2023-12-26 18:00:01,354][105692] Updated weights for policy 0, policy_version 361931 (0.0008) [2023-12-26 18:00:01,422][105692] Updated weights for policy 0, policy_version 361941 (0.0008) [2023-12-26 18:00:01,487][105692] Updated weights for policy 0, policy_version 361951 (0.0009) [2023-12-26 18:00:01,810][105620] Updated weights for policy 1, policy_version 362544 (0.0008) [2023-12-26 18:00:01,864][105620] Updated weights for policy 1, policy_version 362554 (0.0009) [2023-12-26 18:00:01,916][105620] Updated weights for policy 1, policy_version 362564 (0.0005) [2023-12-26 18:00:02,231][105692] Updated weights for policy 0, policy_version 361961 (0.0008) [2023-12-26 18:00:02,293][105692] Updated weights for policy 0, policy_version 361971 (0.0009) [2023-12-26 18:00:02,349][105692] Updated weights for policy 0, policy_version 361981 (0.0009) [2023-12-26 18:00:02,411][105692] Updated weights for policy 0, policy_version 361991 (0.0009) [2023-12-26 18:00:02,571][105620] Updated weights for policy 1, policy_version 362574 (0.0005) [2023-12-26 18:00:02,623][105620] Updated weights for policy 1, policy_version 362584 (0.0008) [2023-12-26 18:00:02,675][105620] Updated weights for policy 1, policy_version 362594 (0.0008) [2023-12-26 18:00:03,172][105692] Updated weights for policy 0, policy_version 362001 (0.0010) [2023-12-26 18:00:03,229][105692] Updated weights for policy 0, policy_version 362011 (0.0010) [2023-12-26 18:00:03,287][105692] Updated weights for policy 0, policy_version 362021 (0.0010) [2023-12-26 18:00:03,393][105620] Updated weights for policy 1, policy_version 362604 (0.0009) [2023-12-26 18:00:03,460][105620] Updated weights for policy 1, policy_version 362614 (0.0010) [2023-12-26 18:00:03,524][105620] Updated weights for policy 1, policy_version 362624 (0.0010) [2023-12-26 18:00:03,876][105692] Updated weights for policy 0, policy_version 362031 (0.0009) [2023-12-26 18:00:03,923][105692] Updated weights for policy 0, policy_version 362041 (0.0005) [2023-12-26 18:00:03,989][105692] Updated weights for policy 0, policy_version 362051 (0.0005) [2023-12-26 18:00:04,197][105620] Updated weights for policy 1, policy_version 362634 (0.0010) [2023-12-26 18:00:04,265][105620] Updated weights for policy 1, policy_version 362644 (0.0010) [2023-12-26 18:00:04,330][105620] Updated weights for policy 1, policy_version 362654 (0.0010) [2023-12-26 18:00:04,383][105620] Updated weights for policy 1, policy_version 362664 (0.0009) [2023-12-26 18:00:04,671][105692] Updated weights for policy 0, policy_version 362061 (0.0008) [2023-12-26 18:00:04,733][105692] Updated weights for policy 0, policy_version 362071 (0.0011) [2023-12-26 18:00:04,795][105692] Updated weights for policy 0, policy_version 362081 (0.0010) [2023-12-26 18:00:05,065][105620] Updated weights for policy 1, policy_version 362674 (0.0011) [2023-12-26 18:00:05,121][105620] Updated weights for policy 1, policy_version 362684 (0.0010) [2023-12-26 18:00:05,178][105620] Updated weights for policy 1, policy_version 362694 (0.0011) [2023-12-26 18:00:05,457][105692] Updated weights for policy 0, policy_version 362091 (0.0011) [2023-12-26 18:00:05,506][105692] Updated weights for policy 0, policy_version 362101 (0.0011) [2023-12-26 18:00:05,558][105692] Updated weights for policy 0, policy_version 362111 (0.0011) [2023-12-26 18:00:05,817][105620] Updated weights for policy 1, policy_version 362704 (0.0006) [2023-12-26 18:00:05,866][105620] Updated weights for policy 1, policy_version 362714 (0.0005) [2023-12-26 18:00:05,925][105620] Updated weights for policy 1, policy_version 362724 (0.0007) [2023-12-26 18:00:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 185581568. Throughput: 0: 9548.3, 1: 9801.9. Samples: 185566716. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-12-26 18:00:06,063][104569] Avg episode reward: [(0, '9357.660'), (1, '9266.145')] [2023-12-26 18:00:06,310][105692] Updated weights for policy 0, policy_version 362121 (0.0011) [2023-12-26 18:00:06,370][105692] Updated weights for policy 0, policy_version 362131 (0.0011) [2023-12-26 18:00:06,434][105692] Updated weights for policy 0, policy_version 362141 (0.0011) [2023-12-26 18:00:06,495][105692] Updated weights for policy 0, policy_version 362151 (0.0011) [2023-12-26 18:00:06,669][105620] Updated weights for policy 1, policy_version 362734 (0.0011) [2023-12-26 18:00:06,729][105620] Updated weights for policy 1, policy_version 362744 (0.0011) [2023-12-26 18:00:06,790][105620] Updated weights for policy 1, policy_version 362754 (0.0011) [2023-12-26 18:00:07,218][105692] Updated weights for policy 0, policy_version 362161 (0.0011) [2023-12-26 18:00:07,280][105692] Updated weights for policy 0, policy_version 362171 (0.0011) [2023-12-26 18:00:07,338][105692] Updated weights for policy 0, policy_version 362181 (0.0010) [2023-12-26 18:00:07,409][105620] Updated weights for policy 1, policy_version 362764 (0.0008) [2023-12-26 18:00:07,458][105620] Updated weights for policy 1, policy_version 362774 (0.0010) [2023-12-26 18:00:07,507][105620] Updated weights for policy 1, policy_version 362784 (0.0010) [2023-12-26 18:00:08,089][105692] Updated weights for policy 0, policy_version 362191 (0.0011) [2023-12-26 18:00:08,148][105692] Updated weights for policy 0, policy_version 362201 (0.0010) [2023-12-26 18:00:08,160][105620] Updated weights for policy 1, policy_version 362794 (0.0010) [2023-12-26 18:00:08,203][105692] Updated weights for policy 0, policy_version 362211 (0.0010) [2023-12-26 18:00:08,220][105620] Updated weights for policy 1, policy_version 362804 (0.0010) [2023-12-26 18:00:08,281][105620] Updated weights for policy 1, policy_version 362814 (0.0010) [2023-12-26 18:00:08,345][105620] Updated weights for policy 1, policy_version 362824 (0.0011) [2023-12-26 18:00:08,949][105692] Updated weights for policy 0, policy_version 362221 (0.0010) [2023-12-26 18:00:08,977][105620] Updated weights for policy 1, policy_version 362834 (0.0011) [2023-12-26 18:00:09,005][105692] Updated weights for policy 0, policy_version 362231 (0.0011) [2023-12-26 18:00:09,036][105620] Updated weights for policy 1, policy_version 362844 (0.0010) [2023-12-26 18:00:09,054][105692] Updated weights for policy 0, policy_version 362241 (0.0011) [2023-12-26 18:00:09,096][105620] Updated weights for policy 1, policy_version 362854 (0.0011) [2023-12-26 18:00:09,769][105692] Updated weights for policy 0, policy_version 362251 (0.0011) [2023-12-26 18:00:09,831][105692] Updated weights for policy 0, policy_version 362261 (0.0010) [2023-12-26 18:00:09,888][105620] Updated weights for policy 1, policy_version 362864 (0.0011) [2023-12-26 18:00:09,891][105692] Updated weights for policy 0, policy_version 362271 (0.0009) [2023-12-26 18:00:09,952][105620] Updated weights for policy 1, policy_version 362874 (0.0009) [2023-12-26 18:00:10,015][105620] Updated weights for policy 1, policy_version 362884 (0.0011) [2023-12-26 18:00:10,651][105692] Updated weights for policy 0, policy_version 362281 (0.0008) [2023-12-26 18:00:10,704][105692] Updated weights for policy 0, policy_version 362291 (0.0008) [2023-12-26 18:00:10,754][105620] Updated weights for policy 1, policy_version 362894 (0.0008) [2023-12-26 18:00:10,763][105692] Updated weights for policy 0, policy_version 362301 (0.0008) [2023-12-26 18:00:10,817][105620] Updated weights for policy 1, policy_version 362904 (0.0009) [2023-12-26 18:00:10,824][105692] Updated weights for policy 0, policy_version 362311 (0.0008) [2023-12-26 18:00:10,880][105620] Updated weights for policy 1, policy_version 362914 (0.0010) [2023-12-26 18:00:11,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 185679872. Throughput: 0: 9568.4, 1: 9843.6. Samples: 185684516. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-12-26 18:00:11,062][104569] Avg episode reward: [(0, '9357.831'), (1, '9266.122')] [2023-12-26 18:00:11,525][105620] Updated weights for policy 1, policy_version 362924 (0.0007) [2023-12-26 18:00:11,578][105620] Updated weights for policy 1, policy_version 362934 (0.0005) [2023-12-26 18:00:11,637][105620] Updated weights for policy 1, policy_version 362944 (0.0008) [2023-12-26 18:00:11,735][105692] Updated weights for policy 0, policy_version 362321 (0.0008) [2023-12-26 18:00:11,794][105692] Updated weights for policy 0, policy_version 362331 (0.0008) [2023-12-26 18:00:11,859][105692] Updated weights for policy 0, policy_version 362341 (0.0007) [2023-12-26 18:00:12,391][105620] Updated weights for policy 1, policy_version 362954 (0.0010) [2023-12-26 18:00:12,448][105620] Updated weights for policy 1, policy_version 362964 (0.0005) [2023-12-26 18:00:12,510][105620] Updated weights for policy 1, policy_version 362974 (0.0006) [2023-12-26 18:00:12,575][105620] Updated weights for policy 1, policy_version 362984 (0.0005) [2023-12-26 18:00:12,670][105692] Updated weights for policy 0, policy_version 362351 (0.0008) [2023-12-26 18:00:12,723][105692] Updated weights for policy 0, policy_version 362361 (0.0009) [2023-12-26 18:00:12,775][105692] Updated weights for policy 0, policy_version 362371 (0.0010) [2023-12-26 18:00:13,110][105620] Updated weights for policy 1, policy_version 362994 (0.0006) [2023-12-26 18:00:13,162][105620] Updated weights for policy 1, policy_version 363004 (0.0005) [2023-12-26 18:00:13,214][105620] Updated weights for policy 1, policy_version 363014 (0.0007) [2023-12-26 18:00:13,643][105692] Updated weights for policy 0, policy_version 362381 (0.0010) [2023-12-26 18:00:13,695][105692] Updated weights for policy 0, policy_version 362391 (0.0007) [2023-12-26 18:00:13,745][105692] Updated weights for policy 0, policy_version 362401 (0.0007) [2023-12-26 18:00:13,852][105620] Updated weights for policy 1, policy_version 363024 (0.0009) [2023-12-26 18:00:13,919][105620] Updated weights for policy 1, policy_version 363034 (0.0010) [2023-12-26 18:00:13,967][105620] Updated weights for policy 1, policy_version 363044 (0.0010) [2023-12-26 18:00:14,447][105692] Updated weights for policy 0, policy_version 362411 (0.0008) [2023-12-26 18:00:14,496][105692] Updated weights for policy 0, policy_version 362421 (0.0008) [2023-12-26 18:00:14,544][105692] Updated weights for policy 0, policy_version 362431 (0.0008) [2023-12-26 18:00:14,708][105620] Updated weights for policy 1, policy_version 363054 (0.0010) [2023-12-26 18:00:14,764][105620] Updated weights for policy 1, policy_version 363064 (0.0010) [2023-12-26 18:00:14,830][105620] Updated weights for policy 1, policy_version 363074 (0.0010) [2023-12-26 18:00:15,403][105620] Updated weights for policy 1, policy_version 363084 (0.0006) [2023-12-26 18:00:15,466][105620] Updated weights for policy 1, policy_version 363094 (0.0010) [2023-12-26 18:00:15,473][105692] Updated weights for policy 0, policy_version 362441 (0.0008) [2023-12-26 18:00:15,525][105620] Updated weights for policy 1, policy_version 363104 (0.0008) [2023-12-26 18:00:15,529][105692] Updated weights for policy 0, policy_version 362451 (0.0009) [2023-12-26 18:00:15,581][105692] Updated weights for policy 0, policy_version 362461 (0.0008) [2023-12-26 18:00:15,635][105692] Updated weights for policy 0, policy_version 362472 (0.0010) [2023-12-26 18:00:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 185769984. Throughput: 0: 9496.0, 1: 9879.6. Samples: 185741660. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-12-26 18:00:16,062][104569] Avg episode reward: [(0, '9358.004'), (1, '9172.626')] [2023-12-26 18:00:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000362472_92807168.pth... [2023-12-26 18:00:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000361384_92528640.pth [2023-12-26 18:00:16,080][105620] Updated weights for policy 1, policy_version 363114 (0.0006) [2023-12-26 18:00:16,141][105620] Updated weights for policy 1, policy_version 363124 (0.0005) [2023-12-26 18:00:16,195][105620] Updated weights for policy 1, policy_version 363134 (0.0005) [2023-12-26 18:00:16,248][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000363144_92971008.pth... [2023-12-26 18:00:16,250][105620] Updated weights for policy 1, policy_version 363144 (0.0006) [2023-12-26 18:00:16,252][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000361960_92667904.pth [2023-12-26 18:00:16,504][105692] Updated weights for policy 0, policy_version 362482 (0.0010) [2023-12-26 18:00:16,560][105692] Updated weights for policy 0, policy_version 362492 (0.0009) [2023-12-26 18:00:16,611][105692] Updated weights for policy 0, policy_version 362502 (0.0008) [2023-12-26 18:00:16,878][105620] Updated weights for policy 1, policy_version 363154 (0.0006) [2023-12-26 18:00:16,944][105620] Updated weights for policy 1, policy_version 363164 (0.0005) [2023-12-26 18:00:17,009][105620] Updated weights for policy 1, policy_version 363174 (0.0005) [2023-12-26 18:00:17,321][105692] Updated weights for policy 0, policy_version 362512 (0.0009) [2023-12-26 18:00:17,367][105692] Updated weights for policy 0, policy_version 362522 (0.0008) [2023-12-26 18:00:17,421][105692] Updated weights for policy 0, policy_version 362532 (0.0009) [2023-12-26 18:00:17,559][105620] Updated weights for policy 1, policy_version 363184 (0.0005) [2023-12-26 18:00:17,617][105620] Updated weights for policy 1, policy_version 363194 (0.0007) [2023-12-26 18:00:17,675][105620] Updated weights for policy 1, policy_version 363204 (0.0009) [2023-12-26 18:00:18,153][105692] Updated weights for policy 0, policy_version 362542 (0.0008) [2023-12-26 18:00:18,210][105692] Updated weights for policy 0, policy_version 362552 (0.0005) [2023-12-26 18:00:18,270][105692] Updated weights for policy 0, policy_version 362562 (0.0006) [2023-12-26 18:00:18,429][105620] Updated weights for policy 1, policy_version 363214 (0.0009) [2023-12-26 18:00:18,489][105620] Updated weights for policy 1, policy_version 363224 (0.0009) [2023-12-26 18:00:18,550][105620] Updated weights for policy 1, policy_version 363234 (0.0009) [2023-12-26 18:00:19,004][105692] Updated weights for policy 0, policy_version 362572 (0.0009) [2023-12-26 18:00:19,056][105692] Updated weights for policy 0, policy_version 362582 (0.0008) [2023-12-26 18:00:19,103][105692] Updated weights for policy 0, policy_version 362592 (0.0008) [2023-12-26 18:00:19,244][105620] Updated weights for policy 1, policy_version 363244 (0.0008) [2023-12-26 18:00:19,311][105620] Updated weights for policy 1, policy_version 363254 (0.0006) [2023-12-26 18:00:19,379][105620] Updated weights for policy 1, policy_version 363264 (0.0007) [2023-12-26 18:00:19,724][105692] Updated weights for policy 0, policy_version 362602 (0.0005) [2023-12-26 18:00:19,773][105692] Updated weights for policy 0, policy_version 362612 (0.0006) [2023-12-26 18:00:19,825][105692] Updated weights for policy 0, policy_version 362622 (0.0008) [2023-12-26 18:00:19,879][105692] Updated weights for policy 0, policy_version 362632 (0.0008) [2023-12-26 18:00:20,050][105620] Updated weights for policy 1, policy_version 363274 (0.0007) [2023-12-26 18:00:20,116][105620] Updated weights for policy 1, policy_version 363284 (0.0010) [2023-12-26 18:00:20,183][105620] Updated weights for policy 1, policy_version 363294 (0.0010) [2023-12-26 18:00:20,251][105620] Updated weights for policy 1, policy_version 363304 (0.0011) [2023-12-26 18:00:20,624][105692] Updated weights for policy 0, policy_version 362642 (0.0008) [2023-12-26 18:00:20,684][105692] Updated weights for policy 0, policy_version 362652 (0.0008) [2023-12-26 18:00:20,744][105692] Updated weights for policy 0, policy_version 362662 (0.0008) [2023-12-26 18:00:20,987][105620] Updated weights for policy 1, policy_version 363314 (0.0010) [2023-12-26 18:00:21,053][105620] Updated weights for policy 1, policy_version 363324 (0.0009) [2023-12-26 18:00:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 185868288. Throughput: 0: 9525.1, 1: 9935.7. Samples: 185861164. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-12-26 18:00:21,062][104569] Avg episode reward: [(0, '9267.650'), (1, '9173.432')] [2023-12-26 18:00:21,119][105620] Updated weights for policy 1, policy_version 363334 (0.0010) [2023-12-26 18:00:21,539][105692] Updated weights for policy 0, policy_version 362672 (0.0008) [2023-12-26 18:00:21,601][105692] Updated weights for policy 0, policy_version 362682 (0.0009) [2023-12-26 18:00:21,669][105692] Updated weights for policy 0, policy_version 362692 (0.0009) [2023-12-26 18:00:21,816][105620] Updated weights for policy 1, policy_version 363344 (0.0009) [2023-12-26 18:00:21,876][105620] Updated weights for policy 1, policy_version 363354 (0.0010) [2023-12-26 18:00:21,939][105620] Updated weights for policy 1, policy_version 363364 (0.0009) [2023-12-26 18:00:22,430][105692] Updated weights for policy 0, policy_version 362702 (0.0008) [2023-12-26 18:00:22,495][105692] Updated weights for policy 0, policy_version 362712 (0.0008) [2023-12-26 18:00:22,560][105692] Updated weights for policy 0, policy_version 362722 (0.0008) [2023-12-26 18:00:22,717][105620] Updated weights for policy 1, policy_version 363374 (0.0009) [2023-12-26 18:00:22,773][105620] Updated weights for policy 1, policy_version 363384 (0.0006) [2023-12-26 18:00:22,842][105620] Updated weights for policy 1, policy_version 363394 (0.0007) [2023-12-26 18:00:23,284][105692] Updated weights for policy 0, policy_version 362732 (0.0008) [2023-12-26 18:00:23,341][105692] Updated weights for policy 0, policy_version 362742 (0.0007) [2023-12-26 18:00:23,407][105692] Updated weights for policy 0, policy_version 362752 (0.0011) [2023-12-26 18:00:23,422][105620] Updated weights for policy 1, policy_version 363404 (0.0007) [2023-12-26 18:00:23,471][105620] Updated weights for policy 1, policy_version 363414 (0.0006) [2023-12-26 18:00:23,537][105620] Updated weights for policy 1, policy_version 363424 (0.0006) [2023-12-26 18:00:24,119][105692] Updated weights for policy 0, policy_version 362762 (0.0010) [2023-12-26 18:00:24,188][105692] Updated weights for policy 0, policy_version 362772 (0.0011) [2023-12-26 18:00:24,195][105620] Updated weights for policy 1, policy_version 363434 (0.0005) [2023-12-26 18:00:24,243][105692] Updated weights for policy 0, policy_version 362782 (0.0010) [2023-12-26 18:00:24,250][105620] Updated weights for policy 1, policy_version 363444 (0.0005) [2023-12-26 18:00:24,303][105692] Updated weights for policy 0, policy_version 362792 (0.0010) [2023-12-26 18:00:24,308][105620] Updated weights for policy 1, policy_version 363454 (0.0006) [2023-12-26 18:00:24,375][105620] Updated weights for policy 1, policy_version 363464 (0.0005) [2023-12-26 18:00:24,995][105692] Updated weights for policy 0, policy_version 362802 (0.0011) [2023-12-26 18:00:25,001][105620] Updated weights for policy 1, policy_version 363474 (0.0006) [2023-12-26 18:00:25,051][105692] Updated weights for policy 0, policy_version 362812 (0.0010) [2023-12-26 18:00:25,057][105620] Updated weights for policy 1, policy_version 363484 (0.0006) [2023-12-26 18:00:25,107][105692] Updated weights for policy 0, policy_version 362822 (0.0011) [2023-12-26 18:00:25,113][105620] Updated weights for policy 1, policy_version 363494 (0.0005) [2023-12-26 18:00:25,739][105692] Updated weights for policy 0, policy_version 362832 (0.0006) [2023-12-26 18:00:25,743][105620] Updated weights for policy 1, policy_version 363504 (0.0008) [2023-12-26 18:00:25,792][105692] Updated weights for policy 0, policy_version 362842 (0.0009) [2023-12-26 18:00:25,810][105620] Updated weights for policy 1, policy_version 363514 (0.0008) [2023-12-26 18:00:25,854][105692] Updated weights for policy 0, policy_version 362852 (0.0007) [2023-12-26 18:00:25,865][105620] Updated weights for policy 1, policy_version 363524 (0.0006) [2023-12-26 18:00:26,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 185974784. Throughput: 0: 9541.1, 1: 9985.8. Samples: 185978736. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-12-26 18:00:26,063][104569] Avg episode reward: [(0, '9268.059'), (1, '9266.439')] [2023-12-26 18:00:26,431][105620] Updated weights for policy 1, policy_version 363534 (0.0005) [2023-12-26 18:00:26,486][105620] Updated weights for policy 1, policy_version 363544 (0.0005) [2023-12-26 18:00:26,537][105620] Updated weights for policy 1, policy_version 363554 (0.0005) [2023-12-26 18:00:26,543][105692] Updated weights for policy 0, policy_version 362862 (0.0007) [2023-12-26 18:00:26,596][105692] Updated weights for policy 0, policy_version 362872 (0.0005) [2023-12-26 18:00:26,644][105692] Updated weights for policy 0, policy_version 362882 (0.0005) [2023-12-26 18:00:27,147][105620] Updated weights for policy 1, policy_version 363564 (0.0007) [2023-12-26 18:00:27,184][105692] Updated weights for policy 0, policy_version 362892 (0.0007) [2023-12-26 18:00:27,196][105620] Updated weights for policy 1, policy_version 363574 (0.0005) [2023-12-26 18:00:27,239][105692] Updated weights for policy 0, policy_version 362902 (0.0009) [2023-12-26 18:00:27,256][105620] Updated weights for policy 1, policy_version 363584 (0.0005) [2023-12-26 18:00:27,298][105692] Updated weights for policy 0, policy_version 362912 (0.0008) [2023-12-26 18:00:27,793][105620] Updated weights for policy 1, policy_version 363594 (0.0005) [2023-12-26 18:00:27,844][105620] Updated weights for policy 1, policy_version 363604 (0.0006) [2023-12-26 18:00:27,891][105620] Updated weights for policy 1, policy_version 363614 (0.0005) [2023-12-26 18:00:27,947][105620] Updated weights for policy 1, policy_version 363624 (0.0005) [2023-12-26 18:00:28,163][105692] Updated weights for policy 0, policy_version 362922 (0.0010) [2023-12-26 18:00:28,215][105692] Updated weights for policy 0, policy_version 362932 (0.0010) [2023-12-26 18:00:28,273][105692] Updated weights for policy 0, policy_version 362942 (0.0010) [2023-12-26 18:00:28,327][105692] Updated weights for policy 0, policy_version 362952 (0.0009) [2023-12-26 18:00:28,530][105620] Updated weights for policy 1, policy_version 363634 (0.0011) [2023-12-26 18:00:28,593][105620] Updated weights for policy 1, policy_version 363644 (0.0011) [2023-12-26 18:00:28,658][105620] Updated weights for policy 1, policy_version 363654 (0.0010) [2023-12-26 18:00:29,110][105692] Updated weights for policy 0, policy_version 362962 (0.0010) [2023-12-26 18:00:29,165][105692] Updated weights for policy 0, policy_version 362972 (0.0012) [2023-12-26 18:00:29,205][105620] Updated weights for policy 1, policy_version 363664 (0.0007) [2023-12-26 18:00:29,213][105692] Updated weights for policy 0, policy_version 362982 (0.0009) [2023-12-26 18:00:29,265][105620] Updated weights for policy 1, policy_version 363674 (0.0011) [2023-12-26 18:00:29,327][105620] Updated weights for policy 1, policy_version 363684 (0.0009) [2023-12-26 18:00:29,891][105620] Updated weights for policy 1, policy_version 363694 (0.0009) [2023-12-26 18:00:29,959][105620] Updated weights for policy 1, policy_version 363704 (0.0010) [2023-12-26 18:00:30,014][105620] Updated weights for policy 1, policy_version 363714 (0.0008) [2023-12-26 18:00:30,085][105692] Updated weights for policy 0, policy_version 362992 (0.0009) [2023-12-26 18:00:30,155][105692] Updated weights for policy 0, policy_version 363002 (0.0010) [2023-12-26 18:00:30,226][105692] Updated weights for policy 0, policy_version 363012 (0.0010) [2023-12-26 18:00:30,593][105620] Updated weights for policy 1, policy_version 363724 (0.0008) [2023-12-26 18:00:30,649][105620] Updated weights for policy 1, policy_version 363734 (0.0008) [2023-12-26 18:00:30,684][105586] KL-divergence is very high: 143.3152 [2023-12-26 18:00:30,689][105586] KL-divergence is very high: 107.7929 [2023-12-26 18:00:30,694][105620] Updated weights for policy 1, policy_version 363744 (0.0005) [2023-12-26 18:00:30,724][105586] KL-divergence is very high: 124.9872 [2023-12-26 18:00:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 186073088. Throughput: 0: 9535.8, 1: 10137.2. Samples: 186044560. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-12-26 18:00:31,062][104569] Avg episode reward: [(0, '9273.091'), (1, '8895.216')] [2023-12-26 18:00:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000363752_93126656.pth... [2023-12-26 18:00:31,069][105692] Updated weights for policy 0, policy_version 363022 (0.0009) [2023-12-26 18:00:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000362536_92815360.pth [2023-12-26 18:00:31,133][105692] Updated weights for policy 0, policy_version 363032 (0.0009) [2023-12-26 18:00:31,202][105692] Updated weights for policy 0, policy_version 363042 (0.0009) [2023-12-26 18:00:31,231][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000363048_92954624.pth... [2023-12-26 18:00:31,234][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000361928_92667904.pth [2023-12-26 18:00:31,330][105586] KL-divergence is very high: 102.4113 [2023-12-26 18:00:31,342][105620] Updated weights for policy 1, policy_version 363754 (0.0007) [2023-12-26 18:00:31,384][105586] KL-divergence is very high: 119.9436 [2023-12-26 18:00:31,413][105620] Updated weights for policy 1, policy_version 363764 (0.0007) [2023-12-26 18:00:31,420][105586] KL-divergence is very high: 207.5935 [2023-12-26 18:00:31,428][105586] KL-divergence is very high: 515.4281 [2023-12-26 18:00:31,441][105586] KL-divergence is very high: 564.2100 [2023-12-26 18:00:31,454][105586] KL-divergence is very high: 344.8161 [2023-12-26 18:00:31,474][105586] KL-divergence is very high: 373.4375 [2023-12-26 18:00:31,482][105586] KL-divergence is very high: 802.2231 [2023-12-26 18:00:31,483][105620] Updated weights for policy 1, policy_version 363774 (0.0009) [2023-12-26 18:00:31,495][105586] KL-divergence is very high: 696.3582 [2023-12-26 18:00:31,508][105586] KL-divergence is very high: 378.4193 [2023-12-26 18:00:31,527][105586] KL-divergence is very high: 326.7635 [2023-12-26 18:00:31,533][105586] KL-divergence is very high: 698.2400 [2023-12-26 18:00:31,546][105620] Updated weights for policy 1, policy_version 363784 (0.0010) [2023-12-26 18:00:32,015][105692] Updated weights for policy 0, policy_version 363052 (0.0008) [2023-12-26 18:00:32,067][105692] Updated weights for policy 0, policy_version 363062 (0.0008) [2023-12-26 18:00:32,117][105692] Updated weights for policy 0, policy_version 363072 (0.0008) [2023-12-26 18:00:32,218][105620] Updated weights for policy 1, policy_version 363794 (0.0010) [2023-12-26 18:00:32,288][105620] Updated weights for policy 1, policy_version 363804 (0.0010) [2023-12-26 18:00:32,352][105620] Updated weights for policy 1, policy_version 363814 (0.0010) [2023-12-26 18:00:32,882][105692] Updated weights for policy 0, policy_version 363082 (0.0008) [2023-12-26 18:00:32,936][105692] Updated weights for policy 0, policy_version 363092 (0.0009) [2023-12-26 18:00:32,992][105692] Updated weights for policy 0, policy_version 363102 (0.0011) [2023-12-26 18:00:33,049][105692] Updated weights for policy 0, policy_version 363112 (0.0011) [2023-12-26 18:00:33,049][105620] Updated weights for policy 1, policy_version 363824 (0.0006) [2023-12-26 18:00:33,097][105620] Updated weights for policy 1, policy_version 363834 (0.0005) [2023-12-26 18:00:33,144][105620] Updated weights for policy 1, policy_version 363844 (0.0005) [2023-12-26 18:00:33,744][105692] Updated weights for policy 0, policy_version 363122 (0.0006) [2023-12-26 18:00:33,795][105692] Updated weights for policy 0, policy_version 363132 (0.0009) [2023-12-26 18:00:33,821][105620] Updated weights for policy 1, policy_version 363854 (0.0007) [2023-12-26 18:00:33,844][105692] Updated weights for policy 0, policy_version 363142 (0.0006) [2023-12-26 18:00:33,877][105620] Updated weights for policy 1, policy_version 363864 (0.0009) [2023-12-26 18:00:33,939][105620] Updated weights for policy 1, policy_version 363874 (0.0008) [2023-12-26 18:00:34,583][105692] Updated weights for policy 0, policy_version 363152 (0.0009) [2023-12-26 18:00:34,634][105692] Updated weights for policy 0, policy_version 363162 (0.0009) [2023-12-26 18:00:34,663][105620] Updated weights for policy 1, policy_version 363884 (0.0008) [2023-12-26 18:00:34,685][105692] Updated weights for policy 0, policy_version 363172 (0.0008) [2023-12-26 18:00:34,731][105620] Updated weights for policy 1, policy_version 363894 (0.0008) [2023-12-26 18:00:34,792][105620] Updated weights for policy 1, policy_version 363904 (0.0010) [2023-12-26 18:00:35,351][105692] Updated weights for policy 0, policy_version 363182 (0.0007) [2023-12-26 18:00:35,409][105692] Updated weights for policy 0, policy_version 363192 (0.0009) [2023-12-26 18:00:35,469][105692] Updated weights for policy 0, policy_version 363202 (0.0010) [2023-12-26 18:00:35,582][105620] Updated weights for policy 1, policy_version 363914 (0.0008) [2023-12-26 18:00:35,636][105620] Updated weights for policy 1, policy_version 363924 (0.0008) [2023-12-26 18:00:35,687][105620] Updated weights for policy 1, policy_version 363934 (0.0009) [2023-12-26 18:00:35,733][105620] Updated weights for policy 1, policy_version 363944 (0.0008) [2023-12-26 18:00:36,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 186171392. Throughput: 0: 9422.8, 1: 10266.1. Samples: 186160676. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-12-26 18:00:36,062][104569] Avg episode reward: [(0, '9273.273'), (1, '8895.178')] [2023-12-26 18:00:36,234][105692] Updated weights for policy 0, policy_version 363212 (0.0009) [2023-12-26 18:00:36,297][105692] Updated weights for policy 0, policy_version 363222 (0.0009) [2023-12-26 18:00:36,359][105692] Updated weights for policy 0, policy_version 363232 (0.0008) [2023-12-26 18:00:36,558][105620] Updated weights for policy 1, policy_version 363954 (0.0010) [2023-12-26 18:00:36,628][105620] Updated weights for policy 1, policy_version 363964 (0.0008) [2023-12-26 18:00:36,694][105620] Updated weights for policy 1, policy_version 363974 (0.0008) [2023-12-26 18:00:37,101][105692] Updated weights for policy 0, policy_version 363242 (0.0007) [2023-12-26 18:00:37,163][105692] Updated weights for policy 0, policy_version 363252 (0.0009) [2023-12-26 18:00:37,229][105692] Updated weights for policy 0, policy_version 363262 (0.0009) [2023-12-26 18:00:37,292][105692] Updated weights for policy 0, policy_version 363272 (0.0009) [2023-12-26 18:00:37,408][105620] Updated weights for policy 1, policy_version 363984 (0.0006) [2023-12-26 18:00:37,469][105620] Updated weights for policy 1, policy_version 363994 (0.0009) [2023-12-26 18:00:37,531][105620] Updated weights for policy 1, policy_version 364004 (0.0009) [2023-12-26 18:00:37,996][105692] Updated weights for policy 0, policy_version 363282 (0.0009) [2023-12-26 18:00:38,060][105692] Updated weights for policy 0, policy_version 363292 (0.0010) [2023-12-26 18:00:38,121][105692] Updated weights for policy 0, policy_version 363302 (0.0009) [2023-12-26 18:00:38,303][105620] Updated weights for policy 1, policy_version 364014 (0.0010) [2023-12-26 18:00:38,368][105620] Updated weights for policy 1, policy_version 364024 (0.0009) [2023-12-26 18:00:38,462][105620] Updated weights for policy 1, policy_version 364034 (0.0009) [2023-12-26 18:00:38,740][105692] Updated weights for policy 0, policy_version 363312 (0.0006) [2023-12-26 18:00:38,800][105692] Updated weights for policy 0, policy_version 363322 (0.0006) [2023-12-26 18:00:38,859][105692] Updated weights for policy 0, policy_version 363332 (0.0007) [2023-12-26 18:00:39,266][105620] Updated weights for policy 1, policy_version 364044 (0.0010) [2023-12-26 18:00:39,320][105620] Updated weights for policy 1, policy_version 364054 (0.0011) [2023-12-26 18:00:39,386][105620] Updated weights for policy 1, policy_version 364064 (0.0009) [2023-12-26 18:00:39,593][105692] Updated weights for policy 0, policy_version 363342 (0.0008) [2023-12-26 18:00:39,654][105692] Updated weights for policy 0, policy_version 363352 (0.0007) [2023-12-26 18:00:39,703][105692] Updated weights for policy 0, policy_version 363362 (0.0008) [2023-12-26 18:00:40,130][105620] Updated weights for policy 1, policy_version 364074 (0.0011) [2023-12-26 18:00:40,194][105620] Updated weights for policy 1, policy_version 364084 (0.0011) [2023-12-26 18:00:40,268][105620] Updated weights for policy 1, policy_version 364094 (0.0011) [2023-12-26 18:00:40,332][105620] Updated weights for policy 1, policy_version 364104 (0.0010) [2023-12-26 18:00:40,539][105692] Updated weights for policy 0, policy_version 363372 (0.0008) [2023-12-26 18:00:40,590][105692] Updated weights for policy 0, policy_version 363382 (0.0008) [2023-12-26 18:00:40,643][105692] Updated weights for policy 0, policy_version 363392 (0.0008) [2023-12-26 18:00:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 186261504. Throughput: 0: 9460.9, 1: 10145.5. Samples: 186272588. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-12-26 18:00:41,062][104569] Avg episode reward: [(0, '9196.785'), (1, '8895.785')] [2023-12-26 18:00:41,094][105620] Updated weights for policy 1, policy_version 364114 (0.0011) [2023-12-26 18:00:41,154][105620] Updated weights for policy 1, policy_version 364124 (0.0009) [2023-12-26 18:00:41,219][105620] Updated weights for policy 1, policy_version 364134 (0.0008) [2023-12-26 18:00:41,315][105692] Updated weights for policy 0, policy_version 363402 (0.0008) [2023-12-26 18:00:41,379][105692] Updated weights for policy 0, policy_version 363412 (0.0008) [2023-12-26 18:00:41,443][105692] Updated weights for policy 0, policy_version 363422 (0.0009) [2023-12-26 18:00:41,500][105692] Updated weights for policy 0, policy_version 363432 (0.0011) [2023-12-26 18:00:41,998][105620] Updated weights for policy 1, policy_version 364144 (0.0008) [2023-12-26 18:00:42,057][105620] Updated weights for policy 1, policy_version 364154 (0.0008) [2023-12-26 18:00:42,118][105620] Updated weights for policy 1, policy_version 364164 (0.0008) [2023-12-26 18:00:42,294][105692] Updated weights for policy 0, policy_version 363442 (0.0008) [2023-12-26 18:00:42,358][105692] Updated weights for policy 0, policy_version 363452 (0.0008) [2023-12-26 18:00:42,423][105692] Updated weights for policy 0, policy_version 363462 (0.0008) [2023-12-26 18:00:42,912][105620] Updated weights for policy 1, policy_version 364174 (0.0010) [2023-12-26 18:00:42,979][105620] Updated weights for policy 1, policy_version 364184 (0.0009) [2023-12-26 18:00:43,041][105620] Updated weights for policy 1, policy_version 364194 (0.0010) [2023-12-26 18:00:43,133][105692] Updated weights for policy 0, policy_version 363472 (0.0008) [2023-12-26 18:00:43,185][105692] Updated weights for policy 0, policy_version 363482 (0.0010) [2023-12-26 18:00:43,240][105692] Updated weights for policy 0, policy_version 363492 (0.0010) [2023-12-26 18:00:43,804][105620] Updated weights for policy 1, policy_version 364204 (0.0007) [2023-12-26 18:00:43,853][105620] Updated weights for policy 1, policy_version 364214 (0.0005) [2023-12-26 18:00:43,914][105620] Updated weights for policy 1, policy_version 364224 (0.0006) [2023-12-26 18:00:43,985][105692] Updated weights for policy 0, policy_version 363502 (0.0009) [2023-12-26 18:00:44,038][105692] Updated weights for policy 0, policy_version 363512 (0.0010) [2023-12-26 18:00:44,092][105692] Updated weights for policy 0, policy_version 363524 (0.0011) [2023-12-26 18:00:44,572][105620] Updated weights for policy 1, policy_version 364234 (0.0006) [2023-12-26 18:00:44,629][105620] Updated weights for policy 1, policy_version 364244 (0.0008) [2023-12-26 18:00:44,667][105586] KL-divergence is very high: 172.3231 [2023-12-26 18:00:44,683][105620] Updated weights for policy 1, policy_version 364254 (0.0006) [2023-12-26 18:00:44,718][105586] KL-divergence is very high: 196.9727 [2023-12-26 18:00:44,743][105620] Updated weights for policy 1, policy_version 364264 (0.0009) [2023-12-26 18:00:44,865][105692] Updated weights for policy 0, policy_version 363534 (0.0007) [2023-12-26 18:00:44,927][105692] Updated weights for policy 0, policy_version 363544 (0.0009) [2023-12-26 18:00:44,988][105692] Updated weights for policy 0, policy_version 363554 (0.0010) [2023-12-26 18:00:45,524][105620] Updated weights for policy 1, policy_version 364274 (0.0008) [2023-12-26 18:00:45,588][105620] Updated weights for policy 1, policy_version 364284 (0.0009) [2023-12-26 18:00:45,648][105620] Updated weights for policy 1, policy_version 364294 (0.0009) [2023-12-26 18:00:45,737][105692] Updated weights for policy 0, policy_version 363564 (0.0009) [2023-12-26 18:00:45,797][105692] Updated weights for policy 0, policy_version 363574 (0.0005) [2023-12-26 18:00:45,857][105692] Updated weights for policy 0, policy_version 363584 (0.0006) [2023-12-26 18:00:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 186359808. Throughput: 0: 9437.9, 1: 10073.8. Samples: 186327460. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-12-26 18:00:46,063][104569] Avg episode reward: [(0, '9282.825'), (1, '8988.176')] [2023-12-26 18:00:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000363592_93093888.pth... [2023-12-26 18:00:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000364296_93265920.pth... [2023-12-26 18:00:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000363144_92971008.pth [2023-12-26 18:00:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000362472_92807168.pth [2023-12-26 18:00:46,387][105620] Updated weights for policy 1, policy_version 364304 (0.0006) [2023-12-26 18:00:46,458][105620] Updated weights for policy 1, policy_version 364314 (0.0005) [2023-12-26 18:00:46,524][105620] Updated weights for policy 1, policy_version 364324 (0.0006) [2023-12-26 18:00:46,556][105692] Updated weights for policy 0, policy_version 363594 (0.0006) [2023-12-26 18:00:46,619][105692] Updated weights for policy 0, policy_version 363604 (0.0009) [2023-12-26 18:00:46,679][105692] Updated weights for policy 0, policy_version 363614 (0.0009) [2023-12-26 18:00:46,739][105692] Updated weights for policy 0, policy_version 363624 (0.0009) [2023-12-26 18:00:47,206][105620] Updated weights for policy 1, policy_version 364334 (0.0005) [2023-12-26 18:00:47,274][105620] Updated weights for policy 1, policy_version 364344 (0.0005) [2023-12-26 18:00:47,339][105620] Updated weights for policy 1, policy_version 364354 (0.0005) [2023-12-26 18:00:47,523][105692] Updated weights for policy 0, policy_version 363634 (0.0006) [2023-12-26 18:00:47,576][105692] Updated weights for policy 0, policy_version 363644 (0.0005) [2023-12-26 18:00:47,628][105692] Updated weights for policy 0, policy_version 363654 (0.0005) [2023-12-26 18:00:47,848][105620] Updated weights for policy 1, policy_version 364364 (0.0006) [2023-12-26 18:00:47,910][105620] Updated weights for policy 1, policy_version 364374 (0.0008) [2023-12-26 18:00:47,973][105620] Updated weights for policy 1, policy_version 364384 (0.0008) [2023-12-26 18:00:48,348][105692] Updated weights for policy 0, policy_version 363664 (0.0008) [2023-12-26 18:00:48,408][105692] Updated weights for policy 0, policy_version 363674 (0.0009) [2023-12-26 18:00:48,465][105692] Updated weights for policy 0, policy_version 363684 (0.0009) [2023-12-26 18:00:48,676][105620] Updated weights for policy 1, policy_version 364394 (0.0008) [2023-12-26 18:00:48,734][105620] Updated weights for policy 1, policy_version 364404 (0.0006) [2023-12-26 18:00:48,800][105620] Updated weights for policy 1, policy_version 364414 (0.0010) [2023-12-26 18:00:48,852][105620] Updated weights for policy 1, policy_version 364424 (0.0006) [2023-12-26 18:00:49,312][105692] Updated weights for policy 0, policy_version 363694 (0.0009) [2023-12-26 18:00:49,372][105692] Updated weights for policy 0, policy_version 363704 (0.0008) [2023-12-26 18:00:49,427][105692] Updated weights for policy 0, policy_version 363714 (0.0007) [2023-12-26 18:00:49,446][105620] Updated weights for policy 1, policy_version 364434 (0.0008) [2023-12-26 18:00:49,499][105620] Updated weights for policy 1, policy_version 364444 (0.0009) [2023-12-26 18:00:49,552][105620] Updated weights for policy 1, policy_version 364454 (0.0008) [2023-12-26 18:00:49,555][105586] KL-divergence is very high: 320.2736 [2023-12-26 18:00:50,199][105692] Updated weights for policy 0, policy_version 363724 (0.0007) [2023-12-26 18:00:50,252][105692] Updated weights for policy 0, policy_version 363734 (0.0008) [2023-12-26 18:00:50,267][105620] Updated weights for policy 1, policy_version 364464 (0.0008) [2023-12-26 18:00:50,305][105692] Updated weights for policy 0, policy_version 363744 (0.0008) [2023-12-26 18:00:50,323][105620] Updated weights for policy 1, policy_version 364474 (0.0007) [2023-12-26 18:00:50,375][105620] Updated weights for policy 1, policy_version 364484 (0.0008) [2023-12-26 18:00:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 186449920. Throughput: 0: 9350.0, 1: 10136.8. Samples: 186443624. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-12-26 18:00:51,062][104569] Avg episode reward: [(0, '906.641'), (1, '8987.956')] [2023-12-26 18:00:51,076][105692] Updated weights for policy 0, policy_version 363754 (0.0007) [2023-12-26 18:00:51,102][105620] Updated weights for policy 1, policy_version 364494 (0.0008) [2023-12-26 18:00:51,143][105692] Updated weights for policy 0, policy_version 363764 (0.0009) [2023-12-26 18:00:51,166][105620] Updated weights for policy 1, policy_version 364504 (0.0007) [2023-12-26 18:00:51,192][105692] Updated weights for policy 0, policy_version 363774 (0.0006) [2023-12-26 18:00:51,230][105620] Updated weights for policy 1, policy_version 364514 (0.0008) [2023-12-26 18:00:51,252][105692] Updated weights for policy 0, policy_version 363784 (0.0007) [2023-12-26 18:00:51,927][105620] Updated weights for policy 1, policy_version 364524 (0.0007) [2023-12-26 18:00:51,981][105620] Updated weights for policy 1, policy_version 364534 (0.0006) [2023-12-26 18:00:52,039][105620] Updated weights for policy 1, policy_version 364544 (0.0006) [2023-12-26 18:00:52,110][105692] Updated weights for policy 0, policy_version 363794 (0.0010) [2023-12-26 18:00:52,165][105585] KL-divergence is very high: 111.2084 [2023-12-26 18:00:52,173][105692] Updated weights for policy 0, policy_version 363804 (0.0010) [2023-12-26 18:00:52,231][105692] Updated weights for policy 0, policy_version 363814 (0.0009) [2023-12-26 18:00:52,703][105620] Updated weights for policy 1, policy_version 364554 (0.0006) [2023-12-26 18:00:52,762][105620] Updated weights for policy 1, policy_version 364564 (0.0009) [2023-12-26 18:00:52,822][105620] Updated weights for policy 1, policy_version 364574 (0.0010) [2023-12-26 18:00:52,876][105620] Updated weights for policy 1, policy_version 364584 (0.0008) [2023-12-26 18:00:53,019][105585] KL-divergence is very high: 147.6750 [2023-12-26 18:00:53,026][105585] KL-divergence is very high: 125.4346 [2023-12-26 18:00:53,055][105692] Updated weights for policy 0, policy_version 363824 (0.0010) [2023-12-26 18:00:53,122][105692] Updated weights for policy 0, policy_version 363834 (0.0010) [2023-12-26 18:00:53,193][105692] Updated weights for policy 0, policy_version 363844 (0.0010) [2023-12-26 18:00:53,543][105620] Updated weights for policy 1, policy_version 364594 (0.0009) [2023-12-26 18:00:53,604][105620] Updated weights for policy 1, policy_version 364604 (0.0009) [2023-12-26 18:00:53,665][105620] Updated weights for policy 1, policy_version 364614 (0.0009) [2023-12-26 18:00:53,953][105692] Updated weights for policy 0, policy_version 363854 (0.0009) [2023-12-26 18:00:54,012][105692] Updated weights for policy 0, policy_version 363864 (0.0009) [2023-12-26 18:00:54,075][105692] Updated weights for policy 0, policy_version 363874 (0.0009) [2023-12-26 18:00:54,418][105620] Updated weights for policy 1, policy_version 364624 (0.0009) [2023-12-26 18:00:54,468][105620] Updated weights for policy 1, policy_version 364634 (0.0008) [2023-12-26 18:00:54,529][105620] Updated weights for policy 1, policy_version 364644 (0.0008) [2023-12-26 18:00:54,786][105692] Updated weights for policy 0, policy_version 363884 (0.0009) [2023-12-26 18:00:54,847][105692] Updated weights for policy 0, policy_version 363894 (0.0009) [2023-12-26 18:00:54,908][105692] Updated weights for policy 0, policy_version 363905 (0.0010) [2023-12-26 18:00:55,185][105620] Updated weights for policy 1, policy_version 364654 (0.0009) [2023-12-26 18:00:55,242][105620] Updated weights for policy 1, policy_version 364664 (0.0008) [2023-12-26 18:00:55,298][105620] Updated weights for policy 1, policy_version 364674 (0.0008) [2023-12-26 18:00:55,729][105692] Updated weights for policy 0, policy_version 363916 (0.0009) [2023-12-26 18:00:55,744][105585] KL-divergence is very high: 110.7447 [2023-12-26 18:00:55,794][105692] Updated weights for policy 0, policy_version 363926 (0.0009) [2023-12-26 18:00:55,849][105692] Updated weights for policy 0, policy_version 363936 (0.0009) [2023-12-26 18:00:56,008][105620] Updated weights for policy 1, policy_version 364684 (0.0010) [2023-12-26 18:00:56,062][105620] Updated weights for policy 1, policy_version 364694 (0.0008) [2023-12-26 18:00:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.6, 300 sec: 19438.6). Total num frames: 186548224. Throughput: 0: 9264.5, 1: 10146.9. Samples: 186558032. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-12-26 18:00:56,063][104569] Avg episode reward: [(0, '1034.741'), (1, '8802.701')] [2023-12-26 18:00:56,105][105586] KL-divergence is very high: 108.8301 [2023-12-26 18:00:56,111][105620] Updated weights for policy 1, policy_version 364704 (0.0009) [2023-12-26 18:00:56,606][105692] Updated weights for policy 0, policy_version 363946 (0.0009) [2023-12-26 18:00:56,657][105692] Updated weights for policy 0, policy_version 363956 (0.0009) [2023-12-26 18:00:56,681][105585] KL-divergence is very high: 127.8933 [2023-12-26 18:00:56,718][105692] Updated weights for policy 0, policy_version 363966 (0.0009) [2023-12-26 18:00:56,741][105585] KL-divergence is very high: 233.2980 [2023-12-26 18:00:56,772][105692] Updated weights for policy 0, policy_version 363976 (0.0009) [2023-12-26 18:00:56,842][105620] Updated weights for policy 1, policy_version 364714 (0.0009) [2023-12-26 18:00:56,895][105620] Updated weights for policy 1, policy_version 364724 (0.0008) [2023-12-26 18:00:56,941][105620] Updated weights for policy 1, policy_version 364734 (0.0009) [2023-12-26 18:00:56,995][105620] Updated weights for policy 1, policy_version 364744 (0.0009) [2023-12-26 18:00:57,480][105585] KL-divergence is very high: 105.7894 [2023-12-26 18:00:57,497][105585] KL-divergence is very high: 119.7263 [2023-12-26 18:00:57,515][105585] KL-divergence is very high: 107.8159 [2023-12-26 18:00:57,519][105585] KL-divergence is very high: 114.2389 [2023-12-26 18:00:57,530][105692] Updated weights for policy 0, policy_version 363986 (0.0009) [2023-12-26 18:00:57,533][105585] KL-divergence is very high: 113.7606 [2023-12-26 18:00:57,552][105585] KL-divergence is very high: 128.4896 [2023-12-26 18:00:57,578][105692] Updated weights for policy 0, policy_version 363996 (0.0009) [2023-12-26 18:00:57,626][105692] Updated weights for policy 0, policy_version 364006 (0.0009) [2023-12-26 18:00:57,762][105620] Updated weights for policy 1, policy_version 364754 (0.0010) [2023-12-26 18:00:57,814][105620] Updated weights for policy 1, policy_version 364764 (0.0010) [2023-12-26 18:00:57,862][105620] Updated weights for policy 1, policy_version 364774 (0.0010) [2023-12-26 18:00:58,380][105692] Updated weights for policy 0, policy_version 364016 (0.0008) [2023-12-26 18:00:58,392][105585] KL-divergence is very high: 118.3179 [2023-12-26 18:00:58,410][105585] KL-divergence is very high: 100.5834 [2023-12-26 18:00:58,442][105692] Updated weights for policy 0, policy_version 364026 (0.0008) [2023-12-26 18:00:58,497][105692] Updated weights for policy 0, policy_version 364036 (0.0008) [2023-12-26 18:00:58,504][105585] KL-divergence is very high: 112.4731 [2023-12-26 18:00:58,514][105585] KL-divergence is very high: 101.2145 [2023-12-26 18:00:58,611][105620] Updated weights for policy 1, policy_version 364784 (0.0008) [2023-12-26 18:00:58,672][105620] Updated weights for policy 1, policy_version 364794 (0.0008) [2023-12-26 18:00:58,735][105620] Updated weights for policy 1, policy_version 364804 (0.0007) [2023-12-26 18:00:59,337][105692] Updated weights for policy 0, policy_version 364046 (0.0008) [2023-12-26 18:00:59,389][105585] KL-divergence is very high: 114.1021 [2023-12-26 18:00:59,396][105585] KL-divergence is very high: 117.1017 [2023-12-26 18:00:59,403][105585] KL-divergence is very high: 108.0427 [2023-12-26 18:00:59,412][105692] Updated weights for policy 0, policy_version 364056 (0.0008) [2023-12-26 18:00:59,468][105585] KL-divergence is very high: 150.1395 [2023-12-26 18:00:59,471][105620] Updated weights for policy 1, policy_version 364814 (0.0008) [2023-12-26 18:00:59,475][105692] Updated weights for policy 0, policy_version 364066 (0.0005) [2023-12-26 18:00:59,538][105620] Updated weights for policy 1, policy_version 364824 (0.0009) [2023-12-26 18:00:59,609][105620] Updated weights for policy 1, policy_version 364834 (0.0006) [2023-12-26 18:01:00,132][105692] Updated weights for policy 0, policy_version 364076 (0.0007) [2023-12-26 18:01:00,148][105585] KL-divergence is very high: 202.7421 [2023-12-26 18:01:00,187][105692] Updated weights for policy 0, policy_version 364086 (0.0009) [2023-12-26 18:01:00,194][105585] KL-divergence is very high: 201.4093 [2023-12-26 18:01:00,242][105585] KL-divergence is very high: 152.4845 [2023-12-26 18:01:00,245][105692] Updated weights for policy 0, policy_version 364096 (0.0008) [2023-12-26 18:01:00,288][105585] KL-divergence is very high: 144.7996 [2023-12-26 18:01:00,341][105620] Updated weights for policy 1, policy_version 364844 (0.0009) [2023-12-26 18:01:00,405][105620] Updated weights for policy 1, policy_version 364854 (0.0008) [2023-12-26 18:01:00,456][105620] Updated weights for policy 1, policy_version 364864 (0.0005) [2023-12-26 18:01:00,996][105620] Updated weights for policy 1, policy_version 364874 (0.0005) [2023-12-26 18:01:01,061][105620] Updated weights for policy 1, policy_version 364884 (0.0007) [2023-12-26 18:01:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 186638336. Throughput: 0: 9315.8, 1: 10062.7. Samples: 186613692. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-12-26 18:01:01,062][104569] Avg episode reward: [(0, '1357.071'), (1, '8709.301')] [2023-12-26 18:01:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000364104_93224960.pth... [2023-12-26 18:01:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000363048_92954624.pth [2023-12-26 18:01:01,110][105692] Updated weights for policy 0, policy_version 364106 (0.0009) [2023-12-26 18:01:01,116][105620] Updated weights for policy 1, policy_version 364894 (0.0008) [2023-12-26 18:01:01,172][105692] Updated weights for policy 0, policy_version 364116 (0.0007) [2023-12-26 18:01:01,178][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000364904_93421568.pth... [2023-12-26 18:01:01,179][105620] Updated weights for policy 1, policy_version 364904 (0.0008) [2023-12-26 18:01:01,183][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000363752_93126656.pth [2023-12-26 18:01:01,228][105692] Updated weights for policy 0, policy_version 364126 (0.0006) [2023-12-26 18:01:01,295][105692] Updated weights for policy 0, policy_version 364136 (0.0008) [2023-12-26 18:01:01,910][105620] Updated weights for policy 1, policy_version 364914 (0.0006) [2023-12-26 18:01:01,970][105620] Updated weights for policy 1, policy_version 364924 (0.0006) [2023-12-26 18:01:02,026][105620] Updated weights for policy 1, policy_version 364934 (0.0005) [2023-12-26 18:01:02,094][105692] Updated weights for policy 0, policy_version 364146 (0.0010) [2023-12-26 18:01:02,147][105692] Updated weights for policy 0, policy_version 364156 (0.0009) [2023-12-26 18:01:02,202][105692] Updated weights for policy 0, policy_version 364166 (0.0010) [2023-12-26 18:01:02,572][105620] Updated weights for policy 1, policy_version 364944 (0.0005) [2023-12-26 18:01:02,633][105620] Updated weights for policy 1, policy_version 364954 (0.0005) [2023-12-26 18:01:02,682][105620] Updated weights for policy 1, policy_version 364964 (0.0005) [2023-12-26 18:01:03,095][105692] Updated weights for policy 0, policy_version 364176 (0.0008) [2023-12-26 18:01:03,140][105692] Updated weights for policy 0, policy_version 364186 (0.0008) [2023-12-26 18:01:03,187][105692] Updated weights for policy 0, policy_version 364196 (0.0008) [2023-12-26 18:01:03,287][105620] Updated weights for policy 1, policy_version 364974 (0.0005) [2023-12-26 18:01:03,343][105620] Updated weights for policy 1, policy_version 364984 (0.0005) [2023-12-26 18:01:03,395][105620] Updated weights for policy 1, policy_version 364994 (0.0005) [2023-12-26 18:01:03,956][105692] Updated weights for policy 0, policy_version 364206 (0.0007) [2023-12-26 18:01:04,018][105692] Updated weights for policy 0, policy_version 364216 (0.0006) [2023-12-26 18:01:04,034][105620] Updated weights for policy 1, policy_version 365004 (0.0008) [2023-12-26 18:01:04,079][105692] Updated weights for policy 0, policy_version 364226 (0.0007) [2023-12-26 18:01:04,094][105620] Updated weights for policy 1, policy_version 365014 (0.0011) [2023-12-26 18:01:04,158][105620] Updated weights for policy 1, policy_version 365024 (0.0011) [2023-12-26 18:01:04,782][105692] Updated weights for policy 0, policy_version 364236 (0.0008) [2023-12-26 18:01:04,832][105692] Updated weights for policy 0, policy_version 364246 (0.0011) [2023-12-26 18:01:04,881][105692] Updated weights for policy 0, policy_version 364256 (0.0010) [2023-12-26 18:01:04,917][105620] Updated weights for policy 1, policy_version 365034 (0.0010) [2023-12-26 18:01:04,980][105620] Updated weights for policy 1, policy_version 365044 (0.0010) [2023-12-26 18:01:05,046][105620] Updated weights for policy 1, policy_version 365054 (0.0010) [2023-12-26 18:01:05,104][105620] Updated weights for policy 1, policy_version 365064 (0.0010) [2023-12-26 18:01:05,580][105692] Updated weights for policy 0, policy_version 364266 (0.0010) [2023-12-26 18:01:05,632][105692] Updated weights for policy 0, policy_version 364276 (0.0009) [2023-12-26 18:01:05,678][105692] Updated weights for policy 0, policy_version 364286 (0.0008) [2023-12-26 18:01:05,730][105692] Updated weights for policy 0, policy_version 364296 (0.0008) [2023-12-26 18:01:05,823][105620] Updated weights for policy 1, policy_version 365074 (0.0010) [2023-12-26 18:01:05,874][105620] Updated weights for policy 1, policy_version 365084 (0.0010) [2023-12-26 18:01:05,922][105620] Updated weights for policy 1, policy_version 365094 (0.0010) [2023-12-26 18:01:06,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 186744832. Throughput: 0: 9264.6, 1: 10030.3. Samples: 186729436. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-12-26 18:01:06,062][104569] Avg episode reward: [(0, '2257.534'), (1, '9080.211')] [2023-12-26 18:01:06,496][105692] Updated weights for policy 0, policy_version 364306 (0.0009) [2023-12-26 18:01:06,560][105692] Updated weights for policy 0, policy_version 364316 (0.0009) [2023-12-26 18:01:06,613][105620] Updated weights for policy 1, policy_version 365104 (0.0010) [2023-12-26 18:01:06,619][105692] Updated weights for policy 0, policy_version 364326 (0.0009) [2023-12-26 18:01:06,677][105620] Updated weights for policy 1, policy_version 365114 (0.0009) [2023-12-26 18:01:06,731][105620] Updated weights for policy 1, policy_version 365124 (0.0008) [2023-12-26 18:01:07,330][105692] Updated weights for policy 0, policy_version 364336 (0.0009) [2023-12-26 18:01:07,391][105692] Updated weights for policy 0, policy_version 364346 (0.0009) [2023-12-26 18:01:07,450][105692] Updated weights for policy 0, policy_version 364356 (0.0009) [2023-12-26 18:01:07,512][105620] Updated weights for policy 1, policy_version 365134 (0.0009) [2023-12-26 18:01:07,572][105620] Updated weights for policy 1, policy_version 365144 (0.0008) [2023-12-26 18:01:07,632][105620] Updated weights for policy 1, policy_version 365154 (0.0009) [2023-12-26 18:01:08,202][105692] Updated weights for policy 0, policy_version 364366 (0.0009) [2023-12-26 18:01:08,250][105692] Updated weights for policy 0, policy_version 364376 (0.0006) [2023-12-26 18:01:08,307][105692] Updated weights for policy 0, policy_version 364386 (0.0008) [2023-12-26 18:01:08,338][105620] Updated weights for policy 1, policy_version 365164 (0.0008) [2023-12-26 18:01:08,399][105620] Updated weights for policy 1, policy_version 365174 (0.0009) [2023-12-26 18:01:08,461][105620] Updated weights for policy 1, policy_version 365184 (0.0010) [2023-12-26 18:01:08,998][105692] Updated weights for policy 0, policy_version 364396 (0.0007) [2023-12-26 18:01:09,049][105692] Updated weights for policy 0, policy_version 364406 (0.0005) [2023-12-26 18:01:09,099][105692] Updated weights for policy 0, policy_version 364416 (0.0005) [2023-12-26 18:01:09,246][105620] Updated weights for policy 1, policy_version 365194 (0.0009) [2023-12-26 18:01:09,282][105586] KL-divergence is very high: 111.1952 [2023-12-26 18:01:09,287][105586] KL-divergence is very high: 142.9803 [2023-12-26 18:01:09,292][105586] KL-divergence is very high: 130.8638 [2023-12-26 18:01:09,297][105586] KL-divergence is very high: 145.2182 [2023-12-26 18:01:09,302][105620] Updated weights for policy 1, policy_version 365204 (0.0006) [2023-12-26 18:01:09,353][105586] KL-divergence is very high: 139.4512 [2023-12-26 18:01:09,361][105586] KL-divergence is very high: 138.3419 [2023-12-26 18:01:09,366][105620] Updated weights for policy 1, policy_version 365214 (0.0007) [2023-12-26 18:01:09,376][105586] KL-divergence is very high: 101.8310 [2023-12-26 18:01:09,409][105586] KL-divergence is very high: 118.2852 [2023-12-26 18:01:09,416][105586] KL-divergence is very high: 108.2608 [2023-12-26 18:01:09,428][105586] KL-divergence is very high: 128.9790 [2023-12-26 18:01:09,433][105620] Updated weights for policy 1, policy_version 365224 (0.0007) [2023-12-26 18:01:09,770][105692] Updated weights for policy 0, policy_version 364426 (0.0008) [2023-12-26 18:01:09,832][105692] Updated weights for policy 0, policy_version 364436 (0.0008) [2023-12-26 18:01:09,896][105692] Updated weights for policy 0, policy_version 364446 (0.0010) [2023-12-26 18:01:09,966][105692] Updated weights for policy 0, policy_version 364456 (0.0008) [2023-12-26 18:01:10,228][105586] KL-divergence is very high: 110.1938 [2023-12-26 18:01:10,235][105586] KL-divergence is very high: 166.4711 [2023-12-26 18:01:10,243][105586] KL-divergence is very high: 231.7696 [2023-12-26 18:01:10,250][105586] KL-divergence is very high: 302.2869 [2023-12-26 18:01:10,256][105586] KL-divergence is very high: 371.0424 [2023-12-26 18:01:10,263][105586] KL-divergence is very high: 381.7798 [2023-12-26 18:01:10,268][105620] Updated weights for policy 1, policy_version 365234 (0.0009) [2023-12-26 18:01:10,269][105586] KL-divergence is very high: 417.1891 [2023-12-26 18:01:10,276][105586] KL-divergence is very high: 286.3406 [2023-12-26 18:01:10,284][105586] KL-divergence is very high: 427.1168 [2023-12-26 18:01:10,294][105586] KL-divergence is very high: 430.5268 [2023-12-26 18:01:10,302][105586] KL-divergence is very high: 321.9728 [2023-12-26 18:01:10,309][105586] KL-divergence is very high: 320.1154 [2023-12-26 18:01:10,316][105586] KL-divergence is very high: 276.3375 [2023-12-26 18:01:10,322][105586] KL-divergence is very high: 236.3837 [2023-12-26 18:01:10,328][105586] KL-divergence is very high: 157.9349 [2023-12-26 18:01:10,339][105586] KL-divergence is very high: 130.3862 [2023-12-26 18:01:10,341][105620] Updated weights for policy 1, policy_version 365244 (0.0009) [2023-12-26 18:01:10,392][105586] KL-divergence is very high: 158.1958 [2023-12-26 18:01:10,404][105620] Updated weights for policy 1, policy_version 365254 (0.0008) [2023-12-26 18:01:10,633][105692] Updated weights for policy 0, policy_version 364466 (0.0006) [2023-12-26 18:01:10,696][105692] Updated weights for policy 0, policy_version 364476 (0.0005) [2023-12-26 18:01:10,760][105692] Updated weights for policy 0, policy_version 364486 (0.0006) [2023-12-26 18:01:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 186834944. Throughput: 0: 9308.1, 1: 9912.6. Samples: 186843664. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-12-26 18:01:11,062][104569] Avg episode reward: [(0, '6938.000'), (1, '2793.464')] [2023-12-26 18:01:11,106][105586] KL-divergence is very high: 288.4282 [2023-12-26 18:01:11,114][105586] KL-divergence is very high: 185.0166 [2023-12-26 18:01:11,121][105586] KL-divergence is very high: 466.8123 [2023-12-26 18:01:11,162][105620] Updated weights for policy 1, policy_version 365264 (0.0008) [2023-12-26 18:01:11,172][105586] KL-divergence is very high: 105.0716 [2023-12-26 18:01:11,178][105586] KL-divergence is very high: 206.5037 [2023-12-26 18:01:11,185][105586] KL-divergence is very high: 135.5202 [2023-12-26 18:01:11,215][105586] KL-divergence is very high: 140.0416 [2023-12-26 18:01:11,221][105586] KL-divergence is very high: 148.5795 [2023-12-26 18:01:11,227][105586] KL-divergence is very high: 442.2756 [2023-12-26 18:01:11,229][105620] Updated weights for policy 1, policy_version 365274 (0.0008) [2023-12-26 18:01:11,261][105586] KL-divergence is very high: 137.2886 [2023-12-26 18:01:11,267][105586] KL-divergence is very high: 103.9999 [2023-12-26 18:01:11,272][105586] KL-divergence is very high: 314.5328 [2023-12-26 18:01:11,283][105620] Updated weights for policy 1, policy_version 365284 (0.0008) [2023-12-26 18:01:11,445][105692] Updated weights for policy 0, policy_version 364496 (0.0009) [2023-12-26 18:01:11,499][105692] Updated weights for policy 0, policy_version 364506 (0.0010) [2023-12-26 18:01:11,556][105692] Updated weights for policy 0, policy_version 364516 (0.0010) [2023-12-26 18:01:12,028][105620] Updated weights for policy 1, policy_version 365294 (0.0009) [2023-12-26 18:01:12,090][105620] Updated weights for policy 1, policy_version 365304 (0.0009) [2023-12-26 18:01:12,145][105620] Updated weights for policy 1, policy_version 365314 (0.0009) [2023-12-26 18:01:12,356][105692] Updated weights for policy 0, policy_version 364526 (0.0009) [2023-12-26 18:01:12,418][105692] Updated weights for policy 0, policy_version 364536 (0.0009) [2023-12-26 18:01:12,475][105692] Updated weights for policy 0, policy_version 364546 (0.0008) [2023-12-26 18:01:12,878][105620] Updated weights for policy 1, policy_version 365324 (0.0009) [2023-12-26 18:01:12,936][105620] Updated weights for policy 1, policy_version 365334 (0.0007) [2023-12-26 18:01:13,001][105620] Updated weights for policy 1, policy_version 365344 (0.0009) [2023-12-26 18:01:13,191][105692] Updated weights for policy 0, policy_version 364556 (0.0009) [2023-12-26 18:01:13,260][105692] Updated weights for policy 0, policy_version 364566 (0.0007) [2023-12-26 18:01:13,328][105692] Updated weights for policy 0, policy_version 364576 (0.0010) [2023-12-26 18:01:13,684][105586] KL-divergence is very high: 105.0209 [2023-12-26 18:01:13,685][105620] Updated weights for policy 1, policy_version 365354 (0.0009) [2023-12-26 18:01:13,689][105586] KL-divergence is very high: 111.3264 [2023-12-26 18:01:13,695][105586] KL-divergence is very high: 144.5734 [2023-12-26 18:01:13,704][105586] KL-divergence is very high: 156.6959 [2023-12-26 18:01:13,715][105586] KL-divergence is very high: 155.6576 [2023-12-26 18:01:13,721][105586] KL-divergence is very high: 152.6909 [2023-12-26 18:01:13,727][105586] KL-divergence is very high: 127.3823 [2023-12-26 18:01:13,732][105586] KL-divergence is very high: 104.9231 [2023-12-26 18:01:13,738][105586] KL-divergence is very high: 109.6953 [2023-12-26 18:01:13,739][105620] Updated weights for policy 1, policy_version 365364 (0.0005) [2023-12-26 18:01:13,765][105586] KL-divergence is very high: 171.6414 [2023-12-26 18:01:13,770][105586] KL-divergence is very high: 218.3716 [2023-12-26 18:01:13,788][105586] KL-divergence is very high: 267.0775 [2023-12-26 18:01:13,796][105620] Updated weights for policy 1, policy_version 365374 (0.0006) [2023-12-26 18:01:13,799][105586] KL-divergence is very high: 275.5246 [2023-12-26 18:01:13,814][105586] KL-divergence is very high: 257.2632 [2023-12-26 18:01:13,819][105586] KL-divergence is very high: 231.2075 [2023-12-26 18:01:13,834][105586] KL-divergence is very high: 183.2951 [2023-12-26 18:01:13,845][105586] KL-divergence is very high: 148.9772 [2023-12-26 18:01:13,851][105620] Updated weights for policy 1, policy_version 365384 (0.0006) [2023-12-26 18:01:14,018][105692] Updated weights for policy 0, policy_version 364586 (0.0010) [2023-12-26 18:01:14,079][105692] Updated weights for policy 0, policy_version 364596 (0.0010) [2023-12-26 18:01:14,136][105692] Updated weights for policy 0, policy_version 364606 (0.0009) [2023-12-26 18:01:14,197][105692] Updated weights for policy 0, policy_version 364616 (0.0006) [2023-12-26 18:01:14,380][105586] KL-divergence is very high: 156.1808 [2023-12-26 18:01:14,398][105586] KL-divergence is very high: 130.3227 [2023-12-26 18:01:14,419][105586] KL-divergence is very high: 120.2089 [2023-12-26 18:01:14,425][105586] KL-divergence is very high: 106.8673 [2023-12-26 18:01:14,435][105620] Updated weights for policy 1, policy_version 365394 (0.0006) [2023-12-26 18:01:14,493][105620] Updated weights for policy 1, policy_version 365404 (0.0006) [2023-12-26 18:01:14,512][105586] KL-divergence is very high: 115.1883 [2023-12-26 18:01:14,519][105586] KL-divergence is very high: 166.5307 [2023-12-26 18:01:14,525][105586] KL-divergence is very high: 183.6516 [2023-12-26 18:01:14,538][105586] KL-divergence is very high: 145.3174 [2023-12-26 18:01:14,555][105620] Updated weights for policy 1, policy_version 365414 (0.0007) [2023-12-26 18:01:14,907][105692] Updated weights for policy 0, policy_version 364626 (0.0011) [2023-12-26 18:01:14,977][105692] Updated weights for policy 0, policy_version 364636 (0.0011) [2023-12-26 18:01:15,046][105692] Updated weights for policy 0, policy_version 364646 (0.0011) [2023-12-26 18:01:15,223][105620] Updated weights for policy 1, policy_version 365424 (0.0011) [2023-12-26 18:01:15,256][105586] KL-divergence is very high: 100.6351 [2023-12-26 18:01:15,287][105620] Updated weights for policy 1, policy_version 365434 (0.0011) [2023-12-26 18:01:15,352][105620] Updated weights for policy 1, policy_version 365444 (0.0011) [2023-12-26 18:01:15,787][105692] Updated weights for policy 0, policy_version 364656 (0.0011) [2023-12-26 18:01:15,853][105692] Updated weights for policy 0, policy_version 364666 (0.0011) [2023-12-26 18:01:15,921][105692] Updated weights for policy 0, policy_version 364676 (0.0010) [2023-12-26 18:01:16,012][105620] Updated weights for policy 1, policy_version 365454 (0.0011) [2023-12-26 18:01:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 186933248. Throughput: 0: 9269.5, 1: 9789.5. Samples: 186902216. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-12-26 18:01:16,063][104569] Avg episode reward: [(0, '9358.981'), (1, '1793.818')] [2023-12-26 18:01:16,067][105620] Updated weights for policy 1, policy_version 365464 (0.0010) [2023-12-26 18:01:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000364680_93372416.pth... [2023-12-26 18:01:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000363592_93093888.pth [2023-12-26 18:01:16,118][105620] Updated weights for policy 1, policy_version 365474 (0.0010) [2023-12-26 18:01:16,119][105586] KL-divergence is very high: 108.0715 [2023-12-26 18:01:16,129][105586] KL-divergence is very high: 162.1474 [2023-12-26 18:01:16,135][105586] KL-divergence is very high: 232.7156 [2023-12-26 18:01:16,141][105586] KL-divergence is very high: 240.6356 [2023-12-26 18:01:16,146][105586] KL-divergence is very high: 243.8101 [2023-12-26 18:01:16,151][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000365480_93569024.pth... [2023-12-26 18:01:16,154][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000364296_93265920.pth [2023-12-26 18:01:16,648][105692] Updated weights for policy 0, policy_version 364686 (0.0007) [2023-12-26 18:01:16,703][105692] Updated weights for policy 0, policy_version 364696 (0.0005) [2023-12-26 18:01:16,762][105692] Updated weights for policy 0, policy_version 364706 (0.0009) [2023-12-26 18:01:16,778][105586] KL-divergence is very high: 138.4114 [2023-12-26 18:01:16,795][105586] KL-divergence is very high: 181.2753 [2023-12-26 18:01:16,811][105620] Updated weights for policy 1, policy_version 365484 (0.0009) [2023-12-26 18:01:16,852][105586] KL-divergence is very high: 107.4151 [2023-12-26 18:01:16,877][105620] Updated weights for policy 1, policy_version 365494 (0.0008) [2023-12-26 18:01:16,901][105586] KL-divergence is very high: 124.3871 [2023-12-26 18:01:16,930][105620] Updated weights for policy 1, policy_version 365504 (0.0007) [2023-12-26 18:01:16,941][105586] KL-divergence is very high: 100.1598 [2023-12-26 18:01:17,461][105692] Updated weights for policy 0, policy_version 364716 (0.0010) [2023-12-26 18:01:17,523][105692] Updated weights for policy 0, policy_version 364726 (0.0009) [2023-12-26 18:01:17,589][105620] Updated weights for policy 1, policy_version 365514 (0.0008) [2023-12-26 18:01:17,591][105692] Updated weights for policy 0, policy_version 364736 (0.0009) [2023-12-26 18:01:17,646][105620] Updated weights for policy 1, policy_version 365524 (0.0008) [2023-12-26 18:01:17,708][105620] Updated weights for policy 1, policy_version 365534 (0.0009) [2023-12-26 18:01:17,775][105620] Updated weights for policy 1, policy_version 365544 (0.0009) [2023-12-26 18:01:18,330][105692] Updated weights for policy 0, policy_version 364746 (0.0006) [2023-12-26 18:01:18,394][105692] Updated weights for policy 0, policy_version 364756 (0.0008) [2023-12-26 18:01:18,456][105692] Updated weights for policy 0, policy_version 364766 (0.0007) [2023-12-26 18:01:18,489][105620] Updated weights for policy 1, policy_version 365554 (0.0007) [2023-12-26 18:01:18,511][105692] Updated weights for policy 0, policy_version 364776 (0.0008) [2023-12-26 18:01:18,553][105620] Updated weights for policy 1, policy_version 365564 (0.0005) [2023-12-26 18:01:18,622][105620] Updated weights for policy 1, policy_version 365574 (0.0005) [2023-12-26 18:01:19,151][105620] Updated weights for policy 1, policy_version 365584 (0.0006) [2023-12-26 18:01:19,208][105620] Updated weights for policy 1, policy_version 365594 (0.0010) [2023-12-26 18:01:19,281][105620] Updated weights for policy 1, policy_version 365604 (0.0010) [2023-12-26 18:01:19,423][105692] Updated weights for policy 0, policy_version 364786 (0.0008) [2023-12-26 18:01:19,485][105692] Updated weights for policy 0, policy_version 364796 (0.0007) [2023-12-26 18:01:19,549][105692] Updated weights for policy 0, policy_version 364806 (0.0006) [2023-12-26 18:01:20,083][105620] Updated weights for policy 1, policy_version 365614 (0.0010) [2023-12-26 18:01:20,138][105620] Updated weights for policy 1, policy_version 365624 (0.0009) [2023-12-26 18:01:20,165][105692] Updated weights for policy 0, policy_version 364816 (0.0006) [2023-12-26 18:01:20,195][105620] Updated weights for policy 1, policy_version 365634 (0.0006) [2023-12-26 18:01:20,226][105692] Updated weights for policy 0, policy_version 364826 (0.0007) [2023-12-26 18:01:20,287][105692] Updated weights for policy 0, policy_version 364836 (0.0008) [2023-12-26 18:01:20,967][105620] Updated weights for policy 1, policy_version 365644 (0.0008) [2023-12-26 18:01:21,027][105620] Updated weights for policy 1, policy_version 365654 (0.0009) [2023-12-26 18:01:21,042][105692] Updated weights for policy 0, policy_version 364846 (0.0008) [2023-12-26 18:01:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 187023360. Throughput: 0: 9301.9, 1: 9777.4. Samples: 187019244. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-12-26 18:01:21,062][104569] Avg episode reward: [(0, '9359.277'), (1, '1798.908')] [2023-12-26 18:01:21,088][105620] Updated weights for policy 1, policy_version 365664 (0.0008) [2023-12-26 18:01:21,098][105692] Updated weights for policy 0, policy_version 364856 (0.0007) [2023-12-26 18:01:21,158][105692] Updated weights for policy 0, policy_version 364866 (0.0009) [2023-12-26 18:01:21,840][105692] Updated weights for policy 0, policy_version 364876 (0.0008) [2023-12-26 18:01:21,904][105692] Updated weights for policy 0, policy_version 364886 (0.0011) [2023-12-26 18:01:21,922][105620] Updated weights for policy 1, policy_version 365674 (0.0008) [2023-12-26 18:01:21,965][105692] Updated weights for policy 0, policy_version 364896 (0.0011) [2023-12-26 18:01:21,983][105620] Updated weights for policy 1, policy_version 365684 (0.0006) [2023-12-26 18:01:22,043][105620] Updated weights for policy 1, policy_version 365694 (0.0007) [2023-12-26 18:01:22,092][105620] Updated weights for policy 1, policy_version 365704 (0.0008) [2023-12-26 18:01:22,657][105692] Updated weights for policy 0, policy_version 364906 (0.0010) [2023-12-26 18:01:22,716][105692] Updated weights for policy 0, policy_version 364916 (0.0007) [2023-12-26 18:01:22,777][105692] Updated weights for policy 0, policy_version 364926 (0.0007) [2023-12-26 18:01:22,843][105692] Updated weights for policy 0, policy_version 364936 (0.0009) [2023-12-26 18:01:22,891][105620] Updated weights for policy 1, policy_version 365714 (0.0009) [2023-12-26 18:01:22,951][105620] Updated weights for policy 1, policy_version 365724 (0.0010) [2023-12-26 18:01:23,005][105620] Updated weights for policy 1, policy_version 365734 (0.0010) [2023-12-26 18:01:23,399][105692] Updated weights for policy 0, policy_version 364946 (0.0005) [2023-12-26 18:01:23,465][105692] Updated weights for policy 0, policy_version 364956 (0.0007) [2023-12-26 18:01:23,521][105692] Updated weights for policy 0, policy_version 364966 (0.0007) [2023-12-26 18:01:23,933][105620] Updated weights for policy 1, policy_version 365744 (0.0009) [2023-12-26 18:01:24,002][105620] Updated weights for policy 1, policy_version 365754 (0.0010) [2023-12-26 18:01:24,050][105692] Updated weights for policy 0, policy_version 364976 (0.0007) [2023-12-26 18:01:24,064][105620] Updated weights for policy 1, policy_version 365764 (0.0007) [2023-12-26 18:01:24,104][105692] Updated weights for policy 0, policy_version 364986 (0.0007) [2023-12-26 18:01:24,153][105692] Updated weights for policy 0, policy_version 364996 (0.0009) [2023-12-26 18:01:24,746][105692] Updated weights for policy 0, policy_version 365006 (0.0005) [2023-12-26 18:01:24,802][105692] Updated weights for policy 0, policy_version 365016 (0.0005) [2023-12-26 18:01:24,860][105692] Updated weights for policy 0, policy_version 365026 (0.0005) [2023-12-26 18:01:24,882][105620] Updated weights for policy 1, policy_version 365774 (0.0008) [2023-12-26 18:01:24,931][105620] Updated weights for policy 1, policy_version 365784 (0.0009) [2023-12-26 18:01:24,996][105620] Updated weights for policy 1, policy_version 365794 (0.0009) [2023-12-26 18:01:25,441][105692] Updated weights for policy 0, policy_version 365036 (0.0006) [2023-12-26 18:01:25,493][105692] Updated weights for policy 0, policy_version 365046 (0.0005) [2023-12-26 18:01:25,538][105692] Updated weights for policy 0, policy_version 365056 (0.0005) [2023-12-26 18:01:25,806][105620] Updated weights for policy 1, policy_version 365804 (0.0009) [2023-12-26 18:01:25,870][105620] Updated weights for policy 1, policy_version 365814 (0.0009) [2023-12-26 18:01:25,934][105620] Updated weights for policy 1, policy_version 365824 (0.0009) [2023-12-26 18:01:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 187129856. Throughput: 0: 9475.3, 1: 9707.1. Samples: 187135796. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-12-26 18:01:26,063][104569] Avg episode reward: [(0, '9359.262'), (1, '6643.470')] [2023-12-26 18:01:26,098][105692] Updated weights for policy 0, policy_version 365066 (0.0006) [2023-12-26 18:01:26,150][105692] Updated weights for policy 0, policy_version 365076 (0.0007) [2023-12-26 18:01:26,196][105692] Updated weights for policy 0, policy_version 365086 (0.0005) [2023-12-26 18:01:26,245][105692] Updated weights for policy 0, policy_version 365096 (0.0005) [2023-12-26 18:01:26,780][105692] Updated weights for policy 0, policy_version 365106 (0.0005) [2023-12-26 18:01:26,803][105620] Updated weights for policy 1, policy_version 365834 (0.0010) [2023-12-26 18:01:26,831][105692] Updated weights for policy 0, policy_version 365116 (0.0005) [2023-12-26 18:01:26,867][105620] Updated weights for policy 1, policy_version 365844 (0.0008) [2023-12-26 18:01:26,879][105692] Updated weights for policy 0, policy_version 365126 (0.0005) [2023-12-26 18:01:26,930][105620] Updated weights for policy 1, policy_version 365854 (0.0008) [2023-12-26 18:01:26,987][105620] Updated weights for policy 1, policy_version 365864 (0.0010) [2023-12-26 18:01:27,512][105692] Updated weights for policy 0, policy_version 365136 (0.0006) [2023-12-26 18:01:27,565][105692] Updated weights for policy 0, policy_version 365146 (0.0005) [2023-12-26 18:01:27,610][105692] Updated weights for policy 0, policy_version 365156 (0.0005) [2023-12-26 18:01:27,801][105620] Updated weights for policy 1, policy_version 365874 (0.0009) [2023-12-26 18:01:27,853][105620] Updated weights for policy 1, policy_version 365885 (0.0010) [2023-12-26 18:01:27,909][105620] Updated weights for policy 1, policy_version 365895 (0.0009) [2023-12-26 18:01:28,147][105692] Updated weights for policy 0, policy_version 365166 (0.0007) [2023-12-26 18:01:28,200][105692] Updated weights for policy 0, policy_version 365176 (0.0008) [2023-12-26 18:01:28,263][105692] Updated weights for policy 0, policy_version 365186 (0.0009) [2023-12-26 18:01:28,782][105620] Updated weights for policy 1, policy_version 365905 (0.0010) [2023-12-26 18:01:28,832][105620] Updated weights for policy 1, policy_version 365915 (0.0009) [2023-12-26 18:01:28,847][105692] Updated weights for policy 0, policy_version 365196 (0.0008) [2023-12-26 18:01:28,884][105620] Updated weights for policy 1, policy_version 365925 (0.0008) [2023-12-26 18:01:28,904][105692] Updated weights for policy 0, policy_version 365206 (0.0006) [2023-12-26 18:01:28,948][105692] Updated weights for policy 0, policy_version 365216 (0.0009) [2023-12-26 18:01:29,582][105620] Updated weights for policy 1, policy_version 365935 (0.0009) [2023-12-26 18:01:29,629][105620] Updated weights for policy 1, policy_version 365945 (0.0008) [2023-12-26 18:01:29,675][105620] Updated weights for policy 1, policy_version 365955 (0.0009) [2023-12-26 18:01:29,705][105692] Updated weights for policy 0, policy_version 365226 (0.0009) [2023-12-26 18:01:29,756][105692] Updated weights for policy 0, policy_version 365236 (0.0008) [2023-12-26 18:01:29,805][105692] Updated weights for policy 0, policy_version 365246 (0.0005) [2023-12-26 18:01:29,873][105692] Updated weights for policy 0, policy_version 365256 (0.0009) [2023-12-26 18:01:30,464][105620] Updated weights for policy 1, policy_version 365965 (0.0008) [2023-12-26 18:01:30,515][105620] Updated weights for policy 1, policy_version 365975 (0.0010) [2023-12-26 18:01:30,542][105692] Updated weights for policy 0, policy_version 365266 (0.0010) [2023-12-26 18:01:30,561][105620] Updated weights for policy 1, policy_version 365985 (0.0010) [2023-12-26 18:01:30,594][105692] Updated weights for policy 0, policy_version 365276 (0.0010) [2023-12-26 18:01:30,651][105692] Updated weights for policy 0, policy_version 365286 (0.0006) [2023-12-26 18:01:31,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 187228160. Throughput: 0: 9637.5, 1: 9667.7. Samples: 187196192. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-12-26 18:01:31,063][104569] Avg episode reward: [(0, '9358.861'), (1, '8488.188')] [2023-12-26 18:01:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000365992_93700096.pth... [2023-12-26 18:01:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000365288_93528064.pth... [2023-12-26 18:01:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000364104_93224960.pth [2023-12-26 18:01:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000364904_93421568.pth [2023-12-26 18:01:31,250][105692] Updated weights for policy 0, policy_version 365296 (0.0008) [2023-12-26 18:01:31,308][105692] Updated weights for policy 0, policy_version 365306 (0.0009) [2023-12-26 18:01:31,315][105620] Updated weights for policy 1, policy_version 365995 (0.0010) [2023-12-26 18:01:31,362][105692] Updated weights for policy 0, policy_version 365316 (0.0009) [2023-12-26 18:01:31,381][105620] Updated weights for policy 1, policy_version 366005 (0.0009) [2023-12-26 18:01:31,440][105620] Updated weights for policy 1, policy_version 366015 (0.0009) [2023-12-26 18:01:32,250][105692] Updated weights for policy 0, policy_version 365326 (0.0007) [2023-12-26 18:01:32,272][105620] Updated weights for policy 1, policy_version 366025 (0.0009) [2023-12-26 18:01:32,309][105692] Updated weights for policy 0, policy_version 365336 (0.0007) [2023-12-26 18:01:32,331][105620] Updated weights for policy 1, policy_version 366035 (0.0010) [2023-12-26 18:01:32,362][105692] Updated weights for policy 0, policy_version 365346 (0.0007) [2023-12-26 18:01:32,395][105620] Updated weights for policy 1, policy_version 366045 (0.0009) [2023-12-26 18:01:32,459][105620] Updated weights for policy 1, policy_version 366055 (0.0008) [2023-12-26 18:01:33,031][105692] Updated weights for policy 0, policy_version 365356 (0.0009) [2023-12-26 18:01:33,074][105620] Updated weights for policy 1, policy_version 366065 (0.0006) [2023-12-26 18:01:33,089][105692] Updated weights for policy 0, policy_version 365366 (0.0010) [2023-12-26 18:01:33,131][105620] Updated weights for policy 1, policy_version 366075 (0.0005) [2023-12-26 18:01:33,132][105692] Updated weights for policy 0, policy_version 365376 (0.0006) [2023-12-26 18:01:33,186][105620] Updated weights for policy 1, policy_version 366085 (0.0005) [2023-12-26 18:01:33,737][105692] Updated weights for policy 0, policy_version 365386 (0.0008) [2023-12-26 18:01:33,758][105620] Updated weights for policy 1, policy_version 366095 (0.0005) [2023-12-26 18:01:33,785][105692] Updated weights for policy 0, policy_version 365396 (0.0005) [2023-12-26 18:01:33,813][105620] Updated weights for policy 1, policy_version 366105 (0.0006) [2023-12-26 18:01:33,833][105692] Updated weights for policy 0, policy_version 365406 (0.0008) [2023-12-26 18:01:33,867][105620] Updated weights for policy 1, policy_version 366115 (0.0005) [2023-12-26 18:01:33,892][105692] Updated weights for policy 0, policy_version 365416 (0.0010) [2023-12-26 18:01:34,478][105620] Updated weights for policy 1, policy_version 366125 (0.0007) [2023-12-26 18:01:34,536][105620] Updated weights for policy 1, policy_version 366135 (0.0008) [2023-12-26 18:01:34,546][105692] Updated weights for policy 0, policy_version 365426 (0.0008) [2023-12-26 18:01:34,598][105620] Updated weights for policy 1, policy_version 366145 (0.0006) [2023-12-26 18:01:34,609][105692] Updated weights for policy 0, policy_version 365436 (0.0008) [2023-12-26 18:01:34,664][105692] Updated weights for policy 0, policy_version 365446 (0.0010) [2023-12-26 18:01:35,173][105620] Updated weights for policy 1, policy_version 366155 (0.0007) [2023-12-26 18:01:35,234][105620] Updated weights for policy 1, policy_version 366165 (0.0008) [2023-12-26 18:01:35,304][105620] Updated weights for policy 1, policy_version 366175 (0.0009) [2023-12-26 18:01:35,379][105692] Updated weights for policy 0, policy_version 365456 (0.0006) [2023-12-26 18:01:35,422][105692] Updated weights for policy 0, policy_version 365466 (0.0005) [2023-12-26 18:01:35,489][105692] Updated weights for policy 0, policy_version 365476 (0.0005) [2023-12-26 18:01:36,054][105620] Updated weights for policy 1, policy_version 366185 (0.0009) [2023-12-26 18:01:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 187326464. Throughput: 0: 9780.3, 1: 9676.4. Samples: 187319172. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-12-26 18:01:36,062][104569] Avg episode reward: [(0, '9358.021'), (1, '8185.832')] [2023-12-26 18:01:36,104][105692] Updated weights for policy 0, policy_version 365486 (0.0007) [2023-12-26 18:01:36,118][105620] Updated weights for policy 1, policy_version 366195 (0.0008) [2023-12-26 18:01:36,174][105692] Updated weights for policy 0, policy_version 365496 (0.0008) [2023-12-26 18:01:36,180][105620] Updated weights for policy 1, policy_version 366205 (0.0008) [2023-12-26 18:01:36,230][105692] Updated weights for policy 0, policy_version 365506 (0.0008) [2023-12-26 18:01:36,241][105620] Updated weights for policy 1, policy_version 366215 (0.0006) [2023-12-26 18:01:36,920][105692] Updated weights for policy 0, policy_version 365516 (0.0009) [2023-12-26 18:01:36,977][105692] Updated weights for policy 0, policy_version 365526 (0.0007) [2023-12-26 18:01:36,987][105620] Updated weights for policy 1, policy_version 366225 (0.0008) [2023-12-26 18:01:37,034][105692] Updated weights for policy 0, policy_version 365536 (0.0007) [2023-12-26 18:01:37,044][105620] Updated weights for policy 1, policy_version 366235 (0.0006) [2023-12-26 18:01:37,107][105620] Updated weights for policy 1, policy_version 366245 (0.0008) [2023-12-26 18:01:37,683][105692] Updated weights for policy 0, policy_version 365546 (0.0007) [2023-12-26 18:01:37,747][105692] Updated weights for policy 0, policy_version 365556 (0.0011) [2023-12-26 18:01:37,800][105692] Updated weights for policy 0, policy_version 365566 (0.0011) [2023-12-26 18:01:37,854][105692] Updated weights for policy 0, policy_version 365576 (0.0011) [2023-12-26 18:01:37,941][105620] Updated weights for policy 1, policy_version 366255 (0.0007) [2023-12-26 18:01:38,004][105620] Updated weights for policy 1, policy_version 366265 (0.0008) [2023-12-26 18:01:38,062][105620] Updated weights for policy 1, policy_version 366275 (0.0010) [2023-12-26 18:01:38,656][105692] Updated weights for policy 0, policy_version 365586 (0.0009) [2023-12-26 18:01:38,666][105620] Updated weights for policy 1, policy_version 366285 (0.0008) [2023-12-26 18:01:38,729][105620] Updated weights for policy 1, policy_version 366295 (0.0005) [2023-12-26 18:01:38,741][105692] Updated weights for policy 0, policy_version 365596 (0.0007) [2023-12-26 18:01:38,796][105620] Updated weights for policy 1, policy_version 366305 (0.0007) [2023-12-26 18:01:38,805][105692] Updated weights for policy 0, policy_version 365606 (0.0005) [2023-12-26 18:01:39,393][105692] Updated weights for policy 0, policy_version 365616 (0.0007) [2023-12-26 18:01:39,460][105692] Updated weights for policy 0, policy_version 365626 (0.0008) [2023-12-26 18:01:39,469][105620] Updated weights for policy 1, policy_version 366315 (0.0010) [2023-12-26 18:01:39,527][105692] Updated weights for policy 0, policy_version 365636 (0.0006) [2023-12-26 18:01:39,529][105620] Updated weights for policy 1, policy_version 366325 (0.0010) [2023-12-26 18:01:39,586][105620] Updated weights for policy 1, policy_version 366335 (0.0011) [2023-12-26 18:01:40,239][105692] Updated weights for policy 0, policy_version 365646 (0.0006) [2023-12-26 18:01:40,305][105692] Updated weights for policy 0, policy_version 365656 (0.0006) [2023-12-26 18:01:40,367][105692] Updated weights for policy 0, policy_version 365666 (0.0006) [2023-12-26 18:01:40,374][105620] Updated weights for policy 1, policy_version 366345 (0.0011) [2023-12-26 18:01:40,426][105620] Updated weights for policy 1, policy_version 366355 (0.0010) [2023-12-26 18:01:40,482][105620] Updated weights for policy 1, policy_version 366365 (0.0011) [2023-12-26 18:01:40,539][105620] Updated weights for policy 1, policy_version 366375 (0.0011) [2023-12-26 18:01:41,030][105692] Updated weights for policy 0, policy_version 365676 (0.0007) [2023-12-26 18:01:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 187424768. Throughput: 0: 9949.7, 1: 9599.6. Samples: 187437748. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-12-26 18:01:41,062][104569] Avg episode reward: [(0, '9355.903'), (1, '8170.523')] [2023-12-26 18:01:41,089][105692] Updated weights for policy 0, policy_version 365686 (0.0008) [2023-12-26 18:01:41,146][105692] Updated weights for policy 0, policy_version 365696 (0.0009) [2023-12-26 18:01:41,290][105620] Updated weights for policy 1, policy_version 366385 (0.0009) [2023-12-26 18:01:41,357][105620] Updated weights for policy 1, policy_version 366395 (0.0010) [2023-12-26 18:01:41,429][105620] Updated weights for policy 1, policy_version 366405 (0.0009) [2023-12-26 18:01:41,911][105692] Updated weights for policy 0, policy_version 365706 (0.0008) [2023-12-26 18:01:41,966][105692] Updated weights for policy 0, policy_version 365716 (0.0009) [2023-12-26 18:01:42,029][105692] Updated weights for policy 0, policy_version 365726 (0.0010) [2023-12-26 18:01:42,090][105692] Updated weights for policy 0, policy_version 365736 (0.0009) [2023-12-26 18:01:42,168][105620] Updated weights for policy 1, policy_version 366415 (0.0008) [2023-12-26 18:01:42,223][105620] Updated weights for policy 1, policy_version 366425 (0.0009) [2023-12-26 18:01:42,280][105620] Updated weights for policy 1, policy_version 366435 (0.0009) [2023-12-26 18:01:42,851][105692] Updated weights for policy 0, policy_version 365746 (0.0010) [2023-12-26 18:01:42,896][105692] Updated weights for policy 0, policy_version 365756 (0.0010) [2023-12-26 18:01:42,944][105692] Updated weights for policy 0, policy_version 365766 (0.0010) [2023-12-26 18:01:43,057][105620] Updated weights for policy 1, policy_version 366445 (0.0008) [2023-12-26 18:01:43,125][105620] Updated weights for policy 1, policy_version 366455 (0.0008) [2023-12-26 18:01:43,196][105620] Updated weights for policy 1, policy_version 366465 (0.0010) [2023-12-26 18:01:43,553][105692] Updated weights for policy 0, policy_version 365776 (0.0006) [2023-12-26 18:01:43,622][105692] Updated weights for policy 0, policy_version 365786 (0.0006) [2023-12-26 18:01:43,681][105692] Updated weights for policy 0, policy_version 365796 (0.0010) [2023-12-26 18:01:43,979][105620] Updated weights for policy 1, policy_version 366475 (0.0008) [2023-12-26 18:01:44,041][105620] Updated weights for policy 1, policy_version 366485 (0.0008) [2023-12-26 18:01:44,095][105620] Updated weights for policy 1, policy_version 366495 (0.0010) [2023-12-26 18:01:44,256][105692] Updated weights for policy 0, policy_version 365806 (0.0007) [2023-12-26 18:01:44,307][105692] Updated weights for policy 0, policy_version 365816 (0.0005) [2023-12-26 18:01:44,351][105692] Updated weights for policy 0, policy_version 365826 (0.0005) [2023-12-26 18:01:44,869][105620] Updated weights for policy 1, policy_version 366505 (0.0008) [2023-12-26 18:01:44,927][105620] Updated weights for policy 1, policy_version 366515 (0.0008) [2023-12-26 18:01:44,933][105692] Updated weights for policy 0, policy_version 365836 (0.0010) [2023-12-26 18:01:44,975][105620] Updated weights for policy 1, policy_version 366525 (0.0006) [2023-12-26 18:01:44,999][105692] Updated weights for policy 0, policy_version 365846 (0.0010) [2023-12-26 18:01:45,022][105620] Updated weights for policy 1, policy_version 366535 (0.0008) [2023-12-26 18:01:45,059][105692] Updated weights for policy 0, policy_version 365856 (0.0011) [2023-12-26 18:01:45,730][105692] Updated weights for policy 0, policy_version 365866 (0.0007) [2023-12-26 18:01:45,775][105692] Updated weights for policy 0, policy_version 365876 (0.0005) [2023-12-26 18:01:45,821][105692] Updated weights for policy 0, policy_version 365886 (0.0005) [2023-12-26 18:01:45,850][105620] Updated weights for policy 1, policy_version 366545 (0.0008) [2023-12-26 18:01:45,865][105692] Updated weights for policy 0, policy_version 365896 (0.0005) [2023-12-26 18:01:45,910][105620] Updated weights for policy 1, policy_version 366555 (0.0009) [2023-12-26 18:01:45,960][105620] Updated weights for policy 1, policy_version 366565 (0.0009) [2023-12-26 18:01:46,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 187531264. Throughput: 0: 9994.1, 1: 9578.2. Samples: 187494448. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-12-26 18:01:46,062][104569] Avg episode reward: [(0, '9355.560'), (1, '7822.816')] [2023-12-26 18:01:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000366568_93847552.pth... [2023-12-26 18:01:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000365896_93683712.pth... [2023-12-26 18:01:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000365480_93569024.pth [2023-12-26 18:01:46,071][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_000366568_93847552.pth [2023-12-26 18:01:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000364680_93372416.pth [2023-12-26 18:01:46,072][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_000365896_93683712.pth [2023-12-26 18:01:46,551][105692] Updated weights for policy 0, policy_version 365906 (0.0010) [2023-12-26 18:01:46,601][105692] Updated weights for policy 0, policy_version 365916 (0.0010) [2023-12-26 18:01:46,653][105692] Updated weights for policy 0, policy_version 365926 (0.0010) [2023-12-26 18:01:46,741][105620] Updated weights for policy 1, policy_version 366575 (0.0008) [2023-12-26 18:01:46,790][105620] Updated weights for policy 1, policy_version 366585 (0.0008) [2023-12-26 18:01:46,844][105620] Updated weights for policy 1, policy_version 366596 (0.0009) [2023-12-26 18:01:47,280][105692] Updated weights for policy 0, policy_version 365936 (0.0006) [2023-12-26 18:01:47,338][105692] Updated weights for policy 0, policy_version 365946 (0.0005) [2023-12-26 18:01:47,396][105692] Updated weights for policy 0, policy_version 365956 (0.0005) [2023-12-26 18:01:47,710][105620] Updated weights for policy 1, policy_version 366606 (0.0009) [2023-12-26 18:01:47,767][105620] Updated weights for policy 1, policy_version 366616 (0.0008) [2023-12-26 18:01:47,819][105620] Updated weights for policy 1, policy_version 366626 (0.0009) [2023-12-26 18:01:48,005][105692] Updated weights for policy 0, policy_version 365966 (0.0006) [2023-12-26 18:01:48,068][105692] Updated weights for policy 0, policy_version 365976 (0.0006) [2023-12-26 18:01:48,123][105692] Updated weights for policy 0, policy_version 365986 (0.0008) [2023-12-26 18:01:48,662][105620] Updated weights for policy 1, policy_version 366636 (0.0007) [2023-12-26 18:01:48,729][105620] Updated weights for policy 1, policy_version 366646 (0.0008) [2023-12-26 18:01:48,777][105692] Updated weights for policy 0, policy_version 365996 (0.0008) [2023-12-26 18:01:48,787][105620] Updated weights for policy 1, policy_version 366656 (0.0009) [2023-12-26 18:01:48,826][105692] Updated weights for policy 0, policy_version 366006 (0.0006) [2023-12-26 18:01:48,887][105692] Updated weights for policy 0, policy_version 366016 (0.0008) [2023-12-26 18:01:49,479][105620] Updated weights for policy 1, policy_version 366666 (0.0008) [2023-12-26 18:01:49,538][105620] Updated weights for policy 1, policy_version 366676 (0.0008) [2023-12-26 18:01:49,584][105692] Updated weights for policy 0, policy_version 366026 (0.0010) [2023-12-26 18:01:49,601][105620] Updated weights for policy 1, policy_version 366686 (0.0007) [2023-12-26 18:01:49,632][105692] Updated weights for policy 0, policy_version 366036 (0.0010) [2023-12-26 18:01:49,659][105620] Updated weights for policy 1, policy_version 366696 (0.0006) [2023-12-26 18:01:49,681][105692] Updated weights for policy 0, policy_version 366046 (0.0010) [2023-12-26 18:01:49,726][105692] Updated weights for policy 0, policy_version 366056 (0.0010) [2023-12-26 18:01:50,419][105620] Updated weights for policy 1, policy_version 366706 (0.0008) [2023-12-26 18:01:50,483][105620] Updated weights for policy 1, policy_version 366716 (0.0007) [2023-12-26 18:01:50,512][105692] Updated weights for policy 0, policy_version 366066 (0.0009) [2023-12-26 18:01:50,547][105620] Updated weights for policy 1, policy_version 366726 (0.0007) [2023-12-26 18:01:50,575][105692] Updated weights for policy 0, policy_version 366076 (0.0007) [2023-12-26 18:01:50,632][105692] Updated weights for policy 0, policy_version 366086 (0.0009) [2023-12-26 18:01:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 187621376. Throughput: 0: 10244.3, 1: 9397.6. Samples: 187613320. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) [2023-12-26 18:01:51,063][104569] Avg episode reward: [(0, '9356.331'), (1, '7687.221')] [2023-12-26 18:01:51,298][105620] Updated weights for policy 1, policy_version 366736 (0.0008) [2023-12-26 18:01:51,370][105620] Updated weights for policy 1, policy_version 366746 (0.0009) [2023-12-26 18:01:51,395][105692] Updated weights for policy 0, policy_version 366096 (0.0009) [2023-12-26 18:01:51,430][105620] Updated weights for policy 1, policy_version 366756 (0.0009) [2023-12-26 18:01:51,442][105692] Updated weights for policy 0, policy_version 366106 (0.0008) [2023-12-26 18:01:51,495][105692] Updated weights for policy 0, policy_version 366116 (0.0008) [2023-12-26 18:01:52,168][105620] Updated weights for policy 1, policy_version 366766 (0.0009) [2023-12-26 18:01:52,222][105620] Updated weights for policy 1, policy_version 366776 (0.0009) [2023-12-26 18:01:52,284][105620] Updated weights for policy 1, policy_version 366786 (0.0009) [2023-12-26 18:01:52,303][105692] Updated weights for policy 0, policy_version 366126 (0.0008) [2023-12-26 18:01:52,361][105692] Updated weights for policy 0, policy_version 366136 (0.0009) [2023-12-26 18:01:52,427][105692] Updated weights for policy 0, policy_version 366146 (0.0009) [2023-12-26 18:01:52,950][105620] Updated weights for policy 1, policy_version 366796 (0.0007) [2023-12-26 18:01:53,006][105620] Updated weights for policy 1, policy_version 366806 (0.0009) [2023-12-26 18:01:53,061][105620] Updated weights for policy 1, policy_version 366816 (0.0009) [2023-12-26 18:01:53,099][105692] Updated weights for policy 0, policy_version 366156 (0.0008) [2023-12-26 18:01:53,149][105692] Updated weights for policy 0, policy_version 366166 (0.0005) [2023-12-26 18:01:53,195][105692] Updated weights for policy 0, policy_version 366176 (0.0005) [2023-12-26 18:01:53,763][105692] Updated weights for policy 0, policy_version 366186 (0.0006) [2023-12-26 18:01:53,817][105692] Updated weights for policy 0, policy_version 366196 (0.0009) [2023-12-26 18:01:53,866][105692] Updated weights for policy 0, policy_version 366206 (0.0008) [2023-12-26 18:01:53,885][105620] Updated weights for policy 1, policy_version 366826 (0.0008) [2023-12-26 18:01:53,912][105692] Updated weights for policy 0, policy_version 366216 (0.0007) [2023-12-26 18:01:53,939][105620] Updated weights for policy 1, policy_version 366836 (0.0009) [2023-12-26 18:01:53,992][105620] Updated weights for policy 1, policy_version 366846 (0.0008) [2023-12-26 18:01:54,042][105620] Updated weights for policy 1, policy_version 366856 (0.0009) [2023-12-26 18:01:54,688][105692] Updated weights for policy 0, policy_version 366226 (0.0009) [2023-12-26 18:01:54,743][105692] Updated weights for policy 0, policy_version 366236 (0.0009) [2023-12-26 18:01:54,793][105692] Updated weights for policy 0, policy_version 366246 (0.0007) [2023-12-26 18:01:54,803][105620] Updated weights for policy 1, policy_version 366866 (0.0008) [2023-12-26 18:01:54,859][105620] Updated weights for policy 1, policy_version 366876 (0.0009) [2023-12-26 18:01:54,908][105620] Updated weights for policy 1, policy_version 366886 (0.0008) [2023-12-26 18:01:55,568][105692] Updated weights for policy 0, policy_version 366256 (0.0009) [2023-12-26 18:01:55,629][105692] Updated weights for policy 0, policy_version 366266 (0.0009) [2023-12-26 18:01:55,683][105620] Updated weights for policy 1, policy_version 366896 (0.0009) [2023-12-26 18:01:55,689][105692] Updated weights for policy 0, policy_version 366276 (0.0007) [2023-12-26 18:01:55,743][105620] Updated weights for policy 1, policy_version 366906 (0.0008) [2023-12-26 18:01:55,807][105620] Updated weights for policy 1, policy_version 366916 (0.0009) [2023-12-26 18:01:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.4, 300 sec: 19438.6). Total num frames: 187719680. Throughput: 0: 10218.4, 1: 9404.3. Samples: 187726684. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) [2023-12-26 18:01:56,062][104569] Avg episode reward: [(0, '9355.912'), (1, '6005.161')] [2023-12-26 18:01:56,422][105692] Updated weights for policy 0, policy_version 366286 (0.0005) [2023-12-26 18:01:56,475][105692] Updated weights for policy 0, policy_version 366296 (0.0005) [2023-12-26 18:01:56,526][105692] Updated weights for policy 0, policy_version 366306 (0.0005) [2023-12-26 18:01:56,549][105620] Updated weights for policy 1, policy_version 366926 (0.0009) [2023-12-26 18:01:56,598][105620] Updated weights for policy 1, policy_version 366936 (0.0009) [2023-12-26 18:01:56,653][105620] Updated weights for policy 1, policy_version 366947 (0.0007) [2023-12-26 18:01:57,209][105620] Updated weights for policy 1, policy_version 366957 (0.0005) [2023-12-26 18:01:57,264][105692] Updated weights for policy 0, policy_version 366316 (0.0007) [2023-12-26 18:01:57,268][105620] Updated weights for policy 1, policy_version 366967 (0.0006) [2023-12-26 18:01:57,329][105692] Updated weights for policy 0, policy_version 366326 (0.0008) [2023-12-26 18:01:57,332][105620] Updated weights for policy 1, policy_version 366977 (0.0005) [2023-12-26 18:01:57,376][105692] Updated weights for policy 0, policy_version 366336 (0.0005) [2023-12-26 18:01:57,930][105692] Updated weights for policy 0, policy_version 366346 (0.0005) [2023-12-26 18:01:57,984][105692] Updated weights for policy 0, policy_version 366356 (0.0005) [2023-12-26 18:01:58,044][105692] Updated weights for policy 0, policy_version 366366 (0.0010) [2023-12-26 18:01:58,050][105620] Updated weights for policy 1, policy_version 366987 (0.0005) [2023-12-26 18:01:58,101][105692] Updated weights for policy 0, policy_version 366376 (0.0010) [2023-12-26 18:01:58,109][105620] Updated weights for policy 1, policy_version 366997 (0.0006) [2023-12-26 18:01:58,177][105620] Updated weights for policy 1, policy_version 367007 (0.0008) [2023-12-26 18:01:58,972][105692] Updated weights for policy 0, policy_version 366386 (0.0009) [2023-12-26 18:01:58,985][105620] Updated weights for policy 1, policy_version 367017 (0.0009) [2023-12-26 18:01:59,033][105692] Updated weights for policy 0, policy_version 366396 (0.0007) [2023-12-26 18:01:59,051][105620] Updated weights for policy 1, policy_version 367027 (0.0009) [2023-12-26 18:01:59,090][105692] Updated weights for policy 0, policy_version 366406 (0.0006) [2023-12-26 18:01:59,111][105620] Updated weights for policy 1, policy_version 367037 (0.0008) [2023-12-26 18:01:59,173][105620] Updated weights for policy 1, policy_version 367047 (0.0008) [2023-12-26 18:01:59,876][105692] Updated weights for policy 0, policy_version 366416 (0.0008) [2023-12-26 18:01:59,947][105692] Updated weights for policy 0, policy_version 366426 (0.0008) [2023-12-26 18:01:59,948][105620] Updated weights for policy 1, policy_version 367057 (0.0007) [2023-12-26 18:01:59,999][105620] Updated weights for policy 1, policy_version 367067 (0.0007) [2023-12-26 18:02:00,005][105692] Updated weights for policy 0, policy_version 366436 (0.0008) [2023-12-26 18:02:00,056][105620] Updated weights for policy 1, policy_version 367077 (0.0008) [2023-12-26 18:02:00,697][105692] Updated weights for policy 0, policy_version 366446 (0.0009) [2023-12-26 18:02:00,742][105692] Updated weights for policy 0, policy_version 366456 (0.0010) [2023-12-26 18:02:00,793][105692] Updated weights for policy 0, policy_version 366466 (0.0010) [2023-12-26 18:02:00,805][105620] Updated weights for policy 1, policy_version 367087 (0.0006) [2023-12-26 18:02:00,861][105620] Updated weights for policy 1, policy_version 367097 (0.0005) [2023-12-26 18:02:00,909][105620] Updated weights for policy 1, policy_version 367107 (0.0005) [2023-12-26 18:02:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 187817984. Throughput: 0: 10245.0, 1: 9401.4. Samples: 187786304. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) [2023-12-26 18:02:01,063][104569] Avg episode reward: [(0, '9355.519'), (1, '5732.127')] [2023-12-26 18:02:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000366472_93831168.pth... [2023-12-26 18:02:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000367112_93986816.pth... [2023-12-26 18:02:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000365288_93528064.pth [2023-12-26 18:02:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000365992_93700096.pth [2023-12-26 18:02:01,517][105620] Updated weights for policy 1, policy_version 367117 (0.0005) [2023-12-26 18:02:01,580][105620] Updated weights for policy 1, policy_version 367127 (0.0010) [2023-12-26 18:02:01,584][105692] Updated weights for policy 0, policy_version 366476 (0.0010) [2023-12-26 18:02:01,644][105620] Updated weights for policy 1, policy_version 367137 (0.0010) [2023-12-26 18:02:01,648][105692] Updated weights for policy 0, policy_version 366486 (0.0011) [2023-12-26 18:02:01,709][105692] Updated weights for policy 0, policy_version 366496 (0.0010) [2023-12-26 18:02:02,198][105620] Updated weights for policy 1, policy_version 367147 (0.0010) [2023-12-26 18:02:02,264][105620] Updated weights for policy 1, policy_version 367157 (0.0011) [2023-12-26 18:02:02,328][105620] Updated weights for policy 1, policy_version 367167 (0.0008) [2023-12-26 18:02:02,490][105692] Updated weights for policy 0, policy_version 366506 (0.0009) [2023-12-26 18:02:02,545][105692] Updated weights for policy 0, policy_version 366516 (0.0008) [2023-12-26 18:02:02,597][105692] Updated weights for policy 0, policy_version 366526 (0.0011) [2023-12-26 18:02:02,645][105692] Updated weights for policy 0, policy_version 366536 (0.0010) [2023-12-26 18:02:02,873][105620] Updated weights for policy 1, policy_version 367177 (0.0006) [2023-12-26 18:02:02,933][105620] Updated weights for policy 1, policy_version 367187 (0.0010) [2023-12-26 18:02:02,985][105620] Updated weights for policy 1, policy_version 367197 (0.0010) [2023-12-26 18:02:03,039][105620] Updated weights for policy 1, policy_version 367207 (0.0010) [2023-12-26 18:02:03,328][105692] Updated weights for policy 0, policy_version 366546 (0.0007) [2023-12-26 18:02:03,386][105692] Updated weights for policy 0, policy_version 366556 (0.0005) [2023-12-26 18:02:03,442][105692] Updated weights for policy 0, policy_version 366566 (0.0005) [2023-12-26 18:02:03,614][105620] Updated weights for policy 1, policy_version 367217 (0.0007) [2023-12-26 18:02:03,667][105620] Updated weights for policy 1, policy_version 367227 (0.0008) [2023-12-26 18:02:03,729][105620] Updated weights for policy 1, policy_version 367237 (0.0005) [2023-12-26 18:02:04,023][105692] Updated weights for policy 0, policy_version 366576 (0.0006) [2023-12-26 18:02:04,071][105692] Updated weights for policy 0, policy_version 366586 (0.0005) [2023-12-26 18:02:04,132][105692] Updated weights for policy 0, policy_version 366596 (0.0007) [2023-12-26 18:02:04,424][105620] Updated weights for policy 1, policy_version 367247 (0.0008) [2023-12-26 18:02:04,479][105620] Updated weights for policy 1, policy_version 367257 (0.0009) [2023-12-26 18:02:04,531][105620] Updated weights for policy 1, policy_version 367267 (0.0009) [2023-12-26 18:02:04,851][105692] Updated weights for policy 0, policy_version 366606 (0.0009) [2023-12-26 18:02:04,898][105692] Updated weights for policy 0, policy_version 366616 (0.0008) [2023-12-26 18:02:04,951][105692] Updated weights for policy 0, policy_version 366626 (0.0006) [2023-12-26 18:02:05,327][105620] Updated weights for policy 1, policy_version 367277 (0.0009) [2023-12-26 18:02:05,385][105620] Updated weights for policy 1, policy_version 367287 (0.0009) [2023-12-26 18:02:05,443][105620] Updated weights for policy 1, policy_version 367297 (0.0006) [2023-12-26 18:02:05,663][105692] Updated weights for policy 0, policy_version 366636 (0.0007) [2023-12-26 18:02:05,724][105692] Updated weights for policy 0, policy_version 366646 (0.0009) [2023-12-26 18:02:05,786][105692] Updated weights for policy 0, policy_version 366656 (0.0009) [2023-12-26 18:02:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 187916288. Throughput: 0: 10301.3, 1: 9423.3. Samples: 187906852. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) [2023-12-26 18:02:06,062][104569] Avg episode reward: [(0, '9357.117'), (1, '7087.129')] [2023-12-26 18:02:06,171][105620] Updated weights for policy 1, policy_version 367307 (0.0009) [2023-12-26 18:02:06,226][105620] Updated weights for policy 1, policy_version 367317 (0.0009) [2023-12-26 18:02:06,285][105620] Updated weights for policy 1, policy_version 367327 (0.0009) [2023-12-26 18:02:06,540][105692] Updated weights for policy 0, policy_version 366666 (0.0010) [2023-12-26 18:02:06,595][105692] Updated weights for policy 0, policy_version 366676 (0.0009) [2023-12-26 18:02:06,643][105692] Updated weights for policy 0, policy_version 366686 (0.0009) [2023-12-26 18:02:06,690][105692] Updated weights for policy 0, policy_version 366696 (0.0008) [2023-12-26 18:02:07,078][105620] Updated weights for policy 1, policy_version 367337 (0.0009) [2023-12-26 18:02:07,134][105620] Updated weights for policy 1, policy_version 367347 (0.0009) [2023-12-26 18:02:07,193][105620] Updated weights for policy 1, policy_version 367357 (0.0009) [2023-12-26 18:02:07,241][105620] Updated weights for policy 1, policy_version 367367 (0.0009) [2023-12-26 18:02:07,436][105692] Updated weights for policy 0, policy_version 366706 (0.0007) [2023-12-26 18:02:07,498][105692] Updated weights for policy 0, policy_version 366716 (0.0008) [2023-12-26 18:02:07,557][105692] Updated weights for policy 0, policy_version 366726 (0.0005) [2023-12-26 18:02:08,113][105692] Updated weights for policy 0, policy_version 366736 (0.0007) [2023-12-26 18:02:08,118][105620] Updated weights for policy 1, policy_version 367377 (0.0009) [2023-12-26 18:02:08,165][105692] Updated weights for policy 0, policy_version 366746 (0.0006) [2023-12-26 18:02:08,182][105620] Updated weights for policy 1, policy_version 367387 (0.0008) [2023-12-26 18:02:08,211][105692] Updated weights for policy 0, policy_version 366756 (0.0008) [2023-12-26 18:02:08,246][105620] Updated weights for policy 1, policy_version 367397 (0.0006) [2023-12-26 18:02:08,873][105692] Updated weights for policy 0, policy_version 366766 (0.0008) [2023-12-26 18:02:08,929][105692] Updated weights for policy 0, policy_version 366776 (0.0009) [2023-12-26 18:02:08,987][105692] Updated weights for policy 0, policy_version 366786 (0.0008) [2023-12-26 18:02:09,016][105620] Updated weights for policy 1, policy_version 367407 (0.0008) [2023-12-26 18:02:09,082][105620] Updated weights for policy 1, policy_version 367417 (0.0010) [2023-12-26 18:02:09,144][105620] Updated weights for policy 1, policy_version 367427 (0.0009) [2023-12-26 18:02:09,720][105692] Updated weights for policy 0, policy_version 366796 (0.0009) [2023-12-26 18:02:09,768][105692] Updated weights for policy 0, policy_version 366806 (0.0009) [2023-12-26 18:02:09,831][105692] Updated weights for policy 0, policy_version 366816 (0.0009) [2023-12-26 18:02:09,974][105620] Updated weights for policy 1, policy_version 367437 (0.0008) [2023-12-26 18:02:10,037][105620] Updated weights for policy 1, policy_version 367447 (0.0008) [2023-12-26 18:02:10,100][105620] Updated weights for policy 1, policy_version 367457 (0.0007) [2023-12-26 18:02:10,625][105692] Updated weights for policy 0, policy_version 366826 (0.0011) [2023-12-26 18:02:10,687][105692] Updated weights for policy 0, policy_version 366836 (0.0010) [2023-12-26 18:02:10,698][105620] Updated weights for policy 1, policy_version 367467 (0.0009) [2023-12-26 18:02:10,744][105692] Updated weights for policy 0, policy_version 366846 (0.0010) [2023-12-26 18:02:10,755][105620] Updated weights for policy 1, policy_version 367477 (0.0007) [2023-12-26 18:02:10,796][105692] Updated weights for policy 0, policy_version 366856 (0.0010) [2023-12-26 18:02:10,811][105620] Updated weights for policy 1, policy_version 367487 (0.0007) [2023-12-26 18:02:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 188014592. Throughput: 0: 10166.6, 1: 9495.1. Samples: 188020572. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) [2023-12-26 18:02:11,062][104569] Avg episode reward: [(0, '9357.506'), (1, '7694.196')] [2023-12-26 18:02:11,516][105692] Updated weights for policy 0, policy_version 366866 (0.0010) [2023-12-26 18:02:11,543][105620] Updated weights for policy 1, policy_version 367497 (0.0009) [2023-12-26 18:02:11,579][105692] Updated weights for policy 0, policy_version 366876 (0.0011) [2023-12-26 18:02:11,610][105620] Updated weights for policy 1, policy_version 367507 (0.0008) [2023-12-26 18:02:11,644][105692] Updated weights for policy 0, policy_version 366886 (0.0009) [2023-12-26 18:02:11,674][105620] Updated weights for policy 1, policy_version 367517 (0.0008) [2023-12-26 18:02:11,745][105620] Updated weights for policy 1, policy_version 367527 (0.0007) [2023-12-26 18:02:12,333][105692] Updated weights for policy 0, policy_version 366896 (0.0007) [2023-12-26 18:02:12,397][105692] Updated weights for policy 0, policy_version 366906 (0.0011) [2023-12-26 18:02:12,455][105692] Updated weights for policy 0, policy_version 366916 (0.0010) [2023-12-26 18:02:12,457][105620] Updated weights for policy 1, policy_version 367537 (0.0006) [2023-12-26 18:02:12,513][105620] Updated weights for policy 1, policy_version 367547 (0.0005) [2023-12-26 18:02:12,561][105620] Updated weights for policy 1, policy_version 367557 (0.0005) [2023-12-26 18:02:13,177][105692] Updated weights for policy 0, policy_version 366926 (0.0010) [2023-12-26 18:02:13,228][105692] Updated weights for policy 0, policy_version 366936 (0.0010) [2023-12-26 18:02:13,242][105620] Updated weights for policy 1, policy_version 367567 (0.0005) [2023-12-26 18:02:13,276][105692] Updated weights for policy 0, policy_version 366946 (0.0010) [2023-12-26 18:02:13,290][105620] Updated weights for policy 1, policy_version 367577 (0.0005) [2023-12-26 18:02:13,353][105620] Updated weights for policy 1, policy_version 367587 (0.0008) [2023-12-26 18:02:14,036][105692] Updated weights for policy 0, policy_version 366956 (0.0010) [2023-12-26 18:02:14,082][105620] Updated weights for policy 1, policy_version 367597 (0.0007) [2023-12-26 18:02:14,084][105692] Updated weights for policy 0, policy_version 366966 (0.0010) [2023-12-26 18:02:14,133][105692] Updated weights for policy 0, policy_version 366976 (0.0007) [2023-12-26 18:02:14,144][105620] Updated weights for policy 1, policy_version 367607 (0.0007) [2023-12-26 18:02:14,210][105620] Updated weights for policy 1, policy_version 367617 (0.0006) [2023-12-26 18:02:14,751][105620] Updated weights for policy 1, policy_version 367627 (0.0005) [2023-12-26 18:02:14,786][105692] Updated weights for policy 0, policy_version 366986 (0.0010) [2023-12-26 18:02:14,832][105620] Updated weights for policy 1, policy_version 367637 (0.0006) [2023-12-26 18:02:14,844][105692] Updated weights for policy 0, policy_version 366996 (0.0008) [2023-12-26 18:02:14,891][105620] Updated weights for policy 1, policy_version 367647 (0.0005) [2023-12-26 18:02:14,901][105692] Updated weights for policy 0, policy_version 367006 (0.0011) [2023-12-26 18:02:14,966][105692] Updated weights for policy 0, policy_version 367016 (0.0011) [2023-12-26 18:02:15,554][105620] Updated weights for policy 1, policy_version 367657 (0.0006) [2023-12-26 18:02:15,614][105620] Updated weights for policy 1, policy_version 367667 (0.0008) [2023-12-26 18:02:15,653][105692] Updated weights for policy 0, policy_version 367026 (0.0005) [2023-12-26 18:02:15,670][105620] Updated weights for policy 1, policy_version 367677 (0.0009) [2023-12-26 18:02:15,713][105692] Updated weights for policy 0, policy_version 367036 (0.0005) [2023-12-26 18:02:15,726][105620] Updated weights for policy 1, policy_version 367687 (0.0008) [2023-12-26 18:02:15,776][105692] Updated weights for policy 0, policy_version 367046 (0.0007) [2023-12-26 18:02:16,063][104569] Fps is (10 sec: 19658.5, 60 sec: 19660.5, 300 sec: 19438.6). Total num frames: 188112896. Throughput: 0: 10015.6, 1: 9597.9. Samples: 188078820. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) [2023-12-26 18:02:16,064][104569] Avg episode reward: [(0, '9357.265'), (1, '8615.843')] [2023-12-26 18:02:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000367048_93978624.pth... [2023-12-26 18:02:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000367688_94134272.pth... [2023-12-26 18:02:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000366568_93847552.pth [2023-12-26 18:02:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000365896_93683712.pth [2023-12-26 18:02:16,318][105620] Updated weights for policy 1, policy_version 367697 (0.0006) [2023-12-26 18:02:16,376][105620] Updated weights for policy 1, policy_version 367707 (0.0006) [2023-12-26 18:02:16,387][105692] Updated weights for policy 0, policy_version 367056 (0.0006) [2023-12-26 18:02:16,431][105620] Updated weights for policy 1, policy_version 367717 (0.0006) [2023-12-26 18:02:16,454][105692] Updated weights for policy 0, policy_version 367066 (0.0008) [2023-12-26 18:02:16,509][105692] Updated weights for policy 0, policy_version 367076 (0.0009) [2023-12-26 18:02:17,005][105620] Updated weights for policy 1, policy_version 367727 (0.0007) [2023-12-26 18:02:17,056][105620] Updated weights for policy 1, policy_version 367737 (0.0008) [2023-12-26 18:02:17,120][105620] Updated weights for policy 1, policy_version 367747 (0.0009) [2023-12-26 18:02:17,282][105692] Updated weights for policy 0, policy_version 367086 (0.0010) [2023-12-26 18:02:17,330][105692] Updated weights for policy 0, policy_version 367096 (0.0009) [2023-12-26 18:02:17,390][105692] Updated weights for policy 0, policy_version 367106 (0.0008) [2023-12-26 18:02:17,901][105620] Updated weights for policy 1, policy_version 367757 (0.0008) [2023-12-26 18:02:17,958][105620] Updated weights for policy 1, policy_version 367767 (0.0009) [2023-12-26 18:02:18,010][105620] Updated weights for policy 1, policy_version 367777 (0.0009) [2023-12-26 18:02:18,076][105692] Updated weights for policy 0, policy_version 367116 (0.0007) [2023-12-26 18:02:18,132][105692] Updated weights for policy 0, policy_version 367126 (0.0005) [2023-12-26 18:02:18,191][105692] Updated weights for policy 0, policy_version 367136 (0.0005) [2023-12-26 18:02:18,826][105620] Updated weights for policy 1, policy_version 367787 (0.0009) [2023-12-26 18:02:18,852][105692] Updated weights for policy 0, policy_version 367146 (0.0006) [2023-12-26 18:02:18,875][105620] Updated weights for policy 1, policy_version 367797 (0.0007) [2023-12-26 18:02:18,915][105692] Updated weights for policy 0, policy_version 367156 (0.0008) [2023-12-26 18:02:18,933][105620] Updated weights for policy 1, policy_version 367807 (0.0007) [2023-12-26 18:02:18,973][105692] Updated weights for policy 0, policy_version 367166 (0.0009) [2023-12-26 18:02:19,027][105692] Updated weights for policy 0, policy_version 367176 (0.0010) [2023-12-26 18:02:19,582][105620] Updated weights for policy 1, policy_version 367817 (0.0006) [2023-12-26 18:02:19,647][105620] Updated weights for policy 1, policy_version 367827 (0.0005) [2023-12-26 18:02:19,712][105620] Updated weights for policy 1, policy_version 367837 (0.0006) [2023-12-26 18:02:19,769][105620] Updated weights for policy 1, policy_version 367847 (0.0006) [2023-12-26 18:02:19,879][105692] Updated weights for policy 0, policy_version 367186 (0.0007) [2023-12-26 18:02:19,954][105692] Updated weights for policy 0, policy_version 367196 (0.0006) [2023-12-26 18:02:20,013][105692] Updated weights for policy 0, policy_version 367206 (0.0009) [2023-12-26 18:02:20,399][105620] Updated weights for policy 1, policy_version 367857 (0.0008) [2023-12-26 18:02:20,464][105620] Updated weights for policy 1, policy_version 367867 (0.0008) [2023-12-26 18:02:20,525][105620] Updated weights for policy 1, policy_version 367877 (0.0005) [2023-12-26 18:02:20,802][105692] Updated weights for policy 0, policy_version 367216 (0.0009) [2023-12-26 18:02:20,858][105692] Updated weights for policy 0, policy_version 367226 (0.0009) [2023-12-26 18:02:20,921][105692] Updated weights for policy 0, policy_version 367236 (0.0009) [2023-12-26 18:02:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19438.6). Total num frames: 188211200. Throughput: 0: 9962.2, 1: 9619.8. Samples: 188200364. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) [2023-12-26 18:02:21,063][104569] Avg episode reward: [(0, '9356.516'), (1, '8994.195')] [2023-12-26 18:02:21,279][105620] Updated weights for policy 1, policy_version 367887 (0.0008) [2023-12-26 18:02:21,338][105620] Updated weights for policy 1, policy_version 367897 (0.0007) [2023-12-26 18:02:21,412][105620] Updated weights for policy 1, policy_version 367907 (0.0008) [2023-12-26 18:02:21,722][105692] Updated weights for policy 0, policy_version 367246 (0.0009) [2023-12-26 18:02:21,789][105692] Updated weights for policy 0, policy_version 367256 (0.0009) [2023-12-26 18:02:21,858][105692] Updated weights for policy 0, policy_version 367266 (0.0009) [2023-12-26 18:02:22,145][105620] Updated weights for policy 1, policy_version 367917 (0.0009) [2023-12-26 18:02:22,203][105620] Updated weights for policy 1, policy_version 367927 (0.0009) [2023-12-26 18:02:22,254][105620] Updated weights for policy 1, policy_version 367937 (0.0008) [2023-12-26 18:02:22,617][105692] Updated weights for policy 0, policy_version 367276 (0.0008) [2023-12-26 18:02:22,669][105692] Updated weights for policy 0, policy_version 367286 (0.0009) [2023-12-26 18:02:22,728][105692] Updated weights for policy 0, policy_version 367296 (0.0010) [2023-12-26 18:02:22,954][105620] Updated weights for policy 1, policy_version 367947 (0.0007) [2023-12-26 18:02:23,019][105620] Updated weights for policy 1, policy_version 367957 (0.0008) [2023-12-26 18:02:23,079][105620] Updated weights for policy 1, policy_version 367967 (0.0010) [2023-12-26 18:02:23,566][105692] Updated weights for policy 0, policy_version 367306 (0.0009) [2023-12-26 18:02:23,616][105692] Updated weights for policy 0, policy_version 367316 (0.0009) [2023-12-26 18:02:23,667][105692] Updated weights for policy 0, policy_version 367326 (0.0008) [2023-12-26 18:02:23,718][105692] Updated weights for policy 0, policy_version 367336 (0.0009) [2023-12-26 18:02:23,757][105620] Updated weights for policy 1, policy_version 367977 (0.0009) [2023-12-26 18:02:23,812][105620] Updated weights for policy 1, policy_version 367987 (0.0010) [2023-12-26 18:02:23,873][105620] Updated weights for policy 1, policy_version 367997 (0.0007) [2023-12-26 18:02:23,932][105620] Updated weights for policy 1, policy_version 368007 (0.0010) [2023-12-26 18:02:24,553][105620] Updated weights for policy 1, policy_version 368017 (0.0006) [2023-12-26 18:02:24,555][105692] Updated weights for policy 0, policy_version 367346 (0.0009) [2023-12-26 18:02:24,603][105620] Updated weights for policy 1, policy_version 368027 (0.0005) [2023-12-26 18:02:24,603][105692] Updated weights for policy 0, policy_version 367356 (0.0009) [2023-12-26 18:02:24,648][105620] Updated weights for policy 1, policy_version 368037 (0.0005) [2023-12-26 18:02:24,651][105692] Updated weights for policy 0, policy_version 367366 (0.0008) [2023-12-26 18:02:25,198][105620] Updated weights for policy 1, policy_version 368047 (0.0005) [2023-12-26 18:02:25,251][105620] Updated weights for policy 1, policy_version 368057 (0.0005) [2023-12-26 18:02:25,294][105620] Updated weights for policy 1, policy_version 368067 (0.0005) [2023-12-26 18:02:25,476][105692] Updated weights for policy 0, policy_version 367376 (0.0009) [2023-12-26 18:02:25,523][105692] Updated weights for policy 0, policy_version 367386 (0.0007) [2023-12-26 18:02:25,590][105692] Updated weights for policy 0, policy_version 367396 (0.0008) [2023-12-26 18:02:25,918][105620] Updated weights for policy 1, policy_version 368077 (0.0008) [2023-12-26 18:02:25,967][105620] Updated weights for policy 1, policy_version 368087 (0.0010) [2023-12-26 18:02:26,022][105620] Updated weights for policy 1, policy_version 368097 (0.0010) [2023-12-26 18:02:26,062][104569] Fps is (10 sec: 19663.1, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 188309504. Throughput: 0: 9749.0, 1: 9733.9. Samples: 188314476. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) [2023-12-26 18:02:26,062][104569] Avg episode reward: [(0, '9356.702'), (1, '8990.906')] [2023-12-26 18:02:26,236][105692] Updated weights for policy 0, policy_version 367406 (0.0006) [2023-12-26 18:02:26,286][105692] Updated weights for policy 0, policy_version 367416 (0.0005) [2023-12-26 18:02:26,333][105692] Updated weights for policy 0, policy_version 367426 (0.0005) [2023-12-26 18:02:26,754][105620] Updated weights for policy 1, policy_version 368107 (0.0010) [2023-12-26 18:02:26,805][105620] Updated weights for policy 1, policy_version 368117 (0.0007) [2023-12-26 18:02:26,847][105692] Updated weights for policy 0, policy_version 367436 (0.0005) [2023-12-26 18:02:26,852][105620] Updated weights for policy 1, policy_version 368127 (0.0010) [2023-12-26 18:02:26,898][105692] Updated weights for policy 0, policy_version 367446 (0.0006) [2023-12-26 18:02:26,946][105692] Updated weights for policy 0, policy_version 367456 (0.0010) [2023-12-26 18:02:27,554][105620] Updated weights for policy 1, policy_version 368137 (0.0010) [2023-12-26 18:02:27,618][105620] Updated weights for policy 1, policy_version 368147 (0.0010) [2023-12-26 18:02:27,666][105692] Updated weights for policy 0, policy_version 367466 (0.0010) [2023-12-26 18:02:27,683][105620] Updated weights for policy 1, policy_version 368157 (0.0010) [2023-12-26 18:02:27,721][105692] Updated weights for policy 0, policy_version 367476 (0.0010) [2023-12-26 18:02:27,741][105620] Updated weights for policy 1, policy_version 368167 (0.0010) [2023-12-26 18:02:27,782][105692] Updated weights for policy 0, policy_version 367486 (0.0010) [2023-12-26 18:02:27,842][105692] Updated weights for policy 0, policy_version 367496 (0.0005) [2023-12-26 18:02:28,389][105692] Updated weights for policy 0, policy_version 367506 (0.0006) [2023-12-26 18:02:28,444][105692] Updated weights for policy 0, policy_version 367516 (0.0005) [2023-12-26 18:02:28,471][105620] Updated weights for policy 1, policy_version 368177 (0.0010) [2023-12-26 18:02:28,507][105692] Updated weights for policy 0, policy_version 367526 (0.0006) [2023-12-26 18:02:28,529][105620] Updated weights for policy 1, policy_version 368187 (0.0010) [2023-12-26 18:02:28,584][105620] Updated weights for policy 1, policy_version 368197 (0.0009) [2023-12-26 18:02:29,167][105692] Updated weights for policy 0, policy_version 367536 (0.0007) [2023-12-26 18:02:29,186][105585] KL-divergence is very high: 184.5245 [2023-12-26 18:02:29,192][105585] KL-divergence is very high: 330.2247 [2023-12-26 18:02:29,198][105585] KL-divergence is very high: 127.6112 [2023-12-26 18:02:29,204][105585] KL-divergence is very high: 319.0826 [2023-12-26 18:02:29,210][105585] KL-divergence is very high: 261.6288 [2023-12-26 18:02:29,232][105692] Updated weights for policy 0, policy_version 367546 (0.0009) [2023-12-26 18:02:29,247][105585] KL-divergence is very high: 144.8421 [2023-12-26 18:02:29,261][105585] KL-divergence is very high: 127.1505 [2023-12-26 18:02:29,267][105585] KL-divergence is very high: 111.9410 [2023-12-26 18:02:29,270][105620] Updated weights for policy 1, policy_version 368207 (0.0007) [2023-12-26 18:02:29,297][105692] Updated weights for policy 0, policy_version 367556 (0.0009) [2023-12-26 18:02:29,332][105620] Updated weights for policy 1, policy_version 368217 (0.0007) [2023-12-26 18:02:29,407][105620] Updated weights for policy 1, policy_version 368227 (0.0008) [2023-12-26 18:02:30,012][105620] Updated weights for policy 1, policy_version 368237 (0.0007) [2023-12-26 18:02:30,067][105620] Updated weights for policy 1, policy_version 368247 (0.0009) [2023-12-26 18:02:30,073][105585] KL-divergence is very high: 158.8081 [2023-12-26 18:02:30,103][105692] Updated weights for policy 0, policy_version 367566 (0.0008) [2023-12-26 18:02:30,118][105585] KL-divergence is very high: 228.1473 [2023-12-26 18:02:30,127][105620] Updated weights for policy 1, policy_version 368257 (0.0008) [2023-12-26 18:02:30,157][105692] Updated weights for policy 0, policy_version 367576 (0.0008) [2023-12-26 18:02:30,163][105585] KL-divergence is very high: 125.9844 [2023-12-26 18:02:30,212][105692] Updated weights for policy 0, policy_version 367586 (0.0008) [2023-12-26 18:02:30,789][105620] Updated weights for policy 1, policy_version 368267 (0.0007) [2023-12-26 18:02:30,845][105620] Updated weights for policy 1, policy_version 368277 (0.0005) [2023-12-26 18:02:30,869][105692] Updated weights for policy 0, policy_version 367596 (0.0008) [2023-12-26 18:02:30,891][105620] Updated weights for policy 1, policy_version 368287 (0.0005) [2023-12-26 18:02:30,920][105692] Updated weights for policy 0, policy_version 367606 (0.0005) [2023-12-26 18:02:30,972][105692] Updated weights for policy 0, policy_version 367616 (0.0005) [2023-12-26 18:02:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.4, 300 sec: 19466.4). Total num frames: 188416000. Throughput: 0: 9866.3, 1: 9774.9. Samples: 188378304. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) [2023-12-26 18:02:31,062][104569] Avg episode reward: [(0, '3041.429'), (1, '9265.499')] [2023-12-26 18:02:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000367624_94126080.pth... [2023-12-26 18:02:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000368296_94289920.pth... [2023-12-26 18:02:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000366472_93831168.pth [2023-12-26 18:02:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000367112_93986816.pth [2023-12-26 18:02:31,525][105620] Updated weights for policy 1, policy_version 368297 (0.0006) [2023-12-26 18:02:31,573][105620] Updated weights for policy 1, policy_version 368307 (0.0010) [2023-12-26 18:02:31,633][105620] Updated weights for policy 1, policy_version 368317 (0.0009) [2023-12-26 18:02:31,700][105620] Updated weights for policy 1, policy_version 368327 (0.0011) [2023-12-26 18:02:31,725][105692] Updated weights for policy 0, policy_version 367626 (0.0006) [2023-12-26 18:02:31,751][105585] KL-divergence is very high: 153.6563 [2023-12-26 18:02:31,763][105585] KL-divergence is very high: 180.1877 [2023-12-26 18:02:31,779][105692] Updated weights for policy 0, policy_version 367636 (0.0006) [2023-12-26 18:02:31,796][105585] KL-divergence is very high: 152.3377 [2023-12-26 18:02:31,808][105585] KL-divergence is very high: 111.7181 [2023-12-26 18:02:31,832][105692] Updated weights for policy 0, policy_version 367646 (0.0007) [2023-12-26 18:02:31,890][105692] Updated weights for policy 0, policy_version 367656 (0.0009) [2023-12-26 18:02:32,497][105620] Updated weights for policy 1, policy_version 368337 (0.0009) [2023-12-26 18:02:32,542][105585] KL-divergence is very high: 115.4484 [2023-12-26 18:02:32,556][105620] Updated weights for policy 1, policy_version 368347 (0.0008) [2023-12-26 18:02:32,587][105585] KL-divergence is very high: 118.6843 [2023-12-26 18:02:32,588][105692] Updated weights for policy 0, policy_version 367666 (0.0008) [2023-12-26 18:02:32,619][105620] Updated weights for policy 1, policy_version 368357 (0.0007) [2023-12-26 18:02:32,628][105585] KL-divergence is very high: 102.6466 [2023-12-26 18:02:32,638][105692] Updated weights for policy 0, policy_version 367676 (0.0006) [2023-12-26 18:02:32,692][105692] Updated weights for policy 0, policy_version 367686 (0.0008) [2023-12-26 18:02:33,366][105620] Updated weights for policy 1, policy_version 368367 (0.0010) [2023-12-26 18:02:33,417][105620] Updated weights for policy 1, policy_version 368377 (0.0010) [2023-12-26 18:02:33,454][105692] Updated weights for policy 0, policy_version 367696 (0.0006) [2023-12-26 18:02:33,475][105620] Updated weights for policy 1, policy_version 368387 (0.0010) [2023-12-26 18:02:33,513][105692] Updated weights for policy 0, policy_version 367706 (0.0006) [2023-12-26 18:02:33,560][105692] Updated weights for policy 0, policy_version 367716 (0.0007) [2023-12-26 18:02:34,232][105620] Updated weights for policy 1, policy_version 368397 (0.0010) [2023-12-26 18:02:34,285][105620] Updated weights for policy 1, policy_version 368407 (0.0011) [2023-12-26 18:02:34,331][105692] Updated weights for policy 0, policy_version 367726 (0.0008) [2023-12-26 18:02:34,341][105620] Updated weights for policy 1, policy_version 368417 (0.0011) [2023-12-26 18:02:34,383][105692] Updated weights for policy 0, policy_version 367736 (0.0006) [2023-12-26 18:02:34,435][105692] Updated weights for policy 0, policy_version 367746 (0.0008) [2023-12-26 18:02:35,095][105620] Updated weights for policy 1, policy_version 368427 (0.0010) [2023-12-26 18:02:35,150][105620] Updated weights for policy 1, policy_version 368437 (0.0010) [2023-12-26 18:02:35,204][105620] Updated weights for policy 1, policy_version 368447 (0.0010) [2023-12-26 18:02:35,214][105692] Updated weights for policy 0, policy_version 367756 (0.0007) [2023-12-26 18:02:35,262][105692] Updated weights for policy 0, policy_version 367766 (0.0006) [2023-12-26 18:02:35,314][105692] Updated weights for policy 0, policy_version 367776 (0.0008) [2023-12-26 18:02:35,939][105620] Updated weights for policy 1, policy_version 368457 (0.0010) [2023-12-26 18:02:35,987][105620] Updated weights for policy 1, policy_version 368467 (0.0009) [2023-12-26 18:02:36,038][105620] Updated weights for policy 1, policy_version 368478 (0.0009) [2023-12-26 18:02:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 188497920. Throughput: 0: 9691.3, 1: 9880.2. Samples: 188494036. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) [2023-12-26 18:02:36,062][104569] Avg episode reward: [(0, '2040.493'), (1, '9357.700')] [2023-12-26 18:02:36,084][105620] Updated weights for policy 1, policy_version 368488 (0.0008) [2023-12-26 18:02:36,085][105692] Updated weights for policy 0, policy_version 367786 (0.0008) [2023-12-26 18:02:36,145][105692] Updated weights for policy 0, policy_version 367796 (0.0009) [2023-12-26 18:02:36,155][105585] KL-divergence is very high: 112.2094 [2023-12-26 18:02:36,196][105692] Updated weights for policy 0, policy_version 367806 (0.0009) [2023-12-26 18:02:36,197][105585] KL-divergence is very high: 112.4049 [2023-12-26 18:02:36,235][105585] KL-divergence is very high: 123.0211 [2023-12-26 18:02:36,244][105692] Updated weights for policy 0, policy_version 367816 (0.0009) [2023-12-26 18:02:36,859][105620] Updated weights for policy 1, policy_version 368498 (0.0009) [2023-12-26 18:02:36,905][105620] Updated weights for policy 1, policy_version 368508 (0.0009) [2023-12-26 18:02:36,965][105620] Updated weights for policy 1, policy_version 368518 (0.0009) [2023-12-26 18:02:37,028][105692] Updated weights for policy 0, policy_version 367826 (0.0009) [2023-12-26 18:02:37,083][105692] Updated weights for policy 0, policy_version 367836 (0.0010) [2023-12-26 18:02:37,141][105692] Updated weights for policy 0, policy_version 367846 (0.0010) [2023-12-26 18:02:37,561][105620] Updated weights for policy 1, policy_version 368528 (0.0006) [2023-12-26 18:02:37,614][105620] Updated weights for policy 1, policy_version 368538 (0.0007) [2023-12-26 18:02:37,672][105620] Updated weights for policy 1, policy_version 368548 (0.0010) [2023-12-26 18:02:37,958][105692] Updated weights for policy 0, policy_version 367856 (0.0008) [2023-12-26 18:02:38,009][105692] Updated weights for policy 0, policy_version 367866 (0.0009) [2023-12-26 18:02:38,070][105692] Updated weights for policy 0, policy_version 367876 (0.0009) [2023-12-26 18:02:38,383][105620] Updated weights for policy 1, policy_version 368558 (0.0009) [2023-12-26 18:02:38,438][105620] Updated weights for policy 1, policy_version 368568 (0.0008) [2023-12-26 18:02:38,500][105620] Updated weights for policy 1, policy_version 368578 (0.0008) [2023-12-26 18:02:38,881][105692] Updated weights for policy 0, policy_version 367886 (0.0009) [2023-12-26 18:02:38,935][105692] Updated weights for policy 0, policy_version 367896 (0.0009) [2023-12-26 18:02:38,990][105692] Updated weights for policy 0, policy_version 367906 (0.0005) [2023-12-26 18:02:39,136][105620] Updated weights for policy 1, policy_version 368588 (0.0007) [2023-12-26 18:02:39,193][105620] Updated weights for policy 1, policy_version 368598 (0.0009) [2023-12-26 18:02:39,253][105620] Updated weights for policy 1, policy_version 368608 (0.0009) [2023-12-26 18:02:39,722][105692] Updated weights for policy 0, policy_version 367916 (0.0007) [2023-12-26 18:02:39,778][105692] Updated weights for policy 0, policy_version 367926 (0.0011) [2023-12-26 18:02:39,837][105692] Updated weights for policy 0, policy_version 367936 (0.0010) [2023-12-26 18:02:40,032][105620] Updated weights for policy 1, policy_version 368618 (0.0009) [2023-12-26 18:02:40,090][105620] Updated weights for policy 1, policy_version 368628 (0.0009) [2023-12-26 18:02:40,152][105620] Updated weights for policy 1, policy_version 368638 (0.0007) [2023-12-26 18:02:40,221][105620] Updated weights for policy 1, policy_version 368648 (0.0006) [2023-12-26 18:02:40,588][105692] Updated weights for policy 0, policy_version 367946 (0.0010) [2023-12-26 18:02:40,650][105692] Updated weights for policy 0, policy_version 367956 (0.0011) [2023-12-26 18:02:40,715][105692] Updated weights for policy 0, policy_version 367966 (0.0010) [2023-12-26 18:02:40,775][105692] Updated weights for policy 0, policy_version 367976 (0.0008) [2023-12-26 18:02:40,973][105620] Updated weights for policy 1, policy_version 368658 (0.0009) [2023-12-26 18:02:41,029][105620] Updated weights for policy 1, policy_version 368668 (0.0008) [2023-12-26 18:02:41,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 188596224. Throughput: 0: 9630.2, 1: 9946.7. Samples: 188607644. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) [2023-12-26 18:02:41,063][104569] Avg episode reward: [(0, '3844.075'), (1, '8994.578')] [2023-12-26 18:02:41,097][105620] Updated weights for policy 1, policy_version 368678 (0.0009) [2023-12-26 18:02:41,471][105692] Updated weights for policy 0, policy_version 367986 (0.0006) [2023-12-26 18:02:41,532][105692] Updated weights for policy 0, policy_version 367996 (0.0006) [2023-12-26 18:02:41,594][105692] Updated weights for policy 0, policy_version 368006 (0.0006) [2023-12-26 18:02:41,943][105620] Updated weights for policy 1, policy_version 368688 (0.0009) [2023-12-26 18:02:41,998][105620] Updated weights for policy 1, policy_version 368698 (0.0009) [2023-12-26 18:02:42,060][105620] Updated weights for policy 1, policy_version 368708 (0.0009) [2023-12-26 18:02:42,206][105692] Updated weights for policy 0, policy_version 368016 (0.0008) [2023-12-26 18:02:42,218][105585] KL-divergence is very high: 259.2643 [2023-12-26 18:02:42,266][105692] Updated weights for policy 0, policy_version 368026 (0.0009) [2023-12-26 18:02:42,267][105585] KL-divergence is very high: 499.6179 [2023-12-26 18:02:42,319][105585] KL-divergence is very high: 570.5953 [2023-12-26 18:02:42,331][105692] Updated weights for policy 0, policy_version 368036 (0.0010) [2023-12-26 18:02:42,824][105620] Updated weights for policy 1, policy_version 368718 (0.0009) [2023-12-26 18:02:42,883][105620] Updated weights for policy 1, policy_version 368728 (0.0009) [2023-12-26 18:02:42,938][105620] Updated weights for policy 1, policy_version 368738 (0.0009) [2023-12-26 18:02:43,019][105692] Updated weights for policy 0, policy_version 368046 (0.0009) [2023-12-26 18:02:43,072][105692] Updated weights for policy 0, policy_version 368056 (0.0006) [2023-12-26 18:02:43,133][105692] Updated weights for policy 0, policy_version 368066 (0.0009) [2023-12-26 18:02:43,710][105620] Updated weights for policy 1, policy_version 368748 (0.0009) [2023-12-26 18:02:43,763][105620] Updated weights for policy 1, policy_version 368759 (0.0010) [2023-12-26 18:02:43,811][105620] Updated weights for policy 1, policy_version 368769 (0.0008) [2023-12-26 18:02:43,872][105692] Updated weights for policy 0, policy_version 368076 (0.0010) [2023-12-26 18:02:43,931][105692] Updated weights for policy 0, policy_version 368086 (0.0010) [2023-12-26 18:02:43,989][105692] Updated weights for policy 0, policy_version 368096 (0.0006) [2023-12-26 18:02:44,588][105692] Updated weights for policy 0, policy_version 368106 (0.0007) [2023-12-26 18:02:44,610][105620] Updated weights for policy 1, policy_version 368779 (0.0007) [2023-12-26 18:02:44,640][105692] Updated weights for policy 0, policy_version 368116 (0.0010) [2023-12-26 18:02:44,662][105620] Updated weights for policy 1, policy_version 368789 (0.0005) [2023-12-26 18:02:44,695][105692] Updated weights for policy 0, policy_version 368126 (0.0010) [2023-12-26 18:02:44,717][105620] Updated weights for policy 1, policy_version 368799 (0.0006) [2023-12-26 18:02:44,757][105692] Updated weights for policy 0, policy_version 368136 (0.0011) [2023-12-26 18:02:45,440][105692] Updated weights for policy 0, policy_version 368146 (0.0010) [2023-12-26 18:02:45,467][105585] KL-divergence is very high: 299.0915 [2023-12-26 18:02:45,480][105585] KL-divergence is very high: 252.4672 [2023-12-26 18:02:45,487][105585] KL-divergence is very high: 249.3074 [2023-12-26 18:02:45,506][105692] Updated weights for policy 0, policy_version 368156 (0.0011) [2023-12-26 18:02:45,519][105585] KL-divergence is very high: 323.8437 [2023-12-26 18:02:45,532][105585] KL-divergence is very high: 218.4085 [2023-12-26 18:02:45,535][105620] Updated weights for policy 1, policy_version 368809 (0.0006) [2023-12-26 18:02:45,538][105585] KL-divergence is very high: 204.6771 [2023-12-26 18:02:45,567][105692] Updated weights for policy 0, policy_version 368166 (0.0010) [2023-12-26 18:02:45,567][105585] KL-divergence is very high: 194.7672 [2023-12-26 18:02:45,594][105620] Updated weights for policy 1, policy_version 368819 (0.0006) [2023-12-26 18:02:45,660][105620] Updated weights for policy 1, policy_version 368829 (0.0008) [2023-12-26 18:02:45,719][105620] Updated weights for policy 1, policy_version 368840 (0.0007) [2023-12-26 18:02:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 188694528. Throughput: 0: 9624.3, 1: 9867.1. Samples: 188663416. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) [2023-12-26 18:02:46,063][104569] Avg episode reward: [(0, '6957.672'), (1, '8811.918')] [2023-12-26 18:02:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000368168_94265344.pth... [2023-12-26 18:02:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000368840_94429184.pth... [2023-12-26 18:02:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000367048_93978624.pth [2023-12-26 18:02:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000367688_94134272.pth [2023-12-26 18:02:46,252][105692] Updated weights for policy 0, policy_version 368176 (0.0006) [2023-12-26 18:02:46,303][105692] Updated weights for policy 0, policy_version 368186 (0.0010) [2023-12-26 18:02:46,365][105692] Updated weights for policy 0, policy_version 368196 (0.0010) [2023-12-26 18:02:46,368][105620] Updated weights for policy 1, policy_version 368850 (0.0007) [2023-12-26 18:02:46,419][105620] Updated weights for policy 1, policy_version 368860 (0.0005) [2023-12-26 18:02:46,471][105620] Updated weights for policy 1, policy_version 368870 (0.0005) [2023-12-26 18:02:47,083][105692] Updated weights for policy 0, policy_version 368206 (0.0010) [2023-12-26 18:02:47,087][105620] Updated weights for policy 1, policy_version 368880 (0.0007) [2023-12-26 18:02:47,142][105692] Updated weights for policy 0, policy_version 368216 (0.0010) [2023-12-26 18:02:47,145][105620] Updated weights for policy 1, policy_version 368890 (0.0010) [2023-12-26 18:02:47,197][105692] Updated weights for policy 0, policy_version 368226 (0.0010) [2023-12-26 18:02:47,201][105620] Updated weights for policy 1, policy_version 368900 (0.0010) [2023-12-26 18:02:47,782][105692] Updated weights for policy 0, policy_version 368236 (0.0008) [2023-12-26 18:02:47,834][105692] Updated weights for policy 0, policy_version 368246 (0.0010) [2023-12-26 18:02:47,882][105692] Updated weights for policy 0, policy_version 368256 (0.0010) [2023-12-26 18:02:47,959][105620] Updated weights for policy 1, policy_version 368910 (0.0009) [2023-12-26 18:02:48,022][105620] Updated weights for policy 1, policy_version 368920 (0.0008) [2023-12-26 18:02:48,071][105620] Updated weights for policy 1, policy_version 368930 (0.0007) [2023-12-26 18:02:48,671][105692] Updated weights for policy 0, policy_version 368266 (0.0010) [2023-12-26 18:02:48,724][105692] Updated weights for policy 0, policy_version 368276 (0.0006) [2023-12-26 18:02:48,775][105692] Updated weights for policy 0, policy_version 368286 (0.0005) [2023-12-26 18:02:48,833][105692] Updated weights for policy 0, policy_version 368296 (0.0009) [2023-12-26 18:02:48,869][105620] Updated weights for policy 1, policy_version 368940 (0.0009) [2023-12-26 18:02:48,932][105620] Updated weights for policy 1, policy_version 368950 (0.0011) [2023-12-26 18:02:48,994][105620] Updated weights for policy 1, policy_version 368960 (0.0009) [2023-12-26 18:02:49,581][105692] Updated weights for policy 0, policy_version 368306 (0.0010) [2023-12-26 18:02:49,644][105692] Updated weights for policy 0, policy_version 368316 (0.0010) [2023-12-26 18:02:49,704][105692] Updated weights for policy 0, policy_version 368326 (0.0011) [2023-12-26 18:02:49,741][105620] Updated weights for policy 1, policy_version 368970 (0.0011) [2023-12-26 18:02:49,802][105620] Updated weights for policy 1, policy_version 368980 (0.0010) [2023-12-26 18:02:49,874][105620] Updated weights for policy 1, policy_version 368990 (0.0010) [2023-12-26 18:02:49,938][105620] Updated weights for policy 1, policy_version 369000 (0.0010) [2023-12-26 18:02:50,446][105692] Updated weights for policy 0, policy_version 368336 (0.0011) [2023-12-26 18:02:50,505][105692] Updated weights for policy 0, policy_version 368346 (0.0011) [2023-12-26 18:02:50,564][105692] Updated weights for policy 0, policy_version 368356 (0.0010) [2023-12-26 18:02:50,607][105620] Updated weights for policy 1, policy_version 369010 (0.0011) [2023-12-26 18:02:50,673][105620] Updated weights for policy 1, policy_version 369020 (0.0010) [2023-12-26 18:02:50,737][105620] Updated weights for policy 1, policy_version 369030 (0.0008) [2023-12-26 18:02:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 188792832. Throughput: 0: 9695.2, 1: 9746.5. Samples: 188781728. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) [2023-12-26 18:02:51,062][104569] Avg episode reward: [(0, '8750.745'), (1, '8997.146')] [2023-12-26 18:02:51,318][105692] Updated weights for policy 0, policy_version 368366 (0.0010) [2023-12-26 18:02:51,386][105692] Updated weights for policy 0, policy_version 368376 (0.0008) [2023-12-26 18:02:51,432][105692] Updated weights for policy 0, policy_version 368386 (0.0005) [2023-12-26 18:02:51,519][105620] Updated weights for policy 1, policy_version 369040 (0.0008) [2023-12-26 18:02:51,575][105620] Updated weights for policy 1, policy_version 369050 (0.0008) [2023-12-26 18:02:51,643][105620] Updated weights for policy 1, policy_version 369060 (0.0008) [2023-12-26 18:02:52,120][105692] Updated weights for policy 0, policy_version 368396 (0.0007) [2023-12-26 18:02:52,172][105692] Updated weights for policy 0, policy_version 368406 (0.0009) [2023-12-26 18:02:52,224][105692] Updated weights for policy 0, policy_version 368416 (0.0009) [2023-12-26 18:02:52,427][105620] Updated weights for policy 1, policy_version 369070 (0.0009) [2023-12-26 18:02:52,479][105620] Updated weights for policy 1, policy_version 369080 (0.0009) [2023-12-26 18:02:52,533][105620] Updated weights for policy 1, policy_version 369090 (0.0008) [2023-12-26 18:02:52,987][105692] Updated weights for policy 0, policy_version 368426 (0.0009) [2023-12-26 18:02:53,043][105692] Updated weights for policy 0, policy_version 368437 (0.0009) [2023-12-26 18:02:53,110][105692] Updated weights for policy 0, policy_version 368447 (0.0011) [2023-12-26 18:02:53,210][105620] Updated weights for policy 1, policy_version 369100 (0.0008) [2023-12-26 18:02:53,268][105620] Updated weights for policy 1, policy_version 369110 (0.0009) [2023-12-26 18:02:53,321][105620] Updated weights for policy 1, policy_version 369120 (0.0010) [2023-12-26 18:02:53,778][105692] Updated weights for policy 0, policy_version 368457 (0.0010) [2023-12-26 18:02:53,826][105692] Updated weights for policy 0, policy_version 368467 (0.0007) [2023-12-26 18:02:53,875][105692] Updated weights for policy 0, policy_version 368477 (0.0009) [2023-12-26 18:02:53,923][105692] Updated weights for policy 0, policy_version 368487 (0.0009) [2023-12-26 18:02:54,106][105620] Updated weights for policy 1, policy_version 369130 (0.0008) [2023-12-26 18:02:54,163][105620] Updated weights for policy 1, policy_version 369140 (0.0005) [2023-12-26 18:02:54,231][105620] Updated weights for policy 1, policy_version 369150 (0.0005) [2023-12-26 18:02:54,290][105620] Updated weights for policy 1, policy_version 369160 (0.0005) [2023-12-26 18:02:54,734][105692] Updated weights for policy 0, policy_version 368497 (0.0007) [2023-12-26 18:02:54,789][105692] Updated weights for policy 0, policy_version 368507 (0.0008) [2023-12-26 18:02:54,845][105692] Updated weights for policy 0, policy_version 368517 (0.0009) [2023-12-26 18:02:54,905][105620] Updated weights for policy 1, policy_version 369170 (0.0010) [2023-12-26 18:02:54,954][105620] Updated weights for policy 1, policy_version 369180 (0.0010) [2023-12-26 18:02:55,010][105620] Updated weights for policy 1, policy_version 369190 (0.0011) [2023-12-26 18:02:55,486][105692] Updated weights for policy 0, policy_version 368527 (0.0010) [2023-12-26 18:02:55,541][105692] Updated weights for policy 0, policy_version 368537 (0.0010) [2023-12-26 18:02:55,596][105692] Updated weights for policy 0, policy_version 368547 (0.0010) [2023-12-26 18:02:55,660][105620] Updated weights for policy 1, policy_version 369200 (0.0009) [2023-12-26 18:02:55,721][105620] Updated weights for policy 1, policy_version 369211 (0.0009) [2023-12-26 18:02:55,784][105620] Updated weights for policy 1, policy_version 369221 (0.0008) [2023-12-26 18:02:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 188891136. Throughput: 0: 9668.4, 1: 9844.8. Samples: 188898668. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) [2023-12-26 18:02:56,063][104569] Avg episode reward: [(0, '9181.716'), (1, '8815.915')] [2023-12-26 18:02:56,201][105692] Updated weights for policy 0, policy_version 368557 (0.0008) [2023-12-26 18:02:56,256][105692] Updated weights for policy 0, policy_version 368567 (0.0005) [2023-12-26 18:02:56,324][105692] Updated weights for policy 0, policy_version 368577 (0.0005) [2023-12-26 18:02:56,381][105620] Updated weights for policy 1, policy_version 369231 (0.0009) [2023-12-26 18:02:56,445][105620] Updated weights for policy 1, policy_version 369241 (0.0010) [2023-12-26 18:02:56,510][105620] Updated weights for policy 1, policy_version 369251 (0.0010) [2023-12-26 18:02:56,986][105692] Updated weights for policy 0, policy_version 368587 (0.0007) [2023-12-26 18:02:57,043][105692] Updated weights for policy 0, policy_version 368597 (0.0010) [2023-12-26 18:02:57,092][105620] Updated weights for policy 1, policy_version 369261 (0.0009) [2023-12-26 18:02:57,094][105692] Updated weights for policy 0, policy_version 368607 (0.0010) [2023-12-26 18:02:57,142][105620] Updated weights for policy 1, policy_version 369271 (0.0006) [2023-12-26 18:02:57,204][105620] Updated weights for policy 1, policy_version 369281 (0.0009) [2023-12-26 18:02:57,838][105692] Updated weights for policy 0, policy_version 368617 (0.0010) [2023-12-26 18:02:57,888][105692] Updated weights for policy 0, policy_version 368627 (0.0005) [2023-12-26 18:02:57,937][105692] Updated weights for policy 0, policy_version 368637 (0.0005) [2023-12-26 18:02:57,976][105620] Updated weights for policy 1, policy_version 369291 (0.0009) [2023-12-26 18:02:57,994][105692] Updated weights for policy 0, policy_version 368647 (0.0008) [2023-12-26 18:02:58,034][105620] Updated weights for policy 1, policy_version 369301 (0.0010) [2023-12-26 18:02:58,084][105620] Updated weights for policy 1, policy_version 369311 (0.0010) [2023-12-26 18:02:58,706][105692] Updated weights for policy 0, policy_version 368657 (0.0010) [2023-12-26 18:02:58,776][105692] Updated weights for policy 0, policy_version 368667 (0.0010) [2023-12-26 18:02:58,850][105692] Updated weights for policy 0, policy_version 368677 (0.0010) [2023-12-26 18:02:58,987][105620] Updated weights for policy 1, policy_version 369321 (0.0010) [2023-12-26 18:02:59,038][105620] Updated weights for policy 1, policy_version 369331 (0.0008) [2023-12-26 18:02:59,077][105586] KL-divergence is very high: 128.0432 [2023-12-26 18:02:59,098][105620] Updated weights for policy 1, policy_version 369341 (0.0007) [2023-12-26 18:02:59,127][105586] KL-divergence is very high: 132.0599 [2023-12-26 18:02:59,159][105620] Updated weights for policy 1, policy_version 369351 (0.0011) [2023-12-26 18:02:59,627][105692] Updated weights for policy 0, policy_version 368687 (0.0010) [2023-12-26 18:02:59,682][105692] Updated weights for policy 0, policy_version 368697 (0.0012) [2023-12-26 18:02:59,734][105692] Updated weights for policy 0, policy_version 368707 (0.0009) [2023-12-26 18:02:59,831][105620] Updated weights for policy 1, policy_version 369361 (0.0011) [2023-12-26 18:02:59,890][105620] Updated weights for policy 1, policy_version 369371 (0.0011) [2023-12-26 18:02:59,955][105620] Updated weights for policy 1, policy_version 369381 (0.0011) [2023-12-26 18:03:00,408][105692] Updated weights for policy 0, policy_version 368717 (0.0008) [2023-12-26 18:03:00,463][105692] Updated weights for policy 0, policy_version 368727 (0.0005) [2023-12-26 18:03:00,524][105692] Updated weights for policy 0, policy_version 368737 (0.0010) [2023-12-26 18:03:00,627][105620] Updated weights for policy 1, policy_version 369391 (0.0007) [2023-12-26 18:03:00,663][105586] KL-divergence is very high: 207.3513 [2023-12-26 18:03:00,676][105620] Updated weights for policy 1, policy_version 369401 (0.0005) [2023-12-26 18:03:00,706][105586] KL-divergence is very high: 402.0337 [2023-12-26 18:03:00,736][105620] Updated weights for policy 1, policy_version 369411 (0.0005) [2023-12-26 18:03:00,758][105586] KL-divergence is very high: 451.4750 [2023-12-26 18:03:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 188989440. Throughput: 0: 9717.9, 1: 9857.1. Samples: 188959676. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) [2023-12-26 18:03:01,062][104569] Avg episode reward: [(0, '9270.874'), (1, '8474.851')] [2023-12-26 18:03:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000368744_94412800.pth... [2023-12-26 18:03:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000369416_94576640.pth... [2023-12-26 18:03:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000367624_94126080.pth [2023-12-26 18:03:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000368296_94289920.pth [2023-12-26 18:03:01,303][105692] Updated weights for policy 0, policy_version 368747 (0.0009) [2023-12-26 18:03:01,358][105620] Updated weights for policy 1, policy_version 369421 (0.0008) [2023-12-26 18:03:01,367][105692] Updated weights for policy 0, policy_version 368757 (0.0007) [2023-12-26 18:03:01,415][105620] Updated weights for policy 1, policy_version 369431 (0.0010) [2023-12-26 18:03:01,420][105692] Updated weights for policy 0, policy_version 368767 (0.0006) [2023-12-26 18:03:01,468][105620] Updated weights for policy 1, policy_version 369441 (0.0010) [2023-12-26 18:03:02,056][105692] Updated weights for policy 0, policy_version 368777 (0.0006) [2023-12-26 18:03:02,107][105692] Updated weights for policy 0, policy_version 368787 (0.0008) [2023-12-26 18:03:02,159][105692] Updated weights for policy 0, policy_version 368797 (0.0008) [2023-12-26 18:03:02,197][105620] Updated weights for policy 1, policy_version 369451 (0.0010) [2023-12-26 18:03:02,208][105692] Updated weights for policy 0, policy_version 368807 (0.0006) [2023-12-26 18:03:02,258][105620] Updated weights for policy 1, policy_version 369461 (0.0010) [2023-12-26 18:03:02,323][105620] Updated weights for policy 1, policy_version 369471 (0.0011) [2023-12-26 18:03:02,967][105620] Updated weights for policy 1, policy_version 369481 (0.0011) [2023-12-26 18:03:02,976][105692] Updated weights for policy 0, policy_version 368817 (0.0010) [2023-12-26 18:03:03,026][105620] Updated weights for policy 1, policy_version 369491 (0.0011) [2023-12-26 18:03:03,036][105692] Updated weights for policy 0, policy_version 368827 (0.0010) [2023-12-26 18:03:03,088][105620] Updated weights for policy 1, policy_version 369501 (0.0010) [2023-12-26 18:03:03,094][105692] Updated weights for policy 0, policy_version 368837 (0.0011) [2023-12-26 18:03:03,146][105620] Updated weights for policy 1, policy_version 369511 (0.0010) [2023-12-26 18:03:03,826][105620] Updated weights for policy 1, policy_version 369521 (0.0006) [2023-12-26 18:03:03,831][105692] Updated weights for policy 0, policy_version 368847 (0.0011) [2023-12-26 18:03:03,891][105620] Updated weights for policy 1, policy_version 369531 (0.0006) [2023-12-26 18:03:03,892][105692] Updated weights for policy 0, policy_version 368857 (0.0010) [2023-12-26 18:03:03,951][105620] Updated weights for policy 1, policy_version 369541 (0.0006) [2023-12-26 18:03:03,953][105692] Updated weights for policy 0, policy_version 368867 (0.0008) [2023-12-26 18:03:04,652][105620] Updated weights for policy 1, policy_version 369551 (0.0009) [2023-12-26 18:03:04,705][105620] Updated weights for policy 1, policy_version 369561 (0.0008) [2023-12-26 18:03:04,714][105586] KL-divergence is very high: 116.0353 [2023-12-26 18:03:04,718][105692] Updated weights for policy 0, policy_version 368878 (0.0007) [2023-12-26 18:03:04,734][105586] KL-divergence is very high: 130.2004 [2023-12-26 18:03:04,767][105586] KL-divergence is very high: 184.3084 [2023-12-26 18:03:04,773][105620] Updated weights for policy 1, policy_version 369571 (0.0009) [2023-12-26 18:03:04,781][105692] Updated weights for policy 0, policy_version 368888 (0.0006) [2023-12-26 18:03:04,787][105586] KL-divergence is very high: 142.6200 [2023-12-26 18:03:04,836][105692] Updated weights for policy 0, policy_version 368898 (0.0005) [2023-12-26 18:03:05,477][105692] Updated weights for policy 0, policy_version 368908 (0.0007) [2023-12-26 18:03:05,495][105620] Updated weights for policy 1, policy_version 369581 (0.0007) [2023-12-26 18:03:05,495][105586] KL-divergence is very high: 201.3817 [2023-12-26 18:03:05,533][105692] Updated weights for policy 0, policy_version 368918 (0.0011) [2023-12-26 18:03:05,543][105586] KL-divergence is very high: 192.2217 [2023-12-26 18:03:05,552][105620] Updated weights for policy 1, policy_version 369591 (0.0005) [2023-12-26 18:03:05,588][105692] Updated weights for policy 0, policy_version 368928 (0.0010) [2023-12-26 18:03:05,589][105586] KL-divergence is very high: 179.1329 [2023-12-26 18:03:05,611][105620] Updated weights for policy 1, policy_version 369601 (0.0006) [2023-12-26 18:03:05,638][105586] KL-divergence is very high: 163.4073 [2023-12-26 18:03:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 189087744. Throughput: 0: 9665.9, 1: 9813.6. Samples: 189076944. Policy #0 lag: (min: 16.0, avg: 40.4, max: 48.0) [2023-12-26 18:03:06,063][104569] Avg episode reward: [(0, '668.373'), (1, '8026.147')] [2023-12-26 18:03:06,272][105692] Updated weights for policy 0, policy_version 368938 (0.0007) [2023-12-26 18:03:06,323][105692] Updated weights for policy 0, policy_version 368948 (0.0008) [2023-12-26 18:03:06,388][105692] Updated weights for policy 0, policy_version 368958 (0.0011) [2023-12-26 18:03:06,425][105620] Updated weights for policy 1, policy_version 369611 (0.0007) [2023-12-26 18:03:06,441][105692] Updated weights for policy 0, policy_version 368968 (0.0011) [2023-12-26 18:03:06,493][105620] Updated weights for policy 1, policy_version 369621 (0.0008) [2023-12-26 18:03:06,545][105620] Updated weights for policy 1, policy_version 369631 (0.0008) [2023-12-26 18:03:07,158][105692] Updated weights for policy 0, policy_version 368978 (0.0010) [2023-12-26 18:03:07,210][105692] Updated weights for policy 0, policy_version 368988 (0.0011) [2023-12-26 18:03:07,262][105692] Updated weights for policy 0, policy_version 368998 (0.0010) [2023-12-26 18:03:07,272][105620] Updated weights for policy 1, policy_version 369642 (0.0009) [2023-12-26 18:03:07,321][105620] Updated weights for policy 1, policy_version 369652 (0.0010) [2023-12-26 18:03:07,366][105620] Updated weights for policy 1, policy_version 369662 (0.0010) [2023-12-26 18:03:07,421][105620] Updated weights for policy 1, policy_version 369672 (0.0010) [2023-12-26 18:03:07,920][105692] Updated weights for policy 0, policy_version 369008 (0.0010) [2023-12-26 18:03:07,969][105692] Updated weights for policy 0, policy_version 369018 (0.0010) [2023-12-26 18:03:08,028][105692] Updated weights for policy 0, policy_version 369028 (0.0010) [2023-12-26 18:03:08,062][105620] Updated weights for policy 1, policy_version 369682 (0.0007) [2023-12-26 18:03:08,120][105620] Updated weights for policy 1, policy_version 369692 (0.0005) [2023-12-26 18:03:08,179][105620] Updated weights for policy 1, policy_version 369702 (0.0005) [2023-12-26 18:03:08,728][105692] Updated weights for policy 0, policy_version 369038 (0.0008) [2023-12-26 18:03:08,756][105620] Updated weights for policy 1, policy_version 369712 (0.0006) [2023-12-26 18:03:08,790][105692] Updated weights for policy 0, policy_version 369048 (0.0008) [2023-12-26 18:03:08,816][105620] Updated weights for policy 1, policy_version 369722 (0.0007) [2023-12-26 18:03:08,848][105692] Updated weights for policy 0, policy_version 369058 (0.0006) [2023-12-26 18:03:08,879][105620] Updated weights for policy 1, policy_version 369732 (0.0008) [2023-12-26 18:03:09,600][105692] Updated weights for policy 0, policy_version 369068 (0.0009) [2023-12-26 18:03:09,630][105620] Updated weights for policy 1, policy_version 369742 (0.0008) [2023-12-26 18:03:09,651][105692] Updated weights for policy 0, policy_version 369078 (0.0008) [2023-12-26 18:03:09,686][105620] Updated weights for policy 1, policy_version 369752 (0.0007) [2023-12-26 18:03:09,708][105692] Updated weights for policy 0, policy_version 369088 (0.0007) [2023-12-26 18:03:09,750][105620] Updated weights for policy 1, policy_version 369762 (0.0008) [2023-12-26 18:03:10,444][105692] Updated weights for policy 0, policy_version 369098 (0.0006) [2023-12-26 18:03:10,494][105692] Updated weights for policy 0, policy_version 369108 (0.0007) [2023-12-26 18:03:10,544][105692] Updated weights for policy 0, policy_version 369118 (0.0005) [2023-12-26 18:03:10,559][105620] Updated weights for policy 1, policy_version 369772 (0.0008) [2023-12-26 18:03:10,598][105692] Updated weights for policy 0, policy_version 369128 (0.0007) [2023-12-26 18:03:10,630][105620] Updated weights for policy 1, policy_version 369782 (0.0008) [2023-12-26 18:03:10,695][105620] Updated weights for policy 1, policy_version 369792 (0.0009) [2023-12-26 18:03:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 189186048. Throughput: 0: 9845.7, 1: 9725.4. Samples: 189195176. Policy #0 lag: (min: 16.0, avg: 40.4, max: 48.0) [2023-12-26 18:03:11,063][104569] Avg episode reward: [(0, '826.596'), (1, '7267.773')] [2023-12-26 18:03:11,435][105620] Updated weights for policy 1, policy_version 369802 (0.0008) [2023-12-26 18:03:11,435][105692] Updated weights for policy 0, policy_version 369138 (0.0009) [2023-12-26 18:03:11,478][105586] KL-divergence is very high: 157.5813 [2023-12-26 18:03:11,496][105692] Updated weights for policy 0, policy_version 369148 (0.0008) [2023-12-26 18:03:11,498][105620] Updated weights for policy 1, policy_version 369812 (0.0008) [2023-12-26 18:03:11,524][105586] KL-divergence is very high: 143.2386 [2023-12-26 18:03:11,531][105586] KL-divergence is very high: 207.6313 [2023-12-26 18:03:11,554][105692] Updated weights for policy 0, policy_version 369158 (0.0009) [2023-12-26 18:03:11,561][105620] Updated weights for policy 1, policy_version 369822 (0.0009) [2023-12-26 18:03:11,574][105586] KL-divergence is very high: 140.0463 [2023-12-26 18:03:11,581][105586] KL-divergence is very high: 192.2515 [2023-12-26 18:03:11,632][105620] Updated weights for policy 1, policy_version 369832 (0.0009) [2023-12-26 18:03:12,379][105692] Updated weights for policy 0, policy_version 369168 (0.0008) [2023-12-26 18:03:12,408][105620] Updated weights for policy 1, policy_version 369842 (0.0008) [2023-12-26 18:03:12,438][105692] Updated weights for policy 0, policy_version 369178 (0.0009) [2023-12-26 18:03:12,468][105620] Updated weights for policy 1, policy_version 369852 (0.0008) [2023-12-26 18:03:12,495][105692] Updated weights for policy 0, policy_version 369188 (0.0006) [2023-12-26 18:03:12,529][105620] Updated weights for policy 1, policy_version 369862 (0.0010) [2023-12-26 18:03:13,206][105692] Updated weights for policy 0, policy_version 369198 (0.0008) [2023-12-26 18:03:13,270][105692] Updated weights for policy 0, policy_version 369208 (0.0009) [2023-12-26 18:03:13,303][105620] Updated weights for policy 1, policy_version 369872 (0.0007) [2023-12-26 18:03:13,336][105692] Updated weights for policy 0, policy_version 369218 (0.0007) [2023-12-26 18:03:13,361][105620] Updated weights for policy 1, policy_version 369882 (0.0008) [2023-12-26 18:03:13,418][105620] Updated weights for policy 1, policy_version 369892 (0.0009) [2023-12-26 18:03:14,061][105692] Updated weights for policy 0, policy_version 369228 (0.0007) [2023-12-26 18:03:14,108][105692] Updated weights for policy 0, policy_version 369238 (0.0009) [2023-12-26 18:03:14,160][105692] Updated weights for policy 0, policy_version 369248 (0.0008) [2023-12-26 18:03:14,169][105620] Updated weights for policy 1, policy_version 369902 (0.0009) [2023-12-26 18:03:14,228][105620] Updated weights for policy 1, policy_version 369912 (0.0007) [2023-12-26 18:03:14,285][105620] Updated weights for policy 1, policy_version 369922 (0.0009) [2023-12-26 18:03:14,913][105692] Updated weights for policy 0, policy_version 369258 (0.0009) [2023-12-26 18:03:14,970][105692] Updated weights for policy 0, policy_version 369268 (0.0009) [2023-12-26 18:03:15,027][105692] Updated weights for policy 0, policy_version 369278 (0.0010) [2023-12-26 18:03:15,036][105620] Updated weights for policy 1, policy_version 369932 (0.0008) [2023-12-26 18:03:15,087][105692] Updated weights for policy 0, policy_version 369288 (0.0009) [2023-12-26 18:03:15,095][105620] Updated weights for policy 1, policy_version 369942 (0.0008) [2023-12-26 18:03:15,148][105620] Updated weights for policy 1, policy_version 369952 (0.0010) [2023-12-26 18:03:15,772][105692] Updated weights for policy 0, policy_version 369298 (0.0005) [2023-12-26 18:03:15,827][105692] Updated weights for policy 0, policy_version 369308 (0.0005) [2023-12-26 18:03:15,887][105692] Updated weights for policy 0, policy_version 369318 (0.0005) [2023-12-26 18:03:15,897][105620] Updated weights for policy 1, policy_version 369962 (0.0011) [2023-12-26 18:03:15,951][105620] Updated weights for policy 1, policy_version 369972 (0.0009) [2023-12-26 18:03:16,010][105620] Updated weights for policy 1, policy_version 369982 (0.0011) [2023-12-26 18:03:16,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.6, 300 sec: 19438.7). Total num frames: 189284352. Throughput: 0: 9671.3, 1: 9681.0. Samples: 189249156. Policy #0 lag: (min: 16.0, avg: 40.4, max: 48.0) [2023-12-26 18:03:16,062][104569] Avg episode reward: [(0, '4415.555'), (1, '6210.763')] [2023-12-26 18:03:16,063][105620] Updated weights for policy 1, policy_version 369992 (0.0010) [2023-12-26 18:03:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000369320_94560256.pth... [2023-12-26 18:03:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000369992_94724096.pth... [2023-12-26 18:03:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000368168_94265344.pth [2023-12-26 18:03:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000368840_94429184.pth [2023-12-26 18:03:16,403][105692] Updated weights for policy 0, policy_version 369328 (0.0009) [2023-12-26 18:03:16,447][105692] Updated weights for policy 0, policy_version 369338 (0.0010) [2023-12-26 18:03:16,501][105692] Updated weights for policy 0, policy_version 369348 (0.0010) [2023-12-26 18:03:16,789][105620] Updated weights for policy 1, policy_version 370002 (0.0005) [2023-12-26 18:03:16,835][105620] Updated weights for policy 1, policy_version 370012 (0.0005) [2023-12-26 18:03:16,892][105620] Updated weights for policy 1, policy_version 370022 (0.0005) [2023-12-26 18:03:17,147][105692] Updated weights for policy 0, policy_version 369358 (0.0007) [2023-12-26 18:03:17,194][105692] Updated weights for policy 0, policy_version 369368 (0.0005) [2023-12-26 18:03:17,247][105692] Updated weights for policy 0, policy_version 369378 (0.0009) [2023-12-26 18:03:17,463][105620] Updated weights for policy 1, policy_version 370032 (0.0009) [2023-12-26 18:03:17,520][105620] Updated weights for policy 1, policy_version 370042 (0.0010) [2023-12-26 18:03:17,580][105620] Updated weights for policy 1, policy_version 370052 (0.0010) [2023-12-26 18:03:17,916][105692] Updated weights for policy 0, policy_version 369388 (0.0009) [2023-12-26 18:03:17,971][105692] Updated weights for policy 0, policy_version 369398 (0.0010) [2023-12-26 18:03:18,040][105692] Updated weights for policy 0, policy_version 369408 (0.0009) [2023-12-26 18:03:18,325][105620] Updated weights for policy 1, policy_version 370062 (0.0008) [2023-12-26 18:03:18,387][105620] Updated weights for policy 1, policy_version 370072 (0.0008) [2023-12-26 18:03:18,438][105620] Updated weights for policy 1, policy_version 370082 (0.0010) [2023-12-26 18:03:18,732][105692] Updated weights for policy 0, policy_version 369418 (0.0006) [2023-12-26 18:03:18,786][105692] Updated weights for policy 0, policy_version 369428 (0.0010) [2023-12-26 18:03:18,837][105692] Updated weights for policy 0, policy_version 369438 (0.0010) [2023-12-26 18:03:18,888][105692] Updated weights for policy 0, policy_version 369448 (0.0010) [2023-12-26 18:03:19,062][105620] Updated weights for policy 1, policy_version 370092 (0.0008) [2023-12-26 18:03:19,117][105620] Updated weights for policy 1, policy_version 370102 (0.0005) [2023-12-26 18:03:19,189][105620] Updated weights for policy 1, policy_version 370112 (0.0007) [2023-12-26 18:03:19,592][105692] Updated weights for policy 0, policy_version 369458 (0.0006) [2023-12-26 18:03:19,648][105692] Updated weights for policy 0, policy_version 369468 (0.0009) [2023-12-26 18:03:19,701][105692] Updated weights for policy 0, policy_version 369478 (0.0009) [2023-12-26 18:03:19,824][105620] Updated weights for policy 1, policy_version 370122 (0.0008) [2023-12-26 18:03:19,885][105620] Updated weights for policy 1, policy_version 370132 (0.0010) [2023-12-26 18:03:19,952][105620] Updated weights for policy 1, policy_version 370142 (0.0009) [2023-12-26 18:03:20,017][105620] Updated weights for policy 1, policy_version 370152 (0.0009) [2023-12-26 18:03:20,431][105692] Updated weights for policy 0, policy_version 369488 (0.0010) [2023-12-26 18:03:20,490][105692] Updated weights for policy 0, policy_version 369498 (0.0011) [2023-12-26 18:03:20,550][105692] Updated weights for policy 0, policy_version 369508 (0.0011) [2023-12-26 18:03:20,686][105620] Updated weights for policy 1, policy_version 370162 (0.0008) [2023-12-26 18:03:20,739][105620] Updated weights for policy 1, policy_version 370172 (0.0008) [2023-12-26 18:03:20,795][105620] Updated weights for policy 1, policy_version 370182 (0.0008) [2023-12-26 18:03:21,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 189382656. Throughput: 0: 9790.9, 1: 9712.2. Samples: 189371676. Policy #0 lag: (min: 16.0, avg: 40.4, max: 48.0) [2023-12-26 18:03:21,062][104569] Avg episode reward: [(0, '6422.633'), (1, '6188.472')] [2023-12-26 18:03:21,344][105692] Updated weights for policy 0, policy_version 369518 (0.0010) [2023-12-26 18:03:21,410][105692] Updated weights for policy 0, policy_version 369528 (0.0009) [2023-12-26 18:03:21,475][105620] Updated weights for policy 1, policy_version 370192 (0.0007) [2023-12-26 18:03:21,477][105692] Updated weights for policy 0, policy_version 369538 (0.0009) [2023-12-26 18:03:21,536][105620] Updated weights for policy 1, policy_version 370202 (0.0006) [2023-12-26 18:03:21,602][105620] Updated weights for policy 1, policy_version 370212 (0.0009) [2023-12-26 18:03:22,265][105692] Updated weights for policy 0, policy_version 369548 (0.0009) [2023-12-26 18:03:22,332][105692] Updated weights for policy 0, policy_version 369558 (0.0008) [2023-12-26 18:03:22,349][105620] Updated weights for policy 1, policy_version 370222 (0.0009) [2023-12-26 18:03:22,400][105692] Updated weights for policy 0, policy_version 369568 (0.0009) [2023-12-26 18:03:22,419][105620] Updated weights for policy 1, policy_version 370232 (0.0007) [2023-12-26 18:03:22,474][105620] Updated weights for policy 1, policy_version 370242 (0.0007) [2023-12-26 18:03:23,100][105692] Updated weights for policy 0, policy_version 369578 (0.0006) [2023-12-26 18:03:23,159][105692] Updated weights for policy 0, policy_version 369588 (0.0008) [2023-12-26 18:03:23,209][105692] Updated weights for policy 0, policy_version 369598 (0.0005) [2023-12-26 18:03:23,223][105620] Updated weights for policy 1, policy_version 370252 (0.0007) [2023-12-26 18:03:23,268][105692] Updated weights for policy 0, policy_version 369608 (0.0006) [2023-12-26 18:03:23,278][105620] Updated weights for policy 1, policy_version 370262 (0.0009) [2023-12-26 18:03:23,338][105620] Updated weights for policy 1, policy_version 370272 (0.0008) [2023-12-26 18:03:23,870][105692] Updated weights for policy 0, policy_version 369618 (0.0005) [2023-12-26 18:03:23,931][105692] Updated weights for policy 0, policy_version 369628 (0.0006) [2023-12-26 18:03:23,989][105692] Updated weights for policy 0, policy_version 369638 (0.0010) [2023-12-26 18:03:24,174][105620] Updated weights for policy 1, policy_version 370282 (0.0009) [2023-12-26 18:03:24,227][105620] Updated weights for policy 1, policy_version 370292 (0.0009) [2023-12-26 18:03:24,280][105620] Updated weights for policy 1, policy_version 370302 (0.0010) [2023-12-26 18:03:24,345][105620] Updated weights for policy 1, policy_version 370312 (0.0009) [2023-12-26 18:03:24,556][105692] Updated weights for policy 0, policy_version 369648 (0.0006) [2023-12-26 18:03:24,620][105692] Updated weights for policy 0, policy_version 369658 (0.0006) [2023-12-26 18:03:24,667][105692] Updated weights for policy 0, policy_version 369668 (0.0007) [2023-12-26 18:03:25,090][105620] Updated weights for policy 1, policy_version 370322 (0.0008) [2023-12-26 18:03:25,134][105620] Updated weights for policy 1, policy_version 370332 (0.0008) [2023-12-26 18:03:25,181][105620] Updated weights for policy 1, policy_version 370342 (0.0008) [2023-12-26 18:03:25,364][105692] Updated weights for policy 0, policy_version 369678 (0.0010) [2023-12-26 18:03:25,411][105692] Updated weights for policy 0, policy_version 369688 (0.0010) [2023-12-26 18:03:25,459][105692] Updated weights for policy 0, policy_version 369698 (0.0010) [2023-12-26 18:03:25,948][105620] Updated weights for policy 1, policy_version 370352 (0.0008) [2023-12-26 18:03:25,992][105620] Updated weights for policy 1, policy_version 370362 (0.0008) [2023-12-26 18:03:26,047][105620] Updated weights for policy 1, policy_version 370372 (0.0008) [2023-12-26 18:03:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 189472768. Throughput: 0: 9877.2, 1: 9662.3. Samples: 189486920. Policy #0 lag: (min: 16.0, avg: 40.4, max: 48.0) [2023-12-26 18:03:26,062][104569] Avg episode reward: [(0, '7601.420'), (1, '7176.583')] [2023-12-26 18:03:26,217][105692] Updated weights for policy 0, policy_version 369708 (0.0010) [2023-12-26 18:03:26,274][105692] Updated weights for policy 0, policy_version 369718 (0.0010) [2023-12-26 18:03:26,335][105692] Updated weights for policy 0, policy_version 369728 (0.0010) [2023-12-26 18:03:26,822][105620] Updated weights for policy 1, policy_version 370382 (0.0008) [2023-12-26 18:03:26,869][105620] Updated weights for policy 1, policy_version 370392 (0.0008) [2023-12-26 18:03:26,913][105620] Updated weights for policy 1, policy_version 370402 (0.0007) [2023-12-26 18:03:27,061][105692] Updated weights for policy 0, policy_version 369738 (0.0010) [2023-12-26 18:03:27,122][105692] Updated weights for policy 0, policy_version 369748 (0.0010) [2023-12-26 18:03:27,179][105692] Updated weights for policy 0, policy_version 369758 (0.0010) [2023-12-26 18:03:27,239][105692] Updated weights for policy 0, policy_version 369768 (0.0010) [2023-12-26 18:03:27,687][105620] Updated weights for policy 1, policy_version 370412 (0.0008) [2023-12-26 18:03:27,749][105620] Updated weights for policy 1, policy_version 370422 (0.0008) [2023-12-26 18:03:27,803][105620] Updated weights for policy 1, policy_version 370432 (0.0008) [2023-12-26 18:03:27,960][105692] Updated weights for policy 0, policy_version 369778 (0.0010) [2023-12-26 18:03:28,021][105692] Updated weights for policy 0, policy_version 369788 (0.0008) [2023-12-26 18:03:28,081][105692] Updated weights for policy 0, policy_version 369798 (0.0010) [2023-12-26 18:03:28,552][105620] Updated weights for policy 1, policy_version 370442 (0.0007) [2023-12-26 18:03:28,609][105620] Updated weights for policy 1, policy_version 370452 (0.0005) [2023-12-26 18:03:28,673][105620] Updated weights for policy 1, policy_version 370462 (0.0008) [2023-12-26 18:03:28,736][105620] Updated weights for policy 1, policy_version 370472 (0.0008) [2023-12-26 18:03:28,807][105692] Updated weights for policy 0, policy_version 369808 (0.0008) [2023-12-26 18:03:28,861][105692] Updated weights for policy 0, policy_version 369818 (0.0005) [2023-12-26 18:03:28,919][105692] Updated weights for policy 0, policy_version 369828 (0.0008) [2023-12-26 18:03:29,388][105620] Updated weights for policy 1, policy_version 370482 (0.0009) [2023-12-26 18:03:29,450][105620] Updated weights for policy 1, policy_version 370492 (0.0009) [2023-12-26 18:03:29,520][105620] Updated weights for policy 1, policy_version 370502 (0.0009) [2023-12-26 18:03:29,587][105692] Updated weights for policy 0, policy_version 369838 (0.0005) [2023-12-26 18:03:29,642][105692] Updated weights for policy 0, policy_version 369848 (0.0008) [2023-12-26 18:03:29,698][105692] Updated weights for policy 0, policy_version 369858 (0.0010) [2023-12-26 18:03:30,327][105620] Updated weights for policy 1, policy_version 370512 (0.0008) [2023-12-26 18:03:30,390][105620] Updated weights for policy 1, policy_version 370522 (0.0008) [2023-12-26 18:03:30,415][105692] Updated weights for policy 0, policy_version 369868 (0.0010) [2023-12-26 18:03:30,452][105620] Updated weights for policy 1, policy_version 370532 (0.0006) [2023-12-26 18:03:30,470][105692] Updated weights for policy 0, policy_version 369878 (0.0010) [2023-12-26 18:03:30,517][105692] Updated weights for policy 0, policy_version 369888 (0.0010) [2023-12-26 18:03:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 189571072. Throughput: 0: 9856.4, 1: 9707.3. Samples: 189543780. Policy #0 lag: (min: 16.0, avg: 40.4, max: 48.0) [2023-12-26 18:03:31,062][104569] Avg episode reward: [(0, '8558.360'), (1, '7907.998')] [2023-12-26 18:03:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000369896_94707712.pth... [2023-12-26 18:03:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000370536_94863360.pth... [2023-12-26 18:03:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000368744_94412800.pth [2023-12-26 18:03:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000369416_94576640.pth [2023-12-26 18:03:31,207][105620] Updated weights for policy 1, policy_version 370542 (0.0007) [2023-12-26 18:03:31,251][105692] Updated weights for policy 0, policy_version 369898 (0.0010) [2023-12-26 18:03:31,269][105620] Updated weights for policy 1, policy_version 370552 (0.0007) [2023-12-26 18:03:31,313][105692] Updated weights for policy 0, policy_version 369908 (0.0010) [2023-12-26 18:03:31,332][105620] Updated weights for policy 1, policy_version 370562 (0.0006) [2023-12-26 18:03:31,380][105692] Updated weights for policy 0, policy_version 369918 (0.0010) [2023-12-26 18:03:31,448][105692] Updated weights for policy 0, policy_version 369928 (0.0010) [2023-12-26 18:03:32,080][105620] Updated weights for policy 1, policy_version 370572 (0.0008) [2023-12-26 18:03:32,133][105620] Updated weights for policy 1, policy_version 370582 (0.0010) [2023-12-26 18:03:32,171][105692] Updated weights for policy 0, policy_version 369938 (0.0008) [2023-12-26 18:03:32,194][105620] Updated weights for policy 1, policy_version 370592 (0.0008) [2023-12-26 18:03:32,233][105692] Updated weights for policy 0, policy_version 369948 (0.0010) [2023-12-26 18:03:32,286][105692] Updated weights for policy 0, policy_version 369958 (0.0007) [2023-12-26 18:03:32,910][105692] Updated weights for policy 0, policy_version 369968 (0.0006) [2023-12-26 18:03:32,969][105692] Updated weights for policy 0, policy_version 369978 (0.0009) [2023-12-26 18:03:32,998][105620] Updated weights for policy 1, policy_version 370602 (0.0006) [2023-12-26 18:03:33,028][105692] Updated weights for policy 0, policy_version 369988 (0.0010) [2023-12-26 18:03:33,057][105620] Updated weights for policy 1, policy_version 370612 (0.0006) [2023-12-26 18:03:33,113][105620] Updated weights for policy 1, policy_version 370622 (0.0008) [2023-12-26 18:03:33,180][105620] Updated weights for policy 1, policy_version 370632 (0.0006) [2023-12-26 18:03:33,587][105692] Updated weights for policy 0, policy_version 369998 (0.0007) [2023-12-26 18:03:33,641][105692] Updated weights for policy 0, policy_version 370008 (0.0005) [2023-12-26 18:03:33,694][105692] Updated weights for policy 0, policy_version 370018 (0.0005) [2023-12-26 18:03:34,033][105620] Updated weights for policy 1, policy_version 370642 (0.0009) [2023-12-26 18:03:34,086][105620] Updated weights for policy 1, policy_version 370654 (0.0010) [2023-12-26 18:03:34,222][105692] Updated weights for policy 0, policy_version 370028 (0.0007) [2023-12-26 18:03:34,275][105692] Updated weights for policy 0, policy_version 370038 (0.0009) [2023-12-26 18:03:34,334][105692] Updated weights for policy 0, policy_version 370048 (0.0009) [2023-12-26 18:03:34,923][105620] Updated weights for policy 1, policy_version 370666 (0.0009) [2023-12-26 18:03:34,979][105620] Updated weights for policy 1, policy_version 370676 (0.0008) [2023-12-26 18:03:35,026][105620] Updated weights for policy 1, policy_version 370686 (0.0009) [2023-12-26 18:03:35,038][105692] Updated weights for policy 0, policy_version 370058 (0.0008) [2023-12-26 18:03:35,068][105620] Updated weights for policy 1, policy_version 370696 (0.0007) [2023-12-26 18:03:35,089][105692] Updated weights for policy 0, policy_version 370068 (0.0007) [2023-12-26 18:03:35,141][105692] Updated weights for policy 0, policy_version 370078 (0.0006) [2023-12-26 18:03:35,196][105692] Updated weights for policy 0, policy_version 370088 (0.0005) [2023-12-26 18:03:35,807][105692] Updated weights for policy 0, policy_version 370098 (0.0009) [2023-12-26 18:03:35,856][105692] Updated weights for policy 0, policy_version 370108 (0.0008) [2023-12-26 18:03:35,875][105620] Updated weights for policy 1, policy_version 370706 (0.0009) [2023-12-26 18:03:35,917][105692] Updated weights for policy 0, policy_version 370118 (0.0006) [2023-12-26 18:03:35,925][105620] Updated weights for policy 1, policy_version 370716 (0.0008) [2023-12-26 18:03:35,976][105620] Updated weights for policy 1, policy_version 370726 (0.0010) [2023-12-26 18:03:36,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.7, 300 sec: 19466.4). Total num frames: 189677568. Throughput: 0: 9886.1, 1: 9624.7. Samples: 189659716. Policy #0 lag: (min: 16.0, avg: 40.4, max: 48.0) [2023-12-26 18:03:36,063][104569] Avg episode reward: [(0, '8914.022'), (1, '8627.023')] [2023-12-26 18:03:36,577][105692] Updated weights for policy 0, policy_version 370128 (0.0008) [2023-12-26 18:03:36,634][105692] Updated weights for policy 0, policy_version 370138 (0.0011) [2023-12-26 18:03:36,692][105692] Updated weights for policy 0, policy_version 370148 (0.0010) [2023-12-26 18:03:36,705][105620] Updated weights for policy 1, policy_version 370736 (0.0007) [2023-12-26 18:03:36,762][105620] Updated weights for policy 1, policy_version 370746 (0.0009) [2023-12-26 18:03:36,815][105620] Updated weights for policy 1, policy_version 370756 (0.0008) [2023-12-26 18:03:37,309][105692] Updated weights for policy 0, policy_version 370158 (0.0009) [2023-12-26 18:03:37,368][105692] Updated weights for policy 0, policy_version 370168 (0.0010) [2023-12-26 18:03:37,430][105692] Updated weights for policy 0, policy_version 370178 (0.0010) [2023-12-26 18:03:37,585][105620] Updated weights for policy 1, policy_version 370766 (0.0007) [2023-12-26 18:03:37,643][105620] Updated weights for policy 1, policy_version 370776 (0.0008) [2023-12-26 18:03:37,702][105620] Updated weights for policy 1, policy_version 370786 (0.0008) [2023-12-26 18:03:38,128][105692] Updated weights for policy 0, policy_version 370188 (0.0011) [2023-12-26 18:03:38,178][105692] Updated weights for policy 0, policy_version 370198 (0.0011) [2023-12-26 18:03:38,235][105692] Updated weights for policy 0, policy_version 370208 (0.0011) [2023-12-26 18:03:38,320][105620] Updated weights for policy 1, policy_version 370796 (0.0007) [2023-12-26 18:03:38,387][105620] Updated weights for policy 1, policy_version 370806 (0.0006) [2023-12-26 18:03:38,443][105620] Updated weights for policy 1, policy_version 370816 (0.0006) [2023-12-26 18:03:38,965][105692] Updated weights for policy 0, policy_version 370218 (0.0010) [2023-12-26 18:03:39,034][105692] Updated weights for policy 0, policy_version 370228 (0.0006) [2023-12-26 18:03:39,100][105692] Updated weights for policy 0, policy_version 370238 (0.0005) [2023-12-26 18:03:39,153][105620] Updated weights for policy 1, policy_version 370826 (0.0009) [2023-12-26 18:03:39,164][105692] Updated weights for policy 0, policy_version 370248 (0.0006) [2023-12-26 18:03:39,215][105620] Updated weights for policy 1, policy_version 370836 (0.0010) [2023-12-26 18:03:39,283][105620] Updated weights for policy 1, policy_version 370846 (0.0009) [2023-12-26 18:03:39,351][105620] Updated weights for policy 1, policy_version 370856 (0.0010) [2023-12-26 18:03:39,811][105692] Updated weights for policy 0, policy_version 370258 (0.0008) [2023-12-26 18:03:39,876][105692] Updated weights for policy 0, policy_version 370268 (0.0008) [2023-12-26 18:03:39,939][105692] Updated weights for policy 0, policy_version 370278 (0.0009) [2023-12-26 18:03:40,159][105620] Updated weights for policy 1, policy_version 370866 (0.0011) [2023-12-26 18:03:40,226][105620] Updated weights for policy 1, policy_version 370876 (0.0007) [2023-12-26 18:03:40,289][105620] Updated weights for policy 1, policy_version 370886 (0.0006) [2023-12-26 18:03:40,682][105692] Updated weights for policy 0, policy_version 370288 (0.0010) [2023-12-26 18:03:40,736][105692] Updated weights for policy 0, policy_version 370298 (0.0010) [2023-12-26 18:03:40,785][105692] Updated weights for policy 0, policy_version 370309 (0.0009) [2023-12-26 18:03:40,825][105620] Updated weights for policy 1, policy_version 370896 (0.0010) [2023-12-26 18:03:40,874][105620] Updated weights for policy 1, policy_version 370906 (0.0011) [2023-12-26 18:03:40,931][105620] Updated weights for policy 1, policy_version 370916 (0.0011) [2023-12-26 18:03:41,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 189775872. Throughput: 0: 9969.5, 1: 9614.8. Samples: 189779960. Policy #0 lag: (min: 16.0, avg: 40.4, max: 48.0) [2023-12-26 18:03:41,063][104569] Avg episode reward: [(0, '9003.005'), (1, '8444.520')] [2023-12-26 18:03:41,652][105692] Updated weights for policy 0, policy_version 370319 (0.0008) [2023-12-26 18:03:41,718][105692] Updated weights for policy 0, policy_version 370329 (0.0009) [2023-12-26 18:03:41,751][105620] Updated weights for policy 1, policy_version 370926 (0.0011) [2023-12-26 18:03:41,785][105692] Updated weights for policy 0, policy_version 370339 (0.0008) [2023-12-26 18:03:41,809][105620] Updated weights for policy 1, policy_version 370936 (0.0010) [2023-12-26 18:03:41,869][105620] Updated weights for policy 1, policy_version 370946 (0.0011) [2023-12-26 18:03:42,566][105692] Updated weights for policy 0, policy_version 370349 (0.0008) [2023-12-26 18:03:42,619][105692] Updated weights for policy 0, policy_version 370359 (0.0008) [2023-12-26 18:03:42,665][105620] Updated weights for policy 1, policy_version 370956 (0.0011) [2023-12-26 18:03:42,670][105692] Updated weights for policy 0, policy_version 370369 (0.0008) [2023-12-26 18:03:42,720][105620] Updated weights for policy 1, policy_version 370966 (0.0010) [2023-12-26 18:03:42,775][105620] Updated weights for policy 1, policy_version 370976 (0.0010) [2023-12-26 18:03:43,444][105692] Updated weights for policy 0, policy_version 370379 (0.0007) [2023-12-26 18:03:43,493][105692] Updated weights for policy 0, policy_version 370389 (0.0008) [2023-12-26 18:03:43,530][105620] Updated weights for policy 1, policy_version 370986 (0.0010) [2023-12-26 18:03:43,544][105692] Updated weights for policy 0, policy_version 370399 (0.0007) [2023-12-26 18:03:43,581][105620] Updated weights for policy 1, policy_version 370996 (0.0010) [2023-12-26 18:03:43,638][105620] Updated weights for policy 1, policy_version 371006 (0.0010) [2023-12-26 18:03:43,685][105620] Updated weights for policy 1, policy_version 371016 (0.0010) [2023-12-26 18:03:44,311][105692] Updated weights for policy 0, policy_version 370409 (0.0009) [2023-12-26 18:03:44,367][105692] Updated weights for policy 0, policy_version 370419 (0.0008) [2023-12-26 18:03:44,410][105692] Updated weights for policy 0, policy_version 370429 (0.0007) [2023-12-26 18:03:44,436][105620] Updated weights for policy 1, policy_version 371026 (0.0010) [2023-12-26 18:03:44,458][105692] Updated weights for policy 0, policy_version 370439 (0.0005) [2023-12-26 18:03:44,480][105620] Updated weights for policy 1, policy_version 371036 (0.0010) [2023-12-26 18:03:44,496][105586] KL-divergence is very high: 318.2820 [2023-12-26 18:03:44,501][105586] KL-divergence is very high: 321.0944 [2023-12-26 18:03:44,525][105620] Updated weights for policy 1, policy_version 371046 (0.0010) [2023-12-26 18:03:45,184][105620] Updated weights for policy 1, policy_version 371056 (0.0011) [2023-12-26 18:03:45,243][105620] Updated weights for policy 1, policy_version 371066 (0.0011) [2023-12-26 18:03:45,298][105620] Updated weights for policy 1, policy_version 371076 (0.0009) [2023-12-26 18:03:45,304][105692] Updated weights for policy 0, policy_version 370449 (0.0007) [2023-12-26 18:03:45,358][105692] Updated weights for policy 0, policy_version 370459 (0.0005) [2023-12-26 18:03:45,413][105692] Updated weights for policy 0, policy_version 370469 (0.0007) [2023-12-26 18:03:46,002][105620] Updated weights for policy 1, policy_version 371086 (0.0009) [2023-12-26 18:03:46,048][105620] Updated weights for policy 1, policy_version 371096 (0.0008) [2023-12-26 18:03:46,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 189857792. Throughput: 0: 9868.3, 1: 9550.8. Samples: 189833540. Policy #0 lag: (min: 16.0, avg: 40.4, max: 48.0) [2023-12-26 18:03:46,063][104569] Avg episode reward: [(0, '9088.572'), (1, '8069.007')] [2023-12-26 18:03:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000370472_94855168.pth... [2023-12-26 18:03:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000369320_94560256.pth [2023-12-26 18:03:46,095][105620] Updated weights for policy 1, policy_version 371106 (0.0008) [2023-12-26 18:03:46,120][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000371112_95010816.pth... [2023-12-26 18:03:46,123][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000369992_94724096.pth [2023-12-26 18:03:46,152][105692] Updated weights for policy 0, policy_version 370479 (0.0008) [2023-12-26 18:03:46,200][105692] Updated weights for policy 0, policy_version 370489 (0.0009) [2023-12-26 18:03:46,261][105692] Updated weights for policy 0, policy_version 370499 (0.0008) [2023-12-26 18:03:46,900][105620] Updated weights for policy 1, policy_version 371116 (0.0008) [2023-12-26 18:03:46,963][105620] Updated weights for policy 1, policy_version 371126 (0.0005) [2023-12-26 18:03:46,970][105692] Updated weights for policy 0, policy_version 370509 (0.0008) [2023-12-26 18:03:47,024][105620] Updated weights for policy 1, policy_version 371136 (0.0005) [2023-12-26 18:03:47,032][105692] Updated weights for policy 0, policy_version 370519 (0.0009) [2023-12-26 18:03:47,090][105692] Updated weights for policy 0, policy_version 370530 (0.0009) [2023-12-26 18:03:47,592][105620] Updated weights for policy 1, policy_version 371146 (0.0006) [2023-12-26 18:03:47,645][105620] Updated weights for policy 1, policy_version 371156 (0.0009) [2023-12-26 18:03:47,695][105620] Updated weights for policy 1, policy_version 371166 (0.0008) [2023-12-26 18:03:47,752][105620] Updated weights for policy 1, policy_version 371176 (0.0009) [2023-12-26 18:03:47,901][105692] Updated weights for policy 0, policy_version 370540 (0.0010) [2023-12-26 18:03:47,955][105692] Updated weights for policy 0, policy_version 370550 (0.0009) [2023-12-26 18:03:48,014][105692] Updated weights for policy 0, policy_version 370560 (0.0009) [2023-12-26 18:03:48,525][105620] Updated weights for policy 1, policy_version 371186 (0.0009) [2023-12-26 18:03:48,590][105620] Updated weights for policy 1, policy_version 371196 (0.0009) [2023-12-26 18:03:48,653][105620] Updated weights for policy 1, policy_version 371206 (0.0010) [2023-12-26 18:03:48,710][105692] Updated weights for policy 0, policy_version 370570 (0.0009) [2023-12-26 18:03:48,769][105692] Updated weights for policy 0, policy_version 370580 (0.0009) [2023-12-26 18:03:48,825][105692] Updated weights for policy 0, policy_version 370590 (0.0009) [2023-12-26 18:03:48,884][105692] Updated weights for policy 0, policy_version 370600 (0.0010) [2023-12-26 18:03:49,342][105620] Updated weights for policy 1, policy_version 371216 (0.0010) [2023-12-26 18:03:49,403][105620] Updated weights for policy 1, policy_version 371226 (0.0009) [2023-12-26 18:03:49,454][105620] Updated weights for policy 1, policy_version 371236 (0.0009) [2023-12-26 18:03:49,729][105692] Updated weights for policy 0, policy_version 370610 (0.0009) [2023-12-26 18:03:49,790][105692] Updated weights for policy 0, policy_version 370620 (0.0008) [2023-12-26 18:03:49,855][105692] Updated weights for policy 0, policy_version 370630 (0.0008) [2023-12-26 18:03:50,141][105620] Updated weights for policy 1, policy_version 371246 (0.0008) [2023-12-26 18:03:50,199][105620] Updated weights for policy 1, policy_version 371256 (0.0009) [2023-12-26 18:03:50,253][105620] Updated weights for policy 1, policy_version 371266 (0.0007) [2023-12-26 18:03:50,518][105692] Updated weights for policy 0, policy_version 370640 (0.0009) [2023-12-26 18:03:50,581][105692] Updated weights for policy 0, policy_version 370650 (0.0009) [2023-12-26 18:03:50,636][105692] Updated weights for policy 0, policy_version 370660 (0.0008) [2023-12-26 18:03:50,934][105620] Updated weights for policy 1, policy_version 371276 (0.0008) [2023-12-26 18:03:50,984][105620] Updated weights for policy 1, policy_version 371286 (0.0010) [2023-12-26 18:03:51,043][105620] Updated weights for policy 1, policy_version 371296 (0.0009) [2023-12-26 18:03:51,062][104569] Fps is (10 sec: 18022.6, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 189956096. Throughput: 0: 9834.8, 1: 9542.3. Samples: 189948908. Policy #0 lag: (min: 16.0, avg: 40.4, max: 48.0) [2023-12-26 18:03:51,062][104569] Avg episode reward: [(0, '8815.820'), (1, '7606.279')] [2023-12-26 18:03:51,414][105692] Updated weights for policy 0, policy_version 370670 (0.0008) [2023-12-26 18:03:51,480][105692] Updated weights for policy 0, policy_version 370680 (0.0010) [2023-12-26 18:03:51,547][105692] Updated weights for policy 0, policy_version 370690 (0.0008) [2023-12-26 18:03:51,731][105620] Updated weights for policy 1, policy_version 371306 (0.0007) [2023-12-26 18:03:51,789][105620] Updated weights for policy 1, policy_version 371316 (0.0009) [2023-12-26 18:03:51,846][105620] Updated weights for policy 1, policy_version 371326 (0.0005) [2023-12-26 18:03:51,904][105620] Updated weights for policy 1, policy_version 371336 (0.0010) [2023-12-26 18:03:52,236][105692] Updated weights for policy 0, policy_version 370700 (0.0006) [2023-12-26 18:03:52,304][105692] Updated weights for policy 0, policy_version 370710 (0.0007) [2023-12-26 18:03:52,365][105692] Updated weights for policy 0, policy_version 370720 (0.0008) [2023-12-26 18:03:52,602][105620] Updated weights for policy 1, policy_version 371346 (0.0011) [2023-12-26 18:03:52,668][105620] Updated weights for policy 1, policy_version 371356 (0.0010) [2023-12-26 18:03:52,727][105620] Updated weights for policy 1, policy_version 371366 (0.0010) [2023-12-26 18:03:53,009][105692] Updated weights for policy 0, policy_version 370730 (0.0008) [2023-12-26 18:03:53,069][105692] Updated weights for policy 0, policy_version 370740 (0.0009) [2023-12-26 18:03:53,133][105692] Updated weights for policy 0, policy_version 370750 (0.0008) [2023-12-26 18:03:53,178][105692] Updated weights for policy 0, policy_version 370760 (0.0008) [2023-12-26 18:03:53,473][105620] Updated weights for policy 1, policy_version 371376 (0.0010) [2023-12-26 18:03:53,525][105620] Updated weights for policy 1, policy_version 371386 (0.0009) [2023-12-26 18:03:53,578][105620] Updated weights for policy 1, policy_version 371396 (0.0008) [2023-12-26 18:03:53,984][105692] Updated weights for policy 0, policy_version 370770 (0.0009) [2023-12-26 18:03:54,048][105692] Updated weights for policy 0, policy_version 370780 (0.0006) [2023-12-26 18:03:54,105][105692] Updated weights for policy 0, policy_version 370790 (0.0005) [2023-12-26 18:03:54,343][105620] Updated weights for policy 1, policy_version 371406 (0.0009) [2023-12-26 18:03:54,400][105620] Updated weights for policy 1, policy_version 371416 (0.0008) [2023-12-26 18:03:54,463][105620] Updated weights for policy 1, policy_version 371426 (0.0009) [2023-12-26 18:03:54,751][105692] Updated weights for policy 0, policy_version 370800 (0.0008) [2023-12-26 18:03:54,798][105692] Updated weights for policy 0, policy_version 370810 (0.0009) [2023-12-26 18:03:54,851][105692] Updated weights for policy 0, policy_version 370820 (0.0009) [2023-12-26 18:03:55,223][105620] Updated weights for policy 1, policy_version 371436 (0.0009) [2023-12-26 18:03:55,283][105620] Updated weights for policy 1, policy_version 371446 (0.0008) [2023-12-26 18:03:55,338][105620] Updated weights for policy 1, policy_version 371456 (0.0009) [2023-12-26 18:03:55,554][105692] Updated weights for policy 0, policy_version 370830 (0.0008) [2023-12-26 18:03:55,600][105692] Updated weights for policy 0, policy_version 370840 (0.0009) [2023-12-26 18:03:55,646][105692] Updated weights for policy 0, policy_version 370850 (0.0008) [2023-12-26 18:03:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 190054400. Throughput: 0: 9799.4, 1: 9521.3. Samples: 190064608. Policy #0 lag: (min: 16.0, avg: 40.4, max: 48.0) [2023-12-26 18:03:56,063][104569] Avg episode reward: [(0, '8729.059'), (1, '7886.053')] [2023-12-26 18:03:56,087][105620] Updated weights for policy 1, policy_version 371466 (0.0009) [2023-12-26 18:03:56,144][105620] Updated weights for policy 1, policy_version 371476 (0.0009) [2023-12-26 18:03:56,195][105620] Updated weights for policy 1, policy_version 371486 (0.0007) [2023-12-26 18:03:56,252][105620] Updated weights for policy 1, policy_version 371496 (0.0009) [2023-12-26 18:03:56,410][105692] Updated weights for policy 0, policy_version 370860 (0.0009) [2023-12-26 18:03:56,459][105692] Updated weights for policy 0, policy_version 370870 (0.0009) [2023-12-26 18:03:56,506][105692] Updated weights for policy 0, policy_version 370880 (0.0009) [2023-12-26 18:03:57,007][105620] Updated weights for policy 1, policy_version 371506 (0.0009) [2023-12-26 18:03:57,042][105586] KL-divergence is very high: 111.7819 [2023-12-26 18:03:57,053][105620] Updated weights for policy 1, policy_version 371516 (0.0009) [2023-12-26 18:03:57,061][105586] KL-divergence is very high: 146.4005 [2023-12-26 18:03:57,079][105586] KL-divergence is very high: 133.4799 [2023-12-26 18:03:57,097][105586] KL-divergence is very high: 149.9227 [2023-12-26 18:03:57,099][105620] Updated weights for policy 1, policy_version 371526 (0.0009) [2023-12-26 18:03:57,277][105692] Updated weights for policy 0, policy_version 370890 (0.0009) [2023-12-26 18:03:57,333][105692] Updated weights for policy 0, policy_version 370900 (0.0010) [2023-12-26 18:03:57,381][105692] Updated weights for policy 0, policy_version 370910 (0.0008) [2023-12-26 18:03:57,430][105692] Updated weights for policy 0, policy_version 370920 (0.0008) [2023-12-26 18:03:57,867][105620] Updated weights for policy 1, policy_version 371536 (0.0009) [2023-12-26 18:03:57,919][105620] Updated weights for policy 1, policy_version 371546 (0.0009) [2023-12-26 18:03:57,971][105620] Updated weights for policy 1, policy_version 371557 (0.0009) [2023-12-26 18:03:58,119][105692] Updated weights for policy 0, policy_version 370930 (0.0009) [2023-12-26 18:03:58,185][105692] Updated weights for policy 0, policy_version 370940 (0.0008) [2023-12-26 18:03:58,240][105692] Updated weights for policy 0, policy_version 370950 (0.0008) [2023-12-26 18:03:58,776][105620] Updated weights for policy 1, policy_version 371568 (0.0008) [2023-12-26 18:03:58,846][105620] Updated weights for policy 1, policy_version 371578 (0.0010) [2023-12-26 18:03:58,917][105620] Updated weights for policy 1, policy_version 371588 (0.0010) [2023-12-26 18:03:59,088][105692] Updated weights for policy 0, policy_version 370960 (0.0008) [2023-12-26 18:03:59,153][105692] Updated weights for policy 0, policy_version 370970 (0.0007) [2023-12-26 18:03:59,214][105692] Updated weights for policy 0, policy_version 370980 (0.0008) [2023-12-26 18:03:59,691][105620] Updated weights for policy 1, policy_version 371598 (0.0010) [2023-12-26 18:03:59,762][105620] Updated weights for policy 1, policy_version 371608 (0.0010) [2023-12-26 18:03:59,833][105620] Updated weights for policy 1, policy_version 371618 (0.0010) [2023-12-26 18:03:59,868][105692] Updated weights for policy 0, policy_version 370990 (0.0008) [2023-12-26 18:03:59,934][105692] Updated weights for policy 0, policy_version 371000 (0.0007) [2023-12-26 18:03:59,993][105692] Updated weights for policy 0, policy_version 371010 (0.0006) [2023-12-26 18:04:00,558][105620] Updated weights for policy 1, policy_version 371628 (0.0011) [2023-12-26 18:04:00,619][105620] Updated weights for policy 1, policy_version 371638 (0.0010) [2023-12-26 18:04:00,678][105692] Updated weights for policy 0, policy_version 371020 (0.0005) [2023-12-26 18:04:00,679][105620] Updated weights for policy 1, policy_version 371648 (0.0010) [2023-12-26 18:04:00,736][105692] Updated weights for policy 0, policy_version 371030 (0.0005) [2023-12-26 18:04:00,784][105692] Updated weights for policy 0, policy_version 371040 (0.0005) [2023-12-26 18:04:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19438.7). Total num frames: 190152704. Throughput: 0: 9840.7, 1: 9531.0. Samples: 190120884. Policy #0 lag: (min: 16.0, avg: 40.4, max: 48.0) [2023-12-26 18:04:01,062][104569] Avg episode reward: [(0, '8821.195'), (1, '7792.744')] [2023-12-26 18:04:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000371048_95002624.pth... [2023-12-26 18:04:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000371656_95150080.pth... [2023-12-26 18:04:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000369896_94707712.pth [2023-12-26 18:04:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000370536_94863360.pth [2023-12-26 18:04:01,344][105620] Updated weights for policy 1, policy_version 371658 (0.0010) [2023-12-26 18:04:01,408][105620] Updated weights for policy 1, policy_version 371668 (0.0011) [2023-12-26 18:04:01,445][105692] Updated weights for policy 0, policy_version 371050 (0.0007) [2023-12-26 18:04:01,453][105620] Updated weights for policy 1, policy_version 371678 (0.0010) [2023-12-26 18:04:01,497][105692] Updated weights for policy 0, policy_version 371060 (0.0005) [2023-12-26 18:04:01,508][105620] Updated weights for policy 1, policy_version 371688 (0.0010) [2023-12-26 18:04:01,553][105692] Updated weights for policy 0, policy_version 371070 (0.0005) [2023-12-26 18:04:01,615][105692] Updated weights for policy 0, policy_version 371080 (0.0006) [2023-12-26 18:04:02,239][105692] Updated weights for policy 0, policy_version 371090 (0.0006) [2023-12-26 18:04:02,283][105620] Updated weights for policy 1, policy_version 371698 (0.0009) [2023-12-26 18:04:02,301][105692] Updated weights for policy 0, policy_version 371100 (0.0007) [2023-12-26 18:04:02,344][105620] Updated weights for policy 1, policy_version 371708 (0.0008) [2023-12-26 18:04:02,359][105692] Updated weights for policy 0, policy_version 371110 (0.0008) [2023-12-26 18:04:02,412][105620] Updated weights for policy 1, policy_version 371718 (0.0010) [2023-12-26 18:04:03,003][105692] Updated weights for policy 0, policy_version 371120 (0.0008) [2023-12-26 18:04:03,068][105692] Updated weights for policy 0, policy_version 371130 (0.0008) [2023-12-26 18:04:03,123][105620] Updated weights for policy 1, policy_version 371728 (0.0010) [2023-12-26 18:04:03,133][105692] Updated weights for policy 0, policy_version 371140 (0.0005) [2023-12-26 18:04:03,181][105620] Updated weights for policy 1, policy_version 371738 (0.0010) [2023-12-26 18:04:03,235][105620] Updated weights for policy 1, policy_version 371748 (0.0010) [2023-12-26 18:04:03,797][105692] Updated weights for policy 0, policy_version 371150 (0.0007) [2023-12-26 18:04:03,861][105692] Updated weights for policy 0, policy_version 371160 (0.0009) [2023-12-26 18:04:03,882][105620] Updated weights for policy 1, policy_version 371758 (0.0008) [2023-12-26 18:04:03,918][105692] Updated weights for policy 0, policy_version 371170 (0.0009) [2023-12-26 18:04:03,944][105620] Updated weights for policy 1, policy_version 371768 (0.0006) [2023-12-26 18:04:03,995][105620] Updated weights for policy 1, policy_version 371778 (0.0008) [2023-12-26 18:04:04,580][105692] Updated weights for policy 0, policy_version 371180 (0.0011) [2023-12-26 18:04:04,642][105692] Updated weights for policy 0, policy_version 371190 (0.0010) [2023-12-26 18:04:04,697][105692] Updated weights for policy 0, policy_version 371200 (0.0010) [2023-12-26 18:04:04,729][105620] Updated weights for policy 1, policy_version 371788 (0.0007) [2023-12-26 18:04:04,776][105620] Updated weights for policy 1, policy_version 371798 (0.0005) [2023-12-26 18:04:04,825][105620] Updated weights for policy 1, policy_version 371808 (0.0009) [2023-12-26 18:04:05,329][105692] Updated weights for policy 0, policy_version 371210 (0.0009) [2023-12-26 18:04:05,389][105692] Updated weights for policy 0, policy_version 371220 (0.0010) [2023-12-26 18:04:05,391][105620] Updated weights for policy 1, policy_version 371818 (0.0005) [2023-12-26 18:04:05,442][105620] Updated weights for policy 1, policy_version 371828 (0.0007) [2023-12-26 18:04:05,448][105692] Updated weights for policy 0, policy_version 371230 (0.0005) [2023-12-26 18:04:05,505][105620] Updated weights for policy 1, policy_version 371838 (0.0005) [2023-12-26 18:04:05,506][105692] Updated weights for policy 0, policy_version 371240 (0.0009) [2023-12-26 18:04:05,556][105620] Updated weights for policy 1, policy_version 371848 (0.0006) [2023-12-26 18:04:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 190251008. Throughput: 0: 9816.4, 1: 9484.8. Samples: 190240236. Policy #0 lag: (min: 16.0, avg: 40.4, max: 48.0) [2023-12-26 18:04:06,063][104569] Avg episode reward: [(0, '8819.517'), (1, '7518.042')] [2023-12-26 18:04:06,201][105692] Updated weights for policy 0, policy_version 371250 (0.0010) [2023-12-26 18:04:06,219][105620] Updated weights for policy 1, policy_version 371858 (0.0006) [2023-12-26 18:04:06,260][105692] Updated weights for policy 0, policy_version 371260 (0.0011) [2023-12-26 18:04:06,278][105620] Updated weights for policy 1, policy_version 371868 (0.0006) [2023-12-26 18:04:06,316][105692] Updated weights for policy 0, policy_version 371270 (0.0010) [2023-12-26 18:04:06,335][105620] Updated weights for policy 1, policy_version 371878 (0.0007) [2023-12-26 18:04:07,059][105692] Updated weights for policy 0, policy_version 371280 (0.0010) [2023-12-26 18:04:07,118][105692] Updated weights for policy 0, policy_version 371290 (0.0011) [2023-12-26 18:04:07,129][105620] Updated weights for policy 1, policy_version 371888 (0.0006) [2023-12-26 18:04:07,174][105692] Updated weights for policy 0, policy_version 371300 (0.0011) [2023-12-26 18:04:07,181][105620] Updated weights for policy 1, policy_version 371898 (0.0005) [2023-12-26 18:04:07,229][105620] Updated weights for policy 1, policy_version 371908 (0.0008) [2023-12-26 18:04:07,768][105692] Updated weights for policy 0, policy_version 371310 (0.0010) [2023-12-26 18:04:07,834][105692] Updated weights for policy 0, policy_version 371320 (0.0011) [2023-12-26 18:04:07,900][105692] Updated weights for policy 0, policy_version 371330 (0.0011) [2023-12-26 18:04:07,947][105620] Updated weights for policy 1, policy_version 371918 (0.0006) [2023-12-26 18:04:07,995][105620] Updated weights for policy 1, policy_version 371928 (0.0010) [2023-12-26 18:04:08,047][105620] Updated weights for policy 1, policy_version 371938 (0.0010) [2023-12-26 18:04:08,582][105692] Updated weights for policy 0, policy_version 371340 (0.0010) [2023-12-26 18:04:08,651][105692] Updated weights for policy 0, policy_version 371350 (0.0011) [2023-12-26 18:04:08,710][105620] Updated weights for policy 1, policy_version 371948 (0.0010) [2023-12-26 18:04:08,712][105692] Updated weights for policy 0, policy_version 371360 (0.0010) [2023-12-26 18:04:08,769][105620] Updated weights for policy 1, policy_version 371958 (0.0008) [2023-12-26 18:04:08,831][105620] Updated weights for policy 1, policy_version 371968 (0.0010) [2023-12-26 18:04:09,466][105692] Updated weights for policy 0, policy_version 371370 (0.0011) [2023-12-26 18:04:09,483][105620] Updated weights for policy 1, policy_version 371978 (0.0010) [2023-12-26 18:04:09,516][105692] Updated weights for policy 0, policy_version 371380 (0.0011) [2023-12-26 18:04:09,539][105620] Updated weights for policy 1, policy_version 371988 (0.0011) [2023-12-26 18:04:09,541][105586] KL-divergence is very high: 103.9830 [2023-12-26 18:04:09,568][105692] Updated weights for policy 0, policy_version 371390 (0.0011) [2023-12-26 18:04:09,573][105586] KL-divergence is very high: 167.3254 [2023-12-26 18:04:09,592][105586] KL-divergence is very high: 123.8603 [2023-12-26 18:04:09,606][105620] Updated weights for policy 1, policy_version 371998 (0.0011) [2023-12-26 18:04:09,610][105586] KL-divergence is very high: 101.3831 [2023-12-26 18:04:09,622][105586] KL-divergence is very high: 107.6862 [2023-12-26 18:04:09,627][105692] Updated weights for policy 0, policy_version 371400 (0.0011) [2023-12-26 18:04:09,659][105620] Updated weights for policy 1, policy_version 372008 (0.0011) [2023-12-26 18:04:10,427][105692] Updated weights for policy 0, policy_version 371410 (0.0010) [2023-12-26 18:04:10,467][105620] Updated weights for policy 1, policy_version 372018 (0.0011) [2023-12-26 18:04:10,490][105692] Updated weights for policy 0, policy_version 371420 (0.0011) [2023-12-26 18:04:10,525][105620] Updated weights for policy 1, policy_version 372028 (0.0010) [2023-12-26 18:04:10,551][105692] Updated weights for policy 0, policy_version 371430 (0.0011) [2023-12-26 18:04:10,590][105620] Updated weights for policy 1, policy_version 372038 (0.0005) [2023-12-26 18:04:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 190349312. Throughput: 0: 9818.7, 1: 9578.8. Samples: 190359808. Policy #0 lag: (min: 16.0, avg: 40.4, max: 48.0) [2023-12-26 18:04:11,062][104569] Avg episode reward: [(0, '9087.822'), (1, '7527.504')] [2023-12-26 18:04:11,177][105692] Updated weights for policy 0, policy_version 371440 (0.0009) [2023-12-26 18:04:11,230][105620] Updated weights for policy 1, policy_version 372048 (0.0010) [2023-12-26 18:04:11,240][105692] Updated weights for policy 0, policy_version 371450 (0.0011) [2023-12-26 18:04:11,294][105620] Updated weights for policy 1, policy_version 372058 (0.0007) [2023-12-26 18:04:11,317][105692] Updated weights for policy 0, policy_version 371460 (0.0008) [2023-12-26 18:04:11,361][105620] Updated weights for policy 1, policy_version 372068 (0.0006) [2023-12-26 18:04:12,093][105620] Updated weights for policy 1, policy_version 372078 (0.0009) [2023-12-26 18:04:12,138][105692] Updated weights for policy 0, policy_version 371470 (0.0007) [2023-12-26 18:04:12,161][105620] Updated weights for policy 1, policy_version 372088 (0.0007) [2023-12-26 18:04:12,198][105692] Updated weights for policy 0, policy_version 371480 (0.0008) [2023-12-26 18:04:12,220][105620] Updated weights for policy 1, policy_version 372098 (0.0008) [2023-12-26 18:04:12,254][105692] Updated weights for policy 0, policy_version 371490 (0.0011) [2023-12-26 18:04:12,841][105620] Updated weights for policy 1, policy_version 372108 (0.0006) [2023-12-26 18:04:12,906][105620] Updated weights for policy 1, policy_version 372118 (0.0007) [2023-12-26 18:04:12,968][105620] Updated weights for policy 1, policy_version 372128 (0.0008) [2023-12-26 18:04:12,983][105692] Updated weights for policy 0, policy_version 371500 (0.0010) [2023-12-26 18:04:13,041][105692] Updated weights for policy 0, policy_version 371510 (0.0010) [2023-12-26 18:04:13,099][105692] Updated weights for policy 0, policy_version 371520 (0.0010) [2023-12-26 18:04:13,738][105620] Updated weights for policy 1, policy_version 372138 (0.0008) [2023-12-26 18:04:13,758][105692] Updated weights for policy 0, policy_version 371530 (0.0009) [2023-12-26 18:04:13,802][105620] Updated weights for policy 1, policy_version 372148 (0.0009) [2023-12-26 18:04:13,809][105692] Updated weights for policy 0, policy_version 371540 (0.0006) [2023-12-26 18:04:13,861][105620] Updated weights for policy 1, policy_version 372158 (0.0008) [2023-12-26 18:04:13,867][105692] Updated weights for policy 0, policy_version 371550 (0.0007) [2023-12-26 18:04:13,918][105692] Updated weights for policy 0, policy_version 371560 (0.0006) [2023-12-26 18:04:13,922][105620] Updated weights for policy 1, policy_version 372168 (0.0008) [2023-12-26 18:04:14,598][105620] Updated weights for policy 1, policy_version 372178 (0.0005) [2023-12-26 18:04:14,644][105620] Updated weights for policy 1, policy_version 372188 (0.0005) [2023-12-26 18:04:14,677][105692] Updated weights for policy 0, policy_version 371570 (0.0008) [2023-12-26 18:04:14,695][105620] Updated weights for policy 1, policy_version 372198 (0.0007) [2023-12-26 18:04:14,732][105692] Updated weights for policy 0, policy_version 371580 (0.0006) [2023-12-26 18:04:14,814][105692] Updated weights for policy 0, policy_version 371590 (0.0006) [2023-12-26 18:04:15,417][105620] Updated weights for policy 1, policy_version 372208 (0.0008) [2023-12-26 18:04:15,465][105620] Updated weights for policy 1, policy_version 372218 (0.0008) [2023-12-26 18:04:15,520][105620] Updated weights for policy 1, policy_version 372228 (0.0008) [2023-12-26 18:04:15,527][105692] Updated weights for policy 0, policy_version 371600 (0.0006) [2023-12-26 18:04:15,582][105692] Updated weights for policy 0, policy_version 371610 (0.0009) [2023-12-26 18:04:15,633][105692] Updated weights for policy 0, policy_version 371620 (0.0009) [2023-12-26 18:04:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.6, 300 sec: 19466.4). Total num frames: 190447616. Throughput: 0: 9824.7, 1: 9596.1. Samples: 190417724. Policy #0 lag: (min: 16.0, avg: 40.4, max: 48.0) [2023-12-26 18:04:16,063][104569] Avg episode reward: [(0, '9003.541'), (1, '7889.139')] [2023-12-26 18:04:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000371624_95150080.pth... [2023-12-26 18:04:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000372232_95297536.pth... [2023-12-26 18:04:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000371112_95010816.pth [2023-12-26 18:04:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000370472_94855168.pth [2023-12-26 18:04:16,255][105620] Updated weights for policy 1, policy_version 372238 (0.0010) [2023-12-26 18:04:16,303][105620] Updated weights for policy 1, policy_version 372248 (0.0010) [2023-12-26 18:04:16,312][105692] Updated weights for policy 0, policy_version 371630 (0.0007) [2023-12-26 18:04:16,355][105620] Updated weights for policy 1, policy_version 372258 (0.0010) [2023-12-26 18:04:16,365][105692] Updated weights for policy 0, policy_version 371640 (0.0005) [2023-12-26 18:04:16,425][105692] Updated weights for policy 0, policy_version 371650 (0.0006) [2023-12-26 18:04:16,967][105620] Updated weights for policy 1, policy_version 372268 (0.0008) [2023-12-26 18:04:17,018][105620] Updated weights for policy 1, policy_version 372278 (0.0005) [2023-12-26 18:04:17,026][105692] Updated weights for policy 0, policy_version 371660 (0.0008) [2023-12-26 18:04:17,067][105620] Updated weights for policy 1, policy_version 372288 (0.0005) [2023-12-26 18:04:17,080][105692] Updated weights for policy 0, policy_version 371670 (0.0006) [2023-12-26 18:04:17,132][105692] Updated weights for policy 0, policy_version 371680 (0.0007) [2023-12-26 18:04:17,584][105620] Updated weights for policy 1, policy_version 372298 (0.0005) [2023-12-26 18:04:17,649][105620] Updated weights for policy 1, policy_version 372308 (0.0006) [2023-12-26 18:04:17,707][105620] Updated weights for policy 1, policy_version 372318 (0.0010) [2023-12-26 18:04:17,768][105620] Updated weights for policy 1, policy_version 372328 (0.0010) [2023-12-26 18:04:17,848][105692] Updated weights for policy 0, policy_version 371690 (0.0010) [2023-12-26 18:04:17,907][105692] Updated weights for policy 0, policy_version 371700 (0.0011) [2023-12-26 18:04:17,966][105692] Updated weights for policy 0, policy_version 371710 (0.0010) [2023-12-26 18:04:18,017][105692] Updated weights for policy 0, policy_version 371720 (0.0008) [2023-12-26 18:04:18,334][105620] Updated weights for policy 1, policy_version 372338 (0.0006) [2023-12-26 18:04:18,397][105620] Updated weights for policy 1, policy_version 372348 (0.0011) [2023-12-26 18:04:18,460][105620] Updated weights for policy 1, policy_version 372358 (0.0011) [2023-12-26 18:04:18,653][105692] Updated weights for policy 0, policy_version 371730 (0.0010) [2023-12-26 18:04:18,702][105692] Updated weights for policy 0, policy_version 371740 (0.0010) [2023-12-26 18:04:18,746][105692] Updated weights for policy 0, policy_version 371750 (0.0010) [2023-12-26 18:04:19,161][105620] Updated weights for policy 1, policy_version 372368 (0.0008) [2023-12-26 18:04:19,216][105620] Updated weights for policy 1, policy_version 372378 (0.0008) [2023-12-26 18:04:19,283][105620] Updated weights for policy 1, policy_version 372388 (0.0008) [2023-12-26 18:04:19,507][105692] Updated weights for policy 0, policy_version 371760 (0.0008) [2023-12-26 18:04:19,570][105692] Updated weights for policy 0, policy_version 371770 (0.0008) [2023-12-26 18:04:19,636][105692] Updated weights for policy 0, policy_version 371780 (0.0008) [2023-12-26 18:04:19,965][105620] Updated weights for policy 1, policy_version 372398 (0.0009) [2023-12-26 18:04:20,018][105620] Updated weights for policy 1, policy_version 372408 (0.0011) [2023-12-26 18:04:20,084][105620] Updated weights for policy 1, policy_version 372418 (0.0008) [2023-12-26 18:04:20,346][105692] Updated weights for policy 0, policy_version 371790 (0.0010) [2023-12-26 18:04:20,413][105692] Updated weights for policy 0, policy_version 371800 (0.0011) [2023-12-26 18:04:20,470][105692] Updated weights for policy 0, policy_version 371810 (0.0011) [2023-12-26 18:04:20,827][105620] Updated weights for policy 1, policy_version 372428 (0.0011) [2023-12-26 18:04:20,890][105620] Updated weights for policy 1, policy_version 372438 (0.0010) [2023-12-26 18:04:20,946][105620] Updated weights for policy 1, policy_version 372448 (0.0011) [2023-12-26 18:04:21,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 190554112. Throughput: 0: 9787.4, 1: 9822.5. Samples: 190542152. Policy #0 lag: (min: 16.0, avg: 40.4, max: 48.0) [2023-12-26 18:04:21,062][104569] Avg episode reward: [(0, '5703.905'), (1, '8164.177')] [2023-12-26 18:04:21,225][105692] Updated weights for policy 0, policy_version 371820 (0.0011) [2023-12-26 18:04:21,290][105692] Updated weights for policy 0, policy_version 371830 (0.0010) [2023-12-26 18:04:21,357][105692] Updated weights for policy 0, policy_version 371840 (0.0010) [2023-12-26 18:04:21,699][105620] Updated weights for policy 1, policy_version 372458 (0.0010) [2023-12-26 18:04:21,772][105620] Updated weights for policy 1, policy_version 372468 (0.0009) [2023-12-26 18:04:21,824][105620] Updated weights for policy 1, policy_version 372478 (0.0010) [2023-12-26 18:04:21,875][105620] Updated weights for policy 1, policy_version 372488 (0.0010) [2023-12-26 18:04:22,127][105692] Updated weights for policy 0, policy_version 371850 (0.0010) [2023-12-26 18:04:22,180][105692] Updated weights for policy 0, policy_version 371860 (0.0011) [2023-12-26 18:04:22,242][105692] Updated weights for policy 0, policy_version 371870 (0.0010) [2023-12-26 18:04:22,300][105692] Updated weights for policy 0, policy_version 371880 (0.0011) [2023-12-26 18:04:22,606][105620] Updated weights for policy 1, policy_version 372498 (0.0007) [2023-12-26 18:04:22,660][105620] Updated weights for policy 1, policy_version 372508 (0.0008) [2023-12-26 18:04:22,713][105620] Updated weights for policy 1, policy_version 372518 (0.0010) [2023-12-26 18:04:23,055][105692] Updated weights for policy 0, policy_version 371890 (0.0010) [2023-12-26 18:04:23,120][105692] Updated weights for policy 0, policy_version 371900 (0.0010) [2023-12-26 18:04:23,196][105692] Updated weights for policy 0, policy_version 371910 (0.0011) [2023-12-26 18:04:23,367][105620] Updated weights for policy 1, policy_version 372528 (0.0006) [2023-12-26 18:04:23,432][105620] Updated weights for policy 1, policy_version 372538 (0.0005) [2023-12-26 18:04:23,498][105620] Updated weights for policy 1, policy_version 372548 (0.0005) [2023-12-26 18:04:23,877][105692] Updated weights for policy 0, policy_version 371920 (0.0006) [2023-12-26 18:04:23,941][105692] Updated weights for policy 0, policy_version 371930 (0.0005) [2023-12-26 18:04:24,002][105692] Updated weights for policy 0, policy_version 371940 (0.0005) [2023-12-26 18:04:24,020][105620] Updated weights for policy 1, policy_version 372558 (0.0008) [2023-12-26 18:04:24,080][105620] Updated weights for policy 1, policy_version 372568 (0.0010) [2023-12-26 18:04:24,140][105620] Updated weights for policy 1, policy_version 372578 (0.0010) [2023-12-26 18:04:24,570][105692] Updated weights for policy 0, policy_version 371950 (0.0005) [2023-12-26 18:04:24,614][105692] Updated weights for policy 0, policy_version 371960 (0.0010) [2023-12-26 18:04:24,659][105692] Updated weights for policy 0, policy_version 371970 (0.0010) [2023-12-26 18:04:24,780][105620] Updated weights for policy 1, policy_version 372588 (0.0008) [2023-12-26 18:04:24,834][105620] Updated weights for policy 1, policy_version 372598 (0.0005) [2023-12-26 18:04:24,906][105620] Updated weights for policy 1, policy_version 372608 (0.0006) [2023-12-26 18:04:25,305][105692] Updated weights for policy 0, policy_version 371980 (0.0009) [2023-12-26 18:04:25,362][105692] Updated weights for policy 0, policy_version 371990 (0.0007) [2023-12-26 18:04:25,418][105692] Updated weights for policy 0, policy_version 372000 (0.0008) [2023-12-26 18:04:25,523][105620] Updated weights for policy 1, policy_version 372618 (0.0010) [2023-12-26 18:04:25,568][105620] Updated weights for policy 1, policy_version 372628 (0.0010) [2023-12-26 18:04:25,613][105620] Updated weights for policy 1, policy_version 372638 (0.0010) [2023-12-26 18:04:25,658][105620] Updated weights for policy 1, policy_version 372648 (0.0010) [2023-12-26 18:04:26,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.7, 300 sec: 19494.2). Total num frames: 190652416. Throughput: 0: 9724.9, 1: 9880.6. Samples: 190662212. Policy #0 lag: (min: 29.0, avg: 53.8, max: 56.0) [2023-12-26 18:04:26,063][104569] Avg episode reward: [(0, '7160.291'), (1, '7523.808')] [2023-12-26 18:04:26,068][105692] Updated weights for policy 0, policy_version 372010 (0.0008) [2023-12-26 18:04:26,127][105692] Updated weights for policy 0, policy_version 372020 (0.0008) [2023-12-26 18:04:26,186][105692] Updated weights for policy 0, policy_version 372030 (0.0008) [2023-12-26 18:04:26,237][105692] Updated weights for policy 0, policy_version 372040 (0.0008) [2023-12-26 18:04:26,436][105620] Updated weights for policy 1, policy_version 372658 (0.0009) [2023-12-26 18:04:26,484][105620] Updated weights for policy 1, policy_version 372668 (0.0008) [2023-12-26 18:04:26,538][105620] Updated weights for policy 1, policy_version 372678 (0.0009) [2023-12-26 18:04:26,973][105692] Updated weights for policy 0, policy_version 372050 (0.0005) [2023-12-26 18:04:27,018][105692] Updated weights for policy 0, policy_version 372060 (0.0005) [2023-12-26 18:04:27,070][105692] Updated weights for policy 0, policy_version 372070 (0.0007) [2023-12-26 18:04:27,152][105620] Updated weights for policy 1, policy_version 372688 (0.0010) [2023-12-26 18:04:27,196][105620] Updated weights for policy 1, policy_version 372698 (0.0010) [2023-12-26 18:04:27,247][105620] Updated weights for policy 1, policy_version 372708 (0.0010) [2023-12-26 18:04:27,818][105692] Updated weights for policy 0, policy_version 372080 (0.0008) [2023-12-26 18:04:27,873][105692] Updated weights for policy 0, policy_version 372090 (0.0008) [2023-12-26 18:04:27,933][105692] Updated weights for policy 0, policy_version 372100 (0.0008) [2023-12-26 18:04:28,003][105620] Updated weights for policy 1, policy_version 372718 (0.0009) [2023-12-26 18:04:28,057][105620] Updated weights for policy 1, policy_version 372728 (0.0009) [2023-12-26 18:04:28,112][105620] Updated weights for policy 1, policy_version 372738 (0.0010) [2023-12-26 18:04:28,642][105692] Updated weights for policy 0, policy_version 372110 (0.0008) [2023-12-26 18:04:28,702][105692] Updated weights for policy 0, policy_version 372120 (0.0008) [2023-12-26 18:04:28,762][105692] Updated weights for policy 0, policy_version 372130 (0.0008) [2023-12-26 18:04:28,831][105620] Updated weights for policy 1, policy_version 372748 (0.0009) [2023-12-26 18:04:28,887][105620] Updated weights for policy 1, policy_version 372758 (0.0005) [2023-12-26 18:04:28,947][105620] Updated weights for policy 1, policy_version 372768 (0.0005) [2023-12-26 18:04:29,525][105692] Updated weights for policy 0, policy_version 372140 (0.0008) [2023-12-26 18:04:29,527][105620] Updated weights for policy 1, policy_version 372778 (0.0006) [2023-12-26 18:04:29,579][105692] Updated weights for policy 0, policy_version 372150 (0.0007) [2023-12-26 18:04:29,589][105620] Updated weights for policy 1, policy_version 372788 (0.0010) [2023-12-26 18:04:29,627][105692] Updated weights for policy 0, policy_version 372160 (0.0005) [2023-12-26 18:04:29,647][105620] Updated weights for policy 1, policy_version 372798 (0.0010) [2023-12-26 18:04:29,702][105620] Updated weights for policy 1, policy_version 372808 (0.0010) [2023-12-26 18:04:30,289][105692] Updated weights for policy 0, policy_version 372170 (0.0007) [2023-12-26 18:04:30,348][105692] Updated weights for policy 0, policy_version 372180 (0.0011) [2023-12-26 18:04:30,359][105620] Updated weights for policy 1, policy_version 372818 (0.0006) [2023-12-26 18:04:30,410][105692] Updated weights for policy 0, policy_version 372190 (0.0011) [2023-12-26 18:04:30,410][105620] Updated weights for policy 1, policy_version 372828 (0.0005) [2023-12-26 18:04:30,468][105692] Updated weights for policy 0, policy_version 372200 (0.0009) [2023-12-26 18:04:30,476][105620] Updated weights for policy 1, policy_version 372838 (0.0009) [2023-12-26 18:04:31,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 190750720. Throughput: 0: 9800.4, 1: 9948.9. Samples: 190722256. Policy #0 lag: (min: 29.0, avg: 53.8, max: 56.0) [2023-12-26 18:04:31,062][104569] Avg episode reward: [(0, '1747.052'), (1, '7796.393')] [2023-12-26 18:04:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000372200_95297536.pth... [2023-12-26 18:04:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000371048_95002624.pth [2023-12-26 18:04:31,101][105620] Updated weights for policy 1, policy_version 372848 (0.0007) [2023-12-26 18:04:31,169][105620] Updated weights for policy 1, policy_version 372858 (0.0008) [2023-12-26 18:04:31,215][105692] Updated weights for policy 0, policy_version 372210 (0.0006) [2023-12-26 18:04:31,239][105620] Updated weights for policy 1, policy_version 372868 (0.0006) [2023-12-26 18:04:31,265][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000372872_95461376.pth... [2023-12-26 18:04:31,269][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000371656_95150080.pth [2023-12-26 18:04:31,282][105692] Updated weights for policy 0, policy_version 372220 (0.0007) [2023-12-26 18:04:31,352][105692] Updated weights for policy 0, policy_version 372230 (0.0007) [2023-12-26 18:04:31,931][105620] Updated weights for policy 1, policy_version 372878 (0.0009) [2023-12-26 18:04:31,994][105620] Updated weights for policy 1, policy_version 372888 (0.0010) [2023-12-26 18:04:32,056][105692] Updated weights for policy 0, policy_version 372240 (0.0007) [2023-12-26 18:04:32,056][105620] Updated weights for policy 1, policy_version 372898 (0.0010) [2023-12-26 18:04:32,074][105585] KL-divergence is very high: 127.3855 [2023-12-26 18:04:32,079][105585] KL-divergence is very high: 122.6360 [2023-12-26 18:04:32,085][105585] KL-divergence is very high: 122.3933 [2023-12-26 18:04:32,090][105585] KL-divergence is very high: 121.6234 [2023-12-26 18:04:32,110][105692] Updated weights for policy 0, policy_version 372250 (0.0008) [2023-12-26 18:04:32,166][105692] Updated weights for policy 0, policy_version 372260 (0.0008) [2023-12-26 18:04:32,816][105620] Updated weights for policy 1, policy_version 372908 (0.0010) [2023-12-26 18:04:32,869][105620] Updated weights for policy 1, policy_version 372918 (0.0007) [2023-12-26 18:04:32,917][105620] Updated weights for policy 1, policy_version 372928 (0.0005) [2023-12-26 18:04:32,937][105585] KL-divergence is very high: 138.3497 [2023-12-26 18:04:32,950][105585] KL-divergence is very high: 103.0664 [2023-12-26 18:04:32,975][105692] Updated weights for policy 0, policy_version 372270 (0.0008) [2023-12-26 18:04:33,031][105692] Updated weights for policy 0, policy_version 372280 (0.0009) [2023-12-26 18:04:33,086][105692] Updated weights for policy 0, policy_version 372290 (0.0009) [2023-12-26 18:04:33,647][105620] Updated weights for policy 1, policy_version 372938 (0.0008) [2023-12-26 18:04:33,697][105620] Updated weights for policy 1, policy_version 372948 (0.0008) [2023-12-26 18:04:33,725][105692] Updated weights for policy 0, policy_version 372300 (0.0010) [2023-12-26 18:04:33,761][105620] Updated weights for policy 1, policy_version 372958 (0.0006) [2023-12-26 18:04:33,785][105692] Updated weights for policy 0, policy_version 372310 (0.0010) [2023-12-26 18:04:33,811][105620] Updated weights for policy 1, policy_version 372968 (0.0006) [2023-12-26 18:04:33,841][105692] Updated weights for policy 0, policy_version 372320 (0.0010) [2023-12-26 18:04:34,416][105620] Updated weights for policy 1, policy_version 372978 (0.0005) [2023-12-26 18:04:34,478][105620] Updated weights for policy 1, policy_version 372988 (0.0009) [2023-12-26 18:04:34,514][105586] KL-divergence is very high: 138.5774 [2023-12-26 18:04:34,521][105586] KL-divergence is very high: 214.6279 [2023-12-26 18:04:34,535][105586] KL-divergence is very high: 285.0161 [2023-12-26 18:04:34,541][105586] KL-divergence is very high: 352.0855 [2023-12-26 18:04:34,541][105620] Updated weights for policy 1, policy_version 372998 (0.0010) [2023-12-26 18:04:34,547][105586] KL-divergence is very high: 110.5972 [2023-12-26 18:04:34,562][105692] Updated weights for policy 0, policy_version 372330 (0.0010) [2023-12-26 18:04:34,622][105692] Updated weights for policy 0, policy_version 372340 (0.0011) [2023-12-26 18:04:34,679][105692] Updated weights for policy 0, policy_version 372350 (0.0008) [2023-12-26 18:04:34,739][105692] Updated weights for policy 0, policy_version 372360 (0.0009) [2023-12-26 18:04:35,080][105586] KL-divergence is very high: 269.3554 [2023-12-26 18:04:35,118][105620] Updated weights for policy 1, policy_version 373008 (0.0006) [2023-12-26 18:04:35,118][105586] KL-divergence is very high: 214.5699 [2023-12-26 18:04:35,157][105586] KL-divergence is very high: 162.6628 [2023-12-26 18:04:35,168][105620] Updated weights for policy 1, policy_version 373018 (0.0006) [2023-12-26 18:04:35,176][105586] KL-divergence is very high: 102.2920 [2023-12-26 18:04:35,196][105586] KL-divergence is very high: 165.8888 [2023-12-26 18:04:35,224][105620] Updated weights for policy 1, policy_version 373028 (0.0009) [2023-12-26 18:04:35,399][105692] Updated weights for policy 0, policy_version 372370 (0.0010) [2023-12-26 18:04:35,460][105692] Updated weights for policy 0, policy_version 372380 (0.0010) [2023-12-26 18:04:35,511][105692] Updated weights for policy 0, policy_version 372390 (0.0010) [2023-12-26 18:04:35,901][105620] Updated weights for policy 1, policy_version 373038 (0.0009) [2023-12-26 18:04:35,952][105620] Updated weights for policy 1, policy_version 373048 (0.0009) [2023-12-26 18:04:36,006][105620] Updated weights for policy 1, policy_version 373058 (0.0006) [2023-12-26 18:04:36,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 190857216. Throughput: 0: 9852.4, 1: 10006.0. Samples: 190842540. Policy #0 lag: (min: 29.0, avg: 53.8, max: 56.0) [2023-12-26 18:04:36,062][104569] Avg episode reward: [(0, '1235.160'), (1, '6960.854')] [2023-12-26 18:04:36,122][105692] Updated weights for policy 0, policy_version 372400 (0.0008) [2023-12-26 18:04:36,191][105692] Updated weights for policy 0, policy_version 372410 (0.0009) [2023-12-26 18:04:36,254][105692] Updated weights for policy 0, policy_version 372420 (0.0011) [2023-12-26 18:04:36,740][105620] Updated weights for policy 1, policy_version 373068 (0.0008) [2023-12-26 18:04:36,788][105620] Updated weights for policy 1, policy_version 373078 (0.0005) [2023-12-26 18:04:36,836][105620] Updated weights for policy 1, policy_version 373088 (0.0006) [2023-12-26 18:04:37,003][105692] Updated weights for policy 0, policy_version 372430 (0.0008) [2023-12-26 18:04:37,071][105692] Updated weights for policy 0, policy_version 372440 (0.0009) [2023-12-26 18:04:37,133][105692] Updated weights for policy 0, policy_version 372450 (0.0010) [2023-12-26 18:04:37,480][105620] Updated weights for policy 1, policy_version 373098 (0.0006) [2023-12-26 18:04:37,536][105620] Updated weights for policy 1, policy_version 373108 (0.0009) [2023-12-26 18:04:37,597][105620] Updated weights for policy 1, policy_version 373118 (0.0009) [2023-12-26 18:04:37,647][105620] Updated weights for policy 1, policy_version 373128 (0.0008) [2023-12-26 18:04:37,812][105692] Updated weights for policy 0, policy_version 372460 (0.0008) [2023-12-26 18:04:37,875][105692] Updated weights for policy 0, policy_version 372470 (0.0009) [2023-12-26 18:04:37,931][105692] Updated weights for policy 0, policy_version 372480 (0.0009) [2023-12-26 18:04:38,509][105620] Updated weights for policy 1, policy_version 373138 (0.0009) [2023-12-26 18:04:38,560][105620] Updated weights for policy 1, policy_version 373148 (0.0008) [2023-12-26 18:04:38,567][105692] Updated weights for policy 0, policy_version 372490 (0.0006) [2023-12-26 18:04:38,609][105620] Updated weights for policy 1, policy_version 373158 (0.0008) [2023-12-26 18:04:38,627][105692] Updated weights for policy 0, policy_version 372500 (0.0008) [2023-12-26 18:04:38,682][105692] Updated weights for policy 0, policy_version 372510 (0.0009) [2023-12-26 18:04:38,738][105692] Updated weights for policy 0, policy_version 372520 (0.0009) [2023-12-26 18:04:39,434][105620] Updated weights for policy 1, policy_version 373168 (0.0008) [2023-12-26 18:04:39,486][105692] Updated weights for policy 0, policy_version 372530 (0.0006) [2023-12-26 18:04:39,496][105620] Updated weights for policy 1, policy_version 373178 (0.0009) [2023-12-26 18:04:39,545][105692] Updated weights for policy 0, policy_version 372540 (0.0005) [2023-12-26 18:04:39,548][105620] Updated weights for policy 1, policy_version 373188 (0.0008) [2023-12-26 18:04:39,608][105692] Updated weights for policy 0, policy_version 372550 (0.0008) [2023-12-26 18:04:40,280][105620] Updated weights for policy 1, policy_version 373198 (0.0008) [2023-12-26 18:04:40,343][105620] Updated weights for policy 1, policy_version 373208 (0.0008) [2023-12-26 18:04:40,349][105692] Updated weights for policy 0, policy_version 372560 (0.0007) [2023-12-26 18:04:40,404][105620] Updated weights for policy 1, policy_version 373218 (0.0009) [2023-12-26 18:04:40,410][105692] Updated weights for policy 0, policy_version 372570 (0.0006) [2023-12-26 18:04:40,466][105692] Updated weights for policy 0, policy_version 372580 (0.0007) [2023-12-26 18:04:41,055][105620] Updated weights for policy 1, policy_version 373228 (0.0011) [2023-12-26 18:04:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 190947328. Throughput: 0: 9876.4, 1: 10018.1. Samples: 190959856. Policy #0 lag: (min: 29.0, avg: 53.8, max: 56.0) [2023-12-26 18:04:41,062][104569] Avg episode reward: [(0, '1563.280'), (1, '7055.538')] [2023-12-26 18:04:41,117][105620] Updated weights for policy 1, policy_version 373238 (0.0008) [2023-12-26 18:04:41,186][105620] Updated weights for policy 1, policy_version 373248 (0.0007) [2023-12-26 18:04:41,230][105692] Updated weights for policy 0, policy_version 372590 (0.0008) [2023-12-26 18:04:41,289][105692] Updated weights for policy 0, policy_version 372600 (0.0009) [2023-12-26 18:04:41,343][105692] Updated weights for policy 0, policy_version 372610 (0.0009) [2023-12-26 18:04:41,904][105620] Updated weights for policy 1, policy_version 373258 (0.0007) [2023-12-26 18:04:41,971][105620] Updated weights for policy 1, policy_version 373268 (0.0006) [2023-12-26 18:04:42,029][105620] Updated weights for policy 1, policy_version 373278 (0.0006) [2023-12-26 18:04:42,092][105620] Updated weights for policy 1, policy_version 373288 (0.0007) [2023-12-26 18:04:42,185][105692] Updated weights for policy 0, policy_version 372620 (0.0009) [2023-12-26 18:04:42,233][105692] Updated weights for policy 0, policy_version 372630 (0.0009) [2023-12-26 18:04:42,293][105692] Updated weights for policy 0, policy_version 372640 (0.0009) [2023-12-26 18:04:42,797][105620] Updated weights for policy 1, policy_version 373298 (0.0011) [2023-12-26 18:04:42,857][105620] Updated weights for policy 1, policy_version 373308 (0.0010) [2023-12-26 18:04:42,919][105620] Updated weights for policy 1, policy_version 373318 (0.0010) [2023-12-26 18:04:43,076][105692] Updated weights for policy 0, policy_version 372650 (0.0009) [2023-12-26 18:04:43,135][105692] Updated weights for policy 0, policy_version 372660 (0.0006) [2023-12-26 18:04:43,190][105692] Updated weights for policy 0, policy_version 372670 (0.0007) [2023-12-26 18:04:43,253][105692] Updated weights for policy 0, policy_version 372680 (0.0006) [2023-12-26 18:04:43,630][105620] Updated weights for policy 1, policy_version 373328 (0.0006) [2023-12-26 18:04:43,683][105620] Updated weights for policy 1, policy_version 373338 (0.0005) [2023-12-26 18:04:43,731][105620] Updated weights for policy 1, policy_version 373348 (0.0005) [2023-12-26 18:04:43,918][105692] Updated weights for policy 0, policy_version 372690 (0.0010) [2023-12-26 18:04:43,993][105692] Updated weights for policy 0, policy_version 372700 (0.0009) [2023-12-26 18:04:44,054][105692] Updated weights for policy 0, policy_version 372710 (0.0009) [2023-12-26 18:04:44,281][105620] Updated weights for policy 1, policy_version 373358 (0.0008) [2023-12-26 18:04:44,336][105620] Updated weights for policy 1, policy_version 373368 (0.0010) [2023-12-26 18:04:44,391][105620] Updated weights for policy 1, policy_version 373378 (0.0011) [2023-12-26 18:04:44,818][105692] Updated weights for policy 0, policy_version 372720 (0.0009) [2023-12-26 18:04:44,870][105692] Updated weights for policy 0, policy_version 372730 (0.0008) [2023-12-26 18:04:44,936][105692] Updated weights for policy 0, policy_version 372740 (0.0008) [2023-12-26 18:04:45,157][105620] Updated weights for policy 1, policy_version 373388 (0.0010) [2023-12-26 18:04:45,226][105620] Updated weights for policy 1, policy_version 373398 (0.0010) [2023-12-26 18:04:45,294][105620] Updated weights for policy 1, policy_version 373408 (0.0011) [2023-12-26 18:04:45,660][105692] Updated weights for policy 0, policy_version 372750 (0.0009) [2023-12-26 18:04:45,704][105692] Updated weights for policy 0, policy_version 372760 (0.0008) [2023-12-26 18:04:45,751][105692] Updated weights for policy 0, policy_version 372770 (0.0008) [2023-12-26 18:04:45,981][105620] Updated weights for policy 1, policy_version 373418 (0.0009) [2023-12-26 18:04:46,048][105620] Updated weights for policy 1, policy_version 373428 (0.0008) [2023-12-26 18:04:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 191045632. Throughput: 0: 9868.7, 1: 10072.4. Samples: 191018236. Policy #0 lag: (min: 29.0, avg: 53.8, max: 56.0) [2023-12-26 18:04:46,062][104569] Avg episode reward: [(0, '6346.223'), (1, '7330.487')] [2023-12-26 18:04:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000372776_95444992.pth... [2023-12-26 18:04:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000371624_95150080.pth [2023-12-26 18:04:46,101][105620] Updated weights for policy 1, policy_version 373438 (0.0011) [2023-12-26 18:04:46,153][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000373448_95608832.pth... [2023-12-26 18:04:46,157][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000372232_95297536.pth [2023-12-26 18:04:46,157][105620] Updated weights for policy 1, policy_version 373448 (0.0010) [2023-12-26 18:04:46,602][105692] Updated weights for policy 0, policy_version 372780 (0.0009) [2023-12-26 18:04:46,667][105692] Updated weights for policy 0, policy_version 372790 (0.0009) [2023-12-26 18:04:46,729][105692] Updated weights for policy 0, policy_version 372800 (0.0008) [2023-12-26 18:04:46,735][105620] Updated weights for policy 1, policy_version 373458 (0.0008) [2023-12-26 18:04:46,781][105620] Updated weights for policy 1, policy_version 373468 (0.0007) [2023-12-26 18:04:46,835][105620] Updated weights for policy 1, policy_version 373478 (0.0009) [2023-12-26 18:04:47,451][105692] Updated weights for policy 0, policy_version 372810 (0.0008) [2023-12-26 18:04:47,492][105620] Updated weights for policy 1, policy_version 373488 (0.0008) [2023-12-26 18:04:47,510][105692] Updated weights for policy 0, policy_version 372820 (0.0007) [2023-12-26 18:04:47,540][105620] Updated weights for policy 1, policy_version 373498 (0.0009) [2023-12-26 18:04:47,562][105692] Updated weights for policy 0, policy_version 372830 (0.0008) [2023-12-26 18:04:47,584][105620] Updated weights for policy 1, policy_version 373508 (0.0005) [2023-12-26 18:04:47,610][105692] Updated weights for policy 0, policy_version 372840 (0.0008) [2023-12-26 18:04:48,139][105620] Updated weights for policy 1, policy_version 373518 (0.0008) [2023-12-26 18:04:48,198][105620] Updated weights for policy 1, policy_version 373528 (0.0010) [2023-12-26 18:04:48,233][105586] KL-divergence is very high: 109.9060 [2023-12-26 18:04:48,243][105620] Updated weights for policy 1, policy_version 373538 (0.0010) [2023-12-26 18:04:48,400][105692] Updated weights for policy 0, policy_version 372850 (0.0010) [2023-12-26 18:04:48,459][105692] Updated weights for policy 0, policy_version 372860 (0.0010) [2023-12-26 18:04:48,522][105692] Updated weights for policy 0, policy_version 372870 (0.0008) [2023-12-26 18:04:48,994][105620] Updated weights for policy 1, policy_version 373548 (0.0008) [2023-12-26 18:04:49,053][105620] Updated weights for policy 1, policy_version 373558 (0.0007) [2023-12-26 18:04:49,112][105620] Updated weights for policy 1, policy_version 373568 (0.0008) [2023-12-26 18:04:49,262][105692] Updated weights for policy 0, policy_version 372880 (0.0009) [2023-12-26 18:04:49,320][105692] Updated weights for policy 0, policy_version 372890 (0.0009) [2023-12-26 18:04:49,393][105692] Updated weights for policy 0, policy_version 372900 (0.0009) [2023-12-26 18:04:49,882][105620] Updated weights for policy 1, policy_version 373578 (0.0007) [2023-12-26 18:04:49,951][105620] Updated weights for policy 1, policy_version 373588 (0.0009) [2023-12-26 18:04:50,022][105620] Updated weights for policy 1, policy_version 373598 (0.0010) [2023-12-26 18:04:50,047][105692] Updated weights for policy 0, policy_version 372910 (0.0007) [2023-12-26 18:04:50,084][105620] Updated weights for policy 1, policy_version 373608 (0.0008) [2023-12-26 18:04:50,098][105692] Updated weights for policy 0, policy_version 372920 (0.0006) [2023-12-26 18:04:50,160][105692] Updated weights for policy 0, policy_version 372930 (0.0008) [2023-12-26 18:04:50,785][105620] Updated weights for policy 1, policy_version 373618 (0.0007) [2023-12-26 18:04:50,841][105620] Updated weights for policy 1, policy_version 373628 (0.0006) [2023-12-26 18:04:50,903][105620] Updated weights for policy 1, policy_version 373638 (0.0007) [2023-12-26 18:04:50,919][105692] Updated weights for policy 0, policy_version 372940 (0.0009) [2023-12-26 18:04:50,977][105692] Updated weights for policy 0, policy_version 372950 (0.0009) [2023-12-26 18:04:51,032][105692] Updated weights for policy 0, policy_version 372960 (0.0010) [2023-12-26 18:04:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 191143936. Throughput: 0: 9721.2, 1: 10153.8. Samples: 191134608. Policy #0 lag: (min: 29.0, avg: 53.8, max: 56.0) [2023-12-26 18:04:51,062][104569] Avg episode reward: [(0, '8472.225'), (1, '5677.717')] [2023-12-26 18:04:51,602][105620] Updated weights for policy 1, policy_version 373648 (0.0009) [2023-12-26 18:04:51,667][105620] Updated weights for policy 1, policy_version 373658 (0.0009) [2023-12-26 18:04:51,737][105620] Updated weights for policy 1, policy_version 373668 (0.0009) [2023-12-26 18:04:51,787][105692] Updated weights for policy 0, policy_version 372970 (0.0008) [2023-12-26 18:04:51,849][105692] Updated weights for policy 0, policy_version 372980 (0.0009) [2023-12-26 18:04:51,908][105692] Updated weights for policy 0, policy_version 372990 (0.0009) [2023-12-26 18:04:51,974][105692] Updated weights for policy 0, policy_version 373000 (0.0009) [2023-12-26 18:04:52,489][105620] Updated weights for policy 1, policy_version 373678 (0.0008) [2023-12-26 18:04:52,543][105620] Updated weights for policy 1, policy_version 373688 (0.0009) [2023-12-26 18:04:52,609][105620] Updated weights for policy 1, policy_version 373698 (0.0008) [2023-12-26 18:04:52,719][105692] Updated weights for policy 0, policy_version 373010 (0.0008) [2023-12-26 18:04:52,773][105692] Updated weights for policy 0, policy_version 373020 (0.0008) [2023-12-26 18:04:52,826][105692] Updated weights for policy 0, policy_version 373030 (0.0010) [2023-12-26 18:04:53,284][105620] Updated weights for policy 1, policy_version 373708 (0.0009) [2023-12-26 18:04:53,331][105620] Updated weights for policy 1, policy_version 373718 (0.0005) [2023-12-26 18:04:53,376][105620] Updated weights for policy 1, policy_version 373728 (0.0005) [2023-12-26 18:04:53,670][105692] Updated weights for policy 0, policy_version 373040 (0.0009) [2023-12-26 18:04:53,717][105692] Updated weights for policy 0, policy_version 373050 (0.0008) [2023-12-26 18:04:53,765][105692] Updated weights for policy 0, policy_version 373060 (0.0008) [2023-12-26 18:04:54,062][105620] Updated weights for policy 1, policy_version 373738 (0.0006) [2023-12-26 18:04:54,120][105620] Updated weights for policy 1, policy_version 373748 (0.0010) [2023-12-26 18:04:54,167][105620] Updated weights for policy 1, policy_version 373758 (0.0006) [2023-12-26 18:04:54,223][105620] Updated weights for policy 1, policy_version 373768 (0.0005) [2023-12-26 18:04:54,456][105692] Updated weights for policy 0, policy_version 373070 (0.0006) [2023-12-26 18:04:54,516][105692] Updated weights for policy 0, policy_version 373080 (0.0007) [2023-12-26 18:04:54,577][105692] Updated weights for policy 0, policy_version 373090 (0.0009) [2023-12-26 18:04:54,869][105620] Updated weights for policy 1, policy_version 373778 (0.0010) [2023-12-26 18:04:54,925][105620] Updated weights for policy 1, policy_version 373788 (0.0010) [2023-12-26 18:04:54,982][105620] Updated weights for policy 1, policy_version 373798 (0.0013) [2023-12-26 18:04:55,227][105692] Updated weights for policy 0, policy_version 373100 (0.0009) [2023-12-26 18:04:55,275][105692] Updated weights for policy 0, policy_version 373110 (0.0008) [2023-12-26 18:04:55,326][105692] Updated weights for policy 0, policy_version 373120 (0.0007) [2023-12-26 18:04:55,740][105620] Updated weights for policy 1, policy_version 373808 (0.0010) [2023-12-26 18:04:55,805][105620] Updated weights for policy 1, policy_version 373818 (0.0010) [2023-12-26 18:04:55,856][105620] Updated weights for policy 1, policy_version 373828 (0.0010) [2023-12-26 18:04:55,933][105692] Updated weights for policy 0, policy_version 373130 (0.0009) [2023-12-26 18:04:55,992][105692] Updated weights for policy 0, policy_version 373140 (0.0005) [2023-12-26 18:04:56,045][105692] Updated weights for policy 0, policy_version 373150 (0.0005) [2023-12-26 18:04:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 191242240. Throughput: 0: 9697.3, 1: 10120.1. Samples: 191251592. Policy #0 lag: (min: 29.0, avg: 53.8, max: 56.0) [2023-12-26 18:04:56,062][104569] Avg episode reward: [(0, '8645.867'), (1, '5742.887')] [2023-12-26 18:04:56,109][105692] Updated weights for policy 0, policy_version 373160 (0.0005) [2023-12-26 18:04:56,540][105620] Updated weights for policy 1, policy_version 373838 (0.0009) [2023-12-26 18:04:56,595][105620] Updated weights for policy 1, policy_version 373848 (0.0010) [2023-12-26 18:04:56,661][105620] Updated weights for policy 1, policy_version 373858 (0.0008) [2023-12-26 18:04:56,675][105692] Updated weights for policy 0, policy_version 373170 (0.0009) [2023-12-26 18:04:56,739][105692] Updated weights for policy 0, policy_version 373180 (0.0010) [2023-12-26 18:04:56,808][105692] Updated weights for policy 0, policy_version 373190 (0.0010) [2023-12-26 18:04:57,186][105620] Updated weights for policy 1, policy_version 373868 (0.0005) [2023-12-26 18:04:57,246][105620] Updated weights for policy 1, policy_version 373878 (0.0005) [2023-12-26 18:04:57,304][105620] Updated weights for policy 1, policy_version 373888 (0.0005) [2023-12-26 18:04:57,485][105692] Updated weights for policy 0, policy_version 373200 (0.0008) [2023-12-26 18:04:57,543][105692] Updated weights for policy 0, policy_version 373210 (0.0010) [2023-12-26 18:04:57,594][105692] Updated weights for policy 0, policy_version 373220 (0.0006) [2023-12-26 18:04:57,997][105620] Updated weights for policy 1, policy_version 373898 (0.0010) [2023-12-26 18:04:58,048][105620] Updated weights for policy 1, policy_version 373908 (0.0010) [2023-12-26 18:04:58,100][105620] Updated weights for policy 1, policy_version 373918 (0.0010) [2023-12-26 18:04:58,124][105692] Updated weights for policy 0, policy_version 373230 (0.0009) [2023-12-26 18:04:58,159][105620] Updated weights for policy 1, policy_version 373928 (0.0009) [2023-12-26 18:04:58,183][105692] Updated weights for policy 0, policy_version 373240 (0.0010) [2023-12-26 18:04:58,247][105692] Updated weights for policy 0, policy_version 373250 (0.0011) [2023-12-26 18:04:58,965][105620] Updated weights for policy 1, policy_version 373938 (0.0008) [2023-12-26 18:04:58,994][105692] Updated weights for policy 0, policy_version 373260 (0.0009) [2023-12-26 18:04:59,030][105620] Updated weights for policy 1, policy_version 373948 (0.0008) [2023-12-26 18:04:59,050][105692] Updated weights for policy 0, policy_version 373270 (0.0007) [2023-12-26 18:04:59,094][105620] Updated weights for policy 1, policy_version 373958 (0.0007) [2023-12-26 18:04:59,100][105692] Updated weights for policy 0, policy_version 373280 (0.0007) [2023-12-26 18:04:59,782][105620] Updated weights for policy 1, policy_version 373968 (0.0006) [2023-12-26 18:04:59,842][105620] Updated weights for policy 1, policy_version 373978 (0.0008) [2023-12-26 18:04:59,846][105692] Updated weights for policy 0, policy_version 373290 (0.0006) [2023-12-26 18:04:59,901][105620] Updated weights for policy 1, policy_version 373988 (0.0006) [2023-12-26 18:04:59,912][105692] Updated weights for policy 0, policy_version 373300 (0.0006) [2023-12-26 18:04:59,974][105692] Updated weights for policy 0, policy_version 373310 (0.0008) [2023-12-26 18:05:00,035][105692] Updated weights for policy 0, policy_version 373320 (0.0007) [2023-12-26 18:05:00,528][105620] Updated weights for policy 1, policy_version 373998 (0.0006) [2023-12-26 18:05:00,579][105620] Updated weights for policy 1, policy_version 374008 (0.0005) [2023-12-26 18:05:00,638][105620] Updated weights for policy 1, policy_version 374018 (0.0005) [2023-12-26 18:05:00,642][105692] Updated weights for policy 0, policy_version 373330 (0.0006) [2023-12-26 18:05:00,689][105692] Updated weights for policy 0, policy_version 373340 (0.0010) [2023-12-26 18:05:00,740][105692] Updated weights for policy 0, policy_version 373350 (0.0010) [2023-12-26 18:05:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19933.9, 300 sec: 19549.7). Total num frames: 191348736. Throughput: 0: 9797.7, 1: 10162.0. Samples: 191315904. Policy #0 lag: (min: 29.0, avg: 53.8, max: 56.0) [2023-12-26 18:05:01,062][104569] Avg episode reward: [(0, '7449.397'), (1, '5578.869')] [2023-12-26 18:05:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000374024_95756288.pth... [2023-12-26 18:05:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000373352_95592448.pth... [2023-12-26 18:05:01,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000372872_95461376.pth [2023-12-26 18:05:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000372200_95297536.pth [2023-12-26 18:05:01,250][105620] Updated weights for policy 1, policy_version 374028 (0.0008) [2023-12-26 18:05:01,315][105620] Updated weights for policy 1, policy_version 374038 (0.0010) [2023-12-26 18:05:01,384][105620] Updated weights for policy 1, policy_version 374048 (0.0012) [2023-12-26 18:05:01,481][105692] Updated weights for policy 0, policy_version 373360 (0.0006) [2023-12-26 18:05:01,545][105692] Updated weights for policy 0, policy_version 373370 (0.0005) [2023-12-26 18:05:01,607][105692] Updated weights for policy 0, policy_version 373380 (0.0006) [2023-12-26 18:05:02,119][105620] Updated weights for policy 1, policy_version 374058 (0.0010) [2023-12-26 18:05:02,171][105620] Updated weights for policy 1, policy_version 374068 (0.0009) [2023-12-26 18:05:02,220][105620] Updated weights for policy 1, policy_version 374078 (0.0005) [2023-12-26 18:05:02,266][105692] Updated weights for policy 0, policy_version 373390 (0.0007) [2023-12-26 18:05:02,286][105620] Updated weights for policy 1, policy_version 374088 (0.0006) [2023-12-26 18:05:02,332][105692] Updated weights for policy 0, policy_version 373400 (0.0006) [2023-12-26 18:05:02,398][105692] Updated weights for policy 0, policy_version 373410 (0.0009) [2023-12-26 18:05:02,903][105620] Updated weights for policy 1, policy_version 374098 (0.0007) [2023-12-26 18:05:02,964][105620] Updated weights for policy 1, policy_version 374108 (0.0007) [2023-12-26 18:05:03,014][105620] Updated weights for policy 1, policy_version 374118 (0.0005) [2023-12-26 18:05:03,220][105692] Updated weights for policy 0, policy_version 373420 (0.0009) [2023-12-26 18:05:03,274][105692] Updated weights for policy 0, policy_version 373432 (0.0010) [2023-12-26 18:05:03,327][105692] Updated weights for policy 0, policy_version 373444 (0.0010) [2023-12-26 18:05:03,537][105620] Updated weights for policy 1, policy_version 374128 (0.0005) [2023-12-26 18:05:03,599][105620] Updated weights for policy 1, policy_version 374138 (0.0005) [2023-12-26 18:05:03,650][105620] Updated weights for policy 1, policy_version 374148 (0.0005) [2023-12-26 18:05:04,208][105620] Updated weights for policy 1, policy_version 374158 (0.0006) [2023-12-26 18:05:04,254][105620] Updated weights for policy 1, policy_version 374168 (0.0009) [2023-12-26 18:05:04,288][105692] Updated weights for policy 0, policy_version 373454 (0.0009) [2023-12-26 18:05:04,317][105620] Updated weights for policy 1, policy_version 374178 (0.0008) [2023-12-26 18:05:04,351][105692] Updated weights for policy 0, policy_version 373464 (0.0007) [2023-12-26 18:05:04,413][105692] Updated weights for policy 0, policy_version 373474 (0.0009) [2023-12-26 18:05:05,043][105620] Updated weights for policy 1, policy_version 374188 (0.0008) [2023-12-26 18:05:05,090][105620] Updated weights for policy 1, policy_version 374198 (0.0007) [2023-12-26 18:05:05,105][105692] Updated weights for policy 0, policy_version 373484 (0.0007) [2023-12-26 18:05:05,146][105620] Updated weights for policy 1, policy_version 374208 (0.0005) [2023-12-26 18:05:05,159][105692] Updated weights for policy 0, policy_version 373494 (0.0005) [2023-12-26 18:05:05,213][105692] Updated weights for policy 0, policy_version 373504 (0.0005) [2023-12-26 18:05:05,730][105692] Updated weights for policy 0, policy_version 373514 (0.0005) [2023-12-26 18:05:05,785][105692] Updated weights for policy 0, policy_version 373524 (0.0005) [2023-12-26 18:05:05,843][105692] Updated weights for policy 0, policy_version 373534 (0.0006) [2023-12-26 18:05:05,903][105692] Updated weights for policy 0, policy_version 373544 (0.0009) [2023-12-26 18:05:05,925][105620] Updated weights for policy 1, policy_version 374218 (0.0006) [2023-12-26 18:05:05,989][105620] Updated weights for policy 1, policy_version 374228 (0.0010) [2023-12-26 18:05:06,055][105620] Updated weights for policy 1, policy_version 374238 (0.0010) [2023-12-26 18:05:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19933.9, 300 sec: 19549.7). Total num frames: 191447040. Throughput: 0: 9684.3, 1: 10168.4. Samples: 191435528. Policy #0 lag: (min: 29.0, avg: 53.8, max: 56.0) [2023-12-26 18:05:06,062][104569] Avg episode reward: [(0, '7003.785'), (1, '5951.210')] [2023-12-26 18:05:06,124][105620] Updated weights for policy 1, policy_version 374248 (0.0009) [2023-12-26 18:05:06,533][105692] Updated weights for policy 0, policy_version 373554 (0.0009) [2023-12-26 18:05:06,595][105692] Updated weights for policy 0, policy_version 373564 (0.0009) [2023-12-26 18:05:06,654][105692] Updated weights for policy 0, policy_version 373574 (0.0009) [2023-12-26 18:05:06,882][105620] Updated weights for policy 1, policy_version 374258 (0.0008) [2023-12-26 18:05:06,943][105620] Updated weights for policy 1, policy_version 374268 (0.0009) [2023-12-26 18:05:07,000][105620] Updated weights for policy 1, policy_version 374278 (0.0008) [2023-12-26 18:05:07,417][105692] Updated weights for policy 0, policy_version 373584 (0.0008) [2023-12-26 18:05:07,475][105692] Updated weights for policy 0, policy_version 373594 (0.0009) [2023-12-26 18:05:07,530][105692] Updated weights for policy 0, policy_version 373604 (0.0009) [2023-12-26 18:05:07,738][105620] Updated weights for policy 1, policy_version 374288 (0.0006) [2023-12-26 18:05:07,796][105620] Updated weights for policy 1, policy_version 374298 (0.0010) [2023-12-26 18:05:07,843][105620] Updated weights for policy 1, policy_version 374308 (0.0009) [2023-12-26 18:05:08,316][105692] Updated weights for policy 0, policy_version 373614 (0.0009) [2023-12-26 18:05:08,391][105692] Updated weights for policy 0, policy_version 373624 (0.0009) [2023-12-26 18:05:08,456][105692] Updated weights for policy 0, policy_version 373634 (0.0009) [2023-12-26 18:05:08,551][105620] Updated weights for policy 1, policy_version 374318 (0.0008) [2023-12-26 18:05:08,574][105586] KL-divergence is very high: 155.5257 [2023-12-26 18:05:08,589][105586] KL-divergence is very high: 114.3176 [2023-12-26 18:05:08,598][105620] Updated weights for policy 1, policy_version 374328 (0.0009) [2023-12-26 18:05:08,608][105586] KL-divergence is very high: 184.6492 [2023-12-26 18:05:08,613][105586] KL-divergence is very high: 280.4842 [2023-12-26 18:05:08,627][105586] KL-divergence is very high: 144.3256 [2023-12-26 18:05:08,648][105586] KL-divergence is very high: 182.1931 [2023-12-26 18:05:08,648][105620] Updated weights for policy 1, policy_version 374338 (0.0009) [2023-12-26 18:05:08,653][105586] KL-divergence is very high: 269.9632 [2023-12-26 18:05:08,672][105586] KL-divergence is very high: 106.1351 [2023-12-26 18:05:09,210][105692] Updated weights for policy 0, policy_version 373644 (0.0009) [2023-12-26 18:05:09,276][105692] Updated weights for policy 0, policy_version 373654 (0.0009) [2023-12-26 18:05:09,338][105692] Updated weights for policy 0, policy_version 373664 (0.0008) [2023-12-26 18:05:09,448][105620] Updated weights for policy 1, policy_version 374348 (0.0010) [2023-12-26 18:05:09,501][105620] Updated weights for policy 1, policy_version 374358 (0.0008) [2023-12-26 18:05:09,565][105620] Updated weights for policy 1, policy_version 374368 (0.0006) [2023-12-26 18:05:10,202][105692] Updated weights for policy 0, policy_version 373674 (0.0007) [2023-12-26 18:05:10,204][105620] Updated weights for policy 1, policy_version 374378 (0.0006) [2023-12-26 18:05:10,264][105692] Updated weights for policy 0, policy_version 373684 (0.0006) [2023-12-26 18:05:10,271][105620] Updated weights for policy 1, policy_version 374388 (0.0010) [2023-12-26 18:05:10,329][105620] Updated weights for policy 1, policy_version 374398 (0.0010) [2023-12-26 18:05:10,329][105692] Updated weights for policy 0, policy_version 373694 (0.0005) [2023-12-26 18:05:10,383][105620] Updated weights for policy 1, policy_version 374408 (0.0011) [2023-12-26 18:05:10,392][105692] Updated weights for policy 0, policy_version 373704 (0.0006) [2023-12-26 18:05:10,967][105692] Updated weights for policy 0, policy_version 373714 (0.0010) [2023-12-26 18:05:11,022][105692] Updated weights for policy 0, policy_version 373724 (0.0010) [2023-12-26 18:05:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 191537152. Throughput: 0: 9683.9, 1: 10088.1. Samples: 191551952. Policy #0 lag: (min: 29.0, avg: 53.8, max: 56.0) [2023-12-26 18:05:11,063][104569] Avg episode reward: [(0, '8038.764'), (1, '6042.067')] [2023-12-26 18:05:11,090][105692] Updated weights for policy 0, policy_version 373734 (0.0008) [2023-12-26 18:05:11,115][105620] Updated weights for policy 1, policy_version 374418 (0.0010) [2023-12-26 18:05:11,180][105620] Updated weights for policy 1, policy_version 374428 (0.0008) [2023-12-26 18:05:11,250][105620] Updated weights for policy 1, policy_version 374438 (0.0009) [2023-12-26 18:05:11,904][105692] Updated weights for policy 0, policy_version 373744 (0.0009) [2023-12-26 18:05:11,965][105692] Updated weights for policy 0, policy_version 373754 (0.0006) [2023-12-26 18:05:11,965][105620] Updated weights for policy 1, policy_version 374448 (0.0008) [2023-12-26 18:05:12,027][105692] Updated weights for policy 0, policy_version 373764 (0.0006) [2023-12-26 18:05:12,032][105620] Updated weights for policy 1, policy_version 374458 (0.0008) [2023-12-26 18:05:12,104][105620] Updated weights for policy 1, policy_version 374468 (0.0008) [2023-12-26 18:05:12,769][105620] Updated weights for policy 1, policy_version 374478 (0.0008) [2023-12-26 18:05:12,823][105692] Updated weights for policy 0, policy_version 373774 (0.0008) [2023-12-26 18:05:12,829][105620] Updated weights for policy 1, policy_version 374488 (0.0011) [2023-12-26 18:05:12,853][105586] KL-divergence is very high: 115.3325 [2023-12-26 18:05:12,865][105586] KL-divergence is very high: 153.7878 [2023-12-26 18:05:12,880][105692] Updated weights for policy 0, policy_version 373784 (0.0008) [2023-12-26 18:05:12,887][105620] Updated weights for policy 1, policy_version 374498 (0.0007) [2023-12-26 18:05:12,889][105586] KL-divergence is very high: 161.4189 [2023-12-26 18:05:12,901][105586] KL-divergence is very high: 155.2173 [2023-12-26 18:05:12,914][105586] KL-divergence is very high: 156.9722 [2023-12-26 18:05:12,939][105692] Updated weights for policy 0, policy_version 373794 (0.0007) [2023-12-26 18:05:13,618][105692] Updated weights for policy 0, policy_version 373804 (0.0005) [2023-12-26 18:05:13,670][105620] Updated weights for policy 1, policy_version 374508 (0.0007) [2023-12-26 18:05:13,673][105692] Updated weights for policy 0, policy_version 373814 (0.0006) [2023-12-26 18:05:13,725][105620] Updated weights for policy 1, policy_version 374518 (0.0007) [2023-12-26 18:05:13,730][105692] Updated weights for policy 0, policy_version 373824 (0.0007) [2023-12-26 18:05:13,786][105620] Updated weights for policy 1, policy_version 374528 (0.0008) [2023-12-26 18:05:14,350][105692] Updated weights for policy 0, policy_version 373834 (0.0006) [2023-12-26 18:05:14,416][105692] Updated weights for policy 0, policy_version 373844 (0.0009) [2023-12-26 18:05:14,481][105692] Updated weights for policy 0, policy_version 373854 (0.0010) [2023-12-26 18:05:14,549][105692] Updated weights for policy 0, policy_version 373864 (0.0010) [2023-12-26 18:05:14,595][105620] Updated weights for policy 1, policy_version 374538 (0.0008) [2023-12-26 18:05:14,660][105620] Updated weights for policy 1, policy_version 374548 (0.0008) [2023-12-26 18:05:14,711][105620] Updated weights for policy 1, policy_version 374558 (0.0008) [2023-12-26 18:05:14,774][105620] Updated weights for policy 1, policy_version 374568 (0.0008) [2023-12-26 18:05:15,268][105692] Updated weights for policy 0, policy_version 373874 (0.0009) [2023-12-26 18:05:15,329][105692] Updated weights for policy 0, policy_version 373884 (0.0009) [2023-12-26 18:05:15,395][105692] Updated weights for policy 0, policy_version 373894 (0.0009) [2023-12-26 18:05:15,494][105620] Updated weights for policy 1, policy_version 374578 (0.0009) [2023-12-26 18:05:15,548][105620] Updated weights for policy 1, policy_version 374588 (0.0008) [2023-12-26 18:05:15,598][105620] Updated weights for policy 1, policy_version 374598 (0.0009) [2023-12-26 18:05:16,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 191635456. Throughput: 0: 9649.2, 1: 10034.1. Samples: 191608008. Policy #0 lag: (min: 29.0, avg: 53.8, max: 56.0) [2023-12-26 18:05:16,063][104569] Avg episode reward: [(0, '8131.978'), (1, '5857.456')] [2023-12-26 18:05:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000373896_95731712.pth... [2023-12-26 18:05:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000374600_95903744.pth... [2023-12-26 18:05:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000372776_95444992.pth [2023-12-26 18:05:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000373448_95608832.pth [2023-12-26 18:05:16,154][105692] Updated weights for policy 0, policy_version 373904 (0.0008) [2023-12-26 18:05:16,213][105620] Updated weights for policy 1, policy_version 374608 (0.0008) [2023-12-26 18:05:16,219][105692] Updated weights for policy 0, policy_version 373914 (0.0008) [2023-12-26 18:05:16,268][105620] Updated weights for policy 1, policy_version 374618 (0.0010) [2023-12-26 18:05:16,278][105692] Updated weights for policy 0, policy_version 373924 (0.0005) [2023-12-26 18:05:16,319][105620] Updated weights for policy 1, policy_version 374628 (0.0010) [2023-12-26 18:05:16,961][105692] Updated weights for policy 0, policy_version 373934 (0.0005) [2023-12-26 18:05:17,009][105692] Updated weights for policy 0, policy_version 373944 (0.0005) [2023-12-26 18:05:17,055][105620] Updated weights for policy 1, policy_version 374638 (0.0008) [2023-12-26 18:05:17,057][105692] Updated weights for policy 0, policy_version 373954 (0.0005) [2023-12-26 18:05:17,109][105620] Updated weights for policy 1, policy_version 374648 (0.0007) [2023-12-26 18:05:17,162][105620] Updated weights for policy 1, policy_version 374658 (0.0008) [2023-12-26 18:05:17,766][105692] Updated weights for policy 0, policy_version 373964 (0.0007) [2023-12-26 18:05:17,813][105692] Updated weights for policy 0, policy_version 373974 (0.0008) [2023-12-26 18:05:17,873][105692] Updated weights for policy 0, policy_version 373984 (0.0009) [2023-12-26 18:05:17,916][105620] Updated weights for policy 1, policy_version 374668 (0.0008) [2023-12-26 18:05:17,980][105620] Updated weights for policy 1, policy_version 374678 (0.0009) [2023-12-26 18:05:18,040][105620] Updated weights for policy 1, policy_version 374688 (0.0009) [2023-12-26 18:05:18,558][105692] Updated weights for policy 0, policy_version 373994 (0.0009) [2023-12-26 18:05:18,625][105692] Updated weights for policy 0, policy_version 374004 (0.0009) [2023-12-26 18:05:18,670][105692] Updated weights for policy 0, policy_version 374014 (0.0008) [2023-12-26 18:05:18,719][105692] Updated weights for policy 0, policy_version 374024 (0.0011) [2023-12-26 18:05:18,835][105620] Updated weights for policy 1, policy_version 374698 (0.0010) [2023-12-26 18:05:18,899][105620] Updated weights for policy 1, policy_version 374708 (0.0009) [2023-12-26 18:05:18,963][105620] Updated weights for policy 1, policy_version 374718 (0.0009) [2023-12-26 18:05:19,020][105620] Updated weights for policy 1, policy_version 374728 (0.0009) [2023-12-26 18:05:19,486][105692] Updated weights for policy 0, policy_version 374034 (0.0011) [2023-12-26 18:05:19,545][105692] Updated weights for policy 0, policy_version 374044 (0.0009) [2023-12-26 18:05:19,610][105692] Updated weights for policy 0, policy_version 374054 (0.0011) [2023-12-26 18:05:19,751][105620] Updated weights for policy 1, policy_version 374738 (0.0005) [2023-12-26 18:05:19,813][105620] Updated weights for policy 1, policy_version 374748 (0.0006) [2023-12-26 18:05:19,883][105620] Updated weights for policy 1, policy_version 374758 (0.0009) [2023-12-26 18:05:20,415][105692] Updated weights for policy 0, policy_version 374064 (0.0011) [2023-12-26 18:05:20,478][105692] Updated weights for policy 0, policy_version 374074 (0.0011) [2023-12-26 18:05:20,528][105692] Updated weights for policy 0, policy_version 374084 (0.0010) [2023-12-26 18:05:20,621][105620] Updated weights for policy 1, policy_version 374768 (0.0007) [2023-12-26 18:05:20,677][105620] Updated weights for policy 1, policy_version 374778 (0.0008) [2023-12-26 18:05:20,726][105620] Updated weights for policy 1, policy_version 374788 (0.0008) [2023-12-26 18:05:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 191733760. Throughput: 0: 9672.3, 1: 9922.9. Samples: 191724324. Policy #0 lag: (min: 29.0, avg: 53.8, max: 56.0) [2023-12-26 18:05:21,062][104569] Avg episode reward: [(0, '9093.066'), (1, '6866.034')] [2023-12-26 18:05:21,309][105692] Updated weights for policy 0, policy_version 374094 (0.0010) [2023-12-26 18:05:21,377][105692] Updated weights for policy 0, policy_version 374104 (0.0011) [2023-12-26 18:05:21,437][105692] Updated weights for policy 0, policy_version 374114 (0.0011) [2023-12-26 18:05:21,461][105620] Updated weights for policy 1, policy_version 374798 (0.0009) [2023-12-26 18:05:21,521][105620] Updated weights for policy 1, policy_version 374808 (0.0011) [2023-12-26 18:05:21,581][105620] Updated weights for policy 1, policy_version 374818 (0.0010) [2023-12-26 18:05:22,205][105692] Updated weights for policy 0, policy_version 374124 (0.0011) [2023-12-26 18:05:22,272][105692] Updated weights for policy 0, policy_version 374134 (0.0010) [2023-12-26 18:05:22,301][105620] Updated weights for policy 1, policy_version 374828 (0.0010) [2023-12-26 18:05:22,334][105692] Updated weights for policy 0, policy_version 374144 (0.0010) [2023-12-26 18:05:22,367][105620] Updated weights for policy 1, policy_version 374838 (0.0009) [2023-12-26 18:05:22,433][105620] Updated weights for policy 1, policy_version 374848 (0.0007) [2023-12-26 18:05:22,931][105692] Updated weights for policy 0, policy_version 374154 (0.0007) [2023-12-26 18:05:22,994][105692] Updated weights for policy 0, policy_version 374164 (0.0011) [2023-12-26 18:05:23,060][105692] Updated weights for policy 0, policy_version 374174 (0.0011) [2023-12-26 18:05:23,126][105692] Updated weights for policy 0, policy_version 374184 (0.0011) [2023-12-26 18:05:23,161][105620] Updated weights for policy 1, policy_version 374858 (0.0010) [2023-12-26 18:05:23,218][105620] Updated weights for policy 1, policy_version 374869 (0.0010) [2023-12-26 18:05:23,264][105620] Updated weights for policy 1, policy_version 374879 (0.0008) [2023-12-26 18:05:23,770][105692] Updated weights for policy 0, policy_version 374194 (0.0005) [2023-12-26 18:05:23,824][105692] Updated weights for policy 0, policy_version 374204 (0.0005) [2023-12-26 18:05:23,882][105692] Updated weights for policy 0, policy_version 374214 (0.0005) [2023-12-26 18:05:23,893][105620] Updated weights for policy 1, policy_version 374889 (0.0008) [2023-12-26 18:05:23,948][105620] Updated weights for policy 1, policy_version 374899 (0.0006) [2023-12-26 18:05:23,996][105620] Updated weights for policy 1, policy_version 374909 (0.0005) [2023-12-26 18:05:24,039][105620] Updated weights for policy 1, policy_version 374919 (0.0005) [2023-12-26 18:05:24,503][105692] Updated weights for policy 0, policy_version 374224 (0.0006) [2023-12-26 18:05:24,562][105692] Updated weights for policy 0, policy_version 374234 (0.0005) [2023-12-26 18:05:24,625][105692] Updated weights for policy 0, policy_version 374244 (0.0005) [2023-12-26 18:05:24,740][105620] Updated weights for policy 1, policy_version 374929 (0.0009) [2023-12-26 18:05:24,793][105620] Updated weights for policy 1, policy_version 374940 (0.0009) [2023-12-26 18:05:24,851][105620] Updated weights for policy 1, policy_version 374951 (0.0010) [2023-12-26 18:05:25,138][105692] Updated weights for policy 0, policy_version 374254 (0.0009) [2023-12-26 18:05:25,193][105692] Updated weights for policy 0, policy_version 374264 (0.0010) [2023-12-26 18:05:25,243][105692] Updated weights for policy 0, policy_version 374274 (0.0010) [2023-12-26 18:05:25,705][105620] Updated weights for policy 1, policy_version 374961 (0.0009) [2023-12-26 18:05:25,768][105620] Updated weights for policy 1, policy_version 374971 (0.0009) [2023-12-26 18:05:25,815][105620] Updated weights for policy 1, policy_version 374981 (0.0009) [2023-12-26 18:05:25,968][105692] Updated weights for policy 0, policy_version 374284 (0.0010) [2023-12-26 18:05:26,025][105692] Updated weights for policy 0, policy_version 374294 (0.0009) [2023-12-26 18:05:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19660.9, 300 sec: 19522.0). Total num frames: 191832064. Throughput: 0: 9699.2, 1: 9898.3. Samples: 191841744. Policy #0 lag: (min: 29.0, avg: 53.8, max: 56.0) [2023-12-26 18:05:26,062][104569] Avg episode reward: [(0, '8913.740'), (1, '5757.574')] [2023-12-26 18:05:26,071][105692] Updated weights for policy 0, policy_version 374304 (0.0008) [2023-12-26 18:05:26,625][105692] Updated weights for policy 0, policy_version 374314 (0.0005) [2023-12-26 18:05:26,674][105692] Updated weights for policy 0, policy_version 374324 (0.0007) [2023-12-26 18:05:26,692][105620] Updated weights for policy 1, policy_version 374991 (0.0007) [2023-12-26 18:05:26,731][105692] Updated weights for policy 0, policy_version 374334 (0.0005) [2023-12-26 18:05:26,753][105620] Updated weights for policy 1, policy_version 375001 (0.0005) [2023-12-26 18:05:26,772][105586] KL-divergence is very high: 109.8897 [2023-12-26 18:05:26,794][105692] Updated weights for policy 0, policy_version 374344 (0.0006) [2023-12-26 18:05:26,807][105620] Updated weights for policy 1, policy_version 375011 (0.0005) [2023-12-26 18:05:27,485][105620] Updated weights for policy 1, policy_version 375021 (0.0007) [2023-12-26 18:05:27,507][105692] Updated weights for policy 0, policy_version 374354 (0.0008) [2023-12-26 18:05:27,541][105620] Updated weights for policy 1, policy_version 375031 (0.0007) [2023-12-26 18:05:27,552][105692] Updated weights for policy 0, policy_version 374364 (0.0006) [2023-12-26 18:05:27,595][105620] Updated weights for policy 1, policy_version 375041 (0.0007) [2023-12-26 18:05:27,601][105692] Updated weights for policy 0, policy_version 374374 (0.0009) [2023-12-26 18:05:28,208][105692] Updated weights for policy 0, policy_version 374384 (0.0007) [2023-12-26 18:05:28,269][105692] Updated weights for policy 0, policy_version 374394 (0.0009) [2023-12-26 18:05:28,327][105692] Updated weights for policy 0, policy_version 374404 (0.0009) [2023-12-26 18:05:28,411][105620] Updated weights for policy 1, policy_version 375051 (0.0008) [2023-12-26 18:05:28,439][105586] KL-divergence is very high: 212.9305 [2023-12-26 18:05:28,445][105586] KL-divergence is very high: 228.4816 [2023-12-26 18:05:28,452][105586] KL-divergence is very high: 164.7620 [2023-12-26 18:05:28,472][105620] Updated weights for policy 1, policy_version 375061 (0.0009) [2023-12-26 18:05:28,490][105586] KL-divergence is very high: 305.4143 [2023-12-26 18:05:28,496][105586] KL-divergence is very high: 278.4654 [2023-12-26 18:05:28,504][105586] KL-divergence is very high: 183.0032 [2023-12-26 18:05:28,537][105620] Updated weights for policy 1, policy_version 375071 (0.0009) [2023-12-26 18:05:28,542][105586] KL-divergence is very high: 230.0724 [2023-12-26 18:05:28,549][105586] KL-divergence is very high: 196.1889 [2023-12-26 18:05:28,559][105586] KL-divergence is very high: 133.1202 [2023-12-26 18:05:29,057][105692] Updated weights for policy 0, policy_version 374414 (0.0009) [2023-12-26 18:05:29,109][105692] Updated weights for policy 0, policy_version 374424 (0.0006) [2023-12-26 18:05:29,159][105692] Updated weights for policy 0, policy_version 374434 (0.0005) [2023-12-26 18:05:29,332][105620] Updated weights for policy 1, policy_version 375081 (0.0009) [2023-12-26 18:05:29,399][105620] Updated weights for policy 1, policy_version 375091 (0.0008) [2023-12-26 18:05:29,459][105620] Updated weights for policy 1, policy_version 375101 (0.0009) [2023-12-26 18:05:29,511][105620] Updated weights for policy 1, policy_version 375111 (0.0009) [2023-12-26 18:05:29,843][105692] Updated weights for policy 0, policy_version 374444 (0.0007) [2023-12-26 18:05:29,914][105692] Updated weights for policy 0, policy_version 374454 (0.0007) [2023-12-26 18:05:29,980][105692] Updated weights for policy 0, policy_version 374464 (0.0009) [2023-12-26 18:05:30,233][105620] Updated weights for policy 1, policy_version 375121 (0.0009) [2023-12-26 18:05:30,294][105620] Updated weights for policy 1, policy_version 375131 (0.0009) [2023-12-26 18:05:30,355][105620] Updated weights for policy 1, policy_version 375141 (0.0009) [2023-12-26 18:05:30,731][105692] Updated weights for policy 0, policy_version 374474 (0.0010) [2023-12-26 18:05:30,777][105692] Updated weights for policy 0, policy_version 374484 (0.0009) [2023-12-26 18:05:30,828][105692] Updated weights for policy 0, policy_version 374494 (0.0009) [2023-12-26 18:05:30,878][105692] Updated weights for policy 0, policy_version 374504 (0.0009) [2023-12-26 18:05:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 191930368. Throughput: 0: 9781.4, 1: 9833.2. Samples: 191900896. Policy #0 lag: (min: 29.0, avg: 53.8, max: 56.0) [2023-12-26 18:05:31,063][104569] Avg episode reward: [(0, '4355.128'), (1, '5757.125')] [2023-12-26 18:05:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000374504_95887360.pth... [2023-12-26 18:05:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000373352_95592448.pth [2023-12-26 18:05:31,085][105620] Updated weights for policy 1, policy_version 375151 (0.0009) [2023-12-26 18:05:31,153][105620] Updated weights for policy 1, policy_version 375161 (0.0009) [2023-12-26 18:05:31,209][105620] Updated weights for policy 1, policy_version 375171 (0.0009) [2023-12-26 18:05:31,241][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000375176_96051200.pth... [2023-12-26 18:05:31,244][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000374024_95756288.pth [2023-12-26 18:05:31,681][105692] Updated weights for policy 0, policy_version 374514 (0.0007) [2023-12-26 18:05:31,753][105692] Updated weights for policy 0, policy_version 374524 (0.0007) [2023-12-26 18:05:31,816][105692] Updated weights for policy 0, policy_version 374534 (0.0008) [2023-12-26 18:05:32,028][105620] Updated weights for policy 1, policy_version 375181 (0.0009) [2023-12-26 18:05:32,089][105620] Updated weights for policy 1, policy_version 375191 (0.0009) [2023-12-26 18:05:32,146][105620] Updated weights for policy 1, policy_version 375201 (0.0009) [2023-12-26 18:05:32,488][105692] Updated weights for policy 0, policy_version 374544 (0.0005) [2023-12-26 18:05:32,543][105692] Updated weights for policy 0, policy_version 374554 (0.0005) [2023-12-26 18:05:32,615][105692] Updated weights for policy 0, policy_version 374564 (0.0008) [2023-12-26 18:05:32,839][105620] Updated weights for policy 1, policy_version 375211 (0.0009) [2023-12-26 18:05:32,899][105620] Updated weights for policy 1, policy_version 375221 (0.0009) [2023-12-26 18:05:32,959][105620] Updated weights for policy 1, policy_version 375231 (0.0005) [2023-12-26 18:05:33,271][105692] Updated weights for policy 0, policy_version 374574 (0.0009) [2023-12-26 18:05:33,332][105692] Updated weights for policy 0, policy_version 374584 (0.0009) [2023-12-26 18:05:33,389][105692] Updated weights for policy 0, policy_version 374594 (0.0009) [2023-12-26 18:05:33,674][105620] Updated weights for policy 1, policy_version 375241 (0.0008) [2023-12-26 18:05:33,730][105620] Updated weights for policy 1, policy_version 375251 (0.0009) [2023-12-26 18:05:33,739][105586] KL-divergence is very high: 149.1465 [2023-12-26 18:05:33,775][105586] KL-divergence is very high: 103.3946 [2023-12-26 18:05:33,776][105620] Updated weights for policy 1, policy_version 375261 (0.0008) [2023-12-26 18:05:33,842][105620] Updated weights for policy 1, policy_version 375271 (0.0007) [2023-12-26 18:05:34,045][105692] Updated weights for policy 0, policy_version 374604 (0.0010) [2023-12-26 18:05:34,109][105692] Updated weights for policy 0, policy_version 374614 (0.0009) [2023-12-26 18:05:34,180][105692] Updated weights for policy 0, policy_version 374624 (0.0009) [2023-12-26 18:05:34,555][105620] Updated weights for policy 1, policy_version 375281 (0.0008) [2023-12-26 18:05:34,589][105586] KL-divergence is very high: 105.4776 [2023-12-26 18:05:34,620][105620] Updated weights for policy 1, policy_version 375291 (0.0008) [2023-12-26 18:05:34,679][105620] Updated weights for policy 1, policy_version 375301 (0.0009) [2023-12-26 18:05:34,993][105692] Updated weights for policy 0, policy_version 374634 (0.0010) [2023-12-26 18:05:35,057][105692] Updated weights for policy 0, policy_version 374644 (0.0007) [2023-12-26 18:05:35,114][105692] Updated weights for policy 0, policy_version 374654 (0.0010) [2023-12-26 18:05:35,171][105692] Updated weights for policy 0, policy_version 374664 (0.0009) [2023-12-26 18:05:35,303][105620] Updated weights for policy 1, policy_version 375311 (0.0006) [2023-12-26 18:05:35,358][105620] Updated weights for policy 1, policy_version 375321 (0.0007) [2023-12-26 18:05:35,413][105620] Updated weights for policy 1, policy_version 375331 (0.0006) [2023-12-26 18:05:35,815][105692] Updated weights for policy 0, policy_version 374674 (0.0010) [2023-12-26 18:05:35,871][105692] Updated weights for policy 0, policy_version 374684 (0.0010) [2023-12-26 18:05:35,945][105692] Updated weights for policy 0, policy_version 374694 (0.0010) [2023-12-26 18:05:36,044][105620] Updated weights for policy 1, policy_version 375341 (0.0007) [2023-12-26 18:05:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 192028672. Throughput: 0: 9864.0, 1: 9736.7. Samples: 192016640. Policy #0 lag: (min: 29.0, avg: 53.8, max: 56.0) [2023-12-26 18:05:36,062][104569] Avg episode reward: [(0, '3831.629'), (1, '5578.682')] [2023-12-26 18:05:36,109][105620] Updated weights for policy 1, policy_version 375351 (0.0006) [2023-12-26 18:05:36,200][105620] Updated weights for policy 1, policy_version 375361 (0.0006) [2023-12-26 18:05:36,714][105692] Updated weights for policy 0, policy_version 374704 (0.0011) [2023-12-26 18:05:36,772][105692] Updated weights for policy 0, policy_version 374714 (0.0010) [2023-12-26 18:05:36,814][105620] Updated weights for policy 1, policy_version 375371 (0.0006) [2023-12-26 18:05:36,834][105692] Updated weights for policy 0, policy_version 374724 (0.0011) [2023-12-26 18:05:36,872][105620] Updated weights for policy 1, policy_version 375381 (0.0006) [2023-12-26 18:05:36,923][105620] Updated weights for policy 1, policy_version 375391 (0.0007) [2023-12-26 18:05:37,569][105692] Updated weights for policy 0, policy_version 374734 (0.0009) [2023-12-26 18:05:37,624][105692] Updated weights for policy 0, policy_version 374744 (0.0010) [2023-12-26 18:05:37,676][105620] Updated weights for policy 1, policy_version 375401 (0.0008) [2023-12-26 18:05:37,679][105692] Updated weights for policy 0, policy_version 374754 (0.0010) [2023-12-26 18:05:37,737][105620] Updated weights for policy 1, policy_version 375411 (0.0008) [2023-12-26 18:05:37,799][105620] Updated weights for policy 1, policy_version 375421 (0.0006) [2023-12-26 18:05:37,862][105620] Updated weights for policy 1, policy_version 375431 (0.0008) [2023-12-26 18:05:38,447][105692] Updated weights for policy 0, policy_version 374764 (0.0008) [2023-12-26 18:05:38,511][105692] Updated weights for policy 0, policy_version 374774 (0.0005) [2023-12-26 18:05:38,575][105692] Updated weights for policy 0, policy_version 374784 (0.0007) [2023-12-26 18:05:38,636][105620] Updated weights for policy 1, policy_version 375441 (0.0009) [2023-12-26 18:05:38,696][105620] Updated weights for policy 1, policy_version 375451 (0.0009) [2023-12-26 18:05:38,758][105620] Updated weights for policy 1, policy_version 375461 (0.0009) [2023-12-26 18:05:39,144][105692] Updated weights for policy 0, policy_version 374794 (0.0008) [2023-12-26 18:05:39,202][105692] Updated weights for policy 0, policy_version 374804 (0.0005) [2023-12-26 18:05:39,268][105692] Updated weights for policy 0, policy_version 374814 (0.0009) [2023-12-26 18:05:39,330][105692] Updated weights for policy 0, policy_version 374824 (0.0009) [2023-12-26 18:05:39,598][105620] Updated weights for policy 1, policy_version 375471 (0.0009) [2023-12-26 18:05:39,661][105620] Updated weights for policy 1, policy_version 375481 (0.0009) [2023-12-26 18:05:39,724][105620] Updated weights for policy 1, policy_version 375491 (0.0009) [2023-12-26 18:05:40,020][105692] Updated weights for policy 0, policy_version 374834 (0.0009) [2023-12-26 18:05:40,071][105692] Updated weights for policy 0, policy_version 374844 (0.0009) [2023-12-26 18:05:40,133][105692] Updated weights for policy 0, policy_version 374854 (0.0009) [2023-12-26 18:05:40,527][105620] Updated weights for policy 1, policy_version 375501 (0.0009) [2023-12-26 18:05:40,578][105620] Updated weights for policy 1, policy_version 375511 (0.0009) [2023-12-26 18:05:40,583][105586] KL-divergence is very high: 116.4222 [2023-12-26 18:05:40,634][105620] Updated weights for policy 1, policy_version 375521 (0.0009) [2023-12-26 18:05:40,829][105692] Updated weights for policy 0, policy_version 374864 (0.0006) [2023-12-26 18:05:40,886][105692] Updated weights for policy 0, policy_version 374874 (0.0006) [2023-12-26 18:05:40,946][105692] Updated weights for policy 0, policy_version 374884 (0.0009) [2023-12-26 18:05:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 192126976. Throughput: 0: 9878.3, 1: 9669.1. Samples: 192131224. Policy #0 lag: (min: 29.0, avg: 53.8, max: 56.0) [2023-12-26 18:05:41,062][104569] Avg episode reward: [(0, '7240.053'), (1, '5388.949')] [2023-12-26 18:05:41,491][105620] Updated weights for policy 1, policy_version 375531 (0.0010) [2023-12-26 18:05:41,557][105620] Updated weights for policy 1, policy_version 375541 (0.0009) [2023-12-26 18:05:41,621][105620] Updated weights for policy 1, policy_version 375551 (0.0008) [2023-12-26 18:05:41,692][105692] Updated weights for policy 0, policy_version 374894 (0.0007) [2023-12-26 18:05:41,764][105692] Updated weights for policy 0, policy_version 374904 (0.0009) [2023-12-26 18:05:41,823][105692] Updated weights for policy 0, policy_version 374914 (0.0008) [2023-12-26 18:05:42,409][105620] Updated weights for policy 1, policy_version 375561 (0.0009) [2023-12-26 18:05:42,475][105620] Updated weights for policy 1, policy_version 375571 (0.0006) [2023-12-26 18:05:42,539][105620] Updated weights for policy 1, policy_version 375581 (0.0007) [2023-12-26 18:05:42,544][105692] Updated weights for policy 0, policy_version 374924 (0.0007) [2023-12-26 18:05:42,599][105620] Updated weights for policy 1, policy_version 375591 (0.0007) [2023-12-26 18:05:42,602][105692] Updated weights for policy 0, policy_version 374934 (0.0006) [2023-12-26 18:05:42,649][105692] Updated weights for policy 0, policy_version 374944 (0.0007) [2023-12-26 18:05:43,246][105692] Updated weights for policy 0, policy_version 374954 (0.0005) [2023-12-26 18:05:43,297][105620] Updated weights for policy 1, policy_version 375601 (0.0008) [2023-12-26 18:05:43,301][105692] Updated weights for policy 0, policy_version 374964 (0.0006) [2023-12-26 18:05:43,357][105620] Updated weights for policy 1, policy_version 375611 (0.0008) [2023-12-26 18:05:43,358][105692] Updated weights for policy 0, policy_version 374974 (0.0006) [2023-12-26 18:05:43,415][105620] Updated weights for policy 1, policy_version 375621 (0.0007) [2023-12-26 18:05:43,416][105692] Updated weights for policy 0, policy_version 374984 (0.0008) [2023-12-26 18:05:44,033][105692] Updated weights for policy 0, policy_version 374994 (0.0010) [2023-12-26 18:05:44,095][105692] Updated weights for policy 0, policy_version 375004 (0.0010) [2023-12-26 18:05:44,158][105692] Updated weights for policy 0, policy_version 375014 (0.0006) [2023-12-26 18:05:44,233][105620] Updated weights for policy 1, policy_version 375631 (0.0010) [2023-12-26 18:05:44,290][105620] Updated weights for policy 1, policy_version 375641 (0.0009) [2023-12-26 18:05:44,349][105620] Updated weights for policy 1, policy_version 375651 (0.0009) [2023-12-26 18:05:44,854][105692] Updated weights for policy 0, policy_version 375024 (0.0010) [2023-12-26 18:05:44,921][105692] Updated weights for policy 0, policy_version 375034 (0.0011) [2023-12-26 18:05:44,985][105692] Updated weights for policy 0, policy_version 375044 (0.0011) [2023-12-26 18:05:45,176][105620] Updated weights for policy 1, policy_version 375661 (0.0009) [2023-12-26 18:05:45,237][105620] Updated weights for policy 1, policy_version 375671 (0.0008) [2023-12-26 18:05:45,290][105620] Updated weights for policy 1, policy_version 375681 (0.0008) [2023-12-26 18:05:45,734][105692] Updated weights for policy 0, policy_version 375054 (0.0011) [2023-12-26 18:05:45,799][105692] Updated weights for policy 0, policy_version 375064 (0.0010) [2023-12-26 18:05:45,864][105692] Updated weights for policy 0, policy_version 375074 (0.0010) [2023-12-26 18:05:46,040][105620] Updated weights for policy 1, policy_version 375691 (0.0009) [2023-12-26 18:05:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 192217088. Throughput: 0: 9815.1, 1: 9582.0. Samples: 192188776. Policy #0 lag: (min: 31.0, avg: 31.3, max: 45.0) [2023-12-26 18:05:46,063][104569] Avg episode reward: [(0, '9186.333'), (1, '6032.686')] [2023-12-26 18:05:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000375080_96034816.pth... [2023-12-26 18:05:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000373896_95731712.pth [2023-12-26 18:05:46,106][105620] Updated weights for policy 1, policy_version 375701 (0.0009) [2023-12-26 18:05:46,175][105620] Updated weights for policy 1, policy_version 375711 (0.0009) [2023-12-26 18:05:46,217][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000375720_96190464.pth... [2023-12-26 18:05:46,221][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000374600_95903744.pth [2023-12-26 18:05:46,491][105692] Updated weights for policy 0, policy_version 375084 (0.0011) [2023-12-26 18:05:46,553][105692] Updated weights for policy 0, policy_version 375094 (0.0010) [2023-12-26 18:05:46,607][105692] Updated weights for policy 0, policy_version 375104 (0.0010) [2023-12-26 18:05:46,981][105620] Updated weights for policy 1, policy_version 375721 (0.0009) [2023-12-26 18:05:47,049][105620] Updated weights for policy 1, policy_version 375731 (0.0008) [2023-12-26 18:05:47,118][105620] Updated weights for policy 1, policy_version 375741 (0.0008) [2023-12-26 18:05:47,177][105620] Updated weights for policy 1, policy_version 375751 (0.0010) [2023-12-26 18:05:47,255][105692] Updated weights for policy 0, policy_version 375114 (0.0009) [2023-12-26 18:05:47,320][105692] Updated weights for policy 0, policy_version 375124 (0.0005) [2023-12-26 18:05:47,380][105692] Updated weights for policy 0, policy_version 375134 (0.0005) [2023-12-26 18:05:47,438][105692] Updated weights for policy 0, policy_version 375144 (0.0005) [2023-12-26 18:05:48,012][105692] Updated weights for policy 0, policy_version 375154 (0.0006) [2023-12-26 18:05:48,014][105620] Updated weights for policy 1, policy_version 375761 (0.0008) [2023-12-26 18:05:48,069][105692] Updated weights for policy 0, policy_version 375164 (0.0006) [2023-12-26 18:05:48,070][105620] Updated weights for policy 1, policy_version 375771 (0.0007) [2023-12-26 18:05:48,134][105692] Updated weights for policy 0, policy_version 375174 (0.0006) [2023-12-26 18:05:48,135][105620] Updated weights for policy 1, policy_version 375781 (0.0009) [2023-12-26 18:05:48,825][105692] Updated weights for policy 0, policy_version 375184 (0.0005) [2023-12-26 18:05:48,893][105692] Updated weights for policy 0, policy_version 375194 (0.0008) [2023-12-26 18:05:48,946][105620] Updated weights for policy 1, policy_version 375791 (0.0008) [2023-12-26 18:05:48,961][105692] Updated weights for policy 0, policy_version 375204 (0.0008) [2023-12-26 18:05:48,998][105620] Updated weights for policy 1, policy_version 375801 (0.0007) [2023-12-26 18:05:49,046][105586] KL-divergence is very high: 105.3831 [2023-12-26 18:05:49,052][105620] Updated weights for policy 1, policy_version 375812 (0.0010) [2023-12-26 18:05:49,565][105692] Updated weights for policy 0, policy_version 375214 (0.0006) [2023-12-26 18:05:49,622][105692] Updated weights for policy 0, policy_version 375224 (0.0006) [2023-12-26 18:05:49,670][105692] Updated weights for policy 0, policy_version 375234 (0.0010) [2023-12-26 18:05:49,843][105620] Updated weights for policy 1, policy_version 375822 (0.0009) [2023-12-26 18:05:49,865][105586] KL-divergence is very high: 234.5237 [2023-12-26 18:05:49,872][105586] KL-divergence is very high: 290.5957 [2023-12-26 18:05:49,886][105586] KL-divergence is very high: 357.0701 [2023-12-26 18:05:49,893][105586] KL-divergence is very high: 210.4747 [2023-12-26 18:05:49,914][105620] Updated weights for policy 1, policy_version 375832 (0.0009) [2023-12-26 18:05:49,924][105586] KL-divergence is very high: 602.6554 [2023-12-26 18:05:49,931][105586] KL-divergence is very high: 647.7328 [2023-12-26 18:05:49,945][105586] KL-divergence is very high: 526.6217 [2023-12-26 18:05:49,951][105586] KL-divergence is very high: 313.7396 [2023-12-26 18:05:49,973][105586] KL-divergence is very high: 472.3487 [2023-12-26 18:05:49,978][105620] Updated weights for policy 1, policy_version 375842 (0.0008) [2023-12-26 18:05:49,981][105586] KL-divergence is very high: 483.9191 [2023-12-26 18:05:49,993][105586] KL-divergence is very high: 408.3803 [2023-12-26 18:05:49,998][105586] KL-divergence is very high: 262.2423 [2023-12-26 18:05:50,465][105692] Updated weights for policy 0, policy_version 375244 (0.0010) [2023-12-26 18:05:50,532][105692] Updated weights for policy 0, policy_version 375254 (0.0010) [2023-12-26 18:05:50,592][105620] Updated weights for policy 1, policy_version 375852 (0.0009) [2023-12-26 18:05:50,595][105692] Updated weights for policy 0, policy_version 375264 (0.0008) [2023-12-26 18:05:50,604][105586] KL-divergence is very high: 206.9357 [2023-12-26 18:05:50,650][105620] Updated weights for policy 1, policy_version 375862 (0.0007) [2023-12-26 18:05:50,650][105586] KL-divergence is very high: 160.5043 [2023-12-26 18:05:50,692][105586] KL-divergence is very high: 142.1042 [2023-12-26 18:05:50,704][105620] Updated weights for policy 1, policy_version 375872 (0.0009) [2023-12-26 18:05:50,742][105586] KL-divergence is very high: 130.6512 [2023-12-26 18:05:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 192315392. Throughput: 0: 9975.5, 1: 9325.3. Samples: 192304068. Policy #0 lag: (min: 31.0, avg: 31.3, max: 45.0) [2023-12-26 18:05:51,062][104569] Avg episode reward: [(0, '9180.774'), (1, '4746.140')] [2023-12-26 18:05:51,328][105692] Updated weights for policy 0, policy_version 375274 (0.0008) [2023-12-26 18:05:51,393][105692] Updated weights for policy 0, policy_version 375284 (0.0009) [2023-12-26 18:05:51,450][105692] Updated weights for policy 0, policy_version 375294 (0.0009) [2023-12-26 18:05:51,471][105620] Updated weights for policy 1, policy_version 375882 (0.0008) [2023-12-26 18:05:51,507][105692] Updated weights for policy 0, policy_version 375304 (0.0009) [2023-12-26 18:05:51,522][105620] Updated weights for policy 1, policy_version 375892 (0.0006) [2023-12-26 18:05:51,585][105620] Updated weights for policy 1, policy_version 375902 (0.0006) [2023-12-26 18:05:51,654][105620] Updated weights for policy 1, policy_version 375912 (0.0006) [2023-12-26 18:05:52,261][105620] Updated weights for policy 1, policy_version 375922 (0.0009) [2023-12-26 18:05:52,321][105620] Updated weights for policy 1, policy_version 375932 (0.0009) [2023-12-26 18:05:52,357][105692] Updated weights for policy 0, policy_version 375314 (0.0007) [2023-12-26 18:05:52,383][105620] Updated weights for policy 1, policy_version 375942 (0.0009) [2023-12-26 18:05:52,421][105692] Updated weights for policy 0, policy_version 375324 (0.0007) [2023-12-26 18:05:52,489][105692] Updated weights for policy 0, policy_version 375335 (0.0010) [2023-12-26 18:05:53,044][105620] Updated weights for policy 1, policy_version 375952 (0.0006) [2023-12-26 18:05:53,111][105620] Updated weights for policy 1, policy_version 375962 (0.0005) [2023-12-26 18:05:53,181][105620] Updated weights for policy 1, policy_version 375972 (0.0005) [2023-12-26 18:05:53,299][105692] Updated weights for policy 0, policy_version 375345 (0.0008) [2023-12-26 18:05:53,354][105692] Updated weights for policy 0, policy_version 375355 (0.0006) [2023-12-26 18:05:53,413][105692] Updated weights for policy 0, policy_version 375365 (0.0005) [2023-12-26 18:05:53,718][105620] Updated weights for policy 1, policy_version 375982 (0.0008) [2023-12-26 18:05:53,770][105620] Updated weights for policy 1, policy_version 375992 (0.0010) [2023-12-26 18:05:53,821][105620] Updated weights for policy 1, policy_version 376002 (0.0010) [2023-12-26 18:05:54,203][105692] Updated weights for policy 0, policy_version 375375 (0.0008) [2023-12-26 18:05:54,260][105692] Updated weights for policy 0, policy_version 375385 (0.0010) [2023-12-26 18:05:54,320][105692] Updated weights for policy 0, policy_version 375395 (0.0010) [2023-12-26 18:05:54,447][105620] Updated weights for policy 1, policy_version 376012 (0.0010) [2023-12-26 18:05:54,495][105620] Updated weights for policy 1, policy_version 376022 (0.0009) [2023-12-26 18:05:54,553][105620] Updated weights for policy 1, policy_version 376032 (0.0010) [2023-12-26 18:05:55,115][105692] Updated weights for policy 0, policy_version 375405 (0.0007) [2023-12-26 18:05:55,168][105692] Updated weights for policy 0, policy_version 375415 (0.0008) [2023-12-26 18:05:55,220][105692] Updated weights for policy 0, policy_version 375425 (0.0008) [2023-12-26 18:05:55,324][105620] Updated weights for policy 1, policy_version 376042 (0.0010) [2023-12-26 18:05:55,394][105620] Updated weights for policy 1, policy_version 376052 (0.0007) [2023-12-26 18:05:55,462][105620] Updated weights for policy 1, policy_version 376062 (0.0007) [2023-12-26 18:05:55,510][105620] Updated weights for policy 1, policy_version 376072 (0.0005) [2023-12-26 18:05:56,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 192405504. Throughput: 0: 9841.3, 1: 9468.2. Samples: 192420876. Policy #0 lag: (min: 31.0, avg: 31.3, max: 45.0) [2023-12-26 18:05:56,062][104569] Avg episode reward: [(0, '9179.941'), (1, '637.975')] [2023-12-26 18:05:56,081][105620] Updated weights for policy 1, policy_version 376082 (0.0008) [2023-12-26 18:05:56,091][105692] Updated weights for policy 0, policy_version 375435 (0.0008) [2023-12-26 18:05:56,145][105692] Updated weights for policy 0, policy_version 375445 (0.0008) [2023-12-26 18:05:56,148][105620] Updated weights for policy 1, policy_version 376092 (0.0008) [2023-12-26 18:05:56,169][105586] KL-divergence is very high: 150.4974 [2023-12-26 18:05:56,204][105692] Updated weights for policy 0, policy_version 375455 (0.0007) [2023-12-26 18:05:56,217][105620] Updated weights for policy 1, policy_version 376102 (0.0008) [2023-12-26 18:05:56,226][105586] KL-divergence is very high: 126.2332 [2023-12-26 18:05:56,770][105620] Updated weights for policy 1, policy_version 376112 (0.0006) [2023-12-26 18:05:56,826][105620] Updated weights for policy 1, policy_version 376122 (0.0005) [2023-12-26 18:05:56,878][105620] Updated weights for policy 1, policy_version 376132 (0.0006) [2023-12-26 18:05:57,062][105692] Updated weights for policy 0, policy_version 375465 (0.0009) [2023-12-26 18:05:57,110][105692] Updated weights for policy 0, policy_version 375475 (0.0009) [2023-12-26 18:05:57,162][105692] Updated weights for policy 0, policy_version 375485 (0.0008) [2023-12-26 18:05:57,221][105692] Updated weights for policy 0, policy_version 375495 (0.0009) [2023-12-26 18:05:57,562][105620] Updated weights for policy 1, policy_version 376142 (0.0010) [2023-12-26 18:05:57,619][105620] Updated weights for policy 1, policy_version 376152 (0.0010) [2023-12-26 18:05:57,663][105620] Updated weights for policy 1, policy_version 376162 (0.0010) [2023-12-26 18:05:57,987][105692] Updated weights for policy 0, policy_version 375505 (0.0009) [2023-12-26 18:05:58,042][105692] Updated weights for policy 0, policy_version 375515 (0.0009) [2023-12-26 18:05:58,101][105692] Updated weights for policy 0, policy_version 375525 (0.0008) [2023-12-26 18:05:58,419][105620] Updated weights for policy 1, policy_version 376172 (0.0010) [2023-12-26 18:05:58,483][105620] Updated weights for policy 1, policy_version 376182 (0.0010) [2023-12-26 18:05:58,548][105620] Updated weights for policy 1, policy_version 376192 (0.0009) [2023-12-26 18:05:58,936][105692] Updated weights for policy 0, policy_version 375535 (0.0007) [2023-12-26 18:05:58,998][105692] Updated weights for policy 0, policy_version 375545 (0.0009) [2023-12-26 18:05:59,047][105692] Updated weights for policy 0, policy_version 375555 (0.0008) [2023-12-26 18:05:59,315][105620] Updated weights for policy 1, policy_version 376202 (0.0008) [2023-12-26 18:05:59,380][105620] Updated weights for policy 1, policy_version 376212 (0.0008) [2023-12-26 18:05:59,433][105620] Updated weights for policy 1, policy_version 376222 (0.0009) [2023-12-26 18:05:59,486][105620] Updated weights for policy 1, policy_version 376232 (0.0009) [2023-12-26 18:05:59,760][105692] Updated weights for policy 0, policy_version 375565 (0.0009) [2023-12-26 18:05:59,808][105692] Updated weights for policy 0, policy_version 375575 (0.0009) [2023-12-26 18:05:59,869][105692] Updated weights for policy 0, policy_version 375585 (0.0008) [2023-12-26 18:06:00,194][105620] Updated weights for policy 1, policy_version 376242 (0.0007) [2023-12-26 18:06:00,247][105620] Updated weights for policy 1, policy_version 376253 (0.0011) [2023-12-26 18:06:00,516][105692] Updated weights for policy 0, policy_version 375595 (0.0007) [2023-12-26 18:06:00,578][105692] Updated weights for policy 0, policy_version 375605 (0.0009) [2023-12-26 18:06:00,639][105692] Updated weights for policy 0, policy_version 375615 (0.0006) [2023-12-26 18:06:00,923][105620] Updated weights for policy 1, policy_version 376266 (0.0010) [2023-12-26 18:06:00,982][105620] Updated weights for policy 1, policy_version 376276 (0.0009) [2023-12-26 18:06:01,035][105620] Updated weights for policy 1, policy_version 376286 (0.0011) [2023-12-26 18:06:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 192503808. Throughput: 0: 9790.1, 1: 9519.3. Samples: 192476928. Policy #0 lag: (min: 31.0, avg: 31.3, max: 45.0) [2023-12-26 18:06:01,063][104569] Avg episode reward: [(0, '9178.987'), (1, '600.367')] [2023-12-26 18:06:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000375624_96174080.pth... [2023-12-26 18:06:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000374504_95887360.pth [2023-12-26 18:06:01,096][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000376296_96337920.pth... [2023-12-26 18:06:01,096][105620] Updated weights for policy 1, policy_version 376296 (0.0010) [2023-12-26 18:06:01,100][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000375176_96051200.pth [2023-12-26 18:06:01,253][105692] Updated weights for policy 0, policy_version 375625 (0.0006) [2023-12-26 18:06:01,305][105692] Updated weights for policy 0, policy_version 375635 (0.0008) [2023-12-26 18:06:01,360][105692] Updated weights for policy 0, policy_version 375645 (0.0008) [2023-12-26 18:06:01,420][105692] Updated weights for policy 0, policy_version 375655 (0.0008) [2023-12-26 18:06:01,792][105620] Updated weights for policy 1, policy_version 376306 (0.0006) [2023-12-26 18:06:01,854][105620] Updated weights for policy 1, policy_version 376316 (0.0005) [2023-12-26 18:06:01,915][105620] Updated weights for policy 1, policy_version 376326 (0.0008) [2023-12-26 18:06:02,181][105692] Updated weights for policy 0, policy_version 375665 (0.0009) [2023-12-26 18:06:02,228][105692] Updated weights for policy 0, policy_version 375675 (0.0008) [2023-12-26 18:06:02,284][105692] Updated weights for policy 0, policy_version 375685 (0.0009) [2023-12-26 18:06:02,591][105620] Updated weights for policy 1, policy_version 376337 (0.0009) [2023-12-26 18:06:02,648][105620] Updated weights for policy 1, policy_version 376347 (0.0009) [2023-12-26 18:06:02,708][105620] Updated weights for policy 1, policy_version 376358 (0.0011) [2023-12-26 18:06:02,896][105692] Updated weights for policy 0, policy_version 375695 (0.0008) [2023-12-26 18:06:02,955][105692] Updated weights for policy 0, policy_version 375705 (0.0009) [2023-12-26 18:06:03,013][105692] Updated weights for policy 0, policy_version 375715 (0.0008) [2023-12-26 18:06:03,559][105620] Updated weights for policy 1, policy_version 376368 (0.0010) [2023-12-26 18:06:03,610][105620] Updated weights for policy 1, policy_version 376378 (0.0009) [2023-12-26 18:06:03,616][105692] Updated weights for policy 0, policy_version 375725 (0.0007) [2023-12-26 18:06:03,669][105620] Updated weights for policy 1, policy_version 376388 (0.0009) [2023-12-26 18:06:03,669][105692] Updated weights for policy 0, policy_version 375735 (0.0005) [2023-12-26 18:06:03,718][105692] Updated weights for policy 0, policy_version 375745 (0.0005) [2023-12-26 18:06:04,365][105692] Updated weights for policy 0, policy_version 375755 (0.0007) [2023-12-26 18:06:04,392][105620] Updated weights for policy 1, policy_version 376398 (0.0007) [2023-12-26 18:06:04,430][105692] Updated weights for policy 0, policy_version 375765 (0.0011) [2023-12-26 18:06:04,457][105620] Updated weights for policy 1, policy_version 376408 (0.0007) [2023-12-26 18:06:04,486][105692] Updated weights for policy 0, policy_version 375775 (0.0011) [2023-12-26 18:06:04,517][105620] Updated weights for policy 1, policy_version 376418 (0.0005) [2023-12-26 18:06:05,191][105692] Updated weights for policy 0, policy_version 375785 (0.0011) [2023-12-26 18:06:05,219][105620] Updated weights for policy 1, policy_version 376428 (0.0007) [2023-12-26 18:06:05,253][105692] Updated weights for policy 0, policy_version 375795 (0.0010) [2023-12-26 18:06:05,276][105620] Updated weights for policy 1, policy_version 376438 (0.0009) [2023-12-26 18:06:05,315][105692] Updated weights for policy 0, policy_version 375805 (0.0010) [2023-12-26 18:06:05,338][105620] Updated weights for policy 1, policy_version 376448 (0.0010) [2023-12-26 18:06:05,373][105692] Updated weights for policy 0, policy_version 375815 (0.0010) [2023-12-26 18:06:05,931][105620] Updated weights for policy 1, policy_version 376458 (0.0006) [2023-12-26 18:06:05,985][105620] Updated weights for policy 1, policy_version 376468 (0.0005) [2023-12-26 18:06:06,017][105692] Updated weights for policy 0, policy_version 375825 (0.0010) [2023-12-26 18:06:06,033][105620] Updated weights for policy 1, policy_version 376478 (0.0005) [2023-12-26 18:06:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 192602112. Throughput: 0: 9849.9, 1: 9527.9. Samples: 192596324. Policy #0 lag: (min: 31.0, avg: 31.3, max: 45.0) [2023-12-26 18:06:06,062][104569] Avg episode reward: [(0, '9267.919'), (1, '964.424')] [2023-12-26 18:06:06,072][105692] Updated weights for policy 0, policy_version 375835 (0.0010) [2023-12-26 18:06:06,083][105620] Updated weights for policy 1, policy_version 376488 (0.0007) [2023-12-26 18:06:06,137][105692] Updated weights for policy 0, policy_version 375845 (0.0010) [2023-12-26 18:06:06,676][105620] Updated weights for policy 1, policy_version 376498 (0.0008) [2023-12-26 18:06:06,732][105620] Updated weights for policy 1, policy_version 376508 (0.0008) [2023-12-26 18:06:06,791][105620] Updated weights for policy 1, policy_version 376518 (0.0008) [2023-12-26 18:06:06,895][105692] Updated weights for policy 0, policy_version 375855 (0.0009) [2023-12-26 18:06:06,953][105692] Updated weights for policy 0, policy_version 375865 (0.0008) [2023-12-26 18:06:07,014][105692] Updated weights for policy 0, policy_version 375875 (0.0008) [2023-12-26 18:06:07,527][105620] Updated weights for policy 1, policy_version 376528 (0.0010) [2023-12-26 18:06:07,581][105620] Updated weights for policy 1, policy_version 376538 (0.0007) [2023-12-26 18:06:07,627][105620] Updated weights for policy 1, policy_version 376548 (0.0005) [2023-12-26 18:06:07,799][105692] Updated weights for policy 0, policy_version 375885 (0.0007) [2023-12-26 18:06:07,849][105692] Updated weights for policy 0, policy_version 375896 (0.0007) [2023-12-26 18:06:07,906][105692] Updated weights for policy 0, policy_version 375906 (0.0009) [2023-12-26 18:06:08,250][105620] Updated weights for policy 1, policy_version 376558 (0.0008) [2023-12-26 18:06:08,298][105620] Updated weights for policy 1, policy_version 376568 (0.0010) [2023-12-26 18:06:08,355][105620] Updated weights for policy 1, policy_version 376578 (0.0010) [2023-12-26 18:06:08,618][105692] Updated weights for policy 0, policy_version 375916 (0.0009) [2023-12-26 18:06:08,663][105692] Updated weights for policy 0, policy_version 375926 (0.0008) [2023-12-26 18:06:08,708][105692] Updated weights for policy 0, policy_version 375936 (0.0008) [2023-12-26 18:06:09,119][105620] Updated weights for policy 1, policy_version 376588 (0.0008) [2023-12-26 18:06:09,182][105620] Updated weights for policy 1, policy_version 376598 (0.0006) [2023-12-26 18:06:09,250][105620] Updated weights for policy 1, policy_version 376608 (0.0007) [2023-12-26 18:06:09,420][105692] Updated weights for policy 0, policy_version 375946 (0.0008) [2023-12-26 18:06:09,483][105692] Updated weights for policy 0, policy_version 375956 (0.0008) [2023-12-26 18:06:09,543][105692] Updated weights for policy 0, policy_version 375966 (0.0008) [2023-12-26 18:06:09,607][105692] Updated weights for policy 0, policy_version 375976 (0.0007) [2023-12-26 18:06:09,991][105620] Updated weights for policy 1, policy_version 376618 (0.0008) [2023-12-26 18:06:10,055][105620] Updated weights for policy 1, policy_version 376628 (0.0011) [2023-12-26 18:06:10,114][105620] Updated weights for policy 1, policy_version 376638 (0.0008) [2023-12-26 18:06:10,174][105620] Updated weights for policy 1, policy_version 376648 (0.0011) [2023-12-26 18:06:10,285][105692] Updated weights for policy 0, policy_version 375986 (0.0008) [2023-12-26 18:06:10,345][105692] Updated weights for policy 0, policy_version 375996 (0.0006) [2023-12-26 18:06:10,402][105692] Updated weights for policy 0, policy_version 376006 (0.0008) [2023-12-26 18:06:10,973][105620] Updated weights for policy 1, policy_version 376658 (0.0009) [2023-12-26 18:06:11,035][105620] Updated weights for policy 1, policy_version 376668 (0.0009) [2023-12-26 18:06:11,040][105692] Updated weights for policy 0, policy_version 376016 (0.0007) [2023-12-26 18:06:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 192700416. Throughput: 0: 9805.2, 1: 9607.8. Samples: 192715332. Policy #0 lag: (min: 31.0, avg: 31.3, max: 45.0) [2023-12-26 18:06:11,063][104569] Avg episode reward: [(0, '9268.215'), (1, '2853.822')] [2023-12-26 18:06:11,095][105620] Updated weights for policy 1, policy_version 376678 (0.0008) [2023-12-26 18:06:11,106][105692] Updated weights for policy 0, policy_version 376026 (0.0006) [2023-12-26 18:06:11,173][105692] Updated weights for policy 0, policy_version 376036 (0.0008) [2023-12-26 18:06:11,935][105620] Updated weights for policy 1, policy_version 376688 (0.0008) [2023-12-26 18:06:11,973][105692] Updated weights for policy 0, policy_version 376046 (0.0010) [2023-12-26 18:06:11,984][105620] Updated weights for policy 1, policy_version 376698 (0.0006) [2023-12-26 18:06:12,028][105692] Updated weights for policy 0, policy_version 376056 (0.0010) [2023-12-26 18:06:12,035][105620] Updated weights for policy 1, policy_version 376708 (0.0009) [2023-12-26 18:06:12,081][105692] Updated weights for policy 0, policy_version 376066 (0.0008) [2023-12-26 18:06:12,692][105620] Updated weights for policy 1, policy_version 376718 (0.0008) [2023-12-26 18:06:12,745][105620] Updated weights for policy 1, policy_version 376728 (0.0008) [2023-12-26 18:06:12,805][105620] Updated weights for policy 1, policy_version 376738 (0.0008) [2023-12-26 18:06:12,807][105692] Updated weights for policy 0, policy_version 376076 (0.0008) [2023-12-26 18:06:12,870][105692] Updated weights for policy 0, policy_version 376086 (0.0010) [2023-12-26 18:06:12,926][105692] Updated weights for policy 0, policy_version 376096 (0.0007) [2023-12-26 18:06:13,578][105620] Updated weights for policy 1, policy_version 376748 (0.0005) [2023-12-26 18:06:13,579][105692] Updated weights for policy 0, policy_version 376106 (0.0005) [2023-12-26 18:06:13,627][105620] Updated weights for policy 1, policy_version 376758 (0.0010) [2023-12-26 18:06:13,638][105692] Updated weights for policy 0, policy_version 376116 (0.0005) [2023-12-26 18:06:13,671][105620] Updated weights for policy 1, policy_version 376768 (0.0010) [2023-12-26 18:06:13,698][105692] Updated weights for policy 0, policy_version 376126 (0.0006) [2023-12-26 18:06:13,762][105692] Updated weights for policy 0, policy_version 376136 (0.0006) [2023-12-26 18:06:14,231][105620] Updated weights for policy 1, policy_version 376778 (0.0009) [2023-12-26 18:06:14,288][105620] Updated weights for policy 1, policy_version 376788 (0.0009) [2023-12-26 18:06:14,346][105620] Updated weights for policy 1, policy_version 376798 (0.0007) [2023-12-26 18:06:14,407][105620] Updated weights for policy 1, policy_version 376808 (0.0007) [2023-12-26 18:06:14,522][105692] Updated weights for policy 0, policy_version 376146 (0.0008) [2023-12-26 18:06:14,574][105692] Updated weights for policy 0, policy_version 376156 (0.0008) [2023-12-26 18:06:14,623][105692] Updated weights for policy 0, policy_version 376166 (0.0009) [2023-12-26 18:06:15,126][105620] Updated weights for policy 1, policy_version 376818 (0.0011) [2023-12-26 18:06:15,189][105620] Updated weights for policy 1, policy_version 376828 (0.0010) [2023-12-26 18:06:15,254][105620] Updated weights for policy 1, policy_version 376838 (0.0011) [2023-12-26 18:06:15,439][105692] Updated weights for policy 0, policy_version 376176 (0.0009) [2023-12-26 18:06:15,501][105692] Updated weights for policy 0, policy_version 376186 (0.0009) [2023-12-26 18:06:15,554][105692] Updated weights for policy 0, policy_version 376196 (0.0011) [2023-12-26 18:06:16,005][105620] Updated weights for policy 1, policy_version 376848 (0.0011) [2023-12-26 18:06:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 192798720. Throughput: 0: 9757.2, 1: 9642.9. Samples: 192773896. Policy #0 lag: (min: 31.0, avg: 31.3, max: 45.0) [2023-12-26 18:06:16,062][104569] Avg episode reward: [(0, '9180.370'), (1, '2516.178')] [2023-12-26 18:06:16,067][105620] Updated weights for policy 1, policy_version 376858 (0.0010) [2023-12-26 18:06:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000376200_96321536.pth... [2023-12-26 18:06:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000375080_96034816.pth [2023-12-26 18:06:16,125][105620] Updated weights for policy 1, policy_version 376868 (0.0010) [2023-12-26 18:06:16,146][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000376872_96485376.pth... [2023-12-26 18:06:16,149][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000375720_96190464.pth [2023-12-26 18:06:16,194][105692] Updated weights for policy 0, policy_version 376206 (0.0007) [2023-12-26 18:06:16,247][105692] Updated weights for policy 0, policy_version 376216 (0.0005) [2023-12-26 18:06:16,298][105692] Updated weights for policy 0, policy_version 376226 (0.0008) [2023-12-26 18:06:16,817][105620] Updated weights for policy 1, policy_version 376878 (0.0007) [2023-12-26 18:06:16,875][105620] Updated weights for policy 1, policy_version 376888 (0.0008) [2023-12-26 18:06:16,927][105620] Updated weights for policy 1, policy_version 376898 (0.0006) [2023-12-26 18:06:16,980][105692] Updated weights for policy 0, policy_version 376236 (0.0010) [2023-12-26 18:06:17,047][105692] Updated weights for policy 0, policy_version 376246 (0.0007) [2023-12-26 18:06:17,095][105692] Updated weights for policy 0, policy_version 376256 (0.0008) [2023-12-26 18:06:17,577][105620] Updated weights for policy 1, policy_version 376908 (0.0007) [2023-12-26 18:06:17,630][105620] Updated weights for policy 1, policy_version 376918 (0.0005) [2023-12-26 18:06:17,684][105620] Updated weights for policy 1, policy_version 376928 (0.0006) [2023-12-26 18:06:17,765][105692] Updated weights for policy 0, policy_version 376266 (0.0008) [2023-12-26 18:06:17,827][105692] Updated weights for policy 0, policy_version 376276 (0.0005) [2023-12-26 18:06:17,873][105692] Updated weights for policy 0, policy_version 376286 (0.0008) [2023-12-26 18:06:17,917][105692] Updated weights for policy 0, policy_version 376296 (0.0008) [2023-12-26 18:06:18,397][105620] Updated weights for policy 1, policy_version 376938 (0.0011) [2023-12-26 18:06:18,466][105620] Updated weights for policy 1, policy_version 376948 (0.0007) [2023-12-26 18:06:18,529][105620] Updated weights for policy 1, policy_version 376958 (0.0010) [2023-12-26 18:06:18,543][105692] Updated weights for policy 0, policy_version 376306 (0.0006) [2023-12-26 18:06:18,564][105586] KL-divergence is very high: 134.2831 [2023-12-26 18:06:18,582][105586] KL-divergence is very high: 126.2318 [2023-12-26 18:06:18,588][105586] KL-divergence is very high: 101.7408 [2023-12-26 18:06:18,595][105620] Updated weights for policy 1, policy_version 376968 (0.0011) [2023-12-26 18:06:18,607][105692] Updated weights for policy 0, policy_version 376316 (0.0010) [2023-12-26 18:06:18,676][105692] Updated weights for policy 0, policy_version 376326 (0.0010) [2023-12-26 18:06:19,231][105586] KL-divergence is very high: 344.3739 [2023-12-26 18:06:19,276][105586] KL-divergence is very high: 160.5046 [2023-12-26 18:06:19,283][105586] KL-divergence is very high: 596.3397 [2023-12-26 18:06:19,290][105586] KL-divergence is very high: 202.6863 [2023-12-26 18:06:19,297][105620] Updated weights for policy 1, policy_version 376978 (0.0011) [2023-12-26 18:06:19,302][105586] KL-divergence is very high: 190.2941 [2023-12-26 18:06:19,311][105586] KL-divergence is very high: 216.6476 [2023-12-26 18:06:19,331][105586] KL-divergence is very high: 181.1152 [2023-12-26 18:06:19,338][105586] KL-divergence is very high: 664.9829 [2023-12-26 18:06:19,345][105586] KL-divergence is very high: 182.7639 [2023-12-26 18:06:19,360][105586] KL-divergence is very high: 130.8646 [2023-12-26 18:06:19,361][105692] Updated weights for policy 0, policy_version 376336 (0.0009) [2023-12-26 18:06:19,365][105620] Updated weights for policy 1, policy_version 376988 (0.0010) [2023-12-26 18:06:19,366][105586] KL-divergence is very high: 164.9951 [2023-12-26 18:06:19,396][105586] KL-divergence is very high: 588.1704 [2023-12-26 18:06:19,402][105586] KL-divergence is very high: 110.3099 [2023-12-26 18:06:19,428][105692] Updated weights for policy 0, policy_version 376346 (0.0006) [2023-12-26 18:06:19,433][105620] Updated weights for policy 1, policy_version 376998 (0.0010) [2023-12-26 18:06:19,498][105692] Updated weights for policy 0, policy_version 376356 (0.0009) [2023-12-26 18:06:20,108][105620] Updated weights for policy 1, policy_version 377008 (0.0009) [2023-12-26 18:06:20,169][105620] Updated weights for policy 1, policy_version 377018 (0.0008) [2023-12-26 18:06:20,227][105620] Updated weights for policy 1, policy_version 377028 (0.0010) [2023-12-26 18:06:20,264][105692] Updated weights for policy 0, policy_version 376366 (0.0007) [2023-12-26 18:06:20,327][105692] Updated weights for policy 0, policy_version 376376 (0.0009) [2023-12-26 18:06:20,391][105692] Updated weights for policy 0, policy_version 376386 (0.0009) [2023-12-26 18:06:21,009][105620] Updated weights for policy 1, policy_version 377038 (0.0009) [2023-12-26 18:06:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 192897024. Throughput: 0: 9780.7, 1: 9691.6. Samples: 192892892. Policy #0 lag: (min: 31.0, avg: 31.3, max: 45.0) [2023-12-26 18:06:21,062][104569] Avg episode reward: [(0, '9268.789'), (1, '2448.723')] [2023-12-26 18:06:21,073][105620] Updated weights for policy 1, policy_version 377048 (0.0009) [2023-12-26 18:06:21,113][105692] Updated weights for policy 0, policy_version 376396 (0.0009) [2023-12-26 18:06:21,136][105620] Updated weights for policy 1, policy_version 377058 (0.0008) [2023-12-26 18:06:21,183][105692] Updated weights for policy 0, policy_version 376406 (0.0009) [2023-12-26 18:06:21,253][105692] Updated weights for policy 0, policy_version 376416 (0.0007) [2023-12-26 18:06:21,936][105692] Updated weights for policy 0, policy_version 376426 (0.0008) [2023-12-26 18:06:21,985][105620] Updated weights for policy 1, policy_version 377068 (0.0009) [2023-12-26 18:06:21,996][105692] Updated weights for policy 0, policy_version 376436 (0.0008) [2023-12-26 18:06:22,042][105620] Updated weights for policy 1, policy_version 377078 (0.0008) [2023-12-26 18:06:22,058][105692] Updated weights for policy 0, policy_version 376446 (0.0007) [2023-12-26 18:06:22,102][105620] Updated weights for policy 1, policy_version 377088 (0.0010) [2023-12-26 18:06:22,116][105692] Updated weights for policy 0, policy_version 376456 (0.0008) [2023-12-26 18:06:22,909][105620] Updated weights for policy 1, policy_version 377098 (0.0010) [2023-12-26 18:06:22,937][105692] Updated weights for policy 0, policy_version 376466 (0.0006) [2023-12-26 18:06:22,975][105620] Updated weights for policy 1, policy_version 377108 (0.0009) [2023-12-26 18:06:22,997][105692] Updated weights for policy 0, policy_version 376476 (0.0005) [2023-12-26 18:06:23,040][105620] Updated weights for policy 1, policy_version 377118 (0.0009) [2023-12-26 18:06:23,057][105692] Updated weights for policy 0, policy_version 376486 (0.0005) [2023-12-26 18:06:23,108][105620] Updated weights for policy 1, policy_version 377128 (0.0009) [2023-12-26 18:06:23,765][105692] Updated weights for policy 0, policy_version 376496 (0.0008) [2023-12-26 18:06:23,798][105620] Updated weights for policy 1, policy_version 377138 (0.0005) [2023-12-26 18:06:23,822][105692] Updated weights for policy 0, policy_version 376506 (0.0010) [2023-12-26 18:06:23,859][105620] Updated weights for policy 1, policy_version 377148 (0.0005) [2023-12-26 18:06:23,875][105692] Updated weights for policy 0, policy_version 376516 (0.0009) [2023-12-26 18:06:23,912][105620] Updated weights for policy 1, policy_version 377158 (0.0005) [2023-12-26 18:06:24,590][105620] Updated weights for policy 1, policy_version 377168 (0.0007) [2023-12-26 18:06:24,600][105692] Updated weights for policy 0, policy_version 376526 (0.0008) [2023-12-26 18:06:24,643][105620] Updated weights for policy 1, policy_version 377178 (0.0006) [2023-12-26 18:06:24,649][105692] Updated weights for policy 0, policy_version 376536 (0.0006) [2023-12-26 18:06:24,694][105620] Updated weights for policy 1, policy_version 377188 (0.0006) [2023-12-26 18:06:24,708][105692] Updated weights for policy 0, policy_version 376546 (0.0007) [2023-12-26 18:06:25,318][105620] Updated weights for policy 1, policy_version 377198 (0.0009) [2023-12-26 18:06:25,376][105620] Updated weights for policy 1, policy_version 377208 (0.0009) [2023-12-26 18:06:25,438][105620] Updated weights for policy 1, policy_version 377218 (0.0011) [2023-12-26 18:06:25,560][105692] Updated weights for policy 0, policy_version 376556 (0.0009) [2023-12-26 18:06:25,620][105692] Updated weights for policy 0, policy_version 376566 (0.0010) [2023-12-26 18:06:25,676][105692] Updated weights for policy 0, policy_version 376576 (0.0009) [2023-12-26 18:06:25,993][105620] Updated weights for policy 1, policy_version 377228 (0.0010) [2023-12-26 18:06:26,044][105620] Updated weights for policy 1, policy_version 377238 (0.0010) [2023-12-26 18:06:26,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19387.6, 300 sec: 19549.7). Total num frames: 192995328. Throughput: 0: 9716.1, 1: 9748.5. Samples: 193007136. Policy #0 lag: (min: 31.0, avg: 31.3, max: 45.0) [2023-12-26 18:06:26,063][104569] Avg episode reward: [(0, '9356.215'), (1, '3087.805')] [2023-12-26 18:06:26,095][105620] Updated weights for policy 1, policy_version 377248 (0.0010) [2023-12-26 18:06:26,517][105692] Updated weights for policy 0, policy_version 376587 (0.0010) [2023-12-26 18:06:26,576][105692] Updated weights for policy 0, policy_version 376597 (0.0009) [2023-12-26 18:06:26,634][105692] Updated weights for policy 0, policy_version 376607 (0.0009) [2023-12-26 18:06:26,758][105620] Updated weights for policy 1, policy_version 377258 (0.0008) [2023-12-26 18:06:26,817][105620] Updated weights for policy 1, policy_version 377268 (0.0008) [2023-12-26 18:06:26,873][105620] Updated weights for policy 1, policy_version 377278 (0.0009) [2023-12-26 18:06:26,929][105620] Updated weights for policy 1, policy_version 377288 (0.0009) [2023-12-26 18:06:27,411][105692] Updated weights for policy 0, policy_version 376617 (0.0010) [2023-12-26 18:06:27,464][105692] Updated weights for policy 0, policy_version 376627 (0.0010) [2023-12-26 18:06:27,517][105692] Updated weights for policy 0, policy_version 376637 (0.0010) [2023-12-26 18:06:27,577][105692] Updated weights for policy 0, policy_version 376647 (0.0008) [2023-12-26 18:06:27,613][105620] Updated weights for policy 1, policy_version 377298 (0.0007) [2023-12-26 18:06:27,671][105620] Updated weights for policy 1, policy_version 377308 (0.0005) [2023-12-26 18:06:27,732][105620] Updated weights for policy 1, policy_version 377318 (0.0005) [2023-12-26 18:06:28,316][105692] Updated weights for policy 0, policy_version 376657 (0.0006) [2023-12-26 18:06:28,322][105620] Updated weights for policy 1, policy_version 377328 (0.0007) [2023-12-26 18:06:28,376][105692] Updated weights for policy 0, policy_version 376667 (0.0007) [2023-12-26 18:06:28,382][105620] Updated weights for policy 1, policy_version 377338 (0.0007) [2023-12-26 18:06:28,433][105692] Updated weights for policy 0, policy_version 376677 (0.0008) [2023-12-26 18:06:28,441][105620] Updated weights for policy 1, policy_version 377348 (0.0005) [2023-12-26 18:06:29,054][105620] Updated weights for policy 1, policy_version 377358 (0.0005) [2023-12-26 18:06:29,114][105620] Updated weights for policy 1, policy_version 377368 (0.0006) [2023-12-26 18:06:29,163][105620] Updated weights for policy 1, policy_version 377378 (0.0009) [2023-12-26 18:06:29,209][105692] Updated weights for policy 0, policy_version 376687 (0.0007) [2023-12-26 18:06:29,272][105692] Updated weights for policy 0, policy_version 376697 (0.0008) [2023-12-26 18:06:29,326][105692] Updated weights for policy 0, policy_version 376707 (0.0008) [2023-12-26 18:06:29,795][105620] Updated weights for policy 1, policy_version 377388 (0.0008) [2023-12-26 18:06:29,858][105620] Updated weights for policy 1, policy_version 377398 (0.0008) [2023-12-26 18:06:29,923][105620] Updated weights for policy 1, policy_version 377408 (0.0008) [2023-12-26 18:06:30,096][105692] Updated weights for policy 0, policy_version 376717 (0.0009) [2023-12-26 18:06:30,153][105692] Updated weights for policy 0, policy_version 376727 (0.0010) [2023-12-26 18:06:30,201][105692] Updated weights for policy 0, policy_version 376737 (0.0010) [2023-12-26 18:06:30,690][105620] Updated weights for policy 1, policy_version 377418 (0.0007) [2023-12-26 18:06:30,750][105620] Updated weights for policy 1, policy_version 377428 (0.0007) [2023-12-26 18:06:30,803][105620] Updated weights for policy 1, policy_version 377438 (0.0009) [2023-12-26 18:06:30,838][105692] Updated weights for policy 0, policy_version 376747 (0.0009) [2023-12-26 18:06:30,854][105620] Updated weights for policy 1, policy_version 377448 (0.0009) [2023-12-26 18:06:30,886][105692] Updated weights for policy 0, policy_version 376757 (0.0007) [2023-12-26 18:06:30,935][105692] Updated weights for policy 0, policy_version 376767 (0.0006) [2023-12-26 18:06:31,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 193101824. Throughput: 0: 9627.3, 1: 9862.3. Samples: 193065808. Policy #0 lag: (min: 31.0, avg: 31.3, max: 45.0) [2023-12-26 18:06:31,063][104569] Avg episode reward: [(0, '7320.696'), (1, '3920.106')] [2023-12-26 18:06:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000376776_96468992.pth... [2023-12-26 18:06:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000377448_96632832.pth... [2023-12-26 18:06:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000375624_96174080.pth [2023-12-26 18:06:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000376296_96337920.pth [2023-12-26 18:06:31,534][105620] Updated weights for policy 1, policy_version 377458 (0.0005) [2023-12-26 18:06:31,589][105620] Updated weights for policy 1, policy_version 377468 (0.0005) [2023-12-26 18:06:31,617][105692] Updated weights for policy 0, policy_version 376777 (0.0005) [2023-12-26 18:06:31,647][105620] Updated weights for policy 1, policy_version 377478 (0.0007) [2023-12-26 18:06:31,678][105692] Updated weights for policy 0, policy_version 376787 (0.0008) [2023-12-26 18:06:31,741][105692] Updated weights for policy 0, policy_version 376797 (0.0008) [2023-12-26 18:06:31,793][105692] Updated weights for policy 0, policy_version 376807 (0.0008) [2023-12-26 18:06:32,368][105620] Updated weights for policy 1, policy_version 377488 (0.0009) [2023-12-26 18:06:32,427][105620] Updated weights for policy 1, policy_version 377498 (0.0008) [2023-12-26 18:06:32,494][105620] Updated weights for policy 1, policy_version 377509 (0.0007) [2023-12-26 18:06:32,561][105692] Updated weights for policy 0, policy_version 376817 (0.0009) [2023-12-26 18:06:32,614][105692] Updated weights for policy 0, policy_version 376827 (0.0009) [2023-12-26 18:06:32,667][105692] Updated weights for policy 0, policy_version 376838 (0.0009) [2023-12-26 18:06:33,083][105620] Updated weights for policy 1, policy_version 377519 (0.0006) [2023-12-26 18:06:33,137][105620] Updated weights for policy 1, policy_version 377529 (0.0007) [2023-12-26 18:06:33,193][105620] Updated weights for policy 1, policy_version 377539 (0.0009) [2023-12-26 18:06:33,348][105692] Updated weights for policy 0, policy_version 376848 (0.0006) [2023-12-26 18:06:33,393][105692] Updated weights for policy 0, policy_version 376858 (0.0005) [2023-12-26 18:06:33,460][105692] Updated weights for policy 0, policy_version 376868 (0.0005) [2023-12-26 18:06:33,957][105692] Updated weights for policy 0, policy_version 376878 (0.0005) [2023-12-26 18:06:33,971][105620] Updated weights for policy 1, policy_version 377549 (0.0010) [2023-12-26 18:06:34,002][105692] Updated weights for policy 0, policy_version 376888 (0.0005) [2023-12-26 18:06:34,022][105620] Updated weights for policy 1, policy_version 377559 (0.0010) [2023-12-26 18:06:34,052][105692] Updated weights for policy 0, policy_version 376898 (0.0005) [2023-12-26 18:06:34,072][105620] Updated weights for policy 1, policy_version 377569 (0.0008) [2023-12-26 18:06:34,723][105692] Updated weights for policy 0, policy_version 376908 (0.0007) [2023-12-26 18:06:34,770][105692] Updated weights for policy 0, policy_version 376918 (0.0008) [2023-12-26 18:06:34,817][105692] Updated weights for policy 0, policy_version 376928 (0.0009) [2023-12-26 18:06:34,838][105620] Updated weights for policy 1, policy_version 377579 (0.0009) [2023-12-26 18:06:34,886][105620] Updated weights for policy 1, policy_version 377589 (0.0008) [2023-12-26 18:06:34,942][105620] Updated weights for policy 1, policy_version 377601 (0.0010) [2023-12-26 18:06:35,585][105692] Updated weights for policy 0, policy_version 376938 (0.0009) [2023-12-26 18:06:35,635][105692] Updated weights for policy 0, policy_version 376948 (0.0008) [2023-12-26 18:06:35,692][105692] Updated weights for policy 0, policy_version 376958 (0.0009) [2023-12-26 18:06:35,752][105692] Updated weights for policy 0, policy_version 376968 (0.0008) [2023-12-26 18:06:35,754][105620] Updated weights for policy 1, policy_version 377612 (0.0009) [2023-12-26 18:06:35,808][105620] Updated weights for policy 1, policy_version 377622 (0.0009) [2023-12-26 18:06:35,855][105620] Updated weights for policy 1, policy_version 377632 (0.0009) [2023-12-26 18:06:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 193200128. Throughput: 0: 9614.9, 1: 10017.6. Samples: 193187532. Policy #0 lag: (min: 31.0, avg: 31.3, max: 45.0) [2023-12-26 18:06:36,063][104569] Avg episode reward: [(0, '4111.192'), (1, '2629.996')] [2023-12-26 18:06:36,527][105692] Updated weights for policy 0, policy_version 376978 (0.0009) [2023-12-26 18:06:36,575][105692] Updated weights for policy 0, policy_version 376988 (0.0009) [2023-12-26 18:06:36,622][105620] Updated weights for policy 1, policy_version 377642 (0.0009) [2023-12-26 18:06:36,626][105692] Updated weights for policy 0, policy_version 376998 (0.0009) [2023-12-26 18:06:36,677][105620] Updated weights for policy 1, policy_version 377652 (0.0009) [2023-12-26 18:06:36,726][105620] Updated weights for policy 1, policy_version 377662 (0.0009) [2023-12-26 18:06:36,776][105620] Updated weights for policy 1, policy_version 377672 (0.0009) [2023-12-26 18:06:37,398][105692] Updated weights for policy 0, policy_version 377008 (0.0009) [2023-12-26 18:06:37,462][105692] Updated weights for policy 0, policy_version 377018 (0.0009) [2023-12-26 18:06:37,501][105620] Updated weights for policy 1, policy_version 377682 (0.0008) [2023-12-26 18:06:37,520][105692] Updated weights for policy 0, policy_version 377028 (0.0006) [2023-12-26 18:06:37,562][105620] Updated weights for policy 1, policy_version 377692 (0.0007) [2023-12-26 18:06:37,617][105620] Updated weights for policy 1, policy_version 377702 (0.0006) [2023-12-26 18:06:38,227][105620] Updated weights for policy 1, policy_version 377712 (0.0009) [2023-12-26 18:06:38,282][105620] Updated weights for policy 1, policy_version 377722 (0.0009) [2023-12-26 18:06:38,337][105620] Updated weights for policy 1, policy_version 377732 (0.0008) [2023-12-26 18:06:38,349][105692] Updated weights for policy 0, policy_version 377038 (0.0007) [2023-12-26 18:06:38,409][105692] Updated weights for policy 0, policy_version 377048 (0.0009) [2023-12-26 18:06:38,474][105692] Updated weights for policy 0, policy_version 377058 (0.0009) [2023-12-26 18:06:39,077][105620] Updated weights for policy 1, policy_version 377742 (0.0009) [2023-12-26 18:06:39,138][105620] Updated weights for policy 1, policy_version 377752 (0.0009) [2023-12-26 18:06:39,194][105620] Updated weights for policy 1, policy_version 377762 (0.0008) [2023-12-26 18:06:39,212][105692] Updated weights for policy 0, policy_version 377068 (0.0009) [2023-12-26 18:06:39,275][105692] Updated weights for policy 0, policy_version 377078 (0.0009) [2023-12-26 18:06:39,327][105692] Updated weights for policy 0, policy_version 377088 (0.0009) [2023-12-26 18:06:39,954][105620] Updated weights for policy 1, policy_version 377772 (0.0009) [2023-12-26 18:06:40,010][105620] Updated weights for policy 1, policy_version 377782 (0.0009) [2023-12-26 18:06:40,077][105620] Updated weights for policy 1, policy_version 377792 (0.0009) [2023-12-26 18:06:40,081][105692] Updated weights for policy 0, policy_version 377098 (0.0008) [2023-12-26 18:06:40,136][105692] Updated weights for policy 0, policy_version 377108 (0.0008) [2023-12-26 18:06:40,193][105692] Updated weights for policy 0, policy_version 377118 (0.0009) [2023-12-26 18:06:40,241][105692] Updated weights for policy 0, policy_version 377128 (0.0009) [2023-12-26 18:06:40,855][105620] Updated weights for policy 1, policy_version 377802 (0.0008) [2023-12-26 18:06:40,910][105620] Updated weights for policy 1, policy_version 377812 (0.0009) [2023-12-26 18:06:40,967][105620] Updated weights for policy 1, policy_version 377822 (0.0009) [2023-12-26 18:06:40,989][105692] Updated weights for policy 0, policy_version 377138 (0.0006) [2023-12-26 18:06:41,026][105620] Updated weights for policy 1, policy_version 377832 (0.0008) [2023-12-26 18:06:41,043][105692] Updated weights for policy 0, policy_version 377148 (0.0006) [2023-12-26 18:06:41,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.6, 300 sec: 19521.9). Total num frames: 193290240. Throughput: 0: 9656.3, 1: 9865.8. Samples: 193299376. Policy #0 lag: (min: 31.0, avg: 31.3, max: 45.0) [2023-12-26 18:06:41,063][104569] Avg episode reward: [(0, '6939.007'), (1, '2905.041')] [2023-12-26 18:06:41,106][105692] Updated weights for policy 0, policy_version 377158 (0.0009) [2023-12-26 18:06:41,817][105620] Updated weights for policy 1, policy_version 377842 (0.0009) [2023-12-26 18:06:41,877][105620] Updated weights for policy 1, policy_version 377852 (0.0007) [2023-12-26 18:06:41,888][105692] Updated weights for policy 0, policy_version 377168 (0.0009) [2023-12-26 18:06:41,935][105620] Updated weights for policy 1, policy_version 377862 (0.0007) [2023-12-26 18:06:41,946][105692] Updated weights for policy 0, policy_version 377178 (0.0005) [2023-12-26 18:06:42,002][105692] Updated weights for policy 0, policy_version 377188 (0.0008) [2023-12-26 18:06:42,680][105692] Updated weights for policy 0, policy_version 377198 (0.0008) [2023-12-26 18:06:42,735][105692] Updated weights for policy 0, policy_version 377208 (0.0008) [2023-12-26 18:06:42,750][105620] Updated weights for policy 1, policy_version 377872 (0.0008) [2023-12-26 18:06:42,796][105692] Updated weights for policy 0, policy_version 377218 (0.0007) [2023-12-26 18:06:42,807][105620] Updated weights for policy 1, policy_version 377882 (0.0008) [2023-12-26 18:06:42,864][105620] Updated weights for policy 1, policy_version 377892 (0.0007) [2023-12-26 18:06:43,383][105692] Updated weights for policy 0, policy_version 377228 (0.0008) [2023-12-26 18:06:43,441][105692] Updated weights for policy 0, policy_version 377238 (0.0009) [2023-12-26 18:06:43,506][105692] Updated weights for policy 0, policy_version 377248 (0.0009) [2023-12-26 18:06:43,707][105620] Updated weights for policy 1, policy_version 377902 (0.0009) [2023-12-26 18:06:43,755][105620] Updated weights for policy 1, policy_version 377912 (0.0009) [2023-12-26 18:06:43,808][105620] Updated weights for policy 1, policy_version 377922 (0.0009) [2023-12-26 18:06:44,245][105692] Updated weights for policy 0, policy_version 377258 (0.0009) [2023-12-26 18:06:44,306][105692] Updated weights for policy 0, policy_version 377268 (0.0009) [2023-12-26 18:06:44,357][105692] Updated weights for policy 0, policy_version 377278 (0.0008) [2023-12-26 18:06:44,409][105692] Updated weights for policy 0, policy_version 377288 (0.0009) [2023-12-26 18:06:44,596][105620] Updated weights for policy 1, policy_version 377932 (0.0009) [2023-12-26 18:06:44,654][105620] Updated weights for policy 1, policy_version 377942 (0.0009) [2023-12-26 18:06:44,702][105620] Updated weights for policy 1, policy_version 377952 (0.0006) [2023-12-26 18:06:45,205][105692] Updated weights for policy 0, policy_version 377298 (0.0008) [2023-12-26 18:06:45,262][105692] Updated weights for policy 0, policy_version 377308 (0.0010) [2023-12-26 18:06:45,319][105692] Updated weights for policy 0, policy_version 377318 (0.0009) [2023-12-26 18:06:45,420][105620] Updated weights for policy 1, policy_version 377962 (0.0006) [2023-12-26 18:06:45,481][105620] Updated weights for policy 1, policy_version 377972 (0.0009) [2023-12-26 18:06:45,544][105620] Updated weights for policy 1, policy_version 377982 (0.0009) [2023-12-26 18:06:45,605][105620] Updated weights for policy 1, policy_version 377992 (0.0009) [2023-12-26 18:06:46,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 193380352. Throughput: 0: 9751.0, 1: 9779.4. Samples: 193355796. Policy #0 lag: (min: 31.0, avg: 31.3, max: 45.0) [2023-12-26 18:06:46,063][104569] Avg episode reward: [(0, '9354.276'), (1, '904.048')] [2023-12-26 18:06:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000377992_96772096.pth... [2023-12-26 18:06:46,068][105692] Updated weights for policy 0, policy_version 377328 (0.0008) [2023-12-26 18:06:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000376872_96485376.pth [2023-12-26 18:06:46,119][105692] Updated weights for policy 0, policy_version 377338 (0.0009) [2023-12-26 18:06:46,179][105692] Updated weights for policy 0, policy_version 377348 (0.0009) [2023-12-26 18:06:46,203][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000377352_96616448.pth... [2023-12-26 18:06:46,207][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000376200_96321536.pth [2023-12-26 18:06:46,375][105620] Updated weights for policy 1, policy_version 378002 (0.0009) [2023-12-26 18:06:46,424][105620] Updated weights for policy 1, policy_version 378012 (0.0009) [2023-12-26 18:06:46,472][105620] Updated weights for policy 1, policy_version 378022 (0.0009) [2023-12-26 18:06:46,921][105692] Updated weights for policy 0, policy_version 377358 (0.0010) [2023-12-26 18:06:46,978][105692] Updated weights for policy 0, policy_version 377368 (0.0009) [2023-12-26 18:06:47,028][105692] Updated weights for policy 0, policy_version 377378 (0.0007) [2023-12-26 18:06:47,265][105620] Updated weights for policy 1, policy_version 378032 (0.0009) [2023-12-26 18:06:47,313][105620] Updated weights for policy 1, policy_version 378042 (0.0009) [2023-12-26 18:06:47,360][105620] Updated weights for policy 1, policy_version 378052 (0.0009) [2023-12-26 18:06:47,756][105692] Updated weights for policy 0, policy_version 377388 (0.0007) [2023-12-26 18:06:47,816][105692] Updated weights for policy 0, policy_version 377398 (0.0009) [2023-12-26 18:06:47,873][105692] Updated weights for policy 0, policy_version 377408 (0.0009) [2023-12-26 18:06:48,139][105620] Updated weights for policy 1, policy_version 378062 (0.0009) [2023-12-26 18:06:48,186][105620] Updated weights for policy 1, policy_version 378072 (0.0008) [2023-12-26 18:06:48,232][105620] Updated weights for policy 1, policy_version 378082 (0.0008) [2023-12-26 18:06:48,619][105692] Updated weights for policy 0, policy_version 377418 (0.0009) [2023-12-26 18:06:48,675][105692] Updated weights for policy 0, policy_version 377428 (0.0009) [2023-12-26 18:06:48,722][105692] Updated weights for policy 0, policy_version 377438 (0.0009) [2023-12-26 18:06:48,778][105692] Updated weights for policy 0, policy_version 377448 (0.0008) [2023-12-26 18:06:49,011][105620] Updated weights for policy 1, policy_version 378092 (0.0009) [2023-12-26 18:06:49,061][105620] Updated weights for policy 1, policy_version 378102 (0.0009) [2023-12-26 18:06:49,108][105620] Updated weights for policy 1, policy_version 378112 (0.0009) [2023-12-26 18:06:49,552][105692] Updated weights for policy 0, policy_version 377458 (0.0009) [2023-12-26 18:06:49,610][105692] Updated weights for policy 0, policy_version 377468 (0.0009) [2023-12-26 18:06:49,671][105692] Updated weights for policy 0, policy_version 377478 (0.0009) [2023-12-26 18:06:49,874][105620] Updated weights for policy 1, policy_version 378122 (0.0009) [2023-12-26 18:06:49,933][105620] Updated weights for policy 1, policy_version 378132 (0.0007) [2023-12-26 18:06:49,989][105620] Updated weights for policy 1, policy_version 378142 (0.0008) [2023-12-26 18:06:50,050][105620] Updated weights for policy 1, policy_version 378152 (0.0006) [2023-12-26 18:06:50,453][105692] Updated weights for policy 0, policy_version 377488 (0.0009) [2023-12-26 18:06:50,519][105692] Updated weights for policy 0, policy_version 377498 (0.0009) [2023-12-26 18:06:50,588][105692] Updated weights for policy 0, policy_version 377508 (0.0008) [2023-12-26 18:06:50,716][105620] Updated weights for policy 1, policy_version 378162 (0.0008) [2023-12-26 18:06:50,774][105620] Updated weights for policy 1, policy_version 378172 (0.0009) [2023-12-26 18:06:50,822][105620] Updated weights for policy 1, policy_version 378182 (0.0008) [2023-12-26 18:06:51,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 193478656. Throughput: 0: 9631.2, 1: 9737.8. Samples: 193467928. Policy #0 lag: (min: 31.0, avg: 31.3, max: 45.0) [2023-12-26 18:06:51,063][104569] Avg episode reward: [(0, '9271.055'), (1, '862.812')] [2023-12-26 18:06:51,347][105692] Updated weights for policy 0, policy_version 377518 (0.0009) [2023-12-26 18:06:51,408][105692] Updated weights for policy 0, policy_version 377528 (0.0009) [2023-12-26 18:06:51,462][105692] Updated weights for policy 0, policy_version 377538 (0.0009) [2023-12-26 18:06:51,501][105620] Updated weights for policy 1, policy_version 378192 (0.0007) [2023-12-26 18:06:51,559][105620] Updated weights for policy 1, policy_version 378202 (0.0009) [2023-12-26 18:06:51,626][105620] Updated weights for policy 1, policy_version 378212 (0.0009) [2023-12-26 18:06:52,225][105692] Updated weights for policy 0, policy_version 377548 (0.0008) [2023-12-26 18:06:52,294][105692] Updated weights for policy 0, policy_version 377558 (0.0009) [2023-12-26 18:06:52,363][105692] Updated weights for policy 0, policy_version 377568 (0.0008) [2023-12-26 18:06:52,376][105620] Updated weights for policy 1, policy_version 378222 (0.0009) [2023-12-26 18:06:52,434][105620] Updated weights for policy 1, policy_version 378232 (0.0007) [2023-12-26 18:06:52,486][105620] Updated weights for policy 1, policy_version 378242 (0.0009) [2023-12-26 18:06:53,034][105692] Updated weights for policy 0, policy_version 377578 (0.0009) [2023-12-26 18:06:53,090][105692] Updated weights for policy 0, policy_version 377588 (0.0010) [2023-12-26 18:06:53,142][105692] Updated weights for policy 0, policy_version 377598 (0.0009) [2023-12-26 18:06:53,197][105692] Updated weights for policy 0, policy_version 377608 (0.0008) [2023-12-26 18:06:53,220][105620] Updated weights for policy 1, policy_version 378252 (0.0008) [2023-12-26 18:06:53,282][105620] Updated weights for policy 1, policy_version 378262 (0.0007) [2023-12-26 18:06:53,346][105620] Updated weights for policy 1, policy_version 378272 (0.0006) [2023-12-26 18:06:53,947][105620] Updated weights for policy 1, policy_version 378282 (0.0007) [2023-12-26 18:06:54,012][105620] Updated weights for policy 1, policy_version 378292 (0.0010) [2023-12-26 18:06:54,050][105692] Updated weights for policy 0, policy_version 377618 (0.0007) [2023-12-26 18:06:54,074][105620] Updated weights for policy 1, policy_version 378302 (0.0011) [2023-12-26 18:06:54,107][105692] Updated weights for policy 0, policy_version 377628 (0.0008) [2023-12-26 18:06:54,122][105620] Updated weights for policy 1, policy_version 378312 (0.0007) [2023-12-26 18:06:54,167][105692] Updated weights for policy 0, policy_version 377638 (0.0009) [2023-12-26 18:06:54,863][105620] Updated weights for policy 1, policy_version 378322 (0.0009) [2023-12-26 18:06:54,917][105620] Updated weights for policy 1, policy_version 378332 (0.0008) [2023-12-26 18:06:54,919][105692] Updated weights for policy 0, policy_version 377648 (0.0007) [2023-12-26 18:06:54,975][105620] Updated weights for policy 1, policy_version 378342 (0.0007) [2023-12-26 18:06:54,982][105692] Updated weights for policy 0, policy_version 377658 (0.0007) [2023-12-26 18:06:55,044][105692] Updated weights for policy 0, policy_version 377668 (0.0006) [2023-12-26 18:06:55,599][105620] Updated weights for policy 1, policy_version 378352 (0.0006) [2023-12-26 18:06:55,657][105620] Updated weights for policy 1, policy_version 378362 (0.0009) [2023-12-26 18:06:55,714][105620] Updated weights for policy 1, policy_version 378372 (0.0010) [2023-12-26 18:06:55,786][105692] Updated weights for policy 0, policy_version 377678 (0.0006) [2023-12-26 18:06:55,845][105692] Updated weights for policy 0, policy_version 377688 (0.0009) [2023-12-26 18:06:55,903][105692] Updated weights for policy 0, policy_version 377698 (0.0009) [2023-12-26 18:06:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 193576960. Throughput: 0: 9534.3, 1: 9734.6. Samples: 193582432. Policy #0 lag: (min: 31.0, avg: 31.3, max: 45.0) [2023-12-26 18:06:56,063][104569] Avg episode reward: [(0, '9114.657'), (1, '413.069')] [2023-12-26 18:06:56,366][105620] Updated weights for policy 1, policy_version 378382 (0.0009) [2023-12-26 18:06:56,415][105620] Updated weights for policy 1, policy_version 378393 (0.0006) [2023-12-26 18:06:56,462][105620] Updated weights for policy 1, policy_version 378403 (0.0005) [2023-12-26 18:06:56,745][105692] Updated weights for policy 0, policy_version 377708 (0.0007) [2023-12-26 18:06:56,788][105692] Updated weights for policy 0, policy_version 377718 (0.0005) [2023-12-26 18:06:56,832][105692] Updated weights for policy 0, policy_version 377728 (0.0005) [2023-12-26 18:06:57,091][105620] Updated weights for policy 1, policy_version 378413 (0.0005) [2023-12-26 18:06:57,146][105620] Updated weights for policy 1, policy_version 378423 (0.0005) [2023-12-26 18:06:57,209][105620] Updated weights for policy 1, policy_version 378433 (0.0005) [2023-12-26 18:06:57,459][105692] Updated weights for policy 0, policy_version 377738 (0.0006) [2023-12-26 18:06:57,512][105692] Updated weights for policy 0, policy_version 377748 (0.0005) [2023-12-26 18:06:57,560][105692] Updated weights for policy 0, policy_version 377758 (0.0005) [2023-12-26 18:06:57,606][105692] Updated weights for policy 0, policy_version 377768 (0.0005) [2023-12-26 18:06:57,847][105620] Updated weights for policy 1, policy_version 378443 (0.0007) [2023-12-26 18:06:57,912][105620] Updated weights for policy 1, policy_version 378453 (0.0010) [2023-12-26 18:06:57,967][105620] Updated weights for policy 1, policy_version 378463 (0.0005) [2023-12-26 18:06:58,162][105692] Updated weights for policy 0, policy_version 377778 (0.0008) [2023-12-26 18:06:58,221][105692] Updated weights for policy 0, policy_version 377788 (0.0008) [2023-12-26 18:06:58,282][105692] Updated weights for policy 0, policy_version 377798 (0.0008) [2023-12-26 18:06:58,689][105620] Updated weights for policy 1, policy_version 378473 (0.0005) [2023-12-26 18:06:58,761][105620] Updated weights for policy 1, policy_version 378483 (0.0009) [2023-12-26 18:06:58,831][105620] Updated weights for policy 1, policy_version 378493 (0.0009) [2023-12-26 18:06:58,904][105620] Updated weights for policy 1, policy_version 378503 (0.0008) [2023-12-26 18:06:59,116][105692] Updated weights for policy 0, policy_version 377808 (0.0007) [2023-12-26 18:06:59,169][105692] Updated weights for policy 0, policy_version 377818 (0.0008) [2023-12-26 18:06:59,234][105692] Updated weights for policy 0, policy_version 377828 (0.0007) [2023-12-26 18:06:59,664][105620] Updated weights for policy 1, policy_version 378513 (0.0005) [2023-12-26 18:06:59,725][105620] Updated weights for policy 1, policy_version 378523 (0.0005) [2023-12-26 18:06:59,795][105620] Updated weights for policy 1, policy_version 378533 (0.0006) [2023-12-26 18:06:59,961][105692] Updated weights for policy 0, policy_version 377838 (0.0008) [2023-12-26 18:07:00,027][105692] Updated weights for policy 0, policy_version 377848 (0.0006) [2023-12-26 18:07:00,092][105692] Updated weights for policy 0, policy_version 377858 (0.0005) [2023-12-26 18:07:00,401][105620] Updated weights for policy 1, policy_version 378543 (0.0008) [2023-12-26 18:07:00,455][105620] Updated weights for policy 1, policy_version 378554 (0.0010) [2023-12-26 18:07:00,507][105620] Updated weights for policy 1, policy_version 378565 (0.0010) [2023-12-26 18:07:00,625][105692] Updated weights for policy 0, policy_version 377868 (0.0006) [2023-12-26 18:07:00,682][105692] Updated weights for policy 0, policy_version 377878 (0.0009) [2023-12-26 18:07:00,746][105692] Updated weights for policy 0, policy_version 377888 (0.0009) [2023-12-26 18:07:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 193675264. Throughput: 0: 9558.7, 1: 9783.9. Samples: 193644316. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 18:07:01,063][104569] Avg episode reward: [(0, '9123.088'), (1, '364.264')] [2023-12-26 18:07:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000377896_96755712.pth... [2023-12-26 18:07:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000378568_96919552.pth... [2023-12-26 18:07:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000376776_96468992.pth [2023-12-26 18:07:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000377448_96632832.pth [2023-12-26 18:07:01,208][105620] Updated weights for policy 1, policy_version 378575 (0.0009) [2023-12-26 18:07:01,267][105620] Updated weights for policy 1, policy_version 378585 (0.0008) [2023-12-26 18:07:01,331][105620] Updated weights for policy 1, policy_version 378595 (0.0007) [2023-12-26 18:07:01,501][105692] Updated weights for policy 0, policy_version 377898 (0.0010) [2023-12-26 18:07:01,562][105692] Updated weights for policy 0, policy_version 377908 (0.0005) [2023-12-26 18:07:01,623][105692] Updated weights for policy 0, policy_version 377918 (0.0006) [2023-12-26 18:07:01,683][105692] Updated weights for policy 0, policy_version 377928 (0.0006) [2023-12-26 18:07:02,079][105620] Updated weights for policy 1, policy_version 378605 (0.0006) [2023-12-26 18:07:02,151][105620] Updated weights for policy 1, policy_version 378615 (0.0005) [2023-12-26 18:07:02,219][105620] Updated weights for policy 1, policy_version 378625 (0.0008) [2023-12-26 18:07:02,298][105692] Updated weights for policy 0, policy_version 377938 (0.0006) [2023-12-26 18:07:02,359][105692] Updated weights for policy 0, policy_version 377948 (0.0008) [2023-12-26 18:07:02,416][105692] Updated weights for policy 0, policy_version 377958 (0.0009) [2023-12-26 18:07:02,812][105620] Updated weights for policy 1, policy_version 378635 (0.0009) [2023-12-26 18:07:02,858][105620] Updated weights for policy 1, policy_version 378645 (0.0008) [2023-12-26 18:07:02,915][105620] Updated weights for policy 1, policy_version 378655 (0.0008) [2023-12-26 18:07:03,171][105692] Updated weights for policy 0, policy_version 377968 (0.0006) [2023-12-26 18:07:03,217][105692] Updated weights for policy 0, policy_version 377978 (0.0005) [2023-12-26 18:07:03,261][105692] Updated weights for policy 0, policy_version 377988 (0.0005) [2023-12-26 18:07:03,652][105620] Updated weights for policy 1, policy_version 378665 (0.0008) [2023-12-26 18:07:03,712][105620] Updated weights for policy 1, policy_version 378675 (0.0005) [2023-12-26 18:07:03,771][105620] Updated weights for policy 1, policy_version 378685 (0.0005) [2023-12-26 18:07:03,824][105620] Updated weights for policy 1, policy_version 378695 (0.0007) [2023-12-26 18:07:03,974][105692] Updated weights for policy 0, policy_version 377998 (0.0007) [2023-12-26 18:07:04,028][105692] Updated weights for policy 0, policy_version 378008 (0.0008) [2023-12-26 18:07:04,078][105692] Updated weights for policy 0, policy_version 378018 (0.0008) [2023-12-26 18:07:04,517][105620] Updated weights for policy 1, policy_version 378705 (0.0010) [2023-12-26 18:07:04,584][105620] Updated weights for policy 1, policy_version 378715 (0.0009) [2023-12-26 18:07:04,637][105620] Updated weights for policy 1, policy_version 378725 (0.0010) [2023-12-26 18:07:04,758][105692] Updated weights for policy 0, policy_version 378028 (0.0008) [2023-12-26 18:07:04,823][105692] Updated weights for policy 0, policy_version 378038 (0.0009) [2023-12-26 18:07:04,878][105692] Updated weights for policy 0, policy_version 378048 (0.0008) [2023-12-26 18:07:05,363][105620] Updated weights for policy 1, policy_version 378735 (0.0007) [2023-12-26 18:07:05,426][105620] Updated weights for policy 1, policy_version 378745 (0.0008) [2023-12-26 18:07:05,480][105620] Updated weights for policy 1, policy_version 378755 (0.0008) [2023-12-26 18:07:05,618][105692] Updated weights for policy 0, policy_version 378058 (0.0006) [2023-12-26 18:07:05,690][105692] Updated weights for policy 0, policy_version 378068 (0.0006) [2023-12-26 18:07:05,749][105692] Updated weights for policy 0, policy_version 378078 (0.0007) [2023-12-26 18:07:05,800][105692] Updated weights for policy 0, policy_version 378088 (0.0005) [2023-12-26 18:07:06,062][104569] Fps is (10 sec: 19659.9, 60 sec: 19524.1, 300 sec: 19521.9). Total num frames: 193773568. Throughput: 0: 9558.6, 1: 9790.6. Samples: 193763616. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 18:07:06,064][104569] Avg episode reward: [(0, '9189.947'), (1, '395.561')] [2023-12-26 18:07:06,251][105620] Updated weights for policy 1, policy_version 378765 (0.0009) [2023-12-26 18:07:06,302][105620] Updated weights for policy 1, policy_version 378775 (0.0009) [2023-12-26 18:07:06,349][105620] Updated weights for policy 1, policy_version 378785 (0.0008) [2023-12-26 18:07:06,447][105692] Updated weights for policy 0, policy_version 378098 (0.0009) [2023-12-26 18:07:06,493][105692] Updated weights for policy 0, policy_version 378108 (0.0008) [2023-12-26 18:07:06,542][105692] Updated weights for policy 0, policy_version 378118 (0.0009) [2023-12-26 18:07:07,140][105620] Updated weights for policy 1, policy_version 378795 (0.0008) [2023-12-26 18:07:07,194][105620] Updated weights for policy 1, policy_version 378805 (0.0005) [2023-12-26 18:07:07,246][105692] Updated weights for policy 0, policy_version 378128 (0.0009) [2023-12-26 18:07:07,254][105620] Updated weights for policy 1, policy_version 378815 (0.0006) [2023-12-26 18:07:07,302][105692] Updated weights for policy 0, policy_version 378138 (0.0008) [2023-12-26 18:07:07,352][105692] Updated weights for policy 0, policy_version 378148 (0.0008) [2023-12-26 18:07:07,941][105620] Updated weights for policy 1, policy_version 378825 (0.0007) [2023-12-26 18:07:08,009][105620] Updated weights for policy 1, policy_version 378835 (0.0009) [2023-12-26 18:07:08,056][105620] Updated weights for policy 1, policy_version 378845 (0.0008) [2023-12-26 18:07:08,112][105620] Updated weights for policy 1, policy_version 378855 (0.0008) [2023-12-26 18:07:08,127][105692] Updated weights for policy 0, policy_version 378158 (0.0008) [2023-12-26 18:07:08,185][105692] Updated weights for policy 0, policy_version 378168 (0.0005) [2023-12-26 18:07:08,231][105692] Updated weights for policy 0, policy_version 378178 (0.0005) [2023-12-26 18:07:08,871][105620] Updated weights for policy 1, policy_version 378865 (0.0009) [2023-12-26 18:07:08,913][105692] Updated weights for policy 0, policy_version 378188 (0.0007) [2023-12-26 18:07:08,923][105620] Updated weights for policy 1, policy_version 378875 (0.0007) [2023-12-26 18:07:08,973][105620] Updated weights for policy 1, policy_version 378885 (0.0006) [2023-12-26 18:07:08,975][105692] Updated weights for policy 0, policy_version 378198 (0.0008) [2023-12-26 18:07:09,035][105692] Updated weights for policy 0, policy_version 378208 (0.0009) [2023-12-26 18:07:09,690][105620] Updated weights for policy 1, policy_version 378895 (0.0007) [2023-12-26 18:07:09,760][105620] Updated weights for policy 1, policy_version 378905 (0.0007) [2023-12-26 18:07:09,787][105692] Updated weights for policy 0, policy_version 378218 (0.0009) [2023-12-26 18:07:09,821][105620] Updated weights for policy 1, policy_version 378915 (0.0008) [2023-12-26 18:07:09,862][105692] Updated weights for policy 0, policy_version 378228 (0.0006) [2023-12-26 18:07:09,926][105692] Updated weights for policy 0, policy_version 378238 (0.0009) [2023-12-26 18:07:09,992][105692] Updated weights for policy 0, policy_version 378248 (0.0009) [2023-12-26 18:07:10,560][105620] Updated weights for policy 1, policy_version 378925 (0.0009) [2023-12-26 18:07:10,620][105620] Updated weights for policy 1, policy_version 378935 (0.0009) [2023-12-26 18:07:10,680][105620] Updated weights for policy 1, policy_version 378945 (0.0008) [2023-12-26 18:07:10,725][105692] Updated weights for policy 0, policy_version 378258 (0.0007) [2023-12-26 18:07:10,780][105692] Updated weights for policy 0, policy_version 378268 (0.0009) [2023-12-26 18:07:10,836][105692] Updated weights for policy 0, policy_version 378278 (0.0006) [2023-12-26 18:07:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 193871872. Throughput: 0: 9611.9, 1: 9744.8. Samples: 193878180. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 18:07:11,063][104569] Avg episode reward: [(0, '9188.995'), (1, '2990.626')] [2023-12-26 18:07:11,415][105620] Updated weights for policy 1, policy_version 378955 (0.0008) [2023-12-26 18:07:11,474][105620] Updated weights for policy 1, policy_version 378965 (0.0008) [2023-12-26 18:07:11,529][105620] Updated weights for policy 1, policy_version 378975 (0.0009) [2023-12-26 18:07:11,688][105692] Updated weights for policy 0, policy_version 378288 (0.0009) [2023-12-26 18:07:11,757][105692] Updated weights for policy 0, policy_version 378298 (0.0010) [2023-12-26 18:07:11,819][105692] Updated weights for policy 0, policy_version 378308 (0.0009) [2023-12-26 18:07:12,257][105620] Updated weights for policy 1, policy_version 378985 (0.0009) [2023-12-26 18:07:12,322][105620] Updated weights for policy 1, policy_version 378995 (0.0008) [2023-12-26 18:07:12,395][105620] Updated weights for policy 1, policy_version 379005 (0.0008) [2023-12-26 18:07:12,457][105620] Updated weights for policy 1, policy_version 379015 (0.0008) [2023-12-26 18:07:12,636][105692] Updated weights for policy 0, policy_version 378318 (0.0009) [2023-12-26 18:07:12,685][105692] Updated weights for policy 0, policy_version 378328 (0.0009) [2023-12-26 18:07:12,740][105692] Updated weights for policy 0, policy_version 378338 (0.0009) [2023-12-26 18:07:13,122][105620] Updated weights for policy 1, policy_version 379025 (0.0008) [2023-12-26 18:07:13,181][105620] Updated weights for policy 1, policy_version 379035 (0.0009) [2023-12-26 18:07:13,247][105620] Updated weights for policy 1, policy_version 379045 (0.0009) [2023-12-26 18:07:13,481][105692] Updated weights for policy 0, policy_version 378348 (0.0009) [2023-12-26 18:07:13,536][105692] Updated weights for policy 0, policy_version 378358 (0.0009) [2023-12-26 18:07:13,589][105692] Updated weights for policy 0, policy_version 378369 (0.0008) [2023-12-26 18:07:14,032][105620] Updated weights for policy 1, policy_version 379055 (0.0010) [2023-12-26 18:07:14,084][105620] Updated weights for policy 1, policy_version 379067 (0.0009) [2023-12-26 18:07:14,138][105620] Updated weights for policy 1, policy_version 379077 (0.0009) [2023-12-26 18:07:14,278][105692] Updated weights for policy 0, policy_version 378379 (0.0008) [2023-12-26 18:07:14,326][105692] Updated weights for policy 0, policy_version 378389 (0.0009) [2023-12-26 18:07:14,371][105692] Updated weights for policy 0, policy_version 378399 (0.0006) [2023-12-26 18:07:14,920][105620] Updated weights for policy 1, policy_version 379087 (0.0007) [2023-12-26 18:07:14,984][105620] Updated weights for policy 1, policy_version 379097 (0.0006) [2023-12-26 18:07:15,055][105620] Updated weights for policy 1, policy_version 379107 (0.0009) [2023-12-26 18:07:15,130][105692] Updated weights for policy 0, policy_version 378409 (0.0006) [2023-12-26 18:07:15,188][105692] Updated weights for policy 0, policy_version 378419 (0.0008) [2023-12-26 18:07:15,241][105692] Updated weights for policy 0, policy_version 378429 (0.0009) [2023-12-26 18:07:15,300][105692] Updated weights for policy 0, policy_version 378439 (0.0009) [2023-12-26 18:07:15,736][105620] Updated weights for policy 1, policy_version 379117 (0.0009) [2023-12-26 18:07:15,800][105620] Updated weights for policy 1, policy_version 379127 (0.0008) [2023-12-26 18:07:15,865][105620] Updated weights for policy 1, policy_version 379137 (0.0009) [2023-12-26 18:07:16,062][104569] Fps is (10 sec: 18842.4, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 193961984. Throughput: 0: 9615.7, 1: 9666.0. Samples: 193933480. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 18:07:16,063][104569] Avg episode reward: [(0, '9267.030'), (1, '3264.748')] [2023-12-26 18:07:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000379144_97067008.pth... [2023-12-26 18:07:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000377992_96772096.pth [2023-12-26 18:07:16,119][105692] Updated weights for policy 0, policy_version 378449 (0.0009) [2023-12-26 18:07:16,179][105692] Updated weights for policy 0, policy_version 378459 (0.0009) [2023-12-26 18:07:16,242][105692] Updated weights for policy 0, policy_version 378469 (0.0009) [2023-12-26 18:07:16,255][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000378472_96903168.pth... [2023-12-26 18:07:16,258][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000377352_96616448.pth [2023-12-26 18:07:16,632][105620] Updated weights for policy 1, policy_version 379147 (0.0009) [2023-12-26 18:07:16,685][105620] Updated weights for policy 1, policy_version 379157 (0.0010) [2023-12-26 18:07:16,738][105620] Updated weights for policy 1, policy_version 379167 (0.0010) [2023-12-26 18:07:16,895][105692] Updated weights for policy 0, policy_version 378479 (0.0007) [2023-12-26 18:07:16,962][105692] Updated weights for policy 0, policy_version 378489 (0.0005) [2023-12-26 18:07:17,023][105692] Updated weights for policy 0, policy_version 378499 (0.0005) [2023-12-26 18:07:17,505][105692] Updated weights for policy 0, policy_version 378509 (0.0005) [2023-12-26 18:07:17,551][105692] Updated weights for policy 0, policy_version 378519 (0.0005) [2023-12-26 18:07:17,599][105692] Updated weights for policy 0, policy_version 378529 (0.0005) [2023-12-26 18:07:17,664][105620] Updated weights for policy 1, policy_version 379177 (0.0011) [2023-12-26 18:07:17,723][105620] Updated weights for policy 1, policy_version 379187 (0.0010) [2023-12-26 18:07:17,781][105620] Updated weights for policy 1, policy_version 379197 (0.0009) [2023-12-26 18:07:17,834][105620] Updated weights for policy 1, policy_version 379207 (0.0010) [2023-12-26 18:07:18,222][105692] Updated weights for policy 0, policy_version 378539 (0.0007) [2023-12-26 18:07:18,283][105692] Updated weights for policy 0, policy_version 378549 (0.0010) [2023-12-26 18:07:18,344][105692] Updated weights for policy 0, policy_version 378559 (0.0010) [2023-12-26 18:07:18,540][105620] Updated weights for policy 1, policy_version 379217 (0.0008) [2023-12-26 18:07:18,595][105620] Updated weights for policy 1, policy_version 379227 (0.0008) [2023-12-26 18:07:18,647][105620] Updated weights for policy 1, policy_version 379237 (0.0008) [2023-12-26 18:07:19,026][105692] Updated weights for policy 0, policy_version 378569 (0.0006) [2023-12-26 18:07:19,075][105692] Updated weights for policy 0, policy_version 378579 (0.0009) [2023-12-26 18:07:19,125][105692] Updated weights for policy 0, policy_version 378590 (0.0009) [2023-12-26 18:07:19,171][105692] Updated weights for policy 0, policy_version 378600 (0.0005) [2023-12-26 18:07:19,451][105620] Updated weights for policy 1, policy_version 379247 (0.0008) [2023-12-26 18:07:19,514][105620] Updated weights for policy 1, policy_version 379257 (0.0009) [2023-12-26 18:07:19,572][105620] Updated weights for policy 1, policy_version 379267 (0.0008) [2023-12-26 18:07:19,948][105692] Updated weights for policy 0, policy_version 378610 (0.0010) [2023-12-26 18:07:20,014][105692] Updated weights for policy 0, policy_version 378620 (0.0008) [2023-12-26 18:07:20,080][105692] Updated weights for policy 0, policy_version 378630 (0.0007) [2023-12-26 18:07:20,328][105620] Updated weights for policy 1, policy_version 379277 (0.0009) [2023-12-26 18:07:20,383][105620] Updated weights for policy 1, policy_version 379287 (0.0009) [2023-12-26 18:07:20,433][105620] Updated weights for policy 1, policy_version 379297 (0.0009) [2023-12-26 18:07:20,721][105692] Updated weights for policy 0, policy_version 378640 (0.0006) [2023-12-26 18:07:20,779][105692] Updated weights for policy 0, policy_version 378650 (0.0006) [2023-12-26 18:07:20,850][105692] Updated weights for policy 0, policy_version 378660 (0.0005) [2023-12-26 18:07:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 194060288. Throughput: 0: 9598.8, 1: 9546.0. Samples: 194049044. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 18:07:21,063][104569] Avg episode reward: [(0, '3716.932'), (1, '4647.383')] [2023-12-26 18:07:21,307][105620] Updated weights for policy 1, policy_version 379307 (0.0010) [2023-12-26 18:07:21,361][105620] Updated weights for policy 1, policy_version 379317 (0.0009) [2023-12-26 18:07:21,433][105620] Updated weights for policy 1, policy_version 379327 (0.0009) [2023-12-26 18:07:21,538][105692] Updated weights for policy 0, policy_version 378670 (0.0007) [2023-12-26 18:07:21,607][105692] Updated weights for policy 0, policy_version 378680 (0.0008) [2023-12-26 18:07:21,671][105692] Updated weights for policy 0, policy_version 378690 (0.0009) [2023-12-26 18:07:22,189][105620] Updated weights for policy 1, policy_version 379337 (0.0009) [2023-12-26 18:07:22,246][105620] Updated weights for policy 1, policy_version 379347 (0.0005) [2023-12-26 18:07:22,310][105620] Updated weights for policy 1, policy_version 379357 (0.0009) [2023-12-26 18:07:22,378][105620] Updated weights for policy 1, policy_version 379367 (0.0010) [2023-12-26 18:07:22,443][105692] Updated weights for policy 0, policy_version 378700 (0.0009) [2023-12-26 18:07:22,502][105692] Updated weights for policy 0, policy_version 378710 (0.0009) [2023-12-26 18:07:22,563][105692] Updated weights for policy 0, policy_version 378720 (0.0009) [2023-12-26 18:07:23,133][105620] Updated weights for policy 1, policy_version 379377 (0.0008) [2023-12-26 18:07:23,198][105620] Updated weights for policy 1, policy_version 379387 (0.0008) [2023-12-26 18:07:23,245][105692] Updated weights for policy 0, policy_version 378730 (0.0009) [2023-12-26 18:07:23,254][105620] Updated weights for policy 1, policy_version 379397 (0.0010) [2023-12-26 18:07:23,307][105692] Updated weights for policy 0, policy_version 378740 (0.0009) [2023-12-26 18:07:23,372][105692] Updated weights for policy 0, policy_version 378750 (0.0010) [2023-12-26 18:07:23,425][105692] Updated weights for policy 0, policy_version 378760 (0.0010) [2023-12-26 18:07:23,935][105620] Updated weights for policy 1, policy_version 379407 (0.0007) [2023-12-26 18:07:24,001][105620] Updated weights for policy 1, policy_version 379417 (0.0008) [2023-12-26 18:07:24,055][105620] Updated weights for policy 1, policy_version 379429 (0.0009) [2023-12-26 18:07:24,084][105692] Updated weights for policy 0, policy_version 378770 (0.0010) [2023-12-26 18:07:24,139][105692] Updated weights for policy 0, policy_version 378780 (0.0010) [2023-12-26 18:07:24,196][105692] Updated weights for policy 0, policy_version 378790 (0.0009) [2023-12-26 18:07:24,641][105620] Updated weights for policy 1, policy_version 379439 (0.0006) [2023-12-26 18:07:24,694][105620] Updated weights for policy 1, policy_version 379449 (0.0006) [2023-12-26 18:07:24,740][105620] Updated weights for policy 1, policy_version 379459 (0.0005) [2023-12-26 18:07:24,948][105692] Updated weights for policy 0, policy_version 378800 (0.0008) [2023-12-26 18:07:25,001][105692] Updated weights for policy 0, policy_version 378810 (0.0010) [2023-12-26 18:07:25,053][105692] Updated weights for policy 0, policy_version 378821 (0.0010) [2023-12-26 18:07:25,309][105620] Updated weights for policy 1, policy_version 379469 (0.0005) [2023-12-26 18:07:25,364][105620] Updated weights for policy 1, policy_version 379479 (0.0006) [2023-12-26 18:07:25,423][105620] Updated weights for policy 1, policy_version 379489 (0.0007) [2023-12-26 18:07:25,830][105692] Updated weights for policy 0, policy_version 378831 (0.0009) [2023-12-26 18:07:25,879][105692] Updated weights for policy 0, policy_version 378841 (0.0008) [2023-12-26 18:07:25,932][105692] Updated weights for policy 0, policy_version 378851 (0.0008) [2023-12-26 18:07:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 194158592. Throughput: 0: 9671.4, 1: 9593.3. Samples: 194166284. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 18:07:26,062][104569] Avg episode reward: [(0, '3682.737'), (1, '3820.833')] [2023-12-26 18:07:26,084][105620] Updated weights for policy 1, policy_version 379499 (0.0008) [2023-12-26 18:07:26,135][105620] Updated weights for policy 1, policy_version 379509 (0.0009) [2023-12-26 18:07:26,184][105620] Updated weights for policy 1, policy_version 379519 (0.0008) [2023-12-26 18:07:26,754][105692] Updated weights for policy 0, policy_version 378861 (0.0008) [2023-12-26 18:07:26,808][105692] Updated weights for policy 0, policy_version 378871 (0.0009) [2023-12-26 18:07:26,846][105620] Updated weights for policy 1, policy_version 379529 (0.0009) [2023-12-26 18:07:26,857][105692] Updated weights for policy 0, policy_version 378881 (0.0008) [2023-12-26 18:07:26,901][105620] Updated weights for policy 1, policy_version 379539 (0.0008) [2023-12-26 18:07:26,955][105620] Updated weights for policy 1, policy_version 379549 (0.0007) [2023-12-26 18:07:27,005][105620] Updated weights for policy 1, policy_version 379559 (0.0005) [2023-12-26 18:07:27,552][105620] Updated weights for policy 1, policy_version 379569 (0.0005) [2023-12-26 18:07:27,610][105620] Updated weights for policy 1, policy_version 379579 (0.0005) [2023-12-26 18:07:27,662][105620] Updated weights for policy 1, policy_version 379589 (0.0005) [2023-12-26 18:07:27,728][105692] Updated weights for policy 0, policy_version 378891 (0.0008) [2023-12-26 18:07:27,782][105692] Updated weights for policy 0, policy_version 378901 (0.0010) [2023-12-26 18:07:27,844][105692] Updated weights for policy 0, policy_version 378911 (0.0010) [2023-12-26 18:07:28,224][105620] Updated weights for policy 1, policy_version 379599 (0.0009) [2023-12-26 18:07:28,267][105620] Updated weights for policy 1, policy_version 379609 (0.0010) [2023-12-26 18:07:28,314][105620] Updated weights for policy 1, policy_version 379619 (0.0010) [2023-12-26 18:07:28,602][105692] Updated weights for policy 0, policy_version 378921 (0.0009) [2023-12-26 18:07:28,657][105692] Updated weights for policy 0, policy_version 378931 (0.0008) [2023-12-26 18:07:28,718][105692] Updated weights for policy 0, policy_version 378941 (0.0006) [2023-12-26 18:07:28,777][105692] Updated weights for policy 0, policy_version 378951 (0.0008) [2023-12-26 18:07:29,075][105620] Updated weights for policy 1, policy_version 379629 (0.0010) [2023-12-26 18:07:29,126][105620] Updated weights for policy 1, policy_version 379639 (0.0010) [2023-12-26 18:07:29,177][105620] Updated weights for policy 1, policy_version 379649 (0.0010) [2023-12-26 18:07:29,437][105692] Updated weights for policy 0, policy_version 378961 (0.0008) [2023-12-26 18:07:29,495][105692] Updated weights for policy 0, policy_version 378971 (0.0007) [2023-12-26 18:07:29,553][105692] Updated weights for policy 0, policy_version 378981 (0.0008) [2023-12-26 18:07:29,915][105620] Updated weights for policy 1, policy_version 379659 (0.0010) [2023-12-26 18:07:29,972][105620] Updated weights for policy 1, policy_version 379669 (0.0009) [2023-12-26 18:07:30,031][105620] Updated weights for policy 1, policy_version 379679 (0.0009) [2023-12-26 18:07:30,346][105692] Updated weights for policy 0, policy_version 378991 (0.0009) [2023-12-26 18:07:30,407][105692] Updated weights for policy 0, policy_version 379001 (0.0008) [2023-12-26 18:07:30,463][105692] Updated weights for policy 0, policy_version 379011 (0.0008) [2023-12-26 18:07:30,778][105620] Updated weights for policy 1, policy_version 379689 (0.0009) [2023-12-26 18:07:30,831][105620] Updated weights for policy 1, policy_version 379699 (0.0005) [2023-12-26 18:07:30,891][105620] Updated weights for policy 1, policy_version 379709 (0.0009) [2023-12-26 18:07:30,949][105620] Updated weights for policy 1, policy_version 379719 (0.0010) [2023-12-26 18:07:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 194256896. Throughput: 0: 9590.8, 1: 9747.9. Samples: 194226040. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 18:07:31,063][104569] Avg episode reward: [(0, '3656.927'), (1, '4469.334')] [2023-12-26 18:07:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000379016_97042432.pth... [2023-12-26 18:07:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000379720_97214464.pth... [2023-12-26 18:07:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000377896_96755712.pth [2023-12-26 18:07:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000378568_96919552.pth [2023-12-26 18:07:31,216][105692] Updated weights for policy 0, policy_version 379021 (0.0007) [2023-12-26 18:07:31,288][105692] Updated weights for policy 0, policy_version 379031 (0.0008) [2023-12-26 18:07:31,353][105692] Updated weights for policy 0, policy_version 379041 (0.0008) [2023-12-26 18:07:31,650][105620] Updated weights for policy 1, policy_version 379729 (0.0009) [2023-12-26 18:07:31,713][105620] Updated weights for policy 1, policy_version 379739 (0.0011) [2023-12-26 18:07:31,778][105620] Updated weights for policy 1, policy_version 379749 (0.0011) [2023-12-26 18:07:32,004][105692] Updated weights for policy 0, policy_version 379051 (0.0007) [2023-12-26 18:07:32,067][105692] Updated weights for policy 0, policy_version 379061 (0.0006) [2023-12-26 18:07:32,127][105692] Updated weights for policy 0, policy_version 379071 (0.0008) [2023-12-26 18:07:32,490][105620] Updated weights for policy 1, policy_version 379759 (0.0007) [2023-12-26 18:07:32,549][105620] Updated weights for policy 1, policy_version 379769 (0.0005) [2023-12-26 18:07:32,603][105620] Updated weights for policy 1, policy_version 379779 (0.0006) [2023-12-26 18:07:32,777][105692] Updated weights for policy 0, policy_version 379081 (0.0010) [2023-12-26 18:07:32,842][105692] Updated weights for policy 0, policy_version 379091 (0.0008) [2023-12-26 18:07:32,914][105692] Updated weights for policy 0, policy_version 379101 (0.0011) [2023-12-26 18:07:32,981][105692] Updated weights for policy 0, policy_version 379111 (0.0010) [2023-12-26 18:07:33,191][105620] Updated weights for policy 1, policy_version 379789 (0.0008) [2023-12-26 18:07:33,240][105620] Updated weights for policy 1, policy_version 379799 (0.0009) [2023-12-26 18:07:33,293][105620] Updated weights for policy 1, policy_version 379810 (0.0010) [2023-12-26 18:07:33,558][105692] Updated weights for policy 0, policy_version 379121 (0.0006) [2023-12-26 18:07:33,620][105692] Updated weights for policy 0, policy_version 379131 (0.0005) [2023-12-26 18:07:33,687][105692] Updated weights for policy 0, policy_version 379141 (0.0005) [2023-12-26 18:07:34,187][105692] Updated weights for policy 0, policy_version 379151 (0.0008) [2023-12-26 18:07:34,219][105620] Updated weights for policy 1, policy_version 379821 (0.0009) [2023-12-26 18:07:34,246][105692] Updated weights for policy 0, policy_version 379161 (0.0008) [2023-12-26 18:07:34,276][105620] Updated weights for policy 1, policy_version 379831 (0.0006) [2023-12-26 18:07:34,309][105692] Updated weights for policy 0, policy_version 379171 (0.0010) [2023-12-26 18:07:34,343][105620] Updated weights for policy 1, policy_version 379841 (0.0006) [2023-12-26 18:07:34,921][105620] Updated weights for policy 1, policy_version 379851 (0.0006) [2023-12-26 18:07:34,979][105620] Updated weights for policy 1, policy_version 379861 (0.0006) [2023-12-26 18:07:35,034][105620] Updated weights for policy 1, policy_version 379871 (0.0005) [2023-12-26 18:07:35,042][105692] Updated weights for policy 0, policy_version 379181 (0.0010) [2023-12-26 18:07:35,087][105692] Updated weights for policy 0, policy_version 379191 (0.0009) [2023-12-26 18:07:35,144][105692] Updated weights for policy 0, policy_version 379201 (0.0010) [2023-12-26 18:07:35,734][105620] Updated weights for policy 1, policy_version 379881 (0.0005) [2023-12-26 18:07:35,790][105620] Updated weights for policy 1, policy_version 379891 (0.0007) [2023-12-26 18:07:35,843][105620] Updated weights for policy 1, policy_version 379901 (0.0010) [2023-12-26 18:07:35,895][105620] Updated weights for policy 1, policy_version 379911 (0.0010) [2023-12-26 18:07:35,916][105692] Updated weights for policy 0, policy_version 379211 (0.0010) [2023-12-26 18:07:35,978][105692] Updated weights for policy 0, policy_version 379221 (0.0007) [2023-12-26 18:07:36,042][105692] Updated weights for policy 0, policy_version 379231 (0.0010) [2023-12-26 18:07:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.3, 300 sec: 19522.0). Total num frames: 194355200. Throughput: 0: 9700.3, 1: 9812.5. Samples: 194346000. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 18:07:36,063][104569] Avg episode reward: [(0, '7338.434'), (1, '4468.232')] [2023-12-26 18:07:36,625][105620] Updated weights for policy 1, policy_version 379921 (0.0010) [2023-12-26 18:07:36,683][105620] Updated weights for policy 1, policy_version 379931 (0.0010) [2023-12-26 18:07:36,746][105620] Updated weights for policy 1, policy_version 379941 (0.0010) [2023-12-26 18:07:36,781][105692] Updated weights for policy 0, policy_version 379241 (0.0009) [2023-12-26 18:07:36,835][105692] Updated weights for policy 0, policy_version 379251 (0.0008) [2023-12-26 18:07:36,895][105692] Updated weights for policy 0, policy_version 379261 (0.0008) [2023-12-26 18:07:36,939][105692] Updated weights for policy 0, policy_version 379271 (0.0008) [2023-12-26 18:07:37,496][105620] Updated weights for policy 1, policy_version 379951 (0.0009) [2023-12-26 18:07:37,545][105586] KL-divergence is very high: 109.1254 [2023-12-26 18:07:37,553][105620] Updated weights for policy 1, policy_version 379961 (0.0008) [2023-12-26 18:07:37,562][105586] KL-divergence is very high: 100.6548 [2023-12-26 18:07:37,573][105586] KL-divergence is very high: 113.7034 [2023-12-26 18:07:37,584][105586] KL-divergence is very high: 130.5087 [2023-12-26 18:07:37,590][105586] KL-divergence is very high: 141.8877 [2023-12-26 18:07:37,605][105620] Updated weights for policy 1, policy_version 379971 (0.0006) [2023-12-26 18:07:37,710][105692] Updated weights for policy 0, policy_version 379281 (0.0007) [2023-12-26 18:07:37,779][105692] Updated weights for policy 0, policy_version 379291 (0.0006) [2023-12-26 18:07:37,843][105692] Updated weights for policy 0, policy_version 379301 (0.0005) [2023-12-26 18:07:38,218][105620] Updated weights for policy 1, policy_version 379981 (0.0005) [2023-12-26 18:07:38,265][105620] Updated weights for policy 1, policy_version 379991 (0.0005) [2023-12-26 18:07:38,321][105620] Updated weights for policy 1, policy_version 380001 (0.0006) [2023-12-26 18:07:38,611][105692] Updated weights for policy 0, policy_version 379311 (0.0007) [2023-12-26 18:07:38,672][105692] Updated weights for policy 0, policy_version 379321 (0.0005) [2023-12-26 18:07:38,736][105692] Updated weights for policy 0, policy_version 379331 (0.0005) [2023-12-26 18:07:38,971][105620] Updated weights for policy 1, policy_version 380011 (0.0010) [2023-12-26 18:07:39,033][105620] Updated weights for policy 1, policy_version 380021 (0.0009) [2023-12-26 18:07:39,093][105620] Updated weights for policy 1, policy_version 380031 (0.0009) [2023-12-26 18:07:39,386][105692] Updated weights for policy 0, policy_version 379341 (0.0008) [2023-12-26 18:07:39,452][105692] Updated weights for policy 0, policy_version 379351 (0.0009) [2023-12-26 18:07:39,516][105692] Updated weights for policy 0, policy_version 379361 (0.0010) [2023-12-26 18:07:39,872][105620] Updated weights for policy 1, policy_version 380041 (0.0010) [2023-12-26 18:07:39,935][105620] Updated weights for policy 1, policy_version 380051 (0.0009) [2023-12-26 18:07:40,003][105620] Updated weights for policy 1, policy_version 380061 (0.0008) [2023-12-26 18:07:40,065][105620] Updated weights for policy 1, policy_version 380071 (0.0009) [2023-12-26 18:07:40,288][105692] Updated weights for policy 0, policy_version 379371 (0.0009) [2023-12-26 18:07:40,350][105692] Updated weights for policy 0, policy_version 379381 (0.0009) [2023-12-26 18:07:40,405][105692] Updated weights for policy 0, policy_version 379391 (0.0009) [2023-12-26 18:07:40,745][105620] Updated weights for policy 1, policy_version 380081 (0.0006) [2023-12-26 18:07:40,797][105620] Updated weights for policy 1, policy_version 380091 (0.0005) [2023-12-26 18:07:40,848][105620] Updated weights for policy 1, policy_version 380101 (0.0007) [2023-12-26 18:07:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 194453504. Throughput: 0: 9748.0, 1: 9788.0. Samples: 194461552. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 18:07:41,063][104569] Avg episode reward: [(0, '9358.482'), (1, '4002.319')] [2023-12-26 18:07:41,118][105692] Updated weights for policy 0, policy_version 379401 (0.0009) [2023-12-26 18:07:41,188][105692] Updated weights for policy 0, policy_version 379411 (0.0009) [2023-12-26 18:07:41,251][105692] Updated weights for policy 0, policy_version 379421 (0.0010) [2023-12-26 18:07:41,314][105692] Updated weights for policy 0, policy_version 379431 (0.0009) [2023-12-26 18:07:41,626][105620] Updated weights for policy 1, policy_version 380111 (0.0008) [2023-12-26 18:07:41,689][105620] Updated weights for policy 1, policy_version 380121 (0.0008) [2023-12-26 18:07:41,749][105620] Updated weights for policy 1, policy_version 380131 (0.0008) [2023-12-26 18:07:42,095][105692] Updated weights for policy 0, policy_version 379441 (0.0007) [2023-12-26 18:07:42,158][105692] Updated weights for policy 0, policy_version 379451 (0.0006) [2023-12-26 18:07:42,219][105692] Updated weights for policy 0, policy_version 379461 (0.0005) [2023-12-26 18:07:42,555][105620] Updated weights for policy 1, policy_version 380141 (0.0009) [2023-12-26 18:07:42,614][105620] Updated weights for policy 1, policy_version 380151 (0.0008) [2023-12-26 18:07:42,670][105620] Updated weights for policy 1, policy_version 380161 (0.0008) [2023-12-26 18:07:42,814][105692] Updated weights for policy 0, policy_version 379471 (0.0007) [2023-12-26 18:07:42,869][105692] Updated weights for policy 0, policy_version 379481 (0.0010) [2023-12-26 18:07:42,925][105692] Updated weights for policy 0, policy_version 379491 (0.0010) [2023-12-26 18:07:43,339][105620] Updated weights for policy 1, policy_version 380171 (0.0008) [2023-12-26 18:07:43,391][105620] Updated weights for policy 1, policy_version 380181 (0.0005) [2023-12-26 18:07:43,442][105620] Updated weights for policy 1, policy_version 380191 (0.0006) [2023-12-26 18:07:43,643][105692] Updated weights for policy 0, policy_version 379501 (0.0010) [2023-12-26 18:07:43,701][105692] Updated weights for policy 0, policy_version 379511 (0.0010) [2023-12-26 18:07:43,758][105692] Updated weights for policy 0, policy_version 379521 (0.0010) [2023-12-26 18:07:44,093][105620] Updated weights for policy 1, policy_version 380201 (0.0006) [2023-12-26 18:07:44,139][105620] Updated weights for policy 1, policy_version 380211 (0.0006) [2023-12-26 18:07:44,174][105586] KL-divergence is very high: 106.4539 [2023-12-26 18:07:44,180][105586] KL-divergence is very high: 163.6507 [2023-12-26 18:07:44,185][105586] KL-divergence is very high: 159.8453 [2023-12-26 18:07:44,190][105586] KL-divergence is very high: 199.4613 [2023-12-26 18:07:44,192][105620] Updated weights for policy 1, policy_version 380221 (0.0008) [2023-12-26 18:07:44,197][105586] KL-divergence is very high: 100.7647 [2023-12-26 18:07:44,203][105586] KL-divergence is very high: 233.3595 [2023-12-26 18:07:44,209][105586] KL-divergence is very high: 217.1786 [2023-12-26 18:07:44,220][105586] KL-divergence is very high: 202.8319 [2023-12-26 18:07:44,227][105586] KL-divergence is very high: 216.1205 [2023-12-26 18:07:44,233][105586] KL-divergence is very high: 179.5242 [2023-12-26 18:07:44,239][105586] KL-divergence is very high: 182.3219 [2023-12-26 18:07:44,250][105620] Updated weights for policy 1, policy_version 380231 (0.0008) [2023-12-26 18:07:44,251][105586] KL-divergence is very high: 144.8230 [2023-12-26 18:07:44,496][105692] Updated weights for policy 0, policy_version 379531 (0.0010) [2023-12-26 18:07:44,547][105692] Updated weights for policy 0, policy_version 379541 (0.0010) [2023-12-26 18:07:44,591][105692] Updated weights for policy 0, policy_version 379551 (0.0010) [2023-12-26 18:07:44,846][105586] KL-divergence is very high: 135.8647 [2023-12-26 18:07:44,893][105620] Updated weights for policy 1, policy_version 380241 (0.0008) [2023-12-26 18:07:44,945][105620] Updated weights for policy 1, policy_version 380251 (0.0010) [2023-12-26 18:07:45,001][105620] Updated weights for policy 1, policy_version 380261 (0.0010) [2023-12-26 18:07:45,310][105692] Updated weights for policy 0, policy_version 379561 (0.0010) [2023-12-26 18:07:45,377][105692] Updated weights for policy 0, policy_version 379571 (0.0007) [2023-12-26 18:07:45,432][105692] Updated weights for policy 0, policy_version 379581 (0.0010) [2023-12-26 18:07:45,491][105692] Updated weights for policy 0, policy_version 379591 (0.0010) [2023-12-26 18:07:45,747][105620] Updated weights for policy 1, policy_version 380271 (0.0007) [2023-12-26 18:07:45,810][105620] Updated weights for policy 1, policy_version 380281 (0.0005) [2023-12-26 18:07:45,870][105620] Updated weights for policy 1, policy_version 380291 (0.0005) [2023-12-26 18:07:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 194551808. Throughput: 0: 9730.3, 1: 9743.7. Samples: 194520644. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 18:07:46,063][104569] Avg episode reward: [(0, '9359.032'), (1, '4468.006')] [2023-12-26 18:07:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000380296_97361920.pth... [2023-12-26 18:07:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000379144_97067008.pth [2023-12-26 18:07:46,120][105692] Updated weights for policy 0, policy_version 379601 (0.0006) [2023-12-26 18:07:46,174][105692] Updated weights for policy 0, policy_version 379611 (0.0005) [2023-12-26 18:07:46,230][105692] Updated weights for policy 0, policy_version 379621 (0.0010) [2023-12-26 18:07:46,245][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000379624_97198080.pth... [2023-12-26 18:07:46,248][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000378472_96903168.pth [2023-12-26 18:07:46,427][105620] Updated weights for policy 1, policy_version 380301 (0.0008) [2023-12-26 18:07:46,485][105620] Updated weights for policy 1, policy_version 380311 (0.0010) [2023-12-26 18:07:46,546][105620] Updated weights for policy 1, policy_version 380321 (0.0006) [2023-12-26 18:07:46,887][105692] Updated weights for policy 0, policy_version 379631 (0.0007) [2023-12-26 18:07:46,950][105692] Updated weights for policy 0, policy_version 379641 (0.0008) [2023-12-26 18:07:47,013][105692] Updated weights for policy 0, policy_version 379651 (0.0009) [2023-12-26 18:07:47,247][105620] Updated weights for policy 1, policy_version 380331 (0.0007) [2023-12-26 18:07:47,308][105620] Updated weights for policy 1, policy_version 380341 (0.0007) [2023-12-26 18:07:47,375][105620] Updated weights for policy 1, policy_version 380351 (0.0005) [2023-12-26 18:07:47,806][105692] Updated weights for policy 0, policy_version 379661 (0.0009) [2023-12-26 18:07:47,866][105692] Updated weights for policy 0, policy_version 379671 (0.0009) [2023-12-26 18:07:47,925][105692] Updated weights for policy 0, policy_version 379681 (0.0009) [2023-12-26 18:07:48,031][105620] Updated weights for policy 1, policy_version 380361 (0.0006) [2023-12-26 18:07:48,082][105620] Updated weights for policy 1, policy_version 380371 (0.0009) [2023-12-26 18:07:48,146][105620] Updated weights for policy 1, policy_version 380381 (0.0005) [2023-12-26 18:07:48,200][105620] Updated weights for policy 1, policy_version 380391 (0.0006) [2023-12-26 18:07:48,745][105692] Updated weights for policy 0, policy_version 379691 (0.0009) [2023-12-26 18:07:48,791][105692] Updated weights for policy 0, policy_version 379701 (0.0009) [2023-12-26 18:07:48,838][105692] Updated weights for policy 0, policy_version 379711 (0.0009) [2023-12-26 18:07:48,849][105620] Updated weights for policy 1, policy_version 380401 (0.0008) [2023-12-26 18:07:48,907][105620] Updated weights for policy 1, policy_version 380411 (0.0006) [2023-12-26 18:07:48,963][105620] Updated weights for policy 1, policy_version 380421 (0.0009) [2023-12-26 18:07:49,571][105692] Updated weights for policy 0, policy_version 379721 (0.0007) [2023-12-26 18:07:49,625][105692] Updated weights for policy 0, policy_version 379731 (0.0005) [2023-12-26 18:07:49,678][105692] Updated weights for policy 0, policy_version 379741 (0.0006) [2023-12-26 18:07:49,679][105585] KL-divergence is very high: 219.9978 [2023-12-26 18:07:49,685][105585] KL-divergence is very high: 234.8025 [2023-12-26 18:07:49,692][105585] KL-divergence is very high: 126.4226 [2023-12-26 18:07:49,730][105585] KL-divergence is very high: 142.9762 [2023-12-26 18:07:49,737][105585] KL-divergence is very high: 176.7321 [2023-12-26 18:07:49,743][105585] KL-divergence is very high: 129.0161 [2023-12-26 18:07:49,745][105692] Updated weights for policy 0, policy_version 379751 (0.0009) [2023-12-26 18:07:49,769][105620] Updated weights for policy 1, policy_version 380431 (0.0009) [2023-12-26 18:07:49,820][105620] Updated weights for policy 1, policy_version 380441 (0.0009) [2023-12-26 18:07:49,883][105620] Updated weights for policy 1, policy_version 380451 (0.0008) [2023-12-26 18:07:50,360][105585] KL-divergence is very high: 162.8047 [2023-12-26 18:07:50,365][105585] KL-divergence is very high: 106.4751 [2023-12-26 18:07:50,371][105585] KL-divergence is very high: 146.7993 [2023-12-26 18:07:50,377][105585] KL-divergence is very high: 209.5114 [2023-12-26 18:07:50,383][105585] KL-divergence is very high: 220.2551 [2023-12-26 18:07:50,394][105585] KL-divergence is very high: 323.9858 [2023-12-26 18:07:50,404][105692] Updated weights for policy 0, policy_version 379761 (0.0007) [2023-12-26 18:07:50,406][105585] KL-divergence is very high: 265.0100 [2023-12-26 18:07:50,411][105585] KL-divergence is very high: 170.5668 [2023-12-26 18:07:50,417][105585] KL-divergence is very high: 110.7105 [2023-12-26 18:07:50,436][105585] KL-divergence is very high: 126.4940 [2023-12-26 18:07:50,442][105585] KL-divergence is very high: 142.4025 [2023-12-26 18:07:50,447][105585] KL-divergence is very high: 254.3762 [2023-12-26 18:07:50,451][105585] KL-divergence is very high: 264.6985 [2023-12-26 18:07:50,456][105692] Updated weights for policy 0, policy_version 379771 (0.0005) [2023-12-26 18:07:50,456][105585] KL-divergence is very high: 201.4774 [2023-12-26 18:07:50,492][105585] KL-divergence is very high: 103.3347 [2023-12-26 18:07:50,500][105585] KL-divergence is very high: 122.6948 [2023-12-26 18:07:50,508][105585] KL-divergence is very high: 141.1553 [2023-12-26 18:07:50,516][105585] KL-divergence is very high: 153.9178 [2023-12-26 18:07:50,523][105692] Updated weights for policy 0, policy_version 379781 (0.0005) [2023-12-26 18:07:50,524][105585] KL-divergence is very high: 144.7769 [2023-12-26 18:07:50,539][105585] KL-divergence is very high: 209.2260 [2023-12-26 18:07:50,685][105620] Updated weights for policy 1, policy_version 380461 (0.0008) [2023-12-26 18:07:50,752][105620] Updated weights for policy 1, policy_version 380471 (0.0009) [2023-12-26 18:07:50,822][105620] Updated weights for policy 1, policy_version 380481 (0.0009) [2023-12-26 18:07:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 194650112. Throughput: 0: 9691.9, 1: 9748.4. Samples: 194638420. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 18:07:51,063][104569] Avg episode reward: [(0, '5549.028'), (1, '5567.347')] [2023-12-26 18:07:51,173][105585] KL-divergence is very high: 102.9600 [2023-12-26 18:07:51,209][105692] Updated weights for policy 0, policy_version 379791 (0.0008) [2023-12-26 18:07:51,273][105692] Updated weights for policy 0, policy_version 379801 (0.0009) [2023-12-26 18:07:51,336][105692] Updated weights for policy 0, policy_version 379811 (0.0008) [2023-12-26 18:07:51,573][105620] Updated weights for policy 1, policy_version 380491 (0.0010) [2023-12-26 18:07:51,645][105620] Updated weights for policy 1, policy_version 380501 (0.0009) [2023-12-26 18:07:51,705][105620] Updated weights for policy 1, policy_version 380511 (0.0007) [2023-12-26 18:07:52,064][105692] Updated weights for policy 0, policy_version 379821 (0.0009) [2023-12-26 18:07:52,134][105692] Updated weights for policy 0, policy_version 379831 (0.0010) [2023-12-26 18:07:52,198][105692] Updated weights for policy 0, policy_version 379841 (0.0009) [2023-12-26 18:07:52,300][105620] Updated weights for policy 1, policy_version 380521 (0.0010) [2023-12-26 18:07:52,365][105620] Updated weights for policy 1, policy_version 380531 (0.0011) [2023-12-26 18:07:52,425][105620] Updated weights for policy 1, policy_version 380541 (0.0011) [2023-12-26 18:07:52,481][105620] Updated weights for policy 1, policy_version 380551 (0.0010) [2023-12-26 18:07:52,936][105692] Updated weights for policy 0, policy_version 379851 (0.0009) [2023-12-26 18:07:52,999][105692] Updated weights for policy 0, policy_version 379861 (0.0008) [2023-12-26 18:07:53,062][105692] Updated weights for policy 0, policy_version 379871 (0.0008) [2023-12-26 18:07:53,235][105620] Updated weights for policy 1, policy_version 380561 (0.0010) [2023-12-26 18:07:53,293][105620] Updated weights for policy 1, policy_version 380571 (0.0010) [2023-12-26 18:07:53,352][105620] Updated weights for policy 1, policy_version 380581 (0.0010) [2023-12-26 18:07:53,795][105692] Updated weights for policy 0, policy_version 379881 (0.0008) [2023-12-26 18:07:53,853][105692] Updated weights for policy 0, policy_version 379891 (0.0007) [2023-12-26 18:07:53,913][105692] Updated weights for policy 0, policy_version 379901 (0.0009) [2023-12-26 18:07:53,974][105692] Updated weights for policy 0, policy_version 379912 (0.0012) [2023-12-26 18:07:54,037][105620] Updated weights for policy 1, policy_version 380591 (0.0009) [2023-12-26 18:07:54,102][105620] Updated weights for policy 1, policy_version 380601 (0.0007) [2023-12-26 18:07:54,157][105620] Updated weights for policy 1, policy_version 380611 (0.0008) [2023-12-26 18:07:54,647][105692] Updated weights for policy 0, policy_version 379922 (0.0005) [2023-12-26 18:07:54,708][105692] Updated weights for policy 0, policy_version 379932 (0.0005) [2023-12-26 18:07:54,769][105692] Updated weights for policy 0, policy_version 379942 (0.0006) [2023-12-26 18:07:54,844][105620] Updated weights for policy 1, policy_version 380621 (0.0008) [2023-12-26 18:07:54,907][105620] Updated weights for policy 1, policy_version 380631 (0.0005) [2023-12-26 18:07:54,960][105620] Updated weights for policy 1, policy_version 380641 (0.0007) [2023-12-26 18:07:55,335][105692] Updated weights for policy 0, policy_version 379952 (0.0009) [2023-12-26 18:07:55,384][105692] Updated weights for policy 0, policy_version 379962 (0.0010) [2023-12-26 18:07:55,429][105692] Updated weights for policy 0, policy_version 379972 (0.0010) [2023-12-26 18:07:55,625][105620] Updated weights for policy 1, policy_version 380651 (0.0009) [2023-12-26 18:07:55,681][105620] Updated weights for policy 1, policy_version 380661 (0.0008) [2023-12-26 18:07:55,695][105586] KL-divergence is very high: 120.7489 [2023-12-26 18:07:55,719][105586] KL-divergence is very high: 118.6423 [2023-12-26 18:07:55,741][105620] Updated weights for policy 1, policy_version 380671 (0.0008) [2023-12-26 18:07:55,743][105586] KL-divergence is very high: 169.7025 [2023-12-26 18:07:55,766][105586] KL-divergence is very high: 132.4809 [2023-12-26 18:07:55,788][105586] KL-divergence is very high: 151.7118 [2023-12-26 18:07:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 194748416. Throughput: 0: 9728.1, 1: 9794.8. Samples: 194756708. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 18:07:56,062][104569] Avg episode reward: [(0, '6687.905'), (1, '5385.502')] [2023-12-26 18:07:56,108][105692] Updated weights for policy 0, policy_version 379982 (0.0009) [2023-12-26 18:07:56,165][105692] Updated weights for policy 0, policy_version 379992 (0.0008) [2023-12-26 18:07:56,213][105692] Updated weights for policy 0, policy_version 380002 (0.0010) [2023-12-26 18:07:56,444][105620] Updated weights for policy 1, policy_version 380681 (0.0008) [2023-12-26 18:07:56,508][105620] Updated weights for policy 1, policy_version 380691 (0.0007) [2023-12-26 18:07:56,565][105620] Updated weights for policy 1, policy_version 380702 (0.0012) [2023-12-26 18:07:56,618][105620] Updated weights for policy 1, policy_version 380712 (0.0009) [2023-12-26 18:07:56,783][105692] Updated weights for policy 0, policy_version 380012 (0.0010) [2023-12-26 18:07:56,838][105692] Updated weights for policy 0, policy_version 380022 (0.0011) [2023-12-26 18:07:56,886][105692] Updated weights for policy 0, policy_version 380032 (0.0010) [2023-12-26 18:07:57,236][105620] Updated weights for policy 1, policy_version 380722 (0.0006) [2023-12-26 18:07:57,277][105620] Updated weights for policy 1, policy_version 380732 (0.0006) [2023-12-26 18:07:57,334][105620] Updated weights for policy 1, policy_version 380742 (0.0005) [2023-12-26 18:07:57,445][105692] Updated weights for policy 0, policy_version 380042 (0.0009) [2023-12-26 18:07:57,514][105692] Updated weights for policy 0, policy_version 380052 (0.0005) [2023-12-26 18:07:57,574][105692] Updated weights for policy 0, policy_version 380062 (0.0005) [2023-12-26 18:07:57,633][105692] Updated weights for policy 0, policy_version 380072 (0.0005) [2023-12-26 18:07:57,875][105620] Updated weights for policy 1, policy_version 380752 (0.0010) [2023-12-26 18:07:57,926][105620] Updated weights for policy 1, policy_version 380762 (0.0010) [2023-12-26 18:07:57,981][105620] Updated weights for policy 1, policy_version 380772 (0.0010) [2023-12-26 18:07:58,149][105692] Updated weights for policy 0, policy_version 380082 (0.0010) [2023-12-26 18:07:58,208][105692] Updated weights for policy 0, policy_version 380092 (0.0011) [2023-12-26 18:07:58,260][105692] Updated weights for policy 0, policy_version 380102 (0.0010) [2023-12-26 18:07:58,717][105620] Updated weights for policy 1, policy_version 380782 (0.0008) [2023-12-26 18:07:58,787][105620] Updated weights for policy 1, policy_version 380792 (0.0007) [2023-12-26 18:07:58,862][105620] Updated weights for policy 1, policy_version 380802 (0.0007) [2023-12-26 18:07:59,095][105692] Updated weights for policy 0, policy_version 380112 (0.0010) [2023-12-26 18:07:59,153][105692] Updated weights for policy 0, policy_version 380122 (0.0010) [2023-12-26 18:07:59,209][105692] Updated weights for policy 0, policy_version 380132 (0.0010) [2023-12-26 18:07:59,586][105620] Updated weights for policy 1, policy_version 380812 (0.0010) [2023-12-26 18:07:59,640][105620] Updated weights for policy 1, policy_version 380822 (0.0010) [2023-12-26 18:07:59,704][105620] Updated weights for policy 1, policy_version 380832 (0.0006) [2023-12-26 18:08:00,004][105692] Updated weights for policy 0, policy_version 380142 (0.0008) [2023-12-26 18:08:00,077][105692] Updated weights for policy 0, policy_version 380152 (0.0006) [2023-12-26 18:08:00,141][105692] Updated weights for policy 0, policy_version 380162 (0.0008) [2023-12-26 18:08:00,299][105620] Updated weights for policy 1, policy_version 380842 (0.0005) [2023-12-26 18:08:00,361][105620] Updated weights for policy 1, policy_version 380852 (0.0006) [2023-12-26 18:08:00,424][105620] Updated weights for policy 1, policy_version 380862 (0.0006) [2023-12-26 18:08:00,473][105620] Updated weights for policy 1, policy_version 380872 (0.0005) [2023-12-26 18:08:00,763][105692] Updated weights for policy 0, policy_version 380172 (0.0008) [2023-12-26 18:08:00,827][105692] Updated weights for policy 0, policy_version 380182 (0.0010) [2023-12-26 18:08:00,881][105692] Updated weights for policy 0, policy_version 380192 (0.0010) [2023-12-26 18:08:01,044][105620] Updated weights for policy 1, policy_version 380882 (0.0009) [2023-12-26 18:08:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 194854912. Throughput: 0: 9902.2, 1: 9869.6. Samples: 194823212. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 18:08:01,063][104569] Avg episode reward: [(0, '7232.167'), (1, '4935.968')] [2023-12-26 18:08:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000380200_97345536.pth... [2023-12-26 18:08:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000379016_97042432.pth [2023-12-26 18:08:01,114][105620] Updated weights for policy 1, policy_version 380892 (0.0009) [2023-12-26 18:08:01,178][105620] Updated weights for policy 1, policy_version 380902 (0.0011) [2023-12-26 18:08:01,184][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000380904_97517568.pth... [2023-12-26 18:08:01,187][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000379720_97214464.pth [2023-12-26 18:08:01,599][105692] Updated weights for policy 0, policy_version 380202 (0.0010) [2023-12-26 18:08:01,657][105692] Updated weights for policy 0, policy_version 380212 (0.0009) [2023-12-26 18:08:01,709][105692] Updated weights for policy 0, policy_version 380222 (0.0008) [2023-12-26 18:08:01,778][105692] Updated weights for policy 0, policy_version 380232 (0.0008) [2023-12-26 18:08:01,933][105620] Updated weights for policy 1, policy_version 380912 (0.0006) [2023-12-26 18:08:01,988][105620] Updated weights for policy 1, policy_version 380922 (0.0006) [2023-12-26 18:08:02,057][105620] Updated weights for policy 1, policy_version 380932 (0.0006) [2023-12-26 18:08:02,512][105692] Updated weights for policy 0, policy_version 380242 (0.0006) [2023-12-26 18:08:02,563][105692] Updated weights for policy 0, policy_version 380252 (0.0008) [2023-12-26 18:08:02,614][105692] Updated weights for policy 0, policy_version 380262 (0.0009) [2023-12-26 18:08:02,795][105620] Updated weights for policy 1, policy_version 380942 (0.0007) [2023-12-26 18:08:02,848][105620] Updated weights for policy 1, policy_version 380952 (0.0005) [2023-12-26 18:08:02,908][105620] Updated weights for policy 1, policy_version 380962 (0.0006) [2023-12-26 18:08:03,415][105692] Updated weights for policy 0, policy_version 380272 (0.0009) [2023-12-26 18:08:03,466][105692] Updated weights for policy 0, policy_version 380282 (0.0007) [2023-12-26 18:08:03,519][105692] Updated weights for policy 0, policy_version 380292 (0.0005) [2023-12-26 18:08:03,536][105620] Updated weights for policy 1, policy_version 380972 (0.0007) [2023-12-26 18:08:03,581][105620] Updated weights for policy 1, policy_version 380982 (0.0005) [2023-12-26 18:08:03,638][105620] Updated weights for policy 1, policy_version 380992 (0.0005) [2023-12-26 18:08:04,246][105620] Updated weights for policy 1, policy_version 381002 (0.0006) [2023-12-26 18:08:04,268][105692] Updated weights for policy 0, policy_version 380302 (0.0008) [2023-12-26 18:08:04,300][105620] Updated weights for policy 1, policy_version 381012 (0.0006) [2023-12-26 18:08:04,325][105692] Updated weights for policy 0, policy_version 380312 (0.0008) [2023-12-26 18:08:04,359][105620] Updated weights for policy 1, policy_version 381022 (0.0008) [2023-12-26 18:08:04,380][105692] Updated weights for policy 0, policy_version 380322 (0.0008) [2023-12-26 18:08:04,417][105620] Updated weights for policy 1, policy_version 381032 (0.0010) [2023-12-26 18:08:05,090][105692] Updated weights for policy 0, policy_version 380332 (0.0010) [2023-12-26 18:08:05,141][105620] Updated weights for policy 1, policy_version 381042 (0.0011) [2023-12-26 18:08:05,145][105692] Updated weights for policy 0, policy_version 380342 (0.0010) [2023-12-26 18:08:05,189][105692] Updated weights for policy 0, policy_version 380352 (0.0010) [2023-12-26 18:08:05,204][105620] Updated weights for policy 1, policy_version 381052 (0.0010) [2023-12-26 18:08:05,266][105620] Updated weights for policy 1, policy_version 381062 (0.0011) [2023-12-26 18:08:05,948][105692] Updated weights for policy 0, policy_version 380362 (0.0010) [2023-12-26 18:08:06,001][105692] Updated weights for policy 0, policy_version 380372 (0.0011) [2023-12-26 18:08:06,009][105620] Updated weights for policy 1, policy_version 381072 (0.0011) [2023-12-26 18:08:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.4, 300 sec: 19522.0). Total num frames: 194945024. Throughput: 0: 9793.2, 1: 10054.1. Samples: 194942172. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 18:08:06,062][104569] Avg episode reward: [(0, '9014.253'), (1, '4566.280')] [2023-12-26 18:08:06,063][105692] Updated weights for policy 0, policy_version 380382 (0.0010) [2023-12-26 18:08:06,072][105620] Updated weights for policy 1, policy_version 381082 (0.0011) [2023-12-26 18:08:06,123][105692] Updated weights for policy 0, policy_version 380392 (0.0012) [2023-12-26 18:08:06,135][105620] Updated weights for policy 1, policy_version 381092 (0.0010) [2023-12-26 18:08:06,893][105620] Updated weights for policy 1, policy_version 381102 (0.0011) [2023-12-26 18:08:06,898][105692] Updated weights for policy 0, policy_version 380402 (0.0011) [2023-12-26 18:08:06,959][105620] Updated weights for policy 1, policy_version 381112 (0.0011) [2023-12-26 18:08:06,964][105692] Updated weights for policy 0, policy_version 380412 (0.0011) [2023-12-26 18:08:07,023][105620] Updated weights for policy 1, policy_version 381122 (0.0011) [2023-12-26 18:08:07,027][105692] Updated weights for policy 0, policy_version 380422 (0.0010) [2023-12-26 18:08:07,647][105692] Updated weights for policy 0, policy_version 380432 (0.0010) [2023-12-26 18:08:07,715][105692] Updated weights for policy 0, policy_version 380442 (0.0010) [2023-12-26 18:08:07,725][105620] Updated weights for policy 1, policy_version 381132 (0.0009) [2023-12-26 18:08:07,766][105692] Updated weights for policy 0, policy_version 380452 (0.0010) [2023-12-26 18:08:07,772][105620] Updated weights for policy 1, policy_version 381142 (0.0005) [2023-12-26 18:08:07,818][105620] Updated weights for policy 1, policy_version 381152 (0.0007) [2023-12-26 18:08:08,500][105620] Updated weights for policy 1, policy_version 381162 (0.0008) [2023-12-26 18:08:08,505][105692] Updated weights for policy 0, policy_version 380462 (0.0009) [2023-12-26 18:08:08,554][105620] Updated weights for policy 1, policy_version 381172 (0.0008) [2023-12-26 18:08:08,570][105692] Updated weights for policy 0, policy_version 380472 (0.0008) [2023-12-26 18:08:08,605][105620] Updated weights for policy 1, policy_version 381182 (0.0008) [2023-12-26 18:08:08,625][105692] Updated weights for policy 0, policy_version 380482 (0.0006) [2023-12-26 18:08:08,659][105620] Updated weights for policy 1, policy_version 381192 (0.0008) [2023-12-26 18:08:09,261][105620] Updated weights for policy 1, policy_version 381202 (0.0007) [2023-12-26 18:08:09,320][105620] Updated weights for policy 1, policy_version 381212 (0.0008) [2023-12-26 18:08:09,348][105692] Updated weights for policy 0, policy_version 380492 (0.0007) [2023-12-26 18:08:09,361][105586] KL-divergence is very high: 109.5842 [2023-12-26 18:08:09,389][105620] Updated weights for policy 1, policy_version 381222 (0.0008) [2023-12-26 18:08:09,421][105692] Updated weights for policy 0, policy_version 380502 (0.0008) [2023-12-26 18:08:09,485][105692] Updated weights for policy 0, policy_version 380512 (0.0008) [2023-12-26 18:08:10,132][105620] Updated weights for policy 1, policy_version 381232 (0.0009) [2023-12-26 18:08:10,139][105586] KL-divergence is very high: 231.8913 [2023-12-26 18:08:10,145][105586] KL-divergence is very high: 123.6951 [2023-12-26 18:08:10,151][105586] KL-divergence is very high: 295.9900 [2023-12-26 18:08:10,183][105586] KL-divergence is very high: 299.2892 [2023-12-26 18:08:10,189][105620] Updated weights for policy 1, policy_version 381242 (0.0008) [2023-12-26 18:08:10,189][105586] KL-divergence is very high: 168.5087 [2023-12-26 18:08:10,196][105586] KL-divergence is very high: 327.7930 [2023-12-26 18:08:10,230][105586] KL-divergence is very high: 247.4567 [2023-12-26 18:08:10,236][105586] KL-divergence is very high: 130.2359 [2023-12-26 18:08:10,243][105586] KL-divergence is very high: 263.8017 [2023-12-26 18:08:10,247][105620] Updated weights for policy 1, policy_version 381252 (0.0006) [2023-12-26 18:08:10,249][105692] Updated weights for policy 0, policy_version 380522 (0.0008) [2023-12-26 18:08:10,315][105692] Updated weights for policy 0, policy_version 380532 (0.0009) [2023-12-26 18:08:10,375][105692] Updated weights for policy 0, policy_version 380543 (0.0009) [2023-12-26 18:08:10,976][105620] Updated weights for policy 1, policy_version 381262 (0.0008) [2023-12-26 18:08:11,039][105620] Updated weights for policy 1, policy_version 381272 (0.0007) [2023-12-26 18:08:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 195043328. Throughput: 0: 9760.6, 1: 10047.8. Samples: 195057664. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 18:08:11,062][104569] Avg episode reward: [(0, '9014.688'), (1, '5948.639')] [2023-12-26 18:08:11,105][105620] Updated weights for policy 1, policy_version 381282 (0.0008) [2023-12-26 18:08:11,197][105692] Updated weights for policy 0, policy_version 380553 (0.0009) [2023-12-26 18:08:11,265][105692] Updated weights for policy 0, policy_version 380563 (0.0008) [2023-12-26 18:08:11,318][105692] Updated weights for policy 0, policy_version 380573 (0.0009) [2023-12-26 18:08:11,381][105692] Updated weights for policy 0, policy_version 380583 (0.0009) [2023-12-26 18:08:11,914][105620] Updated weights for policy 1, policy_version 381292 (0.0009) [2023-12-26 18:08:11,974][105620] Updated weights for policy 1, policy_version 381302 (0.0009) [2023-12-26 18:08:12,033][105620] Updated weights for policy 1, policy_version 381312 (0.0009) [2023-12-26 18:08:12,073][105692] Updated weights for policy 0, policy_version 380593 (0.0008) [2023-12-26 18:08:12,138][105692] Updated weights for policy 0, policy_version 380603 (0.0009) [2023-12-26 18:08:12,204][105692] Updated weights for policy 0, policy_version 380613 (0.0009) [2023-12-26 18:08:12,800][105620] Updated weights for policy 1, policy_version 381322 (0.0008) [2023-12-26 18:08:12,852][105620] Updated weights for policy 1, policy_version 381332 (0.0008) [2023-12-26 18:08:12,910][105620] Updated weights for policy 1, policy_version 381342 (0.0008) [2023-12-26 18:08:12,962][105692] Updated weights for policy 0, policy_version 380623 (0.0010) [2023-12-26 18:08:12,969][105620] Updated weights for policy 1, policy_version 381352 (0.0009) [2023-12-26 18:08:13,010][105692] Updated weights for policy 0, policy_version 380633 (0.0010) [2023-12-26 18:08:13,061][105692] Updated weights for policy 0, policy_version 380643 (0.0010) [2023-12-26 18:08:13,686][105620] Updated weights for policy 1, policy_version 381362 (0.0008) [2023-12-26 18:08:13,748][105620] Updated weights for policy 1, policy_version 381372 (0.0006) [2023-12-26 18:08:13,807][105620] Updated weights for policy 1, policy_version 381382 (0.0008) [2023-12-26 18:08:13,833][105692] Updated weights for policy 0, policy_version 380653 (0.0010) [2023-12-26 18:08:13,898][105692] Updated weights for policy 0, policy_version 380663 (0.0011) [2023-12-26 18:08:13,952][105692] Updated weights for policy 0, policy_version 380673 (0.0010) [2023-12-26 18:08:14,510][105620] Updated weights for policy 1, policy_version 381392 (0.0007) [2023-12-26 18:08:14,567][105620] Updated weights for policy 1, policy_version 381402 (0.0009) [2023-12-26 18:08:14,594][105692] Updated weights for policy 0, policy_version 380683 (0.0010) [2023-12-26 18:08:14,626][105620] Updated weights for policy 1, policy_version 381412 (0.0007) [2023-12-26 18:08:14,653][105692] Updated weights for policy 0, policy_version 380693 (0.0007) [2023-12-26 18:08:14,705][105692] Updated weights for policy 0, policy_version 380703 (0.0005) [2023-12-26 18:08:15,319][105620] Updated weights for policy 1, policy_version 381422 (0.0009) [2023-12-26 18:08:15,375][105620] Updated weights for policy 1, policy_version 381432 (0.0008) [2023-12-26 18:08:15,431][105620] Updated weights for policy 1, policy_version 381442 (0.0005) [2023-12-26 18:08:15,475][105692] Updated weights for policy 0, policy_version 380713 (0.0006) [2023-12-26 18:08:15,533][105692] Updated weights for policy 0, policy_version 380723 (0.0010) [2023-12-26 18:08:15,591][105692] Updated weights for policy 0, policy_version 380733 (0.0010) [2023-12-26 18:08:15,649][105692] Updated weights for policy 0, policy_version 380744 (0.0011) [2023-12-26 18:08:16,058][105620] Updated weights for policy 1, policy_version 381452 (0.0006) [2023-12-26 18:08:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 195141632. Throughput: 0: 9799.3, 1: 9914.9. Samples: 195113180. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 18:08:16,063][104569] Avg episode reward: [(0, '9359.160'), (1, '5946.353')] [2023-12-26 18:08:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000380744_97484800.pth... [2023-12-26 18:08:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000379624_97198080.pth [2023-12-26 18:08:16,121][105620] Updated weights for policy 1, policy_version 381462 (0.0008) [2023-12-26 18:08:16,188][105620] Updated weights for policy 1, policy_version 381472 (0.0010) [2023-12-26 18:08:16,228][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000381480_97665024.pth... [2023-12-26 18:08:16,231][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000380296_97361920.pth [2023-12-26 18:08:16,284][105692] Updated weights for policy 0, policy_version 380754 (0.0005) [2023-12-26 18:08:16,339][105692] Updated weights for policy 0, policy_version 380764 (0.0006) [2023-12-26 18:08:16,393][105692] Updated weights for policy 0, policy_version 380774 (0.0011) [2023-12-26 18:08:16,860][105620] Updated weights for policy 1, policy_version 381482 (0.0010) [2023-12-26 18:08:16,918][105620] Updated weights for policy 1, policy_version 381492 (0.0005) [2023-12-26 18:08:16,986][105620] Updated weights for policy 1, policy_version 381502 (0.0006) [2023-12-26 18:08:17,040][105692] Updated weights for policy 0, policy_version 380784 (0.0011) [2023-12-26 18:08:17,045][105620] Updated weights for policy 1, policy_version 381512 (0.0007) [2023-12-26 18:08:17,105][105692] Updated weights for policy 0, policy_version 380794 (0.0010) [2023-12-26 18:08:17,176][105692] Updated weights for policy 0, policy_version 380804 (0.0007) [2023-12-26 18:08:17,710][105620] Updated weights for policy 1, policy_version 381522 (0.0006) [2023-12-26 18:08:17,774][105620] Updated weights for policy 1, policy_version 381532 (0.0008) [2023-12-26 18:08:17,835][105620] Updated weights for policy 1, policy_version 381542 (0.0009) [2023-12-26 18:08:17,884][105692] Updated weights for policy 0, policy_version 380814 (0.0008) [2023-12-26 18:08:17,933][105692] Updated weights for policy 0, policy_version 380824 (0.0011) [2023-12-26 18:08:17,994][105692] Updated weights for policy 0, policy_version 380834 (0.0011) [2023-12-26 18:08:18,474][105620] Updated weights for policy 1, policy_version 381552 (0.0009) [2023-12-26 18:08:18,548][105620] Updated weights for policy 1, policy_version 381562 (0.0008) [2023-12-26 18:08:18,624][105620] Updated weights for policy 1, policy_version 381572 (0.0008) [2023-12-26 18:08:18,664][105692] Updated weights for policy 0, policy_version 380844 (0.0009) [2023-12-26 18:08:18,721][105692] Updated weights for policy 0, policy_version 380854 (0.0006) [2023-12-26 18:08:18,789][105692] Updated weights for policy 0, policy_version 380864 (0.0008) [2023-12-26 18:08:19,372][105620] Updated weights for policy 1, policy_version 381582 (0.0009) [2023-12-26 18:08:19,431][105620] Updated weights for policy 1, policy_version 381592 (0.0008) [2023-12-26 18:08:19,473][105692] Updated weights for policy 0, policy_version 380874 (0.0007) [2023-12-26 18:08:19,487][105620] Updated weights for policy 1, policy_version 381602 (0.0009) [2023-12-26 18:08:19,532][105692] Updated weights for policy 0, policy_version 380884 (0.0008) [2023-12-26 18:08:19,591][105692] Updated weights for policy 0, policy_version 380894 (0.0008) [2023-12-26 18:08:19,649][105692] Updated weights for policy 0, policy_version 380904 (0.0009) [2023-12-26 18:08:20,253][105620] Updated weights for policy 1, policy_version 381612 (0.0007) [2023-12-26 18:08:20,321][105620] Updated weights for policy 1, policy_version 381622 (0.0005) [2023-12-26 18:08:20,380][105620] Updated weights for policy 1, policy_version 381632 (0.0008) [2023-12-26 18:08:20,453][105692] Updated weights for policy 0, policy_version 380914 (0.0009) [2023-12-26 18:08:20,509][105692] Updated weights for policy 0, policy_version 380924 (0.0009) [2023-12-26 18:08:20,571][105692] Updated weights for policy 0, policy_version 380934 (0.0008) [2023-12-26 18:08:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 195239936. Throughput: 0: 9779.7, 1: 9933.4. Samples: 195233092. Policy #0 lag: (min: 28.0, avg: 29.8, max: 56.0) [2023-12-26 18:08:21,062][104569] Avg episode reward: [(0, '9359.168'), (1, '5404.212')] [2023-12-26 18:08:21,086][105620] Updated weights for policy 1, policy_version 381642 (0.0007) [2023-12-26 18:08:21,155][105620] Updated weights for policy 1, policy_version 381652 (0.0009) [2023-12-26 18:08:21,221][105620] Updated weights for policy 1, policy_version 381662 (0.0009) [2023-12-26 18:08:21,288][105620] Updated weights for policy 1, policy_version 381672 (0.0008) [2023-12-26 18:08:21,304][105692] Updated weights for policy 0, policy_version 380944 (0.0009) [2023-12-26 18:08:21,370][105692] Updated weights for policy 0, policy_version 380954 (0.0008) [2023-12-26 18:08:21,436][105692] Updated weights for policy 0, policy_version 380964 (0.0008) [2023-12-26 18:08:22,084][105620] Updated weights for policy 1, policy_version 381682 (0.0009) [2023-12-26 18:08:22,147][105620] Updated weights for policy 1, policy_version 381692 (0.0009) [2023-12-26 18:08:22,182][105692] Updated weights for policy 0, policy_version 380974 (0.0008) [2023-12-26 18:08:22,209][105620] Updated weights for policy 1, policy_version 381702 (0.0007) [2023-12-26 18:08:22,241][105692] Updated weights for policy 0, policy_version 380984 (0.0006) [2023-12-26 18:08:22,307][105692] Updated weights for policy 0, policy_version 380994 (0.0009) [2023-12-26 18:08:22,999][105620] Updated weights for policy 1, policy_version 381712 (0.0008) [2023-12-26 18:08:23,044][105692] Updated weights for policy 0, policy_version 381004 (0.0008) [2023-12-26 18:08:23,058][105620] Updated weights for policy 1, policy_version 381722 (0.0008) [2023-12-26 18:08:23,101][105692] Updated weights for policy 0, policy_version 381014 (0.0008) [2023-12-26 18:08:23,111][105620] Updated weights for policy 1, policy_version 381732 (0.0006) [2023-12-26 18:08:23,161][105692] Updated weights for policy 0, policy_version 381024 (0.0008) [2023-12-26 18:08:23,873][105620] Updated weights for policy 1, policy_version 381742 (0.0008) [2023-12-26 18:08:23,884][105692] Updated weights for policy 0, policy_version 381034 (0.0009) [2023-12-26 18:08:23,920][105620] Updated weights for policy 1, policy_version 381752 (0.0008) [2023-12-26 18:08:23,942][105692] Updated weights for policy 0, policy_version 381044 (0.0008) [2023-12-26 18:08:23,972][105620] Updated weights for policy 1, policy_version 381762 (0.0006) [2023-12-26 18:08:23,995][105692] Updated weights for policy 0, policy_version 381054 (0.0006) [2023-12-26 18:08:24,048][105692] Updated weights for policy 0, policy_version 381064 (0.0008) [2023-12-26 18:08:24,663][105620] Updated weights for policy 1, policy_version 381772 (0.0005) [2023-12-26 18:08:24,711][105620] Updated weights for policy 1, policy_version 381782 (0.0009) [2023-12-26 18:08:24,725][105586] KL-divergence is very high: 157.3435 [2023-12-26 18:08:24,757][105586] KL-divergence is very high: 117.8558 [2023-12-26 18:08:24,764][105620] Updated weights for policy 1, policy_version 381792 (0.0009) [2023-12-26 18:08:24,767][105586] KL-divergence is very high: 146.5522 [2023-12-26 18:08:24,774][105692] Updated weights for policy 0, policy_version 381074 (0.0005) [2023-12-26 18:08:24,793][105586] KL-divergence is very high: 103.1634 [2023-12-26 18:08:24,842][105692] Updated weights for policy 0, policy_version 381084 (0.0005) [2023-12-26 18:08:24,912][105692] Updated weights for policy 0, policy_version 381094 (0.0005) [2023-12-26 18:08:25,490][105692] Updated weights for policy 0, policy_version 381104 (0.0009) [2023-12-26 18:08:25,544][105692] Updated weights for policy 0, policy_version 381114 (0.0009) [2023-12-26 18:08:25,560][105620] Updated weights for policy 1, policy_version 381803 (0.0009) [2023-12-26 18:08:25,592][105692] Updated weights for policy 0, policy_version 381124 (0.0007) [2023-12-26 18:08:25,609][105620] Updated weights for policy 1, policy_version 381813 (0.0006) [2023-12-26 18:08:25,667][105620] Updated weights for policy 1, policy_version 381823 (0.0008) [2023-12-26 18:08:26,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 195338240. Throughput: 0: 9804.0, 1: 9859.8. Samples: 195346424. Policy #0 lag: (min: 28.0, avg: 29.8, max: 56.0) [2023-12-26 18:08:26,062][104569] Avg episode reward: [(0, '9359.206'), (1, '5224.768')] [2023-12-26 18:08:26,373][105620] Updated weights for policy 1, policy_version 381833 (0.0008) [2023-12-26 18:08:26,389][105692] Updated weights for policy 0, policy_version 381134 (0.0008) [2023-12-26 18:08:26,420][105620] Updated weights for policy 1, policy_version 381843 (0.0008) [2023-12-26 18:08:26,442][105692] Updated weights for policy 0, policy_version 381144 (0.0007) [2023-12-26 18:08:26,473][105620] Updated weights for policy 1, policy_version 381853 (0.0006) [2023-12-26 18:08:26,492][105692] Updated weights for policy 0, policy_version 381154 (0.0006) [2023-12-26 18:08:26,532][105620] Updated weights for policy 1, policy_version 381863 (0.0008) [2023-12-26 18:08:27,062][105692] Updated weights for policy 0, policy_version 381164 (0.0008) [2023-12-26 18:08:27,114][105692] Updated weights for policy 0, policy_version 381174 (0.0005) [2023-12-26 18:08:27,164][105692] Updated weights for policy 0, policy_version 381184 (0.0005) [2023-12-26 18:08:27,425][105620] Updated weights for policy 1, policy_version 381873 (0.0005) [2023-12-26 18:08:27,470][105620] Updated weights for policy 1, policy_version 381883 (0.0005) [2023-12-26 18:08:27,525][105620] Updated weights for policy 1, policy_version 381893 (0.0005) [2023-12-26 18:08:27,702][105692] Updated weights for policy 0, policy_version 381194 (0.0007) [2023-12-26 18:08:27,765][105692] Updated weights for policy 0, policy_version 381204 (0.0007) [2023-12-26 18:08:27,823][105692] Updated weights for policy 0, policy_version 381214 (0.0009) [2023-12-26 18:08:27,874][105692] Updated weights for policy 0, policy_version 381224 (0.0010) [2023-12-26 18:08:28,163][105620] Updated weights for policy 1, policy_version 381903 (0.0009) [2023-12-26 18:08:28,225][105620] Updated weights for policy 1, policy_version 381913 (0.0008) [2023-12-26 18:08:28,276][105620] Updated weights for policy 1, policy_version 381923 (0.0005) [2023-12-26 18:08:28,573][105692] Updated weights for policy 0, policy_version 381234 (0.0010) [2023-12-26 18:08:28,621][105692] Updated weights for policy 0, policy_version 381244 (0.0010) [2023-12-26 18:08:28,680][105692] Updated weights for policy 0, policy_version 381254 (0.0010) [2023-12-26 18:08:28,928][105620] Updated weights for policy 1, policy_version 381933 (0.0007) [2023-12-26 18:08:28,981][105620] Updated weights for policy 1, policy_version 381943 (0.0009) [2023-12-26 18:08:29,035][105620] Updated weights for policy 1, policy_version 381953 (0.0007) [2023-12-26 18:08:29,374][105692] Updated weights for policy 0, policy_version 381264 (0.0009) [2023-12-26 18:08:29,435][105692] Updated weights for policy 0, policy_version 381274 (0.0010) [2023-12-26 18:08:29,490][105692] Updated weights for policy 0, policy_version 381284 (0.0010) [2023-12-26 18:08:29,750][105620] Updated weights for policy 1, policy_version 381963 (0.0006) [2023-12-26 18:08:29,797][105620] Updated weights for policy 1, policy_version 381973 (0.0006) [2023-12-26 18:08:29,848][105586] KL-divergence is very high: 108.0125 [2023-12-26 18:08:29,860][105620] Updated weights for policy 1, policy_version 381983 (0.0008) [2023-12-26 18:08:29,865][105586] KL-divergence is very high: 149.1745 [2023-12-26 18:08:29,871][105586] KL-divergence is very high: 167.1882 [2023-12-26 18:08:29,878][105586] KL-divergence is very high: 164.4077 [2023-12-26 18:08:29,890][105586] KL-divergence is very high: 139.6143 [2023-12-26 18:08:29,897][105586] KL-divergence is very high: 198.3815 [2023-12-26 18:08:30,258][105692] Updated weights for policy 0, policy_version 381295 (0.0009) [2023-12-26 18:08:30,308][105692] Updated weights for policy 0, policy_version 381305 (0.0009) [2023-12-26 18:08:30,361][105692] Updated weights for policy 0, policy_version 381315 (0.0009) [2023-12-26 18:08:30,468][105620] Updated weights for policy 1, policy_version 381993 (0.0007) [2023-12-26 18:08:30,537][105620] Updated weights for policy 1, policy_version 382003 (0.0006) [2023-12-26 18:08:30,596][105620] Updated weights for policy 1, policy_version 382013 (0.0005) [2023-12-26 18:08:30,647][105620] Updated weights for policy 1, policy_version 382024 (0.0007) [2023-12-26 18:08:30,999][105692] Updated weights for policy 0, policy_version 381325 (0.0009) [2023-12-26 18:08:31,060][105692] Updated weights for policy 0, policy_version 381335 (0.0006) [2023-12-26 18:08:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 195436544. Throughput: 0: 9849.3, 1: 9869.6. Samples: 195407996. Policy #0 lag: (min: 28.0, avg: 29.8, max: 56.0) [2023-12-26 18:08:31,063][104569] Avg episode reward: [(0, '9359.248'), (1, '5868.639')] [2023-12-26 18:08:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000382024_97804288.pth... [2023-12-26 18:08:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000380904_97517568.pth [2023-12-26 18:08:31,127][105692] Updated weights for policy 0, policy_version 381345 (0.0010) [2023-12-26 18:08:31,175][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000381352_97640448.pth... [2023-12-26 18:08:31,180][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000380200_97345536.pth [2023-12-26 18:08:31,274][105620] Updated weights for policy 1, policy_version 382034 (0.0009) [2023-12-26 18:08:31,337][105620] Updated weights for policy 1, policy_version 382044 (0.0008) [2023-12-26 18:08:31,403][105620] Updated weights for policy 1, policy_version 382054 (0.0009) [2023-12-26 18:08:31,835][105692] Updated weights for policy 0, policy_version 381355 (0.0010) [2023-12-26 18:08:31,891][105692] Updated weights for policy 0, policy_version 381365 (0.0010) [2023-12-26 18:08:31,949][105692] Updated weights for policy 0, policy_version 381375 (0.0010) [2023-12-26 18:08:31,996][105620] Updated weights for policy 1, policy_version 382064 (0.0008) [2023-12-26 18:08:32,051][105620] Updated weights for policy 1, policy_version 382074 (0.0008) [2023-12-26 18:08:32,113][105620] Updated weights for policy 1, policy_version 382084 (0.0011) [2023-12-26 18:08:32,666][105692] Updated weights for policy 0, policy_version 381385 (0.0010) [2023-12-26 18:08:32,725][105692] Updated weights for policy 0, policy_version 381395 (0.0011) [2023-12-26 18:08:32,765][105620] Updated weights for policy 1, policy_version 382094 (0.0008) [2023-12-26 18:08:32,790][105692] Updated weights for policy 0, policy_version 381405 (0.0009) [2023-12-26 18:08:32,832][105620] Updated weights for policy 1, policy_version 382104 (0.0005) [2023-12-26 18:08:32,850][105692] Updated weights for policy 0, policy_version 381415 (0.0008) [2023-12-26 18:08:32,892][105620] Updated weights for policy 1, policy_version 382114 (0.0008) [2023-12-26 18:08:33,400][105692] Updated weights for policy 0, policy_version 381425 (0.0008) [2023-12-26 18:08:33,451][105692] Updated weights for policy 0, policy_version 381435 (0.0010) [2023-12-26 18:08:33,498][105692] Updated weights for policy 0, policy_version 381445 (0.0010) [2023-12-26 18:08:33,572][105620] Updated weights for policy 1, policy_version 382124 (0.0008) [2023-12-26 18:08:33,634][105620] Updated weights for policy 1, policy_version 382134 (0.0005) [2023-12-26 18:08:33,701][105620] Updated weights for policy 1, policy_version 382144 (0.0007) [2023-12-26 18:08:34,249][105692] Updated weights for policy 0, policy_version 381455 (0.0010) [2023-12-26 18:08:34,305][105692] Updated weights for policy 0, policy_version 381465 (0.0010) [2023-12-26 18:08:34,325][105620] Updated weights for policy 1, policy_version 382154 (0.0009) [2023-12-26 18:08:34,361][105692] Updated weights for policy 0, policy_version 381475 (0.0011) [2023-12-26 18:08:34,379][105620] Updated weights for policy 1, policy_version 382164 (0.0008) [2023-12-26 18:08:34,446][105620] Updated weights for policy 1, policy_version 382174 (0.0008) [2023-12-26 18:08:34,508][105620] Updated weights for policy 1, policy_version 382184 (0.0008) [2023-12-26 18:08:35,052][105692] Updated weights for policy 0, policy_version 381485 (0.0010) [2023-12-26 18:08:35,099][105692] Updated weights for policy 0, policy_version 381495 (0.0010) [2023-12-26 18:08:35,144][105692] Updated weights for policy 0, policy_version 381505 (0.0010) [2023-12-26 18:08:35,278][105620] Updated weights for policy 1, policy_version 382194 (0.0008) [2023-12-26 18:08:35,334][105620] Updated weights for policy 1, policy_version 382204 (0.0008) [2023-12-26 18:08:35,383][105620] Updated weights for policy 1, policy_version 382214 (0.0008) [2023-12-26 18:08:35,882][105692] Updated weights for policy 0, policy_version 381515 (0.0009) [2023-12-26 18:08:35,937][105692] Updated weights for policy 0, policy_version 381525 (0.0006) [2023-12-26 18:08:35,997][105692] Updated weights for policy 0, policy_version 381535 (0.0006) [2023-12-26 18:08:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 195543040. Throughput: 0: 9898.4, 1: 9914.8. Samples: 195530012. Policy #0 lag: (min: 28.0, avg: 29.8, max: 56.0) [2023-12-26 18:08:36,062][104569] Avg episode reward: [(0, '9359.433'), (1, '6518.829')] [2023-12-26 18:08:36,177][105620] Updated weights for policy 1, policy_version 382224 (0.0010) [2023-12-26 18:08:36,237][105620] Updated weights for policy 1, policy_version 382234 (0.0011) [2023-12-26 18:08:36,297][105620] Updated weights for policy 1, policy_version 382244 (0.0011) [2023-12-26 18:08:36,715][105692] Updated weights for policy 0, policy_version 381545 (0.0011) [2023-12-26 18:08:36,773][105692] Updated weights for policy 0, policy_version 381555 (0.0010) [2023-12-26 18:08:36,832][105692] Updated weights for policy 0, policy_version 381565 (0.0010) [2023-12-26 18:08:36,891][105692] Updated weights for policy 0, policy_version 381575 (0.0011) [2023-12-26 18:08:37,049][105620] Updated weights for policy 1, policy_version 382254 (0.0008) [2023-12-26 18:08:37,106][105620] Updated weights for policy 1, policy_version 382264 (0.0011) [2023-12-26 18:08:37,165][105620] Updated weights for policy 1, policy_version 382274 (0.0011) [2023-12-26 18:08:37,558][105692] Updated weights for policy 0, policy_version 381585 (0.0010) [2023-12-26 18:08:37,617][105692] Updated weights for policy 0, policy_version 381595 (0.0010) [2023-12-26 18:08:37,677][105692] Updated weights for policy 0, policy_version 381605 (0.0010) [2023-12-26 18:08:37,865][105620] Updated weights for policy 1, policy_version 382284 (0.0011) [2023-12-26 18:08:37,925][105620] Updated weights for policy 1, policy_version 382294 (0.0010) [2023-12-26 18:08:37,989][105620] Updated weights for policy 1, policy_version 382304 (0.0006) [2023-12-26 18:08:38,424][105692] Updated weights for policy 0, policy_version 381615 (0.0011) [2023-12-26 18:08:38,475][105692] Updated weights for policy 0, policy_version 381625 (0.0010) [2023-12-26 18:08:38,530][105692] Updated weights for policy 0, policy_version 381635 (0.0010) [2023-12-26 18:08:38,654][105620] Updated weights for policy 1, policy_version 382314 (0.0006) [2023-12-26 18:08:38,713][105620] Updated weights for policy 1, policy_version 382324 (0.0011) [2023-12-26 18:08:38,765][105620] Updated weights for policy 1, policy_version 382334 (0.0010) [2023-12-26 18:08:38,824][105620] Updated weights for policy 1, policy_version 382344 (0.0010) [2023-12-26 18:08:39,244][105692] Updated weights for policy 0, policy_version 381645 (0.0010) [2023-12-26 18:08:39,304][105692] Updated weights for policy 0, policy_version 381655 (0.0009) [2023-12-26 18:08:39,357][105692] Updated weights for policy 0, policy_version 381665 (0.0008) [2023-12-26 18:08:39,571][105620] Updated weights for policy 1, policy_version 382354 (0.0009) [2023-12-26 18:08:39,619][105620] Updated weights for policy 1, policy_version 382364 (0.0009) [2023-12-26 18:08:39,674][105620] Updated weights for policy 1, policy_version 382374 (0.0009) [2023-12-26 18:08:40,216][105692] Updated weights for policy 0, policy_version 381675 (0.0008) [2023-12-26 18:08:40,271][105692] Updated weights for policy 0, policy_version 381685 (0.0009) [2023-12-26 18:08:40,327][105692] Updated weights for policy 0, policy_version 381695 (0.0009) [2023-12-26 18:08:40,393][105620] Updated weights for policy 1, policy_version 382384 (0.0008) [2023-12-26 18:08:40,457][105620] Updated weights for policy 1, policy_version 382394 (0.0008) [2023-12-26 18:08:40,517][105620] Updated weights for policy 1, policy_version 382404 (0.0009) [2023-12-26 18:08:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 195633152. Throughput: 0: 9842.3, 1: 9889.2. Samples: 195644624. Policy #0 lag: (min: 28.0, avg: 29.8, max: 56.0) [2023-12-26 18:08:41,062][104569] Avg episode reward: [(0, '9359.433'), (1, '6427.836')] [2023-12-26 18:08:41,108][105692] Updated weights for policy 0, policy_version 381705 (0.0008) [2023-12-26 18:08:41,170][105692] Updated weights for policy 0, policy_version 381715 (0.0010) [2023-12-26 18:08:41,225][105692] Updated weights for policy 0, policy_version 381725 (0.0008) [2023-12-26 18:08:41,259][105620] Updated weights for policy 1, policy_version 382414 (0.0008) [2023-12-26 18:08:41,286][105692] Updated weights for policy 0, policy_version 381735 (0.0006) [2023-12-26 18:08:41,318][105620] Updated weights for policy 1, policy_version 382424 (0.0008) [2023-12-26 18:08:41,387][105620] Updated weights for policy 1, policy_version 382434 (0.0009) [2023-12-26 18:08:42,071][105620] Updated weights for policy 1, policy_version 382444 (0.0008) [2023-12-26 18:08:42,133][105620] Updated weights for policy 1, policy_version 382454 (0.0008) [2023-12-26 18:08:42,134][105586] KL-divergence is very high: 141.2139 [2023-12-26 18:08:42,141][105586] KL-divergence is very high: 137.6124 [2023-12-26 18:08:42,148][105692] Updated weights for policy 0, policy_version 381745 (0.0008) [2023-12-26 18:08:42,157][105586] KL-divergence is very high: 109.4835 [2023-12-26 18:08:42,181][105586] KL-divergence is very high: 305.2622 [2023-12-26 18:08:42,188][105586] KL-divergence is very high: 266.8765 [2023-12-26 18:08:42,195][105620] Updated weights for policy 1, policy_version 382464 (0.0006) [2023-12-26 18:08:42,207][105586] KL-divergence is very high: 156.7960 [2023-12-26 18:08:42,209][105692] Updated weights for policy 0, policy_version 381755 (0.0009) [2023-12-26 18:08:42,230][105586] KL-divergence is very high: 294.7348 [2023-12-26 18:08:42,237][105586] KL-divergence is very high: 254.5967 [2023-12-26 18:08:42,275][105692] Updated weights for policy 0, policy_version 381765 (0.0009) [2023-12-26 18:08:42,917][105620] Updated weights for policy 1, policy_version 382474 (0.0006) [2023-12-26 18:08:42,968][105620] Updated weights for policy 1, policy_version 382484 (0.0005) [2023-12-26 18:08:42,975][105586] KL-divergence is very high: 497.7686 [2023-12-26 18:08:43,014][105692] Updated weights for policy 0, policy_version 381775 (0.0008) [2023-12-26 18:08:43,015][105586] KL-divergence is very high: 864.8419 [2023-12-26 18:08:43,020][105620] Updated weights for policy 1, policy_version 382494 (0.0007) [2023-12-26 18:08:43,053][105586] KL-divergence is very high: 928.8755 [2023-12-26 18:08:43,068][105620] Updated weights for policy 1, policy_version 382504 (0.0006) [2023-12-26 18:08:43,071][105692] Updated weights for policy 0, policy_version 381785 (0.0008) [2023-12-26 18:08:43,129][105692] Updated weights for policy 0, policy_version 381795 (0.0009) [2023-12-26 18:08:43,750][105692] Updated weights for policy 0, policy_version 381805 (0.0010) [2023-12-26 18:08:43,798][105692] Updated weights for policy 0, policy_version 381815 (0.0008) [2023-12-26 18:08:43,837][105586] KL-divergence is very high: 573.2156 [2023-12-26 18:08:43,855][105692] Updated weights for policy 0, policy_version 381825 (0.0008) [2023-12-26 18:08:43,861][105620] Updated weights for policy 1, policy_version 382514 (0.0008) [2023-12-26 18:08:43,883][105586] KL-divergence is very high: 576.1111 [2023-12-26 18:08:43,913][105620] Updated weights for policy 1, policy_version 382524 (0.0008) [2023-12-26 18:08:43,926][105586] KL-divergence is very high: 496.7583 [2023-12-26 18:08:43,973][105586] KL-divergence is very high: 466.5366 [2023-12-26 18:08:43,974][105620] Updated weights for policy 1, policy_version 382534 (0.0008) [2023-12-26 18:08:44,447][105692] Updated weights for policy 0, policy_version 381835 (0.0007) [2023-12-26 18:08:44,512][105692] Updated weights for policy 0, policy_version 381845 (0.0008) [2023-12-26 18:08:44,575][105692] Updated weights for policy 0, policy_version 381855 (0.0009) [2023-12-26 18:08:44,823][105620] Updated weights for policy 1, policy_version 382544 (0.0009) [2023-12-26 18:08:44,880][105620] Updated weights for policy 1, policy_version 382554 (0.0009) [2023-12-26 18:08:44,933][105620] Updated weights for policy 1, policy_version 382565 (0.0010) [2023-12-26 18:08:45,153][105692] Updated weights for policy 0, policy_version 381865 (0.0007) [2023-12-26 18:08:45,204][105692] Updated weights for policy 0, policy_version 381875 (0.0009) [2023-12-26 18:08:45,252][105692] Updated weights for policy 0, policy_version 381885 (0.0007) [2023-12-26 18:08:45,299][105692] Updated weights for policy 0, policy_version 381895 (0.0006) [2023-12-26 18:08:45,805][105620] Updated weights for policy 1, policy_version 382575 (0.0008) [2023-12-26 18:08:45,852][105620] Updated weights for policy 1, policy_version 382585 (0.0005) [2023-12-26 18:08:45,860][105692] Updated weights for policy 0, policy_version 381905 (0.0007) [2023-12-26 18:08:45,899][105620] Updated weights for policy 1, policy_version 382595 (0.0005) [2023-12-26 18:08:45,910][105692] Updated weights for policy 0, policy_version 381915 (0.0008) [2023-12-26 18:08:45,968][105692] Updated weights for policy 0, policy_version 381925 (0.0009) [2023-12-26 18:08:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 195739648. Throughput: 0: 9687.6, 1: 9816.9. Samples: 195700912. Policy #0 lag: (min: 28.0, avg: 29.8, max: 56.0) [2023-12-26 18:08:46,063][104569] Avg episode reward: [(0, '9266.923'), (1, '6723.426')] [2023-12-26 18:08:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000381928_97787904.pth... [2023-12-26 18:08:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000382600_97951744.pth... [2023-12-26 18:08:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000381480_97665024.pth [2023-12-26 18:08:46,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000380744_97484800.pth [2023-12-26 18:08:46,452][105620] Updated weights for policy 1, policy_version 382605 (0.0005) [2023-12-26 18:08:46,508][105620] Updated weights for policy 1, policy_version 382615 (0.0005) [2023-12-26 18:08:46,568][105620] Updated weights for policy 1, policy_version 382625 (0.0005) [2023-12-26 18:08:46,657][105692] Updated weights for policy 0, policy_version 381935 (0.0007) [2023-12-26 18:08:46,707][105692] Updated weights for policy 0, policy_version 381945 (0.0005) [2023-12-26 18:08:46,768][105692] Updated weights for policy 0, policy_version 381955 (0.0005) [2023-12-26 18:08:47,066][105620] Updated weights for policy 1, policy_version 382635 (0.0006) [2023-12-26 18:08:47,127][105620] Updated weights for policy 1, policy_version 382645 (0.0005) [2023-12-26 18:08:47,180][105620] Updated weights for policy 1, policy_version 382655 (0.0005) [2023-12-26 18:08:47,295][105692] Updated weights for policy 0, policy_version 381965 (0.0007) [2023-12-26 18:08:47,349][105692] Updated weights for policy 0, policy_version 381975 (0.0005) [2023-12-26 18:08:47,420][105692] Updated weights for policy 0, policy_version 381985 (0.0007) [2023-12-26 18:08:47,680][105620] Updated weights for policy 1, policy_version 382665 (0.0005) [2023-12-26 18:08:47,751][105620] Updated weights for policy 1, policy_version 382675 (0.0006) [2023-12-26 18:08:47,812][105620] Updated weights for policy 1, policy_version 382685 (0.0010) [2023-12-26 18:08:47,858][105620] Updated weights for policy 1, policy_version 382695 (0.0007) [2023-12-26 18:08:48,036][105692] Updated weights for policy 0, policy_version 381995 (0.0007) [2023-12-26 18:08:48,083][105692] Updated weights for policy 0, policy_version 382005 (0.0005) [2023-12-26 18:08:48,139][105692] Updated weights for policy 0, policy_version 382015 (0.0009) [2023-12-26 18:08:48,528][105620] Updated weights for policy 1, policy_version 382705 (0.0010) [2023-12-26 18:08:48,536][105586] KL-divergence is very high: 144.2111 [2023-12-26 18:08:48,593][105586] KL-divergence is very high: 233.1981 [2023-12-26 18:08:48,600][105620] Updated weights for policy 1, policy_version 382715 (0.0011) [2023-12-26 18:08:48,646][105586] KL-divergence is very high: 221.1388 [2023-12-26 18:08:48,666][105620] Updated weights for policy 1, policy_version 382725 (0.0011) [2023-12-26 18:08:48,826][105692] Updated weights for policy 0, policy_version 382025 (0.0006) [2023-12-26 18:08:48,881][105692] Updated weights for policy 0, policy_version 382035 (0.0010) [2023-12-26 18:08:48,938][105692] Updated weights for policy 0, policy_version 382047 (0.0011) [2023-12-26 18:08:49,223][105620] Updated weights for policy 1, policy_version 382735 (0.0009) [2023-12-26 18:08:49,289][105620] Updated weights for policy 1, policy_version 382745 (0.0007) [2023-12-26 18:08:49,353][105620] Updated weights for policy 1, policy_version 382755 (0.0010) [2023-12-26 18:08:49,794][105692] Updated weights for policy 0, policy_version 382057 (0.0008) [2023-12-26 18:08:49,861][105692] Updated weights for policy 0, policy_version 382067 (0.0009) [2023-12-26 18:08:49,923][105692] Updated weights for policy 0, policy_version 382077 (0.0009) [2023-12-26 18:08:49,986][105692] Updated weights for policy 0, policy_version 382087 (0.0010) [2023-12-26 18:08:50,101][105620] Updated weights for policy 1, policy_version 382765 (0.0010) [2023-12-26 18:08:50,152][105620] Updated weights for policy 1, policy_version 382775 (0.0009) [2023-12-26 18:08:50,211][105620] Updated weights for policy 1, policy_version 382785 (0.0009) [2023-12-26 18:08:50,671][105692] Updated weights for policy 0, policy_version 382097 (0.0006) [2023-12-26 18:08:50,728][105692] Updated weights for policy 0, policy_version 382107 (0.0005) [2023-12-26 18:08:50,786][105692] Updated weights for policy 0, policy_version 382117 (0.0005) [2023-12-26 18:08:50,977][105620] Updated weights for policy 1, policy_version 382795 (0.0008) [2023-12-26 18:08:51,036][105620] Updated weights for policy 1, policy_version 382805 (0.0006) [2023-12-26 18:08:51,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 195837952. Throughput: 0: 9881.3, 1: 9826.7. Samples: 195829032. Policy #0 lag: (min: 28.0, avg: 29.8, max: 56.0) [2023-12-26 18:08:51,063][104569] Avg episode reward: [(0, '9266.938'), (1, '6756.178')] [2023-12-26 18:08:51,090][105620] Updated weights for policy 1, policy_version 382815 (0.0007) [2023-12-26 18:08:51,525][105692] Updated weights for policy 0, policy_version 382127 (0.0009) [2023-12-26 18:08:51,583][105692] Updated weights for policy 0, policy_version 382137 (0.0008) [2023-12-26 18:08:51,648][105692] Updated weights for policy 0, policy_version 382147 (0.0008) [2023-12-26 18:08:51,821][105620] Updated weights for policy 1, policy_version 382825 (0.0009) [2023-12-26 18:08:51,895][105620] Updated weights for policy 1, policy_version 382835 (0.0010) [2023-12-26 18:08:51,965][105620] Updated weights for policy 1, policy_version 382845 (0.0009) [2023-12-26 18:08:52,030][105620] Updated weights for policy 1, policy_version 382855 (0.0008) [2023-12-26 18:08:52,404][105692] Updated weights for policy 0, policy_version 382157 (0.0009) [2023-12-26 18:08:52,452][105692] Updated weights for policy 0, policy_version 382167 (0.0009) [2023-12-26 18:08:52,511][105692] Updated weights for policy 0, policy_version 382177 (0.0009) [2023-12-26 18:08:52,724][105620] Updated weights for policy 1, policy_version 382865 (0.0009) [2023-12-26 18:08:52,775][105620] Updated weights for policy 1, policy_version 382875 (0.0009) [2023-12-26 18:08:52,830][105620] Updated weights for policy 1, policy_version 382885 (0.0009) [2023-12-26 18:08:53,285][105692] Updated weights for policy 0, policy_version 382187 (0.0009) [2023-12-26 18:08:53,336][105692] Updated weights for policy 0, policy_version 382197 (0.0009) [2023-12-26 18:08:53,386][105692] Updated weights for policy 0, policy_version 382207 (0.0009) [2023-12-26 18:08:53,567][105620] Updated weights for policy 1, policy_version 382895 (0.0008) [2023-12-26 18:08:53,626][105620] Updated weights for policy 1, policy_version 382905 (0.0010) [2023-12-26 18:08:53,672][105620] Updated weights for policy 1, policy_version 382915 (0.0008) [2023-12-26 18:08:54,148][105692] Updated weights for policy 0, policy_version 382217 (0.0009) [2023-12-26 18:08:54,209][105692] Updated weights for policy 0, policy_version 382227 (0.0009) [2023-12-26 18:08:54,263][105692] Updated weights for policy 0, policy_version 382237 (0.0009) [2023-12-26 18:08:54,320][105692] Updated weights for policy 0, policy_version 382247 (0.0009) [2023-12-26 18:08:54,431][105620] Updated weights for policy 1, policy_version 382925 (0.0007) [2023-12-26 18:08:54,485][105620] Updated weights for policy 1, policy_version 382935 (0.0005) [2023-12-26 18:08:54,538][105620] Updated weights for policy 1, policy_version 382945 (0.0006) [2023-12-26 18:08:55,111][105692] Updated weights for policy 0, policy_version 382257 (0.0010) [2023-12-26 18:08:55,163][105692] Updated weights for policy 0, policy_version 382267 (0.0009) [2023-12-26 18:08:55,165][105585] KL-divergence is very high: 114.4947 [2023-12-26 18:08:55,191][105620] Updated weights for policy 1, policy_version 382955 (0.0005) [2023-12-26 18:08:55,209][105692] Updated weights for policy 0, policy_version 382277 (0.0008) [2023-12-26 18:08:55,234][105620] Updated weights for policy 1, policy_version 382965 (0.0005) [2023-12-26 18:08:55,290][105620] Updated weights for policy 1, policy_version 382975 (0.0005) [2023-12-26 18:08:55,955][105620] Updated weights for policy 1, policy_version 382985 (0.0006) [2023-12-26 18:08:56,001][105620] Updated weights for policy 1, policy_version 382995 (0.0008) [2023-12-26 18:08:56,023][105692] Updated weights for policy 0, policy_version 382287 (0.0007) [2023-12-26 18:08:56,061][105620] Updated weights for policy 1, policy_version 383005 (0.0006) [2023-12-26 18:08:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 195928064. Throughput: 0: 9858.7, 1: 9833.0. Samples: 195943796. Policy #0 lag: (min: 28.0, avg: 29.8, max: 56.0) [2023-12-26 18:08:56,062][104569] Avg episode reward: [(0, '9268.849'), (1, '6542.205')] [2023-12-26 18:08:56,084][105692] Updated weights for policy 0, policy_version 382297 (0.0007) [2023-12-26 18:08:56,119][105620] Updated weights for policy 1, policy_version 383015 (0.0008) [2023-12-26 18:08:56,146][105692] Updated weights for policy 0, policy_version 382307 (0.0007) [2023-12-26 18:08:56,870][105620] Updated weights for policy 1, policy_version 383025 (0.0006) [2023-12-26 18:08:56,896][105692] Updated weights for policy 0, policy_version 382317 (0.0008) [2023-12-26 18:08:56,914][105620] Updated weights for policy 1, policy_version 383035 (0.0006) [2023-12-26 18:08:56,955][105692] Updated weights for policy 0, policy_version 382327 (0.0007) [2023-12-26 18:08:56,960][105620] Updated weights for policy 1, policy_version 383045 (0.0009) [2023-12-26 18:08:57,007][105692] Updated weights for policy 0, policy_version 382337 (0.0008) [2023-12-26 18:08:57,714][105620] Updated weights for policy 1, policy_version 383055 (0.0009) [2023-12-26 18:08:57,754][105692] Updated weights for policy 0, policy_version 382347 (0.0008) [2023-12-26 18:08:57,761][105620] Updated weights for policy 1, policy_version 383065 (0.0007) [2023-12-26 18:08:57,799][105692] Updated weights for policy 0, policy_version 382357 (0.0007) [2023-12-26 18:08:57,809][105620] Updated weights for policy 1, policy_version 383075 (0.0006) [2023-12-26 18:08:57,853][105692] Updated weights for policy 0, policy_version 382367 (0.0007) [2023-12-26 18:08:58,559][105620] Updated weights for policy 1, policy_version 383085 (0.0008) [2023-12-26 18:08:58,621][105620] Updated weights for policy 1, policy_version 383095 (0.0009) [2023-12-26 18:08:58,642][105692] Updated weights for policy 0, policy_version 382377 (0.0008) [2023-12-26 18:08:58,681][105620] Updated weights for policy 1, policy_version 383105 (0.0009) [2023-12-26 18:08:58,704][105692] Updated weights for policy 0, policy_version 382387 (0.0008) [2023-12-26 18:08:58,778][105692] Updated weights for policy 0, policy_version 382397 (0.0007) [2023-12-26 18:08:58,848][105692] Updated weights for policy 0, policy_version 382407 (0.0008) [2023-12-26 18:08:59,478][105620] Updated weights for policy 1, policy_version 383115 (0.0009) [2023-12-26 18:08:59,537][105620] Updated weights for policy 1, policy_version 383125 (0.0008) [2023-12-26 18:08:59,595][105620] Updated weights for policy 1, policy_version 383135 (0.0007) [2023-12-26 18:08:59,613][105692] Updated weights for policy 0, policy_version 382417 (0.0006) [2023-12-26 18:08:59,667][105692] Updated weights for policy 0, policy_version 382427 (0.0008) [2023-12-26 18:08:59,722][105692] Updated weights for policy 0, policy_version 382437 (0.0007) [2023-12-26 18:09:00,290][105620] Updated weights for policy 1, policy_version 383145 (0.0007) [2023-12-26 18:09:00,354][105620] Updated weights for policy 1, policy_version 383155 (0.0009) [2023-12-26 18:09:00,402][105620] Updated weights for policy 1, policy_version 383165 (0.0008) [2023-12-26 18:09:00,459][105620] Updated weights for policy 1, policy_version 383175 (0.0009) [2023-12-26 18:09:00,461][105692] Updated weights for policy 0, policy_version 382447 (0.0006) [2023-12-26 18:09:00,518][105692] Updated weights for policy 0, policy_version 382457 (0.0005) [2023-12-26 18:09:00,577][105692] Updated weights for policy 0, policy_version 382467 (0.0005) [2023-12-26 18:09:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 196026368. Throughput: 0: 9853.0, 1: 9842.7. Samples: 195999484. Policy #0 lag: (min: 28.0, avg: 29.8, max: 56.0) [2023-12-26 18:09:01,062][104569] Avg episode reward: [(0, '9268.846'), (1, '6372.697')] [2023-12-26 18:09:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000382472_97927168.pth... [2023-12-26 18:09:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000383176_98099200.pth... [2023-12-26 18:09:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000381352_97640448.pth [2023-12-26 18:09:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000382024_97804288.pth [2023-12-26 18:09:01,224][105620] Updated weights for policy 1, policy_version 383185 (0.0007) [2023-12-26 18:09:01,240][105692] Updated weights for policy 0, policy_version 382477 (0.0005) [2023-12-26 18:09:01,280][105620] Updated weights for policy 1, policy_version 383195 (0.0008) [2023-12-26 18:09:01,300][105692] Updated weights for policy 0, policy_version 382487 (0.0006) [2023-12-26 18:09:01,342][105620] Updated weights for policy 1, policy_version 383205 (0.0007) [2023-12-26 18:09:01,358][105692] Updated weights for policy 0, policy_version 382497 (0.0007) [2023-12-26 18:09:01,384][105585] KL-divergence is very high: 101.6845 [2023-12-26 18:09:02,042][105692] Updated weights for policy 0, policy_version 382507 (0.0007) [2023-12-26 18:09:02,103][105692] Updated weights for policy 0, policy_version 382517 (0.0009) [2023-12-26 18:09:02,128][105620] Updated weights for policy 1, policy_version 383215 (0.0008) [2023-12-26 18:09:02,163][105692] Updated weights for policy 0, policy_version 382527 (0.0007) [2023-12-26 18:09:02,185][105620] Updated weights for policy 1, policy_version 383225 (0.0008) [2023-12-26 18:09:02,233][105620] Updated weights for policy 1, policy_version 383235 (0.0006) [2023-12-26 18:09:02,818][105692] Updated weights for policy 0, policy_version 382537 (0.0010) [2023-12-26 18:09:02,883][105692] Updated weights for policy 0, policy_version 382547 (0.0009) [2023-12-26 18:09:02,945][105692] Updated weights for policy 0, policy_version 382557 (0.0010) [2023-12-26 18:09:02,996][105692] Updated weights for policy 0, policy_version 382567 (0.0010) [2023-12-26 18:09:03,050][105620] Updated weights for policy 1, policy_version 383245 (0.0009) [2023-12-26 18:09:03,097][105620] Updated weights for policy 1, policy_version 383255 (0.0010) [2023-12-26 18:09:03,166][105620] Updated weights for policy 1, policy_version 383265 (0.0010) [2023-12-26 18:09:03,591][105692] Updated weights for policy 0, policy_version 382577 (0.0006) [2023-12-26 18:09:03,641][105692] Updated weights for policy 0, policy_version 382587 (0.0005) [2023-12-26 18:09:03,685][105692] Updated weights for policy 0, policy_version 382597 (0.0007) [2023-12-26 18:09:03,838][105620] Updated weights for policy 1, policy_version 383275 (0.0010) [2023-12-26 18:09:03,901][105620] Updated weights for policy 1, policy_version 383285 (0.0009) [2023-12-26 18:09:03,964][105620] Updated weights for policy 1, policy_version 383295 (0.0009) [2023-12-26 18:09:04,263][105692] Updated weights for policy 0, policy_version 382607 (0.0011) [2023-12-26 18:09:04,324][105692] Updated weights for policy 0, policy_version 382617 (0.0011) [2023-12-26 18:09:04,388][105692] Updated weights for policy 0, policy_version 382627 (0.0010) [2023-12-26 18:09:04,597][105620] Updated weights for policy 1, policy_version 383305 (0.0006) [2023-12-26 18:09:04,648][105620] Updated weights for policy 1, policy_version 383315 (0.0006) [2023-12-26 18:09:04,703][105620] Updated weights for policy 1, policy_version 383325 (0.0010) [2023-12-26 18:09:04,765][105620] Updated weights for policy 1, policy_version 383335 (0.0010) [2023-12-26 18:09:05,056][105692] Updated weights for policy 0, policy_version 382637 (0.0008) [2023-12-26 18:09:05,121][105692] Updated weights for policy 0, policy_version 382647 (0.0006) [2023-12-26 18:09:05,185][105692] Updated weights for policy 0, policy_version 382657 (0.0005) [2023-12-26 18:09:05,363][105620] Updated weights for policy 1, policy_version 383345 (0.0010) [2023-12-26 18:09:05,416][105620] Updated weights for policy 1, policy_version 383355 (0.0009) [2023-12-26 18:09:05,467][105620] Updated weights for policy 1, policy_version 383365 (0.0010) [2023-12-26 18:09:05,763][105692] Updated weights for policy 0, policy_version 382667 (0.0006) [2023-12-26 18:09:05,818][105692] Updated weights for policy 0, policy_version 382677 (0.0008) [2023-12-26 18:09:05,872][105692] Updated weights for policy 0, policy_version 382687 (0.0008) [2023-12-26 18:09:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 196132864. Throughput: 0: 9868.7, 1: 9797.4. Samples: 196118068. Policy #0 lag: (min: 28.0, avg: 29.8, max: 56.0) [2023-12-26 18:09:06,062][104569] Avg episode reward: [(0, '9268.824'), (1, '6744.244')] [2023-12-26 18:09:06,232][105620] Updated weights for policy 1, policy_version 383375 (0.0007) [2023-12-26 18:09:06,304][105620] Updated weights for policy 1, policy_version 383385 (0.0009) [2023-12-26 18:09:06,370][105620] Updated weights for policy 1, policy_version 383395 (0.0007) [2023-12-26 18:09:06,610][105692] Updated weights for policy 0, policy_version 382697 (0.0006) [2023-12-26 18:09:06,662][105692] Updated weights for policy 0, policy_version 382707 (0.0009) [2023-12-26 18:09:06,720][105692] Updated weights for policy 0, policy_version 382717 (0.0010) [2023-12-26 18:09:06,779][105692] Updated weights for policy 0, policy_version 382728 (0.0010) [2023-12-26 18:09:06,999][105620] Updated weights for policy 1, policy_version 383405 (0.0007) [2023-12-26 18:09:07,070][105620] Updated weights for policy 1, policy_version 383415 (0.0010) [2023-12-26 18:09:07,143][105620] Updated weights for policy 1, policy_version 383425 (0.0008) [2023-12-26 18:09:07,457][105692] Updated weights for policy 0, policy_version 382738 (0.0006) [2023-12-26 18:09:07,522][105692] Updated weights for policy 0, policy_version 382748 (0.0008) [2023-12-26 18:09:07,585][105692] Updated weights for policy 0, policy_version 382758 (0.0010) [2023-12-26 18:09:07,874][105620] Updated weights for policy 1, policy_version 383435 (0.0007) [2023-12-26 18:09:07,929][105620] Updated weights for policy 1, policy_version 383445 (0.0007) [2023-12-26 18:09:07,990][105620] Updated weights for policy 1, policy_version 383455 (0.0008) [2023-12-26 18:09:08,277][105692] Updated weights for policy 0, policy_version 382768 (0.0010) [2023-12-26 18:09:08,343][105692] Updated weights for policy 0, policy_version 382778 (0.0010) [2023-12-26 18:09:08,400][105692] Updated weights for policy 0, policy_version 382788 (0.0010) [2023-12-26 18:09:08,646][105620] Updated weights for policy 1, policy_version 383465 (0.0008) [2023-12-26 18:09:08,695][105620] Updated weights for policy 1, policy_version 383475 (0.0008) [2023-12-26 18:09:08,746][105620] Updated weights for policy 1, policy_version 383485 (0.0008) [2023-12-26 18:09:08,812][105620] Updated weights for policy 1, policy_version 383495 (0.0008) [2023-12-26 18:09:09,136][105692] Updated weights for policy 0, policy_version 382798 (0.0010) [2023-12-26 18:09:09,188][105692] Updated weights for policy 0, policy_version 382808 (0.0008) [2023-12-26 18:09:09,254][105692] Updated weights for policy 0, policy_version 382818 (0.0007) [2023-12-26 18:09:09,620][105620] Updated weights for policy 1, policy_version 383505 (0.0008) [2023-12-26 18:09:09,680][105620] Updated weights for policy 1, policy_version 383515 (0.0008) [2023-12-26 18:09:09,739][105620] Updated weights for policy 1, policy_version 383525 (0.0008) [2023-12-26 18:09:09,968][105692] Updated weights for policy 0, policy_version 382828 (0.0008) [2023-12-26 18:09:10,030][105692] Updated weights for policy 0, policy_version 382838 (0.0009) [2023-12-26 18:09:10,083][105692] Updated weights for policy 0, policy_version 382848 (0.0010) [2023-12-26 18:09:10,542][105620] Updated weights for policy 1, policy_version 383535 (0.0008) [2023-12-26 18:09:10,603][105620] Updated weights for policy 1, policy_version 383545 (0.0008) [2023-12-26 18:09:10,655][105620] Updated weights for policy 1, policy_version 383555 (0.0008) [2023-12-26 18:09:10,789][105692] Updated weights for policy 0, policy_version 382858 (0.0008) [2023-12-26 18:09:10,844][105692] Updated weights for policy 0, policy_version 382868 (0.0010) [2023-12-26 18:09:10,892][105692] Updated weights for policy 0, policy_version 382878 (0.0010) [2023-12-26 18:09:10,940][105692] Updated weights for policy 0, policy_version 382888 (0.0010) [2023-12-26 18:09:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 196231168. Throughput: 0: 9943.8, 1: 9830.9. Samples: 196236288. Policy #0 lag: (min: 28.0, avg: 29.8, max: 56.0) [2023-12-26 18:09:11,063][104569] Avg episode reward: [(0, '9359.576'), (1, '7644.321')] [2023-12-26 18:09:11,378][105620] Updated weights for policy 1, policy_version 383565 (0.0008) [2023-12-26 18:09:11,436][105620] Updated weights for policy 1, policy_version 383575 (0.0007) [2023-12-26 18:09:11,502][105620] Updated weights for policy 1, policy_version 383585 (0.0008) [2023-12-26 18:09:11,776][105692] Updated weights for policy 0, policy_version 382898 (0.0011) [2023-12-26 18:09:11,823][105692] Updated weights for policy 0, policy_version 382908 (0.0006) [2023-12-26 18:09:11,878][105692] Updated weights for policy 0, policy_version 382918 (0.0009) [2023-12-26 18:09:12,296][105620] Updated weights for policy 1, policy_version 383595 (0.0008) [2023-12-26 18:09:12,362][105620] Updated weights for policy 1, policy_version 383605 (0.0008) [2023-12-26 18:09:12,424][105620] Updated weights for policy 1, policy_version 383615 (0.0008) [2023-12-26 18:09:12,611][105692] Updated weights for policy 0, policy_version 382928 (0.0011) [2023-12-26 18:09:12,669][105692] Updated weights for policy 0, policy_version 382938 (0.0007) [2023-12-26 18:09:12,730][105692] Updated weights for policy 0, policy_version 382948 (0.0011) [2023-12-26 18:09:13,202][105620] Updated weights for policy 1, policy_version 383625 (0.0008) [2023-12-26 18:09:13,255][105620] Updated weights for policy 1, policy_version 383635 (0.0005) [2023-12-26 18:09:13,299][105620] Updated weights for policy 1, policy_version 383645 (0.0005) [2023-12-26 18:09:13,352][105620] Updated weights for policy 1, policy_version 383655 (0.0008) [2023-12-26 18:09:13,442][105692] Updated weights for policy 0, policy_version 382958 (0.0010) [2023-12-26 18:09:13,507][105692] Updated weights for policy 0, policy_version 382968 (0.0010) [2023-12-26 18:09:13,570][105692] Updated weights for policy 0, policy_version 382978 (0.0006) [2023-12-26 18:09:13,976][105620] Updated weights for policy 1, policy_version 383665 (0.0006) [2023-12-26 18:09:14,022][105620] Updated weights for policy 1, policy_version 383675 (0.0005) [2023-12-26 18:09:14,081][105620] Updated weights for policy 1, policy_version 383685 (0.0006) [2023-12-26 18:09:14,201][105692] Updated weights for policy 0, policy_version 382988 (0.0007) [2023-12-26 18:09:14,255][105692] Updated weights for policy 0, policy_version 382998 (0.0010) [2023-12-26 18:09:14,312][105692] Updated weights for policy 0, policy_version 383008 (0.0010) [2023-12-26 18:09:14,755][105620] Updated weights for policy 1, policy_version 383695 (0.0007) [2023-12-26 18:09:14,811][105620] Updated weights for policy 1, policy_version 383705 (0.0008) [2023-12-26 18:09:14,873][105620] Updated weights for policy 1, policy_version 383715 (0.0008) [2023-12-26 18:09:15,065][105692] Updated weights for policy 0, policy_version 383018 (0.0010) [2023-12-26 18:09:15,132][105692] Updated weights for policy 0, policy_version 383028 (0.0011) [2023-12-26 18:09:15,197][105692] Updated weights for policy 0, policy_version 383038 (0.0011) [2023-12-26 18:09:15,263][105692] Updated weights for policy 0, policy_version 383048 (0.0011) [2023-12-26 18:09:15,629][105620] Updated weights for policy 1, policy_version 383725 (0.0009) [2023-12-26 18:09:15,692][105620] Updated weights for policy 1, policy_version 383735 (0.0009) [2023-12-26 18:09:15,749][105620] Updated weights for policy 1, policy_version 383745 (0.0009) [2023-12-26 18:09:15,981][105692] Updated weights for policy 0, policy_version 383058 (0.0009) [2023-12-26 18:09:16,041][105692] Updated weights for policy 0, policy_version 383068 (0.0009) [2023-12-26 18:09:16,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 196321280. Throughput: 0: 9854.8, 1: 9821.6. Samples: 196293436. Policy #0 lag: (min: 28.0, avg: 29.8, max: 56.0) [2023-12-26 18:09:16,063][104569] Avg episode reward: [(0, '9359.450'), (1, '7442.225')] [2023-12-26 18:09:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000383752_98246656.pth... [2023-12-26 18:09:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000382600_97951744.pth [2023-12-26 18:09:16,100][105692] Updated weights for policy 0, policy_version 383078 (0.0009) [2023-12-26 18:09:16,110][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000383080_98082816.pth... [2023-12-26 18:09:16,115][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000381928_97787904.pth [2023-12-26 18:09:16,501][105620] Updated weights for policy 1, policy_version 383755 (0.0009) [2023-12-26 18:09:16,557][105620] Updated weights for policy 1, policy_version 383765 (0.0009) [2023-12-26 18:09:16,607][105620] Updated weights for policy 1, policy_version 383775 (0.0008) [2023-12-26 18:09:16,807][105692] Updated weights for policy 0, policy_version 383088 (0.0006) [2023-12-26 18:09:16,855][105692] Updated weights for policy 0, policy_version 383098 (0.0005) [2023-12-26 18:09:16,900][105692] Updated weights for policy 0, policy_version 383108 (0.0007) [2023-12-26 18:09:17,310][105620] Updated weights for policy 1, policy_version 383786 (0.0009) [2023-12-26 18:09:17,366][105620] Updated weights for policy 1, policy_version 383796 (0.0005) [2023-12-26 18:09:17,415][105620] Updated weights for policy 1, policy_version 383806 (0.0005) [2023-12-26 18:09:17,471][105620] Updated weights for policy 1, policy_version 383816 (0.0005) [2023-12-26 18:09:17,703][105692] Updated weights for policy 0, policy_version 383118 (0.0010) [2023-12-26 18:09:17,758][105692] Updated weights for policy 0, policy_version 383128 (0.0010) [2023-12-26 18:09:17,816][105692] Updated weights for policy 0, policy_version 383138 (0.0010) [2023-12-26 18:09:18,018][105620] Updated weights for policy 1, policy_version 383826 (0.0006) [2023-12-26 18:09:18,070][105620] Updated weights for policy 1, policy_version 383836 (0.0007) [2023-12-26 18:09:18,123][105620] Updated weights for policy 1, policy_version 383846 (0.0005) [2023-12-26 18:09:18,453][105692] Updated weights for policy 0, policy_version 383148 (0.0011) [2023-12-26 18:09:18,520][105692] Updated weights for policy 0, policy_version 383158 (0.0011) [2023-12-26 18:09:18,588][105692] Updated weights for policy 0, policy_version 383168 (0.0007) [2023-12-26 18:09:18,838][105620] Updated weights for policy 1, policy_version 383856 (0.0005) [2023-12-26 18:09:18,907][105620] Updated weights for policy 1, policy_version 383866 (0.0005) [2023-12-26 18:09:18,975][105620] Updated weights for policy 1, policy_version 383876 (0.0006) [2023-12-26 18:09:19,301][105692] Updated weights for policy 0, policy_version 383178 (0.0007) [2023-12-26 18:09:19,373][105692] Updated weights for policy 0, policy_version 383188 (0.0008) [2023-12-26 18:09:19,424][105692] Updated weights for policy 0, policy_version 383198 (0.0006) [2023-12-26 18:09:19,474][105692] Updated weights for policy 0, policy_version 383208 (0.0007) [2023-12-26 18:09:19,650][105620] Updated weights for policy 1, policy_version 383886 (0.0009) [2023-12-26 18:09:19,704][105620] Updated weights for policy 1, policy_version 383897 (0.0010) [2023-12-26 18:09:19,773][105620] Updated weights for policy 1, policy_version 383908 (0.0009) [2023-12-26 18:09:20,101][105692] Updated weights for policy 0, policy_version 383218 (0.0006) [2023-12-26 18:09:20,167][105692] Updated weights for policy 0, policy_version 383228 (0.0006) [2023-12-26 18:09:20,238][105692] Updated weights for policy 0, policy_version 383238 (0.0006) [2023-12-26 18:09:20,498][105620] Updated weights for policy 1, policy_version 383918 (0.0007) [2023-12-26 18:09:20,560][105620] Updated weights for policy 1, policy_version 383928 (0.0006) [2023-12-26 18:09:20,628][105620] Updated weights for policy 1, policy_version 383938 (0.0008) [2023-12-26 18:09:20,943][105692] Updated weights for policy 0, policy_version 383248 (0.0009) [2023-12-26 18:09:21,002][105692] Updated weights for policy 0, policy_version 383258 (0.0009) [2023-12-26 18:09:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 196419584. Throughput: 0: 9826.1, 1: 9773.0. Samples: 196411972. Policy #0 lag: (min: 28.0, avg: 29.8, max: 56.0) [2023-12-26 18:09:21,063][104569] Avg episode reward: [(0, '9357.772'), (1, '5994.665')] [2023-12-26 18:09:21,077][105692] Updated weights for policy 0, policy_version 383268 (0.0009) [2023-12-26 18:09:21,396][105620] Updated weights for policy 1, policy_version 383948 (0.0009) [2023-12-26 18:09:21,460][105586] KL-divergence is very high: 102.3975 [2023-12-26 18:09:21,461][105620] Updated weights for policy 1, policy_version 383958 (0.0009) [2023-12-26 18:09:21,490][105586] KL-divergence is very high: 149.3647 [2023-12-26 18:09:21,508][105586] KL-divergence is very high: 141.5110 [2023-12-26 18:09:21,521][105620] Updated weights for policy 1, policy_version 383968 (0.0010) [2023-12-26 18:09:21,539][105586] KL-divergence is very high: 153.0061 [2023-12-26 18:09:21,556][105586] KL-divergence is very high: 127.4042 [2023-12-26 18:09:21,848][105692] Updated weights for policy 0, policy_version 383278 (0.0008) [2023-12-26 18:09:21,912][105692] Updated weights for policy 0, policy_version 383288 (0.0006) [2023-12-26 18:09:21,983][105692] Updated weights for policy 0, policy_version 383298 (0.0006) [2023-12-26 18:09:22,239][105620] Updated weights for policy 1, policy_version 383978 (0.0009) [2023-12-26 18:09:22,297][105620] Updated weights for policy 1, policy_version 383988 (0.0008) [2023-12-26 18:09:22,354][105620] Updated weights for policy 1, policy_version 383998 (0.0008) [2023-12-26 18:09:22,411][105620] Updated weights for policy 1, policy_version 384008 (0.0006) [2023-12-26 18:09:22,593][105692] Updated weights for policy 0, policy_version 383308 (0.0007) [2023-12-26 18:09:22,653][105692] Updated weights for policy 0, policy_version 383318 (0.0009) [2023-12-26 18:09:22,715][105692] Updated weights for policy 0, policy_version 383328 (0.0009) [2023-12-26 18:09:23,100][105620] Updated weights for policy 1, policy_version 384018 (0.0009) [2023-12-26 18:09:23,159][105620] Updated weights for policy 1, policy_version 384028 (0.0009) [2023-12-26 18:09:23,219][105620] Updated weights for policy 1, policy_version 384038 (0.0010) [2023-12-26 18:09:23,406][105692] Updated weights for policy 0, policy_version 383338 (0.0009) [2023-12-26 18:09:23,466][105692] Updated weights for policy 0, policy_version 383348 (0.0009) [2023-12-26 18:09:23,533][105692] Updated weights for policy 0, policy_version 383358 (0.0010) [2023-12-26 18:09:23,591][105692] Updated weights for policy 0, policy_version 383368 (0.0008) [2023-12-26 18:09:23,916][105620] Updated weights for policy 1, policy_version 384048 (0.0009) [2023-12-26 18:09:23,974][105620] Updated weights for policy 1, policy_version 384058 (0.0010) [2023-12-26 18:09:24,042][105620] Updated weights for policy 1, policy_version 384068 (0.0009) [2023-12-26 18:09:24,188][105692] Updated weights for policy 0, policy_version 383378 (0.0005) [2023-12-26 18:09:24,240][105692] Updated weights for policy 0, policy_version 383388 (0.0005) [2023-12-26 18:09:24,297][105692] Updated weights for policy 0, policy_version 383398 (0.0005) [2023-12-26 18:09:24,664][105586] KL-divergence is very high: 104.6777 [2023-12-26 18:09:24,664][105620] Updated weights for policy 1, policy_version 384078 (0.0010) [2023-12-26 18:09:24,718][105620] Updated weights for policy 1, policy_version 384088 (0.0007) [2023-12-26 18:09:24,719][105586] KL-divergence is very high: 119.6684 [2023-12-26 18:09:24,730][105586] KL-divergence is very high: 147.3714 [2023-12-26 18:09:24,780][105620] Updated weights for policy 1, policy_version 384098 (0.0005) [2023-12-26 18:09:25,001][105692] Updated weights for policy 0, policy_version 383408 (0.0008) [2023-12-26 18:09:25,056][105692] Updated weights for policy 0, policy_version 383418 (0.0005) [2023-12-26 18:09:25,120][105692] Updated weights for policy 0, policy_version 383428 (0.0005) [2023-12-26 18:09:25,394][105586] KL-divergence is very high: 212.9261 [2023-12-26 18:09:25,400][105586] KL-divergence is very high: 201.4440 [2023-12-26 18:09:25,405][105620] Updated weights for policy 1, policy_version 384108 (0.0007) [2023-12-26 18:09:25,436][105586] KL-divergence is very high: 141.3487 [2023-12-26 18:09:25,442][105586] KL-divergence is very high: 129.3380 [2023-12-26 18:09:25,459][105620] Updated weights for policy 1, policy_version 384118 (0.0009) [2023-12-26 18:09:25,483][105586] KL-divergence is very high: 138.9895 [2023-12-26 18:09:25,489][105586] KL-divergence is very high: 163.9726 [2023-12-26 18:09:25,523][105620] Updated weights for policy 1, policy_version 384128 (0.0009) [2023-12-26 18:09:25,536][105586] KL-divergence is very high: 212.2495 [2023-12-26 18:09:25,543][105586] KL-divergence is very high: 179.3191 [2023-12-26 18:09:25,739][105692] Updated weights for policy 0, policy_version 383438 (0.0008) [2023-12-26 18:09:25,804][105692] Updated weights for policy 0, policy_version 383448 (0.0009) [2023-12-26 18:09:25,855][105692] Updated weights for policy 0, policy_version 383458 (0.0009) [2023-12-26 18:09:26,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 196526080. Throughput: 0: 9924.8, 1: 9821.7. Samples: 196533216. Policy #0 lag: (min: 28.0, avg: 29.8, max: 56.0) [2023-12-26 18:09:26,062][104569] Avg episode reward: [(0, '9357.660'), (1, '806.388')] [2023-12-26 18:09:26,221][105620] Updated weights for policy 1, policy_version 384138 (0.0009) [2023-12-26 18:09:26,279][105620] Updated weights for policy 1, policy_version 384148 (0.0009) [2023-12-26 18:09:26,338][105620] Updated weights for policy 1, policy_version 384158 (0.0008) [2023-12-26 18:09:26,391][105620] Updated weights for policy 1, policy_version 384168 (0.0010) [2023-12-26 18:09:26,573][105692] Updated weights for policy 0, policy_version 383468 (0.0009) [2023-12-26 18:09:26,621][105692] Updated weights for policy 0, policy_version 383478 (0.0010) [2023-12-26 18:09:26,668][105692] Updated weights for policy 0, policy_version 383488 (0.0010) [2023-12-26 18:09:27,235][105620] Updated weights for policy 1, policy_version 384178 (0.0009) [2023-12-26 18:09:27,247][105692] Updated weights for policy 0, policy_version 383498 (0.0009) [2023-12-26 18:09:27,297][105620] Updated weights for policy 1, policy_version 384188 (0.0008) [2023-12-26 18:09:27,313][105692] Updated weights for policy 0, policy_version 383508 (0.0007) [2023-12-26 18:09:27,359][105620] Updated weights for policy 1, policy_version 384198 (0.0006) [2023-12-26 18:09:27,363][105692] Updated weights for policy 0, policy_version 383518 (0.0005) [2023-12-26 18:09:27,408][105692] Updated weights for policy 0, policy_version 383528 (0.0005) [2023-12-26 18:09:27,917][105692] Updated weights for policy 0, policy_version 383538 (0.0005) [2023-12-26 18:09:27,963][105692] Updated weights for policy 0, policy_version 383548 (0.0009) [2023-12-26 18:09:28,011][105692] Updated weights for policy 0, policy_version 383558 (0.0009) [2023-12-26 18:09:28,163][105620] Updated weights for policy 1, policy_version 384208 (0.0008) [2023-12-26 18:09:28,213][105620] Updated weights for policy 1, policy_version 384218 (0.0009) [2023-12-26 18:09:28,265][105620] Updated weights for policy 1, policy_version 384228 (0.0009) [2023-12-26 18:09:28,626][105692] Updated weights for policy 0, policy_version 383568 (0.0006) [2023-12-26 18:09:28,695][105692] Updated weights for policy 0, policy_version 383578 (0.0008) [2023-12-26 18:09:28,763][105692] Updated weights for policy 0, policy_version 383588 (0.0008) [2023-12-26 18:09:29,118][105620] Updated weights for policy 1, policy_version 384238 (0.0010) [2023-12-26 18:09:29,171][105620] Updated weights for policy 1, policy_version 384248 (0.0009) [2023-12-26 18:09:29,228][105620] Updated weights for policy 1, policy_version 384258 (0.0010) [2023-12-26 18:09:29,339][105692] Updated weights for policy 0, policy_version 383598 (0.0010) [2023-12-26 18:09:29,407][105692] Updated weights for policy 0, policy_version 383608 (0.0011) [2023-12-26 18:09:29,463][105692] Updated weights for policy 0, policy_version 383618 (0.0010) [2023-12-26 18:09:29,957][105620] Updated weights for policy 1, policy_version 384268 (0.0008) [2023-12-26 18:09:30,015][105620] Updated weights for policy 1, policy_version 384278 (0.0010) [2023-12-26 18:09:30,074][105620] Updated weights for policy 1, policy_version 384288 (0.0007) [2023-12-26 18:09:30,215][105692] Updated weights for policy 0, policy_version 383628 (0.0008) [2023-12-26 18:09:30,270][105692] Updated weights for policy 0, policy_version 383638 (0.0010) [2023-12-26 18:09:30,326][105692] Updated weights for policy 0, policy_version 383648 (0.0009) [2023-12-26 18:09:30,881][105620] Updated weights for policy 1, policy_version 384298 (0.0008) [2023-12-26 18:09:30,936][105620] Updated weights for policy 1, policy_version 384308 (0.0008) [2023-12-26 18:09:30,970][105692] Updated weights for policy 0, policy_version 383658 (0.0006) [2023-12-26 18:09:30,980][105620] Updated weights for policy 1, policy_version 384318 (0.0007) [2023-12-26 18:09:31,022][105692] Updated weights for policy 0, policy_version 383668 (0.0010) [2023-12-26 18:09:31,033][105620] Updated weights for policy 1, policy_version 384328 (0.0006) [2023-12-26 18:09:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 196624384. Throughput: 0: 10060.6, 1: 9774.8. Samples: 196593500. Policy #0 lag: (min: 28.0, avg: 29.8, max: 56.0) [2023-12-26 18:09:31,062][104569] Avg episode reward: [(0, '9356.298'), (1, '459.286')] [2023-12-26 18:09:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000384328_98394112.pth... [2023-12-26 18:09:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000383176_98099200.pth [2023-12-26 18:09:31,084][105692] Updated weights for policy 0, policy_version 383678 (0.0010) [2023-12-26 18:09:31,148][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000383688_98238464.pth... [2023-12-26 18:09:31,149][105692] Updated weights for policy 0, policy_version 383688 (0.0007) [2023-12-26 18:09:31,153][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000382472_97927168.pth [2023-12-26 18:09:31,859][105620] Updated weights for policy 1, policy_version 384338 (0.0007) [2023-12-26 18:09:31,890][105692] Updated weights for policy 0, policy_version 383698 (0.0009) [2023-12-26 18:09:31,913][105620] Updated weights for policy 1, policy_version 384348 (0.0007) [2023-12-26 18:09:31,940][105692] Updated weights for policy 0, policy_version 383708 (0.0007) [2023-12-26 18:09:31,973][105620] Updated weights for policy 1, policy_version 384358 (0.0009) [2023-12-26 18:09:31,996][105692] Updated weights for policy 0, policy_version 383718 (0.0008) [2023-12-26 18:09:32,678][105692] Updated weights for policy 0, policy_version 383728 (0.0009) [2023-12-26 18:09:32,729][105620] Updated weights for policy 1, policy_version 384368 (0.0008) [2023-12-26 18:09:32,737][105692] Updated weights for policy 0, policy_version 383738 (0.0007) [2023-12-26 18:09:32,784][105620] Updated weights for policy 1, policy_version 384378 (0.0008) [2023-12-26 18:09:32,799][105692] Updated weights for policy 0, policy_version 383748 (0.0005) [2023-12-26 18:09:32,850][105620] Updated weights for policy 1, policy_version 384388 (0.0006) [2023-12-26 18:09:33,342][105692] Updated weights for policy 0, policy_version 383758 (0.0006) [2023-12-26 18:09:33,397][105692] Updated weights for policy 0, policy_version 383768 (0.0006) [2023-12-26 18:09:33,416][105620] Updated weights for policy 1, policy_version 384398 (0.0005) [2023-12-26 18:09:33,460][105692] Updated weights for policy 0, policy_version 383778 (0.0005) [2023-12-26 18:09:33,471][105620] Updated weights for policy 1, policy_version 384408 (0.0007) [2023-12-26 18:09:33,522][105620] Updated weights for policy 1, policy_version 384419 (0.0010) [2023-12-26 18:09:34,091][105692] Updated weights for policy 0, policy_version 383789 (0.0008) [2023-12-26 18:09:34,155][105692] Updated weights for policy 0, policy_version 383799 (0.0009) [2023-12-26 18:09:34,214][105692] Updated weights for policy 0, policy_version 383809 (0.0008) [2023-12-26 18:09:34,224][105620] Updated weights for policy 1, policy_version 384429 (0.0007) [2023-12-26 18:09:34,283][105620] Updated weights for policy 1, policy_version 384439 (0.0007) [2023-12-26 18:09:34,344][105620] Updated weights for policy 1, policy_version 384449 (0.0009) [2023-12-26 18:09:34,878][105692] Updated weights for policy 0, policy_version 383819 (0.0009) [2023-12-26 18:09:34,932][105692] Updated weights for policy 0, policy_version 383829 (0.0006) [2023-12-26 18:09:34,987][105692] Updated weights for policy 0, policy_version 383839 (0.0009) [2023-12-26 18:09:35,119][105620] Updated weights for policy 1, policy_version 384459 (0.0009) [2023-12-26 18:09:35,183][105620] Updated weights for policy 1, policy_version 384469 (0.0009) [2023-12-26 18:09:35,236][105620] Updated weights for policy 1, policy_version 384479 (0.0010) [2023-12-26 18:09:35,645][105692] Updated weights for policy 0, policy_version 383849 (0.0009) [2023-12-26 18:09:35,711][105692] Updated weights for policy 0, policy_version 383859 (0.0007) [2023-12-26 18:09:35,771][105692] Updated weights for policy 0, policy_version 383869 (0.0005) [2023-12-26 18:09:35,829][105692] Updated weights for policy 0, policy_version 383879 (0.0005) [2023-12-26 18:09:36,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 196722688. Throughput: 0: 9986.8, 1: 9642.8. Samples: 196712368. Policy #0 lag: (min: 28.0, avg: 29.8, max: 56.0) [2023-12-26 18:09:36,063][104569] Avg episode reward: [(0, '9356.945'), (1, '469.504')] [2023-12-26 18:09:36,084][105620] Updated weights for policy 1, policy_version 384490 (0.0010) [2023-12-26 18:09:36,147][105620] Updated weights for policy 1, policy_version 384500 (0.0010) [2023-12-26 18:09:36,216][105620] Updated weights for policy 1, policy_version 384510 (0.0009) [2023-12-26 18:09:36,279][105620] Updated weights for policy 1, policy_version 384520 (0.0009) [2023-12-26 18:09:36,416][105692] Updated weights for policy 0, policy_version 383889 (0.0005) [2023-12-26 18:09:36,474][105692] Updated weights for policy 0, policy_version 383899 (0.0006) [2023-12-26 18:09:36,535][105692] Updated weights for policy 0, policy_version 383909 (0.0005) [2023-12-26 18:09:37,093][105692] Updated weights for policy 0, policy_version 383919 (0.0006) [2023-12-26 18:09:37,156][105692] Updated weights for policy 0, policy_version 383929 (0.0006) [2023-12-26 18:09:37,159][105620] Updated weights for policy 1, policy_version 384530 (0.0011) [2023-12-26 18:09:37,206][105692] Updated weights for policy 0, policy_version 383939 (0.0006) [2023-12-26 18:09:37,215][105620] Updated weights for policy 1, policy_version 384540 (0.0011) [2023-12-26 18:09:37,277][105620] Updated weights for policy 1, policy_version 384550 (0.0010) [2023-12-26 18:09:37,964][105692] Updated weights for policy 0, policy_version 383949 (0.0008) [2023-12-26 18:09:38,016][105692] Updated weights for policy 0, policy_version 383959 (0.0008) [2023-12-26 18:09:38,026][105620] Updated weights for policy 1, policy_version 384560 (0.0011) [2023-12-26 18:09:38,072][105692] Updated weights for policy 0, policy_version 383969 (0.0005) [2023-12-26 18:09:38,089][105620] Updated weights for policy 1, policy_version 384570 (0.0011) [2023-12-26 18:09:38,137][105620] Updated weights for policy 1, policy_version 384580 (0.0010) [2023-12-26 18:09:38,876][105692] Updated weights for policy 0, policy_version 383979 (0.0006) [2023-12-26 18:09:38,886][105620] Updated weights for policy 1, policy_version 384590 (0.0010) [2023-12-26 18:09:38,925][105692] Updated weights for policy 0, policy_version 383989 (0.0006) [2023-12-26 18:09:38,945][105620] Updated weights for policy 1, policy_version 384600 (0.0010) [2023-12-26 18:09:38,979][105692] Updated weights for policy 0, policy_version 383999 (0.0008) [2023-12-26 18:09:39,004][105620] Updated weights for policy 1, policy_version 384610 (0.0010) [2023-12-26 18:09:39,751][105620] Updated weights for policy 1, policy_version 384620 (0.0010) [2023-12-26 18:09:39,776][105692] Updated weights for policy 0, policy_version 384009 (0.0006) [2023-12-26 18:09:39,811][105620] Updated weights for policy 1, policy_version 384630 (0.0011) [2023-12-26 18:09:39,842][105692] Updated weights for policy 0, policy_version 384019 (0.0006) [2023-12-26 18:09:39,878][105620] Updated weights for policy 1, policy_version 384640 (0.0010) [2023-12-26 18:09:39,907][105692] Updated weights for policy 0, policy_version 384029 (0.0010) [2023-12-26 18:09:39,971][105692] Updated weights for policy 0, policy_version 384039 (0.0008) [2023-12-26 18:09:40,622][105620] Updated weights for policy 1, policy_version 384650 (0.0010) [2023-12-26 18:09:40,686][105620] Updated weights for policy 1, policy_version 384660 (0.0005) [2023-12-26 18:09:40,723][105692] Updated weights for policy 0, policy_version 384049 (0.0011) [2023-12-26 18:09:40,748][105620] Updated weights for policy 1, policy_version 384670 (0.0010) [2023-12-26 18:09:40,786][105692] Updated weights for policy 0, policy_version 384059 (0.0011) [2023-12-26 18:09:40,808][105620] Updated weights for policy 1, policy_version 384680 (0.0009) [2023-12-26 18:09:40,844][105692] Updated weights for policy 0, policy_version 384069 (0.0010) [2023-12-26 18:09:41,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 196820992. Throughput: 0: 10091.9, 1: 9523.6. Samples: 196826492. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-26 18:09:41,063][104569] Avg episode reward: [(0, '9356.868'), (1, '514.710')] [2023-12-26 18:09:41,552][105620] Updated weights for policy 1, policy_version 384690 (0.0009) [2023-12-26 18:09:41,555][105692] Updated weights for policy 0, policy_version 384079 (0.0009) [2023-12-26 18:09:41,616][105692] Updated weights for policy 0, policy_version 384089 (0.0009) [2023-12-26 18:09:41,617][105620] Updated weights for policy 1, policy_version 384700 (0.0006) [2023-12-26 18:09:41,688][105620] Updated weights for policy 1, policy_version 384710 (0.0006) [2023-12-26 18:09:41,691][105692] Updated weights for policy 0, policy_version 384099 (0.0010) [2023-12-26 18:09:42,398][105692] Updated weights for policy 0, policy_version 384109 (0.0010) [2023-12-26 18:09:42,467][105692] Updated weights for policy 0, policy_version 384119 (0.0007) [2023-12-26 18:09:42,474][105620] Updated weights for policy 1, policy_version 384720 (0.0008) [2023-12-26 18:09:42,521][105692] Updated weights for policy 0, policy_version 384129 (0.0006) [2023-12-26 18:09:42,531][105620] Updated weights for policy 1, policy_version 384730 (0.0009) [2023-12-26 18:09:42,584][105620] Updated weights for policy 1, policy_version 384740 (0.0009) [2023-12-26 18:09:43,213][105692] Updated weights for policy 0, policy_version 384139 (0.0007) [2023-12-26 18:09:43,261][105692] Updated weights for policy 0, policy_version 384149 (0.0010) [2023-12-26 18:09:43,305][105692] Updated weights for policy 0, policy_version 384159 (0.0010) [2023-12-26 18:09:43,315][105620] Updated weights for policy 1, policy_version 384750 (0.0008) [2023-12-26 18:09:43,372][105620] Updated weights for policy 1, policy_version 384760 (0.0006) [2023-12-26 18:09:43,420][105620] Updated weights for policy 1, policy_version 384770 (0.0008) [2023-12-26 18:09:44,032][105620] Updated weights for policy 1, policy_version 384780 (0.0007) [2023-12-26 18:09:44,049][105692] Updated weights for policy 0, policy_version 384169 (0.0010) [2023-12-26 18:09:44,092][105620] Updated weights for policy 1, policy_version 384790 (0.0008) [2023-12-26 18:09:44,115][105692] Updated weights for policy 0, policy_version 384179 (0.0010) [2023-12-26 18:09:44,152][105620] Updated weights for policy 1, policy_version 384800 (0.0008) [2023-12-26 18:09:44,167][105692] Updated weights for policy 0, policy_version 384189 (0.0010) [2023-12-26 18:09:44,224][105692] Updated weights for policy 0, policy_version 384199 (0.0005) [2023-12-26 18:09:44,737][105620] Updated weights for policy 1, policy_version 384810 (0.0008) [2023-12-26 18:09:44,803][105620] Updated weights for policy 1, policy_version 384820 (0.0008) [2023-12-26 18:09:44,850][105692] Updated weights for policy 0, policy_version 384209 (0.0010) [2023-12-26 18:09:44,864][105620] Updated weights for policy 1, policy_version 384830 (0.0006) [2023-12-26 18:09:44,910][105692] Updated weights for policy 0, policy_version 384219 (0.0011) [2023-12-26 18:09:44,928][105620] Updated weights for policy 1, policy_version 384840 (0.0006) [2023-12-26 18:09:44,958][105692] Updated weights for policy 0, policy_version 384229 (0.0006) [2023-12-26 18:09:45,673][105692] Updated weights for policy 0, policy_version 384239 (0.0009) [2023-12-26 18:09:45,691][105620] Updated weights for policy 1, policy_version 384850 (0.0006) [2023-12-26 18:09:45,731][105692] Updated weights for policy 0, policy_version 384249 (0.0010) [2023-12-26 18:09:45,743][105620] Updated weights for policy 1, policy_version 384860 (0.0006) [2023-12-26 18:09:45,795][105692] Updated weights for policy 0, policy_version 384259 (0.0005) [2023-12-26 18:09:45,796][105620] Updated weights for policy 1, policy_version 384870 (0.0009) [2023-12-26 18:09:46,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 196919296. Throughput: 0: 10111.2, 1: 9545.5. Samples: 196884036. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-26 18:09:46,062][104569] Avg episode reward: [(0, '9356.771'), (1, '475.196')] [2023-12-26 18:09:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000384872_98533376.pth... [2023-12-26 18:09:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000384264_98385920.pth... [2023-12-26 18:09:46,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000383080_98082816.pth [2023-12-26 18:09:46,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000383752_98246656.pth [2023-12-26 18:09:46,334][105692] Updated weights for policy 0, policy_version 384269 (0.0005) [2023-12-26 18:09:46,378][105692] Updated weights for policy 0, policy_version 384279 (0.0005) [2023-12-26 18:09:46,430][105692] Updated weights for policy 0, policy_version 384289 (0.0005) [2023-12-26 18:09:46,695][105620] Updated weights for policy 1, policy_version 384880 (0.0009) [2023-12-26 18:09:46,750][105620] Updated weights for policy 1, policy_version 384890 (0.0008) [2023-12-26 18:09:46,797][105620] Updated weights for policy 1, policy_version 384900 (0.0008) [2023-12-26 18:09:47,077][105692] Updated weights for policy 0, policy_version 384299 (0.0007) [2023-12-26 18:09:47,129][105692] Updated weights for policy 0, policy_version 384309 (0.0010) [2023-12-26 18:09:47,180][105692] Updated weights for policy 0, policy_version 384319 (0.0010) [2023-12-26 18:09:47,559][105620] Updated weights for policy 1, policy_version 384910 (0.0008) [2023-12-26 18:09:47,613][105620] Updated weights for policy 1, policy_version 384920 (0.0009) [2023-12-26 18:09:47,669][105620] Updated weights for policy 1, policy_version 384930 (0.0009) [2023-12-26 18:09:47,864][105692] Updated weights for policy 0, policy_version 384329 (0.0010) [2023-12-26 18:09:47,922][105692] Updated weights for policy 0, policy_version 384339 (0.0010) [2023-12-26 18:09:47,982][105692] Updated weights for policy 0, policy_version 384349 (0.0006) [2023-12-26 18:09:48,041][105692] Updated weights for policy 0, policy_version 384359 (0.0007) [2023-12-26 18:09:48,547][105620] Updated weights for policy 1, policy_version 384940 (0.0009) [2023-12-26 18:09:48,609][105620] Updated weights for policy 1, policy_version 384950 (0.0009) [2023-12-26 18:09:48,616][105692] Updated weights for policy 0, policy_version 384369 (0.0007) [2023-12-26 18:09:48,669][105620] Updated weights for policy 1, policy_version 384960 (0.0007) [2023-12-26 18:09:48,671][105692] Updated weights for policy 0, policy_version 384379 (0.0008) [2023-12-26 18:09:48,744][105692] Updated weights for policy 0, policy_version 384389 (0.0009) [2023-12-26 18:09:49,426][105620] Updated weights for policy 1, policy_version 384970 (0.0007) [2023-12-26 18:09:49,477][105620] Updated weights for policy 1, policy_version 384980 (0.0009) [2023-12-26 18:09:49,527][105692] Updated weights for policy 0, policy_version 384399 (0.0008) [2023-12-26 18:09:49,531][105620] Updated weights for policy 1, policy_version 384990 (0.0009) [2023-12-26 18:09:49,585][105692] Updated weights for policy 0, policy_version 384409 (0.0010) [2023-12-26 18:09:49,588][105620] Updated weights for policy 1, policy_version 385000 (0.0008) [2023-12-26 18:09:49,641][105692] Updated weights for policy 0, policy_version 384419 (0.0009) [2023-12-26 18:09:50,393][105620] Updated weights for policy 1, policy_version 385010 (0.0009) [2023-12-26 18:09:50,400][105692] Updated weights for policy 0, policy_version 384429 (0.0008) [2023-12-26 18:09:50,451][105620] Updated weights for policy 1, policy_version 385020 (0.0008) [2023-12-26 18:09:50,454][105692] Updated weights for policy 0, policy_version 384439 (0.0008) [2023-12-26 18:09:50,508][105620] Updated weights for policy 1, policy_version 385030 (0.0007) [2023-12-26 18:09:50,511][105692] Updated weights for policy 0, policy_version 384449 (0.0006) [2023-12-26 18:09:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 197009408. Throughput: 0: 10147.9, 1: 9483.1. Samples: 197001464. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-26 18:09:51,063][104569] Avg episode reward: [(0, '9356.768'), (1, '507.582')] [2023-12-26 18:09:51,237][105692] Updated weights for policy 0, policy_version 384459 (0.0008) [2023-12-26 18:09:51,300][105620] Updated weights for policy 1, policy_version 385040 (0.0008) [2023-12-26 18:09:51,300][105692] Updated weights for policy 0, policy_version 384469 (0.0008) [2023-12-26 18:09:51,369][105692] Updated weights for policy 0, policy_version 384479 (0.0008) [2023-12-26 18:09:51,369][105620] Updated weights for policy 1, policy_version 385050 (0.0007) [2023-12-26 18:09:51,433][105620] Updated weights for policy 1, policy_version 385060 (0.0009) [2023-12-26 18:09:52,105][105692] Updated weights for policy 0, policy_version 384489 (0.0008) [2023-12-26 18:09:52,172][105692] Updated weights for policy 0, policy_version 384499 (0.0006) [2023-12-26 18:09:52,202][105620] Updated weights for policy 1, policy_version 385070 (0.0008) [2023-12-26 18:09:52,225][105692] Updated weights for policy 0, policy_version 384509 (0.0007) [2023-12-26 18:09:52,261][105620] Updated weights for policy 1, policy_version 385080 (0.0009) [2023-12-26 18:09:52,287][105692] Updated weights for policy 0, policy_version 384519 (0.0008) [2023-12-26 18:09:52,319][105620] Updated weights for policy 1, policy_version 385090 (0.0007) [2023-12-26 18:09:52,932][105692] Updated weights for policy 0, policy_version 384529 (0.0007) [2023-12-26 18:09:52,983][105692] Updated weights for policy 0, policy_version 384539 (0.0009) [2023-12-26 18:09:53,031][105692] Updated weights for policy 0, policy_version 384549 (0.0009) [2023-12-26 18:09:53,149][105620] Updated weights for policy 1, policy_version 385100 (0.0008) [2023-12-26 18:09:53,204][105620] Updated weights for policy 1, policy_version 385110 (0.0008) [2023-12-26 18:09:53,261][105620] Updated weights for policy 1, policy_version 385120 (0.0009) [2023-12-26 18:09:53,652][105692] Updated weights for policy 0, policy_version 384559 (0.0008) [2023-12-26 18:09:53,713][105692] Updated weights for policy 0, policy_version 384569 (0.0008) [2023-12-26 18:09:53,779][105692] Updated weights for policy 0, policy_version 384579 (0.0009) [2023-12-26 18:09:54,010][105620] Updated weights for policy 1, policy_version 385130 (0.0010) [2023-12-26 18:09:54,078][105620] Updated weights for policy 1, policy_version 385140 (0.0009) [2023-12-26 18:09:54,135][105620] Updated weights for policy 1, policy_version 385150 (0.0008) [2023-12-26 18:09:54,189][105620] Updated weights for policy 1, policy_version 385160 (0.0009) [2023-12-26 18:09:54,485][105692] Updated weights for policy 0, policy_version 384589 (0.0007) [2023-12-26 18:09:54,544][105692] Updated weights for policy 0, policy_version 384599 (0.0005) [2023-12-26 18:09:54,602][105692] Updated weights for policy 0, policy_version 384609 (0.0007) [2023-12-26 18:09:54,990][105620] Updated weights for policy 1, policy_version 385170 (0.0009) [2023-12-26 18:09:55,041][105620] Updated weights for policy 1, policy_version 385180 (0.0009) [2023-12-26 18:09:55,091][105620] Updated weights for policy 1, policy_version 385190 (0.0009) [2023-12-26 18:09:55,274][105692] Updated weights for policy 0, policy_version 384619 (0.0006) [2023-12-26 18:09:55,321][105692] Updated weights for policy 0, policy_version 384629 (0.0009) [2023-12-26 18:09:55,373][105692] Updated weights for policy 0, policy_version 384639 (0.0009) [2023-12-26 18:09:55,863][105620] Updated weights for policy 1, policy_version 385200 (0.0008) [2023-12-26 18:09:55,918][105620] Updated weights for policy 1, policy_version 385210 (0.0009) [2023-12-26 18:09:55,979][105620] Updated weights for policy 1, policy_version 385220 (0.0006) [2023-12-26 18:09:56,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 197107712. Throughput: 0: 10112.9, 1: 9405.1. Samples: 197114600. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-26 18:09:56,063][104569] Avg episode reward: [(0, '9356.378'), (1, '509.480')] [2023-12-26 18:09:56,181][105692] Updated weights for policy 0, policy_version 384649 (0.0009) [2023-12-26 18:09:56,245][105692] Updated weights for policy 0, policy_version 384659 (0.0010) [2023-12-26 18:09:56,307][105692] Updated weights for policy 0, policy_version 384669 (0.0009) [2023-12-26 18:09:56,372][105692] Updated weights for policy 0, policy_version 384679 (0.0008) [2023-12-26 18:09:56,628][105620] Updated weights for policy 1, policy_version 385230 (0.0007) [2023-12-26 18:09:56,685][105620] Updated weights for policy 1, policy_version 385240 (0.0009) [2023-12-26 18:09:56,740][105620] Updated weights for policy 1, policy_version 385250 (0.0010) [2023-12-26 18:09:57,060][105692] Updated weights for policy 0, policy_version 384689 (0.0006) [2023-12-26 18:09:57,118][105692] Updated weights for policy 0, policy_version 384699 (0.0005) [2023-12-26 18:09:57,181][105692] Updated weights for policy 0, policy_version 384709 (0.0006) [2023-12-26 18:09:57,417][105620] Updated weights for policy 1, policy_version 385260 (0.0009) [2023-12-26 18:09:57,472][105620] Updated weights for policy 1, policy_version 385270 (0.0009) [2023-12-26 18:09:57,527][105620] Updated weights for policy 1, policy_version 385280 (0.0012) [2023-12-26 18:09:57,735][105692] Updated weights for policy 0, policy_version 384719 (0.0005) [2023-12-26 18:09:57,779][105692] Updated weights for policy 0, policy_version 384729 (0.0005) [2023-12-26 18:09:57,831][105692] Updated weights for policy 0, policy_version 384739 (0.0005) [2023-12-26 18:09:58,252][105620] Updated weights for policy 1, policy_version 385291 (0.0010) [2023-12-26 18:09:58,319][105620] Updated weights for policy 1, policy_version 385301 (0.0008) [2023-12-26 18:09:58,390][105620] Updated weights for policy 1, policy_version 385311 (0.0008) [2023-12-26 18:09:58,475][105692] Updated weights for policy 0, policy_version 384749 (0.0007) [2023-12-26 18:09:58,540][105692] Updated weights for policy 0, policy_version 384759 (0.0008) [2023-12-26 18:09:58,600][105692] Updated weights for policy 0, policy_version 384769 (0.0008) [2023-12-26 18:09:59,179][105620] Updated weights for policy 1, policy_version 385321 (0.0008) [2023-12-26 18:09:59,234][105620] Updated weights for policy 1, policy_version 385331 (0.0010) [2023-12-26 18:09:59,293][105620] Updated weights for policy 1, policy_version 385341 (0.0010) [2023-12-26 18:09:59,336][105692] Updated weights for policy 0, policy_version 384779 (0.0008) [2023-12-26 18:09:59,355][105620] Updated weights for policy 1, policy_version 385351 (0.0008) [2023-12-26 18:09:59,392][105692] Updated weights for policy 0, policy_version 384789 (0.0008) [2023-12-26 18:09:59,438][105692] Updated weights for policy 0, policy_version 384799 (0.0008) [2023-12-26 18:10:00,040][105620] Updated weights for policy 1, policy_version 385361 (0.0010) [2023-12-26 18:10:00,097][105620] Updated weights for policy 1, policy_version 385371 (0.0010) [2023-12-26 18:10:00,097][105692] Updated weights for policy 0, policy_version 384809 (0.0008) [2023-12-26 18:10:00,155][105692] Updated weights for policy 0, policy_version 384819 (0.0005) [2023-12-26 18:10:00,155][105620] Updated weights for policy 1, policy_version 385381 (0.0010) [2023-12-26 18:10:00,212][105692] Updated weights for policy 0, policy_version 384829 (0.0005) [2023-12-26 18:10:00,269][105692] Updated weights for policy 0, policy_version 384839 (0.0005) [2023-12-26 18:10:00,823][105692] Updated weights for policy 0, policy_version 384849 (0.0005) [2023-12-26 18:10:00,876][105692] Updated weights for policy 0, policy_version 384859 (0.0005) [2023-12-26 18:10:00,903][105620] Updated weights for policy 1, policy_version 385391 (0.0010) [2023-12-26 18:10:00,931][105692] Updated weights for policy 0, policy_version 384869 (0.0005) [2023-12-26 18:10:00,954][105620] Updated weights for policy 1, policy_version 385401 (0.0010) [2023-12-26 18:10:01,005][105620] Updated weights for policy 1, policy_version 385411 (0.0010) [2023-12-26 18:10:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 197214208. Throughput: 0: 10175.5, 1: 9421.5. Samples: 197175296. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-26 18:10:01,063][104569] Avg episode reward: [(0, '9356.229'), (1, '775.141')] [2023-12-26 18:10:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000384872_98541568.pth... [2023-12-26 18:10:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000385416_98672640.pth... [2023-12-26 18:10:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000384328_98394112.pth [2023-12-26 18:10:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000383688_98238464.pth [2023-12-26 18:10:01,550][105692] Updated weights for policy 0, policy_version 384879 (0.0009) [2023-12-26 18:10:01,607][105692] Updated weights for policy 0, policy_version 384889 (0.0010) [2023-12-26 18:10:01,681][105692] Updated weights for policy 0, policy_version 384899 (0.0011) [2023-12-26 18:10:01,795][105620] Updated weights for policy 1, policy_version 385421 (0.0009) [2023-12-26 18:10:01,851][105620] Updated weights for policy 1, policy_version 385431 (0.0008) [2023-12-26 18:10:01,906][105620] Updated weights for policy 1, policy_version 385441 (0.0008) [2023-12-26 18:10:02,414][105692] Updated weights for policy 0, policy_version 384909 (0.0011) [2023-12-26 18:10:02,472][105692] Updated weights for policy 0, policy_version 384919 (0.0010) [2023-12-26 18:10:02,535][105692] Updated weights for policy 0, policy_version 384929 (0.0008) [2023-12-26 18:10:02,654][105620] Updated weights for policy 1, policy_version 385451 (0.0009) [2023-12-26 18:10:02,706][105620] Updated weights for policy 1, policy_version 385462 (0.0007) [2023-12-26 18:10:02,762][105620] Updated weights for policy 1, policy_version 385472 (0.0008) [2023-12-26 18:10:03,177][105692] Updated weights for policy 0, policy_version 384939 (0.0011) [2023-12-26 18:10:03,231][105692] Updated weights for policy 0, policy_version 384949 (0.0010) [2023-12-26 18:10:03,292][105692] Updated weights for policy 0, policy_version 384959 (0.0010) [2023-12-26 18:10:03,507][105620] Updated weights for policy 1, policy_version 385482 (0.0008) [2023-12-26 18:10:03,565][105620] Updated weights for policy 1, policy_version 385492 (0.0008) [2023-12-26 18:10:03,617][105620] Updated weights for policy 1, policy_version 385502 (0.0008) [2023-12-26 18:10:03,671][105620] Updated weights for policy 1, policy_version 385512 (0.0005) [2023-12-26 18:10:04,033][105692] Updated weights for policy 0, policy_version 384969 (0.0010) [2023-12-26 18:10:04,107][105692] Updated weights for policy 0, policy_version 384979 (0.0006) [2023-12-26 18:10:04,163][105692] Updated weights for policy 0, policy_version 384989 (0.0009) [2023-12-26 18:10:04,212][105692] Updated weights for policy 0, policy_version 384999 (0.0010) [2023-12-26 18:10:04,326][105620] Updated weights for policy 1, policy_version 385522 (0.0008) [2023-12-26 18:10:04,385][105620] Updated weights for policy 1, policy_version 385532 (0.0008) [2023-12-26 18:10:04,442][105620] Updated weights for policy 1, policy_version 385542 (0.0008) [2023-12-26 18:10:04,919][105692] Updated weights for policy 0, policy_version 385009 (0.0010) [2023-12-26 18:10:04,938][105585] KL-divergence is very high: 181.8961 [2023-12-26 18:10:04,943][105585] KL-divergence is very high: 205.7276 [2023-12-26 18:10:04,971][105692] Updated weights for policy 0, policy_version 385019 (0.0010) [2023-12-26 18:10:04,975][105585] KL-divergence is very high: 218.0128 [2023-12-26 18:10:04,980][105585] KL-divergence is very high: 225.2579 [2023-12-26 18:10:05,019][105585] KL-divergence is very high: 186.4809 [2023-12-26 18:10:05,027][105585] KL-divergence is very high: 188.3999 [2023-12-26 18:10:05,027][105692] Updated weights for policy 0, policy_version 385029 (0.0010) [2023-12-26 18:10:05,093][105620] Updated weights for policy 1, policy_version 385552 (0.0006) [2023-12-26 18:10:05,163][105620] Updated weights for policy 1, policy_version 385562 (0.0005) [2023-12-26 18:10:05,234][105620] Updated weights for policy 1, policy_version 385572 (0.0005) [2023-12-26 18:10:05,621][105692] Updated weights for policy 0, policy_version 385039 (0.0007) [2023-12-26 18:10:05,677][105692] Updated weights for policy 0, policy_version 385049 (0.0005) [2023-12-26 18:10:05,747][105692] Updated weights for policy 0, policy_version 385059 (0.0005) [2023-12-26 18:10:05,787][105620] Updated weights for policy 1, policy_version 385582 (0.0006) [2023-12-26 18:10:05,842][105620] Updated weights for policy 1, policy_version 385592 (0.0005) [2023-12-26 18:10:05,901][105620] Updated weights for policy 1, policy_version 385602 (0.0006) [2023-12-26 18:10:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 197312512. Throughput: 0: 10240.6, 1: 9375.5. Samples: 197294700. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-26 18:10:06,063][104569] Avg episode reward: [(0, '9356.158'), (1, '690.068')] [2023-12-26 18:10:06,326][105692] Updated weights for policy 0, policy_version 385069 (0.0008) [2023-12-26 18:10:06,384][105692] Updated weights for policy 0, policy_version 385079 (0.0008) [2023-12-26 18:10:06,451][105692] Updated weights for policy 0, policy_version 385089 (0.0011) [2023-12-26 18:10:06,490][105620] Updated weights for policy 1, policy_version 385612 (0.0006) [2023-12-26 18:10:06,554][105620] Updated weights for policy 1, policy_version 385622 (0.0008) [2023-12-26 18:10:06,618][105620] Updated weights for policy 1, policy_version 385632 (0.0007) [2023-12-26 18:10:07,027][105692] Updated weights for policy 0, policy_version 385099 (0.0008) [2023-12-26 18:10:07,086][105692] Updated weights for policy 0, policy_version 385109 (0.0006) [2023-12-26 18:10:07,151][105692] Updated weights for policy 0, policy_version 385119 (0.0006) [2023-12-26 18:10:07,359][105620] Updated weights for policy 1, policy_version 385642 (0.0011) [2023-12-26 18:10:07,426][105620] Updated weights for policy 1, policy_version 385652 (0.0008) [2023-12-26 18:10:07,483][105620] Updated weights for policy 1, policy_version 385662 (0.0008) [2023-12-26 18:10:07,538][105620] Updated weights for policy 1, policy_version 385672 (0.0008) [2023-12-26 18:10:07,779][105692] Updated weights for policy 0, policy_version 385129 (0.0007) [2023-12-26 18:10:07,835][105692] Updated weights for policy 0, policy_version 385139 (0.0005) [2023-12-26 18:10:07,887][105692] Updated weights for policy 0, policy_version 385149 (0.0005) [2023-12-26 18:10:07,948][105692] Updated weights for policy 0, policy_version 385159 (0.0010) [2023-12-26 18:10:08,270][105620] Updated weights for policy 1, policy_version 385682 (0.0005) [2023-12-26 18:10:08,321][105620] Updated weights for policy 1, policy_version 385692 (0.0005) [2023-12-26 18:10:08,392][105620] Updated weights for policy 1, policy_version 385702 (0.0006) [2023-12-26 18:10:08,608][105692] Updated weights for policy 0, policy_version 385169 (0.0006) [2023-12-26 18:10:08,674][105692] Updated weights for policy 0, policy_version 385179 (0.0006) [2023-12-26 18:10:08,740][105692] Updated weights for policy 0, policy_version 385189 (0.0005) [2023-12-26 18:10:09,207][105620] Updated weights for policy 1, policy_version 385712 (0.0008) [2023-12-26 18:10:09,262][105620] Updated weights for policy 1, policy_version 385722 (0.0008) [2023-12-26 18:10:09,284][105692] Updated weights for policy 0, policy_version 385199 (0.0008) [2023-12-26 18:10:09,327][105620] Updated weights for policy 1, policy_version 385732 (0.0008) [2023-12-26 18:10:09,343][105692] Updated weights for policy 0, policy_version 385209 (0.0008) [2023-12-26 18:10:09,407][105692] Updated weights for policy 0, policy_version 385219 (0.0013) [2023-12-26 18:10:10,144][105692] Updated weights for policy 0, policy_version 385229 (0.0009) [2023-12-26 18:10:10,179][105620] Updated weights for policy 1, policy_version 385742 (0.0006) [2023-12-26 18:10:10,191][105585] KL-divergence is very high: 127.3930 [2023-12-26 18:10:10,197][105585] KL-divergence is very high: 135.1572 [2023-12-26 18:10:10,203][105692] Updated weights for policy 0, policy_version 385239 (0.0008) [2023-12-26 18:10:10,239][105585] KL-divergence is very high: 152.2533 [2023-12-26 18:10:10,239][105620] Updated weights for policy 1, policy_version 385752 (0.0007) [2023-12-26 18:10:10,245][105585] KL-divergence is very high: 147.3817 [2023-12-26 18:10:10,262][105692] Updated weights for policy 0, policy_version 385249 (0.0008) [2023-12-26 18:10:10,288][105585] KL-divergence is very high: 111.7755 [2023-12-26 18:10:10,295][105620] Updated weights for policy 1, policy_version 385762 (0.0008) [2023-12-26 18:10:10,958][105692] Updated weights for policy 0, policy_version 385259 (0.0008) [2023-12-26 18:10:11,023][105692] Updated weights for policy 0, policy_version 385269 (0.0009) [2023-12-26 18:10:11,048][105620] Updated weights for policy 1, policy_version 385772 (0.0007) [2023-12-26 18:10:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 197402624. Throughput: 0: 10300.6, 1: 9347.6. Samples: 197417388. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-26 18:10:11,062][104569] Avg episode reward: [(0, '9355.958'), (1, '686.877')] [2023-12-26 18:10:11,088][105692] Updated weights for policy 0, policy_version 385279 (0.0007) [2023-12-26 18:10:11,115][105620] Updated weights for policy 1, policy_version 385782 (0.0008) [2023-12-26 18:10:11,181][105620] Updated weights for policy 1, policy_version 385792 (0.0008) [2023-12-26 18:10:11,851][105692] Updated weights for policy 0, policy_version 385289 (0.0009) [2023-12-26 18:10:11,889][105620] Updated weights for policy 1, policy_version 385802 (0.0008) [2023-12-26 18:10:11,917][105692] Updated weights for policy 0, policy_version 385299 (0.0009) [2023-12-26 18:10:11,953][105620] Updated weights for policy 1, policy_version 385812 (0.0007) [2023-12-26 18:10:11,976][105692] Updated weights for policy 0, policy_version 385309 (0.0007) [2023-12-26 18:10:12,013][105620] Updated weights for policy 1, policy_version 385822 (0.0009) [2023-12-26 18:10:12,037][105692] Updated weights for policy 0, policy_version 385319 (0.0007) [2023-12-26 18:10:12,068][105620] Updated weights for policy 1, policy_version 385832 (0.0009) [2023-12-26 18:10:12,738][105692] Updated weights for policy 0, policy_version 385329 (0.0005) [2023-12-26 18:10:12,803][105692] Updated weights for policy 0, policy_version 385339 (0.0007) [2023-12-26 18:10:12,862][105620] Updated weights for policy 1, policy_version 385842 (0.0008) [2023-12-26 18:10:12,865][105692] Updated weights for policy 0, policy_version 385349 (0.0006) [2023-12-26 18:10:12,919][105620] Updated weights for policy 1, policy_version 385852 (0.0009) [2023-12-26 18:10:12,965][105620] Updated weights for policy 1, policy_version 385862 (0.0008) [2023-12-26 18:10:13,595][105692] Updated weights for policy 0, policy_version 385359 (0.0009) [2023-12-26 18:10:13,640][105620] Updated weights for policy 1, policy_version 385872 (0.0006) [2023-12-26 18:10:13,651][105692] Updated weights for policy 0, policy_version 385369 (0.0009) [2023-12-26 18:10:13,689][105620] Updated weights for policy 1, policy_version 385882 (0.0005) [2023-12-26 18:10:13,698][105692] Updated weights for policy 0, policy_version 385379 (0.0009) [2023-12-26 18:10:13,738][105620] Updated weights for policy 1, policy_version 385892 (0.0009) [2023-12-26 18:10:14,338][105620] Updated weights for policy 1, policy_version 385902 (0.0007) [2023-12-26 18:10:14,389][105620] Updated weights for policy 1, policy_version 385912 (0.0007) [2023-12-26 18:10:14,444][105620] Updated weights for policy 1, policy_version 385922 (0.0010) [2023-12-26 18:10:14,510][105692] Updated weights for policy 0, policy_version 385389 (0.0008) [2023-12-26 18:10:14,564][105692] Updated weights for policy 0, policy_version 385399 (0.0008) [2023-12-26 18:10:14,627][105692] Updated weights for policy 0, policy_version 385409 (0.0008) [2023-12-26 18:10:15,203][105620] Updated weights for policy 1, policy_version 385932 (0.0010) [2023-12-26 18:10:15,270][105620] Updated weights for policy 1, policy_version 385942 (0.0011) [2023-12-26 18:10:15,309][105692] Updated weights for policy 0, policy_version 385419 (0.0008) [2023-12-26 18:10:15,334][105620] Updated weights for policy 1, policy_version 385952 (0.0011) [2023-12-26 18:10:15,373][105692] Updated weights for policy 0, policy_version 385429 (0.0010) [2023-12-26 18:10:15,441][105692] Updated weights for policy 0, policy_version 385439 (0.0007) [2023-12-26 18:10:16,012][105620] Updated weights for policy 1, policy_version 385962 (0.0011) [2023-12-26 18:10:16,060][105620] Updated weights for policy 1, policy_version 385972 (0.0010) [2023-12-26 18:10:16,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 197500928. Throughput: 0: 10187.5, 1: 9407.5. Samples: 197475272. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-26 18:10:16,062][104569] Avg episode reward: [(0, '9355.956'), (1, '624.527')] [2023-12-26 18:10:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000385448_98689024.pth... [2023-12-26 18:10:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000384264_98385920.pth [2023-12-26 18:10:16,112][105620] Updated weights for policy 1, policy_version 385982 (0.0010) [2023-12-26 18:10:16,135][105692] Updated weights for policy 0, policy_version 385449 (0.0007) [2023-12-26 18:10:16,166][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000385992_98820096.pth... [2023-12-26 18:10:16,167][105620] Updated weights for policy 1, policy_version 385992 (0.0010) [2023-12-26 18:10:16,169][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000384872_98533376.pth [2023-12-26 18:10:16,190][105692] Updated weights for policy 0, policy_version 385459 (0.0007) [2023-12-26 18:10:16,244][105692] Updated weights for policy 0, policy_version 385469 (0.0008) [2023-12-26 18:10:16,306][105692] Updated weights for policy 0, policy_version 385479 (0.0008) [2023-12-26 18:10:16,787][105620] Updated weights for policy 1, policy_version 386002 (0.0005) [2023-12-26 18:10:16,838][105620] Updated weights for policy 1, policy_version 386012 (0.0005) [2023-12-26 18:10:16,898][105620] Updated weights for policy 1, policy_version 386022 (0.0005) [2023-12-26 18:10:17,009][105692] Updated weights for policy 0, policy_version 385489 (0.0006) [2023-12-26 18:10:17,058][105692] Updated weights for policy 0, policy_version 385499 (0.0005) [2023-12-26 18:10:17,110][105692] Updated weights for policy 0, policy_version 385509 (0.0005) [2023-12-26 18:10:17,472][105620] Updated weights for policy 1, policy_version 386032 (0.0005) [2023-12-26 18:10:17,529][105620] Updated weights for policy 1, policy_version 386042 (0.0005) [2023-12-26 18:10:17,577][105620] Updated weights for policy 1, policy_version 386052 (0.0005) [2023-12-26 18:10:17,741][105692] Updated weights for policy 0, policy_version 385519 (0.0007) [2023-12-26 18:10:17,803][105692] Updated weights for policy 0, policy_version 385529 (0.0008) [2023-12-26 18:10:17,870][105692] Updated weights for policy 0, policy_version 385539 (0.0008) [2023-12-26 18:10:18,167][105620] Updated weights for policy 1, policy_version 386062 (0.0008) [2023-12-26 18:10:18,222][105620] Updated weights for policy 1, policy_version 386072 (0.0010) [2023-12-26 18:10:18,270][105620] Updated weights for policy 1, policy_version 386082 (0.0010) [2023-12-26 18:10:18,534][105692] Updated weights for policy 0, policy_version 385549 (0.0007) [2023-12-26 18:10:18,596][105692] Updated weights for policy 0, policy_version 385559 (0.0009) [2023-12-26 18:10:18,648][105692] Updated weights for policy 0, policy_version 385569 (0.0009) [2023-12-26 18:10:19,073][105620] Updated weights for policy 1, policy_version 386092 (0.0010) [2023-12-26 18:10:19,125][105620] Updated weights for policy 1, policy_version 386102 (0.0010) [2023-12-26 18:10:19,178][105620] Updated weights for policy 1, policy_version 386112 (0.0009) [2023-12-26 18:10:19,321][105692] Updated weights for policy 0, policy_version 385579 (0.0009) [2023-12-26 18:10:19,389][105692] Updated weights for policy 0, policy_version 385589 (0.0009) [2023-12-26 18:10:19,451][105692] Updated weights for policy 0, policy_version 385599 (0.0009) [2023-12-26 18:10:19,882][105620] Updated weights for policy 1, policy_version 386122 (0.0007) [2023-12-26 18:10:19,953][105620] Updated weights for policy 1, policy_version 386132 (0.0009) [2023-12-26 18:10:20,016][105620] Updated weights for policy 1, policy_version 386142 (0.0006) [2023-12-26 18:10:20,081][105620] Updated weights for policy 1, policy_version 386152 (0.0006) [2023-12-26 18:10:20,280][105692] Updated weights for policy 0, policy_version 385609 (0.0009) [2023-12-26 18:10:20,346][105692] Updated weights for policy 0, policy_version 385619 (0.0008) [2023-12-26 18:10:20,425][105692] Updated weights for policy 0, policy_version 385629 (0.0010) [2023-12-26 18:10:20,492][105692] Updated weights for policy 0, policy_version 385639 (0.0009) [2023-12-26 18:10:20,717][105620] Updated weights for policy 1, policy_version 386162 (0.0008) [2023-12-26 18:10:20,775][105620] Updated weights for policy 1, policy_version 386172 (0.0006) [2023-12-26 18:10:20,832][105620] Updated weights for policy 1, policy_version 386182 (0.0006) [2023-12-26 18:10:21,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 197607424. Throughput: 0: 10132.4, 1: 9530.6. Samples: 197597200. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-26 18:10:21,063][104569] Avg episode reward: [(0, '9355.061'), (1, '873.799')] [2023-12-26 18:10:21,253][105692] Updated weights for policy 0, policy_version 385649 (0.0009) [2023-12-26 18:10:21,317][105692] Updated weights for policy 0, policy_version 385659 (0.0007) [2023-12-26 18:10:21,383][105692] Updated weights for policy 0, policy_version 385669 (0.0012) [2023-12-26 18:10:21,554][105620] Updated weights for policy 1, policy_version 386192 (0.0008) [2023-12-26 18:10:21,608][105620] Updated weights for policy 1, policy_version 386202 (0.0009) [2023-12-26 18:10:21,678][105620] Updated weights for policy 1, policy_version 386212 (0.0007) [2023-12-26 18:10:22,060][105692] Updated weights for policy 0, policy_version 385679 (0.0006) [2023-12-26 18:10:22,129][105692] Updated weights for policy 0, policy_version 385689 (0.0007) [2023-12-26 18:10:22,188][105692] Updated weights for policy 0, policy_version 385699 (0.0011) [2023-12-26 18:10:22,472][105620] Updated weights for policy 1, policy_version 386222 (0.0009) [2023-12-26 18:10:22,525][105620] Updated weights for policy 1, policy_version 386232 (0.0010) [2023-12-26 18:10:22,582][105620] Updated weights for policy 1, policy_version 386242 (0.0011) [2023-12-26 18:10:22,946][105692] Updated weights for policy 0, policy_version 385709 (0.0011) [2023-12-26 18:10:23,005][105692] Updated weights for policy 0, policy_version 385719 (0.0011) [2023-12-26 18:10:23,070][105692] Updated weights for policy 0, policy_version 385729 (0.0007) [2023-12-26 18:10:23,299][105620] Updated weights for policy 1, policy_version 386252 (0.0010) [2023-12-26 18:10:23,350][105620] Updated weights for policy 1, policy_version 386262 (0.0008) [2023-12-26 18:10:23,413][105620] Updated weights for policy 1, policy_version 386272 (0.0008) [2023-12-26 18:10:23,738][105692] Updated weights for policy 0, policy_version 385739 (0.0009) [2023-12-26 18:10:23,791][105692] Updated weights for policy 0, policy_version 385750 (0.0010) [2023-12-26 18:10:23,848][105692] Updated weights for policy 0, policy_version 385760 (0.0010) [2023-12-26 18:10:24,130][105620] Updated weights for policy 1, policy_version 386282 (0.0008) [2023-12-26 18:10:24,197][105620] Updated weights for policy 1, policy_version 386292 (0.0008) [2023-12-26 18:10:24,259][105620] Updated weights for policy 1, policy_version 386302 (0.0009) [2023-12-26 18:10:24,505][105692] Updated weights for policy 0, policy_version 385770 (0.0008) [2023-12-26 18:10:24,558][105692] Updated weights for policy 0, policy_version 385780 (0.0006) [2023-12-26 18:10:24,615][105692] Updated weights for policy 0, policy_version 385790 (0.0008) [2023-12-26 18:10:24,669][105692] Updated weights for policy 0, policy_version 385800 (0.0005) [2023-12-26 18:10:25,050][105620] Updated weights for policy 1, policy_version 386313 (0.0008) [2023-12-26 18:10:25,100][105620] Updated weights for policy 1, policy_version 386323 (0.0009) [2023-12-26 18:10:25,155][105620] Updated weights for policy 1, policy_version 386333 (0.0008) [2023-12-26 18:10:25,214][105620] Updated weights for policy 1, policy_version 386343 (0.0008) [2023-12-26 18:10:25,345][105692] Updated weights for policy 0, policy_version 385810 (0.0011) [2023-12-26 18:10:25,412][105692] Updated weights for policy 0, policy_version 385820 (0.0010) [2023-12-26 18:10:25,470][105692] Updated weights for policy 0, policy_version 385830 (0.0010) [2023-12-26 18:10:26,012][105620] Updated weights for policy 1, policy_version 386353 (0.0008) [2023-12-26 18:10:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 197697536. Throughput: 0: 10081.8, 1: 9601.6. Samples: 197712244. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-26 18:10:26,062][104569] Avg episode reward: [(0, '9355.249'), (1, '403.483')] [2023-12-26 18:10:26,071][105620] Updated weights for policy 1, policy_version 386363 (0.0008) [2023-12-26 18:10:26,118][105692] Updated weights for policy 0, policy_version 385840 (0.0011) [2023-12-26 18:10:26,132][105620] Updated weights for policy 1, policy_version 386373 (0.0007) [2023-12-26 18:10:26,176][105692] Updated weights for policy 0, policy_version 385850 (0.0010) [2023-12-26 18:10:26,231][105692] Updated weights for policy 0, policy_version 385860 (0.0010) [2023-12-26 18:10:26,877][105620] Updated weights for policy 1, policy_version 386383 (0.0005) [2023-12-26 18:10:26,931][105692] Updated weights for policy 0, policy_version 385870 (0.0010) [2023-12-26 18:10:26,934][105620] Updated weights for policy 1, policy_version 386393 (0.0005) [2023-12-26 18:10:26,975][105692] Updated weights for policy 0, policy_version 385880 (0.0010) [2023-12-26 18:10:26,978][105620] Updated weights for policy 1, policy_version 386403 (0.0005) [2023-12-26 18:10:27,019][105692] Updated weights for policy 0, policy_version 385890 (0.0010) [2023-12-26 18:10:27,529][105620] Updated weights for policy 1, policy_version 386413 (0.0005) [2023-12-26 18:10:27,598][105620] Updated weights for policy 1, policy_version 386423 (0.0005) [2023-12-26 18:10:27,658][105620] Updated weights for policy 1, policy_version 386433 (0.0005) [2023-12-26 18:10:27,709][105692] Updated weights for policy 0, policy_version 385900 (0.0008) [2023-12-26 18:10:27,757][105692] Updated weights for policy 0, policy_version 385910 (0.0005) [2023-12-26 18:10:27,807][105692] Updated weights for policy 0, policy_version 385920 (0.0006) [2023-12-26 18:10:28,348][105692] Updated weights for policy 0, policy_version 385930 (0.0008) [2023-12-26 18:10:28,355][105620] Updated weights for policy 1, policy_version 386443 (0.0006) [2023-12-26 18:10:28,409][105692] Updated weights for policy 0, policy_version 385940 (0.0008) [2023-12-26 18:10:28,411][105620] Updated weights for policy 1, policy_version 386453 (0.0007) [2023-12-26 18:10:28,462][105692] Updated weights for policy 0, policy_version 385950 (0.0006) [2023-12-26 18:10:28,472][105620] Updated weights for policy 1, policy_version 386463 (0.0009) [2023-12-26 18:10:28,522][105692] Updated weights for policy 0, policy_version 385960 (0.0006) [2023-12-26 18:10:29,222][105620] Updated weights for policy 1, policy_version 386473 (0.0008) [2023-12-26 18:10:29,238][105692] Updated weights for policy 0, policy_version 385970 (0.0009) [2023-12-26 18:10:29,282][105620] Updated weights for policy 1, policy_version 386483 (0.0008) [2023-12-26 18:10:29,289][105692] Updated weights for policy 0, policy_version 385980 (0.0006) [2023-12-26 18:10:29,337][105620] Updated weights for policy 1, policy_version 386493 (0.0007) [2023-12-26 18:10:29,340][105692] Updated weights for policy 0, policy_version 385990 (0.0007) [2023-12-26 18:10:29,402][105620] Updated weights for policy 1, policy_version 386503 (0.0007) [2023-12-26 18:10:29,982][105620] Updated weights for policy 1, policy_version 386513 (0.0009) [2023-12-26 18:10:30,043][105620] Updated weights for policy 1, policy_version 386523 (0.0009) [2023-12-26 18:10:30,094][105620] Updated weights for policy 1, policy_version 386533 (0.0008) [2023-12-26 18:10:30,155][105692] Updated weights for policy 0, policy_version 386000 (0.0008) [2023-12-26 18:10:30,203][105692] Updated weights for policy 0, policy_version 386010 (0.0009) [2023-12-26 18:10:30,251][105692] Updated weights for policy 0, policy_version 386020 (0.0007) [2023-12-26 18:10:30,783][105620] Updated weights for policy 1, policy_version 386543 (0.0007) [2023-12-26 18:10:30,847][105620] Updated weights for policy 1, policy_version 386553 (0.0007) [2023-12-26 18:10:30,892][105692] Updated weights for policy 0, policy_version 386030 (0.0008) [2023-12-26 18:10:30,909][105620] Updated weights for policy 1, policy_version 386563 (0.0010) [2023-12-26 18:10:30,954][105692] Updated weights for policy 0, policy_version 386040 (0.0010) [2023-12-26 18:10:31,018][105692] Updated weights for policy 0, policy_version 386050 (0.0009) [2023-12-26 18:10:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 197812224. Throughput: 0: 10160.2, 1: 9635.7. Samples: 197774852. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-26 18:10:31,063][104569] Avg episode reward: [(0, '9355.667'), (1, '434.966')] [2023-12-26 18:10:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000386056_98844672.pth... [2023-12-26 18:10:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000386568_98967552.pth... [2023-12-26 18:10:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000384872_98541568.pth [2023-12-26 18:10:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000385416_98672640.pth [2023-12-26 18:10:31,606][105620] Updated weights for policy 1, policy_version 386573 (0.0008) [2023-12-26 18:10:31,672][105620] Updated weights for policy 1, policy_version 386583 (0.0008) [2023-12-26 18:10:31,735][105692] Updated weights for policy 0, policy_version 386060 (0.0009) [2023-12-26 18:10:31,741][105620] Updated weights for policy 1, policy_version 386593 (0.0008) [2023-12-26 18:10:31,781][105692] Updated weights for policy 0, policy_version 386070 (0.0006) [2023-12-26 18:10:31,836][105692] Updated weights for policy 0, policy_version 386080 (0.0006) [2023-12-26 18:10:32,482][105620] Updated weights for policy 1, policy_version 386603 (0.0009) [2023-12-26 18:10:32,513][105692] Updated weights for policy 0, policy_version 386090 (0.0007) [2023-12-26 18:10:32,538][105620] Updated weights for policy 1, policy_version 386613 (0.0011) [2023-12-26 18:10:32,568][105692] Updated weights for policy 0, policy_version 386100 (0.0006) [2023-12-26 18:10:32,597][105620] Updated weights for policy 1, policy_version 386623 (0.0011) [2023-12-26 18:10:32,627][105692] Updated weights for policy 0, policy_version 386110 (0.0005) [2023-12-26 18:10:32,680][105692] Updated weights for policy 0, policy_version 386120 (0.0007) [2023-12-26 18:10:33,290][105692] Updated weights for policy 0, policy_version 386130 (0.0005) [2023-12-26 18:10:33,341][105692] Updated weights for policy 0, policy_version 386140 (0.0005) [2023-12-26 18:10:33,353][105620] Updated weights for policy 1, policy_version 386633 (0.0010) [2023-12-26 18:10:33,394][105692] Updated weights for policy 0, policy_version 386150 (0.0007) [2023-12-26 18:10:33,408][105620] Updated weights for policy 1, policy_version 386643 (0.0010) [2023-12-26 18:10:33,462][105620] Updated weights for policy 1, policy_version 386653 (0.0009) [2023-12-26 18:10:33,533][105620] Updated weights for policy 1, policy_version 386663 (0.0006) [2023-12-26 18:10:33,908][105692] Updated weights for policy 0, policy_version 386160 (0.0005) [2023-12-26 18:10:33,952][105692] Updated weights for policy 0, policy_version 386170 (0.0005) [2023-12-26 18:10:34,004][105692] Updated weights for policy 0, policy_version 386180 (0.0005) [2023-12-26 18:10:34,188][105620] Updated weights for policy 1, policy_version 386673 (0.0009) [2023-12-26 18:10:34,248][105620] Updated weights for policy 1, policy_version 386683 (0.0008) [2023-12-26 18:10:34,312][105620] Updated weights for policy 1, policy_version 386693 (0.0010) [2023-12-26 18:10:34,702][105692] Updated weights for policy 0, policy_version 386190 (0.0005) [2023-12-26 18:10:34,768][105692] Updated weights for policy 0, policy_version 386200 (0.0007) [2023-12-26 18:10:34,815][105692] Updated weights for policy 0, policy_version 386210 (0.0008) [2023-12-26 18:10:35,038][105620] Updated weights for policy 1, policy_version 386703 (0.0010) [2023-12-26 18:10:35,093][105620] Updated weights for policy 1, policy_version 386713 (0.0009) [2023-12-26 18:10:35,153][105620] Updated weights for policy 1, policy_version 386723 (0.0008) [2023-12-26 18:10:35,516][105692] Updated weights for policy 0, policy_version 386220 (0.0009) [2023-12-26 18:10:35,577][105692] Updated weights for policy 0, policy_version 386230 (0.0009) [2023-12-26 18:10:35,631][105585] KL-divergence is very high: 115.0938 [2023-12-26 18:10:35,647][105692] Updated weights for policy 0, policy_version 386240 (0.0010) [2023-12-26 18:10:35,684][105585] KL-divergence is very high: 128.0330 [2023-12-26 18:10:35,813][105620] Updated weights for policy 1, policy_version 386733 (0.0010) [2023-12-26 18:10:35,858][105620] Updated weights for policy 1, policy_version 386743 (0.0010) [2023-12-26 18:10:35,903][105620] Updated weights for policy 1, policy_version 386753 (0.0010) [2023-12-26 18:10:36,062][104569] Fps is (10 sec: 21298.7, 60 sec: 19797.3, 300 sec: 19605.2). Total num frames: 197910528. Throughput: 0: 10152.6, 1: 9743.0. Samples: 197896768. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-26 18:10:36,063][104569] Avg episode reward: [(0, '9355.535'), (1, '523.070')] [2023-12-26 18:10:36,391][105692] Updated weights for policy 0, policy_version 386250 (0.0010) [2023-12-26 18:10:36,457][105692] Updated weights for policy 0, policy_version 386260 (0.0011) [2023-12-26 18:10:36,527][105692] Updated weights for policy 0, policy_version 386270 (0.0010) [2023-12-26 18:10:36,593][105692] Updated weights for policy 0, policy_version 386280 (0.0010) [2023-12-26 18:10:36,608][105620] Updated weights for policy 1, policy_version 386763 (0.0009) [2023-12-26 18:10:36,674][105620] Updated weights for policy 1, policy_version 386773 (0.0008) [2023-12-26 18:10:36,736][105620] Updated weights for policy 1, policy_version 386783 (0.0007) [2023-12-26 18:10:37,326][105692] Updated weights for policy 0, policy_version 386290 (0.0011) [2023-12-26 18:10:37,378][105692] Updated weights for policy 0, policy_version 386300 (0.0010) [2023-12-26 18:10:37,427][105620] Updated weights for policy 1, policy_version 386793 (0.0006) [2023-12-26 18:10:37,444][105692] Updated weights for policy 0, policy_version 386310 (0.0010) [2023-12-26 18:10:37,481][105620] Updated weights for policy 1, policy_version 386803 (0.0007) [2023-12-26 18:10:37,529][105620] Updated weights for policy 1, policy_version 386813 (0.0008) [2023-12-26 18:10:37,574][105620] Updated weights for policy 1, policy_version 386823 (0.0006) [2023-12-26 18:10:38,179][105692] Updated weights for policy 0, policy_version 386320 (0.0009) [2023-12-26 18:10:38,236][105692] Updated weights for policy 0, policy_version 386330 (0.0006) [2023-12-26 18:10:38,286][105692] Updated weights for policy 0, policy_version 386340 (0.0006) [2023-12-26 18:10:38,347][105620] Updated weights for policy 1, policy_version 386833 (0.0007) [2023-12-26 18:10:38,418][105620] Updated weights for policy 1, policy_version 386843 (0.0006) [2023-12-26 18:10:38,490][105620] Updated weights for policy 1, policy_version 386853 (0.0006) [2023-12-26 18:10:38,937][105692] Updated weights for policy 0, policy_version 386350 (0.0005) [2023-12-26 18:10:39,009][105692] Updated weights for policy 0, policy_version 386360 (0.0005) [2023-12-26 18:10:39,068][105620] Updated weights for policy 1, policy_version 386863 (0.0006) [2023-12-26 18:10:39,070][105692] Updated weights for policy 0, policy_version 386370 (0.0005) [2023-12-26 18:10:39,121][105620] Updated weights for policy 1, policy_version 386873 (0.0009) [2023-12-26 18:10:39,174][105620] Updated weights for policy 1, policy_version 386883 (0.0009) [2023-12-26 18:10:39,691][105692] Updated weights for policy 0, policy_version 386380 (0.0006) [2023-12-26 18:10:39,750][105692] Updated weights for policy 0, policy_version 386390 (0.0011) [2023-12-26 18:10:39,814][105692] Updated weights for policy 0, policy_version 386400 (0.0011) [2023-12-26 18:10:39,996][105620] Updated weights for policy 1, policy_version 386893 (0.0009) [2023-12-26 18:10:40,066][105620] Updated weights for policy 1, policy_version 386903 (0.0008) [2023-12-26 18:10:40,130][105620] Updated weights for policy 1, policy_version 386913 (0.0008) [2023-12-26 18:10:40,554][105692] Updated weights for policy 0, policy_version 386410 (0.0010) [2023-12-26 18:10:40,616][105692] Updated weights for policy 0, policy_version 386420 (0.0011) [2023-12-26 18:10:40,669][105692] Updated weights for policy 0, policy_version 386430 (0.0011) [2023-12-26 18:10:40,717][105692] Updated weights for policy 0, policy_version 386440 (0.0010) [2023-12-26 18:10:40,907][105620] Updated weights for policy 1, policy_version 386923 (0.0009) [2023-12-26 18:10:40,966][105620] Updated weights for policy 1, policy_version 386933 (0.0008) [2023-12-26 18:10:41,041][105620] Updated weights for policy 1, policy_version 386943 (0.0008) [2023-12-26 18:10:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 198000640. Throughput: 0: 10132.5, 1: 9840.6. Samples: 198013384. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-26 18:10:41,062][104569] Avg episode reward: [(0, '9355.527'), (1, '703.031')] [2023-12-26 18:10:41,453][105692] Updated weights for policy 0, policy_version 386450 (0.0009) [2023-12-26 18:10:41,520][105692] Updated weights for policy 0, policy_version 386460 (0.0008) [2023-12-26 18:10:41,582][105692] Updated weights for policy 0, policy_version 386470 (0.0005) [2023-12-26 18:10:41,792][105620] Updated weights for policy 1, policy_version 386953 (0.0008) [2023-12-26 18:10:41,851][105620] Updated weights for policy 1, policy_version 386963 (0.0008) [2023-12-26 18:10:41,911][105620] Updated weights for policy 1, policy_version 386973 (0.0008) [2023-12-26 18:10:41,972][105620] Updated weights for policy 1, policy_version 386983 (0.0008) [2023-12-26 18:10:42,287][105692] Updated weights for policy 0, policy_version 386480 (0.0009) [2023-12-26 18:10:42,315][105585] KL-divergence is very high: 302.4095 [2023-12-26 18:10:42,329][105585] KL-divergence is very high: 161.5557 [2023-12-26 18:10:42,366][105692] Updated weights for policy 0, policy_version 386490 (0.0008) [2023-12-26 18:10:42,383][105585] KL-divergence is very high: 421.8250 [2023-12-26 18:10:42,396][105585] KL-divergence is very high: 162.2609 [2023-12-26 18:10:42,439][105692] Updated weights for policy 0, policy_version 386500 (0.0011) [2023-12-26 18:10:42,441][105585] KL-divergence is very high: 340.2123 [2023-12-26 18:10:42,454][105585] KL-divergence is very high: 118.8342 [2023-12-26 18:10:42,768][105620] Updated weights for policy 1, policy_version 386993 (0.0009) [2023-12-26 18:10:42,827][105620] Updated weights for policy 1, policy_version 387003 (0.0011) [2023-12-26 18:10:42,880][105620] Updated weights for policy 1, policy_version 387013 (0.0010) [2023-12-26 18:10:43,054][105692] Updated weights for policy 0, policy_version 386510 (0.0007) [2023-12-26 18:10:43,115][105692] Updated weights for policy 0, policy_version 386520 (0.0007) [2023-12-26 18:10:43,163][105692] Updated weights for policy 0, policy_version 386530 (0.0010) [2023-12-26 18:10:43,580][105620] Updated weights for policy 1, policy_version 387023 (0.0007) [2023-12-26 18:10:43,643][105620] Updated weights for policy 1, policy_version 387033 (0.0008) [2023-12-26 18:10:43,694][105620] Updated weights for policy 1, policy_version 387043 (0.0008) [2023-12-26 18:10:43,874][105692] Updated weights for policy 0, policy_version 386540 (0.0010) [2023-12-26 18:10:43,925][105692] Updated weights for policy 0, policy_version 386550 (0.0010) [2023-12-26 18:10:43,973][105692] Updated weights for policy 0, policy_version 386560 (0.0010) [2023-12-26 18:10:44,342][105620] Updated weights for policy 1, policy_version 387053 (0.0008) [2023-12-26 18:10:44,396][105620] Updated weights for policy 1, policy_version 387063 (0.0007) [2023-12-26 18:10:44,461][105620] Updated weights for policy 1, policy_version 387073 (0.0008) [2023-12-26 18:10:44,726][105692] Updated weights for policy 0, policy_version 386570 (0.0009) [2023-12-26 18:10:44,786][105692] Updated weights for policy 0, policy_version 386580 (0.0006) [2023-12-26 18:10:44,841][105692] Updated weights for policy 0, policy_version 386590 (0.0006) [2023-12-26 18:10:44,910][105692] Updated weights for policy 0, policy_version 386600 (0.0006) [2023-12-26 18:10:45,093][105620] Updated weights for policy 1, policy_version 387083 (0.0008) [2023-12-26 18:10:45,162][105620] Updated weights for policy 1, policy_version 387093 (0.0006) [2023-12-26 18:10:45,250][105620] Updated weights for policy 1, policy_version 387103 (0.0007) [2023-12-26 18:10:45,503][105692] Updated weights for policy 0, policy_version 386610 (0.0006) [2023-12-26 18:10:45,565][105692] Updated weights for policy 0, policy_version 386620 (0.0007) [2023-12-26 18:10:45,632][105692] Updated weights for policy 0, policy_version 386630 (0.0009) [2023-12-26 18:10:45,889][105620] Updated weights for policy 1, policy_version 387113 (0.0011) [2023-12-26 18:10:45,955][105620] Updated weights for policy 1, policy_version 387123 (0.0005) [2023-12-26 18:10:46,007][105620] Updated weights for policy 1, policy_version 387133 (0.0007) [2023-12-26 18:10:46,054][105620] Updated weights for policy 1, policy_version 387143 (0.0009) [2023-12-26 18:10:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 198107136. Throughput: 0: 10106.1, 1: 9803.1. Samples: 198071212. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-26 18:10:46,063][104569] Avg episode reward: [(0, '9355.126'), (1, '485.704')] [2023-12-26 18:10:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000386632_98992128.pth... [2023-12-26 18:10:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000387144_99115008.pth... [2023-12-26 18:10:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000385448_98689024.pth [2023-12-26 18:10:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000385992_98820096.pth [2023-12-26 18:10:46,223][105692] Updated weights for policy 0, policy_version 386640 (0.0006) [2023-12-26 18:10:46,272][105692] Updated weights for policy 0, policy_version 386650 (0.0005) [2023-12-26 18:10:46,323][105692] Updated weights for policy 0, policy_version 386660 (0.0005) [2023-12-26 18:10:46,786][105620] Updated weights for policy 1, policy_version 387153 (0.0008) [2023-12-26 18:10:46,845][105620] Updated weights for policy 1, policy_version 387163 (0.0005) [2023-12-26 18:10:46,901][105620] Updated weights for policy 1, policy_version 387173 (0.0005) [2023-12-26 18:10:47,015][105692] Updated weights for policy 0, policy_version 386670 (0.0008) [2023-12-26 18:10:47,083][105692] Updated weights for policy 0, policy_version 386680 (0.0010) [2023-12-26 18:10:47,144][105692] Updated weights for policy 0, policy_version 386690 (0.0010) [2023-12-26 18:10:47,592][105620] Updated weights for policy 1, policy_version 387183 (0.0005) [2023-12-26 18:10:47,656][105620] Updated weights for policy 1, policy_version 387193 (0.0005) [2023-12-26 18:10:47,708][105620] Updated weights for policy 1, policy_version 387203 (0.0005) [2023-12-26 18:10:47,807][105692] Updated weights for policy 0, policy_version 386700 (0.0008) [2023-12-26 18:10:47,854][105692] Updated weights for policy 0, policy_version 386710 (0.0005) [2023-12-26 18:10:47,899][105692] Updated weights for policy 0, policy_version 386720 (0.0005) [2023-12-26 18:10:48,354][105620] Updated weights for policy 1, policy_version 387213 (0.0006) [2023-12-26 18:10:48,428][105620] Updated weights for policy 1, policy_version 387223 (0.0006) [2023-12-26 18:10:48,489][105620] Updated weights for policy 1, policy_version 387233 (0.0005) [2023-12-26 18:10:48,503][105692] Updated weights for policy 0, policy_version 386730 (0.0006) [2023-12-26 18:10:48,561][105692] Updated weights for policy 0, policy_version 386740 (0.0009) [2023-12-26 18:10:48,628][105692] Updated weights for policy 0, policy_version 386750 (0.0009) [2023-12-26 18:10:48,691][105692] Updated weights for policy 0, policy_version 386760 (0.0007) [2023-12-26 18:10:49,060][105620] Updated weights for policy 1, policy_version 387243 (0.0005) [2023-12-26 18:10:49,115][105620] Updated weights for policy 1, policy_version 387253 (0.0006) [2023-12-26 18:10:49,171][105620] Updated weights for policy 1, policy_version 387263 (0.0008) [2023-12-26 18:10:49,485][105692] Updated weights for policy 0, policy_version 386770 (0.0005) [2023-12-26 18:10:49,534][105692] Updated weights for policy 0, policy_version 386780 (0.0006) [2023-12-26 18:10:49,584][105692] Updated weights for policy 0, policy_version 386790 (0.0009) [2023-12-26 18:10:50,002][105620] Updated weights for policy 1, policy_version 387273 (0.0008) [2023-12-26 18:10:50,065][105620] Updated weights for policy 1, policy_version 387283 (0.0009) [2023-12-26 18:10:50,127][105620] Updated weights for policy 1, policy_version 387293 (0.0009) [2023-12-26 18:10:50,181][105620] Updated weights for policy 1, policy_version 387303 (0.0009) [2023-12-26 18:10:50,202][105692] Updated weights for policy 0, policy_version 386800 (0.0009) [2023-12-26 18:10:50,260][105692] Updated weights for policy 0, policy_version 386810 (0.0006) [2023-12-26 18:10:50,329][105692] Updated weights for policy 0, policy_version 386820 (0.0010) [2023-12-26 18:10:50,908][105620] Updated weights for policy 1, policy_version 387313 (0.0009) [2023-12-26 18:10:50,974][105620] Updated weights for policy 1, policy_version 387323 (0.0008) [2023-12-26 18:10:51,006][105692] Updated weights for policy 0, policy_version 386830 (0.0008) [2023-12-26 18:10:51,043][105620] Updated weights for policy 1, policy_version 387333 (0.0008) [2023-12-26 18:10:51,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19933.9, 300 sec: 19660.8). Total num frames: 198205440. Throughput: 0: 10110.1, 1: 9878.4. Samples: 198194180. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-26 18:10:51,063][104569] Avg episode reward: [(0, '9355.034'), (1, '476.215')] [2023-12-26 18:10:51,076][105692] Updated weights for policy 0, policy_version 386840 (0.0009) [2023-12-26 18:10:51,150][105692] Updated weights for policy 0, policy_version 386850 (0.0009) [2023-12-26 18:10:51,836][105620] Updated weights for policy 1, policy_version 387343 (0.0008) [2023-12-26 18:10:51,895][105620] Updated weights for policy 1, policy_version 387353 (0.0009) [2023-12-26 18:10:51,948][105692] Updated weights for policy 0, policy_version 386860 (0.0009) [2023-12-26 18:10:51,950][105620] Updated weights for policy 1, policy_version 387363 (0.0007) [2023-12-26 18:10:52,010][105692] Updated weights for policy 0, policy_version 386870 (0.0008) [2023-12-26 18:10:52,073][105692] Updated weights for policy 0, policy_version 386880 (0.0007) [2023-12-26 18:10:52,735][105620] Updated weights for policy 1, policy_version 387373 (0.0008) [2023-12-26 18:10:52,788][105692] Updated weights for policy 0, policy_version 386890 (0.0008) [2023-12-26 18:10:52,794][105620] Updated weights for policy 1, policy_version 387383 (0.0010) [2023-12-26 18:10:52,852][105692] Updated weights for policy 0, policy_version 386900 (0.0007) [2023-12-26 18:10:52,854][105620] Updated weights for policy 1, policy_version 387393 (0.0008) [2023-12-26 18:10:52,910][105692] Updated weights for policy 0, policy_version 386910 (0.0006) [2023-12-26 18:10:52,975][105692] Updated weights for policy 0, policy_version 386920 (0.0006) [2023-12-26 18:10:53,579][105692] Updated weights for policy 0, policy_version 386930 (0.0006) [2023-12-26 18:10:53,638][105692] Updated weights for policy 0, policy_version 386940 (0.0010) [2023-12-26 18:10:53,700][105692] Updated weights for policy 0, policy_version 386950 (0.0007) [2023-12-26 18:10:53,703][105620] Updated weights for policy 1, policy_version 387403 (0.0008) [2023-12-26 18:10:53,762][105620] Updated weights for policy 1, policy_version 387413 (0.0008) [2023-12-26 18:10:53,810][105620] Updated weights for policy 1, policy_version 387423 (0.0008) [2023-12-26 18:10:54,395][105692] Updated weights for policy 0, policy_version 386960 (0.0010) [2023-12-26 18:10:54,452][105692] Updated weights for policy 0, policy_version 386970 (0.0010) [2023-12-26 18:10:54,516][105692] Updated weights for policy 0, policy_version 386980 (0.0008) [2023-12-26 18:10:54,586][105620] Updated weights for policy 1, policy_version 387433 (0.0009) [2023-12-26 18:10:54,639][105620] Updated weights for policy 1, policy_version 387444 (0.0010) [2023-12-26 18:10:54,694][105620] Updated weights for policy 1, policy_version 387455 (0.0011) [2023-12-26 18:10:55,153][105692] Updated weights for policy 0, policy_version 386990 (0.0009) [2023-12-26 18:10:55,161][105585] KL-divergence is very high: 178.0393 [2023-12-26 18:10:55,180][105585] KL-divergence is very high: 266.5484 [2023-12-26 18:10:55,209][105585] KL-divergence is very high: 339.0287 [2023-12-26 18:10:55,214][105692] Updated weights for policy 0, policy_version 387000 (0.0010) [2023-12-26 18:10:55,227][105585] KL-divergence is very high: 336.8246 [2023-12-26 18:10:55,256][105585] KL-divergence is very high: 273.7921 [2023-12-26 18:10:55,274][105585] KL-divergence is very high: 195.4079 [2023-12-26 18:10:55,275][105692] Updated weights for policy 0, policy_version 387010 (0.0010) [2023-12-26 18:10:55,304][105585] KL-divergence is very high: 123.9605 [2023-12-26 18:10:55,521][105620] Updated weights for policy 1, policy_version 387465 (0.0009) [2023-12-26 18:10:55,573][105620] Updated weights for policy 1, policy_version 387475 (0.0007) [2023-12-26 18:10:55,628][105620] Updated weights for policy 1, policy_version 387485 (0.0006) [2023-12-26 18:10:55,693][105620] Updated weights for policy 1, policy_version 387495 (0.0006) [2023-12-26 18:10:55,999][105692] Updated weights for policy 0, policy_version 387020 (0.0010) [2023-12-26 18:10:56,047][105692] Updated weights for policy 0, policy_version 387030 (0.0010) [2023-12-26 18:10:56,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 198295552. Throughput: 0: 10011.2, 1: 9787.6. Samples: 198308332. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-26 18:10:56,062][104569] Avg episode reward: [(0, '9355.131'), (1, '663.681')] [2023-12-26 18:10:56,099][105692] Updated weights for policy 0, policy_version 387040 (0.0010) [2023-12-26 18:10:56,303][105620] Updated weights for policy 1, policy_version 387505 (0.0005) [2023-12-26 18:10:56,367][105620] Updated weights for policy 1, policy_version 387515 (0.0011) [2023-12-26 18:10:56,422][105620] Updated weights for policy 1, policy_version 387525 (0.0005) [2023-12-26 18:10:56,787][105692] Updated weights for policy 0, policy_version 387050 (0.0009) [2023-12-26 18:10:56,842][105692] Updated weights for policy 0, policy_version 387060 (0.0005) [2023-12-26 18:10:56,889][105692] Updated weights for policy 0, policy_version 387070 (0.0008) [2023-12-26 18:10:56,938][105692] Updated weights for policy 0, policy_version 387080 (0.0008) [2023-12-26 18:10:57,043][105620] Updated weights for policy 1, policy_version 387535 (0.0009) [2023-12-26 18:10:57,093][105620] Updated weights for policy 1, policy_version 387545 (0.0010) [2023-12-26 18:10:57,149][105620] Updated weights for policy 1, policy_version 387555 (0.0010) [2023-12-26 18:10:57,634][105692] Updated weights for policy 0, policy_version 387090 (0.0011) [2023-12-26 18:10:57,682][105692] Updated weights for policy 0, policy_version 387100 (0.0010) [2023-12-26 18:10:57,735][105692] Updated weights for policy 0, policy_version 387110 (0.0009) [2023-12-26 18:10:57,837][105620] Updated weights for policy 1, policy_version 387565 (0.0010) [2023-12-26 18:10:57,884][105620] Updated weights for policy 1, policy_version 387575 (0.0010) [2023-12-26 18:10:57,936][105620] Updated weights for policy 1, policy_version 387585 (0.0010) [2023-12-26 18:10:58,427][105692] Updated weights for policy 0, policy_version 387120 (0.0008) [2023-12-26 18:10:58,490][105692] Updated weights for policy 0, policy_version 387130 (0.0010) [2023-12-26 18:10:58,550][105692] Updated weights for policy 0, policy_version 387140 (0.0010) [2023-12-26 18:10:58,672][105620] Updated weights for policy 1, policy_version 387595 (0.0009) [2023-12-26 18:10:58,740][105620] Updated weights for policy 1, policy_version 387605 (0.0008) [2023-12-26 18:10:58,813][105620] Updated weights for policy 1, policy_version 387615 (0.0009) [2023-12-26 18:10:59,398][105692] Updated weights for policy 0, policy_version 387150 (0.0010) [2023-12-26 18:10:59,448][105692] Updated weights for policy 0, policy_version 387160 (0.0009) [2023-12-26 18:10:59,498][105692] Updated weights for policy 0, policy_version 387170 (0.0009) [2023-12-26 18:10:59,621][105620] Updated weights for policy 1, policy_version 387625 (0.0009) [2023-12-26 18:10:59,670][105620] Updated weights for policy 1, policy_version 387635 (0.0009) [2023-12-26 18:10:59,726][105620] Updated weights for policy 1, policy_version 387646 (0.0009) [2023-12-26 18:11:00,189][105692] Updated weights for policy 0, policy_version 387180 (0.0009) [2023-12-26 18:11:00,240][105692] Updated weights for policy 0, policy_version 387190 (0.0009) [2023-12-26 18:11:00,294][105692] Updated weights for policy 0, policy_version 387200 (0.0010) [2023-12-26 18:11:00,474][105620] Updated weights for policy 1, policy_version 387657 (0.0010) [2023-12-26 18:11:00,532][105620] Updated weights for policy 1, policy_version 387667 (0.0007) [2023-12-26 18:11:00,595][105620] Updated weights for policy 1, policy_version 387677 (0.0008) [2023-12-26 18:11:00,649][105620] Updated weights for policy 1, policy_version 387687 (0.0008) [2023-12-26 18:11:00,975][105692] Updated weights for policy 0, policy_version 387210 (0.0007) [2023-12-26 18:11:01,021][105692] Updated weights for policy 0, policy_version 387220 (0.0009) [2023-12-26 18:11:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 198393856. Throughput: 0: 10045.6, 1: 9794.0. Samples: 198368056. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-12-26 18:11:01,062][104569] Avg episode reward: [(0, '9355.151'), (1, '3792.209')] [2023-12-26 18:11:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000387688_99254272.pth... [2023-12-26 18:11:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000386568_98967552.pth [2023-12-26 18:11:01,080][105692] Updated weights for policy 0, policy_version 387230 (0.0009) [2023-12-26 18:11:01,136][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000387240_99147776.pth... [2023-12-26 18:11:01,137][105692] Updated weights for policy 0, policy_version 387240 (0.0008) [2023-12-26 18:11:01,139][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000386056_98844672.pth [2023-12-26 18:11:01,379][105620] Updated weights for policy 1, policy_version 387697 (0.0009) [2023-12-26 18:11:01,432][105620] Updated weights for policy 1, policy_version 387707 (0.0009) [2023-12-26 18:11:01,478][105620] Updated weights for policy 1, policy_version 387717 (0.0009) [2023-12-26 18:11:01,941][105692] Updated weights for policy 0, policy_version 387250 (0.0009) [2023-12-26 18:11:01,997][105692] Updated weights for policy 0, policy_version 387260 (0.0009) [2023-12-26 18:11:02,058][105692] Updated weights for policy 0, policy_version 387270 (0.0009) [2023-12-26 18:11:02,218][105620] Updated weights for policy 1, policy_version 387727 (0.0009) [2023-12-26 18:11:02,279][105620] Updated weights for policy 1, policy_version 387737 (0.0010) [2023-12-26 18:11:02,340][105620] Updated weights for policy 1, policy_version 387747 (0.0009) [2023-12-26 18:11:02,825][105692] Updated weights for policy 0, policy_version 387280 (0.0009) [2023-12-26 18:11:02,881][105692] Updated weights for policy 0, policy_version 387290 (0.0009) [2023-12-26 18:11:02,940][105692] Updated weights for policy 0, policy_version 387300 (0.0009) [2023-12-26 18:11:03,080][105620] Updated weights for policy 1, policy_version 387757 (0.0006) [2023-12-26 18:11:03,135][105620] Updated weights for policy 1, policy_version 387767 (0.0007) [2023-12-26 18:11:03,196][105620] Updated weights for policy 1, policy_version 387777 (0.0008) [2023-12-26 18:11:03,623][105692] Updated weights for policy 0, policy_version 387310 (0.0007) [2023-12-26 18:11:03,681][105692] Updated weights for policy 0, policy_version 387320 (0.0005) [2023-12-26 18:11:03,738][105692] Updated weights for policy 0, policy_version 387330 (0.0005) [2023-12-26 18:11:03,958][105620] Updated weights for policy 1, policy_version 387787 (0.0009) [2023-12-26 18:11:04,013][105620] Updated weights for policy 1, policy_version 387797 (0.0009) [2023-12-26 18:11:04,072][105620] Updated weights for policy 1, policy_version 387807 (0.0009) [2023-12-26 18:11:04,375][105692] Updated weights for policy 0, policy_version 387340 (0.0007) [2023-12-26 18:11:04,433][105692] Updated weights for policy 0, policy_version 387350 (0.0009) [2023-12-26 18:11:04,479][105692] Updated weights for policy 0, policy_version 387360 (0.0008) [2023-12-26 18:11:04,831][105620] Updated weights for policy 1, policy_version 387817 (0.0009) [2023-12-26 18:11:04,881][105620] Updated weights for policy 1, policy_version 387827 (0.0009) [2023-12-26 18:11:04,928][105620] Updated weights for policy 1, policy_version 387838 (0.0009) [2023-12-26 18:11:04,979][105620] Updated weights for policy 1, policy_version 387848 (0.0009) [2023-12-26 18:11:05,226][105692] Updated weights for policy 0, policy_version 387370 (0.0008) [2023-12-26 18:11:05,290][105692] Updated weights for policy 0, policy_version 387380 (0.0007) [2023-12-26 18:11:05,351][105692] Updated weights for policy 0, policy_version 387390 (0.0009) [2023-12-26 18:11:05,418][105692] Updated weights for policy 0, policy_version 387400 (0.0009) [2023-12-26 18:11:05,781][105620] Updated weights for policy 1, policy_version 387858 (0.0009) [2023-12-26 18:11:05,848][105620] Updated weights for policy 1, policy_version 387868 (0.0009) [2023-12-26 18:11:05,916][105620] Updated weights for policy 1, policy_version 387878 (0.0009) [2023-12-26 18:11:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.9, 300 sec: 19633.0). Total num frames: 198492160. Throughput: 0: 10001.7, 1: 9661.6. Samples: 198482048. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-12-26 18:11:06,062][104569] Avg episode reward: [(0, '9355.255'), (1, '6567.509')] [2023-12-26 18:11:06,083][105692] Updated weights for policy 0, policy_version 387410 (0.0008) [2023-12-26 18:11:06,142][105692] Updated weights for policy 0, policy_version 387420 (0.0008) [2023-12-26 18:11:06,201][105692] Updated weights for policy 0, policy_version 387430 (0.0006) [2023-12-26 18:11:06,658][105620] Updated weights for policy 1, policy_version 387888 (0.0007) [2023-12-26 18:11:06,723][105620] Updated weights for policy 1, policy_version 387898 (0.0006) [2023-12-26 18:11:06,790][105620] Updated weights for policy 1, policy_version 387908 (0.0007) [2023-12-26 18:11:06,868][105692] Updated weights for policy 0, policy_version 387440 (0.0008) [2023-12-26 18:11:06,928][105692] Updated weights for policy 0, policy_version 387450 (0.0006) [2023-12-26 18:11:07,002][105692] Updated weights for policy 0, policy_version 387460 (0.0009) [2023-12-26 18:11:07,375][105620] Updated weights for policy 1, policy_version 387918 (0.0009) [2023-12-26 18:11:07,427][105620] Updated weights for policy 1, policy_version 387928 (0.0009) [2023-12-26 18:11:07,484][105620] Updated weights for policy 1, policy_version 387938 (0.0008) [2023-12-26 18:11:07,808][105692] Updated weights for policy 0, policy_version 387470 (0.0009) [2023-12-26 18:11:07,854][105692] Updated weights for policy 0, policy_version 387480 (0.0008) [2023-12-26 18:11:07,905][105692] Updated weights for policy 0, policy_version 387490 (0.0009) [2023-12-26 18:11:08,119][105620] Updated weights for policy 1, policy_version 387948 (0.0008) [2023-12-26 18:11:08,170][105620] Updated weights for policy 1, policy_version 387958 (0.0009) [2023-12-26 18:11:08,223][105620] Updated weights for policy 1, policy_version 387968 (0.0009) [2023-12-26 18:11:08,727][105692] Updated weights for policy 0, policy_version 387501 (0.0008) [2023-12-26 18:11:08,791][105692] Updated weights for policy 0, policy_version 387511 (0.0009) [2023-12-26 18:11:08,850][105692] Updated weights for policy 0, policy_version 387521 (0.0010) [2023-12-26 18:11:08,944][105620] Updated weights for policy 1, policy_version 387978 (0.0008) [2023-12-26 18:11:09,005][105620] Updated weights for policy 1, policy_version 387988 (0.0006) [2023-12-26 18:11:09,071][105620] Updated weights for policy 1, policy_version 387998 (0.0007) [2023-12-26 18:11:09,136][105620] Updated weights for policy 1, policy_version 388008 (0.0009) [2023-12-26 18:11:09,646][105692] Updated weights for policy 0, policy_version 387531 (0.0009) [2023-12-26 18:11:09,705][105692] Updated weights for policy 0, policy_version 387541 (0.0009) [2023-12-26 18:11:09,736][105585] KL-divergence is very high: 112.6953 [2023-12-26 18:11:09,767][105692] Updated weights for policy 0, policy_version 387551 (0.0010) [2023-12-26 18:11:09,867][105620] Updated weights for policy 1, policy_version 388018 (0.0007) [2023-12-26 18:11:09,937][105620] Updated weights for policy 1, policy_version 388028 (0.0008) [2023-12-26 18:11:10,005][105620] Updated weights for policy 1, policy_version 388038 (0.0008) [2023-12-26 18:11:10,527][105692] Updated weights for policy 0, policy_version 387561 (0.0010) [2023-12-26 18:11:10,582][105692] Updated weights for policy 0, policy_version 387571 (0.0010) [2023-12-26 18:11:10,624][105620] Updated weights for policy 1, policy_version 388048 (0.0007) [2023-12-26 18:11:10,634][105692] Updated weights for policy 0, policy_version 387581 (0.0010) [2023-12-26 18:11:10,672][105620] Updated weights for policy 1, policy_version 388058 (0.0005) [2023-12-26 18:11:10,685][105692] Updated weights for policy 0, policy_version 387591 (0.0011) [2023-12-26 18:11:10,722][105620] Updated weights for policy 1, policy_version 388068 (0.0007) [2023-12-26 18:11:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 198590464. Throughput: 0: 9957.6, 1: 9717.3. Samples: 198597616. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-12-26 18:11:11,062][104569] Avg episode reward: [(0, '9355.464'), (1, '7845.210')] [2023-12-26 18:11:11,452][105692] Updated weights for policy 0, policy_version 387601 (0.0011) [2023-12-26 18:11:11,508][105692] Updated weights for policy 0, policy_version 387611 (0.0010) [2023-12-26 18:11:11,536][105620] Updated weights for policy 1, policy_version 388078 (0.0007) [2023-12-26 18:11:11,567][105692] Updated weights for policy 0, policy_version 387621 (0.0010) [2023-12-26 18:11:11,596][105620] Updated weights for policy 1, policy_version 388088 (0.0005) [2023-12-26 18:11:11,665][105620] Updated weights for policy 1, policy_version 388098 (0.0009) [2023-12-26 18:11:12,316][105692] Updated weights for policy 0, policy_version 387631 (0.0008) [2023-12-26 18:11:12,383][105692] Updated weights for policy 0, policy_version 387641 (0.0008) [2023-12-26 18:11:12,450][105620] Updated weights for policy 1, policy_version 388108 (0.0008) [2023-12-26 18:11:12,460][105692] Updated weights for policy 0, policy_version 387651 (0.0006) [2023-12-26 18:11:12,519][105620] Updated weights for policy 1, policy_version 388118 (0.0008) [2023-12-26 18:11:12,583][105620] Updated weights for policy 1, policy_version 388128 (0.0009) [2023-12-26 18:11:13,040][105692] Updated weights for policy 0, policy_version 387661 (0.0006) [2023-12-26 18:11:13,102][105692] Updated weights for policy 0, policy_version 387671 (0.0008) [2023-12-26 18:11:13,151][105692] Updated weights for policy 0, policy_version 387681 (0.0006) [2023-12-26 18:11:13,460][105620] Updated weights for policy 1, policy_version 388138 (0.0008) [2023-12-26 18:11:13,530][105620] Updated weights for policy 1, policy_version 388148 (0.0006) [2023-12-26 18:11:13,592][105620] Updated weights for policy 1, policy_version 388158 (0.0005) [2023-12-26 18:11:13,640][105620] Updated weights for policy 1, policy_version 388168 (0.0005) [2023-12-26 18:11:13,668][105692] Updated weights for policy 0, policy_version 387691 (0.0005) [2023-12-26 18:11:13,733][105692] Updated weights for policy 0, policy_version 387701 (0.0005) [2023-12-26 18:11:13,794][105692] Updated weights for policy 0, policy_version 387711 (0.0005) [2023-12-26 18:11:14,179][105620] Updated weights for policy 1, policy_version 388178 (0.0007) [2023-12-26 18:11:14,250][105620] Updated weights for policy 1, policy_version 388188 (0.0005) [2023-12-26 18:11:14,311][105620] Updated weights for policy 1, policy_version 388198 (0.0006) [2023-12-26 18:11:14,361][105692] Updated weights for policy 0, policy_version 387721 (0.0008) [2023-12-26 18:11:14,409][105692] Updated weights for policy 0, policy_version 387731 (0.0007) [2023-12-26 18:11:14,463][105692] Updated weights for policy 0, policy_version 387741 (0.0005) [2023-12-26 18:11:14,517][105692] Updated weights for policy 0, policy_version 387751 (0.0005) [2023-12-26 18:11:14,883][105620] Updated weights for policy 1, policy_version 388208 (0.0009) [2023-12-26 18:11:14,949][105620] Updated weights for policy 1, policy_version 388218 (0.0007) [2023-12-26 18:11:15,016][105620] Updated weights for policy 1, policy_version 388228 (0.0006) [2023-12-26 18:11:15,204][105692] Updated weights for policy 0, policy_version 387761 (0.0010) [2023-12-26 18:11:15,270][105692] Updated weights for policy 0, policy_version 387771 (0.0006) [2023-12-26 18:11:15,339][105692] Updated weights for policy 0, policy_version 387781 (0.0010) [2023-12-26 18:11:15,641][105620] Updated weights for policy 1, policy_version 388238 (0.0007) [2023-12-26 18:11:15,695][105620] Updated weights for policy 1, policy_version 388248 (0.0008) [2023-12-26 18:11:15,753][105620] Updated weights for policy 1, policy_version 388258 (0.0008) [2023-12-26 18:11:16,042][105692] Updated weights for policy 0, policy_version 387791 (0.0010) [2023-12-26 18:11:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 198688768. Throughput: 0: 9927.9, 1: 9665.3. Samples: 198656544. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-12-26 18:11:16,062][104569] Avg episode reward: [(0, '9355.530'), (1, '840.840')] [2023-12-26 18:11:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000388264_99401728.pth... [2023-12-26 18:11:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000387144_99115008.pth [2023-12-26 18:11:16,106][105692] Updated weights for policy 0, policy_version 387801 (0.0010) [2023-12-26 18:11:16,168][105692] Updated weights for policy 0, policy_version 387811 (0.0010) [2023-12-26 18:11:16,193][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000387816_99295232.pth... [2023-12-26 18:11:16,197][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000386632_98992128.pth [2023-12-26 18:11:16,453][105620] Updated weights for policy 1, policy_version 388268 (0.0008) [2023-12-26 18:11:16,515][105620] Updated weights for policy 1, policy_version 388278 (0.0008) [2023-12-26 18:11:16,576][105620] Updated weights for policy 1, policy_version 388288 (0.0008) [2023-12-26 18:11:16,900][105692] Updated weights for policy 0, policy_version 387821 (0.0010) [2023-12-26 18:11:16,960][105692] Updated weights for policy 0, policy_version 387831 (0.0008) [2023-12-26 18:11:17,021][105692] Updated weights for policy 0, policy_version 387841 (0.0005) [2023-12-26 18:11:17,321][105620] Updated weights for policy 1, policy_version 388298 (0.0008) [2023-12-26 18:11:17,379][105620] Updated weights for policy 1, policy_version 388308 (0.0008) [2023-12-26 18:11:17,435][105620] Updated weights for policy 1, policy_version 388318 (0.0008) [2023-12-26 18:11:17,497][105620] Updated weights for policy 1, policy_version 388328 (0.0008) [2023-12-26 18:11:17,734][105692] Updated weights for policy 0, policy_version 387851 (0.0008) [2023-12-26 18:11:17,798][105692] Updated weights for policy 0, policy_version 387861 (0.0007) [2023-12-26 18:11:17,856][105692] Updated weights for policy 0, policy_version 387871 (0.0005) [2023-12-26 18:11:18,085][105620] Updated weights for policy 1, policy_version 388338 (0.0005) [2023-12-26 18:11:18,154][105620] Updated weights for policy 1, policy_version 388348 (0.0006) [2023-12-26 18:11:18,215][105620] Updated weights for policy 1, policy_version 388358 (0.0005) [2023-12-26 18:11:18,381][105692] Updated weights for policy 0, policy_version 387881 (0.0006) [2023-12-26 18:11:18,440][105692] Updated weights for policy 0, policy_version 387891 (0.0010) [2023-12-26 18:11:18,495][105692] Updated weights for policy 0, policy_version 387901 (0.0010) [2023-12-26 18:11:18,551][105692] Updated weights for policy 0, policy_version 387911 (0.0010) [2023-12-26 18:11:18,775][105620] Updated weights for policy 1, policy_version 388368 (0.0008) [2023-12-26 18:11:18,834][105620] Updated weights for policy 1, policy_version 388378 (0.0008) [2023-12-26 18:11:18,896][105620] Updated weights for policy 1, policy_version 388388 (0.0008) [2023-12-26 18:11:19,305][105692] Updated weights for policy 0, policy_version 387921 (0.0011) [2023-12-26 18:11:19,373][105692] Updated weights for policy 0, policy_version 387931 (0.0009) [2023-12-26 18:11:19,443][105692] Updated weights for policy 0, policy_version 387941 (0.0011) [2023-12-26 18:11:19,620][105620] Updated weights for policy 1, policy_version 388398 (0.0007) [2023-12-26 18:11:19,680][105620] Updated weights for policy 1, policy_version 388408 (0.0008) [2023-12-26 18:11:19,736][105620] Updated weights for policy 1, policy_version 388418 (0.0008) [2023-12-26 18:11:20,161][105692] Updated weights for policy 0, policy_version 387951 (0.0011) [2023-12-26 18:11:20,231][105692] Updated weights for policy 0, policy_version 387961 (0.0011) [2023-12-26 18:11:20,291][105692] Updated weights for policy 0, policy_version 387971 (0.0011) [2023-12-26 18:11:20,455][105620] Updated weights for policy 1, policy_version 388428 (0.0007) [2023-12-26 18:11:20,518][105620] Updated weights for policy 1, policy_version 388438 (0.0005) [2023-12-26 18:11:20,591][105620] Updated weights for policy 1, policy_version 388448 (0.0007) [2023-12-26 18:11:20,973][105692] Updated weights for policy 0, policy_version 387981 (0.0008) [2023-12-26 18:11:21,028][105585] KL-divergence is very high: 110.9619 [2023-12-26 18:11:21,040][105692] Updated weights for policy 0, policy_version 387991 (0.0009) [2023-12-26 18:11:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 198787072. Throughput: 0: 9907.6, 1: 9719.0. Samples: 198779960. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-12-26 18:11:21,063][104569] Avg episode reward: [(0, '9355.765'), (1, '1134.373')] [2023-12-26 18:11:21,111][105692] Updated weights for policy 0, policy_version 388001 (0.0010) [2023-12-26 18:11:21,247][105620] Updated weights for policy 1, policy_version 388458 (0.0008) [2023-12-26 18:11:21,303][105620] Updated weights for policy 1, policy_version 388468 (0.0008) [2023-12-26 18:11:21,374][105620] Updated weights for policy 1, policy_version 388478 (0.0008) [2023-12-26 18:11:21,432][105620] Updated weights for policy 1, policy_version 388488 (0.0008) [2023-12-26 18:11:21,832][105692] Updated weights for policy 0, policy_version 388011 (0.0011) [2023-12-26 18:11:21,890][105692] Updated weights for policy 0, policy_version 388021 (0.0009) [2023-12-26 18:11:21,947][105692] Updated weights for policy 0, policy_version 388031 (0.0010) [2023-12-26 18:11:22,179][105620] Updated weights for policy 1, policy_version 388498 (0.0007) [2023-12-26 18:11:22,242][105620] Updated weights for policy 1, policy_version 388508 (0.0009) [2023-12-26 18:11:22,302][105620] Updated weights for policy 1, policy_version 388518 (0.0008) [2023-12-26 18:11:22,810][105692] Updated weights for policy 0, policy_version 388041 (0.0009) [2023-12-26 18:11:22,876][105692] Updated weights for policy 0, policy_version 388051 (0.0009) [2023-12-26 18:11:22,927][105692] Updated weights for policy 0, policy_version 388061 (0.0009) [2023-12-26 18:11:22,984][105692] Updated weights for policy 0, policy_version 388071 (0.0009) [2023-12-26 18:11:23,018][105620] Updated weights for policy 1, policy_version 388528 (0.0008) [2023-12-26 18:11:23,072][105620] Updated weights for policy 1, policy_version 388538 (0.0009) [2023-12-26 18:11:23,134][105620] Updated weights for policy 1, policy_version 388548 (0.0009) [2023-12-26 18:11:23,740][105692] Updated weights for policy 0, policy_version 388081 (0.0005) [2023-12-26 18:11:23,796][105692] Updated weights for policy 0, policy_version 388091 (0.0005) [2023-12-26 18:11:23,844][105692] Updated weights for policy 0, policy_version 388101 (0.0005) [2023-12-26 18:11:23,907][105620] Updated weights for policy 1, policy_version 388558 (0.0010) [2023-12-26 18:11:23,959][105620] Updated weights for policy 1, policy_version 388568 (0.0010) [2023-12-26 18:11:24,007][105620] Updated weights for policy 1, policy_version 388578 (0.0010) [2023-12-26 18:11:24,460][105692] Updated weights for policy 0, policy_version 388111 (0.0009) [2023-12-26 18:11:24,508][105692] Updated weights for policy 0, policy_version 388121 (0.0010) [2023-12-26 18:11:24,560][105692] Updated weights for policy 0, policy_version 388131 (0.0010) [2023-12-26 18:11:24,757][105620] Updated weights for policy 1, policy_version 388588 (0.0010) [2023-12-26 18:11:24,814][105620] Updated weights for policy 1, policy_version 388598 (0.0010) [2023-12-26 18:11:24,872][105620] Updated weights for policy 1, policy_version 388608 (0.0010) [2023-12-26 18:11:25,308][105692] Updated weights for policy 0, policy_version 388141 (0.0010) [2023-12-26 18:11:25,363][105692] Updated weights for policy 0, policy_version 388151 (0.0010) [2023-12-26 18:11:25,424][105692] Updated weights for policy 0, policy_version 388161 (0.0010) [2023-12-26 18:11:25,596][105620] Updated weights for policy 1, policy_version 388618 (0.0010) [2023-12-26 18:11:25,640][105620] Updated weights for policy 1, policy_version 388628 (0.0010) [2023-12-26 18:11:25,686][105620] Updated weights for policy 1, policy_version 388638 (0.0010) [2023-12-26 18:11:25,738][105620] Updated weights for policy 1, policy_version 388648 (0.0010) [2023-12-26 18:11:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 198885376. Throughput: 0: 9884.3, 1: 9703.6. Samples: 198894836. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-12-26 18:11:26,063][104569] Avg episode reward: [(0, '9355.836'), (1, '6210.167')] [2023-12-26 18:11:26,156][105692] Updated weights for policy 0, policy_version 388171 (0.0010) [2023-12-26 18:11:26,204][105692] Updated weights for policy 0, policy_version 388181 (0.0010) [2023-12-26 18:11:26,248][105692] Updated weights for policy 0, policy_version 388191 (0.0010) [2023-12-26 18:11:26,404][105620] Updated weights for policy 1, policy_version 388658 (0.0008) [2023-12-26 18:11:26,453][105620] Updated weights for policy 1, policy_version 388668 (0.0008) [2023-12-26 18:11:26,503][105620] Updated weights for policy 1, policy_version 388678 (0.0008) [2023-12-26 18:11:27,016][105692] Updated weights for policy 0, policy_version 388201 (0.0010) [2023-12-26 18:11:27,070][105692] Updated weights for policy 0, policy_version 388211 (0.0010) [2023-12-26 18:11:27,127][105692] Updated weights for policy 0, policy_version 388221 (0.0010) [2023-12-26 18:11:27,151][105620] Updated weights for policy 1, policy_version 388688 (0.0008) [2023-12-26 18:11:27,194][105692] Updated weights for policy 0, policy_version 388231 (0.0006) [2023-12-26 18:11:27,197][105620] Updated weights for policy 1, policy_version 388698 (0.0008) [2023-12-26 18:11:27,244][105620] Updated weights for policy 1, policy_version 388708 (0.0008) [2023-12-26 18:11:27,872][105692] Updated weights for policy 0, policy_version 388241 (0.0006) [2023-12-26 18:11:27,922][105692] Updated weights for policy 0, policy_version 388251 (0.0005) [2023-12-26 18:11:27,959][105620] Updated weights for policy 1, policy_version 388718 (0.0008) [2023-12-26 18:11:27,973][105692] Updated weights for policy 0, policy_version 388261 (0.0009) [2023-12-26 18:11:28,013][105620] Updated weights for policy 1, policy_version 388728 (0.0007) [2023-12-26 18:11:28,075][105620] Updated weights for policy 1, policy_version 388738 (0.0008) [2023-12-26 18:11:28,682][105692] Updated weights for policy 0, policy_version 388271 (0.0010) [2023-12-26 18:11:28,737][105692] Updated weights for policy 0, policy_version 388281 (0.0010) [2023-12-26 18:11:28,787][105692] Updated weights for policy 0, policy_version 388291 (0.0010) [2023-12-26 18:11:28,815][105620] Updated weights for policy 1, policy_version 388748 (0.0009) [2023-12-26 18:11:28,874][105620] Updated weights for policy 1, policy_version 388758 (0.0007) [2023-12-26 18:11:28,930][105620] Updated weights for policy 1, policy_version 388768 (0.0008) [2023-12-26 18:11:29,530][105692] Updated weights for policy 0, policy_version 388301 (0.0009) [2023-12-26 18:11:29,597][105692] Updated weights for policy 0, policy_version 388311 (0.0010) [2023-12-26 18:11:29,655][105620] Updated weights for policy 1, policy_version 388778 (0.0007) [2023-12-26 18:11:29,661][105692] Updated weights for policy 0, policy_version 388321 (0.0009) [2023-12-26 18:11:29,704][105620] Updated weights for policy 1, policy_version 388788 (0.0006) [2023-12-26 18:11:29,755][105620] Updated weights for policy 1, policy_version 388798 (0.0009) [2023-12-26 18:11:29,805][105620] Updated weights for policy 1, policy_version 388808 (0.0008) [2023-12-26 18:11:30,365][105692] Updated weights for policy 0, policy_version 388331 (0.0009) [2023-12-26 18:11:30,420][105692] Updated weights for policy 0, policy_version 388341 (0.0010) [2023-12-26 18:11:30,474][105692] Updated weights for policy 0, policy_version 388351 (0.0010) [2023-12-26 18:11:30,617][105620] Updated weights for policy 1, policy_version 388818 (0.0008) [2023-12-26 18:11:30,671][105620] Updated weights for policy 1, policy_version 388828 (0.0008) [2023-12-26 18:11:30,722][105620] Updated weights for policy 1, policy_version 388838 (0.0008) [2023-12-26 18:11:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 198983680. Throughput: 0: 9876.2, 1: 9761.2. Samples: 198954892. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-12-26 18:11:31,063][104569] Avg episode reward: [(0, '9355.832'), (1, '7420.225')] [2023-12-26 18:11:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000388360_99434496.pth... [2023-12-26 18:11:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000388840_99549184.pth... [2023-12-26 18:11:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000387688_99254272.pth [2023-12-26 18:11:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000387240_99147776.pth [2023-12-26 18:11:31,217][105692] Updated weights for policy 0, policy_version 388361 (0.0010) [2023-12-26 18:11:31,282][105692] Updated weights for policy 0, policy_version 388371 (0.0011) [2023-12-26 18:11:31,340][105692] Updated weights for policy 0, policy_version 388381 (0.0010) [2023-12-26 18:11:31,410][105692] Updated weights for policy 0, policy_version 388391 (0.0011) [2023-12-26 18:11:31,472][105620] Updated weights for policy 1, policy_version 388848 (0.0009) [2023-12-26 18:11:31,530][105620] Updated weights for policy 1, policy_version 388859 (0.0011) [2023-12-26 18:11:31,578][105620] Updated weights for policy 1, policy_version 388869 (0.0008) [2023-12-26 18:11:32,067][105692] Updated weights for policy 0, policy_version 388401 (0.0005) [2023-12-26 18:11:32,123][105692] Updated weights for policy 0, policy_version 388411 (0.0005) [2023-12-26 18:11:32,179][105692] Updated weights for policy 0, policy_version 388421 (0.0005) [2023-12-26 18:11:32,416][105620] Updated weights for policy 1, policy_version 388879 (0.0009) [2023-12-26 18:11:32,478][105620] Updated weights for policy 1, policy_version 388889 (0.0010) [2023-12-26 18:11:32,532][105620] Updated weights for policy 1, policy_version 388899 (0.0009) [2023-12-26 18:11:32,731][105692] Updated weights for policy 0, policy_version 388431 (0.0006) [2023-12-26 18:11:32,782][105692] Updated weights for policy 0, policy_version 388441 (0.0009) [2023-12-26 18:11:32,833][105692] Updated weights for policy 0, policy_version 388451 (0.0009) [2023-12-26 18:11:33,367][105620] Updated weights for policy 1, policy_version 388909 (0.0008) [2023-12-26 18:11:33,420][105620] Updated weights for policy 1, policy_version 388919 (0.0009) [2023-12-26 18:11:33,486][105620] Updated weights for policy 1, policy_version 388929 (0.0009) [2023-12-26 18:11:33,515][105692] Updated weights for policy 0, policy_version 388461 (0.0007) [2023-12-26 18:11:33,579][105692] Updated weights for policy 0, policy_version 388471 (0.0008) [2023-12-26 18:11:33,632][105692] Updated weights for policy 0, policy_version 388481 (0.0008) [2023-12-26 18:11:34,242][105692] Updated weights for policy 0, policy_version 388491 (0.0007) [2023-12-26 18:11:34,294][105692] Updated weights for policy 0, policy_version 388501 (0.0011) [2023-12-26 18:11:34,332][105620] Updated weights for policy 1, policy_version 388939 (0.0009) [2023-12-26 18:11:34,355][105692] Updated weights for policy 0, policy_version 388511 (0.0011) [2023-12-26 18:11:34,386][105620] Updated weights for policy 1, policy_version 388949 (0.0006) [2023-12-26 18:11:34,438][105620] Updated weights for policy 1, policy_version 388959 (0.0007) [2023-12-26 18:11:35,114][105692] Updated weights for policy 0, policy_version 388521 (0.0011) [2023-12-26 18:11:35,132][105585] KL-divergence is very high: 162.8057 [2023-12-26 18:11:35,166][105692] Updated weights for policy 0, policy_version 388531 (0.0010) [2023-12-26 18:11:35,175][105585] KL-divergence is very high: 171.9427 [2023-12-26 18:11:35,216][105620] Updated weights for policy 1, policy_version 388969 (0.0008) [2023-12-26 18:11:35,221][105585] KL-divergence is very high: 130.1495 [2023-12-26 18:11:35,222][105692] Updated weights for policy 0, policy_version 388541 (0.0009) [2023-12-26 18:11:35,257][105620] Updated weights for policy 1, policy_version 388979 (0.0007) [2023-12-26 18:11:35,276][105692] Updated weights for policy 0, policy_version 388551 (0.0006) [2023-12-26 18:11:35,306][105620] Updated weights for policy 1, policy_version 388989 (0.0007) [2023-12-26 18:11:35,361][105620] Updated weights for policy 1, policy_version 388999 (0.0009) [2023-12-26 18:11:36,026][105692] Updated weights for policy 0, policy_version 388561 (0.0008) [2023-12-26 18:11:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 199073792. Throughput: 0: 9868.8, 1: 9583.3. Samples: 199069524. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-12-26 18:11:36,062][104569] Avg episode reward: [(0, '9355.579'), (1, '7602.773')] [2023-12-26 18:11:36,080][105692] Updated weights for policy 0, policy_version 388571 (0.0008) [2023-12-26 18:11:36,141][105692] Updated weights for policy 0, policy_version 388581 (0.0008) [2023-12-26 18:11:36,151][105620] Updated weights for policy 1, policy_version 389009 (0.0007) [2023-12-26 18:11:36,217][105620] Updated weights for policy 1, policy_version 389020 (0.0010) [2023-12-26 18:11:36,288][105620] Updated weights for policy 1, policy_version 389030 (0.0010) [2023-12-26 18:11:36,833][105692] Updated weights for policy 0, policy_version 388591 (0.0008) [2023-12-26 18:11:36,892][105692] Updated weights for policy 0, policy_version 388601 (0.0009) [2023-12-26 18:11:36,950][105692] Updated weights for policy 0, policy_version 388611 (0.0008) [2023-12-26 18:11:37,066][105620] Updated weights for policy 1, policy_version 389040 (0.0009) [2023-12-26 18:11:37,123][105620] Updated weights for policy 1, policy_version 389050 (0.0010) [2023-12-26 18:11:37,176][105620] Updated weights for policy 1, policy_version 389060 (0.0010) [2023-12-26 18:11:37,584][105692] Updated weights for policy 0, policy_version 388621 (0.0007) [2023-12-26 18:11:37,631][105692] Updated weights for policy 0, policy_version 388631 (0.0009) [2023-12-26 18:11:37,681][105692] Updated weights for policy 0, policy_version 388641 (0.0008) [2023-12-26 18:11:37,976][105620] Updated weights for policy 1, policy_version 389070 (0.0008) [2023-12-26 18:11:38,028][105620] Updated weights for policy 1, policy_version 389080 (0.0009) [2023-12-26 18:11:38,086][105620] Updated weights for policy 1, policy_version 389090 (0.0009) [2023-12-26 18:11:38,461][105692] Updated weights for policy 0, policy_version 388651 (0.0009) [2023-12-26 18:11:38,518][105692] Updated weights for policy 0, policy_version 388661 (0.0010) [2023-12-26 18:11:38,580][105692] Updated weights for policy 0, policy_version 388671 (0.0009) [2023-12-26 18:11:38,779][105620] Updated weights for policy 1, policy_version 389100 (0.0007) [2023-12-26 18:11:38,826][105620] Updated weights for policy 1, policy_version 389110 (0.0005) [2023-12-26 18:11:38,884][105620] Updated weights for policy 1, policy_version 389120 (0.0005) [2023-12-26 18:11:39,415][105692] Updated weights for policy 0, policy_version 388681 (0.0010) [2023-12-26 18:11:39,468][105620] Updated weights for policy 1, policy_version 389130 (0.0005) [2023-12-26 18:11:39,485][105692] Updated weights for policy 0, policy_version 388691 (0.0007) [2023-12-26 18:11:39,530][105620] Updated weights for policy 1, policy_version 389140 (0.0008) [2023-12-26 18:11:39,552][105692] Updated weights for policy 0, policy_version 388701 (0.0007) [2023-12-26 18:11:39,589][105620] Updated weights for policy 1, policy_version 389150 (0.0009) [2023-12-26 18:11:39,603][105692] Updated weights for policy 0, policy_version 388711 (0.0005) [2023-12-26 18:11:39,648][105620] Updated weights for policy 1, policy_version 389160 (0.0011) [2023-12-26 18:11:40,290][105692] Updated weights for policy 0, policy_version 388721 (0.0006) [2023-12-26 18:11:40,323][105620] Updated weights for policy 1, policy_version 389170 (0.0010) [2023-12-26 18:11:40,347][105692] Updated weights for policy 0, policy_version 388731 (0.0008) [2023-12-26 18:11:40,386][105620] Updated weights for policy 1, policy_version 389180 (0.0010) [2023-12-26 18:11:40,414][105692] Updated weights for policy 0, policy_version 388741 (0.0009) [2023-12-26 18:11:40,449][105620] Updated weights for policy 1, policy_version 389190 (0.0010) [2023-12-26 18:11:41,022][105692] Updated weights for policy 0, policy_version 388751 (0.0008) [2023-12-26 18:11:41,061][105585] KL-divergence is very high: 361.4883 [2023-12-26 18:11:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 199172096. Throughput: 0: 9831.0, 1: 9668.9. Samples: 199185828. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-12-26 18:11:41,062][104569] Avg episode reward: [(0, '9263.333'), (1, '7419.995')] [2023-12-26 18:11:41,089][105692] Updated weights for policy 0, policy_version 388761 (0.0007) [2023-12-26 18:11:41,110][105585] KL-divergence is very high: 208.5368 [2023-12-26 18:11:41,156][105692] Updated weights for policy 0, policy_version 388771 (0.0008) [2023-12-26 18:11:41,177][105620] Updated weights for policy 1, policy_version 389200 (0.0009) [2023-12-26 18:11:41,241][105620] Updated weights for policy 1, policy_version 389210 (0.0010) [2023-12-26 18:11:41,308][105620] Updated weights for policy 1, policy_version 389220 (0.0008) [2023-12-26 18:11:41,867][105692] Updated weights for policy 0, policy_version 388781 (0.0008) [2023-12-26 18:11:41,944][105692] Updated weights for policy 0, policy_version 388791 (0.0011) [2023-12-26 18:11:42,007][105692] Updated weights for policy 0, policy_version 388801 (0.0011) [2023-12-26 18:11:42,040][105620] Updated weights for policy 1, policy_version 389230 (0.0010) [2023-12-26 18:11:42,092][105620] Updated weights for policy 1, policy_version 389240 (0.0010) [2023-12-26 18:11:42,148][105620] Updated weights for policy 1, policy_version 389250 (0.0006) [2023-12-26 18:11:42,642][105692] Updated weights for policy 0, policy_version 388811 (0.0010) [2023-12-26 18:11:42,694][105692] Updated weights for policy 0, policy_version 388821 (0.0011) [2023-12-26 18:11:42,739][105692] Updated weights for policy 0, policy_version 388831 (0.0010) [2023-12-26 18:11:42,872][105620] Updated weights for policy 1, policy_version 389260 (0.0007) [2023-12-26 18:11:42,926][105620] Updated weights for policy 1, policy_version 389270 (0.0010) [2023-12-26 18:11:42,974][105620] Updated weights for policy 1, policy_version 389280 (0.0010) [2023-12-26 18:11:43,504][105692] Updated weights for policy 0, policy_version 388841 (0.0011) [2023-12-26 18:11:43,562][105692] Updated weights for policy 0, policy_version 388851 (0.0008) [2023-12-26 18:11:43,622][105692] Updated weights for policy 0, policy_version 388861 (0.0006) [2023-12-26 18:11:43,624][105620] Updated weights for policy 1, policy_version 389290 (0.0009) [2023-12-26 18:11:43,675][105692] Updated weights for policy 0, policy_version 388871 (0.0007) [2023-12-26 18:11:43,687][105620] Updated weights for policy 1, policy_version 389300 (0.0008) [2023-12-26 18:11:43,746][105620] Updated weights for policy 1, policy_version 389310 (0.0009) [2023-12-26 18:11:43,793][105620] Updated weights for policy 1, policy_version 389320 (0.0009) [2023-12-26 18:11:44,419][105692] Updated weights for policy 0, policy_version 388881 (0.0008) [2023-12-26 18:11:44,454][105620] Updated weights for policy 1, policy_version 389330 (0.0007) [2023-12-26 18:11:44,480][105692] Updated weights for policy 0, policy_version 388891 (0.0007) [2023-12-26 18:11:44,513][105620] Updated weights for policy 1, policy_version 389340 (0.0007) [2023-12-26 18:11:44,536][105692] Updated weights for policy 0, policy_version 388901 (0.0006) [2023-12-26 18:11:44,566][105620] Updated weights for policy 1, policy_version 389350 (0.0006) [2023-12-26 18:11:45,219][105692] Updated weights for policy 0, policy_version 388911 (0.0006) [2023-12-26 18:11:45,281][105692] Updated weights for policy 0, policy_version 388921 (0.0006) [2023-12-26 18:11:45,342][105692] Updated weights for policy 0, policy_version 388931 (0.0006) [2023-12-26 18:11:45,390][105620] Updated weights for policy 1, policy_version 389360 (0.0008) [2023-12-26 18:11:45,453][105620] Updated weights for policy 1, policy_version 389370 (0.0010) [2023-12-26 18:11:45,504][105620] Updated weights for policy 1, policy_version 389380 (0.0008) [2023-12-26 18:11:45,973][105692] Updated weights for policy 0, policy_version 388941 (0.0007) [2023-12-26 18:11:46,022][105692] Updated weights for policy 0, policy_version 388951 (0.0007) [2023-12-26 18:11:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 199270400. Throughput: 0: 9822.1, 1: 9670.5. Samples: 199245224. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-12-26 18:11:46,063][104569] Avg episode reward: [(0, '9263.326'), (1, '7973.342')] [2023-12-26 18:11:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000389384_99688448.pth... [2023-12-26 18:11:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000388264_99401728.pth [2023-12-26 18:11:46,074][105692] Updated weights for policy 0, policy_version 388961 (0.0006) [2023-12-26 18:11:46,119][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000388968_99590144.pth... [2023-12-26 18:11:46,124][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000387816_99295232.pth [2023-12-26 18:11:46,327][105620] Updated weights for policy 1, policy_version 389390 (0.0008) [2023-12-26 18:11:46,390][105620] Updated weights for policy 1, policy_version 389400 (0.0005) [2023-12-26 18:11:46,441][105620] Updated weights for policy 1, policy_version 389410 (0.0008) [2023-12-26 18:11:46,687][105692] Updated weights for policy 0, policy_version 388971 (0.0005) [2023-12-26 18:11:46,743][105585] KL-divergence is very high: 129.5257 [2023-12-26 18:11:46,744][105692] Updated weights for policy 0, policy_version 388981 (0.0006) [2023-12-26 18:11:46,754][105585] KL-divergence is very high: 139.4314 [2023-12-26 18:11:46,783][105585] KL-divergence is very high: 196.0262 [2023-12-26 18:11:46,791][105692] Updated weights for policy 0, policy_version 388991 (0.0006) [2023-12-26 18:11:46,792][105585] KL-divergence is very high: 157.0976 [2023-12-26 18:11:46,821][105585] KL-divergence is very high: 131.0874 [2023-12-26 18:11:47,203][105620] Updated weights for policy 1, policy_version 389420 (0.0007) [2023-12-26 18:11:47,253][105620] Updated weights for policy 1, policy_version 389430 (0.0007) [2023-12-26 18:11:47,297][105620] Updated weights for policy 1, policy_version 389440 (0.0008) [2023-12-26 18:11:47,501][105692] Updated weights for policy 0, policy_version 389001 (0.0010) [2023-12-26 18:11:47,551][105692] Updated weights for policy 0, policy_version 389011 (0.0010) [2023-12-26 18:11:47,608][105692] Updated weights for policy 0, policy_version 389021 (0.0010) [2023-12-26 18:11:47,675][105692] Updated weights for policy 0, policy_version 389031 (0.0007) [2023-12-26 18:11:48,081][105620] Updated weights for policy 1, policy_version 389450 (0.0007) [2023-12-26 18:11:48,143][105620] Updated weights for policy 1, policy_version 389460 (0.0008) [2023-12-26 18:11:48,198][105620] Updated weights for policy 1, policy_version 389470 (0.0008) [2023-12-26 18:11:48,258][105620] Updated weights for policy 1, policy_version 389480 (0.0007) [2023-12-26 18:11:48,327][105692] Updated weights for policy 0, policy_version 389041 (0.0009) [2023-12-26 18:11:48,388][105692] Updated weights for policy 0, policy_version 389051 (0.0010) [2023-12-26 18:11:48,444][105692] Updated weights for policy 0, policy_version 389061 (0.0011) [2023-12-26 18:11:48,846][105620] Updated weights for policy 1, policy_version 389490 (0.0008) [2023-12-26 18:11:48,899][105620] Updated weights for policy 1, policy_version 389500 (0.0006) [2023-12-26 18:11:48,954][105620] Updated weights for policy 1, policy_version 389510 (0.0011) [2023-12-26 18:11:49,199][105692] Updated weights for policy 0, policy_version 389071 (0.0009) [2023-12-26 18:11:49,263][105692] Updated weights for policy 0, policy_version 389081 (0.0008) [2023-12-26 18:11:49,322][105692] Updated weights for policy 0, policy_version 389091 (0.0008) [2023-12-26 18:11:49,678][105620] Updated weights for policy 1, policy_version 389520 (0.0009) [2023-12-26 18:11:49,736][105620] Updated weights for policy 1, policy_version 389530 (0.0006) [2023-12-26 18:11:49,795][105620] Updated weights for policy 1, policy_version 389540 (0.0008) [2023-12-26 18:11:49,999][105692] Updated weights for policy 0, policy_version 389101 (0.0006) [2023-12-26 18:11:50,044][105692] Updated weights for policy 0, policy_version 389111 (0.0006) [2023-12-26 18:11:50,095][105692] Updated weights for policy 0, policy_version 389121 (0.0006) [2023-12-26 18:11:50,590][105620] Updated weights for policy 1, policy_version 389550 (0.0008) [2023-12-26 18:11:50,648][105620] Updated weights for policy 1, policy_version 389560 (0.0009) [2023-12-26 18:11:50,701][105620] Updated weights for policy 1, policy_version 389570 (0.0008) [2023-12-26 18:11:50,790][105692] Updated weights for policy 0, policy_version 389131 (0.0009) [2023-12-26 18:11:50,838][105692] Updated weights for policy 0, policy_version 389141 (0.0009) [2023-12-26 18:11:50,897][105692] Updated weights for policy 0, policy_version 389151 (0.0009) [2023-12-26 18:11:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 199376896. Throughput: 0: 9895.0, 1: 9684.6. Samples: 199363132. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-12-26 18:11:51,063][104569] Avg episode reward: [(0, '9263.535'), (1, '8160.305')] [2023-12-26 18:11:51,404][105620] Updated weights for policy 1, policy_version 389580 (0.0008) [2023-12-26 18:11:51,468][105620] Updated weights for policy 1, policy_version 389590 (0.0008) [2023-12-26 18:11:51,529][105620] Updated weights for policy 1, policy_version 389600 (0.0009) [2023-12-26 18:11:51,721][105692] Updated weights for policy 0, policy_version 389161 (0.0009) [2023-12-26 18:11:51,788][105692] Updated weights for policy 0, policy_version 389171 (0.0010) [2023-12-26 18:11:51,854][105692] Updated weights for policy 0, policy_version 389181 (0.0010) [2023-12-26 18:11:51,916][105692] Updated weights for policy 0, policy_version 389191 (0.0010) [2023-12-26 18:11:52,279][105620] Updated weights for policy 1, policy_version 389610 (0.0008) [2023-12-26 18:11:52,336][105620] Updated weights for policy 1, policy_version 389620 (0.0008) [2023-12-26 18:11:52,402][105620] Updated weights for policy 1, policy_version 389630 (0.0008) [2023-12-26 18:11:52,464][105620] Updated weights for policy 1, policy_version 389640 (0.0008) [2023-12-26 18:11:52,621][105692] Updated weights for policy 0, policy_version 389201 (0.0008) [2023-12-26 18:11:52,683][105692] Updated weights for policy 0, policy_version 389211 (0.0009) [2023-12-26 18:11:52,749][105692] Updated weights for policy 0, policy_version 389221 (0.0008) [2023-12-26 18:11:53,213][105620] Updated weights for policy 1, policy_version 389650 (0.0009) [2023-12-26 18:11:53,263][105620] Updated weights for policy 1, policy_version 389660 (0.0008) [2023-12-26 18:11:53,312][105620] Updated weights for policy 1, policy_version 389670 (0.0010) [2023-12-26 18:11:53,409][105692] Updated weights for policy 0, policy_version 389231 (0.0009) [2023-12-26 18:11:53,469][105692] Updated weights for policy 0, policy_version 389241 (0.0011) [2023-12-26 18:11:53,537][105692] Updated weights for policy 0, policy_version 389251 (0.0011) [2023-12-26 18:11:54,031][105620] Updated weights for policy 1, policy_version 389680 (0.0010) [2023-12-26 18:11:54,093][105620] Updated weights for policy 1, policy_version 389690 (0.0010) [2023-12-26 18:11:54,152][105620] Updated weights for policy 1, policy_version 389700 (0.0010) [2023-12-26 18:11:54,265][105692] Updated weights for policy 0, policy_version 389261 (0.0011) [2023-12-26 18:11:54,301][105585] KL-divergence is very high: 158.7978 [2023-12-26 18:11:54,327][105692] Updated weights for policy 0, policy_version 389271 (0.0011) [2023-12-26 18:11:54,346][105585] KL-divergence is very high: 133.4032 [2023-12-26 18:11:54,383][105692] Updated weights for policy 0, policy_version 389281 (0.0011) [2023-12-26 18:11:54,842][105620] Updated weights for policy 1, policy_version 389710 (0.0010) [2023-12-26 18:11:54,894][105620] Updated weights for policy 1, policy_version 389720 (0.0010) [2023-12-26 18:11:54,954][105620] Updated weights for policy 1, policy_version 389730 (0.0009) [2023-12-26 18:11:55,122][105692] Updated weights for policy 0, policy_version 389291 (0.0011) [2023-12-26 18:11:55,188][105692] Updated weights for policy 0, policy_version 389301 (0.0009) [2023-12-26 18:11:55,242][105692] Updated weights for policy 0, policy_version 389311 (0.0010) [2023-12-26 18:11:55,591][105620] Updated weights for policy 1, policy_version 389740 (0.0006) [2023-12-26 18:11:55,647][105620] Updated weights for policy 1, policy_version 389750 (0.0010) [2023-12-26 18:11:55,705][105620] Updated weights for policy 1, policy_version 389760 (0.0010) [2023-12-26 18:11:55,882][105692] Updated weights for policy 0, policy_version 389321 (0.0007) [2023-12-26 18:11:55,943][105692] Updated weights for policy 0, policy_version 389331 (0.0011) [2023-12-26 18:11:56,001][105692] Updated weights for policy 0, policy_version 389341 (0.0010) [2023-12-26 18:11:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 199467008. Throughput: 0: 9935.3, 1: 9653.1. Samples: 199479092. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-12-26 18:11:56,062][104569] Avg episode reward: [(0, '9355.687'), (1, '8622.637')] [2023-12-26 18:11:56,065][105692] Updated weights for policy 0, policy_version 389351 (0.0010) [2023-12-26 18:11:56,438][105620] Updated weights for policy 1, policy_version 389770 (0.0010) [2023-12-26 18:11:56,488][105620] Updated weights for policy 1, policy_version 389780 (0.0008) [2023-12-26 18:11:56,534][105620] Updated weights for policy 1, policy_version 389790 (0.0009) [2023-12-26 18:11:56,583][105620] Updated weights for policy 1, policy_version 389800 (0.0009) [2023-12-26 18:11:56,759][105692] Updated weights for policy 0, policy_version 389361 (0.0007) [2023-12-26 18:11:56,826][105692] Updated weights for policy 0, policy_version 389371 (0.0006) [2023-12-26 18:11:56,893][105692] Updated weights for policy 0, policy_version 389381 (0.0006) [2023-12-26 18:11:57,380][105620] Updated weights for policy 1, policy_version 389810 (0.0006) [2023-12-26 18:11:57,447][105620] Updated weights for policy 1, policy_version 389820 (0.0005) [2023-12-26 18:11:57,517][105620] Updated weights for policy 1, policy_version 389830 (0.0005) [2023-12-26 18:11:57,560][105692] Updated weights for policy 0, policy_version 389391 (0.0010) [2023-12-26 18:11:57,604][105692] Updated weights for policy 0, policy_version 389401 (0.0010) [2023-12-26 18:11:57,649][105692] Updated weights for policy 0, policy_version 389411 (0.0010) [2023-12-26 18:11:58,130][105620] Updated weights for policy 1, policy_version 389840 (0.0008) [2023-12-26 18:11:58,202][105620] Updated weights for policy 1, policy_version 389850 (0.0009) [2023-12-26 18:11:58,278][105620] Updated weights for policy 1, policy_version 389860 (0.0009) [2023-12-26 18:11:58,360][105692] Updated weights for policy 0, policy_version 389421 (0.0009) [2023-12-26 18:11:58,423][105692] Updated weights for policy 0, policy_version 389431 (0.0008) [2023-12-26 18:11:58,485][105692] Updated weights for policy 0, policy_version 389441 (0.0008) [2023-12-26 18:11:59,051][105620] Updated weights for policy 1, policy_version 389870 (0.0009) [2023-12-26 18:11:59,122][105620] Updated weights for policy 1, policy_version 389880 (0.0008) [2023-12-26 18:11:59,184][105620] Updated weights for policy 1, policy_version 389890 (0.0008) [2023-12-26 18:11:59,302][105692] Updated weights for policy 0, policy_version 389451 (0.0008) [2023-12-26 18:11:59,366][105692] Updated weights for policy 0, policy_version 389461 (0.0009) [2023-12-26 18:11:59,430][105692] Updated weights for policy 0, policy_version 389471 (0.0009) [2023-12-26 18:11:59,908][105620] Updated weights for policy 1, policy_version 389900 (0.0009) [2023-12-26 18:11:59,970][105620] Updated weights for policy 1, policy_version 389910 (0.0006) [2023-12-26 18:12:00,028][105620] Updated weights for policy 1, policy_version 389920 (0.0006) [2023-12-26 18:12:00,204][105692] Updated weights for policy 0, policy_version 389481 (0.0009) [2023-12-26 18:12:00,266][105692] Updated weights for policy 0, policy_version 389492 (0.0010) [2023-12-26 18:12:00,326][105692] Updated weights for policy 0, policy_version 389502 (0.0009) [2023-12-26 18:12:00,380][105692] Updated weights for policy 0, policy_version 389512 (0.0010) [2023-12-26 18:12:00,647][105620] Updated weights for policy 1, policy_version 389930 (0.0007) [2023-12-26 18:12:00,701][105620] Updated weights for policy 1, policy_version 389940 (0.0008) [2023-12-26 18:12:00,749][105620] Updated weights for policy 1, policy_version 389950 (0.0008) [2023-12-26 18:12:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19633.1). Total num frames: 199565312. Throughput: 0: 9913.0, 1: 9673.0. Samples: 199537920. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-12-26 18:12:01,063][104569] Avg episode reward: [(0, '9355.628'), (1, '8529.998')] [2023-12-26 18:12:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000389960_99835904.pth... [2023-12-26 18:12:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000388840_99549184.pth [2023-12-26 18:12:01,090][105692] Updated weights for policy 0, policy_version 389522 (0.0009) [2023-12-26 18:12:01,149][105692] Updated weights for policy 0, policy_version 389532 (0.0010) [2023-12-26 18:12:01,207][105585] KL-divergence is very high: 106.9701 [2023-12-26 18:12:01,208][105692] Updated weights for policy 0, policy_version 389542 (0.0008) [2023-12-26 18:12:01,216][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000389544_99737600.pth... [2023-12-26 18:12:01,244][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000388360_99434496.pth [2023-12-26 18:12:01,547][105620] Updated weights for policy 1, policy_version 389961 (0.0010) [2023-12-26 18:12:01,604][105620] Updated weights for policy 1, policy_version 389971 (0.0006) [2023-12-26 18:12:01,668][105620] Updated weights for policy 1, policy_version 389981 (0.0008) [2023-12-26 18:12:01,732][105620] Updated weights for policy 1, policy_version 389991 (0.0008) [2023-12-26 18:12:01,953][105692] Updated weights for policy 0, policy_version 389552 (0.0010) [2023-12-26 18:12:02,009][105692] Updated weights for policy 0, policy_version 389562 (0.0010) [2023-12-26 18:12:02,065][105692] Updated weights for policy 0, policy_version 389572 (0.0010) [2023-12-26 18:12:02,476][105620] Updated weights for policy 1, policy_version 390001 (0.0008) [2023-12-26 18:12:02,528][105620] Updated weights for policy 1, policy_version 390011 (0.0008) [2023-12-26 18:12:02,579][105620] Updated weights for policy 1, policy_version 390021 (0.0008) [2023-12-26 18:12:02,815][105692] Updated weights for policy 0, policy_version 389582 (0.0010) [2023-12-26 18:12:02,860][105692] Updated weights for policy 0, policy_version 389592 (0.0010) [2023-12-26 18:12:02,912][105692] Updated weights for policy 0, policy_version 389602 (0.0010) [2023-12-26 18:12:03,354][105620] Updated weights for policy 1, policy_version 390031 (0.0008) [2023-12-26 18:12:03,412][105620] Updated weights for policy 1, policy_version 390041 (0.0008) [2023-12-26 18:12:03,478][105620] Updated weights for policy 1, policy_version 390051 (0.0008) [2023-12-26 18:12:03,653][105692] Updated weights for policy 0, policy_version 389612 (0.0008) [2023-12-26 18:12:03,722][105692] Updated weights for policy 0, policy_version 389622 (0.0005) [2023-12-26 18:12:03,785][105692] Updated weights for policy 0, policy_version 389632 (0.0005) [2023-12-26 18:12:04,243][105620] Updated weights for policy 1, policy_version 390061 (0.0007) [2023-12-26 18:12:04,297][105620] Updated weights for policy 1, policy_version 390071 (0.0008) [2023-12-26 18:12:04,356][105620] Updated weights for policy 1, policy_version 390081 (0.0006) [2023-12-26 18:12:04,455][105692] Updated weights for policy 0, policy_version 389642 (0.0007) [2023-12-26 18:12:04,521][105692] Updated weights for policy 0, policy_version 389652 (0.0011) [2023-12-26 18:12:04,590][105692] Updated weights for policy 0, policy_version 389662 (0.0011) [2023-12-26 18:12:04,655][105692] Updated weights for policy 0, policy_version 389672 (0.0010) [2023-12-26 18:12:05,036][105620] Updated weights for policy 1, policy_version 390091 (0.0007) [2023-12-26 18:12:05,094][105620] Updated weights for policy 1, policy_version 390101 (0.0005) [2023-12-26 18:12:05,149][105620] Updated weights for policy 1, policy_version 390111 (0.0005) [2023-12-26 18:12:05,301][105692] Updated weights for policy 0, policy_version 389682 (0.0005) [2023-12-26 18:12:05,345][105692] Updated weights for policy 0, policy_version 389692 (0.0005) [2023-12-26 18:12:05,400][105692] Updated weights for policy 0, policy_version 389702 (0.0005) [2023-12-26 18:12:05,711][105620] Updated weights for policy 1, policy_version 390121 (0.0005) [2023-12-26 18:12:05,772][105620] Updated weights for policy 1, policy_version 390131 (0.0006) [2023-12-26 18:12:05,829][105620] Updated weights for policy 1, policy_version 390141 (0.0005) [2023-12-26 18:12:05,890][105620] Updated weights for policy 1, policy_version 390151 (0.0005) [2023-12-26 18:12:05,975][105692] Updated weights for policy 0, policy_version 389712 (0.0008) [2023-12-26 18:12:06,034][105692] Updated weights for policy 0, policy_version 389722 (0.0009) [2023-12-26 18:12:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 199663616. Throughput: 0: 9808.0, 1: 9571.4. Samples: 199652032. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-12-26 18:12:06,062][104569] Avg episode reward: [(0, '9355.631'), (1, '8251.565')] [2023-12-26 18:12:06,095][105692] Updated weights for policy 0, policy_version 389732 (0.0009) [2023-12-26 18:12:06,532][105620] Updated weights for policy 1, policy_version 390161 (0.0010) [2023-12-26 18:12:06,595][105620] Updated weights for policy 1, policy_version 390171 (0.0011) [2023-12-26 18:12:06,655][105620] Updated weights for policy 1, policy_version 390181 (0.0011) [2023-12-26 18:12:06,845][105692] Updated weights for policy 0, policy_version 389742 (0.0008) [2023-12-26 18:12:06,902][105692] Updated weights for policy 0, policy_version 389752 (0.0008) [2023-12-26 18:12:06,968][105692] Updated weights for policy 0, policy_version 389762 (0.0008) [2023-12-26 18:12:07,382][105620] Updated weights for policy 1, policy_version 390191 (0.0011) [2023-12-26 18:12:07,452][105620] Updated weights for policy 1, policy_version 390201 (0.0010) [2023-12-26 18:12:07,519][105620] Updated weights for policy 1, policy_version 390211 (0.0011) [2023-12-26 18:12:07,715][105692] Updated weights for policy 0, policy_version 389772 (0.0007) [2023-12-26 18:12:07,743][105585] KL-divergence is very high: 165.8508 [2023-12-26 18:12:07,765][105585] KL-divergence is very high: 245.5033 [2023-12-26 18:12:07,780][105585] KL-divergence is very high: 354.7406 [2023-12-26 18:12:07,785][105692] Updated weights for policy 0, policy_version 389782 (0.0005) [2023-12-26 18:12:07,801][105585] KL-divergence is very high: 153.3688 [2023-12-26 18:12:07,834][105585] KL-divergence is very high: 580.2586 [2023-12-26 18:12:07,857][105692] Updated weights for policy 0, policy_version 389792 (0.0009) [2023-12-26 18:12:07,887][105585] KL-divergence is very high: 406.4375 [2023-12-26 18:12:08,124][105620] Updated weights for policy 1, policy_version 390221 (0.0010) [2023-12-26 18:12:08,179][105620] Updated weights for policy 1, policy_version 390231 (0.0010) [2023-12-26 18:12:08,234][105620] Updated weights for policy 1, policy_version 390241 (0.0010) [2023-12-26 18:12:08,475][105692] Updated weights for policy 0, policy_version 389802 (0.0009) [2023-12-26 18:12:08,533][105692] Updated weights for policy 0, policy_version 389812 (0.0006) [2023-12-26 18:12:08,594][105692] Updated weights for policy 0, policy_version 389822 (0.0005) [2023-12-26 18:12:08,661][105692] Updated weights for policy 0, policy_version 389832 (0.0005) [2023-12-26 18:12:08,987][105620] Updated weights for policy 1, policy_version 390251 (0.0011) [2023-12-26 18:12:09,038][105620] Updated weights for policy 1, policy_version 390261 (0.0010) [2023-12-26 18:12:09,096][105620] Updated weights for policy 1, policy_version 390271 (0.0010) [2023-12-26 18:12:09,333][105692] Updated weights for policy 0, policy_version 389842 (0.0006) [2023-12-26 18:12:09,399][105692] Updated weights for policy 0, policy_version 389852 (0.0009) [2023-12-26 18:12:09,457][105692] Updated weights for policy 0, policy_version 389862 (0.0008) [2023-12-26 18:12:09,821][105620] Updated weights for policy 1, policy_version 390281 (0.0010) [2023-12-26 18:12:09,881][105620] Updated weights for policy 1, policy_version 390291 (0.0010) [2023-12-26 18:12:09,905][105586] KL-divergence is very high: 215.3899 [2023-12-26 18:12:09,942][105620] Updated weights for policy 1, policy_version 390301 (0.0008) [2023-12-26 18:12:09,957][105586] KL-divergence is very high: 340.1190 [2023-12-26 18:12:09,964][105586] KL-divergence is very high: 102.6006 [2023-12-26 18:12:10,009][105620] Updated weights for policy 1, policy_version 390311 (0.0006) [2023-12-26 18:12:10,009][105586] KL-divergence is very high: 345.0493 [2023-12-26 18:12:10,236][105692] Updated weights for policy 0, policy_version 389872 (0.0010) [2023-12-26 18:12:10,299][105692] Updated weights for policy 0, policy_version 389882 (0.0011) [2023-12-26 18:12:10,362][105692] Updated weights for policy 0, policy_version 389892 (0.0011) [2023-12-26 18:12:10,680][105620] Updated weights for policy 1, policy_version 390321 (0.0006) [2023-12-26 18:12:10,743][105620] Updated weights for policy 1, policy_version 390331 (0.0011) [2023-12-26 18:12:10,805][105620] Updated weights for policy 1, policy_version 390341 (0.0011) [2023-12-26 18:12:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 199761920. Throughput: 0: 9852.7, 1: 9654.1. Samples: 199772640. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-12-26 18:12:11,062][104569] Avg episode reward: [(0, '9263.428'), (1, '8072.949')] [2023-12-26 18:12:11,156][105692] Updated weights for policy 0, policy_version 389902 (0.0008) [2023-12-26 18:12:11,221][105692] Updated weights for policy 0, policy_version 389912 (0.0007) [2023-12-26 18:12:11,288][105692] Updated weights for policy 0, policy_version 389922 (0.0008) [2023-12-26 18:12:11,555][105620] Updated weights for policy 1, policy_version 390351 (0.0009) [2023-12-26 18:12:11,618][105620] Updated weights for policy 1, policy_version 390361 (0.0009) [2023-12-26 18:12:11,682][105620] Updated weights for policy 1, policy_version 390371 (0.0009) [2023-12-26 18:12:12,031][105692] Updated weights for policy 0, policy_version 389932 (0.0006) [2023-12-26 18:12:12,092][105692] Updated weights for policy 0, policy_version 389942 (0.0007) [2023-12-26 18:12:12,151][105692] Updated weights for policy 0, policy_version 389952 (0.0007) [2023-12-26 18:12:12,396][105620] Updated weights for policy 1, policy_version 390381 (0.0009) [2023-12-26 18:12:12,460][105620] Updated weights for policy 1, policy_version 390391 (0.0008) [2023-12-26 18:12:12,523][105620] Updated weights for policy 1, policy_version 390401 (0.0007) [2023-12-26 18:12:12,859][105692] Updated weights for policy 0, policy_version 389962 (0.0006) [2023-12-26 18:12:12,923][105692] Updated weights for policy 0, policy_version 389972 (0.0006) [2023-12-26 18:12:12,981][105692] Updated weights for policy 0, policy_version 389982 (0.0008) [2023-12-26 18:12:13,039][105692] Updated weights for policy 0, policy_version 389992 (0.0009) [2023-12-26 18:12:13,129][105620] Updated weights for policy 1, policy_version 390411 (0.0005) [2023-12-26 18:12:13,191][105620] Updated weights for policy 1, policy_version 390421 (0.0005) [2023-12-26 18:12:13,253][105620] Updated weights for policy 1, policy_version 390431 (0.0006) [2023-12-26 18:12:13,649][105692] Updated weights for policy 0, policy_version 390002 (0.0008) [2023-12-26 18:12:13,711][105692] Updated weights for policy 0, policy_version 390012 (0.0008) [2023-12-26 18:12:13,758][105692] Updated weights for policy 0, policy_version 390022 (0.0007) [2023-12-26 18:12:13,986][105620] Updated weights for policy 1, policy_version 390442 (0.0010) [2023-12-26 18:12:14,034][105620] Updated weights for policy 1, policy_version 390452 (0.0010) [2023-12-26 18:12:14,093][105620] Updated weights for policy 1, policy_version 390462 (0.0005) [2023-12-26 18:12:14,104][105586] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000005 [2023-12-26 18:12:14,442][105692] Updated weights for policy 0, policy_version 390032 (0.0008) [2023-12-26 18:12:14,509][105692] Updated weights for policy 0, policy_version 390042 (0.0010) [2023-12-26 18:12:14,574][105692] Updated weights for policy 0, policy_version 390052 (0.0009) [2023-12-26 18:12:14,781][105620] Updated weights for policy 1, policy_version 390472 (0.0007) [2023-12-26 18:12:14,843][105620] Updated weights for policy 1, policy_version 390482 (0.0008) [2023-12-26 18:12:14,909][105620] Updated weights for policy 1, policy_version 390492 (0.0008) [2023-12-26 18:12:15,324][105692] Updated weights for policy 0, policy_version 390062 (0.0008) [2023-12-26 18:12:15,384][105692] Updated weights for policy 0, policy_version 390072 (0.0008) [2023-12-26 18:12:15,444][105692] Updated weights for policy 0, policy_version 390082 (0.0009) [2023-12-26 18:12:15,671][105620] Updated weights for policy 1, policy_version 390502 (0.0010) [2023-12-26 18:12:15,719][105620] Updated weights for policy 1, policy_version 390512 (0.0010) [2023-12-26 18:12:15,768][105620] Updated weights for policy 1, policy_version 390522 (0.0010) [2023-12-26 18:12:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 199860224. Throughput: 0: 9854.9, 1: 9623.6. Samples: 199831420. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-12-26 18:12:16,062][104569] Avg episode reward: [(0, '9171.461'), (1, '8345.620')] [2023-12-26 18:12:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000390088_99876864.pth... [2023-12-26 18:12:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000390528_99983360.pth... [2023-12-26 18:12:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000388968_99590144.pth [2023-12-26 18:12:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000389384_99688448.pth [2023-12-26 18:12:16,118][105692] Updated weights for policy 0, policy_version 390092 (0.0007) [2023-12-26 18:12:16,176][105692] Updated weights for policy 0, policy_version 390102 (0.0006) [2023-12-26 18:12:16,229][105692] Updated weights for policy 0, policy_version 390112 (0.0010) [2023-12-26 18:12:16,459][105620] Updated weights for policy 1, policy_version 390532 (0.0009) [2023-12-26 18:12:16,523][105620] Updated weights for policy 1, policy_version 390542 (0.0010) [2023-12-26 18:12:16,588][105620] Updated weights for policy 1, policy_version 390552 (0.0010) [2023-12-26 18:12:16,913][105692] Updated weights for policy 0, policy_version 390122 (0.0006) [2023-12-26 18:12:16,967][105692] Updated weights for policy 0, policy_version 390132 (0.0009) [2023-12-26 18:12:17,020][105692] Updated weights for policy 0, policy_version 390142 (0.0009) [2023-12-26 18:12:17,075][105692] Updated weights for policy 0, policy_version 390152 (0.0009) [2023-12-26 18:12:17,187][105620] Updated weights for policy 1, policy_version 390562 (0.0008) [2023-12-26 18:12:17,233][105620] Updated weights for policy 1, policy_version 390572 (0.0005) [2023-12-26 18:12:17,287][105620] Updated weights for policy 1, policy_version 390582 (0.0005) [2023-12-26 18:12:17,348][105620] Updated weights for policy 1, policy_version 390592 (0.0009) [2023-12-26 18:12:17,943][105620] Updated weights for policy 1, policy_version 390602 (0.0009) [2023-12-26 18:12:17,946][105692] Updated weights for policy 0, policy_version 390162 (0.0009) [2023-12-26 18:12:17,999][105620] Updated weights for policy 1, policy_version 390612 (0.0005) [2023-12-26 18:12:18,000][105692] Updated weights for policy 0, policy_version 390172 (0.0008) [2023-12-26 18:12:18,047][105620] Updated weights for policy 1, policy_version 390622 (0.0005) [2023-12-26 18:12:18,049][105692] Updated weights for policy 0, policy_version 390182 (0.0009) [2023-12-26 18:12:18,619][105620] Updated weights for policy 1, policy_version 390632 (0.0005) [2023-12-26 18:12:18,687][105620] Updated weights for policy 1, policy_version 390642 (0.0005) [2023-12-26 18:12:18,747][105620] Updated weights for policy 1, policy_version 390652 (0.0005) [2023-12-26 18:12:18,961][105692] Updated weights for policy 0, policy_version 390192 (0.0009) [2023-12-26 18:12:19,017][105692] Updated weights for policy 0, policy_version 390202 (0.0009) [2023-12-26 18:12:19,072][105692] Updated weights for policy 0, policy_version 390212 (0.0009) [2023-12-26 18:12:19,325][105620] Updated weights for policy 1, policy_version 390662 (0.0008) [2023-12-26 18:12:19,392][105620] Updated weights for policy 1, policy_version 390672 (0.0009) [2023-12-26 18:12:19,452][105620] Updated weights for policy 1, policy_version 390682 (0.0009) [2023-12-26 18:12:19,852][105692] Updated weights for policy 0, policy_version 390222 (0.0009) [2023-12-26 18:12:19,917][105692] Updated weights for policy 0, policy_version 390232 (0.0008) [2023-12-26 18:12:19,984][105692] Updated weights for policy 0, policy_version 390242 (0.0006) [2023-12-26 18:12:20,223][105620] Updated weights for policy 1, policy_version 390692 (0.0007) [2023-12-26 18:12:20,284][105620] Updated weights for policy 1, policy_version 390702 (0.0005) [2023-12-26 18:12:20,351][105620] Updated weights for policy 1, policy_version 390712 (0.0008) [2023-12-26 18:12:20,734][105692] Updated weights for policy 0, policy_version 390252 (0.0006) [2023-12-26 18:12:20,792][105692] Updated weights for policy 0, policy_version 390262 (0.0006) [2023-12-26 18:12:20,856][105692] Updated weights for policy 0, policy_version 390272 (0.0006) [2023-12-26 18:12:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 199958528. Throughput: 0: 9725.1, 1: 9844.3. Samples: 199950148. Policy #0 lag: (min: 1.0, avg: 21.8, max: 33.0) [2023-12-26 18:12:21,062][104569] Avg episode reward: [(0, '9355.205'), (1, '8253.670')] [2023-12-26 18:12:21,109][105620] Updated weights for policy 1, policy_version 390722 (0.0008) [2023-12-26 18:12:21,171][105620] Updated weights for policy 1, policy_version 390732 (0.0010) [2023-12-26 18:12:21,222][105620] Updated weights for policy 1, policy_version 390742 (0.0010) [2023-12-26 18:12:21,277][105620] Updated weights for policy 1, policy_version 390752 (0.0009) [2023-12-26 18:12:21,477][105692] Updated weights for policy 0, policy_version 390282 (0.0006) [2023-12-26 18:12:21,542][105692] Updated weights for policy 0, policy_version 390292 (0.0008) [2023-12-26 18:12:21,609][105692] Updated weights for policy 0, policy_version 390302 (0.0009) [2023-12-26 18:12:21,676][105692] Updated weights for policy 0, policy_version 390312 (0.0007) [2023-12-26 18:12:22,056][105620] Updated weights for policy 1, policy_version 390762 (0.0009) [2023-12-26 18:12:22,116][105620] Updated weights for policy 1, policy_version 390772 (0.0009) [2023-12-26 18:12:22,179][105620] Updated weights for policy 1, policy_version 390782 (0.0009) [2023-12-26 18:12:22,478][105692] Updated weights for policy 0, policy_version 390322 (0.0008) [2023-12-26 18:12:22,543][105692] Updated weights for policy 0, policy_version 390332 (0.0009) [2023-12-26 18:12:22,598][105692] Updated weights for policy 0, policy_version 390342 (0.0008) [2023-12-26 18:12:22,918][105620] Updated weights for policy 1, policy_version 390792 (0.0010) [2023-12-26 18:12:22,975][105620] Updated weights for policy 1, policy_version 390802 (0.0010) [2023-12-26 18:12:23,030][105586] KL-divergence is very high: 150.6058 [2023-12-26 18:12:23,037][105620] Updated weights for policy 1, policy_version 390812 (0.0010) [2023-12-26 18:12:23,290][105692] Updated weights for policy 0, policy_version 390352 (0.0008) [2023-12-26 18:12:23,347][105692] Updated weights for policy 0, policy_version 390362 (0.0008) [2023-12-26 18:12:23,411][105692] Updated weights for policy 0, policy_version 390372 (0.0008) [2023-12-26 18:12:23,772][105620] Updated weights for policy 1, policy_version 390822 (0.0010) [2023-12-26 18:12:23,827][105620] Updated weights for policy 1, policy_version 390832 (0.0010) [2023-12-26 18:12:23,888][105620] Updated weights for policy 1, policy_version 390842 (0.0010) [2023-12-26 18:12:24,144][105692] Updated weights for policy 0, policy_version 390382 (0.0008) [2023-12-26 18:12:24,199][105692] Updated weights for policy 0, policy_version 390392 (0.0008) [2023-12-26 18:12:24,257][105692] Updated weights for policy 0, policy_version 390402 (0.0006) [2023-12-26 18:12:24,620][105620] Updated weights for policy 1, policy_version 390852 (0.0010) [2023-12-26 18:12:24,678][105620] Updated weights for policy 1, policy_version 390862 (0.0010) [2023-12-26 18:12:24,738][105620] Updated weights for policy 1, policy_version 390872 (0.0010) [2023-12-26 18:12:25,028][105692] Updated weights for policy 0, policy_version 390412 (0.0007) [2023-12-26 18:12:25,088][105692] Updated weights for policy 0, policy_version 390422 (0.0007) [2023-12-26 18:12:25,144][105692] Updated weights for policy 0, policy_version 390432 (0.0008) [2023-12-26 18:12:25,474][105620] Updated weights for policy 1, policy_version 390882 (0.0010) [2023-12-26 18:12:25,521][105620] Updated weights for policy 1, policy_version 390892 (0.0010) [2023-12-26 18:12:25,565][105620] Updated weights for policy 1, policy_version 390902 (0.0010) [2023-12-26 18:12:25,613][105620] Updated weights for policy 1, policy_version 390912 (0.0010) [2023-12-26 18:12:25,775][105692] Updated weights for policy 0, policy_version 390442 (0.0008) [2023-12-26 18:12:25,834][105692] Updated weights for policy 0, policy_version 390452 (0.0008) [2023-12-26 18:12:25,902][105692] Updated weights for policy 0, policy_version 390462 (0.0009) [2023-12-26 18:12:25,954][105692] Updated weights for policy 0, policy_version 390472 (0.0008) [2023-12-26 18:12:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 200056832. Throughput: 0: 9690.6, 1: 9797.2. Samples: 200062780. Policy #0 lag: (min: 1.0, avg: 21.8, max: 33.0) [2023-12-26 18:12:26,062][104569] Avg episode reward: [(0, '9355.205'), (1, '8075.399')] [2023-12-26 18:12:26,388][105620] Updated weights for policy 1, policy_version 390922 (0.0010) [2023-12-26 18:12:26,440][105620] Updated weights for policy 1, policy_version 390932 (0.0010) [2023-12-26 18:12:26,488][105620] Updated weights for policy 1, policy_version 390942 (0.0010) [2023-12-26 18:12:26,717][105692] Updated weights for policy 0, policy_version 390482 (0.0008) [2023-12-26 18:12:26,769][105692] Updated weights for policy 0, policy_version 390492 (0.0008) [2023-12-26 18:12:26,838][105692] Updated weights for policy 0, policy_version 390502 (0.0009) [2023-12-26 18:12:27,188][105620] Updated weights for policy 1, policy_version 390952 (0.0010) [2023-12-26 18:12:27,239][105620] Updated weights for policy 1, policy_version 390962 (0.0010) [2023-12-26 18:12:27,294][105620] Updated weights for policy 1, policy_version 390972 (0.0010) [2023-12-26 18:12:27,552][105692] Updated weights for policy 0, policy_version 390512 (0.0007) [2023-12-26 18:12:27,606][105692] Updated weights for policy 0, policy_version 390522 (0.0005) [2023-12-26 18:12:27,658][105692] Updated weights for policy 0, policy_version 390532 (0.0005) [2023-12-26 18:12:27,975][105620] Updated weights for policy 1, policy_version 390982 (0.0010) [2023-12-26 18:12:28,037][105620] Updated weights for policy 1, policy_version 390992 (0.0010) [2023-12-26 18:12:28,091][105620] Updated weights for policy 1, policy_version 391002 (0.0010) [2023-12-26 18:12:28,268][105585] KL-divergence is very high: 109.3291 [2023-12-26 18:12:28,300][105692] Updated weights for policy 0, policy_version 390542 (0.0007) [2023-12-26 18:12:28,361][105692] Updated weights for policy 0, policy_version 390552 (0.0009) [2023-12-26 18:12:28,424][105692] Updated weights for policy 0, policy_version 390562 (0.0006) [2023-12-26 18:12:28,835][105620] Updated weights for policy 1, policy_version 391012 (0.0010) [2023-12-26 18:12:28,901][105620] Updated weights for policy 1, policy_version 391022 (0.0009) [2023-12-26 18:12:28,951][105620] Updated weights for policy 1, policy_version 391032 (0.0010) [2023-12-26 18:12:29,087][105692] Updated weights for policy 0, policy_version 390572 (0.0007) [2023-12-26 18:12:29,136][105692] Updated weights for policy 0, policy_version 390582 (0.0008) [2023-12-26 18:12:29,188][105692] Updated weights for policy 0, policy_version 390592 (0.0008) [2023-12-26 18:12:29,694][105620] Updated weights for policy 1, policy_version 391042 (0.0010) [2023-12-26 18:12:29,748][105620] Updated weights for policy 1, policy_version 391052 (0.0010) [2023-12-26 18:12:29,810][105620] Updated weights for policy 1, policy_version 391062 (0.0007) [2023-12-26 18:12:29,876][105620] Updated weights for policy 1, policy_version 391072 (0.0007) [2023-12-26 18:12:29,995][105692] Updated weights for policy 0, policy_version 390602 (0.0009) [2023-12-26 18:12:30,054][105692] Updated weights for policy 0, policy_version 390612 (0.0011) [2023-12-26 18:12:30,116][105692] Updated weights for policy 0, policy_version 390622 (0.0009) [2023-12-26 18:12:30,176][105692] Updated weights for policy 0, policy_version 390632 (0.0008) [2023-12-26 18:12:30,488][105620] Updated weights for policy 1, policy_version 391082 (0.0005) [2023-12-26 18:12:30,539][105620] Updated weights for policy 1, policy_version 391092 (0.0009) [2023-12-26 18:12:30,595][105620] Updated weights for policy 1, policy_version 391102 (0.0008) [2023-12-26 18:12:30,835][105692] Updated weights for policy 0, policy_version 390642 (0.0007) [2023-12-26 18:12:30,882][105692] Updated weights for policy 0, policy_version 390652 (0.0008) [2023-12-26 18:12:30,930][105692] Updated weights for policy 0, policy_version 390662 (0.0008) [2023-12-26 18:12:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19660.8). Total num frames: 200155136. Throughput: 0: 9699.9, 1: 9791.1. Samples: 200122320. Policy #0 lag: (min: 1.0, avg: 21.8, max: 33.0) [2023-12-26 18:12:31,063][104569] Avg episode reward: [(0, '9354.997'), (1, '8536.316')] [2023-12-26 18:12:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000390664_100024320.pth... [2023-12-26 18:12:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000391104_100130816.pth... [2023-12-26 18:12:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000389544_99737600.pth [2023-12-26 18:12:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000389960_99835904.pth [2023-12-26 18:12:31,313][105620] Updated weights for policy 1, policy_version 391112 (0.0009) [2023-12-26 18:12:31,397][105620] Updated weights for policy 1, policy_version 391122 (0.0008) [2023-12-26 18:12:31,467][105620] Updated weights for policy 1, policy_version 391132 (0.0009) [2023-12-26 18:12:31,702][105692] Updated weights for policy 0, policy_version 390672 (0.0008) [2023-12-26 18:12:31,759][105692] Updated weights for policy 0, policy_version 390682 (0.0009) [2023-12-26 18:12:31,812][105692] Updated weights for policy 0, policy_version 390692 (0.0008) [2023-12-26 18:12:32,122][105620] Updated weights for policy 1, policy_version 391142 (0.0007) [2023-12-26 18:12:32,173][105620] Updated weights for policy 1, policy_version 391152 (0.0005) [2023-12-26 18:12:32,224][105620] Updated weights for policy 1, policy_version 391162 (0.0006) [2023-12-26 18:12:32,510][105692] Updated weights for policy 0, policy_version 390702 (0.0009) [2023-12-26 18:12:32,568][105692] Updated weights for policy 0, policy_version 390712 (0.0010) [2023-12-26 18:12:32,636][105692] Updated weights for policy 0, policy_version 390722 (0.0010) [2023-12-26 18:12:32,765][105620] Updated weights for policy 1, policy_version 391172 (0.0006) [2023-12-26 18:12:32,815][105620] Updated weights for policy 1, policy_version 391182 (0.0005) [2023-12-26 18:12:32,861][105620] Updated weights for policy 1, policy_version 391192 (0.0005) [2023-12-26 18:12:33,273][105692] Updated weights for policy 0, policy_version 390732 (0.0006) [2023-12-26 18:12:33,327][105692] Updated weights for policy 0, policy_version 390742 (0.0010) [2023-12-26 18:12:33,328][105585] KL-divergence is very high: 315.1244 [2023-12-26 18:12:33,377][105585] KL-divergence is very high: 397.9427 [2023-12-26 18:12:33,390][105692] Updated weights for policy 0, policy_version 390752 (0.0011) [2023-12-26 18:12:33,432][105585] KL-divergence is very high: 339.0368 [2023-12-26 18:12:33,521][105620] Updated weights for policy 1, policy_version 391202 (0.0007) [2023-12-26 18:12:33,573][105620] Updated weights for policy 1, policy_version 391212 (0.0005) [2023-12-26 18:12:33,623][105620] Updated weights for policy 1, policy_version 391222 (0.0005) [2023-12-26 18:12:33,672][105620] Updated weights for policy 1, policy_version 391232 (0.0007) [2023-12-26 18:12:34,007][105692] Updated weights for policy 0, policy_version 390762 (0.0008) [2023-12-26 18:12:34,055][105692] Updated weights for policy 0, policy_version 390772 (0.0010) [2023-12-26 18:12:34,117][105692] Updated weights for policy 0, policy_version 390782 (0.0011) [2023-12-26 18:12:34,180][105692] Updated weights for policy 0, policy_version 390792 (0.0010) [2023-12-26 18:12:34,287][105620] Updated weights for policy 1, policy_version 391242 (0.0008) [2023-12-26 18:12:34,339][105620] Updated weights for policy 1, policy_version 391252 (0.0008) [2023-12-26 18:12:34,396][105620] Updated weights for policy 1, policy_version 391262 (0.0009) [2023-12-26 18:12:34,895][105692] Updated weights for policy 0, policy_version 390802 (0.0008) [2023-12-26 18:12:34,957][105692] Updated weights for policy 0, policy_version 390812 (0.0010) [2023-12-26 18:12:35,022][105692] Updated weights for policy 0, policy_version 390822 (0.0008) [2023-12-26 18:12:35,105][105620] Updated weights for policy 1, policy_version 391272 (0.0007) [2023-12-26 18:12:35,166][105620] Updated weights for policy 1, policy_version 391282 (0.0005) [2023-12-26 18:12:35,232][105620] Updated weights for policy 1, policy_version 391292 (0.0005) [2023-12-26 18:12:35,683][105692] Updated weights for policy 0, policy_version 390832 (0.0006) [2023-12-26 18:12:35,727][105692] Updated weights for policy 0, policy_version 390842 (0.0005) [2023-12-26 18:12:35,771][105692] Updated weights for policy 0, policy_version 390852 (0.0005) [2023-12-26 18:12:35,834][105620] Updated weights for policy 1, policy_version 391302 (0.0005) [2023-12-26 18:12:35,901][105620] Updated weights for policy 1, policy_version 391312 (0.0005) [2023-12-26 18:12:35,957][105620] Updated weights for policy 1, policy_version 391322 (0.0005) [2023-12-26 18:12:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 200261632. Throughput: 0: 9662.6, 1: 9925.1. Samples: 200244580. Policy #0 lag: (min: 1.0, avg: 21.8, max: 33.0) [2023-12-26 18:12:36,063][104569] Avg episode reward: [(0, '9263.605'), (1, '8993.534')] [2023-12-26 18:12:36,443][105692] Updated weights for policy 0, policy_version 390862 (0.0007) [2023-12-26 18:12:36,501][105692] Updated weights for policy 0, policy_version 390872 (0.0009) [2023-12-26 18:12:36,512][105620] Updated weights for policy 1, policy_version 391332 (0.0007) [2023-12-26 18:12:36,568][105692] Updated weights for policy 0, policy_version 390882 (0.0006) [2023-12-26 18:12:36,573][105620] Updated weights for policy 1, policy_version 391342 (0.0009) [2023-12-26 18:12:36,633][105620] Updated weights for policy 1, policy_version 391352 (0.0007) [2023-12-26 18:12:37,235][105692] Updated weights for policy 0, policy_version 390892 (0.0006) [2023-12-26 18:12:37,297][105692] Updated weights for policy 0, policy_version 390902 (0.0011) [2023-12-26 18:12:37,358][105692] Updated weights for policy 0, policy_version 390912 (0.0006) [2023-12-26 18:12:37,410][105620] Updated weights for policy 1, policy_version 391362 (0.0009) [2023-12-26 18:12:37,468][105620] Updated weights for policy 1, policy_version 391372 (0.0010) [2023-12-26 18:12:37,526][105620] Updated weights for policy 1, policy_version 391382 (0.0010) [2023-12-26 18:12:37,586][105620] Updated weights for policy 1, policy_version 391392 (0.0010) [2023-12-26 18:12:38,084][105692] Updated weights for policy 0, policy_version 390922 (0.0006) [2023-12-26 18:12:38,146][105692] Updated weights for policy 0, policy_version 390932 (0.0005) [2023-12-26 18:12:38,207][105692] Updated weights for policy 0, policy_version 390942 (0.0005) [2023-12-26 18:12:38,256][105692] Updated weights for policy 0, policy_version 390952 (0.0005) [2023-12-26 18:12:38,331][105620] Updated weights for policy 1, policy_version 391402 (0.0009) [2023-12-26 18:12:38,398][105620] Updated weights for policy 1, policy_version 391412 (0.0009) [2023-12-26 18:12:38,467][105620] Updated weights for policy 1, policy_version 391423 (0.0007) [2023-12-26 18:12:38,863][105692] Updated weights for policy 0, policy_version 390962 (0.0005) [2023-12-26 18:12:38,911][105692] Updated weights for policy 0, policy_version 390972 (0.0005) [2023-12-26 18:12:38,960][105692] Updated weights for policy 0, policy_version 390982 (0.0005) [2023-12-26 18:12:39,277][105620] Updated weights for policy 1, policy_version 391433 (0.0010) [2023-12-26 18:12:39,334][105620] Updated weights for policy 1, policy_version 391443 (0.0011) [2023-12-26 18:12:39,403][105620] Updated weights for policy 1, policy_version 391454 (0.0010) [2023-12-26 18:12:39,641][105692] Updated weights for policy 0, policy_version 390992 (0.0009) [2023-12-26 18:12:39,671][105585] KL-divergence is very high: 225.9124 [2023-12-26 18:12:39,703][105692] Updated weights for policy 0, policy_version 391002 (0.0011) [2023-12-26 18:12:39,718][105585] KL-divergence is very high: 261.9146 [2023-12-26 18:12:39,759][105692] Updated weights for policy 0, policy_version 391012 (0.0011) [2023-12-26 18:12:39,766][105585] KL-divergence is very high: 159.6038 [2023-12-26 18:12:40,152][105620] Updated weights for policy 1, policy_version 391464 (0.0010) [2023-12-26 18:12:40,208][105620] Updated weights for policy 1, policy_version 391474 (0.0010) [2023-12-26 18:12:40,269][105620] Updated weights for policy 1, policy_version 391484 (0.0010) [2023-12-26 18:12:40,504][105692] Updated weights for policy 0, policy_version 391022 (0.0010) [2023-12-26 18:12:40,575][105692] Updated weights for policy 0, policy_version 391032 (0.0010) [2023-12-26 18:12:40,648][105692] Updated weights for policy 0, policy_version 391042 (0.0010) [2023-12-26 18:12:40,948][105620] Updated weights for policy 1, policy_version 391494 (0.0008) [2023-12-26 18:12:40,997][105620] Updated weights for policy 1, policy_version 391504 (0.0008) [2023-12-26 18:12:41,060][105620] Updated weights for policy 1, policy_version 391514 (0.0008) [2023-12-26 18:12:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 200351744. Throughput: 0: 9734.4, 1: 9937.9. Samples: 200364344. Policy #0 lag: (min: 1.0, avg: 21.8, max: 33.0) [2023-12-26 18:12:41,062][104569] Avg episode reward: [(0, '9171.975'), (1, '8993.840')] [2023-12-26 18:12:41,379][105692] Updated weights for policy 0, policy_version 391052 (0.0009) [2023-12-26 18:12:41,455][105692] Updated weights for policy 0, policy_version 391062 (0.0010) [2023-12-26 18:12:41,512][105692] Updated weights for policy 0, policy_version 391072 (0.0011) [2023-12-26 18:12:41,815][105620] Updated weights for policy 1, policy_version 391524 (0.0008) [2023-12-26 18:12:41,882][105620] Updated weights for policy 1, policy_version 391534 (0.0008) [2023-12-26 18:12:41,935][105620] Updated weights for policy 1, policy_version 391544 (0.0011) [2023-12-26 18:12:42,213][105692] Updated weights for policy 0, policy_version 391082 (0.0011) [2023-12-26 18:12:42,280][105692] Updated weights for policy 0, policy_version 391092 (0.0011) [2023-12-26 18:12:42,346][105692] Updated weights for policy 0, policy_version 391102 (0.0010) [2023-12-26 18:12:42,413][105692] Updated weights for policy 0, policy_version 391112 (0.0007) [2023-12-26 18:12:42,688][105620] Updated weights for policy 1, policy_version 391554 (0.0011) [2023-12-26 18:12:42,754][105620] Updated weights for policy 1, policy_version 391564 (0.0011) [2023-12-26 18:12:42,816][105620] Updated weights for policy 1, policy_version 391574 (0.0010) [2023-12-26 18:12:42,877][105620] Updated weights for policy 1, policy_version 391584 (0.0010) [2023-12-26 18:12:43,120][105692] Updated weights for policy 0, policy_version 391122 (0.0010) [2023-12-26 18:12:43,171][105692] Updated weights for policy 0, policy_version 391132 (0.0010) [2023-12-26 18:12:43,222][105692] Updated weights for policy 0, policy_version 391142 (0.0010) [2023-12-26 18:12:43,519][105620] Updated weights for policy 1, policy_version 391594 (0.0008) [2023-12-26 18:12:43,575][105620] Updated weights for policy 1, policy_version 391604 (0.0006) [2023-12-26 18:12:43,642][105620] Updated weights for policy 1, policy_version 391614 (0.0008) [2023-12-26 18:12:43,963][105692] Updated weights for policy 0, policy_version 391152 (0.0010) [2023-12-26 18:12:44,027][105692] Updated weights for policy 0, policy_version 391162 (0.0009) [2023-12-26 18:12:44,088][105692] Updated weights for policy 0, policy_version 391172 (0.0010) [2023-12-26 18:12:44,187][105620] Updated weights for policy 1, policy_version 391624 (0.0006) [2023-12-26 18:12:44,252][105620] Updated weights for policy 1, policy_version 391634 (0.0006) [2023-12-26 18:12:44,309][105620] Updated weights for policy 1, policy_version 391644 (0.0008) [2023-12-26 18:12:44,817][105692] Updated weights for policy 0, policy_version 391182 (0.0010) [2023-12-26 18:12:44,877][105692] Updated weights for policy 0, policy_version 391192 (0.0010) [2023-12-26 18:12:44,900][105620] Updated weights for policy 1, policy_version 391654 (0.0006) [2023-12-26 18:12:44,940][105692] Updated weights for policy 0, policy_version 391202 (0.0010) [2023-12-26 18:12:44,961][105620] Updated weights for policy 1, policy_version 391664 (0.0006) [2023-12-26 18:12:45,021][105620] Updated weights for policy 1, policy_version 391674 (0.0008) [2023-12-26 18:12:45,588][105620] Updated weights for policy 1, policy_version 391684 (0.0007) [2023-12-26 18:12:45,643][105620] Updated weights for policy 1, policy_version 391694 (0.0010) [2023-12-26 18:12:45,688][105692] Updated weights for policy 0, policy_version 391212 (0.0008) [2023-12-26 18:12:45,695][105620] Updated weights for policy 1, policy_version 391704 (0.0011) [2023-12-26 18:12:45,751][105692] Updated weights for policy 0, policy_version 391222 (0.0005) [2023-12-26 18:12:45,798][105692] Updated weights for policy 0, policy_version 391232 (0.0009) [2023-12-26 18:12:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19688.6). Total num frames: 200458240. Throughput: 0: 9696.1, 1: 9967.7. Samples: 200422792. Policy #0 lag: (min: 1.0, avg: 21.8, max: 33.0) [2023-12-26 18:12:46,062][104569] Avg episode reward: [(0, '9263.972'), (1, '8904.010')] [2023-12-26 18:12:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000391712_100286464.pth... [2023-12-26 18:12:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000391240_100171776.pth... [2023-12-26 18:12:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000390528_99983360.pth [2023-12-26 18:12:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000390088_99876864.pth [2023-12-26 18:12:46,414][105620] Updated weights for policy 1, policy_version 391714 (0.0011) [2023-12-26 18:12:46,454][105692] Updated weights for policy 0, policy_version 391242 (0.0009) [2023-12-26 18:12:46,476][105620] Updated weights for policy 1, policy_version 391724 (0.0010) [2023-12-26 18:12:46,493][105585] KL-divergence is very high: 299.6191 [2023-12-26 18:12:46,499][105585] KL-divergence is very high: 203.0365 [2023-12-26 18:12:46,510][105692] Updated weights for policy 0, policy_version 391252 (0.0006) [2023-12-26 18:12:46,526][105620] Updated weights for policy 1, policy_version 391734 (0.0010) [2023-12-26 18:12:46,543][105585] KL-divergence is very high: 219.0392 [2023-12-26 18:12:46,550][105585] KL-divergence is very high: 120.3961 [2023-12-26 18:12:46,572][105692] Updated weights for policy 0, policy_version 391262 (0.0007) [2023-12-26 18:12:46,592][105620] Updated weights for policy 1, policy_version 391744 (0.0011) [2023-12-26 18:12:46,637][105692] Updated weights for policy 0, policy_version 391272 (0.0006) [2023-12-26 18:12:47,308][105620] Updated weights for policy 1, policy_version 391754 (0.0010) [2023-12-26 18:12:47,355][105620] Updated weights for policy 1, policy_version 391764 (0.0008) [2023-12-26 18:12:47,362][105692] Updated weights for policy 0, policy_version 391282 (0.0006) [2023-12-26 18:12:47,410][105620] Updated weights for policy 1, policy_version 391774 (0.0010) [2023-12-26 18:12:47,417][105692] Updated weights for policy 0, policy_version 391292 (0.0006) [2023-12-26 18:12:47,465][105692] Updated weights for policy 0, policy_version 391302 (0.0008) [2023-12-26 18:12:48,165][105620] Updated weights for policy 1, policy_version 391784 (0.0010) [2023-12-26 18:12:48,221][105620] Updated weights for policy 1, policy_version 391794 (0.0007) [2023-12-26 18:12:48,235][105692] Updated weights for policy 0, policy_version 391312 (0.0010) [2023-12-26 18:12:48,271][105620] Updated weights for policy 1, policy_version 391804 (0.0005) [2023-12-26 18:12:48,298][105692] Updated weights for policy 0, policy_version 391322 (0.0010) [2023-12-26 18:12:48,357][105692] Updated weights for policy 0, policy_version 391332 (0.0010) [2023-12-26 18:12:48,852][105620] Updated weights for policy 1, policy_version 391814 (0.0008) [2023-12-26 18:12:48,917][105620] Updated weights for policy 1, policy_version 391824 (0.0010) [2023-12-26 18:12:48,919][105586] KL-divergence is very high: 142.8658 [2023-12-26 18:12:48,967][105586] KL-divergence is very high: 132.7488 [2023-12-26 18:12:48,978][105620] Updated weights for policy 1, policy_version 391834 (0.0010) [2023-12-26 18:12:49,101][105692] Updated weights for policy 0, policy_version 391342 (0.0010) [2023-12-26 18:12:49,159][105692] Updated weights for policy 0, policy_version 391352 (0.0010) [2023-12-26 18:12:49,225][105692] Updated weights for policy 0, policy_version 391362 (0.0011) [2023-12-26 18:12:49,633][105620] Updated weights for policy 1, policy_version 391844 (0.0010) [2023-12-26 18:12:49,692][105620] Updated weights for policy 1, policy_version 391854 (0.0011) [2023-12-26 18:12:49,740][105620] Updated weights for policy 1, policy_version 391864 (0.0010) [2023-12-26 18:12:49,973][105692] Updated weights for policy 0, policy_version 391372 (0.0009) [2023-12-26 18:12:50,033][105692] Updated weights for policy 0, policy_version 391382 (0.0011) [2023-12-26 18:12:50,098][105692] Updated weights for policy 0, policy_version 391392 (0.0010) [2023-12-26 18:12:50,506][105620] Updated weights for policy 1, policy_version 391874 (0.0010) [2023-12-26 18:12:50,554][105620] Updated weights for policy 1, policy_version 391884 (0.0010) [2023-12-26 18:12:50,620][105620] Updated weights for policy 1, policy_version 391894 (0.0011) [2023-12-26 18:12:50,677][105620] Updated weights for policy 1, policy_version 391904 (0.0010) [2023-12-26 18:12:50,766][105692] Updated weights for policy 0, policy_version 391402 (0.0011) [2023-12-26 18:12:50,828][105692] Updated weights for policy 0, policy_version 391412 (0.0009) [2023-12-26 18:12:50,896][105692] Updated weights for policy 0, policy_version 391422 (0.0008) [2023-12-26 18:12:50,956][105692] Updated weights for policy 0, policy_version 391432 (0.0008) [2023-12-26 18:12:51,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 200556544. Throughput: 0: 9708.9, 1: 10089.6. Samples: 200542964. Policy #0 lag: (min: 1.0, avg: 21.8, max: 33.0) [2023-12-26 18:12:51,062][104569] Avg episode reward: [(0, '9355.983'), (1, '8258.581')] [2023-12-26 18:12:51,409][105620] Updated weights for policy 1, policy_version 391914 (0.0009) [2023-12-26 18:12:51,458][105620] Updated weights for policy 1, policy_version 391924 (0.0005) [2023-12-26 18:12:51,520][105620] Updated weights for policy 1, policy_version 391934 (0.0005) [2023-12-26 18:12:51,763][105692] Updated weights for policy 0, policy_version 391442 (0.0008) [2023-12-26 18:12:51,826][105692] Updated weights for policy 0, policy_version 391452 (0.0009) [2023-12-26 18:12:51,889][105692] Updated weights for policy 0, policy_version 391462 (0.0009) [2023-12-26 18:12:52,168][105620] Updated weights for policy 1, policy_version 391944 (0.0006) [2023-12-26 18:12:52,223][105620] Updated weights for policy 1, policy_version 391954 (0.0005) [2023-12-26 18:12:52,283][105620] Updated weights for policy 1, policy_version 391964 (0.0006) [2023-12-26 18:12:52,758][105692] Updated weights for policy 0, policy_version 391472 (0.0009) [2023-12-26 18:12:52,823][105692] Updated weights for policy 0, policy_version 391482 (0.0008) [2023-12-26 18:12:52,873][105692] Updated weights for policy 0, policy_version 391492 (0.0008) [2023-12-26 18:12:52,981][105620] Updated weights for policy 1, policy_version 391974 (0.0007) [2023-12-26 18:12:53,033][105620] Updated weights for policy 1, policy_version 391984 (0.0008) [2023-12-26 18:12:53,093][105620] Updated weights for policy 1, policy_version 391994 (0.0008) [2023-12-26 18:12:53,571][105692] Updated weights for policy 0, policy_version 391502 (0.0007) [2023-12-26 18:12:53,626][105692] Updated weights for policy 0, policy_version 391512 (0.0006) [2023-12-26 18:12:53,687][105692] Updated weights for policy 0, policy_version 391522 (0.0007) [2023-12-26 18:12:53,885][105620] Updated weights for policy 1, policy_version 392004 (0.0007) [2023-12-26 18:12:53,947][105620] Updated weights for policy 1, policy_version 392014 (0.0006) [2023-12-26 18:12:54,006][105620] Updated weights for policy 1, policy_version 392024 (0.0010) [2023-12-26 18:12:54,248][105692] Updated weights for policy 0, policy_version 391532 (0.0006) [2023-12-26 18:12:54,305][105692] Updated weights for policy 0, policy_version 391542 (0.0008) [2023-12-26 18:12:54,358][105692] Updated weights for policy 0, policy_version 391552 (0.0010) [2023-12-26 18:12:54,775][105620] Updated weights for policy 1, policy_version 392034 (0.0010) [2023-12-26 18:12:54,828][105620] Updated weights for policy 1, policy_version 392044 (0.0008) [2023-12-26 18:12:54,877][105620] Updated weights for policy 1, policy_version 392054 (0.0008) [2023-12-26 18:12:54,933][105620] Updated weights for policy 1, policy_version 392064 (0.0008) [2023-12-26 18:12:55,091][105692] Updated weights for policy 0, policy_version 391562 (0.0010) [2023-12-26 18:12:55,152][105692] Updated weights for policy 0, policy_version 391572 (0.0010) [2023-12-26 18:12:55,213][105692] Updated weights for policy 0, policy_version 391582 (0.0010) [2023-12-26 18:12:55,264][105692] Updated weights for policy 0, policy_version 391592 (0.0010) [2023-12-26 18:12:55,726][105620] Updated weights for policy 1, policy_version 392074 (0.0008) [2023-12-26 18:12:55,782][105620] Updated weights for policy 1, policy_version 392084 (0.0008) [2023-12-26 18:12:55,844][105620] Updated weights for policy 1, policy_version 392094 (0.0008) [2023-12-26 18:12:55,991][105692] Updated weights for policy 0, policy_version 391602 (0.0010) [2023-12-26 18:12:56,035][105692] Updated weights for policy 0, policy_version 391612 (0.0010) [2023-12-26 18:12:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 200646656. Throughput: 0: 9685.0, 1: 9981.3. Samples: 200657628. Policy #0 lag: (min: 1.0, avg: 21.8, max: 33.0) [2023-12-26 18:12:56,062][104569] Avg episode reward: [(0, '9266.913'), (1, '8455.952')] [2023-12-26 18:12:56,089][105692] Updated weights for policy 0, policy_version 391622 (0.0010) [2023-12-26 18:12:56,606][105620] Updated weights for policy 1, policy_version 392104 (0.0009) [2023-12-26 18:12:56,666][105620] Updated weights for policy 1, policy_version 392114 (0.0010) [2023-12-26 18:12:56,721][105620] Updated weights for policy 1, policy_version 392124 (0.0009) [2023-12-26 18:12:56,858][105692] Updated weights for policy 0, policy_version 391632 (0.0010) [2023-12-26 18:12:56,908][105692] Updated weights for policy 0, policy_version 391642 (0.0010) [2023-12-26 18:12:56,955][105692] Updated weights for policy 0, policy_version 391652 (0.0010) [2023-12-26 18:12:57,418][105620] Updated weights for policy 1, policy_version 392134 (0.0007) [2023-12-26 18:12:57,461][105620] Updated weights for policy 1, policy_version 392144 (0.0005) [2023-12-26 18:12:57,507][105620] Updated weights for policy 1, policy_version 392154 (0.0005) [2023-12-26 18:12:57,688][105692] Updated weights for policy 0, policy_version 391662 (0.0007) [2023-12-26 18:12:57,741][105692] Updated weights for policy 0, policy_version 391672 (0.0006) [2023-12-26 18:12:57,795][105692] Updated weights for policy 0, policy_version 391682 (0.0010) [2023-12-26 18:12:58,149][105620] Updated weights for policy 1, policy_version 392164 (0.0006) [2023-12-26 18:12:58,215][105620] Updated weights for policy 1, policy_version 392174 (0.0008) [2023-12-26 18:12:58,270][105620] Updated weights for policy 1, policy_version 392184 (0.0008) [2023-12-26 18:12:58,482][105692] Updated weights for policy 0, policy_version 391692 (0.0010) [2023-12-26 18:12:58,546][105692] Updated weights for policy 0, policy_version 391702 (0.0008) [2023-12-26 18:12:58,597][105692] Updated weights for policy 0, policy_version 391712 (0.0009) [2023-12-26 18:12:58,981][105620] Updated weights for policy 1, policy_version 392194 (0.0008) [2023-12-26 18:12:59,042][105620] Updated weights for policy 1, policy_version 392204 (0.0010) [2023-12-26 18:12:59,094][105620] Updated weights for policy 1, policy_version 392214 (0.0011) [2023-12-26 18:12:59,144][105620] Updated weights for policy 1, policy_version 392224 (0.0010) [2023-12-26 18:12:59,306][105692] Updated weights for policy 0, policy_version 391722 (0.0008) [2023-12-26 18:12:59,340][105585] KL-divergence is very high: 228.4005 [2023-12-26 18:12:59,367][105585] KL-divergence is very high: 472.1246 [2023-12-26 18:12:59,371][105692] Updated weights for policy 0, policy_version 391732 (0.0008) [2023-12-26 18:12:59,373][105585] KL-divergence is very high: 396.4841 [2023-12-26 18:12:59,377][105585] KL-divergence is very high: 490.7170 [2023-12-26 18:12:59,390][105585] KL-divergence is very high: 734.8731 [2023-12-26 18:12:59,416][105585] KL-divergence is very high: 608.6668 [2023-12-26 18:12:59,422][105585] KL-divergence is very high: 457.6876 [2023-12-26 18:12:59,428][105585] KL-divergence is very high: 490.2768 [2023-12-26 18:12:59,433][105692] Updated weights for policy 0, policy_version 391742 (0.0006) [2023-12-26 18:12:59,440][105585] KL-divergence is very high: 620.5567 [2023-12-26 18:12:59,463][105585] KL-divergence is very high: 378.1432 [2023-12-26 18:12:59,469][105585] KL-divergence is very high: 251.5069 [2023-12-26 18:12:59,475][105585] KL-divergence is very high: 268.6995 [2023-12-26 18:12:59,487][105585] KL-divergence is very high: 319.9162 [2023-12-26 18:12:59,492][105692] Updated weights for policy 0, policy_version 391752 (0.0006) [2023-12-26 18:12:59,937][105620] Updated weights for policy 1, policy_version 392234 (0.0008) [2023-12-26 18:13:00,004][105620] Updated weights for policy 1, policy_version 392244 (0.0011) [2023-12-26 18:13:00,061][105585] KL-divergence is very high: 215.7684 [2023-12-26 18:13:00,067][105620] Updated weights for policy 1, policy_version 392254 (0.0011) [2023-12-26 18:13:00,067][105585] KL-divergence is very high: 128.1988 [2023-12-26 18:13:00,096][105585] KL-divergence is very high: 165.5542 [2023-12-26 18:13:00,118][105692] Updated weights for policy 0, policy_version 391762 (0.0008) [2023-12-26 18:13:00,137][105585] KL-divergence is very high: 253.4974 [2023-12-26 18:13:00,166][105585] KL-divergence is very high: 123.4748 [2023-12-26 18:13:00,173][105692] Updated weights for policy 0, policy_version 391772 (0.0009) [2023-12-26 18:13:00,184][105585] KL-divergence is very high: 259.7764 [2023-12-26 18:13:00,229][105585] KL-divergence is very high: 182.1848 [2023-12-26 18:13:00,230][105692] Updated weights for policy 0, policy_version 391782 (0.0009) [2023-12-26 18:13:00,699][105620] Updated weights for policy 1, policy_version 392264 (0.0007) [2023-12-26 18:13:00,751][105620] Updated weights for policy 1, policy_version 392274 (0.0005) [2023-12-26 18:13:00,820][105620] Updated weights for policy 1, policy_version 392284 (0.0006) [2023-12-26 18:13:00,985][105692] Updated weights for policy 0, policy_version 391792 (0.0010) [2023-12-26 18:13:01,041][105692] Updated weights for policy 0, policy_version 391802 (0.0008) [2023-12-26 18:13:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 200744960. Throughput: 0: 9673.2, 1: 10001.2. Samples: 200716768. Policy #0 lag: (min: 1.0, avg: 21.8, max: 33.0) [2023-12-26 18:13:01,062][104569] Avg episode reward: [(0, '9177.731'), (1, '8811.970')] [2023-12-26 18:13:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000392288_100433920.pth... [2023-12-26 18:13:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000391104_100130816.pth [2023-12-26 18:13:01,102][105692] Updated weights for policy 0, policy_version 391812 (0.0010) [2023-12-26 18:13:01,128][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000391816_100319232.pth... [2023-12-26 18:13:01,133][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000390664_100024320.pth [2023-12-26 18:13:01,442][105620] Updated weights for policy 1, policy_version 392294 (0.0007) [2023-12-26 18:13:01,499][105620] Updated weights for policy 1, policy_version 392304 (0.0008) [2023-12-26 18:13:01,560][105620] Updated weights for policy 1, policy_version 392314 (0.0008) [2023-12-26 18:13:01,865][105692] Updated weights for policy 0, policy_version 391822 (0.0007) [2023-12-26 18:13:01,928][105692] Updated weights for policy 0, policy_version 391832 (0.0005) [2023-12-26 18:13:01,994][105692] Updated weights for policy 0, policy_version 391842 (0.0008) [2023-12-26 18:13:02,365][105620] Updated weights for policy 1, policy_version 392324 (0.0010) [2023-12-26 18:13:02,429][105620] Updated weights for policy 1, policy_version 392334 (0.0010) [2023-12-26 18:13:02,487][105620] Updated weights for policy 1, policy_version 392344 (0.0010) [2023-12-26 18:13:02,665][105692] Updated weights for policy 0, policy_version 391852 (0.0008) [2023-12-26 18:13:02,722][105692] Updated weights for policy 0, policy_version 391862 (0.0008) [2023-12-26 18:13:02,772][105692] Updated weights for policy 0, policy_version 391872 (0.0009) [2023-12-26 18:13:03,157][105620] Updated weights for policy 1, policy_version 392354 (0.0010) [2023-12-26 18:13:03,214][105620] Updated weights for policy 1, policy_version 392364 (0.0009) [2023-12-26 18:13:03,260][105620] Updated weights for policy 1, policy_version 392374 (0.0009) [2023-12-26 18:13:03,320][105620] Updated weights for policy 1, policy_version 392384 (0.0009) [2023-12-26 18:13:03,511][105692] Updated weights for policy 0, policy_version 391882 (0.0009) [2023-12-26 18:13:03,556][105692] Updated weights for policy 0, policy_version 391892 (0.0009) [2023-12-26 18:13:03,602][105692] Updated weights for policy 0, policy_version 391902 (0.0009) [2023-12-26 18:13:03,648][105692] Updated weights for policy 0, policy_version 391912 (0.0008) [2023-12-26 18:13:04,063][105620] Updated weights for policy 1, policy_version 392394 (0.0010) [2023-12-26 18:13:04,120][105620] Updated weights for policy 1, policy_version 392404 (0.0011) [2023-12-26 18:13:04,177][105620] Updated weights for policy 1, policy_version 392414 (0.0011) [2023-12-26 18:13:04,453][105692] Updated weights for policy 0, policy_version 391922 (0.0010) [2023-12-26 18:13:04,503][105692] Updated weights for policy 0, policy_version 391932 (0.0010) [2023-12-26 18:13:04,552][105692] Updated weights for policy 0, policy_version 391942 (0.0010) [2023-12-26 18:13:04,822][105620] Updated weights for policy 1, policy_version 392424 (0.0006) [2023-12-26 18:13:04,880][105620] Updated weights for policy 1, policy_version 392434 (0.0010) [2023-12-26 18:13:04,946][105620] Updated weights for policy 1, policy_version 392444 (0.0010) [2023-12-26 18:13:05,274][105692] Updated weights for policy 0, policy_version 391952 (0.0009) [2023-12-26 18:13:05,326][105692] Updated weights for policy 0, policy_version 391962 (0.0009) [2023-12-26 18:13:05,390][105692] Updated weights for policy 0, policy_version 391972 (0.0006) [2023-12-26 18:13:05,680][105620] Updated weights for policy 1, policy_version 392454 (0.0007) [2023-12-26 18:13:05,726][105620] Updated weights for policy 1, policy_version 392464 (0.0005) [2023-12-26 18:13:05,777][105620] Updated weights for policy 1, policy_version 392474 (0.0005) [2023-12-26 18:13:05,930][105692] Updated weights for policy 0, policy_version 391982 (0.0006) [2023-12-26 18:13:05,986][105692] Updated weights for policy 0, policy_version 391992 (0.0007) [2023-12-26 18:13:06,046][105692] Updated weights for policy 0, policy_version 392002 (0.0010) [2023-12-26 18:13:06,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.9, 300 sec: 19660.8). Total num frames: 200843264. Throughput: 0: 9725.5, 1: 9910.8. Samples: 200833776. Policy #0 lag: (min: 1.0, avg: 21.8, max: 33.0) [2023-12-26 18:13:06,062][104569] Avg episode reward: [(0, '9266.644'), (1, '8882.516')] [2023-12-26 18:13:06,539][105620] Updated weights for policy 1, policy_version 392484 (0.0008) [2023-12-26 18:13:06,607][105620] Updated weights for policy 1, policy_version 392494 (0.0008) [2023-12-26 18:13:06,671][105620] Updated weights for policy 1, policy_version 392504 (0.0008) [2023-12-26 18:13:06,780][105692] Updated weights for policy 0, policy_version 392012 (0.0010) [2023-12-26 18:13:06,834][105692] Updated weights for policy 0, policy_version 392022 (0.0010) [2023-12-26 18:13:06,896][105692] Updated weights for policy 0, policy_version 392032 (0.0010) [2023-12-26 18:13:07,439][105620] Updated weights for policy 1, policy_version 392514 (0.0009) [2023-12-26 18:13:07,496][105620] Updated weights for policy 1, policy_version 392524 (0.0008) [2023-12-26 18:13:07,553][105620] Updated weights for policy 1, policy_version 392535 (0.0010) [2023-12-26 18:13:07,561][105692] Updated weights for policy 0, policy_version 392042 (0.0009) [2023-12-26 18:13:07,615][105692] Updated weights for policy 0, policy_version 392052 (0.0005) [2023-12-26 18:13:07,660][105692] Updated weights for policy 0, policy_version 392062 (0.0005) [2023-12-26 18:13:07,723][105692] Updated weights for policy 0, policy_version 392072 (0.0006) [2023-12-26 18:13:08,319][105620] Updated weights for policy 1, policy_version 392545 (0.0008) [2023-12-26 18:13:08,356][105692] Updated weights for policy 0, policy_version 392082 (0.0008) [2023-12-26 18:13:08,369][105586] KL-divergence is very high: 117.0606 [2023-12-26 18:13:08,385][105620] Updated weights for policy 1, policy_version 392555 (0.0007) [2023-12-26 18:13:08,412][105586] KL-divergence is very high: 101.5539 [2023-12-26 18:13:08,414][105692] Updated weights for policy 0, policy_version 392092 (0.0008) [2023-12-26 18:13:08,418][105586] KL-divergence is very high: 100.6043 [2023-12-26 18:13:08,431][105586] KL-divergence is very high: 106.5975 [2023-12-26 18:13:08,450][105620] Updated weights for policy 1, policy_version 392565 (0.0007) [2023-12-26 18:13:08,458][105586] KL-divergence is very high: 145.8714 [2023-12-26 18:13:08,469][105586] KL-divergence is very high: 119.8376 [2023-12-26 18:13:08,472][105692] Updated weights for policy 0, policy_version 392102 (0.0007) [2023-12-26 18:13:08,475][105586] KL-divergence is very high: 120.4638 [2023-12-26 18:13:08,506][105586] KL-divergence is very high: 126.0188 [2023-12-26 18:13:08,507][105620] Updated weights for policy 1, policy_version 392575 (0.0009) [2023-12-26 18:13:09,065][105692] Updated weights for policy 0, policy_version 392112 (0.0005) [2023-12-26 18:13:09,128][105692] Updated weights for policy 0, policy_version 392122 (0.0007) [2023-12-26 18:13:09,189][105692] Updated weights for policy 0, policy_version 392132 (0.0009) [2023-12-26 18:13:09,314][105620] Updated weights for policy 1, policy_version 392585 (0.0009) [2023-12-26 18:13:09,382][105620] Updated weights for policy 1, policy_version 392595 (0.0008) [2023-12-26 18:13:09,449][105620] Updated weights for policy 1, policy_version 392605 (0.0008) [2023-12-26 18:13:09,957][105692] Updated weights for policy 0, policy_version 392142 (0.0009) [2023-12-26 18:13:10,010][105692] Updated weights for policy 0, policy_version 392152 (0.0009) [2023-12-26 18:13:10,060][105692] Updated weights for policy 0, policy_version 392163 (0.0010) [2023-12-26 18:13:10,169][105620] Updated weights for policy 1, policy_version 392615 (0.0006) [2023-12-26 18:13:10,240][105620] Updated weights for policy 1, policy_version 392625 (0.0008) [2023-12-26 18:13:10,303][105620] Updated weights for policy 1, policy_version 392635 (0.0010) [2023-12-26 18:13:10,854][105692] Updated weights for policy 0, policy_version 392173 (0.0007) [2023-12-26 18:13:10,902][105692] Updated weights for policy 0, policy_version 392183 (0.0006) [2023-12-26 18:13:10,958][105692] Updated weights for policy 0, policy_version 392193 (0.0005) [2023-12-26 18:13:11,043][105620] Updated weights for policy 1, policy_version 392645 (0.0009) [2023-12-26 18:13:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 200941568. Throughput: 0: 9831.7, 1: 9888.1. Samples: 200950172. Policy #0 lag: (min: 1.0, avg: 21.8, max: 33.0) [2023-12-26 18:13:11,062][104569] Avg episode reward: [(0, '9266.750'), (1, '8451.261')] [2023-12-26 18:13:11,101][105620] Updated weights for policy 1, policy_version 392655 (0.0009) [2023-12-26 18:13:11,166][105620] Updated weights for policy 1, policy_version 392665 (0.0009) [2023-12-26 18:13:11,687][105692] Updated weights for policy 0, policy_version 392203 (0.0006) [2023-12-26 18:13:11,759][105692] Updated weights for policy 0, policy_version 392213 (0.0009) [2023-12-26 18:13:11,812][105692] Updated weights for policy 0, policy_version 392223 (0.0011) [2023-12-26 18:13:11,985][105620] Updated weights for policy 1, policy_version 392675 (0.0009) [2023-12-26 18:13:12,041][105620] Updated weights for policy 1, policy_version 392685 (0.0009) [2023-12-26 18:13:12,094][105620] Updated weights for policy 1, policy_version 392695 (0.0009) [2023-12-26 18:13:12,472][105692] Updated weights for policy 0, policy_version 392233 (0.0011) [2023-12-26 18:13:12,517][105692] Updated weights for policy 0, policy_version 392243 (0.0010) [2023-12-26 18:13:12,569][105692] Updated weights for policy 0, policy_version 392253 (0.0011) [2023-12-26 18:13:12,631][105692] Updated weights for policy 0, policy_version 392263 (0.0011) [2023-12-26 18:13:12,911][105620] Updated weights for policy 1, policy_version 392705 (0.0009) [2023-12-26 18:13:12,964][105620] Updated weights for policy 1, policy_version 392715 (0.0009) [2023-12-26 18:13:13,026][105620] Updated weights for policy 1, policy_version 392725 (0.0009) [2023-12-26 18:13:13,089][105620] Updated weights for policy 1, policy_version 392735 (0.0009) [2023-12-26 18:13:13,317][105692] Updated weights for policy 0, policy_version 392273 (0.0009) [2023-12-26 18:13:13,376][105692] Updated weights for policy 0, policy_version 392283 (0.0009) [2023-12-26 18:13:13,430][105692] Updated weights for policy 0, policy_version 392293 (0.0009) [2023-12-26 18:13:13,874][105620] Updated weights for policy 1, policy_version 392745 (0.0009) [2023-12-26 18:13:13,932][105620] Updated weights for policy 1, policy_version 392755 (0.0008) [2023-12-26 18:13:14,001][105620] Updated weights for policy 1, policy_version 392765 (0.0008) [2023-12-26 18:13:14,107][105692] Updated weights for policy 0, policy_version 392303 (0.0009) [2023-12-26 18:13:14,167][105692] Updated weights for policy 0, policy_version 392313 (0.0009) [2023-12-26 18:13:14,221][105692] Updated weights for policy 0, policy_version 392323 (0.0008) [2023-12-26 18:13:14,722][105620] Updated weights for policy 1, policy_version 392775 (0.0009) [2023-12-26 18:13:14,772][105620] Updated weights for policy 1, policy_version 392785 (0.0008) [2023-12-26 18:13:14,835][105620] Updated weights for policy 1, policy_version 392795 (0.0008) [2023-12-26 18:13:14,976][105692] Updated weights for policy 0, policy_version 392333 (0.0008) [2023-12-26 18:13:15,035][105692] Updated weights for policy 0, policy_version 392343 (0.0009) [2023-12-26 18:13:15,085][105692] Updated weights for policy 0, policy_version 392353 (0.0007) [2023-12-26 18:13:15,582][105620] Updated weights for policy 1, policy_version 392805 (0.0009) [2023-12-26 18:13:15,645][105620] Updated weights for policy 1, policy_version 392815 (0.0008) [2023-12-26 18:13:15,709][105620] Updated weights for policy 1, policy_version 392825 (0.0008) [2023-12-26 18:13:15,802][105692] Updated weights for policy 0, policy_version 392363 (0.0008) [2023-12-26 18:13:15,861][105692] Updated weights for policy 0, policy_version 392373 (0.0005) [2023-12-26 18:13:15,917][105692] Updated weights for policy 0, policy_version 392383 (0.0009) [2023-12-26 18:13:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 201039872. Throughput: 0: 9825.0, 1: 9814.9. Samples: 201006112. Policy #0 lag: (min: 1.0, avg: 21.8, max: 33.0) [2023-12-26 18:13:16,062][104569] Avg episode reward: [(0, '9356.455'), (1, '8652.903')] [2023-12-26 18:13:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000392392_100466688.pth... [2023-12-26 18:13:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000392832_100573184.pth... [2023-12-26 18:13:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000391712_100286464.pth [2023-12-26 18:13:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000391240_100171776.pth [2023-12-26 18:13:16,457][105692] Updated weights for policy 0, policy_version 392393 (0.0009) [2023-12-26 18:13:16,521][105692] Updated weights for policy 0, policy_version 392403 (0.0010) [2023-12-26 18:13:16,549][105620] Updated weights for policy 1, policy_version 392835 (0.0008) [2023-12-26 18:13:16,579][105692] Updated weights for policy 0, policy_version 392413 (0.0010) [2023-12-26 18:13:16,609][105620] Updated weights for policy 1, policy_version 392845 (0.0007) [2023-12-26 18:13:16,638][105692] Updated weights for policy 0, policy_version 392423 (0.0007) [2023-12-26 18:13:16,669][105620] Updated weights for policy 1, policy_version 392855 (0.0008) [2023-12-26 18:13:17,190][105692] Updated weights for policy 0, policy_version 392433 (0.0005) [2023-12-26 18:13:17,257][105692] Updated weights for policy 0, policy_version 392443 (0.0005) [2023-12-26 18:13:17,317][105692] Updated weights for policy 0, policy_version 392453 (0.0005) [2023-12-26 18:13:17,547][105620] Updated weights for policy 1, policy_version 392865 (0.0009) [2023-12-26 18:13:17,612][105620] Updated weights for policy 1, policy_version 392875 (0.0009) [2023-12-26 18:13:17,664][105620] Updated weights for policy 1, policy_version 392885 (0.0009) [2023-12-26 18:13:17,711][105620] Updated weights for policy 1, policy_version 392895 (0.0009) [2023-12-26 18:13:17,869][105692] Updated weights for policy 0, policy_version 392463 (0.0008) [2023-12-26 18:13:17,928][105692] Updated weights for policy 0, policy_version 392473 (0.0006) [2023-12-26 18:13:17,989][105692] Updated weights for policy 0, policy_version 392483 (0.0009) [2023-12-26 18:13:18,523][105620] Updated weights for policy 1, policy_version 392905 (0.0010) [2023-12-26 18:13:18,586][105620] Updated weights for policy 1, policy_version 392915 (0.0010) [2023-12-26 18:13:18,642][105620] Updated weights for policy 1, policy_version 392925 (0.0010) [2023-12-26 18:13:18,677][105692] Updated weights for policy 0, policy_version 392493 (0.0008) [2023-12-26 18:13:18,740][105692] Updated weights for policy 0, policy_version 392503 (0.0007) [2023-12-26 18:13:18,796][105692] Updated weights for policy 0, policy_version 392513 (0.0008) [2023-12-26 18:13:19,469][105620] Updated weights for policy 1, policy_version 392935 (0.0009) [2023-12-26 18:13:19,472][105692] Updated weights for policy 0, policy_version 392523 (0.0009) [2023-12-26 18:13:19,535][105692] Updated weights for policy 0, policy_version 392533 (0.0009) [2023-12-26 18:13:19,536][105620] Updated weights for policy 1, policy_version 392945 (0.0009) [2023-12-26 18:13:19,590][105620] Updated weights for policy 1, policy_version 392955 (0.0008) [2023-12-26 18:13:19,592][105692] Updated weights for policy 0, policy_version 392543 (0.0006) [2023-12-26 18:13:19,641][105585] KL-divergence is very high: 156.4631 [2023-12-26 18:13:20,321][105620] Updated weights for policy 1, policy_version 392965 (0.0008) [2023-12-26 18:13:20,357][105692] Updated weights for policy 0, policy_version 392553 (0.0008) [2023-12-26 18:13:20,381][105620] Updated weights for policy 1, policy_version 392975 (0.0008) [2023-12-26 18:13:20,420][105692] Updated weights for policy 0, policy_version 392563 (0.0009) [2023-12-26 18:13:20,442][105620] Updated weights for policy 1, policy_version 392985 (0.0006) [2023-12-26 18:13:20,477][105692] Updated weights for policy 0, policy_version 392573 (0.0008) [2023-12-26 18:13:20,534][105692] Updated weights for policy 0, policy_version 392583 (0.0008) [2023-12-26 18:13:21,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 201129984. Throughput: 0: 9931.0, 1: 9577.2. Samples: 201122452. Policy #0 lag: (min: 1.0, avg: 21.8, max: 33.0) [2023-12-26 18:13:21,063][104569] Avg episode reward: [(0, '9356.651'), (1, '9268.342')] [2023-12-26 18:13:21,162][105620] Updated weights for policy 1, policy_version 392995 (0.0008) [2023-12-26 18:13:21,229][105620] Updated weights for policy 1, policy_version 393005 (0.0011) [2023-12-26 18:13:21,294][105620] Updated weights for policy 1, policy_version 393016 (0.0011) [2023-12-26 18:13:21,348][105692] Updated weights for policy 0, policy_version 392593 (0.0008) [2023-12-26 18:13:21,416][105692] Updated weights for policy 0, policy_version 392603 (0.0008) [2023-12-26 18:13:21,476][105692] Updated weights for policy 0, policy_version 392613 (0.0008) [2023-12-26 18:13:22,044][105620] Updated weights for policy 1, policy_version 393026 (0.0009) [2023-12-26 18:13:22,103][105620] Updated weights for policy 1, policy_version 393036 (0.0009) [2023-12-26 18:13:22,183][105620] Updated weights for policy 1, policy_version 393046 (0.0011) [2023-12-26 18:13:22,207][105692] Updated weights for policy 0, policy_version 392623 (0.0009) [2023-12-26 18:13:22,239][105620] Updated weights for policy 1, policy_version 393056 (0.0010) [2023-12-26 18:13:22,266][105692] Updated weights for policy 0, policy_version 392633 (0.0008) [2023-12-26 18:13:22,320][105692] Updated weights for policy 0, policy_version 392643 (0.0008) [2023-12-26 18:13:22,902][105620] Updated weights for policy 1, policy_version 393066 (0.0011) [2023-12-26 18:13:22,956][105620] Updated weights for policy 1, policy_version 393076 (0.0010) [2023-12-26 18:13:23,015][105620] Updated weights for policy 1, policy_version 393086 (0.0008) [2023-12-26 18:13:23,074][105692] Updated weights for policy 0, policy_version 392653 (0.0010) [2023-12-26 18:13:23,141][105692] Updated weights for policy 0, policy_version 392663 (0.0011) [2023-12-26 18:13:23,201][105692] Updated weights for policy 0, policy_version 392673 (0.0010) [2023-12-26 18:13:23,679][105620] Updated weights for policy 1, policy_version 393096 (0.0010) [2023-12-26 18:13:23,746][105620] Updated weights for policy 1, policy_version 393106 (0.0007) [2023-12-26 18:13:23,801][105620] Updated weights for policy 1, policy_version 393116 (0.0006) [2023-12-26 18:13:23,803][105692] Updated weights for policy 0, policy_version 392683 (0.0010) [2023-12-26 18:13:23,853][105692] Updated weights for policy 0, policy_version 392693 (0.0009) [2023-12-26 18:13:23,910][105692] Updated weights for policy 0, policy_version 392703 (0.0007) [2023-12-26 18:13:24,554][105692] Updated weights for policy 0, policy_version 392713 (0.0006) [2023-12-26 18:13:24,559][105620] Updated weights for policy 1, policy_version 393126 (0.0008) [2023-12-26 18:13:24,617][105620] Updated weights for policy 1, policy_version 393136 (0.0006) [2023-12-26 18:13:24,617][105692] Updated weights for policy 0, policy_version 392723 (0.0011) [2023-12-26 18:13:24,680][105692] Updated weights for policy 0, policy_version 392733 (0.0010) [2023-12-26 18:13:24,682][105620] Updated weights for policy 1, policy_version 393146 (0.0008) [2023-12-26 18:13:24,739][105692] Updated weights for policy 0, policy_version 392743 (0.0010) [2023-12-26 18:13:25,383][105692] Updated weights for policy 0, policy_version 392753 (0.0010) [2023-12-26 18:13:25,425][105620] Updated weights for policy 1, policy_version 393156 (0.0008) [2023-12-26 18:13:25,438][105692] Updated weights for policy 0, policy_version 392763 (0.0011) [2023-12-26 18:13:25,474][105620] Updated weights for policy 1, policy_version 393166 (0.0010) [2023-12-26 18:13:25,497][105692] Updated weights for policy 0, policy_version 392773 (0.0010) [2023-12-26 18:13:25,525][105620] Updated weights for policy 1, policy_version 393176 (0.0010) [2023-12-26 18:13:26,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 201228288. Throughput: 0: 9867.9, 1: 9575.0. Samples: 201239276. Policy #0 lag: (min: 1.0, avg: 21.8, max: 33.0) [2023-12-26 18:13:26,063][104569] Avg episode reward: [(0, '9356.302'), (1, '9006.633')] [2023-12-26 18:13:26,205][105620] Updated weights for policy 1, policy_version 393186 (0.0010) [2023-12-26 18:13:26,249][105692] Updated weights for policy 0, policy_version 392783 (0.0009) [2023-12-26 18:13:26,257][105620] Updated weights for policy 1, policy_version 393196 (0.0010) [2023-12-26 18:13:26,313][105692] Updated weights for policy 0, policy_version 392793 (0.0008) [2023-12-26 18:13:26,318][105620] Updated weights for policy 1, policy_version 393206 (0.0010) [2023-12-26 18:13:26,370][105620] Updated weights for policy 1, policy_version 393216 (0.0010) [2023-12-26 18:13:26,379][105692] Updated weights for policy 0, policy_version 392803 (0.0008) [2023-12-26 18:13:26,953][105692] Updated weights for policy 0, policy_version 392813 (0.0006) [2023-12-26 18:13:27,003][105692] Updated weights for policy 0, policy_version 392823 (0.0006) [2023-12-26 18:13:27,047][105620] Updated weights for policy 1, policy_version 393226 (0.0007) [2023-12-26 18:13:27,055][105692] Updated weights for policy 0, policy_version 392833 (0.0009) [2023-12-26 18:13:27,095][105620] Updated weights for policy 1, policy_version 393236 (0.0006) [2023-12-26 18:13:27,142][105620] Updated weights for policy 1, policy_version 393246 (0.0009) [2023-12-26 18:13:27,662][105692] Updated weights for policy 0, policy_version 392843 (0.0008) [2023-12-26 18:13:27,724][105692] Updated weights for policy 0, policy_version 392853 (0.0005) [2023-12-26 18:13:27,774][105692] Updated weights for policy 0, policy_version 392863 (0.0005) [2023-12-26 18:13:28,000][105620] Updated weights for policy 1, policy_version 393256 (0.0009) [2023-12-26 18:13:28,053][105620] Updated weights for policy 1, policy_version 393266 (0.0010) [2023-12-26 18:13:28,110][105620] Updated weights for policy 1, policy_version 393276 (0.0010) [2023-12-26 18:13:28,344][105692] Updated weights for policy 0, policy_version 392873 (0.0006) [2023-12-26 18:13:28,397][105692] Updated weights for policy 0, policy_version 392883 (0.0008) [2023-12-26 18:13:28,442][105692] Updated weights for policy 0, policy_version 392893 (0.0008) [2023-12-26 18:13:28,497][105692] Updated weights for policy 0, policy_version 392903 (0.0008) [2023-12-26 18:13:28,876][105620] Updated weights for policy 1, policy_version 393286 (0.0010) [2023-12-26 18:13:28,927][105620] Updated weights for policy 1, policy_version 393296 (0.0010) [2023-12-26 18:13:28,989][105620] Updated weights for policy 1, policy_version 393306 (0.0010) [2023-12-26 18:13:29,300][105692] Updated weights for policy 0, policy_version 392913 (0.0009) [2023-12-26 18:13:29,371][105692] Updated weights for policy 0, policy_version 392923 (0.0009) [2023-12-26 18:13:29,420][105692] Updated weights for policy 0, policy_version 392933 (0.0008) [2023-12-26 18:13:29,759][105620] Updated weights for policy 1, policy_version 393316 (0.0010) [2023-12-26 18:13:29,824][105620] Updated weights for policy 1, policy_version 393326 (0.0010) [2023-12-26 18:13:29,882][105620] Updated weights for policy 1, policy_version 393336 (0.0010) [2023-12-26 18:13:30,167][105692] Updated weights for policy 0, policy_version 392943 (0.0009) [2023-12-26 18:13:30,227][105692] Updated weights for policy 0, policy_version 392953 (0.0007) [2023-12-26 18:13:30,288][105692] Updated weights for policy 0, policy_version 392963 (0.0008) [2023-12-26 18:13:30,613][105620] Updated weights for policy 1, policy_version 393346 (0.0009) [2023-12-26 18:13:30,665][105620] Updated weights for policy 1, policy_version 393356 (0.0005) [2023-12-26 18:13:30,717][105620] Updated weights for policy 1, policy_version 393366 (0.0005) [2023-12-26 18:13:30,762][105620] Updated weights for policy 1, policy_version 393376 (0.0005) [2023-12-26 18:13:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.2). Total num frames: 201326592. Throughput: 0: 9960.9, 1: 9526.0. Samples: 201299704. Policy #0 lag: (min: 1.0, avg: 21.8, max: 33.0) [2023-12-26 18:13:31,062][104569] Avg episode reward: [(0, '9356.155'), (1, '9006.600')] [2023-12-26 18:13:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000392968_100614144.pth... [2023-12-26 18:13:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000393376_100712448.pth... [2023-12-26 18:13:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000391816_100319232.pth [2023-12-26 18:13:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000392288_100433920.pth [2023-12-26 18:13:31,134][105692] Updated weights for policy 0, policy_version 392973 (0.0009) [2023-12-26 18:13:31,194][105692] Updated weights for policy 0, policy_version 392983 (0.0008) [2023-12-26 18:13:31,243][105692] Updated weights for policy 0, policy_version 392993 (0.0007) [2023-12-26 18:13:31,349][105620] Updated weights for policy 1, policy_version 393386 (0.0008) [2023-12-26 18:13:31,419][105620] Updated weights for policy 1, policy_version 393396 (0.0008) [2023-12-26 18:13:31,475][105620] Updated weights for policy 1, policy_version 393406 (0.0008) [2023-12-26 18:13:32,074][105692] Updated weights for policy 0, policy_version 393003 (0.0008) [2023-12-26 18:13:32,114][105585] KL-divergence is very high: 104.4812 [2023-12-26 18:13:32,132][105692] Updated weights for policy 0, policy_version 393013 (0.0006) [2023-12-26 18:13:32,184][105620] Updated weights for policy 1, policy_version 393416 (0.0009) [2023-12-26 18:13:32,189][105692] Updated weights for policy 0, policy_version 393023 (0.0007) [2023-12-26 18:13:32,232][105620] Updated weights for policy 1, policy_version 393426 (0.0007) [2023-12-26 18:13:32,296][105620] Updated weights for policy 1, policy_version 393436 (0.0008) [2023-12-26 18:13:32,818][105692] Updated weights for policy 0, policy_version 393033 (0.0006) [2023-12-26 18:13:32,865][105692] Updated weights for policy 0, policy_version 393043 (0.0009) [2023-12-26 18:13:32,912][105692] Updated weights for policy 0, policy_version 393053 (0.0009) [2023-12-26 18:13:32,973][105692] Updated weights for policy 0, policy_version 393063 (0.0008) [2023-12-26 18:13:33,096][105620] Updated weights for policy 1, policy_version 393446 (0.0009) [2023-12-26 18:13:33,156][105620] Updated weights for policy 1, policy_version 393456 (0.0008) [2023-12-26 18:13:33,208][105620] Updated weights for policy 1, policy_version 393466 (0.0008) [2023-12-26 18:13:33,696][105692] Updated weights for policy 0, policy_version 393073 (0.0009) [2023-12-26 18:13:33,742][105692] Updated weights for policy 0, policy_version 393083 (0.0009) [2023-12-26 18:13:33,788][105692] Updated weights for policy 0, policy_version 393093 (0.0008) [2023-12-26 18:13:33,965][105620] Updated weights for policy 1, policy_version 393476 (0.0008) [2023-12-26 18:13:34,008][105620] Updated weights for policy 1, policy_version 393486 (0.0005) [2023-12-26 18:13:34,054][105620] Updated weights for policy 1, policy_version 393496 (0.0008) [2023-12-26 18:13:34,656][105620] Updated weights for policy 1, policy_version 393506 (0.0009) [2023-12-26 18:13:34,658][105692] Updated weights for policy 0, policy_version 393103 (0.0009) [2023-12-26 18:13:34,712][105692] Updated weights for policy 0, policy_version 393113 (0.0007) [2023-12-26 18:13:34,714][105620] Updated weights for policy 1, policy_version 393516 (0.0007) [2023-12-26 18:13:34,773][105692] Updated weights for policy 0, policy_version 393123 (0.0006) [2023-12-26 18:13:34,775][105620] Updated weights for policy 1, policy_version 393526 (0.0008) [2023-12-26 18:13:34,830][105620] Updated weights for policy 1, policy_version 393536 (0.0007) [2023-12-26 18:13:35,541][105692] Updated weights for policy 0, policy_version 393133 (0.0008) [2023-12-26 18:13:35,596][105692] Updated weights for policy 0, policy_version 393143 (0.0009) [2023-12-26 18:13:35,607][105620] Updated weights for policy 1, policy_version 393546 (0.0006) [2023-12-26 18:13:35,649][105692] Updated weights for policy 0, policy_version 393153 (0.0008) [2023-12-26 18:13:35,662][105620] Updated weights for policy 1, policy_version 393556 (0.0006) [2023-12-26 18:13:35,727][105620] Updated weights for policy 1, policy_version 393566 (0.0005) [2023-12-26 18:13:36,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 201424896. Throughput: 0: 9913.6, 1: 9439.6. Samples: 201413856. Policy #0 lag: (min: 1.0, avg: 21.8, max: 33.0) [2023-12-26 18:13:36,063][104569] Avg episode reward: [(0, '9356.068'), (1, '9265.122')] [2023-12-26 18:13:36,381][105620] Updated weights for policy 1, policy_version 393576 (0.0008) [2023-12-26 18:13:36,435][105620] Updated weights for policy 1, policy_version 393586 (0.0010) [2023-12-26 18:13:36,451][105692] Updated weights for policy 0, policy_version 393163 (0.0007) [2023-12-26 18:13:36,496][105620] Updated weights for policy 1, policy_version 393596 (0.0008) [2023-12-26 18:13:36,506][105692] Updated weights for policy 0, policy_version 393173 (0.0008) [2023-12-26 18:13:36,568][105692] Updated weights for policy 0, policy_version 393183 (0.0008) [2023-12-26 18:13:37,171][105692] Updated weights for policy 0, policy_version 393193 (0.0009) [2023-12-26 18:13:37,232][105692] Updated weights for policy 0, policy_version 393203 (0.0010) [2023-12-26 18:13:37,289][105692] Updated weights for policy 0, policy_version 393213 (0.0008) [2023-12-26 18:13:37,305][105620] Updated weights for policy 1, policy_version 393606 (0.0008) [2023-12-26 18:13:37,347][105692] Updated weights for policy 0, policy_version 393223 (0.0006) [2023-12-26 18:13:37,361][105620] Updated weights for policy 1, policy_version 393616 (0.0009) [2023-12-26 18:13:37,415][105620] Updated weights for policy 1, policy_version 393626 (0.0010) [2023-12-26 18:13:38,000][105692] Updated weights for policy 0, policy_version 393233 (0.0006) [2023-12-26 18:13:38,036][105585] KL-divergence is very high: 197.3472 [2023-12-26 18:13:38,065][105692] Updated weights for policy 0, policy_version 393243 (0.0006) [2023-12-26 18:13:38,072][105585] KL-divergence is very high: 149.8335 [2023-12-26 18:13:38,082][105585] KL-divergence is very high: 219.2315 [2023-12-26 18:13:38,111][105585] KL-divergence is very high: 110.9456 [2023-12-26 18:13:38,117][105692] Updated weights for policy 0, policy_version 393253 (0.0006) [2023-12-26 18:13:38,124][105585] KL-divergence is very high: 179.6016 [2023-12-26 18:13:38,130][105620] Updated weights for policy 1, policy_version 393636 (0.0010) [2023-12-26 18:13:38,184][105620] Updated weights for policy 1, policy_version 393646 (0.0010) [2023-12-26 18:13:38,251][105620] Updated weights for policy 1, policy_version 393656 (0.0008) [2023-12-26 18:13:38,669][105585] KL-divergence is very high: 145.0791 [2023-12-26 18:13:38,710][105692] Updated weights for policy 0, policy_version 393263 (0.0008) [2023-12-26 18:13:38,718][105585] KL-divergence is very high: 135.0844 [2023-12-26 18:13:38,778][105692] Updated weights for policy 0, policy_version 393273 (0.0005) [2023-12-26 18:13:38,846][105692] Updated weights for policy 0, policy_version 393283 (0.0007) [2023-12-26 18:13:38,914][105620] Updated weights for policy 1, policy_version 393666 (0.0007) [2023-12-26 18:13:38,976][105620] Updated weights for policy 1, policy_version 393676 (0.0007) [2023-12-26 18:13:39,041][105620] Updated weights for policy 1, policy_version 393686 (0.0009) [2023-12-26 18:13:39,109][105620] Updated weights for policy 1, policy_version 393696 (0.0006) [2023-12-26 18:13:39,462][105692] Updated weights for policy 0, policy_version 393293 (0.0009) [2023-12-26 18:13:39,526][105692] Updated weights for policy 0, policy_version 393303 (0.0010) [2023-12-26 18:13:39,592][105692] Updated weights for policy 0, policy_version 393313 (0.0009) [2023-12-26 18:13:39,781][105620] Updated weights for policy 1, policy_version 393706 (0.0007) [2023-12-26 18:13:39,846][105620] Updated weights for policy 1, policy_version 393716 (0.0009) [2023-12-26 18:13:39,900][105620] Updated weights for policy 1, policy_version 393726 (0.0009) [2023-12-26 18:13:40,289][105692] Updated weights for policy 0, policy_version 393323 (0.0010) [2023-12-26 18:13:40,351][105692] Updated weights for policy 0, policy_version 393333 (0.0009) [2023-12-26 18:13:40,411][105692] Updated weights for policy 0, policy_version 393343 (0.0009) [2023-12-26 18:13:40,687][105620] Updated weights for policy 1, policy_version 393736 (0.0010) [2023-12-26 18:13:40,749][105620] Updated weights for policy 1, policy_version 393746 (0.0010) [2023-12-26 18:13:40,816][105620] Updated weights for policy 1, policy_version 393756 (0.0010) [2023-12-26 18:13:41,019][105692] Updated weights for policy 0, policy_version 393353 (0.0009) [2023-12-26 18:13:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 201523200. Throughput: 0: 9954.8, 1: 9445.6. Samples: 201530644. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 18:13:41,063][104569] Avg episode reward: [(0, '9355.760'), (1, '9265.022')] [2023-12-26 18:13:41,080][105692] Updated weights for policy 0, policy_version 393363 (0.0008) [2023-12-26 18:13:41,139][105692] Updated weights for policy 0, policy_version 393373 (0.0009) [2023-12-26 18:13:41,205][105692] Updated weights for policy 0, policy_version 393383 (0.0010) [2023-12-26 18:13:41,622][105620] Updated weights for policy 1, policy_version 393766 (0.0010) [2023-12-26 18:13:41,681][105620] Updated weights for policy 1, policy_version 393776 (0.0009) [2023-12-26 18:13:41,750][105620] Updated weights for policy 1, policy_version 393786 (0.0008) [2023-12-26 18:13:41,993][105692] Updated weights for policy 0, policy_version 393393 (0.0009) [2023-12-26 18:13:42,053][105692] Updated weights for policy 0, policy_version 393403 (0.0010) [2023-12-26 18:13:42,110][105692] Updated weights for policy 0, policy_version 393413 (0.0008) [2023-12-26 18:13:42,414][105620] Updated weights for policy 1, policy_version 393796 (0.0006) [2023-12-26 18:13:42,468][105620] Updated weights for policy 1, policy_version 393806 (0.0006) [2023-12-26 18:13:42,515][105620] Updated weights for policy 1, policy_version 393816 (0.0005) [2023-12-26 18:13:43,002][105692] Updated weights for policy 0, policy_version 393423 (0.0009) [2023-12-26 18:13:43,060][105692] Updated weights for policy 0, policy_version 393433 (0.0010) [2023-12-26 18:13:43,079][105620] Updated weights for policy 1, policy_version 393826 (0.0009) [2023-12-26 18:13:43,113][105692] Updated weights for policy 0, policy_version 393443 (0.0008) [2023-12-26 18:13:43,135][105620] Updated weights for policy 1, policy_version 393836 (0.0005) [2023-12-26 18:13:43,181][105620] Updated weights for policy 1, policy_version 393846 (0.0005) [2023-12-26 18:13:43,239][105620] Updated weights for policy 1, policy_version 393856 (0.0005) [2023-12-26 18:13:43,839][105620] Updated weights for policy 1, policy_version 393866 (0.0005) [2023-12-26 18:13:43,877][105692] Updated weights for policy 0, policy_version 393453 (0.0007) [2023-12-26 18:13:43,901][105620] Updated weights for policy 1, policy_version 393876 (0.0005) [2023-12-26 18:13:43,931][105692] Updated weights for policy 0, policy_version 393463 (0.0005) [2023-12-26 18:13:43,958][105620] Updated weights for policy 1, policy_version 393886 (0.0005) [2023-12-26 18:13:43,987][105692] Updated weights for policy 0, policy_version 393473 (0.0005) [2023-12-26 18:13:44,537][105620] Updated weights for policy 1, policy_version 393896 (0.0006) [2023-12-26 18:13:44,599][105620] Updated weights for policy 1, policy_version 393906 (0.0011) [2023-12-26 18:13:44,660][105620] Updated weights for policy 1, policy_version 393916 (0.0006) [2023-12-26 18:13:44,676][105692] Updated weights for policy 0, policy_version 393483 (0.0006) [2023-12-26 18:13:44,729][105692] Updated weights for policy 0, policy_version 393493 (0.0006) [2023-12-26 18:13:44,790][105692] Updated weights for policy 0, policy_version 393503 (0.0007) [2023-12-26 18:13:45,326][105620] Updated weights for policy 1, policy_version 393926 (0.0008) [2023-12-26 18:13:45,386][105620] Updated weights for policy 1, policy_version 393936 (0.0009) [2023-12-26 18:13:45,448][105620] Updated weights for policy 1, policy_version 393946 (0.0009) [2023-12-26 18:13:45,480][105692] Updated weights for policy 0, policy_version 393513 (0.0007) [2023-12-26 18:13:45,541][105692] Updated weights for policy 0, policy_version 393523 (0.0006) [2023-12-26 18:13:45,599][105692] Updated weights for policy 0, policy_version 393533 (0.0006) [2023-12-26 18:13:45,665][105692] Updated weights for policy 0, policy_version 393543 (0.0005) [2023-12-26 18:13:46,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19387.6, 300 sec: 19605.2). Total num frames: 201621504. Throughput: 0: 9914.1, 1: 9510.2. Samples: 201590864. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 18:13:46,063][104569] Avg episode reward: [(0, '9355.429'), (1, '9265.048')] [2023-12-26 18:13:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000393544_100761600.pth... [2023-12-26 18:13:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000393952_100859904.pth... [2023-12-26 18:13:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000392832_100573184.pth [2023-12-26 18:13:46,083][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000392392_100466688.pth [2023-12-26 18:13:46,147][105620] Updated weights for policy 1, policy_version 393956 (0.0010) [2023-12-26 18:13:46,209][105620] Updated weights for policy 1, policy_version 393966 (0.0011) [2023-12-26 18:13:46,254][105692] Updated weights for policy 0, policy_version 393553 (0.0010) [2023-12-26 18:13:46,260][105620] Updated weights for policy 1, policy_version 393976 (0.0010) [2023-12-26 18:13:46,311][105692] Updated weights for policy 0, policy_version 393563 (0.0010) [2023-12-26 18:13:46,371][105692] Updated weights for policy 0, policy_version 393573 (0.0009) [2023-12-26 18:13:46,927][105620] Updated weights for policy 1, policy_version 393986 (0.0009) [2023-12-26 18:13:46,995][105620] Updated weights for policy 1, policy_version 393996 (0.0005) [2023-12-26 18:13:47,019][105692] Updated weights for policy 0, policy_version 393583 (0.0005) [2023-12-26 18:13:47,063][105620] Updated weights for policy 1, policy_version 394006 (0.0007) [2023-12-26 18:13:47,083][105692] Updated weights for policy 0, policy_version 393593 (0.0008) [2023-12-26 18:13:47,123][105620] Updated weights for policy 1, policy_version 394016 (0.0005) [2023-12-26 18:13:47,138][105692] Updated weights for policy 0, policy_version 393603 (0.0010) [2023-12-26 18:13:47,638][105620] Updated weights for policy 1, policy_version 394026 (0.0005) [2023-12-26 18:13:47,702][105620] Updated weights for policy 1, policy_version 394036 (0.0008) [2023-12-26 18:13:47,767][105620] Updated weights for policy 1, policy_version 394046 (0.0006) [2023-12-26 18:13:47,825][105692] Updated weights for policy 0, policy_version 393613 (0.0010) [2023-12-26 18:13:47,873][105692] Updated weights for policy 0, policy_version 393623 (0.0010) [2023-12-26 18:13:47,920][105692] Updated weights for policy 0, policy_version 393633 (0.0010) [2023-12-26 18:13:48,281][105620] Updated weights for policy 1, policy_version 394056 (0.0005) [2023-12-26 18:13:48,343][105620] Updated weights for policy 1, policy_version 394066 (0.0006) [2023-12-26 18:13:48,404][105620] Updated weights for policy 1, policy_version 394076 (0.0006) [2023-12-26 18:13:48,648][105692] Updated weights for policy 0, policy_version 393643 (0.0010) [2023-12-26 18:13:48,714][105692] Updated weights for policy 0, policy_version 393653 (0.0008) [2023-12-26 18:13:48,785][105692] Updated weights for policy 0, policy_version 393663 (0.0006) [2023-12-26 18:13:49,008][105620] Updated weights for policy 1, policy_version 394086 (0.0008) [2023-12-26 18:13:49,056][105620] Updated weights for policy 1, policy_version 394096 (0.0010) [2023-12-26 18:13:49,118][105620] Updated weights for policy 1, policy_version 394106 (0.0011) [2023-12-26 18:13:49,400][105692] Updated weights for policy 0, policy_version 393673 (0.0006) [2023-12-26 18:13:49,448][105692] Updated weights for policy 0, policy_version 393683 (0.0010) [2023-12-26 18:13:49,507][105692] Updated weights for policy 0, policy_version 393694 (0.0010) [2023-12-26 18:13:49,563][105692] Updated weights for policy 0, policy_version 393704 (0.0009) [2023-12-26 18:13:49,884][105620] Updated weights for policy 1, policy_version 394116 (0.0011) [2023-12-26 18:13:49,948][105620] Updated weights for policy 1, policy_version 394126 (0.0010) [2023-12-26 18:13:49,997][105620] Updated weights for policy 1, policy_version 394136 (0.0010) [2023-12-26 18:13:50,313][105692] Updated weights for policy 0, policy_version 393714 (0.0008) [2023-12-26 18:13:50,369][105692] Updated weights for policy 0, policy_version 393724 (0.0008) [2023-12-26 18:13:50,430][105692] Updated weights for policy 0, policy_version 393734 (0.0008) [2023-12-26 18:13:50,781][105620] Updated weights for policy 1, policy_version 394146 (0.0010) [2023-12-26 18:13:50,847][105620] Updated weights for policy 1, policy_version 394156 (0.0011) [2023-12-26 18:13:50,914][105620] Updated weights for policy 1, policy_version 394166 (0.0011) [2023-12-26 18:13:50,972][105620] Updated weights for policy 1, policy_version 394176 (0.0010) [2023-12-26 18:13:51,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 201728000. Throughput: 0: 10003.4, 1: 9617.8. Samples: 201716728. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 18:13:51,062][104569] Avg episode reward: [(0, '9355.487'), (1, '9264.798')] [2023-12-26 18:13:51,201][105692] Updated weights for policy 0, policy_version 393744 (0.0008) [2023-12-26 18:13:51,265][105692] Updated weights for policy 0, policy_version 393754 (0.0008) [2023-12-26 18:13:51,325][105692] Updated weights for policy 0, policy_version 393764 (0.0008) [2023-12-26 18:13:51,728][105620] Updated weights for policy 1, policy_version 394186 (0.0010) [2023-12-26 18:13:51,777][105620] Updated weights for policy 1, policy_version 394196 (0.0010) [2023-12-26 18:13:51,836][105620] Updated weights for policy 1, policy_version 394206 (0.0010) [2023-12-26 18:13:52,089][105692] Updated weights for policy 0, policy_version 393774 (0.0008) [2023-12-26 18:13:52,148][105585] KL-divergence is very high: 995.7990 [2023-12-26 18:13:52,155][105692] Updated weights for policy 0, policy_version 393784 (0.0008) [2023-12-26 18:13:52,194][105585] KL-divergence is very high: 1679.3993 [2023-12-26 18:13:52,212][105692] Updated weights for policy 0, policy_version 393794 (0.0008) [2023-12-26 18:13:52,240][105585] KL-divergence is very high: 1724.0605 [2023-12-26 18:13:52,610][105620] Updated weights for policy 1, policy_version 394216 (0.0011) [2023-12-26 18:13:52,655][105620] Updated weights for policy 1, policy_version 394226 (0.0010) [2023-12-26 18:13:52,708][105620] Updated weights for policy 1, policy_version 394236 (0.0010) [2023-12-26 18:13:52,955][105585] KL-divergence is very high: 132.7716 [2023-12-26 18:13:52,966][105692] Updated weights for policy 0, policy_version 393804 (0.0009) [2023-12-26 18:13:52,998][105585] KL-divergence is very high: 119.6144 [2023-12-26 18:13:53,017][105692] Updated weights for policy 0, policy_version 393814 (0.0011) [2023-12-26 18:13:53,043][105585] KL-divergence is very high: 107.4777 [2023-12-26 18:13:53,076][105692] Updated weights for policy 0, policy_version 393824 (0.0010) [2023-12-26 18:13:53,491][105620] Updated weights for policy 1, policy_version 394246 (0.0011) [2023-12-26 18:13:53,540][105620] Updated weights for policy 1, policy_version 394256 (0.0011) [2023-12-26 18:13:53,608][105620] Updated weights for policy 1, policy_version 394266 (0.0011) [2023-12-26 18:13:53,783][105692] Updated weights for policy 0, policy_version 393834 (0.0010) [2023-12-26 18:13:53,836][105692] Updated weights for policy 0, policy_version 393844 (0.0009) [2023-12-26 18:13:53,891][105692] Updated weights for policy 0, policy_version 393854 (0.0008) [2023-12-26 18:13:53,959][105692] Updated weights for policy 0, policy_version 393864 (0.0008) [2023-12-26 18:13:54,365][105620] Updated weights for policy 1, policy_version 394276 (0.0010) [2023-12-26 18:13:54,423][105620] Updated weights for policy 1, policy_version 394286 (0.0009) [2023-12-26 18:13:54,484][105620] Updated weights for policy 1, policy_version 394296 (0.0009) [2023-12-26 18:13:54,656][105692] Updated weights for policy 0, policy_version 393874 (0.0009) [2023-12-26 18:13:54,715][105692] Updated weights for policy 0, policy_version 393884 (0.0009) [2023-12-26 18:13:54,781][105692] Updated weights for policy 0, policy_version 393894 (0.0009) [2023-12-26 18:13:55,176][105620] Updated weights for policy 1, policy_version 394306 (0.0009) [2023-12-26 18:13:55,243][105620] Updated weights for policy 1, policy_version 394316 (0.0008) [2023-12-26 18:13:55,310][105620] Updated weights for policy 1, policy_version 394326 (0.0008) [2023-12-26 18:13:55,373][105620] Updated weights for policy 1, policy_version 394336 (0.0007) [2023-12-26 18:13:55,544][105692] Updated weights for policy 0, policy_version 393904 (0.0007) [2023-12-26 18:13:55,599][105692] Updated weights for policy 0, policy_version 393914 (0.0010) [2023-12-26 18:13:55,657][105692] Updated weights for policy 0, policy_version 393924 (0.0010) [2023-12-26 18:13:55,984][105620] Updated weights for policy 1, policy_version 394346 (0.0006) [2023-12-26 18:13:56,036][105620] Updated weights for policy 1, policy_version 394356 (0.0007) [2023-12-26 18:13:56,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 201818112. Throughput: 0: 9900.8, 1: 9654.6. Samples: 201830164. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 18:13:56,063][104569] Avg episode reward: [(0, '9355.521'), (1, '8991.072')] [2023-12-26 18:13:56,092][105620] Updated weights for policy 1, policy_version 394366 (0.0007) [2023-12-26 18:13:56,448][105692] Updated weights for policy 0, policy_version 393934 (0.0009) [2023-12-26 18:13:56,504][105692] Updated weights for policy 0, policy_version 393944 (0.0008) [2023-12-26 18:13:56,555][105692] Updated weights for policy 0, policy_version 393954 (0.0005) [2023-12-26 18:13:56,865][105620] Updated weights for policy 1, policy_version 394376 (0.0009) [2023-12-26 18:13:56,918][105620] Updated weights for policy 1, policy_version 394386 (0.0010) [2023-12-26 18:13:56,974][105620] Updated weights for policy 1, policy_version 394396 (0.0010) [2023-12-26 18:13:57,111][105692] Updated weights for policy 0, policy_version 393964 (0.0005) [2023-12-26 18:13:57,158][105692] Updated weights for policy 0, policy_version 393974 (0.0006) [2023-12-26 18:13:57,205][105692] Updated weights for policy 0, policy_version 393984 (0.0009) [2023-12-26 18:13:57,788][105620] Updated weights for policy 1, policy_version 394406 (0.0009) [2023-12-26 18:13:57,838][105620] Updated weights for policy 1, policy_version 394416 (0.0009) [2023-12-26 18:13:57,879][105692] Updated weights for policy 0, policy_version 393994 (0.0007) [2023-12-26 18:13:57,892][105620] Updated weights for policy 1, policy_version 394426 (0.0009) [2023-12-26 18:13:57,910][105585] KL-divergence is very high: 163.9445 [2023-12-26 18:13:57,934][105692] Updated weights for policy 0, policy_version 394004 (0.0008) [2023-12-26 18:13:57,956][105585] KL-divergence is very high: 155.6790 [2023-12-26 18:13:57,988][105692] Updated weights for policy 0, policy_version 394014 (0.0010) [2023-12-26 18:13:58,037][105692] Updated weights for policy 0, policy_version 394024 (0.0008) [2023-12-26 18:13:58,630][105620] Updated weights for policy 1, policy_version 394436 (0.0007) [2023-12-26 18:13:58,702][105620] Updated weights for policy 1, policy_version 394446 (0.0007) [2023-12-26 18:13:58,767][105620] Updated weights for policy 1, policy_version 394456 (0.0007) [2023-12-26 18:13:58,883][105692] Updated weights for policy 0, policy_version 394034 (0.0006) [2023-12-26 18:13:58,936][105692] Updated weights for policy 0, policy_version 394044 (0.0007) [2023-12-26 18:13:59,000][105692] Updated weights for policy 0, policy_version 394054 (0.0006) [2023-12-26 18:13:59,584][105620] Updated weights for policy 1, policy_version 394466 (0.0008) [2023-12-26 18:13:59,640][105620] Updated weights for policy 1, policy_version 394476 (0.0010) [2023-12-26 18:13:59,652][105692] Updated weights for policy 0, policy_version 394064 (0.0010) [2023-12-26 18:13:59,698][105620] Updated weights for policy 1, policy_version 394486 (0.0010) [2023-12-26 18:13:59,701][105692] Updated weights for policy 0, policy_version 394074 (0.0010) [2023-12-26 18:13:59,750][105692] Updated weights for policy 0, policy_version 394084 (0.0007) [2023-12-26 18:13:59,757][105620] Updated weights for policy 1, policy_version 394496 (0.0010) [2023-12-26 18:14:00,379][105692] Updated weights for policy 0, policy_version 394094 (0.0005) [2023-12-26 18:14:00,437][105692] Updated weights for policy 0, policy_version 394104 (0.0005) [2023-12-26 18:14:00,492][105692] Updated weights for policy 0, policy_version 394114 (0.0008) [2023-12-26 18:14:00,514][105620] Updated weights for policy 1, policy_version 394506 (0.0011) [2023-12-26 18:14:00,572][105620] Updated weights for policy 1, policy_version 394516 (0.0010) [2023-12-26 18:14:00,626][105620] Updated weights for policy 1, policy_version 394526 (0.0010) [2023-12-26 18:14:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 201916416. Throughput: 0: 9902.1, 1: 9671.9. Samples: 201886944. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 18:14:01,062][104569] Avg episode reward: [(0, '9263.248'), (1, '9174.284')] [2023-12-26 18:14:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000394528_101007360.pth... [2023-12-26 18:14:01,071][105692] Updated weights for policy 0, policy_version 394124 (0.0010) [2023-12-26 18:14:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000393376_100712448.pth [2023-12-26 18:14:01,128][105692] Updated weights for policy 0, policy_version 394134 (0.0011) [2023-12-26 18:14:01,189][105692] Updated weights for policy 0, policy_version 394144 (0.0011) [2023-12-26 18:14:01,225][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000394152_100917248.pth... [2023-12-26 18:14:01,228][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000392968_100614144.pth [2023-12-26 18:14:01,387][105620] Updated weights for policy 1, policy_version 394536 (0.0010) [2023-12-26 18:14:01,442][105620] Updated weights for policy 1, policy_version 394546 (0.0011) [2023-12-26 18:14:01,501][105620] Updated weights for policy 1, policy_version 394556 (0.0010) [2023-12-26 18:14:01,955][105692] Updated weights for policy 0, policy_version 394154 (0.0010) [2023-12-26 18:14:02,011][105692] Updated weights for policy 0, policy_version 394164 (0.0008) [2023-12-26 18:14:02,066][105692] Updated weights for policy 0, policy_version 394174 (0.0008) [2023-12-26 18:14:02,127][105692] Updated weights for policy 0, policy_version 394184 (0.0008) [2023-12-26 18:14:02,273][105620] Updated weights for policy 1, policy_version 394566 (0.0010) [2023-12-26 18:14:02,322][105620] Updated weights for policy 1, policy_version 394576 (0.0010) [2023-12-26 18:14:02,387][105620] Updated weights for policy 1, policy_version 394586 (0.0010) [2023-12-26 18:14:02,877][105692] Updated weights for policy 0, policy_version 394194 (0.0008) [2023-12-26 18:14:02,928][105692] Updated weights for policy 0, policy_version 394204 (0.0010) [2023-12-26 18:14:02,980][105692] Updated weights for policy 0, policy_version 394214 (0.0005) [2023-12-26 18:14:03,160][105620] Updated weights for policy 1, policy_version 394596 (0.0009) [2023-12-26 18:14:03,224][105620] Updated weights for policy 1, policy_version 394606 (0.0008) [2023-12-26 18:14:03,281][105620] Updated weights for policy 1, policy_version 394616 (0.0008) [2023-12-26 18:14:03,622][105692] Updated weights for policy 0, policy_version 394224 (0.0007) [2023-12-26 18:14:03,646][105585] KL-divergence is very high: 256.6252 [2023-12-26 18:14:03,671][105692] Updated weights for policy 0, policy_version 394234 (0.0007) [2023-12-26 18:14:03,689][105585] KL-divergence is very high: 325.8071 [2023-12-26 18:14:03,729][105692] Updated weights for policy 0, policy_version 394244 (0.0005) [2023-12-26 18:14:03,736][105585] KL-divergence is very high: 165.9957 [2023-12-26 18:14:03,990][105620] Updated weights for policy 1, policy_version 394626 (0.0008) [2023-12-26 18:14:04,054][105620] Updated weights for policy 1, policy_version 394636 (0.0008) [2023-12-26 18:14:04,114][105620] Updated weights for policy 1, policy_version 394646 (0.0009) [2023-12-26 18:14:04,173][105620] Updated weights for policy 1, policy_version 394656 (0.0011) [2023-12-26 18:14:04,442][105692] Updated weights for policy 0, policy_version 394254 (0.0008) [2023-12-26 18:14:04,491][105692] Updated weights for policy 0, policy_version 394264 (0.0008) [2023-12-26 18:14:04,541][105692] Updated weights for policy 0, policy_version 394274 (0.0008) [2023-12-26 18:14:04,838][105620] Updated weights for policy 1, policy_version 394666 (0.0009) [2023-12-26 18:14:04,887][105620] Updated weights for policy 1, policy_version 394676 (0.0008) [2023-12-26 18:14:04,943][105620] Updated weights for policy 1, policy_version 394686 (0.0010) [2023-12-26 18:14:05,444][105692] Updated weights for policy 0, policy_version 394284 (0.0010) [2023-12-26 18:14:05,497][105692] Updated weights for policy 0, policy_version 394295 (0.0010) [2023-12-26 18:14:05,546][105692] Updated weights for policy 0, policy_version 394305 (0.0009) [2023-12-26 18:14:05,547][105620] Updated weights for policy 1, policy_version 394696 (0.0006) [2023-12-26 18:14:05,608][105620] Updated weights for policy 1, policy_version 394706 (0.0005) [2023-12-26 18:14:05,669][105620] Updated weights for policy 1, policy_version 394716 (0.0008) [2023-12-26 18:14:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 202014720. Throughput: 0: 9832.0, 1: 9758.9. Samples: 202004040. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 18:14:06,062][104569] Avg episode reward: [(0, '9262.780'), (1, '9265.664')] [2023-12-26 18:14:06,348][105692] Updated weights for policy 0, policy_version 394315 (0.0009) [2023-12-26 18:14:06,382][105620] Updated weights for policy 1, policy_version 394726 (0.0011) [2023-12-26 18:14:06,401][105692] Updated weights for policy 0, policy_version 394325 (0.0009) [2023-12-26 18:14:06,444][105620] Updated weights for policy 1, policy_version 394736 (0.0010) [2023-12-26 18:14:06,464][105692] Updated weights for policy 0, policy_version 394335 (0.0006) [2023-12-26 18:14:06,504][105620] Updated weights for policy 1, policy_version 394746 (0.0006) [2023-12-26 18:14:07,084][105620] Updated weights for policy 1, policy_version 394756 (0.0006) [2023-12-26 18:14:07,147][105620] Updated weights for policy 1, policy_version 394766 (0.0005) [2023-12-26 18:14:07,208][105620] Updated weights for policy 1, policy_version 394776 (0.0005) [2023-12-26 18:14:07,370][105692] Updated weights for policy 0, policy_version 394345 (0.0008) [2023-12-26 18:14:07,417][105692] Updated weights for policy 0, policy_version 394355 (0.0008) [2023-12-26 18:14:07,477][105692] Updated weights for policy 0, policy_version 394365 (0.0009) [2023-12-26 18:14:07,533][105692] Updated weights for policy 0, policy_version 394376 (0.0010) [2023-12-26 18:14:07,776][105620] Updated weights for policy 1, policy_version 394786 (0.0006) [2023-12-26 18:14:07,841][105620] Updated weights for policy 1, policy_version 394796 (0.0009) [2023-12-26 18:14:07,900][105620] Updated weights for policy 1, policy_version 394806 (0.0008) [2023-12-26 18:14:07,947][105620] Updated weights for policy 1, policy_version 394816 (0.0008) [2023-12-26 18:14:08,371][105692] Updated weights for policy 0, policy_version 394386 (0.0009) [2023-12-26 18:14:08,438][105692] Updated weights for policy 0, policy_version 394396 (0.0008) [2023-12-26 18:14:08,505][105692] Updated weights for policy 0, policy_version 394406 (0.0008) [2023-12-26 18:14:08,662][105620] Updated weights for policy 1, policy_version 394826 (0.0009) [2023-12-26 18:14:08,715][105620] Updated weights for policy 1, policy_version 394836 (0.0009) [2023-12-26 18:14:08,772][105620] Updated weights for policy 1, policy_version 394846 (0.0009) [2023-12-26 18:14:09,204][105692] Updated weights for policy 0, policy_version 394416 (0.0009) [2023-12-26 18:14:09,258][105692] Updated weights for policy 0, policy_version 394426 (0.0009) [2023-12-26 18:14:09,305][105692] Updated weights for policy 0, policy_version 394436 (0.0008) [2023-12-26 18:14:09,471][105620] Updated weights for policy 1, policy_version 394856 (0.0008) [2023-12-26 18:14:09,539][105620] Updated weights for policy 1, policy_version 394866 (0.0008) [2023-12-26 18:14:09,613][105620] Updated weights for policy 1, policy_version 394876 (0.0008) [2023-12-26 18:14:10,130][105692] Updated weights for policy 0, policy_version 394446 (0.0009) [2023-12-26 18:14:10,178][105692] Updated weights for policy 0, policy_version 394456 (0.0009) [2023-12-26 18:14:10,237][105692] Updated weights for policy 0, policy_version 394466 (0.0009) [2023-12-26 18:14:10,345][105620] Updated weights for policy 1, policy_version 394886 (0.0007) [2023-12-26 18:14:10,413][105620] Updated weights for policy 1, policy_version 394896 (0.0008) [2023-12-26 18:14:10,467][105620] Updated weights for policy 1, policy_version 394906 (0.0008) [2023-12-26 18:14:11,012][105692] Updated weights for policy 0, policy_version 394476 (0.0008) [2023-12-26 18:14:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 202104832. Throughput: 0: 9686.8, 1: 9837.1. Samples: 202117848. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 18:14:11,062][104569] Avg episode reward: [(0, '9262.636'), (1, '8639.501')] [2023-12-26 18:14:11,072][105692] Updated weights for policy 0, policy_version 394486 (0.0008) [2023-12-26 18:14:11,132][105692] Updated weights for policy 0, policy_version 394496 (0.0009) [2023-12-26 18:14:11,232][105620] Updated weights for policy 1, policy_version 394916 (0.0008) [2023-12-26 18:14:11,286][105620] Updated weights for policy 1, policy_version 394926 (0.0010) [2023-12-26 18:14:11,343][105620] Updated weights for policy 1, policy_version 394936 (0.0007) [2023-12-26 18:14:11,915][105692] Updated weights for policy 0, policy_version 394506 (0.0007) [2023-12-26 18:14:11,976][105692] Updated weights for policy 0, policy_version 394516 (0.0008) [2023-12-26 18:14:12,028][105692] Updated weights for policy 0, policy_version 394526 (0.0008) [2023-12-26 18:14:12,082][105692] Updated weights for policy 0, policy_version 394536 (0.0007) [2023-12-26 18:14:12,085][105620] Updated weights for policy 1, policy_version 394946 (0.0009) [2023-12-26 18:14:12,137][105620] Updated weights for policy 1, policy_version 394956 (0.0005) [2023-12-26 18:14:12,185][105620] Updated weights for policy 1, policy_version 394966 (0.0005) [2023-12-26 18:14:12,248][105620] Updated weights for policy 1, policy_version 394976 (0.0005) [2023-12-26 18:14:12,774][105692] Updated weights for policy 0, policy_version 394546 (0.0005) [2023-12-26 18:14:12,793][105585] KL-divergence is very high: 134.0957 [2023-12-26 18:14:12,802][105620] Updated weights for policy 1, policy_version 394986 (0.0009) [2023-12-26 18:14:12,834][105692] Updated weights for policy 0, policy_version 394556 (0.0005) [2023-12-26 18:14:12,839][105585] KL-divergence is very high: 173.7460 [2023-12-26 18:14:12,859][105620] Updated weights for policy 1, policy_version 394996 (0.0009) [2023-12-26 18:14:12,882][105692] Updated weights for policy 0, policy_version 394566 (0.0005) [2023-12-26 18:14:12,919][105620] Updated weights for policy 1, policy_version 395006 (0.0009) [2023-12-26 18:14:13,610][105620] Updated weights for policy 1, policy_version 395016 (0.0007) [2023-12-26 18:14:13,637][105692] Updated weights for policy 0, policy_version 394576 (0.0006) [2023-12-26 18:14:13,663][105620] Updated weights for policy 1, policy_version 395026 (0.0007) [2023-12-26 18:14:13,679][105692] Updated weights for policy 0, policy_version 394586 (0.0006) [2023-12-26 18:14:13,709][105620] Updated weights for policy 1, policy_version 395036 (0.0006) [2023-12-26 18:14:13,738][105692] Updated weights for policy 0, policy_version 394596 (0.0007) [2023-12-26 18:14:14,326][105620] Updated weights for policy 1, policy_version 395046 (0.0008) [2023-12-26 18:14:14,392][105620] Updated weights for policy 1, policy_version 395056 (0.0010) [2023-12-26 18:14:14,454][105620] Updated weights for policy 1, policy_version 395066 (0.0010) [2023-12-26 18:14:14,503][105692] Updated weights for policy 0, policy_version 394606 (0.0007) [2023-12-26 18:14:14,558][105692] Updated weights for policy 0, policy_version 394616 (0.0008) [2023-12-26 18:14:14,616][105692] Updated weights for policy 0, policy_version 394626 (0.0009) [2023-12-26 18:14:15,266][105620] Updated weights for policy 1, policy_version 395076 (0.0010) [2023-12-26 18:14:15,305][105692] Updated weights for policy 0, policy_version 394636 (0.0009) [2023-12-26 18:14:15,315][105620] Updated weights for policy 1, policy_version 395086 (0.0008) [2023-12-26 18:14:15,366][105692] Updated weights for policy 0, policy_version 394646 (0.0007) [2023-12-26 18:14:15,373][105620] Updated weights for policy 1, policy_version 395096 (0.0006) [2023-12-26 18:14:15,427][105692] Updated weights for policy 0, policy_version 394656 (0.0007) [2023-12-26 18:14:16,048][105692] Updated weights for policy 0, policy_version 394666 (0.0008) [2023-12-26 18:14:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 202203136. Throughput: 0: 9603.3, 1: 9908.9. Samples: 202177752. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 18:14:16,063][104569] Avg episode reward: [(0, '9354.998'), (1, '8555.730')] [2023-12-26 18:14:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000395104_101154816.pth... [2023-12-26 18:14:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000393952_100859904.pth [2023-12-26 18:14:16,106][105692] Updated weights for policy 0, policy_version 394676 (0.0009) [2023-12-26 18:14:16,160][105692] Updated weights for policy 0, policy_version 394686 (0.0009) [2023-12-26 18:14:16,199][105620] Updated weights for policy 1, policy_version 395106 (0.0008) [2023-12-26 18:14:16,210][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000394696_101056512.pth... [2023-12-26 18:14:16,212][105692] Updated weights for policy 0, policy_version 394696 (0.0009) [2023-12-26 18:14:16,214][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000393544_100761600.pth [2023-12-26 18:14:16,251][105620] Updated weights for policy 1, policy_version 395116 (0.0008) [2023-12-26 18:14:16,304][105620] Updated weights for policy 1, policy_version 395126 (0.0008) [2023-12-26 18:14:16,350][105620] Updated weights for policy 1, policy_version 395136 (0.0008) [2023-12-26 18:14:16,953][105692] Updated weights for policy 0, policy_version 394706 (0.0009) [2023-12-26 18:14:17,012][105692] Updated weights for policy 0, policy_version 394716 (0.0009) [2023-12-26 18:14:17,066][105692] Updated weights for policy 0, policy_version 394726 (0.0009) [2023-12-26 18:14:17,122][105620] Updated weights for policy 1, policy_version 395146 (0.0005) [2023-12-26 18:14:17,177][105620] Updated weights for policy 1, policy_version 395156 (0.0005) [2023-12-26 18:14:17,229][105620] Updated weights for policy 1, policy_version 395166 (0.0005) [2023-12-26 18:14:17,825][105620] Updated weights for policy 1, policy_version 395176 (0.0008) [2023-12-26 18:14:17,882][105620] Updated weights for policy 1, policy_version 395186 (0.0006) [2023-12-26 18:14:17,884][105692] Updated weights for policy 0, policy_version 394736 (0.0009) [2023-12-26 18:14:17,934][105692] Updated weights for policy 0, policy_version 394746 (0.0007) [2023-12-26 18:14:17,936][105620] Updated weights for policy 1, policy_version 395196 (0.0006) [2023-12-26 18:14:17,981][105692] Updated weights for policy 0, policy_version 394756 (0.0007) [2023-12-26 18:14:18,652][105692] Updated weights for policy 0, policy_version 394766 (0.0006) [2023-12-26 18:14:18,704][105692] Updated weights for policy 0, policy_version 394776 (0.0005) [2023-12-26 18:14:18,748][105620] Updated weights for policy 1, policy_version 395206 (0.0009) [2023-12-26 18:14:18,760][105692] Updated weights for policy 0, policy_version 394786 (0.0005) [2023-12-26 18:14:18,810][105620] Updated weights for policy 1, policy_version 395216 (0.0009) [2023-12-26 18:14:18,867][105620] Updated weights for policy 1, policy_version 395226 (0.0009) [2023-12-26 18:14:19,431][105692] Updated weights for policy 0, policy_version 394796 (0.0006) [2023-12-26 18:14:19,497][105692] Updated weights for policy 0, policy_version 394806 (0.0007) [2023-12-26 18:14:19,561][105692] Updated weights for policy 0, policy_version 394816 (0.0008) [2023-12-26 18:14:19,712][105620] Updated weights for policy 1, policy_version 395236 (0.0009) [2023-12-26 18:14:19,780][105620] Updated weights for policy 1, policy_version 395246 (0.0009) [2023-12-26 18:14:19,839][105620] Updated weights for policy 1, policy_version 395256 (0.0009) [2023-12-26 18:14:20,178][105692] Updated weights for policy 0, policy_version 394826 (0.0006) [2023-12-26 18:14:20,240][105692] Updated weights for policy 0, policy_version 394836 (0.0007) [2023-12-26 18:14:20,293][105692] Updated weights for policy 0, policy_version 394846 (0.0010) [2023-12-26 18:14:20,341][105692] Updated weights for policy 0, policy_version 394856 (0.0010) [2023-12-26 18:14:20,675][105620] Updated weights for policy 1, policy_version 395266 (0.0009) [2023-12-26 18:14:20,728][105620] Updated weights for policy 1, policy_version 395276 (0.0008) [2023-12-26 18:14:20,781][105620] Updated weights for policy 1, policy_version 395286 (0.0008) [2023-12-26 18:14:20,837][105620] Updated weights for policy 1, policy_version 395296 (0.0008) [2023-12-26 18:14:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 202301440. Throughput: 0: 9687.1, 1: 9830.7. Samples: 202292156. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 18:14:21,062][104569] Avg episode reward: [(0, '9079.008'), (1, '8912.669')] [2023-12-26 18:14:21,098][105692] Updated weights for policy 0, policy_version 394866 (0.0009) [2023-12-26 18:14:21,162][105692] Updated weights for policy 0, policy_version 394876 (0.0009) [2023-12-26 18:14:21,228][105692] Updated weights for policy 0, policy_version 394886 (0.0010) [2023-12-26 18:14:21,582][105620] Updated weights for policy 1, policy_version 395306 (0.0007) [2023-12-26 18:14:21,649][105620] Updated weights for policy 1, policy_version 395316 (0.0008) [2023-12-26 18:14:21,709][105620] Updated weights for policy 1, policy_version 395326 (0.0008) [2023-12-26 18:14:22,021][105692] Updated weights for policy 0, policy_version 394896 (0.0010) [2023-12-26 18:14:22,082][105692] Updated weights for policy 0, policy_version 394906 (0.0011) [2023-12-26 18:14:22,152][105692] Updated weights for policy 0, policy_version 394916 (0.0011) [2023-12-26 18:14:22,458][105620] Updated weights for policy 1, policy_version 395336 (0.0006) [2023-12-26 18:14:22,522][105620] Updated weights for policy 1, policy_version 395346 (0.0006) [2023-12-26 18:14:22,581][105620] Updated weights for policy 1, policy_version 395356 (0.0007) [2023-12-26 18:14:22,895][105692] Updated weights for policy 0, policy_version 394926 (0.0010) [2023-12-26 18:14:22,945][105692] Updated weights for policy 0, policy_version 394936 (0.0010) [2023-12-26 18:14:22,994][105692] Updated weights for policy 0, policy_version 394946 (0.0010) [2023-12-26 18:14:23,308][105620] Updated weights for policy 1, policy_version 395366 (0.0008) [2023-12-26 18:14:23,356][105620] Updated weights for policy 1, policy_version 395376 (0.0007) [2023-12-26 18:14:23,416][105620] Updated weights for policy 1, policy_version 395386 (0.0008) [2023-12-26 18:14:23,712][105692] Updated weights for policy 0, policy_version 394956 (0.0010) [2023-12-26 18:14:23,768][105692] Updated weights for policy 0, policy_version 394966 (0.0010) [2023-12-26 18:14:23,820][105692] Updated weights for policy 0, policy_version 394976 (0.0010) [2023-12-26 18:14:24,244][105620] Updated weights for policy 1, policy_version 395396 (0.0009) [2023-12-26 18:14:24,303][105620] Updated weights for policy 1, policy_version 395406 (0.0008) [2023-12-26 18:14:24,368][105620] Updated weights for policy 1, policy_version 395416 (0.0008) [2023-12-26 18:14:24,531][105692] Updated weights for policy 0, policy_version 394986 (0.0009) [2023-12-26 18:14:24,587][105692] Updated weights for policy 0, policy_version 394996 (0.0008) [2023-12-26 18:14:24,638][105692] Updated weights for policy 0, policy_version 395006 (0.0009) [2023-12-26 18:14:24,698][105692] Updated weights for policy 0, policy_version 395016 (0.0010) [2023-12-26 18:14:24,954][105620] Updated weights for policy 1, policy_version 395426 (0.0009) [2023-12-26 18:14:25,016][105620] Updated weights for policy 1, policy_version 395436 (0.0008) [2023-12-26 18:14:25,071][105620] Updated weights for policy 1, policy_version 395446 (0.0008) [2023-12-26 18:14:25,133][105620] Updated weights for policy 1, policy_version 395456 (0.0008) [2023-12-26 18:14:25,356][105692] Updated weights for policy 0, policy_version 395026 (0.0005) [2023-12-26 18:14:25,408][105692] Updated weights for policy 0, policy_version 395036 (0.0005) [2023-12-26 18:14:25,457][105692] Updated weights for policy 0, policy_version 395046 (0.0010) [2023-12-26 18:14:25,850][105620] Updated weights for policy 1, policy_version 395466 (0.0010) [2023-12-26 18:14:25,908][105620] Updated weights for policy 1, policy_version 395476 (0.0010) [2023-12-26 18:14:25,973][105620] Updated weights for policy 1, policy_version 395487 (0.0007) [2023-12-26 18:14:26,038][105692] Updated weights for policy 0, policy_version 395056 (0.0006) [2023-12-26 18:14:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 202399744. Throughput: 0: 9657.8, 1: 9817.6. Samples: 202407036. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 18:14:26,062][104569] Avg episode reward: [(0, '9079.050'), (1, '8912.740')] [2023-12-26 18:14:26,090][105692] Updated weights for policy 0, policy_version 395066 (0.0007) [2023-12-26 18:14:26,151][105692] Updated weights for policy 0, policy_version 395076 (0.0011) [2023-12-26 18:14:26,701][105620] Updated weights for policy 1, policy_version 395497 (0.0009) [2023-12-26 18:14:26,740][105692] Updated weights for policy 0, policy_version 395086 (0.0007) [2023-12-26 18:14:26,745][105620] Updated weights for policy 1, policy_version 395507 (0.0005) [2023-12-26 18:14:26,796][105620] Updated weights for policy 1, policy_version 395517 (0.0008) [2023-12-26 18:14:26,799][105692] Updated weights for policy 0, policy_version 395096 (0.0005) [2023-12-26 18:14:26,847][105692] Updated weights for policy 0, policy_version 395106 (0.0009) [2023-12-26 18:14:27,386][105692] Updated weights for policy 0, policy_version 395116 (0.0007) [2023-12-26 18:14:27,448][105692] Updated weights for policy 0, policy_version 395126 (0.0010) [2023-12-26 18:14:27,507][105692] Updated weights for policy 0, policy_version 395136 (0.0010) [2023-12-26 18:14:27,633][105620] Updated weights for policy 1, policy_version 395527 (0.0008) [2023-12-26 18:14:27,653][105586] KL-divergence is very high: 186.4092 [2023-12-26 18:14:27,695][105620] Updated weights for policy 1, policy_version 395537 (0.0010) [2023-12-26 18:14:27,699][105586] KL-divergence is very high: 234.3918 [2023-12-26 18:14:27,738][105586] KL-divergence is very high: 297.4324 [2023-12-26 18:14:27,742][105620] Updated weights for policy 1, policy_version 395547 (0.0008) [2023-12-26 18:14:28,123][105692] Updated weights for policy 0, policy_version 395146 (0.0010) [2023-12-26 18:14:28,178][105692] Updated weights for policy 0, policy_version 395156 (0.0010) [2023-12-26 18:14:28,235][105692] Updated weights for policy 0, policy_version 395166 (0.0006) [2023-12-26 18:14:28,293][105692] Updated weights for policy 0, policy_version 395176 (0.0005) [2023-12-26 18:14:28,462][105620] Updated weights for policy 1, policy_version 395557 (0.0009) [2023-12-26 18:14:28,520][105620] Updated weights for policy 1, policy_version 395567 (0.0010) [2023-12-26 18:14:28,574][105620] Updated weights for policy 1, policy_version 395577 (0.0005) [2023-12-26 18:14:28,850][105692] Updated weights for policy 0, policy_version 395186 (0.0005) [2023-12-26 18:14:28,900][105692] Updated weights for policy 0, policy_version 395196 (0.0008) [2023-12-26 18:14:28,951][105692] Updated weights for policy 0, policy_version 395206 (0.0010) [2023-12-26 18:14:29,215][105620] Updated weights for policy 1, policy_version 395587 (0.0007) [2023-12-26 18:14:29,275][105620] Updated weights for policy 1, policy_version 395597 (0.0009) [2023-12-26 18:14:29,335][105620] Updated weights for policy 1, policy_version 395607 (0.0009) [2023-12-26 18:14:29,711][105692] Updated weights for policy 0, policy_version 395216 (0.0010) [2023-12-26 18:14:29,773][105692] Updated weights for policy 0, policy_version 395226 (0.0010) [2023-12-26 18:14:29,825][105692] Updated weights for policy 0, policy_version 395236 (0.0009) [2023-12-26 18:14:29,988][105620] Updated weights for policy 1, policy_version 395617 (0.0008) [2023-12-26 18:14:30,050][105620] Updated weights for policy 1, policy_version 395627 (0.0008) [2023-12-26 18:14:30,107][105620] Updated weights for policy 1, policy_version 395637 (0.0008) [2023-12-26 18:14:30,165][105620] Updated weights for policy 1, policy_version 395647 (0.0009) [2023-12-26 18:14:30,608][105692] Updated weights for policy 0, policy_version 395246 (0.0008) [2023-12-26 18:14:30,656][105692] Updated weights for policy 0, policy_version 395256 (0.0009) [2023-12-26 18:14:30,703][105692] Updated weights for policy 0, policy_version 395266 (0.0009) [2023-12-26 18:14:30,909][105620] Updated weights for policy 1, policy_version 395657 (0.0011) [2023-12-26 18:14:30,960][105620] Updated weights for policy 1, policy_version 395667 (0.0010) [2023-12-26 18:14:31,019][105620] Updated weights for policy 1, policy_version 395677 (0.0010) [2023-12-26 18:14:31,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 202506240. Throughput: 0: 9830.7, 1: 9727.7. Samples: 202470988. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 18:14:31,063][104569] Avg episode reward: [(0, '9263.247'), (1, '8818.962')] [2023-12-26 18:14:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000395272_101203968.pth... [2023-12-26 18:14:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000395680_101302272.pth... [2023-12-26 18:14:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000394152_100917248.pth [2023-12-26 18:14:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000394528_101007360.pth [2023-12-26 18:14:31,553][105692] Updated weights for policy 0, policy_version 395276 (0.0009) [2023-12-26 18:14:31,618][105692] Updated weights for policy 0, policy_version 395286 (0.0008) [2023-12-26 18:14:31,674][105692] Updated weights for policy 0, policy_version 395296 (0.0009) [2023-12-26 18:14:31,784][105620] Updated weights for policy 1, policy_version 395687 (0.0010) [2023-12-26 18:14:31,848][105620] Updated weights for policy 1, policy_version 395697 (0.0008) [2023-12-26 18:14:31,904][105620] Updated weights for policy 1, policy_version 395707 (0.0008) [2023-12-26 18:14:32,450][105692] Updated weights for policy 0, policy_version 395306 (0.0008) [2023-12-26 18:14:32,505][105692] Updated weights for policy 0, policy_version 395316 (0.0008) [2023-12-26 18:14:32,564][105692] Updated weights for policy 0, policy_version 395326 (0.0008) [2023-12-26 18:14:32,579][105620] Updated weights for policy 1, policy_version 395717 (0.0008) [2023-12-26 18:14:32,616][105692] Updated weights for policy 0, policy_version 395336 (0.0007) [2023-12-26 18:14:32,634][105620] Updated weights for policy 1, policy_version 395727 (0.0011) [2023-12-26 18:14:32,698][105620] Updated weights for policy 1, policy_version 395737 (0.0011) [2023-12-26 18:14:33,388][105692] Updated weights for policy 0, policy_version 395346 (0.0008) [2023-12-26 18:14:33,439][105620] Updated weights for policy 1, policy_version 395747 (0.0009) [2023-12-26 18:14:33,449][105692] Updated weights for policy 0, policy_version 395357 (0.0009) [2023-12-26 18:14:33,493][105620] Updated weights for policy 1, policy_version 395757 (0.0005) [2023-12-26 18:14:33,517][105692] Updated weights for policy 0, policy_version 395367 (0.0008) [2023-12-26 18:14:33,555][105620] Updated weights for policy 1, policy_version 395767 (0.0005) [2023-12-26 18:14:34,247][105620] Updated weights for policy 1, policy_version 395777 (0.0008) [2023-12-26 18:14:34,277][105692] Updated weights for policy 0, policy_version 395377 (0.0007) [2023-12-26 18:14:34,311][105620] Updated weights for policy 1, policy_version 395787 (0.0010) [2023-12-26 18:14:34,345][105692] Updated weights for policy 0, policy_version 395387 (0.0006) [2023-12-26 18:14:34,368][105620] Updated weights for policy 1, policy_version 395797 (0.0009) [2023-12-26 18:14:34,406][105692] Updated weights for policy 0, policy_version 395397 (0.0007) [2023-12-26 18:14:34,426][105620] Updated weights for policy 1, policy_version 395807 (0.0008) [2023-12-26 18:14:35,089][105692] Updated weights for policy 0, policy_version 395407 (0.0006) [2023-12-26 18:14:35,115][105620] Updated weights for policy 1, policy_version 395817 (0.0006) [2023-12-26 18:14:35,138][105692] Updated weights for policy 0, policy_version 395417 (0.0007) [2023-12-26 18:14:35,166][105620] Updated weights for policy 1, policy_version 395827 (0.0005) [2023-12-26 18:14:35,193][105692] Updated weights for policy 0, policy_version 395427 (0.0010) [2023-12-26 18:14:35,216][105620] Updated weights for policy 1, policy_version 395837 (0.0006) [2023-12-26 18:14:35,739][105620] Updated weights for policy 1, policy_version 395847 (0.0005) [2023-12-26 18:14:35,796][105620] Updated weights for policy 1, policy_version 395857 (0.0005) [2023-12-26 18:14:35,845][105620] Updated weights for policy 1, policy_version 395867 (0.0005) [2023-12-26 18:14:35,866][105692] Updated weights for policy 0, policy_version 395437 (0.0008) [2023-12-26 18:14:35,927][105692] Updated weights for policy 0, policy_version 395447 (0.0009) [2023-12-26 18:14:35,988][105692] Updated weights for policy 0, policy_version 395457 (0.0010) [2023-12-26 18:14:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 202604544. Throughput: 0: 9677.2, 1: 9606.0. Samples: 202584476. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 18:14:36,062][104569] Avg episode reward: [(0, '9263.260'), (1, '8826.898')] [2023-12-26 18:14:36,468][105620] Updated weights for policy 1, policy_version 395877 (0.0008) [2023-12-26 18:14:36,531][105620] Updated weights for policy 1, policy_version 395887 (0.0010) [2023-12-26 18:14:36,591][105620] Updated weights for policy 1, policy_version 395897 (0.0005) [2023-12-26 18:14:36,670][105692] Updated weights for policy 0, policy_version 395467 (0.0010) [2023-12-26 18:14:36,726][105692] Updated weights for policy 0, policy_version 395477 (0.0010) [2023-12-26 18:14:36,772][105585] KL-divergence is very high: 103.5219 [2023-12-26 18:14:36,786][105692] Updated weights for policy 0, policy_version 395487 (0.0011) [2023-12-26 18:14:37,345][105620] Updated weights for policy 1, policy_version 395907 (0.0008) [2023-12-26 18:14:37,404][105620] Updated weights for policy 1, policy_version 395917 (0.0008) [2023-12-26 18:14:37,460][105620] Updated weights for policy 1, policy_version 395927 (0.0008) [2023-12-26 18:14:37,505][105692] Updated weights for policy 0, policy_version 395497 (0.0011) [2023-12-26 18:14:37,557][105692] Updated weights for policy 0, policy_version 395507 (0.0010) [2023-12-26 18:14:37,609][105692] Updated weights for policy 0, policy_version 395517 (0.0010) [2023-12-26 18:14:37,670][105692] Updated weights for policy 0, policy_version 395527 (0.0011) [2023-12-26 18:14:38,275][105620] Updated weights for policy 1, policy_version 395937 (0.0008) [2023-12-26 18:14:38,340][105620] Updated weights for policy 1, policy_version 395947 (0.0008) [2023-12-26 18:14:38,341][105692] Updated weights for policy 0, policy_version 395537 (0.0007) [2023-12-26 18:14:38,353][105585] KL-divergence is very high: 119.5744 [2023-12-26 18:14:38,391][105585] KL-divergence is very high: 174.8753 [2023-12-26 18:14:38,397][105692] Updated weights for policy 0, policy_version 395547 (0.0007) [2023-12-26 18:14:38,402][105620] Updated weights for policy 1, policy_version 395957 (0.0008) [2023-12-26 18:14:38,414][105585] KL-divergence is very high: 210.5699 [2023-12-26 18:14:38,434][105585] KL-divergence is very high: 281.5050 [2023-12-26 18:14:38,447][105692] Updated weights for policy 0, policy_version 395557 (0.0005) [2023-12-26 18:14:38,452][105585] KL-divergence is very high: 195.1938 [2023-12-26 18:14:38,452][105620] Updated weights for policy 1, policy_version 395967 (0.0010) [2023-12-26 18:14:39,084][105692] Updated weights for policy 0, policy_version 395567 (0.0009) [2023-12-26 18:14:39,148][105692] Updated weights for policy 0, policy_version 395577 (0.0007) [2023-12-26 18:14:39,203][105692] Updated weights for policy 0, policy_version 395587 (0.0010) [2023-12-26 18:14:39,299][105620] Updated weights for policy 1, policy_version 395977 (0.0009) [2023-12-26 18:14:39,368][105620] Updated weights for policy 1, policy_version 395987 (0.0012) [2023-12-26 18:14:39,429][105620] Updated weights for policy 1, policy_version 395997 (0.0010) [2023-12-26 18:14:39,989][105692] Updated weights for policy 0, policy_version 395597 (0.0010) [2023-12-26 18:14:40,057][105692] Updated weights for policy 0, policy_version 395607 (0.0007) [2023-12-26 18:14:40,120][105692] Updated weights for policy 0, policy_version 395617 (0.0010) [2023-12-26 18:14:40,171][105620] Updated weights for policy 1, policy_version 396007 (0.0007) [2023-12-26 18:14:40,229][105620] Updated weights for policy 1, policy_version 396017 (0.0006) [2023-12-26 18:14:40,298][105620] Updated weights for policy 1, policy_version 396027 (0.0006) [2023-12-26 18:14:40,751][105692] Updated weights for policy 0, policy_version 395627 (0.0009) [2023-12-26 18:14:40,809][105692] Updated weights for policy 0, policy_version 395637 (0.0006) [2023-12-26 18:14:40,862][105692] Updated weights for policy 0, policy_version 395647 (0.0009) [2023-12-26 18:14:40,929][105620] Updated weights for policy 1, policy_version 396037 (0.0009) [2023-12-26 18:14:40,988][105620] Updated weights for policy 1, policy_version 396047 (0.0010) [2023-12-26 18:14:41,059][105620] Updated weights for policy 1, policy_version 396057 (0.0010) [2023-12-26 18:14:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 202694656. Throughput: 0: 9772.5, 1: 9676.5. Samples: 202705372. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 18:14:41,062][104569] Avg episode reward: [(0, '8895.488'), (1, '8638.683')] [2023-12-26 18:14:41,576][105692] Updated weights for policy 0, policy_version 395657 (0.0010) [2023-12-26 18:14:41,642][105692] Updated weights for policy 0, policy_version 395667 (0.0008) [2023-12-26 18:14:41,701][105692] Updated weights for policy 0, policy_version 395677 (0.0007) [2023-12-26 18:14:41,765][105692] Updated weights for policy 0, policy_version 395687 (0.0008) [2023-12-26 18:14:41,816][105620] Updated weights for policy 1, policy_version 396067 (0.0009) [2023-12-26 18:14:41,881][105620] Updated weights for policy 1, policy_version 396077 (0.0009) [2023-12-26 18:14:41,882][105586] KL-divergence is very high: 114.2398 [2023-12-26 18:14:41,937][105586] KL-divergence is very high: 206.0532 [2023-12-26 18:14:41,953][105620] Updated weights for policy 1, policy_version 396087 (0.0009) [2023-12-26 18:14:41,996][105586] KL-divergence is very high: 220.0948 [2023-12-26 18:14:42,453][105692] Updated weights for policy 0, policy_version 395697 (0.0009) [2023-12-26 18:14:42,508][105692] Updated weights for policy 0, policy_version 395707 (0.0006) [2023-12-26 18:14:42,576][105692] Updated weights for policy 0, policy_version 395717 (0.0009) [2023-12-26 18:14:42,672][105620] Updated weights for policy 1, policy_version 396097 (0.0009) [2023-12-26 18:14:42,726][105620] Updated weights for policy 1, policy_version 396107 (0.0009) [2023-12-26 18:14:42,778][105620] Updated weights for policy 1, policy_version 396117 (0.0009) [2023-12-26 18:14:42,835][105620] Updated weights for policy 1, policy_version 396127 (0.0009) [2023-12-26 18:14:43,283][105692] Updated weights for policy 0, policy_version 395727 (0.0007) [2023-12-26 18:14:43,330][105585] KL-divergence is very high: 1366.5880 [2023-12-26 18:14:43,342][105692] Updated weights for policy 0, policy_version 395737 (0.0009) [2023-12-26 18:14:43,366][105585] KL-divergence is very high: 111.9872 [2023-12-26 18:14:43,378][105585] KL-divergence is very high: 2317.6526 [2023-12-26 18:14:43,399][105692] Updated weights for policy 0, policy_version 395747 (0.0008) [2023-12-26 18:14:43,410][105585] KL-divergence is very high: 104.7864 [2023-12-26 18:14:43,420][105585] KL-divergence is very high: 2354.1350 [2023-12-26 18:14:43,627][105620] Updated weights for policy 1, policy_version 396137 (0.0009) [2023-12-26 18:14:43,673][105620] Updated weights for policy 1, policy_version 396147 (0.0009) [2023-12-26 18:14:43,735][105620] Updated weights for policy 1, policy_version 396157 (0.0009) [2023-12-26 18:14:44,071][105585] KL-divergence is very high: 112.5442 [2023-12-26 18:14:44,082][105692] Updated weights for policy 0, policy_version 395757 (0.0007) [2023-12-26 18:14:44,136][105692] Updated weights for policy 0, policy_version 395767 (0.0005) [2023-12-26 18:14:44,184][105692] Updated weights for policy 0, policy_version 395777 (0.0005) [2023-12-26 18:14:44,588][105620] Updated weights for policy 1, policy_version 396167 (0.0009) [2023-12-26 18:14:44,638][105620] Updated weights for policy 1, policy_version 396177 (0.0009) [2023-12-26 18:14:44,696][105620] Updated weights for policy 1, policy_version 396187 (0.0009) [2023-12-26 18:14:44,762][105692] Updated weights for policy 0, policy_version 395787 (0.0006) [2023-12-26 18:14:44,818][105692] Updated weights for policy 0, policy_version 395797 (0.0009) [2023-12-26 18:14:44,872][105692] Updated weights for policy 0, policy_version 395807 (0.0009) [2023-12-26 18:14:45,471][105620] Updated weights for policy 1, policy_version 396197 (0.0010) [2023-12-26 18:14:45,524][105620] Updated weights for policy 1, policy_version 396207 (0.0009) [2023-12-26 18:14:45,587][105620] Updated weights for policy 1, policy_version 396217 (0.0010) [2023-12-26 18:14:45,589][105692] Updated weights for policy 0, policy_version 395817 (0.0009) [2023-12-26 18:14:45,638][105692] Updated weights for policy 0, policy_version 395827 (0.0005) [2023-12-26 18:14:45,694][105692] Updated weights for policy 0, policy_version 395837 (0.0008) [2023-12-26 18:14:45,748][105692] Updated weights for policy 0, policy_version 395847 (0.0007) [2023-12-26 18:14:46,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19524.3, 300 sec: 19605.2). Total num frames: 202792960. Throughput: 0: 9764.5, 1: 9681.9. Samples: 202762036. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 18:14:46,063][104569] Avg episode reward: [(0, '8896.310'), (1, '8550.923')] [2023-12-26 18:14:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000395848_101351424.pth... [2023-12-26 18:14:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000396224_101441536.pth... [2023-12-26 18:14:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000394696_101056512.pth [2023-12-26 18:14:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000395104_101154816.pth [2023-12-26 18:14:46,355][105692] Updated weights for policy 0, policy_version 395857 (0.0008) [2023-12-26 18:14:46,387][105620] Updated weights for policy 1, policy_version 396227 (0.0009) [2023-12-26 18:14:46,400][105692] Updated weights for policy 0, policy_version 395867 (0.0006) [2023-12-26 18:14:46,439][105620] Updated weights for policy 1, policy_version 396237 (0.0005) [2023-12-26 18:14:46,458][105692] Updated weights for policy 0, policy_version 395877 (0.0005) [2023-12-26 18:14:46,494][105620] Updated weights for policy 1, policy_version 396247 (0.0005) [2023-12-26 18:14:47,115][105692] Updated weights for policy 0, policy_version 395887 (0.0005) [2023-12-26 18:14:47,170][105692] Updated weights for policy 0, policy_version 395897 (0.0005) [2023-12-26 18:14:47,218][105692] Updated weights for policy 0, policy_version 395907 (0.0006) [2023-12-26 18:14:47,231][105620] Updated weights for policy 1, policy_version 396257 (0.0006) [2023-12-26 18:14:47,280][105620] Updated weights for policy 1, policy_version 396267 (0.0006) [2023-12-26 18:14:47,331][105620] Updated weights for policy 1, policy_version 396277 (0.0005) [2023-12-26 18:14:47,390][105620] Updated weights for policy 1, policy_version 396287 (0.0005) [2023-12-26 18:14:47,810][105692] Updated weights for policy 0, policy_version 395917 (0.0009) [2023-12-26 18:14:47,865][105692] Updated weights for policy 0, policy_version 395927 (0.0010) [2023-12-26 18:14:47,919][105692] Updated weights for policy 0, policy_version 395937 (0.0010) [2023-12-26 18:14:48,020][105620] Updated weights for policy 1, policy_version 396297 (0.0010) [2023-12-26 18:14:48,085][105620] Updated weights for policy 1, policy_version 396307 (0.0010) [2023-12-26 18:14:48,150][105620] Updated weights for policy 1, policy_version 396317 (0.0010) [2023-12-26 18:14:48,635][105692] Updated weights for policy 0, policy_version 395947 (0.0009) [2023-12-26 18:14:48,701][105692] Updated weights for policy 0, policy_version 395957 (0.0006) [2023-12-26 18:14:48,757][105692] Updated weights for policy 0, policy_version 395967 (0.0010) [2023-12-26 18:14:48,882][105620] Updated weights for policy 1, policy_version 396327 (0.0011) [2023-12-26 18:14:48,942][105620] Updated weights for policy 1, policy_version 396337 (0.0009) [2023-12-26 18:14:49,006][105620] Updated weights for policy 1, policy_version 396347 (0.0011) [2023-12-26 18:14:49,447][105692] Updated weights for policy 0, policy_version 395977 (0.0009) [2023-12-26 18:14:49,503][105692] Updated weights for policy 0, policy_version 395987 (0.0005) [2023-12-26 18:14:49,563][105692] Updated weights for policy 0, policy_version 395997 (0.0005) [2023-12-26 18:14:49,624][105692] Updated weights for policy 0, policy_version 396007 (0.0008) [2023-12-26 18:14:49,737][105620] Updated weights for policy 1, policy_version 396357 (0.0010) [2023-12-26 18:14:49,798][105620] Updated weights for policy 1, policy_version 396367 (0.0010) [2023-12-26 18:14:49,866][105620] Updated weights for policy 1, policy_version 396377 (0.0009) [2023-12-26 18:14:50,329][105692] Updated weights for policy 0, policy_version 396017 (0.0009) [2023-12-26 18:14:50,393][105692] Updated weights for policy 0, policy_version 396027 (0.0010) [2023-12-26 18:14:50,448][105692] Updated weights for policy 0, policy_version 396037 (0.0006) [2023-12-26 18:14:50,529][105620] Updated weights for policy 1, policy_version 396387 (0.0010) [2023-12-26 18:14:50,586][105620] Updated weights for policy 1, policy_version 396397 (0.0010) [2023-12-26 18:14:50,653][105620] Updated weights for policy 1, policy_version 396407 (0.0008) [2023-12-26 18:14:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 202891264. Throughput: 0: 9836.8, 1: 9668.6. Samples: 202881784. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 18:14:51,063][104569] Avg episode reward: [(0, '9173.597'), (1, '8905.798')] [2023-12-26 18:14:51,275][105692] Updated weights for policy 0, policy_version 396047 (0.0006) [2023-12-26 18:14:51,317][105620] Updated weights for policy 1, policy_version 396417 (0.0008) [2023-12-26 18:14:51,340][105692] Updated weights for policy 0, policy_version 396057 (0.0007) [2023-12-26 18:14:51,382][105620] Updated weights for policy 1, policy_version 396427 (0.0010) [2023-12-26 18:14:51,405][105692] Updated weights for policy 0, policy_version 396067 (0.0007) [2023-12-26 18:14:51,436][105620] Updated weights for policy 1, policy_version 396437 (0.0011) [2023-12-26 18:14:51,485][105620] Updated weights for policy 1, policy_version 396447 (0.0011) [2023-12-26 18:14:52,171][105692] Updated weights for policy 0, policy_version 396077 (0.0007) [2023-12-26 18:14:52,185][105620] Updated weights for policy 1, policy_version 396457 (0.0008) [2023-12-26 18:14:52,227][105692] Updated weights for policy 0, policy_version 396087 (0.0008) [2023-12-26 18:14:52,230][105620] Updated weights for policy 1, policy_version 396467 (0.0006) [2023-12-26 18:14:52,286][105620] Updated weights for policy 1, policy_version 396477 (0.0008) [2023-12-26 18:14:52,288][105692] Updated weights for policy 0, policy_version 396097 (0.0007) [2023-12-26 18:14:52,940][105620] Updated weights for policy 1, policy_version 396487 (0.0006) [2023-12-26 18:14:52,991][105620] Updated weights for policy 1, policy_version 396497 (0.0009) [2023-12-26 18:14:53,039][105620] Updated weights for policy 1, policy_version 396507 (0.0010) [2023-12-26 18:14:53,088][105692] Updated weights for policy 0, policy_version 396107 (0.0010) [2023-12-26 18:14:53,148][105692] Updated weights for policy 0, policy_version 396117 (0.0011) [2023-12-26 18:14:53,206][105692] Updated weights for policy 0, policy_version 396127 (0.0010) [2023-12-26 18:14:53,746][105620] Updated weights for policy 1, policy_version 396517 (0.0009) [2023-12-26 18:14:53,810][105620] Updated weights for policy 1, policy_version 396527 (0.0005) [2023-12-26 18:14:53,882][105620] Updated weights for policy 1, policy_version 396537 (0.0008) [2023-12-26 18:14:53,958][105692] Updated weights for policy 0, policy_version 396137 (0.0010) [2023-12-26 18:14:54,014][105692] Updated weights for policy 0, policy_version 396147 (0.0011) [2023-12-26 18:14:54,059][105692] Updated weights for policy 0, policy_version 396157 (0.0010) [2023-12-26 18:14:54,108][105692] Updated weights for policy 0, policy_version 396167 (0.0010) [2023-12-26 18:14:54,541][105620] Updated weights for policy 1, policy_version 396547 (0.0008) [2023-12-26 18:14:54,594][105620] Updated weights for policy 1, policy_version 396557 (0.0010) [2023-12-26 18:14:54,642][105620] Updated weights for policy 1, policy_version 396567 (0.0008) [2023-12-26 18:14:54,757][105692] Updated weights for policy 0, policy_version 396177 (0.0006) [2023-12-26 18:14:54,828][105692] Updated weights for policy 0, policy_version 396187 (0.0006) [2023-12-26 18:14:54,897][105692] Updated weights for policy 0, policy_version 396197 (0.0009) [2023-12-26 18:14:55,277][105620] Updated weights for policy 1, policy_version 396577 (0.0005) [2023-12-26 18:14:55,333][105620] Updated weights for policy 1, policy_version 396587 (0.0005) [2023-12-26 18:14:55,388][105620] Updated weights for policy 1, policy_version 396597 (0.0007) [2023-12-26 18:14:55,436][105620] Updated weights for policy 1, policy_version 396607 (0.0010) [2023-12-26 18:14:55,510][105692] Updated weights for policy 0, policy_version 396207 (0.0007) [2023-12-26 18:14:55,560][105692] Updated weights for policy 0, policy_version 396217 (0.0006) [2023-12-26 18:14:55,603][105692] Updated weights for policy 0, policy_version 396227 (0.0010) [2023-12-26 18:14:56,035][105620] Updated weights for policy 1, policy_version 396617 (0.0006) [2023-12-26 18:14:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 202989568. Throughput: 0: 9947.5, 1: 9695.9. Samples: 203001804. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 18:14:56,063][104569] Avg episode reward: [(0, '9173.060'), (1, '8718.371')] [2023-12-26 18:14:56,089][105620] Updated weights for policy 1, policy_version 396627 (0.0005) [2023-12-26 18:14:56,153][105620] Updated weights for policy 1, policy_version 396637 (0.0005) [2023-12-26 18:14:56,393][105692] Updated weights for policy 0, policy_version 396237 (0.0010) [2023-12-26 18:14:56,441][105585] KL-divergence is very high: 101.5224 [2023-12-26 18:14:56,453][105692] Updated weights for policy 0, policy_version 396247 (0.0010) [2023-12-26 18:14:56,515][105692] Updated weights for policy 0, policy_version 396257 (0.0010) [2023-12-26 18:14:56,766][105620] Updated weights for policy 1, policy_version 396647 (0.0009) [2023-12-26 18:14:56,833][105620] Updated weights for policy 1, policy_version 396657 (0.0005) [2023-12-26 18:14:56,893][105620] Updated weights for policy 1, policy_version 396667 (0.0005) [2023-12-26 18:14:57,186][105692] Updated weights for policy 0, policy_version 396267 (0.0009) [2023-12-26 18:14:57,241][105692] Updated weights for policy 0, policy_version 396277 (0.0009) [2023-12-26 18:14:57,299][105692] Updated weights for policy 0, policy_version 396287 (0.0009) [2023-12-26 18:14:57,466][105620] Updated weights for policy 1, policy_version 396677 (0.0007) [2023-12-26 18:14:57,521][105620] Updated weights for policy 1, policy_version 396688 (0.0009) [2023-12-26 18:14:57,572][105620] Updated weights for policy 1, policy_version 396698 (0.0009) [2023-12-26 18:14:57,920][105692] Updated weights for policy 0, policy_version 396297 (0.0008) [2023-12-26 18:14:57,970][105692] Updated weights for policy 0, policy_version 396307 (0.0006) [2023-12-26 18:14:58,027][105692] Updated weights for policy 0, policy_version 396317 (0.0009) [2023-12-26 18:14:58,086][105692] Updated weights for policy 0, policy_version 396327 (0.0009) [2023-12-26 18:14:58,427][105620] Updated weights for policy 1, policy_version 396708 (0.0009) [2023-12-26 18:14:58,489][105620] Updated weights for policy 1, policy_version 396718 (0.0007) [2023-12-26 18:14:58,556][105620] Updated weights for policy 1, policy_version 396728 (0.0008) [2023-12-26 18:14:58,855][105692] Updated weights for policy 0, policy_version 396337 (0.0009) [2023-12-26 18:14:58,918][105692] Updated weights for policy 0, policy_version 396347 (0.0009) [2023-12-26 18:14:58,972][105692] Updated weights for policy 0, policy_version 396357 (0.0008) [2023-12-26 18:14:59,398][105620] Updated weights for policy 1, policy_version 396738 (0.0008) [2023-12-26 18:14:59,458][105620] Updated weights for policy 1, policy_version 396748 (0.0005) [2023-12-26 18:14:59,516][105620] Updated weights for policy 1, policy_version 396758 (0.0005) [2023-12-26 18:14:59,571][105620] Updated weights for policy 1, policy_version 396768 (0.0007) [2023-12-26 18:14:59,770][105692] Updated weights for policy 0, policy_version 396367 (0.0010) [2023-12-26 18:14:59,829][105692] Updated weights for policy 0, policy_version 396377 (0.0009) [2023-12-26 18:14:59,891][105692] Updated weights for policy 0, policy_version 396387 (0.0009) [2023-12-26 18:15:00,260][105620] Updated weights for policy 1, policy_version 396778 (0.0008) [2023-12-26 18:15:00,265][105586] KL-divergence is very high: 131.4950 [2023-12-26 18:15:00,295][105586] KL-divergence is very high: 112.0273 [2023-12-26 18:15:00,309][105586] KL-divergence is very high: 138.3380 [2023-12-26 18:15:00,318][105620] Updated weights for policy 1, policy_version 396788 (0.0009) [2023-12-26 18:15:00,373][105620] Updated weights for policy 1, policy_version 396798 (0.0009) [2023-12-26 18:15:00,645][105692] Updated weights for policy 0, policy_version 396397 (0.0009) [2023-12-26 18:15:00,704][105692] Updated weights for policy 0, policy_version 396407 (0.0009) [2023-12-26 18:15:00,765][105692] Updated weights for policy 0, policy_version 396418 (0.0009) [2023-12-26 18:15:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 203087872. Throughput: 0: 9971.5, 1: 9658.6. Samples: 203061104. Policy #0 lag: (min: 6.0, avg: 8.3, max: 38.0) [2023-12-26 18:15:01,062][104569] Avg episode reward: [(0, '8618.591'), (1, '8277.807')] [2023-12-26 18:15:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000396424_101498880.pth... [2023-12-26 18:15:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000395272_101203968.pth [2023-12-26 18:15:01,099][105620] Updated weights for policy 1, policy_version 396808 (0.0007) [2023-12-26 18:15:01,158][105620] Updated weights for policy 1, policy_version 396818 (0.0006) [2023-12-26 18:15:01,217][105620] Updated weights for policy 1, policy_version 396828 (0.0006) [2023-12-26 18:15:01,246][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000396832_101597184.pth... [2023-12-26 18:15:01,251][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000395680_101302272.pth [2023-12-26 18:15:01,547][105692] Updated weights for policy 0, policy_version 396428 (0.0007) [2023-12-26 18:15:01,613][105692] Updated weights for policy 0, policy_version 396438 (0.0008) [2023-12-26 18:15:01,668][105692] Updated weights for policy 0, policy_version 396448 (0.0008) [2023-12-26 18:15:01,840][105620] Updated weights for policy 1, policy_version 396838 (0.0009) [2023-12-26 18:15:01,891][105620] Updated weights for policy 1, policy_version 396848 (0.0010) [2023-12-26 18:15:01,947][105620] Updated weights for policy 1, policy_version 396858 (0.0010) [2023-12-26 18:15:02,290][105692] Updated weights for policy 0, policy_version 396458 (0.0009) [2023-12-26 18:15:02,356][105692] Updated weights for policy 0, policy_version 396468 (0.0009) [2023-12-26 18:15:02,418][105692] Updated weights for policy 0, policy_version 396478 (0.0008) [2023-12-26 18:15:02,470][105692] Updated weights for policy 0, policy_version 396488 (0.0011) [2023-12-26 18:15:02,678][105620] Updated weights for policy 1, policy_version 396868 (0.0010) [2023-12-26 18:15:02,735][105620] Updated weights for policy 1, policy_version 396878 (0.0010) [2023-12-26 18:15:02,797][105620] Updated weights for policy 1, policy_version 396888 (0.0010) [2023-12-26 18:15:03,145][105692] Updated weights for policy 0, policy_version 396498 (0.0006) [2023-12-26 18:15:03,195][105692] Updated weights for policy 0, policy_version 396508 (0.0005) [2023-12-26 18:15:03,223][105585] KL-divergence is very high: 185.8114 [2023-12-26 18:15:03,250][105692] Updated weights for policy 0, policy_version 396518 (0.0005) [2023-12-26 18:15:03,497][105620] Updated weights for policy 1, policy_version 396898 (0.0010) [2023-12-26 18:15:03,547][105620] Updated weights for policy 1, policy_version 396908 (0.0010) [2023-12-26 18:15:03,604][105620] Updated weights for policy 1, policy_version 396918 (0.0010) [2023-12-26 18:15:03,665][105620] Updated weights for policy 1, policy_version 396928 (0.0010) [2023-12-26 18:15:03,888][105692] Updated weights for policy 0, policy_version 396528 (0.0007) [2023-12-26 18:15:03,899][105585] KL-divergence is very high: 459.5460 [2023-12-26 18:15:03,923][105585] KL-divergence is very high: 170.7526 [2023-12-26 18:15:03,944][105585] KL-divergence is very high: 704.9673 [2023-12-26 18:15:03,945][105692] Updated weights for policy 0, policy_version 396538 (0.0005) [2023-12-26 18:15:03,964][105585] KL-divergence is very high: 181.7241 [2023-12-26 18:15:03,985][105585] KL-divergence is very high: 589.3709 [2023-12-26 18:15:03,998][105692] Updated weights for policy 0, policy_version 396548 (0.0006) [2023-12-26 18:15:04,014][105585] KL-divergence is very high: 136.5734 [2023-12-26 18:15:04,385][105620] Updated weights for policy 1, policy_version 396938 (0.0007) [2023-12-26 18:15:04,452][105620] Updated weights for policy 1, policy_version 396948 (0.0008) [2023-12-26 18:15:04,519][105620] Updated weights for policy 1, policy_version 396958 (0.0008) [2023-12-26 18:15:04,692][105692] Updated weights for policy 0, policy_version 396558 (0.0008) [2023-12-26 18:15:04,754][105692] Updated weights for policy 0, policy_version 396568 (0.0009) [2023-12-26 18:15:04,816][105692] Updated weights for policy 0, policy_version 396578 (0.0008) [2023-12-26 18:15:05,134][105620] Updated weights for policy 1, policy_version 396968 (0.0008) [2023-12-26 18:15:05,183][105620] Updated weights for policy 1, policy_version 396978 (0.0009) [2023-12-26 18:15:05,241][105620] Updated weights for policy 1, policy_version 396988 (0.0005) [2023-12-26 18:15:05,612][105692] Updated weights for policy 0, policy_version 396588 (0.0009) [2023-12-26 18:15:05,677][105692] Updated weights for policy 0, policy_version 396598 (0.0009) [2023-12-26 18:15:05,726][105692] Updated weights for policy 0, policy_version 396608 (0.0008) [2023-12-26 18:15:05,835][105620] Updated weights for policy 1, policy_version 396998 (0.0007) [2023-12-26 18:15:05,889][105620] Updated weights for policy 1, policy_version 397008 (0.0009) [2023-12-26 18:15:05,960][105620] Updated weights for policy 1, policy_version 397018 (0.0009) [2023-12-26 18:15:06,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.7, 300 sec: 19633.0). Total num frames: 203194368. Throughput: 0: 9967.0, 1: 9737.7. Samples: 203178872. Policy #0 lag: (min: 6.0, avg: 8.3, max: 38.0) [2023-12-26 18:15:06,063][104569] Avg episode reward: [(0, '8524.470'), (1, '8366.855')] [2023-12-26 18:15:06,416][105692] Updated weights for policy 0, policy_version 396618 (0.0008) [2023-12-26 18:15:06,480][105692] Updated weights for policy 0, policy_version 396628 (0.0008) [2023-12-26 18:15:06,543][105692] Updated weights for policy 0, policy_version 396638 (0.0008) [2023-12-26 18:15:06,603][105692] Updated weights for policy 0, policy_version 396648 (0.0008) [2023-12-26 18:15:06,733][105620] Updated weights for policy 1, policy_version 397028 (0.0009) [2023-12-26 18:15:06,795][105620] Updated weights for policy 1, policy_version 397038 (0.0010) [2023-12-26 18:15:06,857][105620] Updated weights for policy 1, policy_version 397048 (0.0010) [2023-12-26 18:15:07,354][105692] Updated weights for policy 0, policy_version 396658 (0.0008) [2023-12-26 18:15:07,411][105692] Updated weights for policy 0, policy_version 396668 (0.0008) [2023-12-26 18:15:07,467][105692] Updated weights for policy 0, policy_version 396678 (0.0008) [2023-12-26 18:15:07,555][105620] Updated weights for policy 1, policy_version 397058 (0.0009) [2023-12-26 18:15:07,613][105620] Updated weights for policy 1, policy_version 397068 (0.0008) [2023-12-26 18:15:07,672][105620] Updated weights for policy 1, policy_version 397078 (0.0008) [2023-12-26 18:15:07,731][105620] Updated weights for policy 1, policy_version 397088 (0.0009) [2023-12-26 18:15:08,277][105692] Updated weights for policy 0, policy_version 396688 (0.0009) [2023-12-26 18:15:08,342][105692] Updated weights for policy 0, policy_version 396698 (0.0009) [2023-12-26 18:15:08,401][105620] Updated weights for policy 1, policy_version 397098 (0.0006) [2023-12-26 18:15:08,403][105692] Updated weights for policy 0, policy_version 396708 (0.0008) [2023-12-26 18:15:08,460][105620] Updated weights for policy 1, policy_version 397108 (0.0007) [2023-12-26 18:15:08,524][105620] Updated weights for policy 1, policy_version 397118 (0.0009) [2023-12-26 18:15:09,216][105692] Updated weights for policy 0, policy_version 396718 (0.0008) [2023-12-26 18:15:09,243][105620] Updated weights for policy 1, policy_version 397128 (0.0007) [2023-12-26 18:15:09,282][105692] Updated weights for policy 0, policy_version 396728 (0.0008) [2023-12-26 18:15:09,309][105620] Updated weights for policy 1, policy_version 397138 (0.0006) [2023-12-26 18:15:09,351][105692] Updated weights for policy 0, policy_version 396738 (0.0009) [2023-12-26 18:15:09,374][105585] KL-divergence is very high: 157.1460 [2023-12-26 18:15:09,377][105620] Updated weights for policy 1, policy_version 397148 (0.0007) [2023-12-26 18:15:10,089][105620] Updated weights for policy 1, policy_version 397158 (0.0007) [2023-12-26 18:15:10,138][105585] KL-divergence is very high: 164.0914 [2023-12-26 18:15:10,151][105620] Updated weights for policy 1, policy_version 397168 (0.0006) [2023-12-26 18:15:10,158][105692] Updated weights for policy 0, policy_version 396748 (0.0009) [2023-12-26 18:15:10,189][105585] KL-divergence is very high: 124.3195 [2023-12-26 18:15:10,204][105620] Updated weights for policy 1, policy_version 397178 (0.0006) [2023-12-26 18:15:10,217][105585] KL-divergence is very high: 120.0703 [2023-12-26 18:15:10,223][105692] Updated weights for policy 0, policy_version 396758 (0.0008) [2023-12-26 18:15:10,244][105585] KL-divergence is very high: 117.8663 [2023-12-26 18:15:10,258][105585] KL-divergence is very high: 119.5610 [2023-12-26 18:15:10,290][105692] Updated weights for policy 0, policy_version 396768 (0.0008) [2023-12-26 18:15:10,299][105585] KL-divergence is very high: 107.8768 [2023-12-26 18:15:10,316][105585] KL-divergence is very high: 105.4072 [2023-12-26 18:15:10,859][105620] Updated weights for policy 1, policy_version 397188 (0.0009) [2023-12-26 18:15:10,912][105620] Updated weights for policy 1, policy_version 397198 (0.0007) [2023-12-26 18:15:10,972][105620] Updated weights for policy 1, policy_version 397208 (0.0009) [2023-12-26 18:15:11,046][105692] Updated weights for policy 0, policy_version 396778 (0.0008) [2023-12-26 18:15:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 203284480. Throughput: 0: 9844.5, 1: 9822.9. Samples: 203292072. Policy #0 lag: (min: 6.0, avg: 8.3, max: 38.0) [2023-12-26 18:15:11,062][104569] Avg episode reward: [(0, '8617.277'), (1, '8639.646')] [2023-12-26 18:15:11,103][105692] Updated weights for policy 0, policy_version 396788 (0.0009) [2023-12-26 18:15:11,168][105692] Updated weights for policy 0, policy_version 396798 (0.0008) [2023-12-26 18:15:11,217][105692] Updated weights for policy 0, policy_version 396808 (0.0009) [2023-12-26 18:15:11,789][105620] Updated weights for policy 1, policy_version 397218 (0.0009) [2023-12-26 18:15:11,841][105620] Updated weights for policy 1, policy_version 397228 (0.0008) [2023-12-26 18:15:11,890][105620] Updated weights for policy 1, policy_version 397238 (0.0009) [2023-12-26 18:15:11,957][105620] Updated weights for policy 1, policy_version 397248 (0.0009) [2023-12-26 18:15:12,034][105692] Updated weights for policy 0, policy_version 396818 (0.0006) [2023-12-26 18:15:12,094][105692] Updated weights for policy 0, policy_version 396828 (0.0010) [2023-12-26 18:15:12,146][105692] Updated weights for policy 0, policy_version 396838 (0.0010) [2023-12-26 18:15:12,784][105620] Updated weights for policy 1, policy_version 397258 (0.0008) [2023-12-26 18:15:12,804][105692] Updated weights for policy 0, policy_version 396848 (0.0009) [2023-12-26 18:15:12,854][105692] Updated weights for policy 0, policy_version 396858 (0.0005) [2023-12-26 18:15:12,855][105620] Updated weights for policy 1, policy_version 397268 (0.0005) [2023-12-26 18:15:12,911][105692] Updated weights for policy 0, policy_version 396868 (0.0005) [2023-12-26 18:15:12,926][105620] Updated weights for policy 1, policy_version 397278 (0.0007) [2023-12-26 18:15:13,481][105692] Updated weights for policy 0, policy_version 396878 (0.0005) [2023-12-26 18:15:13,527][105692] Updated weights for policy 0, policy_version 396888 (0.0007) [2023-12-26 18:15:13,570][105692] Updated weights for policy 0, policy_version 396898 (0.0006) [2023-12-26 18:15:13,615][105620] Updated weights for policy 1, policy_version 397288 (0.0009) [2023-12-26 18:15:13,678][105620] Updated weights for policy 1, policy_version 397298 (0.0010) [2023-12-26 18:15:13,738][105620] Updated weights for policy 1, policy_version 397308 (0.0008) [2023-12-26 18:15:14,113][105692] Updated weights for policy 0, policy_version 396908 (0.0005) [2023-12-26 18:15:14,140][105585] KL-divergence is very high: 217.4079 [2023-12-26 18:15:14,158][105585] KL-divergence is very high: 184.6075 [2023-12-26 18:15:14,167][105692] Updated weights for policy 0, policy_version 396918 (0.0005) [2023-12-26 18:15:14,190][105585] KL-divergence is very high: 274.1862 [2023-12-26 18:15:14,210][105585] KL-divergence is very high: 161.4509 [2023-12-26 18:15:14,234][105585] KL-divergence is very high: 135.3012 [2023-12-26 18:15:14,234][105692] Updated weights for policy 0, policy_version 396928 (0.0009) [2023-12-26 18:15:14,240][105585] KL-divergence is very high: 200.2340 [2023-12-26 18:15:14,246][105585] KL-divergence is very high: 170.4680 [2023-12-26 18:15:14,252][105585] KL-divergence is very high: 175.4370 [2023-12-26 18:15:14,259][105585] KL-divergence is very high: 131.0330 [2023-12-26 18:15:14,265][105585] KL-divergence is very high: 155.5350 [2023-12-26 18:15:14,564][105620] Updated weights for policy 1, policy_version 397318 (0.0007) [2023-12-26 18:15:14,616][105620] Updated weights for policy 1, policy_version 397328 (0.0006) [2023-12-26 18:15:14,680][105620] Updated weights for policy 1, policy_version 397338 (0.0005) [2023-12-26 18:15:14,881][105692] Updated weights for policy 0, policy_version 396938 (0.0010) [2023-12-26 18:15:14,900][105585] KL-divergence is very high: 146.5845 [2023-12-26 18:15:14,942][105692] Updated weights for policy 0, policy_version 396948 (0.0011) [2023-12-26 18:15:14,949][105585] KL-divergence is very high: 190.4769 [2023-12-26 18:15:14,998][105585] KL-divergence is very high: 138.9725 [2023-12-26 18:15:15,006][105692] Updated weights for policy 0, policy_version 396958 (0.0011) [2023-12-26 18:15:15,053][105585] KL-divergence is very high: 132.0293 [2023-12-26 18:15:15,071][105692] Updated weights for policy 0, policy_version 396968 (0.0008) [2023-12-26 18:15:15,366][105620] Updated weights for policy 1, policy_version 397348 (0.0007) [2023-12-26 18:15:15,429][105620] Updated weights for policy 1, policy_version 397358 (0.0008) [2023-12-26 18:15:15,492][105620] Updated weights for policy 1, policy_version 397368 (0.0008) [2023-12-26 18:15:15,732][105692] Updated weights for policy 0, policy_version 396978 (0.0010) [2023-12-26 18:15:15,788][105692] Updated weights for policy 0, policy_version 396988 (0.0011) [2023-12-26 18:15:15,839][105692] Updated weights for policy 0, policy_version 396998 (0.0010) [2023-12-26 18:15:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 203382784. Throughput: 0: 9744.5, 1: 9769.8. Samples: 203349136. Policy #0 lag: (min: 6.0, avg: 8.3, max: 38.0) [2023-12-26 18:15:16,063][104569] Avg episode reward: [(0, '7211.599'), (1, '7164.663')] [2023-12-26 18:15:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000397000_101646336.pth... [2023-12-26 18:15:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000397376_101736448.pth... [2023-12-26 18:15:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000396224_101441536.pth [2023-12-26 18:15:16,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000395848_101351424.pth [2023-12-26 18:15:16,113][105620] Updated weights for policy 1, policy_version 397378 (0.0008) [2023-12-26 18:15:16,164][105620] Updated weights for policy 1, policy_version 397388 (0.0005) [2023-12-26 18:15:16,217][105620] Updated weights for policy 1, policy_version 397398 (0.0005) [2023-12-26 18:15:16,268][105620] Updated weights for policy 1, policy_version 397408 (0.0005) [2023-12-26 18:15:16,550][105692] Updated weights for policy 0, policy_version 397008 (0.0006) [2023-12-26 18:15:16,605][105692] Updated weights for policy 0, policy_version 397018 (0.0005) [2023-12-26 18:15:16,662][105692] Updated weights for policy 0, policy_version 397028 (0.0005) [2023-12-26 18:15:16,907][105620] Updated weights for policy 1, policy_version 397418 (0.0005) [2023-12-26 18:15:16,964][105620] Updated weights for policy 1, policy_version 397429 (0.0010) [2023-12-26 18:15:17,017][105620] Updated weights for policy 1, policy_version 397440 (0.0010) [2023-12-26 18:15:17,165][105692] Updated weights for policy 0, policy_version 397038 (0.0005) [2023-12-26 18:15:17,224][105692] Updated weights for policy 0, policy_version 397048 (0.0005) [2023-12-26 18:15:17,293][105692] Updated weights for policy 0, policy_version 397058 (0.0005) [2023-12-26 18:15:17,802][105620] Updated weights for policy 1, policy_version 397450 (0.0009) [2023-12-26 18:15:17,833][105692] Updated weights for policy 0, policy_version 397068 (0.0007) [2023-12-26 18:15:17,852][105620] Updated weights for policy 1, policy_version 397460 (0.0007) [2023-12-26 18:15:17,879][105692] Updated weights for policy 0, policy_version 397078 (0.0007) [2023-12-26 18:15:17,901][105620] Updated weights for policy 1, policy_version 397470 (0.0008) [2023-12-26 18:15:17,929][105692] Updated weights for policy 0, policy_version 397088 (0.0008) [2023-12-26 18:15:18,601][105692] Updated weights for policy 0, policy_version 397098 (0.0008) [2023-12-26 18:15:18,663][105692] Updated weights for policy 0, policy_version 397108 (0.0007) [2023-12-26 18:15:18,714][105620] Updated weights for policy 1, policy_version 397480 (0.0008) [2023-12-26 18:15:18,721][105692] Updated weights for policy 0, policy_version 397118 (0.0006) [2023-12-26 18:15:18,778][105620] Updated weights for policy 1, policy_version 397490 (0.0006) [2023-12-26 18:15:18,787][105692] Updated weights for policy 0, policy_version 397128 (0.0008) [2023-12-26 18:15:18,847][105620] Updated weights for policy 1, policy_version 397500 (0.0006) [2023-12-26 18:15:19,393][105620] Updated weights for policy 1, policy_version 397510 (0.0006) [2023-12-26 18:15:19,449][105620] Updated weights for policy 1, policy_version 397520 (0.0005) [2023-12-26 18:15:19,510][105620] Updated weights for policy 1, policy_version 397530 (0.0006) [2023-12-26 18:15:19,598][105692] Updated weights for policy 0, policy_version 397138 (0.0008) [2023-12-26 18:15:19,659][105692] Updated weights for policy 0, policy_version 397148 (0.0008) [2023-12-26 18:15:19,719][105692] Updated weights for policy 0, policy_version 397158 (0.0008) [2023-12-26 18:15:20,207][105620] Updated weights for policy 1, policy_version 397540 (0.0009) [2023-12-26 18:15:20,271][105620] Updated weights for policy 1, policy_version 397550 (0.0008) [2023-12-26 18:15:20,331][105620] Updated weights for policy 1, policy_version 397560 (0.0008) [2023-12-26 18:15:20,362][105692] Updated weights for policy 0, policy_version 397168 (0.0010) [2023-12-26 18:15:20,428][105692] Updated weights for policy 0, policy_version 397178 (0.0008) [2023-12-26 18:15:20,487][105692] Updated weights for policy 0, policy_version 397188 (0.0006) [2023-12-26 18:15:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 203481088. Throughput: 0: 9958.1, 1: 9805.5. Samples: 203473840. Policy #0 lag: (min: 6.0, avg: 8.3, max: 38.0) [2023-12-26 18:15:21,063][104569] Avg episode reward: [(0, '6160.008'), (1, '6942.209')] [2023-12-26 18:15:21,181][105692] Updated weights for policy 0, policy_version 397198 (0.0008) [2023-12-26 18:15:21,201][105620] Updated weights for policy 1, policy_version 397570 (0.0009) [2023-12-26 18:15:21,241][105692] Updated weights for policy 0, policy_version 397208 (0.0008) [2023-12-26 18:15:21,263][105620] Updated weights for policy 1, policy_version 397580 (0.0007) [2023-12-26 18:15:21,299][105692] Updated weights for policy 0, policy_version 397218 (0.0010) [2023-12-26 18:15:21,318][105620] Updated weights for policy 1, policy_version 397590 (0.0007) [2023-12-26 18:15:21,384][105620] Updated weights for policy 1, policy_version 397600 (0.0007) [2023-12-26 18:15:22,054][105692] Updated weights for policy 0, policy_version 397228 (0.0008) [2023-12-26 18:15:22,081][105620] Updated weights for policy 1, policy_version 397610 (0.0008) [2023-12-26 18:15:22,116][105692] Updated weights for policy 0, policy_version 397238 (0.0006) [2023-12-26 18:15:22,136][105620] Updated weights for policy 1, policy_version 397620 (0.0008) [2023-12-26 18:15:22,171][105692] Updated weights for policy 0, policy_version 397248 (0.0008) [2023-12-26 18:15:22,201][105620] Updated weights for policy 1, policy_version 397630 (0.0008) [2023-12-26 18:15:22,972][105692] Updated weights for policy 0, policy_version 397258 (0.0007) [2023-12-26 18:15:22,995][105620] Updated weights for policy 1, policy_version 397640 (0.0008) [2023-12-26 18:15:23,034][105692] Updated weights for policy 0, policy_version 397268 (0.0008) [2023-12-26 18:15:23,053][105620] Updated weights for policy 1, policy_version 397650 (0.0007) [2023-12-26 18:15:23,091][105692] Updated weights for policy 0, policy_version 397278 (0.0006) [2023-12-26 18:15:23,110][105620] Updated weights for policy 1, policy_version 397660 (0.0006) [2023-12-26 18:15:23,150][105692] Updated weights for policy 0, policy_version 397288 (0.0008) [2023-12-26 18:15:23,841][105692] Updated weights for policy 0, policy_version 397298 (0.0010) [2023-12-26 18:15:23,848][105585] KL-divergence is very high: 158.5526 [2023-12-26 18:15:23,881][105620] Updated weights for policy 1, policy_version 397670 (0.0008) [2023-12-26 18:15:23,886][105692] Updated weights for policy 0, policy_version 397308 (0.0010) [2023-12-26 18:15:23,886][105585] KL-divergence is very high: 253.2834 [2023-12-26 18:15:23,925][105585] KL-divergence is very high: 243.1855 [2023-12-26 18:15:23,934][105692] Updated weights for policy 0, policy_version 397318 (0.0010) [2023-12-26 18:15:23,942][105620] Updated weights for policy 1, policy_version 397680 (0.0008) [2023-12-26 18:15:24,004][105620] Updated weights for policy 1, policy_version 397690 (0.0008) [2023-12-26 18:15:24,687][105585] KL-divergence is very high: 296.4571 [2023-12-26 18:15:24,703][105692] Updated weights for policy 0, policy_version 397328 (0.0010) [2023-12-26 18:15:24,731][105585] KL-divergence is very high: 284.3896 [2023-12-26 18:15:24,755][105692] Updated weights for policy 0, policy_version 397338 (0.0010) [2023-12-26 18:15:24,769][105585] KL-divergence is very high: 201.6242 [2023-12-26 18:15:24,769][105620] Updated weights for policy 1, policy_version 397700 (0.0009) [2023-12-26 18:15:24,800][105692] Updated weights for policy 0, policy_version 397348 (0.0010) [2023-12-26 18:15:24,805][105585] KL-divergence is very high: 168.5152 [2023-12-26 18:15:24,827][105620] Updated weights for policy 1, policy_version 397710 (0.0010) [2023-12-26 18:15:24,884][105620] Updated weights for policy 1, policy_version 397720 (0.0010) [2023-12-26 18:15:25,418][105692] Updated weights for policy 0, policy_version 397358 (0.0010) [2023-12-26 18:15:25,472][105692] Updated weights for policy 0, policy_version 397368 (0.0010) [2023-12-26 18:15:25,498][105585] KL-divergence is very high: 161.2562 [2023-12-26 18:15:25,531][105692] Updated weights for policy 0, policy_version 397378 (0.0010) [2023-12-26 18:15:25,542][105585] KL-divergence is very high: 120.3899 [2023-12-26 18:15:25,609][105620] Updated weights for policy 1, policy_version 397730 (0.0006) [2023-12-26 18:15:25,668][105620] Updated weights for policy 1, policy_version 397740 (0.0006) [2023-12-26 18:15:25,723][105620] Updated weights for policy 1, policy_version 397750 (0.0006) [2023-12-26 18:15:25,781][105620] Updated weights for policy 1, policy_version 397760 (0.0006) [2023-12-26 18:15:26,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 203579392. Throughput: 0: 9902.1, 1: 9715.8. Samples: 203588180. Policy #0 lag: (min: 6.0, avg: 8.3, max: 38.0) [2023-12-26 18:15:26,063][104569] Avg episode reward: [(0, '7385.626'), (1, '8448.451')] [2023-12-26 18:15:26,285][105692] Updated weights for policy 0, policy_version 397388 (0.0010) [2023-12-26 18:15:26,351][105692] Updated weights for policy 0, policy_version 397398 (0.0010) [2023-12-26 18:15:26,405][105620] Updated weights for policy 1, policy_version 397770 (0.0007) [2023-12-26 18:15:26,409][105692] Updated weights for policy 0, policy_version 397408 (0.0010) [2023-12-26 18:15:26,450][105620] Updated weights for policy 1, policy_version 397780 (0.0007) [2023-12-26 18:15:26,496][105620] Updated weights for policy 1, policy_version 397790 (0.0007) [2023-12-26 18:15:27,039][105692] Updated weights for policy 0, policy_version 397418 (0.0009) [2023-12-26 18:15:27,101][105692] Updated weights for policy 0, policy_version 397428 (0.0005) [2023-12-26 18:15:27,146][105620] Updated weights for policy 1, policy_version 397800 (0.0009) [2023-12-26 18:15:27,169][105692] Updated weights for policy 0, policy_version 397438 (0.0005) [2023-12-26 18:15:27,194][105620] Updated weights for policy 1, policy_version 397810 (0.0009) [2023-12-26 18:15:27,225][105692] Updated weights for policy 0, policy_version 397448 (0.0005) [2023-12-26 18:15:27,244][105620] Updated weights for policy 1, policy_version 397820 (0.0009) [2023-12-26 18:15:27,720][105692] Updated weights for policy 0, policy_version 397458 (0.0005) [2023-12-26 18:15:27,768][105692] Updated weights for policy 0, policy_version 397468 (0.0005) [2023-12-26 18:15:27,813][105692] Updated weights for policy 0, policy_version 397478 (0.0005) [2023-12-26 18:15:28,162][105620] Updated weights for policy 1, policy_version 397830 (0.0008) [2023-12-26 18:15:28,211][105620] Updated weights for policy 1, policy_version 397840 (0.0008) [2023-12-26 18:15:28,272][105620] Updated weights for policy 1, policy_version 397850 (0.0009) [2023-12-26 18:15:28,372][105692] Updated weights for policy 0, policy_version 397488 (0.0008) [2023-12-26 18:15:28,429][105692] Updated weights for policy 0, policy_version 397498 (0.0010) [2023-12-26 18:15:28,487][105692] Updated weights for policy 0, policy_version 397508 (0.0010) [2023-12-26 18:15:29,040][105620] Updated weights for policy 1, policy_version 397860 (0.0008) [2023-12-26 18:15:29,054][105692] Updated weights for policy 0, policy_version 397518 (0.0007) [2023-12-26 18:15:29,087][105620] Updated weights for policy 1, policy_version 397870 (0.0009) [2023-12-26 18:15:29,098][105692] Updated weights for policy 0, policy_version 397528 (0.0005) [2023-12-26 18:15:29,139][105620] Updated weights for policy 1, policy_version 397880 (0.0008) [2023-12-26 18:15:29,157][105692] Updated weights for policy 0, policy_version 397538 (0.0005) [2023-12-26 18:15:29,753][105692] Updated weights for policy 0, policy_version 397548 (0.0006) [2023-12-26 18:15:29,816][105692] Updated weights for policy 0, policy_version 397558 (0.0006) [2023-12-26 18:15:29,822][105620] Updated weights for policy 1, policy_version 397890 (0.0008) [2023-12-26 18:15:29,883][105692] Updated weights for policy 0, policy_version 397568 (0.0008) [2023-12-26 18:15:29,883][105620] Updated weights for policy 1, policy_version 397900 (0.0007) [2023-12-26 18:15:29,949][105620] Updated weights for policy 1, policy_version 397910 (0.0007) [2023-12-26 18:15:30,019][105620] Updated weights for policy 1, policy_version 397920 (0.0006) [2023-12-26 18:15:30,515][105692] Updated weights for policy 0, policy_version 397578 (0.0009) [2023-12-26 18:15:30,561][105692] Updated weights for policy 0, policy_version 397588 (0.0008) [2023-12-26 18:15:30,615][105692] Updated weights for policy 0, policy_version 397598 (0.0010) [2023-12-26 18:15:30,672][105692] Updated weights for policy 0, policy_version 397608 (0.0008) [2023-12-26 18:15:30,675][105620] Updated weights for policy 1, policy_version 397930 (0.0006) [2023-12-26 18:15:30,727][105620] Updated weights for policy 1, policy_version 397940 (0.0008) [2023-12-26 18:15:30,783][105620] Updated weights for policy 1, policy_version 397950 (0.0009) [2023-12-26 18:15:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 203685888. Throughput: 0: 9987.0, 1: 9742.2. Samples: 203649844. Policy #0 lag: (min: 6.0, avg: 8.3, max: 38.0) [2023-12-26 18:15:31,062][104569] Avg episode reward: [(0, '8621.460'), (1, '7892.381')] [2023-12-26 18:15:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000397608_101801984.pth... [2023-12-26 18:15:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000397952_101883904.pth... [2023-12-26 18:15:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000396424_101498880.pth [2023-12-26 18:15:31,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000396832_101597184.pth [2023-12-26 18:15:31,434][105692] Updated weights for policy 0, policy_version 397618 (0.0011) [2023-12-26 18:15:31,482][105692] Updated weights for policy 0, policy_version 397628 (0.0010) [2023-12-26 18:15:31,544][105692] Updated weights for policy 0, policy_version 397638 (0.0009) [2023-12-26 18:15:31,580][105620] Updated weights for policy 1, policy_version 397960 (0.0008) [2023-12-26 18:15:31,643][105620] Updated weights for policy 1, policy_version 397970 (0.0008) [2023-12-26 18:15:31,706][105620] Updated weights for policy 1, policy_version 397980 (0.0008) [2023-12-26 18:15:32,357][105620] Updated weights for policy 1, policy_version 397990 (0.0008) [2023-12-26 18:15:32,391][105692] Updated weights for policy 0, policy_version 397648 (0.0009) [2023-12-26 18:15:32,416][105620] Updated weights for policy 1, policy_version 398000 (0.0007) [2023-12-26 18:15:32,446][105692] Updated weights for policy 0, policy_version 397658 (0.0006) [2023-12-26 18:15:32,476][105620] Updated weights for policy 1, policy_version 398010 (0.0008) [2023-12-26 18:15:32,502][105692] Updated weights for policy 0, policy_version 397668 (0.0007) [2023-12-26 18:15:33,175][105620] Updated weights for policy 1, policy_version 398020 (0.0009) [2023-12-26 18:15:33,215][105692] Updated weights for policy 0, policy_version 397678 (0.0007) [2023-12-26 18:15:33,234][105620] Updated weights for policy 1, policy_version 398030 (0.0008) [2023-12-26 18:15:33,266][105692] Updated weights for policy 0, policy_version 397688 (0.0006) [2023-12-26 18:15:33,286][105620] Updated weights for policy 1, policy_version 398040 (0.0006) [2023-12-26 18:15:33,319][105692] Updated weights for policy 0, policy_version 397698 (0.0005) [2023-12-26 18:15:33,858][105692] Updated weights for policy 0, policy_version 397708 (0.0009) [2023-12-26 18:15:33,902][105692] Updated weights for policy 0, policy_version 397718 (0.0010) [2023-12-26 18:15:33,950][105692] Updated weights for policy 0, policy_version 397728 (0.0010) [2023-12-26 18:15:34,100][105620] Updated weights for policy 1, policy_version 398051 (0.0009) [2023-12-26 18:15:34,162][105620] Updated weights for policy 1, policy_version 398061 (0.0011) [2023-12-26 18:15:34,221][105620] Updated weights for policy 1, policy_version 398071 (0.0011) [2023-12-26 18:15:34,737][105692] Updated weights for policy 0, policy_version 397738 (0.0010) [2023-12-26 18:15:34,788][105692] Updated weights for policy 0, policy_version 397748 (0.0010) [2023-12-26 18:15:34,798][105585] KL-divergence is very high: 141.6958 [2023-12-26 18:15:34,815][105585] KL-divergence is very high: 131.7884 [2023-12-26 18:15:34,843][105692] Updated weights for policy 0, policy_version 397758 (0.0010) [2023-12-26 18:15:34,857][105585] KL-divergence is very high: 112.4461 [2023-12-26 18:15:34,894][105692] Updated weights for policy 0, policy_version 397768 (0.0010) [2023-12-26 18:15:34,976][105620] Updated weights for policy 1, policy_version 398081 (0.0011) [2023-12-26 18:15:35,050][105620] Updated weights for policy 1, policy_version 398091 (0.0009) [2023-12-26 18:15:35,110][105620] Updated weights for policy 1, policy_version 398101 (0.0007) [2023-12-26 18:15:35,171][105620] Updated weights for policy 1, policy_version 398111 (0.0005) [2023-12-26 18:15:35,576][105692] Updated weights for policy 0, policy_version 397778 (0.0005) [2023-12-26 18:15:35,640][105692] Updated weights for policy 0, policy_version 397788 (0.0006) [2023-12-26 18:15:35,693][105692] Updated weights for policy 0, policy_version 397798 (0.0010) [2023-12-26 18:15:35,702][105620] Updated weights for policy 1, policy_version 398121 (0.0007) [2023-12-26 18:15:35,752][105620] Updated weights for policy 1, policy_version 398131 (0.0008) [2023-12-26 18:15:35,801][105620] Updated weights for policy 1, policy_version 398141 (0.0008) [2023-12-26 18:15:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 203784192. Throughput: 0: 9941.2, 1: 9784.2. Samples: 203769424. Policy #0 lag: (min: 6.0, avg: 8.3, max: 38.0) [2023-12-26 18:15:36,063][104569] Avg episode reward: [(0, '8529.954'), (1, '7695.848')] [2023-12-26 18:15:36,470][105692] Updated weights for policy 0, policy_version 397808 (0.0011) [2023-12-26 18:15:36,537][105692] Updated weights for policy 0, policy_version 397818 (0.0011) [2023-12-26 18:15:36,601][105692] Updated weights for policy 0, policy_version 397828 (0.0011) [2023-12-26 18:15:36,636][105620] Updated weights for policy 1, policy_version 398151 (0.0008) [2023-12-26 18:15:36,697][105620] Updated weights for policy 1, policy_version 398161 (0.0008) [2023-12-26 18:15:36,761][105620] Updated weights for policy 1, policy_version 398171 (0.0008) [2023-12-26 18:15:37,339][105692] Updated weights for policy 0, policy_version 397838 (0.0011) [2023-12-26 18:15:37,353][105585] KL-divergence is very high: 151.8036 [2023-12-26 18:15:37,384][105585] KL-divergence is very high: 611.4691 [2023-12-26 18:15:37,396][105585] KL-divergence is very high: 767.6559 [2023-12-26 18:15:37,403][105585] KL-divergence is very high: 846.6808 [2023-12-26 18:15:37,404][105692] Updated weights for policy 0, policy_version 397848 (0.0011) [2023-12-26 18:15:37,409][105585] KL-divergence is very high: 835.4164 [2023-12-26 18:15:37,434][105585] KL-divergence is very high: 561.2228 [2023-12-26 18:15:37,446][105585] KL-divergence is very high: 493.3547 [2023-12-26 18:15:37,453][105585] KL-divergence is very high: 319.8107 [2023-12-26 18:15:37,460][105585] KL-divergence is very high: 231.0673 [2023-12-26 18:15:37,466][105692] Updated weights for policy 0, policy_version 397858 (0.0010) [2023-12-26 18:15:37,501][105585] KL-divergence is very high: 354.8515 [2023-12-26 18:15:37,521][105620] Updated weights for policy 1, policy_version 398181 (0.0008) [2023-12-26 18:15:37,579][105620] Updated weights for policy 1, policy_version 398191 (0.0009) [2023-12-26 18:15:37,641][105620] Updated weights for policy 1, policy_version 398201 (0.0009) [2023-12-26 18:15:38,162][105692] Updated weights for policy 0, policy_version 397868 (0.0009) [2023-12-26 18:15:38,182][105585] KL-divergence is very high: 135.2998 [2023-12-26 18:15:38,226][105692] Updated weights for policy 0, policy_version 397878 (0.0009) [2023-12-26 18:15:38,232][105585] KL-divergence is very high: 132.0624 [2023-12-26 18:15:38,289][105692] Updated weights for policy 0, policy_version 397888 (0.0009) [2023-12-26 18:15:38,333][105585] KL-divergence is very high: 109.3371 [2023-12-26 18:15:38,450][105620] Updated weights for policy 1, policy_version 398211 (0.0008) [2023-12-26 18:15:38,512][105620] Updated weights for policy 1, policy_version 398221 (0.0009) [2023-12-26 18:15:38,570][105620] Updated weights for policy 1, policy_version 398231 (0.0008) [2023-12-26 18:15:39,037][105692] Updated weights for policy 0, policy_version 397898 (0.0009) [2023-12-26 18:15:39,095][105692] Updated weights for policy 0, policy_version 397908 (0.0009) [2023-12-26 18:15:39,154][105692] Updated weights for policy 0, policy_version 397918 (0.0009) [2023-12-26 18:15:39,209][105692] Updated weights for policy 0, policy_version 397928 (0.0009) [2023-12-26 18:15:39,335][105620] Updated weights for policy 1, policy_version 398241 (0.0009) [2023-12-26 18:15:39,410][105620] Updated weights for policy 1, policy_version 398251 (0.0009) [2023-12-26 18:15:39,475][105620] Updated weights for policy 1, policy_version 398261 (0.0008) [2023-12-26 18:15:39,542][105620] Updated weights for policy 1, policy_version 398271 (0.0008) [2023-12-26 18:15:40,039][105692] Updated weights for policy 0, policy_version 397938 (0.0009) [2023-12-26 18:15:40,097][105692] Updated weights for policy 0, policy_version 397948 (0.0009) [2023-12-26 18:15:40,156][105692] Updated weights for policy 0, policy_version 397958 (0.0009) [2023-12-26 18:15:40,265][105620] Updated weights for policy 1, policy_version 398281 (0.0009) [2023-12-26 18:15:40,320][105620] Updated weights for policy 1, policy_version 398291 (0.0009) [2023-12-26 18:15:40,382][105620] Updated weights for policy 1, policy_version 398301 (0.0008) [2023-12-26 18:15:40,949][105692] Updated weights for policy 0, policy_version 397968 (0.0009) [2023-12-26 18:15:41,003][105692] Updated weights for policy 0, policy_version 397978 (0.0005) [2023-12-26 18:15:41,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 203866112. Throughput: 0: 9920.3, 1: 9624.0. Samples: 203881296. Policy #0 lag: (min: 6.0, avg: 8.3, max: 38.0) [2023-12-26 18:15:41,063][104569] Avg episode reward: [(0, '7884.987'), (1, '7928.525')] [2023-12-26 18:15:41,068][105692] Updated weights for policy 0, policy_version 397988 (0.0010) [2023-12-26 18:15:41,176][105620] Updated weights for policy 1, policy_version 398311 (0.0009) [2023-12-26 18:15:41,237][105620] Updated weights for policy 1, policy_version 398321 (0.0009) [2023-12-26 18:15:41,302][105620] Updated weights for policy 1, policy_version 398331 (0.0007) [2023-12-26 18:15:41,903][105692] Updated weights for policy 0, policy_version 397998 (0.0009) [2023-12-26 18:15:41,960][105585] KL-divergence is very high: 123.9924 [2023-12-26 18:15:41,973][105692] Updated weights for policy 0, policy_version 398008 (0.0008) [2023-12-26 18:15:42,013][105585] KL-divergence is very high: 157.6855 [2023-12-26 18:15:42,037][105692] Updated weights for policy 0, policy_version 398018 (0.0008) [2023-12-26 18:15:42,061][105585] KL-divergence is very high: 113.5379 [2023-12-26 18:15:42,102][105620] Updated weights for policy 1, policy_version 398341 (0.0007) [2023-12-26 18:15:42,168][105620] Updated weights for policy 1, policy_version 398351 (0.0008) [2023-12-26 18:15:42,236][105620] Updated weights for policy 1, policy_version 398361 (0.0008) [2023-12-26 18:15:42,826][105692] Updated weights for policy 0, policy_version 398028 (0.0009) [2023-12-26 18:15:42,893][105692] Updated weights for policy 0, policy_version 398038 (0.0011) [2023-12-26 18:15:42,935][105620] Updated weights for policy 1, policy_version 398371 (0.0009) [2023-12-26 18:15:42,960][105692] Updated weights for policy 0, policy_version 398048 (0.0011) [2023-12-26 18:15:42,994][105620] Updated weights for policy 1, policy_version 398381 (0.0010) [2023-12-26 18:15:43,050][105620] Updated weights for policy 1, policy_version 398391 (0.0010) [2023-12-26 18:15:43,635][105692] Updated weights for policy 0, policy_version 398058 (0.0010) [2023-12-26 18:15:43,649][105620] Updated weights for policy 1, policy_version 398401 (0.0010) [2023-12-26 18:15:43,694][105692] Updated weights for policy 0, policy_version 398068 (0.0009) [2023-12-26 18:15:43,695][105620] Updated weights for policy 1, policy_version 398411 (0.0010) [2023-12-26 18:15:43,745][105620] Updated weights for policy 1, policy_version 398421 (0.0010) [2023-12-26 18:15:43,749][105692] Updated weights for policy 0, policy_version 398078 (0.0011) [2023-12-26 18:15:43,795][105620] Updated weights for policy 1, policy_version 398431 (0.0010) [2023-12-26 18:15:43,808][105692] Updated weights for policy 0, policy_version 398088 (0.0011) [2023-12-26 18:15:44,452][105692] Updated weights for policy 0, policy_version 398098 (0.0006) [2023-12-26 18:15:44,514][105692] Updated weights for policy 0, policy_version 398108 (0.0005) [2023-12-26 18:15:44,569][105692] Updated weights for policy 0, policy_version 398118 (0.0005) [2023-12-26 18:15:44,572][105620] Updated weights for policy 1, policy_version 398441 (0.0010) [2023-12-26 18:15:44,626][105620] Updated weights for policy 1, policy_version 398451 (0.0010) [2023-12-26 18:15:44,685][105620] Updated weights for policy 1, policy_version 398461 (0.0009) [2023-12-26 18:15:45,248][105692] Updated weights for policy 0, policy_version 398128 (0.0008) [2023-12-26 18:15:45,297][105692] Updated weights for policy 0, policy_version 398138 (0.0010) [2023-12-26 18:15:45,346][105692] Updated weights for policy 0, policy_version 398148 (0.0010) [2023-12-26 18:15:45,470][105620] Updated weights for policy 1, policy_version 398471 (0.0009) [2023-12-26 18:15:45,525][105620] Updated weights for policy 1, policy_version 398481 (0.0010) [2023-12-26 18:15:45,580][105620] Updated weights for policy 1, policy_version 398491 (0.0010) [2023-12-26 18:15:46,033][105692] Updated weights for policy 0, policy_version 398158 (0.0011) [2023-12-26 18:15:46,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19524.4, 300 sec: 19522.0). Total num frames: 203964416. Throughput: 0: 9864.3, 1: 9608.3. Samples: 203937368. Policy #0 lag: (min: 6.0, avg: 8.3, max: 38.0) [2023-12-26 18:15:46,062][104569] Avg episode reward: [(0, '8066.861'), (1, '8189.795')] [2023-12-26 18:15:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000398496_102023168.pth... [2023-12-26 18:15:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000397376_101736448.pth [2023-12-26 18:15:46,081][105692] Updated weights for policy 0, policy_version 398168 (0.0010) [2023-12-26 18:15:46,113][105585] KL-divergence is very high: 106.8026 [2023-12-26 18:15:46,129][105692] Updated weights for policy 0, policy_version 398178 (0.0010) [2023-12-26 18:15:46,161][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000398184_101949440.pth... [2023-12-26 18:15:46,167][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000397000_101646336.pth [2023-12-26 18:15:46,363][105620] Updated weights for policy 1, policy_version 398501 (0.0008) [2023-12-26 18:15:46,414][105620] Updated weights for policy 1, policy_version 398511 (0.0008) [2023-12-26 18:15:46,460][105620] Updated weights for policy 1, policy_version 398521 (0.0008) [2023-12-26 18:15:46,829][105692] Updated weights for policy 0, policy_version 398188 (0.0009) [2023-12-26 18:15:46,891][105692] Updated weights for policy 0, policy_version 398198 (0.0011) [2023-12-26 18:15:46,949][105692] Updated weights for policy 0, policy_version 398208 (0.0011) [2023-12-26 18:15:47,260][105620] Updated weights for policy 1, policy_version 398531 (0.0009) [2023-12-26 18:15:47,319][105620] Updated weights for policy 1, policy_version 398541 (0.0009) [2023-12-26 18:15:47,385][105620] Updated weights for policy 1, policy_version 398551 (0.0009) [2023-12-26 18:15:47,535][105692] Updated weights for policy 0, policy_version 398218 (0.0007) [2023-12-26 18:15:47,579][105692] Updated weights for policy 0, policy_version 398228 (0.0010) [2023-12-26 18:15:47,627][105692] Updated weights for policy 0, policy_version 398238 (0.0010) [2023-12-26 18:15:47,678][105692] Updated weights for policy 0, policy_version 398248 (0.0010) [2023-12-26 18:15:48,164][105620] Updated weights for policy 1, policy_version 398561 (0.0010) [2023-12-26 18:15:48,220][105620] Updated weights for policy 1, policy_version 398571 (0.0006) [2023-12-26 18:15:48,263][105620] Updated weights for policy 1, policy_version 398581 (0.0005) [2023-12-26 18:15:48,320][105620] Updated weights for policy 1, policy_version 398591 (0.0009) [2023-12-26 18:15:48,437][105692] Updated weights for policy 0, policy_version 398258 (0.0008) [2023-12-26 18:15:48,467][105585] KL-divergence is very high: 364.2713 [2023-12-26 18:15:48,496][105692] Updated weights for policy 0, policy_version 398268 (0.0009) [2023-12-26 18:15:48,513][105585] KL-divergence is very high: 509.1320 [2023-12-26 18:15:48,548][105692] Updated weights for policy 0, policy_version 398278 (0.0009) [2023-12-26 18:15:48,555][105585] KL-divergence is very high: 435.0915 [2023-12-26 18:15:49,063][105620] Updated weights for policy 1, policy_version 398601 (0.0009) [2023-12-26 18:15:49,111][105620] Updated weights for policy 1, policy_version 398611 (0.0009) [2023-12-26 18:15:49,161][105620] Updated weights for policy 1, policy_version 398621 (0.0008) [2023-12-26 18:15:49,232][105585] KL-divergence is very high: 223.1142 [2023-12-26 18:15:49,274][105692] Updated weights for policy 0, policy_version 398288 (0.0009) [2023-12-26 18:15:49,335][105692] Updated weights for policy 0, policy_version 398298 (0.0010) [2023-12-26 18:15:49,400][105692] Updated weights for policy 0, policy_version 398308 (0.0008) [2023-12-26 18:15:49,961][105620] Updated weights for policy 1, policy_version 398631 (0.0008) [2023-12-26 18:15:50,019][105620] Updated weights for policy 1, policy_version 398641 (0.0008) [2023-12-26 18:15:50,071][105692] Updated weights for policy 0, policy_version 398318 (0.0008) [2023-12-26 18:15:50,077][105620] Updated weights for policy 1, policy_version 398651 (0.0006) [2023-12-26 18:15:50,088][105585] KL-divergence is very high: 159.5773 [2023-12-26 18:15:50,135][105692] Updated weights for policy 0, policy_version 398328 (0.0009) [2023-12-26 18:15:50,141][105585] KL-divergence is very high: 278.7107 [2023-12-26 18:15:50,194][105585] KL-divergence is very high: 306.3116 [2023-12-26 18:15:50,200][105692] Updated weights for policy 0, policy_version 398338 (0.0010) [2023-12-26 18:15:50,745][105620] Updated weights for policy 1, policy_version 398661 (0.0008) [2023-12-26 18:15:50,816][105620] Updated weights for policy 1, policy_version 398671 (0.0010) [2023-12-26 18:15:50,889][105620] Updated weights for policy 1, policy_version 398681 (0.0007) [2023-12-26 18:15:50,933][105692] Updated weights for policy 0, policy_version 398348 (0.0010) [2023-12-26 18:15:50,993][105692] Updated weights for policy 0, policy_version 398358 (0.0008) [2023-12-26 18:15:51,046][105692] Updated weights for policy 0, policy_version 398368 (0.0008) [2023-12-26 18:15:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 204062720. Throughput: 0: 9916.0, 1: 9520.8. Samples: 204053528. Policy #0 lag: (min: 6.0, avg: 8.3, max: 38.0) [2023-12-26 18:15:51,062][104569] Avg episode reward: [(0, '8712.015'), (1, '8447.456')] [2023-12-26 18:15:51,703][105620] Updated weights for policy 1, policy_version 398691 (0.0010) [2023-12-26 18:15:51,763][105620] Updated weights for policy 1, policy_version 398701 (0.0009) [2023-12-26 18:15:51,797][105692] Updated weights for policy 0, policy_version 398378 (0.0007) [2023-12-26 18:15:51,829][105620] Updated weights for policy 1, policy_version 398711 (0.0009) [2023-12-26 18:15:51,852][105692] Updated weights for policy 0, policy_version 398388 (0.0005) [2023-12-26 18:15:51,909][105692] Updated weights for policy 0, policy_version 398398 (0.0006) [2023-12-26 18:15:51,970][105692] Updated weights for policy 0, policy_version 398408 (0.0011) [2023-12-26 18:15:52,555][105692] Updated weights for policy 0, policy_version 398418 (0.0011) [2023-12-26 18:15:52,614][105692] Updated weights for policy 0, policy_version 398428 (0.0011) [2023-12-26 18:15:52,653][105620] Updated weights for policy 1, policy_version 398721 (0.0008) [2023-12-26 18:15:52,666][105692] Updated weights for policy 0, policy_version 398438 (0.0010) [2023-12-26 18:15:52,713][105620] Updated weights for policy 1, policy_version 398731 (0.0006) [2023-12-26 18:15:52,785][105620] Updated weights for policy 1, policy_version 398741 (0.0005) [2023-12-26 18:15:52,854][105620] Updated weights for policy 1, policy_version 398751 (0.0005) [2023-12-26 18:15:53,340][105692] Updated weights for policy 0, policy_version 398448 (0.0006) [2023-12-26 18:15:53,407][105692] Updated weights for policy 0, policy_version 398458 (0.0008) [2023-12-26 18:15:53,475][105692] Updated weights for policy 0, policy_version 398468 (0.0008) [2023-12-26 18:15:53,533][105620] Updated weights for policy 1, policy_version 398761 (0.0008) [2023-12-26 18:15:53,586][105620] Updated weights for policy 1, policy_version 398772 (0.0010) [2023-12-26 18:15:53,644][105620] Updated weights for policy 1, policy_version 398782 (0.0006) [2023-12-26 18:15:54,124][105692] Updated weights for policy 0, policy_version 398478 (0.0006) [2023-12-26 18:15:54,170][105692] Updated weights for policy 0, policy_version 398488 (0.0007) [2023-12-26 18:15:54,228][105692] Updated weights for policy 0, policy_version 398498 (0.0010) [2023-12-26 18:15:54,426][105620] Updated weights for policy 1, policy_version 398792 (0.0008) [2023-12-26 18:15:54,492][105620] Updated weights for policy 1, policy_version 398802 (0.0006) [2023-12-26 18:15:54,556][105620] Updated weights for policy 1, policy_version 398812 (0.0007) [2023-12-26 18:15:54,998][105692] Updated weights for policy 0, policy_version 398508 (0.0010) [2023-12-26 18:15:55,038][105585] KL-divergence is very high: 170.8583 [2023-12-26 18:15:55,063][105692] Updated weights for policy 0, policy_version 398518 (0.0008) [2023-12-26 18:15:55,066][105585] KL-divergence is very high: 119.9388 [2023-12-26 18:15:55,085][105585] KL-divergence is very high: 474.8865 [2023-12-26 18:15:55,093][105585] KL-divergence is very high: 167.9088 [2023-12-26 18:15:55,115][105585] KL-divergence is very high: 123.2859 [2023-12-26 18:15:55,129][105692] Updated weights for policy 0, policy_version 398528 (0.0008) [2023-12-26 18:15:55,135][105585] KL-divergence is very high: 611.3216 [2023-12-26 18:15:55,296][105620] Updated weights for policy 1, policy_version 398822 (0.0010) [2023-12-26 18:15:55,348][105620] Updated weights for policy 1, policy_version 398832 (0.0010) [2023-12-26 18:15:55,407][105620] Updated weights for policy 1, policy_version 398842 (0.0010) [2023-12-26 18:15:55,735][105692] Updated weights for policy 0, policy_version 398538 (0.0008) [2023-12-26 18:15:55,786][105692] Updated weights for policy 0, policy_version 398548 (0.0005) [2023-12-26 18:15:55,850][105692] Updated weights for policy 0, policy_version 398558 (0.0009) [2023-12-26 18:15:55,896][105692] Updated weights for policy 0, policy_version 398568 (0.0006) [2023-12-26 18:15:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 204161024. Throughput: 0: 10073.6, 1: 9431.0. Samples: 204169780. Policy #0 lag: (min: 6.0, avg: 8.3, max: 38.0) [2023-12-26 18:15:56,063][104569] Avg episode reward: [(0, '8530.794'), (1, '8550.994')] [2023-12-26 18:15:56,082][105620] Updated weights for policy 1, policy_version 398852 (0.0008) [2023-12-26 18:15:56,146][105620] Updated weights for policy 1, policy_version 398862 (0.0005) [2023-12-26 18:15:56,208][105620] Updated weights for policy 1, policy_version 398872 (0.0005) [2023-12-26 18:15:56,516][105692] Updated weights for policy 0, policy_version 398578 (0.0005) [2023-12-26 18:15:56,531][105585] KL-divergence is very high: 148.9232 [2023-12-26 18:15:56,562][105692] Updated weights for policy 0, policy_version 398588 (0.0005) [2023-12-26 18:15:56,569][105585] KL-divergence is very high: 190.2685 [2023-12-26 18:15:56,619][105585] KL-divergence is very high: 122.9805 [2023-12-26 18:15:56,625][105692] Updated weights for policy 0, policy_version 398598 (0.0005) [2023-12-26 18:15:56,791][105620] Updated weights for policy 1, policy_version 398882 (0.0005) [2023-12-26 18:15:56,839][105620] Updated weights for policy 1, policy_version 398892 (0.0005) [2023-12-26 18:15:56,897][105620] Updated weights for policy 1, policy_version 398902 (0.0005) [2023-12-26 18:15:56,954][105620] Updated weights for policy 1, policy_version 398912 (0.0005) [2023-12-26 18:15:57,360][105692] Updated weights for policy 0, policy_version 398608 (0.0007) [2023-12-26 18:15:57,382][105585] KL-divergence is very high: 173.8367 [2023-12-26 18:15:57,417][105692] Updated weights for policy 0, policy_version 398618 (0.0006) [2023-12-26 18:15:57,428][105585] KL-divergence is very high: 234.4183 [2023-12-26 18:15:57,476][105692] Updated weights for policy 0, policy_version 398628 (0.0005) [2023-12-26 18:15:57,477][105585] KL-divergence is very high: 184.0422 [2023-12-26 18:15:57,528][105620] Updated weights for policy 1, policy_version 398922 (0.0005) [2023-12-26 18:15:57,597][105620] Updated weights for policy 1, policy_version 398932 (0.0005) [2023-12-26 18:15:57,657][105620] Updated weights for policy 1, policy_version 398942 (0.0006) [2023-12-26 18:15:58,139][105692] Updated weights for policy 0, policy_version 398638 (0.0009) [2023-12-26 18:15:58,203][105692] Updated weights for policy 0, policy_version 398648 (0.0011) [2023-12-26 18:15:58,263][105692] Updated weights for policy 0, policy_version 398658 (0.0008) [2023-12-26 18:15:58,321][105620] Updated weights for policy 1, policy_version 398952 (0.0010) [2023-12-26 18:15:58,388][105620] Updated weights for policy 1, policy_version 398962 (0.0011) [2023-12-26 18:15:58,457][105620] Updated weights for policy 1, policy_version 398972 (0.0010) [2023-12-26 18:15:59,082][105692] Updated weights for policy 0, policy_version 398668 (0.0008) [2023-12-26 18:15:59,139][105692] Updated weights for policy 0, policy_version 398678 (0.0009) [2023-12-26 18:15:59,195][105692] Updated weights for policy 0, policy_version 398688 (0.0009) [2023-12-26 18:15:59,232][105620] Updated weights for policy 1, policy_version 398982 (0.0008) [2023-12-26 18:15:59,316][105620] Updated weights for policy 1, policy_version 398992 (0.0009) [2023-12-26 18:15:59,379][105620] Updated weights for policy 1, policy_version 399002 (0.0009) [2023-12-26 18:15:59,862][105692] Updated weights for policy 0, policy_version 398698 (0.0007) [2023-12-26 18:15:59,928][105692] Updated weights for policy 0, policy_version 398708 (0.0007) [2023-12-26 18:15:59,987][105692] Updated weights for policy 0, policy_version 398718 (0.0009) [2023-12-26 18:16:00,048][105692] Updated weights for policy 0, policy_version 398728 (0.0010) [2023-12-26 18:16:00,134][105620] Updated weights for policy 1, policy_version 399012 (0.0009) [2023-12-26 18:16:00,197][105620] Updated weights for policy 1, policy_version 399022 (0.0011) [2023-12-26 18:16:00,254][105620] Updated weights for policy 1, policy_version 399032 (0.0011) [2023-12-26 18:16:00,827][105692] Updated weights for policy 0, policy_version 398738 (0.0010) [2023-12-26 18:16:00,880][105692] Updated weights for policy 0, policy_version 398748 (0.0009) [2023-12-26 18:16:00,890][105620] Updated weights for policy 1, policy_version 399042 (0.0010) [2023-12-26 18:16:00,938][105692] Updated weights for policy 0, policy_version 398758 (0.0006) [2023-12-26 18:16:00,954][105620] Updated weights for policy 1, policy_version 399052 (0.0005) [2023-12-26 18:16:01,002][105620] Updated weights for policy 1, policy_version 399062 (0.0008) [2023-12-26 18:16:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 204259328. Throughput: 0: 10074.7, 1: 9538.4. Samples: 204231720. Policy #0 lag: (min: 6.0, avg: 8.3, max: 38.0) [2023-12-26 18:16:01,062][104569] Avg episode reward: [(0, '8439.568'), (1, '8913.872')] [2023-12-26 18:16:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000398760_102096896.pth... [2023-12-26 18:16:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000399072_102170624.pth... [2023-12-26 18:16:01,071][105620] Updated weights for policy 1, policy_version 399072 (0.0008) [2023-12-26 18:16:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000397608_101801984.pth [2023-12-26 18:16:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000397952_101883904.pth [2023-12-26 18:16:01,696][105692] Updated weights for policy 0, policy_version 398768 (0.0007) [2023-12-26 18:16:01,736][105620] Updated weights for policy 1, policy_version 399082 (0.0008) [2023-12-26 18:16:01,759][105585] KL-divergence is very high: 146.1546 [2023-12-26 18:16:01,767][105692] Updated weights for policy 0, policy_version 398778 (0.0007) [2023-12-26 18:16:01,800][105620] Updated weights for policy 1, policy_version 399092 (0.0006) [2023-12-26 18:16:01,814][105585] KL-divergence is very high: 120.8700 [2023-12-26 18:16:01,835][105692] Updated weights for policy 0, policy_version 398788 (0.0008) [2023-12-26 18:16:01,858][105620] Updated weights for policy 1, policy_version 399102 (0.0005) [2023-12-26 18:16:02,428][105585] KL-divergence is very high: 267.7171 [2023-12-26 18:16:02,436][105692] Updated weights for policy 0, policy_version 398798 (0.0007) [2023-12-26 18:16:02,437][105620] Updated weights for policy 1, policy_version 399112 (0.0007) [2023-12-26 18:16:02,477][105585] KL-divergence is very high: 455.6115 [2023-12-26 18:16:02,497][105692] Updated weights for policy 0, policy_version 398808 (0.0009) [2023-12-26 18:16:02,506][105620] Updated weights for policy 1, policy_version 399122 (0.0006) [2023-12-26 18:16:02,534][105585] KL-divergence is very high: 264.8417 [2023-12-26 18:16:02,563][105692] Updated weights for policy 0, policy_version 398818 (0.0006) [2023-12-26 18:16:02,568][105620] Updated weights for policy 1, policy_version 399132 (0.0007) [2023-12-26 18:16:03,238][105692] Updated weights for policy 0, policy_version 398828 (0.0006) [2023-12-26 18:16:03,239][105620] Updated weights for policy 1, policy_version 399142 (0.0007) [2023-12-26 18:16:03,293][105692] Updated weights for policy 0, policy_version 398838 (0.0005) [2023-12-26 18:16:03,299][105620] Updated weights for policy 1, policy_version 399152 (0.0006) [2023-12-26 18:16:03,352][105692] Updated weights for policy 0, policy_version 398848 (0.0008) [2023-12-26 18:16:03,353][105620] Updated weights for policy 1, policy_version 399162 (0.0005) [2023-12-26 18:16:04,026][105692] Updated weights for policy 0, policy_version 398858 (0.0008) [2023-12-26 18:16:04,050][105620] Updated weights for policy 1, policy_version 399172 (0.0006) [2023-12-26 18:16:04,090][105692] Updated weights for policy 0, policy_version 398868 (0.0007) [2023-12-26 18:16:04,115][105620] Updated weights for policy 1, policy_version 399182 (0.0007) [2023-12-26 18:16:04,156][105692] Updated weights for policy 0, policy_version 398878 (0.0007) [2023-12-26 18:16:04,183][105620] Updated weights for policy 1, policy_version 399192 (0.0010) [2023-12-26 18:16:04,214][105692] Updated weights for policy 0, policy_version 398888 (0.0007) [2023-12-26 18:16:04,917][105692] Updated weights for policy 0, policy_version 398898 (0.0005) [2023-12-26 18:16:04,978][105692] Updated weights for policy 0, policy_version 398908 (0.0010) [2023-12-26 18:16:04,995][105620] Updated weights for policy 1, policy_version 399202 (0.0010) [2023-12-26 18:16:05,036][105692] Updated weights for policy 0, policy_version 398918 (0.0010) [2023-12-26 18:16:05,047][105620] Updated weights for policy 1, policy_version 399212 (0.0010) [2023-12-26 18:16:05,112][105620] Updated weights for policy 1, policy_version 399222 (0.0010) [2023-12-26 18:16:05,180][105620] Updated weights for policy 1, policy_version 399232 (0.0011) [2023-12-26 18:16:05,672][105692] Updated weights for policy 0, policy_version 398928 (0.0010) [2023-12-26 18:16:05,738][105692] Updated weights for policy 0, policy_version 398938 (0.0007) [2023-12-26 18:16:05,803][105692] Updated weights for policy 0, policy_version 398948 (0.0005) [2023-12-26 18:16:05,912][105620] Updated weights for policy 1, policy_version 399242 (0.0009) [2023-12-26 18:16:05,983][105620] Updated weights for policy 1, policy_version 399252 (0.0005) [2023-12-26 18:16:06,053][105620] Updated weights for policy 1, policy_version 399262 (0.0005) [2023-12-26 18:16:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 204357632. Throughput: 0: 9934.7, 1: 9527.5. Samples: 204349644. Policy #0 lag: (min: 6.0, avg: 8.3, max: 38.0) [2023-12-26 18:16:06,063][104569] Avg episode reward: [(0, '8440.452'), (1, '8739.760')] [2023-12-26 18:16:06,412][105692] Updated weights for policy 0, policy_version 398958 (0.0008) [2023-12-26 18:16:06,470][105692] Updated weights for policy 0, policy_version 398968 (0.0010) [2023-12-26 18:16:06,530][105692] Updated weights for policy 0, policy_version 398978 (0.0011) [2023-12-26 18:16:06,555][105585] KL-divergence is very high: 106.0543 [2023-12-26 18:16:06,721][105620] Updated weights for policy 1, policy_version 399272 (0.0009) [2023-12-26 18:16:06,750][105586] KL-divergence is very high: 247.6141 [2023-12-26 18:16:06,758][105586] KL-divergence is very high: 200.7699 [2023-12-26 18:16:06,765][105586] KL-divergence is very high: 344.9892 [2023-12-26 18:16:06,787][105620] Updated weights for policy 1, policy_version 399282 (0.0011) [2023-12-26 18:16:06,800][105586] KL-divergence is very high: 609.5586 [2023-12-26 18:16:06,805][105586] KL-divergence is very high: 441.5396 [2023-12-26 18:16:06,810][105586] KL-divergence is very high: 611.6706 [2023-12-26 18:16:06,831][105620] Updated weights for policy 1, policy_version 399292 (0.0010) [2023-12-26 18:16:06,837][105586] KL-divergence is very high: 665.0241 [2023-12-26 18:16:06,846][105586] KL-divergence is very high: 429.0197 [2023-12-26 18:16:06,852][105586] KL-divergence is very high: 600.0428 [2023-12-26 18:16:07,242][105692] Updated weights for policy 0, policy_version 398988 (0.0008) [2023-12-26 18:16:07,303][105692] Updated weights for policy 0, policy_version 398998 (0.0005) [2023-12-26 18:16:07,363][105692] Updated weights for policy 0, policy_version 399008 (0.0005) [2023-12-26 18:16:07,602][105620] Updated weights for policy 1, policy_version 399302 (0.0011) [2023-12-26 18:16:07,661][105620] Updated weights for policy 1, policy_version 399312 (0.0011) [2023-12-26 18:16:07,724][105620] Updated weights for policy 1, policy_version 399322 (0.0011) [2023-12-26 18:16:08,018][105692] Updated weights for policy 0, policy_version 399018 (0.0008) [2023-12-26 18:16:08,082][105692] Updated weights for policy 0, policy_version 399028 (0.0005) [2023-12-26 18:16:08,137][105692] Updated weights for policy 0, policy_version 399038 (0.0005) [2023-12-26 18:16:08,204][105692] Updated weights for policy 0, policy_version 399048 (0.0007) [2023-12-26 18:16:08,447][105620] Updated weights for policy 1, policy_version 399332 (0.0009) [2023-12-26 18:16:08,512][105620] Updated weights for policy 1, policy_version 399342 (0.0006) [2023-12-26 18:16:08,573][105620] Updated weights for policy 1, policy_version 399352 (0.0009) [2023-12-26 18:16:08,847][105692] Updated weights for policy 0, policy_version 399058 (0.0009) [2023-12-26 18:16:08,920][105692] Updated weights for policy 0, policy_version 399068 (0.0011) [2023-12-26 18:16:08,990][105692] Updated weights for policy 0, policy_version 399078 (0.0011) [2023-12-26 18:16:09,189][105620] Updated weights for policy 1, policy_version 399362 (0.0010) [2023-12-26 18:16:09,258][105620] Updated weights for policy 1, policy_version 399372 (0.0008) [2023-12-26 18:16:09,321][105620] Updated weights for policy 1, policy_version 399382 (0.0010) [2023-12-26 18:16:09,382][105620] Updated weights for policy 1, policy_version 399392 (0.0009) [2023-12-26 18:16:09,699][105585] KL-divergence is very high: 435.5604 [2023-12-26 18:16:09,700][105692] Updated weights for policy 0, policy_version 399088 (0.0009) [2023-12-26 18:16:09,711][105585] KL-divergence is very high: 449.1081 [2023-12-26 18:16:09,722][105585] KL-divergence is very high: 157.6002 [2023-12-26 18:16:09,748][105585] KL-divergence is very high: 644.0850 [2023-12-26 18:16:09,762][105692] Updated weights for policy 0, policy_version 399098 (0.0010) [2023-12-26 18:16:09,762][105585] KL-divergence is very high: 516.2015 [2023-12-26 18:16:09,777][105585] KL-divergence is very high: 127.8839 [2023-12-26 18:16:09,804][105585] KL-divergence is very high: 571.9914 [2023-12-26 18:16:09,817][105585] KL-divergence is very high: 377.8106 [2023-12-26 18:16:09,834][105692] Updated weights for policy 0, policy_version 399108 (0.0011) [2023-12-26 18:16:10,056][105620] Updated weights for policy 1, policy_version 399402 (0.0011) [2023-12-26 18:16:10,116][105620] Updated weights for policy 1, policy_version 399412 (0.0009) [2023-12-26 18:16:10,177][105620] Updated weights for policy 1, policy_version 399422 (0.0011) [2023-12-26 18:16:10,668][105692] Updated weights for policy 0, policy_version 399118 (0.0010) [2023-12-26 18:16:10,715][105692] Updated weights for policy 0, policy_version 399128 (0.0009) [2023-12-26 18:16:10,764][105692] Updated weights for policy 0, policy_version 399138 (0.0008) [2023-12-26 18:16:10,804][105620] Updated weights for policy 1, policy_version 399432 (0.0010) [2023-12-26 18:16:10,855][105620] Updated weights for policy 1, policy_version 399442 (0.0010) [2023-12-26 18:16:10,916][105620] Updated weights for policy 1, policy_version 399452 (0.0010) [2023-12-26 18:16:11,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 204464128. Throughput: 0: 9978.6, 1: 9600.3. Samples: 204469228. Policy #0 lag: (min: 6.0, avg: 8.3, max: 38.0) [2023-12-26 18:16:11,063][104569] Avg episode reward: [(0, '8070.992'), (1, '8082.460')] [2023-12-26 18:16:11,583][105692] Updated weights for policy 0, policy_version 399148 (0.0007) [2023-12-26 18:16:11,644][105692] Updated weights for policy 0, policy_version 399158 (0.0008) [2023-12-26 18:16:11,687][105620] Updated weights for policy 1, policy_version 399462 (0.0010) [2023-12-26 18:16:11,706][105692] Updated weights for policy 0, policy_version 399168 (0.0007) [2023-12-26 18:16:11,751][105620] Updated weights for policy 1, policy_version 399472 (0.0009) [2023-12-26 18:16:11,812][105620] Updated weights for policy 1, policy_version 399482 (0.0006) [2023-12-26 18:16:12,519][105692] Updated weights for policy 0, policy_version 399178 (0.0008) [2023-12-26 18:16:12,525][105620] Updated weights for policy 1, policy_version 399492 (0.0006) [2023-12-26 18:16:12,589][105620] Updated weights for policy 1, policy_version 399502 (0.0008) [2023-12-26 18:16:12,590][105692] Updated weights for policy 0, policy_version 399188 (0.0006) [2023-12-26 18:16:12,653][105620] Updated weights for policy 1, policy_version 399512 (0.0007) [2023-12-26 18:16:12,655][105692] Updated weights for policy 0, policy_version 399198 (0.0006) [2023-12-26 18:16:12,717][105692] Updated weights for policy 0, policy_version 399208 (0.0007) [2023-12-26 18:16:13,398][105620] Updated weights for policy 1, policy_version 399522 (0.0009) [2023-12-26 18:16:13,417][105692] Updated weights for policy 0, policy_version 399218 (0.0011) [2023-12-26 18:16:13,449][105620] Updated weights for policy 1, policy_version 399532 (0.0010) [2023-12-26 18:16:13,471][105692] Updated weights for policy 0, policy_version 399228 (0.0010) [2023-12-26 18:16:13,507][105620] Updated weights for policy 1, policy_version 399542 (0.0010) [2023-12-26 18:16:13,529][105692] Updated weights for policy 0, policy_version 399238 (0.0010) [2023-12-26 18:16:13,562][105620] Updated weights for policy 1, policy_version 399552 (0.0010) [2023-12-26 18:16:14,185][105620] Updated weights for policy 1, policy_version 399562 (0.0005) [2023-12-26 18:16:14,196][105692] Updated weights for policy 0, policy_version 399248 (0.0010) [2023-12-26 18:16:14,244][105692] Updated weights for policy 0, policy_version 399258 (0.0010) [2023-12-26 18:16:14,246][105620] Updated weights for policy 1, policy_version 399572 (0.0006) [2023-12-26 18:16:14,248][105585] KL-divergence is very high: 108.7700 [2023-12-26 18:16:14,295][105692] Updated weights for policy 0, policy_version 399268 (0.0010) [2023-12-26 18:16:14,309][105620] Updated weights for policy 1, policy_version 399582 (0.0005) [2023-12-26 18:16:14,843][105620] Updated weights for policy 1, policy_version 399592 (0.0006) [2023-12-26 18:16:14,911][105620] Updated weights for policy 1, policy_version 399602 (0.0008) [2023-12-26 18:16:14,973][105620] Updated weights for policy 1, policy_version 399612 (0.0007) [2023-12-26 18:16:15,063][105692] Updated weights for policy 0, policy_version 399278 (0.0010) [2023-12-26 18:16:15,119][105692] Updated weights for policy 0, policy_version 399288 (0.0010) [2023-12-26 18:16:15,125][105585] KL-divergence is very high: 395.5613 [2023-12-26 18:16:15,180][105585] KL-divergence is very high: 577.7658 [2023-12-26 18:16:15,188][105692] Updated weights for policy 0, policy_version 399298 (0.0011) [2023-12-26 18:16:15,544][105620] Updated weights for policy 1, policy_version 399622 (0.0006) [2023-12-26 18:16:15,605][105620] Updated weights for policy 1, policy_version 399632 (0.0006) [2023-12-26 18:16:15,663][105620] Updated weights for policy 1, policy_version 399642 (0.0008) [2023-12-26 18:16:15,790][105585] KL-divergence is very high: 129.2840 [2023-12-26 18:16:15,814][105692] Updated weights for policy 0, policy_version 399308 (0.0010) [2023-12-26 18:16:15,867][105692] Updated weights for policy 0, policy_version 399318 (0.0008) [2023-12-26 18:16:15,921][105692] Updated weights for policy 0, policy_version 399328 (0.0005) [2023-12-26 18:16:16,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 204562432. Throughput: 0: 9826.2, 1: 9613.8. Samples: 204524648. Policy #0 lag: (min: 6.0, avg: 8.3, max: 38.0) [2023-12-26 18:16:16,063][104569] Avg episode reward: [(0, '7702.681'), (1, '8537.990')] [2023-12-26 18:16:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000399336_102244352.pth... [2023-12-26 18:16:16,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000399648_102318080.pth... [2023-12-26 18:16:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000398184_101949440.pth [2023-12-26 18:16:16,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000398496_102023168.pth [2023-12-26 18:16:16,289][105620] Updated weights for policy 1, policy_version 399652 (0.0010) [2023-12-26 18:16:16,351][105620] Updated weights for policy 1, policy_version 399662 (0.0010) [2023-12-26 18:16:16,413][105620] Updated weights for policy 1, policy_version 399672 (0.0009) [2023-12-26 18:16:16,449][105692] Updated weights for policy 0, policy_version 399338 (0.0005) [2023-12-26 18:16:16,513][105692] Updated weights for policy 0, policy_version 399348 (0.0005) [2023-12-26 18:16:16,573][105692] Updated weights for policy 0, policy_version 399358 (0.0006) [2023-12-26 18:16:16,637][105692] Updated weights for policy 0, policy_version 399368 (0.0010) [2023-12-26 18:16:17,170][105692] Updated weights for policy 0, policy_version 399378 (0.0005) [2023-12-26 18:16:17,193][105620] Updated weights for policy 1, policy_version 399682 (0.0009) [2023-12-26 18:16:17,217][105692] Updated weights for policy 0, policy_version 399388 (0.0005) [2023-12-26 18:16:17,251][105620] Updated weights for policy 1, policy_version 399692 (0.0010) [2023-12-26 18:16:17,266][105692] Updated weights for policy 0, policy_version 399398 (0.0005) [2023-12-26 18:16:17,306][105620] Updated weights for policy 1, policy_version 399702 (0.0010) [2023-12-26 18:16:17,358][105620] Updated weights for policy 1, policy_version 399712 (0.0010) [2023-12-26 18:16:17,963][105692] Updated weights for policy 0, policy_version 399408 (0.0009) [2023-12-26 18:16:18,012][105620] Updated weights for policy 1, policy_version 399722 (0.0009) [2023-12-26 18:16:18,018][105692] Updated weights for policy 0, policy_version 399418 (0.0010) [2023-12-26 18:16:18,066][105692] Updated weights for policy 0, policy_version 399428 (0.0010) [2023-12-26 18:16:18,066][105620] Updated weights for policy 1, policy_version 399732 (0.0005) [2023-12-26 18:16:18,115][105620] Updated weights for policy 1, policy_version 399742 (0.0005) [2023-12-26 18:16:18,787][105692] Updated weights for policy 0, policy_version 399438 (0.0007) [2023-12-26 18:16:18,804][105620] Updated weights for policy 1, policy_version 399752 (0.0008) [2023-12-26 18:16:18,845][105692] Updated weights for policy 0, policy_version 399448 (0.0006) [2023-12-26 18:16:18,868][105620] Updated weights for policy 1, policy_version 399762 (0.0006) [2023-12-26 18:16:18,912][105692] Updated weights for policy 0, policy_version 399458 (0.0006) [2023-12-26 18:16:18,934][105620] Updated weights for policy 1, policy_version 399772 (0.0009) [2023-12-26 18:16:19,525][105692] Updated weights for policy 0, policy_version 399468 (0.0007) [2023-12-26 18:16:19,587][105692] Updated weights for policy 0, policy_version 399478 (0.0008) [2023-12-26 18:16:19,650][105692] Updated weights for policy 0, policy_version 399488 (0.0007) [2023-12-26 18:16:19,664][105620] Updated weights for policy 1, policy_version 399782 (0.0011) [2023-12-26 18:16:19,717][105620] Updated weights for policy 1, policy_version 399792 (0.0010) [2023-12-26 18:16:19,774][105620] Updated weights for policy 1, policy_version 399802 (0.0011) [2023-12-26 18:16:20,406][105692] Updated weights for policy 0, policy_version 399498 (0.0007) [2023-12-26 18:16:20,455][105585] KL-divergence is very high: 154.2011 [2023-12-26 18:16:20,476][105692] Updated weights for policy 0, policy_version 399508 (0.0006) [2023-12-26 18:16:20,510][105585] KL-divergence is very high: 131.7749 [2023-12-26 18:16:20,542][105692] Updated weights for policy 0, policy_version 399518 (0.0006) [2023-12-26 18:16:20,554][105620] Updated weights for policy 1, policy_version 399812 (0.0009) [2023-12-26 18:16:20,611][105692] Updated weights for policy 0, policy_version 399528 (0.0007) [2023-12-26 18:16:20,622][105620] Updated weights for policy 1, policy_version 399822 (0.0008) [2023-12-26 18:16:20,681][105620] Updated weights for policy 1, policy_version 399832 (0.0009) [2023-12-26 18:16:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 204660736. Throughput: 0: 9875.9, 1: 9718.0. Samples: 204651148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:16:21,063][104569] Avg episode reward: [(0, '7706.028'), (1, '8607.898')] [2023-12-26 18:16:21,262][105692] Updated weights for policy 0, policy_version 399538 (0.0011) [2023-12-26 18:16:21,322][105692] Updated weights for policy 0, policy_version 399548 (0.0011) [2023-12-26 18:16:21,343][105585] KL-divergence is very high: 117.7626 [2023-12-26 18:16:21,387][105692] Updated weights for policy 0, policy_version 399558 (0.0010) [2023-12-26 18:16:21,395][105585] KL-divergence is very high: 141.7206 [2023-12-26 18:16:21,488][105620] Updated weights for policy 1, policy_version 399842 (0.0009) [2023-12-26 18:16:21,552][105620] Updated weights for policy 1, policy_version 399852 (0.0008) [2023-12-26 18:16:21,620][105620] Updated weights for policy 1, policy_version 399862 (0.0008) [2023-12-26 18:16:21,683][105620] Updated weights for policy 1, policy_version 399872 (0.0008) [2023-12-26 18:16:22,113][105585] KL-divergence is very high: 221.6804 [2023-12-26 18:16:22,119][105585] KL-divergence is very high: 224.9644 [2023-12-26 18:16:22,159][105692] Updated weights for policy 0, policy_version 399568 (0.0009) [2023-12-26 18:16:22,161][105585] KL-divergence is very high: 246.9351 [2023-12-26 18:16:22,169][105585] KL-divergence is very high: 232.5228 [2023-12-26 18:16:22,214][105585] KL-divergence is very high: 210.0594 [2023-12-26 18:16:22,221][105585] KL-divergence is very high: 195.9261 [2023-12-26 18:16:22,226][105692] Updated weights for policy 0, policy_version 399578 (0.0006) [2023-12-26 18:16:22,271][105585] KL-divergence is very high: 174.9648 [2023-12-26 18:16:22,278][105585] KL-divergence is very high: 168.7619 [2023-12-26 18:16:22,297][105692] Updated weights for policy 0, policy_version 399588 (0.0008) [2023-12-26 18:16:22,339][105620] Updated weights for policy 1, policy_version 399882 (0.0006) [2023-12-26 18:16:22,402][105620] Updated weights for policy 1, policy_version 399892 (0.0007) [2023-12-26 18:16:22,464][105620] Updated weights for policy 1, policy_version 399902 (0.0007) [2023-12-26 18:16:22,961][105692] Updated weights for policy 0, policy_version 399598 (0.0010) [2023-12-26 18:16:23,010][105692] Updated weights for policy 0, policy_version 399608 (0.0010) [2023-12-26 18:16:23,058][105692] Updated weights for policy 0, policy_version 399618 (0.0010) [2023-12-26 18:16:23,257][105620] Updated weights for policy 1, policy_version 399912 (0.0009) [2023-12-26 18:16:23,308][105620] Updated weights for policy 1, policy_version 399922 (0.0008) [2023-12-26 18:16:23,356][105620] Updated weights for policy 1, policy_version 399932 (0.0008) [2023-12-26 18:16:23,762][105692] Updated weights for policy 0, policy_version 399628 (0.0009) [2023-12-26 18:16:23,820][105692] Updated weights for policy 0, policy_version 399638 (0.0009) [2023-12-26 18:16:23,865][105692] Updated weights for policy 0, policy_version 399648 (0.0008) [2023-12-26 18:16:24,166][105620] Updated weights for policy 1, policy_version 399942 (0.0007) [2023-12-26 18:16:24,222][105620] Updated weights for policy 1, policy_version 399952 (0.0008) [2023-12-26 18:16:24,281][105620] Updated weights for policy 1, policy_version 399962 (0.0008) [2023-12-26 18:16:24,581][105692] Updated weights for policy 0, policy_version 399658 (0.0008) [2023-12-26 18:16:24,640][105692] Updated weights for policy 0, policy_version 399668 (0.0010) [2023-12-26 18:16:24,706][105692] Updated weights for policy 0, policy_version 399678 (0.0011) [2023-12-26 18:16:24,772][105692] Updated weights for policy 0, policy_version 399688 (0.0010) [2023-12-26 18:16:25,021][105620] Updated weights for policy 1, policy_version 399972 (0.0008) [2023-12-26 18:16:25,088][105620] Updated weights for policy 1, policy_version 399982 (0.0008) [2023-12-26 18:16:25,151][105620] Updated weights for policy 1, policy_version 399992 (0.0008) [2023-12-26 18:16:25,505][105692] Updated weights for policy 0, policy_version 399698 (0.0010) [2023-12-26 18:16:25,557][105692] Updated weights for policy 0, policy_version 399708 (0.0010) [2023-12-26 18:16:25,612][105692] Updated weights for policy 0, policy_version 399718 (0.0010) [2023-12-26 18:16:25,782][105620] Updated weights for policy 1, policy_version 400002 (0.0007) [2023-12-26 18:16:25,837][105620] Updated weights for policy 1, policy_version 400012 (0.0007) [2023-12-26 18:16:25,889][105620] Updated weights for policy 1, policy_version 400022 (0.0008) [2023-12-26 18:16:25,937][105620] Updated weights for policy 1, policy_version 400032 (0.0008) [2023-12-26 18:16:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 204759040. Throughput: 0: 9919.6, 1: 9716.3. Samples: 204764908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:16:26,062][104569] Avg episode reward: [(0, '7159.201'), (1, '8621.343')] [2023-12-26 18:16:26,325][105692] Updated weights for policy 0, policy_version 399728 (0.0010) [2023-12-26 18:16:26,376][105692] Updated weights for policy 0, policy_version 399738 (0.0010) [2023-12-26 18:16:26,439][105692] Updated weights for policy 0, policy_version 399748 (0.0010) [2023-12-26 18:16:26,667][105620] Updated weights for policy 1, policy_version 400042 (0.0009) [2023-12-26 18:16:26,724][105620] Updated weights for policy 1, policy_version 400052 (0.0010) [2023-12-26 18:16:26,782][105620] Updated weights for policy 1, policy_version 400062 (0.0011) [2023-12-26 18:16:27,040][105692] Updated weights for policy 0, policy_version 399758 (0.0008) [2023-12-26 18:16:27,094][105692] Updated weights for policy 0, policy_version 399768 (0.0010) [2023-12-26 18:16:27,158][105692] Updated weights for policy 0, policy_version 399778 (0.0010) [2023-12-26 18:16:27,590][105620] Updated weights for policy 1, policy_version 400072 (0.0009) [2023-12-26 18:16:27,648][105620] Updated weights for policy 1, policy_version 400082 (0.0010) [2023-12-26 18:16:27,712][105620] Updated weights for policy 1, policy_version 400092 (0.0009) [2023-12-26 18:16:27,814][105692] Updated weights for policy 0, policy_version 399788 (0.0008) [2023-12-26 18:16:27,881][105692] Updated weights for policy 0, policy_version 399798 (0.0005) [2023-12-26 18:16:27,943][105692] Updated weights for policy 0, policy_version 399808 (0.0008) [2023-12-26 18:16:28,528][105620] Updated weights for policy 1, policy_version 400102 (0.0008) [2023-12-26 18:16:28,584][105620] Updated weights for policy 1, policy_version 400112 (0.0005) [2023-12-26 18:16:28,597][105692] Updated weights for policy 0, policy_version 399818 (0.0009) [2023-12-26 18:16:28,650][105620] Updated weights for policy 1, policy_version 400122 (0.0008) [2023-12-26 18:16:28,663][105692] Updated weights for policy 0, policy_version 399828 (0.0006) [2023-12-26 18:16:28,723][105692] Updated weights for policy 0, policy_version 399838 (0.0008) [2023-12-26 18:16:28,778][105692] Updated weights for policy 0, policy_version 399848 (0.0009) [2023-12-26 18:16:29,332][105620] Updated weights for policy 1, policy_version 400132 (0.0009) [2023-12-26 18:16:29,402][105620] Updated weights for policy 1, policy_version 400142 (0.0009) [2023-12-26 18:16:29,457][105620] Updated weights for policy 1, policy_version 400152 (0.0009) [2023-12-26 18:16:29,538][105692] Updated weights for policy 0, policy_version 399858 (0.0007) [2023-12-26 18:16:29,585][105692] Updated weights for policy 0, policy_version 399868 (0.0009) [2023-12-26 18:16:29,642][105692] Updated weights for policy 0, policy_version 399878 (0.0009) [2023-12-26 18:16:30,079][105620] Updated weights for policy 1, policy_version 400162 (0.0008) [2023-12-26 18:16:30,132][105620] Updated weights for policy 1, policy_version 400172 (0.0006) [2023-12-26 18:16:30,194][105620] Updated weights for policy 1, policy_version 400183 (0.0007) [2023-12-26 18:16:30,338][105692] Updated weights for policy 0, policy_version 399888 (0.0007) [2023-12-26 18:16:30,398][105692] Updated weights for policy 0, policy_version 399898 (0.0006) [2023-12-26 18:16:30,471][105692] Updated weights for policy 0, policy_version 399908 (0.0005) [2023-12-26 18:16:30,892][105620] Updated weights for policy 1, policy_version 400193 (0.0010) [2023-12-26 18:16:30,960][105620] Updated weights for policy 1, policy_version 400203 (0.0006) [2023-12-26 18:16:31,026][105620] Updated weights for policy 1, policy_version 400213 (0.0006) [2023-12-26 18:16:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 204849152. Throughput: 0: 10011.2, 1: 9685.2. Samples: 204823708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:16:31,062][104569] Avg episode reward: [(0, '7155.638'), (1, '8272.594')] [2023-12-26 18:16:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000399912_102391808.pth... [2023-12-26 18:16:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000398760_102096896.pth [2023-12-26 18:16:31,086][105620] Updated weights for policy 1, policy_version 400223 (0.0007) [2023-12-26 18:16:31,089][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000400224_102465536.pth... [2023-12-26 18:16:31,092][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000399072_102170624.pth [2023-12-26 18:16:31,196][105692] Updated weights for policy 0, policy_version 399918 (0.0007) [2023-12-26 18:16:31,262][105692] Updated weights for policy 0, policy_version 399928 (0.0005) [2023-12-26 18:16:31,316][105692] Updated weights for policy 0, policy_version 399938 (0.0006) [2023-12-26 18:16:31,689][105620] Updated weights for policy 1, policy_version 400233 (0.0009) [2023-12-26 18:16:31,754][105620] Updated weights for policy 1, policy_version 400243 (0.0009) [2023-12-26 18:16:31,815][105620] Updated weights for policy 1, policy_version 400253 (0.0009) [2023-12-26 18:16:32,076][105692] Updated weights for policy 0, policy_version 399948 (0.0007) [2023-12-26 18:16:32,139][105692] Updated weights for policy 0, policy_version 399958 (0.0006) [2023-12-26 18:16:32,197][105692] Updated weights for policy 0, policy_version 399968 (0.0005) [2023-12-26 18:16:32,645][105620] Updated weights for policy 1, policy_version 400263 (0.0009) [2023-12-26 18:16:32,703][105620] Updated weights for policy 1, policy_version 400273 (0.0009) [2023-12-26 18:16:32,761][105620] Updated weights for policy 1, policy_version 400283 (0.0009) [2023-12-26 18:16:32,810][105692] Updated weights for policy 0, policy_version 399978 (0.0007) [2023-12-26 18:16:32,871][105692] Updated weights for policy 0, policy_version 399988 (0.0009) [2023-12-26 18:16:32,933][105692] Updated weights for policy 0, policy_version 399998 (0.0009) [2023-12-26 18:16:32,979][105692] Updated weights for policy 0, policy_version 400008 (0.0008) [2023-12-26 18:16:33,560][105620] Updated weights for policy 1, policy_version 400293 (0.0009) [2023-12-26 18:16:33,614][105620] Updated weights for policy 1, policy_version 400303 (0.0006) [2023-12-26 18:16:33,625][105692] Updated weights for policy 0, policy_version 400018 (0.0007) [2023-12-26 18:16:33,659][105620] Updated weights for policy 1, policy_version 400313 (0.0005) [2023-12-26 18:16:33,676][105692] Updated weights for policy 0, policy_version 400028 (0.0008) [2023-12-26 18:16:33,730][105692] Updated weights for policy 0, policy_version 400038 (0.0010) [2023-12-26 18:16:34,347][105620] Updated weights for policy 1, policy_version 400323 (0.0006) [2023-12-26 18:16:34,414][105620] Updated weights for policy 1, policy_version 400333 (0.0009) [2023-12-26 18:16:34,477][105620] Updated weights for policy 1, policy_version 400343 (0.0008) [2023-12-26 18:16:34,514][105692] Updated weights for policy 0, policy_version 400048 (0.0010) [2023-12-26 18:16:34,565][105692] Updated weights for policy 0, policy_version 400059 (0.0008) [2023-12-26 18:16:34,617][105692] Updated weights for policy 0, policy_version 400069 (0.0008) [2023-12-26 18:16:35,241][105620] Updated weights for policy 1, policy_version 400353 (0.0008) [2023-12-26 18:16:35,303][105620] Updated weights for policy 1, policy_version 400363 (0.0008) [2023-12-26 18:16:35,357][105692] Updated weights for policy 0, policy_version 400079 (0.0007) [2023-12-26 18:16:35,361][105620] Updated weights for policy 1, policy_version 400374 (0.0010) [2023-12-26 18:16:35,408][105692] Updated weights for policy 0, policy_version 400089 (0.0007) [2023-12-26 18:16:35,409][105620] Updated weights for policy 1, policy_version 400384 (0.0006) [2023-12-26 18:16:35,451][105585] KL-divergence is very high: 111.4006 [2023-12-26 18:16:35,459][105692] Updated weights for policy 0, policy_version 400099 (0.0009) [2023-12-26 18:16:35,471][105585] KL-divergence is very high: 104.4592 [2023-12-26 18:16:36,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 204947456. Throughput: 0: 9958.0, 1: 9756.2. Samples: 204940668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:16:36,063][104569] Avg episode reward: [(0, '7797.808'), (1, '8372.168')] [2023-12-26 18:16:36,172][105620] Updated weights for policy 1, policy_version 400394 (0.0009) [2023-12-26 18:16:36,173][105692] Updated weights for policy 0, policy_version 400109 (0.0008) [2023-12-26 18:16:36,231][105692] Updated weights for policy 0, policy_version 400119 (0.0006) [2023-12-26 18:16:36,232][105620] Updated weights for policy 1, policy_version 400404 (0.0010) [2023-12-26 18:16:36,283][105692] Updated weights for policy 0, policy_version 400129 (0.0008) [2023-12-26 18:16:36,294][105620] Updated weights for policy 1, policy_version 400414 (0.0009) [2023-12-26 18:16:36,963][105692] Updated weights for policy 0, policy_version 400139 (0.0008) [2023-12-26 18:16:37,019][105620] Updated weights for policy 1, policy_version 400424 (0.0010) [2023-12-26 18:16:37,025][105692] Updated weights for policy 0, policy_version 400149 (0.0011) [2023-12-26 18:16:37,081][105620] Updated weights for policy 1, policy_version 400434 (0.0010) [2023-12-26 18:16:37,087][105692] Updated weights for policy 0, policy_version 400159 (0.0010) [2023-12-26 18:16:37,139][105620] Updated weights for policy 1, policy_version 400444 (0.0009) [2023-12-26 18:16:37,830][105692] Updated weights for policy 0, policy_version 400169 (0.0008) [2023-12-26 18:16:37,873][105620] Updated weights for policy 1, policy_version 400454 (0.0010) [2023-12-26 18:16:37,891][105692] Updated weights for policy 0, policy_version 400179 (0.0008) [2023-12-26 18:16:37,918][105620] Updated weights for policy 1, policy_version 400464 (0.0010) [2023-12-26 18:16:37,943][105692] Updated weights for policy 0, policy_version 400189 (0.0007) [2023-12-26 18:16:37,962][105620] Updated weights for policy 1, policy_version 400474 (0.0010) [2023-12-26 18:16:37,995][105692] Updated weights for policy 0, policy_version 400199 (0.0007) [2023-12-26 18:16:38,752][105620] Updated weights for policy 1, policy_version 400484 (0.0010) [2023-12-26 18:16:38,767][105692] Updated weights for policy 0, policy_version 400209 (0.0006) [2023-12-26 18:16:38,811][105620] Updated weights for policy 1, policy_version 400494 (0.0010) [2023-12-26 18:16:38,825][105692] Updated weights for policy 0, policy_version 400219 (0.0005) [2023-12-26 18:16:38,871][105620] Updated weights for policy 1, policy_version 400504 (0.0010) [2023-12-26 18:16:38,885][105692] Updated weights for policy 0, policy_version 400229 (0.0006) [2023-12-26 18:16:39,657][105620] Updated weights for policy 1, policy_version 400514 (0.0011) [2023-12-26 18:16:39,683][105692] Updated weights for policy 0, policy_version 400239 (0.0007) [2023-12-26 18:16:39,720][105620] Updated weights for policy 1, policy_version 400524 (0.0008) [2023-12-26 18:16:39,725][105585] KL-divergence is very high: 146.1269 [2023-12-26 18:16:39,751][105692] Updated weights for policy 0, policy_version 400249 (0.0007) [2023-12-26 18:16:39,778][105585] KL-divergence is very high: 132.0384 [2023-12-26 18:16:39,780][105620] Updated weights for policy 1, policy_version 400534 (0.0009) [2023-12-26 18:16:39,795][105585] KL-divergence is very high: 216.5185 [2023-12-26 18:16:39,822][105692] Updated weights for policy 0, policy_version 400259 (0.0008) [2023-12-26 18:16:39,835][105585] KL-divergence is very high: 159.9778 [2023-12-26 18:16:39,848][105620] Updated weights for policy 1, policy_version 400544 (0.0010) [2023-12-26 18:16:39,849][105585] KL-divergence is very high: 251.9433 [2023-12-26 18:16:40,540][105620] Updated weights for policy 1, policy_version 400554 (0.0010) [2023-12-26 18:16:40,579][105585] KL-divergence is very high: 265.0283 [2023-12-26 18:16:40,597][105620] Updated weights for policy 1, policy_version 400564 (0.0010) [2023-12-26 18:16:40,602][105586] KL-divergence is very high: 117.2770 [2023-12-26 18:16:40,609][105692] Updated weights for policy 0, policy_version 400269 (0.0009) [2023-12-26 18:16:40,625][105585] KL-divergence is very high: 243.1383 [2023-12-26 18:16:40,652][105586] KL-divergence is very high: 110.3322 [2023-12-26 18:16:40,659][105620] Updated weights for policy 1, policy_version 400574 (0.0010) [2023-12-26 18:16:40,666][105692] Updated weights for policy 0, policy_version 400279 (0.0006) [2023-12-26 18:16:40,672][105585] KL-divergence is very high: 187.6523 [2023-12-26 18:16:40,715][105585] KL-divergence is very high: 161.0834 [2023-12-26 18:16:40,720][105692] Updated weights for policy 0, policy_version 400289 (0.0008) [2023-12-26 18:16:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 205045760. Throughput: 0: 9872.1, 1: 9762.2. Samples: 205053324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:16:41,063][104569] Avg episode reward: [(0, '7890.499'), (1, '8372.075')] [2023-12-26 18:16:41,401][105586] KL-divergence is very high: 116.2466 [2023-12-26 18:16:41,419][105620] Updated weights for policy 1, policy_version 400584 (0.0008) [2023-12-26 18:16:41,482][105620] Updated weights for policy 1, policy_version 400594 (0.0009) [2023-12-26 18:16:41,513][105692] Updated weights for policy 0, policy_version 400299 (0.0008) [2023-12-26 18:16:41,540][105620] Updated weights for policy 1, policy_version 400604 (0.0007) [2023-12-26 18:16:41,570][105692] Updated weights for policy 0, policy_version 400309 (0.0009) [2023-12-26 18:16:41,630][105692] Updated weights for policy 0, policy_version 400319 (0.0009) [2023-12-26 18:16:42,303][105620] Updated weights for policy 1, policy_version 400614 (0.0006) [2023-12-26 18:16:42,366][105620] Updated weights for policy 1, policy_version 400624 (0.0008) [2023-12-26 18:16:42,432][105620] Updated weights for policy 1, policy_version 400634 (0.0009) [2023-12-26 18:16:42,435][105692] Updated weights for policy 0, policy_version 400329 (0.0007) [2023-12-26 18:16:42,493][105692] Updated weights for policy 0, policy_version 400339 (0.0008) [2023-12-26 18:16:42,494][105585] KL-divergence is very high: 180.9832 [2023-12-26 18:16:42,500][105585] KL-divergence is very high: 147.4985 [2023-12-26 18:16:42,518][105585] KL-divergence is very high: 133.9978 [2023-12-26 18:16:42,542][105585] KL-divergence is very high: 168.1740 [2023-12-26 18:16:42,549][105585] KL-divergence is very high: 114.6834 [2023-12-26 18:16:42,555][105692] Updated weights for policy 0, policy_version 400349 (0.0010) [2023-12-26 18:16:42,618][105692] Updated weights for policy 0, policy_version 400359 (0.0011) [2023-12-26 18:16:43,063][105620] Updated weights for policy 1, policy_version 400644 (0.0005) [2023-12-26 18:16:43,138][105620] Updated weights for policy 1, policy_version 400654 (0.0008) [2023-12-26 18:16:43,203][105620] Updated weights for policy 1, policy_version 400664 (0.0009) [2023-12-26 18:16:43,244][105692] Updated weights for policy 0, policy_version 400369 (0.0006) [2023-12-26 18:16:43,307][105692] Updated weights for policy 0, policy_version 400379 (0.0005) [2023-12-26 18:16:43,368][105692] Updated weights for policy 0, policy_version 400389 (0.0005) [2023-12-26 18:16:43,854][105620] Updated weights for policy 1, policy_version 400674 (0.0008) [2023-12-26 18:16:43,893][105692] Updated weights for policy 0, policy_version 400399 (0.0007) [2023-12-26 18:16:43,907][105620] Updated weights for policy 1, policy_version 400684 (0.0010) [2023-12-26 18:16:43,948][105692] Updated weights for policy 0, policy_version 400409 (0.0007) [2023-12-26 18:16:43,967][105620] Updated weights for policy 1, policy_version 400694 (0.0009) [2023-12-26 18:16:43,997][105692] Updated weights for policy 0, policy_version 400419 (0.0006) [2023-12-26 18:16:44,018][105620] Updated weights for policy 1, policy_version 400704 (0.0010) [2023-12-26 18:16:44,625][105620] Updated weights for policy 1, policy_version 400714 (0.0008) [2023-12-26 18:16:44,683][105620] Updated weights for policy 1, policy_version 400724 (0.0009) [2023-12-26 18:16:44,747][105620] Updated weights for policy 1, policy_version 400734 (0.0008) [2023-12-26 18:16:44,754][105692] Updated weights for policy 0, policy_version 400429 (0.0007) [2023-12-26 18:16:44,821][105692] Updated weights for policy 0, policy_version 400439 (0.0009) [2023-12-26 18:16:44,888][105692] Updated weights for policy 0, policy_version 400449 (0.0010) [2023-12-26 18:16:45,404][105620] Updated weights for policy 1, policy_version 400744 (0.0008) [2023-12-26 18:16:45,466][105620] Updated weights for policy 1, policy_version 400754 (0.0010) [2023-12-26 18:16:45,535][105620] Updated weights for policy 1, policy_version 400764 (0.0010) [2023-12-26 18:16:45,551][105692] Updated weights for policy 0, policy_version 400460 (0.0009) [2023-12-26 18:16:45,606][105692] Updated weights for policy 0, policy_version 400470 (0.0010) [2023-12-26 18:16:45,655][105692] Updated weights for policy 0, policy_version 400480 (0.0010) [2023-12-26 18:16:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 205144064. Throughput: 0: 9851.4, 1: 9720.7. Samples: 205112464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:16:46,063][104569] Avg episode reward: [(0, '7522.129'), (1, '8369.485')] [2023-12-26 18:16:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000400488_102539264.pth... [2023-12-26 18:16:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000400768_102604800.pth... [2023-12-26 18:16:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000399336_102244352.pth [2023-12-26 18:16:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000399648_102318080.pth [2023-12-26 18:16:46,172][105620] Updated weights for policy 1, policy_version 400774 (0.0010) [2023-12-26 18:16:46,232][105692] Updated weights for policy 0, policy_version 400490 (0.0007) [2023-12-26 18:16:46,240][105620] Updated weights for policy 1, policy_version 400784 (0.0010) [2023-12-26 18:16:46,291][105620] Updated weights for policy 1, policy_version 400794 (0.0010) [2023-12-26 18:16:46,293][105692] Updated weights for policy 0, policy_version 400500 (0.0010) [2023-12-26 18:16:46,352][105692] Updated weights for policy 0, policy_version 400510 (0.0009) [2023-12-26 18:16:46,411][105692] Updated weights for policy 0, policy_version 400520 (0.0008) [2023-12-26 18:16:46,967][105620] Updated weights for policy 1, policy_version 400804 (0.0010) [2023-12-26 18:16:47,022][105620] Updated weights for policy 1, policy_version 400814 (0.0010) [2023-12-26 18:16:47,033][105692] Updated weights for policy 0, policy_version 400530 (0.0010) [2023-12-26 18:16:47,083][105620] Updated weights for policy 1, policy_version 400824 (0.0010) [2023-12-26 18:16:47,092][105692] Updated weights for policy 0, policy_version 400540 (0.0008) [2023-12-26 18:16:47,154][105692] Updated weights for policy 0, policy_version 400550 (0.0005) [2023-12-26 18:16:47,646][105620] Updated weights for policy 1, policy_version 400834 (0.0009) [2023-12-26 18:16:47,713][105620] Updated weights for policy 1, policy_version 400844 (0.0005) [2023-12-26 18:16:47,780][105620] Updated weights for policy 1, policy_version 400854 (0.0006) [2023-12-26 18:16:47,782][105692] Updated weights for policy 0, policy_version 400560 (0.0008) [2023-12-26 18:16:47,832][105692] Updated weights for policy 0, policy_version 400570 (0.0007) [2023-12-26 18:16:47,833][105620] Updated weights for policy 1, policy_version 400864 (0.0005) [2023-12-26 18:16:47,877][105692] Updated weights for policy 0, policy_version 400580 (0.0010) [2023-12-26 18:16:48,377][105620] Updated weights for policy 1, policy_version 400874 (0.0011) [2023-12-26 18:16:48,429][105620] Updated weights for policy 1, policy_version 400884 (0.0010) [2023-12-26 18:16:48,491][105620] Updated weights for policy 1, policy_version 400894 (0.0010) [2023-12-26 18:16:48,644][105692] Updated weights for policy 0, policy_version 400590 (0.0007) [2023-12-26 18:16:48,696][105692] Updated weights for policy 0, policy_version 400600 (0.0005) [2023-12-26 18:16:48,746][105692] Updated weights for policy 0, policy_version 400610 (0.0010) [2023-12-26 18:16:49,236][105620] Updated weights for policy 1, policy_version 400904 (0.0010) [2023-12-26 18:16:49,301][105620] Updated weights for policy 1, policy_version 400914 (0.0009) [2023-12-26 18:16:49,358][105692] Updated weights for policy 0, policy_version 400620 (0.0010) [2023-12-26 18:16:49,374][105620] Updated weights for policy 1, policy_version 400924 (0.0008) [2023-12-26 18:16:49,422][105692] Updated weights for policy 0, policy_version 400630 (0.0006) [2023-12-26 18:16:49,493][105692] Updated weights for policy 0, policy_version 400640 (0.0005) [2023-12-26 18:16:50,089][105692] Updated weights for policy 0, policy_version 400650 (0.0006) [2023-12-26 18:16:50,154][105692] Updated weights for policy 0, policy_version 400660 (0.0007) [2023-12-26 18:16:50,157][105620] Updated weights for policy 1, policy_version 400934 (0.0006) [2023-12-26 18:16:50,215][105692] Updated weights for policy 0, policy_version 400670 (0.0007) [2023-12-26 18:16:50,219][105620] Updated weights for policy 1, policy_version 400944 (0.0006) [2023-12-26 18:16:50,277][105620] Updated weights for policy 1, policy_version 400954 (0.0005) [2023-12-26 18:16:50,284][105692] Updated weights for policy 0, policy_version 400680 (0.0010) [2023-12-26 18:16:50,961][105620] Updated weights for policy 1, policy_version 400964 (0.0009) [2023-12-26 18:16:51,012][105692] Updated weights for policy 0, policy_version 400690 (0.0010) [2023-12-26 18:16:51,024][105620] Updated weights for policy 1, policy_version 400974 (0.0008) [2023-12-26 18:16:51,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 205242368. Throughput: 0: 9971.2, 1: 9795.2. Samples: 205239124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:16:51,062][104569] Avg episode reward: [(0, '7978.673'), (1, '8017.517')] [2023-12-26 18:16:51,076][105692] Updated weights for policy 0, policy_version 400700 (0.0011) [2023-12-26 18:16:51,091][105620] Updated weights for policy 1, policy_version 400984 (0.0007) [2023-12-26 18:16:51,140][105692] Updated weights for policy 0, policy_version 400710 (0.0011) [2023-12-26 18:16:51,824][105620] Updated weights for policy 1, policy_version 400994 (0.0008) [2023-12-26 18:16:51,880][105620] Updated weights for policy 1, policy_version 401004 (0.0011) [2023-12-26 18:16:51,919][105692] Updated weights for policy 0, policy_version 400720 (0.0008) [2023-12-26 18:16:51,936][105620] Updated weights for policy 1, policy_version 401014 (0.0010) [2023-12-26 18:16:51,983][105692] Updated weights for policy 0, policy_version 400730 (0.0010) [2023-12-26 18:16:51,991][105620] Updated weights for policy 1, policy_version 401024 (0.0008) [2023-12-26 18:16:52,046][105692] Updated weights for policy 0, policy_version 400740 (0.0011) [2023-12-26 18:16:52,707][105692] Updated weights for policy 0, policy_version 400750 (0.0008) [2023-12-26 18:16:52,720][105620] Updated weights for policy 1, policy_version 401034 (0.0010) [2023-12-26 18:16:52,751][105585] KL-divergence is very high: 125.3305 [2023-12-26 18:16:52,759][105585] KL-divergence is very high: 113.9830 [2023-12-26 18:16:52,778][105692] Updated weights for policy 0, policy_version 400760 (0.0006) [2023-12-26 18:16:52,783][105620] Updated weights for policy 1, policy_version 401044 (0.0011) [2023-12-26 18:16:52,800][105585] KL-divergence is very high: 202.5346 [2023-12-26 18:16:52,805][105585] KL-divergence is very high: 149.6739 [2023-12-26 18:16:52,838][105692] Updated weights for policy 0, policy_version 400770 (0.0005) [2023-12-26 18:16:52,846][105620] Updated weights for policy 1, policy_version 401054 (0.0011) [2023-12-26 18:16:52,852][105585] KL-divergence is very high: 169.0755 [2023-12-26 18:16:52,860][105585] KL-divergence is very high: 110.2788 [2023-12-26 18:16:53,392][105585] KL-divergence is very high: 111.9037 [2023-12-26 18:16:53,417][105692] Updated weights for policy 0, policy_version 400780 (0.0007) [2023-12-26 18:16:53,484][105692] Updated weights for policy 0, policy_version 400790 (0.0008) [2023-12-26 18:16:53,491][105620] Updated weights for policy 1, policy_version 401064 (0.0010) [2023-12-26 18:16:53,535][105620] Updated weights for policy 1, policy_version 401074 (0.0010) [2023-12-26 18:16:53,545][105692] Updated weights for policy 0, policy_version 400800 (0.0008) [2023-12-26 18:16:53,597][105620] Updated weights for policy 1, policy_version 401084 (0.0010) [2023-12-26 18:16:54,120][105692] Updated weights for policy 0, policy_version 400810 (0.0008) [2023-12-26 18:16:54,186][105692] Updated weights for policy 0, policy_version 400820 (0.0008) [2023-12-26 18:16:54,259][105692] Updated weights for policy 0, policy_version 400830 (0.0009) [2023-12-26 18:16:54,313][105692] Updated weights for policy 0, policy_version 400840 (0.0007) [2023-12-26 18:16:54,347][105620] Updated weights for policy 1, policy_version 401094 (0.0010) [2023-12-26 18:16:54,408][105620] Updated weights for policy 1, policy_version 401104 (0.0009) [2023-12-26 18:16:54,466][105620] Updated weights for policy 1, policy_version 401114 (0.0006) [2023-12-26 18:16:54,898][105692] Updated weights for policy 0, policy_version 400850 (0.0005) [2023-12-26 18:16:54,970][105692] Updated weights for policy 0, policy_version 400860 (0.0011) [2023-12-26 18:16:55,035][105692] Updated weights for policy 0, policy_version 400870 (0.0011) [2023-12-26 18:16:55,137][105620] Updated weights for policy 1, policy_version 401124 (0.0009) [2023-12-26 18:16:55,192][105620] Updated weights for policy 1, policy_version 401134 (0.0010) [2023-12-26 18:16:55,258][105620] Updated weights for policy 1, policy_version 401144 (0.0011) [2023-12-26 18:16:55,741][105692] Updated weights for policy 0, policy_version 400880 (0.0007) [2023-12-26 18:16:55,798][105692] Updated weights for policy 0, policy_version 400890 (0.0005) [2023-12-26 18:16:55,850][105620] Updated weights for policy 1, policy_version 401154 (0.0007) [2023-12-26 18:16:55,857][105692] Updated weights for policy 0, policy_version 400900 (0.0007) [2023-12-26 18:16:55,898][105620] Updated weights for policy 1, policy_version 401164 (0.0010) [2023-12-26 18:16:55,954][105620] Updated weights for policy 1, policy_version 401174 (0.0010) [2023-12-26 18:16:56,022][105620] Updated weights for policy 1, policy_version 401184 (0.0010) [2023-12-26 18:16:56,062][104569] Fps is (10 sec: 21299.3, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 205357056. Throughput: 0: 9997.0, 1: 9813.6. Samples: 205360704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:16:56,063][104569] Avg episode reward: [(0, '8623.952'), (1, '8297.465')] [2023-12-26 18:16:56,439][105692] Updated weights for policy 0, policy_version 400910 (0.0007) [2023-12-26 18:16:56,504][105692] Updated weights for policy 0, policy_version 400920 (0.0005) [2023-12-26 18:16:56,556][105692] Updated weights for policy 0, policy_version 400930 (0.0005) [2023-12-26 18:16:56,672][105620] Updated weights for policy 1, policy_version 401194 (0.0005) [2023-12-26 18:16:56,738][105620] Updated weights for policy 1, policy_version 401204 (0.0007) [2023-12-26 18:16:56,799][105620] Updated weights for policy 1, policy_version 401214 (0.0010) [2023-12-26 18:16:57,094][105692] Updated weights for policy 0, policy_version 400940 (0.0008) [2023-12-26 18:16:57,151][105692] Updated weights for policy 0, policy_version 400950 (0.0010) [2023-12-26 18:16:57,215][105692] Updated weights for policy 0, policy_version 400960 (0.0010) [2023-12-26 18:16:57,474][105620] Updated weights for policy 1, policy_version 401224 (0.0010) [2023-12-26 18:16:57,529][105620] Updated weights for policy 1, policy_version 401234 (0.0010) [2023-12-26 18:16:57,582][105620] Updated weights for policy 1, policy_version 401244 (0.0007) [2023-12-26 18:16:57,784][105692] Updated weights for policy 0, policy_version 400970 (0.0010) [2023-12-26 18:16:57,839][105692] Updated weights for policy 0, policy_version 400980 (0.0005) [2023-12-26 18:16:57,885][105692] Updated weights for policy 0, policy_version 400990 (0.0006) [2023-12-26 18:16:57,932][105692] Updated weights for policy 0, policy_version 401000 (0.0005) [2023-12-26 18:16:58,245][105620] Updated weights for policy 1, policy_version 401254 (0.0009) [2023-12-26 18:16:58,300][105620] Updated weights for policy 1, policy_version 401264 (0.0010) [2023-12-26 18:16:58,367][105620] Updated weights for policy 1, policy_version 401274 (0.0008) [2023-12-26 18:16:58,642][105692] Updated weights for policy 0, policy_version 401010 (0.0008) [2023-12-26 18:16:58,714][105692] Updated weights for policy 0, policy_version 401020 (0.0009) [2023-12-26 18:16:58,789][105692] Updated weights for policy 0, policy_version 401030 (0.0008) [2023-12-26 18:16:59,150][105620] Updated weights for policy 1, policy_version 401284 (0.0008) [2023-12-26 18:16:59,218][105620] Updated weights for policy 1, policy_version 401294 (0.0008) [2023-12-26 18:16:59,286][105620] Updated weights for policy 1, policy_version 401304 (0.0008) [2023-12-26 18:16:59,591][105692] Updated weights for policy 0, policy_version 401040 (0.0009) [2023-12-26 18:16:59,643][105692] Updated weights for policy 0, policy_version 401050 (0.0009) [2023-12-26 18:16:59,707][105692] Updated weights for policy 0, policy_version 401060 (0.0010) [2023-12-26 18:17:00,027][105620] Updated weights for policy 1, policy_version 401314 (0.0008) [2023-12-26 18:17:00,090][105620] Updated weights for policy 1, policy_version 401324 (0.0007) [2023-12-26 18:17:00,152][105620] Updated weights for policy 1, policy_version 401334 (0.0007) [2023-12-26 18:17:00,200][105620] Updated weights for policy 1, policy_version 401344 (0.0008) [2023-12-26 18:17:00,474][105692] Updated weights for policy 0, policy_version 401070 (0.0009) [2023-12-26 18:17:00,529][105692] Updated weights for policy 0, policy_version 401080 (0.0005) [2023-12-26 18:17:00,575][105692] Updated weights for policy 0, policy_version 401090 (0.0005) [2023-12-26 18:17:00,971][105620] Updated weights for policy 1, policy_version 401354 (0.0008) [2023-12-26 18:17:01,015][105620] Updated weights for policy 1, policy_version 401364 (0.0008) [2023-12-26 18:17:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 205447168. Throughput: 0: 10161.7, 1: 9846.4. Samples: 205425008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:17:01,062][104569] Avg episode reward: [(0, '5919.781'), (1, '3522.812')] [2023-12-26 18:17:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000401096_102694912.pth... [2023-12-26 18:17:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000399912_102391808.pth [2023-12-26 18:17:01,079][105620] Updated weights for policy 1, policy_version 401374 (0.0008) [2023-12-26 18:17:01,088][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000401376_102760448.pth... [2023-12-26 18:17:01,092][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000400224_102465536.pth [2023-12-26 18:17:01,260][105692] Updated weights for policy 0, policy_version 401100 (0.0007) [2023-12-26 18:17:01,316][105692] Updated weights for policy 0, policy_version 401110 (0.0009) [2023-12-26 18:17:01,386][105692] Updated weights for policy 0, policy_version 401120 (0.0007) [2023-12-26 18:17:01,925][105620] Updated weights for policy 1, policy_version 401384 (0.0008) [2023-12-26 18:17:01,984][105620] Updated weights for policy 1, policy_version 401394 (0.0008) [2023-12-26 18:17:02,030][105692] Updated weights for policy 0, policy_version 401130 (0.0010) [2023-12-26 18:17:02,042][105620] Updated weights for policy 1, policy_version 401404 (0.0008) [2023-12-26 18:17:02,078][105692] Updated weights for policy 0, policy_version 401140 (0.0010) [2023-12-26 18:17:02,134][105692] Updated weights for policy 0, policy_version 401150 (0.0010) [2023-12-26 18:17:02,190][105692] Updated weights for policy 0, policy_version 401160 (0.0010) [2023-12-26 18:17:02,853][105692] Updated weights for policy 0, policy_version 401170 (0.0005) [2023-12-26 18:17:02,868][105620] Updated weights for policy 1, policy_version 401414 (0.0007) [2023-12-26 18:17:02,897][105585] KL-divergence is very high: 104.3267 [2023-12-26 18:17:02,908][105692] Updated weights for policy 0, policy_version 401180 (0.0005) [2023-12-26 18:17:02,929][105620] Updated weights for policy 1, policy_version 401424 (0.0006) [2023-12-26 18:17:02,959][105692] Updated weights for policy 0, policy_version 401190 (0.0005) [2023-12-26 18:17:02,980][105620] Updated weights for policy 1, policy_version 401434 (0.0008) [2023-12-26 18:17:03,504][105585] KL-divergence is very high: 108.6233 [2023-12-26 18:17:03,556][105692] Updated weights for policy 0, policy_version 401200 (0.0005) [2023-12-26 18:17:03,558][105585] KL-divergence is very high: 227.9964 [2023-12-26 18:17:03,602][105585] KL-divergence is very high: 352.5618 [2023-12-26 18:17:03,614][105692] Updated weights for policy 0, policy_version 401210 (0.0006) [2023-12-26 18:17:03,650][105585] KL-divergence is very high: 301.8419 [2023-12-26 18:17:03,672][105692] Updated weights for policy 0, policy_version 401220 (0.0005) [2023-12-26 18:17:03,709][105586] KL-divergence is very high: 129.4352 [2023-12-26 18:17:03,715][105620] Updated weights for policy 1, policy_version 401444 (0.0010) [2023-12-26 18:17:03,761][105620] Updated weights for policy 1, policy_version 401454 (0.0005) [2023-12-26 18:17:03,810][105620] Updated weights for policy 1, policy_version 401464 (0.0005) [2023-12-26 18:17:03,810][105586] KL-divergence is very high: 100.8878 [2023-12-26 18:17:03,815][105586] KL-divergence is very high: 111.7368 [2023-12-26 18:17:03,825][105586] KL-divergence is very high: 124.6276 [2023-12-26 18:17:04,276][105692] Updated weights for policy 0, policy_version 401230 (0.0006) [2023-12-26 18:17:04,329][105585] KL-divergence is very high: 116.0196 [2023-12-26 18:17:04,334][105692] Updated weights for policy 0, policy_version 401240 (0.0006) [2023-12-26 18:17:04,375][105585] KL-divergence is very high: 197.5291 [2023-12-26 18:17:04,392][105692] Updated weights for policy 0, policy_version 401250 (0.0010) [2023-12-26 18:17:04,404][105585] KL-divergence is very high: 101.7569 [2023-12-26 18:17:04,422][105585] KL-divergence is very high: 179.8218 [2023-12-26 18:17:04,471][105620] Updated weights for policy 1, policy_version 401474 (0.0007) [2023-12-26 18:17:04,516][105620] Updated weights for policy 1, policy_version 401484 (0.0008) [2023-12-26 18:17:04,536][105586] KL-divergence is very high: 116.4508 [2023-12-26 18:17:04,541][105586] KL-divergence is very high: 118.2448 [2023-12-26 18:17:04,547][105586] KL-divergence is very high: 114.8282 [2023-12-26 18:17:04,567][105620] Updated weights for policy 1, policy_version 401494 (0.0005) [2023-12-26 18:17:04,621][105620] Updated weights for policy 1, policy_version 401504 (0.0009) [2023-12-26 18:17:05,085][105692] Updated weights for policy 0, policy_version 401260 (0.0010) [2023-12-26 18:17:05,133][105692] Updated weights for policy 0, policy_version 401270 (0.0010) [2023-12-26 18:17:05,199][105692] Updated weights for policy 0, policy_version 401280 (0.0010) [2023-12-26 18:17:05,224][105620] Updated weights for policy 1, policy_version 401514 (0.0005) [2023-12-26 18:17:05,273][105620] Updated weights for policy 1, policy_version 401524 (0.0005) [2023-12-26 18:17:05,332][105620] Updated weights for policy 1, policy_version 401534 (0.0005) [2023-12-26 18:17:05,860][105692] Updated weights for policy 0, policy_version 401290 (0.0010) [2023-12-26 18:17:05,927][105692] Updated weights for policy 0, policy_version 401300 (0.0009) [2023-12-26 18:17:05,928][105620] Updated weights for policy 1, policy_version 401544 (0.0006) [2023-12-26 18:17:05,985][105620] Updated weights for policy 1, policy_version 401554 (0.0005) [2023-12-26 18:17:05,988][105692] Updated weights for policy 0, policy_version 401310 (0.0010) [2023-12-26 18:17:06,046][105620] Updated weights for policy 1, policy_version 401564 (0.0005) [2023-12-26 18:17:06,049][105692] Updated weights for policy 0, policy_version 401320 (0.0010) [2023-12-26 18:17:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 205553664. Throughput: 0: 10079.4, 1: 9715.9. Samples: 205541936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:17:06,062][104569] Avg episode reward: [(0, '4015.783'), (1, '2081.055')] [2023-12-26 18:17:06,718][105620] Updated weights for policy 1, policy_version 401574 (0.0008) [2023-12-26 18:17:06,778][105620] Updated weights for policy 1, policy_version 401584 (0.0011) [2023-12-26 18:17:06,796][105692] Updated weights for policy 0, policy_version 401330 (0.0010) [2023-12-26 18:17:06,841][105620] Updated weights for policy 1, policy_version 401594 (0.0011) [2023-12-26 18:17:06,859][105692] Updated weights for policy 0, policy_version 401340 (0.0005) [2023-12-26 18:17:06,921][105692] Updated weights for policy 0, policy_version 401350 (0.0010) [2023-12-26 18:17:07,558][105620] Updated weights for policy 1, policy_version 401604 (0.0011) [2023-12-26 18:17:07,604][105692] Updated weights for policy 0, policy_version 401360 (0.0007) [2023-12-26 18:17:07,614][105620] Updated weights for policy 1, policy_version 401614 (0.0011) [2023-12-26 18:17:07,664][105620] Updated weights for policy 1, policy_version 401624 (0.0010) [2023-12-26 18:17:07,664][105692] Updated weights for policy 0, policy_version 401370 (0.0008) [2023-12-26 18:17:07,728][105692] Updated weights for policy 0, policy_version 401380 (0.0009) [2023-12-26 18:17:08,210][105620] Updated weights for policy 1, policy_version 401634 (0.0006) [2023-12-26 18:17:08,268][105620] Updated weights for policy 1, policy_version 401644 (0.0005) [2023-12-26 18:17:08,327][105620] Updated weights for policy 1, policy_version 401654 (0.0007) [2023-12-26 18:17:08,385][105692] Updated weights for policy 0, policy_version 401390 (0.0010) [2023-12-26 18:17:08,386][105620] Updated weights for policy 1, policy_version 401664 (0.0007) [2023-12-26 18:17:08,455][105692] Updated weights for policy 0, policy_version 401400 (0.0009) [2023-12-26 18:17:08,519][105692] Updated weights for policy 0, policy_version 401410 (0.0008) [2023-12-26 18:17:09,028][105620] Updated weights for policy 1, policy_version 401674 (0.0005) [2023-12-26 18:17:09,067][105692] Updated weights for policy 0, policy_version 401420 (0.0005) [2023-12-26 18:17:09,082][105620] Updated weights for policy 1, policy_version 401684 (0.0009) [2023-12-26 18:17:09,109][105585] KL-divergence is very high: 101.3745 [2023-12-26 18:17:09,124][105692] Updated weights for policy 0, policy_version 401430 (0.0005) [2023-12-26 18:17:09,141][105620] Updated weights for policy 1, policy_version 401694 (0.0007) [2023-12-26 18:17:09,181][105692] Updated weights for policy 0, policy_version 401440 (0.0005) [2023-12-26 18:17:09,827][105620] Updated weights for policy 1, policy_version 401704 (0.0009) [2023-12-26 18:17:09,893][105620] Updated weights for policy 1, policy_version 401714 (0.0010) [2023-12-26 18:17:09,903][105692] Updated weights for policy 0, policy_version 401450 (0.0008) [2023-12-26 18:17:09,911][105585] KL-divergence is very high: 153.7366 [2023-12-26 18:17:09,923][105585] KL-divergence is very high: 121.8645 [2023-12-26 18:17:09,936][105585] KL-divergence is very high: 154.9116 [2023-12-26 18:17:09,942][105585] KL-divergence is very high: 265.6237 [2023-12-26 18:17:09,952][105585] KL-divergence is very high: 361.1266 [2023-12-26 18:17:09,957][105585] KL-divergence is very high: 460.7876 [2023-12-26 18:17:09,958][105620] Updated weights for policy 1, policy_version 401724 (0.0011) [2023-12-26 18:17:09,963][105585] KL-divergence is very high: 270.7335 [2023-12-26 18:17:09,964][105692] Updated weights for policy 0, policy_version 401460 (0.0008) [2023-12-26 18:17:09,969][105585] KL-divergence is very high: 358.4611 [2023-12-26 18:17:09,981][105585] KL-divergence is very high: 190.3090 [2023-12-26 18:17:09,987][105585] KL-divergence is very high: 286.7986 [2023-12-26 18:17:10,001][105585] KL-divergence is very high: 281.5656 [2023-12-26 18:17:10,007][105585] KL-divergence is very high: 201.1821 [2023-12-26 18:17:10,014][105585] KL-divergence is very high: 129.4155 [2023-12-26 18:17:10,020][105585] KL-divergence is very high: 182.1846 [2023-12-26 18:17:10,024][105692] Updated weights for policy 0, policy_version 401470 (0.0007) [2023-12-26 18:17:10,038][105585] KL-divergence is very high: 104.6388 [2023-12-26 18:17:10,088][105692] Updated weights for policy 0, policy_version 401480 (0.0009) [2023-12-26 18:17:10,742][105620] Updated weights for policy 1, policy_version 401734 (0.0011) [2023-12-26 18:17:10,799][105620] Updated weights for policy 1, policy_version 401744 (0.0011) [2023-12-26 18:17:10,860][105620] Updated weights for policy 1, policy_version 401754 (0.0010) [2023-12-26 18:17:10,875][105692] Updated weights for policy 0, policy_version 401490 (0.0010) [2023-12-26 18:17:10,924][105585] KL-divergence is very high: 127.9408 [2023-12-26 18:17:10,941][105692] Updated weights for policy 0, policy_version 401500 (0.0011) [2023-12-26 18:17:10,972][105585] KL-divergence is very high: 102.0507 [2023-12-26 18:17:10,990][105692] Updated weights for policy 0, policy_version 401510 (0.0011) [2023-12-26 18:17:11,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19933.9, 300 sec: 19660.8). Total num frames: 205660160. Throughput: 0: 10130.0, 1: 9845.9. Samples: 205663824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:17:11,062][104569] Avg episode reward: [(0, '2998.731'), (1, '6741.545')] [2023-12-26 18:17:11,611][105620] Updated weights for policy 1, policy_version 401764 (0.0010) [2023-12-26 18:17:11,687][105620] Updated weights for policy 1, policy_version 401776 (0.0008) [2023-12-26 18:17:11,755][105620] Updated weights for policy 1, policy_version 401786 (0.0008) [2023-12-26 18:17:11,786][105692] Updated weights for policy 0, policy_version 401520 (0.0008) [2023-12-26 18:17:11,851][105692] Updated weights for policy 0, policy_version 401530 (0.0010) [2023-12-26 18:17:11,886][105585] KL-divergence is very high: 120.3700 [2023-12-26 18:17:11,922][105692] Updated weights for policy 0, policy_version 401540 (0.0008) [2023-12-26 18:17:11,940][105585] KL-divergence is very high: 135.7242 [2023-12-26 18:17:12,556][105620] Updated weights for policy 1, policy_version 401796 (0.0008) [2023-12-26 18:17:12,573][105692] Updated weights for policy 0, policy_version 401550 (0.0010) [2023-12-26 18:17:12,615][105620] Updated weights for policy 1, policy_version 401806 (0.0006) [2023-12-26 18:17:12,626][105692] Updated weights for policy 0, policy_version 401560 (0.0010) [2023-12-26 18:17:12,676][105620] Updated weights for policy 1, policy_version 401816 (0.0006) [2023-12-26 18:17:12,681][105692] Updated weights for policy 0, policy_version 401570 (0.0011) [2023-12-26 18:17:13,340][105620] Updated weights for policy 1, policy_version 401826 (0.0007) [2023-12-26 18:17:13,394][105620] Updated weights for policy 1, policy_version 401836 (0.0005) [2023-12-26 18:17:13,407][105692] Updated weights for policy 0, policy_version 401580 (0.0010) [2023-12-26 18:17:13,453][105620] Updated weights for policy 1, policy_version 401846 (0.0006) [2023-12-26 18:17:13,455][105692] Updated weights for policy 0, policy_version 401590 (0.0010) [2023-12-26 18:17:13,500][105692] Updated weights for policy 0, policy_version 401600 (0.0010) [2023-12-26 18:17:13,523][105620] Updated weights for policy 1, policy_version 401856 (0.0005) [2023-12-26 18:17:14,054][105620] Updated weights for policy 1, policy_version 401866 (0.0006) [2023-12-26 18:17:14,109][105620] Updated weights for policy 1, policy_version 401876 (0.0005) [2023-12-26 18:17:14,156][105620] Updated weights for policy 1, policy_version 401886 (0.0007) [2023-12-26 18:17:14,272][105692] Updated weights for policy 0, policy_version 401610 (0.0010) [2023-12-26 18:17:14,320][105692] Updated weights for policy 0, policy_version 401620 (0.0009) [2023-12-26 18:17:14,325][105585] KL-divergence is very high: 171.6688 [2023-12-26 18:17:14,377][105585] KL-divergence is very high: 124.3374 [2023-12-26 18:17:14,385][105692] Updated weights for policy 0, policy_version 401630 (0.0010) [2023-12-26 18:17:14,445][105692] Updated weights for policy 0, policy_version 401640 (0.0010) [2023-12-26 18:17:14,827][105620] Updated weights for policy 1, policy_version 401896 (0.0009) [2023-12-26 18:17:14,890][105620] Updated weights for policy 1, policy_version 401906 (0.0009) [2023-12-26 18:17:14,948][105620] Updated weights for policy 1, policy_version 401916 (0.0007) [2023-12-26 18:17:15,085][105692] Updated weights for policy 0, policy_version 401650 (0.0008) [2023-12-26 18:17:15,134][105692] Updated weights for policy 0, policy_version 401660 (0.0008) [2023-12-26 18:17:15,183][105692] Updated weights for policy 0, policy_version 401670 (0.0009) [2023-12-26 18:17:15,615][105620] Updated weights for policy 1, policy_version 401926 (0.0007) [2023-12-26 18:17:15,678][105620] Updated weights for policy 1, policy_version 401936 (0.0006) [2023-12-26 18:17:15,753][105620] Updated weights for policy 1, policy_version 401946 (0.0006) [2023-12-26 18:17:15,892][105692] Updated weights for policy 0, policy_version 401680 (0.0006) [2023-12-26 18:17:15,952][105692] Updated weights for policy 0, policy_version 401690 (0.0005) [2023-12-26 18:17:16,005][105692] Updated weights for policy 0, policy_version 401700 (0.0005) [2023-12-26 18:17:16,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19934.0, 300 sec: 19660.8). Total num frames: 205758464. Throughput: 0: 10062.0, 1: 9898.0. Samples: 205721912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:17:16,062][104569] Avg episode reward: [(0, '5999.674'), (1, '8819.023')] [2023-12-26 18:17:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000401952_102907904.pth... [2023-12-26 18:17:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000401704_102850560.pth... [2023-12-26 18:17:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000400768_102604800.pth [2023-12-26 18:17:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000400488_102539264.pth [2023-12-26 18:17:16,295][105620] Updated weights for policy 1, policy_version 401956 (0.0009) [2023-12-26 18:17:16,347][105620] Updated weights for policy 1, policy_version 401966 (0.0011) [2023-12-26 18:17:16,399][105620] Updated weights for policy 1, policy_version 401976 (0.0007) [2023-12-26 18:17:16,548][105692] Updated weights for policy 0, policy_version 401710 (0.0005) [2023-12-26 18:17:16,603][105692] Updated weights for policy 0, policy_version 401720 (0.0008) [2023-12-26 18:17:16,647][105692] Updated weights for policy 0, policy_version 401730 (0.0008) [2023-12-26 18:17:17,057][105620] Updated weights for policy 1, policy_version 401986 (0.0005) [2023-12-26 18:17:17,111][105620] Updated weights for policy 1, policy_version 401996 (0.0005) [2023-12-26 18:17:17,166][105620] Updated weights for policy 1, policy_version 402006 (0.0005) [2023-12-26 18:17:17,217][105620] Updated weights for policy 1, policy_version 402016 (0.0005) [2023-12-26 18:17:17,286][105692] Updated weights for policy 0, policy_version 401740 (0.0007) [2023-12-26 18:17:17,355][105692] Updated weights for policy 0, policy_version 401750 (0.0005) [2023-12-26 18:17:17,418][105692] Updated weights for policy 0, policy_version 401760 (0.0005) [2023-12-26 18:17:17,778][105620] Updated weights for policy 1, policy_version 402026 (0.0010) [2023-12-26 18:17:17,831][105620] Updated weights for policy 1, policy_version 402036 (0.0008) [2023-12-26 18:17:17,890][105620] Updated weights for policy 1, policy_version 402046 (0.0008) [2023-12-26 18:17:17,967][105692] Updated weights for policy 0, policy_version 401770 (0.0006) [2023-12-26 18:17:18,023][105692] Updated weights for policy 0, policy_version 401780 (0.0009) [2023-12-26 18:17:18,087][105692] Updated weights for policy 0, policy_version 401790 (0.0009) [2023-12-26 18:17:18,145][105692] Updated weights for policy 0, policy_version 401800 (0.0009) [2023-12-26 18:17:18,613][105620] Updated weights for policy 1, policy_version 402056 (0.0007) [2023-12-26 18:17:18,663][105620] Updated weights for policy 1, policy_version 402066 (0.0008) [2023-12-26 18:17:18,718][105620] Updated weights for policy 1, policy_version 402076 (0.0008) [2023-12-26 18:17:18,851][105692] Updated weights for policy 0, policy_version 401810 (0.0010) [2023-12-26 18:17:18,910][105692] Updated weights for policy 0, policy_version 401820 (0.0010) [2023-12-26 18:17:18,981][105692] Updated weights for policy 0, policy_version 401830 (0.0010) [2023-12-26 18:17:19,429][105620] Updated weights for policy 1, policy_version 402086 (0.0007) [2023-12-26 18:17:19,474][105620] Updated weights for policy 1, policy_version 402096 (0.0008) [2023-12-26 18:17:19,538][105620] Updated weights for policy 1, policy_version 402106 (0.0008) [2023-12-26 18:17:19,703][105692] Updated weights for policy 0, policy_version 401840 (0.0010) [2023-12-26 18:17:19,770][105692] Updated weights for policy 0, policy_version 401850 (0.0011) [2023-12-26 18:17:19,839][105692] Updated weights for policy 0, policy_version 401860 (0.0011) [2023-12-26 18:17:20,300][105620] Updated weights for policy 1, policy_version 402116 (0.0008) [2023-12-26 18:17:20,372][105620] Updated weights for policy 1, policy_version 402126 (0.0010) [2023-12-26 18:17:20,439][105620] Updated weights for policy 1, policy_version 402136 (0.0010) [2023-12-26 18:17:20,516][105692] Updated weights for policy 0, policy_version 401870 (0.0008) [2023-12-26 18:17:20,582][105692] Updated weights for policy 0, policy_version 401880 (0.0007) [2023-12-26 18:17:20,646][105692] Updated weights for policy 0, policy_version 401890 (0.0011) [2023-12-26 18:17:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19660.8). Total num frames: 205856768. Throughput: 0: 10157.3, 1: 10007.3. Samples: 205848076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:17:21,063][104569] Avg episode reward: [(0, '6939.513'), (1, '7684.787')] [2023-12-26 18:17:21,218][105620] Updated weights for policy 1, policy_version 402146 (0.0009) [2023-12-26 18:17:21,283][105620] Updated weights for policy 1, policy_version 402156 (0.0008) [2023-12-26 18:17:21,309][105586] KL-divergence is very high: 106.9366 [2023-12-26 18:17:21,316][105586] KL-divergence is very high: 128.1182 [2023-12-26 18:17:21,328][105586] KL-divergence is very high: 137.9489 [2023-12-26 18:17:21,334][105586] KL-divergence is very high: 148.3736 [2023-12-26 18:17:21,346][105620] Updated weights for policy 1, policy_version 402166 (0.0008) [2023-12-26 18:17:21,414][105620] Updated weights for policy 1, policy_version 402176 (0.0009) [2023-12-26 18:17:21,422][105692] Updated weights for policy 0, policy_version 401900 (0.0011) [2023-12-26 18:17:21,478][105692] Updated weights for policy 0, policy_version 401910 (0.0010) [2023-12-26 18:17:21,535][105692] Updated weights for policy 0, policy_version 401920 (0.0011) [2023-12-26 18:17:22,083][105586] KL-divergence is very high: 102.4413 [2023-12-26 18:17:22,091][105586] KL-divergence is very high: 236.9806 [2023-12-26 18:17:22,122][105586] KL-divergence is very high: 106.4629 [2023-12-26 18:17:22,136][105586] KL-divergence is very high: 142.6068 [2023-12-26 18:17:22,150][105586] KL-divergence is very high: 139.4066 [2023-12-26 18:17:22,157][105586] KL-divergence is very high: 114.1416 [2023-12-26 18:17:22,157][105620] Updated weights for policy 1, policy_version 402186 (0.0008) [2023-12-26 18:17:22,186][105586] KL-divergence is very high: 143.0020 [2023-12-26 18:17:22,198][105586] KL-divergence is very high: 172.2650 [2023-12-26 18:17:22,205][105586] KL-divergence is very high: 101.6980 [2023-12-26 18:17:22,220][105620] Updated weights for policy 1, policy_version 402196 (0.0007) [2023-12-26 18:17:22,241][105586] KL-divergence is very high: 181.8426 [2023-12-26 18:17:22,254][105586] KL-divergence is very high: 158.9202 [2023-12-26 18:17:22,262][105586] KL-divergence is very high: 101.9157 [2023-12-26 18:17:22,287][105620] Updated weights for policy 1, policy_version 402206 (0.0007) [2023-12-26 18:17:22,294][105586] KL-divergence is very high: 138.3047 [2023-12-26 18:17:22,308][105692] Updated weights for policy 0, policy_version 401930 (0.0009) [2023-12-26 18:17:22,362][105692] Updated weights for policy 0, policy_version 401940 (0.0011) [2023-12-26 18:17:22,428][105692] Updated weights for policy 0, policy_version 401950 (0.0011) [2023-12-26 18:17:22,485][105692] Updated weights for policy 0, policy_version 401960 (0.0011) [2023-12-26 18:17:23,015][105620] Updated weights for policy 1, policy_version 402216 (0.0008) [2023-12-26 18:17:23,066][105620] Updated weights for policy 1, policy_version 402226 (0.0008) [2023-12-26 18:17:23,111][105586] KL-divergence is very high: 124.1663 [2023-12-26 18:17:23,129][105620] Updated weights for policy 1, policy_version 402236 (0.0008) [2023-12-26 18:17:23,237][105692] Updated weights for policy 0, policy_version 401970 (0.0010) [2023-12-26 18:17:23,300][105692] Updated weights for policy 0, policy_version 401980 (0.0010) [2023-12-26 18:17:23,355][105692] Updated weights for policy 0, policy_version 401990 (0.0010) [2023-12-26 18:17:23,854][105620] Updated weights for policy 1, policy_version 402246 (0.0009) [2023-12-26 18:17:23,878][105586] KL-divergence is very high: 186.6572 [2023-12-26 18:17:23,918][105620] Updated weights for policy 1, policy_version 402256 (0.0009) [2023-12-26 18:17:23,924][105586] KL-divergence is very high: 124.6707 [2023-12-26 18:17:23,931][105586] KL-divergence is very high: 356.4810 [2023-12-26 18:17:23,971][105586] KL-divergence is very high: 128.9920 [2023-12-26 18:17:23,977][105620] Updated weights for policy 1, policy_version 402266 (0.0007) [2023-12-26 18:17:23,978][105586] KL-divergence is very high: 363.9101 [2023-12-26 18:17:23,996][105692] Updated weights for policy 0, policy_version 402000 (0.0008) [2023-12-26 18:17:24,045][105692] Updated weights for policy 0, policy_version 402010 (0.0009) [2023-12-26 18:17:24,100][105692] Updated weights for policy 0, policy_version 402020 (0.0009) [2023-12-26 18:17:24,601][105586] KL-divergence is very high: 208.9387 [2023-12-26 18:17:24,623][105620] Updated weights for policy 1, policy_version 402276 (0.0006) [2023-12-26 18:17:24,643][105586] KL-divergence is very high: 176.9249 [2023-12-26 18:17:24,681][105620] Updated weights for policy 1, policy_version 402286 (0.0007) [2023-12-26 18:17:24,690][105586] KL-divergence is very high: 150.5794 [2023-12-26 18:17:24,732][105620] Updated weights for policy 1, policy_version 402296 (0.0007) [2023-12-26 18:17:24,734][105586] KL-divergence is very high: 137.6658 [2023-12-26 18:17:24,966][105692] Updated weights for policy 0, policy_version 402030 (0.0008) [2023-12-26 18:17:25,019][105692] Updated weights for policy 0, policy_version 402040 (0.0010) [2023-12-26 18:17:25,072][105692] Updated weights for policy 0, policy_version 402050 (0.0010) [2023-12-26 18:17:25,349][105620] Updated weights for policy 1, policy_version 402306 (0.0006) [2023-12-26 18:17:25,405][105620] Updated weights for policy 1, policy_version 402316 (0.0009) [2023-12-26 18:17:25,459][105620] Updated weights for policy 1, policy_version 402326 (0.0008) [2023-12-26 18:17:25,526][105620] Updated weights for policy 1, policy_version 402336 (0.0006) [2023-12-26 18:17:25,806][105692] Updated weights for policy 0, policy_version 402060 (0.0009) [2023-12-26 18:17:25,861][105692] Updated weights for policy 0, policy_version 402072 (0.0011) [2023-12-26 18:17:25,915][105692] Updated weights for policy 0, policy_version 402082 (0.0010) [2023-12-26 18:17:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19933.9, 300 sec: 19660.8). Total num frames: 205955072. Throughput: 0: 10162.0, 1: 10075.5. Samples: 205964008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:17:26,062][104569] Avg episode reward: [(0, '7812.872'), (1, '2108.899')] [2023-12-26 18:17:26,126][105586] KL-divergence is very high: 106.0915 [2023-12-26 18:17:26,132][105586] KL-divergence is very high: 109.7860 [2023-12-26 18:17:26,142][105586] KL-divergence is very high: 115.6489 [2023-12-26 18:17:26,149][105620] Updated weights for policy 1, policy_version 402347 (0.0007) [2023-12-26 18:17:26,195][105620] Updated weights for policy 1, policy_version 402357 (0.0008) [2023-12-26 18:17:26,248][105620] Updated weights for policy 1, policy_version 402367 (0.0008) [2023-12-26 18:17:26,612][105692] Updated weights for policy 0, policy_version 402092 (0.0008) [2023-12-26 18:17:26,678][105692] Updated weights for policy 0, policy_version 402102 (0.0005) [2023-12-26 18:17:26,745][105692] Updated weights for policy 0, policy_version 402112 (0.0005) [2023-12-26 18:17:27,011][105620] Updated weights for policy 1, policy_version 402377 (0.0006) [2023-12-26 18:17:27,026][105586] KL-divergence is very high: 163.4566 [2023-12-26 18:17:27,068][105620] Updated weights for policy 1, policy_version 402387 (0.0005) [2023-12-26 18:17:27,076][105586] KL-divergence is very high: 138.1846 [2023-12-26 18:17:27,131][105586] KL-divergence is very high: 149.6825 [2023-12-26 18:17:27,137][105620] Updated weights for policy 1, policy_version 402397 (0.0005) [2023-12-26 18:17:27,282][105692] Updated weights for policy 0, policy_version 402122 (0.0005) [2023-12-26 18:17:27,344][105692] Updated weights for policy 0, policy_version 402133 (0.0009) [2023-12-26 18:17:27,402][105692] Updated weights for policy 0, policy_version 402143 (0.0010) [2023-12-26 18:17:27,624][105620] Updated weights for policy 1, policy_version 402407 (0.0005) [2023-12-26 18:17:27,674][105620] Updated weights for policy 1, policy_version 402417 (0.0005) [2023-12-26 18:17:27,726][105620] Updated weights for policy 1, policy_version 402427 (0.0007) [2023-12-26 18:17:27,999][105692] Updated weights for policy 0, policy_version 402153 (0.0009) [2023-12-26 18:17:28,059][105692] Updated weights for policy 0, policy_version 402163 (0.0006) [2023-12-26 18:17:28,125][105692] Updated weights for policy 0, policy_version 402173 (0.0009) [2023-12-26 18:17:28,181][105692] Updated weights for policy 0, policy_version 402183 (0.0009) [2023-12-26 18:17:28,347][105620] Updated weights for policy 1, policy_version 402437 (0.0009) [2023-12-26 18:17:28,397][105620] Updated weights for policy 1, policy_version 402447 (0.0007) [2023-12-26 18:17:28,452][105620] Updated weights for policy 1, policy_version 402457 (0.0007) [2023-12-26 18:17:28,872][105692] Updated weights for policy 0, policy_version 402193 (0.0010) [2023-12-26 18:17:28,926][105692] Updated weights for policy 0, policy_version 402204 (0.0011) [2023-12-26 18:17:28,977][105692] Updated weights for policy 0, policy_version 402214 (0.0010) [2023-12-26 18:17:29,067][105620] Updated weights for policy 1, policy_version 402467 (0.0006) [2023-12-26 18:17:29,136][105620] Updated weights for policy 1, policy_version 402477 (0.0006) [2023-12-26 18:17:29,186][105620] Updated weights for policy 1, policy_version 402487 (0.0009) [2023-12-26 18:17:29,741][105620] Updated weights for policy 1, policy_version 402497 (0.0009) [2023-12-26 18:17:29,800][105620] Updated weights for policy 1, policy_version 402507 (0.0006) [2023-12-26 18:17:29,861][105620] Updated weights for policy 1, policy_version 402517 (0.0007) [2023-12-26 18:17:29,876][105692] Updated weights for policy 0, policy_version 402224 (0.0009) [2023-12-26 18:17:29,921][105620] Updated weights for policy 1, policy_version 402527 (0.0006) [2023-12-26 18:17:29,937][105692] Updated weights for policy 0, policy_version 402234 (0.0008) [2023-12-26 18:17:29,944][105585] KL-divergence is very high: 255.4151 [2023-12-26 18:17:29,966][105585] KL-divergence is very high: 218.4430 [2023-12-26 18:17:29,973][105585] KL-divergence is very high: 135.6401 [2023-12-26 18:17:29,992][105585] KL-divergence is very high: 331.6364 [2023-12-26 18:17:29,999][105692] Updated weights for policy 0, policy_version 402244 (0.0008) [2023-12-26 18:17:30,021][105585] KL-divergence is very high: 194.2836 [2023-12-26 18:17:30,606][105620] Updated weights for policy 1, policy_version 402537 (0.0006) [2023-12-26 18:17:30,661][105620] Updated weights for policy 1, policy_version 402547 (0.0006) [2023-12-26 18:17:30,713][105620] Updated weights for policy 1, policy_version 402557 (0.0008) [2023-12-26 18:17:30,735][105585] KL-divergence is very high: 115.9164 [2023-12-26 18:17:30,741][105692] Updated weights for policy 0, policy_version 402254 (0.0010) [2023-12-26 18:17:30,773][105585] KL-divergence is very high: 105.2503 [2023-12-26 18:17:30,786][105692] Updated weights for policy 0, policy_version 402264 (0.0007) [2023-12-26 18:17:30,805][105585] KL-divergence is very high: 105.3448 [2023-12-26 18:17:30,810][105585] KL-divergence is very high: 202.3268 [2023-12-26 18:17:30,839][105692] Updated weights for policy 0, policy_version 402274 (0.0010) [2023-12-26 18:17:30,853][105585] KL-divergence is very high: 113.1989 [2023-12-26 18:17:30,859][105585] KL-divergence is very high: 216.0775 [2023-12-26 18:17:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 20206.9, 300 sec: 19660.8). Total num frames: 206061568. Throughput: 0: 10212.1, 1: 10152.2. Samples: 206028856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:17:31,063][104569] Avg episode reward: [(0, '8088.456'), (1, '2903.413')] [2023-12-26 18:17:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000402280_102998016.pth... [2023-12-26 18:17:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000402560_103063552.pth... [2023-12-26 18:17:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000401376_102760448.pth [2023-12-26 18:17:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000401096_102694912.pth [2023-12-26 18:17:31,384][105620] Updated weights for policy 1, policy_version 402567 (0.0009) [2023-12-26 18:17:31,443][105620] Updated weights for policy 1, policy_version 402577 (0.0008) [2023-12-26 18:17:31,493][105620] Updated weights for policy 1, policy_version 402587 (0.0008) [2023-12-26 18:17:31,532][105692] Updated weights for policy 0, policy_version 402284 (0.0010) [2023-12-26 18:17:31,586][105692] Updated weights for policy 0, policy_version 402294 (0.0009) [2023-12-26 18:17:31,646][105692] Updated weights for policy 0, policy_version 402304 (0.0007) [2023-12-26 18:17:32,259][105620] Updated weights for policy 1, policy_version 402597 (0.0008) [2023-12-26 18:17:32,326][105620] Updated weights for policy 1, policy_version 402607 (0.0008) [2023-12-26 18:17:32,363][105692] Updated weights for policy 0, policy_version 402314 (0.0006) [2023-12-26 18:17:32,390][105620] Updated weights for policy 1, policy_version 402617 (0.0007) [2023-12-26 18:17:32,421][105692] Updated weights for policy 0, policy_version 402324 (0.0008) [2023-12-26 18:17:32,481][105692] Updated weights for policy 0, policy_version 402334 (0.0008) [2023-12-26 18:17:32,533][105692] Updated weights for policy 0, policy_version 402344 (0.0008) [2023-12-26 18:17:32,979][105620] Updated weights for policy 1, policy_version 402627 (0.0005) [2023-12-26 18:17:33,028][105620] Updated weights for policy 1, policy_version 402637 (0.0009) [2023-12-26 18:17:33,073][105620] Updated weights for policy 1, policy_version 402647 (0.0008) [2023-12-26 18:17:33,314][105692] Updated weights for policy 0, policy_version 402354 (0.0008) [2023-12-26 18:17:33,373][105692] Updated weights for policy 0, policy_version 402364 (0.0008) [2023-12-26 18:17:33,440][105692] Updated weights for policy 0, policy_version 402374 (0.0009) [2023-12-26 18:17:33,840][105620] Updated weights for policy 1, policy_version 402657 (0.0008) [2023-12-26 18:17:33,886][105620] Updated weights for policy 1, policy_version 402667 (0.0005) [2023-12-26 18:17:33,899][105586] KL-divergence is very high: 113.9165 [2023-12-26 18:17:33,938][105620] Updated weights for policy 1, policy_version 402677 (0.0005) [2023-12-26 18:17:33,948][105586] KL-divergence is very high: 229.8175 [2023-12-26 18:17:33,991][105586] KL-divergence is very high: 266.6348 [2023-12-26 18:17:33,998][105620] Updated weights for policy 1, policy_version 402687 (0.0005) [2023-12-26 18:17:34,228][105692] Updated weights for policy 0, policy_version 402384 (0.0009) [2023-12-26 18:17:34,236][105585] KL-divergence is very high: 116.6645 [2023-12-26 18:17:34,296][105585] KL-divergence is very high: 119.5478 [2023-12-26 18:17:34,304][105692] Updated weights for policy 0, policy_version 402394 (0.0008) [2023-12-26 18:17:34,349][105585] KL-divergence is very high: 119.9621 [2023-12-26 18:17:34,369][105692] Updated weights for policy 0, policy_version 402404 (0.0009) [2023-12-26 18:17:34,563][105620] Updated weights for policy 1, policy_version 402697 (0.0006) [2023-12-26 18:17:34,629][105620] Updated weights for policy 1, policy_version 402707 (0.0006) [2023-12-26 18:17:34,696][105620] Updated weights for policy 1, policy_version 402717 (0.0009) [2023-12-26 18:17:35,162][105692] Updated weights for policy 0, policy_version 402414 (0.0009) [2023-12-26 18:17:35,223][105692] Updated weights for policy 0, policy_version 402424 (0.0009) [2023-12-26 18:17:35,282][105692] Updated weights for policy 0, policy_version 402434 (0.0008) [2023-12-26 18:17:35,304][105620] Updated weights for policy 1, policy_version 402727 (0.0007) [2023-12-26 18:17:35,364][105620] Updated weights for policy 1, policy_version 402737 (0.0008) [2023-12-26 18:17:35,421][105620] Updated weights for policy 1, policy_version 402747 (0.0009) [2023-12-26 18:17:36,036][105692] Updated weights for policy 0, policy_version 402444 (0.0008) [2023-12-26 18:17:36,062][104569] Fps is (10 sec: 19660.0, 60 sec: 20070.3, 300 sec: 19660.8). Total num frames: 206151680. Throughput: 0: 10024.2, 1: 10149.6. Samples: 206146952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:17:36,063][104569] Avg episode reward: [(0, '4928.700'), (1, '6747.569')] [2023-12-26 18:17:36,104][105692] Updated weights for policy 0, policy_version 402454 (0.0010) [2023-12-26 18:17:36,122][105620] Updated weights for policy 1, policy_version 402757 (0.0008) [2023-12-26 18:17:36,136][105585] KL-divergence is very high: 114.2280 [2023-12-26 18:17:36,163][105692] Updated weights for policy 0, policy_version 402464 (0.0007) [2023-12-26 18:17:36,186][105585] KL-divergence is very high: 113.9746 [2023-12-26 18:17:36,187][105620] Updated weights for policy 1, policy_version 402767 (0.0007) [2023-12-26 18:17:36,254][105620] Updated weights for policy 1, policy_version 402777 (0.0008) [2023-12-26 18:17:36,847][105692] Updated weights for policy 0, policy_version 402474 (0.0007) [2023-12-26 18:17:36,884][105620] Updated weights for policy 1, policy_version 402787 (0.0009) [2023-12-26 18:17:36,918][105692] Updated weights for policy 0, policy_version 402484 (0.0006) [2023-12-26 18:17:36,954][105620] Updated weights for policy 1, policy_version 402797 (0.0008) [2023-12-26 18:17:36,987][105692] Updated weights for policy 0, policy_version 402494 (0.0006) [2023-12-26 18:17:37,023][105620] Updated weights for policy 1, policy_version 402807 (0.0007) [2023-12-26 18:17:37,053][105692] Updated weights for policy 0, policy_version 402504 (0.0007) [2023-12-26 18:17:37,612][105692] Updated weights for policy 0, policy_version 402514 (0.0005) [2023-12-26 18:17:37,668][105692] Updated weights for policy 0, policy_version 402524 (0.0005) [2023-12-26 18:17:37,696][105620] Updated weights for policy 1, policy_version 402817 (0.0007) [2023-12-26 18:17:37,726][105692] Updated weights for policy 0, policy_version 402534 (0.0008) [2023-12-26 18:17:37,751][105620] Updated weights for policy 1, policy_version 402827 (0.0005) [2023-12-26 18:17:37,814][105620] Updated weights for policy 1, policy_version 402837 (0.0005) [2023-12-26 18:17:37,872][105620] Updated weights for policy 1, policy_version 402847 (0.0006) [2023-12-26 18:17:38,432][105692] Updated weights for policy 0, policy_version 402544 (0.0008) [2023-12-26 18:17:38,491][105692] Updated weights for policy 0, policy_version 402554 (0.0009) [2023-12-26 18:17:38,508][105620] Updated weights for policy 1, policy_version 402857 (0.0009) [2023-12-26 18:17:38,554][105692] Updated weights for policy 0, policy_version 402564 (0.0006) [2023-12-26 18:17:38,572][105620] Updated weights for policy 1, policy_version 402867 (0.0007) [2023-12-26 18:17:38,631][105620] Updated weights for policy 1, policy_version 402877 (0.0009) [2023-12-26 18:17:39,184][105692] Updated weights for policy 0, policy_version 402574 (0.0007) [2023-12-26 18:17:39,244][105692] Updated weights for policy 0, policy_version 402584 (0.0007) [2023-12-26 18:17:39,301][105692] Updated weights for policy 0, policy_version 402594 (0.0009) [2023-12-26 18:17:39,437][105620] Updated weights for policy 1, policy_version 402887 (0.0009) [2023-12-26 18:17:39,497][105620] Updated weights for policy 1, policy_version 402897 (0.0009) [2023-12-26 18:17:39,549][105620] Updated weights for policy 1, policy_version 402907 (0.0009) [2023-12-26 18:17:40,062][105692] Updated weights for policy 0, policy_version 402604 (0.0009) [2023-12-26 18:17:40,124][105692] Updated weights for policy 0, policy_version 402614 (0.0009) [2023-12-26 18:17:40,186][105692] Updated weights for policy 0, policy_version 402624 (0.0009) [2023-12-26 18:17:40,329][105620] Updated weights for policy 1, policy_version 402917 (0.0009) [2023-12-26 18:17:40,397][105620] Updated weights for policy 1, policy_version 402927 (0.0008) [2023-12-26 18:17:40,458][105620] Updated weights for policy 1, policy_version 402937 (0.0009) [2023-12-26 18:17:40,953][105692] Updated weights for policy 0, policy_version 402634 (0.0009) [2023-12-26 18:17:41,013][105692] Updated weights for policy 0, policy_version 402644 (0.0008) [2023-12-26 18:17:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 20070.4, 300 sec: 19633.0). Total num frames: 206249984. Throughput: 0: 9959.4, 1: 10124.6. Samples: 206264480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:17:41,062][104569] Avg episode reward: [(0, '6079.230'), (1, '8731.559')] [2023-12-26 18:17:41,084][105692] Updated weights for policy 0, policy_version 402654 (0.0009) [2023-12-26 18:17:41,151][105692] Updated weights for policy 0, policy_version 402664 (0.0011) [2023-12-26 18:17:41,242][105620] Updated weights for policy 1, policy_version 402947 (0.0009) [2023-12-26 18:17:41,309][105620] Updated weights for policy 1, policy_version 402957 (0.0008) [2023-12-26 18:17:41,373][105620] Updated weights for policy 1, policy_version 402967 (0.0008) [2023-12-26 18:17:41,957][105692] Updated weights for policy 0, policy_version 402674 (0.0008) [2023-12-26 18:17:42,023][105692] Updated weights for policy 0, policy_version 402684 (0.0008) [2023-12-26 18:17:42,080][105692] Updated weights for policy 0, policy_version 402694 (0.0010) [2023-12-26 18:17:42,118][105620] Updated weights for policy 1, policy_version 402977 (0.0008) [2023-12-26 18:17:42,177][105620] Updated weights for policy 1, policy_version 402987 (0.0007) [2023-12-26 18:17:42,224][105620] Updated weights for policy 1, policy_version 402997 (0.0009) [2023-12-26 18:17:42,282][105620] Updated weights for policy 1, policy_version 403007 (0.0008) [2023-12-26 18:17:42,849][105692] Updated weights for policy 0, policy_version 402704 (0.0011) [2023-12-26 18:17:42,911][105692] Updated weights for policy 0, policy_version 402714 (0.0010) [2023-12-26 18:17:42,972][105692] Updated weights for policy 0, policy_version 402724 (0.0010) [2023-12-26 18:17:42,986][105620] Updated weights for policy 1, policy_version 403017 (0.0006) [2023-12-26 18:17:43,052][105620] Updated weights for policy 1, policy_version 403027 (0.0008) [2023-12-26 18:17:43,114][105620] Updated weights for policy 1, policy_version 403037 (0.0008) [2023-12-26 18:17:43,653][105692] Updated weights for policy 0, policy_version 402734 (0.0007) [2023-12-26 18:17:43,719][105692] Updated weights for policy 0, policy_version 402744 (0.0005) [2023-12-26 18:17:43,727][105585] KL-divergence is very high: 174.8183 [2023-12-26 18:17:43,778][105585] KL-divergence is very high: 183.9343 [2023-12-26 18:17:43,784][105692] Updated weights for policy 0, policy_version 402754 (0.0005) [2023-12-26 18:17:43,930][105620] Updated weights for policy 1, policy_version 403047 (0.0008) [2023-12-26 18:17:43,982][105620] Updated weights for policy 1, policy_version 403057 (0.0008) [2023-12-26 18:17:44,036][105620] Updated weights for policy 1, policy_version 403067 (0.0009) [2023-12-26 18:17:44,377][105692] Updated weights for policy 0, policy_version 402764 (0.0007) [2023-12-26 18:17:44,437][105692] Updated weights for policy 0, policy_version 402774 (0.0010) [2023-12-26 18:17:44,505][105692] Updated weights for policy 0, policy_version 402784 (0.0010) [2023-12-26 18:17:44,825][105620] Updated weights for policy 1, policy_version 403077 (0.0009) [2023-12-26 18:17:44,890][105620] Updated weights for policy 1, policy_version 403087 (0.0009) [2023-12-26 18:17:44,941][105620] Updated weights for policy 1, policy_version 403097 (0.0006) [2023-12-26 18:17:45,222][105692] Updated weights for policy 0, policy_version 402794 (0.0010) [2023-12-26 18:17:45,281][105692] Updated weights for policy 0, policy_version 402804 (0.0009) [2023-12-26 18:17:45,336][105692] Updated weights for policy 0, policy_version 402814 (0.0008) [2023-12-26 18:17:45,395][105692] Updated weights for policy 0, policy_version 402824 (0.0009) [2023-12-26 18:17:45,643][105620] Updated weights for policy 1, policy_version 403107 (0.0005) [2023-12-26 18:17:45,703][105620] Updated weights for policy 1, policy_version 403117 (0.0005) [2023-12-26 18:17:45,769][105620] Updated weights for policy 1, policy_version 403127 (0.0009) [2023-12-26 18:17:46,062][104569] Fps is (10 sec: 19661.4, 60 sec: 20070.4, 300 sec: 19633.0). Total num frames: 206348288. Throughput: 0: 9810.5, 1: 10059.0. Samples: 206319136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:17:46,062][104569] Avg episode reward: [(0, '7058.743'), (1, '8816.831')] [2023-12-26 18:17:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000403136_103211008.pth... [2023-12-26 18:17:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000401952_102907904.pth [2023-12-26 18:17:46,125][105692] Updated weights for policy 0, policy_version 402834 (0.0009) [2023-12-26 18:17:46,174][105692] Updated weights for policy 0, policy_version 402844 (0.0009) [2023-12-26 18:17:46,220][105692] Updated weights for policy 0, policy_version 402854 (0.0008) [2023-12-26 18:17:46,227][105585] KL-divergence is very high: 104.4713 [2023-12-26 18:17:46,231][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000402856_103145472.pth... [2023-12-26 18:17:46,234][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000401704_102850560.pth [2023-12-26 18:17:46,497][105620] Updated weights for policy 1, policy_version 403137 (0.0009) [2023-12-26 18:17:46,548][105620] Updated weights for policy 1, policy_version 403147 (0.0007) [2023-12-26 18:17:46,597][105620] Updated weights for policy 1, policy_version 403158 (0.0008) [2023-12-26 18:17:46,641][105620] Updated weights for policy 1, policy_version 403168 (0.0007) [2023-12-26 18:17:46,933][105692] Updated weights for policy 0, policy_version 402864 (0.0005) [2023-12-26 18:17:46,985][105692] Updated weights for policy 0, policy_version 402874 (0.0006) [2023-12-26 18:17:47,039][105692] Updated weights for policy 0, policy_version 402884 (0.0006) [2023-12-26 18:17:47,306][105620] Updated weights for policy 1, policy_version 403178 (0.0005) [2023-12-26 18:17:47,354][105620] Updated weights for policy 1, policy_version 403188 (0.0005) [2023-12-26 18:17:47,406][105620] Updated weights for policy 1, policy_version 403198 (0.0008) [2023-12-26 18:17:47,744][105692] Updated weights for policy 0, policy_version 402894 (0.0006) [2023-12-26 18:17:47,803][105692] Updated weights for policy 0, policy_version 402904 (0.0005) [2023-12-26 18:17:47,863][105692] Updated weights for policy 0, policy_version 402914 (0.0005) [2023-12-26 18:17:48,068][105620] Updated weights for policy 1, policy_version 403208 (0.0010) [2023-12-26 18:17:48,128][105620] Updated weights for policy 1, policy_version 403218 (0.0011) [2023-12-26 18:17:48,194][105620] Updated weights for policy 1, policy_version 403228 (0.0011) [2023-12-26 18:17:48,566][105692] Updated weights for policy 0, policy_version 402924 (0.0007) [2023-12-26 18:17:48,621][105692] Updated weights for policy 0, policy_version 402934 (0.0008) [2023-12-26 18:17:48,679][105692] Updated weights for policy 0, policy_version 402944 (0.0006) [2023-12-26 18:17:48,988][105620] Updated weights for policy 1, policy_version 403238 (0.0010) [2023-12-26 18:17:49,038][105620] Updated weights for policy 1, policy_version 403248 (0.0009) [2023-12-26 18:17:49,091][105620] Updated weights for policy 1, policy_version 403258 (0.0007) [2023-12-26 18:17:49,302][105692] Updated weights for policy 0, policy_version 402954 (0.0006) [2023-12-26 18:17:49,342][105585] KL-divergence is very high: 102.9204 [2023-12-26 18:17:49,349][105585] KL-divergence is very high: 174.2804 [2023-12-26 18:17:49,366][105692] Updated weights for policy 0, policy_version 402964 (0.0009) [2023-12-26 18:17:49,385][105585] KL-divergence is very high: 187.5454 [2023-12-26 18:17:49,390][105585] KL-divergence is very high: 285.3222 [2023-12-26 18:17:49,419][105692] Updated weights for policy 0, policy_version 402974 (0.0010) [2023-12-26 18:17:49,427][105585] KL-divergence is very high: 203.4116 [2023-12-26 18:17:49,433][105585] KL-divergence is very high: 313.7078 [2023-12-26 18:17:49,472][105692] Updated weights for policy 0, policy_version 402984 (0.0009) [2023-12-26 18:17:49,792][105620] Updated weights for policy 1, policy_version 403268 (0.0007) [2023-12-26 18:17:49,856][105620] Updated weights for policy 1, policy_version 403278 (0.0009) [2023-12-26 18:17:49,917][105620] Updated weights for policy 1, policy_version 403288 (0.0007) [2023-12-26 18:17:50,217][105585] KL-divergence is very high: 165.6297 [2023-12-26 18:17:50,262][105585] KL-divergence is very high: 172.7305 [2023-12-26 18:17:50,275][105692] Updated weights for policy 0, policy_version 402994 (0.0010) [2023-12-26 18:17:50,305][105585] KL-divergence is very high: 176.3550 [2023-12-26 18:17:50,328][105692] Updated weights for policy 0, policy_version 403004 (0.0009) [2023-12-26 18:17:50,354][105585] KL-divergence is very high: 143.5815 [2023-12-26 18:17:50,394][105692] Updated weights for policy 0, policy_version 403014 (0.0010) [2023-12-26 18:17:50,506][105620] Updated weights for policy 1, policy_version 403298 (0.0006) [2023-12-26 18:17:50,566][105620] Updated weights for policy 1, policy_version 403308 (0.0006) [2023-12-26 18:17:50,631][105620] Updated weights for policy 1, policy_version 403318 (0.0006) [2023-12-26 18:17:50,688][105620] Updated weights for policy 1, policy_version 403328 (0.0006) [2023-12-26 18:17:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 20070.4, 300 sec: 19660.8). Total num frames: 206446592. Throughput: 0: 9796.1, 1: 10097.2. Samples: 206437132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:17:51,062][104569] Avg episode reward: [(0, '7530.031'), (1, '8553.289')] [2023-12-26 18:17:51,144][105692] Updated weights for policy 0, policy_version 403024 (0.0009) [2023-12-26 18:17:51,213][105692] Updated weights for policy 0, policy_version 403034 (0.0009) [2023-12-26 18:17:51,275][105692] Updated weights for policy 0, policy_version 403044 (0.0010) [2023-12-26 18:17:51,355][105620] Updated weights for policy 1, policy_version 403338 (0.0008) [2023-12-26 18:17:51,424][105620] Updated weights for policy 1, policy_version 403348 (0.0010) [2023-12-26 18:17:51,487][105620] Updated weights for policy 1, policy_version 403358 (0.0010) [2023-12-26 18:17:52,055][105692] Updated weights for policy 0, policy_version 403054 (0.0009) [2023-12-26 18:17:52,116][105692] Updated weights for policy 0, policy_version 403064 (0.0010) [2023-12-26 18:17:52,163][105692] Updated weights for policy 0, policy_version 403074 (0.0007) [2023-12-26 18:17:52,185][105620] Updated weights for policy 1, policy_version 403368 (0.0008) [2023-12-26 18:17:52,245][105620] Updated weights for policy 1, policy_version 403378 (0.0006) [2023-12-26 18:17:52,305][105620] Updated weights for policy 1, policy_version 403388 (0.0006) [2023-12-26 18:17:52,911][105692] Updated weights for policy 0, policy_version 403084 (0.0007) [2023-12-26 18:17:52,969][105692] Updated weights for policy 0, policy_version 403094 (0.0009) [2023-12-26 18:17:53,011][105620] Updated weights for policy 1, policy_version 403398 (0.0008) [2023-12-26 18:17:53,019][105692] Updated weights for policy 0, policy_version 403104 (0.0005) [2023-12-26 18:17:53,069][105620] Updated weights for policy 1, policy_version 403408 (0.0006) [2023-12-26 18:17:53,123][105620] Updated weights for policy 1, policy_version 403418 (0.0005) [2023-12-26 18:17:53,659][105692] Updated weights for policy 0, policy_version 403114 (0.0005) [2023-12-26 18:17:53,723][105692] Updated weights for policy 0, policy_version 403124 (0.0005) [2023-12-26 18:17:53,782][105692] Updated weights for policy 0, policy_version 403134 (0.0008) [2023-12-26 18:17:53,845][105692] Updated weights for policy 0, policy_version 403144 (0.0009) [2023-12-26 18:17:53,869][105620] Updated weights for policy 1, policy_version 403428 (0.0006) [2023-12-26 18:17:53,924][105620] Updated weights for policy 1, policy_version 403438 (0.0005) [2023-12-26 18:17:53,974][105620] Updated weights for policy 1, policy_version 403448 (0.0005) [2023-12-26 18:17:54,543][105620] Updated weights for policy 1, policy_version 403458 (0.0005) [2023-12-26 18:17:54,578][105692] Updated weights for policy 0, policy_version 403154 (0.0008) [2023-12-26 18:17:54,589][105620] Updated weights for policy 1, policy_version 403468 (0.0008) [2023-12-26 18:17:54,628][105692] Updated weights for policy 0, policy_version 403164 (0.0008) [2023-12-26 18:17:54,639][105620] Updated weights for policy 1, policy_version 403478 (0.0010) [2023-12-26 18:17:54,677][105692] Updated weights for policy 0, policy_version 403174 (0.0007) [2023-12-26 18:17:54,687][105620] Updated weights for policy 1, policy_version 403488 (0.0010) [2023-12-26 18:17:55,365][105692] Updated weights for policy 0, policy_version 403184 (0.0007) [2023-12-26 18:17:55,417][105692] Updated weights for policy 0, policy_version 403194 (0.0006) [2023-12-26 18:17:55,419][105620] Updated weights for policy 1, policy_version 403498 (0.0010) [2023-12-26 18:17:55,473][105620] Updated weights for policy 1, policy_version 403508 (0.0011) [2023-12-26 18:17:55,477][105692] Updated weights for policy 0, policy_version 403204 (0.0007) [2023-12-26 18:17:55,532][105620] Updated weights for policy 1, policy_version 403518 (0.0010) [2023-12-26 18:17:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 206544896. Throughput: 0: 9741.1, 1: 10092.3. Samples: 206556324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:17:56,063][104569] Avg episode reward: [(0, '7621.154'), (1, '8649.020')] [2023-12-26 18:17:56,153][105620] Updated weights for policy 1, policy_version 403528 (0.0011) [2023-12-26 18:17:56,210][105620] Updated weights for policy 1, policy_version 403538 (0.0011) [2023-12-26 18:17:56,232][105692] Updated weights for policy 0, policy_version 403214 (0.0006) [2023-12-26 18:17:56,266][105620] Updated weights for policy 1, policy_version 403548 (0.0011) [2023-12-26 18:17:56,284][105692] Updated weights for policy 0, policy_version 403224 (0.0005) [2023-12-26 18:17:56,331][105692] Updated weights for policy 0, policy_version 403234 (0.0008) [2023-12-26 18:17:56,949][105620] Updated weights for policy 1, policy_version 403558 (0.0008) [2023-12-26 18:17:57,000][105620] Updated weights for policy 1, policy_version 403568 (0.0007) [2023-12-26 18:17:57,050][105620] Updated weights for policy 1, policy_version 403578 (0.0009) [2023-12-26 18:17:57,076][105692] Updated weights for policy 0, policy_version 403244 (0.0008) [2023-12-26 18:17:57,124][105692] Updated weights for policy 0, policy_version 403254 (0.0007) [2023-12-26 18:17:57,170][105692] Updated weights for policy 0, policy_version 403264 (0.0008) [2023-12-26 18:17:57,693][105620] Updated weights for policy 1, policy_version 403588 (0.0006) [2023-12-26 18:17:57,748][105620] Updated weights for policy 1, policy_version 403598 (0.0009) [2023-12-26 18:17:57,806][105620] Updated weights for policy 1, policy_version 403608 (0.0010) [2023-12-26 18:17:57,809][105692] Updated weights for policy 0, policy_version 403274 (0.0009) [2023-12-26 18:17:57,854][105692] Updated weights for policy 0, policy_version 403284 (0.0009) [2023-12-26 18:17:57,902][105692] Updated weights for policy 0, policy_version 403294 (0.0008) [2023-12-26 18:17:57,954][105692] Updated weights for policy 0, policy_version 403304 (0.0009) [2023-12-26 18:17:58,477][105620] Updated weights for policy 1, policy_version 403618 (0.0010) [2023-12-26 18:17:58,540][105620] Updated weights for policy 1, policy_version 403628 (0.0008) [2023-12-26 18:17:58,607][105620] Updated weights for policy 1, policy_version 403638 (0.0008) [2023-12-26 18:17:58,671][105620] Updated weights for policy 1, policy_version 403648 (0.0008) [2023-12-26 18:17:58,859][105692] Updated weights for policy 0, policy_version 403314 (0.0009) [2023-12-26 18:17:58,923][105692] Updated weights for policy 0, policy_version 403324 (0.0011) [2023-12-26 18:17:58,983][105692] Updated weights for policy 0, policy_version 403334 (0.0008) [2023-12-26 18:17:59,431][105620] Updated weights for policy 1, policy_version 403658 (0.0008) [2023-12-26 18:17:59,476][105620] Updated weights for policy 1, policy_version 403668 (0.0008) [2023-12-26 18:17:59,524][105620] Updated weights for policy 1, policy_version 403678 (0.0008) [2023-12-26 18:17:59,735][105692] Updated weights for policy 0, policy_version 403344 (0.0008) [2023-12-26 18:17:59,793][105692] Updated weights for policy 0, policy_version 403354 (0.0010) [2023-12-26 18:17:59,854][105692] Updated weights for policy 0, policy_version 403364 (0.0009) [2023-12-26 18:18:00,172][105620] Updated weights for policy 1, policy_version 403688 (0.0006) [2023-12-26 18:18:00,229][105620] Updated weights for policy 1, policy_version 403698 (0.0006) [2023-12-26 18:18:00,294][105620] Updated weights for policy 1, policy_version 403708 (0.0006) [2023-12-26 18:18:00,691][105692] Updated weights for policy 0, policy_version 403374 (0.0007) [2023-12-26 18:18:00,712][105585] KL-divergence is very high: 210.2737 [2023-12-26 18:18:00,764][105692] Updated weights for policy 0, policy_version 403384 (0.0005) [2023-12-26 18:18:00,769][105585] KL-divergence is very high: 343.9992 [2023-12-26 18:18:00,805][105585] KL-divergence is very high: 366.6281 [2023-12-26 18:18:00,810][105692] Updated weights for policy 0, policy_version 403394 (0.0005) [2023-12-26 18:18:00,935][105620] Updated weights for policy 1, policy_version 403718 (0.0008) [2023-12-26 18:18:00,986][105620] Updated weights for policy 1, policy_version 403728 (0.0006) [2023-12-26 18:18:01,060][105620] Updated weights for policy 1, policy_version 403738 (0.0008) [2023-12-26 18:18:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19933.8, 300 sec: 19660.8). Total num frames: 206643200. Throughput: 0: 9753.2, 1: 10113.1. Samples: 206615900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:18:01,063][104569] Avg episode reward: [(0, '8182.160'), (1, '8828.912')] [2023-12-26 18:18:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000403400_103284736.pth... [2023-12-26 18:18:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000402280_102998016.pth [2023-12-26 18:18:01,088][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000403744_103366656.pth... [2023-12-26 18:18:01,091][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000402560_103063552.pth [2023-12-26 18:18:01,458][105692] Updated weights for policy 0, policy_version 403404 (0.0007) [2023-12-26 18:18:01,518][105692] Updated weights for policy 0, policy_version 403414 (0.0009) [2023-12-26 18:18:01,579][105692] Updated weights for policy 0, policy_version 403424 (0.0008) [2023-12-26 18:18:01,832][105620] Updated weights for policy 1, policy_version 403748 (0.0010) [2023-12-26 18:18:01,877][105620] Updated weights for policy 1, policy_version 403758 (0.0010) [2023-12-26 18:18:01,940][105620] Updated weights for policy 1, policy_version 403768 (0.0010) [2023-12-26 18:18:02,351][105692] Updated weights for policy 0, policy_version 403434 (0.0008) [2023-12-26 18:18:02,402][105692] Updated weights for policy 0, policy_version 403444 (0.0008) [2023-12-26 18:18:02,407][105585] KL-divergence is very high: 372.7216 [2023-12-26 18:18:02,445][105585] KL-divergence is very high: 701.6954 [2023-12-26 18:18:02,450][105692] Updated weights for policy 0, policy_version 403454 (0.0008) [2023-12-26 18:18:02,495][105585] KL-divergence is very high: 722.5269 [2023-12-26 18:18:02,513][105692] Updated weights for policy 0, policy_version 403464 (0.0008) [2023-12-26 18:18:02,697][105620] Updated weights for policy 1, policy_version 403778 (0.0010) [2023-12-26 18:18:02,752][105620] Updated weights for policy 1, policy_version 403788 (0.0010) [2023-12-26 18:18:02,803][105620] Updated weights for policy 1, policy_version 403798 (0.0010) [2023-12-26 18:18:02,861][105620] Updated weights for policy 1, policy_version 403808 (0.0010) [2023-12-26 18:18:03,276][105692] Updated weights for policy 0, policy_version 403474 (0.0010) [2023-12-26 18:18:03,340][105692] Updated weights for policy 0, policy_version 403484 (0.0010) [2023-12-26 18:18:03,394][105692] Updated weights for policy 0, policy_version 403494 (0.0010) [2023-12-26 18:18:03,553][105620] Updated weights for policy 1, policy_version 403818 (0.0005) [2023-12-26 18:18:03,604][105620] Updated weights for policy 1, policy_version 403828 (0.0005) [2023-12-26 18:18:03,650][105620] Updated weights for policy 1, policy_version 403838 (0.0005) [2023-12-26 18:18:04,129][105692] Updated weights for policy 0, policy_version 403504 (0.0009) [2023-12-26 18:18:04,184][105692] Updated weights for policy 0, policy_version 403514 (0.0010) [2023-12-26 18:18:04,247][105692] Updated weights for policy 0, policy_version 403524 (0.0011) [2023-12-26 18:18:04,279][105620] Updated weights for policy 1, policy_version 403848 (0.0009) [2023-12-26 18:18:04,339][105620] Updated weights for policy 1, policy_version 403858 (0.0010) [2023-12-26 18:18:04,401][105620] Updated weights for policy 1, policy_version 403868 (0.0010) [2023-12-26 18:18:04,952][105692] Updated weights for policy 0, policy_version 403534 (0.0007) [2023-12-26 18:18:05,013][105692] Updated weights for policy 0, policy_version 403544 (0.0005) [2023-12-26 18:18:05,071][105692] Updated weights for policy 0, policy_version 403554 (0.0006) [2023-12-26 18:18:05,113][105620] Updated weights for policy 1, policy_version 403878 (0.0011) [2023-12-26 18:18:05,170][105620] Updated weights for policy 1, policy_version 403888 (0.0009) [2023-12-26 18:18:05,227][105620] Updated weights for policy 1, policy_version 403898 (0.0009) [2023-12-26 18:18:05,611][105692] Updated weights for policy 0, policy_version 403564 (0.0005) [2023-12-26 18:18:05,659][105692] Updated weights for policy 0, policy_version 403574 (0.0005) [2023-12-26 18:18:05,713][105692] Updated weights for policy 0, policy_version 403584 (0.0005) [2023-12-26 18:18:05,917][105620] Updated weights for policy 1, policy_version 403908 (0.0010) [2023-12-26 18:18:05,964][105620] Updated weights for policy 1, policy_version 403918 (0.0010) [2023-12-26 18:18:06,016][105620] Updated weights for policy 1, policy_version 403928 (0.0010) [2023-12-26 18:18:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 206741504. Throughput: 0: 9579.9, 1: 10034.6. Samples: 206730732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:18:06,063][104569] Avg episode reward: [(0, '7737.002'), (1, '8917.628')] [2023-12-26 18:18:06,271][105692] Updated weights for policy 0, policy_version 403594 (0.0005) [2023-12-26 18:18:06,337][105692] Updated weights for policy 0, policy_version 403604 (0.0006) [2023-12-26 18:18:06,400][105692] Updated weights for policy 0, policy_version 403614 (0.0010) [2023-12-26 18:18:06,463][105692] Updated weights for policy 0, policy_version 403624 (0.0011) [2023-12-26 18:18:06,769][105620] Updated weights for policy 1, policy_version 403938 (0.0009) [2023-12-26 18:18:06,840][105620] Updated weights for policy 1, policy_version 403948 (0.0005) [2023-12-26 18:18:06,911][105620] Updated weights for policy 1, policy_version 403958 (0.0005) [2023-12-26 18:18:06,974][105620] Updated weights for policy 1, policy_version 403968 (0.0005) [2023-12-26 18:18:07,202][105692] Updated weights for policy 0, policy_version 403634 (0.0005) [2023-12-26 18:18:07,252][105692] Updated weights for policy 0, policy_version 403644 (0.0005) [2023-12-26 18:18:07,298][105692] Updated weights for policy 0, policy_version 403654 (0.0005) [2023-12-26 18:18:07,585][105620] Updated weights for policy 1, policy_version 403978 (0.0010) [2023-12-26 18:18:07,637][105620] Updated weights for policy 1, policy_version 403988 (0.0010) [2023-12-26 18:18:07,698][105620] Updated weights for policy 1, policy_version 403998 (0.0011) [2023-12-26 18:18:07,853][105692] Updated weights for policy 0, policy_version 403664 (0.0009) [2023-12-26 18:18:07,907][105692] Updated weights for policy 0, policy_version 403674 (0.0010) [2023-12-26 18:18:07,964][105692] Updated weights for policy 0, policy_version 403684 (0.0010) [2023-12-26 18:18:08,450][105620] Updated weights for policy 1, policy_version 404008 (0.0011) [2023-12-26 18:18:08,511][105620] Updated weights for policy 1, policy_version 404018 (0.0010) [2023-12-26 18:18:08,577][105620] Updated weights for policy 1, policy_version 404028 (0.0010) [2023-12-26 18:18:08,730][105692] Updated weights for policy 0, policy_version 403694 (0.0010) [2023-12-26 18:18:08,786][105692] Updated weights for policy 0, policy_version 403704 (0.0010) [2023-12-26 18:18:08,839][105692] Updated weights for policy 0, policy_version 403714 (0.0010) [2023-12-26 18:18:09,268][105620] Updated weights for policy 1, policy_version 404038 (0.0011) [2023-12-26 18:18:09,321][105620] Updated weights for policy 1, policy_version 404048 (0.0010) [2023-12-26 18:18:09,390][105620] Updated weights for policy 1, policy_version 404058 (0.0010) [2023-12-26 18:18:09,572][105692] Updated weights for policy 0, policy_version 403724 (0.0010) [2023-12-26 18:18:09,636][105692] Updated weights for policy 0, policy_version 403734 (0.0008) [2023-12-26 18:18:09,702][105692] Updated weights for policy 0, policy_version 403744 (0.0008) [2023-12-26 18:18:10,165][105620] Updated weights for policy 1, policy_version 404068 (0.0012) [2023-12-26 18:18:10,231][105620] Updated weights for policy 1, policy_version 404078 (0.0011) [2023-12-26 18:18:10,290][105620] Updated weights for policy 1, policy_version 404088 (0.0011) [2023-12-26 18:18:10,467][105692] Updated weights for policy 0, policy_version 403754 (0.0008) [2023-12-26 18:18:10,519][105692] Updated weights for policy 0, policy_version 403764 (0.0008) [2023-12-26 18:18:10,569][105692] Updated weights for policy 0, policy_version 403774 (0.0008) [2023-12-26 18:18:10,636][105692] Updated weights for policy 0, policy_version 403784 (0.0008) [2023-12-26 18:18:11,048][105620] Updated weights for policy 1, policy_version 404098 (0.0011) [2023-12-26 18:18:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 206839808. Throughput: 0: 9706.9, 1: 10010.7. Samples: 206851304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:18:11,062][104569] Avg episode reward: [(0, '7822.047'), (1, '8918.410')] [2023-12-26 18:18:11,111][105620] Updated weights for policy 1, policy_version 404108 (0.0011) [2023-12-26 18:18:11,190][105620] Updated weights for policy 1, policy_version 404118 (0.0009) [2023-12-26 18:18:11,251][105620] Updated weights for policy 1, policy_version 404128 (0.0009) [2023-12-26 18:18:11,308][105692] Updated weights for policy 0, policy_version 403794 (0.0007) [2023-12-26 18:18:11,375][105692] Updated weights for policy 0, policy_version 403804 (0.0008) [2023-12-26 18:18:11,428][105692] Updated weights for policy 0, policy_version 403814 (0.0007) [2023-12-26 18:18:12,018][105620] Updated weights for policy 1, policy_version 404138 (0.0007) [2023-12-26 18:18:12,069][105620] Updated weights for policy 1, policy_version 404148 (0.0006) [2023-12-26 18:18:12,117][105620] Updated weights for policy 1, policy_version 404158 (0.0006) [2023-12-26 18:18:12,204][105692] Updated weights for policy 0, policy_version 403824 (0.0009) [2023-12-26 18:18:12,265][105692] Updated weights for policy 0, policy_version 403834 (0.0011) [2023-12-26 18:18:12,329][105692] Updated weights for policy 0, policy_version 403844 (0.0010) [2023-12-26 18:18:12,833][105620] Updated weights for policy 1, policy_version 404168 (0.0008) [2023-12-26 18:18:12,895][105620] Updated weights for policy 1, policy_version 404178 (0.0010) [2023-12-26 18:18:12,960][105620] Updated weights for policy 1, policy_version 404188 (0.0009) [2023-12-26 18:18:12,993][105692] Updated weights for policy 0, policy_version 403854 (0.0010) [2023-12-26 18:18:13,058][105692] Updated weights for policy 0, policy_version 403864 (0.0011) [2023-12-26 18:18:13,113][105692] Updated weights for policy 0, policy_version 403874 (0.0010) [2023-12-26 18:18:13,710][105692] Updated weights for policy 0, policy_version 403884 (0.0010) [2023-12-26 18:18:13,759][105620] Updated weights for policy 1, policy_version 404198 (0.0008) [2023-12-26 18:18:13,768][105692] Updated weights for policy 0, policy_version 403894 (0.0010) [2023-12-26 18:18:13,810][105620] Updated weights for policy 1, policy_version 404208 (0.0006) [2023-12-26 18:18:13,834][105692] Updated weights for policy 0, policy_version 403904 (0.0010) [2023-12-26 18:18:13,865][105620] Updated weights for policy 1, policy_version 404218 (0.0007) [2023-12-26 18:18:14,559][105692] Updated weights for policy 0, policy_version 403914 (0.0010) [2023-12-26 18:18:14,570][105620] Updated weights for policy 1, policy_version 404228 (0.0006) [2023-12-26 18:18:14,621][105692] Updated weights for policy 0, policy_version 403924 (0.0010) [2023-12-26 18:18:14,624][105620] Updated weights for policy 1, policy_version 404238 (0.0008) [2023-12-26 18:18:14,673][105692] Updated weights for policy 0, policy_version 403934 (0.0010) [2023-12-26 18:18:14,683][105620] Updated weights for policy 1, policy_version 404248 (0.0005) [2023-12-26 18:18:14,727][105692] Updated weights for policy 0, policy_version 403944 (0.0010) [2023-12-26 18:18:15,484][105692] Updated weights for policy 0, policy_version 403954 (0.0011) [2023-12-26 18:18:15,490][105620] Updated weights for policy 1, policy_version 404258 (0.0006) [2023-12-26 18:18:15,548][105620] Updated weights for policy 1, policy_version 404268 (0.0005) [2023-12-26 18:18:15,550][105692] Updated weights for policy 0, policy_version 403964 (0.0011) [2023-12-26 18:18:15,610][105620] Updated weights for policy 1, policy_version 404278 (0.0005) [2023-12-26 18:18:15,612][105692] Updated weights for policy 0, policy_version 403974 (0.0010) [2023-12-26 18:18:15,668][105620] Updated weights for policy 1, policy_version 404288 (0.0007) [2023-12-26 18:18:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.7, 300 sec: 19688.6). Total num frames: 206938112. Throughput: 0: 9678.8, 1: 9890.6. Samples: 206909484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:18:16,063][104569] Avg episode reward: [(0, '8447.439'), (1, '8823.805')] [2023-12-26 18:18:16,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000404288_103505920.pth... [2023-12-26 18:18:16,073][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000403976_103432192.pth... [2023-12-26 18:18:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000403136_103211008.pth [2023-12-26 18:18:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000402856_103145472.pth [2023-12-26 18:18:16,328][105692] Updated weights for policy 0, policy_version 403984 (0.0007) [2023-12-26 18:18:16,345][105620] Updated weights for policy 1, policy_version 404298 (0.0007) [2023-12-26 18:18:16,387][105692] Updated weights for policy 0, policy_version 403994 (0.0007) [2023-12-26 18:18:16,401][105620] Updated weights for policy 1, policy_version 404308 (0.0008) [2023-12-26 18:18:16,446][105620] Updated weights for policy 1, policy_version 404318 (0.0007) [2023-12-26 18:18:16,449][105692] Updated weights for policy 0, policy_version 404004 (0.0008) [2023-12-26 18:18:17,100][105692] Updated weights for policy 0, policy_version 404014 (0.0007) [2023-12-26 18:18:17,148][105692] Updated weights for policy 0, policy_version 404024 (0.0005) [2023-12-26 18:18:17,201][105692] Updated weights for policy 0, policy_version 404034 (0.0005) [2023-12-26 18:18:17,230][105585] KL-divergence is very high: 181.6714 [2023-12-26 18:18:17,271][105620] Updated weights for policy 1, policy_version 404328 (0.0009) [2023-12-26 18:18:17,332][105620] Updated weights for policy 1, policy_version 404338 (0.0009) [2023-12-26 18:18:17,393][105620] Updated weights for policy 1, policy_version 404348 (0.0009) [2023-12-26 18:18:17,813][105692] Updated weights for policy 0, policy_version 404044 (0.0007) [2023-12-26 18:18:17,871][105692] Updated weights for policy 0, policy_version 404054 (0.0007) [2023-12-26 18:18:17,934][105692] Updated weights for policy 0, policy_version 404064 (0.0007) [2023-12-26 18:18:18,141][105620] Updated weights for policy 1, policy_version 404358 (0.0007) [2023-12-26 18:18:18,192][105620] Updated weights for policy 1, policy_version 404369 (0.0010) [2023-12-26 18:18:18,248][105620] Updated weights for policy 1, policy_version 404379 (0.0009) [2023-12-26 18:18:18,607][105692] Updated weights for policy 0, policy_version 404074 (0.0008) [2023-12-26 18:18:18,662][105692] Updated weights for policy 0, policy_version 404084 (0.0005) [2023-12-26 18:18:18,721][105692] Updated weights for policy 0, policy_version 404094 (0.0005) [2023-12-26 18:18:18,777][105692] Updated weights for policy 0, policy_version 404104 (0.0005) [2023-12-26 18:18:19,092][105620] Updated weights for policy 1, policy_version 404389 (0.0009) [2023-12-26 18:18:19,160][105620] Updated weights for policy 1, policy_version 404399 (0.0008) [2023-12-26 18:18:19,226][105620] Updated weights for policy 1, policy_version 404409 (0.0008) [2023-12-26 18:18:19,418][105692] Updated weights for policy 0, policy_version 404114 (0.0005) [2023-12-26 18:18:19,467][105692] Updated weights for policy 0, policy_version 404124 (0.0006) [2023-12-26 18:18:19,533][105692] Updated weights for policy 0, policy_version 404134 (0.0009) [2023-12-26 18:18:19,975][105620] Updated weights for policy 1, policy_version 404419 (0.0009) [2023-12-26 18:18:20,030][105620] Updated weights for policy 1, policy_version 404429 (0.0008) [2023-12-26 18:18:20,079][105620] Updated weights for policy 1, policy_version 404439 (0.0008) [2023-12-26 18:18:20,319][105692] Updated weights for policy 0, policy_version 404144 (0.0010) [2023-12-26 18:18:20,382][105692] Updated weights for policy 0, policy_version 404155 (0.0009) [2023-12-26 18:18:20,434][105692] Updated weights for policy 0, policy_version 404165 (0.0009) [2023-12-26 18:18:20,751][105620] Updated weights for policy 1, policy_version 404449 (0.0008) [2023-12-26 18:18:20,817][105620] Updated weights for policy 1, policy_version 404459 (0.0009) [2023-12-26 18:18:20,871][105620] Updated weights for policy 1, policy_version 404469 (0.0010) [2023-12-26 18:18:20,938][105620] Updated weights for policy 1, policy_version 404479 (0.0009) [2023-12-26 18:18:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 207036416. Throughput: 0: 9794.4, 1: 9717.6. Samples: 207024984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:18:21,063][104569] Avg episode reward: [(0, '8544.551'), (1, '8639.621')] [2023-12-26 18:18:21,195][105692] Updated weights for policy 0, policy_version 404175 (0.0008) [2023-12-26 18:18:21,264][105692] Updated weights for policy 0, policy_version 404185 (0.0008) [2023-12-26 18:18:21,332][105692] Updated weights for policy 0, policy_version 404195 (0.0008) [2023-12-26 18:18:21,743][105620] Updated weights for policy 1, policy_version 404489 (0.0009) [2023-12-26 18:18:21,800][105620] Updated weights for policy 1, policy_version 404499 (0.0006) [2023-12-26 18:18:21,855][105620] Updated weights for policy 1, policy_version 404509 (0.0006) [2023-12-26 18:18:22,121][105692] Updated weights for policy 0, policy_version 404205 (0.0009) [2023-12-26 18:18:22,187][105692] Updated weights for policy 0, policy_version 404215 (0.0009) [2023-12-26 18:18:22,249][105692] Updated weights for policy 0, policy_version 404225 (0.0009) [2023-12-26 18:18:22,527][105620] Updated weights for policy 1, policy_version 404519 (0.0006) [2023-12-26 18:18:22,596][105620] Updated weights for policy 1, policy_version 404529 (0.0008) [2023-12-26 18:18:22,660][105620] Updated weights for policy 1, policy_version 404539 (0.0007) [2023-12-26 18:18:22,938][105692] Updated weights for policy 0, policy_version 404235 (0.0009) [2023-12-26 18:18:22,998][105692] Updated weights for policy 0, policy_version 404245 (0.0009) [2023-12-26 18:18:23,055][105692] Updated weights for policy 0, policy_version 404255 (0.0010) [2023-12-26 18:18:23,210][105620] Updated weights for policy 1, policy_version 404549 (0.0006) [2023-12-26 18:18:23,272][105620] Updated weights for policy 1, policy_version 404559 (0.0006) [2023-12-26 18:18:23,339][105620] Updated weights for policy 1, policy_version 404569 (0.0006) [2023-12-26 18:18:23,811][105692] Updated weights for policy 0, policy_version 404265 (0.0010) [2023-12-26 18:18:23,855][105692] Updated weights for policy 0, policy_version 404275 (0.0005) [2023-12-26 18:18:23,898][105692] Updated weights for policy 0, policy_version 404285 (0.0005) [2023-12-26 18:18:23,944][105692] Updated weights for policy 0, policy_version 404295 (0.0005) [2023-12-26 18:18:24,038][105620] Updated weights for policy 1, policy_version 404579 (0.0008) [2023-12-26 18:18:24,090][105620] Updated weights for policy 1, policy_version 404589 (0.0009) [2023-12-26 18:18:24,150][105620] Updated weights for policy 1, policy_version 404599 (0.0010) [2023-12-26 18:18:24,517][105692] Updated weights for policy 0, policy_version 404305 (0.0009) [2023-12-26 18:18:24,576][105692] Updated weights for policy 0, policy_version 404315 (0.0010) [2023-12-26 18:18:24,630][105692] Updated weights for policy 0, policy_version 404325 (0.0010) [2023-12-26 18:18:24,853][105620] Updated weights for policy 1, policy_version 404609 (0.0008) [2023-12-26 18:18:24,915][105620] Updated weights for policy 1, policy_version 404619 (0.0005) [2023-12-26 18:18:24,977][105620] Updated weights for policy 1, policy_version 404629 (0.0008) [2023-12-26 18:18:25,024][105620] Updated weights for policy 1, policy_version 404639 (0.0008) [2023-12-26 18:18:25,359][105692] Updated weights for policy 0, policy_version 404336 (0.0011) [2023-12-26 18:18:25,420][105692] Updated weights for policy 0, policy_version 404346 (0.0009) [2023-12-26 18:18:25,481][105692] Updated weights for policy 0, policy_version 404356 (0.0009) [2023-12-26 18:18:25,724][105620] Updated weights for policy 1, policy_version 404649 (0.0007) [2023-12-26 18:18:25,768][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000008 [2023-12-26 18:18:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 207134720. Throughput: 0: 9765.8, 1: 9746.0. Samples: 207142512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:18:26,062][104569] Avg episode reward: [(0, '5971.943'), (1, '8635.979')] [2023-12-26 18:18:26,257][105692] Updated weights for policy 0, policy_version 404366 (0.0008) [2023-12-26 18:18:26,316][105692] Updated weights for policy 0, policy_version 404376 (0.0009) [2023-12-26 18:18:26,371][105692] Updated weights for policy 0, policy_version 404386 (0.0009) [2023-12-26 18:18:26,520][105620] Updated weights for policy 1, policy_version 404659 (0.0009) [2023-12-26 18:18:26,573][105620] Updated weights for policy 1, policy_version 404669 (0.0009) [2023-12-26 18:18:26,623][105620] Updated weights for policy 1, policy_version 404679 (0.0009) [2023-12-26 18:18:27,113][105692] Updated weights for policy 0, policy_version 404396 (0.0009) [2023-12-26 18:18:27,170][105692] Updated weights for policy 0, policy_version 404406 (0.0009) [2023-12-26 18:18:27,219][105692] Updated weights for policy 0, policy_version 404416 (0.0008) [2023-12-26 18:18:27,382][105620] Updated weights for policy 1, policy_version 404689 (0.0009) [2023-12-26 18:18:27,446][105620] Updated weights for policy 1, policy_version 404699 (0.0006) [2023-12-26 18:18:27,509][105620] Updated weights for policy 1, policy_version 404709 (0.0007) [2023-12-26 18:18:27,580][105620] Updated weights for policy 1, policy_version 404719 (0.0008) [2023-12-26 18:18:28,018][105692] Updated weights for policy 0, policy_version 404426 (0.0009) [2023-12-26 18:18:28,078][105692] Updated weights for policy 0, policy_version 404436 (0.0008) [2023-12-26 18:18:28,143][105692] Updated weights for policy 0, policy_version 404446 (0.0008) [2023-12-26 18:18:28,202][105692] Updated weights for policy 0, policy_version 404456 (0.0008) [2023-12-26 18:18:28,230][105620] Updated weights for policy 1, policy_version 404729 (0.0010) [2023-12-26 18:18:28,278][105620] Updated weights for policy 1, policy_version 404739 (0.0010) [2023-12-26 18:18:28,323][105620] Updated weights for policy 1, policy_version 404749 (0.0010) [2023-12-26 18:18:28,970][105692] Updated weights for policy 0, policy_version 404466 (0.0009) [2023-12-26 18:18:29,001][105620] Updated weights for policy 1, policy_version 404759 (0.0006) [2023-12-26 18:18:29,027][105692] Updated weights for policy 0, policy_version 404476 (0.0007) [2023-12-26 18:18:29,048][105620] Updated weights for policy 1, policy_version 404769 (0.0009) [2023-12-26 18:18:29,084][105692] Updated weights for policy 0, policy_version 404486 (0.0008) [2023-12-26 18:18:29,103][105620] Updated weights for policy 1, policy_version 404779 (0.0010) [2023-12-26 18:18:29,746][105620] Updated weights for policy 1, policy_version 404789 (0.0008) [2023-12-26 18:18:29,810][105620] Updated weights for policy 1, policy_version 404799 (0.0005) [2023-12-26 18:18:29,880][105620] Updated weights for policy 1, policy_version 404809 (0.0007) [2023-12-26 18:18:29,920][105692] Updated weights for policy 0, policy_version 404496 (0.0009) [2023-12-26 18:18:29,986][105585] KL-divergence is very high: 106.1185 [2023-12-26 18:18:29,993][105692] Updated weights for policy 0, policy_version 404506 (0.0008) [2023-12-26 18:18:29,993][105585] KL-divergence is very high: 105.8353 [2023-12-26 18:18:30,000][105585] KL-divergence is very high: 103.7463 [2023-12-26 18:18:30,057][105692] Updated weights for policy 0, policy_version 404516 (0.0009) [2023-12-26 18:18:30,553][105620] Updated weights for policy 1, policy_version 404819 (0.0007) [2023-12-26 18:18:30,604][105620] Updated weights for policy 1, policy_version 404829 (0.0010) [2023-12-26 18:18:30,654][105620] Updated weights for policy 1, policy_version 404839 (0.0010) [2023-12-26 18:18:30,806][105692] Updated weights for policy 0, policy_version 404526 (0.0008) [2023-12-26 18:18:30,850][105692] Updated weights for policy 0, policy_version 404536 (0.0008) [2023-12-26 18:18:30,904][105692] Updated weights for policy 0, policy_version 404546 (0.0008) [2023-12-26 18:18:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19688.6). Total num frames: 207233024. Throughput: 0: 9778.6, 1: 9795.0. Samples: 207199948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:18:31,063][104569] Avg episode reward: [(0, '3404.053'), (1, '8814.468')] [2023-12-26 18:18:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000404552_103579648.pth... [2023-12-26 18:18:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000404848_103653376.pth... [2023-12-26 18:18:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000403744_103366656.pth [2023-12-26 18:18:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000403400_103284736.pth [2023-12-26 18:18:31,421][105620] Updated weights for policy 1, policy_version 404849 (0.0010) [2023-12-26 18:18:31,480][105620] Updated weights for policy 1, policy_version 404859 (0.0010) [2023-12-26 18:18:31,546][105620] Updated weights for policy 1, policy_version 404869 (0.0011) [2023-12-26 18:18:31,606][105620] Updated weights for policy 1, policy_version 404879 (0.0011) [2023-12-26 18:18:31,696][105692] Updated weights for policy 0, policy_version 404556 (0.0008) [2023-12-26 18:18:31,768][105692] Updated weights for policy 0, policy_version 404566 (0.0007) [2023-12-26 18:18:31,830][105692] Updated weights for policy 0, policy_version 404576 (0.0010) [2023-12-26 18:18:32,390][105620] Updated weights for policy 1, policy_version 404889 (0.0011) [2023-12-26 18:18:32,449][105620] Updated weights for policy 1, policy_version 404899 (0.0010) [2023-12-26 18:18:32,515][105620] Updated weights for policy 1, policy_version 404909 (0.0008) [2023-12-26 18:18:32,546][105692] Updated weights for policy 0, policy_version 404586 (0.0011) [2023-12-26 18:18:32,601][105692] Updated weights for policy 0, policy_version 404596 (0.0010) [2023-12-26 18:18:32,663][105692] Updated weights for policy 0, policy_version 404606 (0.0010) [2023-12-26 18:18:32,719][105692] Updated weights for policy 0, policy_version 404616 (0.0010) [2023-12-26 18:18:33,258][105620] Updated weights for policy 1, policy_version 404919 (0.0009) [2023-12-26 18:18:33,339][105620] Updated weights for policy 1, policy_version 404929 (0.0009) [2023-12-26 18:18:33,398][105620] Updated weights for policy 1, policy_version 404939 (0.0008) [2023-12-26 18:18:33,445][105692] Updated weights for policy 0, policy_version 404626 (0.0007) [2023-12-26 18:18:33,493][105692] Updated weights for policy 0, policy_version 404636 (0.0009) [2023-12-26 18:18:33,545][105692] Updated weights for policy 0, policy_version 404646 (0.0009) [2023-12-26 18:18:34,136][105620] Updated weights for policy 1, policy_version 404949 (0.0008) [2023-12-26 18:18:34,205][105620] Updated weights for policy 1, policy_version 404959 (0.0007) [2023-12-26 18:18:34,258][105620] Updated weights for policy 1, policy_version 404969 (0.0008) [2023-12-26 18:18:34,315][105692] Updated weights for policy 0, policy_version 404656 (0.0010) [2023-12-26 18:18:34,375][105692] Updated weights for policy 0, policy_version 404666 (0.0011) [2023-12-26 18:18:34,431][105692] Updated weights for policy 0, policy_version 404676 (0.0011) [2023-12-26 18:18:35,007][105692] Updated weights for policy 0, policy_version 404686 (0.0008) [2023-12-26 18:18:35,055][105692] Updated weights for policy 0, policy_version 404696 (0.0010) [2023-12-26 18:18:35,087][105620] Updated weights for policy 1, policy_version 404979 (0.0007) [2023-12-26 18:18:35,112][105692] Updated weights for policy 0, policy_version 404706 (0.0008) [2023-12-26 18:18:35,149][105620] Updated weights for policy 1, policy_version 404989 (0.0007) [2023-12-26 18:18:35,215][105620] Updated weights for policy 1, policy_version 404999 (0.0008) [2023-12-26 18:18:35,750][105692] Updated weights for policy 0, policy_version 404716 (0.0009) [2023-12-26 18:18:35,802][105692] Updated weights for policy 0, policy_version 404726 (0.0009) [2023-12-26 18:18:35,863][105692] Updated weights for policy 0, policy_version 404736 (0.0007) [2023-12-26 18:18:36,003][105620] Updated weights for policy 1, policy_version 405009 (0.0009) [2023-12-26 18:18:36,051][105620] Updated weights for policy 1, policy_version 405019 (0.0008) [2023-12-26 18:18:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.4, 300 sec: 19660.8). Total num frames: 207323136. Throughput: 0: 9666.0, 1: 9772.9. Samples: 207311884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:18:36,062][104569] Avg episode reward: [(0, '2091.994'), (1, '8715.493')] [2023-12-26 18:18:36,099][105620] Updated weights for policy 1, policy_version 405029 (0.0008) [2023-12-26 18:18:36,164][105620] Updated weights for policy 1, policy_version 405039 (0.0009) [2023-12-26 18:18:36,560][105692] Updated weights for policy 0, policy_version 404746 (0.0006) [2023-12-26 18:18:36,612][105692] Updated weights for policy 0, policy_version 404756 (0.0010) [2023-12-26 18:18:36,664][105692] Updated weights for policy 0, policy_version 404766 (0.0010) [2023-12-26 18:18:36,723][105692] Updated weights for policy 0, policy_version 404776 (0.0010) [2023-12-26 18:18:36,925][105620] Updated weights for policy 1, policy_version 405049 (0.0008) [2023-12-26 18:18:36,990][105620] Updated weights for policy 1, policy_version 405059 (0.0007) [2023-12-26 18:18:37,052][105620] Updated weights for policy 1, policy_version 405069 (0.0006) [2023-12-26 18:18:37,482][105692] Updated weights for policy 0, policy_version 404786 (0.0005) [2023-12-26 18:18:37,532][105692] Updated weights for policy 0, policy_version 404796 (0.0005) [2023-12-26 18:18:37,538][105585] KL-divergence is very high: 113.6672 [2023-12-26 18:18:37,587][105692] Updated weights for policy 0, policy_version 404806 (0.0005) [2023-12-26 18:18:37,794][105620] Updated weights for policy 1, policy_version 405079 (0.0008) [2023-12-26 18:18:37,862][105620] Updated weights for policy 1, policy_version 405089 (0.0009) [2023-12-26 18:18:37,919][105620] Updated weights for policy 1, policy_version 405099 (0.0008) [2023-12-26 18:18:38,275][105692] Updated weights for policy 0, policy_version 404816 (0.0009) [2023-12-26 18:18:38,328][105692] Updated weights for policy 0, policy_version 404826 (0.0010) [2023-12-26 18:18:38,386][105692] Updated weights for policy 0, policy_version 404836 (0.0008) [2023-12-26 18:18:38,699][105620] Updated weights for policy 1, policy_version 405109 (0.0009) [2023-12-26 18:18:38,753][105620] Updated weights for policy 1, policy_version 405119 (0.0010) [2023-12-26 18:18:38,812][105620] Updated weights for policy 1, policy_version 405130 (0.0010) [2023-12-26 18:18:38,982][105692] Updated weights for policy 0, policy_version 404846 (0.0007) [2023-12-26 18:18:39,053][105692] Updated weights for policy 0, policy_version 404856 (0.0007) [2023-12-26 18:18:39,107][105692] Updated weights for policy 0, policy_version 404866 (0.0010) [2023-12-26 18:18:39,681][105620] Updated weights for policy 1, policy_version 405140 (0.0009) [2023-12-26 18:18:39,738][105620] Updated weights for policy 1, policy_version 405150 (0.0008) [2023-12-26 18:18:39,794][105620] Updated weights for policy 1, policy_version 405160 (0.0007) [2023-12-26 18:18:39,796][105692] Updated weights for policy 0, policy_version 404876 (0.0010) [2023-12-26 18:18:39,819][105585] KL-divergence is very high: 190.9733 [2023-12-26 18:18:39,841][105585] KL-divergence is very high: 201.7242 [2023-12-26 18:18:39,864][105692] Updated weights for policy 0, policy_version 404886 (0.0010) [2023-12-26 18:18:39,877][105585] KL-divergence is very high: 268.5226 [2023-12-26 18:18:39,898][105585] KL-divergence is very high: 181.8786 [2023-12-26 18:18:39,932][105692] Updated weights for policy 0, policy_version 404896 (0.0011) [2023-12-26 18:18:39,932][105585] KL-divergence is very high: 211.7769 [2023-12-26 18:18:40,540][105692] Updated weights for policy 0, policy_version 404906 (0.0009) [2023-12-26 18:18:40,584][105692] Updated weights for policy 0, policy_version 404916 (0.0005) [2023-12-26 18:18:40,632][105692] Updated weights for policy 0, policy_version 404926 (0.0005) [2023-12-26 18:18:40,654][105620] Updated weights for policy 1, policy_version 405170 (0.0007) [2023-12-26 18:18:40,694][105692] Updated weights for policy 0, policy_version 404936 (0.0008) [2023-12-26 18:18:40,713][105620] Updated weights for policy 1, policy_version 405180 (0.0009) [2023-12-26 18:18:40,770][105620] Updated weights for policy 1, policy_version 405190 (0.0008) [2023-12-26 18:18:40,820][105620] Updated weights for policy 1, policy_version 405200 (0.0008) [2023-12-26 18:18:41,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 207421440. Throughput: 0: 9793.9, 1: 9570.2. Samples: 207427708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:18:41,062][104569] Avg episode reward: [(0, '5253.687'), (1, '8808.519')] [2023-12-26 18:18:41,408][105692] Updated weights for policy 0, policy_version 404946 (0.0009) [2023-12-26 18:18:41,461][105692] Updated weights for policy 0, policy_version 404956 (0.0006) [2023-12-26 18:18:41,510][105692] Updated weights for policy 0, policy_version 404966 (0.0011) [2023-12-26 18:18:41,634][105620] Updated weights for policy 1, policy_version 405210 (0.0008) [2023-12-26 18:18:41,691][105620] Updated weights for policy 1, policy_version 405220 (0.0008) [2023-12-26 18:18:41,762][105620] Updated weights for policy 1, policy_version 405230 (0.0008) [2023-12-26 18:18:42,274][105692] Updated weights for policy 0, policy_version 404976 (0.0011) [2023-12-26 18:18:42,335][105692] Updated weights for policy 0, policy_version 404986 (0.0011) [2023-12-26 18:18:42,391][105692] Updated weights for policy 0, policy_version 404996 (0.0011) [2023-12-26 18:18:42,523][105620] Updated weights for policy 1, policy_version 405240 (0.0006) [2023-12-26 18:18:42,590][105620] Updated weights for policy 1, policy_version 405250 (0.0005) [2023-12-26 18:18:42,657][105620] Updated weights for policy 1, policy_version 405260 (0.0007) [2023-12-26 18:18:43,182][105692] Updated weights for policy 0, policy_version 405006 (0.0011) [2023-12-26 18:18:43,236][105692] Updated weights for policy 0, policy_version 405016 (0.0007) [2023-12-26 18:18:43,249][105620] Updated weights for policy 1, policy_version 405270 (0.0007) [2023-12-26 18:18:43,287][105692] Updated weights for policy 0, policy_version 405026 (0.0008) [2023-12-26 18:18:43,297][105620] Updated weights for policy 1, policy_version 405280 (0.0005) [2023-12-26 18:18:43,350][105620] Updated weights for policy 1, policy_version 405290 (0.0005) [2023-12-26 18:18:43,906][105620] Updated weights for policy 1, policy_version 405300 (0.0007) [2023-12-26 18:18:43,976][105620] Updated weights for policy 1, policy_version 405310 (0.0010) [2023-12-26 18:18:43,978][105692] Updated weights for policy 0, policy_version 405036 (0.0008) [2023-12-26 18:18:44,003][105585] KL-divergence is very high: 189.2246 [2023-12-26 18:18:44,013][105585] KL-divergence is very high: 621.4868 [2023-12-26 18:18:44,025][105620] Updated weights for policy 1, policy_version 405320 (0.0008) [2023-12-26 18:18:44,029][105692] Updated weights for policy 0, policy_version 405046 (0.0005) [2023-12-26 18:18:44,046][105585] KL-divergence is very high: 399.4866 [2023-12-26 18:18:44,057][105585] KL-divergence is very high: 1050.9695 [2023-12-26 18:18:44,085][105692] Updated weights for policy 0, policy_version 405056 (0.0005) [2023-12-26 18:18:44,090][105585] KL-divergence is very high: 412.1613 [2023-12-26 18:18:44,104][105585] KL-divergence is very high: 1129.1039 [2023-12-26 18:18:44,635][105692] Updated weights for policy 0, policy_version 405066 (0.0005) [2023-12-26 18:18:44,696][105692] Updated weights for policy 0, policy_version 405076 (0.0007) [2023-12-26 18:18:44,756][105692] Updated weights for policy 0, policy_version 405086 (0.0009) [2023-12-26 18:18:44,824][105692] Updated weights for policy 0, policy_version 405096 (0.0009) [2023-12-26 18:18:44,840][105620] Updated weights for policy 1, policy_version 405330 (0.0009) [2023-12-26 18:18:44,895][105620] Updated weights for policy 1, policy_version 405340 (0.0009) [2023-12-26 18:18:44,947][105620] Updated weights for policy 1, policy_version 405350 (0.0008) [2023-12-26 18:18:45,007][105620] Updated weights for policy 1, policy_version 405360 (0.0007) [2023-12-26 18:18:45,389][105692] Updated weights for policy 0, policy_version 405106 (0.0006) [2023-12-26 18:18:45,438][105692] Updated weights for policy 0, policy_version 405116 (0.0005) [2023-12-26 18:18:45,487][105692] Updated weights for policy 0, policy_version 405126 (0.0005) [2023-12-26 18:18:45,840][105620] Updated weights for policy 1, policy_version 405370 (0.0008) [2023-12-26 18:18:45,842][105586] KL-divergence is very high: 196.4976 [2023-12-26 18:18:45,882][105586] KL-divergence is very high: 201.4637 [2023-12-26 18:18:45,888][105586] KL-divergence is very high: 357.2653 [2023-12-26 18:18:45,899][105620] Updated weights for policy 1, policy_version 405380 (0.0008) [2023-12-26 18:18:45,934][105586] KL-divergence is very high: 221.2245 [2023-12-26 18:18:45,941][105586] KL-divergence is very high: 389.9614 [2023-12-26 18:18:45,962][105620] Updated weights for policy 1, policy_version 405390 (0.0008) [2023-12-26 18:18:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 207519744. Throughput: 0: 9773.2, 1: 9579.3. Samples: 207486764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:18:46,063][104569] Avg episode reward: [(0, '4150.631'), (1, '8454.940')] [2023-12-26 18:18:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000405128_103727104.pth... [2023-12-26 18:18:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000405392_103792640.pth... [2023-12-26 18:18:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000404288_103505920.pth [2023-12-26 18:18:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000403976_103432192.pth [2023-12-26 18:18:46,186][105692] Updated weights for policy 0, policy_version 405136 (0.0009) [2023-12-26 18:18:46,248][105692] Updated weights for policy 0, policy_version 405146 (0.0010) [2023-12-26 18:18:46,313][105692] Updated weights for policy 0, policy_version 405156 (0.0010) [2023-12-26 18:18:46,690][105586] KL-divergence is very high: 179.9746 [2023-12-26 18:18:46,726][105620] Updated weights for policy 1, policy_version 405400 (0.0006) [2023-12-26 18:18:46,733][105586] KL-divergence is very high: 168.7037 [2023-12-26 18:18:46,775][105586] KL-divergence is very high: 145.1663 [2023-12-26 18:18:46,781][105620] Updated weights for policy 1, policy_version 405410 (0.0005) [2023-12-26 18:18:46,822][105586] KL-divergence is very high: 126.8632 [2023-12-26 18:18:46,838][105620] Updated weights for policy 1, policy_version 405420 (0.0006) [2023-12-26 18:18:47,014][105692] Updated weights for policy 0, policy_version 405166 (0.0010) [2023-12-26 18:18:47,062][105692] Updated weights for policy 0, policy_version 405176 (0.0010) [2023-12-26 18:18:47,117][105692] Updated weights for policy 0, policy_version 405186 (0.0010) [2023-12-26 18:18:47,414][105620] Updated weights for policy 1, policy_version 405430 (0.0005) [2023-12-26 18:18:47,460][105620] Updated weights for policy 1, policy_version 405440 (0.0010) [2023-12-26 18:18:47,508][105620] Updated weights for policy 1, policy_version 405450 (0.0010) [2023-12-26 18:18:47,714][105692] Updated weights for policy 0, policy_version 405196 (0.0006) [2023-12-26 18:18:47,772][105692] Updated weights for policy 0, policy_version 405206 (0.0007) [2023-12-26 18:18:47,824][105692] Updated weights for policy 0, policy_version 405216 (0.0009) [2023-12-26 18:18:48,160][105620] Updated weights for policy 1, policy_version 405460 (0.0008) [2023-12-26 18:18:48,209][105620] Updated weights for policy 1, policy_version 405470 (0.0005) [2023-12-26 18:18:48,257][105620] Updated weights for policy 1, policy_version 405480 (0.0005) [2023-12-26 18:18:48,486][105692] Updated weights for policy 0, policy_version 405226 (0.0006) [2023-12-26 18:18:48,552][105692] Updated weights for policy 0, policy_version 405236 (0.0005) [2023-12-26 18:18:48,605][105692] Updated weights for policy 0, policy_version 405246 (0.0005) [2023-12-26 18:18:48,669][105692] Updated weights for policy 0, policy_version 405256 (0.0005) [2023-12-26 18:18:48,917][105620] Updated weights for policy 1, policy_version 405490 (0.0006) [2023-12-26 18:18:48,977][105620] Updated weights for policy 1, policy_version 405500 (0.0008) [2023-12-26 18:18:49,041][105620] Updated weights for policy 1, policy_version 405510 (0.0008) [2023-12-26 18:18:49,098][105620] Updated weights for policy 1, policy_version 405520 (0.0008) [2023-12-26 18:18:49,269][105692] Updated weights for policy 0, policy_version 405266 (0.0008) [2023-12-26 18:18:49,305][105585] KL-divergence is very high: 142.6501 [2023-12-26 18:18:49,314][105692] Updated weights for policy 0, policy_version 405276 (0.0010) [2023-12-26 18:18:49,349][105585] KL-divergence is very high: 135.0127 [2023-12-26 18:18:49,362][105585] KL-divergence is very high: 134.9375 [2023-12-26 18:18:49,376][105692] Updated weights for policy 0, policy_version 405286 (0.0010) [2023-12-26 18:18:49,881][105620] Updated weights for policy 1, policy_version 405530 (0.0006) [2023-12-26 18:18:49,949][105620] Updated weights for policy 1, policy_version 405540 (0.0007) [2023-12-26 18:18:50,008][105620] Updated weights for policy 1, policy_version 405550 (0.0007) [2023-12-26 18:18:50,153][105692] Updated weights for policy 0, policy_version 405296 (0.0011) [2023-12-26 18:18:50,215][105692] Updated weights for policy 0, policy_version 405306 (0.0010) [2023-12-26 18:18:50,276][105692] Updated weights for policy 0, policy_version 405316 (0.0010) [2023-12-26 18:18:50,714][105620] Updated weights for policy 1, policy_version 405560 (0.0008) [2023-12-26 18:18:50,780][105620] Updated weights for policy 1, policy_version 405570 (0.0009) [2023-12-26 18:18:50,835][105620] Updated weights for policy 1, policy_version 405580 (0.0008) [2023-12-26 18:18:51,005][105692] Updated weights for policy 0, policy_version 405326 (0.0011) [2023-12-26 18:18:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19660.8). Total num frames: 207618048. Throughput: 0: 9985.2, 1: 9546.0. Samples: 207609632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:18:51,063][104569] Avg episode reward: [(0, '7623.913'), (1, '8178.513')] [2023-12-26 18:18:51,070][105692] Updated weights for policy 0, policy_version 405336 (0.0009) [2023-12-26 18:18:51,135][105692] Updated weights for policy 0, policy_version 405346 (0.0009) [2023-12-26 18:18:51,640][105620] Updated weights for policy 1, policy_version 405590 (0.0008) [2023-12-26 18:18:51,706][105620] Updated weights for policy 1, policy_version 405600 (0.0008) [2023-12-26 18:18:51,769][105620] Updated weights for policy 1, policy_version 405610 (0.0008) [2023-12-26 18:18:51,796][105692] Updated weights for policy 0, policy_version 405356 (0.0007) [2023-12-26 18:18:51,855][105692] Updated weights for policy 0, policy_version 405366 (0.0010) [2023-12-26 18:18:51,909][105692] Updated weights for policy 0, policy_version 405376 (0.0010) [2023-12-26 18:18:52,469][105620] Updated weights for policy 1, policy_version 405620 (0.0006) [2023-12-26 18:18:52,533][105620] Updated weights for policy 1, policy_version 405630 (0.0006) [2023-12-26 18:18:52,601][105620] Updated weights for policy 1, policy_version 405640 (0.0006) [2023-12-26 18:18:52,683][105692] Updated weights for policy 0, policy_version 405386 (0.0010) [2023-12-26 18:18:52,744][105692] Updated weights for policy 0, policy_version 405396 (0.0010) [2023-12-26 18:18:52,807][105692] Updated weights for policy 0, policy_version 405406 (0.0011) [2023-12-26 18:18:52,873][105692] Updated weights for policy 0, policy_version 405416 (0.0011) [2023-12-26 18:18:53,166][105620] Updated weights for policy 1, policy_version 405650 (0.0006) [2023-12-26 18:18:53,213][105620] Updated weights for policy 1, policy_version 405660 (0.0008) [2023-12-26 18:18:53,262][105620] Updated weights for policy 1, policy_version 405670 (0.0006) [2023-12-26 18:18:53,322][105620] Updated weights for policy 1, policy_version 405680 (0.0005) [2023-12-26 18:18:53,531][105692] Updated weights for policy 0, policy_version 405426 (0.0010) [2023-12-26 18:18:53,589][105692] Updated weights for policy 0, policy_version 405436 (0.0010) [2023-12-26 18:18:53,676][105692] Updated weights for policy 0, policy_version 405446 (0.0010) [2023-12-26 18:18:53,879][105620] Updated weights for policy 1, policy_version 405690 (0.0010) [2023-12-26 18:18:53,930][105620] Updated weights for policy 1, policy_version 405700 (0.0010) [2023-12-26 18:18:53,979][105620] Updated weights for policy 1, policy_version 405710 (0.0010) [2023-12-26 18:18:54,369][105692] Updated weights for policy 0, policy_version 405456 (0.0010) [2023-12-26 18:18:54,430][105692] Updated weights for policy 0, policy_version 405466 (0.0010) [2023-12-26 18:18:54,478][105692] Updated weights for policy 0, policy_version 405476 (0.0010) [2023-12-26 18:18:54,734][105620] Updated weights for policy 1, policy_version 405720 (0.0008) [2023-12-26 18:18:54,782][105620] Updated weights for policy 1, policy_version 405730 (0.0008) [2023-12-26 18:18:54,830][105620] Updated weights for policy 1, policy_version 405740 (0.0008) [2023-12-26 18:18:55,176][105692] Updated weights for policy 0, policy_version 405486 (0.0007) [2023-12-26 18:18:55,228][105692] Updated weights for policy 0, policy_version 405496 (0.0005) [2023-12-26 18:18:55,236][105585] KL-divergence is very high: 176.2799 [2023-12-26 18:18:55,292][105585] KL-divergence is very high: 218.1309 [2023-12-26 18:18:55,299][105692] Updated weights for policy 0, policy_version 405506 (0.0006) [2023-12-26 18:18:55,692][105620] Updated weights for policy 1, policy_version 405750 (0.0009) [2023-12-26 18:18:55,743][105620] Updated weights for policy 1, policy_version 405760 (0.0009) [2023-12-26 18:18:55,791][105620] Updated weights for policy 1, policy_version 405770 (0.0008) [2023-12-26 18:18:55,939][105692] Updated weights for policy 0, policy_version 405516 (0.0007) [2023-12-26 18:18:56,008][105692] Updated weights for policy 0, policy_version 405526 (0.0010) [2023-12-26 18:18:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 207716352. Throughput: 0: 9910.6, 1: 9544.5. Samples: 207726784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:18:56,062][104569] Avg episode reward: [(0, '8170.633'), (1, '8810.277')] [2023-12-26 18:18:56,073][105692] Updated weights for policy 0, policy_version 405536 (0.0011) [2023-12-26 18:18:56,600][105620] Updated weights for policy 1, policy_version 405780 (0.0008) [2023-12-26 18:18:56,659][105620] Updated weights for policy 1, policy_version 405790 (0.0008) [2023-12-26 18:18:56,717][105620] Updated weights for policy 1, policy_version 405800 (0.0007) [2023-12-26 18:18:56,733][105692] Updated weights for policy 0, policy_version 405546 (0.0011) [2023-12-26 18:18:56,762][105585] KL-divergence is very high: 101.3445 [2023-12-26 18:18:56,784][105692] Updated weights for policy 0, policy_version 405556 (0.0010) [2023-12-26 18:18:56,784][105585] KL-divergence is very high: 276.3882 [2023-12-26 18:18:56,806][105585] KL-divergence is very high: 326.0103 [2023-12-26 18:18:56,828][105585] KL-divergence is very high: 402.9077 [2023-12-26 18:18:56,842][105692] Updated weights for policy 0, policy_version 405566 (0.0010) [2023-12-26 18:18:56,852][105585] KL-divergence is very high: 302.2177 [2023-12-26 18:18:56,871][105585] KL-divergence is very high: 337.6930 [2023-12-26 18:18:56,890][105692] Updated weights for policy 0, policy_version 405576 (0.0010) [2023-12-26 18:18:57,467][105620] Updated weights for policy 1, policy_version 405810 (0.0006) [2023-12-26 18:18:57,533][105620] Updated weights for policy 1, policy_version 405820 (0.0008) [2023-12-26 18:18:57,570][105692] Updated weights for policy 0, policy_version 405586 (0.0005) [2023-12-26 18:18:57,580][105620] Updated weights for policy 1, policy_version 405830 (0.0008) [2023-12-26 18:18:57,615][105692] Updated weights for policy 0, policy_version 405596 (0.0005) [2023-12-26 18:18:57,668][105692] Updated weights for policy 0, policy_version 405606 (0.0005) [2023-12-26 18:18:58,201][105692] Updated weights for policy 0, policy_version 405616 (0.0007) [2023-12-26 18:18:58,246][105692] Updated weights for policy 0, policy_version 405626 (0.0008) [2023-12-26 18:18:58,296][105692] Updated weights for policy 0, policy_version 405636 (0.0007) [2023-12-26 18:18:58,429][105620] Updated weights for policy 1, policy_version 405841 (0.0009) [2023-12-26 18:18:58,492][105620] Updated weights for policy 1, policy_version 405851 (0.0008) [2023-12-26 18:18:58,561][105620] Updated weights for policy 1, policy_version 405861 (0.0008) [2023-12-26 18:18:58,624][105620] Updated weights for policy 1, policy_version 405871 (0.0008) [2023-12-26 18:18:59,037][105692] Updated weights for policy 0, policy_version 405646 (0.0008) [2023-12-26 18:18:59,096][105692] Updated weights for policy 0, policy_version 405656 (0.0008) [2023-12-26 18:18:59,159][105692] Updated weights for policy 0, policy_version 405666 (0.0009) [2023-12-26 18:18:59,458][105620] Updated weights for policy 1, policy_version 405881 (0.0006) [2023-12-26 18:18:59,519][105586] KL-divergence is very high: 103.7289 [2023-12-26 18:18:59,525][105620] Updated weights for policy 1, policy_version 405891 (0.0005) [2023-12-26 18:18:59,590][105620] Updated weights for policy 1, policy_version 405901 (0.0006) [2023-12-26 18:18:59,912][105692] Updated weights for policy 0, policy_version 405676 (0.0009) [2023-12-26 18:18:59,973][105692] Updated weights for policy 0, policy_version 405686 (0.0008) [2023-12-26 18:19:00,029][105692] Updated weights for policy 0, policy_version 405696 (0.0009) [2023-12-26 18:19:00,043][105585] KL-divergence is very high: 124.3965 [2023-12-26 18:19:00,255][105620] Updated weights for policy 1, policy_version 405911 (0.0006) [2023-12-26 18:19:00,320][105620] Updated weights for policy 1, policy_version 405921 (0.0005) [2023-12-26 18:19:00,369][105620] Updated weights for policy 1, policy_version 405931 (0.0006) [2023-12-26 18:19:00,798][105692] Updated weights for policy 0, policy_version 405706 (0.0009) [2023-12-26 18:19:00,853][105692] Updated weights for policy 0, policy_version 405716 (0.0010) [2023-12-26 18:19:00,908][105585] KL-divergence is very high: 145.7438 [2023-12-26 18:19:00,908][105692] Updated weights for policy 0, policy_version 405726 (0.0008) [2023-12-26 18:19:00,934][105620] Updated weights for policy 1, policy_version 405941 (0.0007) [2023-12-26 18:19:00,953][105585] KL-divergence is very high: 245.8920 [2023-12-26 18:19:00,967][105692] Updated weights for policy 0, policy_version 405736 (0.0008) [2023-12-26 18:19:00,984][105620] Updated weights for policy 1, policy_version 405951 (0.0008) [2023-12-26 18:19:01,035][105620] Updated weights for policy 1, policy_version 405961 (0.0006) [2023-12-26 18:19:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 207814656. Throughput: 0: 9930.6, 1: 9508.7. Samples: 207784252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:19:01,063][104569] Avg episode reward: [(0, '8439.725'), (1, '8808.028')] [2023-12-26 18:19:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000405736_103882752.pth... [2023-12-26 18:19:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000404552_103579648.pth [2023-12-26 18:19:01,075][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000405968_103940096.pth... [2023-12-26 18:19:01,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000404848_103653376.pth [2023-12-26 18:19:01,760][105692] Updated weights for policy 0, policy_version 405746 (0.0007) [2023-12-26 18:19:01,818][105692] Updated weights for policy 0, policy_version 405756 (0.0007) [2023-12-26 18:19:01,824][105620] Updated weights for policy 1, policy_version 405971 (0.0009) [2023-12-26 18:19:01,880][105692] Updated weights for policy 0, policy_version 405766 (0.0006) [2023-12-26 18:19:01,884][105620] Updated weights for policy 1, policy_version 405981 (0.0008) [2023-12-26 18:19:01,947][105620] Updated weights for policy 1, policy_version 405991 (0.0008) [2023-12-26 18:19:02,522][105692] Updated weights for policy 0, policy_version 405776 (0.0007) [2023-12-26 18:19:02,576][105692] Updated weights for policy 0, policy_version 405786 (0.0009) [2023-12-26 18:19:02,634][105692] Updated weights for policy 0, policy_version 405796 (0.0009) [2023-12-26 18:19:02,673][105620] Updated weights for policy 1, policy_version 406001 (0.0009) [2023-12-26 18:19:02,731][105620] Updated weights for policy 1, policy_version 406011 (0.0009) [2023-12-26 18:19:02,779][105620] Updated weights for policy 1, policy_version 406021 (0.0009) [2023-12-26 18:19:02,834][105620] Updated weights for policy 1, policy_version 406031 (0.0009) [2023-12-26 18:19:03,402][105692] Updated weights for policy 0, policy_version 405806 (0.0009) [2023-12-26 18:19:03,456][105692] Updated weights for policy 0, policy_version 405816 (0.0009) [2023-12-26 18:19:03,507][105692] Updated weights for policy 0, policy_version 405826 (0.0009) [2023-12-26 18:19:03,599][105620] Updated weights for policy 1, policy_version 406041 (0.0009) [2023-12-26 18:19:03,650][105620] Updated weights for policy 1, policy_version 406051 (0.0009) [2023-12-26 18:19:03,705][105620] Updated weights for policy 1, policy_version 406061 (0.0008) [2023-12-26 18:19:04,338][105692] Updated weights for policy 0, policy_version 405836 (0.0009) [2023-12-26 18:19:04,364][105620] Updated weights for policy 1, policy_version 406071 (0.0005) [2023-12-26 18:19:04,390][105692] Updated weights for policy 0, policy_version 405846 (0.0009) [2023-12-26 18:19:04,422][105620] Updated weights for policy 1, policy_version 406081 (0.0005) [2023-12-26 18:19:04,444][105692] Updated weights for policy 0, policy_version 405856 (0.0007) [2023-12-26 18:19:04,479][105620] Updated weights for policy 1, policy_version 406091 (0.0008) [2023-12-26 18:19:05,137][105620] Updated weights for policy 1, policy_version 406101 (0.0007) [2023-12-26 18:19:05,196][105620] Updated weights for policy 1, policy_version 406111 (0.0005) [2023-12-26 18:19:05,239][105692] Updated weights for policy 0, policy_version 405866 (0.0007) [2023-12-26 18:19:05,257][105620] Updated weights for policy 1, policy_version 406121 (0.0005) [2023-12-26 18:19:05,294][105692] Updated weights for policy 0, policy_version 405876 (0.0009) [2023-12-26 18:19:05,361][105692] Updated weights for policy 0, policy_version 405886 (0.0009) [2023-12-26 18:19:05,415][105692] Updated weights for policy 0, policy_version 405896 (0.0009) [2023-12-26 18:19:05,959][105620] Updated weights for policy 1, policy_version 406131 (0.0007) [2023-12-26 18:19:06,022][105620] Updated weights for policy 1, policy_version 406141 (0.0008) [2023-12-26 18:19:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19660.8). Total num frames: 207904768. Throughput: 0: 9828.4, 1: 9614.2. Samples: 207899896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:19:06,062][104569] Avg episode reward: [(0, '8805.872'), (1, '8989.467')] [2023-12-26 18:19:06,076][105620] Updated weights for policy 1, policy_version 406151 (0.0008) [2023-12-26 18:19:06,100][105692] Updated weights for policy 0, policy_version 405906 (0.0007) [2023-12-26 18:19:06,165][105692] Updated weights for policy 0, policy_version 405916 (0.0007) [2023-12-26 18:19:06,225][105692] Updated weights for policy 0, policy_version 405926 (0.0006) [2023-12-26 18:19:06,836][105620] Updated weights for policy 1, policy_version 406161 (0.0007) [2023-12-26 18:19:06,898][105620] Updated weights for policy 1, policy_version 406171 (0.0009) [2023-12-26 18:19:06,951][105620] Updated weights for policy 1, policy_version 406181 (0.0006) [2023-12-26 18:19:06,959][105692] Updated weights for policy 0, policy_version 405936 (0.0007) [2023-12-26 18:19:07,010][105620] Updated weights for policy 1, policy_version 406191 (0.0007) [2023-12-26 18:19:07,018][105692] Updated weights for policy 0, policy_version 405946 (0.0009) [2023-12-26 18:19:07,080][105692] Updated weights for policy 0, policy_version 405956 (0.0009) [2023-12-26 18:19:07,703][105620] Updated weights for policy 1, policy_version 406201 (0.0009) [2023-12-26 18:19:07,764][105620] Updated weights for policy 1, policy_version 406211 (0.0009) [2023-12-26 18:19:07,824][105620] Updated weights for policy 1, policy_version 406221 (0.0008) [2023-12-26 18:19:07,847][105692] Updated weights for policy 0, policy_version 405966 (0.0008) [2023-12-26 18:19:07,901][105692] Updated weights for policy 0, policy_version 405976 (0.0009) [2023-12-26 18:19:07,960][105692] Updated weights for policy 0, policy_version 405986 (0.0008) [2023-12-26 18:19:08,443][105620] Updated weights for policy 1, policy_version 406231 (0.0006) [2023-12-26 18:19:08,498][105620] Updated weights for policy 1, policy_version 406241 (0.0007) [2023-12-26 18:19:08,559][105620] Updated weights for policy 1, policy_version 406251 (0.0009) [2023-12-26 18:19:08,821][105692] Updated weights for policy 0, policy_version 405996 (0.0010) [2023-12-26 18:19:08,873][105692] Updated weights for policy 0, policy_version 406006 (0.0009) [2023-12-26 18:19:08,929][105692] Updated weights for policy 0, policy_version 406016 (0.0005) [2023-12-26 18:19:09,252][105620] Updated weights for policy 1, policy_version 406261 (0.0009) [2023-12-26 18:19:09,322][105620] Updated weights for policy 1, policy_version 406271 (0.0006) [2023-12-26 18:19:09,388][105620] Updated weights for policy 1, policy_version 406281 (0.0008) [2023-12-26 18:19:09,650][105692] Updated weights for policy 0, policy_version 406026 (0.0008) [2023-12-26 18:19:09,717][105692] Updated weights for policy 0, policy_version 406036 (0.0008) [2023-12-26 18:19:09,771][105692] Updated weights for policy 0, policy_version 406046 (0.0005) [2023-12-26 18:19:09,834][105692] Updated weights for policy 0, policy_version 406056 (0.0007) [2023-12-26 18:19:10,215][105620] Updated weights for policy 1, policy_version 406291 (0.0010) [2023-12-26 18:19:10,268][105620] Updated weights for policy 1, policy_version 406301 (0.0009) [2023-12-26 18:19:10,322][105620] Updated weights for policy 1, policy_version 406311 (0.0009) [2023-12-26 18:19:10,467][105692] Updated weights for policy 0, policy_version 406066 (0.0008) [2023-12-26 18:19:10,525][105692] Updated weights for policy 0, policy_version 406076 (0.0010) [2023-12-26 18:19:10,580][105692] Updated weights for policy 0, policy_version 406087 (0.0009) [2023-12-26 18:19:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 208003072. Throughput: 0: 9800.4, 1: 9575.4. Samples: 208014428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:19:11,063][104569] Avg episode reward: [(0, '8653.225'), (1, '9081.082')] [2023-12-26 18:19:11,069][105620] Updated weights for policy 1, policy_version 406321 (0.0007) [2023-12-26 18:19:11,144][105620] Updated weights for policy 1, policy_version 406331 (0.0009) [2023-12-26 18:19:11,210][105620] Updated weights for policy 1, policy_version 406341 (0.0009) [2023-12-26 18:19:11,274][105620] Updated weights for policy 1, policy_version 406351 (0.0008) [2023-12-26 18:19:11,311][105692] Updated weights for policy 0, policy_version 406097 (0.0007) [2023-12-26 18:19:11,313][105585] KL-divergence is very high: 195.1309 [2023-12-26 18:19:11,371][105585] KL-divergence is very high: 339.7672 [2023-12-26 18:19:11,382][105692] Updated weights for policy 0, policy_version 406107 (0.0009) [2023-12-26 18:19:11,416][105585] KL-divergence is very high: 218.6610 [2023-12-26 18:19:11,440][105692] Updated weights for policy 0, policy_version 406117 (0.0010) [2023-12-26 18:19:11,997][105620] Updated weights for policy 1, policy_version 406361 (0.0009) [2023-12-26 18:19:12,045][105620] Updated weights for policy 1, policy_version 406371 (0.0009) [2023-12-26 18:19:12,108][105620] Updated weights for policy 1, policy_version 406381 (0.0008) [2023-12-26 18:19:12,186][105692] Updated weights for policy 0, policy_version 406127 (0.0008) [2023-12-26 18:19:12,247][105692] Updated weights for policy 0, policy_version 406137 (0.0007) [2023-12-26 18:19:12,309][105692] Updated weights for policy 0, policy_version 406147 (0.0009) [2023-12-26 18:19:12,802][105620] Updated weights for policy 1, policy_version 406391 (0.0006) [2023-12-26 18:19:12,854][105620] Updated weights for policy 1, policy_version 406401 (0.0007) [2023-12-26 18:19:12,901][105620] Updated weights for policy 1, policy_version 406411 (0.0008) [2023-12-26 18:19:13,114][105692] Updated weights for policy 0, policy_version 406157 (0.0009) [2023-12-26 18:19:13,165][105692] Updated weights for policy 0, policy_version 406167 (0.0009) [2023-12-26 18:19:13,231][105692] Updated weights for policy 0, policy_version 406177 (0.0006) [2023-12-26 18:19:13,613][105620] Updated weights for policy 1, policy_version 406421 (0.0010) [2023-12-26 18:19:13,667][105620] Updated weights for policy 1, policy_version 406431 (0.0010) [2023-12-26 18:19:13,729][105620] Updated weights for policy 1, policy_version 406441 (0.0010) [2023-12-26 18:19:13,909][105692] Updated weights for policy 0, policy_version 406187 (0.0010) [2023-12-26 18:19:13,970][105692] Updated weights for policy 0, policy_version 406197 (0.0010) [2023-12-26 18:19:14,025][105692] Updated weights for policy 0, policy_version 406207 (0.0010) [2023-12-26 18:19:14,338][105620] Updated weights for policy 1, policy_version 406451 (0.0009) [2023-12-26 18:19:14,394][105620] Updated weights for policy 1, policy_version 406461 (0.0005) [2023-12-26 18:19:14,442][105620] Updated weights for policy 1, policy_version 406471 (0.0005) [2023-12-26 18:19:14,777][105692] Updated weights for policy 0, policy_version 406217 (0.0010) [2023-12-26 18:19:14,837][105692] Updated weights for policy 0, policy_version 406227 (0.0008) [2023-12-26 18:19:14,891][105692] Updated weights for policy 0, policy_version 406237 (0.0008) [2023-12-26 18:19:14,950][105692] Updated weights for policy 0, policy_version 406247 (0.0009) [2023-12-26 18:19:15,121][105620] Updated weights for policy 1, policy_version 406481 (0.0006) [2023-12-26 18:19:15,184][105620] Updated weights for policy 1, policy_version 406491 (0.0011) [2023-12-26 18:19:15,249][105620] Updated weights for policy 1, policy_version 406501 (0.0010) [2023-12-26 18:19:15,316][105620] Updated weights for policy 1, policy_version 406511 (0.0006) [2023-12-26 18:19:15,689][105692] Updated weights for policy 0, policy_version 406257 (0.0010) [2023-12-26 18:19:15,751][105692] Updated weights for policy 0, policy_version 406267 (0.0010) [2023-12-26 18:19:15,812][105692] Updated weights for policy 0, policy_version 406277 (0.0010) [2023-12-26 18:19:16,033][105620] Updated weights for policy 1, policy_version 406521 (0.0006) [2023-12-26 18:19:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19660.8). Total num frames: 208101376. Throughput: 0: 9799.1, 1: 9554.7. Samples: 208070864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:19:16,062][104569] Avg episode reward: [(0, '8653.287'), (1, '8817.902')] [2023-12-26 18:19:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000406280_104022016.pth... [2023-12-26 18:19:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000405128_103727104.pth [2023-12-26 18:19:16,092][105620] Updated weights for policy 1, policy_version 406531 (0.0010) [2023-12-26 18:19:16,151][105620] Updated weights for policy 1, policy_version 406541 (0.0010) [2023-12-26 18:19:16,165][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000406544_104087552.pth... [2023-12-26 18:19:16,169][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000405392_103792640.pth [2023-12-26 18:19:16,545][105692] Updated weights for policy 0, policy_version 406287 (0.0010) [2023-12-26 18:19:16,601][105692] Updated weights for policy 0, policy_version 406297 (0.0010) [2023-12-26 18:19:16,628][105585] KL-divergence is very high: 152.2995 [2023-12-26 18:19:16,656][105692] Updated weights for policy 0, policy_version 406307 (0.0010) [2023-12-26 18:19:16,669][105585] KL-divergence is very high: 212.6799 [2023-12-26 18:19:16,756][105620] Updated weights for policy 1, policy_version 406551 (0.0007) [2023-12-26 18:19:16,817][105620] Updated weights for policy 1, policy_version 406561 (0.0005) [2023-12-26 18:19:16,877][105620] Updated weights for policy 1, policy_version 406571 (0.0005) [2023-12-26 18:19:17,379][105692] Updated weights for policy 0, policy_version 406317 (0.0010) [2023-12-26 18:19:17,415][105620] Updated weights for policy 1, policy_version 406581 (0.0005) [2023-12-26 18:19:17,428][105692] Updated weights for policy 0, policy_version 406327 (0.0010) [2023-12-26 18:19:17,479][105620] Updated weights for policy 1, policy_version 406591 (0.0006) [2023-12-26 18:19:17,479][105692] Updated weights for policy 0, policy_version 406337 (0.0010) [2023-12-26 18:19:17,543][105620] Updated weights for policy 1, policy_version 406601 (0.0005) [2023-12-26 18:19:18,141][105620] Updated weights for policy 1, policy_version 406611 (0.0007) [2023-12-26 18:19:18,192][105620] Updated weights for policy 1, policy_version 406621 (0.0005) [2023-12-26 18:19:18,250][105620] Updated weights for policy 1, policy_version 406631 (0.0010) [2023-12-26 18:19:18,253][105692] Updated weights for policy 0, policy_version 406347 (0.0010) [2023-12-26 18:19:18,307][105692] Updated weights for policy 0, policy_version 406357 (0.0010) [2023-12-26 18:19:18,370][105692] Updated weights for policy 0, policy_version 406367 (0.0011) [2023-12-26 18:19:19,004][105620] Updated weights for policy 1, policy_version 406641 (0.0010) [2023-12-26 18:19:19,052][105620] Updated weights for policy 1, policy_version 406651 (0.0010) [2023-12-26 18:19:19,104][105620] Updated weights for policy 1, policy_version 406661 (0.0010) [2023-12-26 18:19:19,119][105692] Updated weights for policy 0, policy_version 406377 (0.0011) [2023-12-26 18:19:19,163][105620] Updated weights for policy 1, policy_version 406671 (0.0010) [2023-12-26 18:19:19,171][105692] Updated weights for policy 0, policy_version 406387 (0.0010) [2023-12-26 18:19:19,226][105692] Updated weights for policy 0, policy_version 406397 (0.0010) [2023-12-26 18:19:19,287][105692] Updated weights for policy 0, policy_version 406407 (0.0009) [2023-12-26 18:19:19,908][105620] Updated weights for policy 1, policy_version 406681 (0.0009) [2023-12-26 18:19:19,970][105620] Updated weights for policy 1, policy_version 406691 (0.0007) [2023-12-26 18:19:20,009][105692] Updated weights for policy 0, policy_version 406417 (0.0010) [2023-12-26 18:19:20,031][105620] Updated weights for policy 1, policy_version 406701 (0.0006) [2023-12-26 18:19:20,068][105692] Updated weights for policy 0, policy_version 406427 (0.0010) [2023-12-26 18:19:20,132][105692] Updated weights for policy 0, policy_version 406437 (0.0011) [2023-12-26 18:19:20,663][105620] Updated weights for policy 1, policy_version 406711 (0.0008) [2023-12-26 18:19:20,727][105620] Updated weights for policy 1, policy_version 406721 (0.0006) [2023-12-26 18:19:20,788][105620] Updated weights for policy 1, policy_version 406731 (0.0008) [2023-12-26 18:19:20,857][105692] Updated weights for policy 0, policy_version 406447 (0.0011) [2023-12-26 18:19:20,917][105692] Updated weights for policy 0, policy_version 406457 (0.0011) [2023-12-26 18:19:20,974][105692] Updated weights for policy 0, policy_version 406467 (0.0011) [2023-12-26 18:19:21,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.2, 300 sec: 19688.6). Total num frames: 208207872. Throughput: 0: 9852.3, 1: 9693.5. Samples: 208191448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:19:21,063][104569] Avg episode reward: [(0, '8451.610'), (1, '8729.388')] [2023-12-26 18:19:21,485][105620] Updated weights for policy 1, policy_version 406741 (0.0008) [2023-12-26 18:19:21,540][105620] Updated weights for policy 1, policy_version 406751 (0.0008) [2023-12-26 18:19:21,597][105620] Updated weights for policy 1, policy_version 406761 (0.0008) [2023-12-26 18:19:21,727][105692] Updated weights for policy 0, policy_version 406477 (0.0009) [2023-12-26 18:19:21,792][105692] Updated weights for policy 0, policy_version 406487 (0.0010) [2023-12-26 18:19:21,850][105692] Updated weights for policy 0, policy_version 406497 (0.0010) [2023-12-26 18:19:22,430][105620] Updated weights for policy 1, policy_version 406771 (0.0008) [2023-12-26 18:19:22,433][105692] Updated weights for policy 0, policy_version 406507 (0.0007) [2023-12-26 18:19:22,490][105620] Updated weights for policy 1, policy_version 406781 (0.0007) [2023-12-26 18:19:22,493][105692] Updated weights for policy 0, policy_version 406517 (0.0010) [2023-12-26 18:19:22,546][105620] Updated weights for policy 1, policy_version 406791 (0.0009) [2023-12-26 18:19:22,552][105692] Updated weights for policy 0, policy_version 406527 (0.0009) [2023-12-26 18:19:23,173][105692] Updated weights for policy 0, policy_version 406537 (0.0009) [2023-12-26 18:19:23,222][105692] Updated weights for policy 0, policy_version 406547 (0.0010) [2023-12-26 18:19:23,277][105692] Updated weights for policy 0, policy_version 406557 (0.0011) [2023-12-26 18:19:23,326][105692] Updated weights for policy 0, policy_version 406567 (0.0011) [2023-12-26 18:19:23,350][105620] Updated weights for policy 1, policy_version 406801 (0.0009) [2023-12-26 18:19:23,405][105620] Updated weights for policy 1, policy_version 406811 (0.0008) [2023-12-26 18:19:23,449][105620] Updated weights for policy 1, policy_version 406821 (0.0008) [2023-12-26 18:19:23,500][105620] Updated weights for policy 1, policy_version 406831 (0.0008) [2023-12-26 18:19:24,090][105692] Updated weights for policy 0, policy_version 406577 (0.0010) [2023-12-26 18:19:24,151][105692] Updated weights for policy 0, policy_version 406587 (0.0010) [2023-12-26 18:19:24,210][105692] Updated weights for policy 0, policy_version 406597 (0.0011) [2023-12-26 18:19:24,262][105620] Updated weights for policy 1, policy_version 406841 (0.0008) [2023-12-26 18:19:24,317][105620] Updated weights for policy 1, policy_version 406851 (0.0008) [2023-12-26 18:19:24,368][105620] Updated weights for policy 1, policy_version 406861 (0.0008) [2023-12-26 18:19:24,957][105692] Updated weights for policy 0, policy_version 406607 (0.0009) [2023-12-26 18:19:25,014][105692] Updated weights for policy 0, policy_version 406617 (0.0005) [2023-12-26 18:19:25,073][105692] Updated weights for policy 0, policy_version 406627 (0.0005) [2023-12-26 18:19:25,155][105620] Updated weights for policy 1, policy_version 406871 (0.0009) [2023-12-26 18:19:25,205][105620] Updated weights for policy 1, policy_version 406881 (0.0008) [2023-12-26 18:19:25,271][105620] Updated weights for policy 1, policy_version 406891 (0.0008) [2023-12-26 18:19:25,681][105692] Updated weights for policy 0, policy_version 406637 (0.0006) [2023-12-26 18:19:25,745][105692] Updated weights for policy 0, policy_version 406647 (0.0005) [2023-12-26 18:19:25,812][105692] Updated weights for policy 0, policy_version 406657 (0.0008) [2023-12-26 18:19:26,050][105620] Updated weights for policy 1, policy_version 406901 (0.0009) [2023-12-26 18:19:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 208297984. Throughput: 0: 9788.5, 1: 9773.8. Samples: 208308012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:19:26,062][104569] Avg episode reward: [(0, '8455.095'), (1, '8817.315')] [2023-12-26 18:19:26,107][105620] Updated weights for policy 1, policy_version 406911 (0.0009) [2023-12-26 18:19:26,158][105620] Updated weights for policy 1, policy_version 406921 (0.0009) [2023-12-26 18:19:26,397][105692] Updated weights for policy 0, policy_version 406667 (0.0006) [2023-12-26 18:19:26,458][105692] Updated weights for policy 0, policy_version 406677 (0.0009) [2023-12-26 18:19:26,512][105692] Updated weights for policy 0, policy_version 406687 (0.0005) [2023-12-26 18:19:26,941][105620] Updated weights for policy 1, policy_version 406931 (0.0008) [2023-12-26 18:19:26,998][105620] Updated weights for policy 1, policy_version 406941 (0.0005) [2023-12-26 18:19:27,055][105620] Updated weights for policy 1, policy_version 406951 (0.0005) [2023-12-26 18:19:27,125][105692] Updated weights for policy 0, policy_version 406697 (0.0007) [2023-12-26 18:19:27,193][105692] Updated weights for policy 0, policy_version 406707 (0.0005) [2023-12-26 18:19:27,245][105692] Updated weights for policy 0, policy_version 406717 (0.0005) [2023-12-26 18:19:27,288][105692] Updated weights for policy 0, policy_version 406727 (0.0005) [2023-12-26 18:19:27,680][105620] Updated weights for policy 1, policy_version 406961 (0.0007) [2023-12-26 18:19:27,738][105620] Updated weights for policy 1, policy_version 406971 (0.0010) [2023-12-26 18:19:27,799][105620] Updated weights for policy 1, policy_version 406981 (0.0010) [2023-12-26 18:19:27,813][105692] Updated weights for policy 0, policy_version 406737 (0.0007) [2023-12-26 18:19:27,857][105620] Updated weights for policy 1, policy_version 406991 (0.0010) [2023-12-26 18:19:27,867][105692] Updated weights for policy 0, policy_version 406747 (0.0005) [2023-12-26 18:19:27,921][105692] Updated weights for policy 0, policy_version 406757 (0.0008) [2023-12-26 18:19:28,597][105620] Updated weights for policy 1, policy_version 407001 (0.0010) [2023-12-26 18:19:28,624][105692] Updated weights for policy 0, policy_version 406767 (0.0006) [2023-12-26 18:19:28,656][105620] Updated weights for policy 1, policy_version 407011 (0.0010) [2023-12-26 18:19:28,669][105692] Updated weights for policy 0, policy_version 406777 (0.0008) [2023-12-26 18:19:28,706][105620] Updated weights for policy 1, policy_version 407021 (0.0010) [2023-12-26 18:19:28,718][105692] Updated weights for policy 0, policy_version 406787 (0.0005) [2023-12-26 18:19:29,312][105620] Updated weights for policy 1, policy_version 407031 (0.0008) [2023-12-26 18:19:29,379][105620] Updated weights for policy 1, policy_version 407041 (0.0009) [2023-12-26 18:19:29,429][105620] Updated weights for policy 1, policy_version 407051 (0.0008) [2023-12-26 18:19:29,435][105692] Updated weights for policy 0, policy_version 406797 (0.0005) [2023-12-26 18:19:29,487][105692] Updated weights for policy 0, policy_version 406807 (0.0008) [2023-12-26 18:19:29,537][105692] Updated weights for policy 0, policy_version 406817 (0.0009) [2023-12-26 18:19:30,147][105620] Updated weights for policy 1, policy_version 407061 (0.0007) [2023-12-26 18:19:30,206][105620] Updated weights for policy 1, policy_version 407071 (0.0008) [2023-12-26 18:19:30,256][105692] Updated weights for policy 0, policy_version 406827 (0.0009) [2023-12-26 18:19:30,266][105620] Updated weights for policy 1, policy_version 407081 (0.0007) [2023-12-26 18:19:30,304][105692] Updated weights for policy 0, policy_version 406837 (0.0010) [2023-12-26 18:19:30,359][105692] Updated weights for policy 0, policy_version 406847 (0.0010) [2023-12-26 18:19:31,035][105692] Updated weights for policy 0, policy_version 406857 (0.0008) [2023-12-26 18:19:31,059][105620] Updated weights for policy 1, policy_version 407091 (0.0006) [2023-12-26 18:19:31,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 208396288. Throughput: 0: 9910.1, 1: 9734.3. Samples: 208370756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:19:31,062][104569] Avg episode reward: [(0, '9179.435'), (1, '9088.004')] [2023-12-26 18:19:31,095][105692] Updated weights for policy 0, policy_version 406867 (0.0009) [2023-12-26 18:19:31,117][105620] Updated weights for policy 1, policy_version 407101 (0.0009) [2023-12-26 18:19:31,156][105692] Updated weights for policy 0, policy_version 406877 (0.0006) [2023-12-26 18:19:31,178][105620] Updated weights for policy 1, policy_version 407111 (0.0008) [2023-12-26 18:19:31,206][105692] Updated weights for policy 0, policy_version 406887 (0.0006) [2023-12-26 18:19:31,210][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000406888_104177664.pth... [2023-12-26 18:19:31,213][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000405736_103882752.pth [2023-12-26 18:19:31,224][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000407120_104235008.pth... [2023-12-26 18:19:31,227][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000405968_103940096.pth [2023-12-26 18:19:31,870][105692] Updated weights for policy 0, policy_version 406897 (0.0008) [2023-12-26 18:19:31,923][105692] Updated weights for policy 0, policy_version 406907 (0.0009) [2023-12-26 18:19:31,969][105692] Updated weights for policy 0, policy_version 406917 (0.0008) [2023-12-26 18:19:31,975][105620] Updated weights for policy 1, policy_version 407121 (0.0008) [2023-12-26 18:19:32,035][105620] Updated weights for policy 1, policy_version 407131 (0.0006) [2023-12-26 18:19:32,091][105620] Updated weights for policy 1, policy_version 407141 (0.0006) [2023-12-26 18:19:32,158][105620] Updated weights for policy 1, policy_version 407151 (0.0008) [2023-12-26 18:19:32,757][105692] Updated weights for policy 0, policy_version 406927 (0.0010) [2023-12-26 18:19:32,813][105692] Updated weights for policy 0, policy_version 406937 (0.0010) [2023-12-26 18:19:32,823][105620] Updated weights for policy 1, policy_version 407161 (0.0006) [2023-12-26 18:19:32,872][105692] Updated weights for policy 0, policy_version 406947 (0.0011) [2023-12-26 18:19:32,879][105620] Updated weights for policy 1, policy_version 407171 (0.0006) [2023-12-26 18:19:32,941][105620] Updated weights for policy 1, policy_version 407181 (0.0007) [2023-12-26 18:19:33,594][105692] Updated weights for policy 0, policy_version 406957 (0.0009) [2023-12-26 18:19:33,653][105692] Updated weights for policy 0, policy_version 406967 (0.0009) [2023-12-26 18:19:33,676][105620] Updated weights for policy 1, policy_version 407191 (0.0008) [2023-12-26 18:19:33,702][105692] Updated weights for policy 0, policy_version 406977 (0.0006) [2023-12-26 18:19:33,731][105620] Updated weights for policy 1, policy_version 407201 (0.0008) [2023-12-26 18:19:33,787][105620] Updated weights for policy 1, policy_version 407211 (0.0009) [2023-12-26 18:19:34,476][105692] Updated weights for policy 0, policy_version 406987 (0.0008) [2023-12-26 18:19:34,537][105620] Updated weights for policy 1, policy_version 407221 (0.0008) [2023-12-26 18:19:34,539][105692] Updated weights for policy 0, policy_version 406997 (0.0007) [2023-12-26 18:19:34,600][105620] Updated weights for policy 1, policy_version 407231 (0.0008) [2023-12-26 18:19:34,602][105692] Updated weights for policy 0, policy_version 407007 (0.0007) [2023-12-26 18:19:34,663][105620] Updated weights for policy 1, policy_version 407241 (0.0008) [2023-12-26 18:19:35,288][105620] Updated weights for policy 1, policy_version 407251 (0.0008) [2023-12-26 18:19:35,290][105692] Updated weights for policy 0, policy_version 407017 (0.0008) [2023-12-26 18:19:35,344][105620] Updated weights for policy 1, policy_version 407261 (0.0007) [2023-12-26 18:19:35,350][105692] Updated weights for policy 0, policy_version 407027 (0.0008) [2023-12-26 18:19:35,405][105620] Updated weights for policy 1, policy_version 407271 (0.0008) [2023-12-26 18:19:35,411][105692] Updated weights for policy 0, policy_version 407037 (0.0008) [2023-12-26 18:19:35,468][105692] Updated weights for policy 0, policy_version 407047 (0.0006) [2023-12-26 18:19:36,055][105692] Updated weights for policy 0, policy_version 407057 (0.0006) [2023-12-26 18:19:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19660.8). Total num frames: 208494592. Throughput: 0: 9769.4, 1: 9711.8. Samples: 208486288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:19:36,062][104569] Avg episode reward: [(0, '9265.371'), (1, '8908.380')] [2023-12-26 18:19:36,089][105620] Updated weights for policy 1, policy_version 407281 (0.0008) [2023-12-26 18:19:36,122][105692] Updated weights for policy 0, policy_version 407067 (0.0007) [2023-12-26 18:19:36,156][105620] Updated weights for policy 1, policy_version 407291 (0.0006) [2023-12-26 18:19:36,180][105692] Updated weights for policy 0, policy_version 407077 (0.0006) [2023-12-26 18:19:36,226][105620] Updated weights for policy 1, policy_version 407301 (0.0006) [2023-12-26 18:19:36,299][105620] Updated weights for policy 1, policy_version 407311 (0.0007) [2023-12-26 18:19:36,845][105692] Updated weights for policy 0, policy_version 407087 (0.0007) [2023-12-26 18:19:36,898][105692] Updated weights for policy 0, policy_version 407097 (0.0008) [2023-12-26 18:19:36,954][105692] Updated weights for policy 0, policy_version 407107 (0.0008) [2023-12-26 18:19:37,015][105620] Updated weights for policy 1, policy_version 407321 (0.0010) [2023-12-26 18:19:37,074][105620] Updated weights for policy 1, policy_version 407331 (0.0009) [2023-12-26 18:19:37,136][105620] Updated weights for policy 1, policy_version 407341 (0.0010) [2023-12-26 18:19:37,750][105692] Updated weights for policy 0, policy_version 407117 (0.0008) [2023-12-26 18:19:37,798][105692] Updated weights for policy 0, policy_version 407127 (0.0008) [2023-12-26 18:19:37,850][105692] Updated weights for policy 0, policy_version 407137 (0.0008) [2023-12-26 18:19:37,882][105620] Updated weights for policy 1, policy_version 407351 (0.0007) [2023-12-26 18:19:37,942][105620] Updated weights for policy 1, policy_version 407361 (0.0006) [2023-12-26 18:19:37,979][105586] KL-divergence is very high: 116.4904 [2023-12-26 18:19:38,006][105620] Updated weights for policy 1, policy_version 407371 (0.0005) [2023-12-26 18:19:38,031][105586] KL-divergence is very high: 107.9489 [2023-12-26 18:19:38,643][105620] Updated weights for policy 1, policy_version 407381 (0.0005) [2023-12-26 18:19:38,669][105692] Updated weights for policy 0, policy_version 407147 (0.0009) [2023-12-26 18:19:38,694][105620] Updated weights for policy 1, policy_version 407391 (0.0005) [2023-12-26 18:19:38,732][105692] Updated weights for policy 0, policy_version 407157 (0.0008) [2023-12-26 18:19:38,751][105620] Updated weights for policy 1, policy_version 407401 (0.0007) [2023-12-26 18:19:38,793][105692] Updated weights for policy 0, policy_version 407167 (0.0008) [2023-12-26 18:19:39,374][105620] Updated weights for policy 1, policy_version 407411 (0.0006) [2023-12-26 18:19:39,441][105620] Updated weights for policy 1, policy_version 407421 (0.0007) [2023-12-26 18:19:39,492][105620] Updated weights for policy 1, policy_version 407431 (0.0009) [2023-12-26 18:19:39,639][105692] Updated weights for policy 0, policy_version 407177 (0.0008) [2023-12-26 18:19:39,692][105692] Updated weights for policy 0, policy_version 407187 (0.0007) [2023-12-26 18:19:39,750][105692] Updated weights for policy 0, policy_version 407197 (0.0008) [2023-12-26 18:19:39,803][105692] Updated weights for policy 0, policy_version 407207 (0.0008) [2023-12-26 18:19:40,189][105620] Updated weights for policy 1, policy_version 407441 (0.0007) [2023-12-26 18:19:40,250][105620] Updated weights for policy 1, policy_version 407451 (0.0009) [2023-12-26 18:19:40,315][105620] Updated weights for policy 1, policy_version 407461 (0.0010) [2023-12-26 18:19:40,377][105620] Updated weights for policy 1, policy_version 407471 (0.0010) [2023-12-26 18:19:40,627][105692] Updated weights for policy 0, policy_version 407217 (0.0010) [2023-12-26 18:19:40,689][105692] Updated weights for policy 0, policy_version 407227 (0.0010) [2023-12-26 18:19:40,752][105692] Updated weights for policy 0, policy_version 407237 (0.0008) [2023-12-26 18:19:41,012][105620] Updated weights for policy 1, policy_version 407481 (0.0010) [2023-12-26 18:19:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 208592896. Throughput: 0: 9715.6, 1: 9770.4. Samples: 208603652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:19:41,062][104569] Avg episode reward: [(0, '9265.184'), (1, '8818.475')] [2023-12-26 18:19:41,080][105620] Updated weights for policy 1, policy_version 407491 (0.0007) [2023-12-26 18:19:41,151][105620] Updated weights for policy 1, policy_version 407501 (0.0007) [2023-12-26 18:19:41,583][105692] Updated weights for policy 0, policy_version 407247 (0.0008) [2023-12-26 18:19:41,655][105692] Updated weights for policy 0, policy_version 407257 (0.0008) [2023-12-26 18:19:41,706][105692] Updated weights for policy 0, policy_version 407267 (0.0011) [2023-12-26 18:19:41,865][105620] Updated weights for policy 1, policy_version 407511 (0.0009) [2023-12-26 18:19:41,927][105620] Updated weights for policy 1, policy_version 407521 (0.0009) [2023-12-26 18:19:41,987][105620] Updated weights for policy 1, policy_version 407531 (0.0006) [2023-12-26 18:19:42,459][105692] Updated weights for policy 0, policy_version 407277 (0.0010) [2023-12-26 18:19:42,504][105692] Updated weights for policy 0, policy_version 407287 (0.0010) [2023-12-26 18:19:42,571][105692] Updated weights for policy 0, policy_version 407297 (0.0008) [2023-12-26 18:19:42,687][105620] Updated weights for policy 1, policy_version 407541 (0.0007) [2023-12-26 18:19:42,747][105620] Updated weights for policy 1, policy_version 407551 (0.0010) [2023-12-26 18:19:42,799][105620] Updated weights for policy 1, policy_version 407561 (0.0010) [2023-12-26 18:19:43,285][105692] Updated weights for policy 0, policy_version 407307 (0.0008) [2023-12-26 18:19:43,333][105692] Updated weights for policy 0, policy_version 407317 (0.0005) [2023-12-26 18:19:43,380][105692] Updated weights for policy 0, policy_version 407327 (0.0005) [2023-12-26 18:19:43,534][105620] Updated weights for policy 1, policy_version 407571 (0.0009) [2023-12-26 18:19:43,581][105620] Updated weights for policy 1, policy_version 407581 (0.0005) [2023-12-26 18:19:43,624][105620] Updated weights for policy 1, policy_version 407591 (0.0005) [2023-12-26 18:19:44,003][105692] Updated weights for policy 0, policy_version 407337 (0.0010) [2023-12-26 18:19:44,048][105692] Updated weights for policy 0, policy_version 407347 (0.0010) [2023-12-26 18:19:44,110][105692] Updated weights for policy 0, policy_version 407357 (0.0010) [2023-12-26 18:19:44,175][105692] Updated weights for policy 0, policy_version 407367 (0.0008) [2023-12-26 18:19:44,277][105620] Updated weights for policy 1, policy_version 407601 (0.0006) [2023-12-26 18:19:44,344][105620] Updated weights for policy 1, policy_version 407611 (0.0006) [2023-12-26 18:19:44,407][105620] Updated weights for policy 1, policy_version 407621 (0.0010) [2023-12-26 18:19:44,468][105620] Updated weights for policy 1, policy_version 407631 (0.0007) [2023-12-26 18:19:44,978][105692] Updated weights for policy 0, policy_version 407377 (0.0007) [2023-12-26 18:19:45,031][105692] Updated weights for policy 0, policy_version 407387 (0.0011) [2023-12-26 18:19:45,094][105692] Updated weights for policy 0, policy_version 407397 (0.0009) [2023-12-26 18:19:45,125][105620] Updated weights for policy 1, policy_version 407641 (0.0007) [2023-12-26 18:19:45,180][105620] Updated weights for policy 1, policy_version 407651 (0.0005) [2023-12-26 18:19:45,233][105620] Updated weights for policy 1, policy_version 407661 (0.0010) [2023-12-26 18:19:45,818][105692] Updated weights for policy 0, policy_version 407407 (0.0011) [2023-12-26 18:19:45,852][105620] Updated weights for policy 1, policy_version 407671 (0.0009) [2023-12-26 18:19:45,877][105692] Updated weights for policy 0, policy_version 407417 (0.0010) [2023-12-26 18:19:45,903][105620] Updated weights for policy 1, policy_version 407681 (0.0010) [2023-12-26 18:19:45,937][105692] Updated weights for policy 0, policy_version 407427 (0.0005) [2023-12-26 18:19:45,965][105620] Updated weights for policy 1, policy_version 407691 (0.0010) [2023-12-26 18:19:46,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 208699392. Throughput: 0: 9628.0, 1: 9856.2. Samples: 208661048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:19:46,063][104569] Avg episode reward: [(0, '9357.926'), (1, '9173.537')] [2023-12-26 18:19:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000407432_104316928.pth... [2023-12-26 18:19:46,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000407696_104382464.pth... [2023-12-26 18:19:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000406280_104022016.pth [2023-12-26 18:19:46,081][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000406544_104087552.pth [2023-12-26 18:19:46,559][105692] Updated weights for policy 0, policy_version 407437 (0.0007) [2023-12-26 18:19:46,613][105692] Updated weights for policy 0, policy_version 407447 (0.0005) [2023-12-26 18:19:46,666][105692] Updated weights for policy 0, policy_version 407457 (0.0005) [2023-12-26 18:19:46,723][105620] Updated weights for policy 1, policy_version 407701 (0.0010) [2023-12-26 18:19:46,781][105620] Updated weights for policy 1, policy_version 407711 (0.0010) [2023-12-26 18:19:46,832][105620] Updated weights for policy 1, policy_version 407721 (0.0010) [2023-12-26 18:19:47,237][105692] Updated weights for policy 0, policy_version 407467 (0.0005) [2023-12-26 18:19:47,287][105692] Updated weights for policy 0, policy_version 407477 (0.0005) [2023-12-26 18:19:47,343][105692] Updated weights for policy 0, policy_version 407487 (0.0005) [2023-12-26 18:19:47,616][105620] Updated weights for policy 1, policy_version 407731 (0.0009) [2023-12-26 18:19:47,677][105620] Updated weights for policy 1, policy_version 407741 (0.0005) [2023-12-26 18:19:47,739][105620] Updated weights for policy 1, policy_version 407751 (0.0007) [2023-12-26 18:19:48,023][105692] Updated weights for policy 0, policy_version 407497 (0.0009) [2023-12-26 18:19:48,076][105692] Updated weights for policy 0, policy_version 407507 (0.0007) [2023-12-26 18:19:48,135][105692] Updated weights for policy 0, policy_version 407517 (0.0008) [2023-12-26 18:19:48,188][105692] Updated weights for policy 0, policy_version 407527 (0.0005) [2023-12-26 18:19:48,435][105620] Updated weights for policy 1, policy_version 407761 (0.0009) [2023-12-26 18:19:48,495][105620] Updated weights for policy 1, policy_version 407771 (0.0007) [2023-12-26 18:19:48,554][105620] Updated weights for policy 1, policy_version 407781 (0.0008) [2023-12-26 18:19:48,599][105620] Updated weights for policy 1, policy_version 407791 (0.0008) [2023-12-26 18:19:48,921][105692] Updated weights for policy 0, policy_version 407537 (0.0010) [2023-12-26 18:19:48,979][105692] Updated weights for policy 0, policy_version 407547 (0.0010) [2023-12-26 18:19:49,031][105692] Updated weights for policy 0, policy_version 407557 (0.0010) [2023-12-26 18:19:49,371][105620] Updated weights for policy 1, policy_version 407801 (0.0008) [2023-12-26 18:19:49,425][105620] Updated weights for policy 1, policy_version 407811 (0.0008) [2023-12-26 18:19:49,470][105620] Updated weights for policy 1, policy_version 407821 (0.0007) [2023-12-26 18:19:49,808][105692] Updated weights for policy 0, policy_version 407567 (0.0010) [2023-12-26 18:19:49,830][105585] KL-divergence is very high: 274.8238 [2023-12-26 18:19:49,879][105692] Updated weights for policy 0, policy_version 407577 (0.0011) [2023-12-26 18:19:49,885][105585] KL-divergence is very high: 419.7384 [2023-12-26 18:19:49,944][105585] KL-divergence is very high: 338.7520 [2023-12-26 18:19:49,950][105692] Updated weights for policy 0, policy_version 407587 (0.0011) [2023-12-26 18:19:50,230][105620] Updated weights for policy 1, policy_version 407831 (0.0007) [2023-12-26 18:19:50,288][105620] Updated weights for policy 1, policy_version 407841 (0.0008) [2023-12-26 18:19:50,352][105620] Updated weights for policy 1, policy_version 407851 (0.0008) [2023-12-26 18:19:50,693][105692] Updated weights for policy 0, policy_version 407597 (0.0008) [2023-12-26 18:19:50,764][105692] Updated weights for policy 0, policy_version 407607 (0.0006) [2023-12-26 18:19:50,831][105692] Updated weights for policy 0, policy_version 407617 (0.0006) [2023-12-26 18:19:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 208789504. Throughput: 0: 9735.0, 1: 9824.4. Samples: 208780068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:19:51,062][104569] Avg episode reward: [(0, '9176.826'), (1, '9177.197')] [2023-12-26 18:19:51,141][105620] Updated weights for policy 1, policy_version 407861 (0.0009) [2023-12-26 18:19:51,189][105620] Updated weights for policy 1, policy_version 407871 (0.0008) [2023-12-26 18:19:51,241][105620] Updated weights for policy 1, policy_version 407881 (0.0008) [2023-12-26 18:19:51,455][105692] Updated weights for policy 0, policy_version 407627 (0.0006) [2023-12-26 18:19:51,522][105692] Updated weights for policy 0, policy_version 407637 (0.0005) [2023-12-26 18:19:51,575][105692] Updated weights for policy 0, policy_version 407647 (0.0006) [2023-12-26 18:19:52,058][105620] Updated weights for policy 1, policy_version 407891 (0.0009) [2023-12-26 18:19:52,120][105620] Updated weights for policy 1, policy_version 407901 (0.0011) [2023-12-26 18:19:52,181][105620] Updated weights for policy 1, policy_version 407911 (0.0008) [2023-12-26 18:19:52,183][105692] Updated weights for policy 0, policy_version 407657 (0.0009) [2023-12-26 18:19:52,238][105692] Updated weights for policy 0, policy_version 407667 (0.0005) [2023-12-26 18:19:52,301][105692] Updated weights for policy 0, policy_version 407677 (0.0009) [2023-12-26 18:19:52,356][105585] KL-divergence is very high: 137.8144 [2023-12-26 18:19:52,360][105692] Updated weights for policy 0, policy_version 407687 (0.0008) [2023-12-26 18:19:52,871][105620] Updated weights for policy 1, policy_version 407921 (0.0010) [2023-12-26 18:19:52,934][105620] Updated weights for policy 1, policy_version 407931 (0.0008) [2023-12-26 18:19:52,994][105620] Updated weights for policy 1, policy_version 407941 (0.0007) [2023-12-26 18:19:53,042][105620] Updated weights for policy 1, policy_version 407951 (0.0005) [2023-12-26 18:19:53,041][105692] Updated weights for policy 0, policy_version 407697 (0.0009) [2023-12-26 18:19:53,098][105692] Updated weights for policy 0, policy_version 407707 (0.0010) [2023-12-26 18:19:53,112][105585] KL-divergence is very high: 109.7423 [2023-12-26 18:19:53,158][105692] Updated weights for policy 0, policy_version 407717 (0.0009) [2023-12-26 18:19:53,730][105620] Updated weights for policy 1, policy_version 407961 (0.0005) [2023-12-26 18:19:53,788][105620] Updated weights for policy 1, policy_version 407971 (0.0005) [2023-12-26 18:19:53,847][105620] Updated weights for policy 1, policy_version 407981 (0.0005) [2023-12-26 18:19:53,935][105692] Updated weights for policy 0, policy_version 407727 (0.0008) [2023-12-26 18:19:53,993][105692] Updated weights for policy 0, policy_version 407737 (0.0009) [2023-12-26 18:19:54,054][105692] Updated weights for policy 0, policy_version 407747 (0.0007) [2023-12-26 18:19:54,458][105620] Updated weights for policy 1, policy_version 407991 (0.0005) [2023-12-26 18:19:54,503][105620] Updated weights for policy 1, policy_version 408001 (0.0005) [2023-12-26 18:19:54,557][105620] Updated weights for policy 1, policy_version 408011 (0.0005) [2023-12-26 18:19:54,778][105692] Updated weights for policy 0, policy_version 407757 (0.0005) [2023-12-26 18:19:54,834][105692] Updated weights for policy 0, policy_version 407767 (0.0006) [2023-12-26 18:19:54,881][105692] Updated weights for policy 0, policy_version 407777 (0.0008) [2023-12-26 18:19:55,255][105620] Updated weights for policy 1, policy_version 408021 (0.0007) [2023-12-26 18:19:55,305][105620] Updated weights for policy 1, policy_version 408031 (0.0005) [2023-12-26 18:19:55,359][105620] Updated weights for policy 1, policy_version 408041 (0.0006) [2023-12-26 18:19:55,657][105692] Updated weights for policy 0, policy_version 407787 (0.0008) [2023-12-26 18:19:55,725][105692] Updated weights for policy 0, policy_version 407797 (0.0005) [2023-12-26 18:19:55,776][105692] Updated weights for policy 0, policy_version 407807 (0.0005) [2023-12-26 18:19:55,999][105620] Updated weights for policy 1, policy_version 408051 (0.0007) [2023-12-26 18:19:56,056][105620] Updated weights for policy 1, policy_version 408061 (0.0009) [2023-12-26 18:19:56,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.2, 300 sec: 19660.8). Total num frames: 208887808. Throughput: 0: 9782.5, 1: 9850.6. Samples: 208897916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:19:56,063][104569] Avg episode reward: [(0, '7793.266'), (1, '9176.904')] [2023-12-26 18:19:56,109][105620] Updated weights for policy 1, policy_version 408071 (0.0010) [2023-12-26 18:19:56,291][105692] Updated weights for policy 0, policy_version 407817 (0.0005) [2023-12-26 18:19:56,362][105692] Updated weights for policy 0, policy_version 407827 (0.0005) [2023-12-26 18:19:56,426][105692] Updated weights for policy 0, policy_version 407837 (0.0008) [2023-12-26 18:19:56,485][105692] Updated weights for policy 0, policy_version 407847 (0.0009) [2023-12-26 18:19:56,788][105620] Updated weights for policy 1, policy_version 408082 (0.0010) [2023-12-26 18:19:56,861][105620] Updated weights for policy 1, policy_version 408092 (0.0010) [2023-12-26 18:19:56,921][105620] Updated weights for policy 1, policy_version 408102 (0.0009) [2023-12-26 18:19:56,976][105620] Updated weights for policy 1, policy_version 408112 (0.0009) [2023-12-26 18:19:57,090][105692] Updated weights for policy 0, policy_version 407857 (0.0010) [2023-12-26 18:19:57,144][105692] Updated weights for policy 0, policy_version 407868 (0.0010) [2023-12-26 18:19:57,192][105692] Updated weights for policy 0, policy_version 407879 (0.0008) [2023-12-26 18:19:57,567][105620] Updated weights for policy 1, policy_version 408122 (0.0007) [2023-12-26 18:19:57,624][105620] Updated weights for policy 1, policy_version 408132 (0.0009) [2023-12-26 18:19:57,681][105620] Updated weights for policy 1, policy_version 408142 (0.0009) [2023-12-26 18:19:57,785][105692] Updated weights for policy 0, policy_version 407889 (0.0009) [2023-12-26 18:19:57,840][105692] Updated weights for policy 0, policy_version 407899 (0.0010) [2023-12-26 18:19:57,894][105692] Updated weights for policy 0, policy_version 407909 (0.0010) [2023-12-26 18:19:58,536][105620] Updated weights for policy 1, policy_version 408152 (0.0009) [2023-12-26 18:19:58,599][105620] Updated weights for policy 1, policy_version 408162 (0.0008) [2023-12-26 18:19:58,623][105692] Updated weights for policy 0, policy_version 407919 (0.0010) [2023-12-26 18:19:58,665][105620] Updated weights for policy 1, policy_version 408172 (0.0009) [2023-12-26 18:19:58,678][105586] KL-divergence is very high: 292.5689 [2023-12-26 18:19:58,686][105692] Updated weights for policy 0, policy_version 407929 (0.0008) [2023-12-26 18:19:58,752][105692] Updated weights for policy 0, policy_version 407939 (0.0011) [2023-12-26 18:19:59,372][105620] Updated weights for policy 1, policy_version 408182 (0.0007) [2023-12-26 18:19:59,426][105620] Updated weights for policy 1, policy_version 408192 (0.0006) [2023-12-26 18:19:59,475][105620] Updated weights for policy 1, policy_version 408202 (0.0005) [2023-12-26 18:19:59,487][105692] Updated weights for policy 0, policy_version 407949 (0.0007) [2023-12-26 18:19:59,540][105692] Updated weights for policy 0, policy_version 407959 (0.0006) [2023-12-26 18:19:59,594][105692] Updated weights for policy 0, policy_version 407969 (0.0009) [2023-12-26 18:20:00,184][105620] Updated weights for policy 1, policy_version 408212 (0.0007) [2023-12-26 18:20:00,243][105620] Updated weights for policy 1, policy_version 408222 (0.0008) [2023-12-26 18:20:00,302][105620] Updated weights for policy 1, policy_version 408232 (0.0008) [2023-12-26 18:20:00,324][105692] Updated weights for policy 0, policy_version 407979 (0.0009) [2023-12-26 18:20:00,381][105692] Updated weights for policy 0, policy_version 407989 (0.0010) [2023-12-26 18:20:00,429][105692] Updated weights for policy 0, policy_version 407999 (0.0010) [2023-12-26 18:20:00,991][105620] Updated weights for policy 1, policy_version 408242 (0.0007) [2023-12-26 18:20:01,047][105620] Updated weights for policy 1, policy_version 408252 (0.0010) [2023-12-26 18:20:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 208986112. Throughput: 0: 9915.6, 1: 9858.1. Samples: 208960684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:20:01,063][104569] Avg episode reward: [(0, '7434.368'), (1, '9172.430')] [2023-12-26 18:20:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000408008_104464384.pth... [2023-12-26 18:20:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000406888_104177664.pth [2023-12-26 18:20:01,106][105620] Updated weights for policy 1, policy_version 408262 (0.0011) [2023-12-26 18:20:01,163][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000408272_104529920.pth... [2023-12-26 18:20:01,166][105620] Updated weights for policy 1, policy_version 408272 (0.0011) [2023-12-26 18:20:01,166][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000407120_104235008.pth [2023-12-26 18:20:01,180][105692] Updated weights for policy 0, policy_version 408009 (0.0010) [2023-12-26 18:20:01,230][105692] Updated weights for policy 0, policy_version 408019 (0.0008) [2023-12-26 18:20:01,237][105585] KL-divergence is very high: 103.1920 [2023-12-26 18:20:01,265][105585] KL-divergence is very high: 103.3293 [2023-12-26 18:20:01,295][105692] Updated weights for policy 0, policy_version 408029 (0.0007) [2023-12-26 18:20:01,366][105692] Updated weights for policy 0, policy_version 408039 (0.0008) [2023-12-26 18:20:01,924][105620] Updated weights for policy 1, policy_version 408282 (0.0010) [2023-12-26 18:20:01,984][105620] Updated weights for policy 1, policy_version 408292 (0.0009) [2023-12-26 18:20:02,050][105620] Updated weights for policy 1, policy_version 408302 (0.0009) [2023-12-26 18:20:02,089][105692] Updated weights for policy 0, policy_version 408049 (0.0007) [2023-12-26 18:20:02,144][105692] Updated weights for policy 0, policy_version 408059 (0.0008) [2023-12-26 18:20:02,199][105692] Updated weights for policy 0, policy_version 408069 (0.0008) [2023-12-26 18:20:02,748][105620] Updated weights for policy 1, policy_version 408312 (0.0010) [2023-12-26 18:20:02,807][105620] Updated weights for policy 1, policy_version 408323 (0.0011) [2023-12-26 18:20:02,864][105692] Updated weights for policy 0, policy_version 408079 (0.0008) [2023-12-26 18:20:02,871][105620] Updated weights for policy 1, policy_version 408333 (0.0008) [2023-12-26 18:20:02,920][105692] Updated weights for policy 0, policy_version 408089 (0.0011) [2023-12-26 18:20:02,969][105692] Updated weights for policy 0, policy_version 408099 (0.0010) [2023-12-26 18:20:03,498][105620] Updated weights for policy 1, policy_version 408343 (0.0007) [2023-12-26 18:20:03,548][105620] Updated weights for policy 1, policy_version 408353 (0.0007) [2023-12-26 18:20:03,608][105620] Updated weights for policy 1, policy_version 408363 (0.0009) [2023-12-26 18:20:03,669][105692] Updated weights for policy 0, policy_version 408109 (0.0008) [2023-12-26 18:20:03,721][105692] Updated weights for policy 0, policy_version 408119 (0.0005) [2023-12-26 18:20:03,776][105692] Updated weights for policy 0, policy_version 408129 (0.0006) [2023-12-26 18:20:04,260][105620] Updated weights for policy 1, policy_version 408373 (0.0008) [2023-12-26 18:20:04,326][105620] Updated weights for policy 1, policy_version 408383 (0.0006) [2023-12-26 18:20:04,391][105620] Updated weights for policy 1, policy_version 408393 (0.0006) [2023-12-26 18:20:04,478][105692] Updated weights for policy 0, policy_version 408139 (0.0008) [2023-12-26 18:20:04,531][105692] Updated weights for policy 0, policy_version 408149 (0.0005) [2023-12-26 18:20:04,581][105692] Updated weights for policy 0, policy_version 408159 (0.0008) [2023-12-26 18:20:05,070][105620] Updated weights for policy 1, policy_version 408403 (0.0007) [2023-12-26 18:20:05,132][105620] Updated weights for policy 1, policy_version 408413 (0.0009) [2023-12-26 18:20:05,190][105620] Updated weights for policy 1, policy_version 408423 (0.0007) [2023-12-26 18:20:05,300][105692] Updated weights for policy 0, policy_version 408169 (0.0009) [2023-12-26 18:20:05,362][105692] Updated weights for policy 0, policy_version 408179 (0.0009) [2023-12-26 18:20:05,423][105692] Updated weights for policy 0, policy_version 408189 (0.0009) [2023-12-26 18:20:05,480][105692] Updated weights for policy 0, policy_version 408199 (0.0011) [2023-12-26 18:20:05,782][105620] Updated weights for policy 1, policy_version 408433 (0.0006) [2023-12-26 18:20:05,833][105620] Updated weights for policy 1, policy_version 408443 (0.0009) [2023-12-26 18:20:05,879][105620] Updated weights for policy 1, policy_version 408453 (0.0008) [2023-12-26 18:20:05,929][105620] Updated weights for policy 1, policy_version 408463 (0.0009) [2023-12-26 18:20:06,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 209092608. Throughput: 0: 9950.5, 1: 9787.5. Samples: 209079656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:20:06,063][104569] Avg episode reward: [(0, '8071.375'), (1, '9175.900')] [2023-12-26 18:20:06,287][105692] Updated weights for policy 0, policy_version 408209 (0.0009) [2023-12-26 18:20:06,348][105692] Updated weights for policy 0, policy_version 408219 (0.0009) [2023-12-26 18:20:06,400][105692] Updated weights for policy 0, policy_version 408229 (0.0009) [2023-12-26 18:20:06,669][105620] Updated weights for policy 1, policy_version 408473 (0.0009) [2023-12-26 18:20:06,732][105620] Updated weights for policy 1, policy_version 408483 (0.0009) [2023-12-26 18:20:06,795][105620] Updated weights for policy 1, policy_version 408493 (0.0009) [2023-12-26 18:20:07,183][105692] Updated weights for policy 0, policy_version 408239 (0.0009) [2023-12-26 18:20:07,242][105692] Updated weights for policy 0, policy_version 408249 (0.0009) [2023-12-26 18:20:07,299][105692] Updated weights for policy 0, policy_version 408259 (0.0009) [2023-12-26 18:20:07,537][105620] Updated weights for policy 1, policy_version 408503 (0.0009) [2023-12-26 18:20:07,583][105620] Updated weights for policy 1, policy_version 408513 (0.0008) [2023-12-26 18:20:07,633][105620] Updated weights for policy 1, policy_version 408523 (0.0009) [2023-12-26 18:20:08,056][105692] Updated weights for policy 0, policy_version 408269 (0.0007) [2023-12-26 18:20:08,113][105692] Updated weights for policy 0, policy_version 408279 (0.0005) [2023-12-26 18:20:08,165][105692] Updated weights for policy 0, policy_version 408289 (0.0006) [2023-12-26 18:20:08,422][105620] Updated weights for policy 1, policy_version 408533 (0.0009) [2023-12-26 18:20:08,483][105620] Updated weights for policy 1, policy_version 408543 (0.0009) [2023-12-26 18:20:08,544][105620] Updated weights for policy 1, policy_version 408553 (0.0009) [2023-12-26 18:20:08,880][105692] Updated weights for policy 0, policy_version 408299 (0.0006) [2023-12-26 18:20:08,937][105692] Updated weights for policy 0, policy_version 408309 (0.0008) [2023-12-26 18:20:09,007][105692] Updated weights for policy 0, policy_version 408319 (0.0005) [2023-12-26 18:20:09,114][105620] Updated weights for policy 1, policy_version 408563 (0.0007) [2023-12-26 18:20:09,174][105620] Updated weights for policy 1, policy_version 408573 (0.0008) [2023-12-26 18:20:09,243][105620] Updated weights for policy 1, policy_version 408583 (0.0009) [2023-12-26 18:20:09,782][105692] Updated weights for policy 0, policy_version 408329 (0.0007) [2023-12-26 18:20:09,850][105692] Updated weights for policy 0, policy_version 408339 (0.0011) [2023-12-26 18:20:09,915][105692] Updated weights for policy 0, policy_version 408349 (0.0010) [2023-12-26 18:20:09,982][105692] Updated weights for policy 0, policy_version 408359 (0.0008) [2023-12-26 18:20:10,017][105620] Updated weights for policy 1, policy_version 408593 (0.0009) [2023-12-26 18:20:10,070][105620] Updated weights for policy 1, policy_version 408603 (0.0007) [2023-12-26 18:20:10,135][105620] Updated weights for policy 1, policy_version 408613 (0.0009) [2023-12-26 18:20:10,200][105620] Updated weights for policy 1, policy_version 408623 (0.0008) [2023-12-26 18:20:10,736][105692] Updated weights for policy 0, policy_version 408369 (0.0007) [2023-12-26 18:20:10,792][105692] Updated weights for policy 0, policy_version 408379 (0.0005) [2023-12-26 18:20:10,852][105692] Updated weights for policy 0, policy_version 408389 (0.0005) [2023-12-26 18:20:10,963][105620] Updated weights for policy 1, policy_version 408633 (0.0009) [2023-12-26 18:20:11,024][105620] Updated weights for policy 1, policy_version 408643 (0.0008) [2023-12-26 18:20:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 209182720. Throughput: 0: 9847.3, 1: 9846.4. Samples: 209194228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:20:11,063][104569] Avg episode reward: [(0, '8646.688'), (1, '9089.790')] [2023-12-26 18:20:11,089][105620] Updated weights for policy 1, policy_version 408653 (0.0010) [2023-12-26 18:20:11,550][105692] Updated weights for policy 0, policy_version 408399 (0.0008) [2023-12-26 18:20:11,621][105692] Updated weights for policy 0, policy_version 408409 (0.0009) [2023-12-26 18:20:11,692][105692] Updated weights for policy 0, policy_version 408419 (0.0008) [2023-12-26 18:20:11,915][105620] Updated weights for policy 1, policy_version 408663 (0.0006) [2023-12-26 18:20:11,970][105620] Updated weights for policy 1, policy_version 408673 (0.0006) [2023-12-26 18:20:12,020][105620] Updated weights for policy 1, policy_version 408683 (0.0005) [2023-12-26 18:20:12,501][105692] Updated weights for policy 0, policy_version 408429 (0.0009) [2023-12-26 18:20:12,553][105692] Updated weights for policy 0, policy_version 408439 (0.0010) [2023-12-26 18:20:12,601][105620] Updated weights for policy 1, policy_version 408693 (0.0007) [2023-12-26 18:20:12,611][105692] Updated weights for policy 0, policy_version 408449 (0.0007) [2023-12-26 18:20:12,660][105620] Updated weights for policy 1, policy_version 408703 (0.0008) [2023-12-26 18:20:12,720][105620] Updated weights for policy 1, policy_version 408713 (0.0009) [2023-12-26 18:20:13,326][105692] Updated weights for policy 0, policy_version 408459 (0.0007) [2023-12-26 18:20:13,385][105692] Updated weights for policy 0, policy_version 408469 (0.0005) [2023-12-26 18:20:13,438][105692] Updated weights for policy 0, policy_version 408479 (0.0005) [2023-12-26 18:20:13,571][105620] Updated weights for policy 1, policy_version 408723 (0.0008) [2023-12-26 18:20:13,629][105620] Updated weights for policy 1, policy_version 408733 (0.0010) [2023-12-26 18:20:13,683][105620] Updated weights for policy 1, policy_version 408743 (0.0010) [2023-12-26 18:20:13,977][105692] Updated weights for policy 0, policy_version 408489 (0.0006) [2023-12-26 18:20:14,038][105692] Updated weights for policy 0, policy_version 408499 (0.0010) [2023-12-26 18:20:14,089][105585] KL-divergence is very high: 104.3966 [2023-12-26 18:20:14,089][105692] Updated weights for policy 0, policy_version 408509 (0.0010) [2023-12-26 18:20:14,148][105692] Updated weights for policy 0, policy_version 408519 (0.0010) [2023-12-26 18:20:14,412][105620] Updated weights for policy 1, policy_version 408753 (0.0008) [2023-12-26 18:20:14,468][105620] Updated weights for policy 1, policy_version 408763 (0.0009) [2023-12-26 18:20:14,519][105620] Updated weights for policy 1, policy_version 408773 (0.0008) [2023-12-26 18:20:14,570][105620] Updated weights for policy 1, policy_version 408783 (0.0008) [2023-12-26 18:20:14,878][105692] Updated weights for policy 0, policy_version 408529 (0.0008) [2023-12-26 18:20:14,950][105692] Updated weights for policy 0, policy_version 408539 (0.0008) [2023-12-26 18:20:15,020][105692] Updated weights for policy 0, policy_version 408549 (0.0008) [2023-12-26 18:20:15,366][105620] Updated weights for policy 1, policy_version 408793 (0.0009) [2023-12-26 18:20:15,430][105620] Updated weights for policy 1, policy_version 408803 (0.0009) [2023-12-26 18:20:15,491][105620] Updated weights for policy 1, policy_version 408813 (0.0009) [2023-12-26 18:20:15,704][105692] Updated weights for policy 0, policy_version 408559 (0.0008) [2023-12-26 18:20:15,759][105692] Updated weights for policy 0, policy_version 408569 (0.0009) [2023-12-26 18:20:15,817][105692] Updated weights for policy 0, policy_version 408579 (0.0010) [2023-12-26 18:20:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 209281024. Throughput: 0: 9733.8, 1: 9825.5. Samples: 209250924. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:20:16,062][104569] Avg episode reward: [(0, '8999.150'), (1, '9003.081')] [2023-12-26 18:20:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000408816_104669184.pth... [2023-12-26 18:20:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000408584_104611840.pth... [2023-12-26 18:20:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000407696_104382464.pth [2023-12-26 18:20:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000407432_104316928.pth [2023-12-26 18:20:16,148][105620] Updated weights for policy 1, policy_version 408823 (0.0010) [2023-12-26 18:20:16,210][105620] Updated weights for policy 1, policy_version 408833 (0.0010) [2023-12-26 18:20:16,271][105620] Updated weights for policy 1, policy_version 408843 (0.0009) [2023-12-26 18:20:16,595][105692] Updated weights for policy 0, policy_version 408589 (0.0009) [2023-12-26 18:20:16,662][105692] Updated weights for policy 0, policy_version 408599 (0.0008) [2023-12-26 18:20:16,723][105692] Updated weights for policy 0, policy_version 408609 (0.0008) [2023-12-26 18:20:17,045][105620] Updated weights for policy 1, policy_version 408853 (0.0009) [2023-12-26 18:20:17,103][105620] Updated weights for policy 1, policy_version 408863 (0.0010) [2023-12-26 18:20:17,167][105620] Updated weights for policy 1, policy_version 408873 (0.0009) [2023-12-26 18:20:17,317][105692] Updated weights for policy 0, policy_version 408619 (0.0008) [2023-12-26 18:20:17,361][105692] Updated weights for policy 0, policy_version 408629 (0.0010) [2023-12-26 18:20:17,416][105692] Updated weights for policy 0, policy_version 408639 (0.0010) [2023-12-26 18:20:18,004][105620] Updated weights for policy 1, policy_version 408883 (0.0008) [2023-12-26 18:20:18,036][105692] Updated weights for policy 0, policy_version 408649 (0.0010) [2023-12-26 18:20:18,066][105620] Updated weights for policy 1, policy_version 408893 (0.0008) [2023-12-26 18:20:18,095][105692] Updated weights for policy 0, policy_version 408659 (0.0008) [2023-12-26 18:20:18,133][105620] Updated weights for policy 1, policy_version 408903 (0.0009) [2023-12-26 18:20:18,151][105692] Updated weights for policy 0, policy_version 408669 (0.0005) [2023-12-26 18:20:18,217][105692] Updated weights for policy 0, policy_version 408679 (0.0005) [2023-12-26 18:20:18,802][105620] Updated weights for policy 1, policy_version 408913 (0.0008) [2023-12-26 18:20:18,865][105620] Updated weights for policy 1, policy_version 408923 (0.0006) [2023-12-26 18:20:18,933][105620] Updated weights for policy 1, policy_version 408933 (0.0006) [2023-12-26 18:20:18,959][105692] Updated weights for policy 0, policy_version 408689 (0.0008) [2023-12-26 18:20:18,991][105620] Updated weights for policy 1, policy_version 408943 (0.0006) [2023-12-26 18:20:19,014][105692] Updated weights for policy 0, policy_version 408699 (0.0008) [2023-12-26 18:20:19,075][105692] Updated weights for policy 0, policy_version 408709 (0.0010) [2023-12-26 18:20:19,574][105620] Updated weights for policy 1, policy_version 408953 (0.0007) [2023-12-26 18:20:19,634][105620] Updated weights for policy 1, policy_version 408963 (0.0009) [2023-12-26 18:20:19,697][105620] Updated weights for policy 1, policy_version 408973 (0.0009) [2023-12-26 18:20:19,916][105692] Updated weights for policy 0, policy_version 408719 (0.0008) [2023-12-26 18:20:19,981][105692] Updated weights for policy 0, policy_version 408729 (0.0007) [2023-12-26 18:20:20,038][105692] Updated weights for policy 0, policy_version 408739 (0.0006) [2023-12-26 18:20:20,454][105620] Updated weights for policy 1, policy_version 408983 (0.0009) [2023-12-26 18:20:20,522][105620] Updated weights for policy 1, policy_version 408993 (0.0009) [2023-12-26 18:20:20,590][105620] Updated weights for policy 1, policy_version 409003 (0.0008) [2023-12-26 18:20:20,733][105692] Updated weights for policy 0, policy_version 408749 (0.0008) [2023-12-26 18:20:20,795][105692] Updated weights for policy 0, policy_version 408759 (0.0009) [2023-12-26 18:20:20,858][105692] Updated weights for policy 0, policy_version 408769 (0.0009) [2023-12-26 18:20:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 209379328. Throughput: 0: 9761.2, 1: 9823.7. Samples: 209367604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:20:21,063][104569] Avg episode reward: [(0, '9263.400'), (1, '9181.383')] [2023-12-26 18:20:21,382][105620] Updated weights for policy 1, policy_version 409013 (0.0008) [2023-12-26 18:20:21,445][105620] Updated weights for policy 1, policy_version 409023 (0.0009) [2023-12-26 18:20:21,495][105620] Updated weights for policy 1, policy_version 409033 (0.0008) [2023-12-26 18:20:21,657][105692] Updated weights for policy 0, policy_version 408779 (0.0008) [2023-12-26 18:20:21,727][105692] Updated weights for policy 0, policy_version 408789 (0.0008) [2023-12-26 18:20:21,785][105692] Updated weights for policy 0, policy_version 408799 (0.0008) [2023-12-26 18:20:22,277][105620] Updated weights for policy 1, policy_version 409043 (0.0007) [2023-12-26 18:20:22,339][105620] Updated weights for policy 1, policy_version 409053 (0.0009) [2023-12-26 18:20:22,406][105620] Updated weights for policy 1, policy_version 409063 (0.0009) [2023-12-26 18:20:22,546][105692] Updated weights for policy 0, policy_version 408809 (0.0009) [2023-12-26 18:20:22,606][105692] Updated weights for policy 0, policy_version 408819 (0.0009) [2023-12-26 18:20:22,669][105692] Updated weights for policy 0, policy_version 408829 (0.0008) [2023-12-26 18:20:22,736][105692] Updated weights for policy 0, policy_version 408839 (0.0008) [2023-12-26 18:20:23,200][105620] Updated weights for policy 1, policy_version 409073 (0.0009) [2023-12-26 18:20:23,249][105620] Updated weights for policy 1, policy_version 409083 (0.0009) [2023-12-26 18:20:23,300][105620] Updated weights for policy 1, policy_version 409093 (0.0009) [2023-12-26 18:20:23,353][105620] Updated weights for policy 1, policy_version 409103 (0.0008) [2023-12-26 18:20:23,407][105692] Updated weights for policy 0, policy_version 408849 (0.0008) [2023-12-26 18:20:23,465][105692] Updated weights for policy 0, policy_version 408859 (0.0009) [2023-12-26 18:20:23,518][105692] Updated weights for policy 0, policy_version 408869 (0.0008) [2023-12-26 18:20:24,040][105620] Updated weights for policy 1, policy_version 409113 (0.0005) [2023-12-26 18:20:24,094][105620] Updated weights for policy 1, policy_version 409123 (0.0005) [2023-12-26 18:20:24,143][105620] Updated weights for policy 1, policy_version 409133 (0.0007) [2023-12-26 18:20:24,248][105692] Updated weights for policy 0, policy_version 408879 (0.0008) [2023-12-26 18:20:24,296][105692] Updated weights for policy 0, policy_version 408889 (0.0008) [2023-12-26 18:20:24,349][105692] Updated weights for policy 0, policy_version 408899 (0.0008) [2023-12-26 18:20:24,855][105620] Updated weights for policy 1, policy_version 409143 (0.0010) [2023-12-26 18:20:24,915][105620] Updated weights for policy 1, policy_version 409153 (0.0007) [2023-12-26 18:20:24,975][105620] Updated weights for policy 1, policy_version 409163 (0.0010) [2023-12-26 18:20:25,104][105692] Updated weights for policy 0, policy_version 408909 (0.0008) [2023-12-26 18:20:25,169][105692] Updated weights for policy 0, policy_version 408919 (0.0009) [2023-12-26 18:20:25,232][105692] Updated weights for policy 0, policy_version 408929 (0.0008) [2023-12-26 18:20:25,669][105620] Updated weights for policy 1, policy_version 409173 (0.0010) [2023-12-26 18:20:25,718][105620] Updated weights for policy 1, policy_version 409183 (0.0008) [2023-12-26 18:20:25,765][105620] Updated weights for policy 1, policy_version 409193 (0.0009) [2023-12-26 18:20:26,010][105692] Updated weights for policy 0, policy_version 408939 (0.0008) [2023-12-26 18:20:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 209469440. Throughput: 0: 9755.3, 1: 9727.8. Samples: 209480392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:20:26,062][104569] Avg episode reward: [(0, '9355.084'), (1, '8991.466')] [2023-12-26 18:20:26,074][105692] Updated weights for policy 0, policy_version 408949 (0.0006) [2023-12-26 18:20:26,122][105692] Updated weights for policy 0, policy_version 408959 (0.0007) [2023-12-26 18:20:26,515][105620] Updated weights for policy 1, policy_version 409203 (0.0010) [2023-12-26 18:20:26,558][105620] Updated weights for policy 1, policy_version 409213 (0.0010) [2023-12-26 18:20:26,606][105620] Updated weights for policy 1, policy_version 409223 (0.0010) [2023-12-26 18:20:26,869][105692] Updated weights for policy 0, policy_version 408969 (0.0008) [2023-12-26 18:20:26,922][105692] Updated weights for policy 0, policy_version 408979 (0.0010) [2023-12-26 18:20:26,977][105692] Updated weights for policy 0, policy_version 408989 (0.0010) [2023-12-26 18:20:27,036][105692] Updated weights for policy 0, policy_version 409000 (0.0010) [2023-12-26 18:20:27,271][105620] Updated weights for policy 1, policy_version 409233 (0.0010) [2023-12-26 18:20:27,322][105620] Updated weights for policy 1, policy_version 409243 (0.0010) [2023-12-26 18:20:27,382][105620] Updated weights for policy 1, policy_version 409253 (0.0010) [2023-12-26 18:20:27,449][105620] Updated weights for policy 1, policy_version 409263 (0.0010) [2023-12-26 18:20:27,778][105692] Updated weights for policy 0, policy_version 409010 (0.0008) [2023-12-26 18:20:27,826][105692] Updated weights for policy 0, policy_version 409020 (0.0008) [2023-12-26 18:20:27,871][105692] Updated weights for policy 0, policy_version 409030 (0.0007) [2023-12-26 18:20:28,122][105620] Updated weights for policy 1, policy_version 409273 (0.0006) [2023-12-26 18:20:28,177][105620] Updated weights for policy 1, policy_version 409283 (0.0005) [2023-12-26 18:20:28,227][105620] Updated weights for policy 1, policy_version 409293 (0.0007) [2023-12-26 18:20:28,689][105692] Updated weights for policy 0, policy_version 409040 (0.0008) [2023-12-26 18:20:28,749][105692] Updated weights for policy 0, policy_version 409050 (0.0008) [2023-12-26 18:20:28,807][105692] Updated weights for policy 0, policy_version 409060 (0.0008) [2023-12-26 18:20:28,886][105620] Updated weights for policy 1, policy_version 409303 (0.0008) [2023-12-26 18:20:28,934][105620] Updated weights for policy 1, policy_version 409313 (0.0009) [2023-12-26 18:20:28,984][105620] Updated weights for policy 1, policy_version 409323 (0.0010) [2023-12-26 18:20:29,570][105692] Updated weights for policy 0, policy_version 409070 (0.0009) [2023-12-26 18:20:29,620][105692] Updated weights for policy 0, policy_version 409081 (0.0009) [2023-12-26 18:20:29,677][105692] Updated weights for policy 0, policy_version 409091 (0.0008) [2023-12-26 18:20:29,683][105620] Updated weights for policy 1, policy_version 409333 (0.0009) [2023-12-26 18:20:29,742][105620] Updated weights for policy 1, policy_version 409343 (0.0007) [2023-12-26 18:20:29,809][105620] Updated weights for policy 1, policy_version 409353 (0.0005) [2023-12-26 18:20:30,478][105692] Updated weights for policy 0, policy_version 409101 (0.0008) [2023-12-26 18:20:30,533][105692] Updated weights for policy 0, policy_version 409111 (0.0009) [2023-12-26 18:20:30,546][105620] Updated weights for policy 1, policy_version 409363 (0.0008) [2023-12-26 18:20:30,590][105692] Updated weights for policy 0, policy_version 409121 (0.0007) [2023-12-26 18:20:30,596][105620] Updated weights for policy 1, policy_version 409373 (0.0006) [2023-12-26 18:20:30,639][105620] Updated weights for policy 1, policy_version 409383 (0.0005) [2023-12-26 18:20:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 209567744. Throughput: 0: 9760.3, 1: 9747.1. Samples: 209538876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:20:31,063][104569] Avg episode reward: [(0, '9173.753'), (1, '8991.854')] [2023-12-26 18:20:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000409128_104751104.pth... [2023-12-26 18:20:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000409392_104816640.pth... [2023-12-26 18:20:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000408008_104464384.pth [2023-12-26 18:20:31,090][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000408272_104529920.pth [2023-12-26 18:20:31,251][105620] Updated weights for policy 1, policy_version 409393 (0.0005) [2023-12-26 18:20:31,313][105620] Updated weights for policy 1, policy_version 409403 (0.0009) [2023-12-26 18:20:31,380][105620] Updated weights for policy 1, policy_version 409413 (0.0008) [2023-12-26 18:20:31,438][105620] Updated weights for policy 1, policy_version 409423 (0.0008) [2023-12-26 18:20:31,439][105692] Updated weights for policy 0, policy_version 409131 (0.0009) [2023-12-26 18:20:31,491][105692] Updated weights for policy 0, policy_version 409141 (0.0009) [2023-12-26 18:20:31,553][105692] Updated weights for policy 0, policy_version 409151 (0.0010) [2023-12-26 18:20:32,206][105620] Updated weights for policy 1, policy_version 409433 (0.0009) [2023-12-26 18:20:32,255][105620] Updated weights for policy 1, policy_version 409443 (0.0008) [2023-12-26 18:20:32,299][105692] Updated weights for policy 0, policy_version 409161 (0.0009) [2023-12-26 18:20:32,321][105620] Updated weights for policy 1, policy_version 409453 (0.0007) [2023-12-26 18:20:32,366][105692] Updated weights for policy 0, policy_version 409171 (0.0009) [2023-12-26 18:20:32,431][105692] Updated weights for policy 0, policy_version 409181 (0.0009) [2023-12-26 18:20:32,479][105692] Updated weights for policy 0, policy_version 409191 (0.0009) [2023-12-26 18:20:33,027][105620] Updated weights for policy 1, policy_version 409463 (0.0005) [2023-12-26 18:20:33,095][105620] Updated weights for policy 1, policy_version 409473 (0.0005) [2023-12-26 18:20:33,159][105620] Updated weights for policy 1, policy_version 409483 (0.0009) [2023-12-26 18:20:33,292][105692] Updated weights for policy 0, policy_version 409201 (0.0009) [2023-12-26 18:20:33,345][105692] Updated weights for policy 0, policy_version 409211 (0.0009) [2023-12-26 18:20:33,392][105692] Updated weights for policy 0, policy_version 409221 (0.0009) [2023-12-26 18:20:33,848][105620] Updated weights for policy 1, policy_version 409493 (0.0007) [2023-12-26 18:20:33,901][105620] Updated weights for policy 1, policy_version 409503 (0.0005) [2023-12-26 18:20:33,951][105620] Updated weights for policy 1, policy_version 409513 (0.0005) [2023-12-26 18:20:34,190][105692] Updated weights for policy 0, policy_version 409231 (0.0010) [2023-12-26 18:20:34,251][105692] Updated weights for policy 0, policy_version 409241 (0.0010) [2023-12-26 18:20:34,321][105692] Updated weights for policy 0, policy_version 409251 (0.0010) [2023-12-26 18:20:34,331][105585] KL-divergence is very high: 105.1700 [2023-12-26 18:20:34,592][105620] Updated weights for policy 1, policy_version 409523 (0.0006) [2023-12-26 18:20:34,657][105620] Updated weights for policy 1, policy_version 409533 (0.0009) [2023-12-26 18:20:34,726][105620] Updated weights for policy 1, policy_version 409543 (0.0007) [2023-12-26 18:20:35,095][105692] Updated weights for policy 0, policy_version 409261 (0.0009) [2023-12-26 18:20:35,148][105692] Updated weights for policy 0, policy_version 409271 (0.0010) [2023-12-26 18:20:35,209][105692] Updated weights for policy 0, policy_version 409281 (0.0010) [2023-12-26 18:20:35,420][105620] Updated weights for policy 1, policy_version 409553 (0.0008) [2023-12-26 18:20:35,471][105620] Updated weights for policy 1, policy_version 409563 (0.0010) [2023-12-26 18:20:35,523][105620] Updated weights for policy 1, policy_version 409573 (0.0010) [2023-12-26 18:20:35,574][105620] Updated weights for policy 1, policy_version 409583 (0.0010) [2023-12-26 18:20:35,915][105692] Updated weights for policy 0, policy_version 409291 (0.0010) [2023-12-26 18:20:35,964][105692] Updated weights for policy 0, policy_version 409301 (0.0010) [2023-12-26 18:20:36,017][105692] Updated weights for policy 0, policy_version 409311 (0.0010) [2023-12-26 18:20:36,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 209657856. Throughput: 0: 9606.9, 1: 9769.4. Samples: 209652004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:20:36,062][104569] Avg episode reward: [(0, '9082.534'), (1, '9086.489')] [2023-12-26 18:20:36,320][105620] Updated weights for policy 1, policy_version 409593 (0.0007) [2023-12-26 18:20:36,383][105620] Updated weights for policy 1, policy_version 409603 (0.0006) [2023-12-26 18:20:36,405][105586] KL-divergence is very high: 138.2023 [2023-12-26 18:20:36,452][105620] Updated weights for policy 1, policy_version 409613 (0.0006) [2023-12-26 18:20:36,458][105586] KL-divergence is very high: 145.2347 [2023-12-26 18:20:36,862][105692] Updated weights for policy 0, policy_version 409321 (0.0011) [2023-12-26 18:20:36,928][105692] Updated weights for policy 0, policy_version 409331 (0.0009) [2023-12-26 18:20:36,975][105585] KL-divergence is very high: 189.7126 [2023-12-26 18:20:36,994][105692] Updated weights for policy 0, policy_version 409341 (0.0009) [2023-12-26 18:20:37,027][105585] KL-divergence is very high: 224.3763 [2023-12-26 18:20:37,054][105692] Updated weights for policy 0, policy_version 409351 (0.0008) [2023-12-26 18:20:37,065][105620] Updated weights for policy 1, policy_version 409623 (0.0007) [2023-12-26 18:20:37,116][105620] Updated weights for policy 1, policy_version 409633 (0.0009) [2023-12-26 18:20:37,174][105620] Updated weights for policy 1, policy_version 409643 (0.0009) [2023-12-26 18:20:37,816][105692] Updated weights for policy 0, policy_version 409361 (0.0009) [2023-12-26 18:20:37,877][105692] Updated weights for policy 0, policy_version 409371 (0.0010) [2023-12-26 18:20:37,892][105620] Updated weights for policy 1, policy_version 409653 (0.0007) [2023-12-26 18:20:37,931][105692] Updated weights for policy 0, policy_version 409381 (0.0009) [2023-12-26 18:20:37,949][105620] Updated weights for policy 1, policy_version 409663 (0.0008) [2023-12-26 18:20:38,009][105620] Updated weights for policy 1, policy_version 409673 (0.0008) [2023-12-26 18:20:38,695][105692] Updated weights for policy 0, policy_version 409391 (0.0009) [2023-12-26 18:20:38,718][105620] Updated weights for policy 1, policy_version 409683 (0.0008) [2023-12-26 18:20:38,751][105692] Updated weights for policy 0, policy_version 409401 (0.0011) [2023-12-26 18:20:38,765][105620] Updated weights for policy 1, policy_version 409693 (0.0005) [2023-12-26 18:20:38,810][105692] Updated weights for policy 0, policy_version 409411 (0.0011) [2023-12-26 18:20:38,823][105620] Updated weights for policy 1, policy_version 409703 (0.0005) [2023-12-26 18:20:39,549][105692] Updated weights for policy 0, policy_version 409421 (0.0011) [2023-12-26 18:20:39,603][105692] Updated weights for policy 0, policy_version 409431 (0.0006) [2023-12-26 18:20:39,608][105620] Updated weights for policy 1, policy_version 409713 (0.0008) [2023-12-26 18:20:39,660][105692] Updated weights for policy 0, policy_version 409441 (0.0008) [2023-12-26 18:20:39,670][105620] Updated weights for policy 1, policy_version 409723 (0.0007) [2023-12-26 18:20:39,732][105620] Updated weights for policy 1, policy_version 409733 (0.0007) [2023-12-26 18:20:39,790][105620] Updated weights for policy 1, policy_version 409743 (0.0009) [2023-12-26 18:20:40,352][105692] Updated weights for policy 0, policy_version 409451 (0.0009) [2023-12-26 18:20:40,413][105692] Updated weights for policy 0, policy_version 409461 (0.0007) [2023-12-26 18:20:40,473][105692] Updated weights for policy 0, policy_version 409471 (0.0010) [2023-12-26 18:20:40,576][105620] Updated weights for policy 1, policy_version 409753 (0.0008) [2023-12-26 18:20:40,627][105620] Updated weights for policy 1, policy_version 409763 (0.0008) [2023-12-26 18:20:40,682][105620] Updated weights for policy 1, policy_version 409773 (0.0008) [2023-12-26 18:20:41,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 209756160. Throughput: 0: 9557.8, 1: 9727.3. Samples: 209765744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:20:41,062][104569] Avg episode reward: [(0, '9082.254'), (1, '7872.855')] [2023-12-26 18:20:41,193][105692] Updated weights for policy 0, policy_version 409481 (0.0011) [2023-12-26 18:20:41,259][105692] Updated weights for policy 0, policy_version 409491 (0.0010) [2023-12-26 18:20:41,320][105692] Updated weights for policy 0, policy_version 409501 (0.0011) [2023-12-26 18:20:41,374][105585] KL-divergence is very high: 162.1944 [2023-12-26 18:20:41,389][105692] Updated weights for policy 0, policy_version 409511 (0.0011) [2023-12-26 18:20:41,455][105620] Updated weights for policy 1, policy_version 409783 (0.0008) [2023-12-26 18:20:41,519][105620] Updated weights for policy 1, policy_version 409793 (0.0008) [2023-12-26 18:20:41,586][105620] Updated weights for policy 1, policy_version 409803 (0.0008) [2023-12-26 18:20:42,133][105692] Updated weights for policy 0, policy_version 409521 (0.0008) [2023-12-26 18:20:42,186][105692] Updated weights for policy 0, policy_version 409531 (0.0008) [2023-12-26 18:20:42,243][105692] Updated weights for policy 0, policy_version 409541 (0.0008) [2023-12-26 18:20:42,304][105620] Updated weights for policy 1, policy_version 409813 (0.0008) [2023-12-26 18:20:42,357][105620] Updated weights for policy 1, policy_version 409823 (0.0009) [2023-12-26 18:20:42,421][105620] Updated weights for policy 1, policy_version 409833 (0.0007) [2023-12-26 18:20:42,929][105692] Updated weights for policy 0, policy_version 409551 (0.0010) [2023-12-26 18:20:42,984][105692] Updated weights for policy 0, policy_version 409561 (0.0010) [2023-12-26 18:20:43,044][105692] Updated weights for policy 0, policy_version 409571 (0.0011) [2023-12-26 18:20:43,179][105620] Updated weights for policy 1, policy_version 409843 (0.0006) [2023-12-26 18:20:43,242][105620] Updated weights for policy 1, policy_version 409853 (0.0005) [2023-12-26 18:20:43,297][105620] Updated weights for policy 1, policy_version 409863 (0.0008) [2023-12-26 18:20:43,730][105692] Updated weights for policy 0, policy_version 409581 (0.0008) [2023-12-26 18:20:43,788][105692] Updated weights for policy 0, policy_version 409591 (0.0005) [2023-12-26 18:20:43,842][105692] Updated weights for policy 0, policy_version 409601 (0.0006) [2023-12-26 18:20:44,036][105620] Updated weights for policy 1, policy_version 409873 (0.0008) [2023-12-26 18:20:44,083][105620] Updated weights for policy 1, policy_version 409883 (0.0009) [2023-12-26 18:20:44,138][105620] Updated weights for policy 1, policy_version 409893 (0.0009) [2023-12-26 18:20:44,188][105620] Updated weights for policy 1, policy_version 409903 (0.0009) [2023-12-26 18:20:44,455][105692] Updated weights for policy 0, policy_version 409611 (0.0007) [2023-12-26 18:20:44,520][105692] Updated weights for policy 0, policy_version 409621 (0.0009) [2023-12-26 18:20:44,578][105692] Updated weights for policy 0, policy_version 409631 (0.0009) [2023-12-26 18:20:45,016][105620] Updated weights for policy 1, policy_version 409913 (0.0010) [2023-12-26 18:20:45,064][105620] Updated weights for policy 1, policy_version 409923 (0.0009) [2023-12-26 18:20:45,111][105620] Updated weights for policy 1, policy_version 409933 (0.0009) [2023-12-26 18:20:45,291][105692] Updated weights for policy 0, policy_version 409641 (0.0009) [2023-12-26 18:20:45,347][105692] Updated weights for policy 0, policy_version 409651 (0.0009) [2023-12-26 18:20:45,403][105692] Updated weights for policy 0, policy_version 409661 (0.0009) [2023-12-26 18:20:45,471][105692] Updated weights for policy 0, policy_version 409671 (0.0007) [2023-12-26 18:20:45,875][105620] Updated weights for policy 1, policy_version 409943 (0.0007) [2023-12-26 18:20:45,930][105620] Updated weights for policy 1, policy_version 409953 (0.0006) [2023-12-26 18:20:45,988][105620] Updated weights for policy 1, policy_version 409963 (0.0008) [2023-12-26 18:20:46,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19251.2, 300 sec: 19633.0). Total num frames: 209854464. Throughput: 0: 9444.9, 1: 9710.8. Samples: 209822696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:20:46,063][104569] Avg episode reward: [(0, '9175.309'), (1, '8058.595')] [2023-12-26 18:20:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000409672_104890368.pth... [2023-12-26 18:20:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000409968_104964096.pth... [2023-12-26 18:20:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000408584_104611840.pth [2023-12-26 18:20:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000408816_104669184.pth [2023-12-26 18:20:46,228][105692] Updated weights for policy 0, policy_version 409681 (0.0006) [2023-12-26 18:20:46,277][105692] Updated weights for policy 0, policy_version 409691 (0.0005) [2023-12-26 18:20:46,331][105692] Updated weights for policy 0, policy_version 409701 (0.0007) [2023-12-26 18:20:46,671][105620] Updated weights for policy 1, policy_version 409973 (0.0008) [2023-12-26 18:20:46,731][105620] Updated weights for policy 1, policy_version 409983 (0.0007) [2023-12-26 18:20:46,783][105620] Updated weights for policy 1, policy_version 409993 (0.0005) [2023-12-26 18:20:46,928][105692] Updated weights for policy 0, policy_version 409711 (0.0006) [2023-12-26 18:20:46,982][105692] Updated weights for policy 0, policy_version 409721 (0.0006) [2023-12-26 18:20:47,044][105692] Updated weights for policy 0, policy_version 409731 (0.0006) [2023-12-26 18:20:47,374][105620] Updated weights for policy 1, policy_version 410003 (0.0006) [2023-12-26 18:20:47,435][105620] Updated weights for policy 1, policy_version 410013 (0.0005) [2023-12-26 18:20:47,490][105620] Updated weights for policy 1, policy_version 410023 (0.0008) [2023-12-26 18:20:47,639][105692] Updated weights for policy 0, policy_version 409741 (0.0007) [2023-12-26 18:20:47,699][105692] Updated weights for policy 0, policy_version 409751 (0.0005) [2023-12-26 18:20:47,752][105692] Updated weights for policy 0, policy_version 409761 (0.0009) [2023-12-26 18:20:48,191][105620] Updated weights for policy 1, policy_version 410033 (0.0010) [2023-12-26 18:20:48,248][105620] Updated weights for policy 1, policy_version 410043 (0.0010) [2023-12-26 18:20:48,303][105620] Updated weights for policy 1, policy_version 410053 (0.0010) [2023-12-26 18:20:48,357][105620] Updated weights for policy 1, policy_version 410063 (0.0010) [2023-12-26 18:20:48,456][105692] Updated weights for policy 0, policy_version 409771 (0.0009) [2023-12-26 18:20:48,515][105692] Updated weights for policy 0, policy_version 409781 (0.0010) [2023-12-26 18:20:48,574][105692] Updated weights for policy 0, policy_version 409791 (0.0009) [2023-12-26 18:20:49,000][105620] Updated weights for policy 1, policy_version 410073 (0.0008) [2023-12-26 18:20:49,066][105620] Updated weights for policy 1, policy_version 410083 (0.0011) [2023-12-26 18:20:49,125][105620] Updated weights for policy 1, policy_version 410093 (0.0011) [2023-12-26 18:20:49,394][105692] Updated weights for policy 0, policy_version 409801 (0.0008) [2023-12-26 18:20:49,446][105692] Updated weights for policy 0, policy_version 409811 (0.0008) [2023-12-26 18:20:49,504][105692] Updated weights for policy 0, policy_version 409821 (0.0008) [2023-12-26 18:20:49,566][105692] Updated weights for policy 0, policy_version 409831 (0.0007) [2023-12-26 18:20:49,844][105620] Updated weights for policy 1, policy_version 410103 (0.0011) [2023-12-26 18:20:49,907][105620] Updated weights for policy 1, policy_version 410113 (0.0011) [2023-12-26 18:20:49,975][105620] Updated weights for policy 1, policy_version 410123 (0.0011) [2023-12-26 18:20:50,311][105692] Updated weights for policy 0, policy_version 409841 (0.0008) [2023-12-26 18:20:50,374][105692] Updated weights for policy 0, policy_version 409851 (0.0007) [2023-12-26 18:20:50,433][105692] Updated weights for policy 0, policy_version 409861 (0.0005) [2023-12-26 18:20:50,731][105620] Updated weights for policy 1, policy_version 410133 (0.0008) [2023-12-26 18:20:50,796][105620] Updated weights for policy 1, policy_version 410143 (0.0011) [2023-12-26 18:20:50,856][105620] Updated weights for policy 1, policy_version 410153 (0.0005) [2023-12-26 18:20:51,046][105692] Updated weights for policy 0, policy_version 409871 (0.0006) [2023-12-26 18:20:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 209952768. Throughput: 0: 9484.4, 1: 9689.7. Samples: 209942488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:20:51,062][104569] Avg episode reward: [(0, '9266.848'), (1, '8927.484')] [2023-12-26 18:20:51,102][105692] Updated weights for policy 0, policy_version 409881 (0.0005) [2023-12-26 18:20:51,166][105692] Updated weights for policy 0, policy_version 409891 (0.0008) [2023-12-26 18:20:51,519][105620] Updated weights for policy 1, policy_version 410163 (0.0008) [2023-12-26 18:20:51,585][105620] Updated weights for policy 1, policy_version 410173 (0.0006) [2023-12-26 18:20:51,663][105620] Updated weights for policy 1, policy_version 410183 (0.0009) [2023-12-26 18:20:51,842][105692] Updated weights for policy 0, policy_version 409901 (0.0007) [2023-12-26 18:20:51,897][105692] Updated weights for policy 0, policy_version 409911 (0.0009) [2023-12-26 18:20:51,958][105692] Updated weights for policy 0, policy_version 409921 (0.0008) [2023-12-26 18:20:52,365][105620] Updated weights for policy 1, policy_version 410193 (0.0009) [2023-12-26 18:20:52,431][105620] Updated weights for policy 1, policy_version 410203 (0.0009) [2023-12-26 18:20:52,491][105620] Updated weights for policy 1, policy_version 410213 (0.0009) [2023-12-26 18:20:52,552][105620] Updated weights for policy 1, policy_version 410223 (0.0008) [2023-12-26 18:20:52,716][105692] Updated weights for policy 0, policy_version 409931 (0.0009) [2023-12-26 18:20:52,777][105692] Updated weights for policy 0, policy_version 409941 (0.0009) [2023-12-26 18:20:52,826][105692] Updated weights for policy 0, policy_version 409951 (0.0009) [2023-12-26 18:20:53,373][105620] Updated weights for policy 1, policy_version 410233 (0.0009) [2023-12-26 18:20:53,435][105620] Updated weights for policy 1, policy_version 410243 (0.0009) [2023-12-26 18:20:53,440][105586] KL-divergence is very high: 126.0283 [2023-12-26 18:20:53,486][105586] KL-divergence is very high: 198.7877 [2023-12-26 18:20:53,491][105620] Updated weights for policy 1, policy_version 410253 (0.0008) [2023-12-26 18:20:53,604][105692] Updated weights for policy 0, policy_version 409961 (0.0009) [2023-12-26 18:20:53,654][105692] Updated weights for policy 0, policy_version 409971 (0.0009) [2023-12-26 18:20:53,701][105692] Updated weights for policy 0, policy_version 409981 (0.0009) [2023-12-26 18:20:53,755][105692] Updated weights for policy 0, policy_version 409991 (0.0008) [2023-12-26 18:20:54,153][105620] Updated weights for policy 1, policy_version 410263 (0.0006) [2023-12-26 18:20:54,208][105620] Updated weights for policy 1, policy_version 410273 (0.0006) [2023-12-26 18:20:54,271][105620] Updated weights for policy 1, policy_version 410283 (0.0008) [2023-12-26 18:20:54,601][105692] Updated weights for policy 0, policy_version 410001 (0.0008) [2023-12-26 18:20:54,661][105692] Updated weights for policy 0, policy_version 410011 (0.0008) [2023-12-26 18:20:54,723][105692] Updated weights for policy 0, policy_version 410021 (0.0008) [2023-12-26 18:20:54,930][105620] Updated weights for policy 1, policy_version 410293 (0.0009) [2023-12-26 18:20:54,986][105620] Updated weights for policy 1, policy_version 410303 (0.0010) [2023-12-26 18:20:55,041][105620] Updated weights for policy 1, policy_version 410313 (0.0010) [2023-12-26 18:20:55,538][105692] Updated weights for policy 0, policy_version 410031 (0.0009) [2023-12-26 18:20:55,591][105692] Updated weights for policy 0, policy_version 410042 (0.0010) [2023-12-26 18:20:55,613][105620] Updated weights for policy 1, policy_version 410323 (0.0011) [2023-12-26 18:20:55,642][105692] Updated weights for policy 0, policy_version 410052 (0.0006) [2023-12-26 18:20:55,671][105620] Updated weights for policy 1, policy_version 410333 (0.0010) [2023-12-26 18:20:55,725][105620] Updated weights for policy 1, policy_version 410343 (0.0010) [2023-12-26 18:20:56,062][104569] Fps is (10 sec: 19661.6, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 210051072. Throughput: 0: 9498.3, 1: 9713.8. Samples: 210058772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:20:56,062][104569] Avg episode reward: [(0, '9357.032'), (1, '9174.419')] [2023-12-26 18:20:56,400][105692] Updated weights for policy 0, policy_version 410062 (0.0007) [2023-12-26 18:20:56,450][105692] Updated weights for policy 0, policy_version 410072 (0.0007) [2023-12-26 18:20:56,466][105620] Updated weights for policy 1, policy_version 410353 (0.0010) [2023-12-26 18:20:56,494][105692] Updated weights for policy 0, policy_version 410082 (0.0010) [2023-12-26 18:20:56,518][105620] Updated weights for policy 1, policy_version 410363 (0.0010) [2023-12-26 18:20:56,558][105620] Updated weights for policy 1, policy_version 410373 (0.0010) [2023-12-26 18:20:56,616][105620] Updated weights for policy 1, policy_version 410383 (0.0010) [2023-12-26 18:20:57,094][105692] Updated weights for policy 0, policy_version 410092 (0.0007) [2023-12-26 18:20:57,153][105692] Updated weights for policy 0, policy_version 410102 (0.0011) [2023-12-26 18:20:57,215][105692] Updated weights for policy 0, policy_version 410112 (0.0011) [2023-12-26 18:20:57,281][105620] Updated weights for policy 1, policy_version 410393 (0.0006) [2023-12-26 18:20:57,338][105620] Updated weights for policy 1, policy_version 410403 (0.0010) [2023-12-26 18:20:57,389][105620] Updated weights for policy 1, policy_version 410413 (0.0010) [2023-12-26 18:20:57,941][105692] Updated weights for policy 0, policy_version 410122 (0.0011) [2023-12-26 18:20:57,954][105620] Updated weights for policy 1, policy_version 410423 (0.0010) [2023-12-26 18:20:57,956][105586] KL-divergence is very high: 376.0091 [2023-12-26 18:20:57,993][105692] Updated weights for policy 0, policy_version 410132 (0.0010) [2023-12-26 18:20:58,006][105586] KL-divergence is very high: 704.3811 [2023-12-26 18:20:58,012][105620] Updated weights for policy 1, policy_version 410433 (0.0011) [2023-12-26 18:20:58,045][105692] Updated weights for policy 0, policy_version 410142 (0.0010) [2023-12-26 18:20:58,049][105586] KL-divergence is very high: 733.3820 [2023-12-26 18:20:58,064][105620] Updated weights for policy 1, policy_version 410443 (0.0010) [2023-12-26 18:20:58,093][105692] Updated weights for policy 0, policy_version 410152 (0.0010) [2023-12-26 18:20:58,843][105620] Updated weights for policy 1, policy_version 410453 (0.0010) [2023-12-26 18:20:58,903][105620] Updated weights for policy 1, policy_version 410463 (0.0009) [2023-12-26 18:20:58,914][105692] Updated weights for policy 0, policy_version 410162 (0.0009) [2023-12-26 18:20:58,965][105620] Updated weights for policy 1, policy_version 410473 (0.0009) [2023-12-26 18:20:58,976][105692] Updated weights for policy 0, policy_version 410172 (0.0009) [2023-12-26 18:20:59,029][105692] Updated weights for policy 0, policy_version 410182 (0.0009) [2023-12-26 18:20:59,757][105620] Updated weights for policy 1, policy_version 410483 (0.0009) [2023-12-26 18:20:59,812][105620] Updated weights for policy 1, policy_version 410493 (0.0008) [2023-12-26 18:20:59,876][105620] Updated weights for policy 1, policy_version 410503 (0.0010) [2023-12-26 18:20:59,929][105692] Updated weights for policy 0, policy_version 410192 (0.0008) [2023-12-26 18:20:59,989][105692] Updated weights for policy 0, policy_version 410202 (0.0006) [2023-12-26 18:21:00,049][105692] Updated weights for policy 0, policy_version 410212 (0.0007) [2023-12-26 18:21:00,554][105620] Updated weights for policy 1, policy_version 410513 (0.0010) [2023-12-26 18:21:00,619][105620] Updated weights for policy 1, policy_version 410523 (0.0010) [2023-12-26 18:21:00,687][105620] Updated weights for policy 1, policy_version 410533 (0.0010) [2023-12-26 18:21:00,740][105692] Updated weights for policy 0, policy_version 410222 (0.0007) [2023-12-26 18:21:00,745][105620] Updated weights for policy 1, policy_version 410543 (0.0010) [2023-12-26 18:21:00,786][105692] Updated weights for policy 0, policy_version 410232 (0.0005) [2023-12-26 18:21:00,831][105692] Updated weights for policy 0, policy_version 410242 (0.0007) [2023-12-26 18:21:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 210149376. Throughput: 0: 9532.3, 1: 9751.2. Samples: 210118680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:21:01,062][104569] Avg episode reward: [(0, '7981.859'), (1, '9082.044')] [2023-12-26 18:21:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000410544_105111552.pth... [2023-12-26 18:21:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000410248_105037824.pth... [2023-12-26 18:21:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000409392_104816640.pth [2023-12-26 18:21:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000409128_104751104.pth [2023-12-26 18:21:01,413][105620] Updated weights for policy 1, policy_version 410553 (0.0010) [2023-12-26 18:21:01,468][105620] Updated weights for policy 1, policy_version 410563 (0.0010) [2023-12-26 18:21:01,472][105692] Updated weights for policy 0, policy_version 410252 (0.0007) [2023-12-26 18:21:01,520][105620] Updated weights for policy 1, policy_version 410573 (0.0010) [2023-12-26 18:21:01,534][105692] Updated weights for policy 0, policy_version 410262 (0.0011) [2023-12-26 18:21:01,591][105692] Updated weights for policy 0, policy_version 410272 (0.0010) [2023-12-26 18:21:02,211][105620] Updated weights for policy 1, policy_version 410583 (0.0010) [2023-12-26 18:21:02,275][105620] Updated weights for policy 1, policy_version 410593 (0.0010) [2023-12-26 18:21:02,332][105620] Updated weights for policy 1, policy_version 410603 (0.0011) [2023-12-26 18:21:02,350][105692] Updated weights for policy 0, policy_version 410282 (0.0010) [2023-12-26 18:21:02,412][105692] Updated weights for policy 0, policy_version 410292 (0.0007) [2023-12-26 18:21:02,473][105692] Updated weights for policy 0, policy_version 410302 (0.0009) [2023-12-26 18:21:02,914][105620] Updated weights for policy 1, policy_version 410613 (0.0008) [2023-12-26 18:21:02,963][105620] Updated weights for policy 1, policy_version 410623 (0.0005) [2023-12-26 18:21:03,015][105620] Updated weights for policy 1, policy_version 410633 (0.0007) [2023-12-26 18:21:03,156][105692] Updated weights for policy 0, policy_version 410313 (0.0007) [2023-12-26 18:21:03,213][105692] Updated weights for policy 0, policy_version 410323 (0.0011) [2023-12-26 18:21:03,269][105692] Updated weights for policy 0, policy_version 410333 (0.0010) [2023-12-26 18:21:03,318][105692] Updated weights for policy 0, policy_version 410343 (0.0009) [2023-12-26 18:21:03,808][105620] Updated weights for policy 1, policy_version 410643 (0.0009) [2023-12-26 18:21:03,875][105620] Updated weights for policy 1, policy_version 410653 (0.0008) [2023-12-26 18:21:03,923][105620] Updated weights for policy 1, policy_version 410663 (0.0008) [2023-12-26 18:21:03,958][105692] Updated weights for policy 0, policy_version 410353 (0.0008) [2023-12-26 18:21:04,014][105692] Updated weights for policy 0, policy_version 410363 (0.0008) [2023-12-26 18:21:04,073][105692] Updated weights for policy 0, policy_version 410373 (0.0008) [2023-12-26 18:21:04,637][105620] Updated weights for policy 1, policy_version 410673 (0.0008) [2023-12-26 18:21:04,697][105620] Updated weights for policy 1, policy_version 410683 (0.0009) [2023-12-26 18:21:04,755][105620] Updated weights for policy 1, policy_version 410693 (0.0009) [2023-12-26 18:21:04,790][105692] Updated weights for policy 0, policy_version 410383 (0.0007) [2023-12-26 18:21:04,811][105620] Updated weights for policy 1, policy_version 410703 (0.0008) [2023-12-26 18:21:04,847][105692] Updated weights for policy 0, policy_version 410393 (0.0005) [2023-12-26 18:21:04,913][105692] Updated weights for policy 0, policy_version 410403 (0.0005) [2023-12-26 18:21:05,506][105620] Updated weights for policy 1, policy_version 410713 (0.0009) [2023-12-26 18:21:05,563][105620] Updated weights for policy 1, policy_version 410723 (0.0008) [2023-12-26 18:21:05,601][105692] Updated weights for policy 0, policy_version 410413 (0.0005) [2023-12-26 18:21:05,615][105620] Updated weights for policy 1, policy_version 410733 (0.0007) [2023-12-26 18:21:05,672][105692] Updated weights for policy 0, policy_version 410423 (0.0006) [2023-12-26 18:21:05,737][105692] Updated weights for policy 0, policy_version 410433 (0.0008) [2023-12-26 18:21:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 210247680. Throughput: 0: 9490.8, 1: 9790.6. Samples: 210235264. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:21:06,063][104569] Avg episode reward: [(0, '802.378'), (1, '9356.786')] [2023-12-26 18:21:06,248][105620] Updated weights for policy 1, policy_version 410743 (0.0006) [2023-12-26 18:21:06,312][105620] Updated weights for policy 1, policy_version 410753 (0.0006) [2023-12-26 18:21:06,371][105620] Updated weights for policy 1, policy_version 410763 (0.0006) [2023-12-26 18:21:06,448][105692] Updated weights for policy 0, policy_version 410443 (0.0010) [2023-12-26 18:21:06,496][105692] Updated weights for policy 0, policy_version 410453 (0.0008) [2023-12-26 18:21:06,545][105692] Updated weights for policy 0, policy_version 410463 (0.0008) [2023-12-26 18:21:07,028][105620] Updated weights for policy 1, policy_version 410773 (0.0006) [2023-12-26 18:21:07,097][105620] Updated weights for policy 1, policy_version 410783 (0.0008) [2023-12-26 18:21:07,159][105620] Updated weights for policy 1, policy_version 410793 (0.0009) [2023-12-26 18:21:07,363][105692] Updated weights for policy 0, policy_version 410473 (0.0007) [2023-12-26 18:21:07,418][105692] Updated weights for policy 0, policy_version 410483 (0.0009) [2023-12-26 18:21:07,478][105692] Updated weights for policy 0, policy_version 410493 (0.0009) [2023-12-26 18:21:07,480][105585] KL-divergence is very high: 122.1004 [2023-12-26 18:21:07,540][105692] Updated weights for policy 0, policy_version 410503 (0.0009) [2023-12-26 18:21:07,812][105620] Updated weights for policy 1, policy_version 410803 (0.0008) [2023-12-26 18:21:07,871][105620] Updated weights for policy 1, policy_version 410813 (0.0006) [2023-12-26 18:21:07,920][105620] Updated weights for policy 1, policy_version 410823 (0.0010) [2023-12-26 18:21:08,371][105692] Updated weights for policy 0, policy_version 410513 (0.0009) [2023-12-26 18:21:08,428][105692] Updated weights for policy 0, policy_version 410523 (0.0009) [2023-12-26 18:21:08,491][105692] Updated weights for policy 0, policy_version 410533 (0.0010) [2023-12-26 18:21:08,546][105620] Updated weights for policy 1, policy_version 410833 (0.0010) [2023-12-26 18:21:08,601][105620] Updated weights for policy 1, policy_version 410843 (0.0005) [2023-12-26 18:21:08,651][105620] Updated weights for policy 1, policy_version 410853 (0.0005) [2023-12-26 18:21:08,702][105620] Updated weights for policy 1, policy_version 410863 (0.0006) [2023-12-26 18:21:09,208][105692] Updated weights for policy 0, policy_version 410543 (0.0008) [2023-12-26 18:21:09,269][105692] Updated weights for policy 0, policy_version 410553 (0.0007) [2023-12-26 18:21:09,328][105692] Updated weights for policy 0, policy_version 410563 (0.0008) [2023-12-26 18:21:09,465][105620] Updated weights for policy 1, policy_version 410873 (0.0008) [2023-12-26 18:21:09,532][105620] Updated weights for policy 1, policy_version 410883 (0.0010) [2023-12-26 18:21:09,595][105620] Updated weights for policy 1, policy_version 410893 (0.0010) [2023-12-26 18:21:10,107][105692] Updated weights for policy 0, policy_version 410573 (0.0008) [2023-12-26 18:21:10,167][105692] Updated weights for policy 0, policy_version 410583 (0.0008) [2023-12-26 18:21:10,230][105692] Updated weights for policy 0, policy_version 410593 (0.0008) [2023-12-26 18:21:10,348][105620] Updated weights for policy 1, policy_version 410903 (0.0011) [2023-12-26 18:21:10,407][105620] Updated weights for policy 1, policy_version 410913 (0.0011) [2023-12-26 18:21:10,473][105620] Updated weights for policy 1, policy_version 410923 (0.0010) [2023-12-26 18:21:11,001][105692] Updated weights for policy 0, policy_version 410603 (0.0008) [2023-12-26 18:21:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 210337792. Throughput: 0: 9472.5, 1: 9892.4. Samples: 210351808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:21:11,063][104569] Avg episode reward: [(0, '1215.894'), (1, '9356.664')] [2023-12-26 18:21:11,071][105692] Updated weights for policy 0, policy_version 410613 (0.0008) [2023-12-26 18:21:11,125][105692] Updated weights for policy 0, policy_version 410623 (0.0008) [2023-12-26 18:21:11,215][105620] Updated weights for policy 1, policy_version 410933 (0.0010) [2023-12-26 18:21:11,277][105620] Updated weights for policy 1, policy_version 410943 (0.0011) [2023-12-26 18:21:11,337][105620] Updated weights for policy 1, policy_version 410953 (0.0010) [2023-12-26 18:21:11,942][105692] Updated weights for policy 0, policy_version 410633 (0.0008) [2023-12-26 18:21:11,996][105692] Updated weights for policy 0, policy_version 410643 (0.0008) [2023-12-26 18:21:12,056][105692] Updated weights for policy 0, policy_version 410653 (0.0008) [2023-12-26 18:21:12,094][105620] Updated weights for policy 1, policy_version 410963 (0.0010) [2023-12-26 18:21:12,124][105692] Updated weights for policy 0, policy_version 410663 (0.0009) [2023-12-26 18:21:12,161][105620] Updated weights for policy 1, policy_version 410973 (0.0007) [2023-12-26 18:21:12,226][105620] Updated weights for policy 1, policy_version 410983 (0.0006) [2023-12-26 18:21:12,840][105692] Updated weights for policy 0, policy_version 410673 (0.0010) [2023-12-26 18:21:12,912][105620] Updated weights for policy 1, policy_version 410993 (0.0007) [2023-12-26 18:21:12,917][105692] Updated weights for policy 0, policy_version 410683 (0.0009) [2023-12-26 18:21:12,959][105620] Updated weights for policy 1, policy_version 411003 (0.0006) [2023-12-26 18:21:12,973][105692] Updated weights for policy 0, policy_version 410693 (0.0008) [2023-12-26 18:21:13,006][105620] Updated weights for policy 1, policy_version 411013 (0.0008) [2023-12-26 18:21:13,056][105620] Updated weights for policy 1, policy_version 411023 (0.0009) [2023-12-26 18:21:13,691][105692] Updated weights for policy 0, policy_version 410703 (0.0008) [2023-12-26 18:21:13,746][105692] Updated weights for policy 0, policy_version 410713 (0.0009) [2023-12-26 18:21:13,787][105620] Updated weights for policy 1, policy_version 411033 (0.0009) [2023-12-26 18:21:13,807][105692] Updated weights for policy 0, policy_version 410723 (0.0006) [2023-12-26 18:21:13,838][105620] Updated weights for policy 1, policy_version 411043 (0.0007) [2023-12-26 18:21:13,896][105620] Updated weights for policy 1, policy_version 411053 (0.0009) [2023-12-26 18:21:14,585][105692] Updated weights for policy 0, policy_version 410733 (0.0007) [2023-12-26 18:21:14,603][105620] Updated weights for policy 1, policy_version 411063 (0.0007) [2023-12-26 18:21:14,633][105692] Updated weights for policy 0, policy_version 410743 (0.0006) [2023-12-26 18:21:14,664][105620] Updated weights for policy 1, policy_version 411073 (0.0009) [2023-12-26 18:21:14,682][105692] Updated weights for policy 0, policy_version 410753 (0.0007) [2023-12-26 18:21:14,722][105620] Updated weights for policy 1, policy_version 411083 (0.0007) [2023-12-26 18:21:15,421][105620] Updated weights for policy 1, policy_version 411093 (0.0009) [2023-12-26 18:21:15,477][105620] Updated weights for policy 1, policy_version 411103 (0.0008) [2023-12-26 18:21:15,479][105692] Updated weights for policy 0, policy_version 410763 (0.0008) [2023-12-26 18:21:15,526][105692] Updated weights for policy 0, policy_version 410773 (0.0009) [2023-12-26 18:21:15,532][105620] Updated weights for policy 1, policy_version 411113 (0.0008) [2023-12-26 18:21:15,590][105692] Updated weights for policy 0, policy_version 410783 (0.0008) [2023-12-26 18:21:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 210436096. Throughput: 0: 9471.4, 1: 9839.2. Samples: 210407852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:21:16,062][104569] Avg episode reward: [(0, '4213.298'), (1, '9267.581')] [2023-12-26 18:21:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000410792_105177088.pth... [2023-12-26 18:21:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000411120_105259008.pth... [2023-12-26 18:21:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000409968_104964096.pth [2023-12-26 18:21:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000409672_104890368.pth [2023-12-26 18:21:16,231][105620] Updated weights for policy 1, policy_version 411123 (0.0007) [2023-12-26 18:21:16,290][105620] Updated weights for policy 1, policy_version 411133 (0.0009) [2023-12-26 18:21:16,349][105620] Updated weights for policy 1, policy_version 411143 (0.0009) [2023-12-26 18:21:16,377][105692] Updated weights for policy 0, policy_version 410793 (0.0009) [2023-12-26 18:21:16,391][105585] KL-divergence is very high: 120.6785 [2023-12-26 18:21:16,398][105585] KL-divergence is very high: 122.8831 [2023-12-26 18:21:16,415][105585] KL-divergence is very high: 372.8588 [2023-12-26 18:21:16,422][105585] KL-divergence is very high: 366.8145 [2023-12-26 18:21:16,433][105585] KL-divergence is very high: 303.3208 [2023-12-26 18:21:16,438][105692] Updated weights for policy 0, policy_version 410803 (0.0008) [2023-12-26 18:21:16,438][105585] KL-divergence is very high: 431.2560 [2023-12-26 18:21:16,443][105585] KL-divergence is very high: 321.9072 [2023-12-26 18:21:16,457][105585] KL-divergence is very high: 427.3064 [2023-12-26 18:21:16,462][105585] KL-divergence is very high: 352.2312 [2023-12-26 18:21:16,474][105585] KL-divergence is very high: 204.5489 [2023-12-26 18:21:16,480][105585] KL-divergence is very high: 278.4286 [2023-12-26 18:21:16,486][105585] KL-divergence is very high: 178.1527 [2023-12-26 18:21:16,492][105692] Updated weights for policy 0, policy_version 410813 (0.0009) [2023-12-26 18:21:16,503][105585] KL-divergence is very high: 196.2217 [2023-12-26 18:21:16,510][105585] KL-divergence is very high: 133.0430 [2023-12-26 18:21:16,546][105692] Updated weights for policy 0, policy_version 410823 (0.0009) [2023-12-26 18:21:16,995][105620] Updated weights for policy 1, policy_version 411153 (0.0007) [2023-12-26 18:21:17,056][105620] Updated weights for policy 1, policy_version 411163 (0.0009) [2023-12-26 18:21:17,116][105620] Updated weights for policy 1, policy_version 411173 (0.0009) [2023-12-26 18:21:17,177][105620] Updated weights for policy 1, policy_version 411183 (0.0009) [2023-12-26 18:21:17,309][105692] Updated weights for policy 0, policy_version 410833 (0.0009) [2023-12-26 18:21:17,358][105692] Updated weights for policy 0, policy_version 410843 (0.0008) [2023-12-26 18:21:17,415][105692] Updated weights for policy 0, policy_version 410853 (0.0009) [2023-12-26 18:21:17,929][105620] Updated weights for policy 1, policy_version 411193 (0.0009) [2023-12-26 18:21:17,992][105620] Updated weights for policy 1, policy_version 411203 (0.0008) [2023-12-26 18:21:18,057][105620] Updated weights for policy 1, policy_version 411213 (0.0009) [2023-12-26 18:21:18,168][105692] Updated weights for policy 0, policy_version 410863 (0.0009) [2023-12-26 18:21:18,218][105692] Updated weights for policy 0, policy_version 410873 (0.0008) [2023-12-26 18:21:18,265][105692] Updated weights for policy 0, policy_version 410883 (0.0009) [2023-12-26 18:21:18,794][105620] Updated weights for policy 1, policy_version 411223 (0.0009) [2023-12-26 18:21:18,844][105620] Updated weights for policy 1, policy_version 411233 (0.0008) [2023-12-26 18:21:18,899][105620] Updated weights for policy 1, policy_version 411243 (0.0009) [2023-12-26 18:21:19,061][105692] Updated weights for policy 0, policy_version 410893 (0.0009) [2023-12-26 18:21:19,123][105692] Updated weights for policy 0, policy_version 410903 (0.0009) [2023-12-26 18:21:19,181][105692] Updated weights for policy 0, policy_version 410913 (0.0009) [2023-12-26 18:21:19,646][105620] Updated weights for policy 1, policy_version 411253 (0.0009) [2023-12-26 18:21:19,701][105620] Updated weights for policy 1, policy_version 411263 (0.0009) [2023-12-26 18:21:19,752][105620] Updated weights for policy 1, policy_version 411273 (0.0008) [2023-12-26 18:21:20,053][105692] Updated weights for policy 0, policy_version 410923 (0.0009) [2023-12-26 18:21:20,112][105692] Updated weights for policy 0, policy_version 410933 (0.0009) [2023-12-26 18:21:20,177][105692] Updated weights for policy 0, policy_version 410943 (0.0009) [2023-12-26 18:21:20,501][105620] Updated weights for policy 1, policy_version 411283 (0.0008) [2023-12-26 18:21:20,569][105620] Updated weights for policy 1, policy_version 411293 (0.0010) [2023-12-26 18:21:20,639][105620] Updated weights for policy 1, policy_version 411303 (0.0009) [2023-12-26 18:21:20,984][105692] Updated weights for policy 0, policy_version 410953 (0.0010) [2023-12-26 18:21:21,054][105692] Updated weights for policy 0, policy_version 410963 (0.0009) [2023-12-26 18:21:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.7, 300 sec: 19549.7). Total num frames: 210526208. Throughput: 0: 9479.5, 1: 9823.5. Samples: 210520640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:21:21,063][104569] Avg episode reward: [(0, '741.101'), (1, '9177.208')] [2023-12-26 18:21:21,114][105692] Updated weights for policy 0, policy_version 410973 (0.0009) [2023-12-26 18:21:21,180][105692] Updated weights for policy 0, policy_version 410983 (0.0008) [2023-12-26 18:21:21,364][105620] Updated weights for policy 1, policy_version 411313 (0.0009) [2023-12-26 18:21:21,431][105620] Updated weights for policy 1, policy_version 411323 (0.0009) [2023-12-26 18:21:21,482][105620] Updated weights for policy 1, policy_version 411333 (0.0007) [2023-12-26 18:21:21,530][105620] Updated weights for policy 1, policy_version 411343 (0.0009) [2023-12-26 18:21:21,956][105692] Updated weights for policy 0, policy_version 410993 (0.0010) [2023-12-26 18:21:22,020][105692] Updated weights for policy 0, policy_version 411003 (0.0010) [2023-12-26 18:21:22,086][105692] Updated weights for policy 0, policy_version 411013 (0.0010) [2023-12-26 18:21:22,325][105620] Updated weights for policy 1, policy_version 411353 (0.0009) [2023-12-26 18:21:22,410][105620] Updated weights for policy 1, policy_version 411363 (0.0009) [2023-12-26 18:21:22,463][105620] Updated weights for policy 1, policy_version 411373 (0.0008) [2023-12-26 18:21:22,836][105692] Updated weights for policy 0, policy_version 411023 (0.0011) [2023-12-26 18:21:22,889][105692] Updated weights for policy 0, policy_version 411033 (0.0010) [2023-12-26 18:21:22,945][105692] Updated weights for policy 0, policy_version 411043 (0.0010) [2023-12-26 18:21:23,233][105620] Updated weights for policy 1, policy_version 411383 (0.0008) [2023-12-26 18:21:23,263][105586] KL-divergence is very high: 119.0123 [2023-12-26 18:21:23,300][105620] Updated weights for policy 1, policy_version 411393 (0.0009) [2023-12-26 18:21:23,315][105586] KL-divergence is very high: 188.7124 [2023-12-26 18:21:23,364][105620] Updated weights for policy 1, policy_version 411403 (0.0008) [2023-12-26 18:21:23,365][105586] KL-divergence is very high: 198.5836 [2023-12-26 18:21:23,710][105692] Updated weights for policy 0, policy_version 411053 (0.0010) [2023-12-26 18:21:23,771][105692] Updated weights for policy 0, policy_version 411063 (0.0010) [2023-12-26 18:21:23,825][105692] Updated weights for policy 0, policy_version 411073 (0.0010) [2023-12-26 18:21:23,985][105620] Updated weights for policy 1, policy_version 411413 (0.0007) [2023-12-26 18:21:24,039][105620] Updated weights for policy 1, policy_version 411423 (0.0005) [2023-12-26 18:21:24,094][105620] Updated weights for policy 1, policy_version 411433 (0.0005) [2023-12-26 18:21:24,441][105692] Updated weights for policy 0, policy_version 411083 (0.0010) [2023-12-26 18:21:24,501][105692] Updated weights for policy 0, policy_version 411093 (0.0011) [2023-12-26 18:21:24,554][105692] Updated weights for policy 0, policy_version 411103 (0.0009) [2023-12-26 18:21:24,754][105620] Updated weights for policy 1, policy_version 411443 (0.0007) [2023-12-26 18:21:24,815][105620] Updated weights for policy 1, policy_version 411453 (0.0010) [2023-12-26 18:21:24,869][105620] Updated weights for policy 1, policy_version 411463 (0.0010) [2023-12-26 18:21:25,083][105692] Updated weights for policy 0, policy_version 411113 (0.0005) [2023-12-26 18:21:25,134][105692] Updated weights for policy 0, policy_version 411123 (0.0005) [2023-12-26 18:21:25,198][105692] Updated weights for policy 0, policy_version 411133 (0.0009) [2023-12-26 18:21:25,255][105692] Updated weights for policy 0, policy_version 411143 (0.0009) [2023-12-26 18:21:25,495][105620] Updated weights for policy 1, policy_version 411474 (0.0009) [2023-12-26 18:21:25,546][105620] Updated weights for policy 1, policy_version 411484 (0.0010) [2023-12-26 18:21:25,593][105620] Updated weights for policy 1, policy_version 411494 (0.0007) [2023-12-26 18:21:25,641][105620] Updated weights for policy 1, policy_version 411504 (0.0008) [2023-12-26 18:21:25,945][105692] Updated weights for policy 0, policy_version 411153 (0.0007) [2023-12-26 18:21:25,996][105692] Updated weights for policy 0, policy_version 411163 (0.0005) [2023-12-26 18:21:26,049][105692] Updated weights for policy 0, policy_version 411173 (0.0005) [2023-12-26 18:21:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 210624512. Throughput: 0: 9517.7, 1: 9834.4. Samples: 210636588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:21:26,063][104569] Avg episode reward: [(0, '1639.367'), (1, '9087.144')] [2023-12-26 18:21:26,474][105620] Updated weights for policy 1, policy_version 411514 (0.0010) [2023-12-26 18:21:26,533][105620] Updated weights for policy 1, policy_version 411524 (0.0010) [2023-12-26 18:21:26,594][105620] Updated weights for policy 1, policy_version 411534 (0.0009) [2023-12-26 18:21:26,648][105692] Updated weights for policy 0, policy_version 411183 (0.0006) [2023-12-26 18:21:26,711][105692] Updated weights for policy 0, policy_version 411193 (0.0007) [2023-12-26 18:21:26,777][105692] Updated weights for policy 0, policy_version 411203 (0.0008) [2023-12-26 18:21:27,296][105692] Updated weights for policy 0, policy_version 411213 (0.0009) [2023-12-26 18:21:27,357][105692] Updated weights for policy 0, policy_version 411223 (0.0010) [2023-12-26 18:21:27,410][105692] Updated weights for policy 0, policy_version 411233 (0.0010) [2023-12-26 18:21:27,447][105620] Updated weights for policy 1, policy_version 411544 (0.0007) [2023-12-26 18:21:27,508][105620] Updated weights for policy 1, policy_version 411554 (0.0008) [2023-12-26 18:21:27,567][105620] Updated weights for policy 1, policy_version 411564 (0.0007) [2023-12-26 18:21:28,086][105692] Updated weights for policy 0, policy_version 411243 (0.0009) [2023-12-26 18:21:28,149][105692] Updated weights for policy 0, policy_version 411253 (0.0010) [2023-12-26 18:21:28,196][105692] Updated weights for policy 0, policy_version 411263 (0.0010) [2023-12-26 18:21:28,367][105620] Updated weights for policy 1, policy_version 411574 (0.0009) [2023-12-26 18:21:28,419][105620] Updated weights for policy 1, policy_version 411584 (0.0008) [2023-12-26 18:21:28,475][105620] Updated weights for policy 1, policy_version 411594 (0.0008) [2023-12-26 18:21:28,886][105692] Updated weights for policy 0, policy_version 411273 (0.0010) [2023-12-26 18:21:28,937][105692] Updated weights for policy 0, policy_version 411283 (0.0010) [2023-12-26 18:21:28,981][105692] Updated weights for policy 0, policy_version 411293 (0.0010) [2023-12-26 18:21:29,028][105692] Updated weights for policy 0, policy_version 411303 (0.0010) [2023-12-26 18:21:29,125][105620] Updated weights for policy 1, policy_version 411604 (0.0007) [2023-12-26 18:21:29,181][105620] Updated weights for policy 1, policy_version 411614 (0.0005) [2023-12-26 18:21:29,248][105620] Updated weights for policy 1, policy_version 411624 (0.0008) [2023-12-26 18:21:29,797][105692] Updated weights for policy 0, policy_version 411313 (0.0010) [2023-12-26 18:21:29,857][105692] Updated weights for policy 0, policy_version 411323 (0.0009) [2023-12-26 18:21:29,902][105620] Updated weights for policy 1, policy_version 411634 (0.0008) [2023-12-26 18:21:29,915][105692] Updated weights for policy 0, policy_version 411333 (0.0006) [2023-12-26 18:21:29,968][105620] Updated weights for policy 1, policy_version 411644 (0.0008) [2023-12-26 18:21:30,028][105620] Updated weights for policy 1, policy_version 411654 (0.0008) [2023-12-26 18:21:30,086][105620] Updated weights for policy 1, policy_version 411664 (0.0008) [2023-12-26 18:21:30,629][105692] Updated weights for policy 0, policy_version 411343 (0.0009) [2023-12-26 18:21:30,681][105692] Updated weights for policy 0, policy_version 411353 (0.0009) [2023-12-26 18:21:30,735][105620] Updated weights for policy 1, policy_version 411674 (0.0006) [2023-12-26 18:21:30,739][105692] Updated weights for policy 0, policy_version 411363 (0.0009) [2023-12-26 18:21:30,788][105620] Updated weights for policy 1, policy_version 411684 (0.0005) [2023-12-26 18:21:30,834][105620] Updated weights for policy 1, policy_version 411694 (0.0005) [2023-12-26 18:21:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 210731008. Throughput: 0: 9609.3, 1: 9800.7. Samples: 210696140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:21:31,063][104569] Avg episode reward: [(0, '6297.600'), (1, '1216.502')] [2023-12-26 18:21:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000411368_105324544.pth... [2023-12-26 18:21:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000411696_105406464.pth... [2023-12-26 18:21:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000410544_105111552.pth [2023-12-26 18:21:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000410248_105037824.pth [2023-12-26 18:21:31,434][105620] Updated weights for policy 1, policy_version 411704 (0.0008) [2023-12-26 18:21:31,499][105620] Updated weights for policy 1, policy_version 411714 (0.0009) [2023-12-26 18:21:31,563][105620] Updated weights for policy 1, policy_version 411724 (0.0007) [2023-12-26 18:21:31,572][105692] Updated weights for policy 0, policy_version 411373 (0.0009) [2023-12-26 18:21:31,664][105692] Updated weights for policy 0, policy_version 411383 (0.0007) [2023-12-26 18:21:31,737][105692] Updated weights for policy 0, policy_version 411393 (0.0007) [2023-12-26 18:21:32,263][105620] Updated weights for policy 1, policy_version 411734 (0.0005) [2023-12-26 18:21:32,323][105620] Updated weights for policy 1, policy_version 411744 (0.0008) [2023-12-26 18:21:32,388][105620] Updated weights for policy 1, policy_version 411754 (0.0008) [2023-12-26 18:21:32,432][105692] Updated weights for policy 0, policy_version 411403 (0.0009) [2023-12-26 18:21:32,484][105692] Updated weights for policy 0, policy_version 411413 (0.0008) [2023-12-26 18:21:32,529][105692] Updated weights for policy 0, policy_version 411423 (0.0008) [2023-12-26 18:21:33,125][105620] Updated weights for policy 1, policy_version 411764 (0.0009) [2023-12-26 18:21:33,183][105620] Updated weights for policy 1, policy_version 411774 (0.0010) [2023-12-26 18:21:33,220][105692] Updated weights for policy 0, policy_version 411433 (0.0008) [2023-12-26 18:21:33,238][105620] Updated weights for policy 1, policy_version 411784 (0.0010) [2023-12-26 18:21:33,286][105692] Updated weights for policy 0, policy_version 411443 (0.0008) [2023-12-26 18:21:33,346][105692] Updated weights for policy 0, policy_version 411453 (0.0009) [2023-12-26 18:21:33,400][105692] Updated weights for policy 0, policy_version 411463 (0.0010) [2023-12-26 18:21:33,928][105620] Updated weights for policy 1, policy_version 411794 (0.0010) [2023-12-26 18:21:33,989][105620] Updated weights for policy 1, policy_version 411804 (0.0010) [2023-12-26 18:21:34,033][105620] Updated weights for policy 1, policy_version 411814 (0.0010) [2023-12-26 18:21:34,080][105620] Updated weights for policy 1, policy_version 411824 (0.0010) [2023-12-26 18:21:34,111][105692] Updated weights for policy 0, policy_version 411473 (0.0006) [2023-12-26 18:21:34,173][105692] Updated weights for policy 0, policy_version 411483 (0.0007) [2023-12-26 18:21:34,240][105692] Updated weights for policy 0, policy_version 411493 (0.0006) [2023-12-26 18:21:34,848][105620] Updated weights for policy 1, policy_version 411834 (0.0008) [2023-12-26 18:21:34,911][105620] Updated weights for policy 1, policy_version 411844 (0.0007) [2023-12-26 18:21:34,928][105692] Updated weights for policy 0, policy_version 411503 (0.0009) [2023-12-26 18:21:34,961][105620] Updated weights for policy 1, policy_version 411854 (0.0009) [2023-12-26 18:21:34,983][105692] Updated weights for policy 0, policy_version 411513 (0.0009) [2023-12-26 18:21:35,035][105692] Updated weights for policy 0, policy_version 411523 (0.0010) [2023-12-26 18:21:35,716][105620] Updated weights for policy 1, policy_version 411864 (0.0007) [2023-12-26 18:21:35,772][105620] Updated weights for policy 1, policy_version 411874 (0.0008) [2023-12-26 18:21:35,796][105692] Updated weights for policy 0, policy_version 411533 (0.0010) [2023-12-26 18:21:35,833][105620] Updated weights for policy 1, policy_version 411884 (0.0010) [2023-12-26 18:21:35,851][105692] Updated weights for policy 0, policy_version 411543 (0.0011) [2023-12-26 18:21:35,911][105692] Updated weights for policy 0, policy_version 411553 (0.0010) [2023-12-26 18:21:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 210829312. Throughput: 0: 9550.2, 1: 9839.8. Samples: 210815040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:21:36,062][104569] Avg episode reward: [(0, '9179.019'), (1, '1113.320')] [2023-12-26 18:21:36,621][105620] Updated weights for policy 1, policy_version 411894 (0.0008) [2023-12-26 18:21:36,677][105692] Updated weights for policy 0, policy_version 411563 (0.0010) [2023-12-26 18:21:36,680][105620] Updated weights for policy 1, policy_version 411904 (0.0007) [2023-12-26 18:21:36,740][105692] Updated weights for policy 0, policy_version 411573 (0.0010) [2023-12-26 18:21:36,741][105620] Updated weights for policy 1, policy_version 411914 (0.0009) [2023-12-26 18:21:36,802][105692] Updated weights for policy 0, policy_version 411583 (0.0010) [2023-12-26 18:21:37,506][105692] Updated weights for policy 0, policy_version 411593 (0.0010) [2023-12-26 18:21:37,526][105620] Updated weights for policy 1, policy_version 411924 (0.0006) [2023-12-26 18:21:37,566][105692] Updated weights for policy 0, policy_version 411603 (0.0011) [2023-12-26 18:21:37,584][105620] Updated weights for policy 1, policy_version 411934 (0.0006) [2023-12-26 18:21:37,626][105692] Updated weights for policy 0, policy_version 411613 (0.0011) [2023-12-26 18:21:37,635][105620] Updated weights for policy 1, policy_version 411944 (0.0006) [2023-12-26 18:21:37,687][105692] Updated weights for policy 0, policy_version 411623 (0.0011) [2023-12-26 18:21:38,328][105692] Updated weights for policy 0, policy_version 411633 (0.0008) [2023-12-26 18:21:38,379][105620] Updated weights for policy 1, policy_version 411954 (0.0006) [2023-12-26 18:21:38,397][105692] Updated weights for policy 0, policy_version 411643 (0.0007) [2023-12-26 18:21:38,436][105620] Updated weights for policy 1, policy_version 411964 (0.0009) [2023-12-26 18:21:38,465][105692] Updated weights for policy 0, policy_version 411653 (0.0007) [2023-12-26 18:21:38,497][105620] Updated weights for policy 1, policy_version 411974 (0.0010) [2023-12-26 18:21:38,560][105620] Updated weights for policy 1, policy_version 411984 (0.0009) [2023-12-26 18:21:39,132][105692] Updated weights for policy 0, policy_version 411663 (0.0010) [2023-12-26 18:21:39,183][105692] Updated weights for policy 0, policy_version 411673 (0.0010) [2023-12-26 18:21:39,200][105620] Updated weights for policy 1, policy_version 411994 (0.0005) [2023-12-26 18:21:39,237][105692] Updated weights for policy 0, policy_version 411683 (0.0010) [2023-12-26 18:21:39,263][105620] Updated weights for policy 1, policy_version 412004 (0.0006) [2023-12-26 18:21:39,322][105620] Updated weights for policy 1, policy_version 412014 (0.0008) [2023-12-26 18:21:39,976][105692] Updated weights for policy 0, policy_version 411693 (0.0010) [2023-12-26 18:21:40,040][105692] Updated weights for policy 0, policy_version 411703 (0.0008) [2023-12-26 18:21:40,099][105620] Updated weights for policy 1, policy_version 412024 (0.0010) [2023-12-26 18:21:40,100][105692] Updated weights for policy 0, policy_version 411713 (0.0007) [2023-12-26 18:21:40,157][105620] Updated weights for policy 1, policy_version 412034 (0.0011) [2023-12-26 18:21:40,207][105620] Updated weights for policy 1, policy_version 412044 (0.0011) [2023-12-26 18:21:40,876][105692] Updated weights for policy 0, policy_version 411723 (0.0007) [2023-12-26 18:21:40,923][105692] Updated weights for policy 0, policy_version 411733 (0.0008) [2023-12-26 18:21:40,928][105585] KL-divergence is very high: 198.6915 [2023-12-26 18:21:40,968][105585] KL-divergence is very high: 327.3658 [2023-12-26 18:21:40,972][105692] Updated weights for policy 0, policy_version 411743 (0.0009) [2023-12-26 18:21:40,987][105620] Updated weights for policy 1, policy_version 412054 (0.0011) [2023-12-26 18:21:41,005][105585] KL-divergence is very high: 345.7112 [2023-12-26 18:21:41,048][105620] Updated weights for policy 1, policy_version 412064 (0.0011) [2023-12-26 18:21:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 210919424. Throughput: 0: 9582.4, 1: 9747.6. Samples: 210928620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:21:41,062][104569] Avg episode reward: [(0, '9263.075'), (1, '1199.260')] [2023-12-26 18:21:41,113][105620] Updated weights for policy 1, policy_version 412074 (0.0010) [2023-12-26 18:21:41,648][105692] Updated weights for policy 0, policy_version 411753 (0.0006) [2023-12-26 18:21:41,714][105692] Updated weights for policy 0, policy_version 411763 (0.0010) [2023-12-26 18:21:41,780][105692] Updated weights for policy 0, policy_version 411773 (0.0009) [2023-12-26 18:21:41,836][105692] Updated weights for policy 0, policy_version 411783 (0.0009) [2023-12-26 18:21:41,882][105620] Updated weights for policy 1, policy_version 412084 (0.0007) [2023-12-26 18:21:41,943][105620] Updated weights for policy 1, policy_version 412094 (0.0007) [2023-12-26 18:21:41,991][105620] Updated weights for policy 1, policy_version 412104 (0.0009) [2023-12-26 18:21:42,611][105692] Updated weights for policy 0, policy_version 411793 (0.0006) [2023-12-26 18:21:42,676][105692] Updated weights for policy 0, policy_version 411803 (0.0006) [2023-12-26 18:21:42,739][105692] Updated weights for policy 0, policy_version 411813 (0.0006) [2023-12-26 18:21:42,786][105620] Updated weights for policy 1, policy_version 412114 (0.0009) [2023-12-26 18:21:42,838][105620] Updated weights for policy 1, policy_version 412124 (0.0009) [2023-12-26 18:21:42,885][105620] Updated weights for policy 1, policy_version 412134 (0.0009) [2023-12-26 18:21:42,933][105620] Updated weights for policy 1, policy_version 412144 (0.0009) [2023-12-26 18:21:43,391][105692] Updated weights for policy 0, policy_version 411823 (0.0009) [2023-12-26 18:21:43,449][105692] Updated weights for policy 0, policy_version 411833 (0.0009) [2023-12-26 18:21:43,504][105692] Updated weights for policy 0, policy_version 411843 (0.0009) [2023-12-26 18:21:43,701][105620] Updated weights for policy 1, policy_version 412154 (0.0009) [2023-12-26 18:21:43,756][105620] Updated weights for policy 1, policy_version 412165 (0.0011) [2023-12-26 18:21:43,815][105620] Updated weights for policy 1, policy_version 412175 (0.0010) [2023-12-26 18:21:44,116][105692] Updated weights for policy 0, policy_version 411853 (0.0007) [2023-12-26 18:21:44,168][105692] Updated weights for policy 0, policy_version 411863 (0.0005) [2023-12-26 18:21:44,228][105692] Updated weights for policy 0, policy_version 411873 (0.0005) [2023-12-26 18:21:44,592][105620] Updated weights for policy 1, policy_version 412185 (0.0006) [2023-12-26 18:21:44,651][105620] Updated weights for policy 1, policy_version 412195 (0.0005) [2023-12-26 18:21:44,706][105620] Updated weights for policy 1, policy_version 412205 (0.0005) [2023-12-26 18:21:44,824][105692] Updated weights for policy 0, policy_version 411883 (0.0007) [2023-12-26 18:21:44,886][105692] Updated weights for policy 0, policy_version 411893 (0.0010) [2023-12-26 18:21:44,942][105692] Updated weights for policy 0, policy_version 411903 (0.0010) [2023-12-26 18:21:45,250][105620] Updated weights for policy 1, policy_version 412215 (0.0005) [2023-12-26 18:21:45,311][105620] Updated weights for policy 1, policy_version 412225 (0.0009) [2023-12-26 18:21:45,377][105620] Updated weights for policy 1, policy_version 412235 (0.0007) [2023-12-26 18:21:45,778][105692] Updated weights for policy 0, policy_version 411913 (0.0010) [2023-12-26 18:21:45,841][105692] Updated weights for policy 0, policy_version 411923 (0.0008) [2023-12-26 18:21:45,904][105692] Updated weights for policy 0, policy_version 411933 (0.0009) [2023-12-26 18:21:45,971][105692] Updated weights for policy 0, policy_version 411943 (0.0010) [2023-12-26 18:21:46,056][105620] Updated weights for policy 1, policy_version 412245 (0.0006) [2023-12-26 18:21:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 211017728. Throughput: 0: 9560.5, 1: 9685.5. Samples: 210984752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:21:46,063][104569] Avg episode reward: [(0, '9263.104'), (1, '6424.695')] [2023-12-26 18:21:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000411944_105472000.pth... [2023-12-26 18:21:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000410792_105177088.pth [2023-12-26 18:21:46,073][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_000411944_105472000.pth [2023-12-26 18:21:46,109][105620] Updated weights for policy 1, policy_version 412255 (0.0005) [2023-12-26 18:21:46,159][105620] Updated weights for policy 1, policy_version 412265 (0.0005) [2023-12-26 18:21:46,201][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000412272_105553920.pth... [2023-12-26 18:21:46,205][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000411120_105259008.pth [2023-12-26 18:21:46,206][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_000412272_105553920.pth [2023-12-26 18:21:46,715][105620] Updated weights for policy 1, policy_version 412275 (0.0007) [2023-12-26 18:21:46,768][105620] Updated weights for policy 1, policy_version 412285 (0.0008) [2023-12-26 18:21:46,822][105620] Updated weights for policy 1, policy_version 412295 (0.0005) [2023-12-26 18:21:46,824][105692] Updated weights for policy 0, policy_version 411953 (0.0009) [2023-12-26 18:21:46,879][105692] Updated weights for policy 0, policy_version 411963 (0.0008) [2023-12-26 18:21:46,944][105692] Updated weights for policy 0, policy_version 411973 (0.0009) [2023-12-26 18:21:47,587][105620] Updated weights for policy 1, policy_version 412305 (0.0006) [2023-12-26 18:21:47,616][105692] Updated weights for policy 0, policy_version 411983 (0.0008) [2023-12-26 18:21:47,646][105620] Updated weights for policy 1, policy_version 412315 (0.0008) [2023-12-26 18:21:47,668][105692] Updated weights for policy 0, policy_version 411993 (0.0006) [2023-12-26 18:21:47,709][105620] Updated weights for policy 1, policy_version 412325 (0.0009) [2023-12-26 18:21:47,728][105692] Updated weights for policy 0, policy_version 412003 (0.0006) [2023-12-26 18:21:47,758][105620] Updated weights for policy 1, policy_version 412335 (0.0006) [2023-12-26 18:21:48,445][105620] Updated weights for policy 1, policy_version 412345 (0.0009) [2023-12-26 18:21:48,499][105620] Updated weights for policy 1, policy_version 412355 (0.0009) [2023-12-26 18:21:48,513][105692] Updated weights for policy 0, policy_version 412013 (0.0006) [2023-12-26 18:21:48,563][105620] Updated weights for policy 1, policy_version 412365 (0.0008) [2023-12-26 18:21:48,574][105692] Updated weights for policy 0, policy_version 412023 (0.0006) [2023-12-26 18:21:48,628][105692] Updated weights for policy 0, policy_version 412033 (0.0009) [2023-12-26 18:21:49,271][105620] Updated weights for policy 1, policy_version 412375 (0.0007) [2023-12-26 18:21:49,320][105620] Updated weights for policy 1, policy_version 412385 (0.0006) [2023-12-26 18:21:49,381][105620] Updated weights for policy 1, policy_version 412395 (0.0008) [2023-12-26 18:21:49,422][105692] Updated weights for policy 0, policy_version 412043 (0.0008) [2023-12-26 18:21:49,485][105692] Updated weights for policy 0, policy_version 412053 (0.0008) [2023-12-26 18:21:49,550][105692] Updated weights for policy 0, policy_version 412063 (0.0007) [2023-12-26 18:21:50,109][105620] Updated weights for policy 1, policy_version 412405 (0.0008) [2023-12-26 18:21:50,164][105620] Updated weights for policy 1, policy_version 412415 (0.0007) [2023-12-26 18:21:50,188][105692] Updated weights for policy 0, policy_version 412073 (0.0006) [2023-12-26 18:21:50,219][105620] Updated weights for policy 1, policy_version 412425 (0.0007) [2023-12-26 18:21:50,239][105692] Updated weights for policy 0, policy_version 412083 (0.0006) [2023-12-26 18:21:50,291][105692] Updated weights for policy 0, policy_version 412093 (0.0006) [2023-12-26 18:21:50,343][105692] Updated weights for policy 0, policy_version 412103 (0.0005) [2023-12-26 18:21:50,942][105620] Updated weights for policy 1, policy_version 412435 (0.0008) [2023-12-26 18:21:50,944][105692] Updated weights for policy 0, policy_version 412113 (0.0007) [2023-12-26 18:21:51,000][105692] Updated weights for policy 0, policy_version 412123 (0.0007) [2023-12-26 18:21:51,010][105620] Updated weights for policy 1, policy_version 412445 (0.0005) [2023-12-26 18:21:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 211107840. Throughput: 0: 9538.2, 1: 9757.4. Samples: 211103568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:21:51,063][104569] Avg episode reward: [(0, '8987.197'), (1, '9088.878')] [2023-12-26 18:21:51,063][105692] Updated weights for policy 0, policy_version 412133 (0.0007) [2023-12-26 18:21:51,076][105620] Updated weights for policy 1, policy_version 412455 (0.0008) [2023-12-26 18:21:51,756][105692] Updated weights for policy 0, policy_version 412143 (0.0008) [2023-12-26 18:21:51,818][105692] Updated weights for policy 0, policy_version 412153 (0.0009) [2023-12-26 18:21:51,841][105620] Updated weights for policy 1, policy_version 412465 (0.0009) [2023-12-26 18:21:51,876][105692] Updated weights for policy 0, policy_version 412163 (0.0007) [2023-12-26 18:21:51,899][105620] Updated weights for policy 1, policy_version 412475 (0.0008) [2023-12-26 18:21:51,951][105620] Updated weights for policy 1, policy_version 412485 (0.0009) [2023-12-26 18:21:52,018][105620] Updated weights for policy 1, policy_version 412495 (0.0010) [2023-12-26 18:21:52,588][105692] Updated weights for policy 0, policy_version 412173 (0.0008) [2023-12-26 18:21:52,647][105692] Updated weights for policy 0, policy_version 412183 (0.0008) [2023-12-26 18:21:52,701][105692] Updated weights for policy 0, policy_version 412193 (0.0010) [2023-12-26 18:21:52,773][105620] Updated weights for policy 1, policy_version 412505 (0.0008) [2023-12-26 18:21:52,840][105620] Updated weights for policy 1, policy_version 412515 (0.0009) [2023-12-26 18:21:52,901][105620] Updated weights for policy 1, policy_version 412525 (0.0009) [2023-12-26 18:21:53,502][105692] Updated weights for policy 0, policy_version 412203 (0.0009) [2023-12-26 18:21:53,557][105692] Updated weights for policy 0, policy_version 412213 (0.0009) [2023-12-26 18:21:53,583][105620] Updated weights for policy 1, policy_version 412535 (0.0007) [2023-12-26 18:21:53,611][105692] Updated weights for policy 0, policy_version 412223 (0.0008) [2023-12-26 18:21:53,636][105620] Updated weights for policy 1, policy_version 412545 (0.0006) [2023-12-26 18:21:53,689][105620] Updated weights for policy 1, policy_version 412555 (0.0006) [2023-12-26 18:21:54,266][105620] Updated weights for policy 1, policy_version 412565 (0.0005) [2023-12-26 18:21:54,328][105692] Updated weights for policy 0, policy_version 412233 (0.0007) [2023-12-26 18:21:54,331][105620] Updated weights for policy 1, policy_version 412575 (0.0006) [2023-12-26 18:21:54,390][105692] Updated weights for policy 0, policy_version 412243 (0.0007) [2023-12-26 18:21:54,391][105620] Updated weights for policy 1, policy_version 412585 (0.0009) [2023-12-26 18:21:54,439][105692] Updated weights for policy 0, policy_version 412253 (0.0008) [2023-12-26 18:21:54,493][105692] Updated weights for policy 0, policy_version 412263 (0.0007) [2023-12-26 18:21:54,989][105620] Updated weights for policy 1, policy_version 412595 (0.0009) [2023-12-26 18:21:55,040][105620] Updated weights for policy 1, policy_version 412605 (0.0007) [2023-12-26 18:21:55,087][105620] Updated weights for policy 1, policy_version 412615 (0.0006) [2023-12-26 18:21:55,281][105692] Updated weights for policy 0, policy_version 412273 (0.0006) [2023-12-26 18:21:55,330][105692] Updated weights for policy 0, policy_version 412283 (0.0005) [2023-12-26 18:21:55,395][105692] Updated weights for policy 0, policy_version 412293 (0.0006) [2023-12-26 18:21:55,658][105620] Updated weights for policy 1, policy_version 412625 (0.0005) [2023-12-26 18:21:55,731][105620] Updated weights for policy 1, policy_version 412635 (0.0005) [2023-12-26 18:21:55,798][105620] Updated weights for policy 1, policy_version 412645 (0.0005) [2023-12-26 18:21:55,871][105620] Updated weights for policy 1, policy_version 412655 (0.0005) [2023-12-26 18:21:56,027][105692] Updated weights for policy 0, policy_version 412303 (0.0010) [2023-12-26 18:21:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 211214336. Throughput: 0: 9631.1, 1: 9774.1. Samples: 211225044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:21:56,062][104569] Avg episode reward: [(0, '7592.006'), (1, '8995.968')] [2023-12-26 18:21:56,072][105692] Updated weights for policy 0, policy_version 412313 (0.0010) [2023-12-26 18:21:56,121][105692] Updated weights for policy 0, policy_version 412323 (0.0010) [2023-12-26 18:21:56,364][105620] Updated weights for policy 1, policy_version 412665 (0.0005) [2023-12-26 18:21:56,420][105620] Updated weights for policy 1, policy_version 412675 (0.0005) [2023-12-26 18:21:56,467][105620] Updated weights for policy 1, policy_version 412685 (0.0005) [2023-12-26 18:21:56,724][105692] Updated weights for policy 0, policy_version 412333 (0.0008) [2023-12-26 18:21:56,784][105692] Updated weights for policy 0, policy_version 412343 (0.0008) [2023-12-26 18:21:56,841][105692] Updated weights for policy 0, policy_version 412353 (0.0007) [2023-12-26 18:21:57,002][105620] Updated weights for policy 1, policy_version 412695 (0.0005) [2023-12-26 18:21:57,048][105620] Updated weights for policy 1, policy_version 412705 (0.0005) [2023-12-26 18:21:57,094][105620] Updated weights for policy 1, policy_version 412715 (0.0005) [2023-12-26 18:21:57,517][105692] Updated weights for policy 0, policy_version 412363 (0.0007) [2023-12-26 18:21:57,575][105692] Updated weights for policy 0, policy_version 412373 (0.0010) [2023-12-26 18:21:57,638][105620] Updated weights for policy 1, policy_version 412725 (0.0005) [2023-12-26 18:21:57,640][105692] Updated weights for policy 0, policy_version 412383 (0.0008) [2023-12-26 18:21:57,698][105620] Updated weights for policy 1, policy_version 412735 (0.0005) [2023-12-26 18:21:57,748][105620] Updated weights for policy 1, policy_version 412745 (0.0005) [2023-12-26 18:21:58,339][105692] Updated weights for policy 0, policy_version 412393 (0.0006) [2023-12-26 18:21:58,397][105692] Updated weights for policy 0, policy_version 412403 (0.0009) [2023-12-26 18:21:58,460][105620] Updated weights for policy 1, policy_version 412755 (0.0010) [2023-12-26 18:21:58,460][105692] Updated weights for policy 0, policy_version 412413 (0.0007) [2023-12-26 18:21:58,524][105692] Updated weights for policy 0, policy_version 412423 (0.0006) [2023-12-26 18:21:58,525][105620] Updated weights for policy 1, policy_version 412765 (0.0010) [2023-12-26 18:21:58,584][105620] Updated weights for policy 1, policy_version 412775 (0.0010) [2023-12-26 18:21:59,211][105692] Updated weights for policy 0, policy_version 412433 (0.0010) [2023-12-26 18:21:59,276][105692] Updated weights for policy 0, policy_version 412443 (0.0010) [2023-12-26 18:21:59,347][105692] Updated weights for policy 0, policy_version 412453 (0.0010) [2023-12-26 18:21:59,356][105620] Updated weights for policy 1, policy_version 412785 (0.0010) [2023-12-26 18:21:59,422][105620] Updated weights for policy 1, policy_version 412795 (0.0009) [2023-12-26 18:21:59,477][105620] Updated weights for policy 1, policy_version 412805 (0.0006) [2023-12-26 18:21:59,533][105620] Updated weights for policy 1, policy_version 412815 (0.0007) [2023-12-26 18:22:00,106][105692] Updated weights for policy 0, policy_version 412463 (0.0010) [2023-12-26 18:22:00,163][105692] Updated weights for policy 0, policy_version 412473 (0.0010) [2023-12-26 18:22:00,173][105620] Updated weights for policy 1, policy_version 412825 (0.0006) [2023-12-26 18:22:00,214][105692] Updated weights for policy 0, policy_version 412483 (0.0008) [2023-12-26 18:22:00,223][105620] Updated weights for policy 1, policy_version 412835 (0.0005) [2023-12-26 18:22:00,272][105620] Updated weights for policy 1, policy_version 412845 (0.0007) [2023-12-26 18:22:00,921][105620] Updated weights for policy 1, policy_version 412855 (0.0009) [2023-12-26 18:22:00,967][105620] Updated weights for policy 1, policy_version 412865 (0.0008) [2023-12-26 18:22:01,019][105620] Updated weights for policy 1, policy_version 412875 (0.0009) [2023-12-26 18:22:01,036][105692] Updated weights for policy 0, policy_version 412493 (0.0008) [2023-12-26 18:22:01,062][104569] Fps is (10 sec: 21298.9, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 211320832. Throughput: 0: 9704.1, 1: 9881.3. Samples: 211289196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:22:01,062][104569] Avg episode reward: [(0, '6999.272'), (1, '8995.843')] [2023-12-26 18:22:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000412880_105709568.pth... [2023-12-26 18:22:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000411696_105406464.pth [2023-12-26 18:22:01,100][105692] Updated weights for policy 0, policy_version 412503 (0.0010) [2023-12-26 18:22:01,166][105692] Updated weights for policy 0, policy_version 412513 (0.0009) [2023-12-26 18:22:01,207][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000412520_105619456.pth... [2023-12-26 18:22:01,212][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000411368_105324544.pth [2023-12-26 18:22:01,794][105620] Updated weights for policy 1, policy_version 412885 (0.0009) [2023-12-26 18:22:01,833][105692] Updated weights for policy 0, policy_version 412523 (0.0008) [2023-12-26 18:22:01,846][105620] Updated weights for policy 1, policy_version 412895 (0.0008) [2023-12-26 18:22:01,886][105692] Updated weights for policy 0, policy_version 412533 (0.0005) [2023-12-26 18:22:01,899][105620] Updated weights for policy 1, policy_version 412905 (0.0008) [2023-12-26 18:22:01,936][105692] Updated weights for policy 0, policy_version 412543 (0.0005) [2023-12-26 18:22:02,631][105620] Updated weights for policy 1, policy_version 412915 (0.0008) [2023-12-26 18:22:02,659][105692] Updated weights for policy 0, policy_version 412553 (0.0009) [2023-12-26 18:22:02,696][105620] Updated weights for policy 1, policy_version 412925 (0.0005) [2023-12-26 18:22:02,717][105692] Updated weights for policy 0, policy_version 412563 (0.0009) [2023-12-26 18:22:02,753][105620] Updated weights for policy 1, policy_version 412935 (0.0005) [2023-12-26 18:22:02,774][105692] Updated weights for policy 0, policy_version 412573 (0.0009) [2023-12-26 18:22:02,831][105692] Updated weights for policy 0, policy_version 412583 (0.0009) [2023-12-26 18:22:03,422][105620] Updated weights for policy 1, policy_version 412945 (0.0006) [2023-12-26 18:22:03,486][105620] Updated weights for policy 1, policy_version 412955 (0.0009) [2023-12-26 18:22:03,548][105692] Updated weights for policy 0, policy_version 412593 (0.0006) [2023-12-26 18:22:03,549][105620] Updated weights for policy 1, policy_version 412965 (0.0010) [2023-12-26 18:22:03,610][105692] Updated weights for policy 0, policy_version 412603 (0.0006) [2023-12-26 18:22:03,619][105620] Updated weights for policy 1, policy_version 412975 (0.0009) [2023-12-26 18:22:03,677][105692] Updated weights for policy 0, policy_version 412613 (0.0005) [2023-12-26 18:22:04,211][105692] Updated weights for policy 0, policy_version 412623 (0.0007) [2023-12-26 18:22:04,268][105692] Updated weights for policy 0, policy_version 412633 (0.0006) [2023-12-26 18:22:04,335][105692] Updated weights for policy 0, policy_version 412643 (0.0009) [2023-12-26 18:22:04,392][105620] Updated weights for policy 1, policy_version 412985 (0.0007) [2023-12-26 18:22:04,459][105620] Updated weights for policy 1, policy_version 412995 (0.0007) [2023-12-26 18:22:04,513][105620] Updated weights for policy 1, policy_version 413005 (0.0007) [2023-12-26 18:22:05,014][105692] Updated weights for policy 0, policy_version 412653 (0.0009) [2023-12-26 18:22:05,074][105692] Updated weights for policy 0, policy_version 412663 (0.0010) [2023-12-26 18:22:05,132][105692] Updated weights for policy 0, policy_version 412675 (0.0010) [2023-12-26 18:22:05,189][105620] Updated weights for policy 1, policy_version 413015 (0.0008) [2023-12-26 18:22:05,247][105620] Updated weights for policy 1, policy_version 413025 (0.0008) [2023-12-26 18:22:05,299][105620] Updated weights for policy 1, policy_version 413035 (0.0006) [2023-12-26 18:22:05,858][105692] Updated weights for policy 0, policy_version 412685 (0.0008) [2023-12-26 18:22:05,915][105692] Updated weights for policy 0, policy_version 412695 (0.0005) [2023-12-26 18:22:05,971][105692] Updated weights for policy 0, policy_version 412705 (0.0005) [2023-12-26 18:22:05,987][105620] Updated weights for policy 1, policy_version 413045 (0.0007) [2023-12-26 18:22:06,041][105620] Updated weights for policy 1, policy_version 413055 (0.0006) [2023-12-26 18:22:06,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 211419136. Throughput: 0: 9811.1, 1: 9881.0. Samples: 211406784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:22:06,063][104569] Avg episode reward: [(0, '8488.424'), (1, '9176.189')] [2023-12-26 18:22:06,100][105620] Updated weights for policy 1, policy_version 413065 (0.0007) [2023-12-26 18:22:06,633][105692] Updated weights for policy 0, policy_version 412715 (0.0007) [2023-12-26 18:22:06,697][105692] Updated weights for policy 0, policy_version 412725 (0.0010) [2023-12-26 18:22:06,762][105692] Updated weights for policy 0, policy_version 412735 (0.0011) [2023-12-26 18:22:06,800][105620] Updated weights for policy 1, policy_version 413075 (0.0007) [2023-12-26 18:22:06,868][105620] Updated weights for policy 1, policy_version 413085 (0.0008) [2023-12-26 18:22:06,933][105620] Updated weights for policy 1, policy_version 413095 (0.0010) [2023-12-26 18:22:07,381][105692] Updated weights for policy 0, policy_version 412745 (0.0011) [2023-12-26 18:22:07,434][105692] Updated weights for policy 0, policy_version 412755 (0.0010) [2023-12-26 18:22:07,485][105692] Updated weights for policy 0, policy_version 412765 (0.0010) [2023-12-26 18:22:07,552][105692] Updated weights for policy 0, policy_version 412775 (0.0010) [2023-12-26 18:22:07,753][105620] Updated weights for policy 1, policy_version 413105 (0.0008) [2023-12-26 18:22:07,810][105620] Updated weights for policy 1, policy_version 413115 (0.0010) [2023-12-26 18:22:07,876][105620] Updated weights for policy 1, policy_version 413125 (0.0008) [2023-12-26 18:22:07,931][105620] Updated weights for policy 1, policy_version 413135 (0.0010) [2023-12-26 18:22:08,303][105692] Updated weights for policy 0, policy_version 412785 (0.0010) [2023-12-26 18:22:08,373][105692] Updated weights for policy 0, policy_version 412795 (0.0010) [2023-12-26 18:22:08,439][105692] Updated weights for policy 0, policy_version 412805 (0.0010) [2023-12-26 18:22:08,559][105620] Updated weights for policy 1, policy_version 413145 (0.0011) [2023-12-26 18:22:08,627][105620] Updated weights for policy 1, policy_version 413155 (0.0011) [2023-12-26 18:22:08,685][105620] Updated weights for policy 1, policy_version 413165 (0.0008) [2023-12-26 18:22:09,196][105692] Updated weights for policy 0, policy_version 412815 (0.0010) [2023-12-26 18:22:09,270][105692] Updated weights for policy 0, policy_version 412825 (0.0009) [2023-12-26 18:22:09,344][105692] Updated weights for policy 0, policy_version 412835 (0.0008) [2023-12-26 18:22:09,356][105620] Updated weights for policy 1, policy_version 413175 (0.0008) [2023-12-26 18:22:09,422][105620] Updated weights for policy 1, policy_version 413185 (0.0008) [2023-12-26 18:22:09,482][105620] Updated weights for policy 1, policy_version 413195 (0.0009) [2023-12-26 18:22:10,140][105692] Updated weights for policy 0, policy_version 412845 (0.0009) [2023-12-26 18:22:10,209][105692] Updated weights for policy 0, policy_version 412855 (0.0008) [2023-12-26 18:22:10,262][105620] Updated weights for policy 1, policy_version 413205 (0.0009) [2023-12-26 18:22:10,271][105692] Updated weights for policy 0, policy_version 412865 (0.0007) [2023-12-26 18:22:10,323][105620] Updated weights for policy 1, policy_version 413215 (0.0007) [2023-12-26 18:22:10,392][105620] Updated weights for policy 1, policy_version 413225 (0.0008) [2023-12-26 18:22:10,929][105692] Updated weights for policy 0, policy_version 412875 (0.0008) [2023-12-26 18:22:10,982][105692] Updated weights for policy 0, policy_version 412885 (0.0008) [2023-12-26 18:22:11,038][105692] Updated weights for policy 0, policy_version 412895 (0.0009) [2023-12-26 18:22:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 211509248. Throughput: 0: 9814.2, 1: 9863.7. Samples: 211522096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:22:11,063][104569] Avg episode reward: [(0, '8922.602'), (1, '9356.749')] [2023-12-26 18:22:11,234][105620] Updated weights for policy 1, policy_version 413235 (0.0010) [2023-12-26 18:22:11,307][105620] Updated weights for policy 1, policy_version 413245 (0.0009) [2023-12-26 18:22:11,380][105620] Updated weights for policy 1, policy_version 413255 (0.0008) [2023-12-26 18:22:11,898][105692] Updated weights for policy 0, policy_version 412905 (0.0007) [2023-12-26 18:22:11,966][105692] Updated weights for policy 0, policy_version 412915 (0.0008) [2023-12-26 18:22:12,036][105692] Updated weights for policy 0, policy_version 412925 (0.0009) [2023-12-26 18:22:12,103][105692] Updated weights for policy 0, policy_version 412935 (0.0008) [2023-12-26 18:22:12,270][105620] Updated weights for policy 1, policy_version 413265 (0.0009) [2023-12-26 18:22:12,343][105620] Updated weights for policy 1, policy_version 413275 (0.0009) [2023-12-26 18:22:12,419][105620] Updated weights for policy 1, policy_version 413285 (0.0010) [2023-12-26 18:22:12,485][105620] Updated weights for policy 1, policy_version 413295 (0.0009) [2023-12-26 18:22:12,951][105692] Updated weights for policy 0, policy_version 412945 (0.0010) [2023-12-26 18:22:13,015][105692] Updated weights for policy 0, policy_version 412955 (0.0008) [2023-12-26 18:22:13,081][105692] Updated weights for policy 0, policy_version 412965 (0.0008) [2023-12-26 18:22:13,383][105620] Updated weights for policy 1, policy_version 413305 (0.0008) [2023-12-26 18:22:13,452][105620] Updated weights for policy 1, policy_version 413315 (0.0008) [2023-12-26 18:22:13,524][105620] Updated weights for policy 1, policy_version 413325 (0.0009) [2023-12-26 18:22:13,921][105692] Updated weights for policy 0, policy_version 412975 (0.0010) [2023-12-26 18:22:13,979][105692] Updated weights for policy 0, policy_version 412985 (0.0009) [2023-12-26 18:22:14,046][105692] Updated weights for policy 0, policy_version 412995 (0.0010) [2023-12-26 18:22:14,269][105620] Updated weights for policy 1, policy_version 413335 (0.0007) [2023-12-26 18:22:14,337][105620] Updated weights for policy 1, policy_version 413345 (0.0008) [2023-12-26 18:22:14,408][105620] Updated weights for policy 1, policy_version 413355 (0.0008) [2023-12-26 18:22:14,812][105692] Updated weights for policy 0, policy_version 413005 (0.0011) [2023-12-26 18:22:14,869][105692] Updated weights for policy 0, policy_version 413015 (0.0011) [2023-12-26 18:22:14,938][105692] Updated weights for policy 0, policy_version 413025 (0.0011) [2023-12-26 18:22:15,130][105620] Updated weights for policy 1, policy_version 413365 (0.0008) [2023-12-26 18:22:15,183][105620] Updated weights for policy 1, policy_version 413375 (0.0009) [2023-12-26 18:22:15,244][105620] Updated weights for policy 1, policy_version 413385 (0.0008) [2023-12-26 18:22:15,703][105692] Updated weights for policy 0, policy_version 413035 (0.0010) [2023-12-26 18:22:15,749][105692] Updated weights for policy 0, policy_version 413045 (0.0005) [2023-12-26 18:22:15,799][105692] Updated weights for policy 0, policy_version 413055 (0.0008) [2023-12-26 18:22:16,025][105620] Updated weights for policy 1, policy_version 413395 (0.0009) [2023-12-26 18:22:16,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 211599360. Throughput: 0: 9654.7, 1: 9805.5. Samples: 211571852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:22:16,063][104569] Avg episode reward: [(0, '9355.297'), (1, '9356.831')] [2023-12-26 18:22:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000413064_105758720.pth... [2023-12-26 18:22:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000411944_105472000.pth [2023-12-26 18:22:16,085][105620] Updated weights for policy 1, policy_version 413405 (0.0008) [2023-12-26 18:22:16,140][105620] Updated weights for policy 1, policy_version 413415 (0.0009) [2023-12-26 18:22:16,185][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000413424_105848832.pth... [2023-12-26 18:22:16,188][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000412272_105553920.pth [2023-12-26 18:22:16,600][105692] Updated weights for policy 0, policy_version 413065 (0.0008) [2023-12-26 18:22:16,647][105692] Updated weights for policy 0, policy_version 413075 (0.0009) [2023-12-26 18:22:16,695][105692] Updated weights for policy 0, policy_version 413085 (0.0009) [2023-12-26 18:22:16,741][105692] Updated weights for policy 0, policy_version 413095 (0.0009) [2023-12-26 18:22:16,818][105620] Updated weights for policy 1, policy_version 413425 (0.0009) [2023-12-26 18:22:16,886][105620] Updated weights for policy 1, policy_version 413435 (0.0009) [2023-12-26 18:22:16,954][105620] Updated weights for policy 1, policy_version 413445 (0.0010) [2023-12-26 18:22:17,023][105620] Updated weights for policy 1, policy_version 413455 (0.0007) [2023-12-26 18:22:17,542][105692] Updated weights for policy 0, policy_version 413105 (0.0008) [2023-12-26 18:22:17,598][105692] Updated weights for policy 0, policy_version 413115 (0.0008) [2023-12-26 18:22:17,655][105692] Updated weights for policy 0, policy_version 413125 (0.0008) [2023-12-26 18:22:17,819][105620] Updated weights for policy 1, policy_version 413465 (0.0008) [2023-12-26 18:22:17,884][105620] Updated weights for policy 1, policy_version 413475 (0.0006) [2023-12-26 18:22:17,945][105620] Updated weights for policy 1, policy_version 413485 (0.0010) [2023-12-26 18:22:18,425][105692] Updated weights for policy 0, policy_version 413135 (0.0008) [2023-12-26 18:22:18,494][105692] Updated weights for policy 0, policy_version 413145 (0.0006) [2023-12-26 18:22:18,564][105692] Updated weights for policy 0, policy_version 413155 (0.0009) [2023-12-26 18:22:18,660][105620] Updated weights for policy 1, policy_version 413495 (0.0011) [2023-12-26 18:22:18,720][105620] Updated weights for policy 1, policy_version 413505 (0.0011) [2023-12-26 18:22:18,783][105620] Updated weights for policy 1, policy_version 413515 (0.0011) [2023-12-26 18:22:19,275][105692] Updated weights for policy 0, policy_version 413165 (0.0010) [2023-12-26 18:22:19,336][105692] Updated weights for policy 0, policy_version 413175 (0.0011) [2023-12-26 18:22:19,399][105692] Updated weights for policy 0, policy_version 413185 (0.0011) [2023-12-26 18:22:19,567][105620] Updated weights for policy 1, policy_version 413525 (0.0010) [2023-12-26 18:22:19,633][105620] Updated weights for policy 1, policy_version 413535 (0.0011) [2023-12-26 18:22:19,699][105620] Updated weights for policy 1, policy_version 413545 (0.0011) [2023-12-26 18:22:20,222][105692] Updated weights for policy 0, policy_version 413195 (0.0007) [2023-12-26 18:22:20,287][105692] Updated weights for policy 0, policy_version 413205 (0.0008) [2023-12-26 18:22:20,356][105692] Updated weights for policy 0, policy_version 413215 (0.0009) [2023-12-26 18:22:20,547][105620] Updated weights for policy 1, policy_version 413555 (0.0010) [2023-12-26 18:22:20,626][105620] Updated weights for policy 1, policy_version 413565 (0.0010) [2023-12-26 18:22:20,692][105620] Updated weights for policy 1, policy_version 413575 (0.0010) [2023-12-26 18:22:21,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 211689472. Throughput: 0: 9580.3, 1: 9683.6. Samples: 211681920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:22:21,063][104569] Avg episode reward: [(0, '9357.329'), (1, '9356.829')] [2023-12-26 18:22:21,156][105692] Updated weights for policy 0, policy_version 413225 (0.0009) [2023-12-26 18:22:21,229][105692] Updated weights for policy 0, policy_version 413235 (0.0009) [2023-12-26 18:22:21,300][105692] Updated weights for policy 0, policy_version 413245 (0.0008) [2023-12-26 18:22:21,366][105692] Updated weights for policy 0, policy_version 413255 (0.0007) [2023-12-26 18:22:21,479][105620] Updated weights for policy 1, policy_version 413585 (0.0011) [2023-12-26 18:22:21,541][105620] Updated weights for policy 1, policy_version 413595 (0.0011) [2023-12-26 18:22:21,606][105620] Updated weights for policy 1, policy_version 413605 (0.0011) [2023-12-26 18:22:21,678][105620] Updated weights for policy 1, policy_version 413615 (0.0011) [2023-12-26 18:22:22,119][105692] Updated weights for policy 0, policy_version 413265 (0.0008) [2023-12-26 18:22:22,179][105692] Updated weights for policy 0, policy_version 413275 (0.0008) [2023-12-26 18:22:22,242][105692] Updated weights for policy 0, policy_version 413285 (0.0009) [2023-12-26 18:22:22,435][105620] Updated weights for policy 1, policy_version 413625 (0.0008) [2023-12-26 18:22:22,505][105620] Updated weights for policy 1, policy_version 413635 (0.0011) [2023-12-26 18:22:22,575][105620] Updated weights for policy 1, policy_version 413645 (0.0010) [2023-12-26 18:22:23,083][105692] Updated weights for policy 0, policy_version 413295 (0.0007) [2023-12-26 18:22:23,151][105692] Updated weights for policy 0, policy_version 413305 (0.0009) [2023-12-26 18:22:23,221][105692] Updated weights for policy 0, policy_version 413315 (0.0009) [2023-12-26 18:22:23,361][105620] Updated weights for policy 1, policy_version 413655 (0.0009) [2023-12-26 18:22:23,429][105620] Updated weights for policy 1, policy_version 413665 (0.0006) [2023-12-26 18:22:23,494][105620] Updated weights for policy 1, policy_version 413675 (0.0006) [2023-12-26 18:22:24,038][105692] Updated weights for policy 0, policy_version 413325 (0.0009) [2023-12-26 18:22:24,103][105692] Updated weights for policy 0, policy_version 413335 (0.0009) [2023-12-26 18:22:24,176][105692] Updated weights for policy 0, policy_version 413345 (0.0009) [2023-12-26 18:22:24,263][105620] Updated weights for policy 1, policy_version 413685 (0.0008) [2023-12-26 18:22:24,334][105620] Updated weights for policy 1, policy_version 413695 (0.0011) [2023-12-26 18:22:24,398][105620] Updated weights for policy 1, policy_version 413705 (0.0010) [2023-12-26 18:22:24,973][105692] Updated weights for policy 0, policy_version 413355 (0.0010) [2023-12-26 18:22:25,039][105692] Updated weights for policy 0, policy_version 413365 (0.0008) [2023-12-26 18:22:25,108][105692] Updated weights for policy 0, policy_version 413375 (0.0009) [2023-12-26 18:22:25,180][105620] Updated weights for policy 1, policy_version 413715 (0.0010) [2023-12-26 18:22:25,244][105620] Updated weights for policy 1, policy_version 413725 (0.0011) [2023-12-26 18:22:25,304][105620] Updated weights for policy 1, policy_version 413735 (0.0011) [2023-12-26 18:22:25,917][105692] Updated weights for policy 0, policy_version 413385 (0.0010) [2023-12-26 18:22:25,985][105692] Updated weights for policy 0, policy_version 413395 (0.0009) [2023-12-26 18:22:26,062][104569] Fps is (10 sec: 17203.3, 60 sec: 19114.7, 300 sec: 19355.3). Total num frames: 211771392. Throughput: 0: 9426.5, 1: 9618.4. Samples: 211785640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:22:26,062][104569] Avg episode reward: [(0, '9357.807'), (1, '9265.028')] [2023-12-26 18:22:26,064][105692] Updated weights for policy 0, policy_version 413405 (0.0010) [2023-12-26 18:22:26,096][105620] Updated weights for policy 1, policy_version 413745 (0.0010) [2023-12-26 18:22:26,130][105692] Updated weights for policy 0, policy_version 413415 (0.0010) [2023-12-26 18:22:26,164][105620] Updated weights for policy 1, policy_version 413755 (0.0009) [2023-12-26 18:22:26,241][105620] Updated weights for policy 1, policy_version 413765 (0.0011) [2023-12-26 18:22:26,308][105620] Updated weights for policy 1, policy_version 413775 (0.0011) [2023-12-26 18:22:26,928][105692] Updated weights for policy 0, policy_version 413425 (0.0008) [2023-12-26 18:22:26,985][105692] Updated weights for policy 0, policy_version 413435 (0.0008) [2023-12-26 18:22:27,043][105692] Updated weights for policy 0, policy_version 413445 (0.0008) [2023-12-26 18:22:27,066][105620] Updated weights for policy 1, policy_version 413785 (0.0011) [2023-12-26 18:22:27,122][105620] Updated weights for policy 1, policy_version 413795 (0.0011) [2023-12-26 18:22:27,176][105620] Updated weights for policy 1, policy_version 413805 (0.0011) [2023-12-26 18:22:27,758][105692] Updated weights for policy 0, policy_version 413455 (0.0007) [2023-12-26 18:22:27,834][105692] Updated weights for policy 0, policy_version 413465 (0.0008) [2023-12-26 18:22:27,897][105692] Updated weights for policy 0, policy_version 413475 (0.0009) [2023-12-26 18:22:27,936][105620] Updated weights for policy 1, policy_version 413815 (0.0011) [2023-12-26 18:22:27,999][105620] Updated weights for policy 1, policy_version 413825 (0.0011) [2023-12-26 18:22:28,061][105620] Updated weights for policy 1, policy_version 413835 (0.0011) [2023-12-26 18:22:28,694][105692] Updated weights for policy 0, policy_version 413485 (0.0007) [2023-12-26 18:22:28,760][105692] Updated weights for policy 0, policy_version 413495 (0.0008) [2023-12-26 18:22:28,826][105692] Updated weights for policy 0, policy_version 413505 (0.0009) [2023-12-26 18:22:28,923][105620] Updated weights for policy 1, policy_version 413845 (0.0010) [2023-12-26 18:22:29,000][105620] Updated weights for policy 1, policy_version 413855 (0.0011) [2023-12-26 18:22:29,071][105620] Updated weights for policy 1, policy_version 413865 (0.0011) [2023-12-26 18:22:29,651][105692] Updated weights for policy 0, policy_version 413515 (0.0009) [2023-12-26 18:22:29,721][105692] Updated weights for policy 0, policy_version 413525 (0.0008) [2023-12-26 18:22:29,793][105692] Updated weights for policy 0, policy_version 413535 (0.0008) [2023-12-26 18:22:29,899][105620] Updated weights for policy 1, policy_version 413875 (0.0010) [2023-12-26 18:22:29,962][105620] Updated weights for policy 1, policy_version 413885 (0.0008) [2023-12-26 18:22:30,024][105620] Updated weights for policy 1, policy_version 413895 (0.0009) [2023-12-26 18:22:30,576][105692] Updated weights for policy 0, policy_version 413545 (0.0008) [2023-12-26 18:22:30,641][105692] Updated weights for policy 0, policy_version 413555 (0.0009) [2023-12-26 18:22:30,702][105692] Updated weights for policy 0, policy_version 413565 (0.0008) [2023-12-26 18:22:30,759][105692] Updated weights for policy 0, policy_version 413575 (0.0010) [2023-12-26 18:22:30,807][105620] Updated weights for policy 1, policy_version 413905 (0.0008) [2023-12-26 18:22:30,872][105620] Updated weights for policy 1, policy_version 413915 (0.0008) [2023-12-26 18:22:30,938][105620] Updated weights for policy 1, policy_version 413925 (0.0008) [2023-12-26 18:22:31,006][105620] Updated weights for policy 1, policy_version 413935 (0.0009) [2023-12-26 18:22:31,062][104569] Fps is (10 sec: 18022.1, 60 sec: 18978.1, 300 sec: 19383.1). Total num frames: 211869696. Throughput: 0: 9391.5, 1: 9595.8. Samples: 211839180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:22:31,063][104569] Avg episode reward: [(0, '9358.275'), (1, '9265.176')] [2023-12-26 18:22:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000413576_105889792.pth... [2023-12-26 18:22:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000413936_105979904.pth... [2023-12-26 18:22:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000412520_105619456.pth [2023-12-26 18:22:31,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000412880_105709568.pth [2023-12-26 18:22:31,681][105692] Updated weights for policy 0, policy_version 413585 (0.0008) [2023-12-26 18:22:31,750][105692] Updated weights for policy 0, policy_version 413595 (0.0008) [2023-12-26 18:22:31,813][105692] Updated weights for policy 0, policy_version 413605 (0.0008) [2023-12-26 18:22:31,885][105620] Updated weights for policy 1, policy_version 413945 (0.0007) [2023-12-26 18:22:31,950][105620] Updated weights for policy 1, policy_version 413955 (0.0009) [2023-12-26 18:22:32,012][105620] Updated weights for policy 1, policy_version 413965 (0.0008) [2023-12-26 18:22:32,549][105692] Updated weights for policy 0, policy_version 413615 (0.0007) [2023-12-26 18:22:32,615][105692] Updated weights for policy 0, policy_version 413625 (0.0009) [2023-12-26 18:22:32,685][105692] Updated weights for policy 0, policy_version 413635 (0.0010) [2023-12-26 18:22:32,767][105620] Updated weights for policy 1, policy_version 413975 (0.0008) [2023-12-26 18:22:32,828][105620] Updated weights for policy 1, policy_version 413985 (0.0009) [2023-12-26 18:22:32,875][105620] Updated weights for policy 1, policy_version 413995 (0.0008) [2023-12-26 18:22:33,399][105692] Updated weights for policy 0, policy_version 413645 (0.0009) [2023-12-26 18:22:33,460][105692] Updated weights for policy 0, policy_version 413655 (0.0011) [2023-12-26 18:22:33,527][105692] Updated weights for policy 0, policy_version 413665 (0.0010) [2023-12-26 18:22:33,633][105620] Updated weights for policy 1, policy_version 414005 (0.0009) [2023-12-26 18:22:33,700][105620] Updated weights for policy 1, policy_version 414015 (0.0009) [2023-12-26 18:22:33,761][105620] Updated weights for policy 1, policy_version 414025 (0.0008) [2023-12-26 18:22:34,237][105692] Updated weights for policy 0, policy_version 413675 (0.0011) [2023-12-26 18:22:34,299][105692] Updated weights for policy 0, policy_version 413685 (0.0011) [2023-12-26 18:22:34,366][105692] Updated weights for policy 0, policy_version 413695 (0.0010) [2023-12-26 18:22:34,592][105620] Updated weights for policy 1, policy_version 414035 (0.0009) [2023-12-26 18:22:34,667][105620] Updated weights for policy 1, policy_version 414045 (0.0009) [2023-12-26 18:22:34,726][105620] Updated weights for policy 1, policy_version 414055 (0.0007) [2023-12-26 18:22:35,132][105692] Updated weights for policy 0, policy_version 413705 (0.0009) [2023-12-26 18:22:35,191][105692] Updated weights for policy 0, policy_version 413715 (0.0011) [2023-12-26 18:22:35,261][105692] Updated weights for policy 0, policy_version 413725 (0.0011) [2023-12-26 18:22:35,321][105692] Updated weights for policy 0, policy_version 413735 (0.0011) [2023-12-26 18:22:35,469][105620] Updated weights for policy 1, policy_version 414065 (0.0007) [2023-12-26 18:22:35,524][105620] Updated weights for policy 1, policy_version 414075 (0.0006) [2023-12-26 18:22:35,588][105620] Updated weights for policy 1, policy_version 414085 (0.0008) [2023-12-26 18:22:35,656][105620] Updated weights for policy 1, policy_version 414095 (0.0008) [2023-12-26 18:22:36,062][104569] Fps is (10 sec: 18022.4, 60 sec: 18705.1, 300 sec: 19327.6). Total num frames: 211951616. Throughput: 0: 9298.2, 1: 9352.4. Samples: 211942844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:22:36,062][104569] Avg episode reward: [(0, '9172.763'), (1, '9265.182')] [2023-12-26 18:22:36,118][105692] Updated weights for policy 0, policy_version 413745 (0.0011) [2023-12-26 18:22:36,186][105692] Updated weights for policy 0, policy_version 413755 (0.0008) [2023-12-26 18:22:36,254][105692] Updated weights for policy 0, policy_version 413765 (0.0010) [2023-12-26 18:22:36,254][105585] KL-divergence is very high: 236.8747 [2023-12-26 18:22:36,384][105620] Updated weights for policy 1, policy_version 414105 (0.0010) [2023-12-26 18:22:36,456][105620] Updated weights for policy 1, policy_version 414115 (0.0010) [2023-12-26 18:22:36,525][105620] Updated weights for policy 1, policy_version 414125 (0.0011) [2023-12-26 18:22:36,999][105692] Updated weights for policy 0, policy_version 413775 (0.0007) [2023-12-26 18:22:37,063][105692] Updated weights for policy 0, policy_version 413785 (0.0007) [2023-12-26 18:22:37,129][105692] Updated weights for policy 0, policy_version 413795 (0.0009) [2023-12-26 18:22:37,247][105620] Updated weights for policy 1, policy_version 414135 (0.0009) [2023-12-26 18:22:37,300][105620] Updated weights for policy 1, policy_version 414145 (0.0008) [2023-12-26 18:22:37,360][105620] Updated weights for policy 1, policy_version 414155 (0.0008) [2023-12-26 18:22:37,878][105692] Updated weights for policy 0, policy_version 413805 (0.0010) [2023-12-26 18:22:37,935][105692] Updated weights for policy 0, policy_version 413815 (0.0008) [2023-12-26 18:22:37,997][105692] Updated weights for policy 0, policy_version 413825 (0.0009) [2023-12-26 18:22:38,164][105620] Updated weights for policy 1, policy_version 414165 (0.0009) [2023-12-26 18:22:38,230][105620] Updated weights for policy 1, policy_version 414175 (0.0009) [2023-12-26 18:22:38,292][105620] Updated weights for policy 1, policy_version 414185 (0.0009) [2023-12-26 18:22:38,818][105692] Updated weights for policy 0, policy_version 413835 (0.0009) [2023-12-26 18:22:38,882][105692] Updated weights for policy 0, policy_version 413845 (0.0010) [2023-12-26 18:22:38,941][105692] Updated weights for policy 0, policy_version 413855 (0.0009) [2023-12-26 18:22:39,162][105620] Updated weights for policy 1, policy_version 414195 (0.0009) [2023-12-26 18:22:39,234][105620] Updated weights for policy 1, policy_version 414205 (0.0012) [2023-12-26 18:22:39,308][105620] Updated weights for policy 1, policy_version 414215 (0.0007) [2023-12-26 18:22:39,920][105692] Updated weights for policy 0, policy_version 413865 (0.0010) [2023-12-26 18:22:39,991][105692] Updated weights for policy 0, policy_version 413875 (0.0010) [2023-12-26 18:22:40,055][105692] Updated weights for policy 0, policy_version 413885 (0.0009) [2023-12-26 18:22:40,119][105692] Updated weights for policy 0, policy_version 413895 (0.0009) [2023-12-26 18:22:40,133][105620] Updated weights for policy 1, policy_version 414225 (0.0009) [2023-12-26 18:22:40,206][105620] Updated weights for policy 1, policy_version 414235 (0.0009) [2023-12-26 18:22:40,266][105620] Updated weights for policy 1, policy_version 414245 (0.0009) [2023-12-26 18:22:40,325][105620] Updated weights for policy 1, policy_version 414255 (0.0009) [2023-12-26 18:22:40,907][105692] Updated weights for policy 0, policy_version 413905 (0.0006) [2023-12-26 18:22:40,968][105692] Updated weights for policy 0, policy_version 413915 (0.0007) [2023-12-26 18:22:41,025][105692] Updated weights for policy 0, policy_version 413925 (0.0009) [2023-12-26 18:22:41,062][104569] Fps is (10 sec: 17203.6, 60 sec: 18705.1, 300 sec: 19299.8). Total num frames: 212041728. Throughput: 0: 9139.2, 1: 9185.8. Samples: 212049668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:22:41,062][104569] Avg episode reward: [(0, '9172.476'), (1, '9357.338')] [2023-12-26 18:22:41,125][105620] Updated weights for policy 1, policy_version 414265 (0.0009) [2023-12-26 18:22:41,187][105620] Updated weights for policy 1, policy_version 414275 (0.0009) [2023-12-26 18:22:41,254][105620] Updated weights for policy 1, policy_version 414285 (0.0009) [2023-12-26 18:22:41,888][105692] Updated weights for policy 0, policy_version 413935 (0.0009) [2023-12-26 18:22:41,956][105692] Updated weights for policy 0, policy_version 413945 (0.0008) [2023-12-26 18:22:42,028][105692] Updated weights for policy 0, policy_version 413955 (0.0009) [2023-12-26 18:22:42,094][105620] Updated weights for policy 1, policy_version 414295 (0.0010) [2023-12-26 18:22:42,153][105620] Updated weights for policy 1, policy_version 414305 (0.0010) [2023-12-26 18:22:42,223][105620] Updated weights for policy 1, policy_version 414315 (0.0012) [2023-12-26 18:22:42,802][105692] Updated weights for policy 0, policy_version 413965 (0.0009) [2023-12-26 18:22:42,865][105692] Updated weights for policy 0, policy_version 413975 (0.0010) [2023-12-26 18:22:42,934][105692] Updated weights for policy 0, policy_version 413985 (0.0009) [2023-12-26 18:22:43,013][105620] Updated weights for policy 1, policy_version 414325 (0.0010) [2023-12-26 18:22:43,076][105620] Updated weights for policy 1, policy_version 414335 (0.0009) [2023-12-26 18:22:43,139][105620] Updated weights for policy 1, policy_version 414345 (0.0009) [2023-12-26 18:22:43,661][105692] Updated weights for policy 0, policy_version 413995 (0.0007) [2023-12-26 18:22:43,725][105692] Updated weights for policy 0, policy_version 414005 (0.0010) [2023-12-26 18:22:43,790][105692] Updated weights for policy 0, policy_version 414015 (0.0009) [2023-12-26 18:22:43,950][105620] Updated weights for policy 1, policy_version 414355 (0.0008) [2023-12-26 18:22:44,013][105620] Updated weights for policy 1, policy_version 414365 (0.0008) [2023-12-26 18:22:44,072][105620] Updated weights for policy 1, policy_version 414375 (0.0008) [2023-12-26 18:22:44,507][105692] Updated weights for policy 0, policy_version 414025 (0.0007) [2023-12-26 18:22:44,574][105692] Updated weights for policy 0, policy_version 414035 (0.0011) [2023-12-26 18:22:44,638][105692] Updated weights for policy 0, policy_version 414045 (0.0011) [2023-12-26 18:22:44,704][105692] Updated weights for policy 0, policy_version 414055 (0.0010) [2023-12-26 18:22:44,871][105620] Updated weights for policy 1, policy_version 414385 (0.0007) [2023-12-26 18:22:44,939][105620] Updated weights for policy 1, policy_version 414395 (0.0009) [2023-12-26 18:22:45,006][105620] Updated weights for policy 1, policy_version 414405 (0.0010) [2023-12-26 18:22:45,084][105620] Updated weights for policy 1, policy_version 414415 (0.0010) [2023-12-26 18:22:45,374][105692] Updated weights for policy 0, policy_version 414065 (0.0008) [2023-12-26 18:22:45,447][105692] Updated weights for policy 0, policy_version 414075 (0.0008) [2023-12-26 18:22:45,511][105692] Updated weights for policy 0, policy_version 414085 (0.0008) [2023-12-26 18:22:45,876][105620] Updated weights for policy 1, policy_version 414425 (0.0007) [2023-12-26 18:22:45,933][105620] Updated weights for policy 1, policy_version 414435 (0.0008) [2023-12-26 18:22:45,992][105620] Updated weights for policy 1, policy_version 414445 (0.0007) [2023-12-26 18:22:46,062][104569] Fps is (10 sec: 18022.0, 60 sec: 18568.5, 300 sec: 19272.0). Total num frames: 212131840. Throughput: 0: 9035.4, 1: 9017.0. Samples: 212101552. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:22:46,063][104569] Avg episode reward: [(0, '9171.618'), (1, '9265.356')] [2023-12-26 18:22:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000414088_106020864.pth... [2023-12-26 18:22:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000414448_106110976.pth... [2023-12-26 18:22:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000413064_105758720.pth [2023-12-26 18:22:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000413424_105848832.pth [2023-12-26 18:22:46,235][105692] Updated weights for policy 0, policy_version 414095 (0.0009) [2023-12-26 18:22:46,294][105692] Updated weights for policy 0, policy_version 414105 (0.0009) [2023-12-26 18:22:46,342][105692] Updated weights for policy 0, policy_version 414115 (0.0009) [2023-12-26 18:22:46,719][105620] Updated weights for policy 1, policy_version 414455 (0.0007) [2023-12-26 18:22:46,781][105620] Updated weights for policy 1, policy_version 414465 (0.0005) [2023-12-26 18:22:46,841][105620] Updated weights for policy 1, policy_version 414475 (0.0007) [2023-12-26 18:22:47,123][105692] Updated weights for policy 0, policy_version 414125 (0.0009) [2023-12-26 18:22:47,183][105692] Updated weights for policy 0, policy_version 414135 (0.0009) [2023-12-26 18:22:47,251][105692] Updated weights for policy 0, policy_version 414145 (0.0009) [2023-12-26 18:22:47,465][105620] Updated weights for policy 1, policy_version 414485 (0.0007) [2023-12-26 18:22:47,524][105620] Updated weights for policy 1, policy_version 414495 (0.0006) [2023-12-26 18:22:47,579][105620] Updated weights for policy 1, policy_version 414505 (0.0009) [2023-12-26 18:22:48,105][105692] Updated weights for policy 0, policy_version 414155 (0.0009) [2023-12-26 18:22:48,175][105692] Updated weights for policy 0, policy_version 414165 (0.0009) [2023-12-26 18:22:48,240][105692] Updated weights for policy 0, policy_version 414175 (0.0009) [2023-12-26 18:22:48,267][105620] Updated weights for policy 1, policy_version 414515 (0.0009) [2023-12-26 18:22:48,337][105620] Updated weights for policy 1, policy_version 414525 (0.0010) [2023-12-26 18:22:48,413][105620] Updated weights for policy 1, policy_version 414535 (0.0011) [2023-12-26 18:22:49,114][105692] Updated weights for policy 0, policy_version 414185 (0.0007) [2023-12-26 18:22:49,179][105692] Updated weights for policy 0, policy_version 414195 (0.0009) [2023-12-26 18:22:49,245][105692] Updated weights for policy 0, policy_version 414205 (0.0008) [2023-12-26 18:22:49,271][105620] Updated weights for policy 1, policy_version 414545 (0.0009) [2023-12-26 18:22:49,314][105692] Updated weights for policy 0, policy_version 414215 (0.0008) [2023-12-26 18:22:49,332][105620] Updated weights for policy 1, policy_version 414555 (0.0007) [2023-12-26 18:22:49,407][105620] Updated weights for policy 1, policy_version 414565 (0.0008) [2023-12-26 18:22:49,469][105620] Updated weights for policy 1, policy_version 414575 (0.0008) [2023-12-26 18:22:50,249][105692] Updated weights for policy 0, policy_version 414225 (0.0008) [2023-12-26 18:22:50,315][105692] Updated weights for policy 0, policy_version 414235 (0.0012) [2023-12-26 18:22:50,317][105620] Updated weights for policy 1, policy_version 414585 (0.0008) [2023-12-26 18:22:50,387][105620] Updated weights for policy 1, policy_version 414595 (0.0008) [2023-12-26 18:22:50,389][105692] Updated weights for policy 0, policy_version 414245 (0.0010) [2023-12-26 18:22:50,462][105620] Updated weights for policy 1, policy_version 414605 (0.0008) [2023-12-26 18:22:51,062][104569] Fps is (10 sec: 17203.1, 60 sec: 18432.0, 300 sec: 19216.5). Total num frames: 212213760. Throughput: 0: 8909.2, 1: 8929.9. Samples: 212209544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:22:51,063][104569] Avg episode reward: [(0, '9263.532'), (1, '9265.320')] [2023-12-26 18:22:51,284][105692] Updated weights for policy 0, policy_version 414255 (0.0007) [2023-12-26 18:22:51,352][105692] Updated weights for policy 0, policy_version 414265 (0.0008) [2023-12-26 18:22:51,371][105620] Updated weights for policy 1, policy_version 414615 (0.0008) [2023-12-26 18:22:51,417][105692] Updated weights for policy 0, policy_version 414275 (0.0007) [2023-12-26 18:22:51,427][105620] Updated weights for policy 1, policy_version 414625 (0.0008) [2023-12-26 18:22:51,487][105620] Updated weights for policy 1, policy_version 414635 (0.0008) [2023-12-26 18:22:52,119][105692] Updated weights for policy 0, policy_version 414285 (0.0008) [2023-12-26 18:22:52,171][105692] Updated weights for policy 0, policy_version 414295 (0.0009) [2023-12-26 18:22:52,229][105692] Updated weights for policy 0, policy_version 414305 (0.0008) [2023-12-26 18:22:52,243][105620] Updated weights for policy 1, policy_version 414645 (0.0009) [2023-12-26 18:22:52,302][105620] Updated weights for policy 1, policy_version 414655 (0.0007) [2023-12-26 18:22:52,361][105620] Updated weights for policy 1, policy_version 414665 (0.0009) [2023-12-26 18:22:52,401][105586] KL-divergence is very high: 105.0614 [2023-12-26 18:22:53,047][105692] Updated weights for policy 0, policy_version 414315 (0.0008) [2023-12-26 18:22:53,059][105620] Updated weights for policy 1, policy_version 414675 (0.0009) [2023-12-26 18:22:53,104][105692] Updated weights for policy 0, policy_version 414325 (0.0009) [2023-12-26 18:22:53,118][105620] Updated weights for policy 1, policy_version 414685 (0.0008) [2023-12-26 18:22:53,163][105692] Updated weights for policy 0, policy_version 414335 (0.0006) [2023-12-26 18:22:53,176][105620] Updated weights for policy 1, policy_version 414695 (0.0007) [2023-12-26 18:22:53,732][105692] Updated weights for policy 0, policy_version 414345 (0.0006) [2023-12-26 18:22:53,784][105692] Updated weights for policy 0, policy_version 414355 (0.0006) [2023-12-26 18:22:53,829][105692] Updated weights for policy 0, policy_version 414365 (0.0005) [2023-12-26 18:22:53,873][105692] Updated weights for policy 0, policy_version 414375 (0.0008) [2023-12-26 18:22:53,995][105620] Updated weights for policy 1, policy_version 414705 (0.0009) [2023-12-26 18:22:54,047][105620] Updated weights for policy 1, policy_version 414715 (0.0008) [2023-12-26 18:22:54,091][105620] Updated weights for policy 1, policy_version 414725 (0.0008) [2023-12-26 18:22:54,153][105620] Updated weights for policy 1, policy_version 414735 (0.0008) [2023-12-26 18:22:54,605][105692] Updated weights for policy 0, policy_version 414385 (0.0011) [2023-12-26 18:22:54,670][105692] Updated weights for policy 0, policy_version 414395 (0.0011) [2023-12-26 18:22:54,735][105692] Updated weights for policy 0, policy_version 414405 (0.0010) [2023-12-26 18:22:54,961][105620] Updated weights for policy 1, policy_version 414745 (0.0008) [2023-12-26 18:22:55,019][105620] Updated weights for policy 1, policy_version 414755 (0.0008) [2023-12-26 18:22:55,080][105620] Updated weights for policy 1, policy_version 414765 (0.0009) [2023-12-26 18:22:55,473][105692] Updated weights for policy 0, policy_version 414415 (0.0007) [2023-12-26 18:22:55,538][105692] Updated weights for policy 0, policy_version 414425 (0.0005) [2023-12-26 18:22:55,586][105692] Updated weights for policy 0, policy_version 414435 (0.0005) [2023-12-26 18:22:55,821][105620] Updated weights for policy 1, policy_version 414775 (0.0006) [2023-12-26 18:22:55,877][105620] Updated weights for policy 1, policy_version 414785 (0.0007) [2023-12-26 18:22:55,939][105620] Updated weights for policy 1, policy_version 414795 (0.0008) [2023-12-26 18:22:56,062][104569] Fps is (10 sec: 18022.8, 60 sec: 18295.5, 300 sec: 19216.5). Total num frames: 212312064. Throughput: 0: 8856.6, 1: 8846.2. Samples: 212318720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:22:56,062][104569] Avg episode reward: [(0, '9355.569'), (1, '9173.407')] [2023-12-26 18:22:56,268][105692] Updated weights for policy 0, policy_version 414445 (0.0011) [2023-12-26 18:22:56,327][105692] Updated weights for policy 0, policy_version 414455 (0.0011) [2023-12-26 18:22:56,386][105692] Updated weights for policy 0, policy_version 414465 (0.0011) [2023-12-26 18:22:56,564][105620] Updated weights for policy 1, policy_version 414805 (0.0010) [2023-12-26 18:22:56,625][105620] Updated weights for policy 1, policy_version 414815 (0.0009) [2023-12-26 18:22:56,690][105620] Updated weights for policy 1, policy_version 414825 (0.0010) [2023-12-26 18:22:57,078][105692] Updated weights for policy 0, policy_version 414475 (0.0011) [2023-12-26 18:22:57,133][105692] Updated weights for policy 0, policy_version 414485 (0.0010) [2023-12-26 18:22:57,183][105692] Updated weights for policy 0, policy_version 414495 (0.0010) [2023-12-26 18:22:57,329][105620] Updated weights for policy 1, policy_version 414835 (0.0009) [2023-12-26 18:22:57,386][105620] Updated weights for policy 1, policy_version 414845 (0.0006) [2023-12-26 18:22:57,452][105620] Updated weights for policy 1, policy_version 414855 (0.0008) [2023-12-26 18:22:57,766][105692] Updated weights for policy 0, policy_version 414505 (0.0010) [2023-12-26 18:22:57,817][105692] Updated weights for policy 0, policy_version 414515 (0.0005) [2023-12-26 18:22:57,868][105692] Updated weights for policy 0, policy_version 414525 (0.0005) [2023-12-26 18:22:57,918][105692] Updated weights for policy 0, policy_version 414535 (0.0005) [2023-12-26 18:22:58,021][105620] Updated weights for policy 1, policy_version 414865 (0.0008) [2023-12-26 18:22:58,083][105620] Updated weights for policy 1, policy_version 414875 (0.0007) [2023-12-26 18:22:58,155][105620] Updated weights for policy 1, policy_version 414885 (0.0008) [2023-12-26 18:22:58,226][105620] Updated weights for policy 1, policy_version 414895 (0.0006) [2023-12-26 18:22:58,638][105692] Updated weights for policy 0, policy_version 414545 (0.0011) [2023-12-26 18:22:58,699][105692] Updated weights for policy 0, policy_version 414555 (0.0011) [2023-12-26 18:22:58,767][105692] Updated weights for policy 0, policy_version 414565 (0.0010) [2023-12-26 18:22:58,950][105620] Updated weights for policy 1, policy_version 414905 (0.0010) [2023-12-26 18:22:58,990][105586] KL-divergence is very high: 101.1769 [2023-12-26 18:22:59,001][105620] Updated weights for policy 1, policy_version 414915 (0.0010) [2023-12-26 18:22:59,036][105586] KL-divergence is very high: 104.7340 [2023-12-26 18:22:59,057][105620] Updated weights for policy 1, policy_version 414925 (0.0009) [2023-12-26 18:22:59,499][105692] Updated weights for policy 0, policy_version 414575 (0.0010) [2023-12-26 18:22:59,555][105692] Updated weights for policy 0, policy_version 414585 (0.0011) [2023-12-26 18:22:59,616][105692] Updated weights for policy 0, policy_version 414595 (0.0010) [2023-12-26 18:22:59,736][105620] Updated weights for policy 1, policy_version 414935 (0.0009) [2023-12-26 18:22:59,787][105620] Updated weights for policy 1, policy_version 414945 (0.0005) [2023-12-26 18:22:59,846][105620] Updated weights for policy 1, policy_version 414955 (0.0008) [2023-12-26 18:23:00,340][105692] Updated weights for policy 0, policy_version 414605 (0.0010) [2023-12-26 18:23:00,398][105692] Updated weights for policy 0, policy_version 414615 (0.0010) [2023-12-26 18:23:00,450][105620] Updated weights for policy 1, policy_version 414965 (0.0008) [2023-12-26 18:23:00,456][105692] Updated weights for policy 0, policy_version 414625 (0.0010) [2023-12-26 18:23:00,508][105620] Updated weights for policy 1, policy_version 414975 (0.0010) [2023-12-26 18:23:00,569][105620] Updated weights for policy 1, policy_version 414985 (0.0010) [2023-12-26 18:23:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 18158.9, 300 sec: 19216.5). Total num frames: 212410368. Throughput: 0: 8983.5, 1: 9010.0. Samples: 212381564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:23:01,063][104569] Avg episode reward: [(0, '9262.673'), (1, '8804.355')] [2023-12-26 18:23:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000414992_106250240.pth... [2023-12-26 18:23:01,073][105692] Updated weights for policy 0, policy_version 414635 (0.0010) [2023-12-26 18:23:01,081][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000413936_105979904.pth [2023-12-26 18:23:01,135][105692] Updated weights for policy 0, policy_version 414645 (0.0009) [2023-12-26 18:23:01,199][105692] Updated weights for policy 0, policy_version 414655 (0.0011) [2023-12-26 18:23:01,228][105620] Updated weights for policy 1, policy_version 414995 (0.0010) [2023-12-26 18:23:01,256][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000414664_106168320.pth... [2023-12-26 18:23:01,262][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000413576_105889792.pth [2023-12-26 18:23:01,295][105620] Updated weights for policy 1, policy_version 415005 (0.0008) [2023-12-26 18:23:01,363][105620] Updated weights for policy 1, policy_version 415015 (0.0011) [2023-12-26 18:23:01,922][105692] Updated weights for policy 0, policy_version 414665 (0.0009) [2023-12-26 18:23:01,977][105692] Updated weights for policy 0, policy_version 414675 (0.0009) [2023-12-26 18:23:02,028][105692] Updated weights for policy 0, policy_version 414685 (0.0009) [2023-12-26 18:23:02,037][105620] Updated weights for policy 1, policy_version 415025 (0.0006) [2023-12-26 18:23:02,081][105692] Updated weights for policy 0, policy_version 414695 (0.0009) [2023-12-26 18:23:02,093][105620] Updated weights for policy 1, policy_version 415035 (0.0005) [2023-12-26 18:23:02,147][105620] Updated weights for policy 1, policy_version 415045 (0.0005) [2023-12-26 18:23:02,198][105620] Updated weights for policy 1, policy_version 415055 (0.0005) [2023-12-26 18:23:02,875][105692] Updated weights for policy 0, policy_version 414705 (0.0007) [2023-12-26 18:23:02,893][105620] Updated weights for policy 1, policy_version 415065 (0.0008) [2023-12-26 18:23:02,927][105692] Updated weights for policy 0, policy_version 414715 (0.0007) [2023-12-26 18:23:02,945][105620] Updated weights for policy 1, policy_version 415075 (0.0009) [2023-12-26 18:23:02,979][105692] Updated weights for policy 0, policy_version 414725 (0.0007) [2023-12-26 18:23:02,994][105620] Updated weights for policy 1, policy_version 415085 (0.0008) [2023-12-26 18:23:03,585][105620] Updated weights for policy 1, policy_version 415095 (0.0006) [2023-12-26 18:23:03,630][105620] Updated weights for policy 1, policy_version 415105 (0.0005) [2023-12-26 18:23:03,686][105620] Updated weights for policy 1, policy_version 415115 (0.0005) [2023-12-26 18:23:03,688][105692] Updated weights for policy 0, policy_version 414735 (0.0005) [2023-12-26 18:23:03,738][105692] Updated weights for policy 0, policy_version 414745 (0.0005) [2023-12-26 18:23:03,781][105692] Updated weights for policy 0, policy_version 414755 (0.0005) [2023-12-26 18:23:04,259][105620] Updated weights for policy 1, policy_version 415125 (0.0007) [2023-12-26 18:23:04,322][105620] Updated weights for policy 1, policy_version 415135 (0.0009) [2023-12-26 18:23:04,374][105620] Updated weights for policy 1, policy_version 415145 (0.0009) [2023-12-26 18:23:04,540][105692] Updated weights for policy 0, policy_version 414765 (0.0006) [2023-12-26 18:23:04,600][105692] Updated weights for policy 0, policy_version 414775 (0.0009) [2023-12-26 18:23:04,659][105692] Updated weights for policy 0, policy_version 414785 (0.0008) [2023-12-26 18:23:05,010][105620] Updated weights for policy 1, policy_version 415155 (0.0009) [2023-12-26 18:23:05,073][105620] Updated weights for policy 1, policy_version 415165 (0.0009) [2023-12-26 18:23:05,125][105620] Updated weights for policy 1, policy_version 415175 (0.0008) [2023-12-26 18:23:05,407][105692] Updated weights for policy 0, policy_version 414795 (0.0008) [2023-12-26 18:23:05,457][105692] Updated weights for policy 0, policy_version 414805 (0.0005) [2023-12-26 18:23:05,507][105692] Updated weights for policy 0, policy_version 414815 (0.0005) [2023-12-26 18:23:05,916][105620] Updated weights for policy 1, policy_version 415185 (0.0009) [2023-12-26 18:23:05,972][105620] Updated weights for policy 1, policy_version 415195 (0.0005) [2023-12-26 18:23:06,028][105620] Updated weights for policy 1, policy_version 415205 (0.0006) [2023-12-26 18:23:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 18159.0, 300 sec: 19216.5). Total num frames: 212508672. Throughput: 0: 9059.0, 1: 9202.8. Samples: 212503696. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 18:23:06,062][104569] Avg episode reward: [(0, '9355.602'), (1, '8528.437')] [2023-12-26 18:23:06,080][105620] Updated weights for policy 1, policy_version 415215 (0.0005) [2023-12-26 18:23:06,165][105692] Updated weights for policy 0, policy_version 414825 (0.0006) [2023-12-26 18:23:06,230][105692] Updated weights for policy 0, policy_version 414835 (0.0010) [2023-12-26 18:23:06,296][105692] Updated weights for policy 0, policy_version 414845 (0.0006) [2023-12-26 18:23:06,362][105692] Updated weights for policy 0, policy_version 414855 (0.0005) [2023-12-26 18:23:06,690][105620] Updated weights for policy 1, policy_version 415225 (0.0011) [2023-12-26 18:23:06,749][105620] Updated weights for policy 1, policy_version 415235 (0.0011) [2023-12-26 18:23:06,812][105620] Updated weights for policy 1, policy_version 415245 (0.0010) [2023-12-26 18:23:07,052][105692] Updated weights for policy 0, policy_version 414865 (0.0011) [2023-12-26 18:23:07,117][105692] Updated weights for policy 0, policy_version 414875 (0.0011) [2023-12-26 18:23:07,169][105692] Updated weights for policy 0, policy_version 414885 (0.0010) [2023-12-26 18:23:07,413][105620] Updated weights for policy 1, policy_version 415255 (0.0011) [2023-12-26 18:23:07,476][105620] Updated weights for policy 1, policy_version 415265 (0.0008) [2023-12-26 18:23:07,530][105620] Updated weights for policy 1, policy_version 415275 (0.0005) [2023-12-26 18:23:07,960][105692] Updated weights for policy 0, policy_version 414895 (0.0009) [2023-12-26 18:23:08,020][105692] Updated weights for policy 0, policy_version 414905 (0.0010) [2023-12-26 18:23:08,088][105692] Updated weights for policy 0, policy_version 414915 (0.0010) [2023-12-26 18:23:08,180][105620] Updated weights for policy 1, policy_version 415285 (0.0009) [2023-12-26 18:23:08,231][105620] Updated weights for policy 1, policy_version 415295 (0.0009) [2023-12-26 18:23:08,294][105620] Updated weights for policy 1, policy_version 415305 (0.0009) [2023-12-26 18:23:08,907][105692] Updated weights for policy 0, policy_version 414925 (0.0009) [2023-12-26 18:23:08,962][105692] Updated weights for policy 0, policy_version 414935 (0.0009) [2023-12-26 18:23:09,004][105620] Updated weights for policy 1, policy_version 415315 (0.0007) [2023-12-26 18:23:09,021][105692] Updated weights for policy 0, policy_version 414945 (0.0010) [2023-12-26 18:23:09,055][105620] Updated weights for policy 1, policy_version 415325 (0.0007) [2023-12-26 18:23:09,102][105620] Updated weights for policy 1, policy_version 415335 (0.0009) [2023-12-26 18:23:09,754][105692] Updated weights for policy 0, policy_version 414955 (0.0010) [2023-12-26 18:23:09,817][105692] Updated weights for policy 0, policy_version 414965 (0.0009) [2023-12-26 18:23:09,886][105692] Updated weights for policy 0, policy_version 414975 (0.0007) [2023-12-26 18:23:09,945][105620] Updated weights for policy 1, policy_version 415345 (0.0009) [2023-12-26 18:23:10,014][105620] Updated weights for policy 1, policy_version 415355 (0.0009) [2023-12-26 18:23:10,067][105620] Updated weights for policy 1, policy_version 415365 (0.0009) [2023-12-26 18:23:10,118][105620] Updated weights for policy 1, policy_version 415375 (0.0008) [2023-12-26 18:23:10,610][105692] Updated weights for policy 0, policy_version 414985 (0.0008) [2023-12-26 18:23:10,657][105692] Updated weights for policy 0, policy_version 414995 (0.0008) [2023-12-26 18:23:10,716][105692] Updated weights for policy 0, policy_version 415005 (0.0009) [2023-12-26 18:23:10,775][105692] Updated weights for policy 0, policy_version 415015 (0.0009) [2023-12-26 18:23:10,860][105620] Updated weights for policy 1, policy_version 415385 (0.0009) [2023-12-26 18:23:10,907][105620] Updated weights for policy 1, policy_version 415395 (0.0008) [2023-12-26 18:23:10,962][105620] Updated weights for policy 1, policy_version 415405 (0.0008) [2023-12-26 18:23:11,062][104569] Fps is (10 sec: 20480.2, 60 sec: 18432.0, 300 sec: 19244.3). Total num frames: 212615168. Throughput: 0: 9177.3, 1: 9338.4. Samples: 212618848. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 18:23:11,062][104569] Avg episode reward: [(0, '6858.118'), (1, '8804.343')] [2023-12-26 18:23:11,656][105692] Updated weights for policy 0, policy_version 415025 (0.0009) [2023-12-26 18:23:11,716][105620] Updated weights for policy 1, policy_version 415415 (0.0009) [2023-12-26 18:23:11,718][105692] Updated weights for policy 0, policy_version 415035 (0.0007) [2023-12-26 18:23:11,785][105620] Updated weights for policy 1, policy_version 415425 (0.0008) [2023-12-26 18:23:11,786][105692] Updated weights for policy 0, policy_version 415045 (0.0008) [2023-12-26 18:23:11,849][105620] Updated weights for policy 1, policy_version 415435 (0.0009) [2023-12-26 18:23:12,478][105620] Updated weights for policy 1, policy_version 415445 (0.0007) [2023-12-26 18:23:12,536][105620] Updated weights for policy 1, policy_version 415455 (0.0007) [2023-12-26 18:23:12,598][105620] Updated weights for policy 1, policy_version 415465 (0.0006) [2023-12-26 18:23:12,612][105692] Updated weights for policy 0, policy_version 415055 (0.0007) [2023-12-26 18:23:12,672][105692] Updated weights for policy 0, policy_version 415065 (0.0009) [2023-12-26 18:23:12,728][105692] Updated weights for policy 0, policy_version 415075 (0.0009) [2023-12-26 18:23:13,181][105620] Updated weights for policy 1, policy_version 415475 (0.0007) [2023-12-26 18:23:13,237][105620] Updated weights for policy 1, policy_version 415485 (0.0006) [2023-12-26 18:23:13,287][105620] Updated weights for policy 1, policy_version 415495 (0.0007) [2023-12-26 18:23:13,565][105692] Updated weights for policy 0, policy_version 415085 (0.0009) [2023-12-26 18:23:13,631][105692] Updated weights for policy 0, policy_version 415095 (0.0009) [2023-12-26 18:23:13,693][105692] Updated weights for policy 0, policy_version 415105 (0.0009) [2023-12-26 18:23:13,966][105620] Updated weights for policy 1, policy_version 415505 (0.0008) [2023-12-26 18:23:14,013][105620] Updated weights for policy 1, policy_version 415515 (0.0008) [2023-12-26 18:23:14,068][105620] Updated weights for policy 1, policy_version 415525 (0.0009) [2023-12-26 18:23:14,127][105620] Updated weights for policy 1, policy_version 415535 (0.0009) [2023-12-26 18:23:14,503][105692] Updated weights for policy 0, policy_version 415115 (0.0009) [2023-12-26 18:23:14,556][105692] Updated weights for policy 0, policy_version 415125 (0.0008) [2023-12-26 18:23:14,607][105692] Updated weights for policy 0, policy_version 415135 (0.0008) [2023-12-26 18:23:14,809][105620] Updated weights for policy 1, policy_version 415545 (0.0008) [2023-12-26 18:23:14,869][105620] Updated weights for policy 1, policy_version 415555 (0.0010) [2023-12-26 18:23:14,932][105620] Updated weights for policy 1, policy_version 415565 (0.0009) [2023-12-26 18:23:15,381][105692] Updated weights for policy 0, policy_version 415145 (0.0009) [2023-12-26 18:23:15,432][105692] Updated weights for policy 0, policy_version 415155 (0.0009) [2023-12-26 18:23:15,479][105692] Updated weights for policy 0, policy_version 415165 (0.0009) [2023-12-26 18:23:15,538][105692] Updated weights for policy 0, policy_version 415175 (0.0009) [2023-12-26 18:23:15,716][105620] Updated weights for policy 1, policy_version 415575 (0.0009) [2023-12-26 18:23:15,767][105620] Updated weights for policy 1, policy_version 415585 (0.0009) [2023-12-26 18:23:15,820][105620] Updated weights for policy 1, policy_version 415595 (0.0009) [2023-12-26 18:23:16,062][104569] Fps is (10 sec: 19660.3, 60 sec: 18431.9, 300 sec: 19216.5). Total num frames: 212705280. Throughput: 0: 9128.6, 1: 9455.9. Samples: 212675484. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 18:23:16,063][104569] Avg episode reward: [(0, '1372.865'), (1, '8807.452')] [2023-12-26 18:23:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000415600_106405888.pth... [2023-12-26 18:23:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000415176_106299392.pth... [2023-12-26 18:23:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000414448_106110976.pth [2023-12-26 18:23:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000414088_106020864.pth [2023-12-26 18:23:16,255][105692] Updated weights for policy 0, policy_version 415185 (0.0008) [2023-12-26 18:23:16,311][105692] Updated weights for policy 0, policy_version 415195 (0.0009) [2023-12-26 18:23:16,369][105692] Updated weights for policy 0, policy_version 415205 (0.0009) [2023-12-26 18:23:16,581][105620] Updated weights for policy 1, policy_version 415605 (0.0007) [2023-12-26 18:23:16,648][105620] Updated weights for policy 1, policy_version 415615 (0.0005) [2023-12-26 18:23:16,708][105620] Updated weights for policy 1, policy_version 415625 (0.0005) [2023-12-26 18:23:17,201][105620] Updated weights for policy 1, policy_version 415635 (0.0005) [2023-12-26 18:23:17,254][105692] Updated weights for policy 0, policy_version 415216 (0.0010) [2023-12-26 18:23:17,259][105620] Updated weights for policy 1, policy_version 415645 (0.0005) [2023-12-26 18:23:17,299][105692] Updated weights for policy 0, policy_version 415226 (0.0007) [2023-12-26 18:23:17,316][105620] Updated weights for policy 1, policy_version 415655 (0.0008) [2023-12-26 18:23:17,360][105692] Updated weights for policy 0, policy_version 415236 (0.0007) [2023-12-26 18:23:17,962][105620] Updated weights for policy 1, policy_version 415665 (0.0010) [2023-12-26 18:23:18,011][105620] Updated weights for policy 1, policy_version 415675 (0.0010) [2023-12-26 18:23:18,063][105620] Updated weights for policy 1, policy_version 415685 (0.0010) [2023-12-26 18:23:18,111][105620] Updated weights for policy 1, policy_version 415695 (0.0010) [2023-12-26 18:23:18,158][105692] Updated weights for policy 0, policy_version 415246 (0.0008) [2023-12-26 18:23:18,207][105692] Updated weights for policy 0, policy_version 415256 (0.0007) [2023-12-26 18:23:18,258][105692] Updated weights for policy 0, policy_version 415266 (0.0008) [2023-12-26 18:23:18,854][105620] Updated weights for policy 1, policy_version 415705 (0.0009) [2023-12-26 18:23:18,925][105620] Updated weights for policy 1, policy_version 415715 (0.0006) [2023-12-26 18:23:18,990][105620] Updated weights for policy 1, policy_version 415725 (0.0007) [2023-12-26 18:23:19,086][105692] Updated weights for policy 0, policy_version 415276 (0.0007) [2023-12-26 18:23:19,138][105692] Updated weights for policy 0, policy_version 415286 (0.0009) [2023-12-26 18:23:19,194][105692] Updated weights for policy 0, policy_version 415296 (0.0008) [2023-12-26 18:23:19,707][105620] Updated weights for policy 1, policy_version 415735 (0.0008) [2023-12-26 18:23:19,764][105620] Updated weights for policy 1, policy_version 415745 (0.0007) [2023-12-26 18:23:19,833][105620] Updated weights for policy 1, policy_version 415755 (0.0009) [2023-12-26 18:23:20,042][105692] Updated weights for policy 0, policy_version 415306 (0.0009) [2023-12-26 18:23:20,094][105692] Updated weights for policy 0, policy_version 415316 (0.0006) [2023-12-26 18:23:20,167][105692] Updated weights for policy 0, policy_version 415326 (0.0007) [2023-12-26 18:23:20,242][105692] Updated weights for policy 0, policy_version 415336 (0.0008) [2023-12-26 18:23:20,484][105620] Updated weights for policy 1, policy_version 415765 (0.0008) [2023-12-26 18:23:20,547][105620] Updated weights for policy 1, policy_version 415775 (0.0009) [2023-12-26 18:23:20,607][105620] Updated weights for policy 1, policy_version 415785 (0.0008) [2023-12-26 18:23:20,963][105692] Updated weights for policy 0, policy_version 415346 (0.0008) [2023-12-26 18:23:21,030][105692] Updated weights for policy 0, policy_version 415356 (0.0009) [2023-12-26 18:23:21,062][104569] Fps is (10 sec: 18022.4, 60 sec: 18432.0, 300 sec: 19188.7). Total num frames: 212795392. Throughput: 0: 9144.3, 1: 9668.1. Samples: 212789400. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 18:23:21,063][104569] Avg episode reward: [(0, '1198.836'), (1, '8807.060')] [2023-12-26 18:23:21,096][105692] Updated weights for policy 0, policy_version 415366 (0.0008) [2023-12-26 18:23:21,321][105620] Updated weights for policy 1, policy_version 415795 (0.0009) [2023-12-26 18:23:21,389][105620] Updated weights for policy 1, policy_version 415805 (0.0009) [2023-12-26 18:23:21,450][105620] Updated weights for policy 1, policy_version 415815 (0.0009) [2023-12-26 18:23:21,897][105692] Updated weights for policy 0, policy_version 415376 (0.0011) [2023-12-26 18:23:21,960][105692] Updated weights for policy 0, policy_version 415386 (0.0011) [2023-12-26 18:23:22,026][105692] Updated weights for policy 0, policy_version 415396 (0.0011) [2023-12-26 18:23:22,128][105620] Updated weights for policy 1, policy_version 415825 (0.0009) [2023-12-26 18:23:22,186][105620] Updated weights for policy 1, policy_version 415835 (0.0007) [2023-12-26 18:23:22,236][105620] Updated weights for policy 1, policy_version 415845 (0.0008) [2023-12-26 18:23:22,297][105620] Updated weights for policy 1, policy_version 415855 (0.0008) [2023-12-26 18:23:22,742][105692] Updated weights for policy 0, policy_version 415406 (0.0011) [2023-12-26 18:23:22,794][105692] Updated weights for policy 0, policy_version 415416 (0.0011) [2023-12-26 18:23:22,846][105692] Updated weights for policy 0, policy_version 415426 (0.0011) [2023-12-26 18:23:23,072][105620] Updated weights for policy 1, policy_version 415865 (0.0010) [2023-12-26 18:23:23,132][105620] Updated weights for policy 1, policy_version 415875 (0.0010) [2023-12-26 18:23:23,186][105620] Updated weights for policy 1, policy_version 415885 (0.0008) [2023-12-26 18:23:23,611][105692] Updated weights for policy 0, policy_version 415436 (0.0010) [2023-12-26 18:23:23,662][105692] Updated weights for policy 0, policy_version 415446 (0.0010) [2023-12-26 18:23:23,713][105692] Updated weights for policy 0, policy_version 415456 (0.0010) [2023-12-26 18:23:23,764][105620] Updated weights for policy 1, policy_version 415895 (0.0009) [2023-12-26 18:23:23,815][105620] Updated weights for policy 1, policy_version 415905 (0.0010) [2023-12-26 18:23:23,863][105620] Updated weights for policy 1, policy_version 415915 (0.0010) [2023-12-26 18:23:24,407][105692] Updated weights for policy 0, policy_version 415466 (0.0010) [2023-12-26 18:23:24,442][105620] Updated weights for policy 1, policy_version 415925 (0.0007) [2023-12-26 18:23:24,466][105692] Updated weights for policy 0, policy_version 415476 (0.0010) [2023-12-26 18:23:24,480][105585] KL-divergence is very high: 144.1091 [2023-12-26 18:23:24,504][105620] Updated weights for policy 1, policy_version 415935 (0.0006) [2023-12-26 18:23:24,515][105585] KL-divergence is very high: 114.7396 [2023-12-26 18:23:24,521][105692] Updated weights for policy 0, policy_version 415486 (0.0011) [2023-12-26 18:23:24,527][105585] KL-divergence is very high: 154.8797 [2023-12-26 18:23:24,566][105620] Updated weights for policy 1, policy_version 415945 (0.0006) [2023-12-26 18:23:24,576][105692] Updated weights for policy 0, policy_version 415496 (0.0010) [2023-12-26 18:23:25,192][105620] Updated weights for policy 1, policy_version 415955 (0.0008) [2023-12-26 18:23:25,249][105620] Updated weights for policy 1, policy_version 415965 (0.0006) [2023-12-26 18:23:25,264][105692] Updated weights for policy 0, policy_version 415506 (0.0008) [2023-12-26 18:23:25,307][105620] Updated weights for policy 1, policy_version 415975 (0.0008) [2023-12-26 18:23:25,317][105692] Updated weights for policy 0, policy_version 415516 (0.0008) [2023-12-26 18:23:25,376][105692] Updated weights for policy 0, policy_version 415526 (0.0007) [2023-12-26 18:23:26,025][105620] Updated weights for policy 1, policy_version 415985 (0.0008) [2023-12-26 18:23:26,057][105692] Updated weights for policy 0, policy_version 415536 (0.0006) [2023-12-26 18:23:26,062][104569] Fps is (10 sec: 18842.0, 60 sec: 18705.1, 300 sec: 19188.7). Total num frames: 212893696. Throughput: 0: 9237.4, 1: 9849.2. Samples: 212908568. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 18:23:26,062][104569] Avg episode reward: [(0, '1530.312'), (1, '8805.095')] [2023-12-26 18:23:26,085][105620] Updated weights for policy 1, policy_version 415995 (0.0009) [2023-12-26 18:23:26,116][105692] Updated weights for policy 0, policy_version 415546 (0.0008) [2023-12-26 18:23:26,134][105620] Updated weights for policy 1, policy_version 416005 (0.0007) [2023-12-26 18:23:26,175][105692] Updated weights for policy 0, policy_version 415556 (0.0006) [2023-12-26 18:23:26,192][105620] Updated weights for policy 1, policy_version 416015 (0.0009) [2023-12-26 18:23:26,890][105692] Updated weights for policy 0, policy_version 415566 (0.0008) [2023-12-26 18:23:26,935][105692] Updated weights for policy 0, policy_version 415576 (0.0007) [2023-12-26 18:23:26,942][105620] Updated weights for policy 1, policy_version 416025 (0.0008) [2023-12-26 18:23:26,990][105692] Updated weights for policy 0, policy_version 415586 (0.0005) [2023-12-26 18:23:26,999][105620] Updated weights for policy 1, policy_version 416035 (0.0009) [2023-12-26 18:23:27,049][105620] Updated weights for policy 1, policy_version 416045 (0.0010) [2023-12-26 18:23:27,750][105692] Updated weights for policy 0, policy_version 415596 (0.0007) [2023-12-26 18:23:27,794][105692] Updated weights for policy 0, policy_version 415606 (0.0009) [2023-12-26 18:23:27,800][105620] Updated weights for policy 1, policy_version 416055 (0.0010) [2023-12-26 18:23:27,839][105692] Updated weights for policy 0, policy_version 415616 (0.0005) [2023-12-26 18:23:27,844][105620] Updated weights for policy 1, policy_version 416065 (0.0010) [2023-12-26 18:23:27,908][105620] Updated weights for policy 1, policy_version 416075 (0.0010) [2023-12-26 18:23:28,618][105692] Updated weights for policy 0, policy_version 415626 (0.0009) [2023-12-26 18:23:28,651][105620] Updated weights for policy 1, policy_version 416085 (0.0007) [2023-12-26 18:23:28,672][105692] Updated weights for policy 0, policy_version 415636 (0.0009) [2023-12-26 18:23:28,698][105620] Updated weights for policy 1, policy_version 416095 (0.0006) [2023-12-26 18:23:28,728][105692] Updated weights for policy 0, policy_version 415646 (0.0007) [2023-12-26 18:23:28,754][105620] Updated weights for policy 1, policy_version 416105 (0.0005) [2023-12-26 18:23:28,783][105692] Updated weights for policy 0, policy_version 415656 (0.0005) [2023-12-26 18:23:29,404][105620] Updated weights for policy 1, policy_version 416115 (0.0007) [2023-12-26 18:23:29,428][105692] Updated weights for policy 0, policy_version 415666 (0.0006) [2023-12-26 18:23:29,458][105620] Updated weights for policy 1, policy_version 416125 (0.0008) [2023-12-26 18:23:29,482][105692] Updated weights for policy 0, policy_version 415676 (0.0007) [2023-12-26 18:23:29,505][105620] Updated weights for policy 1, policy_version 416135 (0.0008) [2023-12-26 18:23:29,541][105692] Updated weights for policy 0, policy_version 415686 (0.0007) [2023-12-26 18:23:30,152][105620] Updated weights for policy 1, policy_version 416145 (0.0008) [2023-12-26 18:23:30,209][105620] Updated weights for policy 1, policy_version 416155 (0.0005) [2023-12-26 18:23:30,255][105620] Updated weights for policy 1, policy_version 416165 (0.0005) [2023-12-26 18:23:30,302][105620] Updated weights for policy 1, policy_version 416175 (0.0005) [2023-12-26 18:23:30,374][105692] Updated weights for policy 0, policy_version 415696 (0.0009) [2023-12-26 18:23:30,421][105692] Updated weights for policy 0, policy_version 415706 (0.0009) [2023-12-26 18:23:30,467][105692] Updated weights for policy 0, policy_version 415716 (0.0008) [2023-12-26 18:23:30,937][105586] KL-divergence is very high: 317.0071 [2023-12-26 18:23:30,941][105620] Updated weights for policy 1, policy_version 416185 (0.0005) [2023-12-26 18:23:30,955][105586] KL-divergence is very high: 130.9911 [2023-12-26 18:23:30,980][105586] KL-divergence is very high: 475.6166 [2023-12-26 18:23:30,994][105620] Updated weights for policy 1, policy_version 416195 (0.0005) [2023-12-26 18:23:31,002][105586] KL-divergence is very high: 149.1977 [2023-12-26 18:23:31,025][105586] KL-divergence is very high: 582.5558 [2023-12-26 18:23:31,053][105586] KL-divergence is very high: 148.6557 [2023-12-26 18:23:31,060][105620] Updated weights for policy 1, policy_version 416205 (0.0007) [2023-12-26 18:23:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 18705.1, 300 sec: 19216.5). Total num frames: 212992000. Throughput: 0: 9295.3, 1: 9905.4. Samples: 212965580. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 18:23:31,062][104569] Avg episode reward: [(0, '2078.397'), (1, '8715.546')] [2023-12-26 18:23:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000415720_106438656.pth... [2023-12-26 18:23:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000414664_106168320.pth [2023-12-26 18:23:31,077][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000416208_106561536.pth... [2023-12-26 18:23:31,082][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000414992_106250240.pth [2023-12-26 18:23:31,263][105692] Updated weights for policy 0, policy_version 415726 (0.0009) [2023-12-26 18:23:31,325][105692] Updated weights for policy 0, policy_version 415736 (0.0008) [2023-12-26 18:23:31,370][105692] Updated weights for policy 0, policy_version 415746 (0.0008) [2023-12-26 18:23:31,761][105620] Updated weights for policy 1, policy_version 416215 (0.0008) [2023-12-26 18:23:31,818][105620] Updated weights for policy 1, policy_version 416225 (0.0008) [2023-12-26 18:23:31,864][105620] Updated weights for policy 1, policy_version 416235 (0.0008) [2023-12-26 18:23:32,180][105692] Updated weights for policy 0, policy_version 415756 (0.0008) [2023-12-26 18:23:32,229][105692] Updated weights for policy 0, policy_version 415766 (0.0008) [2023-12-26 18:23:32,285][105692] Updated weights for policy 0, policy_version 415776 (0.0008) [2023-12-26 18:23:32,528][105620] Updated weights for policy 1, policy_version 416245 (0.0007) [2023-12-26 18:23:32,588][105620] Updated weights for policy 1, policy_version 416255 (0.0009) [2023-12-26 18:23:32,643][105620] Updated weights for policy 1, policy_version 416265 (0.0005) [2023-12-26 18:23:33,030][105692] Updated weights for policy 0, policy_version 415786 (0.0007) [2023-12-26 18:23:33,078][105692] Updated weights for policy 0, policy_version 415796 (0.0005) [2023-12-26 18:23:33,125][105692] Updated weights for policy 0, policy_version 415806 (0.0005) [2023-12-26 18:23:33,174][105692] Updated weights for policy 0, policy_version 415816 (0.0010) [2023-12-26 18:23:33,327][105620] Updated weights for policy 1, policy_version 416275 (0.0007) [2023-12-26 18:23:33,378][105620] Updated weights for policy 1, policy_version 416285 (0.0010) [2023-12-26 18:23:33,428][105620] Updated weights for policy 1, policy_version 416295 (0.0010) [2023-12-26 18:23:33,828][105692] Updated weights for policy 0, policy_version 415826 (0.0005) [2023-12-26 18:23:33,872][105692] Updated weights for policy 0, policy_version 415836 (0.0005) [2023-12-26 18:23:33,916][105692] Updated weights for policy 0, policy_version 415846 (0.0005) [2023-12-26 18:23:34,180][105620] Updated weights for policy 1, policy_version 416305 (0.0009) [2023-12-26 18:23:34,227][105620] Updated weights for policy 1, policy_version 416315 (0.0006) [2023-12-26 18:23:34,284][105620] Updated weights for policy 1, policy_version 416325 (0.0008) [2023-12-26 18:23:34,338][105620] Updated weights for policy 1, policy_version 416335 (0.0007) [2023-12-26 18:23:34,575][105692] Updated weights for policy 0, policy_version 415856 (0.0009) [2023-12-26 18:23:34,628][105692] Updated weights for policy 0, policy_version 415866 (0.0010) [2023-12-26 18:23:34,690][105692] Updated weights for policy 0, policy_version 415876 (0.0010) [2023-12-26 18:23:34,923][105620] Updated weights for policy 1, policy_version 416345 (0.0005) [2023-12-26 18:23:34,977][105620] Updated weights for policy 1, policy_version 416355 (0.0007) [2023-12-26 18:23:35,029][105620] Updated weights for policy 1, policy_version 416365 (0.0007) [2023-12-26 18:23:35,434][105692] Updated weights for policy 0, policy_version 415886 (0.0010) [2023-12-26 18:23:35,499][105692] Updated weights for policy 0, policy_version 415896 (0.0010) [2023-12-26 18:23:35,557][105692] Updated weights for policy 0, policy_version 415906 (0.0010) [2023-12-26 18:23:35,613][105620] Updated weights for policy 1, policy_version 416375 (0.0006) [2023-12-26 18:23:35,672][105620] Updated weights for policy 1, policy_version 416385 (0.0008) [2023-12-26 18:23:35,732][105620] Updated weights for policy 1, policy_version 416395 (0.0008) [2023-12-26 18:23:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19114.7, 300 sec: 19244.3). Total num frames: 213098496. Throughput: 0: 9406.7, 1: 10099.9. Samples: 213087340. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 18:23:36,062][104569] Avg episode reward: [(0, '2233.783'), (1, '8807.175')] [2023-12-26 18:23:36,289][105692] Updated weights for policy 0, policy_version 415916 (0.0010) [2023-12-26 18:23:36,356][105692] Updated weights for policy 0, policy_version 415926 (0.0009) [2023-12-26 18:23:36,420][105692] Updated weights for policy 0, policy_version 415936 (0.0009) [2023-12-26 18:23:36,471][105620] Updated weights for policy 1, policy_version 416405 (0.0007) [2023-12-26 18:23:36,542][105620] Updated weights for policy 1, policy_version 416415 (0.0010) [2023-12-26 18:23:36,614][105620] Updated weights for policy 1, policy_version 416425 (0.0008) [2023-12-26 18:23:37,033][105692] Updated weights for policy 0, policy_version 415946 (0.0006) [2023-12-26 18:23:37,081][105692] Updated weights for policy 0, policy_version 415956 (0.0006) [2023-12-26 18:23:37,137][105692] Updated weights for policy 0, policy_version 415966 (0.0005) [2023-12-26 18:23:37,184][105692] Updated weights for policy 0, policy_version 415976 (0.0005) [2023-12-26 18:23:37,301][105620] Updated weights for policy 1, policy_version 416435 (0.0010) [2023-12-26 18:23:37,359][105620] Updated weights for policy 1, policy_version 416445 (0.0010) [2023-12-26 18:23:37,422][105620] Updated weights for policy 1, policy_version 416455 (0.0011) [2023-12-26 18:23:37,772][105692] Updated weights for policy 0, policy_version 415986 (0.0007) [2023-12-26 18:23:37,841][105692] Updated weights for policy 0, policy_version 415996 (0.0009) [2023-12-26 18:23:37,908][105692] Updated weights for policy 0, policy_version 416006 (0.0009) [2023-12-26 18:23:38,097][105620] Updated weights for policy 1, policy_version 416465 (0.0010) [2023-12-26 18:23:38,157][105620] Updated weights for policy 1, policy_version 416475 (0.0009) [2023-12-26 18:23:38,218][105620] Updated weights for policy 1, policy_version 416485 (0.0009) [2023-12-26 18:23:38,279][105620] Updated weights for policy 1, policy_version 416495 (0.0008) [2023-12-26 18:23:38,670][105692] Updated weights for policy 0, policy_version 416016 (0.0009) [2023-12-26 18:23:38,722][105692] Updated weights for policy 0, policy_version 416026 (0.0007) [2023-12-26 18:23:38,771][105692] Updated weights for policy 0, policy_version 416036 (0.0006) [2023-12-26 18:23:39,049][105620] Updated weights for policy 1, policy_version 416505 (0.0011) [2023-12-26 18:23:39,106][105620] Updated weights for policy 1, policy_version 416515 (0.0011) [2023-12-26 18:23:39,162][105620] Updated weights for policy 1, policy_version 416525 (0.0011) [2023-12-26 18:23:39,414][105692] Updated weights for policy 0, policy_version 416046 (0.0007) [2023-12-26 18:23:39,482][105692] Updated weights for policy 0, policy_version 416056 (0.0008) [2023-12-26 18:23:39,539][105692] Updated weights for policy 0, policy_version 416066 (0.0008) [2023-12-26 18:23:39,894][105620] Updated weights for policy 1, policy_version 416535 (0.0011) [2023-12-26 18:23:39,959][105620] Updated weights for policy 1, policy_version 416545 (0.0011) [2023-12-26 18:23:40,023][105620] Updated weights for policy 1, policy_version 416555 (0.0009) [2023-12-26 18:23:40,330][105692] Updated weights for policy 0, policy_version 416076 (0.0007) [2023-12-26 18:23:40,395][105692] Updated weights for policy 0, policy_version 416086 (0.0008) [2023-12-26 18:23:40,452][105692] Updated weights for policy 0, policy_version 416096 (0.0011) [2023-12-26 18:23:40,754][105620] Updated weights for policy 1, policy_version 416565 (0.0008) [2023-12-26 18:23:40,817][105620] Updated weights for policy 1, policy_version 416575 (0.0010) [2023-12-26 18:23:40,879][105620] Updated weights for policy 1, policy_version 416585 (0.0010) [2023-12-26 18:23:41,055][105692] Updated weights for policy 0, policy_version 416106 (0.0008) [2023-12-26 18:23:41,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19251.2, 300 sec: 19244.3). Total num frames: 213196800. Throughput: 0: 9499.8, 1: 10200.2. Samples: 213205224. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 18:23:41,063][104569] Avg episode reward: [(0, '4744.177'), (1, '8807.181')] [2023-12-26 18:23:41,114][105692] Updated weights for policy 0, policy_version 416116 (0.0010) [2023-12-26 18:23:41,180][105692] Updated weights for policy 0, policy_version 416126 (0.0011) [2023-12-26 18:23:41,235][105692] Updated weights for policy 0, policy_version 416136 (0.0010) [2023-12-26 18:23:41,654][105620] Updated weights for policy 1, policy_version 416595 (0.0010) [2023-12-26 18:23:41,730][105620] Updated weights for policy 1, policy_version 416605 (0.0011) [2023-12-26 18:23:41,797][105620] Updated weights for policy 1, policy_version 416615 (0.0010) [2023-12-26 18:23:42,000][105692] Updated weights for policy 0, policy_version 416146 (0.0011) [2023-12-26 18:23:42,050][105692] Updated weights for policy 0, policy_version 416156 (0.0009) [2023-12-26 18:23:42,098][105692] Updated weights for policy 0, policy_version 416166 (0.0008) [2023-12-26 18:23:42,508][105620] Updated weights for policy 1, policy_version 416625 (0.0010) [2023-12-26 18:23:42,570][105620] Updated weights for policy 1, policy_version 416635 (0.0006) [2023-12-26 18:23:42,628][105620] Updated weights for policy 1, policy_version 416645 (0.0005) [2023-12-26 18:23:42,690][105620] Updated weights for policy 1, policy_version 416655 (0.0005) [2023-12-26 18:23:42,873][105692] Updated weights for policy 0, policy_version 416176 (0.0008) [2023-12-26 18:23:42,924][105692] Updated weights for policy 0, policy_version 416186 (0.0010) [2023-12-26 18:23:42,977][105692] Updated weights for policy 0, policy_version 416196 (0.0010) [2023-12-26 18:23:43,332][105620] Updated weights for policy 1, policy_version 416665 (0.0009) [2023-12-26 18:23:43,377][105620] Updated weights for policy 1, policy_version 416675 (0.0008) [2023-12-26 18:23:43,426][105620] Updated weights for policy 1, policy_version 416685 (0.0008) [2023-12-26 18:23:43,611][105692] Updated weights for policy 0, policy_version 416206 (0.0006) [2023-12-26 18:23:43,669][105692] Updated weights for policy 0, policy_version 416216 (0.0006) [2023-12-26 18:23:43,732][105692] Updated weights for policy 0, policy_version 416226 (0.0011) [2023-12-26 18:23:44,284][105692] Updated weights for policy 0, policy_version 416236 (0.0006) [2023-12-26 18:23:44,308][105620] Updated weights for policy 1, policy_version 416695 (0.0010) [2023-12-26 18:23:44,337][105692] Updated weights for policy 0, policy_version 416246 (0.0005) [2023-12-26 18:23:44,359][105620] Updated weights for policy 1, policy_version 416705 (0.0009) [2023-12-26 18:23:44,385][105692] Updated weights for policy 0, policy_version 416256 (0.0008) [2023-12-26 18:23:44,411][105620] Updated weights for policy 1, policy_version 416715 (0.0005) [2023-12-26 18:23:44,981][105692] Updated weights for policy 0, policy_version 416266 (0.0009) [2023-12-26 18:23:45,038][105692] Updated weights for policy 0, policy_version 416276 (0.0009) [2023-12-26 18:23:45,090][105692] Updated weights for policy 0, policy_version 416286 (0.0011) [2023-12-26 18:23:45,148][105692] Updated weights for policy 0, policy_version 416296 (0.0010) [2023-12-26 18:23:45,231][105620] Updated weights for policy 1, policy_version 416725 (0.0006) [2023-12-26 18:23:45,300][105620] Updated weights for policy 1, policy_version 416735 (0.0007) [2023-12-26 18:23:45,363][105620] Updated weights for policy 1, policy_version 416745 (0.0008) [2023-12-26 18:23:45,733][105692] Updated weights for policy 0, policy_version 416306 (0.0005) [2023-12-26 18:23:45,780][105692] Updated weights for policy 0, policy_version 416316 (0.0006) [2023-12-26 18:23:45,836][105692] Updated weights for policy 0, policy_version 416326 (0.0005) [2023-12-26 18:23:46,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19387.7, 300 sec: 19244.2). Total num frames: 213295104. Throughput: 0: 9468.0, 1: 10130.0. Samples: 213263476. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 18:23:46,063][104569] Avg episode reward: [(0, '8438.512'), (1, '9172.222')] [2023-12-26 18:23:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000416752_106700800.pth... [2023-12-26 18:23:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000416328_106594304.pth... [2023-12-26 18:23:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000415600_106405888.pth [2023-12-26 18:23:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000415176_106299392.pth [2023-12-26 18:23:46,143][105620] Updated weights for policy 1, policy_version 416755 (0.0009) [2023-12-26 18:23:46,195][105620] Updated weights for policy 1, policy_version 416765 (0.0008) [2023-12-26 18:23:46,252][105620] Updated weights for policy 1, policy_version 416775 (0.0009) [2023-12-26 18:23:46,537][105692] Updated weights for policy 0, policy_version 416336 (0.0010) [2023-12-26 18:23:46,593][105692] Updated weights for policy 0, policy_version 416346 (0.0010) [2023-12-26 18:23:46,643][105692] Updated weights for policy 0, policy_version 416356 (0.0010) [2023-12-26 18:23:47,010][105620] Updated weights for policy 1, policy_version 416785 (0.0008) [2023-12-26 18:23:47,060][105620] Updated weights for policy 1, policy_version 416795 (0.0009) [2023-12-26 18:23:47,109][105620] Updated weights for policy 1, policy_version 416805 (0.0008) [2023-12-26 18:23:47,158][105620] Updated weights for policy 1, policy_version 416815 (0.0008) [2023-12-26 18:23:47,415][105692] Updated weights for policy 0, policy_version 416366 (0.0010) [2023-12-26 18:23:47,475][105692] Updated weights for policy 0, policy_version 416376 (0.0006) [2023-12-26 18:23:47,548][105692] Updated weights for policy 0, policy_version 416386 (0.0006) [2023-12-26 18:23:47,778][105620] Updated weights for policy 1, policy_version 416825 (0.0006) [2023-12-26 18:23:47,836][105620] Updated weights for policy 1, policy_version 416835 (0.0009) [2023-12-26 18:23:47,895][105620] Updated weights for policy 1, policy_version 416845 (0.0005) [2023-12-26 18:23:48,336][105692] Updated weights for policy 0, policy_version 416396 (0.0010) [2023-12-26 18:23:48,402][105692] Updated weights for policy 0, policy_version 416406 (0.0009) [2023-12-26 18:23:48,462][105692] Updated weights for policy 0, policy_version 416416 (0.0009) [2023-12-26 18:23:48,482][105620] Updated weights for policy 1, policy_version 416855 (0.0007) [2023-12-26 18:23:48,544][105620] Updated weights for policy 1, policy_version 416865 (0.0008) [2023-12-26 18:23:48,605][105620] Updated weights for policy 1, policy_version 416875 (0.0008) [2023-12-26 18:23:49,206][105692] Updated weights for policy 0, policy_version 416426 (0.0008) [2023-12-26 18:23:49,272][105692] Updated weights for policy 0, policy_version 416436 (0.0009) [2023-12-26 18:23:49,341][105692] Updated weights for policy 0, policy_version 416446 (0.0007) [2023-12-26 18:23:49,360][105620] Updated weights for policy 1, policy_version 416885 (0.0008) [2023-12-26 18:23:49,409][105692] Updated weights for policy 0, policy_version 416456 (0.0008) [2023-12-26 18:23:49,427][105620] Updated weights for policy 1, policy_version 416895 (0.0007) [2023-12-26 18:23:49,493][105620] Updated weights for policy 1, policy_version 416905 (0.0009) [2023-12-26 18:23:50,176][105692] Updated weights for policy 0, policy_version 416466 (0.0010) [2023-12-26 18:23:50,192][105620] Updated weights for policy 1, policy_version 416915 (0.0008) [2023-12-26 18:23:50,241][105692] Updated weights for policy 0, policy_version 416476 (0.0009) [2023-12-26 18:23:50,249][105620] Updated weights for policy 1, policy_version 416925 (0.0006) [2023-12-26 18:23:50,301][105620] Updated weights for policy 1, policy_version 416935 (0.0005) [2023-12-26 18:23:50,306][105692] Updated weights for policy 0, policy_version 416486 (0.0010) [2023-12-26 18:23:50,977][105692] Updated weights for policy 0, policy_version 416496 (0.0010) [2023-12-26 18:23:50,994][105620] Updated weights for policy 1, policy_version 416945 (0.0009) [2023-12-26 18:23:51,046][105692] Updated weights for policy 0, policy_version 416507 (0.0010) [2023-12-26 18:23:51,058][105620] Updated weights for policy 1, policy_version 416955 (0.0009) [2023-12-26 18:23:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19216.5). Total num frames: 213385216. Throughput: 0: 9524.2, 1: 9987.5. Samples: 213381720. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 18:23:51,062][104569] Avg episode reward: [(0, '9266.004'), (1, '8896.966')] [2023-12-26 18:23:51,112][105692] Updated weights for policy 0, policy_version 416518 (0.0007) [2023-12-26 18:23:51,129][105620] Updated weights for policy 1, policy_version 416965 (0.0007) [2023-12-26 18:23:51,187][105620] Updated weights for policy 1, policy_version 416975 (0.0006) [2023-12-26 18:23:51,852][105692] Updated weights for policy 0, policy_version 416528 (0.0010) [2023-12-26 18:23:51,888][105620] Updated weights for policy 1, policy_version 416985 (0.0009) [2023-12-26 18:23:51,916][105692] Updated weights for policy 0, policy_version 416538 (0.0007) [2023-12-26 18:23:51,951][105620] Updated weights for policy 1, policy_version 416995 (0.0005) [2023-12-26 18:23:51,973][105692] Updated weights for policy 0, policy_version 416548 (0.0010) [2023-12-26 18:23:52,005][105620] Updated weights for policy 1, policy_version 417005 (0.0008) [2023-12-26 18:23:52,677][105692] Updated weights for policy 0, policy_version 416558 (0.0009) [2023-12-26 18:23:52,735][105692] Updated weights for policy 0, policy_version 416568 (0.0009) [2023-12-26 18:23:52,742][105620] Updated weights for policy 1, policy_version 417015 (0.0010) [2023-12-26 18:23:52,795][105692] Updated weights for policy 0, policy_version 416578 (0.0006) [2023-12-26 18:23:52,800][105620] Updated weights for policy 1, policy_version 417025 (0.0010) [2023-12-26 18:23:52,863][105620] Updated weights for policy 1, policy_version 417035 (0.0010) [2023-12-26 18:23:53,378][105692] Updated weights for policy 0, policy_version 416588 (0.0007) [2023-12-26 18:23:53,424][105692] Updated weights for policy 0, policy_version 416598 (0.0008) [2023-12-26 18:23:53,465][105692] Updated weights for policy 0, policy_version 416608 (0.0006) [2023-12-26 18:23:53,585][105620] Updated weights for policy 1, policy_version 417045 (0.0010) [2023-12-26 18:23:53,632][105620] Updated weights for policy 1, policy_version 417055 (0.0010) [2023-12-26 18:23:53,684][105620] Updated weights for policy 1, policy_version 417065 (0.0010) [2023-12-26 18:23:54,207][105692] Updated weights for policy 0, policy_version 416618 (0.0008) [2023-12-26 18:23:54,267][105692] Updated weights for policy 0, policy_version 416628 (0.0010) [2023-12-26 18:23:54,331][105692] Updated weights for policy 0, policy_version 416638 (0.0010) [2023-12-26 18:23:54,392][105692] Updated weights for policy 0, policy_version 416648 (0.0010) [2023-12-26 18:23:54,428][105620] Updated weights for policy 1, policy_version 417075 (0.0010) [2023-12-26 18:23:54,483][105620] Updated weights for policy 1, policy_version 417085 (0.0010) [2023-12-26 18:23:54,541][105620] Updated weights for policy 1, policy_version 417095 (0.0010) [2023-12-26 18:23:55,096][105692] Updated weights for policy 0, policy_version 416658 (0.0006) [2023-12-26 18:23:55,149][105692] Updated weights for policy 0, policy_version 416668 (0.0005) [2023-12-26 18:23:55,201][105692] Updated weights for policy 0, policy_version 416678 (0.0005) [2023-12-26 18:23:55,285][105620] Updated weights for policy 1, policy_version 417105 (0.0009) [2023-12-26 18:23:55,336][105620] Updated weights for policy 1, policy_version 417115 (0.0009) [2023-12-26 18:23:55,401][105620] Updated weights for policy 1, policy_version 417125 (0.0007) [2023-12-26 18:23:55,463][105620] Updated weights for policy 1, policy_version 417135 (0.0007) [2023-12-26 18:23:55,804][105692] Updated weights for policy 0, policy_version 416688 (0.0008) [2023-12-26 18:23:55,850][105692] Updated weights for policy 0, policy_version 416698 (0.0009) [2023-12-26 18:23:55,896][105692] Updated weights for policy 0, policy_version 416708 (0.0008) [2023-12-26 18:23:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.8, 300 sec: 19244.3). Total num frames: 213491712. Throughput: 0: 9606.8, 1: 9978.8. Samples: 213500204. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 18:23:56,063][104569] Avg episode reward: [(0, '9267.134'), (1, '9081.286')] [2023-12-26 18:23:56,181][105620] Updated weights for policy 1, policy_version 417145 (0.0009) [2023-12-26 18:23:56,247][105620] Updated weights for policy 1, policy_version 417155 (0.0009) [2023-12-26 18:23:56,305][105620] Updated weights for policy 1, policy_version 417165 (0.0009) [2023-12-26 18:23:56,643][105692] Updated weights for policy 0, policy_version 416718 (0.0010) [2023-12-26 18:23:56,697][105692] Updated weights for policy 0, policy_version 416728 (0.0010) [2023-12-26 18:23:56,747][105692] Updated weights for policy 0, policy_version 416738 (0.0010) [2023-12-26 18:23:56,986][105620] Updated weights for policy 1, policy_version 417175 (0.0008) [2023-12-26 18:23:57,036][105620] Updated weights for policy 1, policy_version 417185 (0.0008) [2023-12-26 18:23:57,094][105620] Updated weights for policy 1, policy_version 417195 (0.0009) [2023-12-26 18:23:57,497][105692] Updated weights for policy 0, policy_version 416748 (0.0008) [2023-12-26 18:23:57,560][105692] Updated weights for policy 0, policy_version 416758 (0.0006) [2023-12-26 18:23:57,615][105692] Updated weights for policy 0, policy_version 416768 (0.0008) [2023-12-26 18:23:57,835][105620] Updated weights for policy 1, policy_version 417205 (0.0008) [2023-12-26 18:23:57,880][105620] Updated weights for policy 1, policy_version 417215 (0.0008) [2023-12-26 18:23:57,927][105620] Updated weights for policy 1, policy_version 417225 (0.0008) [2023-12-26 18:23:58,365][105692] Updated weights for policy 0, policy_version 416778 (0.0009) [2023-12-26 18:23:58,426][105692] Updated weights for policy 0, policy_version 416788 (0.0009) [2023-12-26 18:23:58,490][105692] Updated weights for policy 0, policy_version 416798 (0.0009) [2023-12-26 18:23:58,545][105692] Updated weights for policy 0, policy_version 416808 (0.0010) [2023-12-26 18:23:58,721][105620] Updated weights for policy 1, policy_version 417235 (0.0008) [2023-12-26 18:23:58,785][105620] Updated weights for policy 1, policy_version 417245 (0.0008) [2023-12-26 18:23:58,849][105620] Updated weights for policy 1, policy_version 417255 (0.0008) [2023-12-26 18:23:59,294][105692] Updated weights for policy 0, policy_version 416818 (0.0009) [2023-12-26 18:23:59,347][105692] Updated weights for policy 0, policy_version 416828 (0.0010) [2023-12-26 18:23:59,409][105692] Updated weights for policy 0, policy_version 416838 (0.0008) [2023-12-26 18:23:59,586][105620] Updated weights for policy 1, policy_version 417265 (0.0008) [2023-12-26 18:23:59,644][105620] Updated weights for policy 1, policy_version 417275 (0.0008) [2023-12-26 18:23:59,692][105620] Updated weights for policy 1, policy_version 417285 (0.0007) [2023-12-26 18:23:59,736][105620] Updated weights for policy 1, policy_version 417295 (0.0008) [2023-12-26 18:24:00,163][105692] Updated weights for policy 0, policy_version 416848 (0.0010) [2023-12-26 18:24:00,224][105692] Updated weights for policy 0, policy_version 416858 (0.0011) [2023-12-26 18:24:00,277][105692] Updated weights for policy 0, policy_version 416868 (0.0010) [2023-12-26 18:24:00,440][105620] Updated weights for policy 1, policy_version 417305 (0.0010) [2023-12-26 18:24:00,502][105620] Updated weights for policy 1, policy_version 417315 (0.0010) [2023-12-26 18:24:00,570][105620] Updated weights for policy 1, policy_version 417325 (0.0005) [2023-12-26 18:24:01,018][105692] Updated weights for policy 0, policy_version 416878 (0.0009) [2023-12-26 18:24:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19244.3). Total num frames: 213581824. Throughput: 0: 9676.1, 1: 9902.4. Samples: 213556516. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 18:24:01,063][104569] Avg episode reward: [(0, '9354.285'), (1, '9081.287')] [2023-12-26 18:24:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000417328_106848256.pth... [2023-12-26 18:24:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000416208_106561536.pth [2023-12-26 18:24:01,079][105692] Updated weights for policy 0, policy_version 416888 (0.0010) [2023-12-26 18:24:01,139][105692] Updated weights for policy 0, policy_version 416898 (0.0011) [2023-12-26 18:24:01,146][105620] Updated weights for policy 1, policy_version 417335 (0.0007) [2023-12-26 18:24:01,170][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000416904_106741760.pth... [2023-12-26 18:24:01,174][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000415720_106438656.pth [2023-12-26 18:24:01,204][105620] Updated weights for policy 1, policy_version 417345 (0.0008) [2023-12-26 18:24:01,255][105620] Updated weights for policy 1, policy_version 417355 (0.0008) [2023-12-26 18:24:01,886][105692] Updated weights for policy 0, policy_version 416908 (0.0008) [2023-12-26 18:24:01,947][105692] Updated weights for policy 0, policy_version 416918 (0.0005) [2023-12-26 18:24:02,002][105692] Updated weights for policy 0, policy_version 416928 (0.0005) [2023-12-26 18:24:02,007][105620] Updated weights for policy 1, policy_version 417365 (0.0009) [2023-12-26 18:24:02,073][105620] Updated weights for policy 1, policy_version 417375 (0.0010) [2023-12-26 18:24:02,129][105620] Updated weights for policy 1, policy_version 417385 (0.0010) [2023-12-26 18:24:02,563][105692] Updated weights for policy 0, policy_version 416938 (0.0007) [2023-12-26 18:24:02,618][105692] Updated weights for policy 0, policy_version 416948 (0.0010) [2023-12-26 18:24:02,678][105692] Updated weights for policy 0, policy_version 416958 (0.0008) [2023-12-26 18:24:02,738][105692] Updated weights for policy 0, policy_version 416968 (0.0010) [2023-12-26 18:24:02,794][105620] Updated weights for policy 1, policy_version 417395 (0.0007) [2023-12-26 18:24:02,851][105620] Updated weights for policy 1, policy_version 417405 (0.0010) [2023-12-26 18:24:02,907][105620] Updated weights for policy 1, policy_version 417415 (0.0008) [2023-12-26 18:24:03,374][105692] Updated weights for policy 0, policy_version 416978 (0.0010) [2023-12-26 18:24:03,428][105692] Updated weights for policy 0, policy_version 416988 (0.0010) [2023-12-26 18:24:03,477][105692] Updated weights for policy 0, policy_version 416998 (0.0010) [2023-12-26 18:24:03,506][105620] Updated weights for policy 1, policy_version 417425 (0.0005) [2023-12-26 18:24:03,570][105620] Updated weights for policy 1, policy_version 417435 (0.0005) [2023-12-26 18:24:03,615][105620] Updated weights for policy 1, policy_version 417445 (0.0007) [2023-12-26 18:24:03,672][105620] Updated weights for policy 1, policy_version 417455 (0.0010) [2023-12-26 18:24:04,073][105692] Updated weights for policy 0, policy_version 417008 (0.0010) [2023-12-26 18:24:04,126][105692] Updated weights for policy 0, policy_version 417018 (0.0010) [2023-12-26 18:24:04,185][105692] Updated weights for policy 0, policy_version 417028 (0.0011) [2023-12-26 18:24:04,378][105620] Updated weights for policy 1, policy_version 417465 (0.0010) [2023-12-26 18:24:04,443][105620] Updated weights for policy 1, policy_version 417475 (0.0010) [2023-12-26 18:24:04,507][105620] Updated weights for policy 1, policy_version 417485 (0.0010) [2023-12-26 18:24:04,962][105692] Updated weights for policy 0, policy_version 417038 (0.0010) [2023-12-26 18:24:05,010][105692] Updated weights for policy 0, policy_version 417048 (0.0010) [2023-12-26 18:24:05,068][105692] Updated weights for policy 0, policy_version 417058 (0.0010) [2023-12-26 18:24:05,148][105620] Updated weights for policy 1, policy_version 417495 (0.0007) [2023-12-26 18:24:05,192][105620] Updated weights for policy 1, policy_version 417505 (0.0010) [2023-12-26 18:24:05,240][105620] Updated weights for policy 1, policy_version 417515 (0.0010) [2023-12-26 18:24:05,746][105692] Updated weights for policy 0, policy_version 417068 (0.0010) [2023-12-26 18:24:05,800][105692] Updated weights for policy 0, policy_version 417078 (0.0010) [2023-12-26 18:24:05,860][105692] Updated weights for policy 0, policy_version 417088 (0.0010) [2023-12-26 18:24:05,990][105620] Updated weights for policy 1, policy_version 417525 (0.0010) [2023-12-26 18:24:06,051][105620] Updated weights for policy 1, policy_version 417535 (0.0010) [2023-12-26 18:24:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19272.0). Total num frames: 213688320. Throughput: 0: 9839.5, 1: 9922.5. Samples: 213678692. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 18:24:06,063][104569] Avg episode reward: [(0, '9354.426'), (1, '9263.903')] [2023-12-26 18:24:06,112][105620] Updated weights for policy 1, policy_version 417545 (0.0010) [2023-12-26 18:24:06,513][105692] Updated weights for policy 0, policy_version 417098 (0.0010) [2023-12-26 18:24:06,582][105692] Updated weights for policy 0, policy_version 417108 (0.0011) [2023-12-26 18:24:06,649][105692] Updated weights for policy 0, policy_version 417118 (0.0010) [2023-12-26 18:24:06,716][105692] Updated weights for policy 0, policy_version 417128 (0.0010) [2023-12-26 18:24:06,845][105620] Updated weights for policy 1, policy_version 417555 (0.0010) [2023-12-26 18:24:06,900][105620] Updated weights for policy 1, policy_version 417565 (0.0008) [2023-12-26 18:24:06,961][105620] Updated weights for policy 1, policy_version 417575 (0.0008) [2023-12-26 18:24:07,430][105692] Updated weights for policy 0, policy_version 417138 (0.0011) [2023-12-26 18:24:07,490][105692] Updated weights for policy 0, policy_version 417148 (0.0011) [2023-12-26 18:24:07,553][105692] Updated weights for policy 0, policy_version 417158 (0.0010) [2023-12-26 18:24:07,712][105620] Updated weights for policy 1, policy_version 417585 (0.0008) [2023-12-26 18:24:07,758][105620] Updated weights for policy 1, policy_version 417595 (0.0007) [2023-12-26 18:24:07,812][105620] Updated weights for policy 1, policy_version 417605 (0.0007) [2023-12-26 18:24:07,870][105620] Updated weights for policy 1, policy_version 417615 (0.0008) [2023-12-26 18:24:08,256][105692] Updated weights for policy 0, policy_version 417168 (0.0010) [2023-12-26 18:24:08,309][105692] Updated weights for policy 0, policy_version 417178 (0.0009) [2023-12-26 18:24:08,370][105692] Updated weights for policy 0, policy_version 417188 (0.0009) [2023-12-26 18:24:08,538][105620] Updated weights for policy 1, policy_version 417625 (0.0009) [2023-12-26 18:24:08,598][105620] Updated weights for policy 1, policy_version 417635 (0.0008) [2023-12-26 18:24:08,654][105620] Updated weights for policy 1, policy_version 417645 (0.0005) [2023-12-26 18:24:09,189][105692] Updated weights for policy 0, policy_version 417198 (0.0010) [2023-12-26 18:24:09,257][105692] Updated weights for policy 0, policy_version 417208 (0.0009) [2023-12-26 18:24:09,318][105692] Updated weights for policy 0, policy_version 417218 (0.0008) [2023-12-26 18:24:09,320][105620] Updated weights for policy 1, policy_version 417655 (0.0007) [2023-12-26 18:24:09,390][105620] Updated weights for policy 1, policy_version 417665 (0.0008) [2023-12-26 18:24:09,456][105620] Updated weights for policy 1, policy_version 417675 (0.0009) [2023-12-26 18:24:10,170][105620] Updated weights for policy 1, policy_version 417685 (0.0009) [2023-12-26 18:24:10,181][105692] Updated weights for policy 0, policy_version 417228 (0.0006) [2023-12-26 18:24:10,229][105620] Updated weights for policy 1, policy_version 417695 (0.0008) [2023-12-26 18:24:10,231][105692] Updated weights for policy 0, policy_version 417238 (0.0006) [2023-12-26 18:24:10,289][105692] Updated weights for policy 0, policy_version 417248 (0.0010) [2023-12-26 18:24:10,291][105620] Updated weights for policy 1, policy_version 417705 (0.0009) [2023-12-26 18:24:10,954][105692] Updated weights for policy 0, policy_version 417258 (0.0009) [2023-12-26 18:24:11,008][105692] Updated weights for policy 0, policy_version 417268 (0.0010) [2023-12-26 18:24:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19244.3). Total num frames: 213778432. Throughput: 0: 9837.3, 1: 9830.4. Samples: 213793616. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 18:24:11,063][104569] Avg episode reward: [(0, '9354.516'), (1, '8806.231')] [2023-12-26 18:24:11,063][105620] Updated weights for policy 1, policy_version 417715 (0.0007) [2023-12-26 18:24:11,071][105692] Updated weights for policy 0, policy_version 417278 (0.0011) [2023-12-26 18:24:11,116][105620] Updated weights for policy 1, policy_version 417725 (0.0009) [2023-12-26 18:24:11,127][105692] Updated weights for policy 0, policy_version 417288 (0.0011) [2023-12-26 18:24:11,182][105620] Updated weights for policy 1, policy_version 417735 (0.0008) [2023-12-26 18:24:11,901][105692] Updated weights for policy 0, policy_version 417298 (0.0011) [2023-12-26 18:24:11,930][105620] Updated weights for policy 1, policy_version 417745 (0.0006) [2023-12-26 18:24:11,961][105692] Updated weights for policy 0, policy_version 417308 (0.0011) [2023-12-26 18:24:11,997][105620] Updated weights for policy 1, policy_version 417755 (0.0005) [2023-12-26 18:24:12,022][105692] Updated weights for policy 0, policy_version 417318 (0.0011) [2023-12-26 18:24:12,068][105620] Updated weights for policy 1, policy_version 417765 (0.0005) [2023-12-26 18:24:12,131][105620] Updated weights for policy 1, policy_version 417775 (0.0006) [2023-12-26 18:24:12,688][105692] Updated weights for policy 0, policy_version 417328 (0.0009) [2023-12-26 18:24:12,757][105692] Updated weights for policy 0, policy_version 417338 (0.0007) [2023-12-26 18:24:12,792][105620] Updated weights for policy 1, policy_version 417785 (0.0009) [2023-12-26 18:24:12,814][105692] Updated weights for policy 0, policy_version 417348 (0.0007) [2023-12-26 18:24:12,846][105620] Updated weights for policy 1, policy_version 417795 (0.0009) [2023-12-26 18:24:12,901][105620] Updated weights for policy 1, policy_version 417805 (0.0008) [2023-12-26 18:24:13,344][105692] Updated weights for policy 0, policy_version 417358 (0.0007) [2023-12-26 18:24:13,410][105692] Updated weights for policy 0, policy_version 417368 (0.0006) [2023-12-26 18:24:13,470][105692] Updated weights for policy 0, policy_version 417378 (0.0005) [2023-12-26 18:24:13,781][105620] Updated weights for policy 1, policy_version 417815 (0.0008) [2023-12-26 18:24:13,841][105620] Updated weights for policy 1, policy_version 417825 (0.0007) [2023-12-26 18:24:13,899][105620] Updated weights for policy 1, policy_version 417835 (0.0008) [2023-12-26 18:24:14,075][105692] Updated weights for policy 0, policy_version 417388 (0.0007) [2023-12-26 18:24:14,125][105692] Updated weights for policy 0, policy_version 417398 (0.0010) [2023-12-26 18:24:14,184][105692] Updated weights for policy 0, policy_version 417408 (0.0011) [2023-12-26 18:24:14,645][105620] Updated weights for policy 1, policy_version 417845 (0.0009) [2023-12-26 18:24:14,687][105620] Updated weights for policy 1, policy_version 417855 (0.0006) [2023-12-26 18:24:14,735][105620] Updated weights for policy 1, policy_version 417865 (0.0005) [2023-12-26 18:24:14,967][105692] Updated weights for policy 0, policy_version 417418 (0.0010) [2023-12-26 18:24:15,042][105692] Updated weights for policy 0, policy_version 417428 (0.0009) [2023-12-26 18:24:15,098][105692] Updated weights for policy 0, policy_version 417438 (0.0011) [2023-12-26 18:24:15,157][105692] Updated weights for policy 0, policy_version 417448 (0.0011) [2023-12-26 18:24:15,463][105620] Updated weights for policy 1, policy_version 417875 (0.0007) [2023-12-26 18:24:15,523][105620] Updated weights for policy 1, policy_version 417885 (0.0009) [2023-12-26 18:24:15,577][105620] Updated weights for policy 1, policy_version 417896 (0.0009) [2023-12-26 18:24:15,837][105692] Updated weights for policy 0, policy_version 417458 (0.0009) [2023-12-26 18:24:15,891][105692] Updated weights for policy 0, policy_version 417468 (0.0005) [2023-12-26 18:24:15,959][105692] Updated weights for policy 0, policy_version 417478 (0.0006) [2023-12-26 18:24:16,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19660.8, 300 sec: 19244.2). Total num frames: 213884928. Throughput: 0: 9906.9, 1: 9810.3. Samples: 213852860. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 18:24:16,063][104569] Avg episode reward: [(0, '9261.676'), (1, '8532.823')] [2023-12-26 18:24:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000417480_106889216.pth... [2023-12-26 18:24:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000417904_106995712.pth... [2023-12-26 18:24:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000416328_106594304.pth [2023-12-26 18:24:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000416752_106700800.pth [2023-12-26 18:24:16,268][105620] Updated weights for policy 1, policy_version 417906 (0.0007) [2023-12-26 18:24:16,339][105620] Updated weights for policy 1, policy_version 417916 (0.0010) [2023-12-26 18:24:16,396][105620] Updated weights for policy 1, policy_version 417926 (0.0010) [2023-12-26 18:24:16,463][105620] Updated weights for policy 1, policy_version 417936 (0.0010) [2023-12-26 18:24:16,557][105692] Updated weights for policy 0, policy_version 417488 (0.0008) [2023-12-26 18:24:16,618][105692] Updated weights for policy 0, policy_version 417498 (0.0010) [2023-12-26 18:24:16,677][105692] Updated weights for policy 0, policy_version 417508 (0.0010) [2023-12-26 18:24:17,126][105620] Updated weights for policy 1, policy_version 417946 (0.0005) [2023-12-26 18:24:17,188][105620] Updated weights for policy 1, policy_version 417956 (0.0005) [2023-12-26 18:24:17,247][105620] Updated weights for policy 1, policy_version 417966 (0.0005) [2023-12-26 18:24:17,281][105692] Updated weights for policy 0, policy_version 417518 (0.0007) [2023-12-26 18:24:17,337][105692] Updated weights for policy 0, policy_version 417528 (0.0005) [2023-12-26 18:24:17,383][105692] Updated weights for policy 0, policy_version 417538 (0.0005) [2023-12-26 18:24:17,746][105620] Updated weights for policy 1, policy_version 417976 (0.0007) [2023-12-26 18:24:17,800][105620] Updated weights for policy 1, policy_version 417986 (0.0008) [2023-12-26 18:24:17,862][105620] Updated weights for policy 1, policy_version 417996 (0.0008) [2023-12-26 18:24:17,956][105692] Updated weights for policy 0, policy_version 417548 (0.0005) [2023-12-26 18:24:18,010][105692] Updated weights for policy 0, policy_version 417558 (0.0005) [2023-12-26 18:24:18,063][105692] Updated weights for policy 0, policy_version 417568 (0.0005) [2023-12-26 18:24:18,660][105620] Updated weights for policy 1, policy_version 418006 (0.0008) [2023-12-26 18:24:18,704][105692] Updated weights for policy 0, policy_version 417578 (0.0007) [2023-12-26 18:24:18,723][105620] Updated weights for policy 1, policy_version 418016 (0.0008) [2023-12-26 18:24:18,760][105692] Updated weights for policy 0, policy_version 417588 (0.0011) [2023-12-26 18:24:18,767][105620] Updated weights for policy 1, policy_version 418026 (0.0008) [2023-12-26 18:24:18,816][105692] Updated weights for policy 0, policy_version 417598 (0.0010) [2023-12-26 18:24:18,871][105692] Updated weights for policy 0, policy_version 417608 (0.0010) [2023-12-26 18:24:19,535][105620] Updated weights for policy 1, policy_version 418036 (0.0009) [2023-12-26 18:24:19,598][105620] Updated weights for policy 1, policy_version 418046 (0.0008) [2023-12-26 18:24:19,655][105692] Updated weights for policy 0, policy_version 417618 (0.0010) [2023-12-26 18:24:19,661][105620] Updated weights for policy 1, policy_version 418056 (0.0007) [2023-12-26 18:24:19,720][105692] Updated weights for policy 0, policy_version 417628 (0.0009) [2023-12-26 18:24:19,787][105692] Updated weights for policy 0, policy_version 417638 (0.0009) [2023-12-26 18:24:20,427][105620] Updated weights for policy 1, policy_version 418066 (0.0007) [2023-12-26 18:24:20,483][105620] Updated weights for policy 1, policy_version 418076 (0.0010) [2023-12-26 18:24:20,531][105620] Updated weights for policy 1, policy_version 418086 (0.0010) [2023-12-26 18:24:20,555][105692] Updated weights for policy 0, policy_version 417648 (0.0010) [2023-12-26 18:24:20,592][105620] Updated weights for policy 1, policy_version 418096 (0.0010) [2023-12-26 18:24:20,622][105692] Updated weights for policy 0, policy_version 417658 (0.0011) [2023-12-26 18:24:20,679][105692] Updated weights for policy 0, policy_version 417668 (0.0011) [2023-12-26 18:24:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19272.0). Total num frames: 213983232. Throughput: 0: 9997.6, 1: 9717.3. Samples: 213974512. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 18:24:21,062][104569] Avg episode reward: [(0, '9261.298'), (1, '8440.557')] [2023-12-26 18:24:21,367][105620] Updated weights for policy 1, policy_version 418106 (0.0009) [2023-12-26 18:24:21,428][105620] Updated weights for policy 1, policy_version 418116 (0.0008) [2023-12-26 18:24:21,445][105692] Updated weights for policy 0, policy_version 417678 (0.0011) [2023-12-26 18:24:21,484][105620] Updated weights for policy 1, policy_version 418126 (0.0006) [2023-12-26 18:24:21,504][105692] Updated weights for policy 0, policy_version 417688 (0.0011) [2023-12-26 18:24:21,567][105692] Updated weights for policy 0, policy_version 417698 (0.0010) [2023-12-26 18:24:22,263][105620] Updated weights for policy 1, policy_version 418136 (0.0008) [2023-12-26 18:24:22,323][105620] Updated weights for policy 1, policy_version 418146 (0.0008) [2023-12-26 18:24:22,331][105692] Updated weights for policy 0, policy_version 417708 (0.0011) [2023-12-26 18:24:22,390][105620] Updated weights for policy 1, policy_version 418156 (0.0009) [2023-12-26 18:24:22,399][105692] Updated weights for policy 0, policy_version 417718 (0.0011) [2023-12-26 18:24:22,462][105692] Updated weights for policy 0, policy_version 417728 (0.0011) [2023-12-26 18:24:23,161][105620] Updated weights for policy 1, policy_version 418166 (0.0007) [2023-12-26 18:24:23,192][105692] Updated weights for policy 0, policy_version 417738 (0.0011) [2023-12-26 18:24:23,210][105620] Updated weights for policy 1, policy_version 418176 (0.0005) [2023-12-26 18:24:23,255][105692] Updated weights for policy 0, policy_version 417748 (0.0011) [2023-12-26 18:24:23,269][105620] Updated weights for policy 1, policy_version 418186 (0.0006) [2023-12-26 18:24:23,314][105692] Updated weights for policy 0, policy_version 417758 (0.0011) [2023-12-26 18:24:23,377][105692] Updated weights for policy 0, policy_version 417768 (0.0011) [2023-12-26 18:24:23,803][105620] Updated weights for policy 1, policy_version 418196 (0.0007) [2023-12-26 18:24:23,852][105620] Updated weights for policy 1, policy_version 418206 (0.0008) [2023-12-26 18:24:23,898][105620] Updated weights for policy 1, policy_version 418216 (0.0008) [2023-12-26 18:24:24,125][105692] Updated weights for policy 0, policy_version 417778 (0.0010) [2023-12-26 18:24:24,184][105692] Updated weights for policy 0, policy_version 417788 (0.0011) [2023-12-26 18:24:24,246][105692] Updated weights for policy 0, policy_version 417798 (0.0011) [2023-12-26 18:24:24,548][105620] Updated weights for policy 1, policy_version 418226 (0.0008) [2023-12-26 18:24:24,609][105620] Updated weights for policy 1, policy_version 418236 (0.0005) [2023-12-26 18:24:24,671][105620] Updated weights for policy 1, policy_version 418246 (0.0006) [2023-12-26 18:24:24,730][105620] Updated weights for policy 1, policy_version 418256 (0.0008) [2023-12-26 18:24:24,914][105692] Updated weights for policy 0, policy_version 417808 (0.0010) [2023-12-26 18:24:24,969][105692] Updated weights for policy 0, policy_version 417818 (0.0010) [2023-12-26 18:24:25,025][105692] Updated weights for policy 0, policy_version 417828 (0.0010) [2023-12-26 18:24:25,350][105620] Updated weights for policy 1, policy_version 418266 (0.0005) [2023-12-26 18:24:25,401][105620] Updated weights for policy 1, policy_version 418276 (0.0005) [2023-12-26 18:24:25,458][105620] Updated weights for policy 1, policy_version 418286 (0.0006) [2023-12-26 18:24:25,798][105692] Updated weights for policy 0, policy_version 417838 (0.0007) [2023-12-26 18:24:25,860][105692] Updated weights for policy 0, policy_version 417848 (0.0005) [2023-12-26 18:24:25,932][105692] Updated weights for policy 0, policy_version 417858 (0.0005) [2023-12-26 18:24:26,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19797.3, 300 sec: 19272.0). Total num frames: 214081536. Throughput: 0: 9895.8, 1: 9769.2. Samples: 214090148. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 18:24:26,062][104569] Avg episode reward: [(0, '9353.845'), (1, '8623.135')] [2023-12-26 18:24:26,083][105620] Updated weights for policy 1, policy_version 418296 (0.0009) [2023-12-26 18:24:26,130][105620] Updated weights for policy 1, policy_version 418306 (0.0009) [2023-12-26 18:24:26,176][105620] Updated weights for policy 1, policy_version 418316 (0.0008) [2023-12-26 18:24:26,506][105692] Updated weights for policy 0, policy_version 417868 (0.0005) [2023-12-26 18:24:26,563][105692] Updated weights for policy 0, policy_version 417878 (0.0005) [2023-12-26 18:24:26,625][105692] Updated weights for policy 0, policy_version 417888 (0.0008) [2023-12-26 18:24:27,012][105620] Updated weights for policy 1, policy_version 418326 (0.0009) [2023-12-26 18:24:27,066][105620] Updated weights for policy 1, policy_version 418336 (0.0010) [2023-12-26 18:24:27,124][105620] Updated weights for policy 1, policy_version 418347 (0.0009) [2023-12-26 18:24:27,218][105692] Updated weights for policy 0, policy_version 417898 (0.0008) [2023-12-26 18:24:27,265][105692] Updated weights for policy 0, policy_version 417908 (0.0009) [2023-12-26 18:24:27,312][105692] Updated weights for policy 0, policy_version 417918 (0.0009) [2023-12-26 18:24:27,362][105692] Updated weights for policy 0, policy_version 417928 (0.0009) [2023-12-26 18:24:27,954][105620] Updated weights for policy 1, policy_version 418357 (0.0007) [2023-12-26 18:24:28,002][105692] Updated weights for policy 0, policy_version 417938 (0.0005) [2023-12-26 18:24:28,004][105620] Updated weights for policy 1, policy_version 418367 (0.0008) [2023-12-26 18:24:28,059][105620] Updated weights for policy 1, policy_version 418377 (0.0005) [2023-12-26 18:24:28,065][105692] Updated weights for policy 0, policy_version 417948 (0.0008) [2023-12-26 18:24:28,133][105692] Updated weights for policy 0, policy_version 417958 (0.0008) [2023-12-26 18:24:28,700][105692] Updated weights for policy 0, policy_version 417968 (0.0007) [2023-12-26 18:24:28,761][105620] Updated weights for policy 1, policy_version 418387 (0.0006) [2023-12-26 18:24:28,765][105692] Updated weights for policy 0, policy_version 417978 (0.0006) [2023-12-26 18:24:28,809][105620] Updated weights for policy 1, policy_version 418397 (0.0009) [2023-12-26 18:24:28,822][105692] Updated weights for policy 0, policy_version 417988 (0.0005) [2023-12-26 18:24:28,871][105620] Updated weights for policy 1, policy_version 418407 (0.0008) [2023-12-26 18:24:29,531][105620] Updated weights for policy 1, policy_version 418417 (0.0008) [2023-12-26 18:24:29,559][105692] Updated weights for policy 0, policy_version 417998 (0.0006) [2023-12-26 18:24:29,591][105620] Updated weights for policy 1, policy_version 418427 (0.0009) [2023-12-26 18:24:29,619][105692] Updated weights for policy 0, policy_version 418008 (0.0005) [2023-12-26 18:24:29,643][105620] Updated weights for policy 1, policy_version 418437 (0.0009) [2023-12-26 18:24:29,671][105692] Updated weights for policy 0, policy_version 418018 (0.0006) [2023-12-26 18:24:29,708][105620] Updated weights for policy 1, policy_version 418447 (0.0007) [2023-12-26 18:24:30,369][105692] Updated weights for policy 0, policy_version 418028 (0.0007) [2023-12-26 18:24:30,408][105620] Updated weights for policy 1, policy_version 418457 (0.0005) [2023-12-26 18:24:30,419][105692] Updated weights for policy 0, policy_version 418038 (0.0008) [2023-12-26 18:24:30,468][105620] Updated weights for policy 1, policy_version 418467 (0.0008) [2023-12-26 18:24:30,470][105692] Updated weights for policy 0, policy_version 418048 (0.0005) [2023-12-26 18:24:30,527][105620] Updated weights for policy 1, policy_version 418477 (0.0008) [2023-12-26 18:24:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19272.0). Total num frames: 214179840. Throughput: 0: 9984.2, 1: 9740.5. Samples: 214151084. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 18:24:31,062][104569] Avg episode reward: [(0, '9353.805'), (1, '8526.372')] [2023-12-26 18:24:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000418056_107036672.pth... [2023-12-26 18:24:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000418480_107143168.pth... [2023-12-26 18:24:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000416904_106741760.pth [2023-12-26 18:24:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000417328_106848256.pth [2023-12-26 18:24:31,219][105620] Updated weights for policy 1, policy_version 418487 (0.0009) [2023-12-26 18:24:31,253][105692] Updated weights for policy 0, policy_version 418058 (0.0007) [2023-12-26 18:24:31,272][105620] Updated weights for policy 1, policy_version 418497 (0.0008) [2023-12-26 18:24:31,315][105692] Updated weights for policy 0, policy_version 418068 (0.0009) [2023-12-26 18:24:31,335][105620] Updated weights for policy 1, policy_version 418507 (0.0007) [2023-12-26 18:24:31,376][105692] Updated weights for policy 0, policy_version 418078 (0.0007) [2023-12-26 18:24:31,437][105692] Updated weights for policy 0, policy_version 418088 (0.0010) [2023-12-26 18:24:32,088][105620] Updated weights for policy 1, policy_version 418517 (0.0008) [2023-12-26 18:24:32,141][105692] Updated weights for policy 0, policy_version 418098 (0.0008) [2023-12-26 18:24:32,150][105620] Updated weights for policy 1, policy_version 418527 (0.0008) [2023-12-26 18:24:32,200][105692] Updated weights for policy 0, policy_version 418108 (0.0007) [2023-12-26 18:24:32,211][105620] Updated weights for policy 1, policy_version 418537 (0.0009) [2023-12-26 18:24:32,256][105692] Updated weights for policy 0, policy_version 418118 (0.0007) [2023-12-26 18:24:32,838][105620] Updated weights for policy 1, policy_version 418547 (0.0007) [2023-12-26 18:24:32,890][105620] Updated weights for policy 1, policy_version 418557 (0.0007) [2023-12-26 18:24:32,937][105620] Updated weights for policy 1, policy_version 418567 (0.0009) [2023-12-26 18:24:33,087][105692] Updated weights for policy 0, policy_version 418128 (0.0009) [2023-12-26 18:24:33,143][105692] Updated weights for policy 0, policy_version 418138 (0.0009) [2023-12-26 18:24:33,197][105692] Updated weights for policy 0, policy_version 418148 (0.0009) [2023-12-26 18:24:33,509][105620] Updated weights for policy 1, policy_version 418577 (0.0007) [2023-12-26 18:24:33,570][105620] Updated weights for policy 1, policy_version 418587 (0.0005) [2023-12-26 18:24:33,624][105620] Updated weights for policy 1, policy_version 418597 (0.0005) [2023-12-26 18:24:33,672][105620] Updated weights for policy 1, policy_version 418607 (0.0005) [2023-12-26 18:24:34,055][105692] Updated weights for policy 0, policy_version 418158 (0.0009) [2023-12-26 18:24:34,103][105692] Updated weights for policy 0, policy_version 418168 (0.0008) [2023-12-26 18:24:34,167][105692] Updated weights for policy 0, policy_version 418178 (0.0009) [2023-12-26 18:24:34,288][105620] Updated weights for policy 1, policy_version 418617 (0.0010) [2023-12-26 18:24:34,353][105620] Updated weights for policy 1, policy_version 418627 (0.0011) [2023-12-26 18:24:34,416][105620] Updated weights for policy 1, policy_version 418637 (0.0010) [2023-12-26 18:24:34,972][105692] Updated weights for policy 0, policy_version 418188 (0.0007) [2023-12-26 18:24:35,022][105620] Updated weights for policy 1, policy_version 418647 (0.0010) [2023-12-26 18:24:35,032][105692] Updated weights for policy 0, policy_version 418198 (0.0007) [2023-12-26 18:24:35,085][105620] Updated weights for policy 1, policy_version 418657 (0.0009) [2023-12-26 18:24:35,087][105692] Updated weights for policy 0, policy_version 418208 (0.0006) [2023-12-26 18:24:35,142][105620] Updated weights for policy 1, policy_version 418667 (0.0008) [2023-12-26 18:24:35,721][105620] Updated weights for policy 1, policy_version 418677 (0.0008) [2023-12-26 18:24:35,769][105620] Updated weights for policy 1, policy_version 418687 (0.0010) [2023-12-26 18:24:35,783][105692] Updated weights for policy 0, policy_version 418218 (0.0005) [2023-12-26 18:24:35,821][105620] Updated weights for policy 1, policy_version 418697 (0.0007) [2023-12-26 18:24:35,843][105692] Updated weights for policy 0, policy_version 418228 (0.0007) [2023-12-26 18:24:35,910][105692] Updated weights for policy 0, policy_version 418238 (0.0008) [2023-12-26 18:24:35,969][105692] Updated weights for policy 0, policy_version 418248 (0.0009) [2023-12-26 18:24:36,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.3, 300 sec: 19299.8). Total num frames: 214286336. Throughput: 0: 9868.8, 1: 9858.7. Samples: 214269456. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 18:24:36,062][104569] Avg episode reward: [(0, '9265.274'), (1, '8525.746')] [2023-12-26 18:24:36,555][105620] Updated weights for policy 1, policy_version 418707 (0.0007) [2023-12-26 18:24:36,622][105620] Updated weights for policy 1, policy_version 418717 (0.0008) [2023-12-26 18:24:36,690][105620] Updated weights for policy 1, policy_version 418727 (0.0009) [2023-12-26 18:24:36,752][105692] Updated weights for policy 0, policy_version 418258 (0.0008) [2023-12-26 18:24:36,803][105692] Updated weights for policy 0, policy_version 418268 (0.0009) [2023-12-26 18:24:36,854][105692] Updated weights for policy 0, policy_version 418278 (0.0008) [2023-12-26 18:24:37,249][105620] Updated weights for policy 1, policy_version 418737 (0.0006) [2023-12-26 18:24:37,312][105620] Updated weights for policy 1, policy_version 418747 (0.0005) [2023-12-26 18:24:37,368][105620] Updated weights for policy 1, policy_version 418757 (0.0005) [2023-12-26 18:24:37,429][105620] Updated weights for policy 1, policy_version 418767 (0.0005) [2023-12-26 18:24:37,771][105692] Updated weights for policy 0, policy_version 418288 (0.0008) [2023-12-26 18:24:37,835][105692] Updated weights for policy 0, policy_version 418298 (0.0008) [2023-12-26 18:24:37,902][105692] Updated weights for policy 0, policy_version 418308 (0.0009) [2023-12-26 18:24:38,015][105620] Updated weights for policy 1, policy_version 418777 (0.0010) [2023-12-26 18:24:38,063][105620] Updated weights for policy 1, policy_version 418787 (0.0010) [2023-12-26 18:24:38,119][105620] Updated weights for policy 1, policy_version 418797 (0.0007) [2023-12-26 18:24:38,622][105692] Updated weights for policy 0, policy_version 418318 (0.0009) [2023-12-26 18:24:38,679][105692] Updated weights for policy 0, policy_version 418328 (0.0008) [2023-12-26 18:24:38,736][105620] Updated weights for policy 1, policy_version 418807 (0.0007) [2023-12-26 18:24:38,741][105692] Updated weights for policy 0, policy_version 418338 (0.0008) [2023-12-26 18:24:38,796][105620] Updated weights for policy 1, policy_version 418817 (0.0005) [2023-12-26 18:24:38,851][105620] Updated weights for policy 1, policy_version 418827 (0.0006) [2023-12-26 18:24:39,417][105620] Updated weights for policy 1, policy_version 418837 (0.0008) [2023-12-26 18:24:39,483][105620] Updated weights for policy 1, policy_version 418847 (0.0008) [2023-12-26 18:24:39,556][105620] Updated weights for policy 1, policy_version 418857 (0.0009) [2023-12-26 18:24:39,587][105692] Updated weights for policy 0, policy_version 418348 (0.0008) [2023-12-26 18:24:39,652][105692] Updated weights for policy 0, policy_version 418358 (0.0009) [2023-12-26 18:24:39,714][105692] Updated weights for policy 0, policy_version 418368 (0.0009) [2023-12-26 18:24:40,335][105620] Updated weights for policy 1, policy_version 418867 (0.0010) [2023-12-26 18:24:40,393][105620] Updated weights for policy 1, policy_version 418877 (0.0010) [2023-12-26 18:24:40,428][105692] Updated weights for policy 0, policy_version 418378 (0.0008) [2023-12-26 18:24:40,451][105620] Updated weights for policy 1, policy_version 418887 (0.0008) [2023-12-26 18:24:40,491][105692] Updated weights for policy 0, policy_version 418388 (0.0005) [2023-12-26 18:24:40,560][105692] Updated weights for policy 0, policy_version 418398 (0.0005) [2023-12-26 18:24:40,624][105692] Updated weights for policy 0, policy_version 418408 (0.0007) [2023-12-26 18:24:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19244.3). Total num frames: 214376448. Throughput: 0: 9730.1, 1: 9972.2. Samples: 214386808. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 18:24:41,062][104569] Avg episode reward: [(0, '9265.483'), (1, '8342.704')] [2023-12-26 18:24:41,172][105620] Updated weights for policy 1, policy_version 418897 (0.0008) [2023-12-26 18:24:41,194][105692] Updated weights for policy 0, policy_version 418418 (0.0007) [2023-12-26 18:24:41,247][105620] Updated weights for policy 1, policy_version 418908 (0.0006) [2023-12-26 18:24:41,265][105692] Updated weights for policy 0, policy_version 418428 (0.0007) [2023-12-26 18:24:41,313][105620] Updated weights for policy 1, policy_version 418918 (0.0010) [2023-12-26 18:24:41,327][105692] Updated weights for policy 0, policy_version 418438 (0.0008) [2023-12-26 18:24:41,385][105620] Updated weights for policy 1, policy_version 418928 (0.0010) [2023-12-26 18:24:42,136][105692] Updated weights for policy 0, policy_version 418448 (0.0006) [2023-12-26 18:24:42,142][105620] Updated weights for policy 1, policy_version 418938 (0.0011) [2023-12-26 18:24:42,196][105692] Updated weights for policy 0, policy_version 418458 (0.0005) [2023-12-26 18:24:42,205][105620] Updated weights for policy 1, policy_version 418948 (0.0011) [2023-12-26 18:24:42,256][105692] Updated weights for policy 0, policy_version 418468 (0.0006) [2023-12-26 18:24:42,258][105620] Updated weights for policy 1, policy_version 418958 (0.0011) [2023-12-26 18:24:42,259][105586] KL-divergence is very high: 245.0786 [2023-12-26 18:24:42,969][105620] Updated weights for policy 1, policy_version 418968 (0.0010) [2023-12-26 18:24:43,031][105620] Updated weights for policy 1, policy_version 418978 (0.0010) [2023-12-26 18:24:43,049][105692] Updated weights for policy 0, policy_version 418478 (0.0007) [2023-12-26 18:24:43,086][105620] Updated weights for policy 1, policy_version 418988 (0.0010) [2023-12-26 18:24:43,108][105692] Updated weights for policy 0, policy_version 418488 (0.0005) [2023-12-26 18:24:43,159][105692] Updated weights for policy 0, policy_version 418498 (0.0008) [2023-12-26 18:24:43,799][105620] Updated weights for policy 1, policy_version 418998 (0.0008) [2023-12-26 18:24:43,856][105620] Updated weights for policy 1, policy_version 419008 (0.0007) [2023-12-26 18:24:43,907][105620] Updated weights for policy 1, policy_version 419018 (0.0005) [2023-12-26 18:24:43,960][105692] Updated weights for policy 0, policy_version 418508 (0.0008) [2023-12-26 18:24:44,026][105692] Updated weights for policy 0, policy_version 418518 (0.0008) [2023-12-26 18:24:44,083][105692] Updated weights for policy 0, policy_version 418528 (0.0008) [2023-12-26 18:24:44,647][105620] Updated weights for policy 1, policy_version 419028 (0.0005) [2023-12-26 18:24:44,712][105620] Updated weights for policy 1, policy_version 419038 (0.0005) [2023-12-26 18:24:44,713][105692] Updated weights for policy 0, policy_version 418538 (0.0008) [2023-12-26 18:24:44,771][105620] Updated weights for policy 1, policy_version 419048 (0.0006) [2023-12-26 18:24:44,779][105692] Updated weights for policy 0, policy_version 418548 (0.0007) [2023-12-26 18:24:44,848][105692] Updated weights for policy 0, policy_version 418558 (0.0007) [2023-12-26 18:24:44,907][105692] Updated weights for policy 0, policy_version 418568 (0.0008) [2023-12-26 18:24:45,456][105620] Updated weights for policy 1, policy_version 419058 (0.0007) [2023-12-26 18:24:45,518][105620] Updated weights for policy 1, policy_version 419068 (0.0005) [2023-12-26 18:24:45,569][105620] Updated weights for policy 1, policy_version 419078 (0.0005) [2023-12-26 18:24:45,623][105620] Updated weights for policy 1, policy_version 419088 (0.0008) [2023-12-26 18:24:45,666][105692] Updated weights for policy 0, policy_version 418578 (0.0008) [2023-12-26 18:24:45,724][105692] Updated weights for policy 0, policy_version 418588 (0.0009) [2023-12-26 18:24:45,774][105692] Updated weights for policy 0, policy_version 418598 (0.0008) [2023-12-26 18:24:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.9, 300 sec: 19272.0). Total num frames: 214474752. Throughput: 0: 9731.0, 1: 9989.9. Samples: 214443956. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 18:24:46,062][104569] Avg episode reward: [(0, '9265.602'), (1, '8434.716')] [2023-12-26 18:24:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000418600_107175936.pth... [2023-12-26 18:24:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000419088_107298816.pth... [2023-12-26 18:24:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000417480_106889216.pth [2023-12-26 18:24:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000417904_106995712.pth [2023-12-26 18:24:46,185][105620] Updated weights for policy 1, policy_version 419098 (0.0006) [2023-12-26 18:24:46,234][105620] Updated weights for policy 1, policy_version 419108 (0.0005) [2023-12-26 18:24:46,294][105620] Updated weights for policy 1, policy_version 419118 (0.0005) [2023-12-26 18:24:46,679][105692] Updated weights for policy 0, policy_version 418608 (0.0006) [2023-12-26 18:24:46,726][105692] Updated weights for policy 0, policy_version 418618 (0.0005) [2023-12-26 18:24:46,783][105692] Updated weights for policy 0, policy_version 418628 (0.0006) [2023-12-26 18:24:46,823][105620] Updated weights for policy 1, policy_version 419128 (0.0009) [2023-12-26 18:24:46,871][105620] Updated weights for policy 1, policy_version 419138 (0.0010) [2023-12-26 18:24:46,919][105620] Updated weights for policy 1, policy_version 419148 (0.0010) [2023-12-26 18:24:47,474][105692] Updated weights for policy 0, policy_version 418638 (0.0010) [2023-12-26 18:24:47,521][105692] Updated weights for policy 0, policy_version 418648 (0.0010) [2023-12-26 18:24:47,585][105692] Updated weights for policy 0, policy_version 418658 (0.0010) [2023-12-26 18:24:47,694][105620] Updated weights for policy 1, policy_version 419158 (0.0009) [2023-12-26 18:24:47,747][105620] Updated weights for policy 1, policy_version 419168 (0.0008) [2023-12-26 18:24:47,799][105620] Updated weights for policy 1, policy_version 419178 (0.0008) [2023-12-26 18:24:48,346][105692] Updated weights for policy 0, policy_version 418668 (0.0010) [2023-12-26 18:24:48,409][105692] Updated weights for policy 0, policy_version 418678 (0.0011) [2023-12-26 18:24:48,453][105620] Updated weights for policy 1, policy_version 419188 (0.0008) [2023-12-26 18:24:48,470][105692] Updated weights for policy 0, policy_version 418688 (0.0009) [2023-12-26 18:24:48,508][105620] Updated weights for policy 1, policy_version 419198 (0.0008) [2023-12-26 18:24:48,556][105620] Updated weights for policy 1, policy_version 419208 (0.0008) [2023-12-26 18:24:49,124][105692] Updated weights for policy 0, policy_version 418698 (0.0010) [2023-12-26 18:24:49,186][105692] Updated weights for policy 0, policy_version 418708 (0.0009) [2023-12-26 18:24:49,256][105692] Updated weights for policy 0, policy_version 418718 (0.0007) [2023-12-26 18:24:49,318][105692] Updated weights for policy 0, policy_version 418728 (0.0010) [2023-12-26 18:24:49,325][105620] Updated weights for policy 1, policy_version 419218 (0.0007) [2023-12-26 18:24:49,391][105620] Updated weights for policy 1, policy_version 419228 (0.0008) [2023-12-26 18:24:49,442][105620] Updated weights for policy 1, policy_version 419238 (0.0005) [2023-12-26 18:24:49,489][105620] Updated weights for policy 1, policy_version 419248 (0.0005) [2023-12-26 18:24:49,970][105692] Updated weights for policy 0, policy_version 418738 (0.0008) [2023-12-26 18:24:50,040][105692] Updated weights for policy 0, policy_version 418748 (0.0007) [2023-12-26 18:24:50,101][105692] Updated weights for policy 0, policy_version 418758 (0.0011) [2023-12-26 18:24:50,180][105620] Updated weights for policy 1, policy_version 419258 (0.0008) [2023-12-26 18:24:50,242][105620] Updated weights for policy 1, policy_version 419268 (0.0008) [2023-12-26 18:24:50,302][105620] Updated weights for policy 1, policy_version 419278 (0.0009) [2023-12-26 18:24:50,828][105692] Updated weights for policy 0, policy_version 418768 (0.0010) [2023-12-26 18:24:50,898][105692] Updated weights for policy 0, policy_version 418778 (0.0010) [2023-12-26 18:24:50,965][105692] Updated weights for policy 0, policy_version 418788 (0.0009) [2023-12-26 18:24:51,010][105620] Updated weights for policy 1, policy_version 419288 (0.0008) [2023-12-26 18:24:51,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.3, 300 sec: 19272.0). Total num frames: 214573056. Throughput: 0: 9649.4, 1: 9995.1. Samples: 214562696. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 18:24:51,062][104569] Avg episode reward: [(0, '9354.292'), (1, '8710.322')] [2023-12-26 18:24:51,077][105620] Updated weights for policy 1, policy_version 419298 (0.0010) [2023-12-26 18:24:51,142][105620] Updated weights for policy 1, policy_version 419308 (0.0008) [2023-12-26 18:24:51,804][105692] Updated weights for policy 0, policy_version 418798 (0.0009) [2023-12-26 18:24:51,834][105620] Updated weights for policy 1, policy_version 419318 (0.0007) [2023-12-26 18:24:51,861][105692] Updated weights for policy 0, policy_version 418808 (0.0007) [2023-12-26 18:24:51,892][105620] Updated weights for policy 1, policy_version 419328 (0.0007) [2023-12-26 18:24:51,910][105692] Updated weights for policy 0, policy_version 418818 (0.0007) [2023-12-26 18:24:51,955][105620] Updated weights for policy 1, policy_version 419338 (0.0008) [2023-12-26 18:24:52,619][105620] Updated weights for policy 1, policy_version 419348 (0.0009) [2023-12-26 18:24:52,629][105692] Updated weights for policy 0, policy_version 418828 (0.0008) [2023-12-26 18:24:52,672][105620] Updated weights for policy 1, policy_version 419358 (0.0007) [2023-12-26 18:24:52,679][105692] Updated weights for policy 0, policy_version 418838 (0.0006) [2023-12-26 18:24:52,726][105620] Updated weights for policy 1, policy_version 419368 (0.0006) [2023-12-26 18:24:52,736][105692] Updated weights for policy 0, policy_version 418848 (0.0007) [2023-12-26 18:24:53,470][105620] Updated weights for policy 1, policy_version 419378 (0.0006) [2023-12-26 18:24:53,514][105692] Updated weights for policy 0, policy_version 418858 (0.0007) [2023-12-26 18:24:53,540][105620] Updated weights for policy 1, policy_version 419388 (0.0005) [2023-12-26 18:24:53,563][105692] Updated weights for policy 0, policy_version 418869 (0.0009) [2023-12-26 18:24:53,605][105620] Updated weights for policy 1, policy_version 419398 (0.0005) [2023-12-26 18:24:53,613][105692] Updated weights for policy 0, policy_version 418879 (0.0009) [2023-12-26 18:24:53,660][105620] Updated weights for policy 1, policy_version 419408 (0.0005) [2023-12-26 18:24:54,183][105620] Updated weights for policy 1, policy_version 419418 (0.0007) [2023-12-26 18:24:54,240][105620] Updated weights for policy 1, policy_version 419428 (0.0007) [2023-12-26 18:24:54,295][105620] Updated weights for policy 1, policy_version 419438 (0.0005) [2023-12-26 18:24:54,443][105692] Updated weights for policy 0, policy_version 418889 (0.0009) [2023-12-26 18:24:54,499][105692] Updated weights for policy 0, policy_version 418899 (0.0006) [2023-12-26 18:24:54,549][105692] Updated weights for policy 0, policy_version 418909 (0.0005) [2023-12-26 18:24:54,605][105692] Updated weights for policy 0, policy_version 418919 (0.0006) [2023-12-26 18:24:54,937][105620] Updated weights for policy 1, policy_version 419448 (0.0008) [2023-12-26 18:24:55,001][105620] Updated weights for policy 1, policy_version 419458 (0.0010) [2023-12-26 18:24:55,053][105620] Updated weights for policy 1, policy_version 419468 (0.0009) [2023-12-26 18:24:55,269][105692] Updated weights for policy 0, policy_version 418929 (0.0008) [2023-12-26 18:24:55,320][105692] Updated weights for policy 0, policy_version 418939 (0.0008) [2023-12-26 18:24:55,386][105692] Updated weights for policy 0, policy_version 418949 (0.0008) [2023-12-26 18:24:55,785][105620] Updated weights for policy 1, policy_version 419478 (0.0010) [2023-12-26 18:24:55,843][105620] Updated weights for policy 1, policy_version 419488 (0.0009) [2023-12-26 18:24:55,897][105620] Updated weights for policy 1, policy_version 419498 (0.0009) [2023-12-26 18:24:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19272.0). Total num frames: 214671360. Throughput: 0: 9630.3, 1: 10049.4. Samples: 214679200. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 18:24:56,062][104569] Avg episode reward: [(0, '9079.534'), (1, '8895.988')] [2023-12-26 18:24:56,145][105692] Updated weights for policy 0, policy_version 418959 (0.0009) [2023-12-26 18:24:56,206][105692] Updated weights for policy 0, policy_version 418969 (0.0009) [2023-12-26 18:24:56,265][105692] Updated weights for policy 0, policy_version 418979 (0.0009) [2023-12-26 18:24:56,597][105620] Updated weights for policy 1, policy_version 419508 (0.0007) [2023-12-26 18:24:56,646][105620] Updated weights for policy 1, policy_version 419518 (0.0008) [2023-12-26 18:24:56,692][105620] Updated weights for policy 1, policy_version 419528 (0.0009) [2023-12-26 18:24:57,074][105692] Updated weights for policy 0, policy_version 418989 (0.0009) [2023-12-26 18:24:57,130][105692] Updated weights for policy 0, policy_version 418999 (0.0009) [2023-12-26 18:24:57,184][105692] Updated weights for policy 0, policy_version 419009 (0.0009) [2023-12-26 18:24:57,352][105620] Updated weights for policy 1, policy_version 419538 (0.0008) [2023-12-26 18:24:57,400][105620] Updated weights for policy 1, policy_version 419548 (0.0009) [2023-12-26 18:24:57,451][105620] Updated weights for policy 1, policy_version 419558 (0.0009) [2023-12-26 18:24:57,498][105620] Updated weights for policy 1, policy_version 419568 (0.0009) [2023-12-26 18:24:58,008][105692] Updated weights for policy 0, policy_version 419019 (0.0009) [2023-12-26 18:24:58,068][105692] Updated weights for policy 0, policy_version 419029 (0.0008) [2023-12-26 18:24:58,130][105692] Updated weights for policy 0, policy_version 419039 (0.0007) [2023-12-26 18:24:58,155][105620] Updated weights for policy 1, policy_version 419578 (0.0008) [2023-12-26 18:24:58,217][105620] Updated weights for policy 1, policy_version 419588 (0.0008) [2023-12-26 18:24:58,280][105620] Updated weights for policy 1, policy_version 419598 (0.0007) [2023-12-26 18:24:58,964][105692] Updated weights for policy 0, policy_version 419049 (0.0007) [2023-12-26 18:24:59,028][105692] Updated weights for policy 0, policy_version 419059 (0.0009) [2023-12-26 18:24:59,034][105620] Updated weights for policy 1, policy_version 419608 (0.0008) [2023-12-26 18:24:59,086][105692] Updated weights for policy 0, policy_version 419069 (0.0006) [2023-12-26 18:24:59,097][105620] Updated weights for policy 1, policy_version 419618 (0.0007) [2023-12-26 18:24:59,153][105692] Updated weights for policy 0, policy_version 419079 (0.0008) [2023-12-26 18:24:59,168][105620] Updated weights for policy 1, policy_version 419628 (0.0008) [2023-12-26 18:24:59,858][105620] Updated weights for policy 1, policy_version 419638 (0.0009) [2023-12-26 18:24:59,918][105620] Updated weights for policy 1, policy_version 419648 (0.0009) [2023-12-26 18:24:59,963][105692] Updated weights for policy 0, policy_version 419089 (0.0008) [2023-12-26 18:24:59,979][105620] Updated weights for policy 1, policy_version 419658 (0.0008) [2023-12-26 18:25:00,013][105692] Updated weights for policy 0, policy_version 419099 (0.0008) [2023-12-26 18:25:00,070][105692] Updated weights for policy 0, policy_version 419109 (0.0010) [2023-12-26 18:25:00,590][105620] Updated weights for policy 1, policy_version 419668 (0.0010) [2023-12-26 18:25:00,649][105620] Updated weights for policy 1, policy_version 419678 (0.0009) [2023-12-26 18:25:00,696][105620] Updated weights for policy 1, policy_version 419688 (0.0008) [2023-12-26 18:25:00,897][105692] Updated weights for policy 0, policy_version 419119 (0.0009) [2023-12-26 18:25:00,950][105692] Updated weights for policy 0, policy_version 419129 (0.0008) [2023-12-26 18:25:01,011][105692] Updated weights for policy 0, policy_version 419139 (0.0008) [2023-12-26 18:25:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19244.3). Total num frames: 214769664. Throughput: 0: 9506.8, 1: 10132.2. Samples: 214736612. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 18:25:01,062][104569] Avg episode reward: [(0, '9079.581'), (1, '8710.910')] [2023-12-26 18:25:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000419144_107315200.pth... [2023-12-26 18:25:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000419696_107454464.pth... [2023-12-26 18:25:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000418480_107143168.pth [2023-12-26 18:25:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000418056_107036672.pth [2023-12-26 18:25:01,465][105620] Updated weights for policy 1, policy_version 419698 (0.0009) [2023-12-26 18:25:01,519][105620] Updated weights for policy 1, policy_version 419708 (0.0009) [2023-12-26 18:25:01,577][105620] Updated weights for policy 1, policy_version 419718 (0.0009) [2023-12-26 18:25:01,635][105620] Updated weights for policy 1, policy_version 419728 (0.0009) [2023-12-26 18:25:01,781][105692] Updated weights for policy 0, policy_version 419149 (0.0008) [2023-12-26 18:25:01,832][105692] Updated weights for policy 0, policy_version 419159 (0.0009) [2023-12-26 18:25:01,886][105692] Updated weights for policy 0, policy_version 419169 (0.0008) [2023-12-26 18:25:02,342][105620] Updated weights for policy 1, policy_version 419738 (0.0007) [2023-12-26 18:25:02,407][105620] Updated weights for policy 1, policy_version 419748 (0.0008) [2023-12-26 18:25:02,460][105620] Updated weights for policy 1, policy_version 419758 (0.0009) [2023-12-26 18:25:02,572][105692] Updated weights for policy 0, policy_version 419179 (0.0010) [2023-12-26 18:25:02,632][105692] Updated weights for policy 0, policy_version 419189 (0.0010) [2023-12-26 18:25:02,696][105692] Updated weights for policy 0, policy_version 419199 (0.0010) [2023-12-26 18:25:03,249][105692] Updated weights for policy 0, policy_version 419209 (0.0005) [2023-12-26 18:25:03,306][105692] Updated weights for policy 0, policy_version 419219 (0.0005) [2023-12-26 18:25:03,348][105620] Updated weights for policy 1, policy_version 419768 (0.0007) [2023-12-26 18:25:03,366][105692] Updated weights for policy 0, policy_version 419229 (0.0005) [2023-12-26 18:25:03,407][105620] Updated weights for policy 1, policy_version 419778 (0.0008) [2023-12-26 18:25:03,431][105692] Updated weights for policy 0, policy_version 419239 (0.0005) [2023-12-26 18:25:03,470][105620] Updated weights for policy 1, policy_version 419788 (0.0010) [2023-12-26 18:25:03,917][105692] Updated weights for policy 0, policy_version 419249 (0.0009) [2023-12-26 18:25:03,969][105692] Updated weights for policy 0, policy_version 419259 (0.0010) [2023-12-26 18:25:04,021][105692] Updated weights for policy 0, policy_version 419269 (0.0010) [2023-12-26 18:25:04,277][105620] Updated weights for policy 1, policy_version 419798 (0.0009) [2023-12-26 18:25:04,326][105620] Updated weights for policy 1, policy_version 419808 (0.0008) [2023-12-26 18:25:04,382][105620] Updated weights for policy 1, policy_version 419818 (0.0008) [2023-12-26 18:25:04,743][105692] Updated weights for policy 0, policy_version 419279 (0.0007) [2023-12-26 18:25:04,794][105692] Updated weights for policy 0, policy_version 419289 (0.0006) [2023-12-26 18:25:04,838][105692] Updated weights for policy 0, policy_version 419299 (0.0005) [2023-12-26 18:25:05,201][105620] Updated weights for policy 1, policy_version 419828 (0.0009) [2023-12-26 18:25:05,249][105620] Updated weights for policy 1, policy_version 419838 (0.0008) [2023-12-26 18:25:05,303][105620] Updated weights for policy 1, policy_version 419848 (0.0006) [2023-12-26 18:25:05,466][105692] Updated weights for policy 0, policy_version 419309 (0.0006) [2023-12-26 18:25:05,524][105692] Updated weights for policy 0, policy_version 419319 (0.0005) [2023-12-26 18:25:05,582][105692] Updated weights for policy 0, policy_version 419329 (0.0010) [2023-12-26 18:25:05,989][105620] Updated weights for policy 1, policy_version 419858 (0.0006) [2023-12-26 18:25:06,048][105620] Updated weights for policy 1, policy_version 419868 (0.0008) [2023-12-26 18:25:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19244.3). Total num frames: 214859776. Throughput: 0: 9443.5, 1: 10047.3. Samples: 214851596. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 18:25:06,062][104569] Avg episode reward: [(0, '9263.416'), (1, '8803.573')] [2023-12-26 18:25:06,102][105620] Updated weights for policy 1, policy_version 419878 (0.0009) [2023-12-26 18:25:06,165][105620] Updated weights for policy 1, policy_version 419888 (0.0008) [2023-12-26 18:25:06,192][105692] Updated weights for policy 0, policy_version 419339 (0.0009) [2023-12-26 18:25:06,242][105692] Updated weights for policy 0, policy_version 419349 (0.0005) [2023-12-26 18:25:06,290][105692] Updated weights for policy 0, policy_version 419359 (0.0006) [2023-12-26 18:25:06,923][105620] Updated weights for policy 1, policy_version 419898 (0.0005) [2023-12-26 18:25:06,972][105620] Updated weights for policy 1, policy_version 419908 (0.0010) [2023-12-26 18:25:07,028][105692] Updated weights for policy 0, policy_version 419369 (0.0005) [2023-12-26 18:25:07,034][105620] Updated weights for policy 1, policy_version 419918 (0.0010) [2023-12-26 18:25:07,093][105692] Updated weights for policy 0, policy_version 419379 (0.0007) [2023-12-26 18:25:07,162][105692] Updated weights for policy 0, policy_version 419389 (0.0006) [2023-12-26 18:25:07,222][105692] Updated weights for policy 0, policy_version 419399 (0.0008) [2023-12-26 18:25:07,717][105620] Updated weights for policy 1, policy_version 419928 (0.0010) [2023-12-26 18:25:07,770][105620] Updated weights for policy 1, policy_version 419939 (0.0010) [2023-12-26 18:25:07,818][105620] Updated weights for policy 1, policy_version 419949 (0.0010) [2023-12-26 18:25:07,889][105692] Updated weights for policy 0, policy_version 419409 (0.0008) [2023-12-26 18:25:07,903][105585] KL-divergence is very high: 108.5288 [2023-12-26 18:25:07,948][105692] Updated weights for policy 0, policy_version 419419 (0.0009) [2023-12-26 18:25:07,949][105585] KL-divergence is very high: 199.3419 [2023-12-26 18:25:07,993][105585] KL-divergence is very high: 175.7283 [2023-12-26 18:25:08,002][105692] Updated weights for policy 0, policy_version 419429 (0.0006) [2023-12-26 18:25:08,610][105620] Updated weights for policy 1, policy_version 419959 (0.0010) [2023-12-26 18:25:08,662][105692] Updated weights for policy 0, policy_version 419439 (0.0007) [2023-12-26 18:25:08,678][105620] Updated weights for policy 1, policy_version 419969 (0.0011) [2023-12-26 18:25:08,725][105692] Updated weights for policy 0, policy_version 419449 (0.0010) [2023-12-26 18:25:08,734][105620] Updated weights for policy 1, policy_version 419979 (0.0011) [2023-12-26 18:25:08,782][105692] Updated weights for policy 0, policy_version 419459 (0.0008) [2023-12-26 18:25:09,378][105620] Updated weights for policy 1, policy_version 419989 (0.0011) [2023-12-26 18:25:09,447][105620] Updated weights for policy 1, policy_version 419999 (0.0011) [2023-12-26 18:25:09,510][105620] Updated weights for policy 1, policy_version 420009 (0.0011) [2023-12-26 18:25:09,538][105692] Updated weights for policy 0, policy_version 419469 (0.0009) [2023-12-26 18:25:09,587][105692] Updated weights for policy 0, policy_version 419479 (0.0010) [2023-12-26 18:25:09,635][105692] Updated weights for policy 0, policy_version 419489 (0.0010) [2023-12-26 18:25:10,284][105620] Updated weights for policy 1, policy_version 420019 (0.0011) [2023-12-26 18:25:10,347][105620] Updated weights for policy 1, policy_version 420029 (0.0011) [2023-12-26 18:25:10,409][105620] Updated weights for policy 1, policy_version 420039 (0.0011) [2023-12-26 18:25:10,442][105692] Updated weights for policy 0, policy_version 419499 (0.0010) [2023-12-26 18:25:10,506][105692] Updated weights for policy 0, policy_version 419509 (0.0009) [2023-12-26 18:25:10,568][105692] Updated weights for policy 0, policy_version 419519 (0.0008) [2023-12-26 18:25:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19244.3). Total num frames: 214958080. Throughput: 0: 9561.5, 1: 9989.7. Samples: 214969952. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 18:25:11,062][104569] Avg episode reward: [(0, '9261.244'), (1, '8894.889')] [2023-12-26 18:25:11,128][105692] Updated weights for policy 0, policy_version 419529 (0.0006) [2023-12-26 18:25:11,147][105620] Updated weights for policy 1, policy_version 420049 (0.0011) [2023-12-26 18:25:11,192][105692] Updated weights for policy 0, policy_version 419539 (0.0006) [2023-12-26 18:25:11,205][105620] Updated weights for policy 1, policy_version 420059 (0.0010) [2023-12-26 18:25:11,257][105692] Updated weights for policy 0, policy_version 419549 (0.0006) [2023-12-26 18:25:11,267][105620] Updated weights for policy 1, policy_version 420069 (0.0011) [2023-12-26 18:25:11,317][105692] Updated weights for policy 0, policy_version 419559 (0.0008) [2023-12-26 18:25:11,334][105620] Updated weights for policy 1, policy_version 420079 (0.0010) [2023-12-26 18:25:11,973][105620] Updated weights for policy 1, policy_version 420089 (0.0010) [2023-12-26 18:25:12,031][105620] Updated weights for policy 1, policy_version 420099 (0.0010) [2023-12-26 18:25:12,094][105620] Updated weights for policy 1, policy_version 420109 (0.0008) [2023-12-26 18:25:12,155][105692] Updated weights for policy 0, policy_version 419569 (0.0008) [2023-12-26 18:25:12,212][105692] Updated weights for policy 0, policy_version 419579 (0.0008) [2023-12-26 18:25:12,273][105692] Updated weights for policy 0, policy_version 419589 (0.0009) [2023-12-26 18:25:12,775][105620] Updated weights for policy 1, policy_version 420119 (0.0008) [2023-12-26 18:25:12,839][105620] Updated weights for policy 1, policy_version 420129 (0.0009) [2023-12-26 18:25:12,897][105620] Updated weights for policy 1, policy_version 420139 (0.0009) [2023-12-26 18:25:13,107][105692] Updated weights for policy 0, policy_version 419599 (0.0009) [2023-12-26 18:25:13,163][105692] Updated weights for policy 0, policy_version 419609 (0.0009) [2023-12-26 18:25:13,223][105692] Updated weights for policy 0, policy_version 419619 (0.0008) [2023-12-26 18:25:13,651][105620] Updated weights for policy 1, policy_version 420149 (0.0010) [2023-12-26 18:25:13,709][105620] Updated weights for policy 1, policy_version 420159 (0.0010) [2023-12-26 18:25:13,757][105620] Updated weights for policy 1, policy_version 420169 (0.0010) [2023-12-26 18:25:13,907][105692] Updated weights for policy 0, policy_version 419629 (0.0008) [2023-12-26 18:25:13,955][105692] Updated weights for policy 0, policy_version 419639 (0.0007) [2023-12-26 18:25:14,006][105692] Updated weights for policy 0, policy_version 419649 (0.0007) [2023-12-26 18:25:14,502][105620] Updated weights for policy 1, policy_version 420179 (0.0010) [2023-12-26 18:25:14,561][105620] Updated weights for policy 1, policy_version 420189 (0.0011) [2023-12-26 18:25:14,620][105620] Updated weights for policy 1, policy_version 420199 (0.0010) [2023-12-26 18:25:14,787][105692] Updated weights for policy 0, policy_version 419659 (0.0008) [2023-12-26 18:25:14,839][105692] Updated weights for policy 0, policy_version 419669 (0.0008) [2023-12-26 18:25:14,905][105692] Updated weights for policy 0, policy_version 419679 (0.0008) [2023-12-26 18:25:15,361][105620] Updated weights for policy 1, policy_version 420209 (0.0010) [2023-12-26 18:25:15,427][105620] Updated weights for policy 1, policy_version 420219 (0.0008) [2023-12-26 18:25:15,498][105620] Updated weights for policy 1, policy_version 420229 (0.0007) [2023-12-26 18:25:15,572][105620] Updated weights for policy 1, policy_version 420239 (0.0006) [2023-12-26 18:25:15,623][105692] Updated weights for policy 0, policy_version 419689 (0.0009) [2023-12-26 18:25:15,690][105692] Updated weights for policy 0, policy_version 419699 (0.0009) [2023-12-26 18:25:15,754][105692] Updated weights for policy 0, policy_version 419709 (0.0010) [2023-12-26 18:25:15,820][105692] Updated weights for policy 0, policy_version 419719 (0.0007) [2023-12-26 18:25:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.3, 300 sec: 19244.2). Total num frames: 215056384. Throughput: 0: 9432.9, 1: 10045.1. Samples: 215027596. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 18:25:16,063][104569] Avg episode reward: [(0, '9260.973'), (1, '8894.293')] [2023-12-26 18:25:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000420240_107593728.pth... [2023-12-26 18:25:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000419720_107462656.pth... [2023-12-26 18:25:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000419088_107298816.pth [2023-12-26 18:25:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000418600_107175936.pth [2023-12-26 18:25:16,171][105620] Updated weights for policy 1, policy_version 420249 (0.0008) [2023-12-26 18:25:16,222][105620] Updated weights for policy 1, policy_version 420259 (0.0006) [2023-12-26 18:25:16,269][105620] Updated weights for policy 1, policy_version 420269 (0.0005) [2023-12-26 18:25:16,500][105692] Updated weights for policy 0, policy_version 419729 (0.0010) [2023-12-26 18:25:16,549][105692] Updated weights for policy 0, policy_version 419739 (0.0010) [2023-12-26 18:25:16,605][105692] Updated weights for policy 0, policy_version 419749 (0.0010) [2023-12-26 18:25:16,861][105620] Updated weights for policy 1, policy_version 420279 (0.0010) [2023-12-26 18:25:16,910][105620] Updated weights for policy 1, policy_version 420289 (0.0006) [2023-12-26 18:25:16,972][105620] Updated weights for policy 1, policy_version 420299 (0.0006) [2023-12-26 18:25:17,349][105692] Updated weights for policy 0, policy_version 419759 (0.0010) [2023-12-26 18:25:17,396][105692] Updated weights for policy 0, policy_version 419769 (0.0010) [2023-12-26 18:25:17,444][105692] Updated weights for policy 0, policy_version 419779 (0.0010) [2023-12-26 18:25:17,494][105620] Updated weights for policy 1, policy_version 420309 (0.0008) [2023-12-26 18:25:17,540][105620] Updated weights for policy 1, policy_version 420319 (0.0005) [2023-12-26 18:25:17,584][105620] Updated weights for policy 1, policy_version 420329 (0.0005) [2023-12-26 18:25:18,170][105692] Updated weights for policy 0, policy_version 419789 (0.0008) [2023-12-26 18:25:18,218][105692] Updated weights for policy 0, policy_version 419799 (0.0005) [2023-12-26 18:25:18,280][105692] Updated weights for policy 0, policy_version 419809 (0.0006) [2023-12-26 18:25:18,287][105620] Updated weights for policy 1, policy_version 420339 (0.0007) [2023-12-26 18:25:18,364][105620] Updated weights for policy 1, policy_version 420349 (0.0010) [2023-12-26 18:25:18,428][105620] Updated weights for policy 1, policy_version 420359 (0.0009) [2023-12-26 18:25:18,991][105692] Updated weights for policy 0, policy_version 419819 (0.0008) [2023-12-26 18:25:19,039][105692] Updated weights for policy 0, policy_version 419829 (0.0010) [2023-12-26 18:25:19,087][105692] Updated weights for policy 0, policy_version 419839 (0.0010) [2023-12-26 18:25:19,126][105620] Updated weights for policy 1, policy_version 420369 (0.0010) [2023-12-26 18:25:19,180][105620] Updated weights for policy 1, policy_version 420379 (0.0010) [2023-12-26 18:25:19,233][105620] Updated weights for policy 1, policy_version 420389 (0.0010) [2023-12-26 18:25:19,292][105620] Updated weights for policy 1, policy_version 420399 (0.0007) [2023-12-26 18:25:19,728][105692] Updated weights for policy 0, policy_version 419849 (0.0010) [2023-12-26 18:25:19,792][105692] Updated weights for policy 0, policy_version 419859 (0.0006) [2023-12-26 18:25:19,860][105692] Updated weights for policy 0, policy_version 419869 (0.0009) [2023-12-26 18:25:19,924][105692] Updated weights for policy 0, policy_version 419879 (0.0011) [2023-12-26 18:25:20,013][105620] Updated weights for policy 1, policy_version 420409 (0.0011) [2023-12-26 18:25:20,077][105620] Updated weights for policy 1, policy_version 420419 (0.0011) [2023-12-26 18:25:20,140][105620] Updated weights for policy 1, policy_version 420429 (0.0011) [2023-12-26 18:25:20,612][105692] Updated weights for policy 0, policy_version 419889 (0.0011) [2023-12-26 18:25:20,667][105692] Updated weights for policy 0, policy_version 419899 (0.0006) [2023-12-26 18:25:20,718][105692] Updated weights for policy 0, policy_version 419909 (0.0005) [2023-12-26 18:25:20,903][105620] Updated weights for policy 1, policy_version 420439 (0.0011) [2023-12-26 18:25:20,960][105586] KL-divergence is very high: 127.9505 [2023-12-26 18:25:20,960][105620] Updated weights for policy 1, policy_version 420449 (0.0011) [2023-12-26 18:25:21,013][105586] KL-divergence is very high: 130.2531 [2023-12-26 18:25:21,027][105620] Updated weights for policy 1, policy_version 420459 (0.0010) [2023-12-26 18:25:21,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19299.8). Total num frames: 215162880. Throughput: 0: 9499.0, 1: 10025.4. Samples: 215148056. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 18:25:21,062][104569] Avg episode reward: [(0, '9262.883'), (1, '8527.514')] [2023-12-26 18:25:21,441][105692] Updated weights for policy 0, policy_version 419919 (0.0006) [2023-12-26 18:25:21,506][105692] Updated weights for policy 0, policy_version 419929 (0.0008) [2023-12-26 18:25:21,572][105692] Updated weights for policy 0, policy_version 419939 (0.0008) [2023-12-26 18:25:21,812][105620] Updated weights for policy 1, policy_version 420469 (0.0009) [2023-12-26 18:25:21,879][105620] Updated weights for policy 1, policy_version 420479 (0.0011) [2023-12-26 18:25:21,947][105620] Updated weights for policy 1, policy_version 420489 (0.0010) [2023-12-26 18:25:22,322][105692] Updated weights for policy 0, policy_version 419949 (0.0008) [2023-12-26 18:25:22,380][105692] Updated weights for policy 0, policy_version 419959 (0.0009) [2023-12-26 18:25:22,427][105692] Updated weights for policy 0, policy_version 419969 (0.0008) [2023-12-26 18:25:22,675][105620] Updated weights for policy 1, policy_version 420499 (0.0009) [2023-12-26 18:25:22,742][105620] Updated weights for policy 1, policy_version 420509 (0.0008) [2023-12-26 18:25:22,803][105620] Updated weights for policy 1, policy_version 420519 (0.0008) [2023-12-26 18:25:23,169][105692] Updated weights for policy 0, policy_version 419979 (0.0008) [2023-12-26 18:25:23,222][105692] Updated weights for policy 0, policy_version 419989 (0.0008) [2023-12-26 18:25:23,279][105692] Updated weights for policy 0, policy_version 419999 (0.0009) [2023-12-26 18:25:23,527][105620] Updated weights for policy 1, policy_version 420529 (0.0011) [2023-12-26 18:25:23,582][105620] Updated weights for policy 1, policy_version 420539 (0.0010) [2023-12-26 18:25:23,626][105620] Updated weights for policy 1, policy_version 420549 (0.0010) [2023-12-26 18:25:23,674][105620] Updated weights for policy 1, policy_version 420559 (0.0010) [2023-12-26 18:25:23,827][105692] Updated weights for policy 0, policy_version 420009 (0.0006) [2023-12-26 18:25:23,873][105692] Updated weights for policy 0, policy_version 420019 (0.0005) [2023-12-26 18:25:23,941][105692] Updated weights for policy 0, policy_version 420029 (0.0005) [2023-12-26 18:25:24,010][105692] Updated weights for policy 0, policy_version 420039 (0.0006) [2023-12-26 18:25:24,443][105620] Updated weights for policy 1, policy_version 420569 (0.0010) [2023-12-26 18:25:24,491][105620] Updated weights for policy 1, policy_version 420579 (0.0010) [2023-12-26 18:25:24,542][105620] Updated weights for policy 1, policy_version 420589 (0.0010) [2023-12-26 18:25:24,625][105692] Updated weights for policy 0, policy_version 420049 (0.0007) [2023-12-26 18:25:24,680][105692] Updated weights for policy 0, policy_version 420059 (0.0008) [2023-12-26 18:25:24,731][105692] Updated weights for policy 0, policy_version 420069 (0.0008) [2023-12-26 18:25:25,282][105620] Updated weights for policy 1, policy_version 420599 (0.0010) [2023-12-26 18:25:25,330][105620] Updated weights for policy 1, policy_version 420609 (0.0010) [2023-12-26 18:25:25,378][105620] Updated weights for policy 1, policy_version 420619 (0.0010) [2023-12-26 18:25:25,406][105692] Updated weights for policy 0, policy_version 420079 (0.0008) [2023-12-26 18:25:25,473][105692] Updated weights for policy 0, policy_version 420089 (0.0005) [2023-12-26 18:25:25,540][105692] Updated weights for policy 0, policy_version 420099 (0.0005) [2023-12-26 18:25:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19272.0). Total num frames: 215252992. Throughput: 0: 9668.5, 1: 9858.5. Samples: 215265520. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 18:25:26,063][104569] Avg episode reward: [(0, '9262.979'), (1, '8528.974')] [2023-12-26 18:25:26,071][105692] Updated weights for policy 0, policy_version 420109 (0.0007) [2023-12-26 18:25:26,075][105620] Updated weights for policy 1, policy_version 420629 (0.0008) [2023-12-26 18:25:26,125][105692] Updated weights for policy 0, policy_version 420119 (0.0007) [2023-12-26 18:25:26,133][105620] Updated weights for policy 1, policy_version 420639 (0.0010) [2023-12-26 18:25:26,184][105620] Updated weights for policy 1, policy_version 420649 (0.0010) [2023-12-26 18:25:26,191][105692] Updated weights for policy 0, policy_version 420129 (0.0005) [2023-12-26 18:25:26,719][105692] Updated weights for policy 0, policy_version 420139 (0.0008) [2023-12-26 18:25:26,771][105692] Updated weights for policy 0, policy_version 420149 (0.0006) [2023-12-26 18:25:26,793][105620] Updated weights for policy 1, policy_version 420659 (0.0009) [2023-12-26 18:25:26,834][105692] Updated weights for policy 0, policy_version 420159 (0.0005) [2023-12-26 18:25:26,847][105620] Updated weights for policy 1, policy_version 420669 (0.0009) [2023-12-26 18:25:26,895][105620] Updated weights for policy 1, policy_version 420679 (0.0010) [2023-12-26 18:25:27,426][105692] Updated weights for policy 0, policy_version 420169 (0.0006) [2023-12-26 18:25:27,498][105692] Updated weights for policy 0, policy_version 420179 (0.0009) [2023-12-26 18:25:27,549][105620] Updated weights for policy 1, policy_version 420689 (0.0010) [2023-12-26 18:25:27,558][105692] Updated weights for policy 0, policy_version 420189 (0.0008) [2023-12-26 18:25:27,606][105620] Updated weights for policy 1, policy_version 420699 (0.0010) [2023-12-26 18:25:27,613][105692] Updated weights for policy 0, policy_version 420199 (0.0006) [2023-12-26 18:25:27,668][105620] Updated weights for policy 1, policy_version 420709 (0.0010) [2023-12-26 18:25:27,729][105620] Updated weights for policy 1, policy_version 420719 (0.0010) [2023-12-26 18:25:28,313][105692] Updated weights for policy 0, policy_version 420209 (0.0006) [2023-12-26 18:25:28,374][105692] Updated weights for policy 0, policy_version 420219 (0.0006) [2023-12-26 18:25:28,437][105692] Updated weights for policy 0, policy_version 420229 (0.0006) [2023-12-26 18:25:28,467][105620] Updated weights for policy 1, policy_version 420729 (0.0008) [2023-12-26 18:25:28,523][105620] Updated weights for policy 1, policy_version 420739 (0.0008) [2023-12-26 18:25:28,577][105620] Updated weights for policy 1, policy_version 420749 (0.0005) [2023-12-26 18:25:29,053][105692] Updated weights for policy 0, policy_version 420239 (0.0009) [2023-12-26 18:25:29,097][105692] Updated weights for policy 0, policy_version 420249 (0.0010) [2023-12-26 18:25:29,149][105692] Updated weights for policy 0, policy_version 420259 (0.0006) [2023-12-26 18:25:29,320][105620] Updated weights for policy 1, policy_version 420759 (0.0010) [2023-12-26 18:25:29,387][105620] Updated weights for policy 1, policy_version 420769 (0.0011) [2023-12-26 18:25:29,453][105620] Updated weights for policy 1, policy_version 420779 (0.0011) [2023-12-26 18:25:29,866][105692] Updated weights for policy 0, policy_version 420269 (0.0007) [2023-12-26 18:25:29,916][105692] Updated weights for policy 0, policy_version 420279 (0.0008) [2023-12-26 18:25:29,971][105692] Updated weights for policy 0, policy_version 420289 (0.0008) [2023-12-26 18:25:30,169][105620] Updated weights for policy 1, policy_version 420789 (0.0010) [2023-12-26 18:25:30,234][105620] Updated weights for policy 1, policy_version 420799 (0.0009) [2023-12-26 18:25:30,296][105620] Updated weights for policy 1, policy_version 420809 (0.0010) [2023-12-26 18:25:30,668][105692] Updated weights for policy 0, policy_version 420299 (0.0007) [2023-12-26 18:25:30,723][105692] Updated weights for policy 0, policy_version 420309 (0.0007) [2023-12-26 18:25:30,771][105692] Updated weights for policy 0, policy_version 420319 (0.0008) [2023-12-26 18:25:31,048][105620] Updated weights for policy 1, policy_version 420819 (0.0010) [2023-12-26 18:25:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19327.6). Total num frames: 215359488. Throughput: 0: 9798.8, 1: 9892.6. Samples: 215330068. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 18:25:31,062][104569] Avg episode reward: [(0, '9168.654'), (1, '8621.487')] [2023-12-26 18:25:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000420328_107618304.pth... [2023-12-26 18:25:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000419144_107315200.pth [2023-12-26 18:25:31,114][105620] Updated weights for policy 1, policy_version 420829 (0.0008) [2023-12-26 18:25:31,180][105620] Updated weights for policy 1, policy_version 420839 (0.0007) [2023-12-26 18:25:31,233][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000420848_107749376.pth... [2023-12-26 18:25:31,236][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000419696_107454464.pth [2023-12-26 18:25:31,596][105692] Updated weights for policy 0, policy_version 420329 (0.0008) [2023-12-26 18:25:31,657][105692] Updated weights for policy 0, policy_version 420339 (0.0009) [2023-12-26 18:25:31,709][105692] Updated weights for policy 0, policy_version 420349 (0.0008) [2023-12-26 18:25:31,768][105692] Updated weights for policy 0, policy_version 420359 (0.0008) [2023-12-26 18:25:31,843][105620] Updated weights for policy 1, policy_version 420849 (0.0006) [2023-12-26 18:25:31,899][105620] Updated weights for policy 1, policy_version 420859 (0.0006) [2023-12-26 18:25:31,953][105620] Updated weights for policy 1, policy_version 420869 (0.0010) [2023-12-26 18:25:32,024][105620] Updated weights for policy 1, policy_version 420879 (0.0009) [2023-12-26 18:25:32,483][105692] Updated weights for policy 0, policy_version 420369 (0.0010) [2023-12-26 18:25:32,550][105692] Updated weights for policy 0, policy_version 420379 (0.0011) [2023-12-26 18:25:32,605][105692] Updated weights for policy 0, policy_version 420389 (0.0010) [2023-12-26 18:25:32,659][105620] Updated weights for policy 1, policy_version 420889 (0.0005) [2023-12-26 18:25:32,705][105620] Updated weights for policy 1, policy_version 420899 (0.0005) [2023-12-26 18:25:32,752][105620] Updated weights for policy 1, policy_version 420909 (0.0005) [2023-12-26 18:25:33,306][105692] Updated weights for policy 0, policy_version 420399 (0.0011) [2023-12-26 18:25:33,352][105620] Updated weights for policy 1, policy_version 420919 (0.0009) [2023-12-26 18:25:33,361][105692] Updated weights for policy 0, policy_version 420409 (0.0010) [2023-12-26 18:25:33,399][105620] Updated weights for policy 1, policy_version 420929 (0.0010) [2023-12-26 18:25:33,419][105692] Updated weights for policy 0, policy_version 420419 (0.0010) [2023-12-26 18:25:33,447][105620] Updated weights for policy 1, policy_version 420939 (0.0010) [2023-12-26 18:25:34,068][105692] Updated weights for policy 0, policy_version 420429 (0.0008) [2023-12-26 18:25:34,114][105692] Updated weights for policy 0, policy_version 420439 (0.0006) [2023-12-26 18:25:34,175][105692] Updated weights for policy 0, policy_version 420449 (0.0008) [2023-12-26 18:25:34,205][105620] Updated weights for policy 1, policy_version 420949 (0.0010) [2023-12-26 18:25:34,270][105620] Updated weights for policy 1, policy_version 420959 (0.0010) [2023-12-26 18:25:34,276][105586] KL-divergence is very high: 117.1669 [2023-12-26 18:25:34,326][105586] KL-divergence is very high: 121.6438 [2023-12-26 18:25:34,332][105620] Updated weights for policy 1, policy_version 420969 (0.0010) [2023-12-26 18:25:34,874][105692] Updated weights for policy 0, policy_version 420459 (0.0007) [2023-12-26 18:25:34,924][105692] Updated weights for policy 0, policy_version 420469 (0.0005) [2023-12-26 18:25:34,979][105692] Updated weights for policy 0, policy_version 420479 (0.0005) [2023-12-26 18:25:35,038][105620] Updated weights for policy 1, policy_version 420979 (0.0010) [2023-12-26 18:25:35,086][105620] Updated weights for policy 1, policy_version 420989 (0.0010) [2023-12-26 18:25:35,147][105620] Updated weights for policy 1, policy_version 420999 (0.0010) [2023-12-26 18:25:35,608][105692] Updated weights for policy 0, policy_version 420489 (0.0006) [2023-12-26 18:25:35,665][105692] Updated weights for policy 0, policy_version 420499 (0.0005) [2023-12-26 18:25:35,716][105692] Updated weights for policy 0, policy_version 420509 (0.0005) [2023-12-26 18:25:35,765][105692] Updated weights for policy 0, policy_version 420519 (0.0005) [2023-12-26 18:25:35,902][105620] Updated weights for policy 1, policy_version 421009 (0.0010) [2023-12-26 18:25:35,965][105620] Updated weights for policy 1, policy_version 421019 (0.0006) [2023-12-26 18:25:36,027][105620] Updated weights for policy 1, policy_version 421029 (0.0005) [2023-12-26 18:25:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19327.6). Total num frames: 215457792. Throughput: 0: 9843.7, 1: 9846.6. Samples: 215448760. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 18:25:36,062][104569] Avg episode reward: [(0, '9168.188'), (1, '9079.897')] [2023-12-26 18:25:36,078][105620] Updated weights for policy 1, policy_version 421039 (0.0005) [2023-12-26 18:25:36,364][105692] Updated weights for policy 0, policy_version 420529 (0.0008) [2023-12-26 18:25:36,424][105692] Updated weights for policy 0, policy_version 420539 (0.0008) [2023-12-26 18:25:36,486][105692] Updated weights for policy 0, policy_version 420549 (0.0006) [2023-12-26 18:25:36,792][105620] Updated weights for policy 1, policy_version 421049 (0.0010) [2023-12-26 18:25:36,856][105620] Updated weights for policy 1, policy_version 421059 (0.0010) [2023-12-26 18:25:36,914][105620] Updated weights for policy 1, policy_version 421069 (0.0010) [2023-12-26 18:25:37,167][105692] Updated weights for policy 0, policy_version 420559 (0.0009) [2023-12-26 18:25:37,226][105692] Updated weights for policy 0, policy_version 420569 (0.0008) [2023-12-26 18:25:37,287][105692] Updated weights for policy 0, policy_version 420579 (0.0006) [2023-12-26 18:25:37,637][105620] Updated weights for policy 1, policy_version 421079 (0.0010) [2023-12-26 18:25:37,686][105620] Updated weights for policy 1, policy_version 421089 (0.0010) [2023-12-26 18:25:37,743][105620] Updated weights for policy 1, policy_version 421099 (0.0010) [2023-12-26 18:25:37,948][105692] Updated weights for policy 0, policy_version 420589 (0.0006) [2023-12-26 18:25:37,993][105692] Updated weights for policy 0, policy_version 420599 (0.0007) [2023-12-26 18:25:38,045][105692] Updated weights for policy 0, policy_version 420609 (0.0007) [2023-12-26 18:25:38,454][105620] Updated weights for policy 1, policy_version 421109 (0.0010) [2023-12-26 18:25:38,512][105620] Updated weights for policy 1, policy_version 421119 (0.0010) [2023-12-26 18:25:38,572][105620] Updated weights for policy 1, policy_version 421129 (0.0010) [2023-12-26 18:25:38,835][105692] Updated weights for policy 0, policy_version 420619 (0.0007) [2023-12-26 18:25:38,897][105692] Updated weights for policy 0, policy_version 420629 (0.0006) [2023-12-26 18:25:38,961][105692] Updated weights for policy 0, policy_version 420639 (0.0008) [2023-12-26 18:25:39,298][105620] Updated weights for policy 1, policy_version 421139 (0.0010) [2023-12-26 18:25:39,361][105620] Updated weights for policy 1, policy_version 421149 (0.0008) [2023-12-26 18:25:39,429][105620] Updated weights for policy 1, policy_version 421159 (0.0008) [2023-12-26 18:25:39,655][105692] Updated weights for policy 0, policy_version 420649 (0.0006) [2023-12-26 18:25:39,721][105692] Updated weights for policy 0, policy_version 420659 (0.0006) [2023-12-26 18:25:39,781][105692] Updated weights for policy 0, policy_version 420669 (0.0008) [2023-12-26 18:25:39,847][105692] Updated weights for policy 0, policy_version 420679 (0.0009) [2023-12-26 18:25:40,144][105620] Updated weights for policy 1, policy_version 421169 (0.0008) [2023-12-26 18:25:40,204][105620] Updated weights for policy 1, policy_version 421179 (0.0009) [2023-12-26 18:25:40,261][105620] Updated weights for policy 1, policy_version 421189 (0.0008) [2023-12-26 18:25:40,317][105620] Updated weights for policy 1, policy_version 421199 (0.0008) [2023-12-26 18:25:40,508][105692] Updated weights for policy 0, policy_version 420689 (0.0007) [2023-12-26 18:25:40,562][105692] Updated weights for policy 0, policy_version 420699 (0.0006) [2023-12-26 18:25:40,616][105692] Updated weights for policy 0, policy_version 420709 (0.0006) [2023-12-26 18:25:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19327.6). Total num frames: 215556096. Throughput: 0: 9987.1, 1: 9750.4. Samples: 215567388. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 18:25:41,063][104569] Avg episode reward: [(0, '9260.940'), (1, '8987.534')] [2023-12-26 18:25:41,214][105620] Updated weights for policy 1, policy_version 421209 (0.0008) [2023-12-26 18:25:41,219][105692] Updated weights for policy 0, policy_version 420719 (0.0007) [2023-12-26 18:25:41,285][105692] Updated weights for policy 0, policy_version 420729 (0.0007) [2023-12-26 18:25:41,286][105620] Updated weights for policy 1, policy_version 421219 (0.0008) [2023-12-26 18:25:41,349][105620] Updated weights for policy 1, policy_version 421229 (0.0009) [2023-12-26 18:25:41,352][105692] Updated weights for policy 0, policy_version 420739 (0.0007) [2023-12-26 18:25:42,054][105620] Updated weights for policy 1, policy_version 421239 (0.0008) [2023-12-26 18:25:42,116][105620] Updated weights for policy 1, policy_version 421249 (0.0009) [2023-12-26 18:25:42,142][105692] Updated weights for policy 0, policy_version 420749 (0.0007) [2023-12-26 18:25:42,176][105620] Updated weights for policy 1, policy_version 421259 (0.0008) [2023-12-26 18:25:42,204][105692] Updated weights for policy 0, policy_version 420759 (0.0006) [2023-12-26 18:25:42,272][105692] Updated weights for policy 0, policy_version 420769 (0.0009) [2023-12-26 18:25:42,956][105620] Updated weights for policy 1, policy_version 421269 (0.0008) [2023-12-26 18:25:43,019][105620] Updated weights for policy 1, policy_version 421279 (0.0008) [2023-12-26 18:25:43,020][105692] Updated weights for policy 0, policy_version 420779 (0.0009) [2023-12-26 18:25:43,074][105620] Updated weights for policy 1, policy_version 421289 (0.0006) [2023-12-26 18:25:43,081][105692] Updated weights for policy 0, policy_version 420789 (0.0009) [2023-12-26 18:25:43,140][105692] Updated weights for policy 0, policy_version 420799 (0.0009) [2023-12-26 18:25:43,804][105692] Updated weights for policy 0, policy_version 420809 (0.0009) [2023-12-26 18:25:43,843][105620] Updated weights for policy 1, policy_version 421299 (0.0007) [2023-12-26 18:25:43,856][105692] Updated weights for policy 0, policy_version 420819 (0.0005) [2023-12-26 18:25:43,890][105620] Updated weights for policy 1, policy_version 421309 (0.0008) [2023-12-26 18:25:43,907][105692] Updated weights for policy 0, policy_version 420829 (0.0005) [2023-12-26 18:25:43,951][105620] Updated weights for policy 1, policy_version 421319 (0.0008) [2023-12-26 18:25:43,956][105692] Updated weights for policy 0, policy_version 420839 (0.0005) [2023-12-26 18:25:44,486][105692] Updated weights for policy 0, policy_version 420849 (0.0006) [2023-12-26 18:25:44,537][105692] Updated weights for policy 0, policy_version 420859 (0.0007) [2023-12-26 18:25:44,588][105692] Updated weights for policy 0, policy_version 420869 (0.0005) [2023-12-26 18:25:44,856][105620] Updated weights for policy 1, policy_version 421329 (0.0009) [2023-12-26 18:25:44,912][105620] Updated weights for policy 1, policy_version 421339 (0.0011) [2023-12-26 18:25:44,965][105620] Updated weights for policy 1, policy_version 421349 (0.0011) [2023-12-26 18:25:45,025][105620] Updated weights for policy 1, policy_version 421359 (0.0011) [2023-12-26 18:25:45,311][105692] Updated weights for policy 0, policy_version 420879 (0.0008) [2023-12-26 18:25:45,378][105692] Updated weights for policy 0, policy_version 420889 (0.0008) [2023-12-26 18:25:45,436][105692] Updated weights for policy 0, policy_version 420899 (0.0007) [2023-12-26 18:25:45,790][105620] Updated weights for policy 1, policy_version 421369 (0.0011) [2023-12-26 18:25:45,839][105620] Updated weights for policy 1, policy_version 421379 (0.0010) [2023-12-26 18:25:45,892][105620] Updated weights for policy 1, policy_version 421389 (0.0010) [2023-12-26 18:25:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19327.6). Total num frames: 215654400. Throughput: 0: 10024.4, 1: 9665.9. Samples: 215622680. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-26 18:25:46,063][104569] Avg episode reward: [(0, '9084.307'), (1, '9172.106')] [2023-12-26 18:25:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000421392_107888640.pth... [2023-12-26 18:25:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000420904_107765760.pth... [2023-12-26 18:25:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000420240_107593728.pth [2023-12-26 18:25:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000419720_107462656.pth [2023-12-26 18:25:46,196][105692] Updated weights for policy 0, policy_version 420909 (0.0008) [2023-12-26 18:25:46,240][105692] Updated weights for policy 0, policy_version 420919 (0.0008) [2023-12-26 18:25:46,288][105692] Updated weights for policy 0, policy_version 420929 (0.0008) [2023-12-26 18:25:46,661][105620] Updated weights for policy 1, policy_version 421399 (0.0011) [2023-12-26 18:25:46,723][105620] Updated weights for policy 1, policy_version 421409 (0.0010) [2023-12-26 18:25:46,778][105620] Updated weights for policy 1, policy_version 421419 (0.0010) [2023-12-26 18:25:47,066][105692] Updated weights for policy 0, policy_version 420939 (0.0009) [2023-12-26 18:25:47,125][105692] Updated weights for policy 0, policy_version 420949 (0.0010) [2023-12-26 18:25:47,176][105692] Updated weights for policy 0, policy_version 420959 (0.0010) [2023-12-26 18:25:47,510][105620] Updated weights for policy 1, policy_version 421429 (0.0010) [2023-12-26 18:25:47,565][105620] Updated weights for policy 1, policy_version 421439 (0.0010) [2023-12-26 18:25:47,613][105620] Updated weights for policy 1, policy_version 421449 (0.0010) [2023-12-26 18:25:47,834][105692] Updated weights for policy 0, policy_version 420969 (0.0010) [2023-12-26 18:25:47,895][105692] Updated weights for policy 0, policy_version 420979 (0.0005) [2023-12-26 18:25:47,958][105692] Updated weights for policy 0, policy_version 420989 (0.0005) [2023-12-26 18:25:48,009][105692] Updated weights for policy 0, policy_version 420999 (0.0005) [2023-12-26 18:25:48,295][105620] Updated weights for policy 1, policy_version 421459 (0.0009) [2023-12-26 18:25:48,364][105620] Updated weights for policy 1, policy_version 421469 (0.0006) [2023-12-26 18:25:48,430][105620] Updated weights for policy 1, policy_version 421479 (0.0006) [2023-12-26 18:25:48,656][105692] Updated weights for policy 0, policy_version 421009 (0.0010) [2023-12-26 18:25:48,718][105692] Updated weights for policy 0, policy_version 421019 (0.0011) [2023-12-26 18:25:48,780][105692] Updated weights for policy 0, policy_version 421029 (0.0010) [2023-12-26 18:25:49,075][105620] Updated weights for policy 1, policy_version 421489 (0.0011) [2023-12-26 18:25:49,138][105620] Updated weights for policy 1, policy_version 421499 (0.0011) [2023-12-26 18:25:49,203][105620] Updated weights for policy 1, policy_version 421509 (0.0010) [2023-12-26 18:25:49,271][105620] Updated weights for policy 1, policy_version 421519 (0.0011) [2023-12-26 18:25:49,467][105692] Updated weights for policy 0, policy_version 421039 (0.0010) [2023-12-26 18:25:49,525][105692] Updated weights for policy 0, policy_version 421049 (0.0010) [2023-12-26 18:25:49,587][105692] Updated weights for policy 0, policy_version 421059 (0.0010) [2023-12-26 18:25:50,012][105620] Updated weights for policy 1, policy_version 421529 (0.0011) [2023-12-26 18:25:50,071][105620] Updated weights for policy 1, policy_version 421539 (0.0011) [2023-12-26 18:25:50,134][105620] Updated weights for policy 1, policy_version 421549 (0.0011) [2023-12-26 18:25:50,316][105692] Updated weights for policy 0, policy_version 421069 (0.0008) [2023-12-26 18:25:50,380][105692] Updated weights for policy 0, policy_version 421079 (0.0010) [2023-12-26 18:25:50,442][105692] Updated weights for policy 0, policy_version 421089 (0.0011) [2023-12-26 18:25:50,763][105620] Updated weights for policy 1, policy_version 421559 (0.0011) [2023-12-26 18:25:50,830][105620] Updated weights for policy 1, policy_version 421569 (0.0011) [2023-12-26 18:25:50,890][105620] Updated weights for policy 1, policy_version 421579 (0.0011) [2023-12-26 18:25:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19327.6). Total num frames: 215752704. Throughput: 0: 10072.4, 1: 9684.8. Samples: 215740668. Policy #0 lag: (min: 24.0, avg: 50.6, max: 56.0) [2023-12-26 18:25:51,062][104569] Avg episode reward: [(0, '8991.894'), (1, '8986.713')] [2023-12-26 18:25:51,163][105692] Updated weights for policy 0, policy_version 421099 (0.0011) [2023-12-26 18:25:51,222][105692] Updated weights for policy 0, policy_version 421109 (0.0009) [2023-12-26 18:25:51,280][105692] Updated weights for policy 0, policy_version 421119 (0.0010) [2023-12-26 18:25:51,597][105620] Updated weights for policy 1, policy_version 421589 (0.0011) [2023-12-26 18:25:51,666][105620] Updated weights for policy 1, policy_version 421599 (0.0008) [2023-12-26 18:25:51,742][105620] Updated weights for policy 1, policy_version 421610 (0.0008) [2023-12-26 18:25:52,073][105692] Updated weights for policy 0, policy_version 421129 (0.0008) [2023-12-26 18:25:52,140][105692] Updated weights for policy 0, policy_version 421139 (0.0008) [2023-12-26 18:25:52,201][105692] Updated weights for policy 0, policy_version 421149 (0.0008) [2023-12-26 18:25:52,263][105692] Updated weights for policy 0, policy_version 421159 (0.0007) [2023-12-26 18:25:52,502][105620] Updated weights for policy 1, policy_version 421620 (0.0010) [2023-12-26 18:25:52,567][105620] Updated weights for policy 1, policy_version 421630 (0.0011) [2023-12-26 18:25:52,633][105620] Updated weights for policy 1, policy_version 421640 (0.0011) [2023-12-26 18:25:52,893][105692] Updated weights for policy 0, policy_version 421169 (0.0008) [2023-12-26 18:25:52,952][105692] Updated weights for policy 0, policy_version 421179 (0.0009) [2023-12-26 18:25:53,007][105692] Updated weights for policy 0, policy_version 421189 (0.0009) [2023-12-26 18:25:53,337][105620] Updated weights for policy 1, policy_version 421650 (0.0010) [2023-12-26 18:25:53,392][105620] Updated weights for policy 1, policy_version 421660 (0.0009) [2023-12-26 18:25:53,446][105620] Updated weights for policy 1, policy_version 421671 (0.0010) [2023-12-26 18:25:53,790][105692] Updated weights for policy 0, policy_version 421199 (0.0010) [2023-12-26 18:25:53,851][105692] Updated weights for policy 0, policy_version 421209 (0.0010) [2023-12-26 18:25:53,910][105692] Updated weights for policy 0, policy_version 421219 (0.0010) [2023-12-26 18:25:54,087][105620] Updated weights for policy 1, policy_version 421681 (0.0009) [2023-12-26 18:25:54,154][105620] Updated weights for policy 1, policy_version 421691 (0.0009) [2023-12-26 18:25:54,210][105620] Updated weights for policy 1, policy_version 421701 (0.0008) [2023-12-26 18:25:54,266][105620] Updated weights for policy 1, policy_version 421711 (0.0008) [2023-12-26 18:25:54,711][105692] Updated weights for policy 0, policy_version 421229 (0.0009) [2023-12-26 18:25:54,758][105692] Updated weights for policy 0, policy_version 421239 (0.0009) [2023-12-26 18:25:54,820][105692] Updated weights for policy 0, policy_version 421249 (0.0009) [2023-12-26 18:25:54,992][105620] Updated weights for policy 1, policy_version 421721 (0.0009) [2023-12-26 18:25:55,044][105620] Updated weights for policy 1, policy_version 421731 (0.0009) [2023-12-26 18:25:55,096][105620] Updated weights for policy 1, policy_version 421741 (0.0008) [2023-12-26 18:25:55,570][105692] Updated weights for policy 0, policy_version 421259 (0.0008) [2023-12-26 18:25:55,620][105692] Updated weights for policy 0, policy_version 421269 (0.0005) [2023-12-26 18:25:55,673][105692] Updated weights for policy 0, policy_version 421279 (0.0005) [2023-12-26 18:25:55,753][105620] Updated weights for policy 1, policy_version 421751 (0.0008) [2023-12-26 18:25:55,806][105620] Updated weights for policy 1, policy_version 421761 (0.0008) [2023-12-26 18:25:55,863][105620] Updated weights for policy 1, policy_version 421771 (0.0005) [2023-12-26 18:25:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.7, 300 sec: 19327.6). Total num frames: 215851008. Throughput: 0: 9980.2, 1: 9725.8. Samples: 215856728. Policy #0 lag: (min: 24.0, avg: 50.6, max: 56.0) [2023-12-26 18:25:56,063][104569] Avg episode reward: [(0, '9080.901'), (1, '8800.683')] [2023-12-26 18:25:56,214][105692] Updated weights for policy 0, policy_version 421289 (0.0007) [2023-12-26 18:25:56,268][105692] Updated weights for policy 0, policy_version 421299 (0.0009) [2023-12-26 18:25:56,272][105585] KL-divergence is very high: 191.3948 [2023-12-26 18:25:56,318][105585] KL-divergence is very high: 291.8618 [2023-12-26 18:25:56,322][105692] Updated weights for policy 0, policy_version 421309 (0.0010) [2023-12-26 18:25:56,371][105585] KL-divergence is very high: 265.3165 [2023-12-26 18:25:56,390][105692] Updated weights for policy 0, policy_version 421319 (0.0010) [2023-12-26 18:25:56,523][105620] Updated weights for policy 1, policy_version 421781 (0.0007) [2023-12-26 18:25:56,579][105620] Updated weights for policy 1, policy_version 421791 (0.0005) [2023-12-26 18:25:56,634][105620] Updated weights for policy 1, policy_version 421801 (0.0005) [2023-12-26 18:25:57,140][105692] Updated weights for policy 0, policy_version 421329 (0.0010) [2023-12-26 18:25:57,183][105620] Updated weights for policy 1, policy_version 421811 (0.0006) [2023-12-26 18:25:57,185][105692] Updated weights for policy 0, policy_version 421340 (0.0007) [2023-12-26 18:25:57,231][105692] Updated weights for policy 0, policy_version 421350 (0.0008) [2023-12-26 18:25:57,237][105620] Updated weights for policy 1, policy_version 421821 (0.0007) [2023-12-26 18:25:57,297][105620] Updated weights for policy 1, policy_version 421831 (0.0006) [2023-12-26 18:25:58,012][105692] Updated weights for policy 0, policy_version 421360 (0.0009) [2023-12-26 18:25:58,012][105585] KL-divergence is very high: 120.9626 [2023-12-26 18:25:58,031][105585] KL-divergence is very high: 291.1730 [2023-12-26 18:25:58,037][105620] Updated weights for policy 1, policy_version 421841 (0.0008) [2023-12-26 18:25:58,060][105585] KL-divergence is very high: 151.0781 [2023-12-26 18:25:58,070][105692] Updated weights for policy 0, policy_version 421370 (0.0010) [2023-12-26 18:25:58,076][105585] KL-divergence is very high: 282.2048 [2023-12-26 18:25:58,098][105620] Updated weights for policy 1, policy_version 421851 (0.0006) [2023-12-26 18:25:58,101][105585] KL-divergence is very high: 110.1358 [2023-12-26 18:25:58,123][105585] KL-divergence is very high: 153.7847 [2023-12-26 18:25:58,127][105692] Updated weights for policy 0, policy_version 421380 (0.0011) [2023-12-26 18:25:58,160][105620] Updated weights for policy 1, policy_version 421861 (0.0008) [2023-12-26 18:25:58,220][105620] Updated weights for policy 1, policy_version 421871 (0.0009) [2023-12-26 18:25:58,942][105692] Updated weights for policy 0, policy_version 421390 (0.0009) [2023-12-26 18:25:59,009][105692] Updated weights for policy 0, policy_version 421400 (0.0008) [2023-12-26 18:25:59,011][105620] Updated weights for policy 1, policy_version 421881 (0.0006) [2023-12-26 18:25:59,061][105692] Updated weights for policy 0, policy_version 421410 (0.0008) [2023-12-26 18:25:59,068][105620] Updated weights for policy 1, policy_version 421891 (0.0006) [2023-12-26 18:25:59,129][105620] Updated weights for policy 1, policy_version 421901 (0.0007) [2023-12-26 18:25:59,854][105620] Updated weights for policy 1, policy_version 421911 (0.0007) [2023-12-26 18:25:59,868][105692] Updated weights for policy 0, policy_version 421420 (0.0009) [2023-12-26 18:25:59,919][105620] Updated weights for policy 1, policy_version 421921 (0.0006) [2023-12-26 18:25:59,923][105692] Updated weights for policy 0, policy_version 421430 (0.0010) [2023-12-26 18:25:59,972][105692] Updated weights for policy 0, policy_version 421440 (0.0010) [2023-12-26 18:25:59,983][105620] Updated weights for policy 1, policy_version 421931 (0.0009) [2023-12-26 18:26:00,611][105692] Updated weights for policy 0, policy_version 421450 (0.0010) [2023-12-26 18:26:00,667][105692] Updated weights for policy 0, policy_version 421460 (0.0005) [2023-12-26 18:26:00,668][105620] Updated weights for policy 1, policy_version 421941 (0.0008) [2023-12-26 18:26:00,716][105620] Updated weights for policy 1, policy_version 421951 (0.0007) [2023-12-26 18:26:00,723][105692] Updated weights for policy 0, policy_version 421470 (0.0005) [2023-12-26 18:26:00,757][105586] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000009 [2023-12-26 18:26:00,781][105692] Updated weights for policy 0, policy_version 421480 (0.0009) [2023-12-26 18:26:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19327.6). Total num frames: 215949312. Throughput: 0: 10011.0, 1: 9756.3. Samples: 215917124. Policy #0 lag: (min: 24.0, avg: 50.6, max: 56.0) [2023-12-26 18:26:01,063][104569] Avg episode reward: [(0, '9078.090'), (1, '8802.696')] [2023-12-26 18:26:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000421480_107913216.pth... [2023-12-26 18:26:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000421960_108036096.pth... [2023-12-26 18:26:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000420328_107618304.pth [2023-12-26 18:26:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000420848_107749376.pth [2023-12-26 18:26:01,429][105620] Updated weights for policy 1, policy_version 421961 (0.0008) [2023-12-26 18:26:01,483][105692] Updated weights for policy 0, policy_version 421490 (0.0009) [2023-12-26 18:26:01,488][105620] Updated weights for policy 1, policy_version 421971 (0.0008) [2023-12-26 18:26:01,537][105692] Updated weights for policy 0, policy_version 421500 (0.0007) [2023-12-26 18:26:01,554][105620] Updated weights for policy 1, policy_version 421981 (0.0008) [2023-12-26 18:26:01,592][105692] Updated weights for policy 0, policy_version 421510 (0.0007) [2023-12-26 18:26:01,620][105620] Updated weights for policy 1, policy_version 421991 (0.0008) [2023-12-26 18:26:02,350][105620] Updated weights for policy 1, policy_version 422001 (0.0008) [2023-12-26 18:26:02,389][105692] Updated weights for policy 0, policy_version 421520 (0.0009) [2023-12-26 18:26:02,412][105620] Updated weights for policy 1, policy_version 422011 (0.0008) [2023-12-26 18:26:02,439][105692] Updated weights for policy 0, policy_version 421530 (0.0006) [2023-12-26 18:26:02,471][105620] Updated weights for policy 1, policy_version 422021 (0.0008) [2023-12-26 18:26:02,495][105692] Updated weights for policy 0, policy_version 421540 (0.0006) [2023-12-26 18:26:03,184][105620] Updated weights for policy 1, policy_version 422031 (0.0008) [2023-12-26 18:26:03,198][105692] Updated weights for policy 0, policy_version 421550 (0.0007) [2023-12-26 18:26:03,236][105620] Updated weights for policy 1, policy_version 422041 (0.0008) [2023-12-26 18:26:03,246][105692] Updated weights for policy 0, policy_version 421560 (0.0006) [2023-12-26 18:26:03,288][105620] Updated weights for policy 1, policy_version 422051 (0.0007) [2023-12-26 18:26:03,298][105692] Updated weights for policy 0, policy_version 421570 (0.0006) [2023-12-26 18:26:03,972][105692] Updated weights for policy 0, policy_version 421580 (0.0007) [2023-12-26 18:26:04,030][105692] Updated weights for policy 0, policy_version 421590 (0.0009) [2023-12-26 18:26:04,077][105620] Updated weights for policy 1, policy_version 422061 (0.0006) [2023-12-26 18:26:04,092][105692] Updated weights for policy 0, policy_version 421600 (0.0008) [2023-12-26 18:26:04,144][105620] Updated weights for policy 1, policy_version 422071 (0.0007) [2023-12-26 18:26:04,207][105620] Updated weights for policy 1, policy_version 422081 (0.0009) [2023-12-26 18:26:04,864][105692] Updated weights for policy 0, policy_version 421610 (0.0006) [2023-12-26 18:26:04,912][105692] Updated weights for policy 0, policy_version 421620 (0.0009) [2023-12-26 18:26:04,958][105692] Updated weights for policy 0, policy_version 421630 (0.0008) [2023-12-26 18:26:04,981][105620] Updated weights for policy 1, policy_version 422091 (0.0009) [2023-12-26 18:26:05,004][105692] Updated weights for policy 0, policy_version 421640 (0.0006) [2023-12-26 18:26:05,041][105620] Updated weights for policy 1, policy_version 422101 (0.0008) [2023-12-26 18:26:05,107][105620] Updated weights for policy 1, policy_version 422111 (0.0008) [2023-12-26 18:26:05,790][105692] Updated weights for policy 0, policy_version 421650 (0.0005) [2023-12-26 18:26:05,842][105692] Updated weights for policy 0, policy_version 421660 (0.0005) [2023-12-26 18:26:05,872][105620] Updated weights for policy 1, policy_version 422121 (0.0009) [2023-12-26 18:26:05,892][105692] Updated weights for policy 0, policy_version 421670 (0.0005) [2023-12-26 18:26:05,935][105620] Updated weights for policy 1, policy_version 422131 (0.0010) [2023-12-26 18:26:06,002][105620] Updated weights for policy 1, policy_version 422141 (0.0010) [2023-12-26 18:26:06,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19660.8, 300 sec: 19327.6). Total num frames: 216039424. Throughput: 0: 10000.3, 1: 9642.9. Samples: 216032000. Policy #0 lag: (min: 24.0, avg: 50.6, max: 56.0) [2023-12-26 18:26:06,062][104569] Avg episode reward: [(0, '8893.788'), (1, '8802.797')] [2023-12-26 18:26:06,069][105620] Updated weights for policy 1, policy_version 422151 (0.0009) [2023-12-26 18:26:06,489][105692] Updated weights for policy 0, policy_version 421680 (0.0005) [2023-12-26 18:26:06,516][105585] KL-divergence is very high: 110.8514 [2023-12-26 18:26:06,543][105585] KL-divergence is very high: 296.0739 [2023-12-26 18:26:06,549][105585] KL-divergence is very high: 150.6398 [2023-12-26 18:26:06,555][105692] Updated weights for policy 0, policy_version 421690 (0.0008) [2023-12-26 18:26:06,568][105585] KL-divergence is very high: 283.6999 [2023-12-26 18:26:06,596][105585] KL-divergence is very high: 431.2519 [2023-12-26 18:26:06,603][105585] KL-divergence is very high: 188.4775 [2023-12-26 18:26:06,621][105692] Updated weights for policy 0, policy_version 421700 (0.0006) [2023-12-26 18:26:06,624][105585] KL-divergence is very high: 276.9231 [2023-12-26 18:26:06,889][105620] Updated weights for policy 1, policy_version 422161 (0.0010) [2023-12-26 18:26:06,946][105620] Updated weights for policy 1, policy_version 422171 (0.0009) [2023-12-26 18:26:07,017][105620] Updated weights for policy 1, policy_version 422181 (0.0010) [2023-12-26 18:26:07,231][105692] Updated weights for policy 0, policy_version 421710 (0.0008) [2023-12-26 18:26:07,298][105692] Updated weights for policy 0, policy_version 421720 (0.0007) [2023-12-26 18:26:07,351][105692] Updated weights for policy 0, policy_version 421730 (0.0008) [2023-12-26 18:26:07,762][105620] Updated weights for policy 1, policy_version 422191 (0.0009) [2023-12-26 18:26:07,819][105620] Updated weights for policy 1, policy_version 422201 (0.0009) [2023-12-26 18:26:07,880][105620] Updated weights for policy 1, policy_version 422211 (0.0008) [2023-12-26 18:26:07,995][105692] Updated weights for policy 0, policy_version 421740 (0.0007) [2023-12-26 18:26:08,050][105692] Updated weights for policy 0, policy_version 421751 (0.0010) [2023-12-26 18:26:08,101][105692] Updated weights for policy 0, policy_version 421762 (0.0008) [2023-12-26 18:26:08,499][105620] Updated weights for policy 1, policy_version 422221 (0.0009) [2023-12-26 18:26:08,551][105620] Updated weights for policy 1, policy_version 422231 (0.0009) [2023-12-26 18:26:08,617][105620] Updated weights for policy 1, policy_version 422241 (0.0007) [2023-12-26 18:26:08,933][105692] Updated weights for policy 0, policy_version 421772 (0.0009) [2023-12-26 18:26:08,998][105692] Updated weights for policy 0, policy_version 421782 (0.0009) [2023-12-26 18:26:09,064][105692] Updated weights for policy 0, policy_version 421792 (0.0009) [2023-12-26 18:26:09,354][105620] Updated weights for policy 1, policy_version 422251 (0.0008) [2023-12-26 18:26:09,421][105620] Updated weights for policy 1, policy_version 422261 (0.0008) [2023-12-26 18:26:09,487][105620] Updated weights for policy 1, policy_version 422271 (0.0009) [2023-12-26 18:26:09,868][105692] Updated weights for policy 0, policy_version 421802 (0.0009) [2023-12-26 18:26:09,929][105692] Updated weights for policy 0, policy_version 421812 (0.0009) [2023-12-26 18:26:09,989][105692] Updated weights for policy 0, policy_version 421822 (0.0011) [2023-12-26 18:26:10,049][105692] Updated weights for policy 0, policy_version 421832 (0.0006) [2023-12-26 18:26:10,166][105620] Updated weights for policy 1, policy_version 422281 (0.0010) [2023-12-26 18:26:10,228][105620] Updated weights for policy 1, policy_version 422291 (0.0009) [2023-12-26 18:26:10,286][105620] Updated weights for policy 1, policy_version 422301 (0.0010) [2023-12-26 18:26:10,352][105620] Updated weights for policy 1, policy_version 422311 (0.0010) [2023-12-26 18:26:10,644][105692] Updated weights for policy 0, policy_version 421842 (0.0011) [2023-12-26 18:26:10,704][105692] Updated weights for policy 0, policy_version 421852 (0.0011) [2023-12-26 18:26:10,756][105692] Updated weights for policy 0, policy_version 421862 (0.0011) [2023-12-26 18:26:11,020][105620] Updated weights for policy 1, policy_version 422321 (0.0008) [2023-12-26 18:26:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19327.6). Total num frames: 216137728. Throughput: 0: 9963.9, 1: 9668.0. Samples: 216148956. Policy #0 lag: (min: 24.0, avg: 50.6, max: 56.0) [2023-12-26 18:26:11,062][104569] Avg episode reward: [(0, '8809.025'), (1, '8710.534')] [2023-12-26 18:26:11,100][105620] Updated weights for policy 1, policy_version 422331 (0.0008) [2023-12-26 18:26:11,167][105620] Updated weights for policy 1, policy_version 422341 (0.0008) [2023-12-26 18:26:11,541][105692] Updated weights for policy 0, policy_version 421872 (0.0011) [2023-12-26 18:26:11,601][105692] Updated weights for policy 0, policy_version 421882 (0.0011) [2023-12-26 18:26:11,669][105692] Updated weights for policy 0, policy_version 421892 (0.0009) [2023-12-26 18:26:12,011][105620] Updated weights for policy 1, policy_version 422351 (0.0009) [2023-12-26 18:26:12,067][105620] Updated weights for policy 1, policy_version 422361 (0.0009) [2023-12-26 18:26:12,119][105620] Updated weights for policy 1, policy_version 422371 (0.0010) [2023-12-26 18:26:12,356][105692] Updated weights for policy 0, policy_version 421902 (0.0009) [2023-12-26 18:26:12,423][105692] Updated weights for policy 0, policy_version 421912 (0.0006) [2023-12-26 18:26:12,484][105692] Updated weights for policy 0, policy_version 421922 (0.0006) [2023-12-26 18:26:12,955][105620] Updated weights for policy 1, policy_version 422381 (0.0008) [2023-12-26 18:26:13,020][105620] Updated weights for policy 1, policy_version 422391 (0.0006) [2023-12-26 18:26:13,079][105620] Updated weights for policy 1, policy_version 422401 (0.0009) [2023-12-26 18:26:13,103][105692] Updated weights for policy 0, policy_version 421932 (0.0007) [2023-12-26 18:26:13,176][105692] Updated weights for policy 0, policy_version 421942 (0.0007) [2023-12-26 18:26:13,242][105692] Updated weights for policy 0, policy_version 421952 (0.0005) [2023-12-26 18:26:13,795][105620] Updated weights for policy 1, policy_version 422411 (0.0008) [2023-12-26 18:26:13,857][105620] Updated weights for policy 1, policy_version 422421 (0.0010) [2023-12-26 18:26:13,874][105586] KL-divergence is very high: 103.3960 [2023-12-26 18:26:13,915][105620] Updated weights for policy 1, policy_version 422431 (0.0010) [2023-12-26 18:26:13,921][105586] KL-divergence is very high: 262.2769 [2023-12-26 18:26:13,934][105586] KL-divergence is very high: 191.4429 [2023-12-26 18:26:13,941][105692] Updated weights for policy 0, policy_version 421962 (0.0006) [2023-12-26 18:26:13,946][105586] KL-divergence is very high: 237.0478 [2023-12-26 18:26:13,990][105692] Updated weights for policy 0, policy_version 421972 (0.0007) [2023-12-26 18:26:14,038][105692] Updated weights for policy 0, policy_version 421982 (0.0008) [2023-12-26 18:26:14,086][105692] Updated weights for policy 0, policy_version 421992 (0.0008) [2023-12-26 18:26:14,668][105620] Updated weights for policy 1, policy_version 422441 (0.0010) [2023-12-26 18:26:14,742][105620] Updated weights for policy 1, policy_version 422451 (0.0010) [2023-12-26 18:26:14,811][105620] Updated weights for policy 1, policy_version 422461 (0.0011) [2023-12-26 18:26:14,864][105620] Updated weights for policy 1, policy_version 422471 (0.0011) [2023-12-26 18:26:14,882][105692] Updated weights for policy 0, policy_version 422002 (0.0006) [2023-12-26 18:26:14,938][105692] Updated weights for policy 0, policy_version 422012 (0.0008) [2023-12-26 18:26:14,991][105692] Updated weights for policy 0, policy_version 422022 (0.0008) [2023-12-26 18:26:15,618][105620] Updated weights for policy 1, policy_version 422481 (0.0011) [2023-12-26 18:26:15,677][105620] Updated weights for policy 1, policy_version 422491 (0.0011) [2023-12-26 18:26:15,726][105620] Updated weights for policy 1, policy_version 422501 (0.0011) [2023-12-26 18:26:15,766][105692] Updated weights for policy 0, policy_version 422032 (0.0008) [2023-12-26 18:26:15,818][105692] Updated weights for policy 0, policy_version 422042 (0.0008) [2023-12-26 18:26:15,870][105692] Updated weights for policy 0, policy_version 422052 (0.0008) [2023-12-26 18:26:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.9, 300 sec: 19355.3). Total num frames: 216236032. Throughput: 0: 9858.6, 1: 9582.5. Samples: 216204916. Policy #0 lag: (min: 24.0, avg: 50.6, max: 56.0) [2023-12-26 18:26:16,062][104569] Avg episode reward: [(0, '8808.019'), (1, '8615.941')] [2023-12-26 18:26:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000422056_108060672.pth... [2023-12-26 18:26:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000422504_108175360.pth... [2023-12-26 18:26:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000420904_107765760.pth [2023-12-26 18:26:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000421392_107888640.pth [2023-12-26 18:26:16,478][105620] Updated weights for policy 1, policy_version 422511 (0.0011) [2023-12-26 18:26:16,543][105620] Updated weights for policy 1, policy_version 422521 (0.0010) [2023-12-26 18:26:16,591][105620] Updated weights for policy 1, policy_version 422531 (0.0010) [2023-12-26 18:26:16,640][105692] Updated weights for policy 0, policy_version 422062 (0.0008) [2023-12-26 18:26:16,684][105692] Updated weights for policy 0, policy_version 422072 (0.0007) [2023-12-26 18:26:16,732][105692] Updated weights for policy 0, policy_version 422082 (0.0008) [2023-12-26 18:26:17,345][105620] Updated weights for policy 1, policy_version 422541 (0.0010) [2023-12-26 18:26:17,400][105620] Updated weights for policy 1, policy_version 422551 (0.0010) [2023-12-26 18:26:17,448][105620] Updated weights for policy 1, policy_version 422561 (0.0010) [2023-12-26 18:26:17,498][105692] Updated weights for policy 0, policy_version 422092 (0.0008) [2023-12-26 18:26:17,546][105692] Updated weights for policy 0, policy_version 422102 (0.0008) [2023-12-26 18:26:17,594][105692] Updated weights for policy 0, policy_version 422112 (0.0008) [2023-12-26 18:26:18,151][105620] Updated weights for policy 1, policy_version 422571 (0.0009) [2023-12-26 18:26:18,199][105620] Updated weights for policy 1, policy_version 422581 (0.0006) [2023-12-26 18:26:18,252][105620] Updated weights for policy 1, policy_version 422591 (0.0008) [2023-12-26 18:26:18,398][105692] Updated weights for policy 0, policy_version 422122 (0.0007) [2023-12-26 18:26:18,450][105692] Updated weights for policy 0, policy_version 422132 (0.0008) [2023-12-26 18:26:18,503][105692] Updated weights for policy 0, policy_version 422142 (0.0008) [2023-12-26 18:26:18,556][105692] Updated weights for policy 0, policy_version 422152 (0.0008) [2023-12-26 18:26:18,948][105620] Updated weights for policy 1, policy_version 422601 (0.0010) [2023-12-26 18:26:19,000][105620] Updated weights for policy 1, policy_version 422611 (0.0010) [2023-12-26 18:26:19,052][105620] Updated weights for policy 1, policy_version 422621 (0.0010) [2023-12-26 18:26:19,106][105620] Updated weights for policy 1, policy_version 422631 (0.0008) [2023-12-26 18:26:19,231][105692] Updated weights for policy 0, policy_version 422162 (0.0007) [2023-12-26 18:26:19,297][105692] Updated weights for policy 0, policy_version 422172 (0.0008) [2023-12-26 18:26:19,365][105692] Updated weights for policy 0, policy_version 422182 (0.0007) [2023-12-26 18:26:19,837][105620] Updated weights for policy 1, policy_version 422641 (0.0007) [2023-12-26 18:26:19,909][105620] Updated weights for policy 1, policy_version 422651 (0.0009) [2023-12-26 18:26:19,971][105620] Updated weights for policy 1, policy_version 422661 (0.0009) [2023-12-26 18:26:20,093][105692] Updated weights for policy 0, policy_version 422192 (0.0008) [2023-12-26 18:26:20,154][105692] Updated weights for policy 0, policy_version 422202 (0.0009) [2023-12-26 18:26:20,210][105692] Updated weights for policy 0, policy_version 422212 (0.0009) [2023-12-26 18:26:20,750][105620] Updated weights for policy 1, policy_version 422671 (0.0008) [2023-12-26 18:26:20,816][105620] Updated weights for policy 1, policy_version 422681 (0.0008) [2023-12-26 18:26:20,881][105620] Updated weights for policy 1, policy_version 422691 (0.0009) [2023-12-26 18:26:20,922][105692] Updated weights for policy 0, policy_version 422222 (0.0007) [2023-12-26 18:26:20,978][105692] Updated weights for policy 0, policy_version 422232 (0.0006) [2023-12-26 18:26:21,039][105692] Updated weights for policy 0, policy_version 422242 (0.0009) [2023-12-26 18:26:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19327.6). Total num frames: 216326144. Throughput: 0: 9796.1, 1: 9525.4. Samples: 216318228. Policy #0 lag: (min: 24.0, avg: 50.6, max: 56.0) [2023-12-26 18:26:21,063][104569] Avg episode reward: [(0, '9173.753'), (1, '8894.906')] [2023-12-26 18:26:21,647][105620] Updated weights for policy 1, policy_version 422701 (0.0009) [2023-12-26 18:26:21,710][105620] Updated weights for policy 1, policy_version 422711 (0.0007) [2023-12-26 18:26:21,780][105620] Updated weights for policy 1, policy_version 422721 (0.0006) [2023-12-26 18:26:21,856][105692] Updated weights for policy 0, policy_version 422252 (0.0008) [2023-12-26 18:26:21,909][105692] Updated weights for policy 0, policy_version 422262 (0.0009) [2023-12-26 18:26:21,962][105692] Updated weights for policy 0, policy_version 422272 (0.0010) [2023-12-26 18:26:22,433][105620] Updated weights for policy 1, policy_version 422731 (0.0008) [2023-12-26 18:26:22,492][105620] Updated weights for policy 1, policy_version 422741 (0.0009) [2023-12-26 18:26:22,543][105620] Updated weights for policy 1, policy_version 422751 (0.0009) [2023-12-26 18:26:22,770][105692] Updated weights for policy 0, policy_version 422282 (0.0008) [2023-12-26 18:26:22,823][105692] Updated weights for policy 0, policy_version 422292 (0.0009) [2023-12-26 18:26:22,879][105692] Updated weights for policy 0, policy_version 422302 (0.0009) [2023-12-26 18:26:22,937][105692] Updated weights for policy 0, policy_version 422312 (0.0009) [2023-12-26 18:26:23,314][105620] Updated weights for policy 1, policy_version 422761 (0.0009) [2023-12-26 18:26:23,368][105620] Updated weights for policy 1, policy_version 422771 (0.0009) [2023-12-26 18:26:23,422][105620] Updated weights for policy 1, policy_version 422782 (0.0009) [2023-12-26 18:26:23,476][105620] Updated weights for policy 1, policy_version 422792 (0.0010) [2023-12-26 18:26:23,654][105692] Updated weights for policy 0, policy_version 422322 (0.0009) [2023-12-26 18:26:23,701][105692] Updated weights for policy 0, policy_version 422332 (0.0009) [2023-12-26 18:26:23,752][105692] Updated weights for policy 0, policy_version 422342 (0.0009) [2023-12-26 18:26:24,200][105620] Updated weights for policy 1, policy_version 422802 (0.0009) [2023-12-26 18:26:24,264][105620] Updated weights for policy 1, policy_version 422812 (0.0009) [2023-12-26 18:26:24,325][105620] Updated weights for policy 1, policy_version 422822 (0.0009) [2023-12-26 18:26:24,556][105692] Updated weights for policy 0, policy_version 422353 (0.0010) [2023-12-26 18:26:24,608][105692] Updated weights for policy 0, policy_version 422363 (0.0010) [2023-12-26 18:26:24,659][105692] Updated weights for policy 0, policy_version 422373 (0.0009) [2023-12-26 18:26:25,000][105620] Updated weights for policy 1, policy_version 422832 (0.0006) [2023-12-26 18:26:25,057][105620] Updated weights for policy 1, policy_version 422842 (0.0009) [2023-12-26 18:26:25,114][105620] Updated weights for policy 1, policy_version 422852 (0.0008) [2023-12-26 18:26:25,466][105692] Updated weights for policy 0, policy_version 422383 (0.0008) [2023-12-26 18:26:25,524][105692] Updated weights for policy 0, policy_version 422393 (0.0009) [2023-12-26 18:26:25,579][105692] Updated weights for policy 0, policy_version 422403 (0.0007) [2023-12-26 18:26:25,846][105620] Updated weights for policy 1, policy_version 422862 (0.0009) [2023-12-26 18:26:25,897][105620] Updated weights for policy 1, policy_version 422872 (0.0009) [2023-12-26 18:26:25,945][105620] Updated weights for policy 1, policy_version 422882 (0.0009) [2023-12-26 18:26:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19299.8). Total num frames: 216424448. Throughput: 0: 9641.1, 1: 9537.5. Samples: 216430424. Policy #0 lag: (min: 24.0, avg: 50.6, max: 56.0) [2023-12-26 18:26:26,062][104569] Avg episode reward: [(0, '9174.074'), (1, '8986.830')] [2023-12-26 18:26:26,335][105692] Updated weights for policy 0, policy_version 422413 (0.0009) [2023-12-26 18:26:26,397][105692] Updated weights for policy 0, policy_version 422423 (0.0009) [2023-12-26 18:26:26,456][105692] Updated weights for policy 0, policy_version 422433 (0.0008) [2023-12-26 18:26:26,636][105620] Updated weights for policy 1, policy_version 422892 (0.0008) [2023-12-26 18:26:26,693][105620] Updated weights for policy 1, policy_version 422902 (0.0009) [2023-12-26 18:26:26,749][105620] Updated weights for policy 1, policy_version 422912 (0.0010) [2023-12-26 18:26:27,194][105692] Updated weights for policy 0, policy_version 422443 (0.0009) [2023-12-26 18:26:27,251][105692] Updated weights for policy 0, policy_version 422453 (0.0009) [2023-12-26 18:26:27,311][105692] Updated weights for policy 0, policy_version 422463 (0.0009) [2023-12-26 18:26:27,530][105620] Updated weights for policy 1, policy_version 422922 (0.0009) [2023-12-26 18:26:27,581][105620] Updated weights for policy 1, policy_version 422932 (0.0009) [2023-12-26 18:26:27,639][105620] Updated weights for policy 1, policy_version 422944 (0.0010) [2023-12-26 18:26:27,868][105692] Updated weights for policy 0, policy_version 422473 (0.0007) [2023-12-26 18:26:27,928][105692] Updated weights for policy 0, policy_version 422483 (0.0008) [2023-12-26 18:26:27,993][105692] Updated weights for policy 0, policy_version 422493 (0.0009) [2023-12-26 18:26:28,042][105692] Updated weights for policy 0, policy_version 422503 (0.0005) [2023-12-26 18:26:28,312][105620] Updated weights for policy 1, policy_version 422954 (0.0009) [2023-12-26 18:26:28,378][105620] Updated weights for policy 1, policy_version 422964 (0.0006) [2023-12-26 18:26:28,439][105620] Updated weights for policy 1, policy_version 422974 (0.0006) [2023-12-26 18:26:28,506][105620] Updated weights for policy 1, policy_version 422984 (0.0006) [2023-12-26 18:26:28,847][105692] Updated weights for policy 0, policy_version 422514 (0.0010) [2023-12-26 18:26:28,901][105692] Updated weights for policy 0, policy_version 422525 (0.0010) [2023-12-26 18:26:28,953][105692] Updated weights for policy 0, policy_version 422535 (0.0009) [2023-12-26 18:26:29,019][105620] Updated weights for policy 1, policy_version 422994 (0.0008) [2023-12-26 18:26:29,070][105620] Updated weights for policy 1, policy_version 423004 (0.0008) [2023-12-26 18:26:29,127][105620] Updated weights for policy 1, policy_version 423014 (0.0008) [2023-12-26 18:26:29,750][105692] Updated weights for policy 0, policy_version 422545 (0.0009) [2023-12-26 18:26:29,800][105692] Updated weights for policy 0, policy_version 422555 (0.0009) [2023-12-26 18:26:29,849][105692] Updated weights for policy 0, policy_version 422565 (0.0008) [2023-12-26 18:26:29,902][105620] Updated weights for policy 1, policy_version 423024 (0.0008) [2023-12-26 18:26:29,960][105620] Updated weights for policy 1, policy_version 423034 (0.0007) [2023-12-26 18:26:30,014][105620] Updated weights for policy 1, policy_version 423044 (0.0009) [2023-12-26 18:26:30,631][105692] Updated weights for policy 0, policy_version 422575 (0.0009) [2023-12-26 18:26:30,686][105692] Updated weights for policy 0, policy_version 422585 (0.0010) [2023-12-26 18:26:30,730][105620] Updated weights for policy 1, policy_version 423054 (0.0007) [2023-12-26 18:26:30,744][105692] Updated weights for policy 0, policy_version 422595 (0.0009) [2023-12-26 18:26:30,784][105620] Updated weights for policy 1, policy_version 423064 (0.0005) [2023-12-26 18:26:30,837][105620] Updated weights for policy 1, policy_version 423074 (0.0005) [2023-12-26 18:26:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19299.8). Total num frames: 216522752. Throughput: 0: 9669.3, 1: 9632.2. Samples: 216491244. Policy #0 lag: (min: 24.0, avg: 50.6, max: 56.0) [2023-12-26 18:26:31,062][104569] Avg episode reward: [(0, '8991.500'), (1, '8986.525')] [2023-12-26 18:26:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000423080_108322816.pth... [2023-12-26 18:26:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000422600_108199936.pth... [2023-12-26 18:26:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000421480_107913216.pth [2023-12-26 18:26:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000421960_108036096.pth [2023-12-26 18:26:31,505][105620] Updated weights for policy 1, policy_version 423084 (0.0007) [2023-12-26 18:26:31,560][105620] Updated weights for policy 1, policy_version 423094 (0.0009) [2023-12-26 18:26:31,560][105692] Updated weights for policy 0, policy_version 422605 (0.0007) [2023-12-26 18:26:31,613][105620] Updated weights for policy 1, policy_version 423104 (0.0010) [2023-12-26 18:26:31,615][105692] Updated weights for policy 0, policy_version 422615 (0.0009) [2023-12-26 18:26:31,673][105692] Updated weights for policy 0, policy_version 422625 (0.0008) [2023-12-26 18:26:32,396][105620] Updated weights for policy 1, policy_version 423114 (0.0007) [2023-12-26 18:26:32,439][105692] Updated weights for policy 0, policy_version 422635 (0.0009) [2023-12-26 18:26:32,454][105620] Updated weights for policy 1, policy_version 423124 (0.0008) [2023-12-26 18:26:32,497][105692] Updated weights for policy 0, policy_version 422645 (0.0006) [2023-12-26 18:26:32,508][105620] Updated weights for policy 1, policy_version 423134 (0.0006) [2023-12-26 18:26:32,555][105692] Updated weights for policy 0, policy_version 422655 (0.0007) [2023-12-26 18:26:32,560][105620] Updated weights for policy 1, policy_version 423144 (0.0005) [2023-12-26 18:26:33,226][105620] Updated weights for policy 1, policy_version 423154 (0.0009) [2023-12-26 18:26:33,272][105620] Updated weights for policy 1, policy_version 423164 (0.0009) [2023-12-26 18:26:33,319][105620] Updated weights for policy 1, policy_version 423174 (0.0009) [2023-12-26 18:26:33,342][105692] Updated weights for policy 0, policy_version 422665 (0.0010) [2023-12-26 18:26:33,400][105692] Updated weights for policy 0, policy_version 422675 (0.0009) [2023-12-26 18:26:33,447][105692] Updated weights for policy 0, policy_version 422685 (0.0008) [2023-12-26 18:26:33,502][105692] Updated weights for policy 0, policy_version 422695 (0.0009) [2023-12-26 18:26:34,031][105620] Updated weights for policy 1, policy_version 423184 (0.0006) [2023-12-26 18:26:34,101][105620] Updated weights for policy 1, policy_version 423194 (0.0008) [2023-12-26 18:26:34,164][105620] Updated weights for policy 1, policy_version 423204 (0.0008) [2023-12-26 18:26:34,236][105692] Updated weights for policy 0, policy_version 422705 (0.0010) [2023-12-26 18:26:34,294][105692] Updated weights for policy 0, policy_version 422715 (0.0009) [2023-12-26 18:26:34,351][105692] Updated weights for policy 0, policy_version 422725 (0.0009) [2023-12-26 18:26:34,854][105620] Updated weights for policy 1, policy_version 423214 (0.0010) [2023-12-26 18:26:34,913][105620] Updated weights for policy 1, policy_version 423224 (0.0010) [2023-12-26 18:26:34,976][105620] Updated weights for policy 1, policy_version 423234 (0.0010) [2023-12-26 18:26:35,056][105692] Updated weights for policy 0, policy_version 422735 (0.0010) [2023-12-26 18:26:35,111][105692] Updated weights for policy 0, policy_version 422745 (0.0010) [2023-12-26 18:26:35,174][105692] Updated weights for policy 0, policy_version 422755 (0.0008) [2023-12-26 18:26:35,598][105620] Updated weights for policy 1, policy_version 423244 (0.0010) [2023-12-26 18:26:35,657][105620] Updated weights for policy 1, policy_version 423254 (0.0010) [2023-12-26 18:26:35,718][105620] Updated weights for policy 1, policy_version 423264 (0.0009) [2023-12-26 18:26:35,835][105692] Updated weights for policy 0, policy_version 422765 (0.0010) [2023-12-26 18:26:35,903][105692] Updated weights for policy 0, policy_version 422775 (0.0010) [2023-12-26 18:26:35,968][105692] Updated weights for policy 0, policy_version 422785 (0.0011) [2023-12-26 18:26:36,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19387.7, 300 sec: 19327.6). Total num frames: 216621056. Throughput: 0: 9518.5, 1: 9674.5. Samples: 216604360. Policy #0 lag: (min: 24.0, avg: 50.6, max: 56.0) [2023-12-26 18:26:36,063][104569] Avg episode reward: [(0, '8990.381'), (1, '8986.575')] [2023-12-26 18:26:36,389][105620] Updated weights for policy 1, policy_version 423274 (0.0008) [2023-12-26 18:26:36,451][105620] Updated weights for policy 1, policy_version 423284 (0.0006) [2023-12-26 18:26:36,512][105620] Updated weights for policy 1, policy_version 423294 (0.0006) [2023-12-26 18:26:36,567][105620] Updated weights for policy 1, policy_version 423304 (0.0008) [2023-12-26 18:26:36,651][105692] Updated weights for policy 0, policy_version 422795 (0.0010) [2023-12-26 18:26:36,712][105692] Updated weights for policy 0, policy_version 422805 (0.0009) [2023-12-26 18:26:36,771][105692] Updated weights for policy 0, policy_version 422815 (0.0011) [2023-12-26 18:26:37,133][105620] Updated weights for policy 1, policy_version 423314 (0.0006) [2023-12-26 18:26:37,180][105620] Updated weights for policy 1, policy_version 423324 (0.0005) [2023-12-26 18:26:37,226][105620] Updated weights for policy 1, policy_version 423334 (0.0005) [2023-12-26 18:26:37,535][105692] Updated weights for policy 0, policy_version 422825 (0.0009) [2023-12-26 18:26:37,599][105692] Updated weights for policy 0, policy_version 422835 (0.0008) [2023-12-26 18:26:37,659][105692] Updated weights for policy 0, policy_version 422845 (0.0006) [2023-12-26 18:26:37,722][105692] Updated weights for policy 0, policy_version 422855 (0.0008) [2023-12-26 18:26:37,898][105620] Updated weights for policy 1, policy_version 423344 (0.0010) [2023-12-26 18:26:37,959][105620] Updated weights for policy 1, policy_version 423354 (0.0009) [2023-12-26 18:26:38,022][105620] Updated weights for policy 1, policy_version 423364 (0.0006) [2023-12-26 18:26:38,343][105692] Updated weights for policy 0, policy_version 422865 (0.0008) [2023-12-26 18:26:38,408][105692] Updated weights for policy 0, policy_version 422875 (0.0010) [2023-12-26 18:26:38,467][105692] Updated weights for policy 0, policy_version 422885 (0.0005) [2023-12-26 18:26:38,648][105620] Updated weights for policy 1, policy_version 423374 (0.0008) [2023-12-26 18:26:38,710][105620] Updated weights for policy 1, policy_version 423384 (0.0011) [2023-12-26 18:26:38,773][105620] Updated weights for policy 1, policy_version 423394 (0.0011) [2023-12-26 18:26:39,046][105692] Updated weights for policy 0, policy_version 422895 (0.0006) [2023-12-26 18:26:39,107][105692] Updated weights for policy 0, policy_version 422905 (0.0007) [2023-12-26 18:26:39,162][105692] Updated weights for policy 0, policy_version 422915 (0.0008) [2023-12-26 18:26:39,520][105620] Updated weights for policy 1, policy_version 423404 (0.0009) [2023-12-26 18:26:39,580][105620] Updated weights for policy 1, policy_version 423414 (0.0008) [2023-12-26 18:26:39,632][105620] Updated weights for policy 1, policy_version 423424 (0.0010) [2023-12-26 18:26:39,901][105692] Updated weights for policy 0, policy_version 422925 (0.0009) [2023-12-26 18:26:39,965][105692] Updated weights for policy 0, policy_version 422935 (0.0008) [2023-12-26 18:26:40,025][105692] Updated weights for policy 0, policy_version 422945 (0.0008) [2023-12-26 18:26:40,390][105620] Updated weights for policy 1, policy_version 423434 (0.0011) [2023-12-26 18:26:40,450][105620] Updated weights for policy 1, policy_version 423444 (0.0011) [2023-12-26 18:26:40,516][105620] Updated weights for policy 1, policy_version 423454 (0.0011) [2023-12-26 18:26:40,572][105620] Updated weights for policy 1, policy_version 423464 (0.0011) [2023-12-26 18:26:40,793][105692] Updated weights for policy 0, policy_version 422955 (0.0008) [2023-12-26 18:26:40,859][105692] Updated weights for policy 0, policy_version 422965 (0.0005) [2023-12-26 18:26:40,916][105692] Updated weights for policy 0, policy_version 422975 (0.0005) [2023-12-26 18:26:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19327.6). Total num frames: 216719360. Throughput: 0: 9586.9, 1: 9714.4. Samples: 216725284. Policy #0 lag: (min: 24.0, avg: 50.6, max: 56.0) [2023-12-26 18:26:41,062][104569] Avg episode reward: [(0, '9176.059'), (1, '8986.123')] [2023-12-26 18:26:41,343][105620] Updated weights for policy 1, policy_version 423474 (0.0009) [2023-12-26 18:26:41,417][105620] Updated weights for policy 1, policy_version 423484 (0.0009) [2023-12-26 18:26:41,479][105620] Updated weights for policy 1, policy_version 423494 (0.0008) [2023-12-26 18:26:41,657][105692] Updated weights for policy 0, policy_version 422985 (0.0006) [2023-12-26 18:26:41,734][105692] Updated weights for policy 0, policy_version 422995 (0.0008) [2023-12-26 18:26:41,793][105692] Updated weights for policy 0, policy_version 423005 (0.0008) [2023-12-26 18:26:41,851][105692] Updated weights for policy 0, policy_version 423015 (0.0008) [2023-12-26 18:26:42,208][105620] Updated weights for policy 1, policy_version 423504 (0.0005) [2023-12-26 18:26:42,264][105620] Updated weights for policy 1, policy_version 423514 (0.0007) [2023-12-26 18:26:42,326][105620] Updated weights for policy 1, policy_version 423524 (0.0009) [2023-12-26 18:26:42,634][105692] Updated weights for policy 0, policy_version 423025 (0.0006) [2023-12-26 18:26:42,699][105692] Updated weights for policy 0, policy_version 423035 (0.0005) [2023-12-26 18:26:42,758][105692] Updated weights for policy 0, policy_version 423045 (0.0006) [2023-12-26 18:26:43,012][105620] Updated weights for policy 1, policy_version 423534 (0.0009) [2023-12-26 18:26:43,068][105620] Updated weights for policy 1, policy_version 423544 (0.0009) [2023-12-26 18:26:43,120][105620] Updated weights for policy 1, policy_version 423554 (0.0008) [2023-12-26 18:26:43,393][105692] Updated weights for policy 0, policy_version 423055 (0.0006) [2023-12-26 18:26:43,449][105692] Updated weights for policy 0, policy_version 423065 (0.0005) [2023-12-26 18:26:43,502][105692] Updated weights for policy 0, policy_version 423075 (0.0005) [2023-12-26 18:26:43,827][105620] Updated weights for policy 1, policy_version 423564 (0.0007) [2023-12-26 18:26:43,887][105620] Updated weights for policy 1, policy_version 423574 (0.0008) [2023-12-26 18:26:43,953][105620] Updated weights for policy 1, policy_version 423584 (0.0008) [2023-12-26 18:26:44,109][105692] Updated weights for policy 0, policy_version 423085 (0.0005) [2023-12-26 18:26:44,162][105692] Updated weights for policy 0, policy_version 423095 (0.0006) [2023-12-26 18:26:44,208][105692] Updated weights for policy 0, policy_version 423105 (0.0005) [2023-12-26 18:26:44,755][105620] Updated weights for policy 1, policy_version 423594 (0.0009) [2023-12-26 18:26:44,811][105620] Updated weights for policy 1, policy_version 423604 (0.0009) [2023-12-26 18:26:44,882][105620] Updated weights for policy 1, policy_version 423614 (0.0009) [2023-12-26 18:26:44,890][105692] Updated weights for policy 0, policy_version 423116 (0.0006) [2023-12-26 18:26:44,947][105620] Updated weights for policy 1, policy_version 423624 (0.0009) [2023-12-26 18:26:44,948][105692] Updated weights for policy 0, policy_version 423126 (0.0009) [2023-12-26 18:26:45,012][105692] Updated weights for policy 0, policy_version 423136 (0.0011) [2023-12-26 18:26:45,734][105620] Updated weights for policy 1, policy_version 423634 (0.0008) [2023-12-26 18:26:45,747][105692] Updated weights for policy 0, policy_version 423146 (0.0010) [2023-12-26 18:26:45,792][105620] Updated weights for policy 1, policy_version 423644 (0.0006) [2023-12-26 18:26:45,805][105692] Updated weights for policy 0, policy_version 423156 (0.0010) [2023-12-26 18:26:45,846][105620] Updated weights for policy 1, policy_version 423654 (0.0005) [2023-12-26 18:26:45,863][105692] Updated weights for policy 0, policy_version 423166 (0.0010) [2023-12-26 18:26:45,924][105692] Updated weights for policy 0, policy_version 423176 (0.0010) [2023-12-26 18:26:46,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.8, 300 sec: 19355.3). Total num frames: 216817664. Throughput: 0: 9572.0, 1: 9669.4. Samples: 216782988. Policy #0 lag: (min: 24.0, avg: 50.6, max: 56.0) [2023-12-26 18:26:46,062][104569] Avg episode reward: [(0, '9085.308'), (1, '9077.480')] [2023-12-26 18:26:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000423176_108347392.pth... [2023-12-26 18:26:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000423656_108470272.pth... [2023-12-26 18:26:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000422056_108060672.pth [2023-12-26 18:26:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000422504_108175360.pth [2023-12-26 18:26:46,610][105620] Updated weights for policy 1, policy_version 423664 (0.0006) [2023-12-26 18:26:46,633][105692] Updated weights for policy 0, policy_version 423186 (0.0011) [2023-12-26 18:26:46,670][105620] Updated weights for policy 1, policy_version 423674 (0.0006) [2023-12-26 18:26:46,685][105692] Updated weights for policy 0, policy_version 423196 (0.0010) [2023-12-26 18:26:46,728][105620] Updated weights for policy 1, policy_version 423684 (0.0006) [2023-12-26 18:26:46,736][105692] Updated weights for policy 0, policy_version 423206 (0.0007) [2023-12-26 18:26:47,273][105620] Updated weights for policy 1, policy_version 423694 (0.0006) [2023-12-26 18:26:47,291][105692] Updated weights for policy 0, policy_version 423216 (0.0005) [2023-12-26 18:26:47,326][105620] Updated weights for policy 1, policy_version 423704 (0.0006) [2023-12-26 18:26:47,344][105692] Updated weights for policy 0, policy_version 423226 (0.0005) [2023-12-26 18:26:47,386][105620] Updated weights for policy 1, policy_version 423714 (0.0006) [2023-12-26 18:26:47,395][105692] Updated weights for policy 0, policy_version 423236 (0.0005) [2023-12-26 18:26:48,013][105692] Updated weights for policy 0, policy_version 423246 (0.0010) [2023-12-26 18:26:48,039][105620] Updated weights for policy 1, policy_version 423724 (0.0008) [2023-12-26 18:26:48,064][105692] Updated weights for policy 0, policy_version 423256 (0.0010) [2023-12-26 18:26:48,087][105620] Updated weights for policy 1, policy_version 423734 (0.0010) [2023-12-26 18:26:48,108][105692] Updated weights for policy 0, policy_version 423266 (0.0010) [2023-12-26 18:26:48,138][105620] Updated weights for policy 1, policy_version 423744 (0.0010) [2023-12-26 18:26:48,818][105692] Updated weights for policy 0, policy_version 423276 (0.0010) [2023-12-26 18:26:48,870][105620] Updated weights for policy 1, policy_version 423754 (0.0009) [2023-12-26 18:26:48,883][105692] Updated weights for policy 0, policy_version 423286 (0.0010) [2023-12-26 18:26:48,929][105620] Updated weights for policy 1, policy_version 423764 (0.0006) [2023-12-26 18:26:48,945][105692] Updated weights for policy 0, policy_version 423296 (0.0010) [2023-12-26 18:26:48,988][105620] Updated weights for policy 1, policy_version 423774 (0.0006) [2023-12-26 18:26:49,045][105620] Updated weights for policy 1, policy_version 423784 (0.0005) [2023-12-26 18:26:49,693][105692] Updated weights for policy 0, policy_version 423306 (0.0010) [2023-12-26 18:26:49,744][105692] Updated weights for policy 0, policy_version 423316 (0.0009) [2023-12-26 18:26:49,798][105620] Updated weights for policy 1, policy_version 423794 (0.0009) [2023-12-26 18:26:49,801][105692] Updated weights for policy 0, policy_version 423326 (0.0005) [2023-12-26 18:26:49,862][105620] Updated weights for policy 1, policy_version 423804 (0.0008) [2023-12-26 18:26:49,864][105692] Updated weights for policy 0, policy_version 423336 (0.0008) [2023-12-26 18:26:49,924][105620] Updated weights for policy 1, policy_version 423814 (0.0009) [2023-12-26 18:26:50,475][105692] Updated weights for policy 0, policy_version 423346 (0.0006) [2023-12-26 18:26:50,539][105692] Updated weights for policy 0, policy_version 423356 (0.0010) [2023-12-26 18:26:50,612][105692] Updated weights for policy 0, policy_version 423366 (0.0008) [2023-12-26 18:26:50,753][105620] Updated weights for policy 1, policy_version 423824 (0.0008) [2023-12-26 18:26:50,815][105620] Updated weights for policy 1, policy_version 423834 (0.0009) [2023-12-26 18:26:50,873][105620] Updated weights for policy 1, policy_version 423844 (0.0009) [2023-12-26 18:26:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19327.6). Total num frames: 216915968. Throughput: 0: 9676.9, 1: 9684.1. Samples: 216903244. Policy #0 lag: (min: 24.0, avg: 50.6, max: 56.0) [2023-12-26 18:26:51,063][104569] Avg episode reward: [(0, '9175.339'), (1, '9077.447')] [2023-12-26 18:26:51,260][105692] Updated weights for policy 0, policy_version 423376 (0.0007) [2023-12-26 18:26:51,318][105692] Updated weights for policy 0, policy_version 423386 (0.0008) [2023-12-26 18:26:51,386][105692] Updated weights for policy 0, policy_version 423396 (0.0008) [2023-12-26 18:26:51,684][105620] Updated weights for policy 1, policy_version 423854 (0.0009) [2023-12-26 18:26:51,752][105620] Updated weights for policy 1, policy_version 423864 (0.0008) [2023-12-26 18:26:51,824][105620] Updated weights for policy 1, policy_version 423874 (0.0006) [2023-12-26 18:26:52,094][105692] Updated weights for policy 0, policy_version 423406 (0.0006) [2023-12-26 18:26:52,147][105692] Updated weights for policy 0, policy_version 423416 (0.0005) [2023-12-26 18:26:52,197][105692] Updated weights for policy 0, policy_version 423426 (0.0006) [2023-12-26 18:26:52,606][105620] Updated weights for policy 1, policy_version 423884 (0.0009) [2023-12-26 18:26:52,669][105620] Updated weights for policy 1, policy_version 423894 (0.0009) [2023-12-26 18:26:52,735][105620] Updated weights for policy 1, policy_version 423904 (0.0009) [2023-12-26 18:26:52,908][105692] Updated weights for policy 0, policy_version 423436 (0.0009) [2023-12-26 18:26:52,959][105692] Updated weights for policy 0, policy_version 423446 (0.0009) [2023-12-26 18:26:53,014][105692] Updated weights for policy 0, policy_version 423456 (0.0009) [2023-12-26 18:26:53,475][105620] Updated weights for policy 1, policy_version 423914 (0.0009) [2023-12-26 18:26:53,524][105620] Updated weights for policy 1, policy_version 423924 (0.0007) [2023-12-26 18:26:53,584][105620] Updated weights for policy 1, policy_version 423934 (0.0005) [2023-12-26 18:26:53,631][105620] Updated weights for policy 1, policy_version 423944 (0.0008) [2023-12-26 18:26:53,744][105692] Updated weights for policy 0, policy_version 423466 (0.0009) [2023-12-26 18:26:53,801][105692] Updated weights for policy 0, policy_version 423476 (0.0009) [2023-12-26 18:26:53,833][105585] KL-divergence is very high: 100.2744 [2023-12-26 18:26:53,864][105692] Updated weights for policy 0, policy_version 423486 (0.0008) [2023-12-26 18:26:53,880][105585] KL-divergence is very high: 102.8865 [2023-12-26 18:26:53,914][105692] Updated weights for policy 0, policy_version 423496 (0.0009) [2023-12-26 18:26:54,386][105620] Updated weights for policy 1, policy_version 423956 (0.0011) [2023-12-26 18:26:54,440][105620] Updated weights for policy 1, policy_version 423967 (0.0010) [2023-12-26 18:26:54,548][105692] Updated weights for policy 0, policy_version 423506 (0.0007) [2023-12-26 18:26:54,602][105692] Updated weights for policy 0, policy_version 423516 (0.0009) [2023-12-26 18:26:54,653][105692] Updated weights for policy 0, policy_version 423526 (0.0010) [2023-12-26 18:26:55,204][105620] Updated weights for policy 1, policy_version 423979 (0.0010) [2023-12-26 18:26:55,256][105620] Updated weights for policy 1, policy_version 423989 (0.0010) [2023-12-26 18:26:55,310][105620] Updated weights for policy 1, policy_version 424000 (0.0010) [2023-12-26 18:26:55,373][105692] Updated weights for policy 0, policy_version 423536 (0.0006) [2023-12-26 18:26:55,421][105692] Updated weights for policy 0, policy_version 423546 (0.0007) [2023-12-26 18:26:55,478][105692] Updated weights for policy 0, policy_version 423556 (0.0008) [2023-12-26 18:26:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.3, 300 sec: 19272.0). Total num frames: 217006080. Throughput: 0: 9696.6, 1: 9628.8. Samples: 217018600. Policy #0 lag: (min: 24.0, avg: 50.6, max: 56.0) [2023-12-26 18:26:56,062][104569] Avg episode reward: [(0, '9175.722'), (1, '8984.602')] [2023-12-26 18:26:56,078][105620] Updated weights for policy 1, policy_version 424010 (0.0009) [2023-12-26 18:26:56,132][105620] Updated weights for policy 1, policy_version 424020 (0.0009) [2023-12-26 18:26:56,182][105620] Updated weights for policy 1, policy_version 424030 (0.0008) [2023-12-26 18:26:56,206][105692] Updated weights for policy 0, policy_version 423566 (0.0008) [2023-12-26 18:26:56,236][105620] Updated weights for policy 1, policy_version 424040 (0.0006) [2023-12-26 18:26:56,263][105692] Updated weights for policy 0, policy_version 423576 (0.0007) [2023-12-26 18:26:56,324][105692] Updated weights for policy 0, policy_version 423586 (0.0009) [2023-12-26 18:26:56,885][105620] Updated weights for policy 1, policy_version 424050 (0.0009) [2023-12-26 18:26:56,933][105620] Updated weights for policy 1, policy_version 424060 (0.0009) [2023-12-26 18:26:56,981][105620] Updated weights for policy 1, policy_version 424070 (0.0009) [2023-12-26 18:26:57,097][105692] Updated weights for policy 0, policy_version 423596 (0.0009) [2023-12-26 18:26:57,148][105692] Updated weights for policy 0, policy_version 423606 (0.0009) [2023-12-26 18:26:57,203][105692] Updated weights for policy 0, policy_version 423616 (0.0009) [2023-12-26 18:26:57,818][105692] Updated weights for policy 0, policy_version 423626 (0.0009) [2023-12-26 18:26:57,828][105620] Updated weights for policy 1, policy_version 424080 (0.0008) [2023-12-26 18:26:57,873][105692] Updated weights for policy 0, policy_version 423636 (0.0006) [2023-12-26 18:26:57,890][105620] Updated weights for policy 1, policy_version 424090 (0.0007) [2023-12-26 18:26:57,932][105692] Updated weights for policy 0, policy_version 423646 (0.0007) [2023-12-26 18:26:57,938][105620] Updated weights for policy 1, policy_version 424100 (0.0006) [2023-12-26 18:26:57,990][105692] Updated weights for policy 0, policy_version 423656 (0.0009) [2023-12-26 18:26:58,707][105620] Updated weights for policy 1, policy_version 424110 (0.0007) [2023-12-26 18:26:58,780][105620] Updated weights for policy 1, policy_version 424120 (0.0007) [2023-12-26 18:26:58,852][105620] Updated weights for policy 1, policy_version 424130 (0.0008) [2023-12-26 18:26:58,859][105692] Updated weights for policy 0, policy_version 423666 (0.0009) [2023-12-26 18:26:58,914][105692] Updated weights for policy 0, policy_version 423676 (0.0010) [2023-12-26 18:26:58,970][105692] Updated weights for policy 0, policy_version 423686 (0.0009) [2023-12-26 18:26:59,573][105620] Updated weights for policy 1, policy_version 424140 (0.0007) [2023-12-26 18:26:59,638][105620] Updated weights for policy 1, policy_version 424150 (0.0008) [2023-12-26 18:26:59,702][105692] Updated weights for policy 0, policy_version 423696 (0.0006) [2023-12-26 18:26:59,706][105620] Updated weights for policy 1, policy_version 424160 (0.0008) [2023-12-26 18:26:59,762][105692] Updated weights for policy 0, policy_version 423706 (0.0006) [2023-12-26 18:26:59,821][105692] Updated weights for policy 0, policy_version 423716 (0.0006) [2023-12-26 18:27:00,437][105620] Updated weights for policy 1, policy_version 424170 (0.0008) [2023-12-26 18:27:00,467][105692] Updated weights for policy 0, policy_version 423726 (0.0008) [2023-12-26 18:27:00,498][105620] Updated weights for policy 1, policy_version 424180 (0.0006) [2023-12-26 18:27:00,512][105692] Updated weights for policy 0, policy_version 423736 (0.0008) [2023-12-26 18:27:00,551][105620] Updated weights for policy 1, policy_version 424190 (0.0008) [2023-12-26 18:27:00,564][105692] Updated weights for policy 0, policy_version 423746 (0.0006) [2023-12-26 18:27:00,599][105620] Updated weights for policy 1, policy_version 424200 (0.0006) [2023-12-26 18:27:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19272.0). Total num frames: 217104384. Throughput: 0: 9683.4, 1: 9658.6. Samples: 217075312. Policy #0 lag: (min: 24.0, avg: 50.6, max: 56.0) [2023-12-26 18:27:01,062][104569] Avg episode reward: [(0, '9175.568'), (1, '8900.321')] [2023-12-26 18:27:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000423752_108494848.pth... [2023-12-26 18:27:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000424200_108609536.pth... [2023-12-26 18:27:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000422600_108199936.pth [2023-12-26 18:27:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000423080_108322816.pth [2023-12-26 18:27:01,227][105620] Updated weights for policy 1, policy_version 424210 (0.0007) [2023-12-26 18:27:01,290][105620] Updated weights for policy 1, policy_version 424220 (0.0006) [2023-12-26 18:27:01,360][105620] Updated weights for policy 1, policy_version 424230 (0.0006) [2023-12-26 18:27:01,371][105692] Updated weights for policy 0, policy_version 423756 (0.0007) [2023-12-26 18:27:01,435][105692] Updated weights for policy 0, policy_version 423766 (0.0009) [2023-12-26 18:27:01,489][105692] Updated weights for policy 0, policy_version 423776 (0.0010) [2023-12-26 18:27:01,941][105620] Updated weights for policy 1, policy_version 424240 (0.0006) [2023-12-26 18:27:01,998][105620] Updated weights for policy 1, policy_version 424250 (0.0008) [2023-12-26 18:27:02,052][105620] Updated weights for policy 1, policy_version 424260 (0.0009) [2023-12-26 18:27:02,246][105692] Updated weights for policy 0, policy_version 423786 (0.0008) [2023-12-26 18:27:02,316][105692] Updated weights for policy 0, policy_version 423796 (0.0006) [2023-12-26 18:27:02,376][105692] Updated weights for policy 0, policy_version 423806 (0.0009) [2023-12-26 18:27:02,433][105692] Updated weights for policy 0, policy_version 423816 (0.0010) [2023-12-26 18:27:02,764][105620] Updated weights for policy 1, policy_version 424270 (0.0007) [2023-12-26 18:27:02,828][105620] Updated weights for policy 1, policy_version 424280 (0.0005) [2023-12-26 18:27:02,892][105620] Updated weights for policy 1, policy_version 424290 (0.0005) [2023-12-26 18:27:03,162][105692] Updated weights for policy 0, policy_version 423826 (0.0008) [2023-12-26 18:27:03,215][105692] Updated weights for policy 0, policy_version 423836 (0.0009) [2023-12-26 18:27:03,265][105692] Updated weights for policy 0, policy_version 423846 (0.0009) [2023-12-26 18:27:03,485][105620] Updated weights for policy 1, policy_version 424300 (0.0006) [2023-12-26 18:27:03,547][105620] Updated weights for policy 1, policy_version 424310 (0.0005) [2023-12-26 18:27:03,604][105620] Updated weights for policy 1, policy_version 424320 (0.0005) [2023-12-26 18:27:04,078][105692] Updated weights for policy 0, policy_version 423856 (0.0008) [2023-12-26 18:27:04,142][105692] Updated weights for policy 0, policy_version 423866 (0.0007) [2023-12-26 18:27:04,181][105620] Updated weights for policy 1, policy_version 424330 (0.0006) [2023-12-26 18:27:04,205][105692] Updated weights for policy 0, policy_version 423876 (0.0006) [2023-12-26 18:27:04,243][105620] Updated weights for policy 1, policy_version 424340 (0.0009) [2023-12-26 18:27:04,305][105620] Updated weights for policy 1, policy_version 424350 (0.0009) [2023-12-26 18:27:04,368][105620] Updated weights for policy 1, policy_version 424360 (0.0010) [2023-12-26 18:27:04,916][105692] Updated weights for policy 0, policy_version 423886 (0.0009) [2023-12-26 18:27:04,977][105692] Updated weights for policy 0, policy_version 423896 (0.0009) [2023-12-26 18:27:05,039][105692] Updated weights for policy 0, policy_version 423906 (0.0009) [2023-12-26 18:27:05,123][105620] Updated weights for policy 1, policy_version 424370 (0.0008) [2023-12-26 18:27:05,185][105620] Updated weights for policy 1, policy_version 424380 (0.0009) [2023-12-26 18:27:05,242][105620] Updated weights for policy 1, policy_version 424390 (0.0009) [2023-12-26 18:27:05,785][105692] Updated weights for policy 0, policy_version 423916 (0.0009) [2023-12-26 18:27:05,838][105692] Updated weights for policy 0, policy_version 423927 (0.0010) [2023-12-26 18:27:05,891][105692] Updated weights for policy 0, policy_version 423938 (0.0010) [2023-12-26 18:27:05,934][105620] Updated weights for policy 1, policy_version 424400 (0.0006) [2023-12-26 18:27:05,989][105620] Updated weights for policy 1, policy_version 424410 (0.0005) [2023-12-26 18:27:06,051][105620] Updated weights for policy 1, policy_version 424420 (0.0005) [2023-12-26 18:27:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19299.8). Total num frames: 217202688. Throughput: 0: 9692.6, 1: 9767.4. Samples: 217193924. Policy #0 lag: (min: 24.0, avg: 50.6, max: 56.0) [2023-12-26 18:27:06,063][104569] Avg episode reward: [(0, '9173.646'), (1, '8716.820')] [2023-12-26 18:27:06,611][105692] Updated weights for policy 0, policy_version 423948 (0.0007) [2023-12-26 18:27:06,674][105692] Updated weights for policy 0, policy_version 423958 (0.0009) [2023-12-26 18:27:06,736][105692] Updated weights for policy 0, policy_version 423968 (0.0009) [2023-12-26 18:27:06,774][105620] Updated weights for policy 1, policy_version 424430 (0.0008) [2023-12-26 18:27:06,831][105620] Updated weights for policy 1, policy_version 424440 (0.0008) [2023-12-26 18:27:06,889][105620] Updated weights for policy 1, policy_version 424450 (0.0010) [2023-12-26 18:27:07,357][105692] Updated weights for policy 0, policy_version 423978 (0.0008) [2023-12-26 18:27:07,415][105692] Updated weights for policy 0, policy_version 423988 (0.0006) [2023-12-26 18:27:07,477][105692] Updated weights for policy 0, policy_version 423998 (0.0006) [2023-12-26 18:27:07,536][105692] Updated weights for policy 0, policy_version 424008 (0.0007) [2023-12-26 18:27:07,603][105620] Updated weights for policy 1, policy_version 424460 (0.0010) [2023-12-26 18:27:07,661][105620] Updated weights for policy 1, policy_version 424470 (0.0008) [2023-12-26 18:27:07,726][105620] Updated weights for policy 1, policy_version 424480 (0.0008) [2023-12-26 18:27:08,198][105692] Updated weights for policy 0, policy_version 424018 (0.0005) [2023-12-26 18:27:08,254][105692] Updated weights for policy 0, policy_version 424028 (0.0005) [2023-12-26 18:27:08,302][105692] Updated weights for policy 0, policy_version 424038 (0.0005) [2023-12-26 18:27:08,514][105620] Updated weights for policy 1, policy_version 424490 (0.0008) [2023-12-26 18:27:08,580][105620] Updated weights for policy 1, policy_version 424500 (0.0008) [2023-12-26 18:27:08,628][105620] Updated weights for policy 1, policy_version 424510 (0.0008) [2023-12-26 18:27:08,690][105620] Updated weights for policy 1, policy_version 424520 (0.0008) [2023-12-26 18:27:08,855][105692] Updated weights for policy 0, policy_version 424048 (0.0006) [2023-12-26 18:27:08,921][105692] Updated weights for policy 0, policy_version 424058 (0.0011) [2023-12-26 18:27:08,980][105692] Updated weights for policy 0, policy_version 424068 (0.0010) [2023-12-26 18:27:09,322][105620] Updated weights for policy 1, policy_version 424530 (0.0006) [2023-12-26 18:27:09,393][105620] Updated weights for policy 1, policy_version 424540 (0.0008) [2023-12-26 18:27:09,464][105620] Updated weights for policy 1, policy_version 424550 (0.0008) [2023-12-26 18:27:09,666][105692] Updated weights for policy 0, policy_version 424078 (0.0008) [2023-12-26 18:27:09,733][105692] Updated weights for policy 0, policy_version 424088 (0.0010) [2023-12-26 18:27:09,797][105692] Updated weights for policy 0, policy_version 424098 (0.0011) [2023-12-26 18:27:10,222][105620] Updated weights for policy 1, policy_version 424560 (0.0008) [2023-12-26 18:27:10,291][105620] Updated weights for policy 1, policy_version 424570 (0.0006) [2023-12-26 18:27:10,350][105620] Updated weights for policy 1, policy_version 424580 (0.0008) [2023-12-26 18:27:10,510][105692] Updated weights for policy 0, policy_version 424108 (0.0008) [2023-12-26 18:27:10,578][105692] Updated weights for policy 0, policy_version 424118 (0.0005) [2023-12-26 18:27:10,631][105692] Updated weights for policy 0, policy_version 424128 (0.0005) [2023-12-26 18:27:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19327.6). Total num frames: 217300992. Throughput: 0: 9818.7, 1: 9780.7. Samples: 217312400. Policy #0 lag: (min: 24.0, avg: 50.6, max: 56.0) [2023-12-26 18:27:11,063][104569] Avg episode reward: [(0, '9266.313'), (1, '8716.839')] [2023-12-26 18:27:11,078][105620] Updated weights for policy 1, policy_version 424590 (0.0009) [2023-12-26 18:27:11,134][105620] Updated weights for policy 1, policy_version 424600 (0.0008) [2023-12-26 18:27:11,202][105620] Updated weights for policy 1, policy_version 424610 (0.0008) [2023-12-26 18:27:11,304][105692] Updated weights for policy 0, policy_version 424138 (0.0006) [2023-12-26 18:27:11,375][105692] Updated weights for policy 0, policy_version 424148 (0.0009) [2023-12-26 18:27:11,429][105692] Updated weights for policy 0, policy_version 424158 (0.0007) [2023-12-26 18:27:11,490][105692] Updated weights for policy 0, policy_version 424168 (0.0005) [2023-12-26 18:27:12,045][105620] Updated weights for policy 1, policy_version 424620 (0.0008) [2023-12-26 18:27:12,098][105620] Updated weights for policy 1, policy_version 424630 (0.0008) [2023-12-26 18:27:12,132][105692] Updated weights for policy 0, policy_version 424178 (0.0009) [2023-12-26 18:27:12,159][105620] Updated weights for policy 1, policy_version 424640 (0.0007) [2023-12-26 18:27:12,191][105692] Updated weights for policy 0, policy_version 424188 (0.0007) [2023-12-26 18:27:12,251][105692] Updated weights for policy 0, policy_version 424198 (0.0008) [2023-12-26 18:27:12,964][105620] Updated weights for policy 1, policy_version 424650 (0.0008) [2023-12-26 18:27:13,001][105692] Updated weights for policy 0, policy_version 424208 (0.0006) [2023-12-26 18:27:13,019][105620] Updated weights for policy 1, policy_version 424660 (0.0008) [2023-12-26 18:27:13,066][105692] Updated weights for policy 0, policy_version 424218 (0.0009) [2023-12-26 18:27:13,072][105620] Updated weights for policy 1, policy_version 424670 (0.0006) [2023-12-26 18:27:13,117][105692] Updated weights for policy 0, policy_version 424228 (0.0007) [2023-12-26 18:27:13,130][105620] Updated weights for policy 1, policy_version 424680 (0.0006) [2023-12-26 18:27:13,795][105620] Updated weights for policy 1, policy_version 424690 (0.0009) [2023-12-26 18:27:13,849][105620] Updated weights for policy 1, policy_version 424700 (0.0009) [2023-12-26 18:27:13,901][105620] Updated weights for policy 1, policy_version 424710 (0.0008) [2023-12-26 18:27:13,921][105692] Updated weights for policy 0, policy_version 424238 (0.0008) [2023-12-26 18:27:13,982][105692] Updated weights for policy 0, policy_version 424248 (0.0009) [2023-12-26 18:27:14,048][105692] Updated weights for policy 0, policy_version 424258 (0.0009) [2023-12-26 18:27:14,620][105620] Updated weights for policy 1, policy_version 424720 (0.0006) [2023-12-26 18:27:14,677][105620] Updated weights for policy 1, policy_version 424730 (0.0006) [2023-12-26 18:27:14,732][105620] Updated weights for policy 1, policy_version 424740 (0.0005) [2023-12-26 18:27:14,817][105692] Updated weights for policy 0, policy_version 424268 (0.0009) [2023-12-26 18:27:14,875][105692] Updated weights for policy 0, policy_version 424278 (0.0010) [2023-12-26 18:27:14,934][105692] Updated weights for policy 0, policy_version 424288 (0.0009) [2023-12-26 18:27:15,439][105620] Updated weights for policy 1, policy_version 424750 (0.0008) [2023-12-26 18:27:15,501][105620] Updated weights for policy 1, policy_version 424760 (0.0009) [2023-12-26 18:27:15,562][105620] Updated weights for policy 1, policy_version 424770 (0.0008) [2023-12-26 18:27:15,648][105692] Updated weights for policy 0, policy_version 424298 (0.0008) [2023-12-26 18:27:15,712][105692] Updated weights for policy 0, policy_version 424308 (0.0009) [2023-12-26 18:27:15,762][105692] Updated weights for policy 0, policy_version 424318 (0.0010) [2023-12-26 18:27:15,809][105692] Updated weights for policy 0, policy_version 424328 (0.0008) [2023-12-26 18:27:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 217399296. Throughput: 0: 9799.6, 1: 9685.1. Samples: 217368060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:27:16,063][104569] Avg episode reward: [(0, '9266.289'), (1, '8709.490')] [2023-12-26 18:27:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000424328_108642304.pth... [2023-12-26 18:27:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000424776_108756992.pth... [2023-12-26 18:27:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000423656_108470272.pth [2023-12-26 18:27:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000423176_108347392.pth [2023-12-26 18:27:16,285][105620] Updated weights for policy 1, policy_version 424780 (0.0009) [2023-12-26 18:27:16,339][105620] Updated weights for policy 1, policy_version 424790 (0.0008) [2023-12-26 18:27:16,386][105620] Updated weights for policy 1, policy_version 424800 (0.0009) [2023-12-26 18:27:16,593][105692] Updated weights for policy 0, policy_version 424338 (0.0009) [2023-12-26 18:27:16,657][105692] Updated weights for policy 0, policy_version 424348 (0.0009) [2023-12-26 18:27:16,713][105692] Updated weights for policy 0, policy_version 424358 (0.0008) [2023-12-26 18:27:17,224][105620] Updated weights for policy 1, policy_version 424810 (0.0008) [2023-12-26 18:27:17,278][105620] Updated weights for policy 1, policy_version 424820 (0.0008) [2023-12-26 18:27:17,301][105692] Updated weights for policy 0, policy_version 424368 (0.0009) [2023-12-26 18:27:17,331][105620] Updated weights for policy 1, policy_version 424830 (0.0008) [2023-12-26 18:27:17,346][105692] Updated weights for policy 0, policy_version 424378 (0.0007) [2023-12-26 18:27:17,384][105620] Updated weights for policy 1, policy_version 424840 (0.0007) [2023-12-26 18:27:17,403][105692] Updated weights for policy 0, policy_version 424388 (0.0005) [2023-12-26 18:27:17,955][105692] Updated weights for policy 0, policy_version 424398 (0.0008) [2023-12-26 18:27:18,006][105692] Updated weights for policy 0, policy_version 424408 (0.0010) [2023-12-26 18:27:18,071][105692] Updated weights for policy 0, policy_version 424418 (0.0010) [2023-12-26 18:27:18,218][105620] Updated weights for policy 1, policy_version 424850 (0.0008) [2023-12-26 18:27:18,277][105620] Updated weights for policy 1, policy_version 424860 (0.0010) [2023-12-26 18:27:18,346][105620] Updated weights for policy 1, policy_version 424870 (0.0009) [2023-12-26 18:27:18,751][105585] KL-divergence is very high: 132.4294 [2023-12-26 18:27:18,776][105692] Updated weights for policy 0, policy_version 424428 (0.0010) [2023-12-26 18:27:18,831][105692] Updated weights for policy 0, policy_version 424438 (0.0010) [2023-12-26 18:27:18,880][105692] Updated weights for policy 0, policy_version 424448 (0.0010) [2023-12-26 18:27:19,104][105620] Updated weights for policy 1, policy_version 424880 (0.0008) [2023-12-26 18:27:19,148][105620] Updated weights for policy 1, policy_version 424890 (0.0008) [2023-12-26 18:27:19,196][105620] Updated weights for policy 1, policy_version 424900 (0.0008) [2023-12-26 18:27:19,661][105692] Updated weights for policy 0, policy_version 424458 (0.0010) [2023-12-26 18:27:19,728][105692] Updated weights for policy 0, policy_version 424468 (0.0009) [2023-12-26 18:27:19,792][105692] Updated weights for policy 0, policy_version 424478 (0.0010) [2023-12-26 18:27:19,856][105692] Updated weights for policy 0, policy_version 424488 (0.0008) [2023-12-26 18:27:19,965][105620] Updated weights for policy 1, policy_version 424910 (0.0009) [2023-12-26 18:27:20,032][105620] Updated weights for policy 1, policy_version 424920 (0.0008) [2023-12-26 18:27:20,095][105620] Updated weights for policy 1, policy_version 424930 (0.0008) [2023-12-26 18:27:20,606][105692] Updated weights for policy 0, policy_version 424498 (0.0009) [2023-12-26 18:27:20,670][105692] Updated weights for policy 0, policy_version 424508 (0.0009) [2023-12-26 18:27:20,725][105692] Updated weights for policy 0, policy_version 424518 (0.0009) [2023-12-26 18:27:20,876][105620] Updated weights for policy 1, policy_version 424940 (0.0009) [2023-12-26 18:27:20,936][105620] Updated weights for policy 1, policy_version 424950 (0.0009) [2023-12-26 18:27:21,002][105620] Updated weights for policy 1, policy_version 424960 (0.0009) [2023-12-26 18:27:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 217497600. Throughput: 0: 9909.4, 1: 9631.4. Samples: 217483692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:27:21,063][104569] Avg episode reward: [(0, '9175.539'), (1, '9171.664')] [2023-12-26 18:27:21,494][105692] Updated weights for policy 0, policy_version 424528 (0.0009) [2023-12-26 18:27:21,561][105692] Updated weights for policy 0, policy_version 424538 (0.0011) [2023-12-26 18:27:21,616][105692] Updated weights for policy 0, policy_version 424548 (0.0013) [2023-12-26 18:27:21,818][105620] Updated weights for policy 1, policy_version 424970 (0.0008) [2023-12-26 18:27:21,869][105620] Updated weights for policy 1, policy_version 424980 (0.0009) [2023-12-26 18:27:21,917][105620] Updated weights for policy 1, policy_version 424990 (0.0009) [2023-12-26 18:27:21,978][105620] Updated weights for policy 1, policy_version 425000 (0.0008) [2023-12-26 18:27:22,327][105692] Updated weights for policy 0, policy_version 424558 (0.0009) [2023-12-26 18:27:22,388][105692] Updated weights for policy 0, policy_version 424568 (0.0009) [2023-12-26 18:27:22,441][105692] Updated weights for policy 0, policy_version 424578 (0.0009) [2023-12-26 18:27:22,783][105620] Updated weights for policy 1, policy_version 425010 (0.0009) [2023-12-26 18:27:22,848][105620] Updated weights for policy 1, policy_version 425020 (0.0008) [2023-12-26 18:27:22,909][105620] Updated weights for policy 1, policy_version 425030 (0.0008) [2023-12-26 18:27:23,233][105692] Updated weights for policy 0, policy_version 424588 (0.0009) [2023-12-26 18:27:23,289][105692] Updated weights for policy 0, policy_version 424598 (0.0009) [2023-12-26 18:27:23,353][105692] Updated weights for policy 0, policy_version 424608 (0.0009) [2023-12-26 18:27:23,671][105620] Updated weights for policy 1, policy_version 425040 (0.0008) [2023-12-26 18:27:23,735][105620] Updated weights for policy 1, policy_version 425050 (0.0008) [2023-12-26 18:27:23,794][105620] Updated weights for policy 1, policy_version 425060 (0.0008) [2023-12-26 18:27:24,068][105692] Updated weights for policy 0, policy_version 424618 (0.0010) [2023-12-26 18:27:24,124][105692] Updated weights for policy 0, policy_version 424628 (0.0010) [2023-12-26 18:27:24,178][105692] Updated weights for policy 0, policy_version 424638 (0.0007) [2023-12-26 18:27:24,235][105692] Updated weights for policy 0, policy_version 424648 (0.0005) [2023-12-26 18:27:24,660][105620] Updated weights for policy 1, policy_version 425070 (0.0009) [2023-12-26 18:27:24,714][105620] Updated weights for policy 1, policy_version 425080 (0.0006) [2023-12-26 18:27:24,763][105620] Updated weights for policy 1, policy_version 425090 (0.0007) [2023-12-26 18:27:24,781][105692] Updated weights for policy 0, policy_version 424658 (0.0006) [2023-12-26 18:27:24,827][105692] Updated weights for policy 0, policy_version 424668 (0.0005) [2023-12-26 18:27:24,871][105692] Updated weights for policy 0, policy_version 424678 (0.0009) [2023-12-26 18:27:25,507][105620] Updated weights for policy 1, policy_version 425100 (0.0009) [2023-12-26 18:27:25,565][105620] Updated weights for policy 1, policy_version 425110 (0.0008) [2023-12-26 18:27:25,595][105692] Updated weights for policy 0, policy_version 424688 (0.0010) [2023-12-26 18:27:25,610][105620] Updated weights for policy 1, policy_version 425120 (0.0006) [2023-12-26 18:27:25,653][105692] Updated weights for policy 0, policy_version 424698 (0.0010) [2023-12-26 18:27:25,714][105692] Updated weights for policy 0, policy_version 424708 (0.0010) [2023-12-26 18:27:26,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 217587712. Throughput: 0: 9884.9, 1: 9459.9. Samples: 217595800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:27:26,062][104569] Avg episode reward: [(0, '9083.230'), (1, '9170.996')] [2023-12-26 18:27:26,358][105620] Updated weights for policy 1, policy_version 425130 (0.0008) [2023-12-26 18:27:26,407][105620] Updated weights for policy 1, policy_version 425140 (0.0005) [2023-12-26 18:27:26,444][105692] Updated weights for policy 0, policy_version 424718 (0.0010) [2023-12-26 18:27:26,459][105620] Updated weights for policy 1, policy_version 425150 (0.0005) [2023-12-26 18:27:26,493][105692] Updated weights for policy 0, policy_version 424728 (0.0010) [2023-12-26 18:27:26,509][105620] Updated weights for policy 1, policy_version 425160 (0.0009) [2023-12-26 18:27:26,550][105692] Updated weights for policy 0, policy_version 424738 (0.0010) [2023-12-26 18:27:27,148][105620] Updated weights for policy 1, policy_version 425170 (0.0010) [2023-12-26 18:27:27,203][105620] Updated weights for policy 1, policy_version 425180 (0.0010) [2023-12-26 18:27:27,258][105620] Updated weights for policy 1, policy_version 425190 (0.0010) [2023-12-26 18:27:27,295][105692] Updated weights for policy 0, policy_version 424748 (0.0010) [2023-12-26 18:27:27,352][105692] Updated weights for policy 0, policy_version 424758 (0.0010) [2023-12-26 18:27:27,402][105692] Updated weights for policy 0, policy_version 424768 (0.0005) [2023-12-26 18:27:27,924][105620] Updated weights for policy 1, policy_version 425200 (0.0008) [2023-12-26 18:27:27,936][105692] Updated weights for policy 0, policy_version 424778 (0.0005) [2023-12-26 18:27:27,976][105620] Updated weights for policy 1, policy_version 425210 (0.0010) [2023-12-26 18:27:27,991][105692] Updated weights for policy 0, policy_version 424788 (0.0010) [2023-12-26 18:27:28,024][105620] Updated weights for policy 1, policy_version 425220 (0.0010) [2023-12-26 18:27:28,045][105692] Updated weights for policy 0, policy_version 424798 (0.0010) [2023-12-26 18:27:28,092][105692] Updated weights for policy 0, policy_version 424808 (0.0010) [2023-12-26 18:27:28,603][105620] Updated weights for policy 1, policy_version 425230 (0.0010) [2023-12-26 18:27:28,655][105620] Updated weights for policy 1, policy_version 425240 (0.0008) [2023-12-26 18:27:28,711][105620] Updated weights for policy 1, policy_version 425250 (0.0010) [2023-12-26 18:27:28,834][105692] Updated weights for policy 0, policy_version 424818 (0.0008) [2023-12-26 18:27:28,886][105692] Updated weights for policy 0, policy_version 424828 (0.0008) [2023-12-26 18:27:28,943][105692] Updated weights for policy 0, policy_version 424838 (0.0008) [2023-12-26 18:27:29,492][105620] Updated weights for policy 1, policy_version 425260 (0.0010) [2023-12-26 18:27:29,557][105620] Updated weights for policy 1, policy_version 425270 (0.0010) [2023-12-26 18:27:29,607][105692] Updated weights for policy 0, policy_version 424848 (0.0010) [2023-12-26 18:27:29,611][105620] Updated weights for policy 1, policy_version 425280 (0.0010) [2023-12-26 18:27:29,662][105692] Updated weights for policy 0, policy_version 424858 (0.0010) [2023-12-26 18:27:29,720][105692] Updated weights for policy 0, policy_version 424868 (0.0010) [2023-12-26 18:27:30,290][105620] Updated weights for policy 1, policy_version 425290 (0.0006) [2023-12-26 18:27:30,345][105620] Updated weights for policy 1, policy_version 425300 (0.0007) [2023-12-26 18:27:30,407][105620] Updated weights for policy 1, policy_version 425310 (0.0011) [2023-12-26 18:27:30,467][105692] Updated weights for policy 0, policy_version 424878 (0.0008) [2023-12-26 18:27:30,469][105620] Updated weights for policy 1, policy_version 425320 (0.0008) [2023-12-26 18:27:30,525][105692] Updated weights for policy 0, policy_version 424888 (0.0005) [2023-12-26 18:27:30,585][105692] Updated weights for policy 0, policy_version 424898 (0.0006) [2023-12-26 18:27:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 217686016. Throughput: 0: 9920.5, 1: 9530.4. Samples: 217658280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:27:31,063][104569] Avg episode reward: [(0, '9266.957'), (1, '9170.997')] [2023-12-26 18:27:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000424904_108789760.pth... [2023-12-26 18:27:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000425320_108896256.pth... [2023-12-26 18:27:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000423752_108494848.pth [2023-12-26 18:27:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000424200_108609536.pth [2023-12-26 18:27:31,129][105692] Updated weights for policy 0, policy_version 424908 (0.0007) [2023-12-26 18:27:31,174][105620] Updated weights for policy 1, policy_version 425330 (0.0006) [2023-12-26 18:27:31,191][105692] Updated weights for policy 0, policy_version 424918 (0.0008) [2023-12-26 18:27:31,231][105620] Updated weights for policy 1, policy_version 425340 (0.0007) [2023-12-26 18:27:31,249][105692] Updated weights for policy 0, policy_version 424928 (0.0006) [2023-12-26 18:27:31,293][105620] Updated weights for policy 1, policy_version 425350 (0.0008) [2023-12-26 18:27:31,932][105692] Updated weights for policy 0, policy_version 424938 (0.0007) [2023-12-26 18:27:31,981][105620] Updated weights for policy 1, policy_version 425360 (0.0008) [2023-12-26 18:27:31,995][105692] Updated weights for policy 0, policy_version 424948 (0.0005) [2023-12-26 18:27:32,045][105620] Updated weights for policy 1, policy_version 425370 (0.0007) [2023-12-26 18:27:32,056][105692] Updated weights for policy 0, policy_version 424958 (0.0006) [2023-12-26 18:27:32,111][105620] Updated weights for policy 1, policy_version 425380 (0.0008) [2023-12-26 18:27:32,116][105692] Updated weights for policy 0, policy_version 424968 (0.0005) [2023-12-26 18:27:32,678][105692] Updated weights for policy 0, policy_version 424978 (0.0005) [2023-12-26 18:27:32,729][105692] Updated weights for policy 0, policy_version 424988 (0.0008) [2023-12-26 18:27:32,744][105620] Updated weights for policy 1, policy_version 425390 (0.0006) [2023-12-26 18:27:32,778][105692] Updated weights for policy 0, policy_version 424998 (0.0010) [2023-12-26 18:27:32,802][105620] Updated weights for policy 1, policy_version 425400 (0.0005) [2023-12-26 18:27:32,859][105620] Updated weights for policy 1, policy_version 425410 (0.0005) [2023-12-26 18:27:33,363][105692] Updated weights for policy 0, policy_version 425008 (0.0006) [2023-12-26 18:27:33,431][105692] Updated weights for policy 0, policy_version 425018 (0.0005) [2023-12-26 18:27:33,451][105620] Updated weights for policy 1, policy_version 425420 (0.0008) [2023-12-26 18:27:33,481][105692] Updated weights for policy 0, policy_version 425028 (0.0010) [2023-12-26 18:27:33,513][105620] Updated weights for policy 1, policy_version 425430 (0.0006) [2023-12-26 18:27:33,571][105620] Updated weights for policy 1, policy_version 425440 (0.0008) [2023-12-26 18:27:34,042][105692] Updated weights for policy 0, policy_version 425038 (0.0007) [2023-12-26 18:27:34,090][105692] Updated weights for policy 0, policy_version 425048 (0.0005) [2023-12-26 18:27:34,154][105692] Updated weights for policy 0, policy_version 425058 (0.0007) [2023-12-26 18:27:34,413][105620] Updated weights for policy 1, policy_version 425450 (0.0008) [2023-12-26 18:27:34,472][105620] Updated weights for policy 1, policy_version 425460 (0.0008) [2023-12-26 18:27:34,536][105620] Updated weights for policy 1, policy_version 425470 (0.0007) [2023-12-26 18:27:34,601][105620] Updated weights for policy 1, policy_version 425480 (0.0006) [2023-12-26 18:27:34,795][105692] Updated weights for policy 0, policy_version 425068 (0.0009) [2023-12-26 18:27:34,855][105692] Updated weights for policy 0, policy_version 425078 (0.0011) [2023-12-26 18:27:34,907][105692] Updated weights for policy 0, policy_version 425088 (0.0010) [2023-12-26 18:27:35,385][105620] Updated weights for policy 1, policy_version 425490 (0.0008) [2023-12-26 18:27:35,443][105620] Updated weights for policy 1, policy_version 425500 (0.0008) [2023-12-26 18:27:35,488][105620] Updated weights for policy 1, policy_version 425510 (0.0008) [2023-12-26 18:27:35,575][105692] Updated weights for policy 0, policy_version 425098 (0.0010) [2023-12-26 18:27:35,634][105692] Updated weights for policy 0, policy_version 425108 (0.0010) [2023-12-26 18:27:35,697][105692] Updated weights for policy 0, policy_version 425118 (0.0010) [2023-12-26 18:27:35,759][105692] Updated weights for policy 0, policy_version 425128 (0.0010) [2023-12-26 18:27:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 217792512. Throughput: 0: 9973.4, 1: 9551.9. Samples: 217781884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:27:36,062][104569] Avg episode reward: [(0, '9268.005'), (1, '9263.207')] [2023-12-26 18:27:36,344][105620] Updated weights for policy 1, policy_version 425520 (0.0007) [2023-12-26 18:27:36,354][105692] Updated weights for policy 0, policy_version 425138 (0.0006) [2023-12-26 18:27:36,396][105620] Updated weights for policy 1, policy_version 425530 (0.0009) [2023-12-26 18:27:36,416][105692] Updated weights for policy 0, policy_version 425148 (0.0006) [2023-12-26 18:27:36,457][105620] Updated weights for policy 1, policy_version 425540 (0.0009) [2023-12-26 18:27:36,476][105692] Updated weights for policy 0, policy_version 425158 (0.0005) [2023-12-26 18:27:37,039][105692] Updated weights for policy 0, policy_version 425168 (0.0005) [2023-12-26 18:27:37,105][105692] Updated weights for policy 0, policy_version 425178 (0.0005) [2023-12-26 18:27:37,164][105692] Updated weights for policy 0, policy_version 425188 (0.0008) [2023-12-26 18:27:37,253][105620] Updated weights for policy 1, policy_version 425550 (0.0007) [2023-12-26 18:27:37,304][105620] Updated weights for policy 1, policy_version 425560 (0.0005) [2023-12-26 18:27:37,352][105620] Updated weights for policy 1, policy_version 425570 (0.0008) [2023-12-26 18:27:37,861][105692] Updated weights for policy 0, policy_version 425198 (0.0011) [2023-12-26 18:27:37,920][105692] Updated weights for policy 0, policy_version 425208 (0.0010) [2023-12-26 18:27:37,978][105692] Updated weights for policy 0, policy_version 425218 (0.0008) [2023-12-26 18:27:38,065][105620] Updated weights for policy 1, policy_version 425580 (0.0007) [2023-12-26 18:27:38,123][105620] Updated weights for policy 1, policy_version 425590 (0.0005) [2023-12-26 18:27:38,180][105620] Updated weights for policy 1, policy_version 425600 (0.0005) [2023-12-26 18:27:38,579][105692] Updated weights for policy 0, policy_version 425228 (0.0008) [2023-12-26 18:27:38,641][105692] Updated weights for policy 0, policy_version 425238 (0.0010) [2023-12-26 18:27:38,703][105692] Updated weights for policy 0, policy_version 425248 (0.0010) [2023-12-26 18:27:38,835][105620] Updated weights for policy 1, policy_version 425610 (0.0006) [2023-12-26 18:27:38,894][105620] Updated weights for policy 1, policy_version 425620 (0.0007) [2023-12-26 18:27:38,965][105620] Updated weights for policy 1, policy_version 425630 (0.0008) [2023-12-26 18:27:39,025][105620] Updated weights for policy 1, policy_version 425640 (0.0008) [2023-12-26 18:27:39,439][105692] Updated weights for policy 0, policy_version 425258 (0.0010) [2023-12-26 18:27:39,502][105692] Updated weights for policy 0, policy_version 425268 (0.0008) [2023-12-26 18:27:39,565][105692] Updated weights for policy 0, policy_version 425278 (0.0008) [2023-12-26 18:27:39,625][105692] Updated weights for policy 0, policy_version 425288 (0.0008) [2023-12-26 18:27:39,789][105620] Updated weights for policy 1, policy_version 425650 (0.0011) [2023-12-26 18:27:39,849][105620] Updated weights for policy 1, policy_version 425660 (0.0011) [2023-12-26 18:27:39,911][105620] Updated weights for policy 1, policy_version 425670 (0.0011) [2023-12-26 18:27:40,303][105692] Updated weights for policy 0, policy_version 425298 (0.0006) [2023-12-26 18:27:40,375][105692] Updated weights for policy 0, policy_version 425308 (0.0006) [2023-12-26 18:27:40,439][105692] Updated weights for policy 0, policy_version 425318 (0.0008) [2023-12-26 18:27:40,647][105620] Updated weights for policy 1, policy_version 425680 (0.0006) [2023-12-26 18:27:40,711][105620] Updated weights for policy 1, policy_version 425690 (0.0008) [2023-12-26 18:27:40,767][105620] Updated weights for policy 1, policy_version 425700 (0.0011) [2023-12-26 18:27:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 217890816. Throughput: 0: 10023.5, 1: 9578.6. Samples: 217900696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:27:41,062][104569] Avg episode reward: [(0, '9268.020'), (1, '9170.320')] [2023-12-26 18:27:41,086][105692] Updated weights for policy 0, policy_version 425328 (0.0009) [2023-12-26 18:27:41,148][105692] Updated weights for policy 0, policy_version 425338 (0.0009) [2023-12-26 18:27:41,205][105692] Updated weights for policy 0, policy_version 425349 (0.0010) [2023-12-26 18:27:41,437][105620] Updated weights for policy 1, policy_version 425710 (0.0008) [2023-12-26 18:27:41,488][105620] Updated weights for policy 1, policy_version 425720 (0.0009) [2023-12-26 18:27:41,547][105620] Updated weights for policy 1, policy_version 425730 (0.0009) [2023-12-26 18:27:41,978][105692] Updated weights for policy 0, policy_version 425359 (0.0009) [2023-12-26 18:27:42,035][105692] Updated weights for policy 0, policy_version 425369 (0.0007) [2023-12-26 18:27:42,094][105692] Updated weights for policy 0, policy_version 425379 (0.0008) [2023-12-26 18:27:42,336][105620] Updated weights for policy 1, policy_version 425740 (0.0010) [2023-12-26 18:27:42,400][105620] Updated weights for policy 1, policy_version 425750 (0.0011) [2023-12-26 18:27:42,466][105620] Updated weights for policy 1, policy_version 425760 (0.0011) [2023-12-26 18:27:42,935][105692] Updated weights for policy 0, policy_version 425389 (0.0008) [2023-12-26 18:27:42,996][105692] Updated weights for policy 0, policy_version 425399 (0.0009) [2023-12-26 18:27:43,064][105692] Updated weights for policy 0, policy_version 425409 (0.0008) [2023-12-26 18:27:43,133][105620] Updated weights for policy 1, policy_version 425770 (0.0008) [2023-12-26 18:27:43,189][105620] Updated weights for policy 1, policy_version 425780 (0.0011) [2023-12-26 18:27:43,252][105620] Updated weights for policy 1, policy_version 425790 (0.0011) [2023-12-26 18:27:43,304][105620] Updated weights for policy 1, policy_version 425800 (0.0010) [2023-12-26 18:27:43,698][105692] Updated weights for policy 0, policy_version 425419 (0.0006) [2023-12-26 18:27:43,759][105692] Updated weights for policy 0, policy_version 425429 (0.0005) [2023-12-26 18:27:43,812][105692] Updated weights for policy 0, policy_version 425439 (0.0008) [2023-12-26 18:27:44,044][105620] Updated weights for policy 1, policy_version 425810 (0.0009) [2023-12-26 18:27:44,101][105620] Updated weights for policy 1, policy_version 425820 (0.0006) [2023-12-26 18:27:44,158][105620] Updated weights for policy 1, policy_version 425830 (0.0005) [2023-12-26 18:27:44,555][105692] Updated weights for policy 0, policy_version 425449 (0.0009) [2023-12-26 18:27:44,612][105692] Updated weights for policy 0, policy_version 425459 (0.0006) [2023-12-26 18:27:44,667][105692] Updated weights for policy 0, policy_version 425469 (0.0009) [2023-12-26 18:27:44,729][105692] Updated weights for policy 0, policy_version 425479 (0.0009) [2023-12-26 18:27:44,770][105620] Updated weights for policy 1, policy_version 425840 (0.0008) [2023-12-26 18:27:44,838][105620] Updated weights for policy 1, policy_version 425850 (0.0009) [2023-12-26 18:27:44,895][105620] Updated weights for policy 1, policy_version 425860 (0.0009) [2023-12-26 18:27:45,533][105692] Updated weights for policy 0, policy_version 425489 (0.0009) [2023-12-26 18:27:45,559][105620] Updated weights for policy 1, policy_version 425870 (0.0008) [2023-12-26 18:27:45,595][105692] Updated weights for policy 0, policy_version 425499 (0.0007) [2023-12-26 18:27:45,619][105620] Updated weights for policy 1, policy_version 425880 (0.0007) [2023-12-26 18:27:45,654][105692] Updated weights for policy 0, policy_version 425509 (0.0009) [2023-12-26 18:27:45,680][105620] Updated weights for policy 1, policy_version 425890 (0.0008) [2023-12-26 18:27:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 217989120. Throughput: 0: 10003.0, 1: 9601.0. Samples: 217957492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:27:46,063][104569] Avg episode reward: [(0, '9266.820'), (1, '9080.034')] [2023-12-26 18:27:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000425512_108945408.pth... [2023-12-26 18:27:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000425896_109043712.pth... [2023-12-26 18:27:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000424328_108642304.pth [2023-12-26 18:27:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000424776_108756992.pth [2023-12-26 18:27:46,322][105620] Updated weights for policy 1, policy_version 425900 (0.0008) [2023-12-26 18:27:46,383][105620] Updated weights for policy 1, policy_version 425910 (0.0005) [2023-12-26 18:27:46,443][105620] Updated weights for policy 1, policy_version 425920 (0.0006) [2023-12-26 18:27:46,473][105692] Updated weights for policy 0, policy_version 425519 (0.0008) [2023-12-26 18:27:46,532][105692] Updated weights for policy 0, policy_version 425529 (0.0009) [2023-12-26 18:27:46,583][105692] Updated weights for policy 0, policy_version 425539 (0.0009) [2023-12-26 18:27:47,040][105620] Updated weights for policy 1, policy_version 425930 (0.0006) [2023-12-26 18:27:47,101][105620] Updated weights for policy 1, policy_version 425940 (0.0009) [2023-12-26 18:27:47,155][105620] Updated weights for policy 1, policy_version 425950 (0.0009) [2023-12-26 18:27:47,202][105620] Updated weights for policy 1, policy_version 425960 (0.0009) [2023-12-26 18:27:47,363][105692] Updated weights for policy 0, policy_version 425549 (0.0010) [2023-12-26 18:27:47,415][105692] Updated weights for policy 0, policy_version 425559 (0.0009) [2023-12-26 18:27:47,467][105692] Updated weights for policy 0, policy_version 425570 (0.0010) [2023-12-26 18:27:47,895][105620] Updated weights for policy 1, policy_version 425970 (0.0009) [2023-12-26 18:27:47,942][105620] Updated weights for policy 1, policy_version 425980 (0.0009) [2023-12-26 18:27:47,989][105620] Updated weights for policy 1, policy_version 425990 (0.0009) [2023-12-26 18:27:48,266][105692] Updated weights for policy 0, policy_version 425580 (0.0009) [2023-12-26 18:27:48,316][105692] Updated weights for policy 0, policy_version 425590 (0.0009) [2023-12-26 18:27:48,375][105692] Updated weights for policy 0, policy_version 425600 (0.0010) [2023-12-26 18:27:48,728][105620] Updated weights for policy 1, policy_version 426000 (0.0008) [2023-12-26 18:27:48,779][105620] Updated weights for policy 1, policy_version 426010 (0.0007) [2023-12-26 18:27:48,833][105620] Updated weights for policy 1, policy_version 426020 (0.0006) [2023-12-26 18:27:49,132][105692] Updated weights for policy 0, policy_version 425610 (0.0010) [2023-12-26 18:27:49,191][105692] Updated weights for policy 0, policy_version 425620 (0.0010) [2023-12-26 18:27:49,255][105692] Updated weights for policy 0, policy_version 425630 (0.0010) [2023-12-26 18:27:49,319][105692] Updated weights for policy 0, policy_version 425640 (0.0011) [2023-12-26 18:27:49,502][105620] Updated weights for policy 1, policy_version 426030 (0.0006) [2023-12-26 18:27:49,570][105620] Updated weights for policy 1, policy_version 426040 (0.0005) [2023-12-26 18:27:49,626][105620] Updated weights for policy 1, policy_version 426050 (0.0005) [2023-12-26 18:27:50,095][105692] Updated weights for policy 0, policy_version 425650 (0.0010) [2023-12-26 18:27:50,155][105692] Updated weights for policy 0, policy_version 425660 (0.0011) [2023-12-26 18:27:50,216][105692] Updated weights for policy 0, policy_version 425670 (0.0008) [2023-12-26 18:27:50,242][105620] Updated weights for policy 1, policy_version 426060 (0.0006) [2023-12-26 18:27:50,305][105620] Updated weights for policy 1, policy_version 426070 (0.0006) [2023-12-26 18:27:50,365][105620] Updated weights for policy 1, policy_version 426080 (0.0007) [2023-12-26 18:27:50,963][105692] Updated weights for policy 0, policy_version 425680 (0.0007) [2023-12-26 18:27:51,018][105692] Updated weights for policy 0, policy_version 425690 (0.0008) [2023-12-26 18:27:51,035][105620] Updated weights for policy 1, policy_version 426090 (0.0006) [2023-12-26 18:27:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 218079232. Throughput: 0: 9959.2, 1: 9623.0. Samples: 218075124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:27:51,063][104569] Avg episode reward: [(0, '9266.449'), (1, '9080.079')] [2023-12-26 18:27:51,082][105692] Updated weights for policy 0, policy_version 425700 (0.0011) [2023-12-26 18:27:51,101][105620] Updated weights for policy 1, policy_version 426100 (0.0008) [2023-12-26 18:27:51,156][105620] Updated weights for policy 1, policy_version 426110 (0.0007) [2023-12-26 18:27:51,206][105620] Updated weights for policy 1, policy_version 426120 (0.0008) [2023-12-26 18:27:51,919][105692] Updated weights for policy 0, policy_version 425710 (0.0009) [2023-12-26 18:27:51,939][105620] Updated weights for policy 1, policy_version 426130 (0.0007) [2023-12-26 18:27:51,964][105692] Updated weights for policy 0, policy_version 425720 (0.0005) [2023-12-26 18:27:51,989][105620] Updated weights for policy 1, policy_version 426140 (0.0007) [2023-12-26 18:27:52,019][105692] Updated weights for policy 0, policy_version 425730 (0.0007) [2023-12-26 18:27:52,050][105620] Updated weights for policy 1, policy_version 426150 (0.0006) [2023-12-26 18:27:52,686][105692] Updated weights for policy 0, policy_version 425740 (0.0006) [2023-12-26 18:27:52,738][105692] Updated weights for policy 0, policy_version 425750 (0.0005) [2023-12-26 18:27:52,786][105692] Updated weights for policy 0, policy_version 425760 (0.0008) [2023-12-26 18:27:52,865][105620] Updated weights for policy 1, policy_version 426160 (0.0008) [2023-12-26 18:27:52,915][105620] Updated weights for policy 1, policy_version 426170 (0.0009) [2023-12-26 18:27:52,966][105620] Updated weights for policy 1, policy_version 426180 (0.0008) [2023-12-26 18:27:53,520][105692] Updated weights for policy 0, policy_version 425770 (0.0009) [2023-12-26 18:27:53,586][105692] Updated weights for policy 0, policy_version 425780 (0.0007) [2023-12-26 18:27:53,656][105692] Updated weights for policy 0, policy_version 425790 (0.0006) [2023-12-26 18:27:53,707][105620] Updated weights for policy 1, policy_version 426190 (0.0008) [2023-12-26 18:27:53,724][105692] Updated weights for policy 0, policy_version 425800 (0.0007) [2023-12-26 18:27:53,770][105620] Updated weights for policy 1, policy_version 426200 (0.0009) [2023-12-26 18:27:53,825][105620] Updated weights for policy 1, policy_version 426210 (0.0010) [2023-12-26 18:27:54,316][105692] Updated weights for policy 0, policy_version 425810 (0.0005) [2023-12-26 18:27:54,376][105692] Updated weights for policy 0, policy_version 425820 (0.0006) [2023-12-26 18:27:54,437][105692] Updated weights for policy 0, policy_version 425830 (0.0006) [2023-12-26 18:27:54,530][105620] Updated weights for policy 1, policy_version 426220 (0.0008) [2023-12-26 18:27:54,585][105620] Updated weights for policy 1, policy_version 426230 (0.0010) [2023-12-26 18:27:54,637][105620] Updated weights for policy 1, policy_version 426240 (0.0010) [2023-12-26 18:27:55,099][105692] Updated weights for policy 0, policy_version 425840 (0.0010) [2023-12-26 18:27:55,157][105692] Updated weights for policy 0, policy_version 425850 (0.0010) [2023-12-26 18:27:55,216][105692] Updated weights for policy 0, policy_version 425860 (0.0010) [2023-12-26 18:27:55,386][105620] Updated weights for policy 1, policy_version 426250 (0.0009) [2023-12-26 18:27:55,458][105620] Updated weights for policy 1, policy_version 426260 (0.0005) [2023-12-26 18:27:55,522][105620] Updated weights for policy 1, policy_version 426270 (0.0005) [2023-12-26 18:27:55,577][105620] Updated weights for policy 1, policy_version 426280 (0.0005) [2023-12-26 18:27:55,934][105692] Updated weights for policy 0, policy_version 425870 (0.0011) [2023-12-26 18:27:55,986][105692] Updated weights for policy 0, policy_version 425880 (0.0010) [2023-12-26 18:27:56,037][105692] Updated weights for policy 0, policy_version 425890 (0.0010) [2023-12-26 18:27:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 218185728. Throughput: 0: 9931.4, 1: 9652.6. Samples: 218193680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:27:56,062][104569] Avg episode reward: [(0, '9358.979'), (1, '9170.340')] [2023-12-26 18:27:56,101][105620] Updated weights for policy 1, policy_version 426290 (0.0007) [2023-12-26 18:27:56,160][105620] Updated weights for policy 1, policy_version 426300 (0.0008) [2023-12-26 18:27:56,221][105620] Updated weights for policy 1, policy_version 426310 (0.0008) [2023-12-26 18:27:56,793][105692] Updated weights for policy 0, policy_version 425900 (0.0010) [2023-12-26 18:27:56,844][105692] Updated weights for policy 0, policy_version 425910 (0.0010) [2023-12-26 18:27:56,896][105620] Updated weights for policy 1, policy_version 426320 (0.0006) [2023-12-26 18:27:56,898][105692] Updated weights for policy 0, policy_version 425920 (0.0010) [2023-12-26 18:27:56,956][105620] Updated weights for policy 1, policy_version 426330 (0.0006) [2023-12-26 18:27:57,017][105620] Updated weights for policy 1, policy_version 426340 (0.0007) [2023-12-26 18:27:57,637][105692] Updated weights for policy 0, policy_version 425930 (0.0010) [2023-12-26 18:27:57,695][105692] Updated weights for policy 0, policy_version 425940 (0.0010) [2023-12-26 18:27:57,746][105692] Updated weights for policy 0, policy_version 425950 (0.0010) [2023-12-26 18:27:57,764][105620] Updated weights for policy 1, policy_version 426350 (0.0008) [2023-12-26 18:27:57,793][105692] Updated weights for policy 0, policy_version 425960 (0.0010) [2023-12-26 18:27:57,827][105620] Updated weights for policy 1, policy_version 426360 (0.0007) [2023-12-26 18:27:57,888][105620] Updated weights for policy 1, policy_version 426370 (0.0008) [2023-12-26 18:27:58,551][105692] Updated weights for policy 0, policy_version 425970 (0.0008) [2023-12-26 18:27:58,614][105692] Updated weights for policy 0, policy_version 425980 (0.0009) [2023-12-26 18:27:58,650][105620] Updated weights for policy 1, policy_version 426380 (0.0007) [2023-12-26 18:27:58,672][105692] Updated weights for policy 0, policy_version 425990 (0.0008) [2023-12-26 18:27:58,715][105620] Updated weights for policy 1, policy_version 426390 (0.0007) [2023-12-26 18:27:58,784][105620] Updated weights for policy 1, policy_version 426400 (0.0007) [2023-12-26 18:27:59,467][105620] Updated weights for policy 1, policy_version 426410 (0.0007) [2023-12-26 18:27:59,477][105692] Updated weights for policy 0, policy_version 426000 (0.0007) [2023-12-26 18:27:59,522][105620] Updated weights for policy 1, policy_version 426420 (0.0008) [2023-12-26 18:27:59,531][105692] Updated weights for policy 0, policy_version 426010 (0.0006) [2023-12-26 18:27:59,583][105620] Updated weights for policy 1, policy_version 426430 (0.0008) [2023-12-26 18:27:59,585][105692] Updated weights for policy 0, policy_version 426020 (0.0007) [2023-12-26 18:27:59,646][105620] Updated weights for policy 1, policy_version 426440 (0.0009) [2023-12-26 18:28:00,352][105692] Updated weights for policy 0, policy_version 426030 (0.0006) [2023-12-26 18:28:00,374][105620] Updated weights for policy 1, policy_version 426450 (0.0008) [2023-12-26 18:28:00,411][105692] Updated weights for policy 0, policy_version 426040 (0.0008) [2023-12-26 18:28:00,428][105620] Updated weights for policy 1, policy_version 426460 (0.0005) [2023-12-26 18:28:00,463][105692] Updated weights for policy 0, policy_version 426050 (0.0009) [2023-12-26 18:28:00,485][105620] Updated weights for policy 1, policy_version 426470 (0.0007) [2023-12-26 18:28:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 218275840. Throughput: 0: 9924.7, 1: 9676.1. Samples: 218250096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:28:01,063][104569] Avg episode reward: [(0, '9359.238'), (1, '9263.404')] [2023-12-26 18:28:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000426472_109191168.pth... [2023-12-26 18:28:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000426056_109084672.pth... [2023-12-26 18:28:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000425320_108896256.pth [2023-12-26 18:28:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000424904_108789760.pth [2023-12-26 18:28:01,220][105692] Updated weights for policy 0, policy_version 426060 (0.0009) [2023-12-26 18:28:01,229][105620] Updated weights for policy 1, policy_version 426480 (0.0010) [2023-12-26 18:28:01,275][105692] Updated weights for policy 0, policy_version 426070 (0.0009) [2023-12-26 18:28:01,284][105620] Updated weights for policy 1, policy_version 426490 (0.0010) [2023-12-26 18:28:01,328][105692] Updated weights for policy 0, policy_version 426080 (0.0010) [2023-12-26 18:28:01,344][105620] Updated weights for policy 1, policy_version 426500 (0.0011) [2023-12-26 18:28:02,020][105620] Updated weights for policy 1, policy_version 426510 (0.0008) [2023-12-26 18:28:02,022][105692] Updated weights for policy 0, policy_version 426090 (0.0008) [2023-12-26 18:28:02,082][105692] Updated weights for policy 0, policy_version 426100 (0.0005) [2023-12-26 18:28:02,088][105620] Updated weights for policy 1, policy_version 426520 (0.0008) [2023-12-26 18:28:02,141][105692] Updated weights for policy 0, policy_version 426110 (0.0006) [2023-12-26 18:28:02,144][105620] Updated weights for policy 1, policy_version 426530 (0.0006) [2023-12-26 18:28:02,208][105692] Updated weights for policy 0, policy_version 426120 (0.0005) [2023-12-26 18:28:02,805][105620] Updated weights for policy 1, policy_version 426540 (0.0007) [2023-12-26 18:28:02,827][105692] Updated weights for policy 0, policy_version 426130 (0.0006) [2023-12-26 18:28:02,858][105620] Updated weights for policy 1, policy_version 426550 (0.0008) [2023-12-26 18:28:02,881][105692] Updated weights for policy 0, policy_version 426140 (0.0008) [2023-12-26 18:28:02,904][105620] Updated weights for policy 1, policy_version 426560 (0.0006) [2023-12-26 18:28:02,929][105692] Updated weights for policy 0, policy_version 426150 (0.0007) [2023-12-26 18:28:03,559][105692] Updated weights for policy 0, policy_version 426160 (0.0005) [2023-12-26 18:28:03,592][105585] KL-divergence is very high: 955.9094 [2023-12-26 18:28:03,618][105692] Updated weights for policy 0, policy_version 426170 (0.0005) [2023-12-26 18:28:03,634][105585] KL-divergence is very high: 1313.1213 [2023-12-26 18:28:03,653][105620] Updated weights for policy 1, policy_version 426570 (0.0010) [2023-12-26 18:28:03,668][105692] Updated weights for policy 0, policy_version 426180 (0.0005) [2023-12-26 18:28:03,674][105585] KL-divergence is very high: 919.9261 [2023-12-26 18:28:03,708][105620] Updated weights for policy 1, policy_version 426580 (0.0011) [2023-12-26 18:28:03,762][105620] Updated weights for policy 1, policy_version 426590 (0.0010) [2023-12-26 18:28:03,818][105620] Updated weights for policy 1, policy_version 426600 (0.0010) [2023-12-26 18:28:04,409][105692] Updated weights for policy 0, policy_version 426190 (0.0007) [2023-12-26 18:28:04,460][105692] Updated weights for policy 0, policy_version 426200 (0.0006) [2023-12-26 18:28:04,461][105620] Updated weights for policy 1, policy_version 426610 (0.0010) [2023-12-26 18:28:04,508][105692] Updated weights for policy 0, policy_version 426210 (0.0006) [2023-12-26 18:28:04,513][105620] Updated weights for policy 1, policy_version 426620 (0.0010) [2023-12-26 18:28:04,564][105620] Updated weights for policy 1, policy_version 426630 (0.0010) [2023-12-26 18:28:05,173][105692] Updated weights for policy 0, policy_version 426220 (0.0007) [2023-12-26 18:28:05,198][105620] Updated weights for policy 1, policy_version 426640 (0.0005) [2023-12-26 18:28:05,222][105692] Updated weights for policy 0, policy_version 426230 (0.0011) [2023-12-26 18:28:05,232][105586] KL-divergence is very high: 133.8873 [2023-12-26 18:28:05,257][105620] Updated weights for policy 1, policy_version 426650 (0.0005) [2023-12-26 18:28:05,264][105692] Updated weights for policy 0, policy_version 426240 (0.0010) [2023-12-26 18:28:05,283][105586] KL-divergence is very high: 156.3601 [2023-12-26 18:28:05,319][105620] Updated weights for policy 1, policy_version 426660 (0.0008) [2023-12-26 18:28:05,331][105586] KL-divergence is very high: 163.1644 [2023-12-26 18:28:05,893][105692] Updated weights for policy 0, policy_version 426250 (0.0009) [2023-12-26 18:28:05,955][105692] Updated weights for policy 0, policy_version 426260 (0.0005) [2023-12-26 18:28:05,994][105586] KL-divergence is very high: 122.8578 [2023-12-26 18:28:06,010][105620] Updated weights for policy 1, policy_version 426670 (0.0011) [2023-12-26 18:28:06,022][105692] Updated weights for policy 0, policy_version 426270 (0.0006) [2023-12-26 18:28:06,049][105586] KL-divergence is very high: 103.6966 [2023-12-26 18:28:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 218374144. Throughput: 0: 9880.9, 1: 9767.4. Samples: 218367864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:28:06,063][104569] Avg episode reward: [(0, '9266.610'), (1, '9263.480')] [2023-12-26 18:28:06,074][105620] Updated weights for policy 1, policy_version 426680 (0.0011) [2023-12-26 18:28:06,080][105692] Updated weights for policy 0, policy_version 426280 (0.0006) [2023-12-26 18:28:06,095][105586] KL-divergence is very high: 100.3763 [2023-12-26 18:28:06,136][105620] Updated weights for policy 1, policy_version 426690 (0.0011) [2023-12-26 18:28:06,729][105692] Updated weights for policy 0, policy_version 426290 (0.0011) [2023-12-26 18:28:06,789][105692] Updated weights for policy 0, policy_version 426300 (0.0011) [2023-12-26 18:28:06,846][105692] Updated weights for policy 0, policy_version 426310 (0.0011) [2023-12-26 18:28:06,875][105620] Updated weights for policy 1, policy_version 426700 (0.0007) [2023-12-26 18:28:06,934][105620] Updated weights for policy 1, policy_version 426710 (0.0010) [2023-12-26 18:28:06,993][105620] Updated weights for policy 1, policy_version 426720 (0.0010) [2023-12-26 18:28:07,615][105692] Updated weights for policy 0, policy_version 426320 (0.0010) [2023-12-26 18:28:07,639][105620] Updated weights for policy 1, policy_version 426730 (0.0009) [2023-12-26 18:28:07,670][105692] Updated weights for policy 0, policy_version 426330 (0.0010) [2023-12-26 18:28:07,701][105620] Updated weights for policy 1, policy_version 426740 (0.0005) [2023-12-26 18:28:07,721][105692] Updated weights for policy 0, policy_version 426340 (0.0007) [2023-12-26 18:28:07,764][105620] Updated weights for policy 1, policy_version 426750 (0.0010) [2023-12-26 18:28:07,825][105620] Updated weights for policy 1, policy_version 426760 (0.0010) [2023-12-26 18:28:08,267][105692] Updated weights for policy 0, policy_version 426350 (0.0005) [2023-12-26 18:28:08,319][105692] Updated weights for policy 0, policy_version 426360 (0.0007) [2023-12-26 18:28:08,346][105620] Updated weights for policy 1, policy_version 426770 (0.0010) [2023-12-26 18:28:08,382][105692] Updated weights for policy 0, policy_version 426370 (0.0009) [2023-12-26 18:28:08,408][105620] Updated weights for policy 1, policy_version 426780 (0.0008) [2023-12-26 18:28:08,460][105620] Updated weights for policy 1, policy_version 426790 (0.0008) [2023-12-26 18:28:09,081][105692] Updated weights for policy 0, policy_version 426380 (0.0009) [2023-12-26 18:28:09,083][105620] Updated weights for policy 1, policy_version 426800 (0.0010) [2023-12-26 18:28:09,141][105620] Updated weights for policy 1, policy_version 426810 (0.0010) [2023-12-26 18:28:09,141][105692] Updated weights for policy 0, policy_version 426390 (0.0007) [2023-12-26 18:28:09,195][105620] Updated weights for policy 1, policy_version 426820 (0.0010) [2023-12-26 18:28:09,203][105692] Updated weights for policy 0, policy_version 426400 (0.0009) [2023-12-26 18:28:09,847][105620] Updated weights for policy 1, policy_version 426830 (0.0008) [2023-12-26 18:28:09,914][105620] Updated weights for policy 1, policy_version 426840 (0.0009) [2023-12-26 18:28:09,971][105620] Updated weights for policy 1, policy_version 426850 (0.0009) [2023-12-26 18:28:10,037][105692] Updated weights for policy 0, policy_version 426410 (0.0009) [2023-12-26 18:28:10,089][105692] Updated weights for policy 0, policy_version 426420 (0.0008) [2023-12-26 18:28:10,149][105692] Updated weights for policy 0, policy_version 426430 (0.0009) [2023-12-26 18:28:10,206][105692] Updated weights for policy 0, policy_version 426440 (0.0009) [2023-12-26 18:28:10,603][105620] Updated weights for policy 1, policy_version 426860 (0.0008) [2023-12-26 18:28:10,660][105620] Updated weights for policy 1, policy_version 426870 (0.0006) [2023-12-26 18:28:10,732][105620] Updated weights for policy 1, policy_version 426880 (0.0005) [2023-12-26 18:28:10,890][105692] Updated weights for policy 0, policy_version 426450 (0.0008) [2023-12-26 18:28:10,943][105692] Updated weights for policy 0, policy_version 426460 (0.0011) [2023-12-26 18:28:11,006][105692] Updated weights for policy 0, policy_version 426470 (0.0011) [2023-12-26 18:28:11,062][104569] Fps is (10 sec: 21299.3, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 218488832. Throughput: 0: 9940.9, 1: 10008.9. Samples: 218493540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:28:11,063][104569] Avg episode reward: [(0, '9266.099'), (1, '9355.622')] [2023-12-26 18:28:11,297][105620] Updated weights for policy 1, policy_version 426890 (0.0006) [2023-12-26 18:28:11,370][105620] Updated weights for policy 1, policy_version 426900 (0.0013) [2023-12-26 18:28:11,440][105620] Updated weights for policy 1, policy_version 426910 (0.0011) [2023-12-26 18:28:11,504][105620] Updated weights for policy 1, policy_version 426920 (0.0011) [2023-12-26 18:28:11,698][105692] Updated weights for policy 0, policy_version 426480 (0.0010) [2023-12-26 18:28:11,742][105585] KL-divergence is very high: 110.6004 [2023-12-26 18:28:11,765][105692] Updated weights for policy 0, policy_version 426490 (0.0010) [2023-12-26 18:28:11,795][105585] KL-divergence is very high: 166.1469 [2023-12-26 18:28:11,834][105692] Updated weights for policy 0, policy_version 426500 (0.0009) [2023-12-26 18:28:11,847][105585] KL-divergence is very high: 139.4992 [2023-12-26 18:28:12,176][105620] Updated weights for policy 1, policy_version 426930 (0.0010) [2023-12-26 18:28:12,234][105620] Updated weights for policy 1, policy_version 426940 (0.0010) [2023-12-26 18:28:12,304][105620] Updated weights for policy 1, policy_version 426950 (0.0010) [2023-12-26 18:28:12,562][105692] Updated weights for policy 0, policy_version 426510 (0.0011) [2023-12-26 18:28:12,616][105692] Updated weights for policy 0, policy_version 426520 (0.0010) [2023-12-26 18:28:12,682][105692] Updated weights for policy 0, policy_version 426530 (0.0011) [2023-12-26 18:28:13,034][105620] Updated weights for policy 1, policy_version 426960 (0.0010) [2023-12-26 18:28:13,079][105620] Updated weights for policy 1, policy_version 426970 (0.0010) [2023-12-26 18:28:13,126][105620] Updated weights for policy 1, policy_version 426980 (0.0010) [2023-12-26 18:28:13,353][105692] Updated weights for policy 0, policy_version 426540 (0.0008) [2023-12-26 18:28:13,407][105692] Updated weights for policy 0, policy_version 426550 (0.0010) [2023-12-26 18:28:13,452][105692] Updated weights for policy 0, policy_version 426560 (0.0010) [2023-12-26 18:28:13,946][105620] Updated weights for policy 1, policy_version 426990 (0.0010) [2023-12-26 18:28:13,994][105620] Updated weights for policy 1, policy_version 427000 (0.0009) [2023-12-26 18:28:14,056][105620] Updated weights for policy 1, policy_version 427010 (0.0009) [2023-12-26 18:28:14,127][105692] Updated weights for policy 0, policy_version 426570 (0.0009) [2023-12-26 18:28:14,178][105692] Updated weights for policy 0, policy_version 426580 (0.0006) [2023-12-26 18:28:14,232][105692] Updated weights for policy 0, policy_version 426590 (0.0006) [2023-12-26 18:28:14,286][105692] Updated weights for policy 0, policy_version 426600 (0.0009) [2023-12-26 18:28:14,806][105620] Updated weights for policy 1, policy_version 427020 (0.0009) [2023-12-26 18:28:14,858][105620] Updated weights for policy 1, policy_version 427030 (0.0008) [2023-12-26 18:28:14,914][105620] Updated weights for policy 1, policy_version 427040 (0.0005) [2023-12-26 18:28:15,027][105692] Updated weights for policy 0, policy_version 426610 (0.0007) [2023-12-26 18:28:15,089][105692] Updated weights for policy 0, policy_version 426620 (0.0009) [2023-12-26 18:28:15,152][105692] Updated weights for policy 0, policy_version 426630 (0.0008) [2023-12-26 18:28:15,680][105620] Updated weights for policy 1, policy_version 427050 (0.0010) [2023-12-26 18:28:15,749][105620] Updated weights for policy 1, policy_version 427060 (0.0009) [2023-12-26 18:28:15,809][105620] Updated weights for policy 1, policy_version 427070 (0.0010) [2023-12-26 18:28:15,873][105620] Updated weights for policy 1, policy_version 427080 (0.0009) [2023-12-26 18:28:15,910][105692] Updated weights for policy 0, policy_version 426640 (0.0007) [2023-12-26 18:28:15,981][105692] Updated weights for policy 0, policy_version 426650 (0.0006) [2023-12-26 18:28:16,048][105692] Updated weights for policy 0, policy_version 426660 (0.0006) [2023-12-26 18:28:16,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 218578944. Throughput: 0: 9934.7, 1: 9934.2. Samples: 218552380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:28:16,062][104569] Avg episode reward: [(0, '9172.778'), (1, '9355.582')] [2023-12-26 18:28:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000427080_109346816.pth... [2023-12-26 18:28:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000425896_109043712.pth [2023-12-26 18:28:16,076][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000426664_109240320.pth... [2023-12-26 18:28:16,080][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000425512_108945408.pth [2023-12-26 18:28:16,560][105620] Updated weights for policy 1, policy_version 427090 (0.0009) [2023-12-26 18:28:16,621][105620] Updated weights for policy 1, policy_version 427100 (0.0009) [2023-12-26 18:28:16,676][105620] Updated weights for policy 1, policy_version 427110 (0.0009) [2023-12-26 18:28:16,746][105692] Updated weights for policy 0, policy_version 426670 (0.0007) [2023-12-26 18:28:16,816][105692] Updated weights for policy 0, policy_version 426680 (0.0005) [2023-12-26 18:28:16,884][105692] Updated weights for policy 0, policy_version 426690 (0.0006) [2023-12-26 18:28:17,408][105620] Updated weights for policy 1, policy_version 427120 (0.0009) [2023-12-26 18:28:17,469][105620] Updated weights for policy 1, policy_version 427130 (0.0009) [2023-12-26 18:28:17,531][105620] Updated weights for policy 1, policy_version 427140 (0.0007) [2023-12-26 18:28:17,582][105692] Updated weights for policy 0, policy_version 426700 (0.0008) [2023-12-26 18:28:17,642][105692] Updated weights for policy 0, policy_version 426710 (0.0009) [2023-12-26 18:28:17,695][105692] Updated weights for policy 0, policy_version 426720 (0.0009) [2023-12-26 18:28:18,171][105620] Updated weights for policy 1, policy_version 427150 (0.0007) [2023-12-26 18:28:18,218][105620] Updated weights for policy 1, policy_version 427160 (0.0008) [2023-12-26 18:28:18,276][105620] Updated weights for policy 1, policy_version 427170 (0.0009) [2023-12-26 18:28:18,435][105692] Updated weights for policy 0, policy_version 426730 (0.0006) [2023-12-26 18:28:18,492][105692] Updated weights for policy 0, policy_version 426740 (0.0010) [2023-12-26 18:28:18,549][105692] Updated weights for policy 0, policy_version 426750 (0.0009) [2023-12-26 18:28:18,612][105692] Updated weights for policy 0, policy_version 426760 (0.0008) [2023-12-26 18:28:18,988][105620] Updated weights for policy 1, policy_version 427180 (0.0009) [2023-12-26 18:28:19,036][105620] Updated weights for policy 1, policy_version 427190 (0.0009) [2023-12-26 18:28:19,085][105620] Updated weights for policy 1, policy_version 427201 (0.0009) [2023-12-26 18:28:19,387][105692] Updated weights for policy 0, policy_version 426770 (0.0009) [2023-12-26 18:28:19,449][105692] Updated weights for policy 0, policy_version 426780 (0.0010) [2023-12-26 18:28:19,515][105692] Updated weights for policy 0, policy_version 426790 (0.0009) [2023-12-26 18:28:19,885][105620] Updated weights for policy 1, policy_version 427211 (0.0008) [2023-12-26 18:28:19,940][105620] Updated weights for policy 1, policy_version 427221 (0.0009) [2023-12-26 18:28:19,996][105620] Updated weights for policy 1, policy_version 427231 (0.0009) [2023-12-26 18:28:20,294][105692] Updated weights for policy 0, policy_version 426800 (0.0009) [2023-12-26 18:28:20,356][105692] Updated weights for policy 0, policy_version 426810 (0.0009) [2023-12-26 18:28:20,412][105692] Updated weights for policy 0, policy_version 426820 (0.0009) [2023-12-26 18:28:20,761][105620] Updated weights for policy 1, policy_version 427241 (0.0008) [2023-12-26 18:28:20,826][105620] Updated weights for policy 1, policy_version 427251 (0.0008) [2023-12-26 18:28:20,884][105620] Updated weights for policy 1, policy_version 427261 (0.0005) [2023-12-26 18:28:20,944][105620] Updated weights for policy 1, policy_version 427271 (0.0008) [2023-12-26 18:28:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19605.2). Total num frames: 218677248. Throughput: 0: 9761.5, 1: 9903.3. Samples: 218666804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:28:21,063][104569] Avg episode reward: [(0, '9172.492'), (1, '9355.709')] [2023-12-26 18:28:21,217][105692] Updated weights for policy 0, policy_version 426830 (0.0009) [2023-12-26 18:28:21,274][105692] Updated weights for policy 0, policy_version 426840 (0.0009) [2023-12-26 18:28:21,328][105692] Updated weights for policy 0, policy_version 426850 (0.0009) [2023-12-26 18:28:21,636][105620] Updated weights for policy 1, policy_version 427281 (0.0009) [2023-12-26 18:28:21,689][105620] Updated weights for policy 1, policy_version 427291 (0.0009) [2023-12-26 18:28:21,755][105620] Updated weights for policy 1, policy_version 427301 (0.0009) [2023-12-26 18:28:22,162][105692] Updated weights for policy 0, policy_version 426860 (0.0009) [2023-12-26 18:28:22,221][105692] Updated weights for policy 0, policy_version 426870 (0.0009) [2023-12-26 18:28:22,285][105692] Updated weights for policy 0, policy_version 426880 (0.0010) [2023-12-26 18:28:22,541][105620] Updated weights for policy 1, policy_version 427311 (0.0008) [2023-12-26 18:28:22,604][105620] Updated weights for policy 1, policy_version 427321 (0.0006) [2023-12-26 18:28:22,668][105620] Updated weights for policy 1, policy_version 427331 (0.0008) [2023-12-26 18:28:23,087][105692] Updated weights for policy 0, policy_version 426890 (0.0009) [2023-12-26 18:28:23,146][105692] Updated weights for policy 0, policy_version 426900 (0.0009) [2023-12-26 18:28:23,203][105692] Updated weights for policy 0, policy_version 426910 (0.0009) [2023-12-26 18:28:23,261][105692] Updated weights for policy 0, policy_version 426920 (0.0009) [2023-12-26 18:28:23,395][105620] Updated weights for policy 1, policy_version 427341 (0.0009) [2023-12-26 18:28:23,446][105620] Updated weights for policy 1, policy_version 427351 (0.0009) [2023-12-26 18:28:23,501][105620] Updated weights for policy 1, policy_version 427361 (0.0009) [2023-12-26 18:28:23,949][105692] Updated weights for policy 0, policy_version 426930 (0.0007) [2023-12-26 18:28:24,003][105692] Updated weights for policy 0, policy_version 426940 (0.0008) [2023-12-26 18:28:24,057][105692] Updated weights for policy 0, policy_version 426950 (0.0008) [2023-12-26 18:28:24,256][105620] Updated weights for policy 1, policy_version 427371 (0.0008) [2023-12-26 18:28:24,303][105620] Updated weights for policy 1, policy_version 427381 (0.0008) [2023-12-26 18:28:24,358][105620] Updated weights for policy 1, policy_version 427391 (0.0006) [2023-12-26 18:28:24,773][105692] Updated weights for policy 0, policy_version 426960 (0.0008) [2023-12-26 18:28:24,834][105692] Updated weights for policy 0, policy_version 426970 (0.0010) [2023-12-26 18:28:24,893][105692] Updated weights for policy 0, policy_version 426980 (0.0005) [2023-12-26 18:28:24,986][105620] Updated weights for policy 1, policy_version 427401 (0.0006) [2023-12-26 18:28:25,045][105620] Updated weights for policy 1, policy_version 427411 (0.0008) [2023-12-26 18:28:25,108][105620] Updated weights for policy 1, policy_version 427421 (0.0008) [2023-12-26 18:28:25,167][105620] Updated weights for policy 1, policy_version 427431 (0.0008) [2023-12-26 18:28:25,557][105692] Updated weights for policy 0, policy_version 426990 (0.0007) [2023-12-26 18:28:25,624][105692] Updated weights for policy 0, policy_version 427000 (0.0010) [2023-12-26 18:28:25,685][105692] Updated weights for policy 0, policy_version 427010 (0.0010) [2023-12-26 18:28:25,847][105620] Updated weights for policy 1, policy_version 427441 (0.0010) [2023-12-26 18:28:25,908][105620] Updated weights for policy 1, policy_version 427451 (0.0010) [2023-12-26 18:28:25,959][105620] Updated weights for policy 1, policy_version 427461 (0.0006) [2023-12-26 18:28:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 218775552. Throughput: 0: 9610.9, 1: 9956.9. Samples: 218781248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:28:26,063][104569] Avg episode reward: [(0, '9357.774'), (1, '9263.230')] [2023-12-26 18:28:26,405][105692] Updated weights for policy 0, policy_version 427020 (0.0010) [2023-12-26 18:28:26,452][105692] Updated weights for policy 0, policy_version 427030 (0.0010) [2023-12-26 18:28:26,504][105692] Updated weights for policy 0, policy_version 427040 (0.0010) [2023-12-26 18:28:26,509][105620] Updated weights for policy 1, policy_version 427471 (0.0008) [2023-12-26 18:28:26,565][105620] Updated weights for policy 1, policy_version 427481 (0.0005) [2023-12-26 18:28:26,629][105620] Updated weights for policy 1, policy_version 427491 (0.0008) [2023-12-26 18:28:27,176][105692] Updated weights for policy 0, policy_version 427050 (0.0010) [2023-12-26 18:28:27,225][105692] Updated weights for policy 0, policy_version 427060 (0.0010) [2023-12-26 18:28:27,247][105620] Updated weights for policy 1, policy_version 427501 (0.0008) [2023-12-26 18:28:27,276][105692] Updated weights for policy 0, policy_version 427070 (0.0010) [2023-12-26 18:28:27,306][105620] Updated weights for policy 1, policy_version 427511 (0.0006) [2023-12-26 18:28:27,328][105692] Updated weights for policy 0, policy_version 427080 (0.0010) [2023-12-26 18:28:27,357][105620] Updated weights for policy 1, policy_version 427521 (0.0007) [2023-12-26 18:28:27,939][105620] Updated weights for policy 1, policy_version 427531 (0.0007) [2023-12-26 18:28:27,987][105620] Updated weights for policy 1, policy_version 427541 (0.0005) [2023-12-26 18:28:28,035][105620] Updated weights for policy 1, policy_version 427551 (0.0005) [2023-12-26 18:28:28,077][105692] Updated weights for policy 0, policy_version 427090 (0.0005) [2023-12-26 18:28:28,134][105692] Updated weights for policy 0, policy_version 427100 (0.0005) [2023-12-26 18:28:28,182][105692] Updated weights for policy 0, policy_version 427110 (0.0008) [2023-12-26 18:28:28,694][105620] Updated weights for policy 1, policy_version 427561 (0.0007) [2023-12-26 18:28:28,748][105620] Updated weights for policy 1, policy_version 427571 (0.0009) [2023-12-26 18:28:28,805][105620] Updated weights for policy 1, policy_version 427581 (0.0009) [2023-12-26 18:28:28,834][105692] Updated weights for policy 0, policy_version 427120 (0.0007) [2023-12-26 18:28:28,868][105620] Updated weights for policy 1, policy_version 427591 (0.0009) [2023-12-26 18:28:28,896][105692] Updated weights for policy 0, policy_version 427130 (0.0006) [2023-12-26 18:28:28,949][105692] Updated weights for policy 0, policy_version 427140 (0.0007) [2023-12-26 18:28:29,652][105620] Updated weights for policy 1, policy_version 427601 (0.0005) [2023-12-26 18:28:29,671][105692] Updated weights for policy 0, policy_version 427150 (0.0006) [2023-12-26 18:28:29,712][105620] Updated weights for policy 1, policy_version 427611 (0.0006) [2023-12-26 18:28:29,725][105692] Updated weights for policy 0, policy_version 427160 (0.0010) [2023-12-26 18:28:29,767][105620] Updated weights for policy 1, policy_version 427621 (0.0006) [2023-12-26 18:28:29,785][105692] Updated weights for policy 0, policy_version 427170 (0.0009) [2023-12-26 18:28:30,508][105620] Updated weights for policy 1, policy_version 427631 (0.0009) [2023-12-26 18:28:30,516][105692] Updated weights for policy 0, policy_version 427180 (0.0007) [2023-12-26 18:28:30,558][105620] Updated weights for policy 1, policy_version 427641 (0.0008) [2023-12-26 18:28:30,564][105692] Updated weights for policy 0, policy_version 427190 (0.0006) [2023-12-26 18:28:30,620][105692] Updated weights for policy 0, policy_version 427200 (0.0005) [2023-12-26 18:28:30,622][105620] Updated weights for policy 1, policy_version 427651 (0.0010) [2023-12-26 18:28:31,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 218873856. Throughput: 0: 9657.9, 1: 10055.0. Samples: 218844568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:28:31,062][104569] Avg episode reward: [(0, '9267.352'), (1, '9171.222')] [2023-12-26 18:28:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000427208_109379584.pth... [2023-12-26 18:28:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000427656_109494272.pth... [2023-12-26 18:28:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000426472_109191168.pth [2023-12-26 18:28:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000426056_109084672.pth [2023-12-26 18:28:31,392][105620] Updated weights for policy 1, policy_version 427661 (0.0009) [2023-12-26 18:28:31,419][105692] Updated weights for policy 0, policy_version 427210 (0.0006) [2023-12-26 18:28:31,452][105620] Updated weights for policy 1, policy_version 427671 (0.0010) [2023-12-26 18:28:31,480][105692] Updated weights for policy 0, policy_version 427220 (0.0006) [2023-12-26 18:28:31,507][105620] Updated weights for policy 1, policy_version 427681 (0.0010) [2023-12-26 18:28:31,538][105692] Updated weights for policy 0, policy_version 427230 (0.0008) [2023-12-26 18:28:31,597][105692] Updated weights for policy 0, policy_version 427240 (0.0008) [2023-12-26 18:28:32,270][105620] Updated weights for policy 1, policy_version 427691 (0.0011) [2023-12-26 18:28:32,278][105692] Updated weights for policy 0, policy_version 427250 (0.0006) [2023-12-26 18:28:32,326][105620] Updated weights for policy 1, policy_version 427701 (0.0011) [2023-12-26 18:28:32,338][105692] Updated weights for policy 0, policy_version 427260 (0.0010) [2023-12-26 18:28:32,390][105620] Updated weights for policy 1, policy_version 427711 (0.0008) [2023-12-26 18:28:32,409][105692] Updated weights for policy 0, policy_version 427270 (0.0007) [2023-12-26 18:28:32,940][105620] Updated weights for policy 1, policy_version 427721 (0.0007) [2023-12-26 18:28:32,989][105620] Updated weights for policy 1, policy_version 427731 (0.0010) [2023-12-26 18:28:33,033][105620] Updated weights for policy 1, policy_version 427741 (0.0010) [2023-12-26 18:28:33,092][105620] Updated weights for policy 1, policy_version 427751 (0.0010) [2023-12-26 18:28:33,153][105692] Updated weights for policy 0, policy_version 427280 (0.0008) [2023-12-26 18:28:33,204][105692] Updated weights for policy 0, policy_version 427290 (0.0008) [2023-12-26 18:28:33,251][105692] Updated weights for policy 0, policy_version 427300 (0.0008) [2023-12-26 18:28:33,844][105620] Updated weights for policy 1, policy_version 427761 (0.0010) [2023-12-26 18:28:33,905][105620] Updated weights for policy 1, policy_version 427771 (0.0010) [2023-12-26 18:28:33,962][105620] Updated weights for policy 1, policy_version 427781 (0.0010) [2023-12-26 18:28:34,012][105692] Updated weights for policy 0, policy_version 427310 (0.0008) [2023-12-26 18:28:34,059][105692] Updated weights for policy 0, policy_version 427320 (0.0008) [2023-12-26 18:28:34,113][105692] Updated weights for policy 0, policy_version 427330 (0.0008) [2023-12-26 18:28:34,724][105620] Updated weights for policy 1, policy_version 427791 (0.0010) [2023-12-26 18:28:34,782][105620] Updated weights for policy 1, policy_version 427801 (0.0008) [2023-12-26 18:28:34,803][105692] Updated weights for policy 0, policy_version 427340 (0.0008) [2023-12-26 18:28:34,841][105620] Updated weights for policy 1, policy_version 427811 (0.0006) [2023-12-26 18:28:34,860][105692] Updated weights for policy 0, policy_version 427350 (0.0008) [2023-12-26 18:28:34,908][105692] Updated weights for policy 0, policy_version 427360 (0.0008) [2023-12-26 18:28:35,490][105620] Updated weights for policy 1, policy_version 427821 (0.0006) [2023-12-26 18:28:35,564][105620] Updated weights for policy 1, policy_version 427831 (0.0008) [2023-12-26 18:28:35,622][105620] Updated weights for policy 1, policy_version 427841 (0.0009) [2023-12-26 18:28:35,701][105692] Updated weights for policy 0, policy_version 427370 (0.0009) [2023-12-26 18:28:35,746][105692] Updated weights for policy 0, policy_version 427380 (0.0007) [2023-12-26 18:28:35,800][105692] Updated weights for policy 0, policy_version 427390 (0.0005) [2023-12-26 18:28:35,856][105692] Updated weights for policy 0, policy_version 427400 (0.0005) [2023-12-26 18:28:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 218972160. Throughput: 0: 9721.0, 1: 9936.4. Samples: 218959704. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-26 18:28:36,062][104569] Avg episode reward: [(0, '9267.217'), (1, '9171.205')] [2023-12-26 18:28:36,428][105620] Updated weights for policy 1, policy_version 427851 (0.0008) [2023-12-26 18:28:36,440][105692] Updated weights for policy 0, policy_version 427410 (0.0009) [2023-12-26 18:28:36,480][105620] Updated weights for policy 1, policy_version 427861 (0.0007) [2023-12-26 18:28:36,508][105692] Updated weights for policy 0, policy_version 427420 (0.0009) [2023-12-26 18:28:36,537][105620] Updated weights for policy 1, policy_version 427871 (0.0006) [2023-12-26 18:28:36,566][105692] Updated weights for policy 0, policy_version 427430 (0.0009) [2023-12-26 18:28:37,162][105620] Updated weights for policy 1, policy_version 427881 (0.0006) [2023-12-26 18:28:37,224][105620] Updated weights for policy 1, policy_version 427891 (0.0011) [2023-12-26 18:28:37,287][105620] Updated weights for policy 1, policy_version 427901 (0.0011) [2023-12-26 18:28:37,344][105620] Updated weights for policy 1, policy_version 427911 (0.0009) [2023-12-26 18:28:37,372][105692] Updated weights for policy 0, policy_version 427440 (0.0009) [2023-12-26 18:28:37,447][105692] Updated weights for policy 0, policy_version 427450 (0.0010) [2023-12-26 18:28:37,512][105692] Updated weights for policy 0, policy_version 427460 (0.0009) [2023-12-26 18:28:38,079][105620] Updated weights for policy 1, policy_version 427921 (0.0010) [2023-12-26 18:28:38,143][105620] Updated weights for policy 1, policy_version 427931 (0.0011) [2023-12-26 18:28:38,200][105620] Updated weights for policy 1, policy_version 427941 (0.0011) [2023-12-26 18:28:38,272][105692] Updated weights for policy 0, policy_version 427470 (0.0009) [2023-12-26 18:28:38,325][105692] Updated weights for policy 0, policy_version 427480 (0.0011) [2023-12-26 18:28:38,391][105692] Updated weights for policy 0, policy_version 427490 (0.0011) [2023-12-26 18:28:38,966][105620] Updated weights for policy 1, policy_version 427951 (0.0011) [2023-12-26 18:28:39,021][105620] Updated weights for policy 1, policy_version 427961 (0.0011) [2023-12-26 18:28:39,076][105620] Updated weights for policy 1, policy_version 427971 (0.0010) [2023-12-26 18:28:39,148][105692] Updated weights for policy 0, policy_version 427500 (0.0011) [2023-12-26 18:28:39,193][105692] Updated weights for policy 0, policy_version 427510 (0.0010) [2023-12-26 18:28:39,253][105692] Updated weights for policy 0, policy_version 427520 (0.0011) [2023-12-26 18:28:39,863][105620] Updated weights for policy 1, policy_version 427981 (0.0011) [2023-12-26 18:28:39,923][105620] Updated weights for policy 1, policy_version 427991 (0.0010) [2023-12-26 18:28:39,965][105692] Updated weights for policy 0, policy_version 427530 (0.0010) [2023-12-26 18:28:39,972][105620] Updated weights for policy 1, policy_version 428001 (0.0008) [2023-12-26 18:28:40,028][105692] Updated weights for policy 0, policy_version 427540 (0.0008) [2023-12-26 18:28:40,093][105692] Updated weights for policy 0, policy_version 427550 (0.0008) [2023-12-26 18:28:40,156][105692] Updated weights for policy 0, policy_version 427560 (0.0008) [2023-12-26 18:28:40,709][105620] Updated weights for policy 1, policy_version 428011 (0.0008) [2023-12-26 18:28:40,778][105620] Updated weights for policy 1, policy_version 428021 (0.0009) [2023-12-26 18:28:40,829][105620] Updated weights for policy 1, policy_version 428031 (0.0006) [2023-12-26 18:28:40,878][105692] Updated weights for policy 0, policy_version 427570 (0.0010) [2023-12-26 18:28:40,936][105692] Updated weights for policy 0, policy_version 427580 (0.0010) [2023-12-26 18:28:40,995][105692] Updated weights for policy 0, policy_version 427590 (0.0010) [2023-12-26 18:28:41,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 219070464. Throughput: 0: 9664.0, 1: 9907.3. Samples: 219074392. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-26 18:28:41,063][104569] Avg episode reward: [(0, '9264.565'), (1, '9079.190')] [2023-12-26 18:28:41,550][105620] Updated weights for policy 1, policy_version 428041 (0.0007) [2023-12-26 18:28:41,626][105620] Updated weights for policy 1, policy_version 428051 (0.0011) [2023-12-26 18:28:41,685][105620] Updated weights for policy 1, policy_version 428061 (0.0009) [2023-12-26 18:28:41,739][105692] Updated weights for policy 0, policy_version 427600 (0.0011) [2023-12-26 18:28:41,752][105620] Updated weights for policy 1, policy_version 428071 (0.0008) [2023-12-26 18:28:41,796][105692] Updated weights for policy 0, policy_version 427610 (0.0007) [2023-12-26 18:28:41,858][105692] Updated weights for policy 0, policy_version 427620 (0.0006) [2023-12-26 18:28:42,495][105692] Updated weights for policy 0, policy_version 427630 (0.0009) [2023-12-26 18:28:42,557][105692] Updated weights for policy 0, policy_version 427640 (0.0011) [2023-12-26 18:28:42,585][105620] Updated weights for policy 1, policy_version 428081 (0.0010) [2023-12-26 18:28:42,621][105692] Updated weights for policy 0, policy_version 427650 (0.0009) [2023-12-26 18:28:42,643][105620] Updated weights for policy 1, policy_version 428091 (0.0006) [2023-12-26 18:28:42,698][105620] Updated weights for policy 1, policy_version 428101 (0.0008) [2023-12-26 18:28:43,278][105692] Updated weights for policy 0, policy_version 427660 (0.0011) [2023-12-26 18:28:43,340][105692] Updated weights for policy 0, policy_version 427670 (0.0010) [2023-12-26 18:28:43,385][105692] Updated weights for policy 0, policy_version 427680 (0.0010) [2023-12-26 18:28:43,437][105620] Updated weights for policy 1, policy_version 428111 (0.0006) [2023-12-26 18:28:43,485][105620] Updated weights for policy 1, policy_version 428121 (0.0005) [2023-12-26 18:28:43,532][105620] Updated weights for policy 1, policy_version 428131 (0.0005) [2023-12-26 18:28:44,027][105692] Updated weights for policy 0, policy_version 427690 (0.0010) [2023-12-26 18:28:44,086][105692] Updated weights for policy 0, policy_version 427700 (0.0010) [2023-12-26 18:28:44,100][105620] Updated weights for policy 1, policy_version 428141 (0.0005) [2023-12-26 18:28:44,141][105692] Updated weights for policy 0, policy_version 427710 (0.0010) [2023-12-26 18:28:44,160][105620] Updated weights for policy 1, policy_version 428151 (0.0006) [2023-12-26 18:28:44,205][105692] Updated weights for policy 0, policy_version 427720 (0.0011) [2023-12-26 18:28:44,226][105620] Updated weights for policy 1, policy_version 428161 (0.0006) [2023-12-26 18:28:44,769][105620] Updated weights for policy 1, policy_version 428171 (0.0006) [2023-12-26 18:28:44,836][105620] Updated weights for policy 1, policy_version 428181 (0.0009) [2023-12-26 18:28:44,893][105620] Updated weights for policy 1, policy_version 428191 (0.0009) [2023-12-26 18:28:44,976][105692] Updated weights for policy 0, policy_version 427730 (0.0006) [2023-12-26 18:28:45,045][105692] Updated weights for policy 0, policy_version 427740 (0.0007) [2023-12-26 18:28:45,112][105692] Updated weights for policy 0, policy_version 427750 (0.0006) [2023-12-26 18:28:45,515][105620] Updated weights for policy 1, policy_version 428201 (0.0007) [2023-12-26 18:28:45,581][105620] Updated weights for policy 1, policy_version 428211 (0.0006) [2023-12-26 18:28:45,641][105620] Updated weights for policy 1, policy_version 428221 (0.0005) [2023-12-26 18:28:45,700][105620] Updated weights for policy 1, policy_version 428231 (0.0006) [2023-12-26 18:28:45,758][105692] Updated weights for policy 0, policy_version 427760 (0.0008) [2023-12-26 18:28:45,815][105692] Updated weights for policy 0, policy_version 427770 (0.0010) [2023-12-26 18:28:45,869][105692] Updated weights for policy 0, policy_version 427780 (0.0010) [2023-12-26 18:28:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19605.2). Total num frames: 219168768. Throughput: 0: 9714.7, 1: 9924.0. Samples: 219133840. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-26 18:28:46,063][104569] Avg episode reward: [(0, '9264.570'), (1, '9080.565')] [2023-12-26 18:28:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000427784_109527040.pth... [2023-12-26 18:28:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000428232_109641728.pth... [2023-12-26 18:28:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000426664_109240320.pth [2023-12-26 18:28:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000427080_109346816.pth [2023-12-26 18:28:46,283][105620] Updated weights for policy 1, policy_version 428241 (0.0006) [2023-12-26 18:28:46,342][105620] Updated weights for policy 1, policy_version 428251 (0.0009) [2023-12-26 18:28:46,391][105620] Updated weights for policy 1, policy_version 428261 (0.0009) [2023-12-26 18:28:46,672][105692] Updated weights for policy 0, policy_version 427790 (0.0009) [2023-12-26 18:28:46,729][105692] Updated weights for policy 0, policy_version 427800 (0.0009) [2023-12-26 18:28:46,791][105692] Updated weights for policy 0, policy_version 427810 (0.0009) [2023-12-26 18:28:47,076][105620] Updated weights for policy 1, policy_version 428271 (0.0008) [2023-12-26 18:28:47,134][105620] Updated weights for policy 1, policy_version 428281 (0.0005) [2023-12-26 18:28:47,199][105620] Updated weights for policy 1, policy_version 428291 (0.0010) [2023-12-26 18:28:47,573][105692] Updated weights for policy 0, policy_version 427820 (0.0008) [2023-12-26 18:28:47,629][105692] Updated weights for policy 0, policy_version 427830 (0.0008) [2023-12-26 18:28:47,687][105692] Updated weights for policy 0, policy_version 427840 (0.0008) [2023-12-26 18:28:47,833][105620] Updated weights for policy 1, policy_version 428301 (0.0008) [2023-12-26 18:28:47,892][105620] Updated weights for policy 1, policy_version 428311 (0.0005) [2023-12-26 18:28:47,938][105620] Updated weights for policy 1, policy_version 428321 (0.0005) [2023-12-26 18:28:48,479][105692] Updated weights for policy 0, policy_version 427850 (0.0009) [2023-12-26 18:28:48,546][105692] Updated weights for policy 0, policy_version 427860 (0.0008) [2023-12-26 18:28:48,569][105620] Updated weights for policy 1, policy_version 428331 (0.0007) [2023-12-26 18:28:48,600][105692] Updated weights for policy 0, policy_version 427870 (0.0006) [2023-12-26 18:28:48,632][105620] Updated weights for policy 1, policy_version 428341 (0.0009) [2023-12-26 18:28:48,650][105692] Updated weights for policy 0, policy_version 427880 (0.0009) [2023-12-26 18:28:48,679][105620] Updated weights for policy 1, policy_version 428351 (0.0008) [2023-12-26 18:28:49,439][105692] Updated weights for policy 0, policy_version 427890 (0.0011) [2023-12-26 18:28:49,458][105620] Updated weights for policy 1, policy_version 428361 (0.0009) [2023-12-26 18:28:49,501][105692] Updated weights for policy 0, policy_version 427900 (0.0011) [2023-12-26 18:28:49,503][105620] Updated weights for policy 1, policy_version 428371 (0.0008) [2023-12-26 18:28:49,559][105692] Updated weights for policy 0, policy_version 427910 (0.0009) [2023-12-26 18:28:49,562][105620] Updated weights for policy 1, policy_version 428381 (0.0008) [2023-12-26 18:28:49,614][105620] Updated weights for policy 1, policy_version 428391 (0.0008) [2023-12-26 18:28:50,323][105620] Updated weights for policy 1, policy_version 428401 (0.0006) [2023-12-26 18:28:50,351][105692] Updated weights for policy 0, policy_version 427920 (0.0011) [2023-12-26 18:28:50,380][105620] Updated weights for policy 1, policy_version 428411 (0.0005) [2023-12-26 18:28:50,406][105692] Updated weights for policy 0, policy_version 427930 (0.0010) [2023-12-26 18:28:50,427][105620] Updated weights for policy 1, policy_version 428421 (0.0005) [2023-12-26 18:28:50,454][105692] Updated weights for policy 0, policy_version 427940 (0.0010) [2023-12-26 18:28:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 219258880. Throughput: 0: 9673.0, 1: 9995.9. Samples: 219252964. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-26 18:28:51,062][104569] Avg episode reward: [(0, '9357.593'), (1, '9083.681')] [2023-12-26 18:28:51,150][105692] Updated weights for policy 0, policy_version 427950 (0.0009) [2023-12-26 18:28:51,162][105620] Updated weights for policy 1, policy_version 428431 (0.0008) [2023-12-26 18:28:51,209][105692] Updated weights for policy 0, policy_version 427960 (0.0011) [2023-12-26 18:28:51,223][105620] Updated weights for policy 1, policy_version 428441 (0.0007) [2023-12-26 18:28:51,265][105692] Updated weights for policy 0, policy_version 427970 (0.0010) [2023-12-26 18:28:51,283][105620] Updated weights for policy 1, policy_version 428451 (0.0006) [2023-12-26 18:28:52,036][105692] Updated weights for policy 0, policy_version 427980 (0.0009) [2023-12-26 18:28:52,038][105620] Updated weights for policy 1, policy_version 428461 (0.0007) [2023-12-26 18:28:52,098][105620] Updated weights for policy 1, policy_version 428471 (0.0006) [2023-12-26 18:28:52,098][105692] Updated weights for policy 0, policy_version 427990 (0.0006) [2023-12-26 18:28:52,159][105620] Updated weights for policy 1, policy_version 428481 (0.0005) [2023-12-26 18:28:52,165][105692] Updated weights for policy 0, policy_version 428000 (0.0006) [2023-12-26 18:28:52,832][105620] Updated weights for policy 1, policy_version 428491 (0.0006) [2023-12-26 18:28:52,845][105692] Updated weights for policy 0, policy_version 428010 (0.0007) [2023-12-26 18:28:52,899][105620] Updated weights for policy 1, policy_version 428501 (0.0007) [2023-12-26 18:28:52,906][105692] Updated weights for policy 0, policy_version 428020 (0.0008) [2023-12-26 18:28:52,959][105620] Updated weights for policy 1, policy_version 428511 (0.0007) [2023-12-26 18:28:52,961][105692] Updated weights for policy 0, policy_version 428030 (0.0007) [2023-12-26 18:28:53,021][105692] Updated weights for policy 0, policy_version 428040 (0.0006) [2023-12-26 18:28:53,646][105620] Updated weights for policy 1, policy_version 428521 (0.0006) [2023-12-26 18:28:53,712][105620] Updated weights for policy 1, policy_version 428531 (0.0007) [2023-12-26 18:28:53,768][105620] Updated weights for policy 1, policy_version 428541 (0.0006) [2023-12-26 18:28:53,769][105692] Updated weights for policy 0, policy_version 428050 (0.0009) [2023-12-26 18:28:53,818][105692] Updated weights for policy 0, policy_version 428060 (0.0009) [2023-12-26 18:28:53,827][105620] Updated weights for policy 1, policy_version 428551 (0.0005) [2023-12-26 18:28:53,866][105692] Updated weights for policy 0, policy_version 428070 (0.0009) [2023-12-26 18:28:54,483][105620] Updated weights for policy 1, policy_version 428561 (0.0008) [2023-12-26 18:28:54,544][105620] Updated weights for policy 1, policy_version 428571 (0.0009) [2023-12-26 18:28:54,609][105620] Updated weights for policy 1, policy_version 428581 (0.0009) [2023-12-26 18:28:54,655][105692] Updated weights for policy 0, policy_version 428080 (0.0008) [2023-12-26 18:28:54,708][105692] Updated weights for policy 0, policy_version 428090 (0.0008) [2023-12-26 18:28:54,758][105692] Updated weights for policy 0, policy_version 428100 (0.0009) [2023-12-26 18:28:55,276][105620] Updated weights for policy 1, policy_version 428591 (0.0010) [2023-12-26 18:28:55,339][105620] Updated weights for policy 1, policy_version 428601 (0.0010) [2023-12-26 18:28:55,393][105620] Updated weights for policy 1, policy_version 428611 (0.0010) [2023-12-26 18:28:55,460][105692] Updated weights for policy 0, policy_version 428110 (0.0007) [2023-12-26 18:28:55,515][105692] Updated weights for policy 0, policy_version 428120 (0.0005) [2023-12-26 18:28:55,574][105692] Updated weights for policy 0, policy_version 428130 (0.0005) [2023-12-26 18:28:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 219357184. Throughput: 0: 9595.6, 1: 9872.2. Samples: 219369596. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-26 18:28:56,063][104569] Avg episode reward: [(0, '9357.267'), (1, '9174.629')] [2023-12-26 18:28:56,158][105620] Updated weights for policy 1, policy_version 428621 (0.0009) [2023-12-26 18:28:56,203][105692] Updated weights for policy 0, policy_version 428140 (0.0008) [2023-12-26 18:28:56,211][105620] Updated weights for policy 1, policy_version 428631 (0.0008) [2023-12-26 18:28:56,251][105692] Updated weights for policy 0, policy_version 428150 (0.0006) [2023-12-26 18:28:56,270][105620] Updated weights for policy 1, policy_version 428641 (0.0007) [2023-12-26 18:28:56,293][105692] Updated weights for policy 0, policy_version 428160 (0.0007) [2023-12-26 18:28:56,963][105692] Updated weights for policy 0, policy_version 428170 (0.0007) [2023-12-26 18:28:57,008][105692] Updated weights for policy 0, policy_version 428180 (0.0008) [2023-12-26 18:28:57,055][105620] Updated weights for policy 1, policy_version 428651 (0.0007) [2023-12-26 18:28:57,057][105692] Updated weights for policy 0, policy_version 428190 (0.0008) [2023-12-26 18:28:57,110][105692] Updated weights for policy 0, policy_version 428200 (0.0006) [2023-12-26 18:28:57,112][105620] Updated weights for policy 1, policy_version 428661 (0.0007) [2023-12-26 18:28:57,159][105620] Updated weights for policy 1, policy_version 428671 (0.0006) [2023-12-26 18:28:57,717][105620] Updated weights for policy 1, policy_version 428681 (0.0005) [2023-12-26 18:28:57,775][105620] Updated weights for policy 1, policy_version 428691 (0.0007) [2023-12-26 18:28:57,819][105620] Updated weights for policy 1, policy_version 428701 (0.0008) [2023-12-26 18:28:57,863][105620] Updated weights for policy 1, policy_version 428711 (0.0008) [2023-12-26 18:28:57,933][105692] Updated weights for policy 0, policy_version 428210 (0.0009) [2023-12-26 18:28:57,993][105692] Updated weights for policy 0, policy_version 428220 (0.0009) [2023-12-26 18:28:58,049][105692] Updated weights for policy 0, policy_version 428230 (0.0009) [2023-12-26 18:28:58,598][105620] Updated weights for policy 1, policy_version 428721 (0.0009) [2023-12-26 18:28:58,657][105620] Updated weights for policy 1, policy_version 428731 (0.0009) [2023-12-26 18:28:58,725][105620] Updated weights for policy 1, policy_version 428741 (0.0007) [2023-12-26 18:28:58,815][105692] Updated weights for policy 0, policy_version 428240 (0.0009) [2023-12-26 18:28:58,878][105692] Updated weights for policy 0, policy_version 428250 (0.0008) [2023-12-26 18:28:58,942][105692] Updated weights for policy 0, policy_version 428260 (0.0007) [2023-12-26 18:28:59,568][105620] Updated weights for policy 1, policy_version 428751 (0.0009) [2023-12-26 18:28:59,615][105620] Updated weights for policy 1, policy_version 428761 (0.0008) [2023-12-26 18:28:59,666][105692] Updated weights for policy 0, policy_version 428270 (0.0008) [2023-12-26 18:28:59,672][105620] Updated weights for policy 1, policy_version 428771 (0.0009) [2023-12-26 18:28:59,726][105692] Updated weights for policy 0, policy_version 428280 (0.0007) [2023-12-26 18:28:59,773][105692] Updated weights for policy 0, policy_version 428290 (0.0009) [2023-12-26 18:29:00,388][105620] Updated weights for policy 1, policy_version 428781 (0.0008) [2023-12-26 18:29:00,443][105620] Updated weights for policy 1, policy_version 428791 (0.0010) [2023-12-26 18:29:00,510][105620] Updated weights for policy 1, policy_version 428801 (0.0006) [2023-12-26 18:29:00,527][105692] Updated weights for policy 0, policy_version 428300 (0.0009) [2023-12-26 18:29:00,587][105692] Updated weights for policy 0, policy_version 428310 (0.0010) [2023-12-26 18:29:00,653][105692] Updated weights for policy 0, policy_version 428320 (0.0010) [2023-12-26 18:29:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 219455488. Throughput: 0: 9573.6, 1: 9894.1. Samples: 219428428. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-26 18:29:01,062][104569] Avg episode reward: [(0, '9356.535'), (1, '9266.017')] [2023-12-26 18:29:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000428328_109666304.pth... [2023-12-26 18:29:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000428808_109789184.pth... [2023-12-26 18:29:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000427208_109379584.pth [2023-12-26 18:29:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000427656_109494272.pth [2023-12-26 18:29:01,145][105620] Updated weights for policy 1, policy_version 428811 (0.0007) [2023-12-26 18:29:01,208][105620] Updated weights for policy 1, policy_version 428821 (0.0007) [2023-12-26 18:29:01,265][105620] Updated weights for policy 1, policy_version 428831 (0.0006) [2023-12-26 18:29:01,389][105692] Updated weights for policy 0, policy_version 428330 (0.0010) [2023-12-26 18:29:01,449][105692] Updated weights for policy 0, policy_version 428340 (0.0011) [2023-12-26 18:29:01,494][105692] Updated weights for policy 0, policy_version 428350 (0.0010) [2023-12-26 18:29:01,546][105692] Updated weights for policy 0, policy_version 428360 (0.0010) [2023-12-26 18:29:01,881][105620] Updated weights for policy 1, policy_version 428841 (0.0005) [2023-12-26 18:29:01,934][105620] Updated weights for policy 1, policy_version 428851 (0.0005) [2023-12-26 18:29:01,986][105620] Updated weights for policy 1, policy_version 428861 (0.0005) [2023-12-26 18:29:02,040][105620] Updated weights for policy 1, policy_version 428871 (0.0008) [2023-12-26 18:29:02,335][105692] Updated weights for policy 0, policy_version 428370 (0.0008) [2023-12-26 18:29:02,401][105692] Updated weights for policy 0, policy_version 428380 (0.0008) [2023-12-26 18:29:02,454][105692] Updated weights for policy 0, policy_version 428390 (0.0010) [2023-12-26 18:29:02,673][105620] Updated weights for policy 1, policy_version 428881 (0.0006) [2023-12-26 18:29:02,737][105620] Updated weights for policy 1, policy_version 428891 (0.0005) [2023-12-26 18:29:02,798][105620] Updated weights for policy 1, policy_version 428901 (0.0010) [2023-12-26 18:29:03,131][105692] Updated weights for policy 0, policy_version 428400 (0.0010) [2023-12-26 18:29:03,184][105692] Updated weights for policy 0, policy_version 428410 (0.0010) [2023-12-26 18:29:03,241][105692] Updated weights for policy 0, policy_version 428420 (0.0010) [2023-12-26 18:29:03,429][105620] Updated weights for policy 1, policy_version 428911 (0.0007) [2023-12-26 18:29:03,478][105620] Updated weights for policy 1, policy_version 428921 (0.0005) [2023-12-26 18:29:03,501][105586] KL-divergence is very high: 124.4166 [2023-12-26 18:29:03,540][105620] Updated weights for policy 1, policy_version 428931 (0.0005) [2023-12-26 18:29:03,548][105586] KL-divergence is very high: 221.0550 [2023-12-26 18:29:03,925][105692] Updated weights for policy 0, policy_version 428430 (0.0010) [2023-12-26 18:29:03,994][105692] Updated weights for policy 0, policy_version 428440 (0.0010) [2023-12-26 18:29:04,056][105692] Updated weights for policy 0, policy_version 428450 (0.0010) [2023-12-26 18:29:04,114][105620] Updated weights for policy 1, policy_version 428941 (0.0008) [2023-12-26 18:29:04,180][105620] Updated weights for policy 1, policy_version 428951 (0.0010) [2023-12-26 18:29:04,242][105620] Updated weights for policy 1, policy_version 428961 (0.0011) [2023-12-26 18:29:04,798][105692] Updated weights for policy 0, policy_version 428460 (0.0011) [2023-12-26 18:29:04,846][105692] Updated weights for policy 0, policy_version 428470 (0.0010) [2023-12-26 18:29:04,891][105692] Updated weights for policy 0, policy_version 428480 (0.0010) [2023-12-26 18:29:04,917][105620] Updated weights for policy 1, policy_version 428971 (0.0011) [2023-12-26 18:29:04,974][105620] Updated weights for policy 1, policy_version 428981 (0.0010) [2023-12-26 18:29:05,036][105620] Updated weights for policy 1, policy_version 428991 (0.0010) [2023-12-26 18:29:05,610][105620] Updated weights for policy 1, policy_version 429001 (0.0010) [2023-12-26 18:29:05,611][105692] Updated weights for policy 0, policy_version 428490 (0.0010) [2023-12-26 18:29:05,660][105620] Updated weights for policy 1, policy_version 429011 (0.0007) [2023-12-26 18:29:05,669][105692] Updated weights for policy 0, policy_version 428500 (0.0010) [2023-12-26 18:29:05,709][105620] Updated weights for policy 1, policy_version 429021 (0.0008) [2023-12-26 18:29:05,724][105692] Updated weights for policy 0, policy_version 428510 (0.0006) [2023-12-26 18:29:05,766][105620] Updated weights for policy 1, policy_version 429032 (0.0009) [2023-12-26 18:29:05,780][105692] Updated weights for policy 0, policy_version 428520 (0.0006) [2023-12-26 18:29:06,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 219561984. Throughput: 0: 9574.7, 1: 10025.6. Samples: 219548816. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-26 18:29:06,063][104569] Avg episode reward: [(0, '9355.782'), (1, '8990.575')] [2023-12-26 18:29:06,468][105692] Updated weights for policy 0, policy_version 428530 (0.0009) [2023-12-26 18:29:06,527][105692] Updated weights for policy 0, policy_version 428540 (0.0007) [2023-12-26 18:29:06,545][105620] Updated weights for policy 1, policy_version 429042 (0.0008) [2023-12-26 18:29:06,588][105692] Updated weights for policy 0, policy_version 428550 (0.0007) [2023-12-26 18:29:06,603][105620] Updated weights for policy 1, policy_version 429052 (0.0006) [2023-12-26 18:29:06,650][105620] Updated weights for policy 1, policy_version 429062 (0.0008) [2023-12-26 18:29:07,259][105692] Updated weights for policy 0, policy_version 428560 (0.0005) [2023-12-26 18:29:07,310][105692] Updated weights for policy 0, policy_version 428570 (0.0006) [2023-12-26 18:29:07,366][105620] Updated weights for policy 1, policy_version 429072 (0.0007) [2023-12-26 18:29:07,381][105692] Updated weights for policy 0, policy_version 428580 (0.0009) [2023-12-26 18:29:07,413][105620] Updated weights for policy 1, policy_version 429082 (0.0007) [2023-12-26 18:29:07,470][105620] Updated weights for policy 1, policy_version 429092 (0.0008) [2023-12-26 18:29:08,026][105692] Updated weights for policy 0, policy_version 428590 (0.0008) [2023-12-26 18:29:08,085][105692] Updated weights for policy 0, policy_version 428600 (0.0009) [2023-12-26 18:29:08,136][105692] Updated weights for policy 0, policy_version 428610 (0.0007) [2023-12-26 18:29:08,292][105620] Updated weights for policy 1, policy_version 429102 (0.0010) [2023-12-26 18:29:08,352][105620] Updated weights for policy 1, policy_version 429112 (0.0008) [2023-12-26 18:29:08,413][105620] Updated weights for policy 1, policy_version 429122 (0.0009) [2023-12-26 18:29:08,771][105692] Updated weights for policy 0, policy_version 428620 (0.0005) [2023-12-26 18:29:08,824][105692] Updated weights for policy 0, policy_version 428630 (0.0006) [2023-12-26 18:29:08,876][105692] Updated weights for policy 0, policy_version 428640 (0.0005) [2023-12-26 18:29:09,245][105620] Updated weights for policy 1, policy_version 429132 (0.0008) [2023-12-26 18:29:09,308][105620] Updated weights for policy 1, policy_version 429142 (0.0010) [2023-12-26 18:29:09,383][105620] Updated weights for policy 1, policy_version 429152 (0.0009) [2023-12-26 18:29:09,601][105692] Updated weights for policy 0, policy_version 428650 (0.0007) [2023-12-26 18:29:09,663][105692] Updated weights for policy 0, policy_version 428660 (0.0008) [2023-12-26 18:29:09,721][105692] Updated weights for policy 0, policy_version 428670 (0.0008) [2023-12-26 18:29:09,773][105692] Updated weights for policy 0, policy_version 428680 (0.0008) [2023-12-26 18:29:10,166][105620] Updated weights for policy 1, policy_version 429162 (0.0008) [2023-12-26 18:29:10,223][105620] Updated weights for policy 1, policy_version 429172 (0.0008) [2023-12-26 18:29:10,259][105586] KL-divergence is very high: 103.0074 [2023-12-26 18:29:10,282][105620] Updated weights for policy 1, policy_version 429182 (0.0010) [2023-12-26 18:29:10,340][105620] Updated weights for policy 1, policy_version 429192 (0.0010) [2023-12-26 18:29:10,500][105692] Updated weights for policy 0, policy_version 428690 (0.0009) [2023-12-26 18:29:10,556][105692] Updated weights for policy 0, policy_version 428700 (0.0009) [2023-12-26 18:29:10,607][105692] Updated weights for policy 0, policy_version 428710 (0.0009) [2023-12-26 18:29:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 219652096. Throughput: 0: 9660.8, 1: 9967.1. Samples: 219664504. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-26 18:29:11,062][104569] Avg episode reward: [(0, '9354.979'), (1, '8990.452')] [2023-12-26 18:29:11,155][105620] Updated weights for policy 1, policy_version 429202 (0.0008) [2023-12-26 18:29:11,211][105620] Updated weights for policy 1, policy_version 429212 (0.0008) [2023-12-26 18:29:11,275][105620] Updated weights for policy 1, policy_version 429222 (0.0009) [2023-12-26 18:29:11,312][105692] Updated weights for policy 0, policy_version 428720 (0.0008) [2023-12-26 18:29:11,373][105692] Updated weights for policy 0, policy_version 428730 (0.0009) [2023-12-26 18:29:11,444][105692] Updated weights for policy 0, policy_version 428740 (0.0007) [2023-12-26 18:29:12,040][105620] Updated weights for policy 1, policy_version 429232 (0.0005) [2023-12-26 18:29:12,105][105620] Updated weights for policy 1, policy_version 429242 (0.0006) [2023-12-26 18:29:12,166][105620] Updated weights for policy 1, policy_version 429252 (0.0005) [2023-12-26 18:29:12,222][105692] Updated weights for policy 0, policy_version 428750 (0.0007) [2023-12-26 18:29:12,283][105692] Updated weights for policy 0, policy_version 428760 (0.0009) [2023-12-26 18:29:12,345][105692] Updated weights for policy 0, policy_version 428770 (0.0009) [2023-12-26 18:29:12,760][105620] Updated weights for policy 1, policy_version 429262 (0.0005) [2023-12-26 18:29:12,821][105620] Updated weights for policy 1, policy_version 429272 (0.0009) [2023-12-26 18:29:12,880][105620] Updated weights for policy 1, policy_version 429282 (0.0009) [2023-12-26 18:29:13,207][105692] Updated weights for policy 0, policy_version 428780 (0.0009) [2023-12-26 18:29:13,272][105692] Updated weights for policy 0, policy_version 428790 (0.0009) [2023-12-26 18:29:13,329][105692] Updated weights for policy 0, policy_version 428800 (0.0010) [2023-12-26 18:29:13,467][105620] Updated weights for policy 1, policy_version 429292 (0.0008) [2023-12-26 18:29:13,520][105620] Updated weights for policy 1, policy_version 429302 (0.0005) [2023-12-26 18:29:13,577][105620] Updated weights for policy 1, policy_version 429312 (0.0005) [2023-12-26 18:29:14,136][105692] Updated weights for policy 0, policy_version 428811 (0.0011) [2023-12-26 18:29:14,195][105692] Updated weights for policy 0, policy_version 428821 (0.0009) [2023-12-26 18:29:14,241][105692] Updated weights for policy 0, policy_version 428831 (0.0008) [2023-12-26 18:29:14,260][105620] Updated weights for policy 1, policy_version 429322 (0.0006) [2023-12-26 18:29:14,323][105620] Updated weights for policy 1, policy_version 429332 (0.0007) [2023-12-26 18:29:14,372][105620] Updated weights for policy 1, policy_version 429342 (0.0006) [2023-12-26 18:29:14,419][105620] Updated weights for policy 1, policy_version 429352 (0.0009) [2023-12-26 18:29:15,072][105692] Updated weights for policy 0, policy_version 428841 (0.0007) [2023-12-26 18:29:15,100][105620] Updated weights for policy 1, policy_version 429362 (0.0007) [2023-12-26 18:29:15,138][105692] Updated weights for policy 0, policy_version 428851 (0.0010) [2023-12-26 18:29:15,160][105620] Updated weights for policy 1, policy_version 429372 (0.0008) [2023-12-26 18:29:15,203][105692] Updated weights for policy 0, policy_version 428861 (0.0007) [2023-12-26 18:29:15,226][105620] Updated weights for policy 1, policy_version 429382 (0.0007) [2023-12-26 18:29:15,268][105692] Updated weights for policy 0, policy_version 428871 (0.0007) [2023-12-26 18:29:15,987][105620] Updated weights for policy 1, policy_version 429392 (0.0009) [2023-12-26 18:29:15,997][105692] Updated weights for policy 0, policy_version 428881 (0.0006) [2023-12-26 18:29:16,018][105586] KL-divergence is very high: 115.6040 [2023-12-26 18:29:16,028][105586] KL-divergence is very high: 111.5866 [2023-12-26 18:29:16,035][105620] Updated weights for policy 1, policy_version 429402 (0.0009) [2023-12-26 18:29:16,044][105692] Updated weights for policy 0, policy_version 428891 (0.0005) [2023-12-26 18:29:16,059][105586] KL-divergence is very high: 137.8486 [2023-12-26 18:29:16,062][104569] Fps is (10 sec: 18021.4, 60 sec: 19387.6, 300 sec: 19521.9). Total num frames: 219742208. Throughput: 0: 9586.7, 1: 9906.2. Samples: 219721760. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-26 18:29:16,064][104569] Avg episode reward: [(0, '9354.206'), (1, '8895.729')] [2023-12-26 18:29:16,071][105586] KL-divergence is very high: 120.7365 [2023-12-26 18:29:16,086][105620] Updated weights for policy 1, policy_version 429412 (0.0008) [2023-12-26 18:29:16,096][105692] Updated weights for policy 0, policy_version 428901 (0.0005) [2023-12-26 18:29:16,100][105586] KL-divergence is very high: 132.4805 [2023-12-26 18:29:16,104][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000429416_109944832.pth... [2023-12-26 18:29:16,108][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000428232_109641728.pth [2023-12-26 18:29:16,109][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000428904_109813760.pth... [2023-12-26 18:29:16,114][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000427784_109527040.pth [2023-12-26 18:29:16,731][105692] Updated weights for policy 0, policy_version 428911 (0.0006) [2023-12-26 18:29:16,787][105692] Updated weights for policy 0, policy_version 428921 (0.0005) [2023-12-26 18:29:16,842][105692] Updated weights for policy 0, policy_version 428931 (0.0006) [2023-12-26 18:29:16,927][105620] Updated weights for policy 1, policy_version 429422 (0.0009) [2023-12-26 18:29:16,993][105620] Updated weights for policy 1, policy_version 429432 (0.0010) [2023-12-26 18:29:17,053][105620] Updated weights for policy 1, policy_version 429442 (0.0010) [2023-12-26 18:29:17,449][105692] Updated weights for policy 0, policy_version 428941 (0.0007) [2023-12-26 18:29:17,506][105692] Updated weights for policy 0, policy_version 428951 (0.0008) [2023-12-26 18:29:17,555][105692] Updated weights for policy 0, policy_version 428961 (0.0008) [2023-12-26 18:29:17,816][105620] Updated weights for policy 1, policy_version 429452 (0.0010) [2023-12-26 18:29:17,873][105620] Updated weights for policy 1, policy_version 429462 (0.0010) [2023-12-26 18:29:17,941][105620] Updated weights for policy 1, policy_version 429472 (0.0010) [2023-12-26 18:29:18,173][105692] Updated weights for policy 0, policy_version 428971 (0.0008) [2023-12-26 18:29:18,238][105692] Updated weights for policy 0, policy_version 428981 (0.0007) [2023-12-26 18:29:18,293][105692] Updated weights for policy 0, policy_version 428991 (0.0008) [2023-12-26 18:29:18,680][105620] Updated weights for policy 1, policy_version 429482 (0.0010) [2023-12-26 18:29:18,742][105620] Updated weights for policy 1, policy_version 429492 (0.0010) [2023-12-26 18:29:18,800][105620] Updated weights for policy 1, policy_version 429502 (0.0010) [2023-12-26 18:29:18,862][105620] Updated weights for policy 1, policy_version 429512 (0.0010) [2023-12-26 18:29:18,964][105692] Updated weights for policy 0, policy_version 429001 (0.0008) [2023-12-26 18:29:19,021][105692] Updated weights for policy 0, policy_version 429011 (0.0008) [2023-12-26 18:29:19,085][105692] Updated weights for policy 0, policy_version 429021 (0.0007) [2023-12-26 18:29:19,143][105692] Updated weights for policy 0, policy_version 429031 (0.0008) [2023-12-26 18:29:19,567][105620] Updated weights for policy 1, policy_version 429522 (0.0011) [2023-12-26 18:29:19,630][105620] Updated weights for policy 1, policy_version 429532 (0.0011) [2023-12-26 18:29:19,685][105620] Updated weights for policy 1, policy_version 429542 (0.0010) [2023-12-26 18:29:19,898][105692] Updated weights for policy 0, policy_version 429041 (0.0007) [2023-12-26 18:29:19,963][105692] Updated weights for policy 0, policy_version 429051 (0.0009) [2023-12-26 18:29:20,028][105692] Updated weights for policy 0, policy_version 429061 (0.0008) [2023-12-26 18:29:20,389][105620] Updated weights for policy 1, policy_version 429552 (0.0010) [2023-12-26 18:29:20,448][105620] Updated weights for policy 1, policy_version 429562 (0.0011) [2023-12-26 18:29:20,510][105620] Updated weights for policy 1, policy_version 429572 (0.0009) [2023-12-26 18:29:20,760][105692] Updated weights for policy 0, policy_version 429071 (0.0007) [2023-12-26 18:29:20,824][105692] Updated weights for policy 0, policy_version 429081 (0.0011) [2023-12-26 18:29:20,880][105692] Updated weights for policy 0, policy_version 429091 (0.0011) [2023-12-26 18:29:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 219848704. Throughput: 0: 9642.9, 1: 9876.0. Samples: 219838056. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-26 18:29:21,063][104569] Avg episode reward: [(0, '9353.680'), (1, '8711.657')] [2023-12-26 18:29:21,261][105620] Updated weights for policy 1, policy_version 429582 (0.0010) [2023-12-26 18:29:21,317][105620] Updated weights for policy 1, policy_version 429592 (0.0011) [2023-12-26 18:29:21,385][105620] Updated weights for policy 1, policy_version 429602 (0.0010) [2023-12-26 18:29:21,662][105692] Updated weights for policy 0, policy_version 429101 (0.0011) [2023-12-26 18:29:21,734][105692] Updated weights for policy 0, policy_version 429111 (0.0011) [2023-12-26 18:29:21,788][105692] Updated weights for policy 0, policy_version 429121 (0.0010) [2023-12-26 18:29:22,097][105620] Updated weights for policy 1, policy_version 429612 (0.0009) [2023-12-26 18:29:22,160][105620] Updated weights for policy 1, policy_version 429622 (0.0006) [2023-12-26 18:29:22,214][105620] Updated weights for policy 1, policy_version 429632 (0.0006) [2023-12-26 18:29:22,569][105692] Updated weights for policy 0, policy_version 429131 (0.0010) [2023-12-26 18:29:22,625][105692] Updated weights for policy 0, policy_version 429141 (0.0008) [2023-12-26 18:29:22,697][105692] Updated weights for policy 0, policy_version 429151 (0.0006) [2023-12-26 18:29:22,874][105620] Updated weights for policy 1, policy_version 429642 (0.0008) [2023-12-26 18:29:22,925][105620] Updated weights for policy 1, policy_version 429652 (0.0009) [2023-12-26 18:29:22,994][105620] Updated weights for policy 1, policy_version 429662 (0.0010) [2023-12-26 18:29:23,056][105620] Updated weights for policy 1, policy_version 429672 (0.0010) [2023-12-26 18:29:23,412][105692] Updated weights for policy 0, policy_version 429162 (0.0007) [2023-12-26 18:29:23,463][105692] Updated weights for policy 0, policy_version 429172 (0.0008) [2023-12-26 18:29:23,507][105692] Updated weights for policy 0, policy_version 429182 (0.0008) [2023-12-26 18:29:23,563][105692] Updated weights for policy 0, policy_version 429192 (0.0008) [2023-12-26 18:29:23,786][105620] Updated weights for policy 1, policy_version 429682 (0.0006) [2023-12-26 18:29:23,832][105620] Updated weights for policy 1, policy_version 429692 (0.0005) [2023-12-26 18:29:23,887][105620] Updated weights for policy 1, policy_version 429702 (0.0005) [2023-12-26 18:29:24,250][105692] Updated weights for policy 0, policy_version 429202 (0.0008) [2023-12-26 18:29:24,302][105692] Updated weights for policy 0, policy_version 429212 (0.0008) [2023-12-26 18:29:24,384][105692] Updated weights for policy 0, policy_version 429222 (0.0008) [2023-12-26 18:29:24,519][105620] Updated weights for policy 1, policy_version 429712 (0.0009) [2023-12-26 18:29:24,567][105620] Updated weights for policy 1, policy_version 429722 (0.0010) [2023-12-26 18:29:24,620][105620] Updated weights for policy 1, policy_version 429732 (0.0010) [2023-12-26 18:29:25,135][105692] Updated weights for policy 0, policy_version 429232 (0.0010) [2023-12-26 18:29:25,191][105692] Updated weights for policy 0, policy_version 429243 (0.0012) [2023-12-26 18:29:25,244][105620] Updated weights for policy 1, policy_version 429742 (0.0008) [2023-12-26 18:29:25,244][105692] Updated weights for policy 0, policy_version 429254 (0.0009) [2023-12-26 18:29:25,306][105620] Updated weights for policy 1, policy_version 429752 (0.0007) [2023-12-26 18:29:25,377][105620] Updated weights for policy 1, policy_version 429762 (0.0005) [2023-12-26 18:29:25,946][105692] Updated weights for policy 0, policy_version 429264 (0.0008) [2023-12-26 18:29:26,009][105692] Updated weights for policy 0, policy_version 429274 (0.0009) [2023-12-26 18:29:26,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 219938816. Throughput: 0: 9619.6, 1: 9922.7. Samples: 219953792. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-26 18:29:26,063][104569] Avg episode reward: [(0, '9353.597'), (1, '8714.766')] [2023-12-26 18:29:26,076][105692] Updated weights for policy 0, policy_version 429284 (0.0009) [2023-12-26 18:29:26,078][105620] Updated weights for policy 1, policy_version 429772 (0.0007) [2023-12-26 18:29:26,137][105620] Updated weights for policy 1, policy_version 429783 (0.0009) [2023-12-26 18:29:26,185][105620] Updated weights for policy 1, policy_version 429793 (0.0006) [2023-12-26 18:29:26,637][105692] Updated weights for policy 0, policy_version 429294 (0.0006) [2023-12-26 18:29:26,696][105692] Updated weights for policy 0, policy_version 429304 (0.0005) [2023-12-26 18:29:26,755][105692] Updated weights for policy 0, policy_version 429314 (0.0007) [2023-12-26 18:29:26,898][105620] Updated weights for policy 1, policy_version 429803 (0.0007) [2023-12-26 18:29:26,956][105620] Updated weights for policy 1, policy_version 429813 (0.0010) [2023-12-26 18:29:27,017][105620] Updated weights for policy 1, policy_version 429823 (0.0010) [2023-12-26 18:29:27,423][105692] Updated weights for policy 0, policy_version 429324 (0.0006) [2023-12-26 18:29:27,471][105692] Updated weights for policy 0, policy_version 429334 (0.0005) [2023-12-26 18:29:27,518][105692] Updated weights for policy 0, policy_version 429344 (0.0005) [2023-12-26 18:29:27,631][105620] Updated weights for policy 1, policy_version 429833 (0.0009) [2023-12-26 18:29:27,683][105620] Updated weights for policy 1, policy_version 429843 (0.0005) [2023-12-26 18:29:27,731][105620] Updated weights for policy 1, policy_version 429853 (0.0005) [2023-12-26 18:29:27,778][105620] Updated weights for policy 1, policy_version 429863 (0.0005) [2023-12-26 18:29:28,168][105692] Updated weights for policy 0, policy_version 429354 (0.0005) [2023-12-26 18:29:28,220][105692] Updated weights for policy 0, policy_version 429364 (0.0005) [2023-12-26 18:29:28,275][105692] Updated weights for policy 0, policy_version 429374 (0.0005) [2023-12-26 18:29:28,334][105692] Updated weights for policy 0, policy_version 429384 (0.0007) [2023-12-26 18:29:28,470][105620] Updated weights for policy 1, policy_version 429873 (0.0010) [2023-12-26 18:29:28,538][105620] Updated weights for policy 1, policy_version 429883 (0.0010) [2023-12-26 18:29:28,602][105620] Updated weights for policy 1, policy_version 429893 (0.0010) [2023-12-26 18:29:28,890][105692] Updated weights for policy 0, policy_version 429394 (0.0005) [2023-12-26 18:29:28,936][105692] Updated weights for policy 0, policy_version 429404 (0.0005) [2023-12-26 18:29:28,995][105692] Updated weights for policy 0, policy_version 429414 (0.0008) [2023-12-26 18:29:29,320][105620] Updated weights for policy 1, policy_version 429903 (0.0010) [2023-12-26 18:29:29,386][105620] Updated weights for policy 1, policy_version 429913 (0.0011) [2023-12-26 18:29:29,446][105620] Updated weights for policy 1, policy_version 429923 (0.0010) [2023-12-26 18:29:29,718][105692] Updated weights for policy 0, policy_version 429424 (0.0007) [2023-12-26 18:29:29,776][105692] Updated weights for policy 0, policy_version 429434 (0.0010) [2023-12-26 18:29:29,831][105692] Updated weights for policy 0, policy_version 429444 (0.0010) [2023-12-26 18:29:30,148][105620] Updated weights for policy 1, policy_version 429933 (0.0008) [2023-12-26 18:29:30,196][105620] Updated weights for policy 1, policy_version 429943 (0.0008) [2023-12-26 18:29:30,255][105620] Updated weights for policy 1, policy_version 429953 (0.0008) [2023-12-26 18:29:30,565][105692] Updated weights for policy 0, policy_version 429454 (0.0010) [2023-12-26 18:29:30,620][105692] Updated weights for policy 0, policy_version 429464 (0.0010) [2023-12-26 18:29:30,668][105692] Updated weights for policy 0, policy_version 429474 (0.0010) [2023-12-26 18:29:30,958][105620] Updated weights for policy 1, policy_version 429963 (0.0008) [2023-12-26 18:29:31,008][105620] Updated weights for policy 1, policy_version 429973 (0.0007) [2023-12-26 18:29:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 220045312. Throughput: 0: 9684.7, 1: 9963.9. Samples: 220018024. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-26 18:29:31,063][104569] Avg episode reward: [(0, '9353.508'), (1, '9174.387')] [2023-12-26 18:29:31,069][105620] Updated weights for policy 1, policy_version 429983 (0.0007) [2023-12-26 18:29:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000429480_109961216.pth... [2023-12-26 18:29:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000428328_109666304.pth [2023-12-26 18:29:31,121][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000429992_110092288.pth... [2023-12-26 18:29:31,143][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000428808_109789184.pth [2023-12-26 18:29:31,343][105692] Updated weights for policy 0, policy_version 429484 (0.0011) [2023-12-26 18:29:31,407][105692] Updated weights for policy 0, policy_version 429494 (0.0009) [2023-12-26 18:29:31,459][105692] Updated weights for policy 0, policy_version 429504 (0.0005) [2023-12-26 18:29:31,939][105620] Updated weights for policy 1, policy_version 429993 (0.0008) [2023-12-26 18:29:31,990][105620] Updated weights for policy 1, policy_version 430003 (0.0009) [2023-12-26 18:29:32,049][105620] Updated weights for policy 1, policy_version 430013 (0.0009) [2023-12-26 18:29:32,093][105692] Updated weights for policy 0, policy_version 429514 (0.0005) [2023-12-26 18:29:32,105][105620] Updated weights for policy 1, policy_version 430023 (0.0010) [2023-12-26 18:29:32,159][105692] Updated weights for policy 0, policy_version 429524 (0.0009) [2023-12-26 18:29:32,214][105692] Updated weights for policy 0, policy_version 429534 (0.0008) [2023-12-26 18:29:32,269][105692] Updated weights for policy 0, policy_version 429544 (0.0006) [2023-12-26 18:29:32,822][105620] Updated weights for policy 1, policy_version 430033 (0.0009) [2023-12-26 18:29:32,887][105620] Updated weights for policy 1, policy_version 430043 (0.0009) [2023-12-26 18:29:32,948][105620] Updated weights for policy 1, policy_version 430053 (0.0008) [2023-12-26 18:29:33,017][105692] Updated weights for policy 0, policy_version 429554 (0.0009) [2023-12-26 18:29:33,072][105692] Updated weights for policy 0, policy_version 429564 (0.0010) [2023-12-26 18:29:33,125][105692] Updated weights for policy 0, policy_version 429574 (0.0010) [2023-12-26 18:29:33,547][105620] Updated weights for policy 1, policy_version 430063 (0.0009) [2023-12-26 18:29:33,599][105620] Updated weights for policy 1, policy_version 430073 (0.0009) [2023-12-26 18:29:33,656][105620] Updated weights for policy 1, policy_version 430083 (0.0010) [2023-12-26 18:29:33,845][105692] Updated weights for policy 0, policy_version 429584 (0.0010) [2023-12-26 18:29:33,893][105692] Updated weights for policy 0, policy_version 429594 (0.0010) [2023-12-26 18:29:33,947][105692] Updated weights for policy 0, policy_version 429604 (0.0010) [2023-12-26 18:29:34,342][105620] Updated weights for policy 1, policy_version 430093 (0.0008) [2023-12-26 18:29:34,397][105620] Updated weights for policy 1, policy_version 430103 (0.0005) [2023-12-26 18:29:34,462][105620] Updated weights for policy 1, policy_version 430113 (0.0005) [2023-12-26 18:29:34,630][105692] Updated weights for policy 0, policy_version 429614 (0.0007) [2023-12-26 18:29:34,695][105692] Updated weights for policy 0, policy_version 429624 (0.0006) [2023-12-26 18:29:34,758][105692] Updated weights for policy 0, policy_version 429634 (0.0006) [2023-12-26 18:29:35,146][105620] Updated weights for policy 1, policy_version 430123 (0.0007) [2023-12-26 18:29:35,197][105620] Updated weights for policy 1, policy_version 430133 (0.0010) [2023-12-26 18:29:35,199][105586] KL-divergence is very high: 109.2321 [2023-12-26 18:29:35,239][105586] KL-divergence is very high: 182.0541 [2023-12-26 18:29:35,244][105620] Updated weights for policy 1, policy_version 430143 (0.0010) [2023-12-26 18:29:35,282][105586] KL-divergence is very high: 174.2572 [2023-12-26 18:29:35,366][105692] Updated weights for policy 0, policy_version 429644 (0.0005) [2023-12-26 18:29:35,414][105692] Updated weights for policy 0, policy_version 429654 (0.0005) [2023-12-26 18:29:35,462][105692] Updated weights for policy 0, policy_version 429664 (0.0005) [2023-12-26 18:29:36,010][105620] Updated weights for policy 1, policy_version 430153 (0.0010) [2023-12-26 18:29:36,059][105692] Updated weights for policy 0, policy_version 429674 (0.0006) [2023-12-26 18:29:36,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 220143616. Throughput: 0: 9787.0, 1: 9862.4. Samples: 220137184. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-26 18:29:36,062][104569] Avg episode reward: [(0, '9172.164'), (1, '9263.892')] [2023-12-26 18:29:36,069][105620] Updated weights for policy 1, policy_version 430163 (0.0010) [2023-12-26 18:29:36,110][105692] Updated weights for policy 0, policy_version 429684 (0.0008) [2023-12-26 18:29:36,132][105620] Updated weights for policy 1, policy_version 430173 (0.0008) [2023-12-26 18:29:36,135][105585] KL-divergence is very high: 109.9350 [2023-12-26 18:29:36,173][105692] Updated weights for policy 0, policy_version 429694 (0.0011) [2023-12-26 18:29:36,185][105585] KL-divergence is very high: 114.5094 [2023-12-26 18:29:36,195][105620] Updated weights for policy 1, policy_version 430183 (0.0006) [2023-12-26 18:29:36,240][105692] Updated weights for policy 0, policy_version 429704 (0.0011) [2023-12-26 18:29:36,876][105620] Updated weights for policy 1, policy_version 430193 (0.0010) [2023-12-26 18:29:36,917][105692] Updated weights for policy 0, policy_version 429714 (0.0011) [2023-12-26 18:29:36,935][105620] Updated weights for policy 1, policy_version 430203 (0.0010) [2023-12-26 18:29:36,973][105692] Updated weights for policy 0, policy_version 429724 (0.0010) [2023-12-26 18:29:36,993][105620] Updated weights for policy 1, policy_version 430213 (0.0010) [2023-12-26 18:29:37,028][105692] Updated weights for policy 0, policy_version 429734 (0.0010) [2023-12-26 18:29:37,673][105620] Updated weights for policy 1, policy_version 430223 (0.0009) [2023-12-26 18:29:37,728][105620] Updated weights for policy 1, policy_version 430233 (0.0010) [2023-12-26 18:29:37,742][105692] Updated weights for policy 0, policy_version 429744 (0.0006) [2023-12-26 18:29:37,789][105620] Updated weights for policy 1, policy_version 430243 (0.0010) [2023-12-26 18:29:37,797][105692] Updated weights for policy 0, policy_version 429754 (0.0009) [2023-12-26 18:29:37,849][105692] Updated weights for policy 0, policy_version 429764 (0.0010) [2023-12-26 18:29:38,426][105620] Updated weights for policy 1, policy_version 430253 (0.0009) [2023-12-26 18:29:38,448][105692] Updated weights for policy 0, policy_version 429774 (0.0009) [2023-12-26 18:29:38,488][105620] Updated weights for policy 1, policy_version 430263 (0.0008) [2023-12-26 18:29:38,501][105692] Updated weights for policy 0, policy_version 429784 (0.0008) [2023-12-26 18:29:38,544][105620] Updated weights for policy 1, policy_version 430273 (0.0008) [2023-12-26 18:29:38,559][105692] Updated weights for policy 0, policy_version 429794 (0.0007) [2023-12-26 18:29:39,185][105692] Updated weights for policy 0, policy_version 429804 (0.0006) [2023-12-26 18:29:39,253][105692] Updated weights for policy 0, policy_version 429814 (0.0012) [2023-12-26 18:29:39,319][105692] Updated weights for policy 0, policy_version 429824 (0.0007) [2023-12-26 18:29:39,360][105620] Updated weights for policy 1, policy_version 430283 (0.0009) [2023-12-26 18:29:39,428][105620] Updated weights for policy 1, policy_version 430293 (0.0007) [2023-12-26 18:29:39,486][105620] Updated weights for policy 1, policy_version 430303 (0.0005) [2023-12-26 18:29:40,061][105692] Updated weights for policy 0, policy_version 429834 (0.0009) [2023-12-26 18:29:40,125][105692] Updated weights for policy 0, policy_version 429844 (0.0009) [2023-12-26 18:29:40,189][105692] Updated weights for policy 0, policy_version 429854 (0.0007) [2023-12-26 18:29:40,197][105620] Updated weights for policy 1, policy_version 430313 (0.0009) [2023-12-26 18:29:40,256][105692] Updated weights for policy 0, policy_version 429864 (0.0007) [2023-12-26 18:29:40,267][105620] Updated weights for policy 1, policy_version 430323 (0.0006) [2023-12-26 18:29:40,333][105620] Updated weights for policy 1, policy_version 430333 (0.0006) [2023-12-26 18:29:40,399][105620] Updated weights for policy 1, policy_version 430343 (0.0010) [2023-12-26 18:29:40,962][105692] Updated weights for policy 0, policy_version 429874 (0.0008) [2023-12-26 18:29:40,980][105620] Updated weights for policy 1, policy_version 430353 (0.0009) [2023-12-26 18:29:41,011][105692] Updated weights for policy 0, policy_version 429884 (0.0005) [2023-12-26 18:29:41,038][105620] Updated weights for policy 1, policy_version 430363 (0.0009) [2023-12-26 18:29:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 220241920. Throughput: 0: 9893.2, 1: 9864.5. Samples: 220258688. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-26 18:29:41,063][104569] Avg episode reward: [(0, '8991.565'), (1, '9263.892')] [2023-12-26 18:29:41,076][105692] Updated weights for policy 0, policy_version 429894 (0.0007) [2023-12-26 18:29:41,099][105620] Updated weights for policy 1, policy_version 430373 (0.0009) [2023-12-26 18:29:41,812][105692] Updated weights for policy 0, policy_version 429904 (0.0008) [2023-12-26 18:29:41,872][105620] Updated weights for policy 1, policy_version 430383 (0.0008) [2023-12-26 18:29:41,877][105692] Updated weights for policy 0, policy_version 429914 (0.0007) [2023-12-26 18:29:41,925][105620] Updated weights for policy 1, policy_version 430393 (0.0007) [2023-12-26 18:29:41,930][105692] Updated weights for policy 0, policy_version 429924 (0.0007) [2023-12-26 18:29:41,979][105620] Updated weights for policy 1, policy_version 430403 (0.0008) [2023-12-26 18:29:42,620][105692] Updated weights for policy 0, policy_version 429934 (0.0005) [2023-12-26 18:29:42,690][105692] Updated weights for policy 0, policy_version 429944 (0.0006) [2023-12-26 18:29:42,751][105620] Updated weights for policy 1, policy_version 430414 (0.0009) [2023-12-26 18:29:42,753][105692] Updated weights for policy 0, policy_version 429954 (0.0011) [2023-12-26 18:29:42,809][105620] Updated weights for policy 1, policy_version 430424 (0.0007) [2023-12-26 18:29:42,874][105620] Updated weights for policy 1, policy_version 430434 (0.0008) [2023-12-26 18:29:43,476][105620] Updated weights for policy 1, policy_version 430444 (0.0007) [2023-12-26 18:29:43,490][105692] Updated weights for policy 0, policy_version 429964 (0.0009) [2023-12-26 18:29:43,540][105620] Updated weights for policy 1, policy_version 430454 (0.0007) [2023-12-26 18:29:43,540][105692] Updated weights for policy 0, policy_version 429974 (0.0009) [2023-12-26 18:29:43,601][105692] Updated weights for policy 0, policy_version 429984 (0.0009) [2023-12-26 18:29:43,606][105620] Updated weights for policy 1, policy_version 430464 (0.0005) [2023-12-26 18:29:44,260][105620] Updated weights for policy 1, policy_version 430474 (0.0005) [2023-12-26 18:29:44,327][105620] Updated weights for policy 1, policy_version 430484 (0.0005) [2023-12-26 18:29:44,390][105620] Updated weights for policy 1, policy_version 430494 (0.0008) [2023-12-26 18:29:44,408][105692] Updated weights for policy 0, policy_version 429994 (0.0010) [2023-12-26 18:29:44,447][105620] Updated weights for policy 1, policy_version 430504 (0.0006) [2023-12-26 18:29:44,462][105692] Updated weights for policy 0, policy_version 430004 (0.0008) [2023-12-26 18:29:44,525][105692] Updated weights for policy 0, policy_version 430014 (0.0006) [2023-12-26 18:29:44,581][105692] Updated weights for policy 0, policy_version 430024 (0.0006) [2023-12-26 18:29:45,071][105620] Updated weights for policy 1, policy_version 430514 (0.0010) [2023-12-26 18:29:45,127][105620] Updated weights for policy 1, policy_version 430524 (0.0009) [2023-12-26 18:29:45,181][105620] Updated weights for policy 1, policy_version 430534 (0.0009) [2023-12-26 18:29:45,265][105692] Updated weights for policy 0, policy_version 430034 (0.0006) [2023-12-26 18:29:45,329][105692] Updated weights for policy 0, policy_version 430044 (0.0007) [2023-12-26 18:29:45,393][105692] Updated weights for policy 0, policy_version 430054 (0.0009) [2023-12-26 18:29:46,019][105620] Updated weights for policy 1, policy_version 430544 (0.0008) [2023-12-26 18:29:46,029][105692] Updated weights for policy 0, policy_version 430064 (0.0007) [2023-12-26 18:29:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 220340224. Throughput: 0: 9877.0, 1: 9858.8. Samples: 220316536. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-26 18:29:46,062][104569] Avg episode reward: [(0, '9173.487'), (1, '9264.117')] [2023-12-26 18:29:46,076][105620] Updated weights for policy 1, policy_version 430554 (0.0007) [2023-12-26 18:29:46,086][105692] Updated weights for policy 0, policy_version 430074 (0.0007) [2023-12-26 18:29:46,134][105692] Updated weights for policy 0, policy_version 430084 (0.0006) [2023-12-26 18:29:46,136][105620] Updated weights for policy 1, policy_version 430564 (0.0007) [2023-12-26 18:29:46,151][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000430088_110116864.pth... [2023-12-26 18:29:46,154][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000428904_109813760.pth [2023-12-26 18:29:46,155][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000430568_110239744.pth... [2023-12-26 18:29:46,158][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000429416_109944832.pth [2023-12-26 18:29:46,820][105692] Updated weights for policy 0, policy_version 430094 (0.0006) [2023-12-26 18:29:46,884][105692] Updated weights for policy 0, policy_version 430104 (0.0006) [2023-12-26 18:29:46,923][105620] Updated weights for policy 1, policy_version 430574 (0.0008) [2023-12-26 18:29:46,941][105692] Updated weights for policy 0, policy_version 430114 (0.0005) [2023-12-26 18:29:46,984][105620] Updated weights for policy 1, policy_version 430584 (0.0009) [2023-12-26 18:29:47,043][105620] Updated weights for policy 1, policy_version 430594 (0.0009) [2023-12-26 18:29:47,617][105692] Updated weights for policy 0, policy_version 430124 (0.0005) [2023-12-26 18:29:47,650][105620] Updated weights for policy 1, policy_version 430604 (0.0006) [2023-12-26 18:29:47,671][105692] Updated weights for policy 0, policy_version 430134 (0.0005) [2023-12-26 18:29:47,699][105620] Updated weights for policy 1, policy_version 430614 (0.0005) [2023-12-26 18:29:47,729][105692] Updated weights for policy 0, policy_version 430144 (0.0005) [2023-12-26 18:29:47,754][105620] Updated weights for policy 1, policy_version 430624 (0.0005) [2023-12-26 18:29:48,370][105692] Updated weights for policy 0, policy_version 430154 (0.0006) [2023-12-26 18:29:48,444][105692] Updated weights for policy 0, policy_version 430164 (0.0006) [2023-12-26 18:29:48,454][105620] Updated weights for policy 1, policy_version 430634 (0.0008) [2023-12-26 18:29:48,505][105692] Updated weights for policy 0, policy_version 430174 (0.0006) [2023-12-26 18:29:48,518][105620] Updated weights for policy 1, policy_version 430644 (0.0010) [2023-12-26 18:29:48,567][105692] Updated weights for policy 0, policy_version 430184 (0.0006) [2023-12-26 18:29:48,571][105620] Updated weights for policy 1, policy_version 430654 (0.0009) [2023-12-26 18:29:48,627][105620] Updated weights for policy 1, policy_version 430664 (0.0009) [2023-12-26 18:29:49,198][105692] Updated weights for policy 0, policy_version 430194 (0.0008) [2023-12-26 18:29:49,261][105692] Updated weights for policy 0, policy_version 430204 (0.0007) [2023-12-26 18:29:49,325][105692] Updated weights for policy 0, policy_version 430214 (0.0007) [2023-12-26 18:29:49,353][105620] Updated weights for policy 1, policy_version 430674 (0.0011) [2023-12-26 18:29:49,413][105620] Updated weights for policy 1, policy_version 430684 (0.0010) [2023-12-26 18:29:49,468][105620] Updated weights for policy 1, policy_version 430694 (0.0010) [2023-12-26 18:29:50,057][105692] Updated weights for policy 0, policy_version 430224 (0.0007) [2023-12-26 18:29:50,123][105692] Updated weights for policy 0, policy_version 430234 (0.0011) [2023-12-26 18:29:50,190][105692] Updated weights for policy 0, policy_version 430244 (0.0009) [2023-12-26 18:29:50,223][105620] Updated weights for policy 1, policy_version 430704 (0.0009) [2023-12-26 18:29:50,278][105620] Updated weights for policy 1, policy_version 430714 (0.0005) [2023-12-26 18:29:50,342][105620] Updated weights for policy 1, policy_version 430724 (0.0009) [2023-12-26 18:29:50,779][105692] Updated weights for policy 0, policy_version 430254 (0.0006) [2023-12-26 18:29:50,834][105692] Updated weights for policy 0, policy_version 430264 (0.0005) [2023-12-26 18:29:50,903][105692] Updated weights for policy 0, policy_version 430274 (0.0010) [2023-12-26 18:29:50,967][105620] Updated weights for policy 1, policy_version 430734 (0.0011) [2023-12-26 18:29:51,028][105620] Updated weights for policy 1, policy_version 430744 (0.0011) [2023-12-26 18:29:51,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 220446720. Throughput: 0: 9949.9, 1: 9739.0. Samples: 220434820. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-26 18:29:51,063][104569] Avg episode reward: [(0, '9263.508'), (1, '9264.205')] [2023-12-26 18:29:51,094][105620] Updated weights for policy 1, policy_version 430754 (0.0011) [2023-12-26 18:29:51,558][105692] Updated weights for policy 0, policy_version 430284 (0.0008) [2023-12-26 18:29:51,638][105692] Updated weights for policy 0, policy_version 430294 (0.0010) [2023-12-26 18:29:51,705][105692] Updated weights for policy 0, policy_version 430304 (0.0007) [2023-12-26 18:29:51,805][105620] Updated weights for policy 1, policy_version 430764 (0.0011) [2023-12-26 18:29:51,862][105620] Updated weights for policy 1, policy_version 430774 (0.0011) [2023-12-26 18:29:51,922][105620] Updated weights for policy 1, policy_version 430784 (0.0011) [2023-12-26 18:29:52,486][105692] Updated weights for policy 0, policy_version 430314 (0.0010) [2023-12-26 18:29:52,545][105692] Updated weights for policy 0, policy_version 430324 (0.0008) [2023-12-26 18:29:52,601][105692] Updated weights for policy 0, policy_version 430334 (0.0008) [2023-12-26 18:29:52,630][105620] Updated weights for policy 1, policy_version 430794 (0.0011) [2023-12-26 18:29:52,657][105692] Updated weights for policy 0, policy_version 430344 (0.0008) [2023-12-26 18:29:52,685][105620] Updated weights for policy 1, policy_version 430804 (0.0006) [2023-12-26 18:29:52,747][105620] Updated weights for policy 1, policy_version 430814 (0.0010) [2023-12-26 18:29:52,805][105620] Updated weights for policy 1, policy_version 430824 (0.0007) [2023-12-26 18:29:53,383][105692] Updated weights for policy 0, policy_version 430354 (0.0008) [2023-12-26 18:29:53,436][105692] Updated weights for policy 0, policy_version 430364 (0.0007) [2023-12-26 18:29:53,447][105620] Updated weights for policy 1, policy_version 430834 (0.0009) [2023-12-26 18:29:53,497][105692] Updated weights for policy 0, policy_version 430374 (0.0006) [2023-12-26 18:29:53,500][105620] Updated weights for policy 1, policy_version 430844 (0.0010) [2023-12-26 18:29:53,547][105620] Updated weights for policy 1, policy_version 430854 (0.0009) [2023-12-26 18:29:54,137][105620] Updated weights for policy 1, policy_version 430864 (0.0010) [2023-12-26 18:29:54,195][105620] Updated weights for policy 1, policy_version 430874 (0.0009) [2023-12-26 18:29:54,260][105620] Updated weights for policy 1, policy_version 430884 (0.0010) [2023-12-26 18:29:54,323][105692] Updated weights for policy 0, policy_version 430384 (0.0007) [2023-12-26 18:29:54,378][105692] Updated weights for policy 0, policy_version 430394 (0.0007) [2023-12-26 18:29:54,434][105692] Updated weights for policy 0, policy_version 430404 (0.0009) [2023-12-26 18:29:54,981][105620] Updated weights for policy 1, policy_version 430894 (0.0009) [2023-12-26 18:29:55,041][105620] Updated weights for policy 1, policy_version 430904 (0.0011) [2023-12-26 18:29:55,097][105620] Updated weights for policy 1, policy_version 430914 (0.0010) [2023-12-26 18:29:55,138][105692] Updated weights for policy 0, policy_version 430414 (0.0006) [2023-12-26 18:29:55,189][105692] Updated weights for policy 0, policy_version 430424 (0.0008) [2023-12-26 18:29:55,240][105692] Updated weights for policy 0, policy_version 430434 (0.0008) [2023-12-26 18:29:55,711][105620] Updated weights for policy 1, policy_version 430924 (0.0008) [2023-12-26 18:29:55,775][105620] Updated weights for policy 1, policy_version 430934 (0.0005) [2023-12-26 18:29:55,846][105620] Updated weights for policy 1, policy_version 430944 (0.0005) [2023-12-26 18:29:56,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 220545024. Throughput: 0: 9891.5, 1: 9894.3. Samples: 220554868. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-26 18:29:56,063][104569] Avg episode reward: [(0, '9262.228'), (1, '9264.008')] [2023-12-26 18:29:56,093][105692] Updated weights for policy 0, policy_version 430444 (0.0008) [2023-12-26 18:29:56,148][105692] Updated weights for policy 0, policy_version 430454 (0.0009) [2023-12-26 18:29:56,211][105692] Updated weights for policy 0, policy_version 430464 (0.0009) [2023-12-26 18:29:56,407][105620] Updated weights for policy 1, policy_version 430954 (0.0007) [2023-12-26 18:29:56,470][105620] Updated weights for policy 1, policy_version 430964 (0.0006) [2023-12-26 18:29:56,531][105620] Updated weights for policy 1, policy_version 430974 (0.0007) [2023-12-26 18:29:56,582][105620] Updated weights for policy 1, policy_version 430984 (0.0005) [2023-12-26 18:29:57,030][105692] Updated weights for policy 0, policy_version 430474 (0.0010) [2023-12-26 18:29:57,096][105692] Updated weights for policy 0, policy_version 430484 (0.0009) [2023-12-26 18:29:57,159][105692] Updated weights for policy 0, policy_version 430494 (0.0008) [2023-12-26 18:29:57,219][105692] Updated weights for policy 0, policy_version 430504 (0.0008) [2023-12-26 18:29:57,262][105620] Updated weights for policy 1, policy_version 430994 (0.0010) [2023-12-26 18:29:57,316][105620] Updated weights for policy 1, policy_version 431004 (0.0010) [2023-12-26 18:29:57,363][105620] Updated weights for policy 1, policy_version 431014 (0.0010) [2023-12-26 18:29:57,877][105692] Updated weights for policy 0, policy_version 430514 (0.0010) [2023-12-26 18:29:57,921][105692] Updated weights for policy 0, policy_version 430524 (0.0008) [2023-12-26 18:29:57,975][105692] Updated weights for policy 0, policy_version 430534 (0.0007) [2023-12-26 18:29:58,116][105620] Updated weights for policy 1, policy_version 431024 (0.0010) [2023-12-26 18:29:58,174][105620] Updated weights for policy 1, policy_version 431034 (0.0010) [2023-12-26 18:29:58,229][105620] Updated weights for policy 1, policy_version 431044 (0.0010) [2023-12-26 18:29:58,786][105692] Updated weights for policy 0, policy_version 430544 (0.0010) [2023-12-26 18:29:58,849][105692] Updated weights for policy 0, policy_version 430554 (0.0010) [2023-12-26 18:29:58,913][105692] Updated weights for policy 0, policy_version 430564 (0.0011) [2023-12-26 18:29:59,043][105620] Updated weights for policy 1, policy_version 431054 (0.0010) [2023-12-26 18:29:59,108][105620] Updated weights for policy 1, policy_version 431064 (0.0010) [2023-12-26 18:29:59,166][105620] Updated weights for policy 1, policy_version 431074 (0.0010) [2023-12-26 18:29:59,679][105692] Updated weights for policy 0, policy_version 430574 (0.0011) [2023-12-26 18:29:59,739][105692] Updated weights for policy 0, policy_version 430584 (0.0010) [2023-12-26 18:29:59,800][105692] Updated weights for policy 0, policy_version 430594 (0.0010) [2023-12-26 18:29:59,813][105620] Updated weights for policy 1, policy_version 431084 (0.0008) [2023-12-26 18:29:59,875][105620] Updated weights for policy 1, policy_version 431094 (0.0007) [2023-12-26 18:29:59,940][105620] Updated weights for policy 1, policy_version 431104 (0.0007) [2023-12-26 18:30:00,521][105692] Updated weights for policy 0, policy_version 430604 (0.0010) [2023-12-26 18:30:00,568][105692] Updated weights for policy 0, policy_version 430614 (0.0010) [2023-12-26 18:30:00,578][105620] Updated weights for policy 1, policy_version 431114 (0.0009) [2023-12-26 18:30:00,612][105692] Updated weights for policy 0, policy_version 430624 (0.0010) [2023-12-26 18:30:00,626][105620] Updated weights for policy 1, policy_version 431124 (0.0010) [2023-12-26 18:30:00,673][105620] Updated weights for policy 1, policy_version 431134 (0.0010) [2023-12-26 18:30:00,721][105620] Updated weights for policy 1, policy_version 431144 (0.0010) [2023-12-26 18:30:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 220643328. Throughput: 0: 9917.8, 1: 9860.9. Samples: 220611792. Policy #0 lag: (min: 45.0, avg: 55.6, max: 56.0) [2023-12-26 18:30:01,063][104569] Avg episode reward: [(0, '9061.865'), (1, '9172.526')] [2023-12-26 18:30:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000431144_110387200.pth... [2023-12-26 18:30:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000430632_110256128.pth... [2023-12-26 18:30:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000429480_109961216.pth [2023-12-26 18:30:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000429992_110092288.pth [2023-12-26 18:30:01,307][105692] Updated weights for policy 0, policy_version 430634 (0.0010) [2023-12-26 18:30:01,368][105692] Updated weights for policy 0, policy_version 430644 (0.0011) [2023-12-26 18:30:01,427][105692] Updated weights for policy 0, policy_version 430654 (0.0010) [2023-12-26 18:30:01,445][105620] Updated weights for policy 1, policy_version 431154 (0.0010) [2023-12-26 18:30:01,489][105692] Updated weights for policy 0, policy_version 430664 (0.0010) [2023-12-26 18:30:01,501][105620] Updated weights for policy 1, policy_version 431164 (0.0010) [2023-12-26 18:30:01,559][105620] Updated weights for policy 1, policy_version 431174 (0.0010) [2023-12-26 18:30:02,211][105692] Updated weights for policy 0, policy_version 430674 (0.0010) [2023-12-26 18:30:02,277][105692] Updated weights for policy 0, policy_version 430684 (0.0009) [2023-12-26 18:30:02,317][105620] Updated weights for policy 1, policy_version 431184 (0.0008) [2023-12-26 18:30:02,335][105692] Updated weights for policy 0, policy_version 430694 (0.0007) [2023-12-26 18:30:02,379][105620] Updated weights for policy 1, policy_version 431194 (0.0008) [2023-12-26 18:30:02,443][105620] Updated weights for policy 1, policy_version 431204 (0.0009) [2023-12-26 18:30:03,082][105692] Updated weights for policy 0, policy_version 430704 (0.0007) [2023-12-26 18:30:03,128][105692] Updated weights for policy 0, policy_version 430714 (0.0009) [2023-12-26 18:30:03,176][105692] Updated weights for policy 0, policy_version 430724 (0.0009) [2023-12-26 18:30:03,199][105620] Updated weights for policy 1, policy_version 431214 (0.0007) [2023-12-26 18:30:03,259][105620] Updated weights for policy 1, policy_version 431224 (0.0009) [2023-12-26 18:30:03,315][105620] Updated weights for policy 1, policy_version 431234 (0.0010) [2023-12-26 18:30:03,800][105692] Updated weights for policy 0, policy_version 430734 (0.0005) [2023-12-26 18:30:03,868][105692] Updated weights for policy 0, policy_version 430744 (0.0007) [2023-12-26 18:30:03,931][105692] Updated weights for policy 0, policy_version 430754 (0.0006) [2023-12-26 18:30:03,988][105620] Updated weights for policy 1, policy_version 431244 (0.0010) [2023-12-26 18:30:04,047][105620] Updated weights for policy 1, policy_version 431254 (0.0010) [2023-12-26 18:30:04,103][105620] Updated weights for policy 1, policy_version 431264 (0.0010) [2023-12-26 18:30:04,495][105692] Updated weights for policy 0, policy_version 430764 (0.0006) [2023-12-26 18:30:04,547][105692] Updated weights for policy 0, policy_version 430774 (0.0006) [2023-12-26 18:30:04,593][105692] Updated weights for policy 0, policy_version 430784 (0.0008) [2023-12-26 18:30:04,831][105620] Updated weights for policy 1, policy_version 431274 (0.0011) [2023-12-26 18:30:04,893][105620] Updated weights for policy 1, policy_version 431284 (0.0010) [2023-12-26 18:30:04,957][105620] Updated weights for policy 1, policy_version 431294 (0.0010) [2023-12-26 18:30:05,012][105620] Updated weights for policy 1, policy_version 431304 (0.0010) [2023-12-26 18:30:05,182][105692] Updated weights for policy 0, policy_version 430794 (0.0008) [2023-12-26 18:30:05,238][105692] Updated weights for policy 0, policy_version 430804 (0.0007) [2023-12-26 18:30:05,299][105692] Updated weights for policy 0, policy_version 430814 (0.0005) [2023-12-26 18:30:05,362][105692] Updated weights for policy 0, policy_version 430824 (0.0005) [2023-12-26 18:30:05,734][105620] Updated weights for policy 1, policy_version 431314 (0.0010) [2023-12-26 18:30:05,792][105620] Updated weights for policy 1, policy_version 431324 (0.0010) [2023-12-26 18:30:05,852][105620] Updated weights for policy 1, policy_version 431334 (0.0010) [2023-12-26 18:30:05,914][105692] Updated weights for policy 0, policy_version 430834 (0.0008) [2023-12-26 18:30:05,970][105692] Updated weights for policy 0, policy_version 430844 (0.0005) [2023-12-26 18:30:06,030][105692] Updated weights for policy 0, policy_version 430854 (0.0005) [2023-12-26 18:30:06,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 220749824. Throughput: 0: 9909.0, 1: 9938.9. Samples: 220731208. Policy #0 lag: (min: 45.0, avg: 55.6, max: 56.0) [2023-12-26 18:30:06,062][104569] Avg episode reward: [(0, '8700.122'), (1, '9080.557')] [2023-12-26 18:30:06,610][105620] Updated weights for policy 1, policy_version 431344 (0.0011) [2023-12-26 18:30:06,683][105620] Updated weights for policy 1, policy_version 431354 (0.0011) [2023-12-26 18:30:06,710][105692] Updated weights for policy 0, policy_version 430864 (0.0006) [2023-12-26 18:30:06,750][105620] Updated weights for policy 1, policy_version 431364 (0.0011) [2023-12-26 18:30:06,771][105692] Updated weights for policy 0, policy_version 430874 (0.0006) [2023-12-26 18:30:06,833][105692] Updated weights for policy 0, policy_version 430884 (0.0008) [2023-12-26 18:30:07,391][105620] Updated weights for policy 1, policy_version 431374 (0.0010) [2023-12-26 18:30:07,393][105692] Updated weights for policy 0, policy_version 430894 (0.0009) [2023-12-26 18:30:07,444][105620] Updated weights for policy 1, policy_version 431384 (0.0010) [2023-12-26 18:30:07,452][105692] Updated weights for policy 0, policy_version 430904 (0.0010) [2023-12-26 18:30:07,493][105620] Updated weights for policy 1, policy_version 431394 (0.0010) [2023-12-26 18:30:07,511][105692] Updated weights for policy 0, policy_version 430914 (0.0010) [2023-12-26 18:30:08,144][105692] Updated weights for policy 0, policy_version 430924 (0.0009) [2023-12-26 18:30:08,198][105692] Updated weights for policy 0, policy_version 430934 (0.0005) [2023-12-26 18:30:08,226][105620] Updated weights for policy 1, policy_version 431404 (0.0008) [2023-12-26 18:30:08,254][105692] Updated weights for policy 0, policy_version 430944 (0.0006) [2023-12-26 18:30:08,280][105620] Updated weights for policy 1, policy_version 431414 (0.0005) [2023-12-26 18:30:08,342][105620] Updated weights for policy 1, policy_version 431424 (0.0009) [2023-12-26 18:30:08,883][105692] Updated weights for policy 0, policy_version 430954 (0.0005) [2023-12-26 18:30:08,950][105692] Updated weights for policy 0, policy_version 430964 (0.0005) [2023-12-26 18:30:09,011][105692] Updated weights for policy 0, policy_version 430974 (0.0008) [2023-12-26 18:30:09,045][105620] Updated weights for policy 1, policy_version 431434 (0.0010) [2023-12-26 18:30:09,069][105692] Updated weights for policy 0, policy_version 430984 (0.0007) [2023-12-26 18:30:09,100][105620] Updated weights for policy 1, policy_version 431444 (0.0010) [2023-12-26 18:30:09,156][105620] Updated weights for policy 1, policy_version 431454 (0.0008) [2023-12-26 18:30:09,212][105620] Updated weights for policy 1, policy_version 431464 (0.0006) [2023-12-26 18:30:09,727][105692] Updated weights for policy 0, policy_version 430994 (0.0010) [2023-12-26 18:30:09,780][105692] Updated weights for policy 0, policy_version 431004 (0.0009) [2023-12-26 18:30:09,842][105692] Updated weights for policy 0, policy_version 431014 (0.0008) [2023-12-26 18:30:09,978][105620] Updated weights for policy 1, policy_version 431474 (0.0009) [2023-12-26 18:30:10,037][105620] Updated weights for policy 1, policy_version 431484 (0.0009) [2023-12-26 18:30:10,096][105620] Updated weights for policy 1, policy_version 431494 (0.0008) [2023-12-26 18:30:10,612][105692] Updated weights for policy 0, policy_version 431024 (0.0008) [2023-12-26 18:30:10,674][105692] Updated weights for policy 0, policy_version 431034 (0.0007) [2023-12-26 18:30:10,730][105620] Updated weights for policy 1, policy_version 431504 (0.0008) [2023-12-26 18:30:10,735][105692] Updated weights for policy 0, policy_version 431044 (0.0007) [2023-12-26 18:30:10,793][105620] Updated weights for policy 1, policy_version 431514 (0.0006) [2023-12-26 18:30:10,855][105620] Updated weights for policy 1, policy_version 431524 (0.0006) [2023-12-26 18:30:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 220848128. Throughput: 0: 10105.8, 1: 9919.7. Samples: 220854936. Policy #0 lag: (min: 45.0, avg: 55.6, max: 56.0) [2023-12-26 18:30:11,063][104569] Avg episode reward: [(0, '8903.356'), (1, '9172.245')] [2023-12-26 18:30:11,507][105692] Updated weights for policy 0, policy_version 431054 (0.0009) [2023-12-26 18:30:11,566][105620] Updated weights for policy 1, policy_version 431534 (0.0006) [2023-12-26 18:30:11,568][105692] Updated weights for policy 0, policy_version 431064 (0.0008) [2023-12-26 18:30:11,634][105620] Updated weights for policy 1, policy_version 431544 (0.0008) [2023-12-26 18:30:11,634][105692] Updated weights for policy 0, policy_version 431074 (0.0009) [2023-12-26 18:30:11,706][105620] Updated weights for policy 1, policy_version 431554 (0.0008) [2023-12-26 18:30:12,415][105692] Updated weights for policy 0, policy_version 431084 (0.0010) [2023-12-26 18:30:12,465][105620] Updated weights for policy 1, policy_version 431564 (0.0008) [2023-12-26 18:30:12,467][105692] Updated weights for policy 0, policy_version 431094 (0.0010) [2023-12-26 18:30:12,514][105620] Updated weights for policy 1, policy_version 431574 (0.0005) [2023-12-26 18:30:12,516][105692] Updated weights for policy 0, policy_version 431104 (0.0010) [2023-12-26 18:30:12,570][105620] Updated weights for policy 1, policy_version 431584 (0.0006) [2023-12-26 18:30:13,229][105692] Updated weights for policy 0, policy_version 431114 (0.0010) [2023-12-26 18:30:13,255][105620] Updated weights for policy 1, policy_version 431594 (0.0008) [2023-12-26 18:30:13,281][105692] Updated weights for policy 0, policy_version 431124 (0.0008) [2023-12-26 18:30:13,311][105620] Updated weights for policy 1, policy_version 431604 (0.0008) [2023-12-26 18:30:13,330][105692] Updated weights for policy 0, policy_version 431134 (0.0007) [2023-12-26 18:30:13,377][105620] Updated weights for policy 1, policy_version 431614 (0.0007) [2023-12-26 18:30:13,388][105692] Updated weights for policy 0, policy_version 431144 (0.0008) [2023-12-26 18:30:13,432][105620] Updated weights for policy 1, policy_version 431624 (0.0008) [2023-12-26 18:30:13,982][105692] Updated weights for policy 0, policy_version 431154 (0.0009) [2023-12-26 18:30:14,040][105692] Updated weights for policy 0, policy_version 431164 (0.0009) [2023-12-26 18:30:14,102][105692] Updated weights for policy 0, policy_version 431174 (0.0006) [2023-12-26 18:30:14,171][105620] Updated weights for policy 1, policy_version 431634 (0.0009) [2023-12-26 18:30:14,226][105620] Updated weights for policy 1, policy_version 431644 (0.0009) [2023-12-26 18:30:14,281][105620] Updated weights for policy 1, policy_version 431655 (0.0011) [2023-12-26 18:30:14,805][105692] Updated weights for policy 0, policy_version 431184 (0.0008) [2023-12-26 18:30:14,867][105692] Updated weights for policy 0, policy_version 431194 (0.0009) [2023-12-26 18:30:14,927][105692] Updated weights for policy 0, policy_version 431204 (0.0009) [2023-12-26 18:30:14,986][105620] Updated weights for policy 1, policy_version 431665 (0.0009) [2023-12-26 18:30:15,045][105620] Updated weights for policy 1, policy_version 431675 (0.0008) [2023-12-26 18:30:15,113][105620] Updated weights for policy 1, policy_version 431685 (0.0008) [2023-12-26 18:30:15,734][105692] Updated weights for policy 0, policy_version 431214 (0.0008) [2023-12-26 18:30:15,788][105692] Updated weights for policy 0, policy_version 431224 (0.0007) [2023-12-26 18:30:15,793][105620] Updated weights for policy 1, policy_version 431695 (0.0009) [2023-12-26 18:30:15,849][105692] Updated weights for policy 0, policy_version 431234 (0.0009) [2023-12-26 18:30:15,859][105620] Updated weights for policy 1, policy_version 431705 (0.0010) [2023-12-26 18:30:15,907][105620] Updated weights for policy 1, policy_version 431715 (0.0010) [2023-12-26 18:30:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 20070.6, 300 sec: 19605.3). Total num frames: 220946432. Throughput: 0: 9977.3, 1: 9866.4. Samples: 220910988. Policy #0 lag: (min: 45.0, avg: 55.6, max: 56.0) [2023-12-26 18:30:16,062][104569] Avg episode reward: [(0, '969.652'), (1, '9173.410')] [2023-12-26 18:30:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000431240_110411776.pth... [2023-12-26 18:30:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000431720_110534656.pth... [2023-12-26 18:30:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000430088_110116864.pth [2023-12-26 18:30:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000430568_110239744.pth [2023-12-26 18:30:16,529][105620] Updated weights for policy 1, policy_version 431725 (0.0008) [2023-12-26 18:30:16,591][105620] Updated weights for policy 1, policy_version 431735 (0.0005) [2023-12-26 18:30:16,652][105620] Updated weights for policy 1, policy_version 431745 (0.0005) [2023-12-26 18:30:16,675][105692] Updated weights for policy 0, policy_version 431244 (0.0008) [2023-12-26 18:30:16,741][105692] Updated weights for policy 0, policy_version 431254 (0.0010) [2023-12-26 18:30:16,803][105692] Updated weights for policy 0, policy_version 431264 (0.0010) [2023-12-26 18:30:17,293][105620] Updated weights for policy 1, policy_version 431755 (0.0007) [2023-12-26 18:30:17,351][105620] Updated weights for policy 1, policy_version 431765 (0.0010) [2023-12-26 18:30:17,411][105620] Updated weights for policy 1, policy_version 431775 (0.0006) [2023-12-26 18:30:17,544][105692] Updated weights for policy 0, policy_version 431274 (0.0009) [2023-12-26 18:30:17,598][105692] Updated weights for policy 0, policy_version 431284 (0.0005) [2023-12-26 18:30:17,645][105692] Updated weights for policy 0, policy_version 431294 (0.0006) [2023-12-26 18:30:17,712][105692] Updated weights for policy 0, policy_version 431304 (0.0008) [2023-12-26 18:30:18,142][105620] Updated weights for policy 1, policy_version 431785 (0.0010) [2023-12-26 18:30:18,197][105620] Updated weights for policy 1, policy_version 431795 (0.0009) [2023-12-26 18:30:18,245][105620] Updated weights for policy 1, policy_version 431805 (0.0008) [2023-12-26 18:30:18,296][105620] Updated weights for policy 1, policy_version 431815 (0.0009) [2023-12-26 18:30:18,396][105692] Updated weights for policy 0, policy_version 431314 (0.0006) [2023-12-26 18:30:18,465][105692] Updated weights for policy 0, policy_version 431324 (0.0006) [2023-12-26 18:30:18,519][105692] Updated weights for policy 0, policy_version 431334 (0.0005) [2023-12-26 18:30:19,022][105620] Updated weights for policy 1, policy_version 431825 (0.0010) [2023-12-26 18:30:19,078][105620] Updated weights for policy 1, policy_version 431835 (0.0010) [2023-12-26 18:30:19,146][105620] Updated weights for policy 1, policy_version 431845 (0.0010) [2023-12-26 18:30:19,210][105692] Updated weights for policy 0, policy_version 431344 (0.0007) [2023-12-26 18:30:19,275][105692] Updated weights for policy 0, policy_version 431354 (0.0008) [2023-12-26 18:30:19,337][105692] Updated weights for policy 0, policy_version 431364 (0.0008) [2023-12-26 18:30:19,852][105620] Updated weights for policy 1, policy_version 431855 (0.0011) [2023-12-26 18:30:19,908][105620] Updated weights for policy 1, policy_version 431865 (0.0010) [2023-12-26 18:30:19,965][105620] Updated weights for policy 1, policy_version 431875 (0.0008) [2023-12-26 18:30:20,107][105692] Updated weights for policy 0, policy_version 431374 (0.0010) [2023-12-26 18:30:20,166][105692] Updated weights for policy 0, policy_version 431384 (0.0009) [2023-12-26 18:30:20,231][105692] Updated weights for policy 0, policy_version 431394 (0.0009) [2023-12-26 18:30:20,638][105620] Updated weights for policy 1, policy_version 431885 (0.0007) [2023-12-26 18:30:20,702][105620] Updated weights for policy 1, policy_version 431895 (0.0009) [2023-12-26 18:30:20,772][105620] Updated weights for policy 1, policy_version 431905 (0.0009) [2023-12-26 18:30:21,001][105692] Updated weights for policy 0, policy_version 431404 (0.0008) [2023-12-26 18:30:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 221036544. Throughput: 0: 9893.8, 1: 9900.8. Samples: 221027940. Policy #0 lag: (min: 45.0, avg: 55.6, max: 56.0) [2023-12-26 18:30:21,063][104569] Avg episode reward: [(0, '538.292'), (1, '9082.172')] [2023-12-26 18:30:21,065][105692] Updated weights for policy 0, policy_version 431414 (0.0009) [2023-12-26 18:30:21,129][105692] Updated weights for policy 0, policy_version 431424 (0.0008) [2023-12-26 18:30:21,548][105620] Updated weights for policy 1, policy_version 431915 (0.0006) [2023-12-26 18:30:21,607][105620] Updated weights for policy 1, policy_version 431925 (0.0006) [2023-12-26 18:30:21,668][105620] Updated weights for policy 1, policy_version 431935 (0.0008) [2023-12-26 18:30:21,894][105692] Updated weights for policy 0, policy_version 431434 (0.0007) [2023-12-26 18:30:21,957][105692] Updated weights for policy 0, policy_version 431444 (0.0008) [2023-12-26 18:30:22,026][105692] Updated weights for policy 0, policy_version 431454 (0.0008) [2023-12-26 18:30:22,096][105692] Updated weights for policy 0, policy_version 431464 (0.0008) [2023-12-26 18:30:22,337][105620] Updated weights for policy 1, policy_version 431945 (0.0008) [2023-12-26 18:30:22,401][105620] Updated weights for policy 1, policy_version 431955 (0.0009) [2023-12-26 18:30:22,454][105620] Updated weights for policy 1, policy_version 431965 (0.0009) [2023-12-26 18:30:22,510][105620] Updated weights for policy 1, policy_version 431975 (0.0009) [2023-12-26 18:30:22,817][105692] Updated weights for policy 0, policy_version 431474 (0.0005) [2023-12-26 18:30:22,880][105692] Updated weights for policy 0, policy_version 431484 (0.0006) [2023-12-26 18:30:22,930][105692] Updated weights for policy 0, policy_version 431494 (0.0005) [2023-12-26 18:30:23,342][105620] Updated weights for policy 1, policy_version 431985 (0.0010) [2023-12-26 18:30:23,396][105620] Updated weights for policy 1, policy_version 431995 (0.0009) [2023-12-26 18:30:23,447][105620] Updated weights for policy 1, policy_version 432005 (0.0008) [2023-12-26 18:30:23,518][105692] Updated weights for policy 0, policy_version 431504 (0.0009) [2023-12-26 18:30:23,566][105692] Updated weights for policy 0, policy_version 431514 (0.0009) [2023-12-26 18:30:23,627][105692] Updated weights for policy 0, policy_version 431524 (0.0008) [2023-12-26 18:30:24,202][105620] Updated weights for policy 1, policy_version 432015 (0.0010) [2023-12-26 18:30:24,260][105620] Updated weights for policy 1, policy_version 432025 (0.0010) [2023-12-26 18:30:24,312][105692] Updated weights for policy 0, policy_version 431534 (0.0007) [2023-12-26 18:30:24,314][105620] Updated weights for policy 1, policy_version 432035 (0.0007) [2023-12-26 18:30:24,362][105692] Updated weights for policy 0, policy_version 431544 (0.0007) [2023-12-26 18:30:24,420][105692] Updated weights for policy 0, policy_version 431554 (0.0007) [2023-12-26 18:30:25,038][105692] Updated weights for policy 0, policy_version 431564 (0.0009) [2023-12-26 18:30:25,097][105692] Updated weights for policy 0, policy_version 431574 (0.0011) [2023-12-26 18:30:25,159][105692] Updated weights for policy 0, policy_version 431584 (0.0010) [2023-12-26 18:30:25,166][105620] Updated weights for policy 1, policy_version 432045 (0.0007) [2023-12-26 18:30:25,214][105620] Updated weights for policy 1, policy_version 432055 (0.0008) [2023-12-26 18:30:25,277][105620] Updated weights for policy 1, policy_version 432065 (0.0008) [2023-12-26 18:30:25,910][105692] Updated weights for policy 0, policy_version 431594 (0.0010) [2023-12-26 18:30:25,961][105692] Updated weights for policy 0, policy_version 431604 (0.0010) [2023-12-26 18:30:26,013][105620] Updated weights for policy 1, policy_version 432075 (0.0007) [2023-12-26 18:30:26,019][105692] Updated weights for policy 0, policy_version 431614 (0.0010) [2023-12-26 18:30:26,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 221126656. Throughput: 0: 9820.7, 1: 9826.0. Samples: 221142788. Policy #0 lag: (min: 45.0, avg: 55.6, max: 56.0) [2023-12-26 18:30:26,062][104569] Avg episode reward: [(0, '981.461'), (1, '9082.225')] [2023-12-26 18:30:26,069][105620] Updated weights for policy 1, policy_version 432085 (0.0007) [2023-12-26 18:30:26,076][105692] Updated weights for policy 0, policy_version 431624 (0.0007) [2023-12-26 18:30:26,130][105620] Updated weights for policy 1, policy_version 432095 (0.0005) [2023-12-26 18:30:26,724][105620] Updated weights for policy 1, policy_version 432105 (0.0005) [2023-12-26 18:30:26,786][105620] Updated weights for policy 1, policy_version 432115 (0.0005) [2023-12-26 18:30:26,797][105692] Updated weights for policy 0, policy_version 431634 (0.0011) [2023-12-26 18:30:26,846][105620] Updated weights for policy 1, policy_version 432125 (0.0006) [2023-12-26 18:30:26,855][105692] Updated weights for policy 0, policy_version 431644 (0.0008) [2023-12-26 18:30:26,904][105620] Updated weights for policy 1, policy_version 432135 (0.0006) [2023-12-26 18:30:26,910][105692] Updated weights for policy 0, policy_version 431654 (0.0005) [2023-12-26 18:30:27,472][105692] Updated weights for policy 0, policy_version 431664 (0.0010) [2023-12-26 18:30:27,528][105620] Updated weights for policy 1, policy_version 432146 (0.0008) [2023-12-26 18:30:27,534][105692] Updated weights for policy 0, policy_version 431674 (0.0006) [2023-12-26 18:30:27,576][105620] Updated weights for policy 1, policy_version 432156 (0.0007) [2023-12-26 18:30:27,596][105692] Updated weights for policy 0, policy_version 431684 (0.0006) [2023-12-26 18:30:27,620][105620] Updated weights for policy 1, policy_version 432166 (0.0007) [2023-12-26 18:30:28,130][105692] Updated weights for policy 0, policy_version 431694 (0.0005) [2023-12-26 18:30:28,174][105692] Updated weights for policy 0, policy_version 431704 (0.0005) [2023-12-26 18:30:28,232][105692] Updated weights for policy 0, policy_version 431714 (0.0005) [2023-12-26 18:30:28,536][105620] Updated weights for policy 1, policy_version 432176 (0.0009) [2023-12-26 18:30:28,609][105620] Updated weights for policy 1, policy_version 432186 (0.0009) [2023-12-26 18:30:28,674][105620] Updated weights for policy 1, policy_version 432196 (0.0009) [2023-12-26 18:30:28,773][105692] Updated weights for policy 0, policy_version 431724 (0.0006) [2023-12-26 18:30:28,838][105692] Updated weights for policy 0, policy_version 431734 (0.0006) [2023-12-26 18:30:28,901][105692] Updated weights for policy 0, policy_version 431744 (0.0006) [2023-12-26 18:30:29,374][105620] Updated weights for policy 1, policy_version 432206 (0.0008) [2023-12-26 18:30:29,431][105620] Updated weights for policy 1, policy_version 432216 (0.0005) [2023-12-26 18:30:29,495][105620] Updated weights for policy 1, policy_version 432226 (0.0005) [2023-12-26 18:30:29,509][105692] Updated weights for policy 0, policy_version 431754 (0.0007) [2023-12-26 18:30:29,563][105692] Updated weights for policy 0, policy_version 431764 (0.0010) [2023-12-26 18:30:29,632][105692] Updated weights for policy 0, policy_version 431774 (0.0010) [2023-12-26 18:30:29,701][105692] Updated weights for policy 0, policy_version 431784 (0.0010) [2023-12-26 18:30:30,171][105620] Updated weights for policy 1, policy_version 432236 (0.0007) [2023-12-26 18:30:30,230][105620] Updated weights for policy 1, policy_version 432246 (0.0008) [2023-12-26 18:30:30,289][105620] Updated weights for policy 1, policy_version 432256 (0.0008) [2023-12-26 18:30:30,443][105692] Updated weights for policy 0, policy_version 431794 (0.0010) [2023-12-26 18:30:30,509][105692] Updated weights for policy 0, policy_version 431804 (0.0005) [2023-12-26 18:30:30,569][105692] Updated weights for policy 0, policy_version 431814 (0.0005) [2023-12-26 18:30:30,974][105620] Updated weights for policy 1, policy_version 432266 (0.0007) [2023-12-26 18:30:31,039][105620] Updated weights for policy 1, policy_version 432276 (0.0006) [2023-12-26 18:30:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 221233152. Throughput: 0: 9928.5, 1: 9818.3. Samples: 221205144. Policy #0 lag: (min: 45.0, avg: 55.6, max: 56.0) [2023-12-26 18:30:31,062][104569] Avg episode reward: [(0, '1281.969'), (1, '9263.865')] [2023-12-26 18:30:31,081][105692] Updated weights for policy 0, policy_version 431824 (0.0006) [2023-12-26 18:30:31,094][105620] Updated weights for policy 1, policy_version 432286 (0.0007) [2023-12-26 18:30:31,142][105692] Updated weights for policy 0, policy_version 431834 (0.0010) [2023-12-26 18:30:31,156][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000432296_110682112.pth... [2023-12-26 18:30:31,157][105620] Updated weights for policy 1, policy_version 432296 (0.0007) [2023-12-26 18:30:31,160][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000431144_110387200.pth [2023-12-26 18:30:31,202][105692] Updated weights for policy 0, policy_version 431844 (0.0011) [2023-12-26 18:30:31,226][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000431848_110567424.pth... [2023-12-26 18:30:31,231][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000430632_110256128.pth [2023-12-26 18:30:31,854][105620] Updated weights for policy 1, policy_version 432306 (0.0005) [2023-12-26 18:30:31,886][105692] Updated weights for policy 0, policy_version 431854 (0.0008) [2023-12-26 18:30:31,915][105620] Updated weights for policy 1, policy_version 432316 (0.0007) [2023-12-26 18:30:31,950][105692] Updated weights for policy 0, policy_version 431864 (0.0010) [2023-12-26 18:30:31,972][105620] Updated weights for policy 1, policy_version 432326 (0.0006) [2023-12-26 18:30:32,009][105692] Updated weights for policy 0, policy_version 431874 (0.0011) [2023-12-26 18:30:32,715][105692] Updated weights for policy 0, policy_version 431884 (0.0009) [2023-12-26 18:30:32,717][105620] Updated weights for policy 1, policy_version 432336 (0.0007) [2023-12-26 18:30:32,763][105692] Updated weights for policy 0, policy_version 431894 (0.0008) [2023-12-26 18:30:32,782][105620] Updated weights for policy 1, policy_version 432346 (0.0008) [2023-12-26 18:30:32,809][105692] Updated weights for policy 0, policy_version 431904 (0.0006) [2023-12-26 18:30:32,844][105620] Updated weights for policy 1, policy_version 432356 (0.0009) [2023-12-26 18:30:33,438][105692] Updated weights for policy 0, policy_version 431914 (0.0006) [2023-12-26 18:30:33,493][105692] Updated weights for policy 0, policy_version 431924 (0.0009) [2023-12-26 18:30:33,552][105692] Updated weights for policy 0, policy_version 431934 (0.0009) [2023-12-26 18:30:33,607][105692] Updated weights for policy 0, policy_version 431944 (0.0008) [2023-12-26 18:30:33,617][105620] Updated weights for policy 1, policy_version 432366 (0.0007) [2023-12-26 18:30:33,664][105620] Updated weights for policy 1, policy_version 432376 (0.0008) [2023-12-26 18:30:33,709][105620] Updated weights for policy 1, policy_version 432386 (0.0008) [2023-12-26 18:30:34,344][105692] Updated weights for policy 0, policy_version 431954 (0.0009) [2023-12-26 18:30:34,408][105692] Updated weights for policy 0, policy_version 431964 (0.0006) [2023-12-26 18:30:34,471][105692] Updated weights for policy 0, policy_version 431974 (0.0009) [2023-12-26 18:30:34,507][105620] Updated weights for policy 1, policy_version 432396 (0.0008) [2023-12-26 18:30:34,565][105620] Updated weights for policy 1, policy_version 432406 (0.0009) [2023-12-26 18:30:34,627][105620] Updated weights for policy 1, policy_version 432416 (0.0009) [2023-12-26 18:30:35,079][105692] Updated weights for policy 0, policy_version 431984 (0.0006) [2023-12-26 18:30:35,147][105692] Updated weights for policy 0, policy_version 431994 (0.0006) [2023-12-26 18:30:35,210][105692] Updated weights for policy 0, policy_version 432004 (0.0007) [2023-12-26 18:30:35,495][105620] Updated weights for policy 1, policy_version 432426 (0.0009) [2023-12-26 18:30:35,557][105620] Updated weights for policy 1, policy_version 432436 (0.0009) [2023-12-26 18:30:35,612][105620] Updated weights for policy 1, policy_version 432446 (0.0009) [2023-12-26 18:30:35,668][105620] Updated weights for policy 1, policy_version 432456 (0.0009) [2023-12-26 18:30:35,893][105692] Updated weights for policy 0, policy_version 432014 (0.0005) [2023-12-26 18:30:35,956][105692] Updated weights for policy 0, policy_version 432024 (0.0007) [2023-12-26 18:30:36,022][105692] Updated weights for policy 0, policy_version 432034 (0.0007) [2023-12-26 18:30:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 221331456. Throughput: 0: 9981.0, 1: 9792.6. Samples: 221324632. Policy #0 lag: (min: 45.0, avg: 55.6, max: 56.0) [2023-12-26 18:30:36,062][104569] Avg episode reward: [(0, '6558.169'), (1, '9173.617')] [2023-12-26 18:30:36,432][105620] Updated weights for policy 1, policy_version 432466 (0.0009) [2023-12-26 18:30:36,488][105620] Updated weights for policy 1, policy_version 432476 (0.0009) [2023-12-26 18:30:36,551][105620] Updated weights for policy 1, policy_version 432486 (0.0008) [2023-12-26 18:30:36,646][105692] Updated weights for policy 0, policy_version 432044 (0.0006) [2023-12-26 18:30:36,702][105692] Updated weights for policy 0, policy_version 432054 (0.0008) [2023-12-26 18:30:36,758][105692] Updated weights for policy 0, policy_version 432064 (0.0007) [2023-12-26 18:30:37,348][105620] Updated weights for policy 1, policy_version 432496 (0.0008) [2023-12-26 18:30:37,408][105620] Updated weights for policy 1, policy_version 432506 (0.0009) [2023-12-26 18:30:37,467][105692] Updated weights for policy 0, policy_version 432074 (0.0009) [2023-12-26 18:30:37,477][105620] Updated weights for policy 1, policy_version 432516 (0.0009) [2023-12-26 18:30:37,529][105692] Updated weights for policy 0, policy_version 432084 (0.0007) [2023-12-26 18:30:37,584][105692] Updated weights for policy 0, policy_version 432094 (0.0009) [2023-12-26 18:30:37,635][105692] Updated weights for policy 0, policy_version 432104 (0.0009) [2023-12-26 18:30:38,246][105620] Updated weights for policy 1, policy_version 432526 (0.0006) [2023-12-26 18:30:38,299][105620] Updated weights for policy 1, policy_version 432536 (0.0009) [2023-12-26 18:30:38,354][105692] Updated weights for policy 0, policy_version 432114 (0.0007) [2023-12-26 18:30:38,367][105620] Updated weights for policy 1, policy_version 432546 (0.0008) [2023-12-26 18:30:38,413][105692] Updated weights for policy 0, policy_version 432124 (0.0008) [2023-12-26 18:30:38,476][105692] Updated weights for policy 0, policy_version 432134 (0.0009) [2023-12-26 18:30:39,054][105620] Updated weights for policy 1, policy_version 432556 (0.0007) [2023-12-26 18:30:39,116][105620] Updated weights for policy 1, policy_version 432566 (0.0009) [2023-12-26 18:30:39,175][105620] Updated weights for policy 1, policy_version 432576 (0.0009) [2023-12-26 18:30:39,224][105692] Updated weights for policy 0, policy_version 432144 (0.0007) [2023-12-26 18:30:39,293][105692] Updated weights for policy 0, policy_version 432154 (0.0007) [2023-12-26 18:30:39,365][105692] Updated weights for policy 0, policy_version 432164 (0.0008) [2023-12-26 18:30:40,001][105692] Updated weights for policy 0, policy_version 432174 (0.0007) [2023-12-26 18:30:40,008][105620] Updated weights for policy 1, policy_version 432586 (0.0009) [2023-12-26 18:30:40,068][105692] Updated weights for policy 0, policy_version 432184 (0.0006) [2023-12-26 18:30:40,068][105620] Updated weights for policy 1, policy_version 432596 (0.0009) [2023-12-26 18:30:40,124][105692] Updated weights for policy 0, policy_version 432194 (0.0006) [2023-12-26 18:30:40,134][105620] Updated weights for policy 1, policy_version 432606 (0.0009) [2023-12-26 18:30:40,194][105620] Updated weights for policy 1, policy_version 432616 (0.0009) [2023-12-26 18:30:40,851][105620] Updated weights for policy 1, policy_version 432626 (0.0007) [2023-12-26 18:30:40,874][105692] Updated weights for policy 0, policy_version 432204 (0.0008) [2023-12-26 18:30:40,903][105620] Updated weights for policy 1, policy_version 432636 (0.0007) [2023-12-26 18:30:40,931][105692] Updated weights for policy 0, policy_version 432214 (0.0006) [2023-12-26 18:30:40,960][105620] Updated weights for policy 1, policy_version 432646 (0.0008) [2023-12-26 18:30:40,991][105692] Updated weights for policy 0, policy_version 432224 (0.0006) [2023-12-26 18:30:41,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19933.8, 300 sec: 19605.3). Total num frames: 221437952. Throughput: 0: 10046.9, 1: 9635.2. Samples: 221440560. Policy #0 lag: (min: 45.0, avg: 55.6, max: 56.0) [2023-12-26 18:30:41,062][104569] Avg episode reward: [(0, '7479.194'), (1, '9173.870')] [2023-12-26 18:30:41,766][105620] Updated weights for policy 1, policy_version 432656 (0.0009) [2023-12-26 18:30:41,771][105692] Updated weights for policy 0, policy_version 432234 (0.0007) [2023-12-26 18:30:41,824][105620] Updated weights for policy 1, policy_version 432666 (0.0007) [2023-12-26 18:30:41,827][105692] Updated weights for policy 0, policy_version 432244 (0.0008) [2023-12-26 18:30:41,882][105692] Updated weights for policy 0, policy_version 432254 (0.0008) [2023-12-26 18:30:41,885][105620] Updated weights for policy 1, policy_version 432676 (0.0008) [2023-12-26 18:30:41,934][105692] Updated weights for policy 0, policy_version 432264 (0.0008) [2023-12-26 18:30:42,653][105620] Updated weights for policy 1, policy_version 432686 (0.0008) [2023-12-26 18:30:42,715][105620] Updated weights for policy 1, policy_version 432696 (0.0008) [2023-12-26 18:30:42,721][105692] Updated weights for policy 0, policy_version 432274 (0.0009) [2023-12-26 18:30:42,770][105620] Updated weights for policy 1, policy_version 432706 (0.0006) [2023-12-26 18:30:42,780][105692] Updated weights for policy 0, policy_version 432284 (0.0009) [2023-12-26 18:30:42,833][105692] Updated weights for policy 0, policy_version 432294 (0.0008) [2023-12-26 18:30:43,479][105620] Updated weights for policy 1, policy_version 432716 (0.0006) [2023-12-26 18:30:43,540][105620] Updated weights for policy 1, policy_version 432726 (0.0007) [2023-12-26 18:30:43,610][105620] Updated weights for policy 1, policy_version 432736 (0.0008) [2023-12-26 18:30:43,658][105692] Updated weights for policy 0, policy_version 432304 (0.0008) [2023-12-26 18:30:43,726][105692] Updated weights for policy 0, policy_version 432314 (0.0009) [2023-12-26 18:30:43,785][105692] Updated weights for policy 0, policy_version 432324 (0.0009) [2023-12-26 18:30:44,249][105620] Updated weights for policy 1, policy_version 432746 (0.0008) [2023-12-26 18:30:44,299][105620] Updated weights for policy 1, policy_version 432756 (0.0009) [2023-12-26 18:30:44,351][105620] Updated weights for policy 1, policy_version 432766 (0.0010) [2023-12-26 18:30:44,406][105620] Updated weights for policy 1, policy_version 432776 (0.0008) [2023-12-26 18:30:44,482][105692] Updated weights for policy 0, policy_version 432334 (0.0009) [2023-12-26 18:30:44,532][105692] Updated weights for policy 0, policy_version 432344 (0.0009) [2023-12-26 18:30:44,584][105692] Updated weights for policy 0, policy_version 432354 (0.0009) [2023-12-26 18:30:45,125][105620] Updated weights for policy 1, policy_version 432786 (0.0010) [2023-12-26 18:30:45,188][105620] Updated weights for policy 1, policy_version 432796 (0.0009) [2023-12-26 18:30:45,251][105620] Updated weights for policy 1, policy_version 432806 (0.0008) [2023-12-26 18:30:45,285][105692] Updated weights for policy 0, policy_version 432364 (0.0008) [2023-12-26 18:30:45,348][105692] Updated weights for policy 0, policy_version 432374 (0.0009) [2023-12-26 18:30:45,410][105692] Updated weights for policy 0, policy_version 432384 (0.0009) [2023-12-26 18:30:46,023][105692] Updated weights for policy 0, policy_version 432394 (0.0009) [2023-12-26 18:30:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 221519872. Throughput: 0: 10021.2, 1: 9615.8. Samples: 221495456. Policy #0 lag: (min: 45.0, avg: 55.6, max: 56.0) [2023-12-26 18:30:46,062][104569] Avg episode reward: [(0, '7217.083'), (1, '9267.013')] [2023-12-26 18:30:46,069][105692] Updated weights for policy 0, policy_version 432404 (0.0009) [2023-12-26 18:30:46,072][105620] Updated weights for policy 1, policy_version 432816 (0.0009) [2023-12-26 18:30:46,127][105692] Updated weights for policy 0, policy_version 432414 (0.0006) [2023-12-26 18:30:46,132][105620] Updated weights for policy 1, policy_version 432826 (0.0009) [2023-12-26 18:30:46,180][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000432424_110714880.pth... [2023-12-26 18:30:46,182][105692] Updated weights for policy 0, policy_version 432424 (0.0006) [2023-12-26 18:30:46,183][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000431240_110411776.pth [2023-12-26 18:30:46,184][105620] Updated weights for policy 1, policy_version 432836 (0.0009) [2023-12-26 18:30:46,208][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000432840_110821376.pth... [2023-12-26 18:30:46,212][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000431720_110534656.pth [2023-12-26 18:30:46,873][105692] Updated weights for policy 0, policy_version 432434 (0.0009) [2023-12-26 18:30:46,936][105692] Updated weights for policy 0, policy_version 432444 (0.0009) [2023-12-26 18:30:46,954][105620] Updated weights for policy 1, policy_version 432846 (0.0008) [2023-12-26 18:30:46,996][105692] Updated weights for policy 0, policy_version 432454 (0.0009) [2023-12-26 18:30:47,023][105620] Updated weights for policy 1, policy_version 432856 (0.0007) [2023-12-26 18:30:47,082][105620] Updated weights for policy 1, policy_version 432866 (0.0008) [2023-12-26 18:30:47,740][105620] Updated weights for policy 1, policy_version 432876 (0.0010) [2023-12-26 18:30:47,761][105692] Updated weights for policy 0, policy_version 432464 (0.0009) [2023-12-26 18:30:47,792][105620] Updated weights for policy 1, policy_version 432886 (0.0010) [2023-12-26 18:30:47,816][105692] Updated weights for policy 0, policy_version 432474 (0.0009) [2023-12-26 18:30:47,840][105620] Updated weights for policy 1, policy_version 432896 (0.0010) [2023-12-26 18:30:47,875][105692] Updated weights for policy 0, policy_version 432484 (0.0010) [2023-12-26 18:30:48,557][105692] Updated weights for policy 0, policy_version 432494 (0.0007) [2023-12-26 18:30:48,592][105620] Updated weights for policy 1, policy_version 432906 (0.0008) [2023-12-26 18:30:48,614][105692] Updated weights for policy 0, policy_version 432504 (0.0005) [2023-12-26 18:30:48,648][105620] Updated weights for policy 1, policy_version 432916 (0.0008) [2023-12-26 18:30:48,672][105692] Updated weights for policy 0, policy_version 432514 (0.0007) [2023-12-26 18:30:48,708][105620] Updated weights for policy 1, policy_version 432927 (0.0009) [2023-12-26 18:30:49,308][105692] Updated weights for policy 0, policy_version 432524 (0.0007) [2023-12-26 18:30:49,373][105692] Updated weights for policy 0, policy_version 432534 (0.0009) [2023-12-26 18:30:49,429][105692] Updated weights for policy 0, policy_version 432544 (0.0009) [2023-12-26 18:30:49,522][105620] Updated weights for policy 1, policy_version 432937 (0.0009) [2023-12-26 18:30:49,576][105620] Updated weights for policy 1, policy_version 432947 (0.0010) [2023-12-26 18:30:49,636][105620] Updated weights for policy 1, policy_version 432957 (0.0009) [2023-12-26 18:30:49,684][105620] Updated weights for policy 1, policy_version 432967 (0.0009) [2023-12-26 18:30:50,094][105692] Updated weights for policy 0, policy_version 432554 (0.0010) [2023-12-26 18:30:50,155][105692] Updated weights for policy 0, policy_version 432564 (0.0010) [2023-12-26 18:30:50,219][105692] Updated weights for policy 0, policy_version 432574 (0.0009) [2023-12-26 18:30:50,280][105692] Updated weights for policy 0, policy_version 432584 (0.0009) [2023-12-26 18:30:50,444][105620] Updated weights for policy 1, policy_version 432977 (0.0010) [2023-12-26 18:30:50,503][105620] Updated weights for policy 1, policy_version 432987 (0.0010) [2023-12-26 18:30:50,562][105620] Updated weights for policy 1, policy_version 432997 (0.0010) [2023-12-26 18:30:51,058][105692] Updated weights for policy 0, policy_version 432594 (0.0007) [2023-12-26 18:30:51,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 221618176. Throughput: 0: 10033.0, 1: 9514.1. Samples: 221610832. Policy #0 lag: (min: 45.0, avg: 55.6, max: 56.0) [2023-12-26 18:30:51,063][104569] Avg episode reward: [(0, '6622.804'), (1, '9267.100')] [2023-12-26 18:30:51,113][105692] Updated weights for policy 0, policy_version 432604 (0.0006) [2023-12-26 18:30:51,166][105692] Updated weights for policy 0, policy_version 432614 (0.0009) [2023-12-26 18:30:51,338][105620] Updated weights for policy 1, policy_version 433007 (0.0009) [2023-12-26 18:30:51,403][105620] Updated weights for policy 1, policy_version 433017 (0.0010) [2023-12-26 18:30:51,456][105620] Updated weights for policy 1, policy_version 433027 (0.0009) [2023-12-26 18:30:51,978][105692] Updated weights for policy 0, policy_version 432624 (0.0009) [2023-12-26 18:30:52,042][105692] Updated weights for policy 0, policy_version 432634 (0.0008) [2023-12-26 18:30:52,106][105692] Updated weights for policy 0, policy_version 432644 (0.0007) [2023-12-26 18:30:52,247][105620] Updated weights for policy 1, policy_version 433037 (0.0010) [2023-12-26 18:30:52,305][105620] Updated weights for policy 1, policy_version 433047 (0.0008) [2023-12-26 18:30:52,368][105620] Updated weights for policy 1, policy_version 433057 (0.0009) [2023-12-26 18:30:52,930][105692] Updated weights for policy 0, policy_version 432654 (0.0009) [2023-12-26 18:30:52,954][105620] Updated weights for policy 1, policy_version 433067 (0.0007) [2023-12-26 18:30:52,989][105692] Updated weights for policy 0, policy_version 432664 (0.0011) [2023-12-26 18:30:53,010][105620] Updated weights for policy 1, policy_version 433077 (0.0005) [2023-12-26 18:30:53,045][105692] Updated weights for policy 0, policy_version 432674 (0.0011) [2023-12-26 18:30:53,066][105620] Updated weights for policy 1, policy_version 433087 (0.0007) [2023-12-26 18:30:53,712][105692] Updated weights for policy 0, policy_version 432684 (0.0010) [2023-12-26 18:30:53,737][105620] Updated weights for policy 1, policy_version 433097 (0.0010) [2023-12-26 18:30:53,763][105692] Updated weights for policy 0, policy_version 432694 (0.0010) [2023-12-26 18:30:53,791][105620] Updated weights for policy 1, policy_version 433107 (0.0010) [2023-12-26 18:30:53,825][105692] Updated weights for policy 0, policy_version 432704 (0.0010) [2023-12-26 18:30:53,843][105620] Updated weights for policy 1, policy_version 433117 (0.0010) [2023-12-26 18:30:53,892][105620] Updated weights for policy 1, policy_version 433127 (0.0008) [2023-12-26 18:30:54,498][105692] Updated weights for policy 0, policy_version 432714 (0.0011) [2023-12-26 18:30:54,519][105620] Updated weights for policy 1, policy_version 433137 (0.0005) [2023-12-26 18:30:54,550][105692] Updated weights for policy 0, policy_version 432724 (0.0011) [2023-12-26 18:30:54,581][105620] Updated weights for policy 1, policy_version 433147 (0.0006) [2023-12-26 18:30:54,602][105692] Updated weights for policy 0, policy_version 432734 (0.0011) [2023-12-26 18:30:54,643][105620] Updated weights for policy 1, policy_version 433157 (0.0007) [2023-12-26 18:30:54,650][105692] Updated weights for policy 0, policy_version 432744 (0.0010) [2023-12-26 18:30:55,279][105620] Updated weights for policy 1, policy_version 433167 (0.0009) [2023-12-26 18:30:55,335][105620] Updated weights for policy 1, policy_version 433177 (0.0007) [2023-12-26 18:30:55,393][105620] Updated weights for policy 1, policy_version 433187 (0.0009) [2023-12-26 18:30:55,422][105692] Updated weights for policy 0, policy_version 432754 (0.0011) [2023-12-26 18:30:55,483][105692] Updated weights for policy 0, policy_version 432764 (0.0009) [2023-12-26 18:30:55,540][105692] Updated weights for policy 0, policy_version 432774 (0.0009) [2023-12-26 18:30:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 221716480. Throughput: 0: 9854.1, 1: 9549.4. Samples: 221728096. Policy #0 lag: (min: 45.0, avg: 55.6, max: 56.0) [2023-12-26 18:30:56,063][104569] Avg episode reward: [(0, '7856.082'), (1, '9356.189')] [2023-12-26 18:30:56,146][105620] Updated weights for policy 1, policy_version 433197 (0.0010) [2023-12-26 18:30:56,208][105620] Updated weights for policy 1, policy_version 433207 (0.0010) [2023-12-26 18:30:56,244][105692] Updated weights for policy 0, policy_version 432784 (0.0008) [2023-12-26 18:30:56,266][105620] Updated weights for policy 1, policy_version 433217 (0.0010) [2023-12-26 18:30:56,299][105692] Updated weights for policy 0, policy_version 432794 (0.0005) [2023-12-26 18:30:56,348][105692] Updated weights for policy 0, policy_version 432804 (0.0009) [2023-12-26 18:30:56,927][105692] Updated weights for policy 0, policy_version 432814 (0.0006) [2023-12-26 18:30:56,982][105620] Updated weights for policy 1, policy_version 433227 (0.0010) [2023-12-26 18:30:56,984][105692] Updated weights for policy 0, policy_version 432824 (0.0006) [2023-12-26 18:30:57,032][105692] Updated weights for policy 0, policy_version 432834 (0.0008) [2023-12-26 18:30:57,037][105620] Updated weights for policy 1, policy_version 433237 (0.0010) [2023-12-26 18:30:57,098][105620] Updated weights for policy 1, policy_version 433247 (0.0005) [2023-12-26 18:30:57,665][105620] Updated weights for policy 1, policy_version 433257 (0.0006) [2023-12-26 18:30:57,666][105692] Updated weights for policy 0, policy_version 432844 (0.0008) [2023-12-26 18:30:57,720][105620] Updated weights for policy 1, policy_version 433267 (0.0007) [2023-12-26 18:30:57,726][105692] Updated weights for policy 0, policy_version 432854 (0.0005) [2023-12-26 18:30:57,776][105692] Updated weights for policy 0, policy_version 432864 (0.0005) [2023-12-26 18:30:57,783][105620] Updated weights for policy 1, policy_version 433277 (0.0011) [2023-12-26 18:30:57,839][105620] Updated weights for policy 1, policy_version 433287 (0.0011) [2023-12-26 18:30:58,395][105692] Updated weights for policy 0, policy_version 432874 (0.0007) [2023-12-26 18:30:58,466][105692] Updated weights for policy 0, policy_version 432884 (0.0008) [2023-12-26 18:30:58,527][105692] Updated weights for policy 0, policy_version 432894 (0.0010) [2023-12-26 18:30:58,557][105620] Updated weights for policy 1, policy_version 433297 (0.0009) [2023-12-26 18:30:58,587][105692] Updated weights for policy 0, policy_version 432904 (0.0010) [2023-12-26 18:30:58,623][105620] Updated weights for policy 1, policy_version 433307 (0.0007) [2023-12-26 18:30:58,684][105620] Updated weights for policy 1, policy_version 433317 (0.0008) [2023-12-26 18:30:59,403][105692] Updated weights for policy 0, policy_version 432914 (0.0009) [2023-12-26 18:30:59,452][105692] Updated weights for policy 0, policy_version 432924 (0.0009) [2023-12-26 18:30:59,500][105620] Updated weights for policy 1, policy_version 433327 (0.0007) [2023-12-26 18:30:59,506][105692] Updated weights for policy 0, policy_version 432934 (0.0009) [2023-12-26 18:30:59,557][105620] Updated weights for policy 1, policy_version 433337 (0.0008) [2023-12-26 18:30:59,614][105620] Updated weights for policy 1, policy_version 433347 (0.0009) [2023-12-26 18:31:00,264][105692] Updated weights for policy 0, policy_version 432944 (0.0008) [2023-12-26 18:31:00,325][105692] Updated weights for policy 0, policy_version 432954 (0.0006) [2023-12-26 18:31:00,368][105620] Updated weights for policy 1, policy_version 433357 (0.0009) [2023-12-26 18:31:00,382][105692] Updated weights for policy 0, policy_version 432964 (0.0006) [2023-12-26 18:31:00,435][105620] Updated weights for policy 1, policy_version 433367 (0.0009) [2023-12-26 18:31:00,505][105620] Updated weights for policy 1, policy_version 433377 (0.0008) [2023-12-26 18:31:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 221814784. Throughput: 0: 9965.5, 1: 9588.2. Samples: 221790904. Policy #0 lag: (min: 45.0, avg: 55.6, max: 56.0) [2023-12-26 18:31:01,063][104569] Avg episode reward: [(0, '9001.810'), (1, '9266.132')] [2023-12-26 18:31:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000433384_110960640.pth... [2023-12-26 18:31:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000432296_110682112.pth [2023-12-26 18:31:01,098][105692] Updated weights for policy 0, policy_version 432974 (0.0007) [2023-12-26 18:31:01,166][105692] Updated weights for policy 0, policy_version 432984 (0.0008) [2023-12-26 18:31:01,191][105620] Updated weights for policy 1, policy_version 433387 (0.0010) [2023-12-26 18:31:01,224][105692] Updated weights for policy 0, policy_version 432994 (0.0008) [2023-12-26 18:31:01,256][105620] Updated weights for policy 1, policy_version 433397 (0.0010) [2023-12-26 18:31:01,258][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000433000_110862336.pth... [2023-12-26 18:31:01,262][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000431848_110567424.pth [2023-12-26 18:31:01,320][105620] Updated weights for policy 1, policy_version 433407 (0.0010) [2023-12-26 18:31:01,976][105692] Updated weights for policy 0, policy_version 433004 (0.0007) [2023-12-26 18:31:02,031][105692] Updated weights for policy 0, policy_version 433014 (0.0005) [2023-12-26 18:31:02,037][105620] Updated weights for policy 1, policy_version 433417 (0.0010) [2023-12-26 18:31:02,088][105692] Updated weights for policy 0, policy_version 433024 (0.0006) [2023-12-26 18:31:02,104][105620] Updated weights for policy 1, policy_version 433427 (0.0008) [2023-12-26 18:31:02,170][105620] Updated weights for policy 1, policy_version 433437 (0.0006) [2023-12-26 18:31:02,230][105620] Updated weights for policy 1, policy_version 433447 (0.0007) [2023-12-26 18:31:02,734][105692] Updated weights for policy 0, policy_version 433034 (0.0007) [2023-12-26 18:31:02,790][105692] Updated weights for policy 0, policy_version 433044 (0.0006) [2023-12-26 18:31:02,846][105620] Updated weights for policy 1, policy_version 433457 (0.0005) [2023-12-26 18:31:02,848][105692] Updated weights for policy 0, policy_version 433054 (0.0008) [2023-12-26 18:31:02,899][105692] Updated weights for policy 0, policy_version 433064 (0.0008) [2023-12-26 18:31:02,905][105620] Updated weights for policy 1, policy_version 433467 (0.0006) [2023-12-26 18:31:02,962][105620] Updated weights for policy 1, policy_version 433477 (0.0005) [2023-12-26 18:31:03,484][105620] Updated weights for policy 1, policy_version 433487 (0.0005) [2023-12-26 18:31:03,544][105620] Updated weights for policy 1, policy_version 433497 (0.0005) [2023-12-26 18:31:03,601][105620] Updated weights for policy 1, policy_version 433507 (0.0005) [2023-12-26 18:31:03,616][105692] Updated weights for policy 0, policy_version 433075 (0.0009) [2023-12-26 18:31:03,683][105692] Updated weights for policy 0, policy_version 433085 (0.0009) [2023-12-26 18:31:03,740][105692] Updated weights for policy 0, policy_version 433095 (0.0010) [2023-12-26 18:31:04,145][105620] Updated weights for policy 1, policy_version 433517 (0.0008) [2023-12-26 18:31:04,204][105620] Updated weights for policy 1, policy_version 433527 (0.0008) [2023-12-26 18:31:04,264][105620] Updated weights for policy 1, policy_version 433537 (0.0010) [2023-12-26 18:31:04,565][105692] Updated weights for policy 0, policy_version 433106 (0.0009) [2023-12-26 18:31:04,619][105692] Updated weights for policy 0, policy_version 433116 (0.0010) [2023-12-26 18:31:04,665][105692] Updated weights for policy 0, policy_version 433126 (0.0009) [2023-12-26 18:31:04,906][105620] Updated weights for policy 1, policy_version 433547 (0.0009) [2023-12-26 18:31:04,962][105620] Updated weights for policy 1, policy_version 433557 (0.0010) [2023-12-26 18:31:05,027][105620] Updated weights for policy 1, policy_version 433567 (0.0010) [2023-12-26 18:31:05,326][105692] Updated weights for policy 0, policy_version 433136 (0.0009) [2023-12-26 18:31:05,381][105692] Updated weights for policy 0, policy_version 433146 (0.0010) [2023-12-26 18:31:05,429][105692] Updated weights for policy 0, policy_version 433156 (0.0010) [2023-12-26 18:31:05,742][105620] Updated weights for policy 1, policy_version 433577 (0.0010) [2023-12-26 18:31:05,796][105620] Updated weights for policy 1, policy_version 433587 (0.0010) [2023-12-26 18:31:05,853][105620] Updated weights for policy 1, policy_version 433597 (0.0009) [2023-12-26 18:31:05,910][105620] Updated weights for policy 1, policy_version 433607 (0.0005) [2023-12-26 18:31:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 221921280. Throughput: 0: 9959.1, 1: 9644.7. Samples: 221910112. Policy #0 lag: (min: 45.0, avg: 55.6, max: 56.0) [2023-12-26 18:31:06,062][104569] Avg episode reward: [(0, '9182.979'), (1, '9088.570')] [2023-12-26 18:31:06,198][105692] Updated weights for policy 0, policy_version 433166 (0.0009) [2023-12-26 18:31:06,262][105692] Updated weights for policy 0, policy_version 433176 (0.0008) [2023-12-26 18:31:06,323][105692] Updated weights for policy 0, policy_version 433186 (0.0008) [2023-12-26 18:31:06,574][105620] Updated weights for policy 1, policy_version 433617 (0.0010) [2023-12-26 18:31:06,627][105620] Updated weights for policy 1, policy_version 433627 (0.0010) [2023-12-26 18:31:06,686][105620] Updated weights for policy 1, policy_version 433637 (0.0011) [2023-12-26 18:31:06,973][105692] Updated weights for policy 0, policy_version 433196 (0.0009) [2023-12-26 18:31:07,038][105692] Updated weights for policy 0, policy_version 433206 (0.0010) [2023-12-26 18:31:07,106][105692] Updated weights for policy 0, policy_version 433216 (0.0010) [2023-12-26 18:31:07,387][105620] Updated weights for policy 1, policy_version 433647 (0.0011) [2023-12-26 18:31:07,444][105620] Updated weights for policy 1, policy_version 433657 (0.0011) [2023-12-26 18:31:07,493][105620] Updated weights for policy 1, policy_version 433667 (0.0010) [2023-12-26 18:31:07,672][105692] Updated weights for policy 0, policy_version 433226 (0.0010) [2023-12-26 18:31:07,727][105692] Updated weights for policy 0, policy_version 433236 (0.0010) [2023-12-26 18:31:07,788][105692] Updated weights for policy 0, policy_version 433246 (0.0010) [2023-12-26 18:31:07,841][105692] Updated weights for policy 0, policy_version 433256 (0.0010) [2023-12-26 18:31:08,238][105620] Updated weights for policy 1, policy_version 433677 (0.0010) [2023-12-26 18:31:08,306][105620] Updated weights for policy 1, policy_version 433687 (0.0010) [2023-12-26 18:31:08,373][105620] Updated weights for policy 1, policy_version 433697 (0.0010) [2023-12-26 18:31:08,574][105692] Updated weights for policy 0, policy_version 433266 (0.0005) [2023-12-26 18:31:08,626][105692] Updated weights for policy 0, policy_version 433276 (0.0005) [2023-12-26 18:31:08,682][105692] Updated weights for policy 0, policy_version 433286 (0.0005) [2023-12-26 18:31:09,068][105620] Updated weights for policy 1, policy_version 433707 (0.0010) [2023-12-26 18:31:09,129][105620] Updated weights for policy 1, policy_version 433717 (0.0010) [2023-12-26 18:31:09,190][105620] Updated weights for policy 1, policy_version 433727 (0.0010) [2023-12-26 18:31:09,359][105692] Updated weights for policy 0, policy_version 433296 (0.0009) [2023-12-26 18:31:09,428][105692] Updated weights for policy 0, policy_version 433306 (0.0010) [2023-12-26 18:31:09,487][105692] Updated weights for policy 0, policy_version 433316 (0.0010) [2023-12-26 18:31:09,945][105620] Updated weights for policy 1, policy_version 433737 (0.0011) [2023-12-26 18:31:10,002][105620] Updated weights for policy 1, policy_version 433747 (0.0011) [2023-12-26 18:31:10,065][105620] Updated weights for policy 1, policy_version 433757 (0.0010) [2023-12-26 18:31:10,117][105620] Updated weights for policy 1, policy_version 433767 (0.0010) [2023-12-26 18:31:10,207][105692] Updated weights for policy 0, policy_version 433326 (0.0009) [2023-12-26 18:31:10,260][105692] Updated weights for policy 0, policy_version 433336 (0.0008) [2023-12-26 18:31:10,309][105692] Updated weights for policy 0, policy_version 433346 (0.0008) [2023-12-26 18:31:10,867][105620] Updated weights for policy 1, policy_version 433777 (0.0009) [2023-12-26 18:31:10,918][105620] Updated weights for policy 1, policy_version 433787 (0.0007) [2023-12-26 18:31:10,969][105620] Updated weights for policy 1, policy_version 433797 (0.0009) [2023-12-26 18:31:11,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 222019584. Throughput: 0: 9983.0, 1: 9689.3. Samples: 222028044. Policy #0 lag: (min: 45.0, avg: 55.6, max: 56.0) [2023-12-26 18:31:11,062][104569] Avg episode reward: [(0, '9175.422'), (1, '8677.391')] [2023-12-26 18:31:11,064][105692] Updated weights for policy 0, policy_version 433356 (0.0008) [2023-12-26 18:31:11,143][105692] Updated weights for policy 0, policy_version 433366 (0.0006) [2023-12-26 18:31:11,214][105692] Updated weights for policy 0, policy_version 433376 (0.0007) [2023-12-26 18:31:11,746][105620] Updated weights for policy 1, policy_version 433807 (0.0009) [2023-12-26 18:31:11,810][105620] Updated weights for policy 1, policy_version 433817 (0.0009) [2023-12-26 18:31:11,865][105620] Updated weights for policy 1, policy_version 433827 (0.0008) [2023-12-26 18:31:11,966][105692] Updated weights for policy 0, policy_version 433386 (0.0010) [2023-12-26 18:31:12,028][105692] Updated weights for policy 0, policy_version 433396 (0.0009) [2023-12-26 18:31:12,091][105692] Updated weights for policy 0, policy_version 433406 (0.0009) [2023-12-26 18:31:12,154][105692] Updated weights for policy 0, policy_version 433416 (0.0009) [2023-12-26 18:31:12,648][105620] Updated weights for policy 1, policy_version 433837 (0.0009) [2023-12-26 18:31:12,701][105620] Updated weights for policy 1, policy_version 433847 (0.0009) [2023-12-26 18:31:12,753][105620] Updated weights for policy 1, policy_version 433857 (0.0008) [2023-12-26 18:31:12,920][105692] Updated weights for policy 0, policy_version 433426 (0.0009) [2023-12-26 18:31:12,974][105692] Updated weights for policy 0, policy_version 433436 (0.0009) [2023-12-26 18:31:13,025][105692] Updated weights for policy 0, policy_version 433446 (0.0009) [2023-12-26 18:31:13,495][105620] Updated weights for policy 1, policy_version 433867 (0.0009) [2023-12-26 18:31:13,563][105620] Updated weights for policy 1, policy_version 433877 (0.0009) [2023-12-26 18:31:13,595][105586] KL-divergence is very high: 128.1046 [2023-12-26 18:31:13,624][105620] Updated weights for policy 1, policy_version 433887 (0.0009) [2023-12-26 18:31:13,624][105586] KL-divergence is very high: 223.5560 [2023-12-26 18:31:13,641][105586] KL-divergence is very high: 138.1119 [2023-12-26 18:31:13,665][105586] KL-divergence is very high: 276.2334 [2023-12-26 18:31:13,797][105692] Updated weights for policy 0, policy_version 433456 (0.0009) [2023-12-26 18:31:13,858][105692] Updated weights for policy 0, policy_version 433466 (0.0009) [2023-12-26 18:31:13,922][105692] Updated weights for policy 0, policy_version 433476 (0.0009) [2023-12-26 18:31:14,384][105620] Updated weights for policy 1, policy_version 433897 (0.0008) [2023-12-26 18:31:14,444][105620] Updated weights for policy 1, policy_version 433907 (0.0010) [2023-12-26 18:31:14,499][105620] Updated weights for policy 1, policy_version 433917 (0.0012) [2023-12-26 18:31:14,550][105620] Updated weights for policy 1, policy_version 433927 (0.0008) [2023-12-26 18:31:14,567][105692] Updated weights for policy 0, policy_version 433486 (0.0008) [2023-12-26 18:31:14,621][105692] Updated weights for policy 0, policy_version 433496 (0.0009) [2023-12-26 18:31:14,671][105692] Updated weights for policy 0, policy_version 433506 (0.0009) [2023-12-26 18:31:15,322][105620] Updated weights for policy 1, policy_version 433937 (0.0008) [2023-12-26 18:31:15,387][105620] Updated weights for policy 1, policy_version 433947 (0.0008) [2023-12-26 18:31:15,448][105692] Updated weights for policy 0, policy_version 433516 (0.0010) [2023-12-26 18:31:15,450][105620] Updated weights for policy 1, policy_version 433957 (0.0008) [2023-12-26 18:31:15,500][105692] Updated weights for policy 0, policy_version 433526 (0.0010) [2023-12-26 18:31:15,559][105692] Updated weights for policy 0, policy_version 433536 (0.0009) [2023-12-26 18:31:16,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 222109696. Throughput: 0: 9858.8, 1: 9655.4. Samples: 222083288. Policy #0 lag: (min: 45.0, avg: 55.6, max: 56.0) [2023-12-26 18:31:16,063][104569] Avg episode reward: [(0, '9175.815'), (1, '8299.303')] [2023-12-26 18:31:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000433960_111108096.pth... [2023-12-26 18:31:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000433544_111001600.pth... [2023-12-26 18:31:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000432840_110821376.pth [2023-12-26 18:31:16,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000432424_110714880.pth [2023-12-26 18:31:16,153][105620] Updated weights for policy 1, policy_version 433967 (0.0008) [2023-12-26 18:31:16,207][105620] Updated weights for policy 1, policy_version 433977 (0.0009) [2023-12-26 18:31:16,252][105620] Updated weights for policy 1, policy_version 433987 (0.0008) [2023-12-26 18:31:16,268][105692] Updated weights for policy 0, policy_version 433546 (0.0008) [2023-12-26 18:31:16,328][105692] Updated weights for policy 0, policy_version 433556 (0.0008) [2023-12-26 18:31:16,386][105692] Updated weights for policy 0, policy_version 433566 (0.0009) [2023-12-26 18:31:16,449][105692] Updated weights for policy 0, policy_version 433576 (0.0009) [2023-12-26 18:31:16,951][105620] Updated weights for policy 1, policy_version 433997 (0.0008) [2023-12-26 18:31:16,999][105620] Updated weights for policy 1, policy_version 434007 (0.0009) [2023-12-26 18:31:17,046][105620] Updated weights for policy 1, policy_version 434017 (0.0009) [2023-12-26 18:31:17,223][105692] Updated weights for policy 0, policy_version 433586 (0.0009) [2023-12-26 18:31:17,291][105692] Updated weights for policy 0, policy_version 433596 (0.0010) [2023-12-26 18:31:17,356][105692] Updated weights for policy 0, policy_version 433606 (0.0008) [2023-12-26 18:31:17,848][105620] Updated weights for policy 1, policy_version 434027 (0.0009) [2023-12-26 18:31:17,906][105620] Updated weights for policy 1, policy_version 434037 (0.0008) [2023-12-26 18:31:17,961][105620] Updated weights for policy 1, policy_version 434047 (0.0009) [2023-12-26 18:31:18,147][105692] Updated weights for policy 0, policy_version 433616 (0.0007) [2023-12-26 18:31:18,207][105692] Updated weights for policy 0, policy_version 433626 (0.0006) [2023-12-26 18:31:18,272][105692] Updated weights for policy 0, policy_version 433636 (0.0009) [2023-12-26 18:31:18,722][105620] Updated weights for policy 1, policy_version 434057 (0.0009) [2023-12-26 18:31:18,782][105620] Updated weights for policy 1, policy_version 434067 (0.0011) [2023-12-26 18:31:18,844][105620] Updated weights for policy 1, policy_version 434077 (0.0010) [2023-12-26 18:31:18,901][105692] Updated weights for policy 0, policy_version 433646 (0.0008) [2023-12-26 18:31:18,903][105620] Updated weights for policy 1, policy_version 434087 (0.0011) [2023-12-26 18:31:18,965][105692] Updated weights for policy 0, policy_version 433656 (0.0008) [2023-12-26 18:31:19,028][105692] Updated weights for policy 0, policy_version 433666 (0.0007) [2023-12-26 18:31:19,600][105620] Updated weights for policy 1, policy_version 434097 (0.0007) [2023-12-26 18:31:19,657][105620] Updated weights for policy 1, policy_version 434107 (0.0008) [2023-12-26 18:31:19,692][105692] Updated weights for policy 0, policy_version 433676 (0.0006) [2023-12-26 18:31:19,722][105620] Updated weights for policy 1, policy_version 434117 (0.0006) [2023-12-26 18:31:19,757][105692] Updated weights for policy 0, policy_version 433686 (0.0006) [2023-12-26 18:31:19,815][105692] Updated weights for policy 0, policy_version 433696 (0.0008) [2023-12-26 18:31:20,413][105620] Updated weights for policy 1, policy_version 434127 (0.0006) [2023-12-26 18:31:20,473][105620] Updated weights for policy 1, policy_version 434137 (0.0008) [2023-12-26 18:31:20,535][105620] Updated weights for policy 1, policy_version 434147 (0.0008) [2023-12-26 18:31:20,570][105692] Updated weights for policy 0, policy_version 433706 (0.0009) [2023-12-26 18:31:20,638][105692] Updated weights for policy 0, policy_version 433716 (0.0010) [2023-12-26 18:31:20,698][105692] Updated weights for policy 0, policy_version 433726 (0.0011) [2023-12-26 18:31:20,754][105692] Updated weights for policy 0, policy_version 433736 (0.0010) [2023-12-26 18:31:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 222208000. Throughput: 0: 9752.7, 1: 9659.4. Samples: 222198176. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:31:21,063][104569] Avg episode reward: [(0, '9357.191'), (1, '9088.004')] [2023-12-26 18:31:21,305][105620] Updated weights for policy 1, policy_version 434157 (0.0008) [2023-12-26 18:31:21,381][105620] Updated weights for policy 1, policy_version 434167 (0.0008) [2023-12-26 18:31:21,443][105620] Updated weights for policy 1, policy_version 434177 (0.0008) [2023-12-26 18:31:21,536][105692] Updated weights for policy 0, policy_version 433746 (0.0009) [2023-12-26 18:31:21,604][105692] Updated weights for policy 0, policy_version 433756 (0.0010) [2023-12-26 18:31:21,670][105692] Updated weights for policy 0, policy_version 433766 (0.0009) [2023-12-26 18:31:22,103][105620] Updated weights for policy 1, policy_version 434187 (0.0006) [2023-12-26 18:31:22,154][105620] Updated weights for policy 1, policy_version 434197 (0.0006) [2023-12-26 18:31:22,212][105620] Updated weights for policy 1, policy_version 434207 (0.0008) [2023-12-26 18:31:22,504][105692] Updated weights for policy 0, policy_version 433776 (0.0010) [2023-12-26 18:31:22,558][105692] Updated weights for policy 0, policy_version 433786 (0.0005) [2023-12-26 18:31:22,614][105692] Updated weights for policy 0, policy_version 433796 (0.0006) [2023-12-26 18:31:22,965][105620] Updated weights for policy 1, policy_version 434217 (0.0008) [2023-12-26 18:31:23,029][105620] Updated weights for policy 1, policy_version 434227 (0.0008) [2023-12-26 18:31:23,093][105620] Updated weights for policy 1, policy_version 434237 (0.0008) [2023-12-26 18:31:23,146][105620] Updated weights for policy 1, policy_version 434247 (0.0008) [2023-12-26 18:31:23,278][105692] Updated weights for policy 0, policy_version 433806 (0.0009) [2023-12-26 18:31:23,330][105692] Updated weights for policy 0, policy_version 433816 (0.0011) [2023-12-26 18:31:23,383][105692] Updated weights for policy 0, policy_version 433827 (0.0007) [2023-12-26 18:31:23,976][105620] Updated weights for policy 1, policy_version 434257 (0.0007) [2023-12-26 18:31:23,979][105692] Updated weights for policy 0, policy_version 433837 (0.0007) [2023-12-26 18:31:24,030][105620] Updated weights for policy 1, policy_version 434267 (0.0008) [2023-12-26 18:31:24,033][105692] Updated weights for policy 0, policy_version 433847 (0.0006) [2023-12-26 18:31:24,082][105620] Updated weights for policy 1, policy_version 434277 (0.0009) [2023-12-26 18:31:24,086][105692] Updated weights for policy 0, policy_version 433857 (0.0006) [2023-12-26 18:31:24,754][105692] Updated weights for policy 0, policy_version 433867 (0.0009) [2023-12-26 18:31:24,813][105692] Updated weights for policy 0, policy_version 433877 (0.0009) [2023-12-26 18:31:24,876][105692] Updated weights for policy 0, policy_version 433887 (0.0008) [2023-12-26 18:31:24,882][105620] Updated weights for policy 1, policy_version 434287 (0.0007) [2023-12-26 18:31:24,943][105620] Updated weights for policy 1, policy_version 434297 (0.0009) [2023-12-26 18:31:24,997][105620] Updated weights for policy 1, policy_version 434307 (0.0009) [2023-12-26 18:31:25,606][105620] Updated weights for policy 1, policy_version 434317 (0.0008) [2023-12-26 18:31:25,670][105620] Updated weights for policy 1, policy_version 434327 (0.0008) [2023-12-26 18:31:25,696][105692] Updated weights for policy 0, policy_version 433897 (0.0007) [2023-12-26 18:31:25,718][105620] Updated weights for policy 1, policy_version 434337 (0.0009) [2023-12-26 18:31:25,741][105692] Updated weights for policy 0, policy_version 433907 (0.0005) [2023-12-26 18:31:25,793][105692] Updated weights for policy 0, policy_version 433917 (0.0008) [2023-12-26 18:31:25,849][105692] Updated weights for policy 0, policy_version 433927 (0.0009) [2023-12-26 18:31:26,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 222306304. Throughput: 0: 9687.9, 1: 9687.7. Samples: 222312460. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:31:26,062][104569] Avg episode reward: [(0, '9357.568'), (1, '8906.209')] [2023-12-26 18:31:26,371][105620] Updated weights for policy 1, policy_version 434347 (0.0008) [2023-12-26 18:31:26,429][105620] Updated weights for policy 1, policy_version 434357 (0.0005) [2023-12-26 18:31:26,477][105620] Updated weights for policy 1, policy_version 434367 (0.0005) [2023-12-26 18:31:26,687][105692] Updated weights for policy 0, policy_version 433937 (0.0009) [2023-12-26 18:31:26,733][105692] Updated weights for policy 0, policy_version 433947 (0.0008) [2023-12-26 18:31:26,795][105692] Updated weights for policy 0, policy_version 433957 (0.0009) [2023-12-26 18:31:27,107][105620] Updated weights for policy 1, policy_version 434377 (0.0006) [2023-12-26 18:31:27,158][105620] Updated weights for policy 1, policy_version 434387 (0.0009) [2023-12-26 18:31:27,216][105620] Updated weights for policy 1, policy_version 434397 (0.0009) [2023-12-26 18:31:27,261][105620] Updated weights for policy 1, policy_version 434407 (0.0008) [2023-12-26 18:31:27,568][105692] Updated weights for policy 0, policy_version 433967 (0.0008) [2023-12-26 18:31:27,615][105692] Updated weights for policy 0, policy_version 433977 (0.0009) [2023-12-26 18:31:27,664][105692] Updated weights for policy 0, policy_version 433987 (0.0009) [2023-12-26 18:31:28,038][105620] Updated weights for policy 1, policy_version 434417 (0.0009) [2023-12-26 18:31:28,096][105620] Updated weights for policy 1, policy_version 434427 (0.0009) [2023-12-26 18:31:28,150][105620] Updated weights for policy 1, policy_version 434437 (0.0009) [2023-12-26 18:31:28,436][105692] Updated weights for policy 0, policy_version 433997 (0.0009) [2023-12-26 18:31:28,489][105692] Updated weights for policy 0, policy_version 434007 (0.0009) [2023-12-26 18:31:28,538][105692] Updated weights for policy 0, policy_version 434017 (0.0009) [2023-12-26 18:31:28,914][105620] Updated weights for policy 1, policy_version 434447 (0.0008) [2023-12-26 18:31:28,966][105620] Updated weights for policy 1, policy_version 434457 (0.0007) [2023-12-26 18:31:29,017][105620] Updated weights for policy 1, policy_version 434467 (0.0005) [2023-12-26 18:31:29,283][105692] Updated weights for policy 0, policy_version 434027 (0.0010) [2023-12-26 18:31:29,346][105692] Updated weights for policy 0, policy_version 434037 (0.0011) [2023-12-26 18:31:29,409][105692] Updated weights for policy 0, policy_version 434047 (0.0011) [2023-12-26 18:31:29,726][105620] Updated weights for policy 1, policy_version 434477 (0.0009) [2023-12-26 18:31:29,773][105620] Updated weights for policy 1, policy_version 434487 (0.0008) [2023-12-26 18:31:29,820][105620] Updated weights for policy 1, policy_version 434497 (0.0009) [2023-12-26 18:31:30,170][105692] Updated weights for policy 0, policy_version 434057 (0.0009) [2023-12-26 18:31:30,229][105692] Updated weights for policy 0, policy_version 434067 (0.0006) [2023-12-26 18:31:30,290][105692] Updated weights for policy 0, policy_version 434077 (0.0006) [2023-12-26 18:31:30,353][105692] Updated weights for policy 0, policy_version 434087 (0.0006) [2023-12-26 18:31:30,674][105620] Updated weights for policy 1, policy_version 434507 (0.0008) [2023-12-26 18:31:30,734][105620] Updated weights for policy 1, policy_version 434517 (0.0008) [2023-12-26 18:31:30,780][105620] Updated weights for policy 1, policy_version 434527 (0.0008) [2023-12-26 18:31:30,973][105692] Updated weights for policy 0, policy_version 434097 (0.0010) [2023-12-26 18:31:31,031][105692] Updated weights for policy 0, policy_version 434107 (0.0010) [2023-12-26 18:31:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 222396416. Throughput: 0: 9692.5, 1: 9728.0. Samples: 222369376. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:31:31,062][104569] Avg episode reward: [(0, '9357.859'), (1, '9085.878')] [2023-12-26 18:31:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000434536_111255552.pth... [2023-12-26 18:31:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000433384_110960640.pth [2023-12-26 18:31:31,085][105692] Updated weights for policy 0, policy_version 434117 (0.0011) [2023-12-26 18:31:31,099][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000434120_111149056.pth... [2023-12-26 18:31:31,102][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000433000_110862336.pth [2023-12-26 18:31:31,520][105620] Updated weights for policy 1, policy_version 434537 (0.0008) [2023-12-26 18:31:31,578][105620] Updated weights for policy 1, policy_version 434547 (0.0011) [2023-12-26 18:31:31,642][105620] Updated weights for policy 1, policy_version 434557 (0.0011) [2023-12-26 18:31:31,714][105620] Updated weights for policy 1, policy_version 434567 (0.0009) [2023-12-26 18:31:31,890][105692] Updated weights for policy 0, policy_version 434127 (0.0010) [2023-12-26 18:31:31,961][105692] Updated weights for policy 0, policy_version 434137 (0.0009) [2023-12-26 18:31:32,019][105692] Updated weights for policy 0, policy_version 434147 (0.0010) [2023-12-26 18:31:32,365][105620] Updated weights for policy 1, policy_version 434577 (0.0009) [2023-12-26 18:31:32,431][105620] Updated weights for policy 1, policy_version 434587 (0.0009) [2023-12-26 18:31:32,492][105620] Updated weights for policy 1, policy_version 434597 (0.0008) [2023-12-26 18:31:32,678][105692] Updated weights for policy 0, policy_version 434157 (0.0007) [2023-12-26 18:31:32,731][105692] Updated weights for policy 0, policy_version 434167 (0.0010) [2023-12-26 18:31:32,789][105692] Updated weights for policy 0, policy_version 434177 (0.0010) [2023-12-26 18:31:33,204][105620] Updated weights for policy 1, policy_version 434607 (0.0006) [2023-12-26 18:31:33,257][105620] Updated weights for policy 1, policy_version 434617 (0.0006) [2023-12-26 18:31:33,305][105620] Updated weights for policy 1, policy_version 434627 (0.0005) [2023-12-26 18:31:33,398][105692] Updated weights for policy 0, policy_version 434187 (0.0010) [2023-12-26 18:31:33,449][105692] Updated weights for policy 0, policy_version 434197 (0.0010) [2023-12-26 18:31:33,497][105692] Updated weights for policy 0, policy_version 434207 (0.0010) [2023-12-26 18:31:33,819][105620] Updated weights for policy 1, policy_version 434637 (0.0005) [2023-12-26 18:31:33,867][105620] Updated weights for policy 1, policy_version 434647 (0.0005) [2023-12-26 18:31:33,925][105620] Updated weights for policy 1, policy_version 434657 (0.0006) [2023-12-26 18:31:34,263][105692] Updated weights for policy 0, policy_version 434217 (0.0010) [2023-12-26 18:31:34,322][105692] Updated weights for policy 0, policy_version 434227 (0.0011) [2023-12-26 18:31:34,383][105692] Updated weights for policy 0, policy_version 434237 (0.0011) [2023-12-26 18:31:34,449][105692] Updated weights for policy 0, policy_version 434247 (0.0011) [2023-12-26 18:31:34,665][105620] Updated weights for policy 1, policy_version 434667 (0.0010) [2023-12-26 18:31:34,727][105620] Updated weights for policy 1, policy_version 434677 (0.0010) [2023-12-26 18:31:34,790][105620] Updated weights for policy 1, policy_version 434687 (0.0010) [2023-12-26 18:31:35,025][105692] Updated weights for policy 0, policy_version 434257 (0.0008) [2023-12-26 18:31:35,090][105692] Updated weights for policy 0, policy_version 434267 (0.0010) [2023-12-26 18:31:35,152][105692] Updated weights for policy 0, policy_version 434277 (0.0009) [2023-12-26 18:31:35,431][105620] Updated weights for policy 1, policy_version 434697 (0.0010) [2023-12-26 18:31:35,492][105620] Updated weights for policy 1, policy_version 434707 (0.0009) [2023-12-26 18:31:35,546][105620] Updated weights for policy 1, policy_version 434717 (0.0009) [2023-12-26 18:31:35,598][105620] Updated weights for policy 1, policy_version 434727 (0.0008) [2023-12-26 18:31:35,799][105692] Updated weights for policy 0, policy_version 434287 (0.0007) [2023-12-26 18:31:35,867][105692] Updated weights for policy 0, policy_version 434297 (0.0007) [2023-12-26 18:31:35,925][105692] Updated weights for policy 0, policy_version 434307 (0.0009) [2023-12-26 18:31:36,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19605.2). Total num frames: 222502912. Throughput: 0: 9672.3, 1: 9802.7. Samples: 222487212. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:31:36,063][104569] Avg episode reward: [(0, '9358.342'), (1, '9177.528')] [2023-12-26 18:31:36,202][105620] Updated weights for policy 1, policy_version 434737 (0.0009) [2023-12-26 18:31:36,261][105620] Updated weights for policy 1, policy_version 434747 (0.0010) [2023-12-26 18:31:36,314][105620] Updated weights for policy 1, policy_version 434757 (0.0009) [2023-12-26 18:31:36,537][105692] Updated weights for policy 0, policy_version 434318 (0.0007) [2023-12-26 18:31:36,607][105692] Updated weights for policy 0, policy_version 434328 (0.0006) [2023-12-26 18:31:36,672][105692] Updated weights for policy 0, policy_version 434338 (0.0008) [2023-12-26 18:31:37,122][105620] Updated weights for policy 1, policy_version 434767 (0.0009) [2023-12-26 18:31:37,183][105620] Updated weights for policy 1, policy_version 434777 (0.0009) [2023-12-26 18:31:37,252][105620] Updated weights for policy 1, policy_version 434787 (0.0009) [2023-12-26 18:31:37,318][105692] Updated weights for policy 0, policy_version 434348 (0.0009) [2023-12-26 18:31:37,374][105692] Updated weights for policy 0, policy_version 434358 (0.0009) [2023-12-26 18:31:37,433][105692] Updated weights for policy 0, policy_version 434368 (0.0009) [2023-12-26 18:31:38,062][105692] Updated weights for policy 0, policy_version 434378 (0.0008) [2023-12-26 18:31:38,081][105620] Updated weights for policy 1, policy_version 434797 (0.0009) [2023-12-26 18:31:38,115][105692] Updated weights for policy 0, policy_version 434388 (0.0005) [2023-12-26 18:31:38,140][105620] Updated weights for policy 1, policy_version 434807 (0.0009) [2023-12-26 18:31:38,170][105692] Updated weights for policy 0, policy_version 434398 (0.0006) [2023-12-26 18:31:38,192][105620] Updated weights for policy 1, policy_version 434817 (0.0007) [2023-12-26 18:31:38,221][105692] Updated weights for policy 0, policy_version 434408 (0.0008) [2023-12-26 18:31:38,921][105620] Updated weights for policy 1, policy_version 434827 (0.0008) [2023-12-26 18:31:38,963][105692] Updated weights for policy 0, policy_version 434418 (0.0007) [2023-12-26 18:31:38,979][105620] Updated weights for policy 1, policy_version 434837 (0.0009) [2023-12-26 18:31:39,009][105692] Updated weights for policy 0, policy_version 434428 (0.0008) [2023-12-26 18:31:39,039][105620] Updated weights for policy 1, policy_version 434847 (0.0008) [2023-12-26 18:31:39,068][105692] Updated weights for policy 0, policy_version 434438 (0.0007) [2023-12-26 18:31:39,803][105620] Updated weights for policy 1, policy_version 434857 (0.0007) [2023-12-26 18:31:39,826][105692] Updated weights for policy 0, policy_version 434448 (0.0007) [2023-12-26 18:31:39,867][105620] Updated weights for policy 1, policy_version 434867 (0.0009) [2023-12-26 18:31:39,890][105692] Updated weights for policy 0, policy_version 434458 (0.0006) [2023-12-26 18:31:39,934][105620] Updated weights for policy 1, policy_version 434877 (0.0008) [2023-12-26 18:31:39,952][105692] Updated weights for policy 0, policy_version 434468 (0.0010) [2023-12-26 18:31:39,990][105620] Updated weights for policy 1, policy_version 434887 (0.0008) [2023-12-26 18:31:40,674][105692] Updated weights for policy 0, policy_version 434478 (0.0006) [2023-12-26 18:31:40,680][105620] Updated weights for policy 1, policy_version 434897 (0.0010) [2023-12-26 18:31:40,730][105692] Updated weights for policy 0, policy_version 434488 (0.0005) [2023-12-26 18:31:40,732][105620] Updated weights for policy 1, policy_version 434907 (0.0010) [2023-12-26 18:31:40,784][105620] Updated weights for policy 1, policy_version 434917 (0.0010) [2023-12-26 18:31:40,786][105692] Updated weights for policy 0, policy_version 434498 (0.0006) [2023-12-26 18:31:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 222601216. Throughput: 0: 9779.0, 1: 9743.0. Samples: 222606588. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:31:41,063][104569] Avg episode reward: [(0, '9358.508'), (1, '8996.847')] [2023-12-26 18:31:41,560][105692] Updated weights for policy 0, policy_version 434508 (0.0006) [2023-12-26 18:31:41,566][105620] Updated weights for policy 1, policy_version 434927 (0.0010) [2023-12-26 18:31:41,613][105692] Updated weights for policy 0, policy_version 434518 (0.0006) [2023-12-26 18:31:41,619][105620] Updated weights for policy 1, policy_version 434937 (0.0010) [2023-12-26 18:31:41,684][105620] Updated weights for policy 1, policy_version 434947 (0.0010) [2023-12-26 18:31:41,690][105692] Updated weights for policy 0, policy_version 434528 (0.0006) [2023-12-26 18:31:42,460][105692] Updated weights for policy 0, policy_version 434538 (0.0007) [2023-12-26 18:31:42,460][105620] Updated weights for policy 1, policy_version 434957 (0.0010) [2023-12-26 18:31:42,510][105692] Updated weights for policy 0, policy_version 434548 (0.0006) [2023-12-26 18:31:42,519][105620] Updated weights for policy 1, policy_version 434967 (0.0010) [2023-12-26 18:31:42,569][105692] Updated weights for policy 0, policy_version 434558 (0.0006) [2023-12-26 18:31:42,575][105620] Updated weights for policy 1, policy_version 434977 (0.0010) [2023-12-26 18:31:42,622][105692] Updated weights for policy 0, policy_version 434568 (0.0006) [2023-12-26 18:31:43,324][105620] Updated weights for policy 1, policy_version 434987 (0.0010) [2023-12-26 18:31:43,376][105620] Updated weights for policy 1, policy_version 434997 (0.0010) [2023-12-26 18:31:43,386][105692] Updated weights for policy 0, policy_version 434578 (0.0006) [2023-12-26 18:31:43,434][105620] Updated weights for policy 1, policy_version 435007 (0.0010) [2023-12-26 18:31:43,445][105692] Updated weights for policy 0, policy_version 434588 (0.0005) [2023-12-26 18:31:43,493][105692] Updated weights for policy 0, policy_version 434598 (0.0006) [2023-12-26 18:31:44,119][105620] Updated weights for policy 1, policy_version 435017 (0.0010) [2023-12-26 18:31:44,164][105620] Updated weights for policy 1, policy_version 435027 (0.0005) [2023-12-26 18:31:44,222][105620] Updated weights for policy 1, policy_version 435037 (0.0008) [2023-12-26 18:31:44,283][105620] Updated weights for policy 1, policy_version 435047 (0.0007) [2023-12-26 18:31:44,296][105692] Updated weights for policy 0, policy_version 434608 (0.0009) [2023-12-26 18:31:44,355][105692] Updated weights for policy 0, policy_version 434618 (0.0006) [2023-12-26 18:31:44,415][105692] Updated weights for policy 0, policy_version 434628 (0.0008) [2023-12-26 18:31:45,022][105620] Updated weights for policy 1, policy_version 435057 (0.0010) [2023-12-26 18:31:45,086][105620] Updated weights for policy 1, policy_version 435067 (0.0011) [2023-12-26 18:31:45,088][105692] Updated weights for policy 0, policy_version 434638 (0.0008) [2023-12-26 18:31:45,148][105692] Updated weights for policy 0, policy_version 434648 (0.0006) [2023-12-26 18:31:45,153][105620] Updated weights for policy 1, policy_version 435077 (0.0011) [2023-12-26 18:31:45,211][105692] Updated weights for policy 0, policy_version 434658 (0.0007) [2023-12-26 18:31:45,882][105620] Updated weights for policy 1, policy_version 435087 (0.0010) [2023-12-26 18:31:45,922][105692] Updated weights for policy 0, policy_version 434668 (0.0006) [2023-12-26 18:31:45,946][105620] Updated weights for policy 1, policy_version 435097 (0.0010) [2023-12-26 18:31:45,973][105692] Updated weights for policy 0, policy_version 434678 (0.0007) [2023-12-26 18:31:46,005][105620] Updated weights for policy 1, policy_version 435107 (0.0006) [2023-12-26 18:31:46,022][105692] Updated weights for policy 0, policy_version 434688 (0.0008) [2023-12-26 18:31:46,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 222699520. Throughput: 0: 9656.6, 1: 9700.6. Samples: 222661976. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:31:46,062][104569] Avg episode reward: [(0, '9358.385'), (1, '8996.711')] [2023-12-26 18:31:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000434696_111296512.pth... [2023-12-26 18:31:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000435112_111403008.pth... [2023-12-26 18:31:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000433544_111001600.pth [2023-12-26 18:31:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000433960_111108096.pth [2023-12-26 18:31:46,556][105620] Updated weights for policy 1, policy_version 435117 (0.0008) [2023-12-26 18:31:46,606][105692] Updated weights for policy 0, policy_version 434698 (0.0008) [2023-12-26 18:31:46,617][105620] Updated weights for policy 1, policy_version 435127 (0.0005) [2023-12-26 18:31:46,658][105692] Updated weights for policy 0, policy_version 434708 (0.0009) [2023-12-26 18:31:46,681][105620] Updated weights for policy 1, policy_version 435137 (0.0005) [2023-12-26 18:31:46,711][105692] Updated weights for policy 0, policy_version 434718 (0.0009) [2023-12-26 18:31:46,763][105692] Updated weights for policy 0, policy_version 434728 (0.0010) [2023-12-26 18:31:47,206][105620] Updated weights for policy 1, policy_version 435147 (0.0006) [2023-12-26 18:31:47,263][105620] Updated weights for policy 1, policy_version 435157 (0.0005) [2023-12-26 18:31:47,319][105620] Updated weights for policy 1, policy_version 435167 (0.0005) [2023-12-26 18:31:47,630][105692] Updated weights for policy 0, policy_version 434738 (0.0011) [2023-12-26 18:31:47,682][105692] Updated weights for policy 0, policy_version 434748 (0.0011) [2023-12-26 18:31:47,737][105692] Updated weights for policy 0, policy_version 434758 (0.0011) [2023-12-26 18:31:47,972][105620] Updated weights for policy 1, policy_version 435177 (0.0006) [2023-12-26 18:31:48,033][105620] Updated weights for policy 1, policy_version 435187 (0.0011) [2023-12-26 18:31:48,107][105620] Updated weights for policy 1, policy_version 435197 (0.0011) [2023-12-26 18:31:48,169][105620] Updated weights for policy 1, policy_version 435207 (0.0011) [2023-12-26 18:31:48,428][105692] Updated weights for policy 0, policy_version 434768 (0.0011) [2023-12-26 18:31:48,492][105692] Updated weights for policy 0, policy_version 434778 (0.0011) [2023-12-26 18:31:48,551][105692] Updated weights for policy 0, policy_version 434788 (0.0011) [2023-12-26 18:31:48,867][105620] Updated weights for policy 1, policy_version 435217 (0.0010) [2023-12-26 18:31:48,911][105620] Updated weights for policy 1, policy_version 435227 (0.0010) [2023-12-26 18:31:48,960][105620] Updated weights for policy 1, policy_version 435237 (0.0010) [2023-12-26 18:31:49,314][105692] Updated weights for policy 0, policy_version 434798 (0.0011) [2023-12-26 18:31:49,379][105692] Updated weights for policy 0, policy_version 434808 (0.0009) [2023-12-26 18:31:49,442][105692] Updated weights for policy 0, policy_version 434818 (0.0011) [2023-12-26 18:31:49,643][105620] Updated weights for policy 1, policy_version 435247 (0.0008) [2023-12-26 18:31:49,700][105620] Updated weights for policy 1, policy_version 435257 (0.0005) [2023-12-26 18:31:49,758][105620] Updated weights for policy 1, policy_version 435267 (0.0008) [2023-12-26 18:31:50,229][105692] Updated weights for policy 0, policy_version 434828 (0.0011) [2023-12-26 18:31:50,289][105692] Updated weights for policy 0, policy_version 434838 (0.0011) [2023-12-26 18:31:50,348][105692] Updated weights for policy 0, policy_version 434848 (0.0011) [2023-12-26 18:31:50,416][105620] Updated weights for policy 1, policy_version 435277 (0.0008) [2023-12-26 18:31:50,469][105620] Updated weights for policy 1, policy_version 435287 (0.0007) [2023-12-26 18:31:50,527][105620] Updated weights for policy 1, policy_version 435297 (0.0010) [2023-12-26 18:31:50,935][105692] Updated weights for policy 0, policy_version 434858 (0.0010) [2023-12-26 18:31:50,984][105692] Updated weights for policy 0, policy_version 434868 (0.0010) [2023-12-26 18:31:51,036][105692] Updated weights for policy 0, policy_version 434878 (0.0010) [2023-12-26 18:31:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 222789632. Throughput: 0: 9686.3, 1: 9700.4. Samples: 222782516. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:31:51,062][104569] Avg episode reward: [(0, '9269.030'), (1, '8999.706')] [2023-12-26 18:31:51,104][105692] Updated weights for policy 0, policy_version 434888 (0.0009) [2023-12-26 18:31:51,295][105620] Updated weights for policy 1, policy_version 435307 (0.0011) [2023-12-26 18:31:51,357][105620] Updated weights for policy 1, policy_version 435317 (0.0009) [2023-12-26 18:31:51,425][105620] Updated weights for policy 1, policy_version 435327 (0.0010) [2023-12-26 18:31:51,897][105692] Updated weights for policy 0, policy_version 434898 (0.0008) [2023-12-26 18:31:51,953][105692] Updated weights for policy 0, policy_version 434908 (0.0008) [2023-12-26 18:31:52,012][105692] Updated weights for policy 0, policy_version 434918 (0.0008) [2023-12-26 18:31:52,148][105620] Updated weights for policy 1, policy_version 435337 (0.0012) [2023-12-26 18:31:52,216][105620] Updated weights for policy 1, policy_version 435347 (0.0010) [2023-12-26 18:31:52,273][105620] Updated weights for policy 1, policy_version 435357 (0.0011) [2023-12-26 18:31:52,327][105620] Updated weights for policy 1, policy_version 435367 (0.0009) [2023-12-26 18:31:52,772][105692] Updated weights for policy 0, policy_version 434928 (0.0008) [2023-12-26 18:31:52,825][105692] Updated weights for policy 0, policy_version 434938 (0.0010) [2023-12-26 18:31:52,878][105692] Updated weights for policy 0, policy_version 434948 (0.0010) [2023-12-26 18:31:53,027][105620] Updated weights for policy 1, policy_version 435377 (0.0009) [2023-12-26 18:31:53,087][105620] Updated weights for policy 1, policy_version 435388 (0.0010) [2023-12-26 18:31:53,140][105620] Updated weights for policy 1, policy_version 435398 (0.0009) [2023-12-26 18:31:53,563][105692] Updated weights for policy 0, policy_version 434958 (0.0009) [2023-12-26 18:31:53,625][105692] Updated weights for policy 0, policy_version 434968 (0.0007) [2023-12-26 18:31:53,687][105692] Updated weights for policy 0, policy_version 434978 (0.0005) [2023-12-26 18:31:53,885][105620] Updated weights for policy 1, policy_version 435408 (0.0010) [2023-12-26 18:31:53,942][105620] Updated weights for policy 1, policy_version 435418 (0.0008) [2023-12-26 18:31:54,006][105620] Updated weights for policy 1, policy_version 435428 (0.0009) [2023-12-26 18:31:54,289][105692] Updated weights for policy 0, policy_version 434988 (0.0005) [2023-12-26 18:31:54,342][105692] Updated weights for policy 0, policy_version 434998 (0.0007) [2023-12-26 18:31:54,394][105692] Updated weights for policy 0, policy_version 435008 (0.0009) [2023-12-26 18:31:54,844][105620] Updated weights for policy 1, policy_version 435438 (0.0009) [2023-12-26 18:31:54,898][105620] Updated weights for policy 1, policy_version 435448 (0.0009) [2023-12-26 18:31:54,951][105620] Updated weights for policy 1, policy_version 435458 (0.0008) [2023-12-26 18:31:55,027][105692] Updated weights for policy 0, policy_version 435018 (0.0008) [2023-12-26 18:31:55,088][105692] Updated weights for policy 0, policy_version 435028 (0.0005) [2023-12-26 18:31:55,145][105692] Updated weights for policy 0, policy_version 435038 (0.0005) [2023-12-26 18:31:55,198][105692] Updated weights for policy 0, policy_version 435048 (0.0005) [2023-12-26 18:31:55,710][105692] Updated weights for policy 0, policy_version 435058 (0.0007) [2023-12-26 18:31:55,765][105692] Updated weights for policy 0, policy_version 435068 (0.0010) [2023-12-26 18:31:55,832][105620] Updated weights for policy 1, policy_version 435468 (0.0008) [2023-12-26 18:31:55,832][105692] Updated weights for policy 0, policy_version 435078 (0.0010) [2023-12-26 18:31:55,895][105620] Updated weights for policy 1, policy_version 435478 (0.0008) [2023-12-26 18:31:55,950][105620] Updated weights for policy 1, policy_version 435488 (0.0008) [2023-12-26 18:31:56,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 222896128. Throughput: 0: 9730.0, 1: 9642.4. Samples: 222899808. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:31:56,063][104569] Avg episode reward: [(0, '9269.136'), (1, '9357.330')] [2023-12-26 18:31:56,452][105692] Updated weights for policy 0, policy_version 435088 (0.0006) [2023-12-26 18:31:56,517][105692] Updated weights for policy 0, policy_version 435098 (0.0005) [2023-12-26 18:31:56,575][105692] Updated weights for policy 0, policy_version 435108 (0.0005) [2023-12-26 18:31:56,801][105620] Updated weights for policy 1, policy_version 435498 (0.0008) [2023-12-26 18:31:56,851][105620] Updated weights for policy 1, policy_version 435508 (0.0005) [2023-12-26 18:31:56,899][105620] Updated weights for policy 1, policy_version 435518 (0.0005) [2023-12-26 18:31:56,946][105620] Updated weights for policy 1, policy_version 435528 (0.0006) [2023-12-26 18:31:57,066][105692] Updated weights for policy 0, policy_version 435118 (0.0005) [2023-12-26 18:31:57,127][105692] Updated weights for policy 0, policy_version 435128 (0.0005) [2023-12-26 18:31:57,191][105692] Updated weights for policy 0, policy_version 435138 (0.0005) [2023-12-26 18:31:57,496][105620] Updated weights for policy 1, policy_version 435538 (0.0005) [2023-12-26 18:31:57,548][105620] Updated weights for policy 1, policy_version 435548 (0.0005) [2023-12-26 18:31:57,599][105620] Updated weights for policy 1, policy_version 435558 (0.0005) [2023-12-26 18:31:57,734][105692] Updated weights for policy 0, policy_version 435148 (0.0007) [2023-12-26 18:31:57,788][105692] Updated weights for policy 0, policy_version 435158 (0.0010) [2023-12-26 18:31:57,872][105692] Updated weights for policy 0, policy_version 435168 (0.0010) [2023-12-26 18:31:58,198][105620] Updated weights for policy 1, policy_version 435568 (0.0007) [2023-12-26 18:31:58,261][105620] Updated weights for policy 1, policy_version 435578 (0.0005) [2023-12-26 18:31:58,323][105620] Updated weights for policy 1, policy_version 435588 (0.0006) [2023-12-26 18:31:58,616][105692] Updated weights for policy 0, policy_version 435178 (0.0010) [2023-12-26 18:31:58,687][105692] Updated weights for policy 0, policy_version 435188 (0.0008) [2023-12-26 18:31:58,758][105692] Updated weights for policy 0, policy_version 435198 (0.0008) [2023-12-26 18:31:58,834][105692] Updated weights for policy 0, policy_version 435208 (0.0009) [2023-12-26 18:31:59,093][105620] Updated weights for policy 1, policy_version 435598 (0.0007) [2023-12-26 18:31:59,163][105620] Updated weights for policy 1, policy_version 435608 (0.0008) [2023-12-26 18:31:59,234][105620] Updated weights for policy 1, policy_version 435618 (0.0009) [2023-12-26 18:31:59,601][105692] Updated weights for policy 0, policy_version 435218 (0.0009) [2023-12-26 18:31:59,658][105692] Updated weights for policy 0, policy_version 435228 (0.0009) [2023-12-26 18:31:59,720][105692] Updated weights for policy 0, policy_version 435238 (0.0006) [2023-12-26 18:32:00,057][105620] Updated weights for policy 1, policy_version 435628 (0.0008) [2023-12-26 18:32:00,109][105620] Updated weights for policy 1, policy_version 435638 (0.0006) [2023-12-26 18:32:00,168][105620] Updated weights for policy 1, policy_version 435648 (0.0005) [2023-12-26 18:32:00,386][105692] Updated weights for policy 0, policy_version 435248 (0.0009) [2023-12-26 18:32:00,437][105692] Updated weights for policy 0, policy_version 435258 (0.0010) [2023-12-26 18:32:00,495][105692] Updated weights for policy 0, policy_version 435268 (0.0010) [2023-12-26 18:32:00,754][105620] Updated weights for policy 1, policy_version 435658 (0.0005) [2023-12-26 18:32:00,798][105620] Updated weights for policy 1, policy_version 435668 (0.0006) [2023-12-26 18:32:00,851][105620] Updated weights for policy 1, policy_version 435679 (0.0010) [2023-12-26 18:32:01,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 222994432. Throughput: 0: 9856.5, 1: 9714.8. Samples: 222963992. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:32:01,062][104569] Avg episode reward: [(0, '9358.686'), (1, '9082.612')] [2023-12-26 18:32:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000435272_111443968.pth... [2023-12-26 18:32:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000435688_111550464.pth... [2023-12-26 18:32:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000434536_111255552.pth [2023-12-26 18:32:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000434120_111149056.pth [2023-12-26 18:32:01,187][105692] Updated weights for policy 0, policy_version 435278 (0.0010) [2023-12-26 18:32:01,232][105692] Updated weights for policy 0, policy_version 435288 (0.0007) [2023-12-26 18:32:01,297][105692] Updated weights for policy 0, policy_version 435298 (0.0008) [2023-12-26 18:32:01,576][105620] Updated weights for policy 1, policy_version 435690 (0.0009) [2023-12-26 18:32:01,643][105620] Updated weights for policy 1, policy_version 435700 (0.0008) [2023-12-26 18:32:01,708][105620] Updated weights for policy 1, policy_version 435710 (0.0010) [2023-12-26 18:32:01,779][105620] Updated weights for policy 1, policy_version 435720 (0.0009) [2023-12-26 18:32:02,112][105692] Updated weights for policy 0, policy_version 435308 (0.0009) [2023-12-26 18:32:02,159][105692] Updated weights for policy 0, policy_version 435318 (0.0009) [2023-12-26 18:32:02,205][105692] Updated weights for policy 0, policy_version 435328 (0.0008) [2023-12-26 18:32:02,503][105620] Updated weights for policy 1, policy_version 435730 (0.0007) [2023-12-26 18:32:02,564][105620] Updated weights for policy 1, policy_version 435740 (0.0007) [2023-12-26 18:32:02,625][105620] Updated weights for policy 1, policy_version 435750 (0.0008) [2023-12-26 18:32:02,991][105692] Updated weights for policy 0, policy_version 435338 (0.0010) [2023-12-26 18:32:03,047][105692] Updated weights for policy 0, policy_version 435348 (0.0009) [2023-12-26 18:32:03,102][105692] Updated weights for policy 0, policy_version 435359 (0.0009) [2023-12-26 18:32:03,270][105620] Updated weights for policy 1, policy_version 435760 (0.0008) [2023-12-26 18:32:03,320][105620] Updated weights for policy 1, policy_version 435770 (0.0006) [2023-12-26 18:32:03,363][105620] Updated weights for policy 1, policy_version 435780 (0.0005) [2023-12-26 18:32:03,944][105692] Updated weights for policy 0, policy_version 435369 (0.0009) [2023-12-26 18:32:03,956][105620] Updated weights for policy 1, policy_version 435790 (0.0007) [2023-12-26 18:32:04,007][105692] Updated weights for policy 0, policy_version 435379 (0.0007) [2023-12-26 18:32:04,012][105620] Updated weights for policy 1, policy_version 435800 (0.0010) [2023-12-26 18:32:04,064][105692] Updated weights for policy 0, policy_version 435389 (0.0008) [2023-12-26 18:32:04,070][105620] Updated weights for policy 1, policy_version 435810 (0.0008) [2023-12-26 18:32:04,130][105692] Updated weights for policy 0, policy_version 435399 (0.0008) [2023-12-26 18:32:04,829][105620] Updated weights for policy 1, policy_version 435820 (0.0007) [2023-12-26 18:32:04,864][105692] Updated weights for policy 0, policy_version 435409 (0.0008) [2023-12-26 18:32:04,878][105620] Updated weights for policy 1, policy_version 435830 (0.0007) [2023-12-26 18:32:04,912][105692] Updated weights for policy 0, policy_version 435419 (0.0007) [2023-12-26 18:32:04,929][105620] Updated weights for policy 1, policy_version 435840 (0.0007) [2023-12-26 18:32:04,961][105692] Updated weights for policy 0, policy_version 435429 (0.0007) [2023-12-26 18:32:05,690][105620] Updated weights for policy 1, policy_version 435850 (0.0006) [2023-12-26 18:32:05,730][105692] Updated weights for policy 0, policy_version 435439 (0.0008) [2023-12-26 18:32:05,736][105620] Updated weights for policy 1, policy_version 435860 (0.0007) [2023-12-26 18:32:05,778][105692] Updated weights for policy 0, policy_version 435449 (0.0007) [2023-12-26 18:32:05,784][105620] Updated weights for policy 1, policy_version 435870 (0.0007) [2023-12-26 18:32:05,826][105692] Updated weights for policy 0, policy_version 435459 (0.0005) [2023-12-26 18:32:05,846][105620] Updated weights for policy 1, policy_version 435880 (0.0007) [2023-12-26 18:32:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 223092736. Throughput: 0: 9800.8, 1: 9772.1. Samples: 223078964. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:32:06,063][104569] Avg episode reward: [(0, '9358.095'), (1, '9082.297')] [2023-12-26 18:32:06,483][105692] Updated weights for policy 0, policy_version 435469 (0.0009) [2023-12-26 18:32:06,552][105692] Updated weights for policy 0, policy_version 435479 (0.0011) [2023-12-26 18:32:06,618][105692] Updated weights for policy 0, policy_version 435489 (0.0011) [2023-12-26 18:32:06,660][105620] Updated weights for policy 1, policy_version 435890 (0.0006) [2023-12-26 18:32:06,720][105620] Updated weights for policy 1, policy_version 435900 (0.0008) [2023-12-26 18:32:06,779][105620] Updated weights for policy 1, policy_version 435910 (0.0008) [2023-12-26 18:32:07,332][105692] Updated weights for policy 0, policy_version 435499 (0.0011) [2023-12-26 18:32:07,397][105692] Updated weights for policy 0, policy_version 435509 (0.0011) [2023-12-26 18:32:07,460][105692] Updated weights for policy 0, policy_version 435519 (0.0011) [2023-12-26 18:32:07,460][105620] Updated weights for policy 1, policy_version 435920 (0.0005) [2023-12-26 18:32:07,516][105620] Updated weights for policy 1, policy_version 435930 (0.0005) [2023-12-26 18:32:07,581][105620] Updated weights for policy 1, policy_version 435940 (0.0005) [2023-12-26 18:32:08,187][105620] Updated weights for policy 1, policy_version 435950 (0.0007) [2023-12-26 18:32:08,196][105692] Updated weights for policy 0, policy_version 435529 (0.0011) [2023-12-26 18:32:08,237][105620] Updated weights for policy 1, policy_version 435960 (0.0007) [2023-12-26 18:32:08,257][105692] Updated weights for policy 0, policy_version 435539 (0.0010) [2023-12-26 18:32:08,287][105620] Updated weights for policy 1, policy_version 435970 (0.0007) [2023-12-26 18:32:08,316][105692] Updated weights for policy 0, policy_version 435549 (0.0011) [2023-12-26 18:32:08,385][105692] Updated weights for policy 0, policy_version 435559 (0.0011) [2023-12-26 18:32:09,019][105620] Updated weights for policy 1, policy_version 435980 (0.0008) [2023-12-26 18:32:09,079][105620] Updated weights for policy 1, policy_version 435990 (0.0009) [2023-12-26 18:32:09,085][105586] KL-divergence is very high: 127.3517 [2023-12-26 18:32:09,097][105586] KL-divergence is very high: 161.0644 [2023-12-26 18:32:09,099][105692] Updated weights for policy 0, policy_version 435569 (0.0011) [2023-12-26 18:32:09,125][105586] KL-divergence is very high: 180.7479 [2023-12-26 18:32:09,130][105586] KL-divergence is very high: 223.1398 [2023-12-26 18:32:09,136][105620] Updated weights for policy 1, policy_version 436000 (0.0006) [2023-12-26 18:32:09,143][105586] KL-divergence is very high: 224.8866 [2023-12-26 18:32:09,154][105692] Updated weights for policy 0, policy_version 435579 (0.0011) [2023-12-26 18:32:09,172][105586] KL-divergence is very high: 172.1985 [2023-12-26 18:32:09,178][105586] KL-divergence is very high: 211.0325 [2023-12-26 18:32:09,212][105692] Updated weights for policy 0, policy_version 435589 (0.0010) [2023-12-26 18:32:09,908][105692] Updated weights for policy 0, policy_version 435599 (0.0009) [2023-12-26 18:32:09,970][105620] Updated weights for policy 1, policy_version 436010 (0.0005) [2023-12-26 18:32:09,976][105692] Updated weights for policy 0, policy_version 435609 (0.0008) [2023-12-26 18:32:10,027][105620] Updated weights for policy 1, policy_version 436020 (0.0007) [2023-12-26 18:32:10,036][105692] Updated weights for policy 0, policy_version 435619 (0.0008) [2023-12-26 18:32:10,085][105620] Updated weights for policy 1, policy_version 436030 (0.0006) [2023-12-26 18:32:10,149][105620] Updated weights for policy 1, policy_version 436040 (0.0008) [2023-12-26 18:32:10,738][105692] Updated weights for policy 0, policy_version 435629 (0.0009) [2023-12-26 18:32:10,789][105692] Updated weights for policy 0, policy_version 435639 (0.0010) [2023-12-26 18:32:10,844][105692] Updated weights for policy 0, policy_version 435649 (0.0010) [2023-12-26 18:32:10,937][105620] Updated weights for policy 1, policy_version 436050 (0.0008) [2023-12-26 18:32:11,003][105620] Updated weights for policy 1, policy_version 436060 (0.0008) [2023-12-26 18:32:11,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 223182848. Throughput: 0: 9821.9, 1: 9774.4. Samples: 223194296. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:32:11,063][104569] Avg episode reward: [(0, '9357.437'), (1, '7097.814')] [2023-12-26 18:32:11,072][105620] Updated weights for policy 1, policy_version 436070 (0.0007) [2023-12-26 18:32:11,632][105692] Updated weights for policy 0, policy_version 435659 (0.0010) [2023-12-26 18:32:11,693][105692] Updated weights for policy 0, policy_version 435669 (0.0007) [2023-12-26 18:32:11,762][105692] Updated weights for policy 0, policy_version 435679 (0.0010) [2023-12-26 18:32:11,831][105620] Updated weights for policy 1, policy_version 436080 (0.0007) [2023-12-26 18:32:11,892][105620] Updated weights for policy 1, policy_version 436090 (0.0008) [2023-12-26 18:32:11,961][105620] Updated weights for policy 1, policy_version 436100 (0.0008) [2023-12-26 18:32:12,415][105692] Updated weights for policy 0, policy_version 435689 (0.0010) [2023-12-26 18:32:12,481][105692] Updated weights for policy 0, policy_version 435699 (0.0007) [2023-12-26 18:32:12,552][105692] Updated weights for policy 0, policy_version 435709 (0.0005) [2023-12-26 18:32:12,621][105692] Updated weights for policy 0, policy_version 435719 (0.0005) [2023-12-26 18:32:12,791][105620] Updated weights for policy 1, policy_version 436110 (0.0007) [2023-12-26 18:32:12,858][105620] Updated weights for policy 1, policy_version 436120 (0.0009) [2023-12-26 18:32:12,916][105620] Updated weights for policy 1, policy_version 436130 (0.0010) [2023-12-26 18:32:13,135][105692] Updated weights for policy 0, policy_version 435729 (0.0006) [2023-12-26 18:32:13,193][105692] Updated weights for policy 0, policy_version 435739 (0.0005) [2023-12-26 18:32:13,253][105692] Updated weights for policy 0, policy_version 435749 (0.0005) [2023-12-26 18:32:13,612][105620] Updated weights for policy 1, policy_version 436140 (0.0008) [2023-12-26 18:32:13,674][105620] Updated weights for policy 1, policy_version 436150 (0.0005) [2023-12-26 18:32:13,726][105620] Updated weights for policy 1, policy_version 436160 (0.0005) [2023-12-26 18:32:13,787][105692] Updated weights for policy 0, policy_version 435759 (0.0006) [2023-12-26 18:32:13,851][105692] Updated weights for policy 0, policy_version 435769 (0.0005) [2023-12-26 18:32:13,913][105692] Updated weights for policy 0, policy_version 435779 (0.0008) [2023-12-26 18:32:14,352][105620] Updated weights for policy 1, policy_version 436170 (0.0007) [2023-12-26 18:32:14,416][105620] Updated weights for policy 1, policy_version 436180 (0.0008) [2023-12-26 18:32:14,480][105620] Updated weights for policy 1, policy_version 436190 (0.0009) [2023-12-26 18:32:14,548][105620] Updated weights for policy 1, policy_version 436200 (0.0010) [2023-12-26 18:32:14,550][105692] Updated weights for policy 0, policy_version 435789 (0.0007) [2023-12-26 18:32:14,595][105692] Updated weights for policy 0, policy_version 435799 (0.0005) [2023-12-26 18:32:14,657][105692] Updated weights for policy 0, policy_version 435809 (0.0008) [2023-12-26 18:32:15,304][105620] Updated weights for policy 1, policy_version 436210 (0.0006) [2023-12-26 18:32:15,376][105620] Updated weights for policy 1, policy_version 436220 (0.0006) [2023-12-26 18:32:15,378][105692] Updated weights for policy 0, policy_version 435819 (0.0010) [2023-12-26 18:32:15,426][105620] Updated weights for policy 1, policy_version 436230 (0.0011) [2023-12-26 18:32:15,434][105692] Updated weights for policy 0, policy_version 435829 (0.0010) [2023-12-26 18:32:15,490][105692] Updated weights for policy 0, policy_version 435839 (0.0010) [2023-12-26 18:32:16,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 223281152. Throughput: 0: 9934.2, 1: 9720.8. Samples: 223253852. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:32:16,062][104569] Avg episode reward: [(0, '9357.500'), (1, '7325.368')] [2023-12-26 18:32:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000436232_111689728.pth... [2023-12-26 18:32:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000435112_111403008.pth [2023-12-26 18:32:16,077][105692] Updated weights for policy 0, policy_version 435849 (0.0009) [2023-12-26 18:32:16,126][105692] Updated weights for policy 0, policy_version 435859 (0.0006) [2023-12-26 18:32:16,134][105620] Updated weights for policy 1, policy_version 436240 (0.0009) [2023-12-26 18:32:16,186][105692] Updated weights for policy 0, policy_version 435869 (0.0009) [2023-12-26 18:32:16,192][105620] Updated weights for policy 1, policy_version 436250 (0.0007) [2023-12-26 18:32:16,235][105692] Updated weights for policy 0, policy_version 435879 (0.0010) [2023-12-26 18:32:16,239][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000435880_111599616.pth... [2023-12-26 18:32:16,243][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000434696_111296512.pth [2023-12-26 18:32:16,249][105620] Updated weights for policy 1, policy_version 436260 (0.0006) [2023-12-26 18:32:16,916][105620] Updated weights for policy 1, policy_version 436270 (0.0009) [2023-12-26 18:32:16,931][105692] Updated weights for policy 0, policy_version 435889 (0.0010) [2023-12-26 18:32:16,972][105620] Updated weights for policy 1, policy_version 436280 (0.0006) [2023-12-26 18:32:16,989][105692] Updated weights for policy 0, policy_version 435899 (0.0010) [2023-12-26 18:32:17,035][105620] Updated weights for policy 1, policy_version 436290 (0.0007) [2023-12-26 18:32:17,048][105692] Updated weights for policy 0, policy_version 435909 (0.0010) [2023-12-26 18:32:17,683][105620] Updated weights for policy 1, policy_version 436300 (0.0008) [2023-12-26 18:32:17,753][105620] Updated weights for policy 1, policy_version 436310 (0.0007) [2023-12-26 18:32:17,790][105692] Updated weights for policy 0, policy_version 435919 (0.0010) [2023-12-26 18:32:17,813][105620] Updated weights for policy 1, policy_version 436320 (0.0011) [2023-12-26 18:32:17,846][105692] Updated weights for policy 0, policy_version 435929 (0.0010) [2023-12-26 18:32:17,905][105692] Updated weights for policy 0, policy_version 435939 (0.0011) [2023-12-26 18:32:18,413][105620] Updated weights for policy 1, policy_version 436330 (0.0011) [2023-12-26 18:32:18,465][105620] Updated weights for policy 1, policy_version 436340 (0.0011) [2023-12-26 18:32:18,511][105620] Updated weights for policy 1, policy_version 436350 (0.0011) [2023-12-26 18:32:18,564][105620] Updated weights for policy 1, policy_version 436360 (0.0010) [2023-12-26 18:32:18,657][105692] Updated weights for policy 0, policy_version 435949 (0.0011) [2023-12-26 18:32:18,723][105692] Updated weights for policy 0, policy_version 435959 (0.0011) [2023-12-26 18:32:18,789][105692] Updated weights for policy 0, policy_version 435969 (0.0011) [2023-12-26 18:32:19,309][105620] Updated weights for policy 1, policy_version 436370 (0.0011) [2023-12-26 18:32:19,369][105620] Updated weights for policy 1, policy_version 436380 (0.0010) [2023-12-26 18:32:19,438][105620] Updated weights for policy 1, policy_version 436390 (0.0011) [2023-12-26 18:32:19,554][105692] Updated weights for policy 0, policy_version 435979 (0.0010) [2023-12-26 18:32:19,605][105692] Updated weights for policy 0, policy_version 435989 (0.0008) [2023-12-26 18:32:19,656][105692] Updated weights for policy 0, policy_version 435999 (0.0008) [2023-12-26 18:32:20,210][105620] Updated weights for policy 1, policy_version 436400 (0.0010) [2023-12-26 18:32:20,269][105620] Updated weights for policy 1, policy_version 436410 (0.0008) [2023-12-26 18:32:20,320][105620] Updated weights for policy 1, policy_version 436420 (0.0006) [2023-12-26 18:32:20,427][105692] Updated weights for policy 0, policy_version 436009 (0.0007) [2023-12-26 18:32:20,483][105692] Updated weights for policy 0, policy_version 436019 (0.0009) [2023-12-26 18:32:20,545][105692] Updated weights for policy 0, policy_version 436029 (0.0009) [2023-12-26 18:32:20,614][105692] Updated weights for policy 0, policy_version 436039 (0.0009) [2023-12-26 18:32:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 223379456. Throughput: 0: 9944.2, 1: 9742.8. Samples: 223373124. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:32:21,062][104569] Avg episode reward: [(0, '9357.293'), (1, '9135.438')] [2023-12-26 18:32:21,067][105620] Updated weights for policy 1, policy_version 436430 (0.0008) [2023-12-26 18:32:21,130][105620] Updated weights for policy 1, policy_version 436440 (0.0009) [2023-12-26 18:32:21,197][105620] Updated weights for policy 1, policy_version 436450 (0.0009) [2023-12-26 18:32:21,305][105692] Updated weights for policy 0, policy_version 436049 (0.0008) [2023-12-26 18:32:21,374][105692] Updated weights for policy 0, policy_version 436059 (0.0010) [2023-12-26 18:32:21,430][105692] Updated weights for policy 0, policy_version 436069 (0.0009) [2023-12-26 18:32:21,953][105620] Updated weights for policy 1, policy_version 436460 (0.0008) [2023-12-26 18:32:22,009][105620] Updated weights for policy 1, policy_version 436470 (0.0005) [2023-12-26 18:32:22,070][105620] Updated weights for policy 1, policy_version 436480 (0.0006) [2023-12-26 18:32:22,226][105692] Updated weights for policy 0, policy_version 436079 (0.0008) [2023-12-26 18:32:22,292][105692] Updated weights for policy 0, policy_version 436089 (0.0010) [2023-12-26 18:32:22,341][105692] Updated weights for policy 0, policy_version 436099 (0.0009) [2023-12-26 18:32:22,733][105620] Updated weights for policy 1, policy_version 436490 (0.0008) [2023-12-26 18:32:22,799][105620] Updated weights for policy 1, policy_version 436500 (0.0005) [2023-12-26 18:32:22,868][105620] Updated weights for policy 1, policy_version 436510 (0.0005) [2023-12-26 18:32:22,937][105620] Updated weights for policy 1, policy_version 436520 (0.0008) [2023-12-26 18:32:23,068][105692] Updated weights for policy 0, policy_version 436109 (0.0009) [2023-12-26 18:32:23,140][105692] Updated weights for policy 0, policy_version 436119 (0.0009) [2023-12-26 18:32:23,201][105692] Updated weights for policy 0, policy_version 436129 (0.0009) [2023-12-26 18:32:23,558][105620] Updated weights for policy 1, policy_version 436530 (0.0009) [2023-12-26 18:32:23,608][105620] Updated weights for policy 1, policy_version 436540 (0.0009) [2023-12-26 18:32:23,659][105620] Updated weights for policy 1, policy_version 436550 (0.0009) [2023-12-26 18:32:23,944][105692] Updated weights for policy 0, policy_version 436139 (0.0009) [2023-12-26 18:32:24,002][105692] Updated weights for policy 0, policy_version 436149 (0.0009) [2023-12-26 18:32:24,049][105692] Updated weights for policy 0, policy_version 436159 (0.0009) [2023-12-26 18:32:24,453][105620] Updated weights for policy 1, policy_version 436560 (0.0009) [2023-12-26 18:32:24,518][105620] Updated weights for policy 1, policy_version 436570 (0.0007) [2023-12-26 18:32:24,576][105620] Updated weights for policy 1, policy_version 436580 (0.0007) [2023-12-26 18:32:24,734][105692] Updated weights for policy 0, policy_version 436169 (0.0008) [2023-12-26 18:32:24,784][105692] Updated weights for policy 0, policy_version 436179 (0.0005) [2023-12-26 18:32:24,831][105692] Updated weights for policy 0, policy_version 436189 (0.0005) [2023-12-26 18:32:24,884][105692] Updated weights for policy 0, policy_version 436199 (0.0005) [2023-12-26 18:32:25,343][105620] Updated weights for policy 1, policy_version 436590 (0.0009) [2023-12-26 18:32:25,394][105620] Updated weights for policy 1, policy_version 436600 (0.0009) [2023-12-26 18:32:25,458][105620] Updated weights for policy 1, policy_version 436610 (0.0009) [2023-12-26 18:32:25,570][105692] Updated weights for policy 0, policy_version 436209 (0.0008) [2023-12-26 18:32:25,629][105692] Updated weights for policy 0, policy_version 436219 (0.0009) [2023-12-26 18:32:25,684][105692] Updated weights for policy 0, policy_version 436229 (0.0008) [2023-12-26 18:32:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 223477760. Throughput: 0: 9853.9, 1: 9714.9. Samples: 223487180. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:32:26,063][104569] Avg episode reward: [(0, '9354.762'), (1, '9135.287')] [2023-12-26 18:32:26,221][105620] Updated weights for policy 1, policy_version 436620 (0.0009) [2023-12-26 18:32:26,269][105620] Updated weights for policy 1, policy_version 436630 (0.0010) [2023-12-26 18:32:26,317][105620] Updated weights for policy 1, policy_version 436640 (0.0010) [2023-12-26 18:32:26,450][105692] Updated weights for policy 0, policy_version 436239 (0.0008) [2023-12-26 18:32:26,519][105692] Updated weights for policy 0, policy_version 436249 (0.0009) [2023-12-26 18:32:26,572][105692] Updated weights for policy 0, policy_version 436259 (0.0009) [2023-12-26 18:32:27,050][105620] Updated weights for policy 1, policy_version 436650 (0.0010) [2023-12-26 18:32:27,098][105620] Updated weights for policy 1, policy_version 436660 (0.0007) [2023-12-26 18:32:27,155][105620] Updated weights for policy 1, policy_version 436670 (0.0008) [2023-12-26 18:32:27,197][105692] Updated weights for policy 0, policy_version 436269 (0.0007) [2023-12-26 18:32:27,203][105620] Updated weights for policy 1, policy_version 436680 (0.0007) [2023-12-26 18:32:27,256][105692] Updated weights for policy 0, policy_version 436279 (0.0005) [2023-12-26 18:32:27,320][105692] Updated weights for policy 0, policy_version 436289 (0.0006) [2023-12-26 18:32:27,838][105692] Updated weights for policy 0, policy_version 436299 (0.0006) [2023-12-26 18:32:27,897][105692] Updated weights for policy 0, policy_version 436309 (0.0005) [2023-12-26 18:32:27,964][105692] Updated weights for policy 0, policy_version 436319 (0.0005) [2023-12-26 18:32:28,033][105620] Updated weights for policy 1, policy_version 436690 (0.0006) [2023-12-26 18:32:28,080][105620] Updated weights for policy 1, policy_version 436700 (0.0005) [2023-12-26 18:32:28,127][105620] Updated weights for policy 1, policy_version 436710 (0.0007) [2023-12-26 18:32:28,628][105692] Updated weights for policy 0, policy_version 436329 (0.0008) [2023-12-26 18:32:28,678][105692] Updated weights for policy 0, policy_version 436339 (0.0009) [2023-12-26 18:32:28,735][105692] Updated weights for policy 0, policy_version 436349 (0.0009) [2023-12-26 18:32:28,792][105692] Updated weights for policy 0, policy_version 436359 (0.0008) [2023-12-26 18:32:28,846][105620] Updated weights for policy 1, policy_version 436720 (0.0007) [2023-12-26 18:32:28,901][105620] Updated weights for policy 1, policy_version 436730 (0.0005) [2023-12-26 18:32:28,958][105620] Updated weights for policy 1, policy_version 436740 (0.0006) [2023-12-26 18:32:29,556][105692] Updated weights for policy 0, policy_version 436369 (0.0010) [2023-12-26 18:32:29,619][105692] Updated weights for policy 0, policy_version 436379 (0.0010) [2023-12-26 18:32:29,672][105692] Updated weights for policy 0, policy_version 436389 (0.0009) [2023-12-26 18:32:29,709][105620] Updated weights for policy 1, policy_version 436750 (0.0007) [2023-12-26 18:32:29,761][105620] Updated weights for policy 1, policy_version 436760 (0.0011) [2023-12-26 18:32:29,815][105620] Updated weights for policy 1, policy_version 436770 (0.0007) [2023-12-26 18:32:30,437][105692] Updated weights for policy 0, policy_version 436399 (0.0007) [2023-12-26 18:32:30,489][105692] Updated weights for policy 0, policy_version 436409 (0.0008) [2023-12-26 18:32:30,538][105692] Updated weights for policy 0, policy_version 436419 (0.0008) [2023-12-26 18:32:30,590][105620] Updated weights for policy 1, policy_version 436780 (0.0009) [2023-12-26 18:32:30,655][105620] Updated weights for policy 1, policy_version 436790 (0.0010) [2023-12-26 18:32:30,719][105620] Updated weights for policy 1, policy_version 436800 (0.0010) [2023-12-26 18:32:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 223576064. Throughput: 0: 9953.8, 1: 9722.9. Samples: 223547428. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:32:31,062][104569] Avg episode reward: [(0, '9263.079'), (1, '9084.188')] [2023-12-26 18:32:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000436424_111738880.pth... [2023-12-26 18:32:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000436808_111837184.pth... [2023-12-26 18:32:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000435272_111443968.pth [2023-12-26 18:32:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000435688_111550464.pth [2023-12-26 18:32:31,344][105692] Updated weights for policy 0, policy_version 436429 (0.0007) [2023-12-26 18:32:31,410][105692] Updated weights for policy 0, policy_version 436439 (0.0008) [2023-12-26 18:32:31,421][105620] Updated weights for policy 1, policy_version 436810 (0.0008) [2023-12-26 18:32:31,471][105620] Updated weights for policy 1, policy_version 436820 (0.0005) [2023-12-26 18:32:31,472][105692] Updated weights for policy 0, policy_version 436449 (0.0008) [2023-12-26 18:32:31,524][105620] Updated weights for policy 1, policy_version 436830 (0.0006) [2023-12-26 18:32:32,158][105620] Updated weights for policy 1, policy_version 436841 (0.0010) [2023-12-26 18:32:32,208][105620] Updated weights for policy 1, policy_version 436851 (0.0005) [2023-12-26 18:32:32,262][105620] Updated weights for policy 1, policy_version 436861 (0.0005) [2023-12-26 18:32:32,270][105692] Updated weights for policy 0, policy_version 436459 (0.0008) [2023-12-26 18:32:32,323][105620] Updated weights for policy 1, policy_version 436871 (0.0006) [2023-12-26 18:32:32,327][105692] Updated weights for policy 0, policy_version 436469 (0.0009) [2023-12-26 18:32:32,387][105692] Updated weights for policy 0, policy_version 436479 (0.0009) [2023-12-26 18:32:32,889][105620] Updated weights for policy 1, policy_version 436881 (0.0008) [2023-12-26 18:32:32,947][105620] Updated weights for policy 1, policy_version 436891 (0.0009) [2023-12-26 18:32:33,006][105620] Updated weights for policy 1, policy_version 436901 (0.0009) [2023-12-26 18:32:33,195][105692] Updated weights for policy 0, policy_version 436489 (0.0009) [2023-12-26 18:32:33,249][105692] Updated weights for policy 0, policy_version 436499 (0.0009) [2023-12-26 18:32:33,311][105692] Updated weights for policy 0, policy_version 436509 (0.0009) [2023-12-26 18:32:33,372][105692] Updated weights for policy 0, policy_version 436519 (0.0009) [2023-12-26 18:32:33,682][105620] Updated weights for policy 1, policy_version 436911 (0.0009) [2023-12-26 18:32:33,733][105620] Updated weights for policy 1, policy_version 436921 (0.0009) [2023-12-26 18:32:33,782][105620] Updated weights for policy 1, policy_version 436931 (0.0009) [2023-12-26 18:32:34,184][105692] Updated weights for policy 0, policy_version 436529 (0.0008) [2023-12-26 18:32:34,247][105692] Updated weights for policy 0, policy_version 436539 (0.0008) [2023-12-26 18:32:34,306][105692] Updated weights for policy 0, policy_version 436549 (0.0006) [2023-12-26 18:32:34,495][105620] Updated weights for policy 1, policy_version 436941 (0.0009) [2023-12-26 18:32:34,558][105620] Updated weights for policy 1, policy_version 436951 (0.0007) [2023-12-26 18:32:34,623][105620] Updated weights for policy 1, policy_version 436961 (0.0006) [2023-12-26 18:32:34,921][105692] Updated weights for policy 0, policy_version 436559 (0.0008) [2023-12-26 18:32:34,973][105692] Updated weights for policy 0, policy_version 436570 (0.0008) [2023-12-26 18:32:35,029][105692] Updated weights for policy 0, policy_version 436580 (0.0011) [2023-12-26 18:32:35,296][105620] Updated weights for policy 1, policy_version 436971 (0.0006) [2023-12-26 18:32:35,345][105620] Updated weights for policy 1, policy_version 436981 (0.0005) [2023-12-26 18:32:35,398][105620] Updated weights for policy 1, policy_version 436991 (0.0005) [2023-12-26 18:32:35,838][105692] Updated weights for policy 0, policy_version 436590 (0.0010) [2023-12-26 18:32:35,895][105692] Updated weights for policy 0, policy_version 436600 (0.0008) [2023-12-26 18:32:35,953][105692] Updated weights for policy 0, policy_version 436610 (0.0008) [2023-12-26 18:32:35,968][105620] Updated weights for policy 1, policy_version 437001 (0.0005) [2023-12-26 18:32:36,028][105620] Updated weights for policy 1, policy_version 437011 (0.0007) [2023-12-26 18:32:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 223674368. Throughput: 0: 9887.0, 1: 9683.8. Samples: 223663196. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:32:36,062][104569] Avg episode reward: [(0, '9263.191'), (1, '8991.537')] [2023-12-26 18:32:36,075][105620] Updated weights for policy 1, policy_version 437021 (0.0007) [2023-12-26 18:32:36,142][105620] Updated weights for policy 1, policy_version 437031 (0.0008) [2023-12-26 18:32:36,769][105620] Updated weights for policy 1, policy_version 437041 (0.0009) [2023-12-26 18:32:36,789][105692] Updated weights for policy 0, policy_version 436620 (0.0009) [2023-12-26 18:32:36,816][105620] Updated weights for policy 1, policy_version 437051 (0.0007) [2023-12-26 18:32:36,844][105692] Updated weights for policy 0, policy_version 436630 (0.0007) [2023-12-26 18:32:36,868][105620] Updated weights for policy 1, policy_version 437061 (0.0008) [2023-12-26 18:32:36,904][105692] Updated weights for policy 0, policy_version 436640 (0.0006) [2023-12-26 18:32:37,481][105692] Updated weights for policy 0, policy_version 436650 (0.0006) [2023-12-26 18:32:37,539][105692] Updated weights for policy 0, policy_version 436660 (0.0009) [2023-12-26 18:32:37,594][105692] Updated weights for policy 0, policy_version 436670 (0.0007) [2023-12-26 18:32:37,649][105692] Updated weights for policy 0, policy_version 436680 (0.0007) [2023-12-26 18:32:37,734][105620] Updated weights for policy 1, policy_version 437071 (0.0009) [2023-12-26 18:32:37,794][105620] Updated weights for policy 1, policy_version 437081 (0.0009) [2023-12-26 18:32:37,843][105620] Updated weights for policy 1, policy_version 437091 (0.0009) [2023-12-26 18:32:38,361][105692] Updated weights for policy 0, policy_version 436690 (0.0008) [2023-12-26 18:32:38,417][105692] Updated weights for policy 0, policy_version 436700 (0.0006) [2023-12-26 18:32:38,473][105692] Updated weights for policy 0, policy_version 436710 (0.0005) [2023-12-26 18:32:38,670][105620] Updated weights for policy 1, policy_version 437101 (0.0009) [2023-12-26 18:32:38,723][105620] Updated weights for policy 1, policy_version 437112 (0.0010) [2023-12-26 18:32:38,780][105620] Updated weights for policy 1, policy_version 437123 (0.0009) [2023-12-26 18:32:39,012][105692] Updated weights for policy 0, policy_version 436720 (0.0006) [2023-12-26 18:32:39,078][105692] Updated weights for policy 0, policy_version 436730 (0.0009) [2023-12-26 18:32:39,134][105692] Updated weights for policy 0, policy_version 436740 (0.0009) [2023-12-26 18:32:39,556][105620] Updated weights for policy 1, policy_version 437133 (0.0009) [2023-12-26 18:32:39,615][105620] Updated weights for policy 1, policy_version 437143 (0.0009) [2023-12-26 18:32:39,672][105620] Updated weights for policy 1, policy_version 437153 (0.0009) [2023-12-26 18:32:39,889][105692] Updated weights for policy 0, policy_version 436750 (0.0008) [2023-12-26 18:32:39,950][105692] Updated weights for policy 0, policy_version 436760 (0.0009) [2023-12-26 18:32:40,013][105692] Updated weights for policy 0, policy_version 436770 (0.0009) [2023-12-26 18:32:40,468][105620] Updated weights for policy 1, policy_version 437163 (0.0008) [2023-12-26 18:32:40,539][105620] Updated weights for policy 1, policy_version 437173 (0.0010) [2023-12-26 18:32:40,592][105620] Updated weights for policy 1, policy_version 437183 (0.0010) [2023-12-26 18:32:40,741][105692] Updated weights for policy 0, policy_version 436780 (0.0009) [2023-12-26 18:32:40,792][105692] Updated weights for policy 0, policy_version 436790 (0.0010) [2023-12-26 18:32:40,847][105692] Updated weights for policy 0, policy_version 436800 (0.0010) [2023-12-26 18:32:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 223772672. Throughput: 0: 9811.0, 1: 9720.7. Samples: 223778732. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:32:41,063][104569] Avg episode reward: [(0, '9174.542'), (1, '8772.872')] [2023-12-26 18:32:41,391][105620] Updated weights for policy 1, policy_version 437193 (0.0010) [2023-12-26 18:32:41,455][105620] Updated weights for policy 1, policy_version 437203 (0.0008) [2023-12-26 18:32:41,511][105620] Updated weights for policy 1, policy_version 437213 (0.0008) [2023-12-26 18:32:41,564][105620] Updated weights for policy 1, policy_version 437223 (0.0008) [2023-12-26 18:32:41,613][105692] Updated weights for policy 0, policy_version 436810 (0.0011) [2023-12-26 18:32:41,674][105692] Updated weights for policy 0, policy_version 436820 (0.0010) [2023-12-26 18:32:41,742][105692] Updated weights for policy 0, policy_version 436830 (0.0008) [2023-12-26 18:32:41,807][105692] Updated weights for policy 0, policy_version 436840 (0.0008) [2023-12-26 18:32:42,366][105620] Updated weights for policy 1, policy_version 437233 (0.0008) [2023-12-26 18:32:42,430][105620] Updated weights for policy 1, policy_version 437243 (0.0007) [2023-12-26 18:32:42,495][105620] Updated weights for policy 1, policy_version 437253 (0.0008) [2023-12-26 18:32:42,523][105692] Updated weights for policy 0, policy_version 436850 (0.0008) [2023-12-26 18:32:42,585][105692] Updated weights for policy 0, policy_version 436860 (0.0008) [2023-12-26 18:32:42,648][105692] Updated weights for policy 0, policy_version 436870 (0.0011) [2023-12-26 18:32:43,235][105692] Updated weights for policy 0, policy_version 436880 (0.0007) [2023-12-26 18:32:43,279][105620] Updated weights for policy 1, policy_version 437263 (0.0008) [2023-12-26 18:32:43,290][105692] Updated weights for policy 0, policy_version 436890 (0.0006) [2023-12-26 18:32:43,324][105620] Updated weights for policy 1, policy_version 437273 (0.0008) [2023-12-26 18:32:43,357][105692] Updated weights for policy 0, policy_version 436900 (0.0006) [2023-12-26 18:32:43,377][105620] Updated weights for policy 1, policy_version 437283 (0.0008) [2023-12-26 18:32:44,014][105692] Updated weights for policy 0, policy_version 436910 (0.0005) [2023-12-26 18:32:44,068][105692] Updated weights for policy 0, policy_version 436920 (0.0006) [2023-12-26 18:32:44,129][105692] Updated weights for policy 0, policy_version 436930 (0.0008) [2023-12-26 18:32:44,203][105620] Updated weights for policy 1, policy_version 437293 (0.0010) [2023-12-26 18:32:44,250][105620] Updated weights for policy 1, policy_version 437303 (0.0008) [2023-12-26 18:32:44,302][105620] Updated weights for policy 1, policy_version 437313 (0.0009) [2023-12-26 18:32:44,758][105692] Updated weights for policy 0, policy_version 436940 (0.0008) [2023-12-26 18:32:44,833][105692] Updated weights for policy 0, policy_version 436950 (0.0007) [2023-12-26 18:32:44,891][105692] Updated weights for policy 0, policy_version 436960 (0.0009) [2023-12-26 18:32:45,139][105620] Updated weights for policy 1, policy_version 437323 (0.0009) [2023-12-26 18:32:45,192][105620] Updated weights for policy 1, policy_version 437333 (0.0009) [2023-12-26 18:32:45,247][105620] Updated weights for policy 1, policy_version 437343 (0.0009) [2023-12-26 18:32:45,576][105692] Updated weights for policy 0, policy_version 436970 (0.0007) [2023-12-26 18:32:45,630][105692] Updated weights for policy 0, policy_version 436980 (0.0005) [2023-12-26 18:32:45,693][105692] Updated weights for policy 0, policy_version 436990 (0.0007) [2023-12-26 18:32:45,745][105692] Updated weights for policy 0, policy_version 437000 (0.0009) [2023-12-26 18:32:46,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19387.6, 300 sec: 19605.2). Total num frames: 223862784. Throughput: 0: 9748.3, 1: 9625.2. Samples: 223835804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:32:46,063][104569] Avg episode reward: [(0, '8994.145'), (1, '8383.407')] [2023-12-26 18:32:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000437000_111886336.pth... [2023-12-26 18:32:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000437352_111976448.pth... [2023-12-26 18:32:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000435880_111599616.pth [2023-12-26 18:32:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000436232_111689728.pth [2023-12-26 18:32:46,118][105620] Updated weights for policy 1, policy_version 437353 (0.0009) [2023-12-26 18:32:46,178][105620] Updated weights for policy 1, policy_version 437363 (0.0008) [2023-12-26 18:32:46,224][105620] Updated weights for policy 1, policy_version 437373 (0.0008) [2023-12-26 18:32:46,281][105620] Updated weights for policy 1, policy_version 437383 (0.0008) [2023-12-26 18:32:46,311][105692] Updated weights for policy 0, policy_version 437010 (0.0008) [2023-12-26 18:32:46,358][105692] Updated weights for policy 0, policy_version 437020 (0.0009) [2023-12-26 18:32:46,412][105692] Updated weights for policy 0, policy_version 437030 (0.0008) [2023-12-26 18:32:47,083][105620] Updated weights for policy 1, policy_version 437393 (0.0009) [2023-12-26 18:32:47,143][105620] Updated weights for policy 1, policy_version 437403 (0.0009) [2023-12-26 18:32:47,156][105692] Updated weights for policy 0, policy_version 437040 (0.0007) [2023-12-26 18:32:47,203][105620] Updated weights for policy 1, policy_version 437413 (0.0009) [2023-12-26 18:32:47,209][105692] Updated weights for policy 0, policy_version 437050 (0.0006) [2023-12-26 18:32:47,270][105692] Updated weights for policy 0, policy_version 437060 (0.0008) [2023-12-26 18:32:47,983][105692] Updated weights for policy 0, policy_version 437070 (0.0007) [2023-12-26 18:32:47,984][105620] Updated weights for policy 1, policy_version 437423 (0.0009) [2023-12-26 18:32:48,049][105692] Updated weights for policy 0, policy_version 437080 (0.0007) [2023-12-26 18:32:48,050][105620] Updated weights for policy 1, policy_version 437433 (0.0008) [2023-12-26 18:32:48,109][105692] Updated weights for policy 0, policy_version 437090 (0.0006) [2023-12-26 18:32:48,110][105620] Updated weights for policy 1, policy_version 437443 (0.0007) [2023-12-26 18:32:48,764][105620] Updated weights for policy 1, policy_version 437453 (0.0008) [2023-12-26 18:32:48,826][105620] Updated weights for policy 1, policy_version 437463 (0.0009) [2023-12-26 18:32:48,876][105620] Updated weights for policy 1, policy_version 437473 (0.0009) [2023-12-26 18:32:48,930][105692] Updated weights for policy 0, policy_version 437100 (0.0007) [2023-12-26 18:32:48,983][105692] Updated weights for policy 0, policy_version 437110 (0.0009) [2023-12-26 18:32:49,034][105692] Updated weights for policy 0, policy_version 437120 (0.0009) [2023-12-26 18:32:49,698][105620] Updated weights for policy 1, policy_version 437483 (0.0009) [2023-12-26 18:32:49,756][105620] Updated weights for policy 1, policy_version 437493 (0.0009) [2023-12-26 18:32:49,802][105692] Updated weights for policy 0, policy_version 437130 (0.0009) [2023-12-26 18:32:49,804][105620] Updated weights for policy 1, policy_version 437503 (0.0008) [2023-12-26 18:32:49,869][105692] Updated weights for policy 0, policy_version 437140 (0.0007) [2023-12-26 18:32:49,932][105692] Updated weights for policy 0, policy_version 437150 (0.0009) [2023-12-26 18:32:49,994][105692] Updated weights for policy 0, policy_version 437160 (0.0008) [2023-12-26 18:32:50,594][105620] Updated weights for policy 1, policy_version 437513 (0.0009) [2023-12-26 18:32:50,657][105620] Updated weights for policy 1, policy_version 437523 (0.0009) [2023-12-26 18:32:50,715][105620] Updated weights for policy 1, policy_version 437533 (0.0009) [2023-12-26 18:32:50,742][105692] Updated weights for policy 0, policy_version 437170 (0.0006) [2023-12-26 18:32:50,780][105620] Updated weights for policy 1, policy_version 437543 (0.0009) [2023-12-26 18:32:50,795][105692] Updated weights for policy 0, policy_version 437180 (0.0005) [2023-12-26 18:32:50,852][105692] Updated weights for policy 0, policy_version 437190 (0.0005) [2023-12-26 18:32:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 223961088. Throughput: 0: 9827.9, 1: 9496.7. Samples: 223948568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:32:51,063][104569] Avg episode reward: [(0, '9174.044'), (1, '7567.264')] [2023-12-26 18:32:51,506][105692] Updated weights for policy 0, policy_version 437200 (0.0008) [2023-12-26 18:32:51,562][105692] Updated weights for policy 0, policy_version 437210 (0.0005) [2023-12-26 18:32:51,618][105692] Updated weights for policy 0, policy_version 437220 (0.0008) [2023-12-26 18:32:51,619][105620] Updated weights for policy 1, policy_version 437553 (0.0008) [2023-12-26 18:32:51,666][105620] Updated weights for policy 1, policy_version 437563 (0.0009) [2023-12-26 18:32:51,722][105620] Updated weights for policy 1, policy_version 437573 (0.0008) [2023-12-26 18:32:52,249][105692] Updated weights for policy 0, policy_version 437230 (0.0007) [2023-12-26 18:32:52,311][105692] Updated weights for policy 0, policy_version 437240 (0.0008) [2023-12-26 18:32:52,370][105692] Updated weights for policy 0, policy_version 437250 (0.0008) [2023-12-26 18:32:52,576][105620] Updated weights for policy 1, policy_version 437583 (0.0008) [2023-12-26 18:32:52,640][105620] Updated weights for policy 1, policy_version 437593 (0.0009) [2023-12-26 18:32:52,702][105620] Updated weights for policy 1, policy_version 437603 (0.0009) [2023-12-26 18:32:53,035][105692] Updated weights for policy 0, policy_version 437260 (0.0009) [2023-12-26 18:32:53,093][105692] Updated weights for policy 0, policy_version 437270 (0.0009) [2023-12-26 18:32:53,147][105692] Updated weights for policy 0, policy_version 437280 (0.0012) [2023-12-26 18:32:53,395][105620] Updated weights for policy 1, policy_version 437613 (0.0008) [2023-12-26 18:32:53,447][105620] Updated weights for policy 1, policy_version 437623 (0.0010) [2023-12-26 18:32:53,499][105620] Updated weights for policy 1, policy_version 437633 (0.0009) [2023-12-26 18:32:53,779][105692] Updated weights for policy 0, policy_version 437291 (0.0008) [2023-12-26 18:32:53,834][105692] Updated weights for policy 0, policy_version 437301 (0.0005) [2023-12-26 18:32:53,886][105692] Updated weights for policy 0, policy_version 437311 (0.0005) [2023-12-26 18:32:54,403][105620] Updated weights for policy 1, policy_version 437643 (0.0010) [2023-12-26 18:32:54,451][105620] Updated weights for policy 1, policy_version 437653 (0.0009) [2023-12-26 18:32:54,459][105692] Updated weights for policy 0, policy_version 437321 (0.0005) [2023-12-26 18:32:54,497][105620] Updated weights for policy 1, policy_version 437663 (0.0007) [2023-12-26 18:32:54,516][105692] Updated weights for policy 0, policy_version 437331 (0.0008) [2023-12-26 18:32:54,571][105692] Updated weights for policy 0, policy_version 437341 (0.0007) [2023-12-26 18:32:54,622][105692] Updated weights for policy 0, policy_version 437351 (0.0008) [2023-12-26 18:32:55,292][105620] Updated weights for policy 1, policy_version 437673 (0.0008) [2023-12-26 18:32:55,349][105692] Updated weights for policy 0, policy_version 437361 (0.0006) [2023-12-26 18:32:55,352][105620] Updated weights for policy 1, policy_version 437683 (0.0008) [2023-12-26 18:32:55,396][105692] Updated weights for policy 0, policy_version 437371 (0.0005) [2023-12-26 18:32:55,405][105620] Updated weights for policy 1, policy_version 437693 (0.0008) [2023-12-26 18:32:55,444][105692] Updated weights for policy 0, policy_version 437381 (0.0007) [2023-12-26 18:32:55,461][105620] Updated weights for policy 1, policy_version 437703 (0.0009) [2023-12-26 18:32:56,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 224051200. Throughput: 0: 9915.7, 1: 9411.9. Samples: 224064036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:32:56,063][104569] Avg episode reward: [(0, '9354.371'), (1, '7366.288')] [2023-12-26 18:32:56,176][105692] Updated weights for policy 0, policy_version 437391 (0.0006) [2023-12-26 18:32:56,227][105692] Updated weights for policy 0, policy_version 437401 (0.0005) [2023-12-26 18:32:56,237][105620] Updated weights for policy 1, policy_version 437713 (0.0008) [2023-12-26 18:32:56,275][105692] Updated weights for policy 0, policy_version 437411 (0.0005) [2023-12-26 18:32:56,289][105620] Updated weights for policy 1, policy_version 437723 (0.0008) [2023-12-26 18:32:56,342][105620] Updated weights for policy 1, policy_version 437733 (0.0009) [2023-12-26 18:32:56,847][105692] Updated weights for policy 0, policy_version 437421 (0.0005) [2023-12-26 18:32:56,897][105692] Updated weights for policy 0, policy_version 437431 (0.0005) [2023-12-26 18:32:56,951][105692] Updated weights for policy 0, policy_version 437441 (0.0006) [2023-12-26 18:32:57,209][105620] Updated weights for policy 1, policy_version 437743 (0.0008) [2023-12-26 18:32:57,268][105620] Updated weights for policy 1, policy_version 437753 (0.0008) [2023-12-26 18:32:57,326][105620] Updated weights for policy 1, policy_version 437763 (0.0008) [2023-12-26 18:32:57,662][105692] Updated weights for policy 0, policy_version 437451 (0.0010) [2023-12-26 18:32:57,721][105692] Updated weights for policy 0, policy_version 437461 (0.0010) [2023-12-26 18:32:57,775][105692] Updated weights for policy 0, policy_version 437471 (0.0010) [2023-12-26 18:32:58,078][105620] Updated weights for policy 1, policy_version 437773 (0.0008) [2023-12-26 18:32:58,139][105620] Updated weights for policy 1, policy_version 437783 (0.0008) [2023-12-26 18:32:58,202][105620] Updated weights for policy 1, policy_version 437793 (0.0006) [2023-12-26 18:32:58,555][105692] Updated weights for policy 0, policy_version 437481 (0.0010) [2023-12-26 18:32:58,617][105692] Updated weights for policy 0, policy_version 437491 (0.0009) [2023-12-26 18:32:58,679][105692] Updated weights for policy 0, policy_version 437501 (0.0009) [2023-12-26 18:32:58,739][105692] Updated weights for policy 0, policy_version 437511 (0.0009) [2023-12-26 18:32:58,996][105620] Updated weights for policy 1, policy_version 437803 (0.0006) [2023-12-26 18:32:59,046][105620] Updated weights for policy 1, policy_version 437813 (0.0008) [2023-12-26 18:32:59,091][105620] Updated weights for policy 1, policy_version 437823 (0.0008) [2023-12-26 18:32:59,532][105692] Updated weights for policy 0, policy_version 437521 (0.0008) [2023-12-26 18:32:59,597][105692] Updated weights for policy 0, policy_version 437531 (0.0006) [2023-12-26 18:32:59,670][105692] Updated weights for policy 0, policy_version 437541 (0.0006) [2023-12-26 18:32:59,802][105620] Updated weights for policy 1, policy_version 437833 (0.0008) [2023-12-26 18:32:59,864][105620] Updated weights for policy 1, policy_version 437843 (0.0009) [2023-12-26 18:32:59,912][105620] Updated weights for policy 1, policy_version 437853 (0.0008) [2023-12-26 18:32:59,974][105620] Updated weights for policy 1, policy_version 437863 (0.0008) [2023-12-26 18:33:00,373][105692] Updated weights for policy 0, policy_version 437551 (0.0010) [2023-12-26 18:33:00,433][105692] Updated weights for policy 0, policy_version 437561 (0.0009) [2023-12-26 18:33:00,489][105692] Updated weights for policy 0, policy_version 437571 (0.0009) [2023-12-26 18:33:00,722][105620] Updated weights for policy 1, policy_version 437873 (0.0006) [2023-12-26 18:33:00,775][105620] Updated weights for policy 1, policy_version 437883 (0.0009) [2023-12-26 18:33:00,822][105620] Updated weights for policy 1, policy_version 437893 (0.0009) [2023-12-26 18:33:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 224149504. Throughput: 0: 9885.6, 1: 9381.9. Samples: 224120888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:33:01,062][104569] Avg episode reward: [(0, '9265.068'), (1, '7985.015')] [2023-12-26 18:33:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000437576_112033792.pth... [2023-12-26 18:33:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000437896_112115712.pth... [2023-12-26 18:33:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000436424_111738880.pth [2023-12-26 18:33:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000436808_111837184.pth [2023-12-26 18:33:01,263][105692] Updated weights for policy 0, policy_version 437581 (0.0008) [2023-12-26 18:33:01,322][105692] Updated weights for policy 0, policy_version 437591 (0.0009) [2023-12-26 18:33:01,385][105692] Updated weights for policy 0, policy_version 437601 (0.0009) [2023-12-26 18:33:01,469][105620] Updated weights for policy 1, policy_version 437903 (0.0007) [2023-12-26 18:33:01,528][105620] Updated weights for policy 1, policy_version 437913 (0.0010) [2023-12-26 18:33:01,583][105620] Updated weights for policy 1, policy_version 437923 (0.0010) [2023-12-26 18:33:02,215][105620] Updated weights for policy 1, policy_version 437933 (0.0009) [2023-12-26 18:33:02,216][105692] Updated weights for policy 0, policy_version 437611 (0.0009) [2023-12-26 18:33:02,273][105620] Updated weights for policy 1, policy_version 437943 (0.0009) [2023-12-26 18:33:02,277][105692] Updated weights for policy 0, policy_version 437621 (0.0009) [2023-12-26 18:33:02,332][105620] Updated weights for policy 1, policy_version 437953 (0.0005) [2023-12-26 18:33:02,341][105692] Updated weights for policy 0, policy_version 437631 (0.0009) [2023-12-26 18:33:03,040][105620] Updated weights for policy 1, policy_version 437963 (0.0008) [2023-12-26 18:33:03,099][105620] Updated weights for policy 1, policy_version 437973 (0.0006) [2023-12-26 18:33:03,138][105692] Updated weights for policy 0, policy_version 437641 (0.0008) [2023-12-26 18:33:03,162][105620] Updated weights for policy 1, policy_version 437983 (0.0010) [2023-12-26 18:33:03,197][105692] Updated weights for policy 0, policy_version 437651 (0.0009) [2023-12-26 18:33:03,255][105692] Updated weights for policy 0, policy_version 437661 (0.0009) [2023-12-26 18:33:03,308][105692] Updated weights for policy 0, policy_version 437671 (0.0008) [2023-12-26 18:33:03,817][105620] Updated weights for policy 1, policy_version 437993 (0.0010) [2023-12-26 18:33:03,883][105620] Updated weights for policy 1, policy_version 438003 (0.0009) [2023-12-26 18:33:03,944][105620] Updated weights for policy 1, policy_version 438013 (0.0010) [2023-12-26 18:33:03,980][105692] Updated weights for policy 0, policy_version 437681 (0.0010) [2023-12-26 18:33:04,005][105620] Updated weights for policy 1, policy_version 438024 (0.0011) [2023-12-26 18:33:04,042][105692] Updated weights for policy 0, policy_version 437691 (0.0011) [2023-12-26 18:33:04,102][105692] Updated weights for policy 0, policy_version 437701 (0.0008) [2023-12-26 18:33:04,716][105692] Updated weights for policy 0, policy_version 437711 (0.0009) [2023-12-26 18:33:04,750][105620] Updated weights for policy 1, policy_version 438034 (0.0009) [2023-12-26 18:33:04,770][105692] Updated weights for policy 0, policy_version 437721 (0.0010) [2023-12-26 18:33:04,816][105620] Updated weights for policy 1, policy_version 438044 (0.0006) [2023-12-26 18:33:04,830][105692] Updated weights for policy 0, policy_version 437731 (0.0005) [2023-12-26 18:33:04,880][105620] Updated weights for policy 1, policy_version 438054 (0.0005) [2023-12-26 18:33:05,379][105692] Updated weights for policy 0, policy_version 437741 (0.0008) [2023-12-26 18:33:05,430][105692] Updated weights for policy 0, policy_version 437751 (0.0007) [2023-12-26 18:33:05,438][105620] Updated weights for policy 1, policy_version 438064 (0.0007) [2023-12-26 18:33:05,481][105692] Updated weights for policy 0, policy_version 437761 (0.0005) [2023-12-26 18:33:05,502][105620] Updated weights for policy 1, policy_version 438074 (0.0008) [2023-12-26 18:33:05,569][105620] Updated weights for policy 1, policy_version 438084 (0.0006) [2023-12-26 18:33:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 224247808. Throughput: 0: 9801.8, 1: 9394.2. Samples: 224236944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:33:06,063][104569] Avg episode reward: [(0, '9085.124'), (1, '9029.013')] [2023-12-26 18:33:06,101][105620] Updated weights for policy 1, policy_version 438094 (0.0009) [2023-12-26 18:33:06,120][105692] Updated weights for policy 0, policy_version 437771 (0.0006) [2023-12-26 18:33:06,164][105620] Updated weights for policy 1, policy_version 438104 (0.0006) [2023-12-26 18:33:06,173][105692] Updated weights for policy 0, policy_version 437781 (0.0009) [2023-12-26 18:33:06,227][105692] Updated weights for policy 0, policy_version 437791 (0.0011) [2023-12-26 18:33:06,230][105620] Updated weights for policy 1, policy_version 438114 (0.0010) [2023-12-26 18:33:06,942][105620] Updated weights for policy 1, policy_version 438124 (0.0009) [2023-12-26 18:33:06,972][105692] Updated weights for policy 0, policy_version 437801 (0.0010) [2023-12-26 18:33:06,994][105620] Updated weights for policy 1, policy_version 438134 (0.0010) [2023-12-26 18:33:07,023][105692] Updated weights for policy 0, policy_version 437811 (0.0005) [2023-12-26 18:33:07,042][105620] Updated weights for policy 1, policy_version 438144 (0.0009) [2023-12-26 18:33:07,079][105692] Updated weights for policy 0, policy_version 437821 (0.0007) [2023-12-26 18:33:07,141][105692] Updated weights for policy 0, policy_version 437831 (0.0008) [2023-12-26 18:33:07,754][105620] Updated weights for policy 1, policy_version 438154 (0.0007) [2023-12-26 18:33:07,811][105620] Updated weights for policy 1, policy_version 438164 (0.0006) [2023-12-26 18:33:07,842][105692] Updated weights for policy 0, policy_version 437841 (0.0009) [2023-12-26 18:33:07,869][105620] Updated weights for policy 1, policy_version 438174 (0.0007) [2023-12-26 18:33:07,902][105692] Updated weights for policy 0, policy_version 437851 (0.0005) [2023-12-26 18:33:07,924][105620] Updated weights for policy 1, policy_version 438184 (0.0009) [2023-12-26 18:33:07,963][105692] Updated weights for policy 0, policy_version 437861 (0.0005) [2023-12-26 18:33:08,630][105620] Updated weights for policy 1, policy_version 438194 (0.0009) [2023-12-26 18:33:08,692][105692] Updated weights for policy 0, policy_version 437871 (0.0009) [2023-12-26 18:33:08,693][105620] Updated weights for policy 1, policy_version 438204 (0.0011) [2023-12-26 18:33:08,745][105620] Updated weights for policy 1, policy_version 438214 (0.0010) [2023-12-26 18:33:08,749][105692] Updated weights for policy 0, policy_version 437881 (0.0011) [2023-12-26 18:33:08,807][105692] Updated weights for policy 0, policy_version 437891 (0.0011) [2023-12-26 18:33:09,392][105620] Updated weights for policy 1, policy_version 438224 (0.0010) [2023-12-26 18:33:09,460][105620] Updated weights for policy 1, policy_version 438234 (0.0010) [2023-12-26 18:33:09,527][105620] Updated weights for policy 1, policy_version 438244 (0.0009) [2023-12-26 18:33:09,573][105692] Updated weights for policy 0, policy_version 437901 (0.0009) [2023-12-26 18:33:09,621][105692] Updated weights for policy 0, policy_version 437911 (0.0007) [2023-12-26 18:33:09,667][105692] Updated weights for policy 0, policy_version 437921 (0.0008) [2023-12-26 18:33:10,305][105620] Updated weights for policy 1, policy_version 438254 (0.0010) [2023-12-26 18:33:10,358][105620] Updated weights for policy 1, policy_version 438264 (0.0011) [2023-12-26 18:33:10,411][105620] Updated weights for policy 1, policy_version 438274 (0.0011) [2023-12-26 18:33:10,476][105692] Updated weights for policy 0, policy_version 437931 (0.0008) [2023-12-26 18:33:10,528][105692] Updated weights for policy 0, policy_version 437941 (0.0008) [2023-12-26 18:33:10,588][105692] Updated weights for policy 0, policy_version 437951 (0.0008) [2023-12-26 18:33:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 224346112. Throughput: 0: 9841.1, 1: 9483.8. Samples: 224356800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:33:11,062][104569] Avg episode reward: [(0, '8861.954'), (1, '9020.993')] [2023-12-26 18:33:11,179][105620] Updated weights for policy 1, policy_version 438284 (0.0009) [2023-12-26 18:33:11,241][105620] Updated weights for policy 1, policy_version 438294 (0.0010) [2023-12-26 18:33:11,298][105692] Updated weights for policy 0, policy_version 437961 (0.0008) [2023-12-26 18:33:11,302][105620] Updated weights for policy 1, policy_version 438304 (0.0011) [2023-12-26 18:33:11,350][105692] Updated weights for policy 0, policy_version 437971 (0.0007) [2023-12-26 18:33:11,416][105692] Updated weights for policy 0, policy_version 437981 (0.0009) [2023-12-26 18:33:11,472][105692] Updated weights for policy 0, policy_version 437991 (0.0009) [2023-12-26 18:33:12,103][105620] Updated weights for policy 1, policy_version 438314 (0.0011) [2023-12-26 18:33:12,166][105620] Updated weights for policy 1, policy_version 438324 (0.0010) [2023-12-26 18:33:12,229][105620] Updated weights for policy 1, policy_version 438334 (0.0010) [2023-12-26 18:33:12,244][105692] Updated weights for policy 0, policy_version 438001 (0.0007) [2023-12-26 18:33:12,289][105620] Updated weights for policy 1, policy_version 438344 (0.0008) [2023-12-26 18:33:12,302][105692] Updated weights for policy 0, policy_version 438011 (0.0008) [2023-12-26 18:33:12,365][105692] Updated weights for policy 0, policy_version 438021 (0.0009) [2023-12-26 18:33:13,029][105620] Updated weights for policy 1, policy_version 438354 (0.0008) [2023-12-26 18:33:13,057][105692] Updated weights for policy 0, policy_version 438031 (0.0009) [2023-12-26 18:33:13,083][105620] Updated weights for policy 1, policy_version 438364 (0.0008) [2023-12-26 18:33:13,118][105692] Updated weights for policy 0, policy_version 438041 (0.0006) [2023-12-26 18:33:13,140][105620] Updated weights for policy 1, policy_version 438374 (0.0010) [2023-12-26 18:33:13,170][105692] Updated weights for policy 0, policy_version 438051 (0.0006) [2023-12-26 18:33:13,883][105620] Updated weights for policy 1, policy_version 438384 (0.0010) [2023-12-26 18:33:13,932][105692] Updated weights for policy 0, policy_version 438061 (0.0009) [2023-12-26 18:33:13,936][105620] Updated weights for policy 1, policy_version 438394 (0.0010) [2023-12-26 18:33:13,982][105692] Updated weights for policy 0, policy_version 438071 (0.0005) [2023-12-26 18:33:13,992][105620] Updated weights for policy 1, policy_version 438404 (0.0007) [2023-12-26 18:33:14,033][105692] Updated weights for policy 0, policy_version 438081 (0.0008) [2023-12-26 18:33:14,729][105620] Updated weights for policy 1, policy_version 438414 (0.0006) [2023-12-26 18:33:14,793][105620] Updated weights for policy 1, policy_version 438424 (0.0007) [2023-12-26 18:33:14,807][105692] Updated weights for policy 0, policy_version 438091 (0.0008) [2023-12-26 18:33:14,847][105620] Updated weights for policy 1, policy_version 438434 (0.0008) [2023-12-26 18:33:14,859][105692] Updated weights for policy 0, policy_version 438101 (0.0008) [2023-12-26 18:33:14,928][105692] Updated weights for policy 0, policy_version 438111 (0.0007) [2023-12-26 18:33:15,591][105620] Updated weights for policy 1, policy_version 438444 (0.0009) [2023-12-26 18:33:15,599][105692] Updated weights for policy 0, policy_version 438121 (0.0008) [2023-12-26 18:33:15,650][105620] Updated weights for policy 1, policy_version 438454 (0.0011) [2023-12-26 18:33:15,663][105692] Updated weights for policy 0, policy_version 438131 (0.0006) [2023-12-26 18:33:15,709][105620] Updated weights for policy 1, policy_version 438464 (0.0010) [2023-12-26 18:33:15,721][105692] Updated weights for policy 0, policy_version 438141 (0.0006) [2023-12-26 18:33:15,774][105692] Updated weights for policy 0, policy_version 438151 (0.0007) [2023-12-26 18:33:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.6, 300 sec: 19549.7). Total num frames: 224444416. Throughput: 0: 9769.8, 1: 9464.6. Samples: 224412980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:33:16,063][104569] Avg episode reward: [(0, '8769.053'), (1, '9172.447')] [2023-12-26 18:33:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000438152_112181248.pth... [2023-12-26 18:33:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000438472_112263168.pth... [2023-12-26 18:33:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000437000_111886336.pth [2023-12-26 18:33:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000437352_111976448.pth [2023-12-26 18:33:16,340][105692] Updated weights for policy 0, policy_version 438161 (0.0006) [2023-12-26 18:33:16,389][105620] Updated weights for policy 1, policy_version 438474 (0.0009) [2023-12-26 18:33:16,392][105692] Updated weights for policy 0, policy_version 438171 (0.0006) [2023-12-26 18:33:16,441][105620] Updated weights for policy 1, policy_version 438484 (0.0007) [2023-12-26 18:33:16,447][105692] Updated weights for policy 0, policy_version 438181 (0.0006) [2023-12-26 18:33:16,496][105620] Updated weights for policy 1, policy_version 438494 (0.0010) [2023-12-26 18:33:16,547][105620] Updated weights for policy 1, policy_version 438504 (0.0010) [2023-12-26 18:33:17,108][105692] Updated weights for policy 0, policy_version 438191 (0.0009) [2023-12-26 18:33:17,169][105692] Updated weights for policy 0, policy_version 438201 (0.0010) [2023-12-26 18:33:17,234][105692] Updated weights for policy 0, policy_version 438211 (0.0010) [2023-12-26 18:33:17,282][105620] Updated weights for policy 1, policy_version 438514 (0.0008) [2023-12-26 18:33:17,347][105620] Updated weights for policy 1, policy_version 438524 (0.0008) [2023-12-26 18:33:17,412][105620] Updated weights for policy 1, policy_version 438534 (0.0008) [2023-12-26 18:33:17,955][105692] Updated weights for policy 0, policy_version 438221 (0.0010) [2023-12-26 18:33:18,002][105692] Updated weights for policy 0, policy_version 438231 (0.0010) [2023-12-26 18:33:18,064][105692] Updated weights for policy 0, policy_version 438241 (0.0010) [2023-12-26 18:33:18,103][105620] Updated weights for policy 1, policy_version 438544 (0.0006) [2023-12-26 18:33:18,159][105620] Updated weights for policy 1, policy_version 438554 (0.0008) [2023-12-26 18:33:18,208][105620] Updated weights for policy 1, policy_version 438564 (0.0008) [2023-12-26 18:33:18,822][105692] Updated weights for policy 0, policy_version 438251 (0.0010) [2023-12-26 18:33:18,885][105692] Updated weights for policy 0, policy_version 438261 (0.0010) [2023-12-26 18:33:18,944][105692] Updated weights for policy 0, policy_version 438271 (0.0010) [2023-12-26 18:33:19,006][105620] Updated weights for policy 1, policy_version 438574 (0.0007) [2023-12-26 18:33:19,054][105620] Updated weights for policy 1, policy_version 438584 (0.0008) [2023-12-26 18:33:19,111][105620] Updated weights for policy 1, policy_version 438594 (0.0009) [2023-12-26 18:33:19,702][105692] Updated weights for policy 0, policy_version 438281 (0.0010) [2023-12-26 18:33:19,759][105692] Updated weights for policy 0, policy_version 438291 (0.0011) [2023-12-26 18:33:19,823][105692] Updated weights for policy 0, policy_version 438301 (0.0011) [2023-12-26 18:33:19,882][105620] Updated weights for policy 1, policy_version 438604 (0.0006) [2023-12-26 18:33:19,892][105692] Updated weights for policy 0, policy_version 438311 (0.0011) [2023-12-26 18:33:19,947][105620] Updated weights for policy 1, policy_version 438614 (0.0007) [2023-12-26 18:33:20,018][105620] Updated weights for policy 1, policy_version 438624 (0.0008) [2023-12-26 18:33:20,643][105692] Updated weights for policy 0, policy_version 438321 (0.0011) [2023-12-26 18:33:20,706][105692] Updated weights for policy 0, policy_version 438331 (0.0011) [2023-12-26 18:33:20,740][105620] Updated weights for policy 1, policy_version 438634 (0.0008) [2023-12-26 18:33:20,775][105692] Updated weights for policy 0, policy_version 438341 (0.0011) [2023-12-26 18:33:20,799][105620] Updated weights for policy 1, policy_version 438644 (0.0007) [2023-12-26 18:33:20,856][105620] Updated weights for policy 1, policy_version 438654 (0.0008) [2023-12-26 18:33:20,916][105620] Updated weights for policy 1, policy_version 438664 (0.0008) [2023-12-26 18:33:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 224542720. Throughput: 0: 9872.7, 1: 9385.0. Samples: 224529792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:33:21,063][104569] Avg episode reward: [(0, '8865.201'), (1, '7905.733')] [2023-12-26 18:33:21,555][105692] Updated weights for policy 0, policy_version 438351 (0.0011) [2023-12-26 18:33:21,605][105692] Updated weights for policy 0, policy_version 438361 (0.0008) [2023-12-26 18:33:21,652][105620] Updated weights for policy 1, policy_version 438674 (0.0009) [2023-12-26 18:33:21,673][105692] Updated weights for policy 0, policy_version 438371 (0.0010) [2023-12-26 18:33:21,722][105620] Updated weights for policy 1, policy_version 438684 (0.0011) [2023-12-26 18:33:21,785][105620] Updated weights for policy 1, policy_version 438694 (0.0007) [2023-12-26 18:33:22,458][105692] Updated weights for policy 0, policy_version 438381 (0.0011) [2023-12-26 18:33:22,521][105692] Updated weights for policy 0, policy_version 438391 (0.0011) [2023-12-26 18:33:22,523][105620] Updated weights for policy 1, policy_version 438704 (0.0006) [2023-12-26 18:33:22,582][105692] Updated weights for policy 0, policy_version 438401 (0.0010) [2023-12-26 18:33:22,588][105620] Updated weights for policy 1, policy_version 438714 (0.0006) [2023-12-26 18:33:22,651][105620] Updated weights for policy 1, policy_version 438724 (0.0006) [2023-12-26 18:33:23,279][105620] Updated weights for policy 1, policy_version 438734 (0.0007) [2023-12-26 18:33:23,321][105692] Updated weights for policy 0, policy_version 438411 (0.0011) [2023-12-26 18:33:23,347][105620] Updated weights for policy 1, policy_version 438744 (0.0006) [2023-12-26 18:33:23,380][105692] Updated weights for policy 0, policy_version 438421 (0.0011) [2023-12-26 18:33:23,403][105620] Updated weights for policy 1, policy_version 438754 (0.0006) [2023-12-26 18:33:23,444][105692] Updated weights for policy 0, policy_version 438431 (0.0011) [2023-12-26 18:33:24,153][105620] Updated weights for policy 1, policy_version 438764 (0.0008) [2023-12-26 18:33:24,195][105692] Updated weights for policy 0, policy_version 438441 (0.0011) [2023-12-26 18:33:24,217][105620] Updated weights for policy 1, policy_version 438774 (0.0006) [2023-12-26 18:33:24,240][105692] Updated weights for policy 0, policy_version 438451 (0.0010) [2023-12-26 18:33:24,277][105620] Updated weights for policy 1, policy_version 438784 (0.0008) [2023-12-26 18:33:24,289][105692] Updated weights for policy 0, policy_version 438461 (0.0010) [2023-12-26 18:33:24,343][105692] Updated weights for policy 0, policy_version 438471 (0.0010) [2023-12-26 18:33:24,898][105620] Updated weights for policy 1, policy_version 438794 (0.0008) [2023-12-26 18:33:24,950][105620] Updated weights for policy 1, policy_version 438804 (0.0010) [2023-12-26 18:33:24,991][105692] Updated weights for policy 0, policy_version 438481 (0.0006) [2023-12-26 18:33:25,006][105620] Updated weights for policy 1, policy_version 438814 (0.0008) [2023-12-26 18:33:25,049][105692] Updated weights for policy 0, policy_version 438491 (0.0005) [2023-12-26 18:33:25,069][105620] Updated weights for policy 1, policy_version 438824 (0.0009) [2023-12-26 18:33:25,109][105692] Updated weights for policy 0, policy_version 438501 (0.0009) [2023-12-26 18:33:25,743][105620] Updated weights for policy 1, policy_version 438834 (0.0008) [2023-12-26 18:33:25,791][105692] Updated weights for policy 0, policy_version 438511 (0.0010) [2023-12-26 18:33:25,797][105620] Updated weights for policy 1, policy_version 438844 (0.0008) [2023-12-26 18:33:25,846][105692] Updated weights for policy 0, policy_version 438521 (0.0010) [2023-12-26 18:33:25,852][105620] Updated weights for policy 1, policy_version 438854 (0.0005) [2023-12-26 18:33:25,908][105692] Updated weights for policy 0, policy_version 438531 (0.0011) [2023-12-26 18:33:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 224641024. Throughput: 0: 9823.6, 1: 9426.0. Samples: 224644960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:33:26,062][104569] Avg episode reward: [(0, '9173.950'), (1, '7945.135')] [2023-12-26 18:33:26,506][105620] Updated weights for policy 1, policy_version 438864 (0.0007) [2023-12-26 18:33:26,550][105620] Updated weights for policy 1, policy_version 438874 (0.0008) [2023-12-26 18:33:26,597][105620] Updated weights for policy 1, policy_version 438884 (0.0008) [2023-12-26 18:33:26,661][105692] Updated weights for policy 0, policy_version 438541 (0.0010) [2023-12-26 18:33:26,715][105692] Updated weights for policy 0, policy_version 438551 (0.0010) [2023-12-26 18:33:26,770][105692] Updated weights for policy 0, policy_version 438561 (0.0008) [2023-12-26 18:33:27,167][105620] Updated weights for policy 1, policy_version 438894 (0.0006) [2023-12-26 18:33:27,226][105620] Updated weights for policy 1, policy_version 438904 (0.0005) [2023-12-26 18:33:27,282][105620] Updated weights for policy 1, policy_version 438914 (0.0005) [2023-12-26 18:33:27,482][105692] Updated weights for policy 0, policy_version 438571 (0.0008) [2023-12-26 18:33:27,529][105692] Updated weights for policy 0, policy_version 438581 (0.0010) [2023-12-26 18:33:27,580][105692] Updated weights for policy 0, policy_version 438591 (0.0010) [2023-12-26 18:33:27,902][105620] Updated weights for policy 1, policy_version 438924 (0.0005) [2023-12-26 18:33:27,960][105620] Updated weights for policy 1, policy_version 438934 (0.0005) [2023-12-26 18:33:28,018][105620] Updated weights for policy 1, policy_version 438944 (0.0006) [2023-12-26 18:33:28,342][105692] Updated weights for policy 0, policy_version 438601 (0.0010) [2023-12-26 18:33:28,403][105692] Updated weights for policy 0, policy_version 438611 (0.0010) [2023-12-26 18:33:28,464][105692] Updated weights for policy 0, policy_version 438621 (0.0010) [2023-12-26 18:33:28,522][105692] Updated weights for policy 0, policy_version 438631 (0.0010) [2023-12-26 18:33:28,597][105620] Updated weights for policy 1, policy_version 438954 (0.0009) [2023-12-26 18:33:28,652][105620] Updated weights for policy 1, policy_version 438964 (0.0010) [2023-12-26 18:33:28,712][105620] Updated weights for policy 1, policy_version 438974 (0.0011) [2023-12-26 18:33:28,771][105620] Updated weights for policy 1, policy_version 438984 (0.0010) [2023-12-26 18:33:29,228][105692] Updated weights for policy 0, policy_version 438641 (0.0008) [2023-12-26 18:33:29,293][105692] Updated weights for policy 0, policy_version 438651 (0.0009) [2023-12-26 18:33:29,358][105692] Updated weights for policy 0, policy_version 438661 (0.0009) [2023-12-26 18:33:29,500][105620] Updated weights for policy 1, policy_version 438994 (0.0007) [2023-12-26 18:33:29,569][105620] Updated weights for policy 1, policy_version 439004 (0.0005) [2023-12-26 18:33:29,640][105620] Updated weights for policy 1, policy_version 439014 (0.0005) [2023-12-26 18:33:30,063][105692] Updated weights for policy 0, policy_version 438671 (0.0010) [2023-12-26 18:33:30,130][105692] Updated weights for policy 0, policy_version 438681 (0.0010) [2023-12-26 18:33:30,179][105692] Updated weights for policy 0, policy_version 438691 (0.0010) [2023-12-26 18:33:30,243][105620] Updated weights for policy 1, policy_version 439024 (0.0008) [2023-12-26 18:33:30,302][105620] Updated weights for policy 1, policy_version 439034 (0.0008) [2023-12-26 18:33:30,358][105620] Updated weights for policy 1, policy_version 439044 (0.0008) [2023-12-26 18:33:30,859][105692] Updated weights for policy 0, policy_version 438701 (0.0008) [2023-12-26 18:33:30,914][105692] Updated weights for policy 0, policy_version 438711 (0.0005) [2023-12-26 18:33:30,972][105692] Updated weights for policy 0, policy_version 438721 (0.0005) [2023-12-26 18:33:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 224739328. Throughput: 0: 9774.9, 1: 9588.8. Samples: 224707168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:33:31,062][104569] Avg episode reward: [(0, '9262.459'), (1, '8169.029')] [2023-12-26 18:33:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000438728_112328704.pth... [2023-12-26 18:33:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000437576_112033792.pth [2023-12-26 18:33:31,097][105586] KL-divergence is very high: 155.2666 [2023-12-26 18:33:31,109][105620] Updated weights for policy 1, policy_version 439054 (0.0008) [2023-12-26 18:33:31,153][105586] KL-divergence is very high: 210.2501 [2023-12-26 18:33:31,175][105620] Updated weights for policy 1, policy_version 439064 (0.0008) [2023-12-26 18:33:31,197][105586] KL-divergence is very high: 183.8936 [2023-12-26 18:33:31,227][105620] Updated weights for policy 1, policy_version 439074 (0.0008) [2023-12-26 18:33:31,240][105586] KL-divergence is very high: 134.7076 [2023-12-26 18:33:31,258][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000439080_112418816.pth... [2023-12-26 18:33:31,262][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000437896_112115712.pth [2023-12-26 18:33:31,705][105692] Updated weights for policy 0, policy_version 438731 (0.0007) [2023-12-26 18:33:31,775][105692] Updated weights for policy 0, policy_version 438741 (0.0009) [2023-12-26 18:33:31,833][105692] Updated weights for policy 0, policy_version 438751 (0.0008) [2023-12-26 18:33:31,994][105620] Updated weights for policy 1, policy_version 439084 (0.0007) [2023-12-26 18:33:32,018][105586] KL-divergence is very high: 104.8352 [2023-12-26 18:33:32,047][105620] Updated weights for policy 1, policy_version 439094 (0.0005) [2023-12-26 18:33:32,058][105586] KL-divergence is very high: 104.6275 [2023-12-26 18:33:32,098][105620] Updated weights for policy 1, policy_version 439104 (0.0008) [2023-12-26 18:33:32,588][105692] Updated weights for policy 0, policy_version 438761 (0.0009) [2023-12-26 18:33:32,646][105692] Updated weights for policy 0, policy_version 438771 (0.0009) [2023-12-26 18:33:32,700][105692] Updated weights for policy 0, policy_version 438781 (0.0009) [2023-12-26 18:33:32,749][105692] Updated weights for policy 0, policy_version 438791 (0.0008) [2023-12-26 18:33:32,847][105620] Updated weights for policy 1, policy_version 439114 (0.0009) [2023-12-26 18:33:32,907][105620] Updated weights for policy 1, policy_version 439124 (0.0006) [2023-12-26 18:33:32,952][105620] Updated weights for policy 1, policy_version 439134 (0.0005) [2023-12-26 18:33:32,996][105620] Updated weights for policy 1, policy_version 439144 (0.0007) [2023-12-26 18:33:33,526][105692] Updated weights for policy 0, policy_version 438801 (0.0009) [2023-12-26 18:33:33,585][105692] Updated weights for policy 0, policy_version 438811 (0.0009) [2023-12-26 18:33:33,646][105692] Updated weights for policy 0, policy_version 438821 (0.0009) [2023-12-26 18:33:33,730][105620] Updated weights for policy 1, policy_version 439154 (0.0009) [2023-12-26 18:33:33,776][105620] Updated weights for policy 1, policy_version 439164 (0.0009) [2023-12-26 18:33:33,834][105620] Updated weights for policy 1, policy_version 439174 (0.0009) [2023-12-26 18:33:34,406][105692] Updated weights for policy 0, policy_version 438831 (0.0009) [2023-12-26 18:33:34,464][105692] Updated weights for policy 0, policy_version 438841 (0.0009) [2023-12-26 18:33:34,519][105692] Updated weights for policy 0, policy_version 438851 (0.0009) [2023-12-26 18:33:34,612][105620] Updated weights for policy 1, policy_version 439184 (0.0009) [2023-12-26 18:33:34,675][105620] Updated weights for policy 1, policy_version 439194 (0.0007) [2023-12-26 18:33:34,737][105620] Updated weights for policy 1, policy_version 439204 (0.0009) [2023-12-26 18:33:35,222][105692] Updated weights for policy 0, policy_version 438861 (0.0009) [2023-12-26 18:33:35,280][105692] Updated weights for policy 0, policy_version 438871 (0.0009) [2023-12-26 18:33:35,327][105692] Updated weights for policy 0, policy_version 438881 (0.0008) [2023-12-26 18:33:35,485][105620] Updated weights for policy 1, policy_version 439214 (0.0009) [2023-12-26 18:33:35,536][105620] Updated weights for policy 1, policy_version 439224 (0.0009) [2023-12-26 18:33:35,590][105620] Updated weights for policy 1, policy_version 439234 (0.0009) [2023-12-26 18:33:36,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 224829440. Throughput: 0: 9728.4, 1: 9671.6. Samples: 224821564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:33:36,063][104569] Avg episode reward: [(0, '9351.990'), (1, '7675.440')] [2023-12-26 18:33:36,101][105692] Updated weights for policy 0, policy_version 438891 (0.0009) [2023-12-26 18:33:36,162][105692] Updated weights for policy 0, policy_version 438901 (0.0009) [2023-12-26 18:33:36,221][105692] Updated weights for policy 0, policy_version 438911 (0.0010) [2023-12-26 18:33:36,299][105620] Updated weights for policy 1, policy_version 439244 (0.0009) [2023-12-26 18:33:36,352][105620] Updated weights for policy 1, policy_version 439254 (0.0009) [2023-12-26 18:33:36,404][105620] Updated weights for policy 1, policy_version 439264 (0.0008) [2023-12-26 18:33:36,924][105692] Updated weights for policy 0, policy_version 438921 (0.0009) [2023-12-26 18:33:36,985][105692] Updated weights for policy 0, policy_version 438931 (0.0006) [2023-12-26 18:33:37,034][105692] Updated weights for policy 0, policy_version 438941 (0.0005) [2023-12-26 18:33:37,083][105692] Updated weights for policy 0, policy_version 438951 (0.0005) [2023-12-26 18:33:37,193][105620] Updated weights for policy 1, policy_version 439274 (0.0009) [2023-12-26 18:33:37,262][105620] Updated weights for policy 1, policy_version 439284 (0.0010) [2023-12-26 18:33:37,324][105620] Updated weights for policy 1, policy_version 439294 (0.0011) [2023-12-26 18:33:37,381][105620] Updated weights for policy 1, policy_version 439304 (0.0011) [2023-12-26 18:33:37,721][105692] Updated weights for policy 0, policy_version 438961 (0.0010) [2023-12-26 18:33:37,779][105692] Updated weights for policy 0, policy_version 438971 (0.0010) [2023-12-26 18:33:37,840][105692] Updated weights for policy 0, policy_version 438981 (0.0010) [2023-12-26 18:33:38,095][105620] Updated weights for policy 1, policy_version 439314 (0.0006) [2023-12-26 18:33:38,157][105620] Updated weights for policy 1, policy_version 439324 (0.0009) [2023-12-26 18:33:38,222][105620] Updated weights for policy 1, policy_version 439334 (0.0008) [2023-12-26 18:33:38,526][105692] Updated weights for policy 0, policy_version 438991 (0.0008) [2023-12-26 18:33:38,588][105692] Updated weights for policy 0, policy_version 439001 (0.0005) [2023-12-26 18:33:38,660][105692] Updated weights for policy 0, policy_version 439011 (0.0006) [2023-12-26 18:33:38,812][105620] Updated weights for policy 1, policy_version 439344 (0.0006) [2023-12-26 18:33:38,885][105620] Updated weights for policy 1, policy_version 439354 (0.0006) [2023-12-26 18:33:38,955][105620] Updated weights for policy 1, policy_version 439364 (0.0005) [2023-12-26 18:33:39,195][105692] Updated weights for policy 0, policy_version 439021 (0.0008) [2023-12-26 18:33:39,252][105692] Updated weights for policy 0, policy_version 439031 (0.0007) [2023-12-26 18:33:39,306][105692] Updated weights for policy 0, policy_version 439041 (0.0008) [2023-12-26 18:33:39,534][105620] Updated weights for policy 1, policy_version 439374 (0.0007) [2023-12-26 18:33:39,595][105620] Updated weights for policy 1, policy_version 439384 (0.0010) [2023-12-26 18:33:39,657][105620] Updated weights for policy 1, policy_version 439394 (0.0009) [2023-12-26 18:33:40,150][105692] Updated weights for policy 0, policy_version 439051 (0.0009) [2023-12-26 18:33:40,212][105692] Updated weights for policy 0, policy_version 439061 (0.0010) [2023-12-26 18:33:40,275][105692] Updated weights for policy 0, policy_version 439071 (0.0010) [2023-12-26 18:33:40,363][105620] Updated weights for policy 1, policy_version 439404 (0.0008) [2023-12-26 18:33:40,413][105620] Updated weights for policy 1, policy_version 439414 (0.0007) [2023-12-26 18:33:40,461][105620] Updated weights for policy 1, policy_version 439424 (0.0009) [2023-12-26 18:33:41,023][105692] Updated weights for policy 0, policy_version 439081 (0.0009) [2023-12-26 18:33:41,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 224927744. Throughput: 0: 9674.7, 1: 9812.1. Samples: 224940940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:33:41,062][104569] Avg episode reward: [(0, '9352.505'), (1, '8046.910')] [2023-12-26 18:33:41,092][105692] Updated weights for policy 0, policy_version 439091 (0.0007) [2023-12-26 18:33:41,156][105692] Updated weights for policy 0, policy_version 439101 (0.0007) [2023-12-26 18:33:41,222][105692] Updated weights for policy 0, policy_version 439111 (0.0010) [2023-12-26 18:33:41,228][105620] Updated weights for policy 1, policy_version 439434 (0.0009) [2023-12-26 18:33:41,300][105620] Updated weights for policy 1, policy_version 439444 (0.0009) [2023-12-26 18:33:41,389][105620] Updated weights for policy 1, policy_version 439454 (0.0008) [2023-12-26 18:33:41,456][105620] Updated weights for policy 1, policy_version 439464 (0.0010) [2023-12-26 18:33:41,956][105692] Updated weights for policy 0, policy_version 439121 (0.0008) [2023-12-26 18:33:42,004][105692] Updated weights for policy 0, policy_version 439131 (0.0009) [2023-12-26 18:33:42,051][105692] Updated weights for policy 0, policy_version 439141 (0.0009) [2023-12-26 18:33:42,171][105620] Updated weights for policy 1, policy_version 439474 (0.0009) [2023-12-26 18:33:42,232][105620] Updated weights for policy 1, policy_version 439484 (0.0009) [2023-12-26 18:33:42,298][105620] Updated weights for policy 1, policy_version 439494 (0.0010) [2023-12-26 18:33:42,852][105692] Updated weights for policy 0, policy_version 439151 (0.0007) [2023-12-26 18:33:42,913][105692] Updated weights for policy 0, policy_version 439161 (0.0006) [2023-12-26 18:33:42,973][105692] Updated weights for policy 0, policy_version 439171 (0.0005) [2023-12-26 18:33:43,077][105620] Updated weights for policy 1, policy_version 439504 (0.0009) [2023-12-26 18:33:43,134][105620] Updated weights for policy 1, policy_version 439514 (0.0008) [2023-12-26 18:33:43,199][105620] Updated weights for policy 1, policy_version 439524 (0.0008) [2023-12-26 18:33:43,620][105692] Updated weights for policy 0, policy_version 439181 (0.0006) [2023-12-26 18:33:43,680][105692] Updated weights for policy 0, policy_version 439191 (0.0007) [2023-12-26 18:33:43,738][105692] Updated weights for policy 0, policy_version 439201 (0.0010) [2023-12-26 18:33:43,982][105620] Updated weights for policy 1, policy_version 439534 (0.0009) [2023-12-26 18:33:44,043][105620] Updated weights for policy 1, policy_version 439544 (0.0009) [2023-12-26 18:33:44,109][105620] Updated weights for policy 1, policy_version 439554 (0.0010) [2023-12-26 18:33:44,298][105692] Updated weights for policy 0, policy_version 439211 (0.0010) [2023-12-26 18:33:44,346][105692] Updated weights for policy 0, policy_version 439221 (0.0008) [2023-12-26 18:33:44,397][105692] Updated weights for policy 0, policy_version 439231 (0.0009) [2023-12-26 18:33:44,877][105620] Updated weights for policy 1, policy_version 439564 (0.0009) [2023-12-26 18:33:44,926][105620] Updated weights for policy 1, policy_version 439574 (0.0008) [2023-12-26 18:33:44,974][105620] Updated weights for policy 1, policy_version 439584 (0.0008) [2023-12-26 18:33:45,139][105692] Updated weights for policy 0, policy_version 439241 (0.0010) [2023-12-26 18:33:45,202][105692] Updated weights for policy 0, policy_version 439251 (0.0007) [2023-12-26 18:33:45,267][105692] Updated weights for policy 0, policy_version 439261 (0.0010) [2023-12-26 18:33:45,337][105692] Updated weights for policy 0, policy_version 439271 (0.0011) [2023-12-26 18:33:45,776][105620] Updated weights for policy 1, policy_version 439594 (0.0009) [2023-12-26 18:33:45,829][105620] Updated weights for policy 1, policy_version 439604 (0.0010) [2023-12-26 18:33:45,888][105620] Updated weights for policy 1, policy_version 439614 (0.0009) [2023-12-26 18:33:45,942][105620] Updated weights for policy 1, policy_version 439624 (0.0008) [2023-12-26 18:33:45,983][105692] Updated weights for policy 0, policy_version 439281 (0.0010) [2023-12-26 18:33:46,031][105692] Updated weights for policy 0, policy_version 439291 (0.0009) [2023-12-26 18:33:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 225026048. Throughput: 0: 9637.0, 1: 9827.1. Samples: 224996772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:33:46,062][104569] Avg episode reward: [(0, '9170.455'), (1, '8936.269')] [2023-12-26 18:33:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000439624_112558080.pth... [2023-12-26 18:33:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000438472_112263168.pth [2023-12-26 18:33:46,077][105692] Updated weights for policy 0, policy_version 439301 (0.0009) [2023-12-26 18:33:46,089][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000439304_112476160.pth... [2023-12-26 18:33:46,092][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000438152_112181248.pth [2023-12-26 18:33:46,718][105620] Updated weights for policy 1, policy_version 439634 (0.0010) [2023-12-26 18:33:46,778][105620] Updated weights for policy 1, policy_version 439644 (0.0010) [2023-12-26 18:33:46,805][105692] Updated weights for policy 0, policy_version 439311 (0.0006) [2023-12-26 18:33:46,838][105620] Updated weights for policy 1, policy_version 439654 (0.0007) [2023-12-26 18:33:46,861][105692] Updated weights for policy 0, policy_version 439321 (0.0008) [2023-12-26 18:33:46,912][105692] Updated weights for policy 0, policy_version 439331 (0.0009) [2023-12-26 18:33:47,502][105692] Updated weights for policy 0, policy_version 439341 (0.0007) [2023-12-26 18:33:47,560][105692] Updated weights for policy 0, policy_version 439351 (0.0005) [2023-12-26 18:33:47,580][105585] KL-divergence is very high: 113.4887 [2023-12-26 18:33:47,589][105585] KL-divergence is very high: 109.4748 [2023-12-26 18:33:47,599][105585] KL-divergence is very high: 106.1160 [2023-12-26 18:33:47,608][105692] Updated weights for policy 0, policy_version 439361 (0.0005) [2023-12-26 18:33:47,618][105585] KL-divergence is very high: 117.9487 [2023-12-26 18:33:47,705][105620] Updated weights for policy 1, policy_version 439664 (0.0008) [2023-12-26 18:33:47,765][105620] Updated weights for policy 1, policy_version 439674 (0.0009) [2023-12-26 18:33:47,818][105620] Updated weights for policy 1, policy_version 439684 (0.0010) [2023-12-26 18:33:48,292][105692] Updated weights for policy 0, policy_version 439371 (0.0006) [2023-12-26 18:33:48,349][105692] Updated weights for policy 0, policy_version 439381 (0.0009) [2023-12-26 18:33:48,402][105692] Updated weights for policy 0, policy_version 439391 (0.0007) [2023-12-26 18:33:48,497][105620] Updated weights for policy 1, policy_version 439694 (0.0008) [2023-12-26 18:33:48,553][105620] Updated weights for policy 1, policy_version 439704 (0.0007) [2023-12-26 18:33:48,627][105620] Updated weights for policy 1, policy_version 439714 (0.0008) [2023-12-26 18:33:49,185][105692] Updated weights for policy 0, policy_version 439401 (0.0008) [2023-12-26 18:33:49,253][105692] Updated weights for policy 0, policy_version 439411 (0.0009) [2023-12-26 18:33:49,296][105620] Updated weights for policy 1, policy_version 439724 (0.0006) [2023-12-26 18:33:49,310][105692] Updated weights for policy 0, policy_version 439421 (0.0008) [2023-12-26 18:33:49,359][105620] Updated weights for policy 1, policy_version 439734 (0.0007) [2023-12-26 18:33:49,371][105692] Updated weights for policy 0, policy_version 439431 (0.0009) [2023-12-26 18:33:49,422][105620] Updated weights for policy 1, policy_version 439744 (0.0009) [2023-12-26 18:33:50,114][105692] Updated weights for policy 0, policy_version 439441 (0.0008) [2023-12-26 18:33:50,177][105692] Updated weights for policy 0, policy_version 439451 (0.0009) [2023-12-26 18:33:50,189][105620] Updated weights for policy 1, policy_version 439754 (0.0009) [2023-12-26 18:33:50,236][105692] Updated weights for policy 0, policy_version 439461 (0.0008) [2023-12-26 18:33:50,250][105620] Updated weights for policy 1, policy_version 439764 (0.0007) [2023-12-26 18:33:50,308][105620] Updated weights for policy 1, policy_version 439774 (0.0009) [2023-12-26 18:33:50,362][105620] Updated weights for policy 1, policy_version 439784 (0.0009) [2023-12-26 18:33:50,975][105692] Updated weights for policy 0, policy_version 439471 (0.0007) [2023-12-26 18:33:51,058][105692] Updated weights for policy 0, policy_version 439481 (0.0009) [2023-12-26 18:33:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 225116160. Throughput: 0: 9748.0, 1: 9717.5. Samples: 225112892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:33:51,063][104569] Avg episode reward: [(0, '7640.271'), (1, '9263.431')] [2023-12-26 18:33:51,123][105692] Updated weights for policy 0, policy_version 439491 (0.0007) [2023-12-26 18:33:51,151][105620] Updated weights for policy 1, policy_version 439794 (0.0008) [2023-12-26 18:33:51,216][105620] Updated weights for policy 1, policy_version 439804 (0.0008) [2023-12-26 18:33:51,285][105620] Updated weights for policy 1, policy_version 439814 (0.0009) [2023-12-26 18:33:51,828][105692] Updated weights for policy 0, policy_version 439501 (0.0009) [2023-12-26 18:33:51,895][105692] Updated weights for policy 0, policy_version 439511 (0.0011) [2023-12-26 18:33:51,951][105692] Updated weights for policy 0, policy_version 439521 (0.0011) [2023-12-26 18:33:52,085][105620] Updated weights for policy 1, policy_version 439824 (0.0008) [2023-12-26 18:33:52,146][105620] Updated weights for policy 1, policy_version 439834 (0.0010) [2023-12-26 18:33:52,202][105620] Updated weights for policy 1, policy_version 439844 (0.0011) [2023-12-26 18:33:52,694][105692] Updated weights for policy 0, policy_version 439531 (0.0010) [2023-12-26 18:33:52,752][105692] Updated weights for policy 0, policy_version 439541 (0.0010) [2023-12-26 18:33:52,817][105692] Updated weights for policy 0, policy_version 439551 (0.0010) [2023-12-26 18:33:52,922][105620] Updated weights for policy 1, policy_version 439854 (0.0011) [2023-12-26 18:33:52,988][105620] Updated weights for policy 1, policy_version 439864 (0.0010) [2023-12-26 18:33:53,057][105620] Updated weights for policy 1, policy_version 439874 (0.0011) [2023-12-26 18:33:53,559][105692] Updated weights for policy 0, policy_version 439561 (0.0010) [2023-12-26 18:33:53,622][105692] Updated weights for policy 0, policy_version 439571 (0.0010) [2023-12-26 18:33:53,659][105620] Updated weights for policy 1, policy_version 439884 (0.0009) [2023-12-26 18:33:53,684][105692] Updated weights for policy 0, policy_version 439581 (0.0010) [2023-12-26 18:33:53,705][105620] Updated weights for policy 1, policy_version 439894 (0.0007) [2023-12-26 18:33:53,739][105692] Updated weights for policy 0, policy_version 439591 (0.0010) [2023-12-26 18:33:53,762][105620] Updated weights for policy 1, policy_version 439904 (0.0005) [2023-12-26 18:33:54,377][105620] Updated weights for policy 1, policy_version 439914 (0.0006) [2023-12-26 18:33:54,436][105620] Updated weights for policy 1, policy_version 439924 (0.0010) [2023-12-26 18:33:54,468][105692] Updated weights for policy 0, policy_version 439601 (0.0010) [2023-12-26 18:33:54,484][105620] Updated weights for policy 1, policy_version 439934 (0.0010) [2023-12-26 18:33:54,512][105692] Updated weights for policy 0, policy_version 439611 (0.0010) [2023-12-26 18:33:54,533][105620] Updated weights for policy 1, policy_version 439944 (0.0010) [2023-12-26 18:33:54,557][105692] Updated weights for policy 0, policy_version 439621 (0.0010) [2023-12-26 18:33:55,250][105692] Updated weights for policy 0, policy_version 439631 (0.0010) [2023-12-26 18:33:55,275][105620] Updated weights for policy 1, policy_version 439954 (0.0011) [2023-12-26 18:33:55,307][105692] Updated weights for policy 0, policy_version 439641 (0.0011) [2023-12-26 18:33:55,335][105620] Updated weights for policy 1, policy_version 439964 (0.0011) [2023-12-26 18:33:55,356][105692] Updated weights for policy 0, policy_version 439651 (0.0010) [2023-12-26 18:33:55,396][105620] Updated weights for policy 1, policy_version 439974 (0.0010) [2023-12-26 18:33:56,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 225214464. Throughput: 0: 9695.4, 1: 9674.3. Samples: 225228440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:33:56,063][104569] Avg episode reward: [(0, '8008.190'), (1, '9263.632')] [2023-12-26 18:33:56,085][105620] Updated weights for policy 1, policy_version 439984 (0.0011) [2023-12-26 18:33:56,115][105692] Updated weights for policy 0, policy_version 439661 (0.0010) [2023-12-26 18:33:56,140][105620] Updated weights for policy 1, policy_version 439994 (0.0010) [2023-12-26 18:33:56,160][105692] Updated weights for policy 0, policy_version 439671 (0.0010) [2023-12-26 18:33:56,195][105620] Updated weights for policy 1, policy_version 440004 (0.0010) [2023-12-26 18:33:56,211][105692] Updated weights for policy 0, policy_version 439681 (0.0010) [2023-12-26 18:33:56,896][105692] Updated weights for policy 0, policy_version 439691 (0.0010) [2023-12-26 18:33:56,934][105620] Updated weights for policy 1, policy_version 440014 (0.0007) [2023-12-26 18:33:56,956][105692] Updated weights for policy 0, policy_version 439701 (0.0009) [2023-12-26 18:33:56,986][105620] Updated weights for policy 1, policy_version 440024 (0.0005) [2023-12-26 18:33:57,008][105692] Updated weights for policy 0, policy_version 439711 (0.0010) [2023-12-26 18:33:57,046][105620] Updated weights for policy 1, policy_version 440034 (0.0005) [2023-12-26 18:33:57,588][105620] Updated weights for policy 1, policy_version 440044 (0.0005) [2023-12-26 18:33:57,634][105620] Updated weights for policy 1, policy_version 440054 (0.0005) [2023-12-26 18:33:57,682][105620] Updated weights for policy 1, policy_version 440064 (0.0006) [2023-12-26 18:33:57,707][105692] Updated weights for policy 0, policy_version 439721 (0.0010) [2023-12-26 18:33:57,771][105692] Updated weights for policy 0, policy_version 439731 (0.0010) [2023-12-26 18:33:57,832][105692] Updated weights for policy 0, policy_version 439741 (0.0010) [2023-12-26 18:33:57,899][105692] Updated weights for policy 0, policy_version 439751 (0.0010) [2023-12-26 18:33:58,293][105620] Updated weights for policy 1, policy_version 440074 (0.0007) [2023-12-26 18:33:58,357][105620] Updated weights for policy 1, policy_version 440084 (0.0008) [2023-12-26 18:33:58,376][105586] KL-divergence is very high: 188.5049 [2023-12-26 18:33:58,383][105586] KL-divergence is very high: 186.3347 [2023-12-26 18:33:58,397][105586] KL-divergence is very high: 221.6888 [2023-12-26 18:33:58,425][105620] Updated weights for policy 1, policy_version 440094 (0.0008) [2023-12-26 18:33:58,432][105586] KL-divergence is very high: 268.0808 [2023-12-26 18:33:58,440][105586] KL-divergence is very high: 256.5756 [2023-12-26 18:33:58,456][105586] KL-divergence is very high: 245.4616 [2023-12-26 18:33:58,488][105586] KL-divergence is very high: 250.5486 [2023-12-26 18:33:58,494][105620] Updated weights for policy 1, policy_version 440104 (0.0007) [2023-12-26 18:33:58,593][105692] Updated weights for policy 0, policy_version 439761 (0.0008) [2023-12-26 18:33:58,654][105692] Updated weights for policy 0, policy_version 439771 (0.0007) [2023-12-26 18:33:58,712][105692] Updated weights for policy 0, policy_version 439781 (0.0008) [2023-12-26 18:33:59,118][105586] KL-divergence is very high: 169.0321 [2023-12-26 18:33:59,166][105586] KL-divergence is very high: 137.1481 [2023-12-26 18:33:59,170][105620] Updated weights for policy 1, policy_version 440114 (0.0008) [2023-12-26 18:33:59,218][105586] KL-divergence is very high: 116.7818 [2023-12-26 18:33:59,247][105620] Updated weights for policy 1, policy_version 440124 (0.0007) [2023-12-26 18:33:59,310][105620] Updated weights for policy 1, policy_version 440134 (0.0009) [2023-12-26 18:33:59,539][105692] Updated weights for policy 0, policy_version 439791 (0.0008) [2023-12-26 18:33:59,596][105692] Updated weights for policy 0, policy_version 439801 (0.0005) [2023-12-26 18:33:59,649][105692] Updated weights for policy 0, policy_version 439811 (0.0007) [2023-12-26 18:34:00,043][105620] Updated weights for policy 1, policy_version 440144 (0.0008) [2023-12-26 18:34:00,096][105620] Updated weights for policy 1, policy_version 440154 (0.0006) [2023-12-26 18:34:00,145][105620] Updated weights for policy 1, policy_version 440164 (0.0010) [2023-12-26 18:34:00,374][105692] Updated weights for policy 0, policy_version 439821 (0.0009) [2023-12-26 18:34:00,432][105692] Updated weights for policy 0, policy_version 439831 (0.0005) [2023-12-26 18:34:00,485][105692] Updated weights for policy 0, policy_version 439841 (0.0007) [2023-12-26 18:34:00,940][105620] Updated weights for policy 1, policy_version 440174 (0.0008) [2023-12-26 18:34:00,988][105620] Updated weights for policy 1, policy_version 440184 (0.0010) [2023-12-26 18:34:01,042][105620] Updated weights for policy 1, policy_version 440194 (0.0011) [2023-12-26 18:34:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 225312768. Throughput: 0: 9725.5, 1: 9768.8. Samples: 225290216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:34:01,062][104569] Avg episode reward: [(0, '8780.128'), (1, '8895.202')] [2023-12-26 18:34:01,076][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000440200_112705536.pth... [2023-12-26 18:34:01,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000439080_112418816.pth [2023-12-26 18:34:01,090][105692] Updated weights for policy 0, policy_version 439851 (0.0006) [2023-12-26 18:34:01,157][105692] Updated weights for policy 0, policy_version 439861 (0.0009) [2023-12-26 18:34:01,223][105692] Updated weights for policy 0, policy_version 439871 (0.0008) [2023-12-26 18:34:01,282][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000439880_112623616.pth... [2023-12-26 18:34:01,288][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000438728_112328704.pth [2023-12-26 18:34:01,745][105620] Updated weights for policy 1, policy_version 440204 (0.0010) [2023-12-26 18:34:01,812][105620] Updated weights for policy 1, policy_version 440214 (0.0009) [2023-12-26 18:34:01,869][105620] Updated weights for policy 1, policy_version 440224 (0.0009) [2023-12-26 18:34:01,948][105692] Updated weights for policy 0, policy_version 439881 (0.0010) [2023-12-26 18:34:02,002][105692] Updated weights for policy 0, policy_version 439891 (0.0006) [2023-12-26 18:34:02,057][105692] Updated weights for policy 0, policy_version 439901 (0.0005) [2023-12-26 18:34:02,108][105692] Updated weights for policy 0, policy_version 439911 (0.0007) [2023-12-26 18:34:02,596][105620] Updated weights for policy 1, policy_version 440234 (0.0009) [2023-12-26 18:34:02,658][105620] Updated weights for policy 1, policy_version 440244 (0.0009) [2023-12-26 18:34:02,718][105620] Updated weights for policy 1, policy_version 440254 (0.0007) [2023-12-26 18:34:02,732][105692] Updated weights for policy 0, policy_version 439921 (0.0008) [2023-12-26 18:34:02,779][105620] Updated weights for policy 1, policy_version 440264 (0.0007) [2023-12-26 18:34:02,791][105692] Updated weights for policy 0, policy_version 439931 (0.0007) [2023-12-26 18:34:02,849][105692] Updated weights for policy 0, policy_version 439941 (0.0008) [2023-12-26 18:34:03,529][105620] Updated weights for policy 1, policy_version 440274 (0.0005) [2023-12-26 18:34:03,579][105620] Updated weights for policy 1, policy_version 440284 (0.0008) [2023-12-26 18:34:03,615][105692] Updated weights for policy 0, policy_version 439951 (0.0008) [2023-12-26 18:34:03,630][105620] Updated weights for policy 1, policy_version 440294 (0.0007) [2023-12-26 18:34:03,669][105692] Updated weights for policy 0, policy_version 439961 (0.0008) [2023-12-26 18:34:03,739][105692] Updated weights for policy 0, policy_version 439971 (0.0009) [2023-12-26 18:34:04,260][105620] Updated weights for policy 1, policy_version 440304 (0.0006) [2023-12-26 18:34:04,326][105620] Updated weights for policy 1, policy_version 440314 (0.0006) [2023-12-26 18:34:04,391][105620] Updated weights for policy 1, policy_version 440324 (0.0006) [2023-12-26 18:34:04,600][105692] Updated weights for policy 0, policy_version 439981 (0.0010) [2023-12-26 18:34:04,667][105692] Updated weights for policy 0, policy_version 439991 (0.0008) [2023-12-26 18:34:04,731][105692] Updated weights for policy 0, policy_version 440001 (0.0008) [2023-12-26 18:34:05,002][105620] Updated weights for policy 1, policy_version 440334 (0.0008) [2023-12-26 18:34:05,067][105620] Updated weights for policy 1, policy_version 440344 (0.0008) [2023-12-26 18:34:05,127][105620] Updated weights for policy 1, policy_version 440354 (0.0008) [2023-12-26 18:34:05,450][105692] Updated weights for policy 0, policy_version 440011 (0.0008) [2023-12-26 18:34:05,503][105692] Updated weights for policy 0, policy_version 440021 (0.0009) [2023-12-26 18:34:05,560][105692] Updated weights for policy 0, policy_version 440031 (0.0009) [2023-12-26 18:34:05,766][105620] Updated weights for policy 1, policy_version 440364 (0.0009) [2023-12-26 18:34:05,821][105620] Updated weights for policy 1, policy_version 440374 (0.0009) [2023-12-26 18:34:05,890][105620] Updated weights for policy 1, policy_version 440384 (0.0009) [2023-12-26 18:34:06,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 225419264. Throughput: 0: 9673.2, 1: 9807.5. Samples: 225406428. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 18:34:06,063][104569] Avg episode reward: [(0, '9157.773'), (1, '8789.784')] [2023-12-26 18:34:06,353][105692] Updated weights for policy 0, policy_version 440041 (0.0009) [2023-12-26 18:34:06,412][105692] Updated weights for policy 0, policy_version 440051 (0.0009) [2023-12-26 18:34:06,467][105692] Updated weights for policy 0, policy_version 440061 (0.0009) [2023-12-26 18:34:06,522][105692] Updated weights for policy 0, policy_version 440071 (0.0009) [2023-12-26 18:34:06,584][105620] Updated weights for policy 1, policy_version 440394 (0.0009) [2023-12-26 18:34:06,646][105620] Updated weights for policy 1, policy_version 440404 (0.0008) [2023-12-26 18:34:06,702][105620] Updated weights for policy 1, policy_version 440414 (0.0005) [2023-12-26 18:34:06,757][105620] Updated weights for policy 1, policy_version 440424 (0.0005) [2023-12-26 18:34:07,337][105692] Updated weights for policy 0, policy_version 440081 (0.0008) [2023-12-26 18:34:07,391][105692] Updated weights for policy 0, policy_version 440091 (0.0008) [2023-12-26 18:34:07,392][105620] Updated weights for policy 1, policy_version 440434 (0.0009) [2023-12-26 18:34:07,447][105692] Updated weights for policy 0, policy_version 440101 (0.0007) [2023-12-26 18:34:07,449][105620] Updated weights for policy 1, policy_version 440444 (0.0008) [2023-12-26 18:34:07,505][105620] Updated weights for policy 1, policy_version 440454 (0.0009) [2023-12-26 18:34:08,208][105692] Updated weights for policy 0, policy_version 440111 (0.0008) [2023-12-26 18:34:08,255][105620] Updated weights for policy 1, policy_version 440464 (0.0008) [2023-12-26 18:34:08,260][105692] Updated weights for policy 0, policy_version 440121 (0.0007) [2023-12-26 18:34:08,314][105620] Updated weights for policy 1, policy_version 440474 (0.0008) [2023-12-26 18:34:08,316][105692] Updated weights for policy 0, policy_version 440131 (0.0005) [2023-12-26 18:34:08,376][105620] Updated weights for policy 1, policy_version 440484 (0.0008) [2023-12-26 18:34:08,989][105692] Updated weights for policy 0, policy_version 440141 (0.0008) [2023-12-26 18:34:09,044][105692] Updated weights for policy 0, policy_version 440151 (0.0010) [2023-12-26 18:34:09,093][105692] Updated weights for policy 0, policy_version 440161 (0.0008) [2023-12-26 18:34:09,113][105620] Updated weights for policy 1, policy_version 440494 (0.0010) [2023-12-26 18:34:09,168][105620] Updated weights for policy 1, policy_version 440504 (0.0010) [2023-12-26 18:34:09,224][105620] Updated weights for policy 1, policy_version 440514 (0.0009) [2023-12-26 18:34:09,832][105692] Updated weights for policy 0, policy_version 440171 (0.0009) [2023-12-26 18:34:09,894][105692] Updated weights for policy 0, policy_version 440181 (0.0008) [2023-12-26 18:34:09,960][105692] Updated weights for policy 0, policy_version 440191 (0.0008) [2023-12-26 18:34:10,029][105620] Updated weights for policy 1, policy_version 440524 (0.0010) [2023-12-26 18:34:10,086][105620] Updated weights for policy 1, policy_version 440534 (0.0008) [2023-12-26 18:34:10,146][105620] Updated weights for policy 1, policy_version 440544 (0.0008) [2023-12-26 18:34:10,719][105692] Updated weights for policy 0, policy_version 440201 (0.0008) [2023-12-26 18:34:10,785][105692] Updated weights for policy 0, policy_version 440211 (0.0008) [2023-12-26 18:34:10,849][105692] Updated weights for policy 0, policy_version 440221 (0.0007) [2023-12-26 18:34:10,908][105692] Updated weights for policy 0, policy_version 440231 (0.0007) [2023-12-26 18:34:10,920][105620] Updated weights for policy 1, policy_version 440554 (0.0008) [2023-12-26 18:34:10,973][105620] Updated weights for policy 1, policy_version 440564 (0.0009) [2023-12-26 18:34:11,032][105620] Updated weights for policy 1, policy_version 440574 (0.0009) [2023-12-26 18:34:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19549.8). Total num frames: 225509376. Throughput: 0: 9663.0, 1: 9794.1. Samples: 225520532. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 18:34:11,062][104569] Avg episode reward: [(0, '9264.522'), (1, '9168.635')] [2023-12-26 18:34:11,099][105620] Updated weights for policy 1, policy_version 440584 (0.0009) [2023-12-26 18:34:11,605][105692] Updated weights for policy 0, policy_version 440241 (0.0006) [2023-12-26 18:34:11,674][105692] Updated weights for policy 0, policy_version 440251 (0.0007) [2023-12-26 18:34:11,742][105692] Updated weights for policy 0, policy_version 440261 (0.0008) [2023-12-26 18:34:11,954][105620] Updated weights for policy 1, policy_version 440594 (0.0006) [2023-12-26 18:34:12,039][105620] Updated weights for policy 1, policy_version 440604 (0.0009) [2023-12-26 18:34:12,096][105620] Updated weights for policy 1, policy_version 440614 (0.0009) [2023-12-26 18:34:12,306][105692] Updated weights for policy 0, policy_version 440271 (0.0008) [2023-12-26 18:34:12,374][105692] Updated weights for policy 0, policy_version 440281 (0.0009) [2023-12-26 18:34:12,443][105692] Updated weights for policy 0, policy_version 440291 (0.0010) [2023-12-26 18:34:12,842][105620] Updated weights for policy 1, policy_version 440624 (0.0009) [2023-12-26 18:34:12,903][105620] Updated weights for policy 1, policy_version 440634 (0.0009) [2023-12-26 18:34:12,970][105620] Updated weights for policy 1, policy_version 440644 (0.0008) [2023-12-26 18:34:13,162][105692] Updated weights for policy 0, policy_version 440301 (0.0009) [2023-12-26 18:34:13,221][105692] Updated weights for policy 0, policy_version 440311 (0.0009) [2023-12-26 18:34:13,268][105692] Updated weights for policy 0, policy_version 440321 (0.0009) [2023-12-26 18:34:13,674][105620] Updated weights for policy 1, policy_version 440654 (0.0010) [2023-12-26 18:34:13,727][105620] Updated weights for policy 1, policy_version 440664 (0.0008) [2023-12-26 18:34:13,782][105620] Updated weights for policy 1, policy_version 440674 (0.0009) [2023-12-26 18:34:13,981][105692] Updated weights for policy 0, policy_version 440331 (0.0008) [2023-12-26 18:34:14,038][105692] Updated weights for policy 0, policy_version 440341 (0.0007) [2023-12-26 18:34:14,086][105692] Updated weights for policy 0, policy_version 440351 (0.0010) [2023-12-26 18:34:14,611][105620] Updated weights for policy 1, policy_version 440684 (0.0011) [2023-12-26 18:34:14,659][105620] Updated weights for policy 1, policy_version 440694 (0.0007) [2023-12-26 18:34:14,707][105620] Updated weights for policy 1, policy_version 440704 (0.0008) [2023-12-26 18:34:14,761][105692] Updated weights for policy 0, policy_version 440361 (0.0010) [2023-12-26 18:34:14,828][105692] Updated weights for policy 0, policy_version 440371 (0.0009) [2023-12-26 18:34:14,899][105692] Updated weights for policy 0, policy_version 440381 (0.0009) [2023-12-26 18:34:14,965][105692] Updated weights for policy 0, policy_version 440391 (0.0009) [2023-12-26 18:34:15,446][105620] Updated weights for policy 1, policy_version 440714 (0.0009) [2023-12-26 18:34:15,514][105620] Updated weights for policy 1, policy_version 440724 (0.0010) [2023-12-26 18:34:15,575][105620] Updated weights for policy 1, policy_version 440734 (0.0010) [2023-12-26 18:34:15,633][105620] Updated weights for policy 1, policy_version 440744 (0.0010) [2023-12-26 18:34:15,722][105692] Updated weights for policy 0, policy_version 440401 (0.0009) [2023-12-26 18:34:15,767][105692] Updated weights for policy 0, policy_version 440411 (0.0007) [2023-12-26 18:34:15,822][105692] Updated weights for policy 0, policy_version 440421 (0.0005) [2023-12-26 18:34:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19521.9). Total num frames: 225607680. Throughput: 0: 9681.2, 1: 9628.2. Samples: 225576092. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 18:34:16,063][104569] Avg episode reward: [(0, '9183.975'), (1, '9084.150')] [2023-12-26 18:34:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000440424_112762880.pth... [2023-12-26 18:34:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000440744_112844800.pth... [2023-12-26 18:34:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000439304_112476160.pth [2023-12-26 18:34:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000439624_112558080.pth [2023-12-26 18:34:16,263][105620] Updated weights for policy 1, policy_version 440754 (0.0006) [2023-12-26 18:34:16,322][105620] Updated weights for policy 1, policy_version 440764 (0.0006) [2023-12-26 18:34:16,371][105692] Updated weights for policy 0, policy_version 440431 (0.0007) [2023-12-26 18:34:16,381][105620] Updated weights for policy 1, policy_version 440774 (0.0007) [2023-12-26 18:34:16,421][105692] Updated weights for policy 0, policy_version 440441 (0.0007) [2023-12-26 18:34:16,477][105692] Updated weights for policy 0, policy_version 440451 (0.0008) [2023-12-26 18:34:17,035][105620] Updated weights for policy 1, policy_version 440784 (0.0009) [2023-12-26 18:34:17,097][105620] Updated weights for policy 1, policy_version 440794 (0.0007) [2023-12-26 18:34:17,160][105620] Updated weights for policy 1, policy_version 440804 (0.0007) [2023-12-26 18:34:17,274][105692] Updated weights for policy 0, policy_version 440461 (0.0009) [2023-12-26 18:34:17,329][105692] Updated weights for policy 0, policy_version 440471 (0.0008) [2023-12-26 18:34:17,387][105692] Updated weights for policy 0, policy_version 440481 (0.0008) [2023-12-26 18:34:17,823][105620] Updated weights for policy 1, policy_version 440814 (0.0005) [2023-12-26 18:34:17,881][105620] Updated weights for policy 1, policy_version 440824 (0.0005) [2023-12-26 18:34:17,934][105620] Updated weights for policy 1, policy_version 440834 (0.0005) [2023-12-26 18:34:18,186][105692] Updated weights for policy 0, policy_version 440491 (0.0009) [2023-12-26 18:34:18,237][105692] Updated weights for policy 0, policy_version 440501 (0.0009) [2023-12-26 18:34:18,293][105692] Updated weights for policy 0, policy_version 440511 (0.0010) [2023-12-26 18:34:18,529][105620] Updated weights for policy 1, policy_version 440844 (0.0005) [2023-12-26 18:34:18,588][105620] Updated weights for policy 1, policy_version 440854 (0.0005) [2023-12-26 18:34:18,642][105620] Updated weights for policy 1, policy_version 440864 (0.0008) [2023-12-26 18:34:19,145][105692] Updated weights for policy 0, policy_version 440521 (0.0009) [2023-12-26 18:34:19,203][105692] Updated weights for policy 0, policy_version 440531 (0.0009) [2023-12-26 18:34:19,262][105692] Updated weights for policy 0, policy_version 440541 (0.0009) [2023-12-26 18:34:19,325][105692] Updated weights for policy 0, policy_version 440551 (0.0009) [2023-12-26 18:34:19,333][105620] Updated weights for policy 1, policy_version 440874 (0.0009) [2023-12-26 18:34:19,398][105620] Updated weights for policy 1, policy_version 440884 (0.0008) [2023-12-26 18:34:19,459][105620] Updated weights for policy 1, policy_version 440894 (0.0009) [2023-12-26 18:34:19,529][105620] Updated weights for policy 1, policy_version 440904 (0.0008) [2023-12-26 18:34:20,165][105692] Updated weights for policy 0, policy_version 440561 (0.0009) [2023-12-26 18:34:20,184][105620] Updated weights for policy 1, policy_version 440914 (0.0007) [2023-12-26 18:34:20,223][105692] Updated weights for policy 0, policy_version 440571 (0.0007) [2023-12-26 18:34:20,246][105620] Updated weights for policy 1, policy_version 440924 (0.0007) [2023-12-26 18:34:20,285][105692] Updated weights for policy 0, policy_version 440581 (0.0007) [2023-12-26 18:34:20,307][105620] Updated weights for policy 1, policy_version 440934 (0.0007) [2023-12-26 18:34:21,010][105620] Updated weights for policy 1, policy_version 440944 (0.0009) [2023-12-26 18:34:21,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 225697792. Throughput: 0: 9692.0, 1: 9719.6. Samples: 225695088. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 18:34:21,063][104569] Avg episode reward: [(0, '9103.605'), (1, '8999.425')] [2023-12-26 18:34:21,081][105620] Updated weights for policy 1, policy_version 440954 (0.0007) [2023-12-26 18:34:21,086][105692] Updated weights for policy 0, policy_version 440591 (0.0007) [2023-12-26 18:34:21,141][105620] Updated weights for policy 1, policy_version 440964 (0.0009) [2023-12-26 18:34:21,146][105692] Updated weights for policy 0, policy_version 440601 (0.0008) [2023-12-26 18:34:21,209][105692] Updated weights for policy 0, policy_version 440611 (0.0009) [2023-12-26 18:34:21,961][105620] Updated weights for policy 1, policy_version 440974 (0.0007) [2023-12-26 18:34:21,973][105692] Updated weights for policy 0, policy_version 440621 (0.0010) [2023-12-26 18:34:22,022][105620] Updated weights for policy 1, policy_version 440984 (0.0007) [2023-12-26 18:34:22,033][105692] Updated weights for policy 0, policy_version 440631 (0.0007) [2023-12-26 18:34:22,086][105620] Updated weights for policy 1, policy_version 440994 (0.0007) [2023-12-26 18:34:22,096][105692] Updated weights for policy 0, policy_version 440641 (0.0006) [2023-12-26 18:34:22,822][105620] Updated weights for policy 1, policy_version 441004 (0.0006) [2023-12-26 18:34:22,887][105620] Updated weights for policy 1, policy_version 441014 (0.0008) [2023-12-26 18:34:22,909][105692] Updated weights for policy 0, policy_version 440651 (0.0009) [2023-12-26 18:34:22,942][105620] Updated weights for policy 1, policy_version 441024 (0.0009) [2023-12-26 18:34:22,962][105692] Updated weights for policy 0, policy_version 440661 (0.0010) [2023-12-26 18:34:23,011][105692] Updated weights for policy 0, policy_version 440671 (0.0007) [2023-12-26 18:34:23,622][105620] Updated weights for policy 1, policy_version 441034 (0.0008) [2023-12-26 18:34:23,686][105620] Updated weights for policy 1, policy_version 441044 (0.0009) [2023-12-26 18:34:23,752][105620] Updated weights for policy 1, policy_version 441054 (0.0009) [2023-12-26 18:34:23,800][105692] Updated weights for policy 0, policy_version 440681 (0.0008) [2023-12-26 18:34:23,813][105620] Updated weights for policy 1, policy_version 441064 (0.0009) [2023-12-26 18:34:23,867][105692] Updated weights for policy 0, policy_version 440691 (0.0005) [2023-12-26 18:34:23,935][105692] Updated weights for policy 0, policy_version 440701 (0.0005) [2023-12-26 18:34:24,003][105692] Updated weights for policy 0, policy_version 440711 (0.0005) [2023-12-26 18:34:24,540][105620] Updated weights for policy 1, policy_version 441074 (0.0009) [2023-12-26 18:34:24,593][105620] Updated weights for policy 1, policy_version 441084 (0.0009) [2023-12-26 18:34:24,615][105692] Updated weights for policy 0, policy_version 440721 (0.0005) [2023-12-26 18:34:24,649][105620] Updated weights for policy 1, policy_version 441094 (0.0009) [2023-12-26 18:34:24,672][105692] Updated weights for policy 0, policy_version 440731 (0.0005) [2023-12-26 18:34:24,727][105692] Updated weights for policy 0, policy_version 440741 (0.0005) [2023-12-26 18:34:25,261][105692] Updated weights for policy 0, policy_version 440751 (0.0008) [2023-12-26 18:34:25,332][105692] Updated weights for policy 0, policy_version 440761 (0.0005) [2023-12-26 18:34:25,399][105692] Updated weights for policy 0, policy_version 440771 (0.0005) [2023-12-26 18:34:25,531][105620] Updated weights for policy 1, policy_version 441104 (0.0007) [2023-12-26 18:34:25,577][105620] Updated weights for policy 1, policy_version 441114 (0.0008) [2023-12-26 18:34:25,633][105620] Updated weights for policy 1, policy_version 441124 (0.0006) [2023-12-26 18:34:25,961][105692] Updated weights for policy 0, policy_version 440781 (0.0008) [2023-12-26 18:34:26,022][105692] Updated weights for policy 0, policy_version 440791 (0.0010) [2023-12-26 18:34:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 225796096. Throughput: 0: 9642.0, 1: 9628.5. Samples: 225808116. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 18:34:26,063][104569] Avg episode reward: [(0, '8955.372'), (1, '9262.253')] [2023-12-26 18:34:26,079][105692] Updated weights for policy 0, policy_version 440801 (0.0008) [2023-12-26 18:34:26,367][105620] Updated weights for policy 1, policy_version 441134 (0.0008) [2023-12-26 18:34:26,420][105620] Updated weights for policy 1, policy_version 441144 (0.0009) [2023-12-26 18:34:26,478][105620] Updated weights for policy 1, policy_version 441154 (0.0010) [2023-12-26 18:34:26,636][105692] Updated weights for policy 0, policy_version 440811 (0.0009) [2023-12-26 18:34:26,688][105692] Updated weights for policy 0, policy_version 440821 (0.0005) [2023-12-26 18:34:26,741][105692] Updated weights for policy 0, policy_version 440831 (0.0005) [2023-12-26 18:34:27,224][105620] Updated weights for policy 1, policy_version 441165 (0.0009) [2023-12-26 18:34:27,277][105620] Updated weights for policy 1, policy_version 441175 (0.0006) [2023-12-26 18:34:27,329][105620] Updated weights for policy 1, policy_version 441185 (0.0006) [2023-12-26 18:34:27,398][105692] Updated weights for policy 0, policy_version 440841 (0.0008) [2023-12-26 18:34:27,455][105692] Updated weights for policy 0, policy_version 440851 (0.0010) [2023-12-26 18:34:27,512][105692] Updated weights for policy 0, policy_version 440861 (0.0010) [2023-12-26 18:34:27,569][105692] Updated weights for policy 0, policy_version 440871 (0.0010) [2023-12-26 18:34:28,057][105620] Updated weights for policy 1, policy_version 441195 (0.0006) [2023-12-26 18:34:28,109][105620] Updated weights for policy 1, policy_version 441205 (0.0010) [2023-12-26 18:34:28,135][105692] Updated weights for policy 0, policy_version 440881 (0.0006) [2023-12-26 18:34:28,163][105620] Updated weights for policy 1, policy_version 441215 (0.0010) [2023-12-26 18:34:28,195][105692] Updated weights for policy 0, policy_version 440891 (0.0005) [2023-12-26 18:34:28,255][105692] Updated weights for policy 0, policy_version 440901 (0.0007) [2023-12-26 18:34:28,813][105692] Updated weights for policy 0, policy_version 440911 (0.0010) [2023-12-26 18:34:28,843][105620] Updated weights for policy 1, policy_version 441225 (0.0008) [2023-12-26 18:34:28,868][105692] Updated weights for policy 0, policy_version 440921 (0.0010) [2023-12-26 18:34:28,898][105620] Updated weights for policy 1, policy_version 441235 (0.0005) [2023-12-26 18:34:28,916][105692] Updated weights for policy 0, policy_version 440931 (0.0010) [2023-12-26 18:34:28,949][105620] Updated weights for policy 1, policy_version 441245 (0.0005) [2023-12-26 18:34:29,012][105620] Updated weights for policy 1, policy_version 441255 (0.0005) [2023-12-26 18:34:29,620][105692] Updated weights for policy 0, policy_version 440941 (0.0008) [2023-12-26 18:34:29,644][105620] Updated weights for policy 1, policy_version 441265 (0.0005) [2023-12-26 18:34:29,685][105692] Updated weights for policy 0, policy_version 440951 (0.0005) [2023-12-26 18:34:29,704][105620] Updated weights for policy 1, policy_version 441275 (0.0005) [2023-12-26 18:34:29,745][105692] Updated weights for policy 0, policy_version 440961 (0.0006) [2023-12-26 18:34:29,761][105620] Updated weights for policy 1, policy_version 441285 (0.0006) [2023-12-26 18:34:30,380][105692] Updated weights for policy 0, policy_version 440971 (0.0007) [2023-12-26 18:34:30,428][105692] Updated weights for policy 0, policy_version 440981 (0.0010) [2023-12-26 18:34:30,436][105620] Updated weights for policy 1, policy_version 441295 (0.0010) [2023-12-26 18:34:30,482][105692] Updated weights for policy 0, policy_version 440991 (0.0010) [2023-12-26 18:34:30,491][105620] Updated weights for policy 1, policy_version 441305 (0.0010) [2023-12-26 18:34:30,538][105620] Updated weights for policy 1, policy_version 441315 (0.0010) [2023-12-26 18:34:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 225902592. Throughput: 0: 9768.8, 1: 9683.1. Samples: 225872108. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 18:34:31,062][104569] Avg episode reward: [(0, '9016.508'), (1, '9086.464')] [2023-12-26 18:34:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000441000_112910336.pth... [2023-12-26 18:34:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000441320_112992256.pth... [2023-12-26 18:34:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000439880_112623616.pth [2023-12-26 18:34:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000440200_112705536.pth [2023-12-26 18:34:31,210][105620] Updated weights for policy 1, policy_version 441325 (0.0010) [2023-12-26 18:34:31,225][105692] Updated weights for policy 0, policy_version 441001 (0.0010) [2023-12-26 18:34:31,272][105620] Updated weights for policy 1, policy_version 441335 (0.0011) [2023-12-26 18:34:31,291][105692] Updated weights for policy 0, policy_version 441011 (0.0010) [2023-12-26 18:34:31,332][105620] Updated weights for policy 1, policy_version 441345 (0.0010) [2023-12-26 18:34:31,346][105692] Updated weights for policy 0, policy_version 441021 (0.0011) [2023-12-26 18:34:31,411][105692] Updated weights for policy 0, policy_version 441031 (0.0011) [2023-12-26 18:34:31,989][105620] Updated weights for policy 1, policy_version 441355 (0.0010) [2023-12-26 18:34:32,039][105692] Updated weights for policy 0, policy_version 441041 (0.0010) [2023-12-26 18:34:32,047][105620] Updated weights for policy 1, policy_version 441365 (0.0010) [2023-12-26 18:34:32,087][105692] Updated weights for policy 0, policy_version 441051 (0.0010) [2023-12-26 18:34:32,106][105620] Updated weights for policy 1, policy_version 441375 (0.0011) [2023-12-26 18:34:32,139][105692] Updated weights for policy 0, policy_version 441061 (0.0010) [2023-12-26 18:34:32,747][105620] Updated weights for policy 1, policy_version 441385 (0.0010) [2023-12-26 18:34:32,805][105620] Updated weights for policy 1, policy_version 441395 (0.0005) [2023-12-26 18:34:32,867][105620] Updated weights for policy 1, policy_version 441405 (0.0005) [2023-12-26 18:34:32,889][105692] Updated weights for policy 0, policy_version 441071 (0.0010) [2023-12-26 18:34:32,924][105620] Updated weights for policy 1, policy_version 441415 (0.0008) [2023-12-26 18:34:32,939][105692] Updated weights for policy 0, policy_version 441081 (0.0006) [2023-12-26 18:34:33,002][105692] Updated weights for policy 0, policy_version 441091 (0.0005) [2023-12-26 18:34:33,529][105620] Updated weights for policy 1, policy_version 441425 (0.0006) [2023-12-26 18:34:33,556][105692] Updated weights for policy 0, policy_version 441101 (0.0008) [2023-12-26 18:34:33,578][105620] Updated weights for policy 1, policy_version 441435 (0.0005) [2023-12-26 18:34:33,609][105692] Updated weights for policy 0, policy_version 441111 (0.0011) [2023-12-26 18:34:33,629][105620] Updated weights for policy 1, policy_version 441445 (0.0005) [2023-12-26 18:34:33,663][105692] Updated weights for policy 0, policy_version 441121 (0.0009) [2023-12-26 18:34:34,255][105620] Updated weights for policy 1, policy_version 441455 (0.0006) [2023-12-26 18:34:34,308][105620] Updated weights for policy 1, policy_version 441465 (0.0008) [2023-12-26 18:34:34,362][105620] Updated weights for policy 1, policy_version 441475 (0.0006) [2023-12-26 18:34:34,384][105692] Updated weights for policy 0, policy_version 441131 (0.0007) [2023-12-26 18:34:34,440][105692] Updated weights for policy 0, policy_version 441143 (0.0010) [2023-12-26 18:34:34,499][105692] Updated weights for policy 0, policy_version 441153 (0.0009) [2023-12-26 18:34:34,963][105620] Updated weights for policy 1, policy_version 441485 (0.0005) [2023-12-26 18:34:35,020][105620] Updated weights for policy 1, policy_version 441495 (0.0005) [2023-12-26 18:34:35,088][105620] Updated weights for policy 1, policy_version 441505 (0.0005) [2023-12-26 18:34:35,222][105692] Updated weights for policy 0, policy_version 441163 (0.0010) [2023-12-26 18:34:35,281][105692] Updated weights for policy 0, policy_version 441173 (0.0010) [2023-12-26 18:34:35,343][105692] Updated weights for policy 0, policy_version 441183 (0.0010) [2023-12-26 18:34:35,635][105620] Updated weights for policy 1, policy_version 441515 (0.0005) [2023-12-26 18:34:35,697][105620] Updated weights for policy 1, policy_version 441525 (0.0007) [2023-12-26 18:34:35,750][105620] Updated weights for policy 1, policy_version 441535 (0.0005) [2023-12-26 18:34:36,062][104569] Fps is (10 sec: 21299.4, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 226009088. Throughput: 0: 9779.0, 1: 9886.2. Samples: 225997824. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 18:34:36,062][104569] Avg episode reward: [(0, '9269.589'), (1, '8914.023')] [2023-12-26 18:34:36,085][105692] Updated weights for policy 0, policy_version 441193 (0.0008) [2023-12-26 18:34:36,145][105692] Updated weights for policy 0, policy_version 441203 (0.0009) [2023-12-26 18:34:36,204][105692] Updated weights for policy 0, policy_version 441213 (0.0009) [2023-12-26 18:34:36,254][105692] Updated weights for policy 0, policy_version 441223 (0.0009) [2023-12-26 18:34:36,455][105620] Updated weights for policy 1, policy_version 441545 (0.0007) [2023-12-26 18:34:36,510][105620] Updated weights for policy 1, policy_version 441555 (0.0010) [2023-12-26 18:34:36,570][105620] Updated weights for policy 1, policy_version 441565 (0.0010) [2023-12-26 18:34:36,621][105620] Updated weights for policy 1, policy_version 441575 (0.0009) [2023-12-26 18:34:36,936][105692] Updated weights for policy 0, policy_version 441233 (0.0009) [2023-12-26 18:34:36,995][105692] Updated weights for policy 0, policy_version 441243 (0.0008) [2023-12-26 18:34:37,051][105692] Updated weights for policy 0, policy_version 441253 (0.0008) [2023-12-26 18:34:37,379][105620] Updated weights for policy 1, policy_version 441585 (0.0009) [2023-12-26 18:34:37,427][105620] Updated weights for policy 1, policy_version 441595 (0.0009) [2023-12-26 18:34:37,483][105620] Updated weights for policy 1, policy_version 441605 (0.0009) [2023-12-26 18:34:37,818][105692] Updated weights for policy 0, policy_version 441263 (0.0008) [2023-12-26 18:34:37,871][105692] Updated weights for policy 0, policy_version 441273 (0.0008) [2023-12-26 18:34:37,930][105692] Updated weights for policy 0, policy_version 441283 (0.0008) [2023-12-26 18:34:38,262][105620] Updated weights for policy 1, policy_version 441615 (0.0007) [2023-12-26 18:34:38,316][105620] Updated weights for policy 1, policy_version 441625 (0.0005) [2023-12-26 18:34:38,377][105620] Updated weights for policy 1, policy_version 441635 (0.0010) [2023-12-26 18:34:38,637][105692] Updated weights for policy 0, policy_version 441293 (0.0008) [2023-12-26 18:34:38,686][105692] Updated weights for policy 0, policy_version 441303 (0.0010) [2023-12-26 18:34:38,734][105692] Updated weights for policy 0, policy_version 441313 (0.0010) [2023-12-26 18:34:39,088][105620] Updated weights for policy 1, policy_version 441645 (0.0011) [2023-12-26 18:34:39,147][105620] Updated weights for policy 1, policy_version 441655 (0.0011) [2023-12-26 18:34:39,203][105620] Updated weights for policy 1, policy_version 441665 (0.0011) [2023-12-26 18:34:39,509][105692] Updated weights for policy 0, policy_version 441323 (0.0010) [2023-12-26 18:34:39,574][105692] Updated weights for policy 0, policy_version 441333 (0.0007) [2023-12-26 18:34:39,642][105692] Updated weights for policy 0, policy_version 441343 (0.0007) [2023-12-26 18:34:39,981][105620] Updated weights for policy 1, policy_version 441675 (0.0010) [2023-12-26 18:34:40,037][105620] Updated weights for policy 1, policy_version 441685 (0.0008) [2023-12-26 18:34:40,100][105620] Updated weights for policy 1, policy_version 441695 (0.0008) [2023-12-26 18:34:40,343][105692] Updated weights for policy 0, policy_version 441353 (0.0008) [2023-12-26 18:34:40,405][105692] Updated weights for policy 0, policy_version 441363 (0.0010) [2023-12-26 18:34:40,470][105692] Updated weights for policy 0, policy_version 441373 (0.0010) [2023-12-26 18:34:40,539][105692] Updated weights for policy 0, policy_version 441383 (0.0010) [2023-12-26 18:34:40,872][105620] Updated weights for policy 1, policy_version 441705 (0.0008) [2023-12-26 18:34:40,925][105620] Updated weights for policy 1, policy_version 441715 (0.0010) [2023-12-26 18:34:40,987][105620] Updated weights for policy 1, policy_version 441725 (0.0010) [2023-12-26 18:34:41,053][105620] Updated weights for policy 1, policy_version 441735 (0.0011) [2023-12-26 18:34:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 226107392. Throughput: 0: 9792.3, 1: 9872.3. Samples: 226113344. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 18:34:41,062][104569] Avg episode reward: [(0, '9265.973'), (1, '9004.458')] [2023-12-26 18:34:41,273][105692] Updated weights for policy 0, policy_version 441393 (0.0011) [2023-12-26 18:34:41,336][105692] Updated weights for policy 0, policy_version 441403 (0.0011) [2023-12-26 18:34:41,405][105692] Updated weights for policy 0, policy_version 441413 (0.0011) [2023-12-26 18:34:41,821][105620] Updated weights for policy 1, policy_version 441745 (0.0009) [2023-12-26 18:34:41,888][105620] Updated weights for policy 1, policy_version 441755 (0.0008) [2023-12-26 18:34:41,953][105620] Updated weights for policy 1, policy_version 441765 (0.0009) [2023-12-26 18:34:42,125][105692] Updated weights for policy 0, policy_version 441423 (0.0008) [2023-12-26 18:34:42,188][105692] Updated weights for policy 0, policy_version 441433 (0.0011) [2023-12-26 18:34:42,237][105692] Updated weights for policy 0, policy_version 441443 (0.0010) [2023-12-26 18:34:42,711][105620] Updated weights for policy 1, policy_version 441775 (0.0006) [2023-12-26 18:34:42,775][105620] Updated weights for policy 1, policy_version 441785 (0.0009) [2023-12-26 18:34:42,831][105620] Updated weights for policy 1, policy_version 441795 (0.0010) [2023-12-26 18:34:42,904][105692] Updated weights for policy 0, policy_version 441453 (0.0008) [2023-12-26 18:34:42,972][105692] Updated weights for policy 0, policy_version 441463 (0.0009) [2023-12-26 18:34:43,027][105692] Updated weights for policy 0, policy_version 441473 (0.0010) [2023-12-26 18:34:43,574][105620] Updated weights for policy 1, policy_version 441805 (0.0011) [2023-12-26 18:34:43,641][105620] Updated weights for policy 1, policy_version 441815 (0.0009) [2023-12-26 18:34:43,700][105620] Updated weights for policy 1, policy_version 441825 (0.0010) [2023-12-26 18:34:43,740][105692] Updated weights for policy 0, policy_version 441483 (0.0010) [2023-12-26 18:34:43,798][105692] Updated weights for policy 0, policy_version 441493 (0.0010) [2023-12-26 18:34:43,863][105692] Updated weights for policy 0, policy_version 441503 (0.0010) [2023-12-26 18:34:44,313][105620] Updated weights for policy 1, policy_version 441835 (0.0010) [2023-12-26 18:34:44,364][105620] Updated weights for policy 1, policy_version 441845 (0.0010) [2023-12-26 18:34:44,422][105620] Updated weights for policy 1, policy_version 441855 (0.0010) [2023-12-26 18:34:44,603][105692] Updated weights for policy 0, policy_version 441513 (0.0010) [2023-12-26 18:34:44,651][105692] Updated weights for policy 0, policy_version 441523 (0.0008) [2023-12-26 18:34:44,703][105692] Updated weights for policy 0, policy_version 441533 (0.0008) [2023-12-26 18:34:44,754][105692] Updated weights for policy 0, policy_version 441543 (0.0008) [2023-12-26 18:34:45,170][105620] Updated weights for policy 1, policy_version 441865 (0.0011) [2023-12-26 18:34:45,233][105620] Updated weights for policy 1, policy_version 441875 (0.0011) [2023-12-26 18:34:45,288][105620] Updated weights for policy 1, policy_version 441885 (0.0010) [2023-12-26 18:34:45,350][105620] Updated weights for policy 1, policy_version 441895 (0.0010) [2023-12-26 18:34:45,530][105692] Updated weights for policy 0, policy_version 441553 (0.0006) [2023-12-26 18:34:45,588][105692] Updated weights for policy 0, policy_version 441563 (0.0005) [2023-12-26 18:34:45,649][105692] Updated weights for policy 0, policy_version 441573 (0.0005) [2023-12-26 18:34:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 226197504. Throughput: 0: 9778.4, 1: 9774.1. Samples: 226170080. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 18:34:46,062][104569] Avg episode reward: [(0, '8701.301'), (1, '9122.288')] [2023-12-26 18:34:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000441576_113057792.pth... [2023-12-26 18:34:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000440424_112762880.pth [2023-12-26 18:34:46,128][105692] Updated weights for policy 0, policy_version 441583 (0.0006) [2023-12-26 18:34:46,134][105620] Updated weights for policy 1, policy_version 441905 (0.0008) [2023-12-26 18:34:46,182][105692] Updated weights for policy 0, policy_version 441593 (0.0005) [2023-12-26 18:34:46,189][105620] Updated weights for policy 1, policy_version 441915 (0.0006) [2023-12-26 18:34:46,231][105692] Updated weights for policy 0, policy_version 441603 (0.0005) [2023-12-26 18:34:46,250][105620] Updated weights for policy 1, policy_version 441925 (0.0008) [2023-12-26 18:34:46,266][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000441928_113147904.pth... [2023-12-26 18:34:46,272][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000440744_112844800.pth [2023-12-26 18:34:46,802][105620] Updated weights for policy 1, policy_version 441935 (0.0010) [2023-12-26 18:34:46,857][105620] Updated weights for policy 1, policy_version 441945 (0.0010) [2023-12-26 18:34:46,913][105620] Updated weights for policy 1, policy_version 441955 (0.0010) [2023-12-26 18:34:46,953][105692] Updated weights for policy 0, policy_version 441613 (0.0005) [2023-12-26 18:34:47,001][105692] Updated weights for policy 0, policy_version 441623 (0.0006) [2023-12-26 18:34:47,066][105692] Updated weights for policy 0, policy_version 441633 (0.0010) [2023-12-26 18:34:47,502][105620] Updated weights for policy 1, policy_version 441965 (0.0008) [2023-12-26 18:34:47,561][105620] Updated weights for policy 1, policy_version 441975 (0.0005) [2023-12-26 18:34:47,607][105620] Updated weights for policy 1, policy_version 441985 (0.0005) [2023-12-26 18:34:47,779][105692] Updated weights for policy 0, policy_version 441643 (0.0011) [2023-12-26 18:34:47,833][105692] Updated weights for policy 0, policy_version 441653 (0.0010) [2023-12-26 18:34:47,850][105585] KL-divergence is very high: 114.9980 [2023-12-26 18:34:47,888][105692] Updated weights for policy 0, policy_version 441663 (0.0010) [2023-12-26 18:34:47,894][105585] KL-divergence is very high: 119.7131 [2023-12-26 18:34:48,128][105620] Updated weights for policy 1, policy_version 441995 (0.0005) [2023-12-26 18:34:48,182][105620] Updated weights for policy 1, policy_version 442005 (0.0005) [2023-12-26 18:34:48,237][105620] Updated weights for policy 1, policy_version 442015 (0.0005) [2023-12-26 18:34:48,694][105692] Updated weights for policy 0, policy_version 441673 (0.0010) [2023-12-26 18:34:48,748][105692] Updated weights for policy 0, policy_version 441683 (0.0010) [2023-12-26 18:34:48,779][105620] Updated weights for policy 1, policy_version 442025 (0.0006) [2023-12-26 18:34:48,810][105692] Updated weights for policy 0, policy_version 441693 (0.0006) [2023-12-26 18:34:48,828][105620] Updated weights for policy 1, policy_version 442035 (0.0010) [2023-12-26 18:34:48,870][105692] Updated weights for policy 0, policy_version 441703 (0.0006) [2023-12-26 18:34:48,887][105620] Updated weights for policy 1, policy_version 442045 (0.0010) [2023-12-26 18:34:48,949][105620] Updated weights for policy 1, policy_version 442055 (0.0010) [2023-12-26 18:34:49,617][105692] Updated weights for policy 0, policy_version 441713 (0.0006) [2023-12-26 18:34:49,679][105692] Updated weights for policy 0, policy_version 441723 (0.0006) [2023-12-26 18:34:49,689][105620] Updated weights for policy 1, policy_version 442065 (0.0008) [2023-12-26 18:34:49,742][105692] Updated weights for policy 0, policy_version 441733 (0.0008) [2023-12-26 18:34:49,751][105620] Updated weights for policy 1, policy_version 442075 (0.0010) [2023-12-26 18:34:49,809][105620] Updated weights for policy 1, policy_version 442085 (0.0010) [2023-12-26 18:34:50,352][105692] Updated weights for policy 0, policy_version 441743 (0.0008) [2023-12-26 18:34:50,408][105692] Updated weights for policy 0, policy_version 441753 (0.0009) [2023-12-26 18:34:50,471][105692] Updated weights for policy 0, policy_version 441763 (0.0009) [2023-12-26 18:34:50,619][105620] Updated weights for policy 1, policy_version 442095 (0.0009) [2023-12-26 18:34:50,682][105620] Updated weights for policy 1, policy_version 442105 (0.0010) [2023-12-26 18:34:50,746][105620] Updated weights for policy 1, policy_version 442115 (0.0011) [2023-12-26 18:34:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 226304000. Throughput: 0: 9819.1, 1: 9890.9. Samples: 226293376. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 18:34:51,062][104569] Avg episode reward: [(0, '8225.627'), (1, '8851.091')] [2023-12-26 18:34:51,266][105692] Updated weights for policy 0, policy_version 441773 (0.0009) [2023-12-26 18:34:51,324][105692] Updated weights for policy 0, policy_version 441783 (0.0008) [2023-12-26 18:34:51,394][105692] Updated weights for policy 0, policy_version 441793 (0.0009) [2023-12-26 18:34:51,554][105620] Updated weights for policy 1, policy_version 442125 (0.0011) [2023-12-26 18:34:51,624][105620] Updated weights for policy 1, policy_version 442135 (0.0010) [2023-12-26 18:34:51,679][105620] Updated weights for policy 1, policy_version 442145 (0.0007) [2023-12-26 18:34:52,174][105692] Updated weights for policy 0, policy_version 441803 (0.0008) [2023-12-26 18:34:52,230][105692] Updated weights for policy 0, policy_version 441813 (0.0008) [2023-12-26 18:34:52,294][105692] Updated weights for policy 0, policy_version 441823 (0.0008) [2023-12-26 18:34:52,416][105620] Updated weights for policy 1, policy_version 442155 (0.0009) [2023-12-26 18:34:52,484][105620] Updated weights for policy 1, policy_version 442165 (0.0009) [2023-12-26 18:34:52,549][105620] Updated weights for policy 1, policy_version 442175 (0.0009) [2023-12-26 18:34:52,928][105692] Updated weights for policy 0, policy_version 441833 (0.0009) [2023-12-26 18:34:52,988][105692] Updated weights for policy 0, policy_version 441843 (0.0009) [2023-12-26 18:34:53,040][105692] Updated weights for policy 0, policy_version 441854 (0.0010) [2023-12-26 18:34:53,192][105620] Updated weights for policy 1, policy_version 442185 (0.0009) [2023-12-26 18:34:53,254][105620] Updated weights for policy 1, policy_version 442195 (0.0009) [2023-12-26 18:34:53,314][105620] Updated weights for policy 1, policy_version 442205 (0.0009) [2023-12-26 18:34:53,377][105620] Updated weights for policy 1, policy_version 442215 (0.0009) [2023-12-26 18:34:53,860][105692] Updated weights for policy 0, policy_version 441865 (0.0010) [2023-12-26 18:34:53,922][105692] Updated weights for policy 0, policy_version 441875 (0.0009) [2023-12-26 18:34:53,974][105692] Updated weights for policy 0, policy_version 441885 (0.0009) [2023-12-26 18:34:54,021][105692] Updated weights for policy 0, policy_version 441895 (0.0008) [2023-12-26 18:34:54,089][105620] Updated weights for policy 1, policy_version 442225 (0.0009) [2023-12-26 18:34:54,146][105620] Updated weights for policy 1, policy_version 442235 (0.0009) [2023-12-26 18:34:54,201][105620] Updated weights for policy 1, policy_version 442245 (0.0007) [2023-12-26 18:34:54,805][105620] Updated weights for policy 1, policy_version 442255 (0.0005) [2023-12-26 18:34:54,863][105620] Updated weights for policy 1, policy_version 442265 (0.0005) [2023-12-26 18:34:54,887][105692] Updated weights for policy 0, policy_version 441905 (0.0008) [2023-12-26 18:34:54,912][105620] Updated weights for policy 1, policy_version 442275 (0.0005) [2023-12-26 18:34:54,940][105692] Updated weights for policy 0, policy_version 441915 (0.0008) [2023-12-26 18:34:54,958][105585] KL-divergence is very high: 236.2423 [2023-12-26 18:34:54,995][105692] Updated weights for policy 0, policy_version 441925 (0.0008) [2023-12-26 18:34:55,008][105585] KL-divergence is very high: 442.1640 [2023-12-26 18:34:55,631][105620] Updated weights for policy 1, policy_version 442285 (0.0008) [2023-12-26 18:34:55,684][105620] Updated weights for policy 1, policy_version 442295 (0.0008) [2023-12-26 18:34:55,688][105692] Updated weights for policy 0, policy_version 441935 (0.0006) [2023-12-26 18:34:55,732][105692] Updated weights for policy 0, policy_version 441945 (0.0005) [2023-12-26 18:34:55,739][105620] Updated weights for policy 1, policy_version 442305 (0.0008) [2023-12-26 18:34:55,778][105692] Updated weights for policy 0, policy_version 441955 (0.0005) [2023-12-26 18:34:56,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19522.0). Total num frames: 226402304. Throughput: 0: 9830.1, 1: 9883.7. Samples: 226407652. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 18:34:56,062][104569] Avg episode reward: [(0, '8613.027'), (1, '8995.860')] [2023-12-26 18:34:56,447][105692] Updated weights for policy 0, policy_version 441965 (0.0005) [2023-12-26 18:34:56,510][105692] Updated weights for policy 0, policy_version 441975 (0.0005) [2023-12-26 18:34:56,515][105620] Updated weights for policy 1, policy_version 442315 (0.0009) [2023-12-26 18:34:56,567][105620] Updated weights for policy 1, policy_version 442325 (0.0010) [2023-12-26 18:34:56,572][105692] Updated weights for policy 0, policy_version 441985 (0.0005) [2023-12-26 18:34:56,615][105620] Updated weights for policy 1, policy_version 442335 (0.0010) [2023-12-26 18:34:57,081][105692] Updated weights for policy 0, policy_version 441995 (0.0005) [2023-12-26 18:34:57,134][105692] Updated weights for policy 0, policy_version 442005 (0.0008) [2023-12-26 18:34:57,189][105692] Updated weights for policy 0, policy_version 442015 (0.0010) [2023-12-26 18:34:57,253][105620] Updated weights for policy 1, policy_version 442345 (0.0010) [2023-12-26 18:34:57,311][105620] Updated weights for policy 1, policy_version 442355 (0.0010) [2023-12-26 18:34:57,370][105620] Updated weights for policy 1, policy_version 442365 (0.0010) [2023-12-26 18:34:57,424][105620] Updated weights for policy 1, policy_version 442375 (0.0010) [2023-12-26 18:34:57,911][105692] Updated weights for policy 0, policy_version 442025 (0.0010) [2023-12-26 18:34:57,973][105692] Updated weights for policy 0, policy_version 442035 (0.0008) [2023-12-26 18:34:58,034][105692] Updated weights for policy 0, policy_version 442045 (0.0007) [2023-12-26 18:34:58,095][105620] Updated weights for policy 1, policy_version 442385 (0.0010) [2023-12-26 18:34:58,098][105692] Updated weights for policy 0, policy_version 442055 (0.0008) [2023-12-26 18:34:58,157][105620] Updated weights for policy 1, policy_version 442395 (0.0010) [2023-12-26 18:34:58,220][105620] Updated weights for policy 1, policy_version 442405 (0.0009) [2023-12-26 18:34:58,889][105692] Updated weights for policy 0, policy_version 442065 (0.0008) [2023-12-26 18:34:58,952][105692] Updated weights for policy 0, policy_version 442075 (0.0006) [2023-12-26 18:34:59,022][105692] Updated weights for policy 0, policy_version 442085 (0.0006) [2023-12-26 18:34:59,038][105620] Updated weights for policy 1, policy_version 442415 (0.0010) [2023-12-26 18:34:59,094][105620] Updated weights for policy 1, policy_version 442425 (0.0011) [2023-12-26 18:34:59,148][105620] Updated weights for policy 1, policy_version 442435 (0.0010) [2023-12-26 18:34:59,666][105692] Updated weights for policy 0, policy_version 442095 (0.0006) [2023-12-26 18:34:59,720][105692] Updated weights for policy 0, policy_version 442105 (0.0005) [2023-12-26 18:34:59,777][105692] Updated weights for policy 0, policy_version 442115 (0.0008) [2023-12-26 18:34:59,866][105620] Updated weights for policy 1, policy_version 442445 (0.0009) [2023-12-26 18:34:59,933][105620] Updated weights for policy 1, policy_version 442455 (0.0008) [2023-12-26 18:35:00,000][105620] Updated weights for policy 1, policy_version 442465 (0.0006) [2023-12-26 18:35:00,375][105692] Updated weights for policy 0, policy_version 442125 (0.0008) [2023-12-26 18:35:00,432][105692] Updated weights for policy 0, policy_version 442135 (0.0005) [2023-12-26 18:35:00,485][105692] Updated weights for policy 0, policy_version 442145 (0.0005) [2023-12-26 18:35:00,604][105620] Updated weights for policy 1, policy_version 442475 (0.0009) [2023-12-26 18:35:00,659][105620] Updated weights for policy 1, policy_version 442485 (0.0005) [2023-12-26 18:35:00,702][105620] Updated weights for policy 1, policy_version 442495 (0.0008) [2023-12-26 18:35:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 226500608. Throughput: 0: 9888.2, 1: 9954.1. Samples: 226468996. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 18:35:01,063][104569] Avg episode reward: [(0, '6022.224'), (1, '9086.482')] [2023-12-26 18:35:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000442152_113205248.pth... [2023-12-26 18:35:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000442504_113295360.pth... [2023-12-26 18:35:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000441320_112992256.pth [2023-12-26 18:35:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000441000_112910336.pth [2023-12-26 18:35:01,147][105692] Updated weights for policy 0, policy_version 442155 (0.0006) [2023-12-26 18:35:01,216][105692] Updated weights for policy 0, policy_version 442165 (0.0010) [2023-12-26 18:35:01,285][105692] Updated weights for policy 0, policy_version 442175 (0.0010) [2023-12-26 18:35:01,436][105620] Updated weights for policy 1, policy_version 442505 (0.0010) [2023-12-26 18:35:01,494][105620] Updated weights for policy 1, policy_version 442515 (0.0008) [2023-12-26 18:35:01,541][105620] Updated weights for policy 1, policy_version 442525 (0.0007) [2023-12-26 18:35:01,605][105620] Updated weights for policy 1, policy_version 442535 (0.0006) [2023-12-26 18:35:01,959][105692] Updated weights for policy 0, policy_version 442185 (0.0009) [2023-12-26 18:35:02,013][105692] Updated weights for policy 0, policy_version 442195 (0.0005) [2023-12-26 18:35:02,079][105692] Updated weights for policy 0, policy_version 442205 (0.0005) [2023-12-26 18:35:02,151][105692] Updated weights for policy 0, policy_version 442215 (0.0005) [2023-12-26 18:35:02,314][105620] Updated weights for policy 1, policy_version 442545 (0.0006) [2023-12-26 18:35:02,382][105620] Updated weights for policy 1, policy_version 442555 (0.0007) [2023-12-26 18:35:02,441][105620] Updated weights for policy 1, policy_version 442565 (0.0010) [2023-12-26 18:35:02,816][105692] Updated weights for policy 0, policy_version 442225 (0.0008) [2023-12-26 18:35:02,877][105692] Updated weights for policy 0, policy_version 442235 (0.0005) [2023-12-26 18:35:02,943][105692] Updated weights for policy 0, policy_version 442245 (0.0006) [2023-12-26 18:35:03,126][105620] Updated weights for policy 1, policy_version 442575 (0.0008) [2023-12-26 18:35:03,178][105620] Updated weights for policy 1, policy_version 442585 (0.0005) [2023-12-26 18:35:03,226][105620] Updated weights for policy 1, policy_version 442595 (0.0009) [2023-12-26 18:35:03,736][105692] Updated weights for policy 0, policy_version 442255 (0.0009) [2023-12-26 18:35:03,774][105620] Updated weights for policy 1, policy_version 442605 (0.0008) [2023-12-26 18:35:03,805][105692] Updated weights for policy 0, policy_version 442265 (0.0009) [2023-12-26 18:35:03,830][105620] Updated weights for policy 1, policy_version 442615 (0.0006) [2023-12-26 18:35:03,868][105692] Updated weights for policy 0, policy_version 442275 (0.0007) [2023-12-26 18:35:03,903][105620] Updated weights for policy 1, policy_version 442625 (0.0008) [2023-12-26 18:35:04,582][105620] Updated weights for policy 1, policy_version 442635 (0.0010) [2023-12-26 18:35:04,643][105620] Updated weights for policy 1, policy_version 442645 (0.0009) [2023-12-26 18:35:04,661][105692] Updated weights for policy 0, policy_version 442285 (0.0007) [2023-12-26 18:35:04,708][105620] Updated weights for policy 1, policy_version 442655 (0.0006) [2023-12-26 18:35:04,732][105692] Updated weights for policy 0, policy_version 442295 (0.0006) [2023-12-26 18:35:04,801][105692] Updated weights for policy 0, policy_version 442305 (0.0008) [2023-12-26 18:35:05,273][105620] Updated weights for policy 1, policy_version 442665 (0.0006) [2023-12-26 18:35:05,325][105620] Updated weights for policy 1, policy_version 442675 (0.0010) [2023-12-26 18:35:05,380][105620] Updated weights for policy 1, policy_version 442685 (0.0010) [2023-12-26 18:35:05,445][105620] Updated weights for policy 1, policy_version 442695 (0.0010) [2023-12-26 18:35:05,472][105692] Updated weights for policy 0, policy_version 442315 (0.0010) [2023-12-26 18:35:05,520][105692] Updated weights for policy 0, policy_version 442325 (0.0010) [2023-12-26 18:35:05,575][105692] Updated weights for policy 0, policy_version 442335 (0.0010) [2023-12-26 18:35:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.9, 300 sec: 19494.2). Total num frames: 226598912. Throughput: 0: 9907.2, 1: 9961.4. Samples: 226589172. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 18:35:06,062][104569] Avg episode reward: [(0, '6503.603'), (1, '8907.589')] [2023-12-26 18:35:06,087][105620] Updated weights for policy 1, policy_version 442705 (0.0010) [2023-12-26 18:35:06,147][105620] Updated weights for policy 1, policy_version 442715 (0.0011) [2023-12-26 18:35:06,180][105692] Updated weights for policy 0, policy_version 442345 (0.0010) [2023-12-26 18:35:06,201][105620] Updated weights for policy 1, policy_version 442725 (0.0010) [2023-12-26 18:35:06,235][105692] Updated weights for policy 0, policy_version 442355 (0.0009) [2023-12-26 18:35:06,299][105692] Updated weights for policy 0, policy_version 442365 (0.0008) [2023-12-26 18:35:06,352][105692] Updated weights for policy 0, policy_version 442375 (0.0008) [2023-12-26 18:35:06,999][105620] Updated weights for policy 1, policy_version 442735 (0.0010) [2023-12-26 18:35:07,065][105620] Updated weights for policy 1, policy_version 442745 (0.0011) [2023-12-26 18:35:07,124][105620] Updated weights for policy 1, policy_version 442755 (0.0010) [2023-12-26 18:35:07,186][105692] Updated weights for policy 0, policy_version 442385 (0.0008) [2023-12-26 18:35:07,246][105692] Updated weights for policy 0, policy_version 442395 (0.0008) [2023-12-26 18:35:07,302][105692] Updated weights for policy 0, policy_version 442405 (0.0008) [2023-12-26 18:35:07,870][105620] Updated weights for policy 1, policy_version 442765 (0.0010) [2023-12-26 18:35:07,929][105620] Updated weights for policy 1, policy_version 442775 (0.0010) [2023-12-26 18:35:07,991][105620] Updated weights for policy 1, policy_version 442785 (0.0010) [2023-12-26 18:35:08,095][105692] Updated weights for policy 0, policy_version 442415 (0.0008) [2023-12-26 18:35:08,155][105692] Updated weights for policy 0, policy_version 442425 (0.0008) [2023-12-26 18:35:08,211][105692] Updated weights for policy 0, policy_version 442435 (0.0008) [2023-12-26 18:35:08,724][105620] Updated weights for policy 1, policy_version 442795 (0.0010) [2023-12-26 18:35:08,769][105620] Updated weights for policy 1, policy_version 442805 (0.0007) [2023-12-26 18:35:08,826][105620] Updated weights for policy 1, policy_version 442815 (0.0007) [2023-12-26 18:35:08,936][105692] Updated weights for policy 0, policy_version 442445 (0.0009) [2023-12-26 18:35:08,994][105692] Updated weights for policy 0, policy_version 442455 (0.0010) [2023-12-26 18:35:09,051][105692] Updated weights for policy 0, policy_version 442465 (0.0010) [2023-12-26 18:35:09,630][105620] Updated weights for policy 1, policy_version 442825 (0.0008) [2023-12-26 18:35:09,691][105620] Updated weights for policy 1, policy_version 442835 (0.0009) [2023-12-26 18:35:09,750][105620] Updated weights for policy 1, policy_version 442845 (0.0008) [2023-12-26 18:35:09,791][105692] Updated weights for policy 0, policy_version 442475 (0.0010) [2023-12-26 18:35:09,806][105620] Updated weights for policy 1, policy_version 442855 (0.0007) [2023-12-26 18:35:09,855][105692] Updated weights for policy 0, policy_version 442485 (0.0011) [2023-12-26 18:35:09,912][105692] Updated weights for policy 0, policy_version 442495 (0.0008) [2023-12-26 18:35:10,517][105620] Updated weights for policy 1, policy_version 442865 (0.0006) [2023-12-26 18:35:10,564][105620] Updated weights for policy 1, policy_version 442875 (0.0006) [2023-12-26 18:35:10,617][105620] Updated weights for policy 1, policy_version 442885 (0.0011) [2023-12-26 18:35:10,709][105692] Updated weights for policy 0, policy_version 442505 (0.0010) [2023-12-26 18:35:10,765][105692] Updated weights for policy 0, policy_version 442515 (0.0010) [2023-12-26 18:35:10,813][105692] Updated weights for policy 0, policy_version 442525 (0.0010) [2023-12-26 18:35:10,872][105692] Updated weights for policy 0, policy_version 442535 (0.0011) [2023-12-26 18:35:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 226697216. Throughput: 0: 9896.0, 1: 10004.3. Samples: 226703628. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 18:35:11,062][104569] Avg episode reward: [(0, '6971.620'), (1, '8747.767')] [2023-12-26 18:35:11,310][105620] Updated weights for policy 1, policy_version 442895 (0.0011) [2023-12-26 18:35:11,376][105620] Updated weights for policy 1, policy_version 442905 (0.0010) [2023-12-26 18:35:11,441][105620] Updated weights for policy 1, policy_version 442915 (0.0006) [2023-12-26 18:35:11,704][105692] Updated weights for policy 0, policy_version 442545 (0.0008) [2023-12-26 18:35:11,771][105692] Updated weights for policy 0, policy_version 442555 (0.0007) [2023-12-26 18:35:11,831][105692] Updated weights for policy 0, policy_version 442565 (0.0005) [2023-12-26 18:35:12,139][105620] Updated weights for policy 1, policy_version 442925 (0.0008) [2023-12-26 18:35:12,202][105620] Updated weights for policy 1, policy_version 442935 (0.0010) [2023-12-26 18:35:12,269][105620] Updated weights for policy 1, policy_version 442945 (0.0011) [2023-12-26 18:35:12,543][105692] Updated weights for policy 0, policy_version 442575 (0.0006) [2023-12-26 18:35:12,604][105692] Updated weights for policy 0, policy_version 442585 (0.0006) [2023-12-26 18:35:12,652][105692] Updated weights for policy 0, policy_version 442595 (0.0005) [2023-12-26 18:35:12,887][105620] Updated weights for policy 1, policy_version 442955 (0.0009) [2023-12-26 18:35:12,941][105620] Updated weights for policy 1, policy_version 442965 (0.0005) [2023-12-26 18:35:13,003][105620] Updated weights for policy 1, policy_version 442975 (0.0007) [2023-12-26 18:35:13,300][105692] Updated weights for policy 0, policy_version 442605 (0.0009) [2023-12-26 18:35:13,352][105692] Updated weights for policy 0, policy_version 442615 (0.0010) [2023-12-26 18:35:13,401][105692] Updated weights for policy 0, policy_version 442625 (0.0009) [2023-12-26 18:35:13,583][105620] Updated weights for policy 1, policy_version 442985 (0.0010) [2023-12-26 18:35:13,642][105620] Updated weights for policy 1, policy_version 442995 (0.0006) [2023-12-26 18:35:13,694][105620] Updated weights for policy 1, policy_version 443005 (0.0010) [2023-12-26 18:35:13,738][105620] Updated weights for policy 1, policy_version 443015 (0.0010) [2023-12-26 18:35:13,964][105692] Updated weights for policy 0, policy_version 442635 (0.0005) [2023-12-26 18:35:14,023][105692] Updated weights for policy 0, policy_version 442645 (0.0005) [2023-12-26 18:35:14,079][105692] Updated weights for policy 0, policy_version 442655 (0.0005) [2023-12-26 18:35:14,456][105620] Updated weights for policy 1, policy_version 443025 (0.0006) [2023-12-26 18:35:14,514][105620] Updated weights for policy 1, policy_version 443035 (0.0005) [2023-12-26 18:35:14,574][105620] Updated weights for policy 1, policy_version 443045 (0.0007) [2023-12-26 18:35:14,764][105692] Updated weights for policy 0, policy_version 442665 (0.0006) [2023-12-26 18:35:14,822][105692] Updated weights for policy 0, policy_version 442675 (0.0009) [2023-12-26 18:35:14,885][105692] Updated weights for policy 0, policy_version 442685 (0.0008) [2023-12-26 18:35:14,944][105692] Updated weights for policy 0, policy_version 442695 (0.0008) [2023-12-26 18:35:15,283][105620] Updated weights for policy 1, policy_version 443055 (0.0008) [2023-12-26 18:35:15,343][105620] Updated weights for policy 1, policy_version 443065 (0.0008) [2023-12-26 18:35:15,408][105620] Updated weights for policy 1, policy_version 443075 (0.0008) [2023-12-26 18:35:15,664][105692] Updated weights for policy 0, policy_version 442705 (0.0006) [2023-12-26 18:35:15,729][105692] Updated weights for policy 0, policy_version 442715 (0.0010) [2023-12-26 18:35:15,799][105692] Updated weights for policy 0, policy_version 442725 (0.0010) [2023-12-26 18:35:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 226795520. Throughput: 0: 9770.1, 1: 10059.4. Samples: 226764436. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 18:35:16,063][104569] Avg episode reward: [(0, '9021.293'), (1, '9031.705')] [2023-12-26 18:35:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000442728_113352704.pth... [2023-12-26 18:35:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000441576_113057792.pth [2023-12-26 18:35:16,102][105620] Updated weights for policy 1, policy_version 443085 (0.0008) [2023-12-26 18:35:16,163][105620] Updated weights for policy 1, policy_version 443095 (0.0010) [2023-12-26 18:35:16,226][105620] Updated weights for policy 1, policy_version 443105 (0.0011) [2023-12-26 18:35:16,266][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000443112_113451008.pth... [2023-12-26 18:35:16,270][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000441928_113147904.pth [2023-12-26 18:35:16,470][105692] Updated weights for policy 0, policy_version 442735 (0.0009) [2023-12-26 18:35:16,516][105692] Updated weights for policy 0, policy_version 442745 (0.0009) [2023-12-26 18:35:16,565][105692] Updated weights for policy 0, policy_version 442756 (0.0008) [2023-12-26 18:35:16,920][105620] Updated weights for policy 1, policy_version 443115 (0.0009) [2023-12-26 18:35:16,980][105620] Updated weights for policy 1, policy_version 443125 (0.0008) [2023-12-26 18:35:17,045][105620] Updated weights for policy 1, policy_version 443135 (0.0005) [2023-12-26 18:35:17,334][105692] Updated weights for policy 0, policy_version 442766 (0.0009) [2023-12-26 18:35:17,380][105692] Updated weights for policy 0, policy_version 442776 (0.0009) [2023-12-26 18:35:17,432][105692] Updated weights for policy 0, policy_version 442786 (0.0008) [2023-12-26 18:35:17,677][105620] Updated weights for policy 1, policy_version 443145 (0.0007) [2023-12-26 18:35:17,726][105620] Updated weights for policy 1, policy_version 443155 (0.0011) [2023-12-26 18:35:17,786][105620] Updated weights for policy 1, policy_version 443165 (0.0011) [2023-12-26 18:35:17,852][105620] Updated weights for policy 1, policy_version 443175 (0.0009) [2023-12-26 18:35:18,097][105692] Updated weights for policy 0, policy_version 442796 (0.0008) [2023-12-26 18:35:18,152][105692] Updated weights for policy 0, policy_version 442806 (0.0006) [2023-12-26 18:35:18,207][105692] Updated weights for policy 0, policy_version 442816 (0.0005) [2023-12-26 18:35:18,476][105620] Updated weights for policy 1, policy_version 443185 (0.0010) [2023-12-26 18:35:18,533][105620] Updated weights for policy 1, policy_version 443195 (0.0007) [2023-12-26 18:35:18,589][105620] Updated weights for policy 1, policy_version 443205 (0.0005) [2023-12-26 18:35:18,886][105692] Updated weights for policy 0, policy_version 442826 (0.0006) [2023-12-26 18:35:18,948][105692] Updated weights for policy 0, policy_version 442836 (0.0010) [2023-12-26 18:35:19,001][105692] Updated weights for policy 0, policy_version 442846 (0.0009) [2023-12-26 18:35:19,052][105692] Updated weights for policy 0, policy_version 442856 (0.0008) [2023-12-26 18:35:19,167][105620] Updated weights for policy 1, policy_version 443215 (0.0005) [2023-12-26 18:35:19,223][105620] Updated weights for policy 1, policy_version 443225 (0.0006) [2023-12-26 18:35:19,285][105620] Updated weights for policy 1, policy_version 443235 (0.0006) [2023-12-26 18:35:19,922][105692] Updated weights for policy 0, policy_version 442866 (0.0009) [2023-12-26 18:35:19,949][105620] Updated weights for policy 1, policy_version 443245 (0.0006) [2023-12-26 18:35:19,984][105692] Updated weights for policy 0, policy_version 442876 (0.0008) [2023-12-26 18:35:20,012][105620] Updated weights for policy 1, policy_version 443255 (0.0006) [2023-12-26 18:35:20,043][105692] Updated weights for policy 0, policy_version 442886 (0.0008) [2023-12-26 18:35:20,075][105620] Updated weights for policy 1, policy_version 443265 (0.0006) [2023-12-26 18:35:20,777][105620] Updated weights for policy 1, policy_version 443275 (0.0008) [2023-12-26 18:35:20,800][105692] Updated weights for policy 0, policy_version 442896 (0.0009) [2023-12-26 18:35:20,831][105620] Updated weights for policy 1, policy_version 443285 (0.0006) [2023-12-26 18:35:20,862][105692] Updated weights for policy 0, policy_version 442906 (0.0008) [2023-12-26 18:35:20,888][105620] Updated weights for policy 1, policy_version 443295 (0.0006) [2023-12-26 18:35:20,919][105692] Updated weights for policy 0, policy_version 442916 (0.0006) [2023-12-26 18:35:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 20070.4, 300 sec: 19577.5). Total num frames: 226902016. Throughput: 0: 9742.8, 1: 10010.2. Samples: 226886712. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 18:35:21,062][104569] Avg episode reward: [(0, '9266.916'), (1, '9083.090')] [2023-12-26 18:35:21,653][105620] Updated weights for policy 1, policy_version 443305 (0.0007) [2023-12-26 18:35:21,716][105620] Updated weights for policy 1, policy_version 443315 (0.0006) [2023-12-26 18:35:21,757][105692] Updated weights for policy 0, policy_version 442926 (0.0008) [2023-12-26 18:35:21,785][105620] Updated weights for policy 1, policy_version 443325 (0.0008) [2023-12-26 18:35:21,814][105692] Updated weights for policy 0, policy_version 442936 (0.0007) [2023-12-26 18:35:21,845][105620] Updated weights for policy 1, policy_version 443335 (0.0008) [2023-12-26 18:35:21,877][105692] Updated weights for policy 0, policy_version 442946 (0.0010) [2023-12-26 18:35:22,582][105620] Updated weights for policy 1, policy_version 443345 (0.0009) [2023-12-26 18:35:22,639][105620] Updated weights for policy 1, policy_version 443355 (0.0008) [2023-12-26 18:35:22,642][105692] Updated weights for policy 0, policy_version 442956 (0.0008) [2023-12-26 18:35:22,693][105692] Updated weights for policy 0, policy_version 442966 (0.0009) [2023-12-26 18:35:22,700][105620] Updated weights for policy 1, policy_version 443365 (0.0007) [2023-12-26 18:35:22,754][105692] Updated weights for policy 0, policy_version 442976 (0.0008) [2023-12-26 18:35:23,333][105620] Updated weights for policy 1, policy_version 443375 (0.0006) [2023-12-26 18:35:23,394][105620] Updated weights for policy 1, policy_version 443385 (0.0007) [2023-12-26 18:35:23,454][105620] Updated weights for policy 1, policy_version 443395 (0.0009) [2023-12-26 18:35:23,589][105692] Updated weights for policy 0, policy_version 442986 (0.0010) [2023-12-26 18:35:23,646][105692] Updated weights for policy 0, policy_version 442996 (0.0009) [2023-12-26 18:35:23,701][105692] Updated weights for policy 0, policy_version 443006 (0.0009) [2023-12-26 18:35:23,763][105692] Updated weights for policy 0, policy_version 443016 (0.0010) [2023-12-26 18:35:24,131][105620] Updated weights for policy 1, policy_version 443405 (0.0009) [2023-12-26 18:35:24,183][105620] Updated weights for policy 1, policy_version 443415 (0.0008) [2023-12-26 18:35:24,234][105620] Updated weights for policy 1, policy_version 443425 (0.0008) [2023-12-26 18:35:24,493][105692] Updated weights for policy 0, policy_version 443026 (0.0006) [2023-12-26 18:35:24,556][105692] Updated weights for policy 0, policy_version 443036 (0.0006) [2023-12-26 18:35:24,624][105692] Updated weights for policy 0, policy_version 443046 (0.0005) [2023-12-26 18:35:25,042][105620] Updated weights for policy 1, policy_version 443435 (0.0008) [2023-12-26 18:35:25,104][105620] Updated weights for policy 1, policy_version 443445 (0.0008) [2023-12-26 18:35:25,165][105620] Updated weights for policy 1, policy_version 443455 (0.0009) [2023-12-26 18:35:25,179][105692] Updated weights for policy 0, policy_version 443056 (0.0006) [2023-12-26 18:35:25,238][105692] Updated weights for policy 0, policy_version 443066 (0.0008) [2023-12-26 18:35:25,292][105692] Updated weights for policy 0, policy_version 443076 (0.0010) [2023-12-26 18:35:25,767][105620] Updated weights for policy 1, policy_version 443465 (0.0006) [2023-12-26 18:35:25,814][105620] Updated weights for policy 1, policy_version 443475 (0.0009) [2023-12-26 18:35:25,874][105620] Updated weights for policy 1, policy_version 443485 (0.0010) [2023-12-26 18:35:25,926][105620] Updated weights for policy 1, policy_version 443495 (0.0010) [2023-12-26 18:35:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19933.9, 300 sec: 19521.9). Total num frames: 226992128. Throughput: 0: 9679.4, 1: 10028.2. Samples: 227000188. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 18:35:26,063][104569] Avg episode reward: [(0, '9006.179'), (1, '9083.562')] [2023-12-26 18:35:26,159][105692] Updated weights for policy 0, policy_version 443086 (0.0009) [2023-12-26 18:35:26,217][105692] Updated weights for policy 0, policy_version 443096 (0.0008) [2023-12-26 18:35:26,276][105692] Updated weights for policy 0, policy_version 443106 (0.0008) [2023-12-26 18:35:26,625][105620] Updated weights for policy 1, policy_version 443505 (0.0009) [2023-12-26 18:35:26,685][105620] Updated weights for policy 1, policy_version 443515 (0.0009) [2023-12-26 18:35:26,747][105620] Updated weights for policy 1, policy_version 443525 (0.0009) [2023-12-26 18:35:27,028][105692] Updated weights for policy 0, policy_version 443116 (0.0008) [2023-12-26 18:35:27,089][105692] Updated weights for policy 0, policy_version 443126 (0.0009) [2023-12-26 18:35:27,143][105692] Updated weights for policy 0, policy_version 443136 (0.0008) [2023-12-26 18:35:27,439][105620] Updated weights for policy 1, policy_version 443535 (0.0006) [2023-12-26 18:35:27,486][105620] Updated weights for policy 1, policy_version 443545 (0.0006) [2023-12-26 18:35:27,532][105620] Updated weights for policy 1, policy_version 443555 (0.0008) [2023-12-26 18:35:27,909][105692] Updated weights for policy 0, policy_version 443146 (0.0009) [2023-12-26 18:35:27,963][105692] Updated weights for policy 0, policy_version 443156 (0.0009) [2023-12-26 18:35:28,009][105692] Updated weights for policy 0, policy_version 443166 (0.0008) [2023-12-26 18:35:28,055][105692] Updated weights for policy 0, policy_version 443176 (0.0009) [2023-12-26 18:35:28,263][105620] Updated weights for policy 1, policy_version 443565 (0.0009) [2023-12-26 18:35:28,316][105620] Updated weights for policy 1, policy_version 443575 (0.0009) [2023-12-26 18:35:28,383][105620] Updated weights for policy 1, policy_version 443585 (0.0008) [2023-12-26 18:35:28,734][105692] Updated weights for policy 0, policy_version 443186 (0.0006) [2023-12-26 18:35:28,782][105692] Updated weights for policy 0, policy_version 443196 (0.0007) [2023-12-26 18:35:28,830][105692] Updated weights for policy 0, policy_version 443206 (0.0010) [2023-12-26 18:35:29,074][105620] Updated weights for policy 1, policy_version 443595 (0.0008) [2023-12-26 18:35:29,125][105620] Updated weights for policy 1, policy_version 443605 (0.0008) [2023-12-26 18:35:29,177][105620] Updated weights for policy 1, policy_version 443615 (0.0008) [2023-12-26 18:35:29,540][105692] Updated weights for policy 0, policy_version 443216 (0.0007) [2023-12-26 18:35:29,585][105692] Updated weights for policy 0, policy_version 443226 (0.0009) [2023-12-26 18:35:29,641][105692] Updated weights for policy 0, policy_version 443236 (0.0005) [2023-12-26 18:35:30,018][105620] Updated weights for policy 1, policy_version 443625 (0.0009) [2023-12-26 18:35:30,066][105620] Updated weights for policy 1, policy_version 443635 (0.0008) [2023-12-26 18:35:30,090][105586] KL-divergence is very high: 117.7068 [2023-12-26 18:35:30,105][105586] KL-divergence is very high: 113.9632 [2023-12-26 18:35:30,114][105620] Updated weights for policy 1, policy_version 443645 (0.0008) [2023-12-26 18:35:30,115][105586] KL-divergence is very high: 143.9860 [2023-12-26 18:35:30,131][105586] KL-divergence is very high: 120.9288 [2023-12-26 18:35:30,155][105586] KL-divergence is very high: 113.9718 [2023-12-26 18:35:30,166][105620] Updated weights for policy 1, policy_version 443655 (0.0008) [2023-12-26 18:35:30,350][105692] Updated weights for policy 0, policy_version 443246 (0.0008) [2023-12-26 18:35:30,405][105692] Updated weights for policy 0, policy_version 443256 (0.0010) [2023-12-26 18:35:30,453][105692] Updated weights for policy 0, policy_version 443266 (0.0010) [2023-12-26 18:35:31,019][105692] Updated weights for policy 0, policy_version 443276 (0.0009) [2023-12-26 18:35:31,051][105620] Updated weights for policy 1, policy_version 443665 (0.0007) [2023-12-26 18:35:31,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 227082240. Throughput: 0: 9650.9, 1: 10088.1. Samples: 227058336. Policy #0 lag: (min: 24.0, avg: 51.8, max: 56.0) [2023-12-26 18:35:31,062][104569] Avg episode reward: [(0, '7423.879'), (1, '6846.595')] [2023-12-26 18:35:31,077][105692] Updated weights for policy 0, policy_version 443286 (0.0010) [2023-12-26 18:35:31,106][105620] Updated weights for policy 1, policy_version 443675 (0.0006) [2023-12-26 18:35:31,132][105692] Updated weights for policy 0, policy_version 443296 (0.0010) [2023-12-26 18:35:31,166][105620] Updated weights for policy 1, policy_version 443685 (0.0008) [2023-12-26 18:35:31,181][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000443688_113598464.pth... [2023-12-26 18:35:31,182][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000443304_113500160.pth... [2023-12-26 18:35:31,184][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000442504_113295360.pth [2023-12-26 18:35:31,187][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000442152_113205248.pth [2023-12-26 18:35:31,857][105692] Updated weights for policy 0, policy_version 443306 (0.0010) [2023-12-26 18:35:31,865][105620] Updated weights for policy 1, policy_version 443695 (0.0007) [2023-12-26 18:35:31,910][105620] Updated weights for policy 1, policy_version 443705 (0.0005) [2023-12-26 18:35:31,917][105692] Updated weights for policy 0, policy_version 443316 (0.0007) [2023-12-26 18:35:31,966][105620] Updated weights for policy 1, policy_version 443715 (0.0005) [2023-12-26 18:35:31,982][105692] Updated weights for policy 0, policy_version 443326 (0.0009) [2023-12-26 18:35:32,044][105692] Updated weights for policy 0, policy_version 443336 (0.0010) [2023-12-26 18:35:32,700][105620] Updated weights for policy 1, policy_version 443725 (0.0008) [2023-12-26 18:35:32,734][105692] Updated weights for policy 0, policy_version 443346 (0.0007) [2023-12-26 18:35:32,759][105620] Updated weights for policy 1, policy_version 443735 (0.0010) [2023-12-26 18:35:32,789][105692] Updated weights for policy 0, policy_version 443356 (0.0005) [2023-12-26 18:35:32,812][105620] Updated weights for policy 1, policy_version 443745 (0.0010) [2023-12-26 18:35:32,834][105692] Updated weights for policy 0, policy_version 443366 (0.0005) [2023-12-26 18:35:33,543][105620] Updated weights for policy 1, policy_version 443755 (0.0010) [2023-12-26 18:35:33,545][105692] Updated weights for policy 0, policy_version 443376 (0.0007) [2023-12-26 18:35:33,598][105620] Updated weights for policy 1, policy_version 443765 (0.0010) [2023-12-26 18:35:33,603][105692] Updated weights for policy 0, policy_version 443386 (0.0005) [2023-12-26 18:35:33,649][105620] Updated weights for policy 1, policy_version 443775 (0.0010) [2023-12-26 18:35:33,651][105692] Updated weights for policy 0, policy_version 443396 (0.0005) [2023-12-26 18:35:34,399][105692] Updated weights for policy 0, policy_version 443406 (0.0007) [2023-12-26 18:35:34,401][105620] Updated weights for policy 1, policy_version 443785 (0.0010) [2023-12-26 18:35:34,456][105692] Updated weights for policy 0, policy_version 443416 (0.0007) [2023-12-26 18:35:34,469][105620] Updated weights for policy 1, policy_version 443795 (0.0011) [2023-12-26 18:35:34,522][105692] Updated weights for policy 0, policy_version 443426 (0.0007) [2023-12-26 18:35:34,532][105620] Updated weights for policy 1, policy_version 443805 (0.0011) [2023-12-26 18:35:34,592][105620] Updated weights for policy 1, policy_version 443815 (0.0011) [2023-12-26 18:35:35,289][105620] Updated weights for policy 1, policy_version 443825 (0.0011) [2023-12-26 18:35:35,306][105692] Updated weights for policy 0, policy_version 443436 (0.0006) [2023-12-26 18:35:35,362][105620] Updated weights for policy 1, policy_version 443835 (0.0011) [2023-12-26 18:35:35,367][105692] Updated weights for policy 0, policy_version 443446 (0.0007) [2023-12-26 18:35:35,415][105692] Updated weights for policy 0, policy_version 443456 (0.0006) [2023-12-26 18:35:35,424][105620] Updated weights for policy 1, policy_version 443845 (0.0010) [2023-12-26 18:35:36,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 227180544. Throughput: 0: 9684.9, 1: 9882.3. Samples: 227173900. Policy #0 lag: (min: 24.0, avg: 51.8, max: 56.0) [2023-12-26 18:35:36,062][104569] Avg episode reward: [(0, '6904.895'), (1, '6704.004')] [2023-12-26 18:35:36,141][105620] Updated weights for policy 1, policy_version 443855 (0.0010) [2023-12-26 18:35:36,146][105692] Updated weights for policy 0, policy_version 443466 (0.0007) [2023-12-26 18:35:36,200][105692] Updated weights for policy 0, policy_version 443476 (0.0007) [2023-12-26 18:35:36,204][105620] Updated weights for policy 1, policy_version 443865 (0.0011) [2023-12-26 18:35:36,260][105620] Updated weights for policy 1, policy_version 443875 (0.0011) [2023-12-26 18:35:36,263][105692] Updated weights for policy 0, policy_version 443486 (0.0006) [2023-12-26 18:35:36,324][105692] Updated weights for policy 0, policy_version 443496 (0.0007) [2023-12-26 18:35:37,004][105620] Updated weights for policy 1, policy_version 443885 (0.0011) [2023-12-26 18:35:37,061][105692] Updated weights for policy 0, policy_version 443506 (0.0005) [2023-12-26 18:35:37,067][105620] Updated weights for policy 1, policy_version 443895 (0.0010) [2023-12-26 18:35:37,121][105692] Updated weights for policy 0, policy_version 443516 (0.0006) [2023-12-26 18:35:37,130][105620] Updated weights for policy 1, policy_version 443905 (0.0006) [2023-12-26 18:35:37,179][105692] Updated weights for policy 0, policy_version 443526 (0.0008) [2023-12-26 18:35:37,864][105620] Updated weights for policy 1, policy_version 443915 (0.0008) [2023-12-26 18:35:37,911][105692] Updated weights for policy 0, policy_version 443536 (0.0007) [2023-12-26 18:35:37,928][105620] Updated weights for policy 1, policy_version 443925 (0.0010) [2023-12-26 18:35:37,974][105692] Updated weights for policy 0, policy_version 443546 (0.0006) [2023-12-26 18:35:37,984][105620] Updated weights for policy 1, policy_version 443935 (0.0010) [2023-12-26 18:35:38,034][105692] Updated weights for policy 0, policy_version 443556 (0.0006) [2023-12-26 18:35:38,727][105620] Updated weights for policy 1, policy_version 443945 (0.0010) [2023-12-26 18:35:38,736][105692] Updated weights for policy 0, policy_version 443566 (0.0007) [2023-12-26 18:35:38,788][105692] Updated weights for policy 0, policy_version 443576 (0.0005) [2023-12-26 18:35:38,789][105620] Updated weights for policy 1, policy_version 443955 (0.0011) [2023-12-26 18:35:38,855][105692] Updated weights for policy 0, policy_version 443586 (0.0006) [2023-12-26 18:35:38,858][105620] Updated weights for policy 1, policy_version 443965 (0.0011) [2023-12-26 18:35:38,913][105620] Updated weights for policy 1, policy_version 443975 (0.0011) [2023-12-26 18:35:39,522][105692] Updated weights for policy 0, policy_version 443596 (0.0007) [2023-12-26 18:35:39,590][105692] Updated weights for policy 0, policy_version 443606 (0.0008) [2023-12-26 18:35:39,630][105620] Updated weights for policy 1, policy_version 443985 (0.0008) [2023-12-26 18:35:39,654][105692] Updated weights for policy 0, policy_version 443616 (0.0008) [2023-12-26 18:35:39,691][105620] Updated weights for policy 1, policy_version 443995 (0.0009) [2023-12-26 18:35:39,761][105620] Updated weights for policy 1, policy_version 444005 (0.0009) [2023-12-26 18:35:40,373][105692] Updated weights for policy 0, policy_version 443626 (0.0008) [2023-12-26 18:35:40,433][105692] Updated weights for policy 0, policy_version 443636 (0.0010) [2023-12-26 18:35:40,483][105692] Updated weights for policy 0, policy_version 443646 (0.0010) [2023-12-26 18:35:40,509][105620] Updated weights for policy 1, policy_version 444015 (0.0008) [2023-12-26 18:35:40,536][105692] Updated weights for policy 0, policy_version 443656 (0.0007) [2023-12-26 18:35:40,570][105620] Updated weights for policy 1, policy_version 444025 (0.0009) [2023-12-26 18:35:40,617][105620] Updated weights for policy 1, policy_version 444035 (0.0008) [2023-12-26 18:35:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 227278848. Throughput: 0: 9699.5, 1: 9849.0. Samples: 227287340. Policy #0 lag: (min: 24.0, avg: 51.8, max: 56.0) [2023-12-26 18:35:41,063][104569] Avg episode reward: [(0, '7607.416'), (1, '8334.816')] [2023-12-26 18:35:41,340][105692] Updated weights for policy 0, policy_version 443666 (0.0010) [2023-12-26 18:35:41,399][105620] Updated weights for policy 1, policy_version 444045 (0.0008) [2023-12-26 18:35:41,411][105692] Updated weights for policy 0, policy_version 443676 (0.0008) [2023-12-26 18:35:41,462][105692] Updated weights for policy 0, policy_version 443686 (0.0007) [2023-12-26 18:35:41,462][105620] Updated weights for policy 1, policy_version 444055 (0.0008) [2023-12-26 18:35:41,529][105620] Updated weights for policy 1, policy_version 444065 (0.0009) [2023-12-26 18:35:42,211][105692] Updated weights for policy 0, policy_version 443696 (0.0008) [2023-12-26 18:35:42,277][105692] Updated weights for policy 0, policy_version 443706 (0.0009) [2023-12-26 18:35:42,306][105620] Updated weights for policy 1, policy_version 444075 (0.0008) [2023-12-26 18:35:42,343][105692] Updated weights for policy 0, policy_version 443716 (0.0008) [2023-12-26 18:35:42,372][105620] Updated weights for policy 1, policy_version 444085 (0.0010) [2023-12-26 18:35:42,432][105620] Updated weights for policy 1, policy_version 444095 (0.0011) [2023-12-26 18:35:43,036][105692] Updated weights for policy 0, policy_version 443726 (0.0008) [2023-12-26 18:35:43,091][105692] Updated weights for policy 0, policy_version 443736 (0.0008) [2023-12-26 18:35:43,133][105620] Updated weights for policy 1, policy_version 444105 (0.0011) [2023-12-26 18:35:43,139][105692] Updated weights for policy 0, policy_version 443746 (0.0007) [2023-12-26 18:35:43,190][105620] Updated weights for policy 1, policy_version 444115 (0.0007) [2023-12-26 18:35:43,249][105620] Updated weights for policy 1, policy_version 444125 (0.0007) [2023-12-26 18:35:43,304][105620] Updated weights for policy 1, policy_version 444135 (0.0009) [2023-12-26 18:35:43,867][105692] Updated weights for policy 0, policy_version 443756 (0.0009) [2023-12-26 18:35:43,917][105692] Updated weights for policy 0, policy_version 443766 (0.0010) [2023-12-26 18:35:43,941][105620] Updated weights for policy 1, policy_version 444145 (0.0006) [2023-12-26 18:35:43,973][105692] Updated weights for policy 0, policy_version 443776 (0.0008) [2023-12-26 18:35:44,001][105620] Updated weights for policy 1, policy_version 444155 (0.0005) [2023-12-26 18:35:44,051][105620] Updated weights for policy 1, policy_version 444165 (0.0006) [2023-12-26 18:35:44,674][105620] Updated weights for policy 1, policy_version 444175 (0.0008) [2023-12-26 18:35:44,740][105620] Updated weights for policy 1, policy_version 444185 (0.0006) [2023-12-26 18:35:44,806][105620] Updated weights for policy 1, policy_version 444195 (0.0008) [2023-12-26 18:35:44,806][105692] Updated weights for policy 0, policy_version 443786 (0.0009) [2023-12-26 18:35:44,855][105692] Updated weights for policy 0, policy_version 443796 (0.0008) [2023-12-26 18:35:44,909][105692] Updated weights for policy 0, policy_version 443806 (0.0009) [2023-12-26 18:35:44,971][105692] Updated weights for policy 0, policy_version 443816 (0.0009) [2023-12-26 18:35:45,480][105620] Updated weights for policy 1, policy_version 444205 (0.0007) [2023-12-26 18:35:45,540][105620] Updated weights for policy 1, policy_version 444215 (0.0006) [2023-12-26 18:35:45,601][105620] Updated weights for policy 1, policy_version 444225 (0.0006) [2023-12-26 18:35:45,780][105692] Updated weights for policy 0, policy_version 443826 (0.0009) [2023-12-26 18:35:45,833][105692] Updated weights for policy 0, policy_version 443836 (0.0010) [2023-12-26 18:35:45,889][105692] Updated weights for policy 0, policy_version 443846 (0.0010) [2023-12-26 18:35:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 227377152. Throughput: 0: 9622.0, 1: 9847.1. Samples: 227345100. Policy #0 lag: (min: 24.0, avg: 51.8, max: 56.0) [2023-12-26 18:35:46,062][104569] Avg episode reward: [(0, '7955.200'), (1, '8904.938')] [2023-12-26 18:35:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000443848_113639424.pth... [2023-12-26 18:35:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000444232_113737728.pth... [2023-12-26 18:35:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000442728_113352704.pth [2023-12-26 18:35:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000443112_113451008.pth [2023-12-26 18:35:46,171][105620] Updated weights for policy 1, policy_version 444235 (0.0007) [2023-12-26 18:35:46,223][105620] Updated weights for policy 1, policy_version 444245 (0.0009) [2023-12-26 18:35:46,283][105620] Updated weights for policy 1, policy_version 444255 (0.0005) [2023-12-26 18:35:46,670][105692] Updated weights for policy 0, policy_version 443856 (0.0008) [2023-12-26 18:35:46,717][105692] Updated weights for policy 0, policy_version 443866 (0.0008) [2023-12-26 18:35:46,765][105692] Updated weights for policy 0, policy_version 443876 (0.0007) [2023-12-26 18:35:46,963][105620] Updated weights for policy 1, policy_version 444265 (0.0006) [2023-12-26 18:35:47,024][105620] Updated weights for policy 1, policy_version 444275 (0.0010) [2023-12-26 18:35:47,068][105620] Updated weights for policy 1, policy_version 444285 (0.0010) [2023-12-26 18:35:47,119][105620] Updated weights for policy 1, policy_version 444295 (0.0010) [2023-12-26 18:35:47,560][105692] Updated weights for policy 0, policy_version 443886 (0.0008) [2023-12-26 18:35:47,622][105692] Updated weights for policy 0, policy_version 443896 (0.0008) [2023-12-26 18:35:47,677][105692] Updated weights for policy 0, policy_version 443906 (0.0008) [2023-12-26 18:35:47,829][105620] Updated weights for policy 1, policy_version 444305 (0.0006) [2023-12-26 18:35:47,877][105620] Updated weights for policy 1, policy_version 444315 (0.0007) [2023-12-26 18:35:47,924][105620] Updated weights for policy 1, policy_version 444325 (0.0006) [2023-12-26 18:35:48,279][105692] Updated weights for policy 0, policy_version 443916 (0.0008) [2023-12-26 18:35:48,349][105692] Updated weights for policy 0, policy_version 443926 (0.0008) [2023-12-26 18:35:48,399][105692] Updated weights for policy 0, policy_version 443936 (0.0009) [2023-12-26 18:35:48,530][105620] Updated weights for policy 1, policy_version 444335 (0.0008) [2023-12-26 18:35:48,589][105620] Updated weights for policy 1, policy_version 444345 (0.0009) [2023-12-26 18:35:48,643][105620] Updated weights for policy 1, policy_version 444355 (0.0008) [2023-12-26 18:35:49,134][105692] Updated weights for policy 0, policy_version 443946 (0.0008) [2023-12-26 18:35:49,187][105692] Updated weights for policy 0, policy_version 443956 (0.0008) [2023-12-26 18:35:49,245][105692] Updated weights for policy 0, policy_version 443966 (0.0009) [2023-12-26 18:35:49,305][105692] Updated weights for policy 0, policy_version 443976 (0.0009) [2023-12-26 18:35:49,410][105620] Updated weights for policy 1, policy_version 444365 (0.0009) [2023-12-26 18:35:49,465][105620] Updated weights for policy 1, policy_version 444376 (0.0009) [2023-12-26 18:35:49,527][105620] Updated weights for policy 1, policy_version 444386 (0.0010) [2023-12-26 18:35:50,038][105692] Updated weights for policy 0, policy_version 443986 (0.0009) [2023-12-26 18:35:50,096][105692] Updated weights for policy 0, policy_version 443996 (0.0008) [2023-12-26 18:35:50,155][105692] Updated weights for policy 0, policy_version 444006 (0.0009) [2023-12-26 18:35:50,348][105620] Updated weights for policy 1, policy_version 444396 (0.0008) [2023-12-26 18:35:50,406][105620] Updated weights for policy 1, policy_version 444406 (0.0009) [2023-12-26 18:35:50,465][105620] Updated weights for policy 1, policy_version 444416 (0.0009) [2023-12-26 18:35:50,918][105692] Updated weights for policy 0, policy_version 444016 (0.0010) [2023-12-26 18:35:50,973][105692] Updated weights for policy 0, policy_version 444026 (0.0010) [2023-12-26 18:35:51,035][105692] Updated weights for policy 0, policy_version 444036 (0.0009) [2023-12-26 18:35:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 227467264. Throughput: 0: 9583.7, 1: 9829.4. Samples: 227462764. Policy #0 lag: (min: 24.0, avg: 51.8, max: 56.0) [2023-12-26 18:35:51,062][104569] Avg episode reward: [(0, '7612.915'), (1, '9080.721')] [2023-12-26 18:35:51,166][105620] Updated weights for policy 1, policy_version 444426 (0.0009) [2023-12-26 18:35:51,223][105620] Updated weights for policy 1, policy_version 444436 (0.0006) [2023-12-26 18:35:51,282][105620] Updated weights for policy 1, policy_version 444446 (0.0007) [2023-12-26 18:35:51,336][105620] Updated weights for policy 1, policy_version 444456 (0.0006) [2023-12-26 18:35:51,819][105692] Updated weights for policy 0, policy_version 444046 (0.0008) [2023-12-26 18:35:51,874][105692] Updated weights for policy 0, policy_version 444056 (0.0008) [2023-12-26 18:35:51,943][105692] Updated weights for policy 0, policy_version 444066 (0.0009) [2023-12-26 18:35:52,078][105620] Updated weights for policy 1, policy_version 444466 (0.0009) [2023-12-26 18:35:52,128][105620] Updated weights for policy 1, policy_version 444476 (0.0009) [2023-12-26 18:35:52,175][105620] Updated weights for policy 1, policy_version 444486 (0.0006) [2023-12-26 18:35:52,723][105692] Updated weights for policy 0, policy_version 444076 (0.0009) [2023-12-26 18:35:52,771][105692] Updated weights for policy 0, policy_version 444086 (0.0009) [2023-12-26 18:35:52,818][105692] Updated weights for policy 0, policy_version 444096 (0.0008) [2023-12-26 18:35:52,908][105620] Updated weights for policy 1, policy_version 444496 (0.0009) [2023-12-26 18:35:52,967][105620] Updated weights for policy 1, policy_version 444506 (0.0010) [2023-12-26 18:35:53,026][105620] Updated weights for policy 1, policy_version 444516 (0.0008) [2023-12-26 18:35:53,588][105620] Updated weights for policy 1, policy_version 444526 (0.0005) [2023-12-26 18:35:53,595][105692] Updated weights for policy 0, policy_version 444106 (0.0009) [2023-12-26 18:35:53,652][105620] Updated weights for policy 1, policy_version 444536 (0.0006) [2023-12-26 18:35:53,657][105692] Updated weights for policy 0, policy_version 444116 (0.0010) [2023-12-26 18:35:53,718][105620] Updated weights for policy 1, policy_version 444546 (0.0007) [2023-12-26 18:35:53,724][105692] Updated weights for policy 0, policy_version 444126 (0.0010) [2023-12-26 18:35:53,791][105692] Updated weights for policy 0, policy_version 444136 (0.0010) [2023-12-26 18:35:54,368][105620] Updated weights for policy 1, policy_version 444556 (0.0006) [2023-12-26 18:35:54,423][105620] Updated weights for policy 1, policy_version 444566 (0.0005) [2023-12-26 18:35:54,489][105620] Updated weights for policy 1, policy_version 444576 (0.0006) [2023-12-26 18:35:54,542][105692] Updated weights for policy 0, policy_version 444146 (0.0007) [2023-12-26 18:35:54,607][105692] Updated weights for policy 0, policy_version 444156 (0.0006) [2023-12-26 18:35:54,680][105692] Updated weights for policy 0, policy_version 444166 (0.0006) [2023-12-26 18:35:55,099][105620] Updated weights for policy 1, policy_version 444586 (0.0007) [2023-12-26 18:35:55,158][105620] Updated weights for policy 1, policy_version 444596 (0.0008) [2023-12-26 18:35:55,212][105620] Updated weights for policy 1, policy_version 444606 (0.0009) [2023-12-26 18:35:55,266][105620] Updated weights for policy 1, policy_version 444616 (0.0009) [2023-12-26 18:35:55,270][105692] Updated weights for policy 0, policy_version 444176 (0.0010) [2023-12-26 18:35:55,324][105692] Updated weights for policy 0, policy_version 444186 (0.0010) [2023-12-26 18:35:55,373][105692] Updated weights for policy 0, policy_version 444196 (0.0006) [2023-12-26 18:35:55,944][105692] Updated weights for policy 0, policy_version 444206 (0.0008) [2023-12-26 18:35:56,011][105692] Updated weights for policy 0, policy_version 444216 (0.0011) [2023-12-26 18:35:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 227565568. Throughput: 0: 9568.1, 1: 9890.7. Samples: 227579272. Policy #0 lag: (min: 24.0, avg: 51.8, max: 56.0) [2023-12-26 18:35:56,062][104569] Avg episode reward: [(0, '8074.862'), (1, '9081.574')] [2023-12-26 18:35:56,076][105692] Updated weights for policy 0, policy_version 444226 (0.0010) [2023-12-26 18:35:56,087][105620] Updated weights for policy 1, policy_version 444626 (0.0006) [2023-12-26 18:35:56,143][105620] Updated weights for policy 1, policy_version 444636 (0.0008) [2023-12-26 18:35:56,191][105620] Updated weights for policy 1, policy_version 444646 (0.0010) [2023-12-26 18:35:56,756][105692] Updated weights for policy 0, policy_version 444236 (0.0010) [2023-12-26 18:35:56,814][105692] Updated weights for policy 0, policy_version 444246 (0.0010) [2023-12-26 18:35:56,852][105620] Updated weights for policy 1, policy_version 444656 (0.0006) [2023-12-26 18:35:56,862][105692] Updated weights for policy 0, policy_version 444256 (0.0010) [2023-12-26 18:35:56,915][105620] Updated weights for policy 1, policy_version 444666 (0.0005) [2023-12-26 18:35:56,978][105620] Updated weights for policy 1, policy_version 444676 (0.0005) [2023-12-26 18:35:57,520][105692] Updated weights for policy 0, policy_version 444266 (0.0009) [2023-12-26 18:35:57,520][105620] Updated weights for policy 1, policy_version 444686 (0.0006) [2023-12-26 18:35:57,573][105620] Updated weights for policy 1, policy_version 444696 (0.0006) [2023-12-26 18:35:57,578][105692] Updated weights for policy 0, policy_version 444276 (0.0005) [2023-12-26 18:35:57,619][105620] Updated weights for policy 1, policy_version 444706 (0.0005) [2023-12-26 18:35:57,631][105692] Updated weights for policy 0, policy_version 444286 (0.0008) [2023-12-26 18:35:57,682][105692] Updated weights for policy 0, policy_version 444296 (0.0010) [2023-12-26 18:35:58,180][105620] Updated weights for policy 1, policy_version 444716 (0.0006) [2023-12-26 18:35:58,243][105620] Updated weights for policy 1, policy_version 444726 (0.0009) [2023-12-26 18:35:58,303][105620] Updated weights for policy 1, policy_version 444736 (0.0007) [2023-12-26 18:35:58,333][105692] Updated weights for policy 0, policy_version 444306 (0.0009) [2023-12-26 18:35:58,392][105692] Updated weights for policy 0, policy_version 444316 (0.0007) [2023-12-26 18:35:58,447][105692] Updated weights for policy 0, policy_version 444326 (0.0008) [2023-12-26 18:35:59,168][105620] Updated weights for policy 1, policy_version 444746 (0.0009) [2023-12-26 18:35:59,235][105692] Updated weights for policy 0, policy_version 444336 (0.0007) [2023-12-26 18:35:59,236][105620] Updated weights for policy 1, policy_version 444756 (0.0008) [2023-12-26 18:35:59,290][105692] Updated weights for policy 0, policy_version 444346 (0.0007) [2023-12-26 18:35:59,296][105620] Updated weights for policy 1, policy_version 444766 (0.0008) [2023-12-26 18:35:59,348][105692] Updated weights for policy 0, policy_version 444356 (0.0006) [2023-12-26 18:35:59,354][105620] Updated weights for policy 1, policy_version 444776 (0.0008) [2023-12-26 18:36:00,066][105620] Updated weights for policy 1, policy_version 444786 (0.0009) [2023-12-26 18:36:00,112][105692] Updated weights for policy 0, policy_version 444366 (0.0009) [2023-12-26 18:36:00,123][105620] Updated weights for policy 1, policy_version 444796 (0.0007) [2023-12-26 18:36:00,173][105692] Updated weights for policy 0, policy_version 444376 (0.0009) [2023-12-26 18:36:00,177][105620] Updated weights for policy 1, policy_version 444806 (0.0007) [2023-12-26 18:36:00,224][105692] Updated weights for policy 0, policy_version 444386 (0.0008) [2023-12-26 18:36:00,933][105620] Updated weights for policy 1, policy_version 444816 (0.0009) [2023-12-26 18:36:00,970][105692] Updated weights for policy 0, policy_version 444396 (0.0008) [2023-12-26 18:36:00,980][105620] Updated weights for policy 1, policy_version 444826 (0.0007) [2023-12-26 18:36:01,019][105692] Updated weights for policy 0, policy_version 444406 (0.0007) [2023-12-26 18:36:01,034][105620] Updated weights for policy 1, policy_version 444836 (0.0007) [2023-12-26 18:36:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 227672064. Throughput: 0: 9624.3, 1: 9895.0. Samples: 227642804. Policy #0 lag: (min: 24.0, avg: 51.8, max: 56.0) [2023-12-26 18:36:01,062][104569] Avg episode reward: [(0, '8334.544'), (1, '8901.123')] [2023-12-26 18:36:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000444840_113893376.pth... [2023-12-26 18:36:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000443688_113598464.pth [2023-12-26 18:36:01,089][105692] Updated weights for policy 0, policy_version 444416 (0.0007) [2023-12-26 18:36:01,141][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000444424_113786880.pth... [2023-12-26 18:36:01,145][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000443304_113500160.pth [2023-12-26 18:36:01,746][105692] Updated weights for policy 0, policy_version 444426 (0.0009) [2023-12-26 18:36:01,793][105692] Updated weights for policy 0, policy_version 444436 (0.0009) [2023-12-26 18:36:01,842][105692] Updated weights for policy 0, policy_version 444446 (0.0008) [2023-12-26 18:36:01,856][105620] Updated weights for policy 1, policy_version 444846 (0.0008) [2023-12-26 18:36:01,903][105692] Updated weights for policy 0, policy_version 444456 (0.0006) [2023-12-26 18:36:01,916][105620] Updated weights for policy 1, policy_version 444856 (0.0009) [2023-12-26 18:36:01,980][105620] Updated weights for policy 1, policy_version 444866 (0.0008) [2023-12-26 18:36:02,645][105692] Updated weights for policy 0, policy_version 444466 (0.0009) [2023-12-26 18:36:02,704][105692] Updated weights for policy 0, policy_version 444476 (0.0008) [2023-12-26 18:36:02,754][105620] Updated weights for policy 1, policy_version 444876 (0.0008) [2023-12-26 18:36:02,764][105692] Updated weights for policy 0, policy_version 444486 (0.0007) [2023-12-26 18:36:02,809][105620] Updated weights for policy 1, policy_version 444886 (0.0008) [2023-12-26 18:36:02,877][105620] Updated weights for policy 1, policy_version 444896 (0.0010) [2023-12-26 18:36:03,344][105692] Updated weights for policy 0, policy_version 444496 (0.0006) [2023-12-26 18:36:03,405][105692] Updated weights for policy 0, policy_version 444506 (0.0009) [2023-12-26 18:36:03,453][105692] Updated weights for policy 0, policy_version 444516 (0.0008) [2023-12-26 18:36:03,725][105620] Updated weights for policy 1, policy_version 444906 (0.0008) [2023-12-26 18:36:03,772][105620] Updated weights for policy 1, policy_version 444916 (0.0006) [2023-12-26 18:36:03,826][105620] Updated weights for policy 1, policy_version 444926 (0.0008) [2023-12-26 18:36:03,885][105620] Updated weights for policy 1, policy_version 444936 (0.0008) [2023-12-26 18:36:04,019][105692] Updated weights for policy 0, policy_version 444526 (0.0005) [2023-12-26 18:36:04,085][105692] Updated weights for policy 0, policy_version 444536 (0.0006) [2023-12-26 18:36:04,149][105692] Updated weights for policy 0, policy_version 444546 (0.0006) [2023-12-26 18:36:04,701][105620] Updated weights for policy 1, policy_version 444946 (0.0008) [2023-12-26 18:36:04,764][105620] Updated weights for policy 1, policy_version 444956 (0.0008) [2023-12-26 18:36:04,818][105692] Updated weights for policy 0, policy_version 444556 (0.0010) [2023-12-26 18:36:04,828][105620] Updated weights for policy 1, policy_version 444966 (0.0007) [2023-12-26 18:36:04,878][105692] Updated weights for policy 0, policy_version 444566 (0.0011) [2023-12-26 18:36:04,940][105692] Updated weights for policy 0, policy_version 444576 (0.0010) [2023-12-26 18:36:05,592][105620] Updated weights for policy 1, policy_version 444976 (0.0008) [2023-12-26 18:36:05,648][105620] Updated weights for policy 1, policy_version 444986 (0.0008) [2023-12-26 18:36:05,661][105692] Updated weights for policy 0, policy_version 444586 (0.0010) [2023-12-26 18:36:05,705][105620] Updated weights for policy 1, policy_version 444996 (0.0006) [2023-12-26 18:36:05,711][105692] Updated weights for policy 0, policy_version 444596 (0.0011) [2023-12-26 18:36:05,771][105692] Updated weights for policy 0, policy_version 444606 (0.0010) [2023-12-26 18:36:05,834][105692] Updated weights for policy 0, policy_version 444616 (0.0011) [2023-12-26 18:36:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 227770368. Throughput: 0: 9643.1, 1: 9695.2. Samples: 227756936. Policy #0 lag: (min: 24.0, avg: 51.8, max: 56.0) [2023-12-26 18:36:06,062][104569] Avg episode reward: [(0, '8445.398'), (1, '2926.921')] [2023-12-26 18:36:06,456][105620] Updated weights for policy 1, policy_version 445006 (0.0008) [2023-12-26 18:36:06,522][105620] Updated weights for policy 1, policy_version 445016 (0.0009) [2023-12-26 18:36:06,581][105620] Updated weights for policy 1, policy_version 445026 (0.0008) [2023-12-26 18:36:06,587][105692] Updated weights for policy 0, policy_version 444626 (0.0006) [2023-12-26 18:36:06,636][105692] Updated weights for policy 0, policy_version 444636 (0.0007) [2023-12-26 18:36:06,686][105692] Updated weights for policy 0, policy_version 444646 (0.0008) [2023-12-26 18:36:07,340][105620] Updated weights for policy 1, policy_version 445036 (0.0009) [2023-12-26 18:36:07,347][105692] Updated weights for policy 0, policy_version 444656 (0.0006) [2023-12-26 18:36:07,393][105620] Updated weights for policy 1, policy_version 445046 (0.0011) [2023-12-26 18:36:07,409][105692] Updated weights for policy 0, policy_version 444666 (0.0005) [2023-12-26 18:36:07,442][105620] Updated weights for policy 1, policy_version 445056 (0.0010) [2023-12-26 18:36:07,472][105692] Updated weights for policy 0, policy_version 444676 (0.0011) [2023-12-26 18:36:08,052][105692] Updated weights for policy 0, policy_version 444686 (0.0010) [2023-12-26 18:36:08,104][105692] Updated weights for policy 0, policy_version 444696 (0.0009) [2023-12-26 18:36:08,135][105620] Updated weights for policy 1, policy_version 445066 (0.0009) [2023-12-26 18:36:08,155][105692] Updated weights for policy 0, policy_version 444706 (0.0010) [2023-12-26 18:36:08,201][105620] Updated weights for policy 1, policy_version 445076 (0.0007) [2023-12-26 18:36:08,262][105620] Updated weights for policy 1, policy_version 445086 (0.0010) [2023-12-26 18:36:08,315][105620] Updated weights for policy 1, policy_version 445096 (0.0010) [2023-12-26 18:36:08,925][105620] Updated weights for policy 1, policy_version 445106 (0.0010) [2023-12-26 18:36:08,983][105620] Updated weights for policy 1, policy_version 445116 (0.0010) [2023-12-26 18:36:09,008][105692] Updated weights for policy 0, policy_version 444716 (0.0007) [2023-12-26 18:36:09,045][105620] Updated weights for policy 1, policy_version 445126 (0.0011) [2023-12-26 18:36:09,063][105692] Updated weights for policy 0, policy_version 444726 (0.0006) [2023-12-26 18:36:09,118][105692] Updated weights for policy 0, policy_version 444736 (0.0009) [2023-12-26 18:36:09,833][105620] Updated weights for policy 1, policy_version 445136 (0.0009) [2023-12-26 18:36:09,871][105692] Updated weights for policy 0, policy_version 444746 (0.0009) [2023-12-26 18:36:09,898][105620] Updated weights for policy 1, policy_version 445146 (0.0007) [2023-12-26 18:36:09,936][105692] Updated weights for policy 0, policy_version 444756 (0.0007) [2023-12-26 18:36:09,962][105620] Updated weights for policy 1, policy_version 445156 (0.0007) [2023-12-26 18:36:09,998][105692] Updated weights for policy 0, policy_version 444766 (0.0008) [2023-12-26 18:36:10,072][105692] Updated weights for policy 0, policy_version 444776 (0.0009) [2023-12-26 18:36:10,544][105620] Updated weights for policy 1, policy_version 445166 (0.0006) [2023-12-26 18:36:10,601][105620] Updated weights for policy 1, policy_version 445176 (0.0005) [2023-12-26 18:36:10,648][105620] Updated weights for policy 1, policy_version 445186 (0.0005) [2023-12-26 18:36:10,896][105692] Updated weights for policy 0, policy_version 444786 (0.0009) [2023-12-26 18:36:10,958][105692] Updated weights for policy 0, policy_version 444796 (0.0006) [2023-12-26 18:36:11,025][105692] Updated weights for policy 0, policy_version 444806 (0.0008) [2023-12-26 18:36:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 227868672. Throughput: 0: 9693.1, 1: 9713.3. Samples: 227873472. Policy #0 lag: (min: 24.0, avg: 51.8, max: 56.0) [2023-12-26 18:36:11,062][104569] Avg episode reward: [(0, '9095.012'), (1, '4242.581')] [2023-12-26 18:36:11,385][105620] Updated weights for policy 1, policy_version 445196 (0.0007) [2023-12-26 18:36:11,453][105620] Updated weights for policy 1, policy_version 445206 (0.0008) [2023-12-26 18:36:11,518][105620] Updated weights for policy 1, policy_version 445216 (0.0009) [2023-12-26 18:36:11,771][105692] Updated weights for policy 0, policy_version 444816 (0.0010) [2023-12-26 18:36:11,832][105692] Updated weights for policy 0, policy_version 444826 (0.0011) [2023-12-26 18:36:11,892][105692] Updated weights for policy 0, policy_version 444836 (0.0011) [2023-12-26 18:36:12,204][105620] Updated weights for policy 1, policy_version 445226 (0.0007) [2023-12-26 18:36:12,266][105620] Updated weights for policy 1, policy_version 445236 (0.0009) [2023-12-26 18:36:12,319][105620] Updated weights for policy 1, policy_version 445246 (0.0009) [2023-12-26 18:36:12,386][105620] Updated weights for policy 1, policy_version 445256 (0.0007) [2023-12-26 18:36:12,525][105692] Updated weights for policy 0, policy_version 444846 (0.0008) [2023-12-26 18:36:12,587][105692] Updated weights for policy 0, policy_version 444856 (0.0005) [2023-12-26 18:36:12,652][105692] Updated weights for policy 0, policy_version 444866 (0.0006) [2023-12-26 18:36:13,176][105620] Updated weights for policy 1, policy_version 445266 (0.0009) [2023-12-26 18:36:13,213][105692] Updated weights for policy 0, policy_version 444876 (0.0006) [2023-12-26 18:36:13,234][105620] Updated weights for policy 1, policy_version 445277 (0.0010) [2023-12-26 18:36:13,268][105692] Updated weights for policy 0, policy_version 444886 (0.0006) [2023-12-26 18:36:13,295][105620] Updated weights for policy 1, policy_version 445287 (0.0009) [2023-12-26 18:36:13,308][105585] KL-divergence is very high: 130.2882 [2023-12-26 18:36:13,326][105692] Updated weights for policy 0, policy_version 444896 (0.0008) [2023-12-26 18:36:14,026][105692] Updated weights for policy 0, policy_version 444906 (0.0010) [2023-12-26 18:36:14,069][105620] Updated weights for policy 1, policy_version 445297 (0.0007) [2023-12-26 18:36:14,082][105692] Updated weights for policy 0, policy_version 444916 (0.0007) [2023-12-26 18:36:14,136][105620] Updated weights for policy 1, policy_version 445307 (0.0008) [2023-12-26 18:36:14,142][105692] Updated weights for policy 0, policy_version 444926 (0.0008) [2023-12-26 18:36:14,188][105620] Updated weights for policy 1, policy_version 445317 (0.0006) [2023-12-26 18:36:14,201][105692] Updated weights for policy 0, policy_version 444936 (0.0009) [2023-12-26 18:36:14,801][105692] Updated weights for policy 0, policy_version 444946 (0.0008) [2023-12-26 18:36:14,858][105692] Updated weights for policy 0, policy_version 444956 (0.0009) [2023-12-26 18:36:14,922][105692] Updated weights for policy 0, policy_version 444966 (0.0009) [2023-12-26 18:36:14,965][105620] Updated weights for policy 1, policy_version 445327 (0.0007) [2023-12-26 18:36:15,028][105620] Updated weights for policy 1, policy_version 445337 (0.0007) [2023-12-26 18:36:15,077][105620] Updated weights for policy 1, policy_version 445347 (0.0005) [2023-12-26 18:36:15,664][105692] Updated weights for policy 0, policy_version 444976 (0.0008) [2023-12-26 18:36:15,715][105692] Updated weights for policy 0, policy_version 444986 (0.0009) [2023-12-26 18:36:15,758][105692] Updated weights for policy 0, policy_version 444996 (0.0006) [2023-12-26 18:36:15,814][105620] Updated weights for policy 1, policy_version 445357 (0.0008) [2023-12-26 18:36:15,865][105620] Updated weights for policy 1, policy_version 445367 (0.0009) [2023-12-26 18:36:15,913][105620] Updated weights for policy 1, policy_version 445377 (0.0009) [2023-12-26 18:36:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 227966976. Throughput: 0: 9751.5, 1: 9657.9. Samples: 227931760. Policy #0 lag: (min: 24.0, avg: 51.8, max: 56.0) [2023-12-26 18:36:16,062][104569] Avg episode reward: [(0, '8912.402'), (1, '6982.965')] [2023-12-26 18:36:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000445000_113934336.pth... [2023-12-26 18:36:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000445384_114032640.pth... [2023-12-26 18:36:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000443848_113639424.pth [2023-12-26 18:36:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000444232_113737728.pth [2023-12-26 18:36:16,531][105692] Updated weights for policy 0, policy_version 445006 (0.0009) [2023-12-26 18:36:16,587][105692] Updated weights for policy 0, policy_version 445016 (0.0009) [2023-12-26 18:36:16,642][105692] Updated weights for policy 0, policy_version 445026 (0.0009) [2023-12-26 18:36:16,685][105620] Updated weights for policy 1, policy_version 445387 (0.0008) [2023-12-26 18:36:16,736][105620] Updated weights for policy 1, policy_version 445397 (0.0009) [2023-12-26 18:36:16,797][105620] Updated weights for policy 1, policy_version 445407 (0.0009) [2023-12-26 18:36:17,365][105692] Updated weights for policy 0, policy_version 445036 (0.0008) [2023-12-26 18:36:17,423][105692] Updated weights for policy 0, policy_version 445046 (0.0009) [2023-12-26 18:36:17,469][105692] Updated weights for policy 0, policy_version 445056 (0.0008) [2023-12-26 18:36:17,583][105620] Updated weights for policy 1, policy_version 445417 (0.0009) [2023-12-26 18:36:17,636][105620] Updated weights for policy 1, policy_version 445427 (0.0009) [2023-12-26 18:36:17,687][105620] Updated weights for policy 1, policy_version 445437 (0.0009) [2023-12-26 18:36:17,738][105620] Updated weights for policy 1, policy_version 445447 (0.0009) [2023-12-26 18:36:18,106][105692] Updated weights for policy 0, policy_version 445066 (0.0009) [2023-12-26 18:36:18,153][105692] Updated weights for policy 0, policy_version 445076 (0.0009) [2023-12-26 18:36:18,204][105692] Updated weights for policy 0, policy_version 445086 (0.0009) [2023-12-26 18:36:18,252][105692] Updated weights for policy 0, policy_version 445096 (0.0009) [2023-12-26 18:36:18,583][105620] Updated weights for policy 1, policy_version 445457 (0.0010) [2023-12-26 18:36:18,637][105620] Updated weights for policy 1, policy_version 445468 (0.0010) [2023-12-26 18:36:18,688][105620] Updated weights for policy 1, policy_version 445478 (0.0009) [2023-12-26 18:36:18,955][105692] Updated weights for policy 0, policy_version 445106 (0.0009) [2023-12-26 18:36:19,007][105692] Updated weights for policy 0, policy_version 445116 (0.0009) [2023-12-26 18:36:19,059][105692] Updated weights for policy 0, policy_version 445126 (0.0009) [2023-12-26 18:36:19,494][105620] Updated weights for policy 1, policy_version 445488 (0.0008) [2023-12-26 18:36:19,550][105620] Updated weights for policy 1, policy_version 445498 (0.0009) [2023-12-26 18:36:19,600][105620] Updated weights for policy 1, policy_version 445508 (0.0008) [2023-12-26 18:36:19,885][105692] Updated weights for policy 0, policy_version 445136 (0.0008) [2023-12-26 18:36:19,952][105692] Updated weights for policy 0, policy_version 445146 (0.0008) [2023-12-26 18:36:20,014][105692] Updated weights for policy 0, policy_version 445156 (0.0006) [2023-12-26 18:36:20,398][105620] Updated weights for policy 1, policy_version 445518 (0.0010) [2023-12-26 18:36:20,462][105620] Updated weights for policy 1, policy_version 445528 (0.0011) [2023-12-26 18:36:20,515][105620] Updated weights for policy 1, policy_version 445538 (0.0011) [2023-12-26 18:36:20,643][105692] Updated weights for policy 0, policy_version 445166 (0.0008) [2023-12-26 18:36:20,702][105692] Updated weights for policy 0, policy_version 445176 (0.0008) [2023-12-26 18:36:20,755][105692] Updated weights for policy 0, policy_version 445186 (0.0008) [2023-12-26 18:36:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 228057088. Throughput: 0: 9730.5, 1: 9632.7. Samples: 228045244. Policy #0 lag: (min: 24.0, avg: 51.8, max: 56.0) [2023-12-26 18:36:21,063][104569] Avg episode reward: [(0, '9003.283'), (1, '8282.981')] [2023-12-26 18:36:21,328][105620] Updated weights for policy 1, policy_version 445548 (0.0011) [2023-12-26 18:36:21,402][105620] Updated weights for policy 1, policy_version 445558 (0.0014) [2023-12-26 18:36:21,455][105620] Updated weights for policy 1, policy_version 445568 (0.0009) [2023-12-26 18:36:21,541][105692] Updated weights for policy 0, policy_version 445196 (0.0009) [2023-12-26 18:36:21,607][105692] Updated weights for policy 0, policy_version 445206 (0.0010) [2023-12-26 18:36:21,674][105692] Updated weights for policy 0, policy_version 445216 (0.0009) [2023-12-26 18:36:22,192][105620] Updated weights for policy 1, policy_version 445578 (0.0007) [2023-12-26 18:36:22,238][105620] Updated weights for policy 1, policy_version 445588 (0.0008) [2023-12-26 18:36:22,303][105620] Updated weights for policy 1, policy_version 445598 (0.0009) [2023-12-26 18:36:22,371][105620] Updated weights for policy 1, policy_version 445608 (0.0009) [2023-12-26 18:36:22,449][105692] Updated weights for policy 0, policy_version 445226 (0.0008) [2023-12-26 18:36:22,505][105692] Updated weights for policy 0, policy_version 445236 (0.0009) [2023-12-26 18:36:22,555][105692] Updated weights for policy 0, policy_version 445246 (0.0010) [2023-12-26 18:36:23,122][105620] Updated weights for policy 1, policy_version 445618 (0.0008) [2023-12-26 18:36:23,192][105620] Updated weights for policy 1, policy_version 445628 (0.0010) [2023-12-26 18:36:23,253][105620] Updated weights for policy 1, policy_version 445638 (0.0010) [2023-12-26 18:36:23,324][105692] Updated weights for policy 0, policy_version 445257 (0.0009) [2023-12-26 18:36:23,377][105692] Updated weights for policy 0, policy_version 445267 (0.0007) [2023-12-26 18:36:23,436][105692] Updated weights for policy 0, policy_version 445277 (0.0009) [2023-12-26 18:36:23,498][105692] Updated weights for policy 0, policy_version 445287 (0.0008) [2023-12-26 18:36:23,955][105620] Updated weights for policy 1, policy_version 445648 (0.0006) [2023-12-26 18:36:24,009][105620] Updated weights for policy 1, policy_version 445658 (0.0007) [2023-12-26 18:36:24,062][105620] Updated weights for policy 1, policy_version 445668 (0.0010) [2023-12-26 18:36:24,198][105692] Updated weights for policy 0, policy_version 445297 (0.0006) [2023-12-26 18:36:24,258][105692] Updated weights for policy 0, policy_version 445307 (0.0006) [2023-12-26 18:36:24,320][105692] Updated weights for policy 0, policy_version 445317 (0.0006) [2023-12-26 18:36:24,809][105620] Updated weights for policy 1, policy_version 445678 (0.0007) [2023-12-26 18:36:24,866][105620] Updated weights for policy 1, policy_version 445688 (0.0007) [2023-12-26 18:36:24,925][105620] Updated weights for policy 1, policy_version 445698 (0.0010) [2023-12-26 18:36:25,058][105692] Updated weights for policy 0, policy_version 445327 (0.0006) [2023-12-26 18:36:25,132][105692] Updated weights for policy 0, policy_version 445337 (0.0005) [2023-12-26 18:36:25,203][105692] Updated weights for policy 0, policy_version 445347 (0.0006) [2023-12-26 18:36:25,540][105620] Updated weights for policy 1, policy_version 445708 (0.0009) [2023-12-26 18:36:25,591][105620] Updated weights for policy 1, policy_version 445718 (0.0005) [2023-12-26 18:36:25,646][105620] Updated weights for policy 1, policy_version 445728 (0.0005) [2023-12-26 18:36:25,933][105692] Updated weights for policy 0, policy_version 445357 (0.0008) [2023-12-26 18:36:25,999][105692] Updated weights for policy 0, policy_version 445367 (0.0010) [2023-12-26 18:36:26,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 228147200. Throughput: 0: 9724.1, 1: 9673.7. Samples: 228160248. Policy #0 lag: (min: 24.0, avg: 51.8, max: 56.0) [2023-12-26 18:36:26,063][104569] Avg episode reward: [(0, '9266.389'), (1, '8321.991')] [2023-12-26 18:36:26,071][105692] Updated weights for policy 0, policy_version 445377 (0.0009) [2023-12-26 18:36:26,221][105620] Updated weights for policy 1, policy_version 445738 (0.0006) [2023-12-26 18:36:26,289][105620] Updated weights for policy 1, policy_version 445748 (0.0008) [2023-12-26 18:36:26,353][105620] Updated weights for policy 1, policy_version 445758 (0.0008) [2023-12-26 18:36:26,419][105620] Updated weights for policy 1, policy_version 445768 (0.0009) [2023-12-26 18:36:26,897][105692] Updated weights for policy 0, policy_version 445387 (0.0009) [2023-12-26 18:36:26,948][105692] Updated weights for policy 0, policy_version 445397 (0.0009) [2023-12-26 18:36:27,002][105692] Updated weights for policy 0, policy_version 445407 (0.0009) [2023-12-26 18:36:27,015][105620] Updated weights for policy 1, policy_version 445778 (0.0005) [2023-12-26 18:36:27,080][105620] Updated weights for policy 1, policy_version 445788 (0.0008) [2023-12-26 18:36:27,145][105620] Updated weights for policy 1, policy_version 445798 (0.0010) [2023-12-26 18:36:27,781][105692] Updated weights for policy 0, policy_version 445417 (0.0009) [2023-12-26 18:36:27,831][105692] Updated weights for policy 0, policy_version 445427 (0.0007) [2023-12-26 18:36:27,845][105620] Updated weights for policy 1, policy_version 445808 (0.0007) [2023-12-26 18:36:27,888][105692] Updated weights for policy 0, policy_version 445437 (0.0010) [2023-12-26 18:36:27,904][105620] Updated weights for policy 1, policy_version 445818 (0.0005) [2023-12-26 18:36:27,940][105692] Updated weights for policy 0, policy_version 445447 (0.0009) [2023-12-26 18:36:27,954][105620] Updated weights for policy 1, policy_version 445828 (0.0005) [2023-12-26 18:36:28,531][105620] Updated weights for policy 1, policy_version 445838 (0.0005) [2023-12-26 18:36:28,581][105620] Updated weights for policy 1, policy_version 445848 (0.0005) [2023-12-26 18:36:28,644][105620] Updated weights for policy 1, policy_version 445858 (0.0008) [2023-12-26 18:36:28,820][105692] Updated weights for policy 0, policy_version 445457 (0.0010) [2023-12-26 18:36:28,872][105692] Updated weights for policy 0, policy_version 445467 (0.0009) [2023-12-26 18:36:28,921][105692] Updated weights for policy 0, policy_version 445477 (0.0008) [2023-12-26 18:36:28,925][105585] KL-divergence is very high: 201.4260 [2023-12-26 18:36:29,275][105620] Updated weights for policy 1, policy_version 445868 (0.0009) [2023-12-26 18:36:29,339][105620] Updated weights for policy 1, policy_version 445878 (0.0009) [2023-12-26 18:36:29,406][105620] Updated weights for policy 1, policy_version 445888 (0.0010) [2023-12-26 18:36:29,724][105692] Updated weights for policy 0, policy_version 445487 (0.0008) [2023-12-26 18:36:29,769][105692] Updated weights for policy 0, policy_version 445497 (0.0008) [2023-12-26 18:36:29,813][105692] Updated weights for policy 0, policy_version 445507 (0.0008) [2023-12-26 18:36:30,147][105620] Updated weights for policy 1, policy_version 445898 (0.0010) [2023-12-26 18:36:30,208][105620] Updated weights for policy 1, policy_version 445908 (0.0010) [2023-12-26 18:36:30,269][105620] Updated weights for policy 1, policy_version 445918 (0.0010) [2023-12-26 18:36:30,327][105620] Updated weights for policy 1, policy_version 445928 (0.0010) [2023-12-26 18:36:30,614][105692] Updated weights for policy 0, policy_version 445517 (0.0009) [2023-12-26 18:36:30,673][105692] Updated weights for policy 0, policy_version 445527 (0.0008) [2023-12-26 18:36:30,730][105692] Updated weights for policy 0, policy_version 445537 (0.0009) [2023-12-26 18:36:30,958][105620] Updated weights for policy 1, policy_version 445938 (0.0010) [2023-12-26 18:36:31,011][105620] Updated weights for policy 1, policy_version 445948 (0.0010) [2023-12-26 18:36:31,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 228245504. Throughput: 0: 9663.3, 1: 9758.5. Samples: 228219080. Policy #0 lag: (min: 24.0, avg: 51.8, max: 56.0) [2023-12-26 18:36:31,062][104569] Avg episode reward: [(0, '9179.972'), (1, '8732.717')] [2023-12-26 18:36:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000445544_114073600.pth... [2023-12-26 18:36:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000444424_113786880.pth [2023-12-26 18:36:31,078][105620] Updated weights for policy 1, policy_version 445958 (0.0010) [2023-12-26 18:36:31,090][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000445960_114180096.pth... [2023-12-26 18:36:31,095][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000444840_113893376.pth [2023-12-26 18:36:31,396][105692] Updated weights for policy 0, policy_version 445547 (0.0007) [2023-12-26 18:36:31,460][105692] Updated weights for policy 0, policy_version 445557 (0.0005) [2023-12-26 18:36:31,528][105692] Updated weights for policy 0, policy_version 445567 (0.0008) [2023-12-26 18:36:31,834][105620] Updated weights for policy 1, policy_version 445968 (0.0010) [2023-12-26 18:36:31,898][105620] Updated weights for policy 1, policy_version 445978 (0.0010) [2023-12-26 18:36:31,967][105620] Updated weights for policy 1, policy_version 445988 (0.0010) [2023-12-26 18:36:32,177][105692] Updated weights for policy 0, policy_version 445577 (0.0008) [2023-12-26 18:36:32,241][105692] Updated weights for policy 0, policy_version 445587 (0.0008) [2023-12-26 18:36:32,306][105692] Updated weights for policy 0, policy_version 445597 (0.0009) [2023-12-26 18:36:32,374][105692] Updated weights for policy 0, policy_version 445607 (0.0008) [2023-12-26 18:36:32,656][105620] Updated weights for policy 1, policy_version 445998 (0.0010) [2023-12-26 18:36:32,708][105620] Updated weights for policy 1, policy_version 446008 (0.0010) [2023-12-26 18:36:32,766][105620] Updated weights for policy 1, policy_version 446018 (0.0010) [2023-12-26 18:36:33,121][105692] Updated weights for policy 0, policy_version 445617 (0.0010) [2023-12-26 18:36:33,175][105692] Updated weights for policy 0, policy_version 445627 (0.0010) [2023-12-26 18:36:33,229][105692] Updated weights for policy 0, policy_version 445637 (0.0010) [2023-12-26 18:36:33,367][105620] Updated weights for policy 1, policy_version 446028 (0.0010) [2023-12-26 18:36:33,423][105620] Updated weights for policy 1, policy_version 446038 (0.0010) [2023-12-26 18:36:33,480][105620] Updated weights for policy 1, policy_version 446048 (0.0010) [2023-12-26 18:36:33,983][105692] Updated weights for policy 0, policy_version 445647 (0.0008) [2023-12-26 18:36:34,031][105692] Updated weights for policy 0, policy_version 445657 (0.0008) [2023-12-26 18:36:34,083][105692] Updated weights for policy 0, policy_version 445667 (0.0008) [2023-12-26 18:36:34,215][105620] Updated weights for policy 1, policy_version 446058 (0.0010) [2023-12-26 18:36:34,271][105620] Updated weights for policy 1, policy_version 446068 (0.0009) [2023-12-26 18:36:34,328][105620] Updated weights for policy 1, policy_version 446078 (0.0009) [2023-12-26 18:36:34,392][105620] Updated weights for policy 1, policy_version 446088 (0.0009) [2023-12-26 18:36:34,858][105692] Updated weights for policy 0, policy_version 445677 (0.0008) [2023-12-26 18:36:34,912][105692] Updated weights for policy 0, policy_version 445687 (0.0008) [2023-12-26 18:36:34,973][105692] Updated weights for policy 0, policy_version 445697 (0.0008) [2023-12-26 18:36:35,205][105620] Updated weights for policy 1, policy_version 446098 (0.0010) [2023-12-26 18:36:35,253][105620] Updated weights for policy 1, policy_version 446108 (0.0010) [2023-12-26 18:36:35,301][105620] Updated weights for policy 1, policy_version 446118 (0.0010) [2023-12-26 18:36:35,752][105692] Updated weights for policy 0, policy_version 445707 (0.0008) [2023-12-26 18:36:35,807][105692] Updated weights for policy 0, policy_version 445717 (0.0008) [2023-12-26 18:36:35,855][105692] Updated weights for policy 0, policy_version 445727 (0.0008) [2023-12-26 18:36:36,051][105620] Updated weights for policy 1, policy_version 446128 (0.0010) [2023-12-26 18:36:36,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 228343808. Throughput: 0: 9670.3, 1: 9697.9. Samples: 228334332. Policy #0 lag: (min: 24.0, avg: 51.8, max: 56.0) [2023-12-26 18:36:36,062][104569] Avg episode reward: [(0, '9090.795'), (1, '8463.083')] [2023-12-26 18:36:36,099][105620] Updated weights for policy 1, policy_version 446138 (0.0010) [2023-12-26 18:36:36,167][105620] Updated weights for policy 1, policy_version 446148 (0.0010) [2023-12-26 18:36:36,625][105692] Updated weights for policy 0, policy_version 445737 (0.0008) [2023-12-26 18:36:36,688][105692] Updated weights for policy 0, policy_version 445747 (0.0011) [2023-12-26 18:36:36,750][105692] Updated weights for policy 0, policy_version 445757 (0.0011) [2023-12-26 18:36:36,810][105692] Updated weights for policy 0, policy_version 445767 (0.0011) [2023-12-26 18:36:36,897][105620] Updated weights for policy 1, policy_version 446158 (0.0009) [2023-12-26 18:36:36,954][105620] Updated weights for policy 1, policy_version 446168 (0.0007) [2023-12-26 18:36:37,005][105620] Updated weights for policy 1, policy_version 446178 (0.0009) [2023-12-26 18:36:37,464][105692] Updated weights for policy 0, policy_version 445777 (0.0007) [2023-12-26 18:36:37,523][105692] Updated weights for policy 0, policy_version 445787 (0.0009) [2023-12-26 18:36:37,582][105692] Updated weights for policy 0, policy_version 445797 (0.0006) [2023-12-26 18:36:37,642][105620] Updated weights for policy 1, policy_version 446188 (0.0008) [2023-12-26 18:36:37,701][105620] Updated weights for policy 1, policy_version 446198 (0.0009) [2023-12-26 18:36:37,760][105620] Updated weights for policy 1, policy_version 446208 (0.0009) [2023-12-26 18:36:38,212][105692] Updated weights for policy 0, policy_version 445807 (0.0005) [2023-12-26 18:36:38,266][105692] Updated weights for policy 0, policy_version 445817 (0.0005) [2023-12-26 18:36:38,323][105692] Updated weights for policy 0, policy_version 445827 (0.0006) [2023-12-26 18:36:38,529][105620] Updated weights for policy 1, policy_version 446218 (0.0009) [2023-12-26 18:36:38,586][105620] Updated weights for policy 1, policy_version 446228 (0.0009) [2023-12-26 18:36:38,636][105620] Updated weights for policy 1, policy_version 446238 (0.0008) [2023-12-26 18:36:38,692][105620] Updated weights for policy 1, policy_version 446248 (0.0008) [2023-12-26 18:36:39,010][105692] Updated weights for policy 0, policy_version 445837 (0.0009) [2023-12-26 18:36:39,074][105692] Updated weights for policy 0, policy_version 445847 (0.0009) [2023-12-26 18:36:39,132][105692] Updated weights for policy 0, policy_version 445857 (0.0010) [2023-12-26 18:36:39,449][105620] Updated weights for policy 1, policy_version 446258 (0.0009) [2023-12-26 18:36:39,513][105620] Updated weights for policy 1, policy_version 446268 (0.0009) [2023-12-26 18:36:39,570][105620] Updated weights for policy 1, policy_version 446278 (0.0005) [2023-12-26 18:36:39,917][105692] Updated weights for policy 0, policy_version 445867 (0.0008) [2023-12-26 18:36:39,952][105585] KL-divergence is very high: 196.5747 [2023-12-26 18:36:39,982][105692] Updated weights for policy 0, policy_version 445877 (0.0008) [2023-12-26 18:36:40,003][105585] KL-divergence is very high: 322.3421 [2023-12-26 18:36:40,053][105692] Updated weights for policy 0, policy_version 445887 (0.0008) [2023-12-26 18:36:40,059][105585] KL-divergence is very high: 232.4059 [2023-12-26 18:36:40,269][105620] Updated weights for policy 1, policy_version 446288 (0.0007) [2023-12-26 18:36:40,338][105620] Updated weights for policy 1, policy_version 446298 (0.0008) [2023-12-26 18:36:40,406][105620] Updated weights for policy 1, policy_version 446308 (0.0008) [2023-12-26 18:36:40,820][105692] Updated weights for policy 0, policy_version 445897 (0.0009) [2023-12-26 18:36:40,878][105692] Updated weights for policy 0, policy_version 445907 (0.0010) [2023-12-26 18:36:40,936][105692] Updated weights for policy 0, policy_version 445917 (0.0010) [2023-12-26 18:36:40,991][105692] Updated weights for policy 0, policy_version 445927 (0.0010) [2023-12-26 18:36:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 228442112. Throughput: 0: 9694.2, 1: 9655.2. Samples: 228449996. Policy #0 lag: (min: 24.0, avg: 51.8, max: 56.0) [2023-12-26 18:36:41,062][104569] Avg episode reward: [(0, '9175.506'), (1, '7621.690')] [2023-12-26 18:36:41,162][105620] Updated weights for policy 1, policy_version 446318 (0.0008) [2023-12-26 18:36:41,226][105620] Updated weights for policy 1, policy_version 446328 (0.0008) [2023-12-26 18:36:41,289][105620] Updated weights for policy 1, policy_version 446338 (0.0008) [2023-12-26 18:36:41,862][105692] Updated weights for policy 0, policy_version 445937 (0.0010) [2023-12-26 18:36:41,922][105692] Updated weights for policy 0, policy_version 445947 (0.0011) [2023-12-26 18:36:41,982][105692] Updated weights for policy 0, policy_version 445957 (0.0011) [2023-12-26 18:36:42,057][105620] Updated weights for policy 1, policy_version 446348 (0.0008) [2023-12-26 18:36:42,121][105620] Updated weights for policy 1, policy_version 446358 (0.0009) [2023-12-26 18:36:42,183][105620] Updated weights for policy 1, policy_version 446368 (0.0008) [2023-12-26 18:36:42,729][105692] Updated weights for policy 0, policy_version 445967 (0.0010) [2023-12-26 18:36:42,788][105692] Updated weights for policy 0, policy_version 445977 (0.0010) [2023-12-26 18:36:42,840][105692] Updated weights for policy 0, policy_version 445987 (0.0010) [2023-12-26 18:36:42,932][105620] Updated weights for policy 1, policy_version 446378 (0.0009) [2023-12-26 18:36:42,985][105620] Updated weights for policy 1, policy_version 446389 (0.0009) [2023-12-26 18:36:43,037][105620] Updated weights for policy 1, policy_version 446399 (0.0010) [2023-12-26 18:36:43,462][105692] Updated weights for policy 0, policy_version 445997 (0.0008) [2023-12-26 18:36:43,515][105692] Updated weights for policy 0, policy_version 446007 (0.0005) [2023-12-26 18:36:43,564][105692] Updated weights for policy 0, policy_version 446017 (0.0005) [2023-12-26 18:36:43,916][105620] Updated weights for policy 1, policy_version 446410 (0.0009) [2023-12-26 18:36:43,967][105620] Updated weights for policy 1, policy_version 446420 (0.0008) [2023-12-26 18:36:44,024][105620] Updated weights for policy 1, policy_version 446430 (0.0009) [2023-12-26 18:36:44,077][105620] Updated weights for policy 1, policy_version 446440 (0.0008) [2023-12-26 18:36:44,167][105692] Updated weights for policy 0, policy_version 446027 (0.0006) [2023-12-26 18:36:44,227][105692] Updated weights for policy 0, policy_version 446037 (0.0006) [2023-12-26 18:36:44,282][105692] Updated weights for policy 0, policy_version 446047 (0.0005) [2023-12-26 18:36:44,796][105692] Updated weights for policy 0, policy_version 446057 (0.0006) [2023-12-26 18:36:44,849][105692] Updated weights for policy 0, policy_version 446067 (0.0005) [2023-12-26 18:36:44,912][105692] Updated weights for policy 0, policy_version 446077 (0.0006) [2023-12-26 18:36:44,964][105692] Updated weights for policy 0, policy_version 446087 (0.0006) [2023-12-26 18:36:44,971][105620] Updated weights for policy 1, policy_version 446450 (0.0010) [2023-12-26 18:36:45,032][105620] Updated weights for policy 1, policy_version 446460 (0.0009) [2023-12-26 18:36:45,094][105620] Updated weights for policy 1, policy_version 446470 (0.0008) [2023-12-26 18:36:45,679][105692] Updated weights for policy 0, policy_version 446097 (0.0010) [2023-12-26 18:36:45,737][105692] Updated weights for policy 0, policy_version 446107 (0.0010) [2023-12-26 18:36:45,791][105692] Updated weights for policy 0, policy_version 446117 (0.0010) [2023-12-26 18:36:45,828][105620] Updated weights for policy 1, policy_version 446480 (0.0006) [2023-12-26 18:36:45,882][105620] Updated weights for policy 1, policy_version 446490 (0.0006) [2023-12-26 18:36:45,937][105620] Updated weights for policy 1, policy_version 446500 (0.0010) [2023-12-26 18:36:46,062][104569] Fps is (10 sec: 19660.0, 60 sec: 19387.6, 300 sec: 19494.2). Total num frames: 228540416. Throughput: 0: 9643.0, 1: 9527.6. Samples: 228505488. Policy #0 lag: (min: 24.0, avg: 51.8, max: 56.0) [2023-12-26 18:36:46,063][104569] Avg episode reward: [(0, '9086.587'), (1, '7804.770')] [2023-12-26 18:36:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000446504_114319360.pth... [2023-12-26 18:36:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000446120_114221056.pth... [2023-12-26 18:36:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000445384_114032640.pth [2023-12-26 18:36:46,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000445000_113934336.pth [2023-12-26 18:36:46,413][105692] Updated weights for policy 0, policy_version 446127 (0.0007) [2023-12-26 18:36:46,469][105692] Updated weights for policy 0, policy_version 446137 (0.0010) [2023-12-26 18:36:46,534][105692] Updated weights for policy 0, policy_version 446147 (0.0008) [2023-12-26 18:36:46,572][105620] Updated weights for policy 1, policy_version 446510 (0.0008) [2023-12-26 18:36:46,630][105620] Updated weights for policy 1, policy_version 446520 (0.0010) [2023-12-26 18:36:46,682][105620] Updated weights for policy 1, policy_version 446530 (0.0010) [2023-12-26 18:36:47,115][105692] Updated weights for policy 0, policy_version 446157 (0.0006) [2023-12-26 18:36:47,168][105692] Updated weights for policy 0, policy_version 446167 (0.0006) [2023-12-26 18:36:47,222][105692] Updated weights for policy 0, policy_version 446177 (0.0006) [2023-12-26 18:36:47,375][105620] Updated weights for policy 1, policy_version 446540 (0.0009) [2023-12-26 18:36:47,443][105620] Updated weights for policy 1, policy_version 446550 (0.0005) [2023-12-26 18:36:47,506][105620] Updated weights for policy 1, policy_version 446560 (0.0005) [2023-12-26 18:36:47,944][105692] Updated weights for policy 0, policy_version 446187 (0.0010) [2023-12-26 18:36:47,992][105692] Updated weights for policy 0, policy_version 446197 (0.0008) [2023-12-26 18:36:48,043][105620] Updated weights for policy 1, policy_version 446570 (0.0006) [2023-12-26 18:36:48,046][105692] Updated weights for policy 0, policy_version 446207 (0.0005) [2023-12-26 18:36:48,092][105620] Updated weights for policy 1, policy_version 446580 (0.0010) [2023-12-26 18:36:48,140][105620] Updated weights for policy 1, policy_version 446590 (0.0010) [2023-12-26 18:36:48,189][105620] Updated weights for policy 1, policy_version 446600 (0.0010) [2023-12-26 18:36:48,825][105692] Updated weights for policy 0, policy_version 446217 (0.0006) [2023-12-26 18:36:48,874][105692] Updated weights for policy 0, policy_version 446227 (0.0008) [2023-12-26 18:36:48,923][105692] Updated weights for policy 0, policy_version 446237 (0.0006) [2023-12-26 18:36:48,931][105620] Updated weights for policy 1, policy_version 446610 (0.0009) [2023-12-26 18:36:48,975][105692] Updated weights for policy 0, policy_version 446247 (0.0010) [2023-12-26 18:36:48,985][105620] Updated weights for policy 1, policy_version 446620 (0.0005) [2023-12-26 18:36:49,046][105620] Updated weights for policy 1, policy_version 446630 (0.0005) [2023-12-26 18:36:49,700][105620] Updated weights for policy 1, policy_version 446640 (0.0008) [2023-12-26 18:36:49,723][105692] Updated weights for policy 0, policy_version 446257 (0.0008) [2023-12-26 18:36:49,763][105620] Updated weights for policy 1, policy_version 446650 (0.0009) [2023-12-26 18:36:49,770][105692] Updated weights for policy 0, policy_version 446267 (0.0007) [2023-12-26 18:36:49,819][105620] Updated weights for policy 1, policy_version 446660 (0.0007) [2023-12-26 18:36:49,824][105692] Updated weights for policy 0, policy_version 446277 (0.0008) [2023-12-26 18:36:50,577][105620] Updated weights for policy 1, policy_version 446670 (0.0008) [2023-12-26 18:36:50,602][105692] Updated weights for policy 0, policy_version 446287 (0.0010) [2023-12-26 18:36:50,640][105620] Updated weights for policy 1, policy_version 446680 (0.0006) [2023-12-26 18:36:50,663][105692] Updated weights for policy 0, policy_version 446297 (0.0011) [2023-12-26 18:36:50,698][105620] Updated weights for policy 1, policy_version 446690 (0.0006) [2023-12-26 18:36:50,720][105692] Updated weights for policy 0, policy_version 446307 (0.0010) [2023-12-26 18:36:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 228638720. Throughput: 0: 9682.8, 1: 9662.4. Samples: 228627472. Policy #0 lag: (min: 24.0, avg: 51.8, max: 56.0) [2023-12-26 18:36:51,063][104569] Avg episode reward: [(0, '9178.203'), (1, '8069.875')] [2023-12-26 18:36:51,361][105620] Updated weights for policy 1, policy_version 446700 (0.0008) [2023-12-26 18:36:51,420][105620] Updated weights for policy 1, policy_version 446710 (0.0008) [2023-12-26 18:36:51,473][105620] Updated weights for policy 1, policy_version 446720 (0.0008) [2023-12-26 18:36:51,508][105692] Updated weights for policy 0, policy_version 446317 (0.0011) [2023-12-26 18:36:51,570][105692] Updated weights for policy 0, policy_version 446327 (0.0011) [2023-12-26 18:36:51,636][105692] Updated weights for policy 0, policy_version 446337 (0.0011) [2023-12-26 18:36:52,284][105620] Updated weights for policy 1, policy_version 446730 (0.0007) [2023-12-26 18:36:52,337][105620] Updated weights for policy 1, policy_version 446740 (0.0008) [2023-12-26 18:36:52,377][105692] Updated weights for policy 0, policy_version 446347 (0.0010) [2023-12-26 18:36:52,396][105620] Updated weights for policy 1, policy_version 446750 (0.0007) [2023-12-26 18:36:52,436][105692] Updated weights for policy 0, policy_version 446357 (0.0010) [2023-12-26 18:36:52,455][105620] Updated weights for policy 1, policy_version 446760 (0.0005) [2023-12-26 18:36:52,493][105692] Updated weights for policy 0, policy_version 446367 (0.0010) [2023-12-26 18:36:53,099][105620] Updated weights for policy 1, policy_version 446770 (0.0009) [2023-12-26 18:36:53,156][105620] Updated weights for policy 1, policy_version 446780 (0.0009) [2023-12-26 18:36:53,218][105620] Updated weights for policy 1, policy_version 446790 (0.0009) [2023-12-26 18:36:53,276][105692] Updated weights for policy 0, policy_version 446377 (0.0008) [2023-12-26 18:36:53,333][105692] Updated weights for policy 0, policy_version 446387 (0.0007) [2023-12-26 18:36:53,385][105692] Updated weights for policy 0, policy_version 446397 (0.0009) [2023-12-26 18:36:53,444][105692] Updated weights for policy 0, policy_version 446408 (0.0012) [2023-12-26 18:36:53,864][105620] Updated weights for policy 1, policy_version 446800 (0.0010) [2023-12-26 18:36:53,924][105620] Updated weights for policy 1, policy_version 446810 (0.0009) [2023-12-26 18:36:53,985][105620] Updated weights for policy 1, policy_version 446820 (0.0009) [2023-12-26 18:36:54,098][105692] Updated weights for policy 0, policy_version 446418 (0.0009) [2023-12-26 18:36:54,149][105692] Updated weights for policy 0, policy_version 446428 (0.0008) [2023-12-26 18:36:54,206][105692] Updated weights for policy 0, policy_version 446438 (0.0009) [2023-12-26 18:36:54,748][105620] Updated weights for policy 1, policy_version 446830 (0.0010) [2023-12-26 18:36:54,804][105620] Updated weights for policy 1, policy_version 446840 (0.0009) [2023-12-26 18:36:54,858][105620] Updated weights for policy 1, policy_version 446850 (0.0009) [2023-12-26 18:36:54,968][105692] Updated weights for policy 0, policy_version 446448 (0.0008) [2023-12-26 18:36:55,035][105692] Updated weights for policy 0, policy_version 446458 (0.0008) [2023-12-26 18:36:55,094][105692] Updated weights for policy 0, policy_version 446468 (0.0008) [2023-12-26 18:36:55,631][105620] Updated weights for policy 1, policy_version 446860 (0.0010) [2023-12-26 18:36:55,685][105620] Updated weights for policy 1, policy_version 446870 (0.0009) [2023-12-26 18:36:55,733][105620] Updated weights for policy 1, policy_version 446880 (0.0009) [2023-12-26 18:36:55,843][105692] Updated weights for policy 0, policy_version 446478 (0.0009) [2023-12-26 18:36:55,909][105692] Updated weights for policy 0, policy_version 446488 (0.0009) [2023-12-26 18:36:55,970][105692] Updated weights for policy 0, policy_version 446498 (0.0010) [2023-12-26 18:36:56,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.1, 300 sec: 19466.4). Total num frames: 228737024. Throughput: 0: 9662.2, 1: 9625.6. Samples: 228741436. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-26 18:36:56,063][104569] Avg episode reward: [(0, '8727.519'), (1, '8558.627')] [2023-12-26 18:36:56,410][105620] Updated weights for policy 1, policy_version 446890 (0.0009) [2023-12-26 18:36:56,473][105620] Updated weights for policy 1, policy_version 446900 (0.0009) [2023-12-26 18:36:56,531][105620] Updated weights for policy 1, policy_version 446910 (0.0009) [2023-12-26 18:36:56,587][105620] Updated weights for policy 1, policy_version 446920 (0.0006) [2023-12-26 18:36:56,799][105692] Updated weights for policy 0, policy_version 446508 (0.0010) [2023-12-26 18:36:56,852][105692] Updated weights for policy 0, policy_version 446518 (0.0010) [2023-12-26 18:36:56,914][105692] Updated weights for policy 0, policy_version 446528 (0.0010) [2023-12-26 18:36:57,185][105620] Updated weights for policy 1, policy_version 446930 (0.0009) [2023-12-26 18:36:57,232][105620] Updated weights for policy 1, policy_version 446940 (0.0009) [2023-12-26 18:36:57,278][105620] Updated weights for policy 1, policy_version 446950 (0.0009) [2023-12-26 18:36:57,732][105692] Updated weights for policy 0, policy_version 446538 (0.0009) [2023-12-26 18:36:57,794][105692] Updated weights for policy 0, policy_version 446548 (0.0009) [2023-12-26 18:36:57,842][105692] Updated weights for policy 0, policy_version 446558 (0.0009) [2023-12-26 18:36:57,900][105692] Updated weights for policy 0, policy_version 446568 (0.0009) [2023-12-26 18:36:57,942][105620] Updated weights for policy 1, policy_version 446960 (0.0006) [2023-12-26 18:36:57,998][105620] Updated weights for policy 1, policy_version 446970 (0.0006) [2023-12-26 18:36:58,059][105620] Updated weights for policy 1, policy_version 446980 (0.0007) [2023-12-26 18:36:58,687][105692] Updated weights for policy 0, policy_version 446578 (0.0009) [2023-12-26 18:36:58,749][105692] Updated weights for policy 0, policy_version 446588 (0.0009) [2023-12-26 18:36:58,759][105620] Updated weights for policy 1, policy_version 446990 (0.0008) [2023-12-26 18:36:58,816][105692] Updated weights for policy 0, policy_version 446598 (0.0007) [2023-12-26 18:36:58,835][105620] Updated weights for policy 1, policy_version 447000 (0.0008) [2023-12-26 18:36:58,903][105620] Updated weights for policy 1, policy_version 447010 (0.0008) [2023-12-26 18:36:59,611][105620] Updated weights for policy 1, policy_version 447020 (0.0008) [2023-12-26 18:36:59,640][105692] Updated weights for policy 0, policy_version 446608 (0.0009) [2023-12-26 18:36:59,666][105620] Updated weights for policy 1, policy_version 447030 (0.0008) [2023-12-26 18:36:59,700][105692] Updated weights for policy 0, policy_version 446618 (0.0006) [2023-12-26 18:36:59,726][105620] Updated weights for policy 1, policy_version 447040 (0.0007) [2023-12-26 18:36:59,755][105692] Updated weights for policy 0, policy_version 446628 (0.0007) [2023-12-26 18:37:00,453][105620] Updated weights for policy 1, policy_version 447050 (0.0009) [2023-12-26 18:37:00,509][105620] Updated weights for policy 1, policy_version 447060 (0.0009) [2023-12-26 18:37:00,524][105692] Updated weights for policy 0, policy_version 446638 (0.0009) [2023-12-26 18:37:00,558][105620] Updated weights for policy 1, policy_version 447070 (0.0007) [2023-12-26 18:37:00,568][105692] Updated weights for policy 0, policy_version 446648 (0.0006) [2023-12-26 18:37:00,606][105620] Updated weights for policy 1, policy_version 447080 (0.0006) [2023-12-26 18:37:00,617][105692] Updated weights for policy 0, policy_version 446658 (0.0007) [2023-12-26 18:37:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19438.7). Total num frames: 228827136. Throughput: 0: 9568.3, 1: 9696.3. Samples: 228798672. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-26 18:37:01,063][104569] Avg episode reward: [(0, '8732.124'), (1, '8559.066')] [2023-12-26 18:37:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000446664_114360320.pth... [2023-12-26 18:37:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000447080_114466816.pth... [2023-12-26 18:37:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000445544_114073600.pth [2023-12-26 18:37:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000445960_114180096.pth [2023-12-26 18:37:01,378][105692] Updated weights for policy 0, policy_version 446668 (0.0007) [2023-12-26 18:37:01,384][105620] Updated weights for policy 1, policy_version 447090 (0.0008) [2023-12-26 18:37:01,436][105692] Updated weights for policy 0, policy_version 446678 (0.0006) [2023-12-26 18:37:01,446][105620] Updated weights for policy 1, policy_version 447100 (0.0008) [2023-12-26 18:37:01,500][105692] Updated weights for policy 0, policy_version 446688 (0.0007) [2023-12-26 18:37:01,511][105620] Updated weights for policy 1, policy_version 447110 (0.0008) [2023-12-26 18:37:02,145][105692] Updated weights for policy 0, policy_version 446698 (0.0008) [2023-12-26 18:37:02,191][105692] Updated weights for policy 0, policy_version 446708 (0.0005) [2023-12-26 18:37:02,238][105692] Updated weights for policy 0, policy_version 446718 (0.0005) [2023-12-26 18:37:02,288][105692] Updated weights for policy 0, policy_version 446728 (0.0006) [2023-12-26 18:37:02,375][105620] Updated weights for policy 1, policy_version 447120 (0.0008) [2023-12-26 18:37:02,430][105620] Updated weights for policy 1, policy_version 447130 (0.0009) [2023-12-26 18:37:02,482][105620] Updated weights for policy 1, policy_version 447140 (0.0009) [2023-12-26 18:37:02,852][105692] Updated weights for policy 0, policy_version 446738 (0.0005) [2023-12-26 18:37:02,910][105692] Updated weights for policy 0, policy_version 446748 (0.0005) [2023-12-26 18:37:02,965][105692] Updated weights for policy 0, policy_version 446758 (0.0005) [2023-12-26 18:37:03,295][105620] Updated weights for policy 1, policy_version 447150 (0.0007) [2023-12-26 18:37:03,355][105620] Updated weights for policy 1, policy_version 447160 (0.0005) [2023-12-26 18:37:03,410][105620] Updated weights for policy 1, policy_version 447170 (0.0007) [2023-12-26 18:37:03,471][105692] Updated weights for policy 0, policy_version 446768 (0.0009) [2023-12-26 18:37:03,512][105692] Updated weights for policy 0, policy_version 446778 (0.0008) [2023-12-26 18:37:03,560][105692] Updated weights for policy 0, policy_version 446788 (0.0005) [2023-12-26 18:37:04,142][105620] Updated weights for policy 1, policy_version 447180 (0.0009) [2023-12-26 18:37:04,157][105692] Updated weights for policy 0, policy_version 446798 (0.0008) [2023-12-26 18:37:04,199][105620] Updated weights for policy 1, policy_version 447190 (0.0006) [2023-12-26 18:37:04,213][105692] Updated weights for policy 0, policy_version 446808 (0.0011) [2023-12-26 18:37:04,263][105620] Updated weights for policy 1, policy_version 447200 (0.0005) [2023-12-26 18:37:04,273][105692] Updated weights for policy 0, policy_version 446818 (0.0011) [2023-12-26 18:37:05,019][105692] Updated weights for policy 0, policy_version 446828 (0.0010) [2023-12-26 18:37:05,020][105620] Updated weights for policy 1, policy_version 447210 (0.0006) [2023-12-26 18:37:05,068][105620] Updated weights for policy 1, policy_version 447220 (0.0006) [2023-12-26 18:37:05,073][105692] Updated weights for policy 0, policy_version 446838 (0.0010) [2023-12-26 18:37:05,122][105620] Updated weights for policy 1, policy_version 447230 (0.0006) [2023-12-26 18:37:05,127][105692] Updated weights for policy 0, policy_version 446848 (0.0010) [2023-12-26 18:37:05,180][105620] Updated weights for policy 1, policy_version 447240 (0.0006) [2023-12-26 18:37:05,804][105692] Updated weights for policy 0, policy_version 446858 (0.0009) [2023-12-26 18:37:05,861][105692] Updated weights for policy 0, policy_version 446868 (0.0005) [2023-12-26 18:37:05,911][105692] Updated weights for policy 0, policy_version 446878 (0.0005) [2023-12-26 18:37:05,975][105692] Updated weights for policy 0, policy_version 446888 (0.0005) [2023-12-26 18:37:05,997][105620] Updated weights for policy 1, policy_version 447250 (0.0009) [2023-12-26 18:37:06,052][105620] Updated weights for policy 1, policy_version 447260 (0.0009) [2023-12-26 18:37:06,062][104569] Fps is (10 sec: 18842.5, 60 sec: 19251.1, 300 sec: 19466.4). Total num frames: 228925440. Throughput: 0: 9621.8, 1: 9723.3. Samples: 228915772. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-26 18:37:06,063][104569] Avg episode reward: [(0, '8911.659'), (1, '8382.388')] [2023-12-26 18:37:06,109][105620] Updated weights for policy 1, policy_version 447270 (0.0009) [2023-12-26 18:37:06,508][105692] Updated weights for policy 0, policy_version 446898 (0.0007) [2023-12-26 18:37:06,569][105692] Updated weights for policy 0, policy_version 446908 (0.0005) [2023-12-26 18:37:06,623][105692] Updated weights for policy 0, policy_version 446918 (0.0008) [2023-12-26 18:37:06,967][105620] Updated weights for policy 1, policy_version 447280 (0.0008) [2023-12-26 18:37:07,018][105620] Updated weights for policy 1, policy_version 447290 (0.0008) [2023-12-26 18:37:07,077][105620] Updated weights for policy 1, policy_version 447300 (0.0008) [2023-12-26 18:37:07,339][105692] Updated weights for policy 0, policy_version 446928 (0.0010) [2023-12-26 18:37:07,398][105692] Updated weights for policy 0, policy_version 446938 (0.0010) [2023-12-26 18:37:07,463][105692] Updated weights for policy 0, policy_version 446948 (0.0010) [2023-12-26 18:37:07,853][105620] Updated weights for policy 1, policy_version 447310 (0.0008) [2023-12-26 18:37:07,916][105620] Updated weights for policy 1, policy_version 447320 (0.0008) [2023-12-26 18:37:07,988][105620] Updated weights for policy 1, policy_version 447330 (0.0008) [2023-12-26 18:37:08,179][105692] Updated weights for policy 0, policy_version 446958 (0.0010) [2023-12-26 18:37:08,237][105692] Updated weights for policy 0, policy_version 446968 (0.0010) [2023-12-26 18:37:08,301][105692] Updated weights for policy 0, policy_version 446978 (0.0010) [2023-12-26 18:37:08,755][105620] Updated weights for policy 1, policy_version 447340 (0.0008) [2023-12-26 18:37:08,820][105620] Updated weights for policy 1, policy_version 447350 (0.0008) [2023-12-26 18:37:08,879][105620] Updated weights for policy 1, policy_version 447360 (0.0009) [2023-12-26 18:37:08,957][105692] Updated weights for policy 0, policy_version 446988 (0.0010) [2023-12-26 18:37:09,014][105692] Updated weights for policy 0, policy_version 446998 (0.0010) [2023-12-26 18:37:09,072][105692] Updated weights for policy 0, policy_version 447008 (0.0010) [2023-12-26 18:37:09,640][105620] Updated weights for policy 1, policy_version 447370 (0.0008) [2023-12-26 18:37:09,706][105620] Updated weights for policy 1, policy_version 447380 (0.0011) [2023-12-26 18:37:09,771][105620] Updated weights for policy 1, policy_version 447390 (0.0011) [2023-12-26 18:37:09,841][105620] Updated weights for policy 1, policy_version 447400 (0.0010) [2023-12-26 18:37:09,856][105692] Updated weights for policy 0, policy_version 447018 (0.0010) [2023-12-26 18:37:09,915][105692] Updated weights for policy 0, policy_version 447028 (0.0008) [2023-12-26 18:37:09,978][105692] Updated weights for policy 0, policy_version 447038 (0.0010) [2023-12-26 18:37:10,041][105692] Updated weights for policy 0, policy_version 447048 (0.0011) [2023-12-26 18:37:10,506][105620] Updated weights for policy 1, policy_version 447410 (0.0007) [2023-12-26 18:37:10,568][105620] Updated weights for policy 1, policy_version 447420 (0.0008) [2023-12-26 18:37:10,636][105620] Updated weights for policy 1, policy_version 447430 (0.0008) [2023-12-26 18:37:10,809][105692] Updated weights for policy 0, policy_version 447058 (0.0011) [2023-12-26 18:37:10,876][105692] Updated weights for policy 0, policy_version 447068 (0.0011) [2023-12-26 18:37:10,942][105692] Updated weights for policy 0, policy_version 447078 (0.0011) [2023-12-26 18:37:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 229023744. Throughput: 0: 9688.6, 1: 9634.6. Samples: 229029788. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-26 18:37:11,063][104569] Avg episode reward: [(0, '9089.411'), (1, '8464.231')] [2023-12-26 18:37:11,318][105620] Updated weights for policy 1, policy_version 447440 (0.0007) [2023-12-26 18:37:11,390][105620] Updated weights for policy 1, policy_version 447450 (0.0007) [2023-12-26 18:37:11,450][105620] Updated weights for policy 1, policy_version 447460 (0.0007) [2023-12-26 18:37:11,693][105692] Updated weights for policy 0, policy_version 447088 (0.0011) [2023-12-26 18:37:11,761][105692] Updated weights for policy 0, policy_version 447098 (0.0007) [2023-12-26 18:37:11,824][105692] Updated weights for policy 0, policy_version 447108 (0.0009) [2023-12-26 18:37:12,051][105620] Updated weights for policy 1, policy_version 447470 (0.0009) [2023-12-26 18:37:12,106][105620] Updated weights for policy 1, policy_version 447480 (0.0009) [2023-12-26 18:37:12,168][105620] Updated weights for policy 1, policy_version 447490 (0.0006) [2023-12-26 18:37:12,490][105692] Updated weights for policy 0, policy_version 447118 (0.0011) [2023-12-26 18:37:12,556][105692] Updated weights for policy 0, policy_version 447128 (0.0011) [2023-12-26 18:37:12,616][105692] Updated weights for policy 0, policy_version 447138 (0.0006) [2023-12-26 18:37:12,908][105620] Updated weights for policy 1, policy_version 447500 (0.0007) [2023-12-26 18:37:12,976][105620] Updated weights for policy 1, policy_version 447510 (0.0008) [2023-12-26 18:37:13,039][105620] Updated weights for policy 1, policy_version 447520 (0.0009) [2023-12-26 18:37:13,263][105692] Updated weights for policy 0, policy_version 447148 (0.0007) [2023-12-26 18:37:13,311][105692] Updated weights for policy 0, policy_version 447158 (0.0009) [2023-12-26 18:37:13,357][105692] Updated weights for policy 0, policy_version 447168 (0.0005) [2023-12-26 18:37:13,684][105620] Updated weights for policy 1, policy_version 447530 (0.0007) [2023-12-26 18:37:13,740][105620] Updated weights for policy 1, policy_version 447540 (0.0005) [2023-12-26 18:37:13,805][105620] Updated weights for policy 1, policy_version 447550 (0.0007) [2023-12-26 18:37:13,875][105620] Updated weights for policy 1, policy_version 447560 (0.0010) [2023-12-26 18:37:13,905][105692] Updated weights for policy 0, policy_version 447178 (0.0005) [2023-12-26 18:37:13,956][105692] Updated weights for policy 0, policy_version 447188 (0.0008) [2023-12-26 18:37:14,007][105692] Updated weights for policy 0, policy_version 447198 (0.0010) [2023-12-26 18:37:14,052][105692] Updated weights for policy 0, policy_version 447208 (0.0010) [2023-12-26 18:37:14,581][105620] Updated weights for policy 1, policy_version 447570 (0.0008) [2023-12-26 18:37:14,637][105620] Updated weights for policy 1, policy_version 447580 (0.0007) [2023-12-26 18:37:14,661][105692] Updated weights for policy 0, policy_version 447218 (0.0008) [2023-12-26 18:37:14,691][105620] Updated weights for policy 1, policy_version 447590 (0.0007) [2023-12-26 18:37:14,717][105692] Updated weights for policy 0, policy_version 447228 (0.0008) [2023-12-26 18:37:14,771][105692] Updated weights for policy 0, policy_version 447238 (0.0008) [2023-12-26 18:37:15,482][105620] Updated weights for policy 1, policy_version 447600 (0.0007) [2023-12-26 18:37:15,511][105692] Updated weights for policy 0, policy_version 447248 (0.0010) [2023-12-26 18:37:15,542][105620] Updated weights for policy 1, policy_version 447610 (0.0008) [2023-12-26 18:37:15,560][105692] Updated weights for policy 0, policy_version 447258 (0.0006) [2023-12-26 18:37:15,604][105620] Updated weights for policy 1, policy_version 447620 (0.0008) [2023-12-26 18:37:15,615][105692] Updated weights for policy 0, policy_version 447268 (0.0005) [2023-12-26 18:37:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 229122048. Throughput: 0: 9785.7, 1: 9577.8. Samples: 229090440. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-26 18:37:16,063][104569] Avg episode reward: [(0, '9179.055'), (1, '8817.146')] [2023-12-26 18:37:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000447624_114606080.pth... [2023-12-26 18:37:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000447272_114515968.pth... [2023-12-26 18:37:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000446504_114319360.pth [2023-12-26 18:37:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000446120_114221056.pth [2023-12-26 18:37:16,321][105692] Updated weights for policy 0, policy_version 447278 (0.0007) [2023-12-26 18:37:16,347][105620] Updated weights for policy 1, policy_version 447630 (0.0008) [2023-12-26 18:37:16,376][105692] Updated weights for policy 0, policy_version 447288 (0.0007) [2023-12-26 18:37:16,405][105620] Updated weights for policy 1, policy_version 447640 (0.0007) [2023-12-26 18:37:16,428][105692] Updated weights for policy 0, policy_version 447298 (0.0006) [2023-12-26 18:37:16,453][105620] Updated weights for policy 1, policy_version 447650 (0.0007) [2023-12-26 18:37:17,118][105620] Updated weights for policy 1, policy_version 447660 (0.0006) [2023-12-26 18:37:17,183][105620] Updated weights for policy 1, policy_version 447670 (0.0008) [2023-12-26 18:37:17,197][105692] Updated weights for policy 0, policy_version 447308 (0.0006) [2023-12-26 18:37:17,250][105620] Updated weights for policy 1, policy_version 447680 (0.0008) [2023-12-26 18:37:17,253][105692] Updated weights for policy 0, policy_version 447318 (0.0007) [2023-12-26 18:37:17,310][105692] Updated weights for policy 0, policy_version 447328 (0.0007) [2023-12-26 18:37:17,888][105620] Updated weights for policy 1, policy_version 447690 (0.0007) [2023-12-26 18:37:17,935][105620] Updated weights for policy 1, policy_version 447700 (0.0009) [2023-12-26 18:37:17,981][105620] Updated weights for policy 1, policy_version 447710 (0.0009) [2023-12-26 18:37:18,091][105692] Updated weights for policy 0, policy_version 447338 (0.0009) [2023-12-26 18:37:18,145][105692] Updated weights for policy 0, policy_version 447348 (0.0009) [2023-12-26 18:37:18,199][105692] Updated weights for policy 0, policy_version 447358 (0.0010) [2023-12-26 18:37:18,259][105692] Updated weights for policy 0, policy_version 447368 (0.0010) [2023-12-26 18:37:18,612][105620] Updated weights for policy 1, policy_version 447721 (0.0009) [2023-12-26 18:37:18,685][105620] Updated weights for policy 1, policy_version 447731 (0.0005) [2023-12-26 18:37:18,750][105620] Updated weights for policy 1, policy_version 447741 (0.0006) [2023-12-26 18:37:18,812][105620] Updated weights for policy 1, policy_version 447751 (0.0009) [2023-12-26 18:37:19,111][105692] Updated weights for policy 0, policy_version 447378 (0.0009) [2023-12-26 18:37:19,176][105692] Updated weights for policy 0, policy_version 447389 (0.0010) [2023-12-26 18:37:19,234][105692] Updated weights for policy 0, policy_version 447400 (0.0010) [2023-12-26 18:37:19,432][105620] Updated weights for policy 1, policy_version 447761 (0.0008) [2023-12-26 18:37:19,495][105620] Updated weights for policy 1, policy_version 447771 (0.0008) [2023-12-26 18:37:19,558][105620] Updated weights for policy 1, policy_version 447781 (0.0008) [2023-12-26 18:37:20,041][105692] Updated weights for policy 0, policy_version 447410 (0.0009) [2023-12-26 18:37:20,103][105692] Updated weights for policy 0, policy_version 447420 (0.0009) [2023-12-26 18:37:20,171][105692] Updated weights for policy 0, policy_version 447430 (0.0008) [2023-12-26 18:37:20,269][105620] Updated weights for policy 1, policy_version 447791 (0.0010) [2023-12-26 18:37:20,327][105586] KL-divergence is very high: 107.7665 [2023-12-26 18:37:20,329][105620] Updated weights for policy 1, policy_version 447801 (0.0011) [2023-12-26 18:37:20,352][105586] KL-divergence is very high: 100.9877 [2023-12-26 18:37:20,359][105586] KL-divergence is very high: 112.3680 [2023-12-26 18:37:20,365][105586] KL-divergence is very high: 105.3581 [2023-12-26 18:37:20,378][105586] KL-divergence is very high: 112.7562 [2023-12-26 18:37:20,389][105620] Updated weights for policy 1, policy_version 447811 (0.0011) [2023-12-26 18:37:20,411][105586] KL-divergence is very high: 102.0304 [2023-12-26 18:37:20,904][105692] Updated weights for policy 0, policy_version 447440 (0.0007) [2023-12-26 18:37:20,953][105692] Updated weights for policy 0, policy_version 447450 (0.0008) [2023-12-26 18:37:21,010][105692] Updated weights for policy 0, policy_version 447460 (0.0008) [2023-12-26 18:37:21,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 229220352. Throughput: 0: 9812.7, 1: 9611.4. Samples: 229208420. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-26 18:37:21,063][104569] Avg episode reward: [(0, '9177.008'), (1, '7283.854')] [2023-12-26 18:37:21,148][105620] Updated weights for policy 1, policy_version 447821 (0.0008) [2023-12-26 18:37:21,213][105620] Updated weights for policy 1, policy_version 447831 (0.0006) [2023-12-26 18:37:21,278][105620] Updated weights for policy 1, policy_version 447841 (0.0007) [2023-12-26 18:37:21,801][105692] Updated weights for policy 0, policy_version 447470 (0.0008) [2023-12-26 18:37:21,867][105692] Updated weights for policy 0, policy_version 447480 (0.0009) [2023-12-26 18:37:21,932][105692] Updated weights for policy 0, policy_version 447490 (0.0008) [2023-12-26 18:37:21,996][105620] Updated weights for policy 1, policy_version 447851 (0.0009) [2023-12-26 18:37:22,048][105620] Updated weights for policy 1, policy_version 447861 (0.0009) [2023-12-26 18:37:22,106][105620] Updated weights for policy 1, policy_version 447871 (0.0010) [2023-12-26 18:37:22,689][105692] Updated weights for policy 0, policy_version 447500 (0.0009) [2023-12-26 18:37:22,741][105692] Updated weights for policy 0, policy_version 447510 (0.0008) [2023-12-26 18:37:22,787][105692] Updated weights for policy 0, policy_version 447520 (0.0007) [2023-12-26 18:37:22,901][105620] Updated weights for policy 1, policy_version 447881 (0.0008) [2023-12-26 18:37:22,963][105620] Updated weights for policy 1, policy_version 447891 (0.0006) [2023-12-26 18:37:23,011][105620] Updated weights for policy 1, policy_version 447901 (0.0005) [2023-12-26 18:37:23,066][105620] Updated weights for policy 1, policy_version 447911 (0.0005) [2023-12-26 18:37:23,441][105692] Updated weights for policy 0, policy_version 447530 (0.0006) [2023-12-26 18:37:23,502][105692] Updated weights for policy 0, policy_version 447540 (0.0006) [2023-12-26 18:37:23,557][105692] Updated weights for policy 0, policy_version 447550 (0.0006) [2023-12-26 18:37:23,615][105692] Updated weights for policy 0, policy_version 447560 (0.0006) [2023-12-26 18:37:23,631][105620] Updated weights for policy 1, policy_version 447921 (0.0005) [2023-12-26 18:37:23,675][105620] Updated weights for policy 1, policy_version 447931 (0.0005) [2023-12-26 18:37:23,721][105620] Updated weights for policy 1, policy_version 447941 (0.0005) [2023-12-26 18:37:24,184][105692] Updated weights for policy 0, policy_version 447570 (0.0008) [2023-12-26 18:37:24,247][105692] Updated weights for policy 0, policy_version 447580 (0.0007) [2023-12-26 18:37:24,298][105620] Updated weights for policy 1, policy_version 447951 (0.0008) [2023-12-26 18:37:24,306][105692] Updated weights for policy 0, policy_version 447590 (0.0007) [2023-12-26 18:37:24,355][105620] Updated weights for policy 1, policy_version 447961 (0.0009) [2023-12-26 18:37:24,418][105620] Updated weights for policy 1, policy_version 447971 (0.0007) [2023-12-26 18:37:24,960][105692] Updated weights for policy 0, policy_version 447600 (0.0005) [2023-12-26 18:37:25,004][105620] Updated weights for policy 1, policy_version 447981 (0.0008) [2023-12-26 18:37:25,026][105692] Updated weights for policy 0, policy_version 447610 (0.0007) [2023-12-26 18:37:25,056][105620] Updated weights for policy 1, policy_version 447991 (0.0010) [2023-12-26 18:37:25,078][105692] Updated weights for policy 0, policy_version 447620 (0.0006) [2023-12-26 18:37:25,101][105620] Updated weights for policy 1, policy_version 448001 (0.0010) [2023-12-26 18:37:25,681][105692] Updated weights for policy 0, policy_version 447630 (0.0009) [2023-12-26 18:37:25,742][105692] Updated weights for policy 0, policy_version 447640 (0.0010) [2023-12-26 18:37:25,789][105620] Updated weights for policy 1, policy_version 448011 (0.0009) [2023-12-26 18:37:25,804][105692] Updated weights for policy 0, policy_version 447650 (0.0010) [2023-12-26 18:37:25,840][105620] Updated weights for policy 1, policy_version 448021 (0.0005) [2023-12-26 18:37:25,890][105620] Updated weights for policy 1, policy_version 448031 (0.0006) [2023-12-26 18:37:26,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19660.9, 300 sec: 19494.2). Total num frames: 229326848. Throughput: 0: 9876.5, 1: 9693.0. Samples: 229330620. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-26 18:37:26,062][104569] Avg episode reward: [(0, '9088.477'), (1, '6945.150')] [2023-12-26 18:37:26,419][105692] Updated weights for policy 0, policy_version 447660 (0.0011) [2023-12-26 18:37:26,476][105692] Updated weights for policy 0, policy_version 447670 (0.0011) [2023-12-26 18:37:26,536][105692] Updated weights for policy 0, policy_version 447680 (0.0011) [2023-12-26 18:37:26,599][105620] Updated weights for policy 1, policy_version 448041 (0.0006) [2023-12-26 18:37:26,645][105586] KL-divergence is very high: 137.9153 [2023-12-26 18:37:26,661][105620] Updated weights for policy 1, policy_version 448051 (0.0010) [2023-12-26 18:37:26,688][105586] KL-divergence is very high: 266.6754 [2023-12-26 18:37:26,712][105620] Updated weights for policy 1, policy_version 448061 (0.0010) [2023-12-26 18:37:26,724][105586] KL-divergence is very high: 293.1566 [2023-12-26 18:37:26,766][105620] Updated weights for policy 1, policy_version 448071 (0.0010) [2023-12-26 18:37:27,137][105692] Updated weights for policy 0, policy_version 447690 (0.0010) [2023-12-26 18:37:27,190][105692] Updated weights for policy 0, policy_version 447700 (0.0005) [2023-12-26 18:37:27,247][105692] Updated weights for policy 0, policy_version 447710 (0.0006) [2023-12-26 18:37:27,297][105692] Updated weights for policy 0, policy_version 447720 (0.0008) [2023-12-26 18:37:27,436][105620] Updated weights for policy 1, policy_version 448081 (0.0010) [2023-12-26 18:37:27,507][105620] Updated weights for policy 1, policy_version 448091 (0.0006) [2023-12-26 18:37:27,579][105620] Updated weights for policy 1, policy_version 448101 (0.0005) [2023-12-26 18:37:27,961][105692] Updated weights for policy 0, policy_version 447730 (0.0005) [2023-12-26 18:37:28,007][105692] Updated weights for policy 0, policy_version 447740 (0.0005) [2023-12-26 18:37:28,060][105692] Updated weights for policy 0, policy_version 447750 (0.0005) [2023-12-26 18:37:28,119][105620] Updated weights for policy 1, policy_version 448111 (0.0009) [2023-12-26 18:37:28,180][105620] Updated weights for policy 1, policy_version 448121 (0.0010) [2023-12-26 18:37:28,225][105620] Updated weights for policy 1, policy_version 448131 (0.0010) [2023-12-26 18:37:28,747][105692] Updated weights for policy 0, policy_version 447760 (0.0005) [2023-12-26 18:37:28,819][105692] Updated weights for policy 0, policy_version 447770 (0.0008) [2023-12-26 18:37:28,850][105620] Updated weights for policy 1, policy_version 448141 (0.0006) [2023-12-26 18:37:28,876][105692] Updated weights for policy 0, policy_version 447780 (0.0005) [2023-12-26 18:37:28,917][105620] Updated weights for policy 1, policy_version 448151 (0.0008) [2023-12-26 18:37:28,973][105620] Updated weights for policy 1, policy_version 448161 (0.0010) [2023-12-26 18:37:29,547][105692] Updated weights for policy 0, policy_version 447790 (0.0010) [2023-12-26 18:37:29,586][105620] Updated weights for policy 1, policy_version 448171 (0.0009) [2023-12-26 18:37:29,596][105692] Updated weights for policy 0, policy_version 447800 (0.0011) [2023-12-26 18:37:29,648][105692] Updated weights for policy 0, policy_version 447810 (0.0008) [2023-12-26 18:37:29,648][105620] Updated weights for policy 1, policy_version 448181 (0.0011) [2023-12-26 18:37:29,711][105620] Updated weights for policy 1, policy_version 448191 (0.0010) [2023-12-26 18:37:30,370][105692] Updated weights for policy 0, policy_version 447820 (0.0008) [2023-12-26 18:37:30,399][105620] Updated weights for policy 1, policy_version 448201 (0.0010) [2023-12-26 18:37:30,436][105692] Updated weights for policy 0, policy_version 447830 (0.0011) [2023-12-26 18:37:30,455][105620] Updated weights for policy 1, policy_version 448211 (0.0007) [2023-12-26 18:37:30,501][105692] Updated weights for policy 0, policy_version 447840 (0.0011) [2023-12-26 18:37:30,513][105620] Updated weights for policy 1, policy_version 448221 (0.0008) [2023-12-26 18:37:30,568][105620] Updated weights for policy 1, policy_version 448231 (0.0010) [2023-12-26 18:37:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 229425152. Throughput: 0: 9946.1, 1: 9823.2. Samples: 229395100. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-26 18:37:31,062][104569] Avg episode reward: [(0, '9003.012'), (1, '8002.348')] [2023-12-26 18:37:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000448232_114761728.pth... [2023-12-26 18:37:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000447080_114466816.pth [2023-12-26 18:37:31,074][105692] Updated weights for policy 0, policy_version 447850 (0.0010) [2023-12-26 18:37:31,130][105692] Updated weights for policy 0, policy_version 447860 (0.0009) [2023-12-26 18:37:31,189][105692] Updated weights for policy 0, policy_version 447870 (0.0007) [2023-12-26 18:37:31,246][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000447880_114671616.pth... [2023-12-26 18:37:31,249][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000446664_114360320.pth [2023-12-26 18:37:31,249][105692] Updated weights for policy 0, policy_version 447880 (0.0007) [2023-12-26 18:37:31,280][105620] Updated weights for policy 1, policy_version 448241 (0.0011) [2023-12-26 18:37:31,342][105620] Updated weights for policy 1, policy_version 448251 (0.0008) [2023-12-26 18:37:31,409][105620] Updated weights for policy 1, policy_version 448261 (0.0009) [2023-12-26 18:37:31,965][105692] Updated weights for policy 0, policy_version 447890 (0.0009) [2023-12-26 18:37:32,019][105692] Updated weights for policy 0, policy_version 447900 (0.0009) [2023-12-26 18:37:32,073][105692] Updated weights for policy 0, policy_version 447910 (0.0008) [2023-12-26 18:37:32,155][105620] Updated weights for policy 1, policy_version 448271 (0.0007) [2023-12-26 18:37:32,207][105620] Updated weights for policy 1, policy_version 448281 (0.0006) [2023-12-26 18:37:32,259][105620] Updated weights for policy 1, policy_version 448291 (0.0010) [2023-12-26 18:37:32,778][105692] Updated weights for policy 0, policy_version 447920 (0.0005) [2023-12-26 18:37:32,831][105692] Updated weights for policy 0, policy_version 447930 (0.0005) [2023-12-26 18:37:32,882][105692] Updated weights for policy 0, policy_version 447940 (0.0005) [2023-12-26 18:37:32,917][105620] Updated weights for policy 1, policy_version 448301 (0.0009) [2023-12-26 18:37:32,970][105620] Updated weights for policy 1, policy_version 448311 (0.0006) [2023-12-26 18:37:33,016][105620] Updated weights for policy 1, policy_version 448321 (0.0005) [2023-12-26 18:37:33,542][105692] Updated weights for policy 0, policy_version 447950 (0.0008) [2023-12-26 18:37:33,590][105692] Updated weights for policy 0, policy_version 447960 (0.0010) [2023-12-26 18:37:33,638][105692] Updated weights for policy 0, policy_version 447970 (0.0010) [2023-12-26 18:37:33,697][105620] Updated weights for policy 1, policy_version 448331 (0.0006) [2023-12-26 18:37:33,755][105620] Updated weights for policy 1, policy_version 448341 (0.0008) [2023-12-26 18:37:33,813][105620] Updated weights for policy 1, policy_version 448351 (0.0009) [2023-12-26 18:37:34,394][105692] Updated weights for policy 0, policy_version 447980 (0.0010) [2023-12-26 18:37:34,459][105692] Updated weights for policy 0, policy_version 447990 (0.0008) [2023-12-26 18:37:34,518][105692] Updated weights for policy 0, policy_version 448000 (0.0008) [2023-12-26 18:37:34,636][105620] Updated weights for policy 1, policy_version 448361 (0.0009) [2023-12-26 18:37:34,691][105620] Updated weights for policy 1, policy_version 448371 (0.0009) [2023-12-26 18:37:34,746][105620] Updated weights for policy 1, policy_version 448381 (0.0009) [2023-12-26 18:37:34,800][105620] Updated weights for policy 1, policy_version 448391 (0.0009) [2023-12-26 18:37:35,214][105692] Updated weights for policy 0, policy_version 448010 (0.0008) [2023-12-26 18:37:35,260][105692] Updated weights for policy 0, policy_version 448020 (0.0006) [2023-12-26 18:37:35,309][105692] Updated weights for policy 0, policy_version 448030 (0.0007) [2023-12-26 18:37:35,367][105692] Updated weights for policy 0, policy_version 448040 (0.0009) [2023-12-26 18:37:35,598][105620] Updated weights for policy 1, policy_version 448401 (0.0008) [2023-12-26 18:37:35,650][105620] Updated weights for policy 1, policy_version 448411 (0.0009) [2023-12-26 18:37:35,708][105620] Updated weights for policy 1, policy_version 448422 (0.0009) [2023-12-26 18:37:36,038][105692] Updated weights for policy 0, policy_version 448050 (0.0007) [2023-12-26 18:37:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 229523456. Throughput: 0: 9881.5, 1: 9822.6. Samples: 229514156. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-26 18:37:36,062][104569] Avg episode reward: [(0, '8913.354'), (1, '8556.116')] [2023-12-26 18:37:36,093][105692] Updated weights for policy 0, policy_version 448060 (0.0009) [2023-12-26 18:37:36,153][105692] Updated weights for policy 0, policy_version 448070 (0.0007) [2023-12-26 18:37:36,611][105620] Updated weights for policy 1, policy_version 448432 (0.0009) [2023-12-26 18:37:36,671][105620] Updated weights for policy 1, policy_version 448442 (0.0010) [2023-12-26 18:37:36,723][105620] Updated weights for policy 1, policy_version 448453 (0.0010) [2023-12-26 18:37:36,737][105692] Updated weights for policy 0, policy_version 448080 (0.0005) [2023-12-26 18:37:36,783][105692] Updated weights for policy 0, policy_version 448090 (0.0005) [2023-12-26 18:37:36,834][105692] Updated weights for policy 0, policy_version 448100 (0.0005) [2023-12-26 18:37:37,388][105692] Updated weights for policy 0, policy_version 448110 (0.0005) [2023-12-26 18:37:37,443][105692] Updated weights for policy 0, policy_version 448120 (0.0007) [2023-12-26 18:37:37,489][105692] Updated weights for policy 0, policy_version 448130 (0.0010) [2023-12-26 18:37:37,576][105620] Updated weights for policy 1, policy_version 448463 (0.0009) [2023-12-26 18:37:37,640][105620] Updated weights for policy 1, policy_version 448473 (0.0009) [2023-12-26 18:37:37,696][105620] Updated weights for policy 1, policy_version 448483 (0.0008) [2023-12-26 18:37:38,186][105692] Updated weights for policy 0, policy_version 448140 (0.0010) [2023-12-26 18:37:38,243][105692] Updated weights for policy 0, policy_version 448150 (0.0010) [2023-12-26 18:37:38,309][105692] Updated weights for policy 0, policy_version 448160 (0.0010) [2023-12-26 18:37:38,495][105620] Updated weights for policy 1, policy_version 448493 (0.0008) [2023-12-26 18:37:38,550][105620] Updated weights for policy 1, policy_version 448503 (0.0010) [2023-12-26 18:37:38,606][105620] Updated weights for policy 1, policy_version 448513 (0.0010) [2023-12-26 18:37:39,051][105692] Updated weights for policy 0, policy_version 448170 (0.0008) [2023-12-26 18:37:39,101][105692] Updated weights for policy 0, policy_version 448180 (0.0009) [2023-12-26 18:37:39,156][105692] Updated weights for policy 0, policy_version 448190 (0.0005) [2023-12-26 18:37:39,215][105692] Updated weights for policy 0, policy_version 448200 (0.0006) [2023-12-26 18:37:39,268][105620] Updated weights for policy 1, policy_version 448523 (0.0009) [2023-12-26 18:37:39,335][105620] Updated weights for policy 1, policy_version 448533 (0.0006) [2023-12-26 18:37:39,404][105620] Updated weights for policy 1, policy_version 448543 (0.0008) [2023-12-26 18:37:39,955][105692] Updated weights for policy 0, policy_version 448210 (0.0010) [2023-12-26 18:37:39,988][105620] Updated weights for policy 1, policy_version 448553 (0.0009) [2023-12-26 18:37:40,012][105692] Updated weights for policy 0, policy_version 448220 (0.0011) [2023-12-26 18:37:40,046][105620] Updated weights for policy 1, policy_version 448563 (0.0008) [2023-12-26 18:37:40,076][105692] Updated weights for policy 0, policy_version 448230 (0.0011) [2023-12-26 18:37:40,106][105620] Updated weights for policy 1, policy_version 448573 (0.0006) [2023-12-26 18:37:40,156][105620] Updated weights for policy 1, policy_version 448583 (0.0006) [2023-12-26 18:37:40,818][105692] Updated weights for policy 0, policy_version 448240 (0.0006) [2023-12-26 18:37:40,875][105692] Updated weights for policy 0, policy_version 448250 (0.0010) [2023-12-26 18:37:40,896][105620] Updated weights for policy 1, policy_version 448593 (0.0010) [2023-12-26 18:37:40,930][105692] Updated weights for policy 0, policy_version 448260 (0.0011) [2023-12-26 18:37:40,952][105620] Updated weights for policy 1, policy_version 448603 (0.0008) [2023-12-26 18:37:41,000][105620] Updated weights for policy 1, policy_version 448613 (0.0008) [2023-12-26 18:37:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 229629952. Throughput: 0: 10014.0, 1: 9772.2. Samples: 229631804. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-26 18:37:41,063][104569] Avg episode reward: [(0, '9358.763'), (1, '8556.080')] [2023-12-26 18:37:41,629][105692] Updated weights for policy 0, policy_version 448270 (0.0009) [2023-12-26 18:37:41,693][105692] Updated weights for policy 0, policy_version 448280 (0.0010) [2023-12-26 18:37:41,757][105620] Updated weights for policy 1, policy_version 448623 (0.0008) [2023-12-26 18:37:41,760][105692] Updated weights for policy 0, policy_version 448290 (0.0009) [2023-12-26 18:37:41,823][105620] Updated weights for policy 1, policy_version 448633 (0.0005) [2023-12-26 18:37:41,883][105620] Updated weights for policy 1, policy_version 448643 (0.0005) [2023-12-26 18:37:42,525][105620] Updated weights for policy 1, policy_version 448653 (0.0006) [2023-12-26 18:37:42,529][105692] Updated weights for policy 0, policy_version 448300 (0.0009) [2023-12-26 18:37:42,575][105692] Updated weights for policy 0, policy_version 448310 (0.0006) [2023-12-26 18:37:42,584][105620] Updated weights for policy 1, policy_version 448663 (0.0011) [2023-12-26 18:37:42,631][105692] Updated weights for policy 0, policy_version 448320 (0.0006) [2023-12-26 18:37:42,644][105620] Updated weights for policy 1, policy_version 448673 (0.0011) [2023-12-26 18:37:43,360][105620] Updated weights for policy 1, policy_version 448683 (0.0010) [2023-12-26 18:37:43,415][105620] Updated weights for policy 1, policy_version 448693 (0.0008) [2023-12-26 18:37:43,425][105692] Updated weights for policy 0, policy_version 448330 (0.0006) [2023-12-26 18:37:43,472][105620] Updated weights for policy 1, policy_version 448703 (0.0006) [2023-12-26 18:37:43,486][105692] Updated weights for policy 0, policy_version 448340 (0.0007) [2023-12-26 18:37:43,545][105692] Updated weights for policy 0, policy_version 448350 (0.0007) [2023-12-26 18:37:43,593][105692] Updated weights for policy 0, policy_version 448360 (0.0008) [2023-12-26 18:37:44,123][105620] Updated weights for policy 1, policy_version 448713 (0.0006) [2023-12-26 18:37:44,172][105620] Updated weights for policy 1, policy_version 448723 (0.0006) [2023-12-26 18:37:44,228][105620] Updated weights for policy 1, policy_version 448733 (0.0005) [2023-12-26 18:37:44,292][105620] Updated weights for policy 1, policy_version 448743 (0.0008) [2023-12-26 18:37:44,428][105692] Updated weights for policy 0, policy_version 448370 (0.0009) [2023-12-26 18:37:44,486][105692] Updated weights for policy 0, policy_version 448380 (0.0008) [2023-12-26 18:37:44,550][105692] Updated weights for policy 0, policy_version 448390 (0.0009) [2023-12-26 18:37:44,914][105620] Updated weights for policy 1, policy_version 448753 (0.0008) [2023-12-26 18:37:44,968][105620] Updated weights for policy 1, policy_version 448763 (0.0009) [2023-12-26 18:37:45,033][105620] Updated weights for policy 1, policy_version 448773 (0.0008) [2023-12-26 18:37:45,313][105692] Updated weights for policy 0, policy_version 448400 (0.0009) [2023-12-26 18:37:45,373][105692] Updated weights for policy 0, policy_version 448410 (0.0009) [2023-12-26 18:37:45,436][105692] Updated weights for policy 0, policy_version 448420 (0.0009) [2023-12-26 18:37:45,732][105620] Updated weights for policy 1, policy_version 448783 (0.0005) [2023-12-26 18:37:45,792][105620] Updated weights for policy 1, policy_version 448793 (0.0005) [2023-12-26 18:37:45,857][105620] Updated weights for policy 1, policy_version 448803 (0.0008) [2023-12-26 18:37:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.9, 300 sec: 19521.9). Total num frames: 229720064. Throughput: 0: 10047.9, 1: 9742.0. Samples: 229689220. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-26 18:37:46,063][104569] Avg episode reward: [(0, '9358.898'), (1, '8820.203')] [2023-12-26 18:37:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000448808_114909184.pth... [2023-12-26 18:37:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000448424_114810880.pth... [2023-12-26 18:37:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000447272_114515968.pth [2023-12-26 18:37:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000447624_114606080.pth [2023-12-26 18:37:46,219][105692] Updated weights for policy 0, policy_version 448430 (0.0009) [2023-12-26 18:37:46,279][105692] Updated weights for policy 0, policy_version 448440 (0.0009) [2023-12-26 18:37:46,342][105692] Updated weights for policy 0, policy_version 448450 (0.0009) [2023-12-26 18:37:46,522][105620] Updated weights for policy 1, policy_version 448813 (0.0009) [2023-12-26 18:37:46,580][105620] Updated weights for policy 1, policy_version 448823 (0.0010) [2023-12-26 18:37:46,639][105620] Updated weights for policy 1, policy_version 448833 (0.0008) [2023-12-26 18:37:47,121][105692] Updated weights for policy 0, policy_version 448460 (0.0009) [2023-12-26 18:37:47,173][105692] Updated weights for policy 0, policy_version 448470 (0.0009) [2023-12-26 18:37:47,225][105692] Updated weights for policy 0, policy_version 448480 (0.0009) [2023-12-26 18:37:47,336][105620] Updated weights for policy 1, policy_version 448843 (0.0009) [2023-12-26 18:37:47,396][105620] Updated weights for policy 1, policy_version 448853 (0.0008) [2023-12-26 18:37:47,456][105620] Updated weights for policy 1, policy_version 448863 (0.0009) [2023-12-26 18:37:48,011][105692] Updated weights for policy 0, policy_version 448490 (0.0009) [2023-12-26 18:37:48,057][105692] Updated weights for policy 0, policy_version 448500 (0.0006) [2023-12-26 18:37:48,116][105692] Updated weights for policy 0, policy_version 448510 (0.0009) [2023-12-26 18:37:48,168][105692] Updated weights for policy 0, policy_version 448520 (0.0007) [2023-12-26 18:37:48,185][105620] Updated weights for policy 1, policy_version 448873 (0.0009) [2023-12-26 18:37:48,242][105620] Updated weights for policy 1, policy_version 448883 (0.0009) [2023-12-26 18:37:48,293][105620] Updated weights for policy 1, policy_version 448893 (0.0006) [2023-12-26 18:37:48,356][105620] Updated weights for policy 1, policy_version 448903 (0.0007) [2023-12-26 18:37:48,924][105692] Updated weights for policy 0, policy_version 448530 (0.0010) [2023-12-26 18:37:48,954][105620] Updated weights for policy 1, policy_version 448913 (0.0006) [2023-12-26 18:37:48,982][105692] Updated weights for policy 0, policy_version 448540 (0.0010) [2023-12-26 18:37:49,010][105620] Updated weights for policy 1, policy_version 448923 (0.0009) [2023-12-26 18:37:49,036][105692] Updated weights for policy 0, policy_version 448550 (0.0009) [2023-12-26 18:37:49,070][105620] Updated weights for policy 1, policy_version 448933 (0.0008) [2023-12-26 18:37:49,734][105692] Updated weights for policy 0, policy_version 448560 (0.0009) [2023-12-26 18:37:49,799][105692] Updated weights for policy 0, policy_version 448570 (0.0010) [2023-12-26 18:37:49,824][105620] Updated weights for policy 1, policy_version 448943 (0.0007) [2023-12-26 18:37:49,865][105692] Updated weights for policy 0, policy_version 448580 (0.0009) [2023-12-26 18:37:49,885][105620] Updated weights for policy 1, policy_version 448953 (0.0008) [2023-12-26 18:37:49,945][105620] Updated weights for policy 1, policy_version 448963 (0.0008) [2023-12-26 18:37:50,582][105692] Updated weights for policy 0, policy_version 448590 (0.0008) [2023-12-26 18:37:50,646][105692] Updated weights for policy 0, policy_version 448600 (0.0009) [2023-12-26 18:37:50,679][105620] Updated weights for policy 1, policy_version 448973 (0.0008) [2023-12-26 18:37:50,714][105692] Updated weights for policy 0, policy_version 448610 (0.0008) [2023-12-26 18:37:50,743][105620] Updated weights for policy 1, policy_version 448983 (0.0008) [2023-12-26 18:37:50,809][105620] Updated weights for policy 1, policy_version 448993 (0.0011) [2023-12-26 18:37:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 229818368. Throughput: 0: 9890.2, 1: 9871.0. Samples: 229805024. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-26 18:37:51,063][104569] Avg episode reward: [(0, '9358.894'), (1, '9174.222')] [2023-12-26 18:37:51,477][105692] Updated weights for policy 0, policy_version 448620 (0.0007) [2023-12-26 18:37:51,533][105692] Updated weights for policy 0, policy_version 448630 (0.0008) [2023-12-26 18:37:51,540][105620] Updated weights for policy 1, policy_version 449003 (0.0010) [2023-12-26 18:37:51,581][105692] Updated weights for policy 0, policy_version 448640 (0.0008) [2023-12-26 18:37:51,588][105620] Updated weights for policy 1, policy_version 449013 (0.0007) [2023-12-26 18:37:51,644][105620] Updated weights for policy 1, policy_version 449023 (0.0008) [2023-12-26 18:37:52,377][105692] Updated weights for policy 0, policy_version 448650 (0.0008) [2023-12-26 18:37:52,392][105620] Updated weights for policy 1, policy_version 449033 (0.0009) [2023-12-26 18:37:52,439][105692] Updated weights for policy 0, policy_version 448660 (0.0009) [2023-12-26 18:37:52,453][105620] Updated weights for policy 1, policy_version 449043 (0.0006) [2023-12-26 18:37:52,496][105692] Updated weights for policy 0, policy_version 448670 (0.0006) [2023-12-26 18:37:52,506][105620] Updated weights for policy 1, policy_version 449053 (0.0006) [2023-12-26 18:37:52,550][105692] Updated weights for policy 0, policy_version 448680 (0.0009) [2023-12-26 18:37:52,563][105620] Updated weights for policy 1, policy_version 449063 (0.0007) [2023-12-26 18:37:53,237][105620] Updated weights for policy 1, policy_version 449073 (0.0006) [2023-12-26 18:37:53,298][105620] Updated weights for policy 1, policy_version 449083 (0.0006) [2023-12-26 18:37:53,315][105692] Updated weights for policy 0, policy_version 448690 (0.0006) [2023-12-26 18:37:53,356][105620] Updated weights for policy 1, policy_version 449093 (0.0008) [2023-12-26 18:37:53,379][105692] Updated weights for policy 0, policy_version 448700 (0.0005) [2023-12-26 18:37:53,433][105692] Updated weights for policy 0, policy_version 448710 (0.0007) [2023-12-26 18:37:53,943][105620] Updated weights for policy 1, policy_version 449103 (0.0010) [2023-12-26 18:37:54,008][105620] Updated weights for policy 1, policy_version 449113 (0.0010) [2023-12-26 18:37:54,057][105620] Updated weights for policy 1, policy_version 449123 (0.0007) [2023-12-26 18:37:54,109][105692] Updated weights for policy 0, policy_version 448720 (0.0008) [2023-12-26 18:37:54,167][105692] Updated weights for policy 0, policy_version 448730 (0.0010) [2023-12-26 18:37:54,224][105692] Updated weights for policy 0, policy_version 448740 (0.0010) [2023-12-26 18:37:54,764][105620] Updated weights for policy 1, policy_version 449133 (0.0008) [2023-12-26 18:37:54,815][105620] Updated weights for policy 1, policy_version 449143 (0.0007) [2023-12-26 18:37:54,878][105620] Updated weights for policy 1, policy_version 449153 (0.0005) [2023-12-26 18:37:54,931][105692] Updated weights for policy 0, policy_version 448750 (0.0009) [2023-12-26 18:37:54,991][105692] Updated weights for policy 0, policy_version 448760 (0.0011) [2023-12-26 18:37:55,050][105692] Updated weights for policy 0, policy_version 448770 (0.0011) [2023-12-26 18:37:55,526][105620] Updated weights for policy 1, policy_version 449163 (0.0005) [2023-12-26 18:37:55,577][105620] Updated weights for policy 1, policy_version 449173 (0.0005) [2023-12-26 18:37:55,624][105620] Updated weights for policy 1, policy_version 449183 (0.0007) [2023-12-26 18:37:55,769][105692] Updated weights for policy 0, policy_version 448780 (0.0010) [2023-12-26 18:37:55,818][105692] Updated weights for policy 0, policy_version 448790 (0.0008) [2023-12-26 18:37:55,864][105692] Updated weights for policy 0, policy_version 448800 (0.0005) [2023-12-26 18:37:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19661.0, 300 sec: 19549.7). Total num frames: 229916672. Throughput: 0: 9816.1, 1: 10022.5. Samples: 229922532. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-26 18:37:56,063][104569] Avg episode reward: [(0, '9270.789'), (1, '9084.045')] [2023-12-26 18:37:56,372][105620] Updated weights for policy 1, policy_version 449193 (0.0007) [2023-12-26 18:37:56,406][105692] Updated weights for policy 0, policy_version 448810 (0.0006) [2023-12-26 18:37:56,428][105620] Updated weights for policy 1, policy_version 449203 (0.0008) [2023-12-26 18:37:56,463][105692] Updated weights for policy 0, policy_version 448820 (0.0010) [2023-12-26 18:37:56,492][105620] Updated weights for policy 1, policy_version 449213 (0.0009) [2023-12-26 18:37:56,521][105692] Updated weights for policy 0, policy_version 448830 (0.0010) [2023-12-26 18:37:56,554][105620] Updated weights for policy 1, policy_version 449223 (0.0007) [2023-12-26 18:37:56,571][105692] Updated weights for policy 0, policy_version 448840 (0.0010) [2023-12-26 18:37:57,175][105692] Updated weights for policy 0, policy_version 448850 (0.0005) [2023-12-26 18:37:57,228][105692] Updated weights for policy 0, policy_version 448860 (0.0006) [2023-12-26 18:37:57,290][105692] Updated weights for policy 0, policy_version 448870 (0.0005) [2023-12-26 18:37:57,344][105620] Updated weights for policy 1, policy_version 449233 (0.0009) [2023-12-26 18:37:57,408][105620] Updated weights for policy 1, policy_version 449243 (0.0008) [2023-12-26 18:37:57,467][105620] Updated weights for policy 1, policy_version 449253 (0.0008) [2023-12-26 18:37:57,953][105692] Updated weights for policy 0, policy_version 448880 (0.0005) [2023-12-26 18:37:58,021][105692] Updated weights for policy 0, policy_version 448890 (0.0005) [2023-12-26 18:37:58,072][105692] Updated weights for policy 0, policy_version 448900 (0.0006) [2023-12-26 18:37:58,285][105620] Updated weights for policy 1, policy_version 449263 (0.0008) [2023-12-26 18:37:58,350][105620] Updated weights for policy 1, policy_version 449273 (0.0009) [2023-12-26 18:37:58,415][105620] Updated weights for policy 1, policy_version 449283 (0.0008) [2023-12-26 18:37:58,851][105692] Updated weights for policy 0, policy_version 448910 (0.0008) [2023-12-26 18:37:58,916][105692] Updated weights for policy 0, policy_version 448920 (0.0008) [2023-12-26 18:37:58,973][105692] Updated weights for policy 0, policy_version 448930 (0.0009) [2023-12-26 18:37:59,184][105620] Updated weights for policy 1, policy_version 449293 (0.0009) [2023-12-26 18:37:59,249][105620] Updated weights for policy 1, policy_version 449303 (0.0011) [2023-12-26 18:37:59,317][105620] Updated weights for policy 1, policy_version 449313 (0.0008) [2023-12-26 18:37:59,814][105692] Updated weights for policy 0, policy_version 448940 (0.0008) [2023-12-26 18:37:59,874][105692] Updated weights for policy 0, policy_version 448950 (0.0008) [2023-12-26 18:37:59,934][105692] Updated weights for policy 0, policy_version 448960 (0.0008) [2023-12-26 18:38:00,048][105620] Updated weights for policy 1, policy_version 449323 (0.0009) [2023-12-26 18:38:00,109][105620] Updated weights for policy 1, policy_version 449333 (0.0009) [2023-12-26 18:38:00,171][105620] Updated weights for policy 1, policy_version 449343 (0.0010) [2023-12-26 18:38:00,652][105692] Updated weights for policy 0, policy_version 448970 (0.0008) [2023-12-26 18:38:00,705][105692] Updated weights for policy 0, policy_version 448981 (0.0010) [2023-12-26 18:38:00,760][105692] Updated weights for policy 0, policy_version 448991 (0.0007) [2023-12-26 18:38:00,902][105620] Updated weights for policy 1, policy_version 449353 (0.0009) [2023-12-26 18:38:00,969][105620] Updated weights for policy 1, policy_version 449363 (0.0005) [2023-12-26 18:38:01,037][105620] Updated weights for policy 1, policy_version 449373 (0.0006) [2023-12-26 18:38:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 230006784. Throughput: 0: 9876.3, 1: 9906.3. Samples: 229980656. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-26 18:38:01,063][104569] Avg episode reward: [(0, '9179.845'), (1, '9086.791')] [2023-12-26 18:38:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000449000_114958336.pth... [2023-12-26 18:38:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000447880_114671616.pth [2023-12-26 18:38:01,102][105620] Updated weights for policy 1, policy_version 449383 (0.0008) [2023-12-26 18:38:01,106][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000449384_115056640.pth... [2023-12-26 18:38:01,110][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000448232_114761728.pth [2023-12-26 18:38:01,469][105692] Updated weights for policy 0, policy_version 449001 (0.0008) [2023-12-26 18:38:01,539][105692] Updated weights for policy 0, policy_version 449011 (0.0008) [2023-12-26 18:38:01,597][105692] Updated weights for policy 0, policy_version 449021 (0.0010) [2023-12-26 18:38:01,662][105692] Updated weights for policy 0, policy_version 449031 (0.0009) [2023-12-26 18:38:01,760][105620] Updated weights for policy 1, policy_version 449393 (0.0009) [2023-12-26 18:38:01,819][105620] Updated weights for policy 1, policy_version 449403 (0.0008) [2023-12-26 18:38:01,877][105620] Updated weights for policy 1, policy_version 449413 (0.0008) [2023-12-26 18:38:02,375][105692] Updated weights for policy 0, policy_version 449041 (0.0011) [2023-12-26 18:38:02,444][105692] Updated weights for policy 0, policy_version 449051 (0.0011) [2023-12-26 18:38:02,502][105692] Updated weights for policy 0, policy_version 449061 (0.0010) [2023-12-26 18:38:02,550][105620] Updated weights for policy 1, policy_version 449423 (0.0006) [2023-12-26 18:38:02,605][105620] Updated weights for policy 1, policy_version 449433 (0.0005) [2023-12-26 18:38:02,664][105620] Updated weights for policy 1, policy_version 449443 (0.0009) [2023-12-26 18:38:03,165][105692] Updated weights for policy 0, policy_version 449071 (0.0010) [2023-12-26 18:38:03,219][105692] Updated weights for policy 0, policy_version 449081 (0.0010) [2023-12-26 18:38:03,270][105692] Updated weights for policy 0, policy_version 449091 (0.0010) [2023-12-26 18:38:03,303][105620] Updated weights for policy 1, policy_version 449453 (0.0008) [2023-12-26 18:38:03,359][105620] Updated weights for policy 1, policy_version 449463 (0.0005) [2023-12-26 18:38:03,415][105620] Updated weights for policy 1, policy_version 449473 (0.0007) [2023-12-26 18:38:03,989][105692] Updated weights for policy 0, policy_version 449101 (0.0011) [2023-12-26 18:38:04,051][105692] Updated weights for policy 0, policy_version 449111 (0.0011) [2023-12-26 18:38:04,111][105692] Updated weights for policy 0, policy_version 449121 (0.0011) [2023-12-26 18:38:04,118][105620] Updated weights for policy 1, policy_version 449483 (0.0009) [2023-12-26 18:38:04,186][105620] Updated weights for policy 1, policy_version 449493 (0.0008) [2023-12-26 18:38:04,251][105620] Updated weights for policy 1, policy_version 449503 (0.0008) [2023-12-26 18:38:04,749][105692] Updated weights for policy 0, policy_version 449131 (0.0009) [2023-12-26 18:38:04,797][105692] Updated weights for policy 0, policy_version 449141 (0.0008) [2023-12-26 18:38:04,841][105692] Updated weights for policy 0, policy_version 449151 (0.0010) [2023-12-26 18:38:04,860][105620] Updated weights for policy 1, policy_version 449513 (0.0006) [2023-12-26 18:38:04,914][105620] Updated weights for policy 1, policy_version 449523 (0.0008) [2023-12-26 18:38:04,964][105620] Updated weights for policy 1, policy_version 449534 (0.0009) [2023-12-26 18:38:05,023][105620] Updated weights for policy 1, policy_version 449544 (0.0007) [2023-12-26 18:38:05,550][105692] Updated weights for policy 0, policy_version 449161 (0.0006) [2023-12-26 18:38:05,598][105692] Updated weights for policy 0, policy_version 449171 (0.0010) [2023-12-26 18:38:05,631][105620] Updated weights for policy 1, policy_version 449554 (0.0010) [2023-12-26 18:38:05,646][105692] Updated weights for policy 0, policy_version 449181 (0.0010) [2023-12-26 18:38:05,682][105620] Updated weights for policy 1, policy_version 449564 (0.0010) [2023-12-26 18:38:05,687][105692] Updated weights for policy 0, policy_version 449191 (0.0005) [2023-12-26 18:38:05,738][105620] Updated weights for policy 1, policy_version 449574 (0.0010) [2023-12-26 18:38:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 230113280. Throughput: 0: 9865.2, 1: 9916.2. Samples: 230098584. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-26 18:38:06,063][104569] Avg episode reward: [(0, '9267.896'), (1, '9264.287')] [2023-12-26 18:38:06,482][105692] Updated weights for policy 0, policy_version 449201 (0.0010) [2023-12-26 18:38:06,520][105620] Updated weights for policy 1, policy_version 449584 (0.0006) [2023-12-26 18:38:06,534][105692] Updated weights for policy 0, policy_version 449211 (0.0011) [2023-12-26 18:38:06,577][105620] Updated weights for policy 1, policy_version 449594 (0.0005) [2023-12-26 18:38:06,590][105692] Updated weights for policy 0, policy_version 449221 (0.0011) [2023-12-26 18:38:06,633][105620] Updated weights for policy 1, policy_version 449604 (0.0007) [2023-12-26 18:38:07,343][105692] Updated weights for policy 0, policy_version 449231 (0.0011) [2023-12-26 18:38:07,395][105620] Updated weights for policy 1, policy_version 449614 (0.0006) [2023-12-26 18:38:07,404][105692] Updated weights for policy 0, policy_version 449241 (0.0011) [2023-12-26 18:38:07,449][105620] Updated weights for policy 1, policy_version 449624 (0.0006) [2023-12-26 18:38:07,455][105692] Updated weights for policy 0, policy_version 449251 (0.0010) [2023-12-26 18:38:07,506][105620] Updated weights for policy 1, policy_version 449634 (0.0007) [2023-12-26 18:38:08,170][105620] Updated weights for policy 1, policy_version 449644 (0.0008) [2023-12-26 18:38:08,178][105692] Updated weights for policy 0, policy_version 449261 (0.0008) [2023-12-26 18:38:08,226][105620] Updated weights for policy 1, policy_version 449654 (0.0006) [2023-12-26 18:38:08,241][105692] Updated weights for policy 0, policy_version 449271 (0.0007) [2023-12-26 18:38:08,282][105620] Updated weights for policy 1, policy_version 449664 (0.0007) [2023-12-26 18:38:08,291][105692] Updated weights for policy 0, policy_version 449281 (0.0007) [2023-12-26 18:38:08,951][105692] Updated weights for policy 0, policy_version 449291 (0.0009) [2023-12-26 18:38:09,001][105692] Updated weights for policy 0, policy_version 449301 (0.0006) [2023-12-26 18:38:09,056][105692] Updated weights for policy 0, policy_version 449311 (0.0005) [2023-12-26 18:38:09,095][105620] Updated weights for policy 1, policy_version 449674 (0.0009) [2023-12-26 18:38:09,151][105620] Updated weights for policy 1, policy_version 449684 (0.0010) [2023-12-26 18:38:09,204][105620] Updated weights for policy 1, policy_version 449695 (0.0010) [2023-12-26 18:38:09,771][105692] Updated weights for policy 0, policy_version 449321 (0.0005) [2023-12-26 18:38:09,833][105692] Updated weights for policy 0, policy_version 449331 (0.0007) [2023-12-26 18:38:09,898][105692] Updated weights for policy 0, policy_version 449341 (0.0006) [2023-12-26 18:38:09,963][105692] Updated weights for policy 0, policy_version 449351 (0.0008) [2023-12-26 18:38:10,106][105620] Updated weights for policy 1, policy_version 449705 (0.0009) [2023-12-26 18:38:10,169][105620] Updated weights for policy 1, policy_version 449715 (0.0008) [2023-12-26 18:38:10,223][105620] Updated weights for policy 1, policy_version 449725 (0.0009) [2023-12-26 18:38:10,272][105620] Updated weights for policy 1, policy_version 449735 (0.0009) [2023-12-26 18:38:10,640][105692] Updated weights for policy 0, policy_version 449361 (0.0009) [2023-12-26 18:38:10,694][105692] Updated weights for policy 0, policy_version 449371 (0.0008) [2023-12-26 18:38:10,753][105692] Updated weights for policy 0, policy_version 449381 (0.0009) [2023-12-26 18:38:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 230203392. Throughput: 0: 9847.7, 1: 9789.5. Samples: 230214296. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-26 18:38:11,063][104569] Avg episode reward: [(0, '9358.896'), (1, '9038.169')] [2023-12-26 18:38:11,064][105620] Updated weights for policy 1, policy_version 449745 (0.0009) [2023-12-26 18:38:11,134][105620] Updated weights for policy 1, policy_version 449755 (0.0009) [2023-12-26 18:38:11,190][105620] Updated weights for policy 1, policy_version 449765 (0.0008) [2023-12-26 18:38:11,494][105692] Updated weights for policy 0, policy_version 449391 (0.0010) [2023-12-26 18:38:11,557][105692] Updated weights for policy 0, policy_version 449401 (0.0011) [2023-12-26 18:38:11,620][105692] Updated weights for policy 0, policy_version 449411 (0.0010) [2023-12-26 18:38:11,989][105620] Updated weights for policy 1, policy_version 449775 (0.0008) [2023-12-26 18:38:12,045][105620] Updated weights for policy 1, policy_version 449785 (0.0008) [2023-12-26 18:38:12,112][105620] Updated weights for policy 1, policy_version 449795 (0.0008) [2023-12-26 18:38:12,315][105692] Updated weights for policy 0, policy_version 449421 (0.0010) [2023-12-26 18:38:12,380][105692] Updated weights for policy 0, policy_version 449431 (0.0010) [2023-12-26 18:38:12,441][105692] Updated weights for policy 0, policy_version 449441 (0.0010) [2023-12-26 18:38:12,853][105620] Updated weights for policy 1, policy_version 449805 (0.0007) [2023-12-26 18:38:12,921][105620] Updated weights for policy 1, policy_version 449815 (0.0009) [2023-12-26 18:38:12,993][105620] Updated weights for policy 1, policy_version 449825 (0.0010) [2023-12-26 18:38:13,183][105692] Updated weights for policy 0, policy_version 449451 (0.0009) [2023-12-26 18:38:13,245][105692] Updated weights for policy 0, policy_version 449461 (0.0009) [2023-12-26 18:38:13,299][105692] Updated weights for policy 0, policy_version 449471 (0.0009) [2023-12-26 18:38:13,716][105620] Updated weights for policy 1, policy_version 449835 (0.0009) [2023-12-26 18:38:13,745][105586] KL-divergence is very high: 104.9945 [2023-12-26 18:38:13,770][105620] Updated weights for policy 1, policy_version 449845 (0.0008) [2023-12-26 18:38:13,820][105620] Updated weights for policy 1, policy_version 449855 (0.0008) [2023-12-26 18:38:14,029][105692] Updated weights for policy 0, policy_version 449481 (0.0008) [2023-12-26 18:38:14,089][105692] Updated weights for policy 0, policy_version 449491 (0.0008) [2023-12-26 18:38:14,149][105692] Updated weights for policy 0, policy_version 449501 (0.0008) [2023-12-26 18:38:14,213][105692] Updated weights for policy 0, policy_version 449511 (0.0009) [2023-12-26 18:38:14,482][105620] Updated weights for policy 1, policy_version 449865 (0.0010) [2023-12-26 18:38:14,534][105620] Updated weights for policy 1, policy_version 449875 (0.0005) [2023-12-26 18:38:14,585][105620] Updated weights for policy 1, policy_version 449885 (0.0005) [2023-12-26 18:38:14,647][105620] Updated weights for policy 1, policy_version 449895 (0.0006) [2023-12-26 18:38:14,905][105692] Updated weights for policy 0, policy_version 449521 (0.0011) [2023-12-26 18:38:14,961][105692] Updated weights for policy 0, policy_version 449531 (0.0010) [2023-12-26 18:38:15,028][105692] Updated weights for policy 0, policy_version 449541 (0.0011) [2023-12-26 18:38:15,192][105620] Updated weights for policy 1, policy_version 449905 (0.0006) [2023-12-26 18:38:15,259][105620] Updated weights for policy 1, policy_version 449915 (0.0007) [2023-12-26 18:38:15,324][105620] Updated weights for policy 1, policy_version 449925 (0.0010) [2023-12-26 18:38:15,775][105692] Updated weights for policy 0, policy_version 449551 (0.0010) [2023-12-26 18:38:15,833][105692] Updated weights for policy 0, policy_version 449561 (0.0010) [2023-12-26 18:38:15,887][105620] Updated weights for policy 1, policy_version 449935 (0.0008) [2023-12-26 18:38:15,893][105692] Updated weights for policy 0, policy_version 449571 (0.0010) [2023-12-26 18:38:15,942][105620] Updated weights for policy 1, policy_version 449945 (0.0005) [2023-12-26 18:38:16,000][105620] Updated weights for policy 1, policy_version 449955 (0.0006) [2023-12-26 18:38:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 230309888. Throughput: 0: 9747.9, 1: 9677.4. Samples: 230269240. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-26 18:38:16,063][104569] Avg episode reward: [(0, '9358.971'), (1, '5778.787')] [2023-12-26 18:38:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000449576_115105792.pth... [2023-12-26 18:38:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000449960_115204096.pth... [2023-12-26 18:38:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000448424_114810880.pth [2023-12-26 18:38:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000448808_114909184.pth [2023-12-26 18:38:16,559][105620] Updated weights for policy 1, policy_version 449965 (0.0008) [2023-12-26 18:38:16,607][105620] Updated weights for policy 1, policy_version 449975 (0.0007) [2023-12-26 18:38:16,657][105620] Updated weights for policy 1, policy_version 449985 (0.0006) [2023-12-26 18:38:16,716][105692] Updated weights for policy 0, policy_version 449581 (0.0008) [2023-12-26 18:38:16,773][105692] Updated weights for policy 0, policy_version 449591 (0.0010) [2023-12-26 18:38:16,837][105692] Updated weights for policy 0, policy_version 449601 (0.0009) [2023-12-26 18:38:17,300][105620] Updated weights for policy 1, policy_version 449995 (0.0005) [2023-12-26 18:38:17,366][105620] Updated weights for policy 1, policy_version 450005 (0.0007) [2023-12-26 18:38:17,420][105620] Updated weights for policy 1, policy_version 450015 (0.0006) [2023-12-26 18:38:17,454][105692] Updated weights for policy 0, policy_version 449611 (0.0008) [2023-12-26 18:38:17,500][105692] Updated weights for policy 0, policy_version 449621 (0.0005) [2023-12-26 18:38:17,550][105692] Updated weights for policy 0, policy_version 449631 (0.0005) [2023-12-26 18:38:18,018][105620] Updated weights for policy 1, policy_version 450025 (0.0005) [2023-12-26 18:38:18,066][105620] Updated weights for policy 1, policy_version 450035 (0.0008) [2023-12-26 18:38:18,114][105620] Updated weights for policy 1, policy_version 450045 (0.0008) [2023-12-26 18:38:18,162][105620] Updated weights for policy 1, policy_version 450055 (0.0008) [2023-12-26 18:38:18,176][105692] Updated weights for policy 0, policy_version 449641 (0.0006) [2023-12-26 18:38:18,234][105692] Updated weights for policy 0, policy_version 449651 (0.0011) [2023-12-26 18:38:18,299][105692] Updated weights for policy 0, policy_version 449661 (0.0009) [2023-12-26 18:38:18,369][105692] Updated weights for policy 0, policy_version 449671 (0.0008) [2023-12-26 18:38:18,810][105620] Updated weights for policy 1, policy_version 450065 (0.0006) [2023-12-26 18:38:18,862][105620] Updated weights for policy 1, policy_version 450075 (0.0008) [2023-12-26 18:38:18,913][105620] Updated weights for policy 1, policy_version 450085 (0.0009) [2023-12-26 18:38:19,065][105692] Updated weights for policy 0, policy_version 449681 (0.0006) [2023-12-26 18:38:19,121][105692] Updated weights for policy 0, policy_version 449691 (0.0008) [2023-12-26 18:38:19,173][105692] Updated weights for policy 0, policy_version 449701 (0.0009) [2023-12-26 18:38:19,659][105620] Updated weights for policy 1, policy_version 450095 (0.0010) [2023-12-26 18:38:19,719][105620] Updated weights for policy 1, policy_version 450105 (0.0011) [2023-12-26 18:38:19,771][105620] Updated weights for policy 1, policy_version 450115 (0.0011) [2023-12-26 18:38:19,896][105692] Updated weights for policy 0, policy_version 449711 (0.0007) [2023-12-26 18:38:19,962][105692] Updated weights for policy 0, policy_version 449721 (0.0008) [2023-12-26 18:38:20,023][105692] Updated weights for policy 0, policy_version 449731 (0.0008) [2023-12-26 18:38:20,526][105620] Updated weights for policy 1, policy_version 450125 (0.0011) [2023-12-26 18:38:20,589][105620] Updated weights for policy 1, policy_version 450135 (0.0010) [2023-12-26 18:38:20,659][105620] Updated weights for policy 1, policy_version 450145 (0.0008) [2023-12-26 18:38:20,834][105692] Updated weights for policy 0, policy_version 449741 (0.0010) [2023-12-26 18:38:20,899][105692] Updated weights for policy 0, policy_version 449751 (0.0009) [2023-12-26 18:38:20,961][105692] Updated weights for policy 0, policy_version 449761 (0.0006) [2023-12-26 18:38:21,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 230408192. Throughput: 0: 9724.8, 1: 9826.1. Samples: 230393948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:38:21,062][104569] Avg episode reward: [(0, '9358.972'), (1, '7245.087')] [2023-12-26 18:38:21,374][105620] Updated weights for policy 1, policy_version 450155 (0.0008) [2023-12-26 18:38:21,438][105620] Updated weights for policy 1, policy_version 450165 (0.0011) [2023-12-26 18:38:21,506][105620] Updated weights for policy 1, policy_version 450175 (0.0010) [2023-12-26 18:38:21,738][105692] Updated weights for policy 0, policy_version 449771 (0.0009) [2023-12-26 18:38:21,791][105692] Updated weights for policy 0, policy_version 449781 (0.0007) [2023-12-26 18:38:21,851][105692] Updated weights for policy 0, policy_version 449791 (0.0008) [2023-12-26 18:38:22,281][105620] Updated weights for policy 1, policy_version 450185 (0.0010) [2023-12-26 18:38:22,351][105620] Updated weights for policy 1, policy_version 450195 (0.0010) [2023-12-26 18:38:22,419][105620] Updated weights for policy 1, policy_version 450205 (0.0010) [2023-12-26 18:38:22,482][105620] Updated weights for policy 1, policy_version 450215 (0.0008) [2023-12-26 18:38:22,551][105692] Updated weights for policy 0, policy_version 449801 (0.0008) [2023-12-26 18:38:22,599][105692] Updated weights for policy 0, policy_version 449811 (0.0009) [2023-12-26 18:38:22,645][105692] Updated weights for policy 0, policy_version 449821 (0.0009) [2023-12-26 18:38:22,691][105692] Updated weights for policy 0, policy_version 449831 (0.0009) [2023-12-26 18:38:23,321][105620] Updated weights for policy 1, policy_version 450225 (0.0006) [2023-12-26 18:38:23,324][105692] Updated weights for policy 0, policy_version 449841 (0.0005) [2023-12-26 18:38:23,379][105620] Updated weights for policy 1, policy_version 450235 (0.0008) [2023-12-26 18:38:23,384][105692] Updated weights for policy 0, policy_version 449851 (0.0008) [2023-12-26 18:38:23,429][105620] Updated weights for policy 1, policy_version 450245 (0.0007) [2023-12-26 18:38:23,445][105692] Updated weights for policy 0, policy_version 449861 (0.0008) [2023-12-26 18:38:23,990][105620] Updated weights for policy 1, policy_version 450255 (0.0005) [2023-12-26 18:38:24,040][105620] Updated weights for policy 1, policy_version 450265 (0.0005) [2023-12-26 18:38:24,086][105620] Updated weights for policy 1, policy_version 450275 (0.0005) [2023-12-26 18:38:24,293][105692] Updated weights for policy 0, policy_version 449871 (0.0008) [2023-12-26 18:38:24,348][105692] Updated weights for policy 0, policy_version 449881 (0.0008) [2023-12-26 18:38:24,400][105692] Updated weights for policy 0, policy_version 449891 (0.0008) [2023-12-26 18:38:24,678][105620] Updated weights for policy 1, policy_version 450285 (0.0006) [2023-12-26 18:38:24,739][105620] Updated weights for policy 1, policy_version 450295 (0.0006) [2023-12-26 18:38:24,801][105620] Updated weights for policy 1, policy_version 450305 (0.0005) [2023-12-26 18:38:25,145][105692] Updated weights for policy 0, policy_version 449901 (0.0008) [2023-12-26 18:38:25,202][105692] Updated weights for policy 0, policy_version 449911 (0.0010) [2023-12-26 18:38:25,260][105692] Updated weights for policy 0, policy_version 449921 (0.0010) [2023-12-26 18:38:25,315][105620] Updated weights for policy 1, policy_version 450315 (0.0007) [2023-12-26 18:38:25,376][105620] Updated weights for policy 1, policy_version 450325 (0.0010) [2023-12-26 18:38:25,435][105620] Updated weights for policy 1, policy_version 450335 (0.0010) [2023-12-26 18:38:25,947][105692] Updated weights for policy 0, policy_version 449931 (0.0010) [2023-12-26 18:38:26,009][105692] Updated weights for policy 0, policy_version 449941 (0.0010) [2023-12-26 18:38:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 230498304. Throughput: 0: 9624.8, 1: 9942.7. Samples: 230512344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:38:26,063][104569] Avg episode reward: [(0, '9358.896'), (1, '8652.315')] [2023-12-26 18:38:26,067][105692] Updated weights for policy 0, policy_version 449951 (0.0009) [2023-12-26 18:38:26,106][105620] Updated weights for policy 1, policy_version 450345 (0.0008) [2023-12-26 18:38:26,167][105620] Updated weights for policy 1, policy_version 450355 (0.0005) [2023-12-26 18:38:26,228][105620] Updated weights for policy 1, policy_version 450365 (0.0005) [2023-12-26 18:38:26,290][105620] Updated weights for policy 1, policy_version 450375 (0.0005) [2023-12-26 18:38:26,855][105620] Updated weights for policy 1, policy_version 450385 (0.0009) [2023-12-26 18:38:26,866][105692] Updated weights for policy 0, policy_version 449961 (0.0009) [2023-12-26 18:38:26,911][105620] Updated weights for policy 1, policy_version 450395 (0.0007) [2023-12-26 18:38:26,914][105692] Updated weights for policy 0, policy_version 449971 (0.0008) [2023-12-26 18:38:26,966][105692] Updated weights for policy 0, policy_version 449982 (0.0007) [2023-12-26 18:38:26,968][105620] Updated weights for policy 1, policy_version 450405 (0.0006) [2023-12-26 18:38:27,017][105692] Updated weights for policy 0, policy_version 449992 (0.0008) [2023-12-26 18:38:27,573][105620] Updated weights for policy 1, policy_version 450415 (0.0006) [2023-12-26 18:38:27,619][105620] Updated weights for policy 1, policy_version 450425 (0.0005) [2023-12-26 18:38:27,664][105620] Updated weights for policy 1, policy_version 450435 (0.0009) [2023-12-26 18:38:27,744][105692] Updated weights for policy 0, policy_version 450002 (0.0008) [2023-12-26 18:38:27,789][105692] Updated weights for policy 0, policy_version 450012 (0.0007) [2023-12-26 18:38:27,837][105692] Updated weights for policy 0, policy_version 450022 (0.0008) [2023-12-26 18:38:28,413][105620] Updated weights for policy 1, policy_version 450445 (0.0010) [2023-12-26 18:38:28,474][105620] Updated weights for policy 1, policy_version 450455 (0.0011) [2023-12-26 18:38:28,534][105620] Updated weights for policy 1, policy_version 450465 (0.0011) [2023-12-26 18:38:28,634][105692] Updated weights for policy 0, policy_version 450032 (0.0008) [2023-12-26 18:38:28,687][105692] Updated weights for policy 0, policy_version 450042 (0.0008) [2023-12-26 18:38:28,783][105692] Updated weights for policy 0, policy_version 450052 (0.0008) [2023-12-26 18:38:29,326][105620] Updated weights for policy 1, policy_version 450475 (0.0011) [2023-12-26 18:38:29,389][105620] Updated weights for policy 1, policy_version 450485 (0.0011) [2023-12-26 18:38:29,447][105620] Updated weights for policy 1, policy_version 450495 (0.0010) [2023-12-26 18:38:29,528][105692] Updated weights for policy 0, policy_version 450062 (0.0009) [2023-12-26 18:38:29,588][105692] Updated weights for policy 0, policy_version 450072 (0.0008) [2023-12-26 18:38:29,633][105692] Updated weights for policy 0, policy_version 450082 (0.0008) [2023-12-26 18:38:30,210][105620] Updated weights for policy 1, policy_version 450505 (0.0010) [2023-12-26 18:38:30,268][105620] Updated weights for policy 1, policy_version 450515 (0.0009) [2023-12-26 18:38:30,321][105620] Updated weights for policy 1, policy_version 450525 (0.0006) [2023-12-26 18:38:30,383][105620] Updated weights for policy 1, policy_version 450535 (0.0007) [2023-12-26 18:38:30,418][105692] Updated weights for policy 0, policy_version 450092 (0.0009) [2023-12-26 18:38:30,473][105692] Updated weights for policy 0, policy_version 450102 (0.0009) [2023-12-26 18:38:30,525][105692] Updated weights for policy 0, policy_version 450112 (0.0009) [2023-12-26 18:38:31,034][105620] Updated weights for policy 1, policy_version 450545 (0.0008) [2023-12-26 18:38:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 230596608. Throughput: 0: 9628.4, 1: 9993.0. Samples: 230572184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:38:31,063][104569] Avg episode reward: [(0, '9358.884'), (1, '7956.306')] [2023-12-26 18:38:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000450120_115245056.pth... [2023-12-26 18:38:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000449000_114958336.pth [2023-12-26 18:38:31,093][105620] Updated weights for policy 1, policy_version 450555 (0.0009) [2023-12-26 18:38:31,154][105620] Updated weights for policy 1, policy_version 450565 (0.0007) [2023-12-26 18:38:31,172][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000450568_115359744.pth... [2023-12-26 18:38:31,176][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000449384_115056640.pth [2023-12-26 18:38:31,334][105692] Updated weights for policy 0, policy_version 450122 (0.0009) [2023-12-26 18:38:31,401][105692] Updated weights for policy 0, policy_version 450132 (0.0009) [2023-12-26 18:38:31,466][105692] Updated weights for policy 0, policy_version 450142 (0.0009) [2023-12-26 18:38:31,528][105692] Updated weights for policy 0, policy_version 450152 (0.0009) [2023-12-26 18:38:31,895][105620] Updated weights for policy 1, policy_version 450575 (0.0008) [2023-12-26 18:38:31,958][105620] Updated weights for policy 1, policy_version 450585 (0.0009) [2023-12-26 18:38:32,017][105620] Updated weights for policy 1, policy_version 450595 (0.0009) [2023-12-26 18:38:32,268][105692] Updated weights for policy 0, policy_version 450162 (0.0010) [2023-12-26 18:38:32,333][105692] Updated weights for policy 0, policy_version 450172 (0.0009) [2023-12-26 18:38:32,389][105692] Updated weights for policy 0, policy_version 450182 (0.0009) [2023-12-26 18:38:32,705][105620] Updated weights for policy 1, policy_version 450605 (0.0007) [2023-12-26 18:38:32,759][105620] Updated weights for policy 1, policy_version 450615 (0.0006) [2023-12-26 18:38:32,809][105620] Updated weights for policy 1, policy_version 450625 (0.0006) [2023-12-26 18:38:33,100][105692] Updated weights for policy 0, policy_version 450192 (0.0006) [2023-12-26 18:38:33,161][105692] Updated weights for policy 0, policy_version 450202 (0.0005) [2023-12-26 18:38:33,212][105692] Updated weights for policy 0, policy_version 450212 (0.0006) [2023-12-26 18:38:33,429][105620] Updated weights for policy 1, policy_version 450635 (0.0007) [2023-12-26 18:38:33,488][105620] Updated weights for policy 1, policy_version 450645 (0.0009) [2023-12-26 18:38:33,561][105620] Updated weights for policy 1, policy_version 450655 (0.0010) [2023-12-26 18:38:33,744][105692] Updated weights for policy 0, policy_version 450222 (0.0006) [2023-12-26 18:38:33,802][105692] Updated weights for policy 0, policy_version 450232 (0.0005) [2023-12-26 18:38:33,863][105692] Updated weights for policy 0, policy_version 450242 (0.0005) [2023-12-26 18:38:34,146][105620] Updated weights for policy 1, policy_version 450665 (0.0007) [2023-12-26 18:38:34,210][105620] Updated weights for policy 1, policy_version 450675 (0.0010) [2023-12-26 18:38:34,274][105620] Updated weights for policy 1, policy_version 450685 (0.0011) [2023-12-26 18:38:34,334][105620] Updated weights for policy 1, policy_version 450695 (0.0011) [2023-12-26 18:38:34,524][105692] Updated weights for policy 0, policy_version 450252 (0.0007) [2023-12-26 18:38:34,578][105692] Updated weights for policy 0, policy_version 450262 (0.0008) [2023-12-26 18:38:34,633][105692] Updated weights for policy 0, policy_version 450272 (0.0007) [2023-12-26 18:38:35,031][105620] Updated weights for policy 1, policy_version 450705 (0.0011) [2023-12-26 18:38:35,083][105620] Updated weights for policy 1, policy_version 450715 (0.0010) [2023-12-26 18:38:35,127][105620] Updated weights for policy 1, policy_version 450725 (0.0010) [2023-12-26 18:38:35,428][105692] Updated weights for policy 0, policy_version 450282 (0.0008) [2023-12-26 18:38:35,493][105692] Updated weights for policy 0, policy_version 450292 (0.0008) [2023-12-26 18:38:35,552][105692] Updated weights for policy 0, policy_version 450302 (0.0009) [2023-12-26 18:38:35,782][105620] Updated weights for policy 1, policy_version 450735 (0.0010) [2023-12-26 18:38:35,843][105620] Updated weights for policy 1, policy_version 450745 (0.0010) [2023-12-26 18:38:35,904][105620] Updated weights for policy 1, policy_version 450755 (0.0010) [2023-12-26 18:38:36,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 230703104. Throughput: 0: 9698.4, 1: 9978.9. Samples: 230690504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:38:36,062][104569] Avg episode reward: [(0, '9358.912'), (1, '7450.396')] [2023-12-26 18:38:36,321][105692] Updated weights for policy 0, policy_version 450313 (0.0011) [2023-12-26 18:38:36,386][105692] Updated weights for policy 0, policy_version 450323 (0.0008) [2023-12-26 18:38:36,447][105692] Updated weights for policy 0, policy_version 450333 (0.0008) [2023-12-26 18:38:36,512][105692] Updated weights for policy 0, policy_version 450343 (0.0010) [2023-12-26 18:38:36,622][105620] Updated weights for policy 1, policy_version 450765 (0.0008) [2023-12-26 18:38:36,685][105620] Updated weights for policy 1, policy_version 450775 (0.0010) [2023-12-26 18:38:36,747][105620] Updated weights for policy 1, policy_version 450785 (0.0010) [2023-12-26 18:38:37,310][105692] Updated weights for policy 0, policy_version 450353 (0.0008) [2023-12-26 18:38:37,365][105692] Updated weights for policy 0, policy_version 450364 (0.0010) [2023-12-26 18:38:37,400][105620] Updated weights for policy 1, policy_version 450795 (0.0009) [2023-12-26 18:38:37,414][105692] Updated weights for policy 0, policy_version 450374 (0.0009) [2023-12-26 18:38:37,455][105620] Updated weights for policy 1, policy_version 450805 (0.0005) [2023-12-26 18:38:37,507][105620] Updated weights for policy 1, policy_version 450815 (0.0005) [2023-12-26 18:38:38,097][105620] Updated weights for policy 1, policy_version 450825 (0.0006) [2023-12-26 18:38:38,148][105620] Updated weights for policy 1, policy_version 450835 (0.0010) [2023-12-26 18:38:38,213][105620] Updated weights for policy 1, policy_version 450845 (0.0011) [2023-12-26 18:38:38,263][105692] Updated weights for policy 0, policy_version 450384 (0.0005) [2023-12-26 18:38:38,268][105620] Updated weights for policy 1, policy_version 450855 (0.0010) [2023-12-26 18:38:38,318][105692] Updated weights for policy 0, policy_version 450394 (0.0008) [2023-12-26 18:38:38,383][105692] Updated weights for policy 0, policy_version 450404 (0.0010) [2023-12-26 18:38:38,875][105620] Updated weights for policy 1, policy_version 450865 (0.0006) [2023-12-26 18:38:38,938][105620] Updated weights for policy 1, policy_version 450875 (0.0005) [2023-12-26 18:38:39,004][105620] Updated weights for policy 1, policy_version 450885 (0.0010) [2023-12-26 18:38:39,264][105692] Updated weights for policy 0, policy_version 450414 (0.0010) [2023-12-26 18:38:39,333][105692] Updated weights for policy 0, policy_version 450424 (0.0008) [2023-12-26 18:38:39,399][105692] Updated weights for policy 0, policy_version 450434 (0.0009) [2023-12-26 18:38:39,652][105620] Updated weights for policy 1, policy_version 450895 (0.0009) [2023-12-26 18:38:39,722][105620] Updated weights for policy 1, policy_version 450905 (0.0006) [2023-12-26 18:38:39,784][105620] Updated weights for policy 1, policy_version 450915 (0.0009) [2023-12-26 18:38:40,188][105692] Updated weights for policy 0, policy_version 450444 (0.0007) [2023-12-26 18:38:40,246][105692] Updated weights for policy 0, policy_version 450454 (0.0007) [2023-12-26 18:38:40,302][105692] Updated weights for policy 0, policy_version 450464 (0.0009) [2023-12-26 18:38:40,435][105620] Updated weights for policy 1, policy_version 450925 (0.0010) [2023-12-26 18:38:40,500][105620] Updated weights for policy 1, policy_version 450935 (0.0009) [2023-12-26 18:38:40,568][105620] Updated weights for policy 1, policy_version 450945 (0.0009) [2023-12-26 18:38:40,941][105692] Updated weights for policy 0, policy_version 450474 (0.0008) [2023-12-26 18:38:40,993][105692] Updated weights for policy 0, policy_version 450484 (0.0006) [2023-12-26 18:38:41,057][105692] Updated weights for policy 0, policy_version 450494 (0.0007) [2023-12-26 18:38:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 230793216. Throughput: 0: 9608.4, 1: 10001.2. Samples: 230804964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:38:41,063][104569] Avg episode reward: [(0, '9358.821'), (1, '7529.199')] [2023-12-26 18:38:41,117][105692] Updated weights for policy 0, policy_version 450504 (0.0009) [2023-12-26 18:38:41,391][105620] Updated weights for policy 1, policy_version 450955 (0.0008) [2023-12-26 18:38:41,453][105620] Updated weights for policy 1, policy_version 450965 (0.0009) [2023-12-26 18:38:41,517][105620] Updated weights for policy 1, policy_version 450975 (0.0009) [2023-12-26 18:38:41,857][105692] Updated weights for policy 0, policy_version 450514 (0.0008) [2023-12-26 18:38:41,922][105692] Updated weights for policy 0, policy_version 450524 (0.0009) [2023-12-26 18:38:41,988][105692] Updated weights for policy 0, policy_version 450534 (0.0009) [2023-12-26 18:38:42,279][105620] Updated weights for policy 1, policy_version 450985 (0.0009) [2023-12-26 18:38:42,344][105620] Updated weights for policy 1, policy_version 450995 (0.0008) [2023-12-26 18:38:42,404][105620] Updated weights for policy 1, policy_version 451005 (0.0008) [2023-12-26 18:38:42,452][105620] Updated weights for policy 1, policy_version 451015 (0.0005) [2023-12-26 18:38:42,779][105692] Updated weights for policy 0, policy_version 450544 (0.0009) [2023-12-26 18:38:42,839][105692] Updated weights for policy 0, policy_version 450554 (0.0009) [2023-12-26 18:38:42,898][105692] Updated weights for policy 0, policy_version 450564 (0.0007) [2023-12-26 18:38:43,199][105620] Updated weights for policy 1, policy_version 451025 (0.0008) [2023-12-26 18:38:43,249][105620] Updated weights for policy 1, policy_version 451035 (0.0007) [2023-12-26 18:38:43,297][105620] Updated weights for policy 1, policy_version 451045 (0.0006) [2023-12-26 18:38:43,484][105692] Updated weights for policy 0, policy_version 450574 (0.0005) [2023-12-26 18:38:43,530][105692] Updated weights for policy 0, policy_version 450584 (0.0005) [2023-12-26 18:38:43,574][105692] Updated weights for policy 0, policy_version 450594 (0.0005) [2023-12-26 18:38:44,132][105620] Updated weights for policy 1, policy_version 451055 (0.0009) [2023-12-26 18:38:44,194][105620] Updated weights for policy 1, policy_version 451065 (0.0009) [2023-12-26 18:38:44,230][105692] Updated weights for policy 0, policy_version 450604 (0.0006) [2023-12-26 18:38:44,249][105620] Updated weights for policy 1, policy_version 451075 (0.0007) [2023-12-26 18:38:44,280][105692] Updated weights for policy 0, policy_version 450614 (0.0007) [2023-12-26 18:38:44,327][105692] Updated weights for policy 0, policy_version 450624 (0.0008) [2023-12-26 18:38:44,975][105692] Updated weights for policy 0, policy_version 450634 (0.0006) [2023-12-26 18:38:45,031][105692] Updated weights for policy 0, policy_version 450644 (0.0007) [2023-12-26 18:38:45,080][105692] Updated weights for policy 0, policy_version 450654 (0.0007) [2023-12-26 18:38:45,085][105620] Updated weights for policy 1, policy_version 451085 (0.0007) [2023-12-26 18:38:45,133][105692] Updated weights for policy 0, policy_version 450664 (0.0008) [2023-12-26 18:38:45,144][105620] Updated weights for policy 1, policy_version 451095 (0.0008) [2023-12-26 18:38:45,200][105620] Updated weights for policy 1, policy_version 451105 (0.0009) [2023-12-26 18:38:45,893][105692] Updated weights for policy 0, policy_version 450674 (0.0009) [2023-12-26 18:38:45,925][105620] Updated weights for policy 1, policy_version 451115 (0.0009) [2023-12-26 18:38:45,952][105692] Updated weights for policy 0, policy_version 450684 (0.0007) [2023-12-26 18:38:45,986][105620] Updated weights for policy 1, policy_version 451125 (0.0008) [2023-12-26 18:38:46,015][105692] Updated weights for policy 0, policy_version 450694 (0.0007) [2023-12-26 18:38:46,043][105620] Updated weights for policy 1, policy_version 451135 (0.0008) [2023-12-26 18:38:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 230891520. Throughput: 0: 9557.5, 1: 10041.5. Samples: 230862604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:38:46,062][104569] Avg episode reward: [(0, '9356.432'), (1, '7598.453')] [2023-12-26 18:38:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000450696_115392512.pth... [2023-12-26 18:38:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000449576_115105792.pth [2023-12-26 18:38:46,093][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000451144_115507200.pth... [2023-12-26 18:38:46,098][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000449960_115204096.pth [2023-12-26 18:38:46,709][105692] Updated weights for policy 0, policy_version 450704 (0.0008) [2023-12-26 18:38:46,768][105692] Updated weights for policy 0, policy_version 450714 (0.0005) [2023-12-26 18:38:46,805][105620] Updated weights for policy 1, policy_version 451145 (0.0009) [2023-12-26 18:38:46,825][105692] Updated weights for policy 0, policy_version 450724 (0.0005) [2023-12-26 18:38:46,857][105620] Updated weights for policy 1, policy_version 451155 (0.0009) [2023-12-26 18:38:46,907][105620] Updated weights for policy 1, policy_version 451165 (0.0009) [2023-12-26 18:38:46,958][105620] Updated weights for policy 1, policy_version 451176 (0.0009) [2023-12-26 18:38:47,534][105692] Updated weights for policy 0, policy_version 450734 (0.0008) [2023-12-26 18:38:47,581][105692] Updated weights for policy 0, policy_version 450744 (0.0009) [2023-12-26 18:38:47,632][105692] Updated weights for policy 0, policy_version 450754 (0.0007) [2023-12-26 18:38:47,708][105620] Updated weights for policy 1, policy_version 451186 (0.0010) [2023-12-26 18:38:47,762][105620] Updated weights for policy 1, policy_version 451196 (0.0009) [2023-12-26 18:38:47,808][105620] Updated weights for policy 1, policy_version 451206 (0.0008) [2023-12-26 18:38:48,381][105692] Updated weights for policy 0, policy_version 450764 (0.0005) [2023-12-26 18:38:48,447][105692] Updated weights for policy 0, policy_version 450774 (0.0007) [2023-12-26 18:38:48,509][105692] Updated weights for policy 0, policy_version 450784 (0.0009) [2023-12-26 18:38:48,562][105620] Updated weights for policy 1, policy_version 451216 (0.0007) [2023-12-26 18:38:48,621][105620] Updated weights for policy 1, policy_version 451226 (0.0009) [2023-12-26 18:38:48,678][105620] Updated weights for policy 1, policy_version 451236 (0.0008) [2023-12-26 18:38:49,116][105692] Updated weights for policy 0, policy_version 450794 (0.0007) [2023-12-26 18:38:49,168][105692] Updated weights for policy 0, policy_version 450804 (0.0005) [2023-12-26 18:38:49,236][105692] Updated weights for policy 0, policy_version 450814 (0.0006) [2023-12-26 18:38:49,306][105692] Updated weights for policy 0, policy_version 450824 (0.0008) [2023-12-26 18:38:49,491][105620] Updated weights for policy 1, policy_version 451246 (0.0008) [2023-12-26 18:38:49,553][105620] Updated weights for policy 1, policy_version 451256 (0.0010) [2023-12-26 18:38:49,627][105620] Updated weights for policy 1, policy_version 451266 (0.0009) [2023-12-26 18:38:49,965][105692] Updated weights for policy 0, policy_version 450834 (0.0010) [2023-12-26 18:38:50,020][105692] Updated weights for policy 0, policy_version 450844 (0.0006) [2023-12-26 18:38:50,079][105692] Updated weights for policy 0, policy_version 450854 (0.0006) [2023-12-26 18:38:50,444][105620] Updated weights for policy 1, policy_version 451276 (0.0010) [2023-12-26 18:38:50,506][105620] Updated weights for policy 1, policy_version 451286 (0.0009) [2023-12-26 18:38:50,560][105620] Updated weights for policy 1, policy_version 451296 (0.0008) [2023-12-26 18:38:50,686][105692] Updated weights for policy 0, policy_version 450864 (0.0009) [2023-12-26 18:38:50,738][105692] Updated weights for policy 0, policy_version 450874 (0.0009) [2023-12-26 18:38:50,784][105692] Updated weights for policy 0, policy_version 450884 (0.0005) [2023-12-26 18:38:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 230989824. Throughput: 0: 9616.8, 1: 9914.0. Samples: 230977468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:38:51,063][104569] Avg episode reward: [(0, '9354.648'), (1, '7851.313')] [2023-12-26 18:38:51,381][105620] Updated weights for policy 1, policy_version 451306 (0.0009) [2023-12-26 18:38:51,443][105620] Updated weights for policy 1, policy_version 451316 (0.0008) [2023-12-26 18:38:51,501][105620] Updated weights for policy 1, policy_version 451326 (0.0008) [2023-12-26 18:38:51,545][105692] Updated weights for policy 0, policy_version 450894 (0.0007) [2023-12-26 18:38:51,559][105620] Updated weights for policy 1, policy_version 451336 (0.0007) [2023-12-26 18:38:51,600][105692] Updated weights for policy 0, policy_version 450904 (0.0008) [2023-12-26 18:38:51,667][105692] Updated weights for policy 0, policy_version 450914 (0.0009) [2023-12-26 18:38:52,368][105692] Updated weights for policy 0, policy_version 450924 (0.0013) [2023-12-26 18:38:52,386][105620] Updated weights for policy 1, policy_version 451346 (0.0008) [2023-12-26 18:38:52,425][105692] Updated weights for policy 0, policy_version 450934 (0.0007) [2023-12-26 18:38:52,442][105620] Updated weights for policy 1, policy_version 451356 (0.0009) [2023-12-26 18:38:52,485][105692] Updated weights for policy 0, policy_version 450944 (0.0008) [2023-12-26 18:38:52,488][105620] Updated weights for policy 1, policy_version 451366 (0.0006) [2023-12-26 18:38:53,103][105692] Updated weights for policy 0, policy_version 450954 (0.0008) [2023-12-26 18:38:53,158][105692] Updated weights for policy 0, policy_version 450964 (0.0005) [2023-12-26 18:38:53,215][105692] Updated weights for policy 0, policy_version 450974 (0.0005) [2023-12-26 18:38:53,262][105692] Updated weights for policy 0, policy_version 450984 (0.0008) [2023-12-26 18:38:53,340][105620] Updated weights for policy 1, policy_version 451376 (0.0008) [2023-12-26 18:38:53,386][105620] Updated weights for policy 1, policy_version 451386 (0.0008) [2023-12-26 18:38:53,433][105620] Updated weights for policy 1, policy_version 451396 (0.0009) [2023-12-26 18:38:53,957][105692] Updated weights for policy 0, policy_version 450994 (0.0008) [2023-12-26 18:38:54,004][105692] Updated weights for policy 0, policy_version 451004 (0.0008) [2023-12-26 18:38:54,053][105692] Updated weights for policy 0, policy_version 451015 (0.0009) [2023-12-26 18:38:54,221][105620] Updated weights for policy 1, policy_version 451406 (0.0010) [2023-12-26 18:38:54,275][105620] Updated weights for policy 1, policy_version 451417 (0.0009) [2023-12-26 18:38:54,340][105620] Updated weights for policy 1, policy_version 451427 (0.0008) [2023-12-26 18:38:54,681][105692] Updated weights for policy 0, policy_version 451025 (0.0005) [2023-12-26 18:38:54,732][105692] Updated weights for policy 0, policy_version 451035 (0.0005) [2023-12-26 18:38:54,783][105692] Updated weights for policy 0, policy_version 451045 (0.0005) [2023-12-26 18:38:55,113][105620] Updated weights for policy 1, policy_version 451437 (0.0010) [2023-12-26 18:38:55,169][105620] Updated weights for policy 1, policy_version 451447 (0.0011) [2023-12-26 18:38:55,218][105620] Updated weights for policy 1, policy_version 451457 (0.0010) [2023-12-26 18:38:55,410][105692] Updated weights for policy 0, policy_version 451055 (0.0009) [2023-12-26 18:38:55,471][105692] Updated weights for policy 0, policy_version 451065 (0.0010) [2023-12-26 18:38:55,525][105692] Updated weights for policy 0, policy_version 451075 (0.0010) [2023-12-26 18:38:55,874][105620] Updated weights for policy 1, policy_version 451467 (0.0009) [2023-12-26 18:38:55,931][105620] Updated weights for policy 1, policy_version 451477 (0.0005) [2023-12-26 18:38:55,979][105620] Updated weights for policy 1, policy_version 451487 (0.0006) [2023-12-26 18:38:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 231088128. Throughput: 0: 9687.4, 1: 9868.6. Samples: 231094312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:38:56,062][104569] Avg episode reward: [(0, '9059.948'), (1, '7986.104')] [2023-12-26 18:38:56,089][105692] Updated weights for policy 0, policy_version 451085 (0.0007) [2023-12-26 18:38:56,156][105692] Updated weights for policy 0, policy_version 451095 (0.0005) [2023-12-26 18:38:56,222][105692] Updated weights for policy 0, policy_version 451105 (0.0005) [2023-12-26 18:38:56,586][105620] Updated weights for policy 1, policy_version 451497 (0.0010) [2023-12-26 18:38:56,637][105620] Updated weights for policy 1, policy_version 451507 (0.0006) [2023-12-26 18:38:56,687][105620] Updated weights for policy 1, policy_version 451517 (0.0005) [2023-12-26 18:38:56,737][105692] Updated weights for policy 0, policy_version 451115 (0.0006) [2023-12-26 18:38:56,751][105620] Updated weights for policy 1, policy_version 451527 (0.0006) [2023-12-26 18:38:56,790][105692] Updated weights for policy 0, policy_version 451125 (0.0009) [2023-12-26 18:38:56,834][105692] Updated weights for policy 0, policy_version 451135 (0.0010) [2023-12-26 18:38:57,384][105620] Updated weights for policy 1, policy_version 451537 (0.0009) [2023-12-26 18:38:57,452][105620] Updated weights for policy 1, policy_version 451547 (0.0009) [2023-12-26 18:38:57,462][105692] Updated weights for policy 0, policy_version 451145 (0.0010) [2023-12-26 18:38:57,499][105620] Updated weights for policy 1, policy_version 451557 (0.0009) [2023-12-26 18:38:57,517][105692] Updated weights for policy 0, policy_version 451155 (0.0005) [2023-12-26 18:38:57,573][105692] Updated weights for policy 0, policy_version 451165 (0.0005) [2023-12-26 18:38:57,633][105692] Updated weights for policy 0, policy_version 451175 (0.0005) [2023-12-26 18:38:58,150][105692] Updated weights for policy 0, policy_version 451185 (0.0008) [2023-12-26 18:38:58,199][105692] Updated weights for policy 0, policy_version 451195 (0.0008) [2023-12-26 18:38:58,254][105692] Updated weights for policy 0, policy_version 451205 (0.0008) [2023-12-26 18:38:58,286][105620] Updated weights for policy 1, policy_version 451567 (0.0008) [2023-12-26 18:38:58,342][105620] Updated weights for policy 1, policy_version 451577 (0.0008) [2023-12-26 18:38:58,398][105620] Updated weights for policy 1, policy_version 451587 (0.0009) [2023-12-26 18:38:59,049][105692] Updated weights for policy 0, policy_version 451215 (0.0008) [2023-12-26 18:38:59,096][105692] Updated weights for policy 0, policy_version 451225 (0.0009) [2023-12-26 18:38:59,144][105692] Updated weights for policy 0, policy_version 451235 (0.0009) [2023-12-26 18:38:59,194][105620] Updated weights for policy 1, policy_version 451597 (0.0009) [2023-12-26 18:38:59,261][105620] Updated weights for policy 1, policy_version 451607 (0.0009) [2023-12-26 18:38:59,322][105620] Updated weights for policy 1, policy_version 451617 (0.0009) [2023-12-26 18:38:59,943][105692] Updated weights for policy 0, policy_version 451245 (0.0009) [2023-12-26 18:39:00,005][105692] Updated weights for policy 0, policy_version 451255 (0.0009) [2023-12-26 18:39:00,066][105692] Updated weights for policy 0, policy_version 451265 (0.0009) [2023-12-26 18:39:00,090][105620] Updated weights for policy 1, policy_version 451627 (0.0008) [2023-12-26 18:39:00,155][105620] Updated weights for policy 1, policy_version 451637 (0.0009) [2023-12-26 18:39:00,220][105620] Updated weights for policy 1, policy_version 451647 (0.0009) [2023-12-26 18:39:00,673][105692] Updated weights for policy 0, policy_version 451275 (0.0009) [2023-12-26 18:39:00,726][105692] Updated weights for policy 0, policy_version 451286 (0.0010) [2023-12-26 18:39:00,779][105692] Updated weights for policy 0, policy_version 451296 (0.0010) [2023-12-26 18:39:00,906][105620] Updated weights for policy 1, policy_version 451657 (0.0009) [2023-12-26 18:39:00,957][105620] Updated weights for policy 1, policy_version 451667 (0.0009) [2023-12-26 18:39:01,015][105620] Updated weights for policy 1, policy_version 451677 (0.0007) [2023-12-26 18:39:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 231186432. Throughput: 0: 9845.4, 1: 9918.7. Samples: 231158624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:39:01,062][104569] Avg episode reward: [(0, '8970.121'), (1, '8041.734')] [2023-12-26 18:39:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000451304_115548160.pth... [2023-12-26 18:39:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000450120_115245056.pth [2023-12-26 18:39:01,083][105620] Updated weights for policy 1, policy_version 451687 (0.0007) [2023-12-26 18:39:01,090][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000451688_115646464.pth... [2023-12-26 18:39:01,096][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000450568_115359744.pth [2023-12-26 18:39:01,579][105692] Updated weights for policy 0, policy_version 451306 (0.0009) [2023-12-26 18:39:01,639][105692] Updated weights for policy 0, policy_version 451316 (0.0007) [2023-12-26 18:39:01,696][105692] Updated weights for policy 0, policy_version 451326 (0.0008) [2023-12-26 18:39:01,712][105620] Updated weights for policy 1, policy_version 451697 (0.0006) [2023-12-26 18:39:01,753][105692] Updated weights for policy 0, policy_version 451336 (0.0007) [2023-12-26 18:39:01,780][105620] Updated weights for policy 1, policy_version 451707 (0.0012) [2023-12-26 18:39:01,832][105620] Updated weights for policy 1, policy_version 451717 (0.0010) [2023-12-26 18:39:02,482][105692] Updated weights for policy 0, policy_version 451346 (0.0006) [2023-12-26 18:39:02,540][105620] Updated weights for policy 1, policy_version 451727 (0.0010) [2023-12-26 18:39:02,546][105692] Updated weights for policy 0, policy_version 451356 (0.0006) [2023-12-26 18:39:02,591][105620] Updated weights for policy 1, policy_version 451737 (0.0010) [2023-12-26 18:39:02,610][105692] Updated weights for policy 0, policy_version 451366 (0.0008) [2023-12-26 18:39:02,639][105620] Updated weights for policy 1, policy_version 451747 (0.0010) [2023-12-26 18:39:03,323][105692] Updated weights for policy 0, policy_version 451376 (0.0009) [2023-12-26 18:39:03,381][105620] Updated weights for policy 1, policy_version 451757 (0.0010) [2023-12-26 18:39:03,384][105692] Updated weights for policy 0, policy_version 451386 (0.0007) [2023-12-26 18:39:03,434][105620] Updated weights for policy 1, policy_version 451767 (0.0006) [2023-12-26 18:39:03,436][105692] Updated weights for policy 0, policy_version 451396 (0.0008) [2023-12-26 18:39:03,490][105620] Updated weights for policy 1, policy_version 451777 (0.0005) [2023-12-26 18:39:04,195][105620] Updated weights for policy 1, policy_version 451787 (0.0008) [2023-12-26 18:39:04,238][105692] Updated weights for policy 0, policy_version 451406 (0.0009) [2023-12-26 18:39:04,256][105620] Updated weights for policy 1, policy_version 451797 (0.0007) [2023-12-26 18:39:04,291][105692] Updated weights for policy 0, policy_version 451416 (0.0006) [2023-12-26 18:39:04,317][105620] Updated weights for policy 1, policy_version 451807 (0.0008) [2023-12-26 18:39:04,356][105692] Updated weights for policy 0, policy_version 451426 (0.0007) [2023-12-26 18:39:05,000][105620] Updated weights for policy 1, policy_version 451817 (0.0008) [2023-12-26 18:39:05,065][105620] Updated weights for policy 1, policy_version 451827 (0.0006) [2023-12-26 18:39:05,117][105620] Updated weights for policy 1, policy_version 451837 (0.0009) [2023-12-26 18:39:05,164][105620] Updated weights for policy 1, policy_version 451847 (0.0008) [2023-12-26 18:39:05,167][105692] Updated weights for policy 0, policy_version 451436 (0.0007) [2023-12-26 18:39:05,224][105692] Updated weights for policy 0, policy_version 451446 (0.0005) [2023-12-26 18:39:05,278][105692] Updated weights for policy 0, policy_version 451456 (0.0005) [2023-12-26 18:39:05,815][105620] Updated weights for policy 1, policy_version 451857 (0.0005) [2023-12-26 18:39:05,862][105620] Updated weights for policy 1, policy_version 451867 (0.0005) [2023-12-26 18:39:05,909][105620] Updated weights for policy 1, policy_version 451877 (0.0005) [2023-12-26 18:39:05,959][105692] Updated weights for policy 0, policy_version 451466 (0.0005) [2023-12-26 18:39:06,017][105692] Updated weights for policy 0, policy_version 451476 (0.0005) [2023-12-26 18:39:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 231284736. Throughput: 0: 9795.0, 1: 9758.8. Samples: 231273864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:39:06,062][104569] Avg episode reward: [(0, '9264.334'), (1, '7750.936')] [2023-12-26 18:39:06,085][105692] Updated weights for policy 0, policy_version 451486 (0.0005) [2023-12-26 18:39:06,149][105692] Updated weights for policy 0, policy_version 451496 (0.0008) [2023-12-26 18:39:06,574][105620] Updated weights for policy 1, policy_version 451887 (0.0007) [2023-12-26 18:39:06,630][105620] Updated weights for policy 1, policy_version 451897 (0.0008) [2023-12-26 18:39:06,683][105620] Updated weights for policy 1, policy_version 451907 (0.0008) [2023-12-26 18:39:06,834][105692] Updated weights for policy 0, policy_version 451506 (0.0011) [2023-12-26 18:39:06,904][105692] Updated weights for policy 0, policy_version 451516 (0.0010) [2023-12-26 18:39:06,980][105692] Updated weights for policy 0, policy_version 451526 (0.0010) [2023-12-26 18:39:07,356][105620] Updated weights for policy 1, policy_version 451917 (0.0006) [2023-12-26 18:39:07,406][105620] Updated weights for policy 1, policy_version 451927 (0.0006) [2023-12-26 18:39:07,455][105620] Updated weights for policy 1, policy_version 451937 (0.0005) [2023-12-26 18:39:07,639][105692] Updated weights for policy 0, policy_version 451536 (0.0011) [2023-12-26 18:39:07,697][105692] Updated weights for policy 0, policy_version 451546 (0.0010) [2023-12-26 18:39:07,756][105692] Updated weights for policy 0, policy_version 451556 (0.0010) [2023-12-26 18:39:08,131][105620] Updated weights for policy 1, policy_version 451947 (0.0007) [2023-12-26 18:39:08,191][105620] Updated weights for policy 1, policy_version 451957 (0.0009) [2023-12-26 18:39:08,252][105620] Updated weights for policy 1, policy_version 451967 (0.0009) [2023-12-26 18:39:08,430][105692] Updated weights for policy 0, policy_version 451566 (0.0011) [2023-12-26 18:39:08,489][105692] Updated weights for policy 0, policy_version 451576 (0.0011) [2023-12-26 18:39:08,548][105692] Updated weights for policy 0, policy_version 451586 (0.0011) [2023-12-26 18:39:09,044][105620] Updated weights for policy 1, policy_version 451977 (0.0010) [2023-12-26 18:39:09,103][105620] Updated weights for policy 1, policy_version 451987 (0.0009) [2023-12-26 18:39:09,122][105692] Updated weights for policy 0, policy_version 451596 (0.0011) [2023-12-26 18:39:09,152][105620] Updated weights for policy 1, policy_version 451997 (0.0005) [2023-12-26 18:39:09,184][105692] Updated weights for policy 0, policy_version 451606 (0.0010) [2023-12-26 18:39:09,211][105620] Updated weights for policy 1, policy_version 452007 (0.0006) [2023-12-26 18:39:09,239][105692] Updated weights for policy 0, policy_version 451616 (0.0009) [2023-12-26 18:39:09,887][105620] Updated weights for policy 1, policy_version 452017 (0.0010) [2023-12-26 18:39:09,957][105620] Updated weights for policy 1, policy_version 452027 (0.0011) [2023-12-26 18:39:10,018][105692] Updated weights for policy 0, policy_version 451626 (0.0009) [2023-12-26 18:39:10,021][105620] Updated weights for policy 1, policy_version 452037 (0.0011) [2023-12-26 18:39:10,080][105692] Updated weights for policy 0, policy_version 451636 (0.0008) [2023-12-26 18:39:10,142][105692] Updated weights for policy 0, policy_version 451646 (0.0008) [2023-12-26 18:39:10,200][105692] Updated weights for policy 0, policy_version 451656 (0.0007) [2023-12-26 18:39:10,773][105620] Updated weights for policy 1, policy_version 452047 (0.0007) [2023-12-26 18:39:10,833][105620] Updated weights for policy 1, policy_version 452057 (0.0005) [2023-12-26 18:39:10,889][105620] Updated weights for policy 1, policy_version 452067 (0.0006) [2023-12-26 18:39:10,909][105692] Updated weights for policy 0, policy_version 451666 (0.0010) [2023-12-26 18:39:10,969][105692] Updated weights for policy 0, policy_version 451676 (0.0010) [2023-12-26 18:39:11,022][105692] Updated weights for policy 0, policy_version 451687 (0.0010) [2023-12-26 18:39:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 231391232. Throughput: 0: 9847.0, 1: 9744.7. Samples: 231393964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:39:11,063][104569] Avg episode reward: [(0, '9264.993'), (1, '8453.614')] [2023-12-26 18:39:11,557][105620] Updated weights for policy 1, policy_version 452077 (0.0007) [2023-12-26 18:39:11,617][105620] Updated weights for policy 1, policy_version 452087 (0.0009) [2023-12-26 18:39:11,689][105620] Updated weights for policy 1, policy_version 452097 (0.0010) [2023-12-26 18:39:11,832][105692] Updated weights for policy 0, policy_version 451697 (0.0009) [2023-12-26 18:39:11,900][105692] Updated weights for policy 0, policy_version 451707 (0.0009) [2023-12-26 18:39:11,960][105692] Updated weights for policy 0, policy_version 451717 (0.0009) [2023-12-26 18:39:12,441][105620] Updated weights for policy 1, policy_version 452107 (0.0007) [2023-12-26 18:39:12,511][105620] Updated weights for policy 1, policy_version 452117 (0.0008) [2023-12-26 18:39:12,574][105620] Updated weights for policy 1, policy_version 452127 (0.0005) [2023-12-26 18:39:12,733][105692] Updated weights for policy 0, policy_version 451727 (0.0009) [2023-12-26 18:39:12,787][105692] Updated weights for policy 0, policy_version 451737 (0.0008) [2023-12-26 18:39:12,835][105692] Updated weights for policy 0, policy_version 451747 (0.0008) [2023-12-26 18:39:13,205][105620] Updated weights for policy 1, policy_version 452137 (0.0006) [2023-12-26 18:39:13,257][105620] Updated weights for policy 1, policy_version 452147 (0.0010) [2023-12-26 18:39:13,302][105620] Updated weights for policy 1, policy_version 452157 (0.0010) [2023-12-26 18:39:13,346][105620] Updated weights for policy 1, policy_version 452167 (0.0010) [2023-12-26 18:39:13,421][105692] Updated weights for policy 0, policy_version 451757 (0.0006) [2023-12-26 18:39:13,468][105692] Updated weights for policy 0, policy_version 451767 (0.0005) [2023-12-26 18:39:13,511][105692] Updated weights for policy 0, policy_version 451777 (0.0005) [2023-12-26 18:39:13,990][105620] Updated weights for policy 1, policy_version 452177 (0.0006) [2023-12-26 18:39:14,044][105620] Updated weights for policy 1, policy_version 452187 (0.0005) [2023-12-26 18:39:14,067][105692] Updated weights for policy 0, policy_version 451787 (0.0006) [2023-12-26 18:39:14,105][105620] Updated weights for policy 1, policy_version 452197 (0.0006) [2023-12-26 18:39:14,121][105692] Updated weights for policy 0, policy_version 451798 (0.0009) [2023-12-26 18:39:14,164][105692] Updated weights for policy 0, policy_version 451808 (0.0006) [2023-12-26 18:39:14,751][105620] Updated weights for policy 1, policy_version 452207 (0.0007) [2023-12-26 18:39:14,817][105620] Updated weights for policy 1, policy_version 452217 (0.0008) [2023-12-26 18:39:14,841][105692] Updated weights for policy 0, policy_version 451818 (0.0008) [2023-12-26 18:39:14,878][105620] Updated weights for policy 1, policy_version 452227 (0.0008) [2023-12-26 18:39:14,899][105692] Updated weights for policy 0, policy_version 451828 (0.0007) [2023-12-26 18:39:14,948][105692] Updated weights for policy 0, policy_version 451838 (0.0010) [2023-12-26 18:39:15,007][105692] Updated weights for policy 0, policy_version 451848 (0.0010) [2023-12-26 18:39:15,579][105620] Updated weights for policy 1, policy_version 452237 (0.0008) [2023-12-26 18:39:15,643][105620] Updated weights for policy 1, policy_version 452247 (0.0010) [2023-12-26 18:39:15,702][105620] Updated weights for policy 1, policy_version 452257 (0.0010) [2023-12-26 18:39:15,754][105692] Updated weights for policy 0, policy_version 451858 (0.0010) [2023-12-26 18:39:15,805][105692] Updated weights for policy 0, policy_version 451868 (0.0010) [2023-12-26 18:39:15,855][105692] Updated weights for policy 0, policy_version 451878 (0.0010) [2023-12-26 18:39:16,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 231489536. Throughput: 0: 9871.4, 1: 9720.4. Samples: 231453812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:39:16,063][104569] Avg episode reward: [(0, '9265.164'), (1, '8634.013')] [2023-12-26 18:39:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000452264_115793920.pth... [2023-12-26 18:39:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000451880_115695616.pth... [2023-12-26 18:39:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000451144_115507200.pth [2023-12-26 18:39:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000450696_115392512.pth [2023-12-26 18:39:16,445][105620] Updated weights for policy 1, policy_version 452267 (0.0010) [2023-12-26 18:39:16,481][105692] Updated weights for policy 0, policy_version 451888 (0.0006) [2023-12-26 18:39:16,503][105620] Updated weights for policy 1, policy_version 452277 (0.0010) [2023-12-26 18:39:16,538][105692] Updated weights for policy 0, policy_version 451898 (0.0005) [2023-12-26 18:39:16,555][105620] Updated weights for policy 1, policy_version 452287 (0.0010) [2023-12-26 18:39:16,591][105692] Updated weights for policy 0, policy_version 451908 (0.0005) [2023-12-26 18:39:17,232][105692] Updated weights for policy 0, policy_version 451918 (0.0005) [2023-12-26 18:39:17,285][105692] Updated weights for policy 0, policy_version 451928 (0.0005) [2023-12-26 18:39:17,287][105620] Updated weights for policy 1, policy_version 452297 (0.0010) [2023-12-26 18:39:17,336][105692] Updated weights for policy 0, policy_version 451938 (0.0005) [2023-12-26 18:39:17,338][105620] Updated weights for policy 1, policy_version 452307 (0.0010) [2023-12-26 18:39:17,386][105620] Updated weights for policy 1, policy_version 452317 (0.0010) [2023-12-26 18:39:17,437][105620] Updated weights for policy 1, policy_version 452327 (0.0010) [2023-12-26 18:39:18,048][105620] Updated weights for policy 1, policy_version 452337 (0.0006) [2023-12-26 18:39:18,096][105620] Updated weights for policy 1, policy_version 452347 (0.0005) [2023-12-26 18:39:18,152][105692] Updated weights for policy 0, policy_version 451948 (0.0007) [2023-12-26 18:39:18,152][105620] Updated weights for policy 1, policy_version 452357 (0.0005) [2023-12-26 18:39:18,220][105692] Updated weights for policy 0, policy_version 451958 (0.0008) [2023-12-26 18:39:18,288][105692] Updated weights for policy 0, policy_version 451968 (0.0009) [2023-12-26 18:39:18,778][105620] Updated weights for policy 1, policy_version 452367 (0.0009) [2023-12-26 18:39:18,832][105620] Updated weights for policy 1, policy_version 452377 (0.0010) [2023-12-26 18:39:18,884][105620] Updated weights for policy 1, policy_version 452387 (0.0010) [2023-12-26 18:39:19,053][105692] Updated weights for policy 0, policy_version 451978 (0.0010) [2023-12-26 18:39:19,123][105692] Updated weights for policy 0, policy_version 451988 (0.0010) [2023-12-26 18:39:19,177][105692] Updated weights for policy 0, policy_version 451998 (0.0010) [2023-12-26 18:39:19,235][105692] Updated weights for policy 0, policy_version 452008 (0.0009) [2023-12-26 18:39:19,512][105620] Updated weights for policy 1, policy_version 452397 (0.0009) [2023-12-26 18:39:19,573][105620] Updated weights for policy 1, policy_version 452407 (0.0009) [2023-12-26 18:39:19,644][105620] Updated weights for policy 1, policy_version 452417 (0.0010) [2023-12-26 18:39:19,936][105692] Updated weights for policy 0, policy_version 452018 (0.0010) [2023-12-26 18:39:19,990][105692] Updated weights for policy 0, policy_version 452028 (0.0006) [2023-12-26 18:39:20,057][105692] Updated weights for policy 0, policy_version 452038 (0.0009) [2023-12-26 18:39:20,462][105620] Updated weights for policy 1, policy_version 452427 (0.0010) [2023-12-26 18:39:20,528][105620] Updated weights for policy 1, policy_version 452437 (0.0008) [2023-12-26 18:39:20,588][105620] Updated weights for policy 1, policy_version 452447 (0.0010) [2023-12-26 18:39:20,687][105692] Updated weights for policy 0, policy_version 452048 (0.0007) [2023-12-26 18:39:20,752][105692] Updated weights for policy 0, policy_version 452058 (0.0006) [2023-12-26 18:39:20,817][105692] Updated weights for policy 0, policy_version 452068 (0.0006) [2023-12-26 18:39:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.9, 300 sec: 19633.0). Total num frames: 231587840. Throughput: 0: 9915.9, 1: 9731.6. Samples: 231574640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:39:21,062][104569] Avg episode reward: [(0, '9177.097'), (1, '9084.755')] [2023-12-26 18:39:21,405][105620] Updated weights for policy 1, policy_version 452457 (0.0008) [2023-12-26 18:39:21,431][105692] Updated weights for policy 0, policy_version 452078 (0.0007) [2023-12-26 18:39:21,470][105620] Updated weights for policy 1, policy_version 452467 (0.0007) [2023-12-26 18:39:21,489][105692] Updated weights for policy 0, policy_version 452088 (0.0005) [2023-12-26 18:39:21,537][105620] Updated weights for policy 1, policy_version 452477 (0.0009) [2023-12-26 18:39:21,556][105692] Updated weights for policy 0, policy_version 452098 (0.0006) [2023-12-26 18:39:21,610][105620] Updated weights for policy 1, policy_version 452487 (0.0008) [2023-12-26 18:39:22,265][105692] Updated weights for policy 0, policy_version 452108 (0.0007) [2023-12-26 18:39:22,330][105692] Updated weights for policy 0, policy_version 452118 (0.0009) [2023-12-26 18:39:22,372][105620] Updated weights for policy 1, policy_version 452497 (0.0009) [2023-12-26 18:39:22,397][105692] Updated weights for policy 0, policy_version 452128 (0.0006) [2023-12-26 18:39:22,432][105620] Updated weights for policy 1, policy_version 452507 (0.0009) [2023-12-26 18:39:22,499][105620] Updated weights for policy 1, policy_version 452517 (0.0009) [2023-12-26 18:39:22,961][105692] Updated weights for policy 0, policy_version 452138 (0.0005) [2023-12-26 18:39:23,024][105692] Updated weights for policy 0, policy_version 452148 (0.0006) [2023-12-26 18:39:23,084][105692] Updated weights for policy 0, policy_version 452158 (0.0007) [2023-12-26 18:39:23,146][105692] Updated weights for policy 0, policy_version 452168 (0.0009) [2023-12-26 18:39:23,375][105620] Updated weights for policy 1, policy_version 452527 (0.0010) [2023-12-26 18:39:23,428][105620] Updated weights for policy 1, policy_version 452537 (0.0010) [2023-12-26 18:39:23,481][105620] Updated weights for policy 1, policy_version 452548 (0.0010) [2023-12-26 18:39:23,723][105692] Updated weights for policy 0, policy_version 452178 (0.0005) [2023-12-26 18:39:23,773][105692] Updated weights for policy 0, policy_version 452188 (0.0005) [2023-12-26 18:39:23,822][105692] Updated weights for policy 0, policy_version 452198 (0.0005) [2023-12-26 18:39:24,348][105692] Updated weights for policy 0, policy_version 452208 (0.0005) [2023-12-26 18:39:24,407][105692] Updated weights for policy 0, policy_version 452218 (0.0007) [2023-12-26 18:39:24,433][105620] Updated weights for policy 1, policy_version 452558 (0.0008) [2023-12-26 18:39:24,460][105692] Updated weights for policy 0, policy_version 452228 (0.0008) [2023-12-26 18:39:24,490][105620] Updated weights for policy 1, policy_version 452568 (0.0008) [2023-12-26 18:39:24,551][105620] Updated weights for policy 1, policy_version 452578 (0.0009) [2023-12-26 18:39:25,085][105692] Updated weights for policy 0, policy_version 452238 (0.0008) [2023-12-26 18:39:25,139][105692] Updated weights for policy 0, policy_version 452248 (0.0009) [2023-12-26 18:39:25,186][105692] Updated weights for policy 0, policy_version 452258 (0.0009) [2023-12-26 18:39:25,279][105620] Updated weights for policy 1, policy_version 452588 (0.0010) [2023-12-26 18:39:25,330][105620] Updated weights for policy 1, policy_version 452598 (0.0009) [2023-12-26 18:39:25,381][105620] Updated weights for policy 1, policy_version 452608 (0.0009) [2023-12-26 18:39:25,968][105692] Updated weights for policy 0, policy_version 452268 (0.0009) [2023-12-26 18:39:26,020][105692] Updated weights for policy 0, policy_version 452278 (0.0009) [2023-12-26 18:39:26,030][105585] KL-divergence is very high: 182.4702 [2023-12-26 18:39:26,054][105620] Updated weights for policy 1, policy_version 452618 (0.0008) [2023-12-26 18:39:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 231677952. Throughput: 0: 10211.7, 1: 9522.9. Samples: 231693020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:39:26,063][104569] Avg episode reward: [(0, '8998.262'), (1, '8999.968')] [2023-12-26 18:39:26,077][105692] Updated weights for policy 0, policy_version 452288 (0.0008) [2023-12-26 18:39:26,078][105585] KL-divergence is very high: 158.8197 [2023-12-26 18:39:26,102][105620] Updated weights for policy 1, policy_version 452628 (0.0005) [2023-12-26 18:39:26,148][105620] Updated weights for policy 1, policy_version 452638 (0.0005) [2023-12-26 18:39:26,195][105620] Updated weights for policy 1, policy_version 452648 (0.0005) [2023-12-26 18:39:26,745][105620] Updated weights for policy 1, policy_version 452658 (0.0009) [2023-12-26 18:39:26,796][105620] Updated weights for policy 1, policy_version 452668 (0.0009) [2023-12-26 18:39:26,844][105620] Updated weights for policy 1, policy_version 452678 (0.0009) [2023-12-26 18:39:26,924][105692] Updated weights for policy 0, policy_version 452298 (0.0009) [2023-12-26 18:39:26,977][105692] Updated weights for policy 0, policy_version 452308 (0.0010) [2023-12-26 18:39:27,031][105692] Updated weights for policy 0, policy_version 452319 (0.0010) [2023-12-26 18:39:27,421][105620] Updated weights for policy 1, policy_version 452688 (0.0006) [2023-12-26 18:39:27,468][105620] Updated weights for policy 1, policy_version 452698 (0.0008) [2023-12-26 18:39:27,518][105620] Updated weights for policy 1, policy_version 452708 (0.0005) [2023-12-26 18:39:27,631][105692] Updated weights for policy 0, policy_version 452331 (0.0008) [2023-12-26 18:39:27,684][105692] Updated weights for policy 0, policy_version 452341 (0.0005) [2023-12-26 18:39:27,736][105692] Updated weights for policy 0, policy_version 452351 (0.0006) [2023-12-26 18:39:28,134][105620] Updated weights for policy 1, policy_version 452718 (0.0006) [2023-12-26 18:39:28,179][105620] Updated weights for policy 1, policy_version 452728 (0.0006) [2023-12-26 18:39:28,230][105620] Updated weights for policy 1, policy_version 452738 (0.0005) [2023-12-26 18:39:28,405][105692] Updated weights for policy 0, policy_version 452361 (0.0006) [2023-12-26 18:39:28,472][105692] Updated weights for policy 0, policy_version 452371 (0.0005) [2023-12-26 18:39:28,535][105692] Updated weights for policy 0, policy_version 452381 (0.0009) [2023-12-26 18:39:28,589][105692] Updated weights for policy 0, policy_version 452392 (0.0009) [2023-12-26 18:39:28,927][105620] Updated weights for policy 1, policy_version 452749 (0.0007) [2023-12-26 18:39:28,974][105620] Updated weights for policy 1, policy_version 452759 (0.0008) [2023-12-26 18:39:29,033][105620] Updated weights for policy 1, policy_version 452769 (0.0009) [2023-12-26 18:39:29,320][105692] Updated weights for policy 0, policy_version 452402 (0.0009) [2023-12-26 18:39:29,384][105692] Updated weights for policy 0, policy_version 452412 (0.0009) [2023-12-26 18:39:29,446][105692] Updated weights for policy 0, policy_version 452422 (0.0009) [2023-12-26 18:39:29,761][105620] Updated weights for policy 1, policy_version 452779 (0.0009) [2023-12-26 18:39:29,825][105620] Updated weights for policy 1, policy_version 452789 (0.0009) [2023-12-26 18:39:29,894][105620] Updated weights for policy 1, policy_version 452799 (0.0007) [2023-12-26 18:39:30,205][105692] Updated weights for policy 0, policy_version 452432 (0.0009) [2023-12-26 18:39:30,261][105692] Updated weights for policy 0, policy_version 452442 (0.0009) [2023-12-26 18:39:30,320][105692] Updated weights for policy 0, policy_version 452452 (0.0009) [2023-12-26 18:39:30,670][105620] Updated weights for policy 1, policy_version 452809 (0.0009) [2023-12-26 18:39:30,733][105620] Updated weights for policy 1, policy_version 452819 (0.0009) [2023-12-26 18:39:30,795][105620] Updated weights for policy 1, policy_version 452829 (0.0009) [2023-12-26 18:39:30,856][105620] Updated weights for policy 1, policy_version 452839 (0.0009) [2023-12-26 18:39:31,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 231784448. Throughput: 0: 10187.5, 1: 9673.8. Samples: 231756368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:39:31,063][104569] Avg episode reward: [(0, '9264.743'), (1, '8822.189')] [2023-12-26 18:39:31,063][105692] Updated weights for policy 0, policy_version 452462 (0.0009) [2023-12-26 18:39:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000452840_115941376.pth... [2023-12-26 18:39:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000451688_115646464.pth [2023-12-26 18:39:31,133][105692] Updated weights for policy 0, policy_version 452472 (0.0009) [2023-12-26 18:39:31,197][105692] Updated weights for policy 0, policy_version 452482 (0.0008) [2023-12-26 18:39:31,236][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000452488_115851264.pth... [2023-12-26 18:39:31,240][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000451304_115548160.pth [2023-12-26 18:39:31,553][105620] Updated weights for policy 1, policy_version 452849 (0.0006) [2023-12-26 18:39:31,599][105620] Updated weights for policy 1, policy_version 452859 (0.0007) [2023-12-26 18:39:31,664][105620] Updated weights for policy 1, policy_version 452869 (0.0009) [2023-12-26 18:39:32,021][105692] Updated weights for policy 0, policy_version 452492 (0.0008) [2023-12-26 18:39:32,070][105692] Updated weights for policy 0, policy_version 452502 (0.0008) [2023-12-26 18:39:32,121][105692] Updated weights for policy 0, policy_version 452512 (0.0009) [2023-12-26 18:39:32,431][105620] Updated weights for policy 1, policy_version 452879 (0.0010) [2023-12-26 18:39:32,500][105620] Updated weights for policy 1, policy_version 452889 (0.0011) [2023-12-26 18:39:32,551][105620] Updated weights for policy 1, policy_version 452899 (0.0010) [2023-12-26 18:39:32,848][105692] Updated weights for policy 0, policy_version 452522 (0.0008) [2023-12-26 18:39:32,903][105692] Updated weights for policy 0, policy_version 452532 (0.0005) [2023-12-26 18:39:32,952][105692] Updated weights for policy 0, policy_version 452542 (0.0005) [2023-12-26 18:39:32,999][105692] Updated weights for policy 0, policy_version 452552 (0.0005) [2023-12-26 18:39:33,279][105620] Updated weights for policy 1, policy_version 452909 (0.0010) [2023-12-26 18:39:33,336][105620] Updated weights for policy 1, policy_version 452919 (0.0010) [2023-12-26 18:39:33,385][105620] Updated weights for policy 1, policy_version 452929 (0.0010) [2023-12-26 18:39:33,533][105692] Updated weights for policy 0, policy_version 452562 (0.0005) [2023-12-26 18:39:33,590][105692] Updated weights for policy 0, policy_version 452572 (0.0005) [2023-12-26 18:39:33,649][105692] Updated weights for policy 0, policy_version 452582 (0.0005) [2023-12-26 18:39:33,945][105620] Updated weights for policy 1, policy_version 452939 (0.0009) [2023-12-26 18:39:33,993][105620] Updated weights for policy 1, policy_version 452949 (0.0005) [2023-12-26 18:39:34,044][105620] Updated weights for policy 1, policy_version 452959 (0.0005) [2023-12-26 18:39:34,195][105692] Updated weights for policy 0, policy_version 452592 (0.0007) [2023-12-26 18:39:34,237][105585] KL-divergence is very high: 139.3811 [2023-12-26 18:39:34,255][105692] Updated weights for policy 0, policy_version 452602 (0.0008) [2023-12-26 18:39:34,287][105585] KL-divergence is very high: 208.0311 [2023-12-26 18:39:34,315][105692] Updated weights for policy 0, policy_version 452612 (0.0009) [2023-12-26 18:39:34,335][105585] KL-divergence is very high: 151.7728 [2023-12-26 18:39:34,730][105620] Updated weights for policy 1, policy_version 452969 (0.0006) [2023-12-26 18:39:34,789][105620] Updated weights for policy 1, policy_version 452979 (0.0010) [2023-12-26 18:39:34,847][105620] Updated weights for policy 1, policy_version 452989 (0.0010) [2023-12-26 18:39:34,905][105620] Updated weights for policy 1, policy_version 452999 (0.0010) [2023-12-26 18:39:34,994][105692] Updated weights for policy 0, policy_version 452622 (0.0008) [2023-12-26 18:39:35,058][105692] Updated weights for policy 0, policy_version 452632 (0.0008) [2023-12-26 18:39:35,110][105692] Updated weights for policy 0, policy_version 452642 (0.0008) [2023-12-26 18:39:35,635][105620] Updated weights for policy 1, policy_version 453009 (0.0006) [2023-12-26 18:39:35,692][105620] Updated weights for policy 1, policy_version 453019 (0.0007) [2023-12-26 18:39:35,713][105692] Updated weights for policy 0, policy_version 452652 (0.0007) [2023-12-26 18:39:35,753][105620] Updated weights for policy 1, policy_version 453029 (0.0006) [2023-12-26 18:39:35,779][105692] Updated weights for policy 0, policy_version 452662 (0.0009) [2023-12-26 18:39:35,841][105692] Updated weights for policy 0, policy_version 452672 (0.0009) [2023-12-26 18:39:36,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 231890944. Throughput: 0: 10179.4, 1: 9769.8. Samples: 231875180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:39:36,062][104569] Avg episode reward: [(0, '9087.253'), (1, '8822.709')] [2023-12-26 18:39:36,308][105620] Updated weights for policy 1, policy_version 453039 (0.0008) [2023-12-26 18:39:36,368][105620] Updated weights for policy 1, policy_version 453049 (0.0010) [2023-12-26 18:39:36,448][105620] Updated weights for policy 1, policy_version 453059 (0.0010) [2023-12-26 18:39:36,683][105692] Updated weights for policy 0, policy_version 452682 (0.0009) [2023-12-26 18:39:36,747][105692] Updated weights for policy 0, policy_version 452692 (0.0005) [2023-12-26 18:39:36,819][105692] Updated weights for policy 0, policy_version 452702 (0.0007) [2023-12-26 18:39:36,874][105692] Updated weights for policy 0, policy_version 452712 (0.0010) [2023-12-26 18:39:37,007][105620] Updated weights for policy 1, policy_version 453069 (0.0011) [2023-12-26 18:39:37,058][105620] Updated weights for policy 1, policy_version 453079 (0.0010) [2023-12-26 18:39:37,110][105620] Updated weights for policy 1, policy_version 453089 (0.0010) [2023-12-26 18:39:37,623][105692] Updated weights for policy 0, policy_version 452722 (0.0009) [2023-12-26 18:39:37,682][105692] Updated weights for policy 0, policy_version 452732 (0.0009) [2023-12-26 18:39:37,738][105692] Updated weights for policy 0, policy_version 452742 (0.0008) [2023-12-26 18:39:37,812][105620] Updated weights for policy 1, policy_version 453099 (0.0010) [2023-12-26 18:39:37,871][105620] Updated weights for policy 1, policy_version 453109 (0.0009) [2023-12-26 18:39:37,930][105620] Updated weights for policy 1, policy_version 453119 (0.0008) [2023-12-26 18:39:38,446][105692] Updated weights for policy 0, policy_version 452752 (0.0008) [2023-12-26 18:39:38,509][105692] Updated weights for policy 0, policy_version 452762 (0.0008) [2023-12-26 18:39:38,562][105692] Updated weights for policy 0, policy_version 452772 (0.0008) [2023-12-26 18:39:38,728][105620] Updated weights for policy 1, policy_version 453129 (0.0009) [2023-12-26 18:39:38,794][105620] Updated weights for policy 1, policy_version 453139 (0.0008) [2023-12-26 18:39:38,864][105620] Updated weights for policy 1, policy_version 453149 (0.0007) [2023-12-26 18:39:38,922][105620] Updated weights for policy 1, policy_version 453159 (0.0005) [2023-12-26 18:39:39,332][105692] Updated weights for policy 0, policy_version 452782 (0.0007) [2023-12-26 18:39:39,398][105692] Updated weights for policy 0, policy_version 452792 (0.0008) [2023-12-26 18:39:39,457][105692] Updated weights for policy 0, policy_version 452802 (0.0008) [2023-12-26 18:39:39,542][105620] Updated weights for policy 1, policy_version 453169 (0.0010) [2023-12-26 18:39:39,613][105620] Updated weights for policy 1, policy_version 453179 (0.0010) [2023-12-26 18:39:39,679][105620] Updated weights for policy 1, policy_version 453189 (0.0010) [2023-12-26 18:39:40,195][105692] Updated weights for policy 0, policy_version 452812 (0.0007) [2023-12-26 18:39:40,261][105692] Updated weights for policy 0, policy_version 452822 (0.0008) [2023-12-26 18:39:40,315][105620] Updated weights for policy 1, policy_version 453199 (0.0011) [2023-12-26 18:39:40,326][105692] Updated weights for policy 0, policy_version 452832 (0.0006) [2023-12-26 18:39:40,372][105620] Updated weights for policy 1, policy_version 453209 (0.0011) [2023-12-26 18:39:40,431][105620] Updated weights for policy 1, policy_version 453219 (0.0010) [2023-12-26 18:39:40,974][105692] Updated weights for policy 0, policy_version 452842 (0.0006) [2023-12-26 18:39:41,030][105692] Updated weights for policy 0, policy_version 452852 (0.0008) [2023-12-26 18:39:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19605.2). Total num frames: 231981056. Throughput: 0: 10050.5, 1: 9937.5. Samples: 231993780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:39:41,063][104569] Avg episode reward: [(0, '9087.584'), (1, '8459.985')] [2023-12-26 18:39:41,088][105692] Updated weights for policy 0, policy_version 452862 (0.0008) [2023-12-26 18:39:41,158][105692] Updated weights for policy 0, policy_version 452872 (0.0008) [2023-12-26 18:39:41,205][105620] Updated weights for policy 1, policy_version 453229 (0.0010) [2023-12-26 18:39:41,264][105620] Updated weights for policy 1, policy_version 453239 (0.0012) [2023-12-26 18:39:41,330][105620] Updated weights for policy 1, policy_version 453249 (0.0009) [2023-12-26 18:39:41,958][105692] Updated weights for policy 0, policy_version 452882 (0.0008) [2023-12-26 18:39:42,016][105692] Updated weights for policy 0, policy_version 452892 (0.0008) [2023-12-26 18:39:42,038][105620] Updated weights for policy 1, policy_version 453259 (0.0009) [2023-12-26 18:39:42,075][105692] Updated weights for policy 0, policy_version 452902 (0.0008) [2023-12-26 18:39:42,128][105620] Updated weights for policy 1, policy_version 453269 (0.0008) [2023-12-26 18:39:42,195][105620] Updated weights for policy 1, policy_version 453279 (0.0008) [2023-12-26 18:39:42,806][105620] Updated weights for policy 1, policy_version 453289 (0.0006) [2023-12-26 18:39:42,864][105620] Updated weights for policy 1, policy_version 453299 (0.0008) [2023-12-26 18:39:42,886][105692] Updated weights for policy 0, policy_version 452912 (0.0007) [2023-12-26 18:39:42,918][105620] Updated weights for policy 1, policy_version 453309 (0.0008) [2023-12-26 18:39:42,944][105692] Updated weights for policy 0, policy_version 452922 (0.0008) [2023-12-26 18:39:42,976][105620] Updated weights for policy 1, policy_version 453319 (0.0008) [2023-12-26 18:39:43,001][105692] Updated weights for policy 0, policy_version 452932 (0.0010) [2023-12-26 18:39:43,674][105620] Updated weights for policy 1, policy_version 453329 (0.0009) [2023-12-26 18:39:43,687][105692] Updated weights for policy 0, policy_version 452942 (0.0007) [2023-12-26 18:39:43,722][105620] Updated weights for policy 1, policy_version 453339 (0.0009) [2023-12-26 18:39:43,738][105692] Updated weights for policy 0, policy_version 452952 (0.0005) [2023-12-26 18:39:43,769][105620] Updated weights for policy 1, policy_version 453349 (0.0008) [2023-12-26 18:39:43,794][105692] Updated weights for policy 0, policy_version 452962 (0.0005) [2023-12-26 18:39:44,373][105692] Updated weights for policy 0, policy_version 452972 (0.0007) [2023-12-26 18:39:44,422][105692] Updated weights for policy 0, policy_version 452982 (0.0005) [2023-12-26 18:39:44,475][105692] Updated weights for policy 0, policy_version 452992 (0.0009) [2023-12-26 18:39:44,633][105620] Updated weights for policy 1, policy_version 453359 (0.0009) [2023-12-26 18:39:44,684][105620] Updated weights for policy 1, policy_version 453369 (0.0007) [2023-12-26 18:39:44,731][105620] Updated weights for policy 1, policy_version 453379 (0.0008) [2023-12-26 18:39:45,183][105692] Updated weights for policy 0, policy_version 453002 (0.0011) [2023-12-26 18:39:45,251][105692] Updated weights for policy 0, policy_version 453012 (0.0011) [2023-12-26 18:39:45,311][105692] Updated weights for policy 0, policy_version 453022 (0.0011) [2023-12-26 18:39:45,371][105692] Updated weights for policy 0, policy_version 453032 (0.0011) [2023-12-26 18:39:45,422][105620] Updated weights for policy 1, policy_version 453389 (0.0008) [2023-12-26 18:39:45,479][105620] Updated weights for policy 1, policy_version 453399 (0.0008) [2023-12-26 18:39:45,533][105620] Updated weights for policy 1, policy_version 453409 (0.0008) [2023-12-26 18:39:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 232079360. Throughput: 0: 9902.8, 1: 9932.3. Samples: 232051204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:39:46,063][104569] Avg episode reward: [(0, '9356.828'), (1, '8299.956')] [2023-12-26 18:39:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000453416_116088832.pth... [2023-12-26 18:39:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000452264_115793920.pth [2023-12-26 18:39:46,081][105692] Updated weights for policy 0, policy_version 453042 (0.0007) [2023-12-26 18:39:46,138][105692] Updated weights for policy 0, policy_version 453052 (0.0008) [2023-12-26 18:39:46,195][105692] Updated weights for policy 0, policy_version 453062 (0.0008) [2023-12-26 18:39:46,202][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000453064_115998720.pth... [2023-12-26 18:39:46,205][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000451880_115695616.pth [2023-12-26 18:39:46,316][105620] Updated weights for policy 1, policy_version 453419 (0.0009) [2023-12-26 18:39:46,369][105620] Updated weights for policy 1, policy_version 453429 (0.0010) [2023-12-26 18:39:46,422][105620] Updated weights for policy 1, policy_version 453439 (0.0009) [2023-12-26 18:39:46,733][105692] Updated weights for policy 0, policy_version 453072 (0.0006) [2023-12-26 18:39:46,800][105692] Updated weights for policy 0, policy_version 453082 (0.0010) [2023-12-26 18:39:46,848][105692] Updated weights for policy 0, policy_version 453092 (0.0010) [2023-12-26 18:39:47,283][105620] Updated weights for policy 1, policy_version 453450 (0.0009) [2023-12-26 18:39:47,337][105620] Updated weights for policy 1, policy_version 453460 (0.0008) [2023-12-26 18:39:47,392][105620] Updated weights for policy 1, policy_version 453470 (0.0008) [2023-12-26 18:39:47,439][105620] Updated weights for policy 1, policy_version 453480 (0.0007) [2023-12-26 18:39:47,517][105692] Updated weights for policy 0, policy_version 453102 (0.0010) [2023-12-26 18:39:47,564][105692] Updated weights for policy 0, policy_version 453112 (0.0010) [2023-12-26 18:39:47,628][105692] Updated weights for policy 0, policy_version 453122 (0.0010) [2023-12-26 18:39:48,219][105620] Updated weights for policy 1, policy_version 453490 (0.0008) [2023-12-26 18:39:48,267][105620] Updated weights for policy 1, policy_version 453500 (0.0008) [2023-12-26 18:39:48,325][105620] Updated weights for policy 1, policy_version 453510 (0.0009) [2023-12-26 18:39:48,361][105692] Updated weights for policy 0, policy_version 453132 (0.0009) [2023-12-26 18:39:48,423][105692] Updated weights for policy 0, policy_version 453142 (0.0006) [2023-12-26 18:39:48,478][105692] Updated weights for policy 0, policy_version 453152 (0.0010) [2023-12-26 18:39:49,027][105692] Updated weights for policy 0, policy_version 453162 (0.0006) [2023-12-26 18:39:49,077][105692] Updated weights for policy 0, policy_version 453172 (0.0005) [2023-12-26 18:39:49,130][105692] Updated weights for policy 0, policy_version 453182 (0.0005) [2023-12-26 18:39:49,187][105692] Updated weights for policy 0, policy_version 453192 (0.0005) [2023-12-26 18:39:49,232][105620] Updated weights for policy 1, policy_version 453520 (0.0009) [2023-12-26 18:39:49,295][105620] Updated weights for policy 1, policy_version 453530 (0.0008) [2023-12-26 18:39:49,363][105620] Updated weights for policy 1, policy_version 453540 (0.0009) [2023-12-26 18:39:49,905][105692] Updated weights for policy 0, policy_version 453203 (0.0010) [2023-12-26 18:39:49,966][105692] Updated weights for policy 0, policy_version 453213 (0.0009) [2023-12-26 18:39:50,018][105692] Updated weights for policy 0, policy_version 453223 (0.0008) [2023-12-26 18:39:50,082][105620] Updated weights for policy 1, policy_version 453550 (0.0007) [2023-12-26 18:39:50,138][105620] Updated weights for policy 1, policy_version 453560 (0.0007) [2023-12-26 18:39:50,198][105620] Updated weights for policy 1, policy_version 453570 (0.0008) [2023-12-26 18:39:50,784][105692] Updated weights for policy 0, policy_version 453233 (0.0010) [2023-12-26 18:39:50,847][105692] Updated weights for policy 0, policy_version 453243 (0.0006) [2023-12-26 18:39:50,906][105692] Updated weights for policy 0, policy_version 453253 (0.0005) [2023-12-26 18:39:50,907][105620] Updated weights for policy 1, policy_version 453580 (0.0009) [2023-12-26 18:39:50,957][105620] Updated weights for policy 1, policy_version 453590 (0.0009) [2023-12-26 18:39:51,010][105620] Updated weights for policy 1, policy_version 453600 (0.0007) [2023-12-26 18:39:51,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 232185856. Throughput: 0: 10068.3, 1: 9817.9. Samples: 232168744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:39:51,062][104569] Avg episode reward: [(0, '9357.998'), (1, '8973.665')] [2023-12-26 18:39:51,556][105692] Updated weights for policy 0, policy_version 453263 (0.0007) [2023-12-26 18:39:51,610][105692] Updated weights for policy 0, policy_version 453273 (0.0008) [2023-12-26 18:39:51,672][105692] Updated weights for policy 0, policy_version 453283 (0.0008) [2023-12-26 18:39:51,803][105620] Updated weights for policy 1, policy_version 453610 (0.0008) [2023-12-26 18:39:51,852][105620] Updated weights for policy 1, policy_version 453620 (0.0010) [2023-12-26 18:39:51,906][105620] Updated weights for policy 1, policy_version 453630 (0.0006) [2023-12-26 18:39:51,964][105620] Updated weights for policy 1, policy_version 453640 (0.0005) [2023-12-26 18:39:52,445][105692] Updated weights for policy 0, policy_version 453293 (0.0009) [2023-12-26 18:39:52,504][105692] Updated weights for policy 0, policy_version 453303 (0.0010) [2023-12-26 18:39:52,559][105692] Updated weights for policy 0, policy_version 453313 (0.0010) [2023-12-26 18:39:52,667][105620] Updated weights for policy 1, policy_version 453650 (0.0008) [2023-12-26 18:39:52,723][105620] Updated weights for policy 1, policy_version 453660 (0.0008) [2023-12-26 18:39:52,786][105620] Updated weights for policy 1, policy_version 453670 (0.0008) [2023-12-26 18:39:53,320][105692] Updated weights for policy 0, policy_version 453323 (0.0010) [2023-12-26 18:39:53,361][105620] Updated weights for policy 1, policy_version 453680 (0.0008) [2023-12-26 18:39:53,374][105692] Updated weights for policy 0, policy_version 453333 (0.0007) [2023-12-26 18:39:53,412][105620] Updated weights for policy 1, policy_version 453690 (0.0008) [2023-12-26 18:39:53,425][105692] Updated weights for policy 0, policy_version 453343 (0.0008) [2023-12-26 18:39:53,459][105620] Updated weights for policy 1, policy_version 453700 (0.0005) [2023-12-26 18:39:54,136][105620] Updated weights for policy 1, policy_version 453710 (0.0008) [2023-12-26 18:39:54,157][105692] Updated weights for policy 0, policy_version 453353 (0.0010) [2023-12-26 18:39:54,195][105620] Updated weights for policy 1, policy_version 453720 (0.0007) [2023-12-26 18:39:54,209][105692] Updated weights for policy 0, policy_version 453363 (0.0010) [2023-12-26 18:39:54,257][105620] Updated weights for policy 1, policy_version 453730 (0.0007) [2023-12-26 18:39:54,262][105692] Updated weights for policy 0, policy_version 453373 (0.0010) [2023-12-26 18:39:54,319][105692] Updated weights for policy 0, policy_version 453383 (0.0008) [2023-12-26 18:39:55,020][105620] Updated weights for policy 1, policy_version 453740 (0.0006) [2023-12-26 18:39:55,052][105692] Updated weights for policy 0, policy_version 453393 (0.0009) [2023-12-26 18:39:55,083][105620] Updated weights for policy 1, policy_version 453750 (0.0008) [2023-12-26 18:39:55,102][105692] Updated weights for policy 0, policy_version 453403 (0.0006) [2023-12-26 18:39:55,144][105620] Updated weights for policy 1, policy_version 453760 (0.0009) [2023-12-26 18:39:55,155][105692] Updated weights for policy 0, policy_version 453413 (0.0006) [2023-12-26 18:39:55,873][105620] Updated weights for policy 1, policy_version 453770 (0.0007) [2023-12-26 18:39:55,930][105620] Updated weights for policy 1, policy_version 453780 (0.0008) [2023-12-26 18:39:55,931][105692] Updated weights for policy 0, policy_version 453423 (0.0009) [2023-12-26 18:39:55,987][105692] Updated weights for policy 0, policy_version 453433 (0.0006) [2023-12-26 18:39:55,989][105620] Updated weights for policy 1, policy_version 453790 (0.0007) [2023-12-26 18:39:56,036][105692] Updated weights for policy 0, policy_version 453443 (0.0007) [2023-12-26 18:39:56,052][105620] Updated weights for policy 1, policy_version 453800 (0.0008) [2023-12-26 18:39:56,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 232284160. Throughput: 0: 9993.7, 1: 9794.1. Samples: 232284412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:39:56,062][104569] Avg episode reward: [(0, '9358.271'), (1, '8633.187')] [2023-12-26 18:39:56,719][105620] Updated weights for policy 1, policy_version 453810 (0.0010) [2023-12-26 18:39:56,777][105620] Updated weights for policy 1, policy_version 453820 (0.0010) [2023-12-26 18:39:56,837][105692] Updated weights for policy 0, policy_version 453453 (0.0007) [2023-12-26 18:39:56,838][105620] Updated weights for policy 1, policy_version 453830 (0.0010) [2023-12-26 18:39:56,894][105692] Updated weights for policy 0, policy_version 453463 (0.0008) [2023-12-26 18:39:56,945][105692] Updated weights for policy 0, policy_version 453473 (0.0008) [2023-12-26 18:39:57,488][105620] Updated weights for policy 1, policy_version 453840 (0.0006) [2023-12-26 18:39:57,543][105620] Updated weights for policy 1, policy_version 453850 (0.0005) [2023-12-26 18:39:57,590][105620] Updated weights for policy 1, policy_version 453860 (0.0005) [2023-12-26 18:39:57,724][105692] Updated weights for policy 0, policy_version 453483 (0.0008) [2023-12-26 18:39:57,780][105692] Updated weights for policy 0, policy_version 453493 (0.0010) [2023-12-26 18:39:57,833][105692] Updated weights for policy 0, policy_version 453504 (0.0009) [2023-12-26 18:39:58,129][105620] Updated weights for policy 1, policy_version 453870 (0.0008) [2023-12-26 18:39:58,189][105620] Updated weights for policy 1, policy_version 453880 (0.0010) [2023-12-26 18:39:58,249][105620] Updated weights for policy 1, policy_version 453890 (0.0011) [2023-12-26 18:39:58,693][105692] Updated weights for policy 0, policy_version 453515 (0.0010) [2023-12-26 18:39:58,759][105692] Updated weights for policy 0, policy_version 453525 (0.0008) [2023-12-26 18:39:58,823][105692] Updated weights for policy 0, policy_version 453535 (0.0008) [2023-12-26 18:39:58,982][105620] Updated weights for policy 1, policy_version 453900 (0.0009) [2023-12-26 18:39:59,046][105620] Updated weights for policy 1, policy_version 453910 (0.0011) [2023-12-26 18:39:59,097][105620] Updated weights for policy 1, policy_version 453920 (0.0010) [2023-12-26 18:39:59,642][105692] Updated weights for policy 0, policy_version 453545 (0.0008) [2023-12-26 18:39:59,706][105692] Updated weights for policy 0, policy_version 453555 (0.0008) [2023-12-26 18:39:59,766][105692] Updated weights for policy 0, policy_version 453565 (0.0011) [2023-12-26 18:39:59,823][105692] Updated weights for policy 0, policy_version 453575 (0.0010) [2023-12-26 18:39:59,872][105620] Updated weights for policy 1, policy_version 453930 (0.0009) [2023-12-26 18:39:59,936][105620] Updated weights for policy 1, policy_version 453940 (0.0007) [2023-12-26 18:39:59,998][105620] Updated weights for policy 1, policy_version 453950 (0.0008) [2023-12-26 18:40:00,051][105620] Updated weights for policy 1, policy_version 453960 (0.0008) [2023-12-26 18:40:00,546][105692] Updated weights for policy 0, policy_version 453585 (0.0010) [2023-12-26 18:40:00,600][105692] Updated weights for policy 0, policy_version 453595 (0.0010) [2023-12-26 18:40:00,658][105692] Updated weights for policy 0, policy_version 453605 (0.0010) [2023-12-26 18:40:00,809][105620] Updated weights for policy 1, policy_version 453970 (0.0008) [2023-12-26 18:40:00,883][105620] Updated weights for policy 1, policy_version 453980 (0.0005) [2023-12-26 18:40:00,940][105620] Updated weights for policy 1, policy_version 453990 (0.0005) [2023-12-26 18:40:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 232374272. Throughput: 0: 9938.2, 1: 9815.0. Samples: 232342708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:40:01,063][104569] Avg episode reward: [(0, '9269.855'), (1, '8811.711')] [2023-12-26 18:40:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000453608_116137984.pth... [2023-12-26 18:40:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000453992_116236288.pth... [2023-12-26 18:40:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000452488_115851264.pth [2023-12-26 18:40:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000452840_115941376.pth [2023-12-26 18:40:01,411][105692] Updated weights for policy 0, policy_version 453615 (0.0010) [2023-12-26 18:40:01,467][105692] Updated weights for policy 0, policy_version 453625 (0.0009) [2023-12-26 18:40:01,530][105692] Updated weights for policy 0, policy_version 453635 (0.0011) [2023-12-26 18:40:01,646][105620] Updated weights for policy 1, policy_version 454000 (0.0009) [2023-12-26 18:40:01,705][105620] Updated weights for policy 1, policy_version 454010 (0.0007) [2023-12-26 18:40:01,772][105620] Updated weights for policy 1, policy_version 454020 (0.0009) [2023-12-26 18:40:02,333][105692] Updated weights for policy 0, policy_version 453645 (0.0010) [2023-12-26 18:40:02,391][105692] Updated weights for policy 0, policy_version 453655 (0.0009) [2023-12-26 18:40:02,442][105692] Updated weights for policy 0, policy_version 453665 (0.0010) [2023-12-26 18:40:02,483][105620] Updated weights for policy 1, policy_version 454030 (0.0010) [2023-12-26 18:40:02,541][105620] Updated weights for policy 1, policy_version 454040 (0.0010) [2023-12-26 18:40:02,599][105620] Updated weights for policy 1, policy_version 454050 (0.0010) [2023-12-26 18:40:03,200][105692] Updated weights for policy 0, policy_version 453675 (0.0010) [2023-12-26 18:40:03,261][105692] Updated weights for policy 0, policy_version 453685 (0.0010) [2023-12-26 18:40:03,279][105620] Updated weights for policy 1, policy_version 454060 (0.0008) [2023-12-26 18:40:03,319][105692] Updated weights for policy 0, policy_version 453695 (0.0010) [2023-12-26 18:40:03,337][105620] Updated weights for policy 1, policy_version 454070 (0.0005) [2023-12-26 18:40:03,387][105620] Updated weights for policy 1, policy_version 454080 (0.0005) [2023-12-26 18:40:03,897][105692] Updated weights for policy 0, policy_version 453705 (0.0010) [2023-12-26 18:40:03,951][105692] Updated weights for policy 0, policy_version 453715 (0.0010) [2023-12-26 18:40:04,004][105692] Updated weights for policy 0, policy_version 453725 (0.0010) [2023-12-26 18:40:04,059][105692] Updated weights for policy 0, policy_version 453735 (0.0010) [2023-12-26 18:40:04,076][105620] Updated weights for policy 1, policy_version 454090 (0.0009) [2023-12-26 18:40:04,142][105620] Updated weights for policy 1, policy_version 454100 (0.0009) [2023-12-26 18:40:04,204][105620] Updated weights for policy 1, policy_version 454110 (0.0010) [2023-12-26 18:40:04,267][105620] Updated weights for policy 1, policy_version 454120 (0.0010) [2023-12-26 18:40:04,799][105692] Updated weights for policy 0, policy_version 453745 (0.0010) [2023-12-26 18:40:04,857][105692] Updated weights for policy 0, policy_version 453755 (0.0010) [2023-12-26 18:40:04,915][105692] Updated weights for policy 0, policy_version 453765 (0.0010) [2023-12-26 18:40:05,000][105620] Updated weights for policy 1, policy_version 454130 (0.0010) [2023-12-26 18:40:05,060][105620] Updated weights for policy 1, policy_version 454140 (0.0010) [2023-12-26 18:40:05,118][105620] Updated weights for policy 1, policy_version 454150 (0.0010) [2023-12-26 18:40:05,542][105692] Updated weights for policy 0, policy_version 453775 (0.0010) [2023-12-26 18:40:05,596][105692] Updated weights for policy 0, policy_version 453785 (0.0010) [2023-12-26 18:40:05,655][105692] Updated weights for policy 0, policy_version 453795 (0.0010) [2023-12-26 18:40:05,847][105620] Updated weights for policy 1, policy_version 454160 (0.0010) [2023-12-26 18:40:05,899][105620] Updated weights for policy 1, policy_version 454170 (0.0010) [2023-12-26 18:40:05,964][105620] Updated weights for policy 1, policy_version 454180 (0.0010) [2023-12-26 18:40:06,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 232472576. Throughput: 0: 9869.9, 1: 9763.3. Samples: 232458136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:40:06,062][104569] Avg episode reward: [(0, '9269.981'), (1, '8132.649')] [2023-12-26 18:40:06,323][105692] Updated weights for policy 0, policy_version 453805 (0.0011) [2023-12-26 18:40:06,385][105692] Updated weights for policy 0, policy_version 453815 (0.0010) [2023-12-26 18:40:06,447][105692] Updated weights for policy 0, policy_version 453825 (0.0006) [2023-12-26 18:40:06,720][105620] Updated weights for policy 1, policy_version 454190 (0.0011) [2023-12-26 18:40:06,779][105620] Updated weights for policy 1, policy_version 454200 (0.0011) [2023-12-26 18:40:06,838][105620] Updated weights for policy 1, policy_version 454210 (0.0010) [2023-12-26 18:40:07,141][105692] Updated weights for policy 0, policy_version 453835 (0.0009) [2023-12-26 18:40:07,195][105692] Updated weights for policy 0, policy_version 453845 (0.0008) [2023-12-26 18:40:07,264][105692] Updated weights for policy 0, policy_version 453855 (0.0010) [2023-12-26 18:40:07,439][105620] Updated weights for policy 1, policy_version 454220 (0.0008) [2023-12-26 18:40:07,499][105620] Updated weights for policy 1, policy_version 454230 (0.0006) [2023-12-26 18:40:07,558][105620] Updated weights for policy 1, policy_version 454240 (0.0005) [2023-12-26 18:40:07,932][105692] Updated weights for policy 0, policy_version 453865 (0.0010) [2023-12-26 18:40:07,984][105692] Updated weights for policy 0, policy_version 453875 (0.0010) [2023-12-26 18:40:08,040][105692] Updated weights for policy 0, policy_version 453885 (0.0011) [2023-12-26 18:40:08,068][105620] Updated weights for policy 1, policy_version 454250 (0.0007) [2023-12-26 18:40:08,095][105692] Updated weights for policy 0, policy_version 453895 (0.0010) [2023-12-26 18:40:08,122][105620] Updated weights for policy 1, policy_version 454260 (0.0010) [2023-12-26 18:40:08,179][105620] Updated weights for policy 1, policy_version 454270 (0.0010) [2023-12-26 18:40:08,244][105620] Updated weights for policy 1, policy_version 454280 (0.0010) [2023-12-26 18:40:08,794][105692] Updated weights for policy 0, policy_version 453905 (0.0008) [2023-12-26 18:40:08,863][105692] Updated weights for policy 0, policy_version 453915 (0.0008) [2023-12-26 18:40:08,919][105692] Updated weights for policy 0, policy_version 453925 (0.0007) [2023-12-26 18:40:08,979][105620] Updated weights for policy 1, policy_version 454290 (0.0010) [2023-12-26 18:40:09,035][105620] Updated weights for policy 1, policy_version 454300 (0.0007) [2023-12-26 18:40:09,088][105620] Updated weights for policy 1, policy_version 454310 (0.0005) [2023-12-26 18:40:09,656][105692] Updated weights for policy 0, policy_version 453935 (0.0008) [2023-12-26 18:40:09,720][105692] Updated weights for policy 0, policy_version 453945 (0.0007) [2023-12-26 18:40:09,728][105620] Updated weights for policy 1, policy_version 454320 (0.0008) [2023-12-26 18:40:09,779][105692] Updated weights for policy 0, policy_version 453955 (0.0006) [2023-12-26 18:40:09,794][105620] Updated weights for policy 1, policy_version 454330 (0.0008) [2023-12-26 18:40:09,864][105620] Updated weights for policy 1, policy_version 454340 (0.0010) [2023-12-26 18:40:10,528][105692] Updated weights for policy 0, policy_version 453965 (0.0008) [2023-12-26 18:40:10,585][105692] Updated weights for policy 0, policy_version 453975 (0.0007) [2023-12-26 18:40:10,591][105620] Updated weights for policy 1, policy_version 454350 (0.0009) [2023-12-26 18:40:10,640][105620] Updated weights for policy 1, policy_version 454360 (0.0010) [2023-12-26 18:40:10,641][105692] Updated weights for policy 0, policy_version 453985 (0.0006) [2023-12-26 18:40:10,709][105620] Updated weights for policy 1, policy_version 454370 (0.0011) [2023-12-26 18:40:11,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 232570880. Throughput: 0: 9735.7, 1: 9940.9. Samples: 232578468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:40:11,062][104569] Avg episode reward: [(0, '9269.171'), (1, '7943.852')] [2023-12-26 18:40:11,434][105692] Updated weights for policy 0, policy_version 453995 (0.0007) [2023-12-26 18:40:11,482][105692] Updated weights for policy 0, policy_version 454005 (0.0007) [2023-12-26 18:40:11,491][105620] Updated weights for policy 1, policy_version 454380 (0.0010) [2023-12-26 18:40:11,537][105692] Updated weights for policy 0, policy_version 454015 (0.0006) [2023-12-26 18:40:11,550][105620] Updated weights for policy 1, policy_version 454390 (0.0011) [2023-12-26 18:40:11,609][105620] Updated weights for policy 1, policy_version 454400 (0.0010) [2023-12-26 18:40:12,342][105692] Updated weights for policy 0, policy_version 454025 (0.0006) [2023-12-26 18:40:12,395][105620] Updated weights for policy 1, policy_version 454410 (0.0010) [2023-12-26 18:40:12,404][105692] Updated weights for policy 0, policy_version 454035 (0.0009) [2023-12-26 18:40:12,458][105620] Updated weights for policy 1, policy_version 454420 (0.0011) [2023-12-26 18:40:12,464][105692] Updated weights for policy 0, policy_version 454045 (0.0006) [2023-12-26 18:40:12,518][105620] Updated weights for policy 1, policy_version 454430 (0.0011) [2023-12-26 18:40:12,524][105692] Updated weights for policy 0, policy_version 454055 (0.0005) [2023-12-26 18:40:12,570][105620] Updated weights for policy 1, policy_version 454440 (0.0011) [2023-12-26 18:40:13,249][105692] Updated weights for policy 0, policy_version 454065 (0.0008) [2023-12-26 18:40:13,267][105620] Updated weights for policy 1, policy_version 454450 (0.0005) [2023-12-26 18:40:13,297][105692] Updated weights for policy 0, policy_version 454075 (0.0008) [2023-12-26 18:40:13,328][105620] Updated weights for policy 1, policy_version 454460 (0.0005) [2023-12-26 18:40:13,352][105692] Updated weights for policy 0, policy_version 454085 (0.0008) [2023-12-26 18:40:13,388][105620] Updated weights for policy 1, policy_version 454470 (0.0007) [2023-12-26 18:40:14,076][105692] Updated weights for policy 0, policy_version 454095 (0.0006) [2023-12-26 18:40:14,128][105692] Updated weights for policy 0, policy_version 454105 (0.0005) [2023-12-26 18:40:14,133][105620] Updated weights for policy 1, policy_version 454480 (0.0009) [2023-12-26 18:40:14,185][105692] Updated weights for policy 0, policy_version 454115 (0.0005) [2023-12-26 18:40:14,189][105620] Updated weights for policy 1, policy_version 454490 (0.0009) [2023-12-26 18:40:14,246][105620] Updated weights for policy 1, policy_version 454500 (0.0009) [2023-12-26 18:40:14,856][105692] Updated weights for policy 0, policy_version 454125 (0.0007) [2023-12-26 18:40:14,914][105692] Updated weights for policy 0, policy_version 454135 (0.0009) [2023-12-26 18:40:14,976][105692] Updated weights for policy 0, policy_version 454145 (0.0009) [2023-12-26 18:40:15,035][105620] Updated weights for policy 1, policy_version 454510 (0.0007) [2023-12-26 18:40:15,098][105620] Updated weights for policy 1, policy_version 454520 (0.0009) [2023-12-26 18:40:15,165][105620] Updated weights for policy 1, policy_version 454530 (0.0010) [2023-12-26 18:40:15,640][105692] Updated weights for policy 0, policy_version 454155 (0.0008) [2023-12-26 18:40:15,701][105692] Updated weights for policy 0, policy_version 454165 (0.0006) [2023-12-26 18:40:15,758][105692] Updated weights for policy 0, policy_version 454175 (0.0008) [2023-12-26 18:40:15,973][105620] Updated weights for policy 1, policy_version 454540 (0.0010) [2023-12-26 18:40:16,045][105620] Updated weights for policy 1, policy_version 454550 (0.0010) [2023-12-26 18:40:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 232660992. Throughput: 0: 9690.0, 1: 9813.9. Samples: 232634040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:40:16,062][104569] Avg episode reward: [(0, '9358.440'), (1, '7991.243')] [2023-12-26 18:40:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000454184_116285440.pth... [2023-12-26 18:40:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000453064_115998720.pth [2023-12-26 18:40:16,106][105620] Updated weights for policy 1, policy_version 454560 (0.0010) [2023-12-26 18:40:16,154][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000454568_116383744.pth... [2023-12-26 18:40:16,159][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000453416_116088832.pth [2023-12-26 18:40:16,422][105692] Updated weights for policy 0, policy_version 454185 (0.0009) [2023-12-26 18:40:16,474][105692] Updated weights for policy 0, policy_version 454195 (0.0011) [2023-12-26 18:40:16,527][105692] Updated weights for policy 0, policy_version 454205 (0.0008) [2023-12-26 18:40:16,582][105692] Updated weights for policy 0, policy_version 454215 (0.0010) [2023-12-26 18:40:16,751][105620] Updated weights for policy 1, policy_version 454570 (0.0010) [2023-12-26 18:40:16,807][105620] Updated weights for policy 1, policy_version 454580 (0.0008) [2023-12-26 18:40:16,860][105620] Updated weights for policy 1, policy_version 454590 (0.0005) [2023-12-26 18:40:16,922][105620] Updated weights for policy 1, policy_version 454600 (0.0008) [2023-12-26 18:40:17,306][105692] Updated weights for policy 0, policy_version 454225 (0.0009) [2023-12-26 18:40:17,358][105692] Updated weights for policy 0, policy_version 454235 (0.0009) [2023-12-26 18:40:17,413][105692] Updated weights for policy 0, policy_version 454245 (0.0009) [2023-12-26 18:40:17,657][105620] Updated weights for policy 1, policy_version 454610 (0.0009) [2023-12-26 18:40:17,708][105620] Updated weights for policy 1, policy_version 454620 (0.0009) [2023-12-26 18:40:17,763][105620] Updated weights for policy 1, policy_version 454630 (0.0008) [2023-12-26 18:40:18,117][105692] Updated weights for policy 0, policy_version 454255 (0.0009) [2023-12-26 18:40:18,164][105692] Updated weights for policy 0, policy_version 454265 (0.0009) [2023-12-26 18:40:18,215][105692] Updated weights for policy 0, policy_version 454275 (0.0009) [2023-12-26 18:40:18,540][105620] Updated weights for policy 1, policy_version 454640 (0.0009) [2023-12-26 18:40:18,590][105620] Updated weights for policy 1, policy_version 454650 (0.0008) [2023-12-26 18:40:18,641][105620] Updated weights for policy 1, policy_version 454660 (0.0009) [2023-12-26 18:40:18,978][105692] Updated weights for policy 0, policy_version 454285 (0.0009) [2023-12-26 18:40:19,042][105692] Updated weights for policy 0, policy_version 454295 (0.0009) [2023-12-26 18:40:19,100][105692] Updated weights for policy 0, policy_version 454305 (0.0009) [2023-12-26 18:40:19,431][105620] Updated weights for policy 1, policy_version 454670 (0.0009) [2023-12-26 18:40:19,496][105620] Updated weights for policy 1, policy_version 454680 (0.0009) [2023-12-26 18:40:19,558][105620] Updated weights for policy 1, policy_version 454690 (0.0009) [2023-12-26 18:40:19,877][105692] Updated weights for policy 0, policy_version 454315 (0.0009) [2023-12-26 18:40:19,943][105692] Updated weights for policy 0, policy_version 454325 (0.0009) [2023-12-26 18:40:20,003][105692] Updated weights for policy 0, policy_version 454335 (0.0008) [2023-12-26 18:40:20,292][105620] Updated weights for policy 1, policy_version 454700 (0.0006) [2023-12-26 18:40:20,347][105620] Updated weights for policy 1, policy_version 454710 (0.0005) [2023-12-26 18:40:20,402][105620] Updated weights for policy 1, policy_version 454720 (0.0006) [2023-12-26 18:40:20,792][105692] Updated weights for policy 0, policy_version 454345 (0.0008) [2023-12-26 18:40:20,857][105692] Updated weights for policy 0, policy_version 454355 (0.0011) [2023-12-26 18:40:20,920][105692] Updated weights for policy 0, policy_version 454365 (0.0010) [2023-12-26 18:40:20,982][105692] Updated weights for policy 0, policy_version 454375 (0.0006) [2023-12-26 18:40:21,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 232759296. Throughput: 0: 9699.2, 1: 9727.4. Samples: 232749376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:40:21,063][104569] Avg episode reward: [(0, '9358.438'), (1, '8360.375')] [2023-12-26 18:40:21,168][105620] Updated weights for policy 1, policy_version 454730 (0.0009) [2023-12-26 18:40:21,228][105620] Updated weights for policy 1, policy_version 454740 (0.0008) [2023-12-26 18:40:21,288][105620] Updated weights for policy 1, policy_version 454750 (0.0008) [2023-12-26 18:40:21,357][105620] Updated weights for policy 1, policy_version 454760 (0.0008) [2023-12-26 18:40:21,672][105692] Updated weights for policy 0, policy_version 454385 (0.0009) [2023-12-26 18:40:21,744][105692] Updated weights for policy 0, policy_version 454395 (0.0010) [2023-12-26 18:40:21,807][105692] Updated weights for policy 0, policy_version 454405 (0.0010) [2023-12-26 18:40:22,238][105620] Updated weights for policy 1, policy_version 454770 (0.0009) [2023-12-26 18:40:22,304][105620] Updated weights for policy 1, policy_version 454780 (0.0009) [2023-12-26 18:40:22,367][105620] Updated weights for policy 1, policy_version 454790 (0.0008) [2023-12-26 18:40:22,392][105692] Updated weights for policy 0, policy_version 454415 (0.0009) [2023-12-26 18:40:22,448][105692] Updated weights for policy 0, policy_version 454425 (0.0010) [2023-12-26 18:40:22,493][105692] Updated weights for policy 0, policy_version 454435 (0.0010) [2023-12-26 18:40:23,133][105620] Updated weights for policy 1, policy_version 454800 (0.0008) [2023-12-26 18:40:23,189][105620] Updated weights for policy 1, policy_version 454810 (0.0008) [2023-12-26 18:40:23,255][105620] Updated weights for policy 1, policy_version 454820 (0.0009) [2023-12-26 18:40:23,284][105692] Updated weights for policy 0, policy_version 454445 (0.0010) [2023-12-26 18:40:23,342][105692] Updated weights for policy 0, policy_version 454455 (0.0010) [2023-12-26 18:40:23,400][105692] Updated weights for policy 0, policy_version 454465 (0.0010) [2023-12-26 18:40:23,997][105620] Updated weights for policy 1, policy_version 454830 (0.0007) [2023-12-26 18:40:24,056][105620] Updated weights for policy 1, policy_version 454840 (0.0008) [2023-12-26 18:40:24,086][105692] Updated weights for policy 0, policy_version 454475 (0.0010) [2023-12-26 18:40:24,120][105620] Updated weights for policy 1, policy_version 454850 (0.0006) [2023-12-26 18:40:24,135][105692] Updated weights for policy 0, policy_version 454485 (0.0010) [2023-12-26 18:40:24,151][105585] KL-divergence is very high: 102.3880 [2023-12-26 18:40:24,190][105692] Updated weights for policy 0, policy_version 454495 (0.0010) [2023-12-26 18:40:24,801][105692] Updated weights for policy 0, policy_version 454505 (0.0010) [2023-12-26 18:40:24,852][105692] Updated weights for policy 0, policy_version 454515 (0.0010) [2023-12-26 18:40:24,885][105620] Updated weights for policy 1, policy_version 454860 (0.0006) [2023-12-26 18:40:24,911][105692] Updated weights for policy 0, policy_version 454525 (0.0010) [2023-12-26 18:40:24,940][105620] Updated weights for policy 1, policy_version 454870 (0.0006) [2023-12-26 18:40:24,963][105692] Updated weights for policy 0, policy_version 454535 (0.0011) [2023-12-26 18:40:24,987][105620] Updated weights for policy 1, policy_version 454880 (0.0006) [2023-12-26 18:40:25,695][105692] Updated weights for policy 0, policy_version 454545 (0.0009) [2023-12-26 18:40:25,742][105620] Updated weights for policy 1, policy_version 454890 (0.0008) [2023-12-26 18:40:25,744][105692] Updated weights for policy 0, policy_version 454555 (0.0008) [2023-12-26 18:40:25,788][105620] Updated weights for policy 1, policy_version 454900 (0.0006) [2023-12-26 18:40:25,790][105692] Updated weights for policy 0, policy_version 454565 (0.0006) [2023-12-26 18:40:25,836][105620] Updated weights for policy 1, policy_version 454910 (0.0008) [2023-12-26 18:40:25,883][105620] Updated weights for policy 1, policy_version 454920 (0.0009) [2023-12-26 18:40:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 232857600. Throughput: 0: 9753.4, 1: 9557.3. Samples: 232862756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:40:26,062][104569] Avg episode reward: [(0, '9178.713'), (1, '8779.661')] [2023-12-26 18:40:26,431][105692] Updated weights for policy 0, policy_version 454575 (0.0005) [2023-12-26 18:40:26,478][105692] Updated weights for policy 0, policy_version 454585 (0.0005) [2023-12-26 18:40:26,524][105692] Updated weights for policy 0, policy_version 454595 (0.0007) [2023-12-26 18:40:26,693][105620] Updated weights for policy 1, policy_version 454930 (0.0005) [2023-12-26 18:40:26,751][105620] Updated weights for policy 1, policy_version 454940 (0.0008) [2023-12-26 18:40:26,809][105620] Updated weights for policy 1, policy_version 454950 (0.0009) [2023-12-26 18:40:27,279][105692] Updated weights for policy 0, policy_version 454606 (0.0009) [2023-12-26 18:40:27,345][105692] Updated weights for policy 0, policy_version 454616 (0.0006) [2023-12-26 18:40:27,399][105692] Updated weights for policy 0, policy_version 454626 (0.0006) [2023-12-26 18:40:27,448][105620] Updated weights for policy 1, policy_version 454960 (0.0007) [2023-12-26 18:40:27,499][105620] Updated weights for policy 1, policy_version 454970 (0.0005) [2023-12-26 18:40:27,561][105620] Updated weights for policy 1, policy_version 454980 (0.0006) [2023-12-26 18:40:28,144][105692] Updated weights for policy 0, policy_version 454636 (0.0009) [2023-12-26 18:40:28,177][105620] Updated weights for policy 1, policy_version 454990 (0.0008) [2023-12-26 18:40:28,192][105692] Updated weights for policy 0, policy_version 454646 (0.0006) [2023-12-26 18:40:28,222][105620] Updated weights for policy 1, policy_version 455000 (0.0006) [2023-12-26 18:40:28,240][105692] Updated weights for policy 0, policy_version 454656 (0.0006) [2023-12-26 18:40:28,267][105620] Updated weights for policy 1, policy_version 455010 (0.0006) [2023-12-26 18:40:28,997][105620] Updated weights for policy 1, policy_version 455020 (0.0007) [2023-12-26 18:40:29,025][105692] Updated weights for policy 0, policy_version 454666 (0.0008) [2023-12-26 18:40:29,047][105620] Updated weights for policy 1, policy_version 455030 (0.0007) [2023-12-26 18:40:29,081][105692] Updated weights for policy 0, policy_version 454676 (0.0007) [2023-12-26 18:40:29,092][105620] Updated weights for policy 1, policy_version 455040 (0.0005) [2023-12-26 18:40:29,134][105692] Updated weights for policy 0, policy_version 454686 (0.0007) [2023-12-26 18:40:29,181][105692] Updated weights for policy 0, policy_version 454696 (0.0009) [2023-12-26 18:40:29,864][105620] Updated weights for policy 1, policy_version 455050 (0.0007) [2023-12-26 18:40:29,928][105620] Updated weights for policy 1, policy_version 455060 (0.0010) [2023-12-26 18:40:29,951][105692] Updated weights for policy 0, policy_version 454706 (0.0008) [2023-12-26 18:40:29,977][105620] Updated weights for policy 1, policy_version 455070 (0.0007) [2023-12-26 18:40:30,007][105692] Updated weights for policy 0, policy_version 454716 (0.0007) [2023-12-26 18:40:30,030][105620] Updated weights for policy 1, policy_version 455080 (0.0007) [2023-12-26 18:40:30,056][105692] Updated weights for policy 0, policy_version 454726 (0.0007) [2023-12-26 18:40:30,765][105620] Updated weights for policy 1, policy_version 455090 (0.0006) [2023-12-26 18:40:30,800][105692] Updated weights for policy 0, policy_version 454736 (0.0007) [2023-12-26 18:40:30,813][105620] Updated weights for policy 1, policy_version 455100 (0.0006) [2023-12-26 18:40:30,847][105692] Updated weights for policy 0, policy_version 454746 (0.0006) [2023-12-26 18:40:30,865][105620] Updated weights for policy 1, policy_version 455110 (0.0007) [2023-12-26 18:40:30,893][105692] Updated weights for policy 0, policy_version 454756 (0.0006) [2023-12-26 18:40:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 232955904. Throughput: 0: 9772.4, 1: 9598.1. Samples: 232922872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:40:31,062][104569] Avg episode reward: [(0, '9178.704'), (1, '8598.303')] [2023-12-26 18:40:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000455112_116523008.pth... [2023-12-26 18:40:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000454760_116432896.pth... [2023-12-26 18:40:31,069][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000453992_116236288.pth [2023-12-26 18:40:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000453608_116137984.pth [2023-12-26 18:40:31,587][105620] Updated weights for policy 1, policy_version 455120 (0.0008) [2023-12-26 18:40:31,656][105620] Updated weights for policy 1, policy_version 455130 (0.0008) [2023-12-26 18:40:31,708][105692] Updated weights for policy 0, policy_version 454766 (0.0007) [2023-12-26 18:40:31,720][105620] Updated weights for policy 1, policy_version 455140 (0.0008) [2023-12-26 18:40:31,772][105692] Updated weights for policy 0, policy_version 454776 (0.0007) [2023-12-26 18:40:31,836][105692] Updated weights for policy 0, policy_version 454786 (0.0008) [2023-12-26 18:40:32,505][105620] Updated weights for policy 1, policy_version 455150 (0.0008) [2023-12-26 18:40:32,564][105620] Updated weights for policy 1, policy_version 455160 (0.0010) [2023-12-26 18:40:32,578][105692] Updated weights for policy 0, policy_version 454796 (0.0008) [2023-12-26 18:40:32,622][105620] Updated weights for policy 1, policy_version 455170 (0.0006) [2023-12-26 18:40:32,637][105692] Updated weights for policy 0, policy_version 454806 (0.0007) [2023-12-26 18:40:32,693][105692] Updated weights for policy 0, policy_version 454816 (0.0006) [2023-12-26 18:40:33,288][105620] Updated weights for policy 1, policy_version 455180 (0.0008) [2023-12-26 18:40:33,332][105620] Updated weights for policy 1, policy_version 455190 (0.0006) [2023-12-26 18:40:33,378][105620] Updated weights for policy 1, policy_version 455200 (0.0005) [2023-12-26 18:40:33,466][105692] Updated weights for policy 0, policy_version 454826 (0.0007) [2023-12-26 18:40:33,519][105692] Updated weights for policy 0, policy_version 454836 (0.0010) [2023-12-26 18:40:33,577][105692] Updated weights for policy 0, policy_version 454847 (0.0010) [2023-12-26 18:40:33,958][105620] Updated weights for policy 1, policy_version 455210 (0.0006) [2023-12-26 18:40:34,004][105620] Updated weights for policy 1, policy_version 455220 (0.0005) [2023-12-26 18:40:34,062][105620] Updated weights for policy 1, policy_version 455230 (0.0007) [2023-12-26 18:40:34,126][105620] Updated weights for policy 1, policy_version 455240 (0.0007) [2023-12-26 18:40:34,469][105692] Updated weights for policy 0, policy_version 454857 (0.0010) [2023-12-26 18:40:34,526][105692] Updated weights for policy 0, policy_version 454867 (0.0010) [2023-12-26 18:40:34,590][105692] Updated weights for policy 0, policy_version 454877 (0.0008) [2023-12-26 18:40:34,650][105692] Updated weights for policy 0, policy_version 454887 (0.0008) [2023-12-26 18:40:34,702][105620] Updated weights for policy 1, policy_version 455250 (0.0010) [2023-12-26 18:40:34,753][105620] Updated weights for policy 1, policy_version 455260 (0.0005) [2023-12-26 18:40:34,810][105620] Updated weights for policy 1, policy_version 455270 (0.0007) [2023-12-26 18:40:35,459][105620] Updated weights for policy 1, policy_version 455280 (0.0007) [2023-12-26 18:40:35,461][105692] Updated weights for policy 0, policy_version 454897 (0.0007) [2023-12-26 18:40:35,521][105620] Updated weights for policy 1, policy_version 455290 (0.0011) [2023-12-26 18:40:35,524][105692] Updated weights for policy 0, policy_version 454907 (0.0006) [2023-12-26 18:40:35,581][105692] Updated weights for policy 0, policy_version 454917 (0.0005) [2023-12-26 18:40:35,583][105620] Updated weights for policy 1, policy_version 455300 (0.0011) [2023-12-26 18:40:36,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19251.1, 300 sec: 19549.7). Total num frames: 233046016. Throughput: 0: 9557.8, 1: 9741.3. Samples: 233037208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:40:36,063][104569] Avg episode reward: [(0, '9358.352'), (1, '8988.870')] [2023-12-26 18:40:36,268][105692] Updated weights for policy 0, policy_version 454927 (0.0008) [2023-12-26 18:40:36,321][105620] Updated weights for policy 1, policy_version 455310 (0.0010) [2023-12-26 18:40:36,335][105692] Updated weights for policy 0, policy_version 454937 (0.0008) [2023-12-26 18:40:36,382][105620] Updated weights for policy 1, policy_version 455320 (0.0007) [2023-12-26 18:40:36,393][105692] Updated weights for policy 0, policy_version 454947 (0.0006) [2023-12-26 18:40:36,438][105620] Updated weights for policy 1, policy_version 455330 (0.0008) [2023-12-26 18:40:37,127][105692] Updated weights for policy 0, policy_version 454957 (0.0007) [2023-12-26 18:40:37,178][105692] Updated weights for policy 0, policy_version 454967 (0.0008) [2023-12-26 18:40:37,180][105620] Updated weights for policy 1, policy_version 455340 (0.0008) [2023-12-26 18:40:37,230][105692] Updated weights for policy 0, policy_version 454977 (0.0006) [2023-12-26 18:40:37,243][105620] Updated weights for policy 1, policy_version 455350 (0.0009) [2023-12-26 18:40:37,310][105620] Updated weights for policy 1, policy_version 455360 (0.0009) [2023-12-26 18:40:37,994][105692] Updated weights for policy 0, policy_version 454987 (0.0008) [2023-12-26 18:40:38,001][105620] Updated weights for policy 1, policy_version 455370 (0.0009) [2023-12-26 18:40:38,055][105692] Updated weights for policy 0, policy_version 454997 (0.0007) [2023-12-26 18:40:38,061][105620] Updated weights for policy 1, policy_version 455380 (0.0007) [2023-12-26 18:40:38,104][105692] Updated weights for policy 0, policy_version 455007 (0.0007) [2023-12-26 18:40:38,121][105620] Updated weights for policy 1, policy_version 455390 (0.0008) [2023-12-26 18:40:38,170][105620] Updated weights for policy 1, policy_version 455400 (0.0009) [2023-12-26 18:40:38,703][105692] Updated weights for policy 0, policy_version 455017 (0.0005) [2023-12-26 18:40:38,767][105692] Updated weights for policy 0, policy_version 455027 (0.0009) [2023-12-26 18:40:38,826][105692] Updated weights for policy 0, policy_version 455037 (0.0009) [2023-12-26 18:40:38,878][105692] Updated weights for policy 0, policy_version 455047 (0.0009) [2023-12-26 18:40:39,002][105620] Updated weights for policy 1, policy_version 455410 (0.0009) [2023-12-26 18:40:39,058][105620] Updated weights for policy 1, policy_version 455420 (0.0008) [2023-12-26 18:40:39,119][105620] Updated weights for policy 1, policy_version 455430 (0.0009) [2023-12-26 18:40:39,624][105692] Updated weights for policy 0, policy_version 455057 (0.0010) [2023-12-26 18:40:39,678][105692] Updated weights for policy 0, policy_version 455067 (0.0008) [2023-12-26 18:40:39,743][105692] Updated weights for policy 0, policy_version 455077 (0.0006) [2023-12-26 18:40:39,837][105620] Updated weights for policy 1, policy_version 455440 (0.0008) [2023-12-26 18:40:39,893][105620] Updated weights for policy 1, policy_version 455450 (0.0008) [2023-12-26 18:40:39,954][105620] Updated weights for policy 1, policy_version 455460 (0.0009) [2023-12-26 18:40:40,371][105692] Updated weights for policy 0, policy_version 455087 (0.0006) [2023-12-26 18:40:40,431][105692] Updated weights for policy 0, policy_version 455097 (0.0006) [2023-12-26 18:40:40,488][105692] Updated weights for policy 0, policy_version 455107 (0.0006) [2023-12-26 18:40:40,779][105620] Updated weights for policy 1, policy_version 455470 (0.0006) [2023-12-26 18:40:40,845][105620] Updated weights for policy 1, policy_version 455480 (0.0009) [2023-12-26 18:40:40,909][105620] Updated weights for policy 1, policy_version 455490 (0.0009) [2023-12-26 18:40:41,062][105692] Updated weights for policy 0, policy_version 455117 (0.0006) [2023-12-26 18:40:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 233144320. Throughput: 0: 9612.1, 1: 9675.0. Samples: 233152336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:40:41,062][104569] Avg episode reward: [(0, '9358.245'), (1, '9080.232')] [2023-12-26 18:40:41,116][105692] Updated weights for policy 0, policy_version 455127 (0.0008) [2023-12-26 18:40:41,180][105692] Updated weights for policy 0, policy_version 455137 (0.0007) [2023-12-26 18:40:41,581][105620] Updated weights for policy 1, policy_version 455500 (0.0005) [2023-12-26 18:40:41,640][105620] Updated weights for policy 1, policy_version 455510 (0.0008) [2023-12-26 18:40:41,707][105620] Updated weights for policy 1, policy_version 455520 (0.0009) [2023-12-26 18:40:42,001][105692] Updated weights for policy 0, policy_version 455147 (0.0007) [2023-12-26 18:40:42,061][105692] Updated weights for policy 0, policy_version 455157 (0.0009) [2023-12-26 18:40:42,117][105692] Updated weights for policy 0, policy_version 455167 (0.0009) [2023-12-26 18:40:42,362][105620] Updated weights for policy 1, policy_version 455530 (0.0009) [2023-12-26 18:40:42,421][105620] Updated weights for policy 1, policy_version 455540 (0.0009) [2023-12-26 18:40:42,470][105620] Updated weights for policy 1, policy_version 455550 (0.0009) [2023-12-26 18:40:42,521][105620] Updated weights for policy 1, policy_version 455560 (0.0009) [2023-12-26 18:40:42,900][105692] Updated weights for policy 0, policy_version 455177 (0.0009) [2023-12-26 18:40:42,956][105692] Updated weights for policy 0, policy_version 455187 (0.0009) [2023-12-26 18:40:43,011][105692] Updated weights for policy 0, policy_version 455197 (0.0009) [2023-12-26 18:40:43,077][105692] Updated weights for policy 0, policy_version 455207 (0.0009) [2023-12-26 18:40:43,286][105620] Updated weights for policy 1, policy_version 455570 (0.0009) [2023-12-26 18:40:43,346][105620] Updated weights for policy 1, policy_version 455580 (0.0008) [2023-12-26 18:40:43,404][105620] Updated weights for policy 1, policy_version 455590 (0.0008) [2023-12-26 18:40:43,769][105692] Updated weights for policy 0, policy_version 455217 (0.0006) [2023-12-26 18:40:43,826][105692] Updated weights for policy 0, policy_version 455227 (0.0005) [2023-12-26 18:40:43,881][105692] Updated weights for policy 0, policy_version 455237 (0.0005) [2023-12-26 18:40:44,264][105620] Updated weights for policy 1, policy_version 455600 (0.0009) [2023-12-26 18:40:44,318][105620] Updated weights for policy 1, policy_version 455610 (0.0010) [2023-12-26 18:40:44,378][105620] Updated weights for policy 1, policy_version 455621 (0.0011) [2023-12-26 18:40:44,413][105692] Updated weights for policy 0, policy_version 455247 (0.0005) [2023-12-26 18:40:44,463][105692] Updated weights for policy 0, policy_version 455257 (0.0006) [2023-12-26 18:40:44,507][105692] Updated weights for policy 0, policy_version 455267 (0.0005) [2023-12-26 18:40:45,093][105692] Updated weights for policy 0, policy_version 455277 (0.0008) [2023-12-26 18:40:45,145][105692] Updated weights for policy 0, policy_version 455287 (0.0011) [2023-12-26 18:40:45,202][105692] Updated weights for policy 0, policy_version 455297 (0.0011) [2023-12-26 18:40:45,209][105620] Updated weights for policy 1, policy_version 455631 (0.0007) [2023-12-26 18:40:45,266][105620] Updated weights for policy 1, policy_version 455641 (0.0007) [2023-12-26 18:40:45,334][105620] Updated weights for policy 1, policy_version 455651 (0.0009) [2023-12-26 18:40:45,899][105692] Updated weights for policy 0, policy_version 455307 (0.0011) [2023-12-26 18:40:45,963][105692] Updated weights for policy 0, policy_version 455317 (0.0010) [2023-12-26 18:40:46,028][105692] Updated weights for policy 0, policy_version 455327 (0.0010) [2023-12-26 18:40:46,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 233234432. Throughput: 0: 9643.3, 1: 9607.5. Samples: 233208992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:40:46,063][104569] Avg episode reward: [(0, '9358.131'), (1, '7595.114')] [2023-12-26 18:40:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000455336_116580352.pth... [2023-12-26 18:40:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000455656_116662272.pth... [2023-12-26 18:40:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000454184_116285440.pth [2023-12-26 18:40:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000454568_116383744.pth [2023-12-26 18:40:46,124][105620] Updated weights for policy 1, policy_version 455661 (0.0008) [2023-12-26 18:40:46,180][105620] Updated weights for policy 1, policy_version 455671 (0.0008) [2023-12-26 18:40:46,233][105620] Updated weights for policy 1, policy_version 455681 (0.0008) [2023-12-26 18:40:46,759][105692] Updated weights for policy 0, policy_version 455337 (0.0010) [2023-12-26 18:40:46,810][105692] Updated weights for policy 0, policy_version 455347 (0.0010) [2023-12-26 18:40:46,860][105692] Updated weights for policy 0, policy_version 455357 (0.0010) [2023-12-26 18:40:46,915][105692] Updated weights for policy 0, policy_version 455367 (0.0010) [2023-12-26 18:40:46,993][105620] Updated weights for policy 1, policy_version 455691 (0.0007) [2023-12-26 18:40:47,059][105620] Updated weights for policy 1, policy_version 455701 (0.0008) [2023-12-26 18:40:47,126][105620] Updated weights for policy 1, policy_version 455711 (0.0008) [2023-12-26 18:40:47,657][105692] Updated weights for policy 0, policy_version 455377 (0.0009) [2023-12-26 18:40:47,718][105692] Updated weights for policy 0, policy_version 455387 (0.0009) [2023-12-26 18:40:47,774][105692] Updated weights for policy 0, policy_version 455397 (0.0007) [2023-12-26 18:40:47,876][105620] Updated weights for policy 1, policy_version 455721 (0.0008) [2023-12-26 18:40:47,938][105620] Updated weights for policy 1, policy_version 455731 (0.0010) [2023-12-26 18:40:47,994][105620] Updated weights for policy 1, policy_version 455741 (0.0009) [2023-12-26 18:40:48,056][105620] Updated weights for policy 1, policy_version 455751 (0.0008) [2023-12-26 18:40:48,457][105692] Updated weights for policy 0, policy_version 455407 (0.0006) [2023-12-26 18:40:48,511][105692] Updated weights for policy 0, policy_version 455417 (0.0007) [2023-12-26 18:40:48,566][105692] Updated weights for policy 0, policy_version 455427 (0.0009) [2023-12-26 18:40:48,845][105620] Updated weights for policy 1, policy_version 455761 (0.0009) [2023-12-26 18:40:48,900][105620] Updated weights for policy 1, policy_version 455771 (0.0009) [2023-12-26 18:40:48,961][105620] Updated weights for policy 1, policy_version 455781 (0.0009) [2023-12-26 18:40:49,299][105692] Updated weights for policy 0, policy_version 455437 (0.0009) [2023-12-26 18:40:49,363][105692] Updated weights for policy 0, policy_version 455447 (0.0009) [2023-12-26 18:40:49,432][105692] Updated weights for policy 0, policy_version 455457 (0.0009) [2023-12-26 18:40:49,723][105620] Updated weights for policy 1, policy_version 455791 (0.0009) [2023-12-26 18:40:49,779][105620] Updated weights for policy 1, policy_version 455801 (0.0009) [2023-12-26 18:40:49,837][105620] Updated weights for policy 1, policy_version 455811 (0.0008) [2023-12-26 18:40:50,194][105692] Updated weights for policy 0, policy_version 455467 (0.0009) [2023-12-26 18:40:50,249][105692] Updated weights for policy 0, policy_version 455478 (0.0009) [2023-12-26 18:40:50,300][105692] Updated weights for policy 0, policy_version 455488 (0.0009) [2023-12-26 18:40:50,501][105620] Updated weights for policy 1, policy_version 455821 (0.0008) [2023-12-26 18:40:50,561][105620] Updated weights for policy 1, policy_version 455831 (0.0009) [2023-12-26 18:40:50,622][105620] Updated weights for policy 1, policy_version 455841 (0.0008) [2023-12-26 18:40:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19549.7). Total num frames: 233332736. Throughput: 0: 9748.5, 1: 9488.2. Samples: 233323784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:40:51,062][104569] Avg episode reward: [(0, '9358.054'), (1, '3833.906')] [2023-12-26 18:40:51,127][105692] Updated weights for policy 0, policy_version 455498 (0.0009) [2023-12-26 18:40:51,189][105692] Updated weights for policy 0, policy_version 455508 (0.0008) [2023-12-26 18:40:51,252][105692] Updated weights for policy 0, policy_version 455518 (0.0009) [2023-12-26 18:40:51,308][105692] Updated weights for policy 0, policy_version 455528 (0.0009) [2023-12-26 18:40:51,349][105620] Updated weights for policy 1, policy_version 455851 (0.0009) [2023-12-26 18:40:51,409][105620] Updated weights for policy 1, policy_version 455861 (0.0010) [2023-12-26 18:40:51,467][105620] Updated weights for policy 1, policy_version 455871 (0.0008) [2023-12-26 18:40:52,098][105692] Updated weights for policy 0, policy_version 455538 (0.0010) [2023-12-26 18:40:52,152][105620] Updated weights for policy 1, policy_version 455881 (0.0008) [2023-12-26 18:40:52,158][105692] Updated weights for policy 0, policy_version 455548 (0.0011) [2023-12-26 18:40:52,208][105620] Updated weights for policy 1, policy_version 455891 (0.0005) [2023-12-26 18:40:52,210][105692] Updated weights for policy 0, policy_version 455558 (0.0011) [2023-12-26 18:40:52,268][105620] Updated weights for policy 1, policy_version 455901 (0.0008) [2023-12-26 18:40:52,331][105620] Updated weights for policy 1, policy_version 455911 (0.0008) [2023-12-26 18:40:52,982][105692] Updated weights for policy 0, policy_version 455568 (0.0010) [2023-12-26 18:40:53,044][105692] Updated weights for policy 0, policy_version 455578 (0.0010) [2023-12-26 18:40:53,107][105620] Updated weights for policy 1, policy_version 455921 (0.0006) [2023-12-26 18:40:53,109][105692] Updated weights for policy 0, policy_version 455588 (0.0010) [2023-12-26 18:40:53,160][105620] Updated weights for policy 1, policy_version 455931 (0.0007) [2023-12-26 18:40:53,204][105620] Updated weights for policy 1, policy_version 455941 (0.0008) [2023-12-26 18:40:53,838][105692] Updated weights for policy 0, policy_version 455598 (0.0010) [2023-12-26 18:40:53,895][105692] Updated weights for policy 0, policy_version 455608 (0.0010) [2023-12-26 18:40:53,943][105692] Updated weights for policy 0, policy_version 455618 (0.0010) [2023-12-26 18:40:53,969][105620] Updated weights for policy 1, policy_version 455951 (0.0007) [2023-12-26 18:40:54,025][105620] Updated weights for policy 1, policy_version 455961 (0.0008) [2023-12-26 18:40:54,087][105620] Updated weights for policy 1, policy_version 455971 (0.0008) [2023-12-26 18:40:54,695][105692] Updated weights for policy 0, policy_version 455628 (0.0010) [2023-12-26 18:40:54,743][105692] Updated weights for policy 0, policy_version 455638 (0.0010) [2023-12-26 18:40:54,797][105692] Updated weights for policy 0, policy_version 455648 (0.0010) [2023-12-26 18:40:54,842][105620] Updated weights for policy 1, policy_version 455981 (0.0007) [2023-12-26 18:40:54,893][105620] Updated weights for policy 1, policy_version 455991 (0.0008) [2023-12-26 18:40:54,952][105620] Updated weights for policy 1, policy_version 456001 (0.0008) [2023-12-26 18:40:55,463][105692] Updated weights for policy 0, policy_version 455658 (0.0010) [2023-12-26 18:40:55,509][105692] Updated weights for policy 0, policy_version 455668 (0.0009) [2023-12-26 18:40:55,565][105692] Updated weights for policy 0, policy_version 455678 (0.0009) [2023-12-26 18:40:55,610][105692] Updated weights for policy 0, policy_version 455688 (0.0006) [2023-12-26 18:40:55,641][105620] Updated weights for policy 1, policy_version 456011 (0.0009) [2023-12-26 18:40:55,697][105620] Updated weights for policy 1, policy_version 456021 (0.0009) [2023-12-26 18:40:55,747][105620] Updated weights for policy 1, policy_version 456031 (0.0009) [2023-12-26 18:40:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19114.6, 300 sec: 19522.0). Total num frames: 233431040. Throughput: 0: 9660.2, 1: 9439.9. Samples: 233437972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:40:56,062][104569] Avg episode reward: [(0, '9357.768'), (1, '7155.959')] [2023-12-26 18:40:56,385][105620] Updated weights for policy 1, policy_version 456041 (0.0008) [2023-12-26 18:40:56,394][105692] Updated weights for policy 0, policy_version 455698 (0.0011) [2023-12-26 18:40:56,444][105620] Updated weights for policy 1, policy_version 456051 (0.0006) [2023-12-26 18:40:56,456][105692] Updated weights for policy 0, policy_version 455708 (0.0010) [2023-12-26 18:40:56,503][105620] Updated weights for policy 1, policy_version 456061 (0.0007) [2023-12-26 18:40:56,520][105692] Updated weights for policy 0, policy_version 455718 (0.0010) [2023-12-26 18:40:56,549][105620] Updated weights for policy 1, policy_version 456071 (0.0006) [2023-12-26 18:40:57,206][105692] Updated weights for policy 0, policy_version 455728 (0.0010) [2023-12-26 18:40:57,220][105620] Updated weights for policy 1, policy_version 456081 (0.0006) [2023-12-26 18:40:57,259][105692] Updated weights for policy 0, policy_version 455738 (0.0010) [2023-12-26 18:40:57,272][105620] Updated weights for policy 1, policy_version 456091 (0.0008) [2023-12-26 18:40:57,317][105692] Updated weights for policy 0, policy_version 455748 (0.0010) [2023-12-26 18:40:57,332][105620] Updated weights for policy 1, policy_version 456101 (0.0006) [2023-12-26 18:40:57,961][105692] Updated weights for policy 0, policy_version 455758 (0.0008) [2023-12-26 18:40:58,014][105620] Updated weights for policy 1, policy_version 456111 (0.0008) [2023-12-26 18:40:58,019][105692] Updated weights for policy 0, policy_version 455768 (0.0010) [2023-12-26 18:40:58,073][105620] Updated weights for policy 1, policy_version 456121 (0.0006) [2023-12-26 18:40:58,078][105692] Updated weights for policy 0, policy_version 455778 (0.0010) [2023-12-26 18:40:58,133][105620] Updated weights for policy 1, policy_version 456131 (0.0008) [2023-12-26 18:40:58,864][105692] Updated weights for policy 0, policy_version 455788 (0.0008) [2023-12-26 18:40:58,927][105692] Updated weights for policy 0, policy_version 455798 (0.0008) [2023-12-26 18:40:58,992][105692] Updated weights for policy 0, policy_version 455808 (0.0008) [2023-12-26 18:40:59,018][105620] Updated weights for policy 1, policy_version 456141 (0.0009) [2023-12-26 18:40:59,086][105620] Updated weights for policy 1, policy_version 456151 (0.0007) [2023-12-26 18:40:59,137][105620] Updated weights for policy 1, policy_version 456161 (0.0008) [2023-12-26 18:40:59,756][105620] Updated weights for policy 1, policy_version 456171 (0.0007) [2023-12-26 18:40:59,822][105620] Updated weights for policy 1, policy_version 456181 (0.0006) [2023-12-26 18:40:59,839][105692] Updated weights for policy 0, policy_version 455818 (0.0007) [2023-12-26 18:40:59,886][105620] Updated weights for policy 1, policy_version 456191 (0.0009) [2023-12-26 18:40:59,897][105692] Updated weights for policy 0, policy_version 455828 (0.0006) [2023-12-26 18:40:59,962][105692] Updated weights for policy 0, policy_version 455838 (0.0008) [2023-12-26 18:41:00,019][105692] Updated weights for policy 0, policy_version 455848 (0.0009) [2023-12-26 18:41:00,475][105620] Updated weights for policy 1, policy_version 456201 (0.0007) [2023-12-26 18:41:00,533][105620] Updated weights for policy 1, policy_version 456211 (0.0009) [2023-12-26 18:41:00,576][105620] Updated weights for policy 1, policy_version 456221 (0.0005) [2023-12-26 18:41:00,630][105620] Updated weights for policy 1, policy_version 456231 (0.0008) [2023-12-26 18:41:00,799][105692] Updated weights for policy 0, policy_version 455858 (0.0008) [2023-12-26 18:41:00,857][105692] Updated weights for policy 0, policy_version 455868 (0.0008) [2023-12-26 18:41:00,911][105692] Updated weights for policy 0, policy_version 455878 (0.0008) [2023-12-26 18:41:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 233529344. Throughput: 0: 9710.5, 1: 9475.5. Samples: 233497412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:41:01,063][104569] Avg episode reward: [(0, '9357.545'), (1, '8817.675')] [2023-12-26 18:41:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000455880_116719616.pth... [2023-12-26 18:41:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000456232_116809728.pth... [2023-12-26 18:41:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000454760_116432896.pth [2023-12-26 18:41:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000455112_116523008.pth [2023-12-26 18:41:01,405][105620] Updated weights for policy 1, policy_version 456241 (0.0010) [2023-12-26 18:41:01,459][105620] Updated weights for policy 1, policy_version 456251 (0.0006) [2023-12-26 18:41:01,509][105620] Updated weights for policy 1, policy_version 456261 (0.0005) [2023-12-26 18:41:01,575][105692] Updated weights for policy 0, policy_version 455888 (0.0006) [2023-12-26 18:41:01,639][105692] Updated weights for policy 0, policy_version 455898 (0.0007) [2023-12-26 18:41:01,693][105692] Updated weights for policy 0, policy_version 455908 (0.0008) [2023-12-26 18:41:02,215][105620] Updated weights for policy 1, policy_version 456271 (0.0009) [2023-12-26 18:41:02,277][105620] Updated weights for policy 1, policy_version 456281 (0.0011) [2023-12-26 18:41:02,323][105620] Updated weights for policy 1, policy_version 456291 (0.0011) [2023-12-26 18:41:02,417][105692] Updated weights for policy 0, policy_version 455918 (0.0008) [2023-12-26 18:41:02,467][105692] Updated weights for policy 0, policy_version 455928 (0.0009) [2023-12-26 18:41:02,527][105692] Updated weights for policy 0, policy_version 455938 (0.0011) [2023-12-26 18:41:03,034][105620] Updated weights for policy 1, policy_version 456301 (0.0011) [2023-12-26 18:41:03,095][105620] Updated weights for policy 1, policy_version 456311 (0.0010) [2023-12-26 18:41:03,152][105620] Updated weights for policy 1, policy_version 456321 (0.0011) [2023-12-26 18:41:03,288][105692] Updated weights for policy 0, policy_version 455948 (0.0011) [2023-12-26 18:41:03,336][105692] Updated weights for policy 0, policy_version 455958 (0.0010) [2023-12-26 18:41:03,387][105692] Updated weights for policy 0, policy_version 455968 (0.0010) [2023-12-26 18:41:03,891][105620] Updated weights for policy 1, policy_version 456331 (0.0010) [2023-12-26 18:41:03,948][105620] Updated weights for policy 1, policy_version 456341 (0.0011) [2023-12-26 18:41:04,008][105620] Updated weights for policy 1, policy_version 456351 (0.0011) [2023-12-26 18:41:04,081][105692] Updated weights for policy 0, policy_version 455978 (0.0010) [2023-12-26 18:41:04,174][105692] Updated weights for policy 0, policy_version 455988 (0.0006) [2023-12-26 18:41:04,234][105692] Updated weights for policy 0, policy_version 455998 (0.0009) [2023-12-26 18:41:04,294][105692] Updated weights for policy 0, policy_version 456008 (0.0011) [2023-12-26 18:41:04,771][105620] Updated weights for policy 1, policy_version 456361 (0.0010) [2023-12-26 18:41:04,833][105620] Updated weights for policy 1, policy_version 456371 (0.0006) [2023-12-26 18:41:04,888][105620] Updated weights for policy 1, policy_version 456381 (0.0006) [2023-12-26 18:41:04,948][105620] Updated weights for policy 1, policy_version 456391 (0.0006) [2023-12-26 18:41:04,962][105692] Updated weights for policy 0, policy_version 456018 (0.0005) [2023-12-26 18:41:05,023][105692] Updated weights for policy 0, policy_version 456028 (0.0010) [2023-12-26 18:41:05,071][105692] Updated weights for policy 0, policy_version 456038 (0.0010) [2023-12-26 18:41:05,639][105620] Updated weights for policy 1, policy_version 456401 (0.0010) [2023-12-26 18:41:05,688][105620] Updated weights for policy 1, policy_version 456411 (0.0010) [2023-12-26 18:41:05,690][105692] Updated weights for policy 0, policy_version 456048 (0.0006) [2023-12-26 18:41:05,734][105692] Updated weights for policy 0, policy_version 456058 (0.0005) [2023-12-26 18:41:05,743][105620] Updated weights for policy 1, policy_version 456421 (0.0010) [2023-12-26 18:41:05,779][105692] Updated weights for policy 0, policy_version 456068 (0.0005) [2023-12-26 18:41:06,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19251.1, 300 sec: 19521.9). Total num frames: 233627648. Throughput: 0: 9611.4, 1: 9559.4. Samples: 233612064. Policy #0 lag: (min: 31.0, avg: 31.2, max: 43.0) [2023-12-26 18:41:06,063][104569] Avg episode reward: [(0, '9265.459'), (1, '8511.135')] [2023-12-26 18:41:06,385][105692] Updated weights for policy 0, policy_version 456078 (0.0008) [2023-12-26 18:41:06,438][105692] Updated weights for policy 0, policy_version 456088 (0.0011) [2023-12-26 18:41:06,449][105620] Updated weights for policy 1, policy_version 456431 (0.0007) [2023-12-26 18:41:06,498][105692] Updated weights for policy 0, policy_version 456098 (0.0011) [2023-12-26 18:41:06,511][105620] Updated weights for policy 1, policy_version 456441 (0.0006) [2023-12-26 18:41:06,578][105620] Updated weights for policy 1, policy_version 456451 (0.0010) [2023-12-26 18:41:07,144][105692] Updated weights for policy 0, policy_version 456108 (0.0010) [2023-12-26 18:41:07,204][105692] Updated weights for policy 0, policy_version 456118 (0.0007) [2023-12-26 18:41:07,232][105620] Updated weights for policy 1, policy_version 456461 (0.0011) [2023-12-26 18:41:07,268][105692] Updated weights for policy 0, policy_version 456128 (0.0010) [2023-12-26 18:41:07,284][105620] Updated weights for policy 1, policy_version 456471 (0.0011) [2023-12-26 18:41:07,337][105620] Updated weights for policy 1, policy_version 456481 (0.0010) [2023-12-26 18:41:07,881][105692] Updated weights for policy 0, policy_version 456138 (0.0008) [2023-12-26 18:41:07,954][105692] Updated weights for policy 0, policy_version 456148 (0.0007) [2023-12-26 18:41:07,993][105620] Updated weights for policy 1, policy_version 456491 (0.0009) [2023-12-26 18:41:08,013][105692] Updated weights for policy 0, policy_version 456158 (0.0006) [2023-12-26 18:41:08,051][105620] Updated weights for policy 1, policy_version 456501 (0.0011) [2023-12-26 18:41:08,067][105692] Updated weights for policy 0, policy_version 456168 (0.0005) [2023-12-26 18:41:08,106][105620] Updated weights for policy 1, policy_version 456511 (0.0010) [2023-12-26 18:41:08,670][105692] Updated weights for policy 0, policy_version 456178 (0.0008) [2023-12-26 18:41:08,735][105692] Updated weights for policy 0, policy_version 456188 (0.0008) [2023-12-26 18:41:08,784][105692] Updated weights for policy 0, policy_version 456198 (0.0008) [2023-12-26 18:41:08,838][105620] Updated weights for policy 1, policy_version 456521 (0.0011) [2023-12-26 18:41:08,904][105620] Updated weights for policy 1, policy_version 456531 (0.0011) [2023-12-26 18:41:08,973][105620] Updated weights for policy 1, policy_version 456541 (0.0010) [2023-12-26 18:41:09,036][105620] Updated weights for policy 1, policy_version 456551 (0.0011) [2023-12-26 18:41:09,576][105692] Updated weights for policy 0, policy_version 456208 (0.0009) [2023-12-26 18:41:09,635][105692] Updated weights for policy 0, policy_version 456218 (0.0009) [2023-12-26 18:41:09,688][105692] Updated weights for policy 0, policy_version 456228 (0.0009) [2023-12-26 18:41:09,779][105620] Updated weights for policy 1, policy_version 456561 (0.0006) [2023-12-26 18:41:09,848][105620] Updated weights for policy 1, policy_version 456571 (0.0008) [2023-12-26 18:41:09,912][105620] Updated weights for policy 1, policy_version 456581 (0.0009) [2023-12-26 18:41:10,515][105692] Updated weights for policy 0, policy_version 456238 (0.0007) [2023-12-26 18:41:10,571][105692] Updated weights for policy 0, policy_version 456248 (0.0005) [2023-12-26 18:41:10,618][105692] Updated weights for policy 0, policy_version 456258 (0.0009) [2023-12-26 18:41:10,687][105620] Updated weights for policy 1, policy_version 456591 (0.0009) [2023-12-26 18:41:10,735][105620] Updated weights for policy 1, policy_version 456601 (0.0009) [2023-12-26 18:41:10,789][105620] Updated weights for policy 1, policy_version 456611 (0.0008) [2023-12-26 18:41:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 233725952. Throughput: 0: 9670.9, 1: 9660.0. Samples: 233732648. Policy #0 lag: (min: 31.0, avg: 31.2, max: 43.0) [2023-12-26 18:41:11,063][104569] Avg episode reward: [(0, '9175.287'), (1, '8783.457')] [2023-12-26 18:41:11,474][105692] Updated weights for policy 0, policy_version 456268 (0.0008) [2023-12-26 18:41:11,491][105620] Updated weights for policy 1, policy_version 456621 (0.0008) [2023-12-26 18:41:11,531][105692] Updated weights for policy 0, policy_version 456278 (0.0006) [2023-12-26 18:41:11,556][105620] Updated weights for policy 1, policy_version 456631 (0.0007) [2023-12-26 18:41:11,579][105692] Updated weights for policy 0, policy_version 456288 (0.0007) [2023-12-26 18:41:11,624][105620] Updated weights for policy 1, policy_version 456641 (0.0009) [2023-12-26 18:41:12,377][105692] Updated weights for policy 0, policy_version 456298 (0.0007) [2023-12-26 18:41:12,416][105620] Updated weights for policy 1, policy_version 456651 (0.0008) [2023-12-26 18:41:12,445][105692] Updated weights for policy 0, policy_version 456308 (0.0008) [2023-12-26 18:41:12,483][105620] Updated weights for policy 1, policy_version 456661 (0.0009) [2023-12-26 18:41:12,512][105692] Updated weights for policy 0, policy_version 456318 (0.0008) [2023-12-26 18:41:12,546][105620] Updated weights for policy 1, policy_version 456671 (0.0005) [2023-12-26 18:41:12,570][105692] Updated weights for policy 0, policy_version 456328 (0.0007) [2023-12-26 18:41:13,249][105692] Updated weights for policy 0, policy_version 456338 (0.0008) [2023-12-26 18:41:13,304][105692] Updated weights for policy 0, policy_version 456348 (0.0009) [2023-12-26 18:41:13,331][105620] Updated weights for policy 1, policy_version 456681 (0.0009) [2023-12-26 18:41:13,366][105692] Updated weights for policy 0, policy_version 456358 (0.0007) [2023-12-26 18:41:13,389][105620] Updated weights for policy 1, policy_version 456691 (0.0008) [2023-12-26 18:41:13,446][105620] Updated weights for policy 1, policy_version 456701 (0.0009) [2023-12-26 18:41:13,500][105620] Updated weights for policy 1, policy_version 456711 (0.0008) [2023-12-26 18:41:14,130][105692] Updated weights for policy 0, policy_version 456368 (0.0008) [2023-12-26 18:41:14,141][105620] Updated weights for policy 1, policy_version 456721 (0.0006) [2023-12-26 18:41:14,195][105620] Updated weights for policy 1, policy_version 456731 (0.0006) [2023-12-26 18:41:14,195][105692] Updated weights for policy 0, policy_version 456378 (0.0011) [2023-12-26 18:41:14,247][105692] Updated weights for policy 0, policy_version 456388 (0.0010) [2023-12-26 18:41:14,252][105620] Updated weights for policy 1, policy_version 456741 (0.0007) [2023-12-26 18:41:14,905][105620] Updated weights for policy 1, policy_version 456751 (0.0007) [2023-12-26 18:41:14,953][105692] Updated weights for policy 0, policy_version 456398 (0.0010) [2023-12-26 18:41:14,963][105620] Updated weights for policy 1, policy_version 456761 (0.0006) [2023-12-26 18:41:15,013][105692] Updated weights for policy 0, policy_version 456408 (0.0011) [2023-12-26 18:41:15,024][105620] Updated weights for policy 1, policy_version 456771 (0.0006) [2023-12-26 18:41:15,066][105692] Updated weights for policy 0, policy_version 456418 (0.0011) [2023-12-26 18:41:15,743][105620] Updated weights for policy 1, policy_version 456781 (0.0006) [2023-12-26 18:41:15,795][105620] Updated weights for policy 1, policy_version 456791 (0.0008) [2023-12-26 18:41:15,817][105692] Updated weights for policy 0, policy_version 456428 (0.0010) [2023-12-26 18:41:15,847][105620] Updated weights for policy 1, policy_version 456801 (0.0010) [2023-12-26 18:41:15,871][105692] Updated weights for policy 0, policy_version 456438 (0.0010) [2023-12-26 18:41:15,919][105692] Updated weights for policy 0, policy_version 456448 (0.0010) [2023-12-26 18:41:16,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 233824256. Throughput: 0: 9628.5, 1: 9609.7. Samples: 233788592. Policy #0 lag: (min: 31.0, avg: 31.2, max: 43.0) [2023-12-26 18:41:16,062][104569] Avg episode reward: [(0, '9266.057'), (1, '9265.193')] [2023-12-26 18:41:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000456456_116867072.pth... [2023-12-26 18:41:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000456808_116957184.pth... [2023-12-26 18:41:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000455336_116580352.pth [2023-12-26 18:41:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000455656_116662272.pth [2023-12-26 18:41:16,577][105620] Updated weights for policy 1, policy_version 456811 (0.0009) [2023-12-26 18:41:16,621][105620] Updated weights for policy 1, policy_version 456821 (0.0008) [2023-12-26 18:41:16,664][105692] Updated weights for policy 0, policy_version 456458 (0.0009) [2023-12-26 18:41:16,666][105620] Updated weights for policy 1, policy_version 456831 (0.0008) [2023-12-26 18:41:16,719][105692] Updated weights for policy 0, policy_version 456468 (0.0010) [2023-12-26 18:41:16,766][105692] Updated weights for policy 0, policy_version 456478 (0.0010) [2023-12-26 18:41:16,814][105692] Updated weights for policy 0, policy_version 456488 (0.0010) [2023-12-26 18:41:17,462][105620] Updated weights for policy 1, policy_version 456841 (0.0006) [2023-12-26 18:41:17,510][105620] Updated weights for policy 1, policy_version 456851 (0.0008) [2023-12-26 18:41:17,562][105620] Updated weights for policy 1, policy_version 456861 (0.0006) [2023-12-26 18:41:17,572][105692] Updated weights for policy 0, policy_version 456498 (0.0010) [2023-12-26 18:41:17,621][105620] Updated weights for policy 1, policy_version 456871 (0.0005) [2023-12-26 18:41:17,623][105692] Updated weights for policy 0, policy_version 456508 (0.0010) [2023-12-26 18:41:17,674][105692] Updated weights for policy 0, policy_version 456518 (0.0010) [2023-12-26 18:41:18,401][105620] Updated weights for policy 1, policy_version 456881 (0.0007) [2023-12-26 18:41:18,438][105692] Updated weights for policy 0, policy_version 456528 (0.0011) [2023-12-26 18:41:18,460][105620] Updated weights for policy 1, policy_version 456891 (0.0005) [2023-12-26 18:41:18,497][105692] Updated weights for policy 0, policy_version 456538 (0.0010) [2023-12-26 18:41:18,509][105620] Updated weights for policy 1, policy_version 456901 (0.0009) [2023-12-26 18:41:18,558][105692] Updated weights for policy 0, policy_version 456548 (0.0010) [2023-12-26 18:41:19,292][105620] Updated weights for policy 1, policy_version 456911 (0.0006) [2023-12-26 18:41:19,321][105692] Updated weights for policy 0, policy_version 456558 (0.0011) [2023-12-26 18:41:19,366][105620] Updated weights for policy 1, policy_version 456921 (0.0007) [2023-12-26 18:41:19,390][105692] Updated weights for policy 0, policy_version 456568 (0.0011) [2023-12-26 18:41:19,438][105620] Updated weights for policy 1, policy_version 456931 (0.0008) [2023-12-26 18:41:19,449][105692] Updated weights for policy 0, policy_version 456578 (0.0011) [2023-12-26 18:41:20,111][105620] Updated weights for policy 1, policy_version 456941 (0.0009) [2023-12-26 18:41:20,169][105620] Updated weights for policy 1, policy_version 456951 (0.0009) [2023-12-26 18:41:20,202][105692] Updated weights for policy 0, policy_version 456588 (0.0009) [2023-12-26 18:41:20,233][105620] Updated weights for policy 1, policy_version 456961 (0.0009) [2023-12-26 18:41:20,260][105692] Updated weights for policy 0, policy_version 456598 (0.0006) [2023-12-26 18:41:20,320][105692] Updated weights for policy 0, policy_version 456608 (0.0007) [2023-12-26 18:41:20,995][105692] Updated weights for policy 0, policy_version 456618 (0.0009) [2023-12-26 18:41:21,057][105620] Updated weights for policy 1, policy_version 456971 (0.0010) [2023-12-26 18:41:21,060][105692] Updated weights for policy 0, policy_version 456628 (0.0008) [2023-12-26 18:41:21,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19114.7, 300 sec: 19522.0). Total num frames: 233906176. Throughput: 0: 9671.6, 1: 9556.8. Samples: 233902484. Policy #0 lag: (min: 31.0, avg: 31.2, max: 43.0) [2023-12-26 18:41:21,063][104569] Avg episode reward: [(0, '9265.755'), (1, '9265.447')] [2023-12-26 18:41:21,082][105585] KL-divergence is very high: 127.0567 [2023-12-26 18:41:21,118][105620] Updated weights for policy 1, policy_version 456981 (0.0008) [2023-12-26 18:41:21,120][105692] Updated weights for policy 0, policy_version 456638 (0.0008) [2023-12-26 18:41:21,133][105585] KL-divergence is very high: 144.1422 [2023-12-26 18:41:21,180][105620] Updated weights for policy 1, policy_version 456991 (0.0009) [2023-12-26 18:41:21,184][105692] Updated weights for policy 0, policy_version 456648 (0.0007) [2023-12-26 18:41:21,903][105620] Updated weights for policy 1, policy_version 457001 (0.0009) [2023-12-26 18:41:21,967][105620] Updated weights for policy 1, policy_version 457011 (0.0007) [2023-12-26 18:41:21,995][105692] Updated weights for policy 0, policy_version 456658 (0.0007) [2023-12-26 18:41:22,026][105620] Updated weights for policy 1, policy_version 457021 (0.0009) [2023-12-26 18:41:22,056][105692] Updated weights for policy 0, policy_version 456668 (0.0010) [2023-12-26 18:41:22,089][105620] Updated weights for policy 1, policy_version 457031 (0.0008) [2023-12-26 18:41:22,113][105692] Updated weights for policy 0, policy_version 456678 (0.0008) [2023-12-26 18:41:22,803][105692] Updated weights for policy 0, policy_version 456688 (0.0005) [2023-12-26 18:41:22,864][105692] Updated weights for policy 0, policy_version 456698 (0.0008) [2023-12-26 18:41:22,868][105620] Updated weights for policy 1, policy_version 457041 (0.0010) [2023-12-26 18:41:22,924][105692] Updated weights for policy 0, policy_version 456708 (0.0008) [2023-12-26 18:41:22,936][105620] Updated weights for policy 1, policy_version 457051 (0.0007) [2023-12-26 18:41:23,005][105620] Updated weights for policy 1, policy_version 457061 (0.0007) [2023-12-26 18:41:23,591][105692] Updated weights for policy 0, policy_version 456718 (0.0008) [2023-12-26 18:41:23,597][105620] Updated weights for policy 1, policy_version 457071 (0.0010) [2023-12-26 18:41:23,635][105692] Updated weights for policy 0, policy_version 456728 (0.0006) [2023-12-26 18:41:23,646][105620] Updated weights for policy 1, policy_version 457081 (0.0010) [2023-12-26 18:41:23,684][105692] Updated weights for policy 0, policy_version 456738 (0.0005) [2023-12-26 18:41:23,700][105620] Updated weights for policy 1, policy_version 457091 (0.0010) [2023-12-26 18:41:24,320][105692] Updated weights for policy 0, policy_version 456748 (0.0007) [2023-12-26 18:41:24,372][105692] Updated weights for policy 0, policy_version 456758 (0.0010) [2023-12-26 18:41:24,423][105692] Updated weights for policy 0, policy_version 456768 (0.0010) [2023-12-26 18:41:24,461][105620] Updated weights for policy 1, policy_version 457101 (0.0010) [2023-12-26 18:41:24,512][105620] Updated weights for policy 1, policy_version 457111 (0.0010) [2023-12-26 18:41:24,560][105620] Updated weights for policy 1, policy_version 457121 (0.0010) [2023-12-26 18:41:25,073][105692] Updated weights for policy 0, policy_version 456778 (0.0007) [2023-12-26 18:41:25,122][105692] Updated weights for policy 0, policy_version 456788 (0.0005) [2023-12-26 18:41:25,182][105692] Updated weights for policy 0, policy_version 456798 (0.0009) [2023-12-26 18:41:25,192][105620] Updated weights for policy 1, policy_version 457131 (0.0010) [2023-12-26 18:41:25,231][105692] Updated weights for policy 0, policy_version 456808 (0.0011) [2023-12-26 18:41:25,248][105620] Updated weights for policy 1, policy_version 457141 (0.0006) [2023-12-26 18:41:25,309][105620] Updated weights for policy 1, policy_version 457151 (0.0008) [2023-12-26 18:41:25,919][105692] Updated weights for policy 0, policy_version 456818 (0.0007) [2023-12-26 18:41:25,969][105692] Updated weights for policy 0, policy_version 456828 (0.0005) [2023-12-26 18:41:26,031][105692] Updated weights for policy 0, policy_version 456838 (0.0006) [2023-12-26 18:41:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 234012672. Throughput: 0: 9690.6, 1: 9579.9. Samples: 234019508. Policy #0 lag: (min: 31.0, avg: 31.2, max: 43.0) [2023-12-26 18:41:26,062][104569] Avg episode reward: [(0, '9265.558'), (1, '9265.348')] [2023-12-26 18:41:26,063][105620] Updated weights for policy 1, policy_version 457161 (0.0008) [2023-12-26 18:41:26,120][105620] Updated weights for policy 1, policy_version 457171 (0.0009) [2023-12-26 18:41:26,178][105620] Updated weights for policy 1, policy_version 457182 (0.0009) [2023-12-26 18:41:26,231][105620] Updated weights for policy 1, policy_version 457192 (0.0009) [2023-12-26 18:41:26,593][105692] Updated weights for policy 0, policy_version 456848 (0.0005) [2023-12-26 18:41:26,637][105692] Updated weights for policy 0, policy_version 456858 (0.0005) [2023-12-26 18:41:26,682][105692] Updated weights for policy 0, policy_version 456868 (0.0005) [2023-12-26 18:41:26,850][105620] Updated weights for policy 1, policy_version 457202 (0.0010) [2023-12-26 18:41:26,918][105620] Updated weights for policy 1, policy_version 457212 (0.0010) [2023-12-26 18:41:26,979][105620] Updated weights for policy 1, policy_version 457222 (0.0010) [2023-12-26 18:41:27,279][105692] Updated weights for policy 0, policy_version 456878 (0.0007) [2023-12-26 18:41:27,334][105692] Updated weights for policy 0, policy_version 456888 (0.0009) [2023-12-26 18:41:27,399][105692] Updated weights for policy 0, policy_version 456898 (0.0007) [2023-12-26 18:41:27,619][105620] Updated weights for policy 1, policy_version 457232 (0.0006) [2023-12-26 18:41:27,673][105620] Updated weights for policy 1, policy_version 457242 (0.0010) [2023-12-26 18:41:27,735][105620] Updated weights for policy 1, policy_version 457252 (0.0010) [2023-12-26 18:41:28,091][105692] Updated weights for policy 0, policy_version 456908 (0.0006) [2023-12-26 18:41:28,151][105692] Updated weights for policy 0, policy_version 456918 (0.0008) [2023-12-26 18:41:28,214][105692] Updated weights for policy 0, policy_version 456928 (0.0008) [2023-12-26 18:41:28,372][105620] Updated weights for policy 1, policy_version 457262 (0.0008) [2023-12-26 18:41:28,431][105620] Updated weights for policy 1, policy_version 457272 (0.0010) [2023-12-26 18:41:28,493][105620] Updated weights for policy 1, policy_version 457282 (0.0010) [2023-12-26 18:41:28,908][105692] Updated weights for policy 0, policy_version 456938 (0.0008) [2023-12-26 18:41:28,966][105692] Updated weights for policy 0, policy_version 456948 (0.0010) [2023-12-26 18:41:29,025][105692] Updated weights for policy 0, policy_version 456958 (0.0010) [2023-12-26 18:41:29,083][105692] Updated weights for policy 0, policy_version 456968 (0.0010) [2023-12-26 18:41:29,227][105620] Updated weights for policy 1, policy_version 457292 (0.0010) [2023-12-26 18:41:29,291][105620] Updated weights for policy 1, policy_version 457302 (0.0010) [2023-12-26 18:41:29,355][105620] Updated weights for policy 1, policy_version 457312 (0.0011) [2023-12-26 18:41:29,800][105692] Updated weights for policy 0, policy_version 456978 (0.0010) [2023-12-26 18:41:29,860][105692] Updated weights for policy 0, policy_version 456988 (0.0010) [2023-12-26 18:41:29,919][105692] Updated weights for policy 0, policy_version 456998 (0.0010) [2023-12-26 18:41:30,102][105620] Updated weights for policy 1, policy_version 457322 (0.0009) [2023-12-26 18:41:30,156][105620] Updated weights for policy 1, policy_version 457332 (0.0005) [2023-12-26 18:41:30,224][105620] Updated weights for policy 1, policy_version 457342 (0.0005) [2023-12-26 18:41:30,291][105620] Updated weights for policy 1, policy_version 457352 (0.0005) [2023-12-26 18:41:30,667][105692] Updated weights for policy 0, policy_version 457008 (0.0010) [2023-12-26 18:41:30,714][105692] Updated weights for policy 0, policy_version 457018 (0.0010) [2023-12-26 18:41:30,771][105692] Updated weights for policy 0, policy_version 457028 (0.0010) [2023-12-26 18:41:30,882][105620] Updated weights for policy 1, policy_version 457362 (0.0008) [2023-12-26 18:41:30,929][105620] Updated weights for policy 1, policy_version 457372 (0.0008) [2023-12-26 18:41:30,988][105620] Updated weights for policy 1, policy_version 457382 (0.0009) [2023-12-26 18:41:31,062][104569] Fps is (10 sec: 21299.3, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 234119168. Throughput: 0: 9797.0, 1: 9644.9. Samples: 234083876. Policy #0 lag: (min: 31.0, avg: 31.2, max: 43.0) [2023-12-26 18:41:31,062][104569] Avg episode reward: [(0, '9354.598'), (1, '9175.329')] [2023-12-26 18:41:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000457032_117014528.pth... [2023-12-26 18:41:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000457384_117104640.pth... [2023-12-26 18:41:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000455880_116719616.pth [2023-12-26 18:41:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000456232_116809728.pth [2023-12-26 18:41:31,500][105692] Updated weights for policy 0, policy_version 457038 (0.0010) [2023-12-26 18:41:31,551][105692] Updated weights for policy 0, policy_version 457048 (0.0010) [2023-12-26 18:41:31,603][105692] Updated weights for policy 0, policy_version 457058 (0.0010) [2023-12-26 18:41:31,697][105620] Updated weights for policy 1, policy_version 457392 (0.0008) [2023-12-26 18:41:31,757][105620] Updated weights for policy 1, policy_version 457402 (0.0007) [2023-12-26 18:41:31,824][105620] Updated weights for policy 1, policy_version 457412 (0.0008) [2023-12-26 18:41:32,298][105692] Updated weights for policy 0, policy_version 457068 (0.0011) [2023-12-26 18:41:32,360][105692] Updated weights for policy 0, policy_version 457078 (0.0010) [2023-12-26 18:41:32,422][105692] Updated weights for policy 0, policy_version 457088 (0.0010) [2023-12-26 18:41:32,500][105620] Updated weights for policy 1, policy_version 457422 (0.0006) [2023-12-26 18:41:32,567][105620] Updated weights for policy 1, policy_version 457432 (0.0007) [2023-12-26 18:41:32,622][105620] Updated weights for policy 1, policy_version 457442 (0.0008) [2023-12-26 18:41:33,156][105692] Updated weights for policy 0, policy_version 457098 (0.0010) [2023-12-26 18:41:33,219][105692] Updated weights for policy 0, policy_version 457108 (0.0011) [2023-12-26 18:41:33,275][105692] Updated weights for policy 0, policy_version 457118 (0.0011) [2023-12-26 18:41:33,291][105620] Updated weights for policy 1, policy_version 457452 (0.0007) [2023-12-26 18:41:33,335][105692] Updated weights for policy 0, policy_version 457128 (0.0011) [2023-12-26 18:41:33,344][105620] Updated weights for policy 1, policy_version 457462 (0.0009) [2023-12-26 18:41:33,401][105620] Updated weights for policy 1, policy_version 457472 (0.0008) [2023-12-26 18:41:33,937][105692] Updated weights for policy 0, policy_version 457138 (0.0005) [2023-12-26 18:41:34,002][105692] Updated weights for policy 0, policy_version 457148 (0.0010) [2023-12-26 18:41:34,056][105692] Updated weights for policy 0, policy_version 457159 (0.0010) [2023-12-26 18:41:34,166][105620] Updated weights for policy 1, policy_version 457483 (0.0010) [2023-12-26 18:41:34,225][105620] Updated weights for policy 1, policy_version 457493 (0.0008) [2023-12-26 18:41:34,281][105620] Updated weights for policy 1, policy_version 457503 (0.0008) [2023-12-26 18:41:34,825][105692] Updated weights for policy 0, policy_version 457169 (0.0009) [2023-12-26 18:41:34,881][105692] Updated weights for policy 0, policy_version 457179 (0.0008) [2023-12-26 18:41:34,936][105692] Updated weights for policy 0, policy_version 457189 (0.0006) [2023-12-26 18:41:34,945][105620] Updated weights for policy 1, policy_version 457513 (0.0008) [2023-12-26 18:41:35,011][105620] Updated weights for policy 1, policy_version 457523 (0.0009) [2023-12-26 18:41:35,079][105620] Updated weights for policy 1, policy_version 457533 (0.0008) [2023-12-26 18:41:35,146][105620] Updated weights for policy 1, policy_version 457543 (0.0010) [2023-12-26 18:41:35,613][105692] Updated weights for policy 0, policy_version 457199 (0.0009) [2023-12-26 18:41:35,662][105692] Updated weights for policy 0, policy_version 457209 (0.0010) [2023-12-26 18:41:35,710][105692] Updated weights for policy 0, policy_version 457219 (0.0010) [2023-12-26 18:41:35,862][105620] Updated weights for policy 1, policy_version 457553 (0.0010) [2023-12-26 18:41:35,909][105620] Updated weights for policy 1, policy_version 457563 (0.0010) [2023-12-26 18:41:35,963][105620] Updated weights for policy 1, policy_version 457573 (0.0010) [2023-12-26 18:41:36,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 234217472. Throughput: 0: 9729.0, 1: 9785.3. Samples: 234201932. Policy #0 lag: (min: 31.0, avg: 31.2, max: 43.0) [2023-12-26 18:41:36,063][104569] Avg episode reward: [(0, '9265.258'), (1, '9084.718')] [2023-12-26 18:41:36,397][105692] Updated weights for policy 0, policy_version 457229 (0.0008) [2023-12-26 18:41:36,455][105692] Updated weights for policy 0, policy_version 457239 (0.0006) [2023-12-26 18:41:36,514][105692] Updated weights for policy 0, policy_version 457249 (0.0006) [2023-12-26 18:41:36,715][105620] Updated weights for policy 1, policy_version 457583 (0.0010) [2023-12-26 18:41:36,763][105620] Updated weights for policy 1, policy_version 457593 (0.0010) [2023-12-26 18:41:36,812][105620] Updated weights for policy 1, policy_version 457603 (0.0010) [2023-12-26 18:41:37,057][105692] Updated weights for policy 0, policy_version 457259 (0.0006) [2023-12-26 18:41:37,102][105692] Updated weights for policy 0, policy_version 457269 (0.0008) [2023-12-26 18:41:37,147][105692] Updated weights for policy 0, policy_version 457279 (0.0008) [2023-12-26 18:41:37,573][105620] Updated weights for policy 1, policy_version 457613 (0.0008) [2023-12-26 18:41:37,634][105620] Updated weights for policy 1, policy_version 457623 (0.0005) [2023-12-26 18:41:37,701][105620] Updated weights for policy 1, policy_version 457633 (0.0006) [2023-12-26 18:41:37,954][105692] Updated weights for policy 0, policy_version 457289 (0.0008) [2023-12-26 18:41:38,022][105692] Updated weights for policy 0, policy_version 457299 (0.0008) [2023-12-26 18:41:38,083][105692] Updated weights for policy 0, policy_version 457309 (0.0008) [2023-12-26 18:41:38,137][105692] Updated weights for policy 0, policy_version 457319 (0.0008) [2023-12-26 18:41:38,406][105620] Updated weights for policy 1, policy_version 457643 (0.0008) [2023-12-26 18:41:38,467][105620] Updated weights for policy 1, policy_version 457653 (0.0005) [2023-12-26 18:41:38,537][105620] Updated weights for policy 1, policy_version 457663 (0.0008) [2023-12-26 18:41:38,839][105692] Updated weights for policy 0, policy_version 457329 (0.0008) [2023-12-26 18:41:38,903][105692] Updated weights for policy 0, policy_version 457339 (0.0008) [2023-12-26 18:41:38,969][105692] Updated weights for policy 0, policy_version 457349 (0.0008) [2023-12-26 18:41:39,183][105620] Updated weights for policy 1, policy_version 457673 (0.0006) [2023-12-26 18:41:39,236][105620] Updated weights for policy 1, policy_version 457683 (0.0007) [2023-12-26 18:41:39,289][105620] Updated weights for policy 1, policy_version 457693 (0.0006) [2023-12-26 18:41:39,359][105620] Updated weights for policy 1, policy_version 457703 (0.0010) [2023-12-26 18:41:39,645][105692] Updated weights for policy 0, policy_version 457359 (0.0008) [2023-12-26 18:41:39,705][105692] Updated weights for policy 0, policy_version 457369 (0.0007) [2023-12-26 18:41:39,771][105692] Updated weights for policy 0, policy_version 457379 (0.0005) [2023-12-26 18:41:40,100][105620] Updated weights for policy 1, policy_version 457713 (0.0011) [2023-12-26 18:41:40,153][105620] Updated weights for policy 1, policy_version 457723 (0.0010) [2023-12-26 18:41:40,210][105620] Updated weights for policy 1, policy_version 457733 (0.0011) [2023-12-26 18:41:40,523][105692] Updated weights for policy 0, policy_version 457389 (0.0010) [2023-12-26 18:41:40,585][105692] Updated weights for policy 0, policy_version 457399 (0.0006) [2023-12-26 18:41:40,646][105692] Updated weights for policy 0, policy_version 457409 (0.0011) [2023-12-26 18:41:40,967][105620] Updated weights for policy 1, policy_version 457743 (0.0011) [2023-12-26 18:41:41,023][105620] Updated weights for policy 1, policy_version 457753 (0.0011) [2023-12-26 18:41:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 234307584. Throughput: 0: 9817.0, 1: 9761.2. Samples: 234318996. Policy #0 lag: (min: 31.0, avg: 31.2, max: 43.0) [2023-12-26 18:41:41,063][104569] Avg episode reward: [(0, '9354.422'), (1, '9083.146')] [2023-12-26 18:41:41,089][105620] Updated weights for policy 1, policy_version 457763 (0.0011) [2023-12-26 18:41:41,324][105692] Updated weights for policy 0, policy_version 457419 (0.0011) [2023-12-26 18:41:41,398][105692] Updated weights for policy 0, policy_version 457429 (0.0010) [2023-12-26 18:41:41,453][105692] Updated weights for policy 0, policy_version 457439 (0.0008) [2023-12-26 18:41:41,850][105620] Updated weights for policy 1, policy_version 457773 (0.0011) [2023-12-26 18:41:41,903][105620] Updated weights for policy 1, policy_version 457783 (0.0009) [2023-12-26 18:41:41,956][105620] Updated weights for policy 1, policy_version 457793 (0.0010) [2023-12-26 18:41:42,133][105692] Updated weights for policy 0, policy_version 457449 (0.0010) [2023-12-26 18:41:42,196][105692] Updated weights for policy 0, policy_version 457459 (0.0009) [2023-12-26 18:41:42,258][105692] Updated weights for policy 0, policy_version 457469 (0.0008) [2023-12-26 18:41:42,324][105692] Updated weights for policy 0, policy_version 457479 (0.0009) [2023-12-26 18:41:42,795][105620] Updated weights for policy 1, policy_version 457803 (0.0010) [2023-12-26 18:41:42,855][105620] Updated weights for policy 1, policy_version 457813 (0.0009) [2023-12-26 18:41:42,915][105620] Updated weights for policy 1, policy_version 457823 (0.0009) [2023-12-26 18:41:43,042][105692] Updated weights for policy 0, policy_version 457489 (0.0008) [2023-12-26 18:41:43,104][105692] Updated weights for policy 0, policy_version 457499 (0.0009) [2023-12-26 18:41:43,169][105692] Updated weights for policy 0, policy_version 457509 (0.0009) [2023-12-26 18:41:43,594][105620] Updated weights for policy 1, policy_version 457833 (0.0009) [2023-12-26 18:41:43,659][105620] Updated weights for policy 1, policy_version 457843 (0.0009) [2023-12-26 18:41:43,721][105620] Updated weights for policy 1, policy_version 457853 (0.0009) [2023-12-26 18:41:43,784][105620] Updated weights for policy 1, policy_version 457863 (0.0009) [2023-12-26 18:41:43,927][105692] Updated weights for policy 0, policy_version 457519 (0.0009) [2023-12-26 18:41:43,992][105692] Updated weights for policy 0, policy_version 457529 (0.0009) [2023-12-26 18:41:44,052][105692] Updated weights for policy 0, policy_version 457539 (0.0009) [2023-12-26 18:41:44,421][105620] Updated weights for policy 1, policy_version 457873 (0.0006) [2023-12-26 18:41:44,476][105620] Updated weights for policy 1, policy_version 457883 (0.0008) [2023-12-26 18:41:44,539][105620] Updated weights for policy 1, policy_version 457893 (0.0009) [2023-12-26 18:41:44,825][105692] Updated weights for policy 0, policy_version 457549 (0.0008) [2023-12-26 18:41:44,884][105692] Updated weights for policy 0, policy_version 457559 (0.0010) [2023-12-26 18:41:44,937][105692] Updated weights for policy 0, policy_version 457569 (0.0011) [2023-12-26 18:41:45,343][105620] Updated weights for policy 1, policy_version 457903 (0.0008) [2023-12-26 18:41:45,400][105620] Updated weights for policy 1, policy_version 457913 (0.0008) [2023-12-26 18:41:45,449][105620] Updated weights for policy 1, policy_version 457923 (0.0007) [2023-12-26 18:41:45,710][105692] Updated weights for policy 0, policy_version 457579 (0.0011) [2023-12-26 18:41:45,761][105692] Updated weights for policy 0, policy_version 457589 (0.0011) [2023-12-26 18:41:45,814][105692] Updated weights for policy 0, policy_version 457599 (0.0011) [2023-12-26 18:41:46,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 234405888. Throughput: 0: 9807.0, 1: 9711.3. Samples: 234375732. Policy #0 lag: (min: 31.0, avg: 31.2, max: 43.0) [2023-12-26 18:41:46,062][104569] Avg episode reward: [(0, '9354.285'), (1, '8628.513')] [2023-12-26 18:41:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000457608_117161984.pth... [2023-12-26 18:41:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000457928_117243904.pth... [2023-12-26 18:41:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000456456_116867072.pth [2023-12-26 18:41:46,075][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_000457608_117161984.pth [2023-12-26 18:41:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000456808_116957184.pth [2023-12-26 18:41:46,080][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_000457928_117243904.pth [2023-12-26 18:41:46,223][105620] Updated weights for policy 1, policy_version 457933 (0.0007) [2023-12-26 18:41:46,288][105620] Updated weights for policy 1, policy_version 457943 (0.0005) [2023-12-26 18:41:46,344][105620] Updated weights for policy 1, policy_version 457953 (0.0005) [2023-12-26 18:41:46,532][105692] Updated weights for policy 0, policy_version 457609 (0.0010) [2023-12-26 18:41:46,585][105692] Updated weights for policy 0, policy_version 457619 (0.0005) [2023-12-26 18:41:46,649][105692] Updated weights for policy 0, policy_version 457629 (0.0007) [2023-12-26 18:41:46,714][105692] Updated weights for policy 0, policy_version 457639 (0.0010) [2023-12-26 18:41:46,945][105620] Updated weights for policy 1, policy_version 457963 (0.0007) [2023-12-26 18:41:47,001][105620] Updated weights for policy 1, policy_version 457973 (0.0009) [2023-12-26 18:41:47,059][105620] Updated weights for policy 1, policy_version 457983 (0.0009) [2023-12-26 18:41:47,252][105692] Updated weights for policy 0, policy_version 457649 (0.0006) [2023-12-26 18:41:47,298][105692] Updated weights for policy 0, policy_version 457659 (0.0005) [2023-12-26 18:41:47,346][105692] Updated weights for policy 0, policy_version 457669 (0.0005) [2023-12-26 18:41:47,873][105692] Updated weights for policy 0, policy_version 457679 (0.0006) [2023-12-26 18:41:47,920][105692] Updated weights for policy 0, policy_version 457689 (0.0005) [2023-12-26 18:41:47,924][105585] KL-divergence is very high: 427.0643 [2023-12-26 18:41:47,962][105585] KL-divergence is very high: 645.6453 [2023-12-26 18:41:47,967][105692] Updated weights for policy 0, policy_version 457699 (0.0005) [2023-12-26 18:41:47,979][105620] Updated weights for policy 1, policy_version 457993 (0.0010) [2023-12-26 18:41:48,041][105620] Updated weights for policy 1, policy_version 458003 (0.0009) [2023-12-26 18:41:48,098][105620] Updated weights for policy 1, policy_version 458013 (0.0009) [2023-12-26 18:41:48,157][105620] Updated weights for policy 1, policy_version 458023 (0.0008) [2023-12-26 18:41:48,689][105692] Updated weights for policy 0, policy_version 457709 (0.0008) [2023-12-26 18:41:48,737][105692] Updated weights for policy 0, policy_version 457719 (0.0010) [2023-12-26 18:41:48,785][105692] Updated weights for policy 0, policy_version 457729 (0.0010) [2023-12-26 18:41:48,927][105620] Updated weights for policy 1, policy_version 458033 (0.0008) [2023-12-26 18:41:48,994][105620] Updated weights for policy 1, policy_version 458043 (0.0007) [2023-12-26 18:41:49,062][105620] Updated weights for policy 1, policy_version 458053 (0.0005) [2023-12-26 18:41:49,526][105692] Updated weights for policy 0, policy_version 457739 (0.0010) [2023-12-26 18:41:49,574][105692] Updated weights for policy 0, policy_version 457749 (0.0009) [2023-12-26 18:41:49,621][105692] Updated weights for policy 0, policy_version 457759 (0.0009) [2023-12-26 18:41:49,792][105620] Updated weights for policy 1, policy_version 458063 (0.0008) [2023-12-26 18:41:49,850][105620] Updated weights for policy 1, policy_version 458073 (0.0010) [2023-12-26 18:41:49,912][105620] Updated weights for policy 1, policy_version 458083 (0.0008) [2023-12-26 18:41:50,438][105692] Updated weights for policy 0, policy_version 457769 (0.0009) [2023-12-26 18:41:50,506][105692] Updated weights for policy 0, policy_version 457779 (0.0008) [2023-12-26 18:41:50,573][105692] Updated weights for policy 0, policy_version 457789 (0.0008) [2023-12-26 18:41:50,595][105620] Updated weights for policy 1, policy_version 458093 (0.0009) [2023-12-26 18:41:50,637][105692] Updated weights for policy 0, policy_version 457799 (0.0007) [2023-12-26 18:41:50,660][105620] Updated weights for policy 1, policy_version 458103 (0.0010) [2023-12-26 18:41:50,713][105620] Updated weights for policy 1, policy_version 458113 (0.0011) [2023-12-26 18:41:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19549.8). Total num frames: 234504192. Throughput: 0: 9925.2, 1: 9645.9. Samples: 234492760. Policy #0 lag: (min: 31.0, avg: 31.2, max: 43.0) [2023-12-26 18:41:51,062][104569] Avg episode reward: [(0, '9082.607'), (1, '5601.421')] [2023-12-26 18:41:51,426][105692] Updated weights for policy 0, policy_version 457809 (0.0008) [2023-12-26 18:41:51,482][105692] Updated weights for policy 0, policy_version 457819 (0.0008) [2023-12-26 18:41:51,491][105620] Updated weights for policy 1, policy_version 458123 (0.0010) [2023-12-26 18:41:51,529][105692] Updated weights for policy 0, policy_version 457829 (0.0007) [2023-12-26 18:41:51,553][105620] Updated weights for policy 1, policy_version 458133 (0.0008) [2023-12-26 18:41:51,614][105620] Updated weights for policy 1, policy_version 458143 (0.0011) [2023-12-26 18:41:52,315][105692] Updated weights for policy 0, policy_version 457839 (0.0007) [2023-12-26 18:41:52,355][105620] Updated weights for policy 1, policy_version 458153 (0.0011) [2023-12-26 18:41:52,380][105692] Updated weights for policy 0, policy_version 457849 (0.0008) [2023-12-26 18:41:52,416][105620] Updated weights for policy 1, policy_version 458163 (0.0007) [2023-12-26 18:41:52,435][105692] Updated weights for policy 0, policy_version 457859 (0.0008) [2023-12-26 18:41:52,471][105620] Updated weights for policy 1, policy_version 458173 (0.0007) [2023-12-26 18:41:52,517][105620] Updated weights for policy 1, policy_version 458183 (0.0008) [2023-12-26 18:41:53,200][105620] Updated weights for policy 1, policy_version 458193 (0.0006) [2023-12-26 18:41:53,230][105692] Updated weights for policy 0, policy_version 457869 (0.0008) [2023-12-26 18:41:53,256][105620] Updated weights for policy 1, policy_version 458203 (0.0005) [2023-12-26 18:41:53,278][105692] Updated weights for policy 0, policy_version 457879 (0.0009) [2023-12-26 18:41:53,310][105620] Updated weights for policy 1, policy_version 458213 (0.0006) [2023-12-26 18:41:53,341][105692] Updated weights for policy 0, policy_version 457889 (0.0008) [2023-12-26 18:41:53,869][105620] Updated weights for policy 1, policy_version 458223 (0.0008) [2023-12-26 18:41:53,920][105620] Updated weights for policy 1, policy_version 458233 (0.0009) [2023-12-26 18:41:53,985][105620] Updated weights for policy 1, policy_version 458243 (0.0009) [2023-12-26 18:41:54,123][105692] Updated weights for policy 0, policy_version 457899 (0.0010) [2023-12-26 18:41:54,187][105692] Updated weights for policy 0, policy_version 457909 (0.0008) [2023-12-26 18:41:54,255][105692] Updated weights for policy 0, policy_version 457919 (0.0008) [2023-12-26 18:41:54,670][105620] Updated weights for policy 1, policy_version 458253 (0.0007) [2023-12-26 18:41:54,720][105620] Updated weights for policy 1, policy_version 458263 (0.0005) [2023-12-26 18:41:54,772][105620] Updated weights for policy 1, policy_version 458274 (0.0008) [2023-12-26 18:41:54,861][105692] Updated weights for policy 0, policy_version 457929 (0.0005) [2023-12-26 18:41:54,917][105692] Updated weights for policy 0, policy_version 457939 (0.0005) [2023-12-26 18:41:54,979][105692] Updated weights for policy 0, policy_version 457949 (0.0006) [2023-12-26 18:41:55,044][105692] Updated weights for policy 0, policy_version 457959 (0.0005) [2023-12-26 18:41:55,408][105620] Updated weights for policy 1, policy_version 458284 (0.0008) [2023-12-26 18:41:55,460][105620] Updated weights for policy 1, policy_version 458294 (0.0005) [2023-12-26 18:41:55,508][105620] Updated weights for policy 1, policy_version 458304 (0.0005) [2023-12-26 18:41:55,559][105692] Updated weights for policy 0, policy_version 457969 (0.0006) [2023-12-26 18:41:55,603][105692] Updated weights for policy 0, policy_version 457979 (0.0008) [2023-12-26 18:41:55,647][105692] Updated weights for policy 0, policy_version 457989 (0.0008) [2023-12-26 18:41:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 234602496. Throughput: 0: 9837.9, 1: 9716.1. Samples: 234612576. Policy #0 lag: (min: 31.0, avg: 31.2, max: 43.0) [2023-12-26 18:41:56,062][104569] Avg episode reward: [(0, '8905.754'), (1, '7057.094')] [2023-12-26 18:41:56,171][105620] Updated weights for policy 1, policy_version 458314 (0.0006) [2023-12-26 18:41:56,224][105620] Updated weights for policy 1, policy_version 458324 (0.0005) [2023-12-26 18:41:56,271][105620] Updated weights for policy 1, policy_version 458334 (0.0005) [2023-12-26 18:41:56,315][105620] Updated weights for policy 1, policy_version 458344 (0.0005) [2023-12-26 18:41:56,316][105692] Updated weights for policy 0, policy_version 457999 (0.0009) [2023-12-26 18:41:56,371][105692] Updated weights for policy 0, policy_version 458009 (0.0010) [2023-12-26 18:41:56,423][105692] Updated weights for policy 0, policy_version 458019 (0.0010) [2023-12-26 18:41:56,984][105620] Updated weights for policy 1, policy_version 458354 (0.0010) [2023-12-26 18:41:57,035][105620] Updated weights for policy 1, policy_version 458364 (0.0010) [2023-12-26 18:41:57,086][105692] Updated weights for policy 0, policy_version 458029 (0.0008) [2023-12-26 18:41:57,088][105620] Updated weights for policy 1, policy_version 458374 (0.0006) [2023-12-26 18:41:57,154][105692] Updated weights for policy 0, policy_version 458039 (0.0005) [2023-12-26 18:41:57,212][105692] Updated weights for policy 0, policy_version 458049 (0.0009) [2023-12-26 18:41:57,677][105620] Updated weights for policy 1, policy_version 458384 (0.0005) [2023-12-26 18:41:57,731][105620] Updated weights for policy 1, policy_version 458394 (0.0006) [2023-12-26 18:41:57,791][105620] Updated weights for policy 1, policy_version 458404 (0.0005) [2023-12-26 18:41:57,841][105692] Updated weights for policy 0, policy_version 458059 (0.0009) [2023-12-26 18:41:57,889][105692] Updated weights for policy 0, policy_version 458069 (0.0005) [2023-12-26 18:41:57,935][105692] Updated weights for policy 0, policy_version 458079 (0.0007) [2023-12-26 18:41:58,443][105620] Updated weights for policy 1, policy_version 458414 (0.0006) [2023-12-26 18:41:58,504][105620] Updated weights for policy 1, policy_version 458424 (0.0007) [2023-12-26 18:41:58,568][105620] Updated weights for policy 1, policy_version 458434 (0.0009) [2023-12-26 18:41:58,714][105692] Updated weights for policy 0, policy_version 458089 (0.0010) [2023-12-26 18:41:58,788][105692] Updated weights for policy 0, policy_version 458099 (0.0011) [2023-12-26 18:41:58,856][105692] Updated weights for policy 0, policy_version 458109 (0.0010) [2023-12-26 18:41:58,912][105692] Updated weights for policy 0, policy_version 458119 (0.0010) [2023-12-26 18:41:59,434][105620] Updated weights for policy 1, policy_version 458444 (0.0009) [2023-12-26 18:41:59,491][105620] Updated weights for policy 1, policy_version 458454 (0.0010) [2023-12-26 18:41:59,546][105620] Updated weights for policy 1, policy_version 458465 (0.0009) [2023-12-26 18:41:59,651][105692] Updated weights for policy 0, policy_version 458129 (0.0008) [2023-12-26 18:41:59,713][105692] Updated weights for policy 0, policy_version 458139 (0.0009) [2023-12-26 18:41:59,760][105692] Updated weights for policy 0, policy_version 458149 (0.0009) [2023-12-26 18:42:00,355][105620] Updated weights for policy 1, policy_version 458475 (0.0009) [2023-12-26 18:42:00,412][105620] Updated weights for policy 1, policy_version 458485 (0.0006) [2023-12-26 18:42:00,431][105692] Updated weights for policy 0, policy_version 458159 (0.0007) [2023-12-26 18:42:00,469][105620] Updated weights for policy 1, policy_version 458495 (0.0005) [2023-12-26 18:42:00,490][105692] Updated weights for policy 0, policy_version 458169 (0.0008) [2023-12-26 18:42:00,550][105692] Updated weights for policy 0, policy_version 458179 (0.0008) [2023-12-26 18:42:01,005][105620] Updated weights for policy 1, policy_version 458505 (0.0007) [2023-12-26 18:42:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 234700800. Throughput: 0: 9917.6, 1: 9769.7. Samples: 234674524. Policy #0 lag: (min: 31.0, avg: 31.2, max: 43.0) [2023-12-26 18:42:01,063][104569] Avg episode reward: [(0, '8997.102'), (1, '9355.324')] [2023-12-26 18:42:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000458184_117309440.pth... [2023-12-26 18:42:01,072][105620] Updated weights for policy 1, policy_version 458515 (0.0008) [2023-12-26 18:42:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000457032_117014528.pth [2023-12-26 18:42:01,127][105620] Updated weights for policy 1, policy_version 458525 (0.0009) [2023-12-26 18:42:01,198][105620] Updated weights for policy 1, policy_version 458535 (0.0008) [2023-12-26 18:42:01,201][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000458536_117399552.pth... [2023-12-26 18:42:01,206][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000457384_117104640.pth [2023-12-26 18:42:01,272][105692] Updated weights for policy 0, policy_version 458189 (0.0009) [2023-12-26 18:42:01,338][105692] Updated weights for policy 0, policy_version 458199 (0.0009) [2023-12-26 18:42:01,402][105692] Updated weights for policy 0, policy_version 458209 (0.0009) [2023-12-26 18:42:01,946][105620] Updated weights for policy 1, policy_version 458545 (0.0006) [2023-12-26 18:42:02,012][105620] Updated weights for policy 1, policy_version 458555 (0.0008) [2023-12-26 18:42:02,079][105620] Updated weights for policy 1, policy_version 458565 (0.0008) [2023-12-26 18:42:02,114][105692] Updated weights for policy 0, policy_version 458219 (0.0008) [2023-12-26 18:42:02,163][105692] Updated weights for policy 0, policy_version 458229 (0.0007) [2023-12-26 18:42:02,233][105692] Updated weights for policy 0, policy_version 458239 (0.0009) [2023-12-26 18:42:02,710][105620] Updated weights for policy 1, policy_version 458575 (0.0007) [2023-12-26 18:42:02,766][105620] Updated weights for policy 1, policy_version 458585 (0.0005) [2023-12-26 18:42:02,824][105620] Updated weights for policy 1, policy_version 458595 (0.0005) [2023-12-26 18:42:03,033][105692] Updated weights for policy 0, policy_version 458249 (0.0009) [2023-12-26 18:42:03,084][105692] Updated weights for policy 0, policy_version 458259 (0.0009) [2023-12-26 18:42:03,137][105692] Updated weights for policy 0, policy_version 458269 (0.0006) [2023-12-26 18:42:03,186][105692] Updated weights for policy 0, policy_version 458279 (0.0005) [2023-12-26 18:42:03,530][105620] Updated weights for policy 1, policy_version 458605 (0.0007) [2023-12-26 18:42:03,574][105620] Updated weights for policy 1, policy_version 458615 (0.0008) [2023-12-26 18:42:03,625][105620] Updated weights for policy 1, policy_version 458625 (0.0007) [2023-12-26 18:42:03,820][105692] Updated weights for policy 0, policy_version 458289 (0.0007) [2023-12-26 18:42:03,890][105692] Updated weights for policy 0, policy_version 458299 (0.0007) [2023-12-26 18:42:03,953][105692] Updated weights for policy 0, policy_version 458309 (0.0006) [2023-12-26 18:42:04,303][105620] Updated weights for policy 1, policy_version 458635 (0.0008) [2023-12-26 18:42:04,369][105620] Updated weights for policy 1, policy_version 458645 (0.0006) [2023-12-26 18:42:04,432][105620] Updated weights for policy 1, policy_version 458655 (0.0007) [2023-12-26 18:42:04,619][105692] Updated weights for policy 0, policy_version 458319 (0.0010) [2023-12-26 18:42:04,683][105692] Updated weights for policy 0, policy_version 458329 (0.0010) [2023-12-26 18:42:04,741][105692] Updated weights for policy 0, policy_version 458339 (0.0010) [2023-12-26 18:42:05,130][105620] Updated weights for policy 1, policy_version 458665 (0.0006) [2023-12-26 18:42:05,175][105620] Updated weights for policy 1, policy_version 458675 (0.0008) [2023-12-26 18:42:05,219][105620] Updated weights for policy 1, policy_version 458685 (0.0008) [2023-12-26 18:42:05,270][105620] Updated weights for policy 1, policy_version 458695 (0.0006) [2023-12-26 18:42:05,466][105692] Updated weights for policy 0, policy_version 458349 (0.0010) [2023-12-26 18:42:05,515][105692] Updated weights for policy 0, policy_version 458359 (0.0010) [2023-12-26 18:42:05,563][105692] Updated weights for policy 0, policy_version 458369 (0.0010) [2023-12-26 18:42:05,929][105620] Updated weights for policy 1, policy_version 458705 (0.0010) [2023-12-26 18:42:05,981][105620] Updated weights for policy 1, policy_version 458715 (0.0009) [2023-12-26 18:42:06,042][105620] Updated weights for policy 1, policy_version 458725 (0.0009) [2023-12-26 18:42:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 234807296. Throughput: 0: 9962.8, 1: 9814.6. Samples: 234792464. Policy #0 lag: (min: 31.0, avg: 31.2, max: 43.0) [2023-12-26 18:42:06,062][104569] Avg episode reward: [(0, '9084.943'), (1, '9355.541')] [2023-12-26 18:42:06,264][105692] Updated weights for policy 0, policy_version 458379 (0.0010) [2023-12-26 18:42:06,324][105692] Updated weights for policy 0, policy_version 458389 (0.0006) [2023-12-26 18:42:06,390][105692] Updated weights for policy 0, policy_version 458399 (0.0007) [2023-12-26 18:42:06,844][105620] Updated weights for policy 1, policy_version 458735 (0.0009) [2023-12-26 18:42:06,892][105620] Updated weights for policy 1, policy_version 458745 (0.0009) [2023-12-26 18:42:06,952][105620] Updated weights for policy 1, policy_version 458755 (0.0009) [2023-12-26 18:42:07,075][105692] Updated weights for policy 0, policy_version 458409 (0.0008) [2023-12-26 18:42:07,140][105692] Updated weights for policy 0, policy_version 458419 (0.0009) [2023-12-26 18:42:07,194][105692] Updated weights for policy 0, policy_version 458429 (0.0008) [2023-12-26 18:42:07,245][105692] Updated weights for policy 0, policy_version 458439 (0.0009) [2023-12-26 18:42:07,644][105620] Updated weights for policy 1, policy_version 458765 (0.0008) [2023-12-26 18:42:07,691][105620] Updated weights for policy 1, policy_version 458775 (0.0009) [2023-12-26 18:42:07,749][105620] Updated weights for policy 1, policy_version 458785 (0.0008) [2023-12-26 18:42:08,015][105692] Updated weights for policy 0, policy_version 458449 (0.0010) [2023-12-26 18:42:08,080][105692] Updated weights for policy 0, policy_version 458459 (0.0009) [2023-12-26 18:42:08,144][105692] Updated weights for policy 0, policy_version 458469 (0.0009) [2023-12-26 18:42:08,473][105620] Updated weights for policy 1, policy_version 458796 (0.0010) [2023-12-26 18:42:08,520][105620] Updated weights for policy 1, policy_version 458806 (0.0009) [2023-12-26 18:42:08,576][105620] Updated weights for policy 1, policy_version 458816 (0.0009) [2023-12-26 18:42:08,920][105692] Updated weights for policy 0, policy_version 458479 (0.0009) [2023-12-26 18:42:08,971][105692] Updated weights for policy 0, policy_version 458489 (0.0009) [2023-12-26 18:42:09,023][105692] Updated weights for policy 0, policy_version 458499 (0.0009) [2023-12-26 18:42:09,286][105620] Updated weights for policy 1, policy_version 458826 (0.0009) [2023-12-26 18:42:09,350][105620] Updated weights for policy 1, policy_version 458836 (0.0009) [2023-12-26 18:42:09,414][105620] Updated weights for policy 1, policy_version 458846 (0.0008) [2023-12-26 18:42:09,482][105620] Updated weights for policy 1, policy_version 458856 (0.0009) [2023-12-26 18:42:09,823][105692] Updated weights for policy 0, policy_version 458509 (0.0008) [2023-12-26 18:42:09,886][105692] Updated weights for policy 0, policy_version 458519 (0.0010) [2023-12-26 18:42:09,951][105692] Updated weights for policy 0, policy_version 458529 (0.0011) [2023-12-26 18:42:10,244][105620] Updated weights for policy 1, policy_version 458866 (0.0008) [2023-12-26 18:42:10,305][105620] Updated weights for policy 1, policy_version 458876 (0.0008) [2023-12-26 18:42:10,365][105620] Updated weights for policy 1, policy_version 458886 (0.0008) [2023-12-26 18:42:10,718][105692] Updated weights for policy 0, policy_version 458539 (0.0011) [2023-12-26 18:42:10,773][105692] Updated weights for policy 0, policy_version 458549 (0.0010) [2023-12-26 18:42:10,835][105692] Updated weights for policy 0, policy_version 458559 (0.0010) [2023-12-26 18:42:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 234897408. Throughput: 0: 9885.8, 1: 9821.4. Samples: 234906332. Policy #0 lag: (min: 31.0, avg: 31.2, max: 43.0) [2023-12-26 18:42:11,062][104569] Avg episode reward: [(0, '8547.391'), (1, '9265.479')] [2023-12-26 18:42:11,141][105620] Updated weights for policy 1, policy_version 458896 (0.0009) [2023-12-26 18:42:11,204][105620] Updated weights for policy 1, policy_version 458906 (0.0009) [2023-12-26 18:42:11,267][105620] Updated weights for policy 1, policy_version 458916 (0.0008) [2023-12-26 18:42:11,626][105692] Updated weights for policy 0, policy_version 458569 (0.0010) [2023-12-26 18:42:11,695][105692] Updated weights for policy 0, policy_version 458579 (0.0011) [2023-12-26 18:42:11,766][105692] Updated weights for policy 0, policy_version 458589 (0.0010) [2023-12-26 18:42:11,828][105692] Updated weights for policy 0, policy_version 458599 (0.0011) [2023-12-26 18:42:12,075][105620] Updated weights for policy 1, policy_version 458926 (0.0008) [2023-12-26 18:42:12,138][105620] Updated weights for policy 1, policy_version 458936 (0.0008) [2023-12-26 18:42:12,193][105620] Updated weights for policy 1, policy_version 458946 (0.0008) [2023-12-26 18:42:12,557][105692] Updated weights for policy 0, policy_version 458609 (0.0006) [2023-12-26 18:42:12,604][105692] Updated weights for policy 0, policy_version 458619 (0.0005) [2023-12-26 18:42:12,669][105692] Updated weights for policy 0, policy_version 458629 (0.0006) [2023-12-26 18:42:12,966][105620] Updated weights for policy 1, policy_version 458956 (0.0009) [2023-12-26 18:42:13,017][105620] Updated weights for policy 1, policy_version 458966 (0.0008) [2023-12-26 18:42:13,075][105620] Updated weights for policy 1, policy_version 458976 (0.0007) [2023-12-26 18:42:13,331][105692] Updated weights for policy 0, policy_version 458639 (0.0005) [2023-12-26 18:42:13,390][105692] Updated weights for policy 0, policy_version 458649 (0.0005) [2023-12-26 18:42:13,460][105692] Updated weights for policy 0, policy_version 458659 (0.0005) [2023-12-26 18:42:13,682][105620] Updated weights for policy 1, policy_version 458986 (0.0006) [2023-12-26 18:42:13,730][105620] Updated weights for policy 1, policy_version 458996 (0.0005) [2023-12-26 18:42:13,779][105620] Updated weights for policy 1, policy_version 459006 (0.0005) [2023-12-26 18:42:13,830][105620] Updated weights for policy 1, policy_version 459016 (0.0005) [2023-12-26 18:42:14,066][105692] Updated weights for policy 0, policy_version 458669 (0.0007) [2023-12-26 18:42:14,117][105692] Updated weights for policy 0, policy_version 458679 (0.0011) [2023-12-26 18:42:14,117][105585] KL-divergence is very high: 143.9494 [2023-12-26 18:42:14,163][105585] KL-divergence is very high: 271.7149 [2023-12-26 18:42:14,177][105692] Updated weights for policy 0, policy_version 458689 (0.0010) [2023-12-26 18:42:14,208][105585] KL-divergence is very high: 305.2370 [2023-12-26 18:42:14,472][105620] Updated weights for policy 1, policy_version 459026 (0.0006) [2023-12-26 18:42:14,533][105620] Updated weights for policy 1, policy_version 459036 (0.0006) [2023-12-26 18:42:14,589][105620] Updated weights for policy 1, policy_version 459046 (0.0005) [2023-12-26 18:42:14,917][105692] Updated weights for policy 0, policy_version 458699 (0.0010) [2023-12-26 18:42:14,981][105692] Updated weights for policy 0, policy_version 458709 (0.0011) [2023-12-26 18:42:15,042][105692] Updated weights for policy 0, policy_version 458719 (0.0011) [2023-12-26 18:42:15,240][105620] Updated weights for policy 1, policy_version 459056 (0.0008) [2023-12-26 18:42:15,300][105620] Updated weights for policy 1, policy_version 459066 (0.0008) [2023-12-26 18:42:15,360][105620] Updated weights for policy 1, policy_version 459076 (0.0008) [2023-12-26 18:42:15,795][105692] Updated weights for policy 0, policy_version 458729 (0.0011) [2023-12-26 18:42:15,850][105692] Updated weights for policy 0, policy_version 458739 (0.0011) [2023-12-26 18:42:15,895][105692] Updated weights for policy 0, policy_version 458749 (0.0010) [2023-12-26 18:42:15,947][105692] Updated weights for policy 0, policy_version 458759 (0.0010) [2023-12-26 18:42:16,000][105620] Updated weights for policy 1, policy_version 459086 (0.0007) [2023-12-26 18:42:16,051][105620] Updated weights for policy 1, policy_version 459096 (0.0005) [2023-12-26 18:42:16,062][104569] Fps is (10 sec: 18840.7, 60 sec: 19524.1, 300 sec: 19577.5). Total num frames: 234995712. Throughput: 0: 9806.8, 1: 9771.5. Samples: 234964904. Policy #0 lag: (min: 31.0, avg: 31.2, max: 43.0) [2023-12-26 18:42:16,064][104569] Avg episode reward: [(0, '8365.758'), (1, '9265.324')] [2023-12-26 18:42:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000458760_117456896.pth... [2023-12-26 18:42:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000457608_117161984.pth [2023-12-26 18:42:16,101][105620] Updated weights for policy 1, policy_version 459106 (0.0005) [2023-12-26 18:42:16,137][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000459112_117547008.pth... [2023-12-26 18:42:16,142][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000457928_117243904.pth [2023-12-26 18:42:16,641][105620] Updated weights for policy 1, policy_version 459116 (0.0005) [2023-12-26 18:42:16,649][105692] Updated weights for policy 0, policy_version 458769 (0.0009) [2023-12-26 18:42:16,688][105620] Updated weights for policy 1, policy_version 459126 (0.0010) [2023-12-26 18:42:16,700][105692] Updated weights for policy 0, policy_version 458779 (0.0007) [2023-12-26 18:42:16,720][105585] KL-divergence is very high: 133.6223 [2023-12-26 18:42:16,734][105620] Updated weights for policy 1, policy_version 459136 (0.0010) [2023-12-26 18:42:16,760][105692] Updated weights for policy 0, policy_version 458789 (0.0006) [2023-12-26 18:42:16,766][105585] KL-divergence is very high: 128.9020 [2023-12-26 18:42:17,385][105620] Updated weights for policy 1, policy_version 459146 (0.0010) [2023-12-26 18:42:17,434][105620] Updated weights for policy 1, policy_version 459156 (0.0009) [2023-12-26 18:42:17,466][105692] Updated weights for policy 0, policy_version 458799 (0.0006) [2023-12-26 18:42:17,480][105620] Updated weights for policy 1, policy_version 459166 (0.0010) [2023-12-26 18:42:17,517][105692] Updated weights for policy 0, policy_version 458809 (0.0005) [2023-12-26 18:42:17,542][105620] Updated weights for policy 1, policy_version 459176 (0.0010) [2023-12-26 18:42:17,563][105692] Updated weights for policy 0, policy_version 458819 (0.0009) [2023-12-26 18:42:18,254][105620] Updated weights for policy 1, policy_version 459186 (0.0009) [2023-12-26 18:42:18,315][105620] Updated weights for policy 1, policy_version 459196 (0.0009) [2023-12-26 18:42:18,354][105692] Updated weights for policy 0, policy_version 458829 (0.0009) [2023-12-26 18:42:18,403][105620] Updated weights for policy 1, policy_version 459206 (0.0007) [2023-12-26 18:42:18,417][105692] Updated weights for policy 0, policy_version 458839 (0.0009) [2023-12-26 18:42:18,492][105692] Updated weights for policy 0, policy_version 458849 (0.0010) [2023-12-26 18:42:18,943][105620] Updated weights for policy 1, policy_version 459216 (0.0005) [2023-12-26 18:42:18,987][105620] Updated weights for policy 1, policy_version 459226 (0.0005) [2023-12-26 18:42:19,036][105620] Updated weights for policy 1, policy_version 459236 (0.0007) [2023-12-26 18:42:19,384][105692] Updated weights for policy 0, policy_version 458859 (0.0010) [2023-12-26 18:42:19,439][105692] Updated weights for policy 0, policy_version 458869 (0.0009) [2023-12-26 18:42:19,497][105692] Updated weights for policy 0, policy_version 458879 (0.0008) [2023-12-26 18:42:19,668][105620] Updated weights for policy 1, policy_version 459246 (0.0007) [2023-12-26 18:42:19,730][105620] Updated weights for policy 1, policy_version 459256 (0.0007) [2023-12-26 18:42:19,790][105620] Updated weights for policy 1, policy_version 459266 (0.0006) [2023-12-26 18:42:20,343][105692] Updated weights for policy 0, policy_version 458889 (0.0009) [2023-12-26 18:42:20,401][105692] Updated weights for policy 0, policy_version 458899 (0.0009) [2023-12-26 18:42:20,407][105620] Updated weights for policy 1, policy_version 459276 (0.0006) [2023-12-26 18:42:20,465][105620] Updated weights for policy 1, policy_version 459286 (0.0007) [2023-12-26 18:42:20,466][105692] Updated weights for policy 0, policy_version 458909 (0.0008) [2023-12-26 18:42:20,517][105620] Updated weights for policy 1, policy_version 459296 (0.0005) [2023-12-26 18:42:20,523][105692] Updated weights for policy 0, policy_version 458919 (0.0009) [2023-12-26 18:42:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 235094016. Throughput: 0: 9741.7, 1: 9896.4. Samples: 235085640. Policy #0 lag: (min: 31.0, avg: 31.2, max: 43.0) [2023-12-26 18:42:21,062][104569] Avg episode reward: [(0, '8815.825'), (1, '9355.818')] [2023-12-26 18:42:21,157][105620] Updated weights for policy 1, policy_version 459306 (0.0006) [2023-12-26 18:42:21,217][105620] Updated weights for policy 1, policy_version 459316 (0.0007) [2023-12-26 18:42:21,244][105692] Updated weights for policy 0, policy_version 458929 (0.0007) [2023-12-26 18:42:21,276][105620] Updated weights for policy 1, policy_version 459326 (0.0007) [2023-12-26 18:42:21,303][105692] Updated weights for policy 0, policy_version 458939 (0.0008) [2023-12-26 18:42:21,336][105620] Updated weights for policy 1, policy_version 459336 (0.0007) [2023-12-26 18:42:21,364][105692] Updated weights for policy 0, policy_version 458949 (0.0008) [2023-12-26 18:42:22,088][105692] Updated weights for policy 0, policy_version 458959 (0.0006) [2023-12-26 18:42:22,119][105620] Updated weights for policy 1, policy_version 459346 (0.0009) [2023-12-26 18:42:22,150][105692] Updated weights for policy 0, policy_version 458969 (0.0007) [2023-12-26 18:42:22,173][105620] Updated weights for policy 1, policy_version 459356 (0.0006) [2023-12-26 18:42:22,211][105692] Updated weights for policy 0, policy_version 458979 (0.0011) [2023-12-26 18:42:22,237][105620] Updated weights for policy 1, policy_version 459366 (0.0006) [2023-12-26 18:42:22,933][105692] Updated weights for policy 0, policy_version 458989 (0.0010) [2023-12-26 18:42:23,000][105692] Updated weights for policy 0, policy_version 458999 (0.0009) [2023-12-26 18:42:23,015][105620] Updated weights for policy 1, policy_version 459376 (0.0006) [2023-12-26 18:42:23,059][105692] Updated weights for policy 0, policy_version 459009 (0.0007) [2023-12-26 18:42:23,066][105620] Updated weights for policy 1, policy_version 459386 (0.0006) [2023-12-26 18:42:23,119][105620] Updated weights for policy 1, policy_version 459396 (0.0006) [2023-12-26 18:42:23,790][105620] Updated weights for policy 1, policy_version 459406 (0.0006) [2023-12-26 18:42:23,847][105620] Updated weights for policy 1, policy_version 459416 (0.0005) [2023-12-26 18:42:23,848][105692] Updated weights for policy 0, policy_version 459019 (0.0009) [2023-12-26 18:42:23,906][105620] Updated weights for policy 1, policy_version 459426 (0.0006) [2023-12-26 18:42:23,908][105692] Updated weights for policy 0, policy_version 459029 (0.0008) [2023-12-26 18:42:23,967][105692] Updated weights for policy 0, policy_version 459039 (0.0009) [2023-12-26 18:42:24,622][105620] Updated weights for policy 1, policy_version 459436 (0.0007) [2023-12-26 18:42:24,676][105620] Updated weights for policy 1, policy_version 459446 (0.0009) [2023-12-26 18:42:24,692][105692] Updated weights for policy 0, policy_version 459049 (0.0009) [2023-12-26 18:42:24,723][105620] Updated weights for policy 1, policy_version 459456 (0.0007) [2023-12-26 18:42:24,753][105692] Updated weights for policy 0, policy_version 459059 (0.0009) [2023-12-26 18:42:24,817][105692] Updated weights for policy 0, policy_version 459069 (0.0009) [2023-12-26 18:42:24,880][105692] Updated weights for policy 0, policy_version 459079 (0.0009) [2023-12-26 18:42:25,417][105620] Updated weights for policy 1, policy_version 459466 (0.0007) [2023-12-26 18:42:25,464][105620] Updated weights for policy 1, policy_version 459476 (0.0009) [2023-12-26 18:42:25,514][105620] Updated weights for policy 1, policy_version 459486 (0.0009) [2023-12-26 18:42:25,561][105692] Updated weights for policy 0, policy_version 459089 (0.0007) [2023-12-26 18:42:25,563][105620] Updated weights for policy 1, policy_version 459496 (0.0007) [2023-12-26 18:42:25,607][105692] Updated weights for policy 0, policy_version 459099 (0.0008) [2023-12-26 18:42:25,653][105692] Updated weights for policy 0, policy_version 459109 (0.0009) [2023-12-26 18:42:26,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 235192320. Throughput: 0: 9657.5, 1: 9936.7. Samples: 235200736. Policy #0 lag: (min: 31.0, avg: 31.2, max: 43.0) [2023-12-26 18:42:26,063][104569] Avg episode reward: [(0, '8905.492'), (1, '9355.855')] [2023-12-26 18:42:26,232][105620] Updated weights for policy 1, policy_version 459506 (0.0008) [2023-12-26 18:42:26,279][105692] Updated weights for policy 0, policy_version 459119 (0.0010) [2023-12-26 18:42:26,284][105620] Updated weights for policy 1, policy_version 459516 (0.0007) [2023-12-26 18:42:26,329][105620] Updated weights for policy 1, policy_version 459526 (0.0005) [2023-12-26 18:42:26,335][105692] Updated weights for policy 0, policy_version 459129 (0.0009) [2023-12-26 18:42:26,393][105692] Updated weights for policy 0, policy_version 459139 (0.0009) [2023-12-26 18:42:27,063][105620] Updated weights for policy 1, policy_version 459536 (0.0008) [2023-12-26 18:42:27,125][105620] Updated weights for policy 1, policy_version 459546 (0.0007) [2023-12-26 18:42:27,150][105692] Updated weights for policy 0, policy_version 459149 (0.0009) [2023-12-26 18:42:27,188][105620] Updated weights for policy 1, policy_version 459556 (0.0005) [2023-12-26 18:42:27,209][105692] Updated weights for policy 0, policy_version 459159 (0.0009) [2023-12-26 18:42:27,272][105692] Updated weights for policy 0, policy_version 459169 (0.0008) [2023-12-26 18:42:27,739][105620] Updated weights for policy 1, policy_version 459566 (0.0006) [2023-12-26 18:42:27,792][105620] Updated weights for policy 1, policy_version 459576 (0.0005) [2023-12-26 18:42:27,844][105620] Updated weights for policy 1, policy_version 459586 (0.0009) [2023-12-26 18:42:28,099][105692] Updated weights for policy 0, policy_version 459179 (0.0008) [2023-12-26 18:42:28,153][105692] Updated weights for policy 0, policy_version 459189 (0.0008) [2023-12-26 18:42:28,204][105692] Updated weights for policy 0, policy_version 459199 (0.0008) [2023-12-26 18:42:28,549][105620] Updated weights for policy 1, policy_version 459596 (0.0010) [2023-12-26 18:42:28,614][105620] Updated weights for policy 1, policy_version 459606 (0.0010) [2023-12-26 18:42:28,658][105620] Updated weights for policy 1, policy_version 459616 (0.0010) [2023-12-26 18:42:28,971][105692] Updated weights for policy 0, policy_version 459209 (0.0008) [2023-12-26 18:42:29,025][105692] Updated weights for policy 0, policy_version 459219 (0.0008) [2023-12-26 18:42:29,072][105692] Updated weights for policy 0, policy_version 459229 (0.0008) [2023-12-26 18:42:29,115][105692] Updated weights for policy 0, policy_version 459239 (0.0007) [2023-12-26 18:42:29,383][105620] Updated weights for policy 1, policy_version 459626 (0.0010) [2023-12-26 18:42:29,435][105620] Updated weights for policy 1, policy_version 459636 (0.0006) [2023-12-26 18:42:29,492][105620] Updated weights for policy 1, policy_version 459646 (0.0009) [2023-12-26 18:42:29,557][105620] Updated weights for policy 1, policy_version 459656 (0.0010) [2023-12-26 18:42:29,986][105692] Updated weights for policy 0, policy_version 459249 (0.0010) [2023-12-26 18:42:30,044][105692] Updated weights for policy 0, policy_version 459259 (0.0010) [2023-12-26 18:42:30,059][105585] KL-divergence is very high: 154.7943 [2023-12-26 18:42:30,100][105692] Updated weights for policy 0, policy_version 459269 (0.0009) [2023-12-26 18:42:30,107][105585] KL-divergence is very high: 167.0037 [2023-12-26 18:42:30,161][105620] Updated weights for policy 1, policy_version 459666 (0.0005) [2023-12-26 18:42:30,222][105620] Updated weights for policy 1, policy_version 459676 (0.0005) [2023-12-26 18:42:30,281][105620] Updated weights for policy 1, policy_version 459686 (0.0005) [2023-12-26 18:42:30,862][105620] Updated weights for policy 1, policy_version 459696 (0.0009) [2023-12-26 18:42:30,904][105620] Updated weights for policy 1, policy_version 459706 (0.0007) [2023-12-26 18:42:30,937][105692] Updated weights for policy 0, policy_version 459279 (0.0008) [2023-12-26 18:42:30,950][105620] Updated weights for policy 1, policy_version 459716 (0.0009) [2023-12-26 18:42:30,995][105692] Updated weights for policy 0, policy_version 459289 (0.0006) [2023-12-26 18:42:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 235290624. Throughput: 0: 9644.6, 1: 10026.5. Samples: 235260932. Policy #0 lag: (min: 4.0, avg: 11.1, max: 36.0) [2023-12-26 18:42:31,062][104569] Avg episode reward: [(0, '8458.401'), (1, '9174.030')] [2023-12-26 18:42:31,064][105692] Updated weights for policy 0, policy_version 459299 (0.0007) [2023-12-26 18:42:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000459720_117702656.pth... [2023-12-26 18:42:31,069][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000458536_117399552.pth [2023-12-26 18:42:31,092][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000459304_117596160.pth... [2023-12-26 18:42:31,096][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000458184_117309440.pth [2023-12-26 18:42:31,706][105620] Updated weights for policy 1, policy_version 459726 (0.0007) [2023-12-26 18:42:31,769][105620] Updated weights for policy 1, policy_version 459736 (0.0007) [2023-12-26 18:42:31,827][105620] Updated weights for policy 1, policy_version 459746 (0.0006) [2023-12-26 18:42:31,847][105692] Updated weights for policy 0, policy_version 459309 (0.0007) [2023-12-26 18:42:31,904][105692] Updated weights for policy 0, policy_version 459319 (0.0008) [2023-12-26 18:42:31,962][105692] Updated weights for policy 0, policy_version 459329 (0.0008) [2023-12-26 18:42:32,398][105620] Updated weights for policy 1, policy_version 459756 (0.0007) [2023-12-26 18:42:32,456][105620] Updated weights for policy 1, policy_version 459766 (0.0008) [2023-12-26 18:42:32,507][105620] Updated weights for policy 1, policy_version 459776 (0.0009) [2023-12-26 18:42:32,774][105692] Updated weights for policy 0, policy_version 459339 (0.0009) [2023-12-26 18:42:32,826][105692] Updated weights for policy 0, policy_version 459349 (0.0008) [2023-12-26 18:42:32,881][105692] Updated weights for policy 0, policy_version 459359 (0.0008) [2023-12-26 18:42:33,203][105620] Updated weights for policy 1, policy_version 459787 (0.0009) [2023-12-26 18:42:33,258][105620] Updated weights for policy 1, policy_version 459797 (0.0010) [2023-12-26 18:42:33,312][105620] Updated weights for policy 1, policy_version 459807 (0.0010) [2023-12-26 18:42:33,679][105692] Updated weights for policy 0, policy_version 459369 (0.0009) [2023-12-26 18:42:33,733][105692] Updated weights for policy 0, policy_version 459379 (0.0007) [2023-12-26 18:42:33,783][105692] Updated weights for policy 0, policy_version 459389 (0.0008) [2023-12-26 18:42:33,830][105692] Updated weights for policy 0, policy_version 459399 (0.0008) [2023-12-26 18:42:34,058][105620] Updated weights for policy 1, policy_version 459817 (0.0010) [2023-12-26 18:42:34,113][105620] Updated weights for policy 1, policy_version 459827 (0.0010) [2023-12-26 18:42:34,174][105620] Updated weights for policy 1, policy_version 459837 (0.0010) [2023-12-26 18:42:34,222][105620] Updated weights for policy 1, policy_version 459847 (0.0009) [2023-12-26 18:42:34,590][105692] Updated weights for policy 0, policy_version 459409 (0.0008) [2023-12-26 18:42:34,651][105692] Updated weights for policy 0, policy_version 459419 (0.0008) [2023-12-26 18:42:34,716][105692] Updated weights for policy 0, policy_version 459429 (0.0009) [2023-12-26 18:42:35,002][105620] Updated weights for policy 1, policy_version 459857 (0.0011) [2023-12-26 18:42:35,062][105620] Updated weights for policy 1, policy_version 459867 (0.0011) [2023-12-26 18:42:35,125][105620] Updated weights for policy 1, policy_version 459877 (0.0011) [2023-12-26 18:42:35,475][105692] Updated weights for policy 0, policy_version 459439 (0.0008) [2023-12-26 18:42:35,525][105692] Updated weights for policy 0, policy_version 459449 (0.0007) [2023-12-26 18:42:35,584][105692] Updated weights for policy 0, policy_version 459459 (0.0008) [2023-12-26 18:42:35,862][105620] Updated weights for policy 1, policy_version 459887 (0.0010) [2023-12-26 18:42:35,910][105620] Updated weights for policy 1, policy_version 459897 (0.0010) [2023-12-26 18:42:35,955][105620] Updated weights for policy 1, policy_version 459907 (0.0007) [2023-12-26 18:42:36,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.4, 300 sec: 19522.0). Total num frames: 235388928. Throughput: 0: 9472.7, 1: 10155.0. Samples: 235376004. Policy #0 lag: (min: 4.0, avg: 11.1, max: 36.0) [2023-12-26 18:42:36,062][104569] Avg episode reward: [(0, '8369.778'), (1, '9173.986')] [2023-12-26 18:42:36,332][105692] Updated weights for policy 0, policy_version 459469 (0.0008) [2023-12-26 18:42:36,400][105692] Updated weights for policy 0, policy_version 459479 (0.0008) [2023-12-26 18:42:36,464][105692] Updated weights for policy 0, policy_version 459489 (0.0008) [2023-12-26 18:42:36,678][105620] Updated weights for policy 1, policy_version 459917 (0.0008) [2023-12-26 18:42:36,745][105620] Updated weights for policy 1, policy_version 459927 (0.0011) [2023-12-26 18:42:36,801][105620] Updated weights for policy 1, policy_version 459937 (0.0011) [2023-12-26 18:42:37,220][105692] Updated weights for policy 0, policy_version 459499 (0.0008) [2023-12-26 18:42:37,283][105692] Updated weights for policy 0, policy_version 459509 (0.0008) [2023-12-26 18:42:37,335][105692] Updated weights for policy 0, policy_version 459519 (0.0008) [2023-12-26 18:42:37,554][105620] Updated weights for policy 1, policy_version 459947 (0.0011) [2023-12-26 18:42:37,610][105620] Updated weights for policy 1, policy_version 459957 (0.0011) [2023-12-26 18:42:37,664][105620] Updated weights for policy 1, policy_version 459967 (0.0010) [2023-12-26 18:42:38,084][105692] Updated weights for policy 0, policy_version 459529 (0.0008) [2023-12-26 18:42:38,147][105692] Updated weights for policy 0, policy_version 459539 (0.0008) [2023-12-26 18:42:38,214][105692] Updated weights for policy 0, policy_version 459549 (0.0008) [2023-12-26 18:42:38,277][105692] Updated weights for policy 0, policy_version 459559 (0.0008) [2023-12-26 18:42:38,454][105620] Updated weights for policy 1, policy_version 459977 (0.0010) [2023-12-26 18:42:38,496][105586] KL-divergence is very high: 116.4889 [2023-12-26 18:42:38,500][105620] Updated weights for policy 1, policy_version 459987 (0.0005) [2023-12-26 18:42:38,501][105586] KL-divergence is very high: 101.5630 [2023-12-26 18:42:38,557][105620] Updated weights for policy 1, policy_version 459997 (0.0005) [2023-12-26 18:42:38,625][105620] Updated weights for policy 1, policy_version 460007 (0.0006) [2023-12-26 18:42:39,034][105692] Updated weights for policy 0, policy_version 459569 (0.0005) [2023-12-26 18:42:39,082][105692] Updated weights for policy 0, policy_version 459579 (0.0005) [2023-12-26 18:42:39,129][105692] Updated weights for policy 0, policy_version 459589 (0.0006) [2023-12-26 18:42:39,312][105620] Updated weights for policy 1, policy_version 460017 (0.0008) [2023-12-26 18:42:39,380][105620] Updated weights for policy 1, policy_version 460027 (0.0009) [2023-12-26 18:42:39,445][105620] Updated weights for policy 1, policy_version 460037 (0.0009) [2023-12-26 18:42:39,891][105692] Updated weights for policy 0, policy_version 459599 (0.0008) [2023-12-26 18:42:39,957][105692] Updated weights for policy 0, policy_version 459609 (0.0008) [2023-12-26 18:42:40,022][105692] Updated weights for policy 0, policy_version 459619 (0.0008) [2023-12-26 18:42:40,112][105620] Updated weights for policy 1, policy_version 460047 (0.0009) [2023-12-26 18:42:40,176][105620] Updated weights for policy 1, policy_version 460057 (0.0011) [2023-12-26 18:42:40,235][105620] Updated weights for policy 1, policy_version 460067 (0.0010) [2023-12-26 18:42:40,804][105692] Updated weights for policy 0, policy_version 459629 (0.0008) [2023-12-26 18:42:40,862][105692] Updated weights for policy 0, policy_version 459639 (0.0005) [2023-12-26 18:42:40,894][105620] Updated weights for policy 1, policy_version 460077 (0.0009) [2023-12-26 18:42:40,924][105692] Updated weights for policy 0, policy_version 459649 (0.0007) [2023-12-26 18:42:40,948][105620] Updated weights for policy 1, policy_version 460087 (0.0009) [2023-12-26 18:42:41,003][105620] Updated weights for policy 1, policy_version 460097 (0.0009) [2023-12-26 18:42:41,022][105586] KL-divergence is very high: 103.3646 [2023-12-26 18:42:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 235487232. Throughput: 0: 9409.3, 1: 10073.7. Samples: 235489312. Policy #0 lag: (min: 4.0, avg: 11.1, max: 36.0) [2023-12-26 18:42:41,062][104569] Avg episode reward: [(0, '8634.022'), (1, '1956.651')] [2023-12-26 18:42:41,581][105692] Updated weights for policy 0, policy_version 459659 (0.0009) [2023-12-26 18:42:41,649][105692] Updated weights for policy 0, policy_version 459669 (0.0010) [2023-12-26 18:42:41,712][105692] Updated weights for policy 0, policy_version 459679 (0.0009) [2023-12-26 18:42:41,787][105620] Updated weights for policy 1, policy_version 460107 (0.0009) [2023-12-26 18:42:41,852][105620] Updated weights for policy 1, policy_version 460117 (0.0008) [2023-12-26 18:42:41,918][105620] Updated weights for policy 1, policy_version 460127 (0.0009) [2023-12-26 18:42:42,444][105692] Updated weights for policy 0, policy_version 459689 (0.0008) [2023-12-26 18:42:42,499][105692] Updated weights for policy 0, policy_version 459699 (0.0009) [2023-12-26 18:42:42,559][105692] Updated weights for policy 0, policy_version 459709 (0.0009) [2023-12-26 18:42:42,626][105692] Updated weights for policy 0, policy_version 459719 (0.0006) [2023-12-26 18:42:42,690][105620] Updated weights for policy 1, policy_version 460137 (0.0009) [2023-12-26 18:42:42,748][105620] Updated weights for policy 1, policy_version 460147 (0.0010) [2023-12-26 18:42:42,807][105620] Updated weights for policy 1, policy_version 460157 (0.0010) [2023-12-26 18:42:42,874][105620] Updated weights for policy 1, policy_version 460167 (0.0010) [2023-12-26 18:42:43,248][105692] Updated weights for policy 0, policy_version 459729 (0.0009) [2023-12-26 18:42:43,299][105692] Updated weights for policy 0, policy_version 459739 (0.0009) [2023-12-26 18:42:43,347][105692] Updated weights for policy 0, policy_version 459749 (0.0009) [2023-12-26 18:42:43,711][105620] Updated weights for policy 1, policy_version 460177 (0.0008) [2023-12-26 18:42:43,759][105620] Updated weights for policy 1, policy_version 460187 (0.0008) [2023-12-26 18:42:43,818][105620] Updated weights for policy 1, policy_version 460197 (0.0008) [2023-12-26 18:42:44,050][105692] Updated weights for policy 0, policy_version 459759 (0.0010) [2023-12-26 18:42:44,108][105692] Updated weights for policy 0, policy_version 459769 (0.0010) [2023-12-26 18:42:44,162][105692] Updated weights for policy 0, policy_version 459779 (0.0010) [2023-12-26 18:42:44,535][105620] Updated weights for policy 1, policy_version 460207 (0.0007) [2023-12-26 18:42:44,596][105620] Updated weights for policy 1, policy_version 460217 (0.0009) [2023-12-26 18:42:44,662][105620] Updated weights for policy 1, policy_version 460227 (0.0009) [2023-12-26 18:42:44,782][105692] Updated weights for policy 0, policy_version 459789 (0.0009) [2023-12-26 18:42:44,844][105692] Updated weights for policy 0, policy_version 459799 (0.0009) [2023-12-26 18:42:44,906][105692] Updated weights for policy 0, policy_version 459809 (0.0010) [2023-12-26 18:42:45,388][105620] Updated weights for policy 1, policy_version 460237 (0.0009) [2023-12-26 18:42:45,446][105620] Updated weights for policy 1, policy_version 460247 (0.0008) [2023-12-26 18:42:45,499][105620] Updated weights for policy 1, policy_version 460257 (0.0008) [2023-12-26 18:42:45,514][105692] Updated weights for policy 0, policy_version 459819 (0.0010) [2023-12-26 18:42:45,578][105692] Updated weights for policy 0, policy_version 459829 (0.0006) [2023-12-26 18:42:45,639][105692] Updated weights for policy 0, policy_version 459839 (0.0006) [2023-12-26 18:42:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 235577344. Throughput: 0: 9386.7, 1: 9972.4. Samples: 235545680. Policy #0 lag: (min: 4.0, avg: 11.1, max: 36.0) [2023-12-26 18:42:46,062][104569] Avg episode reward: [(0, '8451.540'), (1, '2171.849')] [2023-12-26 18:42:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000459848_117735424.pth... [2023-12-26 18:42:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000460264_117841920.pth... [2023-12-26 18:42:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000458760_117456896.pth [2023-12-26 18:42:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000459112_117547008.pth [2023-12-26 18:42:46,198][105692] Updated weights for policy 0, policy_version 459849 (0.0008) [2023-12-26 18:42:46,250][105620] Updated weights for policy 1, policy_version 460267 (0.0009) [2023-12-26 18:42:46,256][105692] Updated weights for policy 0, policy_version 459859 (0.0010) [2023-12-26 18:42:46,309][105620] Updated weights for policy 1, policy_version 460277 (0.0010) [2023-12-26 18:42:46,316][105692] Updated weights for policy 0, policy_version 459869 (0.0009) [2023-12-26 18:42:46,361][105620] Updated weights for policy 1, policy_version 460287 (0.0010) [2023-12-26 18:42:46,379][105692] Updated weights for policy 0, policy_version 459879 (0.0006) [2023-12-26 18:42:46,918][105620] Updated weights for policy 1, policy_version 460297 (0.0007) [2023-12-26 18:42:46,944][105692] Updated weights for policy 0, policy_version 459889 (0.0008) [2023-12-26 18:42:46,976][105620] Updated weights for policy 1, policy_version 460307 (0.0008) [2023-12-26 18:42:47,002][105692] Updated weights for policy 0, policy_version 459899 (0.0010) [2023-12-26 18:42:47,024][105620] Updated weights for policy 1, policy_version 460317 (0.0010) [2023-12-26 18:42:47,061][105692] Updated weights for policy 0, policy_version 459909 (0.0010) [2023-12-26 18:42:47,081][105620] Updated weights for policy 1, policy_version 460327 (0.0009) [2023-12-26 18:42:47,756][105692] Updated weights for policy 0, policy_version 459919 (0.0008) [2023-12-26 18:42:47,783][105585] KL-divergence is very high: 100.0775 [2023-12-26 18:42:47,784][105620] Updated weights for policy 1, policy_version 460337 (0.0010) [2023-12-26 18:42:47,811][105692] Updated weights for policy 0, policy_version 459929 (0.0010) [2023-12-26 18:42:47,825][105585] KL-divergence is very high: 122.5784 [2023-12-26 18:42:47,828][105620] Updated weights for policy 1, policy_version 460347 (0.0010) [2023-12-26 18:42:47,856][105692] Updated weights for policy 0, policy_version 459939 (0.0010) [2023-12-26 18:42:47,861][105585] KL-divergence is very high: 106.8408 [2023-12-26 18:42:47,880][105620] Updated weights for policy 1, policy_version 460357 (0.0010) [2023-12-26 18:42:48,546][105692] Updated weights for policy 0, policy_version 459949 (0.0010) [2023-12-26 18:42:48,609][105692] Updated weights for policy 0, policy_version 459959 (0.0011) [2023-12-26 18:42:48,628][105620] Updated weights for policy 1, policy_version 460367 (0.0011) [2023-12-26 18:42:48,668][105692] Updated weights for policy 0, policy_version 459969 (0.0010) [2023-12-26 18:42:48,684][105620] Updated weights for policy 1, policy_version 460377 (0.0010) [2023-12-26 18:42:48,744][105620] Updated weights for policy 1, policy_version 460387 (0.0011) [2023-12-26 18:42:49,388][105692] Updated weights for policy 0, policy_version 459979 (0.0010) [2023-12-26 18:42:49,454][105692] Updated weights for policy 0, policy_version 459989 (0.0010) [2023-12-26 18:42:49,488][105620] Updated weights for policy 1, policy_version 460397 (0.0011) [2023-12-26 18:42:49,510][105692] Updated weights for policy 0, policy_version 459999 (0.0006) [2023-12-26 18:42:49,550][105620] Updated weights for policy 1, policy_version 460407 (0.0011) [2023-12-26 18:42:49,615][105620] Updated weights for policy 1, policy_version 460417 (0.0010) [2023-12-26 18:42:50,178][105692] Updated weights for policy 0, policy_version 460009 (0.0008) [2023-12-26 18:42:50,244][105692] Updated weights for policy 0, policy_version 460019 (0.0011) [2023-12-26 18:42:50,264][105620] Updated weights for policy 1, policy_version 460427 (0.0008) [2023-12-26 18:42:50,293][105692] Updated weights for policy 0, policy_version 460029 (0.0007) [2023-12-26 18:42:50,328][105620] Updated weights for policy 1, policy_version 460437 (0.0010) [2023-12-26 18:42:50,356][105692] Updated weights for policy 0, policy_version 460039 (0.0011) [2023-12-26 18:42:50,383][105620] Updated weights for policy 1, policy_version 460447 (0.0008) [2023-12-26 18:42:51,061][105692] Updated weights for policy 0, policy_version 460049 (0.0011) [2023-12-26 18:42:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 235675648. Throughput: 0: 9508.1, 1: 9961.9. Samples: 235668612. Policy #0 lag: (min: 4.0, avg: 11.1, max: 36.0) [2023-12-26 18:42:51,062][104569] Avg episode reward: [(0, '8284.291'), (1, '6757.598')] [2023-12-26 18:42:51,120][105620] Updated weights for policy 1, policy_version 460457 (0.0008) [2023-12-26 18:42:51,123][105692] Updated weights for policy 0, policy_version 460059 (0.0008) [2023-12-26 18:42:51,177][105620] Updated weights for policy 1, policy_version 460467 (0.0006) [2023-12-26 18:42:51,179][105692] Updated weights for policy 0, policy_version 460069 (0.0006) [2023-12-26 18:42:51,236][105620] Updated weights for policy 1, policy_version 460477 (0.0008) [2023-12-26 18:42:51,301][105620] Updated weights for policy 1, policy_version 460487 (0.0009) [2023-12-26 18:42:51,898][105692] Updated weights for policy 0, policy_version 460079 (0.0007) [2023-12-26 18:42:51,949][105692] Updated weights for policy 0, policy_version 460089 (0.0009) [2023-12-26 18:42:51,998][105692] Updated weights for policy 0, policy_version 460099 (0.0008) [2023-12-26 18:42:52,028][105620] Updated weights for policy 1, policy_version 460497 (0.0009) [2023-12-26 18:42:52,089][105620] Updated weights for policy 1, policy_version 460507 (0.0007) [2023-12-26 18:42:52,159][105620] Updated weights for policy 1, policy_version 460517 (0.0006) [2023-12-26 18:42:52,780][105692] Updated weights for policy 0, policy_version 460109 (0.0009) [2023-12-26 18:42:52,832][105692] Updated weights for policy 0, policy_version 460119 (0.0009) [2023-12-26 18:42:52,867][105585] KL-divergence is very high: 168.5431 [2023-12-26 18:42:52,879][105620] Updated weights for policy 1, policy_version 460527 (0.0010) [2023-12-26 18:42:52,884][105585] KL-divergence is very high: 120.6786 [2023-12-26 18:42:52,886][105692] Updated weights for policy 0, policy_version 460129 (0.0008) [2023-12-26 18:42:52,910][105585] KL-divergence is very high: 165.7190 [2023-12-26 18:42:52,939][105620] Updated weights for policy 1, policy_version 460537 (0.0007) [2023-12-26 18:42:53,000][105620] Updated weights for policy 1, policy_version 460547 (0.0009) [2023-12-26 18:42:53,521][105692] Updated weights for policy 0, policy_version 460139 (0.0008) [2023-12-26 18:42:53,583][105692] Updated weights for policy 0, policy_version 460149 (0.0005) [2023-12-26 18:42:53,643][105692] Updated weights for policy 0, policy_version 460159 (0.0006) [2023-12-26 18:42:53,868][105620] Updated weights for policy 1, policy_version 460557 (0.0010) [2023-12-26 18:42:53,920][105620] Updated weights for policy 1, policy_version 460567 (0.0009) [2023-12-26 18:42:53,973][105620] Updated weights for policy 1, policy_version 460577 (0.0010) [2023-12-26 18:42:54,206][105692] Updated weights for policy 0, policy_version 460169 (0.0005) [2023-12-26 18:42:54,256][105692] Updated weights for policy 0, policy_version 460179 (0.0009) [2023-12-26 18:42:54,322][105692] Updated weights for policy 0, policy_version 460189 (0.0010) [2023-12-26 18:42:54,328][105585] KL-divergence is very high: 104.4999 [2023-12-26 18:42:54,375][105585] KL-divergence is very high: 144.4136 [2023-12-26 18:42:54,380][105692] Updated weights for policy 0, policy_version 460199 (0.0009) [2023-12-26 18:42:54,860][105620] Updated weights for policy 1, policy_version 460587 (0.0008) [2023-12-26 18:42:54,909][105620] Updated weights for policy 1, policy_version 460597 (0.0008) [2023-12-26 18:42:54,954][105692] Updated weights for policy 0, policy_version 460209 (0.0010) [2023-12-26 18:42:54,969][105620] Updated weights for policy 1, policy_version 460607 (0.0007) [2023-12-26 18:42:55,001][105692] Updated weights for policy 0, policy_version 460219 (0.0010) [2023-12-26 18:42:55,045][105692] Updated weights for policy 0, policy_version 460229 (0.0009) [2023-12-26 18:42:55,562][105620] Updated weights for policy 1, policy_version 460617 (0.0006) [2023-12-26 18:42:55,625][105620] Updated weights for policy 1, policy_version 460627 (0.0005) [2023-12-26 18:42:55,680][105620] Updated weights for policy 1, policy_version 460637 (0.0005) [2023-12-26 18:42:55,688][105692] Updated weights for policy 0, policy_version 460239 (0.0007) [2023-12-26 18:42:55,738][105620] Updated weights for policy 1, policy_version 460647 (0.0005) [2023-12-26 18:42:55,747][105692] Updated weights for policy 0, policy_version 460249 (0.0005) [2023-12-26 18:42:55,795][105692] Updated weights for policy 0, policy_version 460259 (0.0006) [2023-12-26 18:42:56,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 235782144. Throughput: 0: 9653.8, 1: 9954.2. Samples: 235788696. Policy #0 lag: (min: 4.0, avg: 11.1, max: 36.0) [2023-12-26 18:42:56,063][104569] Avg episode reward: [(0, '8470.490'), (1, '9082.588')] [2023-12-26 18:42:56,286][105620] Updated weights for policy 1, policy_version 460657 (0.0005) [2023-12-26 18:42:56,342][105620] Updated weights for policy 1, policy_version 460667 (0.0005) [2023-12-26 18:42:56,344][105692] Updated weights for policy 0, policy_version 460269 (0.0008) [2023-12-26 18:42:56,390][105620] Updated weights for policy 1, policy_version 460677 (0.0005) [2023-12-26 18:42:56,400][105692] Updated weights for policy 0, policy_version 460279 (0.0007) [2023-12-26 18:42:56,454][105692] Updated weights for policy 0, policy_version 460289 (0.0006) [2023-12-26 18:42:57,000][105692] Updated weights for policy 0, policy_version 460299 (0.0005) [2023-12-26 18:42:57,047][105620] Updated weights for policy 1, policy_version 460687 (0.0007) [2023-12-26 18:42:57,059][105692] Updated weights for policy 0, policy_version 460309 (0.0007) [2023-12-26 18:42:57,092][105620] Updated weights for policy 1, policy_version 460697 (0.0008) [2023-12-26 18:42:57,112][105692] Updated weights for policy 0, policy_version 460319 (0.0007) [2023-12-26 18:42:57,143][105620] Updated weights for policy 1, policy_version 460707 (0.0006) [2023-12-26 18:42:57,794][105620] Updated weights for policy 1, policy_version 460717 (0.0009) [2023-12-26 18:42:57,831][105692] Updated weights for policy 0, policy_version 460329 (0.0006) [2023-12-26 18:42:57,848][105620] Updated weights for policy 1, policy_version 460727 (0.0008) [2023-12-26 18:42:57,882][105692] Updated weights for policy 0, policy_version 460339 (0.0006) [2023-12-26 18:42:57,912][105620] Updated weights for policy 1, policy_version 460737 (0.0007) [2023-12-26 18:42:57,934][105692] Updated weights for policy 0, policy_version 460349 (0.0007) [2023-12-26 18:42:57,986][105692] Updated weights for policy 0, policy_version 460359 (0.0008) [2023-12-26 18:42:58,664][105620] Updated weights for policy 1, policy_version 460747 (0.0007) [2023-12-26 18:42:58,735][105620] Updated weights for policy 1, policy_version 460757 (0.0009) [2023-12-26 18:42:58,799][105620] Updated weights for policy 1, policy_version 460767 (0.0009) [2023-12-26 18:42:58,833][105692] Updated weights for policy 0, policy_version 460369 (0.0008) [2023-12-26 18:42:58,888][105692] Updated weights for policy 0, policy_version 460379 (0.0007) [2023-12-26 18:42:58,945][105692] Updated weights for policy 0, policy_version 460389 (0.0005) [2023-12-26 18:42:59,571][105692] Updated weights for policy 0, policy_version 460399 (0.0009) [2023-12-26 18:42:59,609][105620] Updated weights for policy 1, policy_version 460777 (0.0007) [2023-12-26 18:42:59,624][105692] Updated weights for policy 0, policy_version 460409 (0.0007) [2023-12-26 18:42:59,660][105620] Updated weights for policy 1, policy_version 460787 (0.0006) [2023-12-26 18:42:59,684][105692] Updated weights for policy 0, policy_version 460419 (0.0007) [2023-12-26 18:42:59,706][105620] Updated weights for policy 1, policy_version 460797 (0.0006) [2023-12-26 18:42:59,754][105620] Updated weights for policy 1, policy_version 460807 (0.0006) [2023-12-26 18:43:00,435][105692] Updated weights for policy 0, policy_version 460429 (0.0008) [2023-12-26 18:43:00,484][105692] Updated weights for policy 0, policy_version 460439 (0.0009) [2023-12-26 18:43:00,526][105620] Updated weights for policy 1, policy_version 460817 (0.0005) [2023-12-26 18:43:00,536][105692] Updated weights for policy 0, policy_version 460449 (0.0008) [2023-12-26 18:43:00,588][105620] Updated weights for policy 1, policy_version 460827 (0.0007) [2023-12-26 18:43:00,645][105620] Updated weights for policy 1, policy_version 460837 (0.0008) [2023-12-26 18:43:01,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 235880448. Throughput: 0: 9714.6, 1: 9982.2. Samples: 235851256. Policy #0 lag: (min: 4.0, avg: 11.1, max: 36.0) [2023-12-26 18:43:01,063][104569] Avg episode reward: [(0, '8027.105'), (1, '9172.774')] [2023-12-26 18:43:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000460456_117891072.pth... [2023-12-26 18:43:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000460840_117989376.pth... [2023-12-26 18:43:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000459304_117596160.pth [2023-12-26 18:43:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000459720_117702656.pth [2023-12-26 18:43:01,298][105620] Updated weights for policy 1, policy_version 460847 (0.0009) [2023-12-26 18:43:01,357][105620] Updated weights for policy 1, policy_version 460857 (0.0009) [2023-12-26 18:43:01,383][105692] Updated weights for policy 0, policy_version 460459 (0.0008) [2023-12-26 18:43:01,412][105620] Updated weights for policy 1, policy_version 460867 (0.0007) [2023-12-26 18:43:01,443][105692] Updated weights for policy 0, policy_version 460469 (0.0008) [2023-12-26 18:43:01,504][105692] Updated weights for policy 0, policy_version 460479 (0.0010) [2023-12-26 18:43:02,026][105620] Updated weights for policy 1, policy_version 460877 (0.0005) [2023-12-26 18:43:02,091][105620] Updated weights for policy 1, policy_version 460887 (0.0006) [2023-12-26 18:43:02,156][105620] Updated weights for policy 1, policy_version 460897 (0.0006) [2023-12-26 18:43:02,267][105692] Updated weights for policy 0, policy_version 460489 (0.0008) [2023-12-26 18:43:02,326][105692] Updated weights for policy 0, policy_version 460499 (0.0008) [2023-12-26 18:43:02,387][105692] Updated weights for policy 0, policy_version 460509 (0.0009) [2023-12-26 18:43:02,439][105692] Updated weights for policy 0, policy_version 460519 (0.0008) [2023-12-26 18:43:02,750][105620] Updated weights for policy 1, policy_version 460907 (0.0006) [2023-12-26 18:43:02,806][105620] Updated weights for policy 1, policy_version 460917 (0.0005) [2023-12-26 18:43:02,855][105620] Updated weights for policy 1, policy_version 460927 (0.0005) [2023-12-26 18:43:03,074][105692] Updated weights for policy 0, policy_version 460529 (0.0006) [2023-12-26 18:43:03,127][105692] Updated weights for policy 0, policy_version 460539 (0.0005) [2023-12-26 18:43:03,178][105692] Updated weights for policy 0, policy_version 460549 (0.0005) [2023-12-26 18:43:03,497][105620] Updated weights for policy 1, policy_version 460937 (0.0006) [2023-12-26 18:43:03,549][105620] Updated weights for policy 1, policy_version 460948 (0.0010) [2023-12-26 18:43:03,603][105620] Updated weights for policy 1, policy_version 460961 (0.0010) [2023-12-26 18:43:03,709][105692] Updated weights for policy 0, policy_version 460559 (0.0009) [2023-12-26 18:43:03,762][105692] Updated weights for policy 0, policy_version 460569 (0.0009) [2023-12-26 18:43:03,814][105692] Updated weights for policy 0, policy_version 460579 (0.0008) [2023-12-26 18:43:04,346][105620] Updated weights for policy 1, policy_version 460972 (0.0008) [2023-12-26 18:43:04,407][105620] Updated weights for policy 1, policy_version 460982 (0.0007) [2023-12-26 18:43:04,465][105620] Updated weights for policy 1, policy_version 460992 (0.0008) [2023-12-26 18:43:04,546][105692] Updated weights for policy 0, policy_version 460589 (0.0008) [2023-12-26 18:43:04,598][105692] Updated weights for policy 0, policy_version 460599 (0.0009) [2023-12-26 18:43:04,652][105692] Updated weights for policy 0, policy_version 460609 (0.0009) [2023-12-26 18:43:05,202][105620] Updated weights for policy 1, policy_version 461002 (0.0009) [2023-12-26 18:43:05,251][105620] Updated weights for policy 1, policy_version 461012 (0.0009) [2023-12-26 18:43:05,308][105620] Updated weights for policy 1, policy_version 461022 (0.0009) [2023-12-26 18:43:05,367][105620] Updated weights for policy 1, policy_version 461032 (0.0008) [2023-12-26 18:43:05,460][105692] Updated weights for policy 0, policy_version 460620 (0.0010) [2023-12-26 18:43:05,518][105692] Updated weights for policy 0, policy_version 460630 (0.0010) [2023-12-26 18:43:05,570][105692] Updated weights for policy 0, policy_version 460640 (0.0009) [2023-12-26 18:43:05,981][105620] Updated weights for policy 1, policy_version 461042 (0.0005) [2023-12-26 18:43:06,033][105620] Updated weights for policy 1, policy_version 461052 (0.0005) [2023-12-26 18:43:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 235978752. Throughput: 0: 9801.2, 1: 9892.7. Samples: 235971872. Policy #0 lag: (min: 4.0, avg: 11.1, max: 36.0) [2023-12-26 18:43:06,063][104569] Avg episode reward: [(0, '7849.035'), (1, '9189.069')] [2023-12-26 18:43:06,088][105620] Updated weights for policy 1, policy_version 461062 (0.0006) [2023-12-26 18:43:06,204][105692] Updated weights for policy 0, policy_version 460650 (0.0009) [2023-12-26 18:43:06,265][105692] Updated weights for policy 0, policy_version 460660 (0.0009) [2023-12-26 18:43:06,327][105692] Updated weights for policy 0, policy_version 460670 (0.0009) [2023-12-26 18:43:06,347][105585] KL-divergence is very high: 126.3652 [2023-12-26 18:43:06,390][105692] Updated weights for policy 0, policy_version 460680 (0.0008) [2023-12-26 18:43:06,685][105620] Updated weights for policy 1, policy_version 461072 (0.0008) [2023-12-26 18:43:06,754][105620] Updated weights for policy 1, policy_version 461082 (0.0008) [2023-12-26 18:43:06,818][105620] Updated weights for policy 1, policy_version 461092 (0.0008) [2023-12-26 18:43:07,282][105692] Updated weights for policy 0, policy_version 460690 (0.0009) [2023-12-26 18:43:07,341][105692] Updated weights for policy 0, policy_version 460700 (0.0009) [2023-12-26 18:43:07,374][105620] Updated weights for policy 1, policy_version 461102 (0.0005) [2023-12-26 18:43:07,399][105692] Updated weights for policy 0, policy_version 460710 (0.0009) [2023-12-26 18:43:07,428][105620] Updated weights for policy 1, policy_version 461112 (0.0005) [2023-12-26 18:43:07,487][105620] Updated weights for policy 1, policy_version 461122 (0.0005) [2023-12-26 18:43:08,084][105620] Updated weights for policy 1, policy_version 461132 (0.0007) [2023-12-26 18:43:08,148][105620] Updated weights for policy 1, policy_version 461142 (0.0009) [2023-12-26 18:43:08,202][105620] Updated weights for policy 1, policy_version 461152 (0.0007) [2023-12-26 18:43:08,215][105585] KL-divergence is very high: 187.7716 [2023-12-26 18:43:08,216][105692] Updated weights for policy 0, policy_version 460720 (0.0008) [2023-12-26 18:43:08,264][105585] KL-divergence is very high: 258.0023 [2023-12-26 18:43:08,279][105692] Updated weights for policy 0, policy_version 460730 (0.0007) [2023-12-26 18:43:08,312][105585] KL-divergence is very high: 261.3442 [2023-12-26 18:43:08,336][105692] Updated weights for policy 0, policy_version 460740 (0.0008) [2023-12-26 18:43:08,987][105620] Updated weights for policy 1, policy_version 461162 (0.0009) [2023-12-26 18:43:09,051][105620] Updated weights for policy 1, policy_version 461172 (0.0009) [2023-12-26 18:43:09,070][105692] Updated weights for policy 0, policy_version 460750 (0.0008) [2023-12-26 18:43:09,105][105620] Updated weights for policy 1, policy_version 461182 (0.0006) [2023-12-26 18:43:09,120][105692] Updated weights for policy 0, policy_version 460760 (0.0006) [2023-12-26 18:43:09,163][105620] Updated weights for policy 1, policy_version 461192 (0.0006) [2023-12-26 18:43:09,178][105692] Updated weights for policy 0, policy_version 460770 (0.0007) [2023-12-26 18:43:09,966][105692] Updated weights for policy 0, policy_version 460780 (0.0008) [2023-12-26 18:43:09,973][105620] Updated weights for policy 1, policy_version 461202 (0.0009) [2023-12-26 18:43:10,026][105692] Updated weights for policy 0, policy_version 460790 (0.0008) [2023-12-26 18:43:10,040][105620] Updated weights for policy 1, policy_version 461212 (0.0007) [2023-12-26 18:43:10,089][105692] Updated weights for policy 0, policy_version 460800 (0.0007) [2023-12-26 18:43:10,108][105620] Updated weights for policy 1, policy_version 461222 (0.0006) [2023-12-26 18:43:10,796][105692] Updated weights for policy 0, policy_version 460810 (0.0009) [2023-12-26 18:43:10,831][105620] Updated weights for policy 1, policy_version 461232 (0.0007) [2023-12-26 18:43:10,860][105692] Updated weights for policy 0, policy_version 460820 (0.0008) [2023-12-26 18:43:10,895][105620] Updated weights for policy 1, policy_version 461242 (0.0007) [2023-12-26 18:43:10,920][105692] Updated weights for policy 0, policy_version 460830 (0.0007) [2023-12-26 18:43:10,943][105620] Updated weights for policy 1, policy_version 461252 (0.0007) [2023-12-26 18:43:10,956][105586] KL-divergence is very high: 344.6917 [2023-12-26 18:43:10,978][105692] Updated weights for policy 0, policy_version 460840 (0.0008) [2023-12-26 18:43:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 236085248. Throughput: 0: 9767.9, 1: 9936.1. Samples: 236087416. Policy #0 lag: (min: 4.0, avg: 11.1, max: 36.0) [2023-12-26 18:43:11,063][104569] Avg episode reward: [(0, '8201.108'), (1, '8756.698')] [2023-12-26 18:43:11,743][105620] Updated weights for policy 1, policy_version 461262 (0.0009) [2023-12-26 18:43:11,763][105692] Updated weights for policy 0, policy_version 460850 (0.0009) [2023-12-26 18:43:11,802][105620] Updated weights for policy 1, policy_version 461272 (0.0009) [2023-12-26 18:43:11,810][105692] Updated weights for policy 0, policy_version 460860 (0.0005) [2023-12-26 18:43:11,860][105692] Updated weights for policy 0, policy_version 460870 (0.0005) [2023-12-26 18:43:11,861][105620] Updated weights for policy 1, policy_version 461282 (0.0009) [2023-12-26 18:43:12,595][105692] Updated weights for policy 0, policy_version 460880 (0.0007) [2023-12-26 18:43:12,636][105620] Updated weights for policy 1, policy_version 461292 (0.0008) [2023-12-26 18:43:12,650][105692] Updated weights for policy 0, policy_version 460890 (0.0008) [2023-12-26 18:43:12,685][105620] Updated weights for policy 1, policy_version 461302 (0.0005) [2023-12-26 18:43:12,703][105692] Updated weights for policy 0, policy_version 460900 (0.0007) [2023-12-26 18:43:12,738][105620] Updated weights for policy 1, policy_version 461312 (0.0006) [2023-12-26 18:43:13,423][105692] Updated weights for policy 0, policy_version 460910 (0.0009) [2023-12-26 18:43:13,476][105620] Updated weights for policy 1, policy_version 461322 (0.0008) [2023-12-26 18:43:13,485][105692] Updated weights for policy 0, policy_version 460920 (0.0009) [2023-12-26 18:43:13,527][105620] Updated weights for policy 1, policy_version 461332 (0.0007) [2023-12-26 18:43:13,552][105692] Updated weights for policy 0, policy_version 460930 (0.0008) [2023-12-26 18:43:13,583][105620] Updated weights for policy 1, policy_version 461342 (0.0005) [2023-12-26 18:43:13,636][105620] Updated weights for policy 1, policy_version 461352 (0.0005) [2023-12-26 18:43:14,192][105692] Updated weights for policy 0, policy_version 460940 (0.0008) [2023-12-26 18:43:14,252][105692] Updated weights for policy 0, policy_version 460950 (0.0009) [2023-12-26 18:43:14,312][105692] Updated weights for policy 0, policy_version 460960 (0.0009) [2023-12-26 18:43:14,314][105620] Updated weights for policy 1, policy_version 461362 (0.0009) [2023-12-26 18:43:14,372][105620] Updated weights for policy 1, policy_version 461372 (0.0007) [2023-12-26 18:43:14,419][105620] Updated weights for policy 1, policy_version 461382 (0.0009) [2023-12-26 18:43:15,061][105692] Updated weights for policy 0, policy_version 460970 (0.0008) [2023-12-26 18:43:15,115][105692] Updated weights for policy 0, policy_version 460980 (0.0009) [2023-12-26 18:43:15,141][105620] Updated weights for policy 1, policy_version 461392 (0.0009) [2023-12-26 18:43:15,172][105692] Updated weights for policy 0, policy_version 460990 (0.0007) [2023-12-26 18:43:15,206][105620] Updated weights for policy 1, policy_version 461402 (0.0011) [2023-12-26 18:43:15,229][105692] Updated weights for policy 0, policy_version 461000 (0.0006) [2023-12-26 18:43:15,264][105620] Updated weights for policy 1, policy_version 461412 (0.0011) [2023-12-26 18:43:15,961][105620] Updated weights for policy 1, policy_version 461422 (0.0011) [2023-12-26 18:43:15,988][105692] Updated weights for policy 0, policy_version 461010 (0.0011) [2023-12-26 18:43:16,011][105620] Updated weights for policy 1, policy_version 461432 (0.0011) [2023-12-26 18:43:16,049][105692] Updated weights for policy 0, policy_version 461020 (0.0011) [2023-12-26 18:43:16,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.4, 300 sec: 19522.0). Total num frames: 236167168. Throughput: 0: 9769.5, 1: 9858.5. Samples: 236144196. Policy #0 lag: (min: 4.0, avg: 11.1, max: 36.0) [2023-12-26 18:43:16,062][104569] Avg episode reward: [(0, '7676.429'), (1, '8611.221')] [2023-12-26 18:43:16,067][105620] Updated weights for policy 1, policy_version 461442 (0.0011) [2023-12-26 18:43:16,101][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000461448_118145024.pth... [2023-12-26 18:43:16,105][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000460264_117841920.pth [2023-12-26 18:43:16,105][105692] Updated weights for policy 0, policy_version 461030 (0.0011) [2023-12-26 18:43:16,113][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000461032_118038528.pth... [2023-12-26 18:43:16,117][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000459848_117735424.pth [2023-12-26 18:43:16,700][105692] Updated weights for policy 0, policy_version 461040 (0.0010) [2023-12-26 18:43:16,766][105692] Updated weights for policy 0, policy_version 461050 (0.0011) [2023-12-26 18:43:16,782][105620] Updated weights for policy 1, policy_version 461452 (0.0009) [2023-12-26 18:43:16,828][105692] Updated weights for policy 0, policy_version 461060 (0.0011) [2023-12-26 18:43:16,840][105620] Updated weights for policy 1, policy_version 461462 (0.0006) [2023-12-26 18:43:16,895][105620] Updated weights for policy 1, policy_version 461472 (0.0010) [2023-12-26 18:43:17,466][105620] Updated weights for policy 1, policy_version 461482 (0.0010) [2023-12-26 18:43:17,493][105692] Updated weights for policy 0, policy_version 461070 (0.0007) [2023-12-26 18:43:17,528][105620] Updated weights for policy 1, policy_version 461492 (0.0010) [2023-12-26 18:43:17,555][105692] Updated weights for policy 0, policy_version 461080 (0.0006) [2023-12-26 18:43:17,583][105620] Updated weights for policy 1, policy_version 461502 (0.0011) [2023-12-26 18:43:17,617][105692] Updated weights for policy 0, policy_version 461090 (0.0010) [2023-12-26 18:43:17,627][105620] Updated weights for policy 1, policy_version 461512 (0.0010) [2023-12-26 18:43:18,192][105692] Updated weights for policy 0, policy_version 461100 (0.0010) [2023-12-26 18:43:18,254][105692] Updated weights for policy 0, policy_version 461110 (0.0010) [2023-12-26 18:43:18,315][105692] Updated weights for policy 0, policy_version 461120 (0.0010) [2023-12-26 18:43:18,350][105620] Updated weights for policy 1, policy_version 461522 (0.0009) [2023-12-26 18:43:18,399][105620] Updated weights for policy 1, policy_version 461532 (0.0010) [2023-12-26 18:43:18,457][105620] Updated weights for policy 1, policy_version 461542 (0.0011) [2023-12-26 18:43:19,011][105692] Updated weights for policy 0, policy_version 461130 (0.0009) [2023-12-26 18:43:19,061][105692] Updated weights for policy 0, policy_version 461140 (0.0006) [2023-12-26 18:43:19,126][105692] Updated weights for policy 0, policy_version 461150 (0.0006) [2023-12-26 18:43:19,170][105692] Updated weights for policy 0, policy_version 461160 (0.0010) [2023-12-26 18:43:19,177][105620] Updated weights for policy 1, policy_version 461552 (0.0008) [2023-12-26 18:43:19,242][105620] Updated weights for policy 1, policy_version 461562 (0.0009) [2023-12-26 18:43:19,307][105620] Updated weights for policy 1, policy_version 461572 (0.0006) [2023-12-26 18:43:19,882][105692] Updated weights for policy 0, policy_version 461170 (0.0009) [2023-12-26 18:43:19,939][105692] Updated weights for policy 0, policy_version 461180 (0.0011) [2023-12-26 18:43:20,000][105692] Updated weights for policy 0, policy_version 461190 (0.0012) [2023-12-26 18:43:20,059][105620] Updated weights for policy 1, policy_version 461582 (0.0008) [2023-12-26 18:43:20,131][105620] Updated weights for policy 1, policy_version 461592 (0.0006) [2023-12-26 18:43:20,208][105620] Updated weights for policy 1, policy_version 461602 (0.0007) [2023-12-26 18:43:20,697][105692] Updated weights for policy 0, policy_version 461200 (0.0010) [2023-12-26 18:43:20,750][105692] Updated weights for policy 0, policy_version 461210 (0.0009) [2023-12-26 18:43:20,803][105692] Updated weights for policy 0, policy_version 461220 (0.0009) [2023-12-26 18:43:20,902][105620] Updated weights for policy 1, policy_version 461612 (0.0009) [2023-12-26 18:43:20,964][105620] Updated weights for policy 1, policy_version 461622 (0.0009) [2023-12-26 18:43:21,030][105620] Updated weights for policy 1, policy_version 461632 (0.0009) [2023-12-26 18:43:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 236273664. Throughput: 0: 9948.8, 1: 9816.3. Samples: 236265432. Policy #0 lag: (min: 4.0, avg: 11.1, max: 36.0) [2023-12-26 18:43:21,062][104569] Avg episode reward: [(0, '7688.261'), (1, '8562.290')] [2023-12-26 18:43:21,641][105692] Updated weights for policy 0, policy_version 461230 (0.0010) [2023-12-26 18:43:21,707][105692] Updated weights for policy 0, policy_version 461240 (0.0010) [2023-12-26 18:43:21,759][105620] Updated weights for policy 1, policy_version 461642 (0.0008) [2023-12-26 18:43:21,774][105692] Updated weights for policy 0, policy_version 461250 (0.0008) [2023-12-26 18:43:21,819][105620] Updated weights for policy 1, policy_version 461652 (0.0007) [2023-12-26 18:43:21,881][105620] Updated weights for policy 1, policy_version 461662 (0.0009) [2023-12-26 18:43:21,949][105620] Updated weights for policy 1, policy_version 461672 (0.0010) [2023-12-26 18:43:22,494][105692] Updated weights for policy 0, policy_version 461260 (0.0009) [2023-12-26 18:43:22,559][105692] Updated weights for policy 0, policy_version 461270 (0.0009) [2023-12-26 18:43:22,628][105692] Updated weights for policy 0, policy_version 461280 (0.0011) [2023-12-26 18:43:22,719][105620] Updated weights for policy 1, policy_version 461682 (0.0007) [2023-12-26 18:43:22,775][105620] Updated weights for policy 1, policy_version 461692 (0.0010) [2023-12-26 18:43:22,838][105620] Updated weights for policy 1, policy_version 461702 (0.0009) [2023-12-26 18:43:23,326][105692] Updated weights for policy 0, policy_version 461290 (0.0010) [2023-12-26 18:43:23,387][105692] Updated weights for policy 0, policy_version 461300 (0.0008) [2023-12-26 18:43:23,453][105692] Updated weights for policy 0, policy_version 461310 (0.0011) [2023-12-26 18:43:23,519][105692] Updated weights for policy 0, policy_version 461320 (0.0010) [2023-12-26 18:43:23,615][105620] Updated weights for policy 1, policy_version 461712 (0.0008) [2023-12-26 18:43:23,663][105620] Updated weights for policy 1, policy_version 461722 (0.0008) [2023-12-26 18:43:23,711][105620] Updated weights for policy 1, policy_version 461732 (0.0008) [2023-12-26 18:43:24,232][105692] Updated weights for policy 0, policy_version 461330 (0.0010) [2023-12-26 18:43:24,293][105692] Updated weights for policy 0, policy_version 461340 (0.0010) [2023-12-26 18:43:24,352][105692] Updated weights for policy 0, policy_version 461350 (0.0010) [2023-12-26 18:43:24,479][105620] Updated weights for policy 1, policy_version 461742 (0.0008) [2023-12-26 18:43:24,531][105620] Updated weights for policy 1, policy_version 461752 (0.0008) [2023-12-26 18:43:24,579][105620] Updated weights for policy 1, policy_version 461762 (0.0008) [2023-12-26 18:43:25,088][105692] Updated weights for policy 0, policy_version 461360 (0.0010) [2023-12-26 18:43:25,140][105692] Updated weights for policy 0, policy_version 461370 (0.0010) [2023-12-26 18:43:25,198][105692] Updated weights for policy 0, policy_version 461380 (0.0010) [2023-12-26 18:43:25,292][105620] Updated weights for policy 1, policy_version 461772 (0.0007) [2023-12-26 18:43:25,347][105620] Updated weights for policy 1, policy_version 461782 (0.0010) [2023-12-26 18:43:25,405][105620] Updated weights for policy 1, policy_version 461792 (0.0010) [2023-12-26 18:43:25,950][105692] Updated weights for policy 0, policy_version 461390 (0.0010) [2023-12-26 18:43:26,004][105692] Updated weights for policy 0, policy_version 461400 (0.0010) [2023-12-26 18:43:26,059][105692] Updated weights for policy 0, policy_version 461410 (0.0010) [2023-12-26 18:43:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 236363776. Throughput: 0: 9975.4, 1: 9778.3. Samples: 236378228. Policy #0 lag: (min: 4.0, avg: 11.1, max: 36.0) [2023-12-26 18:43:26,062][104569] Avg episode reward: [(0, '7848.743'), (1, '8408.165')] [2023-12-26 18:43:26,115][105620] Updated weights for policy 1, policy_version 461802 (0.0009) [2023-12-26 18:43:26,176][105620] Updated weights for policy 1, policy_version 461812 (0.0007) [2023-12-26 18:43:26,234][105620] Updated weights for policy 1, policy_version 461822 (0.0010) [2023-12-26 18:43:26,291][105620] Updated weights for policy 1, policy_version 461832 (0.0010) [2023-12-26 18:43:26,741][105692] Updated weights for policy 0, policy_version 461420 (0.0010) [2023-12-26 18:43:26,794][105692] Updated weights for policy 0, policy_version 461430 (0.0008) [2023-12-26 18:43:26,843][105692] Updated weights for policy 0, policy_version 461440 (0.0010) [2023-12-26 18:43:27,022][105620] Updated weights for policy 1, policy_version 461842 (0.0010) [2023-12-26 18:43:27,069][105620] Updated weights for policy 1, policy_version 461852 (0.0010) [2023-12-26 18:43:27,120][105620] Updated weights for policy 1, policy_version 461862 (0.0008) [2023-12-26 18:43:27,515][105692] Updated weights for policy 0, policy_version 461450 (0.0007) [2023-12-26 18:43:27,588][105692] Updated weights for policy 0, policy_version 461460 (0.0008) [2023-12-26 18:43:27,647][105692] Updated weights for policy 0, policy_version 461470 (0.0005) [2023-12-26 18:43:27,700][105620] Updated weights for policy 1, policy_version 461872 (0.0006) [2023-12-26 18:43:27,701][105692] Updated weights for policy 0, policy_version 461480 (0.0009) [2023-12-26 18:43:27,753][105620] Updated weights for policy 1, policy_version 461882 (0.0006) [2023-12-26 18:43:27,811][105620] Updated weights for policy 1, policy_version 461892 (0.0006) [2023-12-26 18:43:28,223][105692] Updated weights for policy 0, policy_version 461490 (0.0010) [2023-12-26 18:43:28,267][105692] Updated weights for policy 0, policy_version 461500 (0.0010) [2023-12-26 18:43:28,314][105692] Updated weights for policy 0, policy_version 461510 (0.0010) [2023-12-26 18:43:28,348][105620] Updated weights for policy 1, policy_version 461902 (0.0006) [2023-12-26 18:43:28,411][105620] Updated weights for policy 1, policy_version 461912 (0.0009) [2023-12-26 18:43:28,467][105620] Updated weights for policy 1, policy_version 461922 (0.0009) [2023-12-26 18:43:29,029][105620] Updated weights for policy 1, policy_version 461932 (0.0006) [2023-12-26 18:43:29,075][105620] Updated weights for policy 1, policy_version 461942 (0.0008) [2023-12-26 18:43:29,086][105692] Updated weights for policy 0, policy_version 461520 (0.0010) [2023-12-26 18:43:29,127][105620] Updated weights for policy 1, policy_version 461952 (0.0005) [2023-12-26 18:43:29,130][105692] Updated weights for policy 0, policy_version 461530 (0.0010) [2023-12-26 18:43:29,174][105692] Updated weights for policy 0, policy_version 461540 (0.0010) [2023-12-26 18:43:29,851][105620] Updated weights for policy 1, policy_version 461962 (0.0007) [2023-12-26 18:43:29,924][105620] Updated weights for policy 1, policy_version 461972 (0.0007) [2023-12-26 18:43:29,961][105692] Updated weights for policy 0, policy_version 461550 (0.0009) [2023-12-26 18:43:29,987][105620] Updated weights for policy 1, policy_version 461982 (0.0007) [2023-12-26 18:43:30,023][105692] Updated weights for policy 0, policy_version 461560 (0.0010) [2023-12-26 18:43:30,045][105620] Updated weights for policy 1, policy_version 461992 (0.0006) [2023-12-26 18:43:30,085][105692] Updated weights for policy 0, policy_version 461570 (0.0010) [2023-12-26 18:43:30,707][105620] Updated weights for policy 1, policy_version 462002 (0.0006) [2023-12-26 18:43:30,758][105620] Updated weights for policy 1, policy_version 462012 (0.0005) [2023-12-26 18:43:30,808][105692] Updated weights for policy 0, policy_version 461580 (0.0009) [2023-12-26 18:43:30,809][105620] Updated weights for policy 1, policy_version 462022 (0.0007) [2023-12-26 18:43:30,856][105692] Updated weights for policy 0, policy_version 461590 (0.0005) [2023-12-26 18:43:30,915][105692] Updated weights for policy 0, policy_version 461600 (0.0009) [2023-12-26 18:43:31,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 236478464. Throughput: 0: 10014.0, 1: 9936.3. Samples: 236443448. Policy #0 lag: (min: 4.0, avg: 11.1, max: 36.0) [2023-12-26 18:43:31,063][104569] Avg episode reward: [(0, '8107.697'), (1, '8590.257')] [2023-12-26 18:43:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000461608_118185984.pth... [2023-12-26 18:43:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000462024_118292480.pth... [2023-12-26 18:43:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000460840_117989376.pth [2023-12-26 18:43:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000460456_117891072.pth [2023-12-26 18:43:31,587][105620] Updated weights for policy 1, policy_version 462032 (0.0009) [2023-12-26 18:43:31,647][105620] Updated weights for policy 1, policy_version 462042 (0.0011) [2023-12-26 18:43:31,648][105692] Updated weights for policy 0, policy_version 461610 (0.0009) [2023-12-26 18:43:31,698][105692] Updated weights for policy 0, policy_version 461620 (0.0006) [2023-12-26 18:43:31,703][105620] Updated weights for policy 1, policy_version 462052 (0.0010) [2023-12-26 18:43:31,759][105692] Updated weights for policy 0, policy_version 461630 (0.0008) [2023-12-26 18:43:31,819][105692] Updated weights for policy 0, policy_version 461640 (0.0008) [2023-12-26 18:43:32,412][105620] Updated weights for policy 1, policy_version 462062 (0.0010) [2023-12-26 18:43:32,476][105620] Updated weights for policy 1, policy_version 462072 (0.0006) [2023-12-26 18:43:32,539][105620] Updated weights for policy 1, policy_version 462082 (0.0009) [2023-12-26 18:43:32,568][105692] Updated weights for policy 0, policy_version 461650 (0.0008) [2023-12-26 18:43:32,617][105692] Updated weights for policy 0, policy_version 461660 (0.0008) [2023-12-26 18:43:32,679][105692] Updated weights for policy 0, policy_version 461670 (0.0009) [2023-12-26 18:43:33,160][105620] Updated weights for policy 1, policy_version 462092 (0.0008) [2023-12-26 18:43:33,221][105620] Updated weights for policy 1, policy_version 462102 (0.0010) [2023-12-26 18:43:33,296][105620] Updated weights for policy 1, policy_version 462112 (0.0010) [2023-12-26 18:43:33,344][105692] Updated weights for policy 0, policy_version 461680 (0.0007) [2023-12-26 18:43:33,398][105692] Updated weights for policy 0, policy_version 461690 (0.0008) [2023-12-26 18:43:33,446][105692] Updated weights for policy 0, policy_version 461700 (0.0007) [2023-12-26 18:43:33,968][105620] Updated weights for policy 1, policy_version 462122 (0.0010) [2023-12-26 18:43:34,024][105620] Updated weights for policy 1, policy_version 462132 (0.0008) [2023-12-26 18:43:34,079][105620] Updated weights for policy 1, policy_version 462142 (0.0009) [2023-12-26 18:43:34,126][105620] Updated weights for policy 1, policy_version 462152 (0.0009) [2023-12-26 18:43:34,203][105692] Updated weights for policy 0, policy_version 461710 (0.0009) [2023-12-26 18:43:34,265][105585] KL-divergence is very high: 1364.7169 [2023-12-26 18:43:34,265][105692] Updated weights for policy 0, policy_version 461720 (0.0009) [2023-12-26 18:43:34,315][105585] KL-divergence is very high: 2172.6165 [2023-12-26 18:43:34,328][105692] Updated weights for policy 0, policy_version 461730 (0.0009) [2023-12-26 18:43:34,903][105620] Updated weights for policy 1, policy_version 462162 (0.0008) [2023-12-26 18:43:34,962][105620] Updated weights for policy 1, policy_version 462172 (0.0008) [2023-12-26 18:43:35,025][105620] Updated weights for policy 1, policy_version 462182 (0.0008) [2023-12-26 18:43:35,089][105692] Updated weights for policy 0, policy_version 461740 (0.0009) [2023-12-26 18:43:35,154][105692] Updated weights for policy 0, policy_version 461750 (0.0010) [2023-12-26 18:43:35,210][105692] Updated weights for policy 0, policy_version 461760 (0.0007) [2023-12-26 18:43:35,777][105620] Updated weights for policy 1, policy_version 462192 (0.0010) [2023-12-26 18:43:35,832][105692] Updated weights for policy 0, policy_version 461770 (0.0006) [2023-12-26 18:43:35,833][105620] Updated weights for policy 1, policy_version 462202 (0.0010) [2023-12-26 18:43:35,885][105692] Updated weights for policy 0, policy_version 461780 (0.0010) [2023-12-26 18:43:35,893][105620] Updated weights for policy 1, policy_version 462212 (0.0011) [2023-12-26 18:43:35,934][105692] Updated weights for policy 0, policy_version 461790 (0.0010) [2023-12-26 18:43:35,993][105692] Updated weights for policy 0, policy_version 461800 (0.0006) [2023-12-26 18:43:36,062][104569] Fps is (10 sec: 21299.1, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 236576768. Throughput: 0: 9869.1, 1: 9937.2. Samples: 236559900. Policy #0 lag: (min: 4.0, avg: 11.1, max: 36.0) [2023-12-26 18:43:36,063][104569] Avg episode reward: [(0, '8547.398'), (1, '8651.485')] [2023-12-26 18:43:36,525][105620] Updated weights for policy 1, policy_version 462222 (0.0009) [2023-12-26 18:43:36,584][105620] Updated weights for policy 1, policy_version 462232 (0.0008) [2023-12-26 18:43:36,644][105620] Updated weights for policy 1, policy_version 462242 (0.0008) [2023-12-26 18:43:36,740][105692] Updated weights for policy 0, policy_version 461810 (0.0010) [2023-12-26 18:43:36,810][105692] Updated weights for policy 0, policy_version 461820 (0.0010) [2023-12-26 18:43:36,876][105692] Updated weights for policy 0, policy_version 461830 (0.0010) [2023-12-26 18:43:37,303][105620] Updated weights for policy 1, policy_version 462252 (0.0007) [2023-12-26 18:43:37,360][105620] Updated weights for policy 1, policy_version 462262 (0.0006) [2023-12-26 18:43:37,408][105620] Updated weights for policy 1, policy_version 462272 (0.0008) [2023-12-26 18:43:37,585][105692] Updated weights for policy 0, policy_version 461840 (0.0006) [2023-12-26 18:43:37,639][105692] Updated weights for policy 0, policy_version 461850 (0.0010) [2023-12-26 18:43:37,688][105692] Updated weights for policy 0, policy_version 461860 (0.0010) [2023-12-26 18:43:38,134][105620] Updated weights for policy 1, policy_version 462282 (0.0007) [2023-12-26 18:43:38,192][105620] Updated weights for policy 1, policy_version 462292 (0.0005) [2023-12-26 18:43:38,240][105620] Updated weights for policy 1, policy_version 462302 (0.0005) [2023-12-26 18:43:38,287][105620] Updated weights for policy 1, policy_version 462312 (0.0005) [2023-12-26 18:43:38,440][105692] Updated weights for policy 0, policy_version 461870 (0.0010) [2023-12-26 18:43:38,492][105692] Updated weights for policy 0, policy_version 461880 (0.0010) [2023-12-26 18:43:38,553][105692] Updated weights for policy 0, policy_version 461890 (0.0010) [2023-12-26 18:43:38,970][105620] Updated weights for policy 1, policy_version 462322 (0.0010) [2023-12-26 18:43:39,031][105620] Updated weights for policy 1, policy_version 462332 (0.0010) [2023-12-26 18:43:39,085][105620] Updated weights for policy 1, policy_version 462342 (0.0009) [2023-12-26 18:43:39,185][105692] Updated weights for policy 0, policy_version 461900 (0.0010) [2023-12-26 18:43:39,240][105692] Updated weights for policy 0, policy_version 461910 (0.0009) [2023-12-26 18:43:39,303][105692] Updated weights for policy 0, policy_version 461920 (0.0009) [2023-12-26 18:43:39,978][105620] Updated weights for policy 1, policy_version 462352 (0.0009) [2023-12-26 18:43:40,034][105620] Updated weights for policy 1, policy_version 462362 (0.0009) [2023-12-26 18:43:40,064][105692] Updated weights for policy 0, policy_version 461930 (0.0009) [2023-12-26 18:43:40,089][105620] Updated weights for policy 1, policy_version 462372 (0.0009) [2023-12-26 18:43:40,147][105692] Updated weights for policy 0, policy_version 461940 (0.0007) [2023-12-26 18:43:40,203][105692] Updated weights for policy 0, policy_version 461950 (0.0005) [2023-12-26 18:43:40,265][105692] Updated weights for policy 0, policy_version 461960 (0.0009) [2023-12-26 18:43:40,730][105620] Updated weights for policy 1, policy_version 462382 (0.0008) [2023-12-26 18:43:40,784][105620] Updated weights for policy 1, policy_version 462392 (0.0010) [2023-12-26 18:43:40,845][105620] Updated weights for policy 1, policy_version 462402 (0.0010) [2023-12-26 18:43:41,019][105692] Updated weights for policy 0, policy_version 461970 (0.0008) [2023-12-26 18:43:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 236666880. Throughput: 0: 9770.1, 1: 9973.3. Samples: 236677144. Policy #0 lag: (min: 4.0, avg: 11.1, max: 36.0) [2023-12-26 18:43:41,062][104569] Avg episode reward: [(0, '8463.682'), (1, '8829.847')] [2023-12-26 18:43:41,082][105692] Updated weights for policy 0, policy_version 461980 (0.0008) [2023-12-26 18:43:41,152][105692] Updated weights for policy 0, policy_version 461990 (0.0007) [2023-12-26 18:43:41,645][105620] Updated weights for policy 1, policy_version 462412 (0.0009) [2023-12-26 18:43:41,716][105620] Updated weights for policy 1, policy_version 462422 (0.0007) [2023-12-26 18:43:41,780][105620] Updated weights for policy 1, policy_version 462432 (0.0008) [2023-12-26 18:43:41,883][105692] Updated weights for policy 0, policy_version 462000 (0.0010) [2023-12-26 18:43:41,944][105692] Updated weights for policy 0, policy_version 462010 (0.0009) [2023-12-26 18:43:42,008][105692] Updated weights for policy 0, policy_version 462020 (0.0010) [2023-12-26 18:43:42,550][105620] Updated weights for policy 1, policy_version 462442 (0.0009) [2023-12-26 18:43:42,588][105692] Updated weights for policy 0, policy_version 462030 (0.0005) [2023-12-26 18:43:42,604][105620] Updated weights for policy 1, policy_version 462452 (0.0009) [2023-12-26 18:43:42,649][105692] Updated weights for policy 0, policy_version 462040 (0.0006) [2023-12-26 18:43:42,663][105620] Updated weights for policy 1, policy_version 462462 (0.0008) [2023-12-26 18:43:42,708][105692] Updated weights for policy 0, policy_version 462050 (0.0010) [2023-12-26 18:43:42,722][105620] Updated weights for policy 1, policy_version 462472 (0.0009) [2023-12-26 18:43:43,416][105692] Updated weights for policy 0, policy_version 462060 (0.0010) [2023-12-26 18:43:43,471][105692] Updated weights for policy 0, policy_version 462070 (0.0010) [2023-12-26 18:43:43,493][105620] Updated weights for policy 1, policy_version 462482 (0.0006) [2023-12-26 18:43:43,526][105692] Updated weights for policy 0, policy_version 462080 (0.0010) [2023-12-26 18:43:43,552][105620] Updated weights for policy 1, policy_version 462492 (0.0005) [2023-12-26 18:43:43,610][105620] Updated weights for policy 1, policy_version 462502 (0.0008) [2023-12-26 18:43:44,269][105692] Updated weights for policy 0, policy_version 462090 (0.0010) [2023-12-26 18:43:44,330][105692] Updated weights for policy 0, policy_version 462100 (0.0010) [2023-12-26 18:43:44,345][105620] Updated weights for policy 1, policy_version 462512 (0.0008) [2023-12-26 18:43:44,382][105692] Updated weights for policy 0, policy_version 462110 (0.0010) [2023-12-26 18:43:44,408][105620] Updated weights for policy 1, policy_version 462522 (0.0006) [2023-12-26 18:43:44,435][105692] Updated weights for policy 0, policy_version 462120 (0.0011) [2023-12-26 18:43:44,470][105620] Updated weights for policy 1, policy_version 462532 (0.0007) [2023-12-26 18:43:45,171][105620] Updated weights for policy 1, policy_version 462542 (0.0008) [2023-12-26 18:43:45,227][105692] Updated weights for policy 0, policy_version 462130 (0.0011) [2023-12-26 18:43:45,229][105620] Updated weights for policy 1, policy_version 462552 (0.0006) [2023-12-26 18:43:45,287][105620] Updated weights for policy 1, policy_version 462562 (0.0008) [2023-12-26 18:43:45,287][105692] Updated weights for policy 0, policy_version 462140 (0.0011) [2023-12-26 18:43:45,348][105692] Updated weights for policy 0, policy_version 462150 (0.0011) [2023-12-26 18:43:46,002][105620] Updated weights for policy 1, policy_version 462572 (0.0006) [2023-12-26 18:43:46,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 236756992. Throughput: 0: 9714.0, 1: 9896.2. Samples: 236733712. Policy #0 lag: (min: 4.0, avg: 11.1, max: 36.0) [2023-12-26 18:43:46,062][104569] Avg episode reward: [(0, '8733.020'), (1, '9180.876')] [2023-12-26 18:43:46,071][105620] Updated weights for policy 1, policy_version 462582 (0.0006) [2023-12-26 18:43:46,090][105692] Updated weights for policy 0, policy_version 462160 (0.0011) [2023-12-26 18:43:46,124][105620] Updated weights for policy 1, policy_version 462592 (0.0005) [2023-12-26 18:43:46,138][105692] Updated weights for policy 0, policy_version 462170 (0.0010) [2023-12-26 18:43:46,165][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000462600_118439936.pth... [2023-12-26 18:43:46,169][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000461448_118145024.pth [2023-12-26 18:43:46,183][105692] Updated weights for policy 0, policy_version 462180 (0.0010) [2023-12-26 18:43:46,198][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000462184_118333440.pth... [2023-12-26 18:43:46,201][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000461032_118038528.pth [2023-12-26 18:43:46,804][105620] Updated weights for policy 1, policy_version 462602 (0.0005) [2023-12-26 18:43:46,871][105620] Updated weights for policy 1, policy_version 462612 (0.0006) [2023-12-26 18:43:46,933][105620] Updated weights for policy 1, policy_version 462622 (0.0008) [2023-12-26 18:43:46,956][105692] Updated weights for policy 0, policy_version 462190 (0.0010) [2023-12-26 18:43:46,983][105620] Updated weights for policy 1, policy_version 462632 (0.0006) [2023-12-26 18:43:47,016][105692] Updated weights for policy 0, policy_version 462200 (0.0010) [2023-12-26 18:43:47,065][105692] Updated weights for policy 0, policy_version 462210 (0.0008) [2023-12-26 18:43:47,573][105620] Updated weights for policy 1, policy_version 462642 (0.0008) [2023-12-26 18:43:47,621][105620] Updated weights for policy 1, policy_version 462652 (0.0007) [2023-12-26 18:43:47,676][105620] Updated weights for policy 1, policy_version 462662 (0.0008) [2023-12-26 18:43:47,813][105692] Updated weights for policy 0, policy_version 462220 (0.0010) [2023-12-26 18:43:47,861][105692] Updated weights for policy 0, policy_version 462230 (0.0010) [2023-12-26 18:43:47,923][105692] Updated weights for policy 0, policy_version 462240 (0.0010) [2023-12-26 18:43:48,387][105620] Updated weights for policy 1, policy_version 462672 (0.0008) [2023-12-26 18:43:48,453][105620] Updated weights for policy 1, policy_version 462682 (0.0007) [2023-12-26 18:43:48,517][105620] Updated weights for policy 1, policy_version 462692 (0.0009) [2023-12-26 18:43:48,677][105692] Updated weights for policy 0, policy_version 462250 (0.0010) [2023-12-26 18:43:48,739][105692] Updated weights for policy 0, policy_version 462260 (0.0011) [2023-12-26 18:43:48,808][105692] Updated weights for policy 0, policy_version 462270 (0.0011) [2023-12-26 18:43:48,863][105692] Updated weights for policy 0, policy_version 462280 (0.0010) [2023-12-26 18:43:49,174][105620] Updated weights for policy 1, policy_version 462702 (0.0006) [2023-12-26 18:43:49,227][105620] Updated weights for policy 1, policy_version 462712 (0.0006) [2023-12-26 18:43:49,290][105620] Updated weights for policy 1, policy_version 462722 (0.0006) [2023-12-26 18:43:49,605][105692] Updated weights for policy 0, policy_version 462290 (0.0010) [2023-12-26 18:43:49,657][105692] Updated weights for policy 0, policy_version 462300 (0.0010) [2023-12-26 18:43:49,718][105692] Updated weights for policy 0, policy_version 462310 (0.0010) [2023-12-26 18:43:49,932][105620] Updated weights for policy 1, policy_version 462732 (0.0007) [2023-12-26 18:43:49,983][105620] Updated weights for policy 1, policy_version 462742 (0.0010) [2023-12-26 18:43:50,037][105620] Updated weights for policy 1, policy_version 462752 (0.0010) [2023-12-26 18:43:50,489][105692] Updated weights for policy 0, policy_version 462320 (0.0010) [2023-12-26 18:43:50,548][105692] Updated weights for policy 0, policy_version 462330 (0.0011) [2023-12-26 18:43:50,615][105692] Updated weights for policy 0, policy_version 462340 (0.0011) [2023-12-26 18:43:50,809][105620] Updated weights for policy 1, policy_version 462762 (0.0011) [2023-12-26 18:43:50,872][105620] Updated weights for policy 1, policy_version 462772 (0.0011) [2023-12-26 18:43:50,931][105620] Updated weights for policy 1, policy_version 462782 (0.0011) [2023-12-26 18:43:50,991][105620] Updated weights for policy 1, policy_version 462792 (0.0009) [2023-12-26 18:43:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 236863488. Throughput: 0: 9639.0, 1: 9894.6. Samples: 236850880. Policy #0 lag: (min: 4.0, avg: 11.1, max: 36.0) [2023-12-26 18:43:51,062][104569] Avg episode reward: [(0, '8907.982'), (1, '9181.287')] [2023-12-26 18:43:51,372][105692] Updated weights for policy 0, policy_version 462350 (0.0011) [2023-12-26 18:43:51,433][105692] Updated weights for policy 0, policy_version 462360 (0.0010) [2023-12-26 18:43:51,481][105692] Updated weights for policy 0, policy_version 462370 (0.0011) [2023-12-26 18:43:51,796][105620] Updated weights for policy 1, policy_version 462802 (0.0009) [2023-12-26 18:43:51,847][105620] Updated weights for policy 1, policy_version 462812 (0.0007) [2023-12-26 18:43:51,908][105620] Updated weights for policy 1, policy_version 462822 (0.0006) [2023-12-26 18:43:52,203][105692] Updated weights for policy 0, policy_version 462380 (0.0010) [2023-12-26 18:43:52,262][105692] Updated weights for policy 0, policy_version 462390 (0.0009) [2023-12-26 18:43:52,310][105692] Updated weights for policy 0, policy_version 462400 (0.0011) [2023-12-26 18:43:52,659][105620] Updated weights for policy 1, policy_version 462832 (0.0006) [2023-12-26 18:43:52,708][105620] Updated weights for policy 1, policy_version 462842 (0.0009) [2023-12-26 18:43:52,769][105620] Updated weights for policy 1, policy_version 462852 (0.0010) [2023-12-26 18:43:53,052][105692] Updated weights for policy 0, policy_version 462410 (0.0009) [2023-12-26 18:43:53,106][105692] Updated weights for policy 0, policy_version 462420 (0.0005) [2023-12-26 18:43:53,152][105692] Updated weights for policy 0, policy_version 462430 (0.0005) [2023-12-26 18:43:53,198][105692] Updated weights for policy 0, policy_version 462440 (0.0005) [2023-12-26 18:43:53,352][105620] Updated weights for policy 1, policy_version 462862 (0.0007) [2023-12-26 18:43:53,404][105620] Updated weights for policy 1, policy_version 462872 (0.0005) [2023-12-26 18:43:53,461][105620] Updated weights for policy 1, policy_version 462882 (0.0005) [2023-12-26 18:43:53,734][105692] Updated weights for policy 0, policy_version 462450 (0.0006) [2023-12-26 18:43:53,787][105692] Updated weights for policy 0, policy_version 462460 (0.0008) [2023-12-26 18:43:53,833][105692] Updated weights for policy 0, policy_version 462470 (0.0008) [2023-12-26 18:43:54,204][105620] Updated weights for policy 1, policy_version 462892 (0.0007) [2023-12-26 18:43:54,263][105620] Updated weights for policy 1, policy_version 462902 (0.0009) [2023-12-26 18:43:54,332][105620] Updated weights for policy 1, policy_version 462912 (0.0010) [2023-12-26 18:43:54,531][105692] Updated weights for policy 0, policy_version 462480 (0.0010) [2023-12-26 18:43:54,586][105692] Updated weights for policy 0, policy_version 462490 (0.0010) [2023-12-26 18:43:54,637][105585] KL-divergence is very high: 259.7388 [2023-12-26 18:43:54,637][105692] Updated weights for policy 0, policy_version 462500 (0.0010) [2023-12-26 18:43:55,083][105620] Updated weights for policy 1, policy_version 462922 (0.0010) [2023-12-26 18:43:55,138][105620] Updated weights for policy 1, policy_version 462932 (0.0010) [2023-12-26 18:43:55,191][105620] Updated weights for policy 1, policy_version 462942 (0.0010) [2023-12-26 18:43:55,245][105620] Updated weights for policy 1, policy_version 462952 (0.0009) [2023-12-26 18:43:55,324][105692] Updated weights for policy 0, policy_version 462510 (0.0009) [2023-12-26 18:43:55,382][105692] Updated weights for policy 0, policy_version 462520 (0.0008) [2023-12-26 18:43:55,446][105692] Updated weights for policy 0, policy_version 462530 (0.0008) [2023-12-26 18:43:56,055][105620] Updated weights for policy 1, policy_version 462962 (0.0009) [2023-12-26 18:43:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 236953600. Throughput: 0: 9763.1, 1: 9801.3. Samples: 236967812. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:43:56,062][104569] Avg episode reward: [(0, '8913.659'), (1, '9269.249')] [2023-12-26 18:43:56,110][105692] Updated weights for policy 0, policy_version 462540 (0.0007) [2023-12-26 18:43:56,117][105620] Updated weights for policy 1, policy_version 462972 (0.0009) [2023-12-26 18:43:56,159][105692] Updated weights for policy 0, policy_version 462550 (0.0007) [2023-12-26 18:43:56,179][105620] Updated weights for policy 1, policy_version 462982 (0.0007) [2023-12-26 18:43:56,211][105692] Updated weights for policy 0, policy_version 462560 (0.0008) [2023-12-26 18:43:56,875][105692] Updated weights for policy 0, policy_version 462570 (0.0008) [2023-12-26 18:43:56,933][105692] Updated weights for policy 0, policy_version 462580 (0.0005) [2023-12-26 18:43:56,983][105620] Updated weights for policy 1, policy_version 462992 (0.0008) [2023-12-26 18:43:56,986][105692] Updated weights for policy 0, policy_version 462590 (0.0005) [2023-12-26 18:43:57,038][105620] Updated weights for policy 1, policy_version 463002 (0.0008) [2023-12-26 18:43:57,041][105692] Updated weights for policy 0, policy_version 462600 (0.0005) [2023-12-26 18:43:57,094][105620] Updated weights for policy 1, policy_version 463012 (0.0009) [2023-12-26 18:43:57,588][105692] Updated weights for policy 0, policy_version 462610 (0.0008) [2023-12-26 18:43:57,646][105692] Updated weights for policy 0, policy_version 462620 (0.0010) [2023-12-26 18:43:57,702][105692] Updated weights for policy 0, policy_version 462630 (0.0010) [2023-12-26 18:43:57,860][105620] Updated weights for policy 1, policy_version 463023 (0.0009) [2023-12-26 18:43:57,908][105620] Updated weights for policy 1, policy_version 463033 (0.0008) [2023-12-26 18:43:57,960][105620] Updated weights for policy 1, policy_version 463043 (0.0009) [2023-12-26 18:43:58,381][105692] Updated weights for policy 0, policy_version 462641 (0.0007) [2023-12-26 18:43:58,441][105692] Updated weights for policy 0, policy_version 462651 (0.0008) [2023-12-26 18:43:58,500][105692] Updated weights for policy 0, policy_version 462661 (0.0008) [2023-12-26 18:43:58,767][105620] Updated weights for policy 1, policy_version 463053 (0.0007) [2023-12-26 18:43:58,841][105620] Updated weights for policy 1, policy_version 463063 (0.0008) [2023-12-26 18:43:58,903][105620] Updated weights for policy 1, policy_version 463073 (0.0007) [2023-12-26 18:43:59,282][105692] Updated weights for policy 0, policy_version 462671 (0.0009) [2023-12-26 18:43:59,334][105692] Updated weights for policy 0, policy_version 462681 (0.0011) [2023-12-26 18:43:59,399][105692] Updated weights for policy 0, policy_version 462691 (0.0011) [2023-12-26 18:43:59,583][105620] Updated weights for policy 1, policy_version 463083 (0.0010) [2023-12-26 18:43:59,641][105620] Updated weights for policy 1, policy_version 463093 (0.0011) [2023-12-26 18:43:59,703][105620] Updated weights for policy 1, policy_version 463103 (0.0010) [2023-12-26 18:44:00,099][105692] Updated weights for policy 0, policy_version 462701 (0.0008) [2023-12-26 18:44:00,156][105692] Updated weights for policy 0, policy_version 462711 (0.0009) [2023-12-26 18:44:00,179][105585] KL-divergence is very high: 204.7733 [2023-12-26 18:44:00,227][105692] Updated weights for policy 0, policy_version 462721 (0.0005) [2023-12-26 18:44:00,235][105585] KL-divergence is very high: 316.5992 [2023-12-26 18:44:00,415][105620] Updated weights for policy 1, policy_version 463113 (0.0010) [2023-12-26 18:44:00,470][105620] Updated weights for policy 1, policy_version 463123 (0.0010) [2023-12-26 18:44:00,528][105620] Updated weights for policy 1, policy_version 463133 (0.0010) [2023-12-26 18:44:00,582][105620] Updated weights for policy 1, policy_version 463143 (0.0010) [2023-12-26 18:44:00,839][105692] Updated weights for policy 0, policy_version 462731 (0.0007) [2023-12-26 18:44:00,886][105692] Updated weights for policy 0, policy_version 462741 (0.0010) [2023-12-26 18:44:00,933][105692] Updated weights for policy 0, policy_version 462751 (0.0010) [2023-12-26 18:44:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 237060096. Throughput: 0: 9840.2, 1: 9767.1. Samples: 237026524. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:44:01,062][104569] Avg episode reward: [(0, '8825.734'), (1, '9181.152')] [2023-12-26 18:44:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000462760_118480896.pth... [2023-12-26 18:44:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000463144_118579200.pth... [2023-12-26 18:44:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000461608_118185984.pth [2023-12-26 18:44:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000462024_118292480.pth [2023-12-26 18:44:01,183][105620] Updated weights for policy 1, policy_version 463153 (0.0008) [2023-12-26 18:44:01,231][105620] Updated weights for policy 1, policy_version 463163 (0.0010) [2023-12-26 18:44:01,290][105620] Updated weights for policy 1, policy_version 463173 (0.0011) [2023-12-26 18:44:01,667][105692] Updated weights for policy 0, policy_version 462761 (0.0010) [2023-12-26 18:44:01,711][105692] Updated weights for policy 0, policy_version 462771 (0.0008) [2023-12-26 18:44:01,774][105692] Updated weights for policy 0, policy_version 462781 (0.0009) [2023-12-26 18:44:01,828][105692] Updated weights for policy 0, policy_version 462791 (0.0008) [2023-12-26 18:44:02,049][105620] Updated weights for policy 1, policy_version 463183 (0.0010) [2023-12-26 18:44:02,107][105620] Updated weights for policy 1, policy_version 463193 (0.0010) [2023-12-26 18:44:02,173][105620] Updated weights for policy 1, policy_version 463203 (0.0010) [2023-12-26 18:44:02,634][105692] Updated weights for policy 0, policy_version 462801 (0.0008) [2023-12-26 18:44:02,692][105692] Updated weights for policy 0, policy_version 462811 (0.0008) [2023-12-26 18:44:02,753][105692] Updated weights for policy 0, policy_version 462821 (0.0009) [2023-12-26 18:44:02,891][105620] Updated weights for policy 1, policy_version 463213 (0.0010) [2023-12-26 18:44:02,949][105620] Updated weights for policy 1, policy_version 463223 (0.0010) [2023-12-26 18:44:03,000][105620] Updated weights for policy 1, policy_version 463233 (0.0010) [2023-12-26 18:44:03,469][105692] Updated weights for policy 0, policy_version 462831 (0.0008) [2023-12-26 18:44:03,538][105692] Updated weights for policy 0, policy_version 462841 (0.0008) [2023-12-26 18:44:03,595][105692] Updated weights for policy 0, policy_version 462851 (0.0007) [2023-12-26 18:44:03,745][105620] Updated weights for policy 1, policy_version 463243 (0.0010) [2023-12-26 18:44:03,802][105620] Updated weights for policy 1, policy_version 463253 (0.0010) [2023-12-26 18:44:03,867][105620] Updated weights for policy 1, policy_version 463263 (0.0010) [2023-12-26 18:44:04,253][105692] Updated weights for policy 0, policy_version 462861 (0.0008) [2023-12-26 18:44:04,310][105692] Updated weights for policy 0, policy_version 462871 (0.0011) [2023-12-26 18:44:04,360][105692] Updated weights for policy 0, policy_version 462881 (0.0011) [2023-12-26 18:44:04,576][105620] Updated weights for policy 1, policy_version 463273 (0.0010) [2023-12-26 18:44:04,625][105620] Updated weights for policy 1, policy_version 463283 (0.0007) [2023-12-26 18:44:04,682][105620] Updated weights for policy 1, policy_version 463293 (0.0005) [2023-12-26 18:44:04,746][105620] Updated weights for policy 1, policy_version 463303 (0.0010) [2023-12-26 18:44:05,074][105692] Updated weights for policy 0, policy_version 462891 (0.0009) [2023-12-26 18:44:05,129][105692] Updated weights for policy 0, policy_version 462901 (0.0007) [2023-12-26 18:44:05,185][105692] Updated weights for policy 0, policy_version 462911 (0.0008) [2023-12-26 18:44:05,454][105620] Updated weights for policy 1, policy_version 463313 (0.0010) [2023-12-26 18:44:05,512][105620] Updated weights for policy 1, policy_version 463323 (0.0010) [2023-12-26 18:44:05,576][105620] Updated weights for policy 1, policy_version 463333 (0.0010) [2023-12-26 18:44:05,825][105692] Updated weights for policy 0, policy_version 462921 (0.0008) [2023-12-26 18:44:05,878][105692] Updated weights for policy 0, policy_version 462931 (0.0008) [2023-12-26 18:44:05,927][105692] Updated weights for policy 0, policy_version 462941 (0.0008) [2023-12-26 18:44:05,979][105692] Updated weights for policy 0, policy_version 462951 (0.0008) [2023-12-26 18:44:06,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 237158400. Throughput: 0: 9771.6, 1: 9749.3. Samples: 237143876. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:44:06,063][104569] Avg episode reward: [(0, '8368.352'), (1, '9181.130')] [2023-12-26 18:44:06,345][105620] Updated weights for policy 1, policy_version 463343 (0.0010) [2023-12-26 18:44:06,414][105620] Updated weights for policy 1, policy_version 463353 (0.0009) [2023-12-26 18:44:06,482][105620] Updated weights for policy 1, policy_version 463363 (0.0011) [2023-12-26 18:44:06,762][105692] Updated weights for policy 0, policy_version 462961 (0.0008) [2023-12-26 18:44:06,825][105692] Updated weights for policy 0, policy_version 462971 (0.0008) [2023-12-26 18:44:06,887][105692] Updated weights for policy 0, policy_version 462981 (0.0008) [2023-12-26 18:44:07,223][105620] Updated weights for policy 1, policy_version 463373 (0.0011) [2023-12-26 18:44:07,284][105620] Updated weights for policy 1, policy_version 463383 (0.0010) [2023-12-26 18:44:07,346][105620] Updated weights for policy 1, policy_version 463393 (0.0009) [2023-12-26 18:44:07,535][105692] Updated weights for policy 0, policy_version 462991 (0.0008) [2023-12-26 18:44:07,585][105692] Updated weights for policy 0, policy_version 463001 (0.0009) [2023-12-26 18:44:07,646][105692] Updated weights for policy 0, policy_version 463011 (0.0008) [2023-12-26 18:44:08,099][105620] Updated weights for policy 1, policy_version 463403 (0.0009) [2023-12-26 18:44:08,158][105620] Updated weights for policy 1, policy_version 463413 (0.0009) [2023-12-26 18:44:08,206][105620] Updated weights for policy 1, policy_version 463423 (0.0006) [2023-12-26 18:44:08,446][105692] Updated weights for policy 0, policy_version 463021 (0.0008) [2023-12-26 18:44:08,500][105692] Updated weights for policy 0, policy_version 463031 (0.0009) [2023-12-26 18:44:08,566][105692] Updated weights for policy 0, policy_version 463041 (0.0009) [2023-12-26 18:44:08,849][105620] Updated weights for policy 1, policy_version 463433 (0.0006) [2023-12-26 18:44:08,902][105620] Updated weights for policy 1, policy_version 463443 (0.0009) [2023-12-26 18:44:08,958][105620] Updated weights for policy 1, policy_version 463453 (0.0009) [2023-12-26 18:44:09,022][105620] Updated weights for policy 1, policy_version 463463 (0.0009) [2023-12-26 18:44:09,362][105692] Updated weights for policy 0, policy_version 463051 (0.0008) [2023-12-26 18:44:09,427][105692] Updated weights for policy 0, policy_version 463061 (0.0008) [2023-12-26 18:44:09,486][105692] Updated weights for policy 0, policy_version 463071 (0.0009) [2023-12-26 18:44:09,786][105620] Updated weights for policy 1, policy_version 463473 (0.0008) [2023-12-26 18:44:09,846][105620] Updated weights for policy 1, policy_version 463483 (0.0009) [2023-12-26 18:44:09,908][105620] Updated weights for policy 1, policy_version 463493 (0.0009) [2023-12-26 18:44:10,261][105692] Updated weights for policy 0, policy_version 463081 (0.0009) [2023-12-26 18:44:10,328][105692] Updated weights for policy 0, policy_version 463091 (0.0011) [2023-12-26 18:44:10,385][105692] Updated weights for policy 0, policy_version 463101 (0.0011) [2023-12-26 18:44:10,439][105692] Updated weights for policy 0, policy_version 463111 (0.0011) [2023-12-26 18:44:10,662][105620] Updated weights for policy 1, policy_version 463503 (0.0008) [2023-12-26 18:44:10,730][105620] Updated weights for policy 1, policy_version 463513 (0.0006) [2023-12-26 18:44:10,791][105620] Updated weights for policy 1, policy_version 463523 (0.0005) [2023-12-26 18:44:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 237248512. Throughput: 0: 9784.2, 1: 9768.4. Samples: 237258096. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:44:11,063][104569] Avg episode reward: [(0, '8375.010'), (1, '9270.340')] [2023-12-26 18:44:11,210][105692] Updated weights for policy 0, policy_version 463121 (0.0009) [2023-12-26 18:44:11,265][105692] Updated weights for policy 0, policy_version 463131 (0.0007) [2023-12-26 18:44:11,325][105692] Updated weights for policy 0, policy_version 463141 (0.0008) [2023-12-26 18:44:11,426][105620] Updated weights for policy 1, policy_version 463533 (0.0007) [2023-12-26 18:44:11,481][105620] Updated weights for policy 1, policy_version 463543 (0.0010) [2023-12-26 18:44:11,536][105620] Updated weights for policy 1, policy_version 463553 (0.0010) [2023-12-26 18:44:12,069][105692] Updated weights for policy 0, policy_version 463151 (0.0007) [2023-12-26 18:44:12,129][105692] Updated weights for policy 0, policy_version 463161 (0.0005) [2023-12-26 18:44:12,183][105692] Updated weights for policy 0, policy_version 463171 (0.0009) [2023-12-26 18:44:12,309][105620] Updated weights for policy 1, policy_version 463563 (0.0010) [2023-12-26 18:44:12,374][105620] Updated weights for policy 1, policy_version 463573 (0.0009) [2023-12-26 18:44:12,444][105620] Updated weights for policy 1, policy_version 463583 (0.0006) [2023-12-26 18:44:12,904][105692] Updated weights for policy 0, policy_version 463181 (0.0010) [2023-12-26 18:44:12,969][105692] Updated weights for policy 0, policy_version 463191 (0.0010) [2023-12-26 18:44:13,030][105692] Updated weights for policy 0, policy_version 463201 (0.0010) [2023-12-26 18:44:13,127][105620] Updated weights for policy 1, policy_version 463593 (0.0006) [2023-12-26 18:44:13,191][105620] Updated weights for policy 1, policy_version 463603 (0.0008) [2023-12-26 18:44:13,249][105620] Updated weights for policy 1, policy_version 463613 (0.0008) [2023-12-26 18:44:13,317][105620] Updated weights for policy 1, policy_version 463623 (0.0009) [2023-12-26 18:44:13,780][105692] Updated weights for policy 0, policy_version 463211 (0.0010) [2023-12-26 18:44:13,834][105692] Updated weights for policy 0, policy_version 463221 (0.0009) [2023-12-26 18:44:13,886][105692] Updated weights for policy 0, policy_version 463231 (0.0005) [2023-12-26 18:44:13,907][105585] KL-divergence is very high: 121.8965 [2023-12-26 18:44:14,072][105620] Updated weights for policy 1, policy_version 463633 (0.0006) [2023-12-26 18:44:14,133][105620] Updated weights for policy 1, policy_version 463643 (0.0008) [2023-12-26 18:44:14,191][105620] Updated weights for policy 1, policy_version 463653 (0.0009) [2023-12-26 18:44:14,613][105692] Updated weights for policy 0, policy_version 463241 (0.0007) [2023-12-26 18:44:14,668][105692] Updated weights for policy 0, policy_version 463251 (0.0006) [2023-12-26 18:44:14,726][105692] Updated weights for policy 0, policy_version 463261 (0.0005) [2023-12-26 18:44:14,733][105585] KL-divergence is very high: 103.7782 [2023-12-26 18:44:14,739][105585] KL-divergence is very high: 131.7772 [2023-12-26 18:44:14,745][105585] KL-divergence is very high: 176.4669 [2023-12-26 18:44:14,756][105585] KL-divergence is very high: 224.8001 [2023-12-26 18:44:14,769][105585] KL-divergence is very high: 223.7442 [2023-12-26 18:44:14,776][105585] KL-divergence is very high: 137.3960 [2023-12-26 18:44:14,782][105585] KL-divergence is very high: 285.9552 [2023-12-26 18:44:14,787][105585] KL-divergence is very high: 258.8260 [2023-12-26 18:44:14,789][105692] Updated weights for policy 0, policy_version 463271 (0.0007) [2023-12-26 18:44:14,892][105620] Updated weights for policy 1, policy_version 463663 (0.0008) [2023-12-26 18:44:14,950][105620] Updated weights for policy 1, policy_version 463673 (0.0007) [2023-12-26 18:44:14,998][105620] Updated weights for policy 1, policy_version 463683 (0.0008) [2023-12-26 18:44:15,509][105692] Updated weights for policy 0, policy_version 463281 (0.0009) [2023-12-26 18:44:15,556][105692] Updated weights for policy 0, policy_version 463291 (0.0008) [2023-12-26 18:44:15,603][105692] Updated weights for policy 0, policy_version 463301 (0.0009) [2023-12-26 18:44:15,707][105620] Updated weights for policy 1, policy_version 463693 (0.0009) [2023-12-26 18:44:15,766][105620] Updated weights for policy 1, policy_version 463703 (0.0008) [2023-12-26 18:44:15,833][105620] Updated weights for policy 1, policy_version 463713 (0.0006) [2023-12-26 18:44:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 237346816. Throughput: 0: 9714.8, 1: 9650.9. Samples: 237314908. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:44:16,063][104569] Avg episode reward: [(0, '5389.506'), (1, '8999.998')] [2023-12-26 18:44:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000463304_118620160.pth... [2023-12-26 18:44:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000463720_118726656.pth... [2023-12-26 18:44:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000462600_118439936.pth [2023-12-26 18:44:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000462184_118333440.pth [2023-12-26 18:44:16,442][105620] Updated weights for policy 1, policy_version 463723 (0.0006) [2023-12-26 18:44:16,461][105692] Updated weights for policy 0, policy_version 463311 (0.0007) [2023-12-26 18:44:16,506][105620] Updated weights for policy 1, policy_version 463733 (0.0008) [2023-12-26 18:44:16,519][105692] Updated weights for policy 0, policy_version 463321 (0.0005) [2023-12-26 18:44:16,571][105692] Updated weights for policy 0, policy_version 463331 (0.0005) [2023-12-26 18:44:16,572][105620] Updated weights for policy 1, policy_version 463743 (0.0008) [2023-12-26 18:44:17,187][105692] Updated weights for policy 0, policy_version 463341 (0.0008) [2023-12-26 18:44:17,189][105620] Updated weights for policy 1, policy_version 463753 (0.0006) [2023-12-26 18:44:17,246][105692] Updated weights for policy 0, policy_version 463351 (0.0006) [2023-12-26 18:44:17,261][105620] Updated weights for policy 1, policy_version 463763 (0.0009) [2023-12-26 18:44:17,309][105692] Updated weights for policy 0, policy_version 463361 (0.0006) [2023-12-26 18:44:17,328][105620] Updated weights for policy 1, policy_version 463773 (0.0008) [2023-12-26 18:44:17,384][105620] Updated weights for policy 1, policy_version 463783 (0.0008) [2023-12-26 18:44:17,960][105692] Updated weights for policy 0, policy_version 463371 (0.0008) [2023-12-26 18:44:18,005][105620] Updated weights for policy 1, policy_version 463793 (0.0006) [2023-12-26 18:44:18,020][105692] Updated weights for policy 0, policy_version 463381 (0.0008) [2023-12-26 18:44:18,063][105620] Updated weights for policy 1, policy_version 463803 (0.0006) [2023-12-26 18:44:18,084][105692] Updated weights for policy 0, policy_version 463391 (0.0007) [2023-12-26 18:44:18,134][105620] Updated weights for policy 1, policy_version 463813 (0.0006) [2023-12-26 18:44:18,715][105692] Updated weights for policy 0, policy_version 463401 (0.0007) [2023-12-26 18:44:18,766][105692] Updated weights for policy 0, policy_version 463411 (0.0005) [2023-12-26 18:44:18,823][105692] Updated weights for policy 0, policy_version 463421 (0.0006) [2023-12-26 18:44:18,840][105620] Updated weights for policy 1, policy_version 463823 (0.0010) [2023-12-26 18:44:18,878][105692] Updated weights for policy 0, policy_version 463431 (0.0006) [2023-12-26 18:44:18,899][105620] Updated weights for policy 1, policy_version 463833 (0.0010) [2023-12-26 18:44:18,962][105620] Updated weights for policy 1, policy_version 463843 (0.0011) [2023-12-26 18:44:19,553][105692] Updated weights for policy 0, policy_version 463441 (0.0008) [2023-12-26 18:44:19,620][105692] Updated weights for policy 0, policy_version 463451 (0.0008) [2023-12-26 18:44:19,668][105620] Updated weights for policy 1, policy_version 463853 (0.0008) [2023-12-26 18:44:19,684][105692] Updated weights for policy 0, policy_version 463461 (0.0010) [2023-12-26 18:44:19,725][105620] Updated weights for policy 1, policy_version 463863 (0.0008) [2023-12-26 18:44:19,788][105620] Updated weights for policy 1, policy_version 463873 (0.0011) [2023-12-26 18:44:20,400][105692] Updated weights for policy 0, policy_version 463471 (0.0007) [2023-12-26 18:44:20,463][105692] Updated weights for policy 0, policy_version 463481 (0.0006) [2023-12-26 18:44:20,516][105692] Updated weights for policy 0, policy_version 463491 (0.0010) [2023-12-26 18:44:20,531][105620] Updated weights for policy 1, policy_version 463883 (0.0011) [2023-12-26 18:44:20,592][105620] Updated weights for policy 1, policy_version 463893 (0.0010) [2023-12-26 18:44:20,654][105620] Updated weights for policy 1, policy_version 463903 (0.0008) [2023-12-26 18:44:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 237445120. Throughput: 0: 9777.8, 1: 9674.3. Samples: 237435244. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:44:21,062][104569] Avg episode reward: [(0, '910.765'), (1, '9182.275')] [2023-12-26 18:44:21,242][105692] Updated weights for policy 0, policy_version 463501 (0.0010) [2023-12-26 18:44:21,303][105692] Updated weights for policy 0, policy_version 463511 (0.0011) [2023-12-26 18:44:21,380][105692] Updated weights for policy 0, policy_version 463521 (0.0010) [2023-12-26 18:44:21,414][105620] Updated weights for policy 1, policy_version 463913 (0.0008) [2023-12-26 18:44:21,475][105620] Updated weights for policy 1, policy_version 463923 (0.0008) [2023-12-26 18:44:21,539][105620] Updated weights for policy 1, policy_version 463933 (0.0008) [2023-12-26 18:44:21,601][105620] Updated weights for policy 1, policy_version 463943 (0.0007) [2023-12-26 18:44:22,122][105692] Updated weights for policy 0, policy_version 463531 (0.0010) [2023-12-26 18:44:22,183][105692] Updated weights for policy 0, policy_version 463541 (0.0011) [2023-12-26 18:44:22,236][105692] Updated weights for policy 0, policy_version 463551 (0.0010) [2023-12-26 18:44:22,387][105620] Updated weights for policy 1, policy_version 463953 (0.0008) [2023-12-26 18:44:22,460][105620] Updated weights for policy 1, policy_version 463963 (0.0006) [2023-12-26 18:44:22,533][105620] Updated weights for policy 1, policy_version 463973 (0.0006) [2023-12-26 18:44:22,956][105692] Updated weights for policy 0, policy_version 463561 (0.0011) [2023-12-26 18:44:23,011][105692] Updated weights for policy 0, policy_version 463571 (0.0009) [2023-12-26 18:44:23,059][105692] Updated weights for policy 0, policy_version 463581 (0.0009) [2023-12-26 18:44:23,114][105692] Updated weights for policy 0, policy_version 463591 (0.0009) [2023-12-26 18:44:23,163][105620] Updated weights for policy 1, policy_version 463983 (0.0008) [2023-12-26 18:44:23,218][105620] Updated weights for policy 1, policy_version 463993 (0.0009) [2023-12-26 18:44:23,273][105620] Updated weights for policy 1, policy_version 464003 (0.0009) [2023-12-26 18:44:23,905][105692] Updated weights for policy 0, policy_version 463601 (0.0006) [2023-12-26 18:44:23,971][105692] Updated weights for policy 0, policy_version 463611 (0.0009) [2023-12-26 18:44:23,985][105620] Updated weights for policy 1, policy_version 464013 (0.0007) [2023-12-26 18:44:24,032][105692] Updated weights for policy 0, policy_version 463621 (0.0011) [2023-12-26 18:44:24,034][105620] Updated weights for policy 1, policy_version 464023 (0.0006) [2023-12-26 18:44:24,088][105620] Updated weights for policy 1, policy_version 464033 (0.0007) [2023-12-26 18:44:24,748][105620] Updated weights for policy 1, policy_version 464043 (0.0008) [2023-12-26 18:44:24,766][105692] Updated weights for policy 0, policy_version 463631 (0.0011) [2023-12-26 18:44:24,795][105620] Updated weights for policy 1, policy_version 464053 (0.0008) [2023-12-26 18:44:24,815][105692] Updated weights for policy 0, policy_version 463641 (0.0010) [2023-12-26 18:44:24,851][105620] Updated weights for policy 1, policy_version 464063 (0.0005) [2023-12-26 18:44:24,877][105692] Updated weights for policy 0, policy_version 463651 (0.0010) [2023-12-26 18:44:25,497][105620] Updated weights for policy 1, policy_version 464073 (0.0010) [2023-12-26 18:44:25,554][105620] Updated weights for policy 1, policy_version 464083 (0.0010) [2023-12-26 18:44:25,562][105692] Updated weights for policy 0, policy_version 463661 (0.0010) [2023-12-26 18:44:25,609][105620] Updated weights for policy 1, policy_version 464093 (0.0010) [2023-12-26 18:44:25,621][105692] Updated weights for policy 0, policy_version 463671 (0.0010) [2023-12-26 18:44:25,665][105620] Updated weights for policy 1, policy_version 464103 (0.0010) [2023-12-26 18:44:25,680][105692] Updated weights for policy 0, policy_version 463681 (0.0010) [2023-12-26 18:44:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 237543424. Throughput: 0: 9741.6, 1: 9675.0. Samples: 237550892. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:44:26,063][104569] Avg episode reward: [(0, '997.331'), (1, '9267.682')] [2023-12-26 18:44:26,413][105692] Updated weights for policy 0, policy_version 463691 (0.0010) [2023-12-26 18:44:26,435][105620] Updated weights for policy 1, policy_version 464113 (0.0010) [2023-12-26 18:44:26,465][105692] Updated weights for policy 0, policy_version 463701 (0.0010) [2023-12-26 18:44:26,493][105620] Updated weights for policy 1, policy_version 464123 (0.0010) [2023-12-26 18:44:26,513][105692] Updated weights for policy 0, policy_version 463711 (0.0010) [2023-12-26 18:44:26,551][105620] Updated weights for policy 1, policy_version 464133 (0.0010) [2023-12-26 18:44:27,187][105692] Updated weights for policy 0, policy_version 463721 (0.0010) [2023-12-26 18:44:27,234][105692] Updated weights for policy 0, policy_version 463731 (0.0010) [2023-12-26 18:44:27,282][105692] Updated weights for policy 0, policy_version 463741 (0.0006) [2023-12-26 18:44:27,288][105620] Updated weights for policy 1, policy_version 464143 (0.0010) [2023-12-26 18:44:27,342][105692] Updated weights for policy 0, policy_version 463751 (0.0008) [2023-12-26 18:44:27,350][105620] Updated weights for policy 1, policy_version 464153 (0.0010) [2023-12-26 18:44:27,408][105620] Updated weights for policy 1, policy_version 464163 (0.0010) [2023-12-26 18:44:28,074][105692] Updated weights for policy 0, policy_version 463761 (0.0010) [2023-12-26 18:44:28,122][105692] Updated weights for policy 0, policy_version 463771 (0.0010) [2023-12-26 18:44:28,138][105620] Updated weights for policy 1, policy_version 464173 (0.0010) [2023-12-26 18:44:28,170][105692] Updated weights for policy 0, policy_version 463781 (0.0010) [2023-12-26 18:44:28,199][105620] Updated weights for policy 1, policy_version 464183 (0.0010) [2023-12-26 18:44:28,253][105620] Updated weights for policy 1, policy_version 464193 (0.0010) [2023-12-26 18:44:28,822][105692] Updated weights for policy 0, policy_version 463791 (0.0010) [2023-12-26 18:44:28,878][105692] Updated weights for policy 0, policy_version 463801 (0.0010) [2023-12-26 18:44:28,942][105692] Updated weights for policy 0, policy_version 463811 (0.0010) [2023-12-26 18:44:28,994][105620] Updated weights for policy 1, policy_version 464203 (0.0010) [2023-12-26 18:44:29,056][105620] Updated weights for policy 1, policy_version 464213 (0.0010) [2023-12-26 18:44:29,112][105620] Updated weights for policy 1, policy_version 464223 (0.0010) [2023-12-26 18:44:29,671][105692] Updated weights for policy 0, policy_version 463821 (0.0008) [2023-12-26 18:44:29,728][105692] Updated weights for policy 0, policy_version 463831 (0.0005) [2023-12-26 18:44:29,788][105692] Updated weights for policy 0, policy_version 463841 (0.0005) [2023-12-26 18:44:29,835][105620] Updated weights for policy 1, policy_version 464233 (0.0007) [2023-12-26 18:44:29,899][105620] Updated weights for policy 1, policy_version 464243 (0.0010) [2023-12-26 18:44:29,967][105620] Updated weights for policy 1, policy_version 464253 (0.0010) [2023-12-26 18:44:30,019][105620] Updated weights for policy 1, policy_version 464263 (0.0010) [2023-12-26 18:44:30,435][105692] Updated weights for policy 0, policy_version 463851 (0.0007) [2023-12-26 18:44:30,492][105692] Updated weights for policy 0, policy_version 463861 (0.0010) [2023-12-26 18:44:30,546][105692] Updated weights for policy 0, policy_version 463871 (0.0010) [2023-12-26 18:44:30,754][105620] Updated weights for policy 1, policy_version 464273 (0.0010) [2023-12-26 18:44:30,801][105620] Updated weights for policy 1, policy_version 464283 (0.0010) [2023-12-26 18:44:30,843][105620] Updated weights for policy 1, policy_version 464293 (0.0008) [2023-12-26 18:44:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 237641728. Throughput: 0: 9749.5, 1: 9706.2. Samples: 237609220. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:44:31,063][104569] Avg episode reward: [(0, '1812.402'), (1, '9177.531')] [2023-12-26 18:44:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000463880_118767616.pth... [2023-12-26 18:44:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000464296_118874112.pth... [2023-12-26 18:44:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000462760_118480896.pth [2023-12-26 18:44:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000463144_118579200.pth [2023-12-26 18:44:31,256][105692] Updated weights for policy 0, policy_version 463881 (0.0010) [2023-12-26 18:44:31,316][105692] Updated weights for policy 0, policy_version 463891 (0.0009) [2023-12-26 18:44:31,388][105692] Updated weights for policy 0, policy_version 463901 (0.0008) [2023-12-26 18:44:31,456][105692] Updated weights for policy 0, policy_version 463911 (0.0007) [2023-12-26 18:44:31,573][105620] Updated weights for policy 1, policy_version 464303 (0.0007) [2023-12-26 18:44:31,635][105620] Updated weights for policy 1, policy_version 464313 (0.0007) [2023-12-26 18:44:31,690][105620] Updated weights for policy 1, policy_version 464323 (0.0007) [2023-12-26 18:44:32,180][105692] Updated weights for policy 0, policy_version 463921 (0.0009) [2023-12-26 18:44:32,212][105620] Updated weights for policy 1, policy_version 464333 (0.0007) [2023-12-26 18:44:32,240][105692] Updated weights for policy 0, policy_version 463931 (0.0009) [2023-12-26 18:44:32,280][105620] Updated weights for policy 1, policy_version 464343 (0.0009) [2023-12-26 18:44:32,302][105692] Updated weights for policy 0, policy_version 463941 (0.0006) [2023-12-26 18:44:32,343][105620] Updated weights for policy 1, policy_version 464353 (0.0011) [2023-12-26 18:44:32,947][105620] Updated weights for policy 1, policy_version 464363 (0.0009) [2023-12-26 18:44:33,003][105620] Updated weights for policy 1, policy_version 464373 (0.0005) [2023-12-26 18:44:33,070][105620] Updated weights for policy 1, policy_version 464383 (0.0005) [2023-12-26 18:44:33,092][105692] Updated weights for policy 0, policy_version 463951 (0.0008) [2023-12-26 18:44:33,146][105692] Updated weights for policy 0, policy_version 463961 (0.0009) [2023-12-26 18:44:33,201][105692] Updated weights for policy 0, policy_version 463971 (0.0009) [2023-12-26 18:44:33,677][105620] Updated weights for policy 1, policy_version 464393 (0.0005) [2023-12-26 18:44:33,726][105620] Updated weights for policy 1, policy_version 464403 (0.0007) [2023-12-26 18:44:33,770][105620] Updated weights for policy 1, policy_version 464413 (0.0010) [2023-12-26 18:44:33,821][105620] Updated weights for policy 1, policy_version 464423 (0.0010) [2023-12-26 18:44:34,002][105692] Updated weights for policy 0, policy_version 463981 (0.0008) [2023-12-26 18:44:34,056][105692] Updated weights for policy 0, policy_version 463991 (0.0008) [2023-12-26 18:44:34,111][105692] Updated weights for policy 0, policy_version 464001 (0.0008) [2023-12-26 18:44:34,576][105620] Updated weights for policy 1, policy_version 464433 (0.0008) [2023-12-26 18:44:34,626][105620] Updated weights for policy 1, policy_version 464443 (0.0008) [2023-12-26 18:44:34,682][105620] Updated weights for policy 1, policy_version 464453 (0.0008) [2023-12-26 18:44:34,877][105692] Updated weights for policy 0, policy_version 464011 (0.0008) [2023-12-26 18:44:34,936][105692] Updated weights for policy 0, policy_version 464021 (0.0005) [2023-12-26 18:44:34,994][105692] Updated weights for policy 0, policy_version 464031 (0.0005) [2023-12-26 18:44:35,503][105620] Updated weights for policy 1, policy_version 464463 (0.0010) [2023-12-26 18:44:35,561][105620] Updated weights for policy 1, policy_version 464473 (0.0010) [2023-12-26 18:44:35,619][105620] Updated weights for policy 1, policy_version 464483 (0.0010) [2023-12-26 18:44:35,679][105692] Updated weights for policy 0, policy_version 464041 (0.0008) [2023-12-26 18:44:35,730][105692] Updated weights for policy 0, policy_version 464051 (0.0007) [2023-12-26 18:44:35,782][105692] Updated weights for policy 0, policy_version 464061 (0.0008) [2023-12-26 18:44:35,839][105692] Updated weights for policy 0, policy_version 464071 (0.0007) [2023-12-26 18:44:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 237740032. Throughput: 0: 9765.8, 1: 9709.6. Samples: 237727272. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:44:36,062][104569] Avg episode reward: [(0, '3295.823'), (1, '9187.142')] [2023-12-26 18:44:36,327][105620] Updated weights for policy 1, policy_version 464493 (0.0010) [2023-12-26 18:44:36,393][105620] Updated weights for policy 1, policy_version 464503 (0.0010) [2023-12-26 18:44:36,460][105620] Updated weights for policy 1, policy_version 464513 (0.0011) [2023-12-26 18:44:36,627][105692] Updated weights for policy 0, policy_version 464081 (0.0009) [2023-12-26 18:44:36,685][105692] Updated weights for policy 0, policy_version 464091 (0.0008) [2023-12-26 18:44:36,743][105692] Updated weights for policy 0, policy_version 464101 (0.0008) [2023-12-26 18:44:37,215][105620] Updated weights for policy 1, policy_version 464523 (0.0010) [2023-12-26 18:44:37,279][105620] Updated weights for policy 1, policy_version 464533 (0.0011) [2023-12-26 18:44:37,338][105620] Updated weights for policy 1, policy_version 464543 (0.0011) [2023-12-26 18:44:37,489][105692] Updated weights for policy 0, policy_version 464111 (0.0008) [2023-12-26 18:44:37,540][105692] Updated weights for policy 0, policy_version 464121 (0.0008) [2023-12-26 18:44:37,590][105692] Updated weights for policy 0, policy_version 464131 (0.0008) [2023-12-26 18:44:38,080][105620] Updated weights for policy 1, policy_version 464553 (0.0010) [2023-12-26 18:44:38,135][105620] Updated weights for policy 1, policy_version 464563 (0.0010) [2023-12-26 18:44:38,193][105620] Updated weights for policy 1, policy_version 464573 (0.0010) [2023-12-26 18:44:38,243][105620] Updated weights for policy 1, policy_version 464583 (0.0010) [2023-12-26 18:44:38,370][105692] Updated weights for policy 0, policy_version 464141 (0.0008) [2023-12-26 18:44:38,429][105692] Updated weights for policy 0, policy_version 464151 (0.0008) [2023-12-26 18:44:38,487][105692] Updated weights for policy 0, policy_version 464161 (0.0008) [2023-12-26 18:44:39,024][105620] Updated weights for policy 1, policy_version 464593 (0.0010) [2023-12-26 18:44:39,055][105586] KL-divergence is very high: 103.9578 [2023-12-26 18:44:39,091][105586] KL-divergence is very high: 298.2339 [2023-12-26 18:44:39,092][105620] Updated weights for policy 1, policy_version 464603 (0.0010) [2023-12-26 18:44:39,111][105586] KL-divergence is very high: 184.3158 [2023-12-26 18:44:39,142][105586] KL-divergence is very high: 329.4559 [2023-12-26 18:44:39,156][105620] Updated weights for policy 1, policy_version 464613 (0.0011) [2023-12-26 18:44:39,161][105586] KL-divergence is very high: 164.5916 [2023-12-26 18:44:39,251][105692] Updated weights for policy 0, policy_version 464171 (0.0008) [2023-12-26 18:44:39,318][105692] Updated weights for policy 0, policy_version 464181 (0.0008) [2023-12-26 18:44:39,377][105692] Updated weights for policy 0, policy_version 464191 (0.0008) [2023-12-26 18:44:39,912][105620] Updated weights for policy 1, policy_version 464623 (0.0011) [2023-12-26 18:44:39,969][105586] KL-divergence is very high: 120.9539 [2023-12-26 18:44:39,976][105620] Updated weights for policy 1, policy_version 464633 (0.0009) [2023-12-26 18:44:40,006][105586] KL-divergence is very high: 115.2231 [2023-12-26 18:44:40,016][105586] KL-divergence is very high: 148.2774 [2023-12-26 18:44:40,031][105620] Updated weights for policy 1, policy_version 464643 (0.0008) [2023-12-26 18:44:40,048][105586] KL-divergence is very high: 108.3946 [2023-12-26 18:44:40,194][105692] Updated weights for policy 0, policy_version 464201 (0.0008) [2023-12-26 18:44:40,246][105692] Updated weights for policy 0, policy_version 464211 (0.0010) [2023-12-26 18:44:40,303][105692] Updated weights for policy 0, policy_version 464221 (0.0009) [2023-12-26 18:44:40,365][105692] Updated weights for policy 0, policy_version 464231 (0.0007) [2023-12-26 18:44:40,745][105586] KL-divergence is very high: 184.5732 [2023-12-26 18:44:40,753][105620] Updated weights for policy 1, policy_version 464653 (0.0008) [2023-12-26 18:44:40,790][105586] KL-divergence is very high: 212.4510 [2023-12-26 18:44:40,804][105620] Updated weights for policy 1, policy_version 464663 (0.0006) [2023-12-26 18:44:40,828][105586] KL-divergence is very high: 190.4118 [2023-12-26 18:44:40,853][105620] Updated weights for policy 1, policy_version 464673 (0.0005) [2023-12-26 18:44:40,873][105586] KL-divergence is very high: 134.1834 [2023-12-26 18:44:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 237830144. Throughput: 0: 9668.6, 1: 9697.3. Samples: 237839280. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:44:41,063][104569] Avg episode reward: [(0, '6456.165'), (1, '7684.495')] [2023-12-26 18:44:41,074][105692] Updated weights for policy 0, policy_version 464241 (0.0008) [2023-12-26 18:44:41,140][105692] Updated weights for policy 0, policy_version 464251 (0.0009) [2023-12-26 18:44:41,202][105692] Updated weights for policy 0, policy_version 464261 (0.0009) [2023-12-26 18:44:41,609][105620] Updated weights for policy 1, policy_version 464683 (0.0007) [2023-12-26 18:44:41,665][105620] Updated weights for policy 1, policy_version 464693 (0.0009) [2023-12-26 18:44:41,717][105620] Updated weights for policy 1, policy_version 464703 (0.0009) [2023-12-26 18:44:42,019][105692] Updated weights for policy 0, policy_version 464271 (0.0009) [2023-12-26 18:44:42,078][105692] Updated weights for policy 0, policy_version 464281 (0.0009) [2023-12-26 18:44:42,140][105692] Updated weights for policy 0, policy_version 464291 (0.0009) [2023-12-26 18:44:42,405][105620] Updated weights for policy 1, policy_version 464713 (0.0007) [2023-12-26 18:44:42,466][105620] Updated weights for policy 1, policy_version 464723 (0.0007) [2023-12-26 18:44:42,530][105620] Updated weights for policy 1, policy_version 464733 (0.0006) [2023-12-26 18:44:42,590][105620] Updated weights for policy 1, policy_version 464743 (0.0009) [2023-12-26 18:44:42,808][105692] Updated weights for policy 0, policy_version 464301 (0.0008) [2023-12-26 18:44:42,864][105692] Updated weights for policy 0, policy_version 464311 (0.0005) [2023-12-26 18:44:42,920][105692] Updated weights for policy 0, policy_version 464321 (0.0005) [2023-12-26 18:44:43,315][105620] Updated weights for policy 1, policy_version 464753 (0.0008) [2023-12-26 18:44:43,374][105620] Updated weights for policy 1, policy_version 464763 (0.0008) [2023-12-26 18:44:43,436][105620] Updated weights for policy 1, policy_version 464773 (0.0008) [2023-12-26 18:44:43,532][105692] Updated weights for policy 0, policy_version 464331 (0.0005) [2023-12-26 18:44:43,590][105692] Updated weights for policy 0, policy_version 464341 (0.0005) [2023-12-26 18:44:43,655][105692] Updated weights for policy 0, policy_version 464351 (0.0005) [2023-12-26 18:44:44,117][105620] Updated weights for policy 1, policy_version 464783 (0.0006) [2023-12-26 18:44:44,182][105620] Updated weights for policy 1, policy_version 464793 (0.0005) [2023-12-26 18:44:44,214][105692] Updated weights for policy 0, policy_version 464361 (0.0005) [2023-12-26 18:44:44,233][105620] Updated weights for policy 1, policy_version 464803 (0.0006) [2023-12-26 18:44:44,263][105692] Updated weights for policy 0, policy_version 464371 (0.0005) [2023-12-26 18:44:44,314][105692] Updated weights for policy 0, policy_version 464381 (0.0006) [2023-12-26 18:44:44,373][105692] Updated weights for policy 0, policy_version 464391 (0.0006) [2023-12-26 18:44:44,839][105620] Updated weights for policy 1, policy_version 464813 (0.0006) [2023-12-26 18:44:44,903][105620] Updated weights for policy 1, policy_version 464823 (0.0007) [2023-12-26 18:44:44,967][105620] Updated weights for policy 1, policy_version 464833 (0.0010) [2023-12-26 18:44:45,078][105692] Updated weights for policy 0, policy_version 464401 (0.0008) [2023-12-26 18:44:45,145][105692] Updated weights for policy 0, policy_version 464411 (0.0010) [2023-12-26 18:44:45,202][105692] Updated weights for policy 0, policy_version 464421 (0.0010) [2023-12-26 18:44:45,578][105620] Updated weights for policy 1, policy_version 464843 (0.0011) [2023-12-26 18:44:45,639][105620] Updated weights for policy 1, policy_version 464853 (0.0010) [2023-12-26 18:44:45,691][105620] Updated weights for policy 1, policy_version 464863 (0.0010) [2023-12-26 18:44:45,903][105692] Updated weights for policy 0, policy_version 464431 (0.0007) [2023-12-26 18:44:45,955][105692] Updated weights for policy 0, policy_version 464441 (0.0006) [2023-12-26 18:44:46,007][105692] Updated weights for policy 0, policy_version 464451 (0.0008) [2023-12-26 18:44:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 237936640. Throughput: 0: 9619.7, 1: 9741.2. Samples: 237897760. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:44:46,062][104569] Avg episode reward: [(0, '8276.348'), (1, '5249.209')] [2023-12-26 18:44:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000464456_118915072.pth... [2023-12-26 18:44:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000464872_119021568.pth... [2023-12-26 18:44:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000463304_118620160.pth [2023-12-26 18:44:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000463720_118726656.pth [2023-12-26 18:44:46,411][105620] Updated weights for policy 1, policy_version 464873 (0.0010) [2023-12-26 18:44:46,463][105620] Updated weights for policy 1, policy_version 464883 (0.0010) [2023-12-26 18:44:46,511][105620] Updated weights for policy 1, policy_version 464893 (0.0010) [2023-12-26 18:44:46,559][105620] Updated weights for policy 1, policy_version 464903 (0.0010) [2023-12-26 18:44:46,636][105692] Updated weights for policy 0, policy_version 464461 (0.0010) [2023-12-26 18:44:46,691][105692] Updated weights for policy 0, policy_version 464471 (0.0009) [2023-12-26 18:44:46,750][105692] Updated weights for policy 0, policy_version 464481 (0.0005) [2023-12-26 18:44:47,258][105620] Updated weights for policy 1, policy_version 464913 (0.0008) [2023-12-26 18:44:47,310][105620] Updated weights for policy 1, policy_version 464923 (0.0010) [2023-12-26 18:44:47,352][105692] Updated weights for policy 0, policy_version 464491 (0.0005) [2023-12-26 18:44:47,362][105620] Updated weights for policy 1, policy_version 464933 (0.0010) [2023-12-26 18:44:47,411][105692] Updated weights for policy 0, policy_version 464501 (0.0006) [2023-12-26 18:44:47,474][105692] Updated weights for policy 0, policy_version 464511 (0.0010) [2023-12-26 18:44:48,106][105620] Updated weights for policy 1, policy_version 464943 (0.0008) [2023-12-26 18:44:48,173][105620] Updated weights for policy 1, policy_version 464953 (0.0005) [2023-12-26 18:44:48,186][105692] Updated weights for policy 0, policy_version 464521 (0.0010) [2023-12-26 18:44:48,233][105620] Updated weights for policy 1, policy_version 464963 (0.0008) [2023-12-26 18:44:48,247][105692] Updated weights for policy 0, policy_version 464531 (0.0010) [2023-12-26 18:44:48,307][105692] Updated weights for policy 0, policy_version 464541 (0.0011) [2023-12-26 18:44:48,369][105692] Updated weights for policy 0, policy_version 464551 (0.0008) [2023-12-26 18:44:48,937][105620] Updated weights for policy 1, policy_version 464973 (0.0011) [2023-12-26 18:44:48,963][105692] Updated weights for policy 0, policy_version 464561 (0.0010) [2023-12-26 18:44:48,999][105620] Updated weights for policy 1, policy_version 464983 (0.0010) [2023-12-26 18:44:49,022][105692] Updated weights for policy 0, policy_version 464571 (0.0011) [2023-12-26 18:44:49,061][105620] Updated weights for policy 1, policy_version 464993 (0.0010) [2023-12-26 18:44:49,074][105692] Updated weights for policy 0, policy_version 464581 (0.0011) [2023-12-26 18:44:49,750][105620] Updated weights for policy 1, policy_version 465003 (0.0009) [2023-12-26 18:44:49,794][105692] Updated weights for policy 0, policy_version 464591 (0.0008) [2023-12-26 18:44:49,795][105620] Updated weights for policy 1, policy_version 465013 (0.0005) [2023-12-26 18:44:49,858][105692] Updated weights for policy 0, policy_version 464601 (0.0008) [2023-12-26 18:44:49,861][105620] Updated weights for policy 1, policy_version 465023 (0.0007) [2023-12-26 18:44:49,905][105692] Updated weights for policy 0, policy_version 464611 (0.0006) [2023-12-26 18:44:50,576][105692] Updated weights for policy 0, policy_version 464621 (0.0009) [2023-12-26 18:44:50,624][105620] Updated weights for policy 1, policy_version 465033 (0.0008) [2023-12-26 18:44:50,634][105692] Updated weights for policy 0, policy_version 464631 (0.0009) [2023-12-26 18:44:50,682][105620] Updated weights for policy 1, policy_version 465043 (0.0007) [2023-12-26 18:44:50,685][105692] Updated weights for policy 0, policy_version 464641 (0.0006) [2023-12-26 18:44:50,739][105620] Updated weights for policy 1, policy_version 465053 (0.0009) [2023-12-26 18:44:50,798][105620] Updated weights for policy 1, policy_version 465063 (0.0009) [2023-12-26 18:44:51,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 238034944. Throughput: 0: 9717.5, 1: 9791.2. Samples: 238021768. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:44:51,062][104569] Avg episode reward: [(0, '8640.140'), (1, '4781.134')] [2023-12-26 18:44:51,475][105692] Updated weights for policy 0, policy_version 464651 (0.0007) [2023-12-26 18:44:51,535][105692] Updated weights for policy 0, policy_version 464661 (0.0008) [2023-12-26 18:44:51,573][105620] Updated weights for policy 1, policy_version 465073 (0.0008) [2023-12-26 18:44:51,602][105692] Updated weights for policy 0, policy_version 464671 (0.0007) [2023-12-26 18:44:51,641][105620] Updated weights for policy 1, policy_version 465083 (0.0008) [2023-12-26 18:44:51,697][105620] Updated weights for policy 1, policy_version 465093 (0.0008) [2023-12-26 18:44:52,334][105692] Updated weights for policy 0, policy_version 464681 (0.0008) [2023-12-26 18:44:52,411][105692] Updated weights for policy 0, policy_version 464691 (0.0007) [2023-12-26 18:44:52,473][105692] Updated weights for policy 0, policy_version 464701 (0.0007) [2023-12-26 18:44:52,475][105620] Updated weights for policy 1, policy_version 465103 (0.0008) [2023-12-26 18:44:52,531][105620] Updated weights for policy 1, policy_version 465113 (0.0008) [2023-12-26 18:44:52,537][105692] Updated weights for policy 0, policy_version 464711 (0.0006) [2023-12-26 18:44:52,584][105620] Updated weights for policy 1, policy_version 465123 (0.0006) [2023-12-26 18:44:53,235][105620] Updated weights for policy 1, policy_version 465133 (0.0007) [2023-12-26 18:44:53,267][105692] Updated weights for policy 0, policy_version 464721 (0.0009) [2023-12-26 18:44:53,290][105620] Updated weights for policy 1, policy_version 465143 (0.0007) [2023-12-26 18:44:53,325][105692] Updated weights for policy 0, policy_version 464731 (0.0010) [2023-12-26 18:44:53,340][105620] Updated weights for policy 1, policy_version 465153 (0.0007) [2023-12-26 18:44:53,380][105692] Updated weights for policy 0, policy_version 464741 (0.0010) [2023-12-26 18:44:53,944][105620] Updated weights for policy 1, policy_version 465163 (0.0008) [2023-12-26 18:44:54,003][105620] Updated weights for policy 1, policy_version 465173 (0.0006) [2023-12-26 18:44:54,050][105620] Updated weights for policy 1, policy_version 465183 (0.0005) [2023-12-26 18:44:54,105][105692] Updated weights for policy 0, policy_version 464751 (0.0010) [2023-12-26 18:44:54,169][105692] Updated weights for policy 0, policy_version 464761 (0.0011) [2023-12-26 18:44:54,232][105692] Updated weights for policy 0, policy_version 464771 (0.0009) [2023-12-26 18:44:54,808][105620] Updated weights for policy 1, policy_version 465193 (0.0006) [2023-12-26 18:44:54,860][105620] Updated weights for policy 1, policy_version 465203 (0.0008) [2023-12-26 18:44:54,873][105692] Updated weights for policy 0, policy_version 464781 (0.0008) [2023-12-26 18:44:54,923][105620] Updated weights for policy 1, policy_version 465213 (0.0005) [2023-12-26 18:44:54,931][105692] Updated weights for policy 0, policy_version 464791 (0.0010) [2023-12-26 18:44:54,985][105692] Updated weights for policy 0, policy_version 464801 (0.0011) [2023-12-26 18:44:54,986][105620] Updated weights for policy 1, policy_version 465223 (0.0006) [2023-12-26 18:44:55,627][105620] Updated weights for policy 1, policy_version 465233 (0.0007) [2023-12-26 18:44:55,687][105620] Updated weights for policy 1, policy_version 465243 (0.0008) [2023-12-26 18:44:55,706][105692] Updated weights for policy 0, policy_version 464811 (0.0009) [2023-12-26 18:44:55,744][105620] Updated weights for policy 1, policy_version 465253 (0.0008) [2023-12-26 18:44:55,754][105692] Updated weights for policy 0, policy_version 464821 (0.0005) [2023-12-26 18:44:55,810][105692] Updated weights for policy 0, policy_version 464831 (0.0007) [2023-12-26 18:44:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 238133248. Throughput: 0: 9750.1, 1: 9816.0. Samples: 238138568. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:44:56,062][104569] Avg episode reward: [(0, '8644.595'), (1, '7907.503')] [2023-12-26 18:44:56,442][105620] Updated weights for policy 1, policy_version 465263 (0.0010) [2023-12-26 18:44:56,494][105620] Updated weights for policy 1, policy_version 465273 (0.0010) [2023-12-26 18:44:56,527][105692] Updated weights for policy 0, policy_version 464841 (0.0010) [2023-12-26 18:44:56,543][105620] Updated weights for policy 1, policy_version 465283 (0.0010) [2023-12-26 18:44:56,578][105692] Updated weights for policy 0, policy_version 464851 (0.0010) [2023-12-26 18:44:56,629][105692] Updated weights for policy 0, policy_version 464861 (0.0010) [2023-12-26 18:44:56,680][105692] Updated weights for policy 0, policy_version 464871 (0.0010) [2023-12-26 18:44:57,308][105620] Updated weights for policy 1, policy_version 465293 (0.0010) [2023-12-26 18:44:57,367][105620] Updated weights for policy 1, policy_version 465303 (0.0010) [2023-12-26 18:44:57,414][105692] Updated weights for policy 0, policy_version 464881 (0.0010) [2023-12-26 18:44:57,421][105620] Updated weights for policy 1, policy_version 465313 (0.0010) [2023-12-26 18:44:57,468][105692] Updated weights for policy 0, policy_version 464891 (0.0010) [2023-12-26 18:44:57,529][105692] Updated weights for policy 0, policy_version 464901 (0.0010) [2023-12-26 18:44:58,041][105620] Updated weights for policy 1, policy_version 465323 (0.0010) [2023-12-26 18:44:58,101][105620] Updated weights for policy 1, policy_version 465333 (0.0010) [2023-12-26 18:44:58,165][105620] Updated weights for policy 1, policy_version 465343 (0.0007) [2023-12-26 18:44:58,167][105692] Updated weights for policy 0, policy_version 464911 (0.0009) [2023-12-26 18:44:58,224][105692] Updated weights for policy 0, policy_version 464921 (0.0006) [2023-12-26 18:44:58,277][105692] Updated weights for policy 0, policy_version 464931 (0.0009) [2023-12-26 18:44:59,036][105620] Updated weights for policy 1, policy_version 465353 (0.0006) [2023-12-26 18:44:59,087][105620] Updated weights for policy 1, policy_version 465363 (0.0009) [2023-12-26 18:44:59,122][105692] Updated weights for policy 0, policy_version 464941 (0.0008) [2023-12-26 18:44:59,151][105620] Updated weights for policy 1, policy_version 465373 (0.0008) [2023-12-26 18:44:59,170][105692] Updated weights for policy 0, policy_version 464951 (0.0006) [2023-12-26 18:44:59,207][105620] Updated weights for policy 1, policy_version 465383 (0.0008) [2023-12-26 18:44:59,227][105692] Updated weights for policy 0, policy_version 464961 (0.0008) [2023-12-26 18:44:59,985][105692] Updated weights for policy 0, policy_version 464971 (0.0007) [2023-12-26 18:45:00,010][105620] Updated weights for policy 1, policy_version 465393 (0.0009) [2023-12-26 18:45:00,049][105692] Updated weights for policy 0, policy_version 464981 (0.0006) [2023-12-26 18:45:00,070][105620] Updated weights for policy 1, policy_version 465403 (0.0007) [2023-12-26 18:45:00,109][105692] Updated weights for policy 0, policy_version 464991 (0.0006) [2023-12-26 18:45:00,134][105620] Updated weights for policy 1, policy_version 465413 (0.0007) [2023-12-26 18:45:00,758][105620] Updated weights for policy 1, policy_version 465423 (0.0008) [2023-12-26 18:45:00,808][105620] Updated weights for policy 1, policy_version 465433 (0.0008) [2023-12-26 18:45:00,865][105620] Updated weights for policy 1, policy_version 465443 (0.0009) [2023-12-26 18:45:00,875][105692] Updated weights for policy 0, policy_version 465001 (0.0009) [2023-12-26 18:45:00,926][105692] Updated weights for policy 0, policy_version 465011 (0.0008) [2023-12-26 18:45:00,980][105692] Updated weights for policy 0, policy_version 465021 (0.0008) [2023-12-26 18:45:01,032][105692] Updated weights for policy 0, policy_version 465031 (0.0009) [2023-12-26 18:45:01,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 238231552. Throughput: 0: 9767.6, 1: 9837.0. Samples: 238197116. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:45:01,063][104569] Avg episode reward: [(0, '8464.547'), (1, '9279.811')] [2023-12-26 18:45:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000465032_119062528.pth... [2023-12-26 18:45:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000465448_119169024.pth... [2023-12-26 18:45:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000463880_118767616.pth [2023-12-26 18:45:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000464296_118874112.pth [2023-12-26 18:45:01,549][105620] Updated weights for policy 1, policy_version 465453 (0.0006) [2023-12-26 18:45:01,598][105620] Updated weights for policy 1, policy_version 465463 (0.0005) [2023-12-26 18:45:01,664][105620] Updated weights for policy 1, policy_version 465473 (0.0007) [2023-12-26 18:45:01,855][105692] Updated weights for policy 0, policy_version 465041 (0.0009) [2023-12-26 18:45:01,907][105692] Updated weights for policy 0, policy_version 465051 (0.0009) [2023-12-26 18:45:01,955][105692] Updated weights for policy 0, policy_version 465061 (0.0009) [2023-12-26 18:45:02,350][105620] Updated weights for policy 1, policy_version 465483 (0.0007) [2023-12-26 18:45:02,420][105620] Updated weights for policy 1, policy_version 465493 (0.0009) [2023-12-26 18:45:02,470][105620] Updated weights for policy 1, policy_version 465503 (0.0009) [2023-12-26 18:45:02,672][105692] Updated weights for policy 0, policy_version 465071 (0.0010) [2023-12-26 18:45:02,731][105692] Updated weights for policy 0, policy_version 465081 (0.0010) [2023-12-26 18:45:02,779][105692] Updated weights for policy 0, policy_version 465091 (0.0008) [2023-12-26 18:45:03,248][105620] Updated weights for policy 1, policy_version 465513 (0.0009) [2023-12-26 18:45:03,301][105620] Updated weights for policy 1, policy_version 465523 (0.0010) [2023-12-26 18:45:03,349][105620] Updated weights for policy 1, policy_version 465533 (0.0010) [2023-12-26 18:45:03,392][105692] Updated weights for policy 0, policy_version 465101 (0.0005) [2023-12-26 18:45:03,397][105620] Updated weights for policy 1, policy_version 465543 (0.0010) [2023-12-26 18:45:03,443][105692] Updated weights for policy 0, policy_version 465111 (0.0005) [2023-12-26 18:45:03,505][105692] Updated weights for policy 0, policy_version 465121 (0.0005) [2023-12-26 18:45:04,139][105620] Updated weights for policy 1, policy_version 465553 (0.0008) [2023-12-26 18:45:04,177][105692] Updated weights for policy 0, policy_version 465131 (0.0006) [2023-12-26 18:45:04,204][105620] Updated weights for policy 1, policy_version 465563 (0.0006) [2023-12-26 18:45:04,239][105692] Updated weights for policy 0, policy_version 465141 (0.0008) [2023-12-26 18:45:04,262][105620] Updated weights for policy 1, policy_version 465573 (0.0006) [2023-12-26 18:45:04,301][105692] Updated weights for policy 0, policy_version 465151 (0.0008) [2023-12-26 18:45:04,906][105620] Updated weights for policy 1, policy_version 465583 (0.0008) [2023-12-26 18:45:04,974][105620] Updated weights for policy 1, policy_version 465593 (0.0010) [2023-12-26 18:45:05,024][105620] Updated weights for policy 1, policy_version 465604 (0.0010) [2023-12-26 18:45:05,056][105692] Updated weights for policy 0, policy_version 465161 (0.0009) [2023-12-26 18:45:05,113][105692] Updated weights for policy 0, policy_version 465171 (0.0009) [2023-12-26 18:45:05,179][105692] Updated weights for policy 0, policy_version 465181 (0.0009) [2023-12-26 18:45:05,234][105692] Updated weights for policy 0, policy_version 465191 (0.0010) [2023-12-26 18:45:05,775][105620] Updated weights for policy 1, policy_version 465614 (0.0009) [2023-12-26 18:45:05,832][105620] Updated weights for policy 1, policy_version 465624 (0.0009) [2023-12-26 18:45:05,878][105620] Updated weights for policy 1, policy_version 465634 (0.0009) [2023-12-26 18:45:05,974][105692] Updated weights for policy 0, policy_version 465201 (0.0008) [2023-12-26 18:45:06,023][105692] Updated weights for policy 0, policy_version 465211 (0.0009) [2023-12-26 18:45:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 238321664. Throughput: 0: 9701.2, 1: 9797.2. Samples: 238312668. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:45:06,063][104569] Avg episode reward: [(0, '8999.751'), (1, '9358.526')] [2023-12-26 18:45:06,079][105692] Updated weights for policy 0, policy_version 465221 (0.0009) [2023-12-26 18:45:06,601][105620] Updated weights for policy 1, policy_version 465644 (0.0010) [2023-12-26 18:45:06,654][105620] Updated weights for policy 1, policy_version 465654 (0.0008) [2023-12-26 18:45:06,712][105620] Updated weights for policy 1, policy_version 465664 (0.0009) [2023-12-26 18:45:06,906][105692] Updated weights for policy 0, policy_version 465231 (0.0008) [2023-12-26 18:45:06,966][105692] Updated weights for policy 0, policy_version 465241 (0.0008) [2023-12-26 18:45:07,014][105692] Updated weights for policy 0, policy_version 465251 (0.0007) [2023-12-26 18:45:07,345][105620] Updated weights for policy 1, policy_version 465674 (0.0005) [2023-12-26 18:45:07,408][105620] Updated weights for policy 1, policy_version 465684 (0.0006) [2023-12-26 18:45:07,474][105620] Updated weights for policy 1, policy_version 465694 (0.0008) [2023-12-26 18:45:07,527][105620] Updated weights for policy 1, policy_version 465704 (0.0010) [2023-12-26 18:45:07,873][105692] Updated weights for policy 0, policy_version 465261 (0.0007) [2023-12-26 18:45:07,938][105692] Updated weights for policy 0, policy_version 465271 (0.0005) [2023-12-26 18:45:08,004][105692] Updated weights for policy 0, policy_version 465281 (0.0006) [2023-12-26 18:45:08,178][105620] Updated weights for policy 1, policy_version 465714 (0.0010) [2023-12-26 18:45:08,231][105620] Updated weights for policy 1, policy_version 465724 (0.0010) [2023-12-26 18:45:08,289][105620] Updated weights for policy 1, policy_version 465734 (0.0005) [2023-12-26 18:45:08,630][105692] Updated weights for policy 0, policy_version 465291 (0.0009) [2023-12-26 18:45:08,674][105692] Updated weights for policy 0, policy_version 465301 (0.0011) [2023-12-26 18:45:08,723][105692] Updated weights for policy 0, policy_version 465311 (0.0010) [2023-12-26 18:45:08,827][105620] Updated weights for policy 1, policy_version 465744 (0.0005) [2023-12-26 18:45:08,876][105620] Updated weights for policy 1, policy_version 465754 (0.0005) [2023-12-26 18:45:08,925][105620] Updated weights for policy 1, policy_version 465764 (0.0005) [2023-12-26 18:45:09,516][105692] Updated weights for policy 0, policy_version 465321 (0.0010) [2023-12-26 18:45:09,573][105692] Updated weights for policy 0, policy_version 465331 (0.0010) [2023-12-26 18:45:09,637][105620] Updated weights for policy 1, policy_version 465774 (0.0005) [2023-12-26 18:45:09,639][105692] Updated weights for policy 0, policy_version 465341 (0.0008) [2023-12-26 18:45:09,692][105620] Updated weights for policy 1, policy_version 465784 (0.0010) [2023-12-26 18:45:09,694][105692] Updated weights for policy 0, policy_version 465351 (0.0007) [2023-12-26 18:45:09,753][105620] Updated weights for policy 1, policy_version 465794 (0.0008) [2023-12-26 18:45:10,466][105620] Updated weights for policy 1, policy_version 465804 (0.0009) [2023-12-26 18:45:10,506][105692] Updated weights for policy 0, policy_version 465361 (0.0006) [2023-12-26 18:45:10,535][105620] Updated weights for policy 1, policy_version 465814 (0.0010) [2023-12-26 18:45:10,567][105692] Updated weights for policy 0, policy_version 465371 (0.0008) [2023-12-26 18:45:10,594][105620] Updated weights for policy 1, policy_version 465824 (0.0010) [2023-12-26 18:45:10,627][105692] Updated weights for policy 0, policy_version 465381 (0.0011) [2023-12-26 18:45:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 238419968. Throughput: 0: 9665.5, 1: 9843.3. Samples: 238428788. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:45:11,063][104569] Avg episode reward: [(0, '8733.112'), (1, '9358.539')] [2023-12-26 18:45:11,338][105692] Updated weights for policy 0, policy_version 465391 (0.0008) [2023-12-26 18:45:11,382][105620] Updated weights for policy 1, policy_version 465834 (0.0010) [2023-12-26 18:45:11,412][105692] Updated weights for policy 0, policy_version 465401 (0.0010) [2023-12-26 18:45:11,455][105620] Updated weights for policy 1, policy_version 465844 (0.0007) [2023-12-26 18:45:11,479][105692] Updated weights for policy 0, policy_version 465411 (0.0008) [2023-12-26 18:45:11,524][105620] Updated weights for policy 1, policy_version 465854 (0.0009) [2023-12-26 18:45:11,590][105620] Updated weights for policy 1, policy_version 465864 (0.0009) [2023-12-26 18:45:12,258][105692] Updated weights for policy 0, policy_version 465421 (0.0008) [2023-12-26 18:45:12,302][105620] Updated weights for policy 1, policy_version 465874 (0.0007) [2023-12-26 18:45:12,320][105692] Updated weights for policy 0, policy_version 465431 (0.0009) [2023-12-26 18:45:12,355][105620] Updated weights for policy 1, policy_version 465884 (0.0008) [2023-12-26 18:45:12,379][105585] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000008 [2023-12-26 18:45:12,414][105620] Updated weights for policy 1, policy_version 465894 (0.0008) [2023-12-26 18:45:13,073][105692] Updated weights for policy 0, policy_version 465441 (0.0008) [2023-12-26 18:45:13,121][105692] Updated weights for policy 0, policy_version 465451 (0.0010) [2023-12-26 18:45:13,160][105620] Updated weights for policy 1, policy_version 465904 (0.0007) [2023-12-26 18:45:13,167][105692] Updated weights for policy 0, policy_version 465461 (0.0011) [2023-12-26 18:45:13,209][105620] Updated weights for policy 1, policy_version 465914 (0.0005) [2023-12-26 18:45:13,219][105692] Updated weights for policy 0, policy_version 465471 (0.0010) [2023-12-26 18:45:13,255][105620] Updated weights for policy 1, policy_version 465924 (0.0007) [2023-12-26 18:45:13,877][105692] Updated weights for policy 0, policy_version 465481 (0.0006) [2023-12-26 18:45:13,941][105692] Updated weights for policy 0, policy_version 465491 (0.0008) [2023-12-26 18:45:13,989][105692] Updated weights for policy 0, policy_version 465501 (0.0010) [2023-12-26 18:45:14,083][105620] Updated weights for policy 1, policy_version 465934 (0.0008) [2023-12-26 18:45:14,140][105620] Updated weights for policy 1, policy_version 465944 (0.0008) [2023-12-26 18:45:14,197][105620] Updated weights for policy 1, policy_version 465954 (0.0008) [2023-12-26 18:45:14,639][105692] Updated weights for policy 0, policy_version 465511 (0.0010) [2023-12-26 18:45:14,694][105692] Updated weights for policy 0, policy_version 465521 (0.0010) [2023-12-26 18:45:14,742][105692] Updated weights for policy 0, policy_version 465531 (0.0010) [2023-12-26 18:45:14,806][105620] Updated weights for policy 1, policy_version 465964 (0.0007) [2023-12-26 18:45:14,865][105620] Updated weights for policy 1, policy_version 465974 (0.0005) [2023-12-26 18:45:14,924][105620] Updated weights for policy 1, policy_version 465984 (0.0006) [2023-12-26 18:45:15,505][105620] Updated weights for policy 1, policy_version 465994 (0.0006) [2023-12-26 18:45:15,557][105620] Updated weights for policy 1, policy_version 466004 (0.0005) [2023-12-26 18:45:15,604][105692] Updated weights for policy 0, policy_version 465541 (0.0009) [2023-12-26 18:45:15,619][105620] Updated weights for policy 1, policy_version 466014 (0.0005) [2023-12-26 18:45:15,652][105692] Updated weights for policy 0, policy_version 465551 (0.0010) [2023-12-26 18:45:15,677][105620] Updated weights for policy 1, policy_version 466024 (0.0007) [2023-12-26 18:45:15,707][105692] Updated weights for policy 0, policy_version 465561 (0.0010) [2023-12-26 18:45:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 238518272. Throughput: 0: 9630.7, 1: 9820.7. Samples: 238484532. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 18:45:16,062][104569] Avg episode reward: [(0, '8733.206'), (1, '9358.695')] [2023-12-26 18:45:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000465568_119201792.pth... [2023-12-26 18:45:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000466024_119316480.pth... [2023-12-26 18:45:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000464456_118915072.pth [2023-12-26 18:45:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000464872_119021568.pth [2023-12-26 18:45:16,381][105620] Updated weights for policy 1, policy_version 466034 (0.0008) [2023-12-26 18:45:16,440][105620] Updated weights for policy 1, policy_version 466044 (0.0006) [2023-12-26 18:45:16,442][105692] Updated weights for policy 0, policy_version 465571 (0.0009) [2023-12-26 18:45:16,500][105692] Updated weights for policy 0, policy_version 465581 (0.0010) [2023-12-26 18:45:16,505][105620] Updated weights for policy 1, policy_version 466054 (0.0006) [2023-12-26 18:45:16,555][105692] Updated weights for policy 0, policy_version 465591 (0.0010) [2023-12-26 18:45:17,227][105620] Updated weights for policy 1, policy_version 466064 (0.0009) [2023-12-26 18:45:17,264][105692] Updated weights for policy 0, policy_version 465601 (0.0010) [2023-12-26 18:45:17,274][105620] Updated weights for policy 1, policy_version 466074 (0.0010) [2023-12-26 18:45:17,316][105692] Updated weights for policy 0, policy_version 465611 (0.0009) [2023-12-26 18:45:17,326][105620] Updated weights for policy 1, policy_version 466084 (0.0010) [2023-12-26 18:45:17,363][105692] Updated weights for policy 0, policy_version 465621 (0.0008) [2023-12-26 18:45:17,417][105692] Updated weights for policy 0, policy_version 465631 (0.0007) [2023-12-26 18:45:18,005][105620] Updated weights for policy 1, policy_version 466094 (0.0009) [2023-12-26 18:45:18,053][105620] Updated weights for policy 1, policy_version 466104 (0.0008) [2023-12-26 18:45:18,100][105620] Updated weights for policy 1, policy_version 466114 (0.0007) [2023-12-26 18:45:18,168][105692] Updated weights for policy 0, policy_version 465641 (0.0010) [2023-12-26 18:45:18,209][105585] KL-divergence is very high: 109.8106 [2023-12-26 18:45:18,219][105692] Updated weights for policy 0, policy_version 465651 (0.0010) [2023-12-26 18:45:18,253][105585] KL-divergence is very high: 127.4871 [2023-12-26 18:45:18,278][105692] Updated weights for policy 0, policy_version 465661 (0.0010) [2023-12-26 18:45:18,750][105620] Updated weights for policy 1, policy_version 466124 (0.0007) [2023-12-26 18:45:18,812][105620] Updated weights for policy 1, policy_version 466134 (0.0008) [2023-12-26 18:45:18,870][105620] Updated weights for policy 1, policy_version 466144 (0.0010) [2023-12-26 18:45:19,052][105692] Updated weights for policy 0, policy_version 465671 (0.0007) [2023-12-26 18:45:19,102][105692] Updated weights for policy 0, policy_version 465681 (0.0005) [2023-12-26 18:45:19,156][105692] Updated weights for policy 0, policy_version 465691 (0.0006) [2023-12-26 18:45:19,621][105620] Updated weights for policy 1, policy_version 466154 (0.0009) [2023-12-26 18:45:19,685][105620] Updated weights for policy 1, policy_version 466164 (0.0011) [2023-12-26 18:45:19,747][105620] Updated weights for policy 1, policy_version 466174 (0.0011) [2023-12-26 18:45:19,815][105620] Updated weights for policy 1, policy_version 466184 (0.0011) [2023-12-26 18:45:19,873][105692] Updated weights for policy 0, policy_version 465701 (0.0006) [2023-12-26 18:45:19,943][105692] Updated weights for policy 0, policy_version 465711 (0.0008) [2023-12-26 18:45:20,010][105692] Updated weights for policy 0, policy_version 465721 (0.0008) [2023-12-26 18:45:20,577][105620] Updated weights for policy 1, policy_version 466194 (0.0011) [2023-12-26 18:45:20,641][105620] Updated weights for policy 1, policy_version 466204 (0.0011) [2023-12-26 18:45:20,698][105620] Updated weights for policy 1, policy_version 466214 (0.0011) [2023-12-26 18:45:20,741][105692] Updated weights for policy 0, policy_version 465731 (0.0008) [2023-12-26 18:45:20,803][105692] Updated weights for policy 0, policy_version 465741 (0.0008) [2023-12-26 18:45:20,864][105692] Updated weights for policy 0, policy_version 465751 (0.0008) [2023-12-26 18:45:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 238616576. Throughput: 0: 9660.8, 1: 9820.9. Samples: 238603948. Policy #0 lag: (min: 13.0, avg: 25.8, max: 45.0) [2023-12-26 18:45:21,063][104569] Avg episode reward: [(0, '8822.145'), (1, '9187.347')] [2023-12-26 18:45:21,468][105620] Updated weights for policy 1, policy_version 466224 (0.0009) [2023-12-26 18:45:21,520][105620] Updated weights for policy 1, policy_version 466234 (0.0009) [2023-12-26 18:45:21,576][105620] Updated weights for policy 1, policy_version 466244 (0.0009) [2023-12-26 18:45:21,650][105692] Updated weights for policy 0, policy_version 465761 (0.0008) [2023-12-26 18:45:21,714][105692] Updated weights for policy 0, policy_version 465771 (0.0009) [2023-12-26 18:45:21,777][105692] Updated weights for policy 0, policy_version 465781 (0.0009) [2023-12-26 18:45:21,833][105692] Updated weights for policy 0, policy_version 465791 (0.0009) [2023-12-26 18:45:22,360][105620] Updated weights for policy 1, policy_version 466254 (0.0009) [2023-12-26 18:45:22,424][105620] Updated weights for policy 1, policy_version 466264 (0.0009) [2023-12-26 18:45:22,483][105620] Updated weights for policy 1, policy_version 466274 (0.0009) [2023-12-26 18:45:22,653][105692] Updated weights for policy 0, policy_version 465801 (0.0009) [2023-12-26 18:45:22,713][105692] Updated weights for policy 0, policy_version 465811 (0.0010) [2023-12-26 18:45:22,772][105692] Updated weights for policy 0, policy_version 465821 (0.0010) [2023-12-26 18:45:23,089][105620] Updated weights for policy 1, policy_version 466284 (0.0008) [2023-12-26 18:45:23,161][105620] Updated weights for policy 1, policy_version 466294 (0.0006) [2023-12-26 18:45:23,215][105620] Updated weights for policy 1, policy_version 466304 (0.0008) [2023-12-26 18:45:23,489][105692] Updated weights for policy 0, policy_version 465831 (0.0010) [2023-12-26 18:45:23,543][105692] Updated weights for policy 0, policy_version 465841 (0.0010) [2023-12-26 18:45:23,598][105692] Updated weights for policy 0, policy_version 465851 (0.0010) [2023-12-26 18:45:23,796][105620] Updated weights for policy 1, policy_version 466314 (0.0009) [2023-12-26 18:45:23,856][105620] Updated weights for policy 1, policy_version 466324 (0.0005) [2023-12-26 18:45:23,918][105620] Updated weights for policy 1, policy_version 466334 (0.0005) [2023-12-26 18:45:23,979][105620] Updated weights for policy 1, policy_version 466344 (0.0005) [2023-12-26 18:45:24,422][105692] Updated weights for policy 0, policy_version 465862 (0.0009) [2023-12-26 18:45:24,485][105692] Updated weights for policy 0, policy_version 465872 (0.0010) [2023-12-26 18:45:24,546][105692] Updated weights for policy 0, policy_version 465882 (0.0008) [2023-12-26 18:45:24,582][105620] Updated weights for policy 1, policy_version 466354 (0.0009) [2023-12-26 18:45:24,640][105620] Updated weights for policy 1, policy_version 466364 (0.0009) [2023-12-26 18:45:24,697][105620] Updated weights for policy 1, policy_version 466374 (0.0008) [2023-12-26 18:45:25,317][105692] Updated weights for policy 0, policy_version 465892 (0.0010) [2023-12-26 18:45:25,370][105692] Updated weights for policy 0, policy_version 465902 (0.0008) [2023-12-26 18:45:25,427][105692] Updated weights for policy 0, policy_version 465912 (0.0005) [2023-12-26 18:45:25,435][105620] Updated weights for policy 1, policy_version 466384 (0.0006) [2023-12-26 18:45:25,495][105620] Updated weights for policy 1, policy_version 466394 (0.0008) [2023-12-26 18:45:25,555][105620] Updated weights for policy 1, policy_version 466404 (0.0010) [2023-12-26 18:45:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 238706688. Throughput: 0: 9645.3, 1: 9919.7. Samples: 238719700. Policy #0 lag: (min: 13.0, avg: 25.8, max: 45.0) [2023-12-26 18:45:26,063][104569] Avg episode reward: [(0, '8822.878'), (1, '9011.053')] [2023-12-26 18:45:26,066][105692] Updated weights for policy 0, policy_version 465922 (0.0005) [2023-12-26 18:45:26,130][105620] Updated weights for policy 1, policy_version 466414 (0.0007) [2023-12-26 18:45:26,132][105692] Updated weights for policy 0, policy_version 465932 (0.0006) [2023-12-26 18:45:26,190][105692] Updated weights for policy 0, policy_version 465942 (0.0005) [2023-12-26 18:45:26,193][105620] Updated weights for policy 1, policy_version 466424 (0.0005) [2023-12-26 18:45:26,209][105586] KL-divergence is very high: 111.1873 [2023-12-26 18:45:26,248][105692] Updated weights for policy 0, policy_version 465952 (0.0005) [2023-12-26 18:45:26,254][105620] Updated weights for policy 1, policy_version 466434 (0.0007) [2023-12-26 18:45:26,832][105692] Updated weights for policy 0, policy_version 465962 (0.0005) [2023-12-26 18:45:26,858][105620] Updated weights for policy 1, policy_version 466444 (0.0008) [2023-12-26 18:45:26,887][105692] Updated weights for policy 0, policy_version 465972 (0.0007) [2023-12-26 18:45:26,918][105620] Updated weights for policy 1, policy_version 466454 (0.0010) [2023-12-26 18:45:26,940][105692] Updated weights for policy 0, policy_version 465982 (0.0009) [2023-12-26 18:45:26,963][105620] Updated weights for policy 1, policy_version 466464 (0.0010) [2023-12-26 18:45:27,623][105692] Updated weights for policy 0, policy_version 465992 (0.0005) [2023-12-26 18:45:27,626][105620] Updated weights for policy 1, policy_version 466474 (0.0010) [2023-12-26 18:45:27,672][105692] Updated weights for policy 0, policy_version 466002 (0.0005) [2023-12-26 18:45:27,673][105620] Updated weights for policy 1, policy_version 466484 (0.0008) [2023-12-26 18:45:27,718][105692] Updated weights for policy 0, policy_version 466012 (0.0005) [2023-12-26 18:45:27,724][105620] Updated weights for policy 1, policy_version 466494 (0.0006) [2023-12-26 18:45:27,767][105620] Updated weights for policy 1, policy_version 466504 (0.0005) [2023-12-26 18:45:28,261][105692] Updated weights for policy 0, policy_version 466022 (0.0005) [2023-12-26 18:45:28,324][105692] Updated weights for policy 0, policy_version 466032 (0.0006) [2023-12-26 18:45:28,383][105692] Updated weights for policy 0, policy_version 466042 (0.0008) [2023-12-26 18:45:28,464][105620] Updated weights for policy 1, policy_version 466514 (0.0008) [2023-12-26 18:45:28,516][105620] Updated weights for policy 1, policy_version 466524 (0.0009) [2023-12-26 18:45:28,567][105620] Updated weights for policy 1, policy_version 466534 (0.0007) [2023-12-26 18:45:28,952][105692] Updated weights for policy 0, policy_version 466052 (0.0007) [2023-12-26 18:45:29,001][105692] Updated weights for policy 0, policy_version 466062 (0.0005) [2023-12-26 18:45:29,061][105692] Updated weights for policy 0, policy_version 466072 (0.0005) [2023-12-26 18:45:29,143][105620] Updated weights for policy 1, policy_version 466544 (0.0005) [2023-12-26 18:45:29,187][105620] Updated weights for policy 1, policy_version 466554 (0.0005) [2023-12-26 18:45:29,240][105620] Updated weights for policy 1, policy_version 466564 (0.0007) [2023-12-26 18:45:29,725][105692] Updated weights for policy 0, policy_version 466082 (0.0006) [2023-12-26 18:45:29,786][105692] Updated weights for policy 0, policy_version 466092 (0.0007) [2023-12-26 18:45:29,847][105692] Updated weights for policy 0, policy_version 466102 (0.0007) [2023-12-26 18:45:29,910][105692] Updated weights for policy 0, policy_version 466112 (0.0008) [2023-12-26 18:45:29,920][105620] Updated weights for policy 1, policy_version 466574 (0.0008) [2023-12-26 18:45:29,977][105620] Updated weights for policy 1, policy_version 466584 (0.0009) [2023-12-26 18:45:30,029][105620] Updated weights for policy 1, policy_version 466594 (0.0009) [2023-12-26 18:45:30,554][105692] Updated weights for policy 0, policy_version 466122 (0.0008) [2023-12-26 18:45:30,621][105692] Updated weights for policy 0, policy_version 466132 (0.0008) [2023-12-26 18:45:30,626][105585] KL-divergence is very high: 109.0660 [2023-12-26 18:45:30,675][105585] KL-divergence is very high: 117.6004 [2023-12-26 18:45:30,680][105692] Updated weights for policy 0, policy_version 466142 (0.0006) [2023-12-26 18:45:30,851][105620] Updated weights for policy 1, policy_version 466604 (0.0009) [2023-12-26 18:45:30,906][105620] Updated weights for policy 1, policy_version 466614 (0.0009) [2023-12-26 18:45:30,958][105620] Updated weights for policy 1, policy_version 466624 (0.0009) [2023-12-26 18:45:31,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 238821376. Throughput: 0: 9726.4, 1: 10009.2. Samples: 238785864. Policy #0 lag: (min: 13.0, avg: 25.8, max: 45.0) [2023-12-26 18:45:31,063][104569] Avg episode reward: [(0, '8642.378'), (1, '9009.395')] [2023-12-26 18:45:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000466144_119349248.pth... [2023-12-26 18:45:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000466632_119472128.pth... [2023-12-26 18:45:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000465032_119062528.pth [2023-12-26 18:45:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000465448_119169024.pth [2023-12-26 18:45:31,264][105692] Updated weights for policy 0, policy_version 466152 (0.0008) [2023-12-26 18:45:31,316][105692] Updated weights for policy 0, policy_version 466162 (0.0005) [2023-12-26 18:45:31,377][105692] Updated weights for policy 0, policy_version 466172 (0.0008) [2023-12-26 18:45:31,813][105620] Updated weights for policy 1, policy_version 466634 (0.0010) [2023-12-26 18:45:31,877][105620] Updated weights for policy 1, policy_version 466644 (0.0010) [2023-12-26 18:45:31,941][105620] Updated weights for policy 1, policy_version 466654 (0.0009) [2023-12-26 18:45:31,975][105692] Updated weights for policy 0, policy_version 466182 (0.0009) [2023-12-26 18:45:32,000][105620] Updated weights for policy 1, policy_version 466664 (0.0010) [2023-12-26 18:45:32,029][105692] Updated weights for policy 0, policy_version 466192 (0.0008) [2023-12-26 18:45:32,078][105692] Updated weights for policy 0, policy_version 466202 (0.0006) [2023-12-26 18:45:32,643][105620] Updated weights for policy 1, policy_version 466674 (0.0006) [2023-12-26 18:45:32,710][105620] Updated weights for policy 1, policy_version 466684 (0.0011) [2023-12-26 18:45:32,729][105692] Updated weights for policy 0, policy_version 466212 (0.0006) [2023-12-26 18:45:32,774][105620] Updated weights for policy 1, policy_version 466694 (0.0011) [2023-12-26 18:45:32,791][105692] Updated weights for policy 0, policy_version 466222 (0.0008) [2023-12-26 18:45:32,854][105692] Updated weights for policy 0, policy_version 466232 (0.0011) [2023-12-26 18:45:33,363][105620] Updated weights for policy 1, policy_version 466704 (0.0010) [2023-12-26 18:45:33,413][105620] Updated weights for policy 1, policy_version 466714 (0.0010) [2023-12-26 18:45:33,467][105620] Updated weights for policy 1, policy_version 466724 (0.0010) [2023-12-26 18:45:33,577][105692] Updated weights for policy 0, policy_version 466242 (0.0010) [2023-12-26 18:45:33,640][105692] Updated weights for policy 0, policy_version 466252 (0.0010) [2023-12-26 18:45:33,687][105692] Updated weights for policy 0, policy_version 466262 (0.0010) [2023-12-26 18:45:33,745][105692] Updated weights for policy 0, policy_version 466272 (0.0010) [2023-12-26 18:45:34,224][105620] Updated weights for policy 1, policy_version 466734 (0.0010) [2023-12-26 18:45:34,287][105620] Updated weights for policy 1, policy_version 466744 (0.0011) [2023-12-26 18:45:34,310][105692] Updated weights for policy 0, policy_version 466282 (0.0010) [2023-12-26 18:45:34,348][105620] Updated weights for policy 1, policy_version 466754 (0.0011) [2023-12-26 18:45:34,369][105692] Updated weights for policy 0, policy_version 466292 (0.0011) [2023-12-26 18:45:34,428][105692] Updated weights for policy 0, policy_version 466302 (0.0010) [2023-12-26 18:45:35,013][105620] Updated weights for policy 1, policy_version 466764 (0.0010) [2023-12-26 18:45:35,061][105620] Updated weights for policy 1, policy_version 466774 (0.0007) [2023-12-26 18:45:35,120][105620] Updated weights for policy 1, policy_version 466784 (0.0010) [2023-12-26 18:45:35,164][105692] Updated weights for policy 0, policy_version 466312 (0.0010) [2023-12-26 18:45:35,228][105692] Updated weights for policy 0, policy_version 466322 (0.0010) [2023-12-26 18:45:35,285][105692] Updated weights for policy 0, policy_version 466332 (0.0010) [2023-12-26 18:45:35,816][105620] Updated weights for policy 1, policy_version 466794 (0.0009) [2023-12-26 18:45:35,883][105620] Updated weights for policy 1, policy_version 466804 (0.0005) [2023-12-26 18:45:35,950][105620] Updated weights for policy 1, policy_version 466814 (0.0007) [2023-12-26 18:45:35,991][105692] Updated weights for policy 0, policy_version 466342 (0.0008) [2023-12-26 18:45:36,007][105620] Updated weights for policy 1, policy_version 466824 (0.0007) [2023-12-26 18:45:36,047][105692] Updated weights for policy 0, policy_version 466352 (0.0008) [2023-12-26 18:45:36,062][104569] Fps is (10 sec: 21299.3, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 238919680. Throughput: 0: 9764.0, 1: 9967.3. Samples: 238909676. Policy #0 lag: (min: 13.0, avg: 25.8, max: 45.0) [2023-12-26 18:45:36,062][104569] Avg episode reward: [(0, '8731.285'), (1, '8923.374')] [2023-12-26 18:45:36,098][105692] Updated weights for policy 0, policy_version 466362 (0.0007) [2023-12-26 18:45:36,680][105620] Updated weights for policy 1, policy_version 466834 (0.0010) [2023-12-26 18:45:36,749][105620] Updated weights for policy 1, policy_version 466844 (0.0011) [2023-12-26 18:45:36,808][105620] Updated weights for policy 1, policy_version 466854 (0.0010) [2023-12-26 18:45:36,848][105692] Updated weights for policy 0, policy_version 466372 (0.0008) [2023-12-26 18:45:36,916][105692] Updated weights for policy 0, policy_version 466382 (0.0007) [2023-12-26 18:45:36,979][105692] Updated weights for policy 0, policy_version 466392 (0.0008) [2023-12-26 18:45:37,480][105620] Updated weights for policy 1, policy_version 466864 (0.0006) [2023-12-26 18:45:37,536][105620] Updated weights for policy 1, policy_version 466874 (0.0007) [2023-12-26 18:45:37,591][105620] Updated weights for policy 1, policy_version 466884 (0.0010) [2023-12-26 18:45:37,768][105692] Updated weights for policy 0, policy_version 466402 (0.0008) [2023-12-26 18:45:37,822][105692] Updated weights for policy 0, policy_version 466412 (0.0008) [2023-12-26 18:45:37,881][105692] Updated weights for policy 0, policy_version 466422 (0.0009) [2023-12-26 18:45:37,937][105692] Updated weights for policy 0, policy_version 466432 (0.0009) [2023-12-26 18:45:38,314][105620] Updated weights for policy 1, policy_version 466894 (0.0010) [2023-12-26 18:45:38,381][105620] Updated weights for policy 1, policy_version 466904 (0.0011) [2023-12-26 18:45:38,446][105620] Updated weights for policy 1, policy_version 466914 (0.0010) [2023-12-26 18:45:38,754][105692] Updated weights for policy 0, policy_version 466442 (0.0008) [2023-12-26 18:45:38,810][105692] Updated weights for policy 0, policy_version 466452 (0.0008) [2023-12-26 18:45:38,859][105692] Updated weights for policy 0, policy_version 466462 (0.0008) [2023-12-26 18:45:39,133][105620] Updated weights for policy 1, policy_version 466924 (0.0010) [2023-12-26 18:45:39,186][105620] Updated weights for policy 1, policy_version 466934 (0.0011) [2023-12-26 18:45:39,253][105620] Updated weights for policy 1, policy_version 466944 (0.0011) [2023-12-26 18:45:39,645][105692] Updated weights for policy 0, policy_version 466472 (0.0008) [2023-12-26 18:45:39,694][105692] Updated weights for policy 0, policy_version 466482 (0.0008) [2023-12-26 18:45:39,755][105692] Updated weights for policy 0, policy_version 466492 (0.0008) [2023-12-26 18:45:40,098][105620] Updated weights for policy 1, policy_version 466954 (0.0010) [2023-12-26 18:45:40,161][105620] Updated weights for policy 1, policy_version 466964 (0.0008) [2023-12-26 18:45:40,217][105620] Updated weights for policy 1, policy_version 466974 (0.0008) [2023-12-26 18:45:40,268][105620] Updated weights for policy 1, policy_version 466984 (0.0008) [2023-12-26 18:45:40,521][105692] Updated weights for policy 0, policy_version 466502 (0.0008) [2023-12-26 18:45:40,584][105692] Updated weights for policy 0, policy_version 466512 (0.0009) [2023-12-26 18:45:40,642][105692] Updated weights for policy 0, policy_version 466522 (0.0010) [2023-12-26 18:45:40,924][105620] Updated weights for policy 1, policy_version 466994 (0.0006) [2023-12-26 18:45:40,975][105620] Updated weights for policy 1, policy_version 467004 (0.0010) [2023-12-26 18:45:41,034][105620] Updated weights for policy 1, policy_version 467014 (0.0010) [2023-12-26 18:45:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 239017984. Throughput: 0: 9688.6, 1: 9982.8. Samples: 239023784. Policy #0 lag: (min: 13.0, avg: 25.8, max: 45.0) [2023-12-26 18:45:41,063][104569] Avg episode reward: [(0, '9090.977'), (1, '9097.551')] [2023-12-26 18:45:41,483][105692] Updated weights for policy 0, policy_version 466532 (0.0008) [2023-12-26 18:45:41,545][105692] Updated weights for policy 0, policy_version 466542 (0.0008) [2023-12-26 18:45:41,603][105692] Updated weights for policy 0, policy_version 466552 (0.0008) [2023-12-26 18:45:41,841][105620] Updated weights for policy 1, policy_version 467024 (0.0009) [2023-12-26 18:45:41,904][105620] Updated weights for policy 1, policy_version 467034 (0.0008) [2023-12-26 18:45:41,953][105620] Updated weights for policy 1, policy_version 467044 (0.0008) [2023-12-26 18:45:42,340][105692] Updated weights for policy 0, policy_version 466562 (0.0009) [2023-12-26 18:45:42,414][105692] Updated weights for policy 0, policy_version 466572 (0.0009) [2023-12-26 18:45:42,475][105692] Updated weights for policy 0, policy_version 466582 (0.0009) [2023-12-26 18:45:42,537][105692] Updated weights for policy 0, policy_version 466592 (0.0008) [2023-12-26 18:45:42,755][105620] Updated weights for policy 1, policy_version 467054 (0.0009) [2023-12-26 18:45:42,802][105620] Updated weights for policy 1, policy_version 467064 (0.0009) [2023-12-26 18:45:42,860][105620] Updated weights for policy 1, policy_version 467074 (0.0010) [2023-12-26 18:45:43,320][105692] Updated weights for policy 0, policy_version 466602 (0.0008) [2023-12-26 18:45:43,378][105692] Updated weights for policy 0, policy_version 466612 (0.0008) [2023-12-26 18:45:43,439][105692] Updated weights for policy 0, policy_version 466622 (0.0008) [2023-12-26 18:45:43,616][105620] Updated weights for policy 1, policy_version 467084 (0.0010) [2023-12-26 18:45:43,664][105620] Updated weights for policy 1, policy_version 467094 (0.0010) [2023-12-26 18:45:43,713][105620] Updated weights for policy 1, policy_version 467104 (0.0010) [2023-12-26 18:45:44,238][105692] Updated weights for policy 0, policy_version 466632 (0.0008) [2023-12-26 18:45:44,290][105692] Updated weights for policy 0, policy_version 466642 (0.0005) [2023-12-26 18:45:44,343][105692] Updated weights for policy 0, policy_version 466652 (0.0005) [2023-12-26 18:45:44,494][105620] Updated weights for policy 1, policy_version 467114 (0.0010) [2023-12-26 18:45:44,556][105620] Updated weights for policy 1, policy_version 467124 (0.0010) [2023-12-26 18:45:44,615][105620] Updated weights for policy 1, policy_version 467134 (0.0010) [2023-12-26 18:45:44,676][105620] Updated weights for policy 1, policy_version 467144 (0.0010) [2023-12-26 18:45:45,028][105692] Updated weights for policy 0, policy_version 466662 (0.0006) [2023-12-26 18:45:45,088][105692] Updated weights for policy 0, policy_version 466672 (0.0006) [2023-12-26 18:45:45,146][105692] Updated weights for policy 0, policy_version 466682 (0.0006) [2023-12-26 18:45:45,431][105620] Updated weights for policy 1, policy_version 467154 (0.0010) [2023-12-26 18:45:45,500][105620] Updated weights for policy 1, policy_version 467164 (0.0008) [2023-12-26 18:45:45,557][105620] Updated weights for policy 1, policy_version 467174 (0.0005) [2023-12-26 18:45:45,833][105692] Updated weights for policy 0, policy_version 466692 (0.0006) [2023-12-26 18:45:45,895][105692] Updated weights for policy 0, policy_version 466702 (0.0005) [2023-12-26 18:45:45,951][105692] Updated weights for policy 0, policy_version 466712 (0.0005) [2023-12-26 18:45:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 239108096. Throughput: 0: 9618.9, 1: 9927.7. Samples: 239076708. Policy #0 lag: (min: 13.0, avg: 25.8, max: 45.0) [2023-12-26 18:45:46,063][104569] Avg episode reward: [(0, '9089.525'), (1, '9087.405')] [2023-12-26 18:45:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000466720_119496704.pth... [2023-12-26 18:45:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000465568_119201792.pth [2023-12-26 18:45:46,103][105620] Updated weights for policy 1, policy_version 467184 (0.0007) [2023-12-26 18:45:46,160][105620] Updated weights for policy 1, policy_version 467194 (0.0005) [2023-12-26 18:45:46,219][105620] Updated weights for policy 1, policy_version 467204 (0.0005) [2023-12-26 18:45:46,240][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000467208_119619584.pth... [2023-12-26 18:45:46,243][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000466024_119316480.pth [2023-12-26 18:45:46,681][105692] Updated weights for policy 0, policy_version 466722 (0.0009) [2023-12-26 18:45:46,748][105692] Updated weights for policy 0, policy_version 466732 (0.0005) [2023-12-26 18:45:46,810][105692] Updated weights for policy 0, policy_version 466742 (0.0006) [2023-12-26 18:45:46,859][105620] Updated weights for policy 1, policy_version 467214 (0.0008) [2023-12-26 18:45:46,864][105692] Updated weights for policy 0, policy_version 466752 (0.0007) [2023-12-26 18:45:46,914][105620] Updated weights for policy 1, policy_version 467224 (0.0010) [2023-12-26 18:45:46,984][105620] Updated weights for policy 1, policy_version 467234 (0.0010) [2023-12-26 18:45:47,598][105620] Updated weights for policy 1, policy_version 467244 (0.0008) [2023-12-26 18:45:47,617][105692] Updated weights for policy 0, policy_version 466762 (0.0009) [2023-12-26 18:45:47,658][105620] Updated weights for policy 1, policy_version 467254 (0.0010) [2023-12-26 18:45:47,676][105692] Updated weights for policy 0, policy_version 466772 (0.0007) [2023-12-26 18:45:47,717][105620] Updated weights for policy 1, policy_version 467264 (0.0010) [2023-12-26 18:45:47,739][105692] Updated weights for policy 0, policy_version 466782 (0.0006) [2023-12-26 18:45:48,393][105620] Updated weights for policy 1, policy_version 467274 (0.0009) [2023-12-26 18:45:48,455][105620] Updated weights for policy 1, policy_version 467284 (0.0009) [2023-12-26 18:45:48,510][105620] Updated weights for policy 1, policy_version 467294 (0.0005) [2023-12-26 18:45:48,556][105692] Updated weights for policy 0, policy_version 466792 (0.0008) [2023-12-26 18:45:48,565][105620] Updated weights for policy 1, policy_version 467304 (0.0006) [2023-12-26 18:45:48,622][105692] Updated weights for policy 0, policy_version 466802 (0.0009) [2023-12-26 18:45:48,687][105692] Updated weights for policy 0, policy_version 466812 (0.0009) [2023-12-26 18:45:49,284][105620] Updated weights for policy 1, policy_version 467314 (0.0008) [2023-12-26 18:45:49,351][105620] Updated weights for policy 1, policy_version 467324 (0.0009) [2023-12-26 18:45:49,419][105620] Updated weights for policy 1, policy_version 467334 (0.0008) [2023-12-26 18:45:49,510][105692] Updated weights for policy 0, policy_version 466822 (0.0010) [2023-12-26 18:45:49,578][105692] Updated weights for policy 0, policy_version 466832 (0.0008) [2023-12-26 18:45:49,645][105692] Updated weights for policy 0, policy_version 466842 (0.0009) [2023-12-26 18:45:50,136][105620] Updated weights for policy 1, policy_version 467344 (0.0008) [2023-12-26 18:45:50,200][105620] Updated weights for policy 1, policy_version 467354 (0.0010) [2023-12-26 18:45:50,254][105620] Updated weights for policy 1, policy_version 467364 (0.0010) [2023-12-26 18:45:50,405][105692] Updated weights for policy 0, policy_version 466852 (0.0008) [2023-12-26 18:45:50,473][105692] Updated weights for policy 0, policy_version 466862 (0.0010) [2023-12-26 18:45:50,544][105692] Updated weights for policy 0, policy_version 466872 (0.0009) [2023-12-26 18:45:50,996][105620] Updated weights for policy 1, policy_version 467374 (0.0009) [2023-12-26 18:45:51,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 239198208. Throughput: 0: 9592.3, 1: 9975.6. Samples: 239193228. Policy #0 lag: (min: 13.0, avg: 25.8, max: 45.0) [2023-12-26 18:45:51,063][104569] Avg episode reward: [(0, '9002.652'), (1, '8996.438')] [2023-12-26 18:45:51,071][105620] Updated weights for policy 1, policy_version 467384 (0.0010) [2023-12-26 18:45:51,132][105620] Updated weights for policy 1, policy_version 467394 (0.0010) [2023-12-26 18:45:51,253][105692] Updated weights for policy 0, policy_version 466882 (0.0009) [2023-12-26 18:45:51,321][105692] Updated weights for policy 0, policy_version 466892 (0.0007) [2023-12-26 18:45:51,390][105692] Updated weights for policy 0, policy_version 466902 (0.0007) [2023-12-26 18:45:51,454][105692] Updated weights for policy 0, policy_version 466912 (0.0009) [2023-12-26 18:45:51,935][105620] Updated weights for policy 1, policy_version 467404 (0.0008) [2023-12-26 18:45:51,997][105620] Updated weights for policy 1, policy_version 467414 (0.0008) [2023-12-26 18:45:52,054][105620] Updated weights for policy 1, policy_version 467424 (0.0009) [2023-12-26 18:45:52,153][105692] Updated weights for policy 0, policy_version 466922 (0.0008) [2023-12-26 18:45:52,201][105692] Updated weights for policy 0, policy_version 466932 (0.0009) [2023-12-26 18:45:52,251][105692] Updated weights for policy 0, policy_version 466942 (0.0008) [2023-12-26 18:45:52,849][105620] Updated weights for policy 1, policy_version 467434 (0.0009) [2023-12-26 18:45:52,897][105620] Updated weights for policy 1, policy_version 467444 (0.0009) [2023-12-26 18:45:52,948][105620] Updated weights for policy 1, policy_version 467454 (0.0009) [2023-12-26 18:45:52,996][105620] Updated weights for policy 1, policy_version 467464 (0.0009) [2023-12-26 18:45:53,056][105692] Updated weights for policy 0, policy_version 466952 (0.0009) [2023-12-26 18:45:53,122][105692] Updated weights for policy 0, policy_version 466962 (0.0010) [2023-12-26 18:45:53,175][105692] Updated weights for policy 0, policy_version 466972 (0.0010) [2023-12-26 18:45:53,724][105620] Updated weights for policy 1, policy_version 467474 (0.0009) [2023-12-26 18:45:53,776][105620] Updated weights for policy 1, policy_version 467484 (0.0009) [2023-12-26 18:45:53,827][105620] Updated weights for policy 1, policy_version 467494 (0.0008) [2023-12-26 18:45:53,852][105692] Updated weights for policy 0, policy_version 466982 (0.0008) [2023-12-26 18:45:53,904][105692] Updated weights for policy 0, policy_version 466992 (0.0008) [2023-12-26 18:45:53,963][105692] Updated weights for policy 0, policy_version 467002 (0.0008) [2023-12-26 18:45:54,567][105620] Updated weights for policy 1, policy_version 467504 (0.0006) [2023-12-26 18:45:54,615][105620] Updated weights for policy 1, policy_version 467514 (0.0005) [2023-12-26 18:45:54,619][105692] Updated weights for policy 0, policy_version 467012 (0.0007) [2023-12-26 18:45:54,674][105620] Updated weights for policy 1, policy_version 467524 (0.0009) [2023-12-26 18:45:54,686][105692] Updated weights for policy 0, policy_version 467022 (0.0007) [2023-12-26 18:45:54,747][105692] Updated weights for policy 0, policy_version 467032 (0.0006) [2023-12-26 18:45:55,362][105692] Updated weights for policy 0, policy_version 467042 (0.0008) [2023-12-26 18:45:55,411][105692] Updated weights for policy 0, policy_version 467052 (0.0010) [2023-12-26 18:45:55,413][105620] Updated weights for policy 1, policy_version 467534 (0.0005) [2023-12-26 18:45:55,463][105692] Updated weights for policy 0, policy_version 467062 (0.0010) [2023-12-26 18:45:55,475][105620] Updated weights for policy 1, policy_version 467544 (0.0005) [2023-12-26 18:45:55,512][105692] Updated weights for policy 0, policy_version 467072 (0.0010) [2023-12-26 18:45:55,536][105620] Updated weights for policy 1, policy_version 467554 (0.0005) [2023-12-26 18:45:56,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 239296512. Throughput: 0: 9669.6, 1: 9891.5. Samples: 239309036. Policy #0 lag: (min: 13.0, avg: 25.8, max: 45.0) [2023-12-26 18:45:56,062][104569] Avg episode reward: [(0, '9004.138'), (1, '8998.751')] [2023-12-26 18:45:56,189][105620] Updated weights for policy 1, policy_version 467564 (0.0008) [2023-12-26 18:45:56,213][105692] Updated weights for policy 0, policy_version 467082 (0.0005) [2023-12-26 18:45:56,237][105620] Updated weights for policy 1, policy_version 467574 (0.0008) [2023-12-26 18:45:56,262][105692] Updated weights for policy 0, policy_version 467092 (0.0005) [2023-12-26 18:45:56,286][105620] Updated weights for policy 1, policy_version 467584 (0.0011) [2023-12-26 18:45:56,310][105692] Updated weights for policy 0, policy_version 467102 (0.0005) [2023-12-26 18:45:56,902][105692] Updated weights for policy 0, policy_version 467112 (0.0009) [2023-12-26 18:45:56,966][105692] Updated weights for policy 0, policy_version 467122 (0.0010) [2023-12-26 18:45:57,017][105692] Updated weights for policy 0, policy_version 467132 (0.0010) [2023-12-26 18:45:57,052][105620] Updated weights for policy 1, policy_version 467594 (0.0009) [2023-12-26 18:45:57,102][105620] Updated weights for policy 1, policy_version 467604 (0.0008) [2023-12-26 18:45:57,160][105620] Updated weights for policy 1, policy_version 467614 (0.0007) [2023-12-26 18:45:57,218][105620] Updated weights for policy 1, policy_version 467624 (0.0008) [2023-12-26 18:45:57,761][105692] Updated weights for policy 0, policy_version 467142 (0.0011) [2023-12-26 18:45:57,819][105692] Updated weights for policy 0, policy_version 467152 (0.0011) [2023-12-26 18:45:57,875][105692] Updated weights for policy 0, policy_version 467162 (0.0005) [2023-12-26 18:45:57,970][105620] Updated weights for policy 1, policy_version 467634 (0.0009) [2023-12-26 18:45:58,035][105620] Updated weights for policy 1, policy_version 467644 (0.0010) [2023-12-26 18:45:58,111][105620] Updated weights for policy 1, policy_version 467654 (0.0010) [2023-12-26 18:45:58,516][105585] KL-divergence is very high: 110.7565 [2023-12-26 18:45:58,528][105692] Updated weights for policy 0, policy_version 467172 (0.0006) [2023-12-26 18:45:58,600][105692] Updated weights for policy 0, policy_version 467182 (0.0007) [2023-12-26 18:45:58,664][105692] Updated weights for policy 0, policy_version 467192 (0.0010) [2023-12-26 18:45:58,948][105620] Updated weights for policy 1, policy_version 467664 (0.0008) [2023-12-26 18:45:59,014][105620] Updated weights for policy 1, policy_version 467674 (0.0010) [2023-12-26 18:45:59,075][105620] Updated weights for policy 1, policy_version 467684 (0.0011) [2023-12-26 18:45:59,429][105692] Updated weights for policy 0, policy_version 467202 (0.0007) [2023-12-26 18:45:59,490][105692] Updated weights for policy 0, policy_version 467212 (0.0006) [2023-12-26 18:45:59,546][105692] Updated weights for policy 0, policy_version 467222 (0.0009) [2023-12-26 18:45:59,612][105692] Updated weights for policy 0, policy_version 467232 (0.0008) [2023-12-26 18:45:59,851][105620] Updated weights for policy 1, policy_version 467694 (0.0009) [2023-12-26 18:45:59,907][105620] Updated weights for policy 1, policy_version 467704 (0.0010) [2023-12-26 18:45:59,971][105620] Updated weights for policy 1, policy_version 467714 (0.0009) [2023-12-26 18:46:00,243][105692] Updated weights for policy 0, policy_version 467242 (0.0009) [2023-12-26 18:46:00,298][105692] Updated weights for policy 0, policy_version 467252 (0.0009) [2023-12-26 18:46:00,358][105692] Updated weights for policy 0, policy_version 467262 (0.0009) [2023-12-26 18:46:00,704][105620] Updated weights for policy 1, policy_version 467724 (0.0008) [2023-12-26 18:46:00,749][105620] Updated weights for policy 1, policy_version 467734 (0.0005) [2023-12-26 18:46:00,795][105620] Updated weights for policy 1, policy_version 467744 (0.0005) [2023-12-26 18:46:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 239394816. Throughput: 0: 9736.5, 1: 9881.0. Samples: 239367320. Policy #0 lag: (min: 13.0, avg: 25.8, max: 45.0) [2023-12-26 18:46:01,062][104569] Avg episode reward: [(0, '8561.996'), (1, '9177.428')] [2023-12-26 18:46:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000467752_119758848.pth... [2023-12-26 18:46:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000466632_119472128.pth [2023-12-26 18:46:01,098][105692] Updated weights for policy 0, policy_version 467272 (0.0007) [2023-12-26 18:46:01,161][105692] Updated weights for policy 0, policy_version 467282 (0.0008) [2023-12-26 18:46:01,227][105692] Updated weights for policy 0, policy_version 467292 (0.0005) [2023-12-26 18:46:01,255][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000467296_119644160.pth... [2023-12-26 18:46:01,259][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000466144_119349248.pth [2023-12-26 18:46:01,404][105620] Updated weights for policy 1, policy_version 467754 (0.0005) [2023-12-26 18:46:01,462][105620] Updated weights for policy 1, policy_version 467764 (0.0007) [2023-12-26 18:46:01,512][105620] Updated weights for policy 1, policy_version 467774 (0.0007) [2023-12-26 18:46:01,576][105620] Updated weights for policy 1, policy_version 467784 (0.0006) [2023-12-26 18:46:01,922][105692] Updated weights for policy 0, policy_version 467302 (0.0008) [2023-12-26 18:46:01,967][105692] Updated weights for policy 0, policy_version 467312 (0.0009) [2023-12-26 18:46:02,015][105692] Updated weights for policy 0, policy_version 467322 (0.0009) [2023-12-26 18:46:02,251][105620] Updated weights for policy 1, policy_version 467794 (0.0009) [2023-12-26 18:46:02,319][105620] Updated weights for policy 1, policy_version 467804 (0.0008) [2023-12-26 18:46:02,379][105620] Updated weights for policy 1, policy_version 467814 (0.0007) [2023-12-26 18:46:02,836][105692] Updated weights for policy 0, policy_version 467332 (0.0009) [2023-12-26 18:46:02,894][105692] Updated weights for policy 0, policy_version 467342 (0.0009) [2023-12-26 18:46:02,952][105692] Updated weights for policy 0, policy_version 467352 (0.0008) [2023-12-26 18:46:03,024][105620] Updated weights for policy 1, policy_version 467824 (0.0010) [2023-12-26 18:46:03,076][105620] Updated weights for policy 1, policy_version 467834 (0.0010) [2023-12-26 18:46:03,133][105620] Updated weights for policy 1, policy_version 467844 (0.0010) [2023-12-26 18:46:03,637][105692] Updated weights for policy 0, policy_version 467362 (0.0008) [2023-12-26 18:46:03,692][105692] Updated weights for policy 0, policy_version 467372 (0.0008) [2023-12-26 18:46:03,747][105692] Updated weights for policy 0, policy_version 467382 (0.0009) [2023-12-26 18:46:03,769][105620] Updated weights for policy 1, policy_version 467854 (0.0008) [2023-12-26 18:46:03,799][105692] Updated weights for policy 0, policy_version 467392 (0.0007) [2023-12-26 18:46:03,820][105620] Updated weights for policy 1, policy_version 467864 (0.0010) [2023-12-26 18:46:03,887][105620] Updated weights for policy 1, policy_version 467874 (0.0011) [2023-12-26 18:46:04,505][105692] Updated weights for policy 0, policy_version 467402 (0.0009) [2023-12-26 18:46:04,567][105692] Updated weights for policy 0, policy_version 467412 (0.0009) [2023-12-26 18:46:04,620][105692] Updated weights for policy 0, policy_version 467422 (0.0009) [2023-12-26 18:46:04,637][105620] Updated weights for policy 1, policy_version 467884 (0.0008) [2023-12-26 18:46:04,694][105620] Updated weights for policy 1, policy_version 467894 (0.0005) [2023-12-26 18:46:04,751][105620] Updated weights for policy 1, policy_version 467904 (0.0005) [2023-12-26 18:46:05,324][105620] Updated weights for policy 1, policy_version 467914 (0.0007) [2023-12-26 18:46:05,382][105620] Updated weights for policy 1, policy_version 467924 (0.0005) [2023-12-26 18:46:05,429][105620] Updated weights for policy 1, policy_version 467934 (0.0007) [2023-12-26 18:46:05,476][105620] Updated weights for policy 1, policy_version 467944 (0.0007) [2023-12-26 18:46:05,487][105692] Updated weights for policy 0, policy_version 467432 (0.0008) [2023-12-26 18:46:05,537][105692] Updated weights for policy 0, policy_version 467442 (0.0008) [2023-12-26 18:46:05,589][105692] Updated weights for policy 0, policy_version 467452 (0.0009) [2023-12-26 18:46:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 239493120. Throughput: 0: 9712.3, 1: 9862.1. Samples: 239484796. Policy #0 lag: (min: 13.0, avg: 25.8, max: 45.0) [2023-12-26 18:46:06,062][104569] Avg episode reward: [(0, '8824.034'), (1, '9359.361')] [2023-12-26 18:46:06,130][105620] Updated weights for policy 1, policy_version 467954 (0.0009) [2023-12-26 18:46:06,193][105620] Updated weights for policy 1, policy_version 467964 (0.0009) [2023-12-26 18:46:06,279][105620] Updated weights for policy 1, policy_version 467974 (0.0009) [2023-12-26 18:46:06,345][105692] Updated weights for policy 0, policy_version 467462 (0.0008) [2023-12-26 18:46:06,410][105692] Updated weights for policy 0, policy_version 467472 (0.0007) [2023-12-26 18:46:06,476][105692] Updated weights for policy 0, policy_version 467482 (0.0009) [2023-12-26 18:46:06,952][105620] Updated weights for policy 1, policy_version 467984 (0.0009) [2023-12-26 18:46:07,008][105620] Updated weights for policy 1, policy_version 467994 (0.0009) [2023-12-26 18:46:07,056][105620] Updated weights for policy 1, policy_version 468004 (0.0009) [2023-12-26 18:46:07,209][105692] Updated weights for policy 0, policy_version 467492 (0.0009) [2023-12-26 18:46:07,271][105692] Updated weights for policy 0, policy_version 467502 (0.0008) [2023-12-26 18:46:07,328][105692] Updated weights for policy 0, policy_version 467512 (0.0010) [2023-12-26 18:46:07,804][105620] Updated weights for policy 1, policy_version 468014 (0.0010) [2023-12-26 18:46:07,857][105620] Updated weights for policy 1, policy_version 468024 (0.0009) [2023-12-26 18:46:07,917][105620] Updated weights for policy 1, policy_version 468034 (0.0005) [2023-12-26 18:46:08,026][105692] Updated weights for policy 0, policy_version 467523 (0.0010) [2023-12-26 18:46:08,077][105692] Updated weights for policy 0, policy_version 467533 (0.0007) [2023-12-26 18:46:08,134][105692] Updated weights for policy 0, policy_version 467543 (0.0006) [2023-12-26 18:46:08,596][105620] Updated weights for policy 1, policy_version 468044 (0.0007) [2023-12-26 18:46:08,658][105620] Updated weights for policy 1, policy_version 468054 (0.0009) [2023-12-26 18:46:08,716][105620] Updated weights for policy 1, policy_version 468064 (0.0009) [2023-12-26 18:46:08,848][105692] Updated weights for policy 0, policy_version 467553 (0.0008) [2023-12-26 18:46:08,907][105692] Updated weights for policy 0, policy_version 467563 (0.0006) [2023-12-26 18:46:08,971][105692] Updated weights for policy 0, policy_version 467573 (0.0008) [2023-12-26 18:46:09,040][105692] Updated weights for policy 0, policy_version 467583 (0.0006) [2023-12-26 18:46:09,501][105620] Updated weights for policy 1, policy_version 468074 (0.0010) [2023-12-26 18:46:09,557][105620] Updated weights for policy 1, policy_version 468084 (0.0010) [2023-12-26 18:46:09,617][105620] Updated weights for policy 1, policy_version 468094 (0.0009) [2023-12-26 18:46:09,678][105620] Updated weights for policy 1, policy_version 468104 (0.0008) [2023-12-26 18:46:09,717][105692] Updated weights for policy 0, policy_version 467593 (0.0008) [2023-12-26 18:46:09,768][105692] Updated weights for policy 0, policy_version 467603 (0.0009) [2023-12-26 18:46:09,829][105692] Updated weights for policy 0, policy_version 467613 (0.0008) [2023-12-26 18:46:10,503][105692] Updated weights for policy 0, policy_version 467623 (0.0007) [2023-12-26 18:46:10,534][105620] Updated weights for policy 1, policy_version 468114 (0.0008) [2023-12-26 18:46:10,565][105692] Updated weights for policy 0, policy_version 467633 (0.0007) [2023-12-26 18:46:10,591][105620] Updated weights for policy 1, policy_version 468124 (0.0008) [2023-12-26 18:46:10,623][105692] Updated weights for policy 0, policy_version 467643 (0.0007) [2023-12-26 18:46:10,642][105620] Updated weights for policy 1, policy_version 468134 (0.0007) [2023-12-26 18:46:11,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 239591424. Throughput: 0: 9768.7, 1: 9811.5. Samples: 239600812. Policy #0 lag: (min: 13.0, avg: 25.8, max: 45.0) [2023-12-26 18:46:11,063][104569] Avg episode reward: [(0, '9000.337'), (1, '9273.527')] [2023-12-26 18:46:11,329][105692] Updated weights for policy 0, policy_version 467653 (0.0007) [2023-12-26 18:46:11,386][105620] Updated weights for policy 1, policy_version 468144 (0.0007) [2023-12-26 18:46:11,401][105692] Updated weights for policy 0, policy_version 467663 (0.0008) [2023-12-26 18:46:11,442][105620] Updated weights for policy 1, policy_version 468154 (0.0007) [2023-12-26 18:46:11,457][105692] Updated weights for policy 0, policy_version 467673 (0.0008) [2023-12-26 18:46:11,458][105586] KL-divergence is very high: 128.8737 [2023-12-26 18:46:11,492][105620] Updated weights for policy 1, policy_version 468164 (0.0008) [2023-12-26 18:46:11,497][105586] KL-divergence is very high: 155.4872 [2023-12-26 18:46:12,128][105692] Updated weights for policy 0, policy_version 467683 (0.0009) [2023-12-26 18:46:12,177][105692] Updated weights for policy 0, policy_version 467693 (0.0010) [2023-12-26 18:46:12,184][105620] Updated weights for policy 1, policy_version 468174 (0.0008) [2023-12-26 18:46:12,222][105692] Updated weights for policy 0, policy_version 467703 (0.0010) [2023-12-26 18:46:12,241][105620] Updated weights for policy 1, policy_version 468184 (0.0005) [2023-12-26 18:46:12,307][105620] Updated weights for policy 1, policy_version 468194 (0.0009) [2023-12-26 18:46:12,998][105692] Updated weights for policy 0, policy_version 467713 (0.0011) [2023-12-26 18:46:13,059][105692] Updated weights for policy 0, policy_version 467723 (0.0008) [2023-12-26 18:46:13,082][105620] Updated weights for policy 1, policy_version 468204 (0.0009) [2023-12-26 18:46:13,125][105692] Updated weights for policy 0, policy_version 467733 (0.0005) [2023-12-26 18:46:13,139][105620] Updated weights for policy 1, policy_version 468214 (0.0009) [2023-12-26 18:46:13,187][105692] Updated weights for policy 0, policy_version 467743 (0.0006) [2023-12-26 18:46:13,188][105620] Updated weights for policy 1, policy_version 468224 (0.0009) [2023-12-26 18:46:13,768][105692] Updated weights for policy 0, policy_version 467753 (0.0006) [2023-12-26 18:46:13,822][105692] Updated weights for policy 0, policy_version 467763 (0.0010) [2023-12-26 18:46:13,880][105692] Updated weights for policy 0, policy_version 467773 (0.0010) [2023-12-26 18:46:14,017][105620] Updated weights for policy 1, policy_version 468234 (0.0009) [2023-12-26 18:46:14,065][105620] Updated weights for policy 1, policy_version 468244 (0.0008) [2023-12-26 18:46:14,114][105620] Updated weights for policy 1, policy_version 468254 (0.0008) [2023-12-26 18:46:14,162][105620] Updated weights for policy 1, policy_version 468264 (0.0007) [2023-12-26 18:46:14,602][105692] Updated weights for policy 0, policy_version 467783 (0.0010) [2023-12-26 18:46:14,654][105692] Updated weights for policy 0, policy_version 467793 (0.0010) [2023-12-26 18:46:14,670][105585] KL-divergence is very high: 219.8444 [2023-12-26 18:46:14,679][105585] KL-divergence is very high: 144.2011 [2023-12-26 18:46:14,706][105692] Updated weights for policy 0, policy_version 467803 (0.0011) [2023-12-26 18:46:14,714][105585] KL-divergence is very high: 285.5864 [2023-12-26 18:46:14,727][105585] KL-divergence is very high: 162.5564 [2023-12-26 18:46:14,996][105620] Updated weights for policy 1, policy_version 468274 (0.0009) [2023-12-26 18:46:15,040][105586] KL-divergence is very high: 111.1733 [2023-12-26 18:46:15,057][105620] Updated weights for policy 1, policy_version 468284 (0.0008) [2023-12-26 18:46:15,066][105586] KL-divergence is very high: 117.7013 [2023-12-26 18:46:15,073][105586] KL-divergence is very high: 110.0952 [2023-12-26 18:46:15,079][105586] KL-divergence is very high: 107.2685 [2023-12-26 18:46:15,085][105586] KL-divergence is very high: 113.4325 [2023-12-26 18:46:15,114][105620] Updated weights for policy 1, policy_version 468294 (0.0008) [2023-12-26 18:46:15,403][105692] Updated weights for policy 0, policy_version 467813 (0.0011) [2023-12-26 18:46:15,469][105692] Updated weights for policy 0, policy_version 467823 (0.0011) [2023-12-26 18:46:15,530][105692] Updated weights for policy 0, policy_version 467833 (0.0010) [2023-12-26 18:46:15,836][105620] Updated weights for policy 1, policy_version 468304 (0.0008) [2023-12-26 18:46:15,894][105620] Updated weights for policy 1, policy_version 468314 (0.0008) [2023-12-26 18:46:15,945][105620] Updated weights for policy 1, policy_version 468324 (0.0006) [2023-12-26 18:46:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19524.2, 300 sec: 19605.2). Total num frames: 239689728. Throughput: 0: 9687.6, 1: 9700.2. Samples: 239658316. Policy #0 lag: (min: 13.0, avg: 25.8, max: 45.0) [2023-12-26 18:46:16,063][104569] Avg episode reward: [(0, '9002.017'), (1, '6564.387')] [2023-12-26 18:46:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000467840_119783424.pth... [2023-12-26 18:46:16,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000468328_119906304.pth... [2023-12-26 18:46:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000466720_119496704.pth [2023-12-26 18:46:16,081][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000467208_119619584.pth [2023-12-26 18:46:16,238][105692] Updated weights for policy 0, policy_version 467843 (0.0010) [2023-12-26 18:46:16,295][105692] Updated weights for policy 0, policy_version 467853 (0.0010) [2023-12-26 18:46:16,347][105692] Updated weights for policy 0, policy_version 467863 (0.0010) [2023-12-26 18:46:16,527][105620] Updated weights for policy 1, policy_version 468334 (0.0008) [2023-12-26 18:46:16,591][105620] Updated weights for policy 1, policy_version 468344 (0.0010) [2023-12-26 18:46:16,655][105620] Updated weights for policy 1, policy_version 468354 (0.0010) [2023-12-26 18:46:17,075][105692] Updated weights for policy 0, policy_version 467873 (0.0010) [2023-12-26 18:46:17,152][105692] Updated weights for policy 0, policy_version 467883 (0.0008) [2023-12-26 18:46:17,201][105692] Updated weights for policy 0, policy_version 467893 (0.0005) [2023-12-26 18:46:17,258][105692] Updated weights for policy 0, policy_version 467903 (0.0005) [2023-12-26 18:46:17,299][105620] Updated weights for policy 1, policy_version 468364 (0.0010) [2023-12-26 18:46:17,353][105620] Updated weights for policy 1, policy_version 468374 (0.0010) [2023-12-26 18:46:17,411][105620] Updated weights for policy 1, policy_version 468384 (0.0010) [2023-12-26 18:46:17,932][105692] Updated weights for policy 0, policy_version 467913 (0.0010) [2023-12-26 18:46:17,979][105692] Updated weights for policy 0, policy_version 467923 (0.0010) [2023-12-26 18:46:18,031][105692] Updated weights for policy 0, policy_version 467933 (0.0010) [2023-12-26 18:46:18,151][105620] Updated weights for policy 1, policy_version 468394 (0.0010) [2023-12-26 18:46:18,198][105620] Updated weights for policy 1, policy_version 468404 (0.0010) [2023-12-26 18:46:18,264][105620] Updated weights for policy 1, policy_version 468414 (0.0009) [2023-12-26 18:46:18,316][105620] Updated weights for policy 1, policy_version 468424 (0.0005) [2023-12-26 18:46:18,754][105692] Updated weights for policy 0, policy_version 467943 (0.0010) [2023-12-26 18:46:18,809][105692] Updated weights for policy 0, policy_version 467953 (0.0010) [2023-12-26 18:46:18,867][105692] Updated weights for policy 0, policy_version 467963 (0.0010) [2023-12-26 18:46:19,074][105620] Updated weights for policy 1, policy_version 468434 (0.0010) [2023-12-26 18:46:19,129][105620] Updated weights for policy 1, policy_version 468444 (0.0010) [2023-12-26 18:46:19,194][105620] Updated weights for policy 1, policy_version 468454 (0.0010) [2023-12-26 18:46:19,602][105692] Updated weights for policy 0, policy_version 467973 (0.0009) [2023-12-26 18:46:19,670][105692] Updated weights for policy 0, policy_version 467983 (0.0006) [2023-12-26 18:46:19,726][105692] Updated weights for policy 0, policy_version 467993 (0.0007) [2023-12-26 18:46:19,943][105620] Updated weights for policy 1, policy_version 468464 (0.0010) [2023-12-26 18:46:20,006][105620] Updated weights for policy 1, policy_version 468474 (0.0009) [2023-12-26 18:46:20,072][105620] Updated weights for policy 1, policy_version 468484 (0.0011) [2023-12-26 18:46:20,455][105692] Updated weights for policy 0, policy_version 468003 (0.0008) [2023-12-26 18:46:20,516][105692] Updated weights for policy 0, policy_version 468013 (0.0008) [2023-12-26 18:46:20,577][105692] Updated weights for policy 0, policy_version 468023 (0.0008) [2023-12-26 18:46:20,827][105620] Updated weights for policy 1, policy_version 468494 (0.0011) [2023-12-26 18:46:20,879][105620] Updated weights for policy 1, policy_version 468504 (0.0011) [2023-12-26 18:46:20,932][105620] Updated weights for policy 1, policy_version 468514 (0.0010) [2023-12-26 18:46:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 239788032. Throughput: 0: 9571.4, 1: 9689.9. Samples: 239776440. Policy #0 lag: (min: 13.0, avg: 25.8, max: 45.0) [2023-12-26 18:46:21,062][104569] Avg episode reward: [(0, '9087.876'), (1, '7307.339')] [2023-12-26 18:46:21,352][105692] Updated weights for policy 0, policy_version 468033 (0.0008) [2023-12-26 18:46:21,421][105692] Updated weights for policy 0, policy_version 468043 (0.0008) [2023-12-26 18:46:21,482][105692] Updated weights for policy 0, policy_version 468053 (0.0008) [2023-12-26 18:46:21,545][105692] Updated weights for policy 0, policy_version 468063 (0.0008) [2023-12-26 18:46:21,755][105620] Updated weights for policy 1, policy_version 468524 (0.0010) [2023-12-26 18:46:21,820][105620] Updated weights for policy 1, policy_version 468534 (0.0009) [2023-12-26 18:46:21,882][105620] Updated weights for policy 1, policy_version 468544 (0.0009) [2023-12-26 18:46:22,373][105692] Updated weights for policy 0, policy_version 468073 (0.0008) [2023-12-26 18:46:22,435][105692] Updated weights for policy 0, policy_version 468083 (0.0009) [2023-12-26 18:46:22,483][105692] Updated weights for policy 0, policy_version 468093 (0.0009) [2023-12-26 18:46:22,623][105620] Updated weights for policy 1, policy_version 468554 (0.0008) [2023-12-26 18:46:22,679][105620] Updated weights for policy 1, policy_version 468564 (0.0009) [2023-12-26 18:46:22,726][105620] Updated weights for policy 1, policy_version 468574 (0.0009) [2023-12-26 18:46:22,775][105620] Updated weights for policy 1, policy_version 468584 (0.0009) [2023-12-26 18:46:23,197][105692] Updated weights for policy 0, policy_version 468103 (0.0006) [2023-12-26 18:46:23,249][105692] Updated weights for policy 0, policy_version 468113 (0.0005) [2023-12-26 18:46:23,299][105692] Updated weights for policy 0, policy_version 468123 (0.0005) [2023-12-26 18:46:23,507][105620] Updated weights for policy 1, policy_version 468594 (0.0005) [2023-12-26 18:46:23,562][105620] Updated weights for policy 1, policy_version 468604 (0.0006) [2023-12-26 18:46:23,620][105620] Updated weights for policy 1, policy_version 468614 (0.0005) [2023-12-26 18:46:23,970][105692] Updated weights for policy 0, policy_version 468133 (0.0005) [2023-12-26 18:46:24,029][105692] Updated weights for policy 0, policy_version 468143 (0.0005) [2023-12-26 18:46:24,078][105692] Updated weights for policy 0, policy_version 468153 (0.0010) [2023-12-26 18:46:24,249][105620] Updated weights for policy 1, policy_version 468624 (0.0007) [2023-12-26 18:46:24,305][105620] Updated weights for policy 1, policy_version 468634 (0.0008) [2023-12-26 18:46:24,362][105620] Updated weights for policy 1, policy_version 468644 (0.0008) [2023-12-26 18:46:24,796][105692] Updated weights for policy 0, policy_version 468163 (0.0010) [2023-12-26 18:46:24,855][105692] Updated weights for policy 0, policy_version 468173 (0.0007) [2023-12-26 18:46:24,912][105692] Updated weights for policy 0, policy_version 468183 (0.0010) [2023-12-26 18:46:25,098][105620] Updated weights for policy 1, policy_version 468654 (0.0009) [2023-12-26 18:46:25,150][105620] Updated weights for policy 1, policy_version 468665 (0.0010) [2023-12-26 18:46:25,205][105620] Updated weights for policy 1, policy_version 468675 (0.0009) [2023-12-26 18:46:25,625][105692] Updated weights for policy 0, policy_version 468193 (0.0011) [2023-12-26 18:46:25,680][105692] Updated weights for policy 0, policy_version 468203 (0.0009) [2023-12-26 18:46:25,734][105692] Updated weights for policy 0, policy_version 468213 (0.0009) [2023-12-26 18:46:25,779][105692] Updated weights for policy 0, policy_version 468223 (0.0008) [2023-12-26 18:46:25,988][105620] Updated weights for policy 1, policy_version 468685 (0.0009) [2023-12-26 18:46:26,049][105620] Updated weights for policy 1, policy_version 468695 (0.0009) [2023-12-26 18:46:26,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 239878144. Throughput: 0: 9614.1, 1: 9636.6. Samples: 239890060. Policy #0 lag: (min: 13.0, avg: 25.8, max: 45.0) [2023-12-26 18:46:26,062][104569] Avg episode reward: [(0, '8726.939'), (1, '8884.089')] [2023-12-26 18:46:26,108][105620] Updated weights for policy 1, policy_version 468705 (0.0009) [2023-12-26 18:46:26,483][105692] Updated weights for policy 0, policy_version 468233 (0.0007) [2023-12-26 18:46:26,534][105692] Updated weights for policy 0, policy_version 468243 (0.0009) [2023-12-26 18:46:26,581][105692] Updated weights for policy 0, policy_version 468254 (0.0006) [2023-12-26 18:46:26,974][105620] Updated weights for policy 1, policy_version 468715 (0.0009) [2023-12-26 18:46:27,029][105620] Updated weights for policy 1, policy_version 468725 (0.0008) [2023-12-26 18:46:27,082][105620] Updated weights for policy 1, policy_version 468735 (0.0008) [2023-12-26 18:46:27,147][105692] Updated weights for policy 0, policy_version 468264 (0.0006) [2023-12-26 18:46:27,198][105692] Updated weights for policy 0, policy_version 468274 (0.0009) [2023-12-26 18:46:27,247][105692] Updated weights for policy 0, policy_version 468284 (0.0008) [2023-12-26 18:46:27,744][105620] Updated weights for policy 1, policy_version 468745 (0.0008) [2023-12-26 18:46:27,796][105620] Updated weights for policy 1, policy_version 468755 (0.0009) [2023-12-26 18:46:27,843][105620] Updated weights for policy 1, policy_version 468765 (0.0008) [2023-12-26 18:46:27,905][105620] Updated weights for policy 1, policy_version 468775 (0.0009) [2023-12-26 18:46:27,994][105692] Updated weights for policy 0, policy_version 468295 (0.0007) [2023-12-26 18:46:28,037][105585] KL-divergence is very high: 119.4085 [2023-12-26 18:46:28,048][105692] Updated weights for policy 0, policy_version 468305 (0.0005) [2023-12-26 18:46:28,089][105585] KL-divergence is very high: 183.3504 [2023-12-26 18:46:28,114][105692] Updated weights for policy 0, policy_version 468315 (0.0005) [2023-12-26 18:46:28,143][105585] KL-divergence is very high: 178.0160 [2023-12-26 18:46:28,614][105620] Updated weights for policy 1, policy_version 468785 (0.0009) [2023-12-26 18:46:28,635][105692] Updated weights for policy 0, policy_version 468325 (0.0005) [2023-12-26 18:46:28,666][105620] Updated weights for policy 1, policy_version 468795 (0.0008) [2023-12-26 18:46:28,683][105692] Updated weights for policy 0, policy_version 468335 (0.0005) [2023-12-26 18:46:28,732][105620] Updated weights for policy 1, policy_version 468805 (0.0008) [2023-12-26 18:46:28,738][105692] Updated weights for policy 0, policy_version 468345 (0.0006) [2023-12-26 18:46:29,322][105692] Updated weights for policy 0, policy_version 468355 (0.0006) [2023-12-26 18:46:29,378][105692] Updated weights for policy 0, policy_version 468365 (0.0008) [2023-12-26 18:46:29,434][105692] Updated weights for policy 0, policy_version 468375 (0.0008) [2023-12-26 18:46:29,559][105620] Updated weights for policy 1, policy_version 468815 (0.0010) [2023-12-26 18:46:29,613][105620] Updated weights for policy 1, policy_version 468825 (0.0010) [2023-12-26 18:46:29,666][105620] Updated weights for policy 1, policy_version 468836 (0.0010) [2023-12-26 18:46:30,142][105692] Updated weights for policy 0, policy_version 468385 (0.0009) [2023-12-26 18:46:30,206][105692] Updated weights for policy 0, policy_version 468395 (0.0010) [2023-12-26 18:46:30,263][105692] Updated weights for policy 0, policy_version 468405 (0.0009) [2023-12-26 18:46:30,315][105692] Updated weights for policy 0, policy_version 468415 (0.0009) [2023-12-26 18:46:30,413][105620] Updated weights for policy 1, policy_version 468846 (0.0009) [2023-12-26 18:46:30,473][105620] Updated weights for policy 1, policy_version 468856 (0.0008) [2023-12-26 18:46:30,533][105620] Updated weights for policy 1, policy_version 468866 (0.0009) [2023-12-26 18:46:31,035][105692] Updated weights for policy 0, policy_version 468425 (0.0009) [2023-12-26 18:46:31,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.3, 300 sec: 19522.0). Total num frames: 239976448. Throughput: 0: 9763.3, 1: 9650.6. Samples: 239950332. Policy #0 lag: (min: 13.0, avg: 25.8, max: 45.0) [2023-12-26 18:46:31,062][104569] Avg episode reward: [(0, '8468.267'), (1, '9356.874')] [2023-12-26 18:46:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000468872_120045568.pth... [2023-12-26 18:46:31,069][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000467752_119758848.pth [2023-12-26 18:46:31,098][105692] Updated weights for policy 0, policy_version 468435 (0.0009) [2023-12-26 18:46:31,161][105692] Updated weights for policy 0, policy_version 468445 (0.0009) [2023-12-26 18:46:31,178][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000468448_119939072.pth... [2023-12-26 18:46:31,183][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000467296_119644160.pth [2023-12-26 18:46:31,323][105620] Updated weights for policy 1, policy_version 468876 (0.0010) [2023-12-26 18:46:31,390][105620] Updated weights for policy 1, policy_version 468886 (0.0010) [2023-12-26 18:46:31,447][105620] Updated weights for policy 1, policy_version 468896 (0.0008) [2023-12-26 18:46:31,930][105692] Updated weights for policy 0, policy_version 468455 (0.0006) [2023-12-26 18:46:31,990][105692] Updated weights for policy 0, policy_version 468465 (0.0006) [2023-12-26 18:46:32,049][105692] Updated weights for policy 0, policy_version 468475 (0.0006) [2023-12-26 18:46:32,195][105620] Updated weights for policy 1, policy_version 468906 (0.0008) [2023-12-26 18:46:32,253][105620] Updated weights for policy 1, policy_version 468916 (0.0010) [2023-12-26 18:46:32,312][105620] Updated weights for policy 1, policy_version 468926 (0.0011) [2023-12-26 18:46:32,365][105620] Updated weights for policy 1, policy_version 468936 (0.0010) [2023-12-26 18:46:32,776][105692] Updated weights for policy 0, policy_version 468485 (0.0008) [2023-12-26 18:46:32,824][105692] Updated weights for policy 0, policy_version 468495 (0.0010) [2023-12-26 18:46:32,868][105692] Updated weights for policy 0, policy_version 468505 (0.0010) [2023-12-26 18:46:33,125][105620] Updated weights for policy 1, policy_version 468946 (0.0005) [2023-12-26 18:46:33,191][105620] Updated weights for policy 1, policy_version 468956 (0.0005) [2023-12-26 18:46:33,244][105620] Updated weights for policy 1, policy_version 468966 (0.0005) [2023-12-26 18:46:33,597][105692] Updated weights for policy 0, policy_version 468515 (0.0010) [2023-12-26 18:46:33,664][105692] Updated weights for policy 0, policy_version 468525 (0.0006) [2023-12-26 18:46:33,726][105692] Updated weights for policy 0, policy_version 468535 (0.0005) [2023-12-26 18:46:33,888][105620] Updated weights for policy 1, policy_version 468976 (0.0005) [2023-12-26 18:46:33,939][105620] Updated weights for policy 1, policy_version 468986 (0.0005) [2023-12-26 18:46:33,990][105620] Updated weights for policy 1, policy_version 468996 (0.0005) [2023-12-26 18:46:34,407][105692] Updated weights for policy 0, policy_version 468545 (0.0007) [2023-12-26 18:46:34,463][105692] Updated weights for policy 0, policy_version 468555 (0.0011) [2023-12-26 18:46:34,518][105692] Updated weights for policy 0, policy_version 468565 (0.0010) [2023-12-26 18:46:34,581][105692] Updated weights for policy 0, policy_version 468575 (0.0011) [2023-12-26 18:46:34,689][105620] Updated weights for policy 1, policy_version 469006 (0.0007) [2023-12-26 18:46:34,749][105620] Updated weights for policy 1, policy_version 469016 (0.0008) [2023-12-26 18:46:34,805][105620] Updated weights for policy 1, policy_version 469026 (0.0008) [2023-12-26 18:46:35,339][105692] Updated weights for policy 0, policy_version 468585 (0.0011) [2023-12-26 18:46:35,404][105585] KL-divergence is very high: 277.8487 [2023-12-26 18:46:35,404][105692] Updated weights for policy 0, policy_version 468595 (0.0010) [2023-12-26 18:46:35,423][105585] KL-divergence is very high: 164.7604 [2023-12-26 18:46:35,455][105585] KL-divergence is very high: 326.8212 [2023-12-26 18:46:35,456][105620] Updated weights for policy 1, policy_version 469036 (0.0007) [2023-12-26 18:46:35,468][105692] Updated weights for policy 0, policy_version 468605 (0.0010) [2023-12-26 18:46:35,474][105585] KL-divergence is very high: 125.9198 [2023-12-26 18:46:35,507][105620] Updated weights for policy 1, policy_version 469046 (0.0005) [2023-12-26 18:46:35,553][105620] Updated weights for policy 1, policy_version 469056 (0.0005) [2023-12-26 18:46:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 240074752. Throughput: 0: 9837.5, 1: 9562.0. Samples: 240066204. Policy #0 lag: (min: 13.0, avg: 25.8, max: 45.0) [2023-12-26 18:46:36,062][104569] Avg episode reward: [(0, '8116.218'), (1, '9356.865')] [2023-12-26 18:46:36,079][105585] KL-divergence is very high: 217.3600 [2023-12-26 18:46:36,104][105585] KL-divergence is very high: 117.0484 [2023-12-26 18:46:36,124][105692] Updated weights for policy 0, policy_version 468615 (0.0010) [2023-12-26 18:46:36,127][105585] KL-divergence is very high: 143.1256 [2023-12-26 18:46:36,153][105585] KL-divergence is very high: 332.7375 [2023-12-26 18:46:36,166][105585] KL-divergence is very high: 235.9470 [2023-12-26 18:46:36,185][105585] KL-divergence is very high: 222.3503 [2023-12-26 18:46:36,191][105692] Updated weights for policy 0, policy_version 468625 (0.0009) [2023-12-26 18:46:36,209][105585] KL-divergence is very high: 158.7882 [2023-12-26 18:46:36,216][105585] KL-divergence is very high: 188.5620 [2023-12-26 18:46:36,219][105620] Updated weights for policy 1, policy_version 469066 (0.0006) [2023-12-26 18:46:36,233][105585] KL-divergence is very high: 419.0353 [2023-12-26 18:46:36,248][105692] Updated weights for policy 0, policy_version 468635 (0.0009) [2023-12-26 18:46:36,261][105585] KL-divergence is very high: 272.3436 [2023-12-26 18:46:36,273][105620] Updated weights for policy 1, policy_version 469076 (0.0009) [2023-12-26 18:46:36,322][105620] Updated weights for policy 1, policy_version 469086 (0.0009) [2023-12-26 18:46:36,380][105620] Updated weights for policy 1, policy_version 469096 (0.0009) [2023-12-26 18:46:36,964][105692] Updated weights for policy 0, policy_version 468645 (0.0009) [2023-12-26 18:46:37,016][105692] Updated weights for policy 0, policy_version 468655 (0.0006) [2023-12-26 18:46:37,068][105692] Updated weights for policy 0, policy_version 468665 (0.0005) [2023-12-26 18:46:37,215][105620] Updated weights for policy 1, policy_version 469106 (0.0008) [2023-12-26 18:46:37,280][105620] Updated weights for policy 1, policy_version 469116 (0.0009) [2023-12-26 18:46:37,346][105620] Updated weights for policy 1, policy_version 469126 (0.0009) [2023-12-26 18:46:37,682][105692] Updated weights for policy 0, policy_version 468675 (0.0005) [2023-12-26 18:46:37,747][105692] Updated weights for policy 0, policy_version 468685 (0.0008) [2023-12-26 18:46:37,807][105692] Updated weights for policy 0, policy_version 468695 (0.0009) [2023-12-26 18:46:38,122][105620] Updated weights for policy 1, policy_version 469136 (0.0008) [2023-12-26 18:46:38,189][105620] Updated weights for policy 1, policy_version 469146 (0.0008) [2023-12-26 18:46:38,253][105620] Updated weights for policy 1, policy_version 469156 (0.0008) [2023-12-26 18:46:38,545][105692] Updated weights for policy 0, policy_version 468705 (0.0010) [2023-12-26 18:46:38,599][105692] Updated weights for policy 0, policy_version 468715 (0.0010) [2023-12-26 18:46:38,646][105692] Updated weights for policy 0, policy_version 468725 (0.0006) [2023-12-26 18:46:38,706][105692] Updated weights for policy 0, policy_version 468735 (0.0010) [2023-12-26 18:46:38,900][105620] Updated weights for policy 1, policy_version 469166 (0.0006) [2023-12-26 18:46:38,959][105620] Updated weights for policy 1, policy_version 469176 (0.0005) [2023-12-26 18:46:39,010][105620] Updated weights for policy 1, policy_version 469186 (0.0005) [2023-12-26 18:46:39,558][105692] Updated weights for policy 0, policy_version 468745 (0.0009) [2023-12-26 18:46:39,618][105692] Updated weights for policy 0, policy_version 468755 (0.0009) [2023-12-26 18:46:39,677][105692] Updated weights for policy 0, policy_version 468765 (0.0008) [2023-12-26 18:46:39,720][105620] Updated weights for policy 1, policy_version 469196 (0.0007) [2023-12-26 18:46:39,785][105620] Updated weights for policy 1, policy_version 469206 (0.0010) [2023-12-26 18:46:39,863][105620] Updated weights for policy 1, policy_version 469216 (0.0008) [2023-12-26 18:46:40,488][105692] Updated weights for policy 0, policy_version 468775 (0.0010) [2023-12-26 18:46:40,555][105692] Updated weights for policy 0, policy_version 468785 (0.0011) [2023-12-26 18:46:40,621][105692] Updated weights for policy 0, policy_version 468795 (0.0011) [2023-12-26 18:46:40,628][105620] Updated weights for policy 1, policy_version 469226 (0.0008) [2023-12-26 18:46:40,686][105620] Updated weights for policy 1, policy_version 469236 (0.0008) [2023-12-26 18:46:40,746][105620] Updated weights for policy 1, policy_version 469246 (0.0009) [2023-12-26 18:46:40,806][105620] Updated weights for policy 1, policy_version 469256 (0.0009) [2023-12-26 18:46:41,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 240173056. Throughput: 0: 9807.2, 1: 9567.3. Samples: 240180888. Policy #0 lag: (min: 13.0, avg: 25.8, max: 45.0) [2023-12-26 18:46:41,063][104569] Avg episode reward: [(0, '7941.951'), (1, '9264.535')] [2023-12-26 18:46:41,314][105692] Updated weights for policy 0, policy_version 468805 (0.0009) [2023-12-26 18:46:41,382][105692] Updated weights for policy 0, policy_version 468815 (0.0009) [2023-12-26 18:46:41,452][105692] Updated weights for policy 0, policy_version 468825 (0.0007) [2023-12-26 18:46:41,572][105620] Updated weights for policy 1, policy_version 469266 (0.0006) [2023-12-26 18:46:41,653][105620] Updated weights for policy 1, policy_version 469276 (0.0007) [2023-12-26 18:46:41,722][105620] Updated weights for policy 1, policy_version 469286 (0.0006) [2023-12-26 18:46:42,179][105692] Updated weights for policy 0, policy_version 468835 (0.0006) [2023-12-26 18:46:42,243][105692] Updated weights for policy 0, policy_version 468845 (0.0008) [2023-12-26 18:46:42,311][105692] Updated weights for policy 0, policy_version 468855 (0.0007) [2023-12-26 18:46:42,337][105620] Updated weights for policy 1, policy_version 469296 (0.0007) [2023-12-26 18:46:42,404][105620] Updated weights for policy 1, policy_version 469306 (0.0008) [2023-12-26 18:46:42,472][105620] Updated weights for policy 1, policy_version 469316 (0.0009) [2023-12-26 18:46:42,937][105692] Updated weights for policy 0, policy_version 468865 (0.0008) [2023-12-26 18:46:42,993][105692] Updated weights for policy 0, policy_version 468875 (0.0005) [2023-12-26 18:46:43,041][105692] Updated weights for policy 0, policy_version 468885 (0.0005) [2023-12-26 18:46:43,096][105692] Updated weights for policy 0, policy_version 468895 (0.0005) [2023-12-26 18:46:43,242][105620] Updated weights for policy 1, policy_version 469326 (0.0009) [2023-12-26 18:46:43,305][105620] Updated weights for policy 1, policy_version 469336 (0.0010) [2023-12-26 18:46:43,363][105620] Updated weights for policy 1, policy_version 469346 (0.0010) [2023-12-26 18:46:43,687][105692] Updated weights for policy 0, policy_version 468905 (0.0009) [2023-12-26 18:46:43,743][105692] Updated weights for policy 0, policy_version 468915 (0.0010) [2023-12-26 18:46:43,797][105692] Updated weights for policy 0, policy_version 468925 (0.0010) [2023-12-26 18:46:44,150][105620] Updated weights for policy 1, policy_version 469356 (0.0009) [2023-12-26 18:46:44,205][105620] Updated weights for policy 1, policy_version 469366 (0.0009) [2023-12-26 18:46:44,258][105620] Updated weights for policy 1, policy_version 469376 (0.0005) [2023-12-26 18:46:44,492][105692] Updated weights for policy 0, policy_version 468935 (0.0010) [2023-12-26 18:46:44,557][105692] Updated weights for policy 0, policy_version 468945 (0.0010) [2023-12-26 18:46:44,625][105692] Updated weights for policy 0, policy_version 468955 (0.0010) [2023-12-26 18:46:44,922][105620] Updated weights for policy 1, policy_version 469386 (0.0007) [2023-12-26 18:46:44,988][105620] Updated weights for policy 1, policy_version 469396 (0.0008) [2023-12-26 18:46:45,056][105620] Updated weights for policy 1, policy_version 469406 (0.0006) [2023-12-26 18:46:45,121][105620] Updated weights for policy 1, policy_version 469416 (0.0006) [2023-12-26 18:46:45,361][105692] Updated weights for policy 0, policy_version 468965 (0.0008) [2023-12-26 18:46:45,415][105692] Updated weights for policy 0, policy_version 468975 (0.0006) [2023-12-26 18:46:45,467][105692] Updated weights for policy 0, policy_version 468985 (0.0007) [2023-12-26 18:46:45,766][105620] Updated weights for policy 1, policy_version 469426 (0.0008) [2023-12-26 18:46:45,820][105620] Updated weights for policy 1, policy_version 469436 (0.0010) [2023-12-26 18:46:45,872][105620] Updated weights for policy 1, policy_version 469446 (0.0009) [2023-12-26 18:46:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 240271360. Throughput: 0: 9800.4, 1: 9607.1. Samples: 240240656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:46:46,062][104569] Avg episode reward: [(0, '8562.368'), (1, '9264.612')] [2023-12-26 18:46:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000469448_120193024.pth... [2023-12-26 18:46:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000468992_120078336.pth... [2023-12-26 18:46:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000468328_119906304.pth [2023-12-26 18:46:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000467840_119783424.pth [2023-12-26 18:46:46,113][105692] Updated weights for policy 0, policy_version 468995 (0.0007) [2023-12-26 18:46:46,171][105692] Updated weights for policy 0, policy_version 469005 (0.0010) [2023-12-26 18:46:46,233][105692] Updated weights for policy 0, policy_version 469015 (0.0010) [2023-12-26 18:46:46,671][105620] Updated weights for policy 1, policy_version 469456 (0.0006) [2023-12-26 18:46:46,729][105620] Updated weights for policy 1, policy_version 469466 (0.0006) [2023-12-26 18:46:46,784][105620] Updated weights for policy 1, policy_version 469476 (0.0008) [2023-12-26 18:46:46,966][105692] Updated weights for policy 0, policy_version 469025 (0.0010) [2023-12-26 18:46:47,031][105692] Updated weights for policy 0, policy_version 469035 (0.0010) [2023-12-26 18:46:47,099][105692] Updated weights for policy 0, policy_version 469045 (0.0010) [2023-12-26 18:46:47,150][105692] Updated weights for policy 0, policy_version 469055 (0.0010) [2023-12-26 18:46:47,355][105620] Updated weights for policy 1, policy_version 469486 (0.0008) [2023-12-26 18:46:47,400][105620] Updated weights for policy 1, policy_version 469496 (0.0008) [2023-12-26 18:46:47,448][105620] Updated weights for policy 1, policy_version 469506 (0.0008) [2023-12-26 18:46:47,842][105692] Updated weights for policy 0, policy_version 469065 (0.0006) [2023-12-26 18:46:47,898][105692] Updated weights for policy 0, policy_version 469075 (0.0010) [2023-12-26 18:46:47,942][105692] Updated weights for policy 0, policy_version 469085 (0.0010) [2023-12-26 18:46:48,180][105620] Updated weights for policy 1, policy_version 469516 (0.0008) [2023-12-26 18:46:48,239][105620] Updated weights for policy 1, policy_version 469526 (0.0010) [2023-12-26 18:46:48,297][105620] Updated weights for policy 1, policy_version 469536 (0.0010) [2023-12-26 18:46:48,608][105692] Updated weights for policy 0, policy_version 469095 (0.0009) [2023-12-26 18:46:48,660][105692] Updated weights for policy 0, policy_version 469105 (0.0010) [2023-12-26 18:46:48,716][105692] Updated weights for policy 0, policy_version 469115 (0.0010) [2023-12-26 18:46:48,980][105620] Updated weights for policy 1, policy_version 469546 (0.0009) [2023-12-26 18:46:49,042][105620] Updated weights for policy 1, policy_version 469556 (0.0010) [2023-12-26 18:46:49,093][105620] Updated weights for policy 1, policy_version 469566 (0.0010) [2023-12-26 18:46:49,137][105620] Updated weights for policy 1, policy_version 469576 (0.0010) [2023-12-26 18:46:49,404][105692] Updated weights for policy 0, policy_version 469125 (0.0009) [2023-12-26 18:46:49,460][105692] Updated weights for policy 0, policy_version 469135 (0.0011) [2023-12-26 18:46:49,512][105692] Updated weights for policy 0, policy_version 469145 (0.0010) [2023-12-26 18:46:49,951][105620] Updated weights for policy 1, policy_version 469586 (0.0007) [2023-12-26 18:46:50,000][105620] Updated weights for policy 1, policy_version 469596 (0.0008) [2023-12-26 18:46:50,058][105620] Updated weights for policy 1, policy_version 469606 (0.0008) [2023-12-26 18:46:50,272][105692] Updated weights for policy 0, policy_version 469155 (0.0010) [2023-12-26 18:46:50,334][105692] Updated weights for policy 0, policy_version 469165 (0.0010) [2023-12-26 18:46:50,394][105692] Updated weights for policy 0, policy_version 469175 (0.0011) [2023-12-26 18:46:50,870][105620] Updated weights for policy 1, policy_version 469616 (0.0009) [2023-12-26 18:46:50,932][105620] Updated weights for policy 1, policy_version 469626 (0.0008) [2023-12-26 18:46:50,997][105620] Updated weights for policy 1, policy_version 469636 (0.0008) [2023-12-26 18:46:51,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 240369664. Throughput: 0: 9856.7, 1: 9584.7. Samples: 240359660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:46:51,062][104569] Avg episode reward: [(0, '8220.298'), (1, '9265.006')] [2023-12-26 18:46:51,143][105692] Updated weights for policy 0, policy_version 469185 (0.0010) [2023-12-26 18:46:51,197][105692] Updated weights for policy 0, policy_version 469195 (0.0009) [2023-12-26 18:46:51,248][105692] Updated weights for policy 0, policy_version 469205 (0.0009) [2023-12-26 18:46:51,312][105692] Updated weights for policy 0, policy_version 469215 (0.0010) [2023-12-26 18:46:51,795][105620] Updated weights for policy 1, policy_version 469646 (0.0008) [2023-12-26 18:46:51,854][105620] Updated weights for policy 1, policy_version 469656 (0.0008) [2023-12-26 18:46:51,918][105620] Updated weights for policy 1, policy_version 469666 (0.0008) [2023-12-26 18:46:52,020][105692] Updated weights for policy 0, policy_version 469225 (0.0010) [2023-12-26 18:46:52,089][105692] Updated weights for policy 0, policy_version 469235 (0.0011) [2023-12-26 18:46:52,151][105692] Updated weights for policy 0, policy_version 469245 (0.0010) [2023-12-26 18:46:52,664][105620] Updated weights for policy 1, policy_version 469676 (0.0009) [2023-12-26 18:46:52,728][105620] Updated weights for policy 1, policy_version 469686 (0.0008) [2023-12-26 18:46:52,795][105620] Updated weights for policy 1, policy_version 469696 (0.0008) [2023-12-26 18:46:52,868][105692] Updated weights for policy 0, policy_version 469255 (0.0008) [2023-12-26 18:46:52,916][105692] Updated weights for policy 0, policy_version 469265 (0.0008) [2023-12-26 18:46:52,966][105692] Updated weights for policy 0, policy_version 469275 (0.0009) [2023-12-26 18:46:53,522][105620] Updated weights for policy 1, policy_version 469706 (0.0009) [2023-12-26 18:46:53,578][105620] Updated weights for policy 1, policy_version 469716 (0.0009) [2023-12-26 18:46:53,642][105620] Updated weights for policy 1, policy_version 469726 (0.0009) [2023-12-26 18:46:53,706][105620] Updated weights for policy 1, policy_version 469736 (0.0009) [2023-12-26 18:46:53,712][105692] Updated weights for policy 0, policy_version 469285 (0.0008) [2023-12-26 18:46:53,773][105692] Updated weights for policy 0, policy_version 469295 (0.0006) [2023-12-26 18:46:53,835][105692] Updated weights for policy 0, policy_version 469305 (0.0009) [2023-12-26 18:46:54,465][105620] Updated weights for policy 1, policy_version 469746 (0.0008) [2023-12-26 18:46:54,525][105620] Updated weights for policy 1, policy_version 469756 (0.0008) [2023-12-26 18:46:54,554][105692] Updated weights for policy 0, policy_version 469315 (0.0008) [2023-12-26 18:46:54,581][105620] Updated weights for policy 1, policy_version 469766 (0.0008) [2023-12-26 18:46:54,613][105692] Updated weights for policy 0, policy_version 469325 (0.0007) [2023-12-26 18:46:54,664][105692] Updated weights for policy 0, policy_version 469335 (0.0010) [2023-12-26 18:46:55,352][105620] Updated weights for policy 1, policy_version 469776 (0.0008) [2023-12-26 18:46:55,388][105692] Updated weights for policy 0, policy_version 469345 (0.0009) [2023-12-26 18:46:55,411][105620] Updated weights for policy 1, policy_version 469786 (0.0008) [2023-12-26 18:46:55,449][105692] Updated weights for policy 0, policy_version 469355 (0.0008) [2023-12-26 18:46:55,475][105620] Updated weights for policy 1, policy_version 469796 (0.0007) [2023-12-26 18:46:55,507][105692] Updated weights for policy 0, policy_version 469365 (0.0007) [2023-12-26 18:46:55,569][105692] Updated weights for policy 0, policy_version 469375 (0.0009) [2023-12-26 18:46:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 240459776. Throughput: 0: 9839.6, 1: 9496.6. Samples: 240470936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:46:56,062][104569] Avg episode reward: [(0, '8393.438'), (1, '9080.610')] [2023-12-26 18:46:56,238][105620] Updated weights for policy 1, policy_version 469806 (0.0008) [2023-12-26 18:46:56,288][105620] Updated weights for policy 1, policy_version 469816 (0.0008) [2023-12-26 18:46:56,308][105692] Updated weights for policy 0, policy_version 469385 (0.0009) [2023-12-26 18:46:56,335][105620] Updated weights for policy 1, policy_version 469826 (0.0008) [2023-12-26 18:46:56,361][105692] Updated weights for policy 0, policy_version 469395 (0.0007) [2023-12-26 18:46:56,411][105692] Updated weights for policy 0, policy_version 469405 (0.0008) [2023-12-26 18:46:57,057][105620] Updated weights for policy 1, policy_version 469836 (0.0008) [2023-12-26 18:46:57,109][105620] Updated weights for policy 1, policy_version 469846 (0.0005) [2023-12-26 18:46:57,162][105620] Updated weights for policy 1, policy_version 469856 (0.0005) [2023-12-26 18:46:57,200][105692] Updated weights for policy 0, policy_version 469415 (0.0008) [2023-12-26 18:46:57,249][105692] Updated weights for policy 0, policy_version 469425 (0.0008) [2023-12-26 18:46:57,315][105692] Updated weights for policy 0, policy_version 469435 (0.0007) [2023-12-26 18:46:57,939][105620] Updated weights for policy 1, policy_version 469866 (0.0007) [2023-12-26 18:46:57,963][105692] Updated weights for policy 0, policy_version 469445 (0.0006) [2023-12-26 18:46:57,996][105620] Updated weights for policy 1, policy_version 469876 (0.0009) [2023-12-26 18:46:58,018][105692] Updated weights for policy 0, policy_version 469455 (0.0007) [2023-12-26 18:46:58,048][105620] Updated weights for policy 1, policy_version 469886 (0.0006) [2023-12-26 18:46:58,079][105692] Updated weights for policy 0, policy_version 469465 (0.0008) [2023-12-26 18:46:58,112][105620] Updated weights for policy 1, policy_version 469896 (0.0009) [2023-12-26 18:46:58,886][105692] Updated weights for policy 0, policy_version 469475 (0.0007) [2023-12-26 18:46:58,950][105692] Updated weights for policy 0, policy_version 469485 (0.0007) [2023-12-26 18:46:58,997][105620] Updated weights for policy 1, policy_version 469906 (0.0007) [2023-12-26 18:46:59,011][105692] Updated weights for policy 0, policy_version 469495 (0.0007) [2023-12-26 18:46:59,055][105620] Updated weights for policy 1, policy_version 469916 (0.0009) [2023-12-26 18:46:59,120][105620] Updated weights for policy 1, policy_version 469926 (0.0009) [2023-12-26 18:46:59,803][105692] Updated weights for policy 0, policy_version 469505 (0.0007) [2023-12-26 18:46:59,809][105620] Updated weights for policy 1, policy_version 469936 (0.0007) [2023-12-26 18:46:59,862][105692] Updated weights for policy 0, policy_version 469515 (0.0007) [2023-12-26 18:46:59,871][105620] Updated weights for policy 1, policy_version 469946 (0.0008) [2023-12-26 18:46:59,926][105692] Updated weights for policy 0, policy_version 469525 (0.0007) [2023-12-26 18:46:59,933][105620] Updated weights for policy 1, policy_version 469956 (0.0009) [2023-12-26 18:46:59,978][105692] Updated weights for policy 0, policy_version 469535 (0.0009) [2023-12-26 18:47:00,647][105620] Updated weights for policy 1, policy_version 469966 (0.0008) [2023-12-26 18:47:00,712][105620] Updated weights for policy 1, policy_version 469976 (0.0006) [2023-12-26 18:47:00,739][105692] Updated weights for policy 0, policy_version 469545 (0.0008) [2023-12-26 18:47:00,765][105620] Updated weights for policy 1, policy_version 469986 (0.0007) [2023-12-26 18:47:00,802][105692] Updated weights for policy 0, policy_version 469555 (0.0008) [2023-12-26 18:47:00,839][105585] KL-divergence is very high: 123.2085 [2023-12-26 18:47:00,866][105692] Updated weights for policy 0, policy_version 469565 (0.0009) [2023-12-26 18:47:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 240558080. Throughput: 0: 9821.3, 1: 9484.7. Samples: 240527080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:47:01,062][104569] Avg episode reward: [(0, '8197.834'), (1, '8988.894')] [2023-12-26 18:47:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000469568_120225792.pth... [2023-12-26 18:47:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000469992_120332288.pth... [2023-12-26 18:47:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000468448_119939072.pth [2023-12-26 18:47:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000468872_120045568.pth [2023-12-26 18:47:01,506][105620] Updated weights for policy 1, policy_version 469996 (0.0007) [2023-12-26 18:47:01,557][105620] Updated weights for policy 1, policy_version 470006 (0.0008) [2023-12-26 18:47:01,614][105692] Updated weights for policy 0, policy_version 469575 (0.0008) [2023-12-26 18:47:01,620][105620] Updated weights for policy 1, policy_version 470016 (0.0008) [2023-12-26 18:47:01,671][105692] Updated weights for policy 0, policy_version 469585 (0.0008) [2023-12-26 18:47:01,728][105692] Updated weights for policy 0, policy_version 469595 (0.0008) [2023-12-26 18:47:02,401][105620] Updated weights for policy 1, policy_version 470026 (0.0007) [2023-12-26 18:47:02,465][105620] Updated weights for policy 1, policy_version 470036 (0.0009) [2023-12-26 18:47:02,501][105692] Updated weights for policy 0, policy_version 469605 (0.0007) [2023-12-26 18:47:02,520][105620] Updated weights for policy 1, policy_version 470046 (0.0009) [2023-12-26 18:47:02,558][105692] Updated weights for policy 0, policy_version 469615 (0.0008) [2023-12-26 18:47:02,577][105620] Updated weights for policy 1, policy_version 470056 (0.0007) [2023-12-26 18:47:02,607][105692] Updated weights for policy 0, policy_version 469625 (0.0007) [2023-12-26 18:47:03,321][105620] Updated weights for policy 1, policy_version 470066 (0.0008) [2023-12-26 18:47:03,366][105620] Updated weights for policy 1, policy_version 470076 (0.0009) [2023-12-26 18:47:03,374][105692] Updated weights for policy 0, policy_version 469635 (0.0008) [2023-12-26 18:47:03,421][105620] Updated weights for policy 1, policy_version 470086 (0.0007) [2023-12-26 18:47:03,423][105692] Updated weights for policy 0, policy_version 469645 (0.0006) [2023-12-26 18:47:03,469][105692] Updated weights for policy 0, policy_version 469655 (0.0009) [2023-12-26 18:47:04,167][105692] Updated weights for policy 0, policy_version 469665 (0.0006) [2023-12-26 18:47:04,216][105620] Updated weights for policy 1, policy_version 470096 (0.0007) [2023-12-26 18:47:04,222][105692] Updated weights for policy 0, policy_version 469675 (0.0008) [2023-12-26 18:47:04,277][105620] Updated weights for policy 1, policy_version 470106 (0.0006) [2023-12-26 18:47:04,283][105692] Updated weights for policy 0, policy_version 469685 (0.0007) [2023-12-26 18:47:04,335][105620] Updated weights for policy 1, policy_version 470116 (0.0006) [2023-12-26 18:47:04,348][105692] Updated weights for policy 0, policy_version 469695 (0.0008) [2023-12-26 18:47:04,940][105620] Updated weights for policy 1, policy_version 470126 (0.0007) [2023-12-26 18:47:05,000][105620] Updated weights for policy 1, policy_version 470136 (0.0005) [2023-12-26 18:47:05,030][105586] KL-divergence is very high: 131.2506 [2023-12-26 18:47:05,071][105620] Updated weights for policy 1, policy_version 470146 (0.0005) [2023-12-26 18:47:05,087][105586] KL-divergence is very high: 138.0064 [2023-12-26 18:47:05,099][105692] Updated weights for policy 0, policy_version 469705 (0.0006) [2023-12-26 18:47:05,169][105692] Updated weights for policy 0, policy_version 469715 (0.0009) [2023-12-26 18:47:05,236][105692] Updated weights for policy 0, policy_version 469725 (0.0010) [2023-12-26 18:47:05,585][105620] Updated weights for policy 1, policy_version 470156 (0.0006) [2023-12-26 18:47:05,642][105620] Updated weights for policy 1, policy_version 470166 (0.0006) [2023-12-26 18:47:05,690][105620] Updated weights for policy 1, policy_version 470176 (0.0005) [2023-12-26 18:47:05,797][105692] Updated weights for policy 0, policy_version 469735 (0.0008) [2023-12-26 18:47:05,843][105692] Updated weights for policy 0, policy_version 469745 (0.0009) [2023-12-26 18:47:05,895][105692] Updated weights for policy 0, policy_version 469755 (0.0005) [2023-12-26 18:47:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 240656384. Throughput: 0: 9728.0, 1: 9455.7. Samples: 240639704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:47:06,063][104569] Avg episode reward: [(0, '6331.746'), (1, '8988.790')] [2023-12-26 18:47:06,375][105620] Updated weights for policy 1, policy_version 470186 (0.0006) [2023-12-26 18:47:06,439][105620] Updated weights for policy 1, policy_version 470196 (0.0008) [2023-12-26 18:47:06,502][105692] Updated weights for policy 0, policy_version 469765 (0.0008) [2023-12-26 18:47:06,505][105620] Updated weights for policy 1, policy_version 470206 (0.0008) [2023-12-26 18:47:06,559][105620] Updated weights for policy 1, policy_version 470216 (0.0008) [2023-12-26 18:47:06,563][105692] Updated weights for policy 0, policy_version 469775 (0.0011) [2023-12-26 18:47:06,629][105692] Updated weights for policy 0, policy_version 469785 (0.0010) [2023-12-26 18:47:07,263][105692] Updated weights for policy 0, policy_version 469795 (0.0010) [2023-12-26 18:47:07,322][105692] Updated weights for policy 0, policy_version 469805 (0.0009) [2023-12-26 18:47:07,359][105620] Updated weights for policy 1, policy_version 470226 (0.0009) [2023-12-26 18:47:07,381][105692] Updated weights for policy 0, policy_version 469815 (0.0010) [2023-12-26 18:47:07,409][105620] Updated weights for policy 1, policy_version 470236 (0.0008) [2023-12-26 18:47:07,460][105620] Updated weights for policy 1, policy_version 470246 (0.0007) [2023-12-26 18:47:08,087][105692] Updated weights for policy 0, policy_version 469825 (0.0010) [2023-12-26 18:47:08,141][105692] Updated weights for policy 0, policy_version 469835 (0.0008) [2023-12-26 18:47:08,194][105692] Updated weights for policy 0, policy_version 469845 (0.0005) [2023-12-26 18:47:08,236][105620] Updated weights for policy 1, policy_version 470256 (0.0006) [2023-12-26 18:47:08,245][105692] Updated weights for policy 0, policy_version 469855 (0.0005) [2023-12-26 18:47:08,295][105620] Updated weights for policy 1, policy_version 470266 (0.0005) [2023-12-26 18:47:08,358][105620] Updated weights for policy 1, policy_version 470276 (0.0007) [2023-12-26 18:47:08,841][105692] Updated weights for policy 0, policy_version 469865 (0.0005) [2023-12-26 18:47:08,899][105692] Updated weights for policy 0, policy_version 469875 (0.0007) [2023-12-26 18:47:08,956][105692] Updated weights for policy 0, policy_version 469885 (0.0010) [2023-12-26 18:47:09,003][105620] Updated weights for policy 1, policy_version 470286 (0.0008) [2023-12-26 18:47:09,068][105620] Updated weights for policy 1, policy_version 470296 (0.0008) [2023-12-26 18:47:09,127][105620] Updated weights for policy 1, policy_version 470306 (0.0008) [2023-12-26 18:47:09,700][105692] Updated weights for policy 0, policy_version 469895 (0.0009) [2023-12-26 18:47:09,762][105692] Updated weights for policy 0, policy_version 469905 (0.0009) [2023-12-26 18:47:09,824][105692] Updated weights for policy 0, policy_version 469915 (0.0009) [2023-12-26 18:47:09,857][105620] Updated weights for policy 1, policy_version 470316 (0.0007) [2023-12-26 18:47:09,924][105620] Updated weights for policy 1, policy_version 470326 (0.0007) [2023-12-26 18:47:09,990][105620] Updated weights for policy 1, policy_version 470336 (0.0006) [2023-12-26 18:47:10,604][105620] Updated weights for policy 1, policy_version 470346 (0.0007) [2023-12-26 18:47:10,611][105692] Updated weights for policy 0, policy_version 469925 (0.0008) [2023-12-26 18:47:10,666][105620] Updated weights for policy 1, policy_version 470356 (0.0007) [2023-12-26 18:47:10,679][105692] Updated weights for policy 0, policy_version 469935 (0.0009) [2023-12-26 18:47:10,728][105620] Updated weights for policy 1, policy_version 470366 (0.0009) [2023-12-26 18:47:10,736][105692] Updated weights for policy 0, policy_version 469945 (0.0010) [2023-12-26 18:47:10,792][105620] Updated weights for policy 1, policy_version 470376 (0.0008) [2023-12-26 18:47:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 240754688. Throughput: 0: 9843.8, 1: 9523.3. Samples: 240761580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:47:11,063][104569] Avg episode reward: [(0, '7342.860'), (1, '9264.615')] [2023-12-26 18:47:11,443][105692] Updated weights for policy 0, policy_version 469955 (0.0011) [2023-12-26 18:47:11,509][105692] Updated weights for policy 0, policy_version 469965 (0.0011) [2023-12-26 18:47:11,576][105692] Updated weights for policy 0, policy_version 469975 (0.0011) [2023-12-26 18:47:11,627][105620] Updated weights for policy 1, policy_version 470386 (0.0007) [2023-12-26 18:47:11,693][105620] Updated weights for policy 1, policy_version 470396 (0.0009) [2023-12-26 18:47:11,763][105620] Updated weights for policy 1, policy_version 470406 (0.0010) [2023-12-26 18:47:12,329][105692] Updated weights for policy 0, policy_version 469985 (0.0010) [2023-12-26 18:47:12,392][105692] Updated weights for policy 0, policy_version 469995 (0.0009) [2023-12-26 18:47:12,456][105692] Updated weights for policy 0, policy_version 470005 (0.0010) [2023-12-26 18:47:12,520][105692] Updated weights for policy 0, policy_version 470015 (0.0009) [2023-12-26 18:47:12,552][105620] Updated weights for policy 1, policy_version 470416 (0.0008) [2023-12-26 18:47:12,605][105620] Updated weights for policy 1, policy_version 470426 (0.0008) [2023-12-26 18:47:12,660][105620] Updated weights for policy 1, policy_version 470436 (0.0009) [2023-12-26 18:47:13,223][105692] Updated weights for policy 0, policy_version 470025 (0.0010) [2023-12-26 18:47:13,274][105692] Updated weights for policy 0, policy_version 470035 (0.0010) [2023-12-26 18:47:13,325][105692] Updated weights for policy 0, policy_version 470045 (0.0010) [2023-12-26 18:47:13,441][105620] Updated weights for policy 1, policy_version 470446 (0.0009) [2023-12-26 18:47:13,500][105620] Updated weights for policy 1, policy_version 470456 (0.0008) [2023-12-26 18:47:13,548][105620] Updated weights for policy 1, policy_version 470466 (0.0008) [2023-12-26 18:47:14,054][105692] Updated weights for policy 0, policy_version 470055 (0.0007) [2023-12-26 18:47:14,105][105692] Updated weights for policy 0, policy_version 470065 (0.0005) [2023-12-26 18:47:14,151][105692] Updated weights for policy 0, policy_version 470075 (0.0005) [2023-12-26 18:47:14,231][105620] Updated weights for policy 1, policy_version 470476 (0.0007) [2023-12-26 18:47:14,287][105620] Updated weights for policy 1, policy_version 470486 (0.0006) [2023-12-26 18:47:14,356][105620] Updated weights for policy 1, policy_version 470496 (0.0006) [2023-12-26 18:47:14,678][105692] Updated weights for policy 0, policy_version 470085 (0.0005) [2023-12-26 18:47:14,747][105692] Updated weights for policy 0, policy_version 470095 (0.0005) [2023-12-26 18:47:14,813][105692] Updated weights for policy 0, policy_version 470105 (0.0008) [2023-12-26 18:47:14,984][105620] Updated weights for policy 1, policy_version 470506 (0.0007) [2023-12-26 18:47:15,043][105620] Updated weights for policy 1, policy_version 470516 (0.0009) [2023-12-26 18:47:15,110][105620] Updated weights for policy 1, policy_version 470526 (0.0009) [2023-12-26 18:47:15,182][105620] Updated weights for policy 1, policy_version 470536 (0.0007) [2023-12-26 18:47:15,428][105692] Updated weights for policy 0, policy_version 470115 (0.0010) [2023-12-26 18:47:15,490][105692] Updated weights for policy 0, policy_version 470125 (0.0010) [2023-12-26 18:47:15,545][105692] Updated weights for policy 0, policy_version 470135 (0.0010) [2023-12-26 18:47:15,705][105620] Updated weights for policy 1, policy_version 470546 (0.0009) [2023-12-26 18:47:15,750][105620] Updated weights for policy 1, policy_version 470556 (0.0010) [2023-12-26 18:47:15,798][105620] Updated weights for policy 1, policy_version 470566 (0.0010) [2023-12-26 18:47:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 240852992. Throughput: 0: 9744.6, 1: 9510.1. Samples: 240816800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:47:16,063][104569] Avg episode reward: [(0, '8109.677'), (1, '9357.229')] [2023-12-26 18:47:16,074][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000470144_120373248.pth... [2023-12-26 18:47:16,074][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000470568_120479744.pth... [2023-12-26 18:47:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000468992_120078336.pth [2023-12-26 18:47:16,081][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000469448_120193024.pth [2023-12-26 18:47:16,154][105692] Updated weights for policy 0, policy_version 470145 (0.0010) [2023-12-26 18:47:16,213][105692] Updated weights for policy 0, policy_version 470155 (0.0007) [2023-12-26 18:47:16,257][105692] Updated weights for policy 0, policy_version 470165 (0.0007) [2023-12-26 18:47:16,308][105692] Updated weights for policy 0, policy_version 470175 (0.0008) [2023-12-26 18:47:16,530][105620] Updated weights for policy 1, policy_version 470576 (0.0009) [2023-12-26 18:47:16,577][105620] Updated weights for policy 1, policy_version 470586 (0.0009) [2023-12-26 18:47:16,634][105620] Updated weights for policy 1, policy_version 470596 (0.0010) [2023-12-26 18:47:17,105][105692] Updated weights for policy 0, policy_version 470185 (0.0009) [2023-12-26 18:47:17,163][105692] Updated weights for policy 0, policy_version 470195 (0.0009) [2023-12-26 18:47:17,219][105692] Updated weights for policy 0, policy_version 470205 (0.0008) [2023-12-26 18:47:17,220][105620] Updated weights for policy 1, policy_version 470606 (0.0007) [2023-12-26 18:47:17,278][105620] Updated weights for policy 1, policy_version 470616 (0.0005) [2023-12-26 18:47:17,326][105620] Updated weights for policy 1, policy_version 470626 (0.0005) [2023-12-26 18:47:17,841][105692] Updated weights for policy 0, policy_version 470215 (0.0006) [2023-12-26 18:47:17,885][105692] Updated weights for policy 0, policy_version 470225 (0.0009) [2023-12-26 18:47:17,948][105692] Updated weights for policy 0, policy_version 470235 (0.0008) [2023-12-26 18:47:17,962][105620] Updated weights for policy 1, policy_version 470636 (0.0006) [2023-12-26 18:47:18,021][105620] Updated weights for policy 1, policy_version 470646 (0.0008) [2023-12-26 18:47:18,074][105620] Updated weights for policy 1, policy_version 470656 (0.0009) [2023-12-26 18:47:18,532][105692] Updated weights for policy 0, policy_version 470245 (0.0006) [2023-12-26 18:47:18,589][105692] Updated weights for policy 0, policy_version 470255 (0.0005) [2023-12-26 18:47:18,648][105692] Updated weights for policy 0, policy_version 470265 (0.0005) [2023-12-26 18:47:18,843][105620] Updated weights for policy 1, policy_version 470666 (0.0010) [2023-12-26 18:47:18,898][105620] Updated weights for policy 1, policy_version 470676 (0.0010) [2023-12-26 18:47:18,950][105620] Updated weights for policy 1, policy_version 470686 (0.0010) [2023-12-26 18:47:19,015][105620] Updated weights for policy 1, policy_version 470696 (0.0010) [2023-12-26 18:47:19,158][105692] Updated weights for policy 0, policy_version 470275 (0.0007) [2023-12-26 18:47:19,216][105692] Updated weights for policy 0, policy_version 470285 (0.0010) [2023-12-26 18:47:19,286][105692] Updated weights for policy 0, policy_version 470296 (0.0007) [2023-12-26 18:47:19,800][105620] Updated weights for policy 1, policy_version 470706 (0.0008) [2023-12-26 18:47:19,862][105620] Updated weights for policy 1, policy_version 470716 (0.0008) [2023-12-26 18:47:19,928][105620] Updated weights for policy 1, policy_version 470726 (0.0009) [2023-12-26 18:47:20,104][105692] Updated weights for policy 0, policy_version 470306 (0.0007) [2023-12-26 18:47:20,161][105692] Updated weights for policy 0, policy_version 470316 (0.0010) [2023-12-26 18:47:20,213][105692] Updated weights for policy 0, policy_version 470326 (0.0010) [2023-12-26 18:47:20,273][105692] Updated weights for policy 0, policy_version 470336 (0.0010) [2023-12-26 18:47:20,722][105620] Updated weights for policy 1, policy_version 470736 (0.0008) [2023-12-26 18:47:20,787][105620] Updated weights for policy 1, policy_version 470746 (0.0008) [2023-12-26 18:47:20,855][105620] Updated weights for policy 1, policy_version 470756 (0.0009) [2023-12-26 18:47:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 240951296. Throughput: 0: 9886.1, 1: 9623.9. Samples: 240944156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:47:21,063][104569] Avg episode reward: [(0, '8465.782'), (1, '9357.343')] [2023-12-26 18:47:21,066][105692] Updated weights for policy 0, policy_version 470346 (0.0011) [2023-12-26 18:47:21,131][105692] Updated weights for policy 0, policy_version 470356 (0.0011) [2023-12-26 18:47:21,197][105692] Updated weights for policy 0, policy_version 470366 (0.0011) [2023-12-26 18:47:21,671][105620] Updated weights for policy 1, policy_version 470766 (0.0008) [2023-12-26 18:47:21,740][105620] Updated weights for policy 1, policy_version 470776 (0.0010) [2023-12-26 18:47:21,798][105620] Updated weights for policy 1, policy_version 470786 (0.0009) [2023-12-26 18:47:21,939][105692] Updated weights for policy 0, policy_version 470376 (0.0008) [2023-12-26 18:47:21,990][105692] Updated weights for policy 0, policy_version 470386 (0.0006) [2023-12-26 18:47:22,057][105692] Updated weights for policy 0, policy_version 470396 (0.0006) [2023-12-26 18:47:22,567][105620] Updated weights for policy 1, policy_version 470796 (0.0008) [2023-12-26 18:47:22,627][105620] Updated weights for policy 1, policy_version 470806 (0.0007) [2023-12-26 18:47:22,672][105692] Updated weights for policy 0, policy_version 470406 (0.0006) [2023-12-26 18:47:22,685][105620] Updated weights for policy 1, policy_version 470816 (0.0006) [2023-12-26 18:47:22,733][105692] Updated weights for policy 0, policy_version 470416 (0.0006) [2023-12-26 18:47:22,802][105692] Updated weights for policy 0, policy_version 470426 (0.0005) [2023-12-26 18:47:23,322][105692] Updated weights for policy 0, policy_version 470436 (0.0005) [2023-12-26 18:47:23,387][105692] Updated weights for policy 0, policy_version 470446 (0.0005) [2023-12-26 18:47:23,438][105620] Updated weights for policy 1, policy_version 470826 (0.0006) [2023-12-26 18:47:23,449][105692] Updated weights for policy 0, policy_version 470456 (0.0005) [2023-12-26 18:47:23,485][105620] Updated weights for policy 1, policy_version 470836 (0.0008) [2023-12-26 18:47:23,544][105620] Updated weights for policy 1, policy_version 470846 (0.0008) [2023-12-26 18:47:23,605][105620] Updated weights for policy 1, policy_version 470856 (0.0010) [2023-12-26 18:47:24,115][105692] Updated weights for policy 0, policy_version 470466 (0.0010) [2023-12-26 18:47:24,167][105692] Updated weights for policy 0, policy_version 470476 (0.0010) [2023-12-26 18:47:24,218][105692] Updated weights for policy 0, policy_version 470486 (0.0010) [2023-12-26 18:47:24,274][105692] Updated weights for policy 0, policy_version 470496 (0.0010) [2023-12-26 18:47:24,407][105620] Updated weights for policy 1, policy_version 470866 (0.0009) [2023-12-26 18:47:24,481][105620] Updated weights for policy 1, policy_version 470876 (0.0010) [2023-12-26 18:47:24,543][105620] Updated weights for policy 1, policy_version 470886 (0.0009) [2023-12-26 18:47:24,957][105692] Updated weights for policy 0, policy_version 470506 (0.0010) [2023-12-26 18:47:25,015][105692] Updated weights for policy 0, policy_version 470516 (0.0010) [2023-12-26 18:47:25,077][105692] Updated weights for policy 0, policy_version 470526 (0.0010) [2023-12-26 18:47:25,278][105620] Updated weights for policy 1, policy_version 470896 (0.0008) [2023-12-26 18:47:25,337][105620] Updated weights for policy 1, policy_version 470906 (0.0008) [2023-12-26 18:47:25,403][105620] Updated weights for policy 1, policy_version 470916 (0.0008) [2023-12-26 18:47:25,805][105692] Updated weights for policy 0, policy_version 470536 (0.0011) [2023-12-26 18:47:25,865][105692] Updated weights for policy 0, policy_version 470546 (0.0010) [2023-12-26 18:47:25,930][105692] Updated weights for policy 0, policy_version 470556 (0.0010) [2023-12-26 18:47:26,049][105620] Updated weights for policy 1, policy_version 470926 (0.0007) [2023-12-26 18:47:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 241049600. Throughput: 0: 9946.2, 1: 9577.2. Samples: 241059436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:47:26,062][104569] Avg episode reward: [(0, '8825.011'), (1, '9357.368')] [2023-12-26 18:47:26,097][105620] Updated weights for policy 1, policy_version 470936 (0.0007) [2023-12-26 18:47:26,163][105620] Updated weights for policy 1, policy_version 470946 (0.0007) [2023-12-26 18:47:26,644][105692] Updated weights for policy 0, policy_version 470566 (0.0010) [2023-12-26 18:47:26,695][105692] Updated weights for policy 0, policy_version 470576 (0.0010) [2023-12-26 18:47:26,758][105692] Updated weights for policy 0, policy_version 470586 (0.0010) [2023-12-26 18:47:26,762][105620] Updated weights for policy 1, policy_version 470956 (0.0007) [2023-12-26 18:47:26,814][105620] Updated weights for policy 1, policy_version 470966 (0.0008) [2023-12-26 18:47:26,867][105620] Updated weights for policy 1, policy_version 470976 (0.0010) [2023-12-26 18:47:27,322][105692] Updated weights for policy 0, policy_version 470596 (0.0008) [2023-12-26 18:47:27,382][105692] Updated weights for policy 0, policy_version 470606 (0.0010) [2023-12-26 18:47:27,426][105692] Updated weights for policy 0, policy_version 470616 (0.0010) [2023-12-26 18:47:27,586][105620] Updated weights for policy 1, policy_version 470987 (0.0009) [2023-12-26 18:47:27,649][105620] Updated weights for policy 1, policy_version 470997 (0.0006) [2023-12-26 18:47:27,699][105620] Updated weights for policy 1, policy_version 471007 (0.0008) [2023-12-26 18:47:28,087][105692] Updated weights for policy 0, policy_version 470627 (0.0008) [2023-12-26 18:47:28,148][105692] Updated weights for policy 0, policy_version 470637 (0.0006) [2023-12-26 18:47:28,194][105692] Updated weights for policy 0, policy_version 470647 (0.0005) [2023-12-26 18:47:28,344][105620] Updated weights for policy 1, policy_version 471017 (0.0008) [2023-12-26 18:47:28,410][105620] Updated weights for policy 1, policy_version 471027 (0.0009) [2023-12-26 18:47:28,467][105620] Updated weights for policy 1, policy_version 471037 (0.0008) [2023-12-26 18:47:28,527][105620] Updated weights for policy 1, policy_version 471047 (0.0008) [2023-12-26 18:47:28,829][105692] Updated weights for policy 0, policy_version 470657 (0.0005) [2023-12-26 18:47:28,879][105692] Updated weights for policy 0, policy_version 470667 (0.0005) [2023-12-26 18:47:28,946][105692] Updated weights for policy 0, policy_version 470677 (0.0005) [2023-12-26 18:47:29,003][105692] Updated weights for policy 0, policy_version 470687 (0.0009) [2023-12-26 18:47:29,148][105620] Updated weights for policy 1, policy_version 471057 (0.0005) [2023-12-26 18:47:29,206][105620] Updated weights for policy 1, policy_version 471067 (0.0007) [2023-12-26 18:47:29,269][105620] Updated weights for policy 1, policy_version 471077 (0.0009) [2023-12-26 18:47:29,713][105692] Updated weights for policy 0, policy_version 470697 (0.0010) [2023-12-26 18:47:29,774][105692] Updated weights for policy 0, policy_version 470707 (0.0010) [2023-12-26 18:47:29,831][105692] Updated weights for policy 0, policy_version 470717 (0.0010) [2023-12-26 18:47:29,999][105620] Updated weights for policy 1, policy_version 471087 (0.0006) [2023-12-26 18:47:30,050][105620] Updated weights for policy 1, policy_version 471097 (0.0008) [2023-12-26 18:47:30,106][105620] Updated weights for policy 1, policy_version 471107 (0.0008) [2023-12-26 18:47:30,564][105692] Updated weights for policy 0, policy_version 470727 (0.0008) [2023-12-26 18:47:30,608][105692] Updated weights for policy 0, policy_version 470737 (0.0008) [2023-12-26 18:47:30,653][105692] Updated weights for policy 0, policy_version 470747 (0.0008) [2023-12-26 18:47:30,799][105620] Updated weights for policy 1, policy_version 471117 (0.0009) [2023-12-26 18:47:30,858][105620] Updated weights for policy 1, policy_version 471127 (0.0009) [2023-12-26 18:47:30,912][105620] Updated weights for policy 1, policy_version 471137 (0.0009) [2023-12-26 18:47:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 241156096. Throughput: 0: 9969.2, 1: 9639.7. Samples: 241123056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:47:31,062][104569] Avg episode reward: [(0, '8909.363'), (1, '9174.407')] [2023-12-26 18:47:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000470752_120528896.pth... [2023-12-26 18:47:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000471144_120627200.pth... [2023-12-26 18:47:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000469992_120332288.pth [2023-12-26 18:47:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000469568_120225792.pth [2023-12-26 18:47:31,406][105692] Updated weights for policy 0, policy_version 470757 (0.0006) [2023-12-26 18:47:31,458][105692] Updated weights for policy 0, policy_version 470767 (0.0005) [2023-12-26 18:47:31,511][105692] Updated weights for policy 0, policy_version 470777 (0.0006) [2023-12-26 18:47:31,688][105620] Updated weights for policy 1, policy_version 471147 (0.0008) [2023-12-26 18:47:31,751][105620] Updated weights for policy 1, policy_version 471157 (0.0008) [2023-12-26 18:47:31,808][105620] Updated weights for policy 1, policy_version 471167 (0.0008) [2023-12-26 18:47:32,174][105692] Updated weights for policy 0, policy_version 470787 (0.0007) [2023-12-26 18:47:32,232][105692] Updated weights for policy 0, policy_version 470797 (0.0005) [2023-12-26 18:47:32,300][105692] Updated weights for policy 0, policy_version 470807 (0.0007) [2023-12-26 18:47:32,654][105620] Updated weights for policy 1, policy_version 471177 (0.0008) [2023-12-26 18:47:32,713][105620] Updated weights for policy 1, policy_version 471187 (0.0011) [2023-12-26 18:47:32,778][105620] Updated weights for policy 1, policy_version 471197 (0.0008) [2023-12-26 18:47:32,849][105620] Updated weights for policy 1, policy_version 471207 (0.0009) [2023-12-26 18:47:32,870][105692] Updated weights for policy 0, policy_version 470817 (0.0007) [2023-12-26 18:47:32,931][105692] Updated weights for policy 0, policy_version 470827 (0.0007) [2023-12-26 18:47:32,988][105692] Updated weights for policy 0, policy_version 470837 (0.0009) [2023-12-26 18:47:33,044][105692] Updated weights for policy 0, policy_version 470847 (0.0005) [2023-12-26 18:47:33,555][105692] Updated weights for policy 0, policy_version 470857 (0.0005) [2023-12-26 18:47:33,612][105692] Updated weights for policy 0, policy_version 470867 (0.0005) [2023-12-26 18:47:33,672][105692] Updated weights for policy 0, policy_version 470877 (0.0009) [2023-12-26 18:47:33,724][105620] Updated weights for policy 1, policy_version 471217 (0.0008) [2023-12-26 18:47:33,768][105620] Updated weights for policy 1, policy_version 471227 (0.0007) [2023-12-26 18:47:33,819][105620] Updated weights for policy 1, policy_version 471238 (0.0009) [2023-12-26 18:47:34,218][105692] Updated weights for policy 0, policy_version 470887 (0.0010) [2023-12-26 18:47:34,279][105692] Updated weights for policy 0, policy_version 470897 (0.0011) [2023-12-26 18:47:34,332][105692] Updated weights for policy 0, policy_version 470907 (0.0011) [2023-12-26 18:47:34,624][105620] Updated weights for policy 1, policy_version 471248 (0.0008) [2023-12-26 18:47:34,685][105620] Updated weights for policy 1, policy_version 471258 (0.0008) [2023-12-26 18:47:34,736][105620] Updated weights for policy 1, policy_version 471268 (0.0007) [2023-12-26 18:47:35,098][105692] Updated weights for policy 0, policy_version 470917 (0.0010) [2023-12-26 18:47:35,163][105692] Updated weights for policy 0, policy_version 470927 (0.0009) [2023-12-26 18:47:35,217][105692] Updated weights for policy 0, policy_version 470937 (0.0009) [2023-12-26 18:47:35,466][105620] Updated weights for policy 1, policy_version 471278 (0.0006) [2023-12-26 18:47:35,527][105620] Updated weights for policy 1, policy_version 471288 (0.0009) [2023-12-26 18:47:35,588][105620] Updated weights for policy 1, policy_version 471298 (0.0009) [2023-12-26 18:47:35,914][105692] Updated weights for policy 0, policy_version 470947 (0.0009) [2023-12-26 18:47:35,966][105692] Updated weights for policy 0, policy_version 470957 (0.0009) [2023-12-26 18:47:35,996][105585] KL-divergence is very high: 130.9073 [2023-12-26 18:47:36,016][105692] Updated weights for policy 0, policy_version 470967 (0.0009) [2023-12-26 18:47:36,034][105585] KL-divergence is very high: 159.7561 [2023-12-26 18:47:36,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 241254400. Throughput: 0: 10050.4, 1: 9522.8. Samples: 241240456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:47:36,063][104569] Avg episode reward: [(0, '8906.515'), (1, '8311.472')] [2023-12-26 18:47:36,220][105620] Updated weights for policy 1, policy_version 471308 (0.0010) [2023-12-26 18:47:36,274][105620] Updated weights for policy 1, policy_version 471318 (0.0009) [2023-12-26 18:47:36,337][105620] Updated weights for policy 1, policy_version 471328 (0.0005) [2023-12-26 18:47:36,796][105692] Updated weights for policy 0, policy_version 470978 (0.0010) [2023-12-26 18:47:36,851][105692] Updated weights for policy 0, policy_version 470988 (0.0005) [2023-12-26 18:47:36,904][105692] Updated weights for policy 0, policy_version 470998 (0.0005) [2023-12-26 18:47:36,969][105692] Updated weights for policy 0, policy_version 471008 (0.0006) [2023-12-26 18:47:37,013][105620] Updated weights for policy 1, policy_version 471338 (0.0006) [2023-12-26 18:47:37,079][105620] Updated weights for policy 1, policy_version 471348 (0.0011) [2023-12-26 18:47:37,132][105620] Updated weights for policy 1, policy_version 471358 (0.0011) [2023-12-26 18:47:37,184][105620] Updated weights for policy 1, policy_version 471368 (0.0008) [2023-12-26 18:47:37,638][105692] Updated weights for policy 0, policy_version 471018 (0.0009) [2023-12-26 18:47:37,696][105692] Updated weights for policy 0, policy_version 471028 (0.0010) [2023-12-26 18:47:37,749][105692] Updated weights for policy 0, policy_version 471038 (0.0010) [2023-12-26 18:47:37,805][105620] Updated weights for policy 1, policy_version 471378 (0.0007) [2023-12-26 18:47:37,850][105620] Updated weights for policy 1, policy_version 471388 (0.0008) [2023-12-26 18:47:37,901][105620] Updated weights for policy 1, policy_version 471398 (0.0006) [2023-12-26 18:47:38,498][105692] Updated weights for policy 0, policy_version 471048 (0.0010) [2023-12-26 18:47:38,561][105692] Updated weights for policy 0, policy_version 471058 (0.0009) [2023-12-26 18:47:38,565][105620] Updated weights for policy 1, policy_version 471408 (0.0006) [2023-12-26 18:47:38,618][105692] Updated weights for policy 0, policy_version 471068 (0.0009) [2023-12-26 18:47:38,626][105620] Updated weights for policy 1, policy_version 471418 (0.0006) [2023-12-26 18:47:38,692][105620] Updated weights for policy 1, policy_version 471428 (0.0006) [2023-12-26 18:47:39,366][105620] Updated weights for policy 1, policy_version 471438 (0.0008) [2023-12-26 18:47:39,421][105692] Updated weights for policy 0, policy_version 471078 (0.0008) [2023-12-26 18:47:39,440][105620] Updated weights for policy 1, policy_version 471448 (0.0008) [2023-12-26 18:47:39,484][105692] Updated weights for policy 0, policy_version 471088 (0.0006) [2023-12-26 18:47:39,502][105620] Updated weights for policy 1, policy_version 471458 (0.0007) [2023-12-26 18:47:39,543][105692] Updated weights for policy 0, policy_version 471098 (0.0007) [2023-12-26 18:47:40,249][105692] Updated weights for policy 0, policy_version 471108 (0.0008) [2023-12-26 18:47:40,314][105692] Updated weights for policy 0, policy_version 471118 (0.0008) [2023-12-26 18:47:40,320][105620] Updated weights for policy 1, policy_version 471468 (0.0007) [2023-12-26 18:47:40,373][105692] Updated weights for policy 0, policy_version 471128 (0.0006) [2023-12-26 18:47:40,376][105620] Updated weights for policy 1, policy_version 471478 (0.0007) [2023-12-26 18:47:40,438][105620] Updated weights for policy 1, policy_version 471488 (0.0008) [2023-12-26 18:47:41,060][105692] Updated weights for policy 0, policy_version 471138 (0.0007) [2023-12-26 18:47:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 241344512. Throughput: 0: 10056.3, 1: 9633.3. Samples: 241356972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:47:41,062][104569] Avg episode reward: [(0, '9359.535'), (1, '8382.611')] [2023-12-26 18:47:41,122][105692] Updated weights for policy 0, policy_version 471148 (0.0008) [2023-12-26 18:47:41,183][105692] Updated weights for policy 0, policy_version 471158 (0.0008) [2023-12-26 18:47:41,238][105692] Updated weights for policy 0, policy_version 471168 (0.0008) [2023-12-26 18:47:41,304][105620] Updated weights for policy 1, policy_version 471498 (0.0010) [2023-12-26 18:47:41,378][105620] Updated weights for policy 1, policy_version 471508 (0.0009) [2023-12-26 18:47:41,440][105620] Updated weights for policy 1, policy_version 471518 (0.0008) [2023-12-26 18:47:41,506][105620] Updated weights for policy 1, policy_version 471528 (0.0009) [2023-12-26 18:47:41,978][105692] Updated weights for policy 0, policy_version 471178 (0.0009) [2023-12-26 18:47:42,038][105692] Updated weights for policy 0, policy_version 471188 (0.0008) [2023-12-26 18:47:42,099][105692] Updated weights for policy 0, policy_version 471198 (0.0008) [2023-12-26 18:47:42,273][105620] Updated weights for policy 1, policy_version 471538 (0.0009) [2023-12-26 18:47:42,342][105620] Updated weights for policy 1, policy_version 471548 (0.0008) [2023-12-26 18:47:42,409][105620] Updated weights for policy 1, policy_version 471558 (0.0010) [2023-12-26 18:47:42,853][105692] Updated weights for policy 0, policy_version 471208 (0.0010) [2023-12-26 18:47:42,921][105692] Updated weights for policy 0, policy_version 471218 (0.0010) [2023-12-26 18:47:42,981][105692] Updated weights for policy 0, policy_version 471228 (0.0008) [2023-12-26 18:47:43,115][105620] Updated weights for policy 1, policy_version 471568 (0.0009) [2023-12-26 18:47:43,173][105620] Updated weights for policy 1, policy_version 471578 (0.0009) [2023-12-26 18:47:43,236][105620] Updated weights for policy 1, policy_version 471588 (0.0009) [2023-12-26 18:47:43,603][105692] Updated weights for policy 0, policy_version 471238 (0.0006) [2023-12-26 18:47:43,658][105692] Updated weights for policy 0, policy_version 471248 (0.0005) [2023-12-26 18:47:43,711][105692] Updated weights for policy 0, policy_version 471258 (0.0005) [2023-12-26 18:47:44,006][105620] Updated weights for policy 1, policy_version 471598 (0.0009) [2023-12-26 18:47:44,059][105620] Updated weights for policy 1, policy_version 471608 (0.0010) [2023-12-26 18:47:44,117][105620] Updated weights for policy 1, policy_version 471619 (0.0009) [2023-12-26 18:47:44,261][105692] Updated weights for policy 0, policy_version 471268 (0.0008) [2023-12-26 18:47:44,329][105692] Updated weights for policy 0, policy_version 471278 (0.0009) [2023-12-26 18:47:44,391][105692] Updated weights for policy 0, policy_version 471288 (0.0009) [2023-12-26 18:47:44,874][105620] Updated weights for policy 1, policy_version 471629 (0.0007) [2023-12-26 18:47:44,929][105620] Updated weights for policy 1, policy_version 471639 (0.0009) [2023-12-26 18:47:44,992][105620] Updated weights for policy 1, policy_version 471649 (0.0010) [2023-12-26 18:47:45,143][105692] Updated weights for policy 0, policy_version 471298 (0.0009) [2023-12-26 18:47:45,195][105692] Updated weights for policy 0, policy_version 471308 (0.0010) [2023-12-26 18:47:45,254][105692] Updated weights for policy 0, policy_version 471318 (0.0010) [2023-12-26 18:47:45,320][105692] Updated weights for policy 0, policy_version 471328 (0.0010) [2023-12-26 18:47:45,737][105620] Updated weights for policy 1, policy_version 471659 (0.0008) [2023-12-26 18:47:45,787][105620] Updated weights for policy 1, policy_version 471669 (0.0008) [2023-12-26 18:47:45,838][105620] Updated weights for policy 1, policy_version 471679 (0.0008) [2023-12-26 18:47:46,057][105692] Updated weights for policy 0, policy_version 471338 (0.0010) [2023-12-26 18:47:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 241442816. Throughput: 0: 10066.6, 1: 9627.3. Samples: 241413308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:47:46,063][104569] Avg episode reward: [(0, '9359.348'), (1, '8310.385')] [2023-12-26 18:47:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000471688_120766464.pth... [2023-12-26 18:47:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000470568_120479744.pth [2023-12-26 18:47:46,115][105692] Updated weights for policy 0, policy_version 471348 (0.0010) [2023-12-26 18:47:46,173][105692] Updated weights for policy 0, policy_version 471358 (0.0010) [2023-12-26 18:47:46,185][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000471360_120684544.pth... [2023-12-26 18:47:46,188][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000470144_120373248.pth [2023-12-26 18:47:46,623][105620] Updated weights for policy 1, policy_version 471689 (0.0008) [2023-12-26 18:47:46,670][105620] Updated weights for policy 1, policy_version 471699 (0.0008) [2023-12-26 18:47:46,718][105620] Updated weights for policy 1, policy_version 471709 (0.0008) [2023-12-26 18:47:46,763][105620] Updated weights for policy 1, policy_version 471719 (0.0008) [2023-12-26 18:47:46,918][105692] Updated weights for policy 0, policy_version 471368 (0.0011) [2023-12-26 18:47:46,966][105692] Updated weights for policy 0, policy_version 471378 (0.0010) [2023-12-26 18:47:47,014][105692] Updated weights for policy 0, policy_version 471388 (0.0010) [2023-12-26 18:47:47,444][105620] Updated weights for policy 1, policy_version 471729 (0.0008) [2023-12-26 18:47:47,507][105620] Updated weights for policy 1, policy_version 471739 (0.0008) [2023-12-26 18:47:47,558][105620] Updated weights for policy 1, policy_version 471749 (0.0008) [2023-12-26 18:47:47,771][105692] Updated weights for policy 0, policy_version 471398 (0.0010) [2023-12-26 18:47:47,823][105692] Updated weights for policy 0, policy_version 471408 (0.0010) [2023-12-26 18:47:47,875][105692] Updated weights for policy 0, policy_version 471418 (0.0010) [2023-12-26 18:47:48,368][105620] Updated weights for policy 1, policy_version 471759 (0.0008) [2023-12-26 18:47:48,424][105620] Updated weights for policy 1, policy_version 471769 (0.0008) [2023-12-26 18:47:48,484][105620] Updated weights for policy 1, policy_version 471779 (0.0008) [2023-12-26 18:47:48,493][105692] Updated weights for policy 0, policy_version 471428 (0.0010) [2023-12-26 18:47:48,545][105692] Updated weights for policy 0, policy_version 471438 (0.0010) [2023-12-26 18:47:48,594][105692] Updated weights for policy 0, policy_version 471448 (0.0010) [2023-12-26 18:47:49,252][105620] Updated weights for policy 1, policy_version 471789 (0.0007) [2023-12-26 18:47:49,318][105620] Updated weights for policy 1, policy_version 471799 (0.0008) [2023-12-26 18:47:49,333][105692] Updated weights for policy 0, policy_version 471458 (0.0011) [2023-12-26 18:47:49,378][105620] Updated weights for policy 1, policy_version 471809 (0.0008) [2023-12-26 18:47:49,397][105692] Updated weights for policy 0, policy_version 471468 (0.0011) [2023-12-26 18:47:49,454][105692] Updated weights for policy 0, policy_version 471478 (0.0007) [2023-12-26 18:47:49,510][105692] Updated weights for policy 0, policy_version 471488 (0.0011) [2023-12-26 18:47:50,119][105620] Updated weights for policy 1, policy_version 471819 (0.0008) [2023-12-26 18:47:50,184][105620] Updated weights for policy 1, policy_version 471829 (0.0006) [2023-12-26 18:47:50,248][105620] Updated weights for policy 1, policy_version 471839 (0.0008) [2023-12-26 18:47:50,251][105692] Updated weights for policy 0, policy_version 471498 (0.0007) [2023-12-26 18:47:50,299][105692] Updated weights for policy 0, policy_version 471508 (0.0007) [2023-12-26 18:47:50,353][105692] Updated weights for policy 0, policy_version 471518 (0.0005) [2023-12-26 18:47:50,948][105692] Updated weights for policy 0, policy_version 471528 (0.0010) [2023-12-26 18:47:51,005][105692] Updated weights for policy 0, policy_version 471538 (0.0011) [2023-12-26 18:47:51,032][105620] Updated weights for policy 1, policy_version 471849 (0.0006) [2023-12-26 18:47:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 241532928. Throughput: 0: 10163.0, 1: 9609.1. Samples: 241529448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:47:51,063][104569] Avg episode reward: [(0, '9358.536'), (1, '9096.696')] [2023-12-26 18:47:51,072][105692] Updated weights for policy 0, policy_version 471548 (0.0009) [2023-12-26 18:47:51,101][105620] Updated weights for policy 1, policy_version 471859 (0.0008) [2023-12-26 18:47:51,159][105620] Updated weights for policy 1, policy_version 471869 (0.0008) [2023-12-26 18:47:51,216][105620] Updated weights for policy 1, policy_version 471879 (0.0008) [2023-12-26 18:47:51,840][105692] Updated weights for policy 0, policy_version 471558 (0.0009) [2023-12-26 18:47:51,909][105692] Updated weights for policy 0, policy_version 471568 (0.0006) [2023-12-26 18:47:51,975][105692] Updated weights for policy 0, policy_version 471578 (0.0006) [2023-12-26 18:47:51,984][105620] Updated weights for policy 1, policy_version 471889 (0.0008) [2023-12-26 18:47:52,039][105620] Updated weights for policy 1, policy_version 471899 (0.0007) [2023-12-26 18:47:52,101][105620] Updated weights for policy 1, policy_version 471909 (0.0008) [2023-12-26 18:47:52,648][105692] Updated weights for policy 0, policy_version 471588 (0.0008) [2023-12-26 18:47:52,706][105692] Updated weights for policy 0, policy_version 471598 (0.0009) [2023-12-26 18:47:52,759][105692] Updated weights for policy 0, policy_version 471608 (0.0009) [2023-12-26 18:47:52,879][105620] Updated weights for policy 1, policy_version 471919 (0.0010) [2023-12-26 18:47:52,937][105620] Updated weights for policy 1, policy_version 471929 (0.0009) [2023-12-26 18:47:52,995][105620] Updated weights for policy 1, policy_version 471939 (0.0009) [2023-12-26 18:47:53,502][105692] Updated weights for policy 0, policy_version 471618 (0.0009) [2023-12-26 18:47:53,557][105692] Updated weights for policy 0, policy_version 471628 (0.0009) [2023-12-26 18:47:53,609][105692] Updated weights for policy 0, policy_version 471638 (0.0009) [2023-12-26 18:47:53,660][105692] Updated weights for policy 0, policy_version 471648 (0.0009) [2023-12-26 18:47:53,723][105620] Updated weights for policy 1, policy_version 471949 (0.0008) [2023-12-26 18:47:53,776][105620] Updated weights for policy 1, policy_version 471959 (0.0008) [2023-12-26 18:47:53,826][105620] Updated weights for policy 1, policy_version 471969 (0.0009) [2023-12-26 18:47:54,442][105692] Updated weights for policy 0, policy_version 471658 (0.0010) [2023-12-26 18:47:54,494][105692] Updated weights for policy 0, policy_version 471668 (0.0010) [2023-12-26 18:47:54,550][105620] Updated weights for policy 1, policy_version 471979 (0.0009) [2023-12-26 18:47:54,556][105692] Updated weights for policy 0, policy_version 471678 (0.0010) [2023-12-26 18:47:54,603][105620] Updated weights for policy 1, policy_version 471989 (0.0007) [2023-12-26 18:47:54,660][105620] Updated weights for policy 1, policy_version 471999 (0.0008) [2023-12-26 18:47:55,314][105692] Updated weights for policy 0, policy_version 471688 (0.0010) [2023-12-26 18:47:55,367][105692] Updated weights for policy 0, policy_version 471698 (0.0010) [2023-12-26 18:47:55,423][105620] Updated weights for policy 1, policy_version 472009 (0.0008) [2023-12-26 18:47:55,434][105692] Updated weights for policy 0, policy_version 471708 (0.0011) [2023-12-26 18:47:55,485][105620] Updated weights for policy 1, policy_version 472019 (0.0007) [2023-12-26 18:47:55,533][105620] Updated weights for policy 1, policy_version 472029 (0.0007) [2023-12-26 18:47:55,585][105620] Updated weights for policy 1, policy_version 472039 (0.0008) [2023-12-26 18:47:56,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 241631232. Throughput: 0: 10060.8, 1: 9502.1. Samples: 241641908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:47:56,063][104569] Avg episode reward: [(0, '9358.107'), (1, '9091.701')] [2023-12-26 18:47:56,185][105692] Updated weights for policy 0, policy_version 471718 (0.0010) [2023-12-26 18:47:56,234][105692] Updated weights for policy 0, policy_version 471728 (0.0010) [2023-12-26 18:47:56,289][105692] Updated weights for policy 0, policy_version 471738 (0.0010) [2023-12-26 18:47:56,370][105620] Updated weights for policy 1, policy_version 472049 (0.0008) [2023-12-26 18:47:56,422][105620] Updated weights for policy 1, policy_version 472059 (0.0008) [2023-12-26 18:47:56,477][105620] Updated weights for policy 1, policy_version 472069 (0.0008) [2023-12-26 18:47:57,042][105692] Updated weights for policy 0, policy_version 471748 (0.0010) [2023-12-26 18:47:57,099][105692] Updated weights for policy 0, policy_version 471758 (0.0010) [2023-12-26 18:47:57,149][105692] Updated weights for policy 0, policy_version 471768 (0.0010) [2023-12-26 18:47:57,234][105620] Updated weights for policy 1, policy_version 472079 (0.0008) [2023-12-26 18:47:57,277][105620] Updated weights for policy 1, policy_version 472089 (0.0008) [2023-12-26 18:47:57,333][105620] Updated weights for policy 1, policy_version 472099 (0.0008) [2023-12-26 18:47:57,890][105692] Updated weights for policy 0, policy_version 471778 (0.0010) [2023-12-26 18:47:57,948][105692] Updated weights for policy 0, policy_version 471788 (0.0010) [2023-12-26 18:47:58,004][105692] Updated weights for policy 0, policy_version 471798 (0.0010) [2023-12-26 18:47:58,058][105692] Updated weights for policy 0, policy_version 471808 (0.0010) [2023-12-26 18:47:58,111][105620] Updated weights for policy 1, policy_version 472109 (0.0009) [2023-12-26 18:47:58,180][105620] Updated weights for policy 1, policy_version 472119 (0.0008) [2023-12-26 18:47:58,238][105620] Updated weights for policy 1, policy_version 472129 (0.0008) [2023-12-26 18:47:58,888][105692] Updated weights for policy 0, policy_version 471818 (0.0011) [2023-12-26 18:47:58,945][105692] Updated weights for policy 0, policy_version 471828 (0.0010) [2023-12-26 18:47:59,004][105692] Updated weights for policy 0, policy_version 471838 (0.0010) [2023-12-26 18:47:59,061][105620] Updated weights for policy 1, policy_version 472139 (0.0008) [2023-12-26 18:47:59,123][105620] Updated weights for policy 1, policy_version 472149 (0.0008) [2023-12-26 18:47:59,182][105620] Updated weights for policy 1, policy_version 472159 (0.0008) [2023-12-26 18:47:59,774][105692] Updated weights for policy 0, policy_version 471848 (0.0010) [2023-12-26 18:47:59,828][105692] Updated weights for policy 0, policy_version 471858 (0.0010) [2023-12-26 18:47:59,881][105692] Updated weights for policy 0, policy_version 471868 (0.0010) [2023-12-26 18:47:59,934][105620] Updated weights for policy 1, policy_version 472169 (0.0008) [2023-12-26 18:47:59,992][105620] Updated weights for policy 1, policy_version 472179 (0.0008) [2023-12-26 18:48:00,044][105620] Updated weights for policy 1, policy_version 472189 (0.0008) [2023-12-26 18:48:00,099][105620] Updated weights for policy 1, policy_version 472199 (0.0008) [2023-12-26 18:48:00,641][105692] Updated weights for policy 0, policy_version 471878 (0.0010) [2023-12-26 18:48:00,692][105692] Updated weights for policy 0, policy_version 471888 (0.0010) [2023-12-26 18:48:00,740][105692] Updated weights for policy 0, policy_version 471898 (0.0010) [2023-12-26 18:48:00,906][105620] Updated weights for policy 1, policy_version 472209 (0.0008) [2023-12-26 18:48:00,970][105620] Updated weights for policy 1, policy_version 472219 (0.0008) [2023-12-26 18:48:01,030][105620] Updated weights for policy 1, policy_version 472229 (0.0007) [2023-12-26 18:48:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 241729536. Throughput: 0: 10058.7, 1: 9506.3. Samples: 241697220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:48:01,063][104569] Avg episode reward: [(0, '9087.314'), (1, '9177.691')] [2023-12-26 18:48:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000471904_120823808.pth... [2023-12-26 18:48:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000472232_120905728.pth... [2023-12-26 18:48:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000470752_120528896.pth [2023-12-26 18:48:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000471144_120627200.pth [2023-12-26 18:48:01,512][105692] Updated weights for policy 0, policy_version 471908 (0.0010) [2023-12-26 18:48:01,567][105692] Updated weights for policy 0, policy_version 471918 (0.0010) [2023-12-26 18:48:01,623][105692] Updated weights for policy 0, policy_version 471928 (0.0011) [2023-12-26 18:48:01,790][105620] Updated weights for policy 1, policy_version 472239 (0.0008) [2023-12-26 18:48:01,849][105620] Updated weights for policy 1, policy_version 472249 (0.0008) [2023-12-26 18:48:01,893][105620] Updated weights for policy 1, policy_version 472259 (0.0008) [2023-12-26 18:48:02,388][105692] Updated weights for policy 0, policy_version 471938 (0.0011) [2023-12-26 18:48:02,446][105692] Updated weights for policy 0, policy_version 471948 (0.0010) [2023-12-26 18:48:02,505][105692] Updated weights for policy 0, policy_version 471958 (0.0010) [2023-12-26 18:48:02,559][105692] Updated weights for policy 0, policy_version 471968 (0.0010) [2023-12-26 18:48:02,620][105620] Updated weights for policy 1, policy_version 472270 (0.0009) [2023-12-26 18:48:02,674][105620] Updated weights for policy 1, policy_version 472280 (0.0010) [2023-12-26 18:48:02,729][105620] Updated weights for policy 1, policy_version 472290 (0.0008) [2023-12-26 18:48:03,294][105692] Updated weights for policy 0, policy_version 471978 (0.0010) [2023-12-26 18:48:03,324][105620] Updated weights for policy 1, policy_version 472300 (0.0007) [2023-12-26 18:48:03,352][105692] Updated weights for policy 0, policy_version 471988 (0.0010) [2023-12-26 18:48:03,372][105620] Updated weights for policy 1, policy_version 472310 (0.0010) [2023-12-26 18:48:03,406][105692] Updated weights for policy 0, policy_version 471998 (0.0010) [2023-12-26 18:48:03,426][105620] Updated weights for policy 1, policy_version 472320 (0.0010) [2023-12-26 18:48:04,100][105620] Updated weights for policy 1, policy_version 472330 (0.0010) [2023-12-26 18:48:04,137][105692] Updated weights for policy 0, policy_version 472008 (0.0010) [2023-12-26 18:48:04,156][105620] Updated weights for policy 1, policy_version 472340 (0.0010) [2023-12-26 18:48:04,200][105692] Updated weights for policy 0, policy_version 472018 (0.0010) [2023-12-26 18:48:04,208][105620] Updated weights for policy 1, policy_version 472350 (0.0011) [2023-12-26 18:48:04,259][105692] Updated weights for policy 0, policy_version 472028 (0.0010) [2023-12-26 18:48:04,272][105620] Updated weights for policy 1, policy_version 472360 (0.0008) [2023-12-26 18:48:04,976][105620] Updated weights for policy 1, policy_version 472370 (0.0005) [2023-12-26 18:48:04,996][105692] Updated weights for policy 0, policy_version 472038 (0.0010) [2023-12-26 18:48:05,039][105620] Updated weights for policy 1, policy_version 472380 (0.0005) [2023-12-26 18:48:05,057][105692] Updated weights for policy 0, policy_version 472048 (0.0010) [2023-12-26 18:48:05,097][105620] Updated weights for policy 1, policy_version 472390 (0.0005) [2023-12-26 18:48:05,116][105692] Updated weights for policy 0, policy_version 472058 (0.0010) [2023-12-26 18:48:05,700][105620] Updated weights for policy 1, policy_version 472400 (0.0006) [2023-12-26 18:48:05,760][105620] Updated weights for policy 1, policy_version 472410 (0.0006) [2023-12-26 18:48:05,817][105620] Updated weights for policy 1, policy_version 472420 (0.0006) [2023-12-26 18:48:05,856][105692] Updated weights for policy 0, policy_version 472068 (0.0010) [2023-12-26 18:48:05,905][105692] Updated weights for policy 0, policy_version 472078 (0.0010) [2023-12-26 18:48:05,964][105692] Updated weights for policy 0, policy_version 472088 (0.0010) [2023-12-26 18:48:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 241827840. Throughput: 0: 9837.0, 1: 9438.3. Samples: 241811544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:48:06,062][104569] Avg episode reward: [(0, '8995.874'), (1, '9176.929')] [2023-12-26 18:48:06,461][105620] Updated weights for policy 1, policy_version 472430 (0.0008) [2023-12-26 18:48:06,531][105620] Updated weights for policy 1, policy_version 472440 (0.0007) [2023-12-26 18:48:06,580][105620] Updated weights for policy 1, policy_version 472450 (0.0011) [2023-12-26 18:48:06,673][105692] Updated weights for policy 0, policy_version 472098 (0.0009) [2023-12-26 18:48:06,731][105692] Updated weights for policy 0, policy_version 472108 (0.0009) [2023-12-26 18:48:06,794][105692] Updated weights for policy 0, policy_version 472118 (0.0009) [2023-12-26 18:48:06,855][105692] Updated weights for policy 0, policy_version 472128 (0.0009) [2023-12-26 18:48:07,237][105620] Updated weights for policy 1, policy_version 472460 (0.0010) [2023-12-26 18:48:07,286][105620] Updated weights for policy 1, policy_version 472470 (0.0010) [2023-12-26 18:48:07,348][105620] Updated weights for policy 1, policy_version 472480 (0.0010) [2023-12-26 18:48:07,623][105692] Updated weights for policy 0, policy_version 472138 (0.0008) [2023-12-26 18:48:07,682][105692] Updated weights for policy 0, policy_version 472148 (0.0008) [2023-12-26 18:48:07,734][105692] Updated weights for policy 0, policy_version 472158 (0.0009) [2023-12-26 18:48:08,060][105620] Updated weights for policy 1, policy_version 472490 (0.0009) [2023-12-26 18:48:08,113][105620] Updated weights for policy 1, policy_version 472500 (0.0006) [2023-12-26 18:48:08,161][105620] Updated weights for policy 1, policy_version 472510 (0.0009) [2023-12-26 18:48:08,216][105620] Updated weights for policy 1, policy_version 472520 (0.0010) [2023-12-26 18:48:08,404][105692] Updated weights for policy 0, policy_version 472168 (0.0009) [2023-12-26 18:48:08,452][105692] Updated weights for policy 0, policy_version 472178 (0.0008) [2023-12-26 18:48:08,505][105692] Updated weights for policy 0, policy_version 472188 (0.0009) [2023-12-26 18:48:08,859][105620] Updated weights for policy 1, policy_version 472530 (0.0010) [2023-12-26 18:48:08,893][105586] KL-divergence is very high: 122.5922 [2023-12-26 18:48:08,911][105620] Updated weights for policy 1, policy_version 472540 (0.0010) [2023-12-26 18:48:08,970][105620] Updated weights for policy 1, policy_version 472550 (0.0011) [2023-12-26 18:48:09,271][105692] Updated weights for policy 0, policy_version 472198 (0.0009) [2023-12-26 18:48:09,321][105692] Updated weights for policy 0, policy_version 472208 (0.0008) [2023-12-26 18:48:09,385][105692] Updated weights for policy 0, policy_version 472218 (0.0007) [2023-12-26 18:48:09,744][105620] Updated weights for policy 1, policy_version 472560 (0.0009) [2023-12-26 18:48:09,807][105620] Updated weights for policy 1, policy_version 472570 (0.0010) [2023-12-26 18:48:09,872][105620] Updated weights for policy 1, policy_version 472580 (0.0011) [2023-12-26 18:48:10,055][105692] Updated weights for policy 0, policy_version 472228 (0.0008) [2023-12-26 18:48:10,111][105692] Updated weights for policy 0, policy_version 472238 (0.0007) [2023-12-26 18:48:10,174][105692] Updated weights for policy 0, policy_version 472248 (0.0006) [2023-12-26 18:48:10,566][105620] Updated weights for policy 1, policy_version 472590 (0.0009) [2023-12-26 18:48:10,617][105620] Updated weights for policy 1, policy_version 472600 (0.0009) [2023-12-26 18:48:10,672][105620] Updated weights for policy 1, policy_version 472610 (0.0009) [2023-12-26 18:48:10,850][105692] Updated weights for policy 0, policy_version 472258 (0.0007) [2023-12-26 18:48:10,914][105692] Updated weights for policy 0, policy_version 472268 (0.0008) [2023-12-26 18:48:10,981][105692] Updated weights for policy 0, policy_version 472278 (0.0008) [2023-12-26 18:48:11,054][105692] Updated weights for policy 0, policy_version 472288 (0.0009) [2023-12-26 18:48:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 241926144. Throughput: 0: 9791.4, 1: 9541.0. Samples: 241929392. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-26 18:48:11,062][104569] Avg episode reward: [(0, '8907.096'), (1, '8598.077')] [2023-12-26 18:48:11,541][105620] Updated weights for policy 1, policy_version 472620 (0.0010) [2023-12-26 18:48:11,607][105620] Updated weights for policy 1, policy_version 472630 (0.0008) [2023-12-26 18:48:11,673][105620] Updated weights for policy 1, policy_version 472640 (0.0008) [2023-12-26 18:48:11,814][105692] Updated weights for policy 0, policy_version 472298 (0.0009) [2023-12-26 18:48:11,877][105692] Updated weights for policy 0, policy_version 472308 (0.0008) [2023-12-26 18:48:11,940][105692] Updated weights for policy 0, policy_version 472318 (0.0009) [2023-12-26 18:48:12,516][105620] Updated weights for policy 1, policy_version 472650 (0.0009) [2023-12-26 18:48:12,583][105620] Updated weights for policy 1, policy_version 472660 (0.0010) [2023-12-26 18:48:12,641][105620] Updated weights for policy 1, policy_version 472670 (0.0008) [2023-12-26 18:48:12,660][105692] Updated weights for policy 0, policy_version 472328 (0.0007) [2023-12-26 18:48:12,702][105620] Updated weights for policy 1, policy_version 472680 (0.0008) [2023-12-26 18:48:12,721][105692] Updated weights for policy 0, policy_version 472338 (0.0007) [2023-12-26 18:48:12,784][105692] Updated weights for policy 0, policy_version 472348 (0.0009) [2023-12-26 18:48:13,440][105620] Updated weights for policy 1, policy_version 472690 (0.0009) [2023-12-26 18:48:13,491][105620] Updated weights for policy 1, policy_version 472700 (0.0009) [2023-12-26 18:48:13,536][105692] Updated weights for policy 0, policy_version 472358 (0.0007) [2023-12-26 18:48:13,538][105620] Updated weights for policy 1, policy_version 472710 (0.0008) [2023-12-26 18:48:13,589][105692] Updated weights for policy 0, policy_version 472368 (0.0009) [2023-12-26 18:48:13,648][105692] Updated weights for policy 0, policy_version 472378 (0.0009) [2023-12-26 18:48:14,287][105620] Updated weights for policy 1, policy_version 472720 (0.0006) [2023-12-26 18:48:14,344][105620] Updated weights for policy 1, policy_version 472730 (0.0005) [2023-12-26 18:48:14,395][105620] Updated weights for policy 1, policy_version 472740 (0.0005) [2023-12-26 18:48:14,408][105692] Updated weights for policy 0, policy_version 472388 (0.0009) [2023-12-26 18:48:14,465][105692] Updated weights for policy 0, policy_version 472398 (0.0010) [2023-12-26 18:48:14,522][105692] Updated weights for policy 0, policy_version 472408 (0.0009) [2023-12-26 18:48:14,949][105620] Updated weights for policy 1, policy_version 472750 (0.0008) [2023-12-26 18:48:15,006][105620] Updated weights for policy 1, policy_version 472760 (0.0010) [2023-12-26 18:48:15,057][105620] Updated weights for policy 1, policy_version 472770 (0.0009) [2023-12-26 18:48:15,346][105692] Updated weights for policy 0, policy_version 472419 (0.0009) [2023-12-26 18:48:15,399][105692] Updated weights for policy 0, policy_version 472429 (0.0009) [2023-12-26 18:48:15,464][105692] Updated weights for policy 0, policy_version 472439 (0.0009) [2023-12-26 18:48:15,781][105620] Updated weights for policy 1, policy_version 472780 (0.0009) [2023-12-26 18:48:15,843][105620] Updated weights for policy 1, policy_version 472790 (0.0009) [2023-12-26 18:48:15,904][105620] Updated weights for policy 1, policy_version 472800 (0.0009) [2023-12-26 18:48:16,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 242016256. Throughput: 0: 9711.7, 1: 9428.7. Samples: 241984376. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-26 18:48:16,063][104569] Avg episode reward: [(0, '9092.023'), (1, '8345.190')] [2023-12-26 18:48:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000472448_120963072.pth... [2023-12-26 18:48:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000472808_121053184.pth... [2023-12-26 18:48:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000471360_120684544.pth [2023-12-26 18:48:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000471688_120766464.pth [2023-12-26 18:48:16,172][105692] Updated weights for policy 0, policy_version 472449 (0.0007) [2023-12-26 18:48:16,235][105692] Updated weights for policy 0, policy_version 472459 (0.0006) [2023-12-26 18:48:16,298][105692] Updated weights for policy 0, policy_version 472469 (0.0005) [2023-12-26 18:48:16,355][105692] Updated weights for policy 0, policy_version 472479 (0.0005) [2023-12-26 18:48:16,689][105620] Updated weights for policy 1, policy_version 472810 (0.0008) [2023-12-26 18:48:16,750][105620] Updated weights for policy 1, policy_version 472820 (0.0005) [2023-12-26 18:48:16,812][105620] Updated weights for policy 1, policy_version 472830 (0.0005) [2023-12-26 18:48:16,868][105620] Updated weights for policy 1, policy_version 472840 (0.0009) [2023-12-26 18:48:16,892][105692] Updated weights for policy 0, policy_version 472489 (0.0008) [2023-12-26 18:48:16,951][105692] Updated weights for policy 0, policy_version 472499 (0.0010) [2023-12-26 18:48:17,009][105692] Updated weights for policy 0, policy_version 472509 (0.0010) [2023-12-26 18:48:17,574][105692] Updated weights for policy 0, policy_version 472519 (0.0007) [2023-12-26 18:48:17,639][105620] Updated weights for policy 1, policy_version 472850 (0.0009) [2023-12-26 18:48:17,639][105692] Updated weights for policy 0, policy_version 472529 (0.0008) [2023-12-26 18:48:17,685][105620] Updated weights for policy 1, policy_version 472860 (0.0008) [2023-12-26 18:48:17,695][105692] Updated weights for policy 0, policy_version 472539 (0.0010) [2023-12-26 18:48:17,742][105620] Updated weights for policy 1, policy_version 472870 (0.0006) [2023-12-26 18:48:18,387][105692] Updated weights for policy 0, policy_version 472549 (0.0011) [2023-12-26 18:48:18,447][105692] Updated weights for policy 0, policy_version 472559 (0.0011) [2023-12-26 18:48:18,480][105620] Updated weights for policy 1, policy_version 472880 (0.0008) [2023-12-26 18:48:18,503][105692] Updated weights for policy 0, policy_version 472569 (0.0011) [2023-12-26 18:48:18,537][105620] Updated weights for policy 1, policy_version 472890 (0.0005) [2023-12-26 18:48:18,601][105620] Updated weights for policy 1, policy_version 472900 (0.0005) [2023-12-26 18:48:19,255][105692] Updated weights for policy 0, policy_version 472579 (0.0011) [2023-12-26 18:48:19,309][105620] Updated weights for policy 1, policy_version 472910 (0.0006) [2023-12-26 18:48:19,322][105692] Updated weights for policy 0, policy_version 472589 (0.0011) [2023-12-26 18:48:19,375][105620] Updated weights for policy 1, policy_version 472920 (0.0007) [2023-12-26 18:48:19,388][105692] Updated weights for policy 0, policy_version 472599 (0.0011) [2023-12-26 18:48:19,433][105620] Updated weights for policy 1, policy_version 472930 (0.0006) [2023-12-26 18:48:20,113][105692] Updated weights for policy 0, policy_version 472609 (0.0010) [2023-12-26 18:48:20,182][105692] Updated weights for policy 0, policy_version 472619 (0.0006) [2023-12-26 18:48:20,212][105620] Updated weights for policy 1, policy_version 472940 (0.0008) [2023-12-26 18:48:20,246][105692] Updated weights for policy 0, policy_version 472629 (0.0006) [2023-12-26 18:48:20,269][105620] Updated weights for policy 1, policy_version 472950 (0.0008) [2023-12-26 18:48:20,305][105692] Updated weights for policy 0, policy_version 472639 (0.0006) [2023-12-26 18:48:20,337][105620] Updated weights for policy 1, policy_version 472960 (0.0007) [2023-12-26 18:48:20,940][105692] Updated weights for policy 0, policy_version 472649 (0.0009) [2023-12-26 18:48:20,977][105585] KL-divergence is very high: 104.2264 [2023-12-26 18:48:20,990][105585] KL-divergence is very high: 156.2601 [2023-12-26 18:48:20,997][105585] KL-divergence is very high: 116.0010 [2023-12-26 18:48:21,002][105692] Updated weights for policy 0, policy_version 472659 (0.0009) [2023-12-26 18:48:21,005][105620] Updated weights for policy 1, policy_version 472970 (0.0009) [2023-12-26 18:48:21,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 242106368. Throughput: 0: 9627.8, 1: 9526.1. Samples: 242102384. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-26 18:48:21,065][104569] Avg episode reward: [(0, '7299.566'), (1, '8837.685')] [2023-12-26 18:48:21,070][105620] Updated weights for policy 1, policy_version 472980 (0.0007) [2023-12-26 18:48:21,073][105692] Updated weights for policy 0, policy_version 472669 (0.0009) [2023-12-26 18:48:21,080][105585] KL-divergence is very high: 100.2896 [2023-12-26 18:48:21,131][105620] Updated weights for policy 1, policy_version 472990 (0.0009) [2023-12-26 18:48:21,192][105620] Updated weights for policy 1, policy_version 473000 (0.0009) [2023-12-26 18:48:21,877][105692] Updated weights for policy 0, policy_version 472679 (0.0009) [2023-12-26 18:48:21,939][105692] Updated weights for policy 0, policy_version 472689 (0.0009) [2023-12-26 18:48:21,996][105692] Updated weights for policy 0, policy_version 472699 (0.0009) [2023-12-26 18:48:22,016][105620] Updated weights for policy 1, policy_version 473010 (0.0009) [2023-12-26 18:48:22,074][105620] Updated weights for policy 1, policy_version 473020 (0.0009) [2023-12-26 18:48:22,136][105620] Updated weights for policy 1, policy_version 473030 (0.0010) [2023-12-26 18:48:22,735][105692] Updated weights for policy 0, policy_version 472709 (0.0009) [2023-12-26 18:48:22,800][105692] Updated weights for policy 0, policy_version 472719 (0.0009) [2023-12-26 18:48:22,853][105692] Updated weights for policy 0, policy_version 472729 (0.0009) [2023-12-26 18:48:22,880][105620] Updated weights for policy 1, policy_version 473040 (0.0008) [2023-12-26 18:48:22,939][105620] Updated weights for policy 1, policy_version 473050 (0.0009) [2023-12-26 18:48:22,993][105620] Updated weights for policy 1, policy_version 473060 (0.0009) [2023-12-26 18:48:23,686][105692] Updated weights for policy 0, policy_version 472739 (0.0007) [2023-12-26 18:48:23,704][105620] Updated weights for policy 1, policy_version 473070 (0.0008) [2023-12-26 18:48:23,746][105692] Updated weights for policy 0, policy_version 472749 (0.0007) [2023-12-26 18:48:23,756][105620] Updated weights for policy 1, policy_version 473080 (0.0006) [2023-12-26 18:48:23,802][105692] Updated weights for policy 0, policy_version 472759 (0.0009) [2023-12-26 18:48:23,807][105620] Updated weights for policy 1, policy_version 473090 (0.0007) [2023-12-26 18:48:24,408][105620] Updated weights for policy 1, policy_version 473100 (0.0007) [2023-12-26 18:48:24,468][105620] Updated weights for policy 1, policy_version 473110 (0.0009) [2023-12-26 18:48:24,533][105620] Updated weights for policy 1, policy_version 473120 (0.0008) [2023-12-26 18:48:24,564][105692] Updated weights for policy 0, policy_version 472769 (0.0009) [2023-12-26 18:48:24,614][105692] Updated weights for policy 0, policy_version 472779 (0.0008) [2023-12-26 18:48:24,668][105692] Updated weights for policy 0, policy_version 472789 (0.0009) [2023-12-26 18:48:24,719][105692] Updated weights for policy 0, policy_version 472799 (0.0009) [2023-12-26 18:48:25,281][105620] Updated weights for policy 1, policy_version 473130 (0.0009) [2023-12-26 18:48:25,335][105620] Updated weights for policy 1, policy_version 473140 (0.0009) [2023-12-26 18:48:25,386][105620] Updated weights for policy 1, policy_version 473150 (0.0009) [2023-12-26 18:48:25,457][105620] Updated weights for policy 1, policy_version 473160 (0.0010) [2023-12-26 18:48:25,472][105692] Updated weights for policy 0, policy_version 472809 (0.0007) [2023-12-26 18:48:25,521][105692] Updated weights for policy 0, policy_version 472819 (0.0007) [2023-12-26 18:48:25,576][105692] Updated weights for policy 0, policy_version 472829 (0.0005) [2023-12-26 18:48:26,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 242204672. Throughput: 0: 9603.3, 1: 9478.1. Samples: 242215632. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-26 18:48:26,062][104569] Avg episode reward: [(0, '6635.614'), (1, '9172.369')] [2023-12-26 18:48:26,197][105620] Updated weights for policy 1, policy_version 473170 (0.0005) [2023-12-26 18:48:26,261][105620] Updated weights for policy 1, policy_version 473180 (0.0006) [2023-12-26 18:48:26,286][105692] Updated weights for policy 0, policy_version 472839 (0.0007) [2023-12-26 18:48:26,319][105620] Updated weights for policy 1, policy_version 473190 (0.0007) [2023-12-26 18:48:26,346][105692] Updated weights for policy 0, policy_version 472849 (0.0008) [2023-12-26 18:48:26,417][105692] Updated weights for policy 0, policy_version 472859 (0.0009) [2023-12-26 18:48:26,943][105620] Updated weights for policy 1, policy_version 473200 (0.0007) [2023-12-26 18:48:27,002][105620] Updated weights for policy 1, policy_version 473210 (0.0009) [2023-12-26 18:48:27,053][105620] Updated weights for policy 1, policy_version 473220 (0.0009) [2023-12-26 18:48:27,205][105692] Updated weights for policy 0, policy_version 472869 (0.0009) [2023-12-26 18:48:27,254][105692] Updated weights for policy 0, policy_version 472879 (0.0008) [2023-12-26 18:48:27,307][105692] Updated weights for policy 0, policy_version 472889 (0.0009) [2023-12-26 18:48:27,687][105620] Updated weights for policy 1, policy_version 473230 (0.0009) [2023-12-26 18:48:27,741][105620] Updated weights for policy 1, policy_version 473240 (0.0009) [2023-12-26 18:48:27,787][105620] Updated weights for policy 1, policy_version 473250 (0.0008) [2023-12-26 18:48:28,106][105692] Updated weights for policy 0, policy_version 472899 (0.0009) [2023-12-26 18:48:28,151][105692] Updated weights for policy 0, policy_version 472909 (0.0008) [2023-12-26 18:48:28,197][105692] Updated weights for policy 0, policy_version 472919 (0.0009) [2023-12-26 18:48:28,504][105620] Updated weights for policy 1, policy_version 473260 (0.0009) [2023-12-26 18:48:28,558][105620] Updated weights for policy 1, policy_version 473270 (0.0005) [2023-12-26 18:48:28,611][105620] Updated weights for policy 1, policy_version 473280 (0.0005) [2023-12-26 18:48:28,941][105692] Updated weights for policy 0, policy_version 472929 (0.0009) [2023-12-26 18:48:29,004][105692] Updated weights for policy 0, policy_version 472939 (0.0009) [2023-12-26 18:48:29,057][105692] Updated weights for policy 0, policy_version 472950 (0.0010) [2023-12-26 18:48:29,166][105620] Updated weights for policy 1, policy_version 473290 (0.0007) [2023-12-26 18:48:29,217][105620] Updated weights for policy 1, policy_version 473300 (0.0006) [2023-12-26 18:48:29,278][105620] Updated weights for policy 1, policy_version 473310 (0.0008) [2023-12-26 18:48:29,326][105620] Updated weights for policy 1, policy_version 473320 (0.0009) [2023-12-26 18:48:29,802][105692] Updated weights for policy 0, policy_version 472961 (0.0010) [2023-12-26 18:48:29,864][105692] Updated weights for policy 0, policy_version 472971 (0.0008) [2023-12-26 18:48:29,923][105692] Updated weights for policy 0, policy_version 472981 (0.0008) [2023-12-26 18:48:29,985][105692] Updated weights for policy 0, policy_version 472991 (0.0008) [2023-12-26 18:48:30,090][105620] Updated weights for policy 1, policy_version 473330 (0.0008) [2023-12-26 18:48:30,151][105620] Updated weights for policy 1, policy_version 473340 (0.0008) [2023-12-26 18:48:30,213][105620] Updated weights for policy 1, policy_version 473350 (0.0008) [2023-12-26 18:48:30,578][105692] Updated weights for policy 0, policy_version 473001 (0.0010) [2023-12-26 18:48:30,636][105692] Updated weights for policy 0, policy_version 473011 (0.0009) [2023-12-26 18:48:30,694][105692] Updated weights for policy 0, policy_version 473023 (0.0011) [2023-12-26 18:48:30,863][105620] Updated weights for policy 1, policy_version 473360 (0.0008) [2023-12-26 18:48:30,914][105620] Updated weights for policy 1, policy_version 473370 (0.0009) [2023-12-26 18:48:30,975][105620] Updated weights for policy 1, policy_version 473380 (0.0009) [2023-12-26 18:48:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 242311168. Throughput: 0: 9570.0, 1: 9598.6. Samples: 242275892. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-26 18:48:31,063][104569] Avg episode reward: [(0, '7417.131'), (1, '9263.854')] [2023-12-26 18:48:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000473024_121110528.pth... [2023-12-26 18:48:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000473384_121200640.pth... [2023-12-26 18:48:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000471904_120823808.pth [2023-12-26 18:48:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000472232_120905728.pth [2023-12-26 18:48:31,494][105692] Updated weights for policy 0, policy_version 473033 (0.0009) [2023-12-26 18:48:31,548][105692] Updated weights for policy 0, policy_version 473044 (0.0010) [2023-12-26 18:48:31,602][105692] Updated weights for policy 0, policy_version 473056 (0.0010) [2023-12-26 18:48:31,662][105620] Updated weights for policy 1, policy_version 473390 (0.0008) [2023-12-26 18:48:31,731][105620] Updated weights for policy 1, policy_version 473400 (0.0008) [2023-12-26 18:48:31,796][105620] Updated weights for policy 1, policy_version 473410 (0.0007) [2023-12-26 18:48:32,446][105620] Updated weights for policy 1, policy_version 473420 (0.0008) [2023-12-26 18:48:32,463][105692] Updated weights for policy 0, policy_version 473066 (0.0008) [2023-12-26 18:48:32,505][105620] Updated weights for policy 1, policy_version 473430 (0.0006) [2023-12-26 18:48:32,511][105692] Updated weights for policy 0, policy_version 473076 (0.0007) [2023-12-26 18:48:32,565][105620] Updated weights for policy 1, policy_version 473440 (0.0005) [2023-12-26 18:48:32,568][105692] Updated weights for policy 0, policy_version 473087 (0.0010) [2023-12-26 18:48:33,149][105620] Updated weights for policy 1, policy_version 473450 (0.0006) [2023-12-26 18:48:33,213][105620] Updated weights for policy 1, policy_version 473460 (0.0009) [2023-12-26 18:48:33,268][105620] Updated weights for policy 1, policy_version 473470 (0.0008) [2023-12-26 18:48:33,322][105620] Updated weights for policy 1, policy_version 473480 (0.0010) [2023-12-26 18:48:33,391][105692] Updated weights for policy 0, policy_version 473097 (0.0008) [2023-12-26 18:48:33,445][105692] Updated weights for policy 0, policy_version 473107 (0.0008) [2023-12-26 18:48:33,501][105692] Updated weights for policy 0, policy_version 473117 (0.0008) [2023-12-26 18:48:33,946][105620] Updated weights for policy 1, policy_version 473490 (0.0005) [2023-12-26 18:48:33,999][105620] Updated weights for policy 1, policy_version 473500 (0.0005) [2023-12-26 18:48:34,054][105620] Updated weights for policy 1, policy_version 473510 (0.0005) [2023-12-26 18:48:34,352][105692] Updated weights for policy 0, policy_version 473127 (0.0006) [2023-12-26 18:48:34,412][105692] Updated weights for policy 0, policy_version 473137 (0.0006) [2023-12-26 18:48:34,472][105692] Updated weights for policy 0, policy_version 473147 (0.0005) [2023-12-26 18:48:34,770][105620] Updated weights for policy 1, policy_version 473520 (0.0009) [2023-12-26 18:48:34,837][105620] Updated weights for policy 1, policy_version 473530 (0.0009) [2023-12-26 18:48:34,888][105620] Updated weights for policy 1, policy_version 473540 (0.0009) [2023-12-26 18:48:35,081][105692] Updated weights for policy 0, policy_version 473157 (0.0006) [2023-12-26 18:48:35,153][105692] Updated weights for policy 0, policy_version 473167 (0.0008) [2023-12-26 18:48:35,216][105692] Updated weights for policy 0, policy_version 473177 (0.0009) [2023-12-26 18:48:35,633][105620] Updated weights for policy 1, policy_version 473550 (0.0007) [2023-12-26 18:48:35,681][105620] Updated weights for policy 1, policy_version 473560 (0.0005) [2023-12-26 18:48:35,732][105620] Updated weights for policy 1, policy_version 473570 (0.0005) [2023-12-26 18:48:35,951][105692] Updated weights for policy 0, policy_version 473187 (0.0009) [2023-12-26 18:48:36,008][105692] Updated weights for policy 0, policy_version 473197 (0.0010) [2023-12-26 18:48:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19114.7, 300 sec: 19438.6). Total num frames: 242401280. Throughput: 0: 9473.8, 1: 9714.3. Samples: 242392912. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-26 18:48:36,063][104569] Avg episode reward: [(0, '8819.966'), (1, '9176.591')] [2023-12-26 18:48:36,066][105692] Updated weights for policy 0, policy_version 473207 (0.0010) [2023-12-26 18:48:36,348][105620] Updated weights for policy 1, policy_version 473580 (0.0005) [2023-12-26 18:48:36,408][105620] Updated weights for policy 1, policy_version 473590 (0.0005) [2023-12-26 18:48:36,477][105620] Updated weights for policy 1, policy_version 473600 (0.0006) [2023-12-26 18:48:36,726][105692] Updated weights for policy 0, policy_version 473217 (0.0010) [2023-12-26 18:48:36,785][105692] Updated weights for policy 0, policy_version 473227 (0.0011) [2023-12-26 18:48:36,840][105692] Updated weights for policy 0, policy_version 473237 (0.0011) [2023-12-26 18:48:36,913][105692] Updated weights for policy 0, policy_version 473247 (0.0005) [2023-12-26 18:48:37,067][105620] Updated weights for policy 1, policy_version 473610 (0.0010) [2023-12-26 18:48:37,123][105620] Updated weights for policy 1, policy_version 473620 (0.0010) [2023-12-26 18:48:37,181][105620] Updated weights for policy 1, policy_version 473630 (0.0010) [2023-12-26 18:48:37,226][105620] Updated weights for policy 1, policy_version 473640 (0.0010) [2023-12-26 18:48:37,565][105692] Updated weights for policy 0, policy_version 473257 (0.0007) [2023-12-26 18:48:37,622][105692] Updated weights for policy 0, policy_version 473267 (0.0006) [2023-12-26 18:48:37,682][105692] Updated weights for policy 0, policy_version 473277 (0.0006) [2023-12-26 18:48:37,833][105620] Updated weights for policy 1, policy_version 473650 (0.0005) [2023-12-26 18:48:37,888][105620] Updated weights for policy 1, policy_version 473660 (0.0005) [2023-12-26 18:48:37,941][105620] Updated weights for policy 1, policy_version 473670 (0.0005) [2023-12-26 18:48:38,303][105692] Updated weights for policy 0, policy_version 473287 (0.0007) [2023-12-26 18:48:38,368][105692] Updated weights for policy 0, policy_version 473297 (0.0008) [2023-12-26 18:48:38,434][105692] Updated weights for policy 0, policy_version 473307 (0.0011) [2023-12-26 18:48:38,500][105620] Updated weights for policy 1, policy_version 473680 (0.0009) [2023-12-26 18:48:38,545][105620] Updated weights for policy 1, policy_version 473690 (0.0010) [2023-12-26 18:48:38,600][105620] Updated weights for policy 1, policy_version 473700 (0.0010) [2023-12-26 18:48:39,146][105692] Updated weights for policy 0, policy_version 473317 (0.0009) [2023-12-26 18:48:39,197][105692] Updated weights for policy 0, policy_version 473327 (0.0010) [2023-12-26 18:48:39,262][105692] Updated weights for policy 0, policy_version 473337 (0.0009) [2023-12-26 18:48:39,354][105620] Updated weights for policy 1, policy_version 473710 (0.0009) [2023-12-26 18:48:39,420][105620] Updated weights for policy 1, policy_version 473720 (0.0009) [2023-12-26 18:48:39,485][105620] Updated weights for policy 1, policy_version 473730 (0.0009) [2023-12-26 18:48:39,996][105692] Updated weights for policy 0, policy_version 473347 (0.0010) [2023-12-26 18:48:40,062][105692] Updated weights for policy 0, policy_version 473357 (0.0010) [2023-12-26 18:48:40,124][105692] Updated weights for policy 0, policy_version 473367 (0.0006) [2023-12-26 18:48:40,277][105620] Updated weights for policy 1, policy_version 473740 (0.0009) [2023-12-26 18:48:40,326][105620] Updated weights for policy 1, policy_version 473750 (0.0011) [2023-12-26 18:48:40,393][105620] Updated weights for policy 1, policy_version 473760 (0.0011) [2023-12-26 18:48:40,738][105692] Updated weights for policy 0, policy_version 473377 (0.0006) [2023-12-26 18:48:40,793][105692] Updated weights for policy 0, policy_version 473387 (0.0010) [2023-12-26 18:48:40,845][105692] Updated weights for policy 0, policy_version 473397 (0.0010) [2023-12-26 18:48:40,897][105692] Updated weights for policy 0, policy_version 473407 (0.0010) [2023-12-26 18:48:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 242507776. Throughput: 0: 9536.4, 1: 9864.5. Samples: 242514952. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-26 18:48:41,062][104569] Avg episode reward: [(0, '8130.554'), (1, '9176.485')] [2023-12-26 18:48:41,160][105620] Updated weights for policy 1, policy_version 473770 (0.0011) [2023-12-26 18:48:41,231][105620] Updated weights for policy 1, policy_version 473780 (0.0010) [2023-12-26 18:48:41,299][105620] Updated weights for policy 1, policy_version 473790 (0.0009) [2023-12-26 18:48:41,362][105620] Updated weights for policy 1, policy_version 473800 (0.0007) [2023-12-26 18:48:41,696][105692] Updated weights for policy 0, policy_version 473417 (0.0008) [2023-12-26 18:48:41,713][105585] KL-divergence is very high: 122.4857 [2023-12-26 18:48:41,720][105585] KL-divergence is very high: 176.5987 [2023-12-26 18:48:41,727][105585] KL-divergence is very high: 222.4964 [2023-12-26 18:48:41,734][105585] KL-divergence is very high: 241.1298 [2023-12-26 18:48:41,741][105585] KL-divergence is very high: 224.9570 [2023-12-26 18:48:41,748][105585] KL-divergence is very high: 170.6118 [2023-12-26 18:48:41,756][105585] KL-divergence is very high: 228.2763 [2023-12-26 18:48:41,763][105585] KL-divergence is very high: 179.6001 [2023-12-26 18:48:41,771][105585] KL-divergence is very high: 186.2883 [2023-12-26 18:48:41,771][105692] Updated weights for policy 0, policy_version 473427 (0.0008) [2023-12-26 18:48:41,777][105585] KL-divergence is very high: 156.1812 [2023-12-26 18:48:41,782][105585] KL-divergence is very high: 139.6126 [2023-12-26 18:48:41,789][105585] KL-divergence is very high: 117.6241 [2023-12-26 18:48:41,827][105692] Updated weights for policy 0, policy_version 473437 (0.0008) [2023-12-26 18:48:41,991][105620] Updated weights for policy 1, policy_version 473810 (0.0011) [2023-12-26 18:48:42,055][105620] Updated weights for policy 1, policy_version 473820 (0.0008) [2023-12-26 18:48:42,124][105620] Updated weights for policy 1, policy_version 473830 (0.0006) [2023-12-26 18:48:42,603][105692] Updated weights for policy 0, policy_version 473447 (0.0010) [2023-12-26 18:48:42,671][105692] Updated weights for policy 0, policy_version 473457 (0.0011) [2023-12-26 18:48:42,731][105692] Updated weights for policy 0, policy_version 473467 (0.0011) [2023-12-26 18:48:42,873][105620] Updated weights for policy 1, policy_version 473840 (0.0008) [2023-12-26 18:48:42,925][105620] Updated weights for policy 1, policy_version 473850 (0.0009) [2023-12-26 18:48:42,990][105620] Updated weights for policy 1, policy_version 473860 (0.0008) [2023-12-26 18:48:43,480][105692] Updated weights for policy 0, policy_version 473477 (0.0010) [2023-12-26 18:48:43,543][105692] Updated weights for policy 0, policy_version 473487 (0.0010) [2023-12-26 18:48:43,598][105692] Updated weights for policy 0, policy_version 473497 (0.0010) [2023-12-26 18:48:43,749][105620] Updated weights for policy 1, policy_version 473870 (0.0008) [2023-12-26 18:48:43,797][105620] Updated weights for policy 1, policy_version 473880 (0.0009) [2023-12-26 18:48:43,848][105620] Updated weights for policy 1, policy_version 473890 (0.0009) [2023-12-26 18:48:44,333][105692] Updated weights for policy 0, policy_version 473507 (0.0010) [2023-12-26 18:48:44,399][105692] Updated weights for policy 0, policy_version 473517 (0.0009) [2023-12-26 18:48:44,453][105692] Updated weights for policy 0, policy_version 473527 (0.0009) [2023-12-26 18:48:44,617][105620] Updated weights for policy 1, policy_version 473900 (0.0009) [2023-12-26 18:48:44,699][105620] Updated weights for policy 1, policy_version 473910 (0.0009) [2023-12-26 18:48:44,756][105620] Updated weights for policy 1, policy_version 473920 (0.0009) [2023-12-26 18:48:45,113][105692] Updated weights for policy 0, policy_version 473537 (0.0009) [2023-12-26 18:48:45,171][105692] Updated weights for policy 0, policy_version 473547 (0.0006) [2023-12-26 18:48:45,230][105692] Updated weights for policy 0, policy_version 473557 (0.0006) [2023-12-26 18:48:45,281][105692] Updated weights for policy 0, policy_version 473567 (0.0006) [2023-12-26 18:48:45,568][105620] Updated weights for policy 1, policy_version 473930 (0.0009) [2023-12-26 18:48:45,635][105620] Updated weights for policy 1, policy_version 473940 (0.0011) [2023-12-26 18:48:45,695][105620] Updated weights for policy 1, policy_version 473950 (0.0011) [2023-12-26 18:48:45,751][105620] Updated weights for policy 1, policy_version 473960 (0.0011) [2023-12-26 18:48:45,926][105692] Updated weights for policy 0, policy_version 473577 (0.0007) [2023-12-26 18:48:45,983][105692] Updated weights for policy 0, policy_version 473587 (0.0008) [2023-12-26 18:48:46,035][105692] Updated weights for policy 0, policy_version 473597 (0.0010) [2023-12-26 18:48:46,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 242606080. Throughput: 0: 9531.5, 1: 9892.1. Samples: 242571280. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-26 18:48:46,062][104569] Avg episode reward: [(0, '7427.457'), (1, '9176.333')] [2023-12-26 18:48:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000473600_121257984.pth... [2023-12-26 18:48:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000473960_121348096.pth... [2023-12-26 18:48:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000472808_121053184.pth [2023-12-26 18:48:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000472448_120963072.pth [2023-12-26 18:48:46,453][105620] Updated weights for policy 1, policy_version 473970 (0.0010) [2023-12-26 18:48:46,504][105620] Updated weights for policy 1, policy_version 473980 (0.0010) [2023-12-26 18:48:46,563][105620] Updated weights for policy 1, policy_version 473990 (0.0010) [2023-12-26 18:48:46,664][105692] Updated weights for policy 0, policy_version 473607 (0.0009) [2023-12-26 18:48:46,716][105692] Updated weights for policy 0, policy_version 473617 (0.0010) [2023-12-26 18:48:46,770][105692] Updated weights for policy 0, policy_version 473627 (0.0010) [2023-12-26 18:48:47,216][105620] Updated weights for policy 1, policy_version 474000 (0.0010) [2023-12-26 18:48:47,274][105620] Updated weights for policy 1, policy_version 474010 (0.0010) [2023-12-26 18:48:47,332][105620] Updated weights for policy 1, policy_version 474020 (0.0010) [2023-12-26 18:48:47,393][105692] Updated weights for policy 0, policy_version 473637 (0.0010) [2023-12-26 18:48:47,441][105692] Updated weights for policy 0, policy_version 473647 (0.0010) [2023-12-26 18:48:47,485][105692] Updated weights for policy 0, policy_version 473657 (0.0010) [2023-12-26 18:48:47,976][105620] Updated weights for policy 1, policy_version 474030 (0.0005) [2023-12-26 18:48:48,044][105620] Updated weights for policy 1, policy_version 474040 (0.0008) [2023-12-26 18:48:48,107][105620] Updated weights for policy 1, policy_version 474050 (0.0011) [2023-12-26 18:48:48,161][105692] Updated weights for policy 0, policy_version 473667 (0.0009) [2023-12-26 18:48:48,216][105692] Updated weights for policy 0, policy_version 473677 (0.0008) [2023-12-26 18:48:48,278][105692] Updated weights for policy 0, policy_version 473687 (0.0006) [2023-12-26 18:48:48,761][105620] Updated weights for policy 1, policy_version 474060 (0.0011) [2023-12-26 18:48:48,827][105620] Updated weights for policy 1, policy_version 474070 (0.0011) [2023-12-26 18:48:48,894][105620] Updated weights for policy 1, policy_version 474080 (0.0011) [2023-12-26 18:48:49,049][105692] Updated weights for policy 0, policy_version 473697 (0.0007) [2023-12-26 18:48:49,104][105692] Updated weights for policy 0, policy_version 473707 (0.0010) [2023-12-26 18:48:49,155][105692] Updated weights for policy 0, policy_version 473717 (0.0010) [2023-12-26 18:48:49,217][105692] Updated weights for policy 0, policy_version 473727 (0.0010) [2023-12-26 18:48:49,618][105620] Updated weights for policy 1, policy_version 474090 (0.0010) [2023-12-26 18:48:49,669][105620] Updated weights for policy 1, policy_version 474100 (0.0007) [2023-12-26 18:48:49,714][105620] Updated weights for policy 1, policy_version 474110 (0.0009) [2023-12-26 18:48:49,778][105620] Updated weights for policy 1, policy_version 474120 (0.0009) [2023-12-26 18:48:50,026][105692] Updated weights for policy 0, policy_version 473737 (0.0008) [2023-12-26 18:48:50,087][105692] Updated weights for policy 0, policy_version 473747 (0.0010) [2023-12-26 18:48:50,145][105692] Updated weights for policy 0, policy_version 473757 (0.0010) [2023-12-26 18:48:50,475][105620] Updated weights for policy 1, policy_version 474130 (0.0011) [2023-12-26 18:48:50,525][105620] Updated weights for policy 1, policy_version 474140 (0.0011) [2023-12-26 18:48:50,581][105620] Updated weights for policy 1, policy_version 474150 (0.0010) [2023-12-26 18:48:50,856][105692] Updated weights for policy 0, policy_version 473767 (0.0009) [2023-12-26 18:48:50,901][105692] Updated weights for policy 0, policy_version 473777 (0.0008) [2023-12-26 18:48:50,962][105692] Updated weights for policy 0, policy_version 473787 (0.0006) [2023-12-26 18:48:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 242704384. Throughput: 0: 9651.9, 1: 9892.6. Samples: 242691048. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-26 18:48:51,063][104569] Avg episode reward: [(0, '7315.752'), (1, '9355.670')] [2023-12-26 18:48:51,408][105620] Updated weights for policy 1, policy_version 474160 (0.0009) [2023-12-26 18:48:51,467][105620] Updated weights for policy 1, policy_version 474170 (0.0009) [2023-12-26 18:48:51,519][105620] Updated weights for policy 1, policy_version 474180 (0.0009) [2023-12-26 18:48:51,595][105692] Updated weights for policy 0, policy_version 473797 (0.0006) [2023-12-26 18:48:51,658][105692] Updated weights for policy 0, policy_version 473807 (0.0008) [2023-12-26 18:48:51,729][105692] Updated weights for policy 0, policy_version 473817 (0.0009) [2023-12-26 18:48:52,237][105620] Updated weights for policy 1, policy_version 474190 (0.0007) [2023-12-26 18:48:52,296][105620] Updated weights for policy 1, policy_version 474200 (0.0009) [2023-12-26 18:48:52,354][105620] Updated weights for policy 1, policy_version 474210 (0.0010) [2023-12-26 18:48:52,447][105692] Updated weights for policy 0, policy_version 473827 (0.0008) [2023-12-26 18:48:52,510][105692] Updated weights for policy 0, policy_version 473837 (0.0006) [2023-12-26 18:48:52,572][105692] Updated weights for policy 0, policy_version 473847 (0.0009) [2023-12-26 18:48:53,046][105620] Updated weights for policy 1, policy_version 474220 (0.0008) [2023-12-26 18:48:53,099][105620] Updated weights for policy 1, policy_version 474230 (0.0005) [2023-12-26 18:48:53,155][105620] Updated weights for policy 1, policy_version 474240 (0.0005) [2023-12-26 18:48:53,348][105692] Updated weights for policy 0, policy_version 473857 (0.0009) [2023-12-26 18:48:53,405][105692] Updated weights for policy 0, policy_version 473867 (0.0009) [2023-12-26 18:48:53,463][105692] Updated weights for policy 0, policy_version 473877 (0.0010) [2023-12-26 18:48:53,519][105692] Updated weights for policy 0, policy_version 473888 (0.0009) [2023-12-26 18:48:53,729][105620] Updated weights for policy 1, policy_version 474250 (0.0008) [2023-12-26 18:48:53,789][105620] Updated weights for policy 1, policy_version 474261 (0.0011) [2023-12-26 18:48:53,845][105620] Updated weights for policy 1, policy_version 474272 (0.0012) [2023-12-26 18:48:54,237][105692] Updated weights for policy 0, policy_version 473898 (0.0009) [2023-12-26 18:48:54,297][105692] Updated weights for policy 0, policy_version 473908 (0.0008) [2023-12-26 18:48:54,353][105692] Updated weights for policy 0, policy_version 473918 (0.0008) [2023-12-26 18:48:54,631][105620] Updated weights for policy 1, policy_version 474282 (0.0008) [2023-12-26 18:48:54,684][105620] Updated weights for policy 1, policy_version 474292 (0.0010) [2023-12-26 18:48:54,729][105620] Updated weights for policy 1, policy_version 474302 (0.0010) [2023-12-26 18:48:54,779][105620] Updated weights for policy 1, policy_version 474312 (0.0010) [2023-12-26 18:48:55,009][105692] Updated weights for policy 0, policy_version 473928 (0.0009) [2023-12-26 18:48:55,064][105692] Updated weights for policy 0, policy_version 473938 (0.0010) [2023-12-26 18:48:55,120][105692] Updated weights for policy 0, policy_version 473948 (0.0005) [2023-12-26 18:48:55,438][105620] Updated weights for policy 1, policy_version 474322 (0.0008) [2023-12-26 18:48:55,496][105620] Updated weights for policy 1, policy_version 474332 (0.0007) [2023-12-26 18:48:55,561][105620] Updated weights for policy 1, policy_version 474342 (0.0008) [2023-12-26 18:48:55,864][105692] Updated weights for policy 0, policy_version 473958 (0.0009) [2023-12-26 18:48:55,913][105692] Updated weights for policy 0, policy_version 473968 (0.0009) [2023-12-26 18:48:55,962][105692] Updated weights for policy 0, policy_version 473978 (0.0008) [2023-12-26 18:48:56,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 242802688. Throughput: 0: 9668.9, 1: 9882.0. Samples: 242809188. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-26 18:48:56,063][104569] Avg episode reward: [(0, '8548.701'), (1, '9356.047')] [2023-12-26 18:48:56,227][105620] Updated weights for policy 1, policy_version 474352 (0.0009) [2023-12-26 18:48:56,277][105620] Updated weights for policy 1, policy_version 474362 (0.0007) [2023-12-26 18:48:56,343][105620] Updated weights for policy 1, policy_version 474372 (0.0009) [2023-12-26 18:48:56,666][105692] Updated weights for policy 0, policy_version 473988 (0.0008) [2023-12-26 18:48:56,710][105692] Updated weights for policy 0, policy_version 473998 (0.0006) [2023-12-26 18:48:56,772][105692] Updated weights for policy 0, policy_version 474008 (0.0005) [2023-12-26 18:48:57,189][105620] Updated weights for policy 1, policy_version 474382 (0.0007) [2023-12-26 18:48:57,235][105620] Updated weights for policy 1, policy_version 474392 (0.0008) [2023-12-26 18:48:57,282][105620] Updated weights for policy 1, policy_version 474402 (0.0009) [2023-12-26 18:48:57,332][105692] Updated weights for policy 0, policy_version 474018 (0.0005) [2023-12-26 18:48:57,391][105692] Updated weights for policy 0, policy_version 474028 (0.0006) [2023-12-26 18:48:57,442][105692] Updated weights for policy 0, policy_version 474038 (0.0009) [2023-12-26 18:48:58,073][105692] Updated weights for policy 0, policy_version 474049 (0.0009) [2023-12-26 18:48:58,128][105620] Updated weights for policy 1, policy_version 474412 (0.0009) [2023-12-26 18:48:58,133][105692] Updated weights for policy 0, policy_version 474059 (0.0007) [2023-12-26 18:48:58,190][105620] Updated weights for policy 1, policy_version 474422 (0.0008) [2023-12-26 18:48:58,192][105692] Updated weights for policy 0, policy_version 474069 (0.0008) [2023-12-26 18:48:58,242][105692] Updated weights for policy 0, policy_version 474079 (0.0008) [2023-12-26 18:48:58,251][105620] Updated weights for policy 1, policy_version 474432 (0.0010) [2023-12-26 18:48:59,085][105620] Updated weights for policy 1, policy_version 474442 (0.0010) [2023-12-26 18:48:59,126][105692] Updated weights for policy 0, policy_version 474089 (0.0009) [2023-12-26 18:48:59,140][105620] Updated weights for policy 1, policy_version 474452 (0.0009) [2023-12-26 18:48:59,178][105692] Updated weights for policy 0, policy_version 474099 (0.0009) [2023-12-26 18:48:59,196][105620] Updated weights for policy 1, policy_version 474462 (0.0009) [2023-12-26 18:48:59,233][105692] Updated weights for policy 0, policy_version 474110 (0.0008) [2023-12-26 18:48:59,262][105620] Updated weights for policy 1, policy_version 474472 (0.0010) [2023-12-26 18:48:59,914][105620] Updated weights for policy 1, policy_version 474482 (0.0010) [2023-12-26 18:48:59,972][105620] Updated weights for policy 1, policy_version 474492 (0.0009) [2023-12-26 18:48:59,996][105692] Updated weights for policy 0, policy_version 474120 (0.0007) [2023-12-26 18:49:00,031][105620] Updated weights for policy 1, policy_version 474502 (0.0006) [2023-12-26 18:49:00,060][105692] Updated weights for policy 0, policy_version 474130 (0.0006) [2023-12-26 18:49:00,117][105692] Updated weights for policy 0, policy_version 474140 (0.0006) [2023-12-26 18:49:00,684][105620] Updated weights for policy 1, policy_version 474512 (0.0005) [2023-12-26 18:49:00,735][105620] Updated weights for policy 1, policy_version 474522 (0.0005) [2023-12-26 18:49:00,785][105620] Updated weights for policy 1, policy_version 474532 (0.0005) [2023-12-26 18:49:00,792][105692] Updated weights for policy 0, policy_version 474150 (0.0007) [2023-12-26 18:49:00,850][105692] Updated weights for policy 0, policy_version 474160 (0.0009) [2023-12-26 18:49:00,908][105692] Updated weights for policy 0, policy_version 474170 (0.0010) [2023-12-26 18:49:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 242900992. Throughput: 0: 9732.4, 1: 9877.5. Samples: 242866820. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-26 18:49:01,063][104569] Avg episode reward: [(0, '8640.584'), (1, '9356.560')] [2023-12-26 18:49:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000474176_121405440.pth... [2023-12-26 18:49:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000474536_121495552.pth... [2023-12-26 18:49:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000473024_121110528.pth [2023-12-26 18:49:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000473384_121200640.pth [2023-12-26 18:49:01,405][105620] Updated weights for policy 1, policy_version 474542 (0.0007) [2023-12-26 18:49:01,458][105620] Updated weights for policy 1, policy_version 474552 (0.0008) [2023-12-26 18:49:01,513][105620] Updated weights for policy 1, policy_version 474562 (0.0010) [2023-12-26 18:49:01,691][105692] Updated weights for policy 0, policy_version 474180 (0.0008) [2023-12-26 18:49:01,750][105692] Updated weights for policy 0, policy_version 474190 (0.0008) [2023-12-26 18:49:01,795][105692] Updated weights for policy 0, policy_version 474200 (0.0006) [2023-12-26 18:49:02,333][105692] Updated weights for policy 0, policy_version 474210 (0.0005) [2023-12-26 18:49:02,359][105620] Updated weights for policy 1, policy_version 474572 (0.0009) [2023-12-26 18:49:02,392][105692] Updated weights for policy 0, policy_version 474220 (0.0010) [2023-12-26 18:49:02,420][105620] Updated weights for policy 1, policy_version 474582 (0.0007) [2023-12-26 18:49:02,451][105692] Updated weights for policy 0, policy_version 474230 (0.0010) [2023-12-26 18:49:02,474][105620] Updated weights for policy 1, policy_version 474592 (0.0005) [2023-12-26 18:49:02,508][105692] Updated weights for policy 0, policy_version 474240 (0.0008) [2023-12-26 18:49:03,203][105692] Updated weights for policy 0, policy_version 474250 (0.0010) [2023-12-26 18:49:03,212][105620] Updated weights for policy 1, policy_version 474602 (0.0010) [2023-12-26 18:49:03,260][105692] Updated weights for policy 0, policy_version 474260 (0.0009) [2023-12-26 18:49:03,269][105620] Updated weights for policy 1, policy_version 474612 (0.0006) [2023-12-26 18:49:03,314][105692] Updated weights for policy 0, policy_version 474270 (0.0009) [2023-12-26 18:49:03,317][105620] Updated weights for policy 1, policy_version 474622 (0.0005) [2023-12-26 18:49:03,370][105620] Updated weights for policy 1, policy_version 474632 (0.0005) [2023-12-26 18:49:04,006][105620] Updated weights for policy 1, policy_version 474642 (0.0011) [2023-12-26 18:49:04,066][105620] Updated weights for policy 1, policy_version 474652 (0.0011) [2023-12-26 18:49:04,122][105692] Updated weights for policy 0, policy_version 474280 (0.0007) [2023-12-26 18:49:04,131][105620] Updated weights for policy 1, policy_version 474662 (0.0009) [2023-12-26 18:49:04,185][105692] Updated weights for policy 0, policy_version 474290 (0.0008) [2023-12-26 18:49:04,252][105692] Updated weights for policy 0, policy_version 474300 (0.0008) [2023-12-26 18:49:04,792][105620] Updated weights for policy 1, policy_version 474672 (0.0007) [2023-12-26 18:49:04,839][105620] Updated weights for policy 1, policy_version 474682 (0.0005) [2023-12-26 18:49:04,896][105620] Updated weights for policy 1, policy_version 474692 (0.0006) [2023-12-26 18:49:04,984][105692] Updated weights for policy 0, policy_version 474310 (0.0007) [2023-12-26 18:49:05,038][105692] Updated weights for policy 0, policy_version 474320 (0.0005) [2023-12-26 18:49:05,092][105692] Updated weights for policy 0, policy_version 474330 (0.0005) [2023-12-26 18:49:05,450][105620] Updated weights for policy 1, policy_version 474703 (0.0007) [2023-12-26 18:49:05,501][105620] Updated weights for policy 1, policy_version 474713 (0.0005) [2023-12-26 18:49:05,557][105620] Updated weights for policy 1, policy_version 474723 (0.0009) [2023-12-26 18:49:05,645][105692] Updated weights for policy 0, policy_version 474340 (0.0006) [2023-12-26 18:49:05,698][105692] Updated weights for policy 0, policy_version 474350 (0.0005) [2023-12-26 18:49:05,751][105692] Updated weights for policy 0, policy_version 474360 (0.0005) [2023-12-26 18:49:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 242999296. Throughput: 0: 9692.2, 1: 9920.7. Samples: 242984968. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-26 18:49:06,063][104569] Avg episode reward: [(0, '8731.224'), (1, '9353.199')] [2023-12-26 18:49:06,256][105620] Updated weights for policy 1, policy_version 474733 (0.0008) [2023-12-26 18:49:06,308][105620] Updated weights for policy 1, policy_version 474743 (0.0007) [2023-12-26 18:49:06,368][105620] Updated weights for policy 1, policy_version 474753 (0.0010) [2023-12-26 18:49:06,465][105692] Updated weights for policy 0, policy_version 474370 (0.0006) [2023-12-26 18:49:06,518][105692] Updated weights for policy 0, policy_version 474380 (0.0008) [2023-12-26 18:49:06,571][105692] Updated weights for policy 0, policy_version 474390 (0.0007) [2023-12-26 18:49:06,628][105692] Updated weights for policy 0, policy_version 474400 (0.0008) [2023-12-26 18:49:07,052][105620] Updated weights for policy 1, policy_version 474763 (0.0010) [2023-12-26 18:49:07,116][105620] Updated weights for policy 1, policy_version 474773 (0.0010) [2023-12-26 18:49:07,172][105620] Updated weights for policy 1, policy_version 474783 (0.0010) [2023-12-26 18:49:07,358][105692] Updated weights for policy 0, policy_version 474410 (0.0008) [2023-12-26 18:49:07,419][105692] Updated weights for policy 0, policy_version 474420 (0.0009) [2023-12-26 18:49:07,484][105692] Updated weights for policy 0, policy_version 474430 (0.0010) [2023-12-26 18:49:07,872][105620] Updated weights for policy 1, policy_version 474793 (0.0009) [2023-12-26 18:49:07,934][105620] Updated weights for policy 1, policy_version 474803 (0.0009) [2023-12-26 18:49:07,996][105620] Updated weights for policy 1, policy_version 474813 (0.0009) [2023-12-26 18:49:08,047][105620] Updated weights for policy 1, policy_version 474823 (0.0009) [2023-12-26 18:49:08,179][105692] Updated weights for policy 0, policy_version 474440 (0.0009) [2023-12-26 18:49:08,247][105692] Updated weights for policy 0, policy_version 474450 (0.0009) [2023-12-26 18:49:08,268][105585] KL-divergence is very high: 124.8388 [2023-12-26 18:49:08,304][105692] Updated weights for policy 0, policy_version 474460 (0.0008) [2023-12-26 18:49:08,315][105585] KL-divergence is very high: 138.8264 [2023-12-26 18:49:08,814][105620] Updated weights for policy 1, policy_version 474833 (0.0005) [2023-12-26 18:49:08,866][105620] Updated weights for policy 1, policy_version 474843 (0.0007) [2023-12-26 18:49:08,917][105620] Updated weights for policy 1, policy_version 474853 (0.0006) [2023-12-26 18:49:09,010][105692] Updated weights for policy 0, policy_version 474470 (0.0010) [2023-12-26 18:49:09,075][105692] Updated weights for policy 0, policy_version 474480 (0.0010) [2023-12-26 18:49:09,141][105692] Updated weights for policy 0, policy_version 474490 (0.0011) [2023-12-26 18:49:09,637][105620] Updated weights for policy 1, policy_version 474863 (0.0009) [2023-12-26 18:49:09,693][105620] Updated weights for policy 1, policy_version 474873 (0.0011) [2023-12-26 18:49:09,749][105620] Updated weights for policy 1, policy_version 474883 (0.0011) [2023-12-26 18:49:09,945][105692] Updated weights for policy 0, policy_version 474500 (0.0011) [2023-12-26 18:49:10,000][105692] Updated weights for policy 0, policy_version 474510 (0.0009) [2023-12-26 18:49:10,052][105692] Updated weights for policy 0, policy_version 474520 (0.0007) [2023-12-26 18:49:10,533][105620] Updated weights for policy 1, policy_version 474893 (0.0009) [2023-12-26 18:49:10,581][105620] Updated weights for policy 1, policy_version 474903 (0.0008) [2023-12-26 18:49:10,638][105620] Updated weights for policy 1, policy_version 474913 (0.0008) [2023-12-26 18:49:10,795][105692] Updated weights for policy 0, policy_version 474530 (0.0009) [2023-12-26 18:49:10,843][105692] Updated weights for policy 0, policy_version 474540 (0.0010) [2023-12-26 18:49:10,895][105692] Updated weights for policy 0, policy_version 474550 (0.0010) [2023-12-26 18:49:10,943][105692] Updated weights for policy 0, policy_version 474560 (0.0010) [2023-12-26 18:49:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 243097600. Throughput: 0: 9771.8, 1: 9961.0. Samples: 243103612. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-26 18:49:11,063][104569] Avg episode reward: [(0, '8826.980'), (1, '9349.687')] [2023-12-26 18:49:11,445][105620] Updated weights for policy 1, policy_version 474923 (0.0008) [2023-12-26 18:49:11,505][105620] Updated weights for policy 1, policy_version 474933 (0.0009) [2023-12-26 18:49:11,569][105620] Updated weights for policy 1, policy_version 474943 (0.0008) [2023-12-26 18:49:11,758][105692] Updated weights for policy 0, policy_version 474570 (0.0009) [2023-12-26 18:49:11,819][105692] Updated weights for policy 0, policy_version 474580 (0.0008) [2023-12-26 18:49:11,868][105692] Updated weights for policy 0, policy_version 474590 (0.0009) [2023-12-26 18:49:12,314][105620] Updated weights for policy 1, policy_version 474953 (0.0009) [2023-12-26 18:49:12,379][105620] Updated weights for policy 1, policy_version 474963 (0.0008) [2023-12-26 18:49:12,442][105620] Updated weights for policy 1, policy_version 474973 (0.0006) [2023-12-26 18:49:12,509][105620] Updated weights for policy 1, policy_version 474983 (0.0006) [2023-12-26 18:49:12,668][105692] Updated weights for policy 0, policy_version 474600 (0.0010) [2023-12-26 18:49:12,724][105692] Updated weights for policy 0, policy_version 474610 (0.0010) [2023-12-26 18:49:12,782][105692] Updated weights for policy 0, policy_version 474620 (0.0010) [2023-12-26 18:49:13,135][105620] Updated weights for policy 1, policy_version 474993 (0.0010) [2023-12-26 18:49:13,194][105620] Updated weights for policy 1, policy_version 475003 (0.0010) [2023-12-26 18:49:13,257][105620] Updated weights for policy 1, policy_version 475013 (0.0010) [2023-12-26 18:49:13,427][105692] Updated weights for policy 0, policy_version 474630 (0.0008) [2023-12-26 18:49:13,492][105692] Updated weights for policy 0, policy_version 474640 (0.0006) [2023-12-26 18:49:13,559][105692] Updated weights for policy 0, policy_version 474650 (0.0009) [2023-12-26 18:49:14,071][105692] Updated weights for policy 0, policy_version 474660 (0.0006) [2023-12-26 18:49:14,126][105692] Updated weights for policy 0, policy_version 474670 (0.0008) [2023-12-26 18:49:14,171][105620] Updated weights for policy 1, policy_version 475023 (0.0007) [2023-12-26 18:49:14,185][105692] Updated weights for policy 0, policy_version 474680 (0.0010) [2023-12-26 18:49:14,229][105620] Updated weights for policy 1, policy_version 475033 (0.0005) [2023-12-26 18:49:14,291][105620] Updated weights for policy 1, policy_version 475043 (0.0008) [2023-12-26 18:49:14,931][105692] Updated weights for policy 0, policy_version 474690 (0.0010) [2023-12-26 18:49:14,995][105692] Updated weights for policy 0, policy_version 474700 (0.0011) [2023-12-26 18:49:15,058][105692] Updated weights for policy 0, policy_version 474710 (0.0011) [2023-12-26 18:49:15,077][105620] Updated weights for policy 1, policy_version 475053 (0.0009) [2023-12-26 18:49:15,108][105692] Updated weights for policy 0, policy_version 474720 (0.0010) [2023-12-26 18:49:15,139][105620] Updated weights for policy 1, policy_version 475063 (0.0007) [2023-12-26 18:49:15,203][105620] Updated weights for policy 1, policy_version 475073 (0.0008) [2023-12-26 18:49:15,858][105692] Updated weights for policy 0, policy_version 474730 (0.0010) [2023-12-26 18:49:15,919][105692] Updated weights for policy 0, policy_version 474740 (0.0010) [2023-12-26 18:49:15,929][105620] Updated weights for policy 1, policy_version 475083 (0.0008) [2023-12-26 18:49:15,970][105692] Updated weights for policy 0, policy_version 474750 (0.0010) [2023-12-26 18:49:15,984][105620] Updated weights for policy 1, policy_version 475093 (0.0006) [2023-12-26 18:49:16,037][105620] Updated weights for policy 1, policy_version 475103 (0.0008) [2023-12-26 18:49:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 243187712. Throughput: 0: 9781.8, 1: 9858.1. Samples: 243159692. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-26 18:49:16,063][104569] Avg episode reward: [(0, '9183.535'), (1, '9349.339')] [2023-12-26 18:49:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000474752_121552896.pth... [2023-12-26 18:49:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000473600_121257984.pth [2023-12-26 18:49:16,086][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000475112_121643008.pth... [2023-12-26 18:49:16,089][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000473960_121348096.pth [2023-12-26 18:49:16,633][105692] Updated weights for policy 0, policy_version 474760 (0.0008) [2023-12-26 18:49:16,667][105620] Updated weights for policy 1, policy_version 475113 (0.0008) [2023-12-26 18:49:16,684][105692] Updated weights for policy 0, policy_version 474770 (0.0005) [2023-12-26 18:49:16,729][105620] Updated weights for policy 1, policy_version 475123 (0.0005) [2023-12-26 18:49:16,746][105692] Updated weights for policy 0, policy_version 474780 (0.0010) [2023-12-26 18:49:16,785][105620] Updated weights for policy 1, policy_version 475133 (0.0007) [2023-12-26 18:49:16,846][105620] Updated weights for policy 1, policy_version 475143 (0.0009) [2023-12-26 18:49:17,359][105620] Updated weights for policy 1, policy_version 475153 (0.0005) [2023-12-26 18:49:17,415][105620] Updated weights for policy 1, policy_version 475163 (0.0005) [2023-12-26 18:49:17,463][105692] Updated weights for policy 0, policy_version 474790 (0.0011) [2023-12-26 18:49:17,474][105620] Updated weights for policy 1, policy_version 475173 (0.0005) [2023-12-26 18:49:17,518][105692] Updated weights for policy 0, policy_version 474800 (0.0011) [2023-12-26 18:49:17,569][105692] Updated weights for policy 0, policy_version 474810 (0.0010) [2023-12-26 18:49:18,107][105620] Updated weights for policy 1, policy_version 475183 (0.0006) [2023-12-26 18:49:18,174][105620] Updated weights for policy 1, policy_version 475193 (0.0005) [2023-12-26 18:49:18,231][105620] Updated weights for policy 1, policy_version 475203 (0.0008) [2023-12-26 18:49:18,242][105692] Updated weights for policy 0, policy_version 474820 (0.0008) [2023-12-26 18:49:18,286][105692] Updated weights for policy 0, policy_version 474830 (0.0005) [2023-12-26 18:49:18,338][105692] Updated weights for policy 0, policy_version 474840 (0.0006) [2023-12-26 18:49:18,956][105620] Updated weights for policy 1, policy_version 475213 (0.0010) [2023-12-26 18:49:18,957][105692] Updated weights for policy 0, policy_version 474850 (0.0006) [2023-12-26 18:49:19,016][105692] Updated weights for policy 0, policy_version 474860 (0.0007) [2023-12-26 18:49:19,018][105620] Updated weights for policy 1, policy_version 475223 (0.0010) [2023-12-26 18:49:19,068][105692] Updated weights for policy 0, policy_version 474870 (0.0008) [2023-12-26 18:49:19,079][105620] Updated weights for policy 1, policy_version 475233 (0.0011) [2023-12-26 18:49:19,115][105692] Updated weights for policy 0, policy_version 474880 (0.0007) [2023-12-26 18:49:19,821][105620] Updated weights for policy 1, policy_version 475243 (0.0010) [2023-12-26 18:49:19,875][105692] Updated weights for policy 0, policy_version 474890 (0.0011) [2023-12-26 18:49:19,893][105620] Updated weights for policy 1, policy_version 475253 (0.0009) [2023-12-26 18:49:19,941][105692] Updated weights for policy 0, policy_version 474900 (0.0011) [2023-12-26 18:49:19,958][105620] Updated weights for policy 1, policy_version 475263 (0.0008) [2023-12-26 18:49:20,008][105692] Updated weights for policy 0, policy_version 474910 (0.0011) [2023-12-26 18:49:20,611][105620] Updated weights for policy 1, policy_version 475273 (0.0007) [2023-12-26 18:49:20,676][105620] Updated weights for policy 1, policy_version 475283 (0.0007) [2023-12-26 18:49:20,699][105692] Updated weights for policy 0, policy_version 474920 (0.0010) [2023-12-26 18:49:20,719][105585] KL-divergence is very high: 199.8013 [2023-12-26 18:49:20,741][105620] Updated weights for policy 1, policy_version 475293 (0.0006) [2023-12-26 18:49:20,756][105692] Updated weights for policy 0, policy_version 474930 (0.0011) [2023-12-26 18:49:20,761][105585] KL-divergence is very high: 283.1690 [2023-12-26 18:49:20,803][105620] Updated weights for policy 1, policy_version 475303 (0.0006) [2023-12-26 18:49:20,810][105585] KL-divergence is very high: 287.3820 [2023-12-26 18:49:20,816][105692] Updated weights for policy 0, policy_version 474940 (0.0011) [2023-12-26 18:49:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 243294208. Throughput: 0: 9921.3, 1: 9814.1. Samples: 243281008. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-26 18:49:21,063][104569] Avg episode reward: [(0, '9092.979'), (1, '9078.266')] [2023-12-26 18:49:21,475][105620] Updated weights for policy 1, policy_version 475313 (0.0011) [2023-12-26 18:49:21,498][105692] Updated weights for policy 0, policy_version 474950 (0.0009) [2023-12-26 18:49:21,533][105620] Updated weights for policy 1, policy_version 475323 (0.0011) [2023-12-26 18:49:21,565][105692] Updated weights for policy 0, policy_version 474960 (0.0009) [2023-12-26 18:49:21,593][105620] Updated weights for policy 1, policy_version 475333 (0.0011) [2023-12-26 18:49:21,631][105692] Updated weights for policy 0, policy_version 474970 (0.0012) [2023-12-26 18:49:22,304][105620] Updated weights for policy 1, policy_version 475343 (0.0011) [2023-12-26 18:49:22,369][105620] Updated weights for policy 1, policy_version 475353 (0.0008) [2023-12-26 18:49:22,380][105692] Updated weights for policy 0, policy_version 474980 (0.0012) [2023-12-26 18:49:22,435][105620] Updated weights for policy 1, policy_version 475363 (0.0008) [2023-12-26 18:49:22,443][105692] Updated weights for policy 0, policy_version 474990 (0.0010) [2023-12-26 18:49:22,513][105692] Updated weights for policy 0, policy_version 475000 (0.0010) [2023-12-26 18:49:23,132][105620] Updated weights for policy 1, policy_version 475373 (0.0008) [2023-12-26 18:49:23,194][105620] Updated weights for policy 1, policy_version 475383 (0.0010) [2023-12-26 18:49:23,250][105692] Updated weights for policy 0, policy_version 475010 (0.0009) [2023-12-26 18:49:23,252][105620] Updated weights for policy 1, policy_version 475393 (0.0010) [2023-12-26 18:49:23,307][105692] Updated weights for policy 0, policy_version 475020 (0.0006) [2023-12-26 18:49:23,367][105692] Updated weights for policy 0, policy_version 475030 (0.0008) [2023-12-26 18:49:23,433][105692] Updated weights for policy 0, policy_version 475040 (0.0008) [2023-12-26 18:49:23,988][105620] Updated weights for policy 1, policy_version 475403 (0.0010) [2023-12-26 18:49:24,032][105620] Updated weights for policy 1, policy_version 475413 (0.0010) [2023-12-26 18:49:24,076][105620] Updated weights for policy 1, policy_version 475423 (0.0010) [2023-12-26 18:49:24,182][105692] Updated weights for policy 0, policy_version 475050 (0.0010) [2023-12-26 18:49:24,250][105692] Updated weights for policy 0, policy_version 475060 (0.0010) [2023-12-26 18:49:24,312][105692] Updated weights for policy 0, policy_version 475070 (0.0010) [2023-12-26 18:49:24,775][105620] Updated weights for policy 1, policy_version 475433 (0.0010) [2023-12-26 18:49:24,836][105620] Updated weights for policy 1, policy_version 475443 (0.0010) [2023-12-26 18:49:24,894][105620] Updated weights for policy 1, policy_version 475453 (0.0010) [2023-12-26 18:49:24,954][105620] Updated weights for policy 1, policy_version 475463 (0.0010) [2023-12-26 18:49:25,065][105692] Updated weights for policy 0, policy_version 475080 (0.0011) [2023-12-26 18:49:25,126][105692] Updated weights for policy 0, policy_version 475090 (0.0011) [2023-12-26 18:49:25,184][105692] Updated weights for policy 0, policy_version 475100 (0.0010) [2023-12-26 18:49:25,676][105620] Updated weights for policy 1, policy_version 475473 (0.0010) [2023-12-26 18:49:25,734][105620] Updated weights for policy 1, policy_version 475483 (0.0010) [2023-12-26 18:49:25,789][105620] Updated weights for policy 1, policy_version 475493 (0.0010) [2023-12-26 18:49:25,895][105692] Updated weights for policy 0, policy_version 475110 (0.0007) [2023-12-26 18:49:25,959][105692] Updated weights for policy 0, policy_version 475120 (0.0005) [2023-12-26 18:49:26,016][105692] Updated weights for policy 0, policy_version 475130 (0.0005) [2023-12-26 18:49:26,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 243392512. Throughput: 0: 9824.0, 1: 9756.5. Samples: 243396072. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-26 18:49:26,062][104569] Avg episode reward: [(0, '9092.919'), (1, '9080.900')] [2023-12-26 18:49:26,424][105620] Updated weights for policy 1, policy_version 475503 (0.0010) [2023-12-26 18:49:26,485][105620] Updated weights for policy 1, policy_version 475513 (0.0010) [2023-12-26 18:49:26,523][105692] Updated weights for policy 0, policy_version 475140 (0.0005) [2023-12-26 18:49:26,535][105620] Updated weights for policy 1, policy_version 475523 (0.0009) [2023-12-26 18:49:26,588][105692] Updated weights for policy 0, policy_version 475150 (0.0008) [2023-12-26 18:49:26,645][105692] Updated weights for policy 0, policy_version 475160 (0.0008) [2023-12-26 18:49:27,234][105620] Updated weights for policy 1, policy_version 475533 (0.0006) [2023-12-26 18:49:27,282][105692] Updated weights for policy 0, policy_version 475170 (0.0006) [2023-12-26 18:49:27,304][105620] Updated weights for policy 1, policy_version 475543 (0.0009) [2023-12-26 18:49:27,334][105692] Updated weights for policy 0, policy_version 475180 (0.0007) [2023-12-26 18:49:27,367][105620] Updated weights for policy 1, policy_version 475553 (0.0010) [2023-12-26 18:49:27,388][105692] Updated weights for policy 0, policy_version 475190 (0.0009) [2023-12-26 18:49:27,448][105692] Updated weights for policy 0, policy_version 475200 (0.0007) [2023-12-26 18:49:27,957][105620] Updated weights for policy 1, policy_version 475563 (0.0010) [2023-12-26 18:49:28,007][105620] Updated weights for policy 1, policy_version 475573 (0.0009) [2023-12-26 18:49:28,058][105620] Updated weights for policy 1, policy_version 475583 (0.0010) [2023-12-26 18:49:28,183][105692] Updated weights for policy 0, policy_version 475210 (0.0008) [2023-12-26 18:49:28,234][105692] Updated weights for policy 0, policy_version 475220 (0.0008) [2023-12-26 18:49:28,290][105692] Updated weights for policy 0, policy_version 475230 (0.0008) [2023-12-26 18:49:28,790][105620] Updated weights for policy 1, policy_version 475593 (0.0010) [2023-12-26 18:49:28,856][105620] Updated weights for policy 1, policy_version 475603 (0.0010) [2023-12-26 18:49:28,917][105620] Updated weights for policy 1, policy_version 475613 (0.0010) [2023-12-26 18:49:28,961][105620] Updated weights for policy 1, policy_version 475623 (0.0008) [2023-12-26 18:49:29,073][105692] Updated weights for policy 0, policy_version 475240 (0.0010) [2023-12-26 18:49:29,144][105692] Updated weights for policy 0, policy_version 475250 (0.0010) [2023-12-26 18:49:29,213][105692] Updated weights for policy 0, policy_version 475260 (0.0011) [2023-12-26 18:49:29,596][105620] Updated weights for policy 1, policy_version 475633 (0.0010) [2023-12-26 18:49:29,654][105620] Updated weights for policy 1, policy_version 475643 (0.0010) [2023-12-26 18:49:29,713][105620] Updated weights for policy 1, policy_version 475653 (0.0010) [2023-12-26 18:49:29,876][105692] Updated weights for policy 0, policy_version 475270 (0.0008) [2023-12-26 18:49:29,942][105692] Updated weights for policy 0, policy_version 475280 (0.0011) [2023-12-26 18:49:30,005][105692] Updated weights for policy 0, policy_version 475290 (0.0007) [2023-12-26 18:49:30,355][105620] Updated weights for policy 1, policy_version 475663 (0.0008) [2023-12-26 18:49:30,418][105620] Updated weights for policy 1, policy_version 475673 (0.0007) [2023-12-26 18:49:30,488][105620] Updated weights for policy 1, policy_version 475683 (0.0005) [2023-12-26 18:49:30,568][105692] Updated weights for policy 0, policy_version 475300 (0.0006) [2023-12-26 18:49:30,621][105692] Updated weights for policy 0, policy_version 475310 (0.0005) [2023-12-26 18:49:30,675][105692] Updated weights for policy 0, policy_version 475320 (0.0007) [2023-12-26 18:49:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 243490816. Throughput: 0: 9901.6, 1: 9821.8. Samples: 243458832. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-26 18:49:31,062][104569] Avg episode reward: [(0, '8913.497'), (1, '8996.275')] [2023-12-26 18:49:31,066][105620] Updated weights for policy 1, policy_version 475693 (0.0007) [2023-12-26 18:49:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000475328_121700352.pth... [2023-12-26 18:49:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000474176_121405440.pth [2023-12-26 18:49:31,133][105620] Updated weights for policy 1, policy_version 475703 (0.0008) [2023-12-26 18:49:31,196][105620] Updated weights for policy 1, policy_version 475713 (0.0008) [2023-12-26 18:49:31,241][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000475720_121798656.pth... [2023-12-26 18:49:31,246][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000474536_121495552.pth [2023-12-26 18:49:31,320][105692] Updated weights for policy 0, policy_version 475330 (0.0009) [2023-12-26 18:49:31,384][105692] Updated weights for policy 0, policy_version 475340 (0.0009) [2023-12-26 18:49:31,439][105692] Updated weights for policy 0, policy_version 475350 (0.0008) [2023-12-26 18:49:31,506][105692] Updated weights for policy 0, policy_version 475360 (0.0005) [2023-12-26 18:49:31,843][105620] Updated weights for policy 1, policy_version 475723 (0.0008) [2023-12-26 18:49:31,910][105620] Updated weights for policy 1, policy_version 475733 (0.0005) [2023-12-26 18:49:31,979][105620] Updated weights for policy 1, policy_version 475743 (0.0006) [2023-12-26 18:49:32,103][105692] Updated weights for policy 0, policy_version 475370 (0.0005) [2023-12-26 18:49:32,164][105692] Updated weights for policy 0, policy_version 475380 (0.0005) [2023-12-26 18:49:32,228][105692] Updated weights for policy 0, policy_version 475390 (0.0005) [2023-12-26 18:49:32,575][105620] Updated weights for policy 1, policy_version 475753 (0.0006) [2023-12-26 18:49:32,636][105620] Updated weights for policy 1, policy_version 475763 (0.0008) [2023-12-26 18:49:32,705][105620] Updated weights for policy 1, policy_version 475773 (0.0005) [2023-12-26 18:49:32,771][105620] Updated weights for policy 1, policy_version 475783 (0.0005) [2023-12-26 18:49:32,959][105692] Updated weights for policy 0, policy_version 475400 (0.0006) [2023-12-26 18:49:33,014][105692] Updated weights for policy 0, policy_version 475410 (0.0005) [2023-12-26 18:49:33,066][105692] Updated weights for policy 0, policy_version 475420 (0.0005) [2023-12-26 18:49:33,395][105620] Updated weights for policy 1, policy_version 475793 (0.0009) [2023-12-26 18:49:33,454][105620] Updated weights for policy 1, policy_version 475803 (0.0008) [2023-12-26 18:49:33,514][105620] Updated weights for policy 1, policy_version 475813 (0.0005) [2023-12-26 18:49:33,674][105692] Updated weights for policy 0, policy_version 475430 (0.0008) [2023-12-26 18:49:33,722][105692] Updated weights for policy 0, policy_version 475440 (0.0010) [2023-12-26 18:49:33,776][105692] Updated weights for policy 0, policy_version 475450 (0.0010) [2023-12-26 18:49:34,187][105620] Updated weights for policy 1, policy_version 475823 (0.0006) [2023-12-26 18:49:34,245][105620] Updated weights for policy 1, policy_version 475833 (0.0005) [2023-12-26 18:49:34,309][105620] Updated weights for policy 1, policy_version 475843 (0.0006) [2023-12-26 18:49:34,443][105692] Updated weights for policy 0, policy_version 475460 (0.0007) [2023-12-26 18:49:34,502][105692] Updated weights for policy 0, policy_version 475470 (0.0008) [2023-12-26 18:49:34,562][105692] Updated weights for policy 0, policy_version 475480 (0.0008) [2023-12-26 18:49:34,950][105620] Updated weights for policy 1, policy_version 475853 (0.0009) [2023-12-26 18:49:35,017][105620] Updated weights for policy 1, policy_version 475863 (0.0011) [2023-12-26 18:49:35,077][105620] Updated weights for policy 1, policy_version 475873 (0.0010) [2023-12-26 18:49:35,133][105692] Updated weights for policy 0, policy_version 475490 (0.0007) [2023-12-26 18:49:35,184][105692] Updated weights for policy 0, policy_version 475500 (0.0008) [2023-12-26 18:49:35,228][105692] Updated weights for policy 0, policy_version 475510 (0.0008) [2023-12-26 18:49:35,278][105692] Updated weights for policy 0, policy_version 475520 (0.0007) [2023-12-26 18:49:35,808][105620] Updated weights for policy 1, policy_version 475883 (0.0009) [2023-12-26 18:49:35,860][105620] Updated weights for policy 1, policy_version 475893 (0.0010) [2023-12-26 18:49:35,913][105620] Updated weights for policy 1, policy_version 475903 (0.0010) [2023-12-26 18:49:35,931][105692] Updated weights for policy 0, policy_version 475530 (0.0005) [2023-12-26 18:49:35,992][105692] Updated weights for policy 0, policy_version 475540 (0.0007) [2023-12-26 18:49:36,050][105692] Updated weights for policy 0, policy_version 475550 (0.0010) [2023-12-26 18:49:36,062][104569] Fps is (10 sec: 21299.2, 60 sec: 20070.4, 300 sec: 19577.5). Total num frames: 243605504. Throughput: 0: 9943.7, 1: 9945.1. Samples: 243586040. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-26 18:49:36,062][104569] Avg episode reward: [(0, '8733.115'), (1, '8993.479')] [2023-12-26 18:49:36,646][105620] Updated weights for policy 1, policy_version 475913 (0.0010) [2023-12-26 18:49:36,705][105620] Updated weights for policy 1, policy_version 475923 (0.0010) [2023-12-26 18:49:36,756][105620] Updated weights for policy 1, policy_version 475933 (0.0010) [2023-12-26 18:49:36,783][105692] Updated weights for policy 0, policy_version 475560 (0.0010) [2023-12-26 18:49:36,819][105620] Updated weights for policy 1, policy_version 475943 (0.0011) [2023-12-26 18:49:36,838][105692] Updated weights for policy 0, policy_version 475570 (0.0009) [2023-12-26 18:49:36,890][105692] Updated weights for policy 0, policy_version 475580 (0.0010) [2023-12-26 18:49:37,498][105620] Updated weights for policy 1, policy_version 475953 (0.0008) [2023-12-26 18:49:37,568][105620] Updated weights for policy 1, policy_version 475963 (0.0008) [2023-12-26 18:49:37,633][105620] Updated weights for policy 1, policy_version 475973 (0.0007) [2023-12-26 18:49:37,672][105692] Updated weights for policy 0, policy_version 475590 (0.0010) [2023-12-26 18:49:37,735][105692] Updated weights for policy 0, policy_version 475600 (0.0009) [2023-12-26 18:49:37,794][105692] Updated weights for policy 0, policy_version 475610 (0.0008) [2023-12-26 18:49:38,304][105620] Updated weights for policy 1, policy_version 475983 (0.0007) [2023-12-26 18:49:38,370][105620] Updated weights for policy 1, policy_version 475993 (0.0007) [2023-12-26 18:49:38,429][105620] Updated weights for policy 1, policy_version 476003 (0.0006) [2023-12-26 18:49:38,510][105692] Updated weights for policy 0, policy_version 475620 (0.0011) [2023-12-26 18:49:38,568][105692] Updated weights for policy 0, policy_version 475630 (0.0008) [2023-12-26 18:49:38,620][105692] Updated weights for policy 0, policy_version 475640 (0.0008) [2023-12-26 18:49:39,084][105620] Updated weights for policy 1, policy_version 476013 (0.0009) [2023-12-26 18:49:39,149][105620] Updated weights for policy 1, policy_version 476023 (0.0009) [2023-12-26 18:49:39,202][105620] Updated weights for policy 1, policy_version 476033 (0.0010) [2023-12-26 18:49:39,232][105692] Updated weights for policy 0, policy_version 475650 (0.0009) [2023-12-26 18:49:39,299][105692] Updated weights for policy 0, policy_version 475660 (0.0006) [2023-12-26 18:49:39,370][105692] Updated weights for policy 0, policy_version 475670 (0.0009) [2023-12-26 18:49:39,436][105692] Updated weights for policy 0, policy_version 475680 (0.0008) [2023-12-26 18:49:39,992][105620] Updated weights for policy 1, policy_version 476043 (0.0008) [2023-12-26 18:49:40,010][105692] Updated weights for policy 0, policy_version 475690 (0.0008) [2023-12-26 18:49:40,056][105620] Updated weights for policy 1, policy_version 476053 (0.0006) [2023-12-26 18:49:40,066][105692] Updated weights for policy 0, policy_version 475700 (0.0008) [2023-12-26 18:49:40,113][105620] Updated weights for policy 1, policy_version 476063 (0.0006) [2023-12-26 18:49:40,123][105692] Updated weights for policy 0, policy_version 475710 (0.0008) [2023-12-26 18:49:40,675][105620] Updated weights for policy 1, policy_version 476073 (0.0006) [2023-12-26 18:49:40,737][105620] Updated weights for policy 1, policy_version 476083 (0.0008) [2023-12-26 18:49:40,774][105692] Updated weights for policy 0, policy_version 475720 (0.0009) [2023-12-26 18:49:40,785][105620] Updated weights for policy 1, policy_version 476093 (0.0009) [2023-12-26 18:49:40,822][105692] Updated weights for policy 0, policy_version 475730 (0.0008) [2023-12-26 18:49:40,837][105620] Updated weights for policy 1, policy_version 476103 (0.0006) [2023-12-26 18:49:40,878][105692] Updated weights for policy 0, policy_version 475740 (0.0007) [2023-12-26 18:49:41,062][104569] Fps is (10 sec: 21299.0, 60 sec: 19933.9, 300 sec: 19549.7). Total num frames: 243703808. Throughput: 0: 10013.0, 1: 9943.9. Samples: 243707248. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 18:49:41,063][104569] Avg episode reward: [(0, '8731.734'), (1, '9175.185')] [2023-12-26 18:49:41,587][105620] Updated weights for policy 1, policy_version 476113 (0.0007) [2023-12-26 18:49:41,652][105620] Updated weights for policy 1, policy_version 476123 (0.0008) [2023-12-26 18:49:41,712][105620] Updated weights for policy 1, policy_version 476133 (0.0007) [2023-12-26 18:49:41,714][105692] Updated weights for policy 0, policy_version 475750 (0.0009) [2023-12-26 18:49:41,776][105692] Updated weights for policy 0, policy_version 475760 (0.0008) [2023-12-26 18:49:41,829][105692] Updated weights for policy 0, policy_version 475770 (0.0008) [2023-12-26 18:49:42,443][105620] Updated weights for policy 1, policy_version 476143 (0.0009) [2023-12-26 18:49:42,505][105620] Updated weights for policy 1, policy_version 476153 (0.0009) [2023-12-26 18:49:42,558][105620] Updated weights for policy 1, policy_version 476163 (0.0006) [2023-12-26 18:49:42,605][105692] Updated weights for policy 0, policy_version 475780 (0.0009) [2023-12-26 18:49:42,668][105692] Updated weights for policy 0, policy_version 475790 (0.0008) [2023-12-26 18:49:42,730][105692] Updated weights for policy 0, policy_version 475800 (0.0009) [2023-12-26 18:49:43,286][105620] Updated weights for policy 1, policy_version 476173 (0.0007) [2023-12-26 18:49:43,336][105620] Updated weights for policy 1, policy_version 476183 (0.0009) [2023-12-26 18:49:43,383][105620] Updated weights for policy 1, policy_version 476193 (0.0008) [2023-12-26 18:49:43,486][105692] Updated weights for policy 0, policy_version 475810 (0.0009) [2023-12-26 18:49:43,533][105692] Updated weights for policy 0, policy_version 475820 (0.0009) [2023-12-26 18:49:43,589][105692] Updated weights for policy 0, policy_version 475830 (0.0009) [2023-12-26 18:49:43,647][105692] Updated weights for policy 0, policy_version 475840 (0.0009) [2023-12-26 18:49:44,028][105620] Updated weights for policy 1, policy_version 476203 (0.0006) [2023-12-26 18:49:44,093][105620] Updated weights for policy 1, policy_version 476213 (0.0008) [2023-12-26 18:49:44,165][105620] Updated weights for policy 1, policy_version 476223 (0.0007) [2023-12-26 18:49:44,482][105692] Updated weights for policy 0, policy_version 475850 (0.0009) [2023-12-26 18:49:44,547][105692] Updated weights for policy 0, policy_version 475860 (0.0009) [2023-12-26 18:49:44,607][105692] Updated weights for policy 0, policy_version 475870 (0.0009) [2023-12-26 18:49:44,866][105620] Updated weights for policy 1, policy_version 476233 (0.0009) [2023-12-26 18:49:44,926][105620] Updated weights for policy 1, policy_version 476243 (0.0009) [2023-12-26 18:49:44,994][105620] Updated weights for policy 1, policy_version 476253 (0.0010) [2023-12-26 18:49:45,063][105620] Updated weights for policy 1, policy_version 476263 (0.0009) [2023-12-26 18:49:45,297][105692] Updated weights for policy 0, policy_version 475880 (0.0007) [2023-12-26 18:49:45,363][105692] Updated weights for policy 0, policy_version 475890 (0.0005) [2023-12-26 18:49:45,431][105692] Updated weights for policy 0, policy_version 475900 (0.0006) [2023-12-26 18:49:45,910][105620] Updated weights for policy 1, policy_version 476273 (0.0009) [2023-12-26 18:49:45,963][105620] Updated weights for policy 1, policy_version 476283 (0.0009) [2023-12-26 18:49:46,011][105620] Updated weights for policy 1, policy_version 476293 (0.0009) [2023-12-26 18:49:46,047][105692] Updated weights for policy 0, policy_version 475910 (0.0007) [2023-12-26 18:49:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.4, 300 sec: 19522.0). Total num frames: 243793920. Throughput: 0: 9929.6, 1: 10017.2. Samples: 243764424. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 18:49:46,062][104569] Avg episode reward: [(0, '8728.389'), (1, '9356.262')] [2023-12-26 18:49:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000476296_121946112.pth... [2023-12-26 18:49:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000475112_121643008.pth [2023-12-26 18:49:46,107][105692] Updated weights for policy 0, policy_version 475920 (0.0009) [2023-12-26 18:49:46,172][105692] Updated weights for policy 0, policy_version 475930 (0.0009) [2023-12-26 18:49:46,200][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000475936_121856000.pth... [2023-12-26 18:49:46,203][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000474752_121552896.pth [2023-12-26 18:49:46,792][105620] Updated weights for policy 1, policy_version 476303 (0.0008) [2023-12-26 18:49:46,839][105620] Updated weights for policy 1, policy_version 476313 (0.0009) [2023-12-26 18:49:46,887][105620] Updated weights for policy 1, policy_version 476323 (0.0009) [2023-12-26 18:49:46,915][105692] Updated weights for policy 0, policy_version 475940 (0.0007) [2023-12-26 18:49:46,966][105692] Updated weights for policy 0, policy_version 475950 (0.0008) [2023-12-26 18:49:47,022][105692] Updated weights for policy 0, policy_version 475960 (0.0010) [2023-12-26 18:49:47,550][105620] Updated weights for policy 1, policy_version 476333 (0.0008) [2023-12-26 18:49:47,611][105620] Updated weights for policy 1, policy_version 476343 (0.0009) [2023-12-26 18:49:47,673][105620] Updated weights for policy 1, policy_version 476353 (0.0009) [2023-12-26 18:49:47,818][105692] Updated weights for policy 0, policy_version 475970 (0.0009) [2023-12-26 18:49:47,878][105692] Updated weights for policy 0, policy_version 475980 (0.0009) [2023-12-26 18:49:47,935][105692] Updated weights for policy 0, policy_version 475990 (0.0009) [2023-12-26 18:49:47,999][105692] Updated weights for policy 0, policy_version 476000 (0.0009) [2023-12-26 18:49:48,410][105620] Updated weights for policy 1, policy_version 476363 (0.0009) [2023-12-26 18:49:48,465][105620] Updated weights for policy 1, policy_version 476373 (0.0008) [2023-12-26 18:49:48,532][105620] Updated weights for policy 1, policy_version 476383 (0.0008) [2023-12-26 18:49:48,772][105692] Updated weights for policy 0, policy_version 476010 (0.0010) [2023-12-26 18:49:48,824][105692] Updated weights for policy 0, policy_version 476020 (0.0010) [2023-12-26 18:49:48,869][105692] Updated weights for policy 0, policy_version 476030 (0.0010) [2023-12-26 18:49:49,288][105620] Updated weights for policy 1, policy_version 476393 (0.0008) [2023-12-26 18:49:49,351][105620] Updated weights for policy 1, policy_version 476403 (0.0008) [2023-12-26 18:49:49,411][105620] Updated weights for policy 1, policy_version 476413 (0.0008) [2023-12-26 18:49:49,462][105620] Updated weights for policy 1, policy_version 476423 (0.0007) [2023-12-26 18:49:49,658][105692] Updated weights for policy 0, policy_version 476040 (0.0009) [2023-12-26 18:49:49,723][105692] Updated weights for policy 0, policy_version 476050 (0.0008) [2023-12-26 18:49:49,780][105692] Updated weights for policy 0, policy_version 476060 (0.0008) [2023-12-26 18:49:50,121][105620] Updated weights for policy 1, policy_version 476433 (0.0008) [2023-12-26 18:49:50,182][105620] Updated weights for policy 1, policy_version 476443 (0.0008) [2023-12-26 18:49:50,241][105620] Updated weights for policy 1, policy_version 476453 (0.0006) [2023-12-26 18:49:50,603][105692] Updated weights for policy 0, policy_version 476070 (0.0008) [2023-12-26 18:49:50,665][105692] Updated weights for policy 0, policy_version 476080 (0.0009) [2023-12-26 18:49:50,726][105692] Updated weights for policy 0, policy_version 476090 (0.0009) [2023-12-26 18:49:50,830][105620] Updated weights for policy 1, policy_version 476463 (0.0008) [2023-12-26 18:49:50,880][105620] Updated weights for policy 1, policy_version 476473 (0.0008) [2023-12-26 18:49:50,935][105620] Updated weights for policy 1, policy_version 476483 (0.0009) [2023-12-26 18:49:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.3, 300 sec: 19521.9). Total num frames: 243892224. Throughput: 0: 9897.8, 1: 9932.5. Samples: 243877328. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 18:49:51,063][104569] Avg episode reward: [(0, '8729.738'), (1, '9356.365')] [2023-12-26 18:49:51,490][105692] Updated weights for policy 0, policy_version 476100 (0.0008) [2023-12-26 18:49:51,544][105692] Updated weights for policy 0, policy_version 476110 (0.0009) [2023-12-26 18:49:51,605][105692] Updated weights for policy 0, policy_version 476120 (0.0006) [2023-12-26 18:49:51,726][105620] Updated weights for policy 1, policy_version 476493 (0.0009) [2023-12-26 18:49:51,794][105620] Updated weights for policy 1, policy_version 476503 (0.0009) [2023-12-26 18:49:51,848][105620] Updated weights for policy 1, policy_version 476514 (0.0010) [2023-12-26 18:49:52,256][105692] Updated weights for policy 0, policy_version 476130 (0.0008) [2023-12-26 18:49:52,307][105692] Updated weights for policy 0, policy_version 476140 (0.0008) [2023-12-26 18:49:52,362][105692] Updated weights for policy 0, policy_version 476150 (0.0009) [2023-12-26 18:49:52,419][105692] Updated weights for policy 0, policy_version 476160 (0.0008) [2023-12-26 18:49:52,630][105620] Updated weights for policy 1, policy_version 476524 (0.0009) [2023-12-26 18:49:52,692][105620] Updated weights for policy 1, policy_version 476534 (0.0009) [2023-12-26 18:49:52,753][105620] Updated weights for policy 1, policy_version 476544 (0.0008) [2023-12-26 18:49:53,118][105692] Updated weights for policy 0, policy_version 476170 (0.0009) [2023-12-26 18:49:53,166][105692] Updated weights for policy 0, policy_version 476180 (0.0009) [2023-12-26 18:49:53,212][105692] Updated weights for policy 0, policy_version 476190 (0.0008) [2023-12-26 18:49:53,550][105620] Updated weights for policy 1, policy_version 476554 (0.0009) [2023-12-26 18:49:53,604][105620] Updated weights for policy 1, policy_version 476564 (0.0009) [2023-12-26 18:49:53,656][105620] Updated weights for policy 1, policy_version 476574 (0.0007) [2023-12-26 18:49:53,708][105620] Updated weights for policy 1, policy_version 476584 (0.0005) [2023-12-26 18:49:54,013][105692] Updated weights for policy 0, policy_version 476200 (0.0008) [2023-12-26 18:49:54,075][105692] Updated weights for policy 0, policy_version 476210 (0.0008) [2023-12-26 18:49:54,128][105692] Updated weights for policy 0, policy_version 476220 (0.0008) [2023-12-26 18:49:54,402][105620] Updated weights for policy 1, policy_version 476594 (0.0010) [2023-12-26 18:49:54,450][105620] Updated weights for policy 1, policy_version 476604 (0.0010) [2023-12-26 18:49:54,501][105620] Updated weights for policy 1, policy_version 476614 (0.0010) [2023-12-26 18:49:54,851][105692] Updated weights for policy 0, policy_version 476230 (0.0009) [2023-12-26 18:49:54,907][105692] Updated weights for policy 0, policy_version 476240 (0.0009) [2023-12-26 18:49:54,954][105692] Updated weights for policy 0, policy_version 476250 (0.0009) [2023-12-26 18:49:55,173][105620] Updated weights for policy 1, policy_version 476624 (0.0006) [2023-12-26 18:49:55,232][105620] Updated weights for policy 1, policy_version 476634 (0.0006) [2023-12-26 18:49:55,285][105620] Updated weights for policy 1, policy_version 476644 (0.0008) [2023-12-26 18:49:55,672][105692] Updated weights for policy 0, policy_version 476260 (0.0008) [2023-12-26 18:49:55,723][105692] Updated weights for policy 0, policy_version 476270 (0.0005) [2023-12-26 18:49:55,777][105692] Updated weights for policy 0, policy_version 476280 (0.0005) [2023-12-26 18:49:55,953][105620] Updated weights for policy 1, policy_version 476654 (0.0006) [2023-12-26 18:49:55,998][105620] Updated weights for policy 1, policy_version 476664 (0.0005) [2023-12-26 18:49:56,055][105620] Updated weights for policy 1, policy_version 476674 (0.0007) [2023-12-26 18:49:56,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 243982336. Throughput: 0: 9836.5, 1: 9933.6. Samples: 243993268. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 18:49:56,063][104569] Avg episode reward: [(0, '7860.601'), (1, '9356.560')] [2023-12-26 18:49:56,293][105692] Updated weights for policy 0, policy_version 476290 (0.0006) [2023-12-26 18:49:56,345][105692] Updated weights for policy 0, policy_version 476300 (0.0010) [2023-12-26 18:49:56,403][105692] Updated weights for policy 0, policy_version 476310 (0.0010) [2023-12-26 18:49:56,451][105692] Updated weights for policy 0, policy_version 476320 (0.0010) [2023-12-26 18:49:56,773][105620] Updated weights for policy 1, policy_version 476684 (0.0011) [2023-12-26 18:49:56,831][105620] Updated weights for policy 1, policy_version 476694 (0.0010) [2023-12-26 18:49:56,880][105620] Updated weights for policy 1, policy_version 476704 (0.0010) [2023-12-26 18:49:57,198][105692] Updated weights for policy 0, policy_version 476330 (0.0008) [2023-12-26 18:49:57,252][105692] Updated weights for policy 0, policy_version 476340 (0.0007) [2023-12-26 18:49:57,302][105692] Updated weights for policy 0, policy_version 476350 (0.0008) [2023-12-26 18:49:57,631][105620] Updated weights for policy 1, policy_version 476714 (0.0011) [2023-12-26 18:49:57,692][105620] Updated weights for policy 1, policy_version 476724 (0.0010) [2023-12-26 18:49:57,756][105620] Updated weights for policy 1, policy_version 476734 (0.0010) [2023-12-26 18:49:57,817][105620] Updated weights for policy 1, policy_version 476744 (0.0010) [2023-12-26 18:49:57,925][105692] Updated weights for policy 0, policy_version 476360 (0.0005) [2023-12-26 18:49:57,975][105692] Updated weights for policy 0, policy_version 476370 (0.0005) [2023-12-26 18:49:58,023][105692] Updated weights for policy 0, policy_version 476380 (0.0005) [2023-12-26 18:49:58,601][105620] Updated weights for policy 1, policy_version 476754 (0.0009) [2023-12-26 18:49:58,670][105620] Updated weights for policy 1, policy_version 476764 (0.0010) [2023-12-26 18:49:58,736][105620] Updated weights for policy 1, policy_version 476774 (0.0010) [2023-12-26 18:49:58,763][105692] Updated weights for policy 0, policy_version 476390 (0.0007) [2023-12-26 18:49:58,823][105692] Updated weights for policy 0, policy_version 476400 (0.0006) [2023-12-26 18:49:58,893][105692] Updated weights for policy 0, policy_version 476410 (0.0006) [2023-12-26 18:49:58,911][105585] KL-divergence is very high: 103.8446 [2023-12-26 18:49:59,459][105620] Updated weights for policy 1, policy_version 476784 (0.0007) [2023-12-26 18:49:59,529][105620] Updated weights for policy 1, policy_version 476794 (0.0006) [2023-12-26 18:49:59,567][105585] KL-divergence is very high: 102.0979 [2023-12-26 18:49:59,580][105692] Updated weights for policy 0, policy_version 476420 (0.0006) [2023-12-26 18:49:59,591][105620] Updated weights for policy 1, policy_version 476804 (0.0009) [2023-12-26 18:49:59,639][105692] Updated weights for policy 0, policy_version 476430 (0.0005) [2023-12-26 18:49:59,704][105692] Updated weights for policy 0, policy_version 476440 (0.0009) [2023-12-26 18:50:00,257][105620] Updated weights for policy 1, policy_version 476814 (0.0011) [2023-12-26 18:50:00,306][105620] Updated weights for policy 1, policy_version 476824 (0.0010) [2023-12-26 18:50:00,372][105620] Updated weights for policy 1, policy_version 476834 (0.0011) [2023-12-26 18:50:00,449][105692] Updated weights for policy 0, policy_version 476450 (0.0010) [2023-12-26 18:50:00,484][105585] KL-divergence is very high: 188.5628 [2023-12-26 18:50:00,492][105585] KL-divergence is very high: 166.7313 [2023-12-26 18:50:00,514][105692] Updated weights for policy 0, policy_version 476460 (0.0009) [2023-12-26 18:50:00,541][105585] KL-divergence is very high: 274.5309 [2023-12-26 18:50:00,548][105585] KL-divergence is very high: 229.2974 [2023-12-26 18:50:00,580][105692] Updated weights for policy 0, policy_version 476470 (0.0009) [2023-12-26 18:50:00,596][105585] KL-divergence is very high: 247.0970 [2023-12-26 18:50:00,603][105585] KL-divergence is very high: 197.1018 [2023-12-26 18:50:00,645][105692] Updated weights for policy 0, policy_version 476480 (0.0009) [2023-12-26 18:50:01,027][105620] Updated weights for policy 1, policy_version 476844 (0.0010) [2023-12-26 18:50:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 244080640. Throughput: 0: 9932.6, 1: 9931.0. Samples: 244053556. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 18:50:01,063][104569] Avg episode reward: [(0, '7944.472'), (1, '9173.813')] [2023-12-26 18:50:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000476480_121995264.pth... [2023-12-26 18:50:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000475328_121700352.pth [2023-12-26 18:50:01,091][105620] Updated weights for policy 1, policy_version 476854 (0.0011) [2023-12-26 18:50:01,155][105620] Updated weights for policy 1, policy_version 476864 (0.0011) [2023-12-26 18:50:01,205][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000476872_122093568.pth... [2023-12-26 18:50:01,209][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000475720_121798656.pth [2023-12-26 18:50:01,362][105692] Updated weights for policy 0, policy_version 476490 (0.0007) [2023-12-26 18:50:01,419][105692] Updated weights for policy 0, policy_version 476500 (0.0011) [2023-12-26 18:50:01,467][105692] Updated weights for policy 0, policy_version 476510 (0.0010) [2023-12-26 18:50:01,896][105620] Updated weights for policy 1, policy_version 476874 (0.0008) [2023-12-26 18:50:01,960][105620] Updated weights for policy 1, policy_version 476884 (0.0010) [2023-12-26 18:50:02,015][105620] Updated weights for policy 1, policy_version 476894 (0.0010) [2023-12-26 18:50:02,069][105620] Updated weights for policy 1, policy_version 476904 (0.0010) [2023-12-26 18:50:02,243][105692] Updated weights for policy 0, policy_version 476520 (0.0011) [2023-12-26 18:50:02,306][105692] Updated weights for policy 0, policy_version 476530 (0.0011) [2023-12-26 18:50:02,375][105692] Updated weights for policy 0, policy_version 476540 (0.0010) [2023-12-26 18:50:02,772][105620] Updated weights for policy 1, policy_version 476914 (0.0005) [2023-12-26 18:50:02,831][105620] Updated weights for policy 1, policy_version 476924 (0.0005) [2023-12-26 18:50:02,892][105620] Updated weights for policy 1, policy_version 476934 (0.0005) [2023-12-26 18:50:03,109][105692] Updated weights for policy 0, policy_version 476550 (0.0010) [2023-12-26 18:50:03,163][105692] Updated weights for policy 0, policy_version 476560 (0.0010) [2023-12-26 18:50:03,209][105692] Updated weights for policy 0, policy_version 476570 (0.0010) [2023-12-26 18:50:03,476][105620] Updated weights for policy 1, policy_version 476944 (0.0009) [2023-12-26 18:50:03,530][105620] Updated weights for policy 1, policy_version 476954 (0.0010) [2023-12-26 18:50:03,584][105620] Updated weights for policy 1, policy_version 476964 (0.0010) [2023-12-26 18:50:03,941][105585] KL-divergence is very high: 130.7411 [2023-12-26 18:50:03,958][105692] Updated weights for policy 0, policy_version 476580 (0.0010) [2023-12-26 18:50:04,021][105692] Updated weights for policy 0, policy_version 476590 (0.0011) [2023-12-26 18:50:04,076][105692] Updated weights for policy 0, policy_version 476600 (0.0011) [2023-12-26 18:50:04,236][105620] Updated weights for policy 1, policy_version 476974 (0.0010) [2023-12-26 18:50:04,287][105620] Updated weights for policy 1, policy_version 476984 (0.0010) [2023-12-26 18:50:04,344][105620] Updated weights for policy 1, policy_version 476994 (0.0011) [2023-12-26 18:50:04,814][105692] Updated weights for policy 0, policy_version 476610 (0.0010) [2023-12-26 18:50:04,878][105692] Updated weights for policy 0, policy_version 476620 (0.0010) [2023-12-26 18:50:04,938][105692] Updated weights for policy 0, policy_version 476630 (0.0010) [2023-12-26 18:50:05,002][105692] Updated weights for policy 0, policy_version 476640 (0.0010) [2023-12-26 18:50:05,103][105620] Updated weights for policy 1, policy_version 477004 (0.0010) [2023-12-26 18:50:05,155][105620] Updated weights for policy 1, policy_version 477014 (0.0010) [2023-12-26 18:50:05,200][105620] Updated weights for policy 1, policy_version 477024 (0.0010) [2023-12-26 18:50:05,720][105692] Updated weights for policy 0, policy_version 476650 (0.0010) [2023-12-26 18:50:05,768][105692] Updated weights for policy 0, policy_version 476660 (0.0010) [2023-12-26 18:50:05,811][105692] Updated weights for policy 0, policy_version 476670 (0.0010) [2023-12-26 18:50:05,943][105620] Updated weights for policy 1, policy_version 477034 (0.0010) [2023-12-26 18:50:06,001][105620] Updated weights for policy 1, policy_version 477044 (0.0010) [2023-12-26 18:50:06,059][105620] Updated weights for policy 1, policy_version 477054 (0.0010) [2023-12-26 18:50:06,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 244178944. Throughput: 0: 9818.4, 1: 9958.1. Samples: 244170952. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 18:50:06,063][104569] Avg episode reward: [(0, '8280.381'), (1, '9264.298')] [2023-12-26 18:50:06,115][105620] Updated weights for policy 1, policy_version 477064 (0.0010) [2023-12-26 18:50:06,509][105692] Updated weights for policy 0, policy_version 476680 (0.0010) [2023-12-26 18:50:06,560][105692] Updated weights for policy 0, policy_version 476690 (0.0005) [2023-12-26 18:50:06,612][105692] Updated weights for policy 0, policy_version 476700 (0.0005) [2023-12-26 18:50:06,800][105620] Updated weights for policy 1, policy_version 477074 (0.0011) [2023-12-26 18:50:06,870][105620] Updated weights for policy 1, policy_version 477084 (0.0011) [2023-12-26 18:50:06,929][105620] Updated weights for policy 1, policy_version 477094 (0.0011) [2023-12-26 18:50:07,159][105692] Updated weights for policy 0, policy_version 476710 (0.0009) [2023-12-26 18:50:07,219][105692] Updated weights for policy 0, policy_version 476720 (0.0011) [2023-12-26 18:50:07,281][105692] Updated weights for policy 0, policy_version 476730 (0.0010) [2023-12-26 18:50:07,663][105620] Updated weights for policy 1, policy_version 477104 (0.0010) [2023-12-26 18:50:07,720][105620] Updated weights for policy 1, policy_version 477114 (0.0010) [2023-12-26 18:50:07,771][105620] Updated weights for policy 1, policy_version 477124 (0.0010) [2023-12-26 18:50:07,964][105692] Updated weights for policy 0, policy_version 476740 (0.0008) [2023-12-26 18:50:08,020][105692] Updated weights for policy 0, policy_version 476750 (0.0008) [2023-12-26 18:50:08,075][105692] Updated weights for policy 0, policy_version 476760 (0.0008) [2023-12-26 18:50:08,511][105620] Updated weights for policy 1, policy_version 477134 (0.0009) [2023-12-26 18:50:08,576][105620] Updated weights for policy 1, policy_version 477144 (0.0009) [2023-12-26 18:50:08,641][105620] Updated weights for policy 1, policy_version 477154 (0.0008) [2023-12-26 18:50:08,736][105692] Updated weights for policy 0, policy_version 476770 (0.0008) [2023-12-26 18:50:08,793][105692] Updated weights for policy 0, policy_version 476780 (0.0008) [2023-12-26 18:50:08,855][105692] Updated weights for policy 0, policy_version 476790 (0.0005) [2023-12-26 18:50:08,914][105692] Updated weights for policy 0, policy_version 476800 (0.0008) [2023-12-26 18:50:09,306][105620] Updated weights for policy 1, policy_version 477164 (0.0007) [2023-12-26 18:50:09,365][105620] Updated weights for policy 1, policy_version 477174 (0.0010) [2023-12-26 18:50:09,434][105620] Updated weights for policy 1, policy_version 477184 (0.0010) [2023-12-26 18:50:09,608][105692] Updated weights for policy 0, policy_version 476810 (0.0011) [2023-12-26 18:50:09,667][105692] Updated weights for policy 0, policy_version 476820 (0.0005) [2023-12-26 18:50:09,724][105692] Updated weights for policy 0, policy_version 476830 (0.0005) [2023-12-26 18:50:10,156][105620] Updated weights for policy 1, policy_version 477194 (0.0008) [2023-12-26 18:50:10,211][105620] Updated weights for policy 1, policy_version 477204 (0.0005) [2023-12-26 18:50:10,268][105620] Updated weights for policy 1, policy_version 477214 (0.0005) [2023-12-26 18:50:10,319][105620] Updated weights for policy 1, policy_version 477224 (0.0005) [2023-12-26 18:50:10,442][105692] Updated weights for policy 0, policy_version 476840 (0.0010) [2023-12-26 18:50:10,501][105692] Updated weights for policy 0, policy_version 476850 (0.0010) [2023-12-26 18:50:10,557][105692] Updated weights for policy 0, policy_version 476860 (0.0010) [2023-12-26 18:50:11,027][105620] Updated weights for policy 1, policy_version 477234 (0.0010) [2023-12-26 18:50:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 244277248. Throughput: 0: 9936.3, 1: 9943.6. Samples: 244290668. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 18:50:11,063][104569] Avg episode reward: [(0, '8365.559'), (1, '9263.889')] [2023-12-26 18:50:11,090][105620] Updated weights for policy 1, policy_version 477244 (0.0010) [2023-12-26 18:50:11,156][105620] Updated weights for policy 1, policy_version 477254 (0.0009) [2023-12-26 18:50:11,239][105692] Updated weights for policy 0, policy_version 476870 (0.0009) [2023-12-26 18:50:11,301][105692] Updated weights for policy 0, policy_version 476880 (0.0008) [2023-12-26 18:50:11,367][105692] Updated weights for policy 0, policy_version 476890 (0.0009) [2023-12-26 18:50:11,977][105620] Updated weights for policy 1, policy_version 477264 (0.0010) [2023-12-26 18:50:12,037][105620] Updated weights for policy 1, policy_version 477274 (0.0009) [2023-12-26 18:50:12,097][105620] Updated weights for policy 1, policy_version 477284 (0.0008) [2023-12-26 18:50:12,098][105692] Updated weights for policy 0, policy_version 476900 (0.0008) [2023-12-26 18:50:12,159][105692] Updated weights for policy 0, policy_version 476910 (0.0008) [2023-12-26 18:50:12,221][105692] Updated weights for policy 0, policy_version 476920 (0.0009) [2023-12-26 18:50:12,852][105620] Updated weights for policy 1, policy_version 477294 (0.0007) [2023-12-26 18:50:12,920][105620] Updated weights for policy 1, policy_version 477304 (0.0007) [2023-12-26 18:50:12,989][105620] Updated weights for policy 1, policy_version 477314 (0.0008) [2023-12-26 18:50:13,025][105692] Updated weights for policy 0, policy_version 476930 (0.0009) [2023-12-26 18:50:13,083][105692] Updated weights for policy 0, policy_version 476940 (0.0007) [2023-12-26 18:50:13,141][105692] Updated weights for policy 0, policy_version 476950 (0.0009) [2023-12-26 18:50:13,196][105692] Updated weights for policy 0, policy_version 476960 (0.0010) [2023-12-26 18:50:13,549][105620] Updated weights for policy 1, policy_version 477324 (0.0008) [2023-12-26 18:50:13,594][105620] Updated weights for policy 1, policy_version 477334 (0.0005) [2023-12-26 18:50:13,639][105620] Updated weights for policy 1, policy_version 477344 (0.0005) [2023-12-26 18:50:14,092][105692] Updated weights for policy 0, policy_version 476970 (0.0008) [2023-12-26 18:50:14,162][105692] Updated weights for policy 0, policy_version 476980 (0.0009) [2023-12-26 18:50:14,211][105692] Updated weights for policy 0, policy_version 476990 (0.0009) [2023-12-26 18:50:14,231][105620] Updated weights for policy 1, policy_version 477354 (0.0006) [2023-12-26 18:50:14,288][105620] Updated weights for policy 1, policy_version 477364 (0.0005) [2023-12-26 18:50:14,357][105620] Updated weights for policy 1, policy_version 477374 (0.0005) [2023-12-26 18:50:14,426][105620] Updated weights for policy 1, policy_version 477384 (0.0005) [2023-12-26 18:50:14,980][105692] Updated weights for policy 0, policy_version 477000 (0.0007) [2023-12-26 18:50:15,044][105692] Updated weights for policy 0, policy_version 477010 (0.0007) [2023-12-26 18:50:15,045][105620] Updated weights for policy 1, policy_version 477394 (0.0009) [2023-12-26 18:50:15,106][105692] Updated weights for policy 0, policy_version 477020 (0.0008) [2023-12-26 18:50:15,116][105620] Updated weights for policy 1, policy_version 477404 (0.0009) [2023-12-26 18:50:15,179][105620] Updated weights for policy 1, policy_version 477414 (0.0009) [2023-12-26 18:50:15,732][105692] Updated weights for policy 0, policy_version 477030 (0.0006) [2023-12-26 18:50:15,779][105692] Updated weights for policy 0, policy_version 477040 (0.0005) [2023-12-26 18:50:15,837][105692] Updated weights for policy 0, policy_version 477050 (0.0006) [2023-12-26 18:50:15,900][105620] Updated weights for policy 1, policy_version 477424 (0.0007) [2023-12-26 18:50:15,961][105620] Updated weights for policy 1, policy_version 477434 (0.0005) [2023-12-26 18:50:16,022][105620] Updated weights for policy 1, policy_version 477444 (0.0007) [2023-12-26 18:50:16,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19933.9, 300 sec: 19549.7). Total num frames: 244383744. Throughput: 0: 9849.5, 1: 9913.4. Samples: 244348164. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 18:50:16,063][104569] Avg episode reward: [(0, '8274.110'), (1, '9173.337')] [2023-12-26 18:50:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000477056_122142720.pth... [2023-12-26 18:50:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000477448_122241024.pth... [2023-12-26 18:50:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000475936_121856000.pth [2023-12-26 18:50:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000476296_121946112.pth [2023-12-26 18:50:16,372][105692] Updated weights for policy 0, policy_version 477060 (0.0005) [2023-12-26 18:50:16,425][105692] Updated weights for policy 0, policy_version 477070 (0.0005) [2023-12-26 18:50:16,482][105692] Updated weights for policy 0, policy_version 477080 (0.0010) [2023-12-26 18:50:16,799][105620] Updated weights for policy 1, policy_version 477455 (0.0010) [2023-12-26 18:50:16,852][105620] Updated weights for policy 1, policy_version 477465 (0.0010) [2023-12-26 18:50:16,911][105620] Updated weights for policy 1, policy_version 477475 (0.0010) [2023-12-26 18:50:17,030][105692] Updated weights for policy 0, policy_version 477090 (0.0011) [2023-12-26 18:50:17,078][105692] Updated weights for policy 0, policy_version 477100 (0.0010) [2023-12-26 18:50:17,129][105692] Updated weights for policy 0, policy_version 477110 (0.0010) [2023-12-26 18:50:17,191][105692] Updated weights for policy 0, policy_version 477120 (0.0007) [2023-12-26 18:50:17,722][105620] Updated weights for policy 1, policy_version 477485 (0.0008) [2023-12-26 18:50:17,777][105620] Updated weights for policy 1, policy_version 477495 (0.0009) [2023-12-26 18:50:17,835][105620] Updated weights for policy 1, policy_version 477505 (0.0009) [2023-12-26 18:50:17,868][105692] Updated weights for policy 0, policy_version 477130 (0.0005) [2023-12-26 18:50:17,916][105692] Updated weights for policy 0, policy_version 477140 (0.0005) [2023-12-26 18:50:17,966][105692] Updated weights for policy 0, policy_version 477150 (0.0009) [2023-12-26 18:50:18,556][105620] Updated weights for policy 1, policy_version 477515 (0.0007) [2023-12-26 18:50:18,604][105620] Updated weights for policy 1, policy_version 477525 (0.0008) [2023-12-26 18:50:18,656][105620] Updated weights for policy 1, policy_version 477535 (0.0009) [2023-12-26 18:50:18,676][105692] Updated weights for policy 0, policy_version 477160 (0.0009) [2023-12-26 18:50:18,733][105692] Updated weights for policy 0, policy_version 477170 (0.0010) [2023-12-26 18:50:18,801][105692] Updated weights for policy 0, policy_version 477180 (0.0010) [2023-12-26 18:50:19,370][105620] Updated weights for policy 1, policy_version 477545 (0.0010) [2023-12-26 18:50:19,430][105620] Updated weights for policy 1, policy_version 477555 (0.0006) [2023-12-26 18:50:19,497][105620] Updated weights for policy 1, policy_version 477565 (0.0007) [2023-12-26 18:50:19,551][105692] Updated weights for policy 0, policy_version 477190 (0.0011) [2023-12-26 18:50:19,561][105620] Updated weights for policy 1, policy_version 477575 (0.0009) [2023-12-26 18:50:19,603][105692] Updated weights for policy 0, policy_version 477200 (0.0009) [2023-12-26 18:50:19,616][105585] KL-divergence is very high: 133.3842 [2023-12-26 18:50:19,661][105692] Updated weights for policy 0, policy_version 477210 (0.0006) [2023-12-26 18:50:19,663][105585] KL-divergence is very high: 159.0783 [2023-12-26 18:50:19,670][105585] KL-divergence is very high: 103.8844 [2023-12-26 18:50:20,297][105620] Updated weights for policy 1, policy_version 477585 (0.0011) [2023-12-26 18:50:20,339][105692] Updated weights for policy 0, policy_version 477220 (0.0005) [2023-12-26 18:50:20,357][105620] Updated weights for policy 1, policy_version 477595 (0.0011) [2023-12-26 18:50:20,398][105692] Updated weights for policy 0, policy_version 477230 (0.0006) [2023-12-26 18:50:20,412][105620] Updated weights for policy 1, policy_version 477605 (0.0011) [2023-12-26 18:50:20,453][105692] Updated weights for policy 0, policy_version 477240 (0.0006) [2023-12-26 18:50:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 244473856. Throughput: 0: 9821.7, 1: 9751.1. Samples: 244466816. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 18:50:21,062][104569] Avg episode reward: [(0, '8364.217'), (1, '9173.771')] [2023-12-26 18:50:21,178][105620] Updated weights for policy 1, policy_version 477615 (0.0009) [2023-12-26 18:50:21,180][105692] Updated weights for policy 0, policy_version 477250 (0.0010) [2023-12-26 18:50:21,237][105692] Updated weights for policy 0, policy_version 477260 (0.0010) [2023-12-26 18:50:21,239][105620] Updated weights for policy 1, policy_version 477625 (0.0006) [2023-12-26 18:50:21,293][105620] Updated weights for policy 1, policy_version 477635 (0.0008) [2023-12-26 18:50:21,304][105692] Updated weights for policy 0, policy_version 477270 (0.0011) [2023-12-26 18:50:21,369][105692] Updated weights for policy 0, policy_version 477280 (0.0013) [2023-12-26 18:50:21,974][105620] Updated weights for policy 1, policy_version 477645 (0.0006) [2023-12-26 18:50:22,034][105620] Updated weights for policy 1, policy_version 477655 (0.0007) [2023-12-26 18:50:22,099][105620] Updated weights for policy 1, policy_version 477665 (0.0010) [2023-12-26 18:50:22,155][105692] Updated weights for policy 0, policy_version 477290 (0.0005) [2023-12-26 18:50:22,218][105692] Updated weights for policy 0, policy_version 477300 (0.0005) [2023-12-26 18:50:22,283][105692] Updated weights for policy 0, policy_version 477310 (0.0006) [2023-12-26 18:50:22,784][105620] Updated weights for policy 1, policy_version 477675 (0.0009) [2023-12-26 18:50:22,834][105620] Updated weights for policy 1, policy_version 477685 (0.0008) [2023-12-26 18:50:22,894][105620] Updated weights for policy 1, policy_version 477695 (0.0008) [2023-12-26 18:50:22,974][105692] Updated weights for policy 0, policy_version 477320 (0.0010) [2023-12-26 18:50:23,020][105692] Updated weights for policy 0, policy_version 477330 (0.0010) [2023-12-26 18:50:23,079][105692] Updated weights for policy 0, policy_version 477340 (0.0010) [2023-12-26 18:50:23,701][105692] Updated weights for policy 0, policy_version 477350 (0.0007) [2023-12-26 18:50:23,754][105620] Updated weights for policy 1, policy_version 477705 (0.0008) [2023-12-26 18:50:23,759][105692] Updated weights for policy 0, policy_version 477360 (0.0005) [2023-12-26 18:50:23,810][105620] Updated weights for policy 1, policy_version 477715 (0.0009) [2023-12-26 18:50:23,813][105692] Updated weights for policy 0, policy_version 477370 (0.0005) [2023-12-26 18:50:23,868][105620] Updated weights for policy 1, policy_version 477725 (0.0008) [2023-12-26 18:50:23,921][105620] Updated weights for policy 1, policy_version 477736 (0.0010) [2023-12-26 18:50:24,325][105692] Updated weights for policy 0, policy_version 477380 (0.0006) [2023-12-26 18:50:24,376][105692] Updated weights for policy 0, policy_version 477390 (0.0005) [2023-12-26 18:50:24,433][105692] Updated weights for policy 0, policy_version 477400 (0.0005) [2023-12-26 18:50:24,682][105620] Updated weights for policy 1, policy_version 477746 (0.0008) [2023-12-26 18:50:24,728][105620] Updated weights for policy 1, policy_version 477756 (0.0007) [2023-12-26 18:50:24,781][105620] Updated weights for policy 1, policy_version 477766 (0.0005) [2023-12-26 18:50:25,030][105692] Updated weights for policy 0, policy_version 477410 (0.0006) [2023-12-26 18:50:25,078][105692] Updated weights for policy 0, policy_version 477420 (0.0010) [2023-12-26 18:50:25,133][105692] Updated weights for policy 0, policy_version 477430 (0.0010) [2023-12-26 18:50:25,182][105692] Updated weights for policy 0, policy_version 477440 (0.0006) [2023-12-26 18:50:25,467][105620] Updated weights for policy 1, policy_version 477776 (0.0005) [2023-12-26 18:50:25,525][105620] Updated weights for policy 1, policy_version 477786 (0.0009) [2023-12-26 18:50:25,583][105620] Updated weights for policy 1, policy_version 477797 (0.0010) [2023-12-26 18:50:25,724][105692] Updated weights for policy 0, policy_version 477450 (0.0005) [2023-12-26 18:50:25,770][105585] KL-divergence is very high: 279.4123 [2023-12-26 18:50:25,779][105692] Updated weights for policy 0, policy_version 477460 (0.0005) [2023-12-26 18:50:25,787][105585] KL-divergence is very high: 106.1317 [2023-12-26 18:50:25,816][105585] KL-divergence is very high: 407.3292 [2023-12-26 18:50:25,838][105692] Updated weights for policy 0, policy_version 477470 (0.0005) [2023-12-26 18:50:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 244580352. Throughput: 0: 9857.8, 1: 9688.8. Samples: 244586844. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 18:50:26,063][104569] Avg episode reward: [(0, '8820.500'), (1, '9082.771')] [2023-12-26 18:50:26,340][105620] Updated weights for policy 1, policy_version 477807 (0.0009) [2023-12-26 18:50:26,351][105585] KL-divergence is very high: 165.3384 [2023-12-26 18:50:26,387][105692] Updated weights for policy 0, policy_version 477480 (0.0005) [2023-12-26 18:50:26,398][105585] KL-divergence is very high: 138.9318 [2023-12-26 18:50:26,403][105620] Updated weights for policy 1, policy_version 477817 (0.0008) [2023-12-26 18:50:26,439][105692] Updated weights for policy 0, policy_version 477490 (0.0005) [2023-12-26 18:50:26,440][105585] KL-divergence is very high: 109.0787 [2023-12-26 18:50:26,458][105620] Updated weights for policy 1, policy_version 477827 (0.0008) [2023-12-26 18:50:26,492][105692] Updated weights for policy 0, policy_version 477500 (0.0005) [2023-12-26 18:50:27,028][105692] Updated weights for policy 0, policy_version 477510 (0.0007) [2023-12-26 18:50:27,099][105692] Updated weights for policy 0, policy_version 477520 (0.0010) [2023-12-26 18:50:27,166][105692] Updated weights for policy 0, policy_version 477530 (0.0011) [2023-12-26 18:50:27,201][105620] Updated weights for policy 1, policy_version 477837 (0.0006) [2023-12-26 18:50:27,254][105620] Updated weights for policy 1, policy_version 477847 (0.0007) [2023-12-26 18:50:27,308][105620] Updated weights for policy 1, policy_version 477857 (0.0010) [2023-12-26 18:50:27,820][105692] Updated weights for policy 0, policy_version 477540 (0.0009) [2023-12-26 18:50:27,863][105620] Updated weights for policy 1, policy_version 477867 (0.0007) [2023-12-26 18:50:27,873][105692] Updated weights for policy 0, policy_version 477550 (0.0007) [2023-12-26 18:50:27,916][105692] Updated weights for policy 0, policy_version 477560 (0.0006) [2023-12-26 18:50:27,917][105620] Updated weights for policy 1, policy_version 477877 (0.0005) [2023-12-26 18:50:27,961][105620] Updated weights for policy 1, policy_version 477887 (0.0005) [2023-12-26 18:50:28,519][105692] Updated weights for policy 0, policy_version 477570 (0.0006) [2023-12-26 18:50:28,581][105692] Updated weights for policy 0, policy_version 477580 (0.0009) [2023-12-26 18:50:28,644][105692] Updated weights for policy 0, policy_version 477590 (0.0009) [2023-12-26 18:50:28,663][105620] Updated weights for policy 1, policy_version 477897 (0.0007) [2023-12-26 18:50:28,706][105692] Updated weights for policy 0, policy_version 477600 (0.0008) [2023-12-26 18:50:28,720][105620] Updated weights for policy 1, policy_version 477907 (0.0007) [2023-12-26 18:50:28,771][105620] Updated weights for policy 1, policy_version 477917 (0.0009) [2023-12-26 18:50:28,818][105620] Updated weights for policy 1, policy_version 477927 (0.0008) [2023-12-26 18:50:29,415][105692] Updated weights for policy 0, policy_version 477610 (0.0008) [2023-12-26 18:50:29,474][105692] Updated weights for policy 0, policy_version 477620 (0.0008) [2023-12-26 18:50:29,532][105692] Updated weights for policy 0, policy_version 477630 (0.0009) [2023-12-26 18:50:29,602][105620] Updated weights for policy 1, policy_version 477937 (0.0010) [2023-12-26 18:50:29,647][105620] Updated weights for policy 1, policy_version 477947 (0.0010) [2023-12-26 18:50:29,698][105620] Updated weights for policy 1, policy_version 477957 (0.0010) [2023-12-26 18:50:30,300][105692] Updated weights for policy 0, policy_version 477640 (0.0008) [2023-12-26 18:50:30,356][105692] Updated weights for policy 0, policy_version 477650 (0.0008) [2023-12-26 18:50:30,405][105692] Updated weights for policy 0, policy_version 477660 (0.0006) [2023-12-26 18:50:30,480][105620] Updated weights for policy 1, policy_version 477967 (0.0009) [2023-12-26 18:50:30,540][105620] Updated weights for policy 1, policy_version 477977 (0.0006) [2023-12-26 18:50:30,591][105620] Updated weights for policy 1, policy_version 477987 (0.0009) [2023-12-26 18:50:31,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.3, 300 sec: 19521.9). Total num frames: 244678656. Throughput: 0: 10022.4, 1: 9704.7. Samples: 244652144. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 18:50:31,063][104569] Avg episode reward: [(0, '9087.825'), (1, '9082.230')] [2023-12-26 18:50:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000477992_122380288.pth... [2023-12-26 18:50:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000477664_122298368.pth... [2023-12-26 18:50:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000476872_122093568.pth [2023-12-26 18:50:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000476480_121995264.pth [2023-12-26 18:50:31,193][105620] Updated weights for policy 1, policy_version 477997 (0.0009) [2023-12-26 18:50:31,219][105692] Updated weights for policy 0, policy_version 477670 (0.0006) [2023-12-26 18:50:31,253][105620] Updated weights for policy 1, policy_version 478007 (0.0009) [2023-12-26 18:50:31,277][105692] Updated weights for policy 0, policy_version 477680 (0.0007) [2023-12-26 18:50:31,308][105620] Updated weights for policy 1, policy_version 478017 (0.0008) [2023-12-26 18:50:31,338][105692] Updated weights for policy 0, policy_version 477690 (0.0007) [2023-12-26 18:50:32,080][105692] Updated weights for policy 0, policy_version 477700 (0.0008) [2023-12-26 18:50:32,090][105620] Updated weights for policy 1, policy_version 478027 (0.0007) [2023-12-26 18:50:32,144][105692] Updated weights for policy 0, policy_version 477710 (0.0009) [2023-12-26 18:50:32,150][105620] Updated weights for policy 1, policy_version 478037 (0.0005) [2023-12-26 18:50:32,204][105692] Updated weights for policy 0, policy_version 477720 (0.0008) [2023-12-26 18:50:32,211][105620] Updated weights for policy 1, policy_version 478047 (0.0007) [2023-12-26 18:50:32,901][105692] Updated weights for policy 0, policy_version 477730 (0.0007) [2023-12-26 18:50:32,949][105692] Updated weights for policy 0, policy_version 477740 (0.0008) [2023-12-26 18:50:32,960][105620] Updated weights for policy 1, policy_version 478057 (0.0007) [2023-12-26 18:50:32,995][105692] Updated weights for policy 0, policy_version 477750 (0.0006) [2023-12-26 18:50:33,017][105620] Updated weights for policy 1, policy_version 478067 (0.0009) [2023-12-26 18:50:33,041][105692] Updated weights for policy 0, policy_version 477760 (0.0005) [2023-12-26 18:50:33,078][105620] Updated weights for policy 1, policy_version 478077 (0.0009) [2023-12-26 18:50:33,138][105620] Updated weights for policy 1, policy_version 478087 (0.0009) [2023-12-26 18:50:33,654][105692] Updated weights for policy 0, policy_version 477770 (0.0005) [2023-12-26 18:50:33,711][105692] Updated weights for policy 0, policy_version 477780 (0.0005) [2023-12-26 18:50:33,766][105692] Updated weights for policy 0, policy_version 477790 (0.0005) [2023-12-26 18:50:33,986][105620] Updated weights for policy 1, policy_version 478097 (0.0008) [2023-12-26 18:50:34,039][105620] Updated weights for policy 1, policy_version 478107 (0.0008) [2023-12-26 18:50:34,100][105620] Updated weights for policy 1, policy_version 478117 (0.0009) [2023-12-26 18:50:34,395][105692] Updated weights for policy 0, policy_version 477800 (0.0009) [2023-12-26 18:50:34,458][105692] Updated weights for policy 0, policy_version 477810 (0.0009) [2023-12-26 18:50:34,516][105692] Updated weights for policy 0, policy_version 477820 (0.0009) [2023-12-26 18:50:34,879][105620] Updated weights for policy 1, policy_version 478127 (0.0008) [2023-12-26 18:50:34,937][105620] Updated weights for policy 1, policy_version 478137 (0.0009) [2023-12-26 18:50:34,998][105620] Updated weights for policy 1, policy_version 478147 (0.0012) [2023-12-26 18:50:35,153][105692] Updated weights for policy 0, policy_version 477830 (0.0008) [2023-12-26 18:50:35,215][105692] Updated weights for policy 0, policy_version 477840 (0.0009) [2023-12-26 18:50:35,272][105692] Updated weights for policy 0, policy_version 477850 (0.0009) [2023-12-26 18:50:35,772][105620] Updated weights for policy 1, policy_version 478157 (0.0009) [2023-12-26 18:50:35,822][105620] Updated weights for policy 1, policy_version 478167 (0.0008) [2023-12-26 18:50:35,870][105620] Updated weights for policy 1, policy_version 478177 (0.0009) [2023-12-26 18:50:36,033][105692] Updated weights for policy 0, policy_version 477860 (0.0010) [2023-12-26 18:50:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 244776960. Throughput: 0: 10068.6, 1: 9695.2. Samples: 244766700. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 18:50:36,062][104569] Avg episode reward: [(0, '9088.043'), (1, '9173.878')] [2023-12-26 18:50:36,086][105692] Updated weights for policy 0, policy_version 477870 (0.0009) [2023-12-26 18:50:36,145][105692] Updated weights for policy 0, policy_version 477880 (0.0008) [2023-12-26 18:50:36,599][105620] Updated weights for policy 1, policy_version 478187 (0.0008) [2023-12-26 18:50:36,657][105620] Updated weights for policy 1, policy_version 478197 (0.0008) [2023-12-26 18:50:36,712][105620] Updated weights for policy 1, policy_version 478207 (0.0005) [2023-12-26 18:50:36,921][105692] Updated weights for policy 0, policy_version 477890 (0.0006) [2023-12-26 18:50:36,982][105692] Updated weights for policy 0, policy_version 477900 (0.0005) [2023-12-26 18:50:37,048][105692] Updated weights for policy 0, policy_version 477910 (0.0005) [2023-12-26 18:50:37,107][105692] Updated weights for policy 0, policy_version 477920 (0.0005) [2023-12-26 18:50:37,369][105620] Updated weights for policy 1, policy_version 478217 (0.0005) [2023-12-26 18:50:37,430][105620] Updated weights for policy 1, policy_version 478227 (0.0007) [2023-12-26 18:50:37,488][105620] Updated weights for policy 1, policy_version 478237 (0.0009) [2023-12-26 18:50:37,547][105620] Updated weights for policy 1, policy_version 478247 (0.0009) [2023-12-26 18:50:37,758][105692] Updated weights for policy 0, policy_version 477930 (0.0009) [2023-12-26 18:50:37,817][105692] Updated weights for policy 0, policy_version 477940 (0.0009) [2023-12-26 18:50:37,875][105692] Updated weights for policy 0, policy_version 477950 (0.0008) [2023-12-26 18:50:38,286][105620] Updated weights for policy 1, policy_version 478257 (0.0009) [2023-12-26 18:50:38,350][105620] Updated weights for policy 1, policy_version 478267 (0.0009) [2023-12-26 18:50:38,412][105620] Updated weights for policy 1, policy_version 478277 (0.0008) [2023-12-26 18:50:38,638][105692] Updated weights for policy 0, policy_version 477960 (0.0009) [2023-12-26 18:50:38,695][105692] Updated weights for policy 0, policy_version 477970 (0.0008) [2023-12-26 18:50:38,754][105692] Updated weights for policy 0, policy_version 477980 (0.0008) [2023-12-26 18:50:39,114][105620] Updated weights for policy 1, policy_version 478287 (0.0009) [2023-12-26 18:50:39,174][105620] Updated weights for policy 1, policy_version 478297 (0.0009) [2023-12-26 18:50:39,240][105620] Updated weights for policy 1, policy_version 478307 (0.0009) [2023-12-26 18:50:39,579][105692] Updated weights for policy 0, policy_version 477990 (0.0009) [2023-12-26 18:50:39,654][105692] Updated weights for policy 0, policy_version 478000 (0.0008) [2023-12-26 18:50:39,724][105692] Updated weights for policy 0, policy_version 478010 (0.0005) [2023-12-26 18:50:40,021][105620] Updated weights for policy 1, policy_version 478317 (0.0009) [2023-12-26 18:50:40,076][105620] Updated weights for policy 1, policy_version 478327 (0.0009) [2023-12-26 18:50:40,134][105620] Updated weights for policy 1, policy_version 478337 (0.0010) [2023-12-26 18:50:40,367][105692] Updated weights for policy 0, policy_version 478020 (0.0005) [2023-12-26 18:50:40,424][105692] Updated weights for policy 0, policy_version 478030 (0.0008) [2023-12-26 18:50:40,491][105692] Updated weights for policy 0, policy_version 478040 (0.0010) [2023-12-26 18:50:40,896][105620] Updated weights for policy 1, policy_version 478347 (0.0009) [2023-12-26 18:50:40,955][105620] Updated weights for policy 1, policy_version 478357 (0.0009) [2023-12-26 18:50:41,005][105620] Updated weights for policy 1, policy_version 478367 (0.0009) [2023-12-26 18:50:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 244867072. Throughput: 0: 10085.4, 1: 9659.1. Samples: 244881768. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 18:50:41,063][104569] Avg episode reward: [(0, '9267.532'), (1, '9176.381')] [2023-12-26 18:50:41,184][105692] Updated weights for policy 0, policy_version 478050 (0.0009) [2023-12-26 18:50:41,237][105692] Updated weights for policy 0, policy_version 478060 (0.0008) [2023-12-26 18:50:41,302][105692] Updated weights for policy 0, policy_version 478070 (0.0009) [2023-12-26 18:50:41,361][105692] Updated weights for policy 0, policy_version 478080 (0.0008) [2023-12-26 18:50:41,767][105620] Updated weights for policy 1, policy_version 478377 (0.0008) [2023-12-26 18:50:41,822][105620] Updated weights for policy 1, policy_version 478387 (0.0009) [2023-12-26 18:50:41,877][105620] Updated weights for policy 1, policy_version 478397 (0.0009) [2023-12-26 18:50:41,929][105620] Updated weights for policy 1, policy_version 478407 (0.0009) [2023-12-26 18:50:42,165][105692] Updated weights for policy 0, policy_version 478090 (0.0008) [2023-12-26 18:50:42,221][105692] Updated weights for policy 0, policy_version 478100 (0.0008) [2023-12-26 18:50:42,274][105692] Updated weights for policy 0, policy_version 478110 (0.0009) [2023-12-26 18:50:42,666][105620] Updated weights for policy 1, policy_version 478417 (0.0011) [2023-12-26 18:50:42,717][105620] Updated weights for policy 1, policy_version 478427 (0.0008) [2023-12-26 18:50:42,769][105620] Updated weights for policy 1, policy_version 478437 (0.0005) [2023-12-26 18:50:43,147][105692] Updated weights for policy 0, policy_version 478120 (0.0008) [2023-12-26 18:50:43,196][105692] Updated weights for policy 0, policy_version 478130 (0.0008) [2023-12-26 18:50:43,250][105692] Updated weights for policy 0, policy_version 478140 (0.0008) [2023-12-26 18:50:43,427][105620] Updated weights for policy 1, policy_version 478447 (0.0006) [2023-12-26 18:50:43,492][105620] Updated weights for policy 1, policy_version 478457 (0.0006) [2023-12-26 18:50:43,538][105620] Updated weights for policy 1, policy_version 478467 (0.0008) [2023-12-26 18:50:44,020][105692] Updated weights for policy 0, policy_version 478150 (0.0009) [2023-12-26 18:50:44,078][105692] Updated weights for policy 0, policy_version 478160 (0.0009) [2023-12-26 18:50:44,087][105620] Updated weights for policy 1, policy_version 478477 (0.0007) [2023-12-26 18:50:44,133][105692] Updated weights for policy 0, policy_version 478170 (0.0007) [2023-12-26 18:50:44,135][105620] Updated weights for policy 1, policy_version 478487 (0.0006) [2023-12-26 18:50:44,190][105620] Updated weights for policy 1, policy_version 478497 (0.0007) [2023-12-26 18:50:44,791][105692] Updated weights for policy 0, policy_version 478180 (0.0007) [2023-12-26 18:50:44,842][105692] Updated weights for policy 0, policy_version 478190 (0.0009) [2023-12-26 18:50:44,904][105692] Updated weights for policy 0, policy_version 478200 (0.0008) [2023-12-26 18:50:44,970][105620] Updated weights for policy 1, policy_version 478507 (0.0008) [2023-12-26 18:50:44,979][105586] KL-divergence is very high: 128.8992 [2023-12-26 18:50:45,024][105620] Updated weights for policy 1, policy_version 478517 (0.0009) [2023-12-26 18:50:45,061][105586] KL-divergence is very high: 102.0426 [2023-12-26 18:50:45,067][105586] KL-divergence is very high: 114.5512 [2023-12-26 18:50:45,084][105620] Updated weights for policy 1, policy_version 478527 (0.0010) [2023-12-26 18:50:45,617][105692] Updated weights for policy 0, policy_version 478210 (0.0008) [2023-12-26 18:50:45,681][105692] Updated weights for policy 0, policy_version 478220 (0.0006) [2023-12-26 18:50:45,740][105692] Updated weights for policy 0, policy_version 478230 (0.0005) [2023-12-26 18:50:45,799][105692] Updated weights for policy 0, policy_version 478240 (0.0005) [2023-12-26 18:50:45,917][105620] Updated weights for policy 1, policy_version 478537 (0.0009) [2023-12-26 18:50:45,917][105586] KL-divergence is very high: 265.5279 [2023-12-26 18:50:45,923][105586] KL-divergence is very high: 153.8040 [2023-12-26 18:50:45,928][105586] KL-divergence is very high: 239.9862 [2023-12-26 18:50:45,949][105586] KL-divergence is very high: 148.7442 [2023-12-26 18:50:45,959][105586] KL-divergence is very high: 263.4952 [2023-12-26 18:50:45,965][105586] KL-divergence is very high: 154.4272 [2023-12-26 18:50:45,970][105586] KL-divergence is very high: 204.2974 [2023-12-26 18:50:45,971][105620] Updated weights for policy 1, policy_version 478547 (0.0008) [2023-12-26 18:50:45,991][105586] KL-divergence is very high: 104.7030 [2023-12-26 18:50:46,022][105586] KL-divergence is very high: 101.4363 [2023-12-26 18:50:46,024][105620] Updated weights for policy 1, policy_version 478557 (0.0009) [2023-12-26 18:50:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 244965376. Throughput: 0: 9978.6, 1: 9724.2. Samples: 244940180. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 18:50:46,062][104569] Avg episode reward: [(0, '8816.124'), (1, '1764.158')] [2023-12-26 18:50:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000478240_122445824.pth... [2023-12-26 18:50:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000477056_122142720.pth [2023-12-26 18:50:46,086][105620] Updated weights for policy 1, policy_version 478567 (0.0009) [2023-12-26 18:50:46,089][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000478568_122527744.pth... [2023-12-26 18:50:46,092][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000477448_122241024.pth [2023-12-26 18:50:46,468][105692] Updated weights for policy 0, policy_version 478250 (0.0008) [2023-12-26 18:50:46,520][105692] Updated weights for policy 0, policy_version 478260 (0.0009) [2023-12-26 18:50:46,569][105692] Updated weights for policy 0, policy_version 478270 (0.0009) [2023-12-26 18:50:46,696][105620] Updated weights for policy 1, policy_version 478577 (0.0009) [2023-12-26 18:50:46,751][105620] Updated weights for policy 1, policy_version 478587 (0.0008) [2023-12-26 18:50:46,798][105620] Updated weights for policy 1, policy_version 478597 (0.0009) [2023-12-26 18:50:47,356][105692] Updated weights for policy 0, policy_version 478280 (0.0009) [2023-12-26 18:50:47,418][105692] Updated weights for policy 0, policy_version 478290 (0.0009) [2023-12-26 18:50:47,483][105692] Updated weights for policy 0, policy_version 478300 (0.0009) [2023-12-26 18:50:47,567][105620] Updated weights for policy 1, policy_version 478607 (0.0009) [2023-12-26 18:50:47,626][105620] Updated weights for policy 1, policy_version 478617 (0.0009) [2023-12-26 18:50:47,665][105586] KL-divergence is very high: 109.6234 [2023-12-26 18:50:47,676][105620] Updated weights for policy 1, policy_version 478627 (0.0009) [2023-12-26 18:50:48,266][105692] Updated weights for policy 0, policy_version 478310 (0.0009) [2023-12-26 18:50:48,320][105692] Updated weights for policy 0, policy_version 478320 (0.0008) [2023-12-26 18:50:48,356][105620] Updated weights for policy 1, policy_version 478637 (0.0009) [2023-12-26 18:50:48,387][105692] Updated weights for policy 0, policy_version 478330 (0.0008) [2023-12-26 18:50:48,413][105620] Updated weights for policy 1, policy_version 478647 (0.0008) [2023-12-26 18:50:48,475][105620] Updated weights for policy 1, policy_version 478657 (0.0010) [2023-12-26 18:50:49,132][105620] Updated weights for policy 1, policy_version 478667 (0.0010) [2023-12-26 18:50:49,169][105692] Updated weights for policy 0, policy_version 478340 (0.0009) [2023-12-26 18:50:49,185][105620] Updated weights for policy 1, policy_version 478677 (0.0009) [2023-12-26 18:50:49,232][105692] Updated weights for policy 0, policy_version 478350 (0.0009) [2023-12-26 18:50:49,243][105620] Updated weights for policy 1, policy_version 478687 (0.0007) [2023-12-26 18:50:49,287][105692] Updated weights for policy 0, policy_version 478360 (0.0007) [2023-12-26 18:50:49,940][105692] Updated weights for policy 0, policy_version 478370 (0.0008) [2023-12-26 18:50:49,994][105692] Updated weights for policy 0, policy_version 478380 (0.0007) [2023-12-26 18:50:50,050][105692] Updated weights for policy 0, policy_version 478390 (0.0008) [2023-12-26 18:50:50,082][105620] Updated weights for policy 1, policy_version 478697 (0.0007) [2023-12-26 18:50:50,101][105692] Updated weights for policy 0, policy_version 478400 (0.0007) [2023-12-26 18:50:50,138][105620] Updated weights for policy 1, policy_version 478707 (0.0008) [2023-12-26 18:50:50,201][105620] Updated weights for policy 1, policy_version 478717 (0.0009) [2023-12-26 18:50:50,258][105620] Updated weights for policy 1, policy_version 478727 (0.0008) [2023-12-26 18:50:50,859][105692] Updated weights for policy 0, policy_version 478410 (0.0011) [2023-12-26 18:50:50,917][105692] Updated weights for policy 0, policy_version 478420 (0.0008) [2023-12-26 18:50:50,957][105620] Updated weights for policy 1, policy_version 478737 (0.0006) [2023-12-26 18:50:50,982][105692] Updated weights for policy 0, policy_version 478430 (0.0010) [2023-12-26 18:50:51,015][105620] Updated weights for policy 1, policy_version 478747 (0.0007) [2023-12-26 18:50:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 245063680. Throughput: 0: 9983.8, 1: 9641.0. Samples: 245054068. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 18:50:51,063][104569] Avg episode reward: [(0, '8728.675'), (1, '1200.376')] [2023-12-26 18:50:51,083][105620] Updated weights for policy 1, policy_version 478757 (0.0007) [2023-12-26 18:50:51,603][105692] Updated weights for policy 0, policy_version 478440 (0.0011) [2023-12-26 18:50:51,667][105692] Updated weights for policy 0, policy_version 478450 (0.0014) [2023-12-26 18:50:51,719][105585] KL-divergence is very high: 142.2454 [2023-12-26 18:50:51,732][105692] Updated weights for policy 0, policy_version 478460 (0.0008) [2023-12-26 18:50:51,894][105620] Updated weights for policy 1, policy_version 478767 (0.0009) [2023-12-26 18:50:51,957][105620] Updated weights for policy 1, policy_version 478777 (0.0006) [2023-12-26 18:50:52,031][105620] Updated weights for policy 1, policy_version 478787 (0.0009) [2023-12-26 18:50:52,351][105692] Updated weights for policy 0, policy_version 478470 (0.0006) [2023-12-26 18:50:52,424][105692] Updated weights for policy 0, policy_version 478480 (0.0006) [2023-12-26 18:50:52,483][105692] Updated weights for policy 0, policy_version 478490 (0.0010) [2023-12-26 18:50:52,775][105620] Updated weights for policy 1, policy_version 478797 (0.0009) [2023-12-26 18:50:52,835][105620] Updated weights for policy 1, policy_version 478807 (0.0008) [2023-12-26 18:50:52,906][105620] Updated weights for policy 1, policy_version 478817 (0.0008) [2023-12-26 18:50:53,181][105692] Updated weights for policy 0, policy_version 478500 (0.0008) [2023-12-26 18:50:53,239][105692] Updated weights for policy 0, policy_version 478510 (0.0005) [2023-12-26 18:50:53,295][105692] Updated weights for policy 0, policy_version 478520 (0.0005) [2023-12-26 18:50:53,719][105620] Updated weights for policy 1, policy_version 478827 (0.0008) [2023-12-26 18:50:53,778][105620] Updated weights for policy 1, policy_version 478837 (0.0010) [2023-12-26 18:50:53,836][105620] Updated weights for policy 1, policy_version 478847 (0.0009) [2023-12-26 18:50:53,839][105692] Updated weights for policy 0, policy_version 478530 (0.0006) [2023-12-26 18:50:53,887][105692] Updated weights for policy 0, policy_version 478540 (0.0007) [2023-12-26 18:50:53,947][105692] Updated weights for policy 0, policy_version 478550 (0.0007) [2023-12-26 18:50:54,022][105692] Updated weights for policy 0, policy_version 478560 (0.0006) [2023-12-26 18:50:54,609][105620] Updated weights for policy 1, policy_version 478857 (0.0006) [2023-12-26 18:50:54,645][105692] Updated weights for policy 0, policy_version 478570 (0.0006) [2023-12-26 18:50:54,666][105620] Updated weights for policy 1, policy_version 478867 (0.0008) [2023-12-26 18:50:54,710][105692] Updated weights for policy 0, policy_version 478580 (0.0008) [2023-12-26 18:50:54,718][105620] Updated weights for policy 1, policy_version 478877 (0.0011) [2023-12-26 18:50:54,772][105692] Updated weights for policy 0, policy_version 478590 (0.0008) [2023-12-26 18:50:54,774][105620] Updated weights for policy 1, policy_version 478887 (0.0010) [2023-12-26 18:50:55,488][105692] Updated weights for policy 0, policy_version 478600 (0.0010) [2023-12-26 18:50:55,530][105585] KL-divergence is very high: 148.1382 [2023-12-26 18:50:55,541][105620] Updated weights for policy 1, policy_version 478897 (0.0007) [2023-12-26 18:50:55,542][105692] Updated weights for policy 0, policy_version 478610 (0.0010) [2023-12-26 18:50:55,577][105585] KL-divergence is very high: 174.7630 [2023-12-26 18:50:55,593][105620] Updated weights for policy 1, policy_version 478907 (0.0006) [2023-12-26 18:50:55,598][105692] Updated weights for policy 0, policy_version 478620 (0.0010) [2023-12-26 18:50:55,643][105620] Updated weights for policy 1, policy_version 478917 (0.0007) [2023-12-26 18:50:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 245161984. Throughput: 0: 10012.2, 1: 9554.1. Samples: 245171152. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 18:50:56,063][104569] Avg episode reward: [(0, '8726.914'), (1, '6488.717')] [2023-12-26 18:50:56,207][105585] KL-divergence is very high: 160.9373 [2023-12-26 18:50:56,236][105692] Updated weights for policy 0, policy_version 478630 (0.0007) [2023-12-26 18:50:56,255][105585] KL-divergence is very high: 167.6062 [2023-12-26 18:50:56,294][105692] Updated weights for policy 0, policy_version 478640 (0.0009) [2023-12-26 18:50:56,303][105585] KL-divergence is very high: 138.7184 [2023-12-26 18:50:56,342][105620] Updated weights for policy 1, policy_version 478927 (0.0009) [2023-12-26 18:50:56,353][105585] KL-divergence is very high: 119.5137 [2023-12-26 18:50:56,358][105692] Updated weights for policy 0, policy_version 478650 (0.0005) [2023-12-26 18:50:56,406][105620] Updated weights for policy 1, policy_version 478937 (0.0006) [2023-12-26 18:50:56,413][105586] KL-divergence is very high: 108.2741 [2023-12-26 18:50:56,444][105586] KL-divergence is very high: 110.0529 [2023-12-26 18:50:56,461][105586] KL-divergence is very high: 112.6356 [2023-12-26 18:50:56,466][105620] Updated weights for policy 1, policy_version 478947 (0.0008) [2023-12-26 18:50:56,983][105692] Updated weights for policy 0, policy_version 478660 (0.0007) [2023-12-26 18:50:57,041][105692] Updated weights for policy 0, policy_version 478670 (0.0010) [2023-12-26 18:50:57,088][105692] Updated weights for policy 0, policy_version 478680 (0.0010) [2023-12-26 18:50:57,214][105620] Updated weights for policy 1, policy_version 478957 (0.0008) [2023-12-26 18:50:57,259][105620] Updated weights for policy 1, policy_version 478967 (0.0008) [2023-12-26 18:50:57,312][105620] Updated weights for policy 1, policy_version 478977 (0.0008) [2023-12-26 18:50:57,827][105692] Updated weights for policy 0, policy_version 478690 (0.0010) [2023-12-26 18:50:57,887][105692] Updated weights for policy 0, policy_version 478700 (0.0010) [2023-12-26 18:50:57,951][105692] Updated weights for policy 0, policy_version 478710 (0.0010) [2023-12-26 18:50:58,015][105692] Updated weights for policy 0, policy_version 478720 (0.0010) [2023-12-26 18:50:58,107][105620] Updated weights for policy 1, policy_version 478987 (0.0008) [2023-12-26 18:50:58,172][105620] Updated weights for policy 1, policy_version 478997 (0.0009) [2023-12-26 18:50:58,226][105620] Updated weights for policy 1, policy_version 479007 (0.0008) [2023-12-26 18:50:58,745][105692] Updated weights for policy 0, policy_version 478730 (0.0008) [2023-12-26 18:50:58,812][105692] Updated weights for policy 0, policy_version 478740 (0.0008) [2023-12-26 18:50:58,877][105692] Updated weights for policy 0, policy_version 478750 (0.0009) [2023-12-26 18:50:59,030][105620] Updated weights for policy 1, policy_version 479017 (0.0008) [2023-12-26 18:50:59,084][105620] Updated weights for policy 1, policy_version 479027 (0.0010) [2023-12-26 18:50:59,132][105620] Updated weights for policy 1, policy_version 479037 (0.0010) [2023-12-26 18:50:59,186][105620] Updated weights for policy 1, policy_version 479047 (0.0007) [2023-12-26 18:50:59,599][105692] Updated weights for policy 0, policy_version 478760 (0.0009) [2023-12-26 18:50:59,657][105692] Updated weights for policy 0, policy_version 478770 (0.0007) [2023-12-26 18:50:59,711][105692] Updated weights for policy 0, policy_version 478780 (0.0005) [2023-12-26 18:50:59,887][105620] Updated weights for policy 1, policy_version 479057 (0.0006) [2023-12-26 18:50:59,960][105620] Updated weights for policy 1, policy_version 479067 (0.0008) [2023-12-26 18:51:00,028][105620] Updated weights for policy 1, policy_version 479077 (0.0007) [2023-12-26 18:51:00,377][105692] Updated weights for policy 0, policy_version 478790 (0.0008) [2023-12-26 18:51:00,438][105692] Updated weights for policy 0, policy_version 478800 (0.0007) [2023-12-26 18:51:00,493][105692] Updated weights for policy 0, policy_version 478810 (0.0009) [2023-12-26 18:51:00,763][105620] Updated weights for policy 1, policy_version 479087 (0.0009) [2023-12-26 18:51:00,816][105620] Updated weights for policy 1, policy_version 479097 (0.0009) [2023-12-26 18:51:00,862][105620] Updated weights for policy 1, policy_version 479107 (0.0008) [2023-12-26 18:51:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 245260288. Throughput: 0: 10081.5, 1: 9519.3. Samples: 245230200. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 18:51:01,063][104569] Avg episode reward: [(0, '8728.896'), (1, '6854.335')] [2023-12-26 18:51:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000478816_122593280.pth... [2023-12-26 18:51:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000479112_122667008.pth... [2023-12-26 18:51:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000477664_122298368.pth [2023-12-26 18:51:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000477992_122380288.pth [2023-12-26 18:51:01,130][105692] Updated weights for policy 0, policy_version 478820 (0.0007) [2023-12-26 18:51:01,184][105692] Updated weights for policy 0, policy_version 478830 (0.0009) [2023-12-26 18:51:01,238][105692] Updated weights for policy 0, policy_version 478840 (0.0010) [2023-12-26 18:51:01,568][105620] Updated weights for policy 1, policy_version 479117 (0.0008) [2023-12-26 18:51:01,630][105620] Updated weights for policy 1, policy_version 479127 (0.0009) [2023-12-26 18:51:01,695][105620] Updated weights for policy 1, policy_version 479137 (0.0009) [2023-12-26 18:51:01,946][105692] Updated weights for policy 0, policy_version 478850 (0.0008) [2023-12-26 18:51:02,006][105692] Updated weights for policy 0, policy_version 478860 (0.0008) [2023-12-26 18:51:02,061][105692] Updated weights for policy 0, policy_version 478870 (0.0009) [2023-12-26 18:51:02,116][105692] Updated weights for policy 0, policy_version 478880 (0.0009) [2023-12-26 18:51:02,517][105620] Updated weights for policy 1, policy_version 479147 (0.0010) [2023-12-26 18:51:02,574][105620] Updated weights for policy 1, policy_version 479157 (0.0011) [2023-12-26 18:51:02,633][105620] Updated weights for policy 1, policy_version 479167 (0.0011) [2023-12-26 18:51:02,837][105692] Updated weights for policy 0, policy_version 478890 (0.0008) [2023-12-26 18:51:02,901][105692] Updated weights for policy 0, policy_version 478900 (0.0008) [2023-12-26 18:51:02,949][105692] Updated weights for policy 0, policy_version 478910 (0.0008) [2023-12-26 18:51:03,391][105620] Updated weights for policy 1, policy_version 479177 (0.0011) [2023-12-26 18:51:03,443][105620] Updated weights for policy 1, policy_version 479187 (0.0010) [2023-12-26 18:51:03,501][105620] Updated weights for policy 1, policy_version 479197 (0.0011) [2023-12-26 18:51:03,563][105620] Updated weights for policy 1, policy_version 479207 (0.0011) [2023-12-26 18:51:03,705][105692] Updated weights for policy 0, policy_version 478920 (0.0008) [2023-12-26 18:51:03,750][105692] Updated weights for policy 0, policy_version 478930 (0.0007) [2023-12-26 18:51:03,796][105692] Updated weights for policy 0, policy_version 478940 (0.0008) [2023-12-26 18:51:04,325][105620] Updated weights for policy 1, policy_version 479217 (0.0010) [2023-12-26 18:51:04,374][105620] Updated weights for policy 1, policy_version 479227 (0.0011) [2023-12-26 18:51:04,424][105620] Updated weights for policy 1, policy_version 479237 (0.0010) [2023-12-26 18:51:04,602][105692] Updated weights for policy 0, policy_version 478950 (0.0008) [2023-12-26 18:51:04,654][105692] Updated weights for policy 0, policy_version 478960 (0.0008) [2023-12-26 18:51:04,699][105692] Updated weights for policy 0, policy_version 478970 (0.0008) [2023-12-26 18:51:05,212][105620] Updated weights for policy 1, policy_version 479247 (0.0011) [2023-12-26 18:51:05,263][105620] Updated weights for policy 1, policy_version 479257 (0.0007) [2023-12-26 18:51:05,320][105620] Updated weights for policy 1, policy_version 479267 (0.0006) [2023-12-26 18:51:05,400][105692] Updated weights for policy 0, policy_version 478980 (0.0009) [2023-12-26 18:51:05,453][105692] Updated weights for policy 0, policy_version 478990 (0.0010) [2023-12-26 18:51:05,516][105692] Updated weights for policy 0, policy_version 479000 (0.0010) [2023-12-26 18:51:05,924][105620] Updated weights for policy 1, policy_version 479277 (0.0008) [2023-12-26 18:51:05,978][105620] Updated weights for policy 1, policy_version 479287 (0.0010) [2023-12-26 18:51:06,047][105620] Updated weights for policy 1, policy_version 479297 (0.0010) [2023-12-26 18:51:06,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 245350400. Throughput: 0: 10007.7, 1: 9494.2. Samples: 245344404. Policy #0 lag: (min: 20.0, avg: 22.5, max: 48.0) [2023-12-26 18:51:06,063][104569] Avg episode reward: [(0, '8725.153'), (1, '6778.936')] [2023-12-26 18:51:06,176][105692] Updated weights for policy 0, policy_version 479010 (0.0009) [2023-12-26 18:51:06,232][105692] Updated weights for policy 0, policy_version 479020 (0.0010) [2023-12-26 18:51:06,292][105692] Updated weights for policy 0, policy_version 479030 (0.0010) [2023-12-26 18:51:06,357][105692] Updated weights for policy 0, policy_version 479040 (0.0008) [2023-12-26 18:51:06,672][105620] Updated weights for policy 1, policy_version 479307 (0.0009) [2023-12-26 18:51:06,731][105620] Updated weights for policy 1, policy_version 479317 (0.0006) [2023-12-26 18:51:06,783][105620] Updated weights for policy 1, policy_version 479327 (0.0006) [2023-12-26 18:51:07,192][105692] Updated weights for policy 0, policy_version 479051 (0.0010) [2023-12-26 18:51:07,245][105692] Updated weights for policy 0, policy_version 479061 (0.0010) [2023-12-26 18:51:07,301][105692] Updated weights for policy 0, policy_version 479071 (0.0010) [2023-12-26 18:51:07,424][105620] Updated weights for policy 1, policy_version 479337 (0.0008) [2023-12-26 18:51:07,486][105620] Updated weights for policy 1, policy_version 479347 (0.0007) [2023-12-26 18:51:07,535][105620] Updated weights for policy 1, policy_version 479357 (0.0005) [2023-12-26 18:51:07,590][105620] Updated weights for policy 1, policy_version 479367 (0.0006) [2023-12-26 18:51:08,124][105620] Updated weights for policy 1, policy_version 479377 (0.0009) [2023-12-26 18:51:08,162][105692] Updated weights for policy 0, policy_version 479081 (0.0008) [2023-12-26 18:51:08,180][105620] Updated weights for policy 1, policy_version 479387 (0.0007) [2023-12-26 18:51:08,225][105692] Updated weights for policy 0, policy_version 479091 (0.0006) [2023-12-26 18:51:08,246][105620] Updated weights for policy 1, policy_version 479397 (0.0005) [2023-12-26 18:51:08,296][105692] Updated weights for policy 0, policy_version 479101 (0.0005) [2023-12-26 18:51:08,851][105620] Updated weights for policy 1, policy_version 479407 (0.0005) [2023-12-26 18:51:08,924][105620] Updated weights for policy 1, policy_version 479417 (0.0005) [2023-12-26 18:51:08,972][105620] Updated weights for policy 1, policy_version 479427 (0.0008) [2023-12-26 18:51:09,060][105692] Updated weights for policy 0, policy_version 479111 (0.0008) [2023-12-26 18:51:09,119][105692] Updated weights for policy 0, policy_version 479121 (0.0009) [2023-12-26 18:51:09,126][105585] KL-divergence is very high: 113.2086 [2023-12-26 18:51:09,167][105585] KL-divergence is very high: 176.0261 [2023-12-26 18:51:09,172][105692] Updated weights for policy 0, policy_version 479131 (0.0009) [2023-12-26 18:51:09,651][105620] Updated weights for policy 1, policy_version 479437 (0.0008) [2023-12-26 18:51:09,716][105620] Updated weights for policy 1, policy_version 479447 (0.0008) [2023-12-26 18:51:09,781][105620] Updated weights for policy 1, policy_version 479457 (0.0006) [2023-12-26 18:51:09,892][105692] Updated weights for policy 0, policy_version 479141 (0.0008) [2023-12-26 18:51:09,957][105692] Updated weights for policy 0, policy_version 479151 (0.0009) [2023-12-26 18:51:10,014][105692] Updated weights for policy 0, policy_version 479161 (0.0009) [2023-12-26 18:51:10,508][105620] Updated weights for policy 1, policy_version 479467 (0.0007) [2023-12-26 18:51:10,570][105620] Updated weights for policy 1, policy_version 479477 (0.0010) [2023-12-26 18:51:10,624][105620] Updated weights for policy 1, policy_version 479487 (0.0008) [2023-12-26 18:51:10,818][105692] Updated weights for policy 0, policy_version 479171 (0.0009) [2023-12-26 18:51:10,881][105692] Updated weights for policy 0, policy_version 479181 (0.0009) [2023-12-26 18:51:10,940][105692] Updated weights for policy 0, policy_version 479191 (0.0009) [2023-12-26 18:51:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 245456896. Throughput: 0: 9821.4, 1: 9641.4. Samples: 245462668. Policy #0 lag: (min: 20.0, avg: 22.5, max: 48.0) [2023-12-26 18:51:11,062][104569] Avg episode reward: [(0, '8726.271'), (1, '8662.695')] [2023-12-26 18:51:11,430][105620] Updated weights for policy 1, policy_version 479497 (0.0009) [2023-12-26 18:51:11,482][105620] Updated weights for policy 1, policy_version 479507 (0.0008) [2023-12-26 18:51:11,530][105620] Updated weights for policy 1, policy_version 479517 (0.0008) [2023-12-26 18:51:11,578][105620] Updated weights for policy 1, policy_version 479527 (0.0008) [2023-12-26 18:51:11,719][105692] Updated weights for policy 0, policy_version 479201 (0.0007) [2023-12-26 18:51:11,790][105692] Updated weights for policy 0, policy_version 479211 (0.0009) [2023-12-26 18:51:11,844][105692] Updated weights for policy 0, policy_version 479221 (0.0010) [2023-12-26 18:51:11,904][105692] Updated weights for policy 0, policy_version 479232 (0.0011) [2023-12-26 18:51:12,226][105620] Updated weights for policy 1, policy_version 479537 (0.0008) [2023-12-26 18:51:12,296][105620] Updated weights for policy 1, policy_version 479547 (0.0007) [2023-12-26 18:51:12,354][105620] Updated weights for policy 1, policy_version 479557 (0.0006) [2023-12-26 18:51:12,769][105692] Updated weights for policy 0, policy_version 479242 (0.0008) [2023-12-26 18:51:12,837][105692] Updated weights for policy 0, policy_version 479252 (0.0009) [2023-12-26 18:51:12,890][105692] Updated weights for policy 0, policy_version 479262 (0.0009) [2023-12-26 18:51:13,076][105620] Updated weights for policy 1, policy_version 479567 (0.0010) [2023-12-26 18:51:13,128][105620] Updated weights for policy 1, policy_version 479577 (0.0010) [2023-12-26 18:51:13,173][105620] Updated weights for policy 1, policy_version 479587 (0.0010) [2023-12-26 18:51:13,747][105692] Updated weights for policy 0, policy_version 479272 (0.0009) [2023-12-26 18:51:13,775][105620] Updated weights for policy 1, policy_version 479597 (0.0010) [2023-12-26 18:51:13,805][105692] Updated weights for policy 0, policy_version 479282 (0.0007) [2023-12-26 18:51:13,828][105620] Updated weights for policy 1, policy_version 479607 (0.0007) [2023-12-26 18:51:13,847][105692] Updated weights for policy 0, policy_version 479292 (0.0006) [2023-12-26 18:51:13,880][105620] Updated weights for policy 1, policy_version 479617 (0.0008) [2023-12-26 18:51:14,564][105620] Updated weights for policy 1, policy_version 479627 (0.0009) [2023-12-26 18:51:14,626][105620] Updated weights for policy 1, policy_version 479637 (0.0008) [2023-12-26 18:51:14,636][105692] Updated weights for policy 0, policy_version 479302 (0.0009) [2023-12-26 18:51:14,684][105620] Updated weights for policy 1, policy_version 479647 (0.0009) [2023-12-26 18:51:14,691][105692] Updated weights for policy 0, policy_version 479312 (0.0008) [2023-12-26 18:51:14,741][105692] Updated weights for policy 0, policy_version 479322 (0.0009) [2023-12-26 18:51:15,450][105620] Updated weights for policy 1, policy_version 479657 (0.0009) [2023-12-26 18:51:15,509][105620] Updated weights for policy 1, policy_version 479667 (0.0010) [2023-12-26 18:51:15,535][105692] Updated weights for policy 0, policy_version 479332 (0.0007) [2023-12-26 18:51:15,560][105620] Updated weights for policy 1, policy_version 479677 (0.0010) [2023-12-26 18:51:15,590][105692] Updated weights for policy 0, policy_version 479342 (0.0005) [2023-12-26 18:51:15,611][105620] Updated weights for policy 1, policy_version 479687 (0.0010) [2023-12-26 18:51:15,639][105692] Updated weights for policy 0, policy_version 479352 (0.0006) [2023-12-26 18:51:16,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 245547008. Throughput: 0: 9613.4, 1: 9654.2. Samples: 245519184. Policy #0 lag: (min: 20.0, avg: 22.5, max: 48.0) [2023-12-26 18:51:16,063][104569] Avg episode reward: [(0, '8634.412'), (1, '8903.817')] [2023-12-26 18:51:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000479360_122732544.pth... [2023-12-26 18:51:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000479688_122814464.pth... [2023-12-26 18:51:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000478568_122527744.pth [2023-12-26 18:51:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000478240_122445824.pth [2023-12-26 18:51:16,276][105620] Updated weights for policy 1, policy_version 479697 (0.0010) [2023-12-26 18:51:16,331][105620] Updated weights for policy 1, policy_version 479707 (0.0010) [2023-12-26 18:51:16,394][105620] Updated weights for policy 1, policy_version 479717 (0.0010) [2023-12-26 18:51:16,458][105692] Updated weights for policy 0, policy_version 479362 (0.0009) [2023-12-26 18:51:16,515][105692] Updated weights for policy 0, policy_version 479372 (0.0009) [2023-12-26 18:51:16,569][105692] Updated weights for policy 0, policy_version 479382 (0.0010) [2023-12-26 18:51:16,630][105692] Updated weights for policy 0, policy_version 479392 (0.0009) [2023-12-26 18:51:16,964][105620] Updated weights for policy 1, policy_version 479727 (0.0006) [2023-12-26 18:51:17,020][105620] Updated weights for policy 1, policy_version 479737 (0.0005) [2023-12-26 18:51:17,068][105620] Updated weights for policy 1, policy_version 479747 (0.0008) [2023-12-26 18:51:17,463][105692] Updated weights for policy 0, policy_version 479402 (0.0009) [2023-12-26 18:51:17,518][105692] Updated weights for policy 0, policy_version 479412 (0.0009) [2023-12-26 18:51:17,580][105692] Updated weights for policy 0, policy_version 479422 (0.0009) [2023-12-26 18:51:17,783][105620] Updated weights for policy 1, policy_version 479757 (0.0009) [2023-12-26 18:51:17,830][105620] Updated weights for policy 1, policy_version 479767 (0.0009) [2023-12-26 18:51:17,878][105620] Updated weights for policy 1, policy_version 479777 (0.0008) [2023-12-26 18:51:18,426][105692] Updated weights for policy 0, policy_version 479432 (0.0009) [2023-12-26 18:51:18,484][105692] Updated weights for policy 0, policy_version 479442 (0.0009) [2023-12-26 18:51:18,491][105620] Updated weights for policy 1, policy_version 479787 (0.0006) [2023-12-26 18:51:18,534][105692] Updated weights for policy 0, policy_version 479452 (0.0006) [2023-12-26 18:51:18,545][105620] Updated weights for policy 1, policy_version 479797 (0.0006) [2023-12-26 18:51:18,599][105620] Updated weights for policy 1, policy_version 479807 (0.0009) [2023-12-26 18:51:19,288][105692] Updated weights for policy 0, policy_version 479462 (0.0007) [2023-12-26 18:51:19,346][105692] Updated weights for policy 0, policy_version 479472 (0.0009) [2023-12-26 18:51:19,381][105620] Updated weights for policy 1, policy_version 479817 (0.0009) [2023-12-26 18:51:19,409][105692] Updated weights for policy 0, policy_version 479482 (0.0007) [2023-12-26 18:51:19,420][105585] KL-divergence is very high: 186.4510 [2023-12-26 18:51:19,432][105620] Updated weights for policy 1, policy_version 479827 (0.0008) [2023-12-26 18:51:19,485][105620] Updated weights for policy 1, policy_version 479837 (0.0008) [2023-12-26 18:51:19,538][105620] Updated weights for policy 1, policy_version 479847 (0.0009) [2023-12-26 18:51:20,189][105692] Updated weights for policy 0, policy_version 479492 (0.0010) [2023-12-26 18:51:20,221][105585] KL-divergence is very high: 462.4714 [2023-12-26 18:51:20,245][105692] Updated weights for policy 0, policy_version 479502 (0.0008) [2023-12-26 18:51:20,268][105585] KL-divergence is very high: 832.3466 [2023-12-26 18:51:20,304][105692] Updated weights for policy 0, policy_version 479512 (0.0008) [2023-12-26 18:51:20,317][105585] KL-divergence is very high: 925.1829 [2023-12-26 18:51:20,327][105620] Updated weights for policy 1, policy_version 479857 (0.0008) [2023-12-26 18:51:20,384][105620] Updated weights for policy 1, policy_version 479867 (0.0009) [2023-12-26 18:51:20,435][105620] Updated weights for policy 1, policy_version 479877 (0.0008) [2023-12-26 18:51:20,996][105692] Updated weights for policy 0, policy_version 479522 (0.0008) [2023-12-26 18:51:21,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 245637120. Throughput: 0: 9489.6, 1: 9752.5. Samples: 245632596. Policy #0 lag: (min: 20.0, avg: 22.5, max: 48.0) [2023-12-26 18:51:21,062][104569] Avg episode reward: [(0, '8728.347'), (1, '9082.999')] [2023-12-26 18:51:21,067][105692] Updated weights for policy 0, policy_version 479532 (0.0007) [2023-12-26 18:51:21,134][105692] Updated weights for policy 0, policy_version 479542 (0.0008) [2023-12-26 18:51:21,191][105692] Updated weights for policy 0, policy_version 479552 (0.0008) [2023-12-26 18:51:21,249][105620] Updated weights for policy 1, policy_version 479887 (0.0009) [2023-12-26 18:51:21,317][105620] Updated weights for policy 1, policy_version 479897 (0.0008) [2023-12-26 18:51:21,386][105620] Updated weights for policy 1, policy_version 479907 (0.0008) [2023-12-26 18:51:21,949][105692] Updated weights for policy 0, policy_version 479562 (0.0009) [2023-12-26 18:51:22,018][105692] Updated weights for policy 0, policy_version 479572 (0.0006) [2023-12-26 18:51:22,085][105692] Updated weights for policy 0, policy_version 479582 (0.0006) [2023-12-26 18:51:22,122][105620] Updated weights for policy 1, policy_version 479917 (0.0007) [2023-12-26 18:51:22,188][105620] Updated weights for policy 1, policy_version 479927 (0.0007) [2023-12-26 18:51:22,251][105620] Updated weights for policy 1, policy_version 479937 (0.0007) [2023-12-26 18:51:22,775][105692] Updated weights for policy 0, policy_version 479592 (0.0006) [2023-12-26 18:51:22,831][105692] Updated weights for policy 0, policy_version 479602 (0.0005) [2023-12-26 18:51:22,884][105692] Updated weights for policy 0, policy_version 479612 (0.0006) [2023-12-26 18:51:23,005][105620] Updated weights for policy 1, policy_version 479947 (0.0009) [2023-12-26 18:51:23,057][105620] Updated weights for policy 1, policy_version 479957 (0.0007) [2023-12-26 18:51:23,114][105620] Updated weights for policy 1, policy_version 479967 (0.0008) [2023-12-26 18:51:23,459][105692] Updated weights for policy 0, policy_version 479622 (0.0007) [2023-12-26 18:51:23,532][105692] Updated weights for policy 0, policy_version 479632 (0.0008) [2023-12-26 18:51:23,581][105692] Updated weights for policy 0, policy_version 479642 (0.0008) [2023-12-26 18:51:23,913][105620] Updated weights for policy 1, policy_version 479977 (0.0008) [2023-12-26 18:51:23,973][105620] Updated weights for policy 1, policy_version 479987 (0.0008) [2023-12-26 18:51:24,021][105620] Updated weights for policy 1, policy_version 479997 (0.0010) [2023-12-26 18:51:24,066][105620] Updated weights for policy 1, policy_version 480007 (0.0010) [2023-12-26 18:51:24,313][105692] Updated weights for policy 0, policy_version 479652 (0.0009) [2023-12-26 18:51:24,368][105692] Updated weights for policy 0, policy_version 479662 (0.0009) [2023-12-26 18:51:24,421][105692] Updated weights for policy 0, policy_version 479672 (0.0008) [2023-12-26 18:51:24,796][105620] Updated weights for policy 1, policy_version 480017 (0.0008) [2023-12-26 18:51:24,859][105620] Updated weights for policy 1, policy_version 480027 (0.0005) [2023-12-26 18:51:24,915][105620] Updated weights for policy 1, policy_version 480037 (0.0005) [2023-12-26 18:51:25,127][105692] Updated weights for policy 0, policy_version 479682 (0.0008) [2023-12-26 18:51:25,181][105692] Updated weights for policy 0, policy_version 479692 (0.0010) [2023-12-26 18:51:25,227][105692] Updated weights for policy 0, policy_version 479702 (0.0010) [2023-12-26 18:51:25,275][105692] Updated weights for policy 0, policy_version 479712 (0.0009) [2023-12-26 18:51:25,504][105620] Updated weights for policy 1, policy_version 480047 (0.0008) [2023-12-26 18:51:25,562][105620] Updated weights for policy 1, policy_version 480057 (0.0005) [2023-12-26 18:51:25,627][105620] Updated weights for policy 1, policy_version 480067 (0.0005) [2023-12-26 18:51:25,961][105692] Updated weights for policy 0, policy_version 479722 (0.0006) [2023-12-26 18:51:26,014][105692] Updated weights for policy 0, policy_version 479732 (0.0005) [2023-12-26 18:51:26,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 245735424. Throughput: 0: 9518.3, 1: 9755.3. Samples: 245749080. Policy #0 lag: (min: 20.0, avg: 22.5, max: 48.0) [2023-12-26 18:51:26,063][104569] Avg episode reward: [(0, '8908.757'), (1, '9264.379')] [2023-12-26 18:51:26,076][105692] Updated weights for policy 0, policy_version 479742 (0.0005) [2023-12-26 18:51:26,198][105620] Updated weights for policy 1, policy_version 480077 (0.0006) [2023-12-26 18:51:26,244][105620] Updated weights for policy 1, policy_version 480087 (0.0005) [2023-12-26 18:51:26,298][105620] Updated weights for policy 1, policy_version 480097 (0.0005) [2023-12-26 18:51:26,668][105692] Updated weights for policy 0, policy_version 479752 (0.0006) [2023-12-26 18:51:26,738][105692] Updated weights for policy 0, policy_version 479762 (0.0008) [2023-12-26 18:51:26,820][105692] Updated weights for policy 0, policy_version 479772 (0.0011) [2023-12-26 18:51:26,829][105620] Updated weights for policy 1, policy_version 480107 (0.0006) [2023-12-26 18:51:26,877][105620] Updated weights for policy 1, policy_version 480117 (0.0005) [2023-12-26 18:51:26,926][105620] Updated weights for policy 1, policy_version 480127 (0.0006) [2023-12-26 18:51:27,376][105692] Updated weights for policy 0, policy_version 479782 (0.0010) [2023-12-26 18:51:27,426][105692] Updated weights for policy 0, policy_version 479792 (0.0010) [2023-12-26 18:51:27,480][105692] Updated weights for policy 0, policy_version 479802 (0.0008) [2023-12-26 18:51:27,613][105620] Updated weights for policy 1, policy_version 480137 (0.0008) [2023-12-26 18:51:27,678][105620] Updated weights for policy 1, policy_version 480147 (0.0007) [2023-12-26 18:51:27,738][105620] Updated weights for policy 1, policy_version 480157 (0.0009) [2023-12-26 18:51:27,799][105620] Updated weights for policy 1, policy_version 480167 (0.0010) [2023-12-26 18:51:28,282][105692] Updated weights for policy 0, policy_version 479812 (0.0009) [2023-12-26 18:51:28,349][105692] Updated weights for policy 0, policy_version 479822 (0.0008) [2023-12-26 18:51:28,406][105692] Updated weights for policy 0, policy_version 479832 (0.0009) [2023-12-26 18:51:28,413][105620] Updated weights for policy 1, policy_version 480177 (0.0006) [2023-12-26 18:51:28,469][105620] Updated weights for policy 1, policy_version 480187 (0.0005) [2023-12-26 18:51:28,519][105620] Updated weights for policy 1, policy_version 480197 (0.0006) [2023-12-26 18:51:29,057][105620] Updated weights for policy 1, policy_version 480207 (0.0005) [2023-12-26 18:51:29,107][105620] Updated weights for policy 1, policy_version 480217 (0.0005) [2023-12-26 18:51:29,165][105620] Updated weights for policy 1, policy_version 480227 (0.0005) [2023-12-26 18:51:29,279][105692] Updated weights for policy 0, policy_version 479842 (0.0009) [2023-12-26 18:51:29,337][105692] Updated weights for policy 0, policy_version 479852 (0.0008) [2023-12-26 18:51:29,396][105692] Updated weights for policy 0, policy_version 479862 (0.0010) [2023-12-26 18:51:29,455][105692] Updated weights for policy 0, policy_version 479872 (0.0009) [2023-12-26 18:51:29,775][105620] Updated weights for policy 1, policy_version 480237 (0.0008) [2023-12-26 18:51:29,838][105620] Updated weights for policy 1, policy_version 480247 (0.0007) [2023-12-26 18:51:29,909][105620] Updated weights for policy 1, policy_version 480257 (0.0006) [2023-12-26 18:51:30,233][105692] Updated weights for policy 0, policy_version 479882 (0.0007) [2023-12-26 18:51:30,286][105692] Updated weights for policy 0, policy_version 479892 (0.0009) [2023-12-26 18:51:30,336][105692] Updated weights for policy 0, policy_version 479902 (0.0008) [2023-12-26 18:51:30,584][105620] Updated weights for policy 1, policy_version 480267 (0.0007) [2023-12-26 18:51:30,638][105620] Updated weights for policy 1, policy_version 480277 (0.0009) [2023-12-26 18:51:30,691][105620] Updated weights for policy 1, policy_version 480287 (0.0009) [2023-12-26 18:51:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 245841920. Throughput: 0: 9575.0, 1: 9853.7. Samples: 245814472. Policy #0 lag: (min: 20.0, avg: 22.5, max: 48.0) [2023-12-26 18:51:31,062][104569] Avg episode reward: [(0, '9180.785'), (1, '9355.832')] [2023-12-26 18:51:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000479904_122871808.pth... [2023-12-26 18:51:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000480296_122970112.pth... [2023-12-26 18:51:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000478816_122593280.pth [2023-12-26 18:51:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000479112_122667008.pth [2023-12-26 18:51:31,148][105692] Updated weights for policy 0, policy_version 479912 (0.0008) [2023-12-26 18:51:31,195][105692] Updated weights for policy 0, policy_version 479922 (0.0008) [2023-12-26 18:51:31,244][105692] Updated weights for policy 0, policy_version 479932 (0.0009) [2023-12-26 18:51:31,337][105620] Updated weights for policy 1, policy_version 480297 (0.0008) [2023-12-26 18:51:31,398][105620] Updated weights for policy 1, policy_version 480307 (0.0008) [2023-12-26 18:51:31,458][105620] Updated weights for policy 1, policy_version 480317 (0.0009) [2023-12-26 18:51:31,510][105620] Updated weights for policy 1, policy_version 480327 (0.0010) [2023-12-26 18:51:31,957][105692] Updated weights for policy 0, policy_version 479942 (0.0009) [2023-12-26 18:51:32,009][105692] Updated weights for policy 0, policy_version 479952 (0.0008) [2023-12-26 18:51:32,062][105692] Updated weights for policy 0, policy_version 479962 (0.0008) [2023-12-26 18:51:32,246][105620] Updated weights for policy 1, policy_version 480337 (0.0011) [2023-12-26 18:51:32,308][105620] Updated weights for policy 1, policy_version 480347 (0.0010) [2023-12-26 18:51:32,376][105620] Updated weights for policy 1, policy_version 480357 (0.0009) [2023-12-26 18:51:32,847][105692] Updated weights for policy 0, policy_version 479972 (0.0007) [2023-12-26 18:51:32,899][105692] Updated weights for policy 0, policy_version 479982 (0.0007) [2023-12-26 18:51:32,958][105692] Updated weights for policy 0, policy_version 479992 (0.0008) [2023-12-26 18:51:33,110][105620] Updated weights for policy 1, policy_version 480367 (0.0011) [2023-12-26 18:51:33,164][105620] Updated weights for policy 1, policy_version 480377 (0.0010) [2023-12-26 18:51:33,211][105620] Updated weights for policy 1, policy_version 480387 (0.0006) [2023-12-26 18:51:33,634][105692] Updated weights for policy 0, policy_version 480002 (0.0008) [2023-12-26 18:51:33,690][105692] Updated weights for policy 0, policy_version 480012 (0.0008) [2023-12-26 18:51:33,748][105692] Updated weights for policy 0, policy_version 480022 (0.0010) [2023-12-26 18:51:33,808][105692] Updated weights for policy 0, policy_version 480032 (0.0009) [2023-12-26 18:51:33,820][105620] Updated weights for policy 1, policy_version 480397 (0.0008) [2023-12-26 18:51:33,874][105620] Updated weights for policy 1, policy_version 480407 (0.0010) [2023-12-26 18:51:33,932][105620] Updated weights for policy 1, policy_version 480417 (0.0010) [2023-12-26 18:51:34,521][105620] Updated weights for policy 1, policy_version 480427 (0.0007) [2023-12-26 18:51:34,576][105620] Updated weights for policy 1, policy_version 480437 (0.0009) [2023-12-26 18:51:34,638][105620] Updated weights for policy 1, policy_version 480447 (0.0009) [2023-12-26 18:51:34,665][105692] Updated weights for policy 0, policy_version 480042 (0.0008) [2023-12-26 18:51:34,728][105692] Updated weights for policy 0, policy_version 480052 (0.0007) [2023-12-26 18:51:34,795][105692] Updated weights for policy 0, policy_version 480062 (0.0009) [2023-12-26 18:51:35,312][105620] Updated weights for policy 1, policy_version 480457 (0.0008) [2023-12-26 18:51:35,371][105620] Updated weights for policy 1, policy_version 480467 (0.0005) [2023-12-26 18:51:35,424][105620] Updated weights for policy 1, policy_version 480477 (0.0005) [2023-12-26 18:51:35,486][105620] Updated weights for policy 1, policy_version 480487 (0.0006) [2023-12-26 18:51:35,586][105692] Updated weights for policy 0, policy_version 480072 (0.0008) [2023-12-26 18:51:35,641][105692] Updated weights for policy 0, policy_version 480082 (0.0007) [2023-12-26 18:51:35,696][105692] Updated weights for policy 0, policy_version 480092 (0.0008) [2023-12-26 18:51:36,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 245940224. Throughput: 0: 9527.6, 1: 9973.2. Samples: 245931600. Policy #0 lag: (min: 20.0, avg: 22.5, max: 48.0) [2023-12-26 18:51:36,062][104569] Avg episode reward: [(0, '9090.854'), (1, '9356.129')] [2023-12-26 18:51:36,126][105620] Updated weights for policy 1, policy_version 480497 (0.0008) [2023-12-26 18:51:36,193][105620] Updated weights for policy 1, policy_version 480507 (0.0010) [2023-12-26 18:51:36,256][105620] Updated weights for policy 1, policy_version 480517 (0.0010) [2023-12-26 18:51:36,511][105692] Updated weights for policy 0, policy_version 480102 (0.0008) [2023-12-26 18:51:36,563][105692] Updated weights for policy 0, policy_version 480112 (0.0008) [2023-12-26 18:51:36,616][105692] Updated weights for policy 0, policy_version 480122 (0.0008) [2023-12-26 18:51:36,959][105620] Updated weights for policy 1, policy_version 480527 (0.0010) [2023-12-26 18:51:37,007][105620] Updated weights for policy 1, policy_version 480537 (0.0010) [2023-12-26 18:51:37,055][105620] Updated weights for policy 1, policy_version 480547 (0.0010) [2023-12-26 18:51:37,441][105692] Updated weights for policy 0, policy_version 480132 (0.0008) [2023-12-26 18:51:37,505][105692] Updated weights for policy 0, policy_version 480142 (0.0009) [2023-12-26 18:51:37,571][105692] Updated weights for policy 0, policy_version 480152 (0.0010) [2023-12-26 18:51:37,687][105620] Updated weights for policy 1, policy_version 480557 (0.0008) [2023-12-26 18:51:37,738][105620] Updated weights for policy 1, policy_version 480567 (0.0007) [2023-12-26 18:51:37,792][105620] Updated weights for policy 1, policy_version 480577 (0.0005) [2023-12-26 18:51:38,347][105620] Updated weights for policy 1, policy_version 480587 (0.0010) [2023-12-26 18:51:38,362][105692] Updated weights for policy 0, policy_version 480162 (0.0009) [2023-12-26 18:51:38,409][105620] Updated weights for policy 1, policy_version 480597 (0.0007) [2023-12-26 18:51:38,416][105692] Updated weights for policy 0, policy_version 480172 (0.0007) [2023-12-26 18:51:38,461][105620] Updated weights for policy 1, policy_version 480607 (0.0007) [2023-12-26 18:51:38,464][105692] Updated weights for policy 0, policy_version 480182 (0.0007) [2023-12-26 18:51:38,514][105692] Updated weights for policy 0, policy_version 480192 (0.0008) [2023-12-26 18:51:39,231][105620] Updated weights for policy 1, policy_version 480617 (0.0007) [2023-12-26 18:51:39,254][105692] Updated weights for policy 0, policy_version 480202 (0.0008) [2023-12-26 18:51:39,293][105620] Updated weights for policy 1, policy_version 480627 (0.0010) [2023-12-26 18:51:39,312][105692] Updated weights for policy 0, policy_version 480212 (0.0007) [2023-12-26 18:51:39,312][105585] KL-divergence is very high: 1191.9301 [2023-12-26 18:51:39,318][105585] KL-divergence is very high: 1074.1759 [2023-12-26 18:51:39,357][105620] Updated weights for policy 1, policy_version 480637 (0.0013) [2023-12-26 18:51:39,369][105585] KL-divergence is very high: 932.9320 [2023-12-26 18:51:39,376][105585] KL-divergence is very high: 718.6840 [2023-12-26 18:51:39,380][105692] Updated weights for policy 0, policy_version 480222 (0.0008) [2023-12-26 18:51:39,429][105620] Updated weights for policy 1, policy_version 480647 (0.0010) [2023-12-26 18:51:40,112][105692] Updated weights for policy 0, policy_version 480232 (0.0010) [2023-12-26 18:51:40,138][105620] Updated weights for policy 1, policy_version 480657 (0.0011) [2023-12-26 18:51:40,173][105692] Updated weights for policy 0, policy_version 480242 (0.0011) [2023-12-26 18:51:40,203][105620] Updated weights for policy 1, policy_version 480667 (0.0010) [2023-12-26 18:51:40,242][105692] Updated weights for policy 0, policy_version 480252 (0.0011) [2023-12-26 18:51:40,264][105620] Updated weights for policy 1, policy_version 480677 (0.0006) [2023-12-26 18:51:40,918][105620] Updated weights for policy 1, policy_version 480687 (0.0005) [2023-12-26 18:51:40,969][105620] Updated weights for policy 1, policy_version 480697 (0.0006) [2023-12-26 18:51:40,997][105692] Updated weights for policy 0, policy_version 480262 (0.0009) [2023-12-26 18:51:41,037][105620] Updated weights for policy 1, policy_version 480707 (0.0006) [2023-12-26 18:51:41,054][105692] Updated weights for policy 0, policy_version 480272 (0.0007) [2023-12-26 18:51:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 246030336. Throughput: 0: 9343.6, 1: 10138.1. Samples: 246047832. Policy #0 lag: (min: 20.0, avg: 22.5, max: 48.0) [2023-12-26 18:51:41,063][104569] Avg episode reward: [(0, '8635.283'), (1, '9173.938')] [2023-12-26 18:51:41,128][105692] Updated weights for policy 0, policy_version 480282 (0.0009) [2023-12-26 18:51:41,735][105620] Updated weights for policy 1, policy_version 480717 (0.0008) [2023-12-26 18:51:41,798][105620] Updated weights for policy 1, policy_version 480727 (0.0007) [2023-12-26 18:51:41,863][105620] Updated weights for policy 1, policy_version 480737 (0.0006) [2023-12-26 18:51:42,000][105692] Updated weights for policy 0, policy_version 480292 (0.0009) [2023-12-26 18:51:42,063][105692] Updated weights for policy 0, policy_version 480302 (0.0008) [2023-12-26 18:51:42,123][105692] Updated weights for policy 0, policy_version 480312 (0.0008) [2023-12-26 18:51:42,566][105620] Updated weights for policy 1, policy_version 480747 (0.0007) [2023-12-26 18:51:42,627][105620] Updated weights for policy 1, policy_version 480757 (0.0008) [2023-12-26 18:51:42,688][105620] Updated weights for policy 1, policy_version 480767 (0.0009) [2023-12-26 18:51:42,805][105692] Updated weights for policy 0, policy_version 480322 (0.0009) [2023-12-26 18:51:42,867][105692] Updated weights for policy 0, policy_version 480332 (0.0008) [2023-12-26 18:51:42,928][105692] Updated weights for policy 0, policy_version 480342 (0.0009) [2023-12-26 18:51:42,984][105692] Updated weights for policy 0, policy_version 480352 (0.0009) [2023-12-26 18:51:43,377][105620] Updated weights for policy 1, policy_version 480777 (0.0008) [2023-12-26 18:51:43,425][105620] Updated weights for policy 1, policy_version 480787 (0.0005) [2023-12-26 18:51:43,477][105620] Updated weights for policy 1, policy_version 480797 (0.0005) [2023-12-26 18:51:43,535][105620] Updated weights for policy 1, policy_version 480807 (0.0005) [2023-12-26 18:51:43,741][105692] Updated weights for policy 0, policy_version 480362 (0.0010) [2023-12-26 18:51:43,788][105692] Updated weights for policy 0, policy_version 480372 (0.0010) [2023-12-26 18:51:43,843][105692] Updated weights for policy 0, policy_version 480382 (0.0010) [2023-12-26 18:51:44,175][105620] Updated weights for policy 1, policy_version 480817 (0.0006) [2023-12-26 18:51:44,240][105620] Updated weights for policy 1, policy_version 480827 (0.0005) [2023-12-26 18:51:44,291][105620] Updated weights for policy 1, policy_version 480837 (0.0005) [2023-12-26 18:51:44,605][105692] Updated weights for policy 0, policy_version 480392 (0.0010) [2023-12-26 18:51:44,654][105692] Updated weights for policy 0, policy_version 480402 (0.0009) [2023-12-26 18:51:44,709][105692] Updated weights for policy 0, policy_version 480412 (0.0007) [2023-12-26 18:51:44,896][105620] Updated weights for policy 1, policy_version 480847 (0.0008) [2023-12-26 18:51:44,952][105620] Updated weights for policy 1, policy_version 480857 (0.0009) [2023-12-26 18:51:45,001][105620] Updated weights for policy 1, policy_version 480867 (0.0008) [2023-12-26 18:51:45,349][105692] Updated weights for policy 0, policy_version 480422 (0.0007) [2023-12-26 18:51:45,414][105692] Updated weights for policy 0, policy_version 480432 (0.0005) [2023-12-26 18:51:45,477][105692] Updated weights for policy 0, policy_version 480442 (0.0009) [2023-12-26 18:51:45,746][105620] Updated weights for policy 1, policy_version 480877 (0.0008) [2023-12-26 18:51:45,798][105620] Updated weights for policy 1, policy_version 480887 (0.0008) [2023-12-26 18:51:45,850][105620] Updated weights for policy 1, policy_version 480897 (0.0008) [2023-12-26 18:51:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 246136832. Throughput: 0: 9263.6, 1: 10179.5. Samples: 246105140. Policy #0 lag: (min: 20.0, avg: 22.5, max: 48.0) [2023-12-26 18:51:46,062][105692] Updated weights for policy 0, policy_version 480452 (0.0008) [2023-12-26 18:51:46,063][104569] Avg episode reward: [(0, '8453.544'), (1, '9173.847')] [2023-12-26 18:51:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000480904_123125760.pth... [2023-12-26 18:51:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000479688_122814464.pth [2023-12-26 18:51:46,111][105692] Updated weights for policy 0, policy_version 480462 (0.0006) [2023-12-26 18:51:46,166][105692] Updated weights for policy 0, policy_version 480472 (0.0006) [2023-12-26 18:51:46,200][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000480480_123019264.pth... [2023-12-26 18:51:46,203][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000479360_122732544.pth [2023-12-26 18:51:46,724][105620] Updated weights for policy 1, policy_version 480907 (0.0008) [2023-12-26 18:51:46,774][105692] Updated weights for policy 0, policy_version 480482 (0.0005) [2023-12-26 18:51:46,788][105620] Updated weights for policy 1, policy_version 480917 (0.0009) [2023-12-26 18:51:46,838][105692] Updated weights for policy 0, policy_version 480492 (0.0008) [2023-12-26 18:51:46,847][105620] Updated weights for policy 1, policy_version 480927 (0.0006) [2023-12-26 18:51:46,897][105692] Updated weights for policy 0, policy_version 480502 (0.0007) [2023-12-26 18:51:46,958][105692] Updated weights for policy 0, policy_version 480512 (0.0009) [2023-12-26 18:51:47,420][105620] Updated weights for policy 1, policy_version 480937 (0.0007) [2023-12-26 18:51:47,467][105620] Updated weights for policy 1, policy_version 480947 (0.0009) [2023-12-26 18:51:47,518][105620] Updated weights for policy 1, policy_version 480957 (0.0009) [2023-12-26 18:51:47,568][105620] Updated weights for policy 1, policy_version 480967 (0.0009) [2023-12-26 18:51:47,770][105692] Updated weights for policy 0, policy_version 480522 (0.0009) [2023-12-26 18:51:47,830][105692] Updated weights for policy 0, policy_version 480532 (0.0009) [2023-12-26 18:51:47,887][105692] Updated weights for policy 0, policy_version 480542 (0.0009) [2023-12-26 18:51:48,243][105620] Updated weights for policy 1, policy_version 480977 (0.0007) [2023-12-26 18:51:48,307][105620] Updated weights for policy 1, policy_version 480987 (0.0008) [2023-12-26 18:51:48,372][105620] Updated weights for policy 1, policy_version 480997 (0.0009) [2023-12-26 18:51:48,671][105692] Updated weights for policy 0, policy_version 480552 (0.0010) [2023-12-26 18:51:48,729][105692] Updated weights for policy 0, policy_version 480562 (0.0009) [2023-12-26 18:51:48,787][105692] Updated weights for policy 0, policy_version 480572 (0.0009) [2023-12-26 18:51:49,107][105620] Updated weights for policy 1, policy_version 481007 (0.0009) [2023-12-26 18:51:49,169][105620] Updated weights for policy 1, policy_version 481017 (0.0009) [2023-12-26 18:51:49,234][105620] Updated weights for policy 1, policy_version 481027 (0.0008) [2023-12-26 18:51:49,576][105692] Updated weights for policy 0, policy_version 480582 (0.0009) [2023-12-26 18:51:49,637][105692] Updated weights for policy 0, policy_version 480592 (0.0009) [2023-12-26 18:51:49,706][105692] Updated weights for policy 0, policy_version 480602 (0.0010) [2023-12-26 18:51:49,923][105620] Updated weights for policy 1, policy_version 481037 (0.0008) [2023-12-26 18:51:49,989][105620] Updated weights for policy 1, policy_version 481047 (0.0006) [2023-12-26 18:51:50,047][105620] Updated weights for policy 1, policy_version 481057 (0.0005) [2023-12-26 18:51:50,541][105692] Updated weights for policy 0, policy_version 480612 (0.0010) [2023-12-26 18:51:50,602][105692] Updated weights for policy 0, policy_version 480622 (0.0009) [2023-12-26 18:51:50,653][105620] Updated weights for policy 1, policy_version 481067 (0.0005) [2023-12-26 18:51:50,657][105692] Updated weights for policy 0, policy_version 480632 (0.0009) [2023-12-26 18:51:50,713][105620] Updated weights for policy 1, policy_version 481077 (0.0009) [2023-12-26 18:51:50,784][105620] Updated weights for policy 1, policy_version 481087 (0.0009) [2023-12-26 18:51:51,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 246235136. Throughput: 0: 9273.2, 1: 10252.1. Samples: 246223036. Policy #0 lag: (min: 20.0, avg: 22.5, max: 48.0) [2023-12-26 18:51:51,062][104569] Avg episode reward: [(0, '8453.947'), (1, '9174.459')] [2023-12-26 18:51:51,425][105692] Updated weights for policy 0, policy_version 480642 (0.0008) [2023-12-26 18:51:51,478][105692] Updated weights for policy 0, policy_version 480652 (0.0008) [2023-12-26 18:51:51,514][105620] Updated weights for policy 1, policy_version 481097 (0.0009) [2023-12-26 18:51:51,528][105692] Updated weights for policy 0, policy_version 480662 (0.0007) [2023-12-26 18:51:51,567][105620] Updated weights for policy 1, policy_version 481107 (0.0009) [2023-12-26 18:51:51,581][105692] Updated weights for policy 0, policy_version 480672 (0.0006) [2023-12-26 18:51:51,619][105620] Updated weights for policy 1, policy_version 481117 (0.0008) [2023-12-26 18:51:51,679][105620] Updated weights for policy 1, policy_version 481127 (0.0009) [2023-12-26 18:51:52,347][105692] Updated weights for policy 0, policy_version 480682 (0.0009) [2023-12-26 18:51:52,410][105692] Updated weights for policy 0, policy_version 480692 (0.0009) [2023-12-26 18:51:52,458][105692] Updated weights for policy 0, policy_version 480702 (0.0006) [2023-12-26 18:51:52,470][105620] Updated weights for policy 1, policy_version 481137 (0.0007) [2023-12-26 18:51:52,526][105620] Updated weights for policy 1, policy_version 481147 (0.0009) [2023-12-26 18:51:52,590][105620] Updated weights for policy 1, policy_version 481157 (0.0009) [2023-12-26 18:51:53,164][105692] Updated weights for policy 0, policy_version 480712 (0.0005) [2023-12-26 18:51:53,215][105692] Updated weights for policy 0, policy_version 480722 (0.0007) [2023-12-26 18:51:53,269][105692] Updated weights for policy 0, policy_version 480732 (0.0009) [2023-12-26 18:51:53,386][105620] Updated weights for policy 1, policy_version 481167 (0.0009) [2023-12-26 18:51:53,432][105620] Updated weights for policy 1, policy_version 481177 (0.0008) [2023-12-26 18:51:53,494][105620] Updated weights for policy 1, policy_version 481187 (0.0009) [2023-12-26 18:51:53,943][105692] Updated weights for policy 0, policy_version 480742 (0.0008) [2023-12-26 18:51:53,994][105692] Updated weights for policy 0, policy_version 480752 (0.0009) [2023-12-26 18:51:54,043][105692] Updated weights for policy 0, policy_version 480762 (0.0009) [2023-12-26 18:51:54,258][105620] Updated weights for policy 1, policy_version 481197 (0.0009) [2023-12-26 18:51:54,306][105620] Updated weights for policy 1, policy_version 481207 (0.0009) [2023-12-26 18:51:54,353][105620] Updated weights for policy 1, policy_version 481217 (0.0009) [2023-12-26 18:51:54,813][105692] Updated weights for policy 0, policy_version 480772 (0.0009) [2023-12-26 18:51:54,869][105692] Updated weights for policy 0, policy_version 480782 (0.0009) [2023-12-26 18:51:54,927][105692] Updated weights for policy 0, policy_version 480792 (0.0008) [2023-12-26 18:51:55,117][105620] Updated weights for policy 1, policy_version 481227 (0.0009) [2023-12-26 18:51:55,169][105620] Updated weights for policy 1, policy_version 481237 (0.0009) [2023-12-26 18:51:55,223][105620] Updated weights for policy 1, policy_version 481249 (0.0011) [2023-12-26 18:51:55,556][105692] Updated weights for policy 0, policy_version 480802 (0.0009) [2023-12-26 18:51:55,623][105692] Updated weights for policy 0, policy_version 480812 (0.0009) [2023-12-26 18:51:55,670][105692] Updated weights for policy 0, policy_version 480822 (0.0009) [2023-12-26 18:51:55,717][105692] Updated weights for policy 0, policy_version 480832 (0.0009) [2023-12-26 18:51:56,049][105620] Updated weights for policy 1, policy_version 481259 (0.0009) [2023-12-26 18:51:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 246325248. Throughput: 0: 9319.6, 1: 10094.7. Samples: 246336316. Policy #0 lag: (min: 20.0, avg: 22.5, max: 48.0) [2023-12-26 18:51:56,063][104569] Avg episode reward: [(0, '8634.469'), (1, '9174.338')] [2023-12-26 18:51:56,109][105620] Updated weights for policy 1, policy_version 481269 (0.0009) [2023-12-26 18:51:56,170][105620] Updated weights for policy 1, policy_version 481279 (0.0008) [2023-12-26 18:51:56,470][105692] Updated weights for policy 0, policy_version 480842 (0.0009) [2023-12-26 18:51:56,517][105692] Updated weights for policy 0, policy_version 480852 (0.0009) [2023-12-26 18:51:56,565][105692] Updated weights for policy 0, policy_version 480862 (0.0009) [2023-12-26 18:51:56,926][105620] Updated weights for policy 1, policy_version 481289 (0.0009) [2023-12-26 18:51:56,994][105620] Updated weights for policy 1, policy_version 481299 (0.0009) [2023-12-26 18:51:57,040][105620] Updated weights for policy 1, policy_version 481309 (0.0008) [2023-12-26 18:51:57,099][105620] Updated weights for policy 1, policy_version 481319 (0.0009) [2023-12-26 18:51:57,287][105692] Updated weights for policy 0, policy_version 480872 (0.0009) [2023-12-26 18:51:57,335][105692] Updated weights for policy 0, policy_version 480882 (0.0009) [2023-12-26 18:51:57,393][105692] Updated weights for policy 0, policy_version 480892 (0.0010) [2023-12-26 18:51:57,825][105620] Updated weights for policy 1, policy_version 481329 (0.0009) [2023-12-26 18:51:57,891][105620] Updated weights for policy 1, policy_version 481339 (0.0009) [2023-12-26 18:51:57,946][105620] Updated weights for policy 1, policy_version 481349 (0.0008) [2023-12-26 18:51:58,197][105692] Updated weights for policy 0, policy_version 480902 (0.0008) [2023-12-26 18:51:58,221][105585] KL-divergence is very high: 139.7177 [2023-12-26 18:51:58,257][105692] Updated weights for policy 0, policy_version 480912 (0.0006) [2023-12-26 18:51:58,272][105585] KL-divergence is very high: 196.4480 [2023-12-26 18:51:58,327][105692] Updated weights for policy 0, policy_version 480922 (0.0008) [2023-12-26 18:51:58,327][105585] KL-divergence is very high: 149.3003 [2023-12-26 18:51:58,738][105620] Updated weights for policy 1, policy_version 481359 (0.0009) [2023-12-26 18:51:58,802][105620] Updated weights for policy 1, policy_version 481369 (0.0009) [2023-12-26 18:51:58,871][105620] Updated weights for policy 1, policy_version 481379 (0.0010) [2023-12-26 18:51:59,051][105692] Updated weights for policy 0, policy_version 480932 (0.0010) [2023-12-26 18:51:59,106][105692] Updated weights for policy 0, policy_version 480942 (0.0010) [2023-12-26 18:51:59,151][105692] Updated weights for policy 0, policy_version 480952 (0.0010) [2023-12-26 18:51:59,617][105620] Updated weights for policy 1, policy_version 481389 (0.0008) [2023-12-26 18:51:59,671][105620] Updated weights for policy 1, policy_version 481399 (0.0005) [2023-12-26 18:51:59,729][105620] Updated weights for policy 1, policy_version 481409 (0.0006) [2023-12-26 18:52:00,002][105692] Updated weights for policy 0, policy_version 480962 (0.0010) [2023-12-26 18:52:00,049][105692] Updated weights for policy 0, policy_version 480972 (0.0008) [2023-12-26 18:52:00,097][105692] Updated weights for policy 0, policy_version 480982 (0.0009) [2023-12-26 18:52:00,131][105585] KL-divergence is very high: 177.4794 [2023-12-26 18:52:00,153][105692] Updated weights for policy 0, policy_version 480992 (0.0009) [2023-12-26 18:52:00,334][105620] Updated weights for policy 1, policy_version 481419 (0.0010) [2023-12-26 18:52:00,399][105620] Updated weights for policy 1, policy_version 481429 (0.0007) [2023-12-26 18:52:00,461][105620] Updated weights for policy 1, policy_version 481439 (0.0006) [2023-12-26 18:52:00,836][105692] Updated weights for policy 0, policy_version 481002 (0.0005) [2023-12-26 18:52:00,887][105692] Updated weights for policy 0, policy_version 481013 (0.0008) [2023-12-26 18:52:00,944][105692] Updated weights for policy 0, policy_version 481024 (0.0009) [2023-12-26 18:52:01,058][105620] Updated weights for policy 1, policy_version 481449 (0.0009) [2023-12-26 18:52:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 246423552. Throughput: 0: 9368.9, 1: 10014.3. Samples: 246391428. Policy #0 lag: (min: 20.0, avg: 22.5, max: 48.0) [2023-12-26 18:52:01,062][104569] Avg episode reward: [(0, '8459.854'), (1, '9355.647')] [2023-12-26 18:52:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000481024_123158528.pth... [2023-12-26 18:52:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000479904_122871808.pth [2023-12-26 18:52:01,110][105620] Updated weights for policy 1, policy_version 481459 (0.0009) [2023-12-26 18:52:01,173][105620] Updated weights for policy 1, policy_version 481469 (0.0009) [2023-12-26 18:52:01,231][105620] Updated weights for policy 1, policy_version 481479 (0.0008) [2023-12-26 18:52:01,236][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000481480_123273216.pth... [2023-12-26 18:52:01,239][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000480296_122970112.pth [2023-12-26 18:52:01,645][105692] Updated weights for policy 0, policy_version 481034 (0.0009) [2023-12-26 18:52:01,692][105692] Updated weights for policy 0, policy_version 481044 (0.0009) [2023-12-26 18:52:01,744][105692] Updated weights for policy 0, policy_version 481054 (0.0009) [2023-12-26 18:52:02,065][105620] Updated weights for policy 1, policy_version 481489 (0.0010) [2023-12-26 18:52:02,130][105620] Updated weights for policy 1, policy_version 481499 (0.0010) [2023-12-26 18:52:02,193][105620] Updated weights for policy 1, policy_version 481509 (0.0008) [2023-12-26 18:52:02,440][105692] Updated weights for policy 0, policy_version 481064 (0.0009) [2023-12-26 18:52:02,488][105692] Updated weights for policy 0, policy_version 481074 (0.0007) [2023-12-26 18:52:02,547][105692] Updated weights for policy 0, policy_version 481084 (0.0009) [2023-12-26 18:52:02,981][105620] Updated weights for policy 1, policy_version 481519 (0.0010) [2023-12-26 18:52:03,038][105620] Updated weights for policy 1, policy_version 481529 (0.0009) [2023-12-26 18:52:03,090][105620] Updated weights for policy 1, policy_version 481539 (0.0009) [2023-12-26 18:52:03,180][105692] Updated weights for policy 0, policy_version 481094 (0.0009) [2023-12-26 18:52:03,233][105692] Updated weights for policy 0, policy_version 481104 (0.0009) [2023-12-26 18:52:03,284][105692] Updated weights for policy 0, policy_version 481114 (0.0009) [2023-12-26 18:52:03,862][105620] Updated weights for policy 1, policy_version 481549 (0.0009) [2023-12-26 18:52:03,921][105620] Updated weights for policy 1, policy_version 481559 (0.0008) [2023-12-26 18:52:03,982][105620] Updated weights for policy 1, policy_version 481569 (0.0008) [2023-12-26 18:52:04,018][105692] Updated weights for policy 0, policy_version 481124 (0.0010) [2023-12-26 18:52:04,077][105692] Updated weights for policy 0, policy_version 481134 (0.0011) [2023-12-26 18:52:04,141][105692] Updated weights for policy 0, policy_version 481144 (0.0011) [2023-12-26 18:52:04,630][105620] Updated weights for policy 1, policy_version 481579 (0.0008) [2023-12-26 18:52:04,682][105620] Updated weights for policy 1, policy_version 481589 (0.0008) [2023-12-26 18:52:04,739][105620] Updated weights for policy 1, policy_version 481599 (0.0006) [2023-12-26 18:52:04,895][105692] Updated weights for policy 0, policy_version 481154 (0.0011) [2023-12-26 18:52:04,943][105692] Updated weights for policy 0, policy_version 481164 (0.0010) [2023-12-26 18:52:04,991][105692] Updated weights for policy 0, policy_version 481174 (0.0010) [2023-12-26 18:52:05,039][105692] Updated weights for policy 0, policy_version 481184 (0.0010) [2023-12-26 18:52:05,445][105620] Updated weights for policy 1, policy_version 481609 (0.0007) [2023-12-26 18:52:05,506][105620] Updated weights for policy 1, policy_version 481619 (0.0008) [2023-12-26 18:52:05,575][105620] Updated weights for policy 1, policy_version 481629 (0.0007) [2023-12-26 18:52:05,640][105620] Updated weights for policy 1, policy_version 481639 (0.0005) [2023-12-26 18:52:05,671][105692] Updated weights for policy 0, policy_version 481194 (0.0011) [2023-12-26 18:52:05,723][105692] Updated weights for policy 0, policy_version 481204 (0.0010) [2023-12-26 18:52:05,781][105692] Updated weights for policy 0, policy_version 481214 (0.0010) [2023-12-26 18:52:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 246521856. Throughput: 0: 9507.2, 1: 9972.2. Samples: 246509172. Policy #0 lag: (min: 20.0, avg: 22.5, max: 48.0) [2023-12-26 18:52:06,063][104569] Avg episode reward: [(0, '8458.572'), (1, '9174.914')] [2023-12-26 18:52:06,246][105620] Updated weights for policy 1, policy_version 481649 (0.0008) [2023-12-26 18:52:06,316][105620] Updated weights for policy 1, policy_version 481659 (0.0008) [2023-12-26 18:52:06,383][105620] Updated weights for policy 1, policy_version 481669 (0.0008) [2023-12-26 18:52:06,504][105692] Updated weights for policy 0, policy_version 481224 (0.0009) [2023-12-26 18:52:06,565][105692] Updated weights for policy 0, policy_version 481234 (0.0011) [2023-12-26 18:52:06,628][105692] Updated weights for policy 0, policy_version 481244 (0.0010) [2023-12-26 18:52:07,116][105620] Updated weights for policy 1, policy_version 481679 (0.0008) [2023-12-26 18:52:07,161][105620] Updated weights for policy 1, policy_version 481689 (0.0008) [2023-12-26 18:52:07,207][105620] Updated weights for policy 1, policy_version 481699 (0.0008) [2023-12-26 18:52:07,371][105692] Updated weights for policy 0, policy_version 481254 (0.0010) [2023-12-26 18:52:07,439][105692] Updated weights for policy 0, policy_version 481264 (0.0010) [2023-12-26 18:52:07,504][105692] Updated weights for policy 0, policy_version 481274 (0.0010) [2023-12-26 18:52:07,923][105620] Updated weights for policy 1, policy_version 481709 (0.0006) [2023-12-26 18:52:07,969][105620] Updated weights for policy 1, policy_version 481719 (0.0005) [2023-12-26 18:52:08,016][105620] Updated weights for policy 1, policy_version 481729 (0.0006) [2023-12-26 18:52:08,218][105692] Updated weights for policy 0, policy_version 481284 (0.0007) [2023-12-26 18:52:08,268][105692] Updated weights for policy 0, policy_version 481294 (0.0005) [2023-12-26 18:52:08,325][105692] Updated weights for policy 0, policy_version 481304 (0.0005) [2023-12-26 18:52:08,621][105620] Updated weights for policy 1, policy_version 481739 (0.0007) [2023-12-26 18:52:08,683][105620] Updated weights for policy 1, policy_version 481749 (0.0010) [2023-12-26 18:52:08,745][105620] Updated weights for policy 1, policy_version 481759 (0.0010) [2023-12-26 18:52:09,033][105692] Updated weights for policy 0, policy_version 481314 (0.0009) [2023-12-26 18:52:09,093][105692] Updated weights for policy 0, policy_version 481324 (0.0008) [2023-12-26 18:52:09,153][105692] Updated weights for policy 0, policy_version 481334 (0.0008) [2023-12-26 18:52:09,213][105692] Updated weights for policy 0, policy_version 481344 (0.0008) [2023-12-26 18:52:09,486][105620] Updated weights for policy 1, policy_version 481769 (0.0011) [2023-12-26 18:52:09,537][105620] Updated weights for policy 1, policy_version 481779 (0.0011) [2023-12-26 18:52:09,606][105620] Updated weights for policy 1, policy_version 481789 (0.0009) [2023-12-26 18:52:09,666][105620] Updated weights for policy 1, policy_version 481799 (0.0010) [2023-12-26 18:52:10,019][105692] Updated weights for policy 0, policy_version 481354 (0.0006) [2023-12-26 18:52:10,083][105692] Updated weights for policy 0, policy_version 481364 (0.0005) [2023-12-26 18:52:10,148][105692] Updated weights for policy 0, policy_version 481374 (0.0006) [2023-12-26 18:52:10,512][105620] Updated weights for policy 1, policy_version 481809 (0.0010) [2023-12-26 18:52:10,564][105620] Updated weights for policy 1, policy_version 481819 (0.0010) [2023-12-26 18:52:10,612][105620] Updated weights for policy 1, policy_version 481829 (0.0010) [2023-12-26 18:52:10,829][105692] Updated weights for policy 0, policy_version 481384 (0.0008) [2023-12-26 18:52:10,878][105692] Updated weights for policy 0, policy_version 481394 (0.0008) [2023-12-26 18:52:10,930][105692] Updated weights for policy 0, policy_version 481404 (0.0008) [2023-12-26 18:52:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 246620160. Throughput: 0: 9495.6, 1: 9994.8. Samples: 246626144. Policy #0 lag: (min: 20.0, avg: 22.5, max: 48.0) [2023-12-26 18:52:11,062][104569] Avg episode reward: [(0, '8639.123'), (1, '9175.190')] [2023-12-26 18:52:11,408][105620] Updated weights for policy 1, policy_version 481839 (0.0012) [2023-12-26 18:52:11,468][105620] Updated weights for policy 1, policy_version 481849 (0.0009) [2023-12-26 18:52:11,524][105620] Updated weights for policy 1, policy_version 481859 (0.0009) [2023-12-26 18:52:11,747][105692] Updated weights for policy 0, policy_version 481414 (0.0007) [2023-12-26 18:52:11,803][105692] Updated weights for policy 0, policy_version 481424 (0.0009) [2023-12-26 18:52:11,865][105692] Updated weights for policy 0, policy_version 481434 (0.0009) [2023-12-26 18:52:12,371][105620] Updated weights for policy 1, policy_version 481869 (0.0009) [2023-12-26 18:52:12,440][105620] Updated weights for policy 1, policy_version 481879 (0.0007) [2023-12-26 18:52:12,503][105620] Updated weights for policy 1, policy_version 481889 (0.0010) [2023-12-26 18:52:12,519][105692] Updated weights for policy 0, policy_version 481444 (0.0008) [2023-12-26 18:52:12,572][105692] Updated weights for policy 0, policy_version 481454 (0.0007) [2023-12-26 18:52:12,634][105692] Updated weights for policy 0, policy_version 481464 (0.0006) [2023-12-26 18:52:13,288][105692] Updated weights for policy 0, policy_version 481474 (0.0006) [2023-12-26 18:52:13,301][105620] Updated weights for policy 1, policy_version 481899 (0.0007) [2023-12-26 18:52:13,347][105692] Updated weights for policy 0, policy_version 481484 (0.0009) [2023-12-26 18:52:13,362][105620] Updated weights for policy 1, policy_version 481909 (0.0005) [2023-12-26 18:52:13,408][105692] Updated weights for policy 0, policy_version 481494 (0.0010) [2023-12-26 18:52:13,421][105620] Updated weights for policy 1, policy_version 481919 (0.0006) [2023-12-26 18:52:13,461][105692] Updated weights for policy 0, policy_version 481504 (0.0007) [2023-12-26 18:52:13,971][105620] Updated weights for policy 1, policy_version 481929 (0.0007) [2023-12-26 18:52:14,031][105620] Updated weights for policy 1, policy_version 481939 (0.0005) [2023-12-26 18:52:14,088][105620] Updated weights for policy 1, policy_version 481949 (0.0005) [2023-12-26 18:52:14,151][105620] Updated weights for policy 1, policy_version 481959 (0.0006) [2023-12-26 18:52:14,249][105692] Updated weights for policy 0, policy_version 481514 (0.0005) [2023-12-26 18:52:14,306][105692] Updated weights for policy 0, policy_version 481524 (0.0008) [2023-12-26 18:52:14,363][105692] Updated weights for policy 0, policy_version 481534 (0.0011) [2023-12-26 18:52:14,815][105620] Updated weights for policy 1, policy_version 481969 (0.0010) [2023-12-26 18:52:14,878][105620] Updated weights for policy 1, policy_version 481979 (0.0011) [2023-12-26 18:52:14,932][105620] Updated weights for policy 1, policy_version 481989 (0.0011) [2023-12-26 18:52:14,974][105692] Updated weights for policy 0, policy_version 481544 (0.0011) [2023-12-26 18:52:15,040][105692] Updated weights for policy 0, policy_version 481554 (0.0010) [2023-12-26 18:52:15,099][105692] Updated weights for policy 0, policy_version 481564 (0.0010) [2023-12-26 18:52:15,574][105620] Updated weights for policy 1, policy_version 481999 (0.0008) [2023-12-26 18:52:15,629][105620] Updated weights for policy 1, policy_version 482009 (0.0010) [2023-12-26 18:52:15,677][105620] Updated weights for policy 1, policy_version 482019 (0.0005) [2023-12-26 18:52:15,850][105692] Updated weights for policy 0, policy_version 481574 (0.0011) [2023-12-26 18:52:15,919][105692] Updated weights for policy 0, policy_version 481584 (0.0010) [2023-12-26 18:52:15,973][105692] Updated weights for policy 0, policy_version 481594 (0.0008) [2023-12-26 18:52:16,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 246718464. Throughput: 0: 9465.3, 1: 9840.7. Samples: 246683240. Policy #0 lag: (min: 20.0, avg: 22.5, max: 48.0) [2023-12-26 18:52:16,062][104569] Avg episode reward: [(0, '8819.270'), (1, '9266.461')] [2023-12-26 18:52:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000481600_123305984.pth... [2023-12-26 18:52:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000482024_123412480.pth... [2023-12-26 18:52:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000480480_123019264.pth [2023-12-26 18:52:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000480904_123125760.pth [2023-12-26 18:52:16,332][105620] Updated weights for policy 1, policy_version 482029 (0.0008) [2023-12-26 18:52:16,388][105620] Updated weights for policy 1, policy_version 482039 (0.0010) [2023-12-26 18:52:16,443][105620] Updated weights for policy 1, policy_version 482049 (0.0010) [2023-12-26 18:52:16,521][105692] Updated weights for policy 0, policy_version 481604 (0.0008) [2023-12-26 18:52:16,587][105692] Updated weights for policy 0, policy_version 481614 (0.0009) [2023-12-26 18:52:16,658][105692] Updated weights for policy 0, policy_version 481624 (0.0008) [2023-12-26 18:52:17,198][105620] Updated weights for policy 1, policy_version 482059 (0.0010) [2023-12-26 18:52:17,246][105620] Updated weights for policy 1, policy_version 482069 (0.0010) [2023-12-26 18:52:17,278][105692] Updated weights for policy 0, policy_version 481634 (0.0008) [2023-12-26 18:52:17,295][105620] Updated weights for policy 1, policy_version 482079 (0.0010) [2023-12-26 18:52:17,329][105692] Updated weights for policy 0, policy_version 481644 (0.0005) [2023-12-26 18:52:17,389][105692] Updated weights for policy 0, policy_version 481654 (0.0008) [2023-12-26 18:52:17,449][105692] Updated weights for policy 0, policy_version 481664 (0.0008) [2023-12-26 18:52:18,056][105620] Updated weights for policy 1, policy_version 482089 (0.0010) [2023-12-26 18:52:18,118][105620] Updated weights for policy 1, policy_version 482099 (0.0010) [2023-12-26 18:52:18,177][105620] Updated weights for policy 1, policy_version 482109 (0.0010) [2023-12-26 18:52:18,221][105692] Updated weights for policy 0, policy_version 481674 (0.0008) [2023-12-26 18:52:18,232][105620] Updated weights for policy 1, policy_version 482119 (0.0010) [2023-12-26 18:52:18,262][105585] KL-divergence is very high: 128.2470 [2023-12-26 18:52:18,278][105692] Updated weights for policy 0, policy_version 481684 (0.0007) [2023-12-26 18:52:18,306][105585] KL-divergence is very high: 127.7800 [2023-12-26 18:52:18,334][105692] Updated weights for policy 0, policy_version 481694 (0.0008) [2023-12-26 18:52:18,958][105620] Updated weights for policy 1, policy_version 482129 (0.0011) [2023-12-26 18:52:19,018][105620] Updated weights for policy 1, policy_version 482139 (0.0010) [2023-12-26 18:52:19,080][105620] Updated weights for policy 1, policy_version 482149 (0.0011) [2023-12-26 18:52:19,122][105692] Updated weights for policy 0, policy_version 481704 (0.0009) [2023-12-26 18:52:19,170][105692] Updated weights for policy 0, policy_version 481714 (0.0008) [2023-12-26 18:52:19,226][105692] Updated weights for policy 0, policy_version 481724 (0.0010) [2023-12-26 18:52:19,739][105620] Updated weights for policy 1, policy_version 482159 (0.0011) [2023-12-26 18:52:19,800][105620] Updated weights for policy 1, policy_version 482169 (0.0011) [2023-12-26 18:52:19,865][105620] Updated weights for policy 1, policy_version 482179 (0.0010) [2023-12-26 18:52:20,033][105692] Updated weights for policy 0, policy_version 481734 (0.0007) [2023-12-26 18:52:20,092][105692] Updated weights for policy 0, policy_version 481744 (0.0005) [2023-12-26 18:52:20,155][105692] Updated weights for policy 0, policy_version 481754 (0.0008) [2023-12-26 18:52:20,640][105620] Updated weights for policy 1, policy_version 482189 (0.0011) [2023-12-26 18:52:20,704][105620] Updated weights for policy 1, policy_version 482199 (0.0011) [2023-12-26 18:52:20,772][105620] Updated weights for policy 1, policy_version 482209 (0.0011) [2023-12-26 18:52:20,884][105692] Updated weights for policy 0, policy_version 481764 (0.0008) [2023-12-26 18:52:20,935][105692] Updated weights for policy 0, policy_version 481774 (0.0009) [2023-12-26 18:52:20,990][105692] Updated weights for policy 0, policy_version 481784 (0.0009) [2023-12-26 18:52:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 246816768. Throughput: 0: 9579.2, 1: 9762.4. Samples: 246801972. Policy #0 lag: (min: 20.0, avg: 22.5, max: 48.0) [2023-12-26 18:52:21,063][104569] Avg episode reward: [(0, '8729.457'), (1, '9265.521')] [2023-12-26 18:52:21,478][105620] Updated weights for policy 1, policy_version 482219 (0.0010) [2023-12-26 18:52:21,526][105620] Updated weights for policy 1, policy_version 482229 (0.0008) [2023-12-26 18:52:21,577][105620] Updated weights for policy 1, policy_version 482239 (0.0008) [2023-12-26 18:52:21,802][105692] Updated weights for policy 0, policy_version 481794 (0.0009) [2023-12-26 18:52:21,869][105692] Updated weights for policy 0, policy_version 481804 (0.0008) [2023-12-26 18:52:21,928][105692] Updated weights for policy 0, policy_version 481814 (0.0008) [2023-12-26 18:52:21,991][105692] Updated weights for policy 0, policy_version 481824 (0.0009) [2023-12-26 18:52:22,417][105620] Updated weights for policy 1, policy_version 482249 (0.0008) [2023-12-26 18:52:22,483][105620] Updated weights for policy 1, policy_version 482259 (0.0009) [2023-12-26 18:52:22,544][105620] Updated weights for policy 1, policy_version 482269 (0.0009) [2023-12-26 18:52:22,606][105620] Updated weights for policy 1, policy_version 482279 (0.0006) [2023-12-26 18:52:22,634][105692] Updated weights for policy 0, policy_version 481834 (0.0009) [2023-12-26 18:52:22,686][105692] Updated weights for policy 0, policy_version 481844 (0.0009) [2023-12-26 18:52:22,745][105692] Updated weights for policy 0, policy_version 481854 (0.0007) [2023-12-26 18:52:23,412][105620] Updated weights for policy 1, policy_version 482289 (0.0008) [2023-12-26 18:52:23,447][105692] Updated weights for policy 0, policy_version 481864 (0.0007) [2023-12-26 18:52:23,458][105620] Updated weights for policy 1, policy_version 482299 (0.0006) [2023-12-26 18:52:23,501][105692] Updated weights for policy 0, policy_version 481874 (0.0006) [2023-12-26 18:52:23,514][105620] Updated weights for policy 1, policy_version 482309 (0.0007) [2023-12-26 18:52:23,556][105692] Updated weights for policy 0, policy_version 481884 (0.0006) [2023-12-26 18:52:24,159][105620] Updated weights for policy 1, policy_version 482319 (0.0008) [2023-12-26 18:52:24,226][105620] Updated weights for policy 1, policy_version 482329 (0.0007) [2023-12-26 18:52:24,227][105692] Updated weights for policy 0, policy_version 481894 (0.0007) [2023-12-26 18:52:24,276][105620] Updated weights for policy 1, policy_version 482339 (0.0008) [2023-12-26 18:52:24,290][105692] Updated weights for policy 0, policy_version 481904 (0.0007) [2023-12-26 18:52:24,343][105692] Updated weights for policy 0, policy_version 481914 (0.0006) [2023-12-26 18:52:24,899][105692] Updated weights for policy 0, policy_version 481924 (0.0005) [2023-12-26 18:52:24,951][105692] Updated weights for policy 0, policy_version 481934 (0.0006) [2023-12-26 18:52:25,004][105692] Updated weights for policy 0, policy_version 481944 (0.0008) [2023-12-26 18:52:25,139][105620] Updated weights for policy 1, policy_version 482349 (0.0007) [2023-12-26 18:52:25,194][105620] Updated weights for policy 1, policy_version 482359 (0.0009) [2023-12-26 18:52:25,252][105620] Updated weights for policy 1, policy_version 482369 (0.0009) [2023-12-26 18:52:25,621][105692] Updated weights for policy 0, policy_version 481954 (0.0009) [2023-12-26 18:52:25,680][105692] Updated weights for policy 0, policy_version 481964 (0.0007) [2023-12-26 18:52:25,729][105692] Updated weights for policy 0, policy_version 481974 (0.0005) [2023-12-26 18:52:25,775][105692] Updated weights for policy 0, policy_version 481984 (0.0005) [2023-12-26 18:52:26,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 246906880. Throughput: 0: 9722.9, 1: 9586.5. Samples: 246916756. Policy #0 lag: (min: 20.0, avg: 22.5, max: 48.0) [2023-12-26 18:52:26,062][104569] Avg episode reward: [(0, '9088.885'), (1, '9176.212')] [2023-12-26 18:52:26,083][105620] Updated weights for policy 1, policy_version 482379 (0.0009) [2023-12-26 18:52:26,135][105620] Updated weights for policy 1, policy_version 482389 (0.0009) [2023-12-26 18:52:26,186][105620] Updated weights for policy 1, policy_version 482399 (0.0009) [2023-12-26 18:52:26,375][105692] Updated weights for policy 0, policy_version 481994 (0.0008) [2023-12-26 18:52:26,422][105692] Updated weights for policy 0, policy_version 482004 (0.0009) [2023-12-26 18:52:26,474][105692] Updated weights for policy 0, policy_version 482014 (0.0009) [2023-12-26 18:52:27,052][105620] Updated weights for policy 1, policy_version 482409 (0.0009) [2023-12-26 18:52:27,084][105692] Updated weights for policy 0, policy_version 482024 (0.0006) [2023-12-26 18:52:27,098][105620] Updated weights for policy 1, policy_version 482419 (0.0007) [2023-12-26 18:52:27,132][105692] Updated weights for policy 0, policy_version 482034 (0.0005) [2023-12-26 18:52:27,162][105620] Updated weights for policy 1, policy_version 482429 (0.0008) [2023-12-26 18:52:27,189][105692] Updated weights for policy 0, policy_version 482044 (0.0007) [2023-12-26 18:52:27,214][105620] Updated weights for policy 1, policy_version 482439 (0.0006) [2023-12-26 18:52:27,934][105692] Updated weights for policy 0, policy_version 482054 (0.0006) [2023-12-26 18:52:27,952][105620] Updated weights for policy 1, policy_version 482449 (0.0007) [2023-12-26 18:52:27,984][105692] Updated weights for policy 0, policy_version 482064 (0.0006) [2023-12-26 18:52:27,998][105620] Updated weights for policy 1, policy_version 482459 (0.0006) [2023-12-26 18:52:28,034][105692] Updated weights for policy 0, policy_version 482074 (0.0005) [2023-12-26 18:52:28,055][105620] Updated weights for policy 1, policy_version 482469 (0.0007) [2023-12-26 18:52:28,748][105692] Updated weights for policy 0, policy_version 482084 (0.0008) [2023-12-26 18:52:28,812][105692] Updated weights for policy 0, policy_version 482094 (0.0007) [2023-12-26 18:52:28,818][105620] Updated weights for policy 1, policy_version 482479 (0.0006) [2023-12-26 18:52:28,862][105692] Updated weights for policy 0, policy_version 482104 (0.0008) [2023-12-26 18:52:28,868][105620] Updated weights for policy 1, policy_version 482489 (0.0006) [2023-12-26 18:52:28,913][105620] Updated weights for policy 1, policy_version 482499 (0.0006) [2023-12-26 18:52:29,467][105692] Updated weights for policy 0, policy_version 482114 (0.0009) [2023-12-26 18:52:29,523][105692] Updated weights for policy 0, policy_version 482124 (0.0006) [2023-12-26 18:52:29,579][105692] Updated weights for policy 0, policy_version 482134 (0.0008) [2023-12-26 18:52:29,636][105692] Updated weights for policy 0, policy_version 482144 (0.0010) [2023-12-26 18:52:29,779][105620] Updated weights for policy 1, policy_version 482509 (0.0009) [2023-12-26 18:52:29,844][105620] Updated weights for policy 1, policy_version 482519 (0.0008) [2023-12-26 18:52:29,902][105620] Updated weights for policy 1, policy_version 482529 (0.0009) [2023-12-26 18:52:30,271][105692] Updated weights for policy 0, policy_version 482154 (0.0009) [2023-12-26 18:52:30,333][105692] Updated weights for policy 0, policy_version 482164 (0.0011) [2023-12-26 18:52:30,399][105692] Updated weights for policy 0, policy_version 482174 (0.0006) [2023-12-26 18:52:30,772][105620] Updated weights for policy 1, policy_version 482539 (0.0008) [2023-12-26 18:52:30,838][105620] Updated weights for policy 1, policy_version 482549 (0.0010) [2023-12-26 18:52:30,889][105620] Updated weights for policy 1, policy_version 482559 (0.0009) [2023-12-26 18:52:30,950][105692] Updated weights for policy 0, policy_version 482184 (0.0006) [2023-12-26 18:52:31,003][105692] Updated weights for policy 0, policy_version 482194 (0.0005) [2023-12-26 18:52:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 247005184. Throughput: 0: 9822.8, 1: 9518.8. Samples: 246975508. Policy #0 lag: (min: 11.0, avg: 20.6, max: 43.0) [2023-12-26 18:52:31,062][104569] Avg episode reward: [(0, '9270.327'), (1, '9084.470')] [2023-12-26 18:52:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000482568_123551744.pth... [2023-12-26 18:52:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000481480_123273216.pth [2023-12-26 18:52:31,080][105692] Updated weights for policy 0, policy_version 482204 (0.0009) [2023-12-26 18:52:31,101][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000482208_123461632.pth... [2023-12-26 18:52:31,105][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000481024_123158528.pth [2023-12-26 18:52:31,698][105620] Updated weights for policy 1, policy_version 482569 (0.0009) [2023-12-26 18:52:31,750][105692] Updated weights for policy 0, policy_version 482214 (0.0010) [2023-12-26 18:52:31,756][105620] Updated weights for policy 1, policy_version 482579 (0.0007) [2023-12-26 18:52:31,798][105692] Updated weights for policy 0, policy_version 482224 (0.0010) [2023-12-26 18:52:31,804][105620] Updated weights for policy 1, policy_version 482589 (0.0005) [2023-12-26 18:52:31,853][105692] Updated weights for policy 0, policy_version 482234 (0.0010) [2023-12-26 18:52:31,859][105620] Updated weights for policy 1, policy_version 482599 (0.0005) [2023-12-26 18:52:32,532][105620] Updated weights for policy 1, policy_version 482609 (0.0010) [2023-12-26 18:52:32,587][105620] Updated weights for policy 1, policy_version 482619 (0.0010) [2023-12-26 18:52:32,618][105692] Updated weights for policy 0, policy_version 482244 (0.0008) [2023-12-26 18:52:32,646][105620] Updated weights for policy 1, policy_version 482629 (0.0011) [2023-12-26 18:52:32,678][105692] Updated weights for policy 0, policy_version 482254 (0.0007) [2023-12-26 18:52:32,729][105692] Updated weights for policy 0, policy_version 482264 (0.0008) [2023-12-26 18:52:33,340][105692] Updated weights for policy 0, policy_version 482274 (0.0008) [2023-12-26 18:52:33,391][105692] Updated weights for policy 0, policy_version 482284 (0.0006) [2023-12-26 18:52:33,397][105620] Updated weights for policy 1, policy_version 482639 (0.0011) [2023-12-26 18:52:33,439][105692] Updated weights for policy 0, policy_version 482294 (0.0006) [2023-12-26 18:52:33,456][105620] Updated weights for policy 1, policy_version 482649 (0.0010) [2023-12-26 18:52:33,496][105692] Updated weights for policy 0, policy_version 482304 (0.0007) [2023-12-26 18:52:33,514][105620] Updated weights for policy 1, policy_version 482659 (0.0010) [2023-12-26 18:52:34,091][105692] Updated weights for policy 0, policy_version 482314 (0.0005) [2023-12-26 18:52:34,146][105692] Updated weights for policy 0, policy_version 482324 (0.0006) [2023-12-26 18:52:34,204][105692] Updated weights for policy 0, policy_version 482334 (0.0008) [2023-12-26 18:52:34,260][105620] Updated weights for policy 1, policy_version 482669 (0.0010) [2023-12-26 18:52:34,327][105620] Updated weights for policy 1, policy_version 482679 (0.0011) [2023-12-26 18:52:34,380][105620] Updated weights for policy 1, policy_version 482689 (0.0010) [2023-12-26 18:52:34,850][105692] Updated weights for policy 0, policy_version 482344 (0.0006) [2023-12-26 18:52:34,919][105692] Updated weights for policy 0, policy_version 482354 (0.0005) [2023-12-26 18:52:34,981][105692] Updated weights for policy 0, policy_version 482364 (0.0006) [2023-12-26 18:52:35,128][105620] Updated weights for policy 1, policy_version 482699 (0.0009) [2023-12-26 18:52:35,180][105620] Updated weights for policy 1, policy_version 482709 (0.0005) [2023-12-26 18:52:35,234][105620] Updated weights for policy 1, policy_version 482719 (0.0005) [2023-12-26 18:52:35,585][105692] Updated weights for policy 0, policy_version 482374 (0.0007) [2023-12-26 18:52:35,650][105692] Updated weights for policy 0, policy_version 482384 (0.0008) [2023-12-26 18:52:35,708][105692] Updated weights for policy 0, policy_version 482394 (0.0009) [2023-12-26 18:52:35,866][105620] Updated weights for policy 1, policy_version 482729 (0.0006) [2023-12-26 18:52:35,923][105620] Updated weights for policy 1, policy_version 482739 (0.0010) [2023-12-26 18:52:35,988][105620] Updated weights for policy 1, policy_version 482749 (0.0010) [2023-12-26 18:52:36,053][105620] Updated weights for policy 1, policy_version 482759 (0.0010) [2023-12-26 18:52:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 247111680. Throughput: 0: 9947.1, 1: 9408.0. Samples: 247094016. Policy #0 lag: (min: 11.0, avg: 20.6, max: 43.0) [2023-12-26 18:52:36,063][104569] Avg episode reward: [(0, '3185.494'), (1, '8902.124')] [2023-12-26 18:52:36,462][105692] Updated weights for policy 0, policy_version 482404 (0.0008) [2023-12-26 18:52:36,525][105692] Updated weights for policy 0, policy_version 482414 (0.0005) [2023-12-26 18:52:36,582][105692] Updated weights for policy 0, policy_version 482424 (0.0006) [2023-12-26 18:52:36,720][105620] Updated weights for policy 1, policy_version 482769 (0.0010) [2023-12-26 18:52:36,781][105620] Updated weights for policy 1, policy_version 482779 (0.0010) [2023-12-26 18:52:36,844][105620] Updated weights for policy 1, policy_version 482789 (0.0010) [2023-12-26 18:52:37,152][105692] Updated weights for policy 0, policy_version 482434 (0.0009) [2023-12-26 18:52:37,211][105692] Updated weights for policy 0, policy_version 482444 (0.0008) [2023-12-26 18:52:37,264][105692] Updated weights for policy 0, policy_version 482454 (0.0008) [2023-12-26 18:52:37,321][105692] Updated weights for policy 0, policy_version 482464 (0.0008) [2023-12-26 18:52:37,588][105620] Updated weights for policy 1, policy_version 482799 (0.0009) [2023-12-26 18:52:37,638][105620] Updated weights for policy 1, policy_version 482809 (0.0009) [2023-12-26 18:52:37,698][105620] Updated weights for policy 1, policy_version 482819 (0.0010) [2023-12-26 18:52:37,995][105692] Updated weights for policy 0, policy_version 482474 (0.0010) [2023-12-26 18:52:38,053][105692] Updated weights for policy 0, policy_version 482484 (0.0010) [2023-12-26 18:52:38,102][105692] Updated weights for policy 0, policy_version 482494 (0.0006) [2023-12-26 18:52:38,339][105620] Updated weights for policy 1, policy_version 482829 (0.0007) [2023-12-26 18:52:38,405][105620] Updated weights for policy 1, policy_version 482839 (0.0006) [2023-12-26 18:52:38,461][105620] Updated weights for policy 1, policy_version 482849 (0.0005) [2023-12-26 18:52:38,852][105692] Updated weights for policy 0, policy_version 482504 (0.0006) [2023-12-26 18:52:38,910][105692] Updated weights for policy 0, policy_version 482514 (0.0005) [2023-12-26 18:52:38,968][105692] Updated weights for policy 0, policy_version 482524 (0.0007) [2023-12-26 18:52:39,021][105620] Updated weights for policy 1, policy_version 482859 (0.0005) [2023-12-26 18:52:39,080][105620] Updated weights for policy 1, policy_version 482869 (0.0008) [2023-12-26 18:52:39,140][105620] Updated weights for policy 1, policy_version 482879 (0.0005) [2023-12-26 18:52:39,726][105692] Updated weights for policy 0, policy_version 482534 (0.0009) [2023-12-26 18:52:39,784][105692] Updated weights for policy 0, policy_version 482544 (0.0006) [2023-12-26 18:52:39,850][105620] Updated weights for policy 1, policy_version 482889 (0.0006) [2023-12-26 18:52:39,853][105692] Updated weights for policy 0, policy_version 482554 (0.0008) [2023-12-26 18:52:39,914][105620] Updated weights for policy 1, policy_version 482899 (0.0008) [2023-12-26 18:52:39,980][105620] Updated weights for policy 1, policy_version 482909 (0.0009) [2023-12-26 18:52:40,039][105620] Updated weights for policy 1, policy_version 482919 (0.0009) [2023-12-26 18:52:40,550][105692] Updated weights for policy 0, policy_version 482564 (0.0006) [2023-12-26 18:52:40,617][105692] Updated weights for policy 0, policy_version 482574 (0.0006) [2023-12-26 18:52:40,675][105692] Updated weights for policy 0, policy_version 482584 (0.0009) [2023-12-26 18:52:40,840][105620] Updated weights for policy 1, policy_version 482929 (0.0008) [2023-12-26 18:52:40,906][105620] Updated weights for policy 1, policy_version 482939 (0.0009) [2023-12-26 18:52:40,964][105620] Updated weights for policy 1, policy_version 482949 (0.0006) [2023-12-26 18:52:41,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 247209984. Throughput: 0: 10018.3, 1: 9506.5. Samples: 247214932. Policy #0 lag: (min: 11.0, avg: 20.6, max: 43.0) [2023-12-26 18:52:41,062][104569] Avg episode reward: [(0, '5425.877'), (1, '8544.451')] [2023-12-26 18:52:41,444][105692] Updated weights for policy 0, policy_version 482594 (0.0008) [2023-12-26 18:52:41,509][105692] Updated weights for policy 0, policy_version 482604 (0.0009) [2023-12-26 18:52:41,571][105692] Updated weights for policy 0, policy_version 482614 (0.0008) [2023-12-26 18:52:41,639][105692] Updated weights for policy 0, policy_version 482624 (0.0008) [2023-12-26 18:52:41,661][105620] Updated weights for policy 1, policy_version 482959 (0.0009) [2023-12-26 18:52:41,732][105620] Updated weights for policy 1, policy_version 482969 (0.0009) [2023-12-26 18:52:41,795][105620] Updated weights for policy 1, policy_version 482979 (0.0009) [2023-12-26 18:52:42,393][105692] Updated weights for policy 0, policy_version 482634 (0.0009) [2023-12-26 18:52:42,453][105692] Updated weights for policy 0, policy_version 482644 (0.0009) [2023-12-26 18:52:42,505][105692] Updated weights for policy 0, policy_version 482654 (0.0008) [2023-12-26 18:52:42,574][105620] Updated weights for policy 1, policy_version 482989 (0.0010) [2023-12-26 18:52:42,633][105620] Updated weights for policy 1, policy_version 482999 (0.0011) [2023-12-26 18:52:42,682][105620] Updated weights for policy 1, policy_version 483009 (0.0010) [2023-12-26 18:52:43,284][105692] Updated weights for policy 0, policy_version 482664 (0.0010) [2023-12-26 18:52:43,341][105692] Updated weights for policy 0, policy_version 482674 (0.0010) [2023-12-26 18:52:43,404][105692] Updated weights for policy 0, policy_version 482684 (0.0010) [2023-12-26 18:52:43,448][105620] Updated weights for policy 1, policy_version 483019 (0.0010) [2023-12-26 18:52:43,504][105620] Updated weights for policy 1, policy_version 483029 (0.0010) [2023-12-26 18:52:43,553][105620] Updated weights for policy 1, policy_version 483039 (0.0011) [2023-12-26 18:52:44,152][105692] Updated weights for policy 0, policy_version 482694 (0.0010) [2023-12-26 18:52:44,210][105692] Updated weights for policy 0, policy_version 482704 (0.0010) [2023-12-26 18:52:44,267][105692] Updated weights for policy 0, policy_version 482714 (0.0010) [2023-12-26 18:52:44,327][105620] Updated weights for policy 1, policy_version 483049 (0.0011) [2023-12-26 18:52:44,392][105620] Updated weights for policy 1, policy_version 483059 (0.0010) [2023-12-26 18:52:44,444][105620] Updated weights for policy 1, policy_version 483069 (0.0010) [2023-12-26 18:52:44,498][105620] Updated weights for policy 1, policy_version 483079 (0.0010) [2023-12-26 18:52:45,014][105692] Updated weights for policy 0, policy_version 482724 (0.0010) [2023-12-26 18:52:45,079][105692] Updated weights for policy 0, policy_version 482734 (0.0010) [2023-12-26 18:52:45,131][105620] Updated weights for policy 1, policy_version 483089 (0.0010) [2023-12-26 18:52:45,139][105692] Updated weights for policy 0, policy_version 482744 (0.0010) [2023-12-26 18:52:45,192][105620] Updated weights for policy 1, policy_version 483099 (0.0011) [2023-12-26 18:52:45,238][105620] Updated weights for policy 1, policy_version 483109 (0.0010) [2023-12-26 18:52:45,768][105692] Updated weights for policy 0, policy_version 482754 (0.0009) [2023-12-26 18:52:45,835][105692] Updated weights for policy 0, policy_version 482764 (0.0009) [2023-12-26 18:52:45,891][105620] Updated weights for policy 1, policy_version 483119 (0.0011) [2023-12-26 18:52:45,893][105692] Updated weights for policy 0, policy_version 482774 (0.0010) [2023-12-26 18:52:45,947][105692] Updated weights for policy 0, policy_version 482784 (0.0005) [2023-12-26 18:52:45,950][105620] Updated weights for policy 1, policy_version 483129 (0.0010) [2023-12-26 18:52:46,002][105620] Updated weights for policy 1, policy_version 483139 (0.0010) [2023-12-26 18:52:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 247308288. Throughput: 0: 10001.6, 1: 9523.2. Samples: 247270044. Policy #0 lag: (min: 11.0, avg: 20.6, max: 43.0) [2023-12-26 18:52:46,063][104569] Avg episode reward: [(0, '6778.965'), (1, '9248.624')] [2023-12-26 18:52:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000482784_123609088.pth... [2023-12-26 18:52:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000483144_123699200.pth... [2023-12-26 18:52:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000481600_123305984.pth [2023-12-26 18:52:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000482024_123412480.pth [2023-12-26 18:52:46,544][105692] Updated weights for policy 0, policy_version 482794 (0.0008) [2023-12-26 18:52:46,592][105692] Updated weights for policy 0, policy_version 482804 (0.0006) [2023-12-26 18:52:46,641][105692] Updated weights for policy 0, policy_version 482814 (0.0005) [2023-12-26 18:52:46,737][105620] Updated weights for policy 1, policy_version 483149 (0.0010) [2023-12-26 18:52:46,792][105620] Updated weights for policy 1, policy_version 483159 (0.0010) [2023-12-26 18:52:46,847][105620] Updated weights for policy 1, policy_version 483169 (0.0010) [2023-12-26 18:52:47,359][105692] Updated weights for policy 0, policy_version 482824 (0.0007) [2023-12-26 18:52:47,405][105585] KL-divergence is very high: 117.0401 [2023-12-26 18:52:47,417][105692] Updated weights for policy 0, policy_version 482834 (0.0008) [2023-12-26 18:52:47,425][105585] KL-divergence is very high: 111.1464 [2023-12-26 18:52:47,441][105585] KL-divergence is very high: 126.1597 [2023-12-26 18:52:47,446][105585] KL-divergence is very high: 129.7383 [2023-12-26 18:52:47,468][105692] Updated weights for policy 0, policy_version 482844 (0.0008) [2023-12-26 18:52:47,575][105620] Updated weights for policy 1, policy_version 483179 (0.0010) [2023-12-26 18:52:47,627][105620] Updated weights for policy 1, policy_version 483189 (0.0010) [2023-12-26 18:52:47,678][105620] Updated weights for policy 1, policy_version 483199 (0.0010) [2023-12-26 18:52:48,184][105692] Updated weights for policy 0, policy_version 482854 (0.0009) [2023-12-26 18:52:48,240][105692] Updated weights for policy 0, policy_version 482864 (0.0010) [2023-12-26 18:52:48,286][105692] Updated weights for policy 0, policy_version 482874 (0.0010) [2023-12-26 18:52:48,422][105620] Updated weights for policy 1, policy_version 483209 (0.0010) [2023-12-26 18:52:48,482][105620] Updated weights for policy 1, policy_version 483219 (0.0008) [2023-12-26 18:52:48,547][105620] Updated weights for policy 1, policy_version 483229 (0.0007) [2023-12-26 18:52:48,617][105620] Updated weights for policy 1, policy_version 483239 (0.0006) [2023-12-26 18:52:48,934][105692] Updated weights for policy 0, policy_version 482884 (0.0008) [2023-12-26 18:52:48,996][105692] Updated weights for policy 0, policy_version 482894 (0.0005) [2023-12-26 18:52:49,055][105692] Updated weights for policy 0, policy_version 482904 (0.0010) [2023-12-26 18:52:49,192][105620] Updated weights for policy 1, policy_version 483249 (0.0005) [2023-12-26 18:52:49,262][105620] Updated weights for policy 1, policy_version 483259 (0.0007) [2023-12-26 18:52:49,328][105620] Updated weights for policy 1, policy_version 483269 (0.0007) [2023-12-26 18:52:49,726][105692] Updated weights for policy 0, policy_version 482914 (0.0009) [2023-12-26 18:52:49,785][105692] Updated weights for policy 0, policy_version 482924 (0.0007) [2023-12-26 18:52:49,853][105692] Updated weights for policy 0, policy_version 482934 (0.0007) [2023-12-26 18:52:49,884][105620] Updated weights for policy 1, policy_version 483279 (0.0008) [2023-12-26 18:52:49,914][105692] Updated weights for policy 0, policy_version 482944 (0.0009) [2023-12-26 18:52:49,943][105620] Updated weights for policy 1, policy_version 483289 (0.0008) [2023-12-26 18:52:49,997][105620] Updated weights for policy 1, policy_version 483299 (0.0007) [2023-12-26 18:52:50,613][105620] Updated weights for policy 1, policy_version 483309 (0.0008) [2023-12-26 18:52:50,676][105620] Updated weights for policy 1, policy_version 483319 (0.0006) [2023-12-26 18:52:50,724][105692] Updated weights for policy 0, policy_version 482954 (0.0008) [2023-12-26 18:52:50,741][105620] Updated weights for policy 1, policy_version 483329 (0.0008) [2023-12-26 18:52:50,779][105692] Updated weights for policy 0, policy_version 482964 (0.0008) [2023-12-26 18:52:50,837][105692] Updated weights for policy 0, policy_version 482974 (0.0008) [2023-12-26 18:52:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 247406592. Throughput: 0: 10041.6, 1: 9590.1. Samples: 247392600. Policy #0 lag: (min: 11.0, avg: 20.6, max: 43.0) [2023-12-26 18:52:51,063][104569] Avg episode reward: [(0, '8362.745'), (1, '9269.593')] [2023-12-26 18:52:51,472][105620] Updated weights for policy 1, policy_version 483339 (0.0008) [2023-12-26 18:52:51,537][105620] Updated weights for policy 1, policy_version 483349 (0.0010) [2023-12-26 18:52:51,601][105620] Updated weights for policy 1, policy_version 483359 (0.0009) [2023-12-26 18:52:51,653][105692] Updated weights for policy 0, policy_version 482984 (0.0010) [2023-12-26 18:52:51,718][105692] Updated weights for policy 0, policy_version 482995 (0.0009) [2023-12-26 18:52:51,783][105692] Updated weights for policy 0, policy_version 483005 (0.0007) [2023-12-26 18:52:52,242][105620] Updated weights for policy 1, policy_version 483369 (0.0007) [2023-12-26 18:52:52,304][105620] Updated weights for policy 1, policy_version 483379 (0.0009) [2023-12-26 18:52:52,366][105620] Updated weights for policy 1, policy_version 483389 (0.0008) [2023-12-26 18:52:52,434][105620] Updated weights for policy 1, policy_version 483399 (0.0005) [2023-12-26 18:52:52,585][105692] Updated weights for policy 0, policy_version 483015 (0.0009) [2023-12-26 18:52:52,632][105692] Updated weights for policy 0, policy_version 483025 (0.0008) [2023-12-26 18:52:52,682][105692] Updated weights for policy 0, policy_version 483035 (0.0008) [2023-12-26 18:52:53,095][105620] Updated weights for policy 1, policy_version 483409 (0.0008) [2023-12-26 18:52:53,150][105620] Updated weights for policy 1, policy_version 483419 (0.0009) [2023-12-26 18:52:53,201][105620] Updated weights for policy 1, policy_version 483429 (0.0009) [2023-12-26 18:52:53,490][105692] Updated weights for policy 0, policy_version 483045 (0.0009) [2023-12-26 18:52:53,545][105692] Updated weights for policy 0, policy_version 483055 (0.0009) [2023-12-26 18:52:53,576][105585] KL-divergence is very high: 104.9651 [2023-12-26 18:52:53,600][105692] Updated weights for policy 0, policy_version 483065 (0.0010) [2023-12-26 18:52:53,851][105620] Updated weights for policy 1, policy_version 483439 (0.0006) [2023-12-26 18:52:53,895][105620] Updated weights for policy 1, policy_version 483449 (0.0005) [2023-12-26 18:52:53,954][105620] Updated weights for policy 1, policy_version 483459 (0.0005) [2023-12-26 18:52:54,485][105620] Updated weights for policy 1, policy_version 483469 (0.0008) [2023-12-26 18:52:54,488][105692] Updated weights for policy 0, policy_version 483075 (0.0009) [2023-12-26 18:52:54,539][105692] Updated weights for policy 0, policy_version 483085 (0.0005) [2023-12-26 18:52:54,540][105620] Updated weights for policy 1, policy_version 483479 (0.0010) [2023-12-26 18:52:54,594][105692] Updated weights for policy 0, policy_version 483095 (0.0005) [2023-12-26 18:52:54,595][105620] Updated weights for policy 1, policy_version 483489 (0.0010) [2023-12-26 18:52:55,269][105692] Updated weights for policy 0, policy_version 483105 (0.0006) [2023-12-26 18:52:55,317][105692] Updated weights for policy 0, policy_version 483115 (0.0008) [2023-12-26 18:52:55,341][105620] Updated weights for policy 1, policy_version 483499 (0.0010) [2023-12-26 18:52:55,364][105692] Updated weights for policy 0, policy_version 483125 (0.0007) [2023-12-26 18:52:55,396][105620] Updated weights for policy 1, policy_version 483509 (0.0010) [2023-12-26 18:52:55,410][105692] Updated weights for policy 0, policy_version 483135 (0.0007) [2023-12-26 18:52:55,447][105620] Updated weights for policy 1, policy_version 483519 (0.0010) [2023-12-26 18:52:56,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 247496704. Throughput: 0: 9946.6, 1: 9675.5. Samples: 247509136. Policy #0 lag: (min: 11.0, avg: 20.6, max: 43.0) [2023-12-26 18:52:56,062][104569] Avg episode reward: [(0, '8546.013'), (1, '9264.913')] [2023-12-26 18:52:56,185][105692] Updated weights for policy 0, policy_version 483145 (0.0010) [2023-12-26 18:52:56,195][105620] Updated weights for policy 1, policy_version 483529 (0.0010) [2023-12-26 18:52:56,240][105620] Updated weights for policy 1, policy_version 483539 (0.0010) [2023-12-26 18:52:56,244][105692] Updated weights for policy 0, policy_version 483155 (0.0011) [2023-12-26 18:52:56,288][105620] Updated weights for policy 1, policy_version 483549 (0.0010) [2023-12-26 18:52:56,303][105692] Updated weights for policy 0, policy_version 483165 (0.0010) [2023-12-26 18:52:56,339][105620] Updated weights for policy 1, policy_version 483559 (0.0010) [2023-12-26 18:52:56,989][105692] Updated weights for policy 0, policy_version 483175 (0.0008) [2023-12-26 18:52:57,033][105692] Updated weights for policy 0, policy_version 483185 (0.0010) [2023-12-26 18:52:57,081][105692] Updated weights for policy 0, policy_version 483195 (0.0008) [2023-12-26 18:52:57,112][105620] Updated weights for policy 1, policy_version 483569 (0.0010) [2023-12-26 18:52:57,159][105620] Updated weights for policy 1, policy_version 483579 (0.0010) [2023-12-26 18:52:57,216][105620] Updated weights for policy 1, policy_version 483589 (0.0010) [2023-12-26 18:52:57,605][105692] Updated weights for policy 0, policy_version 483205 (0.0008) [2023-12-26 18:52:57,662][105692] Updated weights for policy 0, policy_version 483215 (0.0010) [2023-12-26 18:52:57,722][105692] Updated weights for policy 0, policy_version 483225 (0.0010) [2023-12-26 18:52:57,906][105620] Updated weights for policy 1, policy_version 483599 (0.0010) [2023-12-26 18:52:57,967][105620] Updated weights for policy 1, policy_version 483609 (0.0010) [2023-12-26 18:52:58,021][105620] Updated weights for policy 1, policy_version 483619 (0.0010) [2023-12-26 18:52:58,364][105692] Updated weights for policy 0, policy_version 483235 (0.0006) [2023-12-26 18:52:58,431][105692] Updated weights for policy 0, policy_version 483245 (0.0007) [2023-12-26 18:52:58,503][105692] Updated weights for policy 0, policy_version 483255 (0.0008) [2023-12-26 18:52:58,938][105620] Updated weights for policy 1, policy_version 483629 (0.0009) [2023-12-26 18:52:59,000][105620] Updated weights for policy 1, policy_version 483639 (0.0009) [2023-12-26 18:52:59,065][105620] Updated weights for policy 1, policy_version 483649 (0.0008) [2023-12-26 18:52:59,234][105692] Updated weights for policy 0, policy_version 483265 (0.0006) [2023-12-26 18:52:59,301][105692] Updated weights for policy 0, policy_version 483275 (0.0008) [2023-12-26 18:52:59,368][105692] Updated weights for policy 0, policy_version 483285 (0.0009) [2023-12-26 18:52:59,424][105692] Updated weights for policy 0, policy_version 483295 (0.0009) [2023-12-26 18:52:59,723][105620] Updated weights for policy 1, policy_version 483659 (0.0008) [2023-12-26 18:52:59,772][105620] Updated weights for policy 1, policy_version 483669 (0.0010) [2023-12-26 18:52:59,835][105620] Updated weights for policy 1, policy_version 483679 (0.0011) [2023-12-26 18:53:00,177][105692] Updated weights for policy 0, policy_version 483305 (0.0008) [2023-12-26 18:53:00,238][105692] Updated weights for policy 0, policy_version 483315 (0.0010) [2023-12-26 18:53:00,302][105692] Updated weights for policy 0, policy_version 483325 (0.0010) [2023-12-26 18:53:00,533][105620] Updated weights for policy 1, policy_version 483689 (0.0010) [2023-12-26 18:53:00,584][105620] Updated weights for policy 1, policy_version 483699 (0.0005) [2023-12-26 18:53:00,644][105620] Updated weights for policy 1, policy_version 483709 (0.0009) [2023-12-26 18:53:00,688][105620] Updated weights for policy 1, policy_version 483719 (0.0010) [2023-12-26 18:53:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 247595008. Throughput: 0: 9997.1, 1: 9668.8. Samples: 247568204. Policy #0 lag: (min: 11.0, avg: 20.6, max: 43.0) [2023-12-26 18:53:01,062][105692] Updated weights for policy 0, policy_version 483335 (0.0009) [2023-12-26 18:53:01,062][104569] Avg episode reward: [(0, '8910.455'), (1, '9265.124')] [2023-12-26 18:53:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000483720_123846656.pth... [2023-12-26 18:53:01,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000482568_123551744.pth [2023-12-26 18:53:01,123][105692] Updated weights for policy 0, policy_version 483345 (0.0008) [2023-12-26 18:53:01,181][105692] Updated weights for policy 0, policy_version 483355 (0.0008) [2023-12-26 18:53:01,206][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000483360_123756544.pth... [2023-12-26 18:53:01,210][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000482208_123461632.pth [2023-12-26 18:53:01,419][105620] Updated weights for policy 1, policy_version 483729 (0.0009) [2023-12-26 18:53:01,485][105620] Updated weights for policy 1, policy_version 483739 (0.0009) [2023-12-26 18:53:01,542][105620] Updated weights for policy 1, policy_version 483749 (0.0009) [2023-12-26 18:53:01,889][105692] Updated weights for policy 0, policy_version 483365 (0.0008) [2023-12-26 18:53:01,940][105692] Updated weights for policy 0, policy_version 483375 (0.0009) [2023-12-26 18:53:01,997][105692] Updated weights for policy 0, policy_version 483385 (0.0009) [2023-12-26 18:53:02,354][105620] Updated weights for policy 1, policy_version 483759 (0.0009) [2023-12-26 18:53:02,418][105620] Updated weights for policy 1, policy_version 483769 (0.0009) [2023-12-26 18:53:02,479][105620] Updated weights for policy 1, policy_version 483779 (0.0009) [2023-12-26 18:53:02,782][105692] Updated weights for policy 0, policy_version 483395 (0.0008) [2023-12-26 18:53:02,838][105692] Updated weights for policy 0, policy_version 483405 (0.0009) [2023-12-26 18:53:02,902][105692] Updated weights for policy 0, policy_version 483415 (0.0009) [2023-12-26 18:53:03,136][105620] Updated weights for policy 1, policy_version 483789 (0.0008) [2023-12-26 18:53:03,186][105620] Updated weights for policy 1, policy_version 483799 (0.0009) [2023-12-26 18:53:03,251][105620] Updated weights for policy 1, policy_version 483809 (0.0009) [2023-12-26 18:53:03,657][105692] Updated weights for policy 0, policy_version 483425 (0.0009) [2023-12-26 18:53:03,717][105692] Updated weights for policy 0, policy_version 483435 (0.0009) [2023-12-26 18:53:03,774][105692] Updated weights for policy 0, policy_version 483445 (0.0009) [2023-12-26 18:53:03,834][105692] Updated weights for policy 0, policy_version 483455 (0.0009) [2023-12-26 18:53:03,983][105620] Updated weights for policy 1, policy_version 483819 (0.0009) [2023-12-26 18:53:04,034][105620] Updated weights for policy 1, policy_version 483829 (0.0009) [2023-12-26 18:53:04,082][105620] Updated weights for policy 1, policy_version 483839 (0.0009) [2023-12-26 18:53:04,553][105692] Updated weights for policy 0, policy_version 483465 (0.0009) [2023-12-26 18:53:04,612][105692] Updated weights for policy 0, policy_version 483475 (0.0009) [2023-12-26 18:53:04,674][105692] Updated weights for policy 0, policy_version 483485 (0.0007) [2023-12-26 18:53:04,885][105620] Updated weights for policy 1, policy_version 483849 (0.0010) [2023-12-26 18:53:04,940][105620] Updated weights for policy 1, policy_version 483859 (0.0006) [2023-12-26 18:53:04,986][105620] Updated weights for policy 1, policy_version 483869 (0.0005) [2023-12-26 18:53:05,038][105620] Updated weights for policy 1, policy_version 483879 (0.0006) [2023-12-26 18:53:05,339][105692] Updated weights for policy 0, policy_version 483495 (0.0008) [2023-12-26 18:53:05,399][105692] Updated weights for policy 0, policy_version 483505 (0.0008) [2023-12-26 18:53:05,457][105692] Updated weights for policy 0, policy_version 483515 (0.0008) [2023-12-26 18:53:05,687][105620] Updated weights for policy 1, policy_version 483889 (0.0009) [2023-12-26 18:53:05,732][105620] Updated weights for policy 1, policy_version 483899 (0.0010) [2023-12-26 18:53:05,776][105620] Updated weights for policy 1, policy_version 483909 (0.0010) [2023-12-26 18:53:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 247693312. Throughput: 0: 9934.5, 1: 9616.1. Samples: 247681748. Policy #0 lag: (min: 11.0, avg: 20.6, max: 43.0) [2023-12-26 18:53:06,062][104569] Avg episode reward: [(0, '8816.310'), (1, '9354.922')] [2023-12-26 18:53:06,251][105692] Updated weights for policy 0, policy_version 483525 (0.0008) [2023-12-26 18:53:06,309][105692] Updated weights for policy 0, policy_version 483535 (0.0010) [2023-12-26 18:53:06,372][105692] Updated weights for policy 0, policy_version 483545 (0.0010) [2023-12-26 18:53:06,480][105620] Updated weights for policy 1, policy_version 483919 (0.0007) [2023-12-26 18:53:06,544][105620] Updated weights for policy 1, policy_version 483929 (0.0006) [2023-12-26 18:53:06,603][105620] Updated weights for policy 1, policy_version 483939 (0.0005) [2023-12-26 18:53:07,222][105620] Updated weights for policy 1, policy_version 483949 (0.0007) [2023-12-26 18:53:07,233][105692] Updated weights for policy 0, policy_version 483555 (0.0009) [2023-12-26 18:53:07,279][105620] Updated weights for policy 1, policy_version 483959 (0.0008) [2023-12-26 18:53:07,286][105692] Updated weights for policy 0, policy_version 483565 (0.0006) [2023-12-26 18:53:07,333][105692] Updated weights for policy 0, policy_version 483575 (0.0007) [2023-12-26 18:53:07,339][105620] Updated weights for policy 1, policy_version 483969 (0.0008) [2023-12-26 18:53:07,993][105620] Updated weights for policy 1, policy_version 483979 (0.0009) [2023-12-26 18:53:08,044][105620] Updated weights for policy 1, policy_version 483989 (0.0007) [2023-12-26 18:53:08,097][105620] Updated weights for policy 1, policy_version 483999 (0.0010) [2023-12-26 18:53:08,148][105692] Updated weights for policy 0, policy_version 483585 (0.0007) [2023-12-26 18:53:08,203][105692] Updated weights for policy 0, policy_version 483595 (0.0009) [2023-12-26 18:53:08,261][105692] Updated weights for policy 0, policy_version 483605 (0.0008) [2023-12-26 18:53:08,312][105692] Updated weights for policy 0, policy_version 483616 (0.0009) [2023-12-26 18:53:08,802][105620] Updated weights for policy 1, policy_version 484009 (0.0011) [2023-12-26 18:53:08,858][105620] Updated weights for policy 1, policy_version 484019 (0.0008) [2023-12-26 18:53:08,910][105620] Updated weights for policy 1, policy_version 484029 (0.0008) [2023-12-26 18:53:08,960][105620] Updated weights for policy 1, policy_version 484039 (0.0008) [2023-12-26 18:53:09,060][105692] Updated weights for policy 0, policy_version 483626 (0.0006) [2023-12-26 18:53:09,113][105692] Updated weights for policy 0, policy_version 483636 (0.0008) [2023-12-26 18:53:09,165][105692] Updated weights for policy 0, policy_version 483646 (0.0009) [2023-12-26 18:53:09,785][105620] Updated weights for policy 1, policy_version 484049 (0.0008) [2023-12-26 18:53:09,857][105620] Updated weights for policy 1, policy_version 484059 (0.0009) [2023-12-26 18:53:09,904][105692] Updated weights for policy 0, policy_version 483656 (0.0007) [2023-12-26 18:53:09,915][105620] Updated weights for policy 1, policy_version 484069 (0.0008) [2023-12-26 18:53:09,969][105692] Updated weights for policy 0, policy_version 483666 (0.0009) [2023-12-26 18:53:10,028][105692] Updated weights for policy 0, policy_version 483676 (0.0010) [2023-12-26 18:53:10,656][105620] Updated weights for policy 1, policy_version 484079 (0.0009) [2023-12-26 18:53:10,707][105620] Updated weights for policy 1, policy_version 484089 (0.0009) [2023-12-26 18:53:10,760][105620] Updated weights for policy 1, policy_version 484099 (0.0010) [2023-12-26 18:53:10,764][105692] Updated weights for policy 0, policy_version 483686 (0.0009) [2023-12-26 18:53:10,825][105692] Updated weights for policy 0, policy_version 483696 (0.0007) [2023-12-26 18:53:10,881][105692] Updated weights for policy 0, policy_version 483706 (0.0008) [2023-12-26 18:53:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 247791616. Throughput: 0: 9803.0, 1: 9739.9. Samples: 247796188. Policy #0 lag: (min: 11.0, avg: 20.6, max: 43.0) [2023-12-26 18:53:11,062][104569] Avg episode reward: [(0, '8727.639'), (1, '9354.719')] [2023-12-26 18:53:11,586][105620] Updated weights for policy 1, policy_version 484109 (0.0011) [2023-12-26 18:53:11,614][105692] Updated weights for policy 0, policy_version 483716 (0.0007) [2023-12-26 18:53:11,649][105620] Updated weights for policy 1, policy_version 484119 (0.0009) [2023-12-26 18:53:11,675][105692] Updated weights for policy 0, policy_version 483726 (0.0008) [2023-12-26 18:53:11,700][105620] Updated weights for policy 1, policy_version 484129 (0.0006) [2023-12-26 18:53:11,744][105692] Updated weights for policy 0, policy_version 483736 (0.0008) [2023-12-26 18:53:12,417][105620] Updated weights for policy 1, policy_version 484139 (0.0010) [2023-12-26 18:53:12,481][105620] Updated weights for policy 1, policy_version 484149 (0.0011) [2023-12-26 18:53:12,535][105692] Updated weights for policy 0, policy_version 483746 (0.0008) [2023-12-26 18:53:12,541][105620] Updated weights for policy 1, policy_version 484159 (0.0011) [2023-12-26 18:53:12,590][105692] Updated weights for policy 0, policy_version 483756 (0.0007) [2023-12-26 18:53:12,657][105692] Updated weights for policy 0, policy_version 483766 (0.0010) [2023-12-26 18:53:12,708][105692] Updated weights for policy 0, policy_version 483776 (0.0009) [2023-12-26 18:53:13,205][105620] Updated weights for policy 1, policy_version 484169 (0.0009) [2023-12-26 18:53:13,257][105620] Updated weights for policy 1, policy_version 484179 (0.0010) [2023-12-26 18:53:13,308][105620] Updated weights for policy 1, policy_version 484189 (0.0010) [2023-12-26 18:53:13,357][105620] Updated weights for policy 1, policy_version 484199 (0.0010) [2023-12-26 18:53:13,414][105692] Updated weights for policy 0, policy_version 483786 (0.0005) [2023-12-26 18:53:13,467][105692] Updated weights for policy 0, policy_version 483796 (0.0005) [2023-12-26 18:53:13,523][105692] Updated weights for policy 0, policy_version 483806 (0.0007) [2023-12-26 18:53:14,112][105620] Updated weights for policy 1, policy_version 484209 (0.0010) [2023-12-26 18:53:14,141][105692] Updated weights for policy 0, policy_version 483816 (0.0007) [2023-12-26 18:53:14,170][105620] Updated weights for policy 1, policy_version 484219 (0.0010) [2023-12-26 18:53:14,192][105692] Updated weights for policy 0, policy_version 483826 (0.0006) [2023-12-26 18:53:14,225][105620] Updated weights for policy 1, policy_version 484229 (0.0010) [2023-12-26 18:53:14,248][105692] Updated weights for policy 0, policy_version 483836 (0.0007) [2023-12-26 18:53:14,966][105620] Updated weights for policy 1, policy_version 484239 (0.0009) [2023-12-26 18:53:14,978][105692] Updated weights for policy 0, policy_version 483846 (0.0008) [2023-12-26 18:53:15,024][105620] Updated weights for policy 1, policy_version 484249 (0.0007) [2023-12-26 18:53:15,036][105692] Updated weights for policy 0, policy_version 483856 (0.0010) [2023-12-26 18:53:15,079][105620] Updated weights for policy 1, policy_version 484259 (0.0006) [2023-12-26 18:53:15,100][105692] Updated weights for policy 0, policy_version 483866 (0.0008) [2023-12-26 18:53:15,741][105620] Updated weights for policy 1, policy_version 484269 (0.0009) [2023-12-26 18:53:15,792][105620] Updated weights for policy 1, policy_version 484279 (0.0010) [2023-12-26 18:53:15,801][105692] Updated weights for policy 0, policy_version 483876 (0.0008) [2023-12-26 18:53:15,843][105620] Updated weights for policy 1, policy_version 484289 (0.0010) [2023-12-26 18:53:15,854][105692] Updated weights for policy 0, policy_version 483886 (0.0006) [2023-12-26 18:53:15,911][105692] Updated weights for policy 0, policy_version 483896 (0.0007) [2023-12-26 18:53:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 247889920. Throughput: 0: 9728.3, 1: 9774.9. Samples: 247853152. Policy #0 lag: (min: 11.0, avg: 20.6, max: 43.0) [2023-12-26 18:53:16,062][104569] Avg episode reward: [(0, '9179.315'), (1, '9265.483')] [2023-12-26 18:53:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000484296_123994112.pth... [2023-12-26 18:53:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000483904_123895808.pth... [2023-12-26 18:53:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000483144_123699200.pth [2023-12-26 18:53:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000482784_123609088.pth [2023-12-26 18:53:16,482][105692] Updated weights for policy 0, policy_version 483906 (0.0007) [2023-12-26 18:53:16,527][105692] Updated weights for policy 0, policy_version 483916 (0.0006) [2023-12-26 18:53:16,584][105692] Updated weights for policy 0, policy_version 483926 (0.0005) [2023-12-26 18:53:16,597][105620] Updated weights for policy 1, policy_version 484299 (0.0010) [2023-12-26 18:53:16,640][105692] Updated weights for policy 0, policy_version 483936 (0.0005) [2023-12-26 18:53:16,641][105620] Updated weights for policy 1, policy_version 484309 (0.0010) [2023-12-26 18:53:16,692][105620] Updated weights for policy 1, policy_version 484319 (0.0010) [2023-12-26 18:53:17,193][105692] Updated weights for policy 0, policy_version 483946 (0.0008) [2023-12-26 18:53:17,237][105692] Updated weights for policy 0, policy_version 483956 (0.0008) [2023-12-26 18:53:17,297][105692] Updated weights for policy 0, policy_version 483966 (0.0008) [2023-12-26 18:53:17,406][105620] Updated weights for policy 1, policy_version 484329 (0.0009) [2023-12-26 18:53:17,464][105620] Updated weights for policy 1, policy_version 484339 (0.0005) [2023-12-26 18:53:17,515][105620] Updated weights for policy 1, policy_version 484349 (0.0005) [2023-12-26 18:53:17,567][105620] Updated weights for policy 1, policy_version 484359 (0.0008) [2023-12-26 18:53:18,148][105692] Updated weights for policy 0, policy_version 483976 (0.0009) [2023-12-26 18:53:18,184][105620] Updated weights for policy 1, policy_version 484369 (0.0009) [2023-12-26 18:53:18,206][105692] Updated weights for policy 0, policy_version 483986 (0.0008) [2023-12-26 18:53:18,238][105620] Updated weights for policy 1, policy_version 484379 (0.0006) [2023-12-26 18:53:18,266][105692] Updated weights for policy 0, policy_version 483996 (0.0007) [2023-12-26 18:53:18,293][105620] Updated weights for policy 1, policy_version 484389 (0.0008) [2023-12-26 18:53:19,002][105692] Updated weights for policy 0, policy_version 484006 (0.0007) [2023-12-26 18:53:19,053][105692] Updated weights for policy 0, policy_version 484016 (0.0006) [2023-12-26 18:53:19,081][105620] Updated weights for policy 1, policy_version 484399 (0.0007) [2023-12-26 18:53:19,115][105692] Updated weights for policy 0, policy_version 484026 (0.0009) [2023-12-26 18:53:19,130][105620] Updated weights for policy 1, policy_version 484409 (0.0006) [2023-12-26 18:53:19,186][105620] Updated weights for policy 1, policy_version 484419 (0.0008) [2023-12-26 18:53:19,868][105692] Updated weights for policy 0, policy_version 484036 (0.0010) [2023-12-26 18:53:19,921][105692] Updated weights for policy 0, policy_version 484046 (0.0010) [2023-12-26 18:53:19,962][105620] Updated weights for policy 1, policy_version 484429 (0.0007) [2023-12-26 18:53:19,983][105692] Updated weights for policy 0, policy_version 484056 (0.0011) [2023-12-26 18:53:20,023][105620] Updated weights for policy 1, policy_version 484439 (0.0006) [2023-12-26 18:53:20,083][105620] Updated weights for policy 1, policy_version 484449 (0.0009) [2023-12-26 18:53:20,681][105692] Updated weights for policy 0, policy_version 484066 (0.0010) [2023-12-26 18:53:20,742][105692] Updated weights for policy 0, policy_version 484076 (0.0007) [2023-12-26 18:53:20,793][105692] Updated weights for policy 0, policy_version 484086 (0.0008) [2023-12-26 18:53:20,851][105692] Updated weights for policy 0, policy_version 484096 (0.0008) [2023-12-26 18:53:20,891][105620] Updated weights for policy 1, policy_version 484459 (0.0009) [2023-12-26 18:53:20,948][105620] Updated weights for policy 1, policy_version 484469 (0.0008) [2023-12-26 18:53:21,008][105620] Updated weights for policy 1, policy_version 484479 (0.0008) [2023-12-26 18:53:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 247980032. Throughput: 0: 9651.4, 1: 9877.9. Samples: 247972832. Policy #0 lag: (min: 11.0, avg: 20.6, max: 43.0) [2023-12-26 18:53:21,063][104569] Avg episode reward: [(0, '9269.108'), (1, '9175.024')] [2023-12-26 18:53:21,542][105692] Updated weights for policy 0, policy_version 484106 (0.0006) [2023-12-26 18:53:21,596][105692] Updated weights for policy 0, policy_version 484116 (0.0006) [2023-12-26 18:53:21,665][105692] Updated weights for policy 0, policy_version 484127 (0.0009) [2023-12-26 18:53:21,877][105620] Updated weights for policy 1, policy_version 484489 (0.0009) [2023-12-26 18:53:21,934][105620] Updated weights for policy 1, policy_version 484499 (0.0009) [2023-12-26 18:53:21,998][105620] Updated weights for policy 1, policy_version 484509 (0.0008) [2023-12-26 18:53:22,065][105620] Updated weights for policy 1, policy_version 484519 (0.0008) [2023-12-26 18:53:22,364][105692] Updated weights for policy 0, policy_version 484137 (0.0010) [2023-12-26 18:53:22,428][105692] Updated weights for policy 0, policy_version 484147 (0.0011) [2023-12-26 18:53:22,484][105692] Updated weights for policy 0, policy_version 484157 (0.0010) [2023-12-26 18:53:22,848][105620] Updated weights for policy 1, policy_version 484529 (0.0008) [2023-12-26 18:53:22,918][105620] Updated weights for policy 1, policy_version 484539 (0.0009) [2023-12-26 18:53:22,983][105620] Updated weights for policy 1, policy_version 484549 (0.0008) [2023-12-26 18:53:23,249][105692] Updated weights for policy 0, policy_version 484167 (0.0011) [2023-12-26 18:53:23,300][105692] Updated weights for policy 0, policy_version 484177 (0.0010) [2023-12-26 18:53:23,351][105692] Updated weights for policy 0, policy_version 484187 (0.0010) [2023-12-26 18:53:23,738][105620] Updated weights for policy 1, policy_version 484559 (0.0008) [2023-12-26 18:53:23,786][105620] Updated weights for policy 1, policy_version 484569 (0.0008) [2023-12-26 18:53:23,833][105620] Updated weights for policy 1, policy_version 484579 (0.0008) [2023-12-26 18:53:24,089][105692] Updated weights for policy 0, policy_version 484197 (0.0010) [2023-12-26 18:53:24,145][105692] Updated weights for policy 0, policy_version 484207 (0.0010) [2023-12-26 18:53:24,202][105692] Updated weights for policy 0, policy_version 484217 (0.0010) [2023-12-26 18:53:24,604][105620] Updated weights for policy 1, policy_version 484589 (0.0010) [2023-12-26 18:53:24,652][105620] Updated weights for policy 1, policy_version 484599 (0.0010) [2023-12-26 18:53:24,707][105620] Updated weights for policy 1, policy_version 484609 (0.0010) [2023-12-26 18:53:24,929][105692] Updated weights for policy 0, policy_version 484227 (0.0010) [2023-12-26 18:53:24,977][105692] Updated weights for policy 0, policy_version 484237 (0.0008) [2023-12-26 18:53:25,026][105692] Updated weights for policy 0, policy_version 484247 (0.0008) [2023-12-26 18:53:25,460][105620] Updated weights for policy 1, policy_version 484619 (0.0010) [2023-12-26 18:53:25,505][105620] Updated weights for policy 1, policy_version 484629 (0.0010) [2023-12-26 18:53:25,556][105620] Updated weights for policy 1, policy_version 484639 (0.0010) [2023-12-26 18:53:25,627][105692] Updated weights for policy 0, policy_version 484257 (0.0008) [2023-12-26 18:53:25,690][105692] Updated weights for policy 0, policy_version 484267 (0.0008) [2023-12-26 18:53:25,744][105692] Updated weights for policy 0, policy_version 484277 (0.0008) [2023-12-26 18:53:25,793][105692] Updated weights for policy 0, policy_version 484287 (0.0008) [2023-12-26 18:53:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 248078336. Throughput: 0: 9623.2, 1: 9721.3. Samples: 248085432. Policy #0 lag: (min: 11.0, avg: 20.6, max: 43.0) [2023-12-26 18:53:26,062][104569] Avg episode reward: [(0, '9269.242'), (1, '9265.416')] [2023-12-26 18:53:26,328][105620] Updated weights for policy 1, policy_version 484649 (0.0010) [2023-12-26 18:53:26,376][105620] Updated weights for policy 1, policy_version 484659 (0.0010) [2023-12-26 18:53:26,438][105620] Updated weights for policy 1, policy_version 484669 (0.0010) [2023-12-26 18:53:26,483][105620] Updated weights for policy 1, policy_version 484679 (0.0010) [2023-12-26 18:53:26,543][105692] Updated weights for policy 0, policy_version 484297 (0.0008) [2023-12-26 18:53:26,607][105692] Updated weights for policy 0, policy_version 484307 (0.0008) [2023-12-26 18:53:26,669][105692] Updated weights for policy 0, policy_version 484317 (0.0008) [2023-12-26 18:53:27,182][105620] Updated weights for policy 1, policy_version 484689 (0.0008) [2023-12-26 18:53:27,239][105620] Updated weights for policy 1, policy_version 484699 (0.0010) [2023-12-26 18:53:27,297][105620] Updated weights for policy 1, policy_version 484709 (0.0010) [2023-12-26 18:53:27,420][105692] Updated weights for policy 0, policy_version 484327 (0.0006) [2023-12-26 18:53:27,480][105692] Updated weights for policy 0, policy_version 484337 (0.0008) [2023-12-26 18:53:27,533][105692] Updated weights for policy 0, policy_version 484347 (0.0009) [2023-12-26 18:53:27,981][105620] Updated weights for policy 1, policy_version 484719 (0.0008) [2023-12-26 18:53:28,034][105620] Updated weights for policy 1, policy_version 484729 (0.0005) [2023-12-26 18:53:28,089][105620] Updated weights for policy 1, policy_version 484739 (0.0006) [2023-12-26 18:53:28,261][105692] Updated weights for policy 0, policy_version 484357 (0.0009) [2023-12-26 18:53:28,329][105692] Updated weights for policy 0, policy_version 484367 (0.0010) [2023-12-26 18:53:28,393][105692] Updated weights for policy 0, policy_version 484377 (0.0009) [2023-12-26 18:53:28,732][105620] Updated weights for policy 1, policy_version 484749 (0.0010) [2023-12-26 18:53:28,791][105620] Updated weights for policy 1, policy_version 484759 (0.0011) [2023-12-26 18:53:28,840][105620] Updated weights for policy 1, policy_version 484769 (0.0010) [2023-12-26 18:53:29,144][105692] Updated weights for policy 0, policy_version 484387 (0.0008) [2023-12-26 18:53:29,189][105585] KL-divergence is very high: 496.7400 [2023-12-26 18:53:29,202][105692] Updated weights for policy 0, policy_version 484397 (0.0007) [2023-12-26 18:53:29,241][105585] KL-divergence is very high: 863.9697 [2023-12-26 18:53:29,263][105692] Updated weights for policy 0, policy_version 484407 (0.0007) [2023-12-26 18:53:29,282][105585] KL-divergence is very high: 998.7864 [2023-12-26 18:53:29,542][105620] Updated weights for policy 1, policy_version 484779 (0.0010) [2023-12-26 18:53:29,607][105620] Updated weights for policy 1, policy_version 484789 (0.0007) [2023-12-26 18:53:29,671][105620] Updated weights for policy 1, policy_version 484799 (0.0005) [2023-12-26 18:53:29,987][105692] Updated weights for policy 0, policy_version 484417 (0.0006) [2023-12-26 18:53:30,046][105692] Updated weights for policy 0, policy_version 484427 (0.0008) [2023-12-26 18:53:30,109][105692] Updated weights for policy 0, policy_version 484437 (0.0008) [2023-12-26 18:53:30,168][105692] Updated weights for policy 0, policy_version 484447 (0.0008) [2023-12-26 18:53:30,294][105620] Updated weights for policy 1, policy_version 484809 (0.0006) [2023-12-26 18:53:30,343][105620] Updated weights for policy 1, policy_version 484819 (0.0011) [2023-12-26 18:53:30,409][105620] Updated weights for policy 1, policy_version 484829 (0.0011) [2023-12-26 18:53:30,464][105620] Updated weights for policy 1, policy_version 484839 (0.0010) [2023-12-26 18:53:30,936][105692] Updated weights for policy 0, policy_version 484457 (0.0009) [2023-12-26 18:53:30,999][105692] Updated weights for policy 0, policy_version 484467 (0.0008) [2023-12-26 18:53:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 248168448. Throughput: 0: 9642.8, 1: 9769.8. Samples: 248143608. Policy #0 lag: (min: 11.0, avg: 20.6, max: 43.0) [2023-12-26 18:53:31,063][104569] Avg episode reward: [(0, '9179.213'), (1, '9355.125')] [2023-12-26 18:53:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000484840_124133376.pth... [2023-12-26 18:53:31,069][105692] Updated weights for policy 0, policy_version 484477 (0.0008) [2023-12-26 18:53:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000483720_123846656.pth [2023-12-26 18:53:31,088][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000484480_124043264.pth... [2023-12-26 18:53:31,092][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000483360_123756544.pth [2023-12-26 18:53:31,197][105620] Updated weights for policy 1, policy_version 484849 (0.0009) [2023-12-26 18:53:31,246][105620] Updated weights for policy 1, policy_version 484859 (0.0009) [2023-12-26 18:53:31,302][105620] Updated weights for policy 1, policy_version 484869 (0.0009) [2023-12-26 18:53:31,827][105692] Updated weights for policy 0, policy_version 484487 (0.0010) [2023-12-26 18:53:31,877][105692] Updated weights for policy 0, policy_version 484497 (0.0009) [2023-12-26 18:53:31,927][105692] Updated weights for policy 0, policy_version 484507 (0.0009) [2023-12-26 18:53:32,086][105620] Updated weights for policy 1, policy_version 484879 (0.0008) [2023-12-26 18:53:32,140][105620] Updated weights for policy 1, policy_version 484889 (0.0009) [2023-12-26 18:53:32,196][105620] Updated weights for policy 1, policy_version 484899 (0.0008) [2023-12-26 18:53:32,708][105692] Updated weights for policy 0, policy_version 484517 (0.0009) [2023-12-26 18:53:32,759][105692] Updated weights for policy 0, policy_version 484527 (0.0009) [2023-12-26 18:53:32,819][105692] Updated weights for policy 0, policy_version 484537 (0.0010) [2023-12-26 18:53:32,929][105620] Updated weights for policy 1, policy_version 484909 (0.0009) [2023-12-26 18:53:32,986][105620] Updated weights for policy 1, policy_version 484919 (0.0009) [2023-12-26 18:53:33,051][105620] Updated weights for policy 1, policy_version 484929 (0.0010) [2023-12-26 18:53:33,423][105692] Updated weights for policy 0, policy_version 484547 (0.0009) [2023-12-26 18:53:33,478][105692] Updated weights for policy 0, policy_version 484557 (0.0011) [2023-12-26 18:53:33,537][105692] Updated weights for policy 0, policy_version 484567 (0.0010) [2023-12-26 18:53:33,890][105620] Updated weights for policy 1, policy_version 484939 (0.0009) [2023-12-26 18:53:33,948][105620] Updated weights for policy 1, policy_version 484949 (0.0008) [2023-12-26 18:53:34,002][105620] Updated weights for policy 1, policy_version 484959 (0.0009) [2023-12-26 18:53:34,240][105692] Updated weights for policy 0, policy_version 484577 (0.0009) [2023-12-26 18:53:34,301][105585] KL-divergence is very high: 420.0573 [2023-12-26 18:53:34,309][105692] Updated weights for policy 0, policy_version 484587 (0.0011) [2023-12-26 18:53:34,349][105585] KL-divergence is very high: 759.9199 [2023-12-26 18:53:34,372][105692] Updated weights for policy 0, policy_version 484597 (0.0010) [2023-12-26 18:53:34,403][105585] KL-divergence is very high: 817.4716 [2023-12-26 18:53:34,431][105692] Updated weights for policy 0, policy_version 484607 (0.0011) [2023-12-26 18:53:34,754][105620] Updated weights for policy 1, policy_version 484969 (0.0008) [2023-12-26 18:53:34,803][105620] Updated weights for policy 1, policy_version 484979 (0.0008) [2023-12-26 18:53:34,849][105620] Updated weights for policy 1, policy_version 484989 (0.0008) [2023-12-26 18:53:34,897][105620] Updated weights for policy 1, policy_version 484999 (0.0008) [2023-12-26 18:53:35,131][105585] KL-divergence is very high: 348.2106 [2023-12-26 18:53:35,147][105585] KL-divergence is very high: 220.5136 [2023-12-26 18:53:35,160][105692] Updated weights for policy 0, policy_version 484617 (0.0010) [2023-12-26 18:53:35,171][105585] KL-divergence is very high: 253.5285 [2023-12-26 18:53:35,185][105585] KL-divergence is very high: 179.1043 [2023-12-26 18:53:35,209][105692] Updated weights for policy 0, policy_version 484627 (0.0009) [2023-12-26 18:53:35,210][105585] KL-divergence is very high: 166.2814 [2023-12-26 18:53:35,225][105585] KL-divergence is very high: 123.8993 [2023-12-26 18:53:35,252][105585] KL-divergence is very high: 138.3617 [2023-12-26 18:53:35,261][105692] Updated weights for policy 0, policy_version 484637 (0.0007) [2023-12-26 18:53:35,607][105620] Updated weights for policy 1, policy_version 485009 (0.0007) [2023-12-26 18:53:35,670][105620] Updated weights for policy 1, policy_version 485019 (0.0008) [2023-12-26 18:53:35,726][105620] Updated weights for policy 1, policy_version 485029 (0.0005) [2023-12-26 18:53:35,999][105692] Updated weights for policy 0, policy_version 484647 (0.0008) [2023-12-26 18:53:36,060][105692] Updated weights for policy 0, policy_version 484657 (0.0005) [2023-12-26 18:53:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.3, 300 sec: 19522.0). Total num frames: 248266752. Throughput: 0: 9575.4, 1: 9656.6. Samples: 248258036. Policy #0 lag: (min: 11.0, avg: 20.6, max: 43.0) [2023-12-26 18:53:36,062][104569] Avg episode reward: [(0, '8362.801'), (1, '9354.210')] [2023-12-26 18:53:36,125][105692] Updated weights for policy 0, policy_version 484667 (0.0007) [2023-12-26 18:53:36,282][105620] Updated weights for policy 1, policy_version 485039 (0.0007) [2023-12-26 18:53:36,338][105620] Updated weights for policy 1, policy_version 485049 (0.0008) [2023-12-26 18:53:36,398][105620] Updated weights for policy 1, policy_version 485059 (0.0008) [2023-12-26 18:53:36,751][105692] Updated weights for policy 0, policy_version 484677 (0.0009) [2023-12-26 18:53:36,811][105692] Updated weights for policy 0, policy_version 484687 (0.0010) [2023-12-26 18:53:36,880][105692] Updated weights for policy 0, policy_version 484697 (0.0010) [2023-12-26 18:53:37,220][105620] Updated weights for policy 1, policy_version 485069 (0.0009) [2023-12-26 18:53:37,286][105620] Updated weights for policy 1, policy_version 485079 (0.0010) [2023-12-26 18:53:37,360][105620] Updated weights for policy 1, policy_version 485089 (0.0009) [2023-12-26 18:53:37,517][105692] Updated weights for policy 0, policy_version 484707 (0.0010) [2023-12-26 18:53:37,573][105692] Updated weights for policy 0, policy_version 484717 (0.0009) [2023-12-26 18:53:37,628][105692] Updated weights for policy 0, policy_version 484727 (0.0006) [2023-12-26 18:53:38,196][105620] Updated weights for policy 1, policy_version 485099 (0.0010) [2023-12-26 18:53:38,198][105692] Updated weights for policy 0, policy_version 484737 (0.0005) [2023-12-26 18:53:38,247][105692] Updated weights for policy 0, policy_version 484747 (0.0006) [2023-12-26 18:53:38,249][105620] Updated weights for policy 1, policy_version 485109 (0.0006) [2023-12-26 18:53:38,292][105692] Updated weights for policy 0, policy_version 484757 (0.0006) [2023-12-26 18:53:38,302][105620] Updated weights for policy 1, policy_version 485119 (0.0007) [2023-12-26 18:53:38,347][105692] Updated weights for policy 0, policy_version 484767 (0.0007) [2023-12-26 18:53:39,058][105620] Updated weights for policy 1, policy_version 485129 (0.0007) [2023-12-26 18:53:39,115][105620] Updated weights for policy 1, policy_version 485139 (0.0005) [2023-12-26 18:53:39,176][105620] Updated weights for policy 1, policy_version 485149 (0.0006) [2023-12-26 18:53:39,183][105692] Updated weights for policy 0, policy_version 484777 (0.0009) [2023-12-26 18:53:39,235][105620] Updated weights for policy 1, policy_version 485159 (0.0009) [2023-12-26 18:53:39,250][105692] Updated weights for policy 0, policy_version 484787 (0.0007) [2023-12-26 18:53:39,318][105692] Updated weights for policy 0, policy_version 484797 (0.0008) [2023-12-26 18:53:39,338][105585] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000005 [2023-12-26 18:53:39,929][105620] Updated weights for policy 1, policy_version 485169 (0.0007) [2023-12-26 18:53:39,988][105620] Updated weights for policy 1, policy_version 485179 (0.0008) [2023-12-26 18:53:40,040][105692] Updated weights for policy 0, policy_version 484807 (0.0008) [2023-12-26 18:53:40,054][105620] Updated weights for policy 1, policy_version 485189 (0.0006) [2023-12-26 18:53:40,099][105692] Updated weights for policy 0, policy_version 484817 (0.0009) [2023-12-26 18:53:40,160][105692] Updated weights for policy 0, policy_version 484827 (0.0011) [2023-12-26 18:53:40,694][105620] Updated weights for policy 1, policy_version 485199 (0.0009) [2023-12-26 18:53:40,752][105620] Updated weights for policy 1, policy_version 485209 (0.0010) [2023-12-26 18:53:40,813][105620] Updated weights for policy 1, policy_version 485219 (0.0006) [2023-12-26 18:53:40,845][105692] Updated weights for policy 0, policy_version 484837 (0.0008) [2023-12-26 18:53:40,899][105692] Updated weights for policy 0, policy_version 484847 (0.0009) [2023-12-26 18:53:40,953][105692] Updated weights for policy 0, policy_version 484857 (0.0010) [2023-12-26 18:53:41,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 248373248. Throughput: 0: 9687.9, 1: 9566.3. Samples: 248375576. Policy #0 lag: (min: 11.0, avg: 20.6, max: 43.0) [2023-12-26 18:53:41,062][104569] Avg episode reward: [(0, '8632.378'), (1, '9354.355')] [2023-12-26 18:53:41,509][105620] Updated weights for policy 1, policy_version 485229 (0.0006) [2023-12-26 18:53:41,559][105620] Updated weights for policy 1, policy_version 485239 (0.0008) [2023-12-26 18:53:41,615][105620] Updated weights for policy 1, policy_version 485249 (0.0009) [2023-12-26 18:53:41,768][105692] Updated weights for policy 0, policy_version 484867 (0.0010) [2023-12-26 18:53:41,822][105692] Updated weights for policy 0, policy_version 484877 (0.0009) [2023-12-26 18:53:41,877][105692] Updated weights for policy 0, policy_version 484887 (0.0010) [2023-12-26 18:53:42,358][105620] Updated weights for policy 1, policy_version 485259 (0.0009) [2023-12-26 18:53:42,419][105620] Updated weights for policy 1, policy_version 485269 (0.0009) [2023-12-26 18:53:42,474][105620] Updated weights for policy 1, policy_version 485279 (0.0009) [2023-12-26 18:53:42,639][105692] Updated weights for policy 0, policy_version 484897 (0.0009) [2023-12-26 18:53:42,696][105692] Updated weights for policy 0, policy_version 484907 (0.0009) [2023-12-26 18:53:42,762][105692] Updated weights for policy 0, policy_version 484917 (0.0009) [2023-12-26 18:53:42,818][105692] Updated weights for policy 0, policy_version 484927 (0.0009) [2023-12-26 18:53:43,215][105620] Updated weights for policy 1, policy_version 485289 (0.0008) [2023-12-26 18:53:43,275][105620] Updated weights for policy 1, policy_version 485300 (0.0010) [2023-12-26 18:53:43,333][105620] Updated weights for policy 1, policy_version 485310 (0.0010) [2023-12-26 18:53:43,385][105620] Updated weights for policy 1, policy_version 485320 (0.0010) [2023-12-26 18:53:43,493][105692] Updated weights for policy 0, policy_version 484937 (0.0005) [2023-12-26 18:53:43,541][105692] Updated weights for policy 0, policy_version 484947 (0.0007) [2023-12-26 18:53:43,597][105692] Updated weights for policy 0, policy_version 484957 (0.0010) [2023-12-26 18:53:44,068][105620] Updated weights for policy 1, policy_version 485330 (0.0008) [2023-12-26 18:53:44,128][105620] Updated weights for policy 1, policy_version 485340 (0.0008) [2023-12-26 18:53:44,191][105620] Updated weights for policy 1, policy_version 485350 (0.0008) [2023-12-26 18:53:44,319][105692] Updated weights for policy 0, policy_version 484967 (0.0010) [2023-12-26 18:53:44,371][105692] Updated weights for policy 0, policy_version 484977 (0.0010) [2023-12-26 18:53:44,424][105692] Updated weights for policy 0, policy_version 484987 (0.0010) [2023-12-26 18:53:44,983][105620] Updated weights for policy 1, policy_version 485360 (0.0008) [2023-12-26 18:53:45,046][105620] Updated weights for policy 1, policy_version 485370 (0.0009) [2023-12-26 18:53:45,109][105620] Updated weights for policy 1, policy_version 485380 (0.0009) [2023-12-26 18:53:45,149][105692] Updated weights for policy 0, policy_version 484997 (0.0009) [2023-12-26 18:53:45,212][105692] Updated weights for policy 0, policy_version 485007 (0.0009) [2023-12-26 18:53:45,272][105692] Updated weights for policy 0, policy_version 485017 (0.0008) [2023-12-26 18:53:45,861][105620] Updated weights for policy 1, policy_version 485390 (0.0008) [2023-12-26 18:53:45,916][105620] Updated weights for policy 1, policy_version 485400 (0.0010) [2023-12-26 18:53:45,956][105692] Updated weights for policy 0, policy_version 485027 (0.0009) [2023-12-26 18:53:45,971][105620] Updated weights for policy 1, policy_version 485410 (0.0008) [2023-12-26 18:53:46,005][105692] Updated weights for policy 0, policy_version 485037 (0.0007) [2023-12-26 18:53:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 248463360. Throughput: 0: 9632.9, 1: 9597.5. Samples: 248433572. Policy #0 lag: (min: 11.0, avg: 20.6, max: 43.0) [2023-12-26 18:53:46,063][104569] Avg episode reward: [(0, '8724.520'), (1, '9349.887')] [2023-12-26 18:53:46,066][105692] Updated weights for policy 0, policy_version 485047 (0.0009) [2023-12-26 18:53:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000485416_124280832.pth... [2023-12-26 18:53:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000484296_123994112.pth [2023-12-26 18:53:46,107][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000485056_124190720.pth... [2023-12-26 18:53:46,110][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000483904_123895808.pth [2023-12-26 18:53:46,684][105692] Updated weights for policy 0, policy_version 485057 (0.0009) [2023-12-26 18:53:46,749][105692] Updated weights for policy 0, policy_version 485067 (0.0009) [2023-12-26 18:53:46,804][105692] Updated weights for policy 0, policy_version 485077 (0.0009) [2023-12-26 18:53:46,807][105620] Updated weights for policy 1, policy_version 485420 (0.0006) [2023-12-26 18:53:46,861][105692] Updated weights for policy 0, policy_version 485087 (0.0009) [2023-12-26 18:53:46,868][105620] Updated weights for policy 1, policy_version 485430 (0.0007) [2023-12-26 18:53:46,929][105620] Updated weights for policy 1, policy_version 485441 (0.0009) [2023-12-26 18:53:47,537][105620] Updated weights for policy 1, policy_version 485451 (0.0009) [2023-12-26 18:53:47,601][105620] Updated weights for policy 1, policy_version 485461 (0.0008) [2023-12-26 18:53:47,659][105692] Updated weights for policy 0, policy_version 485097 (0.0007) [2023-12-26 18:53:47,665][105620] Updated weights for policy 1, policy_version 485471 (0.0008) [2023-12-26 18:53:47,711][105692] Updated weights for policy 0, policy_version 485107 (0.0008) [2023-12-26 18:53:47,771][105692] Updated weights for policy 0, policy_version 485117 (0.0008) [2023-12-26 18:53:48,353][105620] Updated weights for policy 1, policy_version 485481 (0.0006) [2023-12-26 18:53:48,412][105620] Updated weights for policy 1, policy_version 485491 (0.0010) [2023-12-26 18:53:48,473][105620] Updated weights for policy 1, policy_version 485501 (0.0010) [2023-12-26 18:53:48,489][105692] Updated weights for policy 0, policy_version 485127 (0.0008) [2023-12-26 18:53:48,535][105620] Updated weights for policy 1, policy_version 485511 (0.0010) [2023-12-26 18:53:48,542][105692] Updated weights for policy 0, policy_version 485137 (0.0009) [2023-12-26 18:53:48,598][105692] Updated weights for policy 0, policy_version 485147 (0.0006) [2023-12-26 18:53:49,122][105620] Updated weights for policy 1, policy_version 485521 (0.0006) [2023-12-26 18:53:49,182][105620] Updated weights for policy 1, policy_version 485531 (0.0005) [2023-12-26 18:53:49,248][105620] Updated weights for policy 1, policy_version 485541 (0.0008) [2023-12-26 18:53:49,449][105692] Updated weights for policy 0, policy_version 485157 (0.0007) [2023-12-26 18:53:49,507][105692] Updated weights for policy 0, policy_version 485167 (0.0009) [2023-12-26 18:53:49,576][105692] Updated weights for policy 0, policy_version 485177 (0.0008) [2023-12-26 18:53:49,993][105620] Updated weights for policy 1, policy_version 485551 (0.0011) [2023-12-26 18:53:50,055][105620] Updated weights for policy 1, policy_version 485561 (0.0008) [2023-12-26 18:53:50,118][105620] Updated weights for policy 1, policy_version 485571 (0.0010) [2023-12-26 18:53:50,367][105692] Updated weights for policy 0, policy_version 485187 (0.0008) [2023-12-26 18:53:50,431][105692] Updated weights for policy 0, policy_version 485197 (0.0008) [2023-12-26 18:53:50,491][105692] Updated weights for policy 0, policy_version 485207 (0.0008) [2023-12-26 18:53:50,893][105620] Updated weights for policy 1, policy_version 485581 (0.0011) [2023-12-26 18:53:50,942][105620] Updated weights for policy 1, policy_version 485591 (0.0010) [2023-12-26 18:53:50,996][105620] Updated weights for policy 1, policy_version 485601 (0.0010) [2023-12-26 18:53:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 248561664. Throughput: 0: 9654.3, 1: 9623.6. Samples: 248549252. Policy #0 lag: (min: 11.0, avg: 20.6, max: 43.0) [2023-12-26 18:53:51,062][104569] Avg episode reward: [(0, '9084.942'), (1, '9344.922')] [2023-12-26 18:53:51,278][105692] Updated weights for policy 0, policy_version 485217 (0.0009) [2023-12-26 18:53:51,327][105692] Updated weights for policy 0, policy_version 485227 (0.0008) [2023-12-26 18:53:51,394][105692] Updated weights for policy 0, policy_version 485237 (0.0008) [2023-12-26 18:53:51,447][105692] Updated weights for policy 0, policy_version 485247 (0.0008) [2023-12-26 18:53:51,800][105620] Updated weights for policy 1, policy_version 485611 (0.0009) [2023-12-26 18:53:51,858][105620] Updated weights for policy 1, policy_version 485621 (0.0008) [2023-12-26 18:53:51,921][105620] Updated weights for policy 1, policy_version 485631 (0.0005) [2023-12-26 18:53:52,293][105692] Updated weights for policy 0, policy_version 485257 (0.0008) [2023-12-26 18:53:52,355][105692] Updated weights for policy 0, policy_version 485267 (0.0009) [2023-12-26 18:53:52,423][105692] Updated weights for policy 0, policy_version 485277 (0.0009) [2023-12-26 18:53:52,572][105620] Updated weights for policy 1, policy_version 485641 (0.0008) [2023-12-26 18:53:52,629][105620] Updated weights for policy 1, policy_version 485651 (0.0007) [2023-12-26 18:53:52,693][105620] Updated weights for policy 1, policy_version 485661 (0.0005) [2023-12-26 18:53:52,757][105620] Updated weights for policy 1, policy_version 485671 (0.0008) [2023-12-26 18:53:53,228][105692] Updated weights for policy 0, policy_version 485287 (0.0008) [2023-12-26 18:53:53,288][105692] Updated weights for policy 0, policy_version 485297 (0.0009) [2023-12-26 18:53:53,336][105692] Updated weights for policy 0, policy_version 485307 (0.0008) [2023-12-26 18:53:53,468][105620] Updated weights for policy 1, policy_version 485681 (0.0006) [2023-12-26 18:53:53,518][105620] Updated weights for policy 1, policy_version 485691 (0.0006) [2023-12-26 18:53:53,574][105620] Updated weights for policy 1, policy_version 485701 (0.0006) [2023-12-26 18:53:54,087][105692] Updated weights for policy 0, policy_version 485317 (0.0007) [2023-12-26 18:53:54,094][105620] Updated weights for policy 1, policy_version 485711 (0.0006) [2023-12-26 18:53:54,151][105692] Updated weights for policy 0, policy_version 485327 (0.0006) [2023-12-26 18:53:54,156][105620] Updated weights for policy 1, policy_version 485721 (0.0010) [2023-12-26 18:53:54,203][105692] Updated weights for policy 0, policy_version 485337 (0.0007) [2023-12-26 18:53:54,225][105620] Updated weights for policy 1, policy_version 485731 (0.0010) [2023-12-26 18:53:54,767][105620] Updated weights for policy 1, policy_version 485741 (0.0010) [2023-12-26 18:53:54,825][105620] Updated weights for policy 1, policy_version 485751 (0.0010) [2023-12-26 18:53:54,887][105620] Updated weights for policy 1, policy_version 485761 (0.0010) [2023-12-26 18:53:54,993][105692] Updated weights for policy 0, policy_version 485347 (0.0006) [2023-12-26 18:53:55,046][105692] Updated weights for policy 0, policy_version 485357 (0.0005) [2023-12-26 18:53:55,100][105692] Updated weights for policy 0, policy_version 485367 (0.0006) [2023-12-26 18:53:55,102][105585] KL-divergence is very high: 137.5198 [2023-12-26 18:53:55,148][105585] KL-divergence is very high: 115.0246 [2023-12-26 18:53:55,627][105620] Updated weights for policy 1, policy_version 485771 (0.0009) [2023-12-26 18:53:55,690][105620] Updated weights for policy 1, policy_version 485781 (0.0007) [2023-12-26 18:53:55,767][105620] Updated weights for policy 1, policy_version 485791 (0.0007) [2023-12-26 18:53:55,848][105692] Updated weights for policy 0, policy_version 485377 (0.0007) [2023-12-26 18:53:55,903][105692] Updated weights for policy 0, policy_version 485387 (0.0009) [2023-12-26 18:53:55,951][105692] Updated weights for policy 0, policy_version 485397 (0.0008) [2023-12-26 18:53:56,003][105692] Updated weights for policy 0, policy_version 485407 (0.0008) [2023-12-26 18:53:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 248659968. Throughput: 0: 9632.3, 1: 9660.5. Samples: 248664368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:53:56,063][104569] Avg episode reward: [(0, '8813.615'), (1, '9070.372')] [2023-12-26 18:53:56,315][105620] Updated weights for policy 1, policy_version 485801 (0.0008) [2023-12-26 18:53:56,365][105620] Updated weights for policy 1, policy_version 485811 (0.0005) [2023-12-26 18:53:56,418][105620] Updated weights for policy 1, policy_version 485821 (0.0005) [2023-12-26 18:53:56,479][105620] Updated weights for policy 1, policy_version 485831 (0.0005) [2023-12-26 18:53:56,744][105692] Updated weights for policy 0, policy_version 485417 (0.0005) [2023-12-26 18:53:56,801][105692] Updated weights for policy 0, policy_version 485427 (0.0007) [2023-12-26 18:53:56,859][105692] Updated weights for policy 0, policy_version 485438 (0.0010) [2023-12-26 18:53:57,001][105620] Updated weights for policy 1, policy_version 485841 (0.0005) [2023-12-26 18:53:57,047][105620] Updated weights for policy 1, policy_version 485851 (0.0005) [2023-12-26 18:53:57,094][105620] Updated weights for policy 1, policy_version 485861 (0.0005) [2023-12-26 18:53:57,616][105620] Updated weights for policy 1, policy_version 485871 (0.0005) [2023-12-26 18:53:57,641][105692] Updated weights for policy 0, policy_version 485448 (0.0006) [2023-12-26 18:53:57,684][105620] Updated weights for policy 1, policy_version 485881 (0.0005) [2023-12-26 18:53:57,701][105692] Updated weights for policy 0, policy_version 485458 (0.0005) [2023-12-26 18:53:57,743][105620] Updated weights for policy 1, policy_version 485891 (0.0005) [2023-12-26 18:53:57,753][105692] Updated weights for policy 0, policy_version 485468 (0.0005) [2023-12-26 18:53:58,252][105620] Updated weights for policy 1, policy_version 485901 (0.0005) [2023-12-26 18:53:58,316][105620] Updated weights for policy 1, policy_version 485911 (0.0007) [2023-12-26 18:53:58,384][105620] Updated weights for policy 1, policy_version 485921 (0.0008) [2023-12-26 18:53:58,491][105692] Updated weights for policy 0, policy_version 485478 (0.0007) [2023-12-26 18:53:58,553][105692] Updated weights for policy 0, policy_version 485488 (0.0008) [2023-12-26 18:53:58,611][105692] Updated weights for policy 0, policy_version 485498 (0.0008) [2023-12-26 18:53:59,162][105620] Updated weights for policy 1, policy_version 485931 (0.0008) [2023-12-26 18:53:59,235][105620] Updated weights for policy 1, policy_version 485941 (0.0009) [2023-12-26 18:53:59,300][105620] Updated weights for policy 1, policy_version 485951 (0.0008) [2023-12-26 18:53:59,438][105692] Updated weights for policy 0, policy_version 485508 (0.0008) [2023-12-26 18:53:59,501][105692] Updated weights for policy 0, policy_version 485518 (0.0008) [2023-12-26 18:53:59,566][105692] Updated weights for policy 0, policy_version 485528 (0.0007) [2023-12-26 18:54:00,093][105620] Updated weights for policy 1, policy_version 485961 (0.0009) [2023-12-26 18:54:00,153][105620] Updated weights for policy 1, policy_version 485971 (0.0008) [2023-12-26 18:54:00,215][105620] Updated weights for policy 1, policy_version 485981 (0.0009) [2023-12-26 18:54:00,250][105692] Updated weights for policy 0, policy_version 485538 (0.0006) [2023-12-26 18:54:00,265][105620] Updated weights for policy 1, policy_version 485991 (0.0007) [2023-12-26 18:54:00,305][105692] Updated weights for policy 0, policy_version 485548 (0.0009) [2023-12-26 18:54:00,362][105692] Updated weights for policy 0, policy_version 485558 (0.0010) [2023-12-26 18:54:00,414][105692] Updated weights for policy 0, policy_version 485568 (0.0008) [2023-12-26 18:54:00,876][105620] Updated weights for policy 1, policy_version 486001 (0.0008) [2023-12-26 18:54:00,929][105620] Updated weights for policy 1, policy_version 486011 (0.0009) [2023-12-26 18:54:00,992][105620] Updated weights for policy 1, policy_version 486021 (0.0007) [2023-12-26 18:54:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 248758272. Throughput: 0: 9628.2, 1: 9812.0. Samples: 248727960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:54:01,063][104569] Avg episode reward: [(0, '9087.975'), (1, '8899.157')] [2023-12-26 18:54:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000485568_124321792.pth... [2023-12-26 18:54:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000486024_124436480.pth... [2023-12-26 18:54:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000484480_124043264.pth [2023-12-26 18:54:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000484840_124133376.pth [2023-12-26 18:54:01,287][105692] Updated weights for policy 0, policy_version 485578 (0.0009) [2023-12-26 18:54:01,351][105692] Updated weights for policy 0, policy_version 485588 (0.0009) [2023-12-26 18:54:01,419][105692] Updated weights for policy 0, policy_version 485598 (0.0009) [2023-12-26 18:54:01,668][105620] Updated weights for policy 1, policy_version 486031 (0.0008) [2023-12-26 18:54:01,740][105620] Updated weights for policy 1, policy_version 486041 (0.0008) [2023-12-26 18:54:01,803][105620] Updated weights for policy 1, policy_version 486051 (0.0007) [2023-12-26 18:54:02,281][105692] Updated weights for policy 0, policy_version 485608 (0.0009) [2023-12-26 18:54:02,336][105692] Updated weights for policy 0, policy_version 485618 (0.0009) [2023-12-26 18:54:02,394][105692] Updated weights for policy 0, policy_version 485628 (0.0009) [2023-12-26 18:54:02,446][105620] Updated weights for policy 1, policy_version 486061 (0.0009) [2023-12-26 18:54:02,511][105620] Updated weights for policy 1, policy_version 486071 (0.0009) [2023-12-26 18:54:02,569][105620] Updated weights for policy 1, policy_version 486081 (0.0008) [2023-12-26 18:54:03,169][105692] Updated weights for policy 0, policy_version 485638 (0.0008) [2023-12-26 18:54:03,215][105692] Updated weights for policy 0, policy_version 485648 (0.0009) [2023-12-26 18:54:03,261][105692] Updated weights for policy 0, policy_version 485658 (0.0008) [2023-12-26 18:54:03,321][105620] Updated weights for policy 1, policy_version 486091 (0.0009) [2023-12-26 18:54:03,370][105620] Updated weights for policy 1, policy_version 486101 (0.0007) [2023-12-26 18:54:03,426][105620] Updated weights for policy 1, policy_version 486111 (0.0005) [2023-12-26 18:54:03,947][105620] Updated weights for policy 1, policy_version 486121 (0.0005) [2023-12-26 18:54:04,014][105620] Updated weights for policy 1, policy_version 486131 (0.0006) [2023-12-26 18:54:04,082][105620] Updated weights for policy 1, policy_version 486141 (0.0009) [2023-12-26 18:54:04,143][105620] Updated weights for policy 1, policy_version 486151 (0.0009) [2023-12-26 18:54:04,160][105692] Updated weights for policy 0, policy_version 485668 (0.0009) [2023-12-26 18:54:04,227][105692] Updated weights for policy 0, policy_version 485678 (0.0008) [2023-12-26 18:54:04,291][105692] Updated weights for policy 0, policy_version 485688 (0.0009) [2023-12-26 18:54:04,806][105620] Updated weights for policy 1, policy_version 486161 (0.0010) [2023-12-26 18:54:04,871][105620] Updated weights for policy 1, policy_version 486171 (0.0010) [2023-12-26 18:54:04,937][105620] Updated weights for policy 1, policy_version 486181 (0.0006) [2023-12-26 18:54:05,115][105692] Updated weights for policy 0, policy_version 485698 (0.0008) [2023-12-26 18:54:05,170][105692] Updated weights for policy 0, policy_version 485709 (0.0010) [2023-12-26 18:54:05,224][105692] Updated weights for policy 0, policy_version 485719 (0.0010) [2023-12-26 18:54:05,461][105620] Updated weights for policy 1, policy_version 486191 (0.0005) [2023-12-26 18:54:05,510][105620] Updated weights for policy 1, policy_version 486201 (0.0005) [2023-12-26 18:54:05,564][105620] Updated weights for policy 1, policy_version 486211 (0.0010) [2023-12-26 18:54:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 248848384. Throughput: 0: 9430.4, 1: 9867.0. Samples: 248841212. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:54:06,062][104569] Avg episode reward: [(0, '9178.292'), (1, '9178.297')] [2023-12-26 18:54:06,109][105692] Updated weights for policy 0, policy_version 485729 (0.0010) [2023-12-26 18:54:06,171][105692] Updated weights for policy 0, policy_version 485739 (0.0009) [2023-12-26 18:54:06,199][105620] Updated weights for policy 1, policy_version 486221 (0.0009) [2023-12-26 18:54:06,227][105692] Updated weights for policy 0, policy_version 485749 (0.0008) [2023-12-26 18:54:06,262][105620] Updated weights for policy 1, policy_version 486231 (0.0009) [2023-12-26 18:54:06,281][105692] Updated weights for policy 0, policy_version 485759 (0.0006) [2023-12-26 18:54:06,318][105620] Updated weights for policy 1, policy_version 486241 (0.0011) [2023-12-26 18:54:07,060][105692] Updated weights for policy 0, policy_version 485769 (0.0008) [2023-12-26 18:54:07,060][105620] Updated weights for policy 1, policy_version 486251 (0.0009) [2023-12-26 18:54:07,113][105620] Updated weights for policy 1, policy_version 486261 (0.0005) [2023-12-26 18:54:07,116][105692] Updated weights for policy 0, policy_version 485779 (0.0008) [2023-12-26 18:54:07,168][105692] Updated weights for policy 0, policy_version 485789 (0.0008) [2023-12-26 18:54:07,174][105620] Updated weights for policy 1, policy_version 486271 (0.0010) [2023-12-26 18:54:07,882][105620] Updated weights for policy 1, policy_version 486281 (0.0010) [2023-12-26 18:54:07,914][105692] Updated weights for policy 0, policy_version 485799 (0.0009) [2023-12-26 18:54:07,934][105620] Updated weights for policy 1, policy_version 486291 (0.0010) [2023-12-26 18:54:07,964][105692] Updated weights for policy 0, policy_version 485809 (0.0007) [2023-12-26 18:54:07,989][105620] Updated weights for policy 1, policy_version 486301 (0.0010) [2023-12-26 18:54:08,016][105692] Updated weights for policy 0, policy_version 485819 (0.0010) [2023-12-26 18:54:08,048][105620] Updated weights for policy 1, policy_version 486311 (0.0010) [2023-12-26 18:54:08,683][105620] Updated weights for policy 1, policy_version 486321 (0.0006) [2023-12-26 18:54:08,747][105620] Updated weights for policy 1, policy_version 486331 (0.0005) [2023-12-26 18:54:08,797][105692] Updated weights for policy 0, policy_version 485829 (0.0009) [2023-12-26 18:54:08,806][105620] Updated weights for policy 1, policy_version 486341 (0.0006) [2023-12-26 18:54:08,856][105692] Updated weights for policy 0, policy_version 485839 (0.0010) [2023-12-26 18:54:08,911][105692] Updated weights for policy 0, policy_version 485849 (0.0010) [2023-12-26 18:54:09,328][105620] Updated weights for policy 1, policy_version 486351 (0.0008) [2023-12-26 18:54:09,392][105620] Updated weights for policy 1, policy_version 486361 (0.0011) [2023-12-26 18:54:09,459][105620] Updated weights for policy 1, policy_version 486371 (0.0011) [2023-12-26 18:54:09,678][105692] Updated weights for policy 0, policy_version 485859 (0.0011) [2023-12-26 18:54:09,733][105692] Updated weights for policy 0, policy_version 485869 (0.0010) [2023-12-26 18:54:09,792][105692] Updated weights for policy 0, policy_version 485879 (0.0010) [2023-12-26 18:54:10,210][105620] Updated weights for policy 1, policy_version 486381 (0.0011) [2023-12-26 18:54:10,286][105620] Updated weights for policy 1, policy_version 486391 (0.0010) [2023-12-26 18:54:10,354][105620] Updated weights for policy 1, policy_version 486401 (0.0010) [2023-12-26 18:54:10,580][105692] Updated weights for policy 0, policy_version 485889 (0.0008) [2023-12-26 18:54:10,643][105692] Updated weights for policy 0, policy_version 485899 (0.0008) [2023-12-26 18:54:10,688][105692] Updated weights for policy 0, policy_version 485909 (0.0008) [2023-12-26 18:54:10,746][105692] Updated weights for policy 0, policy_version 485919 (0.0009) [2023-12-26 18:54:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 248946688. Throughput: 0: 9310.1, 1: 10064.7. Samples: 248957300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:54:11,062][104569] Avg episode reward: [(0, '9269.017'), (1, '9354.781')] [2023-12-26 18:54:11,112][105620] Updated weights for policy 1, policy_version 486411 (0.0011) [2023-12-26 18:54:11,189][105620] Updated weights for policy 1, policy_version 486421 (0.0011) [2023-12-26 18:54:11,246][105620] Updated weights for policy 1, policy_version 486431 (0.0010) [2023-12-26 18:54:11,550][105692] Updated weights for policy 0, policy_version 485929 (0.0009) [2023-12-26 18:54:11,613][105692] Updated weights for policy 0, policy_version 485939 (0.0008) [2023-12-26 18:54:11,682][105692] Updated weights for policy 0, policy_version 485949 (0.0009) [2023-12-26 18:54:11,960][105620] Updated weights for policy 1, policy_version 486441 (0.0010) [2023-12-26 18:54:12,018][105620] Updated weights for policy 1, policy_version 486451 (0.0009) [2023-12-26 18:54:12,073][105620] Updated weights for policy 1, policy_version 486461 (0.0008) [2023-12-26 18:54:12,135][105620] Updated weights for policy 1, policy_version 486471 (0.0009) [2023-12-26 18:54:12,439][105692] Updated weights for policy 0, policy_version 485959 (0.0009) [2023-12-26 18:54:12,490][105692] Updated weights for policy 0, policy_version 485969 (0.0009) [2023-12-26 18:54:12,543][105692] Updated weights for policy 0, policy_version 485979 (0.0009) [2023-12-26 18:54:12,883][105620] Updated weights for policy 1, policy_version 486481 (0.0008) [2023-12-26 18:54:12,930][105620] Updated weights for policy 1, policy_version 486491 (0.0008) [2023-12-26 18:54:12,981][105620] Updated weights for policy 1, policy_version 486501 (0.0009) [2023-12-26 18:54:13,311][105692] Updated weights for policy 0, policy_version 485989 (0.0009) [2023-12-26 18:54:13,358][105692] Updated weights for policy 0, policy_version 485999 (0.0009) [2023-12-26 18:54:13,404][105692] Updated weights for policy 0, policy_version 486009 (0.0009) [2023-12-26 18:54:13,751][105620] Updated weights for policy 1, policy_version 486511 (0.0009) [2023-12-26 18:54:13,812][105620] Updated weights for policy 1, policy_version 486521 (0.0008) [2023-12-26 18:54:13,859][105620] Updated weights for policy 1, policy_version 486531 (0.0009) [2023-12-26 18:54:14,175][105692] Updated weights for policy 0, policy_version 486020 (0.0009) [2023-12-26 18:54:14,233][105692] Updated weights for policy 0, policy_version 486030 (0.0007) [2023-12-26 18:54:14,282][105692] Updated weights for policy 0, policy_version 486040 (0.0008) [2023-12-26 18:54:14,642][105620] Updated weights for policy 1, policy_version 486541 (0.0009) [2023-12-26 18:54:14,694][105620] Updated weights for policy 1, policy_version 486551 (0.0009) [2023-12-26 18:54:14,746][105620] Updated weights for policy 1, policy_version 486561 (0.0009) [2023-12-26 18:54:15,013][105692] Updated weights for policy 0, policy_version 486050 (0.0008) [2023-12-26 18:54:15,066][105692] Updated weights for policy 0, policy_version 486060 (0.0008) [2023-12-26 18:54:15,114][105692] Updated weights for policy 0, policy_version 486070 (0.0008) [2023-12-26 18:54:15,178][105692] Updated weights for policy 0, policy_version 486080 (0.0008) [2023-12-26 18:54:15,534][105620] Updated weights for policy 1, policy_version 486571 (0.0008) [2023-12-26 18:54:15,589][105620] Updated weights for policy 1, policy_version 486581 (0.0005) [2023-12-26 18:54:15,644][105620] Updated weights for policy 1, policy_version 486591 (0.0005) [2023-12-26 18:54:15,975][105692] Updated weights for policy 0, policy_version 486090 (0.0008) [2023-12-26 18:54:16,036][105692] Updated weights for policy 0, policy_version 486100 (0.0009) [2023-12-26 18:54:16,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19114.6, 300 sec: 19466.4). Total num frames: 249036800. Throughput: 0: 9279.6, 1: 10010.6. Samples: 249011664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:54:16,063][104569] Avg episode reward: [(0, '9177.881'), (1, '9354.590')] [2023-12-26 18:54:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000486600_124583936.pth... [2023-12-26 18:54:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000485416_124280832.pth [2023-12-26 18:54:16,091][105692] Updated weights for policy 0, policy_version 486110 (0.0007) [2023-12-26 18:54:16,099][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000486112_124461056.pth... [2023-12-26 18:54:16,102][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000485056_124190720.pth [2023-12-26 18:54:16,325][105620] Updated weights for policy 1, policy_version 486601 (0.0006) [2023-12-26 18:54:16,387][105620] Updated weights for policy 1, policy_version 486611 (0.0009) [2023-12-26 18:54:16,441][105620] Updated weights for policy 1, policy_version 486621 (0.0009) [2023-12-26 18:54:16,496][105620] Updated weights for policy 1, policy_version 486631 (0.0009) [2023-12-26 18:54:16,763][105692] Updated weights for policy 0, policy_version 486120 (0.0005) [2023-12-26 18:54:16,788][105585] KL-divergence is very high: 346.2229 [2023-12-26 18:54:16,826][105692] Updated weights for policy 0, policy_version 486130 (0.0005) [2023-12-26 18:54:16,837][105585] KL-divergence is very high: 405.8846 [2023-12-26 18:54:16,875][105692] Updated weights for policy 0, policy_version 486140 (0.0005) [2023-12-26 18:54:16,875][105585] KL-divergence is very high: 294.1154 [2023-12-26 18:54:17,342][105620] Updated weights for policy 1, policy_version 486641 (0.0009) [2023-12-26 18:54:17,403][105620] Updated weights for policy 1, policy_version 486651 (0.0009) [2023-12-26 18:54:17,456][105620] Updated weights for policy 1, policy_version 486661 (0.0009) [2023-12-26 18:54:17,472][105692] Updated weights for policy 0, policy_version 486150 (0.0008) [2023-12-26 18:54:17,525][105692] Updated weights for policy 0, policy_version 486160 (0.0009) [2023-12-26 18:54:17,584][105692] Updated weights for policy 0, policy_version 486170 (0.0010) [2023-12-26 18:54:18,162][105620] Updated weights for policy 1, policy_version 486671 (0.0009) [2023-12-26 18:54:18,224][105620] Updated weights for policy 1, policy_version 486681 (0.0009) [2023-12-26 18:54:18,290][105620] Updated weights for policy 1, policy_version 486691 (0.0008) [2023-12-26 18:54:18,292][105692] Updated weights for policy 0, policy_version 486180 (0.0008) [2023-12-26 18:54:18,358][105692] Updated weights for policy 0, policy_version 486190 (0.0008) [2023-12-26 18:54:18,416][105692] Updated weights for policy 0, policy_version 486200 (0.0008) [2023-12-26 18:54:19,066][105692] Updated weights for policy 0, policy_version 486210 (0.0009) [2023-12-26 18:54:19,085][105620] Updated weights for policy 1, policy_version 486701 (0.0006) [2023-12-26 18:54:19,127][105692] Updated weights for policy 0, policy_version 486220 (0.0008) [2023-12-26 18:54:19,145][105620] Updated weights for policy 1, policy_version 486711 (0.0006) [2023-12-26 18:54:19,184][105692] Updated weights for policy 0, policy_version 486230 (0.0008) [2023-12-26 18:54:19,202][105620] Updated weights for policy 1, policy_version 486721 (0.0006) [2023-12-26 18:54:19,246][105692] Updated weights for policy 0, policy_version 486240 (0.0007) [2023-12-26 18:54:19,966][105620] Updated weights for policy 1, policy_version 486731 (0.0008) [2023-12-26 18:54:20,028][105620] Updated weights for policy 1, policy_version 486741 (0.0008) [2023-12-26 18:54:20,037][105692] Updated weights for policy 0, policy_version 486250 (0.0008) [2023-12-26 18:54:20,093][105620] Updated weights for policy 1, policy_version 486751 (0.0009) [2023-12-26 18:54:20,097][105692] Updated weights for policy 0, policy_version 486260 (0.0006) [2023-12-26 18:54:20,148][105692] Updated weights for policy 0, policy_version 486270 (0.0007) [2023-12-26 18:54:20,894][105620] Updated weights for policy 1, policy_version 486761 (0.0008) [2023-12-26 18:54:20,899][105692] Updated weights for policy 0, policy_version 486280 (0.0007) [2023-12-26 18:54:20,955][105620] Updated weights for policy 1, policy_version 486771 (0.0008) [2023-12-26 18:54:20,963][105692] Updated weights for policy 0, policy_version 486290 (0.0008) [2023-12-26 18:54:21,018][105620] Updated weights for policy 1, policy_version 486781 (0.0009) [2023-12-26 18:54:21,030][105692] Updated weights for policy 0, policy_version 486300 (0.0009) [2023-12-26 18:54:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 249135104. Throughput: 0: 9311.1, 1: 9987.9. Samples: 249126492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:54:21,062][104569] Avg episode reward: [(0, '9179.309'), (1, '9354.466')] [2023-12-26 18:54:21,081][105620] Updated weights for policy 1, policy_version 486791 (0.0007) [2023-12-26 18:54:21,773][105692] Updated weights for policy 0, policy_version 486310 (0.0010) [2023-12-26 18:54:21,834][105692] Updated weights for policy 0, policy_version 486320 (0.0008) [2023-12-26 18:54:21,839][105620] Updated weights for policy 1, policy_version 486801 (0.0007) [2023-12-26 18:54:21,894][105692] Updated weights for policy 0, policy_version 486330 (0.0007) [2023-12-26 18:54:21,899][105620] Updated weights for policy 1, policy_version 486811 (0.0009) [2023-12-26 18:54:21,955][105620] Updated weights for policy 1, policy_version 486821 (0.0008) [2023-12-26 18:54:22,590][105692] Updated weights for policy 0, policy_version 486340 (0.0006) [2023-12-26 18:54:22,653][105692] Updated weights for policy 0, policy_version 486350 (0.0009) [2023-12-26 18:54:22,707][105620] Updated weights for policy 1, policy_version 486831 (0.0009) [2023-12-26 18:54:22,715][105692] Updated weights for policy 0, policy_version 486360 (0.0008) [2023-12-26 18:54:22,771][105620] Updated weights for policy 1, policy_version 486841 (0.0007) [2023-12-26 18:54:22,834][105620] Updated weights for policy 1, policy_version 486851 (0.0009) [2023-12-26 18:54:23,377][105692] Updated weights for policy 0, policy_version 486370 (0.0010) [2023-12-26 18:54:23,436][105692] Updated weights for policy 0, policy_version 486380 (0.0011) [2023-12-26 18:54:23,488][105692] Updated weights for policy 0, policy_version 486390 (0.0011) [2023-12-26 18:54:23,540][105692] Updated weights for policy 0, policy_version 486400 (0.0011) [2023-12-26 18:54:23,570][105620] Updated weights for policy 1, policy_version 486861 (0.0009) [2023-12-26 18:54:23,633][105620] Updated weights for policy 1, policy_version 486871 (0.0011) [2023-12-26 18:54:23,699][105620] Updated weights for policy 1, policy_version 486881 (0.0010) [2023-12-26 18:54:24,220][105692] Updated weights for policy 0, policy_version 486410 (0.0006) [2023-12-26 18:54:24,283][105692] Updated weights for policy 0, policy_version 486420 (0.0005) [2023-12-26 18:54:24,347][105692] Updated weights for policy 0, policy_version 486430 (0.0006) [2023-12-26 18:54:24,456][105620] Updated weights for policy 1, policy_version 486891 (0.0009) [2023-12-26 18:54:24,515][105620] Updated weights for policy 1, policy_version 486901 (0.0009) [2023-12-26 18:54:24,559][105620] Updated weights for policy 1, policy_version 486911 (0.0010) [2023-12-26 18:54:24,929][105692] Updated weights for policy 0, policy_version 486440 (0.0010) [2023-12-26 18:54:24,981][105692] Updated weights for policy 0, policy_version 486450 (0.0011) [2023-12-26 18:54:25,040][105692] Updated weights for policy 0, policy_version 486460 (0.0006) [2023-12-26 18:54:25,299][105620] Updated weights for policy 1, policy_version 486921 (0.0010) [2023-12-26 18:54:25,354][105620] Updated weights for policy 1, policy_version 486931 (0.0010) [2023-12-26 18:54:25,409][105620] Updated weights for policy 1, policy_version 486941 (0.0010) [2023-12-26 18:54:25,461][105620] Updated weights for policy 1, policy_version 486951 (0.0010) [2023-12-26 18:54:25,733][105692] Updated weights for policy 0, policy_version 486470 (0.0005) [2023-12-26 18:54:25,797][105692] Updated weights for policy 0, policy_version 486480 (0.0006) [2023-12-26 18:54:25,841][105692] Updated weights for policy 0, policy_version 486490 (0.0008) [2023-12-26 18:54:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 249233408. Throughput: 0: 9317.1, 1: 9912.5. Samples: 249240912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:54:26,062][104569] Avg episode reward: [(0, '9179.544'), (1, '9354.995')] [2023-12-26 18:54:26,214][105620] Updated weights for policy 1, policy_version 486961 (0.0011) [2023-12-26 18:54:26,266][105620] Updated weights for policy 1, policy_version 486971 (0.0010) [2023-12-26 18:54:26,319][105620] Updated weights for policy 1, policy_version 486981 (0.0010) [2023-12-26 18:54:26,417][105692] Updated weights for policy 0, policy_version 486500 (0.0008) [2023-12-26 18:54:26,475][105692] Updated weights for policy 0, policy_version 486510 (0.0010) [2023-12-26 18:54:26,526][105692] Updated weights for policy 0, policy_version 486520 (0.0010) [2023-12-26 18:54:27,072][105620] Updated weights for policy 1, policy_version 486991 (0.0010) [2023-12-26 18:54:27,133][105620] Updated weights for policy 1, policy_version 487001 (0.0010) [2023-12-26 18:54:27,168][105692] Updated weights for policy 0, policy_version 486530 (0.0010) [2023-12-26 18:54:27,188][105620] Updated weights for policy 1, policy_version 487011 (0.0010) [2023-12-26 18:54:27,218][105692] Updated weights for policy 0, policy_version 486540 (0.0010) [2023-12-26 18:54:27,265][105692] Updated weights for policy 0, policy_version 486550 (0.0010) [2023-12-26 18:54:27,314][105692] Updated weights for policy 0, policy_version 486560 (0.0007) [2023-12-26 18:54:27,907][105620] Updated weights for policy 1, policy_version 487021 (0.0010) [2023-12-26 18:54:27,955][105620] Updated weights for policy 1, policy_version 487031 (0.0010) [2023-12-26 18:54:27,999][105620] Updated weights for policy 1, policy_version 487041 (0.0010) [2023-12-26 18:54:28,061][105692] Updated weights for policy 0, policy_version 486570 (0.0006) [2023-12-26 18:54:28,106][105692] Updated weights for policy 0, policy_version 486580 (0.0006) [2023-12-26 18:54:28,160][105692] Updated weights for policy 0, policy_version 486590 (0.0008) [2023-12-26 18:54:28,790][105620] Updated weights for policy 1, policy_version 487051 (0.0009) [2023-12-26 18:54:28,812][105692] Updated weights for policy 0, policy_version 486600 (0.0006) [2023-12-26 18:54:28,846][105620] Updated weights for policy 1, policy_version 487061 (0.0007) [2023-12-26 18:54:28,868][105692] Updated weights for policy 0, policy_version 486610 (0.0006) [2023-12-26 18:54:28,898][105620] Updated weights for policy 1, policy_version 487071 (0.0007) [2023-12-26 18:54:28,920][105692] Updated weights for policy 0, policy_version 486620 (0.0007) [2023-12-26 18:54:29,612][105692] Updated weights for policy 0, policy_version 486630 (0.0010) [2023-12-26 18:54:29,663][105692] Updated weights for policy 0, policy_version 486640 (0.0010) [2023-12-26 18:54:29,681][105620] Updated weights for policy 1, policy_version 487081 (0.0007) [2023-12-26 18:54:29,713][105692] Updated weights for policy 0, policy_version 486650 (0.0009) [2023-12-26 18:54:29,738][105620] Updated weights for policy 1, policy_version 487091 (0.0007) [2023-12-26 18:54:29,797][105620] Updated weights for policy 1, policy_version 487101 (0.0010) [2023-12-26 18:54:29,858][105620] Updated weights for policy 1, policy_version 487111 (0.0010) [2023-12-26 18:54:30,357][105692] Updated weights for policy 0, policy_version 486660 (0.0005) [2023-12-26 18:54:30,416][105692] Updated weights for policy 0, policy_version 486670 (0.0006) [2023-12-26 18:54:30,488][105692] Updated weights for policy 0, policy_version 486680 (0.0010) [2023-12-26 18:54:30,651][105620] Updated weights for policy 1, policy_version 487121 (0.0009) [2023-12-26 18:54:30,708][105620] Updated weights for policy 1, policy_version 487131 (0.0008) [2023-12-26 18:54:30,763][105620] Updated weights for policy 1, policy_version 487141 (0.0006) [2023-12-26 18:54:31,023][105692] Updated weights for policy 0, policy_version 486690 (0.0009) [2023-12-26 18:54:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 249331712. Throughput: 0: 9384.5, 1: 9888.2. Samples: 249300844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:54:31,063][104569] Avg episode reward: [(0, '9357.987'), (1, '9352.173')] [2023-12-26 18:54:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000487144_124723200.pth... [2023-12-26 18:54:31,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000486024_124436480.pth [2023-12-26 18:54:31,087][105692] Updated weights for policy 0, policy_version 486700 (0.0008) [2023-12-26 18:54:31,153][105692] Updated weights for policy 0, policy_version 486710 (0.0009) [2023-12-26 18:54:31,213][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000486720_124616704.pth... [2023-12-26 18:54:31,214][105692] Updated weights for policy 0, policy_version 486720 (0.0010) [2023-12-26 18:54:31,218][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000485568_124321792.pth [2023-12-26 18:54:31,555][105620] Updated weights for policy 1, policy_version 487151 (0.0009) [2023-12-26 18:54:31,619][105620] Updated weights for policy 1, policy_version 487161 (0.0007) [2023-12-26 18:54:31,681][105620] Updated weights for policy 1, policy_version 487171 (0.0011) [2023-12-26 18:54:31,960][105692] Updated weights for policy 0, policy_version 486730 (0.0006) [2023-12-26 18:54:32,019][105692] Updated weights for policy 0, policy_version 486740 (0.0010) [2023-12-26 18:54:32,083][105692] Updated weights for policy 0, policy_version 486750 (0.0009) [2023-12-26 18:54:32,334][105620] Updated weights for policy 1, policy_version 487181 (0.0011) [2023-12-26 18:54:32,401][105620] Updated weights for policy 1, policy_version 487191 (0.0011) [2023-12-26 18:54:32,468][105620] Updated weights for policy 1, policy_version 487201 (0.0010) [2023-12-26 18:54:32,735][105692] Updated weights for policy 0, policy_version 486760 (0.0010) [2023-12-26 18:54:32,789][105692] Updated weights for policy 0, policy_version 486770 (0.0010) [2023-12-26 18:54:32,847][105692] Updated weights for policy 0, policy_version 486780 (0.0010) [2023-12-26 18:54:33,123][105620] Updated weights for policy 1, policy_version 487211 (0.0008) [2023-12-26 18:54:33,177][105620] Updated weights for policy 1, policy_version 487221 (0.0008) [2023-12-26 18:54:33,231][105620] Updated weights for policy 1, policy_version 487231 (0.0007) [2023-12-26 18:54:33,587][105692] Updated weights for policy 0, policy_version 486790 (0.0010) [2023-12-26 18:54:33,657][105692] Updated weights for policy 0, policy_version 486800 (0.0010) [2023-12-26 18:54:33,717][105692] Updated weights for policy 0, policy_version 486810 (0.0010) [2023-12-26 18:54:33,834][105620] Updated weights for policy 1, policy_version 487241 (0.0009) [2023-12-26 18:54:33,894][105620] Updated weights for policy 1, policy_version 487251 (0.0007) [2023-12-26 18:54:33,955][105620] Updated weights for policy 1, policy_version 487261 (0.0005) [2023-12-26 18:54:34,013][105620] Updated weights for policy 1, policy_version 487271 (0.0007) [2023-12-26 18:54:34,395][105692] Updated weights for policy 0, policy_version 486820 (0.0009) [2023-12-26 18:54:34,450][105692] Updated weights for policy 0, policy_version 486830 (0.0008) [2023-12-26 18:54:34,510][105692] Updated weights for policy 0, policy_version 486840 (0.0008) [2023-12-26 18:54:34,680][105620] Updated weights for policy 1, policy_version 487281 (0.0006) [2023-12-26 18:54:34,733][105620] Updated weights for policy 1, policy_version 487291 (0.0007) [2023-12-26 18:54:34,785][105620] Updated weights for policy 1, policy_version 487301 (0.0008) [2023-12-26 18:54:35,258][105692] Updated weights for policy 0, policy_version 486850 (0.0009) [2023-12-26 18:54:35,311][105585] KL-divergence is very high: 139.0779 [2023-12-26 18:54:35,326][105692] Updated weights for policy 0, policy_version 486860 (0.0010) [2023-12-26 18:54:35,360][105585] KL-divergence is very high: 306.4160 [2023-12-26 18:54:35,372][105585] KL-divergence is very high: 127.3196 [2023-12-26 18:54:35,384][105692] Updated weights for policy 0, policy_version 486870 (0.0010) [2023-12-26 18:54:35,407][105585] KL-divergence is very high: 278.5740 [2023-12-26 18:54:35,419][105585] KL-divergence is very high: 108.7278 [2023-12-26 18:54:35,445][105692] Updated weights for policy 0, policy_version 486880 (0.0010) [2023-12-26 18:54:35,509][105620] Updated weights for policy 1, policy_version 487311 (0.0007) [2023-12-26 18:54:35,575][105620] Updated weights for policy 1, policy_version 487321 (0.0005) [2023-12-26 18:54:35,640][105620] Updated weights for policy 1, policy_version 487331 (0.0005) [2023-12-26 18:54:36,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.6, 300 sec: 19410.9). Total num frames: 249430016. Throughput: 0: 9475.8, 1: 9903.7. Samples: 249421336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:54:36,063][104569] Avg episode reward: [(0, '9082.910'), (1, '9347.433')] [2023-12-26 18:54:36,109][105585] KL-divergence is very high: 211.8788 [2023-12-26 18:54:36,118][105620] Updated weights for policy 1, policy_version 487341 (0.0007) [2023-12-26 18:54:36,161][105585] KL-divergence is very high: 141.1230 [2023-12-26 18:54:36,172][105692] Updated weights for policy 0, policy_version 486890 (0.0009) [2023-12-26 18:54:36,182][105620] Updated weights for policy 1, policy_version 487351 (0.0006) [2023-12-26 18:54:36,236][105692] Updated weights for policy 0, policy_version 486900 (0.0009) [2023-12-26 18:54:36,242][105620] Updated weights for policy 1, policy_version 487361 (0.0006) [2023-12-26 18:54:36,294][105692] Updated weights for policy 0, policy_version 486910 (0.0007) [2023-12-26 18:54:36,852][105692] Updated weights for policy 0, policy_version 486920 (0.0009) [2023-12-26 18:54:36,918][105692] Updated weights for policy 0, policy_version 486930 (0.0010) [2023-12-26 18:54:36,984][105692] Updated weights for policy 0, policy_version 486940 (0.0008) [2023-12-26 18:54:37,049][105620] Updated weights for policy 1, policy_version 487371 (0.0007) [2023-12-26 18:54:37,104][105620] Updated weights for policy 1, policy_version 487381 (0.0008) [2023-12-26 18:54:37,150][105620] Updated weights for policy 1, policy_version 487391 (0.0006) [2023-12-26 18:54:37,701][105692] Updated weights for policy 0, policy_version 486950 (0.0010) [2023-12-26 18:54:37,756][105692] Updated weights for policy 0, policy_version 486960 (0.0010) [2023-12-26 18:54:37,816][105692] Updated weights for policy 0, policy_version 486970 (0.0010) [2023-12-26 18:54:37,865][105620] Updated weights for policy 1, policy_version 487401 (0.0006) [2023-12-26 18:54:37,931][105620] Updated weights for policy 1, policy_version 487411 (0.0008) [2023-12-26 18:54:37,993][105620] Updated weights for policy 1, policy_version 487421 (0.0010) [2023-12-26 18:54:38,050][105620] Updated weights for policy 1, policy_version 487431 (0.0010) [2023-12-26 18:54:38,500][105692] Updated weights for policy 0, policy_version 486980 (0.0008) [2023-12-26 18:54:38,562][105692] Updated weights for policy 0, policy_version 486990 (0.0005) [2023-12-26 18:54:38,614][105692] Updated weights for policy 0, policy_version 487000 (0.0007) [2023-12-26 18:54:38,759][105620] Updated weights for policy 1, policy_version 487441 (0.0006) [2023-12-26 18:54:38,806][105620] Updated weights for policy 1, policy_version 487451 (0.0005) [2023-12-26 18:54:38,858][105620] Updated weights for policy 1, policy_version 487461 (0.0007) [2023-12-26 18:54:39,335][105692] Updated weights for policy 0, policy_version 487010 (0.0010) [2023-12-26 18:54:39,408][105692] Updated weights for policy 0, policy_version 487020 (0.0010) [2023-12-26 18:54:39,474][105692] Updated weights for policy 0, policy_version 487030 (0.0009) [2023-12-26 18:54:39,508][105620] Updated weights for policy 1, policy_version 487471 (0.0008) [2023-12-26 18:54:39,537][105692] Updated weights for policy 0, policy_version 487040 (0.0008) [2023-12-26 18:54:39,575][105620] Updated weights for policy 1, policy_version 487481 (0.0006) [2023-12-26 18:54:39,639][105620] Updated weights for policy 1, policy_version 487491 (0.0009) [2023-12-26 18:54:40,284][105692] Updated weights for policy 0, policy_version 487050 (0.0011) [2023-12-26 18:54:40,340][105692] Updated weights for policy 0, policy_version 487060 (0.0011) [2023-12-26 18:54:40,358][105620] Updated weights for policy 1, policy_version 487501 (0.0011) [2023-12-26 18:54:40,405][105692] Updated weights for policy 0, policy_version 487070 (0.0009) [2023-12-26 18:54:40,418][105620] Updated weights for policy 1, policy_version 487511 (0.0011) [2023-12-26 18:54:40,477][105620] Updated weights for policy 1, policy_version 487521 (0.0010) [2023-12-26 18:54:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 249528320. Throughput: 0: 9591.0, 1: 9883.3. Samples: 249540712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:54:41,062][104569] Avg episode reward: [(0, '9083.119'), (1, '9255.233')] [2023-12-26 18:54:41,076][105692] Updated weights for policy 0, policy_version 487080 (0.0008) [2023-12-26 18:54:41,144][105692] Updated weights for policy 0, policy_version 487090 (0.0009) [2023-12-26 18:54:41,207][105692] Updated weights for policy 0, policy_version 487100 (0.0011) [2023-12-26 18:54:41,245][105620] Updated weights for policy 1, policy_version 487531 (0.0010) [2023-12-26 18:54:41,315][105620] Updated weights for policy 1, policy_version 487541 (0.0006) [2023-12-26 18:54:41,386][105620] Updated weights for policy 1, policy_version 487551 (0.0008) [2023-12-26 18:54:41,979][105692] Updated weights for policy 0, policy_version 487110 (0.0008) [2023-12-26 18:54:42,039][105692] Updated weights for policy 0, policy_version 487120 (0.0009) [2023-12-26 18:54:42,097][105620] Updated weights for policy 1, policy_version 487561 (0.0008) [2023-12-26 18:54:42,109][105692] Updated weights for policy 0, policy_version 487130 (0.0009) [2023-12-26 18:54:42,165][105620] Updated weights for policy 1, policy_version 487571 (0.0007) [2023-12-26 18:54:42,236][105620] Updated weights for policy 1, policy_version 487581 (0.0006) [2023-12-26 18:54:42,301][105620] Updated weights for policy 1, policy_version 487591 (0.0008) [2023-12-26 18:54:42,815][105692] Updated weights for policy 0, policy_version 487140 (0.0007) [2023-12-26 18:54:42,866][105692] Updated weights for policy 0, policy_version 487150 (0.0007) [2023-12-26 18:54:42,932][105692] Updated weights for policy 0, policy_version 487160 (0.0007) [2023-12-26 18:54:43,020][105620] Updated weights for policy 1, policy_version 487601 (0.0009) [2023-12-26 18:54:43,075][105620] Updated weights for policy 1, policy_version 487611 (0.0009) [2023-12-26 18:54:43,123][105620] Updated weights for policy 1, policy_version 487621 (0.0008) [2023-12-26 18:54:43,644][105692] Updated weights for policy 0, policy_version 487170 (0.0010) [2023-12-26 18:54:43,711][105692] Updated weights for policy 0, policy_version 487181 (0.0010) [2023-12-26 18:54:43,763][105692] Updated weights for policy 0, policy_version 487192 (0.0010) [2023-12-26 18:54:43,799][105620] Updated weights for policy 1, policy_version 487631 (0.0007) [2023-12-26 18:54:43,845][105620] Updated weights for policy 1, policy_version 487641 (0.0008) [2023-12-26 18:54:43,893][105620] Updated weights for policy 1, policy_version 487651 (0.0005) [2023-12-26 18:54:44,429][105692] Updated weights for policy 0, policy_version 487202 (0.0007) [2023-12-26 18:54:44,488][105692] Updated weights for policy 0, policy_version 487212 (0.0009) [2023-12-26 18:54:44,520][105620] Updated weights for policy 1, policy_version 487661 (0.0005) [2023-12-26 18:54:44,553][105692] Updated weights for policy 0, policy_version 487222 (0.0009) [2023-12-26 18:54:44,580][105620] Updated weights for policy 1, policy_version 487671 (0.0007) [2023-12-26 18:54:44,611][105692] Updated weights for policy 0, policy_version 487232 (0.0006) [2023-12-26 18:54:44,635][105620] Updated weights for policy 1, policy_version 487681 (0.0007) [2023-12-26 18:54:45,269][105620] Updated weights for policy 1, policy_version 487691 (0.0009) [2023-12-26 18:54:45,324][105620] Updated weights for policy 1, policy_version 487701 (0.0006) [2023-12-26 18:54:45,378][105620] Updated weights for policy 1, policy_version 487711 (0.0006) [2023-12-26 18:54:45,438][105692] Updated weights for policy 0, policy_version 487242 (0.0008) [2023-12-26 18:54:45,495][105692] Updated weights for policy 0, policy_version 487252 (0.0009) [2023-12-26 18:54:45,547][105692] Updated weights for policy 0, policy_version 487262 (0.0009) [2023-12-26 18:54:46,038][105620] Updated weights for policy 1, policy_version 487721 (0.0008) [2023-12-26 18:54:46,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 249626624. Throughput: 0: 9613.8, 1: 9730.3. Samples: 249598448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:54:46,063][104569] Avg episode reward: [(0, '7804.897'), (1, '9257.305')] [2023-12-26 18:54:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000487264_124755968.pth... [2023-12-26 18:54:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000486112_124461056.pth [2023-12-26 18:54:46,091][105620] Updated weights for policy 1, policy_version 487731 (0.0006) [2023-12-26 18:54:46,148][105620] Updated weights for policy 1, policy_version 487741 (0.0005) [2023-12-26 18:54:46,207][105620] Updated weights for policy 1, policy_version 487751 (0.0007) [2023-12-26 18:54:46,211][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000487752_124878848.pth... [2023-12-26 18:54:46,213][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000486600_124583936.pth [2023-12-26 18:54:46,344][105692] Updated weights for policy 0, policy_version 487272 (0.0010) [2023-12-26 18:54:46,402][105692] Updated weights for policy 0, policy_version 487282 (0.0010) [2023-12-26 18:54:46,450][105692] Updated weights for policy 0, policy_version 487292 (0.0010) [2023-12-26 18:54:46,906][105620] Updated weights for policy 1, policy_version 487761 (0.0010) [2023-12-26 18:54:46,955][105620] Updated weights for policy 1, policy_version 487771 (0.0010) [2023-12-26 18:54:47,007][105620] Updated weights for policy 1, policy_version 487781 (0.0010) [2023-12-26 18:54:47,087][105692] Updated weights for policy 0, policy_version 487302 (0.0010) [2023-12-26 18:54:47,128][105692] Updated weights for policy 0, policy_version 487312 (0.0008) [2023-12-26 18:54:47,182][105692] Updated weights for policy 0, policy_version 487322 (0.0008) [2023-12-26 18:54:47,741][105620] Updated weights for policy 1, policy_version 487791 (0.0010) [2023-12-26 18:54:47,794][105620] Updated weights for policy 1, policy_version 487801 (0.0009) [2023-12-26 18:54:47,863][105620] Updated weights for policy 1, policy_version 487811 (0.0010) [2023-12-26 18:54:47,877][105692] Updated weights for policy 0, policy_version 487332 (0.0006) [2023-12-26 18:54:47,925][105692] Updated weights for policy 0, policy_version 487342 (0.0006) [2023-12-26 18:54:47,982][105692] Updated weights for policy 0, policy_version 487352 (0.0005) [2023-12-26 18:54:48,613][105692] Updated weights for policy 0, policy_version 487362 (0.0007) [2023-12-26 18:54:48,671][105692] Updated weights for policy 0, policy_version 487372 (0.0008) [2023-12-26 18:54:48,704][105620] Updated weights for policy 1, policy_version 487821 (0.0008) [2023-12-26 18:54:48,727][105692] Updated weights for policy 0, policy_version 487382 (0.0009) [2023-12-26 18:54:48,761][105620] Updated weights for policy 1, policy_version 487831 (0.0007) [2023-12-26 18:54:48,776][105692] Updated weights for policy 0, policy_version 487392 (0.0006) [2023-12-26 18:54:48,818][105620] Updated weights for policy 1, policy_version 487841 (0.0008) [2023-12-26 18:54:49,532][105692] Updated weights for policy 0, policy_version 487402 (0.0008) [2023-12-26 18:54:49,577][105620] Updated weights for policy 1, policy_version 487851 (0.0008) [2023-12-26 18:54:49,583][105692] Updated weights for policy 0, policy_version 487412 (0.0009) [2023-12-26 18:54:49,638][105620] Updated weights for policy 1, policy_version 487861 (0.0008) [2023-12-26 18:54:49,639][105692] Updated weights for policy 0, policy_version 487422 (0.0005) [2023-12-26 18:54:49,710][105620] Updated weights for policy 1, policy_version 487871 (0.0009) [2023-12-26 18:54:50,295][105692] Updated weights for policy 0, policy_version 487432 (0.0008) [2023-12-26 18:54:50,361][105692] Updated weights for policy 0, policy_version 487442 (0.0009) [2023-12-26 18:54:50,408][105692] Updated weights for policy 0, policy_version 487452 (0.0009) [2023-12-26 18:54:50,518][105620] Updated weights for policy 1, policy_version 487881 (0.0010) [2023-12-26 18:54:50,579][105620] Updated weights for policy 1, policy_version 487891 (0.0009) [2023-12-26 18:54:50,643][105620] Updated weights for policy 1, policy_version 487901 (0.0008) [2023-12-26 18:54:50,706][105620] Updated weights for policy 1, policy_version 487911 (0.0010) [2023-12-26 18:54:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 249724928. Throughput: 0: 9756.1, 1: 9674.3. Samples: 249715580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:54:51,062][104569] Avg episode reward: [(0, '7277.689'), (1, '9351.648')] [2023-12-26 18:54:51,113][105692] Updated weights for policy 0, policy_version 487462 (0.0007) [2023-12-26 18:54:51,177][105692] Updated weights for policy 0, policy_version 487472 (0.0009) [2023-12-26 18:54:51,236][105692] Updated weights for policy 0, policy_version 487482 (0.0010) [2023-12-26 18:54:51,503][105620] Updated weights for policy 1, policy_version 487921 (0.0007) [2023-12-26 18:54:51,567][105620] Updated weights for policy 1, policy_version 487931 (0.0009) [2023-12-26 18:54:51,631][105620] Updated weights for policy 1, policy_version 487941 (0.0009) [2023-12-26 18:54:51,928][105692] Updated weights for policy 0, policy_version 487492 (0.0009) [2023-12-26 18:54:51,984][105692] Updated weights for policy 0, policy_version 487502 (0.0009) [2023-12-26 18:54:52,036][105692] Updated weights for policy 0, policy_version 487512 (0.0009) [2023-12-26 18:54:52,399][105620] Updated weights for policy 1, policy_version 487951 (0.0007) [2023-12-26 18:54:52,453][105620] Updated weights for policy 1, policy_version 487961 (0.0009) [2023-12-26 18:54:52,500][105620] Updated weights for policy 1, policy_version 487971 (0.0008) [2023-12-26 18:54:52,815][105692] Updated weights for policy 0, policy_version 487522 (0.0008) [2023-12-26 18:54:52,871][105692] Updated weights for policy 0, policy_version 487532 (0.0009) [2023-12-26 18:54:52,920][105692] Updated weights for policy 0, policy_version 487542 (0.0010) [2023-12-26 18:54:52,983][105692] Updated weights for policy 0, policy_version 487552 (0.0010) [2023-12-26 18:54:53,305][105620] Updated weights for policy 1, policy_version 487981 (0.0009) [2023-12-26 18:54:53,358][105620] Updated weights for policy 1, policy_version 487991 (0.0009) [2023-12-26 18:54:53,411][105620] Updated weights for policy 1, policy_version 488001 (0.0010) [2023-12-26 18:54:53,575][105692] Updated weights for policy 0, policy_version 487562 (0.0005) [2023-12-26 18:54:53,633][105692] Updated weights for policy 0, policy_version 487572 (0.0007) [2023-12-26 18:54:53,699][105692] Updated weights for policy 0, policy_version 487582 (0.0010) [2023-12-26 18:54:54,273][105692] Updated weights for policy 0, policy_version 487592 (0.0010) [2023-12-26 18:54:54,291][105620] Updated weights for policy 1, policy_version 488012 (0.0010) [2023-12-26 18:54:54,327][105692] Updated weights for policy 0, policy_version 487602 (0.0009) [2023-12-26 18:54:54,347][105620] Updated weights for policy 1, policy_version 488022 (0.0010) [2023-12-26 18:54:54,383][105692] Updated weights for policy 0, policy_version 487612 (0.0008) [2023-12-26 18:54:54,398][105620] Updated weights for policy 1, policy_version 488032 (0.0010) [2023-12-26 18:54:54,982][105692] Updated weights for policy 0, policy_version 487622 (0.0007) [2023-12-26 18:54:55,041][105692] Updated weights for policy 0, policy_version 487632 (0.0009) [2023-12-26 18:54:55,096][105692] Updated weights for policy 0, policy_version 487642 (0.0010) [2023-12-26 18:54:55,168][105620] Updated weights for policy 1, policy_version 488042 (0.0010) [2023-12-26 18:54:55,219][105620] Updated weights for policy 1, policy_version 488052 (0.0008) [2023-12-26 18:54:55,271][105620] Updated weights for policy 1, policy_version 488062 (0.0008) [2023-12-26 18:54:55,340][105620] Updated weights for policy 1, policy_version 488072 (0.0005) [2023-12-26 18:54:55,706][105692] Updated weights for policy 0, policy_version 487652 (0.0010) [2023-12-26 18:54:55,758][105692] Updated weights for policy 0, policy_version 487662 (0.0010) [2023-12-26 18:54:55,812][105692] Updated weights for policy 0, policy_version 487672 (0.0010) [2023-12-26 18:54:55,945][105620] Updated weights for policy 1, policy_version 488082 (0.0009) [2023-12-26 18:54:56,006][105620] Updated weights for policy 1, policy_version 488092 (0.0007) [2023-12-26 18:54:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 249823232. Throughput: 0: 9960.0, 1: 9503.7. Samples: 249833164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:54:56,062][104569] Avg episode reward: [(0, '8995.145'), (1, '9352.807')] [2023-12-26 18:54:56,070][105620] Updated weights for policy 1, policy_version 488102 (0.0005) [2023-12-26 18:54:56,469][105692] Updated weights for policy 0, policy_version 487682 (0.0009) [2023-12-26 18:54:56,523][105692] Updated weights for policy 0, policy_version 487692 (0.0005) [2023-12-26 18:54:56,569][105692] Updated weights for policy 0, policy_version 487702 (0.0005) [2023-12-26 18:54:56,614][105692] Updated weights for policy 0, policy_version 487712 (0.0007) [2023-12-26 18:54:56,616][105620] Updated weights for policy 1, policy_version 488112 (0.0007) [2023-12-26 18:54:56,664][105620] Updated weights for policy 1, policy_version 488122 (0.0008) [2023-12-26 18:54:56,716][105620] Updated weights for policy 1, policy_version 488132 (0.0008) [2023-12-26 18:54:57,289][105692] Updated weights for policy 0, policy_version 487722 (0.0010) [2023-12-26 18:54:57,340][105692] Updated weights for policy 0, policy_version 487732 (0.0006) [2023-12-26 18:54:57,373][105620] Updated weights for policy 1, policy_version 488142 (0.0006) [2023-12-26 18:54:57,408][105692] Updated weights for policy 0, policy_version 487742 (0.0009) [2023-12-26 18:54:57,419][105620] Updated weights for policy 1, policy_version 488152 (0.0005) [2023-12-26 18:54:57,471][105620] Updated weights for policy 1, policy_version 488162 (0.0007) [2023-12-26 18:54:58,008][105692] Updated weights for policy 0, policy_version 487752 (0.0009) [2023-12-26 18:54:58,066][105692] Updated weights for policy 0, policy_version 487762 (0.0010) [2023-12-26 18:54:58,125][105692] Updated weights for policy 0, policy_version 487772 (0.0008) [2023-12-26 18:54:58,295][105620] Updated weights for policy 1, policy_version 488172 (0.0008) [2023-12-26 18:54:58,357][105620] Updated weights for policy 1, policy_version 488182 (0.0008) [2023-12-26 18:54:58,411][105620] Updated weights for policy 1, policy_version 488192 (0.0008) [2023-12-26 18:54:58,837][105692] Updated weights for policy 0, policy_version 487782 (0.0008) [2023-12-26 18:54:58,918][105692] Updated weights for policy 0, policy_version 487792 (0.0008) [2023-12-26 18:54:58,978][105692] Updated weights for policy 0, policy_version 487802 (0.0008) [2023-12-26 18:54:59,247][105620] Updated weights for policy 1, policy_version 488202 (0.0008) [2023-12-26 18:54:59,307][105620] Updated weights for policy 1, policy_version 488212 (0.0009) [2023-12-26 18:54:59,376][105620] Updated weights for policy 1, policy_version 488222 (0.0009) [2023-12-26 18:54:59,421][105620] Updated weights for policy 1, policy_version 488232 (0.0006) [2023-12-26 18:54:59,772][105692] Updated weights for policy 0, policy_version 487812 (0.0008) [2023-12-26 18:54:59,835][105692] Updated weights for policy 0, policy_version 487822 (0.0009) [2023-12-26 18:54:59,896][105692] Updated weights for policy 0, policy_version 487832 (0.0007) [2023-12-26 18:55:00,142][105620] Updated weights for policy 1, policy_version 488242 (0.0008) [2023-12-26 18:55:00,201][105620] Updated weights for policy 1, policy_version 488252 (0.0007) [2023-12-26 18:55:00,248][105620] Updated weights for policy 1, policy_version 488262 (0.0007) [2023-12-26 18:55:00,548][105692] Updated weights for policy 0, policy_version 487842 (0.0006) [2023-12-26 18:55:00,613][105692] Updated weights for policy 0, policy_version 487852 (0.0005) [2023-12-26 18:55:00,616][105585] KL-divergence is very high: 224.2158 [2023-12-26 18:55:00,663][105585] KL-divergence is very high: 367.8998 [2023-12-26 18:55:00,672][105692] Updated weights for policy 0, policy_version 487862 (0.0005) [2023-12-26 18:55:00,706][105585] KL-divergence is very high: 352.4766 [2023-12-26 18:55:00,726][105692] Updated weights for policy 0, policy_version 487872 (0.0006) [2023-12-26 18:55:01,031][105620] Updated weights for policy 1, policy_version 488272 (0.0006) [2023-12-26 18:55:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 249921536. Throughput: 0: 10087.0, 1: 9540.5. Samples: 249894904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:55:01,063][104569] Avg episode reward: [(0, '9175.690'), (1, '9353.115')] [2023-12-26 18:55:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000487872_124911616.pth... [2023-12-26 18:55:01,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000486720_124616704.pth [2023-12-26 18:55:01,099][105620] Updated weights for policy 1, policy_version 488282 (0.0008) [2023-12-26 18:55:01,167][105620] Updated weights for policy 1, policy_version 488292 (0.0008) [2023-12-26 18:55:01,189][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000488296_125018112.pth... [2023-12-26 18:55:01,194][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000487144_124723200.pth [2023-12-26 18:55:01,280][105692] Updated weights for policy 0, policy_version 487882 (0.0009) [2023-12-26 18:55:01,328][105692] Updated weights for policy 0, policy_version 487892 (0.0009) [2023-12-26 18:55:01,382][105692] Updated weights for policy 0, policy_version 487902 (0.0009) [2023-12-26 18:55:01,919][105620] Updated weights for policy 1, policy_version 488302 (0.0007) [2023-12-26 18:55:01,989][105620] Updated weights for policy 1, policy_version 488312 (0.0006) [2023-12-26 18:55:02,048][105620] Updated weights for policy 1, policy_version 488322 (0.0006) [2023-12-26 18:55:02,092][105692] Updated weights for policy 0, policy_version 487912 (0.0006) [2023-12-26 18:55:02,144][105692] Updated weights for policy 0, policy_version 487922 (0.0005) [2023-12-26 18:55:02,207][105692] Updated weights for policy 0, policy_version 487932 (0.0006) [2023-12-26 18:55:02,639][105620] Updated weights for policy 1, policy_version 488332 (0.0006) [2023-12-26 18:55:02,700][105620] Updated weights for policy 1, policy_version 488342 (0.0008) [2023-12-26 18:55:02,749][105620] Updated weights for policy 1, policy_version 488352 (0.0010) [2023-12-26 18:55:02,908][105692] Updated weights for policy 0, policy_version 487942 (0.0006) [2023-12-26 18:55:02,962][105692] Updated weights for policy 0, policy_version 487952 (0.0010) [2023-12-26 18:55:03,020][105692] Updated weights for policy 0, policy_version 487962 (0.0010) [2023-12-26 18:55:03,387][105620] Updated weights for policy 1, policy_version 488362 (0.0009) [2023-12-26 18:55:03,452][105620] Updated weights for policy 1, policy_version 488372 (0.0006) [2023-12-26 18:55:03,515][105620] Updated weights for policy 1, policy_version 488382 (0.0009) [2023-12-26 18:55:03,566][105620] Updated weights for policy 1, policy_version 488392 (0.0010) [2023-12-26 18:55:03,657][105692] Updated weights for policy 0, policy_version 487972 (0.0007) [2023-12-26 18:55:03,721][105692] Updated weights for policy 0, policy_version 487982 (0.0005) [2023-12-26 18:55:03,784][105692] Updated weights for policy 0, policy_version 487992 (0.0005) [2023-12-26 18:55:04,265][105620] Updated weights for policy 1, policy_version 488402 (0.0008) [2023-12-26 18:55:04,313][105692] Updated weights for policy 0, policy_version 488002 (0.0007) [2023-12-26 18:55:04,320][105620] Updated weights for policy 1, policy_version 488412 (0.0008) [2023-12-26 18:55:04,367][105620] Updated weights for policy 1, policy_version 488422 (0.0007) [2023-12-26 18:55:04,373][105692] Updated weights for policy 0, policy_version 488012 (0.0011) [2023-12-26 18:55:04,435][105692] Updated weights for policy 0, policy_version 488022 (0.0010) [2023-12-26 18:55:04,494][105692] Updated weights for policy 0, policy_version 488032 (0.0011) [2023-12-26 18:55:05,180][105620] Updated weights for policy 1, policy_version 488432 (0.0008) [2023-12-26 18:55:05,211][105692] Updated weights for policy 0, policy_version 488042 (0.0010) [2023-12-26 18:55:05,229][105620] Updated weights for policy 1, policy_version 488442 (0.0007) [2023-12-26 18:55:05,266][105692] Updated weights for policy 0, policy_version 488052 (0.0010) [2023-12-26 18:55:05,273][105620] Updated weights for policy 1, policy_version 488452 (0.0005) [2023-12-26 18:55:05,317][105692] Updated weights for policy 0, policy_version 488062 (0.0011) [2023-12-26 18:55:05,941][105692] Updated weights for policy 0, policy_version 488072 (0.0006) [2023-12-26 18:55:06,003][105692] Updated weights for policy 0, policy_version 488082 (0.0005) [2023-12-26 18:55:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 250019840. Throughput: 0: 10141.9, 1: 9616.1. Samples: 250015600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:55:06,062][104569] Avg episode reward: [(0, '9082.870'), (1, '9351.916')] [2023-12-26 18:55:06,067][105620] Updated weights for policy 1, policy_version 488462 (0.0006) [2023-12-26 18:55:06,068][105692] Updated weights for policy 0, policy_version 488092 (0.0005) [2023-12-26 18:55:06,133][105620] Updated weights for policy 1, policy_version 488472 (0.0007) [2023-12-26 18:55:06,198][105620] Updated weights for policy 1, policy_version 488482 (0.0007) [2023-12-26 18:55:06,724][105692] Updated weights for policy 0, policy_version 488102 (0.0009) [2023-12-26 18:55:06,788][105692] Updated weights for policy 0, policy_version 488112 (0.0008) [2023-12-26 18:55:06,795][105620] Updated weights for policy 1, policy_version 488492 (0.0006) [2023-12-26 18:55:06,852][105692] Updated weights for policy 0, policy_version 488122 (0.0009) [2023-12-26 18:55:06,853][105620] Updated weights for policy 1, policy_version 488502 (0.0006) [2023-12-26 18:55:06,911][105620] Updated weights for policy 1, policy_version 488512 (0.0009) [2023-12-26 18:55:07,541][105692] Updated weights for policy 0, policy_version 488132 (0.0011) [2023-12-26 18:55:07,594][105692] Updated weights for policy 0, policy_version 488142 (0.0011) [2023-12-26 18:55:07,654][105620] Updated weights for policy 1, policy_version 488522 (0.0007) [2023-12-26 18:55:07,656][105692] Updated weights for policy 0, policy_version 488152 (0.0010) [2023-12-26 18:55:07,705][105620] Updated weights for policy 1, policy_version 488532 (0.0006) [2023-12-26 18:55:07,755][105620] Updated weights for policy 1, policy_version 488542 (0.0008) [2023-12-26 18:55:07,803][105620] Updated weights for policy 1, policy_version 488552 (0.0008) [2023-12-26 18:55:08,339][105692] Updated weights for policy 0, policy_version 488162 (0.0010) [2023-12-26 18:55:08,396][105692] Updated weights for policy 0, policy_version 488172 (0.0011) [2023-12-26 18:55:08,457][105692] Updated weights for policy 0, policy_version 488182 (0.0011) [2023-12-26 18:55:08,513][105692] Updated weights for policy 0, policy_version 488192 (0.0009) [2023-12-26 18:55:08,630][105620] Updated weights for policy 1, policy_version 488562 (0.0008) [2023-12-26 18:55:08,693][105620] Updated weights for policy 1, policy_version 488572 (0.0008) [2023-12-26 18:55:08,757][105620] Updated weights for policy 1, policy_version 488582 (0.0008) [2023-12-26 18:55:09,191][105692] Updated weights for policy 0, policy_version 488202 (0.0005) [2023-12-26 18:55:09,257][105692] Updated weights for policy 0, policy_version 488212 (0.0007) [2023-12-26 18:55:09,321][105692] Updated weights for policy 0, policy_version 488222 (0.0006) [2023-12-26 18:55:09,513][105620] Updated weights for policy 1, policy_version 488592 (0.0008) [2023-12-26 18:55:09,572][105620] Updated weights for policy 1, policy_version 488602 (0.0008) [2023-12-26 18:55:09,624][105620] Updated weights for policy 1, policy_version 488612 (0.0008) [2023-12-26 18:55:10,035][105692] Updated weights for policy 0, policy_version 488232 (0.0010) [2023-12-26 18:55:10,091][105692] Updated weights for policy 0, policy_version 488242 (0.0011) [2023-12-26 18:55:10,148][105692] Updated weights for policy 0, policy_version 488252 (0.0011) [2023-12-26 18:55:10,305][105620] Updated weights for policy 1, policy_version 488622 (0.0008) [2023-12-26 18:55:10,373][105620] Updated weights for policy 1, policy_version 488632 (0.0009) [2023-12-26 18:55:10,446][105620] Updated weights for policy 1, policy_version 488642 (0.0008) [2023-12-26 18:55:10,811][105692] Updated weights for policy 0, policy_version 488262 (0.0008) [2023-12-26 18:55:10,866][105692] Updated weights for policy 0, policy_version 488272 (0.0009) [2023-12-26 18:55:10,925][105692] Updated weights for policy 0, policy_version 488282 (0.0008) [2023-12-26 18:55:11,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 250126336. Throughput: 0: 10175.7, 1: 9666.1. Samples: 250133796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:55:11,063][104569] Avg episode reward: [(0, '9172.849'), (1, '9351.086')] [2023-12-26 18:55:11,108][105620] Updated weights for policy 1, policy_version 488652 (0.0009) [2023-12-26 18:55:11,176][105620] Updated weights for policy 1, policy_version 488662 (0.0007) [2023-12-26 18:55:11,248][105620] Updated weights for policy 1, policy_version 488672 (0.0008) [2023-12-26 18:55:11,694][105692] Updated weights for policy 0, policy_version 488292 (0.0009) [2023-12-26 18:55:11,762][105692] Updated weights for policy 0, policy_version 488302 (0.0009) [2023-12-26 18:55:11,817][105692] Updated weights for policy 0, policy_version 488312 (0.0009) [2023-12-26 18:55:11,935][105620] Updated weights for policy 1, policy_version 488682 (0.0007) [2023-12-26 18:55:11,990][105620] Updated weights for policy 1, policy_version 488692 (0.0008) [2023-12-26 18:55:12,052][105620] Updated weights for policy 1, policy_version 488702 (0.0008) [2023-12-26 18:55:12,105][105620] Updated weights for policy 1, policy_version 488712 (0.0009) [2023-12-26 18:55:12,642][105692] Updated weights for policy 0, policy_version 488322 (0.0009) [2023-12-26 18:55:12,700][105692] Updated weights for policy 0, policy_version 488332 (0.0009) [2023-12-26 18:55:12,760][105692] Updated weights for policy 0, policy_version 488342 (0.0010) [2023-12-26 18:55:12,822][105692] Updated weights for policy 0, policy_version 488352 (0.0008) [2023-12-26 18:55:12,858][105620] Updated weights for policy 1, policy_version 488722 (0.0008) [2023-12-26 18:55:12,907][105620] Updated weights for policy 1, policy_version 488732 (0.0008) [2023-12-26 18:55:12,960][105620] Updated weights for policy 1, policy_version 488742 (0.0009) [2023-12-26 18:55:13,429][105692] Updated weights for policy 0, policy_version 488362 (0.0005) [2023-12-26 18:55:13,481][105692] Updated weights for policy 0, policy_version 488372 (0.0007) [2023-12-26 18:55:13,537][105692] Updated weights for policy 0, policy_version 488382 (0.0011) [2023-12-26 18:55:13,821][105620] Updated weights for policy 1, policy_version 488752 (0.0008) [2023-12-26 18:55:13,878][105620] Updated weights for policy 1, policy_version 488762 (0.0008) [2023-12-26 18:55:13,938][105620] Updated weights for policy 1, policy_version 488772 (0.0009) [2023-12-26 18:55:14,270][105692] Updated weights for policy 0, policy_version 488392 (0.0010) [2023-12-26 18:55:14,332][105692] Updated weights for policy 0, policy_version 488402 (0.0010) [2023-12-26 18:55:14,390][105692] Updated weights for policy 0, policy_version 488412 (0.0010) [2023-12-26 18:55:14,707][105620] Updated weights for policy 1, policy_version 488782 (0.0009) [2023-12-26 18:55:14,771][105620] Updated weights for policy 1, policy_version 488792 (0.0008) [2023-12-26 18:55:14,840][105620] Updated weights for policy 1, policy_version 488802 (0.0006) [2023-12-26 18:55:15,161][105692] Updated weights for policy 0, policy_version 488422 (0.0011) [2023-12-26 18:55:15,228][105692] Updated weights for policy 0, policy_version 488432 (0.0009) [2023-12-26 18:55:15,291][105692] Updated weights for policy 0, policy_version 488442 (0.0005) [2023-12-26 18:55:15,573][105620] Updated weights for policy 1, policy_version 488812 (0.0008) [2023-12-26 18:55:15,626][105620] Updated weights for policy 1, policy_version 488822 (0.0009) [2023-12-26 18:55:15,693][105620] Updated weights for policy 1, policy_version 488832 (0.0010) [2023-12-26 18:55:15,805][105692] Updated weights for policy 0, policy_version 488452 (0.0005) [2023-12-26 18:55:15,869][105692] Updated weights for policy 0, policy_version 488462 (0.0006) [2023-12-26 18:55:15,930][105692] Updated weights for policy 0, policy_version 488472 (0.0009) [2023-12-26 18:55:16,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 250224640. Throughput: 0: 10111.9, 1: 9649.4. Samples: 250190104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:55:16,062][104569] Avg episode reward: [(0, '8632.682'), (1, '9351.563')] [2023-12-26 18:55:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000488840_125157376.pth... [2023-12-26 18:55:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000488480_125067264.pth... [2023-12-26 18:55:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000487752_124878848.pth [2023-12-26 18:55:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000487264_124755968.pth [2023-12-26 18:55:16,499][105692] Updated weights for policy 0, policy_version 488482 (0.0010) [2023-12-26 18:55:16,552][105620] Updated weights for policy 1, policy_version 488842 (0.0009) [2023-12-26 18:55:16,555][105692] Updated weights for policy 0, policy_version 488492 (0.0010) [2023-12-26 18:55:16,606][105692] Updated weights for policy 0, policy_version 488502 (0.0010) [2023-12-26 18:55:16,612][105620] Updated weights for policy 1, policy_version 488852 (0.0006) [2023-12-26 18:55:16,657][105692] Updated weights for policy 0, policy_version 488512 (0.0009) [2023-12-26 18:55:16,666][105620] Updated weights for policy 1, policy_version 488862 (0.0007) [2023-12-26 18:55:16,724][105620] Updated weights for policy 1, policy_version 488872 (0.0009) [2023-12-26 18:55:17,383][105692] Updated weights for policy 0, policy_version 488522 (0.0010) [2023-12-26 18:55:17,388][105620] Updated weights for policy 1, policy_version 488882 (0.0010) [2023-12-26 18:55:17,434][105692] Updated weights for policy 0, policy_version 488532 (0.0010) [2023-12-26 18:55:17,440][105620] Updated weights for policy 1, policy_version 488892 (0.0010) [2023-12-26 18:55:17,482][105692] Updated weights for policy 0, policy_version 488542 (0.0010) [2023-12-26 18:55:17,495][105620] Updated weights for policy 1, policy_version 488902 (0.0010) [2023-12-26 18:55:18,233][105620] Updated weights for policy 1, policy_version 488912 (0.0007) [2023-12-26 18:55:18,234][105692] Updated weights for policy 0, policy_version 488552 (0.0010) [2023-12-26 18:55:18,262][105585] KL-divergence is very high: 249.8914 [2023-12-26 18:55:18,281][105620] Updated weights for policy 1, policy_version 488922 (0.0008) [2023-12-26 18:55:18,290][105692] Updated weights for policy 0, policy_version 488562 (0.0009) [2023-12-26 18:55:18,299][105585] KL-divergence is very high: 396.7020 [2023-12-26 18:55:18,327][105620] Updated weights for policy 1, policy_version 488932 (0.0009) [2023-12-26 18:55:18,352][105585] KL-divergence is very high: 351.7050 [2023-12-26 18:55:18,353][105692] Updated weights for policy 0, policy_version 488572 (0.0007) [2023-12-26 18:55:19,031][105620] Updated weights for policy 1, policy_version 488942 (0.0007) [2023-12-26 18:55:19,061][105692] Updated weights for policy 0, policy_version 488582 (0.0010) [2023-12-26 18:55:19,084][105620] Updated weights for policy 1, policy_version 488952 (0.0006) [2023-12-26 18:55:19,123][105692] Updated weights for policy 0, policy_version 488592 (0.0010) [2023-12-26 18:55:19,148][105620] Updated weights for policy 1, policy_version 488962 (0.0009) [2023-12-26 18:55:19,188][105692] Updated weights for policy 0, policy_version 488602 (0.0010) [2023-12-26 18:55:19,834][105620] Updated weights for policy 1, policy_version 488972 (0.0007) [2023-12-26 18:55:19,895][105620] Updated weights for policy 1, policy_version 488982 (0.0008) [2023-12-26 18:55:19,951][105692] Updated weights for policy 0, policy_version 488612 (0.0010) [2023-12-26 18:55:19,959][105620] Updated weights for policy 1, policy_version 488992 (0.0008) [2023-12-26 18:55:20,008][105692] Updated weights for policy 0, policy_version 488622 (0.0009) [2023-12-26 18:55:20,014][105585] KL-divergence is very high: 103.2569 [2023-12-26 18:55:20,059][105585] KL-divergence is very high: 140.6642 [2023-12-26 18:55:20,066][105692] Updated weights for policy 0, policy_version 488632 (0.0010) [2023-12-26 18:55:20,621][105620] Updated weights for policy 1, policy_version 489002 (0.0007) [2023-12-26 18:55:20,678][105620] Updated weights for policy 1, policy_version 489012 (0.0010) [2023-12-26 18:55:20,734][105620] Updated weights for policy 1, policy_version 489022 (0.0011) [2023-12-26 18:55:20,805][105620] Updated weights for policy 1, policy_version 489032 (0.0011) [2023-12-26 18:55:20,892][105692] Updated weights for policy 0, policy_version 488642 (0.0008) [2023-12-26 18:55:20,956][105692] Updated weights for policy 0, policy_version 488652 (0.0008) [2023-12-26 18:55:21,024][105692] Updated weights for policy 0, policy_version 488662 (0.0008) [2023-12-26 18:55:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 250314752. Throughput: 0: 10086.3, 1: 9610.9. Samples: 250307704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:55:21,063][104569] Avg episode reward: [(0, '7855.469'), (1, '9260.276')] [2023-12-26 18:55:21,090][105692] Updated weights for policy 0, policy_version 488672 (0.0008) [2023-12-26 18:55:21,622][105620] Updated weights for policy 1, policy_version 489042 (0.0009) [2023-12-26 18:55:21,681][105620] Updated weights for policy 1, policy_version 489052 (0.0009) [2023-12-26 18:55:21,747][105620] Updated weights for policy 1, policy_version 489062 (0.0009) [2023-12-26 18:55:21,822][105692] Updated weights for policy 0, policy_version 488682 (0.0009) [2023-12-26 18:55:21,890][105692] Updated weights for policy 0, policy_version 488692 (0.0010) [2023-12-26 18:55:21,963][105692] Updated weights for policy 0, policy_version 488702 (0.0010) [2023-12-26 18:55:22,510][105620] Updated weights for policy 1, policy_version 489072 (0.0007) [2023-12-26 18:55:22,581][105620] Updated weights for policy 1, policy_version 489082 (0.0008) [2023-12-26 18:55:22,648][105620] Updated weights for policy 1, policy_version 489092 (0.0008) [2023-12-26 18:55:22,754][105692] Updated weights for policy 0, policy_version 488712 (0.0010) [2023-12-26 18:55:22,812][105692] Updated weights for policy 0, policy_version 488722 (0.0011) [2023-12-26 18:55:22,869][105692] Updated weights for policy 0, policy_version 488732 (0.0008) [2023-12-26 18:55:23,293][105620] Updated weights for policy 1, policy_version 489102 (0.0010) [2023-12-26 18:55:23,355][105620] Updated weights for policy 1, policy_version 489112 (0.0007) [2023-12-26 18:55:23,416][105620] Updated weights for policy 1, policy_version 489122 (0.0005) [2023-12-26 18:55:23,706][105692] Updated weights for policy 0, policy_version 488742 (0.0009) [2023-12-26 18:55:23,758][105692] Updated weights for policy 0, policy_version 488752 (0.0008) [2023-12-26 18:55:23,805][105692] Updated weights for policy 0, policy_version 488762 (0.0008) [2023-12-26 18:55:23,980][105620] Updated weights for policy 1, policy_version 489132 (0.0005) [2023-12-26 18:55:24,039][105620] Updated weights for policy 1, policy_version 489142 (0.0005) [2023-12-26 18:55:24,097][105620] Updated weights for policy 1, policy_version 489152 (0.0005) [2023-12-26 18:55:24,499][105692] Updated weights for policy 0, policy_version 488772 (0.0007) [2023-12-26 18:55:24,564][105692] Updated weights for policy 0, policy_version 488782 (0.0006) [2023-12-26 18:55:24,626][105692] Updated weights for policy 0, policy_version 488792 (0.0008) [2023-12-26 18:55:24,781][105620] Updated weights for policy 1, policy_version 489162 (0.0007) [2023-12-26 18:55:24,828][105620] Updated weights for policy 1, policy_version 489172 (0.0010) [2023-12-26 18:55:24,882][105620] Updated weights for policy 1, policy_version 489182 (0.0010) [2023-12-26 18:55:24,939][105620] Updated weights for policy 1, policy_version 489192 (0.0010) [2023-12-26 18:55:25,358][105692] Updated weights for policy 0, policy_version 488802 (0.0008) [2023-12-26 18:55:25,412][105692] Updated weights for policy 0, policy_version 488813 (0.0009) [2023-12-26 18:55:25,470][105692] Updated weights for policy 0, policy_version 488823 (0.0010) [2023-12-26 18:55:25,545][105620] Updated weights for policy 1, policy_version 489202 (0.0010) [2023-12-26 18:55:25,592][105620] Updated weights for policy 1, policy_version 489212 (0.0010) [2023-12-26 18:55:25,647][105620] Updated weights for policy 1, policy_version 489222 (0.0010) [2023-12-26 18:55:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 250413056. Throughput: 0: 9978.6, 1: 9612.5. Samples: 250422312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:55:26,064][104569] Avg episode reward: [(0, '7950.895'), (1, '9260.579')] [2023-12-26 18:55:26,309][105692] Updated weights for policy 0, policy_version 488834 (0.0007) [2023-12-26 18:55:26,315][105620] Updated weights for policy 1, policy_version 489232 (0.0006) [2023-12-26 18:55:26,367][105692] Updated weights for policy 0, policy_version 488844 (0.0005) [2023-12-26 18:55:26,372][105620] Updated weights for policy 1, policy_version 489242 (0.0010) [2023-12-26 18:55:26,428][105692] Updated weights for policy 0, policy_version 488854 (0.0006) [2023-12-26 18:55:26,433][105620] Updated weights for policy 1, policy_version 489252 (0.0010) [2023-12-26 18:55:26,484][105692] Updated weights for policy 0, policy_version 488864 (0.0008) [2023-12-26 18:55:27,127][105620] Updated weights for policy 1, policy_version 489262 (0.0009) [2023-12-26 18:55:27,175][105692] Updated weights for policy 0, policy_version 488874 (0.0009) [2023-12-26 18:55:27,177][105620] Updated weights for policy 1, policy_version 489272 (0.0005) [2023-12-26 18:55:27,222][105620] Updated weights for policy 1, policy_version 489282 (0.0005) [2023-12-26 18:55:27,225][105692] Updated weights for policy 0, policy_version 488884 (0.0009) [2023-12-26 18:55:27,269][105692] Updated weights for policy 0, policy_version 488894 (0.0007) [2023-12-26 18:55:27,896][105620] Updated weights for policy 1, policy_version 489292 (0.0009) [2023-12-26 18:55:27,946][105620] Updated weights for policy 1, policy_version 489302 (0.0010) [2023-12-26 18:55:28,000][105620] Updated weights for policy 1, policy_version 489312 (0.0010) [2023-12-26 18:55:28,055][105692] Updated weights for policy 0, policy_version 488904 (0.0007) [2023-12-26 18:55:28,109][105692] Updated weights for policy 0, policy_version 488914 (0.0008) [2023-12-26 18:55:28,159][105692] Updated weights for policy 0, policy_version 488924 (0.0008) [2023-12-26 18:55:28,750][105620] Updated weights for policy 1, policy_version 489322 (0.0010) [2023-12-26 18:55:28,807][105620] Updated weights for policy 1, policy_version 489332 (0.0010) [2023-12-26 18:55:28,865][105620] Updated weights for policy 1, policy_version 489342 (0.0010) [2023-12-26 18:55:28,924][105692] Updated weights for policy 0, policy_version 488934 (0.0007) [2023-12-26 18:55:28,928][105620] Updated weights for policy 1, policy_version 489352 (0.0010) [2023-12-26 18:55:28,979][105692] Updated weights for policy 0, policy_version 488944 (0.0008) [2023-12-26 18:55:29,039][105692] Updated weights for policy 0, policy_version 488954 (0.0008) [2023-12-26 18:55:29,613][105620] Updated weights for policy 1, policy_version 489362 (0.0005) [2023-12-26 18:55:29,682][105620] Updated weights for policy 1, policy_version 489372 (0.0007) [2023-12-26 18:55:29,743][105620] Updated weights for policy 1, policy_version 489382 (0.0009) [2023-12-26 18:55:29,847][105692] Updated weights for policy 0, policy_version 488964 (0.0008) [2023-12-26 18:55:29,913][105692] Updated weights for policy 0, policy_version 488974 (0.0008) [2023-12-26 18:55:29,973][105692] Updated weights for policy 0, policy_version 488984 (0.0009) [2023-12-26 18:55:30,467][105620] Updated weights for policy 1, policy_version 489392 (0.0009) [2023-12-26 18:55:30,517][105620] Updated weights for policy 1, policy_version 489402 (0.0009) [2023-12-26 18:55:30,567][105620] Updated weights for policy 1, policy_version 489412 (0.0009) [2023-12-26 18:55:30,613][105692] Updated weights for policy 0, policy_version 488994 (0.0009) [2023-12-26 18:55:30,674][105692] Updated weights for policy 0, policy_version 489004 (0.0009) [2023-12-26 18:55:30,732][105692] Updated weights for policy 0, policy_version 489014 (0.0009) [2023-12-26 18:55:30,794][105692] Updated weights for policy 0, policy_version 489024 (0.0009) [2023-12-26 18:55:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 250511360. Throughput: 0: 9955.7, 1: 9650.1. Samples: 250480708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:55:31,063][104569] Avg episode reward: [(0, '8029.127'), (1, '9353.256')] [2023-12-26 18:55:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000489024_125206528.pth... [2023-12-26 18:55:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000489416_125304832.pth... [2023-12-26 18:55:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000487872_124911616.pth [2023-12-26 18:55:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000488296_125018112.pth [2023-12-26 18:55:31,256][105620] Updated weights for policy 1, policy_version 489422 (0.0009) [2023-12-26 18:55:31,303][105620] Updated weights for policy 1, policy_version 489432 (0.0008) [2023-12-26 18:55:31,370][105620] Updated weights for policy 1, policy_version 489442 (0.0008) [2023-12-26 18:55:31,479][105692] Updated weights for policy 0, policy_version 489035 (0.0010) [2023-12-26 18:55:31,531][105692] Updated weights for policy 0, policy_version 489045 (0.0009) [2023-12-26 18:55:31,591][105692] Updated weights for policy 0, policy_version 489055 (0.0009) [2023-12-26 18:55:32,012][105620] Updated weights for policy 1, policy_version 489452 (0.0007) [2023-12-26 18:55:32,079][105620] Updated weights for policy 1, policy_version 489462 (0.0007) [2023-12-26 18:55:32,140][105620] Updated weights for policy 1, policy_version 489472 (0.0008) [2023-12-26 18:55:32,443][105692] Updated weights for policy 0, policy_version 489065 (0.0009) [2023-12-26 18:55:32,500][105692] Updated weights for policy 0, policy_version 489075 (0.0008) [2023-12-26 18:55:32,561][105692] Updated weights for policy 0, policy_version 489085 (0.0006) [2023-12-26 18:55:32,749][105620] Updated weights for policy 1, policy_version 489482 (0.0007) [2023-12-26 18:55:32,819][105620] Updated weights for policy 1, policy_version 489492 (0.0005) [2023-12-26 18:55:32,890][105620] Updated weights for policy 1, policy_version 489502 (0.0005) [2023-12-26 18:55:32,957][105620] Updated weights for policy 1, policy_version 489512 (0.0006) [2023-12-26 18:55:33,380][105692] Updated weights for policy 0, policy_version 489095 (0.0010) [2023-12-26 18:55:33,434][105692] Updated weights for policy 0, policy_version 489105 (0.0007) [2023-12-26 18:55:33,461][105620] Updated weights for policy 1, policy_version 489522 (0.0005) [2023-12-26 18:55:33,491][105692] Updated weights for policy 0, policy_version 489115 (0.0006) [2023-12-26 18:55:33,513][105620] Updated weights for policy 1, policy_version 489532 (0.0005) [2023-12-26 18:55:33,561][105620] Updated weights for policy 1, policy_version 489542 (0.0005) [2023-12-26 18:55:34,106][105620] Updated weights for policy 1, policy_version 489552 (0.0005) [2023-12-26 18:55:34,175][105620] Updated weights for policy 1, policy_version 489562 (0.0007) [2023-12-26 18:55:34,234][105620] Updated weights for policy 1, policy_version 489572 (0.0006) [2023-12-26 18:55:34,339][105692] Updated weights for policy 0, policy_version 489125 (0.0008) [2023-12-26 18:55:34,398][105692] Updated weights for policy 0, policy_version 489135 (0.0008) [2023-12-26 18:55:34,462][105692] Updated weights for policy 0, policy_version 489145 (0.0008) [2023-12-26 18:55:34,933][105620] Updated weights for policy 1, policy_version 489582 (0.0008) [2023-12-26 18:55:34,993][105620] Updated weights for policy 1, policy_version 489592 (0.0011) [2023-12-26 18:55:35,011][105586] KL-divergence is very high: 202.2650 [2023-12-26 18:55:35,049][105620] Updated weights for policy 1, policy_version 489602 (0.0011) [2023-12-26 18:55:35,054][105586] KL-divergence is very high: 244.2106 [2023-12-26 18:55:35,200][105692] Updated weights for policy 0, policy_version 489155 (0.0009) [2023-12-26 18:55:35,254][105692] Updated weights for policy 0, policy_version 489165 (0.0010) [2023-12-26 18:55:35,304][105692] Updated weights for policy 0, policy_version 489175 (0.0005) [2023-12-26 18:55:35,790][105620] Updated weights for policy 1, policy_version 489612 (0.0009) [2023-12-26 18:55:35,844][105620] Updated weights for policy 1, policy_version 489622 (0.0008) [2023-12-26 18:55:35,894][105620] Updated weights for policy 1, policy_version 489632 (0.0006) [2023-12-26 18:55:35,995][105692] Updated weights for policy 0, policy_version 489185 (0.0005) [2023-12-26 18:55:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.9, 300 sec: 19466.4). Total num frames: 250609664. Throughput: 0: 9864.2, 1: 9767.6. Samples: 250599012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:55:36,063][104569] Avg episode reward: [(0, '8459.851'), (1, '9353.710')] [2023-12-26 18:55:36,065][105692] Updated weights for policy 0, policy_version 489195 (0.0005) [2023-12-26 18:55:36,126][105692] Updated weights for policy 0, policy_version 489205 (0.0008) [2023-12-26 18:55:36,182][105692] Updated weights for policy 0, policy_version 489215 (0.0007) [2023-12-26 18:55:36,497][105620] Updated weights for policy 1, policy_version 489642 (0.0007) [2023-12-26 18:55:36,563][105620] Updated weights for policy 1, policy_version 489652 (0.0011) [2023-12-26 18:55:36,626][105620] Updated weights for policy 1, policy_version 489662 (0.0011) [2023-12-26 18:55:36,690][105620] Updated weights for policy 1, policy_version 489672 (0.0010) [2023-12-26 18:55:36,833][105692] Updated weights for policy 0, policy_version 489225 (0.0006) [2023-12-26 18:55:36,885][105692] Updated weights for policy 0, policy_version 489235 (0.0008) [2023-12-26 18:55:36,940][105692] Updated weights for policy 0, policy_version 489245 (0.0008) [2023-12-26 18:55:37,381][105620] Updated weights for policy 1, policy_version 489682 (0.0005) [2023-12-26 18:55:37,449][105620] Updated weights for policy 1, policy_version 489692 (0.0009) [2023-12-26 18:55:37,514][105620] Updated weights for policy 1, policy_version 489702 (0.0010) [2023-12-26 18:55:37,598][105692] Updated weights for policy 0, policy_version 489255 (0.0006) [2023-12-26 18:55:37,662][105692] Updated weights for policy 0, policy_version 489265 (0.0005) [2023-12-26 18:55:37,734][105692] Updated weights for policy 0, policy_version 489275 (0.0005) [2023-12-26 18:55:38,152][105620] Updated weights for policy 1, policy_version 489712 (0.0010) [2023-12-26 18:55:38,203][105620] Updated weights for policy 1, policy_version 489722 (0.0010) [2023-12-26 18:55:38,238][105692] Updated weights for policy 0, policy_version 489285 (0.0006) [2023-12-26 18:55:38,250][105620] Updated weights for policy 1, policy_version 489732 (0.0010) [2023-12-26 18:55:38,292][105692] Updated weights for policy 0, policy_version 489295 (0.0005) [2023-12-26 18:55:38,357][105692] Updated weights for policy 0, policy_version 489305 (0.0007) [2023-12-26 18:55:38,898][105620] Updated weights for policy 1, policy_version 489742 (0.0007) [2023-12-26 18:55:38,955][105620] Updated weights for policy 1, policy_version 489752 (0.0005) [2023-12-26 18:55:39,012][105620] Updated weights for policy 1, policy_version 489762 (0.0005) [2023-12-26 18:55:39,106][105692] Updated weights for policy 0, policy_version 489315 (0.0007) [2023-12-26 18:55:39,159][105692] Updated weights for policy 0, policy_version 489325 (0.0008) [2023-12-26 18:55:39,225][105692] Updated weights for policy 0, policy_version 489335 (0.0008) [2023-12-26 18:55:39,774][105620] Updated weights for policy 1, policy_version 489772 (0.0008) [2023-12-26 18:55:39,838][105620] Updated weights for policy 1, policy_version 489782 (0.0009) [2023-12-26 18:55:39,873][105692] Updated weights for policy 0, policy_version 489345 (0.0008) [2023-12-26 18:55:39,905][105620] Updated weights for policy 1, policy_version 489792 (0.0009) [2023-12-26 18:55:39,941][105692] Updated weights for policy 0, policy_version 489355 (0.0008) [2023-12-26 18:55:39,999][105692] Updated weights for policy 0, policy_version 489365 (0.0010) [2023-12-26 18:55:40,057][105692] Updated weights for policy 0, policy_version 489375 (0.0009) [2023-12-26 18:55:40,604][105620] Updated weights for policy 1, policy_version 489802 (0.0006) [2023-12-26 18:55:40,661][105620] Updated weights for policy 1, policy_version 489812 (0.0005) [2023-12-26 18:55:40,719][105620] Updated weights for policy 1, policy_version 489822 (0.0005) [2023-12-26 18:55:40,782][105620] Updated weights for policy 1, policy_version 489832 (0.0008) [2023-12-26 18:55:40,892][105692] Updated weights for policy 0, policy_version 489385 (0.0010) [2023-12-26 18:55:40,950][105692] Updated weights for policy 0, policy_version 489395 (0.0010) [2023-12-26 18:55:41,007][105692] Updated weights for policy 0, policy_version 489405 (0.0009) [2023-12-26 18:55:41,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 250716160. Throughput: 0: 9822.4, 1: 9913.2. Samples: 250721268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:55:41,062][104569] Avg episode reward: [(0, '8548.724'), (1, '9261.126')] [2023-12-26 18:55:41,470][105620] Updated weights for policy 1, policy_version 489842 (0.0008) [2023-12-26 18:55:41,532][105620] Updated weights for policy 1, policy_version 489852 (0.0009) [2023-12-26 18:55:41,602][105620] Updated weights for policy 1, policy_version 489862 (0.0009) [2023-12-26 18:55:41,793][105692] Updated weights for policy 0, policy_version 489415 (0.0007) [2023-12-26 18:55:41,846][105692] Updated weights for policy 0, policy_version 489425 (0.0008) [2023-12-26 18:55:41,899][105692] Updated weights for policy 0, policy_version 489435 (0.0010) [2023-12-26 18:55:42,325][105620] Updated weights for policy 1, policy_version 489872 (0.0009) [2023-12-26 18:55:42,389][105620] Updated weights for policy 1, policy_version 489882 (0.0008) [2023-12-26 18:55:42,444][105620] Updated weights for policy 1, policy_version 489892 (0.0009) [2023-12-26 18:55:42,738][105692] Updated weights for policy 0, policy_version 489445 (0.0010) [2023-12-26 18:55:42,807][105692] Updated weights for policy 0, policy_version 489455 (0.0007) [2023-12-26 18:55:42,871][105692] Updated weights for policy 0, policy_version 489465 (0.0008) [2023-12-26 18:55:43,073][105620] Updated weights for policy 1, policy_version 489902 (0.0007) [2023-12-26 18:55:43,133][105620] Updated weights for policy 1, policy_version 489912 (0.0006) [2023-12-26 18:55:43,192][105620] Updated weights for policy 1, policy_version 489922 (0.0005) [2023-12-26 18:55:43,681][105692] Updated weights for policy 0, policy_version 489475 (0.0008) [2023-12-26 18:55:43,736][105692] Updated weights for policy 0, policy_version 489485 (0.0006) [2023-12-26 18:55:43,739][105620] Updated weights for policy 1, policy_version 489932 (0.0005) [2023-12-26 18:55:43,792][105692] Updated weights for policy 0, policy_version 489495 (0.0008) [2023-12-26 18:55:43,792][105620] Updated weights for policy 1, policy_version 489942 (0.0007) [2023-12-26 18:55:43,840][105620] Updated weights for policy 1, policy_version 489952 (0.0010) [2023-12-26 18:55:44,497][105620] Updated weights for policy 1, policy_version 489962 (0.0010) [2023-12-26 18:55:44,526][105692] Updated weights for policy 0, policy_version 489505 (0.0007) [2023-12-26 18:55:44,562][105620] Updated weights for policy 1, policy_version 489972 (0.0006) [2023-12-26 18:55:44,590][105692] Updated weights for policy 0, policy_version 489515 (0.0008) [2023-12-26 18:55:44,622][105620] Updated weights for policy 1, policy_version 489982 (0.0006) [2023-12-26 18:55:44,649][105692] Updated weights for policy 0, policy_version 489525 (0.0008) [2023-12-26 18:55:44,674][105620] Updated weights for policy 1, policy_version 489992 (0.0011) [2023-12-26 18:55:44,706][105692] Updated weights for policy 0, policy_version 489535 (0.0007) [2023-12-26 18:55:45,415][105692] Updated weights for policy 0, policy_version 489545 (0.0006) [2023-12-26 18:55:45,456][105620] Updated weights for policy 1, policy_version 490002 (0.0009) [2023-12-26 18:55:45,475][105692] Updated weights for policy 0, policy_version 489555 (0.0006) [2023-12-26 18:55:45,509][105620] Updated weights for policy 1, policy_version 490012 (0.0007) [2023-12-26 18:55:45,540][105692] Updated weights for policy 0, policy_version 489565 (0.0008) [2023-12-26 18:55:45,573][105620] Updated weights for policy 1, policy_version 490022 (0.0008) [2023-12-26 18:55:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 250806272. Throughput: 0: 9682.9, 1: 9946.3. Samples: 250778216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:55:46,062][104569] Avg episode reward: [(0, '8637.240'), (1, '9167.360')] [2023-12-26 18:55:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000490024_125460480.pth... [2023-12-26 18:55:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000488840_125157376.pth [2023-12-26 18:55:46,099][105692] Updated weights for policy 0, policy_version 489575 (0.0008) [2023-12-26 18:55:46,149][105692] Updated weights for policy 0, policy_version 489585 (0.0009) [2023-12-26 18:55:46,200][105692] Updated weights for policy 0, policy_version 489595 (0.0009) [2023-12-26 18:55:46,223][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000489600_125353984.pth... [2023-12-26 18:55:46,226][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000488480_125067264.pth [2023-12-26 18:55:46,397][105620] Updated weights for policy 1, policy_version 490032 (0.0006) [2023-12-26 18:55:46,459][105620] Updated weights for policy 1, policy_version 490042 (0.0008) [2023-12-26 18:55:46,509][105620] Updated weights for policy 1, policy_version 490052 (0.0008) [2023-12-26 18:55:46,972][105692] Updated weights for policy 0, policy_version 489605 (0.0009) [2023-12-26 18:55:47,025][105692] Updated weights for policy 0, policy_version 489615 (0.0009) [2023-12-26 18:55:47,078][105692] Updated weights for policy 0, policy_version 489626 (0.0010) [2023-12-26 18:55:47,179][105620] Updated weights for policy 1, policy_version 490062 (0.0007) [2023-12-26 18:55:47,234][105620] Updated weights for policy 1, policy_version 490072 (0.0005) [2023-12-26 18:55:47,290][105620] Updated weights for policy 1, policy_version 490082 (0.0005) [2023-12-26 18:55:47,919][105692] Updated weights for policy 0, policy_version 489637 (0.0010) [2023-12-26 18:55:47,939][105620] Updated weights for policy 1, policy_version 490092 (0.0006) [2023-12-26 18:55:47,963][105692] Updated weights for policy 0, policy_version 489647 (0.0008) [2023-12-26 18:55:47,990][105620] Updated weights for policy 1, policy_version 490102 (0.0007) [2023-12-26 18:55:48,012][105692] Updated weights for policy 0, policy_version 489657 (0.0006) [2023-12-26 18:55:48,043][105620] Updated weights for policy 1, policy_version 490112 (0.0006) [2023-12-26 18:55:48,766][105620] Updated weights for policy 1, policy_version 490122 (0.0008) [2023-12-26 18:55:48,805][105692] Updated weights for policy 0, policy_version 489667 (0.0007) [2023-12-26 18:55:48,823][105620] Updated weights for policy 1, policy_version 490132 (0.0010) [2023-12-26 18:55:48,857][105692] Updated weights for policy 0, policy_version 489677 (0.0005) [2023-12-26 18:55:48,872][105620] Updated weights for policy 1, policy_version 490142 (0.0009) [2023-12-26 18:55:48,915][105692] Updated weights for policy 0, policy_version 489687 (0.0005) [2023-12-26 18:55:48,932][105620] Updated weights for policy 1, policy_version 490152 (0.0009) [2023-12-26 18:55:49,575][105692] Updated weights for policy 0, policy_version 489697 (0.0007) [2023-12-26 18:55:49,627][105692] Updated weights for policy 0, policy_version 489707 (0.0009) [2023-12-26 18:55:49,682][105692] Updated weights for policy 0, policy_version 489717 (0.0009) [2023-12-26 18:55:49,743][105692] Updated weights for policy 0, policy_version 489727 (0.0009) [2023-12-26 18:55:49,745][105620] Updated weights for policy 1, policy_version 490162 (0.0009) [2023-12-26 18:55:49,795][105620] Updated weights for policy 1, policy_version 490172 (0.0009) [2023-12-26 18:55:49,854][105620] Updated weights for policy 1, policy_version 490182 (0.0009) [2023-12-26 18:55:50,525][105692] Updated weights for policy 0, policy_version 489737 (0.0009) [2023-12-26 18:55:50,599][105692] Updated weights for policy 0, policy_version 489747 (0.0009) [2023-12-26 18:55:50,662][105692] Updated weights for policy 0, policy_version 489757 (0.0008) [2023-12-26 18:55:50,669][105620] Updated weights for policy 1, policy_version 490192 (0.0010) [2023-12-26 18:55:50,715][105620] Updated weights for policy 1, policy_version 490202 (0.0011) [2023-12-26 18:55:50,778][105620] Updated weights for policy 1, policy_version 490212 (0.0011) [2023-12-26 18:55:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 250904576. Throughput: 0: 9614.7, 1: 9902.1. Samples: 250893856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:55:51,062][104569] Avg episode reward: [(0, '8466.375'), (1, '9174.559')] [2023-12-26 18:55:51,359][105692] Updated weights for policy 0, policy_version 489767 (0.0007) [2023-12-26 18:55:51,423][105692] Updated weights for policy 0, policy_version 489777 (0.0008) [2023-12-26 18:55:51,483][105692] Updated weights for policy 0, policy_version 489787 (0.0006) [2023-12-26 18:55:51,591][105620] Updated weights for policy 1, policy_version 490222 (0.0009) [2023-12-26 18:55:51,662][105620] Updated weights for policy 1, policy_version 490232 (0.0008) [2023-12-26 18:55:51,726][105620] Updated weights for policy 1, policy_version 490242 (0.0008) [2023-12-26 18:55:52,178][105692] Updated weights for policy 0, policy_version 489797 (0.0007) [2023-12-26 18:55:52,227][105692] Updated weights for policy 0, policy_version 489807 (0.0009) [2023-12-26 18:55:52,286][105692] Updated weights for policy 0, policy_version 489817 (0.0010) [2023-12-26 18:55:52,482][105620] Updated weights for policy 1, policy_version 490252 (0.0008) [2023-12-26 18:55:52,537][105620] Updated weights for policy 1, policy_version 490262 (0.0009) [2023-12-26 18:55:52,596][105620] Updated weights for policy 1, policy_version 490272 (0.0009) [2023-12-26 18:55:53,071][105692] Updated weights for policy 0, policy_version 489827 (0.0008) [2023-12-26 18:55:53,135][105692] Updated weights for policy 0, policy_version 489837 (0.0006) [2023-12-26 18:55:53,201][105692] Updated weights for policy 0, policy_version 489847 (0.0005) [2023-12-26 18:55:53,429][105620] Updated weights for policy 1, policy_version 490282 (0.0009) [2023-12-26 18:55:53,479][105620] Updated weights for policy 1, policy_version 490292 (0.0010) [2023-12-26 18:55:53,534][105620] Updated weights for policy 1, policy_version 490302 (0.0010) [2023-12-26 18:55:53,582][105620] Updated weights for policy 1, policy_version 490312 (0.0010) [2023-12-26 18:55:53,838][105692] Updated weights for policy 0, policy_version 489857 (0.0005) [2023-12-26 18:55:53,896][105692] Updated weights for policy 0, policy_version 489867 (0.0008) [2023-12-26 18:55:53,945][105692] Updated weights for policy 0, policy_version 489877 (0.0008) [2023-12-26 18:55:54,000][105692] Updated weights for policy 0, policy_version 489887 (0.0008) [2023-12-26 18:55:54,326][105620] Updated weights for policy 1, policy_version 490322 (0.0010) [2023-12-26 18:55:54,378][105620] Updated weights for policy 1, policy_version 490332 (0.0010) [2023-12-26 18:55:54,432][105620] Updated weights for policy 1, policy_version 490342 (0.0010) [2023-12-26 18:55:54,775][105692] Updated weights for policy 0, policy_version 489897 (0.0008) [2023-12-26 18:55:54,828][105692] Updated weights for policy 0, policy_version 489907 (0.0008) [2023-12-26 18:55:54,830][105585] KL-divergence is very high: 150.3433 [2023-12-26 18:55:54,876][105585] KL-divergence is very high: 150.9855 [2023-12-26 18:55:54,890][105692] Updated weights for policy 0, policy_version 489917 (0.0008) [2023-12-26 18:55:55,190][105620] Updated weights for policy 1, policy_version 490352 (0.0010) [2023-12-26 18:55:55,241][105620] Updated weights for policy 1, policy_version 490362 (0.0010) [2023-12-26 18:55:55,288][105620] Updated weights for policy 1, policy_version 490372 (0.0010) [2023-12-26 18:55:55,671][105692] Updated weights for policy 0, policy_version 489927 (0.0008) [2023-12-26 18:55:55,722][105692] Updated weights for policy 0, policy_version 489937 (0.0008) [2023-12-26 18:55:55,789][105692] Updated weights for policy 0, policy_version 489947 (0.0009) [2023-12-26 18:55:56,051][105620] Updated weights for policy 1, policy_version 490382 (0.0009) [2023-12-26 18:55:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 250994688. Throughput: 0: 9522.8, 1: 9844.9. Samples: 251005340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:55:56,062][104569] Avg episode reward: [(0, '8815.626'), (1, '9268.786')] [2023-12-26 18:55:56,110][105620] Updated weights for policy 1, policy_version 490392 (0.0009) [2023-12-26 18:55:56,164][105620] Updated weights for policy 1, policy_version 490402 (0.0009) [2023-12-26 18:55:56,469][105692] Updated weights for policy 0, policy_version 489957 (0.0008) [2023-12-26 18:55:56,520][105692] Updated weights for policy 0, policy_version 489967 (0.0008) [2023-12-26 18:55:56,579][105692] Updated weights for policy 0, policy_version 489977 (0.0008) [2023-12-26 18:55:56,925][105620] Updated weights for policy 1, policy_version 490412 (0.0010) [2023-12-26 18:55:56,987][105620] Updated weights for policy 1, policy_version 490422 (0.0010) [2023-12-26 18:55:57,035][105620] Updated weights for policy 1, policy_version 490432 (0.0010) [2023-12-26 18:55:57,298][105692] Updated weights for policy 0, policy_version 489987 (0.0007) [2023-12-26 18:55:57,357][105692] Updated weights for policy 0, policy_version 489997 (0.0009) [2023-12-26 18:55:57,405][105692] Updated weights for policy 0, policy_version 490008 (0.0008) [2023-12-26 18:55:57,700][105620] Updated weights for policy 1, policy_version 490442 (0.0009) [2023-12-26 18:55:57,760][105620] Updated weights for policy 1, policy_version 490452 (0.0006) [2023-12-26 18:55:57,815][105620] Updated weights for policy 1, policy_version 490462 (0.0005) [2023-12-26 18:55:57,872][105620] Updated weights for policy 1, policy_version 490472 (0.0006) [2023-12-26 18:55:58,213][105692] Updated weights for policy 0, policy_version 490018 (0.0008) [2023-12-26 18:55:58,274][105692] Updated weights for policy 0, policy_version 490028 (0.0008) [2023-12-26 18:55:58,343][105692] Updated weights for policy 0, policy_version 490038 (0.0008) [2023-12-26 18:55:58,409][105692] Updated weights for policy 0, policy_version 490048 (0.0008) [2023-12-26 18:55:58,556][105620] Updated weights for policy 1, policy_version 490482 (0.0010) [2023-12-26 18:55:58,620][105620] Updated weights for policy 1, policy_version 490492 (0.0011) [2023-12-26 18:55:58,684][105620] Updated weights for policy 1, policy_version 490502 (0.0010) [2023-12-26 18:55:59,193][105692] Updated weights for policy 0, policy_version 490058 (0.0008) [2023-12-26 18:55:59,253][105692] Updated weights for policy 0, policy_version 490068 (0.0009) [2023-12-26 18:55:59,301][105692] Updated weights for policy 0, policy_version 490078 (0.0008) [2023-12-26 18:55:59,420][105620] Updated weights for policy 1, policy_version 490512 (0.0008) [2023-12-26 18:55:59,484][105620] Updated weights for policy 1, policy_version 490522 (0.0009) [2023-12-26 18:55:59,545][105620] Updated weights for policy 1, policy_version 490532 (0.0009) [2023-12-26 18:55:59,976][105692] Updated weights for policy 0, policy_version 490088 (0.0008) [2023-12-26 18:56:00,031][105692] Updated weights for policy 0, policy_version 490098 (0.0009) [2023-12-26 18:56:00,083][105692] Updated weights for policy 0, policy_version 490108 (0.0010) [2023-12-26 18:56:00,298][105620] Updated weights for policy 1, policy_version 490542 (0.0009) [2023-12-26 18:56:00,347][105620] Updated weights for policy 1, policy_version 490552 (0.0008) [2023-12-26 18:56:00,400][105620] Updated weights for policy 1, policy_version 490562 (0.0005) [2023-12-26 18:56:00,836][105692] Updated weights for policy 0, policy_version 490118 (0.0007) [2023-12-26 18:56:00,898][105692] Updated weights for policy 0, policy_version 490128 (0.0005) [2023-12-26 18:56:00,953][105692] Updated weights for policy 0, policy_version 490138 (0.0005) [2023-12-26 18:56:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 251092992. Throughput: 0: 9506.6, 1: 9888.3. Samples: 251062872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:56:01,062][104569] Avg episode reward: [(0, '8906.784'), (1, '9264.541')] [2023-12-26 18:56:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000490568_125599744.pth... [2023-12-26 18:56:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000490144_125493248.pth... [2023-12-26 18:56:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000489024_125206528.pth [2023-12-26 18:56:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000489416_125304832.pth [2023-12-26 18:56:01,118][105620] Updated weights for policy 1, policy_version 490572 (0.0005) [2023-12-26 18:56:01,182][105620] Updated weights for policy 1, policy_version 490582 (0.0007) [2023-12-26 18:56:01,248][105620] Updated weights for policy 1, policy_version 490592 (0.0008) [2023-12-26 18:56:01,614][105692] Updated weights for policy 0, policy_version 490148 (0.0006) [2023-12-26 18:56:01,674][105692] Updated weights for policy 0, policy_version 490158 (0.0010) [2023-12-26 18:56:01,731][105692] Updated weights for policy 0, policy_version 490168 (0.0011) [2023-12-26 18:56:01,934][105620] Updated weights for policy 1, policy_version 490602 (0.0006) [2023-12-26 18:56:01,990][105620] Updated weights for policy 1, policy_version 490612 (0.0008) [2023-12-26 18:56:02,046][105620] Updated weights for policy 1, policy_version 490622 (0.0008) [2023-12-26 18:56:02,097][105620] Updated weights for policy 1, policy_version 490632 (0.0008) [2023-12-26 18:56:02,453][105692] Updated weights for policy 0, policy_version 490178 (0.0011) [2023-12-26 18:56:02,509][105692] Updated weights for policy 0, policy_version 490188 (0.0010) [2023-12-26 18:56:02,560][105692] Updated weights for policy 0, policy_version 490198 (0.0010) [2023-12-26 18:56:02,609][105692] Updated weights for policy 0, policy_version 490208 (0.0010) [2023-12-26 18:56:02,838][105620] Updated weights for policy 1, policy_version 490642 (0.0008) [2023-12-26 18:56:02,904][105620] Updated weights for policy 1, policy_version 490652 (0.0008) [2023-12-26 18:56:02,965][105620] Updated weights for policy 1, policy_version 490662 (0.0008) [2023-12-26 18:56:03,264][105692] Updated weights for policy 0, policy_version 490218 (0.0005) [2023-12-26 18:56:03,308][105692] Updated weights for policy 0, policy_version 490228 (0.0005) [2023-12-26 18:56:03,355][105692] Updated weights for policy 0, policy_version 490238 (0.0005) [2023-12-26 18:56:03,725][105620] Updated weights for policy 1, policy_version 490672 (0.0008) [2023-12-26 18:56:03,783][105620] Updated weights for policy 1, policy_version 490682 (0.0008) [2023-12-26 18:56:03,842][105620] Updated weights for policy 1, policy_version 490692 (0.0008) [2023-12-26 18:56:03,994][105692] Updated weights for policy 0, policy_version 490248 (0.0009) [2023-12-26 18:56:04,049][105692] Updated weights for policy 0, policy_version 490258 (0.0006) [2023-12-26 18:56:04,111][105692] Updated weights for policy 0, policy_version 490268 (0.0007) [2023-12-26 18:56:04,584][105620] Updated weights for policy 1, policy_version 490702 (0.0007) [2023-12-26 18:56:04,644][105620] Updated weights for policy 1, policy_version 490712 (0.0008) [2023-12-26 18:56:04,703][105620] Updated weights for policy 1, policy_version 490722 (0.0009) [2023-12-26 18:56:04,821][105692] Updated weights for policy 0, policy_version 490278 (0.0007) [2023-12-26 18:56:04,870][105585] KL-divergence is very high: 122.8251 [2023-12-26 18:56:04,890][105692] Updated weights for policy 0, policy_version 490288 (0.0005) [2023-12-26 18:56:04,922][105585] KL-divergence is very high: 212.3927 [2023-12-26 18:56:04,953][105692] Updated weights for policy 0, policy_version 490298 (0.0005) [2023-12-26 18:56:04,974][105585] KL-divergence is very high: 218.5702 [2023-12-26 18:56:05,428][105620] Updated weights for policy 1, policy_version 490732 (0.0009) [2023-12-26 18:56:05,479][105620] Updated weights for policy 1, policy_version 490742 (0.0009) [2023-12-26 18:56:05,528][105585] KL-divergence is very high: 118.3491 [2023-12-26 18:56:05,531][105620] Updated weights for policy 1, policy_version 490752 (0.0009) [2023-12-26 18:56:05,542][105692] Updated weights for policy 0, policy_version 490308 (0.0005) [2023-12-26 18:56:05,596][105692] Updated weights for policy 0, policy_version 490318 (0.0005) [2023-12-26 18:56:05,644][105692] Updated weights for policy 0, policy_version 490328 (0.0005) [2023-12-26 18:56:06,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 251191296. Throughput: 0: 9489.8, 1: 9882.0. Samples: 251179436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:56:06,063][104569] Avg episode reward: [(0, '8998.144'), (1, '9258.223')] [2023-12-26 18:56:06,184][105620] Updated weights for policy 1, policy_version 490762 (0.0008) [2023-12-26 18:56:06,243][105620] Updated weights for policy 1, policy_version 490772 (0.0006) [2023-12-26 18:56:06,259][105692] Updated weights for policy 0, policy_version 490338 (0.0006) [2023-12-26 18:56:06,308][105620] Updated weights for policy 1, policy_version 490782 (0.0007) [2023-12-26 18:56:06,326][105692] Updated weights for policy 0, policy_version 490348 (0.0007) [2023-12-26 18:56:06,366][105620] Updated weights for policy 1, policy_version 490792 (0.0007) [2023-12-26 18:56:06,389][105692] Updated weights for policy 0, policy_version 490358 (0.0006) [2023-12-26 18:56:06,458][105692] Updated weights for policy 0, policy_version 490368 (0.0009) [2023-12-26 18:56:07,061][105620] Updated weights for policy 1, policy_version 490802 (0.0009) [2023-12-26 18:56:07,112][105620] Updated weights for policy 1, policy_version 490812 (0.0009) [2023-12-26 18:56:07,162][105620] Updated weights for policy 1, policy_version 490822 (0.0009) [2023-12-26 18:56:07,206][105692] Updated weights for policy 0, policy_version 490378 (0.0009) [2023-12-26 18:56:07,253][105692] Updated weights for policy 0, policy_version 490388 (0.0008) [2023-12-26 18:56:07,305][105692] Updated weights for policy 0, policy_version 490398 (0.0009) [2023-12-26 18:56:07,772][105620] Updated weights for policy 1, policy_version 490832 (0.0006) [2023-12-26 18:56:07,833][105620] Updated weights for policy 1, policy_version 490842 (0.0005) [2023-12-26 18:56:07,891][105620] Updated weights for policy 1, policy_version 490852 (0.0005) [2023-12-26 18:56:08,042][105692] Updated weights for policy 0, policy_version 490408 (0.0009) [2023-12-26 18:56:08,108][105692] Updated weights for policy 0, policy_version 490418 (0.0010) [2023-12-26 18:56:08,162][105692] Updated weights for policy 0, policy_version 490428 (0.0010) [2023-12-26 18:56:08,414][105620] Updated weights for policy 1, policy_version 490862 (0.0007) [2023-12-26 18:56:08,466][105620] Updated weights for policy 1, policy_version 490872 (0.0008) [2023-12-26 18:56:08,515][105620] Updated weights for policy 1, policy_version 490882 (0.0008) [2023-12-26 18:56:08,930][105692] Updated weights for policy 0, policy_version 490439 (0.0011) [2023-12-26 18:56:08,974][105692] Updated weights for policy 0, policy_version 490449 (0.0010) [2023-12-26 18:56:09,029][105692] Updated weights for policy 0, policy_version 490459 (0.0010) [2023-12-26 18:56:09,191][105620] Updated weights for policy 1, policy_version 490892 (0.0010) [2023-12-26 18:56:09,256][105620] Updated weights for policy 1, policy_version 490902 (0.0010) [2023-12-26 18:56:09,322][105620] Updated weights for policy 1, policy_version 490912 (0.0009) [2023-12-26 18:56:09,774][105692] Updated weights for policy 0, policy_version 490469 (0.0005) [2023-12-26 18:56:09,832][105692] Updated weights for policy 0, policy_version 490479 (0.0006) [2023-12-26 18:56:09,896][105692] Updated weights for policy 0, policy_version 490489 (0.0007) [2023-12-26 18:56:10,006][105620] Updated weights for policy 1, policy_version 490922 (0.0010) [2023-12-26 18:56:10,059][105620] Updated weights for policy 1, policy_version 490932 (0.0010) [2023-12-26 18:56:10,106][105620] Updated weights for policy 1, policy_version 490942 (0.0009) [2023-12-26 18:56:10,163][105620] Updated weights for policy 1, policy_version 490952 (0.0008) [2023-12-26 18:56:10,502][105692] Updated weights for policy 0, policy_version 490499 (0.0007) [2023-12-26 18:56:10,549][105692] Updated weights for policy 0, policy_version 490509 (0.0005) [2023-12-26 18:56:10,599][105692] Updated weights for policy 0, policy_version 490519 (0.0005) [2023-12-26 18:56:10,883][105620] Updated weights for policy 1, policy_version 490962 (0.0008) [2023-12-26 18:56:10,939][105620] Updated weights for policy 1, policy_version 490972 (0.0006) [2023-12-26 18:56:10,993][105620] Updated weights for policy 1, policy_version 490982 (0.0006) [2023-12-26 18:56:11,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 251297792. Throughput: 0: 9628.7, 1: 9952.0. Samples: 251303444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:56:11,063][104569] Avg episode reward: [(0, '9178.050'), (1, '9199.534')] [2023-12-26 18:56:11,275][105692] Updated weights for policy 0, policy_version 490529 (0.0006) [2023-12-26 18:56:11,324][105692] Updated weights for policy 0, policy_version 490539 (0.0008) [2023-12-26 18:56:11,388][105692] Updated weights for policy 0, policy_version 490549 (0.0009) [2023-12-26 18:56:11,442][105692] Updated weights for policy 0, policy_version 490559 (0.0008) [2023-12-26 18:56:11,701][105620] Updated weights for policy 1, policy_version 490992 (0.0009) [2023-12-26 18:56:11,776][105620] Updated weights for policy 1, policy_version 491002 (0.0010) [2023-12-26 18:56:11,843][105620] Updated weights for policy 1, policy_version 491012 (0.0011) [2023-12-26 18:56:12,247][105692] Updated weights for policy 0, policy_version 490569 (0.0010) [2023-12-26 18:56:12,314][105692] Updated weights for policy 0, policy_version 490579 (0.0010) [2023-12-26 18:56:12,381][105692] Updated weights for policy 0, policy_version 490589 (0.0009) [2023-12-26 18:56:12,501][105620] Updated weights for policy 1, policy_version 491022 (0.0008) [2023-12-26 18:56:12,566][105620] Updated weights for policy 1, policy_version 491032 (0.0009) [2023-12-26 18:56:12,625][105620] Updated weights for policy 1, policy_version 491042 (0.0011) [2023-12-26 18:56:13,103][105692] Updated weights for policy 0, policy_version 490599 (0.0008) [2023-12-26 18:56:13,151][105692] Updated weights for policy 0, policy_version 490609 (0.0008) [2023-12-26 18:56:13,204][105692] Updated weights for policy 0, policy_version 490619 (0.0008) [2023-12-26 18:56:13,334][105620] Updated weights for policy 1, policy_version 491052 (0.0010) [2023-12-26 18:56:13,386][105620] Updated weights for policy 1, policy_version 491062 (0.0010) [2023-12-26 18:56:13,436][105620] Updated weights for policy 1, policy_version 491072 (0.0010) [2023-12-26 18:56:14,005][105692] Updated weights for policy 0, policy_version 490629 (0.0008) [2023-12-26 18:56:14,061][105692] Updated weights for policy 0, policy_version 490639 (0.0008) [2023-12-26 18:56:14,118][105692] Updated weights for policy 0, policy_version 490649 (0.0009) [2023-12-26 18:56:14,176][105620] Updated weights for policy 1, policy_version 491082 (0.0009) [2023-12-26 18:56:14,223][105620] Updated weights for policy 1, policy_version 491092 (0.0005) [2023-12-26 18:56:14,269][105620] Updated weights for policy 1, policy_version 491102 (0.0005) [2023-12-26 18:56:14,317][105620] Updated weights for policy 1, policy_version 491112 (0.0005) [2023-12-26 18:56:14,851][105692] Updated weights for policy 0, policy_version 490659 (0.0007) [2023-12-26 18:56:14,909][105692] Updated weights for policy 0, policy_version 490670 (0.0009) [2023-12-26 18:56:14,975][105692] Updated weights for policy 0, policy_version 490680 (0.0007) [2023-12-26 18:56:14,997][105620] Updated weights for policy 1, policy_version 491122 (0.0007) [2023-12-26 18:56:15,061][105620] Updated weights for policy 1, policy_version 491132 (0.0006) [2023-12-26 18:56:15,130][105620] Updated weights for policy 1, policy_version 491142 (0.0007) [2023-12-26 18:56:15,673][105692] Updated weights for policy 0, policy_version 490690 (0.0009) [2023-12-26 18:56:15,725][105692] Updated weights for policy 0, policy_version 490700 (0.0008) [2023-12-26 18:56:15,766][105620] Updated weights for policy 1, policy_version 491152 (0.0011) [2023-12-26 18:56:15,806][105692] Updated weights for policy 0, policy_version 490710 (0.0010) [2023-12-26 18:56:15,823][105620] Updated weights for policy 1, policy_version 491162 (0.0011) [2023-12-26 18:56:15,858][105692] Updated weights for policy 0, policy_version 490720 (0.0008) [2023-12-26 18:56:15,874][105620] Updated weights for policy 1, policy_version 491172 (0.0009) [2023-12-26 18:56:16,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 251396096. Throughput: 0: 9630.8, 1: 9922.1. Samples: 251360584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:56:16,062][104569] Avg episode reward: [(0, '8633.551'), (1, '9031.447')] [2023-12-26 18:56:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000490720_125640704.pth... [2023-12-26 18:56:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000491176_125755392.pth... [2023-12-26 18:56:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000490024_125460480.pth [2023-12-26 18:56:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000489600_125353984.pth [2023-12-26 18:56:16,521][105620] Updated weights for policy 1, policy_version 491182 (0.0008) [2023-12-26 18:56:16,577][105620] Updated weights for policy 1, policy_version 491192 (0.0008) [2023-12-26 18:56:16,635][105620] Updated weights for policy 1, policy_version 491202 (0.0008) [2023-12-26 18:56:16,649][105692] Updated weights for policy 0, policy_version 490730 (0.0007) [2023-12-26 18:56:16,709][105692] Updated weights for policy 0, policy_version 490740 (0.0008) [2023-12-26 18:56:16,768][105692] Updated weights for policy 0, policy_version 490750 (0.0009) [2023-12-26 18:56:17,323][105620] Updated weights for policy 1, policy_version 491212 (0.0007) [2023-12-26 18:56:17,370][105620] Updated weights for policy 1, policy_version 491222 (0.0008) [2023-12-26 18:56:17,428][105620] Updated weights for policy 1, policy_version 491232 (0.0007) [2023-12-26 18:56:17,521][105692] Updated weights for policy 0, policy_version 490760 (0.0009) [2023-12-26 18:56:17,578][105692] Updated weights for policy 0, policy_version 490771 (0.0010) [2023-12-26 18:56:17,640][105692] Updated weights for policy 0, policy_version 490781 (0.0008) [2023-12-26 18:56:17,994][105620] Updated weights for policy 1, policy_version 491242 (0.0005) [2023-12-26 18:56:18,049][105620] Updated weights for policy 1, policy_version 491252 (0.0008) [2023-12-26 18:56:18,106][105620] Updated weights for policy 1, policy_version 491262 (0.0008) [2023-12-26 18:56:18,175][105620] Updated weights for policy 1, policy_version 491272 (0.0009) [2023-12-26 18:56:18,342][105692] Updated weights for policy 0, policy_version 490791 (0.0007) [2023-12-26 18:56:18,396][105692] Updated weights for policy 0, policy_version 490801 (0.0007) [2023-12-26 18:56:18,457][105692] Updated weights for policy 0, policy_version 490811 (0.0006) [2023-12-26 18:56:18,871][105620] Updated weights for policy 1, policy_version 491282 (0.0009) [2023-12-26 18:56:18,933][105620] Updated weights for policy 1, policy_version 491292 (0.0009) [2023-12-26 18:56:18,991][105620] Updated weights for policy 1, policy_version 491302 (0.0009) [2023-12-26 18:56:19,158][105692] Updated weights for policy 0, policy_version 490821 (0.0009) [2023-12-26 18:56:19,209][105692] Updated weights for policy 0, policy_version 490832 (0.0009) [2023-12-26 18:56:19,269][105692] Updated weights for policy 0, policy_version 490842 (0.0009) [2023-12-26 18:56:19,766][105620] Updated weights for policy 1, policy_version 491312 (0.0010) [2023-12-26 18:56:19,829][105620] Updated weights for policy 1, policy_version 491322 (0.0009) [2023-12-26 18:56:19,893][105620] Updated weights for policy 1, policy_version 491332 (0.0009) [2023-12-26 18:56:20,029][105692] Updated weights for policy 0, policy_version 490852 (0.0009) [2023-12-26 18:56:20,083][105692] Updated weights for policy 0, policy_version 490862 (0.0008) [2023-12-26 18:56:20,145][105692] Updated weights for policy 0, policy_version 490872 (0.0008) [2023-12-26 18:56:20,623][105620] Updated weights for policy 1, policy_version 491342 (0.0009) [2023-12-26 18:56:20,682][105620] Updated weights for policy 1, policy_version 491352 (0.0009) [2023-12-26 18:56:20,745][105620] Updated weights for policy 1, policy_version 491362 (0.0009) [2023-12-26 18:56:20,949][105692] Updated weights for policy 0, policy_version 490882 (0.0008) [2023-12-26 18:56:21,003][105585] KL-divergence is very high: 105.3568 [2023-12-26 18:56:21,011][105692] Updated weights for policy 0, policy_version 490892 (0.0009) [2023-12-26 18:56:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 251486208. Throughput: 0: 9682.2, 1: 9861.4. Samples: 251478476. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:56:21,063][104569] Avg episode reward: [(0, '8449.102'), (1, '9175.367')] [2023-12-26 18:56:21,065][105585] KL-divergence is very high: 126.8933 [2023-12-26 18:56:21,087][105692] Updated weights for policy 0, policy_version 490902 (0.0009) [2023-12-26 18:56:21,154][105692] Updated weights for policy 0, policy_version 490912 (0.0009) [2023-12-26 18:56:21,565][105620] Updated weights for policy 1, policy_version 491372 (0.0008) [2023-12-26 18:56:21,616][105620] Updated weights for policy 1, policy_version 491382 (0.0009) [2023-12-26 18:56:21,700][105620] Updated weights for policy 1, policy_version 491392 (0.0009) [2023-12-26 18:56:21,814][105692] Updated weights for policy 0, policy_version 490922 (0.0008) [2023-12-26 18:56:21,880][105692] Updated weights for policy 0, policy_version 490932 (0.0007) [2023-12-26 18:56:21,940][105692] Updated weights for policy 0, policy_version 490942 (0.0008) [2023-12-26 18:56:22,445][105620] Updated weights for policy 1, policy_version 491402 (0.0009) [2023-12-26 18:56:22,494][105620] Updated weights for policy 1, policy_version 491412 (0.0010) [2023-12-26 18:56:22,550][105620] Updated weights for policy 1, policy_version 491422 (0.0011) [2023-12-26 18:56:22,609][105620] Updated weights for policy 1, policy_version 491432 (0.0011) [2023-12-26 18:56:22,678][105692] Updated weights for policy 0, policy_version 490952 (0.0010) [2023-12-26 18:56:22,733][105692] Updated weights for policy 0, policy_version 490962 (0.0006) [2023-12-26 18:56:22,792][105692] Updated weights for policy 0, policy_version 490972 (0.0010) [2023-12-26 18:56:23,352][105620] Updated weights for policy 1, policy_version 491442 (0.0009) [2023-12-26 18:56:23,415][105620] Updated weights for policy 1, policy_version 491452 (0.0008) [2023-12-26 18:56:23,477][105620] Updated weights for policy 1, policy_version 491462 (0.0006) [2023-12-26 18:56:23,545][105692] Updated weights for policy 0, policy_version 490982 (0.0010) [2023-12-26 18:56:23,606][105692] Updated weights for policy 0, policy_version 490992 (0.0010) [2023-12-26 18:56:23,672][105692] Updated weights for policy 0, policy_version 491002 (0.0011) [2023-12-26 18:56:24,161][105620] Updated weights for policy 1, policy_version 491472 (0.0008) [2023-12-26 18:56:24,223][105620] Updated weights for policy 1, policy_version 491482 (0.0010) [2023-12-26 18:56:24,274][105620] Updated weights for policy 1, policy_version 491492 (0.0009) [2023-12-26 18:56:24,298][105692] Updated weights for policy 0, policy_version 491012 (0.0010) [2023-12-26 18:56:24,353][105692] Updated weights for policy 0, policy_version 491022 (0.0010) [2023-12-26 18:56:24,417][105692] Updated weights for policy 0, policy_version 491032 (0.0010) [2023-12-26 18:56:24,969][105620] Updated weights for policy 1, policy_version 491502 (0.0006) [2023-12-26 18:56:25,020][105620] Updated weights for policy 1, policy_version 491512 (0.0005) [2023-12-26 18:56:25,069][105620] Updated weights for policy 1, policy_version 491522 (0.0009) [2023-12-26 18:56:25,147][105692] Updated weights for policy 0, policy_version 491042 (0.0010) [2023-12-26 18:56:25,198][105692] Updated weights for policy 0, policy_version 491052 (0.0010) [2023-12-26 18:56:25,248][105692] Updated weights for policy 0, policy_version 491062 (0.0010) [2023-12-26 18:56:25,295][105692] Updated weights for policy 0, policy_version 491072 (0.0010) [2023-12-26 18:56:25,738][105620] Updated weights for policy 1, policy_version 491532 (0.0010) [2023-12-26 18:56:25,793][105620] Updated weights for policy 1, policy_version 491542 (0.0010) [2023-12-26 18:56:25,848][105620] Updated weights for policy 1, policy_version 491552 (0.0010) [2023-12-26 18:56:26,040][105692] Updated weights for policy 0, policy_version 491082 (0.0006) [2023-12-26 18:56:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 251584512. Throughput: 0: 9598.3, 1: 9780.3. Samples: 251593304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:56:26,062][104569] Avg episode reward: [(0, '8723.988'), (1, '9261.302')] [2023-12-26 18:56:26,108][105692] Updated weights for policy 0, policy_version 491092 (0.0006) [2023-12-26 18:56:26,174][105692] Updated weights for policy 0, policy_version 491102 (0.0006) [2023-12-26 18:56:26,591][105620] Updated weights for policy 1, policy_version 491562 (0.0009) [2023-12-26 18:56:26,644][105620] Updated weights for policy 1, policy_version 491572 (0.0010) [2023-12-26 18:56:26,683][105692] Updated weights for policy 0, policy_version 491112 (0.0009) [2023-12-26 18:56:26,692][105620] Updated weights for policy 1, policy_version 491582 (0.0010) [2023-12-26 18:56:26,739][105692] Updated weights for policy 0, policy_version 491122 (0.0008) [2023-12-26 18:56:26,740][105620] Updated weights for policy 1, policy_version 491592 (0.0010) [2023-12-26 18:56:26,802][105692] Updated weights for policy 0, policy_version 491132 (0.0005) [2023-12-26 18:56:27,368][105692] Updated weights for policy 0, policy_version 491142 (0.0005) [2023-12-26 18:56:27,418][105692] Updated weights for policy 0, policy_version 491152 (0.0005) [2023-12-26 18:56:27,452][105620] Updated weights for policy 1, policy_version 491602 (0.0005) [2023-12-26 18:56:27,473][105692] Updated weights for policy 0, policy_version 491162 (0.0009) [2023-12-26 18:56:27,509][105620] Updated weights for policy 1, policy_version 491612 (0.0005) [2023-12-26 18:56:27,569][105620] Updated weights for policy 1, policy_version 491622 (0.0008) [2023-12-26 18:56:28,103][105620] Updated weights for policy 1, policy_version 491632 (0.0008) [2023-12-26 18:56:28,113][105692] Updated weights for policy 0, policy_version 491172 (0.0007) [2023-12-26 18:56:28,161][105692] Updated weights for policy 0, policy_version 491182 (0.0006) [2023-12-26 18:56:28,162][105620] Updated weights for policy 1, policy_version 491642 (0.0010) [2023-12-26 18:56:28,208][105692] Updated weights for policy 0, policy_version 491192 (0.0007) [2023-12-26 18:56:28,215][105620] Updated weights for policy 1, policy_version 491652 (0.0010) [2023-12-26 18:56:28,219][105586] KL-divergence is very high: 120.4557 [2023-12-26 18:56:28,965][105620] Updated weights for policy 1, policy_version 491662 (0.0010) [2023-12-26 18:56:29,000][105692] Updated weights for policy 0, policy_version 491202 (0.0006) [2023-12-26 18:56:29,024][105620] Updated weights for policy 1, policy_version 491672 (0.0010) [2023-12-26 18:56:29,054][105692] Updated weights for policy 0, policy_version 491212 (0.0007) [2023-12-26 18:56:29,083][105620] Updated weights for policy 1, policy_version 491682 (0.0010) [2023-12-26 18:56:29,112][105692] Updated weights for policy 0, policy_version 491222 (0.0006) [2023-12-26 18:56:29,160][105692] Updated weights for policy 0, policy_version 491232 (0.0008) [2023-12-26 18:56:29,854][105620] Updated weights for policy 1, policy_version 491692 (0.0011) [2023-12-26 18:56:29,904][105620] Updated weights for policy 1, policy_version 491702 (0.0011) [2023-12-26 18:56:29,939][105692] Updated weights for policy 0, policy_version 491242 (0.0008) [2023-12-26 18:56:29,968][105620] Updated weights for policy 1, policy_version 491712 (0.0011) [2023-12-26 18:56:29,998][105692] Updated weights for policy 0, policy_version 491252 (0.0006) [2023-12-26 18:56:30,052][105692] Updated weights for policy 0, policy_version 491262 (0.0007) [2023-12-26 18:56:30,656][105620] Updated weights for policy 1, policy_version 491722 (0.0011) [2023-12-26 18:56:30,714][105620] Updated weights for policy 1, policy_version 491732 (0.0010) [2023-12-26 18:56:30,736][105692] Updated weights for policy 0, policy_version 491272 (0.0006) [2023-12-26 18:56:30,770][105620] Updated weights for policy 1, policy_version 491742 (0.0011) [2023-12-26 18:56:30,786][105692] Updated weights for policy 0, policy_version 491282 (0.0005) [2023-12-26 18:56:30,814][105620] Updated weights for policy 1, policy_version 491752 (0.0010) [2023-12-26 18:56:30,840][105692] Updated weights for policy 0, policy_version 491292 (0.0006) [2023-12-26 18:56:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 251691008. Throughput: 0: 9753.4, 1: 9780.6. Samples: 251657248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:56:31,062][104569] Avg episode reward: [(0, '8904.135'), (1, '9170.532')] [2023-12-26 18:56:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000491752_125902848.pth... [2023-12-26 18:56:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000491296_125788160.pth... [2023-12-26 18:56:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000490144_125493248.pth [2023-12-26 18:56:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000490568_125599744.pth [2023-12-26 18:56:31,509][105692] Updated weights for policy 0, policy_version 491302 (0.0009) [2023-12-26 18:56:31,532][105620] Updated weights for policy 1, policy_version 491762 (0.0005) [2023-12-26 18:56:31,564][105692] Updated weights for policy 0, policy_version 491312 (0.0010) [2023-12-26 18:56:31,579][105620] Updated weights for policy 1, policy_version 491772 (0.0006) [2023-12-26 18:56:31,627][105692] Updated weights for policy 0, policy_version 491322 (0.0011) [2023-12-26 18:56:31,633][105620] Updated weights for policy 1, policy_version 491782 (0.0010) [2023-12-26 18:56:32,246][105620] Updated weights for policy 1, policy_version 491792 (0.0006) [2023-12-26 18:56:32,309][105620] Updated weights for policy 1, policy_version 491802 (0.0006) [2023-12-26 18:56:32,383][105620] Updated weights for policy 1, policy_version 491812 (0.0008) [2023-12-26 18:56:32,439][105692] Updated weights for policy 0, policy_version 491332 (0.0010) [2023-12-26 18:56:32,485][105692] Updated weights for policy 0, policy_version 491342 (0.0009) [2023-12-26 18:56:32,541][105692] Updated weights for policy 0, policy_version 491352 (0.0005) [2023-12-26 18:56:33,132][105620] Updated weights for policy 1, policy_version 491822 (0.0009) [2023-12-26 18:56:33,190][105692] Updated weights for policy 0, policy_version 491362 (0.0006) [2023-12-26 18:56:33,200][105620] Updated weights for policy 1, policy_version 491832 (0.0008) [2023-12-26 18:56:33,239][105692] Updated weights for policy 0, policy_version 491372 (0.0007) [2023-12-26 18:56:33,257][105620] Updated weights for policy 1, policy_version 491842 (0.0007) [2023-12-26 18:56:33,289][105692] Updated weights for policy 0, policy_version 491382 (0.0005) [2023-12-26 18:56:33,343][105692] Updated weights for policy 0, policy_version 491392 (0.0008) [2023-12-26 18:56:33,822][105620] Updated weights for policy 1, policy_version 491852 (0.0007) [2023-12-26 18:56:33,876][105620] Updated weights for policy 1, policy_version 491862 (0.0005) [2023-12-26 18:56:33,910][105692] Updated weights for policy 0, policy_version 491402 (0.0005) [2023-12-26 18:56:33,924][105620] Updated weights for policy 1, policy_version 491872 (0.0005) [2023-12-26 18:56:33,958][105692] Updated weights for policy 0, policy_version 491412 (0.0007) [2023-12-26 18:56:33,968][105585] KL-divergence is very high: 151.8714 [2023-12-26 18:56:34,010][105692] Updated weights for policy 0, policy_version 491422 (0.0008) [2023-12-26 18:56:34,011][105585] KL-divergence is very high: 163.9508 [2023-12-26 18:56:34,656][105620] Updated weights for policy 1, policy_version 491882 (0.0008) [2023-12-26 18:56:34,688][105692] Updated weights for policy 0, policy_version 491432 (0.0007) [2023-12-26 18:56:34,711][105620] Updated weights for policy 1, policy_version 491892 (0.0007) [2023-12-26 18:56:34,744][105692] Updated weights for policy 0, policy_version 491442 (0.0006) [2023-12-26 18:56:34,769][105620] Updated weights for policy 1, policy_version 491902 (0.0007) [2023-12-26 18:56:34,800][105692] Updated weights for policy 0, policy_version 491452 (0.0005) [2023-12-26 18:56:34,825][105620] Updated weights for policy 1, policy_version 491912 (0.0008) [2023-12-26 18:56:35,423][105692] Updated weights for policy 0, policy_version 491462 (0.0008) [2023-12-26 18:56:35,472][105692] Updated weights for policy 0, policy_version 491472 (0.0009) [2023-12-26 18:56:35,521][105692] Updated weights for policy 0, policy_version 491482 (0.0008) [2023-12-26 18:56:35,631][105620] Updated weights for policy 1, policy_version 491922 (0.0009) [2023-12-26 18:56:35,678][105620] Updated weights for policy 1, policy_version 491932 (0.0009) [2023-12-26 18:56:35,733][105620] Updated weights for policy 1, policy_version 491942 (0.0009) [2023-12-26 18:56:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 251789312. Throughput: 0: 9799.8, 1: 9832.5. Samples: 251777312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:56:36,063][104569] Avg episode reward: [(0, '8813.435'), (1, '9172.972')] [2023-12-26 18:56:36,328][105692] Updated weights for policy 0, policy_version 491492 (0.0009) [2023-12-26 18:56:36,390][105692] Updated weights for policy 0, policy_version 491502 (0.0008) [2023-12-26 18:56:36,445][105620] Updated weights for policy 1, policy_version 491952 (0.0007) [2023-12-26 18:56:36,451][105692] Updated weights for policy 0, policy_version 491512 (0.0008) [2023-12-26 18:56:36,512][105620] Updated weights for policy 1, policy_version 491962 (0.0007) [2023-12-26 18:56:36,571][105620] Updated weights for policy 1, policy_version 491972 (0.0008) [2023-12-26 18:56:37,057][105692] Updated weights for policy 0, policy_version 491522 (0.0007) [2023-12-26 18:56:37,124][105692] Updated weights for policy 0, policy_version 491532 (0.0005) [2023-12-26 18:56:37,176][105692] Updated weights for policy 0, policy_version 491542 (0.0005) [2023-12-26 18:56:37,224][105585] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000005 [2023-12-26 18:56:37,225][105692] Updated weights for policy 0, policy_version 491552 (0.0005) [2023-12-26 18:56:37,328][105620] Updated weights for policy 1, policy_version 491982 (0.0009) [2023-12-26 18:56:37,391][105620] Updated weights for policy 1, policy_version 491992 (0.0010) [2023-12-26 18:56:37,457][105620] Updated weights for policy 1, policy_version 492002 (0.0010) [2023-12-26 18:56:37,798][105692] Updated weights for policy 0, policy_version 491562 (0.0008) [2023-12-26 18:56:37,845][105692] Updated weights for policy 0, policy_version 491572 (0.0009) [2023-12-26 18:56:37,891][105692] Updated weights for policy 0, policy_version 491582 (0.0008) [2023-12-26 18:56:38,218][105620] Updated weights for policy 1, policy_version 492012 (0.0009) [2023-12-26 18:56:38,281][105620] Updated weights for policy 1, policy_version 492022 (0.0009) [2023-12-26 18:56:38,339][105620] Updated weights for policy 1, policy_version 492032 (0.0009) [2023-12-26 18:56:38,721][105692] Updated weights for policy 0, policy_version 491592 (0.0009) [2023-12-26 18:56:38,780][105692] Updated weights for policy 0, policy_version 491602 (0.0009) [2023-12-26 18:56:38,838][105692] Updated weights for policy 0, policy_version 491612 (0.0009) [2023-12-26 18:56:39,072][105620] Updated weights for policy 1, policy_version 492042 (0.0007) [2023-12-26 18:56:39,118][105620] Updated weights for policy 1, policy_version 492052 (0.0005) [2023-12-26 18:56:39,164][105620] Updated weights for policy 1, policy_version 492062 (0.0005) [2023-12-26 18:56:39,219][105620] Updated weights for policy 1, policy_version 492072 (0.0008) [2023-12-26 18:56:39,628][105692] Updated weights for policy 0, policy_version 491622 (0.0009) [2023-12-26 18:56:39,686][105692] Updated weights for policy 0, policy_version 491632 (0.0007) [2023-12-26 18:56:39,755][105692] Updated weights for policy 0, policy_version 491642 (0.0006) [2023-12-26 18:56:40,040][105620] Updated weights for policy 1, policy_version 492082 (0.0009) [2023-12-26 18:56:40,100][105620] Updated weights for policy 1, policy_version 492092 (0.0009) [2023-12-26 18:56:40,161][105620] Updated weights for policy 1, policy_version 492102 (0.0009) [2023-12-26 18:56:40,370][105692] Updated weights for policy 0, policy_version 491652 (0.0007) [2023-12-26 18:56:40,433][105692] Updated weights for policy 0, policy_version 491662 (0.0005) [2023-12-26 18:56:40,489][105692] Updated weights for policy 0, policy_version 491672 (0.0005) [2023-12-26 18:56:40,887][105620] Updated weights for policy 1, policy_version 492112 (0.0009) [2023-12-26 18:56:40,935][105620] Updated weights for policy 1, policy_version 492122 (0.0009) [2023-12-26 18:56:40,986][105620] Updated weights for policy 1, policy_version 492132 (0.0009) [2023-12-26 18:56:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 251887616. Throughput: 0: 9881.3, 1: 9858.5. Samples: 251893632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:56:41,062][104569] Avg episode reward: [(0, '9174.777'), (1, '9264.497')] [2023-12-26 18:56:41,099][105692] Updated weights for policy 0, policy_version 491682 (0.0006) [2023-12-26 18:56:41,165][105692] Updated weights for policy 0, policy_version 491692 (0.0009) [2023-12-26 18:56:41,232][105692] Updated weights for policy 0, policy_version 491702 (0.0008) [2023-12-26 18:56:41,295][105692] Updated weights for policy 0, policy_version 491712 (0.0009) [2023-12-26 18:56:41,843][105620] Updated weights for policy 1, policy_version 492142 (0.0009) [2023-12-26 18:56:41,902][105620] Updated weights for policy 1, policy_version 492152 (0.0009) [2023-12-26 18:56:41,961][105620] Updated weights for policy 1, policy_version 492162 (0.0009) [2023-12-26 18:56:42,067][105692] Updated weights for policy 0, policy_version 491722 (0.0006) [2023-12-26 18:56:42,117][105692] Updated weights for policy 0, policy_version 491732 (0.0008) [2023-12-26 18:56:42,163][105692] Updated weights for policy 0, policy_version 491742 (0.0005) [2023-12-26 18:56:42,762][105620] Updated weights for policy 1, policy_version 492172 (0.0008) [2023-12-26 18:56:42,824][105620] Updated weights for policy 1, policy_version 492182 (0.0009) [2023-12-26 18:56:42,883][105620] Updated weights for policy 1, policy_version 492192 (0.0007) [2023-12-26 18:56:42,888][105692] Updated weights for policy 0, policy_version 491752 (0.0008) [2023-12-26 18:56:42,948][105692] Updated weights for policy 0, policy_version 491762 (0.0008) [2023-12-26 18:56:42,995][105692] Updated weights for policy 0, policy_version 491772 (0.0009) [2023-12-26 18:56:43,633][105620] Updated weights for policy 1, policy_version 492202 (0.0008) [2023-12-26 18:56:43,685][105620] Updated weights for policy 1, policy_version 492212 (0.0009) [2023-12-26 18:56:43,729][105692] Updated weights for policy 0, policy_version 491782 (0.0007) [2023-12-26 18:56:43,746][105620] Updated weights for policy 1, policy_version 492222 (0.0008) [2023-12-26 18:56:43,780][105692] Updated weights for policy 0, policy_version 491792 (0.0007) [2023-12-26 18:56:43,805][105620] Updated weights for policy 1, policy_version 492232 (0.0009) [2023-12-26 18:56:43,833][105692] Updated weights for policy 0, policy_version 491802 (0.0005) [2023-12-26 18:56:44,475][105692] Updated weights for policy 0, policy_version 491812 (0.0007) [2023-12-26 18:56:44,529][105692] Updated weights for policy 0, policy_version 491822 (0.0008) [2023-12-26 18:56:44,592][105692] Updated weights for policy 0, policy_version 491832 (0.0007) [2023-12-26 18:56:44,640][105620] Updated weights for policy 1, policy_version 492242 (0.0008) [2023-12-26 18:56:44,697][105620] Updated weights for policy 1, policy_version 492252 (0.0009) [2023-12-26 18:56:44,762][105620] Updated weights for policy 1, policy_version 492262 (0.0009) [2023-12-26 18:56:45,379][105692] Updated weights for policy 0, policy_version 491842 (0.0010) [2023-12-26 18:56:45,444][105692] Updated weights for policy 0, policy_version 491852 (0.0009) [2023-12-26 18:56:45,508][105692] Updated weights for policy 0, policy_version 491862 (0.0008) [2023-12-26 18:56:45,531][105620] Updated weights for policy 1, policy_version 492272 (0.0008) [2023-12-26 18:56:45,570][105692] Updated weights for policy 0, policy_version 491872 (0.0007) [2023-12-26 18:56:45,598][105620] Updated weights for policy 1, policy_version 492282 (0.0008) [2023-12-26 18:56:45,661][105620] Updated weights for policy 1, policy_version 492292 (0.0009) [2023-12-26 18:56:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 251977728. Throughput: 0: 9889.1, 1: 9796.9. Samples: 251948744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 18:56:46,062][104569] Avg episode reward: [(0, '9180.571'), (1, '9264.571')] [2023-12-26 18:56:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000491872_125935616.pth... [2023-12-26 18:56:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000492296_126042112.pth... [2023-12-26 18:56:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000490720_125640704.pth [2023-12-26 18:56:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000491176_125755392.pth [2023-12-26 18:56:46,287][105692] Updated weights for policy 0, policy_version 491882 (0.0009) [2023-12-26 18:56:46,349][105692] Updated weights for policy 0, policy_version 491892 (0.0010) [2023-12-26 18:56:46,405][105692] Updated weights for policy 0, policy_version 491902 (0.0005) [2023-12-26 18:56:46,423][105620] Updated weights for policy 1, policy_version 492302 (0.0008) [2023-12-26 18:56:46,472][105620] Updated weights for policy 1, policy_version 492312 (0.0009) [2023-12-26 18:56:46,530][105620] Updated weights for policy 1, policy_version 492322 (0.0008) [2023-12-26 18:56:47,108][105692] Updated weights for policy 0, policy_version 491912 (0.0010) [2023-12-26 18:56:47,163][105692] Updated weights for policy 0, policy_version 491922 (0.0010) [2023-12-26 18:56:47,212][105692] Updated weights for policy 0, policy_version 491932 (0.0010) [2023-12-26 18:56:47,335][105620] Updated weights for policy 1, policy_version 492332 (0.0008) [2023-12-26 18:56:47,384][105620] Updated weights for policy 1, policy_version 492342 (0.0008) [2023-12-26 18:56:47,436][105620] Updated weights for policy 1, policy_version 492352 (0.0007) [2023-12-26 18:56:47,970][105692] Updated weights for policy 0, policy_version 491942 (0.0009) [2023-12-26 18:56:48,030][105692] Updated weights for policy 0, policy_version 491952 (0.0009) [2023-12-26 18:56:48,082][105692] Updated weights for policy 0, policy_version 491962 (0.0006) [2023-12-26 18:56:48,223][105620] Updated weights for policy 1, policy_version 492362 (0.0008) [2023-12-26 18:56:48,285][105620] Updated weights for policy 1, policy_version 492372 (0.0010) [2023-12-26 18:56:48,343][105620] Updated weights for policy 1, policy_version 492382 (0.0009) [2023-12-26 18:56:48,407][105620] Updated weights for policy 1, policy_version 492392 (0.0008) [2023-12-26 18:56:48,753][105692] Updated weights for policy 0, policy_version 491972 (0.0007) [2023-12-26 18:56:48,820][105692] Updated weights for policy 0, policy_version 491982 (0.0011) [2023-12-26 18:56:48,880][105692] Updated weights for policy 0, policy_version 491992 (0.0011) [2023-12-26 18:56:49,230][105620] Updated weights for policy 1, policy_version 492402 (0.0009) [2023-12-26 18:56:49,296][105620] Updated weights for policy 1, policy_version 492412 (0.0009) [2023-12-26 18:56:49,365][105620] Updated weights for policy 1, policy_version 492422 (0.0009) [2023-12-26 18:56:49,508][105692] Updated weights for policy 0, policy_version 492002 (0.0010) [2023-12-26 18:56:49,571][105692] Updated weights for policy 0, policy_version 492012 (0.0008) [2023-12-26 18:56:49,630][105692] Updated weights for policy 0, policy_version 492023 (0.0011) [2023-12-26 18:56:50,072][105620] Updated weights for policy 1, policy_version 492432 (0.0009) [2023-12-26 18:56:50,136][105620] Updated weights for policy 1, policy_version 492442 (0.0009) [2023-12-26 18:56:50,194][105620] Updated weights for policy 1, policy_version 492452 (0.0009) [2023-12-26 18:56:50,373][105692] Updated weights for policy 0, policy_version 492033 (0.0010) [2023-12-26 18:56:50,435][105692] Updated weights for policy 0, policy_version 492043 (0.0009) [2023-12-26 18:56:50,491][105692] Updated weights for policy 0, policy_version 492053 (0.0009) [2023-12-26 18:56:50,559][105692] Updated weights for policy 0, policy_version 492063 (0.0009) [2023-12-26 18:56:50,934][105620] Updated weights for policy 1, policy_version 492462 (0.0008) [2023-12-26 18:56:51,001][105620] Updated weights for policy 1, policy_version 492472 (0.0008) [2023-12-26 18:56:51,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 252067840. Throughput: 0: 9889.9, 1: 9743.7. Samples: 252062944. Policy #0 lag: (min: 31.0, avg: 31.6, max: 51.0) [2023-12-26 18:56:51,062][104569] Avg episode reward: [(0, '8820.152'), (1, '9355.185')] [2023-12-26 18:56:51,067][105620] Updated weights for policy 1, policy_version 492482 (0.0009) [2023-12-26 18:56:51,315][105692] Updated weights for policy 0, policy_version 492073 (0.0008) [2023-12-26 18:56:51,378][105692] Updated weights for policy 0, policy_version 492083 (0.0008) [2023-12-26 18:56:51,434][105692] Updated weights for policy 0, policy_version 492093 (0.0008) [2023-12-26 18:56:51,847][105620] Updated weights for policy 1, policy_version 492492 (0.0008) [2023-12-26 18:56:51,899][105620] Updated weights for policy 1, policy_version 492502 (0.0009) [2023-12-26 18:56:51,950][105620] Updated weights for policy 1, policy_version 492512 (0.0008) [2023-12-26 18:56:52,219][105692] Updated weights for policy 0, policy_version 492103 (0.0006) [2023-12-26 18:56:52,242][105585] KL-divergence is very high: 106.1708 [2023-12-26 18:56:52,256][105585] KL-divergence is very high: 129.5571 [2023-12-26 18:56:52,270][105585] KL-divergence is very high: 136.2641 [2023-12-26 18:56:52,290][105692] Updated weights for policy 0, policy_version 492113 (0.0007) [2023-12-26 18:56:52,295][105585] KL-divergence is very high: 136.5653 [2023-12-26 18:56:52,307][105585] KL-divergence is very high: 115.6305 [2023-12-26 18:56:52,349][105692] Updated weights for policy 0, policy_version 492123 (0.0008) [2023-12-26 18:56:52,690][105620] Updated weights for policy 1, policy_version 492522 (0.0009) [2023-12-26 18:56:52,752][105620] Updated weights for policy 1, policy_version 492532 (0.0010) [2023-12-26 18:56:52,811][105620] Updated weights for policy 1, policy_version 492542 (0.0010) [2023-12-26 18:56:52,882][105620] Updated weights for policy 1, policy_version 492552 (0.0010) [2023-12-26 18:56:53,102][105692] Updated weights for policy 0, policy_version 492133 (0.0008) [2023-12-26 18:56:53,151][105692] Updated weights for policy 0, policy_version 492143 (0.0008) [2023-12-26 18:56:53,203][105692] Updated weights for policy 0, policy_version 492153 (0.0008) [2023-12-26 18:56:53,585][105620] Updated weights for policy 1, policy_version 492562 (0.0009) [2023-12-26 18:56:53,640][105620] Updated weights for policy 1, policy_version 492572 (0.0008) [2023-12-26 18:56:53,701][105620] Updated weights for policy 1, policy_version 492582 (0.0008) [2023-12-26 18:56:53,822][105692] Updated weights for policy 0, policy_version 492163 (0.0008) [2023-12-26 18:56:53,875][105692] Updated weights for policy 0, policy_version 492173 (0.0007) [2023-12-26 18:56:53,930][105692] Updated weights for policy 0, policy_version 492183 (0.0006) [2023-12-26 18:56:54,481][105620] Updated weights for policy 1, policy_version 492592 (0.0008) [2023-12-26 18:56:54,532][105692] Updated weights for policy 0, policy_version 492193 (0.0006) [2023-12-26 18:56:54,550][105620] Updated weights for policy 1, policy_version 492602 (0.0008) [2023-12-26 18:56:54,582][105692] Updated weights for policy 0, policy_version 492203 (0.0006) [2023-12-26 18:56:54,608][105620] Updated weights for policy 1, policy_version 492612 (0.0009) [2023-12-26 18:56:54,627][105692] Updated weights for policy 0, policy_version 492213 (0.0006) [2023-12-26 18:56:54,691][105692] Updated weights for policy 0, policy_version 492223 (0.0009) [2023-12-26 18:56:55,375][105692] Updated weights for policy 0, policy_version 492233 (0.0009) [2023-12-26 18:56:55,398][105620] Updated weights for policy 1, policy_version 492622 (0.0009) [2023-12-26 18:56:55,428][105692] Updated weights for policy 0, policy_version 492243 (0.0007) [2023-12-26 18:56:55,443][105620] Updated weights for policy 1, policy_version 492632 (0.0006) [2023-12-26 18:56:55,478][105692] Updated weights for policy 0, policy_version 492253 (0.0007) [2023-12-26 18:56:55,492][105620] Updated weights for policy 1, policy_version 492642 (0.0007) [2023-12-26 18:56:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 252166144. Throughput: 0: 9855.1, 1: 9565.8. Samples: 252177384. Policy #0 lag: (min: 31.0, avg: 31.6, max: 51.0) [2023-12-26 18:56:56,062][104569] Avg episode reward: [(0, '8456.961'), (1, '9355.077')] [2023-12-26 18:56:56,190][105692] Updated weights for policy 0, policy_version 492263 (0.0007) [2023-12-26 18:56:56,246][105620] Updated weights for policy 1, policy_version 492652 (0.0005) [2023-12-26 18:56:56,257][105692] Updated weights for policy 0, policy_version 492273 (0.0007) [2023-12-26 18:56:56,292][105620] Updated weights for policy 1, policy_version 492662 (0.0005) [2023-12-26 18:56:56,321][105692] Updated weights for policy 0, policy_version 492283 (0.0008) [2023-12-26 18:56:56,338][105620] Updated weights for policy 1, policy_version 492672 (0.0007) [2023-12-26 18:56:56,863][105692] Updated weights for policy 0, policy_version 492293 (0.0007) [2023-12-26 18:56:56,924][105692] Updated weights for policy 0, policy_version 492303 (0.0006) [2023-12-26 18:56:56,972][105692] Updated weights for policy 0, policy_version 492313 (0.0010) [2023-12-26 18:56:57,211][105620] Updated weights for policy 1, policy_version 492682 (0.0009) [2023-12-26 18:56:57,264][105620] Updated weights for policy 1, policy_version 492692 (0.0009) [2023-12-26 18:56:57,322][105620] Updated weights for policy 1, policy_version 492702 (0.0008) [2023-12-26 18:56:57,370][105620] Updated weights for policy 1, policy_version 492712 (0.0008) [2023-12-26 18:56:57,580][105692] Updated weights for policy 0, policy_version 492323 (0.0010) [2023-12-26 18:56:57,638][105692] Updated weights for policy 0, policy_version 492333 (0.0010) [2023-12-26 18:56:57,692][105692] Updated weights for policy 0, policy_version 492343 (0.0010) [2023-12-26 18:56:58,221][105620] Updated weights for policy 1, policy_version 492722 (0.0006) [2023-12-26 18:56:58,283][105620] Updated weights for policy 1, policy_version 492732 (0.0007) [2023-12-26 18:56:58,304][105692] Updated weights for policy 0, policy_version 492353 (0.0010) [2023-12-26 18:56:58,358][105620] Updated weights for policy 1, policy_version 492742 (0.0008) [2023-12-26 18:56:58,372][105692] Updated weights for policy 0, policy_version 492363 (0.0009) [2023-12-26 18:56:58,395][105585] KL-divergence is very high: 265.4528 [2023-12-26 18:56:58,441][105692] Updated weights for policy 0, policy_version 492373 (0.0008) [2023-12-26 18:56:58,448][105585] KL-divergence is very high: 687.9066 [2023-12-26 18:56:58,498][105585] KL-divergence is very high: 589.5104 [2023-12-26 18:56:58,506][105692] Updated weights for policy 0, policy_version 492383 (0.0010) [2023-12-26 18:56:59,178][105620] Updated weights for policy 1, policy_version 492752 (0.0006) [2023-12-26 18:56:59,251][105620] Updated weights for policy 1, policy_version 492762 (0.0009) [2023-12-26 18:56:59,315][105620] Updated weights for policy 1, policy_version 492772 (0.0006) [2023-12-26 18:56:59,372][105692] Updated weights for policy 0, policy_version 492393 (0.0008) [2023-12-26 18:56:59,426][105692] Updated weights for policy 0, policy_version 492403 (0.0009) [2023-12-26 18:56:59,478][105692] Updated weights for policy 0, policy_version 492413 (0.0009) [2023-12-26 18:56:59,926][105620] Updated weights for policy 1, policy_version 492782 (0.0007) [2023-12-26 18:56:59,993][105620] Updated weights for policy 1, policy_version 492792 (0.0007) [2023-12-26 18:57:00,046][105620] Updated weights for policy 1, policy_version 492802 (0.0005) [2023-12-26 18:57:00,293][105692] Updated weights for policy 0, policy_version 492423 (0.0010) [2023-12-26 18:57:00,348][105692] Updated weights for policy 0, policy_version 492433 (0.0011) [2023-12-26 18:57:00,400][105692] Updated weights for policy 0, policy_version 492443 (0.0010) [2023-12-26 18:57:00,675][105620] Updated weights for policy 1, policy_version 492812 (0.0006) [2023-12-26 18:57:00,726][105620] Updated weights for policy 1, policy_version 492822 (0.0007) [2023-12-26 18:57:00,785][105620] Updated weights for policy 1, policy_version 492832 (0.0008) [2023-12-26 18:57:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 252264448. Throughput: 0: 9960.8, 1: 9494.7. Samples: 252236080. Policy #0 lag: (min: 31.0, avg: 31.6, max: 51.0) [2023-12-26 18:57:01,062][104569] Avg episode reward: [(0, '8901.255'), (1, '9266.262')] [2023-12-26 18:57:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000492448_126083072.pth... [2023-12-26 18:57:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000492840_126181376.pth... [2023-12-26 18:57:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000491296_125788160.pth [2023-12-26 18:57:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000491752_125902848.pth [2023-12-26 18:57:01,165][105692] Updated weights for policy 0, policy_version 492453 (0.0009) [2023-12-26 18:57:01,226][105692] Updated weights for policy 0, policy_version 492463 (0.0010) [2023-12-26 18:57:01,288][105692] Updated weights for policy 0, policy_version 492473 (0.0010) [2023-12-26 18:57:01,562][105620] Updated weights for policy 1, policy_version 492842 (0.0008) [2023-12-26 18:57:01,622][105620] Updated weights for policy 1, policy_version 492852 (0.0008) [2023-12-26 18:57:01,687][105620] Updated weights for policy 1, policy_version 492862 (0.0008) [2023-12-26 18:57:01,759][105620] Updated weights for policy 1, policy_version 492872 (0.0009) [2023-12-26 18:57:01,943][105692] Updated weights for policy 0, policy_version 492483 (0.0009) [2023-12-26 18:57:02,004][105692] Updated weights for policy 0, policy_version 492493 (0.0008) [2023-12-26 18:57:02,056][105692] Updated weights for policy 0, policy_version 492503 (0.0005) [2023-12-26 18:57:02,524][105620] Updated weights for policy 1, policy_version 492882 (0.0005) [2023-12-26 18:57:02,584][105620] Updated weights for policy 1, policy_version 492892 (0.0006) [2023-12-26 18:57:02,649][105620] Updated weights for policy 1, policy_version 492902 (0.0010) [2023-12-26 18:57:02,671][105692] Updated weights for policy 0, policy_version 492513 (0.0005) [2023-12-26 18:57:02,739][105692] Updated weights for policy 0, policy_version 492523 (0.0008) [2023-12-26 18:57:02,800][105692] Updated weights for policy 0, policy_version 492533 (0.0009) [2023-12-26 18:57:02,864][105692] Updated weights for policy 0, policy_version 492543 (0.0009) [2023-12-26 18:57:03,372][105620] Updated weights for policy 1, policy_version 492912 (0.0009) [2023-12-26 18:57:03,419][105620] Updated weights for policy 1, policy_version 492922 (0.0006) [2023-12-26 18:57:03,465][105620] Updated weights for policy 1, policy_version 492932 (0.0005) [2023-12-26 18:57:03,539][105692] Updated weights for policy 0, policy_version 492553 (0.0009) [2023-12-26 18:57:03,605][105692] Updated weights for policy 0, policy_version 492563 (0.0009) [2023-12-26 18:57:03,660][105692] Updated weights for policy 0, policy_version 492573 (0.0008) [2023-12-26 18:57:04,193][105620] Updated weights for policy 1, policy_version 492942 (0.0008) [2023-12-26 18:57:04,247][105620] Updated weights for policy 1, policy_version 492952 (0.0009) [2023-12-26 18:57:04,310][105620] Updated weights for policy 1, policy_version 492962 (0.0006) [2023-12-26 18:57:04,342][105692] Updated weights for policy 0, policy_version 492583 (0.0008) [2023-12-26 18:57:04,375][105585] KL-divergence is very high: 348.9666 [2023-12-26 18:57:04,406][105692] Updated weights for policy 0, policy_version 492593 (0.0006) [2023-12-26 18:57:04,424][105585] KL-divergence is very high: 547.9105 [2023-12-26 18:57:04,469][105692] Updated weights for policy 0, policy_version 492603 (0.0008) [2023-12-26 18:57:04,475][105585] KL-divergence is very high: 438.2023 [2023-12-26 18:57:05,007][105620] Updated weights for policy 1, policy_version 492972 (0.0007) [2023-12-26 18:57:05,062][105620] Updated weights for policy 1, policy_version 492982 (0.0007) [2023-12-26 18:57:05,113][105620] Updated weights for policy 1, policy_version 492992 (0.0006) [2023-12-26 18:57:05,200][105692] Updated weights for policy 0, policy_version 492613 (0.0010) [2023-12-26 18:57:05,253][105692] Updated weights for policy 0, policy_version 492623 (0.0009) [2023-12-26 18:57:05,310][105692] Updated weights for policy 0, policy_version 492633 (0.0010) [2023-12-26 18:57:05,673][105620] Updated weights for policy 1, policy_version 493002 (0.0008) [2023-12-26 18:57:05,735][105620] Updated weights for policy 1, policy_version 493012 (0.0007) [2023-12-26 18:57:05,797][105620] Updated weights for policy 1, policy_version 493022 (0.0008) [2023-12-26 18:57:05,867][105620] Updated weights for policy 1, policy_version 493032 (0.0005) [2023-12-26 18:57:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 252362752. Throughput: 0: 9986.8, 1: 9435.2. Samples: 252352468. Policy #0 lag: (min: 31.0, avg: 31.6, max: 51.0) [2023-12-26 18:57:06,063][104569] Avg episode reward: [(0, '8545.225'), (1, '9086.294')] [2023-12-26 18:57:06,206][105692] Updated weights for policy 0, policy_version 492643 (0.0010) [2023-12-26 18:57:06,265][105692] Updated weights for policy 0, policy_version 492653 (0.0009) [2023-12-26 18:57:06,325][105692] Updated weights for policy 0, policy_version 492663 (0.0008) [2023-12-26 18:57:06,494][105620] Updated weights for policy 1, policy_version 493042 (0.0009) [2023-12-26 18:57:06,556][105620] Updated weights for policy 1, policy_version 493052 (0.0009) [2023-12-26 18:57:06,625][105620] Updated weights for policy 1, policy_version 493062 (0.0009) [2023-12-26 18:57:06,956][105692] Updated weights for policy 0, policy_version 492673 (0.0006) [2023-12-26 18:57:07,022][105692] Updated weights for policy 0, policy_version 492683 (0.0008) [2023-12-26 18:57:07,087][105692] Updated weights for policy 0, policy_version 492693 (0.0008) [2023-12-26 18:57:07,150][105692] Updated weights for policy 0, policy_version 492703 (0.0008) [2023-12-26 18:57:07,384][105620] Updated weights for policy 1, policy_version 493072 (0.0006) [2023-12-26 18:57:07,432][105620] Updated weights for policy 1, policy_version 493082 (0.0006) [2023-12-26 18:57:07,484][105620] Updated weights for policy 1, policy_version 493092 (0.0005) [2023-12-26 18:57:07,907][105692] Updated weights for policy 0, policy_version 492713 (0.0010) [2023-12-26 18:57:07,961][105692] Updated weights for policy 0, policy_version 492723 (0.0008) [2023-12-26 18:57:08,017][105692] Updated weights for policy 0, policy_version 492733 (0.0008) [2023-12-26 18:57:08,075][105620] Updated weights for policy 1, policy_version 493102 (0.0007) [2023-12-26 18:57:08,148][105620] Updated weights for policy 1, policy_version 493112 (0.0009) [2023-12-26 18:57:08,217][105620] Updated weights for policy 1, policy_version 493122 (0.0008) [2023-12-26 18:57:08,757][105692] Updated weights for policy 0, policy_version 492743 (0.0006) [2023-12-26 18:57:08,815][105692] Updated weights for policy 0, policy_version 492753 (0.0005) [2023-12-26 18:57:08,871][105692] Updated weights for policy 0, policy_version 492763 (0.0005) [2023-12-26 18:57:08,879][105620] Updated weights for policy 1, policy_version 493132 (0.0010) [2023-12-26 18:57:08,927][105620] Updated weights for policy 1, policy_version 493142 (0.0010) [2023-12-26 18:57:08,986][105620] Updated weights for policy 1, policy_version 493152 (0.0011) [2023-12-26 18:57:09,605][105692] Updated weights for policy 0, policy_version 492773 (0.0007) [2023-12-26 18:57:09,652][105692] Updated weights for policy 0, policy_version 492783 (0.0009) [2023-12-26 18:57:09,706][105692] Updated weights for policy 0, policy_version 492793 (0.0008) [2023-12-26 18:57:09,786][105620] Updated weights for policy 1, policy_version 493162 (0.0010) [2023-12-26 18:57:09,849][105620] Updated weights for policy 1, policy_version 493172 (0.0007) [2023-12-26 18:57:09,911][105620] Updated weights for policy 1, policy_version 493182 (0.0009) [2023-12-26 18:57:09,979][105620] Updated weights for policy 1, policy_version 493192 (0.0009) [2023-12-26 18:57:10,499][105692] Updated weights for policy 0, policy_version 492803 (0.0009) [2023-12-26 18:57:10,550][105692] Updated weights for policy 0, policy_version 492813 (0.0009) [2023-12-26 18:57:10,606][105692] Updated weights for policy 0, policy_version 492824 (0.0009) [2023-12-26 18:57:10,755][105620] Updated weights for policy 1, policy_version 493202 (0.0006) [2023-12-26 18:57:10,820][105620] Updated weights for policy 1, policy_version 493212 (0.0009) [2023-12-26 18:57:10,864][105620] Updated weights for policy 1, policy_version 493222 (0.0008) [2023-12-26 18:57:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 252461056. Throughput: 0: 9961.1, 1: 9492.2. Samples: 252468704. Policy #0 lag: (min: 31.0, avg: 31.6, max: 51.0) [2023-12-26 18:57:11,062][104569] Avg episode reward: [(0, '8819.864'), (1, '9265.209')] [2023-12-26 18:57:11,436][105692] Updated weights for policy 0, policy_version 492834 (0.0008) [2023-12-26 18:57:11,495][105692] Updated weights for policy 0, policy_version 492844 (0.0007) [2023-12-26 18:57:11,545][105692] Updated weights for policy 0, policy_version 492854 (0.0007) [2023-12-26 18:57:11,546][105620] Updated weights for policy 1, policy_version 493232 (0.0010) [2023-12-26 18:57:11,593][105692] Updated weights for policy 0, policy_version 492864 (0.0006) [2023-12-26 18:57:11,603][105620] Updated weights for policy 1, policy_version 493242 (0.0011) [2023-12-26 18:57:11,666][105620] Updated weights for policy 1, policy_version 493252 (0.0010) [2023-12-26 18:57:12,405][105692] Updated weights for policy 0, policy_version 492874 (0.0009) [2023-12-26 18:57:12,420][105620] Updated weights for policy 1, policy_version 493262 (0.0009) [2023-12-26 18:57:12,463][105692] Updated weights for policy 0, policy_version 492884 (0.0006) [2023-12-26 18:57:12,479][105620] Updated weights for policy 1, policy_version 493272 (0.0010) [2023-12-26 18:57:12,525][105692] Updated weights for policy 0, policy_version 492894 (0.0007) [2023-12-26 18:57:12,531][105620] Updated weights for policy 1, policy_version 493282 (0.0010) [2023-12-26 18:57:13,195][105620] Updated weights for policy 1, policy_version 493292 (0.0010) [2023-12-26 18:57:13,249][105620] Updated weights for policy 1, policy_version 493302 (0.0010) [2023-12-26 18:57:13,282][105692] Updated weights for policy 0, policy_version 492904 (0.0009) [2023-12-26 18:57:13,300][105620] Updated weights for policy 1, policy_version 493312 (0.0010) [2023-12-26 18:57:13,337][105692] Updated weights for policy 0, policy_version 492914 (0.0010) [2023-12-26 18:57:13,392][105692] Updated weights for policy 0, policy_version 492924 (0.0008) [2023-12-26 18:57:14,034][105620] Updated weights for policy 1, policy_version 493322 (0.0010) [2023-12-26 18:57:14,062][105692] Updated weights for policy 0, policy_version 492934 (0.0006) [2023-12-26 18:57:14,085][105620] Updated weights for policy 1, policy_version 493332 (0.0010) [2023-12-26 18:57:14,128][105692] Updated weights for policy 0, policy_version 492944 (0.0005) [2023-12-26 18:57:14,147][105620] Updated weights for policy 1, policy_version 493342 (0.0010) [2023-12-26 18:57:14,196][105692] Updated weights for policy 0, policy_version 492954 (0.0005) [2023-12-26 18:57:14,209][105620] Updated weights for policy 1, policy_version 493352 (0.0010) [2023-12-26 18:57:14,686][105692] Updated weights for policy 0, policy_version 492964 (0.0005) [2023-12-26 18:57:14,740][105692] Updated weights for policy 0, policy_version 492974 (0.0005) [2023-12-26 18:57:14,799][105692] Updated weights for policy 0, policy_version 492984 (0.0007) [2023-12-26 18:57:14,920][105620] Updated weights for policy 1, policy_version 493362 (0.0011) [2023-12-26 18:57:14,986][105620] Updated weights for policy 1, policy_version 493372 (0.0011) [2023-12-26 18:57:15,055][105620] Updated weights for policy 1, policy_version 493382 (0.0011) [2023-12-26 18:57:15,446][105692] Updated weights for policy 0, policy_version 492994 (0.0008) [2023-12-26 18:57:15,495][105692] Updated weights for policy 0, policy_version 493004 (0.0008) [2023-12-26 18:57:15,548][105692] Updated weights for policy 0, policy_version 493014 (0.0008) [2023-12-26 18:57:15,607][105692] Updated weights for policy 0, policy_version 493024 (0.0008) [2023-12-26 18:57:15,792][105620] Updated weights for policy 1, policy_version 493392 (0.0011) [2023-12-26 18:57:15,861][105620] Updated weights for policy 1, policy_version 493402 (0.0011) [2023-12-26 18:57:15,927][105620] Updated weights for policy 1, policy_version 493412 (0.0010) [2023-12-26 18:57:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 252559360. Throughput: 0: 9822.7, 1: 9461.2. Samples: 252525028. Policy #0 lag: (min: 31.0, avg: 31.6, max: 51.0) [2023-12-26 18:57:16,062][104569] Avg episode reward: [(0, '9001.144'), (1, '9355.230')] [2023-12-26 18:57:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000493024_126230528.pth... [2023-12-26 18:57:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000493416_126328832.pth... [2023-12-26 18:57:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000491872_125935616.pth [2023-12-26 18:57:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000492296_126042112.pth [2023-12-26 18:57:16,321][105692] Updated weights for policy 0, policy_version 493034 (0.0008) [2023-12-26 18:57:16,382][105692] Updated weights for policy 0, policy_version 493044 (0.0007) [2023-12-26 18:57:16,442][105692] Updated weights for policy 0, policy_version 493054 (0.0008) [2023-12-26 18:57:16,629][105620] Updated weights for policy 1, policy_version 493422 (0.0007) [2023-12-26 18:57:16,674][105620] Updated weights for policy 1, policy_version 493432 (0.0005) [2023-12-26 18:57:16,718][105620] Updated weights for policy 1, policy_version 493442 (0.0005) [2023-12-26 18:57:17,157][105692] Updated weights for policy 0, policy_version 493064 (0.0010) [2023-12-26 18:57:17,205][105692] Updated weights for policy 0, policy_version 493074 (0.0010) [2023-12-26 18:57:17,253][105692] Updated weights for policy 0, policy_version 493084 (0.0010) [2023-12-26 18:57:17,331][105620] Updated weights for policy 1, policy_version 493452 (0.0007) [2023-12-26 18:57:17,392][105620] Updated weights for policy 1, policy_version 493462 (0.0010) [2023-12-26 18:57:17,447][105620] Updated weights for policy 1, policy_version 493472 (0.0010) [2023-12-26 18:57:17,950][105692] Updated weights for policy 0, policy_version 493094 (0.0010) [2023-12-26 18:57:18,016][105692] Updated weights for policy 0, policy_version 493104 (0.0009) [2023-12-26 18:57:18,080][105692] Updated weights for policy 0, policy_version 493114 (0.0005) [2023-12-26 18:57:18,188][105620] Updated weights for policy 1, policy_version 493482 (0.0010) [2023-12-26 18:57:18,243][105620] Updated weights for policy 1, policy_version 493492 (0.0010) [2023-12-26 18:57:18,302][105620] Updated weights for policy 1, policy_version 493502 (0.0010) [2023-12-26 18:57:18,370][105620] Updated weights for policy 1, policy_version 493512 (0.0010) [2023-12-26 18:57:18,752][105692] Updated weights for policy 0, policy_version 493124 (0.0007) [2023-12-26 18:57:18,809][105692] Updated weights for policy 0, policy_version 493134 (0.0008) [2023-12-26 18:57:18,867][105692] Updated weights for policy 0, policy_version 493144 (0.0008) [2023-12-26 18:57:19,137][105620] Updated weights for policy 1, policy_version 493522 (0.0010) [2023-12-26 18:57:19,185][105620] Updated weights for policy 1, policy_version 493532 (0.0010) [2023-12-26 18:57:19,248][105620] Updated weights for policy 1, policy_version 493542 (0.0009) [2023-12-26 18:57:19,675][105692] Updated weights for policy 0, policy_version 493154 (0.0009) [2023-12-26 18:57:19,742][105692] Updated weights for policy 0, policy_version 493164 (0.0011) [2023-12-26 18:57:19,810][105692] Updated weights for policy 0, policy_version 493174 (0.0011) [2023-12-26 18:57:19,870][105692] Updated weights for policy 0, policy_version 493184 (0.0009) [2023-12-26 18:57:19,872][105620] Updated weights for policy 1, policy_version 493552 (0.0010) [2023-12-26 18:57:19,935][105620] Updated weights for policy 1, policy_version 493562 (0.0010) [2023-12-26 18:57:19,995][105620] Updated weights for policy 1, policy_version 493572 (0.0010) [2023-12-26 18:57:20,508][105692] Updated weights for policy 0, policy_version 493194 (0.0011) [2023-12-26 18:57:20,573][105692] Updated weights for policy 0, policy_version 493204 (0.0011) [2023-12-26 18:57:20,636][105692] Updated weights for policy 0, policy_version 493214 (0.0011) [2023-12-26 18:57:20,765][105620] Updated weights for policy 1, policy_version 493582 (0.0011) [2023-12-26 18:57:20,831][105620] Updated weights for policy 1, policy_version 493592 (0.0011) [2023-12-26 18:57:20,890][105620] Updated weights for policy 1, policy_version 493602 (0.0010) [2023-12-26 18:57:21,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 252657664. Throughput: 0: 9839.9, 1: 9452.0. Samples: 252645448. Policy #0 lag: (min: 31.0, avg: 31.6, max: 51.0) [2023-12-26 18:57:21,063][104569] Avg episode reward: [(0, '9087.632'), (1, '9355.108')] [2023-12-26 18:57:21,401][105692] Updated weights for policy 0, policy_version 493224 (0.0009) [2023-12-26 18:57:21,447][105692] Updated weights for policy 0, policy_version 493234 (0.0007) [2023-12-26 18:57:21,497][105692] Updated weights for policy 0, policy_version 493244 (0.0010) [2023-12-26 18:57:21,654][105620] Updated weights for policy 1, policy_version 493612 (0.0010) [2023-12-26 18:57:21,725][105620] Updated weights for policy 1, policy_version 493622 (0.0008) [2023-12-26 18:57:21,784][105620] Updated weights for policy 1, policy_version 493632 (0.0008) [2023-12-26 18:57:22,309][105692] Updated weights for policy 0, policy_version 493254 (0.0009) [2023-12-26 18:57:22,372][105692] Updated weights for policy 0, policy_version 493264 (0.0009) [2023-12-26 18:57:22,428][105692] Updated weights for policy 0, policy_version 493274 (0.0008) [2023-12-26 18:57:22,542][105620] Updated weights for policy 1, policy_version 493642 (0.0009) [2023-12-26 18:57:22,604][105620] Updated weights for policy 1, policy_version 493652 (0.0011) [2023-12-26 18:57:22,674][105620] Updated weights for policy 1, policy_version 493662 (0.0011) [2023-12-26 18:57:22,734][105620] Updated weights for policy 1, policy_version 493672 (0.0011) [2023-12-26 18:57:23,214][105692] Updated weights for policy 0, policy_version 493284 (0.0008) [2023-12-26 18:57:23,275][105692] Updated weights for policy 0, policy_version 493294 (0.0010) [2023-12-26 18:57:23,333][105692] Updated weights for policy 0, policy_version 493304 (0.0010) [2023-12-26 18:57:23,422][105620] Updated weights for policy 1, policy_version 493682 (0.0005) [2023-12-26 18:57:23,485][105620] Updated weights for policy 1, policy_version 493692 (0.0006) [2023-12-26 18:57:23,544][105620] Updated weights for policy 1, policy_version 493702 (0.0006) [2023-12-26 18:57:24,039][105620] Updated weights for policy 1, policy_version 493712 (0.0006) [2023-12-26 18:57:24,105][105620] Updated weights for policy 1, policy_version 493722 (0.0009) [2023-12-26 18:57:24,160][105620] Updated weights for policy 1, policy_version 493732 (0.0009) [2023-12-26 18:57:24,195][105692] Updated weights for policy 0, policy_version 493315 (0.0010) [2023-12-26 18:57:24,257][105692] Updated weights for policy 0, policy_version 493325 (0.0009) [2023-12-26 18:57:24,322][105692] Updated weights for policy 0, policy_version 493335 (0.0009) [2023-12-26 18:57:24,813][105620] Updated weights for policy 1, policy_version 493742 (0.0006) [2023-12-26 18:57:24,869][105620] Updated weights for policy 1, policy_version 493752 (0.0007) [2023-12-26 18:57:24,922][105620] Updated weights for policy 1, policy_version 493762 (0.0009) [2023-12-26 18:57:25,151][105692] Updated weights for policy 0, policy_version 493345 (0.0010) [2023-12-26 18:57:25,217][105692] Updated weights for policy 0, policy_version 493355 (0.0009) [2023-12-26 18:57:25,282][105692] Updated weights for policy 0, policy_version 493365 (0.0009) [2023-12-26 18:57:25,347][105692] Updated weights for policy 0, policy_version 493375 (0.0009) [2023-12-26 18:57:25,593][105620] Updated weights for policy 1, policy_version 493772 (0.0008) [2023-12-26 18:57:25,655][105620] Updated weights for policy 1, policy_version 493782 (0.0005) [2023-12-26 18:57:25,719][105620] Updated weights for policy 1, policy_version 493792 (0.0008) [2023-12-26 18:57:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 252747776. Throughput: 0: 9681.1, 1: 9537.2. Samples: 252758460. Policy #0 lag: (min: 31.0, avg: 31.6, max: 51.0) [2023-12-26 18:57:26,063][104569] Avg episode reward: [(0, '8909.576'), (1, '9354.948')] [2023-12-26 18:57:26,094][105692] Updated weights for policy 0, policy_version 493385 (0.0006) [2023-12-26 18:57:26,146][105692] Updated weights for policy 0, policy_version 493395 (0.0009) [2023-12-26 18:57:26,197][105692] Updated weights for policy 0, policy_version 493405 (0.0009) [2023-12-26 18:57:26,351][105620] Updated weights for policy 1, policy_version 493802 (0.0009) [2023-12-26 18:57:26,418][105620] Updated weights for policy 1, policy_version 493812 (0.0005) [2023-12-26 18:57:26,485][105620] Updated weights for policy 1, policy_version 493822 (0.0010) [2023-12-26 18:57:26,545][105620] Updated weights for policy 1, policy_version 493832 (0.0010) [2023-12-26 18:57:26,954][105692] Updated weights for policy 0, policy_version 493415 (0.0009) [2023-12-26 18:57:27,016][105692] Updated weights for policy 0, policy_version 493425 (0.0006) [2023-12-26 18:57:27,088][105692] Updated weights for policy 0, policy_version 493435 (0.0005) [2023-12-26 18:57:27,197][105620] Updated weights for policy 1, policy_version 493842 (0.0009) [2023-12-26 18:57:27,244][105620] Updated weights for policy 1, policy_version 493852 (0.0009) [2023-12-26 18:57:27,292][105620] Updated weights for policy 1, policy_version 493862 (0.0009) [2023-12-26 18:57:27,645][105692] Updated weights for policy 0, policy_version 493445 (0.0007) [2023-12-26 18:57:27,699][105692] Updated weights for policy 0, policy_version 493455 (0.0009) [2023-12-26 18:57:27,753][105692] Updated weights for policy 0, policy_version 493465 (0.0009) [2023-12-26 18:57:28,108][105620] Updated weights for policy 1, policy_version 493872 (0.0009) [2023-12-26 18:57:28,161][105620] Updated weights for policy 1, policy_version 493882 (0.0010) [2023-12-26 18:57:28,223][105620] Updated weights for policy 1, policy_version 493892 (0.0010) [2023-12-26 18:57:28,404][105692] Updated weights for policy 0, policy_version 493475 (0.0008) [2023-12-26 18:57:28,458][105692] Updated weights for policy 0, policy_version 493485 (0.0008) [2023-12-26 18:57:28,517][105692] Updated weights for policy 0, policy_version 493495 (0.0007) [2023-12-26 18:57:28,959][105620] Updated weights for policy 1, policy_version 493902 (0.0010) [2023-12-26 18:57:29,013][105620] Updated weights for policy 1, policy_version 493912 (0.0010) [2023-12-26 18:57:29,069][105620] Updated weights for policy 1, policy_version 493922 (0.0009) [2023-12-26 18:57:29,250][105692] Updated weights for policy 0, policy_version 493505 (0.0008) [2023-12-26 18:57:29,320][105692] Updated weights for policy 0, policy_version 493515 (0.0009) [2023-12-26 18:57:29,389][105692] Updated weights for policy 0, policy_version 493525 (0.0009) [2023-12-26 18:57:29,446][105692] Updated weights for policy 0, policy_version 493535 (0.0009) [2023-12-26 18:57:29,779][105620] Updated weights for policy 1, policy_version 493932 (0.0009) [2023-12-26 18:57:29,844][105620] Updated weights for policy 1, policy_version 493942 (0.0009) [2023-12-26 18:57:29,911][105620] Updated weights for policy 1, policy_version 493952 (0.0010) [2023-12-26 18:57:30,203][105692] Updated weights for policy 0, policy_version 493545 (0.0010) [2023-12-26 18:57:30,258][105692] Updated weights for policy 0, policy_version 493555 (0.0011) [2023-12-26 18:57:30,309][105692] Updated weights for policy 0, policy_version 493565 (0.0008) [2023-12-26 18:57:30,574][105620] Updated weights for policy 1, policy_version 493962 (0.0009) [2023-12-26 18:57:30,635][105620] Updated weights for policy 1, policy_version 493972 (0.0010) [2023-12-26 18:57:30,692][105620] Updated weights for policy 1, policy_version 493982 (0.0010) [2023-12-26 18:57:30,749][105620] Updated weights for policy 1, policy_version 493992 (0.0010) [2023-12-26 18:57:31,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19251.2, 300 sec: 19438.7). Total num frames: 252846080. Throughput: 0: 9727.1, 1: 9603.1. Samples: 252818604. Policy #0 lag: (min: 31.0, avg: 31.6, max: 51.0) [2023-12-26 18:57:31,062][104569] Avg episode reward: [(0, '8817.783'), (1, '9354.878')] [2023-12-26 18:57:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000493568_126369792.pth... [2023-12-26 18:57:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000493992_126476288.pth... [2023-12-26 18:57:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000492840_126181376.pth [2023-12-26 18:57:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000492448_126083072.pth [2023-12-26 18:57:31,129][105692] Updated weights for policy 0, policy_version 493575 (0.0009) [2023-12-26 18:57:31,186][105692] Updated weights for policy 0, policy_version 493585 (0.0009) [2023-12-26 18:57:31,243][105692] Updated weights for policy 0, policy_version 493595 (0.0009) [2023-12-26 18:57:31,440][105620] Updated weights for policy 1, policy_version 494002 (0.0010) [2023-12-26 18:57:31,510][105620] Updated weights for policy 1, policy_version 494012 (0.0010) [2023-12-26 18:57:31,567][105620] Updated weights for policy 1, policy_version 494022 (0.0010) [2023-12-26 18:57:31,925][105692] Updated weights for policy 0, policy_version 493605 (0.0007) [2023-12-26 18:57:31,977][105692] Updated weights for policy 0, policy_version 493615 (0.0008) [2023-12-26 18:57:32,037][105692] Updated weights for policy 0, policy_version 493625 (0.0009) [2023-12-26 18:57:32,339][105620] Updated weights for policy 1, policy_version 494032 (0.0009) [2023-12-26 18:57:32,406][105620] Updated weights for policy 1, policy_version 494042 (0.0010) [2023-12-26 18:57:32,464][105620] Updated weights for policy 1, policy_version 494052 (0.0009) [2023-12-26 18:57:32,711][105692] Updated weights for policy 0, policy_version 493635 (0.0007) [2023-12-26 18:57:32,766][105692] Updated weights for policy 0, policy_version 493645 (0.0009) [2023-12-26 18:57:32,819][105692] Updated weights for policy 0, policy_version 493655 (0.0009) [2023-12-26 18:57:33,286][105620] Updated weights for policy 1, policy_version 494062 (0.0009) [2023-12-26 18:57:33,345][105620] Updated weights for policy 1, policy_version 494072 (0.0008) [2023-12-26 18:57:33,401][105620] Updated weights for policy 1, policy_version 494082 (0.0010) [2023-12-26 18:57:33,433][105692] Updated weights for policy 0, policy_version 493665 (0.0009) [2023-12-26 18:57:33,491][105692] Updated weights for policy 0, policy_version 493675 (0.0006) [2023-12-26 18:57:33,548][105692] Updated weights for policy 0, policy_version 493685 (0.0008) [2023-12-26 18:57:33,593][105692] Updated weights for policy 0, policy_version 493695 (0.0009) [2023-12-26 18:57:34,170][105620] Updated weights for policy 1, policy_version 494092 (0.0006) [2023-12-26 18:57:34,215][105692] Updated weights for policy 0, policy_version 493705 (0.0006) [2023-12-26 18:57:34,238][105620] Updated weights for policy 1, policy_version 494102 (0.0007) [2023-12-26 18:57:34,284][105692] Updated weights for policy 0, policy_version 493715 (0.0006) [2023-12-26 18:57:34,297][105620] Updated weights for policy 1, policy_version 494112 (0.0008) [2023-12-26 18:57:34,350][105692] Updated weights for policy 0, policy_version 493725 (0.0006) [2023-12-26 18:57:34,880][105620] Updated weights for policy 1, policy_version 494122 (0.0007) [2023-12-26 18:57:34,933][105620] Updated weights for policy 1, policy_version 494132 (0.0009) [2023-12-26 18:57:34,979][105620] Updated weights for policy 1, policy_version 494142 (0.0009) [2023-12-26 18:57:35,026][105620] Updated weights for policy 1, policy_version 494152 (0.0008) [2023-12-26 18:57:35,110][105692] Updated weights for policy 0, policy_version 493735 (0.0009) [2023-12-26 18:57:35,160][105692] Updated weights for policy 0, policy_version 493745 (0.0009) [2023-12-26 18:57:35,215][105692] Updated weights for policy 0, policy_version 493755 (0.0009) [2023-12-26 18:57:35,715][105620] Updated weights for policy 1, policy_version 494162 (0.0010) [2023-12-26 18:57:35,762][105620] Updated weights for policy 1, policy_version 494172 (0.0010) [2023-12-26 18:57:35,813][105620] Updated weights for policy 1, policy_version 494182 (0.0010) [2023-12-26 18:57:35,958][105692] Updated weights for policy 0, policy_version 493765 (0.0009) [2023-12-26 18:57:36,014][105692] Updated weights for policy 0, policy_version 493775 (0.0010) [2023-12-26 18:57:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 252944384. Throughput: 0: 9722.5, 1: 9681.8. Samples: 252936136. Policy #0 lag: (min: 31.0, avg: 31.6, max: 51.0) [2023-12-26 18:57:36,062][104569] Avg episode reward: [(0, '8637.836'), (1, '9264.428')] [2023-12-26 18:57:36,077][105692] Updated weights for policy 0, policy_version 493785 (0.0010) [2023-12-26 18:57:36,443][105620] Updated weights for policy 1, policy_version 494192 (0.0010) [2023-12-26 18:57:36,511][105620] Updated weights for policy 1, policy_version 494202 (0.0011) [2023-12-26 18:57:36,576][105620] Updated weights for policy 1, policy_version 494212 (0.0010) [2023-12-26 18:57:36,909][105692] Updated weights for policy 0, policy_version 493795 (0.0008) [2023-12-26 18:57:36,965][105692] Updated weights for policy 0, policy_version 493805 (0.0009) [2023-12-26 18:57:37,024][105692] Updated weights for policy 0, policy_version 493815 (0.0009) [2023-12-26 18:57:37,322][105620] Updated weights for policy 1, policy_version 494222 (0.0009) [2023-12-26 18:57:37,383][105620] Updated weights for policy 1, policy_version 494232 (0.0008) [2023-12-26 18:57:37,445][105620] Updated weights for policy 1, policy_version 494242 (0.0007) [2023-12-26 18:57:37,821][105692] Updated weights for policy 0, policy_version 493825 (0.0009) [2023-12-26 18:57:37,872][105692] Updated weights for policy 0, policy_version 493835 (0.0009) [2023-12-26 18:57:37,921][105692] Updated weights for policy 0, policy_version 493845 (0.0009) [2023-12-26 18:57:37,982][105692] Updated weights for policy 0, policy_version 493855 (0.0008) [2023-12-26 18:57:38,151][105620] Updated weights for policy 1, policy_version 494252 (0.0006) [2023-12-26 18:57:38,206][105620] Updated weights for policy 1, policy_version 494262 (0.0006) [2023-12-26 18:57:38,269][105620] Updated weights for policy 1, policy_version 494272 (0.0006) [2023-12-26 18:57:38,770][105692] Updated weights for policy 0, policy_version 493865 (0.0008) [2023-12-26 18:57:38,828][105692] Updated weights for policy 0, policy_version 493875 (0.0010) [2023-12-26 18:57:38,883][105692] Updated weights for policy 0, policy_version 493885 (0.0010) [2023-12-26 18:57:38,943][105620] Updated weights for policy 1, policy_version 494282 (0.0006) [2023-12-26 18:57:39,011][105620] Updated weights for policy 1, policy_version 494292 (0.0009) [2023-12-26 18:57:39,071][105620] Updated weights for policy 1, policy_version 494302 (0.0007) [2023-12-26 18:57:39,141][105620] Updated weights for policy 1, policy_version 494312 (0.0006) [2023-12-26 18:57:39,706][105692] Updated weights for policy 0, policy_version 493895 (0.0007) [2023-12-26 18:57:39,740][105585] KL-divergence is very high: 116.1092 [2023-12-26 18:57:39,774][105692] Updated weights for policy 0, policy_version 493905 (0.0006) [2023-12-26 18:57:39,797][105585] KL-divergence is very high: 154.2337 [2023-12-26 18:57:39,830][105620] Updated weights for policy 1, policy_version 494322 (0.0008) [2023-12-26 18:57:39,847][105692] Updated weights for policy 0, policy_version 493915 (0.0007) [2023-12-26 18:57:39,854][105585] KL-divergence is very high: 141.9311 [2023-12-26 18:57:39,895][105620] Updated weights for policy 1, policy_version 494332 (0.0007) [2023-12-26 18:57:39,958][105620] Updated weights for policy 1, policy_version 494342 (0.0009) [2023-12-26 18:57:40,568][105692] Updated weights for policy 0, policy_version 493925 (0.0007) [2023-12-26 18:57:40,596][105620] Updated weights for policy 1, policy_version 494352 (0.0008) [2023-12-26 18:57:40,628][105692] Updated weights for policy 0, policy_version 493935 (0.0008) [2023-12-26 18:57:40,649][105620] Updated weights for policy 1, policy_version 494362 (0.0007) [2023-12-26 18:57:40,686][105692] Updated weights for policy 0, policy_version 493945 (0.0010) [2023-12-26 18:57:40,699][105620] Updated weights for policy 1, policy_version 494372 (0.0008) [2023-12-26 18:57:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 253042688. Throughput: 0: 9609.7, 1: 9805.8. Samples: 253051084. Policy #0 lag: (min: 31.0, avg: 31.6, max: 51.0) [2023-12-26 18:57:41,063][104569] Avg episode reward: [(0, '8995.238'), (1, '9355.233')] [2023-12-26 18:57:41,345][105692] Updated weights for policy 0, policy_version 493955 (0.0009) [2023-12-26 18:57:41,369][105620] Updated weights for policy 1, policy_version 494382 (0.0009) [2023-12-26 18:57:41,411][105692] Updated weights for policy 0, policy_version 493965 (0.0007) [2023-12-26 18:57:41,433][105620] Updated weights for policy 1, policy_version 494392 (0.0009) [2023-12-26 18:57:41,471][105692] Updated weights for policy 0, policy_version 493975 (0.0006) [2023-12-26 18:57:41,494][105620] Updated weights for policy 1, policy_version 494402 (0.0009) [2023-12-26 18:57:42,111][105692] Updated weights for policy 0, policy_version 493985 (0.0006) [2023-12-26 18:57:42,173][105692] Updated weights for policy 0, policy_version 493995 (0.0009) [2023-12-26 18:57:42,230][105692] Updated weights for policy 0, policy_version 494005 (0.0009) [2023-12-26 18:57:42,290][105692] Updated weights for policy 0, policy_version 494015 (0.0009) [2023-12-26 18:57:42,311][105620] Updated weights for policy 1, policy_version 494412 (0.0009) [2023-12-26 18:57:42,375][105620] Updated weights for policy 1, policy_version 494422 (0.0009) [2023-12-26 18:57:42,428][105620] Updated weights for policy 1, policy_version 494432 (0.0007) [2023-12-26 18:57:42,905][105692] Updated weights for policy 0, policy_version 494025 (0.0009) [2023-12-26 18:57:42,961][105692] Updated weights for policy 0, policy_version 494035 (0.0009) [2023-12-26 18:57:43,013][105692] Updated weights for policy 0, policy_version 494045 (0.0009) [2023-12-26 18:57:43,072][105620] Updated weights for policy 1, policy_version 494443 (0.0010) [2023-12-26 18:57:43,125][105620] Updated weights for policy 1, policy_version 494453 (0.0009) [2023-12-26 18:57:43,180][105620] Updated weights for policy 1, policy_version 494463 (0.0009) [2023-12-26 18:57:43,778][105692] Updated weights for policy 0, policy_version 494055 (0.0009) [2023-12-26 18:57:43,832][105692] Updated weights for policy 0, policy_version 494065 (0.0008) [2023-12-26 18:57:43,881][105692] Updated weights for policy 0, policy_version 494075 (0.0008) [2023-12-26 18:57:43,949][105620] Updated weights for policy 1, policy_version 494473 (0.0009) [2023-12-26 18:57:44,001][105620] Updated weights for policy 1, policy_version 494483 (0.0009) [2023-12-26 18:57:44,055][105620] Updated weights for policy 1, policy_version 494493 (0.0010) [2023-12-26 18:57:44,113][105620] Updated weights for policy 1, policy_version 494504 (0.0010) [2023-12-26 18:57:44,482][105692] Updated weights for policy 0, policy_version 494085 (0.0007) [2023-12-26 18:57:44,528][105692] Updated weights for policy 0, policy_version 494095 (0.0005) [2023-12-26 18:57:44,583][105692] Updated weights for policy 0, policy_version 494105 (0.0005) [2023-12-26 18:57:44,988][105620] Updated weights for policy 1, policy_version 494514 (0.0010) [2023-12-26 18:57:45,050][105620] Updated weights for policy 1, policy_version 494524 (0.0009) [2023-12-26 18:57:45,115][105620] Updated weights for policy 1, policy_version 494534 (0.0009) [2023-12-26 18:57:45,220][105692] Updated weights for policy 0, policy_version 494115 (0.0006) [2023-12-26 18:57:45,283][105692] Updated weights for policy 0, policy_version 494125 (0.0009) [2023-12-26 18:57:45,348][105692] Updated weights for policy 0, policy_version 494135 (0.0007) [2023-12-26 18:57:45,902][105620] Updated weights for policy 1, policy_version 494544 (0.0010) [2023-12-26 18:57:45,956][105620] Updated weights for policy 1, policy_version 494555 (0.0010) [2023-12-26 18:57:46,025][105620] Updated weights for policy 1, policy_version 494565 (0.0009) [2023-12-26 18:57:46,033][105692] Updated weights for policy 0, policy_version 494145 (0.0006) [2023-12-26 18:57:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 253140992. Throughput: 0: 9559.0, 1: 9862.3. Samples: 253110040. Policy #0 lag: (min: 31.0, avg: 31.6, max: 51.0) [2023-12-26 18:57:46,063][104569] Avg episode reward: [(0, '8814.971'), (1, '9355.328')] [2023-12-26 18:57:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000494568_126623744.pth... [2023-12-26 18:57:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000493416_126328832.pth [2023-12-26 18:57:46,094][105692] Updated weights for policy 0, policy_version 494155 (0.0007) [2023-12-26 18:57:46,149][105692] Updated weights for policy 0, policy_version 494165 (0.0007) [2023-12-26 18:57:46,219][105692] Updated weights for policy 0, policy_version 494175 (0.0009) [2023-12-26 18:57:46,226][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000494176_126525440.pth... [2023-12-26 18:57:46,231][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000493024_126230528.pth [2023-12-26 18:57:46,809][105620] Updated weights for policy 1, policy_version 494575 (0.0009) [2023-12-26 18:57:46,870][105620] Updated weights for policy 1, policy_version 494585 (0.0009) [2023-12-26 18:57:46,916][105620] Updated weights for policy 1, policy_version 494595 (0.0008) [2023-12-26 18:57:46,939][105692] Updated weights for policy 0, policy_version 494185 (0.0007) [2023-12-26 18:57:46,994][105692] Updated weights for policy 0, policy_version 494195 (0.0009) [2023-12-26 18:57:47,053][105692] Updated weights for policy 0, policy_version 494205 (0.0009) [2023-12-26 18:57:47,641][105620] Updated weights for policy 1, policy_version 494605 (0.0009) [2023-12-26 18:57:47,704][105620] Updated weights for policy 1, policy_version 494615 (0.0009) [2023-12-26 18:57:47,769][105620] Updated weights for policy 1, policy_version 494625 (0.0009) [2023-12-26 18:57:47,839][105692] Updated weights for policy 0, policy_version 494215 (0.0008) [2023-12-26 18:57:47,895][105692] Updated weights for policy 0, policy_version 494225 (0.0008) [2023-12-26 18:57:47,958][105692] Updated weights for policy 0, policy_version 494235 (0.0009) [2023-12-26 18:57:48,480][105620] Updated weights for policy 1, policy_version 494635 (0.0009) [2023-12-26 18:57:48,532][105620] Updated weights for policy 1, policy_version 494645 (0.0009) [2023-12-26 18:57:48,590][105620] Updated weights for policy 1, policy_version 494656 (0.0010) [2023-12-26 18:57:48,668][105692] Updated weights for policy 0, policy_version 494245 (0.0008) [2023-12-26 18:57:48,723][105692] Updated weights for policy 0, policy_version 494255 (0.0005) [2023-12-26 18:57:48,779][105692] Updated weights for policy 0, policy_version 494265 (0.0006) [2023-12-26 18:57:49,410][105692] Updated weights for policy 0, policy_version 494275 (0.0010) [2023-12-26 18:57:49,425][105620] Updated weights for policy 1, policy_version 494667 (0.0009) [2023-12-26 18:57:49,470][105692] Updated weights for policy 0, policy_version 494285 (0.0011) [2023-12-26 18:57:49,493][105620] Updated weights for policy 1, policy_version 494677 (0.0008) [2023-12-26 18:57:49,526][105692] Updated weights for policy 0, policy_version 494295 (0.0011) [2023-12-26 18:57:49,557][105620] Updated weights for policy 1, policy_version 494687 (0.0005) [2023-12-26 18:57:50,303][105692] Updated weights for policy 0, policy_version 494305 (0.0010) [2023-12-26 18:57:50,316][105620] Updated weights for policy 1, policy_version 494697 (0.0007) [2023-12-26 18:57:50,369][105692] Updated weights for policy 0, policy_version 494315 (0.0011) [2023-12-26 18:57:50,385][105620] Updated weights for policy 1, policy_version 494707 (0.0006) [2023-12-26 18:57:50,422][105692] Updated weights for policy 0, policy_version 494325 (0.0011) [2023-12-26 18:57:50,445][105620] Updated weights for policy 1, policy_version 494717 (0.0006) [2023-12-26 18:57:50,475][105692] Updated weights for policy 0, policy_version 494335 (0.0010) [2023-12-26 18:57:50,507][105620] Updated weights for policy 1, policy_version 494727 (0.0007) [2023-12-26 18:57:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 253231104. Throughput: 0: 9622.4, 1: 9752.5. Samples: 253224340. Policy #0 lag: (min: 31.0, avg: 31.6, max: 51.0) [2023-12-26 18:57:51,062][104569] Avg episode reward: [(0, '8816.088'), (1, '9264.145')] [2023-12-26 18:57:51,243][105692] Updated weights for policy 0, policy_version 494345 (0.0008) [2023-12-26 18:57:51,256][105620] Updated weights for policy 1, policy_version 494737 (0.0008) [2023-12-26 18:57:51,310][105692] Updated weights for policy 0, policy_version 494355 (0.0006) [2023-12-26 18:57:51,322][105620] Updated weights for policy 1, policy_version 494747 (0.0007) [2023-12-26 18:57:51,376][105692] Updated weights for policy 0, policy_version 494365 (0.0007) [2023-12-26 18:57:51,398][105620] Updated weights for policy 1, policy_version 494757 (0.0007) [2023-12-26 18:57:52,062][105620] Updated weights for policy 1, policy_version 494767 (0.0007) [2023-12-26 18:57:52,110][105692] Updated weights for policy 0, policy_version 494375 (0.0007) [2023-12-26 18:57:52,122][105620] Updated weights for policy 1, policy_version 494777 (0.0007) [2023-12-26 18:57:52,167][105692] Updated weights for policy 0, policy_version 494385 (0.0008) [2023-12-26 18:57:52,178][105620] Updated weights for policy 1, policy_version 494787 (0.0006) [2023-12-26 18:57:52,224][105692] Updated weights for policy 0, policy_version 494395 (0.0007) [2023-12-26 18:57:52,888][105692] Updated weights for policy 0, policy_version 494405 (0.0010) [2023-12-26 18:57:52,911][105620] Updated weights for policy 1, policy_version 494797 (0.0006) [2023-12-26 18:57:52,949][105692] Updated weights for policy 0, policy_version 494415 (0.0009) [2023-12-26 18:57:52,968][105620] Updated weights for policy 1, policy_version 494807 (0.0008) [2023-12-26 18:57:53,010][105692] Updated weights for policy 0, policy_version 494425 (0.0008) [2023-12-26 18:57:53,028][105620] Updated weights for policy 1, policy_version 494817 (0.0007) [2023-12-26 18:57:53,610][105620] Updated weights for policy 1, policy_version 494827 (0.0009) [2023-12-26 18:57:53,670][105620] Updated weights for policy 1, policy_version 494837 (0.0009) [2023-12-26 18:57:53,738][105620] Updated weights for policy 1, policy_version 494847 (0.0009) [2023-12-26 18:57:53,807][105692] Updated weights for policy 0, policy_version 494435 (0.0007) [2023-12-26 18:57:53,866][105692] Updated weights for policy 0, policy_version 494445 (0.0008) [2023-12-26 18:57:53,915][105692] Updated weights for policy 0, policy_version 494456 (0.0010) [2023-12-26 18:57:54,514][105692] Updated weights for policy 0, policy_version 494466 (0.0009) [2023-12-26 18:57:54,551][105620] Updated weights for policy 1, policy_version 494857 (0.0010) [2023-12-26 18:57:54,578][105692] Updated weights for policy 0, policy_version 494476 (0.0009) [2023-12-26 18:57:54,618][105620] Updated weights for policy 1, policy_version 494867 (0.0008) [2023-12-26 18:57:54,630][105692] Updated weights for policy 0, policy_version 494486 (0.0009) [2023-12-26 18:57:54,679][105692] Updated weights for policy 0, policy_version 494496 (0.0005) [2023-12-26 18:57:54,682][105620] Updated weights for policy 1, policy_version 494877 (0.0008) [2023-12-26 18:57:54,739][105620] Updated weights for policy 1, policy_version 494887 (0.0009) [2023-12-26 18:57:55,286][105692] Updated weights for policy 0, policy_version 494506 (0.0009) [2023-12-26 18:57:55,351][105692] Updated weights for policy 0, policy_version 494516 (0.0008) [2023-12-26 18:57:55,402][105692] Updated weights for policy 0, policy_version 494526 (0.0005) [2023-12-26 18:57:55,497][105620] Updated weights for policy 1, policy_version 494897 (0.0010) [2023-12-26 18:57:55,561][105620] Updated weights for policy 1, policy_version 494907 (0.0008) [2023-12-26 18:57:55,635][105620] Updated weights for policy 1, policy_version 494917 (0.0006) [2023-12-26 18:57:56,015][105692] Updated weights for policy 0, policy_version 494536 (0.0009) [2023-12-26 18:57:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 253329408. Throughput: 0: 9699.5, 1: 9679.6. Samples: 253340764. Policy #0 lag: (min: 31.0, avg: 31.6, max: 51.0) [2023-12-26 18:57:56,063][104569] Avg episode reward: [(0, '8822.477'), (1, '9263.967')] [2023-12-26 18:57:56,081][105692] Updated weights for policy 0, policy_version 494546 (0.0008) [2023-12-26 18:57:56,148][105692] Updated weights for policy 0, policy_version 494556 (0.0009) [2023-12-26 18:57:56,329][105620] Updated weights for policy 1, policy_version 494927 (0.0008) [2023-12-26 18:57:56,378][105620] Updated weights for policy 1, policy_version 494937 (0.0009) [2023-12-26 18:57:56,434][105620] Updated weights for policy 1, policy_version 494947 (0.0010) [2023-12-26 18:57:56,775][105692] Updated weights for policy 0, policy_version 494566 (0.0008) [2023-12-26 18:57:56,810][105585] KL-divergence is very high: 180.1756 [2023-12-26 18:57:56,824][105585] KL-divergence is very high: 108.9113 [2023-12-26 18:57:56,839][105692] Updated weights for policy 0, policy_version 494576 (0.0007) [2023-12-26 18:57:56,847][105585] KL-divergence is very high: 115.5278 [2023-12-26 18:57:56,858][105585] KL-divergence is very high: 237.8419 [2023-12-26 18:57:56,870][105585] KL-divergence is very high: 180.1725 [2023-12-26 18:57:56,892][105585] KL-divergence is very high: 123.8292 [2023-12-26 18:57:56,897][105692] Updated weights for policy 0, policy_version 494586 (0.0008) [2023-12-26 18:57:56,903][105585] KL-divergence is very high: 154.1624 [2023-12-26 18:57:56,915][105585] KL-divergence is very high: 179.1063 [2023-12-26 18:57:57,110][105620] Updated weights for policy 1, policy_version 494957 (0.0009) [2023-12-26 18:57:57,165][105620] Updated weights for policy 1, policy_version 494967 (0.0009) [2023-12-26 18:57:57,213][105620] Updated weights for policy 1, policy_version 494977 (0.0008) [2023-12-26 18:57:57,590][105692] Updated weights for policy 0, policy_version 494596 (0.0009) [2023-12-26 18:57:57,656][105692] Updated weights for policy 0, policy_version 494606 (0.0008) [2023-12-26 18:57:57,723][105692] Updated weights for policy 0, policy_version 494616 (0.0005) [2023-12-26 18:57:57,811][105620] Updated weights for policy 1, policy_version 494987 (0.0006) [2023-12-26 18:57:57,868][105620] Updated weights for policy 1, policy_version 494997 (0.0006) [2023-12-26 18:57:57,928][105620] Updated weights for policy 1, policy_version 495007 (0.0006) [2023-12-26 18:57:58,412][105692] Updated weights for policy 0, policy_version 494626 (0.0006) [2023-12-26 18:57:58,477][105692] Updated weights for policy 0, policy_version 494636 (0.0009) [2023-12-26 18:57:58,545][105692] Updated weights for policy 0, policy_version 494646 (0.0008) [2023-12-26 18:57:58,553][105585] KL-divergence is very high: 113.8965 [2023-12-26 18:57:58,608][105585] KL-divergence is very high: 104.7615 [2023-12-26 18:57:58,613][105692] Updated weights for policy 0, policy_version 494656 (0.0006) [2023-12-26 18:57:58,655][105620] Updated weights for policy 1, policy_version 495017 (0.0006) [2023-12-26 18:57:58,715][105620] Updated weights for policy 1, policy_version 495027 (0.0010) [2023-12-26 18:57:58,783][105620] Updated weights for policy 1, policy_version 495037 (0.0010) [2023-12-26 18:57:58,842][105620] Updated weights for policy 1, policy_version 495047 (0.0010) [2023-12-26 18:57:59,346][105692] Updated weights for policy 0, policy_version 494666 (0.0007) [2023-12-26 18:57:59,406][105692] Updated weights for policy 0, policy_version 494676 (0.0008) [2023-12-26 18:57:59,468][105692] Updated weights for policy 0, policy_version 494686 (0.0008) [2023-12-26 18:57:59,599][105620] Updated weights for policy 1, policy_version 495057 (0.0011) [2023-12-26 18:57:59,668][105620] Updated weights for policy 1, policy_version 495067 (0.0011) [2023-12-26 18:57:59,737][105620] Updated weights for policy 1, policy_version 495077 (0.0010) [2023-12-26 18:58:00,062][105692] Updated weights for policy 0, policy_version 494696 (0.0010) [2023-12-26 18:58:00,117][105692] Updated weights for policy 0, policy_version 494706 (0.0010) [2023-12-26 18:58:00,175][105692] Updated weights for policy 0, policy_version 494716 (0.0010) [2023-12-26 18:58:00,439][105620] Updated weights for policy 1, policy_version 495087 (0.0010) [2023-12-26 18:58:00,496][105620] Updated weights for policy 1, policy_version 495097 (0.0010) [2023-12-26 18:58:00,557][105620] Updated weights for policy 1, policy_version 495107 (0.0007) [2023-12-26 18:58:00,855][105692] Updated weights for policy 0, policy_version 494726 (0.0008) [2023-12-26 18:58:00,912][105692] Updated weights for policy 0, policy_version 494736 (0.0005) [2023-12-26 18:58:00,979][105692] Updated weights for policy 0, policy_version 494746 (0.0010) [2023-12-26 18:58:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 253435904. Throughput: 0: 9780.5, 1: 9706.7. Samples: 253401948. Policy #0 lag: (min: 31.0, avg: 31.6, max: 51.0) [2023-12-26 18:58:01,063][104569] Avg episode reward: [(0, '8821.202'), (1, '9355.003')] [2023-12-26 18:58:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000494752_126672896.pth... [2023-12-26 18:58:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000495112_126763008.pth... [2023-12-26 18:58:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000493992_126476288.pth [2023-12-26 18:58:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000493568_126369792.pth [2023-12-26 18:58:01,304][105620] Updated weights for policy 1, policy_version 495117 (0.0010) [2023-12-26 18:58:01,362][105620] Updated weights for policy 1, policy_version 495127 (0.0011) [2023-12-26 18:58:01,423][105620] Updated weights for policy 1, policy_version 495137 (0.0011) [2023-12-26 18:58:01,711][105692] Updated weights for policy 0, policy_version 494756 (0.0009) [2023-12-26 18:58:01,770][105692] Updated weights for policy 0, policy_version 494766 (0.0010) [2023-12-26 18:58:01,832][105692] Updated weights for policy 0, policy_version 494776 (0.0011) [2023-12-26 18:58:02,155][105620] Updated weights for policy 1, policy_version 495147 (0.0009) [2023-12-26 18:58:02,208][105620] Updated weights for policy 1, policy_version 495157 (0.0010) [2023-12-26 18:58:02,259][105620] Updated weights for policy 1, policy_version 495167 (0.0009) [2023-12-26 18:58:02,418][105692] Updated weights for policy 0, policy_version 494786 (0.0010) [2023-12-26 18:58:02,469][105692] Updated weights for policy 0, policy_version 494796 (0.0010) [2023-12-26 18:58:02,531][105692] Updated weights for policy 0, policy_version 494806 (0.0010) [2023-12-26 18:58:02,585][105692] Updated weights for policy 0, policy_version 494816 (0.0010) [2023-12-26 18:58:02,956][105620] Updated weights for policy 1, policy_version 495177 (0.0008) [2023-12-26 18:58:03,007][105620] Updated weights for policy 1, policy_version 495187 (0.0008) [2023-12-26 18:58:03,055][105620] Updated weights for policy 1, policy_version 495197 (0.0008) [2023-12-26 18:58:03,103][105620] Updated weights for policy 1, policy_version 495207 (0.0007) [2023-12-26 18:58:03,333][105692] Updated weights for policy 0, policy_version 494826 (0.0005) [2023-12-26 18:58:03,378][105692] Updated weights for policy 0, policy_version 494836 (0.0005) [2023-12-26 18:58:03,424][105692] Updated weights for policy 0, policy_version 494846 (0.0005) [2023-12-26 18:58:03,929][105620] Updated weights for policy 1, policy_version 495217 (0.0006) [2023-12-26 18:58:03,982][105620] Updated weights for policy 1, policy_version 495227 (0.0007) [2023-12-26 18:58:04,038][105692] Updated weights for policy 0, policy_version 494856 (0.0007) [2023-12-26 18:58:04,044][105620] Updated weights for policy 1, policy_version 495237 (0.0008) [2023-12-26 18:58:04,101][105692] Updated weights for policy 0, policy_version 494866 (0.0011) [2023-12-26 18:58:04,166][105692] Updated weights for policy 0, policy_version 494876 (0.0011) [2023-12-26 18:58:04,780][105620] Updated weights for policy 1, policy_version 495247 (0.0008) [2023-12-26 18:58:04,828][105620] Updated weights for policy 1, policy_version 495257 (0.0008) [2023-12-26 18:58:04,873][105620] Updated weights for policy 1, policy_version 495267 (0.0008) [2023-12-26 18:58:04,946][105692] Updated weights for policy 0, policy_version 494886 (0.0007) [2023-12-26 18:58:05,006][105692] Updated weights for policy 0, policy_version 494896 (0.0010) [2023-12-26 18:58:05,062][105692] Updated weights for policy 0, policy_version 494906 (0.0007) [2023-12-26 18:58:05,689][105620] Updated weights for policy 1, policy_version 495278 (0.0009) [2023-12-26 18:58:05,718][105692] Updated weights for policy 0, policy_version 494916 (0.0007) [2023-12-26 18:58:05,740][105620] Updated weights for policy 1, policy_version 495288 (0.0007) [2023-12-26 18:58:05,783][105692] Updated weights for policy 0, policy_version 494926 (0.0010) [2023-12-26 18:58:05,795][105620] Updated weights for policy 1, policy_version 495298 (0.0007) [2023-12-26 18:58:05,842][105692] Updated weights for policy 0, policy_version 494936 (0.0010) [2023-12-26 18:58:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 253534208. Throughput: 0: 9773.9, 1: 9647.8. Samples: 253519420. Policy #0 lag: (min: 31.0, avg: 31.6, max: 51.0) [2023-12-26 18:58:06,063][104569] Avg episode reward: [(0, '8907.740'), (1, '9351.850')] [2023-12-26 18:58:06,458][105620] Updated weights for policy 1, policy_version 495308 (0.0007) [2023-12-26 18:58:06,506][105620] Updated weights for policy 1, policy_version 495318 (0.0008) [2023-12-26 18:58:06,568][105620] Updated weights for policy 1, policy_version 495328 (0.0005) [2023-12-26 18:58:06,595][105692] Updated weights for policy 0, policy_version 494946 (0.0010) [2023-12-26 18:58:06,651][105692] Updated weights for policy 0, policy_version 494956 (0.0011) [2023-12-26 18:58:06,721][105692] Updated weights for policy 0, policy_version 494966 (0.0011) [2023-12-26 18:58:06,784][105692] Updated weights for policy 0, policy_version 494976 (0.0010) [2023-12-26 18:58:07,110][105620] Updated weights for policy 1, policy_version 495338 (0.0006) [2023-12-26 18:58:07,168][105620] Updated weights for policy 1, policy_version 495348 (0.0006) [2023-12-26 18:58:07,215][105620] Updated weights for policy 1, policy_version 495358 (0.0010) [2023-12-26 18:58:07,267][105620] Updated weights for policy 1, policy_version 495368 (0.0010) [2023-12-26 18:58:07,526][105692] Updated weights for policy 0, policy_version 494986 (0.0011) [2023-12-26 18:58:07,582][105692] Updated weights for policy 0, policy_version 494996 (0.0010) [2023-12-26 18:58:07,635][105692] Updated weights for policy 0, policy_version 495006 (0.0009) [2023-12-26 18:58:07,885][105620] Updated weights for policy 1, policy_version 495378 (0.0005) [2023-12-26 18:58:07,938][105620] Updated weights for policy 1, policy_version 495388 (0.0008) [2023-12-26 18:58:07,996][105620] Updated weights for policy 1, policy_version 495398 (0.0010) [2023-12-26 18:58:08,358][105692] Updated weights for policy 0, policy_version 495016 (0.0010) [2023-12-26 18:58:08,413][105692] Updated weights for policy 0, policy_version 495026 (0.0011) [2023-12-26 18:58:08,472][105692] Updated weights for policy 0, policy_version 495036 (0.0010) [2023-12-26 18:58:08,656][105620] Updated weights for policy 1, policy_version 495408 (0.0010) [2023-12-26 18:58:08,711][105620] Updated weights for policy 1, policy_version 495418 (0.0010) [2023-12-26 18:58:08,762][105620] Updated weights for policy 1, policy_version 495428 (0.0010) [2023-12-26 18:58:09,172][105692] Updated weights for policy 0, policy_version 495046 (0.0007) [2023-12-26 18:58:09,238][105692] Updated weights for policy 0, policy_version 495056 (0.0007) [2023-12-26 18:58:09,300][105692] Updated weights for policy 0, policy_version 495066 (0.0011) [2023-12-26 18:58:09,521][105620] Updated weights for policy 1, policy_version 495438 (0.0010) [2023-12-26 18:58:09,587][105620] Updated weights for policy 1, policy_version 495448 (0.0010) [2023-12-26 18:58:09,653][105620] Updated weights for policy 1, policy_version 495458 (0.0006) [2023-12-26 18:58:10,072][105692] Updated weights for policy 0, policy_version 495076 (0.0009) [2023-12-26 18:58:10,141][105692] Updated weights for policy 0, policy_version 495086 (0.0006) [2023-12-26 18:58:10,203][105692] Updated weights for policy 0, policy_version 495096 (0.0007) [2023-12-26 18:58:10,361][105620] Updated weights for policy 1, policy_version 495468 (0.0007) [2023-12-26 18:58:10,420][105620] Updated weights for policy 1, policy_version 495478 (0.0005) [2023-12-26 18:58:10,481][105620] Updated weights for policy 1, policy_version 495488 (0.0008) [2023-12-26 18:58:10,801][105692] Updated weights for policy 0, policy_version 495106 (0.0007) [2023-12-26 18:58:10,871][105692] Updated weights for policy 0, policy_version 495116 (0.0007) [2023-12-26 18:58:10,927][105692] Updated weights for policy 0, policy_version 495126 (0.0005) [2023-12-26 18:58:10,982][105692] Updated weights for policy 0, policy_version 495136 (0.0005) [2023-12-26 18:58:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 253632512. Throughput: 0: 9880.5, 1: 9699.8. Samples: 253639568. Policy #0 lag: (min: 31.0, avg: 31.6, max: 51.0) [2023-12-26 18:58:11,062][104569] Avg episode reward: [(0, '9087.608'), (1, '9348.348')] [2023-12-26 18:58:11,167][105620] Updated weights for policy 1, policy_version 495498 (0.0010) [2023-12-26 18:58:11,222][105620] Updated weights for policy 1, policy_version 495508 (0.0010) [2023-12-26 18:58:11,287][105620] Updated weights for policy 1, policy_version 495518 (0.0008) [2023-12-26 18:58:11,358][105620] Updated weights for policy 1, policy_version 495528 (0.0010) [2023-12-26 18:58:11,715][105692] Updated weights for policy 0, policy_version 495146 (0.0007) [2023-12-26 18:58:11,780][105692] Updated weights for policy 0, policy_version 495156 (0.0007) [2023-12-26 18:58:11,832][105692] Updated weights for policy 0, policy_version 495166 (0.0009) [2023-12-26 18:58:12,023][105620] Updated weights for policy 1, policy_version 495538 (0.0009) [2023-12-26 18:58:12,088][105620] Updated weights for policy 1, policy_version 495548 (0.0008) [2023-12-26 18:58:12,147][105620] Updated weights for policy 1, policy_version 495558 (0.0009) [2023-12-26 18:58:12,671][105692] Updated weights for policy 0, policy_version 495176 (0.0008) [2023-12-26 18:58:12,733][105692] Updated weights for policy 0, policy_version 495186 (0.0009) [2023-12-26 18:58:12,786][105692] Updated weights for policy 0, policy_version 495196 (0.0008) [2023-12-26 18:58:12,800][105620] Updated weights for policy 1, policy_version 495568 (0.0008) [2023-12-26 18:58:12,861][105620] Updated weights for policy 1, policy_version 495578 (0.0009) [2023-12-26 18:58:12,915][105620] Updated weights for policy 1, policy_version 495588 (0.0009) [2023-12-26 18:58:13,562][105620] Updated weights for policy 1, policy_version 495598 (0.0008) [2023-12-26 18:58:13,582][105692] Updated weights for policy 0, policy_version 495206 (0.0007) [2023-12-26 18:58:13,629][105692] Updated weights for policy 0, policy_version 495216 (0.0006) [2023-12-26 18:58:13,631][105620] Updated weights for policy 1, policy_version 495608 (0.0006) [2023-12-26 18:58:13,677][105692] Updated weights for policy 0, policy_version 495226 (0.0007) [2023-12-26 18:58:13,701][105620] Updated weights for policy 1, policy_version 495618 (0.0008) [2023-12-26 18:58:14,362][105692] Updated weights for policy 0, policy_version 495236 (0.0007) [2023-12-26 18:58:14,388][105620] Updated weights for policy 1, policy_version 495628 (0.0006) [2023-12-26 18:58:14,414][105692] Updated weights for policy 0, policy_version 495246 (0.0009) [2023-12-26 18:58:14,444][105620] Updated weights for policy 1, policy_version 495638 (0.0005) [2023-12-26 18:58:14,464][105692] Updated weights for policy 0, policy_version 495256 (0.0008) [2023-12-26 18:58:14,492][105620] Updated weights for policy 1, policy_version 495648 (0.0006) [2023-12-26 18:58:15,130][105620] Updated weights for policy 1, policy_version 495658 (0.0006) [2023-12-26 18:58:15,187][105620] Updated weights for policy 1, policy_version 495668 (0.0008) [2023-12-26 18:58:15,216][105692] Updated weights for policy 0, policy_version 495266 (0.0007) [2023-12-26 18:58:15,250][105620] Updated weights for policy 1, policy_version 495678 (0.0008) [2023-12-26 18:58:15,282][105692] Updated weights for policy 0, policy_version 495276 (0.0006) [2023-12-26 18:58:15,307][105620] Updated weights for policy 1, policy_version 495688 (0.0007) [2023-12-26 18:58:15,339][105692] Updated weights for policy 0, policy_version 495286 (0.0008) [2023-12-26 18:58:15,396][105692] Updated weights for policy 0, policy_version 495296 (0.0008) [2023-12-26 18:58:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 253722624. Throughput: 0: 9805.0, 1: 9728.7. Samples: 253697624. Policy #0 lag: (min: 31.0, avg: 31.6, max: 51.0) [2023-12-26 18:58:16,063][104569] Avg episode reward: [(0, '9177.682'), (1, '9352.901')] [2023-12-26 18:58:16,084][105620] Updated weights for policy 1, policy_version 495698 (0.0008) [2023-12-26 18:58:16,106][105692] Updated weights for policy 0, policy_version 495306 (0.0010) [2023-12-26 18:58:16,136][105620] Updated weights for policy 1, policy_version 495708 (0.0006) [2023-12-26 18:58:16,161][105692] Updated weights for policy 0, policy_version 495316 (0.0007) [2023-12-26 18:58:16,199][105620] Updated weights for policy 1, policy_version 495718 (0.0007) [2023-12-26 18:58:16,208][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000495720_126918656.pth... [2023-12-26 18:58:16,213][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000494568_126623744.pth [2023-12-26 18:58:16,214][105692] Updated weights for policy 0, policy_version 495326 (0.0007) [2023-12-26 18:58:16,227][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000495328_126820352.pth... [2023-12-26 18:58:16,230][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000494176_126525440.pth [2023-12-26 18:58:16,882][105692] Updated weights for policy 0, policy_version 495336 (0.0009) [2023-12-26 18:58:16,945][105692] Updated weights for policy 0, policy_version 495346 (0.0008) [2023-12-26 18:58:16,969][105620] Updated weights for policy 1, policy_version 495728 (0.0008) [2023-12-26 18:58:17,007][105692] Updated weights for policy 0, policy_version 495356 (0.0009) [2023-12-26 18:58:17,025][105620] Updated weights for policy 1, policy_version 495738 (0.0008) [2023-12-26 18:58:17,085][105620] Updated weights for policy 1, policy_version 495748 (0.0008) [2023-12-26 18:58:17,746][105692] Updated weights for policy 0, policy_version 495366 (0.0009) [2023-12-26 18:58:17,801][105692] Updated weights for policy 0, policy_version 495376 (0.0010) [2023-12-26 18:58:17,839][105620] Updated weights for policy 1, policy_version 495758 (0.0009) [2023-12-26 18:58:17,863][105692] Updated weights for policy 0, policy_version 495386 (0.0010) [2023-12-26 18:58:17,897][105620] Updated weights for policy 1, policy_version 495768 (0.0010) [2023-12-26 18:58:17,953][105620] Updated weights for policy 1, policy_version 495778 (0.0008) [2023-12-26 18:58:18,581][105692] Updated weights for policy 0, policy_version 495396 (0.0010) [2023-12-26 18:58:18,647][105692] Updated weights for policy 0, policy_version 495406 (0.0010) [2023-12-26 18:58:18,683][105620] Updated weights for policy 1, policy_version 495788 (0.0007) [2023-12-26 18:58:18,715][105692] Updated weights for policy 0, policy_version 495416 (0.0007) [2023-12-26 18:58:18,752][105620] Updated weights for policy 1, policy_version 495798 (0.0006) [2023-12-26 18:58:18,820][105620] Updated weights for policy 1, policy_version 495808 (0.0007) [2023-12-26 18:58:19,399][105692] Updated weights for policy 0, policy_version 495426 (0.0008) [2023-12-26 18:58:19,469][105692] Updated weights for policy 0, policy_version 495436 (0.0009) [2023-12-26 18:58:19,538][105692] Updated weights for policy 0, policy_version 495446 (0.0009) [2023-12-26 18:58:19,543][105620] Updated weights for policy 1, policy_version 495818 (0.0008) [2023-12-26 18:58:19,596][105620] Updated weights for policy 1, policy_version 495828 (0.0006) [2023-12-26 18:58:19,597][105692] Updated weights for policy 0, policy_version 495456 (0.0009) [2023-12-26 18:58:19,657][105620] Updated weights for policy 1, policy_version 495838 (0.0008) [2023-12-26 18:58:19,719][105620] Updated weights for policy 1, policy_version 495848 (0.0009) [2023-12-26 18:58:20,350][105692] Updated weights for policy 0, policy_version 495466 (0.0009) [2023-12-26 18:58:20,408][105692] Updated weights for policy 0, policy_version 495476 (0.0008) [2023-12-26 18:58:20,469][105692] Updated weights for policy 0, policy_version 495486 (0.0006) [2023-12-26 18:58:20,501][105620] Updated weights for policy 1, policy_version 495858 (0.0009) [2023-12-26 18:58:20,568][105620] Updated weights for policy 1, policy_version 495868 (0.0008) [2023-12-26 18:58:20,632][105620] Updated weights for policy 1, policy_version 495878 (0.0008) [2023-12-26 18:58:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 253820928. Throughput: 0: 9780.8, 1: 9707.2. Samples: 253813096. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 18:58:21,062][104569] Avg episode reward: [(0, '9268.449'), (1, '9354.246')] [2023-12-26 18:58:21,153][105692] Updated weights for policy 0, policy_version 495496 (0.0007) [2023-12-26 18:58:21,213][105692] Updated weights for policy 0, policy_version 495506 (0.0008) [2023-12-26 18:58:21,283][105692] Updated weights for policy 0, policy_version 495516 (0.0008) [2023-12-26 18:58:21,465][105620] Updated weights for policy 1, policy_version 495888 (0.0009) [2023-12-26 18:58:21,519][105620] Updated weights for policy 1, policy_version 495898 (0.0009) [2023-12-26 18:58:21,578][105620] Updated weights for policy 1, policy_version 495908 (0.0010) [2023-12-26 18:58:22,071][105692] Updated weights for policy 0, policy_version 495526 (0.0009) [2023-12-26 18:58:22,135][105692] Updated weights for policy 0, policy_version 495536 (0.0007) [2023-12-26 18:58:22,195][105692] Updated weights for policy 0, policy_version 495546 (0.0007) [2023-12-26 18:58:22,356][105620] Updated weights for policy 1, policy_version 495918 (0.0009) [2023-12-26 18:58:22,410][105620] Updated weights for policy 1, policy_version 495928 (0.0008) [2023-12-26 18:58:22,472][105620] Updated weights for policy 1, policy_version 495938 (0.0009) [2023-12-26 18:58:22,951][105692] Updated weights for policy 0, policy_version 495556 (0.0009) [2023-12-26 18:58:23,022][105692] Updated weights for policy 0, policy_version 495566 (0.0010) [2023-12-26 18:58:23,047][105585] KL-divergence is very high: 133.9766 [2023-12-26 18:58:23,078][105692] Updated weights for policy 0, policy_version 495576 (0.0006) [2023-12-26 18:58:23,094][105585] KL-divergence is very high: 144.9778 [2023-12-26 18:58:23,271][105620] Updated weights for policy 1, policy_version 495948 (0.0009) [2023-12-26 18:58:23,332][105620] Updated weights for policy 1, policy_version 495958 (0.0009) [2023-12-26 18:58:23,386][105620] Updated weights for policy 1, policy_version 495968 (0.0009) [2023-12-26 18:58:23,770][105692] Updated weights for policy 0, policy_version 495586 (0.0007) [2023-12-26 18:58:23,825][105692] Updated weights for policy 0, policy_version 495596 (0.0009) [2023-12-26 18:58:23,844][105585] KL-divergence is very high: 110.5703 [2023-12-26 18:58:23,880][105692] Updated weights for policy 0, policy_version 495606 (0.0006) [2023-12-26 18:58:23,886][105585] KL-divergence is very high: 174.1924 [2023-12-26 18:58:23,928][105585] KL-divergence is very high: 146.6615 [2023-12-26 18:58:23,933][105692] Updated weights for policy 0, policy_version 495616 (0.0006) [2023-12-26 18:58:24,154][105620] Updated weights for policy 1, policy_version 495978 (0.0009) [2023-12-26 18:58:24,214][105620] Updated weights for policy 1, policy_version 495988 (0.0010) [2023-12-26 18:58:24,276][105620] Updated weights for policy 1, policy_version 495998 (0.0010) [2023-12-26 18:58:24,333][105620] Updated weights for policy 1, policy_version 496008 (0.0010) [2023-12-26 18:58:24,514][105585] KL-divergence is very high: 130.3748 [2023-12-26 18:58:24,559][105692] Updated weights for policy 0, policy_version 495626 (0.0009) [2023-12-26 18:58:24,560][105585] KL-divergence is very high: 104.4684 [2023-12-26 18:58:24,613][105692] Updated weights for policy 0, policy_version 495636 (0.0009) [2023-12-26 18:58:24,668][105692] Updated weights for policy 0, policy_version 495646 (0.0007) [2023-12-26 18:58:25,169][105620] Updated weights for policy 1, policy_version 496018 (0.0009) [2023-12-26 18:58:25,228][105620] Updated weights for policy 1, policy_version 496028 (0.0008) [2023-12-26 18:58:25,290][105620] Updated weights for policy 1, policy_version 496038 (0.0009) [2023-12-26 18:58:25,337][105692] Updated weights for policy 0, policy_version 495656 (0.0008) [2023-12-26 18:58:25,399][105692] Updated weights for policy 0, policy_version 495666 (0.0009) [2023-12-26 18:58:25,457][105692] Updated weights for policy 0, policy_version 495676 (0.0009) [2023-12-26 18:58:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 253911040. Throughput: 0: 9878.2, 1: 9526.0. Samples: 253924272. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 18:58:26,063][104569] Avg episode reward: [(0, '8900.577'), (1, '9352.986')] [2023-12-26 18:58:26,090][105692] Updated weights for policy 0, policy_version 495686 (0.0008) [2023-12-26 18:58:26,091][105620] Updated weights for policy 1, policy_version 496048 (0.0008) [2023-12-26 18:58:26,147][105692] Updated weights for policy 0, policy_version 495696 (0.0007) [2023-12-26 18:58:26,147][105620] Updated weights for policy 1, policy_version 496058 (0.0009) [2023-12-26 18:58:26,201][105620] Updated weights for policy 1, policy_version 496068 (0.0010) [2023-12-26 18:58:26,208][105692] Updated weights for policy 0, policy_version 495706 (0.0006) [2023-12-26 18:58:26,879][105692] Updated weights for policy 0, policy_version 495716 (0.0007) [2023-12-26 18:58:26,924][105620] Updated weights for policy 1, policy_version 496078 (0.0007) [2023-12-26 18:58:26,930][105692] Updated weights for policy 0, policy_version 495726 (0.0008) [2023-12-26 18:58:26,980][105692] Updated weights for policy 0, policy_version 495736 (0.0009) [2023-12-26 18:58:26,986][105620] Updated weights for policy 1, policy_version 496088 (0.0008) [2023-12-26 18:58:27,044][105620] Updated weights for policy 1, policy_version 496098 (0.0008) [2023-12-26 18:58:27,692][105692] Updated weights for policy 0, policy_version 495746 (0.0006) [2023-12-26 18:58:27,738][105692] Updated weights for policy 0, policy_version 495756 (0.0009) [2023-12-26 18:58:27,745][105620] Updated weights for policy 1, policy_version 496108 (0.0008) [2023-12-26 18:58:27,791][105692] Updated weights for policy 0, policy_version 495766 (0.0007) [2023-12-26 18:58:27,795][105620] Updated weights for policy 1, policy_version 496118 (0.0006) [2023-12-26 18:58:27,837][105692] Updated weights for policy 0, policy_version 495776 (0.0008) [2023-12-26 18:58:27,846][105620] Updated weights for policy 1, policy_version 496128 (0.0009) [2023-12-26 18:58:28,440][105620] Updated weights for policy 1, policy_version 496138 (0.0008) [2023-12-26 18:58:28,497][105620] Updated weights for policy 1, policy_version 496148 (0.0005) [2023-12-26 18:58:28,561][105620] Updated weights for policy 1, policy_version 496158 (0.0005) [2023-12-26 18:58:28,562][105692] Updated weights for policy 0, policy_version 495786 (0.0009) [2023-12-26 18:58:28,627][105620] Updated weights for policy 1, policy_version 496168 (0.0005) [2023-12-26 18:58:28,627][105692] Updated weights for policy 0, policy_version 495796 (0.0007) [2023-12-26 18:58:28,695][105692] Updated weights for policy 0, policy_version 495806 (0.0005) [2023-12-26 18:58:29,304][105620] Updated weights for policy 1, policy_version 496178 (0.0010) [2023-12-26 18:58:29,331][105692] Updated weights for policy 0, policy_version 495816 (0.0006) [2023-12-26 18:58:29,368][105620] Updated weights for policy 1, policy_version 496188 (0.0012) [2023-12-26 18:58:29,397][105692] Updated weights for policy 0, policy_version 495826 (0.0008) [2023-12-26 18:58:29,432][105620] Updated weights for policy 1, policy_version 496198 (0.0009) [2023-12-26 18:58:29,456][105692] Updated weights for policy 0, policy_version 495836 (0.0007) [2023-12-26 18:58:30,122][105620] Updated weights for policy 1, policy_version 496208 (0.0010) [2023-12-26 18:58:30,180][105620] Updated weights for policy 1, policy_version 496218 (0.0010) [2023-12-26 18:58:30,183][105692] Updated weights for policy 0, policy_version 495846 (0.0006) [2023-12-26 18:58:30,230][105692] Updated weights for policy 0, policy_version 495856 (0.0005) [2023-12-26 18:58:30,242][105620] Updated weights for policy 1, policy_version 496228 (0.0011) [2023-12-26 18:58:30,291][105692] Updated weights for policy 0, policy_version 495866 (0.0005) [2023-12-26 18:58:30,976][105620] Updated weights for policy 1, policy_version 496238 (0.0010) [2023-12-26 18:58:31,012][105692] Updated weights for policy 0, policy_version 495876 (0.0007) [2023-12-26 18:58:31,044][105620] Updated weights for policy 1, policy_version 496248 (0.0009) [2023-12-26 18:58:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 254009344. Throughput: 0: 9882.8, 1: 9583.3. Samples: 253986012. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 18:58:31,062][104569] Avg episode reward: [(0, '8809.789'), (1, '9351.131')] [2023-12-26 18:58:31,076][105692] Updated weights for policy 0, policy_version 495886 (0.0007) [2023-12-26 18:58:31,100][105620] Updated weights for policy 1, policy_version 496258 (0.0009) [2023-12-26 18:58:31,135][105692] Updated weights for policy 0, policy_version 495896 (0.0009) [2023-12-26 18:58:31,138][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000496264_127057920.pth... [2023-12-26 18:58:31,143][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000495112_126763008.pth [2023-12-26 18:58:31,175][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000495904_126967808.pth... [2023-12-26 18:58:31,178][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000494752_126672896.pth [2023-12-26 18:58:31,832][105620] Updated weights for policy 1, policy_version 496268 (0.0010) [2023-12-26 18:58:31,852][105692] Updated weights for policy 0, policy_version 495906 (0.0007) [2023-12-26 18:58:31,876][105620] Updated weights for policy 1, policy_version 496278 (0.0006) [2023-12-26 18:58:31,914][105692] Updated weights for policy 0, policy_version 495916 (0.0007) [2023-12-26 18:58:31,937][105620] Updated weights for policy 1, policy_version 496288 (0.0007) [2023-12-26 18:58:31,971][105692] Updated weights for policy 0, policy_version 495926 (0.0009) [2023-12-26 18:58:32,035][105692] Updated weights for policy 0, policy_version 495936 (0.0006) [2023-12-26 18:58:32,661][105620] Updated weights for policy 1, policy_version 496298 (0.0007) [2023-12-26 18:58:32,712][105620] Updated weights for policy 1, policy_version 496308 (0.0008) [2023-12-26 18:58:32,730][105692] Updated weights for policy 0, policy_version 495946 (0.0006) [2023-12-26 18:58:32,771][105620] Updated weights for policy 1, policy_version 496318 (0.0009) [2023-12-26 18:58:32,782][105692] Updated weights for policy 0, policy_version 495956 (0.0005) [2023-12-26 18:58:32,829][105692] Updated weights for policy 0, policy_version 495966 (0.0005) [2023-12-26 18:58:32,830][105620] Updated weights for policy 1, policy_version 496328 (0.0008) [2023-12-26 18:58:33,427][105692] Updated weights for policy 0, policy_version 495976 (0.0005) [2023-12-26 18:58:33,475][105692] Updated weights for policy 0, policy_version 495986 (0.0005) [2023-12-26 18:58:33,532][105692] Updated weights for policy 0, policy_version 495996 (0.0005) [2023-12-26 18:58:33,653][105620] Updated weights for policy 1, policy_version 496339 (0.0010) [2023-12-26 18:58:33,705][105620] Updated weights for policy 1, policy_version 496350 (0.0010) [2023-12-26 18:58:33,757][105620] Updated weights for policy 1, policy_version 496360 (0.0009) [2023-12-26 18:58:34,064][105692] Updated weights for policy 0, policy_version 496006 (0.0007) [2023-12-26 18:58:34,118][105692] Updated weights for policy 0, policy_version 496016 (0.0006) [2023-12-26 18:58:34,173][105692] Updated weights for policy 0, policy_version 496026 (0.0008) [2023-12-26 18:58:34,665][105620] Updated weights for policy 1, policy_version 496370 (0.0009) [2023-12-26 18:58:34,716][105620] Updated weights for policy 1, policy_version 496380 (0.0008) [2023-12-26 18:58:34,771][105620] Updated weights for policy 1, policy_version 496390 (0.0009) [2023-12-26 18:58:34,832][105692] Updated weights for policy 0, policy_version 496036 (0.0007) [2023-12-26 18:58:34,889][105692] Updated weights for policy 0, policy_version 496046 (0.0009) [2023-12-26 18:58:34,943][105692] Updated weights for policy 0, policy_version 496057 (0.0007) [2023-12-26 18:58:35,561][105692] Updated weights for policy 0, policy_version 496067 (0.0006) [2023-12-26 18:58:35,611][105692] Updated weights for policy 0, policy_version 496077 (0.0007) [2023-12-26 18:58:35,613][105620] Updated weights for policy 1, policy_version 496400 (0.0009) [2023-12-26 18:58:35,663][105692] Updated weights for policy 0, policy_version 496087 (0.0005) [2023-12-26 18:58:35,669][105620] Updated weights for policy 1, policy_version 496410 (0.0008) [2023-12-26 18:58:35,717][105620] Updated weights for policy 1, policy_version 496420 (0.0006) [2023-12-26 18:58:36,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 254115840. Throughput: 0: 9925.1, 1: 9607.2. Samples: 254103292. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 18:58:36,062][104569] Avg episode reward: [(0, '8993.625'), (1, '9349.537')] [2023-12-26 18:58:36,422][105692] Updated weights for policy 0, policy_version 496097 (0.0009) [2023-12-26 18:58:36,461][105620] Updated weights for policy 1, policy_version 496430 (0.0009) [2023-12-26 18:58:36,487][105692] Updated weights for policy 0, policy_version 496107 (0.0010) [2023-12-26 18:58:36,526][105620] Updated weights for policy 1, policy_version 496440 (0.0007) [2023-12-26 18:58:36,550][105692] Updated weights for policy 0, policy_version 496117 (0.0008) [2023-12-26 18:58:36,593][105620] Updated weights for policy 1, policy_version 496450 (0.0007) [2023-12-26 18:58:36,609][105692] Updated weights for policy 0, policy_version 496127 (0.0007) [2023-12-26 18:58:37,252][105692] Updated weights for policy 0, policy_version 496137 (0.0010) [2023-12-26 18:58:37,307][105692] Updated weights for policy 0, policy_version 496147 (0.0009) [2023-12-26 18:58:37,311][105620] Updated weights for policy 1, policy_version 496460 (0.0008) [2023-12-26 18:58:37,361][105692] Updated weights for policy 0, policy_version 496157 (0.0010) [2023-12-26 18:58:37,371][105620] Updated weights for policy 1, policy_version 496470 (0.0008) [2023-12-26 18:58:37,435][105620] Updated weights for policy 1, policy_version 496480 (0.0005) [2023-12-26 18:58:37,973][105620] Updated weights for policy 1, policy_version 496490 (0.0006) [2023-12-26 18:58:38,026][105620] Updated weights for policy 1, policy_version 496500 (0.0006) [2023-12-26 18:58:38,064][105692] Updated weights for policy 0, policy_version 496167 (0.0007) [2023-12-26 18:58:38,087][105620] Updated weights for policy 1, policy_version 496510 (0.0007) [2023-12-26 18:58:38,136][105692] Updated weights for policy 0, policy_version 496177 (0.0006) [2023-12-26 18:58:38,151][105620] Updated weights for policy 1, policy_version 496520 (0.0006) [2023-12-26 18:58:38,197][105692] Updated weights for policy 0, policy_version 496187 (0.0006) [2023-12-26 18:58:38,218][105585] KL-divergence is very high: 150.7061 [2023-12-26 18:58:38,838][105692] Updated weights for policy 0, policy_version 496197 (0.0008) [2023-12-26 18:58:38,849][105620] Updated weights for policy 1, policy_version 496530 (0.0010) [2023-12-26 18:58:38,890][105692] Updated weights for policy 0, policy_version 496207 (0.0011) [2023-12-26 18:58:38,901][105620] Updated weights for policy 1, policy_version 496540 (0.0010) [2023-12-26 18:58:38,953][105692] Updated weights for policy 0, policy_version 496217 (0.0011) [2023-12-26 18:58:38,960][105620] Updated weights for policy 1, policy_version 496550 (0.0007) [2023-12-26 18:58:39,671][105620] Updated weights for policy 1, policy_version 496560 (0.0007) [2023-12-26 18:58:39,718][105692] Updated weights for policy 0, policy_version 496227 (0.0010) [2023-12-26 18:58:39,738][105620] Updated weights for policy 1, policy_version 496570 (0.0009) [2023-12-26 18:58:39,781][105692] Updated weights for policy 0, policy_version 496237 (0.0011) [2023-12-26 18:58:39,801][105620] Updated weights for policy 1, policy_version 496580 (0.0011) [2023-12-26 18:58:39,843][105692] Updated weights for policy 0, policy_version 496247 (0.0011) [2023-12-26 18:58:40,501][105692] Updated weights for policy 0, policy_version 496257 (0.0010) [2023-12-26 18:58:40,523][105620] Updated weights for policy 1, policy_version 496590 (0.0011) [2023-12-26 18:58:40,562][105692] Updated weights for policy 0, policy_version 496267 (0.0006) [2023-12-26 18:58:40,580][105620] Updated weights for policy 1, policy_version 496600 (0.0011) [2023-12-26 18:58:40,614][105692] Updated weights for policy 0, policy_version 496277 (0.0006) [2023-12-26 18:58:40,628][105620] Updated weights for policy 1, policy_version 496610 (0.0010) [2023-12-26 18:58:40,660][105692] Updated weights for policy 0, policy_version 496287 (0.0005) [2023-12-26 18:58:41,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 254214144. Throughput: 0: 9941.1, 1: 9645.2. Samples: 254222144. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 18:58:41,063][104569] Avg episode reward: [(0, '8992.430'), (1, '9347.522')] [2023-12-26 18:58:41,391][105620] Updated weights for policy 1, policy_version 496620 (0.0010) [2023-12-26 18:58:41,394][105692] Updated weights for policy 0, policy_version 496297 (0.0007) [2023-12-26 18:58:41,439][105620] Updated weights for policy 1, policy_version 496630 (0.0006) [2023-12-26 18:58:41,451][105692] Updated weights for policy 0, policy_version 496307 (0.0009) [2023-12-26 18:58:41,500][105620] Updated weights for policy 1, policy_version 496640 (0.0006) [2023-12-26 18:58:41,510][105692] Updated weights for policy 0, policy_version 496317 (0.0009) [2023-12-26 18:58:41,525][105585] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000001 [2023-12-26 18:58:42,131][105620] Updated weights for policy 1, policy_version 496650 (0.0007) [2023-12-26 18:58:42,190][105620] Updated weights for policy 1, policy_version 496660 (0.0005) [2023-12-26 18:58:42,253][105620] Updated weights for policy 1, policy_version 496670 (0.0007) [2023-12-26 18:58:42,322][105620] Updated weights for policy 1, policy_version 496680 (0.0007) [2023-12-26 18:58:42,342][105692] Updated weights for policy 0, policy_version 496327 (0.0008) [2023-12-26 18:58:42,403][105692] Updated weights for policy 0, policy_version 496337 (0.0008) [2023-12-26 18:58:42,454][105692] Updated weights for policy 0, policy_version 496347 (0.0008) [2023-12-26 18:58:43,000][105620] Updated weights for policy 1, policy_version 496690 (0.0009) [2023-12-26 18:58:43,051][105620] Updated weights for policy 1, policy_version 496700 (0.0009) [2023-12-26 18:58:43,112][105620] Updated weights for policy 1, policy_version 496710 (0.0009) [2023-12-26 18:58:43,208][105692] Updated weights for policy 0, policy_version 496357 (0.0008) [2023-12-26 18:58:43,256][105692] Updated weights for policy 0, policy_version 496367 (0.0009) [2023-12-26 18:58:43,303][105692] Updated weights for policy 0, policy_version 496377 (0.0009) [2023-12-26 18:58:43,864][105620] Updated weights for policy 1, policy_version 496720 (0.0010) [2023-12-26 18:58:43,934][105620] Updated weights for policy 1, policy_version 496730 (0.0010) [2023-12-26 18:58:43,997][105620] Updated weights for policy 1, policy_version 496740 (0.0009) [2023-12-26 18:58:44,015][105692] Updated weights for policy 0, policy_version 496387 (0.0008) [2023-12-26 18:58:44,063][105692] Updated weights for policy 0, policy_version 496397 (0.0008) [2023-12-26 18:58:44,110][105692] Updated weights for policy 0, policy_version 496407 (0.0008) [2023-12-26 18:58:44,782][105620] Updated weights for policy 1, policy_version 496750 (0.0008) [2023-12-26 18:58:44,819][105692] Updated weights for policy 0, policy_version 496417 (0.0006) [2023-12-26 18:58:44,841][105620] Updated weights for policy 1, policy_version 496760 (0.0009) [2023-12-26 18:58:44,873][105692] Updated weights for policy 0, policy_version 496427 (0.0007) [2023-12-26 18:58:44,904][105620] Updated weights for policy 1, policy_version 496770 (0.0009) [2023-12-26 18:58:44,926][105692] Updated weights for policy 0, policy_version 496437 (0.0008) [2023-12-26 18:58:44,980][105692] Updated weights for policy 0, policy_version 496447 (0.0010) [2023-12-26 18:58:45,676][105620] Updated weights for policy 1, policy_version 496780 (0.0008) [2023-12-26 18:58:45,738][105620] Updated weights for policy 1, policy_version 496790 (0.0009) [2023-12-26 18:58:45,755][105692] Updated weights for policy 0, policy_version 496457 (0.0005) [2023-12-26 18:58:45,796][105620] Updated weights for policy 1, policy_version 496800 (0.0008) [2023-12-26 18:58:45,813][105692] Updated weights for policy 0, policy_version 496467 (0.0005) [2023-12-26 18:58:45,863][105692] Updated weights for policy 0, policy_version 496477 (0.0005) [2023-12-26 18:58:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 254312448. Throughput: 0: 9856.6, 1: 9620.6. Samples: 254278424. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 18:58:46,062][104569] Avg episode reward: [(0, '9268.376'), (1, '9259.971')] [2023-12-26 18:58:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000496480_127115264.pth... [2023-12-26 18:58:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000496808_127197184.pth... [2023-12-26 18:58:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000495328_126820352.pth [2023-12-26 18:58:46,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000495720_126918656.pth [2023-12-26 18:58:46,376][105692] Updated weights for policy 0, policy_version 496487 (0.0005) [2023-12-26 18:58:46,441][105692] Updated weights for policy 0, policy_version 496497 (0.0007) [2023-12-26 18:58:46,500][105692] Updated weights for policy 0, policy_version 496507 (0.0009) [2023-12-26 18:58:46,531][105620] Updated weights for policy 1, policy_version 496810 (0.0007) [2023-12-26 18:58:46,589][105620] Updated weights for policy 1, policy_version 496820 (0.0008) [2023-12-26 18:58:46,647][105620] Updated weights for policy 1, policy_version 496830 (0.0008) [2023-12-26 18:58:46,714][105620] Updated weights for policy 1, policy_version 496840 (0.0006) [2023-12-26 18:58:47,135][105692] Updated weights for policy 0, policy_version 496517 (0.0009) [2023-12-26 18:58:47,196][105692] Updated weights for policy 0, policy_version 496527 (0.0008) [2023-12-26 18:58:47,256][105692] Updated weights for policy 0, policy_version 496537 (0.0006) [2023-12-26 18:58:47,345][105620] Updated weights for policy 1, policy_version 496850 (0.0008) [2023-12-26 18:58:47,393][105620] Updated weights for policy 1, policy_version 496860 (0.0005) [2023-12-26 18:58:47,437][105620] Updated weights for policy 1, policy_version 496870 (0.0005) [2023-12-26 18:58:47,999][105692] Updated weights for policy 0, policy_version 496547 (0.0006) [2023-12-26 18:58:48,008][105620] Updated weights for policy 1, policy_version 496880 (0.0006) [2023-12-26 18:58:48,059][105692] Updated weights for policy 0, policy_version 496557 (0.0007) [2023-12-26 18:58:48,069][105620] Updated weights for policy 1, policy_version 496890 (0.0005) [2023-12-26 18:58:48,118][105692] Updated weights for policy 0, policy_version 496567 (0.0009) [2023-12-26 18:58:48,121][105620] Updated weights for policy 1, policy_version 496900 (0.0005) [2023-12-26 18:58:48,735][105620] Updated weights for policy 1, policy_version 496910 (0.0006) [2023-12-26 18:58:48,798][105620] Updated weights for policy 1, policy_version 496920 (0.0005) [2023-12-26 18:58:48,860][105620] Updated weights for policy 1, policy_version 496930 (0.0006) [2023-12-26 18:58:48,875][105692] Updated weights for policy 0, policy_version 496577 (0.0010) [2023-12-26 18:58:48,927][105692] Updated weights for policy 0, policy_version 496588 (0.0010) [2023-12-26 18:58:48,981][105692] Updated weights for policy 0, policy_version 496599 (0.0010) [2023-12-26 18:58:49,433][105620] Updated weights for policy 1, policy_version 496940 (0.0007) [2023-12-26 18:58:49,492][105620] Updated weights for policy 1, policy_version 496950 (0.0010) [2023-12-26 18:58:49,556][105620] Updated weights for policy 1, policy_version 496960 (0.0009) [2023-12-26 18:58:49,732][105692] Updated weights for policy 0, policy_version 496609 (0.0009) [2023-12-26 18:58:49,780][105692] Updated weights for policy 0, policy_version 496619 (0.0010) [2023-12-26 18:58:49,842][105692] Updated weights for policy 0, policy_version 496629 (0.0006) [2023-12-26 18:58:49,912][105692] Updated weights for policy 0, policy_version 496639 (0.0008) [2023-12-26 18:58:50,146][105620] Updated weights for policy 1, policy_version 496970 (0.0010) [2023-12-26 18:58:50,204][105620] Updated weights for policy 1, policy_version 496980 (0.0010) [2023-12-26 18:58:50,269][105620] Updated weights for policy 1, policy_version 496990 (0.0010) [2023-12-26 18:58:50,331][105620] Updated weights for policy 1, policy_version 497000 (0.0010) [2023-12-26 18:58:50,662][105692] Updated weights for policy 0, policy_version 496649 (0.0010) [2023-12-26 18:58:50,740][105692] Updated weights for policy 0, policy_version 496659 (0.0008) [2023-12-26 18:58:50,788][105692] Updated weights for policy 0, policy_version 496669 (0.0008) [2023-12-26 18:58:51,006][105620] Updated weights for policy 1, policy_version 497010 (0.0005) [2023-12-26 18:58:51,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 254410752. Throughput: 0: 9859.8, 1: 9751.8. Samples: 254401940. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 18:58:51,062][104569] Avg episode reward: [(0, '9174.062'), (1, '9261.098')] [2023-12-26 18:58:51,071][105620] Updated weights for policy 1, policy_version 497020 (0.0010) [2023-12-26 18:58:51,130][105620] Updated weights for policy 1, policy_version 497030 (0.0010) [2023-12-26 18:58:51,525][105692] Updated weights for policy 0, policy_version 496679 (0.0010) [2023-12-26 18:58:51,587][105692] Updated weights for policy 0, policy_version 496689 (0.0011) [2023-12-26 18:58:51,652][105692] Updated weights for policy 0, policy_version 496699 (0.0011) [2023-12-26 18:58:51,787][105620] Updated weights for policy 1, policy_version 497040 (0.0008) [2023-12-26 18:58:51,838][105620] Updated weights for policy 1, policy_version 497050 (0.0008) [2023-12-26 18:58:51,900][105620] Updated weights for policy 1, policy_version 497060 (0.0005) [2023-12-26 18:58:52,457][105692] Updated weights for policy 0, policy_version 496709 (0.0009) [2023-12-26 18:58:52,505][105620] Updated weights for policy 1, policy_version 497070 (0.0008) [2023-12-26 18:58:52,517][105692] Updated weights for policy 0, policy_version 496719 (0.0005) [2023-12-26 18:58:52,567][105620] Updated weights for policy 1, policy_version 497080 (0.0010) [2023-12-26 18:58:52,577][105692] Updated weights for policy 0, policy_version 496729 (0.0007) [2023-12-26 18:58:52,622][105620] Updated weights for policy 1, policy_version 497090 (0.0010) [2023-12-26 18:58:53,303][105692] Updated weights for policy 0, policy_version 496739 (0.0007) [2023-12-26 18:58:53,353][105620] Updated weights for policy 1, policy_version 497100 (0.0010) [2023-12-26 18:58:53,356][105692] Updated weights for policy 0, policy_version 496749 (0.0007) [2023-12-26 18:58:53,400][105620] Updated weights for policy 1, policy_version 497110 (0.0006) [2023-12-26 18:58:53,413][105692] Updated weights for policy 0, policy_version 496759 (0.0007) [2023-12-26 18:58:53,452][105620] Updated weights for policy 1, policy_version 497120 (0.0005) [2023-12-26 18:58:54,147][105692] Updated weights for policy 0, policy_version 496769 (0.0007) [2023-12-26 18:58:54,207][105692] Updated weights for policy 0, policy_version 496779 (0.0008) [2023-12-26 18:58:54,237][105620] Updated weights for policy 1, policy_version 497130 (0.0007) [2023-12-26 18:58:54,259][105692] Updated weights for policy 0, policy_version 496789 (0.0008) [2023-12-26 18:58:54,287][105620] Updated weights for policy 1, policy_version 497140 (0.0006) [2023-12-26 18:58:54,311][105692] Updated weights for policy 0, policy_version 496799 (0.0006) [2023-12-26 18:58:54,343][105620] Updated weights for policy 1, policy_version 497150 (0.0009) [2023-12-26 18:58:54,408][105620] Updated weights for policy 1, policy_version 497160 (0.0009) [2023-12-26 18:58:55,079][105620] Updated weights for policy 1, policy_version 497170 (0.0007) [2023-12-26 18:58:55,081][105692] Updated weights for policy 0, policy_version 496809 (0.0006) [2023-12-26 18:58:55,137][105692] Updated weights for policy 0, policy_version 496819 (0.0006) [2023-12-26 18:58:55,139][105620] Updated weights for policy 1, policy_version 497180 (0.0010) [2023-12-26 18:58:55,188][105620] Updated weights for policy 1, policy_version 497190 (0.0010) [2023-12-26 18:58:55,194][105692] Updated weights for policy 0, policy_version 496829 (0.0006) [2023-12-26 18:58:55,794][105620] Updated weights for policy 1, policy_version 497200 (0.0009) [2023-12-26 18:58:55,849][105620] Updated weights for policy 1, policy_version 497210 (0.0009) [2023-12-26 18:58:55,900][105620] Updated weights for policy 1, policy_version 497220 (0.0009) [2023-12-26 18:58:56,019][105692] Updated weights for policy 0, policy_version 496839 (0.0010) [2023-12-26 18:58:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 254509056. Throughput: 0: 9794.5, 1: 9728.2. Samples: 254518088. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 18:58:56,062][104569] Avg episode reward: [(0, '9173.955'), (1, '9348.153')] [2023-12-26 18:58:56,090][105692] Updated weights for policy 0, policy_version 496849 (0.0009) [2023-12-26 18:58:56,150][105692] Updated weights for policy 0, policy_version 496859 (0.0009) [2023-12-26 18:58:56,591][105620] Updated weights for policy 1, policy_version 497230 (0.0009) [2023-12-26 18:58:56,646][105620] Updated weights for policy 1, policy_version 497240 (0.0010) [2023-12-26 18:58:56,702][105620] Updated weights for policy 1, policy_version 497250 (0.0010) [2023-12-26 18:58:56,867][105692] Updated weights for policy 0, policy_version 496869 (0.0007) [2023-12-26 18:58:56,928][105692] Updated weights for policy 0, policy_version 496879 (0.0005) [2023-12-26 18:58:56,985][105692] Updated weights for policy 0, policy_version 496889 (0.0005) [2023-12-26 18:58:57,349][105620] Updated weights for policy 1, policy_version 497260 (0.0009) [2023-12-26 18:58:57,406][105620] Updated weights for policy 1, policy_version 497270 (0.0011) [2023-12-26 18:58:57,461][105620] Updated weights for policy 1, policy_version 497280 (0.0010) [2023-12-26 18:58:57,530][105692] Updated weights for policy 0, policy_version 496899 (0.0005) [2023-12-26 18:58:57,573][105692] Updated weights for policy 0, policy_version 496909 (0.0005) [2023-12-26 18:58:57,618][105692] Updated weights for policy 0, policy_version 496919 (0.0005) [2023-12-26 18:58:58,107][105620] Updated weights for policy 1, policy_version 497290 (0.0009) [2023-12-26 18:58:58,169][105620] Updated weights for policy 1, policy_version 497300 (0.0008) [2023-12-26 18:58:58,227][105620] Updated weights for policy 1, policy_version 497310 (0.0008) [2023-12-26 18:58:58,291][105620] Updated weights for policy 1, policy_version 497320 (0.0006) [2023-12-26 18:58:58,427][105692] Updated weights for policy 0, policy_version 496929 (0.0006) [2023-12-26 18:58:58,498][105692] Updated weights for policy 0, policy_version 496939 (0.0008) [2023-12-26 18:58:58,564][105692] Updated weights for policy 0, policy_version 496949 (0.0008) [2023-12-26 18:58:58,621][105692] Updated weights for policy 0, policy_version 496959 (0.0008) [2023-12-26 18:58:58,968][105620] Updated weights for policy 1, policy_version 497330 (0.0009) [2023-12-26 18:58:59,015][105620] Updated weights for policy 1, policy_version 497340 (0.0008) [2023-12-26 18:58:59,068][105620] Updated weights for policy 1, policy_version 497350 (0.0009) [2023-12-26 18:58:59,409][105692] Updated weights for policy 0, policy_version 496969 (0.0009) [2023-12-26 18:58:59,472][105692] Updated weights for policy 0, policy_version 496979 (0.0009) [2023-12-26 18:58:59,538][105692] Updated weights for policy 0, policy_version 496989 (0.0009) [2023-12-26 18:58:59,858][105620] Updated weights for policy 1, policy_version 497360 (0.0009) [2023-12-26 18:58:59,918][105620] Updated weights for policy 1, policy_version 497370 (0.0008) [2023-12-26 18:58:59,984][105620] Updated weights for policy 1, policy_version 497380 (0.0009) [2023-12-26 18:59:00,334][105692] Updated weights for policy 0, policy_version 496999 (0.0010) [2023-12-26 18:59:00,393][105692] Updated weights for policy 0, policy_version 497010 (0.0009) [2023-12-26 18:59:00,446][105692] Updated weights for policy 0, policy_version 497020 (0.0009) [2023-12-26 18:59:00,616][105620] Updated weights for policy 1, policy_version 497390 (0.0009) [2023-12-26 18:59:00,669][105620] Updated weights for policy 1, policy_version 497400 (0.0009) [2023-12-26 18:59:00,716][105620] Updated weights for policy 1, policy_version 497410 (0.0009) [2023-12-26 18:59:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 254607360. Throughput: 0: 9845.9, 1: 9744.8. Samples: 254579204. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 18:59:01,062][104569] Avg episode reward: [(0, '9266.812'), (1, '9347.791')] [2023-12-26 18:59:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000497416_127352832.pth... [2023-12-26 18:59:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000497024_127254528.pth... [2023-12-26 18:59:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000496264_127057920.pth [2023-12-26 18:59:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000495904_126967808.pth [2023-12-26 18:59:01,194][105692] Updated weights for policy 0, policy_version 497030 (0.0009) [2023-12-26 18:59:01,248][105692] Updated weights for policy 0, policy_version 497040 (0.0008) [2023-12-26 18:59:01,304][105692] Updated weights for policy 0, policy_version 497050 (0.0008) [2023-12-26 18:59:01,461][105620] Updated weights for policy 1, policy_version 497420 (0.0009) [2023-12-26 18:59:01,510][105620] Updated weights for policy 1, policy_version 497430 (0.0008) [2023-12-26 18:59:01,595][105620] Updated weights for policy 1, policy_version 497440 (0.0009) [2023-12-26 18:59:02,112][105692] Updated weights for policy 0, policy_version 497060 (0.0010) [2023-12-26 18:59:02,165][105692] Updated weights for policy 0, policy_version 497070 (0.0010) [2023-12-26 18:59:02,230][105692] Updated weights for policy 0, policy_version 497080 (0.0010) [2023-12-26 18:59:02,246][105620] Updated weights for policy 1, policy_version 497450 (0.0008) [2023-12-26 18:59:02,302][105620] Updated weights for policy 1, policy_version 497461 (0.0008) [2023-12-26 18:59:02,364][105620] Updated weights for policy 1, policy_version 497471 (0.0006) [2023-12-26 18:59:03,013][105620] Updated weights for policy 1, policy_version 497481 (0.0006) [2023-12-26 18:59:03,057][105692] Updated weights for policy 0, policy_version 497090 (0.0008) [2023-12-26 18:59:03,081][105620] Updated weights for policy 1, policy_version 497491 (0.0006) [2023-12-26 18:59:03,113][105692] Updated weights for policy 0, policy_version 497100 (0.0007) [2023-12-26 18:59:03,138][105620] Updated weights for policy 1, policy_version 497501 (0.0006) [2023-12-26 18:59:03,162][105692] Updated weights for policy 0, policy_version 497110 (0.0009) [2023-12-26 18:59:03,199][105620] Updated weights for policy 1, policy_version 497511 (0.0005) [2023-12-26 18:59:03,210][105692] Updated weights for policy 0, policy_version 497120 (0.0009) [2023-12-26 18:59:03,764][105620] Updated weights for policy 1, policy_version 497521 (0.0005) [2023-12-26 18:59:03,823][105620] Updated weights for policy 1, policy_version 497531 (0.0006) [2023-12-26 18:59:03,887][105620] Updated weights for policy 1, policy_version 497541 (0.0009) [2023-12-26 18:59:04,057][105692] Updated weights for policy 0, policy_version 497130 (0.0009) [2023-12-26 18:59:04,119][105692] Updated weights for policy 0, policy_version 497140 (0.0009) [2023-12-26 18:59:04,180][105692] Updated weights for policy 0, policy_version 497150 (0.0008) [2023-12-26 18:59:04,571][105620] Updated weights for policy 1, policy_version 497551 (0.0008) [2023-12-26 18:59:04,618][105620] Updated weights for policy 1, policy_version 497561 (0.0009) [2023-12-26 18:59:04,676][105620] Updated weights for policy 1, policy_version 497571 (0.0009) [2023-12-26 18:59:04,936][105692] Updated weights for policy 0, policy_version 497160 (0.0010) [2023-12-26 18:59:04,997][105692] Updated weights for policy 0, policy_version 497170 (0.0009) [2023-12-26 18:59:05,054][105692] Updated weights for policy 0, policy_version 497180 (0.0008) [2023-12-26 18:59:05,413][105620] Updated weights for policy 1, policy_version 497581 (0.0009) [2023-12-26 18:59:05,471][105620] Updated weights for policy 1, policy_version 497591 (0.0010) [2023-12-26 18:59:05,525][105620] Updated weights for policy 1, policy_version 497601 (0.0009) [2023-12-26 18:59:05,764][105692] Updated weights for policy 0, policy_version 497190 (0.0009) [2023-12-26 18:59:05,824][105692] Updated weights for policy 0, policy_version 497200 (0.0008) [2023-12-26 18:59:05,885][105692] Updated weights for policy 0, policy_version 497210 (0.0010) [2023-12-26 18:59:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 254705664. Throughput: 0: 9725.7, 1: 9835.9. Samples: 254693368. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 18:59:06,063][104569] Avg episode reward: [(0, '9266.440'), (1, '9346.976')] [2023-12-26 18:59:06,292][105620] Updated weights for policy 1, policy_version 497611 (0.0009) [2023-12-26 18:59:06,352][105620] Updated weights for policy 1, policy_version 497621 (0.0009) [2023-12-26 18:59:06,408][105620] Updated weights for policy 1, policy_version 497631 (0.0009) [2023-12-26 18:59:06,615][105692] Updated weights for policy 0, policy_version 497220 (0.0010) [2023-12-26 18:59:06,673][105692] Updated weights for policy 0, policy_version 497230 (0.0009) [2023-12-26 18:59:06,734][105692] Updated weights for policy 0, policy_version 497240 (0.0009) [2023-12-26 18:59:07,114][105620] Updated weights for policy 1, policy_version 497641 (0.0010) [2023-12-26 18:59:07,164][105620] Updated weights for policy 1, policy_version 497651 (0.0009) [2023-12-26 18:59:07,213][105620] Updated weights for policy 1, policy_version 497661 (0.0009) [2023-12-26 18:59:07,268][105620] Updated weights for policy 1, policy_version 497671 (0.0009) [2023-12-26 18:59:07,512][105692] Updated weights for policy 0, policy_version 497250 (0.0010) [2023-12-26 18:59:07,563][105692] Updated weights for policy 0, policy_version 497261 (0.0008) [2023-12-26 18:59:07,624][105692] Updated weights for policy 0, policy_version 497271 (0.0009) [2023-12-26 18:59:07,997][105620] Updated weights for policy 1, policy_version 497681 (0.0007) [2023-12-26 18:59:08,052][105620] Updated weights for policy 1, policy_version 497691 (0.0007) [2023-12-26 18:59:08,105][105620] Updated weights for policy 1, policy_version 497701 (0.0009) [2023-12-26 18:59:08,352][105692] Updated weights for policy 0, policy_version 497281 (0.0009) [2023-12-26 18:59:08,411][105585] KL-divergence is very high: 102.2863 [2023-12-26 18:59:08,412][105692] Updated weights for policy 0, policy_version 497291 (0.0008) [2023-12-26 18:59:08,459][105585] KL-divergence is very high: 171.8752 [2023-12-26 18:59:08,471][105692] Updated weights for policy 0, policy_version 497301 (0.0008) [2023-12-26 18:59:08,504][105585] KL-divergence is very high: 171.2024 [2023-12-26 18:59:08,524][105692] Updated weights for policy 0, policy_version 497311 (0.0009) [2023-12-26 18:59:08,748][105620] Updated weights for policy 1, policy_version 497711 (0.0006) [2023-12-26 18:59:08,811][105620] Updated weights for policy 1, policy_version 497721 (0.0005) [2023-12-26 18:59:08,872][105620] Updated weights for policy 1, policy_version 497731 (0.0006) [2023-12-26 18:59:09,343][105692] Updated weights for policy 0, policy_version 497321 (0.0009) [2023-12-26 18:59:09,415][105692] Updated weights for policy 0, policy_version 497331 (0.0007) [2023-12-26 18:59:09,479][105692] Updated weights for policy 0, policy_version 497341 (0.0010) [2023-12-26 18:59:09,534][105620] Updated weights for policy 1, policy_version 497741 (0.0006) [2023-12-26 18:59:09,598][105620] Updated weights for policy 1, policy_version 497751 (0.0009) [2023-12-26 18:59:09,661][105620] Updated weights for policy 1, policy_version 497761 (0.0009) [2023-12-26 18:59:10,257][105692] Updated weights for policy 0, policy_version 497351 (0.0010) [2023-12-26 18:59:10,314][105692] Updated weights for policy 0, policy_version 497361 (0.0011) [2023-12-26 18:59:10,351][105585] KL-divergence is very high: 139.8642 [2023-12-26 18:59:10,353][105620] Updated weights for policy 1, policy_version 497771 (0.0009) [2023-12-26 18:59:10,377][105692] Updated weights for policy 0, policy_version 497371 (0.0011) [2023-12-26 18:59:10,396][105585] KL-divergence is very high: 125.9523 [2023-12-26 18:59:10,411][105620] Updated weights for policy 1, policy_version 497781 (0.0006) [2023-12-26 18:59:10,471][105620] Updated weights for policy 1, policy_version 497791 (0.0006) [2023-12-26 18:59:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 254795776. Throughput: 0: 9657.8, 1: 9975.7. Samples: 254807780. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 18:59:11,063][104569] Avg episode reward: [(0, '9173.373'), (1, '9256.689')] [2023-12-26 18:59:11,098][105692] Updated weights for policy 0, policy_version 497381 (0.0010) [2023-12-26 18:59:11,167][105692] Updated weights for policy 0, policy_version 497391 (0.0008) [2023-12-26 18:59:11,227][105692] Updated weights for policy 0, policy_version 497401 (0.0008) [2023-12-26 18:59:11,236][105620] Updated weights for policy 1, policy_version 497801 (0.0008) [2023-12-26 18:59:11,296][105620] Updated weights for policy 1, policy_version 497811 (0.0009) [2023-12-26 18:59:11,351][105620] Updated weights for policy 1, policy_version 497821 (0.0008) [2023-12-26 18:59:11,419][105620] Updated weights for policy 1, policy_version 497831 (0.0008) [2023-12-26 18:59:12,017][105692] Updated weights for policy 0, policy_version 497411 (0.0007) [2023-12-26 18:59:12,077][105585] KL-divergence is very high: 195.1579 [2023-12-26 18:59:12,081][105692] Updated weights for policy 0, policy_version 497421 (0.0010) [2023-12-26 18:59:12,123][105585] KL-divergence is very high: 214.3082 [2023-12-26 18:59:12,137][105620] Updated weights for policy 1, policy_version 497841 (0.0008) [2023-12-26 18:59:12,139][105692] Updated weights for policy 0, policy_version 497431 (0.0007) [2023-12-26 18:59:12,167][105585] KL-divergence is very high: 175.1312 [2023-12-26 18:59:12,191][105620] Updated weights for policy 1, policy_version 497851 (0.0008) [2023-12-26 18:59:12,250][105620] Updated weights for policy 1, policy_version 497861 (0.0008) [2023-12-26 18:59:12,911][105692] Updated weights for policy 0, policy_version 497441 (0.0007) [2023-12-26 18:59:12,980][105692] Updated weights for policy 0, policy_version 497451 (0.0011) [2023-12-26 18:59:13,020][105620] Updated weights for policy 1, policy_version 497871 (0.0006) [2023-12-26 18:59:13,035][105692] Updated weights for policy 0, policy_version 497461 (0.0010) [2023-12-26 18:59:13,076][105620] Updated weights for policy 1, policy_version 497881 (0.0007) [2023-12-26 18:59:13,084][105692] Updated weights for policy 0, policy_version 497471 (0.0010) [2023-12-26 18:59:13,123][105620] Updated weights for policy 1, policy_version 497891 (0.0007) [2023-12-26 18:59:13,825][105692] Updated weights for policy 0, policy_version 497481 (0.0011) [2023-12-26 18:59:13,874][105620] Updated weights for policy 1, policy_version 497901 (0.0007) [2023-12-26 18:59:13,876][105692] Updated weights for policy 0, policy_version 497491 (0.0010) [2023-12-26 18:59:13,930][105620] Updated weights for policy 1, policy_version 497911 (0.0007) [2023-12-26 18:59:13,938][105692] Updated weights for policy 0, policy_version 497501 (0.0010) [2023-12-26 18:59:13,980][105620] Updated weights for policy 1, policy_version 497921 (0.0008) [2023-12-26 18:59:14,678][105692] Updated weights for policy 0, policy_version 497511 (0.0010) [2023-12-26 18:59:14,723][105692] Updated weights for policy 0, policy_version 497521 (0.0010) [2023-12-26 18:59:14,741][105620] Updated weights for policy 1, policy_version 497931 (0.0008) [2023-12-26 18:59:14,786][105692] Updated weights for policy 0, policy_version 497531 (0.0010) [2023-12-26 18:59:14,801][105620] Updated weights for policy 1, policy_version 497941 (0.0007) [2023-12-26 18:59:14,860][105620] Updated weights for policy 1, policy_version 497951 (0.0008) [2023-12-26 18:59:15,554][105692] Updated weights for policy 0, policy_version 497541 (0.0011) [2023-12-26 18:59:15,606][105692] Updated weights for policy 0, policy_version 497551 (0.0010) [2023-12-26 18:59:15,621][105620] Updated weights for policy 1, policy_version 497961 (0.0008) [2023-12-26 18:59:15,662][105692] Updated weights for policy 0, policy_version 497561 (0.0010) [2023-12-26 18:59:15,669][105620] Updated weights for policy 1, policy_version 497971 (0.0010) [2023-12-26 18:59:15,728][105620] Updated weights for policy 1, policy_version 497981 (0.0010) [2023-12-26 18:59:15,789][105620] Updated weights for policy 1, policy_version 497991 (0.0010) [2023-12-26 18:59:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 254894080. Throughput: 0: 9587.0, 1: 9912.2. Samples: 254863476. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 18:59:16,062][104569] Avg episode reward: [(0, '8713.737'), (1, '9256.468')] [2023-12-26 18:59:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000497568_127393792.pth... [2023-12-26 18:59:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000497992_127500288.pth... [2023-12-26 18:59:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000496480_127115264.pth [2023-12-26 18:59:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000496808_127197184.pth [2023-12-26 18:59:16,386][105692] Updated weights for policy 0, policy_version 497571 (0.0009) [2023-12-26 18:59:16,444][105692] Updated weights for policy 0, policy_version 497581 (0.0005) [2023-12-26 18:59:16,492][105620] Updated weights for policy 1, policy_version 498001 (0.0008) [2023-12-26 18:59:16,508][105692] Updated weights for policy 0, policy_version 497591 (0.0006) [2023-12-26 18:59:16,546][105620] Updated weights for policy 1, policy_version 498011 (0.0008) [2023-12-26 18:59:16,602][105620] Updated weights for policy 1, policy_version 498021 (0.0006) [2023-12-26 18:59:17,197][105692] Updated weights for policy 0, policy_version 497601 (0.0010) [2023-12-26 18:59:17,211][105620] Updated weights for policy 1, policy_version 498031 (0.0007) [2023-12-26 18:59:17,252][105692] Updated weights for policy 0, policy_version 497611 (0.0011) [2023-12-26 18:59:17,267][105620] Updated weights for policy 1, policy_version 498041 (0.0006) [2023-12-26 18:59:17,315][105692] Updated weights for policy 0, policy_version 497621 (0.0010) [2023-12-26 18:59:17,323][105620] Updated weights for policy 1, policy_version 498051 (0.0008) [2023-12-26 18:59:17,370][105692] Updated weights for policy 0, policy_version 497631 (0.0011) [2023-12-26 18:59:18,015][105692] Updated weights for policy 0, policy_version 497641 (0.0006) [2023-12-26 18:59:18,060][105585] KL-divergence is very high: 146.3284 [2023-12-26 18:59:18,063][105692] Updated weights for policy 0, policy_version 497651 (0.0006) [2023-12-26 18:59:18,102][105620] Updated weights for policy 1, policy_version 498061 (0.0009) [2023-12-26 18:59:18,104][105585] KL-divergence is very high: 112.8368 [2023-12-26 18:59:18,116][105692] Updated weights for policy 0, policy_version 497661 (0.0005) [2023-12-26 18:59:18,151][105620] Updated weights for policy 1, policy_version 498071 (0.0009) [2023-12-26 18:59:18,202][105620] Updated weights for policy 1, policy_version 498081 (0.0009) [2023-12-26 18:59:18,688][105692] Updated weights for policy 0, policy_version 497671 (0.0008) [2023-12-26 18:59:18,754][105692] Updated weights for policy 0, policy_version 497681 (0.0009) [2023-12-26 18:59:18,810][105692] Updated weights for policy 0, policy_version 497691 (0.0008) [2023-12-26 18:59:19,065][105620] Updated weights for policy 1, policy_version 498091 (0.0009) [2023-12-26 18:59:19,130][105620] Updated weights for policy 1, policy_version 498101 (0.0010) [2023-12-26 18:59:19,189][105620] Updated weights for policy 1, policy_version 498111 (0.0010) [2023-12-26 18:59:19,425][105692] Updated weights for policy 0, policy_version 497701 (0.0005) [2023-12-26 18:59:19,485][105692] Updated weights for policy 0, policy_version 497711 (0.0007) [2023-12-26 18:59:19,552][105692] Updated weights for policy 0, policy_version 497721 (0.0006) [2023-12-26 18:59:19,844][105620] Updated weights for policy 1, policy_version 498121 (0.0009) [2023-12-26 18:59:19,925][105620] Updated weights for policy 1, policy_version 498131 (0.0009) [2023-12-26 18:59:19,991][105620] Updated weights for policy 1, policy_version 498141 (0.0009) [2023-12-26 18:59:20,057][105620] Updated weights for policy 1, policy_version 498151 (0.0008) [2023-12-26 18:59:20,256][105692] Updated weights for policy 0, policy_version 497731 (0.0009) [2023-12-26 18:59:20,304][105692] Updated weights for policy 0, policy_version 497741 (0.0009) [2023-12-26 18:59:20,350][105692] Updated weights for policy 0, policy_version 497751 (0.0009) [2023-12-26 18:59:20,737][105620] Updated weights for policy 1, policy_version 498161 (0.0009) [2023-12-26 18:59:20,801][105620] Updated weights for policy 1, policy_version 498171 (0.0009) [2023-12-26 18:59:20,856][105620] Updated weights for policy 1, policy_version 498181 (0.0009) [2023-12-26 18:59:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 254992384. Throughput: 0: 9549.8, 1: 9963.9. Samples: 254981408. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 18:59:21,062][104569] Avg episode reward: [(0, '8627.100'), (1, '9345.959')] [2023-12-26 18:59:21,165][105692] Updated weights for policy 0, policy_version 497761 (0.0010) [2023-12-26 18:59:21,223][105692] Updated weights for policy 0, policy_version 497771 (0.0009) [2023-12-26 18:59:21,291][105692] Updated weights for policy 0, policy_version 497781 (0.0005) [2023-12-26 18:59:21,357][105692] Updated weights for policy 0, policy_version 497791 (0.0007) [2023-12-26 18:59:21,615][105620] Updated weights for policy 1, policy_version 498191 (0.0009) [2023-12-26 18:59:21,679][105620] Updated weights for policy 1, policy_version 498201 (0.0009) [2023-12-26 18:59:21,742][105620] Updated weights for policy 1, policy_version 498211 (0.0009) [2023-12-26 18:59:21,990][105692] Updated weights for policy 0, policy_version 497801 (0.0007) [2023-12-26 18:59:22,056][105692] Updated weights for policy 0, policy_version 497811 (0.0008) [2023-12-26 18:59:22,118][105692] Updated weights for policy 0, policy_version 497821 (0.0009) [2023-12-26 18:59:22,586][105620] Updated weights for policy 1, policy_version 498221 (0.0009) [2023-12-26 18:59:22,648][105620] Updated weights for policy 1, policy_version 498231 (0.0009) [2023-12-26 18:59:22,715][105620] Updated weights for policy 1, policy_version 498241 (0.0008) [2023-12-26 18:59:22,865][105692] Updated weights for policy 0, policy_version 497831 (0.0009) [2023-12-26 18:59:22,916][105692] Updated weights for policy 0, policy_version 497841 (0.0008) [2023-12-26 18:59:22,975][105692] Updated weights for policy 0, policy_version 497851 (0.0009) [2023-12-26 18:59:23,450][105620] Updated weights for policy 1, policy_version 498251 (0.0008) [2023-12-26 18:59:23,500][105620] Updated weights for policy 1, policy_version 498261 (0.0005) [2023-12-26 18:59:23,547][105620] Updated weights for policy 1, policy_version 498271 (0.0005) [2023-12-26 18:59:23,657][105692] Updated weights for policy 0, policy_version 497861 (0.0009) [2023-12-26 18:59:23,710][105692] Updated weights for policy 0, policy_version 497871 (0.0010) [2023-12-26 18:59:23,764][105692] Updated weights for policy 0, policy_version 497881 (0.0009) [2023-12-26 18:59:24,107][105620] Updated weights for policy 1, policy_version 498281 (0.0006) [2023-12-26 18:59:24,170][105620] Updated weights for policy 1, policy_version 498291 (0.0009) [2023-12-26 18:59:24,229][105620] Updated weights for policy 1, policy_version 498301 (0.0009) [2023-12-26 18:59:24,287][105620] Updated weights for policy 1, policy_version 498311 (0.0009) [2023-12-26 18:59:24,595][105692] Updated weights for policy 0, policy_version 497891 (0.0010) [2023-12-26 18:59:24,658][105692] Updated weights for policy 0, policy_version 497901 (0.0008) [2023-12-26 18:59:24,716][105692] Updated weights for policy 0, policy_version 497911 (0.0008) [2023-12-26 18:59:24,982][105620] Updated weights for policy 1, policy_version 498321 (0.0009) [2023-12-26 18:59:25,037][105620] Updated weights for policy 1, policy_version 498331 (0.0009) [2023-12-26 18:59:25,096][105620] Updated weights for policy 1, policy_version 498341 (0.0009) [2023-12-26 18:59:25,476][105692] Updated weights for policy 0, policy_version 497921 (0.0009) [2023-12-26 18:59:25,533][105692] Updated weights for policy 0, policy_version 497931 (0.0009) [2023-12-26 18:59:25,595][105692] Updated weights for policy 0, policy_version 497941 (0.0009) [2023-12-26 18:59:25,655][105692] Updated weights for policy 0, policy_version 497951 (0.0009) [2023-12-26 18:59:25,850][105620] Updated weights for policy 1, policy_version 498351 (0.0009) [2023-12-26 18:59:25,900][105620] Updated weights for policy 1, policy_version 498361 (0.0009) [2023-12-26 18:59:25,953][105620] Updated weights for policy 1, policy_version 498372 (0.0009) [2023-12-26 18:59:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 255090688. Throughput: 0: 9461.1, 1: 9943.8. Samples: 255095360. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 18:59:26,062][104569] Avg episode reward: [(0, '8539.178'), (1, '9347.600')] [2023-12-26 18:59:26,434][105692] Updated weights for policy 0, policy_version 497961 (0.0009) [2023-12-26 18:59:26,488][105692] Updated weights for policy 0, policy_version 497971 (0.0008) [2023-12-26 18:59:26,542][105692] Updated weights for policy 0, policy_version 497981 (0.0009) [2023-12-26 18:59:26,661][105620] Updated weights for policy 1, policy_version 498382 (0.0009) [2023-12-26 18:59:26,718][105620] Updated weights for policy 1, policy_version 498392 (0.0007) [2023-12-26 18:59:26,778][105620] Updated weights for policy 1, policy_version 498402 (0.0005) [2023-12-26 18:59:27,249][105692] Updated weights for policy 0, policy_version 497991 (0.0009) [2023-12-26 18:59:27,300][105692] Updated weights for policy 0, policy_version 498001 (0.0009) [2023-12-26 18:59:27,359][105692] Updated weights for policy 0, policy_version 498011 (0.0009) [2023-12-26 18:59:27,481][105620] Updated weights for policy 1, policy_version 498412 (0.0006) [2023-12-26 18:59:27,543][105620] Updated weights for policy 1, policy_version 498422 (0.0008) [2023-12-26 18:59:27,612][105620] Updated weights for policy 1, policy_version 498432 (0.0009) [2023-12-26 18:59:28,110][105692] Updated weights for policy 0, policy_version 498021 (0.0009) [2023-12-26 18:59:28,156][105692] Updated weights for policy 0, policy_version 498031 (0.0008) [2023-12-26 18:59:28,204][105692] Updated weights for policy 0, policy_version 498041 (0.0009) [2023-12-26 18:59:28,334][105620] Updated weights for policy 1, policy_version 498442 (0.0008) [2023-12-26 18:59:28,397][105620] Updated weights for policy 1, policy_version 498452 (0.0009) [2023-12-26 18:59:28,456][105620] Updated weights for policy 1, policy_version 498462 (0.0009) [2023-12-26 18:59:28,511][105620] Updated weights for policy 1, policy_version 498472 (0.0009) [2023-12-26 18:59:29,017][105692] Updated weights for policy 0, policy_version 498051 (0.0009) [2023-12-26 18:59:29,075][105692] Updated weights for policy 0, policy_version 498061 (0.0009) [2023-12-26 18:59:29,122][105692] Updated weights for policy 0, policy_version 498071 (0.0009) [2023-12-26 18:59:29,196][105620] Updated weights for policy 1, policy_version 498482 (0.0009) [2023-12-26 18:59:29,265][105620] Updated weights for policy 1, policy_version 498492 (0.0009) [2023-12-26 18:59:29,323][105620] Updated weights for policy 1, policy_version 498502 (0.0009) [2023-12-26 18:59:29,756][105692] Updated weights for policy 0, policy_version 498081 (0.0006) [2023-12-26 18:59:29,815][105692] Updated weights for policy 0, policy_version 498091 (0.0006) [2023-12-26 18:59:29,882][105692] Updated weights for policy 0, policy_version 498101 (0.0007) [2023-12-26 18:59:29,944][105692] Updated weights for policy 0, policy_version 498111 (0.0007) [2023-12-26 18:59:30,127][105620] Updated weights for policy 1, policy_version 498512 (0.0009) [2023-12-26 18:59:30,189][105620] Updated weights for policy 1, policy_version 498522 (0.0011) [2023-12-26 18:59:30,247][105620] Updated weights for policy 1, policy_version 498532 (0.0009) [2023-12-26 18:59:30,563][105692] Updated weights for policy 0, policy_version 498121 (0.0009) [2023-12-26 18:59:30,616][105692] Updated weights for policy 0, policy_version 498131 (0.0009) [2023-12-26 18:59:30,671][105692] Updated weights for policy 0, policy_version 498141 (0.0009) [2023-12-26 18:59:30,873][105620] Updated weights for policy 1, policy_version 498542 (0.0006) [2023-12-26 18:59:30,934][105620] Updated weights for policy 1, policy_version 498552 (0.0005) [2023-12-26 18:59:30,986][105620] Updated weights for policy 1, policy_version 498562 (0.0006) [2023-12-26 18:59:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 255188992. Throughput: 0: 9492.7, 1: 9944.5. Samples: 255153100. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 18:59:31,062][104569] Avg episode reward: [(0, '2559.779'), (1, '9349.606')] [2023-12-26 18:59:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000498144_127541248.pth... [2023-12-26 18:59:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000498568_127647744.pth... [2023-12-26 18:59:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000497024_127254528.pth [2023-12-26 18:59:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000497416_127352832.pth [2023-12-26 18:59:31,466][105692] Updated weights for policy 0, policy_version 498151 (0.0010) [2023-12-26 18:59:31,519][105692] Updated weights for policy 0, policy_version 498161 (0.0009) [2023-12-26 18:59:31,571][105692] Updated weights for policy 0, policy_version 498171 (0.0009) [2023-12-26 18:59:31,580][105620] Updated weights for policy 1, policy_version 498572 (0.0005) [2023-12-26 18:59:31,645][105620] Updated weights for policy 1, policy_version 498582 (0.0007) [2023-12-26 18:59:31,700][105620] Updated weights for policy 1, policy_version 498592 (0.0006) [2023-12-26 18:59:32,266][105620] Updated weights for policy 1, policy_version 498602 (0.0007) [2023-12-26 18:59:32,328][105620] Updated weights for policy 1, policy_version 498612 (0.0009) [2023-12-26 18:59:32,393][105620] Updated weights for policy 1, policy_version 498622 (0.0008) [2023-12-26 18:59:32,450][105620] Updated weights for policy 1, policy_version 498632 (0.0008) [2023-12-26 18:59:32,458][105692] Updated weights for policy 0, policy_version 498181 (0.0008) [2023-12-26 18:59:32,525][105692] Updated weights for policy 0, policy_version 498191 (0.0008) [2023-12-26 18:59:32,588][105692] Updated weights for policy 0, policy_version 498201 (0.0008) [2023-12-26 18:59:33,093][105620] Updated weights for policy 1, policy_version 498642 (0.0006) [2023-12-26 18:59:33,140][105620] Updated weights for policy 1, policy_version 498652 (0.0008) [2023-12-26 18:59:33,187][105620] Updated weights for policy 1, policy_version 498662 (0.0008) [2023-12-26 18:59:33,386][105692] Updated weights for policy 0, policy_version 498211 (0.0009) [2023-12-26 18:59:33,433][105692] Updated weights for policy 0, policy_version 498221 (0.0009) [2023-12-26 18:59:33,479][105692] Updated weights for policy 0, policy_version 498231 (0.0009) [2023-12-26 18:59:33,914][105620] Updated weights for policy 1, policy_version 498672 (0.0006) [2023-12-26 18:59:33,969][105620] Updated weights for policy 1, policy_version 498682 (0.0005) [2023-12-26 18:59:34,030][105620] Updated weights for policy 1, policy_version 498692 (0.0005) [2023-12-26 18:59:34,311][105692] Updated weights for policy 0, policy_version 498241 (0.0008) [2023-12-26 18:59:34,370][105692] Updated weights for policy 0, policy_version 498251 (0.0010) [2023-12-26 18:59:34,427][105692] Updated weights for policy 0, policy_version 498261 (0.0009) [2023-12-26 18:59:34,493][105692] Updated weights for policy 0, policy_version 498271 (0.0008) [2023-12-26 18:59:34,595][105620] Updated weights for policy 1, policy_version 498702 (0.0005) [2023-12-26 18:59:34,655][105620] Updated weights for policy 1, policy_version 498712 (0.0005) [2023-12-26 18:59:34,714][105620] Updated weights for policy 1, policy_version 498722 (0.0005) [2023-12-26 18:59:35,314][105692] Updated weights for policy 0, policy_version 498281 (0.0009) [2023-12-26 18:59:35,370][105692] Updated weights for policy 0, policy_version 498291 (0.0009) [2023-12-26 18:59:35,375][105620] Updated weights for policy 1, policy_version 498732 (0.0007) [2023-12-26 18:59:35,418][105692] Updated weights for policy 0, policy_version 498301 (0.0009) [2023-12-26 18:59:35,421][105620] Updated weights for policy 1, policy_version 498742 (0.0005) [2023-12-26 18:59:35,471][105620] Updated weights for policy 1, policy_version 498752 (0.0005) [2023-12-26 18:59:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 255279104. Throughput: 0: 9347.6, 1: 9965.4. Samples: 255271028. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 18:59:36,062][104569] Avg episode reward: [(0, '690.656'), (1, '9350.762')] [2023-12-26 18:59:36,164][105620] Updated weights for policy 1, policy_version 498762 (0.0005) [2023-12-26 18:59:36,178][105692] Updated weights for policy 0, policy_version 498311 (0.0008) [2023-12-26 18:59:36,220][105620] Updated weights for policy 1, policy_version 498772 (0.0008) [2023-12-26 18:59:36,238][105692] Updated weights for policy 0, policy_version 498321 (0.0008) [2023-12-26 18:59:36,277][105620] Updated weights for policy 1, policy_version 498782 (0.0008) [2023-12-26 18:59:36,293][105692] Updated weights for policy 0, policy_version 498331 (0.0008) [2023-12-26 18:59:36,340][105620] Updated weights for policy 1, policy_version 498792 (0.0009) [2023-12-26 18:59:37,001][105692] Updated weights for policy 0, policy_version 498341 (0.0008) [2023-12-26 18:59:37,063][105692] Updated weights for policy 0, policy_version 498351 (0.0006) [2023-12-26 18:59:37,115][105692] Updated weights for policy 0, policy_version 498361 (0.0005) [2023-12-26 18:59:37,119][105620] Updated weights for policy 1, policy_version 498802 (0.0008) [2023-12-26 18:59:37,173][105620] Updated weights for policy 1, policy_version 498812 (0.0009) [2023-12-26 18:59:37,231][105620] Updated weights for policy 1, policy_version 498824 (0.0010) [2023-12-26 18:59:37,715][105692] Updated weights for policy 0, policy_version 498371 (0.0006) [2023-12-26 18:59:37,777][105692] Updated weights for policy 0, policy_version 498381 (0.0007) [2023-12-26 18:59:37,844][105692] Updated weights for policy 0, policy_version 498391 (0.0010) [2023-12-26 18:59:38,061][105620] Updated weights for policy 1, policy_version 498834 (0.0008) [2023-12-26 18:59:38,108][105620] Updated weights for policy 1, policy_version 498844 (0.0008) [2023-12-26 18:59:38,167][105620] Updated weights for policy 1, policy_version 498854 (0.0008) [2023-12-26 18:59:38,551][105692] Updated weights for policy 0, policy_version 498401 (0.0010) [2023-12-26 18:59:38,601][105692] Updated weights for policy 0, policy_version 498411 (0.0009) [2023-12-26 18:59:38,661][105692] Updated weights for policy 0, policy_version 498421 (0.0008) [2023-12-26 18:59:38,723][105692] Updated weights for policy 0, policy_version 498431 (0.0008) [2023-12-26 18:59:38,931][105620] Updated weights for policy 1, policy_version 498864 (0.0009) [2023-12-26 18:59:38,993][105620] Updated weights for policy 1, policy_version 498874 (0.0009) [2023-12-26 18:59:39,051][105620] Updated weights for policy 1, policy_version 498884 (0.0009) [2023-12-26 18:59:39,497][105692] Updated weights for policy 0, policy_version 498441 (0.0008) [2023-12-26 18:59:39,562][105692] Updated weights for policy 0, policy_version 498452 (0.0009) [2023-12-26 18:59:39,616][105692] Updated weights for policy 0, policy_version 498463 (0.0010) [2023-12-26 18:59:39,787][105620] Updated weights for policy 1, policy_version 498894 (0.0007) [2023-12-26 18:59:39,862][105620] Updated weights for policy 1, policy_version 498904 (0.0009) [2023-12-26 18:59:39,928][105620] Updated weights for policy 1, policy_version 498914 (0.0011) [2023-12-26 18:59:40,341][105692] Updated weights for policy 0, policy_version 498473 (0.0006) [2023-12-26 18:59:40,401][105692] Updated weights for policy 0, policy_version 498483 (0.0006) [2023-12-26 18:59:40,452][105692] Updated weights for policy 0, policy_version 498493 (0.0006) [2023-12-26 18:59:40,599][105620] Updated weights for policy 1, policy_version 498924 (0.0011) [2023-12-26 18:59:40,661][105620] Updated weights for policy 1, policy_version 498934 (0.0006) [2023-12-26 18:59:40,728][105620] Updated weights for policy 1, policy_version 498944 (0.0009) [2023-12-26 18:59:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 255377408. Throughput: 0: 9427.3, 1: 9870.2. Samples: 255386476. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 18:59:41,062][104569] Avg episode reward: [(0, '2573.935'), (1, '9351.188')] [2023-12-26 18:59:41,143][105692] Updated weights for policy 0, policy_version 498503 (0.0008) [2023-12-26 18:59:41,209][105692] Updated weights for policy 0, policy_version 498513 (0.0009) [2023-12-26 18:59:41,267][105692] Updated weights for policy 0, policy_version 498523 (0.0009) [2023-12-26 18:59:41,453][105620] Updated weights for policy 1, policy_version 498954 (0.0008) [2023-12-26 18:59:41,515][105620] Updated weights for policy 1, policy_version 498964 (0.0005) [2023-12-26 18:59:41,574][105620] Updated weights for policy 1, policy_version 498974 (0.0007) [2023-12-26 18:59:41,637][105620] Updated weights for policy 1, policy_version 498984 (0.0008) [2023-12-26 18:59:42,147][105692] Updated weights for policy 0, policy_version 498533 (0.0008) [2023-12-26 18:59:42,217][105692] Updated weights for policy 0, policy_version 498543 (0.0008) [2023-12-26 18:59:42,270][105620] Updated weights for policy 1, policy_version 498994 (0.0007) [2023-12-26 18:59:42,279][105692] Updated weights for policy 0, policy_version 498553 (0.0009) [2023-12-26 18:59:42,326][105620] Updated weights for policy 1, policy_version 499004 (0.0008) [2023-12-26 18:59:42,385][105620] Updated weights for policy 1, policy_version 499014 (0.0008) [2023-12-26 18:59:42,945][105692] Updated weights for policy 0, policy_version 498563 (0.0009) [2023-12-26 18:59:43,007][105620] Updated weights for policy 1, policy_version 499024 (0.0006) [2023-12-26 18:59:43,013][105692] Updated weights for policy 0, policy_version 498573 (0.0009) [2023-12-26 18:59:43,058][105620] Updated weights for policy 1, policy_version 499034 (0.0005) [2023-12-26 18:59:43,079][105692] Updated weights for policy 0, policy_version 498583 (0.0009) [2023-12-26 18:59:43,110][105620] Updated weights for policy 1, policy_version 499044 (0.0007) [2023-12-26 18:59:43,754][105620] Updated weights for policy 1, policy_version 499054 (0.0006) [2023-12-26 18:59:43,807][105620] Updated weights for policy 1, policy_version 499064 (0.0005) [2023-12-26 18:59:43,872][105620] Updated weights for policy 1, policy_version 499074 (0.0007) [2023-12-26 18:59:43,895][105692] Updated weights for policy 0, policy_version 498593 (0.0008) [2023-12-26 18:59:43,946][105692] Updated weights for policy 0, policy_version 498604 (0.0010) [2023-12-26 18:59:43,993][105692] Updated weights for policy 0, policy_version 498614 (0.0008) [2023-12-26 18:59:44,047][105692] Updated weights for policy 0, policy_version 498624 (0.0009) [2023-12-26 18:59:44,555][105620] Updated weights for policy 1, policy_version 499084 (0.0008) [2023-12-26 18:59:44,601][105620] Updated weights for policy 1, policy_version 499094 (0.0009) [2023-12-26 18:59:44,648][105620] Updated weights for policy 1, policy_version 499104 (0.0008) [2023-12-26 18:59:44,838][105692] Updated weights for policy 0, policy_version 498634 (0.0007) [2023-12-26 18:59:44,901][105692] Updated weights for policy 0, policy_version 498644 (0.0006) [2023-12-26 18:59:44,963][105692] Updated weights for policy 0, policy_version 498654 (0.0007) [2023-12-26 18:59:45,462][105620] Updated weights for policy 1, policy_version 499114 (0.0008) [2023-12-26 18:59:45,523][105620] Updated weights for policy 1, policy_version 499124 (0.0006) [2023-12-26 18:59:45,585][105620] Updated weights for policy 1, policy_version 499134 (0.0007) [2023-12-26 18:59:45,608][105692] Updated weights for policy 0, policy_version 498664 (0.0010) [2023-12-26 18:59:45,634][105620] Updated weights for policy 1, policy_version 499144 (0.0005) [2023-12-26 18:59:45,664][105692] Updated weights for policy 0, policy_version 498674 (0.0010) [2023-12-26 18:59:45,716][105585] KL-divergence is very high: 131.9066 [2023-12-26 18:59:45,723][105692] Updated weights for policy 0, policy_version 498684 (0.0010) [2023-12-26 18:59:46,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19387.6, 300 sec: 19494.2). Total num frames: 255475712. Throughput: 0: 9372.1, 1: 9879.5. Samples: 255445532. Policy #0 lag: (min: 9.0, avg: 22.2, max: 41.0) [2023-12-26 18:59:46,063][104569] Avg episode reward: [(0, '1201.001'), (1, '9350.922')] [2023-12-26 18:59:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000499144_127795200.pth... [2023-12-26 18:59:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000498688_127680512.pth... [2023-12-26 18:59:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000497992_127500288.pth [2023-12-26 18:59:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000497568_127393792.pth [2023-12-26 18:59:46,143][105620] Updated weights for policy 1, policy_version 499154 (0.0006) [2023-12-26 18:59:46,198][105620] Updated weights for policy 1, policy_version 499164 (0.0005) [2023-12-26 18:59:46,241][105620] Updated weights for policy 1, policy_version 499174 (0.0005) [2023-12-26 18:59:46,416][105692] Updated weights for policy 0, policy_version 498694 (0.0006) [2023-12-26 18:59:46,475][105692] Updated weights for policy 0, policy_version 498704 (0.0006) [2023-12-26 18:59:46,533][105692] Updated weights for policy 0, policy_version 498714 (0.0006) [2023-12-26 18:59:46,554][105585] KL-divergence is very high: 116.6773 [2023-12-26 18:59:46,859][105620] Updated weights for policy 1, policy_version 499184 (0.0005) [2023-12-26 18:59:46,913][105620] Updated weights for policy 1, policy_version 499194 (0.0005) [2023-12-26 18:59:46,974][105620] Updated weights for policy 1, policy_version 499204 (0.0006) [2023-12-26 18:59:47,217][105692] Updated weights for policy 0, policy_version 498724 (0.0007) [2023-12-26 18:59:47,268][105692] Updated weights for policy 0, policy_version 498734 (0.0010) [2023-12-26 18:59:47,319][105692] Updated weights for policy 0, policy_version 498744 (0.0010) [2023-12-26 18:59:47,672][105620] Updated weights for policy 1, policy_version 499214 (0.0009) [2023-12-26 18:59:47,731][105620] Updated weights for policy 1, policy_version 499224 (0.0010) [2023-12-26 18:59:47,792][105620] Updated weights for policy 1, policy_version 499234 (0.0010) [2023-12-26 18:59:47,998][105692] Updated weights for policy 0, policy_version 498754 (0.0010) [2023-12-26 18:59:48,053][105692] Updated weights for policy 0, policy_version 498764 (0.0005) [2023-12-26 18:59:48,120][105692] Updated weights for policy 0, policy_version 498774 (0.0005) [2023-12-26 18:59:48,185][105692] Updated weights for policy 0, policy_version 498784 (0.0008) [2023-12-26 18:59:48,525][105620] Updated weights for policy 1, policy_version 499244 (0.0009) [2023-12-26 18:59:48,587][105620] Updated weights for policy 1, policy_version 499254 (0.0006) [2023-12-26 18:59:48,651][105620] Updated weights for policy 1, policy_version 499264 (0.0006) [2023-12-26 18:59:48,811][105692] Updated weights for policy 0, policy_version 498794 (0.0005) [2023-12-26 18:59:48,867][105692] Updated weights for policy 0, policy_version 498804 (0.0010) [2023-12-26 18:59:48,929][105692] Updated weights for policy 0, policy_version 498814 (0.0006) [2023-12-26 18:59:49,400][105620] Updated weights for policy 1, policy_version 499274 (0.0006) [2023-12-26 18:59:49,464][105620] Updated weights for policy 1, policy_version 499284 (0.0006) [2023-12-26 18:59:49,527][105620] Updated weights for policy 1, policy_version 499294 (0.0006) [2023-12-26 18:59:49,587][105620] Updated weights for policy 1, policy_version 499304 (0.0009) [2023-12-26 18:59:49,595][105692] Updated weights for policy 0, policy_version 498824 (0.0010) [2023-12-26 18:59:49,661][105692] Updated weights for policy 0, policy_version 498834 (0.0010) [2023-12-26 18:59:49,723][105692] Updated weights for policy 0, policy_version 498844 (0.0010) [2023-12-26 18:59:50,269][105620] Updated weights for policy 1, policy_version 499314 (0.0007) [2023-12-26 18:59:50,322][105620] Updated weights for policy 1, policy_version 499324 (0.0006) [2023-12-26 18:59:50,387][105620] Updated weights for policy 1, policy_version 499334 (0.0008) [2023-12-26 18:59:50,476][105692] Updated weights for policy 0, policy_version 498854 (0.0010) [2023-12-26 18:59:50,528][105692] Updated weights for policy 0, policy_version 498864 (0.0010) [2023-12-26 18:59:50,595][105692] Updated weights for policy 0, policy_version 498874 (0.0008) [2023-12-26 18:59:51,027][105620] Updated weights for policy 1, policy_version 499344 (0.0008) [2023-12-26 18:59:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 255574016. Throughput: 0: 9524.4, 1: 9876.0. Samples: 255566388. Policy #0 lag: (min: 9.0, avg: 22.2, max: 41.0) [2023-12-26 18:59:51,063][104569] Avg episode reward: [(0, '3819.866'), (1, '9350.586')] [2023-12-26 18:59:51,094][105620] Updated weights for policy 1, policy_version 499354 (0.0008) [2023-12-26 18:59:51,157][105620] Updated weights for policy 1, policy_version 499364 (0.0008) [2023-12-26 18:59:51,350][105692] Updated weights for policy 0, policy_version 498884 (0.0010) [2023-12-26 18:59:51,411][105692] Updated weights for policy 0, policy_version 498894 (0.0011) [2023-12-26 18:59:51,467][105692] Updated weights for policy 0, policy_version 498904 (0.0009) [2023-12-26 18:59:51,926][105620] Updated weights for policy 1, policy_version 499374 (0.0009) [2023-12-26 18:59:51,975][105620] Updated weights for policy 1, policy_version 499384 (0.0009) [2023-12-26 18:59:52,027][105620] Updated weights for policy 1, policy_version 499394 (0.0009) [2023-12-26 18:59:52,182][105692] Updated weights for policy 0, policy_version 498914 (0.0008) [2023-12-26 18:59:52,242][105692] Updated weights for policy 0, policy_version 498924 (0.0007) [2023-12-26 18:59:52,307][105692] Updated weights for policy 0, policy_version 498934 (0.0006) [2023-12-26 18:59:52,366][105692] Updated weights for policy 0, policy_version 498944 (0.0007) [2023-12-26 18:59:52,758][105620] Updated weights for policy 1, policy_version 499404 (0.0009) [2023-12-26 18:59:52,824][105620] Updated weights for policy 1, policy_version 499414 (0.0008) [2023-12-26 18:59:52,881][105620] Updated weights for policy 1, policy_version 499424 (0.0006) [2023-12-26 18:59:53,076][105692] Updated weights for policy 0, policy_version 498954 (0.0009) [2023-12-26 18:59:53,130][105692] Updated weights for policy 0, policy_version 498964 (0.0009) [2023-12-26 18:59:53,187][105692] Updated weights for policy 0, policy_version 498974 (0.0009) [2023-12-26 18:59:53,569][105620] Updated weights for policy 1, policy_version 499434 (0.0005) [2023-12-26 18:59:53,637][105620] Updated weights for policy 1, policy_version 499444 (0.0006) [2023-12-26 18:59:53,687][105620] Updated weights for policy 1, policy_version 499454 (0.0009) [2023-12-26 18:59:53,742][105620] Updated weights for policy 1, policy_version 499464 (0.0009) [2023-12-26 18:59:53,985][105692] Updated weights for policy 0, policy_version 498984 (0.0010) [2023-12-26 18:59:54,047][105692] Updated weights for policy 0, policy_version 498994 (0.0009) [2023-12-26 18:59:54,119][105692] Updated weights for policy 0, policy_version 499004 (0.0010) [2023-12-26 18:59:54,355][105620] Updated weights for policy 1, policy_version 499474 (0.0009) [2023-12-26 18:59:54,418][105620] Updated weights for policy 1, policy_version 499484 (0.0009) [2023-12-26 18:59:54,472][105620] Updated weights for policy 1, policy_version 499494 (0.0009) [2023-12-26 18:59:54,927][105692] Updated weights for policy 0, policy_version 499014 (0.0010) [2023-12-26 18:59:54,977][105692] Updated weights for policy 0, policy_version 499024 (0.0008) [2023-12-26 18:59:55,031][105692] Updated weights for policy 0, policy_version 499034 (0.0009) [2023-12-26 18:59:55,119][105620] Updated weights for policy 1, policy_version 499504 (0.0009) [2023-12-26 18:59:55,172][105620] Updated weights for policy 1, policy_version 499514 (0.0009) [2023-12-26 18:59:55,224][105620] Updated weights for policy 1, policy_version 499524 (0.0009) [2023-12-26 18:59:55,803][105692] Updated weights for policy 0, policy_version 499044 (0.0008) [2023-12-26 18:59:55,853][105692] Updated weights for policy 0, policy_version 499054 (0.0006) [2023-12-26 18:59:55,910][105692] Updated weights for policy 0, policy_version 499064 (0.0005) [2023-12-26 18:59:55,981][105620] Updated weights for policy 1, policy_version 499534 (0.0008) [2023-12-26 18:59:56,031][105620] Updated weights for policy 1, policy_version 499544 (0.0008) [2023-12-26 18:59:56,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 255672320. Throughput: 0: 9513.2, 1: 9888.4. Samples: 255680848. Policy #0 lag: (min: 9.0, avg: 22.2, max: 41.0) [2023-12-26 18:59:56,062][104569] Avg episode reward: [(0, '5712.552'), (1, '9350.936')] [2023-12-26 18:59:56,092][105620] Updated weights for policy 1, policy_version 499554 (0.0009) [2023-12-26 18:59:56,549][105692] Updated weights for policy 0, policy_version 499074 (0.0007) [2023-12-26 18:59:56,605][105692] Updated weights for policy 0, policy_version 499084 (0.0010) [2023-12-26 18:59:56,652][105692] Updated weights for policy 0, policy_version 499094 (0.0010) [2023-12-26 18:59:56,703][105692] Updated weights for policy 0, policy_version 499104 (0.0010) [2023-12-26 18:59:56,767][105620] Updated weights for policy 1, policy_version 499564 (0.0007) [2023-12-26 18:59:56,822][105620] Updated weights for policy 1, policy_version 499574 (0.0005) [2023-12-26 18:59:56,889][105620] Updated weights for policy 1, policy_version 499584 (0.0006) [2023-12-26 18:59:57,346][105692] Updated weights for policy 0, policy_version 499114 (0.0010) [2023-12-26 18:59:57,390][105692] Updated weights for policy 0, policy_version 499124 (0.0010) [2023-12-26 18:59:57,434][105692] Updated weights for policy 0, policy_version 499134 (0.0010) [2023-12-26 18:59:57,491][105620] Updated weights for policy 1, policy_version 499594 (0.0006) [2023-12-26 18:59:57,546][105620] Updated weights for policy 1, policy_version 499604 (0.0010) [2023-12-26 18:59:57,594][105620] Updated weights for policy 1, policy_version 499614 (0.0010) [2023-12-26 18:59:57,646][105620] Updated weights for policy 1, policy_version 499624 (0.0006) [2023-12-26 18:59:58,204][105692] Updated weights for policy 0, policy_version 499144 (0.0008) [2023-12-26 18:59:58,269][105692] Updated weights for policy 0, policy_version 499154 (0.0008) [2023-12-26 18:59:58,287][105620] Updated weights for policy 1, policy_version 499634 (0.0007) [2023-12-26 18:59:58,341][105692] Updated weights for policy 0, policy_version 499164 (0.0008) [2023-12-26 18:59:58,360][105620] Updated weights for policy 1, policy_version 499645 (0.0010) [2023-12-26 18:59:58,425][105620] Updated weights for policy 1, policy_version 499655 (0.0009) [2023-12-26 18:59:59,259][105692] Updated weights for policy 0, policy_version 499174 (0.0009) [2023-12-26 18:59:59,274][105620] Updated weights for policy 1, policy_version 499665 (0.0008) [2023-12-26 18:59:59,320][105692] Updated weights for policy 0, policy_version 499184 (0.0005) [2023-12-26 18:59:59,338][105620] Updated weights for policy 1, policy_version 499675 (0.0011) [2023-12-26 18:59:59,384][105692] Updated weights for policy 0, policy_version 499194 (0.0008) [2023-12-26 18:59:59,410][105620] Updated weights for policy 1, policy_version 499685 (0.0010) [2023-12-26 19:00:00,081][105620] Updated weights for policy 1, policy_version 499695 (0.0005) [2023-12-26 19:00:00,101][105692] Updated weights for policy 0, policy_version 499204 (0.0009) [2023-12-26 19:00:00,143][105620] Updated weights for policy 1, policy_version 499705 (0.0007) [2023-12-26 19:00:00,164][105692] Updated weights for policy 0, policy_version 499214 (0.0009) [2023-12-26 19:00:00,195][105620] Updated weights for policy 1, policy_version 499715 (0.0005) [2023-12-26 19:00:00,223][105692] Updated weights for policy 0, policy_version 499224 (0.0008) [2023-12-26 19:00:00,836][105620] Updated weights for policy 1, policy_version 499725 (0.0007) [2023-12-26 19:00:00,884][105620] Updated weights for policy 1, policy_version 499735 (0.0005) [2023-12-26 19:00:00,937][105620] Updated weights for policy 1, policy_version 499745 (0.0006) [2023-12-26 19:00:00,971][105692] Updated weights for policy 0, policy_version 499234 (0.0009) [2023-12-26 19:00:01,022][105692] Updated weights for policy 0, policy_version 499244 (0.0009) [2023-12-26 19:00:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 255770624. Throughput: 0: 9562.0, 1: 9943.9. Samples: 255741244. Policy #0 lag: (min: 9.0, avg: 22.2, max: 41.0) [2023-12-26 19:00:01,062][104569] Avg episode reward: [(0, '6179.756'), (1, '9264.484')] [2023-12-26 19:00:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000499752_127950848.pth... [2023-12-26 19:00:01,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000498568_127647744.pth [2023-12-26 19:00:01,089][105692] Updated weights for policy 0, policy_version 499254 (0.0008) [2023-12-26 19:00:01,157][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000499264_127827968.pth... [2023-12-26 19:00:01,159][105692] Updated weights for policy 0, policy_version 499264 (0.0008) [2023-12-26 19:00:01,160][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000498144_127541248.pth [2023-12-26 19:00:01,664][105620] Updated weights for policy 1, policy_version 499755 (0.0009) [2023-12-26 19:00:01,722][105620] Updated weights for policy 1, policy_version 499765 (0.0008) [2023-12-26 19:00:01,783][105620] Updated weights for policy 1, policy_version 499775 (0.0008) [2023-12-26 19:00:01,925][105692] Updated weights for policy 0, policy_version 499274 (0.0007) [2023-12-26 19:00:01,984][105692] Updated weights for policy 0, policy_version 499284 (0.0005) [2023-12-26 19:00:02,038][105692] Updated weights for policy 0, policy_version 499294 (0.0005) [2023-12-26 19:00:02,491][105620] Updated weights for policy 1, policy_version 499785 (0.0007) [2023-12-26 19:00:02,552][105620] Updated weights for policy 1, policy_version 499795 (0.0009) [2023-12-26 19:00:02,606][105620] Updated weights for policy 1, policy_version 499805 (0.0009) [2023-12-26 19:00:02,636][105692] Updated weights for policy 0, policy_version 499304 (0.0006) [2023-12-26 19:00:02,663][105620] Updated weights for policy 1, policy_version 499815 (0.0007) [2023-12-26 19:00:02,693][105692] Updated weights for policy 0, policy_version 499314 (0.0007) [2023-12-26 19:00:02,741][105692] Updated weights for policy 0, policy_version 499324 (0.0008) [2023-12-26 19:00:03,313][105692] Updated weights for policy 0, policy_version 499334 (0.0007) [2023-12-26 19:00:03,361][105692] Updated weights for policy 0, policy_version 499344 (0.0005) [2023-12-26 19:00:03,363][105620] Updated weights for policy 1, policy_version 499825 (0.0009) [2023-12-26 19:00:03,411][105692] Updated weights for policy 0, policy_version 499354 (0.0006) [2023-12-26 19:00:03,415][105620] Updated weights for policy 1, policy_version 499835 (0.0008) [2023-12-26 19:00:03,468][105620] Updated weights for policy 1, policy_version 499845 (0.0008) [2023-12-26 19:00:03,981][105692] Updated weights for policy 0, policy_version 499364 (0.0008) [2023-12-26 19:00:04,028][105692] Updated weights for policy 0, policy_version 499374 (0.0009) [2023-12-26 19:00:04,078][105692] Updated weights for policy 0, policy_version 499384 (0.0009) [2023-12-26 19:00:04,121][105585] KL-divergence is very high: 115.2279 [2023-12-26 19:00:04,301][105620] Updated weights for policy 1, policy_version 499855 (0.0008) [2023-12-26 19:00:04,362][105620] Updated weights for policy 1, policy_version 499865 (0.0007) [2023-12-26 19:00:04,424][105620] Updated weights for policy 1, policy_version 499875 (0.0007) [2023-12-26 19:00:04,790][105692] Updated weights for policy 0, policy_version 499394 (0.0009) [2023-12-26 19:00:04,843][105692] Updated weights for policy 0, policy_version 499404 (0.0010) [2023-12-26 19:00:04,896][105692] Updated weights for policy 0, policy_version 499414 (0.0010) [2023-12-26 19:00:04,949][105692] Updated weights for policy 0, policy_version 499424 (0.0010) [2023-12-26 19:00:05,076][105620] Updated weights for policy 1, policy_version 499885 (0.0007) [2023-12-26 19:00:05,132][105620] Updated weights for policy 1, policy_version 499895 (0.0005) [2023-12-26 19:00:05,189][105620] Updated weights for policy 1, policy_version 499905 (0.0006) [2023-12-26 19:00:05,711][105620] Updated weights for policy 1, policy_version 499915 (0.0005) [2023-12-26 19:00:05,757][105620] Updated weights for policy 1, policy_version 499925 (0.0005) [2023-12-26 19:00:05,808][105620] Updated weights for policy 1, policy_version 499935 (0.0008) [2023-12-26 19:00:05,859][105692] Updated weights for policy 0, policy_version 499434 (0.0007) [2023-12-26 19:00:05,924][105692] Updated weights for policy 0, policy_version 499444 (0.0010) [2023-12-26 19:00:05,986][105692] Updated weights for policy 0, policy_version 499454 (0.0009) [2023-12-26 19:00:06,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 255877120. Throughput: 0: 9558.0, 1: 9974.6. Samples: 255860380. Policy #0 lag: (min: 9.0, avg: 22.2, max: 41.0) [2023-12-26 19:00:06,063][104569] Avg episode reward: [(0, '6638.810'), (1, '9265.050')] [2023-12-26 19:00:06,430][105620] Updated weights for policy 1, policy_version 499945 (0.0008) [2023-12-26 19:00:06,492][105620] Updated weights for policy 1, policy_version 499955 (0.0010) [2023-12-26 19:00:06,549][105620] Updated weights for policy 1, policy_version 499965 (0.0010) [2023-12-26 19:00:06,602][105620] Updated weights for policy 1, policy_version 499975 (0.0009) [2023-12-26 19:00:06,742][105692] Updated weights for policy 0, policy_version 499464 (0.0009) [2023-12-26 19:00:06,816][105692] Updated weights for policy 0, policy_version 499474 (0.0010) [2023-12-26 19:00:06,875][105692] Updated weights for policy 0, policy_version 499484 (0.0009) [2023-12-26 19:00:07,302][105620] Updated weights for policy 1, policy_version 499985 (0.0006) [2023-12-26 19:00:07,361][105620] Updated weights for policy 1, policy_version 499995 (0.0008) [2023-12-26 19:00:07,408][105620] Updated weights for policy 1, policy_version 500005 (0.0009) [2023-12-26 19:00:07,693][105692] Updated weights for policy 0, policy_version 499494 (0.0009) [2023-12-26 19:00:07,744][105692] Updated weights for policy 0, policy_version 499504 (0.0009) [2023-12-26 19:00:07,805][105692] Updated weights for policy 0, policy_version 499514 (0.0009) [2023-12-26 19:00:08,058][105620] Updated weights for policy 1, policy_version 500015 (0.0009) [2023-12-26 19:00:08,108][105620] Updated weights for policy 1, policy_version 500025 (0.0008) [2023-12-26 19:00:08,157][105620] Updated weights for policy 1, policy_version 500035 (0.0008) [2023-12-26 19:00:08,576][105692] Updated weights for policy 0, policy_version 499524 (0.0008) [2023-12-26 19:00:08,641][105692] Updated weights for policy 0, policy_version 499534 (0.0007) [2023-12-26 19:00:08,707][105692] Updated weights for policy 0, policy_version 499544 (0.0010) [2023-12-26 19:00:08,950][105620] Updated weights for policy 1, policy_version 500045 (0.0009) [2023-12-26 19:00:09,008][105620] Updated weights for policy 1, policy_version 500055 (0.0009) [2023-12-26 19:00:09,065][105620] Updated weights for policy 1, policy_version 500065 (0.0008) [2023-12-26 19:00:09,421][105692] Updated weights for policy 0, policy_version 499554 (0.0010) [2023-12-26 19:00:09,484][105692] Updated weights for policy 0, policy_version 499564 (0.0009) [2023-12-26 19:00:09,550][105692] Updated weights for policy 0, policy_version 499574 (0.0006) [2023-12-26 19:00:09,617][105692] Updated weights for policy 0, policy_version 499584 (0.0008) [2023-12-26 19:00:09,874][105620] Updated weights for policy 1, policy_version 500075 (0.0009) [2023-12-26 19:00:09,937][105620] Updated weights for policy 1, policy_version 500085 (0.0009) [2023-12-26 19:00:10,001][105620] Updated weights for policy 1, policy_version 500095 (0.0008) [2023-12-26 19:00:10,250][105692] Updated weights for policy 0, policy_version 499594 (0.0005) [2023-12-26 19:00:10,308][105692] Updated weights for policy 0, policy_version 499604 (0.0005) [2023-12-26 19:00:10,374][105692] Updated weights for policy 0, policy_version 499614 (0.0006) [2023-12-26 19:00:10,751][105620] Updated weights for policy 1, policy_version 500105 (0.0010) [2023-12-26 19:00:10,814][105620] Updated weights for policy 1, policy_version 500115 (0.0009) [2023-12-26 19:00:10,893][105620] Updated weights for policy 1, policy_version 500125 (0.0008) [2023-12-26 19:00:10,959][105620] Updated weights for policy 1, policy_version 500135 (0.0007) [2023-12-26 19:00:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 255967232. Throughput: 0: 9544.9, 1: 10026.9. Samples: 255976092. Policy #0 lag: (min: 9.0, avg: 22.2, max: 41.0) [2023-12-26 19:00:11,063][104569] Avg episode reward: [(0, '7648.114'), (1, '9169.501')] [2023-12-26 19:00:11,069][105692] Updated weights for policy 0, policy_version 499624 (0.0009) [2023-12-26 19:00:11,130][105692] Updated weights for policy 0, policy_version 499634 (0.0010) [2023-12-26 19:00:11,193][105692] Updated weights for policy 0, policy_version 499644 (0.0009) [2023-12-26 19:00:11,673][105620] Updated weights for policy 1, policy_version 500145 (0.0009) [2023-12-26 19:00:11,747][105620] Updated weights for policy 1, policy_version 500155 (0.0010) [2023-12-26 19:00:11,808][105620] Updated weights for policy 1, policy_version 500165 (0.0006) [2023-12-26 19:00:11,908][105692] Updated weights for policy 0, policy_version 499654 (0.0007) [2023-12-26 19:00:11,977][105692] Updated weights for policy 0, policy_version 499664 (0.0008) [2023-12-26 19:00:12,033][105692] Updated weights for policy 0, policy_version 499674 (0.0009) [2023-12-26 19:00:12,497][105620] Updated weights for policy 1, policy_version 500175 (0.0005) [2023-12-26 19:00:12,559][105620] Updated weights for policy 1, policy_version 500185 (0.0006) [2023-12-26 19:00:12,626][105620] Updated weights for policy 1, policy_version 500195 (0.0009) [2023-12-26 19:00:12,678][105692] Updated weights for policy 0, policy_version 499684 (0.0008) [2023-12-26 19:00:12,733][105692] Updated weights for policy 0, policy_version 499694 (0.0011) [2023-12-26 19:00:12,795][105692] Updated weights for policy 0, policy_version 499704 (0.0010) [2023-12-26 19:00:13,310][105620] Updated weights for policy 1, policy_version 500205 (0.0011) [2023-12-26 19:00:13,368][105620] Updated weights for policy 1, policy_version 500215 (0.0010) [2023-12-26 19:00:13,430][105620] Updated weights for policy 1, policy_version 500225 (0.0007) [2023-12-26 19:00:13,534][105692] Updated weights for policy 0, policy_version 499714 (0.0010) [2023-12-26 19:00:13,583][105692] Updated weights for policy 0, policy_version 499724 (0.0010) [2023-12-26 19:00:13,631][105692] Updated weights for policy 0, policy_version 499734 (0.0010) [2023-12-26 19:00:13,679][105692] Updated weights for policy 0, policy_version 499744 (0.0010) [2023-12-26 19:00:13,963][105620] Updated weights for policy 1, policy_version 500235 (0.0005) [2023-12-26 19:00:14,020][105620] Updated weights for policy 1, policy_version 500245 (0.0005) [2023-12-26 19:00:14,073][105620] Updated weights for policy 1, policy_version 500255 (0.0006) [2023-12-26 19:00:14,468][105692] Updated weights for policy 0, policy_version 499754 (0.0011) [2023-12-26 19:00:14,517][105692] Updated weights for policy 0, policy_version 499764 (0.0010) [2023-12-26 19:00:14,572][105692] Updated weights for policy 0, policy_version 499774 (0.0010) [2023-12-26 19:00:14,638][105620] Updated weights for policy 1, policy_version 500265 (0.0006) [2023-12-26 19:00:14,686][105620] Updated weights for policy 1, policy_version 500275 (0.0010) [2023-12-26 19:00:14,730][105620] Updated weights for policy 1, policy_version 500285 (0.0010) [2023-12-26 19:00:14,787][105620] Updated weights for policy 1, policy_version 500295 (0.0009) [2023-12-26 19:00:15,372][105692] Updated weights for policy 0, policy_version 499784 (0.0010) [2023-12-26 19:00:15,397][105620] Updated weights for policy 1, policy_version 500305 (0.0006) [2023-12-26 19:00:15,432][105692] Updated weights for policy 0, policy_version 499794 (0.0008) [2023-12-26 19:00:15,447][105620] Updated weights for policy 1, policy_version 500315 (0.0005) [2023-12-26 19:00:15,487][105692] Updated weights for policy 0, policy_version 499804 (0.0008) [2023-12-26 19:00:15,497][105620] Updated weights for policy 1, policy_version 500325 (0.0005) [2023-12-26 19:00:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 256065536. Throughput: 0: 9564.6, 1: 10059.7. Samples: 256036196. Policy #0 lag: (min: 9.0, avg: 22.2, max: 41.0) [2023-12-26 19:00:16,063][104569] Avg episode reward: [(0, '8295.835'), (1, '9169.892')] [2023-12-26 19:00:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000500328_128098304.pth... [2023-12-26 19:00:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000499144_127795200.pth [2023-12-26 19:00:16,106][105692] Updated weights for policy 0, policy_version 499814 (0.0006) [2023-12-26 19:00:16,148][105620] Updated weights for policy 1, policy_version 500335 (0.0008) [2023-12-26 19:00:16,172][105692] Updated weights for policy 0, policy_version 499824 (0.0008) [2023-12-26 19:00:16,206][105620] Updated weights for policy 1, policy_version 500345 (0.0009) [2023-12-26 19:00:16,235][105692] Updated weights for policy 0, policy_version 499834 (0.0005) [2023-12-26 19:00:16,259][105620] Updated weights for policy 1, policy_version 500355 (0.0009) [2023-12-26 19:00:16,273][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000499840_127975424.pth... [2023-12-26 19:00:16,277][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000498688_127680512.pth [2023-12-26 19:00:16,912][105692] Updated weights for policy 0, policy_version 499844 (0.0006) [2023-12-26 19:00:16,967][105692] Updated weights for policy 0, policy_version 499854 (0.0005) [2023-12-26 19:00:17,004][105620] Updated weights for policy 1, policy_version 500365 (0.0009) [2023-12-26 19:00:17,026][105692] Updated weights for policy 0, policy_version 499864 (0.0007) [2023-12-26 19:00:17,059][105620] Updated weights for policy 1, policy_version 500375 (0.0007) [2023-12-26 19:00:17,112][105620] Updated weights for policy 1, policy_version 500385 (0.0005) [2023-12-26 19:00:17,688][105692] Updated weights for policy 0, policy_version 499874 (0.0007) [2023-12-26 19:00:17,746][105692] Updated weights for policy 0, policy_version 499884 (0.0006) [2023-12-26 19:00:17,815][105692] Updated weights for policy 0, policy_version 499894 (0.0005) [2023-12-26 19:00:17,846][105620] Updated weights for policy 1, policy_version 500395 (0.0007) [2023-12-26 19:00:17,877][105692] Updated weights for policy 0, policy_version 499904 (0.0005) [2023-12-26 19:00:17,900][105620] Updated weights for policy 1, policy_version 500405 (0.0009) [2023-12-26 19:00:17,973][105620] Updated weights for policy 1, policy_version 500415 (0.0010) [2023-12-26 19:00:18,485][105692] Updated weights for policy 0, policy_version 499914 (0.0009) [2023-12-26 19:00:18,548][105692] Updated weights for policy 0, policy_version 499924 (0.0009) [2023-12-26 19:00:18,602][105692] Updated weights for policy 0, policy_version 499934 (0.0008) [2023-12-26 19:00:18,613][105620] Updated weights for policy 1, policy_version 500425 (0.0009) [2023-12-26 19:00:18,673][105620] Updated weights for policy 1, policy_version 500435 (0.0006) [2023-12-26 19:00:18,738][105620] Updated weights for policy 1, policy_version 500445 (0.0006) [2023-12-26 19:00:18,799][105620] Updated weights for policy 1, policy_version 500455 (0.0005) [2023-12-26 19:00:19,378][105620] Updated weights for policy 1, policy_version 500465 (0.0008) [2023-12-26 19:00:19,426][105692] Updated weights for policy 0, policy_version 499944 (0.0010) [2023-12-26 19:00:19,436][105620] Updated weights for policy 1, policy_version 500475 (0.0006) [2023-12-26 19:00:19,489][105692] Updated weights for policy 0, policy_version 499954 (0.0011) [2023-12-26 19:00:19,501][105620] Updated weights for policy 1, policy_version 500485 (0.0007) [2023-12-26 19:00:19,552][105692] Updated weights for policy 0, policy_version 499964 (0.0010) [2023-12-26 19:00:20,309][105620] Updated weights for policy 1, policy_version 500495 (0.0008) [2023-12-26 19:00:20,333][105692] Updated weights for policy 0, policy_version 499974 (0.0009) [2023-12-26 19:00:20,376][105620] Updated weights for policy 1, policy_version 500505 (0.0008) [2023-12-26 19:00:20,399][105692] Updated weights for policy 0, policy_version 499984 (0.0010) [2023-12-26 19:00:20,433][105620] Updated weights for policy 1, policy_version 500515 (0.0009) [2023-12-26 19:00:20,463][105692] Updated weights for policy 0, policy_version 499994 (0.0009) [2023-12-26 19:00:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 256163840. Throughput: 0: 9648.4, 1: 10049.2. Samples: 256157424. Policy #0 lag: (min: 9.0, avg: 22.2, max: 41.0) [2023-12-26 19:00:21,062][104569] Avg episode reward: [(0, '9004.470'), (1, '9261.271')] [2023-12-26 19:00:21,222][105692] Updated weights for policy 0, policy_version 500004 (0.0011) [2023-12-26 19:00:21,230][105620] Updated weights for policy 1, policy_version 500525 (0.0008) [2023-12-26 19:00:21,288][105692] Updated weights for policy 0, policy_version 500014 (0.0008) [2023-12-26 19:00:21,290][105620] Updated weights for policy 1, policy_version 500535 (0.0007) [2023-12-26 19:00:21,349][105692] Updated weights for policy 0, policy_version 500024 (0.0008) [2023-12-26 19:00:21,351][105620] Updated weights for policy 1, policy_version 500545 (0.0007) [2023-12-26 19:00:22,045][105620] Updated weights for policy 1, policy_version 500555 (0.0008) [2023-12-26 19:00:22,107][105620] Updated weights for policy 1, policy_version 500565 (0.0009) [2023-12-26 19:00:22,143][105692] Updated weights for policy 0, policy_version 500034 (0.0008) [2023-12-26 19:00:22,169][105620] Updated weights for policy 1, policy_version 500575 (0.0007) [2023-12-26 19:00:22,205][105692] Updated weights for policy 0, policy_version 500044 (0.0007) [2023-12-26 19:00:22,271][105692] Updated weights for policy 0, policy_version 500054 (0.0009) [2023-12-26 19:00:22,327][105692] Updated weights for policy 0, policy_version 500064 (0.0009) [2023-12-26 19:00:22,883][105620] Updated weights for policy 1, policy_version 500585 (0.0008) [2023-12-26 19:00:22,947][105620] Updated weights for policy 1, policy_version 500595 (0.0009) [2023-12-26 19:00:22,997][105620] Updated weights for policy 1, policy_version 500605 (0.0008) [2023-12-26 19:00:23,049][105620] Updated weights for policy 1, policy_version 500615 (0.0009) [2023-12-26 19:00:23,106][105692] Updated weights for policy 0, policy_version 500074 (0.0009) [2023-12-26 19:00:23,168][105692] Updated weights for policy 0, policy_version 500084 (0.0009) [2023-12-26 19:00:23,232][105692] Updated weights for policy 0, policy_version 500094 (0.0009) [2023-12-26 19:00:23,825][105620] Updated weights for policy 1, policy_version 500625 (0.0009) [2023-12-26 19:00:23,876][105620] Updated weights for policy 1, policy_version 500635 (0.0009) [2023-12-26 19:00:23,918][105692] Updated weights for policy 0, policy_version 500104 (0.0008) [2023-12-26 19:00:23,936][105620] Updated weights for policy 1, policy_version 500645 (0.0009) [2023-12-26 19:00:23,975][105692] Updated weights for policy 0, policy_version 500114 (0.0008) [2023-12-26 19:00:24,037][105692] Updated weights for policy 0, policy_version 500124 (0.0009) [2023-12-26 19:00:24,683][105692] Updated weights for policy 0, policy_version 500134 (0.0009) [2023-12-26 19:00:24,727][105692] Updated weights for policy 0, policy_version 500144 (0.0007) [2023-12-26 19:00:24,747][105620] Updated weights for policy 1, policy_version 500655 (0.0008) [2023-12-26 19:00:24,777][105692] Updated weights for policy 0, policy_version 500154 (0.0005) [2023-12-26 19:00:24,804][105620] Updated weights for policy 1, policy_version 500665 (0.0007) [2023-12-26 19:00:24,860][105620] Updated weights for policy 1, policy_version 500675 (0.0008) [2023-12-26 19:00:25,398][105692] Updated weights for policy 0, policy_version 500164 (0.0007) [2023-12-26 19:00:25,459][105692] Updated weights for policy 0, policy_version 500174 (0.0008) [2023-12-26 19:00:25,522][105692] Updated weights for policy 0, policy_version 500184 (0.0007) [2023-12-26 19:00:25,698][105620] Updated weights for policy 1, policy_version 500685 (0.0009) [2023-12-26 19:00:25,766][105620] Updated weights for policy 1, policy_version 500695 (0.0009) [2023-12-26 19:00:25,829][105620] Updated weights for policy 1, policy_version 500705 (0.0010) [2023-12-26 19:00:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 256262144. Throughput: 0: 9624.8, 1: 9986.9. Samples: 256269004. Policy #0 lag: (min: 9.0, avg: 22.2, max: 41.0) [2023-12-26 19:00:26,063][104569] Avg episode reward: [(0, '9090.850'), (1, '9353.860')] [2023-12-26 19:00:26,115][105692] Updated weights for policy 0, policy_version 500194 (0.0007) [2023-12-26 19:00:26,165][105692] Updated weights for policy 0, policy_version 500204 (0.0007) [2023-12-26 19:00:26,220][105692] Updated weights for policy 0, policy_version 500214 (0.0010) [2023-12-26 19:00:26,285][105692] Updated weights for policy 0, policy_version 500224 (0.0008) [2023-12-26 19:00:26,676][105620] Updated weights for policy 1, policy_version 500715 (0.0010) [2023-12-26 19:00:26,742][105620] Updated weights for policy 1, policy_version 500725 (0.0008) [2023-12-26 19:00:26,791][105620] Updated weights for policy 1, policy_version 500735 (0.0008) [2023-12-26 19:00:26,909][105692] Updated weights for policy 0, policy_version 500234 (0.0010) [2023-12-26 19:00:26,966][105692] Updated weights for policy 0, policy_version 500244 (0.0009) [2023-12-26 19:00:27,025][105692] Updated weights for policy 0, policy_version 500254 (0.0005) [2023-12-26 19:00:27,599][105620] Updated weights for policy 1, policy_version 500745 (0.0008) [2023-12-26 19:00:27,649][105620] Updated weights for policy 1, policy_version 500755 (0.0009) [2023-12-26 19:00:27,649][105692] Updated weights for policy 0, policy_version 500264 (0.0005) [2023-12-26 19:00:27,697][105620] Updated weights for policy 1, policy_version 500766 (0.0009) [2023-12-26 19:00:27,704][105692] Updated weights for policy 0, policy_version 500274 (0.0005) [2023-12-26 19:00:27,744][105620] Updated weights for policy 1, policy_version 500776 (0.0008) [2023-12-26 19:00:27,761][105692] Updated weights for policy 0, policy_version 500284 (0.0006) [2023-12-26 19:00:28,288][105692] Updated weights for policy 0, policy_version 500294 (0.0009) [2023-12-26 19:00:28,339][105692] Updated weights for policy 0, policy_version 500304 (0.0010) [2023-12-26 19:00:28,401][105692] Updated weights for policy 0, policy_version 500314 (0.0010) [2023-12-26 19:00:28,632][105620] Updated weights for policy 1, policy_version 500786 (0.0008) [2023-12-26 19:00:28,695][105620] Updated weights for policy 1, policy_version 500796 (0.0008) [2023-12-26 19:00:28,761][105620] Updated weights for policy 1, policy_version 500806 (0.0008) [2023-12-26 19:00:29,159][105692] Updated weights for policy 0, policy_version 500324 (0.0010) [2023-12-26 19:00:29,214][105692] Updated weights for policy 0, policy_version 500334 (0.0006) [2023-12-26 19:00:29,282][105692] Updated weights for policy 0, policy_version 500345 (0.0008) [2023-12-26 19:00:29,517][105620] Updated weights for policy 1, policy_version 500816 (0.0008) [2023-12-26 19:00:29,572][105620] Updated weights for policy 1, policy_version 500826 (0.0008) [2023-12-26 19:00:29,628][105620] Updated weights for policy 1, policy_version 500836 (0.0008) [2023-12-26 19:00:29,879][105692] Updated weights for policy 0, policy_version 500355 (0.0009) [2023-12-26 19:00:29,953][105692] Updated weights for policy 0, policy_version 500365 (0.0010) [2023-12-26 19:00:30,019][105692] Updated weights for policy 0, policy_version 500375 (0.0011) [2023-12-26 19:00:30,451][105620] Updated weights for policy 1, policy_version 500846 (0.0008) [2023-12-26 19:00:30,495][105620] Updated weights for policy 1, policy_version 500856 (0.0008) [2023-12-26 19:00:30,539][105620] Updated weights for policy 1, policy_version 500866 (0.0008) [2023-12-26 19:00:30,696][105692] Updated weights for policy 0, policy_version 500385 (0.0010) [2023-12-26 19:00:30,744][105692] Updated weights for policy 0, policy_version 500395 (0.0007) [2023-12-26 19:00:30,791][105692] Updated weights for policy 0, policy_version 500405 (0.0009) [2023-12-26 19:00:30,845][105692] Updated weights for policy 0, policy_version 500415 (0.0010) [2023-12-26 19:00:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 256360448. Throughput: 0: 9769.8, 1: 9851.3. Samples: 256328472. Policy #0 lag: (min: 9.0, avg: 22.2, max: 41.0) [2023-12-26 19:00:31,062][104569] Avg episode reward: [(0, '8275.492'), (1, '9353.437')] [2023-12-26 19:00:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000500416_128122880.pth... [2023-12-26 19:00:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000500872_128237568.pth... [2023-12-26 19:00:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000499264_127827968.pth [2023-12-26 19:00:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000499752_127950848.pth [2023-12-26 19:00:31,333][105620] Updated weights for policy 1, policy_version 500876 (0.0008) [2023-12-26 19:00:31,392][105620] Updated weights for policy 1, policy_version 500886 (0.0008) [2023-12-26 19:00:31,454][105620] Updated weights for policy 1, policy_version 500896 (0.0009) [2023-12-26 19:00:31,585][105692] Updated weights for policy 0, policy_version 500425 (0.0009) [2023-12-26 19:00:31,639][105692] Updated weights for policy 0, policy_version 500435 (0.0007) [2023-12-26 19:00:31,694][105692] Updated weights for policy 0, policy_version 500445 (0.0005) [2023-12-26 19:00:32,207][105620] Updated weights for policy 1, policy_version 500906 (0.0009) [2023-12-26 19:00:32,268][105620] Updated weights for policy 1, policy_version 500916 (0.0008) [2023-12-26 19:00:32,329][105620] Updated weights for policy 1, policy_version 500926 (0.0006) [2023-12-26 19:00:32,396][105620] Updated weights for policy 1, policy_version 500936 (0.0008) [2023-12-26 19:00:32,404][105692] Updated weights for policy 0, policy_version 500455 (0.0008) [2023-12-26 19:00:32,458][105692] Updated weights for policy 0, policy_version 500465 (0.0006) [2023-12-26 19:00:32,515][105692] Updated weights for policy 0, policy_version 500475 (0.0005) [2023-12-26 19:00:33,089][105692] Updated weights for policy 0, policy_version 500485 (0.0005) [2023-12-26 19:00:33,151][105692] Updated weights for policy 0, policy_version 500495 (0.0006) [2023-12-26 19:00:33,178][105620] Updated weights for policy 1, policy_version 500946 (0.0006) [2023-12-26 19:00:33,214][105692] Updated weights for policy 0, policy_version 500506 (0.0009) [2023-12-26 19:00:33,235][105620] Updated weights for policy 1, policy_version 500956 (0.0008) [2023-12-26 19:00:33,290][105620] Updated weights for policy 1, policy_version 500966 (0.0008) [2023-12-26 19:00:33,801][105692] Updated weights for policy 0, policy_version 500516 (0.0006) [2023-12-26 19:00:33,847][105692] Updated weights for policy 0, policy_version 500526 (0.0005) [2023-12-26 19:00:33,897][105692] Updated weights for policy 0, policy_version 500536 (0.0005) [2023-12-26 19:00:34,100][105620] Updated weights for policy 1, policy_version 500976 (0.0007) [2023-12-26 19:00:34,169][105620] Updated weights for policy 1, policy_version 500986 (0.0009) [2023-12-26 19:00:34,229][105620] Updated weights for policy 1, policy_version 500996 (0.0007) [2023-12-26 19:00:34,481][105692] Updated weights for policy 0, policy_version 500546 (0.0005) [2023-12-26 19:00:34,536][105692] Updated weights for policy 0, policy_version 500556 (0.0006) [2023-12-26 19:00:34,592][105692] Updated weights for policy 0, policy_version 500566 (0.0006) [2023-12-26 19:00:34,642][105692] Updated weights for policy 0, policy_version 500576 (0.0006) [2023-12-26 19:00:35,020][105620] Updated weights for policy 1, policy_version 501006 (0.0008) [2023-12-26 19:00:35,071][105620] Updated weights for policy 1, policy_version 501016 (0.0009) [2023-12-26 19:00:35,134][105620] Updated weights for policy 1, policy_version 501026 (0.0010) [2023-12-26 19:00:35,208][105692] Updated weights for policy 0, policy_version 500586 (0.0005) [2023-12-26 19:00:35,260][105692] Updated weights for policy 0, policy_version 500596 (0.0005) [2023-12-26 19:00:35,303][105692] Updated weights for policy 0, policy_version 500606 (0.0005) [2023-12-26 19:00:35,931][105620] Updated weights for policy 1, policy_version 501036 (0.0009) [2023-12-26 19:00:35,987][105620] Updated weights for policy 1, policy_version 501046 (0.0008) [2023-12-26 19:00:35,999][105692] Updated weights for policy 0, policy_version 500616 (0.0006) [2023-12-26 19:00:36,046][105620] Updated weights for policy 1, policy_version 501056 (0.0007) [2023-12-26 19:00:36,056][105692] Updated weights for policy 0, policy_version 500626 (0.0008) [2023-12-26 19:00:36,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 256450560. Throughput: 0: 9857.2, 1: 9691.4. Samples: 256446076. Policy #0 lag: (min: 9.0, avg: 22.2, max: 41.0) [2023-12-26 19:00:36,062][104569] Avg episode reward: [(0, '8523.820'), (1, '9353.254')] [2023-12-26 19:00:36,110][105692] Updated weights for policy 0, policy_version 500636 (0.0008) [2023-12-26 19:00:36,826][105692] Updated weights for policy 0, policy_version 500646 (0.0007) [2023-12-26 19:00:36,839][105620] Updated weights for policy 1, policy_version 501066 (0.0008) [2023-12-26 19:00:36,884][105692] Updated weights for policy 0, policy_version 500656 (0.0005) [2023-12-26 19:00:36,891][105620] Updated weights for policy 1, policy_version 501076 (0.0009) [2023-12-26 19:00:36,943][105692] Updated weights for policy 0, policy_version 500666 (0.0007) [2023-12-26 19:00:36,944][105620] Updated weights for policy 1, policy_version 501086 (0.0007) [2023-12-26 19:00:36,995][105620] Updated weights for policy 1, policy_version 501096 (0.0009) [2023-12-26 19:00:37,579][105692] Updated weights for policy 0, policy_version 500676 (0.0007) [2023-12-26 19:00:37,634][105692] Updated weights for policy 0, policy_version 500686 (0.0009) [2023-12-26 19:00:37,693][105692] Updated weights for policy 0, policy_version 500696 (0.0010) [2023-12-26 19:00:37,812][105620] Updated weights for policy 1, policy_version 501106 (0.0009) [2023-12-26 19:00:37,858][105620] Updated weights for policy 1, policy_version 501116 (0.0008) [2023-12-26 19:00:37,905][105620] Updated weights for policy 1, policy_version 501126 (0.0009) [2023-12-26 19:00:38,471][105692] Updated weights for policy 0, policy_version 500706 (0.0009) [2023-12-26 19:00:38,530][105692] Updated weights for policy 0, policy_version 500716 (0.0009) [2023-12-26 19:00:38,573][105620] Updated weights for policy 1, policy_version 501136 (0.0006) [2023-12-26 19:00:38,591][105692] Updated weights for policy 0, policy_version 500726 (0.0008) [2023-12-26 19:00:38,637][105620] Updated weights for policy 1, policy_version 501146 (0.0008) [2023-12-26 19:00:38,650][105692] Updated weights for policy 0, policy_version 500736 (0.0006) [2023-12-26 19:00:38,700][105620] Updated weights for policy 1, policy_version 501156 (0.0006) [2023-12-26 19:00:39,263][105620] Updated weights for policy 1, policy_version 501166 (0.0007) [2023-12-26 19:00:39,329][105620] Updated weights for policy 1, policy_version 501176 (0.0010) [2023-12-26 19:00:39,396][105620] Updated weights for policy 1, policy_version 501186 (0.0010) [2023-12-26 19:00:39,466][105692] Updated weights for policy 0, policy_version 500746 (0.0007) [2023-12-26 19:00:39,529][105692] Updated weights for policy 0, policy_version 500756 (0.0007) [2023-12-26 19:00:39,593][105692] Updated weights for policy 0, policy_version 500766 (0.0009) [2023-12-26 19:00:40,239][105620] Updated weights for policy 1, policy_version 501196 (0.0008) [2023-12-26 19:00:40,302][105692] Updated weights for policy 0, policy_version 500776 (0.0006) [2023-12-26 19:00:40,304][105620] Updated weights for policy 1, policy_version 501206 (0.0009) [2023-12-26 19:00:40,365][105692] Updated weights for policy 0, policy_version 500786 (0.0007) [2023-12-26 19:00:40,366][105620] Updated weights for policy 1, policy_version 501216 (0.0008) [2023-12-26 19:00:40,426][105692] Updated weights for policy 0, policy_version 500796 (0.0008) [2023-12-26 19:00:41,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 256548864. Throughput: 0: 9959.2, 1: 9620.2. Samples: 256561924. Policy #0 lag: (min: 9.0, avg: 22.2, max: 41.0) [2023-12-26 19:00:41,063][104569] Avg episode reward: [(0, '8852.974'), (1, '9353.751')] [2023-12-26 19:00:41,076][105620] Updated weights for policy 1, policy_version 501226 (0.0008) [2023-12-26 19:00:41,153][105620] Updated weights for policy 1, policy_version 501236 (0.0009) [2023-12-26 19:00:41,215][105620] Updated weights for policy 1, policy_version 501246 (0.0009) [2023-12-26 19:00:41,248][105692] Updated weights for policy 0, policy_version 500806 (0.0009) [2023-12-26 19:00:41,274][105620] Updated weights for policy 1, policy_version 501256 (0.0009) [2023-12-26 19:00:41,322][105692] Updated weights for policy 0, policy_version 500816 (0.0009) [2023-12-26 19:00:41,410][105692] Updated weights for policy 0, policy_version 500826 (0.0010) [2023-12-26 19:00:42,091][105620] Updated weights for policy 1, policy_version 501266 (0.0009) [2023-12-26 19:00:42,157][105620] Updated weights for policy 1, policy_version 501276 (0.0009) [2023-12-26 19:00:42,197][105692] Updated weights for policy 0, policy_version 500836 (0.0009) [2023-12-26 19:00:42,221][105620] Updated weights for policy 1, policy_version 501286 (0.0008) [2023-12-26 19:00:42,265][105692] Updated weights for policy 0, policy_version 500846 (0.0008) [2023-12-26 19:00:42,334][105692] Updated weights for policy 0, policy_version 500856 (0.0009) [2023-12-26 19:00:42,953][105620] Updated weights for policy 1, policy_version 501296 (0.0006) [2023-12-26 19:00:43,000][105620] Updated weights for policy 1, policy_version 501306 (0.0005) [2023-12-26 19:00:43,047][105620] Updated weights for policy 1, policy_version 501316 (0.0005) [2023-12-26 19:00:43,104][105692] Updated weights for policy 0, policy_version 500866 (0.0008) [2023-12-26 19:00:43,162][105692] Updated weights for policy 0, policy_version 500876 (0.0009) [2023-12-26 19:00:43,221][105692] Updated weights for policy 0, policy_version 500886 (0.0010) [2023-12-26 19:00:43,274][105692] Updated weights for policy 0, policy_version 500896 (0.0010) [2023-12-26 19:00:43,684][105620] Updated weights for policy 1, policy_version 501326 (0.0008) [2023-12-26 19:00:43,746][105620] Updated weights for policy 1, policy_version 501336 (0.0009) [2023-12-26 19:00:43,816][105620] Updated weights for policy 1, policy_version 501346 (0.0010) [2023-12-26 19:00:44,028][105692] Updated weights for policy 0, policy_version 500906 (0.0006) [2023-12-26 19:00:44,077][105692] Updated weights for policy 0, policy_version 500916 (0.0009) [2023-12-26 19:00:44,129][105692] Updated weights for policy 0, policy_version 500926 (0.0009) [2023-12-26 19:00:44,592][105620] Updated weights for policy 1, policy_version 501356 (0.0009) [2023-12-26 19:00:44,652][105620] Updated weights for policy 1, policy_version 501366 (0.0008) [2023-12-26 19:00:44,704][105620] Updated weights for policy 1, policy_version 501376 (0.0008) [2023-12-26 19:00:44,891][105692] Updated weights for policy 0, policy_version 500936 (0.0011) [2023-12-26 19:00:44,955][105692] Updated weights for policy 0, policy_version 500946 (0.0011) [2023-12-26 19:00:45,020][105692] Updated weights for policy 0, policy_version 500956 (0.0011) [2023-12-26 19:00:45,486][105620] Updated weights for policy 1, policy_version 501386 (0.0008) [2023-12-26 19:00:45,536][105620] Updated weights for policy 1, policy_version 501396 (0.0008) [2023-12-26 19:00:45,586][105620] Updated weights for policy 1, policy_version 501406 (0.0009) [2023-12-26 19:00:45,650][105620] Updated weights for policy 1, policy_version 501416 (0.0008) [2023-12-26 19:00:45,775][105692] Updated weights for policy 0, policy_version 500966 (0.0011) [2023-12-26 19:00:45,827][105692] Updated weights for policy 0, policy_version 500976 (0.0011) [2023-12-26 19:00:45,885][105692] Updated weights for policy 0, policy_version 500986 (0.0011) [2023-12-26 19:00:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.4, 300 sec: 19466.4). Total num frames: 256647168. Throughput: 0: 9871.2, 1: 9570.4. Samples: 256616116. Policy #0 lag: (min: 9.0, avg: 22.2, max: 41.0) [2023-12-26 19:00:46,062][104569] Avg episode reward: [(0, '8756.341'), (1, '9354.457')] [2023-12-26 19:00:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000500992_128270336.pth... [2023-12-26 19:00:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000501416_128376832.pth... [2023-12-26 19:00:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000499840_127975424.pth [2023-12-26 19:00:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000500328_128098304.pth [2023-12-26 19:00:46,419][105620] Updated weights for policy 1, policy_version 501426 (0.0008) [2023-12-26 19:00:46,475][105620] Updated weights for policy 1, policy_version 501436 (0.0008) [2023-12-26 19:00:46,523][105620] Updated weights for policy 1, policy_version 501446 (0.0007) [2023-12-26 19:00:46,651][105692] Updated weights for policy 0, policy_version 500996 (0.0011) [2023-12-26 19:00:46,707][105692] Updated weights for policy 0, policy_version 501006 (0.0010) [2023-12-26 19:00:46,763][105692] Updated weights for policy 0, policy_version 501016 (0.0011) [2023-12-26 19:00:47,306][105620] Updated weights for policy 1, policy_version 501456 (0.0009) [2023-12-26 19:00:47,359][105620] Updated weights for policy 1, policy_version 501466 (0.0009) [2023-12-26 19:00:47,415][105620] Updated weights for policy 1, policy_version 501476 (0.0009) [2023-12-26 19:00:47,489][105692] Updated weights for policy 0, policy_version 501026 (0.0010) [2023-12-26 19:00:47,556][105692] Updated weights for policy 0, policy_version 501036 (0.0009) [2023-12-26 19:00:47,615][105692] Updated weights for policy 0, policy_version 501046 (0.0006) [2023-12-26 19:00:47,676][105692] Updated weights for policy 0, policy_version 501056 (0.0008) [2023-12-26 19:00:48,261][105620] Updated weights for policy 1, policy_version 501486 (0.0009) [2023-12-26 19:00:48,316][105692] Updated weights for policy 0, policy_version 501066 (0.0009) [2023-12-26 19:00:48,317][105620] Updated weights for policy 1, policy_version 501496 (0.0008) [2023-12-26 19:00:48,376][105620] Updated weights for policy 1, policy_version 501506 (0.0009) [2023-12-26 19:00:48,383][105692] Updated weights for policy 0, policy_version 501076 (0.0008) [2023-12-26 19:00:48,444][105692] Updated weights for policy 0, policy_version 501086 (0.0008) [2023-12-26 19:00:49,139][105620] Updated weights for policy 1, policy_version 501516 (0.0006) [2023-12-26 19:00:49,166][105692] Updated weights for policy 0, policy_version 501096 (0.0009) [2023-12-26 19:00:49,203][105620] Updated weights for policy 1, policy_version 501526 (0.0005) [2023-12-26 19:00:49,228][105692] Updated weights for policy 0, policy_version 501106 (0.0008) [2023-12-26 19:00:49,270][105620] Updated weights for policy 1, policy_version 501536 (0.0009) [2023-12-26 19:00:49,291][105692] Updated weights for policy 0, policy_version 501116 (0.0009) [2023-12-26 19:00:50,010][105620] Updated weights for policy 1, policy_version 501546 (0.0008) [2023-12-26 19:00:50,073][105620] Updated weights for policy 1, policy_version 501556 (0.0009) [2023-12-26 19:00:50,126][105692] Updated weights for policy 0, policy_version 501126 (0.0008) [2023-12-26 19:00:50,140][105620] Updated weights for policy 1, policy_version 501566 (0.0007) [2023-12-26 19:00:50,190][105692] Updated weights for policy 0, policy_version 501136 (0.0008) [2023-12-26 19:00:50,196][105620] Updated weights for policy 1, policy_version 501576 (0.0006) [2023-12-26 19:00:50,255][105692] Updated weights for policy 0, policy_version 501146 (0.0009) [2023-12-26 19:00:50,916][105620] Updated weights for policy 1, policy_version 501586 (0.0009) [2023-12-26 19:00:50,979][105620] Updated weights for policy 1, policy_version 501596 (0.0009) [2023-12-26 19:00:51,046][105620] Updated weights for policy 1, policy_version 501606 (0.0009) [2023-12-26 19:00:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 256737280. Throughput: 0: 9784.3, 1: 9480.6. Samples: 256727296. Policy #0 lag: (min: 9.0, avg: 22.2, max: 41.0) [2023-12-26 19:00:51,062][104569] Avg episode reward: [(0, '8818.128'), (1, '9353.012')] [2023-12-26 19:00:51,109][105692] Updated weights for policy 0, policy_version 501156 (0.0009) [2023-12-26 19:00:51,178][105692] Updated weights for policy 0, policy_version 501166 (0.0009) [2023-12-26 19:00:51,237][105692] Updated weights for policy 0, policy_version 501176 (0.0009) [2023-12-26 19:00:51,746][105620] Updated weights for policy 1, policy_version 501616 (0.0008) [2023-12-26 19:00:51,809][105620] Updated weights for policy 1, policy_version 501626 (0.0009) [2023-12-26 19:00:51,876][105620] Updated weights for policy 1, policy_version 501636 (0.0010) [2023-12-26 19:00:51,985][105692] Updated weights for policy 0, policy_version 501186 (0.0008) [2023-12-26 19:00:52,056][105692] Updated weights for policy 0, policy_version 501196 (0.0009) [2023-12-26 19:00:52,122][105692] Updated weights for policy 0, policy_version 501206 (0.0007) [2023-12-26 19:00:52,187][105692] Updated weights for policy 0, policy_version 501216 (0.0007) [2023-12-26 19:00:52,631][105620] Updated weights for policy 1, policy_version 501646 (0.0007) [2023-12-26 19:00:52,696][105620] Updated weights for policy 1, policy_version 501656 (0.0005) [2023-12-26 19:00:52,758][105620] Updated weights for policy 1, policy_version 501666 (0.0007) [2023-12-26 19:00:52,816][105692] Updated weights for policy 0, policy_version 501226 (0.0007) [2023-12-26 19:00:52,885][105692] Updated weights for policy 0, policy_version 501236 (0.0007) [2023-12-26 19:00:52,951][105692] Updated weights for policy 0, policy_version 501246 (0.0007) [2023-12-26 19:00:53,466][105620] Updated weights for policy 1, policy_version 501676 (0.0007) [2023-12-26 19:00:53,517][105620] Updated weights for policy 1, policy_version 501686 (0.0009) [2023-12-26 19:00:53,569][105620] Updated weights for policy 1, policy_version 501697 (0.0010) [2023-12-26 19:00:53,625][105692] Updated weights for policy 0, policy_version 501256 (0.0009) [2023-12-26 19:00:53,678][105692] Updated weights for policy 0, policy_version 501267 (0.0009) [2023-12-26 19:00:53,730][105692] Updated weights for policy 0, policy_version 501277 (0.0008) [2023-12-26 19:00:54,181][105620] Updated weights for policy 1, policy_version 501707 (0.0005) [2023-12-26 19:00:54,229][105620] Updated weights for policy 1, policy_version 501717 (0.0005) [2023-12-26 19:00:54,283][105620] Updated weights for policy 1, policy_version 501727 (0.0005) [2023-12-26 19:00:54,565][105692] Updated weights for policy 0, policy_version 501287 (0.0008) [2023-12-26 19:00:54,615][105692] Updated weights for policy 0, policy_version 501297 (0.0009) [2023-12-26 19:00:54,667][105692] Updated weights for policy 0, policy_version 501307 (0.0010) [2023-12-26 19:00:54,830][105620] Updated weights for policy 1, policy_version 501737 (0.0005) [2023-12-26 19:00:54,882][105620] Updated weights for policy 1, policy_version 501747 (0.0005) [2023-12-26 19:00:54,929][105620] Updated weights for policy 1, policy_version 501757 (0.0005) [2023-12-26 19:00:54,991][105620] Updated weights for policy 1, policy_version 501767 (0.0010) [2023-12-26 19:00:55,382][105692] Updated weights for policy 0, policy_version 501317 (0.0007) [2023-12-26 19:00:55,438][105692] Updated weights for policy 0, policy_version 501327 (0.0005) [2023-12-26 19:00:55,504][105692] Updated weights for policy 0, policy_version 501337 (0.0005) [2023-12-26 19:00:55,650][105620] Updated weights for policy 1, policy_version 501777 (0.0010) [2023-12-26 19:00:55,700][105620] Updated weights for policy 1, policy_version 501787 (0.0008) [2023-12-26 19:00:55,748][105620] Updated weights for policy 1, policy_version 501797 (0.0005) [2023-12-26 19:00:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 256835584. Throughput: 0: 9815.9, 1: 9517.0. Samples: 256846072. Policy #0 lag: (min: 9.0, avg: 22.2, max: 41.0) [2023-12-26 19:00:56,062][104569] Avg episode reward: [(0, '8732.395'), (1, '9259.419')] [2023-12-26 19:00:56,128][105692] Updated weights for policy 0, policy_version 501347 (0.0007) [2023-12-26 19:00:56,192][105692] Updated weights for policy 0, policy_version 501357 (0.0010) [2023-12-26 19:00:56,246][105692] Updated weights for policy 0, policy_version 501367 (0.0010) [2023-12-26 19:00:56,377][105620] Updated weights for policy 1, policy_version 501807 (0.0009) [2023-12-26 19:00:56,421][105620] Updated weights for policy 1, policy_version 501817 (0.0010) [2023-12-26 19:00:56,469][105620] Updated weights for policy 1, policy_version 501827 (0.0010) [2023-12-26 19:00:56,981][105692] Updated weights for policy 0, policy_version 501377 (0.0010) [2023-12-26 19:00:57,048][105692] Updated weights for policy 0, policy_version 501387 (0.0010) [2023-12-26 19:00:57,098][105692] Updated weights for policy 0, policy_version 501397 (0.0010) [2023-12-26 19:00:57,146][105692] Updated weights for policy 0, policy_version 501407 (0.0010) [2023-12-26 19:00:57,174][105620] Updated weights for policy 1, policy_version 501837 (0.0008) [2023-12-26 19:00:57,235][105620] Updated weights for policy 1, policy_version 501847 (0.0005) [2023-12-26 19:00:57,290][105620] Updated weights for policy 1, policy_version 501857 (0.0006) [2023-12-26 19:00:57,902][105692] Updated weights for policy 0, policy_version 501417 (0.0010) [2023-12-26 19:00:57,961][105692] Updated weights for policy 0, policy_version 501427 (0.0010) [2023-12-26 19:00:57,965][105620] Updated weights for policy 1, policy_version 501867 (0.0006) [2023-12-26 19:00:58,013][105692] Updated weights for policy 0, policy_version 501437 (0.0010) [2023-12-26 19:00:58,020][105620] Updated weights for policy 1, policy_version 501877 (0.0006) [2023-12-26 19:00:58,073][105620] Updated weights for policy 1, policy_version 501887 (0.0008) [2023-12-26 19:00:58,836][105692] Updated weights for policy 0, policy_version 501447 (0.0008) [2023-12-26 19:00:58,896][105620] Updated weights for policy 1, policy_version 501897 (0.0009) [2023-12-26 19:00:58,903][105692] Updated weights for policy 0, policy_version 501457 (0.0006) [2023-12-26 19:00:58,963][105620] Updated weights for policy 1, policy_version 501907 (0.0007) [2023-12-26 19:00:58,964][105692] Updated weights for policy 0, policy_version 501467 (0.0007) [2023-12-26 19:00:59,023][105620] Updated weights for policy 1, policy_version 501917 (0.0008) [2023-12-26 19:00:59,076][105620] Updated weights for policy 1, policy_version 501927 (0.0008) [2023-12-26 19:00:59,612][105692] Updated weights for policy 0, policy_version 501477 (0.0007) [2023-12-26 19:00:59,678][105692] Updated weights for policy 0, policy_version 501487 (0.0006) [2023-12-26 19:00:59,743][105692] Updated weights for policy 0, policy_version 501497 (0.0005) [2023-12-26 19:00:59,973][105620] Updated weights for policy 1, policy_version 501937 (0.0009) [2023-12-26 19:01:00,029][105620] Updated weights for policy 1, policy_version 501947 (0.0009) [2023-12-26 19:01:00,086][105620] Updated weights for policy 1, policy_version 501957 (0.0009) [2023-12-26 19:01:00,447][105692] Updated weights for policy 0, policy_version 501508 (0.0007) [2023-12-26 19:01:00,502][105692] Updated weights for policy 0, policy_version 501518 (0.0005) [2023-12-26 19:01:00,502][105585] KL-divergence is very high: 107.0956 [2023-12-26 19:01:00,522][105585] KL-divergence is very high: 224.6218 [2023-12-26 19:01:00,552][105585] KL-divergence is very high: 192.5023 [2023-12-26 19:01:00,562][105692] Updated weights for policy 0, policy_version 501528 (0.0005) [2023-12-26 19:01:00,569][105585] KL-divergence is very high: 332.4241 [2023-12-26 19:01:00,599][105585] KL-divergence is very high: 187.1808 [2023-12-26 19:01:00,872][105620] Updated weights for policy 1, policy_version 501967 (0.0008) [2023-12-26 19:01:00,918][105620] Updated weights for policy 1, policy_version 501977 (0.0008) [2023-12-26 19:01:00,965][105620] Updated weights for policy 1, policy_version 501987 (0.0009) [2023-12-26 19:01:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 256933888. Throughput: 0: 9805.0, 1: 9473.3. Samples: 256903720. Policy #0 lag: (min: 9.0, avg: 22.2, max: 41.0) [2023-12-26 19:01:01,063][104569] Avg episode reward: [(0, '8911.128'), (1, '9259.709')] [2023-12-26 19:01:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000501536_128409600.pth... [2023-12-26 19:01:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000501992_128524288.pth... [2023-12-26 19:01:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000500416_128122880.pth [2023-12-26 19:01:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000500872_128237568.pth [2023-12-26 19:01:01,208][105692] Updated weights for policy 0, policy_version 501538 (0.0006) [2023-12-26 19:01:01,275][105692] Updated weights for policy 0, policy_version 501548 (0.0010) [2023-12-26 19:01:01,347][105692] Updated weights for policy 0, policy_version 501558 (0.0010) [2023-12-26 19:01:01,409][105692] Updated weights for policy 0, policy_version 501568 (0.0009) [2023-12-26 19:01:01,717][105620] Updated weights for policy 1, policy_version 501997 (0.0007) [2023-12-26 19:01:01,780][105620] Updated weights for policy 1, policy_version 502007 (0.0009) [2023-12-26 19:01:01,837][105620] Updated weights for policy 1, policy_version 502017 (0.0009) [2023-12-26 19:01:02,153][105692] Updated weights for policy 0, policy_version 501578 (0.0009) [2023-12-26 19:01:02,174][105585] KL-divergence is very high: 146.7885 [2023-12-26 19:01:02,219][105692] Updated weights for policy 0, policy_version 501588 (0.0006) [2023-12-26 19:01:02,227][105585] KL-divergence is very high: 255.6337 [2023-12-26 19:01:02,273][105585] KL-divergence is very high: 231.1745 [2023-12-26 19:01:02,280][105692] Updated weights for policy 0, policy_version 501598 (0.0007) [2023-12-26 19:01:02,606][105620] Updated weights for policy 1, policy_version 502027 (0.0009) [2023-12-26 19:01:02,665][105620] Updated weights for policy 1, policy_version 502037 (0.0008) [2023-12-26 19:01:02,726][105620] Updated weights for policy 1, policy_version 502047 (0.0010) [2023-12-26 19:01:02,901][105692] Updated weights for policy 0, policy_version 501608 (0.0006) [2023-12-26 19:01:02,966][105692] Updated weights for policy 0, policy_version 501618 (0.0007) [2023-12-26 19:01:03,013][105692] Updated weights for policy 0, policy_version 501628 (0.0008) [2023-12-26 19:01:03,479][105620] Updated weights for policy 1, policy_version 502057 (0.0009) [2023-12-26 19:01:03,536][105620] Updated weights for policy 1, policy_version 502067 (0.0009) [2023-12-26 19:01:03,584][105620] Updated weights for policy 1, policy_version 502078 (0.0009) [2023-12-26 19:01:03,631][105620] Updated weights for policy 1, policy_version 502088 (0.0008) [2023-12-26 19:01:03,726][105692] Updated weights for policy 0, policy_version 501638 (0.0009) [2023-12-26 19:01:03,783][105692] Updated weights for policy 0, policy_version 501648 (0.0009) [2023-12-26 19:01:03,833][105692] Updated weights for policy 0, policy_version 501658 (0.0008) [2023-12-26 19:01:04,359][105620] Updated weights for policy 1, policy_version 502098 (0.0008) [2023-12-26 19:01:04,421][105620] Updated weights for policy 1, policy_version 502108 (0.0006) [2023-12-26 19:01:04,485][105620] Updated weights for policy 1, policy_version 502118 (0.0009) [2023-12-26 19:01:04,659][105692] Updated weights for policy 0, policy_version 501668 (0.0009) [2023-12-26 19:01:04,720][105692] Updated weights for policy 0, policy_version 501678 (0.0005) [2023-12-26 19:01:04,754][105585] KL-divergence is very high: 161.2099 [2023-12-26 19:01:04,770][105692] Updated weights for policy 0, policy_version 501688 (0.0007) [2023-12-26 19:01:04,798][105585] KL-divergence is very high: 153.4720 [2023-12-26 19:01:05,228][105620] Updated weights for policy 1, policy_version 502128 (0.0006) [2023-12-26 19:01:05,279][105620] Updated weights for policy 1, policy_version 502138 (0.0005) [2023-12-26 19:01:05,334][105620] Updated weights for policy 1, policy_version 502148 (0.0006) [2023-12-26 19:01:05,522][105585] KL-divergence is very high: 155.4904 [2023-12-26 19:01:05,527][105692] Updated weights for policy 0, policy_version 501698 (0.0009) [2023-12-26 19:01:05,528][105585] KL-divergence is very high: 132.7536 [2023-12-26 19:01:05,579][105692] Updated weights for policy 0, policy_version 501708 (0.0009) [2023-12-26 19:01:05,602][105585] KL-divergence is very high: 464.3142 [2023-12-26 19:01:05,607][105585] KL-divergence is very high: 409.1575 [2023-12-26 19:01:05,626][105692] Updated weights for policy 0, policy_version 501718 (0.0009) [2023-12-26 19:01:05,641][105585] KL-divergence is very high: 840.4456 [2023-12-26 19:01:05,645][105585] KL-divergence is very high: 708.4939 [2023-12-26 19:01:05,674][105692] Updated weights for policy 0, policy_version 501728 (0.0009) [2023-12-26 19:01:05,969][105620] Updated weights for policy 1, policy_version 502158 (0.0008) [2023-12-26 19:01:06,022][105620] Updated weights for policy 1, policy_version 502168 (0.0009) [2023-12-26 19:01:06,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19114.7, 300 sec: 19410.9). Total num frames: 257024000. Throughput: 0: 9805.0, 1: 9302.9. Samples: 257017284. Policy #0 lag: (min: 9.0, avg: 22.2, max: 41.0) [2023-12-26 19:01:06,063][104569] Avg episode reward: [(0, '8447.644'), (1, '9353.287')] [2023-12-26 19:01:06,083][105620] Updated weights for policy 1, policy_version 502178 (0.0009) [2023-12-26 19:01:06,427][105692] Updated weights for policy 0, policy_version 501738 (0.0008) [2023-12-26 19:01:06,489][105692] Updated weights for policy 0, policy_version 501748 (0.0008) [2023-12-26 19:01:06,551][105692] Updated weights for policy 0, policy_version 501758 (0.0010) [2023-12-26 19:01:06,762][105620] Updated weights for policy 1, policy_version 502188 (0.0007) [2023-12-26 19:01:06,811][105620] Updated weights for policy 1, policy_version 502198 (0.0006) [2023-12-26 19:01:06,864][105620] Updated weights for policy 1, policy_version 502208 (0.0005) [2023-12-26 19:01:07,340][105692] Updated weights for policy 0, policy_version 501768 (0.0009) [2023-12-26 19:01:07,395][105692] Updated weights for policy 0, policy_version 501778 (0.0008) [2023-12-26 19:01:07,448][105692] Updated weights for policy 0, policy_version 501788 (0.0008) [2023-12-26 19:01:07,611][105620] Updated weights for policy 1, policy_version 502218 (0.0006) [2023-12-26 19:01:07,671][105620] Updated weights for policy 1, policy_version 502228 (0.0009) [2023-12-26 19:01:07,732][105620] Updated weights for policy 1, policy_version 502238 (0.0008) [2023-12-26 19:01:07,786][105620] Updated weights for policy 1, policy_version 502248 (0.0009) [2023-12-26 19:01:08,143][105692] Updated weights for policy 0, policy_version 501798 (0.0007) [2023-12-26 19:01:08,203][105692] Updated weights for policy 0, policy_version 501808 (0.0009) [2023-12-26 19:01:08,269][105692] Updated weights for policy 0, policy_version 501818 (0.0007) [2023-12-26 19:01:08,578][105620] Updated weights for policy 1, policy_version 502258 (0.0009) [2023-12-26 19:01:08,640][105620] Updated weights for policy 1, policy_version 502268 (0.0009) [2023-12-26 19:01:08,710][105620] Updated weights for policy 1, policy_version 502278 (0.0010) [2023-12-26 19:01:08,887][105692] Updated weights for policy 0, policy_version 501828 (0.0007) [2023-12-26 19:01:08,944][105692] Updated weights for policy 0, policy_version 501838 (0.0009) [2023-12-26 19:01:08,996][105692] Updated weights for policy 0, policy_version 501848 (0.0010) [2023-12-26 19:01:09,500][105620] Updated weights for policy 1, policy_version 502288 (0.0009) [2023-12-26 19:01:09,557][105620] Updated weights for policy 1, policy_version 502298 (0.0008) [2023-12-26 19:01:09,622][105620] Updated weights for policy 1, policy_version 502308 (0.0010) [2023-12-26 19:01:09,695][105692] Updated weights for policy 0, policy_version 501858 (0.0008) [2023-12-26 19:01:09,755][105692] Updated weights for policy 0, policy_version 501868 (0.0009) [2023-12-26 19:01:09,820][105692] Updated weights for policy 0, policy_version 501878 (0.0011) [2023-12-26 19:01:09,882][105692] Updated weights for policy 0, policy_version 501888 (0.0011) [2023-12-26 19:01:10,434][105620] Updated weights for policy 1, policy_version 502318 (0.0008) [2023-12-26 19:01:10,495][105620] Updated weights for policy 1, policy_version 502328 (0.0009) [2023-12-26 19:01:10,559][105620] Updated weights for policy 1, policy_version 502338 (0.0008) [2023-12-26 19:01:10,619][105692] Updated weights for policy 0, policy_version 501898 (0.0011) [2023-12-26 19:01:10,671][105692] Updated weights for policy 0, policy_version 501908 (0.0011) [2023-12-26 19:01:10,723][105692] Updated weights for policy 0, policy_version 501918 (0.0011) [2023-12-26 19:01:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 257122304. Throughput: 0: 9809.5, 1: 9347.4. Samples: 257131064. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 19:01:11,062][104569] Avg episode reward: [(0, '7780.648'), (1, '9263.184')] [2023-12-26 19:01:11,386][105620] Updated weights for policy 1, policy_version 502348 (0.0008) [2023-12-26 19:01:11,452][105620] Updated weights for policy 1, policy_version 502358 (0.0007) [2023-12-26 19:01:11,459][105692] Updated weights for policy 0, policy_version 501928 (0.0010) [2023-12-26 19:01:11,510][105620] Updated weights for policy 1, policy_version 502368 (0.0006) [2023-12-26 19:01:11,519][105692] Updated weights for policy 0, policy_version 501938 (0.0010) [2023-12-26 19:01:11,574][105692] Updated weights for policy 0, policy_version 501948 (0.0010) [2023-12-26 19:01:12,264][105620] Updated weights for policy 1, policy_version 502378 (0.0006) [2023-12-26 19:01:12,293][105692] Updated weights for policy 0, policy_version 501958 (0.0008) [2023-12-26 19:01:12,328][105620] Updated weights for policy 1, policy_version 502388 (0.0009) [2023-12-26 19:01:12,355][105692] Updated weights for policy 0, policy_version 501968 (0.0007) [2023-12-26 19:01:12,397][105620] Updated weights for policy 1, policy_version 502398 (0.0008) [2023-12-26 19:01:12,425][105692] Updated weights for policy 0, policy_version 501978 (0.0009) [2023-12-26 19:01:12,459][105620] Updated weights for policy 1, policy_version 502408 (0.0008) [2023-12-26 19:01:13,167][105692] Updated weights for policy 0, policy_version 501988 (0.0006) [2023-12-26 19:01:13,219][105620] Updated weights for policy 1, policy_version 502418 (0.0009) [2023-12-26 19:01:13,229][105692] Updated weights for policy 0, policy_version 501998 (0.0006) [2023-12-26 19:01:13,279][105620] Updated weights for policy 1, policy_version 502428 (0.0006) [2023-12-26 19:01:13,295][105692] Updated weights for policy 0, policy_version 502008 (0.0006) [2023-12-26 19:01:13,347][105620] Updated weights for policy 1, policy_version 502438 (0.0005) [2023-12-26 19:01:13,853][105692] Updated weights for policy 0, policy_version 502018 (0.0006) [2023-12-26 19:01:13,897][105692] Updated weights for policy 0, policy_version 502028 (0.0005) [2023-12-26 19:01:13,950][105692] Updated weights for policy 0, policy_version 502038 (0.0005) [2023-12-26 19:01:14,008][105692] Updated weights for policy 0, policy_version 502048 (0.0005) [2023-12-26 19:01:14,094][105620] Updated weights for policy 1, policy_version 502448 (0.0008) [2023-12-26 19:01:14,145][105620] Updated weights for policy 1, policy_version 502458 (0.0008) [2023-12-26 19:01:14,193][105620] Updated weights for policy 1, policy_version 502468 (0.0008) [2023-12-26 19:01:14,628][105692] Updated weights for policy 0, policy_version 502058 (0.0010) [2023-12-26 19:01:14,676][105692] Updated weights for policy 0, policy_version 502068 (0.0010) [2023-12-26 19:01:14,721][105692] Updated weights for policy 0, policy_version 502078 (0.0010) [2023-12-26 19:01:14,988][105620] Updated weights for policy 1, policy_version 502478 (0.0007) [2023-12-26 19:01:15,057][105620] Updated weights for policy 1, policy_version 502488 (0.0006) [2023-12-26 19:01:15,122][105620] Updated weights for policy 1, policy_version 502498 (0.0005) [2023-12-26 19:01:15,520][105692] Updated weights for policy 0, policy_version 502088 (0.0011) [2023-12-26 19:01:15,579][105692] Updated weights for policy 0, policy_version 502098 (0.0010) [2023-12-26 19:01:15,637][105692] Updated weights for policy 0, policy_version 502108 (0.0007) [2023-12-26 19:01:15,666][105620] Updated weights for policy 1, policy_version 502508 (0.0005) [2023-12-26 19:01:15,719][105620] Updated weights for policy 1, policy_version 502518 (0.0005) [2023-12-26 19:01:15,764][105620] Updated weights for policy 1, policy_version 502528 (0.0005) [2023-12-26 19:01:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 257220608. Throughput: 0: 9709.4, 1: 9381.3. Samples: 257187556. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 19:01:16,063][104569] Avg episode reward: [(0, '7889.417'), (1, '9082.539')] [2023-12-26 19:01:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000502112_128557056.pth... [2023-12-26 19:01:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000502536_128663552.pth... [2023-12-26 19:01:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000500992_128270336.pth [2023-12-26 19:01:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000501416_128376832.pth [2023-12-26 19:01:16,330][105692] Updated weights for policy 0, policy_version 502118 (0.0007) [2023-12-26 19:01:16,375][105692] Updated weights for policy 0, policy_version 502128 (0.0006) [2023-12-26 19:01:16,388][105620] Updated weights for policy 1, policy_version 502538 (0.0005) [2023-12-26 19:01:16,421][105692] Updated weights for policy 0, policy_version 502138 (0.0005) [2023-12-26 19:01:16,447][105620] Updated weights for policy 1, policy_version 502548 (0.0006) [2023-12-26 19:01:16,509][105620] Updated weights for policy 1, policy_version 502558 (0.0006) [2023-12-26 19:01:16,562][105620] Updated weights for policy 1, policy_version 502568 (0.0005) [2023-12-26 19:01:16,976][105692] Updated weights for policy 0, policy_version 502148 (0.0005) [2023-12-26 19:01:17,033][105692] Updated weights for policy 0, policy_version 502158 (0.0005) [2023-12-26 19:01:17,094][105692] Updated weights for policy 0, policy_version 502168 (0.0005) [2023-12-26 19:01:17,288][105620] Updated weights for policy 1, policy_version 502578 (0.0010) [2023-12-26 19:01:17,349][105620] Updated weights for policy 1, policy_version 502588 (0.0010) [2023-12-26 19:01:17,413][105620] Updated weights for policy 1, policy_version 502598 (0.0010) [2023-12-26 19:01:17,606][105692] Updated weights for policy 0, policy_version 502178 (0.0005) [2023-12-26 19:01:17,664][105692] Updated weights for policy 0, policy_version 502188 (0.0005) [2023-12-26 19:01:17,727][105692] Updated weights for policy 0, policy_version 502198 (0.0009) [2023-12-26 19:01:17,793][105692] Updated weights for policy 0, policy_version 502208 (0.0009) [2023-12-26 19:01:18,186][105620] Updated weights for policy 1, policy_version 502608 (0.0010) [2023-12-26 19:01:18,245][105620] Updated weights for policy 1, policy_version 502618 (0.0011) [2023-12-26 19:01:18,299][105620] Updated weights for policy 1, policy_version 502628 (0.0011) [2023-12-26 19:01:18,485][105692] Updated weights for policy 0, policy_version 502218 (0.0006) [2023-12-26 19:01:18,548][105692] Updated weights for policy 0, policy_version 502228 (0.0011) [2023-12-26 19:01:18,615][105692] Updated weights for policy 0, policy_version 502238 (0.0011) [2023-12-26 19:01:19,015][105620] Updated weights for policy 1, policy_version 502638 (0.0007) [2023-12-26 19:01:19,073][105620] Updated weights for policy 1, policy_version 502648 (0.0006) [2023-12-26 19:01:19,119][105620] Updated weights for policy 1, policy_version 502658 (0.0008) [2023-12-26 19:01:19,291][105692] Updated weights for policy 0, policy_version 502248 (0.0010) [2023-12-26 19:01:19,353][105692] Updated weights for policy 0, policy_version 502258 (0.0008) [2023-12-26 19:01:19,413][105692] Updated weights for policy 0, policy_version 502268 (0.0009) [2023-12-26 19:01:19,903][105620] Updated weights for policy 1, policy_version 502668 (0.0009) [2023-12-26 19:01:19,970][105620] Updated weights for policy 1, policy_version 502678 (0.0009) [2023-12-26 19:01:19,982][105692] Updated weights for policy 0, policy_version 502278 (0.0007) [2023-12-26 19:01:20,033][105620] Updated weights for policy 1, policy_version 502688 (0.0007) [2023-12-26 19:01:20,049][105692] Updated weights for policy 0, policy_version 502288 (0.0006) [2023-12-26 19:01:20,116][105692] Updated weights for policy 0, policy_version 502298 (0.0008) [2023-12-26 19:01:20,679][105692] Updated weights for policy 0, policy_version 502308 (0.0006) [2023-12-26 19:01:20,740][105692] Updated weights for policy 0, policy_version 502318 (0.0010) [2023-12-26 19:01:20,765][105620] Updated weights for policy 1, policy_version 502698 (0.0007) [2023-12-26 19:01:20,800][105692] Updated weights for policy 0, policy_version 502328 (0.0011) [2023-12-26 19:01:20,819][105585] KL-divergence is very high: 111.5129 [2023-12-26 19:01:20,831][105620] Updated weights for policy 1, policy_version 502708 (0.0007) [2023-12-26 19:01:20,894][105620] Updated weights for policy 1, policy_version 502718 (0.0007) [2023-12-26 19:01:20,960][105620] Updated weights for policy 1, policy_version 502728 (0.0009) [2023-12-26 19:01:21,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 257327104. Throughput: 0: 9711.9, 1: 9490.4. Samples: 257310180. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 19:01:21,063][104569] Avg episode reward: [(0, '8251.199'), (1, '9081.469')] [2023-12-26 19:01:21,533][105692] Updated weights for policy 0, policy_version 502338 (0.0010) [2023-12-26 19:01:21,598][105692] Updated weights for policy 0, policy_version 502348 (0.0006) [2023-12-26 19:01:21,663][105692] Updated weights for policy 0, policy_version 502358 (0.0006) [2023-12-26 19:01:21,728][105620] Updated weights for policy 1, policy_version 502738 (0.0011) [2023-12-26 19:01:21,728][105692] Updated weights for policy 0, policy_version 502368 (0.0007) [2023-12-26 19:01:21,784][105620] Updated weights for policy 1, policy_version 502748 (0.0011) [2023-12-26 19:01:21,845][105620] Updated weights for policy 1, policy_version 502758 (0.0009) [2023-12-26 19:01:22,426][105692] Updated weights for policy 0, policy_version 502378 (0.0011) [2023-12-26 19:01:22,478][105692] Updated weights for policy 0, policy_version 502388 (0.0006) [2023-12-26 19:01:22,544][105692] Updated weights for policy 0, policy_version 502398 (0.0006) [2023-12-26 19:01:22,559][105620] Updated weights for policy 1, policy_version 502768 (0.0010) [2023-12-26 19:01:22,613][105620] Updated weights for policy 1, policy_version 502778 (0.0011) [2023-12-26 19:01:22,664][105620] Updated weights for policy 1, policy_version 502788 (0.0010) [2023-12-26 19:01:23,204][105692] Updated weights for policy 0, policy_version 502408 (0.0005) [2023-12-26 19:01:23,259][105692] Updated weights for policy 0, policy_version 502418 (0.0006) [2023-12-26 19:01:23,319][105692] Updated weights for policy 0, policy_version 502428 (0.0009) [2023-12-26 19:01:23,443][105620] Updated weights for policy 1, policy_version 502798 (0.0010) [2023-12-26 19:01:23,501][105620] Updated weights for policy 1, policy_version 502808 (0.0010) [2023-12-26 19:01:23,554][105620] Updated weights for policy 1, policy_version 502818 (0.0010) [2023-12-26 19:01:24,057][105692] Updated weights for policy 0, policy_version 502438 (0.0010) [2023-12-26 19:01:24,109][105692] Updated weights for policy 0, policy_version 502448 (0.0010) [2023-12-26 19:01:24,154][105692] Updated weights for policy 0, policy_version 502458 (0.0010) [2023-12-26 19:01:24,254][105620] Updated weights for policy 1, policy_version 502828 (0.0007) [2023-12-26 19:01:24,305][105620] Updated weights for policy 1, policy_version 502838 (0.0010) [2023-12-26 19:01:24,357][105620] Updated weights for policy 1, policy_version 502848 (0.0010) [2023-12-26 19:01:24,902][105692] Updated weights for policy 0, policy_version 502468 (0.0010) [2023-12-26 19:01:24,961][105692] Updated weights for policy 0, policy_version 502478 (0.0010) [2023-12-26 19:01:25,016][105692] Updated weights for policy 0, policy_version 502488 (0.0010) [2023-12-26 19:01:25,027][105585] KL-divergence is very high: 257.9885 [2023-12-26 19:01:25,050][105620] Updated weights for policy 1, policy_version 502858 (0.0009) [2023-12-26 19:01:25,104][105620] Updated weights for policy 1, policy_version 502868 (0.0008) [2023-12-26 19:01:25,152][105620] Updated weights for policy 1, policy_version 502878 (0.0008) [2023-12-26 19:01:25,216][105620] Updated weights for policy 1, policy_version 502888 (0.0007) [2023-12-26 19:01:25,644][105692] Updated weights for policy 0, policy_version 502498 (0.0010) [2023-12-26 19:01:25,697][105692] Updated weights for policy 0, policy_version 502508 (0.0007) [2023-12-26 19:01:25,744][105692] Updated weights for policy 0, policy_version 502518 (0.0007) [2023-12-26 19:01:25,798][105692] Updated weights for policy 0, policy_version 502528 (0.0005) [2023-12-26 19:01:26,024][105620] Updated weights for policy 1, policy_version 502899 (0.0010) [2023-12-26 19:01:26,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 257417216. Throughput: 0: 9755.2, 1: 9490.7. Samples: 257427988. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 19:01:26,063][104569] Avg episode reward: [(0, '7867.270'), (1, '9263.068')] [2023-12-26 19:01:26,070][105620] Updated weights for policy 1, policy_version 502909 (0.0008) [2023-12-26 19:01:26,124][105620] Updated weights for policy 1, policy_version 502920 (0.0010) [2023-12-26 19:01:26,345][105692] Updated weights for policy 0, policy_version 502538 (0.0005) [2023-12-26 19:01:26,398][105692] Updated weights for policy 0, policy_version 502548 (0.0010) [2023-12-26 19:01:26,442][105692] Updated weights for policy 0, policy_version 502558 (0.0010) [2023-12-26 19:01:26,927][105620] Updated weights for policy 1, policy_version 502930 (0.0005) [2023-12-26 19:01:26,975][105620] Updated weights for policy 1, policy_version 502940 (0.0005) [2023-12-26 19:01:27,026][105620] Updated weights for policy 1, policy_version 502950 (0.0006) [2023-12-26 19:01:27,208][105692] Updated weights for policy 0, policy_version 502568 (0.0010) [2023-12-26 19:01:27,273][105692] Updated weights for policy 0, policy_version 502578 (0.0010) [2023-12-26 19:01:27,332][105692] Updated weights for policy 0, policy_version 502588 (0.0008) [2023-12-26 19:01:27,625][105620] Updated weights for policy 1, policy_version 502960 (0.0006) [2023-12-26 19:01:27,684][105620] Updated weights for policy 1, policy_version 502970 (0.0005) [2023-12-26 19:01:27,736][105620] Updated weights for policy 1, policy_version 502980 (0.0005) [2023-12-26 19:01:27,890][105692] Updated weights for policy 0, policy_version 502598 (0.0006) [2023-12-26 19:01:27,935][105692] Updated weights for policy 0, policy_version 502608 (0.0010) [2023-12-26 19:01:27,989][105692] Updated weights for policy 0, policy_version 502618 (0.0010) [2023-12-26 19:01:28,423][105620] Updated weights for policy 1, policy_version 502990 (0.0008) [2023-12-26 19:01:28,489][105620] Updated weights for policy 1, policy_version 503000 (0.0011) [2023-12-26 19:01:28,551][105620] Updated weights for policy 1, policy_version 503010 (0.0010) [2023-12-26 19:01:28,741][105692] Updated weights for policy 0, policy_version 502628 (0.0009) [2023-12-26 19:01:28,793][105692] Updated weights for policy 0, policy_version 502638 (0.0010) [2023-12-26 19:01:28,855][105692] Updated weights for policy 0, policy_version 502648 (0.0010) [2023-12-26 19:01:29,292][105620] Updated weights for policy 1, policy_version 503020 (0.0008) [2023-12-26 19:01:29,372][105620] Updated weights for policy 1, policy_version 503030 (0.0006) [2023-12-26 19:01:29,424][105586] KL-divergence is very high: 129.2834 [2023-12-26 19:01:29,436][105620] Updated weights for policy 1, policy_version 503040 (0.0008) [2023-12-26 19:01:29,487][105692] Updated weights for policy 0, policy_version 502658 (0.0009) [2023-12-26 19:01:29,555][105692] Updated weights for policy 0, policy_version 502668 (0.0005) [2023-12-26 19:01:29,627][105692] Updated weights for policy 0, policy_version 502678 (0.0009) [2023-12-26 19:01:29,685][105692] Updated weights for policy 0, policy_version 502688 (0.0010) [2023-12-26 19:01:30,115][105620] Updated weights for policy 1, policy_version 503050 (0.0008) [2023-12-26 19:01:30,168][105620] Updated weights for policy 1, policy_version 503060 (0.0008) [2023-12-26 19:01:30,223][105620] Updated weights for policy 1, policy_version 503070 (0.0008) [2023-12-26 19:01:30,278][105620] Updated weights for policy 1, policy_version 503080 (0.0008) [2023-12-26 19:01:30,375][105692] Updated weights for policy 0, policy_version 502698 (0.0010) [2023-12-26 19:01:30,438][105692] Updated weights for policy 0, policy_version 502708 (0.0010) [2023-12-26 19:01:30,494][105692] Updated weights for policy 0, policy_version 502718 (0.0011) [2023-12-26 19:01:31,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 257515520. Throughput: 0: 9902.7, 1: 9523.7. Samples: 257490304. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 19:01:31,062][104569] Avg episode reward: [(0, '8627.563'), (1, '9081.987')] [2023-12-26 19:01:31,065][105620] Updated weights for policy 1, policy_version 503090 (0.0008) [2023-12-26 19:01:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000502720_128712704.pth... [2023-12-26 19:01:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000501536_128409600.pth [2023-12-26 19:01:31,127][105620] Updated weights for policy 1, policy_version 503100 (0.0006) [2023-12-26 19:01:31,192][105620] Updated weights for policy 1, policy_version 503110 (0.0008) [2023-12-26 19:01:31,202][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000503112_128811008.pth... [2023-12-26 19:01:31,207][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000501992_128524288.pth [2023-12-26 19:01:31,264][105692] Updated weights for policy 0, policy_version 502728 (0.0010) [2023-12-26 19:01:31,329][105692] Updated weights for policy 0, policy_version 502738 (0.0010) [2023-12-26 19:01:31,397][105692] Updated weights for policy 0, policy_version 502748 (0.0009) [2023-12-26 19:01:31,913][105620] Updated weights for policy 1, policy_version 503120 (0.0006) [2023-12-26 19:01:31,975][105620] Updated weights for policy 1, policy_version 503130 (0.0010) [2023-12-26 19:01:32,036][105620] Updated weights for policy 1, policy_version 503140 (0.0009) [2023-12-26 19:01:32,101][105692] Updated weights for policy 0, policy_version 502758 (0.0008) [2023-12-26 19:01:32,161][105692] Updated weights for policy 0, policy_version 502768 (0.0005) [2023-12-26 19:01:32,228][105692] Updated weights for policy 0, policy_version 502778 (0.0010) [2023-12-26 19:01:32,768][105620] Updated weights for policy 1, policy_version 503150 (0.0008) [2023-12-26 19:01:32,831][105620] Updated weights for policy 1, policy_version 503160 (0.0008) [2023-12-26 19:01:32,891][105620] Updated weights for policy 1, policy_version 503170 (0.0008) [2023-12-26 19:01:32,903][105692] Updated weights for policy 0, policy_version 502788 (0.0011) [2023-12-26 19:01:32,959][105692] Updated weights for policy 0, policy_version 502798 (0.0010) [2023-12-26 19:01:33,014][105692] Updated weights for policy 0, policy_version 502808 (0.0010) [2023-12-26 19:01:33,532][105620] Updated weights for policy 1, policy_version 503180 (0.0006) [2023-12-26 19:01:33,583][105620] Updated weights for policy 1, policy_version 503190 (0.0010) [2023-12-26 19:01:33,634][105620] Updated weights for policy 1, policy_version 503200 (0.0010) [2023-12-26 19:01:33,669][105692] Updated weights for policy 0, policy_version 502818 (0.0010) [2023-12-26 19:01:33,718][105692] Updated weights for policy 0, policy_version 502828 (0.0005) [2023-12-26 19:01:33,764][105692] Updated weights for policy 0, policy_version 502838 (0.0005) [2023-12-26 19:01:33,811][105692] Updated weights for policy 0, policy_version 502848 (0.0005) [2023-12-26 19:01:34,385][105620] Updated weights for policy 1, policy_version 503210 (0.0011) [2023-12-26 19:01:34,397][105585] KL-divergence is very high: 181.0870 [2023-12-26 19:01:34,422][105585] KL-divergence is very high: 782.6890 [2023-12-26 19:01:34,434][105692] Updated weights for policy 0, policy_version 502858 (0.0009) [2023-12-26 19:01:34,445][105585] KL-divergence is very high: 703.9023 [2023-12-26 19:01:34,450][105620] Updated weights for policy 1, policy_version 503220 (0.0011) [2023-12-26 19:01:34,470][105585] KL-divergence is very high: 1395.6140 [2023-12-26 19:01:34,493][105585] KL-divergence is very high: 797.2449 [2023-12-26 19:01:34,494][105692] Updated weights for policy 0, policy_version 502868 (0.0008) [2023-12-26 19:01:34,507][105620] Updated weights for policy 1, policy_version 503230 (0.0011) [2023-12-26 19:01:34,517][105585] KL-divergence is very high: 1336.1774 [2023-12-26 19:01:34,542][105585] KL-divergence is very high: 602.8087 [2023-12-26 19:01:34,554][105692] Updated weights for policy 0, policy_version 502878 (0.0006) [2023-12-26 19:01:34,571][105620] Updated weights for policy 1, policy_version 503240 (0.0010) [2023-12-26 19:01:35,276][105620] Updated weights for policy 1, policy_version 503250 (0.0010) [2023-12-26 19:01:35,295][105692] Updated weights for policy 0, policy_version 502888 (0.0010) [2023-12-26 19:01:35,320][105585] KL-divergence is very high: 181.2803 [2023-12-26 19:01:35,324][105620] Updated weights for policy 1, policy_version 503260 (0.0010) [2023-12-26 19:01:35,349][105692] Updated weights for policy 0, policy_version 502898 (0.0010) [2023-12-26 19:01:35,364][105585] KL-divergence is very high: 280.3290 [2023-12-26 19:01:35,372][105620] Updated weights for policy 1, policy_version 503270 (0.0010) [2023-12-26 19:01:35,404][105692] Updated weights for policy 0, policy_version 502908 (0.0010) [2023-12-26 19:01:35,412][105585] KL-divergence is very high: 248.7545 [2023-12-26 19:01:36,005][105692] Updated weights for policy 0, policy_version 502918 (0.0007) [2023-12-26 19:01:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 257613824. Throughput: 0: 9980.8, 1: 9597.1. Samples: 257608304. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 19:01:36,063][104569] Avg episode reward: [(0, '8442.375'), (1, '9082.116')] [2023-12-26 19:01:36,063][105692] Updated weights for policy 0, policy_version 502928 (0.0008) [2023-12-26 19:01:36,119][105620] Updated weights for policy 1, policy_version 503280 (0.0009) [2023-12-26 19:01:36,125][105692] Updated weights for policy 0, policy_version 502938 (0.0007) [2023-12-26 19:01:36,177][105620] Updated weights for policy 1, policy_version 503290 (0.0010) [2023-12-26 19:01:36,236][105620] Updated weights for policy 1, policy_version 503300 (0.0009) [2023-12-26 19:01:36,846][105620] Updated weights for policy 1, policy_version 503310 (0.0007) [2023-12-26 19:01:36,847][105692] Updated weights for policy 0, policy_version 502948 (0.0009) [2023-12-26 19:01:36,900][105692] Updated weights for policy 0, policy_version 502958 (0.0009) [2023-12-26 19:01:36,907][105620] Updated weights for policy 1, policy_version 503320 (0.0005) [2023-12-26 19:01:36,949][105692] Updated weights for policy 0, policy_version 502968 (0.0009) [2023-12-26 19:01:36,968][105620] Updated weights for policy 1, policy_version 503330 (0.0010) [2023-12-26 19:01:37,524][105620] Updated weights for policy 1, policy_version 503340 (0.0009) [2023-12-26 19:01:37,575][105620] Updated weights for policy 1, policy_version 503350 (0.0010) [2023-12-26 19:01:37,631][105620] Updated weights for policy 1, policy_version 503360 (0.0011) [2023-12-26 19:01:37,815][105692] Updated weights for policy 0, policy_version 502978 (0.0006) [2023-12-26 19:01:37,874][105692] Updated weights for policy 0, policy_version 502988 (0.0008) [2023-12-26 19:01:37,922][105692] Updated weights for policy 0, policy_version 502998 (0.0008) [2023-12-26 19:01:37,977][105692] Updated weights for policy 0, policy_version 503008 (0.0007) [2023-12-26 19:01:38,281][105620] Updated weights for policy 1, policy_version 503370 (0.0009) [2023-12-26 19:01:38,371][105620] Updated weights for policy 1, policy_version 503380 (0.0008) [2023-12-26 19:01:38,430][105620] Updated weights for policy 1, policy_version 503390 (0.0008) [2023-12-26 19:01:38,485][105620] Updated weights for policy 1, policy_version 503400 (0.0005) [2023-12-26 19:01:38,647][105692] Updated weights for policy 0, policy_version 503018 (0.0010) [2023-12-26 19:01:38,711][105692] Updated weights for policy 0, policy_version 503028 (0.0011) [2023-12-26 19:01:38,774][105692] Updated weights for policy 0, policy_version 503038 (0.0011) [2023-12-26 19:01:39,033][105620] Updated weights for policy 1, policy_version 503410 (0.0011) [2023-12-26 19:01:39,098][105620] Updated weights for policy 1, policy_version 503420 (0.0010) [2023-12-26 19:01:39,155][105620] Updated weights for policy 1, policy_version 503430 (0.0006) [2023-12-26 19:01:39,514][105692] Updated weights for policy 0, policy_version 503048 (0.0006) [2023-12-26 19:01:39,578][105692] Updated weights for policy 0, policy_version 503058 (0.0008) [2023-12-26 19:01:39,642][105692] Updated weights for policy 0, policy_version 503068 (0.0010) [2023-12-26 19:01:39,861][105620] Updated weights for policy 1, policy_version 503440 (0.0008) [2023-12-26 19:01:39,922][105620] Updated weights for policy 1, policy_version 503450 (0.0009) [2023-12-26 19:01:39,983][105620] Updated weights for policy 1, policy_version 503460 (0.0009) [2023-12-26 19:01:40,398][105692] Updated weights for policy 0, policy_version 503078 (0.0009) [2023-12-26 19:01:40,461][105692] Updated weights for policy 0, policy_version 503088 (0.0011) [2023-12-26 19:01:40,517][105692] Updated weights for policy 0, policy_version 503098 (0.0011) [2023-12-26 19:01:40,751][105620] Updated weights for policy 1, policy_version 503470 (0.0009) [2023-12-26 19:01:40,813][105620] Updated weights for policy 1, policy_version 503480 (0.0009) [2023-12-26 19:01:40,870][105620] Updated weights for policy 1, policy_version 503490 (0.0009) [2023-12-26 19:01:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 257720320. Throughput: 0: 9980.8, 1: 9598.7. Samples: 257727152. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 19:01:41,062][104569] Avg episode reward: [(0, '8271.283'), (1, '9355.657')] [2023-12-26 19:01:41,183][105692] Updated weights for policy 0, policy_version 503108 (0.0010) [2023-12-26 19:01:41,237][105692] Updated weights for policy 0, policy_version 503118 (0.0008) [2023-12-26 19:01:41,306][105692] Updated weights for policy 0, policy_version 503128 (0.0007) [2023-12-26 19:01:41,700][105620] Updated weights for policy 1, policy_version 503500 (0.0008) [2023-12-26 19:01:41,780][105620] Updated weights for policy 1, policy_version 503510 (0.0009) [2023-12-26 19:01:41,841][105620] Updated weights for policy 1, policy_version 503520 (0.0009) [2023-12-26 19:01:42,026][105692] Updated weights for policy 0, policy_version 503138 (0.0008) [2023-12-26 19:01:42,057][105585] KL-divergence is very high: 177.1680 [2023-12-26 19:01:42,088][105692] Updated weights for policy 0, policy_version 503148 (0.0009) [2023-12-26 19:01:42,109][105585] KL-divergence is very high: 359.6750 [2023-12-26 19:01:42,147][105692] Updated weights for policy 0, policy_version 503158 (0.0009) [2023-12-26 19:01:42,152][105585] KL-divergence is very high: 346.8503 [2023-12-26 19:01:42,193][105585] KL-divergence is very high: 301.5111 [2023-12-26 19:01:42,199][105692] Updated weights for policy 0, policy_version 503168 (0.0009) [2023-12-26 19:01:42,573][105620] Updated weights for policy 1, policy_version 503530 (0.0009) [2023-12-26 19:01:42,633][105620] Updated weights for policy 1, policy_version 503540 (0.0009) [2023-12-26 19:01:42,697][105620] Updated weights for policy 1, policy_version 503550 (0.0009) [2023-12-26 19:01:42,758][105620] Updated weights for policy 1, policy_version 503560 (0.0005) [2023-12-26 19:01:42,964][105585] KL-divergence is very high: 312.0060 [2023-12-26 19:01:42,973][105585] KL-divergence is very high: 192.1614 [2023-12-26 19:01:42,986][105585] KL-divergence is very high: 309.1669 [2023-12-26 19:01:43,012][105585] KL-divergence is very high: 156.3971 [2023-12-26 19:01:43,022][105692] Updated weights for policy 0, policy_version 503178 (0.0009) [2023-12-26 19:01:43,029][105585] KL-divergence is very high: 161.4838 [2023-12-26 19:01:43,086][105692] Updated weights for policy 0, policy_version 503188 (0.0009) [2023-12-26 19:01:43,143][105692] Updated weights for policy 0, policy_version 503198 (0.0009) [2023-12-26 19:01:43,418][105620] Updated weights for policy 1, policy_version 503570 (0.0009) [2023-12-26 19:01:43,469][105620] Updated weights for policy 1, policy_version 503580 (0.0009) [2023-12-26 19:01:43,523][105620] Updated weights for policy 1, policy_version 503590 (0.0009) [2023-12-26 19:01:43,813][105692] Updated weights for policy 0, policy_version 503208 (0.0006) [2023-12-26 19:01:43,861][105692] Updated weights for policy 0, policy_version 503218 (0.0005) [2023-12-26 19:01:43,917][105692] Updated weights for policy 0, policy_version 503228 (0.0005) [2023-12-26 19:01:44,415][105692] Updated weights for policy 0, policy_version 503238 (0.0005) [2023-12-26 19:01:44,422][105620] Updated weights for policy 1, policy_version 503600 (0.0008) [2023-12-26 19:01:44,469][105692] Updated weights for policy 0, policy_version 503248 (0.0007) [2023-12-26 19:01:44,472][105620] Updated weights for policy 1, policy_version 503610 (0.0007) [2023-12-26 19:01:44,514][105692] Updated weights for policy 0, policy_version 503258 (0.0008) [2023-12-26 19:01:44,523][105620] Updated weights for policy 1, policy_version 503620 (0.0008) [2023-12-26 19:01:45,183][105692] Updated weights for policy 0, policy_version 503268 (0.0008) [2023-12-26 19:01:45,239][105692] Updated weights for policy 0, policy_version 503278 (0.0008) [2023-12-26 19:01:45,296][105692] Updated weights for policy 0, policy_version 503288 (0.0011) [2023-12-26 19:01:45,386][105620] Updated weights for policy 1, policy_version 503630 (0.0008) [2023-12-26 19:01:45,433][105620] Updated weights for policy 1, policy_version 503640 (0.0005) [2023-12-26 19:01:45,480][105620] Updated weights for policy 1, policy_version 503650 (0.0007) [2023-12-26 19:01:46,000][105692] Updated weights for policy 0, policy_version 503298 (0.0008) [2023-12-26 19:01:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 257810432. Throughput: 0: 9982.5, 1: 9553.1. Samples: 257782820. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 19:01:46,062][104569] Avg episode reward: [(0, '7331.409'), (1, '9355.926')] [2023-12-26 19:01:46,066][105692] Updated weights for policy 0, policy_version 503308 (0.0011) [2023-12-26 19:01:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000503656_128950272.pth... [2023-12-26 19:01:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000502536_128663552.pth [2023-12-26 19:01:46,074][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_000503656_128950272.pth [2023-12-26 19:01:46,117][105692] Updated weights for policy 0, policy_version 503318 (0.0010) [2023-12-26 19:01:46,169][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000503328_128868352.pth... [2023-12-26 19:01:46,169][105692] Updated weights for policy 0, policy_version 503328 (0.0010) [2023-12-26 19:01:46,173][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000502112_128557056.pth [2023-12-26 19:01:46,174][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_000503328_128868352.pth [2023-12-26 19:01:46,175][105620] Updated weights for policy 1, policy_version 503660 (0.0007) [2023-12-26 19:01:46,240][105620] Updated weights for policy 1, policy_version 503670 (0.0010) [2023-12-26 19:01:46,292][105620] Updated weights for policy 1, policy_version 503680 (0.0009) [2023-12-26 19:01:46,806][105692] Updated weights for policy 0, policy_version 503338 (0.0006) [2023-12-26 19:01:46,874][105692] Updated weights for policy 0, policy_version 503348 (0.0010) [2023-12-26 19:01:46,943][105692] Updated weights for policy 0, policy_version 503358 (0.0010) [2023-12-26 19:01:46,982][105620] Updated weights for policy 1, policy_version 503690 (0.0009) [2023-12-26 19:01:47,045][105620] Updated weights for policy 1, policy_version 503700 (0.0005) [2023-12-26 19:01:47,104][105620] Updated weights for policy 1, policy_version 503710 (0.0006) [2023-12-26 19:01:47,155][105620] Updated weights for policy 1, policy_version 503720 (0.0006) [2023-12-26 19:01:47,626][105692] Updated weights for policy 0, policy_version 503368 (0.0011) [2023-12-26 19:01:47,685][105692] Updated weights for policy 0, policy_version 503378 (0.0010) [2023-12-26 19:01:47,751][105620] Updated weights for policy 1, policy_version 503730 (0.0009) [2023-12-26 19:01:47,754][105692] Updated weights for policy 0, policy_version 503388 (0.0011) [2023-12-26 19:01:47,812][105620] Updated weights for policy 1, policy_version 503740 (0.0009) [2023-12-26 19:01:47,863][105620] Updated weights for policy 1, policy_version 503750 (0.0008) [2023-12-26 19:01:48,384][105692] Updated weights for policy 0, policy_version 503398 (0.0008) [2023-12-26 19:01:48,439][105692] Updated weights for policy 0, policy_version 503408 (0.0006) [2023-12-26 19:01:48,491][105692] Updated weights for policy 0, policy_version 503418 (0.0008) [2023-12-26 19:01:48,636][105620] Updated weights for policy 1, policy_version 503760 (0.0006) [2023-12-26 19:01:48,691][105620] Updated weights for policy 1, policy_version 503770 (0.0005) [2023-12-26 19:01:48,752][105620] Updated weights for policy 1, policy_version 503780 (0.0005) [2023-12-26 19:01:49,098][105692] Updated weights for policy 0, policy_version 503428 (0.0007) [2023-12-26 19:01:49,166][105692] Updated weights for policy 0, policy_version 503438 (0.0006) [2023-12-26 19:01:49,224][105692] Updated weights for policy 0, policy_version 503448 (0.0006) [2023-12-26 19:01:49,422][105620] Updated weights for policy 1, policy_version 503790 (0.0008) [2023-12-26 19:01:49,484][105620] Updated weights for policy 1, policy_version 503800 (0.0008) [2023-12-26 19:01:49,549][105620] Updated weights for policy 1, policy_version 503810 (0.0008) [2023-12-26 19:01:49,952][105692] Updated weights for policy 0, policy_version 503458 (0.0009) [2023-12-26 19:01:50,008][105692] Updated weights for policy 0, policy_version 503468 (0.0011) [2023-12-26 19:01:50,066][105692] Updated weights for policy 0, policy_version 503478 (0.0010) [2023-12-26 19:01:50,126][105692] Updated weights for policy 0, policy_version 503488 (0.0008) [2023-12-26 19:01:50,260][105620] Updated weights for policy 1, policy_version 503820 (0.0008) [2023-12-26 19:01:50,310][105620] Updated weights for policy 1, policy_version 503830 (0.0007) [2023-12-26 19:01:50,359][105620] Updated weights for policy 1, policy_version 503840 (0.0005) [2023-12-26 19:01:50,835][105692] Updated weights for policy 0, policy_version 503498 (0.0010) [2023-12-26 19:01:50,884][105692] Updated weights for policy 0, policy_version 503508 (0.0010) [2023-12-26 19:01:50,941][105692] Updated weights for policy 0, policy_version 503518 (0.0006) [2023-12-26 19:01:50,973][105620] Updated weights for policy 1, policy_version 503850 (0.0006) [2023-12-26 19:01:51,041][105620] Updated weights for policy 1, policy_version 503860 (0.0007) [2023-12-26 19:01:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 257916928. Throughput: 0: 10111.2, 1: 9627.4. Samples: 257905520. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 19:01:51,062][104569] Avg episode reward: [(0, '6974.458'), (1, '9356.111')] [2023-12-26 19:01:51,104][105620] Updated weights for policy 1, policy_version 503870 (0.0006) [2023-12-26 19:01:51,177][105620] Updated weights for policy 1, policy_version 503880 (0.0007) [2023-12-26 19:01:51,612][105692] Updated weights for policy 0, policy_version 503528 (0.0007) [2023-12-26 19:01:51,670][105692] Updated weights for policy 0, policy_version 503538 (0.0009) [2023-12-26 19:01:51,735][105692] Updated weights for policy 0, policy_version 503548 (0.0009) [2023-12-26 19:01:51,938][105620] Updated weights for policy 1, policy_version 503890 (0.0007) [2023-12-26 19:01:51,992][105620] Updated weights for policy 1, policy_version 503900 (0.0006) [2023-12-26 19:01:52,052][105620] Updated weights for policy 1, policy_version 503910 (0.0008) [2023-12-26 19:01:52,449][105692] Updated weights for policy 0, policy_version 503558 (0.0008) [2023-12-26 19:01:52,505][105692] Updated weights for policy 0, policy_version 503568 (0.0010) [2023-12-26 19:01:52,566][105692] Updated weights for policy 0, policy_version 503578 (0.0010) [2023-12-26 19:01:52,744][105620] Updated weights for policy 1, policy_version 503920 (0.0010) [2023-12-26 19:01:52,796][105620] Updated weights for policy 1, policy_version 503930 (0.0011) [2023-12-26 19:01:52,852][105620] Updated weights for policy 1, policy_version 503940 (0.0010) [2023-12-26 19:01:53,327][105692] Updated weights for policy 0, policy_version 503588 (0.0010) [2023-12-26 19:01:53,383][105692] Updated weights for policy 0, policy_version 503598 (0.0010) [2023-12-26 19:01:53,443][105692] Updated weights for policy 0, policy_version 503608 (0.0010) [2023-12-26 19:01:53,610][105620] Updated weights for policy 1, policy_version 503950 (0.0010) [2023-12-26 19:01:53,668][105620] Updated weights for policy 1, policy_version 503960 (0.0010) [2023-12-26 19:01:53,725][105620] Updated weights for policy 1, policy_version 503970 (0.0010) [2023-12-26 19:01:54,121][105692] Updated weights for policy 0, policy_version 503618 (0.0010) [2023-12-26 19:01:54,173][105692] Updated weights for policy 0, policy_version 503628 (0.0007) [2023-12-26 19:01:54,231][105692] Updated weights for policy 0, policy_version 503638 (0.0006) [2023-12-26 19:01:54,285][105692] Updated weights for policy 0, policy_version 503648 (0.0006) [2023-12-26 19:01:54,377][105620] Updated weights for policy 1, policy_version 503980 (0.0008) [2023-12-26 19:01:54,436][105620] Updated weights for policy 1, policy_version 503990 (0.0006) [2023-12-26 19:01:54,490][105620] Updated weights for policy 1, policy_version 504000 (0.0006) [2023-12-26 19:01:54,920][105692] Updated weights for policy 0, policy_version 503658 (0.0010) [2023-12-26 19:01:54,973][105692] Updated weights for policy 0, policy_version 503668 (0.0010) [2023-12-26 19:01:55,026][105692] Updated weights for policy 0, policy_version 503678 (0.0008) [2023-12-26 19:01:55,163][105620] Updated weights for policy 1, policy_version 504010 (0.0010) [2023-12-26 19:01:55,220][105620] Updated weights for policy 1, policy_version 504020 (0.0008) [2023-12-26 19:01:55,280][105620] Updated weights for policy 1, policy_version 504030 (0.0009) [2023-12-26 19:01:55,326][105620] Updated weights for policy 1, policy_version 504040 (0.0008) [2023-12-26 19:01:55,724][105692] Updated weights for policy 0, policy_version 503688 (0.0008) [2023-12-26 19:01:55,753][105585] KL-divergence is very high: 101.1961 [2023-12-26 19:01:55,772][105692] Updated weights for policy 0, policy_version 503698 (0.0009) [2023-12-26 19:01:55,798][105585] KL-divergence is very high: 268.0103 [2023-12-26 19:01:55,839][105692] Updated weights for policy 0, policy_version 503708 (0.0010) [2023-12-26 19:01:55,851][105585] KL-divergence is very high: 306.0779 [2023-12-26 19:01:56,017][105620] Updated weights for policy 1, policy_version 504050 (0.0008) [2023-12-26 19:01:56,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 258015232. Throughput: 0: 10145.3, 1: 9712.4. Samples: 258024664. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 19:01:56,063][104569] Avg episode reward: [(0, '6931.259'), (1, '9356.185')] [2023-12-26 19:01:56,066][105620] Updated weights for policy 1, policy_version 504060 (0.0006) [2023-12-26 19:01:56,120][105620] Updated weights for policy 1, policy_version 504070 (0.0009) [2023-12-26 19:01:56,616][105692] Updated weights for policy 0, policy_version 503718 (0.0009) [2023-12-26 19:01:56,665][105692] Updated weights for policy 0, policy_version 503728 (0.0009) [2023-12-26 19:01:56,717][105692] Updated weights for policy 0, policy_version 503738 (0.0009) [2023-12-26 19:01:56,825][105620] Updated weights for policy 1, policy_version 504080 (0.0006) [2023-12-26 19:01:56,879][105620] Updated weights for policy 1, policy_version 504090 (0.0005) [2023-12-26 19:01:56,925][105620] Updated weights for policy 1, policy_version 504100 (0.0005) [2023-12-26 19:01:57,339][105692] Updated weights for policy 0, policy_version 503748 (0.0009) [2023-12-26 19:01:57,402][105692] Updated weights for policy 0, policy_version 503758 (0.0006) [2023-12-26 19:01:57,456][105692] Updated weights for policy 0, policy_version 503768 (0.0005) [2023-12-26 19:01:57,519][105620] Updated weights for policy 1, policy_version 504110 (0.0005) [2023-12-26 19:01:57,587][105620] Updated weights for policy 1, policy_version 504120 (0.0005) [2023-12-26 19:01:57,647][105620] Updated weights for policy 1, policy_version 504130 (0.0007) [2023-12-26 19:01:58,031][105692] Updated weights for policy 0, policy_version 503778 (0.0008) [2023-12-26 19:01:58,078][105692] Updated weights for policy 0, policy_version 503788 (0.0006) [2023-12-26 19:01:58,138][105692] Updated weights for policy 0, policy_version 503798 (0.0006) [2023-12-26 19:01:58,155][105620] Updated weights for policy 1, policy_version 504140 (0.0006) [2023-12-26 19:01:58,188][105692] Updated weights for policy 0, policy_version 503808 (0.0006) [2023-12-26 19:01:58,214][105620] Updated weights for policy 1, policy_version 504150 (0.0008) [2023-12-26 19:01:58,276][105620] Updated weights for policy 1, policy_version 504160 (0.0008) [2023-12-26 19:01:59,031][105692] Updated weights for policy 0, policy_version 503818 (0.0008) [2023-12-26 19:01:59,067][105620] Updated weights for policy 1, policy_version 504170 (0.0007) [2023-12-26 19:01:59,086][105692] Updated weights for policy 0, policy_version 503828 (0.0007) [2023-12-26 19:01:59,126][105620] Updated weights for policy 1, policy_version 504180 (0.0008) [2023-12-26 19:01:59,148][105692] Updated weights for policy 0, policy_version 503838 (0.0007) [2023-12-26 19:01:59,192][105620] Updated weights for policy 1, policy_version 504190 (0.0007) [2023-12-26 19:01:59,256][105620] Updated weights for policy 1, policy_version 504200 (0.0007) [2023-12-26 19:01:59,885][105620] Updated weights for policy 1, policy_version 504210 (0.0008) [2023-12-26 19:01:59,903][105692] Updated weights for policy 0, policy_version 503848 (0.0010) [2023-12-26 19:01:59,943][105620] Updated weights for policy 1, policy_version 504220 (0.0010) [2023-12-26 19:01:59,969][105692] Updated weights for policy 0, policy_version 503858 (0.0009) [2023-12-26 19:02:00,000][105620] Updated weights for policy 1, policy_version 504230 (0.0005) [2023-12-26 19:02:00,032][105692] Updated weights for policy 0, policy_version 503868 (0.0009) [2023-12-26 19:02:00,674][105620] Updated weights for policy 1, policy_version 504240 (0.0010) [2023-12-26 19:02:00,732][105620] Updated weights for policy 1, policy_version 504250 (0.0009) [2023-12-26 19:02:00,754][105692] Updated weights for policy 0, policy_version 503878 (0.0008) [2023-12-26 19:02:00,780][105620] Updated weights for policy 1, policy_version 504260 (0.0010) [2023-12-26 19:02:00,823][105692] Updated weights for policy 0, policy_version 503888 (0.0008) [2023-12-26 19:02:00,864][105585] KL-divergence is very high: 122.6073 [2023-12-26 19:02:00,890][105692] Updated weights for policy 0, policy_version 503898 (0.0010) [2023-12-26 19:02:00,910][105585] KL-divergence is very high: 120.8870 [2023-12-26 19:02:01,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19522.0). Total num frames: 258121728. Throughput: 0: 10192.2, 1: 9815.8. Samples: 258087912. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 19:02:01,062][104569] Avg episode reward: [(0, '8295.976'), (1, '9356.256')] [2023-12-26 19:02:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000503904_129015808.pth... [2023-12-26 19:02:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000504264_129105920.pth... [2023-12-26 19:02:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000502720_128712704.pth [2023-12-26 19:02:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000503112_128811008.pth [2023-12-26 19:02:01,483][105620] Updated weights for policy 1, policy_version 504270 (0.0006) [2023-12-26 19:02:01,536][105620] Updated weights for policy 1, policy_version 504280 (0.0006) [2023-12-26 19:02:01,592][105620] Updated weights for policy 1, policy_version 504290 (0.0005) [2023-12-26 19:02:01,699][105692] Updated weights for policy 0, policy_version 503908 (0.0009) [2023-12-26 19:02:01,765][105692] Updated weights for policy 0, policy_version 503918 (0.0008) [2023-12-26 19:02:01,831][105692] Updated weights for policy 0, policy_version 503928 (0.0008) [2023-12-26 19:02:02,293][105620] Updated weights for policy 1, policy_version 504300 (0.0007) [2023-12-26 19:02:02,345][105620] Updated weights for policy 1, policy_version 504310 (0.0005) [2023-12-26 19:02:02,410][105620] Updated weights for policy 1, policy_version 504320 (0.0008) [2023-12-26 19:02:02,615][105692] Updated weights for policy 0, policy_version 503938 (0.0008) [2023-12-26 19:02:02,661][105692] Updated weights for policy 0, policy_version 503948 (0.0008) [2023-12-26 19:02:02,711][105692] Updated weights for policy 0, policy_version 503958 (0.0009) [2023-12-26 19:02:02,758][105692] Updated weights for policy 0, policy_version 503968 (0.0008) [2023-12-26 19:02:03,044][105620] Updated weights for policy 1, policy_version 504330 (0.0005) [2023-12-26 19:02:03,095][105620] Updated weights for policy 1, policy_version 504340 (0.0006) [2023-12-26 19:02:03,161][105620] Updated weights for policy 1, policy_version 504350 (0.0009) [2023-12-26 19:02:03,215][105620] Updated weights for policy 1, policy_version 504360 (0.0008) [2023-12-26 19:02:03,478][105692] Updated weights for policy 0, policy_version 503978 (0.0005) [2023-12-26 19:02:03,524][105585] KL-divergence is very high: 103.5671 [2023-12-26 19:02:03,530][105692] Updated weights for policy 0, policy_version 503988 (0.0005) [2023-12-26 19:02:03,566][105585] KL-divergence is very high: 110.5630 [2023-12-26 19:02:03,583][105692] Updated weights for policy 0, policy_version 503998 (0.0005) [2023-12-26 19:02:03,927][105620] Updated weights for policy 1, policy_version 504370 (0.0010) [2023-12-26 19:02:03,986][105620] Updated weights for policy 1, policy_version 504380 (0.0011) [2023-12-26 19:02:04,045][105620] Updated weights for policy 1, policy_version 504390 (0.0010) [2023-12-26 19:02:04,247][105692] Updated weights for policy 0, policy_version 504008 (0.0007) [2023-12-26 19:02:04,301][105692] Updated weights for policy 0, policy_version 504018 (0.0008) [2023-12-26 19:02:04,353][105692] Updated weights for policy 0, policy_version 504028 (0.0008) [2023-12-26 19:02:04,756][105620] Updated weights for policy 1, policy_version 504400 (0.0010) [2023-12-26 19:02:04,810][105620] Updated weights for policy 1, policy_version 504410 (0.0009) [2023-12-26 19:02:04,867][105620] Updated weights for policy 1, policy_version 504420 (0.0008) [2023-12-26 19:02:05,071][105692] Updated weights for policy 0, policy_version 504038 (0.0006) [2023-12-26 19:02:05,134][105692] Updated weights for policy 0, policy_version 504048 (0.0006) [2023-12-26 19:02:05,188][105692] Updated weights for policy 0, policy_version 504058 (0.0009) [2023-12-26 19:02:05,628][105620] Updated weights for policy 1, policy_version 504430 (0.0008) [2023-12-26 19:02:05,682][105620] Updated weights for policy 1, policy_version 504440 (0.0009) [2023-12-26 19:02:05,728][105620] Updated weights for policy 1, policy_version 504450 (0.0008) [2023-12-26 19:02:05,886][105692] Updated weights for policy 0, policy_version 504068 (0.0009) [2023-12-26 19:02:05,950][105692] Updated weights for policy 0, policy_version 504078 (0.0008) [2023-12-26 19:02:06,001][105692] Updated weights for policy 0, policy_version 504088 (0.0009) [2023-12-26 19:02:06,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19933.9, 300 sec: 19522.0). Total num frames: 258220032. Throughput: 0: 10021.4, 1: 9858.3. Samples: 258204768. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 19:02:06,063][104569] Avg episode reward: [(0, '8111.527'), (1, '9175.025')] [2023-12-26 19:02:06,542][105620] Updated weights for policy 1, policy_version 504460 (0.0009) [2023-12-26 19:02:06,605][105620] Updated weights for policy 1, policy_version 504470 (0.0008) [2023-12-26 19:02:06,666][105620] Updated weights for policy 1, policy_version 504480 (0.0009) [2023-12-26 19:02:06,755][105692] Updated weights for policy 0, policy_version 504098 (0.0009) [2023-12-26 19:02:06,815][105692] Updated weights for policy 0, policy_version 504108 (0.0008) [2023-12-26 19:02:06,877][105692] Updated weights for policy 0, policy_version 504118 (0.0009) [2023-12-26 19:02:06,937][105692] Updated weights for policy 0, policy_version 504128 (0.0009) [2023-12-26 19:02:07,338][105620] Updated weights for policy 1, policy_version 504490 (0.0009) [2023-12-26 19:02:07,387][105620] Updated weights for policy 1, policy_version 504500 (0.0008) [2023-12-26 19:02:07,447][105620] Updated weights for policy 1, policy_version 504510 (0.0008) [2023-12-26 19:02:07,501][105620] Updated weights for policy 1, policy_version 504520 (0.0010) [2023-12-26 19:02:07,659][105692] Updated weights for policy 0, policy_version 504138 (0.0010) [2023-12-26 19:02:07,716][105692] Updated weights for policy 0, policy_version 504148 (0.0010) [2023-12-26 19:02:07,770][105692] Updated weights for policy 0, policy_version 504158 (0.0010) [2023-12-26 19:02:08,271][105620] Updated weights for policy 1, policy_version 504530 (0.0008) [2023-12-26 19:02:08,323][105620] Updated weights for policy 1, policy_version 504540 (0.0008) [2023-12-26 19:02:08,393][105620] Updated weights for policy 1, policy_version 504550 (0.0008) [2023-12-26 19:02:08,518][105692] Updated weights for policy 0, policy_version 504168 (0.0006) [2023-12-26 19:02:08,583][105692] Updated weights for policy 0, policy_version 504178 (0.0005) [2023-12-26 19:02:08,642][105692] Updated weights for policy 0, policy_version 504188 (0.0005) [2023-12-26 19:02:09,147][105620] Updated weights for policy 1, policy_version 504560 (0.0009) [2023-12-26 19:02:09,210][105620] Updated weights for policy 1, policy_version 504570 (0.0009) [2023-12-26 19:02:09,246][105692] Updated weights for policy 0, policy_version 504198 (0.0009) [2023-12-26 19:02:09,278][105620] Updated weights for policy 1, policy_version 504580 (0.0008) [2023-12-26 19:02:09,311][105692] Updated weights for policy 0, policy_version 504208 (0.0010) [2023-12-26 19:02:09,379][105692] Updated weights for policy 0, policy_version 504218 (0.0009) [2023-12-26 19:02:10,060][105620] Updated weights for policy 1, policy_version 504590 (0.0009) [2023-12-26 19:02:10,107][105692] Updated weights for policy 0, policy_version 504228 (0.0008) [2023-12-26 19:02:10,123][105620] Updated weights for policy 1, policy_version 504600 (0.0006) [2023-12-26 19:02:10,171][105692] Updated weights for policy 0, policy_version 504238 (0.0006) [2023-12-26 19:02:10,173][105620] Updated weights for policy 1, policy_version 504610 (0.0008) [2023-12-26 19:02:10,233][105692] Updated weights for policy 0, policy_version 504248 (0.0008) [2023-12-26 19:02:10,807][105620] Updated weights for policy 1, policy_version 504620 (0.0006) [2023-12-26 19:02:10,860][105620] Updated weights for policy 1, policy_version 504630 (0.0006) [2023-12-26 19:02:10,863][105692] Updated weights for policy 0, policy_version 504258 (0.0008) [2023-12-26 19:02:10,914][105620] Updated weights for policy 1, policy_version 504640 (0.0006) [2023-12-26 19:02:10,927][105692] Updated weights for policy 0, policy_version 504268 (0.0011) [2023-12-26 19:02:10,986][105692] Updated weights for policy 0, policy_version 504278 (0.0010) [2023-12-26 19:02:11,049][105692] Updated weights for policy 0, policy_version 504288 (0.0008) [2023-12-26 19:02:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19933.9, 300 sec: 19522.0). Total num frames: 258318336. Throughput: 0: 9987.2, 1: 9872.3. Samples: 258321668. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 19:02:11,062][104569] Avg episode reward: [(0, '8633.318'), (1, '9175.128')] [2023-12-26 19:02:11,598][105620] Updated weights for policy 1, policy_version 504650 (0.0007) [2023-12-26 19:02:11,665][105620] Updated weights for policy 1, policy_version 504660 (0.0009) [2023-12-26 19:02:11,730][105620] Updated weights for policy 1, policy_version 504670 (0.0008) [2023-12-26 19:02:11,781][105692] Updated weights for policy 0, policy_version 504298 (0.0011) [2023-12-26 19:02:11,792][105620] Updated weights for policy 1, policy_version 504680 (0.0008) [2023-12-26 19:02:11,837][105692] Updated weights for policy 0, policy_version 504308 (0.0010) [2023-12-26 19:02:11,890][105692] Updated weights for policy 0, policy_version 504318 (0.0011) [2023-12-26 19:02:12,563][105620] Updated weights for policy 1, policy_version 504690 (0.0007) [2023-12-26 19:02:12,594][105692] Updated weights for policy 0, policy_version 504328 (0.0010) [2023-12-26 19:02:12,625][105620] Updated weights for policy 1, policy_version 504700 (0.0006) [2023-12-26 19:02:12,653][105692] Updated weights for policy 0, policy_version 504338 (0.0011) [2023-12-26 19:02:12,680][105620] Updated weights for policy 1, policy_version 504710 (0.0007) [2023-12-26 19:02:12,707][105692] Updated weights for policy 0, policy_version 504348 (0.0009) [2023-12-26 19:02:13,270][105620] Updated weights for policy 1, policy_version 504720 (0.0006) [2023-12-26 19:02:13,326][105620] Updated weights for policy 1, policy_version 504730 (0.0005) [2023-12-26 19:02:13,340][105692] Updated weights for policy 0, policy_version 504358 (0.0008) [2023-12-26 19:02:13,382][105620] Updated weights for policy 1, policy_version 504740 (0.0006) [2023-12-26 19:02:13,403][105692] Updated weights for policy 0, policy_version 504368 (0.0008) [2023-12-26 19:02:13,455][105692] Updated weights for policy 0, policy_version 504378 (0.0009) [2023-12-26 19:02:14,030][105620] Updated weights for policy 1, policy_version 504750 (0.0008) [2023-12-26 19:02:14,091][105620] Updated weights for policy 1, policy_version 504760 (0.0006) [2023-12-26 19:02:14,150][105620] Updated weights for policy 1, policy_version 504770 (0.0006) [2023-12-26 19:02:14,196][105692] Updated weights for policy 0, policy_version 504388 (0.0010) [2023-12-26 19:02:14,248][105692] Updated weights for policy 0, policy_version 504398 (0.0010) [2023-12-26 19:02:14,303][105692] Updated weights for policy 0, policy_version 504408 (0.0010) [2023-12-26 19:02:14,822][105620] Updated weights for policy 1, policy_version 504780 (0.0008) [2023-12-26 19:02:14,888][105620] Updated weights for policy 1, policy_version 504790 (0.0008) [2023-12-26 19:02:14,946][105620] Updated weights for policy 1, policy_version 504800 (0.0010) [2023-12-26 19:02:14,979][105692] Updated weights for policy 0, policy_version 504418 (0.0010) [2023-12-26 19:02:15,039][105692] Updated weights for policy 0, policy_version 504428 (0.0011) [2023-12-26 19:02:15,109][105692] Updated weights for policy 0, policy_version 504438 (0.0011) [2023-12-26 19:02:15,175][105692] Updated weights for policy 0, policy_version 504448 (0.0011) [2023-12-26 19:02:15,624][105620] Updated weights for policy 1, policy_version 504810 (0.0010) [2023-12-26 19:02:15,682][105620] Updated weights for policy 1, policy_version 504820 (0.0008) [2023-12-26 19:02:15,745][105620] Updated weights for policy 1, policy_version 504830 (0.0008) [2023-12-26 19:02:15,788][105620] Updated weights for policy 1, policy_version 504840 (0.0008) [2023-12-26 19:02:15,902][105692] Updated weights for policy 0, policy_version 504458 (0.0005) [2023-12-26 19:02:15,954][105692] Updated weights for policy 0, policy_version 504468 (0.0008) [2023-12-26 19:02:16,002][105692] Updated weights for policy 0, policy_version 504478 (0.0010) [2023-12-26 19:02:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19933.8, 300 sec: 19522.0). Total num frames: 258416640. Throughput: 0: 9919.0, 1: 9883.8. Samples: 258381432. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 19:02:16,063][104569] Avg episode reward: [(0, '9084.115'), (1, '9356.537')] [2023-12-26 19:02:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000504840_129253376.pth... [2023-12-26 19:02:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000504480_129163264.pth... [2023-12-26 19:02:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000503656_128950272.pth [2023-12-26 19:02:16,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000503328_128868352.pth [2023-12-26 19:02:16,558][105620] Updated weights for policy 1, policy_version 504850 (0.0007) [2023-12-26 19:02:16,613][105620] Updated weights for policy 1, policy_version 504860 (0.0008) [2023-12-26 19:02:16,677][105620] Updated weights for policy 1, policy_version 504870 (0.0009) [2023-12-26 19:02:16,718][105692] Updated weights for policy 0, policy_version 504488 (0.0011) [2023-12-26 19:02:16,770][105692] Updated weights for policy 0, policy_version 504498 (0.0010) [2023-12-26 19:02:16,825][105692] Updated weights for policy 0, policy_version 504508 (0.0011) [2023-12-26 19:02:17,423][105620] Updated weights for policy 1, policy_version 504880 (0.0008) [2023-12-26 19:02:17,483][105620] Updated weights for policy 1, policy_version 504890 (0.0008) [2023-12-26 19:02:17,547][105620] Updated weights for policy 1, policy_version 504900 (0.0008) [2023-12-26 19:02:17,585][105692] Updated weights for policy 0, policy_version 504518 (0.0010) [2023-12-26 19:02:17,640][105692] Updated weights for policy 0, policy_version 504528 (0.0010) [2023-12-26 19:02:17,691][105692] Updated weights for policy 0, policy_version 504538 (0.0010) [2023-12-26 19:02:18,192][105620] Updated weights for policy 1, policy_version 504910 (0.0005) [2023-12-26 19:02:18,237][105620] Updated weights for policy 1, policy_version 504920 (0.0005) [2023-12-26 19:02:18,298][105620] Updated weights for policy 1, policy_version 504930 (0.0007) [2023-12-26 19:02:18,463][105692] Updated weights for policy 0, policy_version 504548 (0.0010) [2023-12-26 19:02:18,522][105692] Updated weights for policy 0, policy_version 504558 (0.0011) [2023-12-26 19:02:18,527][105585] KL-divergence is very high: 247.8189 [2023-12-26 19:02:18,573][105585] KL-divergence is very high: 312.9618 [2023-12-26 19:02:18,578][105692] Updated weights for policy 0, policy_version 504568 (0.0011) [2023-12-26 19:02:18,615][105585] KL-divergence is very high: 199.0986 [2023-12-26 19:02:19,015][105620] Updated weights for policy 1, policy_version 504940 (0.0011) [2023-12-26 19:02:19,059][105620] Updated weights for policy 1, policy_version 504950 (0.0010) [2023-12-26 19:02:19,122][105620] Updated weights for policy 1, policy_version 504960 (0.0010) [2023-12-26 19:02:19,319][105692] Updated weights for policy 0, policy_version 504578 (0.0011) [2023-12-26 19:02:19,380][105692] Updated weights for policy 0, policy_version 504588 (0.0010) [2023-12-26 19:02:19,447][105692] Updated weights for policy 0, policy_version 504598 (0.0011) [2023-12-26 19:02:19,509][105692] Updated weights for policy 0, policy_version 504608 (0.0011) [2023-12-26 19:02:19,871][105620] Updated weights for policy 1, policy_version 504970 (0.0010) [2023-12-26 19:02:19,938][105620] Updated weights for policy 1, policy_version 504980 (0.0009) [2023-12-26 19:02:19,996][105620] Updated weights for policy 1, policy_version 504990 (0.0009) [2023-12-26 19:02:20,055][105620] Updated weights for policy 1, policy_version 505000 (0.0010) [2023-12-26 19:02:20,204][105692] Updated weights for policy 0, policy_version 504618 (0.0005) [2023-12-26 19:02:20,250][105692] Updated weights for policy 0, policy_version 504628 (0.0010) [2023-12-26 19:02:20,306][105692] Updated weights for policy 0, policy_version 504638 (0.0009) [2023-12-26 19:02:20,847][105620] Updated weights for policy 1, policy_version 505010 (0.0005) [2023-12-26 19:02:20,902][105620] Updated weights for policy 1, policy_version 505020 (0.0005) [2023-12-26 19:02:20,962][105620] Updated weights for policy 1, policy_version 505030 (0.0006) [2023-12-26 19:02:20,989][105692] Updated weights for policy 0, policy_version 504648 (0.0010) [2023-12-26 19:02:21,050][105692] Updated weights for policy 0, policy_version 504658 (0.0010) [2023-12-26 19:02:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 258506752. Throughput: 0: 9851.6, 1: 9902.6. Samples: 258497240. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 19:02:21,062][104569] Avg episode reward: [(0, '8991.004'), (1, '9266.813')] [2023-12-26 19:02:21,098][105692] Updated weights for policy 0, policy_version 504668 (0.0011) [2023-12-26 19:02:21,657][105620] Updated weights for policy 1, policy_version 505040 (0.0009) [2023-12-26 19:02:21,714][105620] Updated weights for policy 1, policy_version 505050 (0.0011) [2023-12-26 19:02:21,771][105620] Updated weights for policy 1, policy_version 505060 (0.0011) [2023-12-26 19:02:21,841][105692] Updated weights for policy 0, policy_version 504678 (0.0007) [2023-12-26 19:02:21,895][105692] Updated weights for policy 0, policy_version 504688 (0.0005) [2023-12-26 19:02:21,960][105692] Updated weights for policy 0, policy_version 504698 (0.0006) [2023-12-26 19:02:22,522][105620] Updated weights for policy 1, policy_version 505070 (0.0011) [2023-12-26 19:02:22,582][105620] Updated weights for policy 1, policy_version 505080 (0.0011) [2023-12-26 19:02:22,612][105692] Updated weights for policy 0, policy_version 504708 (0.0007) [2023-12-26 19:02:22,635][105620] Updated weights for policy 1, policy_version 505090 (0.0011) [2023-12-26 19:02:22,672][105692] Updated weights for policy 0, policy_version 504718 (0.0010) [2023-12-26 19:02:22,740][105692] Updated weights for policy 0, policy_version 504728 (0.0011) [2023-12-26 19:02:23,325][105692] Updated weights for policy 0, policy_version 504738 (0.0007) [2023-12-26 19:02:23,369][105620] Updated weights for policy 1, policy_version 505100 (0.0011) [2023-12-26 19:02:23,379][105692] Updated weights for policy 0, policy_version 504748 (0.0005) [2023-12-26 19:02:23,422][105620] Updated weights for policy 1, policy_version 505110 (0.0011) [2023-12-26 19:02:23,427][105692] Updated weights for policy 0, policy_version 504758 (0.0005) [2023-12-26 19:02:23,474][105620] Updated weights for policy 1, policy_version 505120 (0.0011) [2023-12-26 19:02:23,476][105692] Updated weights for policy 0, policy_version 504768 (0.0005) [2023-12-26 19:02:24,113][105692] Updated weights for policy 0, policy_version 504778 (0.0009) [2023-12-26 19:02:24,168][105620] Updated weights for policy 1, policy_version 505130 (0.0009) [2023-12-26 19:02:24,171][105692] Updated weights for policy 0, policy_version 504788 (0.0007) [2023-12-26 19:02:24,216][105620] Updated weights for policy 1, policy_version 505140 (0.0005) [2023-12-26 19:02:24,222][105692] Updated weights for policy 0, policy_version 504798 (0.0011) [2023-12-26 19:02:24,259][105620] Updated weights for policy 1, policy_version 505150 (0.0007) [2023-12-26 19:02:24,308][105620] Updated weights for policy 1, policy_version 505160 (0.0008) [2023-12-26 19:02:24,825][105692] Updated weights for policy 0, policy_version 504808 (0.0010) [2023-12-26 19:02:24,873][105692] Updated weights for policy 0, policy_version 504818 (0.0010) [2023-12-26 19:02:24,921][105692] Updated weights for policy 0, policy_version 504828 (0.0010) [2023-12-26 19:02:25,077][105620] Updated weights for policy 1, policy_version 505170 (0.0005) [2023-12-26 19:02:25,142][105620] Updated weights for policy 1, policy_version 505180 (0.0006) [2023-12-26 19:02:25,194][105620] Updated weights for policy 1, policy_version 505190 (0.0006) [2023-12-26 19:02:25,685][105692] Updated weights for policy 0, policy_version 504838 (0.0010) [2023-12-26 19:02:25,743][105692] Updated weights for policy 0, policy_version 504848 (0.0010) [2023-12-26 19:02:25,768][105620] Updated weights for policy 1, policy_version 505200 (0.0005) [2023-12-26 19:02:25,791][105692] Updated weights for policy 0, policy_version 504858 (0.0010) [2023-12-26 19:02:25,826][105620] Updated weights for policy 1, policy_version 505210 (0.0005) [2023-12-26 19:02:25,877][105620] Updated weights for policy 1, policy_version 505220 (0.0006) [2023-12-26 19:02:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19933.9, 300 sec: 19549.7). Total num frames: 258613248. Throughput: 0: 9976.4, 1: 9839.3. Samples: 258618856. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 19:02:26,062][104569] Avg episode reward: [(0, '9083.454'), (1, '9266.848')] [2023-12-26 19:02:26,376][105692] Updated weights for policy 0, policy_version 504868 (0.0009) [2023-12-26 19:02:26,443][105692] Updated weights for policy 0, policy_version 504878 (0.0009) [2023-12-26 19:02:26,501][105692] Updated weights for policy 0, policy_version 504888 (0.0010) [2023-12-26 19:02:26,601][105620] Updated weights for policy 1, policy_version 505230 (0.0006) [2023-12-26 19:02:26,652][105620] Updated weights for policy 1, policy_version 505240 (0.0009) [2023-12-26 19:02:26,714][105620] Updated weights for policy 1, policy_version 505250 (0.0010) [2023-12-26 19:02:27,192][105692] Updated weights for policy 0, policy_version 504898 (0.0009) [2023-12-26 19:02:27,253][105692] Updated weights for policy 0, policy_version 504908 (0.0008) [2023-12-26 19:02:27,305][105692] Updated weights for policy 0, policy_version 504918 (0.0010) [2023-12-26 19:02:27,359][105620] Updated weights for policy 1, policy_version 505260 (0.0010) [2023-12-26 19:02:27,360][105692] Updated weights for policy 0, policy_version 504928 (0.0010) [2023-12-26 19:02:27,420][105620] Updated weights for policy 1, policy_version 505270 (0.0010) [2023-12-26 19:02:27,470][105620] Updated weights for policy 1, policy_version 505280 (0.0010) [2023-12-26 19:02:28,087][105692] Updated weights for policy 0, policy_version 504938 (0.0010) [2023-12-26 19:02:28,148][105692] Updated weights for policy 0, policy_version 504948 (0.0010) [2023-12-26 19:02:28,203][105692] Updated weights for policy 0, policy_version 504958 (0.0006) [2023-12-26 19:02:28,216][105620] Updated weights for policy 1, policy_version 505290 (0.0010) [2023-12-26 19:02:28,262][105620] Updated weights for policy 1, policy_version 505300 (0.0007) [2023-12-26 19:02:28,309][105620] Updated weights for policy 1, policy_version 505310 (0.0008) [2023-12-26 19:02:28,376][105620] Updated weights for policy 1, policy_version 505320 (0.0008) [2023-12-26 19:02:28,934][105692] Updated weights for policy 0, policy_version 504968 (0.0007) [2023-12-26 19:02:28,991][105692] Updated weights for policy 0, policy_version 504978 (0.0010) [2023-12-26 19:02:29,055][105692] Updated weights for policy 0, policy_version 504988 (0.0009) [2023-12-26 19:02:29,113][105620] Updated weights for policy 1, policy_version 505330 (0.0005) [2023-12-26 19:02:29,173][105620] Updated weights for policy 1, policy_version 505340 (0.0005) [2023-12-26 19:02:29,222][105620] Updated weights for policy 1, policy_version 505350 (0.0006) [2023-12-26 19:02:29,701][105692] Updated weights for policy 0, policy_version 504998 (0.0008) [2023-12-26 19:02:29,749][105692] Updated weights for policy 0, policy_version 505008 (0.0010) [2023-12-26 19:02:29,804][105692] Updated weights for policy 0, policy_version 505018 (0.0010) [2023-12-26 19:02:29,868][105620] Updated weights for policy 1, policy_version 505360 (0.0008) [2023-12-26 19:02:29,930][105620] Updated weights for policy 1, policy_version 505370 (0.0009) [2023-12-26 19:02:29,987][105620] Updated weights for policy 1, policy_version 505380 (0.0008) [2023-12-26 19:02:30,580][105692] Updated weights for policy 0, policy_version 505028 (0.0011) [2023-12-26 19:02:30,635][105692] Updated weights for policy 0, policy_version 505038 (0.0011) [2023-12-26 19:02:30,645][105620] Updated weights for policy 1, policy_version 505390 (0.0007) [2023-12-26 19:02:30,691][105620] Updated weights for policy 1, policy_version 505400 (0.0005) [2023-12-26 19:02:30,697][105692] Updated weights for policy 0, policy_version 505048 (0.0011) [2023-12-26 19:02:30,747][105620] Updated weights for policy 1, policy_version 505410 (0.0005) [2023-12-26 19:02:31,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19933.8, 300 sec: 19549.7). Total num frames: 258711552. Throughput: 0: 10018.5, 1: 9881.3. Samples: 258678312. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 19:02:31,063][104569] Avg episode reward: [(0, '8813.830'), (1, '9265.145')] [2023-12-26 19:02:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000505056_129310720.pth... [2023-12-26 19:02:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000505416_129400832.pth... [2023-12-26 19:02:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000503904_129015808.pth [2023-12-26 19:02:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000504264_129105920.pth [2023-12-26 19:02:31,461][105692] Updated weights for policy 0, policy_version 505058 (0.0010) [2023-12-26 19:02:31,514][105692] Updated weights for policy 0, policy_version 505068 (0.0010) [2023-12-26 19:02:31,534][105620] Updated weights for policy 1, policy_version 505420 (0.0009) [2023-12-26 19:02:31,570][105692] Updated weights for policy 0, policy_version 505078 (0.0010) [2023-12-26 19:02:31,587][105620] Updated weights for policy 1, policy_version 505430 (0.0011) [2023-12-26 19:02:31,623][105692] Updated weights for policy 0, policy_version 505088 (0.0009) [2023-12-26 19:02:31,647][105620] Updated weights for policy 1, policy_version 505440 (0.0008) [2023-12-26 19:02:32,395][105620] Updated weights for policy 1, policy_version 505450 (0.0009) [2023-12-26 19:02:32,402][105692] Updated weights for policy 0, policy_version 505098 (0.0008) [2023-12-26 19:02:32,452][105692] Updated weights for policy 0, policy_version 505108 (0.0008) [2023-12-26 19:02:32,458][105620] Updated weights for policy 1, policy_version 505460 (0.0011) [2023-12-26 19:02:32,507][105692] Updated weights for policy 0, policy_version 505118 (0.0008) [2023-12-26 19:02:32,518][105620] Updated weights for policy 1, policy_version 505470 (0.0011) [2023-12-26 19:02:32,572][105620] Updated weights for policy 1, policy_version 505480 (0.0011) [2023-12-26 19:02:33,296][105692] Updated weights for policy 0, policy_version 505128 (0.0008) [2023-12-26 19:02:33,335][105620] Updated weights for policy 1, policy_version 505490 (0.0007) [2023-12-26 19:02:33,356][105692] Updated weights for policy 0, policy_version 505138 (0.0007) [2023-12-26 19:02:33,405][105620] Updated weights for policy 1, policy_version 505500 (0.0008) [2023-12-26 19:02:33,419][105692] Updated weights for policy 0, policy_version 505148 (0.0008) [2023-12-26 19:02:33,470][105620] Updated weights for policy 1, policy_version 505510 (0.0007) [2023-12-26 19:02:34,124][105692] Updated weights for policy 0, policy_version 505158 (0.0009) [2023-12-26 19:02:34,181][105620] Updated weights for policy 1, policy_version 505520 (0.0008) [2023-12-26 19:02:34,187][105692] Updated weights for policy 0, policy_version 505168 (0.0008) [2023-12-26 19:02:34,236][105620] Updated weights for policy 1, policy_version 505530 (0.0009) [2023-12-26 19:02:34,246][105692] Updated weights for policy 0, policy_version 505178 (0.0006) [2023-12-26 19:02:34,295][105620] Updated weights for policy 1, policy_version 505540 (0.0007) [2023-12-26 19:02:34,973][105620] Updated weights for policy 1, policy_version 505550 (0.0007) [2023-12-26 19:02:35,034][105620] Updated weights for policy 1, policy_version 505560 (0.0006) [2023-12-26 19:02:35,062][105692] Updated weights for policy 0, policy_version 505188 (0.0007) [2023-12-26 19:02:35,093][105620] Updated weights for policy 1, policy_version 505570 (0.0005) [2023-12-26 19:02:35,127][105692] Updated weights for policy 0, policy_version 505198 (0.0007) [2023-12-26 19:02:35,177][105692] Updated weights for policy 0, policy_version 505208 (0.0008) [2023-12-26 19:02:35,681][105620] Updated weights for policy 1, policy_version 505580 (0.0007) [2023-12-26 19:02:35,735][105620] Updated weights for policy 1, policy_version 505590 (0.0009) [2023-12-26 19:02:35,785][105620] Updated weights for policy 1, policy_version 505600 (0.0008) [2023-12-26 19:02:36,010][105692] Updated weights for policy 0, policy_version 505218 (0.0009) [2023-12-26 19:02:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.4, 300 sec: 19522.0). Total num frames: 258801664. Throughput: 0: 9844.2, 1: 9894.1. Samples: 258793740. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-26 19:02:36,062][104569] Avg episode reward: [(0, '8568.086'), (1, '9265.208')] [2023-12-26 19:02:36,065][105692] Updated weights for policy 0, policy_version 505229 (0.0010) [2023-12-26 19:02:36,119][105692] Updated weights for policy 0, policy_version 505239 (0.0008) [2023-12-26 19:02:36,401][105620] Updated weights for policy 1, policy_version 505610 (0.0008) [2023-12-26 19:02:36,462][105620] Updated weights for policy 1, policy_version 505620 (0.0006) [2023-12-26 19:02:36,517][105620] Updated weights for policy 1, policy_version 505630 (0.0010) [2023-12-26 19:02:36,570][105620] Updated weights for policy 1, policy_version 505640 (0.0008) [2023-12-26 19:02:36,999][105692] Updated weights for policy 0, policy_version 505249 (0.0008) [2023-12-26 19:02:37,060][105692] Updated weights for policy 0, policy_version 505259 (0.0008) [2023-12-26 19:02:37,114][105692] Updated weights for policy 0, policy_version 505269 (0.0010) [2023-12-26 19:02:37,162][105692] Updated weights for policy 0, policy_version 505279 (0.0008) [2023-12-26 19:02:37,218][105620] Updated weights for policy 1, policy_version 505650 (0.0010) [2023-12-26 19:02:37,283][105620] Updated weights for policy 1, policy_version 505660 (0.0010) [2023-12-26 19:02:37,331][105620] Updated weights for policy 1, policy_version 505670 (0.0010) [2023-12-26 19:02:37,975][105692] Updated weights for policy 0, policy_version 505289 (0.0009) [2023-12-26 19:02:38,029][105620] Updated weights for policy 1, policy_version 505680 (0.0010) [2023-12-26 19:02:38,031][105692] Updated weights for policy 0, policy_version 505299 (0.0007) [2023-12-26 19:02:38,078][105620] Updated weights for policy 1, policy_version 505690 (0.0010) [2023-12-26 19:02:38,080][105692] Updated weights for policy 0, policy_version 505309 (0.0005) [2023-12-26 19:02:38,130][105620] Updated weights for policy 1, policy_version 505700 (0.0010) [2023-12-26 19:02:38,812][105620] Updated weights for policy 1, policy_version 505710 (0.0010) [2023-12-26 19:02:38,873][105620] Updated weights for policy 1, policy_version 505720 (0.0010) [2023-12-26 19:02:38,903][105692] Updated weights for policy 0, policy_version 505319 (0.0005) [2023-12-26 19:02:38,933][105620] Updated weights for policy 1, policy_version 505730 (0.0010) [2023-12-26 19:02:38,952][105692] Updated weights for policy 0, policy_version 505329 (0.0006) [2023-12-26 19:02:39,001][105692] Updated weights for policy 0, policy_version 505339 (0.0007) [2023-12-26 19:02:39,631][105620] Updated weights for policy 1, policy_version 505740 (0.0010) [2023-12-26 19:02:39,688][105620] Updated weights for policy 1, policy_version 505750 (0.0006) [2023-12-26 19:02:39,752][105620] Updated weights for policy 1, policy_version 505760 (0.0006) [2023-12-26 19:02:39,846][105692] Updated weights for policy 0, policy_version 505349 (0.0009) [2023-12-26 19:02:39,895][105692] Updated weights for policy 0, policy_version 505359 (0.0010) [2023-12-26 19:02:39,955][105692] Updated weights for policy 0, policy_version 505369 (0.0009) [2023-12-26 19:02:40,448][105620] Updated weights for policy 1, policy_version 505770 (0.0007) [2023-12-26 19:02:40,506][105620] Updated weights for policy 1, policy_version 505780 (0.0009) [2023-12-26 19:02:40,561][105620] Updated weights for policy 1, policy_version 505790 (0.0009) [2023-12-26 19:02:40,617][105620] Updated weights for policy 1, policy_version 505800 (0.0009) [2023-12-26 19:02:40,740][105692] Updated weights for policy 0, policy_version 505379 (0.0007) [2023-12-26 19:02:40,799][105692] Updated weights for policy 0, policy_version 505389 (0.0009) [2023-12-26 19:02:40,860][105692] Updated weights for policy 0, policy_version 505399 (0.0010) [2023-12-26 19:02:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 258899968. Throughput: 0: 9668.5, 1: 9953.5. Samples: 258907656. Policy #0 lag: (min: 26.0, avg: 43.7, max: 58.0) [2023-12-26 19:02:41,062][104569] Avg episode reward: [(0, '9092.252'), (1, '9356.897')] [2023-12-26 19:02:41,341][105620] Updated weights for policy 1, policy_version 505810 (0.0008) [2023-12-26 19:02:41,410][105620] Updated weights for policy 1, policy_version 505820 (0.0008) [2023-12-26 19:02:41,471][105620] Updated weights for policy 1, policy_version 505830 (0.0008) [2023-12-26 19:02:41,674][105692] Updated weights for policy 0, policy_version 505409 (0.0010) [2023-12-26 19:02:41,745][105692] Updated weights for policy 0, policy_version 505419 (0.0008) [2023-12-26 19:02:41,810][105692] Updated weights for policy 0, policy_version 505429 (0.0008) [2023-12-26 19:02:41,869][105692] Updated weights for policy 0, policy_version 505439 (0.0008) [2023-12-26 19:02:42,194][105620] Updated weights for policy 1, policy_version 505840 (0.0009) [2023-12-26 19:02:42,257][105620] Updated weights for policy 1, policy_version 505850 (0.0010) [2023-12-26 19:02:42,321][105620] Updated weights for policy 1, policy_version 505860 (0.0009) [2023-12-26 19:02:42,645][105692] Updated weights for policy 0, policy_version 505449 (0.0009) [2023-12-26 19:02:42,704][105692] Updated weights for policy 0, policy_version 505459 (0.0008) [2023-12-26 19:02:42,767][105692] Updated weights for policy 0, policy_version 505469 (0.0009) [2023-12-26 19:02:43,055][105620] Updated weights for policy 1, policy_version 505870 (0.0007) [2023-12-26 19:02:43,113][105620] Updated weights for policy 1, policy_version 505880 (0.0006) [2023-12-26 19:02:43,172][105620] Updated weights for policy 1, policy_version 505890 (0.0006) [2023-12-26 19:02:43,538][105692] Updated weights for policy 0, policy_version 505479 (0.0009) [2023-12-26 19:02:43,586][105692] Updated weights for policy 0, policy_version 505489 (0.0008) [2023-12-26 19:02:43,633][105692] Updated weights for policy 0, policy_version 505499 (0.0009) [2023-12-26 19:02:43,824][105620] Updated weights for policy 1, policy_version 505900 (0.0005) [2023-12-26 19:02:43,884][105620] Updated weights for policy 1, policy_version 505910 (0.0005) [2023-12-26 19:02:43,945][105620] Updated weights for policy 1, policy_version 505920 (0.0005) [2023-12-26 19:02:44,479][105620] Updated weights for policy 1, policy_version 505930 (0.0006) [2023-12-26 19:02:44,518][105692] Updated weights for policy 0, policy_version 505509 (0.0008) [2023-12-26 19:02:44,549][105620] Updated weights for policy 1, policy_version 505940 (0.0008) [2023-12-26 19:02:44,577][105692] Updated weights for policy 0, policy_version 505519 (0.0008) [2023-12-26 19:02:44,611][105620] Updated weights for policy 1, policy_version 505950 (0.0007) [2023-12-26 19:02:44,637][105692] Updated weights for policy 0, policy_version 505529 (0.0009) [2023-12-26 19:02:44,672][105620] Updated weights for policy 1, policy_version 505960 (0.0008) [2023-12-26 19:02:45,385][105692] Updated weights for policy 0, policy_version 505539 (0.0008) [2023-12-26 19:02:45,407][105620] Updated weights for policy 1, policy_version 505970 (0.0006) [2023-12-26 19:02:45,445][105692] Updated weights for policy 0, policy_version 505550 (0.0009) [2023-12-26 19:02:45,471][105620] Updated weights for policy 1, policy_version 505980 (0.0005) [2023-12-26 19:02:45,500][105692] Updated weights for policy 0, policy_version 505560 (0.0009) [2023-12-26 19:02:45,532][105620] Updated weights for policy 1, policy_version 505990 (0.0005) [2023-12-26 19:02:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 258990080. Throughput: 0: 9566.8, 1: 9905.0. Samples: 258964144. Policy #0 lag: (min: 26.0, avg: 43.7, max: 58.0) [2023-12-26 19:02:46,062][104569] Avg episode reward: [(0, '9267.338'), (1, '9356.956')] [2023-12-26 19:02:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000505568_129441792.pth... [2023-12-26 19:02:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000505992_129548288.pth... [2023-12-26 19:02:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000504480_129163264.pth [2023-12-26 19:02:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000504840_129253376.pth [2023-12-26 19:02:46,146][105620] Updated weights for policy 1, policy_version 506000 (0.0005) [2023-12-26 19:02:46,153][105692] Updated weights for policy 0, policy_version 505570 (0.0008) [2023-12-26 19:02:46,202][105620] Updated weights for policy 1, policy_version 506010 (0.0006) [2023-12-26 19:02:46,209][105692] Updated weights for policy 0, policy_version 505580 (0.0009) [2023-12-26 19:02:46,247][105620] Updated weights for policy 1, policy_version 506020 (0.0006) [2023-12-26 19:02:46,257][105692] Updated weights for policy 0, policy_version 505590 (0.0007) [2023-12-26 19:02:46,309][105692] Updated weights for policy 0, policy_version 505600 (0.0008) [2023-12-26 19:02:46,876][105620] Updated weights for policy 1, policy_version 506030 (0.0008) [2023-12-26 19:02:46,893][105586] KL-divergence is very high: 111.8936 [2023-12-26 19:02:46,925][105620] Updated weights for policy 1, policy_version 506041 (0.0009) [2023-12-26 19:02:46,930][105586] KL-divergence is very high: 162.5604 [2023-12-26 19:02:46,968][105586] KL-divergence is very high: 138.7272 [2023-12-26 19:02:46,972][105620] Updated weights for policy 1, policy_version 506051 (0.0009) [2023-12-26 19:02:47,095][105692] Updated weights for policy 0, policy_version 505610 (0.0009) [2023-12-26 19:02:47,153][105692] Updated weights for policy 0, policy_version 505620 (0.0009) [2023-12-26 19:02:47,201][105692] Updated weights for policy 0, policy_version 505630 (0.0009) [2023-12-26 19:02:47,711][105620] Updated weights for policy 1, policy_version 506061 (0.0008) [2023-12-26 19:02:47,763][105620] Updated weights for policy 1, policy_version 506071 (0.0010) [2023-12-26 19:02:47,810][105620] Updated weights for policy 1, policy_version 506081 (0.0010) [2023-12-26 19:02:47,900][105692] Updated weights for policy 0, policy_version 505640 (0.0008) [2023-12-26 19:02:47,959][105692] Updated weights for policy 0, policy_version 505650 (0.0008) [2023-12-26 19:02:48,011][105692] Updated weights for policy 0, policy_version 505660 (0.0008) [2023-12-26 19:02:48,597][105620] Updated weights for policy 1, policy_version 506091 (0.0010) [2023-12-26 19:02:48,652][105620] Updated weights for policy 1, policy_version 506101 (0.0010) [2023-12-26 19:02:48,704][105620] Updated weights for policy 1, policy_version 506111 (0.0010) [2023-12-26 19:02:48,791][105692] Updated weights for policy 0, policy_version 505670 (0.0008) [2023-12-26 19:02:48,843][105692] Updated weights for policy 0, policy_version 505680 (0.0008) [2023-12-26 19:02:48,896][105692] Updated weights for policy 0, policy_version 505690 (0.0008) [2023-12-26 19:02:49,458][105620] Updated weights for policy 1, policy_version 506121 (0.0010) [2023-12-26 19:02:49,514][105620] Updated weights for policy 1, policy_version 506131 (0.0010) [2023-12-26 19:02:49,573][105620] Updated weights for policy 1, policy_version 506141 (0.0010) [2023-12-26 19:02:49,633][105620] Updated weights for policy 1, policy_version 506151 (0.0010) [2023-12-26 19:02:49,691][105692] Updated weights for policy 0, policy_version 505700 (0.0008) [2023-12-26 19:02:49,743][105692] Updated weights for policy 0, policy_version 505710 (0.0008) [2023-12-26 19:02:49,802][105692] Updated weights for policy 0, policy_version 505720 (0.0008) [2023-12-26 19:02:50,300][105620] Updated weights for policy 1, policy_version 506161 (0.0010) [2023-12-26 19:02:50,356][105620] Updated weights for policy 1, policy_version 506171 (0.0010) [2023-12-26 19:02:50,415][105620] Updated weights for policy 1, policy_version 506181 (0.0010) [2023-12-26 19:02:50,634][105692] Updated weights for policy 0, policy_version 505730 (0.0009) [2023-12-26 19:02:50,702][105692] Updated weights for policy 0, policy_version 505740 (0.0009) [2023-12-26 19:02:50,766][105692] Updated weights for policy 0, policy_version 505750 (0.0008) [2023-12-26 19:02:50,826][105692] Updated weights for policy 0, policy_version 505760 (0.0008) [2023-12-26 19:02:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 259088384. Throughput: 0: 9552.6, 1: 9897.4. Samples: 259080020. Policy #0 lag: (min: 26.0, avg: 43.7, max: 58.0) [2023-12-26 19:02:51,063][104569] Avg episode reward: [(0, '8996.318'), (1, '9265.957')] [2023-12-26 19:02:51,154][105620] Updated weights for policy 1, policy_version 506191 (0.0011) [2023-12-26 19:02:51,210][105620] Updated weights for policy 1, policy_version 506201 (0.0010) [2023-12-26 19:02:51,271][105620] Updated weights for policy 1, policy_version 506211 (0.0009) [2023-12-26 19:02:51,567][105692] Updated weights for policy 0, policy_version 505770 (0.0009) [2023-12-26 19:02:51,631][105692] Updated weights for policy 0, policy_version 505780 (0.0009) [2023-12-26 19:02:51,690][105692] Updated weights for policy 0, policy_version 505790 (0.0009) [2023-12-26 19:02:52,049][105620] Updated weights for policy 1, policy_version 506221 (0.0009) [2023-12-26 19:02:52,104][105620] Updated weights for policy 1, policy_version 506231 (0.0010) [2023-12-26 19:02:52,156][105620] Updated weights for policy 1, policy_version 506241 (0.0010) [2023-12-26 19:02:52,468][105692] Updated weights for policy 0, policy_version 505800 (0.0008) [2023-12-26 19:02:52,531][105692] Updated weights for policy 0, policy_version 505810 (0.0008) [2023-12-26 19:02:52,580][105692] Updated weights for policy 0, policy_version 505820 (0.0008) [2023-12-26 19:02:52,920][105620] Updated weights for policy 1, policy_version 506251 (0.0010) [2023-12-26 19:02:52,974][105620] Updated weights for policy 1, policy_version 506261 (0.0011) [2023-12-26 19:02:53,022][105620] Updated weights for policy 1, policy_version 506271 (0.0011) [2023-12-26 19:02:53,371][105692] Updated weights for policy 0, policy_version 505830 (0.0007) [2023-12-26 19:02:53,431][105692] Updated weights for policy 0, policy_version 505840 (0.0005) [2023-12-26 19:02:53,497][105692] Updated weights for policy 0, policy_version 505850 (0.0006) [2023-12-26 19:02:53,657][105620] Updated weights for policy 1, policy_version 506281 (0.0010) [2023-12-26 19:02:53,724][105620] Updated weights for policy 1, policy_version 506291 (0.0006) [2023-12-26 19:02:53,789][105620] Updated weights for policy 1, policy_version 506301 (0.0006) [2023-12-26 19:02:53,842][105620] Updated weights for policy 1, policy_version 506311 (0.0005) [2023-12-26 19:02:54,192][105692] Updated weights for policy 0, policy_version 505860 (0.0007) [2023-12-26 19:02:54,245][105692] Updated weights for policy 0, policy_version 505870 (0.0009) [2023-12-26 19:02:54,306][105692] Updated weights for policy 0, policy_version 505880 (0.0009) [2023-12-26 19:02:54,356][105620] Updated weights for policy 1, policy_version 506321 (0.0010) [2023-12-26 19:02:54,411][105620] Updated weights for policy 1, policy_version 506331 (0.0010) [2023-12-26 19:02:54,470][105620] Updated weights for policy 1, policy_version 506341 (0.0011) [2023-12-26 19:02:55,120][105620] Updated weights for policy 1, policy_version 506351 (0.0009) [2023-12-26 19:02:55,137][105692] Updated weights for policy 0, policy_version 505890 (0.0009) [2023-12-26 19:02:55,173][105620] Updated weights for policy 1, policy_version 506361 (0.0005) [2023-12-26 19:02:55,189][105692] Updated weights for policy 0, policy_version 505900 (0.0008) [2023-12-26 19:02:55,231][105620] Updated weights for policy 1, policy_version 506371 (0.0005) [2023-12-26 19:02:55,238][105692] Updated weights for policy 0, policy_version 505910 (0.0009) [2023-12-26 19:02:55,294][105692] Updated weights for policy 0, policy_version 505920 (0.0010) [2023-12-26 19:02:55,767][105620] Updated weights for policy 1, policy_version 506381 (0.0005) [2023-12-26 19:02:55,828][105620] Updated weights for policy 1, policy_version 506391 (0.0005) [2023-12-26 19:02:55,874][105620] Updated weights for policy 1, policy_version 506401 (0.0005) [2023-12-26 19:02:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 259186688. Throughput: 0: 9413.7, 1: 10038.9. Samples: 259197036. Policy #0 lag: (min: 26.0, avg: 43.7, max: 58.0) [2023-12-26 19:02:56,063][104569] Avg episode reward: [(0, '8995.281'), (1, '9266.345')] [2023-12-26 19:02:56,228][105692] Updated weights for policy 0, policy_version 505930 (0.0010) [2023-12-26 19:02:56,282][105692] Updated weights for policy 0, policy_version 505941 (0.0010) [2023-12-26 19:02:56,336][105692] Updated weights for policy 0, policy_version 505951 (0.0010) [2023-12-26 19:02:56,390][105620] Updated weights for policy 1, policy_version 506411 (0.0005) [2023-12-26 19:02:56,451][105620] Updated weights for policy 1, policy_version 506421 (0.0005) [2023-12-26 19:02:56,504][105620] Updated weights for policy 1, policy_version 506431 (0.0005) [2023-12-26 19:02:57,112][105620] Updated weights for policy 1, policy_version 506441 (0.0005) [2023-12-26 19:02:57,165][105620] Updated weights for policy 1, policy_version 506451 (0.0005) [2023-12-26 19:02:57,167][105692] Updated weights for policy 0, policy_version 505961 (0.0009) [2023-12-26 19:02:57,215][105620] Updated weights for policy 1, policy_version 506461 (0.0006) [2023-12-26 19:02:57,218][105692] Updated weights for policy 0, policy_version 505971 (0.0008) [2023-12-26 19:02:57,261][105620] Updated weights for policy 1, policy_version 506471 (0.0006) [2023-12-26 19:02:57,275][105692] Updated weights for policy 0, policy_version 505981 (0.0008) [2023-12-26 19:02:57,866][105620] Updated weights for policy 1, policy_version 506481 (0.0006) [2023-12-26 19:02:57,909][105620] Updated weights for policy 1, policy_version 506491 (0.0005) [2023-12-26 19:02:57,955][105620] Updated weights for policy 1, policy_version 506501 (0.0005) [2023-12-26 19:02:58,145][105692] Updated weights for policy 0, policy_version 505991 (0.0008) [2023-12-26 19:02:58,206][105692] Updated weights for policy 0, policy_version 506001 (0.0008) [2023-12-26 19:02:58,265][105692] Updated weights for policy 0, policy_version 506011 (0.0008) [2023-12-26 19:02:58,634][105620] Updated weights for policy 1, policy_version 506511 (0.0009) [2023-12-26 19:02:58,700][105620] Updated weights for policy 1, policy_version 506521 (0.0010) [2023-12-26 19:02:58,763][105620] Updated weights for policy 1, policy_version 506531 (0.0010) [2023-12-26 19:02:59,000][105692] Updated weights for policy 0, policy_version 506021 (0.0008) [2023-12-26 19:02:59,050][105692] Updated weights for policy 0, policy_version 506031 (0.0008) [2023-12-26 19:02:59,100][105692] Updated weights for policy 0, policy_version 506041 (0.0008) [2023-12-26 19:02:59,456][105620] Updated weights for policy 1, policy_version 506541 (0.0009) [2023-12-26 19:02:59,518][105620] Updated weights for policy 1, policy_version 506551 (0.0009) [2023-12-26 19:02:59,584][105620] Updated weights for policy 1, policy_version 506561 (0.0009) [2023-12-26 19:02:59,958][105692] Updated weights for policy 0, policy_version 506051 (0.0009) [2023-12-26 19:03:00,005][105692] Updated weights for policy 0, policy_version 506061 (0.0009) [2023-12-26 19:03:00,057][105692] Updated weights for policy 0, policy_version 506071 (0.0010) [2023-12-26 19:03:00,191][105620] Updated weights for policy 1, policy_version 506571 (0.0009) [2023-12-26 19:03:00,243][105620] Updated weights for policy 1, policy_version 506581 (0.0006) [2023-12-26 19:03:00,295][105620] Updated weights for policy 1, policy_version 506591 (0.0009) [2023-12-26 19:03:00,739][105692] Updated weights for policy 0, policy_version 506081 (0.0010) [2023-12-26 19:03:00,806][105692] Updated weights for policy 0, policy_version 506091 (0.0007) [2023-12-26 19:03:00,863][105692] Updated weights for policy 0, policy_version 506101 (0.0008) [2023-12-26 19:03:00,917][105692] Updated weights for policy 0, policy_version 506111 (0.0009) [2023-12-26 19:03:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 259284992. Throughput: 0: 9328.1, 1: 10099.6. Samples: 259255672. Policy #0 lag: (min: 26.0, avg: 43.7, max: 58.0) [2023-12-26 19:03:01,062][104569] Avg episode reward: [(0, '8997.645'), (1, '9174.773')] [2023-12-26 19:03:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000506112_129581056.pth... [2023-12-26 19:03:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000506600_129703936.pth... [2023-12-26 19:03:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000505416_129400832.pth [2023-12-26 19:03:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000505056_129310720.pth [2023-12-26 19:03:01,109][105620] Updated weights for policy 1, policy_version 506601 (0.0009) [2023-12-26 19:03:01,169][105620] Updated weights for policy 1, policy_version 506611 (0.0009) [2023-12-26 19:03:01,223][105620] Updated weights for policy 1, policy_version 506621 (0.0006) [2023-12-26 19:03:01,281][105620] Updated weights for policy 1, policy_version 506631 (0.0008) [2023-12-26 19:03:01,666][105692] Updated weights for policy 0, policy_version 506121 (0.0006) [2023-12-26 19:03:01,733][105692] Updated weights for policy 0, policy_version 506131 (0.0007) [2023-12-26 19:03:01,795][105692] Updated weights for policy 0, policy_version 506141 (0.0008) [2023-12-26 19:03:02,046][105620] Updated weights for policy 1, policy_version 506641 (0.0008) [2023-12-26 19:03:02,100][105620] Updated weights for policy 1, policy_version 506651 (0.0009) [2023-12-26 19:03:02,159][105620] Updated weights for policy 1, policy_version 506661 (0.0009) [2023-12-26 19:03:02,476][105692] Updated weights for policy 0, policy_version 506151 (0.0006) [2023-12-26 19:03:02,538][105692] Updated weights for policy 0, policy_version 506161 (0.0005) [2023-12-26 19:03:02,602][105692] Updated weights for policy 0, policy_version 506171 (0.0008) [2023-12-26 19:03:02,961][105620] Updated weights for policy 1, policy_version 506671 (0.0009) [2023-12-26 19:03:03,013][105620] Updated weights for policy 1, policy_version 506681 (0.0007) [2023-12-26 19:03:03,074][105620] Updated weights for policy 1, policy_version 506691 (0.0008) [2023-12-26 19:03:03,255][105692] Updated weights for policy 0, policy_version 506181 (0.0009) [2023-12-26 19:03:03,311][105692] Updated weights for policy 0, policy_version 506191 (0.0009) [2023-12-26 19:03:03,360][105692] Updated weights for policy 0, policy_version 506201 (0.0010) [2023-12-26 19:03:03,802][105620] Updated weights for policy 1, policy_version 506701 (0.0009) [2023-12-26 19:03:03,870][105620] Updated weights for policy 1, policy_version 506711 (0.0010) [2023-12-26 19:03:03,930][105620] Updated weights for policy 1, policy_version 506721 (0.0011) [2023-12-26 19:03:04,095][105692] Updated weights for policy 0, policy_version 506211 (0.0008) [2023-12-26 19:03:04,159][105692] Updated weights for policy 0, policy_version 506221 (0.0009) [2023-12-26 19:03:04,229][105692] Updated weights for policy 0, policy_version 506231 (0.0011) [2023-12-26 19:03:04,609][105620] Updated weights for policy 1, policy_version 506731 (0.0010) [2023-12-26 19:03:04,671][105620] Updated weights for policy 1, policy_version 506741 (0.0009) [2023-12-26 19:03:04,727][105620] Updated weights for policy 1, policy_version 506751 (0.0007) [2023-12-26 19:03:04,950][105692] Updated weights for policy 0, policy_version 506241 (0.0009) [2023-12-26 19:03:05,015][105692] Updated weights for policy 0, policy_version 506251 (0.0009) [2023-12-26 19:03:05,081][105692] Updated weights for policy 0, policy_version 506261 (0.0009) [2023-12-26 19:03:05,132][105692] Updated weights for policy 0, policy_version 506271 (0.0009) [2023-12-26 19:03:05,440][105620] Updated weights for policy 1, policy_version 506761 (0.0009) [2023-12-26 19:03:05,498][105620] Updated weights for policy 1, policy_version 506771 (0.0009) [2023-12-26 19:03:05,553][105620] Updated weights for policy 1, policy_version 506781 (0.0009) [2023-12-26 19:03:05,606][105620] Updated weights for policy 1, policy_version 506791 (0.0006) [2023-12-26 19:03:05,967][105692] Updated weights for policy 0, policy_version 506281 (0.0009) [2023-12-26 19:03:06,042][105692] Updated weights for policy 0, policy_version 506291 (0.0010) [2023-12-26 19:03:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 259375104. Throughput: 0: 9332.2, 1: 10064.7. Samples: 259370100. Policy #0 lag: (min: 26.0, avg: 43.7, max: 58.0) [2023-12-26 19:03:06,062][104569] Avg episode reward: [(0, '8998.472'), (1, '9174.780')] [2023-12-26 19:03:06,114][105692] Updated weights for policy 0, policy_version 506301 (0.0010) [2023-12-26 19:03:06,191][105620] Updated weights for policy 1, policy_version 506801 (0.0010) [2023-12-26 19:03:06,252][105620] Updated weights for policy 1, policy_version 506811 (0.0010) [2023-12-26 19:03:06,307][105620] Updated weights for policy 1, policy_version 506821 (0.0010) [2023-12-26 19:03:06,877][105692] Updated weights for policy 0, policy_version 506311 (0.0007) [2023-12-26 19:03:06,926][105692] Updated weights for policy 0, policy_version 506321 (0.0008) [2023-12-26 19:03:06,975][105692] Updated weights for policy 0, policy_version 506331 (0.0008) [2023-12-26 19:03:07,048][105620] Updated weights for policy 1, policy_version 506831 (0.0010) [2023-12-26 19:03:07,114][105620] Updated weights for policy 1, policy_version 506841 (0.0010) [2023-12-26 19:03:07,176][105620] Updated weights for policy 1, policy_version 506851 (0.0010) [2023-12-26 19:03:07,695][105692] Updated weights for policy 0, policy_version 506341 (0.0008) [2023-12-26 19:03:07,755][105692] Updated weights for policy 0, policy_version 506351 (0.0009) [2023-12-26 19:03:07,782][105620] Updated weights for policy 1, policy_version 506861 (0.0010) [2023-12-26 19:03:07,810][105692] Updated weights for policy 0, policy_version 506361 (0.0010) [2023-12-26 19:03:07,840][105620] Updated weights for policy 1, policy_version 506871 (0.0010) [2023-12-26 19:03:07,897][105620] Updated weights for policy 1, policy_version 506881 (0.0010) [2023-12-26 19:03:08,491][105620] Updated weights for policy 1, policy_version 506891 (0.0009) [2023-12-26 19:03:08,531][105692] Updated weights for policy 0, policy_version 506371 (0.0010) [2023-12-26 19:03:08,546][105620] Updated weights for policy 1, policy_version 506901 (0.0008) [2023-12-26 19:03:08,588][105620] Updated weights for policy 1, policy_version 506911 (0.0008) [2023-12-26 19:03:08,590][105692] Updated weights for policy 0, policy_version 506381 (0.0010) [2023-12-26 19:03:08,651][105692] Updated weights for policy 0, policy_version 506391 (0.0007) [2023-12-26 19:03:09,356][105692] Updated weights for policy 0, policy_version 506401 (0.0008) [2023-12-26 19:03:09,358][105620] Updated weights for policy 1, policy_version 506921 (0.0010) [2023-12-26 19:03:09,424][105692] Updated weights for policy 0, policy_version 506411 (0.0006) [2023-12-26 19:03:09,426][105620] Updated weights for policy 1, policy_version 506931 (0.0011) [2023-12-26 19:03:09,486][105692] Updated weights for policy 0, policy_version 506421 (0.0007) [2023-12-26 19:03:09,486][105620] Updated weights for policy 1, policy_version 506941 (0.0008) [2023-12-26 19:03:09,542][105692] Updated weights for policy 0, policy_version 506431 (0.0009) [2023-12-26 19:03:09,550][105620] Updated weights for policy 1, policy_version 506951 (0.0006) [2023-12-26 19:03:10,244][105620] Updated weights for policy 1, policy_version 506961 (0.0010) [2023-12-26 19:03:10,293][105620] Updated weights for policy 1, policy_version 506971 (0.0010) [2023-12-26 19:03:10,319][105692] Updated weights for policy 0, policy_version 506441 (0.0009) [2023-12-26 19:03:10,349][105620] Updated weights for policy 1, policy_version 506981 (0.0010) [2023-12-26 19:03:10,380][105692] Updated weights for policy 0, policy_version 506451 (0.0007) [2023-12-26 19:03:10,449][105692] Updated weights for policy 0, policy_version 506461 (0.0009) [2023-12-26 19:03:11,002][105620] Updated weights for policy 1, policy_version 506991 (0.0006) [2023-12-26 19:03:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 259473408. Throughput: 0: 9166.7, 1: 10143.2. Samples: 259487800. Policy #0 lag: (min: 26.0, avg: 43.7, max: 58.0) [2023-12-26 19:03:11,062][104569] Avg episode reward: [(0, '8816.736'), (1, '9265.056')] [2023-12-26 19:03:11,069][105620] Updated weights for policy 1, policy_version 507001 (0.0009) [2023-12-26 19:03:11,133][105620] Updated weights for policy 1, policy_version 507011 (0.0006) [2023-12-26 19:03:11,283][105692] Updated weights for policy 0, policy_version 506471 (0.0009) [2023-12-26 19:03:11,344][105692] Updated weights for policy 0, policy_version 506481 (0.0009) [2023-12-26 19:03:11,409][105692] Updated weights for policy 0, policy_version 506491 (0.0007) [2023-12-26 19:03:11,813][105620] Updated weights for policy 1, policy_version 507021 (0.0008) [2023-12-26 19:03:11,879][105620] Updated weights for policy 1, policy_version 507031 (0.0009) [2023-12-26 19:03:11,942][105620] Updated weights for policy 1, policy_version 507041 (0.0009) [2023-12-26 19:03:12,089][105692] Updated weights for policy 0, policy_version 506501 (0.0007) [2023-12-26 19:03:12,152][105692] Updated weights for policy 0, policy_version 506511 (0.0009) [2023-12-26 19:03:12,200][105692] Updated weights for policy 0, policy_version 506521 (0.0008) [2023-12-26 19:03:12,645][105620] Updated weights for policy 1, policy_version 507051 (0.0008) [2023-12-26 19:03:12,669][105586] KL-divergence is very high: 100.5140 [2023-12-26 19:03:12,694][105620] Updated weights for policy 1, policy_version 507061 (0.0008) [2023-12-26 19:03:12,708][105586] KL-divergence is very high: 139.7339 [2023-12-26 19:03:12,741][105620] Updated weights for policy 1, policy_version 507071 (0.0007) [2023-12-26 19:03:12,748][105586] KL-divergence is very high: 113.1549 [2023-12-26 19:03:12,976][105692] Updated weights for policy 0, policy_version 506531 (0.0008) [2023-12-26 19:03:13,024][105692] Updated weights for policy 0, policy_version 506541 (0.0008) [2023-12-26 19:03:13,071][105692] Updated weights for policy 0, policy_version 506551 (0.0008) [2023-12-26 19:03:13,490][105620] Updated weights for policy 1, policy_version 507081 (0.0006) [2023-12-26 19:03:13,538][105620] Updated weights for policy 1, policy_version 507091 (0.0010) [2023-12-26 19:03:13,600][105620] Updated weights for policy 1, policy_version 507101 (0.0010) [2023-12-26 19:03:13,664][105620] Updated weights for policy 1, policy_version 507111 (0.0010) [2023-12-26 19:03:13,857][105692] Updated weights for policy 0, policy_version 506561 (0.0007) [2023-12-26 19:03:13,906][105692] Updated weights for policy 0, policy_version 506571 (0.0008) [2023-12-26 19:03:13,965][105692] Updated weights for policy 0, policy_version 506581 (0.0009) [2023-12-26 19:03:14,019][105692] Updated weights for policy 0, policy_version 506591 (0.0009) [2023-12-26 19:03:14,312][105620] Updated weights for policy 1, policy_version 507121 (0.0006) [2023-12-26 19:03:14,360][105620] Updated weights for policy 1, policy_version 507131 (0.0006) [2023-12-26 19:03:14,417][105620] Updated weights for policy 1, policy_version 507141 (0.0009) [2023-12-26 19:03:14,662][105692] Updated weights for policy 0, policy_version 506601 (0.0006) [2023-12-26 19:03:14,718][105692] Updated weights for policy 0, policy_version 506611 (0.0007) [2023-12-26 19:03:14,776][105692] Updated weights for policy 0, policy_version 506621 (0.0008) [2023-12-26 19:03:15,188][105620] Updated weights for policy 1, policy_version 507151 (0.0007) [2023-12-26 19:03:15,257][105620] Updated weights for policy 1, policy_version 507161 (0.0006) [2023-12-26 19:03:15,323][105620] Updated weights for policy 1, policy_version 507171 (0.0006) [2023-12-26 19:03:15,473][105692] Updated weights for policy 0, policy_version 506631 (0.0011) [2023-12-26 19:03:15,542][105692] Updated weights for policy 0, policy_version 506641 (0.0010) [2023-12-26 19:03:15,604][105692] Updated weights for policy 0, policy_version 506651 (0.0010) [2023-12-26 19:03:15,872][105620] Updated weights for policy 1, policy_version 507181 (0.0008) [2023-12-26 19:03:15,919][105620] Updated weights for policy 1, policy_version 507191 (0.0008) [2023-12-26 19:03:15,973][105620] Updated weights for policy 1, policy_version 507201 (0.0005) [2023-12-26 19:03:16,062][104569] Fps is (10 sec: 20479.4, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 259579904. Throughput: 0: 9105.1, 1: 10148.9. Samples: 259544744. Policy #0 lag: (min: 26.0, avg: 43.7, max: 58.0) [2023-12-26 19:03:16,063][104569] Avg episode reward: [(0, '8903.207'), (1, '9083.294')] [2023-12-26 19:03:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000506656_129720320.pth... [2023-12-26 19:03:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000507208_129859584.pth... [2023-12-26 19:03:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000505568_129441792.pth [2023-12-26 19:03:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000505992_129548288.pth [2023-12-26 19:03:16,345][105692] Updated weights for policy 0, policy_version 506661 (0.0010) [2023-12-26 19:03:16,396][105692] Updated weights for policy 0, policy_version 506671 (0.0008) [2023-12-26 19:03:16,450][105692] Updated weights for policy 0, policy_version 506681 (0.0009) [2023-12-26 19:03:16,630][105620] Updated weights for policy 1, policy_version 507211 (0.0006) [2023-12-26 19:03:16,687][105620] Updated weights for policy 1, policy_version 507221 (0.0009) [2023-12-26 19:03:16,744][105620] Updated weights for policy 1, policy_version 507231 (0.0008) [2023-12-26 19:03:17,224][105692] Updated weights for policy 0, policy_version 506691 (0.0009) [2023-12-26 19:03:17,274][105692] Updated weights for policy 0, policy_version 506701 (0.0009) [2023-12-26 19:03:17,328][105692] Updated weights for policy 0, policy_version 506711 (0.0008) [2023-12-26 19:03:17,502][105620] Updated weights for policy 1, policy_version 507241 (0.0009) [2023-12-26 19:03:17,554][105620] Updated weights for policy 1, policy_version 507252 (0.0008) [2023-12-26 19:03:17,609][105620] Updated weights for policy 1, policy_version 507262 (0.0005) [2023-12-26 19:03:17,665][105620] Updated weights for policy 1, policy_version 507272 (0.0005) [2023-12-26 19:03:18,097][105692] Updated weights for policy 0, policy_version 506721 (0.0009) [2023-12-26 19:03:18,158][105692] Updated weights for policy 0, policy_version 506731 (0.0009) [2023-12-26 19:03:18,206][105692] Updated weights for policy 0, policy_version 506741 (0.0009) [2023-12-26 19:03:18,267][105692] Updated weights for policy 0, policy_version 506751 (0.0009) [2023-12-26 19:03:18,335][105620] Updated weights for policy 1, policy_version 507282 (0.0008) [2023-12-26 19:03:18,391][105620] Updated weights for policy 1, policy_version 507292 (0.0008) [2023-12-26 19:03:18,449][105620] Updated weights for policy 1, policy_version 507302 (0.0010) [2023-12-26 19:03:19,023][105692] Updated weights for policy 0, policy_version 506761 (0.0009) [2023-12-26 19:03:19,079][105692] Updated weights for policy 0, policy_version 506771 (0.0008) [2023-12-26 19:03:19,137][105692] Updated weights for policy 0, policy_version 506781 (0.0008) [2023-12-26 19:03:19,178][105620] Updated weights for policy 1, policy_version 507312 (0.0010) [2023-12-26 19:03:19,240][105620] Updated weights for policy 1, policy_version 507322 (0.0011) [2023-12-26 19:03:19,307][105620] Updated weights for policy 1, policy_version 507332 (0.0006) [2023-12-26 19:03:19,834][105692] Updated weights for policy 0, policy_version 506791 (0.0008) [2023-12-26 19:03:19,896][105692] Updated weights for policy 0, policy_version 506801 (0.0007) [2023-12-26 19:03:19,967][105692] Updated weights for policy 0, policy_version 506811 (0.0009) [2023-12-26 19:03:20,068][105620] Updated weights for policy 1, policy_version 507342 (0.0008) [2023-12-26 19:03:20,135][105620] Updated weights for policy 1, policy_version 507352 (0.0009) [2023-12-26 19:03:20,201][105620] Updated weights for policy 1, policy_version 507362 (0.0007) [2023-12-26 19:03:20,749][105692] Updated weights for policy 0, policy_version 506821 (0.0009) [2023-12-26 19:03:20,809][105692] Updated weights for policy 0, policy_version 506831 (0.0009) [2023-12-26 19:03:20,838][105620] Updated weights for policy 1, policy_version 507372 (0.0007) [2023-12-26 19:03:20,867][105692] Updated weights for policy 0, policy_version 506841 (0.0008) [2023-12-26 19:03:20,903][105620] Updated weights for policy 1, policy_version 507382 (0.0009) [2023-12-26 19:03:20,965][105620] Updated weights for policy 1, policy_version 507392 (0.0010) [2023-12-26 19:03:21,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 259678208. Throughput: 0: 9116.1, 1: 10174.0. Samples: 259661792. Policy #0 lag: (min: 26.0, avg: 43.7, max: 58.0) [2023-12-26 19:03:21,062][104569] Avg episode reward: [(0, '8993.644'), (1, '9085.238')] [2023-12-26 19:03:21,632][105692] Updated weights for policy 0, policy_version 506851 (0.0009) [2023-12-26 19:03:21,691][105692] Updated weights for policy 0, policy_version 506861 (0.0009) [2023-12-26 19:03:21,722][105620] Updated weights for policy 1, policy_version 507402 (0.0009) [2023-12-26 19:03:21,753][105692] Updated weights for policy 0, policy_version 506871 (0.0008) [2023-12-26 19:03:21,784][105620] Updated weights for policy 1, policy_version 507412 (0.0008) [2023-12-26 19:03:21,845][105620] Updated weights for policy 1, policy_version 507422 (0.0008) [2023-12-26 19:03:21,907][105620] Updated weights for policy 1, policy_version 507432 (0.0009) [2023-12-26 19:03:22,457][105692] Updated weights for policy 0, policy_version 506881 (0.0006) [2023-12-26 19:03:22,513][105692] Updated weights for policy 0, policy_version 506891 (0.0006) [2023-12-26 19:03:22,567][105692] Updated weights for policy 0, policy_version 506901 (0.0010) [2023-12-26 19:03:22,615][105620] Updated weights for policy 1, policy_version 507442 (0.0008) [2023-12-26 19:03:22,622][105692] Updated weights for policy 0, policy_version 506911 (0.0009) [2023-12-26 19:03:22,664][105620] Updated weights for policy 1, policy_version 507452 (0.0006) [2023-12-26 19:03:22,718][105620] Updated weights for policy 1, policy_version 507462 (0.0006) [2023-12-26 19:03:23,278][105692] Updated weights for policy 0, policy_version 506921 (0.0006) [2023-12-26 19:03:23,334][105692] Updated weights for policy 0, policy_version 506931 (0.0007) [2023-12-26 19:03:23,392][105692] Updated weights for policy 0, policy_version 506941 (0.0005) [2023-12-26 19:03:23,431][105620] Updated weights for policy 1, policy_version 507472 (0.0009) [2023-12-26 19:03:23,491][105620] Updated weights for policy 1, policy_version 507482 (0.0009) [2023-12-26 19:03:23,542][105620] Updated weights for policy 1, policy_version 507492 (0.0011) [2023-12-26 19:03:23,936][105692] Updated weights for policy 0, policy_version 506951 (0.0007) [2023-12-26 19:03:23,992][105692] Updated weights for policy 0, policy_version 506961 (0.0008) [2023-12-26 19:03:24,055][105692] Updated weights for policy 0, policy_version 506971 (0.0008) [2023-12-26 19:03:24,263][105620] Updated weights for policy 1, policy_version 507502 (0.0010) [2023-12-26 19:03:24,321][105620] Updated weights for policy 1, policy_version 507512 (0.0010) [2023-12-26 19:03:24,380][105620] Updated weights for policy 1, policy_version 507522 (0.0010) [2023-12-26 19:03:24,694][105692] Updated weights for policy 0, policy_version 506981 (0.0007) [2023-12-26 19:03:24,745][105692] Updated weights for policy 0, policy_version 506991 (0.0005) [2023-12-26 19:03:24,795][105692] Updated weights for policy 0, policy_version 507001 (0.0005) [2023-12-26 19:03:25,140][105620] Updated weights for policy 1, policy_version 507532 (0.0010) [2023-12-26 19:03:25,198][105620] Updated weights for policy 1, policy_version 507542 (0.0010) [2023-12-26 19:03:25,256][105620] Updated weights for policy 1, policy_version 507552 (0.0010) [2023-12-26 19:03:25,345][105692] Updated weights for policy 0, policy_version 507011 (0.0006) [2023-12-26 19:03:25,411][105692] Updated weights for policy 0, policy_version 507021 (0.0006) [2023-12-26 19:03:25,471][105692] Updated weights for policy 0, policy_version 507031 (0.0011) [2023-12-26 19:03:25,814][105620] Updated weights for policy 1, policy_version 507562 (0.0008) [2023-12-26 19:03:25,880][105620] Updated weights for policy 1, policy_version 507572 (0.0010) [2023-12-26 19:03:25,941][105620] Updated weights for policy 1, policy_version 507582 (0.0010) [2023-12-26 19:03:25,990][105620] Updated weights for policy 1, policy_version 507592 (0.0010) [2023-12-26 19:03:26,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 259776512. Throughput: 0: 9366.1, 1: 10108.0. Samples: 259783988. Policy #0 lag: (min: 26.0, avg: 43.7, max: 58.0) [2023-12-26 19:03:26,063][104569] Avg episode reward: [(0, '9174.759'), (1, '9176.321')] [2023-12-26 19:03:26,076][105692] Updated weights for policy 0, policy_version 507041 (0.0010) [2023-12-26 19:03:26,138][105692] Updated weights for policy 0, policy_version 507051 (0.0010) [2023-12-26 19:03:26,189][105692] Updated weights for policy 0, policy_version 507061 (0.0005) [2023-12-26 19:03:26,250][105692] Updated weights for policy 0, policy_version 507071 (0.0005) [2023-12-26 19:03:26,711][105620] Updated weights for policy 1, policy_version 507602 (0.0010) [2023-12-26 19:03:26,764][105620] Updated weights for policy 1, policy_version 507612 (0.0007) [2023-12-26 19:03:26,789][105692] Updated weights for policy 0, policy_version 507081 (0.0005) [2023-12-26 19:03:26,813][105620] Updated weights for policy 1, policy_version 507622 (0.0005) [2023-12-26 19:03:26,840][105692] Updated weights for policy 0, policy_version 507091 (0.0005) [2023-12-26 19:03:26,884][105692] Updated weights for policy 0, policy_version 507101 (0.0005) [2023-12-26 19:03:27,418][105620] Updated weights for policy 1, policy_version 507632 (0.0006) [2023-12-26 19:03:27,479][105620] Updated weights for policy 1, policy_version 507642 (0.0006) [2023-12-26 19:03:27,486][105692] Updated weights for policy 0, policy_version 507111 (0.0005) [2023-12-26 19:03:27,532][105692] Updated weights for policy 0, policy_version 507121 (0.0005) [2023-12-26 19:03:27,532][105620] Updated weights for policy 1, policy_version 507652 (0.0009) [2023-12-26 19:03:27,583][105692] Updated weights for policy 0, policy_version 507131 (0.0005) [2023-12-26 19:03:28,110][105620] Updated weights for policy 1, policy_version 507662 (0.0007) [2023-12-26 19:03:28,160][105620] Updated weights for policy 1, policy_version 507672 (0.0010) [2023-12-26 19:03:28,207][105620] Updated weights for policy 1, policy_version 507682 (0.0010) [2023-12-26 19:03:28,338][105692] Updated weights for policy 0, policy_version 507141 (0.0006) [2023-12-26 19:03:28,398][105692] Updated weights for policy 0, policy_version 507151 (0.0009) [2023-12-26 19:03:28,456][105692] Updated weights for policy 0, policy_version 507161 (0.0008) [2023-12-26 19:03:28,903][105620] Updated weights for policy 1, policy_version 507692 (0.0006) [2023-12-26 19:03:28,957][105620] Updated weights for policy 1, policy_version 507702 (0.0005) [2023-12-26 19:03:29,020][105620] Updated weights for policy 1, policy_version 507712 (0.0007) [2023-12-26 19:03:29,143][105692] Updated weights for policy 0, policy_version 507171 (0.0008) [2023-12-26 19:03:29,204][105692] Updated weights for policy 0, policy_version 507181 (0.0008) [2023-12-26 19:03:29,267][105692] Updated weights for policy 0, policy_version 507191 (0.0008) [2023-12-26 19:03:29,672][105620] Updated weights for policy 1, policy_version 507722 (0.0006) [2023-12-26 19:03:29,734][105620] Updated weights for policy 1, policy_version 507732 (0.0008) [2023-12-26 19:03:29,795][105620] Updated weights for policy 1, policy_version 507742 (0.0008) [2023-12-26 19:03:29,862][105620] Updated weights for policy 1, policy_version 507752 (0.0008) [2023-12-26 19:03:29,884][105692] Updated weights for policy 0, policy_version 507201 (0.0006) [2023-12-26 19:03:29,942][105692] Updated weights for policy 0, policy_version 507211 (0.0009) [2023-12-26 19:03:30,000][105692] Updated weights for policy 0, policy_version 507221 (0.0009) [2023-12-26 19:03:30,056][105692] Updated weights for policy 0, policy_version 507231 (0.0009) [2023-12-26 19:03:30,608][105620] Updated weights for policy 1, policy_version 507762 (0.0010) [2023-12-26 19:03:30,659][105620] Updated weights for policy 1, policy_version 507772 (0.0009) [2023-12-26 19:03:30,708][105620] Updated weights for policy 1, policy_version 507782 (0.0009) [2023-12-26 19:03:30,809][105692] Updated weights for policy 0, policy_version 507241 (0.0008) [2023-12-26 19:03:30,862][105692] Updated weights for policy 0, policy_version 507251 (0.0009) [2023-12-26 19:03:30,916][105692] Updated weights for policy 0, policy_version 507261 (0.0008) [2023-12-26 19:03:31,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 259883008. Throughput: 0: 9495.6, 1: 10163.6. Samples: 259848808. Policy #0 lag: (min: 26.0, avg: 43.7, max: 58.0) [2023-12-26 19:03:31,062][104569] Avg episode reward: [(0, '7734.226'), (1, '9174.600')] [2023-12-26 19:03:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000507264_129875968.pth... [2023-12-26 19:03:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000507784_130007040.pth... [2023-12-26 19:03:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000506112_129581056.pth [2023-12-26 19:03:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000506600_129703936.pth [2023-12-26 19:03:31,533][105620] Updated weights for policy 1, policy_version 507792 (0.0009) [2023-12-26 19:03:31,586][105620] Updated weights for policy 1, policy_version 507802 (0.0009) [2023-12-26 19:03:31,652][105620] Updated weights for policy 1, policy_version 507812 (0.0009) [2023-12-26 19:03:31,652][105692] Updated weights for policy 0, policy_version 507271 (0.0008) [2023-12-26 19:03:31,710][105692] Updated weights for policy 0, policy_version 507281 (0.0007) [2023-12-26 19:03:31,765][105692] Updated weights for policy 0, policy_version 507291 (0.0008) [2023-12-26 19:03:32,431][105692] Updated weights for policy 0, policy_version 507301 (0.0008) [2023-12-26 19:03:32,453][105620] Updated weights for policy 1, policy_version 507822 (0.0007) [2023-12-26 19:03:32,496][105692] Updated weights for policy 0, policy_version 507311 (0.0008) [2023-12-26 19:03:32,514][105620] Updated weights for policy 1, policy_version 507832 (0.0008) [2023-12-26 19:03:32,559][105692] Updated weights for policy 0, policy_version 507321 (0.0008) [2023-12-26 19:03:32,573][105620] Updated weights for policy 1, policy_version 507842 (0.0006) [2023-12-26 19:03:33,210][105692] Updated weights for policy 0, policy_version 507331 (0.0007) [2023-12-26 19:03:33,262][105692] Updated weights for policy 0, policy_version 507341 (0.0005) [2023-12-26 19:03:33,315][105692] Updated weights for policy 0, policy_version 507351 (0.0006) [2023-12-26 19:03:33,374][105620] Updated weights for policy 1, policy_version 507852 (0.0008) [2023-12-26 19:03:33,434][105620] Updated weights for policy 1, policy_version 507862 (0.0005) [2023-12-26 19:03:33,503][105620] Updated weights for policy 1, policy_version 507872 (0.0006) [2023-12-26 19:03:33,989][105692] Updated weights for policy 0, policy_version 507361 (0.0009) [2023-12-26 19:03:34,052][105692] Updated weights for policy 0, policy_version 507371 (0.0008) [2023-12-26 19:03:34,116][105692] Updated weights for policy 0, policy_version 507381 (0.0006) [2023-12-26 19:03:34,181][105692] Updated weights for policy 0, policy_version 507391 (0.0007) [2023-12-26 19:03:34,247][105620] Updated weights for policy 1, policy_version 507882 (0.0007) [2023-12-26 19:03:34,313][105620] Updated weights for policy 1, policy_version 507892 (0.0008) [2023-12-26 19:03:34,379][105620] Updated weights for policy 1, policy_version 507902 (0.0008) [2023-12-26 19:03:34,438][105620] Updated weights for policy 1, policy_version 507912 (0.0009) [2023-12-26 19:03:34,879][105692] Updated weights for policy 0, policy_version 507401 (0.0009) [2023-12-26 19:03:34,935][105692] Updated weights for policy 0, policy_version 507411 (0.0008) [2023-12-26 19:03:34,990][105692] Updated weights for policy 0, policy_version 507421 (0.0009) [2023-12-26 19:03:35,168][105620] Updated weights for policy 1, policy_version 507922 (0.0009) [2023-12-26 19:03:35,229][105620] Updated weights for policy 1, policy_version 507932 (0.0009) [2023-12-26 19:03:35,290][105620] Updated weights for policy 1, policy_version 507942 (0.0009) [2023-12-26 19:03:35,774][105692] Updated weights for policy 0, policy_version 507431 (0.0010) [2023-12-26 19:03:35,831][105692] Updated weights for policy 0, policy_version 507441 (0.0010) [2023-12-26 19:03:35,893][105692] Updated weights for policy 0, policy_version 507451 (0.0007) [2023-12-26 19:03:35,978][105620] Updated weights for policy 1, policy_version 507952 (0.0010) [2023-12-26 19:03:36,041][105620] Updated weights for policy 1, policy_version 507962 (0.0009) [2023-12-26 19:03:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 259973120. Throughput: 0: 9609.5, 1: 10037.8. Samples: 259964148. Policy #0 lag: (min: 26.0, avg: 43.7, max: 58.0) [2023-12-26 19:03:36,062][104569] Avg episode reward: [(0, '6998.275'), (1, '9266.150')] [2023-12-26 19:03:36,109][105620] Updated weights for policy 1, policy_version 507972 (0.0008) [2023-12-26 19:03:36,609][105692] Updated weights for policy 0, policy_version 507461 (0.0008) [2023-12-26 19:03:36,672][105692] Updated weights for policy 0, policy_version 507471 (0.0009) [2023-12-26 19:03:36,734][105692] Updated weights for policy 0, policy_version 507481 (0.0009) [2023-12-26 19:03:36,914][105620] Updated weights for policy 1, policy_version 507982 (0.0009) [2023-12-26 19:03:36,964][105620] Updated weights for policy 1, policy_version 507992 (0.0008) [2023-12-26 19:03:37,025][105620] Updated weights for policy 1, policy_version 508002 (0.0007) [2023-12-26 19:03:37,426][105692] Updated weights for policy 0, policy_version 507491 (0.0008) [2023-12-26 19:03:37,482][105692] Updated weights for policy 0, policy_version 507501 (0.0005) [2023-12-26 19:03:37,528][105692] Updated weights for policy 0, policy_version 507511 (0.0005) [2023-12-26 19:03:37,694][105620] Updated weights for policy 1, policy_version 508012 (0.0006) [2023-12-26 19:03:37,751][105620] Updated weights for policy 1, policy_version 508022 (0.0008) [2023-12-26 19:03:37,795][105620] Updated weights for policy 1, policy_version 508032 (0.0008) [2023-12-26 19:03:38,143][105692] Updated weights for policy 0, policy_version 507521 (0.0006) [2023-12-26 19:03:38,202][105692] Updated weights for policy 0, policy_version 507531 (0.0010) [2023-12-26 19:03:38,260][105692] Updated weights for policy 0, policy_version 507541 (0.0010) [2023-12-26 19:03:38,311][105692] Updated weights for policy 0, policy_version 507551 (0.0010) [2023-12-26 19:03:38,548][105620] Updated weights for policy 1, policy_version 508042 (0.0008) [2023-12-26 19:03:38,620][105620] Updated weights for policy 1, policy_version 508052 (0.0008) [2023-12-26 19:03:38,671][105620] Updated weights for policy 1, policy_version 508062 (0.0010) [2023-12-26 19:03:38,724][105620] Updated weights for policy 1, policy_version 508072 (0.0007) [2023-12-26 19:03:38,978][105692] Updated weights for policy 0, policy_version 507561 (0.0007) [2023-12-26 19:03:39,038][105692] Updated weights for policy 0, policy_version 507571 (0.0005) [2023-12-26 19:03:39,092][105692] Updated weights for policy 0, policy_version 507581 (0.0005) [2023-12-26 19:03:39,433][105620] Updated weights for policy 1, policy_version 508082 (0.0008) [2023-12-26 19:03:39,492][105620] Updated weights for policy 1, policy_version 508092 (0.0008) [2023-12-26 19:03:39,554][105620] Updated weights for policy 1, policy_version 508102 (0.0008) [2023-12-26 19:03:39,757][105692] Updated weights for policy 0, policy_version 507591 (0.0006) [2023-12-26 19:03:39,807][105692] Updated weights for policy 0, policy_version 507601 (0.0008) [2023-12-26 19:03:39,875][105692] Updated weights for policy 0, policy_version 507611 (0.0009) [2023-12-26 19:03:40,252][105620] Updated weights for policy 1, policy_version 508112 (0.0006) [2023-12-26 19:03:40,301][105620] Updated weights for policy 1, policy_version 508122 (0.0005) [2023-12-26 19:03:40,361][105620] Updated weights for policy 1, policy_version 508132 (0.0005) [2023-12-26 19:03:40,578][105692] Updated weights for policy 0, policy_version 507621 (0.0007) [2023-12-26 19:03:40,629][105692] Updated weights for policy 0, policy_version 507631 (0.0005) [2023-12-26 19:03:40,688][105692] Updated weights for policy 0, policy_version 507641 (0.0011) [2023-12-26 19:03:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 260071424. Throughput: 0: 9764.2, 1: 9930.4. Samples: 260083292. Policy #0 lag: (min: 26.0, avg: 43.7, max: 58.0) [2023-12-26 19:03:41,062][104569] Avg episode reward: [(0, '8757.570'), (1, '9180.049')] [2023-12-26 19:03:41,073][105620] Updated weights for policy 1, policy_version 508142 (0.0007) [2023-12-26 19:03:41,128][105620] Updated weights for policy 1, policy_version 508152 (0.0008) [2023-12-26 19:03:41,196][105620] Updated weights for policy 1, policy_version 508162 (0.0009) [2023-12-26 19:03:41,346][105692] Updated weights for policy 0, policy_version 507651 (0.0010) [2023-12-26 19:03:41,416][105692] Updated weights for policy 0, policy_version 507661 (0.0008) [2023-12-26 19:03:41,468][105692] Updated weights for policy 0, policy_version 507671 (0.0009) [2023-12-26 19:03:42,030][105620] Updated weights for policy 1, policy_version 508172 (0.0009) [2023-12-26 19:03:42,095][105620] Updated weights for policy 1, policy_version 508182 (0.0008) [2023-12-26 19:03:42,156][105620] Updated weights for policy 1, policy_version 508192 (0.0009) [2023-12-26 19:03:42,214][105692] Updated weights for policy 0, policy_version 507681 (0.0009) [2023-12-26 19:03:42,283][105692] Updated weights for policy 0, policy_version 507691 (0.0010) [2023-12-26 19:03:42,342][105692] Updated weights for policy 0, policy_version 507701 (0.0008) [2023-12-26 19:03:42,416][105692] Updated weights for policy 0, policy_version 507711 (0.0008) [2023-12-26 19:03:42,865][105620] Updated weights for policy 1, policy_version 508202 (0.0008) [2023-12-26 19:03:42,931][105620] Updated weights for policy 1, policy_version 508212 (0.0009) [2023-12-26 19:03:42,991][105620] Updated weights for policy 1, policy_version 508222 (0.0009) [2023-12-26 19:03:43,052][105620] Updated weights for policy 1, policy_version 508232 (0.0009) [2023-12-26 19:03:43,100][105692] Updated weights for policy 0, policy_version 507721 (0.0007) [2023-12-26 19:03:43,159][105692] Updated weights for policy 0, policy_version 507731 (0.0006) [2023-12-26 19:03:43,216][105692] Updated weights for policy 0, policy_version 507741 (0.0007) [2023-12-26 19:03:43,734][105620] Updated weights for policy 1, policy_version 508242 (0.0005) [2023-12-26 19:03:43,796][105620] Updated weights for policy 1, policy_version 508252 (0.0007) [2023-12-26 19:03:43,851][105620] Updated weights for policy 1, policy_version 508262 (0.0010) [2023-12-26 19:03:43,930][105692] Updated weights for policy 0, policy_version 507751 (0.0007) [2023-12-26 19:03:43,977][105692] Updated weights for policy 0, policy_version 507761 (0.0005) [2023-12-26 19:03:44,045][105692] Updated weights for policy 0, policy_version 507771 (0.0007) [2023-12-26 19:03:44,509][105620] Updated weights for policy 1, policy_version 508272 (0.0009) [2023-12-26 19:03:44,558][105620] Updated weights for policy 1, policy_version 508282 (0.0008) [2023-12-26 19:03:44,621][105620] Updated weights for policy 1, policy_version 508292 (0.0005) [2023-12-26 19:03:44,639][105692] Updated weights for policy 0, policy_version 507781 (0.0009) [2023-12-26 19:03:44,685][105692] Updated weights for policy 0, policy_version 507791 (0.0007) [2023-12-26 19:03:44,740][105692] Updated weights for policy 0, policy_version 507801 (0.0010) [2023-12-26 19:03:45,361][105620] Updated weights for policy 1, policy_version 508302 (0.0009) [2023-12-26 19:03:45,435][105620] Updated weights for policy 1, policy_version 508312 (0.0010) [2023-12-26 19:03:45,454][105692] Updated weights for policy 0, policy_version 507811 (0.0008) [2023-12-26 19:03:45,497][105620] Updated weights for policy 1, policy_version 508322 (0.0010) [2023-12-26 19:03:45,509][105692] Updated weights for policy 0, policy_version 507821 (0.0007) [2023-12-26 19:03:45,561][105692] Updated weights for policy 0, policy_version 507831 (0.0010) [2023-12-26 19:03:46,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 260169728. Throughput: 0: 9846.3, 1: 9816.3. Samples: 260140496. Policy #0 lag: (min: 26.0, avg: 43.7, max: 58.0) [2023-12-26 19:03:46,063][104569] Avg episode reward: [(0, '9089.340'), (1, '9089.322')] [2023-12-26 19:03:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000507840_130023424.pth... [2023-12-26 19:03:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000508328_130146304.pth... [2023-12-26 19:03:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000506656_129720320.pth [2023-12-26 19:03:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000507208_129859584.pth [2023-12-26 19:03:46,200][105692] Updated weights for policy 0, policy_version 507841 (0.0010) [2023-12-26 19:03:46,216][105620] Updated weights for policy 1, policy_version 508332 (0.0010) [2023-12-26 19:03:46,256][105692] Updated weights for policy 0, policy_version 507851 (0.0008) [2023-12-26 19:03:46,271][105620] Updated weights for policy 1, policy_version 508342 (0.0010) [2023-12-26 19:03:46,311][105692] Updated weights for policy 0, policy_version 507861 (0.0011) [2023-12-26 19:03:46,332][105620] Updated weights for policy 1, policy_version 508352 (0.0011) [2023-12-26 19:03:46,368][105692] Updated weights for policy 0, policy_version 507871 (0.0005) [2023-12-26 19:03:47,048][105692] Updated weights for policy 0, policy_version 507881 (0.0008) [2023-12-26 19:03:47,051][105620] Updated weights for policy 1, policy_version 508362 (0.0010) [2023-12-26 19:03:47,101][105692] Updated weights for policy 0, policy_version 507891 (0.0008) [2023-12-26 19:03:47,103][105620] Updated weights for policy 1, policy_version 508372 (0.0006) [2023-12-26 19:03:47,151][105620] Updated weights for policy 1, policy_version 508382 (0.0007) [2023-12-26 19:03:47,161][105692] Updated weights for policy 0, policy_version 507901 (0.0008) [2023-12-26 19:03:47,210][105620] Updated weights for policy 1, policy_version 508392 (0.0006) [2023-12-26 19:03:47,793][105620] Updated weights for policy 1, policy_version 508402 (0.0006) [2023-12-26 19:03:47,805][105692] Updated weights for policy 0, policy_version 507911 (0.0008) [2023-12-26 19:03:47,846][105620] Updated weights for policy 1, policy_version 508412 (0.0005) [2023-12-26 19:03:47,858][105692] Updated weights for policy 0, policy_version 507921 (0.0009) [2023-12-26 19:03:47,898][105620] Updated weights for policy 1, policy_version 508422 (0.0008) [2023-12-26 19:03:47,911][105692] Updated weights for policy 0, policy_version 507931 (0.0007) [2023-12-26 19:03:48,569][105620] Updated weights for policy 1, policy_version 508432 (0.0009) [2023-12-26 19:03:48,631][105620] Updated weights for policy 1, policy_version 508442 (0.0009) [2023-12-26 19:03:48,690][105620] Updated weights for policy 1, policy_version 508452 (0.0008) [2023-12-26 19:03:48,696][105692] Updated weights for policy 0, policy_version 507941 (0.0008) [2023-12-26 19:03:48,759][105692] Updated weights for policy 0, policy_version 507951 (0.0008) [2023-12-26 19:03:48,816][105692] Updated weights for policy 0, policy_version 507961 (0.0008) [2023-12-26 19:03:49,487][105620] Updated weights for policy 1, policy_version 508462 (0.0008) [2023-12-26 19:03:49,523][105692] Updated weights for policy 0, policy_version 507971 (0.0006) [2023-12-26 19:03:49,546][105620] Updated weights for policy 1, policy_version 508472 (0.0008) [2023-12-26 19:03:49,577][105692] Updated weights for policy 0, policy_version 507981 (0.0005) [2023-12-26 19:03:49,592][105620] Updated weights for policy 1, policy_version 508482 (0.0007) [2023-12-26 19:03:49,624][105692] Updated weights for policy 0, policy_version 507991 (0.0006) [2023-12-26 19:03:50,377][105620] Updated weights for policy 1, policy_version 508492 (0.0008) [2023-12-26 19:03:50,379][105692] Updated weights for policy 0, policy_version 508001 (0.0008) [2023-12-26 19:03:50,434][105620] Updated weights for policy 1, policy_version 508502 (0.0007) [2023-12-26 19:03:50,436][105692] Updated weights for policy 0, policy_version 508011 (0.0007) [2023-12-26 19:03:50,485][105692] Updated weights for policy 0, policy_version 508021 (0.0006) [2023-12-26 19:03:50,491][105620] Updated weights for policy 1, policy_version 508512 (0.0007) [2023-12-26 19:03:50,531][105692] Updated weights for policy 0, policy_version 508031 (0.0006) [2023-12-26 19:03:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 260268032. Throughput: 0: 9932.3, 1: 9863.5. Samples: 260260908. Policy #0 lag: (min: 26.0, avg: 43.7, max: 58.0) [2023-12-26 19:03:51,062][104569] Avg episode reward: [(0, '8910.232'), (1, '8997.849')] [2023-12-26 19:03:51,215][105620] Updated weights for policy 1, policy_version 508522 (0.0007) [2023-12-26 19:03:51,280][105620] Updated weights for policy 1, policy_version 508532 (0.0008) [2023-12-26 19:03:51,304][105692] Updated weights for policy 0, policy_version 508041 (0.0006) [2023-12-26 19:03:51,343][105620] Updated weights for policy 1, policy_version 508542 (0.0008) [2023-12-26 19:03:51,363][105692] Updated weights for policy 0, policy_version 508051 (0.0007) [2023-12-26 19:03:51,416][105620] Updated weights for policy 1, policy_version 508552 (0.0009) [2023-12-26 19:03:51,433][105692] Updated weights for policy 0, policy_version 508061 (0.0007) [2023-12-26 19:03:52,089][105620] Updated weights for policy 1, policy_version 508562 (0.0009) [2023-12-26 19:03:52,145][105620] Updated weights for policy 1, policy_version 508572 (0.0008) [2023-12-26 19:03:52,194][105620] Updated weights for policy 1, policy_version 508582 (0.0008) [2023-12-26 19:03:52,204][105692] Updated weights for policy 0, policy_version 508071 (0.0006) [2023-12-26 19:03:52,270][105692] Updated weights for policy 0, policy_version 508081 (0.0007) [2023-12-26 19:03:52,326][105692] Updated weights for policy 0, policy_version 508091 (0.0008) [2023-12-26 19:03:52,976][105620] Updated weights for policy 1, policy_version 508592 (0.0009) [2023-12-26 19:03:53,031][105620] Updated weights for policy 1, policy_version 508602 (0.0008) [2023-12-26 19:03:53,041][105692] Updated weights for policy 0, policy_version 508101 (0.0009) [2023-12-26 19:03:53,080][105620] Updated weights for policy 1, policy_version 508612 (0.0006) [2023-12-26 19:03:53,094][105692] Updated weights for policy 0, policy_version 508111 (0.0007) [2023-12-26 19:03:53,153][105692] Updated weights for policy 0, policy_version 508121 (0.0008) [2023-12-26 19:03:53,732][105620] Updated weights for policy 1, policy_version 508622 (0.0009) [2023-12-26 19:03:53,789][105620] Updated weights for policy 1, policy_version 508632 (0.0006) [2023-12-26 19:03:53,842][105620] Updated weights for policy 1, policy_version 508642 (0.0005) [2023-12-26 19:03:53,955][105692] Updated weights for policy 0, policy_version 508131 (0.0008) [2023-12-26 19:03:54,006][105692] Updated weights for policy 0, policy_version 508141 (0.0005) [2023-12-26 19:03:54,056][105692] Updated weights for policy 0, policy_version 508151 (0.0006) [2023-12-26 19:03:54,455][105620] Updated weights for policy 1, policy_version 508652 (0.0008) [2023-12-26 19:03:54,520][105620] Updated weights for policy 1, policy_version 508662 (0.0010) [2023-12-26 19:03:54,571][105620] Updated weights for policy 1, policy_version 508672 (0.0010) [2023-12-26 19:03:54,703][105692] Updated weights for policy 0, policy_version 508161 (0.0006) [2023-12-26 19:03:54,760][105692] Updated weights for policy 0, policy_version 508171 (0.0010) [2023-12-26 19:03:54,815][105692] Updated weights for policy 0, policy_version 508181 (0.0010) [2023-12-26 19:03:54,869][105692] Updated weights for policy 0, policy_version 508191 (0.0010) [2023-12-26 19:03:55,245][105620] Updated weights for policy 1, policy_version 508682 (0.0010) [2023-12-26 19:03:55,304][105620] Updated weights for policy 1, policy_version 508692 (0.0008) [2023-12-26 19:03:55,360][105620] Updated weights for policy 1, policy_version 508702 (0.0007) [2023-12-26 19:03:55,415][105620] Updated weights for policy 1, policy_version 508712 (0.0008) [2023-12-26 19:03:55,581][105692] Updated weights for policy 0, policy_version 508201 (0.0010) [2023-12-26 19:03:55,625][105692] Updated weights for policy 0, policy_version 508211 (0.0010) [2023-12-26 19:03:55,676][105692] Updated weights for policy 0, policy_version 508221 (0.0010) [2023-12-26 19:03:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 260366336. Throughput: 0: 9964.1, 1: 9798.2. Samples: 260377104. Policy #0 lag: (min: 26.0, avg: 43.7, max: 58.0) [2023-12-26 19:03:56,063][104569] Avg episode reward: [(0, '8819.686'), (1, '9265.436')] [2023-12-26 19:03:56,157][105620] Updated weights for policy 1, policy_version 508722 (0.0008) [2023-12-26 19:03:56,207][105620] Updated weights for policy 1, policy_version 508732 (0.0009) [2023-12-26 19:03:56,275][105620] Updated weights for policy 1, policy_version 508742 (0.0008) [2023-12-26 19:03:56,352][105692] Updated weights for policy 0, policy_version 508231 (0.0007) [2023-12-26 19:03:56,403][105692] Updated weights for policy 0, policy_version 508241 (0.0007) [2023-12-26 19:03:56,451][105692] Updated weights for policy 0, policy_version 508251 (0.0010) [2023-12-26 19:03:56,854][105620] Updated weights for policy 1, policy_version 508752 (0.0006) [2023-12-26 19:03:56,902][105620] Updated weights for policy 1, policy_version 508762 (0.0006) [2023-12-26 19:03:56,953][105620] Updated weights for policy 1, policy_version 508772 (0.0010) [2023-12-26 19:03:57,018][105692] Updated weights for policy 0, policy_version 508261 (0.0009) [2023-12-26 19:03:57,073][105692] Updated weights for policy 0, policy_version 508271 (0.0010) [2023-12-26 19:03:57,128][105692] Updated weights for policy 0, policy_version 508281 (0.0010) [2023-12-26 19:03:57,582][105620] Updated weights for policy 1, policy_version 508782 (0.0010) [2023-12-26 19:03:57,628][105620] Updated weights for policy 1, policy_version 508792 (0.0007) [2023-12-26 19:03:57,680][105620] Updated weights for policy 1, policy_version 508802 (0.0005) [2023-12-26 19:03:57,859][105692] Updated weights for policy 0, policy_version 508291 (0.0010) [2023-12-26 19:03:57,913][105692] Updated weights for policy 0, policy_version 508301 (0.0010) [2023-12-26 19:03:57,977][105692] Updated weights for policy 0, policy_version 508311 (0.0010) [2023-12-26 19:03:58,318][105620] Updated weights for policy 1, policy_version 508812 (0.0006) [2023-12-26 19:03:58,391][105620] Updated weights for policy 1, policy_version 508822 (0.0008) [2023-12-26 19:03:58,471][105620] Updated weights for policy 1, policy_version 508832 (0.0010) [2023-12-26 19:03:58,778][105692] Updated weights for policy 0, policy_version 508321 (0.0009) [2023-12-26 19:03:58,843][105692] Updated weights for policy 0, policy_version 508331 (0.0009) [2023-12-26 19:03:58,905][105692] Updated weights for policy 0, policy_version 508341 (0.0010) [2023-12-26 19:03:58,974][105692] Updated weights for policy 0, policy_version 508351 (0.0007) [2023-12-26 19:03:59,228][105620] Updated weights for policy 1, policy_version 508842 (0.0008) [2023-12-26 19:03:59,284][105620] Updated weights for policy 1, policy_version 508852 (0.0008) [2023-12-26 19:03:59,349][105620] Updated weights for policy 1, policy_version 508862 (0.0007) [2023-12-26 19:03:59,413][105620] Updated weights for policy 1, policy_version 508872 (0.0007) [2023-12-26 19:03:59,676][105692] Updated weights for policy 0, policy_version 508361 (0.0009) [2023-12-26 19:03:59,727][105692] Updated weights for policy 0, policy_version 508371 (0.0006) [2023-12-26 19:03:59,783][105692] Updated weights for policy 0, policy_version 508381 (0.0005) [2023-12-26 19:04:00,007][105620] Updated weights for policy 1, policy_version 508882 (0.0008) [2023-12-26 19:04:00,070][105620] Updated weights for policy 1, policy_version 508892 (0.0008) [2023-12-26 19:04:00,127][105620] Updated weights for policy 1, policy_version 508902 (0.0007) [2023-12-26 19:04:00,424][105692] Updated weights for policy 0, policy_version 508391 (0.0009) [2023-12-26 19:04:00,489][105692] Updated weights for policy 0, policy_version 508401 (0.0010) [2023-12-26 19:04:00,553][105692] Updated weights for policy 0, policy_version 508411 (0.0010) [2023-12-26 19:04:00,837][105620] Updated weights for policy 1, policy_version 508912 (0.0006) [2023-12-26 19:04:00,893][105620] Updated weights for policy 1, policy_version 508922 (0.0006) [2023-12-26 19:04:00,944][105620] Updated weights for policy 1, policy_version 508932 (0.0010) [2023-12-26 19:04:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 260472832. Throughput: 0: 10017.6, 1: 9845.0. Samples: 260438552. Policy #0 lag: (min: 26.0, avg: 43.7, max: 58.0) [2023-12-26 19:04:01,062][104569] Avg episode reward: [(0, '8996.832'), (1, '9175.868')] [2023-12-26 19:04:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000508416_130170880.pth... [2023-12-26 19:04:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000508936_130301952.pth... [2023-12-26 19:04:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000507264_129875968.pth [2023-12-26 19:04:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000507784_130007040.pth [2023-12-26 19:04:01,287][105692] Updated weights for policy 0, policy_version 508421 (0.0010) [2023-12-26 19:04:01,353][105692] Updated weights for policy 0, policy_version 508431 (0.0010) [2023-12-26 19:04:01,408][105692] Updated weights for policy 0, policy_version 508441 (0.0010) [2023-12-26 19:04:01,654][105620] Updated weights for policy 1, policy_version 508942 (0.0009) [2023-12-26 19:04:01,717][105620] Updated weights for policy 1, policy_version 508952 (0.0006) [2023-12-26 19:04:01,770][105620] Updated weights for policy 1, policy_version 508962 (0.0006) [2023-12-26 19:04:02,146][105692] Updated weights for policy 0, policy_version 508451 (0.0010) [2023-12-26 19:04:02,207][105692] Updated weights for policy 0, policy_version 508461 (0.0008) [2023-12-26 19:04:02,278][105692] Updated weights for policy 0, policy_version 508471 (0.0008) [2023-12-26 19:04:02,470][105620] Updated weights for policy 1, policy_version 508972 (0.0007) [2023-12-26 19:04:02,531][105620] Updated weights for policy 1, policy_version 508982 (0.0009) [2023-12-26 19:04:02,582][105620] Updated weights for policy 1, policy_version 508992 (0.0009) [2023-12-26 19:04:02,900][105692] Updated weights for policy 0, policy_version 508481 (0.0010) [2023-12-26 19:04:02,953][105692] Updated weights for policy 0, policy_version 508491 (0.0005) [2023-12-26 19:04:03,018][105692] Updated weights for policy 0, policy_version 508501 (0.0006) [2023-12-26 19:04:03,076][105692] Updated weights for policy 0, policy_version 508511 (0.0005) [2023-12-26 19:04:03,309][105620] Updated weights for policy 1, policy_version 509002 (0.0009) [2023-12-26 19:04:03,362][105620] Updated weights for policy 1, policy_version 509012 (0.0009) [2023-12-26 19:04:03,418][105620] Updated weights for policy 1, policy_version 509022 (0.0009) [2023-12-26 19:04:03,475][105620] Updated weights for policy 1, policy_version 509032 (0.0009) [2023-12-26 19:04:03,753][105692] Updated weights for policy 0, policy_version 508521 (0.0009) [2023-12-26 19:04:03,798][105692] Updated weights for policy 0, policy_version 508531 (0.0008) [2023-12-26 19:04:03,862][105692] Updated weights for policy 0, policy_version 508541 (0.0008) [2023-12-26 19:04:04,210][105620] Updated weights for policy 1, policy_version 509042 (0.0008) [2023-12-26 19:04:04,279][105620] Updated weights for policy 1, policy_version 509052 (0.0006) [2023-12-26 19:04:04,336][105620] Updated weights for policy 1, policy_version 509062 (0.0006) [2023-12-26 19:04:04,723][105692] Updated weights for policy 0, policy_version 508551 (0.0009) [2023-12-26 19:04:04,786][105692] Updated weights for policy 0, policy_version 508561 (0.0009) [2023-12-26 19:04:04,850][105692] Updated weights for policy 0, policy_version 508571 (0.0010) [2023-12-26 19:04:05,014][105620] Updated weights for policy 1, policy_version 509072 (0.0009) [2023-12-26 19:04:05,070][105620] Updated weights for policy 1, policy_version 509082 (0.0009) [2023-12-26 19:04:05,123][105620] Updated weights for policy 1, policy_version 509092 (0.0009) [2023-12-26 19:04:05,681][105692] Updated weights for policy 0, policy_version 508581 (0.0008) [2023-12-26 19:04:05,728][105692] Updated weights for policy 0, policy_version 508591 (0.0009) [2023-12-26 19:04:05,759][105620] Updated weights for policy 1, policy_version 509102 (0.0007) [2023-12-26 19:04:05,770][105692] Updated weights for policy 0, policy_version 508601 (0.0006) [2023-12-26 19:04:05,804][105620] Updated weights for policy 1, policy_version 509112 (0.0007) [2023-12-26 19:04:05,855][105620] Updated weights for policy 1, policy_version 509122 (0.0009) [2023-12-26 19:04:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19933.8, 300 sec: 19577.5). Total num frames: 260571136. Throughput: 0: 10050.6, 1: 9837.2. Samples: 260556748. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-12-26 19:04:06,062][104569] Avg episode reward: [(0, '9178.544'), (1, '9266.068')] [2023-12-26 19:04:06,593][105692] Updated weights for policy 0, policy_version 508611 (0.0007) [2023-12-26 19:04:06,600][105620] Updated weights for policy 1, policy_version 509132 (0.0007) [2023-12-26 19:04:06,647][105692] Updated weights for policy 0, policy_version 508621 (0.0007) [2023-12-26 19:04:06,662][105620] Updated weights for policy 1, policy_version 509142 (0.0006) [2023-12-26 19:04:06,696][105692] Updated weights for policy 0, policy_version 508631 (0.0007) [2023-12-26 19:04:06,722][105620] Updated weights for policy 1, policy_version 509152 (0.0008) [2023-12-26 19:04:07,432][105692] Updated weights for policy 0, policy_version 508641 (0.0007) [2023-12-26 19:04:07,481][105620] Updated weights for policy 1, policy_version 509162 (0.0007) [2023-12-26 19:04:07,491][105692] Updated weights for policy 0, policy_version 508651 (0.0008) [2023-12-26 19:04:07,538][105620] Updated weights for policy 1, policy_version 509172 (0.0006) [2023-12-26 19:04:07,548][105692] Updated weights for policy 0, policy_version 508661 (0.0007) [2023-12-26 19:04:07,597][105620] Updated weights for policy 1, policy_version 509182 (0.0007) [2023-12-26 19:04:07,603][105692] Updated weights for policy 0, policy_version 508671 (0.0009) [2023-12-26 19:04:07,655][105620] Updated weights for policy 1, policy_version 509192 (0.0009) [2023-12-26 19:04:08,376][105692] Updated weights for policy 0, policy_version 508681 (0.0008) [2023-12-26 19:04:08,422][105620] Updated weights for policy 1, policy_version 509202 (0.0007) [2023-12-26 19:04:08,433][105692] Updated weights for policy 0, policy_version 508691 (0.0006) [2023-12-26 19:04:08,480][105620] Updated weights for policy 1, policy_version 509212 (0.0006) [2023-12-26 19:04:08,494][105692] Updated weights for policy 0, policy_version 508701 (0.0007) [2023-12-26 19:04:08,543][105620] Updated weights for policy 1, policy_version 509222 (0.0008) [2023-12-26 19:04:09,272][105692] Updated weights for policy 0, policy_version 508711 (0.0007) [2023-12-26 19:04:09,273][105620] Updated weights for policy 1, policy_version 509232 (0.0008) [2023-12-26 19:04:09,339][105620] Updated weights for policy 1, policy_version 509242 (0.0007) [2023-12-26 19:04:09,343][105692] Updated weights for policy 0, policy_version 508721 (0.0009) [2023-12-26 19:04:09,404][105620] Updated weights for policy 1, policy_version 509252 (0.0007) [2023-12-26 19:04:09,410][105692] Updated weights for policy 0, policy_version 508731 (0.0009) [2023-12-26 19:04:10,138][105620] Updated weights for policy 1, policy_version 509262 (0.0006) [2023-12-26 19:04:10,202][105620] Updated weights for policy 1, policy_version 509272 (0.0008) [2023-12-26 19:04:10,223][105692] Updated weights for policy 0, policy_version 508741 (0.0008) [2023-12-26 19:04:10,261][105620] Updated weights for policy 1, policy_version 509282 (0.0008) [2023-12-26 19:04:10,273][105692] Updated weights for policy 0, policy_version 508751 (0.0006) [2023-12-26 19:04:10,325][105692] Updated weights for policy 0, policy_version 508761 (0.0008) [2023-12-26 19:04:10,963][105620] Updated weights for policy 1, policy_version 509292 (0.0009) [2023-12-26 19:04:11,017][105620] Updated weights for policy 1, policy_version 509302 (0.0008) [2023-12-26 19:04:11,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 260653056. Throughput: 0: 9846.8, 1: 9794.9. Samples: 260667868. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-12-26 19:04:11,062][104569] Avg episode reward: [(0, '9179.363'), (1, '9177.950')] [2023-12-26 19:04:11,085][105620] Updated weights for policy 1, policy_version 509312 (0.0009) [2023-12-26 19:04:11,104][105692] Updated weights for policy 0, policy_version 508771 (0.0008) [2023-12-26 19:04:11,168][105692] Updated weights for policy 0, policy_version 508781 (0.0007) [2023-12-26 19:04:11,233][105692] Updated weights for policy 0, policy_version 508791 (0.0008) [2023-12-26 19:04:11,861][105620] Updated weights for policy 1, policy_version 509322 (0.0008) [2023-12-26 19:04:11,911][105620] Updated weights for policy 1, policy_version 509332 (0.0005) [2023-12-26 19:04:11,973][105620] Updated weights for policy 1, policy_version 509342 (0.0006) [2023-12-26 19:04:12,007][105692] Updated weights for policy 0, policy_version 508801 (0.0008) [2023-12-26 19:04:12,040][105620] Updated weights for policy 1, policy_version 509352 (0.0008) [2023-12-26 19:04:12,072][105692] Updated weights for policy 0, policy_version 508811 (0.0007) [2023-12-26 19:04:12,133][105692] Updated weights for policy 0, policy_version 508821 (0.0008) [2023-12-26 19:04:12,194][105692] Updated weights for policy 0, policy_version 508831 (0.0010) [2023-12-26 19:04:12,768][105620] Updated weights for policy 1, policy_version 509362 (0.0009) [2023-12-26 19:04:12,831][105620] Updated weights for policy 1, policy_version 509372 (0.0009) [2023-12-26 19:04:12,859][105692] Updated weights for policy 0, policy_version 508841 (0.0007) [2023-12-26 19:04:12,889][105620] Updated weights for policy 1, policy_version 509382 (0.0008) [2023-12-26 19:04:12,923][105692] Updated weights for policy 0, policy_version 508851 (0.0007) [2023-12-26 19:04:12,981][105692] Updated weights for policy 0, policy_version 508861 (0.0009) [2023-12-26 19:04:13,567][105620] Updated weights for policy 1, policy_version 509392 (0.0006) [2023-12-26 19:04:13,569][105692] Updated weights for policy 0, policy_version 508871 (0.0008) [2023-12-26 19:04:13,620][105620] Updated weights for policy 1, policy_version 509402 (0.0007) [2023-12-26 19:04:13,624][105692] Updated weights for policy 0, policy_version 508881 (0.0008) [2023-12-26 19:04:13,681][105620] Updated weights for policy 1, policy_version 509412 (0.0007) [2023-12-26 19:04:13,687][105692] Updated weights for policy 0, policy_version 508891 (0.0008) [2023-12-26 19:04:14,329][105620] Updated weights for policy 1, policy_version 509422 (0.0006) [2023-12-26 19:04:14,385][105620] Updated weights for policy 1, policy_version 509432 (0.0008) [2023-12-26 19:04:14,449][105620] Updated weights for policy 1, policy_version 509442 (0.0009) [2023-12-26 19:04:14,476][105692] Updated weights for policy 0, policy_version 508901 (0.0007) [2023-12-26 19:04:14,534][105692] Updated weights for policy 0, policy_version 508911 (0.0009) [2023-12-26 19:04:14,581][105692] Updated weights for policy 0, policy_version 508921 (0.0009) [2023-12-26 19:04:15,222][105620] Updated weights for policy 1, policy_version 509452 (0.0009) [2023-12-26 19:04:15,276][105620] Updated weights for policy 1, policy_version 509462 (0.0009) [2023-12-26 19:04:15,303][105692] Updated weights for policy 0, policy_version 508931 (0.0008) [2023-12-26 19:04:15,338][105620] Updated weights for policy 1, policy_version 509472 (0.0007) [2023-12-26 19:04:15,368][105692] Updated weights for policy 0, policy_version 508941 (0.0008) [2023-12-26 19:04:15,426][105692] Updated weights for policy 0, policy_version 508951 (0.0009) [2023-12-26 19:04:15,949][105620] Updated weights for policy 1, policy_version 509482 (0.0006) [2023-12-26 19:04:15,995][105620] Updated weights for policy 1, policy_version 509492 (0.0008) [2023-12-26 19:04:16,048][105620] Updated weights for policy 1, policy_version 509502 (0.0009) [2023-12-26 19:04:16,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19524.4, 300 sec: 19522.0). Total num frames: 260751360. Throughput: 0: 9784.5, 1: 9727.9. Samples: 260726868. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-12-26 19:04:16,062][104569] Avg episode reward: [(0, '9269.423'), (1, '9178.091')] [2023-12-26 19:04:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000508960_130310144.pth... [2023-12-26 19:04:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000507840_130023424.pth [2023-12-26 19:04:16,101][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000509512_130449408.pth... [2023-12-26 19:04:16,102][105620] Updated weights for policy 1, policy_version 509512 (0.0009) [2023-12-26 19:04:16,104][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000508328_130146304.pth [2023-12-26 19:04:16,251][105692] Updated weights for policy 0, policy_version 508961 (0.0009) [2023-12-26 19:04:16,302][105692] Updated weights for policy 0, policy_version 508971 (0.0009) [2023-12-26 19:04:16,353][105692] Updated weights for policy 0, policy_version 508981 (0.0009) [2023-12-26 19:04:16,412][105692] Updated weights for policy 0, policy_version 508991 (0.0009) [2023-12-26 19:04:16,892][105620] Updated weights for policy 1, policy_version 509522 (0.0009) [2023-12-26 19:04:16,952][105620] Updated weights for policy 1, policy_version 509532 (0.0009) [2023-12-26 19:04:17,005][105620] Updated weights for policy 1, policy_version 509542 (0.0008) [2023-12-26 19:04:17,201][105692] Updated weights for policy 0, policy_version 509001 (0.0008) [2023-12-26 19:04:17,256][105692] Updated weights for policy 0, policy_version 509011 (0.0009) [2023-12-26 19:04:17,311][105692] Updated weights for policy 0, policy_version 509021 (0.0009) [2023-12-26 19:04:17,719][105620] Updated weights for policy 1, policy_version 509552 (0.0009) [2023-12-26 19:04:17,765][105620] Updated weights for policy 1, policy_version 509562 (0.0009) [2023-12-26 19:04:17,815][105620] Updated weights for policy 1, policy_version 509572 (0.0008) [2023-12-26 19:04:18,088][105692] Updated weights for policy 0, policy_version 509031 (0.0009) [2023-12-26 19:04:18,139][105692] Updated weights for policy 0, policy_version 509041 (0.0009) [2023-12-26 19:04:18,203][105692] Updated weights for policy 0, policy_version 509051 (0.0009) [2023-12-26 19:04:18,572][105620] Updated weights for policy 1, policy_version 509582 (0.0009) [2023-12-26 19:04:18,619][105620] Updated weights for policy 1, policy_version 509592 (0.0008) [2023-12-26 19:04:18,677][105620] Updated weights for policy 1, policy_version 509602 (0.0009) [2023-12-26 19:04:18,983][105692] Updated weights for policy 0, policy_version 509061 (0.0009) [2023-12-26 19:04:19,002][105585] KL-divergence is very high: 135.7505 [2023-12-26 19:04:19,042][105692] Updated weights for policy 0, policy_version 509071 (0.0009) [2023-12-26 19:04:19,045][105585] KL-divergence is very high: 205.2905 [2023-12-26 19:04:19,092][105585] KL-divergence is very high: 153.8022 [2023-12-26 19:04:19,101][105692] Updated weights for policy 0, policy_version 509081 (0.0009) [2023-12-26 19:04:19,432][105620] Updated weights for policy 1, policy_version 509612 (0.0007) [2023-12-26 19:04:19,492][105620] Updated weights for policy 1, policy_version 509622 (0.0006) [2023-12-26 19:04:19,545][105620] Updated weights for policy 1, policy_version 509632 (0.0009) [2023-12-26 19:04:19,919][105692] Updated weights for policy 0, policy_version 509091 (0.0008) [2023-12-26 19:04:19,981][105692] Updated weights for policy 0, policy_version 509101 (0.0009) [2023-12-26 19:04:20,043][105692] Updated weights for policy 0, policy_version 509111 (0.0009) [2023-12-26 19:04:20,207][105620] Updated weights for policy 1, policy_version 509642 (0.0009) [2023-12-26 19:04:20,273][105620] Updated weights for policy 1, policy_version 509652 (0.0008) [2023-12-26 19:04:20,330][105620] Updated weights for policy 1, policy_version 509662 (0.0006) [2023-12-26 19:04:20,388][105620] Updated weights for policy 1, policy_version 509672 (0.0006) [2023-12-26 19:04:20,842][105692] Updated weights for policy 0, policy_version 509121 (0.0010) [2023-12-26 19:04:20,906][105692] Updated weights for policy 0, policy_version 509131 (0.0009) [2023-12-26 19:04:20,969][105692] Updated weights for policy 0, policy_version 509141 (0.0010) [2023-12-26 19:04:21,037][105692] Updated weights for policy 0, policy_version 509151 (0.0010) [2023-12-26 19:04:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 260849664. Throughput: 0: 9629.0, 1: 9802.1. Samples: 260838548. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-12-26 19:04:21,062][104569] Avg episode reward: [(0, '9265.775'), (1, '9172.156')] [2023-12-26 19:04:21,086][105620] Updated weights for policy 1, policy_version 509682 (0.0008) [2023-12-26 19:04:21,155][105620] Updated weights for policy 1, policy_version 509692 (0.0009) [2023-12-26 19:04:21,217][105620] Updated weights for policy 1, policy_version 509702 (0.0009) [2023-12-26 19:04:21,737][105692] Updated weights for policy 0, policy_version 509161 (0.0007) [2023-12-26 19:04:21,797][105692] Updated weights for policy 0, policy_version 509171 (0.0008) [2023-12-26 19:04:21,847][105692] Updated weights for policy 0, policy_version 509181 (0.0008) [2023-12-26 19:04:22,006][105620] Updated weights for policy 1, policy_version 509712 (0.0008) [2023-12-26 19:04:22,075][105620] Updated weights for policy 1, policy_version 509722 (0.0008) [2023-12-26 19:04:22,139][105620] Updated weights for policy 1, policy_version 509732 (0.0009) [2023-12-26 19:04:22,538][105692] Updated weights for policy 0, policy_version 509191 (0.0009) [2023-12-26 19:04:22,593][105692] Updated weights for policy 0, policy_version 509201 (0.0009) [2023-12-26 19:04:22,632][105585] KL-divergence is very high: 112.0120 [2023-12-26 19:04:22,652][105692] Updated weights for policy 0, policy_version 509211 (0.0009) [2023-12-26 19:04:22,879][105620] Updated weights for policy 1, policy_version 509742 (0.0009) [2023-12-26 19:04:22,939][105620] Updated weights for policy 1, policy_version 509752 (0.0009) [2023-12-26 19:04:22,994][105620] Updated weights for policy 1, policy_version 509762 (0.0007) [2023-12-26 19:04:23,468][105692] Updated weights for policy 0, policy_version 509221 (0.0009) [2023-12-26 19:04:23,516][105692] Updated weights for policy 0, policy_version 509231 (0.0009) [2023-12-26 19:04:23,565][105692] Updated weights for policy 0, policy_version 509241 (0.0009) [2023-12-26 19:04:23,668][105620] Updated weights for policy 1, policy_version 509772 (0.0007) [2023-12-26 19:04:23,722][105620] Updated weights for policy 1, policy_version 509782 (0.0008) [2023-12-26 19:04:23,787][105620] Updated weights for policy 1, policy_version 509792 (0.0009) [2023-12-26 19:04:24,222][105692] Updated weights for policy 0, policy_version 509251 (0.0009) [2023-12-26 19:04:24,255][105585] KL-divergence is very high: 175.0834 [2023-12-26 19:04:24,271][105692] Updated weights for policy 0, policy_version 509261 (0.0010) [2023-12-26 19:04:24,293][105585] KL-divergence is very high: 299.3243 [2023-12-26 19:04:24,319][105692] Updated weights for policy 0, policy_version 509271 (0.0010) [2023-12-26 19:04:24,334][105585] KL-divergence is very high: 284.0089 [2023-12-26 19:04:24,539][105620] Updated weights for policy 1, policy_version 509802 (0.0010) [2023-12-26 19:04:24,602][105620] Updated weights for policy 1, policy_version 509812 (0.0009) [2023-12-26 19:04:24,672][105620] Updated weights for policy 1, policy_version 509822 (0.0009) [2023-12-26 19:04:24,734][105620] Updated weights for policy 1, policy_version 509832 (0.0008) [2023-12-26 19:04:25,054][105692] Updated weights for policy 0, policy_version 509281 (0.0010) [2023-12-26 19:04:25,116][105692] Updated weights for policy 0, policy_version 509291 (0.0010) [2023-12-26 19:04:25,164][105692] Updated weights for policy 0, policy_version 509301 (0.0010) [2023-12-26 19:04:25,208][105692] Updated weights for policy 0, policy_version 509311 (0.0007) [2023-12-26 19:04:25,310][105620] Updated weights for policy 1, policy_version 509842 (0.0005) [2023-12-26 19:04:25,371][105620] Updated weights for policy 1, policy_version 509852 (0.0005) [2023-12-26 19:04:25,416][105620] Updated weights for policy 1, policy_version 509862 (0.0005) [2023-12-26 19:04:25,865][105692] Updated weights for policy 0, policy_version 509321 (0.0008) [2023-12-26 19:04:25,930][105692] Updated weights for policy 0, policy_version 509331 (0.0007) [2023-12-26 19:04:25,933][105620] Updated weights for policy 1, policy_version 509872 (0.0005) [2023-12-26 19:04:25,978][105585] KL-divergence is very high: 134.1433 [2023-12-26 19:04:25,990][105620] Updated weights for policy 1, policy_version 509882 (0.0008) [2023-12-26 19:04:25,992][105692] Updated weights for policy 0, policy_version 509341 (0.0008) [2023-12-26 19:04:26,043][105620] Updated weights for policy 1, policy_version 509892 (0.0009) [2023-12-26 19:04:26,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 260956160. Throughput: 0: 9563.3, 1: 9849.9. Samples: 260956884. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-12-26 19:04:26,062][104569] Avg episode reward: [(0, '9080.931'), (1, '9172.655')] [2023-12-26 19:04:26,551][105692] Updated weights for policy 0, policy_version 509351 (0.0009) [2023-12-26 19:04:26,610][105692] Updated weights for policy 0, policy_version 509361 (0.0010) [2023-12-26 19:04:26,658][105692] Updated weights for policy 0, policy_version 509371 (0.0009) [2023-12-26 19:04:26,861][105620] Updated weights for policy 1, policy_version 509902 (0.0010) [2023-12-26 19:04:26,919][105620] Updated weights for policy 1, policy_version 509912 (0.0010) [2023-12-26 19:04:26,979][105620] Updated weights for policy 1, policy_version 509922 (0.0010) [2023-12-26 19:04:27,411][105692] Updated weights for policy 0, policy_version 509381 (0.0009) [2023-12-26 19:04:27,455][105692] Updated weights for policy 0, policy_version 509391 (0.0008) [2023-12-26 19:04:27,509][105692] Updated weights for policy 0, policy_version 509401 (0.0008) [2023-12-26 19:04:27,694][105620] Updated weights for policy 1, policy_version 509932 (0.0008) [2023-12-26 19:04:27,762][105620] Updated weights for policy 1, policy_version 509942 (0.0005) [2023-12-26 19:04:27,819][105620] Updated weights for policy 1, policy_version 509952 (0.0005) [2023-12-26 19:04:28,223][105692] Updated weights for policy 0, policy_version 509411 (0.0009) [2023-12-26 19:04:28,274][105692] Updated weights for policy 0, policy_version 509421 (0.0006) [2023-12-26 19:04:28,343][105692] Updated weights for policy 0, policy_version 509431 (0.0007) [2023-12-26 19:04:28,429][105620] Updated weights for policy 1, policy_version 509962 (0.0010) [2023-12-26 19:04:28,477][105620] Updated weights for policy 1, policy_version 509972 (0.0008) [2023-12-26 19:04:28,536][105620] Updated weights for policy 1, policy_version 509982 (0.0008) [2023-12-26 19:04:28,594][105620] Updated weights for policy 1, policy_version 509992 (0.0008) [2023-12-26 19:04:29,051][105692] Updated weights for policy 0, policy_version 509441 (0.0007) [2023-12-26 19:04:29,112][105692] Updated weights for policy 0, policy_version 509451 (0.0005) [2023-12-26 19:04:29,178][105692] Updated weights for policy 0, policy_version 509461 (0.0009) [2023-12-26 19:04:29,241][105692] Updated weights for policy 0, policy_version 509471 (0.0011) [2023-12-26 19:04:29,390][105620] Updated weights for policy 1, policy_version 510002 (0.0009) [2023-12-26 19:04:29,448][105620] Updated weights for policy 1, policy_version 510012 (0.0009) [2023-12-26 19:04:29,508][105620] Updated weights for policy 1, policy_version 510022 (0.0007) [2023-12-26 19:04:29,937][105692] Updated weights for policy 0, policy_version 509481 (0.0008) [2023-12-26 19:04:29,999][105692] Updated weights for policy 0, policy_version 509491 (0.0008) [2023-12-26 19:04:30,057][105692] Updated weights for policy 0, policy_version 509501 (0.0009) [2023-12-26 19:04:30,199][105620] Updated weights for policy 1, policy_version 510032 (0.0005) [2023-12-26 19:04:30,264][105620] Updated weights for policy 1, policy_version 510042 (0.0005) [2023-12-26 19:04:30,330][105620] Updated weights for policy 1, policy_version 510052 (0.0005) [2023-12-26 19:04:30,782][105692] Updated weights for policy 0, policy_version 509511 (0.0009) [2023-12-26 19:04:30,843][105692] Updated weights for policy 0, policy_version 509521 (0.0009) [2023-12-26 19:04:30,899][105692] Updated weights for policy 0, policy_version 509531 (0.0007) [2023-12-26 19:04:30,948][105620] Updated weights for policy 1, policy_version 510062 (0.0006) [2023-12-26 19:04:31,003][105620] Updated weights for policy 1, policy_version 510072 (0.0005) [2023-12-26 19:04:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 261046272. Throughput: 0: 9604.3, 1: 9863.4. Samples: 261016540. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-12-26 19:04:31,063][104569] Avg episode reward: [(0, '8902.446'), (1, '9176.562')] [2023-12-26 19:04:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000509536_130457600.pth... [2023-12-26 19:04:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000508416_130170880.pth [2023-12-26 19:04:31,077][105620] Updated weights for policy 1, policy_version 510082 (0.0008) [2023-12-26 19:04:31,105][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000510088_130596864.pth... [2023-12-26 19:04:31,108][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000508936_130301952.pth [2023-12-26 19:04:31,672][105692] Updated weights for policy 0, policy_version 509541 (0.0009) [2023-12-26 19:04:31,731][105692] Updated weights for policy 0, policy_version 509551 (0.0008) [2023-12-26 19:04:31,793][105692] Updated weights for policy 0, policy_version 509561 (0.0008) [2023-12-26 19:04:31,817][105620] Updated weights for policy 1, policy_version 510092 (0.0008) [2023-12-26 19:04:31,872][105620] Updated weights for policy 1, policy_version 510102 (0.0007) [2023-12-26 19:04:31,924][105620] Updated weights for policy 1, policy_version 510112 (0.0006) [2023-12-26 19:04:32,397][105692] Updated weights for policy 0, policy_version 509571 (0.0008) [2023-12-26 19:04:32,458][105692] Updated weights for policy 0, policy_version 509581 (0.0009) [2023-12-26 19:04:32,516][105692] Updated weights for policy 0, policy_version 509591 (0.0009) [2023-12-26 19:04:32,750][105620] Updated weights for policy 1, policy_version 510122 (0.0009) [2023-12-26 19:04:32,796][105620] Updated weights for policy 1, policy_version 510132 (0.0008) [2023-12-26 19:04:32,850][105620] Updated weights for policy 1, policy_version 510142 (0.0008) [2023-12-26 19:04:32,910][105620] Updated weights for policy 1, policy_version 510152 (0.0009) [2023-12-26 19:04:33,306][105692] Updated weights for policy 0, policy_version 509601 (0.0009) [2023-12-26 19:04:33,357][105692] Updated weights for policy 0, policy_version 509611 (0.0009) [2023-12-26 19:04:33,410][105692] Updated weights for policy 0, policy_version 509621 (0.0010) [2023-12-26 19:04:33,468][105692] Updated weights for policy 0, policy_version 509631 (0.0010) [2023-12-26 19:04:33,524][105620] Updated weights for policy 1, policy_version 510162 (0.0008) [2023-12-26 19:04:33,575][105620] Updated weights for policy 1, policy_version 510172 (0.0009) [2023-12-26 19:04:33,632][105620] Updated weights for policy 1, policy_version 510182 (0.0009) [2023-12-26 19:04:34,231][105692] Updated weights for policy 0, policy_version 509641 (0.0010) [2023-12-26 19:04:34,294][105692] Updated weights for policy 0, policy_version 509651 (0.0006) [2023-12-26 19:04:34,356][105692] Updated weights for policy 0, policy_version 509661 (0.0005) [2023-12-26 19:04:34,416][105620] Updated weights for policy 1, policy_version 510192 (0.0010) [2023-12-26 19:04:34,481][105620] Updated weights for policy 1, policy_version 510202 (0.0008) [2023-12-26 19:04:34,570][105620] Updated weights for policy 1, policy_version 510212 (0.0009) [2023-12-26 19:04:35,005][105692] Updated weights for policy 0, policy_version 509671 (0.0006) [2023-12-26 19:04:35,067][105692] Updated weights for policy 0, policy_version 509681 (0.0010) [2023-12-26 19:04:35,139][105692] Updated weights for policy 0, policy_version 509691 (0.0011) [2023-12-26 19:04:35,323][105620] Updated weights for policy 1, policy_version 510222 (0.0008) [2023-12-26 19:04:35,372][105620] Updated weights for policy 1, policy_version 510232 (0.0008) [2023-12-26 19:04:35,421][105620] Updated weights for policy 1, policy_version 510242 (0.0008) [2023-12-26 19:04:35,817][105692] Updated weights for policy 0, policy_version 509701 (0.0011) [2023-12-26 19:04:35,886][105692] Updated weights for policy 0, policy_version 509711 (0.0010) [2023-12-26 19:04:35,942][105692] Updated weights for policy 0, policy_version 509721 (0.0010) [2023-12-26 19:04:36,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 261144576. Throughput: 0: 9523.9, 1: 9832.2. Samples: 261131936. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-12-26 19:04:36,063][104569] Avg episode reward: [(0, '8810.151'), (1, '9268.042')] [2023-12-26 19:04:36,172][105620] Updated weights for policy 1, policy_version 510252 (0.0008) [2023-12-26 19:04:36,238][105620] Updated weights for policy 1, policy_version 510262 (0.0008) [2023-12-26 19:04:36,299][105620] Updated weights for policy 1, policy_version 510272 (0.0006) [2023-12-26 19:04:36,693][105692] Updated weights for policy 0, policy_version 509731 (0.0010) [2023-12-26 19:04:36,754][105692] Updated weights for policy 0, policy_version 509741 (0.0007) [2023-12-26 19:04:36,814][105692] Updated weights for policy 0, policy_version 509751 (0.0008) [2023-12-26 19:04:36,919][105620] Updated weights for policy 1, policy_version 510282 (0.0006) [2023-12-26 19:04:36,965][105620] Updated weights for policy 1, policy_version 510292 (0.0008) [2023-12-26 19:04:37,017][105620] Updated weights for policy 1, policy_version 510302 (0.0008) [2023-12-26 19:04:37,074][105620] Updated weights for policy 1, policy_version 510312 (0.0005) [2023-12-26 19:04:37,479][105692] Updated weights for policy 0, policy_version 509761 (0.0011) [2023-12-26 19:04:37,545][105692] Updated weights for policy 0, policy_version 509771 (0.0011) [2023-12-26 19:04:37,597][105692] Updated weights for policy 0, policy_version 509781 (0.0010) [2023-12-26 19:04:37,653][105692] Updated weights for policy 0, policy_version 509791 (0.0010) [2023-12-26 19:04:37,870][105620] Updated weights for policy 1, policy_version 510322 (0.0008) [2023-12-26 19:04:37,929][105620] Updated weights for policy 1, policy_version 510332 (0.0009) [2023-12-26 19:04:37,980][105620] Updated weights for policy 1, policy_version 510342 (0.0009) [2023-12-26 19:04:38,377][105692] Updated weights for policy 0, policy_version 509801 (0.0008) [2023-12-26 19:04:38,436][105692] Updated weights for policy 0, policy_version 509811 (0.0006) [2023-12-26 19:04:38,498][105692] Updated weights for policy 0, policy_version 509821 (0.0008) [2023-12-26 19:04:38,719][105620] Updated weights for policy 1, policy_version 510352 (0.0007) [2023-12-26 19:04:38,778][105620] Updated weights for policy 1, policy_version 510362 (0.0009) [2023-12-26 19:04:38,826][105620] Updated weights for policy 1, policy_version 510372 (0.0009) [2023-12-26 19:04:39,181][105692] Updated weights for policy 0, policy_version 509831 (0.0009) [2023-12-26 19:04:39,239][105692] Updated weights for policy 0, policy_version 509841 (0.0010) [2023-12-26 19:04:39,304][105692] Updated weights for policy 0, policy_version 509851 (0.0010) [2023-12-26 19:04:39,632][105620] Updated weights for policy 1, policy_version 510382 (0.0008) [2023-12-26 19:04:39,692][105620] Updated weights for policy 1, policy_version 510392 (0.0008) [2023-12-26 19:04:39,752][105620] Updated weights for policy 1, policy_version 510402 (0.0008) [2023-12-26 19:04:40,050][105692] Updated weights for policy 0, policy_version 509861 (0.0010) [2023-12-26 19:04:40,113][105692] Updated weights for policy 0, policy_version 509871 (0.0011) [2023-12-26 19:04:40,177][105692] Updated weights for policy 0, policy_version 509881 (0.0011) [2023-12-26 19:04:40,471][105620] Updated weights for policy 1, policy_version 510412 (0.0009) [2023-12-26 19:04:40,525][105620] Updated weights for policy 1, policy_version 510422 (0.0008) [2023-12-26 19:04:40,581][105620] Updated weights for policy 1, policy_version 510432 (0.0008) [2023-12-26 19:04:40,868][105692] Updated weights for policy 0, policy_version 509891 (0.0011) [2023-12-26 19:04:40,921][105692] Updated weights for policy 0, policy_version 509901 (0.0009) [2023-12-26 19:04:40,973][105692] Updated weights for policy 0, policy_version 509911 (0.0009) [2023-12-26 19:04:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 261242880. Throughput: 0: 9564.5, 1: 9764.6. Samples: 261246912. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-12-26 19:04:41,063][104569] Avg episode reward: [(0, '9082.702'), (1, '9266.116')] [2023-12-26 19:04:41,421][105620] Updated weights for policy 1, policy_version 510442 (0.0008) [2023-12-26 19:04:41,477][105620] Updated weights for policy 1, policy_version 510452 (0.0008) [2023-12-26 19:04:41,532][105620] Updated weights for policy 1, policy_version 510462 (0.0008) [2023-12-26 19:04:41,600][105620] Updated weights for policy 1, policy_version 510472 (0.0008) [2023-12-26 19:04:41,784][105692] Updated weights for policy 0, policy_version 509921 (0.0009) [2023-12-26 19:04:41,844][105692] Updated weights for policy 0, policy_version 509931 (0.0008) [2023-12-26 19:04:41,901][105692] Updated weights for policy 0, policy_version 509941 (0.0007) [2023-12-26 19:04:41,963][105692] Updated weights for policy 0, policy_version 509951 (0.0009) [2023-12-26 19:04:42,400][105620] Updated weights for policy 1, policy_version 510482 (0.0009) [2023-12-26 19:04:42,462][105620] Updated weights for policy 1, policy_version 510492 (0.0009) [2023-12-26 19:04:42,528][105620] Updated weights for policy 1, policy_version 510502 (0.0009) [2023-12-26 19:04:42,710][105692] Updated weights for policy 0, policy_version 509961 (0.0009) [2023-12-26 19:04:42,767][105692] Updated weights for policy 0, policy_version 509971 (0.0007) [2023-12-26 19:04:42,835][105692] Updated weights for policy 0, policy_version 509981 (0.0005) [2023-12-26 19:04:43,215][105620] Updated weights for policy 1, policy_version 510512 (0.0006) [2023-12-26 19:04:43,282][105620] Updated weights for policy 1, policy_version 510522 (0.0006) [2023-12-26 19:04:43,335][105620] Updated weights for policy 1, policy_version 510532 (0.0005) [2023-12-26 19:04:43,429][105692] Updated weights for policy 0, policy_version 509991 (0.0009) [2023-12-26 19:04:43,473][105692] Updated weights for policy 0, policy_version 510001 (0.0010) [2023-12-26 19:04:43,517][105692] Updated weights for policy 0, policy_version 510011 (0.0010) [2023-12-26 19:04:44,020][105620] Updated weights for policy 1, policy_version 510542 (0.0008) [2023-12-26 19:04:44,074][105620] Updated weights for policy 1, policy_version 510553 (0.0010) [2023-12-26 19:04:44,126][105620] Updated weights for policy 1, policy_version 510563 (0.0009) [2023-12-26 19:04:44,182][105692] Updated weights for policy 0, policy_version 510021 (0.0008) [2023-12-26 19:04:44,239][105692] Updated weights for policy 0, policy_version 510031 (0.0007) [2023-12-26 19:04:44,297][105692] Updated weights for policy 0, policy_version 510041 (0.0008) [2023-12-26 19:04:44,803][105620] Updated weights for policy 1, policy_version 510573 (0.0009) [2023-12-26 19:04:44,867][105620] Updated weights for policy 1, policy_version 510583 (0.0006) [2023-12-26 19:04:44,934][105620] Updated weights for policy 1, policy_version 510593 (0.0006) [2023-12-26 19:04:45,085][105692] Updated weights for policy 0, policy_version 510051 (0.0009) [2023-12-26 19:04:45,144][105692] Updated weights for policy 0, policy_version 510061 (0.0009) [2023-12-26 19:04:45,190][105585] KL-divergence is very high: 109.1044 [2023-12-26 19:04:45,204][105692] Updated weights for policy 0, policy_version 510071 (0.0009) [2023-12-26 19:04:45,560][105620] Updated weights for policy 1, policy_version 510603 (0.0006) [2023-12-26 19:04:45,609][105620] Updated weights for policy 1, policy_version 510613 (0.0008) [2023-12-26 19:04:45,653][105620] Updated weights for policy 1, policy_version 510623 (0.0007) [2023-12-26 19:04:45,982][105692] Updated weights for policy 0, policy_version 510081 (0.0009) [2023-12-26 19:04:46,040][105692] Updated weights for policy 0, policy_version 510091 (0.0010) [2023-12-26 19:04:46,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 261332992. Throughput: 0: 9539.4, 1: 9700.7. Samples: 261304356. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-12-26 19:04:46,062][104569] Avg episode reward: [(0, '8954.481'), (1, '9266.043')] [2023-12-26 19:04:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000510632_130736128.pth... [2023-12-26 19:04:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000509512_130449408.pth [2023-12-26 19:04:46,091][105692] Updated weights for policy 0, policy_version 510101 (0.0010) [2023-12-26 19:04:46,136][105692] Updated weights for policy 0, policy_version 510111 (0.0010) [2023-12-26 19:04:46,139][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000510112_130605056.pth... [2023-12-26 19:04:46,142][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000508960_130310144.pth [2023-12-26 19:04:46,403][105620] Updated weights for policy 1, policy_version 510633 (0.0008) [2023-12-26 19:04:46,467][105620] Updated weights for policy 1, policy_version 510643 (0.0009) [2023-12-26 19:04:46,520][105620] Updated weights for policy 1, policy_version 510653 (0.0008) [2023-12-26 19:04:46,564][105620] Updated weights for policy 1, policy_version 510663 (0.0009) [2023-12-26 19:04:46,771][105692] Updated weights for policy 0, policy_version 510121 (0.0009) [2023-12-26 19:04:46,823][105692] Updated weights for policy 0, policy_version 510131 (0.0009) [2023-12-26 19:04:46,882][105692] Updated weights for policy 0, policy_version 510141 (0.0006) [2023-12-26 19:04:47,277][105620] Updated weights for policy 1, policy_version 510673 (0.0008) [2023-12-26 19:04:47,332][105620] Updated weights for policy 1, policy_version 510683 (0.0008) [2023-12-26 19:04:47,387][105620] Updated weights for policy 1, policy_version 510693 (0.0008) [2023-12-26 19:04:47,546][105692] Updated weights for policy 0, policy_version 510151 (0.0009) [2023-12-26 19:04:47,602][105692] Updated weights for policy 0, policy_version 510161 (0.0011) [2023-12-26 19:04:47,655][105692] Updated weights for policy 0, policy_version 510171 (0.0010) [2023-12-26 19:04:48,145][105620] Updated weights for policy 1, policy_version 510703 (0.0006) [2023-12-26 19:04:48,209][105620] Updated weights for policy 1, policy_version 510713 (0.0009) [2023-12-26 19:04:48,265][105620] Updated weights for policy 1, policy_version 510723 (0.0009) [2023-12-26 19:04:48,278][105692] Updated weights for policy 0, policy_version 510181 (0.0007) [2023-12-26 19:04:48,342][105692] Updated weights for policy 0, policy_version 510191 (0.0008) [2023-12-26 19:04:48,405][105692] Updated weights for policy 0, policy_version 510201 (0.0011) [2023-12-26 19:04:49,014][105620] Updated weights for policy 1, policy_version 510733 (0.0008) [2023-12-26 19:04:49,070][105620] Updated weights for policy 1, policy_version 510743 (0.0005) [2023-12-26 19:04:49,136][105620] Updated weights for policy 1, policy_version 510753 (0.0007) [2023-12-26 19:04:49,147][105692] Updated weights for policy 0, policy_version 510211 (0.0010) [2023-12-26 19:04:49,213][105692] Updated weights for policy 0, policy_version 510221 (0.0010) [2023-12-26 19:04:49,248][105585] KL-divergence is very high: 139.1650 [2023-12-26 19:04:49,278][105692] Updated weights for policy 0, policy_version 510231 (0.0011) [2023-12-26 19:04:49,296][105585] KL-divergence is very high: 152.6053 [2023-12-26 19:04:49,899][105620] Updated weights for policy 1, policy_version 510763 (0.0008) [2023-12-26 19:04:49,968][105620] Updated weights for policy 1, policy_version 510773 (0.0009) [2023-12-26 19:04:50,022][105620] Updated weights for policy 1, policy_version 510783 (0.0009) [2023-12-26 19:04:50,037][105692] Updated weights for policy 0, policy_version 510241 (0.0011) [2023-12-26 19:04:50,103][105692] Updated weights for policy 0, policy_version 510251 (0.0009) [2023-12-26 19:04:50,161][105692] Updated weights for policy 0, policy_version 510261 (0.0009) [2023-12-26 19:04:50,227][105692] Updated weights for policy 0, policy_version 510271 (0.0009) [2023-12-26 19:04:50,873][105620] Updated weights for policy 1, policy_version 510793 (0.0009) [2023-12-26 19:04:50,889][105692] Updated weights for policy 0, policy_version 510281 (0.0009) [2023-12-26 19:04:50,928][105620] Updated weights for policy 1, policy_version 510803 (0.0006) [2023-12-26 19:04:50,953][105692] Updated weights for policy 0, policy_version 510291 (0.0007) [2023-12-26 19:04:50,983][105620] Updated weights for policy 1, policy_version 510813 (0.0007) [2023-12-26 19:04:51,016][105692] Updated weights for policy 0, policy_version 510301 (0.0006) [2023-12-26 19:04:51,053][105620] Updated weights for policy 1, policy_version 510823 (0.0008) [2023-12-26 19:04:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 261439488. Throughput: 0: 9574.0, 1: 9671.8. Samples: 261422808. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-12-26 19:04:51,062][104569] Avg episode reward: [(0, '9009.133'), (1, '9265.210')] [2023-12-26 19:04:51,734][105692] Updated weights for policy 0, policy_version 510311 (0.0009) [2023-12-26 19:04:51,781][105620] Updated weights for policy 1, policy_version 510833 (0.0009) [2023-12-26 19:04:51,794][105692] Updated weights for policy 0, policy_version 510321 (0.0010) [2023-12-26 19:04:51,842][105620] Updated weights for policy 1, policy_version 510843 (0.0011) [2023-12-26 19:04:51,852][105692] Updated weights for policy 0, policy_version 510331 (0.0008) [2023-12-26 19:04:51,901][105620] Updated weights for policy 1, policy_version 510853 (0.0011) [2023-12-26 19:04:52,589][105620] Updated weights for policy 1, policy_version 510863 (0.0007) [2023-12-26 19:04:52,638][105692] Updated weights for policy 0, policy_version 510341 (0.0007) [2023-12-26 19:04:52,638][105620] Updated weights for policy 1, policy_version 510873 (0.0006) [2023-12-26 19:04:52,691][105620] Updated weights for policy 1, policy_version 510883 (0.0006) [2023-12-26 19:04:52,697][105692] Updated weights for policy 0, policy_version 510351 (0.0008) [2023-12-26 19:04:52,756][105692] Updated weights for policy 0, policy_version 510361 (0.0007) [2023-12-26 19:04:53,339][105692] Updated weights for policy 0, policy_version 510371 (0.0008) [2023-12-26 19:04:53,386][105692] Updated weights for policy 0, policy_version 510381 (0.0009) [2023-12-26 19:04:53,433][105692] Updated weights for policy 0, policy_version 510391 (0.0009) [2023-12-26 19:04:53,494][105620] Updated weights for policy 1, policy_version 510893 (0.0007) [2023-12-26 19:04:53,558][105620] Updated weights for policy 1, policy_version 510903 (0.0009) [2023-12-26 19:04:53,619][105620] Updated weights for policy 1, policy_version 510913 (0.0009) [2023-12-26 19:04:54,192][105692] Updated weights for policy 0, policy_version 510401 (0.0009) [2023-12-26 19:04:54,247][105692] Updated weights for policy 0, policy_version 510411 (0.0009) [2023-12-26 19:04:54,302][105692] Updated weights for policy 0, policy_version 510421 (0.0009) [2023-12-26 19:04:54,352][105620] Updated weights for policy 1, policy_version 510923 (0.0008) [2023-12-26 19:04:54,354][105692] Updated weights for policy 0, policy_version 510431 (0.0008) [2023-12-26 19:04:54,408][105620] Updated weights for policy 1, policy_version 510933 (0.0008) [2023-12-26 19:04:54,466][105620] Updated weights for policy 1, policy_version 510943 (0.0009) [2023-12-26 19:04:55,141][105620] Updated weights for policy 1, policy_version 510953 (0.0008) [2023-12-26 19:04:55,177][105692] Updated weights for policy 0, policy_version 510441 (0.0008) [2023-12-26 19:04:55,206][105620] Updated weights for policy 1, policy_version 510963 (0.0007) [2023-12-26 19:04:55,234][105692] Updated weights for policy 0, policy_version 510451 (0.0009) [2023-12-26 19:04:55,266][105620] Updated weights for policy 1, policy_version 510973 (0.0011) [2023-12-26 19:04:55,292][105692] Updated weights for policy 0, policy_version 510461 (0.0008) [2023-12-26 19:04:55,321][105620] Updated weights for policy 1, policy_version 510983 (0.0010) [2023-12-26 19:04:56,041][105692] Updated weights for policy 0, policy_version 510471 (0.0006) [2023-12-26 19:04:56,043][105620] Updated weights for policy 1, policy_version 510993 (0.0010) [2023-12-26 19:04:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 261521408. Throughput: 0: 9658.1, 1: 9652.8. Samples: 261536860. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-12-26 19:04:56,063][104569] Avg episode reward: [(0, '7930.580'), (1, '9265.264')] [2023-12-26 19:04:56,085][105585] KL-divergence is very high: 125.3758 [2023-12-26 19:04:56,091][105585] KL-divergence is very high: 154.0287 [2023-12-26 19:04:56,098][105620] Updated weights for policy 1, policy_version 511003 (0.0010) [2023-12-26 19:04:56,104][105692] Updated weights for policy 0, policy_version 510481 (0.0005) [2023-12-26 19:04:56,104][105585] KL-divergence is very high: 157.4904 [2023-12-26 19:04:56,116][105585] KL-divergence is very high: 227.1317 [2023-12-26 19:04:56,122][105585] KL-divergence is very high: 240.9261 [2023-12-26 19:04:56,127][105585] KL-divergence is very high: 244.8006 [2023-12-26 19:04:56,132][105585] KL-divergence is very high: 237.1158 [2023-12-26 19:04:56,136][105585] KL-divergence is very high: 224.6148 [2023-12-26 19:04:56,146][105585] KL-divergence is very high: 160.2672 [2023-12-26 19:04:56,154][105620] Updated weights for policy 1, policy_version 511013 (0.0010) [2023-12-26 19:04:56,155][105585] KL-divergence is very high: 176.2687 [2023-12-26 19:04:56,156][105692] Updated weights for policy 0, policy_version 510491 (0.0005) [2023-12-26 19:04:56,160][105585] KL-divergence is very high: 169.4366 [2023-12-26 19:04:56,165][105585] KL-divergence is very high: 160.9108 [2023-12-26 19:04:56,170][105585] KL-divergence is very high: 148.6231 [2023-12-26 19:04:56,174][105585] KL-divergence is very high: 135.6906 [2023-12-26 19:04:56,735][105620] Updated weights for policy 1, policy_version 511023 (0.0006) [2023-12-26 19:04:56,782][105620] Updated weights for policy 1, policy_version 511033 (0.0005) [2023-12-26 19:04:56,831][105620] Updated weights for policy 1, policy_version 511043 (0.0005) [2023-12-26 19:04:56,991][105585] KL-divergence is very high: 124.7685 [2023-12-26 19:04:56,996][105585] KL-divergence is very high: 112.6579 [2023-12-26 19:04:57,017][105692] Updated weights for policy 0, policy_version 510501 (0.0009) [2023-12-26 19:04:57,069][105692] Updated weights for policy 0, policy_version 510512 (0.0010) [2023-12-26 19:04:57,083][105585] KL-divergence is very high: 100.7525 [2023-12-26 19:04:57,110][105585] KL-divergence is very high: 104.6157 [2023-12-26 19:04:57,123][105692] Updated weights for policy 0, policy_version 510522 (0.0010) [2023-12-26 19:04:57,126][105585] KL-divergence is very high: 102.7601 [2023-12-26 19:04:57,405][105620] Updated weights for policy 1, policy_version 511053 (0.0007) [2023-12-26 19:04:57,452][105620] Updated weights for policy 1, policy_version 511063 (0.0008) [2023-12-26 19:04:57,509][105620] Updated weights for policy 1, policy_version 511073 (0.0005) [2023-12-26 19:04:57,968][105692] Updated weights for policy 0, policy_version 510532 (0.0010) [2023-12-26 19:04:58,024][105692] Updated weights for policy 0, policy_version 510542 (0.0009) [2023-12-26 19:04:58,082][105692] Updated weights for policy 0, policy_version 510553 (0.0010) [2023-12-26 19:04:58,116][105620] Updated weights for policy 1, policy_version 511083 (0.0006) [2023-12-26 19:04:58,184][105620] Updated weights for policy 1, policy_version 511093 (0.0008) [2023-12-26 19:04:58,248][105620] Updated weights for policy 1, policy_version 511103 (0.0008) [2023-12-26 19:04:58,934][105692] Updated weights for policy 0, policy_version 510563 (0.0009) [2023-12-26 19:04:58,951][105620] Updated weights for policy 1, policy_version 511113 (0.0009) [2023-12-26 19:04:59,003][105692] Updated weights for policy 0, policy_version 510573 (0.0008) [2023-12-26 19:04:59,011][105620] Updated weights for policy 1, policy_version 511123 (0.0008) [2023-12-26 19:04:59,065][105692] Updated weights for policy 0, policy_version 510583 (0.0008) [2023-12-26 19:04:59,067][105620] Updated weights for policy 1, policy_version 511133 (0.0008) [2023-12-26 19:04:59,071][105585] KL-divergence is very high: 106.5238 [2023-12-26 19:04:59,104][105585] KL-divergence is very high: 154.2056 [2023-12-26 19:04:59,118][105585] KL-divergence is very high: 140.3045 [2023-12-26 19:04:59,125][105620] Updated weights for policy 1, policy_version 511143 (0.0007) [2023-12-26 19:04:59,801][105620] Updated weights for policy 1, policy_version 511153 (0.0008) [2023-12-26 19:04:59,866][105620] Updated weights for policy 1, policy_version 511163 (0.0009) [2023-12-26 19:04:59,889][105692] Updated weights for policy 0, policy_version 510593 (0.0006) [2023-12-26 19:04:59,918][105620] Updated weights for policy 1, policy_version 511173 (0.0008) [2023-12-26 19:04:59,932][105585] KL-divergence is very high: 102.4032 [2023-12-26 19:04:59,940][105585] KL-divergence is very high: 102.6637 [2023-12-26 19:04:59,954][105692] Updated weights for policy 0, policy_version 510603 (0.0007) [2023-12-26 19:04:59,960][105585] KL-divergence is very high: 112.4506 [2023-12-26 19:05:00,016][105692] Updated weights for policy 0, policy_version 510613 (0.0009) [2023-12-26 19:05:00,051][105585] KL-divergence is very high: 102.2043 [2023-12-26 19:05:00,069][105692] Updated weights for policy 0, policy_version 510623 (0.0009) [2023-12-26 19:05:00,603][105620] Updated weights for policy 1, policy_version 511183 (0.0010) [2023-12-26 19:05:00,655][105620] Updated weights for policy 1, policy_version 511193 (0.0010) [2023-12-26 19:05:00,710][105620] Updated weights for policy 1, policy_version 511203 (0.0011) [2023-12-26 19:05:00,857][105692] Updated weights for policy 0, policy_version 510633 (0.0008) [2023-12-26 19:05:00,912][105692] Updated weights for policy 0, policy_version 510643 (0.0008) [2023-12-26 19:05:00,963][105692] Updated weights for policy 0, policy_version 510653 (0.0008) [2023-12-26 19:05:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 261627904. Throughput: 0: 9563.9, 1: 9735.5. Samples: 261595340. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-12-26 19:05:01,062][104569] Avg episode reward: [(0, '2582.190'), (1, '8998.140')] [2023-12-26 19:05:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000510656_130744320.pth... [2023-12-26 19:05:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000511208_130883584.pth... [2023-12-26 19:05:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000509536_130457600.pth [2023-12-26 19:05:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000510088_130596864.pth [2023-12-26 19:05:01,455][105620] Updated weights for policy 1, policy_version 511213 (0.0010) [2023-12-26 19:05:01,506][105620] Updated weights for policy 1, policy_version 511223 (0.0008) [2023-12-26 19:05:01,572][105620] Updated weights for policy 1, policy_version 511233 (0.0007) [2023-12-26 19:05:01,766][105692] Updated weights for policy 0, policy_version 510663 (0.0009) [2023-12-26 19:05:01,820][105692] Updated weights for policy 0, policy_version 510674 (0.0010) [2023-12-26 19:05:01,880][105692] Updated weights for policy 0, policy_version 510684 (0.0009) [2023-12-26 19:05:02,144][105620] Updated weights for policy 1, policy_version 511243 (0.0008) [2023-12-26 19:05:02,208][105620] Updated weights for policy 1, policy_version 511253 (0.0005) [2023-12-26 19:05:02,262][105620] Updated weights for policy 1, policy_version 511263 (0.0008) [2023-12-26 19:05:02,522][105692] Updated weights for policy 0, policy_version 510694 (0.0007) [2023-12-26 19:05:02,590][105692] Updated weights for policy 0, policy_version 510704 (0.0005) [2023-12-26 19:05:02,652][105692] Updated weights for policy 0, policy_version 510714 (0.0008) [2023-12-26 19:05:03,079][105620] Updated weights for policy 1, policy_version 511273 (0.0010) [2023-12-26 19:05:03,136][105620] Updated weights for policy 1, policy_version 511283 (0.0009) [2023-12-26 19:05:03,180][105620] Updated weights for policy 1, policy_version 511293 (0.0008) [2023-12-26 19:05:03,210][105692] Updated weights for policy 0, policy_version 510724 (0.0010) [2023-12-26 19:05:03,225][105620] Updated weights for policy 1, policy_version 511303 (0.0006) [2023-12-26 19:05:03,255][105692] Updated weights for policy 0, policy_version 510734 (0.0010) [2023-12-26 19:05:03,302][105692] Updated weights for policy 0, policy_version 510744 (0.0010) [2023-12-26 19:05:03,873][105692] Updated weights for policy 0, policy_version 510754 (0.0010) [2023-12-26 19:05:03,940][105692] Updated weights for policy 0, policy_version 510764 (0.0010) [2023-12-26 19:05:04,008][105692] Updated weights for policy 0, policy_version 510774 (0.0011) [2023-12-26 19:05:04,067][105692] Updated weights for policy 0, policy_version 510784 (0.0010) [2023-12-26 19:05:04,078][105620] Updated weights for policy 1, policy_version 511313 (0.0006) [2023-12-26 19:05:04,142][105620] Updated weights for policy 1, policy_version 511323 (0.0007) [2023-12-26 19:05:04,206][105620] Updated weights for policy 1, policy_version 511333 (0.0006) [2023-12-26 19:05:04,804][105692] Updated weights for policy 0, policy_version 510794 (0.0011) [2023-12-26 19:05:04,860][105692] Updated weights for policy 0, policy_version 510804 (0.0010) [2023-12-26 19:05:04,910][105692] Updated weights for policy 0, policy_version 510814 (0.0010) [2023-12-26 19:05:04,943][105620] Updated weights for policy 1, policy_version 511343 (0.0007) [2023-12-26 19:05:04,998][105620] Updated weights for policy 1, policy_version 511353 (0.0009) [2023-12-26 19:05:05,062][105620] Updated weights for policy 1, policy_version 511363 (0.0008) [2023-12-26 19:05:05,655][105692] Updated weights for policy 0, policy_version 510824 (0.0010) [2023-12-26 19:05:05,703][105692] Updated weights for policy 0, policy_version 510834 (0.0010) [2023-12-26 19:05:05,756][105692] Updated weights for policy 0, policy_version 510844 (0.0010) [2023-12-26 19:05:05,803][105620] Updated weights for policy 1, policy_version 511373 (0.0008) [2023-12-26 19:05:05,862][105620] Updated weights for policy 1, policy_version 511383 (0.0008) [2023-12-26 19:05:05,927][105620] Updated weights for policy 1, policy_version 511393 (0.0008) [2023-12-26 19:05:06,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 261726208. Throughput: 0: 9680.5, 1: 9731.4. Samples: 261712084. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-12-26 19:05:06,062][104569] Avg episode reward: [(0, '791.259'), (1, '8997.456')] [2023-12-26 19:05:06,447][105692] Updated weights for policy 0, policy_version 510854 (0.0007) [2023-12-26 19:05:06,515][105692] Updated weights for policy 0, policy_version 510864 (0.0006) [2023-12-26 19:05:06,581][105692] Updated weights for policy 0, policy_version 510874 (0.0011) [2023-12-26 19:05:06,667][105620] Updated weights for policy 1, policy_version 511403 (0.0008) [2023-12-26 19:05:06,732][105620] Updated weights for policy 1, policy_version 511413 (0.0008) [2023-12-26 19:05:06,797][105620] Updated weights for policy 1, policy_version 511423 (0.0009) [2023-12-26 19:05:07,220][105692] Updated weights for policy 0, policy_version 510884 (0.0010) [2023-12-26 19:05:07,277][105692] Updated weights for policy 0, policy_version 510894 (0.0009) [2023-12-26 19:05:07,337][105692] Updated weights for policy 0, policy_version 510904 (0.0006) [2023-12-26 19:05:07,627][105620] Updated weights for policy 1, policy_version 511433 (0.0009) [2023-12-26 19:05:07,684][105620] Updated weights for policy 1, policy_version 511443 (0.0008) [2023-12-26 19:05:07,746][105620] Updated weights for policy 1, policy_version 511453 (0.0009) [2023-12-26 19:05:07,805][105620] Updated weights for policy 1, policy_version 511463 (0.0011) [2023-12-26 19:05:07,995][105692] Updated weights for policy 0, policy_version 510914 (0.0006) [2023-12-26 19:05:08,064][105692] Updated weights for policy 0, policy_version 510924 (0.0005) [2023-12-26 19:05:08,130][105692] Updated weights for policy 0, policy_version 510934 (0.0005) [2023-12-26 19:05:08,188][105692] Updated weights for policy 0, policy_version 510944 (0.0005) [2023-12-26 19:05:08,441][105620] Updated weights for policy 1, policy_version 511473 (0.0011) [2023-12-26 19:05:08,504][105620] Updated weights for policy 1, policy_version 511483 (0.0011) [2023-12-26 19:05:08,563][105620] Updated weights for policy 1, policy_version 511493 (0.0011) [2023-12-26 19:05:08,785][105692] Updated weights for policy 0, policy_version 510954 (0.0008) [2023-12-26 19:05:08,840][105692] Updated weights for policy 0, policy_version 510964 (0.0008) [2023-12-26 19:05:08,897][105692] Updated weights for policy 0, policy_version 510974 (0.0008) [2023-12-26 19:05:09,290][105620] Updated weights for policy 1, policy_version 511503 (0.0011) [2023-12-26 19:05:09,357][105620] Updated weights for policy 1, policy_version 511513 (0.0011) [2023-12-26 19:05:09,422][105620] Updated weights for policy 1, policy_version 511523 (0.0010) [2023-12-26 19:05:09,680][105692] Updated weights for policy 0, policy_version 510984 (0.0010) [2023-12-26 19:05:09,733][105692] Updated weights for policy 0, policy_version 510994 (0.0010) [2023-12-26 19:05:09,796][105692] Updated weights for policy 0, policy_version 511004 (0.0011) [2023-12-26 19:05:10,148][105620] Updated weights for policy 1, policy_version 511533 (0.0010) [2023-12-26 19:05:10,205][105620] Updated weights for policy 1, policy_version 511543 (0.0011) [2023-12-26 19:05:10,261][105620] Updated weights for policy 1, policy_version 511553 (0.0010) [2023-12-26 19:05:10,537][105692] Updated weights for policy 0, policy_version 511014 (0.0010) [2023-12-26 19:05:10,595][105692] Updated weights for policy 0, policy_version 511024 (0.0008) [2023-12-26 19:05:10,656][105692] Updated weights for policy 0, policy_version 511034 (0.0007) [2023-12-26 19:05:10,961][105620] Updated weights for policy 1, policy_version 511563 (0.0010) [2023-12-26 19:05:11,030][105620] Updated weights for policy 1, policy_version 511573 (0.0008) [2023-12-26 19:05:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 261816320. Throughput: 0: 9726.3, 1: 9641.2. Samples: 261828424. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-12-26 19:05:11,063][104569] Avg episode reward: [(0, '1266.391'), (1, '9087.880')] [2023-12-26 19:05:11,096][105620] Updated weights for policy 1, policy_version 511583 (0.0009) [2023-12-26 19:05:11,426][105692] Updated weights for policy 0, policy_version 511044 (0.0006) [2023-12-26 19:05:11,493][105692] Updated weights for policy 0, policy_version 511054 (0.0006) [2023-12-26 19:05:11,556][105692] Updated weights for policy 0, policy_version 511064 (0.0010) [2023-12-26 19:05:11,954][105620] Updated weights for policy 1, policy_version 511593 (0.0010) [2023-12-26 19:05:12,008][105620] Updated weights for policy 1, policy_version 511603 (0.0011) [2023-12-26 19:05:12,058][105620] Updated weights for policy 1, policy_version 511613 (0.0011) [2023-12-26 19:05:12,118][105620] Updated weights for policy 1, policy_version 511623 (0.0011) [2023-12-26 19:05:12,297][105692] Updated weights for policy 0, policy_version 511074 (0.0011) [2023-12-26 19:05:12,358][105692] Updated weights for policy 0, policy_version 511084 (0.0010) [2023-12-26 19:05:12,415][105692] Updated weights for policy 0, policy_version 511094 (0.0006) [2023-12-26 19:05:12,469][105692] Updated weights for policy 0, policy_version 511104 (0.0011) [2023-12-26 19:05:12,899][105620] Updated weights for policy 1, policy_version 511633 (0.0010) [2023-12-26 19:05:12,950][105620] Updated weights for policy 1, policy_version 511643 (0.0010) [2023-12-26 19:05:12,998][105620] Updated weights for policy 1, policy_version 511653 (0.0010) [2023-12-26 19:05:13,170][105692] Updated weights for policy 0, policy_version 511114 (0.0006) [2023-12-26 19:05:13,238][105692] Updated weights for policy 0, policy_version 511124 (0.0005) [2023-12-26 19:05:13,307][105692] Updated weights for policy 0, policy_version 511134 (0.0005) [2023-12-26 19:05:13,797][105620] Updated weights for policy 1, policy_version 511663 (0.0008) [2023-12-26 19:05:13,818][105692] Updated weights for policy 0, policy_version 511144 (0.0009) [2023-12-26 19:05:13,851][105620] Updated weights for policy 1, policy_version 511673 (0.0006) [2023-12-26 19:05:13,876][105692] Updated weights for policy 0, policy_version 511154 (0.0010) [2023-12-26 19:05:13,910][105620] Updated weights for policy 1, policy_version 511683 (0.0006) [2023-12-26 19:05:13,936][105692] Updated weights for policy 0, policy_version 511164 (0.0008) [2023-12-26 19:05:14,510][105692] Updated weights for policy 0, policy_version 511174 (0.0008) [2023-12-26 19:05:14,562][105692] Updated weights for policy 0, policy_version 511184 (0.0010) [2023-12-26 19:05:14,607][105692] Updated weights for policy 0, policy_version 511194 (0.0010) [2023-12-26 19:05:14,695][105620] Updated weights for policy 1, policy_version 511693 (0.0006) [2023-12-26 19:05:14,743][105620] Updated weights for policy 1, policy_version 511703 (0.0008) [2023-12-26 19:05:14,797][105620] Updated weights for policy 1, policy_version 511713 (0.0008) [2023-12-26 19:05:15,315][105692] Updated weights for policy 0, policy_version 511204 (0.0011) [2023-12-26 19:05:15,368][105692] Updated weights for policy 0, policy_version 511214 (0.0011) [2023-12-26 19:05:15,417][105692] Updated weights for policy 0, policy_version 511224 (0.0010) [2023-12-26 19:05:15,549][105620] Updated weights for policy 1, policy_version 511723 (0.0007) [2023-12-26 19:05:15,606][105620] Updated weights for policy 1, policy_version 511733 (0.0007) [2023-12-26 19:05:15,665][105620] Updated weights for policy 1, policy_version 511743 (0.0008) [2023-12-26 19:05:16,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19387.6, 300 sec: 19494.2). Total num frames: 261914624. Throughput: 0: 9696.5, 1: 9598.0. Samples: 261884796. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-12-26 19:05:16,063][104569] Avg episode reward: [(0, '3186.631'), (1, '8995.925')] [2023-12-26 19:05:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000511752_131022848.pth... [2023-12-26 19:05:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000511232_130891776.pth... [2023-12-26 19:05:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000510632_130736128.pth [2023-12-26 19:05:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000510112_130605056.pth [2023-12-26 19:05:16,157][105692] Updated weights for policy 0, policy_version 511234 (0.0009) [2023-12-26 19:05:16,219][105692] Updated weights for policy 0, policy_version 511244 (0.0005) [2023-12-26 19:05:16,277][105692] Updated weights for policy 0, policy_version 511254 (0.0009) [2023-12-26 19:05:16,326][105692] Updated weights for policy 0, policy_version 511264 (0.0008) [2023-12-26 19:05:16,426][105620] Updated weights for policy 1, policy_version 511753 (0.0008) [2023-12-26 19:05:16,477][105620] Updated weights for policy 1, policy_version 511763 (0.0009) [2023-12-26 19:05:16,531][105620] Updated weights for policy 1, policy_version 511773 (0.0009) [2023-12-26 19:05:16,587][105620] Updated weights for policy 1, policy_version 511783 (0.0008) [2023-12-26 19:05:16,975][105692] Updated weights for policy 0, policy_version 511274 (0.0005) [2023-12-26 19:05:17,030][105692] Updated weights for policy 0, policy_version 511284 (0.0005) [2023-12-26 19:05:17,081][105692] Updated weights for policy 0, policy_version 511294 (0.0005) [2023-12-26 19:05:17,415][105620] Updated weights for policy 1, policy_version 511793 (0.0009) [2023-12-26 19:05:17,475][105620] Updated weights for policy 1, policy_version 511803 (0.0009) [2023-12-26 19:05:17,535][105620] Updated weights for policy 1, policy_version 511813 (0.0009) [2023-12-26 19:05:17,695][105692] Updated weights for policy 0, policy_version 511304 (0.0009) [2023-12-26 19:05:17,751][105692] Updated weights for policy 0, policy_version 511314 (0.0009) [2023-12-26 19:05:17,815][105692] Updated weights for policy 0, policy_version 511324 (0.0009) [2023-12-26 19:05:18,318][105620] Updated weights for policy 1, policy_version 511823 (0.0009) [2023-12-26 19:05:18,379][105620] Updated weights for policy 1, policy_version 511833 (0.0008) [2023-12-26 19:05:18,437][105620] Updated weights for policy 1, policy_version 511843 (0.0006) [2023-12-26 19:05:18,558][105692] Updated weights for policy 0, policy_version 511334 (0.0009) [2023-12-26 19:05:18,616][105692] Updated weights for policy 0, policy_version 511344 (0.0010) [2023-12-26 19:05:18,679][105692] Updated weights for policy 0, policy_version 511354 (0.0010) [2023-12-26 19:05:19,020][105620] Updated weights for policy 1, policy_version 511853 (0.0005) [2023-12-26 19:05:19,082][105620] Updated weights for policy 1, policy_version 511863 (0.0005) [2023-12-26 19:05:19,149][105620] Updated weights for policy 1, policy_version 511873 (0.0006) [2023-12-26 19:05:19,545][105692] Updated weights for policy 0, policy_version 511364 (0.0008) [2023-12-26 19:05:19,608][105692] Updated weights for policy 0, policy_version 511374 (0.0005) [2023-12-26 19:05:19,677][105692] Updated weights for policy 0, policy_version 511384 (0.0006) [2023-12-26 19:05:19,788][105620] Updated weights for policy 1, policy_version 511883 (0.0007) [2023-12-26 19:05:19,849][105620] Updated weights for policy 1, policy_version 511893 (0.0008) [2023-12-26 19:05:19,912][105620] Updated weights for policy 1, policy_version 511903 (0.0010) [2023-12-26 19:05:20,282][105692] Updated weights for policy 0, policy_version 511394 (0.0006) [2023-12-26 19:05:20,356][105692] Updated weights for policy 0, policy_version 511404 (0.0005) [2023-12-26 19:05:20,409][105692] Updated weights for policy 0, policy_version 511414 (0.0005) [2023-12-26 19:05:20,464][105692] Updated weights for policy 0, policy_version 511424 (0.0005) [2023-12-26 19:05:20,792][105620] Updated weights for policy 1, policy_version 511913 (0.0009) [2023-12-26 19:05:20,855][105620] Updated weights for policy 1, policy_version 511923 (0.0011) [2023-12-26 19:05:20,911][105620] Updated weights for policy 1, policy_version 511933 (0.0009) [2023-12-26 19:05:20,967][105620] Updated weights for policy 1, policy_version 511943 (0.0010) [2023-12-26 19:05:21,024][105692] Updated weights for policy 0, policy_version 511434 (0.0008) [2023-12-26 19:05:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 262012928. Throughput: 0: 9751.3, 1: 9581.5. Samples: 262001912. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-12-26 19:05:21,063][104569] Avg episode reward: [(0, '7080.884'), (1, '9087.778')] [2023-12-26 19:05:21,085][105692] Updated weights for policy 0, policy_version 511444 (0.0009) [2023-12-26 19:05:21,152][105692] Updated weights for policy 0, policy_version 511454 (0.0008) [2023-12-26 19:05:21,699][105620] Updated weights for policy 1, policy_version 511953 (0.0008) [2023-12-26 19:05:21,774][105620] Updated weights for policy 1, policy_version 511963 (0.0008) [2023-12-26 19:05:21,835][105620] Updated weights for policy 1, policy_version 511973 (0.0005) [2023-12-26 19:05:21,951][105692] Updated weights for policy 0, policy_version 511464 (0.0009) [2023-12-26 19:05:22,007][105692] Updated weights for policy 0, policy_version 511474 (0.0010) [2023-12-26 19:05:22,069][105692] Updated weights for policy 0, policy_version 511484 (0.0008) [2023-12-26 19:05:22,573][105620] Updated weights for policy 1, policy_version 511983 (0.0009) [2023-12-26 19:05:22,641][105620] Updated weights for policy 1, policy_version 511993 (0.0010) [2023-12-26 19:05:22,691][105620] Updated weights for policy 1, policy_version 512003 (0.0010) [2023-12-26 19:05:22,746][105692] Updated weights for policy 0, policy_version 511494 (0.0008) [2023-12-26 19:05:22,796][105692] Updated weights for policy 0, policy_version 511504 (0.0009) [2023-12-26 19:05:22,843][105692] Updated weights for policy 0, policy_version 511514 (0.0009) [2023-12-26 19:05:23,437][105620] Updated weights for policy 1, policy_version 512013 (0.0008) [2023-12-26 19:05:23,490][105620] Updated weights for policy 1, policy_version 512023 (0.0005) [2023-12-26 19:05:23,556][105620] Updated weights for policy 1, policy_version 512033 (0.0005) [2023-12-26 19:05:23,665][105692] Updated weights for policy 0, policy_version 511524 (0.0010) [2023-12-26 19:05:23,717][105692] Updated weights for policy 0, policy_version 511534 (0.0009) [2023-12-26 19:05:23,774][105692] Updated weights for policy 0, policy_version 511544 (0.0009) [2023-12-26 19:05:24,083][105620] Updated weights for policy 1, policy_version 512043 (0.0007) [2023-12-26 19:05:24,135][105620] Updated weights for policy 1, policy_version 512053 (0.0007) [2023-12-26 19:05:24,201][105620] Updated weights for policy 1, policy_version 512063 (0.0009) [2023-12-26 19:05:24,451][105692] Updated weights for policy 0, policy_version 511554 (0.0009) [2023-12-26 19:05:24,499][105692] Updated weights for policy 0, policy_version 511564 (0.0005) [2023-12-26 19:05:24,559][105692] Updated weights for policy 0, policy_version 511574 (0.0009) [2023-12-26 19:05:24,610][105692] Updated weights for policy 0, policy_version 511584 (0.0010) [2023-12-26 19:05:25,031][105620] Updated weights for policy 1, policy_version 512073 (0.0009) [2023-12-26 19:05:25,088][105620] Updated weights for policy 1, policy_version 512083 (0.0010) [2023-12-26 19:05:25,139][105620] Updated weights for policy 1, policy_version 512093 (0.0010) [2023-12-26 19:05:25,190][105620] Updated weights for policy 1, policy_version 512103 (0.0010) [2023-12-26 19:05:25,276][105692] Updated weights for policy 0, policy_version 511594 (0.0009) [2023-12-26 19:05:25,344][105692] Updated weights for policy 0, policy_version 511604 (0.0010) [2023-12-26 19:05:25,405][105692] Updated weights for policy 0, policy_version 511614 (0.0010) [2023-12-26 19:05:25,859][105620] Updated weights for policy 1, policy_version 512113 (0.0010) [2023-12-26 19:05:25,916][105620] Updated weights for policy 1, policy_version 512124 (0.0012) [2023-12-26 19:05:25,923][105586] KL-divergence is very high: 130.6581 [2023-12-26 19:05:25,976][105586] KL-divergence is very high: 154.8729 [2023-12-26 19:05:25,985][105620] Updated weights for policy 1, policy_version 512134 (0.0010) [2023-12-26 19:05:26,030][105692] Updated weights for policy 0, policy_version 511624 (0.0010) [2023-12-26 19:05:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.1, 300 sec: 19494.2). Total num frames: 262111232. Throughput: 0: 9784.9, 1: 9585.8. Samples: 262118600. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-12-26 19:05:26,064][104569] Avg episode reward: [(0, '8101.364'), (1, '8911.245')] [2023-12-26 19:05:26,082][105692] Updated weights for policy 0, policy_version 511634 (0.0010) [2023-12-26 19:05:26,146][105692] Updated weights for policy 0, policy_version 511644 (0.0010) [2023-12-26 19:05:26,767][105620] Updated weights for policy 1, policy_version 512144 (0.0009) [2023-12-26 19:05:26,771][105586] KL-divergence is very high: 103.3402 [2023-12-26 19:05:26,813][105586] KL-divergence is very high: 106.2342 [2023-12-26 19:05:26,817][105620] Updated weights for policy 1, policy_version 512154 (0.0008) [2023-12-26 19:05:26,856][105586] KL-divergence is very high: 115.0001 [2023-12-26 19:05:26,872][105620] Updated weights for policy 1, policy_version 512164 (0.0007) [2023-12-26 19:05:26,898][105692] Updated weights for policy 0, policy_version 511654 (0.0011) [2023-12-26 19:05:26,947][105692] Updated weights for policy 0, policy_version 511664 (0.0011) [2023-12-26 19:05:26,996][105692] Updated weights for policy 0, policy_version 511674 (0.0010) [2023-12-26 19:05:27,482][105620] Updated weights for policy 1, policy_version 512174 (0.0007) [2023-12-26 19:05:27,544][105620] Updated weights for policy 1, policy_version 512184 (0.0008) [2023-12-26 19:05:27,588][105620] Updated weights for policy 1, policy_version 512194 (0.0008) [2023-12-26 19:05:27,734][105692] Updated weights for policy 0, policy_version 511684 (0.0010) [2023-12-26 19:05:27,792][105692] Updated weights for policy 0, policy_version 511694 (0.0010) [2023-12-26 19:05:27,849][105692] Updated weights for policy 0, policy_version 511704 (0.0010) [2023-12-26 19:05:28,306][105620] Updated weights for policy 1, policy_version 512204 (0.0007) [2023-12-26 19:05:28,369][105620] Updated weights for policy 1, policy_version 512214 (0.0008) [2023-12-26 19:05:28,426][105620] Updated weights for policy 1, policy_version 512224 (0.0008) [2023-12-26 19:05:28,496][105692] Updated weights for policy 0, policy_version 511714 (0.0009) [2023-12-26 19:05:28,552][105692] Updated weights for policy 0, policy_version 511724 (0.0010) [2023-12-26 19:05:28,616][105692] Updated weights for policy 0, policy_version 511734 (0.0010) [2023-12-26 19:05:28,670][105692] Updated weights for policy 0, policy_version 511744 (0.0010) [2023-12-26 19:05:29,118][105620] Updated weights for policy 1, policy_version 512234 (0.0008) [2023-12-26 19:05:29,183][105620] Updated weights for policy 1, policy_version 512244 (0.0005) [2023-12-26 19:05:29,249][105620] Updated weights for policy 1, policy_version 512254 (0.0007) [2023-12-26 19:05:29,312][105620] Updated weights for policy 1, policy_version 512264 (0.0009) [2023-12-26 19:05:29,422][105692] Updated weights for policy 0, policy_version 511754 (0.0011) [2023-12-26 19:05:29,487][105692] Updated weights for policy 0, policy_version 511764 (0.0010) [2023-12-26 19:05:29,548][105692] Updated weights for policy 0, policy_version 511774 (0.0010) [2023-12-26 19:05:29,904][105620] Updated weights for policy 1, policy_version 512274 (0.0009) [2023-12-26 19:05:29,968][105620] Updated weights for policy 1, policy_version 512285 (0.0009) [2023-12-26 19:05:30,032][105620] Updated weights for policy 1, policy_version 512295 (0.0008) [2023-12-26 19:05:30,267][105692] Updated weights for policy 0, policy_version 511784 (0.0011) [2023-12-26 19:05:30,296][105585] KL-divergence is very high: 106.6365 [2023-12-26 19:05:30,325][105692] Updated weights for policy 0, policy_version 511794 (0.0011) [2023-12-26 19:05:30,343][105585] KL-divergence is very high: 173.7944 [2023-12-26 19:05:30,384][105692] Updated weights for policy 0, policy_version 511804 (0.0010) [2023-12-26 19:05:30,389][105585] KL-divergence is very high: 166.1213 [2023-12-26 19:05:30,773][105620] Updated weights for policy 1, policy_version 512305 (0.0006) [2023-12-26 19:05:30,825][105620] Updated weights for policy 1, policy_version 512315 (0.0005) [2023-12-26 19:05:30,869][105620] Updated weights for policy 1, policy_version 512325 (0.0005) [2023-12-26 19:05:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 262209536. Throughput: 0: 9818.7, 1: 9616.2. Samples: 262178932. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-12-26 19:05:31,063][104569] Avg episode reward: [(0, '7682.710'), (1, '8817.700')] [2023-12-26 19:05:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000512328_131170304.pth... [2023-12-26 19:05:31,072][105692] Updated weights for policy 0, policy_version 511814 (0.0009) [2023-12-26 19:05:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000511208_130883584.pth [2023-12-26 19:05:31,148][105692] Updated weights for policy 0, policy_version 511824 (0.0006) [2023-12-26 19:05:31,215][105692] Updated weights for policy 0, policy_version 511834 (0.0009) [2023-12-26 19:05:31,257][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000511840_131047424.pth... [2023-12-26 19:05:31,261][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000510656_130744320.pth [2023-12-26 19:05:31,531][105620] Updated weights for policy 1, policy_version 512335 (0.0005) [2023-12-26 19:05:31,598][105620] Updated weights for policy 1, policy_version 512345 (0.0005) [2023-12-26 19:05:31,670][105620] Updated weights for policy 1, policy_version 512355 (0.0009) [2023-12-26 19:05:31,856][105692] Updated weights for policy 0, policy_version 511844 (0.0008) [2023-12-26 19:05:31,914][105692] Updated weights for policy 0, policy_version 511854 (0.0011) [2023-12-26 19:05:31,966][105692] Updated weights for policy 0, policy_version 511864 (0.0010) [2023-12-26 19:05:32,381][105620] Updated weights for policy 1, policy_version 512365 (0.0008) [2023-12-26 19:05:32,435][105620] Updated weights for policy 1, policy_version 512375 (0.0005) [2023-12-26 19:05:32,481][105620] Updated weights for policy 1, policy_version 512385 (0.0005) [2023-12-26 19:05:32,723][105692] Updated weights for policy 0, policy_version 511874 (0.0010) [2023-12-26 19:05:32,780][105692] Updated weights for policy 0, policy_version 511884 (0.0010) [2023-12-26 19:05:32,837][105692] Updated weights for policy 0, policy_version 511894 (0.0010) [2023-12-26 19:05:32,891][105692] Updated weights for policy 0, policy_version 511904 (0.0010) [2023-12-26 19:05:33,157][105620] Updated weights for policy 1, policy_version 512395 (0.0005) [2023-12-26 19:05:33,210][105620] Updated weights for policy 1, policy_version 512405 (0.0005) [2023-12-26 19:05:33,262][105620] Updated weights for policy 1, policy_version 512415 (0.0005) [2023-12-26 19:05:33,628][105692] Updated weights for policy 0, policy_version 511914 (0.0011) [2023-12-26 19:05:33,678][105692] Updated weights for policy 0, policy_version 511924 (0.0010) [2023-12-26 19:05:33,739][105692] Updated weights for policy 0, policy_version 511934 (0.0010) [2023-12-26 19:05:33,896][105620] Updated weights for policy 1, policy_version 512425 (0.0006) [2023-12-26 19:05:33,957][105620] Updated weights for policy 1, policy_version 512435 (0.0010) [2023-12-26 19:05:34,021][105620] Updated weights for policy 1, policy_version 512445 (0.0010) [2023-12-26 19:05:34,080][105620] Updated weights for policy 1, policy_version 512455 (0.0010) [2023-12-26 19:05:34,445][105692] Updated weights for policy 0, policy_version 511944 (0.0011) [2023-12-26 19:05:34,504][105692] Updated weights for policy 0, policy_version 511954 (0.0010) [2023-12-26 19:05:34,565][105692] Updated weights for policy 0, policy_version 511964 (0.0008) [2023-12-26 19:05:34,755][105620] Updated weights for policy 1, policy_version 512465 (0.0006) [2023-12-26 19:05:34,821][105620] Updated weights for policy 1, policy_version 512475 (0.0007) [2023-12-26 19:05:34,853][105586] KL-divergence is very high: 113.2194 [2023-12-26 19:05:34,883][105620] Updated weights for policy 1, policy_version 512485 (0.0007) [2023-12-26 19:05:35,193][105692] Updated weights for policy 0, policy_version 511974 (0.0008) [2023-12-26 19:05:35,244][105692] Updated weights for policy 0, policy_version 511984 (0.0010) [2023-12-26 19:05:35,296][105692] Updated weights for policy 0, policy_version 511994 (0.0010) [2023-12-26 19:05:35,492][105620] Updated weights for policy 1, policy_version 512495 (0.0008) [2023-12-26 19:05:35,542][105586] KL-divergence is very high: 158.3447 [2023-12-26 19:05:35,547][105620] Updated weights for policy 1, policy_version 512505 (0.0008) [2023-12-26 19:05:35,548][105586] KL-divergence is very high: 146.5194 [2023-12-26 19:05:35,554][105586] KL-divergence is very high: 130.8925 [2023-12-26 19:05:35,559][105586] KL-divergence is very high: 105.6040 [2023-12-26 19:05:35,606][105620] Updated weights for policy 1, policy_version 512515 (0.0007) [2023-12-26 19:05:35,614][105586] KL-divergence is very high: 113.7884 [2023-12-26 19:05:35,946][105692] Updated weights for policy 0, policy_version 512004 (0.0008) [2023-12-26 19:05:36,000][105692] Updated weights for policy 0, policy_version 512014 (0.0005) [2023-12-26 19:05:36,055][105692] Updated weights for policy 0, policy_version 512024 (0.0009) [2023-12-26 19:05:36,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 262307840. Throughput: 0: 9777.4, 1: 9696.1. Samples: 262299116. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 19:05:36,063][104569] Avg episode reward: [(0, '6615.136'), (1, '3749.029')] [2023-12-26 19:05:36,284][105586] KL-divergence is very high: 178.9160 [2023-12-26 19:05:36,291][105586] KL-divergence is very high: 236.6923 [2023-12-26 19:05:36,313][105620] Updated weights for policy 1, policy_version 512525 (0.0007) [2023-12-26 19:05:36,340][105586] KL-divergence is very high: 105.0764 [2023-12-26 19:05:36,348][105586] KL-divergence is very high: 139.1425 [2023-12-26 19:05:36,379][105620] Updated weights for policy 1, policy_version 512535 (0.0008) [2023-12-26 19:05:36,392][105586] KL-divergence is very high: 103.2159 [2023-12-26 19:05:36,398][105586] KL-divergence is very high: 138.8127 [2023-12-26 19:05:36,441][105620] Updated weights for policy 1, policy_version 512545 (0.0008) [2023-12-26 19:05:36,442][105586] KL-divergence is very high: 101.0405 [2023-12-26 19:05:36,448][105586] KL-divergence is very high: 132.4113 [2023-12-26 19:05:36,730][105692] Updated weights for policy 0, policy_version 512034 (0.0008) [2023-12-26 19:05:36,800][105692] Updated weights for policy 0, policy_version 512044 (0.0011) [2023-12-26 19:05:36,866][105692] Updated weights for policy 0, policy_version 512054 (0.0011) [2023-12-26 19:05:36,928][105692] Updated weights for policy 0, policy_version 512064 (0.0011) [2023-12-26 19:05:37,120][105620] Updated weights for policy 1, policy_version 512555 (0.0009) [2023-12-26 19:05:37,173][105620] Updated weights for policy 1, policy_version 512565 (0.0009) [2023-12-26 19:05:37,230][105620] Updated weights for policy 1, policy_version 512575 (0.0009) [2023-12-26 19:05:37,537][105692] Updated weights for policy 0, policy_version 512074 (0.0005) [2023-12-26 19:05:37,599][105692] Updated weights for policy 0, policy_version 512084 (0.0007) [2023-12-26 19:05:37,662][105692] Updated weights for policy 0, policy_version 512094 (0.0007) [2023-12-26 19:05:38,134][105620] Updated weights for policy 1, policy_version 512585 (0.0009) [2023-12-26 19:05:38,195][105620] Updated weights for policy 1, policy_version 512595 (0.0009) [2023-12-26 19:05:38,252][105620] Updated weights for policy 1, policy_version 512605 (0.0008) [2023-12-26 19:05:38,266][105692] Updated weights for policy 0, policy_version 512104 (0.0008) [2023-12-26 19:05:38,315][105620] Updated weights for policy 1, policy_version 512615 (0.0006) [2023-12-26 19:05:38,322][105692] Updated weights for policy 0, policy_version 512114 (0.0008) [2023-12-26 19:05:38,387][105692] Updated weights for policy 0, policy_version 512124 (0.0009) [2023-12-26 19:05:39,071][105692] Updated weights for policy 0, policy_version 512134 (0.0009) [2023-12-26 19:05:39,126][105692] Updated weights for policy 0, policy_version 512144 (0.0007) [2023-12-26 19:05:39,127][105620] Updated weights for policy 1, policy_version 512625 (0.0008) [2023-12-26 19:05:39,153][105585] KL-divergence is very high: 192.6618 [2023-12-26 19:05:39,183][105692] Updated weights for policy 0, policy_version 512154 (0.0006) [2023-12-26 19:05:39,196][105620] Updated weights for policy 1, policy_version 512635 (0.0009) [2023-12-26 19:05:39,201][105585] KL-divergence is very high: 242.6767 [2023-12-26 19:05:39,260][105620] Updated weights for policy 1, policy_version 512645 (0.0008) [2023-12-26 19:05:39,931][105692] Updated weights for policy 0, policy_version 512164 (0.0010) [2023-12-26 19:05:39,994][105692] Updated weights for policy 0, policy_version 512174 (0.0010) [2023-12-26 19:05:40,049][105692] Updated weights for policy 0, policy_version 512184 (0.0009) [2023-12-26 19:05:40,064][105620] Updated weights for policy 1, policy_version 512655 (0.0008) [2023-12-26 19:05:40,067][105586] KL-divergence is very high: 100.7975 [2023-12-26 19:05:40,078][105586] KL-divergence is very high: 119.0723 [2023-12-26 19:05:40,083][105586] KL-divergence is very high: 115.9854 [2023-12-26 19:05:40,109][105586] KL-divergence is very high: 123.8268 [2023-12-26 19:05:40,117][105586] KL-divergence is very high: 109.7972 [2023-12-26 19:05:40,124][105620] Updated weights for policy 1, policy_version 512665 (0.0007) [2023-12-26 19:05:40,185][105620] Updated weights for policy 1, policy_version 512675 (0.0009) [2023-12-26 19:05:40,831][105692] Updated weights for policy 0, policy_version 512194 (0.0007) [2023-12-26 19:05:40,887][105692] Updated weights for policy 0, policy_version 512204 (0.0009) [2023-12-26 19:05:40,921][105620] Updated weights for policy 1, policy_version 512685 (0.0009) [2023-12-26 19:05:40,950][105692] Updated weights for policy 0, policy_version 512214 (0.0008) [2023-12-26 19:05:40,980][105620] Updated weights for policy 1, policy_version 512695 (0.0006) [2023-12-26 19:05:41,006][105692] Updated weights for policy 0, policy_version 512224 (0.0006) [2023-12-26 19:05:41,053][105620] Updated weights for policy 1, policy_version 512705 (0.0011) [2023-12-26 19:05:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 262406144. Throughput: 0: 9869.5, 1: 9674.2. Samples: 262416328. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 19:05:41,063][104569] Avg episode reward: [(0, '6537.738'), (1, '1249.718')] [2023-12-26 19:05:41,072][105586] KL-divergence is very high: 155.2682 [2023-12-26 19:05:41,078][105586] KL-divergence is very high: 154.7682 [2023-12-26 19:05:41,085][105586] KL-divergence is very high: 119.3070 [2023-12-26 19:05:41,091][105586] KL-divergence is very high: 180.3519 [2023-12-26 19:05:41,798][105692] Updated weights for policy 0, policy_version 512234 (0.0009) [2023-12-26 19:05:41,830][105586] KL-divergence is very high: 169.0950 [2023-12-26 19:05:41,836][105586] KL-divergence is very high: 101.1568 [2023-12-26 19:05:41,846][105620] Updated weights for policy 1, policy_version 512715 (0.0012) [2023-12-26 19:05:41,849][105692] Updated weights for policy 0, policy_version 512244 (0.0007) [2023-12-26 19:05:41,864][105586] KL-divergence is very high: 106.3803 [2023-12-26 19:05:41,869][105586] KL-divergence is very high: 177.6311 [2023-12-26 19:05:41,876][105586] KL-divergence is very high: 158.3177 [2023-12-26 19:05:41,883][105586] KL-divergence is very high: 105.4687 [2023-12-26 19:05:41,901][105692] Updated weights for policy 0, policy_version 512254 (0.0006) [2023-12-26 19:05:41,907][105620] Updated weights for policy 1, policy_version 512725 (0.0010) [2023-12-26 19:05:41,964][105586] KL-divergence is very high: 173.0238 [2023-12-26 19:05:41,970][105620] Updated weights for policy 1, policy_version 512735 (0.0011) [2023-12-26 19:05:41,971][105586] KL-divergence is very high: 122.5234 [2023-12-26 19:05:41,978][105586] KL-divergence is very high: 203.8759 [2023-12-26 19:05:41,984][105586] KL-divergence is very high: 107.3761 [2023-12-26 19:05:41,991][105586] KL-divergence is very high: 203.4944 [2023-12-26 19:05:41,997][105586] KL-divergence is very high: 177.3742 [2023-12-26 19:05:42,003][105586] KL-divergence is very high: 203.0281 [2023-12-26 19:05:42,009][105586] KL-divergence is very high: 135.7551 [2023-12-26 19:05:42,016][105586] KL-divergence is very high: 130.4685 [2023-12-26 19:05:42,687][105692] Updated weights for policy 0, policy_version 512264 (0.0007) [2023-12-26 19:05:42,733][105620] Updated weights for policy 1, policy_version 512745 (0.0011) [2023-12-26 19:05:42,747][105692] Updated weights for policy 0, policy_version 512274 (0.0007) [2023-12-26 19:05:42,748][105586] KL-divergence is very high: 192.0550 [2023-12-26 19:05:42,764][105586] KL-divergence is very high: 166.6766 [2023-12-26 19:05:42,782][105586] KL-divergence is very high: 210.8998 [2023-12-26 19:05:42,787][105620] Updated weights for policy 1, policy_version 512755 (0.0011) [2023-12-26 19:05:42,788][105586] KL-divergence is very high: 191.4113 [2023-12-26 19:05:42,795][105586] KL-divergence is very high: 416.1458 [2023-12-26 19:05:42,809][105692] Updated weights for policy 0, policy_version 512284 (0.0006) [2023-12-26 19:05:42,813][105586] KL-divergence is very high: 201.8321 [2023-12-26 19:05:42,832][105586] KL-divergence is very high: 142.7157 [2023-12-26 19:05:42,838][105586] KL-divergence is very high: 116.5326 [2023-12-26 19:05:42,845][105586] KL-divergence is very high: 208.5187 [2023-12-26 19:05:42,850][105620] Updated weights for policy 1, policy_version 512765 (0.0011) [2023-12-26 19:05:42,863][105586] KL-divergence is very high: 112.0323 [2023-12-26 19:05:42,902][105620] Updated weights for policy 1, policy_version 512775 (0.0010) [2023-12-26 19:05:43,558][105692] Updated weights for policy 0, policy_version 512294 (0.0007) [2023-12-26 19:05:43,606][105692] Updated weights for policy 0, policy_version 512304 (0.0008) [2023-12-26 19:05:43,657][105620] Updated weights for policy 1, policy_version 512785 (0.0011) [2023-12-26 19:05:43,660][105692] Updated weights for policy 0, policy_version 512314 (0.0006) [2023-12-26 19:05:43,705][105620] Updated weights for policy 1, policy_version 512795 (0.0010) [2023-12-26 19:05:43,753][105620] Updated weights for policy 1, policy_version 512805 (0.0010) [2023-12-26 19:05:44,432][105692] Updated weights for policy 0, policy_version 512324 (0.0007) [2023-12-26 19:05:44,480][105692] Updated weights for policy 0, policy_version 512334 (0.0008) [2023-12-26 19:05:44,513][105620] Updated weights for policy 1, policy_version 512815 (0.0010) [2023-12-26 19:05:44,528][105692] Updated weights for policy 0, policy_version 512344 (0.0008) [2023-12-26 19:05:44,558][105620] Updated weights for policy 1, policy_version 512825 (0.0010) [2023-12-26 19:05:44,610][105620] Updated weights for policy 1, policy_version 512835 (0.0010) [2023-12-26 19:05:45,322][105692] Updated weights for policy 0, policy_version 512354 (0.0006) [2023-12-26 19:05:45,381][105692] Updated weights for policy 0, policy_version 512364 (0.0008) [2023-12-26 19:05:45,381][105620] Updated weights for policy 1, policy_version 512845 (0.0011) [2023-12-26 19:05:45,435][105692] Updated weights for policy 0, policy_version 512374 (0.0006) [2023-12-26 19:05:45,436][105620] Updated weights for policy 1, policy_version 512855 (0.0010) [2023-12-26 19:05:45,485][105620] Updated weights for policy 1, policy_version 512865 (0.0010) [2023-12-26 19:05:45,487][105692] Updated weights for policy 0, policy_version 512384 (0.0005) [2023-12-26 19:05:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 262496256. Throughput: 0: 9904.9, 1: 9542.4. Samples: 262470468. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 19:05:46,062][104569] Avg episode reward: [(0, '1655.854'), (1, '1568.950')] [2023-12-26 19:05:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000512384_131186688.pth... [2023-12-26 19:05:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000512872_131309568.pth... [2023-12-26 19:05:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000511752_131022848.pth [2023-12-26 19:05:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000511232_130891776.pth [2023-12-26 19:05:46,241][105620] Updated weights for policy 1, policy_version 512875 (0.0010) [2023-12-26 19:05:46,255][105692] Updated weights for policy 0, policy_version 512394 (0.0006) [2023-12-26 19:05:46,289][105620] Updated weights for policy 1, policy_version 512885 (0.0010) [2023-12-26 19:05:46,303][105692] Updated weights for policy 0, policy_version 512404 (0.0005) [2023-12-26 19:05:46,337][105620] Updated weights for policy 1, policy_version 512895 (0.0010) [2023-12-26 19:05:46,352][105692] Updated weights for policy 0, policy_version 512414 (0.0005) [2023-12-26 19:05:47,103][105620] Updated weights for policy 1, policy_version 512905 (0.0010) [2023-12-26 19:05:47,116][105692] Updated weights for policy 0, policy_version 512424 (0.0008) [2023-12-26 19:05:47,162][105620] Updated weights for policy 1, policy_version 512915 (0.0009) [2023-12-26 19:05:47,172][105692] Updated weights for policy 0, policy_version 512434 (0.0006) [2023-12-26 19:05:47,218][105620] Updated weights for policy 1, policy_version 512925 (0.0008) [2023-12-26 19:05:47,224][105692] Updated weights for policy 0, policy_version 512444 (0.0006) [2023-12-26 19:05:47,272][105620] Updated weights for policy 1, policy_version 512935 (0.0008) [2023-12-26 19:05:47,980][105692] Updated weights for policy 0, policy_version 512454 (0.0007) [2023-12-26 19:05:48,017][105620] Updated weights for policy 1, policy_version 512945 (0.0006) [2023-12-26 19:05:48,040][105692] Updated weights for policy 0, policy_version 512464 (0.0009) [2023-12-26 19:05:48,068][105620] Updated weights for policy 1, policy_version 512955 (0.0005) [2023-12-26 19:05:48,094][105692] Updated weights for policy 0, policy_version 512474 (0.0008) [2023-12-26 19:05:48,112][105620] Updated weights for policy 1, policy_version 512965 (0.0005) [2023-12-26 19:05:48,771][105620] Updated weights for policy 1, policy_version 512975 (0.0008) [2023-12-26 19:05:48,833][105620] Updated weights for policy 1, policy_version 512985 (0.0009) [2023-12-26 19:05:48,888][105620] Updated weights for policy 1, policy_version 512995 (0.0008) [2023-12-26 19:05:48,889][105692] Updated weights for policy 0, policy_version 512484 (0.0008) [2023-12-26 19:05:48,955][105692] Updated weights for policy 0, policy_version 512494 (0.0006) [2023-12-26 19:05:49,020][105692] Updated weights for policy 0, policy_version 512504 (0.0006) [2023-12-26 19:05:49,668][105620] Updated weights for policy 1, policy_version 513005 (0.0007) [2023-12-26 19:05:49,732][105620] Updated weights for policy 1, policy_version 513015 (0.0009) [2023-12-26 19:05:49,743][105692] Updated weights for policy 0, policy_version 512514 (0.0007) [2023-12-26 19:05:49,789][105620] Updated weights for policy 1, policy_version 513025 (0.0008) [2023-12-26 19:05:49,798][105692] Updated weights for policy 0, policy_version 512524 (0.0006) [2023-12-26 19:05:49,862][105692] Updated weights for policy 0, policy_version 512534 (0.0010) [2023-12-26 19:05:49,928][105692] Updated weights for policy 0, policy_version 512544 (0.0010) [2023-12-26 19:05:50,436][105620] Updated weights for policy 1, policy_version 513035 (0.0007) [2023-12-26 19:05:50,488][105620] Updated weights for policy 1, policy_version 513045 (0.0007) [2023-12-26 19:05:50,548][105620] Updated weights for policy 1, policy_version 513055 (0.0007) [2023-12-26 19:05:50,755][105692] Updated weights for policy 0, policy_version 512554 (0.0010) [2023-12-26 19:05:50,810][105692] Updated weights for policy 0, policy_version 512564 (0.0010) [2023-12-26 19:05:50,856][105692] Updated weights for policy 0, policy_version 512574 (0.0009) [2023-12-26 19:05:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.1, 300 sec: 19521.9). Total num frames: 262594560. Throughput: 0: 9827.7, 1: 9534.5. Samples: 262583388. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 19:05:51,063][104569] Avg episode reward: [(0, '1806.588'), (1, '6677.143')] [2023-12-26 19:05:51,196][105620] Updated weights for policy 1, policy_version 513065 (0.0008) [2023-12-26 19:05:51,242][105620] Updated weights for policy 1, policy_version 513075 (0.0008) [2023-12-26 19:05:51,293][105620] Updated weights for policy 1, policy_version 513085 (0.0008) [2023-12-26 19:05:51,347][105620] Updated weights for policy 1, policy_version 513095 (0.0008) [2023-12-26 19:05:51,707][105692] Updated weights for policy 0, policy_version 512584 (0.0010) [2023-12-26 19:05:51,773][105692] Updated weights for policy 0, policy_version 512594 (0.0008) [2023-12-26 19:05:51,837][105692] Updated weights for policy 0, policy_version 512604 (0.0005) [2023-12-26 19:05:52,124][105620] Updated weights for policy 1, policy_version 513105 (0.0009) [2023-12-26 19:05:52,185][105620] Updated weights for policy 1, policy_version 513115 (0.0009) [2023-12-26 19:05:52,240][105620] Updated weights for policy 1, policy_version 513125 (0.0009) [2023-12-26 19:05:52,436][105692] Updated weights for policy 0, policy_version 512614 (0.0006) [2023-12-26 19:05:52,498][105692] Updated weights for policy 0, policy_version 512624 (0.0005) [2023-12-26 19:05:52,551][105692] Updated weights for policy 0, policy_version 512634 (0.0005) [2023-12-26 19:05:52,984][105620] Updated weights for policy 1, policy_version 513135 (0.0008) [2023-12-26 19:05:53,039][105620] Updated weights for policy 1, policy_version 513145 (0.0008) [2023-12-26 19:05:53,086][105620] Updated weights for policy 1, policy_version 513155 (0.0009) [2023-12-26 19:05:53,162][105692] Updated weights for policy 0, policy_version 512644 (0.0007) [2023-12-26 19:05:53,225][105692] Updated weights for policy 0, policy_version 512654 (0.0009) [2023-12-26 19:05:53,281][105692] Updated weights for policy 0, policy_version 512664 (0.0009) [2023-12-26 19:05:53,848][105620] Updated weights for policy 1, policy_version 513165 (0.0007) [2023-12-26 19:05:53,905][105620] Updated weights for policy 1, policy_version 513175 (0.0007) [2023-12-26 19:05:53,952][105620] Updated weights for policy 1, policy_version 513185 (0.0009) [2023-12-26 19:05:54,044][105692] Updated weights for policy 0, policy_version 512674 (0.0009) [2023-12-26 19:05:54,106][105692] Updated weights for policy 0, policy_version 512684 (0.0009) [2023-12-26 19:05:54,165][105692] Updated weights for policy 0, policy_version 512694 (0.0010) [2023-12-26 19:05:54,229][105692] Updated weights for policy 0, policy_version 512704 (0.0010) [2023-12-26 19:05:54,754][105620] Updated weights for policy 1, policy_version 513195 (0.0009) [2023-12-26 19:05:54,809][105620] Updated weights for policy 1, policy_version 513205 (0.0009) [2023-12-26 19:05:54,841][105692] Updated weights for policy 0, policy_version 512714 (0.0005) [2023-12-26 19:05:54,863][105620] Updated weights for policy 1, policy_version 513215 (0.0008) [2023-12-26 19:05:54,893][105692] Updated weights for policy 0, policy_version 512724 (0.0010) [2023-12-26 19:05:54,946][105692] Updated weights for policy 0, policy_version 512734 (0.0008) [2023-12-26 19:05:55,644][105620] Updated weights for policy 1, policy_version 513225 (0.0005) [2023-12-26 19:05:55,667][105692] Updated weights for policy 0, policy_version 512744 (0.0008) [2023-12-26 19:05:55,704][105620] Updated weights for policy 1, policy_version 513235 (0.0008) [2023-12-26 19:05:55,735][105692] Updated weights for policy 0, policy_version 512754 (0.0006) [2023-12-26 19:05:55,757][105620] Updated weights for policy 1, policy_version 513245 (0.0008) [2023-12-26 19:05:55,784][105692] Updated weights for policy 0, policy_version 512764 (0.0009) [2023-12-26 19:05:55,807][105620] Updated weights for policy 1, policy_version 513255 (0.0006) [2023-12-26 19:05:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 262692864. Throughput: 0: 9798.8, 1: 9542.1. Samples: 262698760. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 19:05:56,062][104569] Avg episode reward: [(0, '6678.057'), (1, '9356.828')] [2023-12-26 19:05:56,414][105692] Updated weights for policy 0, policy_version 512774 (0.0010) [2023-12-26 19:05:56,479][105692] Updated weights for policy 0, policy_version 512784 (0.0010) [2023-12-26 19:05:56,537][105692] Updated weights for policy 0, policy_version 512794 (0.0010) [2023-12-26 19:05:56,607][105620] Updated weights for policy 1, policy_version 513265 (0.0009) [2023-12-26 19:05:56,658][105620] Updated weights for policy 1, policy_version 513275 (0.0008) [2023-12-26 19:05:56,705][105620] Updated weights for policy 1, policy_version 513285 (0.0007) [2023-12-26 19:05:57,199][105692] Updated weights for policy 0, policy_version 512804 (0.0008) [2023-12-26 19:05:57,245][105692] Updated weights for policy 0, policy_version 512814 (0.0005) [2023-12-26 19:05:57,291][105692] Updated weights for policy 0, policy_version 512824 (0.0005) [2023-12-26 19:05:57,523][105620] Updated weights for policy 1, policy_version 513295 (0.0009) [2023-12-26 19:05:57,576][105620] Updated weights for policy 1, policy_version 513305 (0.0009) [2023-12-26 19:05:57,633][105620] Updated weights for policy 1, policy_version 513315 (0.0009) [2023-12-26 19:05:58,040][105692] Updated weights for policy 0, policy_version 512834 (0.0008) [2023-12-26 19:05:58,093][105692] Updated weights for policy 0, policy_version 512846 (0.0010) [2023-12-26 19:05:58,152][105692] Updated weights for policy 0, policy_version 512856 (0.0009) [2023-12-26 19:05:58,257][105620] Updated weights for policy 1, policy_version 513325 (0.0008) [2023-12-26 19:05:58,324][105620] Updated weights for policy 1, policy_version 513335 (0.0008) [2023-12-26 19:05:58,385][105620] Updated weights for policy 1, policy_version 513345 (0.0008) [2023-12-26 19:05:59,108][105692] Updated weights for policy 0, policy_version 512866 (0.0009) [2023-12-26 19:05:59,168][105692] Updated weights for policy 0, policy_version 512876 (0.0009) [2023-12-26 19:05:59,213][105620] Updated weights for policy 1, policy_version 513355 (0.0009) [2023-12-26 19:05:59,223][105692] Updated weights for policy 0, policy_version 512886 (0.0008) [2023-12-26 19:05:59,277][105620] Updated weights for policy 1, policy_version 513365 (0.0010) [2023-12-26 19:05:59,283][105692] Updated weights for policy 0, policy_version 512896 (0.0007) [2023-12-26 19:05:59,345][105620] Updated weights for policy 1, policy_version 513375 (0.0009) [2023-12-26 19:06:00,051][105692] Updated weights for policy 0, policy_version 512906 (0.0008) [2023-12-26 19:06:00,088][105620] Updated weights for policy 1, policy_version 513385 (0.0009) [2023-12-26 19:06:00,109][105692] Updated weights for policy 0, policy_version 512916 (0.0009) [2023-12-26 19:06:00,139][105620] Updated weights for policy 1, policy_version 513395 (0.0005) [2023-12-26 19:06:00,157][105692] Updated weights for policy 0, policy_version 512926 (0.0008) [2023-12-26 19:06:00,187][105620] Updated weights for policy 1, policy_version 513405 (0.0008) [2023-12-26 19:06:00,232][105620] Updated weights for policy 1, policy_version 513415 (0.0010) [2023-12-26 19:06:00,901][105620] Updated weights for policy 1, policy_version 513425 (0.0010) [2023-12-26 19:06:00,934][105692] Updated weights for policy 0, policy_version 512936 (0.0011) [2023-12-26 19:06:00,953][105620] Updated weights for policy 1, policy_version 513435 (0.0010) [2023-12-26 19:06:00,983][105692] Updated weights for policy 0, policy_version 512946 (0.0010) [2023-12-26 19:06:01,003][105620] Updated weights for policy 1, policy_version 513445 (0.0010) [2023-12-26 19:06:01,032][105692] Updated weights for policy 0, policy_version 512956 (0.0010) [2023-12-26 19:06:01,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 262791168. Throughput: 0: 9789.8, 1: 9566.5. Samples: 262755820. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 19:06:01,062][104569] Avg episode reward: [(0, '9177.702'), (1, '9265.412')] [2023-12-26 19:06:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000513448_131457024.pth... [2023-12-26 19:06:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000512960_131334144.pth... [2023-12-26 19:06:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000511840_131047424.pth [2023-12-26 19:06:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000512328_131170304.pth [2023-12-26 19:06:01,747][105620] Updated weights for policy 1, policy_version 513455 (0.0010) [2023-12-26 19:06:01,767][105692] Updated weights for policy 0, policy_version 512966 (0.0009) [2023-12-26 19:06:01,806][105620] Updated weights for policy 1, policy_version 513465 (0.0005) [2023-12-26 19:06:01,820][105692] Updated weights for policy 0, policy_version 512976 (0.0009) [2023-12-26 19:06:01,863][105620] Updated weights for policy 1, policy_version 513475 (0.0005) [2023-12-26 19:06:01,872][105692] Updated weights for policy 0, policy_version 512986 (0.0009) [2023-12-26 19:06:02,448][105620] Updated weights for policy 1, policy_version 513485 (0.0005) [2023-12-26 19:06:02,508][105620] Updated weights for policy 1, policy_version 513495 (0.0005) [2023-12-26 19:06:02,575][105620] Updated weights for policy 1, policy_version 513505 (0.0006) [2023-12-26 19:06:02,751][105692] Updated weights for policy 0, policy_version 512996 (0.0010) [2023-12-26 19:06:02,808][105692] Updated weights for policy 0, policy_version 513006 (0.0011) [2023-12-26 19:06:02,857][105692] Updated weights for policy 0, policy_version 513016 (0.0010) [2023-12-26 19:06:03,178][105620] Updated weights for policy 1, policy_version 513515 (0.0005) [2023-12-26 19:06:03,242][105620] Updated weights for policy 1, policy_version 513525 (0.0005) [2023-12-26 19:06:03,295][105620] Updated weights for policy 1, policy_version 513535 (0.0005) [2023-12-26 19:06:03,596][105692] Updated weights for policy 0, policy_version 513026 (0.0010) [2023-12-26 19:06:03,654][105692] Updated weights for policy 0, policy_version 513036 (0.0010) [2023-12-26 19:06:03,716][105692] Updated weights for policy 0, policy_version 513046 (0.0010) [2023-12-26 19:06:03,763][105692] Updated weights for policy 0, policy_version 513056 (0.0010) [2023-12-26 19:06:03,793][105620] Updated weights for policy 1, policy_version 513545 (0.0006) [2023-12-26 19:06:03,856][105620] Updated weights for policy 1, policy_version 513555 (0.0011) [2023-12-26 19:06:03,920][105620] Updated weights for policy 1, policy_version 513565 (0.0011) [2023-12-26 19:06:03,983][105620] Updated weights for policy 1, policy_version 513575 (0.0009) [2023-12-26 19:06:04,535][105692] Updated weights for policy 0, policy_version 513066 (0.0011) [2023-12-26 19:06:04,591][105692] Updated weights for policy 0, policy_version 513076 (0.0010) [2023-12-26 19:06:04,653][105692] Updated weights for policy 0, policy_version 513086 (0.0011) [2023-12-26 19:06:04,682][105620] Updated weights for policy 1, policy_version 513585 (0.0011) [2023-12-26 19:06:04,730][105620] Updated weights for policy 1, policy_version 513595 (0.0010) [2023-12-26 19:06:04,782][105620] Updated weights for policy 1, policy_version 513605 (0.0011) [2023-12-26 19:06:05,392][105692] Updated weights for policy 0, policy_version 513096 (0.0009) [2023-12-26 19:06:05,447][105692] Updated weights for policy 0, policy_version 513106 (0.0010) [2023-12-26 19:06:05,513][105692] Updated weights for policy 0, policy_version 513116 (0.0011) [2023-12-26 19:06:05,556][105620] Updated weights for policy 1, policy_version 513615 (0.0011) [2023-12-26 19:06:05,614][105620] Updated weights for policy 1, policy_version 513625 (0.0007) [2023-12-26 19:06:05,677][105620] Updated weights for policy 1, policy_version 513635 (0.0006) [2023-12-26 19:06:06,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19251.1, 300 sec: 19521.9). Total num frames: 262881280. Throughput: 0: 9659.1, 1: 9678.1. Samples: 262872092. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 19:06:06,063][104569] Avg episode reward: [(0, '9260.677'), (1, '9175.331')] [2023-12-26 19:06:06,256][105692] Updated weights for policy 0, policy_version 513126 (0.0011) [2023-12-26 19:06:06,322][105692] Updated weights for policy 0, policy_version 513136 (0.0011) [2023-12-26 19:06:06,372][105620] Updated weights for policy 1, policy_version 513645 (0.0009) [2023-12-26 19:06:06,389][105692] Updated weights for policy 0, policy_version 513146 (0.0010) [2023-12-26 19:06:06,431][105620] Updated weights for policy 1, policy_version 513655 (0.0008) [2023-12-26 19:06:06,491][105620] Updated weights for policy 1, policy_version 513665 (0.0008) [2023-12-26 19:06:07,120][105692] Updated weights for policy 0, policy_version 513156 (0.0010) [2023-12-26 19:06:07,179][105692] Updated weights for policy 0, policy_version 513166 (0.0010) [2023-12-26 19:06:07,200][105620] Updated weights for policy 1, policy_version 513675 (0.0008) [2023-12-26 19:06:07,244][105692] Updated weights for policy 0, policy_version 513176 (0.0011) [2023-12-26 19:06:07,255][105620] Updated weights for policy 1, policy_version 513685 (0.0008) [2023-12-26 19:06:07,312][105620] Updated weights for policy 1, policy_version 513695 (0.0007) [2023-12-26 19:06:07,949][105620] Updated weights for policy 1, policy_version 513705 (0.0005) [2023-12-26 19:06:07,987][105692] Updated weights for policy 0, policy_version 513186 (0.0010) [2023-12-26 19:06:08,014][105620] Updated weights for policy 1, policy_version 513715 (0.0009) [2023-12-26 19:06:08,038][105692] Updated weights for policy 0, policy_version 513196 (0.0009) [2023-12-26 19:06:08,076][105620] Updated weights for policy 1, policy_version 513725 (0.0008) [2023-12-26 19:06:08,089][105692] Updated weights for policy 0, policy_version 513206 (0.0010) [2023-12-26 19:06:08,139][105620] Updated weights for policy 1, policy_version 513735 (0.0007) [2023-12-26 19:06:08,141][105692] Updated weights for policy 0, policy_version 513216 (0.0010) [2023-12-26 19:06:08,840][105620] Updated weights for policy 1, policy_version 513745 (0.0010) [2023-12-26 19:06:08,878][105692] Updated weights for policy 0, policy_version 513226 (0.0006) [2023-12-26 19:06:08,899][105620] Updated weights for policy 1, policy_version 513755 (0.0009) [2023-12-26 19:06:08,930][105692] Updated weights for policy 0, policy_version 513236 (0.0008) [2023-12-26 19:06:08,952][105620] Updated weights for policy 1, policy_version 513765 (0.0007) [2023-12-26 19:06:08,990][105692] Updated weights for policy 0, policy_version 513246 (0.0007) [2023-12-26 19:06:09,657][105692] Updated weights for policy 0, policy_version 513256 (0.0009) [2023-12-26 19:06:09,713][105692] Updated weights for policy 0, policy_version 513266 (0.0009) [2023-12-26 19:06:09,773][105692] Updated weights for policy 0, policy_version 513276 (0.0009) [2023-12-26 19:06:09,776][105620] Updated weights for policy 1, policy_version 513775 (0.0008) [2023-12-26 19:06:09,839][105620] Updated weights for policy 1, policy_version 513785 (0.0008) [2023-12-26 19:06:09,904][105620] Updated weights for policy 1, policy_version 513795 (0.0008) [2023-12-26 19:06:10,504][105692] Updated weights for policy 0, policy_version 513286 (0.0009) [2023-12-26 19:06:10,563][105692] Updated weights for policy 0, policy_version 513297 (0.0010) [2023-12-26 19:06:10,616][105692] Updated weights for policy 0, policy_version 513307 (0.0006) [2023-12-26 19:06:10,620][105620] Updated weights for policy 1, policy_version 513805 (0.0007) [2023-12-26 19:06:10,675][105620] Updated weights for policy 1, policy_version 513815 (0.0008) [2023-12-26 19:06:10,736][105620] Updated weights for policy 1, policy_version 513825 (0.0005) [2023-12-26 19:06:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 262979584. Throughput: 0: 9594.1, 1: 9718.7. Samples: 262987668. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 19:06:11,062][104569] Avg episode reward: [(0, '9260.758'), (1, '9265.965')] [2023-12-26 19:06:11,454][105692] Updated weights for policy 0, policy_version 513317 (0.0007) [2023-12-26 19:06:11,480][105620] Updated weights for policy 1, policy_version 513835 (0.0006) [2023-12-26 19:06:11,517][105692] Updated weights for policy 0, policy_version 513327 (0.0006) [2023-12-26 19:06:11,537][105620] Updated weights for policy 1, policy_version 513845 (0.0008) [2023-12-26 19:06:11,578][105692] Updated weights for policy 0, policy_version 513337 (0.0007) [2023-12-26 19:06:11,593][105620] Updated weights for policy 1, policy_version 513855 (0.0008) [2023-12-26 19:06:12,337][105620] Updated weights for policy 1, policy_version 513865 (0.0009) [2023-12-26 19:06:12,375][105692] Updated weights for policy 0, policy_version 513347 (0.0009) [2023-12-26 19:06:12,406][105620] Updated weights for policy 1, policy_version 513875 (0.0008) [2023-12-26 19:06:12,438][105692] Updated weights for policy 0, policy_version 513357 (0.0007) [2023-12-26 19:06:12,474][105620] Updated weights for policy 1, policy_version 513885 (0.0008) [2023-12-26 19:06:12,498][105692] Updated weights for policy 0, policy_version 513367 (0.0008) [2023-12-26 19:06:12,538][105620] Updated weights for policy 1, policy_version 513895 (0.0007) [2023-12-26 19:06:13,258][105692] Updated weights for policy 0, policy_version 513377 (0.0008) [2023-12-26 19:06:13,290][105620] Updated weights for policy 1, policy_version 513905 (0.0010) [2023-12-26 19:06:13,321][105692] Updated weights for policy 0, policy_version 513387 (0.0006) [2023-12-26 19:06:13,352][105620] Updated weights for policy 1, policy_version 513915 (0.0010) [2023-12-26 19:06:13,370][105692] Updated weights for policy 0, policy_version 513397 (0.0006) [2023-12-26 19:06:13,414][105620] Updated weights for policy 1, policy_version 513925 (0.0010) [2023-12-26 19:06:13,420][105692] Updated weights for policy 0, policy_version 513407 (0.0008) [2023-12-26 19:06:14,143][105620] Updated weights for policy 1, policy_version 513935 (0.0010) [2023-12-26 19:06:14,199][105692] Updated weights for policy 0, policy_version 513417 (0.0008) [2023-12-26 19:06:14,208][105620] Updated weights for policy 1, policy_version 513945 (0.0010) [2023-12-26 19:06:14,251][105692] Updated weights for policy 0, policy_version 513427 (0.0006) [2023-12-26 19:06:14,267][105620] Updated weights for policy 1, policy_version 513955 (0.0008) [2023-12-26 19:06:14,305][105692] Updated weights for policy 0, policy_version 513437 (0.0009) [2023-12-26 19:06:14,889][105620] Updated weights for policy 1, policy_version 513965 (0.0007) [2023-12-26 19:06:14,946][105620] Updated weights for policy 1, policy_version 513975 (0.0007) [2023-12-26 19:06:15,006][105620] Updated weights for policy 1, policy_version 513985 (0.0009) [2023-12-26 19:06:15,025][105692] Updated weights for policy 0, policy_version 513447 (0.0007) [2023-12-26 19:06:15,081][105692] Updated weights for policy 0, policy_version 513457 (0.0008) [2023-12-26 19:06:15,138][105692] Updated weights for policy 0, policy_version 513467 (0.0010) [2023-12-26 19:06:15,722][105620] Updated weights for policy 1, policy_version 513995 (0.0007) [2023-12-26 19:06:15,799][105620] Updated weights for policy 1, policy_version 514005 (0.0005) [2023-12-26 19:06:15,834][105692] Updated weights for policy 0, policy_version 513477 (0.0007) [2023-12-26 19:06:15,855][105620] Updated weights for policy 1, policy_version 514015 (0.0009) [2023-12-26 19:06:15,889][105692] Updated weights for policy 0, policy_version 513487 (0.0005) [2023-12-26 19:06:15,943][105692] Updated weights for policy 0, policy_version 513497 (0.0008) [2023-12-26 19:06:16,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 263077888. Throughput: 0: 9511.7, 1: 9664.5. Samples: 263041864. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 19:06:16,063][104569] Avg episode reward: [(0, '9265.101'), (1, '9175.131')] [2023-12-26 19:06:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000513504_131473408.pth... [2023-12-26 19:06:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000514024_131604480.pth... [2023-12-26 19:06:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000512872_131309568.pth [2023-12-26 19:06:16,094][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000512384_131186688.pth [2023-12-26 19:06:16,536][105620] Updated weights for policy 1, policy_version 514025 (0.0008) [2023-12-26 19:06:16,582][105620] Updated weights for policy 1, policy_version 514035 (0.0009) [2023-12-26 19:06:16,632][105620] Updated weights for policy 1, policy_version 514045 (0.0009) [2023-12-26 19:06:16,664][105692] Updated weights for policy 0, policy_version 513507 (0.0008) [2023-12-26 19:06:16,692][105620] Updated weights for policy 1, policy_version 514055 (0.0009) [2023-12-26 19:06:16,722][105692] Updated weights for policy 0, policy_version 513517 (0.0009) [2023-12-26 19:06:16,777][105692] Updated weights for policy 0, policy_version 513527 (0.0009) [2023-12-26 19:06:17,320][105620] Updated weights for policy 1, policy_version 514065 (0.0010) [2023-12-26 19:06:17,368][105620] Updated weights for policy 1, policy_version 514075 (0.0010) [2023-12-26 19:06:17,420][105620] Updated weights for policy 1, policy_version 514085 (0.0011) [2023-12-26 19:06:17,616][105692] Updated weights for policy 0, policy_version 513537 (0.0010) [2023-12-26 19:06:17,681][105692] Updated weights for policy 0, policy_version 513547 (0.0009) [2023-12-26 19:06:17,734][105692] Updated weights for policy 0, policy_version 513557 (0.0009) [2023-12-26 19:06:17,791][105692] Updated weights for policy 0, policy_version 513568 (0.0011) [2023-12-26 19:06:18,096][105620] Updated weights for policy 1, policy_version 514095 (0.0011) [2023-12-26 19:06:18,148][105620] Updated weights for policy 1, policy_version 514105 (0.0010) [2023-12-26 19:06:18,203][105620] Updated weights for policy 1, policy_version 514115 (0.0010) [2023-12-26 19:06:18,635][105692] Updated weights for policy 0, policy_version 513578 (0.0011) [2023-12-26 19:06:18,696][105692] Updated weights for policy 0, policy_version 513588 (0.0010) [2023-12-26 19:06:18,759][105692] Updated weights for policy 0, policy_version 513598 (0.0011) [2023-12-26 19:06:18,860][105620] Updated weights for policy 1, policy_version 514125 (0.0008) [2023-12-26 19:06:18,929][105620] Updated weights for policy 1, policy_version 514135 (0.0006) [2023-12-26 19:06:18,989][105620] Updated weights for policy 1, policy_version 514145 (0.0008) [2023-12-26 19:06:19,496][105692] Updated weights for policy 0, policy_version 513608 (0.0011) [2023-12-26 19:06:19,560][105692] Updated weights for policy 0, policy_version 513618 (0.0011) [2023-12-26 19:06:19,625][105692] Updated weights for policy 0, policy_version 513628 (0.0011) [2023-12-26 19:06:19,669][105620] Updated weights for policy 1, policy_version 514155 (0.0008) [2023-12-26 19:06:19,731][105620] Updated weights for policy 1, policy_version 514165 (0.0008) [2023-12-26 19:06:19,789][105620] Updated weights for policy 1, policy_version 514175 (0.0008) [2023-12-26 19:06:20,454][105692] Updated weights for policy 0, policy_version 513638 (0.0010) [2023-12-26 19:06:20,518][105692] Updated weights for policy 0, policy_version 513648 (0.0011) [2023-12-26 19:06:20,556][105620] Updated weights for policy 1, policy_version 514185 (0.0009) [2023-12-26 19:06:20,581][105692] Updated weights for policy 0, policy_version 513658 (0.0011) [2023-12-26 19:06:20,621][105620] Updated weights for policy 1, policy_version 514195 (0.0008) [2023-12-26 19:06:20,682][105620] Updated weights for policy 1, policy_version 514205 (0.0007) [2023-12-26 19:06:20,756][105620] Updated weights for policy 1, policy_version 514215 (0.0007) [2023-12-26 19:06:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 263168000. Throughput: 0: 9457.0, 1: 9649.1. Samples: 263158888. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 19:06:21,062][104569] Avg episode reward: [(0, '9086.810'), (1, '9116.574')] [2023-12-26 19:06:21,319][105692] Updated weights for policy 0, policy_version 513668 (0.0007) [2023-12-26 19:06:21,390][105692] Updated weights for policy 0, policy_version 513678 (0.0014) [2023-12-26 19:06:21,426][105620] Updated weights for policy 1, policy_version 514225 (0.0009) [2023-12-26 19:06:21,456][105692] Updated weights for policy 0, policy_version 513688 (0.0006) [2023-12-26 19:06:21,478][105620] Updated weights for policy 1, policy_version 514235 (0.0009) [2023-12-26 19:06:21,536][105620] Updated weights for policy 1, policy_version 514245 (0.0009) [2023-12-26 19:06:22,132][105692] Updated weights for policy 0, policy_version 513698 (0.0006) [2023-12-26 19:06:22,188][105692] Updated weights for policy 0, policy_version 513708 (0.0009) [2023-12-26 19:06:22,237][105692] Updated weights for policy 0, policy_version 513718 (0.0009) [2023-12-26 19:06:22,297][105692] Updated weights for policy 0, policy_version 513728 (0.0008) [2023-12-26 19:06:22,318][105620] Updated weights for policy 1, policy_version 514255 (0.0009) [2023-12-26 19:06:22,382][105620] Updated weights for policy 1, policy_version 514265 (0.0008) [2023-12-26 19:06:22,441][105620] Updated weights for policy 1, policy_version 514275 (0.0009) [2023-12-26 19:06:23,093][105692] Updated weights for policy 0, policy_version 513738 (0.0005) [2023-12-26 19:06:23,148][105692] Updated weights for policy 0, policy_version 513748 (0.0005) [2023-12-26 19:06:23,206][105692] Updated weights for policy 0, policy_version 513758 (0.0005) [2023-12-26 19:06:23,246][105620] Updated weights for policy 1, policy_version 514285 (0.0007) [2023-12-26 19:06:23,320][105620] Updated weights for policy 1, policy_version 514295 (0.0006) [2023-12-26 19:06:23,381][105620] Updated weights for policy 1, policy_version 514305 (0.0005) [2023-12-26 19:06:23,768][105692] Updated weights for policy 0, policy_version 513768 (0.0008) [2023-12-26 19:06:23,812][105692] Updated weights for policy 0, policy_version 513778 (0.0005) [2023-12-26 19:06:23,874][105692] Updated weights for policy 0, policy_version 513788 (0.0005) [2023-12-26 19:06:24,092][105620] Updated weights for policy 1, policy_version 514315 (0.0007) [2023-12-26 19:06:24,149][105620] Updated weights for policy 1, policy_version 514325 (0.0010) [2023-12-26 19:06:24,209][105620] Updated weights for policy 1, policy_version 514336 (0.0010) [2023-12-26 19:06:24,434][105692] Updated weights for policy 0, policy_version 513798 (0.0005) [2023-12-26 19:06:24,500][105692] Updated weights for policy 0, policy_version 513808 (0.0005) [2023-12-26 19:06:24,568][105692] Updated weights for policy 0, policy_version 513818 (0.0005) [2023-12-26 19:06:24,896][105620] Updated weights for policy 1, policy_version 514346 (0.0008) [2023-12-26 19:06:24,951][105620] Updated weights for policy 1, policy_version 514356 (0.0008) [2023-12-26 19:06:25,008][105620] Updated weights for policy 1, policy_version 514366 (0.0009) [2023-12-26 19:06:25,061][105620] Updated weights for policy 1, policy_version 514376 (0.0010) [2023-12-26 19:06:25,170][105692] Updated weights for policy 0, policy_version 513828 (0.0007) [2023-12-26 19:06:25,223][105692] Updated weights for policy 0, policy_version 513838 (0.0010) [2023-12-26 19:06:25,282][105692] Updated weights for policy 0, policy_version 513848 (0.0010) [2023-12-26 19:06:25,681][105620] Updated weights for policy 1, policy_version 514386 (0.0005) [2023-12-26 19:06:25,738][105620] Updated weights for policy 1, policy_version 514396 (0.0006) [2023-12-26 19:06:25,796][105620] Updated weights for policy 1, policy_version 514406 (0.0010) [2023-12-26 19:06:26,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 263266304. Throughput: 0: 9413.9, 1: 9701.7. Samples: 263276532. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 19:06:26,063][104569] Avg episode reward: [(0, '9087.324'), (1, '9207.028')] [2023-12-26 19:06:26,137][105692] Updated weights for policy 0, policy_version 513858 (0.0009) [2023-12-26 19:06:26,200][105692] Updated weights for policy 0, policy_version 513868 (0.0007) [2023-12-26 19:06:26,255][105692] Updated weights for policy 0, policy_version 513878 (0.0006) [2023-12-26 19:06:26,325][105692] Updated weights for policy 0, policy_version 513888 (0.0005) [2023-12-26 19:06:26,354][105620] Updated weights for policy 1, policy_version 514416 (0.0009) [2023-12-26 19:06:26,405][105620] Updated weights for policy 1, policy_version 514426 (0.0010) [2023-12-26 19:06:26,457][105620] Updated weights for policy 1, policy_version 514436 (0.0010) [2023-12-26 19:06:26,898][105692] Updated weights for policy 0, policy_version 513898 (0.0005) [2023-12-26 19:06:26,961][105692] Updated weights for policy 0, policy_version 513908 (0.0005) [2023-12-26 19:06:27,020][105692] Updated weights for policy 0, policy_version 513918 (0.0006) [2023-12-26 19:06:27,185][105620] Updated weights for policy 1, policy_version 514446 (0.0010) [2023-12-26 19:06:27,238][105620] Updated weights for policy 1, policy_version 514456 (0.0009) [2023-12-26 19:06:27,296][105620] Updated weights for policy 1, policy_version 514466 (0.0010) [2023-12-26 19:06:27,602][105692] Updated weights for policy 0, policy_version 513928 (0.0005) [2023-12-26 19:06:27,651][105692] Updated weights for policy 0, policy_version 513938 (0.0006) [2023-12-26 19:06:27,698][105692] Updated weights for policy 0, policy_version 513948 (0.0005) [2023-12-26 19:06:28,022][105620] Updated weights for policy 1, policy_version 514476 (0.0010) [2023-12-26 19:06:28,069][105620] Updated weights for policy 1, policy_version 514486 (0.0010) [2023-12-26 19:06:28,113][105620] Updated weights for policy 1, policy_version 514496 (0.0010) [2023-12-26 19:06:28,299][105692] Updated weights for policy 0, policy_version 513958 (0.0008) [2023-12-26 19:06:28,356][105692] Updated weights for policy 0, policy_version 513968 (0.0010) [2023-12-26 19:06:28,404][105692] Updated weights for policy 0, policy_version 513978 (0.0010) [2023-12-26 19:06:28,772][105620] Updated weights for policy 1, policy_version 514507 (0.0009) [2023-12-26 19:06:28,839][105620] Updated weights for policy 1, policy_version 514517 (0.0005) [2023-12-26 19:06:28,906][105620] Updated weights for policy 1, policy_version 514527 (0.0005) [2023-12-26 19:06:29,075][105692] Updated weights for policy 0, policy_version 513988 (0.0008) [2023-12-26 19:06:29,126][105692] Updated weights for policy 0, policy_version 513998 (0.0009) [2023-12-26 19:06:29,180][105692] Updated weights for policy 0, policy_version 514008 (0.0005) [2023-12-26 19:06:29,478][105620] Updated weights for policy 1, policy_version 514537 (0.0005) [2023-12-26 19:06:29,532][105620] Updated weights for policy 1, policy_version 514547 (0.0005) [2023-12-26 19:06:29,605][105620] Updated weights for policy 1, policy_version 514557 (0.0005) [2023-12-26 19:06:29,676][105620] Updated weights for policy 1, policy_version 514567 (0.0006) [2023-12-26 19:06:29,769][105692] Updated weights for policy 0, policy_version 514018 (0.0007) [2023-12-26 19:06:29,830][105692] Updated weights for policy 0, policy_version 514028 (0.0008) [2023-12-26 19:06:29,885][105692] Updated weights for policy 0, policy_version 514038 (0.0007) [2023-12-26 19:06:29,944][105692] Updated weights for policy 0, policy_version 514048 (0.0009) [2023-12-26 19:06:30,400][105620] Updated weights for policy 1, policy_version 514577 (0.0009) [2023-12-26 19:06:30,460][105620] Updated weights for policy 1, policy_version 514587 (0.0009) [2023-12-26 19:06:30,514][105620] Updated weights for policy 1, policy_version 514597 (0.0010) [2023-12-26 19:06:30,601][105692] Updated weights for policy 0, policy_version 514058 (0.0009) [2023-12-26 19:06:30,651][105692] Updated weights for policy 0, policy_version 514069 (0.0009) [2023-12-26 19:06:30,699][105692] Updated weights for policy 0, policy_version 514079 (0.0006) [2023-12-26 19:06:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 263372800. Throughput: 0: 9540.6, 1: 9800.9. Samples: 263340836. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 19:06:31,062][104569] Avg episode reward: [(0, '8903.998'), (1, '9174.458')] [2023-12-26 19:06:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000514080_131620864.pth... [2023-12-26 19:06:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000514600_131751936.pth... [2023-12-26 19:06:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000513448_131457024.pth [2023-12-26 19:06:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000512960_131334144.pth [2023-12-26 19:06:31,196][105620] Updated weights for policy 1, policy_version 514608 (0.0008) [2023-12-26 19:06:31,255][105620] Updated weights for policy 1, policy_version 514618 (0.0010) [2023-12-26 19:06:31,312][105620] Updated weights for policy 1, policy_version 514628 (0.0011) [2023-12-26 19:06:31,314][105692] Updated weights for policy 0, policy_version 514089 (0.0006) [2023-12-26 19:06:31,375][105692] Updated weights for policy 0, policy_version 514099 (0.0007) [2023-12-26 19:06:31,436][105692] Updated weights for policy 0, policy_version 514109 (0.0008) [2023-12-26 19:06:32,054][105620] Updated weights for policy 1, policy_version 514638 (0.0007) [2023-12-26 19:06:32,114][105692] Updated weights for policy 0, policy_version 514119 (0.0007) [2023-12-26 19:06:32,122][105620] Updated weights for policy 1, policy_version 514648 (0.0005) [2023-12-26 19:06:32,172][105692] Updated weights for policy 0, policy_version 514129 (0.0007) [2023-12-26 19:06:32,189][105620] Updated weights for policy 1, policy_version 514658 (0.0008) [2023-12-26 19:06:32,230][105692] Updated weights for policy 0, policy_version 514139 (0.0007) [2023-12-26 19:06:32,750][105620] Updated weights for policy 1, policy_version 514668 (0.0007) [2023-12-26 19:06:32,815][105620] Updated weights for policy 1, policy_version 514678 (0.0010) [2023-12-26 19:06:32,880][105620] Updated weights for policy 1, policy_version 514688 (0.0010) [2023-12-26 19:06:32,942][105692] Updated weights for policy 0, policy_version 514149 (0.0008) [2023-12-26 19:06:32,990][105692] Updated weights for policy 0, policy_version 514159 (0.0008) [2023-12-26 19:06:32,994][105585] KL-divergence is very high: 122.1522 [2023-12-26 19:06:33,036][105585] KL-divergence is very high: 137.6378 [2023-12-26 19:06:33,041][105692] Updated weights for policy 0, policy_version 514169 (0.0008) [2023-12-26 19:06:33,590][105620] Updated weights for policy 1, policy_version 514698 (0.0010) [2023-12-26 19:06:33,648][105620] Updated weights for policy 1, policy_version 514708 (0.0009) [2023-12-26 19:06:33,708][105620] Updated weights for policy 1, policy_version 514718 (0.0008) [2023-12-26 19:06:33,768][105620] Updated weights for policy 1, policy_version 514728 (0.0006) [2023-12-26 19:06:33,816][105692] Updated weights for policy 0, policy_version 514179 (0.0009) [2023-12-26 19:06:33,868][105692] Updated weights for policy 0, policy_version 514189 (0.0009) [2023-12-26 19:06:33,919][105692] Updated weights for policy 0, policy_version 514199 (0.0010) [2023-12-26 19:06:34,388][105620] Updated weights for policy 1, policy_version 514738 (0.0006) [2023-12-26 19:06:34,449][105620] Updated weights for policy 1, policy_version 514748 (0.0006) [2023-12-26 19:06:34,509][105620] Updated weights for policy 1, policy_version 514758 (0.0005) [2023-12-26 19:06:34,661][105692] Updated weights for policy 0, policy_version 514209 (0.0009) [2023-12-26 19:06:34,723][105692] Updated weights for policy 0, policy_version 514219 (0.0009) [2023-12-26 19:06:34,779][105692] Updated weights for policy 0, policy_version 514229 (0.0009) [2023-12-26 19:06:34,846][105692] Updated weights for policy 0, policy_version 514239 (0.0009) [2023-12-26 19:06:35,133][105620] Updated weights for policy 1, policy_version 514768 (0.0008) [2023-12-26 19:06:35,188][105620] Updated weights for policy 1, policy_version 514778 (0.0007) [2023-12-26 19:06:35,252][105620] Updated weights for policy 1, policy_version 514788 (0.0005) [2023-12-26 19:06:35,612][105692] Updated weights for policy 0, policy_version 514249 (0.0009) [2023-12-26 19:06:35,665][105692] Updated weights for policy 0, policy_version 514259 (0.0008) [2023-12-26 19:06:35,715][105692] Updated weights for policy 0, policy_version 514269 (0.0008) [2023-12-26 19:06:35,908][105620] Updated weights for policy 1, policy_version 514798 (0.0008) [2023-12-26 19:06:35,963][105620] Updated weights for policy 1, policy_version 514808 (0.0008) [2023-12-26 19:06:36,027][105620] Updated weights for policy 1, policy_version 514818 (0.0007) [2023-12-26 19:06:36,062][104569] Fps is (10 sec: 21299.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 263479296. Throughput: 0: 9669.7, 1: 9907.0. Samples: 263464336. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 19:06:36,062][104569] Avg episode reward: [(0, '8714.722'), (1, '9174.281')] [2023-12-26 19:06:36,577][105692] Updated weights for policy 0, policy_version 514279 (0.0009) [2023-12-26 19:06:36,613][105620] Updated weights for policy 1, policy_version 514828 (0.0008) [2023-12-26 19:06:36,633][105692] Updated weights for policy 0, policy_version 514289 (0.0006) [2023-12-26 19:06:36,672][105620] Updated weights for policy 1, policy_version 514838 (0.0011) [2023-12-26 19:06:36,690][105692] Updated weights for policy 0, policy_version 514299 (0.0005) [2023-12-26 19:06:36,725][105620] Updated weights for policy 1, policy_version 514848 (0.0010) [2023-12-26 19:06:37,284][105692] Updated weights for policy 0, policy_version 514309 (0.0007) [2023-12-26 19:06:37,343][105692] Updated weights for policy 0, policy_version 514319 (0.0005) [2023-12-26 19:06:37,355][105620] Updated weights for policy 1, policy_version 514858 (0.0010) [2023-12-26 19:06:37,402][105692] Updated weights for policy 0, policy_version 514329 (0.0006) [2023-12-26 19:06:37,404][105620] Updated weights for policy 1, policy_version 514868 (0.0010) [2023-12-26 19:06:37,457][105620] Updated weights for policy 1, policy_version 514878 (0.0010) [2023-12-26 19:06:37,512][105620] Updated weights for policy 1, policy_version 514888 (0.0010) [2023-12-26 19:06:37,974][105692] Updated weights for policy 0, policy_version 514339 (0.0005) [2023-12-26 19:06:38,031][105692] Updated weights for policy 0, policy_version 514349 (0.0005) [2023-12-26 19:06:38,089][105692] Updated weights for policy 0, policy_version 514359 (0.0005) [2023-12-26 19:06:38,293][105620] Updated weights for policy 1, policy_version 514898 (0.0007) [2023-12-26 19:06:38,363][105620] Updated weights for policy 1, policy_version 514908 (0.0008) [2023-12-26 19:06:38,424][105620] Updated weights for policy 1, policy_version 514918 (0.0005) [2023-12-26 19:06:38,714][105692] Updated weights for policy 0, policy_version 514369 (0.0006) [2023-12-26 19:06:38,777][105692] Updated weights for policy 0, policy_version 514379 (0.0008) [2023-12-26 19:06:38,842][105692] Updated weights for policy 0, policy_version 514389 (0.0005) [2023-12-26 19:06:38,901][105692] Updated weights for policy 0, policy_version 514399 (0.0007) [2023-12-26 19:06:39,035][105620] Updated weights for policy 1, policy_version 514928 (0.0009) [2023-12-26 19:06:39,090][105620] Updated weights for policy 1, policy_version 514938 (0.0010) [2023-12-26 19:06:39,150][105620] Updated weights for policy 1, policy_version 514948 (0.0011) [2023-12-26 19:06:39,609][105692] Updated weights for policy 0, policy_version 514409 (0.0010) [2023-12-26 19:06:39,669][105692] Updated weights for policy 0, policy_version 514419 (0.0011) [2023-12-26 19:06:39,739][105692] Updated weights for policy 0, policy_version 514429 (0.0010) [2023-12-26 19:06:39,920][105620] Updated weights for policy 1, policy_version 514958 (0.0011) [2023-12-26 19:06:39,991][105620] Updated weights for policy 1, policy_version 514968 (0.0009) [2023-12-26 19:06:40,058][105620] Updated weights for policy 1, policy_version 514978 (0.0011) [2023-12-26 19:06:40,487][105692] Updated weights for policy 0, policy_version 514439 (0.0011) [2023-12-26 19:06:40,536][105692] Updated weights for policy 0, policy_version 514449 (0.0010) [2023-12-26 19:06:40,591][105692] Updated weights for policy 0, policy_version 514459 (0.0010) [2023-12-26 19:06:40,803][105620] Updated weights for policy 1, policy_version 514988 (0.0011) [2023-12-26 19:06:40,856][105620] Updated weights for policy 1, policy_version 514998 (0.0010) [2023-12-26 19:06:40,913][105620] Updated weights for policy 1, policy_version 515008 (0.0010) [2023-12-26 19:06:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 263577600. Throughput: 0: 9691.4, 1: 9976.4. Samples: 263583812. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 19:06:41,062][104569] Avg episode reward: [(0, '8989.145'), (1, '9081.669')] [2023-12-26 19:06:41,243][105692] Updated weights for policy 0, policy_version 514469 (0.0009) [2023-12-26 19:06:41,299][105692] Updated weights for policy 0, policy_version 514479 (0.0010) [2023-12-26 19:06:41,378][105692] Updated weights for policy 0, policy_version 514489 (0.0009) [2023-12-26 19:06:41,733][105620] Updated weights for policy 1, policy_version 515018 (0.0010) [2023-12-26 19:06:41,789][105620] Updated weights for policy 1, policy_version 515028 (0.0008) [2023-12-26 19:06:41,846][105620] Updated weights for policy 1, policy_version 515038 (0.0008) [2023-12-26 19:06:41,910][105620] Updated weights for policy 1, policy_version 515048 (0.0009) [2023-12-26 19:06:42,071][105692] Updated weights for policy 0, policy_version 514499 (0.0010) [2023-12-26 19:06:42,127][105692] Updated weights for policy 0, policy_version 514509 (0.0010) [2023-12-26 19:06:42,183][105692] Updated weights for policy 0, policy_version 514519 (0.0011) [2023-12-26 19:06:42,735][105620] Updated weights for policy 1, policy_version 515058 (0.0009) [2023-12-26 19:06:42,805][105620] Updated weights for policy 1, policy_version 515068 (0.0009) [2023-12-26 19:06:42,864][105620] Updated weights for policy 1, policy_version 515078 (0.0010) [2023-12-26 19:06:42,895][105692] Updated weights for policy 0, policy_version 514529 (0.0009) [2023-12-26 19:06:42,954][105692] Updated weights for policy 0, policy_version 514539 (0.0009) [2023-12-26 19:06:43,018][105692] Updated weights for policy 0, policy_version 514549 (0.0009) [2023-12-26 19:06:43,078][105692] Updated weights for policy 0, policy_version 514559 (0.0009) [2023-12-26 19:06:43,635][105620] Updated weights for policy 1, policy_version 515088 (0.0008) [2023-12-26 19:06:43,689][105620] Updated weights for policy 1, policy_version 515098 (0.0007) [2023-12-26 19:06:43,706][105692] Updated weights for policy 0, policy_version 514569 (0.0010) [2023-12-26 19:06:43,749][105620] Updated weights for policy 1, policy_version 515108 (0.0007) [2023-12-26 19:06:43,761][105692] Updated weights for policy 0, policy_version 514579 (0.0006) [2023-12-26 19:06:43,810][105692] Updated weights for policy 0, policy_version 514589 (0.0006) [2023-12-26 19:06:44,372][105620] Updated weights for policy 1, policy_version 515118 (0.0007) [2023-12-26 19:06:44,374][105692] Updated weights for policy 0, policy_version 514599 (0.0007) [2023-12-26 19:06:44,419][105620] Updated weights for policy 1, policy_version 515128 (0.0007) [2023-12-26 19:06:44,427][105692] Updated weights for policy 0, policy_version 514609 (0.0006) [2023-12-26 19:06:44,466][105620] Updated weights for policy 1, policy_version 515138 (0.0008) [2023-12-26 19:06:44,471][105692] Updated weights for policy 0, policy_version 514619 (0.0005) [2023-12-26 19:06:45,147][105620] Updated weights for policy 1, policy_version 515148 (0.0008) [2023-12-26 19:06:45,154][105692] Updated weights for policy 0, policy_version 514629 (0.0008) [2023-12-26 19:06:45,199][105620] Updated weights for policy 1, policy_version 515158 (0.0006) [2023-12-26 19:06:45,200][105692] Updated weights for policy 0, policy_version 514639 (0.0011) [2023-12-26 19:06:45,256][105692] Updated weights for policy 0, policy_version 514649 (0.0011) [2023-12-26 19:06:45,256][105620] Updated weights for policy 1, policy_version 515168 (0.0005) [2023-12-26 19:06:45,821][105620] Updated weights for policy 1, policy_version 515178 (0.0006) [2023-12-26 19:06:45,879][105620] Updated weights for policy 1, policy_version 515188 (0.0008) [2023-12-26 19:06:45,923][105692] Updated weights for policy 0, policy_version 514659 (0.0011) [2023-12-26 19:06:45,942][105620] Updated weights for policy 1, policy_version 515198 (0.0006) [2023-12-26 19:06:45,985][105692] Updated weights for policy 0, policy_version 514669 (0.0010) [2023-12-26 19:06:45,995][105620] Updated weights for policy 1, policy_version 515208 (0.0006) [2023-12-26 19:06:46,040][105692] Updated weights for policy 0, policy_version 514679 (0.0010) [2023-12-26 19:06:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 263675904. Throughput: 0: 9725.8, 1: 9947.2. Samples: 263641104. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 19:06:46,062][104569] Avg episode reward: [(0, '9169.643'), (1, '9149.050')] [2023-12-26 19:06:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000515208_131907584.pth... [2023-12-26 19:06:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000514024_131604480.pth [2023-12-26 19:06:46,089][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000514688_131776512.pth... [2023-12-26 19:06:46,095][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000513504_131473408.pth [2023-12-26 19:06:46,684][105692] Updated weights for policy 0, policy_version 514689 (0.0010) [2023-12-26 19:06:46,731][105620] Updated weights for policy 1, policy_version 515218 (0.0009) [2023-12-26 19:06:46,749][105692] Updated weights for policy 0, policy_version 514699 (0.0011) [2023-12-26 19:06:46,786][105620] Updated weights for policy 1, policy_version 515228 (0.0005) [2023-12-26 19:06:46,811][105692] Updated weights for policy 0, policy_version 514709 (0.0010) [2023-12-26 19:06:46,848][105620] Updated weights for policy 1, policy_version 515238 (0.0005) [2023-12-26 19:06:46,873][105692] Updated weights for policy 0, policy_version 514719 (0.0010) [2023-12-26 19:06:47,393][105620] Updated weights for policy 1, policy_version 515248 (0.0005) [2023-12-26 19:06:47,440][105620] Updated weights for policy 1, policy_version 515258 (0.0005) [2023-12-26 19:06:47,483][105620] Updated weights for policy 1, policy_version 515268 (0.0005) [2023-12-26 19:06:47,599][105692] Updated weights for policy 0, policy_version 514729 (0.0009) [2023-12-26 19:06:47,651][105692] Updated weights for policy 0, policy_version 514739 (0.0009) [2023-12-26 19:06:47,697][105692] Updated weights for policy 0, policy_version 514749 (0.0009) [2023-12-26 19:06:48,182][105620] Updated weights for policy 1, policy_version 515278 (0.0008) [2023-12-26 19:06:48,239][105620] Updated weights for policy 1, policy_version 515288 (0.0008) [2023-12-26 19:06:48,304][105620] Updated weights for policy 1, policy_version 515298 (0.0008) [2023-12-26 19:06:48,458][105692] Updated weights for policy 0, policy_version 514759 (0.0010) [2023-12-26 19:06:48,513][105692] Updated weights for policy 0, policy_version 514769 (0.0010) [2023-12-26 19:06:48,572][105692] Updated weights for policy 0, policy_version 514779 (0.0010) [2023-12-26 19:06:49,011][105620] Updated weights for policy 1, policy_version 515308 (0.0009) [2023-12-26 19:06:49,068][105620] Updated weights for policy 1, policy_version 515318 (0.0008) [2023-12-26 19:06:49,131][105620] Updated weights for policy 1, policy_version 515328 (0.0008) [2023-12-26 19:06:49,320][105692] Updated weights for policy 0, policy_version 514789 (0.0009) [2023-12-26 19:06:49,387][105692] Updated weights for policy 0, policy_version 514799 (0.0010) [2023-12-26 19:06:49,447][105692] Updated weights for policy 0, policy_version 514809 (0.0011) [2023-12-26 19:06:49,844][105620] Updated weights for policy 1, policy_version 515338 (0.0008) [2023-12-26 19:06:49,905][105620] Updated weights for policy 1, policy_version 515348 (0.0006) [2023-12-26 19:06:49,972][105620] Updated weights for policy 1, policy_version 515358 (0.0009) [2023-12-26 19:06:50,021][105620] Updated weights for policy 1, policy_version 515368 (0.0008) [2023-12-26 19:06:50,219][105692] Updated weights for policy 0, policy_version 514819 (0.0011) [2023-12-26 19:06:50,277][105692] Updated weights for policy 0, policy_version 514829 (0.0011) [2023-12-26 19:06:50,335][105692] Updated weights for policy 0, policy_version 514839 (0.0010) [2023-12-26 19:06:50,756][105620] Updated weights for policy 1, policy_version 515378 (0.0008) [2023-12-26 19:06:50,821][105620] Updated weights for policy 1, policy_version 515388 (0.0011) [2023-12-26 19:06:50,879][105620] Updated weights for policy 1, policy_version 515398 (0.0011) [2023-12-26 19:06:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.9, 300 sec: 19522.0). Total num frames: 263774208. Throughput: 0: 9863.0, 1: 9981.5. Samples: 263765088. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 19:06:51,062][104569] Avg episode reward: [(0, '9169.694'), (1, '8807.014')] [2023-12-26 19:06:51,093][105692] Updated weights for policy 0, policy_version 514849 (0.0010) [2023-12-26 19:06:51,150][105692] Updated weights for policy 0, policy_version 514859 (0.0011) [2023-12-26 19:06:51,202][105692] Updated weights for policy 0, policy_version 514869 (0.0010) [2023-12-26 19:06:51,254][105692] Updated weights for policy 0, policy_version 514879 (0.0011) [2023-12-26 19:06:51,701][105620] Updated weights for policy 1, policy_version 515408 (0.0011) [2023-12-26 19:06:51,773][105620] Updated weights for policy 1, policy_version 515418 (0.0008) [2023-12-26 19:06:51,840][105620] Updated weights for policy 1, policy_version 515428 (0.0005) [2023-12-26 19:06:52,046][105585] KL-divergence is very high: 208.7889 [2023-12-26 19:06:52,051][105692] Updated weights for policy 0, policy_version 514889 (0.0011) [2023-12-26 19:06:52,064][105585] KL-divergence is very high: 637.3270 [2023-12-26 19:06:52,070][105585] KL-divergence is very high: 128.4102 [2023-12-26 19:06:52,077][105585] KL-divergence is very high: 134.8946 [2023-12-26 19:06:52,083][105585] KL-divergence is very high: 151.0164 [2023-12-26 19:06:52,096][105585] KL-divergence is very high: 568.7845 [2023-12-26 19:06:52,103][105585] KL-divergence is very high: 163.1409 [2023-12-26 19:06:52,109][105585] KL-divergence is very high: 154.5928 [2023-12-26 19:06:52,114][105692] Updated weights for policy 0, policy_version 514899 (0.0011) [2023-12-26 19:06:52,115][105585] KL-divergence is very high: 1066.5920 [2023-12-26 19:06:52,121][105585] KL-divergence is very high: 160.8598 [2023-12-26 19:06:52,128][105585] KL-divergence is very high: 133.2705 [2023-12-26 19:06:52,134][105585] KL-divergence is very high: 139.2004 [2023-12-26 19:06:52,146][105585] KL-divergence is very high: 605.0099 [2023-12-26 19:06:52,153][105585] KL-divergence is very high: 156.3435 [2023-12-26 19:06:52,159][105585] KL-divergence is very high: 136.2057 [2023-12-26 19:06:52,165][105585] KL-divergence is very high: 1045.8893 [2023-12-26 19:06:52,172][105585] KL-divergence is very high: 132.4534 [2023-12-26 19:06:52,179][105692] Updated weights for policy 0, policy_version 514909 (0.0011) [2023-12-26 19:06:52,536][105620] Updated weights for policy 1, policy_version 515438 (0.0008) [2023-12-26 19:06:52,598][105620] Updated weights for policy 1, policy_version 515448 (0.0010) [2023-12-26 19:06:52,664][105620] Updated weights for policy 1, policy_version 515458 (0.0010) [2023-12-26 19:06:52,804][105692] Updated weights for policy 0, policy_version 514919 (0.0010) [2023-12-26 19:06:52,865][105692] Updated weights for policy 0, policy_version 514929 (0.0009) [2023-12-26 19:06:52,920][105692] Updated weights for policy 0, policy_version 514939 (0.0008) [2023-12-26 19:06:53,289][105620] Updated weights for policy 1, policy_version 515468 (0.0008) [2023-12-26 19:06:53,337][105620] Updated weights for policy 1, policy_version 515478 (0.0005) [2023-12-26 19:06:53,390][105620] Updated weights for policy 1, policy_version 515488 (0.0006) [2023-12-26 19:06:53,743][105692] Updated weights for policy 0, policy_version 514949 (0.0010) [2023-12-26 19:06:53,801][105692] Updated weights for policy 0, policy_version 514959 (0.0010) [2023-12-26 19:06:53,866][105692] Updated weights for policy 0, policy_version 514969 (0.0010) [2023-12-26 19:06:53,986][105620] Updated weights for policy 1, policy_version 515498 (0.0008) [2023-12-26 19:06:54,044][105620] Updated weights for policy 1, policy_version 515508 (0.0008) [2023-12-26 19:06:54,094][105620] Updated weights for policy 1, policy_version 515518 (0.0007) [2023-12-26 19:06:54,149][105620] Updated weights for policy 1, policy_version 515528 (0.0009) [2023-12-26 19:06:54,538][105692] Updated weights for policy 0, policy_version 514979 (0.0010) [2023-12-26 19:06:54,591][105692] Updated weights for policy 0, policy_version 514989 (0.0009) [2023-12-26 19:06:54,658][105692] Updated weights for policy 0, policy_version 514999 (0.0009) [2023-12-26 19:06:54,775][105620] Updated weights for policy 1, policy_version 515538 (0.0005) [2023-12-26 19:06:54,836][105620] Updated weights for policy 1, policy_version 515548 (0.0005) [2023-12-26 19:06:54,891][105620] Updated weights for policy 1, policy_version 515558 (0.0005) [2023-12-26 19:06:55,408][105692] Updated weights for policy 0, policy_version 515009 (0.0010) [2023-12-26 19:06:55,455][105692] Updated weights for policy 0, policy_version 515019 (0.0010) [2023-12-26 19:06:55,491][105620] Updated weights for policy 1, policy_version 515568 (0.0005) [2023-12-26 19:06:55,517][105692] Updated weights for policy 0, policy_version 515029 (0.0010) [2023-12-26 19:06:55,549][105620] Updated weights for policy 1, policy_version 515578 (0.0005) [2023-12-26 19:06:55,574][105692] Updated weights for policy 0, policy_version 515039 (0.0009) [2023-12-26 19:06:55,609][105620] Updated weights for policy 1, policy_version 515588 (0.0005) [2023-12-26 19:06:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 263872512. Throughput: 0: 9844.4, 1: 10059.8. Samples: 263883360. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 19:06:56,062][104569] Avg episode reward: [(0, '8988.683'), (1, '9185.488')] [2023-12-26 19:06:56,149][105620] Updated weights for policy 1, policy_version 515598 (0.0006) [2023-12-26 19:06:56,201][105620] Updated weights for policy 1, policy_version 515608 (0.0008) [2023-12-26 19:06:56,246][105620] Updated weights for policy 1, policy_version 515618 (0.0008) [2023-12-26 19:06:56,324][105692] Updated weights for policy 0, policy_version 515049 (0.0010) [2023-12-26 19:06:56,367][105692] Updated weights for policy 0, policy_version 515059 (0.0010) [2023-12-26 19:06:56,418][105692] Updated weights for policy 0, policy_version 515069 (0.0010) [2023-12-26 19:06:56,996][105620] Updated weights for policy 1, policy_version 515628 (0.0008) [2023-12-26 19:06:57,040][105620] Updated weights for policy 1, policy_version 515638 (0.0008) [2023-12-26 19:06:57,083][105620] Updated weights for policy 1, policy_version 515648 (0.0007) [2023-12-26 19:06:57,163][105692] Updated weights for policy 0, policy_version 515079 (0.0007) [2023-12-26 19:06:57,208][105692] Updated weights for policy 0, policy_version 515089 (0.0005) [2023-12-26 19:06:57,265][105692] Updated weights for policy 0, policy_version 515099 (0.0005) [2023-12-26 19:06:57,787][105620] Updated weights for policy 1, policy_version 515658 (0.0008) [2023-12-26 19:06:57,843][105620] Updated weights for policy 1, policy_version 515668 (0.0005) [2023-12-26 19:06:57,854][105692] Updated weights for policy 0, policy_version 515109 (0.0008) [2023-12-26 19:06:57,904][105620] Updated weights for policy 1, policy_version 515678 (0.0005) [2023-12-26 19:06:57,918][105692] Updated weights for policy 0, policy_version 515119 (0.0007) [2023-12-26 19:06:57,971][105620] Updated weights for policy 1, policy_version 515688 (0.0005) [2023-12-26 19:06:57,985][105692] Updated weights for policy 0, policy_version 515129 (0.0009) [2023-12-26 19:06:58,683][105620] Updated weights for policy 1, policy_version 515698 (0.0008) [2023-12-26 19:06:58,691][105692] Updated weights for policy 0, policy_version 515140 (0.0007) [2023-12-26 19:06:58,761][105692] Updated weights for policy 0, policy_version 515150 (0.0011) [2023-12-26 19:06:58,762][105620] Updated weights for policy 1, policy_version 515708 (0.0008) [2023-12-26 19:06:58,762][105585] KL-divergence is very high: 101.1122 [2023-12-26 19:06:58,788][105585] KL-divergence is very high: 220.2098 [2023-12-26 19:06:58,817][105585] KL-divergence is very high: 197.6856 [2023-12-26 19:06:58,829][105620] Updated weights for policy 1, policy_version 515718 (0.0008) [2023-12-26 19:06:58,830][105692] Updated weights for policy 0, policy_version 515160 (0.0008) [2023-12-26 19:06:58,846][105585] KL-divergence is very high: 197.0605 [2023-12-26 19:06:58,871][105585] KL-divergence is very high: 139.7461 [2023-12-26 19:06:59,508][105620] Updated weights for policy 1, policy_version 515728 (0.0009) [2023-12-26 19:06:59,562][105620] Updated weights for policy 1, policy_version 515739 (0.0010) [2023-12-26 19:06:59,590][105692] Updated weights for policy 0, policy_version 515170 (0.0006) [2023-12-26 19:06:59,613][105620] Updated weights for policy 1, policy_version 515749 (0.0008) [2023-12-26 19:06:59,651][105692] Updated weights for policy 0, policy_version 515180 (0.0007) [2023-12-26 19:06:59,708][105692] Updated weights for policy 0, policy_version 515190 (0.0010) [2023-12-26 19:06:59,769][105692] Updated weights for policy 0, policy_version 515200 (0.0010) [2023-12-26 19:07:00,391][105620] Updated weights for policy 1, policy_version 515759 (0.0009) [2023-12-26 19:07:00,444][105620] Updated weights for policy 1, policy_version 515769 (0.0010) [2023-12-26 19:07:00,479][105692] Updated weights for policy 0, policy_version 515210 (0.0009) [2023-12-26 19:07:00,499][105620] Updated weights for policy 1, policy_version 515779 (0.0010) [2023-12-26 19:07:00,530][105692] Updated weights for policy 0, policy_version 515220 (0.0010) [2023-12-26 19:07:00,588][105692] Updated weights for policy 0, policy_version 515230 (0.0010) [2023-12-26 19:07:01,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.7, 300 sec: 19494.2). Total num frames: 263970816. Throughput: 0: 9930.1, 1: 10106.3. Samples: 263943504. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-26 19:07:01,063][104569] Avg episode reward: [(0, '8895.409'), (1, '9352.785')] [2023-12-26 19:07:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000515232_131915776.pth... [2023-12-26 19:07:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000515784_132055040.pth... [2023-12-26 19:07:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000514080_131620864.pth [2023-12-26 19:07:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000514600_131751936.pth [2023-12-26 19:07:01,136][105620] Updated weights for policy 1, policy_version 515789 (0.0010) [2023-12-26 19:07:01,200][105620] Updated weights for policy 1, policy_version 515799 (0.0007) [2023-12-26 19:07:01,254][105620] Updated weights for policy 1, policy_version 515809 (0.0006) [2023-12-26 19:07:01,314][105692] Updated weights for policy 0, policy_version 515240 (0.0010) [2023-12-26 19:07:01,375][105692] Updated weights for policy 0, policy_version 515250 (0.0010) [2023-12-26 19:07:01,432][105692] Updated weights for policy 0, policy_version 515260 (0.0010) [2023-12-26 19:07:01,945][105620] Updated weights for policy 1, policy_version 515819 (0.0010) [2023-12-26 19:07:02,012][105620] Updated weights for policy 1, policy_version 515829 (0.0011) [2023-12-26 19:07:02,072][105620] Updated weights for policy 1, policy_version 515839 (0.0011) [2023-12-26 19:07:02,184][105692] Updated weights for policy 0, policy_version 515270 (0.0008) [2023-12-26 19:07:02,250][105692] Updated weights for policy 0, policy_version 515280 (0.0006) [2023-12-26 19:07:02,319][105692] Updated weights for policy 0, policy_version 515290 (0.0006) [2023-12-26 19:07:02,729][105620] Updated weights for policy 1, policy_version 515849 (0.0011) [2023-12-26 19:07:02,791][105620] Updated weights for policy 1, policy_version 515859 (0.0010) [2023-12-26 19:07:02,857][105620] Updated weights for policy 1, policy_version 515869 (0.0010) [2023-12-26 19:07:02,913][105620] Updated weights for policy 1, policy_version 515879 (0.0010) [2023-12-26 19:07:02,919][105692] Updated weights for policy 0, policy_version 515300 (0.0009) [2023-12-26 19:07:02,976][105692] Updated weights for policy 0, policy_version 515310 (0.0008) [2023-12-26 19:07:03,024][105692] Updated weights for policy 0, policy_version 515320 (0.0008) [2023-12-26 19:07:03,566][105620] Updated weights for policy 1, policy_version 515889 (0.0010) [2023-12-26 19:07:03,626][105620] Updated weights for policy 1, policy_version 515899 (0.0009) [2023-12-26 19:07:03,630][105692] Updated weights for policy 0, policy_version 515330 (0.0007) [2023-12-26 19:07:03,675][105620] Updated weights for policy 1, policy_version 515909 (0.0009) [2023-12-26 19:07:03,684][105692] Updated weights for policy 0, policy_version 515340 (0.0005) [2023-12-26 19:07:03,732][105692] Updated weights for policy 0, policy_version 515350 (0.0005) [2023-12-26 19:07:03,786][105692] Updated weights for policy 0, policy_version 515360 (0.0006) [2023-12-26 19:07:04,490][105620] Updated weights for policy 1, policy_version 515919 (0.0008) [2023-12-26 19:07:04,513][105692] Updated weights for policy 0, policy_version 515370 (0.0008) [2023-12-26 19:07:04,544][105620] Updated weights for policy 1, policy_version 515929 (0.0006) [2023-12-26 19:07:04,570][105692] Updated weights for policy 0, policy_version 515380 (0.0006) [2023-12-26 19:07:04,603][105620] Updated weights for policy 1, policy_version 515939 (0.0008) [2023-12-26 19:07:04,625][105692] Updated weights for policy 0, policy_version 515390 (0.0009) [2023-12-26 19:07:05,316][105620] Updated weights for policy 1, policy_version 515949 (0.0007) [2023-12-26 19:07:05,358][105692] Updated weights for policy 0, policy_version 515400 (0.0008) [2023-12-26 19:07:05,372][105620] Updated weights for policy 1, policy_version 515959 (0.0007) [2023-12-26 19:07:05,415][105692] Updated weights for policy 0, policy_version 515410 (0.0009) [2023-12-26 19:07:05,421][105620] Updated weights for policy 1, policy_version 515969 (0.0006) [2023-12-26 19:07:05,469][105692] Updated weights for policy 0, policy_version 515420 (0.0008) [2023-12-26 19:07:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19494.2). Total num frames: 264069120. Throughput: 0: 10015.3, 1: 10069.6. Samples: 264062708. Policy #0 lag: (min: 4.0, avg: 14.0, max: 36.0) [2023-12-26 19:07:06,062][104569] Avg episode reward: [(0, '8984.161'), (1, '9175.129')] [2023-12-26 19:07:06,123][105620] Updated weights for policy 1, policy_version 515979 (0.0008) [2023-12-26 19:07:06,182][105620] Updated weights for policy 1, policy_version 515989 (0.0011) [2023-12-26 19:07:06,205][105692] Updated weights for policy 0, policy_version 515430 (0.0007) [2023-12-26 19:07:06,231][105620] Updated weights for policy 1, policy_version 515999 (0.0011) [2023-12-26 19:07:06,263][105692] Updated weights for policy 0, policy_version 515440 (0.0007) [2023-12-26 19:07:06,316][105692] Updated weights for policy 0, policy_version 515450 (0.0009) [2023-12-26 19:07:06,790][105620] Updated weights for policy 1, policy_version 516009 (0.0007) [2023-12-26 19:07:06,849][105620] Updated weights for policy 1, policy_version 516019 (0.0007) [2023-12-26 19:07:06,909][105620] Updated weights for policy 1, policy_version 516029 (0.0005) [2023-12-26 19:07:06,969][105620] Updated weights for policy 1, policy_version 516039 (0.0010) [2023-12-26 19:07:07,192][105692] Updated weights for policy 0, policy_version 515460 (0.0010) [2023-12-26 19:07:07,244][105692] Updated weights for policy 0, policy_version 515470 (0.0008) [2023-12-26 19:07:07,305][105692] Updated weights for policy 0, policy_version 515480 (0.0008) [2023-12-26 19:07:07,658][105620] Updated weights for policy 1, policy_version 516049 (0.0010) [2023-12-26 19:07:07,717][105620] Updated weights for policy 1, policy_version 516059 (0.0010) [2023-12-26 19:07:07,775][105620] Updated weights for policy 1, policy_version 516069 (0.0010) [2023-12-26 19:07:08,001][105692] Updated weights for policy 0, policy_version 515490 (0.0008) [2023-12-26 19:07:08,055][105692] Updated weights for policy 0, policy_version 515500 (0.0005) [2023-12-26 19:07:08,123][105692] Updated weights for policy 0, policy_version 515510 (0.0006) [2023-12-26 19:07:08,175][105692] Updated weights for policy 0, policy_version 515520 (0.0008) [2023-12-26 19:07:08,492][105620] Updated weights for policy 1, policy_version 516079 (0.0010) [2023-12-26 19:07:08,550][105620] Updated weights for policy 1, policy_version 516089 (0.0011) [2023-12-26 19:07:08,605][105620] Updated weights for policy 1, policy_version 516099 (0.0010) [2023-12-26 19:07:08,885][105692] Updated weights for policy 0, policy_version 515530 (0.0008) [2023-12-26 19:07:08,937][105692] Updated weights for policy 0, policy_version 515540 (0.0010) [2023-12-26 19:07:08,996][105692] Updated weights for policy 0, policy_version 515550 (0.0010) [2023-12-26 19:07:09,264][105620] Updated weights for policy 1, policy_version 516109 (0.0009) [2023-12-26 19:07:09,340][105620] Updated weights for policy 1, policy_version 516119 (0.0008) [2023-12-26 19:07:09,407][105620] Updated weights for policy 1, policy_version 516129 (0.0009) [2023-12-26 19:07:09,865][105692] Updated weights for policy 0, policy_version 515560 (0.0009) [2023-12-26 19:07:09,932][105692] Updated weights for policy 0, policy_version 515570 (0.0008) [2023-12-26 19:07:09,988][105692] Updated weights for policy 0, policy_version 515580 (0.0007) [2023-12-26 19:07:10,128][105620] Updated weights for policy 1, policy_version 516139 (0.0008) [2023-12-26 19:07:10,222][105620] Updated weights for policy 1, policy_version 516149 (0.0011) [2023-12-26 19:07:10,279][105620] Updated weights for policy 1, policy_version 516159 (0.0011) [2023-12-26 19:07:10,786][105692] Updated weights for policy 0, policy_version 515590 (0.0007) [2023-12-26 19:07:10,855][105692] Updated weights for policy 0, policy_version 515600 (0.0008) [2023-12-26 19:07:10,905][105620] Updated weights for policy 1, policy_version 516169 (0.0010) [2023-12-26 19:07:10,911][105692] Updated weights for policy 0, policy_version 515610 (0.0010) [2023-12-26 19:07:10,957][105620] Updated weights for policy 1, policy_version 516179 (0.0009) [2023-12-26 19:07:11,011][105620] Updated weights for policy 1, policy_version 516189 (0.0009) [2023-12-26 19:07:11,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19797.4, 300 sec: 19494.2). Total num frames: 264167424. Throughput: 0: 9913.7, 1: 10137.3. Samples: 264178820. Policy #0 lag: (min: 4.0, avg: 14.0, max: 36.0) [2023-12-26 19:07:11,062][104569] Avg episode reward: [(0, '9262.192'), (1, '8764.058')] [2023-12-26 19:07:11,074][105620] Updated weights for policy 1, policy_version 516199 (0.0009) [2023-12-26 19:07:11,653][105692] Updated weights for policy 0, policy_version 515620 (0.0007) [2023-12-26 19:07:11,722][105692] Updated weights for policy 0, policy_version 515630 (0.0010) [2023-12-26 19:07:11,788][105620] Updated weights for policy 1, policy_version 516209 (0.0008) [2023-12-26 19:07:11,798][105692] Updated weights for policy 0, policy_version 515640 (0.0008) [2023-12-26 19:07:11,838][105620] Updated weights for policy 1, policy_version 516219 (0.0006) [2023-12-26 19:07:11,885][105620] Updated weights for policy 1, policy_version 516229 (0.0009) [2023-12-26 19:07:12,507][105692] Updated weights for policy 0, policy_version 515650 (0.0007) [2023-12-26 19:07:12,556][105620] Updated weights for policy 1, policy_version 516239 (0.0007) [2023-12-26 19:07:12,570][105692] Updated weights for policy 0, policy_version 515660 (0.0008) [2023-12-26 19:07:12,607][105586] KL-divergence is very high: 247.6670 [2023-12-26 19:07:12,614][105620] Updated weights for policy 1, policy_version 516249 (0.0007) [2023-12-26 19:07:12,614][105586] KL-divergence is very high: 150.7813 [2023-12-26 19:07:12,621][105586] KL-divergence is very high: 283.4298 [2023-12-26 19:07:12,629][105692] Updated weights for policy 0, policy_version 515670 (0.0009) [2023-12-26 19:07:12,634][105586] KL-divergence is very high: 177.8340 [2023-12-26 19:07:12,641][105586] KL-divergence is very high: 122.8662 [2023-12-26 19:07:12,661][105586] KL-divergence is very high: 110.3071 [2023-12-26 19:07:12,680][105620] Updated weights for policy 1, policy_version 516259 (0.0007) [2023-12-26 19:07:12,686][105692] Updated weights for policy 0, policy_version 515680 (0.0006) [2023-12-26 19:07:13,306][105692] Updated weights for policy 0, policy_version 515690 (0.0005) [2023-12-26 19:07:13,355][105692] Updated weights for policy 0, policy_version 515700 (0.0005) [2023-12-26 19:07:13,405][105692] Updated weights for policy 0, policy_version 515710 (0.0007) [2023-12-26 19:07:13,432][105620] Updated weights for policy 1, policy_version 516269 (0.0008) [2023-12-26 19:07:13,495][105620] Updated weights for policy 1, policy_version 516279 (0.0009) [2023-12-26 19:07:13,549][105620] Updated weights for policy 1, policy_version 516290 (0.0010) [2023-12-26 19:07:13,985][105692] Updated weights for policy 0, policy_version 515720 (0.0008) [2023-12-26 19:07:14,046][105692] Updated weights for policy 0, policy_version 515730 (0.0009) [2023-12-26 19:07:14,109][105692] Updated weights for policy 0, policy_version 515740 (0.0009) [2023-12-26 19:07:14,371][105620] Updated weights for policy 1, policy_version 516300 (0.0010) [2023-12-26 19:07:14,432][105620] Updated weights for policy 1, policy_version 516310 (0.0009) [2023-12-26 19:07:14,486][105620] Updated weights for policy 1, policy_version 516320 (0.0005) [2023-12-26 19:07:14,906][105692] Updated weights for policy 0, policy_version 515750 (0.0009) [2023-12-26 19:07:14,975][105692] Updated weights for policy 0, policy_version 515760 (0.0006) [2023-12-26 19:07:15,039][105692] Updated weights for policy 0, policy_version 515770 (0.0010) [2023-12-26 19:07:15,048][105620] Updated weights for policy 1, policy_version 516330 (0.0005) [2023-12-26 19:07:15,111][105620] Updated weights for policy 1, policy_version 516340 (0.0007) [2023-12-26 19:07:15,175][105620] Updated weights for policy 1, policy_version 516350 (0.0009) [2023-12-26 19:07:15,238][105620] Updated weights for policy 1, policy_version 516360 (0.0009) [2023-12-26 19:07:15,631][105692] Updated weights for policy 0, policy_version 515780 (0.0007) [2023-12-26 19:07:15,694][105692] Updated weights for policy 0, policy_version 515790 (0.0006) [2023-12-26 19:07:15,753][105692] Updated weights for policy 0, policy_version 515800 (0.0008) [2023-12-26 19:07:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19522.0). Total num frames: 264265728. Throughput: 0: 9845.2, 1: 10059.7. Samples: 264236552. Policy #0 lag: (min: 4.0, avg: 14.0, max: 36.0) [2023-12-26 19:07:16,063][104569] Avg episode reward: [(0, '9169.523'), (1, '7747.013')] [2023-12-26 19:07:16,065][105620] Updated weights for policy 1, policy_version 516370 (0.0010) [2023-12-26 19:07:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000515808_132063232.pth... [2023-12-26 19:07:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000514688_131776512.pth [2023-12-26 19:07:16,125][105620] Updated weights for policy 1, policy_version 516380 (0.0010) [2023-12-26 19:07:16,183][105620] Updated weights for policy 1, policy_version 516390 (0.0010) [2023-12-26 19:07:16,194][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000516392_132210688.pth... [2023-12-26 19:07:16,198][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000515208_131907584.pth [2023-12-26 19:07:16,463][105692] Updated weights for policy 0, policy_version 515810 (0.0008) [2023-12-26 19:07:16,529][105692] Updated weights for policy 0, policy_version 515820 (0.0008) [2023-12-26 19:07:16,593][105692] Updated weights for policy 0, policy_version 515830 (0.0008) [2023-12-26 19:07:16,648][105692] Updated weights for policy 0, policy_version 515840 (0.0008) [2023-12-26 19:07:16,913][105620] Updated weights for policy 1, policy_version 516400 (0.0010) [2023-12-26 19:07:16,980][105620] Updated weights for policy 1, policy_version 516410 (0.0010) [2023-12-26 19:07:17,037][105620] Updated weights for policy 1, policy_version 516420 (0.0010) [2023-12-26 19:07:17,348][105692] Updated weights for policy 0, policy_version 515850 (0.0005) [2023-12-26 19:07:17,394][105692] Updated weights for policy 0, policy_version 515860 (0.0005) [2023-12-26 19:07:17,448][105692] Updated weights for policy 0, policy_version 515870 (0.0005) [2023-12-26 19:07:17,764][105620] Updated weights for policy 1, policy_version 516430 (0.0007) [2023-12-26 19:07:17,817][105620] Updated weights for policy 1, policy_version 516440 (0.0010) [2023-12-26 19:07:17,862][105620] Updated weights for policy 1, policy_version 516450 (0.0010) [2023-12-26 19:07:18,039][105692] Updated weights for policy 0, policy_version 515880 (0.0005) [2023-12-26 19:07:18,086][105692] Updated weights for policy 0, policy_version 515890 (0.0005) [2023-12-26 19:07:18,130][105692] Updated weights for policy 0, policy_version 515900 (0.0005) [2023-12-26 19:07:18,505][105620] Updated weights for policy 1, policy_version 516460 (0.0009) [2023-12-26 19:07:18,567][105620] Updated weights for policy 1, policy_version 516470 (0.0011) [2023-12-26 19:07:18,628][105620] Updated weights for policy 1, policy_version 516480 (0.0007) [2023-12-26 19:07:18,810][105692] Updated weights for policy 0, policy_version 515910 (0.0006) [2023-12-26 19:07:18,861][105692] Updated weights for policy 0, policy_version 515920 (0.0008) [2023-12-26 19:07:18,917][105692] Updated weights for policy 0, policy_version 515930 (0.0008) [2023-12-26 19:07:19,332][105620] Updated weights for policy 1, policy_version 516490 (0.0010) [2023-12-26 19:07:19,393][105620] Updated weights for policy 1, policy_version 516500 (0.0009) [2023-12-26 19:07:19,443][105620] Updated weights for policy 1, policy_version 516510 (0.0007) [2023-12-26 19:07:19,503][105620] Updated weights for policy 1, policy_version 516520 (0.0009) [2023-12-26 19:07:19,693][105692] Updated weights for policy 0, policy_version 515940 (0.0009) [2023-12-26 19:07:19,752][105692] Updated weights for policy 0, policy_version 515950 (0.0010) [2023-12-26 19:07:19,817][105692] Updated weights for policy 0, policy_version 515960 (0.0008) [2023-12-26 19:07:20,199][105620] Updated weights for policy 1, policy_version 516530 (0.0008) [2023-12-26 19:07:20,275][105620] Updated weights for policy 1, policy_version 516540 (0.0006) [2023-12-26 19:07:20,336][105620] Updated weights for policy 1, policy_version 516550 (0.0006) [2023-12-26 19:07:20,622][105692] Updated weights for policy 0, policy_version 515970 (0.0006) [2023-12-26 19:07:20,680][105692] Updated weights for policy 0, policy_version 515980 (0.0008) [2023-12-26 19:07:20,735][105692] Updated weights for policy 0, policy_version 515990 (0.0009) [2023-12-26 19:07:20,791][105692] Updated weights for policy 0, policy_version 516000 (0.0009) [2023-12-26 19:07:21,034][105620] Updated weights for policy 1, policy_version 516560 (0.0008) [2023-12-26 19:07:21,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19933.8, 300 sec: 19494.2). Total num frames: 264364032. Throughput: 0: 9836.5, 1: 10002.7. Samples: 264357104. Policy #0 lag: (min: 4.0, avg: 14.0, max: 36.0) [2023-12-26 19:07:21,063][104569] Avg episode reward: [(0, '9077.043'), (1, '8283.332')] [2023-12-26 19:07:21,103][105620] Updated weights for policy 1, policy_version 516570 (0.0006) [2023-12-26 19:07:21,161][105620] Updated weights for policy 1, policy_version 516580 (0.0006) [2023-12-26 19:07:21,647][105692] Updated weights for policy 0, policy_version 516010 (0.0009) [2023-12-26 19:07:21,715][105692] Updated weights for policy 0, policy_version 516020 (0.0009) [2023-12-26 19:07:21,786][105692] Updated weights for policy 0, policy_version 516030 (0.0008) [2023-12-26 19:07:21,856][105620] Updated weights for policy 1, policy_version 516590 (0.0007) [2023-12-26 19:07:21,919][105620] Updated weights for policy 1, policy_version 516600 (0.0008) [2023-12-26 19:07:21,980][105620] Updated weights for policy 1, policy_version 516610 (0.0008) [2023-12-26 19:07:22,510][105692] Updated weights for policy 0, policy_version 516040 (0.0006) [2023-12-26 19:07:22,579][105692] Updated weights for policy 0, policy_version 516050 (0.0006) [2023-12-26 19:07:22,647][105692] Updated weights for policy 0, policy_version 516060 (0.0009) [2023-12-26 19:07:22,744][105620] Updated weights for policy 1, policy_version 516620 (0.0008) [2023-12-26 19:07:22,807][105620] Updated weights for policy 1, policy_version 516630 (0.0009) [2023-12-26 19:07:22,864][105620] Updated weights for policy 1, policy_version 516640 (0.0008) [2023-12-26 19:07:23,410][105692] Updated weights for policy 0, policy_version 516070 (0.0009) [2023-12-26 19:07:23,463][105692] Updated weights for policy 0, policy_version 516080 (0.0010) [2023-12-26 19:07:23,465][105620] Updated weights for policy 1, policy_version 516650 (0.0007) [2023-12-26 19:07:23,512][105692] Updated weights for policy 0, policy_version 516090 (0.0008) [2023-12-26 19:07:23,521][105620] Updated weights for policy 1, policy_version 516660 (0.0009) [2023-12-26 19:07:23,579][105620] Updated weights for policy 1, policy_version 516670 (0.0005) [2023-12-26 19:07:23,637][105620] Updated weights for policy 1, policy_version 516680 (0.0005) [2023-12-26 19:07:24,219][105620] Updated weights for policy 1, policy_version 516690 (0.0005) [2023-12-26 19:07:24,290][105620] Updated weights for policy 1, policy_version 516700 (0.0005) [2023-12-26 19:07:24,358][105620] Updated weights for policy 1, policy_version 516710 (0.0006) [2023-12-26 19:07:24,381][105692] Updated weights for policy 0, policy_version 516100 (0.0009) [2023-12-26 19:07:24,432][105692] Updated weights for policy 0, policy_version 516110 (0.0009) [2023-12-26 19:07:24,488][105692] Updated weights for policy 0, policy_version 516120 (0.0009) [2023-12-26 19:07:24,961][105620] Updated weights for policy 1, policy_version 516720 (0.0005) [2023-12-26 19:07:25,026][105620] Updated weights for policy 1, policy_version 516730 (0.0008) [2023-12-26 19:07:25,085][105620] Updated weights for policy 1, policy_version 516740 (0.0007) [2023-12-26 19:07:25,339][105692] Updated weights for policy 0, policy_version 516130 (0.0010) [2023-12-26 19:07:25,394][105692] Updated weights for policy 0, policy_version 516141 (0.0010) [2023-12-26 19:07:25,448][105692] Updated weights for policy 0, policy_version 516153 (0.0011) [2023-12-26 19:07:25,595][105620] Updated weights for policy 1, policy_version 516750 (0.0006) [2023-12-26 19:07:25,641][105620] Updated weights for policy 1, policy_version 516760 (0.0008) [2023-12-26 19:07:25,690][105620] Updated weights for policy 1, policy_version 516770 (0.0008) [2023-12-26 19:07:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19933.9, 300 sec: 19494.2). Total num frames: 264462336. Throughput: 0: 9664.8, 1: 10048.2. Samples: 264470900. Policy #0 lag: (min: 4.0, avg: 14.0, max: 36.0) [2023-12-26 19:07:26,063][104569] Avg episode reward: [(0, '9077.605'), (1, '8916.690')] [2023-12-26 19:07:26,172][105692] Updated weights for policy 0, policy_version 516163 (0.0010) [2023-12-26 19:07:26,223][105692] Updated weights for policy 0, policy_version 516173 (0.0009) [2023-12-26 19:07:26,251][105585] KL-divergence is very high: 130.8981 [2023-12-26 19:07:26,277][105692] Updated weights for policy 0, policy_version 516183 (0.0009) [2023-12-26 19:07:26,301][105585] KL-divergence is very high: 135.2161 [2023-12-26 19:07:26,484][105620] Updated weights for policy 1, policy_version 516780 (0.0009) [2023-12-26 19:07:26,535][105620] Updated weights for policy 1, policy_version 516790 (0.0008) [2023-12-26 19:07:26,581][105620] Updated weights for policy 1, policy_version 516800 (0.0008) [2023-12-26 19:07:27,043][105692] Updated weights for policy 0, policy_version 516193 (0.0009) [2023-12-26 19:07:27,101][105692] Updated weights for policy 0, policy_version 516204 (0.0010) [2023-12-26 19:07:27,153][105692] Updated weights for policy 0, policy_version 516214 (0.0009) [2023-12-26 19:07:27,207][105692] Updated weights for policy 0, policy_version 516224 (0.0009) [2023-12-26 19:07:27,264][105620] Updated weights for policy 1, policy_version 516810 (0.0009) [2023-12-26 19:07:27,314][105620] Updated weights for policy 1, policy_version 516820 (0.0009) [2023-12-26 19:07:27,366][105620] Updated weights for policy 1, policy_version 516830 (0.0008) [2023-12-26 19:07:27,816][105585] KL-divergence is very high: 220.4646 [2023-12-26 19:07:27,821][105692] Updated weights for policy 0, policy_version 516234 (0.0005) [2023-12-26 19:07:27,860][105585] KL-divergence is very high: 361.8632 [2023-12-26 19:07:27,874][105692] Updated weights for policy 0, policy_version 516244 (0.0005) [2023-12-26 19:07:27,897][105585] KL-divergence is very high: 400.2956 [2023-12-26 19:07:27,919][105692] Updated weights for policy 0, policy_version 516254 (0.0005) [2023-12-26 19:07:28,114][105620] Updated weights for policy 1, policy_version 516841 (0.0011) [2023-12-26 19:07:28,166][105620] Updated weights for policy 1, policy_version 516851 (0.0008) [2023-12-26 19:07:28,219][105620] Updated weights for policy 1, policy_version 516861 (0.0009) [2023-12-26 19:07:28,264][105620] Updated weights for policy 1, policy_version 516871 (0.0008) [2023-12-26 19:07:28,533][105692] Updated weights for policy 0, policy_version 516264 (0.0009) [2023-12-26 19:07:28,589][105692] Updated weights for policy 0, policy_version 516274 (0.0008) [2023-12-26 19:07:28,647][105692] Updated weights for policy 0, policy_version 516284 (0.0010) [2023-12-26 19:07:28,946][105620] Updated weights for policy 1, policy_version 516881 (0.0006) [2023-12-26 19:07:29,014][105620] Updated weights for policy 1, policy_version 516891 (0.0005) [2023-12-26 19:07:29,074][105620] Updated weights for policy 1, policy_version 516901 (0.0005) [2023-12-26 19:07:29,356][105692] Updated weights for policy 0, policy_version 516294 (0.0010) [2023-12-26 19:07:29,417][105692] Updated weights for policy 0, policy_version 516304 (0.0007) [2023-12-26 19:07:29,474][105692] Updated weights for policy 0, policy_version 516314 (0.0008) [2023-12-26 19:07:29,736][105620] Updated weights for policy 1, policy_version 516911 (0.0008) [2023-12-26 19:07:29,787][105620] Updated weights for policy 1, policy_version 516921 (0.0009) [2023-12-26 19:07:29,846][105620] Updated weights for policy 1, policy_version 516931 (0.0009) [2023-12-26 19:07:30,083][105692] Updated weights for policy 0, policy_version 516324 (0.0008) [2023-12-26 19:07:30,147][105692] Updated weights for policy 0, policy_version 516334 (0.0009) [2023-12-26 19:07:30,201][105692] Updated weights for policy 0, policy_version 516344 (0.0010) [2023-12-26 19:07:30,489][105620] Updated weights for policy 1, policy_version 516941 (0.0009) [2023-12-26 19:07:30,541][105620] Updated weights for policy 1, policy_version 516951 (0.0010) [2023-12-26 19:07:30,592][105620] Updated weights for policy 1, policy_version 516961 (0.0008) [2023-12-26 19:07:30,840][105692] Updated weights for policy 0, policy_version 516354 (0.0008) [2023-12-26 19:07:30,895][105692] Updated weights for policy 0, policy_version 516364 (0.0005) [2023-12-26 19:07:30,952][105692] Updated weights for policy 0, policy_version 516374 (0.0005) [2023-12-26 19:07:31,008][105692] Updated weights for policy 0, policy_version 516384 (0.0005) [2023-12-26 19:07:31,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19933.8, 300 sec: 19549.7). Total num frames: 264568832. Throughput: 0: 9687.6, 1: 10130.6. Samples: 264532928. Policy #0 lag: (min: 4.0, avg: 14.0, max: 36.0) [2023-12-26 19:07:31,063][104569] Avg episode reward: [(0, '9078.780'), (1, '9173.326')] [2023-12-26 19:07:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000516384_132210688.pth... [2023-12-26 19:07:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000516968_132358144.pth... [2023-12-26 19:07:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000515232_131915776.pth [2023-12-26 19:07:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000515784_132055040.pth [2023-12-26 19:07:31,402][105620] Updated weights for policy 1, policy_version 516971 (0.0008) [2023-12-26 19:07:31,457][105620] Updated weights for policy 1, policy_version 516981 (0.0008) [2023-12-26 19:07:31,502][105620] Updated weights for policy 1, policy_version 516991 (0.0008) [2023-12-26 19:07:31,726][105692] Updated weights for policy 0, policy_version 516394 (0.0006) [2023-12-26 19:07:31,783][105692] Updated weights for policy 0, policy_version 516404 (0.0010) [2023-12-26 19:07:31,838][105692] Updated weights for policy 0, policy_version 516414 (0.0010) [2023-12-26 19:07:32,170][105620] Updated weights for policy 1, policy_version 517001 (0.0008) [2023-12-26 19:07:32,231][105620] Updated weights for policy 1, policy_version 517011 (0.0005) [2023-12-26 19:07:32,291][105620] Updated weights for policy 1, policy_version 517021 (0.0007) [2023-12-26 19:07:32,355][105620] Updated weights for policy 1, policy_version 517031 (0.0007) [2023-12-26 19:07:32,578][105692] Updated weights for policy 0, policy_version 516424 (0.0010) [2023-12-26 19:07:32,629][105692] Updated weights for policy 0, policy_version 516434 (0.0010) [2023-12-26 19:07:32,681][105692] Updated weights for policy 0, policy_version 516444 (0.0010) [2023-12-26 19:07:32,927][105620] Updated weights for policy 1, policy_version 517041 (0.0010) [2023-12-26 19:07:32,978][105620] Updated weights for policy 1, policy_version 517051 (0.0009) [2023-12-26 19:07:33,026][105620] Updated weights for policy 1, policy_version 517061 (0.0009) [2023-12-26 19:07:33,351][105692] Updated weights for policy 0, policy_version 516454 (0.0010) [2023-12-26 19:07:33,402][105692] Updated weights for policy 0, policy_version 516464 (0.0006) [2023-12-26 19:07:33,454][105692] Updated weights for policy 0, policy_version 516474 (0.0006) [2023-12-26 19:07:33,699][105620] Updated weights for policy 1, policy_version 517071 (0.0008) [2023-12-26 19:07:33,747][105620] Updated weights for policy 1, policy_version 517081 (0.0009) [2023-12-26 19:07:33,808][105620] Updated weights for policy 1, policy_version 517091 (0.0009) [2023-12-26 19:07:34,146][105692] Updated weights for policy 0, policy_version 516484 (0.0006) [2023-12-26 19:07:34,208][105692] Updated weights for policy 0, policy_version 516494 (0.0008) [2023-12-26 19:07:34,270][105692] Updated weights for policy 0, policy_version 516504 (0.0009) [2023-12-26 19:07:34,582][105620] Updated weights for policy 1, policy_version 517101 (0.0008) [2023-12-26 19:07:34,644][105620] Updated weights for policy 1, policy_version 517111 (0.0009) [2023-12-26 19:07:34,699][105620] Updated weights for policy 1, policy_version 517121 (0.0009) [2023-12-26 19:07:34,948][105692] Updated weights for policy 0, policy_version 516514 (0.0009) [2023-12-26 19:07:35,008][105692] Updated weights for policy 0, policy_version 516524 (0.0009) [2023-12-26 19:07:35,070][105692] Updated weights for policy 0, policy_version 516534 (0.0009) [2023-12-26 19:07:35,121][105692] Updated weights for policy 0, policy_version 516544 (0.0009) [2023-12-26 19:07:35,457][105620] Updated weights for policy 1, policy_version 517131 (0.0009) [2023-12-26 19:07:35,509][105620] Updated weights for policy 1, policy_version 517141 (0.0009) [2023-12-26 19:07:35,562][105620] Updated weights for policy 1, policy_version 517151 (0.0005) [2023-12-26 19:07:35,878][105692] Updated weights for policy 0, policy_version 516554 (0.0005) [2023-12-26 19:07:35,937][105692] Updated weights for policy 0, policy_version 516564 (0.0005) [2023-12-26 19:07:35,996][105692] Updated weights for policy 0, policy_version 516574 (0.0007) [2023-12-26 19:07:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 264667136. Throughput: 0: 9702.4, 1: 10054.9. Samples: 264654168. Policy #0 lag: (min: 4.0, avg: 14.0, max: 36.0) [2023-12-26 19:07:36,062][104569] Avg episode reward: [(0, '9078.938'), (1, '9081.193')] [2023-12-26 19:07:36,353][105620] Updated weights for policy 1, policy_version 517161 (0.0006) [2023-12-26 19:07:36,412][105620] Updated weights for policy 1, policy_version 517171 (0.0009) [2023-12-26 19:07:36,472][105620] Updated weights for policy 1, policy_version 517181 (0.0009) [2023-12-26 19:07:36,537][105620] Updated weights for policy 1, policy_version 517191 (0.0009) [2023-12-26 19:07:36,625][105692] Updated weights for policy 0, policy_version 516584 (0.0009) [2023-12-26 19:07:36,677][105692] Updated weights for policy 0, policy_version 516594 (0.0009) [2023-12-26 19:07:36,736][105692] Updated weights for policy 0, policy_version 516604 (0.0009) [2023-12-26 19:07:37,300][105620] Updated weights for policy 1, policy_version 517201 (0.0009) [2023-12-26 19:07:37,360][105620] Updated weights for policy 1, policy_version 517211 (0.0010) [2023-12-26 19:07:37,417][105620] Updated weights for policy 1, policy_version 517221 (0.0009) [2023-12-26 19:07:37,479][105692] Updated weights for policy 0, policy_version 516614 (0.0009) [2023-12-26 19:07:37,543][105692] Updated weights for policy 0, policy_version 516624 (0.0008) [2023-12-26 19:07:37,593][105692] Updated weights for policy 0, policy_version 516634 (0.0005) [2023-12-26 19:07:38,182][105620] Updated weights for policy 1, policy_version 517231 (0.0010) [2023-12-26 19:07:38,240][105620] Updated weights for policy 1, policy_version 517241 (0.0010) [2023-12-26 19:07:38,292][105620] Updated weights for policy 1, policy_version 517251 (0.0010) [2023-12-26 19:07:38,338][105692] Updated weights for policy 0, policy_version 516644 (0.0007) [2023-12-26 19:07:38,395][105692] Updated weights for policy 0, policy_version 516654 (0.0008) [2023-12-26 19:07:38,444][105692] Updated weights for policy 0, policy_version 516664 (0.0008) [2023-12-26 19:07:39,053][105620] Updated weights for policy 1, policy_version 517261 (0.0010) [2023-12-26 19:07:39,101][105620] Updated weights for policy 1, policy_version 517271 (0.0010) [2023-12-26 19:07:39,153][105620] Updated weights for policy 1, policy_version 517281 (0.0010) [2023-12-26 19:07:39,207][105692] Updated weights for policy 0, policy_version 516674 (0.0008) [2023-12-26 19:07:39,274][105692] Updated weights for policy 0, policy_version 516684 (0.0008) [2023-12-26 19:07:39,331][105692] Updated weights for policy 0, policy_version 516694 (0.0008) [2023-12-26 19:07:39,397][105692] Updated weights for policy 0, policy_version 516704 (0.0008) [2023-12-26 19:07:39,967][105620] Updated weights for policy 1, policy_version 517291 (0.0009) [2023-12-26 19:07:40,032][105620] Updated weights for policy 1, policy_version 517301 (0.0008) [2023-12-26 19:07:40,102][105620] Updated weights for policy 1, policy_version 517311 (0.0006) [2023-12-26 19:07:40,170][105692] Updated weights for policy 0, policy_version 516714 (0.0009) [2023-12-26 19:07:40,241][105692] Updated weights for policy 0, policy_version 516724 (0.0010) [2023-12-26 19:07:40,302][105692] Updated weights for policy 0, policy_version 516734 (0.0009) [2023-12-26 19:07:40,770][105620] Updated weights for policy 1, policy_version 517321 (0.0006) [2023-12-26 19:07:40,828][105620] Updated weights for policy 1, policy_version 517331 (0.0010) [2023-12-26 19:07:40,880][105620] Updated weights for policy 1, policy_version 517341 (0.0010) [2023-12-26 19:07:40,933][105620] Updated weights for policy 1, policy_version 517351 (0.0010) [2023-12-26 19:07:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 264757248. Throughput: 0: 9711.4, 1: 9908.3. Samples: 264766248. Policy #0 lag: (min: 4.0, avg: 14.0, max: 36.0) [2023-12-26 19:07:41,063][104569] Avg episode reward: [(0, '9355.083'), (1, '8989.679')] [2023-12-26 19:07:41,078][105692] Updated weights for policy 0, policy_version 516744 (0.0008) [2023-12-26 19:07:41,147][105692] Updated weights for policy 0, policy_version 516754 (0.0006) [2023-12-26 19:07:41,212][105692] Updated weights for policy 0, policy_version 516764 (0.0010) [2023-12-26 19:07:41,686][105620] Updated weights for policy 1, policy_version 517361 (0.0009) [2023-12-26 19:07:41,758][105620] Updated weights for policy 1, policy_version 517371 (0.0009) [2023-12-26 19:07:41,827][105620] Updated weights for policy 1, policy_version 517381 (0.0009) [2023-12-26 19:07:42,015][105692] Updated weights for policy 0, policy_version 516774 (0.0010) [2023-12-26 19:07:42,083][105692] Updated weights for policy 0, policy_version 516784 (0.0009) [2023-12-26 19:07:42,152][105692] Updated weights for policy 0, policy_version 516794 (0.0010) [2023-12-26 19:07:42,513][105620] Updated weights for policy 1, policy_version 517391 (0.0008) [2023-12-26 19:07:42,571][105620] Updated weights for policy 1, policy_version 517401 (0.0010) [2023-12-26 19:07:42,629][105620] Updated weights for policy 1, policy_version 517411 (0.0009) [2023-12-26 19:07:42,799][105692] Updated weights for policy 0, policy_version 516804 (0.0007) [2023-12-26 19:07:42,850][105692] Updated weights for policy 0, policy_version 516814 (0.0005) [2023-12-26 19:07:42,902][105692] Updated weights for policy 0, policy_version 516824 (0.0005) [2023-12-26 19:07:43,451][105620] Updated weights for policy 1, policy_version 517421 (0.0010) [2023-12-26 19:07:43,461][105692] Updated weights for policy 0, policy_version 516834 (0.0006) [2023-12-26 19:07:43,507][105620] Updated weights for policy 1, policy_version 517431 (0.0006) [2023-12-26 19:07:43,510][105692] Updated weights for policy 0, policy_version 516844 (0.0007) [2023-12-26 19:07:43,558][105692] Updated weights for policy 0, policy_version 516854 (0.0007) [2023-12-26 19:07:43,565][105620] Updated weights for policy 1, policy_version 517441 (0.0007) [2023-12-26 19:07:43,607][105692] Updated weights for policy 0, policy_version 516864 (0.0005) [2023-12-26 19:07:44,325][105692] Updated weights for policy 0, policy_version 516874 (0.0008) [2023-12-26 19:07:44,327][105620] Updated weights for policy 1, policy_version 517451 (0.0009) [2023-12-26 19:07:44,362][105585] KL-divergence is very high: 189.1535 [2023-12-26 19:07:44,380][105692] Updated weights for policy 0, policy_version 516884 (0.0006) [2023-12-26 19:07:44,380][105620] Updated weights for policy 1, policy_version 517461 (0.0007) [2023-12-26 19:07:44,409][105585] KL-divergence is very high: 180.5792 [2023-12-26 19:07:44,435][105692] Updated weights for policy 0, policy_version 516894 (0.0008) [2023-12-26 19:07:44,437][105620] Updated weights for policy 1, policy_version 517471 (0.0006) [2023-12-26 19:07:45,124][105620] Updated weights for policy 1, policy_version 517481 (0.0009) [2023-12-26 19:07:45,155][105692] Updated weights for policy 0, policy_version 516904 (0.0007) [2023-12-26 19:07:45,183][105620] Updated weights for policy 1, policy_version 517491 (0.0006) [2023-12-26 19:07:45,212][105692] Updated weights for policy 0, policy_version 516914 (0.0007) [2023-12-26 19:07:45,249][105620] Updated weights for policy 1, policy_version 517501 (0.0008) [2023-12-26 19:07:45,269][105692] Updated weights for policy 0, policy_version 516924 (0.0006) [2023-12-26 19:07:45,319][105620] Updated weights for policy 1, policy_version 517511 (0.0008) [2023-12-26 19:07:45,889][105692] Updated weights for policy 0, policy_version 516934 (0.0006) [2023-12-26 19:07:45,954][105692] Updated weights for policy 0, policy_version 516944 (0.0005) [2023-12-26 19:07:46,011][105692] Updated weights for policy 0, policy_version 516954 (0.0009) [2023-12-26 19:07:46,037][105620] Updated weights for policy 1, policy_version 517521 (0.0007) [2023-12-26 19:07:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 264855552. Throughput: 0: 9698.4, 1: 9864.6. Samples: 264823836. Policy #0 lag: (min: 4.0, avg: 14.0, max: 36.0) [2023-12-26 19:07:46,063][104569] Avg episode reward: [(0, '9084.735'), (1, '8990.687')] [2023-12-26 19:07:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000516960_132358144.pth... [2023-12-26 19:07:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000515808_132063232.pth [2023-12-26 19:07:46,101][105620] Updated weights for policy 1, policy_version 517531 (0.0007) [2023-12-26 19:07:46,157][105620] Updated weights for policy 1, policy_version 517541 (0.0010) [2023-12-26 19:07:46,172][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000517544_132505600.pth... [2023-12-26 19:07:46,175][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000516392_132210688.pth [2023-12-26 19:07:46,639][105692] Updated weights for policy 0, policy_version 516964 (0.0006) [2023-12-26 19:07:46,705][105692] Updated weights for policy 0, policy_version 516974 (0.0005) [2023-12-26 19:07:46,755][105692] Updated weights for policy 0, policy_version 516984 (0.0005) [2023-12-26 19:07:46,925][105620] Updated weights for policy 1, policy_version 517551 (0.0010) [2023-12-26 19:07:46,975][105620] Updated weights for policy 1, policy_version 517561 (0.0010) [2023-12-26 19:07:47,028][105620] Updated weights for policy 1, policy_version 517571 (0.0008) [2023-12-26 19:07:47,295][105692] Updated weights for policy 0, policy_version 516994 (0.0006) [2023-12-26 19:07:47,349][105692] Updated weights for policy 0, policy_version 517004 (0.0010) [2023-12-26 19:07:47,410][105692] Updated weights for policy 0, policy_version 517014 (0.0010) [2023-12-26 19:07:47,472][105692] Updated weights for policy 0, policy_version 517024 (0.0010) [2023-12-26 19:07:47,589][105620] Updated weights for policy 1, policy_version 517581 (0.0006) [2023-12-26 19:07:47,633][105620] Updated weights for policy 1, policy_version 517591 (0.0010) [2023-12-26 19:07:47,677][105620] Updated weights for policy 1, policy_version 517601 (0.0010) [2023-12-26 19:07:48,153][105692] Updated weights for policy 0, policy_version 517034 (0.0006) [2023-12-26 19:07:48,198][105692] Updated weights for policy 0, policy_version 517044 (0.0005) [2023-12-26 19:07:48,244][105692] Updated weights for policy 0, policy_version 517054 (0.0005) [2023-12-26 19:07:48,348][105620] Updated weights for policy 1, policy_version 517611 (0.0010) [2023-12-26 19:07:48,404][105620] Updated weights for policy 1, policy_version 517621 (0.0008) [2023-12-26 19:07:48,460][105620] Updated weights for policy 1, policy_version 517631 (0.0008) [2023-12-26 19:07:48,961][105692] Updated weights for policy 0, policy_version 517064 (0.0008) [2023-12-26 19:07:49,012][105692] Updated weights for policy 0, policy_version 517074 (0.0009) [2023-12-26 19:07:49,058][105692] Updated weights for policy 0, policy_version 517084 (0.0005) [2023-12-26 19:07:49,064][105620] Updated weights for policy 1, policy_version 517641 (0.0008) [2023-12-26 19:07:49,123][105620] Updated weights for policy 1, policy_version 517651 (0.0009) [2023-12-26 19:07:49,178][105620] Updated weights for policy 1, policy_version 517661 (0.0010) [2023-12-26 19:07:49,248][105620] Updated weights for policy 1, policy_version 517671 (0.0011) [2023-12-26 19:07:49,812][105692] Updated weights for policy 0, policy_version 517094 (0.0007) [2023-12-26 19:07:49,879][105692] Updated weights for policy 0, policy_version 517104 (0.0009) [2023-12-26 19:07:49,946][105692] Updated weights for policy 0, policy_version 517114 (0.0008) [2023-12-26 19:07:49,994][105620] Updated weights for policy 1, policy_version 517681 (0.0008) [2023-12-26 19:07:50,056][105620] Updated weights for policy 1, policy_version 517691 (0.0008) [2023-12-26 19:07:50,122][105620] Updated weights for policy 1, policy_version 517701 (0.0010) [2023-12-26 19:07:50,713][105692] Updated weights for policy 0, policy_version 517124 (0.0007) [2023-12-26 19:07:50,765][105692] Updated weights for policy 0, policy_version 517134 (0.0010) [2023-12-26 19:07:50,816][105620] Updated weights for policy 1, policy_version 517711 (0.0007) [2023-12-26 19:07:50,830][105692] Updated weights for policy 0, policy_version 517144 (0.0009) [2023-12-26 19:07:50,868][105620] Updated weights for policy 1, policy_version 517721 (0.0005) [2023-12-26 19:07:50,930][105620] Updated weights for policy 1, policy_version 517731 (0.0005) [2023-12-26 19:07:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 264962048. Throughput: 0: 9764.5, 1: 9894.1. Samples: 264947348. Policy #0 lag: (min: 4.0, avg: 14.0, max: 36.0) [2023-12-26 19:07:51,062][104569] Avg episode reward: [(0, '8905.352'), (1, '9173.652')] [2023-12-26 19:07:51,636][105692] Updated weights for policy 0, policy_version 517155 (0.0009) [2023-12-26 19:07:51,645][105620] Updated weights for policy 1, policy_version 517741 (0.0007) [2023-12-26 19:07:51,701][105692] Updated weights for policy 0, policy_version 517165 (0.0006) [2023-12-26 19:07:51,714][105620] Updated weights for policy 1, policy_version 517751 (0.0009) [2023-12-26 19:07:51,764][105692] Updated weights for policy 0, policy_version 517175 (0.0007) [2023-12-26 19:07:51,781][105620] Updated weights for policy 1, policy_version 517761 (0.0008) [2023-12-26 19:07:52,485][105620] Updated weights for policy 1, policy_version 517771 (0.0009) [2023-12-26 19:07:52,531][105692] Updated weights for policy 0, policy_version 517185 (0.0007) [2023-12-26 19:07:52,548][105620] Updated weights for policy 1, policy_version 517781 (0.0008) [2023-12-26 19:07:52,591][105692] Updated weights for policy 0, policy_version 517195 (0.0011) [2023-12-26 19:07:52,606][105620] Updated weights for policy 1, policy_version 517791 (0.0006) [2023-12-26 19:07:52,651][105692] Updated weights for policy 0, policy_version 517205 (0.0011) [2023-12-26 19:07:52,708][105692] Updated weights for policy 0, policy_version 517215 (0.0011) [2023-12-26 19:07:53,305][105620] Updated weights for policy 1, policy_version 517801 (0.0007) [2023-12-26 19:07:53,361][105620] Updated weights for policy 1, policy_version 517811 (0.0010) [2023-12-26 19:07:53,410][105620] Updated weights for policy 1, policy_version 517821 (0.0010) [2023-12-26 19:07:53,418][105692] Updated weights for policy 0, policy_version 517225 (0.0009) [2023-12-26 19:07:53,467][105692] Updated weights for policy 0, policy_version 517235 (0.0010) [2023-12-26 19:07:53,469][105620] Updated weights for policy 1, policy_version 517831 (0.0010) [2023-12-26 19:07:53,518][105692] Updated weights for policy 0, policy_version 517245 (0.0010) [2023-12-26 19:07:54,212][105692] Updated weights for policy 0, policy_version 517255 (0.0007) [2023-12-26 19:07:54,214][105620] Updated weights for policy 1, policy_version 517841 (0.0006) [2023-12-26 19:07:54,261][105692] Updated weights for policy 0, policy_version 517265 (0.0008) [2023-12-26 19:07:54,267][105620] Updated weights for policy 1, policy_version 517851 (0.0005) [2023-12-26 19:07:54,320][105692] Updated weights for policy 0, policy_version 517275 (0.0009) [2023-12-26 19:07:54,320][105620] Updated weights for policy 1, policy_version 517861 (0.0006) [2023-12-26 19:07:54,936][105620] Updated weights for policy 1, policy_version 517871 (0.0007) [2023-12-26 19:07:54,990][105620] Updated weights for policy 1, policy_version 517881 (0.0006) [2023-12-26 19:07:55,039][105620] Updated weights for policy 1, policy_version 517891 (0.0005) [2023-12-26 19:07:55,092][105692] Updated weights for policy 0, policy_version 517285 (0.0009) [2023-12-26 19:07:55,140][105692] Updated weights for policy 0, policy_version 517295 (0.0009) [2023-12-26 19:07:55,186][105692] Updated weights for policy 0, policy_version 517305 (0.0008) [2023-12-26 19:07:55,801][105620] Updated weights for policy 1, policy_version 517901 (0.0007) [2023-12-26 19:07:55,836][105692] Updated weights for policy 0, policy_version 517315 (0.0008) [2023-12-26 19:07:55,851][105620] Updated weights for policy 1, policy_version 517911 (0.0008) [2023-12-26 19:07:55,893][105692] Updated weights for policy 0, policy_version 517325 (0.0008) [2023-12-26 19:07:55,911][105620] Updated weights for policy 1, policy_version 517921 (0.0006) [2023-12-26 19:07:55,955][105692] Updated weights for policy 0, policy_version 517335 (0.0006) [2023-12-26 19:07:56,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 265060352. Throughput: 0: 9791.3, 1: 9845.9. Samples: 265062496. Policy #0 lag: (min: 4.0, avg: 14.0, max: 36.0) [2023-12-26 19:07:56,062][104569] Avg episode reward: [(0, '8905.684'), (1, '9354.531')] [2023-12-26 19:07:56,470][105620] Updated weights for policy 1, policy_version 517931 (0.0005) [2023-12-26 19:07:56,522][105620] Updated weights for policy 1, policy_version 517941 (0.0005) [2023-12-26 19:07:56,577][105620] Updated weights for policy 1, policy_version 517951 (0.0005) [2023-12-26 19:07:56,777][105692] Updated weights for policy 0, policy_version 517345 (0.0009) [2023-12-26 19:07:56,847][105692] Updated weights for policy 0, policy_version 517355 (0.0010) [2023-12-26 19:07:56,907][105692] Updated weights for policy 0, policy_version 517365 (0.0009) [2023-12-26 19:07:56,959][105692] Updated weights for policy 0, policy_version 517375 (0.0010) [2023-12-26 19:07:57,086][105620] Updated weights for policy 1, policy_version 517961 (0.0005) [2023-12-26 19:07:57,146][105620] Updated weights for policy 1, policy_version 517971 (0.0005) [2023-12-26 19:07:57,222][105620] Updated weights for policy 1, policy_version 517981 (0.0006) [2023-12-26 19:07:57,279][105620] Updated weights for policy 1, policy_version 517991 (0.0010) [2023-12-26 19:07:57,709][105692] Updated weights for policy 0, policy_version 517385 (0.0008) [2023-12-26 19:07:57,760][105692] Updated weights for policy 0, policy_version 517395 (0.0010) [2023-12-26 19:07:57,809][105692] Updated weights for policy 0, policy_version 517405 (0.0010) [2023-12-26 19:07:57,813][105620] Updated weights for policy 1, policy_version 518001 (0.0006) [2023-12-26 19:07:57,869][105620] Updated weights for policy 1, policy_version 518011 (0.0005) [2023-12-26 19:07:57,914][105620] Updated weights for policy 1, policy_version 518021 (0.0005) [2023-12-26 19:07:58,521][105692] Updated weights for policy 0, policy_version 517415 (0.0009) [2023-12-26 19:07:58,570][105620] Updated weights for policy 1, policy_version 518031 (0.0009) [2023-12-26 19:07:58,580][105692] Updated weights for policy 0, policy_version 517425 (0.0006) [2023-12-26 19:07:58,623][105620] Updated weights for policy 1, policy_version 518041 (0.0010) [2023-12-26 19:07:58,645][105692] Updated weights for policy 0, policy_version 517435 (0.0005) [2023-12-26 19:07:58,675][105620] Updated weights for policy 1, policy_version 518051 (0.0010) [2023-12-26 19:07:59,391][105620] Updated weights for policy 1, policy_version 518061 (0.0009) [2023-12-26 19:07:59,448][105620] Updated weights for policy 1, policy_version 518071 (0.0009) [2023-12-26 19:07:59,464][105692] Updated weights for policy 0, policy_version 517445 (0.0006) [2023-12-26 19:07:59,502][105620] Updated weights for policy 1, policy_version 518081 (0.0007) [2023-12-26 19:07:59,527][105692] Updated weights for policy 0, policy_version 517455 (0.0009) [2023-12-26 19:07:59,583][105692] Updated weights for policy 0, policy_version 517465 (0.0008) [2023-12-26 19:08:00,240][105620] Updated weights for policy 1, policy_version 518091 (0.0007) [2023-12-26 19:08:00,267][105692] Updated weights for policy 0, policy_version 517475 (0.0005) [2023-12-26 19:08:00,301][105620] Updated weights for policy 1, policy_version 518101 (0.0009) [2023-12-26 19:08:00,328][105692] Updated weights for policy 0, policy_version 517485 (0.0005) [2023-12-26 19:08:00,365][105620] Updated weights for policy 1, policy_version 518111 (0.0008) [2023-12-26 19:08:00,382][105692] Updated weights for policy 0, policy_version 517495 (0.0006) [2023-12-26 19:08:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 265150464. Throughput: 0: 9757.5, 1: 10000.3. Samples: 265125652. Policy #0 lag: (min: 4.0, avg: 14.0, max: 36.0) [2023-12-26 19:08:01,062][104569] Avg episode reward: [(0, '9081.483'), (1, '9265.515')] [2023-12-26 19:08:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000517504_132497408.pth... [2023-12-26 19:08:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000518120_132653056.pth... [2023-12-26 19:08:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000516968_132358144.pth [2023-12-26 19:08:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000516384_132210688.pth [2023-12-26 19:08:01,099][105692] Updated weights for policy 0, policy_version 517505 (0.0006) [2023-12-26 19:08:01,118][105620] Updated weights for policy 1, policy_version 518121 (0.0008) [2023-12-26 19:08:01,159][105692] Updated weights for policy 0, policy_version 517515 (0.0007) [2023-12-26 19:08:01,177][105620] Updated weights for policy 1, policy_version 518131 (0.0007) [2023-12-26 19:08:01,222][105620] Updated weights for policy 1, policy_version 518141 (0.0008) [2023-12-26 19:08:01,223][105692] Updated weights for policy 0, policy_version 517525 (0.0008) [2023-12-26 19:08:01,279][105620] Updated weights for policy 1, policy_version 518151 (0.0006) [2023-12-26 19:08:01,285][105692] Updated weights for policy 0, policy_version 517535 (0.0009) [2023-12-26 19:08:02,011][105692] Updated weights for policy 0, policy_version 517545 (0.0005) [2023-12-26 19:08:02,059][105620] Updated weights for policy 1, policy_version 518161 (0.0008) [2023-12-26 19:08:02,070][105692] Updated weights for policy 0, policy_version 517555 (0.0005) [2023-12-26 19:08:02,125][105620] Updated weights for policy 1, policy_version 518171 (0.0009) [2023-12-26 19:08:02,128][105692] Updated weights for policy 0, policy_version 517565 (0.0007) [2023-12-26 19:08:02,176][105620] Updated weights for policy 1, policy_version 518181 (0.0009) [2023-12-26 19:08:02,771][105692] Updated weights for policy 0, policy_version 517575 (0.0007) [2023-12-26 19:08:02,830][105692] Updated weights for policy 0, policy_version 517585 (0.0009) [2023-12-26 19:08:02,894][105692] Updated weights for policy 0, policy_version 517595 (0.0006) [2023-12-26 19:08:02,946][105620] Updated weights for policy 1, policy_version 518191 (0.0009) [2023-12-26 19:08:02,998][105620] Updated weights for policy 1, policy_version 518202 (0.0009) [2023-12-26 19:08:03,051][105620] Updated weights for policy 1, policy_version 518213 (0.0010) [2023-12-26 19:08:03,422][105692] Updated weights for policy 0, policy_version 517605 (0.0005) [2023-12-26 19:08:03,468][105692] Updated weights for policy 0, policy_version 517615 (0.0005) [2023-12-26 19:08:03,517][105692] Updated weights for policy 0, policy_version 517625 (0.0006) [2023-12-26 19:08:03,914][105620] Updated weights for policy 1, policy_version 518224 (0.0009) [2023-12-26 19:08:03,973][105620] Updated weights for policy 1, policy_version 518234 (0.0008) [2023-12-26 19:08:04,032][105620] Updated weights for policy 1, policy_version 518244 (0.0008) [2023-12-26 19:08:04,271][105692] Updated weights for policy 0, policy_version 517635 (0.0008) [2023-12-26 19:08:04,337][105692] Updated weights for policy 0, policy_version 517645 (0.0010) [2023-12-26 19:08:04,406][105692] Updated weights for policy 0, policy_version 517655 (0.0010) [2023-12-26 19:08:04,731][105620] Updated weights for policy 1, policy_version 518254 (0.0009) [2023-12-26 19:08:04,791][105620] Updated weights for policy 1, policy_version 518264 (0.0008) [2023-12-26 19:08:04,859][105620] Updated weights for policy 1, policy_version 518274 (0.0005) [2023-12-26 19:08:05,091][105692] Updated weights for policy 0, policy_version 517665 (0.0010) [2023-12-26 19:08:05,155][105692] Updated weights for policy 0, policy_version 517675 (0.0009) [2023-12-26 19:08:05,212][105692] Updated weights for policy 0, policy_version 517685 (0.0009) [2023-12-26 19:08:05,263][105692] Updated weights for policy 0, policy_version 517695 (0.0009) [2023-12-26 19:08:05,506][105620] Updated weights for policy 1, policy_version 518284 (0.0006) [2023-12-26 19:08:05,557][105620] Updated weights for policy 1, policy_version 518294 (0.0008) [2023-12-26 19:08:05,618][105620] Updated weights for policy 1, policy_version 518304 (0.0010) [2023-12-26 19:08:05,850][105692] Updated weights for policy 0, policy_version 517705 (0.0006) [2023-12-26 19:08:05,906][105692] Updated weights for policy 0, policy_version 517715 (0.0009) [2023-12-26 19:08:05,966][105692] Updated weights for policy 0, policy_version 517725 (0.0010) [2023-12-26 19:08:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 265256960. Throughput: 0: 9728.6, 1: 9923.4. Samples: 265241440. Policy #0 lag: (min: 4.0, avg: 14.0, max: 36.0) [2023-12-26 19:08:06,062][104569] Avg episode reward: [(0, '9174.640'), (1, '9265.662')] [2023-12-26 19:08:06,437][105620] Updated weights for policy 1, policy_version 518314 (0.0010) [2023-12-26 19:08:06,488][105620] Updated weights for policy 1, policy_version 518324 (0.0008) [2023-12-26 19:08:06,554][105620] Updated weights for policy 1, policy_version 518334 (0.0008) [2023-12-26 19:08:06,621][105620] Updated weights for policy 1, policy_version 518344 (0.0008) [2023-12-26 19:08:06,708][105692] Updated weights for policy 0, policy_version 517735 (0.0007) [2023-12-26 19:08:06,771][105692] Updated weights for policy 0, policy_version 517745 (0.0005) [2023-12-26 19:08:06,826][105692] Updated weights for policy 0, policy_version 517755 (0.0007) [2023-12-26 19:08:07,436][105620] Updated weights for policy 1, policy_version 518354 (0.0008) [2023-12-26 19:08:07,442][105692] Updated weights for policy 0, policy_version 517765 (0.0007) [2023-12-26 19:08:07,489][105620] Updated weights for policy 1, policy_version 518364 (0.0006) [2023-12-26 19:08:07,499][105692] Updated weights for policy 0, policy_version 517775 (0.0007) [2023-12-26 19:08:07,539][105620] Updated weights for policy 1, policy_version 518374 (0.0008) [2023-12-26 19:08:07,556][105692] Updated weights for policy 0, policy_version 517785 (0.0009) [2023-12-26 19:08:08,245][105692] Updated weights for policy 0, policy_version 517795 (0.0009) [2023-12-26 19:08:08,305][105692] Updated weights for policy 0, policy_version 517805 (0.0009) [2023-12-26 19:08:08,337][105620] Updated weights for policy 1, policy_version 518384 (0.0008) [2023-12-26 19:08:08,367][105692] Updated weights for policy 0, policy_version 517815 (0.0008) [2023-12-26 19:08:08,398][105620] Updated weights for policy 1, policy_version 518394 (0.0008) [2023-12-26 19:08:08,459][105620] Updated weights for policy 1, policy_version 518404 (0.0007) [2023-12-26 19:08:09,140][105692] Updated weights for policy 0, policy_version 517825 (0.0008) [2023-12-26 19:08:09,191][105620] Updated weights for policy 1, policy_version 518414 (0.0007) [2023-12-26 19:08:09,203][105692] Updated weights for policy 0, policy_version 517835 (0.0008) [2023-12-26 19:08:09,247][105620] Updated weights for policy 1, policy_version 518424 (0.0006) [2023-12-26 19:08:09,269][105692] Updated weights for policy 0, policy_version 517845 (0.0008) [2023-12-26 19:08:09,305][105620] Updated weights for policy 1, policy_version 518434 (0.0005) [2023-12-26 19:08:09,334][105692] Updated weights for policy 0, policy_version 517855 (0.0008) [2023-12-26 19:08:10,034][105620] Updated weights for policy 1, policy_version 518444 (0.0008) [2023-12-26 19:08:10,084][105620] Updated weights for policy 1, policy_version 518454 (0.0008) [2023-12-26 19:08:10,122][105692] Updated weights for policy 0, policy_version 517865 (0.0008) [2023-12-26 19:08:10,141][105620] Updated weights for policy 1, policy_version 518464 (0.0009) [2023-12-26 19:08:10,181][105692] Updated weights for policy 0, policy_version 517875 (0.0008) [2023-12-26 19:08:10,187][105585] KL-divergence is very high: 163.7623 [2023-12-26 19:08:10,239][105585] KL-divergence is very high: 165.3477 [2023-12-26 19:08:10,247][105692] Updated weights for policy 0, policy_version 517885 (0.0009) [2023-12-26 19:08:10,902][105620] Updated weights for policy 1, policy_version 518474 (0.0006) [2023-12-26 19:08:10,958][105620] Updated weights for policy 1, policy_version 518484 (0.0010) [2023-12-26 19:08:11,021][105620] Updated weights for policy 1, policy_version 518494 (0.0008) [2023-12-26 19:08:11,024][105692] Updated weights for policy 0, policy_version 517895 (0.0007) [2023-12-26 19:08:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 265338880. Throughput: 0: 9893.9, 1: 9768.7. Samples: 265355716. Policy #0 lag: (min: 4.0, avg: 14.0, max: 36.0) [2023-12-26 19:08:11,062][104569] Avg episode reward: [(0, '9263.506'), (1, '9354.907')] [2023-12-26 19:08:11,086][105620] Updated weights for policy 1, policy_version 518504 (0.0009) [2023-12-26 19:08:11,088][105692] Updated weights for policy 0, policy_version 517905 (0.0006) [2023-12-26 19:08:11,150][105692] Updated weights for policy 0, policy_version 517915 (0.0009) [2023-12-26 19:08:11,906][105692] Updated weights for policy 0, policy_version 517925 (0.0009) [2023-12-26 19:08:11,910][105620] Updated weights for policy 1, policy_version 518514 (0.0009) [2023-12-26 19:08:11,972][105620] Updated weights for policy 1, policy_version 518524 (0.0009) [2023-12-26 19:08:11,974][105692] Updated weights for policy 0, policy_version 517935 (0.0009) [2023-12-26 19:08:12,037][105620] Updated weights for policy 1, policy_version 518534 (0.0008) [2023-12-26 19:08:12,039][105692] Updated weights for policy 0, policy_version 517945 (0.0008) [2023-12-26 19:08:12,718][105692] Updated weights for policy 0, policy_version 517955 (0.0007) [2023-12-26 19:08:12,776][105692] Updated weights for policy 0, policy_version 517965 (0.0008) [2023-12-26 19:08:12,802][105620] Updated weights for policy 1, policy_version 518544 (0.0006) [2023-12-26 19:08:12,837][105692] Updated weights for policy 0, policy_version 517975 (0.0010) [2023-12-26 19:08:12,858][105620] Updated weights for policy 1, policy_version 518554 (0.0007) [2023-12-26 19:08:12,918][105620] Updated weights for policy 1, policy_version 518564 (0.0009) [2023-12-26 19:08:13,472][105692] Updated weights for policy 0, policy_version 517985 (0.0006) [2023-12-26 19:08:13,533][105692] Updated weights for policy 0, policy_version 517995 (0.0010) [2023-12-26 19:08:13,591][105692] Updated weights for policy 0, policy_version 518005 (0.0010) [2023-12-26 19:08:13,638][105620] Updated weights for policy 1, policy_version 518574 (0.0007) [2023-12-26 19:08:13,640][105692] Updated weights for policy 0, policy_version 518015 (0.0010) [2023-12-26 19:08:13,696][105620] Updated weights for policy 1, policy_version 518584 (0.0007) [2023-12-26 19:08:13,756][105620] Updated weights for policy 1, policy_version 518594 (0.0007) [2023-12-26 19:08:14,309][105620] Updated weights for policy 1, policy_version 518604 (0.0005) [2023-12-26 19:08:14,327][105692] Updated weights for policy 0, policy_version 518025 (0.0010) [2023-12-26 19:08:14,360][105620] Updated weights for policy 1, policy_version 518614 (0.0005) [2023-12-26 19:08:14,382][105692] Updated weights for policy 0, policy_version 518035 (0.0010) [2023-12-26 19:08:14,419][105620] Updated weights for policy 1, policy_version 518624 (0.0005) [2023-12-26 19:08:14,440][105692] Updated weights for policy 0, policy_version 518045 (0.0010) [2023-12-26 19:08:14,974][105620] Updated weights for policy 1, policy_version 518634 (0.0005) [2023-12-26 19:08:15,027][105620] Updated weights for policy 1, policy_version 518644 (0.0008) [2023-12-26 19:08:15,077][105620] Updated weights for policy 1, policy_version 518654 (0.0008) [2023-12-26 19:08:15,134][105620] Updated weights for policy 1, policy_version 518664 (0.0008) [2023-12-26 19:08:15,163][105692] Updated weights for policy 0, policy_version 518055 (0.0011) [2023-12-26 19:08:15,226][105692] Updated weights for policy 0, policy_version 518065 (0.0011) [2023-12-26 19:08:15,288][105692] Updated weights for policy 0, policy_version 518075 (0.0011) [2023-12-26 19:08:15,912][105620] Updated weights for policy 1, policy_version 518674 (0.0008) [2023-12-26 19:08:15,969][105620] Updated weights for policy 1, policy_version 518684 (0.0008) [2023-12-26 19:08:16,013][105692] Updated weights for policy 0, policy_version 518085 (0.0011) [2023-12-26 19:08:16,027][105620] Updated weights for policy 1, policy_version 518694 (0.0006) [2023-12-26 19:08:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 265445376. Throughput: 0: 9832.8, 1: 9713.2. Samples: 265412496. Policy #0 lag: (min: 4.0, avg: 14.0, max: 36.0) [2023-12-26 19:08:16,062][104569] Avg episode reward: [(0, '9078.021'), (1, '9354.883')] [2023-12-26 19:08:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000518696_132800512.pth... [2023-12-26 19:08:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000517544_132505600.pth [2023-12-26 19:08:16,072][105692] Updated weights for policy 0, policy_version 518095 (0.0010) [2023-12-26 19:08:16,128][105692] Updated weights for policy 0, policy_version 518105 (0.0010) [2023-12-26 19:08:16,171][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000518112_132653056.pth... [2023-12-26 19:08:16,175][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000516960_132358144.pth [2023-12-26 19:08:16,778][105620] Updated weights for policy 1, policy_version 518704 (0.0005) [2023-12-26 19:08:16,826][105620] Updated weights for policy 1, policy_version 518714 (0.0005) [2023-12-26 19:08:16,877][105620] Updated weights for policy 1, policy_version 518724 (0.0005) [2023-12-26 19:08:16,887][105692] Updated weights for policy 0, policy_version 518115 (0.0010) [2023-12-26 19:08:16,939][105692] Updated weights for policy 0, policy_version 518125 (0.0010) [2023-12-26 19:08:17,004][105692] Updated weights for policy 0, policy_version 518135 (0.0010) [2023-12-26 19:08:17,562][105620] Updated weights for policy 1, policy_version 518734 (0.0007) [2023-12-26 19:08:17,610][105620] Updated weights for policy 1, policy_version 518744 (0.0008) [2023-12-26 19:08:17,660][105620] Updated weights for policy 1, policy_version 518754 (0.0007) [2023-12-26 19:08:17,714][105692] Updated weights for policy 0, policy_version 518145 (0.0010) [2023-12-26 19:08:17,766][105692] Updated weights for policy 0, policy_version 518155 (0.0010) [2023-12-26 19:08:17,818][105692] Updated weights for policy 0, policy_version 518165 (0.0010) [2023-12-26 19:08:17,869][105692] Updated weights for policy 0, policy_version 518175 (0.0010) [2023-12-26 19:08:18,336][105620] Updated weights for policy 1, policy_version 518764 (0.0008) [2023-12-26 19:08:18,401][105620] Updated weights for policy 1, policy_version 518774 (0.0007) [2023-12-26 19:08:18,464][105620] Updated weights for policy 1, policy_version 518784 (0.0007) [2023-12-26 19:08:18,702][105692] Updated weights for policy 0, policy_version 518185 (0.0006) [2023-12-26 19:08:18,768][105692] Updated weights for policy 0, policy_version 518195 (0.0008) [2023-12-26 19:08:18,832][105692] Updated weights for policy 0, policy_version 518205 (0.0008) [2023-12-26 19:08:18,999][105620] Updated weights for policy 1, policy_version 518794 (0.0005) [2023-12-26 19:08:19,050][105620] Updated weights for policy 1, policy_version 518804 (0.0005) [2023-12-26 19:08:19,110][105620] Updated weights for policy 1, policy_version 518814 (0.0007) [2023-12-26 19:08:19,175][105620] Updated weights for policy 1, policy_version 518824 (0.0009) [2023-12-26 19:08:19,529][105692] Updated weights for policy 0, policy_version 518215 (0.0009) [2023-12-26 19:08:19,595][105692] Updated weights for policy 0, policy_version 518225 (0.0009) [2023-12-26 19:08:19,662][105692] Updated weights for policy 0, policy_version 518235 (0.0008) [2023-12-26 19:08:19,908][105620] Updated weights for policy 1, policy_version 518834 (0.0008) [2023-12-26 19:08:19,965][105620] Updated weights for policy 1, policy_version 518844 (0.0007) [2023-12-26 19:08:20,025][105620] Updated weights for policy 1, policy_version 518854 (0.0006) [2023-12-26 19:08:20,434][105692] Updated weights for policy 0, policy_version 518245 (0.0008) [2023-12-26 19:08:20,492][105692] Updated weights for policy 0, policy_version 518255 (0.0008) [2023-12-26 19:08:20,555][105692] Updated weights for policy 0, policy_version 518265 (0.0008) [2023-12-26 19:08:20,833][105620] Updated weights for policy 1, policy_version 518864 (0.0009) [2023-12-26 19:08:20,895][105620] Updated weights for policy 1, policy_version 518874 (0.0009) [2023-12-26 19:08:20,943][105620] Updated weights for policy 1, policy_version 518884 (0.0008) [2023-12-26 19:08:21,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 265543680. Throughput: 0: 9739.3, 1: 9767.6. Samples: 265531980. Policy #0 lag: (min: 4.0, avg: 14.0, max: 36.0) [2023-12-26 19:08:21,062][104569] Avg episode reward: [(0, '9078.259'), (1, '9263.078')] [2023-12-26 19:08:21,356][105692] Updated weights for policy 0, policy_version 518275 (0.0009) [2023-12-26 19:08:21,428][105692] Updated weights for policy 0, policy_version 518285 (0.0010) [2023-12-26 19:08:21,487][105692] Updated weights for policy 0, policy_version 518295 (0.0009) [2023-12-26 19:08:21,667][105620] Updated weights for policy 1, policy_version 518894 (0.0008) [2023-12-26 19:08:21,735][105620] Updated weights for policy 1, policy_version 518904 (0.0008) [2023-12-26 19:08:21,797][105620] Updated weights for policy 1, policy_version 518914 (0.0009) [2023-12-26 19:08:22,199][105692] Updated weights for policy 0, policy_version 518305 (0.0008) [2023-12-26 19:08:22,264][105692] Updated weights for policy 0, policy_version 518315 (0.0008) [2023-12-26 19:08:22,326][105692] Updated weights for policy 0, policy_version 518325 (0.0009) [2023-12-26 19:08:22,393][105692] Updated weights for policy 0, policy_version 518335 (0.0007) [2023-12-26 19:08:22,638][105620] Updated weights for policy 1, policy_version 518924 (0.0010) [2023-12-26 19:08:22,704][105620] Updated weights for policy 1, policy_version 518934 (0.0009) [2023-12-26 19:08:22,772][105620] Updated weights for policy 1, policy_version 518944 (0.0009) [2023-12-26 19:08:23,119][105692] Updated weights for policy 0, policy_version 518345 (0.0006) [2023-12-26 19:08:23,186][105692] Updated weights for policy 0, policy_version 518355 (0.0006) [2023-12-26 19:08:23,254][105692] Updated weights for policy 0, policy_version 518365 (0.0006) [2023-12-26 19:08:23,551][105620] Updated weights for policy 1, policy_version 518954 (0.0010) [2023-12-26 19:08:23,608][105620] Updated weights for policy 1, policy_version 518964 (0.0007) [2023-12-26 19:08:23,668][105620] Updated weights for policy 1, policy_version 518974 (0.0005) [2023-12-26 19:08:23,736][105620] Updated weights for policy 1, policy_version 518984 (0.0005) [2023-12-26 19:08:23,959][105692] Updated weights for policy 0, policy_version 518375 (0.0009) [2023-12-26 19:08:24,007][105692] Updated weights for policy 0, policy_version 518385 (0.0010) [2023-12-26 19:08:24,055][105692] Updated weights for policy 0, policy_version 518395 (0.0010) [2023-12-26 19:08:24,269][105620] Updated weights for policy 1, policy_version 518994 (0.0006) [2023-12-26 19:08:24,335][105620] Updated weights for policy 1, policy_version 519004 (0.0007) [2023-12-26 19:08:24,383][105620] Updated weights for policy 1, policy_version 519014 (0.0006) [2023-12-26 19:08:24,825][105692] Updated weights for policy 0, policy_version 518405 (0.0010) [2023-12-26 19:08:24,890][105692] Updated weights for policy 0, policy_version 518415 (0.0010) [2023-12-26 19:08:24,954][105692] Updated weights for policy 0, policy_version 518425 (0.0010) [2023-12-26 19:08:25,065][105620] Updated weights for policy 1, policy_version 519024 (0.0007) [2023-12-26 19:08:25,116][105620] Updated weights for policy 1, policy_version 519034 (0.0008) [2023-12-26 19:08:25,170][105620] Updated weights for policy 1, policy_version 519044 (0.0008) [2023-12-26 19:08:25,625][105692] Updated weights for policy 0, policy_version 518435 (0.0010) [2023-12-26 19:08:25,663][105585] KL-divergence is very high: 121.5700 [2023-12-26 19:08:25,688][105692] Updated weights for policy 0, policy_version 518445 (0.0009) [2023-12-26 19:08:25,712][105585] KL-divergence is very high: 160.4353 [2023-12-26 19:08:25,750][105692] Updated weights for policy 0, policy_version 518455 (0.0011) [2023-12-26 19:08:25,952][105620] Updated weights for policy 1, policy_version 519054 (0.0007) [2023-12-26 19:08:26,004][105620] Updated weights for policy 1, policy_version 519064 (0.0008) [2023-12-26 19:08:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 265633792. Throughput: 0: 9738.3, 1: 9799.4. Samples: 265645444. Policy #0 lag: (min: 4.0, avg: 14.0, max: 36.0) [2023-12-26 19:08:26,062][104569] Avg episode reward: [(0, '9264.807'), (1, '9263.066')] [2023-12-26 19:08:26,072][105620] Updated weights for policy 1, policy_version 519074 (0.0005) [2023-12-26 19:08:26,461][105692] Updated weights for policy 0, policy_version 518465 (0.0010) [2023-12-26 19:08:26,524][105692] Updated weights for policy 0, policy_version 518475 (0.0011) [2023-12-26 19:08:26,572][105692] Updated weights for policy 0, policy_version 518485 (0.0010) [2023-12-26 19:08:26,623][105692] Updated weights for policy 0, policy_version 518495 (0.0010) [2023-12-26 19:08:26,697][105620] Updated weights for policy 1, policy_version 519084 (0.0007) [2023-12-26 19:08:26,757][105620] Updated weights for policy 1, policy_version 519094 (0.0007) [2023-12-26 19:08:26,821][105620] Updated weights for policy 1, policy_version 519104 (0.0008) [2023-12-26 19:08:27,297][105692] Updated weights for policy 0, policy_version 518505 (0.0009) [2023-12-26 19:08:27,354][105692] Updated weights for policy 0, policy_version 518515 (0.0005) [2023-12-26 19:08:27,417][105692] Updated weights for policy 0, policy_version 518525 (0.0008) [2023-12-26 19:08:27,596][105620] Updated weights for policy 1, policy_version 519114 (0.0008) [2023-12-26 19:08:27,641][105620] Updated weights for policy 1, policy_version 519124 (0.0008) [2023-12-26 19:08:27,691][105620] Updated weights for policy 1, policy_version 519134 (0.0008) [2023-12-26 19:08:27,746][105620] Updated weights for policy 1, policy_version 519144 (0.0010) [2023-12-26 19:08:28,113][105692] Updated weights for policy 0, policy_version 518535 (0.0008) [2023-12-26 19:08:28,169][105692] Updated weights for policy 0, policy_version 518545 (0.0009) [2023-12-26 19:08:28,223][105692] Updated weights for policy 0, policy_version 518555 (0.0010) [2023-12-26 19:08:28,540][105620] Updated weights for policy 1, policy_version 519154 (0.0010) [2023-12-26 19:08:28,592][105620] Updated weights for policy 1, policy_version 519164 (0.0009) [2023-12-26 19:08:28,649][105620] Updated weights for policy 1, policy_version 519174 (0.0008) [2023-12-26 19:08:28,890][105692] Updated weights for policy 0, policy_version 518565 (0.0011) [2023-12-26 19:08:28,948][105692] Updated weights for policy 0, policy_version 518575 (0.0010) [2023-12-26 19:08:29,007][105692] Updated weights for policy 0, policy_version 518585 (0.0010) [2023-12-26 19:08:29,431][105620] Updated weights for policy 1, policy_version 519184 (0.0009) [2023-12-26 19:08:29,487][105620] Updated weights for policy 1, policy_version 519194 (0.0008) [2023-12-26 19:08:29,538][105620] Updated weights for policy 1, policy_version 519204 (0.0009) [2023-12-26 19:08:29,728][105692] Updated weights for policy 0, policy_version 518595 (0.0009) [2023-12-26 19:08:29,780][105692] Updated weights for policy 0, policy_version 518605 (0.0009) [2023-12-26 19:08:29,839][105692] Updated weights for policy 0, policy_version 518615 (0.0009) [2023-12-26 19:08:30,326][105620] Updated weights for policy 1, policy_version 519214 (0.0010) [2023-12-26 19:08:30,386][105620] Updated weights for policy 1, policy_version 519224 (0.0009) [2023-12-26 19:08:30,447][105620] Updated weights for policy 1, policy_version 519234 (0.0009) [2023-12-26 19:08:30,532][105692] Updated weights for policy 0, policy_version 518625 (0.0009) [2023-12-26 19:08:30,585][105692] Updated weights for policy 0, policy_version 518635 (0.0005) [2023-12-26 19:08:30,642][105692] Updated weights for policy 0, policy_version 518645 (0.0005) [2023-12-26 19:08:30,687][105692] Updated weights for policy 0, policy_version 518655 (0.0005) [2023-12-26 19:08:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19521.9). Total num frames: 265732096. Throughput: 0: 9742.9, 1: 9815.2. Samples: 265703952. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) [2023-12-26 19:08:31,062][104569] Avg episode reward: [(0, '9266.943'), (1, '9354.750')] [2023-12-26 19:08:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000519240_132939776.pth... [2023-12-26 19:08:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000518656_132792320.pth... [2023-12-26 19:08:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000518120_132653056.pth [2023-12-26 19:08:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000517504_132497408.pth [2023-12-26 19:08:31,267][105692] Updated weights for policy 0, policy_version 518665 (0.0007) [2023-12-26 19:08:31,284][105620] Updated weights for policy 1, policy_version 519244 (0.0008) [2023-12-26 19:08:31,326][105692] Updated weights for policy 0, policy_version 518675 (0.0008) [2023-12-26 19:08:31,346][105620] Updated weights for policy 1, policy_version 519254 (0.0006) [2023-12-26 19:08:31,393][105692] Updated weights for policy 0, policy_version 518685 (0.0007) [2023-12-26 19:08:31,415][105620] Updated weights for policy 1, policy_version 519264 (0.0008) [2023-12-26 19:08:32,095][105620] Updated weights for policy 1, policy_version 519274 (0.0009) [2023-12-26 19:08:32,162][105692] Updated weights for policy 0, policy_version 518695 (0.0006) [2023-12-26 19:08:32,164][105620] Updated weights for policy 1, policy_version 519284 (0.0007) [2023-12-26 19:08:32,226][105692] Updated weights for policy 0, policy_version 518705 (0.0005) [2023-12-26 19:08:32,231][105620] Updated weights for policy 1, policy_version 519294 (0.0008) [2023-12-26 19:08:32,286][105692] Updated weights for policy 0, policy_version 518715 (0.0009) [2023-12-26 19:08:32,288][105620] Updated weights for policy 1, policy_version 519304 (0.0007) [2023-12-26 19:08:32,962][105620] Updated weights for policy 1, policy_version 519314 (0.0006) [2023-12-26 19:08:33,003][105692] Updated weights for policy 0, policy_version 518725 (0.0008) [2023-12-26 19:08:33,016][105620] Updated weights for policy 1, policy_version 519324 (0.0005) [2023-12-26 19:08:33,060][105692] Updated weights for policy 0, policy_version 518735 (0.0008) [2023-12-26 19:08:33,070][105620] Updated weights for policy 1, policy_version 519334 (0.0007) [2023-12-26 19:08:33,119][105692] Updated weights for policy 0, policy_version 518745 (0.0008) [2023-12-26 19:08:33,602][105620] Updated weights for policy 1, policy_version 519344 (0.0006) [2023-12-26 19:08:33,654][105620] Updated weights for policy 1, policy_version 519354 (0.0005) [2023-12-26 19:08:33,709][105620] Updated weights for policy 1, policy_version 519364 (0.0005) [2023-12-26 19:08:34,032][105692] Updated weights for policy 0, policy_version 518755 (0.0009) [2023-12-26 19:08:34,090][105692] Updated weights for policy 0, policy_version 518765 (0.0010) [2023-12-26 19:08:34,160][105692] Updated weights for policy 0, policy_version 518775 (0.0010) [2023-12-26 19:08:34,220][105620] Updated weights for policy 1, policy_version 519374 (0.0008) [2023-12-26 19:08:34,268][105620] Updated weights for policy 1, policy_version 519384 (0.0010) [2023-12-26 19:08:34,320][105620] Updated weights for policy 1, policy_version 519394 (0.0010) [2023-12-26 19:08:34,915][105692] Updated weights for policy 0, policy_version 518785 (0.0007) [2023-12-26 19:08:34,977][105692] Updated weights for policy 0, policy_version 518795 (0.0005) [2023-12-26 19:08:35,040][105692] Updated weights for policy 0, policy_version 518805 (0.0006) [2023-12-26 19:08:35,090][105620] Updated weights for policy 1, policy_version 519404 (0.0010) [2023-12-26 19:08:35,098][105692] Updated weights for policy 0, policy_version 518815 (0.0006) [2023-12-26 19:08:35,133][105620] Updated weights for policy 1, policy_version 519414 (0.0010) [2023-12-26 19:08:35,187][105620] Updated weights for policy 1, policy_version 519424 (0.0010) [2023-12-26 19:08:35,668][105692] Updated weights for policy 0, policy_version 518825 (0.0006) [2023-12-26 19:08:35,736][105692] Updated weights for policy 0, policy_version 518835 (0.0005) [2023-12-26 19:08:35,801][105692] Updated weights for policy 0, policy_version 518845 (0.0005) [2023-12-26 19:08:35,807][105620] Updated weights for policy 1, policy_version 519434 (0.0009) [2023-12-26 19:08:35,868][105620] Updated weights for policy 1, policy_version 519444 (0.0007) [2023-12-26 19:08:35,921][105620] Updated weights for policy 1, policy_version 519454 (0.0009) [2023-12-26 19:08:35,978][105620] Updated weights for policy 1, policy_version 519464 (0.0009) [2023-12-26 19:08:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 265838592. Throughput: 0: 9614.6, 1: 9801.8. Samples: 265821084. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) [2023-12-26 19:08:36,063][104569] Avg episode reward: [(0, '9266.518'), (1, '9264.470')] [2023-12-26 19:08:36,380][105692] Updated weights for policy 0, policy_version 518855 (0.0009) [2023-12-26 19:08:36,444][105692] Updated weights for policy 0, policy_version 518865 (0.0010) [2023-12-26 19:08:36,509][105692] Updated weights for policy 0, policy_version 518875 (0.0009) [2023-12-26 19:08:36,734][105620] Updated weights for policy 1, policy_version 519474 (0.0008) [2023-12-26 19:08:36,786][105620] Updated weights for policy 1, policy_version 519484 (0.0007) [2023-12-26 19:08:36,837][105620] Updated weights for policy 1, policy_version 519494 (0.0009) [2023-12-26 19:08:37,213][105692] Updated weights for policy 0, policy_version 518885 (0.0010) [2023-12-26 19:08:37,268][105692] Updated weights for policy 0, policy_version 518895 (0.0006) [2023-12-26 19:08:37,316][105692] Updated weights for policy 0, policy_version 518905 (0.0009) [2023-12-26 19:08:37,607][105620] Updated weights for policy 1, policy_version 519504 (0.0009) [2023-12-26 19:08:37,653][105620] Updated weights for policy 1, policy_version 519514 (0.0008) [2023-12-26 19:08:37,712][105620] Updated weights for policy 1, policy_version 519524 (0.0009) [2023-12-26 19:08:38,030][105692] Updated weights for policy 0, policy_version 518915 (0.0008) [2023-12-26 19:08:38,089][105692] Updated weights for policy 0, policy_version 518925 (0.0005) [2023-12-26 19:08:38,155][105692] Updated weights for policy 0, policy_version 518935 (0.0007) [2023-12-26 19:08:38,554][105620] Updated weights for policy 1, policy_version 519534 (0.0009) [2023-12-26 19:08:38,603][105620] Updated weights for policy 1, policy_version 519544 (0.0010) [2023-12-26 19:08:38,648][105620] Updated weights for policy 1, policy_version 519554 (0.0010) [2023-12-26 19:08:38,870][105692] Updated weights for policy 0, policy_version 518945 (0.0009) [2023-12-26 19:08:38,924][105692] Updated weights for policy 0, policy_version 518955 (0.0010) [2023-12-26 19:08:38,979][105692] Updated weights for policy 0, policy_version 518967 (0.0010) [2023-12-26 19:08:39,274][105620] Updated weights for policy 1, policy_version 519564 (0.0009) [2023-12-26 19:08:39,342][105620] Updated weights for policy 1, policy_version 519574 (0.0011) [2023-12-26 19:08:39,402][105620] Updated weights for policy 1, policy_version 519584 (0.0010) [2023-12-26 19:08:39,773][105692] Updated weights for policy 0, policy_version 518977 (0.0007) [2023-12-26 19:08:39,845][105692] Updated weights for policy 0, policy_version 518987 (0.0009) [2023-12-26 19:08:39,907][105692] Updated weights for policy 0, policy_version 518997 (0.0008) [2023-12-26 19:08:39,969][105692] Updated weights for policy 0, policy_version 519007 (0.0009) [2023-12-26 19:08:40,177][105620] Updated weights for policy 1, policy_version 519594 (0.0011) [2023-12-26 19:08:40,240][105620] Updated weights for policy 1, policy_version 519604 (0.0011) [2023-12-26 19:08:40,303][105620] Updated weights for policy 1, policy_version 519614 (0.0011) [2023-12-26 19:08:40,369][105620] Updated weights for policy 1, policy_version 519624 (0.0011) [2023-12-26 19:08:40,750][105692] Updated weights for policy 0, policy_version 519017 (0.0009) [2023-12-26 19:08:40,805][105692] Updated weights for policy 0, policy_version 519028 (0.0010) [2023-12-26 19:08:40,872][105692] Updated weights for policy 0, policy_version 519038 (0.0009) [2023-12-26 19:08:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 265928704. Throughput: 0: 9693.1, 1: 9777.0. Samples: 265938648. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) [2023-12-26 19:08:41,063][104569] Avg episode reward: [(0, '9084.153'), (1, '9264.438')] [2023-12-26 19:08:41,082][105620] Updated weights for policy 1, policy_version 519634 (0.0011) [2023-12-26 19:08:41,146][105620] Updated weights for policy 1, policy_version 519644 (0.0011) [2023-12-26 19:08:41,209][105620] Updated weights for policy 1, policy_version 519654 (0.0011) [2023-12-26 19:08:41,689][105692] Updated weights for policy 0, policy_version 519048 (0.0009) [2023-12-26 19:08:41,758][105692] Updated weights for policy 0, policy_version 519058 (0.0009) [2023-12-26 19:08:41,828][105692] Updated weights for policy 0, policy_version 519068 (0.0008) [2023-12-26 19:08:41,990][105620] Updated weights for policy 1, policy_version 519664 (0.0011) [2023-12-26 19:08:42,041][105620] Updated weights for policy 1, policy_version 519674 (0.0010) [2023-12-26 19:08:42,098][105620] Updated weights for policy 1, policy_version 519684 (0.0006) [2023-12-26 19:08:42,488][105692] Updated weights for policy 0, policy_version 519078 (0.0008) [2023-12-26 19:08:42,553][105692] Updated weights for policy 0, policy_version 519088 (0.0011) [2023-12-26 19:08:42,614][105692] Updated weights for policy 0, policy_version 519098 (0.0011) [2023-12-26 19:08:42,850][105620] Updated weights for policy 1, policy_version 519694 (0.0008) [2023-12-26 19:08:42,904][105620] Updated weights for policy 1, policy_version 519704 (0.0008) [2023-12-26 19:08:42,964][105620] Updated weights for policy 1, policy_version 519714 (0.0008) [2023-12-26 19:08:43,363][105692] Updated weights for policy 0, policy_version 519108 (0.0011) [2023-12-26 19:08:43,421][105692] Updated weights for policy 0, policy_version 519118 (0.0011) [2023-12-26 19:08:43,487][105692] Updated weights for policy 0, policy_version 519128 (0.0010) [2023-12-26 19:08:43,670][105620] Updated weights for policy 1, policy_version 519724 (0.0007) [2023-12-26 19:08:43,735][105620] Updated weights for policy 1, policy_version 519734 (0.0005) [2023-12-26 19:08:43,743][105586] KL-divergence is very high: 125.0109 [2023-12-26 19:08:43,784][105586] KL-divergence is very high: 216.4109 [2023-12-26 19:08:43,790][105620] Updated weights for policy 1, policy_version 519744 (0.0008) [2023-12-26 19:08:43,825][105586] KL-divergence is very high: 215.4754 [2023-12-26 19:08:44,237][105692] Updated weights for policy 0, policy_version 519138 (0.0011) [2023-12-26 19:08:44,288][105692] Updated weights for policy 0, policy_version 519148 (0.0010) [2023-12-26 19:08:44,339][105692] Updated weights for policy 0, policy_version 519158 (0.0009) [2023-12-26 19:08:44,396][105692] Updated weights for policy 0, policy_version 519168 (0.0009) [2023-12-26 19:08:44,413][105620] Updated weights for policy 1, policy_version 519754 (0.0008) [2023-12-26 19:08:44,468][105620] Updated weights for policy 1, policy_version 519764 (0.0008) [2023-12-26 19:08:44,518][105620] Updated weights for policy 1, policy_version 519774 (0.0008) [2023-12-26 19:08:44,568][105620] Updated weights for policy 1, policy_version 519784 (0.0008) [2023-12-26 19:08:45,149][105692] Updated weights for policy 0, policy_version 519178 (0.0011) [2023-12-26 19:08:45,201][105692] Updated weights for policy 0, policy_version 519188 (0.0011) [2023-12-26 19:08:45,261][105692] Updated weights for policy 0, policy_version 519198 (0.0011) [2023-12-26 19:08:45,345][105620] Updated weights for policy 1, policy_version 519794 (0.0008) [2023-12-26 19:08:45,407][105620] Updated weights for policy 1, policy_version 519804 (0.0009) [2023-12-26 19:08:45,489][105620] Updated weights for policy 1, policy_version 519814 (0.0008) [2023-12-26 19:08:46,019][105692] Updated weights for policy 0, policy_version 519208 (0.0010) [2023-12-26 19:08:46,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 266018816. Throughput: 0: 9676.9, 1: 9639.3. Samples: 265994880. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) [2023-12-26 19:08:46,063][104569] Avg episode reward: [(0, '8909.460'), (1, '8993.997')] [2023-12-26 19:08:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000519816_133087232.pth... [2023-12-26 19:08:46,073][105692] Updated weights for policy 0, policy_version 519218 (0.0010) [2023-12-26 19:08:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000518696_132800512.pth [2023-12-26 19:08:46,132][105692] Updated weights for policy 0, policy_version 519228 (0.0011) [2023-12-26 19:08:46,154][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000519232_132939776.pth... [2023-12-26 19:08:46,158][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000518112_132653056.pth [2023-12-26 19:08:46,171][105620] Updated weights for policy 1, policy_version 519824 (0.0006) [2023-12-26 19:08:46,216][105620] Updated weights for policy 1, policy_version 519834 (0.0008) [2023-12-26 19:08:46,261][105620] Updated weights for policy 1, policy_version 519844 (0.0010) [2023-12-26 19:08:46,863][105692] Updated weights for policy 0, policy_version 519238 (0.0011) [2023-12-26 19:08:46,878][105620] Updated weights for policy 1, policy_version 519854 (0.0007) [2023-12-26 19:08:46,922][105692] Updated weights for policy 0, policy_version 519248 (0.0008) [2023-12-26 19:08:46,935][105620] Updated weights for policy 1, policy_version 519864 (0.0007) [2023-12-26 19:08:46,987][105692] Updated weights for policy 0, policy_version 519258 (0.0007) [2023-12-26 19:08:47,001][105620] Updated weights for policy 1, policy_version 519874 (0.0009) [2023-12-26 19:08:47,547][105692] Updated weights for policy 0, policy_version 519268 (0.0006) [2023-12-26 19:08:47,606][105692] Updated weights for policy 0, policy_version 519278 (0.0005) [2023-12-26 19:08:47,620][105620] Updated weights for policy 1, policy_version 519884 (0.0007) [2023-12-26 19:08:47,662][105692] Updated weights for policy 0, policy_version 519288 (0.0011) [2023-12-26 19:08:47,690][105620] Updated weights for policy 1, policy_version 519894 (0.0006) [2023-12-26 19:08:47,753][105620] Updated weights for policy 1, policy_version 519904 (0.0010) [2023-12-26 19:08:48,252][105692] Updated weights for policy 0, policy_version 519298 (0.0010) [2023-12-26 19:08:48,304][105692] Updated weights for policy 0, policy_version 519308 (0.0005) [2023-12-26 19:08:48,322][105620] Updated weights for policy 1, policy_version 519914 (0.0006) [2023-12-26 19:08:48,370][105692] Updated weights for policy 0, policy_version 519318 (0.0007) [2023-12-26 19:08:48,384][105620] Updated weights for policy 1, policy_version 519924 (0.0006) [2023-12-26 19:08:48,433][105692] Updated weights for policy 0, policy_version 519328 (0.0007) [2023-12-26 19:08:48,443][105620] Updated weights for policy 1, policy_version 519934 (0.0006) [2023-12-26 19:08:48,503][105620] Updated weights for policy 1, policy_version 519944 (0.0005) [2023-12-26 19:08:49,119][105620] Updated weights for policy 1, policy_version 519954 (0.0010) [2023-12-26 19:08:49,121][105692] Updated weights for policy 0, policy_version 519338 (0.0007) [2023-12-26 19:08:49,174][105692] Updated weights for policy 0, policy_version 519348 (0.0005) [2023-12-26 19:08:49,178][105620] Updated weights for policy 1, policy_version 519964 (0.0011) [2023-12-26 19:08:49,227][105692] Updated weights for policy 0, policy_version 519358 (0.0007) [2023-12-26 19:08:49,240][105620] Updated weights for policy 1, policy_version 519974 (0.0011) [2023-12-26 19:08:49,874][105620] Updated weights for policy 1, policy_version 519984 (0.0008) [2023-12-26 19:08:49,922][105692] Updated weights for policy 0, policy_version 519368 (0.0011) [2023-12-26 19:08:49,932][105620] Updated weights for policy 1, policy_version 519994 (0.0008) [2023-12-26 19:08:49,989][105692] Updated weights for policy 0, policy_version 519378 (0.0010) [2023-12-26 19:08:49,992][105620] Updated weights for policy 1, policy_version 520004 (0.0007) [2023-12-26 19:08:50,055][105692] Updated weights for policy 0, policy_version 519388 (0.0011) [2023-12-26 19:08:50,607][105620] Updated weights for policy 1, policy_version 520014 (0.0010) [2023-12-26 19:08:50,657][105620] Updated weights for policy 1, policy_version 520024 (0.0009) [2023-12-26 19:08:50,718][105620] Updated weights for policy 1, policy_version 520034 (0.0009) [2023-12-26 19:08:50,784][105692] Updated weights for policy 0, policy_version 519398 (0.0008) [2023-12-26 19:08:50,839][105692] Updated weights for policy 0, policy_version 519408 (0.0009) [2023-12-26 19:08:50,891][105692] Updated weights for policy 0, policy_version 519418 (0.0009) [2023-12-26 19:08:51,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 266133504. Throughput: 0: 9689.3, 1: 9808.2. Samples: 266118828. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) [2023-12-26 19:08:51,062][104569] Avg episode reward: [(0, '8906.778'), (1, '9172.453')] [2023-12-26 19:08:51,512][105620] Updated weights for policy 1, policy_version 520044 (0.0009) [2023-12-26 19:08:51,574][105620] Updated weights for policy 1, policy_version 520054 (0.0008) [2023-12-26 19:08:51,636][105620] Updated weights for policy 1, policy_version 520064 (0.0008) [2023-12-26 19:08:51,650][105692] Updated weights for policy 0, policy_version 519428 (0.0007) [2023-12-26 19:08:51,722][105692] Updated weights for policy 0, policy_version 519438 (0.0008) [2023-12-26 19:08:51,814][105692] Updated weights for policy 0, policy_version 519448 (0.0008) [2023-12-26 19:08:52,324][105620] Updated weights for policy 1, policy_version 520074 (0.0008) [2023-12-26 19:08:52,391][105620] Updated weights for policy 1, policy_version 520084 (0.0008) [2023-12-26 19:08:52,461][105620] Updated weights for policy 1, policy_version 520094 (0.0010) [2023-12-26 19:08:52,523][105620] Updated weights for policy 1, policy_version 520104 (0.0010) [2023-12-26 19:08:52,557][105692] Updated weights for policy 0, policy_version 519458 (0.0008) [2023-12-26 19:08:52,614][105692] Updated weights for policy 0, policy_version 519468 (0.0005) [2023-12-26 19:08:52,671][105692] Updated weights for policy 0, policy_version 519478 (0.0006) [2023-12-26 19:08:52,722][105692] Updated weights for policy 0, policy_version 519488 (0.0008) [2023-12-26 19:08:53,281][105620] Updated weights for policy 1, policy_version 520114 (0.0011) [2023-12-26 19:08:53,336][105620] Updated weights for policy 1, policy_version 520124 (0.0010) [2023-12-26 19:08:53,395][105620] Updated weights for policy 1, policy_version 520134 (0.0009) [2023-12-26 19:08:53,450][105692] Updated weights for policy 0, policy_version 519498 (0.0007) [2023-12-26 19:08:53,506][105692] Updated weights for policy 0, policy_version 519508 (0.0008) [2023-12-26 19:08:53,568][105692] Updated weights for policy 0, policy_version 519518 (0.0008) [2023-12-26 19:08:54,146][105620] Updated weights for policy 1, policy_version 520144 (0.0011) [2023-12-26 19:08:54,199][105620] Updated weights for policy 1, policy_version 520154 (0.0011) [2023-12-26 19:08:54,250][105620] Updated weights for policy 1, policy_version 520164 (0.0010) [2023-12-26 19:08:54,323][105692] Updated weights for policy 0, policy_version 519528 (0.0009) [2023-12-26 19:08:54,367][105692] Updated weights for policy 0, policy_version 519538 (0.0007) [2023-12-26 19:08:54,423][105692] Updated weights for policy 0, policy_version 519548 (0.0008) [2023-12-26 19:08:54,986][105620] Updated weights for policy 1, policy_version 520174 (0.0007) [2023-12-26 19:08:55,040][105620] Updated weights for policy 1, policy_version 520184 (0.0008) [2023-12-26 19:08:55,089][105620] Updated weights for policy 1, policy_version 520194 (0.0010) [2023-12-26 19:08:55,176][105692] Updated weights for policy 0, policy_version 519558 (0.0008) [2023-12-26 19:08:55,228][105692] Updated weights for policy 0, policy_version 519568 (0.0008) [2023-12-26 19:08:55,285][105692] Updated weights for policy 0, policy_version 519578 (0.0008) [2023-12-26 19:08:55,819][105620] Updated weights for policy 1, policy_version 520204 (0.0010) [2023-12-26 19:08:55,884][105620] Updated weights for policy 1, policy_version 520214 (0.0011) [2023-12-26 19:08:55,939][105620] Updated weights for policy 1, policy_version 520224 (0.0006) [2023-12-26 19:08:56,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 266223616. Throughput: 0: 9629.9, 1: 9853.2. Samples: 266232460. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) [2023-12-26 19:08:56,062][105692] Updated weights for policy 0, policy_version 519588 (0.0009) [2023-12-26 19:08:56,063][104569] Avg episode reward: [(0, '8812.824'), (1, '9354.043')] [2023-12-26 19:08:56,117][105692] Updated weights for policy 0, policy_version 519598 (0.0008) [2023-12-26 19:08:56,183][105692] Updated weights for policy 0, policy_version 519608 (0.0007) [2023-12-26 19:08:56,659][105620] Updated weights for policy 1, policy_version 520234 (0.0009) [2023-12-26 19:08:56,707][105620] Updated weights for policy 1, policy_version 520244 (0.0010) [2023-12-26 19:08:56,748][105692] Updated weights for policy 0, policy_version 519618 (0.0008) [2023-12-26 19:08:56,762][105620] Updated weights for policy 1, policy_version 520254 (0.0010) [2023-12-26 19:08:56,815][105692] Updated weights for policy 0, policy_version 519628 (0.0005) [2023-12-26 19:08:56,816][105620] Updated weights for policy 1, policy_version 520264 (0.0010) [2023-12-26 19:08:56,882][105692] Updated weights for policy 0, policy_version 519638 (0.0005) [2023-12-26 19:08:56,936][105692] Updated weights for policy 0, policy_version 519648 (0.0009) [2023-12-26 19:08:57,571][105620] Updated weights for policy 1, policy_version 520274 (0.0010) [2023-12-26 19:08:57,573][105692] Updated weights for policy 0, policy_version 519658 (0.0005) [2023-12-26 19:08:57,612][105585] KL-divergence is very high: 185.1580 [2023-12-26 19:08:57,623][105692] Updated weights for policy 0, policy_version 519668 (0.0005) [2023-12-26 19:08:57,625][105620] Updated weights for policy 1, policy_version 520284 (0.0010) [2023-12-26 19:08:57,649][105585] KL-divergence is very high: 167.4870 [2023-12-26 19:08:57,670][105692] Updated weights for policy 0, policy_version 519678 (0.0006) [2023-12-26 19:08:57,682][105620] Updated weights for policy 1, policy_version 520294 (0.0010) [2023-12-26 19:08:58,410][105692] Updated weights for policy 0, policy_version 519688 (0.0007) [2023-12-26 19:08:58,464][105620] Updated weights for policy 1, policy_version 520304 (0.0010) [2023-12-26 19:08:58,479][105692] Updated weights for policy 0, policy_version 519698 (0.0007) [2023-12-26 19:08:58,525][105620] Updated weights for policy 1, policy_version 520314 (0.0010) [2023-12-26 19:08:58,542][105692] Updated weights for policy 0, policy_version 519708 (0.0006) [2023-12-26 19:08:58,588][105620] Updated weights for policy 1, policy_version 520324 (0.0010) [2023-12-26 19:08:59,302][105692] Updated weights for policy 0, policy_version 519718 (0.0008) [2023-12-26 19:08:59,352][105620] Updated weights for policy 1, policy_version 520334 (0.0010) [2023-12-26 19:08:59,371][105692] Updated weights for policy 0, policy_version 519728 (0.0007) [2023-12-26 19:08:59,413][105620] Updated weights for policy 1, policy_version 520344 (0.0007) [2023-12-26 19:08:59,434][105692] Updated weights for policy 0, policy_version 519738 (0.0008) [2023-12-26 19:08:59,472][105620] Updated weights for policy 1, policy_version 520354 (0.0006) [2023-12-26 19:09:00,005][105692] Updated weights for policy 0, policy_version 519748 (0.0007) [2023-12-26 19:09:00,059][105692] Updated weights for policy 0, policy_version 519758 (0.0005) [2023-12-26 19:09:00,106][105692] Updated weights for policy 0, policy_version 519768 (0.0007) [2023-12-26 19:09:00,290][105620] Updated weights for policy 1, policy_version 520364 (0.0009) [2023-12-26 19:09:00,354][105620] Updated weights for policy 1, policy_version 520374 (0.0007) [2023-12-26 19:09:00,422][105620] Updated weights for policy 1, policy_version 520384 (0.0010) [2023-12-26 19:09:00,711][105692] Updated weights for policy 0, policy_version 519778 (0.0006) [2023-12-26 19:09:00,757][105692] Updated weights for policy 0, policy_version 519788 (0.0008) [2023-12-26 19:09:00,803][105692] Updated weights for policy 0, policy_version 519798 (0.0009) [2023-12-26 19:09:00,850][105692] Updated weights for policy 0, policy_version 519808 (0.0009) [2023-12-26 19:09:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 266321920. Throughput: 0: 9666.5, 1: 9852.3. Samples: 266290844. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) [2023-12-26 19:09:01,063][104569] Avg episode reward: [(0, '8636.101'), (1, '9354.089')] [2023-12-26 19:09:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000520392_133234688.pth... [2023-12-26 19:09:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000519808_133087232.pth... [2023-12-26 19:09:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000518656_132792320.pth [2023-12-26 19:09:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000519240_132939776.pth [2023-12-26 19:09:01,138][105620] Updated weights for policy 1, policy_version 520394 (0.0009) [2023-12-26 19:09:01,187][105620] Updated weights for policy 1, policy_version 520404 (0.0008) [2023-12-26 19:09:01,237][105620] Updated weights for policy 1, policy_version 520414 (0.0008) [2023-12-26 19:09:01,295][105620] Updated weights for policy 1, policy_version 520424 (0.0008) [2023-12-26 19:09:01,676][105692] Updated weights for policy 0, policy_version 519818 (0.0010) [2023-12-26 19:09:01,731][105692] Updated weights for policy 0, policy_version 519828 (0.0010) [2023-12-26 19:09:01,786][105692] Updated weights for policy 0, policy_version 519838 (0.0010) [2023-12-26 19:09:02,106][105620] Updated weights for policy 1, policy_version 520434 (0.0009) [2023-12-26 19:09:02,160][105620] Updated weights for policy 1, policy_version 520444 (0.0009) [2023-12-26 19:09:02,206][105620] Updated weights for policy 1, policy_version 520454 (0.0008) [2023-12-26 19:09:02,543][105692] Updated weights for policy 0, policy_version 519848 (0.0010) [2023-12-26 19:09:02,601][105692] Updated weights for policy 0, policy_version 519858 (0.0009) [2023-12-26 19:09:02,660][105692] Updated weights for policy 0, policy_version 519868 (0.0009) [2023-12-26 19:09:02,959][105620] Updated weights for policy 1, policy_version 520464 (0.0009) [2023-12-26 19:09:03,020][105620] Updated weights for policy 1, policy_version 520474 (0.0009) [2023-12-26 19:09:03,069][105620] Updated weights for policy 1, policy_version 520484 (0.0009) [2023-12-26 19:09:03,351][105692] Updated weights for policy 0, policy_version 519878 (0.0008) [2023-12-26 19:09:03,411][105692] Updated weights for policy 0, policy_version 519888 (0.0005) [2023-12-26 19:09:03,469][105692] Updated weights for policy 0, policy_version 519898 (0.0005) [2023-12-26 19:09:03,832][105620] Updated weights for policy 1, policy_version 520494 (0.0009) [2023-12-26 19:09:03,891][105620] Updated weights for policy 1, policy_version 520504 (0.0009) [2023-12-26 19:09:03,946][105620] Updated weights for policy 1, policy_version 520514 (0.0009) [2023-12-26 19:09:04,180][105692] Updated weights for policy 0, policy_version 519908 (0.0007) [2023-12-26 19:09:04,244][105692] Updated weights for policy 0, policy_version 519918 (0.0008) [2023-12-26 19:09:04,301][105692] Updated weights for policy 0, policy_version 519928 (0.0008) [2023-12-26 19:09:04,714][105620] Updated weights for policy 1, policy_version 520524 (0.0009) [2023-12-26 19:09:04,769][105620] Updated weights for policy 1, policy_version 520534 (0.0009) [2023-12-26 19:09:04,820][105620] Updated weights for policy 1, policy_version 520544 (0.0009) [2023-12-26 19:09:05,005][105692] Updated weights for policy 0, policy_version 519938 (0.0007) [2023-12-26 19:09:05,063][105692] Updated weights for policy 0, policy_version 519948 (0.0009) [2023-12-26 19:09:05,127][105692] Updated weights for policy 0, policy_version 519958 (0.0008) [2023-12-26 19:09:05,185][105692] Updated weights for policy 0, policy_version 519968 (0.0009) [2023-12-26 19:09:05,618][105620] Updated weights for policy 1, policy_version 520554 (0.0009) [2023-12-26 19:09:05,665][105620] Updated weights for policy 1, policy_version 520564 (0.0009) [2023-12-26 19:09:05,713][105620] Updated weights for policy 1, policy_version 520574 (0.0008) [2023-12-26 19:09:05,760][105620] Updated weights for policy 1, policy_version 520584 (0.0008) [2023-12-26 19:09:05,865][105692] Updated weights for policy 0, policy_version 519978 (0.0009) [2023-12-26 19:09:05,918][105692] Updated weights for policy 0, policy_version 519988 (0.0010) [2023-12-26 19:09:05,970][105692] Updated weights for policy 0, policy_version 519998 (0.0009) [2023-12-26 19:09:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 266420224. Throughput: 0: 9726.8, 1: 9679.3. Samples: 266405256. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) [2023-12-26 19:09:06,063][104569] Avg episode reward: [(0, '8909.035'), (1, '9176.666')] [2023-12-26 19:09:06,397][105620] Updated weights for policy 1, policy_version 520594 (0.0009) [2023-12-26 19:09:06,450][105620] Updated weights for policy 1, policy_version 520604 (0.0011) [2023-12-26 19:09:06,504][105620] Updated weights for policy 1, policy_version 520614 (0.0008) [2023-12-26 19:09:06,826][105692] Updated weights for policy 0, policy_version 520008 (0.0009) [2023-12-26 19:09:06,882][105692] Updated weights for policy 0, policy_version 520018 (0.0008) [2023-12-26 19:09:06,935][105692] Updated weights for policy 0, policy_version 520028 (0.0008) [2023-12-26 19:09:07,238][105620] Updated weights for policy 1, policy_version 520624 (0.0006) [2023-12-26 19:09:07,302][105620] Updated weights for policy 1, policy_version 520634 (0.0011) [2023-12-26 19:09:07,350][105620] Updated weights for policy 1, policy_version 520644 (0.0010) [2023-12-26 19:09:07,676][105692] Updated weights for policy 0, policy_version 520038 (0.0006) [2023-12-26 19:09:07,729][105692] Updated weights for policy 0, policy_version 520048 (0.0005) [2023-12-26 19:09:07,789][105692] Updated weights for policy 0, policy_version 520058 (0.0007) [2023-12-26 19:09:07,909][105620] Updated weights for policy 1, policy_version 520654 (0.0006) [2023-12-26 19:09:07,961][105620] Updated weights for policy 1, policy_version 520664 (0.0006) [2023-12-26 19:09:08,012][105620] Updated weights for policy 1, policy_version 520674 (0.0010) [2023-12-26 19:09:08,523][105692] Updated weights for policy 0, policy_version 520068 (0.0009) [2023-12-26 19:09:08,574][105692] Updated weights for policy 0, policy_version 520078 (0.0008) [2023-12-26 19:09:08,634][105692] Updated weights for policy 0, policy_version 520088 (0.0009) [2023-12-26 19:09:08,746][105620] Updated weights for policy 1, policy_version 520684 (0.0010) [2023-12-26 19:09:08,813][105620] Updated weights for policy 1, policy_version 520694 (0.0011) [2023-12-26 19:09:08,877][105620] Updated weights for policy 1, policy_version 520704 (0.0011) [2023-12-26 19:09:09,418][105692] Updated weights for policy 0, policy_version 520098 (0.0010) [2023-12-26 19:09:09,473][105692] Updated weights for policy 0, policy_version 520108 (0.0008) [2023-12-26 19:09:09,529][105692] Updated weights for policy 0, policy_version 520118 (0.0008) [2023-12-26 19:09:09,562][105620] Updated weights for policy 1, policy_version 520714 (0.0011) [2023-12-26 19:09:09,581][105692] Updated weights for policy 0, policy_version 520128 (0.0008) [2023-12-26 19:09:09,626][105620] Updated weights for policy 1, policy_version 520724 (0.0010) [2023-12-26 19:09:09,678][105620] Updated weights for policy 1, policy_version 520734 (0.0010) [2023-12-26 19:09:09,741][105620] Updated weights for policy 1, policy_version 520744 (0.0011) [2023-12-26 19:09:10,371][105692] Updated weights for policy 0, policy_version 520138 (0.0008) [2023-12-26 19:09:10,427][105692] Updated weights for policy 0, policy_version 520148 (0.0008) [2023-12-26 19:09:10,486][105692] Updated weights for policy 0, policy_version 520158 (0.0006) [2023-12-26 19:09:10,488][105620] Updated weights for policy 1, policy_version 520754 (0.0010) [2023-12-26 19:09:10,544][105620] Updated weights for policy 1, policy_version 520764 (0.0006) [2023-12-26 19:09:10,604][105620] Updated weights for policy 1, policy_version 520774 (0.0009) [2023-12-26 19:09:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 266510336. Throughput: 0: 9710.2, 1: 9738.8. Samples: 266520652. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) [2023-12-26 19:09:11,062][104569] Avg episode reward: [(0, '8725.914'), (1, '9084.694')] [2023-12-26 19:09:11,171][105692] Updated weights for policy 0, policy_version 520168 (0.0008) [2023-12-26 19:09:11,237][105692] Updated weights for policy 0, policy_version 520178 (0.0008) [2023-12-26 19:09:11,298][105692] Updated weights for policy 0, policy_version 520188 (0.0009) [2023-12-26 19:09:11,391][105620] Updated weights for policy 1, policy_version 520784 (0.0008) [2023-12-26 19:09:11,450][105620] Updated weights for policy 1, policy_version 520794 (0.0006) [2023-12-26 19:09:11,515][105620] Updated weights for policy 1, policy_version 520804 (0.0007) [2023-12-26 19:09:12,083][105692] Updated weights for policy 0, policy_version 520198 (0.0009) [2023-12-26 19:09:12,147][105692] Updated weights for policy 0, policy_version 520208 (0.0009) [2023-12-26 19:09:12,205][105692] Updated weights for policy 0, policy_version 520218 (0.0009) [2023-12-26 19:09:12,222][105620] Updated weights for policy 1, policy_version 520814 (0.0006) [2023-12-26 19:09:12,283][105620] Updated weights for policy 1, policy_version 520824 (0.0007) [2023-12-26 19:09:12,349][105620] Updated weights for policy 1, policy_version 520834 (0.0008) [2023-12-26 19:09:12,853][105692] Updated weights for policy 0, policy_version 520228 (0.0008) [2023-12-26 19:09:12,916][105692] Updated weights for policy 0, policy_version 520238 (0.0009) [2023-12-26 19:09:12,982][105692] Updated weights for policy 0, policy_version 520248 (0.0009) [2023-12-26 19:09:13,144][105620] Updated weights for policy 1, policy_version 520844 (0.0008) [2023-12-26 19:09:13,196][105620] Updated weights for policy 1, policy_version 520854 (0.0005) [2023-12-26 19:09:13,245][105620] Updated weights for policy 1, policy_version 520864 (0.0008) [2023-12-26 19:09:13,742][105692] Updated weights for policy 0, policy_version 520258 (0.0009) [2023-12-26 19:09:13,800][105692] Updated weights for policy 0, policy_version 520268 (0.0009) [2023-12-26 19:09:13,861][105692] Updated weights for policy 0, policy_version 520278 (0.0007) [2023-12-26 19:09:13,918][105692] Updated weights for policy 0, policy_version 520288 (0.0006) [2023-12-26 19:09:14,000][105620] Updated weights for policy 1, policy_version 520874 (0.0009) [2023-12-26 19:09:14,052][105620] Updated weights for policy 1, policy_version 520884 (0.0009) [2023-12-26 19:09:14,107][105620] Updated weights for policy 1, policy_version 520894 (0.0009) [2023-12-26 19:09:14,159][105620] Updated weights for policy 1, policy_version 520904 (0.0009) [2023-12-26 19:09:14,655][105692] Updated weights for policy 0, policy_version 520298 (0.0010) [2023-12-26 19:09:14,713][105692] Updated weights for policy 0, policy_version 520308 (0.0006) [2023-12-26 19:09:14,762][105692] Updated weights for policy 0, policy_version 520318 (0.0007) [2023-12-26 19:09:14,869][105620] Updated weights for policy 1, policy_version 520914 (0.0006) [2023-12-26 19:09:14,930][105620] Updated weights for policy 1, policy_version 520924 (0.0006) [2023-12-26 19:09:14,996][105620] Updated weights for policy 1, policy_version 520934 (0.0006) [2023-12-26 19:09:15,504][105692] Updated weights for policy 0, policy_version 520328 (0.0011) [2023-12-26 19:09:15,565][105692] Updated weights for policy 0, policy_version 520338 (0.0010) [2023-12-26 19:09:15,627][105620] Updated weights for policy 1, policy_version 520944 (0.0006) [2023-12-26 19:09:15,628][105692] Updated weights for policy 0, policy_version 520348 (0.0010) [2023-12-26 19:09:15,686][105620] Updated weights for policy 1, policy_version 520954 (0.0005) [2023-12-26 19:09:15,749][105620] Updated weights for policy 1, policy_version 520964 (0.0005) [2023-12-26 19:09:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 266608640. Throughput: 0: 9678.8, 1: 9731.1. Samples: 266577400. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) [2023-12-26 19:09:16,063][104569] Avg episode reward: [(0, '8818.113'), (1, '9173.277')] [2023-12-26 19:09:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000520352_133226496.pth... [2023-12-26 19:09:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000520968_133382144.pth... [2023-12-26 19:09:16,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000519816_133087232.pth [2023-12-26 19:09:16,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000519232_132939776.pth [2023-12-26 19:09:16,373][105692] Updated weights for policy 0, policy_version 520358 (0.0011) [2023-12-26 19:09:16,426][105620] Updated weights for policy 1, policy_version 520974 (0.0007) [2023-12-26 19:09:16,432][105692] Updated weights for policy 0, policy_version 520368 (0.0010) [2023-12-26 19:09:16,477][105620] Updated weights for policy 1, policy_version 520984 (0.0008) [2023-12-26 19:09:16,493][105692] Updated weights for policy 0, policy_version 520378 (0.0010) [2023-12-26 19:09:16,532][105620] Updated weights for policy 1, policy_version 520994 (0.0008) [2023-12-26 19:09:17,232][105692] Updated weights for policy 0, policy_version 520388 (0.0010) [2023-12-26 19:09:17,287][105692] Updated weights for policy 0, policy_version 520398 (0.0010) [2023-12-26 19:09:17,297][105620] Updated weights for policy 1, policy_version 521004 (0.0007) [2023-12-26 19:09:17,345][105692] Updated weights for policy 0, policy_version 520408 (0.0010) [2023-12-26 19:09:17,359][105620] Updated weights for policy 1, policy_version 521014 (0.0006) [2023-12-26 19:09:17,420][105620] Updated weights for policy 1, policy_version 521024 (0.0007) [2023-12-26 19:09:18,034][105692] Updated weights for policy 0, policy_version 520418 (0.0009) [2023-12-26 19:09:18,097][105692] Updated weights for policy 0, policy_version 520428 (0.0009) [2023-12-26 19:09:18,155][105692] Updated weights for policy 0, policy_version 520438 (0.0006) [2023-12-26 19:09:18,191][105620] Updated weights for policy 1, policy_version 521034 (0.0008) [2023-12-26 19:09:18,206][105692] Updated weights for policy 0, policy_version 520448 (0.0005) [2023-12-26 19:09:18,250][105620] Updated weights for policy 1, policy_version 521044 (0.0009) [2023-12-26 19:09:18,302][105620] Updated weights for policy 1, policy_version 521054 (0.0007) [2023-12-26 19:09:18,370][105620] Updated weights for policy 1, policy_version 521064 (0.0006) [2023-12-26 19:09:18,875][105692] Updated weights for policy 0, policy_version 520458 (0.0006) [2023-12-26 19:09:18,940][105692] Updated weights for policy 0, policy_version 520468 (0.0006) [2023-12-26 19:09:18,988][105692] Updated weights for policy 0, policy_version 520478 (0.0009) [2023-12-26 19:09:19,199][105620] Updated weights for policy 1, policy_version 521074 (0.0010) [2023-12-26 19:09:19,275][105620] Updated weights for policy 1, policy_version 521084 (0.0010) [2023-12-26 19:09:19,342][105620] Updated weights for policy 1, policy_version 521094 (0.0010) [2023-12-26 19:09:19,699][105692] Updated weights for policy 0, policy_version 520488 (0.0010) [2023-12-26 19:09:19,748][105692] Updated weights for policy 0, policy_version 520498 (0.0009) [2023-12-26 19:09:19,804][105692] Updated weights for policy 0, policy_version 520508 (0.0009) [2023-12-26 19:09:20,067][105620] Updated weights for policy 1, policy_version 521104 (0.0008) [2023-12-26 19:09:20,131][105620] Updated weights for policy 1, policy_version 521114 (0.0007) [2023-12-26 19:09:20,188][105620] Updated weights for policy 1, policy_version 521124 (0.0006) [2023-12-26 19:09:20,582][105692] Updated weights for policy 0, policy_version 520518 (0.0008) [2023-12-26 19:09:20,642][105692] Updated weights for policy 0, policy_version 520528 (0.0007) [2023-12-26 19:09:20,695][105692] Updated weights for policy 0, policy_version 520538 (0.0011) [2023-12-26 19:09:20,806][105620] Updated weights for policy 1, policy_version 521134 (0.0009) [2023-12-26 19:09:20,869][105620] Updated weights for policy 1, policy_version 521144 (0.0011) [2023-12-26 19:09:20,934][105620] Updated weights for policy 1, policy_version 521154 (0.0008) [2023-12-26 19:09:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 266706944. Throughput: 0: 9709.9, 1: 9647.6. Samples: 266692168. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) [2023-12-26 19:09:21,063][104569] Avg episode reward: [(0, '8997.341'), (1, '9263.567')] [2023-12-26 19:09:21,409][105692] Updated weights for policy 0, policy_version 520548 (0.0009) [2023-12-26 19:09:21,469][105692] Updated weights for policy 0, policy_version 520558 (0.0010) [2023-12-26 19:09:21,526][105692] Updated weights for policy 0, policy_version 520568 (0.0009) [2023-12-26 19:09:21,574][105620] Updated weights for policy 1, policy_version 521164 (0.0007) [2023-12-26 19:09:21,639][105620] Updated weights for policy 1, policy_version 521174 (0.0010) [2023-12-26 19:09:21,706][105620] Updated weights for policy 1, policy_version 521184 (0.0010) [2023-12-26 19:09:22,249][105692] Updated weights for policy 0, policy_version 520578 (0.0008) [2023-12-26 19:09:22,312][105692] Updated weights for policy 0, policy_version 520588 (0.0009) [2023-12-26 19:09:22,377][105692] Updated weights for policy 0, policy_version 520598 (0.0010) [2023-12-26 19:09:22,442][105692] Updated weights for policy 0, policy_version 520608 (0.0009) [2023-12-26 19:09:22,512][105620] Updated weights for policy 1, policy_version 521194 (0.0009) [2023-12-26 19:09:22,566][105620] Updated weights for policy 1, policy_version 521204 (0.0010) [2023-12-26 19:09:22,624][105620] Updated weights for policy 1, policy_version 521214 (0.0008) [2023-12-26 19:09:22,673][105620] Updated weights for policy 1, policy_version 521224 (0.0008) [2023-12-26 19:09:23,160][105692] Updated weights for policy 0, policy_version 520618 (0.0006) [2023-12-26 19:09:23,207][105692] Updated weights for policy 0, policy_version 520628 (0.0010) [2023-12-26 19:09:23,262][105692] Updated weights for policy 0, policy_version 520638 (0.0007) [2023-12-26 19:09:23,495][105620] Updated weights for policy 1, policy_version 521234 (0.0008) [2023-12-26 19:09:23,544][105620] Updated weights for policy 1, policy_version 521244 (0.0008) [2023-12-26 19:09:23,598][105620] Updated weights for policy 1, policy_version 521254 (0.0007) [2023-12-26 19:09:23,953][105692] Updated weights for policy 0, policy_version 520648 (0.0009) [2023-12-26 19:09:24,005][105692] Updated weights for policy 0, policy_version 520658 (0.0010) [2023-12-26 19:09:24,059][105692] Updated weights for policy 0, policy_version 520668 (0.0010) [2023-12-26 19:09:24,250][105620] Updated weights for policy 1, policy_version 521264 (0.0008) [2023-12-26 19:09:24,318][105620] Updated weights for policy 1, policy_version 521274 (0.0009) [2023-12-26 19:09:24,387][105620] Updated weights for policy 1, policy_version 521284 (0.0006) [2023-12-26 19:09:24,809][105692] Updated weights for policy 0, policy_version 520678 (0.0010) [2023-12-26 19:09:24,874][105692] Updated weights for policy 0, policy_version 520688 (0.0010) [2023-12-26 19:09:24,942][105692] Updated weights for policy 0, policy_version 520698 (0.0010) [2023-12-26 19:09:25,038][105620] Updated weights for policy 1, policy_version 521294 (0.0007) [2023-12-26 19:09:25,096][105620] Updated weights for policy 1, policy_version 521304 (0.0010) [2023-12-26 19:09:25,153][105620] Updated weights for policy 1, policy_version 521314 (0.0010) [2023-12-26 19:09:25,637][105692] Updated weights for policy 0, policy_version 520708 (0.0009) [2023-12-26 19:09:25,684][105692] Updated weights for policy 0, policy_version 520718 (0.0008) [2023-12-26 19:09:25,743][105692] Updated weights for policy 0, policy_version 520728 (0.0008) [2023-12-26 19:09:25,873][105620] Updated weights for policy 1, policy_version 521324 (0.0008) [2023-12-26 19:09:25,934][105620] Updated weights for policy 1, policy_version 521334 (0.0006) [2023-12-26 19:09:25,992][105620] Updated weights for policy 1, policy_version 521344 (0.0005) [2023-12-26 19:09:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 266805248. Throughput: 0: 9667.2, 1: 9682.9. Samples: 266809404. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) [2023-12-26 19:09:26,063][104569] Avg episode reward: [(0, '9357.889'), (1, '9171.581')] [2023-12-26 19:09:26,394][105692] Updated weights for policy 0, policy_version 520738 (0.0008) [2023-12-26 19:09:26,442][105692] Updated weights for policy 0, policy_version 520748 (0.0009) [2023-12-26 19:09:26,505][105692] Updated weights for policy 0, policy_version 520758 (0.0011) [2023-12-26 19:09:26,512][105620] Updated weights for policy 1, policy_version 521354 (0.0005) [2023-12-26 19:09:26,557][105692] Updated weights for policy 0, policy_version 520768 (0.0010) [2023-12-26 19:09:26,567][105620] Updated weights for policy 1, policy_version 521364 (0.0007) [2023-12-26 19:09:26,624][105620] Updated weights for policy 1, policy_version 521374 (0.0009) [2023-12-26 19:09:26,682][105620] Updated weights for policy 1, policy_version 521384 (0.0008) [2023-12-26 19:09:27,295][105692] Updated weights for policy 0, policy_version 520778 (0.0010) [2023-12-26 19:09:27,326][105620] Updated weights for policy 1, policy_version 521394 (0.0006) [2023-12-26 19:09:27,348][105692] Updated weights for policy 0, policy_version 520788 (0.0010) [2023-12-26 19:09:27,389][105692] Updated weights for policy 0, policy_version 520798 (0.0010) [2023-12-26 19:09:27,390][105620] Updated weights for policy 1, policy_version 521404 (0.0008) [2023-12-26 19:09:27,452][105620] Updated weights for policy 1, policy_version 521415 (0.0009) [2023-12-26 19:09:28,067][105692] Updated weights for policy 0, policy_version 520808 (0.0006) [2023-12-26 19:09:28,082][105620] Updated weights for policy 1, policy_version 521425 (0.0010) [2023-12-26 19:09:28,115][105692] Updated weights for policy 0, policy_version 520818 (0.0009) [2023-12-26 19:09:28,136][105620] Updated weights for policy 1, policy_version 521435 (0.0010) [2023-12-26 19:09:28,167][105692] Updated weights for policy 0, policy_version 520828 (0.0006) [2023-12-26 19:09:28,198][105620] Updated weights for policy 1, policy_version 521446 (0.0011) [2023-12-26 19:09:28,807][105692] Updated weights for policy 0, policy_version 520838 (0.0010) [2023-12-26 19:09:28,869][105692] Updated weights for policy 0, policy_version 520848 (0.0010) [2023-12-26 19:09:28,887][105620] Updated weights for policy 1, policy_version 521456 (0.0006) [2023-12-26 19:09:28,927][105692] Updated weights for policy 0, policy_version 520858 (0.0010) [2023-12-26 19:09:28,941][105620] Updated weights for policy 1, policy_version 521466 (0.0007) [2023-12-26 19:09:29,001][105620] Updated weights for policy 1, policy_version 521476 (0.0007) [2023-12-26 19:09:29,615][105692] Updated weights for policy 0, policy_version 520868 (0.0008) [2023-12-26 19:09:29,673][105692] Updated weights for policy 0, policy_version 520878 (0.0010) [2023-12-26 19:09:29,741][105692] Updated weights for policy 0, policy_version 520888 (0.0008) [2023-12-26 19:09:29,786][105620] Updated weights for policy 1, policy_version 521486 (0.0009) [2023-12-26 19:09:29,846][105620] Updated weights for policy 1, policy_version 521496 (0.0010) [2023-12-26 19:09:29,902][105620] Updated weights for policy 1, policy_version 521506 (0.0010) [2023-12-26 19:09:30,431][105692] Updated weights for policy 0, policy_version 520898 (0.0007) [2023-12-26 19:09:30,496][105692] Updated weights for policy 0, policy_version 520908 (0.0010) [2023-12-26 19:09:30,551][105692] Updated weights for policy 0, policy_version 520918 (0.0010) [2023-12-26 19:09:30,609][105692] Updated weights for policy 0, policy_version 520928 (0.0010) [2023-12-26 19:09:30,616][105620] Updated weights for policy 1, policy_version 521516 (0.0011) [2023-12-26 19:09:30,675][105620] Updated weights for policy 1, policy_version 521526 (0.0009) [2023-12-26 19:09:30,733][105620] Updated weights for policy 1, policy_version 521536 (0.0006) [2023-12-26 19:09:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 266903552. Throughput: 0: 9735.1, 1: 9766.2. Samples: 266872432. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) [2023-12-26 19:09:31,062][104569] Avg episode reward: [(0, '9268.222'), (1, '9354.218')] [2023-12-26 19:09:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000520928_133373952.pth... [2023-12-26 19:09:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000521544_133529600.pth... [2023-12-26 19:09:31,069][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000520392_133234688.pth [2023-12-26 19:09:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000519808_133087232.pth [2023-12-26 19:09:31,308][105692] Updated weights for policy 0, policy_version 520938 (0.0008) [2023-12-26 19:09:31,375][105692] Updated weights for policy 0, policy_version 520948 (0.0009) [2023-12-26 19:09:31,421][105620] Updated weights for policy 1, policy_version 521546 (0.0008) [2023-12-26 19:09:31,435][105692] Updated weights for policy 0, policy_version 520958 (0.0008) [2023-12-26 19:09:31,480][105620] Updated weights for policy 1, policy_version 521556 (0.0007) [2023-12-26 19:09:31,537][105620] Updated weights for policy 1, policy_version 521566 (0.0008) [2023-12-26 19:09:31,591][105620] Updated weights for policy 1, policy_version 521576 (0.0008) [2023-12-26 19:09:32,200][105692] Updated weights for policy 0, policy_version 520968 (0.0011) [2023-12-26 19:09:32,248][105692] Updated weights for policy 0, policy_version 520978 (0.0010) [2023-12-26 19:09:32,291][105620] Updated weights for policy 1, policy_version 521586 (0.0007) [2023-12-26 19:09:32,303][105692] Updated weights for policy 0, policy_version 520988 (0.0011) [2023-12-26 19:09:32,354][105620] Updated weights for policy 1, policy_version 521596 (0.0008) [2023-12-26 19:09:32,417][105620] Updated weights for policy 1, policy_version 521606 (0.0011) [2023-12-26 19:09:33,007][105620] Updated weights for policy 1, policy_version 521616 (0.0010) [2023-12-26 19:09:33,053][105620] Updated weights for policy 1, policy_version 521626 (0.0008) [2023-12-26 19:09:33,058][105692] Updated weights for policy 0, policy_version 520998 (0.0010) [2023-12-26 19:09:33,101][105620] Updated weights for policy 1, policy_version 521636 (0.0008) [2023-12-26 19:09:33,122][105692] Updated weights for policy 0, policy_version 521008 (0.0010) [2023-12-26 19:09:33,183][105692] Updated weights for policy 0, policy_version 521018 (0.0010) [2023-12-26 19:09:33,751][105620] Updated weights for policy 1, policy_version 521646 (0.0010) [2023-12-26 19:09:33,806][105620] Updated weights for policy 1, policy_version 521656 (0.0010) [2023-12-26 19:09:33,864][105620] Updated weights for policy 1, policy_version 521666 (0.0005) [2023-12-26 19:09:33,912][105692] Updated weights for policy 0, policy_version 521028 (0.0010) [2023-12-26 19:09:33,962][105692] Updated weights for policy 0, policy_version 521038 (0.0010) [2023-12-26 19:09:34,014][105692] Updated weights for policy 0, policy_version 521048 (0.0008) [2023-12-26 19:09:34,576][105620] Updated weights for policy 1, policy_version 521676 (0.0008) [2023-12-26 19:09:34,646][105620] Updated weights for policy 1, policy_version 521686 (0.0011) [2023-12-26 19:09:34,679][105692] Updated weights for policy 0, policy_version 521058 (0.0007) [2023-12-26 19:09:34,711][105620] Updated weights for policy 1, policy_version 521696 (0.0011) [2023-12-26 19:09:34,743][105692] Updated weights for policy 0, policy_version 521068 (0.0006) [2023-12-26 19:09:34,803][105692] Updated weights for policy 0, policy_version 521078 (0.0010) [2023-12-26 19:09:34,860][105692] Updated weights for policy 0, policy_version 521088 (0.0009) [2023-12-26 19:09:35,443][105620] Updated weights for policy 1, policy_version 521706 (0.0011) [2023-12-26 19:09:35,498][105620] Updated weights for policy 1, policy_version 521716 (0.0008) [2023-12-26 19:09:35,522][105692] Updated weights for policy 0, policy_version 521098 (0.0010) [2023-12-26 19:09:35,552][105620] Updated weights for policy 1, policy_version 521726 (0.0005) [2023-12-26 19:09:35,584][105692] Updated weights for policy 0, policy_version 521108 (0.0010) [2023-12-26 19:09:35,601][105620] Updated weights for policy 1, policy_version 521736 (0.0006) [2023-12-26 19:09:35,641][105692] Updated weights for policy 0, policy_version 521118 (0.0010) [2023-12-26 19:09:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 267001856. Throughput: 0: 9705.2, 1: 9685.2. Samples: 266991400. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) [2023-12-26 19:09:36,062][104569] Avg episode reward: [(0, '9268.146'), (1, '9354.008')] [2023-12-26 19:09:36,314][105620] Updated weights for policy 1, policy_version 521746 (0.0011) [2023-12-26 19:09:36,371][105620] Updated weights for policy 1, policy_version 521756 (0.0011) [2023-12-26 19:09:36,380][105692] Updated weights for policy 0, policy_version 521128 (0.0011) [2023-12-26 19:09:36,427][105620] Updated weights for policy 1, policy_version 521766 (0.0010) [2023-12-26 19:09:36,443][105692] Updated weights for policy 0, policy_version 521138 (0.0011) [2023-12-26 19:09:36,502][105692] Updated weights for policy 0, policy_version 521148 (0.0011) [2023-12-26 19:09:37,144][105620] Updated weights for policy 1, policy_version 521776 (0.0007) [2023-12-26 19:09:37,205][105620] Updated weights for policy 1, policy_version 521786 (0.0007) [2023-12-26 19:09:37,229][105692] Updated weights for policy 0, policy_version 521158 (0.0011) [2023-12-26 19:09:37,255][105620] Updated weights for policy 1, policy_version 521796 (0.0007) [2023-12-26 19:09:37,284][105692] Updated weights for policy 0, policy_version 521168 (0.0008) [2023-12-26 19:09:37,348][105692] Updated weights for policy 0, policy_version 521178 (0.0009) [2023-12-26 19:09:37,911][105620] Updated weights for policy 1, policy_version 521806 (0.0006) [2023-12-26 19:09:37,960][105620] Updated weights for policy 1, policy_version 521816 (0.0005) [2023-12-26 19:09:38,011][105620] Updated weights for policy 1, policy_version 521826 (0.0006) [2023-12-26 19:09:38,162][105692] Updated weights for policy 0, policy_version 521188 (0.0007) [2023-12-26 19:09:38,224][105692] Updated weights for policy 0, policy_version 521198 (0.0006) [2023-12-26 19:09:38,293][105692] Updated weights for policy 0, policy_version 521208 (0.0009) [2023-12-26 19:09:38,665][105620] Updated weights for policy 1, policy_version 521836 (0.0008) [2023-12-26 19:09:38,712][105620] Updated weights for policy 1, policy_version 521846 (0.0009) [2023-12-26 19:09:38,772][105620] Updated weights for policy 1, policy_version 521856 (0.0009) [2023-12-26 19:09:38,964][105692] Updated weights for policy 0, policy_version 521218 (0.0009) [2023-12-26 19:09:39,018][105692] Updated weights for policy 0, policy_version 521228 (0.0010) [2023-12-26 19:09:39,074][105692] Updated weights for policy 0, policy_version 521238 (0.0010) [2023-12-26 19:09:39,130][105692] Updated weights for policy 0, policy_version 521248 (0.0010) [2023-12-26 19:09:39,563][105620] Updated weights for policy 1, policy_version 521866 (0.0009) [2023-12-26 19:09:39,619][105620] Updated weights for policy 1, policy_version 521876 (0.0007) [2023-12-26 19:09:39,690][105620] Updated weights for policy 1, policy_version 521886 (0.0005) [2023-12-26 19:09:39,747][105620] Updated weights for policy 1, policy_version 521896 (0.0009) [2023-12-26 19:09:39,916][105692] Updated weights for policy 0, policy_version 521258 (0.0010) [2023-12-26 19:09:39,979][105692] Updated weights for policy 0, policy_version 521268 (0.0009) [2023-12-26 19:09:40,037][105692] Updated weights for policy 0, policy_version 521278 (0.0009) [2023-12-26 19:09:40,421][105620] Updated weights for policy 1, policy_version 521906 (0.0008) [2023-12-26 19:09:40,471][105620] Updated weights for policy 1, policy_version 521916 (0.0005) [2023-12-26 19:09:40,523][105620] Updated weights for policy 1, policy_version 521926 (0.0010) [2023-12-26 19:09:40,699][105692] Updated weights for policy 0, policy_version 521288 (0.0006) [2023-12-26 19:09:40,751][105692] Updated weights for policy 0, policy_version 521298 (0.0005) [2023-12-26 19:09:40,812][105692] Updated weights for policy 0, policy_version 521308 (0.0008) [2023-12-26 19:09:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 267100160. Throughput: 0: 9725.9, 1: 9746.0. Samples: 267108696. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) [2023-12-26 19:09:41,063][104569] Avg episode reward: [(0, '9267.794'), (1, '9353.954')] [2023-12-26 19:09:41,225][105620] Updated weights for policy 1, policy_version 521936 (0.0008) [2023-12-26 19:09:41,293][105620] Updated weights for policy 1, policy_version 521946 (0.0009) [2023-12-26 19:09:41,366][105620] Updated weights for policy 1, policy_version 521956 (0.0009) [2023-12-26 19:09:41,499][105692] Updated weights for policy 0, policy_version 521318 (0.0008) [2023-12-26 19:09:41,554][105692] Updated weights for policy 0, policy_version 521328 (0.0009) [2023-12-26 19:09:41,619][105692] Updated weights for policy 0, policy_version 521338 (0.0009) [2023-12-26 19:09:42,212][105620] Updated weights for policy 1, policy_version 521966 (0.0010) [2023-12-26 19:09:42,269][105620] Updated weights for policy 1, policy_version 521976 (0.0011) [2023-12-26 19:09:42,334][105620] Updated weights for policy 1, policy_version 521986 (0.0011) [2023-12-26 19:09:42,403][105692] Updated weights for policy 0, policy_version 521348 (0.0009) [2023-12-26 19:09:42,458][105692] Updated weights for policy 0, policy_version 521358 (0.0008) [2023-12-26 19:09:42,525][105692] Updated weights for policy 0, policy_version 521368 (0.0008) [2023-12-26 19:09:43,040][105620] Updated weights for policy 1, policy_version 521996 (0.0011) [2023-12-26 19:09:43,099][105620] Updated weights for policy 1, policy_version 522006 (0.0010) [2023-12-26 19:09:43,159][105620] Updated weights for policy 1, policy_version 522016 (0.0010) [2023-12-26 19:09:43,311][105692] Updated weights for policy 0, policy_version 521378 (0.0008) [2023-12-26 19:09:43,362][105692] Updated weights for policy 0, policy_version 521388 (0.0005) [2023-12-26 19:09:43,415][105692] Updated weights for policy 0, policy_version 521398 (0.0005) [2023-12-26 19:09:43,474][105692] Updated weights for policy 0, policy_version 521408 (0.0006) [2023-12-26 19:09:43,883][105620] Updated weights for policy 1, policy_version 522026 (0.0010) [2023-12-26 19:09:43,943][105620] Updated weights for policy 1, policy_version 522036 (0.0010) [2023-12-26 19:09:44,005][105620] Updated weights for policy 1, policy_version 522046 (0.0010) [2023-12-26 19:09:44,062][105620] Updated weights for policy 1, policy_version 522056 (0.0011) [2023-12-26 19:09:44,062][105692] Updated weights for policy 0, policy_version 521418 (0.0006) [2023-12-26 19:09:44,128][105692] Updated weights for policy 0, policy_version 521428 (0.0006) [2023-12-26 19:09:44,128][105585] KL-divergence is very high: 218.1140 [2023-12-26 19:09:44,146][105585] KL-divergence is very high: 349.1828 [2023-12-26 19:09:44,177][105585] KL-divergence is very high: 569.9568 [2023-12-26 19:09:44,189][105692] Updated weights for policy 0, policy_version 521438 (0.0008) [2023-12-26 19:09:44,195][105585] KL-divergence is very high: 545.3282 [2023-12-26 19:09:44,719][105586] KL-divergence is very high: 152.9262 [2023-12-26 19:09:44,726][105620] Updated weights for policy 1, policy_version 522066 (0.0010) [2023-12-26 19:09:44,777][105586] KL-divergence is very high: 221.0277 [2023-12-26 19:09:44,797][105620] Updated weights for policy 1, policy_version 522076 (0.0011) [2023-12-26 19:09:44,822][105586] KL-divergence is very high: 206.9191 [2023-12-26 19:09:44,849][105620] Updated weights for policy 1, policy_version 522086 (0.0010) [2023-12-26 19:09:44,905][105585] KL-divergence is very high: 549.3627 [2023-12-26 19:09:44,932][105585] KL-divergence is very high: 466.4923 [2023-12-26 19:09:44,943][105692] Updated weights for policy 0, policy_version 521448 (0.0008) [2023-12-26 19:09:44,947][105585] KL-divergence is very high: 445.1187 [2023-12-26 19:09:44,973][105585] KL-divergence is very high: 337.7229 [2023-12-26 19:09:44,991][105585] KL-divergence is very high: 313.8678 [2023-12-26 19:09:44,997][105692] Updated weights for policy 0, policy_version 521458 (0.0009) [2023-12-26 19:09:45,026][105585] KL-divergence is very high: 226.8060 [2023-12-26 19:09:45,044][105585] KL-divergence is very high: 215.6288 [2023-12-26 19:09:45,063][105692] Updated weights for policy 0, policy_version 521468 (0.0009) [2023-12-26 19:09:45,075][105585] KL-divergence is very high: 152.8238 [2023-12-26 19:09:45,602][105620] Updated weights for policy 1, policy_version 522096 (0.0010) [2023-12-26 19:09:45,664][105620] Updated weights for policy 1, policy_version 522106 (0.0009) [2023-12-26 19:09:45,722][105620] Updated weights for policy 1, policy_version 522116 (0.0009) [2023-12-26 19:09:45,804][105692] Updated weights for policy 0, policy_version 521478 (0.0009) [2023-12-26 19:09:45,866][105692] Updated weights for policy 0, policy_version 521488 (0.0009) [2023-12-26 19:09:45,925][105692] Updated weights for policy 0, policy_version 521498 (0.0007) [2023-12-26 19:09:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 267198464. Throughput: 0: 9676.5, 1: 9750.5. Samples: 267165060. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) [2023-12-26 19:09:46,062][104569] Avg episode reward: [(0, '8724.928'), (1, '9083.710')] [2023-12-26 19:09:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000522120_133677056.pth... [2023-12-26 19:09:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000521504_133521408.pth... [2023-12-26 19:09:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000520968_133382144.pth [2023-12-26 19:09:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000520352_133226496.pth [2023-12-26 19:09:46,451][105620] Updated weights for policy 1, policy_version 522126 (0.0009) [2023-12-26 19:09:46,511][105620] Updated weights for policy 1, policy_version 522136 (0.0009) [2023-12-26 19:09:46,569][105620] Updated weights for policy 1, policy_version 522146 (0.0009) [2023-12-26 19:09:46,698][105692] Updated weights for policy 0, policy_version 521508 (0.0010) [2023-12-26 19:09:46,753][105692] Updated weights for policy 0, policy_version 521518 (0.0009) [2023-12-26 19:09:46,819][105692] Updated weights for policy 0, policy_version 521528 (0.0006) [2023-12-26 19:09:47,328][105620] Updated weights for policy 1, policy_version 522156 (0.0009) [2023-12-26 19:09:47,384][105620] Updated weights for policy 1, policy_version 522166 (0.0008) [2023-12-26 19:09:47,434][105620] Updated weights for policy 1, policy_version 522176 (0.0008) [2023-12-26 19:09:47,546][105692] Updated weights for policy 0, policy_version 521538 (0.0009) [2023-12-26 19:09:47,611][105692] Updated weights for policy 0, policy_version 521548 (0.0007) [2023-12-26 19:09:47,669][105692] Updated weights for policy 0, policy_version 521558 (0.0005) [2023-12-26 19:09:47,719][105692] Updated weights for policy 0, policy_version 521568 (0.0006) [2023-12-26 19:09:48,093][105620] Updated weights for policy 1, policy_version 522186 (0.0008) [2023-12-26 19:09:48,158][105620] Updated weights for policy 1, policy_version 522196 (0.0009) [2023-12-26 19:09:48,211][105620] Updated weights for policy 1, policy_version 522206 (0.0008) [2023-12-26 19:09:48,258][105620] Updated weights for policy 1, policy_version 522216 (0.0009) [2023-12-26 19:09:48,296][105692] Updated weights for policy 0, policy_version 521578 (0.0008) [2023-12-26 19:09:48,350][105692] Updated weights for policy 0, policy_version 521588 (0.0009) [2023-12-26 19:09:48,417][105692] Updated weights for policy 0, policy_version 521598 (0.0009) [2023-12-26 19:09:48,994][105620] Updated weights for policy 1, policy_version 522226 (0.0011) [2023-12-26 19:09:49,065][105620] Updated weights for policy 1, policy_version 522236 (0.0011) [2023-12-26 19:09:49,136][105620] Updated weights for policy 1, policy_version 522246 (0.0011) [2023-12-26 19:09:49,157][105692] Updated weights for policy 0, policy_version 521608 (0.0008) [2023-12-26 19:09:49,227][105692] Updated weights for policy 0, policy_version 521618 (0.0007) [2023-12-26 19:09:49,292][105692] Updated weights for policy 0, policy_version 521628 (0.0007) [2023-12-26 19:09:49,863][105620] Updated weights for policy 1, policy_version 522256 (0.0011) [2023-12-26 19:09:49,872][105692] Updated weights for policy 0, policy_version 521638 (0.0009) [2023-12-26 19:09:49,927][105620] Updated weights for policy 1, policy_version 522266 (0.0011) [2023-12-26 19:09:49,937][105692] Updated weights for policy 0, policy_version 521648 (0.0006) [2023-12-26 19:09:49,993][105620] Updated weights for policy 1, policy_version 522276 (0.0010) [2023-12-26 19:09:50,003][105692] Updated weights for policy 0, policy_version 521658 (0.0007) [2023-12-26 19:09:50,728][105620] Updated weights for policy 1, policy_version 522286 (0.0011) [2023-12-26 19:09:50,753][105692] Updated weights for policy 0, policy_version 521668 (0.0008) [2023-12-26 19:09:50,791][105620] Updated weights for policy 1, policy_version 522296 (0.0011) [2023-12-26 19:09:50,817][105692] Updated weights for policy 0, policy_version 521678 (0.0005) [2023-12-26 19:09:50,850][105620] Updated weights for policy 1, policy_version 522306 (0.0011) [2023-12-26 19:09:50,876][105692] Updated weights for policy 0, policy_version 521688 (0.0005) [2023-12-26 19:09:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 267296768. Throughput: 0: 9670.0, 1: 9815.5. Samples: 267282100. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) [2023-12-26 19:09:51,062][104569] Avg episode reward: [(0, '8907.598'), (1, '9174.387')] [2023-12-26 19:09:51,600][105620] Updated weights for policy 1, policy_version 522316 (0.0011) [2023-12-26 19:09:51,605][105692] Updated weights for policy 0, policy_version 521698 (0.0007) [2023-12-26 19:09:51,672][105620] Updated weights for policy 1, policy_version 522326 (0.0011) [2023-12-26 19:09:51,677][105692] Updated weights for policy 0, policy_version 521708 (0.0007) [2023-12-26 19:09:51,741][105692] Updated weights for policy 0, policy_version 521718 (0.0008) [2023-12-26 19:09:51,753][105620] Updated weights for policy 1, policy_version 522336 (0.0011) [2023-12-26 19:09:51,796][105692] Updated weights for policy 0, policy_version 521728 (0.0006) [2023-12-26 19:09:52,385][105620] Updated weights for policy 1, policy_version 522346 (0.0011) [2023-12-26 19:09:52,431][105692] Updated weights for policy 0, policy_version 521738 (0.0008) [2023-12-26 19:09:52,455][105620] Updated weights for policy 1, policy_version 522356 (0.0011) [2023-12-26 19:09:52,492][105692] Updated weights for policy 0, policy_version 521748 (0.0007) [2023-12-26 19:09:52,518][105620] Updated weights for policy 1, policy_version 522366 (0.0011) [2023-12-26 19:09:52,548][105692] Updated weights for policy 0, policy_version 521758 (0.0005) [2023-12-26 19:09:52,581][105620] Updated weights for policy 1, policy_version 522376 (0.0011) [2023-12-26 19:09:53,260][105692] Updated weights for policy 0, policy_version 521768 (0.0010) [2023-12-26 19:09:53,321][105692] Updated weights for policy 0, policy_version 521778 (0.0008) [2023-12-26 19:09:53,327][105620] Updated weights for policy 1, policy_version 522386 (0.0005) [2023-12-26 19:09:53,377][105692] Updated weights for policy 0, policy_version 521788 (0.0008) [2023-12-26 19:09:53,380][105620] Updated weights for policy 1, policy_version 522396 (0.0005) [2023-12-26 19:09:53,442][105620] Updated weights for policy 1, policy_version 522406 (0.0005) [2023-12-26 19:09:54,049][105620] Updated weights for policy 1, policy_version 522416 (0.0008) [2023-12-26 19:09:54,099][105620] Updated weights for policy 1, policy_version 522426 (0.0008) [2023-12-26 19:09:54,162][105620] Updated weights for policy 1, policy_version 522436 (0.0008) [2023-12-26 19:09:54,180][105692] Updated weights for policy 0, policy_version 521798 (0.0008) [2023-12-26 19:09:54,235][105692] Updated weights for policy 0, policy_version 521808 (0.0009) [2023-12-26 19:09:54,282][105692] Updated weights for policy 0, policy_version 521818 (0.0009) [2023-12-26 19:09:54,878][105620] Updated weights for policy 1, policy_version 522446 (0.0009) [2023-12-26 19:09:54,931][105620] Updated weights for policy 1, policy_version 522456 (0.0008) [2023-12-26 19:09:54,982][105620] Updated weights for policy 1, policy_version 522466 (0.0009) [2023-12-26 19:09:55,062][105692] Updated weights for policy 0, policy_version 521829 (0.0010) [2023-12-26 19:09:55,119][105692] Updated weights for policy 0, policy_version 521839 (0.0010) [2023-12-26 19:09:55,176][105692] Updated weights for policy 0, policy_version 521849 (0.0010) [2023-12-26 19:09:55,677][105620] Updated weights for policy 1, policy_version 522476 (0.0007) [2023-12-26 19:09:55,733][105620] Updated weights for policy 1, policy_version 522486 (0.0005) [2023-12-26 19:09:55,783][105620] Updated weights for policy 1, policy_version 522496 (0.0005) [2023-12-26 19:09:56,025][105692] Updated weights for policy 0, policy_version 521859 (0.0010) [2023-12-26 19:09:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 267386880. Throughput: 0: 9697.9, 1: 9815.7. Samples: 267398764. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) [2023-12-26 19:09:56,062][104569] Avg episode reward: [(0, '8908.425'), (1, '9174.030')] [2023-12-26 19:09:56,090][105692] Updated weights for policy 0, policy_version 521869 (0.0009) [2023-12-26 19:09:56,141][105692] Updated weights for policy 0, policy_version 521879 (0.0009) [2023-12-26 19:09:56,423][105620] Updated weights for policy 1, policy_version 522506 (0.0006) [2023-12-26 19:09:56,475][105620] Updated weights for policy 1, policy_version 522516 (0.0009) [2023-12-26 19:09:56,536][105620] Updated weights for policy 1, policy_version 522526 (0.0009) [2023-12-26 19:09:56,586][105620] Updated weights for policy 1, policy_version 522536 (0.0009) [2023-12-26 19:09:56,918][105692] Updated weights for policy 0, policy_version 521889 (0.0009) [2023-12-26 19:09:56,979][105692] Updated weights for policy 0, policy_version 521899 (0.0010) [2023-12-26 19:09:57,032][105692] Updated weights for policy 0, policy_version 521909 (0.0010) [2023-12-26 19:09:57,207][105620] Updated weights for policy 1, policy_version 522546 (0.0009) [2023-12-26 19:09:57,256][105620] Updated weights for policy 1, policy_version 522556 (0.0008) [2023-12-26 19:09:57,303][105620] Updated weights for policy 1, policy_version 522566 (0.0005) [2023-12-26 19:09:57,888][105620] Updated weights for policy 1, policy_version 522576 (0.0006) [2023-12-26 19:09:57,905][105692] Updated weights for policy 0, policy_version 521921 (0.0010) [2023-12-26 19:09:57,937][105620] Updated weights for policy 1, policy_version 522586 (0.0005) [2023-12-26 19:09:57,960][105692] Updated weights for policy 0, policy_version 521931 (0.0009) [2023-12-26 19:09:57,983][105620] Updated weights for policy 1, policy_version 522596 (0.0005) [2023-12-26 19:09:58,019][105692] Updated weights for policy 0, policy_version 521941 (0.0008) [2023-12-26 19:09:58,073][105692] Updated weights for policy 0, policy_version 521951 (0.0009) [2023-12-26 19:09:58,700][105620] Updated weights for policy 1, policy_version 522606 (0.0008) [2023-12-26 19:09:58,756][105620] Updated weights for policy 1, policy_version 522616 (0.0011) [2023-12-26 19:09:58,819][105620] Updated weights for policy 1, policy_version 522626 (0.0010) [2023-12-26 19:09:58,903][105692] Updated weights for policy 0, policy_version 521961 (0.0008) [2023-12-26 19:09:58,961][105692] Updated weights for policy 0, policy_version 521971 (0.0008) [2023-12-26 19:09:59,019][105692] Updated weights for policy 0, policy_version 521981 (0.0008) [2023-12-26 19:09:59,492][105620] Updated weights for policy 1, policy_version 522636 (0.0011) [2023-12-26 19:09:59,551][105620] Updated weights for policy 1, policy_version 522646 (0.0010) [2023-12-26 19:09:59,616][105620] Updated weights for policy 1, policy_version 522656 (0.0010) [2023-12-26 19:09:59,715][105692] Updated weights for policy 0, policy_version 521991 (0.0008) [2023-12-26 19:09:59,770][105692] Updated weights for policy 0, policy_version 522001 (0.0008) [2023-12-26 19:09:59,833][105692] Updated weights for policy 0, policy_version 522011 (0.0008) [2023-12-26 19:10:00,335][105620] Updated weights for policy 1, policy_version 522666 (0.0009) [2023-12-26 19:10:00,382][105620] Updated weights for policy 1, policy_version 522676 (0.0009) [2023-12-26 19:10:00,434][105620] Updated weights for policy 1, policy_version 522686 (0.0010) [2023-12-26 19:10:00,490][105620] Updated weights for policy 1, policy_version 522696 (0.0010) [2023-12-26 19:10:00,595][105692] Updated weights for policy 0, policy_version 522021 (0.0009) [2023-12-26 19:10:00,652][105692] Updated weights for policy 0, policy_version 522031 (0.0010) [2023-12-26 19:10:00,708][105692] Updated weights for policy 0, policy_version 522041 (0.0009) [2023-12-26 19:10:01,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 267485184. Throughput: 0: 9633.8, 1: 9898.0. Samples: 267456332. Policy #0 lag: (min: 36.0, avg: 47.6, max: 48.0) [2023-12-26 19:10:01,063][104569] Avg episode reward: [(0, '8999.471'), (1, '9084.204')] [2023-12-26 19:10:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000522048_133660672.pth... [2023-12-26 19:10:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000520928_133373952.pth [2023-12-26 19:10:01,116][105620] Updated weights for policy 1, policy_version 522706 (0.0009) [2023-12-26 19:10:01,175][105620] Updated weights for policy 1, policy_version 522716 (0.0010) [2023-12-26 19:10:01,233][105620] Updated weights for policy 1, policy_version 522726 (0.0010) [2023-12-26 19:10:01,246][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000522728_133832704.pth... [2023-12-26 19:10:01,251][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000521544_133529600.pth [2023-12-26 19:10:01,517][105692] Updated weights for policy 0, policy_version 522051 (0.0008) [2023-12-26 19:10:01,573][105692] Updated weights for policy 0, policy_version 522061 (0.0010) [2023-12-26 19:10:01,635][105692] Updated weights for policy 0, policy_version 522071 (0.0010) [2023-12-26 19:10:01,923][105620] Updated weights for policy 1, policy_version 522736 (0.0009) [2023-12-26 19:10:01,974][105620] Updated weights for policy 1, policy_version 522746 (0.0007) [2023-12-26 19:10:02,035][105620] Updated weights for policy 1, policy_version 522756 (0.0008) [2023-12-26 19:10:02,433][105692] Updated weights for policy 0, policy_version 522081 (0.0009) [2023-12-26 19:10:02,507][105692] Updated weights for policy 0, policy_version 522091 (0.0010) [2023-12-26 19:10:02,573][105692] Updated weights for policy 0, policy_version 522101 (0.0009) [2023-12-26 19:10:02,642][105692] Updated weights for policy 0, policy_version 522111 (0.0009) [2023-12-26 19:10:02,662][105620] Updated weights for policy 1, policy_version 522766 (0.0006) [2023-12-26 19:10:02,716][105620] Updated weights for policy 1, policy_version 522776 (0.0005) [2023-12-26 19:10:02,770][105620] Updated weights for policy 1, policy_version 522786 (0.0009) [2023-12-26 19:10:03,391][105620] Updated weights for policy 1, policy_version 522796 (0.0009) [2023-12-26 19:10:03,439][105692] Updated weights for policy 0, policy_version 522121 (0.0008) [2023-12-26 19:10:03,441][105620] Updated weights for policy 1, policy_version 522806 (0.0006) [2023-12-26 19:10:03,488][105692] Updated weights for policy 0, policy_version 522131 (0.0006) [2023-12-26 19:10:03,490][105620] Updated weights for policy 1, policy_version 522816 (0.0006) [2023-12-26 19:10:03,540][105692] Updated weights for policy 0, policy_version 522141 (0.0007) [2023-12-26 19:10:04,170][105620] Updated weights for policy 1, policy_version 522826 (0.0006) [2023-12-26 19:10:04,229][105620] Updated weights for policy 1, policy_version 522836 (0.0006) [2023-12-26 19:10:04,287][105620] Updated weights for policy 1, policy_version 522846 (0.0006) [2023-12-26 19:10:04,342][105620] Updated weights for policy 1, policy_version 522856 (0.0006) [2023-12-26 19:10:04,369][105692] Updated weights for policy 0, policy_version 522151 (0.0009) [2023-12-26 19:10:04,426][105692] Updated weights for policy 0, policy_version 522161 (0.0007) [2023-12-26 19:10:04,478][105692] Updated weights for policy 0, policy_version 522171 (0.0010) [2023-12-26 19:10:04,913][105620] Updated weights for policy 1, policy_version 522866 (0.0005) [2023-12-26 19:10:04,963][105620] Updated weights for policy 1, policy_version 522876 (0.0005) [2023-12-26 19:10:05,019][105620] Updated weights for policy 1, policy_version 522886 (0.0006) [2023-12-26 19:10:05,382][105692] Updated weights for policy 0, policy_version 522182 (0.0009) [2023-12-26 19:10:05,431][105692] Updated weights for policy 0, policy_version 522192 (0.0006) [2023-12-26 19:10:05,480][105692] Updated weights for policy 0, policy_version 522202 (0.0007) [2023-12-26 19:10:05,599][105620] Updated weights for policy 1, policy_version 522896 (0.0008) [2023-12-26 19:10:05,649][105620] Updated weights for policy 1, policy_version 522906 (0.0007) [2023-12-26 19:10:05,698][105620] Updated weights for policy 1, policy_version 522916 (0.0005) [2023-12-26 19:10:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 267583488. Throughput: 0: 9530.9, 1: 10067.1. Samples: 267574080. Policy #0 lag: (min: 36.0, avg: 47.6, max: 48.0) [2023-12-26 19:10:06,062][104569] Avg episode reward: [(0, '9090.215'), (1, '9084.322')] [2023-12-26 19:10:06,297][105692] Updated weights for policy 0, policy_version 522212 (0.0009) [2023-12-26 19:10:06,361][105692] Updated weights for policy 0, policy_version 522222 (0.0007) [2023-12-26 19:10:06,362][105620] Updated weights for policy 1, policy_version 522926 (0.0007) [2023-12-26 19:10:06,417][105692] Updated weights for policy 0, policy_version 522232 (0.0006) [2023-12-26 19:10:06,420][105620] Updated weights for policy 1, policy_version 522936 (0.0008) [2023-12-26 19:10:06,482][105620] Updated weights for policy 1, policy_version 522946 (0.0007) [2023-12-26 19:10:07,199][105692] Updated weights for policy 0, policy_version 522242 (0.0006) [2023-12-26 19:10:07,209][105620] Updated weights for policy 1, policy_version 522956 (0.0009) [2023-12-26 19:10:07,252][105692] Updated weights for policy 0, policy_version 522252 (0.0006) [2023-12-26 19:10:07,274][105620] Updated weights for policy 1, policy_version 522966 (0.0009) [2023-12-26 19:10:07,312][105692] Updated weights for policy 0, policy_version 522262 (0.0007) [2023-12-26 19:10:07,335][105620] Updated weights for policy 1, policy_version 522976 (0.0009) [2023-12-26 19:10:07,374][105692] Updated weights for policy 0, policy_version 522272 (0.0007) [2023-12-26 19:10:07,999][105692] Updated weights for policy 0, policy_version 522282 (0.0009) [2023-12-26 19:10:08,049][105692] Updated weights for policy 0, policy_version 522292 (0.0009) [2023-12-26 19:10:08,104][105692] Updated weights for policy 0, policy_version 522302 (0.0009) [2023-12-26 19:10:08,144][105620] Updated weights for policy 1, policy_version 522986 (0.0007) [2023-12-26 19:10:08,200][105620] Updated weights for policy 1, policy_version 522996 (0.0009) [2023-12-26 19:10:08,247][105620] Updated weights for policy 1, policy_version 523006 (0.0008) [2023-12-26 19:10:08,295][105620] Updated weights for policy 1, policy_version 523016 (0.0005) [2023-12-26 19:10:08,929][105620] Updated weights for policy 1, policy_version 523026 (0.0007) [2023-12-26 19:10:08,955][105692] Updated weights for policy 0, policy_version 522312 (0.0006) [2023-12-26 19:10:08,983][105620] Updated weights for policy 1, policy_version 523036 (0.0007) [2023-12-26 19:10:09,019][105692] Updated weights for policy 0, policy_version 522322 (0.0006) [2023-12-26 19:10:09,034][105620] Updated weights for policy 1, policy_version 523046 (0.0010) [2023-12-26 19:10:09,064][105692] Updated weights for policy 0, policy_version 522332 (0.0007) [2023-12-26 19:10:09,721][105620] Updated weights for policy 1, policy_version 523056 (0.0010) [2023-12-26 19:10:09,774][105620] Updated weights for policy 1, policy_version 523066 (0.0011) [2023-12-26 19:10:09,784][105692] Updated weights for policy 0, policy_version 522342 (0.0007) [2023-12-26 19:10:09,838][105620] Updated weights for policy 1, policy_version 523076 (0.0009) [2023-12-26 19:10:09,847][105692] Updated weights for policy 0, policy_version 522352 (0.0008) [2023-12-26 19:10:09,910][105692] Updated weights for policy 0, policy_version 522362 (0.0008) [2023-12-26 19:10:10,588][105620] Updated weights for policy 1, policy_version 523086 (0.0011) [2023-12-26 19:10:10,632][105692] Updated weights for policy 0, policy_version 522372 (0.0007) [2023-12-26 19:10:10,637][105620] Updated weights for policy 1, policy_version 523096 (0.0008) [2023-12-26 19:10:10,683][105620] Updated weights for policy 1, policy_version 523106 (0.0005) [2023-12-26 19:10:10,689][105692] Updated weights for policy 0, policy_version 522382 (0.0008) [2023-12-26 19:10:10,748][105692] Updated weights for policy 0, policy_version 522392 (0.0008) [2023-12-26 19:10:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 267681792. Throughput: 0: 9465.0, 1: 10090.7. Samples: 267689408. Policy #0 lag: (min: 36.0, avg: 47.6, max: 48.0) [2023-12-26 19:10:11,063][104569] Avg episode reward: [(0, '9179.371'), (1, '9354.572')] [2023-12-26 19:10:11,382][105620] Updated weights for policy 1, policy_version 523116 (0.0007) [2023-12-26 19:10:11,442][105620] Updated weights for policy 1, policy_version 523126 (0.0009) [2023-12-26 19:10:11,507][105620] Updated weights for policy 1, policy_version 523136 (0.0009) [2023-12-26 19:10:11,595][105692] Updated weights for policy 0, policy_version 522402 (0.0007) [2023-12-26 19:10:11,666][105692] Updated weights for policy 0, policy_version 522412 (0.0008) [2023-12-26 19:10:11,736][105692] Updated weights for policy 0, policy_version 522422 (0.0009) [2023-12-26 19:10:11,804][105692] Updated weights for policy 0, policy_version 522432 (0.0009) [2023-12-26 19:10:12,232][105620] Updated weights for policy 1, policy_version 523146 (0.0007) [2023-12-26 19:10:12,298][105620] Updated weights for policy 1, policy_version 523156 (0.0009) [2023-12-26 19:10:12,364][105620] Updated weights for policy 1, policy_version 523166 (0.0008) [2023-12-26 19:10:12,430][105620] Updated weights for policy 1, policy_version 523176 (0.0009) [2023-12-26 19:10:12,574][105692] Updated weights for policy 0, policy_version 522442 (0.0009) [2023-12-26 19:10:12,633][105692] Updated weights for policy 0, policy_version 522452 (0.0010) [2023-12-26 19:10:12,687][105692] Updated weights for policy 0, policy_version 522463 (0.0010) [2023-12-26 19:10:13,081][105620] Updated weights for policy 1, policy_version 523186 (0.0005) [2023-12-26 19:10:13,135][105620] Updated weights for policy 1, policy_version 523196 (0.0005) [2023-12-26 19:10:13,186][105620] Updated weights for policy 1, policy_version 523206 (0.0005) [2023-12-26 19:10:13,510][105692] Updated weights for policy 0, policy_version 522473 (0.0008) [2023-12-26 19:10:13,564][105692] Updated weights for policy 0, policy_version 522483 (0.0005) [2023-12-26 19:10:13,611][105692] Updated weights for policy 0, policy_version 522493 (0.0007) [2023-12-26 19:10:13,894][105620] Updated weights for policy 1, policy_version 523216 (0.0009) [2023-12-26 19:10:13,942][105620] Updated weights for policy 1, policy_version 523226 (0.0010) [2023-12-26 19:10:13,991][105620] Updated weights for policy 1, policy_version 523236 (0.0010) [2023-12-26 19:10:14,359][105692] Updated weights for policy 0, policy_version 522503 (0.0009) [2023-12-26 19:10:14,417][105692] Updated weights for policy 0, policy_version 522513 (0.0009) [2023-12-26 19:10:14,476][105692] Updated weights for policy 0, policy_version 522523 (0.0009) [2023-12-26 19:10:14,711][105620] Updated weights for policy 1, policy_version 523246 (0.0010) [2023-12-26 19:10:14,764][105620] Updated weights for policy 1, policy_version 523256 (0.0010) [2023-12-26 19:10:14,832][105620] Updated weights for policy 1, policy_version 523266 (0.0008) [2023-12-26 19:10:15,291][105692] Updated weights for policy 0, policy_version 522533 (0.0009) [2023-12-26 19:10:15,344][105692] Updated weights for policy 0, policy_version 522543 (0.0009) [2023-12-26 19:10:15,394][105692] Updated weights for policy 0, policy_version 522553 (0.0008) [2023-12-26 19:10:15,587][105620] Updated weights for policy 1, policy_version 523276 (0.0007) [2023-12-26 19:10:15,639][105620] Updated weights for policy 1, policy_version 523286 (0.0006) [2023-12-26 19:10:15,690][105620] Updated weights for policy 1, policy_version 523296 (0.0010) [2023-12-26 19:10:16,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19387.6, 300 sec: 19521.9). Total num frames: 267771904. Throughput: 0: 9373.9, 1: 10024.3. Samples: 267745356. Policy #0 lag: (min: 36.0, avg: 47.6, max: 48.0) [2023-12-26 19:10:16,063][104569] Avg episode reward: [(0, '9265.979'), (1, '9354.742')] [2023-12-26 19:10:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000522560_133791744.pth... [2023-12-26 19:10:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000523304_133980160.pth... [2023-12-26 19:10:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000521504_133521408.pth [2023-12-26 19:10:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000522120_133677056.pth [2023-12-26 19:10:16,231][105692] Updated weights for policy 0, policy_version 522563 (0.0008) [2023-12-26 19:10:16,288][105692] Updated weights for policy 0, policy_version 522573 (0.0009) [2023-12-26 19:10:16,312][105620] Updated weights for policy 1, policy_version 523306 (0.0009) [2023-12-26 19:10:16,350][105692] Updated weights for policy 0, policy_version 522583 (0.0007) [2023-12-26 19:10:16,372][105620] Updated weights for policy 1, policy_version 523316 (0.0007) [2023-12-26 19:10:16,431][105620] Updated weights for policy 1, policy_version 523326 (0.0007) [2023-12-26 19:10:16,492][105620] Updated weights for policy 1, policy_version 523336 (0.0008) [2023-12-26 19:10:17,087][105620] Updated weights for policy 1, policy_version 523346 (0.0005) [2023-12-26 19:10:17,142][105620] Updated weights for policy 1, policy_version 523356 (0.0005) [2023-12-26 19:10:17,196][105620] Updated weights for policy 1, policy_version 523366 (0.0008) [2023-12-26 19:10:17,197][105692] Updated weights for policy 0, policy_version 522593 (0.0007) [2023-12-26 19:10:17,251][105692] Updated weights for policy 0, policy_version 522603 (0.0010) [2023-12-26 19:10:17,306][105692] Updated weights for policy 0, policy_version 522614 (0.0011) [2023-12-26 19:10:17,844][105620] Updated weights for policy 1, policy_version 523376 (0.0008) [2023-12-26 19:10:17,896][105620] Updated weights for policy 1, policy_version 523386 (0.0008) [2023-12-26 19:10:17,954][105620] Updated weights for policy 1, policy_version 523396 (0.0009) [2023-12-26 19:10:18,124][105692] Updated weights for policy 0, policy_version 522625 (0.0010) [2023-12-26 19:10:18,187][105692] Updated weights for policy 0, policy_version 522635 (0.0010) [2023-12-26 19:10:18,254][105692] Updated weights for policy 0, policy_version 522645 (0.0010) [2023-12-26 19:10:18,322][105692] Updated weights for policy 0, policy_version 522655 (0.0010) [2023-12-26 19:10:18,641][105620] Updated weights for policy 1, policy_version 523406 (0.0009) [2023-12-26 19:10:18,702][105620] Updated weights for policy 1, policy_version 523416 (0.0009) [2023-12-26 19:10:18,763][105620] Updated weights for policy 1, policy_version 523426 (0.0009) [2023-12-26 19:10:19,071][105692] Updated weights for policy 0, policy_version 522665 (0.0005) [2023-12-26 19:10:19,127][105692] Updated weights for policy 0, policy_version 522675 (0.0005) [2023-12-26 19:10:19,173][105692] Updated weights for policy 0, policy_version 522685 (0.0005) [2023-12-26 19:10:19,576][105620] Updated weights for policy 1, policy_version 523436 (0.0009) [2023-12-26 19:10:19,635][105620] Updated weights for policy 1, policy_version 523446 (0.0009) [2023-12-26 19:10:19,683][105586] KL-divergence is very high: 137.4970 [2023-12-26 19:10:19,687][105620] Updated weights for policy 1, policy_version 523456 (0.0009) [2023-12-26 19:10:19,720][105586] KL-divergence is very high: 176.9727 [2023-12-26 19:10:19,871][105692] Updated weights for policy 0, policy_version 522695 (0.0006) [2023-12-26 19:10:19,932][105692] Updated weights for policy 0, policy_version 522705 (0.0008) [2023-12-26 19:10:19,992][105692] Updated weights for policy 0, policy_version 522715 (0.0009) [2023-12-26 19:10:20,477][105620] Updated weights for policy 1, policy_version 523466 (0.0009) [2023-12-26 19:10:20,532][105620] Updated weights for policy 1, policy_version 523476 (0.0009) [2023-12-26 19:10:20,590][105620] Updated weights for policy 1, policy_version 523486 (0.0009) [2023-12-26 19:10:20,653][105620] Updated weights for policy 1, policy_version 523496 (0.0009) [2023-12-26 19:10:20,727][105692] Updated weights for policy 0, policy_version 522725 (0.0010) [2023-12-26 19:10:20,776][105692] Updated weights for policy 0, policy_version 522735 (0.0009) [2023-12-26 19:10:20,831][105692] Updated weights for policy 0, policy_version 522745 (0.0008) [2023-12-26 19:10:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 267870208. Throughput: 0: 9255.8, 1: 10018.4. Samples: 267858740. Policy #0 lag: (min: 36.0, avg: 47.6, max: 48.0) [2023-12-26 19:10:21,063][104569] Avg episode reward: [(0, '9083.202'), (1, '9175.587')] [2023-12-26 19:10:21,472][105620] Updated weights for policy 1, policy_version 523506 (0.0009) [2023-12-26 19:10:21,526][105620] Updated weights for policy 1, policy_version 523516 (0.0008) [2023-12-26 19:10:21,545][105692] Updated weights for policy 0, policy_version 522755 (0.0008) [2023-12-26 19:10:21,585][105620] Updated weights for policy 1, policy_version 523526 (0.0009) [2023-12-26 19:10:21,608][105692] Updated weights for policy 0, policy_version 522765 (0.0010) [2023-12-26 19:10:21,676][105692] Updated weights for policy 0, policy_version 522775 (0.0008) [2023-12-26 19:10:22,235][105620] Updated weights for policy 1, policy_version 523536 (0.0009) [2023-12-26 19:10:22,296][105620] Updated weights for policy 1, policy_version 523546 (0.0007) [2023-12-26 19:10:22,352][105692] Updated weights for policy 0, policy_version 522785 (0.0009) [2023-12-26 19:10:22,364][105620] Updated weights for policy 1, policy_version 523556 (0.0007) [2023-12-26 19:10:22,420][105692] Updated weights for policy 0, policy_version 522795 (0.0008) [2023-12-26 19:10:22,476][105692] Updated weights for policy 0, policy_version 522805 (0.0009) [2023-12-26 19:10:22,531][105692] Updated weights for policy 0, policy_version 522815 (0.0010) [2023-12-26 19:10:23,050][105620] Updated weights for policy 1, policy_version 523566 (0.0008) [2023-12-26 19:10:23,105][105620] Updated weights for policy 1, policy_version 523576 (0.0009) [2023-12-26 19:10:23,157][105620] Updated weights for policy 1, policy_version 523586 (0.0009) [2023-12-26 19:10:23,333][105692] Updated weights for policy 0, policy_version 522825 (0.0010) [2023-12-26 19:10:23,379][105692] Updated weights for policy 0, policy_version 522835 (0.0008) [2023-12-26 19:10:23,426][105692] Updated weights for policy 0, policy_version 522845 (0.0009) [2023-12-26 19:10:23,835][105620] Updated weights for policy 1, policy_version 523596 (0.0009) [2023-12-26 19:10:23,885][105620] Updated weights for policy 1, policy_version 523606 (0.0009) [2023-12-26 19:10:23,935][105620] Updated weights for policy 1, policy_version 523616 (0.0008) [2023-12-26 19:10:24,244][105692] Updated weights for policy 0, policy_version 522855 (0.0009) [2023-12-26 19:10:24,300][105692] Updated weights for policy 0, policy_version 522865 (0.0008) [2023-12-26 19:10:24,360][105692] Updated weights for policy 0, policy_version 522875 (0.0010) [2023-12-26 19:10:24,608][105620] Updated weights for policy 1, policy_version 523626 (0.0009) [2023-12-26 19:10:24,666][105620] Updated weights for policy 1, policy_version 523636 (0.0008) [2023-12-26 19:10:24,729][105620] Updated weights for policy 1, policy_version 523646 (0.0008) [2023-12-26 19:10:24,780][105620] Updated weights for policy 1, policy_version 523656 (0.0005) [2023-12-26 19:10:25,070][105692] Updated weights for policy 0, policy_version 522885 (0.0006) [2023-12-26 19:10:25,130][105692] Updated weights for policy 0, policy_version 522895 (0.0006) [2023-12-26 19:10:25,192][105692] Updated weights for policy 0, policy_version 522905 (0.0010) [2023-12-26 19:10:25,346][105620] Updated weights for policy 1, policy_version 523666 (0.0008) [2023-12-26 19:10:25,412][105620] Updated weights for policy 1, policy_version 523676 (0.0008) [2023-12-26 19:10:25,478][105620] Updated weights for policy 1, policy_version 523686 (0.0008) [2023-12-26 19:10:25,909][105692] Updated weights for policy 0, policy_version 522915 (0.0010) [2023-12-26 19:10:25,955][105692] Updated weights for policy 0, policy_version 522925 (0.0009) [2023-12-26 19:10:26,003][105692] Updated weights for policy 0, policy_version 522935 (0.0005) [2023-12-26 19:10:26,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 267968512. Throughput: 0: 9242.2, 1: 10025.0. Samples: 267975716. Policy #0 lag: (min: 36.0, avg: 47.6, max: 48.0) [2023-12-26 19:10:26,062][104569] Avg episode reward: [(0, '9176.286'), (1, '9176.326')] [2023-12-26 19:10:26,144][105620] Updated weights for policy 1, policy_version 523696 (0.0006) [2023-12-26 19:10:26,200][105620] Updated weights for policy 1, policy_version 523706 (0.0007) [2023-12-26 19:10:26,252][105620] Updated weights for policy 1, policy_version 523716 (0.0008) [2023-12-26 19:10:26,763][105692] Updated weights for policy 0, policy_version 522945 (0.0010) [2023-12-26 19:10:26,811][105692] Updated weights for policy 0, policy_version 522955 (0.0009) [2023-12-26 19:10:26,857][105692] Updated weights for policy 0, policy_version 522965 (0.0009) [2023-12-26 19:10:26,904][105692] Updated weights for policy 0, policy_version 522975 (0.0007) [2023-12-26 19:10:26,923][105620] Updated weights for policy 1, policy_version 523726 (0.0008) [2023-12-26 19:10:26,988][105620] Updated weights for policy 1, policy_version 523736 (0.0005) [2023-12-26 19:10:27,051][105620] Updated weights for policy 1, policy_version 523746 (0.0008) [2023-12-26 19:10:27,566][105692] Updated weights for policy 0, policy_version 522985 (0.0007) [2023-12-26 19:10:27,611][105692] Updated weights for policy 0, policy_version 522995 (0.0005) [2023-12-26 19:10:27,658][105692] Updated weights for policy 0, policy_version 523005 (0.0007) [2023-12-26 19:10:27,831][105620] Updated weights for policy 1, policy_version 523756 (0.0009) [2023-12-26 19:10:27,884][105620] Updated weights for policy 1, policy_version 523766 (0.0006) [2023-12-26 19:10:27,929][105620] Updated weights for policy 1, policy_version 523776 (0.0005) [2023-12-26 19:10:28,240][105692] Updated weights for policy 0, policy_version 523015 (0.0010) [2023-12-26 19:10:28,295][105692] Updated weights for policy 0, policy_version 523027 (0.0011) [2023-12-26 19:10:28,355][105692] Updated weights for policy 0, policy_version 523037 (0.0009) [2023-12-26 19:10:28,605][105620] Updated weights for policy 1, policy_version 523786 (0.0006) [2023-12-26 19:10:28,653][105620] Updated weights for policy 1, policy_version 523796 (0.0009) [2023-12-26 19:10:28,705][105620] Updated weights for policy 1, policy_version 523806 (0.0009) [2023-12-26 19:10:28,755][105620] Updated weights for policy 1, policy_version 523816 (0.0009) [2023-12-26 19:10:29,071][105692] Updated weights for policy 0, policy_version 523047 (0.0009) [2023-12-26 19:10:29,125][105692] Updated weights for policy 0, policy_version 523057 (0.0009) [2023-12-26 19:10:29,182][105692] Updated weights for policy 0, policy_version 523067 (0.0009) [2023-12-26 19:10:29,565][105620] Updated weights for policy 1, policy_version 523826 (0.0009) [2023-12-26 19:10:29,626][105620] Updated weights for policy 1, policy_version 523836 (0.0009) [2023-12-26 19:10:29,687][105620] Updated weights for policy 1, policy_version 523846 (0.0008) [2023-12-26 19:10:29,898][105692] Updated weights for policy 0, policy_version 523077 (0.0009) [2023-12-26 19:10:29,958][105692] Updated weights for policy 0, policy_version 523087 (0.0010) [2023-12-26 19:10:30,012][105692] Updated weights for policy 0, policy_version 523097 (0.0009) [2023-12-26 19:10:30,452][105620] Updated weights for policy 1, policy_version 523856 (0.0008) [2023-12-26 19:10:30,515][105620] Updated weights for policy 1, policy_version 523866 (0.0008) [2023-12-26 19:10:30,565][105620] Updated weights for policy 1, policy_version 523876 (0.0009) [2023-12-26 19:10:30,778][105692] Updated weights for policy 0, policy_version 523107 (0.0009) [2023-12-26 19:10:30,825][105692] Updated weights for policy 0, policy_version 523117 (0.0009) [2023-12-26 19:10:30,885][105692] Updated weights for policy 0, policy_version 523127 (0.0008) [2023-12-26 19:10:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 268066816. Throughput: 0: 9290.4, 1: 10054.0. Samples: 268035556. Policy #0 lag: (min: 36.0, avg: 47.6, max: 48.0) [2023-12-26 19:10:31,062][104569] Avg episode reward: [(0, '9358.869'), (1, '9176.628')] [2023-12-26 19:10:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000523136_133939200.pth... [2023-12-26 19:10:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000523880_134127616.pth... [2023-12-26 19:10:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000522728_133832704.pth [2023-12-26 19:10:31,088][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000522048_133660672.pth [2023-12-26 19:10:31,243][105620] Updated weights for policy 1, policy_version 523886 (0.0007) [2023-12-26 19:10:31,314][105620] Updated weights for policy 1, policy_version 523896 (0.0010) [2023-12-26 19:10:31,380][105620] Updated weights for policy 1, policy_version 523906 (0.0009) [2023-12-26 19:10:31,634][105692] Updated weights for policy 0, policy_version 523137 (0.0008) [2023-12-26 19:10:31,687][105585] KL-divergence is very high: 198.8870 [2023-12-26 19:10:31,692][105692] Updated weights for policy 0, policy_version 523147 (0.0009) [2023-12-26 19:10:31,739][105585] KL-divergence is very high: 340.8672 [2023-12-26 19:10:31,759][105692] Updated weights for policy 0, policy_version 523157 (0.0009) [2023-12-26 19:10:31,791][105585] KL-divergence is very high: 232.7643 [2023-12-26 19:10:31,821][105692] Updated weights for policy 0, policy_version 523167 (0.0008) [2023-12-26 19:10:32,181][105620] Updated weights for policy 1, policy_version 523916 (0.0009) [2023-12-26 19:10:32,232][105620] Updated weights for policy 1, policy_version 523926 (0.0009) [2023-12-26 19:10:32,294][105620] Updated weights for policy 1, policy_version 523936 (0.0009) [2023-12-26 19:10:32,477][105692] Updated weights for policy 0, policy_version 523177 (0.0005) [2023-12-26 19:10:32,522][105692] Updated weights for policy 0, policy_version 523187 (0.0005) [2023-12-26 19:10:32,573][105692] Updated weights for policy 0, policy_version 523197 (0.0005) [2023-12-26 19:10:33,109][105620] Updated weights for policy 1, policy_version 523946 (0.0009) [2023-12-26 19:10:33,112][105692] Updated weights for policy 0, policy_version 523207 (0.0006) [2023-12-26 19:10:33,165][105620] Updated weights for policy 1, policy_version 523956 (0.0008) [2023-12-26 19:10:33,167][105692] Updated weights for policy 0, policy_version 523217 (0.0007) [2023-12-26 19:10:33,219][105692] Updated weights for policy 0, policy_version 523227 (0.0006) [2023-12-26 19:10:33,225][105620] Updated weights for policy 1, policy_version 523966 (0.0008) [2023-12-26 19:10:33,284][105620] Updated weights for policy 1, policy_version 523976 (0.0006) [2023-12-26 19:10:33,888][105620] Updated weights for policy 1, policy_version 523986 (0.0009) [2023-12-26 19:10:33,935][105620] Updated weights for policy 1, policy_version 523996 (0.0008) [2023-12-26 19:10:33,981][105620] Updated weights for policy 1, policy_version 524006 (0.0008) [2023-12-26 19:10:34,008][105692] Updated weights for policy 0, policy_version 523237 (0.0007) [2023-12-26 19:10:34,070][105692] Updated weights for policy 0, policy_version 523247 (0.0008) [2023-12-26 19:10:34,132][105692] Updated weights for policy 0, policy_version 523257 (0.0009) [2023-12-26 19:10:34,758][105620] Updated weights for policy 1, policy_version 524016 (0.0009) [2023-12-26 19:10:34,810][105620] Updated weights for policy 1, policy_version 524026 (0.0009) [2023-12-26 19:10:34,862][105620] Updated weights for policy 1, policy_version 524036 (0.0009) [2023-12-26 19:10:34,907][105692] Updated weights for policy 0, policy_version 523267 (0.0007) [2023-12-26 19:10:34,960][105692] Updated weights for policy 0, policy_version 523277 (0.0009) [2023-12-26 19:10:35,020][105692] Updated weights for policy 0, policy_version 523287 (0.0009) [2023-12-26 19:10:35,605][105620] Updated weights for policy 1, policy_version 524046 (0.0007) [2023-12-26 19:10:35,656][105620] Updated weights for policy 1, policy_version 524056 (0.0005) [2023-12-26 19:10:35,709][105620] Updated weights for policy 1, policy_version 524066 (0.0005) [2023-12-26 19:10:35,819][105692] Updated weights for policy 0, policy_version 523297 (0.0009) [2023-12-26 19:10:35,867][105692] Updated weights for policy 0, policy_version 523307 (0.0009) [2023-12-26 19:10:35,923][105692] Updated weights for policy 0, policy_version 523317 (0.0008) [2023-12-26 19:10:35,984][105692] Updated weights for policy 0, policy_version 523327 (0.0008) [2023-12-26 19:10:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 268165120. Throughput: 0: 9289.4, 1: 10022.8. Samples: 268151152. Policy #0 lag: (min: 36.0, avg: 47.6, max: 48.0) [2023-12-26 19:10:36,063][104569] Avg episode reward: [(0, '9265.651'), (1, '9174.720')] [2023-12-26 19:10:36,319][105620] Updated weights for policy 1, policy_version 524076 (0.0007) [2023-12-26 19:10:36,376][105620] Updated weights for policy 1, policy_version 524086 (0.0008) [2023-12-26 19:10:36,433][105620] Updated weights for policy 1, policy_version 524096 (0.0008) [2023-12-26 19:10:36,821][105692] Updated weights for policy 0, policy_version 523337 (0.0009) [2023-12-26 19:10:36,880][105692] Updated weights for policy 0, policy_version 523347 (0.0009) [2023-12-26 19:10:36,941][105692] Updated weights for policy 0, policy_version 523357 (0.0009) [2023-12-26 19:10:37,136][105620] Updated weights for policy 1, policy_version 524106 (0.0008) [2023-12-26 19:10:37,195][105620] Updated weights for policy 1, policy_version 524116 (0.0009) [2023-12-26 19:10:37,243][105620] Updated weights for policy 1, policy_version 524126 (0.0009) [2023-12-26 19:10:37,298][105620] Updated weights for policy 1, policy_version 524136 (0.0009) [2023-12-26 19:10:37,697][105692] Updated weights for policy 0, policy_version 523367 (0.0010) [2023-12-26 19:10:37,756][105692] Updated weights for policy 0, policy_version 523377 (0.0008) [2023-12-26 19:10:37,813][105692] Updated weights for policy 0, policy_version 523387 (0.0009) [2023-12-26 19:10:38,016][105620] Updated weights for policy 1, policy_version 524146 (0.0006) [2023-12-26 19:10:38,073][105620] Updated weights for policy 1, policy_version 524156 (0.0009) [2023-12-26 19:10:38,136][105620] Updated weights for policy 1, policy_version 524166 (0.0009) [2023-12-26 19:10:38,615][105692] Updated weights for policy 0, policy_version 523398 (0.0009) [2023-12-26 19:10:38,672][105692] Updated weights for policy 0, policy_version 523408 (0.0009) [2023-12-26 19:10:38,733][105692] Updated weights for policy 0, policy_version 523418 (0.0008) [2023-12-26 19:10:38,862][105620] Updated weights for policy 1, policy_version 524176 (0.0009) [2023-12-26 19:10:38,918][105620] Updated weights for policy 1, policy_version 524186 (0.0009) [2023-12-26 19:10:38,965][105620] Updated weights for policy 1, policy_version 524196 (0.0008) [2023-12-26 19:10:39,528][105692] Updated weights for policy 0, policy_version 523428 (0.0009) [2023-12-26 19:10:39,584][105692] Updated weights for policy 0, policy_version 523438 (0.0008) [2023-12-26 19:10:39,637][105692] Updated weights for policy 0, policy_version 523448 (0.0008) [2023-12-26 19:10:39,766][105620] Updated weights for policy 1, policy_version 524206 (0.0010) [2023-12-26 19:10:39,830][105620] Updated weights for policy 1, policy_version 524216 (0.0011) [2023-12-26 19:10:39,897][105620] Updated weights for policy 1, policy_version 524226 (0.0011) [2023-12-26 19:10:40,428][105692] Updated weights for policy 0, policy_version 523458 (0.0009) [2023-12-26 19:10:40,488][105692] Updated weights for policy 0, policy_version 523468 (0.0009) [2023-12-26 19:10:40,533][105692] Updated weights for policy 0, policy_version 523478 (0.0008) [2023-12-26 19:10:40,582][105692] Updated weights for policy 0, policy_version 523488 (0.0008) [2023-12-26 19:10:40,606][105620] Updated weights for policy 1, policy_version 524236 (0.0009) [2023-12-26 19:10:40,676][105620] Updated weights for policy 1, policy_version 524246 (0.0005) [2023-12-26 19:10:40,746][105620] Updated weights for policy 1, policy_version 524256 (0.0006) [2023-12-26 19:10:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 268255232. Throughput: 0: 9209.4, 1: 10019.3. Samples: 268264056. Policy #0 lag: (min: 36.0, avg: 47.6, max: 48.0) [2023-12-26 19:10:41,063][104569] Avg episode reward: [(0, '9357.682'), (1, '9172.697')] [2023-12-26 19:10:41,291][105692] Updated weights for policy 0, policy_version 523498 (0.0009) [2023-12-26 19:10:41,356][105692] Updated weights for policy 0, policy_version 523508 (0.0008) [2023-12-26 19:10:41,418][105620] Updated weights for policy 1, policy_version 524266 (0.0009) [2023-12-26 19:10:41,424][105692] Updated weights for policy 0, policy_version 523518 (0.0007) [2023-12-26 19:10:41,474][105620] Updated weights for policy 1, policy_version 524276 (0.0009) [2023-12-26 19:10:41,542][105620] Updated weights for policy 1, policy_version 524286 (0.0007) [2023-12-26 19:10:41,616][105620] Updated weights for policy 1, policy_version 524296 (0.0009) [2023-12-26 19:10:42,159][105692] Updated weights for policy 0, policy_version 523528 (0.0008) [2023-12-26 19:10:42,219][105692] Updated weights for policy 0, policy_version 523538 (0.0008) [2023-12-26 19:10:42,269][105620] Updated weights for policy 1, policy_version 524306 (0.0010) [2023-12-26 19:10:42,287][105692] Updated weights for policy 0, policy_version 523548 (0.0007) [2023-12-26 19:10:42,335][105620] Updated weights for policy 1, policy_version 524316 (0.0008) [2023-12-26 19:10:42,402][105620] Updated weights for policy 1, policy_version 524326 (0.0009) [2023-12-26 19:10:42,959][105692] Updated weights for policy 0, policy_version 523558 (0.0008) [2023-12-26 19:10:43,014][105692] Updated weights for policy 0, policy_version 523568 (0.0009) [2023-12-26 19:10:43,069][105620] Updated weights for policy 1, policy_version 524336 (0.0010) [2023-12-26 19:10:43,070][105692] Updated weights for policy 0, policy_version 523578 (0.0007) [2023-12-26 19:10:43,117][105620] Updated weights for policy 1, policy_version 524346 (0.0010) [2023-12-26 19:10:43,172][105620] Updated weights for policy 1, policy_version 524356 (0.0010) [2023-12-26 19:10:43,745][105692] Updated weights for policy 0, policy_version 523588 (0.0007) [2023-12-26 19:10:43,759][105620] Updated weights for policy 1, policy_version 524366 (0.0010) [2023-12-26 19:10:43,793][105692] Updated weights for policy 0, policy_version 523598 (0.0005) [2023-12-26 19:10:43,818][105620] Updated weights for policy 1, policy_version 524376 (0.0010) [2023-12-26 19:10:43,848][105692] Updated weights for policy 0, policy_version 523608 (0.0006) [2023-12-26 19:10:43,877][105620] Updated weights for policy 1, policy_version 524386 (0.0011) [2023-12-26 19:10:44,491][105620] Updated weights for policy 1, policy_version 524396 (0.0009) [2023-12-26 19:10:44,530][105692] Updated weights for policy 0, policy_version 523618 (0.0006) [2023-12-26 19:10:44,553][105620] Updated weights for policy 1, policy_version 524406 (0.0006) [2023-12-26 19:10:44,584][105692] Updated weights for policy 0, policy_version 523628 (0.0008) [2023-12-26 19:10:44,612][105620] Updated weights for policy 1, policy_version 524416 (0.0010) [2023-12-26 19:10:44,638][105692] Updated weights for policy 0, policy_version 523638 (0.0006) [2023-12-26 19:10:44,691][105692] Updated weights for policy 0, policy_version 523648 (0.0007) [2023-12-26 19:10:45,368][105620] Updated weights for policy 1, policy_version 524426 (0.0010) [2023-12-26 19:10:45,388][105692] Updated weights for policy 0, policy_version 523658 (0.0011) [2023-12-26 19:10:45,429][105620] Updated weights for policy 1, policy_version 524436 (0.0011) [2023-12-26 19:10:45,445][105692] Updated weights for policy 0, policy_version 523668 (0.0011) [2023-12-26 19:10:45,486][105620] Updated weights for policy 1, policy_version 524446 (0.0011) [2023-12-26 19:10:45,502][105692] Updated weights for policy 0, policy_version 523678 (0.0011) [2023-12-26 19:10:45,543][105620] Updated weights for policy 1, policy_version 524456 (0.0010) [2023-12-26 19:10:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 268353536. Throughput: 0: 9313.3, 1: 9999.7. Samples: 268325420. Policy #0 lag: (min: 36.0, avg: 47.6, max: 48.0) [2023-12-26 19:10:46,063][104569] Avg episode reward: [(0, '9177.662'), (1, '9171.883')] [2023-12-26 19:10:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000523680_134078464.pth... [2023-12-26 19:10:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000524456_134275072.pth... [2023-12-26 19:10:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000522560_133791744.pth [2023-12-26 19:10:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000523304_133980160.pth [2023-12-26 19:10:46,233][105692] Updated weights for policy 0, policy_version 523688 (0.0007) [2023-12-26 19:10:46,286][105692] Updated weights for policy 0, policy_version 523698 (0.0011) [2023-12-26 19:10:46,313][105620] Updated weights for policy 1, policy_version 524466 (0.0011) [2023-12-26 19:10:46,342][105692] Updated weights for policy 0, policy_version 523708 (0.0011) [2023-12-26 19:10:46,362][105620] Updated weights for policy 1, policy_version 524476 (0.0006) [2023-12-26 19:10:46,409][105620] Updated weights for policy 1, policy_version 524486 (0.0005) [2023-12-26 19:10:46,905][105692] Updated weights for policy 0, policy_version 523718 (0.0007) [2023-12-26 19:10:46,951][105692] Updated weights for policy 0, policy_version 523728 (0.0005) [2023-12-26 19:10:47,000][105692] Updated weights for policy 0, policy_version 523738 (0.0005) [2023-12-26 19:10:47,067][105620] Updated weights for policy 1, policy_version 524496 (0.0008) [2023-12-26 19:10:47,107][105586] KL-divergence is very high: 105.9485 [2023-12-26 19:10:47,118][105620] Updated weights for policy 1, policy_version 524506 (0.0007) [2023-12-26 19:10:47,167][105620] Updated weights for policy 1, policy_version 524516 (0.0008) [2023-12-26 19:10:47,635][105692] Updated weights for policy 0, policy_version 523748 (0.0009) [2023-12-26 19:10:47,703][105692] Updated weights for policy 0, policy_version 523758 (0.0010) [2023-12-26 19:10:47,764][105692] Updated weights for policy 0, policy_version 523768 (0.0010) [2023-12-26 19:10:47,786][105620] Updated weights for policy 1, policy_version 524526 (0.0007) [2023-12-26 19:10:47,835][105620] Updated weights for policy 1, policy_version 524536 (0.0007) [2023-12-26 19:10:47,895][105620] Updated weights for policy 1, policy_version 524546 (0.0008) [2023-12-26 19:10:48,336][105692] Updated weights for policy 0, policy_version 523778 (0.0010) [2023-12-26 19:10:48,401][105692] Updated weights for policy 0, policy_version 523788 (0.0008) [2023-12-26 19:10:48,454][105692] Updated weights for policy 0, policy_version 523798 (0.0005) [2023-12-26 19:10:48,506][105692] Updated weights for policy 0, policy_version 523808 (0.0006) [2023-12-26 19:10:48,719][105620] Updated weights for policy 1, policy_version 524556 (0.0009) [2023-12-26 19:10:48,788][105620] Updated weights for policy 1, policy_version 524566 (0.0008) [2023-12-26 19:10:48,853][105620] Updated weights for policy 1, policy_version 524576 (0.0005) [2023-12-26 19:10:49,140][105692] Updated weights for policy 0, policy_version 523818 (0.0006) [2023-12-26 19:10:49,198][105692] Updated weights for policy 0, policy_version 523828 (0.0007) [2023-12-26 19:10:49,266][105692] Updated weights for policy 0, policy_version 523838 (0.0008) [2023-12-26 19:10:49,538][105620] Updated weights for policy 1, policy_version 524586 (0.0006) [2023-12-26 19:10:49,597][105620] Updated weights for policy 1, policy_version 524596 (0.0010) [2023-12-26 19:10:49,652][105620] Updated weights for policy 1, policy_version 524606 (0.0010) [2023-12-26 19:10:49,699][105620] Updated weights for policy 1, policy_version 524616 (0.0007) [2023-12-26 19:10:49,893][105692] Updated weights for policy 0, policy_version 523848 (0.0009) [2023-12-26 19:10:49,959][105692] Updated weights for policy 0, policy_version 523858 (0.0008) [2023-12-26 19:10:50,011][105692] Updated weights for policy 0, policy_version 523868 (0.0010) [2023-12-26 19:10:50,401][105620] Updated weights for policy 1, policy_version 524626 (0.0009) [2023-12-26 19:10:50,471][105620] Updated weights for policy 1, policy_version 524636 (0.0008) [2023-12-26 19:10:50,541][105620] Updated weights for policy 1, policy_version 524646 (0.0008) [2023-12-26 19:10:50,716][105692] Updated weights for policy 0, policy_version 523878 (0.0009) [2023-12-26 19:10:50,779][105692] Updated weights for policy 0, policy_version 523888 (0.0009) [2023-12-26 19:10:50,838][105692] Updated weights for policy 0, policy_version 523898 (0.0007) [2023-12-26 19:10:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 268460032. Throughput: 0: 9527.5, 1: 9891.1. Samples: 268447916. Policy #0 lag: (min: 36.0, avg: 47.6, max: 48.0) [2023-12-26 19:10:51,062][104569] Avg episode reward: [(0, '9177.318'), (1, '9262.162')] [2023-12-26 19:10:51,135][105620] Updated weights for policy 1, policy_version 524656 (0.0009) [2023-12-26 19:10:51,200][105620] Updated weights for policy 1, policy_version 524666 (0.0009) [2023-12-26 19:10:51,261][105620] Updated weights for policy 1, policy_version 524676 (0.0009) [2023-12-26 19:10:51,581][105692] Updated weights for policy 0, policy_version 523908 (0.0007) [2023-12-26 19:10:51,644][105692] Updated weights for policy 0, policy_version 523918 (0.0008) [2023-12-26 19:10:51,692][105692] Updated weights for policy 0, policy_version 523928 (0.0006) [2023-12-26 19:10:52,032][105620] Updated weights for policy 1, policy_version 524686 (0.0006) [2023-12-26 19:10:52,078][105620] Updated weights for policy 1, policy_version 524696 (0.0009) [2023-12-26 19:10:52,125][105620] Updated weights for policy 1, policy_version 524706 (0.0009) [2023-12-26 19:10:52,461][105692] Updated weights for policy 0, policy_version 523938 (0.0008) [2023-12-26 19:10:52,516][105692] Updated weights for policy 0, policy_version 523948 (0.0009) [2023-12-26 19:10:52,583][105692] Updated weights for policy 0, policy_version 523958 (0.0008) [2023-12-26 19:10:52,651][105692] Updated weights for policy 0, policy_version 523968 (0.0010) [2023-12-26 19:10:52,856][105620] Updated weights for policy 1, policy_version 524716 (0.0008) [2023-12-26 19:10:52,903][105620] Updated weights for policy 1, policy_version 524726 (0.0009) [2023-12-26 19:10:52,950][105620] Updated weights for policy 1, policy_version 524736 (0.0008) [2023-12-26 19:10:53,357][105692] Updated weights for policy 0, policy_version 523978 (0.0009) [2023-12-26 19:10:53,411][105692] Updated weights for policy 0, policy_version 523988 (0.0009) [2023-12-26 19:10:53,468][105692] Updated weights for policy 0, policy_version 523998 (0.0009) [2023-12-26 19:10:53,764][105620] Updated weights for policy 1, policy_version 524746 (0.0008) [2023-12-26 19:10:53,818][105620] Updated weights for policy 1, policy_version 524756 (0.0009) [2023-12-26 19:10:53,865][105620] Updated weights for policy 1, policy_version 524766 (0.0009) [2023-12-26 19:10:53,921][105620] Updated weights for policy 1, policy_version 524776 (0.0007) [2023-12-26 19:10:54,208][105692] Updated weights for policy 0, policy_version 524008 (0.0009) [2023-12-26 19:10:54,263][105692] Updated weights for policy 0, policy_version 524018 (0.0005) [2023-12-26 19:10:54,318][105692] Updated weights for policy 0, policy_version 524028 (0.0008) [2023-12-26 19:10:54,676][105620] Updated weights for policy 1, policy_version 524786 (0.0009) [2023-12-26 19:10:54,704][105586] KL-divergence is very high: 149.8838 [2023-12-26 19:10:54,726][105620] Updated weights for policy 1, policy_version 524796 (0.0009) [2023-12-26 19:10:54,746][105586] KL-divergence is very high: 154.5965 [2023-12-26 19:10:54,787][105620] Updated weights for policy 1, policy_version 524806 (0.0009) [2023-12-26 19:10:54,995][105692] Updated weights for policy 0, policy_version 524038 (0.0009) [2023-12-26 19:10:55,043][105692] Updated weights for policy 0, policy_version 524048 (0.0009) [2023-12-26 19:10:55,094][105692] Updated weights for policy 0, policy_version 524058 (0.0009) [2023-12-26 19:10:55,569][105620] Updated weights for policy 1, policy_version 524816 (0.0008) [2023-12-26 19:10:55,619][105620] Updated weights for policy 1, policy_version 524826 (0.0009) [2023-12-26 19:10:55,672][105620] Updated weights for policy 1, policy_version 524836 (0.0009) [2023-12-26 19:10:55,816][105692] Updated weights for policy 0, policy_version 524068 (0.0009) [2023-12-26 19:10:55,878][105692] Updated weights for policy 0, policy_version 524078 (0.0007) [2023-12-26 19:10:55,931][105692] Updated weights for policy 0, policy_version 524088 (0.0005) [2023-12-26 19:10:56,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 268558336. Throughput: 0: 9593.9, 1: 9805.4. Samples: 268562376. Policy #0 lag: (min: 36.0, avg: 47.6, max: 48.0) [2023-12-26 19:10:56,062][104569] Avg episode reward: [(0, '9267.463'), (1, '9262.030')] [2023-12-26 19:10:56,498][105620] Updated weights for policy 1, policy_version 524847 (0.0010) [2023-12-26 19:10:56,529][105692] Updated weights for policy 0, policy_version 524098 (0.0005) [2023-12-26 19:10:56,548][105620] Updated weights for policy 1, policy_version 524857 (0.0010) [2023-12-26 19:10:56,581][105692] Updated weights for policy 0, policy_version 524108 (0.0005) [2023-12-26 19:10:56,600][105620] Updated weights for policy 1, policy_version 524867 (0.0009) [2023-12-26 19:10:56,633][105692] Updated weights for policy 0, policy_version 524118 (0.0006) [2023-12-26 19:10:56,691][105692] Updated weights for policy 0, policy_version 524128 (0.0008) [2023-12-26 19:10:57,346][105692] Updated weights for policy 0, policy_version 524138 (0.0008) [2023-12-26 19:10:57,405][105620] Updated weights for policy 1, policy_version 524877 (0.0007) [2023-12-26 19:10:57,409][105692] Updated weights for policy 0, policy_version 524148 (0.0010) [2023-12-26 19:10:57,461][105620] Updated weights for policy 1, policy_version 524887 (0.0007) [2023-12-26 19:10:57,471][105692] Updated weights for policy 0, policy_version 524158 (0.0010) [2023-12-26 19:10:57,516][105620] Updated weights for policy 1, policy_version 524897 (0.0006) [2023-12-26 19:10:58,163][105692] Updated weights for policy 0, policy_version 524168 (0.0008) [2023-12-26 19:10:58,218][105692] Updated weights for policy 0, policy_version 524178 (0.0007) [2023-12-26 19:10:58,256][105620] Updated weights for policy 1, policy_version 524907 (0.0009) [2023-12-26 19:10:58,281][105692] Updated weights for policy 0, policy_version 524188 (0.0009) [2023-12-26 19:10:58,326][105620] Updated weights for policy 1, policy_version 524917 (0.0008) [2023-12-26 19:10:58,391][105620] Updated weights for policy 1, policy_version 524927 (0.0009) [2023-12-26 19:10:59,059][105692] Updated weights for policy 0, policy_version 524198 (0.0007) [2023-12-26 19:10:59,105][105692] Updated weights for policy 0, policy_version 524208 (0.0008) [2023-12-26 19:10:59,107][105620] Updated weights for policy 1, policy_version 524937 (0.0008) [2023-12-26 19:10:59,155][105620] Updated weights for policy 1, policy_version 524947 (0.0006) [2023-12-26 19:10:59,165][105692] Updated weights for policy 0, policy_version 524218 (0.0008) [2023-12-26 19:10:59,207][105620] Updated weights for policy 1, policy_version 524957 (0.0006) [2023-12-26 19:10:59,268][105620] Updated weights for policy 1, policy_version 524967 (0.0010) [2023-12-26 19:10:59,986][105620] Updated weights for policy 1, policy_version 524977 (0.0009) [2023-12-26 19:11:00,012][105692] Updated weights for policy 0, policy_version 524228 (0.0010) [2023-12-26 19:11:00,043][105620] Updated weights for policy 1, policy_version 524987 (0.0007) [2023-12-26 19:11:00,069][105692] Updated weights for policy 0, policy_version 524238 (0.0006) [2023-12-26 19:11:00,100][105620] Updated weights for policy 1, policy_version 524997 (0.0006) [2023-12-26 19:11:00,127][105692] Updated weights for policy 0, policy_version 524248 (0.0007) [2023-12-26 19:11:00,839][105620] Updated weights for policy 1, policy_version 525007 (0.0008) [2023-12-26 19:11:00,886][105692] Updated weights for policy 0, policy_version 524258 (0.0008) [2023-12-26 19:11:00,887][105620] Updated weights for policy 1, policy_version 525017 (0.0009) [2023-12-26 19:11:00,939][105620] Updated weights for policy 1, policy_version 525027 (0.0006) [2023-12-26 19:11:00,941][105692] Updated weights for policy 0, policy_version 524268 (0.0006) [2023-12-26 19:11:01,002][105692] Updated weights for policy 0, policy_version 524278 (0.0008) [2023-12-26 19:11:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 268648448. Throughput: 0: 9706.0, 1: 9755.6. Samples: 268621124. Policy #0 lag: (min: 36.0, avg: 47.6, max: 48.0) [2023-12-26 19:11:01,063][104569] Avg episode reward: [(0, '9267.697'), (1, '9172.004')] [2023-12-26 19:11:01,068][105692] Updated weights for policy 0, policy_version 524288 (0.0010) [2023-12-26 19:11:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000525032_134422528.pth... [2023-12-26 19:11:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000524288_134234112.pth... [2023-12-26 19:11:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000523880_134127616.pth [2023-12-26 19:11:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000523136_133939200.pth [2023-12-26 19:11:01,808][105620] Updated weights for policy 1, policy_version 525037 (0.0007) [2023-12-26 19:11:01,827][105692] Updated weights for policy 0, policy_version 524298 (0.0005) [2023-12-26 19:11:01,863][105620] Updated weights for policy 1, policy_version 525047 (0.0008) [2023-12-26 19:11:01,880][105692] Updated weights for policy 0, policy_version 524308 (0.0005) [2023-12-26 19:11:01,915][105620] Updated weights for policy 1, policy_version 525057 (0.0008) [2023-12-26 19:11:01,934][105692] Updated weights for policy 0, policy_version 524318 (0.0005) [2023-12-26 19:11:02,606][105692] Updated weights for policy 0, policy_version 524328 (0.0005) [2023-12-26 19:11:02,666][105692] Updated weights for policy 0, policy_version 524338 (0.0005) [2023-12-26 19:11:02,720][105692] Updated weights for policy 0, policy_version 524348 (0.0008) [2023-12-26 19:11:02,740][105620] Updated weights for policy 1, policy_version 525067 (0.0008) [2023-12-26 19:11:02,795][105620] Updated weights for policy 1, policy_version 525077 (0.0010) [2023-12-26 19:11:02,855][105620] Updated weights for policy 1, policy_version 525087 (0.0008) [2023-12-26 19:11:03,360][105692] Updated weights for policy 0, policy_version 524358 (0.0006) [2023-12-26 19:11:03,425][105692] Updated weights for policy 0, policy_version 524368 (0.0006) [2023-12-26 19:11:03,488][105692] Updated weights for policy 0, policy_version 524378 (0.0007) [2023-12-26 19:11:03,656][105620] Updated weights for policy 1, policy_version 525097 (0.0008) [2023-12-26 19:11:03,702][105620] Updated weights for policy 1, policy_version 525107 (0.0009) [2023-12-26 19:11:03,749][105620] Updated weights for policy 1, policy_version 525117 (0.0009) [2023-12-26 19:11:03,810][105620] Updated weights for policy 1, policy_version 525127 (0.0008) [2023-12-26 19:11:04,169][105692] Updated weights for policy 0, policy_version 524388 (0.0009) [2023-12-26 19:11:04,229][105692] Updated weights for policy 0, policy_version 524398 (0.0009) [2023-12-26 19:11:04,285][105692] Updated weights for policy 0, policy_version 524408 (0.0009) [2023-12-26 19:11:04,606][105620] Updated weights for policy 1, policy_version 525137 (0.0009) [2023-12-26 19:11:04,657][105620] Updated weights for policy 1, policy_version 525147 (0.0009) [2023-12-26 19:11:04,704][105620] Updated weights for policy 1, policy_version 525157 (0.0008) [2023-12-26 19:11:05,051][105692] Updated weights for policy 0, policy_version 524418 (0.0009) [2023-12-26 19:11:05,106][105692] Updated weights for policy 0, policy_version 524428 (0.0009) [2023-12-26 19:11:05,159][105692] Updated weights for policy 0, policy_version 524438 (0.0008) [2023-12-26 19:11:05,220][105692] Updated weights for policy 0, policy_version 524448 (0.0010) [2023-12-26 19:11:05,477][105620] Updated weights for policy 1, policy_version 525167 (0.0010) [2023-12-26 19:11:05,538][105620] Updated weights for policy 1, policy_version 525177 (0.0010) [2023-12-26 19:11:05,590][105620] Updated weights for policy 1, policy_version 525187 (0.0009) [2023-12-26 19:11:05,821][105692] Updated weights for policy 0, policy_version 524458 (0.0007) [2023-12-26 19:11:05,876][105692] Updated weights for policy 0, policy_version 524468 (0.0005) [2023-12-26 19:11:05,938][105692] Updated weights for policy 0, policy_version 524478 (0.0007) [2023-12-26 19:11:06,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 268746752. Throughput: 0: 9785.9, 1: 9651.5. Samples: 268733424. Policy #0 lag: (min: 36.0, avg: 47.6, max: 48.0) [2023-12-26 19:11:06,062][104569] Avg episode reward: [(0, '9175.568'), (1, '9171.995')] [2023-12-26 19:11:06,459][105620] Updated weights for policy 1, policy_version 525197 (0.0009) [2023-12-26 19:11:06,514][105620] Updated weights for policy 1, policy_version 525207 (0.0008) [2023-12-26 19:11:06,539][105692] Updated weights for policy 0, policy_version 524488 (0.0007) [2023-12-26 19:11:06,578][105620] Updated weights for policy 1, policy_version 525217 (0.0008) [2023-12-26 19:11:06,601][105692] Updated weights for policy 0, policy_version 524498 (0.0008) [2023-12-26 19:11:06,664][105692] Updated weights for policy 0, policy_version 524508 (0.0007) [2023-12-26 19:11:07,283][105692] Updated weights for policy 0, policy_version 524518 (0.0005) [2023-12-26 19:11:07,319][105620] Updated weights for policy 1, policy_version 525227 (0.0007) [2023-12-26 19:11:07,344][105692] Updated weights for policy 0, policy_version 524528 (0.0006) [2023-12-26 19:11:07,382][105620] Updated weights for policy 1, policy_version 525237 (0.0009) [2023-12-26 19:11:07,405][105692] Updated weights for policy 0, policy_version 524538 (0.0008) [2023-12-26 19:11:07,438][105620] Updated weights for policy 1, policy_version 525247 (0.0007) [2023-12-26 19:11:07,987][105692] Updated weights for policy 0, policy_version 524548 (0.0007) [2023-12-26 19:11:08,052][105692] Updated weights for policy 0, policy_version 524558 (0.0005) [2023-12-26 19:11:08,118][105692] Updated weights for policy 0, policy_version 524568 (0.0006) [2023-12-26 19:11:08,266][105620] Updated weights for policy 1, policy_version 525257 (0.0010) [2023-12-26 19:11:08,319][105620] Updated weights for policy 1, policy_version 525267 (0.0011) [2023-12-26 19:11:08,376][105620] Updated weights for policy 1, policy_version 525277 (0.0010) [2023-12-26 19:11:08,425][105620] Updated weights for policy 1, policy_version 525287 (0.0010) [2023-12-26 19:11:08,770][105692] Updated weights for policy 0, policy_version 524578 (0.0007) [2023-12-26 19:11:08,827][105692] Updated weights for policy 0, policy_version 524588 (0.0010) [2023-12-26 19:11:08,892][105692] Updated weights for policy 0, policy_version 524598 (0.0010) [2023-12-26 19:11:08,953][105692] Updated weights for policy 0, policy_version 524608 (0.0011) [2023-12-26 19:11:09,204][105620] Updated weights for policy 1, policy_version 525297 (0.0011) [2023-12-26 19:11:09,274][105620] Updated weights for policy 1, policy_version 525307 (0.0011) [2023-12-26 19:11:09,331][105620] Updated weights for policy 1, policy_version 525317 (0.0011) [2023-12-26 19:11:09,590][105585] KL-divergence is very high: 143.6102 [2023-12-26 19:11:09,594][105692] Updated weights for policy 0, policy_version 524618 (0.0008) [2023-12-26 19:11:09,640][105585] KL-divergence is very high: 210.6858 [2023-12-26 19:11:09,656][105692] Updated weights for policy 0, policy_version 524628 (0.0005) [2023-12-26 19:11:09,684][105585] KL-divergence is very high: 163.3891 [2023-12-26 19:11:09,710][105692] Updated weights for policy 0, policy_version 524638 (0.0006) [2023-12-26 19:11:10,072][105620] Updated weights for policy 1, policy_version 525327 (0.0008) [2023-12-26 19:11:10,131][105620] Updated weights for policy 1, policy_version 525337 (0.0007) [2023-12-26 19:11:10,180][105620] Updated weights for policy 1, policy_version 525347 (0.0010) [2023-12-26 19:11:10,366][105692] Updated weights for policy 0, policy_version 524648 (0.0010) [2023-12-26 19:11:10,426][105692] Updated weights for policy 0, policy_version 524658 (0.0011) [2023-12-26 19:11:10,441][105585] KL-divergence is very high: 128.3925 [2023-12-26 19:11:10,465][105585] KL-divergence is very high: 128.6235 [2023-12-26 19:11:10,488][105585] KL-divergence is very high: 121.3730 [2023-12-26 19:11:10,489][105692] Updated weights for policy 0, policy_version 524668 (0.0010) [2023-12-26 19:11:10,825][105620] Updated weights for policy 1, policy_version 525357 (0.0008) [2023-12-26 19:11:10,883][105620] Updated weights for policy 1, policy_version 525367 (0.0006) [2023-12-26 19:11:10,937][105620] Updated weights for policy 1, policy_version 525377 (0.0005) [2023-12-26 19:11:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 268845056. Throughput: 0: 9956.6, 1: 9546.0. Samples: 268853336. Policy #0 lag: (min: 36.0, avg: 47.6, max: 48.0) [2023-12-26 19:11:11,062][104569] Avg episode reward: [(0, '8988.599'), (1, '9263.450')] [2023-12-26 19:11:11,213][105692] Updated weights for policy 0, policy_version 524678 (0.0009) [2023-12-26 19:11:11,280][105692] Updated weights for policy 0, policy_version 524688 (0.0009) [2023-12-26 19:11:11,341][105692] Updated weights for policy 0, policy_version 524698 (0.0009) [2023-12-26 19:11:11,653][105620] Updated weights for policy 1, policy_version 525387 (0.0006) [2023-12-26 19:11:11,706][105620] Updated weights for policy 1, policy_version 525397 (0.0010) [2023-12-26 19:11:11,771][105620] Updated weights for policy 1, policy_version 525407 (0.0009) [2023-12-26 19:11:12,084][105692] Updated weights for policy 0, policy_version 524708 (0.0010) [2023-12-26 19:11:12,148][105692] Updated weights for policy 0, policy_version 524718 (0.0009) [2023-12-26 19:11:12,210][105692] Updated weights for policy 0, policy_version 524728 (0.0009) [2023-12-26 19:11:12,556][105620] Updated weights for policy 1, policy_version 525417 (0.0008) [2023-12-26 19:11:12,624][105620] Updated weights for policy 1, policy_version 525427 (0.0007) [2023-12-26 19:11:12,691][105620] Updated weights for policy 1, policy_version 525437 (0.0005) [2023-12-26 19:11:12,750][105620] Updated weights for policy 1, policy_version 525447 (0.0005) [2023-12-26 19:11:12,910][105692] Updated weights for policy 0, policy_version 524738 (0.0007) [2023-12-26 19:11:12,976][105692] Updated weights for policy 0, policy_version 524748 (0.0008) [2023-12-26 19:11:13,040][105692] Updated weights for policy 0, policy_version 524758 (0.0008) [2023-12-26 19:11:13,107][105692] Updated weights for policy 0, policy_version 524768 (0.0008) [2023-12-26 19:11:13,414][105620] Updated weights for policy 1, policy_version 525457 (0.0010) [2023-12-26 19:11:13,466][105620] Updated weights for policy 1, policy_version 525467 (0.0010) [2023-12-26 19:11:13,514][105620] Updated weights for policy 1, policy_version 525477 (0.0010) [2023-12-26 19:11:13,838][105692] Updated weights for policy 0, policy_version 524778 (0.0008) [2023-12-26 19:11:13,893][105692] Updated weights for policy 0, policy_version 524788 (0.0007) [2023-12-26 19:11:13,954][105692] Updated weights for policy 0, policy_version 524798 (0.0005) [2023-12-26 19:11:14,269][105620] Updated weights for policy 1, policy_version 525487 (0.0010) [2023-12-26 19:11:14,317][105620] Updated weights for policy 1, policy_version 525497 (0.0010) [2023-12-26 19:11:14,368][105620] Updated weights for policy 1, policy_version 525507 (0.0010) [2023-12-26 19:11:14,665][105692] Updated weights for policy 0, policy_version 524808 (0.0008) [2023-12-26 19:11:14,733][105692] Updated weights for policy 0, policy_version 524818 (0.0008) [2023-12-26 19:11:14,800][105692] Updated weights for policy 0, policy_version 524828 (0.0008) [2023-12-26 19:11:15,125][105620] Updated weights for policy 1, policy_version 525517 (0.0010) [2023-12-26 19:11:15,193][105620] Updated weights for policy 1, policy_version 525527 (0.0010) [2023-12-26 19:11:15,260][105620] Updated weights for policy 1, policy_version 525537 (0.0011) [2023-12-26 19:11:15,577][105692] Updated weights for policy 0, policy_version 524838 (0.0008) [2023-12-26 19:11:15,638][105692] Updated weights for policy 0, policy_version 524848 (0.0008) [2023-12-26 19:11:15,698][105692] Updated weights for policy 0, policy_version 524858 (0.0009) [2023-12-26 19:11:15,954][105620] Updated weights for policy 1, policy_version 525547 (0.0009) [2023-12-26 19:11:16,011][105620] Updated weights for policy 1, policy_version 525557 (0.0005) [2023-12-26 19:11:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 268935168. Throughput: 0: 9904.5, 1: 9531.2. Samples: 268910168. Policy #0 lag: (min: 36.0, avg: 47.6, max: 48.0) [2023-12-26 19:11:16,063][104569] Avg episode reward: [(0, '8988.391'), (1, '9354.197')] [2023-12-26 19:11:16,068][105620] Updated weights for policy 1, policy_version 525567 (0.0005) [2023-12-26 19:11:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000524864_134381568.pth... [2023-12-26 19:11:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000523680_134078464.pth [2023-12-26 19:11:16,118][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000525576_134561792.pth... [2023-12-26 19:11:16,121][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000524456_134275072.pth [2023-12-26 19:11:16,587][105692] Updated weights for policy 0, policy_version 524868 (0.0008) [2023-12-26 19:11:16,606][105620] Updated weights for policy 1, policy_version 525577 (0.0006) [2023-12-26 19:11:16,642][105692] Updated weights for policy 0, policy_version 524878 (0.0007) [2023-12-26 19:11:16,665][105620] Updated weights for policy 1, policy_version 525587 (0.0007) [2023-12-26 19:11:16,699][105692] Updated weights for policy 0, policy_version 524888 (0.0010) [2023-12-26 19:11:16,723][105620] Updated weights for policy 1, policy_version 525597 (0.0006) [2023-12-26 19:11:16,783][105620] Updated weights for policy 1, policy_version 525607 (0.0006) [2023-12-26 19:11:17,322][105692] Updated weights for policy 0, policy_version 524898 (0.0009) [2023-12-26 19:11:17,374][105692] Updated weights for policy 0, policy_version 524908 (0.0005) [2023-12-26 19:11:17,410][105620] Updated weights for policy 1, policy_version 525617 (0.0006) [2023-12-26 19:11:17,430][105692] Updated weights for policy 0, policy_version 524918 (0.0005) [2023-12-26 19:11:17,465][105620] Updated weights for policy 1, policy_version 525627 (0.0007) [2023-12-26 19:11:17,479][105692] Updated weights for policy 0, policy_version 524928 (0.0007) [2023-12-26 19:11:17,526][105620] Updated weights for policy 1, policy_version 525637 (0.0008) [2023-12-26 19:11:18,100][105692] Updated weights for policy 0, policy_version 524938 (0.0006) [2023-12-26 19:11:18,109][105620] Updated weights for policy 1, policy_version 525647 (0.0007) [2023-12-26 19:11:18,152][105692] Updated weights for policy 0, policy_version 524948 (0.0008) [2023-12-26 19:11:18,163][105620] Updated weights for policy 1, policy_version 525657 (0.0005) [2023-12-26 19:11:18,208][105692] Updated weights for policy 0, policy_version 524958 (0.0010) [2023-12-26 19:11:18,225][105620] Updated weights for policy 1, policy_version 525667 (0.0005) [2023-12-26 19:11:18,799][105620] Updated weights for policy 1, policy_version 525677 (0.0005) [2023-12-26 19:11:18,862][105620] Updated weights for policy 1, policy_version 525687 (0.0007) [2023-12-26 19:11:18,921][105620] Updated weights for policy 1, policy_version 525697 (0.0011) [2023-12-26 19:11:19,030][105692] Updated weights for policy 0, policy_version 524968 (0.0008) [2023-12-26 19:11:19,093][105692] Updated weights for policy 0, policy_version 524978 (0.0007) [2023-12-26 19:11:19,158][105692] Updated weights for policy 0, policy_version 524988 (0.0008) [2023-12-26 19:11:19,645][105620] Updated weights for policy 1, policy_version 525707 (0.0010) [2023-12-26 19:11:19,710][105620] Updated weights for policy 1, policy_version 525717 (0.0008) [2023-12-26 19:11:19,763][105620] Updated weights for policy 1, policy_version 525727 (0.0010) [2023-12-26 19:11:19,861][105692] Updated weights for policy 0, policy_version 524998 (0.0008) [2023-12-26 19:11:19,926][105692] Updated weights for policy 0, policy_version 525008 (0.0008) [2023-12-26 19:11:19,981][105692] Updated weights for policy 0, policy_version 525018 (0.0009) [2023-12-26 19:11:20,615][105620] Updated weights for policy 1, policy_version 525737 (0.0009) [2023-12-26 19:11:20,648][105692] Updated weights for policy 0, policy_version 525028 (0.0009) [2023-12-26 19:11:20,706][105620] Updated weights for policy 1, policy_version 525747 (0.0009) [2023-12-26 19:11:20,707][105692] Updated weights for policy 0, policy_version 525038 (0.0008) [2023-12-26 19:11:20,765][105692] Updated weights for policy 0, policy_version 525048 (0.0006) [2023-12-26 19:11:20,767][105620] Updated weights for policy 1, policy_version 525757 (0.0007) [2023-12-26 19:11:20,830][105620] Updated weights for policy 1, policy_version 525767 (0.0007) [2023-12-26 19:11:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 269041664. Throughput: 0: 9844.6, 1: 9668.9. Samples: 269029260. Policy #0 lag: (min: 36.0, avg: 47.6, max: 48.0) [2023-12-26 19:11:21,062][104569] Avg episode reward: [(0, '7281.122'), (1, '9261.738')] [2023-12-26 19:11:21,465][105692] Updated weights for policy 0, policy_version 525058 (0.0008) [2023-12-26 19:11:21,522][105692] Updated weights for policy 0, policy_version 525068 (0.0006) [2023-12-26 19:11:21,585][105692] Updated weights for policy 0, policy_version 525078 (0.0006) [2023-12-26 19:11:21,591][105620] Updated weights for policy 1, policy_version 525777 (0.0006) [2023-12-26 19:11:21,650][105692] Updated weights for policy 0, policy_version 525088 (0.0007) [2023-12-26 19:11:21,663][105620] Updated weights for policy 1, policy_version 525787 (0.0008) [2023-12-26 19:11:21,725][105620] Updated weights for policy 1, policy_version 525797 (0.0007) [2023-12-26 19:11:22,305][105620] Updated weights for policy 1, policy_version 525807 (0.0008) [2023-12-26 19:11:22,378][105620] Updated weights for policy 1, policy_version 525817 (0.0009) [2023-12-26 19:11:22,438][105620] Updated weights for policy 1, policy_version 525827 (0.0006) [2023-12-26 19:11:22,449][105692] Updated weights for policy 0, policy_version 525098 (0.0008) [2023-12-26 19:11:22,506][105692] Updated weights for policy 0, policy_version 525108 (0.0008) [2023-12-26 19:11:22,558][105692] Updated weights for policy 0, policy_version 525118 (0.0008) [2023-12-26 19:11:23,124][105620] Updated weights for policy 1, policy_version 525837 (0.0009) [2023-12-26 19:11:23,187][105620] Updated weights for policy 1, policy_version 525847 (0.0009) [2023-12-26 19:11:23,252][105620] Updated weights for policy 1, policy_version 525857 (0.0009) [2023-12-26 19:11:23,410][105692] Updated weights for policy 0, policy_version 525128 (0.0009) [2023-12-26 19:11:23,471][105692] Updated weights for policy 0, policy_version 525138 (0.0009) [2023-12-26 19:11:23,519][105692] Updated weights for policy 0, policy_version 525148 (0.0009) [2023-12-26 19:11:23,989][105620] Updated weights for policy 1, policy_version 525867 (0.0009) [2023-12-26 19:11:24,035][105620] Updated weights for policy 1, policy_version 525877 (0.0009) [2023-12-26 19:11:24,098][105620] Updated weights for policy 1, policy_version 525887 (0.0011) [2023-12-26 19:11:24,271][105692] Updated weights for policy 0, policy_version 525158 (0.0009) [2023-12-26 19:11:24,325][105692] Updated weights for policy 0, policy_version 525168 (0.0009) [2023-12-26 19:11:24,391][105692] Updated weights for policy 0, policy_version 525178 (0.0009) [2023-12-26 19:11:24,879][105620] Updated weights for policy 1, policy_version 525897 (0.0010) [2023-12-26 19:11:24,945][105620] Updated weights for policy 1, policy_version 525907 (0.0010) [2023-12-26 19:11:25,008][105620] Updated weights for policy 1, policy_version 525917 (0.0009) [2023-12-26 19:11:25,015][105692] Updated weights for policy 0, policy_version 525188 (0.0008) [2023-12-26 19:11:25,076][105620] Updated weights for policy 1, policy_version 525927 (0.0007) [2023-12-26 19:11:25,077][105692] Updated weights for policy 0, policy_version 525198 (0.0010) [2023-12-26 19:11:25,127][105692] Updated weights for policy 0, policy_version 525208 (0.0009) [2023-12-26 19:11:25,701][105692] Updated weights for policy 0, policy_version 525218 (0.0008) [2023-12-26 19:11:25,747][105692] Updated weights for policy 0, policy_version 525228 (0.0010) [2023-12-26 19:11:25,792][105692] Updated weights for policy 0, policy_version 525238 (0.0010) [2023-12-26 19:11:25,836][105692] Updated weights for policy 0, policy_version 525248 (0.0010) [2023-12-26 19:11:25,902][105620] Updated weights for policy 1, policy_version 525937 (0.0008) [2023-12-26 19:11:25,960][105620] Updated weights for policy 1, policy_version 525947 (0.0008) [2023-12-26 19:11:26,016][105620] Updated weights for policy 1, policy_version 525957 (0.0009) [2023-12-26 19:11:26,062][104569] Fps is (10 sec: 20480.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 269139968. Throughput: 0: 9946.7, 1: 9590.5. Samples: 269143228. Policy #0 lag: (min: 36.0, avg: 47.6, max: 48.0) [2023-12-26 19:11:26,062][104569] Avg episode reward: [(0, '5165.574'), (1, '9172.225')] [2023-12-26 19:11:26,438][105692] Updated weights for policy 0, policy_version 525258 (0.0006) [2023-12-26 19:11:26,487][105692] Updated weights for policy 0, policy_version 525268 (0.0006) [2023-12-26 19:11:26,533][105692] Updated weights for policy 0, policy_version 525278 (0.0005) [2023-12-26 19:11:26,724][105620] Updated weights for policy 1, policy_version 525967 (0.0009) [2023-12-26 19:11:26,772][105620] Updated weights for policy 1, policy_version 525977 (0.0008) [2023-12-26 19:11:26,827][105620] Updated weights for policy 1, policy_version 525987 (0.0008) [2023-12-26 19:11:27,167][105692] Updated weights for policy 0, policy_version 525288 (0.0010) [2023-12-26 19:11:27,223][105692] Updated weights for policy 0, policy_version 525298 (0.0006) [2023-12-26 19:11:27,269][105692] Updated weights for policy 0, policy_version 525308 (0.0005) [2023-12-26 19:11:27,650][105620] Updated weights for policy 1, policy_version 525997 (0.0008) [2023-12-26 19:11:27,705][105620] Updated weights for policy 1, policy_version 526007 (0.0008) [2023-12-26 19:11:27,760][105620] Updated weights for policy 1, policy_version 526017 (0.0008) [2023-12-26 19:11:27,980][105692] Updated weights for policy 0, policy_version 525318 (0.0009) [2023-12-26 19:11:28,033][105692] Updated weights for policy 0, policy_version 525328 (0.0008) [2023-12-26 19:11:28,101][105692] Updated weights for policy 0, policy_version 525338 (0.0005) [2023-12-26 19:11:28,599][105620] Updated weights for policy 1, policy_version 526027 (0.0009) [2023-12-26 19:11:28,647][105692] Updated weights for policy 0, policy_version 525348 (0.0005) [2023-12-26 19:11:28,659][105620] Updated weights for policy 1, policy_version 526037 (0.0010) [2023-12-26 19:11:28,707][105692] Updated weights for policy 0, policy_version 525358 (0.0011) [2023-12-26 19:11:28,715][105620] Updated weights for policy 1, policy_version 526047 (0.0011) [2023-12-26 19:11:28,770][105692] Updated weights for policy 0, policy_version 525368 (0.0011) [2023-12-26 19:11:29,377][105692] Updated weights for policy 0, policy_version 525378 (0.0011) [2023-12-26 19:11:29,419][105620] Updated weights for policy 1, policy_version 526057 (0.0008) [2023-12-26 19:11:29,439][105692] Updated weights for policy 0, policy_version 525388 (0.0010) [2023-12-26 19:11:29,487][105620] Updated weights for policy 1, policy_version 526067 (0.0005) [2023-12-26 19:11:29,487][105692] Updated weights for policy 0, policy_version 525398 (0.0010) [2023-12-26 19:11:29,549][105692] Updated weights for policy 0, policy_version 525408 (0.0010) [2023-12-26 19:11:29,552][105620] Updated weights for policy 1, policy_version 526077 (0.0005) [2023-12-26 19:11:29,614][105620] Updated weights for policy 1, policy_version 526087 (0.0005) [2023-12-26 19:11:30,251][105620] Updated weights for policy 1, policy_version 526097 (0.0009) [2023-12-26 19:11:30,310][105620] Updated weights for policy 1, policy_version 526107 (0.0009) [2023-12-26 19:11:30,336][105692] Updated weights for policy 0, policy_version 525418 (0.0005) [2023-12-26 19:11:30,369][105620] Updated weights for policy 1, policy_version 526117 (0.0010) [2023-12-26 19:11:30,395][105692] Updated weights for policy 0, policy_version 525428 (0.0007) [2023-12-26 19:11:30,453][105692] Updated weights for policy 0, policy_version 525438 (0.0008) [2023-12-26 19:11:31,020][105620] Updated weights for policy 1, policy_version 526127 (0.0007) [2023-12-26 19:11:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 269230080. Throughput: 0: 10014.1, 1: 9519.5. Samples: 269204428. Policy #0 lag: (min: 29.0, avg: 52.3, max: 57.0) [2023-12-26 19:11:31,062][104569] Avg episode reward: [(0, '6870.078'), (1, '9173.575')] [2023-12-26 19:11:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000525440_134529024.pth... [2023-12-26 19:11:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000524288_134234112.pth [2023-12-26 19:11:31,086][105620] Updated weights for policy 1, policy_version 526137 (0.0006) [2023-12-26 19:11:31,155][105620] Updated weights for policy 1, policy_version 526147 (0.0008) [2023-12-26 19:11:31,178][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000526152_134709248.pth... [2023-12-26 19:11:31,181][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000525032_134422528.pth [2023-12-26 19:11:31,224][105692] Updated weights for policy 0, policy_version 525448 (0.0010) [2023-12-26 19:11:31,293][105692] Updated weights for policy 0, policy_version 525458 (0.0008) [2023-12-26 19:11:31,360][105692] Updated weights for policy 0, policy_version 525468 (0.0008) [2023-12-26 19:11:31,796][105620] Updated weights for policy 1, policy_version 526157 (0.0008) [2023-12-26 19:11:31,847][105620] Updated weights for policy 1, policy_version 526167 (0.0006) [2023-12-26 19:11:31,901][105620] Updated weights for policy 1, policy_version 526177 (0.0005) [2023-12-26 19:11:32,143][105692] Updated weights for policy 0, policy_version 525478 (0.0006) [2023-12-26 19:11:32,206][105692] Updated weights for policy 0, policy_version 525488 (0.0005) [2023-12-26 19:11:32,267][105692] Updated weights for policy 0, policy_version 525498 (0.0006) [2023-12-26 19:11:32,614][105620] Updated weights for policy 1, policy_version 526187 (0.0006) [2023-12-26 19:11:32,665][105620] Updated weights for policy 1, policy_version 526197 (0.0006) [2023-12-26 19:11:32,712][105620] Updated weights for policy 1, policy_version 526207 (0.0005) [2023-12-26 19:11:32,882][105692] Updated weights for policy 0, policy_version 525508 (0.0007) [2023-12-26 19:11:32,929][105692] Updated weights for policy 0, policy_version 525518 (0.0010) [2023-12-26 19:11:32,975][105692] Updated weights for policy 0, policy_version 525528 (0.0008) [2023-12-26 19:11:33,364][105620] Updated weights for policy 1, policy_version 526217 (0.0006) [2023-12-26 19:11:33,416][105620] Updated weights for policy 1, policy_version 526227 (0.0008) [2023-12-26 19:11:33,466][105620] Updated weights for policy 1, policy_version 526237 (0.0008) [2023-12-26 19:11:33,517][105620] Updated weights for policy 1, policy_version 526247 (0.0008) [2023-12-26 19:11:33,764][105692] Updated weights for policy 0, policy_version 525538 (0.0009) [2023-12-26 19:11:33,814][105692] Updated weights for policy 0, policy_version 525548 (0.0008) [2023-12-26 19:11:33,864][105692] Updated weights for policy 0, policy_version 525558 (0.0009) [2023-12-26 19:11:33,910][105692] Updated weights for policy 0, policy_version 525568 (0.0005) [2023-12-26 19:11:34,324][105620] Updated weights for policy 1, policy_version 526257 (0.0010) [2023-12-26 19:11:34,373][105620] Updated weights for policy 1, policy_version 526267 (0.0010) [2023-12-26 19:11:34,425][105620] Updated weights for policy 1, policy_version 526277 (0.0010) [2023-12-26 19:11:34,592][105692] Updated weights for policy 0, policy_version 525578 (0.0008) [2023-12-26 19:11:34,650][105692] Updated weights for policy 0, policy_version 525588 (0.0008) [2023-12-26 19:11:34,707][105692] Updated weights for policy 0, policy_version 525598 (0.0008) [2023-12-26 19:11:35,193][105620] Updated weights for policy 1, policy_version 526287 (0.0010) [2023-12-26 19:11:35,248][105620] Updated weights for policy 1, policy_version 526297 (0.0010) [2023-12-26 19:11:35,313][105620] Updated weights for policy 1, policy_version 526307 (0.0010) [2023-12-26 19:11:35,374][105692] Updated weights for policy 0, policy_version 525608 (0.0006) [2023-12-26 19:11:35,436][105692] Updated weights for policy 0, policy_version 525618 (0.0005) [2023-12-26 19:11:35,493][105692] Updated weights for policy 0, policy_version 525628 (0.0005) [2023-12-26 19:11:36,049][105620] Updated weights for policy 1, policy_version 526317 (0.0010) [2023-12-26 19:11:36,062][104569] Fps is (10 sec: 18840.8, 60 sec: 19387.6, 300 sec: 19494.2). Total num frames: 269328384. Throughput: 0: 9918.3, 1: 9527.7. Samples: 269322996. Policy #0 lag: (min: 29.0, avg: 52.3, max: 57.0) [2023-12-26 19:11:36,064][104569] Avg episode reward: [(0, '8647.453'), (1, '9263.355')] [2023-12-26 19:11:36,100][105620] Updated weights for policy 1, policy_version 526327 (0.0010) [2023-12-26 19:11:36,137][105692] Updated weights for policy 0, policy_version 525638 (0.0007) [2023-12-26 19:11:36,158][105620] Updated weights for policy 1, policy_version 526337 (0.0012) [2023-12-26 19:11:36,191][105692] Updated weights for policy 0, policy_version 525648 (0.0006) [2023-12-26 19:11:36,260][105692] Updated weights for policy 0, policy_version 525658 (0.0006) [2023-12-26 19:11:36,859][105692] Updated weights for policy 0, policy_version 525668 (0.0008) [2023-12-26 19:11:36,902][105620] Updated weights for policy 1, policy_version 526347 (0.0011) [2023-12-26 19:11:36,908][105692] Updated weights for policy 0, policy_version 525678 (0.0011) [2023-12-26 19:11:36,953][105620] Updated weights for policy 1, policy_version 526357 (0.0010) [2023-12-26 19:11:36,960][105692] Updated weights for policy 0, policy_version 525688 (0.0010) [2023-12-26 19:11:37,005][105620] Updated weights for policy 1, policy_version 526367 (0.0010) [2023-12-26 19:11:37,590][105692] Updated weights for policy 0, policy_version 525698 (0.0009) [2023-12-26 19:11:37,636][105692] Updated weights for policy 0, policy_version 525708 (0.0005) [2023-12-26 19:11:37,691][105692] Updated weights for policy 0, policy_version 525718 (0.0005) [2023-12-26 19:11:37,713][105620] Updated weights for policy 1, policy_version 526377 (0.0010) [2023-12-26 19:11:37,751][105692] Updated weights for policy 0, policy_version 525728 (0.0006) [2023-12-26 19:11:37,771][105620] Updated weights for policy 1, policy_version 526387 (0.0006) [2023-12-26 19:11:37,836][105620] Updated weights for policy 1, policy_version 526397 (0.0006) [2023-12-26 19:11:37,896][105620] Updated weights for policy 1, policy_version 526407 (0.0009) [2023-12-26 19:11:38,488][105692] Updated weights for policy 0, policy_version 525738 (0.0009) [2023-12-26 19:11:38,520][105620] Updated weights for policy 1, policy_version 526417 (0.0007) [2023-12-26 19:11:38,531][105692] Updated weights for policy 0, policy_version 525748 (0.0007) [2023-12-26 19:11:38,579][105620] Updated weights for policy 1, policy_version 526427 (0.0007) [2023-12-26 19:11:38,585][105692] Updated weights for policy 0, policy_version 525758 (0.0008) [2023-12-26 19:11:38,639][105620] Updated weights for policy 1, policy_version 526437 (0.0008) [2023-12-26 19:11:39,373][105620] Updated weights for policy 1, policy_version 526447 (0.0007) [2023-12-26 19:11:39,382][105692] Updated weights for policy 0, policy_version 525768 (0.0008) [2023-12-26 19:11:39,432][105620] Updated weights for policy 1, policy_version 526457 (0.0006) [2023-12-26 19:11:39,447][105692] Updated weights for policy 0, policy_version 525778 (0.0007) [2023-12-26 19:11:39,500][105620] Updated weights for policy 1, policy_version 526467 (0.0009) [2023-12-26 19:11:39,510][105692] Updated weights for policy 0, policy_version 525788 (0.0006) [2023-12-26 19:11:40,103][105692] Updated weights for policy 0, policy_version 525798 (0.0007) [2023-12-26 19:11:40,160][105692] Updated weights for policy 0, policy_version 525808 (0.0010) [2023-12-26 19:11:40,229][105692] Updated weights for policy 0, policy_version 525818 (0.0009) [2023-12-26 19:11:40,337][105620] Updated weights for policy 1, policy_version 526477 (0.0009) [2023-12-26 19:11:40,400][105620] Updated weights for policy 1, policy_version 526487 (0.0007) [2023-12-26 19:11:40,472][105620] Updated weights for policy 1, policy_version 526497 (0.0006) [2023-12-26 19:11:41,012][105692] Updated weights for policy 0, policy_version 525828 (0.0008) [2023-12-26 19:11:41,037][105585] KL-divergence is very high: 180.3072 [2023-12-26 19:11:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 269426688. Throughput: 0: 9988.3, 1: 9554.2. Samples: 269441792. Policy #0 lag: (min: 29.0, avg: 52.3, max: 57.0) [2023-12-26 19:11:41,063][104569] Avg episode reward: [(0, '9085.122'), (1, '9352.807')] [2023-12-26 19:11:41,073][105692] Updated weights for policy 0, policy_version 525838 (0.0008) [2023-12-26 19:11:41,086][105585] KL-divergence is very high: 245.3377 [2023-12-26 19:11:41,114][105620] Updated weights for policy 1, policy_version 526507 (0.0007) [2023-12-26 19:11:41,141][105692] Updated weights for policy 0, policy_version 525848 (0.0008) [2023-12-26 19:11:41,142][105585] KL-divergence is very high: 213.8524 [2023-12-26 19:11:41,175][105620] Updated weights for policy 1, policy_version 526517 (0.0008) [2023-12-26 19:11:41,232][105620] Updated weights for policy 1, policy_version 526527 (0.0008) [2023-12-26 19:11:41,916][105692] Updated weights for policy 0, policy_version 525858 (0.0007) [2023-12-26 19:11:41,980][105692] Updated weights for policy 0, policy_version 525868 (0.0009) [2023-12-26 19:11:42,009][105620] Updated weights for policy 1, policy_version 526537 (0.0008) [2023-12-26 19:11:42,043][105692] Updated weights for policy 0, policy_version 525878 (0.0009) [2023-12-26 19:11:42,070][105620] Updated weights for policy 1, policy_version 526547 (0.0006) [2023-12-26 19:11:42,105][105692] Updated weights for policy 0, policy_version 525888 (0.0008) [2023-12-26 19:11:42,130][105620] Updated weights for policy 1, policy_version 526557 (0.0007) [2023-12-26 19:11:42,200][105620] Updated weights for policy 1, policy_version 526567 (0.0010) [2023-12-26 19:11:42,736][105692] Updated weights for policy 0, policy_version 525898 (0.0008) [2023-12-26 19:11:42,790][105692] Updated weights for policy 0, policy_version 525908 (0.0010) [2023-12-26 19:11:42,839][105692] Updated weights for policy 0, policy_version 525918 (0.0009) [2023-12-26 19:11:42,973][105620] Updated weights for policy 1, policy_version 526577 (0.0009) [2023-12-26 19:11:43,023][105620] Updated weights for policy 1, policy_version 526587 (0.0008) [2023-12-26 19:11:43,079][105620] Updated weights for policy 1, policy_version 526597 (0.0006) [2023-12-26 19:11:43,575][105692] Updated weights for policy 0, policy_version 525928 (0.0009) [2023-12-26 19:11:43,629][105692] Updated weights for policy 0, policy_version 525938 (0.0005) [2023-12-26 19:11:43,635][105620] Updated weights for policy 1, policy_version 526607 (0.0006) [2023-12-26 19:11:43,698][105692] Updated weights for policy 0, policy_version 525948 (0.0005) [2023-12-26 19:11:43,701][105620] Updated weights for policy 1, policy_version 526617 (0.0006) [2023-12-26 19:11:43,756][105620] Updated weights for policy 1, policy_version 526627 (0.0009) [2023-12-26 19:11:44,289][105692] Updated weights for policy 0, policy_version 525958 (0.0005) [2023-12-26 19:11:44,344][105692] Updated weights for policy 0, policy_version 525968 (0.0006) [2023-12-26 19:11:44,395][105692] Updated weights for policy 0, policy_version 525978 (0.0006) [2023-12-26 19:11:44,564][105620] Updated weights for policy 1, policy_version 526637 (0.0010) [2023-12-26 19:11:44,618][105620] Updated weights for policy 1, policy_version 526647 (0.0008) [2023-12-26 19:11:44,668][105620] Updated weights for policy 1, policy_version 526657 (0.0009) [2023-12-26 19:11:45,056][105692] Updated weights for policy 0, policy_version 525988 (0.0006) [2023-12-26 19:11:45,117][105692] Updated weights for policy 0, policy_version 525998 (0.0008) [2023-12-26 19:11:45,184][105692] Updated weights for policy 0, policy_version 526008 (0.0008) [2023-12-26 19:11:45,450][105620] Updated weights for policy 1, policy_version 526667 (0.0009) [2023-12-26 19:11:45,518][105620] Updated weights for policy 1, policy_version 526677 (0.0011) [2023-12-26 19:11:45,573][105620] Updated weights for policy 1, policy_version 526687 (0.0010) [2023-12-26 19:11:45,830][105692] Updated weights for policy 0, policy_version 526018 (0.0008) [2023-12-26 19:11:45,881][105692] Updated weights for policy 0, policy_version 526028 (0.0009) [2023-12-26 19:11:45,938][105692] Updated weights for policy 0, policy_version 526038 (0.0010) [2023-12-26 19:11:45,990][105692] Updated weights for policy 0, policy_version 526048 (0.0010) [2023-12-26 19:11:46,062][104569] Fps is (10 sec: 20480.8, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 269533184. Throughput: 0: 9935.6, 1: 9583.4. Samples: 269499476. Policy #0 lag: (min: 29.0, avg: 52.3, max: 57.0) [2023-12-26 19:11:46,062][104569] Avg episode reward: [(0, '8997.135'), (1, '9352.454')] [2023-12-26 19:11:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000526048_134684672.pth... [2023-12-26 19:11:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000526696_134848512.pth... [2023-12-26 19:11:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000524864_134381568.pth [2023-12-26 19:11:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000525576_134561792.pth [2023-12-26 19:11:46,234][105620] Updated weights for policy 1, policy_version 526697 (0.0010) [2023-12-26 19:11:46,284][105620] Updated weights for policy 1, policy_version 526707 (0.0010) [2023-12-26 19:11:46,335][105620] Updated weights for policy 1, policy_version 526717 (0.0010) [2023-12-26 19:11:46,379][105620] Updated weights for policy 1, policy_version 526727 (0.0010) [2023-12-26 19:11:46,728][105692] Updated weights for policy 0, policy_version 526058 (0.0005) [2023-12-26 19:11:46,788][105692] Updated weights for policy 0, policy_version 526068 (0.0005) [2023-12-26 19:11:46,854][105692] Updated weights for policy 0, policy_version 526078 (0.0005) [2023-12-26 19:11:47,109][105620] Updated weights for policy 1, policy_version 526737 (0.0011) [2023-12-26 19:11:47,162][105620] Updated weights for policy 1, policy_version 526747 (0.0011) [2023-12-26 19:11:47,218][105620] Updated weights for policy 1, policy_version 526757 (0.0009) [2023-12-26 19:11:47,394][105692] Updated weights for policy 0, policy_version 526088 (0.0009) [2023-12-26 19:11:47,450][105692] Updated weights for policy 0, policy_version 526098 (0.0009) [2023-12-26 19:11:47,509][105692] Updated weights for policy 0, policy_version 526108 (0.0009) [2023-12-26 19:11:47,981][105620] Updated weights for policy 1, policy_version 526767 (0.0009) [2023-12-26 19:11:48,044][105620] Updated weights for policy 1, policy_version 526777 (0.0007) [2023-12-26 19:11:48,104][105620] Updated weights for policy 1, policy_version 526787 (0.0006) [2023-12-26 19:11:48,296][105692] Updated weights for policy 0, policy_version 526118 (0.0009) [2023-12-26 19:11:48,357][105692] Updated weights for policy 0, policy_version 526128 (0.0009) [2023-12-26 19:11:48,423][105692] Updated weights for policy 0, policy_version 526138 (0.0009) [2023-12-26 19:11:48,806][105620] Updated weights for policy 1, policy_version 526797 (0.0009) [2023-12-26 19:11:48,864][105620] Updated weights for policy 1, policy_version 526807 (0.0009) [2023-12-26 19:11:48,926][105620] Updated weights for policy 1, policy_version 526817 (0.0009) [2023-12-26 19:11:49,164][105692] Updated weights for policy 0, policy_version 526148 (0.0009) [2023-12-26 19:11:49,211][105692] Updated weights for policy 0, policy_version 526158 (0.0009) [2023-12-26 19:11:49,292][105692] Updated weights for policy 0, policy_version 526168 (0.0009) [2023-12-26 19:11:49,689][105620] Updated weights for policy 1, policy_version 526827 (0.0009) [2023-12-26 19:11:49,747][105620] Updated weights for policy 1, policy_version 526837 (0.0010) [2023-12-26 19:11:49,802][105620] Updated weights for policy 1, policy_version 526847 (0.0009) [2023-12-26 19:11:50,009][105692] Updated weights for policy 0, policy_version 526178 (0.0008) [2023-12-26 19:11:50,063][105692] Updated weights for policy 0, policy_version 526188 (0.0007) [2023-12-26 19:11:50,116][105692] Updated weights for policy 0, policy_version 526198 (0.0006) [2023-12-26 19:11:50,171][105692] Updated weights for policy 0, policy_version 526208 (0.0009) [2023-12-26 19:11:50,593][105620] Updated weights for policy 1, policy_version 526857 (0.0008) [2023-12-26 19:11:50,653][105620] Updated weights for policy 1, policy_version 526867 (0.0007) [2023-12-26 19:11:50,713][105620] Updated weights for policy 1, policy_version 526877 (0.0007) [2023-12-26 19:11:50,774][105620] Updated weights for policy 1, policy_version 526887 (0.0005) [2023-12-26 19:11:50,973][105692] Updated weights for policy 0, policy_version 526218 (0.0010) [2023-12-26 19:11:51,027][105692] Updated weights for policy 0, policy_version 526229 (0.0010) [2023-12-26 19:11:51,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 269623296. Throughput: 0: 10007.7, 1: 9608.3. Samples: 269616140. Policy #0 lag: (min: 29.0, avg: 52.3, max: 57.0) [2023-12-26 19:11:51,062][104569] Avg episode reward: [(0, '8916.447'), (1, '9351.800')] [2023-12-26 19:11:51,084][105692] Updated weights for policy 0, policy_version 526239 (0.0008) [2023-12-26 19:11:51,393][105620] Updated weights for policy 1, policy_version 526897 (0.0008) [2023-12-26 19:11:51,452][105620] Updated weights for policy 1, policy_version 526907 (0.0010) [2023-12-26 19:11:51,515][105620] Updated weights for policy 1, policy_version 526917 (0.0008) [2023-12-26 19:11:51,888][105692] Updated weights for policy 0, policy_version 526249 (0.0010) [2023-12-26 19:11:51,942][105692] Updated weights for policy 0, policy_version 526259 (0.0008) [2023-12-26 19:11:51,996][105692] Updated weights for policy 0, policy_version 526269 (0.0009) [2023-12-26 19:11:52,010][105585] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000006 [2023-12-26 19:11:52,221][105620] Updated weights for policy 1, policy_version 526927 (0.0008) [2023-12-26 19:11:52,283][105620] Updated weights for policy 1, policy_version 526937 (0.0009) [2023-12-26 19:11:52,346][105620] Updated weights for policy 1, policy_version 526947 (0.0008) [2023-12-26 19:11:52,779][105692] Updated weights for policy 0, policy_version 526279 (0.0009) [2023-12-26 19:11:52,841][105692] Updated weights for policy 0, policy_version 526289 (0.0009) [2023-12-26 19:11:52,900][105692] Updated weights for policy 0, policy_version 526299 (0.0008) [2023-12-26 19:11:53,121][105620] Updated weights for policy 1, policy_version 526957 (0.0009) [2023-12-26 19:11:53,185][105620] Updated weights for policy 1, policy_version 526967 (0.0009) [2023-12-26 19:11:53,242][105620] Updated weights for policy 1, policy_version 526977 (0.0009) [2023-12-26 19:11:53,672][105692] Updated weights for policy 0, policy_version 526309 (0.0009) [2023-12-26 19:11:53,732][105692] Updated weights for policy 0, policy_version 526319 (0.0009) [2023-12-26 19:11:53,795][105692] Updated weights for policy 0, policy_version 526329 (0.0010) [2023-12-26 19:11:53,945][105620] Updated weights for policy 1, policy_version 526987 (0.0009) [2023-12-26 19:11:53,991][105620] Updated weights for policy 1, policy_version 526997 (0.0008) [2023-12-26 19:11:54,041][105620] Updated weights for policy 1, policy_version 527007 (0.0009) [2023-12-26 19:11:54,550][105692] Updated weights for policy 0, policy_version 526339 (0.0010) [2023-12-26 19:11:54,599][105692] Updated weights for policy 0, policy_version 526349 (0.0008) [2023-12-26 19:11:54,653][105692] Updated weights for policy 0, policy_version 526359 (0.0009) [2023-12-26 19:11:54,810][105620] Updated weights for policy 1, policy_version 527017 (0.0009) [2023-12-26 19:11:54,858][105620] Updated weights for policy 1, policy_version 527027 (0.0009) [2023-12-26 19:11:54,909][105620] Updated weights for policy 1, policy_version 527037 (0.0009) [2023-12-26 19:11:54,964][105620] Updated weights for policy 1, policy_version 527047 (0.0009) [2023-12-26 19:11:55,420][105692] Updated weights for policy 0, policy_version 526369 (0.0009) [2023-12-26 19:11:55,470][105692] Updated weights for policy 0, policy_version 526379 (0.0009) [2023-12-26 19:11:55,522][105692] Updated weights for policy 0, policy_version 526389 (0.0009) [2023-12-26 19:11:55,571][105692] Updated weights for policy 0, policy_version 526399 (0.0009) [2023-12-26 19:11:55,719][105620] Updated weights for policy 1, policy_version 527057 (0.0006) [2023-12-26 19:11:55,775][105620] Updated weights for policy 1, policy_version 527067 (0.0006) [2023-12-26 19:11:55,824][105620] Updated weights for policy 1, policy_version 527077 (0.0009) [2023-12-26 19:11:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 269721600. Throughput: 0: 9804.5, 1: 9664.1. Samples: 269729424. Policy #0 lag: (min: 29.0, avg: 52.3, max: 57.0) [2023-12-26 19:11:56,063][104569] Avg episode reward: [(0, '6862.105'), (1, '9351.803')] [2023-12-26 19:11:56,377][105692] Updated weights for policy 0, policy_version 526409 (0.0009) [2023-12-26 19:11:56,435][105692] Updated weights for policy 0, policy_version 526419 (0.0009) [2023-12-26 19:11:56,504][105692] Updated weights for policy 0, policy_version 526429 (0.0005) [2023-12-26 19:11:56,559][105620] Updated weights for policy 1, policy_version 527087 (0.0009) [2023-12-26 19:11:56,621][105620] Updated weights for policy 1, policy_version 527097 (0.0009) [2023-12-26 19:11:56,677][105620] Updated weights for policy 1, policy_version 527107 (0.0006) [2023-12-26 19:11:57,268][105692] Updated weights for policy 0, policy_version 526439 (0.0009) [2023-12-26 19:11:57,288][105620] Updated weights for policy 1, policy_version 527117 (0.0008) [2023-12-26 19:11:57,321][105692] Updated weights for policy 0, policy_version 526449 (0.0007) [2023-12-26 19:11:57,348][105620] Updated weights for policy 1, policy_version 527127 (0.0009) [2023-12-26 19:11:57,378][105692] Updated weights for policy 0, policy_version 526459 (0.0005) [2023-12-26 19:11:57,404][105620] Updated weights for policy 1, policy_version 527137 (0.0010) [2023-12-26 19:11:58,038][105620] Updated weights for policy 1, policy_version 527147 (0.0006) [2023-12-26 19:11:58,088][105620] Updated weights for policy 1, policy_version 527157 (0.0005) [2023-12-26 19:11:58,156][105620] Updated weights for policy 1, policy_version 527167 (0.0008) [2023-12-26 19:11:58,210][105692] Updated weights for policy 0, policy_version 526469 (0.0005) [2023-12-26 19:11:58,272][105692] Updated weights for policy 0, policy_version 526479 (0.0008) [2023-12-26 19:11:58,349][105692] Updated weights for policy 0, policy_version 526489 (0.0009) [2023-12-26 19:11:58,929][105620] Updated weights for policy 1, policy_version 527177 (0.0008) [2023-12-26 19:11:58,989][105620] Updated weights for policy 1, policy_version 527187 (0.0011) [2023-12-26 19:11:59,045][105620] Updated weights for policy 1, policy_version 527197 (0.0010) [2023-12-26 19:11:59,111][105620] Updated weights for policy 1, policy_version 527207 (0.0011) [2023-12-26 19:11:59,147][105692] Updated weights for policy 0, policy_version 526499 (0.0009) [2023-12-26 19:11:59,204][105692] Updated weights for policy 0, policy_version 526509 (0.0009) [2023-12-26 19:11:59,271][105692] Updated weights for policy 0, policy_version 526519 (0.0009) [2023-12-26 19:11:59,898][105620] Updated weights for policy 1, policy_version 527217 (0.0011) [2023-12-26 19:11:59,958][105620] Updated weights for policy 1, policy_version 527227 (0.0011) [2023-12-26 19:11:59,987][105692] Updated weights for policy 0, policy_version 526529 (0.0009) [2023-12-26 19:12:00,021][105620] Updated weights for policy 1, policy_version 527237 (0.0011) [2023-12-26 19:12:00,049][105692] Updated weights for policy 0, policy_version 526539 (0.0008) [2023-12-26 19:12:00,102][105692] Updated weights for policy 0, policy_version 526549 (0.0006) [2023-12-26 19:12:00,152][105692] Updated weights for policy 0, policy_version 526559 (0.0006) [2023-12-26 19:12:00,797][105620] Updated weights for policy 1, policy_version 527247 (0.0007) [2023-12-26 19:12:00,845][105692] Updated weights for policy 0, policy_version 526569 (0.0007) [2023-12-26 19:12:00,853][105620] Updated weights for policy 1, policy_version 527257 (0.0005) [2023-12-26 19:12:00,892][105692] Updated weights for policy 0, policy_version 526579 (0.0006) [2023-12-26 19:12:00,910][105620] Updated weights for policy 1, policy_version 527267 (0.0008) [2023-12-26 19:12:00,948][105692] Updated weights for policy 0, policy_version 526589 (0.0007) [2023-12-26 19:12:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 269819904. Throughput: 0: 9759.2, 1: 9701.4. Samples: 269785888. Policy #0 lag: (min: 29.0, avg: 52.3, max: 57.0) [2023-12-26 19:12:01,062][104569] Avg episode reward: [(0, '6730.929'), (1, '9351.972')] [2023-12-26 19:12:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000526592_134823936.pth... [2023-12-26 19:12:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000527272_134995968.pth... [2023-12-26 19:12:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000525440_134529024.pth [2023-12-26 19:12:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000526152_134709248.pth [2023-12-26 19:12:01,557][105692] Updated weights for policy 0, policy_version 526599 (0.0009) [2023-12-26 19:12:01,611][105692] Updated weights for policy 0, policy_version 526609 (0.0009) [2023-12-26 19:12:01,625][105620] Updated weights for policy 1, policy_version 527277 (0.0008) [2023-12-26 19:12:01,677][105692] Updated weights for policy 0, policy_version 526619 (0.0006) [2023-12-26 19:12:01,691][105620] Updated weights for policy 1, policy_version 527287 (0.0006) [2023-12-26 19:12:01,755][105620] Updated weights for policy 1, policy_version 527297 (0.0008) [2023-12-26 19:12:02,377][105692] Updated weights for policy 0, policy_version 526629 (0.0007) [2023-12-26 19:12:02,436][105692] Updated weights for policy 0, policy_version 526639 (0.0009) [2023-12-26 19:12:02,505][105692] Updated weights for policy 0, policy_version 526649 (0.0010) [2023-12-26 19:12:02,519][105620] Updated weights for policy 1, policy_version 527307 (0.0008) [2023-12-26 19:12:02,581][105620] Updated weights for policy 1, policy_version 527317 (0.0008) [2023-12-26 19:12:02,638][105620] Updated weights for policy 1, policy_version 527327 (0.0008) [2023-12-26 19:12:03,177][105692] Updated weights for policy 0, policy_version 526659 (0.0007) [2023-12-26 19:12:03,238][105692] Updated weights for policy 0, policy_version 526669 (0.0008) [2023-12-26 19:12:03,309][105692] Updated weights for policy 0, policy_version 526679 (0.0006) [2023-12-26 19:12:03,332][105620] Updated weights for policy 1, policy_version 527337 (0.0006) [2023-12-26 19:12:03,391][105620] Updated weights for policy 1, policy_version 527348 (0.0010) [2023-12-26 19:12:03,443][105620] Updated weights for policy 1, policy_version 527359 (0.0010) [2023-12-26 19:12:03,800][105692] Updated weights for policy 0, policy_version 526689 (0.0006) [2023-12-26 19:12:03,858][105692] Updated weights for policy 0, policy_version 526699 (0.0009) [2023-12-26 19:12:03,912][105692] Updated weights for policy 0, policy_version 526709 (0.0009) [2023-12-26 19:12:03,973][105692] Updated weights for policy 0, policy_version 526719 (0.0010) [2023-12-26 19:12:04,242][105620] Updated weights for policy 1, policy_version 527369 (0.0009) [2023-12-26 19:12:04,295][105620] Updated weights for policy 1, policy_version 527379 (0.0006) [2023-12-26 19:12:04,356][105620] Updated weights for policy 1, policy_version 527389 (0.0005) [2023-12-26 19:12:04,428][105620] Updated weights for policy 1, policy_version 527399 (0.0005) [2023-12-26 19:12:04,819][105692] Updated weights for policy 0, policy_version 526729 (0.0009) [2023-12-26 19:12:04,869][105692] Updated weights for policy 0, policy_version 526739 (0.0009) [2023-12-26 19:12:04,919][105692] Updated weights for policy 0, policy_version 526749 (0.0009) [2023-12-26 19:12:05,006][105620] Updated weights for policy 1, policy_version 527409 (0.0009) [2023-12-26 19:12:05,077][105620] Updated weights for policy 1, policy_version 527419 (0.0010) [2023-12-26 19:12:05,139][105620] Updated weights for policy 1, policy_version 527429 (0.0010) [2023-12-26 19:12:05,572][105692] Updated weights for policy 0, policy_version 526759 (0.0010) [2023-12-26 19:12:05,637][105692] Updated weights for policy 0, policy_version 526769 (0.0011) [2023-12-26 19:12:05,698][105692] Updated weights for policy 0, policy_version 526779 (0.0010) [2023-12-26 19:12:05,850][105620] Updated weights for policy 1, policy_version 527439 (0.0007) [2023-12-26 19:12:05,913][105620] Updated weights for policy 1, policy_version 527449 (0.0005) [2023-12-26 19:12:05,961][105620] Updated weights for policy 1, policy_version 527459 (0.0008) [2023-12-26 19:12:06,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 269918208. Throughput: 0: 9847.9, 1: 9585.8. Samples: 269903780. Policy #0 lag: (min: 29.0, avg: 52.3, max: 57.0) [2023-12-26 19:12:06,063][104569] Avg episode reward: [(0, '7699.393'), (1, '9261.940')] [2023-12-26 19:12:06,434][105692] Updated weights for policy 0, policy_version 526789 (0.0011) [2023-12-26 19:12:06,493][105692] Updated weights for policy 0, policy_version 526799 (0.0011) [2023-12-26 19:12:06,552][105692] Updated weights for policy 0, policy_version 526809 (0.0011) [2023-12-26 19:12:06,712][105620] Updated weights for policy 1, policy_version 527469 (0.0008) [2023-12-26 19:12:06,777][105620] Updated weights for policy 1, policy_version 527479 (0.0008) [2023-12-26 19:12:06,833][105620] Updated weights for policy 1, policy_version 527489 (0.0008) [2023-12-26 19:12:07,285][105692] Updated weights for policy 0, policy_version 526819 (0.0011) [2023-12-26 19:12:07,341][105692] Updated weights for policy 0, policy_version 526829 (0.0011) [2023-12-26 19:12:07,397][105692] Updated weights for policy 0, policy_version 526839 (0.0011) [2023-12-26 19:12:07,591][105620] Updated weights for policy 1, policy_version 527499 (0.0008) [2023-12-26 19:12:07,658][105620] Updated weights for policy 1, policy_version 527509 (0.0008) [2023-12-26 19:12:07,727][105620] Updated weights for policy 1, policy_version 527519 (0.0008) [2023-12-26 19:12:08,127][105692] Updated weights for policy 0, policy_version 526849 (0.0010) [2023-12-26 19:12:08,182][105692] Updated weights for policy 0, policy_version 526859 (0.0010) [2023-12-26 19:12:08,231][105692] Updated weights for policy 0, policy_version 526869 (0.0010) [2023-12-26 19:12:08,293][105692] Updated weights for policy 0, policy_version 526879 (0.0010) [2023-12-26 19:12:08,468][105620] Updated weights for policy 1, policy_version 527529 (0.0008) [2023-12-26 19:12:08,538][105620] Updated weights for policy 1, policy_version 527539 (0.0009) [2023-12-26 19:12:08,579][105586] KL-divergence is very high: 123.9662 [2023-12-26 19:12:08,605][105620] Updated weights for policy 1, policy_version 527549 (0.0009) [2023-12-26 19:12:08,633][105586] KL-divergence is very high: 198.9232 [2023-12-26 19:12:08,671][105620] Updated weights for policy 1, policy_version 527559 (0.0010) [2023-12-26 19:12:09,108][105692] Updated weights for policy 0, policy_version 526889 (0.0010) [2023-12-26 19:12:09,170][105692] Updated weights for policy 0, policy_version 526899 (0.0006) [2023-12-26 19:12:09,233][105692] Updated weights for policy 0, policy_version 526909 (0.0009) [2023-12-26 19:12:09,317][105620] Updated weights for policy 1, policy_version 527569 (0.0008) [2023-12-26 19:12:09,390][105620] Updated weights for policy 1, policy_version 527579 (0.0008) [2023-12-26 19:12:09,453][105620] Updated weights for policy 1, policy_version 527589 (0.0007) [2023-12-26 19:12:09,861][105692] Updated weights for policy 0, policy_version 526919 (0.0008) [2023-12-26 19:12:09,926][105692] Updated weights for policy 0, policy_version 526930 (0.0009) [2023-12-26 19:12:09,991][105692] Updated weights for policy 0, policy_version 526940 (0.0010) [2023-12-26 19:12:10,101][105620] Updated weights for policy 1, policy_version 527599 (0.0008) [2023-12-26 19:12:10,164][105620] Updated weights for policy 1, policy_version 527609 (0.0010) [2023-12-26 19:12:10,219][105620] Updated weights for policy 1, policy_version 527619 (0.0009) [2023-12-26 19:12:10,762][105692] Updated weights for policy 0, policy_version 526950 (0.0009) [2023-12-26 19:12:10,811][105692] Updated weights for policy 0, policy_version 526960 (0.0009) [2023-12-26 19:12:10,861][105692] Updated weights for policy 0, policy_version 526970 (0.0008) [2023-12-26 19:12:10,921][105620] Updated weights for policy 1, policy_version 527629 (0.0006) [2023-12-26 19:12:10,975][105620] Updated weights for policy 1, policy_version 527639 (0.0005) [2023-12-26 19:12:11,031][105620] Updated weights for policy 1, policy_version 527649 (0.0007) [2023-12-26 19:12:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 270008320. Throughput: 0: 9839.5, 1: 9637.7. Samples: 270019704. Policy #0 lag: (min: 29.0, avg: 52.3, max: 57.0) [2023-12-26 19:12:11,063][104569] Avg episode reward: [(0, '8200.763'), (1, '9171.323')] [2023-12-26 19:12:11,694][105692] Updated weights for policy 0, policy_version 526980 (0.0009) [2023-12-26 19:12:11,762][105692] Updated weights for policy 0, policy_version 526990 (0.0010) [2023-12-26 19:12:11,767][105620] Updated weights for policy 1, policy_version 527659 (0.0009) [2023-12-26 19:12:11,822][105692] Updated weights for policy 0, policy_version 527000 (0.0011) [2023-12-26 19:12:11,829][105620] Updated weights for policy 1, policy_version 527669 (0.0008) [2023-12-26 19:12:11,891][105620] Updated weights for policy 1, policy_version 527679 (0.0008) [2023-12-26 19:12:12,528][105692] Updated weights for policy 0, policy_version 527010 (0.0010) [2023-12-26 19:12:12,574][105620] Updated weights for policy 1, policy_version 527689 (0.0008) [2023-12-26 19:12:12,583][105692] Updated weights for policy 0, policy_version 527020 (0.0007) [2023-12-26 19:12:12,629][105620] Updated weights for policy 1, policy_version 527699 (0.0006) [2023-12-26 19:12:12,642][105692] Updated weights for policy 0, policy_version 527030 (0.0011) [2023-12-26 19:12:12,685][105620] Updated weights for policy 1, policy_version 527709 (0.0006) [2023-12-26 19:12:12,705][105692] Updated weights for policy 0, policy_version 527040 (0.0011) [2023-12-26 19:12:12,735][105620] Updated weights for policy 1, policy_version 527719 (0.0009) [2023-12-26 19:12:13,407][105620] Updated weights for policy 1, policy_version 527729 (0.0008) [2023-12-26 19:12:13,452][105692] Updated weights for policy 0, policy_version 527050 (0.0009) [2023-12-26 19:12:13,462][105620] Updated weights for policy 1, policy_version 527739 (0.0005) [2023-12-26 19:12:13,509][105692] Updated weights for policy 0, policy_version 527060 (0.0009) [2023-12-26 19:12:13,526][105620] Updated weights for policy 1, policy_version 527749 (0.0005) [2023-12-26 19:12:13,570][105692] Updated weights for policy 0, policy_version 527070 (0.0010) [2023-12-26 19:12:13,570][105585] KL-divergence is very high: 121.5727 [2023-12-26 19:12:14,210][105620] Updated weights for policy 1, policy_version 527759 (0.0007) [2023-12-26 19:12:14,267][105620] Updated weights for policy 1, policy_version 527769 (0.0008) [2023-12-26 19:12:14,322][105620] Updated weights for policy 1, policy_version 527779 (0.0008) [2023-12-26 19:12:14,329][105692] Updated weights for policy 0, policy_version 527080 (0.0007) [2023-12-26 19:12:14,376][105692] Updated weights for policy 0, policy_version 527090 (0.0007) [2023-12-26 19:12:14,422][105692] Updated weights for policy 0, policy_version 527100 (0.0008) [2023-12-26 19:12:15,087][105620] Updated weights for policy 1, policy_version 527789 (0.0009) [2023-12-26 19:12:15,157][105620] Updated weights for policy 1, policy_version 527799 (0.0009) [2023-12-26 19:12:15,226][105620] Updated weights for policy 1, policy_version 527809 (0.0008) [2023-12-26 19:12:15,237][105692] Updated weights for policy 0, policy_version 527110 (0.0007) [2023-12-26 19:12:15,303][105692] Updated weights for policy 0, policy_version 527120 (0.0007) [2023-12-26 19:12:15,370][105692] Updated weights for policy 0, policy_version 527130 (0.0008) [2023-12-26 19:12:15,928][105620] Updated weights for policy 1, policy_version 527819 (0.0009) [2023-12-26 19:12:15,992][105620] Updated weights for policy 1, policy_version 527829 (0.0008) [2023-12-26 19:12:16,062][104569] Fps is (10 sec: 18022.6, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 270098432. Throughput: 0: 9694.2, 1: 9689.2. Samples: 270076684. Policy #0 lag: (min: 29.0, avg: 52.3, max: 57.0) [2023-12-26 19:12:16,062][104569] Avg episode reward: [(0, '8640.528'), (1, '9077.979')] [2023-12-26 19:12:16,064][105620] Updated weights for policy 1, policy_version 527839 (0.0006) [2023-12-26 19:12:16,094][105692] Updated weights for policy 0, policy_version 527140 (0.0009) [2023-12-26 19:12:16,112][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000527848_135143424.pth... [2023-12-26 19:12:16,115][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000526696_134848512.pth [2023-12-26 19:12:16,144][105692] Updated weights for policy 0, policy_version 527150 (0.0007) [2023-12-26 19:12:16,195][105692] Updated weights for policy 0, policy_version 527160 (0.0010) [2023-12-26 19:12:16,234][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000527168_134971392.pth... [2023-12-26 19:12:16,237][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000526048_134684672.pth [2023-12-26 19:12:16,664][105620] Updated weights for policy 1, policy_version 527849 (0.0007) [2023-12-26 19:12:16,720][105620] Updated weights for policy 1, policy_version 527859 (0.0005) [2023-12-26 19:12:16,773][105620] Updated weights for policy 1, policy_version 527869 (0.0005) [2023-12-26 19:12:16,818][105620] Updated weights for policy 1, policy_version 527879 (0.0005) [2023-12-26 19:12:17,023][105692] Updated weights for policy 0, policy_version 527170 (0.0008) [2023-12-26 19:12:17,079][105692] Updated weights for policy 0, policy_version 527181 (0.0010) [2023-12-26 19:12:17,137][105692] Updated weights for policy 0, policy_version 527191 (0.0009) [2023-12-26 19:12:17,416][105620] Updated weights for policy 1, policy_version 527889 (0.0008) [2023-12-26 19:12:17,466][105620] Updated weights for policy 1, policy_version 527899 (0.0008) [2023-12-26 19:12:17,519][105620] Updated weights for policy 1, policy_version 527909 (0.0008) [2023-12-26 19:12:17,914][105692] Updated weights for policy 0, policy_version 527201 (0.0009) [2023-12-26 19:12:17,975][105692] Updated weights for policy 0, policy_version 527211 (0.0009) [2023-12-26 19:12:18,037][105692] Updated weights for policy 0, policy_version 527221 (0.0008) [2023-12-26 19:12:18,093][105692] Updated weights for policy 0, policy_version 527231 (0.0005) [2023-12-26 19:12:18,274][105620] Updated weights for policy 1, policy_version 527919 (0.0007) [2023-12-26 19:12:18,333][105620] Updated weights for policy 1, policy_version 527929 (0.0008) [2023-12-26 19:12:18,390][105620] Updated weights for policy 1, policy_version 527939 (0.0009) [2023-12-26 19:12:18,889][105692] Updated weights for policy 0, policy_version 527241 (0.0010) [2023-12-26 19:12:18,954][105692] Updated weights for policy 0, policy_version 527251 (0.0009) [2023-12-26 19:12:19,009][105692] Updated weights for policy 0, policy_version 527261 (0.0009) [2023-12-26 19:12:19,061][105620] Updated weights for policy 1, policy_version 527949 (0.0009) [2023-12-26 19:12:19,121][105620] Updated weights for policy 1, policy_version 527959 (0.0008) [2023-12-26 19:12:19,174][105620] Updated weights for policy 1, policy_version 527969 (0.0009) [2023-12-26 19:12:19,809][105692] Updated weights for policy 0, policy_version 527271 (0.0008) [2023-12-26 19:12:19,877][105692] Updated weights for policy 0, policy_version 527281 (0.0009) [2023-12-26 19:12:19,945][105692] Updated weights for policy 0, policy_version 527291 (0.0009) [2023-12-26 19:12:19,956][105620] Updated weights for policy 1, policy_version 527979 (0.0008) [2023-12-26 19:12:20,017][105620] Updated weights for policy 1, policy_version 527989 (0.0008) [2023-12-26 19:12:20,082][105620] Updated weights for policy 1, policy_version 527999 (0.0009) [2023-12-26 19:12:20,742][105692] Updated weights for policy 0, policy_version 527301 (0.0008) [2023-12-26 19:12:20,805][105692] Updated weights for policy 0, policy_version 527311 (0.0009) [2023-12-26 19:12:20,855][105620] Updated weights for policy 1, policy_version 528009 (0.0009) [2023-12-26 19:12:20,865][105692] Updated weights for policy 0, policy_version 527321 (0.0009) [2023-12-26 19:12:20,910][105620] Updated weights for policy 1, policy_version 528019 (0.0006) [2023-12-26 19:12:20,975][105620] Updated weights for policy 1, policy_version 528029 (0.0008) [2023-12-26 19:12:21,043][105620] Updated weights for policy 1, policy_version 528039 (0.0010) [2023-12-26 19:12:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 270204928. Throughput: 0: 9597.3, 1: 9688.7. Samples: 270190860. Policy #0 lag: (min: 29.0, avg: 52.3, max: 57.0) [2023-12-26 19:12:21,062][104569] Avg episode reward: [(0, '8730.584'), (1, '9079.197')] [2023-12-26 19:12:21,666][105692] Updated weights for policy 0, policy_version 527331 (0.0007) [2023-12-26 19:12:21,730][105692] Updated weights for policy 0, policy_version 527341 (0.0009) [2023-12-26 19:12:21,800][105692] Updated weights for policy 0, policy_version 527351 (0.0008) [2023-12-26 19:12:21,809][105620] Updated weights for policy 1, policy_version 528049 (0.0009) [2023-12-26 19:12:21,871][105620] Updated weights for policy 1, policy_version 528059 (0.0007) [2023-12-26 19:12:21,937][105620] Updated weights for policy 1, policy_version 528069 (0.0009) [2023-12-26 19:12:22,600][105692] Updated weights for policy 0, policy_version 527361 (0.0007) [2023-12-26 19:12:22,661][105692] Updated weights for policy 0, policy_version 527371 (0.0009) [2023-12-26 19:12:22,690][105620] Updated weights for policy 1, policy_version 528079 (0.0007) [2023-12-26 19:12:22,713][105692] Updated weights for policy 0, policy_version 527381 (0.0009) [2023-12-26 19:12:22,746][105620] Updated weights for policy 1, policy_version 528089 (0.0006) [2023-12-26 19:12:22,772][105692] Updated weights for policy 0, policy_version 527391 (0.0008) [2023-12-26 19:12:22,800][105620] Updated weights for policy 1, policy_version 528099 (0.0007) [2023-12-26 19:12:23,402][105692] Updated weights for policy 0, policy_version 527401 (0.0009) [2023-12-26 19:12:23,462][105692] Updated weights for policy 0, policy_version 527411 (0.0008) [2023-12-26 19:12:23,514][105692] Updated weights for policy 0, policy_version 527421 (0.0008) [2023-12-26 19:12:23,572][105620] Updated weights for policy 1, policy_version 528109 (0.0010) [2023-12-26 19:12:23,632][105620] Updated weights for policy 1, policy_version 528119 (0.0010) [2023-12-26 19:12:23,685][105620] Updated weights for policy 1, policy_version 528129 (0.0010) [2023-12-26 19:12:24,322][105692] Updated weights for policy 0, policy_version 527431 (0.0008) [2023-12-26 19:12:24,386][105692] Updated weights for policy 0, policy_version 527441 (0.0010) [2023-12-26 19:12:24,433][105620] Updated weights for policy 1, policy_version 528139 (0.0009) [2023-12-26 19:12:24,439][105692] Updated weights for policy 0, policy_version 527451 (0.0011) [2023-12-26 19:12:24,483][105620] Updated weights for policy 1, policy_version 528149 (0.0006) [2023-12-26 19:12:24,531][105620] Updated weights for policy 1, policy_version 528159 (0.0008) [2023-12-26 19:12:25,194][105692] Updated weights for policy 0, policy_version 527461 (0.0010) [2023-12-26 19:12:25,204][105620] Updated weights for policy 1, policy_version 528169 (0.0008) [2023-12-26 19:12:25,255][105692] Updated weights for policy 0, policy_version 527471 (0.0010) [2023-12-26 19:12:25,261][105620] Updated weights for policy 1, policy_version 528179 (0.0007) [2023-12-26 19:12:25,319][105692] Updated weights for policy 0, policy_version 527481 (0.0010) [2023-12-26 19:12:25,320][105620] Updated weights for policy 1, policy_version 528189 (0.0007) [2023-12-26 19:12:25,373][105620] Updated weights for policy 1, policy_version 528199 (0.0006) [2023-12-26 19:12:25,971][105692] Updated weights for policy 0, policy_version 527491 (0.0009) [2023-12-26 19:12:26,015][105692] Updated weights for policy 0, policy_version 527501 (0.0007) [2023-12-26 19:12:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19114.7, 300 sec: 19383.1). Total num frames: 270286848. Throughput: 0: 9463.4, 1: 9660.7. Samples: 270302376. Policy #0 lag: (min: 29.0, avg: 52.3, max: 57.0) [2023-12-26 19:12:26,062][104569] Avg episode reward: [(0, '8732.706'), (1, '9171.096')] [2023-12-26 19:12:26,073][105692] Updated weights for policy 0, policy_version 527511 (0.0010) [2023-12-26 19:12:26,114][105620] Updated weights for policy 1, policy_version 528209 (0.0010) [2023-12-26 19:12:26,172][105620] Updated weights for policy 1, policy_version 528219 (0.0010) [2023-12-26 19:12:26,230][105620] Updated weights for policy 1, policy_version 528229 (0.0010) [2023-12-26 19:12:26,706][105692] Updated weights for policy 0, policy_version 527521 (0.0010) [2023-12-26 19:12:26,758][105692] Updated weights for policy 0, policy_version 527531 (0.0005) [2023-12-26 19:12:26,815][105692] Updated weights for policy 0, policy_version 527541 (0.0006) [2023-12-26 19:12:26,871][105692] Updated weights for policy 0, policy_version 527551 (0.0006) [2023-12-26 19:12:26,971][105620] Updated weights for policy 1, policy_version 528239 (0.0010) [2023-12-26 19:12:27,029][105620] Updated weights for policy 1, policy_version 528249 (0.0010) [2023-12-26 19:12:27,097][105620] Updated weights for policy 1, policy_version 528259 (0.0010) [2023-12-26 19:12:27,415][105692] Updated weights for policy 0, policy_version 527561 (0.0008) [2023-12-26 19:12:27,426][105585] KL-divergence is very high: 568.2216 [2023-12-26 19:12:27,471][105585] KL-divergence is very high: 898.1559 [2023-12-26 19:12:27,472][105692] Updated weights for policy 0, policy_version 527571 (0.0008) [2023-12-26 19:12:27,511][105585] KL-divergence is very high: 845.4774 [2023-12-26 19:12:27,524][105692] Updated weights for policy 0, policy_version 527581 (0.0008) [2023-12-26 19:12:27,747][105620] Updated weights for policy 1, policy_version 528269 (0.0008) [2023-12-26 19:12:27,797][105620] Updated weights for policy 1, policy_version 528279 (0.0007) [2023-12-26 19:12:27,852][105620] Updated weights for policy 1, policy_version 528289 (0.0005) [2023-12-26 19:12:28,381][105620] Updated weights for policy 1, policy_version 528299 (0.0005) [2023-12-26 19:12:28,430][105692] Updated weights for policy 0, policy_version 527591 (0.0007) [2023-12-26 19:12:28,432][105620] Updated weights for policy 1, policy_version 528309 (0.0008) [2023-12-26 19:12:28,485][105620] Updated weights for policy 1, policy_version 528319 (0.0010) [2023-12-26 19:12:28,487][105692] Updated weights for policy 0, policy_version 527601 (0.0006) [2023-12-26 19:12:28,541][105692] Updated weights for policy 0, policy_version 527611 (0.0006) [2023-12-26 19:12:29,083][105620] Updated weights for policy 1, policy_version 528329 (0.0010) [2023-12-26 19:12:29,130][105620] Updated weights for policy 1, policy_version 528339 (0.0005) [2023-12-26 19:12:29,183][105620] Updated weights for policy 1, policy_version 528349 (0.0005) [2023-12-26 19:12:29,239][105620] Updated weights for policy 1, policy_version 528359 (0.0007) [2023-12-26 19:12:29,432][105692] Updated weights for policy 0, policy_version 527621 (0.0008) [2023-12-26 19:12:29,496][105692] Updated weights for policy 0, policy_version 527631 (0.0008) [2023-12-26 19:12:29,555][105692] Updated weights for policy 0, policy_version 527641 (0.0008) [2023-12-26 19:12:29,995][105620] Updated weights for policy 1, policy_version 528369 (0.0009) [2023-12-26 19:12:30,054][105620] Updated weights for policy 1, policy_version 528379 (0.0009) [2023-12-26 19:12:30,104][105620] Updated weights for policy 1, policy_version 528389 (0.0008) [2023-12-26 19:12:30,304][105692] Updated weights for policy 0, policy_version 527651 (0.0008) [2023-12-26 19:12:30,363][105692] Updated weights for policy 0, policy_version 527661 (0.0009) [2023-12-26 19:12:30,411][105692] Updated weights for policy 0, policy_version 527671 (0.0009) [2023-12-26 19:12:30,852][105620] Updated weights for policy 1, policy_version 528399 (0.0007) [2023-12-26 19:12:30,905][105620] Updated weights for policy 1, policy_version 528409 (0.0008) [2023-12-26 19:12:30,964][105620] Updated weights for policy 1, policy_version 528419 (0.0008) [2023-12-26 19:12:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 270393344. Throughput: 0: 9492.9, 1: 9734.6. Samples: 270364712. Policy #0 lag: (min: 29.0, avg: 52.3, max: 57.0) [2023-12-26 19:12:31,062][104569] Avg episode reward: [(0, '8905.894'), (1, '9081.021')] [2023-12-26 19:12:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000528424_135290880.pth... [2023-12-26 19:12:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000527680_135102464.pth... [2023-12-26 19:12:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000526592_134823936.pth [2023-12-26 19:12:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000527272_134995968.pth [2023-12-26 19:12:31,106][105692] Updated weights for policy 0, policy_version 527681 (0.0008) [2023-12-26 19:12:31,166][105692] Updated weights for policy 0, policy_version 527691 (0.0008) [2023-12-26 19:12:31,213][105692] Updated weights for policy 0, policy_version 527701 (0.0005) [2023-12-26 19:12:31,272][105692] Updated weights for policy 0, policy_version 527711 (0.0006) [2023-12-26 19:12:31,794][105620] Updated weights for policy 1, policy_version 528429 (0.0007) [2023-12-26 19:12:31,845][105620] Updated weights for policy 1, policy_version 528439 (0.0009) [2023-12-26 19:12:31,895][105692] Updated weights for policy 0, policy_version 527721 (0.0008) [2023-12-26 19:12:31,898][105620] Updated weights for policy 1, policy_version 528449 (0.0006) [2023-12-26 19:12:31,948][105692] Updated weights for policy 0, policy_version 527731 (0.0008) [2023-12-26 19:12:32,010][105692] Updated weights for policy 0, policy_version 527741 (0.0008) [2023-12-26 19:12:32,677][105620] Updated weights for policy 1, policy_version 528459 (0.0009) [2023-12-26 19:12:32,740][105620] Updated weights for policy 1, policy_version 528469 (0.0009) [2023-12-26 19:12:32,763][105692] Updated weights for policy 0, policy_version 527751 (0.0008) [2023-12-26 19:12:32,792][105620] Updated weights for policy 1, policy_version 528479 (0.0009) [2023-12-26 19:12:32,815][105692] Updated weights for policy 0, policy_version 527761 (0.0007) [2023-12-26 19:12:32,878][105692] Updated weights for policy 0, policy_version 527771 (0.0007) [2023-12-26 19:12:33,517][105620] Updated weights for policy 1, policy_version 528489 (0.0006) [2023-12-26 19:12:33,578][105620] Updated weights for policy 1, policy_version 528499 (0.0009) [2023-12-26 19:12:33,636][105620] Updated weights for policy 1, policy_version 528509 (0.0007) [2023-12-26 19:12:33,638][105692] Updated weights for policy 0, policy_version 527781 (0.0009) [2023-12-26 19:12:33,691][105620] Updated weights for policy 1, policy_version 528519 (0.0006) [2023-12-26 19:12:33,697][105692] Updated weights for policy 0, policy_version 527791 (0.0007) [2023-12-26 19:12:33,738][105585] KL-divergence is very high: 114.7499 [2023-12-26 19:12:33,750][105692] Updated weights for policy 0, policy_version 527801 (0.0009) [2023-12-26 19:12:33,786][105585] KL-divergence is very high: 102.8863 [2023-12-26 19:12:34,371][105620] Updated weights for policy 1, policy_version 528529 (0.0009) [2023-12-26 19:12:34,428][105620] Updated weights for policy 1, policy_version 528539 (0.0008) [2023-12-26 19:12:34,491][105620] Updated weights for policy 1, policy_version 528549 (0.0009) [2023-12-26 19:12:34,543][105692] Updated weights for policy 0, policy_version 527811 (0.0009) [2023-12-26 19:12:34,607][105692] Updated weights for policy 0, policy_version 527821 (0.0009) [2023-12-26 19:12:34,670][105692] Updated weights for policy 0, policy_version 527831 (0.0009) [2023-12-26 19:12:35,219][105620] Updated weights for policy 1, policy_version 528559 (0.0008) [2023-12-26 19:12:35,268][105620] Updated weights for policy 1, policy_version 528569 (0.0008) [2023-12-26 19:12:35,314][105620] Updated weights for policy 1, policy_version 528579 (0.0008) [2023-12-26 19:12:35,374][105692] Updated weights for policy 0, policy_version 527841 (0.0009) [2023-12-26 19:12:35,429][105692] Updated weights for policy 0, policy_version 527851 (0.0009) [2023-12-26 19:12:35,487][105692] Updated weights for policy 0, policy_version 527861 (0.0009) [2023-12-26 19:12:35,549][105692] Updated weights for policy 0, policy_version 527871 (0.0009) [2023-12-26 19:12:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.3, 300 sec: 19410.9). Total num frames: 270483456. Throughput: 0: 9395.4, 1: 9759.6. Samples: 270478112. Policy #0 lag: (min: 29.0, avg: 52.3, max: 57.0) [2023-12-26 19:12:36,062][104569] Avg episode reward: [(0, '8989.374'), (1, '8989.965')] [2023-12-26 19:12:36,072][105620] Updated weights for policy 1, policy_version 528589 (0.0009) [2023-12-26 19:12:36,137][105620] Updated weights for policy 1, policy_version 528599 (0.0008) [2023-12-26 19:12:36,200][105620] Updated weights for policy 1, policy_version 528609 (0.0008) [2023-12-26 19:12:36,270][105692] Updated weights for policy 0, policy_version 527881 (0.0006) [2023-12-26 19:12:36,324][105692] Updated weights for policy 0, policy_version 527891 (0.0006) [2023-12-26 19:12:36,387][105692] Updated weights for policy 0, policy_version 527901 (0.0010) [2023-12-26 19:12:36,800][105620] Updated weights for policy 1, policy_version 528619 (0.0008) [2023-12-26 19:12:36,863][105620] Updated weights for policy 1, policy_version 528629 (0.0007) [2023-12-26 19:12:36,924][105620] Updated weights for policy 1, policy_version 528639 (0.0007) [2023-12-26 19:12:37,217][105692] Updated weights for policy 0, policy_version 527911 (0.0008) [2023-12-26 19:12:37,268][105692] Updated weights for policy 0, policy_version 527921 (0.0009) [2023-12-26 19:12:37,321][105692] Updated weights for policy 0, policy_version 527931 (0.0006) [2023-12-26 19:12:37,650][105620] Updated weights for policy 1, policy_version 528649 (0.0008) [2023-12-26 19:12:37,706][105620] Updated weights for policy 1, policy_version 528659 (0.0009) [2023-12-26 19:12:37,762][105620] Updated weights for policy 1, policy_version 528669 (0.0009) [2023-12-26 19:12:37,827][105620] Updated weights for policy 1, policy_version 528679 (0.0009) [2023-12-26 19:12:38,066][105692] Updated weights for policy 0, policy_version 527941 (0.0007) [2023-12-26 19:12:38,114][105692] Updated weights for policy 0, policy_version 527951 (0.0009) [2023-12-26 19:12:38,174][105692] Updated weights for policy 0, policy_version 527961 (0.0007) [2023-12-26 19:12:38,447][105620] Updated weights for policy 1, policy_version 528689 (0.0006) [2023-12-26 19:12:38,500][105620] Updated weights for policy 1, policy_version 528699 (0.0005) [2023-12-26 19:12:38,557][105620] Updated weights for policy 1, policy_version 528709 (0.0008) [2023-12-26 19:12:39,105][105692] Updated weights for policy 0, policy_version 527971 (0.0009) [2023-12-26 19:12:39,133][105620] Updated weights for policy 1, policy_version 528719 (0.0006) [2023-12-26 19:12:39,159][105692] Updated weights for policy 0, policy_version 527981 (0.0007) [2023-12-26 19:12:39,198][105620] Updated weights for policy 1, policy_version 528729 (0.0005) [2023-12-26 19:12:39,221][105692] Updated weights for policy 0, policy_version 527991 (0.0009) [2023-12-26 19:12:39,267][105620] Updated weights for policy 1, policy_version 528739 (0.0008) [2023-12-26 19:12:40,011][105620] Updated weights for policy 1, policy_version 528749 (0.0008) [2023-12-26 19:12:40,012][105692] Updated weights for policy 0, policy_version 528001 (0.0008) [2023-12-26 19:12:40,076][105620] Updated weights for policy 1, policy_version 528759 (0.0009) [2023-12-26 19:12:40,078][105692] Updated weights for policy 0, policy_version 528011 (0.0007) [2023-12-26 19:12:40,140][105692] Updated weights for policy 0, policy_version 528021 (0.0006) [2023-12-26 19:12:40,142][105620] Updated weights for policy 1, policy_version 528769 (0.0007) [2023-12-26 19:12:40,203][105692] Updated weights for policy 0, policy_version 528031 (0.0007) [2023-12-26 19:12:40,899][105620] Updated weights for policy 1, policy_version 528779 (0.0006) [2023-12-26 19:12:40,942][105692] Updated weights for policy 0, policy_version 528041 (0.0009) [2023-12-26 19:12:40,957][105620] Updated weights for policy 1, policy_version 528789 (0.0006) [2023-12-26 19:12:40,992][105692] Updated weights for policy 0, policy_version 528051 (0.0008) [2023-12-26 19:12:41,015][105620] Updated weights for policy 1, policy_version 528799 (0.0006) [2023-12-26 19:12:41,054][105692] Updated weights for policy 0, policy_version 528061 (0.0007) [2023-12-26 19:12:41,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19114.7, 300 sec: 19383.1). Total num frames: 270573568. Throughput: 0: 9377.0, 1: 9802.1. Samples: 270592488. Policy #0 lag: (min: 29.0, avg: 52.3, max: 57.0) [2023-12-26 19:12:41,063][104569] Avg episode reward: [(0, '8990.970'), (1, '9082.154')] [2023-12-26 19:12:41,714][105692] Updated weights for policy 0, policy_version 528071 (0.0009) [2023-12-26 19:12:41,778][105692] Updated weights for policy 0, policy_version 528081 (0.0009) [2023-12-26 19:12:41,816][105620] Updated weights for policy 1, policy_version 528809 (0.0008) [2023-12-26 19:12:41,838][105692] Updated weights for policy 0, policy_version 528091 (0.0009) [2023-12-26 19:12:41,865][105620] Updated weights for policy 1, policy_version 528819 (0.0006) [2023-12-26 19:12:41,912][105620] Updated weights for policy 1, policy_version 528829 (0.0009) [2023-12-26 19:12:41,959][105620] Updated weights for policy 1, policy_version 528839 (0.0009) [2023-12-26 19:12:42,571][105692] Updated weights for policy 0, policy_version 528101 (0.0010) [2023-12-26 19:12:42,632][105692] Updated weights for policy 0, policy_version 528111 (0.0008) [2023-12-26 19:12:42,697][105692] Updated weights for policy 0, policy_version 528121 (0.0009) [2023-12-26 19:12:42,776][105620] Updated weights for policy 1, policy_version 528849 (0.0008) [2023-12-26 19:12:42,829][105620] Updated weights for policy 1, policy_version 528859 (0.0006) [2023-12-26 19:12:42,876][105620] Updated weights for policy 1, policy_version 528869 (0.0008) [2023-12-26 19:12:43,415][105692] Updated weights for policy 0, policy_version 528131 (0.0009) [2023-12-26 19:12:43,469][105692] Updated weights for policy 0, policy_version 528141 (0.0009) [2023-12-26 19:12:43,478][105620] Updated weights for policy 1, policy_version 528879 (0.0006) [2023-12-26 19:12:43,525][105692] Updated weights for policy 0, policy_version 528151 (0.0009) [2023-12-26 19:12:43,534][105620] Updated weights for policy 1, policy_version 528889 (0.0006) [2023-12-26 19:12:43,587][105620] Updated weights for policy 1, policy_version 528899 (0.0005) [2023-12-26 19:12:44,124][105620] Updated weights for policy 1, policy_version 528909 (0.0007) [2023-12-26 19:12:44,181][105620] Updated weights for policy 1, policy_version 528919 (0.0009) [2023-12-26 19:12:44,232][105620] Updated weights for policy 1, policy_version 528929 (0.0009) [2023-12-26 19:12:44,345][105692] Updated weights for policy 0, policy_version 528161 (0.0008) [2023-12-26 19:12:44,404][105692] Updated weights for policy 0, policy_version 528171 (0.0005) [2023-12-26 19:12:44,469][105692] Updated weights for policy 0, policy_version 528181 (0.0005) [2023-12-26 19:12:44,525][105692] Updated weights for policy 0, policy_version 528191 (0.0005) [2023-12-26 19:12:44,911][105620] Updated weights for policy 1, policy_version 528939 (0.0008) [2023-12-26 19:12:44,973][105620] Updated weights for policy 1, policy_version 528949 (0.0006) [2023-12-26 19:12:45,043][105620] Updated weights for policy 1, policy_version 528959 (0.0006) [2023-12-26 19:12:45,137][105692] Updated weights for policy 0, policy_version 528201 (0.0007) [2023-12-26 19:12:45,190][105692] Updated weights for policy 0, policy_version 528211 (0.0009) [2023-12-26 19:12:45,256][105692] Updated weights for policy 0, policy_version 528221 (0.0008) [2023-12-26 19:12:45,753][105620] Updated weights for policy 1, policy_version 528969 (0.0008) [2023-12-26 19:12:45,811][105620] Updated weights for policy 1, policy_version 528979 (0.0009) [2023-12-26 19:12:45,868][105620] Updated weights for policy 1, policy_version 528989 (0.0008) [2023-12-26 19:12:45,915][105620] Updated weights for policy 1, policy_version 528999 (0.0009) [2023-12-26 19:12:46,011][105692] Updated weights for policy 0, policy_version 528231 (0.0009) [2023-12-26 19:12:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19114.7, 300 sec: 19383.1). Total num frames: 270680064. Throughput: 0: 9427.7, 1: 9807.6. Samples: 270651480. Policy #0 lag: (min: 29.0, avg: 52.3, max: 57.0) [2023-12-26 19:12:46,062][104569] Avg episode reward: [(0, '1593.401'), (1, '9172.270')] [2023-12-26 19:12:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000529000_135438336.pth... [2023-12-26 19:12:46,074][105692] Updated weights for policy 0, policy_version 528241 (0.0009) [2023-12-26 19:12:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000527848_135143424.pth [2023-12-26 19:12:46,136][105692] Updated weights for policy 0, policy_version 528251 (0.0009) [2023-12-26 19:12:46,164][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000528256_135249920.pth... [2023-12-26 19:12:46,168][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000527168_134971392.pth [2023-12-26 19:12:46,652][105620] Updated weights for policy 1, policy_version 529009 (0.0009) [2023-12-26 19:12:46,705][105620] Updated weights for policy 1, policy_version 529019 (0.0009) [2023-12-26 19:12:46,770][105620] Updated weights for policy 1, policy_version 529029 (0.0009) [2023-12-26 19:12:46,884][105692] Updated weights for policy 0, policy_version 528261 (0.0008) [2023-12-26 19:12:46,930][105692] Updated weights for policy 0, policy_version 528271 (0.0008) [2023-12-26 19:12:46,976][105692] Updated weights for policy 0, policy_version 528281 (0.0009) [2023-12-26 19:12:47,510][105620] Updated weights for policy 1, policy_version 529039 (0.0006) [2023-12-26 19:12:47,569][105620] Updated weights for policy 1, policy_version 529049 (0.0005) [2023-12-26 19:12:47,635][105620] Updated weights for policy 1, policy_version 529059 (0.0005) [2023-12-26 19:12:47,814][105692] Updated weights for policy 0, policy_version 528291 (0.0009) [2023-12-26 19:12:47,875][105692] Updated weights for policy 0, policy_version 528301 (0.0010) [2023-12-26 19:12:47,927][105692] Updated weights for policy 0, policy_version 528311 (0.0009) [2023-12-26 19:12:48,138][105620] Updated weights for policy 1, policy_version 529069 (0.0005) [2023-12-26 19:12:48,186][105620] Updated weights for policy 1, policy_version 529079 (0.0005) [2023-12-26 19:12:48,247][105620] Updated weights for policy 1, policy_version 529089 (0.0005) [2023-12-26 19:12:48,690][105692] Updated weights for policy 0, policy_version 528321 (0.0008) [2023-12-26 19:12:48,760][105692] Updated weights for policy 0, policy_version 528331 (0.0006) [2023-12-26 19:12:48,826][105692] Updated weights for policy 0, policy_version 528341 (0.0006) [2023-12-26 19:12:48,887][105692] Updated weights for policy 0, policy_version 528351 (0.0008) [2023-12-26 19:12:48,994][105620] Updated weights for policy 1, policy_version 529099 (0.0007) [2023-12-26 19:12:49,056][105620] Updated weights for policy 1, policy_version 529109 (0.0009) [2023-12-26 19:12:49,116][105620] Updated weights for policy 1, policy_version 529119 (0.0009) [2023-12-26 19:12:49,579][105692] Updated weights for policy 0, policy_version 528361 (0.0009) [2023-12-26 19:12:49,640][105692] Updated weights for policy 0, policy_version 528371 (0.0009) [2023-12-26 19:12:49,687][105692] Updated weights for policy 0, policy_version 528381 (0.0009) [2023-12-26 19:12:49,857][105620] Updated weights for policy 1, policy_version 529129 (0.0009) [2023-12-26 19:12:49,925][105620] Updated weights for policy 1, policy_version 529139 (0.0009) [2023-12-26 19:12:49,988][105620] Updated weights for policy 1, policy_version 529149 (0.0009) [2023-12-26 19:12:50,046][105620] Updated weights for policy 1, policy_version 529159 (0.0009) [2023-12-26 19:12:50,467][105692] Updated weights for policy 0, policy_version 528391 (0.0010) [2023-12-26 19:12:50,525][105692] Updated weights for policy 0, policy_version 528401 (0.0009) [2023-12-26 19:12:50,585][105692] Updated weights for policy 0, policy_version 528411 (0.0008) [2023-12-26 19:12:50,766][105620] Updated weights for policy 1, policy_version 529169 (0.0009) [2023-12-26 19:12:50,832][105620] Updated weights for policy 1, policy_version 529179 (0.0009) [2023-12-26 19:12:50,890][105620] Updated weights for policy 1, policy_version 529189 (0.0009) [2023-12-26 19:12:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 270778368. Throughput: 0: 9335.3, 1: 9861.9. Samples: 270767652. Policy #0 lag: (min: 29.0, avg: 52.3, max: 57.0) [2023-12-26 19:12:51,063][104569] Avg episode reward: [(0, '1633.382'), (1, '9170.864')] [2023-12-26 19:12:51,384][105692] Updated weights for policy 0, policy_version 528421 (0.0009) [2023-12-26 19:12:51,449][105692] Updated weights for policy 0, policy_version 528431 (0.0009) [2023-12-26 19:12:51,510][105692] Updated weights for policy 0, policy_version 528441 (0.0011) [2023-12-26 19:12:51,590][105620] Updated weights for policy 1, policy_version 529199 (0.0006) [2023-12-26 19:12:51,661][105620] Updated weights for policy 1, policy_version 529209 (0.0007) [2023-12-26 19:12:51,718][105620] Updated weights for policy 1, policy_version 529220 (0.0009) [2023-12-26 19:12:52,187][105692] Updated weights for policy 0, policy_version 528451 (0.0009) [2023-12-26 19:12:52,249][105692] Updated weights for policy 0, policy_version 528461 (0.0009) [2023-12-26 19:12:52,303][105692] Updated weights for policy 0, policy_version 528471 (0.0008) [2023-12-26 19:12:52,448][105620] Updated weights for policy 1, policy_version 529230 (0.0010) [2023-12-26 19:12:52,511][105620] Updated weights for policy 1, policy_version 529240 (0.0010) [2023-12-26 19:12:52,556][105620] Updated weights for policy 1, policy_version 529250 (0.0010) [2023-12-26 19:12:53,031][105692] Updated weights for policy 0, policy_version 528481 (0.0008) [2023-12-26 19:12:53,083][105692] Updated weights for policy 0, policy_version 528491 (0.0010) [2023-12-26 19:12:53,138][105692] Updated weights for policy 0, policy_version 528501 (0.0010) [2023-12-26 19:12:53,199][105692] Updated weights for policy 0, policy_version 528511 (0.0010) [2023-12-26 19:12:53,318][105620] Updated weights for policy 1, policy_version 529260 (0.0010) [2023-12-26 19:12:53,386][105620] Updated weights for policy 1, policy_version 529270 (0.0010) [2023-12-26 19:12:53,443][105620] Updated weights for policy 1, policy_version 529280 (0.0010) [2023-12-26 19:12:53,886][105692] Updated weights for policy 0, policy_version 528521 (0.0009) [2023-12-26 19:12:53,949][105692] Updated weights for policy 0, policy_version 528531 (0.0008) [2023-12-26 19:12:54,010][105692] Updated weights for policy 0, policy_version 528541 (0.0007) [2023-12-26 19:12:54,159][105620] Updated weights for policy 1, policy_version 529290 (0.0010) [2023-12-26 19:12:54,220][105620] Updated weights for policy 1, policy_version 529300 (0.0010) [2023-12-26 19:12:54,279][105620] Updated weights for policy 1, policy_version 529310 (0.0010) [2023-12-26 19:12:54,339][105620] Updated weights for policy 1, policy_version 529320 (0.0010) [2023-12-26 19:12:54,702][105692] Updated weights for policy 0, policy_version 528551 (0.0007) [2023-12-26 19:12:54,753][105692] Updated weights for policy 0, policy_version 528561 (0.0008) [2023-12-26 19:12:54,805][105692] Updated weights for policy 0, policy_version 528571 (0.0008) [2023-12-26 19:12:55,086][105620] Updated weights for policy 1, policy_version 529330 (0.0010) [2023-12-26 19:12:55,138][105620] Updated weights for policy 1, policy_version 529340 (0.0010) [2023-12-26 19:12:55,189][105620] Updated weights for policy 1, policy_version 529350 (0.0010) [2023-12-26 19:12:55,522][105692] Updated weights for policy 0, policy_version 528581 (0.0009) [2023-12-26 19:12:55,587][105692] Updated weights for policy 0, policy_version 528591 (0.0010) [2023-12-26 19:12:55,655][105692] Updated weights for policy 0, policy_version 528601 (0.0009) [2023-12-26 19:12:55,940][105620] Updated weights for policy 1, policy_version 529360 (0.0010) [2023-12-26 19:12:56,008][105620] Updated weights for policy 1, policy_version 529370 (0.0006) [2023-12-26 19:12:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19383.1). Total num frames: 270868480. Throughput: 0: 9335.4, 1: 9836.3. Samples: 270882428. Policy #0 lag: (min: 29.0, avg: 52.3, max: 57.0) [2023-12-26 19:12:56,062][104569] Avg episode reward: [(0, '1743.704'), (1, '9169.415')] [2023-12-26 19:12:56,079][105620] Updated weights for policy 1, policy_version 529380 (0.0006) [2023-12-26 19:12:56,266][105692] Updated weights for policy 0, policy_version 528611 (0.0010) [2023-12-26 19:12:56,328][105692] Updated weights for policy 0, policy_version 528621 (0.0008) [2023-12-26 19:12:56,370][105692] Updated weights for policy 0, policy_version 528631 (0.0007) [2023-12-26 19:12:56,767][105620] Updated weights for policy 1, policy_version 529390 (0.0010) [2023-12-26 19:12:56,815][105620] Updated weights for policy 1, policy_version 529400 (0.0010) [2023-12-26 19:12:56,860][105620] Updated weights for policy 1, policy_version 529410 (0.0010) [2023-12-26 19:12:57,103][105692] Updated weights for policy 0, policy_version 528641 (0.0009) [2023-12-26 19:12:57,154][105692] Updated weights for policy 0, policy_version 528651 (0.0010) [2023-12-26 19:12:57,207][105692] Updated weights for policy 0, policy_version 528661 (0.0010) [2023-12-26 19:12:57,270][105692] Updated weights for policy 0, policy_version 528671 (0.0009) [2023-12-26 19:12:57,622][105620] Updated weights for policy 1, policy_version 529420 (0.0010) [2023-12-26 19:12:57,669][105620] Updated weights for policy 1, policy_version 529430 (0.0010) [2023-12-26 19:12:57,728][105620] Updated weights for policy 1, policy_version 529440 (0.0010) [2023-12-26 19:12:57,881][105692] Updated weights for policy 0, policy_version 528681 (0.0008) [2023-12-26 19:12:57,929][105692] Updated weights for policy 0, policy_version 528691 (0.0008) [2023-12-26 19:12:57,988][105692] Updated weights for policy 0, policy_version 528701 (0.0008) [2023-12-26 19:12:58,477][105620] Updated weights for policy 1, policy_version 529450 (0.0010) [2023-12-26 19:12:58,544][105620] Updated weights for policy 1, policy_version 529460 (0.0007) [2023-12-26 19:12:58,610][105620] Updated weights for policy 1, policy_version 529470 (0.0008) [2023-12-26 19:12:58,673][105620] Updated weights for policy 1, policy_version 529480 (0.0008) [2023-12-26 19:12:58,768][105692] Updated weights for policy 0, policy_version 528711 (0.0008) [2023-12-26 19:12:58,841][105692] Updated weights for policy 0, policy_version 528721 (0.0008) [2023-12-26 19:12:58,910][105692] Updated weights for policy 0, policy_version 528731 (0.0008) [2023-12-26 19:12:59,507][105620] Updated weights for policy 1, policy_version 529490 (0.0010) [2023-12-26 19:12:59,576][105620] Updated weights for policy 1, policy_version 529500 (0.0008) [2023-12-26 19:12:59,625][105692] Updated weights for policy 0, policy_version 528741 (0.0009) [2023-12-26 19:12:59,638][105620] Updated weights for policy 1, policy_version 529510 (0.0010) [2023-12-26 19:12:59,678][105692] Updated weights for policy 0, policy_version 528751 (0.0006) [2023-12-26 19:12:59,730][105692] Updated weights for policy 0, policy_version 528761 (0.0007) [2023-12-26 19:13:00,321][105620] Updated weights for policy 1, policy_version 529520 (0.0008) [2023-12-26 19:13:00,388][105620] Updated weights for policy 1, policy_version 529530 (0.0006) [2023-12-26 19:13:00,454][105620] Updated weights for policy 1, policy_version 529540 (0.0005) [2023-12-26 19:13:00,480][105692] Updated weights for policy 0, policy_version 528771 (0.0009) [2023-12-26 19:13:00,546][105692] Updated weights for policy 0, policy_version 528781 (0.0010) [2023-12-26 19:13:00,605][105692] Updated weights for policy 0, policy_version 528791 (0.0008) [2023-12-26 19:13:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19114.6, 300 sec: 19355.3). Total num frames: 270966784. Throughput: 0: 9417.7, 1: 9782.6. Samples: 270940696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:13:01,063][104569] Avg episode reward: [(0, '6502.385'), (1, '9081.275')] [2023-12-26 19:13:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000528800_135389184.pth... [2023-12-26 19:13:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000529544_135577600.pth... [2023-12-26 19:13:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000528424_135290880.pth [2023-12-26 19:13:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000527680_135102464.pth [2023-12-26 19:13:01,159][105620] Updated weights for policy 1, policy_version 529550 (0.0008) [2023-12-26 19:13:01,224][105692] Updated weights for policy 0, policy_version 528801 (0.0006) [2023-12-26 19:13:01,224][105620] Updated weights for policy 1, policy_version 529560 (0.0007) [2023-12-26 19:13:01,283][105692] Updated weights for policy 0, policy_version 528811 (0.0007) [2023-12-26 19:13:01,291][105620] Updated weights for policy 1, policy_version 529570 (0.0007) [2023-12-26 19:13:01,337][105692] Updated weights for policy 0, policy_version 528821 (0.0008) [2023-12-26 19:13:01,402][105692] Updated weights for policy 0, policy_version 528831 (0.0009) [2023-12-26 19:13:01,947][105620] Updated weights for policy 1, policy_version 529580 (0.0006) [2023-12-26 19:13:02,005][105620] Updated weights for policy 1, policy_version 529590 (0.0006) [2023-12-26 19:13:02,061][105620] Updated weights for policy 1, policy_version 529600 (0.0005) [2023-12-26 19:13:02,130][105692] Updated weights for policy 0, policy_version 528841 (0.0010) [2023-12-26 19:13:02,178][105692] Updated weights for policy 0, policy_version 528851 (0.0011) [2023-12-26 19:13:02,237][105692] Updated weights for policy 0, policy_version 528861 (0.0010) [2023-12-26 19:13:02,666][105620] Updated weights for policy 1, policy_version 529610 (0.0006) [2023-12-26 19:13:02,730][105620] Updated weights for policy 1, policy_version 529620 (0.0010) [2023-12-26 19:13:02,784][105620] Updated weights for policy 1, policy_version 529630 (0.0010) [2023-12-26 19:13:02,843][105620] Updated weights for policy 1, policy_version 529640 (0.0010) [2023-12-26 19:13:02,957][105692] Updated weights for policy 0, policy_version 528871 (0.0008) [2023-12-26 19:13:03,026][105692] Updated weights for policy 0, policy_version 528881 (0.0008) [2023-12-26 19:13:03,087][105692] Updated weights for policy 0, policy_version 528891 (0.0009) [2023-12-26 19:13:03,458][105620] Updated weights for policy 1, policy_version 529650 (0.0010) [2023-12-26 19:13:03,512][105620] Updated weights for policy 1, policy_version 529660 (0.0010) [2023-12-26 19:13:03,570][105620] Updated weights for policy 1, policy_version 529670 (0.0010) [2023-12-26 19:13:03,781][105692] Updated weights for policy 0, policy_version 528901 (0.0007) [2023-12-26 19:13:03,827][105692] Updated weights for policy 0, policy_version 528911 (0.0005) [2023-12-26 19:13:03,885][105692] Updated weights for policy 0, policy_version 528921 (0.0010) [2023-12-26 19:13:04,261][105620] Updated weights for policy 1, policy_version 529680 (0.0007) [2023-12-26 19:13:04,325][105620] Updated weights for policy 1, policy_version 529690 (0.0005) [2023-12-26 19:13:04,384][105620] Updated weights for policy 1, policy_version 529700 (0.0010) [2023-12-26 19:13:04,659][105692] Updated weights for policy 0, policy_version 528931 (0.0006) [2023-12-26 19:13:04,710][105692] Updated weights for policy 0, policy_version 528941 (0.0005) [2023-12-26 19:13:04,765][105692] Updated weights for policy 0, policy_version 528951 (0.0008) [2023-12-26 19:13:05,001][105620] Updated weights for policy 1, policy_version 529710 (0.0007) [2023-12-26 19:13:05,062][105620] Updated weights for policy 1, policy_version 529720 (0.0009) [2023-12-26 19:13:05,114][105620] Updated weights for policy 1, policy_version 529730 (0.0008) [2023-12-26 19:13:05,596][105692] Updated weights for policy 0, policy_version 528963 (0.0010) [2023-12-26 19:13:05,650][105692] Updated weights for policy 0, policy_version 528973 (0.0008) [2023-12-26 19:13:05,706][105692] Updated weights for policy 0, policy_version 528983 (0.0010) [2023-12-26 19:13:05,720][105620] Updated weights for policy 1, policy_version 529740 (0.0006) [2023-12-26 19:13:05,775][105620] Updated weights for policy 1, policy_version 529750 (0.0010) [2023-12-26 19:13:05,829][105620] Updated weights for policy 1, policy_version 529760 (0.0010) [2023-12-26 19:13:06,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 271073280. Throughput: 0: 9500.0, 1: 9824.2. Samples: 271060448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:13:06,062][104569] Avg episode reward: [(0, '7920.658'), (1, '9081.447')] [2023-12-26 19:13:06,455][105692] Updated weights for policy 0, policy_version 528993 (0.0008) [2023-12-26 19:13:06,516][105692] Updated weights for policy 0, policy_version 529003 (0.0008) [2023-12-26 19:13:06,582][105692] Updated weights for policy 0, policy_version 529013 (0.0008) [2023-12-26 19:13:06,589][105620] Updated weights for policy 1, policy_version 529770 (0.0011) [2023-12-26 19:13:06,643][105692] Updated weights for policy 0, policy_version 529023 (0.0008) [2023-12-26 19:13:06,653][105620] Updated weights for policy 1, policy_version 529780 (0.0011) [2023-12-26 19:13:06,712][105620] Updated weights for policy 1, policy_version 529790 (0.0011) [2023-12-26 19:13:06,778][105620] Updated weights for policy 1, policy_version 529800 (0.0011) [2023-12-26 19:13:07,360][105692] Updated weights for policy 0, policy_version 529033 (0.0008) [2023-12-26 19:13:07,409][105692] Updated weights for policy 0, policy_version 529043 (0.0008) [2023-12-26 19:13:07,459][105692] Updated weights for policy 0, policy_version 529053 (0.0009) [2023-12-26 19:13:07,515][105620] Updated weights for policy 1, policy_version 529810 (0.0009) [2023-12-26 19:13:07,560][105620] Updated weights for policy 1, policy_version 529820 (0.0010) [2023-12-26 19:13:07,608][105620] Updated weights for policy 1, policy_version 529830 (0.0010) [2023-12-26 19:13:08,275][105692] Updated weights for policy 0, policy_version 529063 (0.0007) [2023-12-26 19:13:08,281][105620] Updated weights for policy 1, policy_version 529840 (0.0009) [2023-12-26 19:13:08,337][105692] Updated weights for policy 0, policy_version 529073 (0.0007) [2023-12-26 19:13:08,348][105620] Updated weights for policy 1, policy_version 529850 (0.0007) [2023-12-26 19:13:08,390][105692] Updated weights for policy 0, policy_version 529083 (0.0009) [2023-12-26 19:13:08,402][105620] Updated weights for policy 1, policy_version 529860 (0.0006) [2023-12-26 19:13:09,115][105692] Updated weights for policy 0, policy_version 529093 (0.0008) [2023-12-26 19:13:09,169][105620] Updated weights for policy 1, policy_version 529870 (0.0006) [2023-12-26 19:13:09,175][105692] Updated weights for policy 0, policy_version 529103 (0.0009) [2023-12-26 19:13:09,231][105620] Updated weights for policy 1, policy_version 529880 (0.0006) [2023-12-26 19:13:09,242][105692] Updated weights for policy 0, policy_version 529113 (0.0006) [2023-12-26 19:13:09,291][105620] Updated weights for policy 1, policy_version 529890 (0.0007) [2023-12-26 19:13:09,936][105692] Updated weights for policy 0, policy_version 529123 (0.0009) [2023-12-26 19:13:09,990][105692] Updated weights for policy 0, policy_version 529133 (0.0009) [2023-12-26 19:13:10,035][105620] Updated weights for policy 1, policy_version 529900 (0.0008) [2023-12-26 19:13:10,053][105692] Updated weights for policy 0, policy_version 529143 (0.0006) [2023-12-26 19:13:10,107][105620] Updated weights for policy 1, policy_version 529910 (0.0008) [2023-12-26 19:13:10,175][105620] Updated weights for policy 1, policy_version 529920 (0.0010) [2023-12-26 19:13:10,812][105692] Updated weights for policy 0, policy_version 529153 (0.0006) [2023-12-26 19:13:10,825][105620] Updated weights for policy 1, policy_version 529930 (0.0010) [2023-12-26 19:13:10,859][105692] Updated weights for policy 0, policy_version 529163 (0.0007) [2023-12-26 19:13:10,883][105620] Updated weights for policy 1, policy_version 529940 (0.0010) [2023-12-26 19:13:10,920][105692] Updated weights for policy 0, policy_version 529173 (0.0009) [2023-12-26 19:13:10,932][105620] Updated weights for policy 1, policy_version 529950 (0.0010) [2023-12-26 19:13:10,978][105692] Updated weights for policy 0, policy_version 529183 (0.0009) [2023-12-26 19:13:10,981][105620] Updated weights for policy 1, policy_version 529960 (0.0010) [2023-12-26 19:13:11,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 271171584. Throughput: 0: 9515.5, 1: 9875.5. Samples: 271174968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:13:11,062][104569] Avg episode reward: [(0, '8194.811'), (1, '9081.470')] [2023-12-26 19:13:11,780][105620] Updated weights for policy 1, policy_version 529970 (0.0010) [2023-12-26 19:13:11,783][105692] Updated weights for policy 0, policy_version 529193 (0.0007) [2023-12-26 19:13:11,838][105620] Updated weights for policy 1, policy_version 529980 (0.0008) [2023-12-26 19:13:11,849][105692] Updated weights for policy 0, policy_version 529203 (0.0009) [2023-12-26 19:13:11,893][105620] Updated weights for policy 1, policy_version 529990 (0.0008) [2023-12-26 19:13:11,907][105692] Updated weights for policy 0, policy_version 529213 (0.0006) [2023-12-26 19:13:12,606][105692] Updated weights for policy 0, policy_version 529223 (0.0007) [2023-12-26 19:13:12,668][105692] Updated weights for policy 0, policy_version 529233 (0.0005) [2023-12-26 19:13:12,699][105620] Updated weights for policy 1, policy_version 530000 (0.0008) [2023-12-26 19:13:12,719][105692] Updated weights for policy 0, policy_version 529243 (0.0006) [2023-12-26 19:13:12,758][105620] Updated weights for policy 1, policy_version 530010 (0.0007) [2023-12-26 19:13:12,811][105620] Updated weights for policy 1, policy_version 530020 (0.0010) [2023-12-26 19:13:13,396][105692] Updated weights for policy 0, policy_version 529253 (0.0008) [2023-12-26 19:13:13,457][105692] Updated weights for policy 0, policy_version 529263 (0.0010) [2023-12-26 19:13:13,505][105620] Updated weights for policy 1, policy_version 530030 (0.0009) [2023-12-26 19:13:13,510][105692] Updated weights for policy 0, policy_version 529273 (0.0008) [2023-12-26 19:13:13,564][105620] Updated weights for policy 1, policy_version 530040 (0.0008) [2023-12-26 19:13:13,631][105620] Updated weights for policy 1, policy_version 530050 (0.0008) [2023-12-26 19:13:14,256][105692] Updated weights for policy 0, policy_version 529283 (0.0010) [2023-12-26 19:13:14,304][105692] Updated weights for policy 0, policy_version 529293 (0.0009) [2023-12-26 19:13:14,352][105692] Updated weights for policy 0, policy_version 529303 (0.0010) [2023-12-26 19:13:14,383][105620] Updated weights for policy 1, policy_version 530060 (0.0009) [2023-12-26 19:13:14,440][105620] Updated weights for policy 1, policy_version 530070 (0.0010) [2023-12-26 19:13:14,501][105620] Updated weights for policy 1, policy_version 530080 (0.0010) [2023-12-26 19:13:15,060][105692] Updated weights for policy 0, policy_version 529313 (0.0010) [2023-12-26 19:13:15,124][105692] Updated weights for policy 0, policy_version 529323 (0.0011) [2023-12-26 19:13:15,157][105620] Updated weights for policy 1, policy_version 530090 (0.0011) [2023-12-26 19:13:15,186][105692] Updated weights for policy 0, policy_version 529333 (0.0011) [2023-12-26 19:13:15,218][105620] Updated weights for policy 1, policy_version 530100 (0.0011) [2023-12-26 19:13:15,248][105692] Updated weights for policy 0, policy_version 529343 (0.0007) [2023-12-26 19:13:15,288][105620] Updated weights for policy 1, policy_version 530110 (0.0011) [2023-12-26 19:13:15,344][105620] Updated weights for policy 1, policy_version 530120 (0.0011) [2023-12-26 19:13:15,909][105692] Updated weights for policy 0, policy_version 529353 (0.0010) [2023-12-26 19:13:15,963][105692] Updated weights for policy 0, policy_version 529363 (0.0010) [2023-12-26 19:13:16,012][105692] Updated weights for policy 0, policy_version 529373 (0.0010) [2023-12-26 19:13:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 271261696. Throughput: 0: 9470.6, 1: 9776.4. Samples: 271230828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:13:16,062][104569] Avg episode reward: [(0, '8205.020'), (1, '9172.947')] [2023-12-26 19:13:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000529376_135536640.pth... [2023-12-26 19:13:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000528256_135249920.pth [2023-12-26 19:13:16,085][105620] Updated weights for policy 1, policy_version 530130 (0.0010) [2023-12-26 19:13:16,153][105620] Updated weights for policy 1, policy_version 530140 (0.0007) [2023-12-26 19:13:16,211][105620] Updated weights for policy 1, policy_version 530150 (0.0010) [2023-12-26 19:13:16,221][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000530152_135733248.pth... [2023-12-26 19:13:16,225][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000529000_135438336.pth [2023-12-26 19:13:16,629][105692] Updated weights for policy 0, policy_version 529383 (0.0010) [2023-12-26 19:13:16,681][105692] Updated weights for policy 0, policy_version 529393 (0.0010) [2023-12-26 19:13:16,740][105692] Updated weights for policy 0, policy_version 529403 (0.0010) [2023-12-26 19:13:16,939][105620] Updated weights for policy 1, policy_version 530160 (0.0007) [2023-12-26 19:13:16,999][105620] Updated weights for policy 1, policy_version 530170 (0.0008) [2023-12-26 19:13:17,056][105620] Updated weights for policy 1, policy_version 530180 (0.0007) [2023-12-26 19:13:17,501][105692] Updated weights for policy 0, policy_version 529413 (0.0010) [2023-12-26 19:13:17,555][105692] Updated weights for policy 0, policy_version 529423 (0.0007) [2023-12-26 19:13:17,606][105692] Updated weights for policy 0, policy_version 529433 (0.0005) [2023-12-26 19:13:17,744][105620] Updated weights for policy 1, policy_version 530190 (0.0008) [2023-12-26 19:13:17,809][105620] Updated weights for policy 1, policy_version 530200 (0.0010) [2023-12-26 19:13:17,870][105620] Updated weights for policy 1, policy_version 530210 (0.0010) [2023-12-26 19:13:18,238][105692] Updated weights for policy 0, policy_version 529443 (0.0007) [2023-12-26 19:13:18,305][105692] Updated weights for policy 0, policy_version 529453 (0.0009) [2023-12-26 19:13:18,364][105692] Updated weights for policy 0, policy_version 529463 (0.0008) [2023-12-26 19:13:18,468][105620] Updated weights for policy 1, policy_version 530220 (0.0008) [2023-12-26 19:13:18,524][105620] Updated weights for policy 1, policy_version 530230 (0.0006) [2023-12-26 19:13:18,582][105620] Updated weights for policy 1, policy_version 530240 (0.0006) [2023-12-26 19:13:19,125][105692] Updated weights for policy 0, policy_version 529473 (0.0010) [2023-12-26 19:13:19,180][105620] Updated weights for policy 1, policy_version 530250 (0.0006) [2023-12-26 19:13:19,190][105692] Updated weights for policy 0, policy_version 529483 (0.0009) [2023-12-26 19:13:19,244][105620] Updated weights for policy 1, policy_version 530260 (0.0008) [2023-12-26 19:13:19,260][105692] Updated weights for policy 0, policy_version 529493 (0.0008) [2023-12-26 19:13:19,310][105620] Updated weights for policy 1, policy_version 530270 (0.0007) [2023-12-26 19:13:19,328][105692] Updated weights for policy 0, policy_version 529503 (0.0007) [2023-12-26 19:13:19,379][105620] Updated weights for policy 1, policy_version 530280 (0.0008) [2023-12-26 19:13:20,069][105692] Updated weights for policy 0, policy_version 529513 (0.0007) [2023-12-26 19:13:20,105][105620] Updated weights for policy 1, policy_version 530290 (0.0007) [2023-12-26 19:13:20,135][105692] Updated weights for policy 0, policy_version 529523 (0.0007) [2023-12-26 19:13:20,177][105620] Updated weights for policy 1, policy_version 530300 (0.0006) [2023-12-26 19:13:20,200][105692] Updated weights for policy 0, policy_version 529533 (0.0007) [2023-12-26 19:13:20,251][105620] Updated weights for policy 1, policy_version 530310 (0.0007) [2023-12-26 19:13:20,812][105692] Updated weights for policy 0, policy_version 529543 (0.0008) [2023-12-26 19:13:20,850][105620] Updated weights for policy 1, policy_version 530320 (0.0011) [2023-12-26 19:13:20,870][105692] Updated weights for policy 0, policy_version 529553 (0.0008) [2023-12-26 19:13:20,910][105620] Updated weights for policy 1, policy_version 530330 (0.0008) [2023-12-26 19:13:20,925][105692] Updated weights for policy 0, policy_version 529563 (0.0008) [2023-12-26 19:13:20,973][105620] Updated weights for policy 1, policy_version 530340 (0.0008) [2023-12-26 19:13:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 271368192. Throughput: 0: 9541.5, 1: 9849.9. Samples: 271350728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:13:21,063][104569] Avg episode reward: [(0, '8469.358'), (1, '9354.600')] [2023-12-26 19:13:21,676][105692] Updated weights for policy 0, policy_version 529573 (0.0007) [2023-12-26 19:13:21,703][105620] Updated weights for policy 1, policy_version 530350 (0.0008) [2023-12-26 19:13:21,732][105692] Updated weights for policy 0, policy_version 529583 (0.0008) [2023-12-26 19:13:21,774][105620] Updated weights for policy 1, policy_version 530360 (0.0007) [2023-12-26 19:13:21,798][105692] Updated weights for policy 0, policy_version 529593 (0.0009) [2023-12-26 19:13:21,834][105620] Updated weights for policy 1, policy_version 530370 (0.0006) [2023-12-26 19:13:22,454][105620] Updated weights for policy 1, policy_version 530380 (0.0006) [2023-12-26 19:13:22,517][105620] Updated weights for policy 1, policy_version 530390 (0.0009) [2023-12-26 19:13:22,579][105620] Updated weights for policy 1, policy_version 530400 (0.0009) [2023-12-26 19:13:22,660][105692] Updated weights for policy 0, policy_version 529603 (0.0009) [2023-12-26 19:13:22,722][105692] Updated weights for policy 0, policy_version 529613 (0.0009) [2023-12-26 19:13:22,783][105692] Updated weights for policy 0, policy_version 529623 (0.0010) [2023-12-26 19:13:23,331][105620] Updated weights for policy 1, policy_version 530410 (0.0010) [2023-12-26 19:13:23,377][105620] Updated weights for policy 1, policy_version 530420 (0.0010) [2023-12-26 19:13:23,421][105692] Updated weights for policy 0, policy_version 529633 (0.0009) [2023-12-26 19:13:23,438][105620] Updated weights for policy 1, policy_version 530430 (0.0011) [2023-12-26 19:13:23,480][105692] Updated weights for policy 0, policy_version 529643 (0.0010) [2023-12-26 19:13:23,491][105620] Updated weights for policy 1, policy_version 530440 (0.0009) [2023-12-26 19:13:23,535][105692] Updated weights for policy 0, policy_version 529653 (0.0010) [2023-12-26 19:13:23,593][105692] Updated weights for policy 0, policy_version 529663 (0.0010) [2023-12-26 19:13:24,131][105620] Updated weights for policy 1, policy_version 530450 (0.0010) [2023-12-26 19:13:24,204][105620] Updated weights for policy 1, policy_version 530460 (0.0010) [2023-12-26 19:13:24,267][105620] Updated weights for policy 1, policy_version 530470 (0.0008) [2023-12-26 19:13:24,273][105692] Updated weights for policy 0, policy_version 529673 (0.0008) [2023-12-26 19:13:24,329][105692] Updated weights for policy 0, policy_version 529683 (0.0008) [2023-12-26 19:13:24,389][105692] Updated weights for policy 0, policy_version 529693 (0.0010) [2023-12-26 19:13:24,797][105620] Updated weights for policy 1, policy_version 530480 (0.0005) [2023-12-26 19:13:24,847][105620] Updated weights for policy 1, policy_version 530490 (0.0005) [2023-12-26 19:13:24,908][105620] Updated weights for policy 1, policy_version 530500 (0.0005) [2023-12-26 19:13:25,125][105692] Updated weights for policy 0, policy_version 529703 (0.0008) [2023-12-26 19:13:25,183][105692] Updated weights for policy 0, policy_version 529713 (0.0010) [2023-12-26 19:13:25,238][105692] Updated weights for policy 0, policy_version 529723 (0.0010) [2023-12-26 19:13:25,553][105620] Updated weights for policy 1, policy_version 530510 (0.0008) [2023-12-26 19:13:25,597][105620] Updated weights for policy 1, policy_version 530520 (0.0010) [2023-12-26 19:13:25,658][105620] Updated weights for policy 1, policy_version 530530 (0.0010) [2023-12-26 19:13:25,967][105692] Updated weights for policy 0, policy_version 529733 (0.0010) [2023-12-26 19:13:26,029][105692] Updated weights for policy 0, policy_version 529743 (0.0010) [2023-12-26 19:13:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 271458304. Throughput: 0: 9630.3, 1: 9888.6. Samples: 271470840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:13:26,063][104569] Avg episode reward: [(0, '8368.275'), (1, '9354.650')] [2023-12-26 19:13:26,077][105692] Updated weights for policy 0, policy_version 529753 (0.0010) [2023-12-26 19:13:26,416][105620] Updated weights for policy 1, policy_version 530540 (0.0010) [2023-12-26 19:13:26,477][105620] Updated weights for policy 1, policy_version 530550 (0.0010) [2023-12-26 19:13:26,539][105620] Updated weights for policy 1, policy_version 530560 (0.0010) [2023-12-26 19:13:26,712][105692] Updated weights for policy 0, policy_version 529763 (0.0010) [2023-12-26 19:13:26,777][105692] Updated weights for policy 0, policy_version 529773 (0.0010) [2023-12-26 19:13:26,840][105692] Updated weights for policy 0, policy_version 529783 (0.0007) [2023-12-26 19:13:27,261][105620] Updated weights for policy 1, policy_version 530570 (0.0010) [2023-12-26 19:13:27,325][105620] Updated weights for policy 1, policy_version 530580 (0.0010) [2023-12-26 19:13:27,383][105692] Updated weights for policy 0, policy_version 529793 (0.0006) [2023-12-26 19:13:27,383][105620] Updated weights for policy 1, policy_version 530590 (0.0010) [2023-12-26 19:13:27,441][105692] Updated weights for policy 0, policy_version 529803 (0.0010) [2023-12-26 19:13:27,444][105620] Updated weights for policy 1, policy_version 530600 (0.0010) [2023-12-26 19:13:27,495][105692] Updated weights for policy 0, policy_version 529813 (0.0010) [2023-12-26 19:13:27,549][105692] Updated weights for policy 0, policy_version 529823 (0.0010) [2023-12-26 19:13:28,171][105620] Updated weights for policy 1, policy_version 530610 (0.0010) [2023-12-26 19:13:28,225][105620] Updated weights for policy 1, policy_version 530620 (0.0010) [2023-12-26 19:13:28,257][105692] Updated weights for policy 0, policy_version 529833 (0.0006) [2023-12-26 19:13:28,278][105620] Updated weights for policy 1, policy_version 530630 (0.0010) [2023-12-26 19:13:28,312][105692] Updated weights for policy 0, policy_version 529843 (0.0006) [2023-12-26 19:13:28,371][105692] Updated weights for policy 0, policy_version 529853 (0.0008) [2023-12-26 19:13:29,048][105620] Updated weights for policy 1, policy_version 530640 (0.0010) [2023-12-26 19:13:29,088][105692] Updated weights for policy 0, policy_version 529863 (0.0006) [2023-12-26 19:13:29,099][105620] Updated weights for policy 1, policy_version 530650 (0.0010) [2023-12-26 19:13:29,145][105692] Updated weights for policy 0, policy_version 529873 (0.0006) [2023-12-26 19:13:29,151][105620] Updated weights for policy 1, policy_version 530660 (0.0010) [2023-12-26 19:13:29,195][105692] Updated weights for policy 0, policy_version 529883 (0.0008) [2023-12-26 19:13:29,870][105692] Updated weights for policy 0, policy_version 529893 (0.0009) [2023-12-26 19:13:29,934][105692] Updated weights for policy 0, policy_version 529903 (0.0008) [2023-12-26 19:13:29,938][105620] Updated weights for policy 1, policy_version 530670 (0.0009) [2023-12-26 19:13:29,990][105620] Updated weights for policy 1, policy_version 530680 (0.0009) [2023-12-26 19:13:29,996][105692] Updated weights for policy 0, policy_version 529913 (0.0007) [2023-12-26 19:13:30,040][105620] Updated weights for policy 1, policy_version 530690 (0.0006) [2023-12-26 19:13:30,714][105692] Updated weights for policy 0, policy_version 529923 (0.0009) [2023-12-26 19:13:30,762][105692] Updated weights for policy 0, policy_version 529933 (0.0009) [2023-12-26 19:13:30,811][105620] Updated weights for policy 1, policy_version 530700 (0.0008) [2023-12-26 19:13:30,817][105692] Updated weights for policy 0, policy_version 529943 (0.0008) [2023-12-26 19:13:30,868][105620] Updated weights for policy 1, policy_version 530710 (0.0006) [2023-12-26 19:13:30,920][105620] Updated weights for policy 1, policy_version 530720 (0.0008) [2023-12-26 19:13:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 271564800. Throughput: 0: 9688.9, 1: 9846.5. Samples: 271530572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:13:31,062][104569] Avg episode reward: [(0, '8731.843'), (1, '9172.387')] [2023-12-26 19:13:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000529952_135684096.pth... [2023-12-26 19:13:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000530728_135880704.pth... [2023-12-26 19:13:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000528800_135389184.pth [2023-12-26 19:13:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000529544_135577600.pth [2023-12-26 19:13:31,616][105692] Updated weights for policy 0, policy_version 529953 (0.0007) [2023-12-26 19:13:31,675][105692] Updated weights for policy 0, policy_version 529963 (0.0007) [2023-12-26 19:13:31,705][105620] Updated weights for policy 1, policy_version 530730 (0.0008) [2023-12-26 19:13:31,736][105692] Updated weights for policy 0, policy_version 529973 (0.0009) [2023-12-26 19:13:31,772][105620] Updated weights for policy 1, policy_version 530740 (0.0008) [2023-12-26 19:13:31,787][105692] Updated weights for policy 0, policy_version 529983 (0.0009) [2023-12-26 19:13:31,820][105620] Updated weights for policy 1, policy_version 530750 (0.0008) [2023-12-26 19:13:31,871][105620] Updated weights for policy 1, policy_version 530760 (0.0009) [2023-12-26 19:13:32,574][105620] Updated weights for policy 1, policy_version 530770 (0.0006) [2023-12-26 19:13:32,630][105692] Updated weights for policy 0, policy_version 529993 (0.0006) [2023-12-26 19:13:32,632][105620] Updated weights for policy 1, policy_version 530780 (0.0007) [2023-12-26 19:13:32,679][105692] Updated weights for policy 0, policy_version 530003 (0.0006) [2023-12-26 19:13:32,681][105620] Updated weights for policy 1, policy_version 530790 (0.0008) [2023-12-26 19:13:32,729][105692] Updated weights for policy 0, policy_version 530013 (0.0008) [2023-12-26 19:13:33,352][105620] Updated weights for policy 1, policy_version 530800 (0.0008) [2023-12-26 19:13:33,400][105620] Updated weights for policy 1, policy_version 530810 (0.0008) [2023-12-26 19:13:33,455][105620] Updated weights for policy 1, policy_version 530820 (0.0005) [2023-12-26 19:13:33,515][105692] Updated weights for policy 0, policy_version 530023 (0.0009) [2023-12-26 19:13:33,571][105692] Updated weights for policy 0, policy_version 530033 (0.0008) [2023-12-26 19:13:33,625][105692] Updated weights for policy 0, policy_version 530043 (0.0009) [2023-12-26 19:13:34,048][105620] Updated weights for policy 1, policy_version 530830 (0.0005) [2023-12-26 19:13:34,101][105620] Updated weights for policy 1, policy_version 530840 (0.0008) [2023-12-26 19:13:34,163][105620] Updated weights for policy 1, policy_version 530850 (0.0007) [2023-12-26 19:13:34,478][105692] Updated weights for policy 0, policy_version 530053 (0.0009) [2023-12-26 19:13:34,530][105692] Updated weights for policy 0, policy_version 530063 (0.0009) [2023-12-26 19:13:34,580][105692] Updated weights for policy 0, policy_version 530073 (0.0009) [2023-12-26 19:13:34,862][105620] Updated weights for policy 1, policy_version 530860 (0.0009) [2023-12-26 19:13:34,918][105620] Updated weights for policy 1, policy_version 530870 (0.0009) [2023-12-26 19:13:34,969][105620] Updated weights for policy 1, policy_version 530880 (0.0009) [2023-12-26 19:13:35,359][105692] Updated weights for policy 0, policy_version 530083 (0.0009) [2023-12-26 19:13:35,413][105692] Updated weights for policy 0, policy_version 530093 (0.0008) [2023-12-26 19:13:35,468][105692] Updated weights for policy 0, policy_version 530103 (0.0005) [2023-12-26 19:13:35,727][105620] Updated weights for policy 1, policy_version 530890 (0.0009) [2023-12-26 19:13:35,789][105620] Updated weights for policy 1, policy_version 530900 (0.0009) [2023-12-26 19:13:35,840][105620] Updated weights for policy 1, policy_version 530910 (0.0009) [2023-12-26 19:13:35,898][105620] Updated weights for policy 1, policy_version 530920 (0.0010) [2023-12-26 19:13:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 271654912. Throughput: 0: 9663.4, 1: 9831.9. Samples: 271644940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:13:36,062][104569] Avg episode reward: [(0, '8731.814'), (1, '9172.209')] [2023-12-26 19:13:36,195][105692] Updated weights for policy 0, policy_version 530113 (0.0008) [2023-12-26 19:13:36,264][105692] Updated weights for policy 0, policy_version 530123 (0.0009) [2023-12-26 19:13:36,323][105692] Updated weights for policy 0, policy_version 530133 (0.0010) [2023-12-26 19:13:36,386][105692] Updated weights for policy 0, policy_version 530143 (0.0009) [2023-12-26 19:13:36,550][105620] Updated weights for policy 1, policy_version 530930 (0.0009) [2023-12-26 19:13:36,607][105620] Updated weights for policy 1, policy_version 530940 (0.0009) [2023-12-26 19:13:36,673][105620] Updated weights for policy 1, policy_version 530950 (0.0009) [2023-12-26 19:13:37,105][105692] Updated weights for policy 0, policy_version 530153 (0.0009) [2023-12-26 19:13:37,164][105692] Updated weights for policy 0, policy_version 530163 (0.0009) [2023-12-26 19:13:37,219][105692] Updated weights for policy 0, policy_version 530173 (0.0009) [2023-12-26 19:13:37,450][105620] Updated weights for policy 1, policy_version 530960 (0.0008) [2023-12-26 19:13:37,518][105620] Updated weights for policy 1, policy_version 530970 (0.0008) [2023-12-26 19:13:37,570][105620] Updated weights for policy 1, policy_version 530980 (0.0009) [2023-12-26 19:13:37,997][105692] Updated weights for policy 0, policy_version 530183 (0.0009) [2023-12-26 19:13:38,055][105692] Updated weights for policy 0, policy_version 530193 (0.0009) [2023-12-26 19:13:38,111][105692] Updated weights for policy 0, policy_version 530203 (0.0009) [2023-12-26 19:13:38,275][105620] Updated weights for policy 1, policy_version 530990 (0.0009) [2023-12-26 19:13:38,341][105620] Updated weights for policy 1, policy_version 531000 (0.0010) [2023-12-26 19:13:38,403][105620] Updated weights for policy 1, policy_version 531010 (0.0009) [2023-12-26 19:13:38,891][105692] Updated weights for policy 0, policy_version 530213 (0.0009) [2023-12-26 19:13:38,958][105692] Updated weights for policy 0, policy_version 530223 (0.0009) [2023-12-26 19:13:39,031][105692] Updated weights for policy 0, policy_version 530233 (0.0009) [2023-12-26 19:13:39,110][105620] Updated weights for policy 1, policy_version 531020 (0.0008) [2023-12-26 19:13:39,171][105620] Updated weights for policy 1, policy_version 531030 (0.0009) [2023-12-26 19:13:39,240][105620] Updated weights for policy 1, policy_version 531040 (0.0009) [2023-12-26 19:13:39,777][105692] Updated weights for policy 0, policy_version 530243 (0.0009) [2023-12-26 19:13:39,841][105692] Updated weights for policy 0, policy_version 530253 (0.0008) [2023-12-26 19:13:39,909][105692] Updated weights for policy 0, policy_version 530263 (0.0008) [2023-12-26 19:13:40,044][105620] Updated weights for policy 1, policy_version 531050 (0.0010) [2023-12-26 19:13:40,102][105620] Updated weights for policy 1, policy_version 531060 (0.0009) [2023-12-26 19:13:40,161][105620] Updated weights for policy 1, policy_version 531070 (0.0009) [2023-12-26 19:13:40,224][105620] Updated weights for policy 1, policy_version 531080 (0.0009) [2023-12-26 19:13:40,697][105692] Updated weights for policy 0, policy_version 530273 (0.0009) [2023-12-26 19:13:40,752][105692] Updated weights for policy 0, policy_version 530283 (0.0009) [2023-12-26 19:13:40,817][105692] Updated weights for policy 0, policy_version 530293 (0.0009) [2023-12-26 19:13:40,881][105692] Updated weights for policy 0, policy_version 530303 (0.0009) [2023-12-26 19:13:40,969][105620] Updated weights for policy 1, policy_version 531090 (0.0007) [2023-12-26 19:13:41,021][105620] Updated weights for policy 1, policy_version 531100 (0.0006) [2023-12-26 19:13:41,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 271745024. Throughput: 0: 9604.3, 1: 9832.6. Samples: 271757092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:13:41,063][104569] Avg episode reward: [(0, '8550.950'), (1, '9354.399')] [2023-12-26 19:13:41,099][105620] Updated weights for policy 1, policy_version 531110 (0.0009) [2023-12-26 19:13:41,652][105692] Updated weights for policy 0, policy_version 530313 (0.0008) [2023-12-26 19:13:41,717][105692] Updated weights for policy 0, policy_version 530323 (0.0008) [2023-12-26 19:13:41,781][105692] Updated weights for policy 0, policy_version 530333 (0.0009) [2023-12-26 19:13:41,841][105620] Updated weights for policy 1, policy_version 531120 (0.0009) [2023-12-26 19:13:41,903][105620] Updated weights for policy 1, policy_version 531130 (0.0009) [2023-12-26 19:13:41,967][105620] Updated weights for policy 1, policy_version 531140 (0.0009) [2023-12-26 19:13:42,543][105692] Updated weights for policy 0, policy_version 530343 (0.0009) [2023-12-26 19:13:42,604][105692] Updated weights for policy 0, policy_version 530353 (0.0011) [2023-12-26 19:13:42,657][105692] Updated weights for policy 0, policy_version 530363 (0.0010) [2023-12-26 19:13:42,719][105620] Updated weights for policy 1, policy_version 531150 (0.0010) [2023-12-26 19:13:42,772][105620] Updated weights for policy 1, policy_version 531160 (0.0010) [2023-12-26 19:13:42,826][105620] Updated weights for policy 1, policy_version 531171 (0.0010) [2023-12-26 19:13:43,354][105692] Updated weights for policy 0, policy_version 530373 (0.0007) [2023-12-26 19:13:43,407][105692] Updated weights for policy 0, policy_version 530383 (0.0009) [2023-12-26 19:13:43,457][105692] Updated weights for policy 0, policy_version 530393 (0.0009) [2023-12-26 19:13:43,566][105620] Updated weights for policy 1, policy_version 531182 (0.0008) [2023-12-26 19:13:43,618][105620] Updated weights for policy 1, policy_version 531192 (0.0009) [2023-12-26 19:13:43,670][105620] Updated weights for policy 1, policy_version 531202 (0.0009) [2023-12-26 19:13:44,097][105692] Updated weights for policy 0, policy_version 530403 (0.0008) [2023-12-26 19:13:44,156][105692] Updated weights for policy 0, policy_version 530413 (0.0008) [2023-12-26 19:13:44,217][105692] Updated weights for policy 0, policy_version 530423 (0.0009) [2023-12-26 19:13:44,490][105620] Updated weights for policy 1, policy_version 531212 (0.0010) [2023-12-26 19:13:44,541][105620] Updated weights for policy 1, policy_version 531222 (0.0009) [2023-12-26 19:13:44,596][105620] Updated weights for policy 1, policy_version 531232 (0.0009) [2023-12-26 19:13:44,965][105692] Updated weights for policy 0, policy_version 530433 (0.0008) [2023-12-26 19:13:45,016][105692] Updated weights for policy 0, policy_version 530443 (0.0007) [2023-12-26 19:13:45,066][105692] Updated weights for policy 0, policy_version 530453 (0.0008) [2023-12-26 19:13:45,113][105692] Updated weights for policy 0, policy_version 530463 (0.0008) [2023-12-26 19:13:45,397][105620] Updated weights for policy 1, policy_version 531242 (0.0009) [2023-12-26 19:13:45,468][105620] Updated weights for policy 1, policy_version 531252 (0.0009) [2023-12-26 19:13:45,524][105620] Updated weights for policy 1, policy_version 531262 (0.0009) [2023-12-26 19:13:45,572][105620] Updated weights for policy 1, policy_version 531272 (0.0009) [2023-12-26 19:13:45,866][105692] Updated weights for policy 0, policy_version 530473 (0.0009) [2023-12-26 19:13:45,925][105692] Updated weights for policy 0, policy_version 530483 (0.0008) [2023-12-26 19:13:45,988][105692] Updated weights for policy 0, policy_version 530493 (0.0009) [2023-12-26 19:13:46,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19387.6, 300 sec: 19355.3). Total num frames: 271843328. Throughput: 0: 9526.2, 1: 9831.0. Samples: 271811772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:13:46,063][104569] Avg episode reward: [(0, '8366.022'), (1, '9173.125')] [2023-12-26 19:13:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000530496_135823360.pth... [2023-12-26 19:13:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000531272_136019968.pth... [2023-12-26 19:13:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000529376_135536640.pth [2023-12-26 19:13:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000530152_135733248.pth [2023-12-26 19:13:46,339][105620] Updated weights for policy 1, policy_version 531282 (0.0009) [2023-12-26 19:13:46,398][105620] Updated weights for policy 1, policy_version 531292 (0.0009) [2023-12-26 19:13:46,460][105620] Updated weights for policy 1, policy_version 531302 (0.0009) [2023-12-26 19:13:46,725][105692] Updated weights for policy 0, policy_version 530503 (0.0007) [2023-12-26 19:13:46,780][105692] Updated weights for policy 0, policy_version 530513 (0.0005) [2023-12-26 19:13:46,842][105692] Updated weights for policy 0, policy_version 530523 (0.0005) [2023-12-26 19:13:47,205][105620] Updated weights for policy 1, policy_version 531312 (0.0006) [2023-12-26 19:13:47,263][105620] Updated weights for policy 1, policy_version 531322 (0.0005) [2023-12-26 19:13:47,310][105620] Updated weights for policy 1, policy_version 531332 (0.0008) [2023-12-26 19:13:47,442][105692] Updated weights for policy 0, policy_version 530533 (0.0007) [2023-12-26 19:13:47,489][105692] Updated weights for policy 0, policy_version 530543 (0.0008) [2023-12-26 19:13:47,533][105692] Updated weights for policy 0, policy_version 530553 (0.0008) [2023-12-26 19:13:47,975][105620] Updated weights for policy 1, policy_version 531342 (0.0010) [2023-12-26 19:13:48,034][105620] Updated weights for policy 1, policy_version 531352 (0.0010) [2023-12-26 19:13:48,097][105620] Updated weights for policy 1, policy_version 531362 (0.0011) [2023-12-26 19:13:48,318][105692] Updated weights for policy 0, policy_version 530563 (0.0009) [2023-12-26 19:13:48,383][105692] Updated weights for policy 0, policy_version 530573 (0.0008) [2023-12-26 19:13:48,442][105692] Updated weights for policy 0, policy_version 530583 (0.0008) [2023-12-26 19:13:48,752][105620] Updated weights for policy 1, policy_version 531372 (0.0010) [2023-12-26 19:13:48,810][105620] Updated weights for policy 1, policy_version 531382 (0.0008) [2023-12-26 19:13:48,875][105620] Updated weights for policy 1, policy_version 531392 (0.0008) [2023-12-26 19:13:49,277][105692] Updated weights for policy 0, policy_version 530593 (0.0008) [2023-12-26 19:13:49,344][105692] Updated weights for policy 0, policy_version 530603 (0.0008) [2023-12-26 19:13:49,408][105692] Updated weights for policy 0, policy_version 530613 (0.0008) [2023-12-26 19:13:49,468][105692] Updated weights for policy 0, policy_version 530623 (0.0009) [2023-12-26 19:13:49,509][105620] Updated weights for policy 1, policy_version 531402 (0.0008) [2023-12-26 19:13:49,573][105620] Updated weights for policy 1, policy_version 531412 (0.0005) [2023-12-26 19:13:49,632][105620] Updated weights for policy 1, policy_version 531422 (0.0005) [2023-12-26 19:13:49,688][105620] Updated weights for policy 1, policy_version 531432 (0.0005) [2023-12-26 19:13:50,247][105692] Updated weights for policy 0, policy_version 530633 (0.0010) [2023-12-26 19:13:50,308][105692] Updated weights for policy 0, policy_version 530643 (0.0011) [2023-12-26 19:13:50,362][105620] Updated weights for policy 1, policy_version 531442 (0.0008) [2023-12-26 19:13:50,374][105692] Updated weights for policy 0, policy_version 530653 (0.0009) [2023-12-26 19:13:50,421][105620] Updated weights for policy 1, policy_version 531452 (0.0008) [2023-12-26 19:13:50,485][105620] Updated weights for policy 1, policy_version 531462 (0.0009) [2023-12-26 19:13:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19355.3). Total num frames: 271933440. Throughput: 0: 9532.9, 1: 9777.3. Samples: 271929408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:13:51,062][104569] Avg episode reward: [(0, '8636.559'), (1, '9173.662')] [2023-12-26 19:13:51,137][105692] Updated weights for policy 0, policy_version 530663 (0.0007) [2023-12-26 19:13:51,204][105620] Updated weights for policy 1, policy_version 531472 (0.0010) [2023-12-26 19:13:51,208][105692] Updated weights for policy 0, policy_version 530673 (0.0006) [2023-12-26 19:13:51,263][105620] Updated weights for policy 1, policy_version 531482 (0.0009) [2023-12-26 19:13:51,285][105692] Updated weights for policy 0, policy_version 530683 (0.0009) [2023-12-26 19:13:51,325][105620] Updated weights for policy 1, policy_version 531492 (0.0007) [2023-12-26 19:13:51,993][105692] Updated weights for policy 0, policy_version 530693 (0.0008) [2023-12-26 19:13:52,062][105692] Updated weights for policy 0, policy_version 530703 (0.0005) [2023-12-26 19:13:52,104][105620] Updated weights for policy 1, policy_version 531502 (0.0008) [2023-12-26 19:13:52,126][105692] Updated weights for policy 0, policy_version 530713 (0.0006) [2023-12-26 19:13:52,168][105620] Updated weights for policy 1, policy_version 531512 (0.0008) [2023-12-26 19:13:52,224][105620] Updated weights for policy 1, policy_version 531522 (0.0009) [2023-12-26 19:13:52,847][105692] Updated weights for policy 0, policy_version 530723 (0.0007) [2023-12-26 19:13:52,905][105692] Updated weights for policy 0, policy_version 530733 (0.0009) [2023-12-26 19:13:52,948][105620] Updated weights for policy 1, policy_version 531532 (0.0007) [2023-12-26 19:13:52,967][105692] Updated weights for policy 0, policy_version 530743 (0.0008) [2023-12-26 19:13:53,002][105620] Updated weights for policy 1, policy_version 531542 (0.0006) [2023-12-26 19:13:53,066][105620] Updated weights for policy 1, policy_version 531552 (0.0008) [2023-12-26 19:13:53,715][105692] Updated weights for policy 0, policy_version 530753 (0.0008) [2023-12-26 19:13:53,761][105692] Updated weights for policy 0, policy_version 530763 (0.0009) [2023-12-26 19:13:53,814][105692] Updated weights for policy 0, policy_version 530773 (0.0008) [2023-12-26 19:13:53,819][105620] Updated weights for policy 1, policy_version 531562 (0.0010) [2023-12-26 19:13:53,876][105692] Updated weights for policy 0, policy_version 530783 (0.0007) [2023-12-26 19:13:53,878][105620] Updated weights for policy 1, policy_version 531572 (0.0007) [2023-12-26 19:13:53,932][105620] Updated weights for policy 1, policy_version 531582 (0.0009) [2023-12-26 19:13:53,982][105620] Updated weights for policy 1, policy_version 531592 (0.0008) [2023-12-26 19:13:54,645][105692] Updated weights for policy 0, policy_version 530793 (0.0009) [2023-12-26 19:13:54,708][105692] Updated weights for policy 0, policy_version 530803 (0.0009) [2023-12-26 19:13:54,743][105620] Updated weights for policy 1, policy_version 531602 (0.0006) [2023-12-26 19:13:54,765][105692] Updated weights for policy 0, policy_version 530813 (0.0008) [2023-12-26 19:13:54,800][105620] Updated weights for policy 1, policy_version 531612 (0.0007) [2023-12-26 19:13:54,865][105620] Updated weights for policy 1, policy_version 531622 (0.0008) [2023-12-26 19:13:55,465][105692] Updated weights for policy 0, policy_version 530823 (0.0008) [2023-12-26 19:13:55,521][105692] Updated weights for policy 0, policy_version 530833 (0.0008) [2023-12-26 19:13:55,580][105692] Updated weights for policy 0, policy_version 530843 (0.0009) [2023-12-26 19:13:55,664][105620] Updated weights for policy 1, policy_version 531632 (0.0009) [2023-12-26 19:13:55,721][105620] Updated weights for policy 1, policy_version 531642 (0.0008) [2023-12-26 19:13:55,778][105620] Updated weights for policy 1, policy_version 531652 (0.0009) [2023-12-26 19:13:56,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 272031744. Throughput: 0: 9532.7, 1: 9693.9. Samples: 272040168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:13:56,063][104569] Avg episode reward: [(0, '8557.591'), (1, '9355.512')] [2023-12-26 19:13:56,392][105692] Updated weights for policy 0, policy_version 530853 (0.0007) [2023-12-26 19:13:56,398][105620] Updated weights for policy 1, policy_version 531662 (0.0008) [2023-12-26 19:13:56,451][105620] Updated weights for policy 1, policy_version 531672 (0.0010) [2023-12-26 19:13:56,452][105692] Updated weights for policy 0, policy_version 530863 (0.0008) [2023-12-26 19:13:56,495][105620] Updated weights for policy 1, policy_version 531682 (0.0010) [2023-12-26 19:13:56,514][105692] Updated weights for policy 0, policy_version 530873 (0.0007) [2023-12-26 19:13:57,133][105692] Updated weights for policy 0, policy_version 530883 (0.0006) [2023-12-26 19:13:57,176][105585] KL-divergence is very high: 176.8769 [2023-12-26 19:13:57,191][105692] Updated weights for policy 0, policy_version 530893 (0.0008) [2023-12-26 19:13:57,208][105620] Updated weights for policy 1, policy_version 531692 (0.0010) [2023-12-26 19:13:57,216][105585] KL-divergence is very high: 287.3131 [2023-12-26 19:13:57,242][105692] Updated weights for policy 0, policy_version 530903 (0.0006) [2023-12-26 19:13:57,255][105585] KL-divergence is very high: 272.6606 [2023-12-26 19:13:57,259][105620] Updated weights for policy 1, policy_version 531702 (0.0010) [2023-12-26 19:13:57,310][105620] Updated weights for policy 1, policy_version 531712 (0.0008) [2023-12-26 19:13:57,859][105692] Updated weights for policy 0, policy_version 530913 (0.0006) [2023-12-26 19:13:57,917][105692] Updated weights for policy 0, policy_version 530923 (0.0008) [2023-12-26 19:13:57,952][105620] Updated weights for policy 1, policy_version 531722 (0.0008) [2023-12-26 19:13:57,966][105692] Updated weights for policy 0, policy_version 530933 (0.0006) [2023-12-26 19:13:58,001][105620] Updated weights for policy 1, policy_version 531732 (0.0009) [2023-12-26 19:13:58,022][105692] Updated weights for policy 0, policy_version 530943 (0.0005) [2023-12-26 19:13:58,058][105620] Updated weights for policy 1, policy_version 531742 (0.0009) [2023-12-26 19:13:58,108][105620] Updated weights for policy 1, policy_version 531752 (0.0010) [2023-12-26 19:13:58,728][105692] Updated weights for policy 0, policy_version 530953 (0.0007) [2023-12-26 19:13:58,806][105692] Updated weights for policy 0, policy_version 530963 (0.0007) [2023-12-26 19:13:58,870][105692] Updated weights for policy 0, policy_version 530973 (0.0008) [2023-12-26 19:13:58,895][105620] Updated weights for policy 1, policy_version 531762 (0.0009) [2023-12-26 19:13:58,954][105620] Updated weights for policy 1, policy_version 531772 (0.0010) [2023-12-26 19:13:59,023][105620] Updated weights for policy 1, policy_version 531782 (0.0011) [2023-12-26 19:13:59,647][105692] Updated weights for policy 0, policy_version 530983 (0.0009) [2023-12-26 19:13:59,705][105692] Updated weights for policy 0, policy_version 530993 (0.0007) [2023-12-26 19:13:59,738][105620] Updated weights for policy 1, policy_version 531792 (0.0008) [2023-12-26 19:13:59,759][105692] Updated weights for policy 0, policy_version 531003 (0.0005) [2023-12-26 19:13:59,799][105620] Updated weights for policy 1, policy_version 531802 (0.0009) [2023-12-26 19:13:59,859][105620] Updated weights for policy 1, policy_version 531813 (0.0009) [2023-12-26 19:14:00,411][105692] Updated weights for policy 0, policy_version 531013 (0.0007) [2023-12-26 19:14:00,473][105692] Updated weights for policy 0, policy_version 531023 (0.0009) [2023-12-26 19:14:00,535][105692] Updated weights for policy 0, policy_version 531033 (0.0009) [2023-12-26 19:14:00,615][105620] Updated weights for policy 1, policy_version 531823 (0.0009) [2023-12-26 19:14:00,660][105620] Updated weights for policy 1, policy_version 531833 (0.0008) [2023-12-26 19:14:00,710][105620] Updated weights for policy 1, policy_version 531843 (0.0009) [2023-12-26 19:14:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 272130048. Throughput: 0: 9581.6, 1: 9765.7. Samples: 272101456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:14:01,063][104569] Avg episode reward: [(0, '9004.506'), (1, '9355.968')] [2023-12-26 19:14:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000531040_135962624.pth... [2023-12-26 19:14:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000531848_136167424.pth... [2023-12-26 19:14:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000530728_135880704.pth [2023-12-26 19:14:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000529952_135684096.pth [2023-12-26 19:14:01,263][105692] Updated weights for policy 0, policy_version 531043 (0.0009) [2023-12-26 19:14:01,322][105692] Updated weights for policy 0, policy_version 531053 (0.0009) [2023-12-26 19:14:01,390][105692] Updated weights for policy 0, policy_version 531063 (0.0008) [2023-12-26 19:14:01,529][105620] Updated weights for policy 1, policy_version 531853 (0.0008) [2023-12-26 19:14:01,585][105620] Updated weights for policy 1, policy_version 531863 (0.0007) [2023-12-26 19:14:01,653][105620] Updated weights for policy 1, policy_version 531873 (0.0009) [2023-12-26 19:14:02,116][105692] Updated weights for policy 0, policy_version 531073 (0.0008) [2023-12-26 19:14:02,177][105692] Updated weights for policy 0, policy_version 531083 (0.0009) [2023-12-26 19:14:02,230][105692] Updated weights for policy 0, policy_version 531093 (0.0009) [2023-12-26 19:14:02,290][105692] Updated weights for policy 0, policy_version 531103 (0.0009) [2023-12-26 19:14:02,305][105620] Updated weights for policy 1, policy_version 531883 (0.0007) [2023-12-26 19:14:02,369][105620] Updated weights for policy 1, policy_version 531893 (0.0007) [2023-12-26 19:14:02,437][105620] Updated weights for policy 1, policy_version 531903 (0.0007) [2023-12-26 19:14:03,014][105620] Updated weights for policy 1, policy_version 531913 (0.0008) [2023-12-26 19:14:03,065][105620] Updated weights for policy 1, policy_version 531923 (0.0005) [2023-12-26 19:14:03,121][105620] Updated weights for policy 1, policy_version 531933 (0.0006) [2023-12-26 19:14:03,149][105692] Updated weights for policy 0, policy_version 531113 (0.0009) [2023-12-26 19:14:03,169][105620] Updated weights for policy 1, policy_version 531943 (0.0010) [2023-12-26 19:14:03,202][105692] Updated weights for policy 0, policy_version 531123 (0.0007) [2023-12-26 19:14:03,256][105692] Updated weights for policy 0, policy_version 531133 (0.0009) [2023-12-26 19:14:03,870][105620] Updated weights for policy 1, policy_version 531953 (0.0011) [2023-12-26 19:14:03,924][105620] Updated weights for policy 1, policy_version 531963 (0.0011) [2023-12-26 19:14:03,979][105620] Updated weights for policy 1, policy_version 531973 (0.0009) [2023-12-26 19:14:04,023][105692] Updated weights for policy 0, policy_version 531144 (0.0010) [2023-12-26 19:14:04,086][105692] Updated weights for policy 0, policy_version 531154 (0.0009) [2023-12-26 19:14:04,144][105692] Updated weights for policy 0, policy_version 531164 (0.0010) [2023-12-26 19:14:04,664][105620] Updated weights for policy 1, policy_version 531983 (0.0008) [2023-12-26 19:14:04,722][105620] Updated weights for policy 1, policy_version 531993 (0.0010) [2023-12-26 19:14:04,777][105620] Updated weights for policy 1, policy_version 532003 (0.0009) [2023-12-26 19:14:04,922][105692] Updated weights for policy 0, policy_version 531174 (0.0009) [2023-12-26 19:14:04,976][105692] Updated weights for policy 0, policy_version 531184 (0.0009) [2023-12-26 19:14:05,037][105692] Updated weights for policy 0, policy_version 531194 (0.0008) [2023-12-26 19:14:05,446][105620] Updated weights for policy 1, policy_version 532013 (0.0007) [2023-12-26 19:14:05,501][105620] Updated weights for policy 1, policy_version 532023 (0.0005) [2023-12-26 19:14:05,564][105620] Updated weights for policy 1, policy_version 532033 (0.0009) [2023-12-26 19:14:05,810][105692] Updated weights for policy 0, policy_version 531204 (0.0009) [2023-12-26 19:14:05,864][105692] Updated weights for policy 0, policy_version 531214 (0.0010) [2023-12-26 19:14:05,919][105692] Updated weights for policy 0, policy_version 531224 (0.0010) [2023-12-26 19:14:06,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 272228352. Throughput: 0: 9491.8, 1: 9716.9. Samples: 272215116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:14:06,063][104569] Avg episode reward: [(0, '9173.379'), (1, '9264.172')] [2023-12-26 19:14:06,324][105620] Updated weights for policy 1, policy_version 532043 (0.0009) [2023-12-26 19:14:06,381][105620] Updated weights for policy 1, policy_version 532053 (0.0008) [2023-12-26 19:14:06,437][105620] Updated weights for policy 1, policy_version 532063 (0.0007) [2023-12-26 19:14:06,619][105692] Updated weights for policy 0, policy_version 531234 (0.0009) [2023-12-26 19:14:06,674][105692] Updated weights for policy 0, policy_version 531244 (0.0006) [2023-12-26 19:14:06,732][105692] Updated weights for policy 0, policy_version 531254 (0.0005) [2023-12-26 19:14:06,780][105692] Updated weights for policy 0, policy_version 531264 (0.0005) [2023-12-26 19:14:07,327][105620] Updated weights for policy 1, policy_version 532073 (0.0009) [2023-12-26 19:14:07,350][105692] Updated weights for policy 0, policy_version 531274 (0.0007) [2023-12-26 19:14:07,392][105620] Updated weights for policy 1, policy_version 532083 (0.0008) [2023-12-26 19:14:07,409][105692] Updated weights for policy 0, policy_version 531284 (0.0007) [2023-12-26 19:14:07,449][105620] Updated weights for policy 1, policy_version 532093 (0.0008) [2023-12-26 19:14:07,466][105692] Updated weights for policy 0, policy_version 531294 (0.0008) [2023-12-26 19:14:07,506][105620] Updated weights for policy 1, policy_version 532103 (0.0009) [2023-12-26 19:14:08,173][105692] Updated weights for policy 0, policy_version 531304 (0.0011) [2023-12-26 19:14:08,225][105692] Updated weights for policy 0, policy_version 531314 (0.0010) [2023-12-26 19:14:08,244][105620] Updated weights for policy 1, policy_version 532113 (0.0006) [2023-12-26 19:14:08,271][105692] Updated weights for policy 0, policy_version 531324 (0.0006) [2023-12-26 19:14:08,296][105620] Updated weights for policy 1, policy_version 532123 (0.0009) [2023-12-26 19:14:08,366][105620] Updated weights for policy 1, policy_version 532133 (0.0009) [2023-12-26 19:14:08,870][105692] Updated weights for policy 0, policy_version 531334 (0.0007) [2023-12-26 19:14:08,932][105692] Updated weights for policy 0, policy_version 531344 (0.0006) [2023-12-26 19:14:08,999][105692] Updated weights for policy 0, policy_version 531354 (0.0007) [2023-12-26 19:14:09,026][105620] Updated weights for policy 1, policy_version 532143 (0.0006) [2023-12-26 19:14:09,092][105620] Updated weights for policy 1, policy_version 532153 (0.0006) [2023-12-26 19:14:09,147][105620] Updated weights for policy 1, policy_version 532163 (0.0010) [2023-12-26 19:14:09,686][105692] Updated weights for policy 0, policy_version 531364 (0.0010) [2023-12-26 19:14:09,749][105692] Updated weights for policy 0, policy_version 531374 (0.0010) [2023-12-26 19:14:09,794][105692] Updated weights for policy 0, policy_version 531384 (0.0008) [2023-12-26 19:14:09,828][105620] Updated weights for policy 1, policy_version 532173 (0.0006) [2023-12-26 19:14:09,892][105620] Updated weights for policy 1, policy_version 532183 (0.0008) [2023-12-26 19:14:09,957][105620] Updated weights for policy 1, policy_version 532193 (0.0009) [2023-12-26 19:14:10,512][105692] Updated weights for policy 0, policy_version 531394 (0.0007) [2023-12-26 19:14:10,559][105692] Updated weights for policy 0, policy_version 531404 (0.0006) [2023-12-26 19:14:10,612][105692] Updated weights for policy 0, policy_version 531414 (0.0008) [2023-12-26 19:14:10,665][105692] Updated weights for policy 0, policy_version 531424 (0.0009) [2023-12-26 19:14:10,810][105620] Updated weights for policy 1, policy_version 532203 (0.0008) [2023-12-26 19:14:10,856][105620] Updated weights for policy 1, policy_version 532213 (0.0005) [2023-12-26 19:14:10,907][105620] Updated weights for policy 1, policy_version 532223 (0.0007) [2023-12-26 19:14:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 272326656. Throughput: 0: 9558.6, 1: 9607.3. Samples: 272333304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:14:11,062][104569] Avg episode reward: [(0, '8992.128'), (1, '1309.111')] [2023-12-26 19:14:11,428][105692] Updated weights for policy 0, policy_version 531434 (0.0008) [2023-12-26 19:14:11,488][105692] Updated weights for policy 0, policy_version 531444 (0.0007) [2023-12-26 19:14:11,555][105692] Updated weights for policy 0, policy_version 531454 (0.0006) [2023-12-26 19:14:11,723][105620] Updated weights for policy 1, policy_version 532234 (0.0009) [2023-12-26 19:14:11,792][105620] Updated weights for policy 1, policy_version 532244 (0.0009) [2023-12-26 19:14:11,853][105620] Updated weights for policy 1, policy_version 532254 (0.0008) [2023-12-26 19:14:11,922][105620] Updated weights for policy 1, policy_version 532264 (0.0008) [2023-12-26 19:14:12,256][105692] Updated weights for policy 0, policy_version 531464 (0.0008) [2023-12-26 19:14:12,319][105692] Updated weights for policy 0, policy_version 531474 (0.0008) [2023-12-26 19:14:12,385][105692] Updated weights for policy 0, policy_version 531484 (0.0007) [2023-12-26 19:14:12,662][105620] Updated weights for policy 1, policy_version 532274 (0.0009) [2023-12-26 19:14:12,718][105620] Updated weights for policy 1, policy_version 532284 (0.0009) [2023-12-26 19:14:12,781][105620] Updated weights for policy 1, policy_version 532294 (0.0011) [2023-12-26 19:14:12,982][105692] Updated weights for policy 0, policy_version 531494 (0.0008) [2023-12-26 19:14:13,040][105692] Updated weights for policy 0, policy_version 531504 (0.0010) [2023-12-26 19:14:13,095][105692] Updated weights for policy 0, policy_version 531514 (0.0011) [2023-12-26 19:14:13,513][105620] Updated weights for policy 1, policy_version 532304 (0.0010) [2023-12-26 19:14:13,571][105620] Updated weights for policy 1, policy_version 532315 (0.0010) [2023-12-26 19:14:13,623][105620] Updated weights for policy 1, policy_version 532325 (0.0009) [2023-12-26 19:14:13,690][105692] Updated weights for policy 0, policy_version 531524 (0.0010) [2023-12-26 19:14:13,748][105692] Updated weights for policy 0, policy_version 531534 (0.0010) [2023-12-26 19:14:13,805][105692] Updated weights for policy 0, policy_version 531544 (0.0010) [2023-12-26 19:14:14,366][105620] Updated weights for policy 1, policy_version 532336 (0.0009) [2023-12-26 19:14:14,427][105620] Updated weights for policy 1, policy_version 532346 (0.0008) [2023-12-26 19:14:14,486][105620] Updated weights for policy 1, policy_version 532356 (0.0005) [2023-12-26 19:14:14,530][105692] Updated weights for policy 0, policy_version 531554 (0.0009) [2023-12-26 19:14:14,582][105692] Updated weights for policy 0, policy_version 531564 (0.0005) [2023-12-26 19:14:14,644][105692] Updated weights for policy 0, policy_version 531574 (0.0005) [2023-12-26 19:14:14,688][105692] Updated weights for policy 0, policy_version 531584 (0.0005) [2023-12-26 19:14:15,204][105620] Updated weights for policy 1, policy_version 532366 (0.0006) [2023-12-26 19:14:15,263][105620] Updated weights for policy 1, policy_version 532376 (0.0008) [2023-12-26 19:14:15,326][105620] Updated weights for policy 1, policy_version 532386 (0.0009) [2023-12-26 19:14:15,388][105692] Updated weights for policy 0, policy_version 531594 (0.0008) [2023-12-26 19:14:15,436][105692] Updated weights for policy 0, policy_version 531604 (0.0008) [2023-12-26 19:14:15,490][105692] Updated weights for policy 0, policy_version 531614 (0.0008) [2023-12-26 19:14:16,058][105620] Updated weights for policy 1, policy_version 532396 (0.0007) [2023-12-26 19:14:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19355.3). Total num frames: 272416768. Throughput: 0: 9543.3, 1: 9587.1. Samples: 272391440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:14:16,062][104569] Avg episode reward: [(0, '8992.506'), (1, '1691.514')] [2023-12-26 19:14:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000531616_136110080.pth... [2023-12-26 19:14:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000530496_135823360.pth [2023-12-26 19:14:16,126][105620] Updated weights for policy 1, policy_version 532406 (0.0006) [2023-12-26 19:14:16,189][105620] Updated weights for policy 1, policy_version 532416 (0.0005) [2023-12-26 19:14:16,239][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000532424_136314880.pth... [2023-12-26 19:14:16,243][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000531272_136019968.pth [2023-12-26 19:14:16,250][105692] Updated weights for policy 0, policy_version 531624 (0.0007) [2023-12-26 19:14:16,302][105692] Updated weights for policy 0, policy_version 531634 (0.0008) [2023-12-26 19:14:16,348][105692] Updated weights for policy 0, policy_version 531644 (0.0009) [2023-12-26 19:14:16,745][105620] Updated weights for policy 1, policy_version 532426 (0.0006) [2023-12-26 19:14:16,809][105620] Updated weights for policy 1, policy_version 532436 (0.0005) [2023-12-26 19:14:16,865][105620] Updated weights for policy 1, policy_version 532446 (0.0009) [2023-12-26 19:14:16,923][105620] Updated weights for policy 1, policy_version 532456 (0.0007) [2023-12-26 19:14:16,994][105692] Updated weights for policy 0, policy_version 531654 (0.0006) [2023-12-26 19:14:17,054][105692] Updated weights for policy 0, policy_version 531664 (0.0005) [2023-12-26 19:14:17,100][105692] Updated weights for policy 0, policy_version 531674 (0.0005) [2023-12-26 19:14:17,573][105620] Updated weights for policy 1, policy_version 532466 (0.0005) [2023-12-26 19:14:17,637][105620] Updated weights for policy 1, policy_version 532476 (0.0005) [2023-12-26 19:14:17,686][105620] Updated weights for policy 1, policy_version 532486 (0.0010) [2023-12-26 19:14:17,788][105692] Updated weights for policy 0, policy_version 531684 (0.0005) [2023-12-26 19:14:17,849][105692] Updated weights for policy 0, policy_version 531694 (0.0005) [2023-12-26 19:14:17,910][105692] Updated weights for policy 0, policy_version 531704 (0.0005) [2023-12-26 19:14:18,268][105620] Updated weights for policy 1, policy_version 532496 (0.0007) [2023-12-26 19:14:18,317][105620] Updated weights for policy 1, policy_version 532506 (0.0008) [2023-12-26 19:14:18,376][105620] Updated weights for policy 1, policy_version 532516 (0.0009) [2023-12-26 19:14:18,485][105692] Updated weights for policy 0, policy_version 531714 (0.0006) [2023-12-26 19:14:18,541][105692] Updated weights for policy 0, policy_version 531724 (0.0011) [2023-12-26 19:14:18,599][105692] Updated weights for policy 0, policy_version 531734 (0.0008) [2023-12-26 19:14:18,662][105692] Updated weights for policy 0, policy_version 531744 (0.0007) [2023-12-26 19:14:19,072][105620] Updated weights for policy 1, policy_version 532526 (0.0008) [2023-12-26 19:14:19,132][105620] Updated weights for policy 1, policy_version 532536 (0.0008) [2023-12-26 19:14:19,191][105620] Updated weights for policy 1, policy_version 532546 (0.0007) [2023-12-26 19:14:19,396][105692] Updated weights for policy 0, policy_version 531754 (0.0008) [2023-12-26 19:14:19,453][105692] Updated weights for policy 0, policy_version 531764 (0.0008) [2023-12-26 19:14:19,517][105692] Updated weights for policy 0, policy_version 531774 (0.0009) [2023-12-26 19:14:19,893][105620] Updated weights for policy 1, policy_version 532556 (0.0007) [2023-12-26 19:14:19,960][105620] Updated weights for policy 1, policy_version 532566 (0.0008) [2023-12-26 19:14:20,024][105620] Updated weights for policy 1, policy_version 532576 (0.0009) [2023-12-26 19:14:20,378][105692] Updated weights for policy 0, policy_version 531784 (0.0009) [2023-12-26 19:14:20,439][105692] Updated weights for policy 0, policy_version 531794 (0.0009) [2023-12-26 19:14:20,504][105692] Updated weights for policy 0, policy_version 531804 (0.0008) [2023-12-26 19:14:20,656][105620] Updated weights for policy 1, policy_version 532586 (0.0006) [2023-12-26 19:14:20,718][105620] Updated weights for policy 1, policy_version 532596 (0.0009) [2023-12-26 19:14:20,780][105620] Updated weights for policy 1, policy_version 532606 (0.0009) [2023-12-26 19:14:20,846][105620] Updated weights for policy 1, policy_version 532616 (0.0006) [2023-12-26 19:14:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 272523264. Throughput: 0: 9652.0, 1: 9644.0. Samples: 272513260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:14:21,062][104569] Avg episode reward: [(0, '8992.256'), (1, '2883.403')] [2023-12-26 19:14:21,409][105692] Updated weights for policy 0, policy_version 531814 (0.0009) [2023-12-26 19:14:21,470][105692] Updated weights for policy 0, policy_version 531824 (0.0008) [2023-12-26 19:14:21,522][105692] Updated weights for policy 0, policy_version 531834 (0.0008) [2023-12-26 19:14:21,581][105620] Updated weights for policy 1, policy_version 532626 (0.0009) [2023-12-26 19:14:21,639][105620] Updated weights for policy 1, policy_version 532636 (0.0008) [2023-12-26 19:14:21,702][105620] Updated weights for policy 1, policy_version 532646 (0.0008) [2023-12-26 19:14:22,356][105692] Updated weights for policy 0, policy_version 531844 (0.0008) [2023-12-26 19:14:22,418][105692] Updated weights for policy 0, policy_version 531854 (0.0008) [2023-12-26 19:14:22,434][105620] Updated weights for policy 1, policy_version 532656 (0.0007) [2023-12-26 19:14:22,480][105692] Updated weights for policy 0, policy_version 531864 (0.0008) [2023-12-26 19:14:22,492][105620] Updated weights for policy 1, policy_version 532666 (0.0007) [2023-12-26 19:14:22,553][105620] Updated weights for policy 1, policy_version 532676 (0.0010) [2023-12-26 19:14:23,236][105620] Updated weights for policy 1, policy_version 532686 (0.0009) [2023-12-26 19:14:23,236][105692] Updated weights for policy 0, policy_version 531874 (0.0007) [2023-12-26 19:14:23,290][105620] Updated weights for policy 1, policy_version 532696 (0.0010) [2023-12-26 19:14:23,290][105692] Updated weights for policy 0, policy_version 531884 (0.0006) [2023-12-26 19:14:23,346][105692] Updated weights for policy 0, policy_version 531894 (0.0005) [2023-12-26 19:14:23,348][105620] Updated weights for policy 1, policy_version 532706 (0.0010) [2023-12-26 19:14:23,403][105692] Updated weights for policy 0, policy_version 531904 (0.0006) [2023-12-26 19:14:24,010][105620] Updated weights for policy 1, policy_version 532716 (0.0010) [2023-12-26 19:14:24,058][105620] Updated weights for policy 1, policy_version 532726 (0.0010) [2023-12-26 19:14:24,061][105585] KL-divergence is very high: 223.4874 [2023-12-26 19:14:24,070][105692] Updated weights for policy 0, policy_version 531914 (0.0005) [2023-12-26 19:14:24,103][105585] KL-divergence is very high: 301.9226 [2023-12-26 19:14:24,106][105620] Updated weights for policy 1, policy_version 532736 (0.0010) [2023-12-26 19:14:24,123][105692] Updated weights for policy 0, policy_version 531924 (0.0007) [2023-12-26 19:14:24,145][105585] KL-divergence is very high: 380.8185 [2023-12-26 19:14:24,174][105692] Updated weights for policy 0, policy_version 531934 (0.0008) [2023-12-26 19:14:24,767][105620] Updated weights for policy 1, policy_version 532746 (0.0010) [2023-12-26 19:14:24,818][105620] Updated weights for policy 1, policy_version 532756 (0.0010) [2023-12-26 19:14:24,866][105620] Updated weights for policy 1, policy_version 532766 (0.0010) [2023-12-26 19:14:24,918][105620] Updated weights for policy 1, policy_version 532776 (0.0010) [2023-12-26 19:14:24,959][105585] KL-divergence is very high: 232.9038 [2023-12-26 19:14:25,002][105585] KL-divergence is very high: 192.3765 [2023-12-26 19:14:25,003][105692] Updated weights for policy 0, policy_version 531944 (0.0008) [2023-12-26 19:14:25,042][105585] KL-divergence is very high: 111.8221 [2023-12-26 19:14:25,053][105692] Updated weights for policy 0, policy_version 531954 (0.0008) [2023-12-26 19:14:25,104][105692] Updated weights for policy 0, policy_version 531964 (0.0009) [2023-12-26 19:14:25,618][105620] Updated weights for policy 1, policy_version 532786 (0.0010) [2023-12-26 19:14:25,673][105620] Updated weights for policy 1, policy_version 532796 (0.0010) [2023-12-26 19:14:25,734][105620] Updated weights for policy 1, policy_version 532806 (0.0010) [2023-12-26 19:14:25,828][105692] Updated weights for policy 0, policy_version 531975 (0.0007) [2023-12-26 19:14:25,885][105692] Updated weights for policy 0, policy_version 531985 (0.0005) [2023-12-26 19:14:25,934][105692] Updated weights for policy 0, policy_version 531995 (0.0005) [2023-12-26 19:14:26,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 272621568. Throughput: 0: 9608.5, 1: 9718.5. Samples: 272626808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:14:26,063][104569] Avg episode reward: [(0, '9175.108'), (1, '6857.536')] [2023-12-26 19:14:26,422][105620] Updated weights for policy 1, policy_version 532816 (0.0010) [2023-12-26 19:14:26,486][105620] Updated weights for policy 1, policy_version 532826 (0.0010) [2023-12-26 19:14:26,534][105620] Updated weights for policy 1, policy_version 532836 (0.0010) [2023-12-26 19:14:26,605][105692] Updated weights for policy 0, policy_version 532005 (0.0008) [2023-12-26 19:14:26,660][105692] Updated weights for policy 0, policy_version 532015 (0.0008) [2023-12-26 19:14:26,717][105692] Updated weights for policy 0, policy_version 532025 (0.0007) [2023-12-26 19:14:27,269][105620] Updated weights for policy 1, policy_version 532846 (0.0010) [2023-12-26 19:14:27,322][105620] Updated weights for policy 1, policy_version 532856 (0.0009) [2023-12-26 19:14:27,370][105620] Updated weights for policy 1, policy_version 532866 (0.0010) [2023-12-26 19:14:27,413][105692] Updated weights for policy 0, policy_version 532035 (0.0006) [2023-12-26 19:14:27,465][105692] Updated weights for policy 0, policy_version 532045 (0.0008) [2023-12-26 19:14:27,513][105692] Updated weights for policy 0, policy_version 532055 (0.0008) [2023-12-26 19:14:28,068][105620] Updated weights for policy 1, policy_version 532876 (0.0008) [2023-12-26 19:14:28,116][105620] Updated weights for policy 1, policy_version 532886 (0.0005) [2023-12-26 19:14:28,169][105620] Updated weights for policy 1, policy_version 532896 (0.0005) [2023-12-26 19:14:28,356][105692] Updated weights for policy 0, policy_version 532065 (0.0008) [2023-12-26 19:14:28,409][105692] Updated weights for policy 0, policy_version 532075 (0.0008) [2023-12-26 19:14:28,469][105692] Updated weights for policy 0, policy_version 532085 (0.0008) [2023-12-26 19:14:28,526][105692] Updated weights for policy 0, policy_version 532095 (0.0008) [2023-12-26 19:14:28,779][105620] Updated weights for policy 1, policy_version 532906 (0.0006) [2023-12-26 19:14:28,841][105620] Updated weights for policy 1, policy_version 532916 (0.0010) [2023-12-26 19:14:28,899][105620] Updated weights for policy 1, policy_version 532926 (0.0010) [2023-12-26 19:14:28,958][105620] Updated weights for policy 1, policy_version 532936 (0.0010) [2023-12-26 19:14:29,380][105692] Updated weights for policy 0, policy_version 532105 (0.0008) [2023-12-26 19:14:29,440][105692] Updated weights for policy 0, policy_version 532115 (0.0008) [2023-12-26 19:14:29,503][105692] Updated weights for policy 0, policy_version 532125 (0.0007) [2023-12-26 19:14:29,631][105620] Updated weights for policy 1, policy_version 532946 (0.0008) [2023-12-26 19:14:29,698][105620] Updated weights for policy 1, policy_version 532956 (0.0008) [2023-12-26 19:14:29,756][105620] Updated weights for policy 1, policy_version 532966 (0.0010) [2023-12-26 19:14:30,209][105692] Updated weights for policy 0, policy_version 532135 (0.0009) [2023-12-26 19:14:30,268][105692] Updated weights for policy 0, policy_version 532145 (0.0010) [2023-12-26 19:14:30,333][105692] Updated weights for policy 0, policy_version 532155 (0.0010) [2023-12-26 19:14:30,393][105620] Updated weights for policy 1, policy_version 532976 (0.0008) [2023-12-26 19:14:30,451][105620] Updated weights for policy 1, policy_version 532986 (0.0007) [2023-12-26 19:14:30,512][105620] Updated weights for policy 1, policy_version 532996 (0.0008) [2023-12-26 19:14:30,993][105692] Updated weights for policy 0, policy_version 532165 (0.0008) [2023-12-26 19:14:31,050][105692] Updated weights for policy 0, policy_version 532175 (0.0007) [2023-12-26 19:14:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19114.6, 300 sec: 19355.3). Total num frames: 272711680. Throughput: 0: 9647.2, 1: 9797.1. Samples: 272686764. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-26 19:14:31,063][104569] Avg episode reward: [(0, '8900.260'), (1, '8903.847')] [2023-12-26 19:14:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000533000_136462336.pth... [2023-12-26 19:14:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000531848_136167424.pth [2023-12-26 19:14:31,107][105692] Updated weights for policy 0, policy_version 532185 (0.0006) [2023-12-26 19:14:31,150][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000532192_136257536.pth... [2023-12-26 19:14:31,155][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000531040_135962624.pth [2023-12-26 19:14:31,347][105620] Updated weights for policy 1, policy_version 533006 (0.0007) [2023-12-26 19:14:31,416][105620] Updated weights for policy 1, policy_version 533017 (0.0010) [2023-12-26 19:14:31,469][105620] Updated weights for policy 1, policy_version 533027 (0.0007) [2023-12-26 19:14:31,827][105692] Updated weights for policy 0, policy_version 532195 (0.0008) [2023-12-26 19:14:31,889][105692] Updated weights for policy 0, policy_version 532205 (0.0010) [2023-12-26 19:14:31,954][105692] Updated weights for policy 0, policy_version 532215 (0.0010) [2023-12-26 19:14:32,145][105620] Updated weights for policy 1, policy_version 533037 (0.0008) [2023-12-26 19:14:32,201][105620] Updated weights for policy 1, policy_version 533047 (0.0005) [2023-12-26 19:14:32,263][105620] Updated weights for policy 1, policy_version 533057 (0.0006) [2023-12-26 19:14:32,771][105692] Updated weights for policy 0, policy_version 532225 (0.0009) [2023-12-26 19:14:32,832][105692] Updated weights for policy 0, policy_version 532235 (0.0014) [2023-12-26 19:14:32,877][105620] Updated weights for policy 1, policy_version 533067 (0.0006) [2023-12-26 19:14:32,882][105692] Updated weights for policy 0, policy_version 532245 (0.0007) [2023-12-26 19:14:32,932][105620] Updated weights for policy 1, policy_version 533077 (0.0006) [2023-12-26 19:14:32,937][105692] Updated weights for policy 0, policy_version 532255 (0.0005) [2023-12-26 19:14:32,984][105620] Updated weights for policy 1, policy_version 533087 (0.0009) [2023-12-26 19:14:33,499][105692] Updated weights for policy 0, policy_version 532265 (0.0008) [2023-12-26 19:14:33,554][105692] Updated weights for policy 0, policy_version 532275 (0.0009) [2023-12-26 19:14:33,602][105692] Updated weights for policy 0, policy_version 532285 (0.0008) [2023-12-26 19:14:33,749][105620] Updated weights for policy 1, policy_version 533097 (0.0007) [2023-12-26 19:14:33,806][105620] Updated weights for policy 1, policy_version 533107 (0.0009) [2023-12-26 19:14:33,870][105620] Updated weights for policy 1, policy_version 533117 (0.0009) [2023-12-26 19:14:33,919][105620] Updated weights for policy 1, policy_version 533127 (0.0008) [2023-12-26 19:14:34,275][105692] Updated weights for policy 0, policy_version 532295 (0.0007) [2023-12-26 19:14:34,346][105692] Updated weights for policy 0, policy_version 532305 (0.0005) [2023-12-26 19:14:34,407][105692] Updated weights for policy 0, policy_version 532315 (0.0007) [2023-12-26 19:14:34,775][105620] Updated weights for policy 1, policy_version 533137 (0.0009) [2023-12-26 19:14:34,848][105620] Updated weights for policy 1, policy_version 533147 (0.0010) [2023-12-26 19:14:34,914][105620] Updated weights for policy 1, policy_version 533157 (0.0008) [2023-12-26 19:14:34,999][105692] Updated weights for policy 0, policy_version 532325 (0.0010) [2023-12-26 19:14:35,063][105692] Updated weights for policy 0, policy_version 532335 (0.0009) [2023-12-26 19:14:35,127][105692] Updated weights for policy 0, policy_version 532345 (0.0007) [2023-12-26 19:14:35,652][105620] Updated weights for policy 1, policy_version 533167 (0.0010) [2023-12-26 19:14:35,697][105620] Updated weights for policy 1, policy_version 533177 (0.0010) [2023-12-26 19:14:35,763][105620] Updated weights for policy 1, policy_version 533187 (0.0010) [2023-12-26 19:14:35,785][105692] Updated weights for policy 0, policy_version 532355 (0.0008) [2023-12-26 19:14:35,839][105692] Updated weights for policy 0, policy_version 532365 (0.0006) [2023-12-26 19:14:35,890][105692] Updated weights for policy 0, policy_version 532375 (0.0008) [2023-12-26 19:14:36,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 272818176. Throughput: 0: 9663.7, 1: 9761.2. Samples: 272803528. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-26 19:14:36,062][104569] Avg episode reward: [(0, '8899.949'), (1, '8723.312')] [2023-12-26 19:14:36,520][105620] Updated weights for policy 1, policy_version 533197 (0.0009) [2023-12-26 19:14:36,587][105620] Updated weights for policy 1, policy_version 533207 (0.0009) [2023-12-26 19:14:36,600][105692] Updated weights for policy 0, policy_version 532385 (0.0008) [2023-12-26 19:14:36,658][105620] Updated weights for policy 1, policy_version 533217 (0.0006) [2023-12-26 19:14:36,659][105692] Updated weights for policy 0, policy_version 532395 (0.0011) [2023-12-26 19:14:36,716][105692] Updated weights for policy 0, policy_version 532405 (0.0011) [2023-12-26 19:14:36,781][105692] Updated weights for policy 0, policy_version 532415 (0.0010) [2023-12-26 19:14:37,336][105620] Updated weights for policy 1, policy_version 533227 (0.0006) [2023-12-26 19:14:37,401][105620] Updated weights for policy 1, policy_version 533237 (0.0008) [2023-12-26 19:14:37,445][105692] Updated weights for policy 0, policy_version 532425 (0.0009) [2023-12-26 19:14:37,460][105620] Updated weights for policy 1, policy_version 533247 (0.0007) [2023-12-26 19:14:37,498][105692] Updated weights for policy 0, policy_version 532435 (0.0006) [2023-12-26 19:14:37,555][105692] Updated weights for policy 0, policy_version 532446 (0.0007) [2023-12-26 19:14:38,066][105620] Updated weights for policy 1, policy_version 533257 (0.0008) [2023-12-26 19:14:38,117][105620] Updated weights for policy 1, policy_version 533267 (0.0009) [2023-12-26 19:14:38,166][105620] Updated weights for policy 1, policy_version 533277 (0.0007) [2023-12-26 19:14:38,221][105620] Updated weights for policy 1, policy_version 533287 (0.0005) [2023-12-26 19:14:38,394][105692] Updated weights for policy 0, policy_version 532456 (0.0009) [2023-12-26 19:14:38,452][105692] Updated weights for policy 0, policy_version 532466 (0.0010) [2023-12-26 19:14:38,511][105692] Updated weights for policy 0, policy_version 532476 (0.0010) [2023-12-26 19:14:38,885][105620] Updated weights for policy 1, policy_version 533297 (0.0008) [2023-12-26 19:14:38,936][105620] Updated weights for policy 1, policy_version 533307 (0.0008) [2023-12-26 19:14:38,992][105620] Updated weights for policy 1, policy_version 533317 (0.0009) [2023-12-26 19:14:39,266][105692] Updated weights for policy 0, policy_version 532486 (0.0009) [2023-12-26 19:14:39,332][105692] Updated weights for policy 0, policy_version 532496 (0.0009) [2023-12-26 19:14:39,395][105692] Updated weights for policy 0, policy_version 532506 (0.0008) [2023-12-26 19:14:39,760][105620] Updated weights for policy 1, policy_version 533327 (0.0009) [2023-12-26 19:14:39,829][105620] Updated weights for policy 1, policy_version 533337 (0.0009) [2023-12-26 19:14:39,894][105620] Updated weights for policy 1, policy_version 533347 (0.0009) [2023-12-26 19:14:40,137][105692] Updated weights for policy 0, policy_version 532516 (0.0009) [2023-12-26 19:14:40,201][105692] Updated weights for policy 0, policy_version 532526 (0.0006) [2023-12-26 19:14:40,252][105692] Updated weights for policy 0, policy_version 532536 (0.0007) [2023-12-26 19:14:40,685][105620] Updated weights for policy 1, policy_version 533357 (0.0009) [2023-12-26 19:14:40,753][105620] Updated weights for policy 1, policy_version 533367 (0.0007) [2023-12-26 19:14:40,815][105620] Updated weights for policy 1, policy_version 533377 (0.0008) [2023-12-26 19:14:40,901][105692] Updated weights for policy 0, policy_version 532546 (0.0006) [2023-12-26 19:14:40,966][105692] Updated weights for policy 0, policy_version 532556 (0.0005) [2023-12-26 19:14:41,028][105692] Updated weights for policy 0, policy_version 532566 (0.0009) [2023-12-26 19:14:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19355.3). Total num frames: 272908288. Throughput: 0: 9716.0, 1: 9825.9. Samples: 272919548. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-26 19:14:41,062][104569] Avg episode reward: [(0, '8993.135'), (1, '8993.332')] [2023-12-26 19:14:41,091][105692] Updated weights for policy 0, policy_version 532576 (0.0007) [2023-12-26 19:14:41,524][105620] Updated weights for policy 1, policy_version 533387 (0.0010) [2023-12-26 19:14:41,584][105620] Updated weights for policy 1, policy_version 533397 (0.0008) [2023-12-26 19:14:41,645][105620] Updated weights for policy 1, policy_version 533407 (0.0008) [2023-12-26 19:14:41,844][105692] Updated weights for policy 0, policy_version 532586 (0.0009) [2023-12-26 19:14:41,903][105692] Updated weights for policy 0, policy_version 532596 (0.0009) [2023-12-26 19:14:41,958][105692] Updated weights for policy 0, policy_version 532606 (0.0009) [2023-12-26 19:14:42,370][105620] Updated weights for policy 1, policy_version 533417 (0.0009) [2023-12-26 19:14:42,439][105620] Updated weights for policy 1, policy_version 533427 (0.0008) [2023-12-26 19:14:42,493][105620] Updated weights for policy 1, policy_version 533437 (0.0009) [2023-12-26 19:14:42,540][105620] Updated weights for policy 1, policy_version 533447 (0.0008) [2023-12-26 19:14:42,770][105692] Updated weights for policy 0, policy_version 532616 (0.0010) [2023-12-26 19:14:42,831][105692] Updated weights for policy 0, policy_version 532626 (0.0009) [2023-12-26 19:14:42,901][105692] Updated weights for policy 0, policy_version 532636 (0.0010) [2023-12-26 19:14:43,157][105620] Updated weights for policy 1, policy_version 533457 (0.0008) [2023-12-26 19:14:43,214][105620] Updated weights for policy 1, policy_version 533467 (0.0009) [2023-12-26 19:14:43,267][105620] Updated weights for policy 1, policy_version 533477 (0.0011) [2023-12-26 19:14:43,531][105692] Updated weights for policy 0, policy_version 532646 (0.0007) [2023-12-26 19:14:43,577][105692] Updated weights for policy 0, policy_version 532656 (0.0005) [2023-12-26 19:14:43,621][105692] Updated weights for policy 0, policy_version 532666 (0.0005) [2023-12-26 19:14:44,051][105620] Updated weights for policy 1, policy_version 533487 (0.0009) [2023-12-26 19:14:44,112][105620] Updated weights for policy 1, policy_version 533497 (0.0009) [2023-12-26 19:14:44,167][105620] Updated weights for policy 1, policy_version 533507 (0.0008) [2023-12-26 19:14:44,286][105692] Updated weights for policy 0, policy_version 532676 (0.0007) [2023-12-26 19:14:44,342][105692] Updated weights for policy 0, policy_version 532686 (0.0009) [2023-12-26 19:14:44,362][105585] KL-divergence is very high: 170.7803 [2023-12-26 19:14:44,393][105585] KL-divergence is very high: 165.0886 [2023-12-26 19:14:44,405][105692] Updated weights for policy 0, policy_version 532696 (0.0008) [2023-12-26 19:14:44,410][105585] KL-divergence is very high: 277.0636 [2023-12-26 19:14:44,438][105585] KL-divergence is very high: 187.5670 [2023-12-26 19:14:44,900][105620] Updated weights for policy 1, policy_version 533517 (0.0008) [2023-12-26 19:14:44,969][105620] Updated weights for policy 1, policy_version 533527 (0.0007) [2023-12-26 19:14:45,016][105692] Updated weights for policy 0, policy_version 532706 (0.0007) [2023-12-26 19:14:45,026][105620] Updated weights for policy 1, policy_version 533537 (0.0006) [2023-12-26 19:14:45,083][105692] Updated weights for policy 0, policy_version 532716 (0.0011) [2023-12-26 19:14:45,150][105692] Updated weights for policy 0, policy_version 532726 (0.0011) [2023-12-26 19:14:45,216][105692] Updated weights for policy 0, policy_version 532736 (0.0009) [2023-12-26 19:14:45,720][105620] Updated weights for policy 1, policy_version 533547 (0.0006) [2023-12-26 19:14:45,770][105620] Updated weights for policy 1, policy_version 533557 (0.0009) [2023-12-26 19:14:45,817][105620] Updated weights for policy 1, policy_version 533567 (0.0008) [2023-12-26 19:14:45,916][105692] Updated weights for policy 0, policy_version 532746 (0.0009) [2023-12-26 19:14:45,967][105692] Updated weights for policy 0, policy_version 532756 (0.0009) [2023-12-26 19:14:46,025][105692] Updated weights for policy 0, policy_version 532766 (0.0009) [2023-12-26 19:14:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.4, 300 sec: 19383.1). Total num frames: 273014784. Throughput: 0: 9682.3, 1: 9806.1. Samples: 272978428. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-26 19:14:46,062][104569] Avg episode reward: [(0, '8810.666'), (1, '9173.343')] [2023-12-26 19:14:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000532768_136404992.pth... [2023-12-26 19:14:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000533576_136609792.pth... [2023-12-26 19:14:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000532424_136314880.pth [2023-12-26 19:14:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000531616_136110080.pth [2023-12-26 19:14:46,599][105620] Updated weights for policy 1, policy_version 533577 (0.0009) [2023-12-26 19:14:46,648][105620] Updated weights for policy 1, policy_version 533588 (0.0009) [2023-12-26 19:14:46,702][105620] Updated weights for policy 1, policy_version 533598 (0.0008) [2023-12-26 19:14:46,751][105692] Updated weights for policy 0, policy_version 532776 (0.0009) [2023-12-26 19:14:46,757][105620] Updated weights for policy 1, policy_version 533608 (0.0008) [2023-12-26 19:14:46,808][105692] Updated weights for policy 0, policy_version 532786 (0.0009) [2023-12-26 19:14:46,869][105692] Updated weights for policy 0, policy_version 532796 (0.0009) [2023-12-26 19:14:47,508][105620] Updated weights for policy 1, policy_version 533618 (0.0009) [2023-12-26 19:14:47,556][105620] Updated weights for policy 1, policy_version 533628 (0.0010) [2023-12-26 19:14:47,615][105620] Updated weights for policy 1, policy_version 533638 (0.0010) [2023-12-26 19:14:47,636][105692] Updated weights for policy 0, policy_version 532806 (0.0008) [2023-12-26 19:14:47,691][105692] Updated weights for policy 0, policy_version 532816 (0.0008) [2023-12-26 19:14:47,742][105692] Updated weights for policy 0, policy_version 532826 (0.0008) [2023-12-26 19:14:48,274][105620] Updated weights for policy 1, policy_version 533648 (0.0006) [2023-12-26 19:14:48,342][105620] Updated weights for policy 1, policy_version 533658 (0.0007) [2023-12-26 19:14:48,404][105620] Updated weights for policy 1, policy_version 533668 (0.0011) [2023-12-26 19:14:48,547][105692] Updated weights for policy 0, policy_version 532836 (0.0008) [2023-12-26 19:14:48,614][105692] Updated weights for policy 0, policy_version 532846 (0.0009) [2023-12-26 19:14:48,671][105692] Updated weights for policy 0, policy_version 532856 (0.0009) [2023-12-26 19:14:49,053][105620] Updated weights for policy 1, policy_version 533678 (0.0009) [2023-12-26 19:14:49,111][105620] Updated weights for policy 1, policy_version 533688 (0.0008) [2023-12-26 19:14:49,172][105620] Updated weights for policy 1, policy_version 533698 (0.0008) [2023-12-26 19:14:49,402][105692] Updated weights for policy 0, policy_version 532866 (0.0009) [2023-12-26 19:14:49,462][105692] Updated weights for policy 0, policy_version 532876 (0.0011) [2023-12-26 19:14:49,514][105692] Updated weights for policy 0, policy_version 532886 (0.0011) [2023-12-26 19:14:49,567][105692] Updated weights for policy 0, policy_version 532896 (0.0006) [2023-12-26 19:14:49,893][105620] Updated weights for policy 1, policy_version 533708 (0.0010) [2023-12-26 19:14:49,959][105620] Updated weights for policy 1, policy_version 533718 (0.0011) [2023-12-26 19:14:50,025][105620] Updated weights for policy 1, policy_version 533728 (0.0007) [2023-12-26 19:14:50,277][105692] Updated weights for policy 0, policy_version 532906 (0.0009) [2023-12-26 19:14:50,341][105692] Updated weights for policy 0, policy_version 532916 (0.0008) [2023-12-26 19:14:50,407][105692] Updated weights for policy 0, policy_version 532926 (0.0006) [2023-12-26 19:14:50,697][105620] Updated weights for policy 1, policy_version 533738 (0.0006) [2023-12-26 19:14:50,745][105620] Updated weights for policy 1, policy_version 533748 (0.0009) [2023-12-26 19:14:50,795][105620] Updated weights for policy 1, policy_version 533758 (0.0008) [2023-12-26 19:14:50,842][105620] Updated weights for policy 1, policy_version 533768 (0.0005) [2023-12-26 19:14:51,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19383.1). Total num frames: 273104896. Throughput: 0: 9745.7, 1: 9813.7. Samples: 273095296. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-26 19:14:51,063][104569] Avg episode reward: [(0, '8900.327'), (1, '9173.117')] [2023-12-26 19:14:51,218][105692] Updated weights for policy 0, policy_version 532936 (0.0009) [2023-12-26 19:14:51,283][105692] Updated weights for policy 0, policy_version 532946 (0.0009) [2023-12-26 19:14:51,347][105692] Updated weights for policy 0, policy_version 532956 (0.0009) [2023-12-26 19:14:51,515][105620] Updated weights for policy 1, policy_version 533778 (0.0008) [2023-12-26 19:14:51,575][105620] Updated weights for policy 1, policy_version 533788 (0.0009) [2023-12-26 19:14:51,641][105620] Updated weights for policy 1, policy_version 533798 (0.0009) [2023-12-26 19:14:52,068][105692] Updated weights for policy 0, policy_version 532966 (0.0009) [2023-12-26 19:14:52,132][105692] Updated weights for policy 0, policy_version 532976 (0.0009) [2023-12-26 19:14:52,190][105692] Updated weights for policy 0, policy_version 532986 (0.0010) [2023-12-26 19:14:52,317][105620] Updated weights for policy 1, policy_version 533808 (0.0011) [2023-12-26 19:14:52,377][105620] Updated weights for policy 1, policy_version 533818 (0.0009) [2023-12-26 19:14:52,440][105620] Updated weights for policy 1, policy_version 533828 (0.0009) [2023-12-26 19:14:52,930][105692] Updated weights for policy 0, policy_version 532996 (0.0009) [2023-12-26 19:14:52,990][105692] Updated weights for policy 0, policy_version 533006 (0.0008) [2023-12-26 19:14:53,060][105692] Updated weights for policy 0, policy_version 533016 (0.0008) [2023-12-26 19:14:53,140][105620] Updated weights for policy 1, policy_version 533838 (0.0008) [2023-12-26 19:14:53,199][105620] Updated weights for policy 1, policy_version 533848 (0.0008) [2023-12-26 19:14:53,259][105620] Updated weights for policy 1, policy_version 533858 (0.0008) [2023-12-26 19:14:53,733][105692] Updated weights for policy 0, policy_version 533026 (0.0007) [2023-12-26 19:14:53,792][105692] Updated weights for policy 0, policy_version 533036 (0.0005) [2023-12-26 19:14:53,838][105692] Updated weights for policy 0, policy_version 533046 (0.0005) [2023-12-26 19:14:53,882][105692] Updated weights for policy 0, policy_version 533056 (0.0005) [2023-12-26 19:14:53,988][105620] Updated weights for policy 1, policy_version 533868 (0.0007) [2023-12-26 19:14:54,050][105620] Updated weights for policy 1, policy_version 533878 (0.0010) [2023-12-26 19:14:54,102][105620] Updated weights for policy 1, policy_version 533888 (0.0010) [2023-12-26 19:14:54,559][105692] Updated weights for policy 0, policy_version 533066 (0.0008) [2023-12-26 19:14:54,619][105692] Updated weights for policy 0, policy_version 533076 (0.0009) [2023-12-26 19:14:54,670][105692] Updated weights for policy 0, policy_version 533086 (0.0005) [2023-12-26 19:14:54,825][105620] Updated weights for policy 1, policy_version 533898 (0.0010) [2023-12-26 19:14:54,875][105620] Updated weights for policy 1, policy_version 533908 (0.0008) [2023-12-26 19:14:54,930][105620] Updated weights for policy 1, policy_version 533918 (0.0009) [2023-12-26 19:14:54,987][105620] Updated weights for policy 1, policy_version 533928 (0.0008) [2023-12-26 19:14:55,406][105692] Updated weights for policy 0, policy_version 533096 (0.0008) [2023-12-26 19:14:55,470][105692] Updated weights for policy 0, policy_version 533106 (0.0008) [2023-12-26 19:14:55,523][105692] Updated weights for policy 0, policy_version 533116 (0.0007) [2023-12-26 19:14:55,749][105620] Updated weights for policy 1, policy_version 533938 (0.0009) [2023-12-26 19:14:55,810][105620] Updated weights for policy 1, policy_version 533948 (0.0009) [2023-12-26 19:14:55,870][105620] Updated weights for policy 1, policy_version 533958 (0.0009) [2023-12-26 19:14:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 273203200. Throughput: 0: 9654.6, 1: 9846.8. Samples: 273210864. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-26 19:14:56,062][104569] Avg episode reward: [(0, '8626.976'), (1, '8994.340')] [2023-12-26 19:14:56,308][105692] Updated weights for policy 0, policy_version 533126 (0.0010) [2023-12-26 19:14:56,355][105692] Updated weights for policy 0, policy_version 533136 (0.0009) [2023-12-26 19:14:56,405][105692] Updated weights for policy 0, policy_version 533146 (0.0009) [2023-12-26 19:14:56,529][105620] Updated weights for policy 1, policy_version 533968 (0.0009) [2023-12-26 19:14:56,590][105620] Updated weights for policy 1, policy_version 533978 (0.0009) [2023-12-26 19:14:56,646][105620] Updated weights for policy 1, policy_version 533988 (0.0009) [2023-12-26 19:14:57,172][105692] Updated weights for policy 0, policy_version 533157 (0.0010) [2023-12-26 19:14:57,219][105692] Updated weights for policy 0, policy_version 533167 (0.0007) [2023-12-26 19:14:57,268][105692] Updated weights for policy 0, policy_version 533177 (0.0005) [2023-12-26 19:14:57,384][105620] Updated weights for policy 1, policy_version 533998 (0.0007) [2023-12-26 19:14:57,448][105620] Updated weights for policy 1, policy_version 534008 (0.0007) [2023-12-26 19:14:57,498][105620] Updated weights for policy 1, policy_version 534018 (0.0008) [2023-12-26 19:14:57,947][105692] Updated weights for policy 0, policy_version 533187 (0.0009) [2023-12-26 19:14:58,001][105692] Updated weights for policy 0, policy_version 533197 (0.0010) [2023-12-26 19:14:58,049][105692] Updated weights for policy 0, policy_version 533207 (0.0010) [2023-12-26 19:14:58,257][105620] Updated weights for policy 1, policy_version 534028 (0.0008) [2023-12-26 19:14:58,325][105620] Updated weights for policy 1, policy_version 534038 (0.0007) [2023-12-26 19:14:58,406][105620] Updated weights for policy 1, policy_version 534048 (0.0009) [2023-12-26 19:14:58,874][105692] Updated weights for policy 0, policy_version 533217 (0.0010) [2023-12-26 19:14:58,937][105692] Updated weights for policy 0, policy_version 533227 (0.0009) [2023-12-26 19:14:59,005][105692] Updated weights for policy 0, policy_version 533237 (0.0009) [2023-12-26 19:14:59,072][105692] Updated weights for policy 0, policy_version 533247 (0.0009) [2023-12-26 19:14:59,178][105620] Updated weights for policy 1, policy_version 534058 (0.0009) [2023-12-26 19:14:59,242][105620] Updated weights for policy 1, policy_version 534068 (0.0009) [2023-12-26 19:14:59,303][105620] Updated weights for policy 1, policy_version 534078 (0.0007) [2023-12-26 19:14:59,370][105620] Updated weights for policy 1, policy_version 534088 (0.0008) [2023-12-26 19:14:59,787][105692] Updated weights for policy 0, policy_version 533257 (0.0010) [2023-12-26 19:14:59,854][105692] Updated weights for policy 0, policy_version 533267 (0.0010) [2023-12-26 19:14:59,910][105692] Updated weights for policy 0, policy_version 533277 (0.0010) [2023-12-26 19:15:00,095][105620] Updated weights for policy 1, policy_version 534098 (0.0010) [2023-12-26 19:15:00,153][105620] Updated weights for policy 1, policy_version 534108 (0.0010) [2023-12-26 19:15:00,211][105620] Updated weights for policy 1, policy_version 534118 (0.0010) [2023-12-26 19:15:00,569][105692] Updated weights for policy 0, policy_version 533287 (0.0007) [2023-12-26 19:15:00,629][105692] Updated weights for policy 0, policy_version 533297 (0.0005) [2023-12-26 19:15:00,678][105692] Updated weights for policy 0, policy_version 533307 (0.0006) [2023-12-26 19:15:00,791][105620] Updated weights for policy 1, policy_version 534128 (0.0008) [2023-12-26 19:15:00,859][105620] Updated weights for policy 1, policy_version 534138 (0.0010) [2023-12-26 19:15:00,922][105620] Updated weights for policy 1, policy_version 534148 (0.0008) [2023-12-26 19:15:01,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 273301504. Throughput: 0: 9611.7, 1: 9866.8. Samples: 273267976. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-26 19:15:01,062][104569] Avg episode reward: [(0, '8534.100'), (1, '9174.856')] [2023-12-26 19:15:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000533312_136544256.pth... [2023-12-26 19:15:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000534152_136757248.pth... [2023-12-26 19:15:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000532192_136257536.pth [2023-12-26 19:15:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000533000_136462336.pth [2023-12-26 19:15:01,346][105692] Updated weights for policy 0, policy_version 533318 (0.0010) [2023-12-26 19:15:01,417][105692] Updated weights for policy 0, policy_version 533328 (0.0009) [2023-12-26 19:15:01,472][105692] Updated weights for policy 0, policy_version 533338 (0.0009) [2023-12-26 19:15:01,544][105620] Updated weights for policy 1, policy_version 534158 (0.0009) [2023-12-26 19:15:01,609][105620] Updated weights for policy 1, policy_version 534168 (0.0007) [2023-12-26 19:15:01,684][105620] Updated weights for policy 1, policy_version 534178 (0.0006) [2023-12-26 19:15:02,289][105620] Updated weights for policy 1, policy_version 534188 (0.0008) [2023-12-26 19:15:02,316][105692] Updated weights for policy 0, policy_version 533348 (0.0009) [2023-12-26 19:15:02,347][105620] Updated weights for policy 1, policy_version 534198 (0.0007) [2023-12-26 19:15:02,378][105692] Updated weights for policy 0, policy_version 533358 (0.0008) [2023-12-26 19:15:02,411][105620] Updated weights for policy 1, policy_version 534208 (0.0006) [2023-12-26 19:15:02,432][105692] Updated weights for policy 0, policy_version 533368 (0.0006) [2023-12-26 19:15:03,149][105692] Updated weights for policy 0, policy_version 533378 (0.0006) [2023-12-26 19:15:03,150][105620] Updated weights for policy 1, policy_version 534218 (0.0007) [2023-12-26 19:15:03,198][105620] Updated weights for policy 1, policy_version 534228 (0.0006) [2023-12-26 19:15:03,206][105692] Updated weights for policy 0, policy_version 533388 (0.0007) [2023-12-26 19:15:03,248][105620] Updated weights for policy 1, policy_version 534238 (0.0009) [2023-12-26 19:15:03,254][105692] Updated weights for policy 0, policy_version 533398 (0.0007) [2023-12-26 19:15:03,274][105585] KL-divergence is very high: 111.1632 [2023-12-26 19:15:03,304][105620] Updated weights for policy 1, policy_version 534248 (0.0007) [2023-12-26 19:15:03,306][105692] Updated weights for policy 0, policy_version 533408 (0.0006) [2023-12-26 19:15:04,053][105620] Updated weights for policy 1, policy_version 534258 (0.0009) [2023-12-26 19:15:04,082][105692] Updated weights for policy 0, policy_version 533418 (0.0010) [2023-12-26 19:15:04,118][105620] Updated weights for policy 1, policy_version 534268 (0.0009) [2023-12-26 19:15:04,140][105692] Updated weights for policy 0, policy_version 533428 (0.0008) [2023-12-26 19:15:04,179][105620] Updated weights for policy 1, policy_version 534278 (0.0008) [2023-12-26 19:15:04,201][105692] Updated weights for policy 0, policy_version 533438 (0.0006) [2023-12-26 19:15:04,878][105692] Updated weights for policy 0, policy_version 533448 (0.0006) [2023-12-26 19:15:04,923][105692] Updated weights for policy 0, policy_version 533458 (0.0005) [2023-12-26 19:15:04,970][105620] Updated weights for policy 1, policy_version 534288 (0.0008) [2023-12-26 19:15:04,972][105692] Updated weights for policy 0, policy_version 533468 (0.0005) [2023-12-26 19:15:05,025][105620] Updated weights for policy 1, policy_version 534298 (0.0009) [2023-12-26 19:15:05,079][105620] Updated weights for policy 1, policy_version 534308 (0.0009) [2023-12-26 19:15:05,598][105692] Updated weights for policy 0, policy_version 533478 (0.0007) [2023-12-26 19:15:05,649][105692] Updated weights for policy 0, policy_version 533489 (0.0010) [2023-12-26 19:15:05,706][105692] Updated weights for policy 0, policy_version 533500 (0.0010) [2023-12-26 19:15:05,823][105620] Updated weights for policy 1, policy_version 534318 (0.0010) [2023-12-26 19:15:05,892][105620] Updated weights for policy 1, policy_version 534328 (0.0009) [2023-12-26 19:15:05,949][105620] Updated weights for policy 1, policy_version 534338 (0.0009) [2023-12-26 19:15:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 273399808. Throughput: 0: 9548.3, 1: 9822.7. Samples: 273384952. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-26 19:15:06,062][104569] Avg episode reward: [(0, '8902.383'), (1, '9085.051')] [2023-12-26 19:15:06,486][105692] Updated weights for policy 0, policy_version 533510 (0.0007) [2023-12-26 19:15:06,551][105692] Updated weights for policy 0, policy_version 533520 (0.0008) [2023-12-26 19:15:06,614][105692] Updated weights for policy 0, policy_version 533530 (0.0009) [2023-12-26 19:15:06,721][105620] Updated weights for policy 1, policy_version 534348 (0.0009) [2023-12-26 19:15:06,771][105620] Updated weights for policy 1, policy_version 534358 (0.0009) [2023-12-26 19:15:06,832][105620] Updated weights for policy 1, policy_version 534368 (0.0009) [2023-12-26 19:15:07,219][105692] Updated weights for policy 0, policy_version 533540 (0.0008) [2023-12-26 19:15:07,281][105692] Updated weights for policy 0, policy_version 533550 (0.0007) [2023-12-26 19:15:07,333][105692] Updated weights for policy 0, policy_version 533560 (0.0009) [2023-12-26 19:15:07,672][105620] Updated weights for policy 1, policy_version 534378 (0.0009) [2023-12-26 19:15:07,727][105620] Updated weights for policy 1, policy_version 534388 (0.0009) [2023-12-26 19:15:07,781][105620] Updated weights for policy 1, policy_version 534398 (0.0010) [2023-12-26 19:15:07,828][105620] Updated weights for policy 1, policy_version 534408 (0.0009) [2023-12-26 19:15:07,990][105692] Updated weights for policy 0, policy_version 533570 (0.0009) [2023-12-26 19:15:08,042][105692] Updated weights for policy 0, policy_version 533580 (0.0009) [2023-12-26 19:15:08,094][105692] Updated weights for policy 0, policy_version 533590 (0.0009) [2023-12-26 19:15:08,149][105692] Updated weights for policy 0, policy_version 533600 (0.0009) [2023-12-26 19:15:08,570][105620] Updated weights for policy 1, policy_version 534418 (0.0010) [2023-12-26 19:15:08,636][105620] Updated weights for policy 1, policy_version 534428 (0.0010) [2023-12-26 19:15:08,705][105620] Updated weights for policy 1, policy_version 534438 (0.0009) [2023-12-26 19:15:08,893][105692] Updated weights for policy 0, policy_version 533610 (0.0009) [2023-12-26 19:15:08,956][105692] Updated weights for policy 0, policy_version 533620 (0.0009) [2023-12-26 19:15:09,022][105692] Updated weights for policy 0, policy_version 533630 (0.0009) [2023-12-26 19:15:09,455][105620] Updated weights for policy 1, policy_version 534448 (0.0009) [2023-12-26 19:15:09,520][105620] Updated weights for policy 1, policy_version 534458 (0.0009) [2023-12-26 19:15:09,579][105620] Updated weights for policy 1, policy_version 534468 (0.0009) [2023-12-26 19:15:09,811][105692] Updated weights for policy 0, policy_version 533640 (0.0008) [2023-12-26 19:15:09,873][105692] Updated weights for policy 0, policy_version 533650 (0.0008) [2023-12-26 19:15:09,937][105692] Updated weights for policy 0, policy_version 533660 (0.0009) [2023-12-26 19:15:10,350][105620] Updated weights for policy 1, policy_version 534478 (0.0008) [2023-12-26 19:15:10,416][105620] Updated weights for policy 1, policy_version 534488 (0.0006) [2023-12-26 19:15:10,483][105620] Updated weights for policy 1, policy_version 534498 (0.0006) [2023-12-26 19:15:10,641][105692] Updated weights for policy 0, policy_version 533670 (0.0008) [2023-12-26 19:15:10,682][105585] KL-divergence is very high: 175.2806 [2023-12-26 19:15:10,689][105585] KL-divergence is very high: 139.3573 [2023-12-26 19:15:10,703][105692] Updated weights for policy 0, policy_version 533680 (0.0008) [2023-12-26 19:15:10,759][105692] Updated weights for policy 0, policy_version 533690 (0.0006) [2023-12-26 19:15:11,041][105620] Updated weights for policy 1, policy_version 534508 (0.0008) [2023-12-26 19:15:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 273489920. Throughput: 0: 9694.3, 1: 9732.4. Samples: 273501004. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-26 19:15:11,063][104569] Avg episode reward: [(0, '8715.038'), (1, '9083.615')] [2023-12-26 19:15:11,105][105620] Updated weights for policy 1, policy_version 534518 (0.0011) [2023-12-26 19:15:11,177][105620] Updated weights for policy 1, policy_version 534528 (0.0011) [2023-12-26 19:15:11,446][105692] Updated weights for policy 0, policy_version 533700 (0.0007) [2023-12-26 19:15:11,513][105692] Updated weights for policy 0, policy_version 533710 (0.0009) [2023-12-26 19:15:11,584][105692] Updated weights for policy 0, policy_version 533720 (0.0010) [2023-12-26 19:15:11,914][105620] Updated weights for policy 1, policy_version 534538 (0.0010) [2023-12-26 19:15:11,979][105620] Updated weights for policy 1, policy_version 534548 (0.0009) [2023-12-26 19:15:12,042][105620] Updated weights for policy 1, policy_version 534558 (0.0009) [2023-12-26 19:15:12,103][105620] Updated weights for policy 1, policy_version 534568 (0.0009) [2023-12-26 19:15:12,345][105692] Updated weights for policy 0, policy_version 533730 (0.0009) [2023-12-26 19:15:12,407][105692] Updated weights for policy 0, policy_version 533740 (0.0009) [2023-12-26 19:15:12,458][105692] Updated weights for policy 0, policy_version 533750 (0.0009) [2023-12-26 19:15:12,509][105692] Updated weights for policy 0, policy_version 533760 (0.0008) [2023-12-26 19:15:12,862][105620] Updated weights for policy 1, policy_version 534578 (0.0009) [2023-12-26 19:15:12,923][105620] Updated weights for policy 1, policy_version 534588 (0.0008) [2023-12-26 19:15:12,984][105620] Updated weights for policy 1, policy_version 534598 (0.0009) [2023-12-26 19:15:13,279][105692] Updated weights for policy 0, policy_version 533770 (0.0008) [2023-12-26 19:15:13,338][105692] Updated weights for policy 0, policy_version 533780 (0.0007) [2023-12-26 19:15:13,397][105692] Updated weights for policy 0, policy_version 533790 (0.0005) [2023-12-26 19:15:13,659][105620] Updated weights for policy 1, policy_version 534608 (0.0006) [2023-12-26 19:15:13,720][105620] Updated weights for policy 1, policy_version 534618 (0.0006) [2023-12-26 19:15:13,783][105620] Updated weights for policy 1, policy_version 534628 (0.0005) [2023-12-26 19:15:14,000][105692] Updated weights for policy 0, policy_version 533800 (0.0005) [2023-12-26 19:15:14,064][105692] Updated weights for policy 0, policy_version 533810 (0.0006) [2023-12-26 19:15:14,125][105692] Updated weights for policy 0, policy_version 533820 (0.0005) [2023-12-26 19:15:14,350][105620] Updated weights for policy 1, policy_version 534638 (0.0009) [2023-12-26 19:15:14,396][105620] Updated weights for policy 1, policy_version 534648 (0.0007) [2023-12-26 19:15:14,455][105620] Updated weights for policy 1, policy_version 534658 (0.0005) [2023-12-26 19:15:14,627][105692] Updated weights for policy 0, policy_version 533830 (0.0005) [2023-12-26 19:15:14,681][105692] Updated weights for policy 0, policy_version 533840 (0.0005) [2023-12-26 19:15:14,733][105692] Updated weights for policy 0, policy_version 533850 (0.0005) [2023-12-26 19:15:15,140][105620] Updated weights for policy 1, policy_version 534668 (0.0005) [2023-12-26 19:15:15,198][105620] Updated weights for policy 1, policy_version 534678 (0.0006) [2023-12-26 19:15:15,261][105620] Updated weights for policy 1, policy_version 534688 (0.0005) [2023-12-26 19:15:15,518][105692] Updated weights for policy 0, policy_version 533860 (0.0007) [2023-12-26 19:15:15,577][105692] Updated weights for policy 0, policy_version 533870 (0.0008) [2023-12-26 19:15:15,640][105692] Updated weights for policy 0, policy_version 533880 (0.0009) [2023-12-26 19:15:15,853][105620] Updated weights for policy 1, policy_version 534698 (0.0006) [2023-12-26 19:15:15,914][105620] Updated weights for policy 1, policy_version 534708 (0.0009) [2023-12-26 19:15:15,972][105620] Updated weights for policy 1, policy_version 534718 (0.0009) [2023-12-26 19:15:16,019][105620] Updated weights for policy 1, policy_version 534728 (0.0009) [2023-12-26 19:15:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 273596416. Throughput: 0: 9681.2, 1: 9693.0. Samples: 273558600. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-26 19:15:16,063][104569] Avg episode reward: [(0, '8621.685'), (1, '9173.334')] [2023-12-26 19:15:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000533888_136691712.pth... [2023-12-26 19:15:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000534728_136904704.pth... [2023-12-26 19:15:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000532768_136404992.pth [2023-12-26 19:15:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000533576_136609792.pth [2023-12-26 19:15:16,383][105692] Updated weights for policy 0, policy_version 533890 (0.0009) [2023-12-26 19:15:16,448][105692] Updated weights for policy 0, policy_version 533900 (0.0006) [2023-12-26 19:15:16,511][105692] Updated weights for policy 0, policy_version 533910 (0.0006) [2023-12-26 19:15:16,579][105692] Updated weights for policy 0, policy_version 533920 (0.0005) [2023-12-26 19:15:16,827][105620] Updated weights for policy 1, policy_version 534738 (0.0005) [2023-12-26 19:15:16,879][105620] Updated weights for policy 1, policy_version 534748 (0.0006) [2023-12-26 19:15:16,931][105620] Updated weights for policy 1, policy_version 534758 (0.0008) [2023-12-26 19:15:17,063][105692] Updated weights for policy 0, policy_version 533930 (0.0005) [2023-12-26 19:15:17,126][105692] Updated weights for policy 0, policy_version 533940 (0.0005) [2023-12-26 19:15:17,181][105692] Updated weights for policy 0, policy_version 533950 (0.0005) [2023-12-26 19:15:17,657][105620] Updated weights for policy 1, policy_version 534768 (0.0006) [2023-12-26 19:15:17,705][105620] Updated weights for policy 1, policy_version 534778 (0.0010) [2023-12-26 19:15:17,765][105692] Updated weights for policy 0, policy_version 533960 (0.0005) [2023-12-26 19:15:17,767][105620] Updated weights for policy 1, policy_version 534788 (0.0010) [2023-12-26 19:15:17,825][105692] Updated weights for policy 0, policy_version 533970 (0.0009) [2023-12-26 19:15:17,876][105692] Updated weights for policy 0, policy_version 533980 (0.0010) [2023-12-26 19:15:18,499][105620] Updated weights for policy 1, policy_version 534798 (0.0007) [2023-12-26 19:15:18,558][105620] Updated weights for policy 1, policy_version 534808 (0.0008) [2023-12-26 19:15:18,577][105692] Updated weights for policy 0, policy_version 533990 (0.0010) [2023-12-26 19:15:18,615][105620] Updated weights for policy 1, policy_version 534818 (0.0007) [2023-12-26 19:15:18,641][105692] Updated weights for policy 0, policy_version 534000 (0.0010) [2023-12-26 19:15:18,701][105692] Updated weights for policy 0, policy_version 534010 (0.0011) [2023-12-26 19:15:19,328][105620] Updated weights for policy 1, policy_version 534828 (0.0006) [2023-12-26 19:15:19,390][105620] Updated weights for policy 1, policy_version 534838 (0.0008) [2023-12-26 19:15:19,444][105692] Updated weights for policy 0, policy_version 534020 (0.0011) [2023-12-26 19:15:19,451][105620] Updated weights for policy 1, policy_version 534848 (0.0005) [2023-12-26 19:15:19,507][105692] Updated weights for policy 0, policy_version 534030 (0.0011) [2023-12-26 19:15:19,574][105692] Updated weights for policy 0, policy_version 534040 (0.0011) [2023-12-26 19:15:20,066][105620] Updated weights for policy 1, policy_version 534858 (0.0008) [2023-12-26 19:15:20,131][105620] Updated weights for policy 1, policy_version 534868 (0.0010) [2023-12-26 19:15:20,202][105620] Updated weights for policy 1, policy_version 534878 (0.0009) [2023-12-26 19:15:20,266][105620] Updated weights for policy 1, policy_version 534888 (0.0006) [2023-12-26 19:15:20,279][105692] Updated weights for policy 0, policy_version 534050 (0.0011) [2023-12-26 19:15:20,336][105692] Updated weights for policy 0, policy_version 534060 (0.0011) [2023-12-26 19:15:20,391][105692] Updated weights for policy 0, policy_version 534070 (0.0011) [2023-12-26 19:15:20,450][105692] Updated weights for policy 0, policy_version 534080 (0.0011) [2023-12-26 19:15:20,996][105620] Updated weights for policy 1, policy_version 534898 (0.0008) [2023-12-26 19:15:21,055][105620] Updated weights for policy 1, policy_version 534908 (0.0008) [2023-12-26 19:15:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 273686528. Throughput: 0: 9774.4, 1: 9755.6. Samples: 273682380. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-26 19:15:21,063][104569] Avg episode reward: [(0, '8714.425'), (1, '9264.269')] [2023-12-26 19:15:21,115][105620] Updated weights for policy 1, policy_version 534918 (0.0006) [2023-12-26 19:15:21,231][105692] Updated weights for policy 0, policy_version 534090 (0.0006) [2023-12-26 19:15:21,297][105692] Updated weights for policy 0, policy_version 534100 (0.0010) [2023-12-26 19:15:21,362][105692] Updated weights for policy 0, policy_version 534110 (0.0010) [2023-12-26 19:15:21,919][105620] Updated weights for policy 1, policy_version 534928 (0.0008) [2023-12-26 19:15:21,977][105620] Updated weights for policy 1, policy_version 534938 (0.0006) [2023-12-26 19:15:22,039][105620] Updated weights for policy 1, policy_version 534948 (0.0006) [2023-12-26 19:15:22,098][105692] Updated weights for policy 0, policy_version 534120 (0.0011) [2023-12-26 19:15:22,162][105692] Updated weights for policy 0, policy_version 534130 (0.0011) [2023-12-26 19:15:22,222][105692] Updated weights for policy 0, policy_version 534140 (0.0011) [2023-12-26 19:15:22,759][105620] Updated weights for policy 1, policy_version 534958 (0.0010) [2023-12-26 19:15:22,823][105620] Updated weights for policy 1, policy_version 534968 (0.0011) [2023-12-26 19:15:22,886][105620] Updated weights for policy 1, policy_version 534978 (0.0011) [2023-12-26 19:15:22,978][105692] Updated weights for policy 0, policy_version 534150 (0.0012) [2023-12-26 19:15:23,015][105585] KL-divergence is very high: 113.7962 [2023-12-26 19:15:23,037][105692] Updated weights for policy 0, policy_version 534160 (0.0010) [2023-12-26 19:15:23,062][105585] KL-divergence is very high: 169.7019 [2023-12-26 19:15:23,081][105585] KL-divergence is very high: 120.1939 [2023-12-26 19:15:23,099][105692] Updated weights for policy 0, policy_version 534170 (0.0010) [2023-12-26 19:15:23,108][105585] KL-divergence is very high: 179.0291 [2023-12-26 19:15:23,126][105585] KL-divergence is very high: 118.5947 [2023-12-26 19:15:23,582][105620] Updated weights for policy 1, policy_version 534988 (0.0010) [2023-12-26 19:15:23,640][105620] Updated weights for policy 1, policy_version 534998 (0.0010) [2023-12-26 19:15:23,652][105585] KL-divergence is very high: 498.4723 [2023-12-26 19:15:23,675][105692] Updated weights for policy 0, policy_version 534180 (0.0010) [2023-12-26 19:15:23,687][105620] Updated weights for policy 1, policy_version 535008 (0.0010) [2023-12-26 19:15:23,698][105585] KL-divergence is very high: 396.3979 [2023-12-26 19:15:23,738][105692] Updated weights for policy 0, policy_version 534190 (0.0008) [2023-12-26 19:15:23,746][105585] KL-divergence is very high: 297.6867 [2023-12-26 19:15:23,786][105585] KL-divergence is very high: 217.0434 [2023-12-26 19:15:23,787][105692] Updated weights for policy 0, policy_version 534200 (0.0008) [2023-12-26 19:15:24,438][105620] Updated weights for policy 1, policy_version 535018 (0.0010) [2023-12-26 19:15:24,489][105620] Updated weights for policy 1, policy_version 535028 (0.0010) [2023-12-26 19:15:24,522][105692] Updated weights for policy 0, policy_version 534210 (0.0007) [2023-12-26 19:15:24,544][105620] Updated weights for policy 1, policy_version 535038 (0.0010) [2023-12-26 19:15:24,571][105692] Updated weights for policy 0, policy_version 534220 (0.0007) [2023-12-26 19:15:24,608][105620] Updated weights for policy 1, policy_version 535048 (0.0009) [2023-12-26 19:15:24,636][105692] Updated weights for policy 0, policy_version 534230 (0.0005) [2023-12-26 19:15:24,704][105692] Updated weights for policy 0, policy_version 534240 (0.0005) [2023-12-26 19:15:25,280][105620] Updated weights for policy 1, policy_version 535058 (0.0011) [2023-12-26 19:15:25,335][105620] Updated weights for policy 1, policy_version 535068 (0.0010) [2023-12-26 19:15:25,385][105620] Updated weights for policy 1, policy_version 535078 (0.0010) [2023-12-26 19:15:25,391][105692] Updated weights for policy 0, policy_version 534250 (0.0011) [2023-12-26 19:15:25,450][105692] Updated weights for policy 0, policy_version 534260 (0.0011) [2023-12-26 19:15:25,504][105692] Updated weights for policy 0, policy_version 534270 (0.0011) [2023-12-26 19:15:26,017][105620] Updated weights for policy 1, policy_version 535088 (0.0006) [2023-12-26 19:15:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 273784832. Throughput: 0: 9785.7, 1: 9762.8. Samples: 273799228. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-26 19:15:26,062][104569] Avg episode reward: [(0, '8807.035'), (1, '9263.948')] [2023-12-26 19:15:26,068][105620] Updated weights for policy 1, policy_version 535098 (0.0006) [2023-12-26 19:15:26,122][105620] Updated weights for policy 1, policy_version 535108 (0.0010) [2023-12-26 19:15:26,258][105692] Updated weights for policy 0, policy_version 534280 (0.0010) [2023-12-26 19:15:26,307][105692] Updated weights for policy 0, policy_version 534290 (0.0010) [2023-12-26 19:15:26,352][105692] Updated weights for policy 0, policy_version 534300 (0.0010) [2023-12-26 19:15:26,732][105620] Updated weights for policy 1, policy_version 535118 (0.0008) [2023-12-26 19:15:26,797][105620] Updated weights for policy 1, policy_version 535128 (0.0005) [2023-12-26 19:15:26,848][105620] Updated weights for policy 1, policy_version 535138 (0.0005) [2023-12-26 19:15:26,958][105692] Updated weights for policy 0, policy_version 534310 (0.0007) [2023-12-26 19:15:27,022][105692] Updated weights for policy 0, policy_version 534320 (0.0005) [2023-12-26 19:15:27,074][105692] Updated weights for policy 0, policy_version 534330 (0.0005) [2023-12-26 19:15:27,348][105620] Updated weights for policy 1, policy_version 535148 (0.0005) [2023-12-26 19:15:27,402][105620] Updated weights for policy 1, policy_version 535158 (0.0005) [2023-12-26 19:15:27,450][105620] Updated weights for policy 1, policy_version 535168 (0.0005) [2023-12-26 19:15:27,721][105692] Updated weights for policy 0, policy_version 534340 (0.0007) [2023-12-26 19:15:27,778][105692] Updated weights for policy 0, policy_version 534350 (0.0010) [2023-12-26 19:15:27,829][105692] Updated weights for policy 0, policy_version 534360 (0.0010) [2023-12-26 19:15:28,002][105620] Updated weights for policy 1, policy_version 535178 (0.0005) [2023-12-26 19:15:28,061][105620] Updated weights for policy 1, policy_version 535188 (0.0005) [2023-12-26 19:15:28,121][105620] Updated weights for policy 1, policy_version 535198 (0.0005) [2023-12-26 19:15:28,171][105620] Updated weights for policy 1, policy_version 535208 (0.0005) [2023-12-26 19:15:28,561][105692] Updated weights for policy 0, policy_version 534370 (0.0010) [2023-12-26 19:15:28,609][105692] Updated weights for policy 0, policy_version 534380 (0.0010) [2023-12-26 19:15:28,656][105692] Updated weights for policy 0, policy_version 534390 (0.0010) [2023-12-26 19:15:28,690][105620] Updated weights for policy 1, policy_version 535218 (0.0005) [2023-12-26 19:15:28,708][105692] Updated weights for policy 0, policy_version 534400 (0.0010) [2023-12-26 19:15:28,751][105620] Updated weights for policy 1, policy_version 535228 (0.0007) [2023-12-26 19:15:28,810][105620] Updated weights for policy 1, policy_version 535238 (0.0008) [2023-12-26 19:15:29,444][105692] Updated weights for policy 0, policy_version 534410 (0.0006) [2023-12-26 19:15:29,506][105692] Updated weights for policy 0, policy_version 534420 (0.0009) [2023-12-26 19:15:29,548][105620] Updated weights for policy 1, policy_version 535248 (0.0007) [2023-12-26 19:15:29,568][105692] Updated weights for policy 0, policy_version 534430 (0.0009) [2023-12-26 19:15:29,613][105620] Updated weights for policy 1, policy_version 535258 (0.0010) [2023-12-26 19:15:29,675][105620] Updated weights for policy 1, policy_version 535268 (0.0008) [2023-12-26 19:15:30,214][105692] Updated weights for policy 0, policy_version 534440 (0.0007) [2023-12-26 19:15:30,251][105585] KL-divergence is very high: 111.6894 [2023-12-26 19:15:30,274][105585] KL-divergence is very high: 514.9105 [2023-12-26 19:15:30,280][105692] Updated weights for policy 0, policy_version 534450 (0.0010) [2023-12-26 19:15:30,289][105585] KL-divergence is very high: 346.7533 [2023-12-26 19:15:30,308][105585] KL-divergence is very high: 228.7842 [2023-12-26 19:15:30,326][105585] KL-divergence is very high: 591.7504 [2023-12-26 19:15:30,335][105585] KL-divergence is very high: 324.5127 [2023-12-26 19:15:30,341][105692] Updated weights for policy 0, policy_version 534460 (0.0011) [2023-12-26 19:15:30,350][105585] KL-divergence is very high: 164.4047 [2023-12-26 19:15:30,399][105620] Updated weights for policy 1, policy_version 535278 (0.0006) [2023-12-26 19:15:30,446][105620] Updated weights for policy 1, policy_version 535288 (0.0005) [2023-12-26 19:15:30,510][105620] Updated weights for policy 1, policy_version 535298 (0.0006) [2023-12-26 19:15:30,976][105692] Updated weights for policy 0, policy_version 534470 (0.0008) [2023-12-26 19:15:31,028][105692] Updated weights for policy 0, policy_version 534480 (0.0006) [2023-12-26 19:15:31,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 273891328. Throughput: 0: 9828.5, 1: 9902.0. Samples: 273866304. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-26 19:15:31,063][104569] Avg episode reward: [(0, '8893.724'), (1, '9174.388')] [2023-12-26 19:15:31,066][105620] Updated weights for policy 1, policy_version 535308 (0.0006) [2023-12-26 19:15:31,080][105692] Updated weights for policy 0, policy_version 534490 (0.0007) [2023-12-26 19:15:31,112][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000534496_136847360.pth... [2023-12-26 19:15:31,117][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000533312_136544256.pth [2023-12-26 19:15:31,123][105620] Updated weights for policy 1, policy_version 535318 (0.0008) [2023-12-26 19:15:31,185][105620] Updated weights for policy 1, policy_version 535328 (0.0008) [2023-12-26 19:15:31,229][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000535336_137060352.pth... [2023-12-26 19:15:31,232][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000534152_136757248.pth [2023-12-26 19:15:31,675][105692] Updated weights for policy 0, policy_version 534500 (0.0006) [2023-12-26 19:15:31,745][105692] Updated weights for policy 0, policy_version 534510 (0.0007) [2023-12-26 19:15:31,807][105692] Updated weights for policy 0, policy_version 534520 (0.0008) [2023-12-26 19:15:31,970][105620] Updated weights for policy 1, policy_version 535338 (0.0008) [2023-12-26 19:15:32,022][105620] Updated weights for policy 1, policy_version 535348 (0.0009) [2023-12-26 19:15:32,082][105620] Updated weights for policy 1, policy_version 535358 (0.0006) [2023-12-26 19:15:32,149][105620] Updated weights for policy 1, policy_version 535368 (0.0005) [2023-12-26 19:15:32,486][105692] Updated weights for policy 0, policy_version 534530 (0.0009) [2023-12-26 19:15:32,544][105692] Updated weights for policy 0, policy_version 534540 (0.0010) [2023-12-26 19:15:32,591][105692] Updated weights for policy 0, policy_version 534550 (0.0008) [2023-12-26 19:15:32,638][105692] Updated weights for policy 0, policy_version 534560 (0.0009) [2023-12-26 19:15:32,769][105620] Updated weights for policy 1, policy_version 535378 (0.0009) [2023-12-26 19:15:32,820][105620] Updated weights for policy 1, policy_version 535388 (0.0008) [2023-12-26 19:15:32,874][105620] Updated weights for policy 1, policy_version 535398 (0.0005) [2023-12-26 19:15:33,378][105692] Updated weights for policy 0, policy_version 534570 (0.0005) [2023-12-26 19:15:33,425][105692] Updated weights for policy 0, policy_version 534580 (0.0005) [2023-12-26 19:15:33,468][105692] Updated weights for policy 0, policy_version 534590 (0.0005) [2023-12-26 19:15:33,579][105620] Updated weights for policy 1, policy_version 535408 (0.0009) [2023-12-26 19:15:33,647][105620] Updated weights for policy 1, policy_version 535418 (0.0010) [2023-12-26 19:15:33,711][105620] Updated weights for policy 1, policy_version 535428 (0.0010) [2023-12-26 19:15:34,067][105692] Updated weights for policy 0, policy_version 534600 (0.0009) [2023-12-26 19:15:34,130][105692] Updated weights for policy 0, policy_version 534610 (0.0011) [2023-12-26 19:15:34,197][105692] Updated weights for policy 0, policy_version 534620 (0.0011) [2023-12-26 19:15:34,323][105620] Updated weights for policy 1, policy_version 535438 (0.0008) [2023-12-26 19:15:34,378][105620] Updated weights for policy 1, policy_version 535448 (0.0006) [2023-12-26 19:15:34,442][105620] Updated weights for policy 1, policy_version 535458 (0.0008) [2023-12-26 19:15:34,992][105692] Updated weights for policy 0, policy_version 534630 (0.0009) [2023-12-26 19:15:35,045][105692] Updated weights for policy 0, policy_version 534640 (0.0009) [2023-12-26 19:15:35,072][105620] Updated weights for policy 1, policy_version 535468 (0.0009) [2023-12-26 19:15:35,091][105692] Updated weights for policy 0, policy_version 534650 (0.0008) [2023-12-26 19:15:35,129][105620] Updated weights for policy 1, policy_version 535478 (0.0006) [2023-12-26 19:15:35,183][105620] Updated weights for policy 1, policy_version 535488 (0.0009) [2023-12-26 19:15:35,715][105692] Updated weights for policy 0, policy_version 534660 (0.0006) [2023-12-26 19:15:35,790][105692] Updated weights for policy 0, policy_version 534670 (0.0005) [2023-12-26 19:15:35,852][105692] Updated weights for policy 0, policy_version 534680 (0.0005) [2023-12-26 19:15:36,044][105620] Updated weights for policy 1, policy_version 535498 (0.0009) [2023-12-26 19:15:36,062][104569] Fps is (10 sec: 21299.1, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 273997824. Throughput: 0: 9907.8, 1: 9956.6. Samples: 273989188. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-26 19:15:36,063][104569] Avg episode reward: [(0, '8708.926'), (1, '9086.793')] [2023-12-26 19:15:36,103][105620] Updated weights for policy 1, policy_version 535508 (0.0009) [2023-12-26 19:15:36,162][105620] Updated weights for policy 1, policy_version 535518 (0.0007) [2023-12-26 19:15:36,371][105692] Updated weights for policy 0, policy_version 534690 (0.0006) [2023-12-26 19:15:36,434][105692] Updated weights for policy 0, policy_version 534700 (0.0009) [2023-12-26 19:15:36,489][105692] Updated weights for policy 0, policy_version 534710 (0.0009) [2023-12-26 19:15:36,537][105692] Updated weights for policy 0, policy_version 534720 (0.0009) [2023-12-26 19:15:36,978][105620] Updated weights for policy 1, policy_version 535530 (0.0010) [2023-12-26 19:15:37,035][105620] Updated weights for policy 1, policy_version 535540 (0.0009) [2023-12-26 19:15:37,089][105620] Updated weights for policy 1, policy_version 535550 (0.0008) [2023-12-26 19:15:37,151][105620] Updated weights for policy 1, policy_version 535560 (0.0009) [2023-12-26 19:15:37,259][105692] Updated weights for policy 0, policy_version 534730 (0.0005) [2023-12-26 19:15:37,321][105692] Updated weights for policy 0, policy_version 534740 (0.0009) [2023-12-26 19:15:37,375][105692] Updated weights for policy 0, policy_version 534750 (0.0008) [2023-12-26 19:15:37,949][105620] Updated weights for policy 1, policy_version 535570 (0.0009) [2023-12-26 19:15:38,011][105620] Updated weights for policy 1, policy_version 535580 (0.0009) [2023-12-26 19:15:38,071][105620] Updated weights for policy 1, policy_version 535590 (0.0009) [2023-12-26 19:15:38,097][105692] Updated weights for policy 0, policy_version 534760 (0.0010) [2023-12-26 19:15:38,144][105692] Updated weights for policy 0, policy_version 534770 (0.0009) [2023-12-26 19:15:38,190][105692] Updated weights for policy 0, policy_version 534780 (0.0008) [2023-12-26 19:15:38,821][105620] Updated weights for policy 1, policy_version 535600 (0.0009) [2023-12-26 19:15:38,879][105620] Updated weights for policy 1, policy_version 535610 (0.0008) [2023-12-26 19:15:38,927][105620] Updated weights for policy 1, policy_version 535620 (0.0009) [2023-12-26 19:15:38,981][105692] Updated weights for policy 0, policy_version 534790 (0.0008) [2023-12-26 19:15:39,042][105692] Updated weights for policy 0, policy_version 534800 (0.0009) [2023-12-26 19:15:39,104][105692] Updated weights for policy 0, policy_version 534810 (0.0009) [2023-12-26 19:15:39,655][105620] Updated weights for policy 1, policy_version 535630 (0.0007) [2023-12-26 19:15:39,716][105620] Updated weights for policy 1, policy_version 535640 (0.0009) [2023-12-26 19:15:39,781][105620] Updated weights for policy 1, policy_version 535650 (0.0009) [2023-12-26 19:15:39,905][105692] Updated weights for policy 0, policy_version 534820 (0.0009) [2023-12-26 19:15:39,973][105692] Updated weights for policy 0, policy_version 534830 (0.0008) [2023-12-26 19:15:40,037][105692] Updated weights for policy 0, policy_version 534840 (0.0008) [2023-12-26 19:15:40,554][105620] Updated weights for policy 1, policy_version 535660 (0.0010) [2023-12-26 19:15:40,607][105620] Updated weights for policy 1, policy_version 535670 (0.0011) [2023-12-26 19:15:40,656][105620] Updated weights for policy 1, policy_version 535680 (0.0010) [2023-12-26 19:15:40,823][105692] Updated weights for policy 0, policy_version 534850 (0.0009) [2023-12-26 19:15:40,884][105692] Updated weights for policy 0, policy_version 534860 (0.0009) [2023-12-26 19:15:40,942][105692] Updated weights for policy 0, policy_version 534870 (0.0008) [2023-12-26 19:15:41,006][105692] Updated weights for policy 0, policy_version 534880 (0.0009) [2023-12-26 19:15:41,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 274096128. Throughput: 0: 9934.6, 1: 9860.5. Samples: 274101648. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-26 19:15:41,063][104569] Avg episode reward: [(0, '8711.162'), (1, '9086.293')] [2023-12-26 19:15:41,436][105620] Updated weights for policy 1, policy_version 535690 (0.0009) [2023-12-26 19:15:41,498][105620] Updated weights for policy 1, policy_version 535700 (0.0008) [2023-12-26 19:15:41,565][105620] Updated weights for policy 1, policy_version 535710 (0.0008) [2023-12-26 19:15:41,631][105620] Updated weights for policy 1, policy_version 535720 (0.0009) [2023-12-26 19:15:41,799][105692] Updated weights for policy 0, policy_version 534890 (0.0008) [2023-12-26 19:15:41,873][105692] Updated weights for policy 0, policy_version 534900 (0.0008) [2023-12-26 19:15:41,934][105692] Updated weights for policy 0, policy_version 534910 (0.0010) [2023-12-26 19:15:42,303][105620] Updated weights for policy 1, policy_version 535730 (0.0009) [2023-12-26 19:15:42,369][105620] Updated weights for policy 1, policy_version 535740 (0.0007) [2023-12-26 19:15:42,435][105620] Updated weights for policy 1, policy_version 535750 (0.0008) [2023-12-26 19:15:42,691][105692] Updated weights for policy 0, policy_version 534920 (0.0006) [2023-12-26 19:15:42,747][105692] Updated weights for policy 0, policy_version 534930 (0.0006) [2023-12-26 19:15:42,805][105692] Updated weights for policy 0, policy_version 534940 (0.0005) [2023-12-26 19:15:43,257][105620] Updated weights for policy 1, policy_version 535760 (0.0009) [2023-12-26 19:15:43,304][105620] Updated weights for policy 1, policy_version 535770 (0.0009) [2023-12-26 19:15:43,360][105620] Updated weights for policy 1, policy_version 535780 (0.0010) [2023-12-26 19:15:43,407][105692] Updated weights for policy 0, policy_version 534950 (0.0005) [2023-12-26 19:15:43,470][105692] Updated weights for policy 0, policy_version 534960 (0.0011) [2023-12-26 19:15:43,528][105692] Updated weights for policy 0, policy_version 534970 (0.0010) [2023-12-26 19:15:44,172][105620] Updated weights for policy 1, policy_version 535790 (0.0009) [2023-12-26 19:15:44,190][105692] Updated weights for policy 0, policy_version 534980 (0.0010) [2023-12-26 19:15:44,236][105620] Updated weights for policy 1, policy_version 535800 (0.0006) [2023-12-26 19:15:44,252][105692] Updated weights for policy 0, policy_version 534990 (0.0010) [2023-12-26 19:15:44,295][105620] Updated weights for policy 1, policy_version 535810 (0.0008) [2023-12-26 19:15:44,310][105692] Updated weights for policy 0, policy_version 535000 (0.0007) [2023-12-26 19:15:44,958][105692] Updated weights for policy 0, policy_version 535010 (0.0006) [2023-12-26 19:15:45,024][105692] Updated weights for policy 0, policy_version 535020 (0.0006) [2023-12-26 19:15:45,041][105620] Updated weights for policy 1, policy_version 535820 (0.0007) [2023-12-26 19:15:45,084][105692] Updated weights for policy 0, policy_version 535030 (0.0007) [2023-12-26 19:15:45,109][105620] Updated weights for policy 1, policy_version 535830 (0.0009) [2023-12-26 19:15:45,148][105692] Updated weights for policy 0, policy_version 535040 (0.0009) [2023-12-26 19:15:45,165][105620] Updated weights for policy 1, policy_version 535840 (0.0006) [2023-12-26 19:15:45,831][105620] Updated weights for policy 1, policy_version 535850 (0.0006) [2023-12-26 19:15:45,894][105620] Updated weights for policy 1, policy_version 535860 (0.0006) [2023-12-26 19:15:45,928][105692] Updated weights for policy 0, policy_version 535050 (0.0008) [2023-12-26 19:15:45,947][105620] Updated weights for policy 1, policy_version 535870 (0.0006) [2023-12-26 19:15:45,980][105692] Updated weights for policy 0, policy_version 535060 (0.0009) [2023-12-26 19:15:45,996][105620] Updated weights for policy 1, policy_version 535880 (0.0008) [2023-12-26 19:15:46,032][105692] Updated weights for policy 0, policy_version 535070 (0.0008) [2023-12-26 19:15:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 274194432. Throughput: 0: 9941.0, 1: 9830.0. Samples: 274157668. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-26 19:15:46,062][104569] Avg episode reward: [(0, '4010.934'), (1, '9083.987')] [2023-12-26 19:15:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000535072_136994816.pth... [2023-12-26 19:15:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000535880_137199616.pth... [2023-12-26 19:15:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000533888_136691712.pth [2023-12-26 19:15:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000534728_136904704.pth [2023-12-26 19:15:46,679][105620] Updated weights for policy 1, policy_version 535890 (0.0006) [2023-12-26 19:15:46,749][105620] Updated weights for policy 1, policy_version 535900 (0.0006) [2023-12-26 19:15:46,809][105692] Updated weights for policy 0, policy_version 535080 (0.0010) [2023-12-26 19:15:46,814][105620] Updated weights for policy 1, policy_version 535910 (0.0005) [2023-12-26 19:15:46,865][105692] Updated weights for policy 0, policy_version 535090 (0.0011) [2023-12-26 19:15:46,927][105692] Updated weights for policy 0, policy_version 535100 (0.0011) [2023-12-26 19:15:47,362][105620] Updated weights for policy 1, policy_version 535920 (0.0007) [2023-12-26 19:15:47,420][105620] Updated weights for policy 1, policy_version 535930 (0.0007) [2023-12-26 19:15:47,476][105620] Updated weights for policy 1, policy_version 535940 (0.0010) [2023-12-26 19:15:47,627][105692] Updated weights for policy 0, policy_version 535110 (0.0007) [2023-12-26 19:15:47,685][105692] Updated weights for policy 0, policy_version 535120 (0.0010) [2023-12-26 19:15:47,744][105692] Updated weights for policy 0, policy_version 535130 (0.0010) [2023-12-26 19:15:48,150][105620] Updated weights for policy 1, policy_version 535950 (0.0009) [2023-12-26 19:15:48,199][105620] Updated weights for policy 1, policy_version 535960 (0.0011) [2023-12-26 19:15:48,248][105620] Updated weights for policy 1, policy_version 535970 (0.0010) [2023-12-26 19:15:48,422][105692] Updated weights for policy 0, policy_version 535140 (0.0007) [2023-12-26 19:15:48,480][105692] Updated weights for policy 0, policy_version 535150 (0.0008) [2023-12-26 19:15:48,543][105692] Updated weights for policy 0, policy_version 535160 (0.0008) [2023-12-26 19:15:49,019][105620] Updated weights for policy 1, policy_version 535980 (0.0008) [2023-12-26 19:15:49,088][105620] Updated weights for policy 1, policy_version 535990 (0.0007) [2023-12-26 19:15:49,152][105620] Updated weights for policy 1, policy_version 536000 (0.0008) [2023-12-26 19:15:49,309][105692] Updated weights for policy 0, policy_version 535170 (0.0009) [2023-12-26 19:15:49,384][105692] Updated weights for policy 0, policy_version 535180 (0.0009) [2023-12-26 19:15:49,447][105692] Updated weights for policy 0, policy_version 535190 (0.0008) [2023-12-26 19:15:49,497][105692] Updated weights for policy 0, policy_version 535200 (0.0008) [2023-12-26 19:15:49,818][105620] Updated weights for policy 1, policy_version 536010 (0.0008) [2023-12-26 19:15:49,887][105620] Updated weights for policy 1, policy_version 536020 (0.0007) [2023-12-26 19:15:49,960][105620] Updated weights for policy 1, policy_version 536030 (0.0008) [2023-12-26 19:15:50,024][105620] Updated weights for policy 1, policy_version 536040 (0.0010) [2023-12-26 19:15:50,219][105692] Updated weights for policy 0, policy_version 535210 (0.0009) [2023-12-26 19:15:50,284][105692] Updated weights for policy 0, policy_version 535220 (0.0009) [2023-12-26 19:15:50,348][105692] Updated weights for policy 0, policy_version 535230 (0.0008) [2023-12-26 19:15:50,819][105620] Updated weights for policy 1, policy_version 536050 (0.0010) [2023-12-26 19:15:50,885][105620] Updated weights for policy 1, policy_version 536060 (0.0009) [2023-12-26 19:15:50,956][105620] Updated weights for policy 1, policy_version 536070 (0.0008) [2023-12-26 19:15:51,037][105692] Updated weights for policy 0, policy_version 535240 (0.0008) [2023-12-26 19:15:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 274284544. Throughput: 0: 9949.9, 1: 9852.6. Samples: 274276064. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-26 19:15:51,063][104569] Avg episode reward: [(0, '5916.781'), (1, '9171.191')] [2023-12-26 19:15:51,103][105692] Updated weights for policy 0, policy_version 535250 (0.0009) [2023-12-26 19:15:51,169][105692] Updated weights for policy 0, policy_version 535260 (0.0010) [2023-12-26 19:15:51,763][105620] Updated weights for policy 1, policy_version 536080 (0.0006) [2023-12-26 19:15:51,829][105620] Updated weights for policy 1, policy_version 536090 (0.0006) [2023-12-26 19:15:51,889][105620] Updated weights for policy 1, policy_version 536100 (0.0007) [2023-12-26 19:15:51,988][105692] Updated weights for policy 0, policy_version 535270 (0.0008) [2023-12-26 19:15:52,059][105692] Updated weights for policy 0, policy_version 535280 (0.0008) [2023-12-26 19:15:52,131][105692] Updated weights for policy 0, policy_version 535290 (0.0011) [2023-12-26 19:15:52,536][105620] Updated weights for policy 1, policy_version 536110 (0.0008) [2023-12-26 19:15:52,604][105620] Updated weights for policy 1, policy_version 536120 (0.0009) [2023-12-26 19:15:52,661][105620] Updated weights for policy 1, policy_version 536130 (0.0009) [2023-12-26 19:15:52,749][105692] Updated weights for policy 0, policy_version 535300 (0.0009) [2023-12-26 19:15:52,815][105692] Updated weights for policy 0, policy_version 535310 (0.0010) [2023-12-26 19:15:52,877][105692] Updated weights for policy 0, policy_version 535320 (0.0009) [2023-12-26 19:15:53,445][105620] Updated weights for policy 1, policy_version 536140 (0.0008) [2023-12-26 19:15:53,501][105620] Updated weights for policy 1, policy_version 536150 (0.0009) [2023-12-26 19:15:53,564][105620] Updated weights for policy 1, policy_version 536160 (0.0009) [2023-12-26 19:15:53,624][105692] Updated weights for policy 0, policy_version 535330 (0.0010) [2023-12-26 19:15:53,678][105692] Updated weights for policy 0, policy_version 535340 (0.0009) [2023-12-26 19:15:53,733][105692] Updated weights for policy 0, policy_version 535350 (0.0009) [2023-12-26 19:15:53,781][105692] Updated weights for policy 0, policy_version 535360 (0.0009) [2023-12-26 19:15:54,265][105620] Updated weights for policy 1, policy_version 536170 (0.0010) [2023-12-26 19:15:54,328][105620] Updated weights for policy 1, policy_version 536180 (0.0010) [2023-12-26 19:15:54,384][105620] Updated weights for policy 1, policy_version 536190 (0.0010) [2023-12-26 19:15:54,438][105620] Updated weights for policy 1, policy_version 536200 (0.0009) [2023-12-26 19:15:54,542][105692] Updated weights for policy 0, policy_version 535370 (0.0010) [2023-12-26 19:15:54,608][105692] Updated weights for policy 0, policy_version 535380 (0.0010) [2023-12-26 19:15:54,662][105692] Updated weights for policy 0, policy_version 535390 (0.0010) [2023-12-26 19:15:55,067][105620] Updated weights for policy 1, policy_version 536210 (0.0011) [2023-12-26 19:15:55,124][105620] Updated weights for policy 1, policy_version 536220 (0.0010) [2023-12-26 19:15:55,175][105620] Updated weights for policy 1, policy_version 536230 (0.0005) [2023-12-26 19:15:55,245][105692] Updated weights for policy 0, policy_version 535400 (0.0006) [2023-12-26 19:15:55,314][105692] Updated weights for policy 0, policy_version 535410 (0.0008) [2023-12-26 19:15:55,380][105692] Updated weights for policy 0, policy_version 535420 (0.0008) [2023-12-26 19:15:55,902][105620] Updated weights for policy 1, policy_version 536240 (0.0010) [2023-12-26 19:15:55,951][105620] Updated weights for policy 1, policy_version 536250 (0.0009) [2023-12-26 19:15:56,004][105620] Updated weights for policy 1, policy_version 536260 (0.0008) [2023-12-26 19:15:56,028][105692] Updated weights for policy 0, policy_version 535430 (0.0006) [2023-12-26 19:15:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19438.7). Total num frames: 274382848. Throughput: 0: 9938.8, 1: 9864.0. Samples: 274392128. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-26 19:15:56,062][104569] Avg episode reward: [(0, '8724.193'), (1, '9260.824')] [2023-12-26 19:15:56,075][105692] Updated weights for policy 0, policy_version 535440 (0.0008) [2023-12-26 19:15:56,123][105692] Updated weights for policy 0, policy_version 535450 (0.0009) [2023-12-26 19:15:56,585][105620] Updated weights for policy 1, policy_version 536270 (0.0009) [2023-12-26 19:15:56,639][105620] Updated weights for policy 1, policy_version 536281 (0.0010) [2023-12-26 19:15:56,693][105620] Updated weights for policy 1, policy_version 536291 (0.0007) [2023-12-26 19:15:56,731][105692] Updated weights for policy 0, policy_version 535460 (0.0008) [2023-12-26 19:15:56,782][105692] Updated weights for policy 0, policy_version 535470 (0.0010) [2023-12-26 19:15:56,835][105692] Updated weights for policy 0, policy_version 535480 (0.0010) [2023-12-26 19:15:57,494][105620] Updated weights for policy 1, policy_version 536301 (0.0007) [2023-12-26 19:15:57,497][105692] Updated weights for policy 0, policy_version 535490 (0.0009) [2023-12-26 19:15:57,538][105620] Updated weights for policy 1, policy_version 536311 (0.0006) [2023-12-26 19:15:57,547][105692] Updated weights for policy 0, policy_version 535500 (0.0010) [2023-12-26 19:15:57,589][105620] Updated weights for policy 1, policy_version 536321 (0.0006) [2023-12-26 19:15:57,605][105692] Updated weights for policy 0, policy_version 535510 (0.0010) [2023-12-26 19:15:57,659][105692] Updated weights for policy 0, policy_version 535520 (0.0010) [2023-12-26 19:15:58,321][105620] Updated weights for policy 1, policy_version 536331 (0.0007) [2023-12-26 19:15:58,364][105692] Updated weights for policy 0, policy_version 535530 (0.0009) [2023-12-26 19:15:58,388][105620] Updated weights for policy 1, policy_version 536341 (0.0007) [2023-12-26 19:15:58,427][105692] Updated weights for policy 0, policy_version 535540 (0.0012) [2023-12-26 19:15:58,450][105620] Updated weights for policy 1, policy_version 536351 (0.0008) [2023-12-26 19:15:58,490][105692] Updated weights for policy 0, policy_version 535550 (0.0011) [2023-12-26 19:15:59,210][105620] Updated weights for policy 1, policy_version 536361 (0.0008) [2023-12-26 19:15:59,281][105620] Updated weights for policy 1, policy_version 536371 (0.0009) [2023-12-26 19:15:59,350][105620] Updated weights for policy 1, policy_version 536381 (0.0008) [2023-12-26 19:15:59,354][105692] Updated weights for policy 0, policy_version 535560 (0.0008) [2023-12-26 19:15:59,411][105620] Updated weights for policy 1, policy_version 536391 (0.0007) [2023-12-26 19:15:59,416][105692] Updated weights for policy 0, policy_version 535570 (0.0010) [2023-12-26 19:15:59,475][105692] Updated weights for policy 0, policy_version 535580 (0.0010) [2023-12-26 19:16:00,109][105620] Updated weights for policy 1, policy_version 536401 (0.0009) [2023-12-26 19:16:00,164][105692] Updated weights for policy 0, policy_version 535590 (0.0008) [2023-12-26 19:16:00,167][105620] Updated weights for policy 1, policy_version 536411 (0.0008) [2023-12-26 19:16:00,224][105620] Updated weights for policy 1, policy_version 536421 (0.0005) [2023-12-26 19:16:00,230][105692] Updated weights for policy 0, policy_version 535600 (0.0007) [2023-12-26 19:16:00,295][105692] Updated weights for policy 0, policy_version 535610 (0.0006) [2023-12-26 19:16:00,883][105692] Updated weights for policy 0, policy_version 535620 (0.0005) [2023-12-26 19:16:00,935][105692] Updated weights for policy 0, policy_version 535630 (0.0005) [2023-12-26 19:16:00,984][105620] Updated weights for policy 1, policy_version 536431 (0.0007) [2023-12-26 19:16:00,994][105692] Updated weights for policy 0, policy_version 535640 (0.0009) [2023-12-26 19:16:01,047][105620] Updated weights for policy 1, policy_version 536441 (0.0007) [2023-12-26 19:16:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 274481152. Throughput: 0: 9992.0, 1: 9869.9. Samples: 274452384. Policy #0 lag: (min: 31.0, avg: 32.3, max: 60.0) [2023-12-26 19:16:01,062][104569] Avg episode reward: [(0, '9264.695'), (1, '9174.673')] [2023-12-26 19:16:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000535648_137142272.pth... [2023-12-26 19:16:01,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000534496_136847360.pth [2023-12-26 19:16:01,102][105620] Updated weights for policy 1, policy_version 536451 (0.0009) [2023-12-26 19:16:01,135][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000536456_137347072.pth... [2023-12-26 19:16:01,138][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000535336_137060352.pth [2023-12-26 19:16:01,709][105692] Updated weights for policy 0, policy_version 535650 (0.0010) [2023-12-26 19:16:01,775][105692] Updated weights for policy 0, policy_version 535660 (0.0011) [2023-12-26 19:16:01,825][105620] Updated weights for policy 1, policy_version 536461 (0.0007) [2023-12-26 19:16:01,834][105692] Updated weights for policy 0, policy_version 535670 (0.0010) [2023-12-26 19:16:01,876][105620] Updated weights for policy 1, policy_version 536471 (0.0009) [2023-12-26 19:16:01,895][105692] Updated weights for policy 0, policy_version 535680 (0.0006) [2023-12-26 19:16:01,928][105620] Updated weights for policy 1, policy_version 536481 (0.0009) [2023-12-26 19:16:02,615][105692] Updated weights for policy 0, policy_version 535690 (0.0005) [2023-12-26 19:16:02,671][105692] Updated weights for policy 0, policy_version 535700 (0.0006) [2023-12-26 19:16:02,681][105620] Updated weights for policy 1, policy_version 536491 (0.0009) [2023-12-26 19:16:02,729][105692] Updated weights for policy 0, policy_version 535710 (0.0006) [2023-12-26 19:16:02,746][105620] Updated weights for policy 1, policy_version 536501 (0.0010) [2023-12-26 19:16:02,812][105620] Updated weights for policy 1, policy_version 536511 (0.0009) [2023-12-26 19:16:03,447][105692] Updated weights for policy 0, policy_version 535720 (0.0009) [2023-12-26 19:16:03,504][105692] Updated weights for policy 0, policy_version 535730 (0.0009) [2023-12-26 19:16:03,511][105620] Updated weights for policy 1, policy_version 536521 (0.0009) [2023-12-26 19:16:03,557][105620] Updated weights for policy 1, policy_version 536531 (0.0006) [2023-12-26 19:16:03,559][105692] Updated weights for policy 0, policy_version 535740 (0.0007) [2023-12-26 19:16:03,616][105620] Updated weights for policy 1, policy_version 536541 (0.0005) [2023-12-26 19:16:03,673][105620] Updated weights for policy 1, policy_version 536551 (0.0005) [2023-12-26 19:16:04,330][105692] Updated weights for policy 0, policy_version 535750 (0.0007) [2023-12-26 19:16:04,379][105620] Updated weights for policy 1, policy_version 536561 (0.0007) [2023-12-26 19:16:04,383][105692] Updated weights for policy 0, policy_version 535760 (0.0006) [2023-12-26 19:16:04,437][105620] Updated weights for policy 1, policy_version 536571 (0.0009) [2023-12-26 19:16:04,442][105692] Updated weights for policy 0, policy_version 535770 (0.0006) [2023-12-26 19:16:04,494][105620] Updated weights for policy 1, policy_version 536581 (0.0009) [2023-12-26 19:16:04,999][105692] Updated weights for policy 0, policy_version 535780 (0.0006) [2023-12-26 19:16:05,051][105692] Updated weights for policy 0, policy_version 535790 (0.0005) [2023-12-26 19:16:05,102][105692] Updated weights for policy 0, policy_version 535800 (0.0005) [2023-12-26 19:16:05,393][105620] Updated weights for policy 1, policy_version 536591 (0.0009) [2023-12-26 19:16:05,439][105620] Updated weights for policy 1, policy_version 536601 (0.0008) [2023-12-26 19:16:05,484][105620] Updated weights for policy 1, policy_version 536611 (0.0008) [2023-12-26 19:16:05,728][105692] Updated weights for policy 0, policy_version 535810 (0.0006) [2023-12-26 19:16:05,784][105692] Updated weights for policy 0, policy_version 535820 (0.0006) [2023-12-26 19:16:05,835][105692] Updated weights for policy 0, policy_version 535830 (0.0006) [2023-12-26 19:16:05,889][105692] Updated weights for policy 0, policy_version 535840 (0.0005) [2023-12-26 19:16:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 274579456. Throughput: 0: 9872.6, 1: 9774.2. Samples: 274566488. Policy #0 lag: (min: 31.0, avg: 32.3, max: 60.0) [2023-12-26 19:16:06,063][104569] Avg episode reward: [(0, '8986.785'), (1, '9089.325')] [2023-12-26 19:16:06,223][105620] Updated weights for policy 1, policy_version 536621 (0.0008) [2023-12-26 19:16:06,293][105620] Updated weights for policy 1, policy_version 536631 (0.0009) [2023-12-26 19:16:06,359][105620] Updated weights for policy 1, policy_version 536641 (0.0008) [2023-12-26 19:16:06,552][105692] Updated weights for policy 0, policy_version 535850 (0.0010) [2023-12-26 19:16:06,598][105692] Updated weights for policy 0, policy_version 535860 (0.0010) [2023-12-26 19:16:06,650][105692] Updated weights for policy 0, policy_version 535870 (0.0011) [2023-12-26 19:16:07,132][105620] Updated weights for policy 1, policy_version 536651 (0.0009) [2023-12-26 19:16:07,195][105620] Updated weights for policy 1, policy_version 536661 (0.0009) [2023-12-26 19:16:07,262][105620] Updated weights for policy 1, policy_version 536671 (0.0009) [2023-12-26 19:16:07,356][105692] Updated weights for policy 0, policy_version 535880 (0.0008) [2023-12-26 19:16:07,407][105692] Updated weights for policy 0, policy_version 535890 (0.0009) [2023-12-26 19:16:07,454][105692] Updated weights for policy 0, policy_version 535900 (0.0008) [2023-12-26 19:16:08,019][105620] Updated weights for policy 1, policy_version 536681 (0.0009) [2023-12-26 19:16:08,083][105620] Updated weights for policy 1, policy_version 536691 (0.0009) [2023-12-26 19:16:08,144][105620] Updated weights for policy 1, policy_version 536701 (0.0010) [2023-12-26 19:16:08,194][105692] Updated weights for policy 0, policy_version 535910 (0.0007) [2023-12-26 19:16:08,202][105620] Updated weights for policy 1, policy_version 536711 (0.0008) [2023-12-26 19:16:08,241][105692] Updated weights for policy 0, policy_version 535920 (0.0005) [2023-12-26 19:16:08,286][105692] Updated weights for policy 0, policy_version 535930 (0.0005) [2023-12-26 19:16:08,906][105620] Updated weights for policy 1, policy_version 536721 (0.0010) [2023-12-26 19:16:08,964][105620] Updated weights for policy 1, policy_version 536731 (0.0010) [2023-12-26 19:16:09,023][105620] Updated weights for policy 1, policy_version 536741 (0.0010) [2023-12-26 19:16:09,027][105692] Updated weights for policy 0, policy_version 535940 (0.0008) [2023-12-26 19:16:09,088][105692] Updated weights for policy 0, policy_version 535950 (0.0009) [2023-12-26 19:16:09,152][105692] Updated weights for policy 0, policy_version 535960 (0.0007) [2023-12-26 19:16:09,726][105620] Updated weights for policy 1, policy_version 536751 (0.0011) [2023-12-26 19:16:09,793][105620] Updated weights for policy 1, policy_version 536761 (0.0011) [2023-12-26 19:16:09,800][105692] Updated weights for policy 0, policy_version 535970 (0.0009) [2023-12-26 19:16:09,853][105620] Updated weights for policy 1, policy_version 536771 (0.0009) [2023-12-26 19:16:09,859][105692] Updated weights for policy 0, policy_version 535980 (0.0007) [2023-12-26 19:16:09,921][105692] Updated weights for policy 0, policy_version 535990 (0.0009) [2023-12-26 19:16:09,988][105692] Updated weights for policy 0, policy_version 536000 (0.0009) [2023-12-26 19:16:10,632][105620] Updated weights for policy 1, policy_version 536781 (0.0007) [2023-12-26 19:16:10,676][105692] Updated weights for policy 0, policy_version 536010 (0.0008) [2023-12-26 19:16:10,690][105620] Updated weights for policy 1, policy_version 536791 (0.0006) [2023-12-26 19:16:10,740][105692] Updated weights for policy 0, policy_version 536020 (0.0006) [2023-12-26 19:16:10,753][105620] Updated weights for policy 1, policy_version 536801 (0.0008) [2023-12-26 19:16:10,802][105692] Updated weights for policy 0, policy_version 536030 (0.0008) [2023-12-26 19:16:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 274677760. Throughput: 0: 9953.5, 1: 9732.7. Samples: 274685108. Policy #0 lag: (min: 31.0, avg: 32.3, max: 60.0) [2023-12-26 19:16:11,062][104569] Avg episode reward: [(0, '8985.846'), (1, '9089.518')] [2023-12-26 19:16:11,359][105620] Updated weights for policy 1, policy_version 536811 (0.0006) [2023-12-26 19:16:11,425][105620] Updated weights for policy 1, policy_version 536821 (0.0008) [2023-12-26 19:16:11,489][105620] Updated weights for policy 1, policy_version 536831 (0.0009) [2023-12-26 19:16:11,613][105692] Updated weights for policy 0, policy_version 536040 (0.0008) [2023-12-26 19:16:11,677][105692] Updated weights for policy 0, policy_version 536050 (0.0008) [2023-12-26 19:16:11,744][105692] Updated weights for policy 0, policy_version 536060 (0.0010) [2023-12-26 19:16:12,194][105620] Updated weights for policy 1, policy_version 536841 (0.0008) [2023-12-26 19:16:12,266][105620] Updated weights for policy 1, policy_version 536851 (0.0007) [2023-12-26 19:16:12,334][105620] Updated weights for policy 1, policy_version 536861 (0.0007) [2023-12-26 19:16:12,399][105620] Updated weights for policy 1, policy_version 536871 (0.0009) [2023-12-26 19:16:12,531][105692] Updated weights for policy 0, policy_version 536070 (0.0007) [2023-12-26 19:16:12,579][105692] Updated weights for policy 0, policy_version 536080 (0.0005) [2023-12-26 19:16:12,635][105692] Updated weights for policy 0, policy_version 536090 (0.0005) [2023-12-26 19:16:13,147][105620] Updated weights for policy 1, policy_version 536881 (0.0010) [2023-12-26 19:16:13,199][105620] Updated weights for policy 1, policy_version 536891 (0.0010) [2023-12-26 19:16:13,251][105692] Updated weights for policy 0, policy_version 536100 (0.0008) [2023-12-26 19:16:13,253][105620] Updated weights for policy 1, policy_version 536901 (0.0010) [2023-12-26 19:16:13,310][105692] Updated weights for policy 0, policy_version 536110 (0.0007) [2023-12-26 19:16:13,369][105692] Updated weights for policy 0, policy_version 536120 (0.0009) [2023-12-26 19:16:13,886][105620] Updated weights for policy 1, policy_version 536911 (0.0010) [2023-12-26 19:16:13,938][105620] Updated weights for policy 1, policy_version 536921 (0.0010) [2023-12-26 19:16:13,971][105692] Updated weights for policy 0, policy_version 536130 (0.0005) [2023-12-26 19:16:13,996][105620] Updated weights for policy 1, policy_version 536931 (0.0010) [2023-12-26 19:16:14,023][105692] Updated weights for policy 0, policy_version 536140 (0.0006) [2023-12-26 19:16:14,070][105692] Updated weights for policy 0, policy_version 536150 (0.0008) [2023-12-26 19:16:14,118][105692] Updated weights for policy 0, policy_version 536160 (0.0008) [2023-12-26 19:16:14,764][105620] Updated weights for policy 1, policy_version 536941 (0.0010) [2023-12-26 19:16:14,828][105620] Updated weights for policy 1, policy_version 536951 (0.0011) [2023-12-26 19:16:14,884][105620] Updated weights for policy 1, policy_version 536961 (0.0011) [2023-12-26 19:16:14,911][105692] Updated weights for policy 0, policy_version 536170 (0.0006) [2023-12-26 19:16:14,966][105692] Updated weights for policy 0, policy_version 536180 (0.0007) [2023-12-26 19:16:15,020][105692] Updated weights for policy 0, policy_version 536190 (0.0008) [2023-12-26 19:16:15,644][105620] Updated weights for policy 1, policy_version 536971 (0.0011) [2023-12-26 19:16:15,702][105620] Updated weights for policy 1, policy_version 536981 (0.0008) [2023-12-26 19:16:15,754][105620] Updated weights for policy 1, policy_version 536991 (0.0005) [2023-12-26 19:16:15,792][105692] Updated weights for policy 0, policy_version 536200 (0.0008) [2023-12-26 19:16:15,806][105585] KL-divergence is very high: 167.8392 [2023-12-26 19:16:15,845][105692] Updated weights for policy 0, policy_version 536210 (0.0010) [2023-12-26 19:16:15,850][105585] KL-divergence is very high: 214.4745 [2023-12-26 19:16:15,900][105585] KL-divergence is very high: 198.1486 [2023-12-26 19:16:15,908][105692] Updated weights for policy 0, policy_version 536220 (0.0010) [2023-12-26 19:16:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 274776064. Throughput: 0: 9896.7, 1: 9595.2. Samples: 274743436. Policy #0 lag: (min: 31.0, avg: 32.3, max: 60.0) [2023-12-26 19:16:16,062][104569] Avg episode reward: [(0, '8805.919'), (1, '9263.142')] [2023-12-26 19:16:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000536224_137289728.pth... [2023-12-26 19:16:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000537000_137486336.pth... [2023-12-26 19:16:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000535072_136994816.pth [2023-12-26 19:16:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000535880_137199616.pth [2023-12-26 19:16:16,424][105620] Updated weights for policy 1, policy_version 537001 (0.0006) [2023-12-26 19:16:16,482][105620] Updated weights for policy 1, policy_version 537011 (0.0007) [2023-12-26 19:16:16,542][105620] Updated weights for policy 1, policy_version 537021 (0.0006) [2023-12-26 19:16:16,595][105620] Updated weights for policy 1, policy_version 537031 (0.0009) [2023-12-26 19:16:16,646][105692] Updated weights for policy 0, policy_version 536230 (0.0009) [2023-12-26 19:16:16,698][105692] Updated weights for policy 0, policy_version 536240 (0.0009) [2023-12-26 19:16:16,762][105692] Updated weights for policy 0, policy_version 536250 (0.0009) [2023-12-26 19:16:17,261][105620] Updated weights for policy 1, policy_version 537041 (0.0009) [2023-12-26 19:16:17,307][105620] Updated weights for policy 1, policy_version 537051 (0.0008) [2023-12-26 19:16:17,354][105620] Updated weights for policy 1, policy_version 537061 (0.0009) [2023-12-26 19:16:17,535][105692] Updated weights for policy 0, policy_version 536260 (0.0007) [2023-12-26 19:16:17,595][105692] Updated weights for policy 0, policy_version 536270 (0.0009) [2023-12-26 19:16:17,657][105692] Updated weights for policy 0, policy_version 536280 (0.0009) [2023-12-26 19:16:18,039][105620] Updated weights for policy 1, policy_version 537071 (0.0009) [2023-12-26 19:16:18,099][105620] Updated weights for policy 1, policy_version 537081 (0.0010) [2023-12-26 19:16:18,153][105620] Updated weights for policy 1, policy_version 537091 (0.0009) [2023-12-26 19:16:18,490][105692] Updated weights for policy 0, policy_version 536290 (0.0009) [2023-12-26 19:16:18,549][105692] Updated weights for policy 0, policy_version 536300 (0.0008) [2023-12-26 19:16:18,602][105692] Updated weights for policy 0, policy_version 536310 (0.0010) [2023-12-26 19:16:18,658][105692] Updated weights for policy 0, policy_version 536320 (0.0010) [2023-12-26 19:16:18,766][105620] Updated weights for policy 1, policy_version 537101 (0.0008) [2023-12-26 19:16:18,829][105620] Updated weights for policy 1, policy_version 537111 (0.0010) [2023-12-26 19:16:18,892][105620] Updated weights for policy 1, policy_version 537121 (0.0010) [2023-12-26 19:16:19,466][105692] Updated weights for policy 0, policy_version 536330 (0.0008) [2023-12-26 19:16:19,525][105692] Updated weights for policy 0, policy_version 536340 (0.0009) [2023-12-26 19:16:19,582][105692] Updated weights for policy 0, policy_version 536350 (0.0009) [2023-12-26 19:16:19,584][105620] Updated weights for policy 1, policy_version 537131 (0.0009) [2023-12-26 19:16:19,647][105620] Updated weights for policy 1, policy_version 537141 (0.0007) [2023-12-26 19:16:19,702][105620] Updated weights for policy 1, policy_version 537151 (0.0005) [2023-12-26 19:16:20,298][105620] Updated weights for policy 1, policy_version 537161 (0.0008) [2023-12-26 19:16:20,365][105620] Updated weights for policy 1, policy_version 537171 (0.0009) [2023-12-26 19:16:20,425][105620] Updated weights for policy 1, policy_version 537181 (0.0010) [2023-12-26 19:16:20,468][105692] Updated weights for policy 0, policy_version 536360 (0.0006) [2023-12-26 19:16:20,484][105620] Updated weights for policy 1, policy_version 537191 (0.0010) [2023-12-26 19:16:20,532][105692] Updated weights for policy 0, policy_version 536370 (0.0005) [2023-12-26 19:16:20,598][105692] Updated weights for policy 0, policy_version 536380 (0.0007) [2023-12-26 19:16:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 274866176. Throughput: 0: 9756.6, 1: 9579.7. Samples: 274859324. Policy #0 lag: (min: 31.0, avg: 32.3, max: 60.0) [2023-12-26 19:16:21,062][104569] Avg episode reward: [(0, '8805.977'), (1, '9262.959')] [2023-12-26 19:16:21,088][105620] Updated weights for policy 1, policy_version 537201 (0.0008) [2023-12-26 19:16:21,140][105620] Updated weights for policy 1, policy_version 537211 (0.0008) [2023-12-26 19:16:21,207][105620] Updated weights for policy 1, policy_version 537221 (0.0008) [2023-12-26 19:16:21,364][105692] Updated weights for policy 0, policy_version 536390 (0.0008) [2023-12-26 19:16:21,428][105692] Updated weights for policy 0, policy_version 536400 (0.0008) [2023-12-26 19:16:21,488][105692] Updated weights for policy 0, policy_version 536410 (0.0008) [2023-12-26 19:16:21,989][105620] Updated weights for policy 1, policy_version 537231 (0.0009) [2023-12-26 19:16:22,040][105620] Updated weights for policy 1, policy_version 537241 (0.0009) [2023-12-26 19:16:22,097][105620] Updated weights for policy 1, policy_version 537251 (0.0009) [2023-12-26 19:16:22,284][105692] Updated weights for policy 0, policy_version 536420 (0.0009) [2023-12-26 19:16:22,342][105692] Updated weights for policy 0, policy_version 536430 (0.0009) [2023-12-26 19:16:22,399][105692] Updated weights for policy 0, policy_version 536440 (0.0010) [2023-12-26 19:16:22,774][105620] Updated weights for policy 1, policy_version 537261 (0.0008) [2023-12-26 19:16:22,828][105620] Updated weights for policy 1, policy_version 537271 (0.0008) [2023-12-26 19:16:22,883][105620] Updated weights for policy 1, policy_version 537281 (0.0009) [2023-12-26 19:16:23,226][105692] Updated weights for policy 0, policy_version 536450 (0.0010) [2023-12-26 19:16:23,281][105692] Updated weights for policy 0, policy_version 536460 (0.0010) [2023-12-26 19:16:23,335][105692] Updated weights for policy 0, policy_version 536470 (0.0011) [2023-12-26 19:16:23,381][105692] Updated weights for policy 0, policy_version 536480 (0.0011) [2023-12-26 19:16:23,670][105620] Updated weights for policy 1, policy_version 537291 (0.0009) [2023-12-26 19:16:23,734][105620] Updated weights for policy 1, policy_version 537301 (0.0009) [2023-12-26 19:16:23,791][105620] Updated weights for policy 1, policy_version 537311 (0.0011) [2023-12-26 19:16:24,151][105692] Updated weights for policy 0, policy_version 536490 (0.0008) [2023-12-26 19:16:24,202][105692] Updated weights for policy 0, policy_version 536500 (0.0008) [2023-12-26 19:16:24,247][105692] Updated weights for policy 0, policy_version 536510 (0.0008) [2023-12-26 19:16:24,538][105620] Updated weights for policy 1, policy_version 537321 (0.0010) [2023-12-26 19:16:24,590][105620] Updated weights for policy 1, policy_version 537331 (0.0010) [2023-12-26 19:16:24,637][105620] Updated weights for policy 1, policy_version 537341 (0.0010) [2023-12-26 19:16:24,691][105620] Updated weights for policy 1, policy_version 537351 (0.0010) [2023-12-26 19:16:25,030][105692] Updated weights for policy 0, policy_version 536520 (0.0006) [2023-12-26 19:16:25,092][105692] Updated weights for policy 0, policy_version 536530 (0.0005) [2023-12-26 19:16:25,153][105692] Updated weights for policy 0, policy_version 536540 (0.0005) [2023-12-26 19:16:25,353][105620] Updated weights for policy 1, policy_version 537361 (0.0009) [2023-12-26 19:16:25,405][105620] Updated weights for policy 1, policy_version 537371 (0.0009) [2023-12-26 19:16:25,458][105620] Updated weights for policy 1, policy_version 537381 (0.0007) [2023-12-26 19:16:25,687][105692] Updated weights for policy 0, policy_version 536550 (0.0008) [2023-12-26 19:16:25,750][105692] Updated weights for policy 0, policy_version 536560 (0.0011) [2023-12-26 19:16:25,808][105692] Updated weights for policy 0, policy_version 536570 (0.0011) [2023-12-26 19:16:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 274964480. Throughput: 0: 9689.7, 1: 9708.8. Samples: 274974580. Policy #0 lag: (min: 31.0, avg: 32.3, max: 60.0) [2023-12-26 19:16:26,062][104569] Avg episode reward: [(0, '9172.437'), (1, '9262.417')] [2023-12-26 19:16:26,154][105620] Updated weights for policy 1, policy_version 537391 (0.0010) [2023-12-26 19:16:26,204][105620] Updated weights for policy 1, policy_version 537401 (0.0010) [2023-12-26 19:16:26,270][105620] Updated weights for policy 1, policy_version 537411 (0.0011) [2023-12-26 19:16:26,459][105692] Updated weights for policy 0, policy_version 536580 (0.0009) [2023-12-26 19:16:26,506][105692] Updated weights for policy 0, policy_version 536590 (0.0005) [2023-12-26 19:16:26,553][105692] Updated weights for policy 0, policy_version 536600 (0.0005) [2023-12-26 19:16:26,955][105620] Updated weights for policy 1, policy_version 537421 (0.0010) [2023-12-26 19:16:27,015][105620] Updated weights for policy 1, policy_version 537431 (0.0010) [2023-12-26 19:16:27,075][105620] Updated weights for policy 1, policy_version 537441 (0.0010) [2023-12-26 19:16:27,123][105692] Updated weights for policy 0, policy_version 536610 (0.0006) [2023-12-26 19:16:27,175][105692] Updated weights for policy 0, policy_version 536620 (0.0010) [2023-12-26 19:16:27,250][105692] Updated weights for policy 0, policy_version 536630 (0.0010) [2023-12-26 19:16:27,329][105692] Updated weights for policy 0, policy_version 536640 (0.0010) [2023-12-26 19:16:27,622][105620] Updated weights for policy 1, policy_version 537451 (0.0009) [2023-12-26 19:16:27,674][105620] Updated weights for policy 1, policy_version 537461 (0.0005) [2023-12-26 19:16:27,749][105620] Updated weights for policy 1, policy_version 537472 (0.0005) [2023-12-26 19:16:28,020][105692] Updated weights for policy 0, policy_version 536650 (0.0005) [2023-12-26 19:16:28,067][105692] Updated weights for policy 0, policy_version 536660 (0.0005) [2023-12-26 19:16:28,128][105692] Updated weights for policy 0, policy_version 536670 (0.0008) [2023-12-26 19:16:28,461][105620] Updated weights for policy 1, policy_version 537482 (0.0010) [2023-12-26 19:16:28,508][105620] Updated weights for policy 1, policy_version 537492 (0.0010) [2023-12-26 19:16:28,563][105620] Updated weights for policy 1, policy_version 537502 (0.0010) [2023-12-26 19:16:28,617][105620] Updated weights for policy 1, policy_version 537512 (0.0010) [2023-12-26 19:16:28,733][105692] Updated weights for policy 0, policy_version 536680 (0.0006) [2023-12-26 19:16:28,791][105692] Updated weights for policy 0, policy_version 536690 (0.0005) [2023-12-26 19:16:28,855][105692] Updated weights for policy 0, policy_version 536700 (0.0009) [2023-12-26 19:16:29,296][105620] Updated weights for policy 1, policy_version 537522 (0.0006) [2023-12-26 19:16:29,357][105620] Updated weights for policy 1, policy_version 537532 (0.0009) [2023-12-26 19:16:29,420][105620] Updated weights for policy 1, policy_version 537542 (0.0005) [2023-12-26 19:16:29,586][105692] Updated weights for policy 0, policy_version 536710 (0.0007) [2023-12-26 19:16:29,636][105692] Updated weights for policy 0, policy_version 536720 (0.0008) [2023-12-26 19:16:29,684][105585] KL-divergence is very high: 114.9980 [2023-12-26 19:16:29,692][105692] Updated weights for policy 0, policy_version 536730 (0.0010) [2023-12-26 19:16:30,043][105620] Updated weights for policy 1, policy_version 537552 (0.0010) [2023-12-26 19:16:30,103][105620] Updated weights for policy 1, policy_version 537562 (0.0009) [2023-12-26 19:16:30,164][105620] Updated weights for policy 1, policy_version 537572 (0.0009) [2023-12-26 19:16:30,435][105692] Updated weights for policy 0, policy_version 536740 (0.0010) [2023-12-26 19:16:30,493][105692] Updated weights for policy 0, policy_version 536750 (0.0009) [2023-12-26 19:16:30,551][105692] Updated weights for policy 0, policy_version 536760 (0.0009) [2023-12-26 19:16:30,908][105620] Updated weights for policy 1, policy_version 537582 (0.0009) [2023-12-26 19:16:30,960][105620] Updated weights for policy 1, policy_version 537592 (0.0010) [2023-12-26 19:16:31,015][105620] Updated weights for policy 1, policy_version 537602 (0.0010) [2023-12-26 19:16:31,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 275070976. Throughput: 0: 9761.4, 1: 9808.6. Samples: 275038320. Policy #0 lag: (min: 31.0, avg: 32.3, max: 60.0) [2023-12-26 19:16:31,062][104569] Avg episode reward: [(0, '9082.179'), (1, '9262.485')] [2023-12-26 19:16:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000536768_137428992.pth... [2023-12-26 19:16:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000537608_137641984.pth... [2023-12-26 19:16:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000535648_137142272.pth [2023-12-26 19:16:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000536456_137347072.pth [2023-12-26 19:16:31,222][105692] Updated weights for policy 0, policy_version 536770 (0.0008) [2023-12-26 19:16:31,281][105692] Updated weights for policy 0, policy_version 536780 (0.0006) [2023-12-26 19:16:31,338][105692] Updated weights for policy 0, policy_version 536790 (0.0009) [2023-12-26 19:16:31,401][105692] Updated weights for policy 0, policy_version 536800 (0.0007) [2023-12-26 19:16:31,764][105620] Updated weights for policy 1, policy_version 537612 (0.0006) [2023-12-26 19:16:31,821][105620] Updated weights for policy 1, policy_version 537622 (0.0008) [2023-12-26 19:16:31,905][105620] Updated weights for policy 1, policy_version 537632 (0.0006) [2023-12-26 19:16:32,113][105692] Updated weights for policy 0, policy_version 536810 (0.0008) [2023-12-26 19:16:32,172][105692] Updated weights for policy 0, policy_version 536820 (0.0008) [2023-12-26 19:16:32,184][105585] KL-divergence is very high: 440.1129 [2023-12-26 19:16:32,230][105692] Updated weights for policy 0, policy_version 536830 (0.0008) [2023-12-26 19:16:32,231][105585] KL-divergence is very high: 712.1884 [2023-12-26 19:16:32,596][105620] Updated weights for policy 1, policy_version 537642 (0.0009) [2023-12-26 19:16:32,657][105620] Updated weights for policy 1, policy_version 537652 (0.0006) [2023-12-26 19:16:32,712][105620] Updated weights for policy 1, policy_version 537662 (0.0005) [2023-12-26 19:16:32,781][105620] Updated weights for policy 1, policy_version 537672 (0.0005) [2023-12-26 19:16:32,995][105692] Updated weights for policy 0, policy_version 536840 (0.0010) [2023-12-26 19:16:33,044][105692] Updated weights for policy 0, policy_version 536850 (0.0010) [2023-12-26 19:16:33,089][105692] Updated weights for policy 0, policy_version 536860 (0.0010) [2023-12-26 19:16:33,412][105620] Updated weights for policy 1, policy_version 537682 (0.0006) [2023-12-26 19:16:33,483][105620] Updated weights for policy 1, policy_version 537692 (0.0005) [2023-12-26 19:16:33,551][105620] Updated weights for policy 1, policy_version 537702 (0.0005) [2023-12-26 19:16:33,784][105692] Updated weights for policy 0, policy_version 536870 (0.0009) [2023-12-26 19:16:33,839][105692] Updated weights for policy 0, policy_version 536880 (0.0009) [2023-12-26 19:16:33,891][105692] Updated weights for policy 0, policy_version 536891 (0.0010) [2023-12-26 19:16:34,032][105620] Updated weights for policy 1, policy_version 537712 (0.0007) [2023-12-26 19:16:34,078][105620] Updated weights for policy 1, policy_version 537722 (0.0006) [2023-12-26 19:16:34,127][105620] Updated weights for policy 1, policy_version 537732 (0.0006) [2023-12-26 19:16:34,673][105692] Updated weights for policy 0, policy_version 536901 (0.0009) [2023-12-26 19:16:34,728][105692] Updated weights for policy 0, policy_version 536911 (0.0008) [2023-12-26 19:16:34,788][105620] Updated weights for policy 1, policy_version 537742 (0.0009) [2023-12-26 19:16:34,790][105692] Updated weights for policy 0, policy_version 536921 (0.0008) [2023-12-26 19:16:34,846][105620] Updated weights for policy 1, policy_version 537752 (0.0010) [2023-12-26 19:16:34,912][105620] Updated weights for policy 1, policy_version 537762 (0.0010) [2023-12-26 19:16:35,470][105692] Updated weights for policy 0, policy_version 536931 (0.0007) [2023-12-26 19:16:35,521][105692] Updated weights for policy 0, policy_version 536941 (0.0008) [2023-12-26 19:16:35,576][105692] Updated weights for policy 0, policy_version 536951 (0.0008) [2023-12-26 19:16:35,639][105620] Updated weights for policy 1, policy_version 537772 (0.0010) [2023-12-26 19:16:35,700][105620] Updated weights for policy 1, policy_version 537782 (0.0010) [2023-12-26 19:16:35,770][105620] Updated weights for policy 1, policy_version 537792 (0.0010) [2023-12-26 19:16:36,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 275169280. Throughput: 0: 9773.5, 1: 9838.9. Samples: 275158620. Policy #0 lag: (min: 31.0, avg: 32.3, max: 60.0) [2023-12-26 19:16:36,062][104569] Avg episode reward: [(0, '7005.326'), (1, '9262.541')] [2023-12-26 19:16:36,348][105692] Updated weights for policy 0, policy_version 536961 (0.0008) [2023-12-26 19:16:36,415][105692] Updated weights for policy 0, policy_version 536971 (0.0008) [2023-12-26 19:16:36,479][105692] Updated weights for policy 0, policy_version 536981 (0.0008) [2023-12-26 19:16:36,506][105620] Updated weights for policy 1, policy_version 537802 (0.0010) [2023-12-26 19:16:36,540][105692] Updated weights for policy 0, policy_version 536991 (0.0006) [2023-12-26 19:16:36,569][105620] Updated weights for policy 1, policy_version 537812 (0.0011) [2023-12-26 19:16:36,628][105620] Updated weights for policy 1, policy_version 537822 (0.0010) [2023-12-26 19:16:36,687][105620] Updated weights for policy 1, policy_version 537832 (0.0011) [2023-12-26 19:16:37,305][105692] Updated weights for policy 0, policy_version 537001 (0.0008) [2023-12-26 19:16:37,365][105692] Updated weights for policy 0, policy_version 537011 (0.0008) [2023-12-26 19:16:37,415][105692] Updated weights for policy 0, policy_version 537021 (0.0006) [2023-12-26 19:16:37,432][105620] Updated weights for policy 1, policy_version 537842 (0.0011) [2023-12-26 19:16:37,488][105620] Updated weights for policy 1, policy_version 537852 (0.0010) [2023-12-26 19:16:37,537][105620] Updated weights for policy 1, policy_version 537862 (0.0010) [2023-12-26 19:16:38,177][105692] Updated weights for policy 0, policy_version 537031 (0.0008) [2023-12-26 19:16:38,225][105692] Updated weights for policy 0, policy_version 537041 (0.0008) [2023-12-26 19:16:38,289][105692] Updated weights for policy 0, policy_version 537051 (0.0007) [2023-12-26 19:16:38,310][105620] Updated weights for policy 1, policy_version 537872 (0.0010) [2023-12-26 19:16:38,370][105620] Updated weights for policy 1, policy_version 537882 (0.0010) [2023-12-26 19:16:38,435][105620] Updated weights for policy 1, policy_version 537892 (0.0010) [2023-12-26 19:16:39,060][105692] Updated weights for policy 0, policy_version 537061 (0.0007) [2023-12-26 19:16:39,109][105692] Updated weights for policy 0, policy_version 537071 (0.0008) [2023-12-26 19:16:39,155][105692] Updated weights for policy 0, policy_version 537081 (0.0008) [2023-12-26 19:16:39,194][105620] Updated weights for policy 1, policy_version 537902 (0.0010) [2023-12-26 19:16:39,254][105620] Updated weights for policy 1, policy_version 537912 (0.0011) [2023-12-26 19:16:39,310][105620] Updated weights for policy 1, policy_version 537922 (0.0010) [2023-12-26 19:16:39,955][105692] Updated weights for policy 0, policy_version 537091 (0.0006) [2023-12-26 19:16:40,019][105692] Updated weights for policy 0, policy_version 537101 (0.0007) [2023-12-26 19:16:40,084][105692] Updated weights for policy 0, policy_version 537111 (0.0008) [2023-12-26 19:16:40,104][105620] Updated weights for policy 1, policy_version 537932 (0.0009) [2023-12-26 19:16:40,165][105620] Updated weights for policy 1, policy_version 537942 (0.0010) [2023-12-26 19:16:40,219][105620] Updated weights for policy 1, policy_version 537952 (0.0010) [2023-12-26 19:16:40,849][105692] Updated weights for policy 0, policy_version 537121 (0.0007) [2023-12-26 19:16:40,887][105620] Updated weights for policy 1, policy_version 537962 (0.0008) [2023-12-26 19:16:40,904][105692] Updated weights for policy 0, policy_version 537131 (0.0006) [2023-12-26 19:16:40,951][105620] Updated weights for policy 1, policy_version 537972 (0.0006) [2023-12-26 19:16:40,971][105692] Updated weights for policy 0, policy_version 537141 (0.0008) [2023-12-26 19:16:41,021][105620] Updated weights for policy 1, policy_version 537982 (0.0006) [2023-12-26 19:16:41,029][105692] Updated weights for policy 0, policy_version 537151 (0.0008) [2023-12-26 19:16:41,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 275259392. Throughput: 0: 9686.8, 1: 9833.3. Samples: 275270536. Policy #0 lag: (min: 31.0, avg: 32.3, max: 60.0) [2023-12-26 19:16:41,062][104569] Avg episode reward: [(0, '3691.068'), (1, '9172.171')] [2023-12-26 19:16:41,082][105620] Updated weights for policy 1, policy_version 537992 (0.0007) [2023-12-26 19:16:41,763][105692] Updated weights for policy 0, policy_version 537161 (0.0008) [2023-12-26 19:16:41,781][105620] Updated weights for policy 1, policy_version 538002 (0.0008) [2023-12-26 19:16:41,818][105692] Updated weights for policy 0, policy_version 537171 (0.0006) [2023-12-26 19:16:41,840][105620] Updated weights for policy 1, policy_version 538012 (0.0008) [2023-12-26 19:16:41,878][105692] Updated weights for policy 0, policy_version 537181 (0.0008) [2023-12-26 19:16:41,903][105620] Updated weights for policy 1, policy_version 538022 (0.0008) [2023-12-26 19:16:42,652][105620] Updated weights for policy 1, policy_version 538032 (0.0006) [2023-12-26 19:16:42,679][105692] Updated weights for policy 0, policy_version 537191 (0.0009) [2023-12-26 19:16:42,711][105620] Updated weights for policy 1, policy_version 538042 (0.0009) [2023-12-26 19:16:42,729][105692] Updated weights for policy 0, policy_version 537201 (0.0008) [2023-12-26 19:16:42,768][105620] Updated weights for policy 1, policy_version 538052 (0.0010) [2023-12-26 19:16:42,793][105692] Updated weights for policy 0, policy_version 537211 (0.0007) [2023-12-26 19:16:43,461][105620] Updated weights for policy 1, policy_version 538062 (0.0010) [2023-12-26 19:16:43,509][105620] Updated weights for policy 1, policy_version 538072 (0.0010) [2023-12-26 19:16:43,562][105620] Updated weights for policy 1, policy_version 538082 (0.0010) [2023-12-26 19:16:43,570][105692] Updated weights for policy 0, policy_version 537221 (0.0006) [2023-12-26 19:16:43,623][105692] Updated weights for policy 0, policy_version 537231 (0.0005) [2023-12-26 19:16:43,679][105692] Updated weights for policy 0, policy_version 537241 (0.0005) [2023-12-26 19:16:44,293][105692] Updated weights for policy 0, policy_version 537251 (0.0006) [2023-12-26 19:16:44,298][105620] Updated weights for policy 1, policy_version 538092 (0.0011) [2023-12-26 19:16:44,331][105585] KL-divergence is very high: 142.6299 [2023-12-26 19:16:44,341][105692] Updated weights for policy 0, policy_version 537261 (0.0006) [2023-12-26 19:16:44,345][105620] Updated weights for policy 1, policy_version 538102 (0.0009) [2023-12-26 19:16:44,367][105585] KL-divergence is very high: 288.7993 [2023-12-26 19:16:44,375][105585] KL-divergence is very high: 311.2273 [2023-12-26 19:16:44,401][105692] Updated weights for policy 0, policy_version 537271 (0.0006) [2023-12-26 19:16:44,403][105620] Updated weights for policy 1, policy_version 538112 (0.0007) [2023-12-26 19:16:44,420][105585] KL-divergence is very high: 147.5099 [2023-12-26 19:16:44,426][105585] KL-divergence is very high: 149.9978 [2023-12-26 19:16:45,140][105620] Updated weights for policy 1, policy_version 538122 (0.0009) [2023-12-26 19:16:45,206][105620] Updated weights for policy 1, policy_version 538132 (0.0011) [2023-12-26 19:16:45,212][105692] Updated weights for policy 0, policy_version 537281 (0.0007) [2023-12-26 19:16:45,248][105585] KL-divergence is very high: 127.8791 [2023-12-26 19:16:45,270][105620] Updated weights for policy 1, policy_version 538142 (0.0011) [2023-12-26 19:16:45,275][105692] Updated weights for policy 0, policy_version 537291 (0.0006) [2023-12-26 19:16:45,333][105620] Updated weights for policy 1, policy_version 538152 (0.0011) [2023-12-26 19:16:45,340][105692] Updated weights for policy 0, policy_version 537301 (0.0006) [2023-12-26 19:16:45,407][105692] Updated weights for policy 0, policy_version 537311 (0.0008) [2023-12-26 19:16:45,979][105620] Updated weights for policy 1, policy_version 538162 (0.0009) [2023-12-26 19:16:46,044][105620] Updated weights for policy 1, policy_version 538172 (0.0009) [2023-12-26 19:16:46,062][104569] Fps is (10 sec: 18021.7, 60 sec: 19251.1, 300 sec: 19410.8). Total num frames: 275349504. Throughput: 0: 9613.8, 1: 9817.4. Samples: 275326796. Policy #0 lag: (min: 31.0, avg: 32.3, max: 60.0) [2023-12-26 19:16:46,063][104569] Avg episode reward: [(0, '6678.491'), (1, '9261.736')] [2023-12-26 19:16:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000537312_137568256.pth... [2023-12-26 19:16:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000536224_137289728.pth [2023-12-26 19:16:46,105][105620] Updated weights for policy 1, policy_version 538182 (0.0008) [2023-12-26 19:16:46,115][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000538184_137789440.pth... [2023-12-26 19:16:46,119][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000537000_137486336.pth [2023-12-26 19:16:46,169][105692] Updated weights for policy 0, policy_version 537321 (0.0009) [2023-12-26 19:16:46,219][105692] Updated weights for policy 0, policy_version 537331 (0.0009) [2023-12-26 19:16:46,269][105692] Updated weights for policy 0, policy_version 537341 (0.0008) [2023-12-26 19:16:46,748][105620] Updated weights for policy 1, policy_version 538192 (0.0009) [2023-12-26 19:16:46,803][105620] Updated weights for policy 1, policy_version 538202 (0.0009) [2023-12-26 19:16:46,848][105620] Updated weights for policy 1, policy_version 538212 (0.0008) [2023-12-26 19:16:46,868][105692] Updated weights for policy 0, policy_version 537351 (0.0008) [2023-12-26 19:16:46,918][105692] Updated weights for policy 0, policy_version 537361 (0.0008) [2023-12-26 19:16:46,965][105692] Updated weights for policy 0, policy_version 537371 (0.0009) [2023-12-26 19:16:47,525][105620] Updated weights for policy 1, policy_version 538222 (0.0007) [2023-12-26 19:16:47,585][105620] Updated weights for policy 1, policy_version 538232 (0.0005) [2023-12-26 19:16:47,648][105620] Updated weights for policy 1, policy_version 538242 (0.0008) [2023-12-26 19:16:47,765][105692] Updated weights for policy 0, policy_version 537381 (0.0009) [2023-12-26 19:16:47,813][105692] Updated weights for policy 0, policy_version 537391 (0.0007) [2023-12-26 19:16:47,859][105692] Updated weights for policy 0, policy_version 537401 (0.0006) [2023-12-26 19:16:48,251][105620] Updated weights for policy 1, policy_version 538252 (0.0008) [2023-12-26 19:16:48,297][105620] Updated weights for policy 1, policy_version 538262 (0.0005) [2023-12-26 19:16:48,358][105620] Updated weights for policy 1, policy_version 538272 (0.0007) [2023-12-26 19:16:48,645][105692] Updated weights for policy 0, policy_version 537411 (0.0007) [2023-12-26 19:16:48,716][105692] Updated weights for policy 0, policy_version 537421 (0.0010) [2023-12-26 19:16:48,782][105692] Updated weights for policy 0, policy_version 537431 (0.0009) [2023-12-26 19:16:48,957][105620] Updated weights for policy 1, policy_version 538282 (0.0007) [2023-12-26 19:16:49,007][105620] Updated weights for policy 1, policy_version 538293 (0.0009) [2023-12-26 19:16:49,053][105620] Updated weights for policy 1, policy_version 538303 (0.0009) [2023-12-26 19:16:49,508][105692] Updated weights for policy 0, policy_version 537441 (0.0010) [2023-12-26 19:16:49,571][105692] Updated weights for policy 0, policy_version 537451 (0.0009) [2023-12-26 19:16:49,635][105692] Updated weights for policy 0, policy_version 537461 (0.0010) [2023-12-26 19:16:49,693][105692] Updated weights for policy 0, policy_version 537471 (0.0009) [2023-12-26 19:16:49,824][105620] Updated weights for policy 1, policy_version 538313 (0.0009) [2023-12-26 19:16:49,894][105620] Updated weights for policy 1, policy_version 538323 (0.0008) [2023-12-26 19:16:49,957][105620] Updated weights for policy 1, policy_version 538333 (0.0008) [2023-12-26 19:16:50,012][105620] Updated weights for policy 1, policy_version 538343 (0.0008) [2023-12-26 19:16:50,416][105692] Updated weights for policy 0, policy_version 537481 (0.0008) [2023-12-26 19:16:50,474][105692] Updated weights for policy 0, policy_version 537491 (0.0009) [2023-12-26 19:16:50,530][105692] Updated weights for policy 0, policy_version 537501 (0.0008) [2023-12-26 19:16:50,766][105620] Updated weights for policy 1, policy_version 538353 (0.0009) [2023-12-26 19:16:50,814][105620] Updated weights for policy 1, policy_version 538363 (0.0009) [2023-12-26 19:16:50,873][105620] Updated weights for policy 1, policy_version 538373 (0.0009) [2023-12-26 19:16:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 275456000. Throughput: 0: 9602.8, 1: 9943.0. Samples: 275446048. Policy #0 lag: (min: 31.0, avg: 32.3, max: 60.0) [2023-12-26 19:16:51,063][104569] Avg episode reward: [(0, '8807.643'), (1, '9261.694')] [2023-12-26 19:16:51,308][105692] Updated weights for policy 0, policy_version 537511 (0.0008) [2023-12-26 19:16:51,361][105692] Updated weights for policy 0, policy_version 537521 (0.0008) [2023-12-26 19:16:51,430][105692] Updated weights for policy 0, policy_version 537531 (0.0008) [2023-12-26 19:16:51,653][105620] Updated weights for policy 1, policy_version 538383 (0.0008) [2023-12-26 19:16:51,728][105620] Updated weights for policy 1, policy_version 538393 (0.0009) [2023-12-26 19:16:51,787][105620] Updated weights for policy 1, policy_version 538403 (0.0009) [2023-12-26 19:16:52,123][105692] Updated weights for policy 0, policy_version 537541 (0.0009) [2023-12-26 19:16:52,186][105692] Updated weights for policy 0, policy_version 537551 (0.0009) [2023-12-26 19:16:52,236][105692] Updated weights for policy 0, policy_version 537561 (0.0009) [2023-12-26 19:16:52,498][105620] Updated weights for policy 1, policy_version 538413 (0.0008) [2023-12-26 19:16:52,552][105620] Updated weights for policy 1, policy_version 538423 (0.0008) [2023-12-26 19:16:52,598][105620] Updated weights for policy 1, policy_version 538433 (0.0009) [2023-12-26 19:16:52,966][105692] Updated weights for policy 0, policy_version 537571 (0.0008) [2023-12-26 19:16:53,022][105692] Updated weights for policy 0, policy_version 537581 (0.0009) [2023-12-26 19:16:53,081][105692] Updated weights for policy 0, policy_version 537591 (0.0010) [2023-12-26 19:16:53,300][105620] Updated weights for policy 1, policy_version 538443 (0.0009) [2023-12-26 19:16:53,349][105620] Updated weights for policy 1, policy_version 538453 (0.0008) [2023-12-26 19:16:53,403][105620] Updated weights for policy 1, policy_version 538463 (0.0009) [2023-12-26 19:16:53,842][105692] Updated weights for policy 0, policy_version 537602 (0.0010) [2023-12-26 19:16:53,902][105692] Updated weights for policy 0, policy_version 537612 (0.0009) [2023-12-26 19:16:53,949][105692] Updated weights for policy 0, policy_version 537622 (0.0008) [2023-12-26 19:16:53,996][105692] Updated weights for policy 0, policy_version 537632 (0.0009) [2023-12-26 19:16:54,192][105620] Updated weights for policy 1, policy_version 538473 (0.0008) [2023-12-26 19:16:54,268][105620] Updated weights for policy 1, policy_version 538483 (0.0005) [2023-12-26 19:16:54,336][105620] Updated weights for policy 1, policy_version 538493 (0.0005) [2023-12-26 19:16:54,392][105620] Updated weights for policy 1, policy_version 538503 (0.0009) [2023-12-26 19:16:54,709][105692] Updated weights for policy 0, policy_version 537642 (0.0009) [2023-12-26 19:16:54,767][105692] Updated weights for policy 0, policy_version 537652 (0.0007) [2023-12-26 19:16:54,817][105692] Updated weights for policy 0, policy_version 537662 (0.0005) [2023-12-26 19:16:55,123][105620] Updated weights for policy 1, policy_version 538513 (0.0009) [2023-12-26 19:16:55,182][105620] Updated weights for policy 1, policy_version 538523 (0.0009) [2023-12-26 19:16:55,238][105620] Updated weights for policy 1, policy_version 538533 (0.0009) [2023-12-26 19:16:55,395][105692] Updated weights for policy 0, policy_version 537672 (0.0009) [2023-12-26 19:16:55,443][105692] Updated weights for policy 0, policy_version 537682 (0.0010) [2023-12-26 19:16:55,490][105692] Updated weights for policy 0, policy_version 537692 (0.0010) [2023-12-26 19:16:56,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 275546112. Throughput: 0: 9521.0, 1: 9920.4. Samples: 275559968. Policy #0 lag: (min: 31.0, avg: 32.3, max: 60.0) [2023-12-26 19:16:56,062][104569] Avg episode reward: [(0, '8991.867'), (1, '9261.999')] [2023-12-26 19:16:56,064][105620] Updated weights for policy 1, policy_version 538543 (0.0009) [2023-12-26 19:16:56,124][105620] Updated weights for policy 1, policy_version 538553 (0.0008) [2023-12-26 19:16:56,169][105692] Updated weights for policy 0, policy_version 537702 (0.0009) [2023-12-26 19:16:56,181][105620] Updated weights for policy 1, policy_version 538563 (0.0007) [2023-12-26 19:16:56,222][105692] Updated weights for policy 0, policy_version 537712 (0.0006) [2023-12-26 19:16:56,268][105692] Updated weights for policy 0, policy_version 537722 (0.0005) [2023-12-26 19:16:56,854][105692] Updated weights for policy 0, policy_version 537732 (0.0008) [2023-12-26 19:16:56,907][105692] Updated weights for policy 0, policy_version 537742 (0.0006) [2023-12-26 19:16:56,963][105692] Updated weights for policy 0, policy_version 537752 (0.0006) [2023-12-26 19:16:56,992][105620] Updated weights for policy 1, policy_version 538573 (0.0008) [2023-12-26 19:16:57,044][105620] Updated weights for policy 1, policy_version 538583 (0.0008) [2023-12-26 19:16:57,104][105620] Updated weights for policy 1, policy_version 538593 (0.0008) [2023-12-26 19:16:57,530][105692] Updated weights for policy 0, policy_version 537762 (0.0007) [2023-12-26 19:16:57,591][105692] Updated weights for policy 0, policy_version 537772 (0.0009) [2023-12-26 19:16:57,648][105692] Updated weights for policy 0, policy_version 537782 (0.0007) [2023-12-26 19:16:57,694][105692] Updated weights for policy 0, policy_version 537792 (0.0008) [2023-12-26 19:16:57,948][105620] Updated weights for policy 1, policy_version 538603 (0.0010) [2023-12-26 19:16:58,004][105620] Updated weights for policy 1, policy_version 538613 (0.0008) [2023-12-26 19:16:58,067][105620] Updated weights for policy 1, policy_version 538623 (0.0005) [2023-12-26 19:16:58,430][105692] Updated weights for policy 0, policy_version 537802 (0.0009) [2023-12-26 19:16:58,498][105692] Updated weights for policy 0, policy_version 537812 (0.0008) [2023-12-26 19:16:58,566][105692] Updated weights for policy 0, policy_version 537822 (0.0008) [2023-12-26 19:16:58,819][105620] Updated weights for policy 1, policy_version 538633 (0.0008) [2023-12-26 19:16:58,884][105620] Updated weights for policy 1, policy_version 538643 (0.0009) [2023-12-26 19:16:58,947][105620] Updated weights for policy 1, policy_version 538653 (0.0007) [2023-12-26 19:16:59,014][105620] Updated weights for policy 1, policy_version 538663 (0.0008) [2023-12-26 19:16:59,365][105692] Updated weights for policy 0, policy_version 537832 (0.0008) [2023-12-26 19:16:59,422][105692] Updated weights for policy 0, policy_version 537842 (0.0008) [2023-12-26 19:16:59,473][105692] Updated weights for policy 0, policy_version 537852 (0.0010) [2023-12-26 19:16:59,712][105620] Updated weights for policy 1, policy_version 538673 (0.0008) [2023-12-26 19:16:59,770][105620] Updated weights for policy 1, policy_version 538683 (0.0008) [2023-12-26 19:16:59,829][105620] Updated weights for policy 1, policy_version 538693 (0.0008) [2023-12-26 19:17:00,228][105692] Updated weights for policy 0, policy_version 537862 (0.0010) [2023-12-26 19:17:00,283][105692] Updated weights for policy 0, policy_version 537872 (0.0010) [2023-12-26 19:17:00,347][105692] Updated weights for policy 0, policy_version 537882 (0.0010) [2023-12-26 19:17:00,560][105620] Updated weights for policy 1, policy_version 538703 (0.0008) [2023-12-26 19:17:00,618][105620] Updated weights for policy 1, policy_version 538713 (0.0008) [2023-12-26 19:17:00,669][105620] Updated weights for policy 1, policy_version 538723 (0.0011) [2023-12-26 19:17:00,998][105692] Updated weights for policy 0, policy_version 537892 (0.0008) [2023-12-26 19:17:01,061][105692] Updated weights for policy 0, policy_version 537902 (0.0006) [2023-12-26 19:17:01,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 275644416. Throughput: 0: 9603.3, 1: 9836.1. Samples: 275618208. Policy #0 lag: (min: 31.0, avg: 32.3, max: 60.0) [2023-12-26 19:17:01,062][104569] Avg episode reward: [(0, '8992.726'), (1, '9264.292')] [2023-12-26 19:17:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000538728_137928704.pth... [2023-12-26 19:17:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000537608_137641984.pth [2023-12-26 19:17:01,122][105692] Updated weights for policy 0, policy_version 537912 (0.0010) [2023-12-26 19:17:01,175][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000537920_137723904.pth... [2023-12-26 19:17:01,180][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000536768_137428992.pth [2023-12-26 19:17:01,514][105620] Updated weights for policy 1, policy_version 538733 (0.0007) [2023-12-26 19:17:01,575][105620] Updated weights for policy 1, policy_version 538743 (0.0009) [2023-12-26 19:17:01,633][105620] Updated weights for policy 1, policy_version 538753 (0.0009) [2023-12-26 19:17:01,786][105692] Updated weights for policy 0, policy_version 537922 (0.0010) [2023-12-26 19:17:01,844][105692] Updated weights for policy 0, policy_version 537932 (0.0010) [2023-12-26 19:17:01,909][105692] Updated weights for policy 0, policy_version 537942 (0.0009) [2023-12-26 19:17:01,965][105692] Updated weights for policy 0, policy_version 537952 (0.0009) [2023-12-26 19:17:02,338][105620] Updated weights for policy 1, policy_version 538763 (0.0007) [2023-12-26 19:17:02,397][105620] Updated weights for policy 1, policy_version 538773 (0.0009) [2023-12-26 19:17:02,459][105620] Updated weights for policy 1, policy_version 538783 (0.0009) [2023-12-26 19:17:02,664][105692] Updated weights for policy 0, policy_version 537962 (0.0009) [2023-12-26 19:17:02,726][105692] Updated weights for policy 0, policy_version 537972 (0.0009) [2023-12-26 19:17:02,787][105692] Updated weights for policy 0, policy_version 537982 (0.0010) [2023-12-26 19:17:03,093][105620] Updated weights for policy 1, policy_version 538793 (0.0008) [2023-12-26 19:17:03,145][105620] Updated weights for policy 1, policy_version 538803 (0.0006) [2023-12-26 19:17:03,191][105620] Updated weights for policy 1, policy_version 538813 (0.0005) [2023-12-26 19:17:03,252][105620] Updated weights for policy 1, policy_version 538823 (0.0005) [2023-12-26 19:17:03,483][105692] Updated weights for policy 0, policy_version 537992 (0.0010) [2023-12-26 19:17:03,534][105692] Updated weights for policy 0, policy_version 538002 (0.0010) [2023-12-26 19:17:03,539][105585] KL-divergence is very high: 122.7311 [2023-12-26 19:17:03,583][105585] KL-divergence is very high: 122.9305 [2023-12-26 19:17:03,588][105692] Updated weights for policy 0, policy_version 538012 (0.0010) [2023-12-26 19:17:03,784][105620] Updated weights for policy 1, policy_version 538833 (0.0005) [2023-12-26 19:17:03,830][105620] Updated weights for policy 1, policy_version 538843 (0.0005) [2023-12-26 19:17:03,895][105620] Updated weights for policy 1, policy_version 538853 (0.0007) [2023-12-26 19:17:04,229][105692] Updated weights for policy 0, policy_version 538022 (0.0009) [2023-12-26 19:17:04,296][105692] Updated weights for policy 0, policy_version 538032 (0.0011) [2023-12-26 19:17:04,362][105692] Updated weights for policy 0, policy_version 538042 (0.0006) [2023-12-26 19:17:04,591][105620] Updated weights for policy 1, policy_version 538863 (0.0005) [2023-12-26 19:17:04,649][105620] Updated weights for policy 1, policy_version 538873 (0.0006) [2023-12-26 19:17:04,707][105620] Updated weights for policy 1, policy_version 538883 (0.0005) [2023-12-26 19:17:05,076][105692] Updated weights for policy 0, policy_version 538052 (0.0009) [2023-12-26 19:17:05,129][105692] Updated weights for policy 0, policy_version 538062 (0.0010) [2023-12-26 19:17:05,186][105692] Updated weights for policy 0, policy_version 538072 (0.0010) [2023-12-26 19:17:05,235][105620] Updated weights for policy 1, policy_version 538893 (0.0005) [2023-12-26 19:17:05,283][105620] Updated weights for policy 1, policy_version 538903 (0.0005) [2023-12-26 19:17:05,341][105620] Updated weights for policy 1, policy_version 538913 (0.0005) [2023-12-26 19:17:05,869][105692] Updated weights for policy 0, policy_version 538082 (0.0010) [2023-12-26 19:17:05,869][105620] Updated weights for policy 1, policy_version 538923 (0.0006) [2023-12-26 19:17:05,925][105692] Updated weights for policy 0, policy_version 538092 (0.0007) [2023-12-26 19:17:05,927][105620] Updated weights for policy 1, policy_version 538933 (0.0007) [2023-12-26 19:17:05,976][105620] Updated weights for policy 1, policy_version 538943 (0.0006) [2023-12-26 19:17:05,985][105692] Updated weights for policy 0, policy_version 538102 (0.0008) [2023-12-26 19:17:06,046][105692] Updated weights for policy 0, policy_version 538112 (0.0007) [2023-12-26 19:17:06,062][104569] Fps is (10 sec: 21299.1, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 275759104. Throughput: 0: 9687.9, 1: 9851.4. Samples: 275738596. Policy #0 lag: (min: 31.0, avg: 32.3, max: 60.0) [2023-12-26 19:17:06,063][104569] Avg episode reward: [(0, '8352.167'), (1, '9264.376')] [2023-12-26 19:17:06,680][105620] Updated weights for policy 1, policy_version 538953 (0.0006) [2023-12-26 19:17:06,687][105692] Updated weights for policy 0, policy_version 538122 (0.0009) [2023-12-26 19:17:06,745][105620] Updated weights for policy 1, policy_version 538963 (0.0009) [2023-12-26 19:17:06,745][105692] Updated weights for policy 0, policy_version 538132 (0.0008) [2023-12-26 19:17:06,795][105692] Updated weights for policy 0, policy_version 538142 (0.0007) [2023-12-26 19:17:06,804][105620] Updated weights for policy 1, policy_version 538973 (0.0007) [2023-12-26 19:17:06,865][105620] Updated weights for policy 1, policy_version 538983 (0.0008) [2023-12-26 19:17:07,425][105692] Updated weights for policy 0, policy_version 538152 (0.0009) [2023-12-26 19:17:07,478][105692] Updated weights for policy 0, policy_version 538162 (0.0009) [2023-12-26 19:17:07,529][105692] Updated weights for policy 0, policy_version 538172 (0.0009) [2023-12-26 19:17:07,668][105620] Updated weights for policy 1, policy_version 538993 (0.0009) [2023-12-26 19:17:07,737][105620] Updated weights for policy 1, policy_version 539003 (0.0005) [2023-12-26 19:17:07,808][105620] Updated weights for policy 1, policy_version 539013 (0.0006) [2023-12-26 19:17:08,115][105692] Updated weights for policy 0, policy_version 538182 (0.0007) [2023-12-26 19:17:08,170][105692] Updated weights for policy 0, policy_version 538192 (0.0010) [2023-12-26 19:17:08,218][105692] Updated weights for policy 0, policy_version 538202 (0.0010) [2023-12-26 19:17:08,497][105620] Updated weights for policy 1, policy_version 539023 (0.0010) [2023-12-26 19:17:08,559][105620] Updated weights for policy 1, policy_version 539033 (0.0010) [2023-12-26 19:17:08,619][105620] Updated weights for policy 1, policy_version 539043 (0.0008) [2023-12-26 19:17:08,974][105692] Updated weights for policy 0, policy_version 538212 (0.0010) [2023-12-26 19:17:09,033][105692] Updated weights for policy 0, policy_version 538222 (0.0011) [2023-12-26 19:17:09,094][105692] Updated weights for policy 0, policy_version 538232 (0.0010) [2023-12-26 19:17:09,335][105620] Updated weights for policy 1, policy_version 539053 (0.0010) [2023-12-26 19:17:09,400][105620] Updated weights for policy 1, policy_version 539063 (0.0008) [2023-12-26 19:17:09,459][105620] Updated weights for policy 1, policy_version 539073 (0.0006) [2023-12-26 19:17:09,825][105692] Updated weights for policy 0, policy_version 538242 (0.0010) [2023-12-26 19:17:09,892][105692] Updated weights for policy 0, policy_version 538252 (0.0011) [2023-12-26 19:17:09,961][105692] Updated weights for policy 0, policy_version 538262 (0.0011) [2023-12-26 19:17:10,024][105692] Updated weights for policy 0, policy_version 538272 (0.0009) [2023-12-26 19:17:10,156][105620] Updated weights for policy 1, policy_version 539083 (0.0009) [2023-12-26 19:17:10,227][105620] Updated weights for policy 1, policy_version 539093 (0.0006) [2023-12-26 19:17:10,296][105620] Updated weights for policy 1, policy_version 539103 (0.0010) [2023-12-26 19:17:10,778][105692] Updated weights for policy 0, policy_version 538282 (0.0006) [2023-12-26 19:17:10,835][105692] Updated weights for policy 0, policy_version 538292 (0.0007) [2023-12-26 19:17:10,894][105692] Updated weights for policy 0, policy_version 538302 (0.0010) [2023-12-26 19:17:11,005][105620] Updated weights for policy 1, policy_version 539113 (0.0010) [2023-12-26 19:17:11,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 275849216. Throughput: 0: 9804.3, 1: 9856.9. Samples: 275859336. Policy #0 lag: (min: 31.0, avg: 32.3, max: 60.0) [2023-12-26 19:17:11,063][104569] Avg episode reward: [(0, '8626.160'), (1, '9353.457')] [2023-12-26 19:17:11,072][105620] Updated weights for policy 1, policy_version 539123 (0.0010) [2023-12-26 19:17:11,138][105620] Updated weights for policy 1, policy_version 539133 (0.0009) [2023-12-26 19:17:11,206][105620] Updated weights for policy 1, policy_version 539143 (0.0011) [2023-12-26 19:17:11,599][105692] Updated weights for policy 0, policy_version 538312 (0.0009) [2023-12-26 19:17:11,637][105585] KL-divergence is very high: 261.5801 [2023-12-26 19:17:11,662][105585] KL-divergence is very high: 102.3908 [2023-12-26 19:17:11,668][105692] Updated weights for policy 0, policy_version 538322 (0.0009) [2023-12-26 19:17:11,681][105585] KL-divergence is very high: 481.3065 [2023-12-26 19:17:11,724][105692] Updated weights for policy 0, policy_version 538332 (0.0008) [2023-12-26 19:17:11,735][105585] KL-divergence is very high: 545.0463 [2023-12-26 19:17:11,963][105620] Updated weights for policy 1, policy_version 539153 (0.0009) [2023-12-26 19:17:12,024][105620] Updated weights for policy 1, policy_version 539163 (0.0009) [2023-12-26 19:17:12,085][105620] Updated weights for policy 1, policy_version 539173 (0.0005) [2023-12-26 19:17:12,473][105692] Updated weights for policy 0, policy_version 538342 (0.0009) [2023-12-26 19:17:12,541][105692] Updated weights for policy 0, policy_version 538352 (0.0007) [2023-12-26 19:17:12,600][105692] Updated weights for policy 0, policy_version 538362 (0.0009) [2023-12-26 19:17:12,760][105620] Updated weights for policy 1, policy_version 539183 (0.0008) [2023-12-26 19:17:12,817][105620] Updated weights for policy 1, policy_version 539193 (0.0010) [2023-12-26 19:17:12,870][105620] Updated weights for policy 1, policy_version 539203 (0.0010) [2023-12-26 19:17:13,294][105692] Updated weights for policy 0, policy_version 538372 (0.0010) [2023-12-26 19:17:13,348][105692] Updated weights for policy 0, policy_version 538382 (0.0010) [2023-12-26 19:17:13,407][105692] Updated weights for policy 0, policy_version 538392 (0.0010) [2023-12-26 19:17:13,625][105620] Updated weights for policy 1, policy_version 539213 (0.0009) [2023-12-26 19:17:13,677][105620] Updated weights for policy 1, policy_version 539223 (0.0009) [2023-12-26 19:17:13,739][105620] Updated weights for policy 1, policy_version 539233 (0.0010) [2023-12-26 19:17:14,030][105692] Updated weights for policy 0, policy_version 538402 (0.0010) [2023-12-26 19:17:14,091][105692] Updated weights for policy 0, policy_version 538412 (0.0010) [2023-12-26 19:17:14,156][105692] Updated weights for policy 0, policy_version 538422 (0.0010) [2023-12-26 19:17:14,216][105692] Updated weights for policy 0, policy_version 538432 (0.0010) [2023-12-26 19:17:14,526][105620] Updated weights for policy 1, policy_version 539243 (0.0009) [2023-12-26 19:17:14,575][105620] Updated weights for policy 1, policy_version 539253 (0.0008) [2023-12-26 19:17:14,623][105620] Updated weights for policy 1, policy_version 539263 (0.0008) [2023-12-26 19:17:14,944][105692] Updated weights for policy 0, policy_version 538442 (0.0008) [2023-12-26 19:17:14,999][105692] Updated weights for policy 0, policy_version 538452 (0.0008) [2023-12-26 19:17:15,060][105692] Updated weights for policy 0, policy_version 538462 (0.0009) [2023-12-26 19:17:15,374][105620] Updated weights for policy 1, policy_version 539273 (0.0008) [2023-12-26 19:17:15,430][105620] Updated weights for policy 1, policy_version 539283 (0.0008) [2023-12-26 19:17:15,491][105620] Updated weights for policy 1, policy_version 539293 (0.0008) [2023-12-26 19:17:15,557][105620] Updated weights for policy 1, policy_version 539303 (0.0009) [2023-12-26 19:17:15,853][105692] Updated weights for policy 0, policy_version 538472 (0.0010) [2023-12-26 19:17:15,906][105692] Updated weights for policy 0, policy_version 538482 (0.0010) [2023-12-26 19:17:15,953][105692] Updated weights for policy 0, policy_version 538492 (0.0009) [2023-12-26 19:17:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 275947520. Throughput: 0: 9722.2, 1: 9773.2. Samples: 275915612. Policy #0 lag: (min: 31.0, avg: 32.3, max: 60.0) [2023-12-26 19:17:16,062][104569] Avg episode reward: [(0, '8809.323'), (1, '9353.463')] [2023-12-26 19:17:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000538496_137871360.pth... [2023-12-26 19:17:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000539304_138076160.pth... [2023-12-26 19:17:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000537312_137568256.pth [2023-12-26 19:17:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000538184_137789440.pth [2023-12-26 19:17:16,265][105620] Updated weights for policy 1, policy_version 539313 (0.0009) [2023-12-26 19:17:16,320][105620] Updated weights for policy 1, policy_version 539323 (0.0009) [2023-12-26 19:17:16,373][105620] Updated weights for policy 1, policy_version 539333 (0.0008) [2023-12-26 19:17:16,623][105692] Updated weights for policy 0, policy_version 538502 (0.0008) [2023-12-26 19:17:16,685][105692] Updated weights for policy 0, policy_version 538512 (0.0009) [2023-12-26 19:17:16,741][105692] Updated weights for policy 0, policy_version 538522 (0.0009) [2023-12-26 19:17:17,197][105620] Updated weights for policy 1, policy_version 539343 (0.0009) [2023-12-26 19:17:17,260][105620] Updated weights for policy 1, policy_version 539353 (0.0010) [2023-12-26 19:17:17,317][105692] Updated weights for policy 0, policy_version 538532 (0.0007) [2023-12-26 19:17:17,325][105620] Updated weights for policy 1, policy_version 539363 (0.0010) [2023-12-26 19:17:17,363][105692] Updated weights for policy 0, policy_version 538542 (0.0005) [2023-12-26 19:17:17,426][105692] Updated weights for policy 0, policy_version 538552 (0.0008) [2023-12-26 19:17:18,093][105620] Updated weights for policy 1, policy_version 539373 (0.0009) [2023-12-26 19:17:18,151][105620] Updated weights for policy 1, policy_version 539383 (0.0008) [2023-12-26 19:17:18,160][105692] Updated weights for policy 0, policy_version 538562 (0.0009) [2023-12-26 19:17:18,210][105620] Updated weights for policy 1, policy_version 539393 (0.0008) [2023-12-26 19:17:18,220][105692] Updated weights for policy 0, policy_version 538572 (0.0010) [2023-12-26 19:17:18,271][105692] Updated weights for policy 0, policy_version 538582 (0.0007) [2023-12-26 19:17:18,323][105692] Updated weights for policy 0, policy_version 538592 (0.0008) [2023-12-26 19:17:18,887][105620] Updated weights for policy 1, policy_version 539403 (0.0007) [2023-12-26 19:17:18,936][105620] Updated weights for policy 1, policy_version 539413 (0.0008) [2023-12-26 19:17:18,986][105620] Updated weights for policy 1, policy_version 539423 (0.0008) [2023-12-26 19:17:19,080][105692] Updated weights for policy 0, policy_version 538602 (0.0006) [2023-12-26 19:17:19,147][105692] Updated weights for policy 0, policy_version 538612 (0.0010) [2023-12-26 19:17:19,202][105692] Updated weights for policy 0, policy_version 538622 (0.0011) [2023-12-26 19:17:19,691][105620] Updated weights for policy 1, policy_version 539433 (0.0007) [2023-12-26 19:17:19,750][105620] Updated weights for policy 1, policy_version 539443 (0.0009) [2023-12-26 19:17:19,799][105620] Updated weights for policy 1, policy_version 539453 (0.0008) [2023-12-26 19:17:19,863][105620] Updated weights for policy 1, policy_version 539463 (0.0008) [2023-12-26 19:17:19,926][105692] Updated weights for policy 0, policy_version 538632 (0.0011) [2023-12-26 19:17:19,992][105692] Updated weights for policy 0, policy_version 538642 (0.0011) [2023-12-26 19:17:20,038][105692] Updated weights for policy 0, policy_version 538652 (0.0011) [2023-12-26 19:17:20,658][105620] Updated weights for policy 1, policy_version 539473 (0.0009) [2023-12-26 19:17:20,721][105620] Updated weights for policy 1, policy_version 539483 (0.0008) [2023-12-26 19:17:20,784][105620] Updated weights for policy 1, policy_version 539493 (0.0008) [2023-12-26 19:17:20,802][105692] Updated weights for policy 0, policy_version 538662 (0.0011) [2023-12-26 19:17:20,858][105692] Updated weights for policy 0, policy_version 538672 (0.0011) [2023-12-26 19:17:20,917][105692] Updated weights for policy 0, policy_version 538682 (0.0011) [2023-12-26 19:17:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 276045824. Throughput: 0: 9751.9, 1: 9647.7. Samples: 276031604. Policy #0 lag: (min: 31.0, avg: 32.3, max: 60.0) [2023-12-26 19:17:21,062][104569] Avg episode reward: [(0, '8630.657'), (1, '9082.453')] [2023-12-26 19:17:21,573][105692] Updated weights for policy 0, policy_version 538692 (0.0009) [2023-12-26 19:17:21,635][105692] Updated weights for policy 0, policy_version 538702 (0.0007) [2023-12-26 19:17:21,661][105620] Updated weights for policy 1, policy_version 539503 (0.0007) [2023-12-26 19:17:21,705][105692] Updated weights for policy 0, policy_version 538712 (0.0009) [2023-12-26 19:17:21,734][105620] Updated weights for policy 1, policy_version 539513 (0.0010) [2023-12-26 19:17:21,792][105620] Updated weights for policy 1, policy_version 539523 (0.0008) [2023-12-26 19:17:22,383][105692] Updated weights for policy 0, policy_version 538722 (0.0007) [2023-12-26 19:17:22,442][105692] Updated weights for policy 0, policy_version 538732 (0.0007) [2023-12-26 19:17:22,502][105692] Updated weights for policy 0, policy_version 538742 (0.0007) [2023-12-26 19:17:22,566][105692] Updated weights for policy 0, policy_version 538752 (0.0009) [2023-12-26 19:17:22,569][105620] Updated weights for policy 1, policy_version 539533 (0.0009) [2023-12-26 19:17:22,621][105620] Updated weights for policy 1, policy_version 539543 (0.0009) [2023-12-26 19:17:22,681][105620] Updated weights for policy 1, policy_version 539553 (0.0009) [2023-12-26 19:17:23,252][105692] Updated weights for policy 0, policy_version 538762 (0.0010) [2023-12-26 19:17:23,323][105692] Updated weights for policy 0, policy_version 538772 (0.0010) [2023-12-26 19:17:23,379][105620] Updated weights for policy 1, policy_version 539563 (0.0008) [2023-12-26 19:17:23,391][105692] Updated weights for policy 0, policy_version 538782 (0.0008) [2023-12-26 19:17:23,436][105620] Updated weights for policy 1, policy_version 539573 (0.0008) [2023-12-26 19:17:23,487][105620] Updated weights for policy 1, policy_version 539583 (0.0010) [2023-12-26 19:17:24,098][105692] Updated weights for policy 0, policy_version 538792 (0.0010) [2023-12-26 19:17:24,163][105620] Updated weights for policy 1, policy_version 539593 (0.0011) [2023-12-26 19:17:24,164][105692] Updated weights for policy 0, policy_version 538802 (0.0010) [2023-12-26 19:17:24,220][105620] Updated weights for policy 1, policy_version 539603 (0.0011) [2023-12-26 19:17:24,220][105692] Updated weights for policy 0, policy_version 538812 (0.0011) [2023-12-26 19:17:24,272][105620] Updated weights for policy 1, policy_version 539613 (0.0010) [2023-12-26 19:17:24,324][105620] Updated weights for policy 1, policy_version 539623 (0.0010) [2023-12-26 19:17:24,876][105692] Updated weights for policy 0, policy_version 538822 (0.0008) [2023-12-26 19:17:24,940][105692] Updated weights for policy 0, policy_version 538832 (0.0008) [2023-12-26 19:17:25,002][105692] Updated weights for policy 0, policy_version 538842 (0.0010) [2023-12-26 19:17:25,076][105620] Updated weights for policy 1, policy_version 539633 (0.0010) [2023-12-26 19:17:25,124][105620] Updated weights for policy 1, policy_version 539643 (0.0010) [2023-12-26 19:17:25,186][105620] Updated weights for policy 1, policy_version 539653 (0.0010) [2023-12-26 19:17:25,732][105692] Updated weights for policy 0, policy_version 538852 (0.0009) [2023-12-26 19:17:25,794][105692] Updated weights for policy 0, policy_version 538862 (0.0005) [2023-12-26 19:17:25,802][105620] Updated weights for policy 1, policy_version 539663 (0.0007) [2023-12-26 19:17:25,853][105692] Updated weights for policy 0, policy_version 538872 (0.0005) [2023-12-26 19:17:25,854][105620] Updated weights for policy 1, policy_version 539673 (0.0008) [2023-12-26 19:17:25,912][105620] Updated weights for policy 1, policy_version 539683 (0.0009) [2023-12-26 19:17:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 276144128. Throughput: 0: 9844.0, 1: 9652.5. Samples: 276147880. Policy #0 lag: (min: 18.0, avg: 29.6, max: 50.0) [2023-12-26 19:17:26,063][104569] Avg episode reward: [(0, '8814.414'), (1, '9082.436')] [2023-12-26 19:17:26,482][105692] Updated weights for policy 0, policy_version 538882 (0.0006) [2023-12-26 19:17:26,530][105692] Updated weights for policy 0, policy_version 538892 (0.0009) [2023-12-26 19:17:26,579][105692] Updated weights for policy 0, policy_version 538902 (0.0009) [2023-12-26 19:17:26,624][105692] Updated weights for policy 0, policy_version 538912 (0.0005) [2023-12-26 19:17:26,667][105620] Updated weights for policy 1, policy_version 539693 (0.0009) [2023-12-26 19:17:26,714][105620] Updated weights for policy 1, policy_version 539703 (0.0009) [2023-12-26 19:17:26,763][105620] Updated weights for policy 1, policy_version 539713 (0.0008) [2023-12-26 19:17:27,290][105692] Updated weights for policy 0, policy_version 538922 (0.0008) [2023-12-26 19:17:27,353][105692] Updated weights for policy 0, policy_version 538932 (0.0006) [2023-12-26 19:17:27,416][105692] Updated weights for policy 0, policy_version 538942 (0.0005) [2023-12-26 19:17:27,466][105620] Updated weights for policy 1, policy_version 539723 (0.0009) [2023-12-26 19:17:27,519][105620] Updated weights for policy 1, policy_version 539734 (0.0010) [2023-12-26 19:17:27,575][105620] Updated weights for policy 1, policy_version 539747 (0.0011) [2023-12-26 19:17:27,934][105692] Updated weights for policy 0, policy_version 538952 (0.0005) [2023-12-26 19:17:28,003][105692] Updated weights for policy 0, policy_version 538962 (0.0005) [2023-12-26 19:17:28,063][105692] Updated weights for policy 0, policy_version 538972 (0.0009) [2023-12-26 19:17:28,250][105620] Updated weights for policy 1, policy_version 539757 (0.0007) [2023-12-26 19:17:28,304][105620] Updated weights for policy 1, policy_version 539767 (0.0009) [2023-12-26 19:17:28,370][105620] Updated weights for policy 1, policy_version 539777 (0.0009) [2023-12-26 19:17:28,750][105692] Updated weights for policy 0, policy_version 538982 (0.0009) [2023-12-26 19:17:28,801][105692] Updated weights for policy 0, policy_version 538992 (0.0009) [2023-12-26 19:17:28,852][105692] Updated weights for policy 0, policy_version 539002 (0.0009) [2023-12-26 19:17:29,128][105620] Updated weights for policy 1, policy_version 539787 (0.0009) [2023-12-26 19:17:29,187][105620] Updated weights for policy 1, policy_version 539797 (0.0009) [2023-12-26 19:17:29,249][105620] Updated weights for policy 1, policy_version 539807 (0.0009) [2023-12-26 19:17:29,632][105692] Updated weights for policy 0, policy_version 539012 (0.0009) [2023-12-26 19:17:29,695][105692] Updated weights for policy 0, policy_version 539022 (0.0009) [2023-12-26 19:17:29,753][105692] Updated weights for policy 0, policy_version 539032 (0.0009) [2023-12-26 19:17:29,984][105620] Updated weights for policy 1, policy_version 539817 (0.0009) [2023-12-26 19:17:30,044][105620] Updated weights for policy 1, policy_version 539827 (0.0009) [2023-12-26 19:17:30,105][105620] Updated weights for policy 1, policy_version 539837 (0.0009) [2023-12-26 19:17:30,155][105620] Updated weights for policy 1, policy_version 539847 (0.0009) [2023-12-26 19:17:30,568][105692] Updated weights for policy 0, policy_version 539042 (0.0009) [2023-12-26 19:17:30,626][105692] Updated weights for policy 0, policy_version 539052 (0.0009) [2023-12-26 19:17:30,689][105692] Updated weights for policy 0, policy_version 539062 (0.0008) [2023-12-26 19:17:30,752][105692] Updated weights for policy 0, policy_version 539072 (0.0005) [2023-12-26 19:17:30,810][105620] Updated weights for policy 1, policy_version 539857 (0.0007) [2023-12-26 19:17:30,867][105620] Updated weights for policy 1, policy_version 539867 (0.0007) [2023-12-26 19:17:30,919][105620] Updated weights for policy 1, policy_version 539877 (0.0010) [2023-12-26 19:17:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 276242432. Throughput: 0: 9962.1, 1: 9673.7. Samples: 276210400. Policy #0 lag: (min: 18.0, avg: 29.6, max: 50.0) [2023-12-26 19:17:31,062][104569] Avg episode reward: [(0, '8815.156'), (1, '8992.778')] [2023-12-26 19:17:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000539072_138018816.pth... [2023-12-26 19:17:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000539880_138223616.pth... [2023-12-26 19:17:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000537920_137723904.pth [2023-12-26 19:17:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000538728_137928704.pth [2023-12-26 19:17:31,375][105692] Updated weights for policy 0, policy_version 539082 (0.0008) [2023-12-26 19:17:31,427][105692] Updated weights for policy 0, policy_version 539092 (0.0008) [2023-12-26 19:17:31,472][105692] Updated weights for policy 0, policy_version 539102 (0.0008) [2023-12-26 19:17:31,675][105620] Updated weights for policy 1, policy_version 539887 (0.0010) [2023-12-26 19:17:31,735][105620] Updated weights for policy 1, policy_version 539897 (0.0008) [2023-12-26 19:17:31,786][105620] Updated weights for policy 1, policy_version 539907 (0.0006) [2023-12-26 19:17:32,290][105692] Updated weights for policy 0, policy_version 539112 (0.0009) [2023-12-26 19:17:32,348][105692] Updated weights for policy 0, policy_version 539122 (0.0007) [2023-12-26 19:17:32,398][105692] Updated weights for policy 0, policy_version 539132 (0.0006) [2023-12-26 19:17:32,444][105620] Updated weights for policy 1, policy_version 539917 (0.0006) [2023-12-26 19:17:32,508][105620] Updated weights for policy 1, policy_version 539927 (0.0005) [2023-12-26 19:17:32,573][105620] Updated weights for policy 1, policy_version 539937 (0.0007) [2023-12-26 19:17:33,050][105692] Updated weights for policy 0, policy_version 539142 (0.0007) [2023-12-26 19:17:33,108][105692] Updated weights for policy 0, policy_version 539152 (0.0006) [2023-12-26 19:17:33,170][105620] Updated weights for policy 1, policy_version 539947 (0.0009) [2023-12-26 19:17:33,172][105692] Updated weights for policy 0, policy_version 539162 (0.0006) [2023-12-26 19:17:33,234][105620] Updated weights for policy 1, policy_version 539957 (0.0009) [2023-12-26 19:17:33,294][105620] Updated weights for policy 1, policy_version 539967 (0.0007) [2023-12-26 19:17:33,740][105692] Updated weights for policy 0, policy_version 539172 (0.0005) [2023-12-26 19:17:33,795][105692] Updated weights for policy 0, policy_version 539182 (0.0005) [2023-12-26 19:17:33,863][105692] Updated weights for policy 0, policy_version 539192 (0.0005) [2023-12-26 19:17:33,952][105620] Updated weights for policy 1, policy_version 539977 (0.0009) [2023-12-26 19:17:34,011][105620] Updated weights for policy 1, policy_version 539987 (0.0005) [2023-12-26 19:17:34,067][105620] Updated weights for policy 1, policy_version 539997 (0.0005) [2023-12-26 19:17:34,138][105620] Updated weights for policy 1, policy_version 540007 (0.0007) [2023-12-26 19:17:34,432][105692] Updated weights for policy 0, policy_version 539202 (0.0006) [2023-12-26 19:17:34,487][105692] Updated weights for policy 0, policy_version 539212 (0.0009) [2023-12-26 19:17:34,547][105692] Updated weights for policy 0, policy_version 539222 (0.0007) [2023-12-26 19:17:34,610][105692] Updated weights for policy 0, policy_version 539232 (0.0009) [2023-12-26 19:17:34,892][105620] Updated weights for policy 1, policy_version 540017 (0.0009) [2023-12-26 19:17:34,949][105620] Updated weights for policy 1, policy_version 540028 (0.0009) [2023-12-26 19:17:35,004][105620] Updated weights for policy 1, policy_version 540038 (0.0008) [2023-12-26 19:17:35,180][105692] Updated weights for policy 0, policy_version 539242 (0.0009) [2023-12-26 19:17:35,242][105692] Updated weights for policy 0, policy_version 539252 (0.0009) [2023-12-26 19:17:35,297][105692] Updated weights for policy 0, policy_version 539262 (0.0009) [2023-12-26 19:17:35,783][105620] Updated weights for policy 1, policy_version 540048 (0.0009) [2023-12-26 19:17:35,849][105620] Updated weights for policy 1, policy_version 540058 (0.0009) [2023-12-26 19:17:35,908][105620] Updated weights for policy 1, policy_version 540068 (0.0009) [2023-12-26 19:17:35,966][105692] Updated weights for policy 0, policy_version 539272 (0.0008) [2023-12-26 19:17:36,013][105692] Updated weights for policy 0, policy_version 539282 (0.0008) [2023-12-26 19:17:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 276340736. Throughput: 0: 10016.3, 1: 9632.0. Samples: 276330220. Policy #0 lag: (min: 18.0, avg: 29.6, max: 50.0) [2023-12-26 19:17:36,063][104569] Avg episode reward: [(0, '9173.557'), (1, '9173.437')] [2023-12-26 19:17:36,073][105692] Updated weights for policy 0, policy_version 539292 (0.0007) [2023-12-26 19:17:36,660][105620] Updated weights for policy 1, policy_version 540078 (0.0009) [2023-12-26 19:17:36,715][105620] Updated weights for policy 1, policy_version 540088 (0.0009) [2023-12-26 19:17:36,756][105692] Updated weights for policy 0, policy_version 539302 (0.0007) [2023-12-26 19:17:36,762][105620] Updated weights for policy 1, policy_version 540098 (0.0007) [2023-12-26 19:17:36,813][105692] Updated weights for policy 0, policy_version 539312 (0.0006) [2023-12-26 19:17:36,872][105692] Updated weights for policy 0, policy_version 539322 (0.0005) [2023-12-26 19:17:37,550][105620] Updated weights for policy 1, policy_version 540108 (0.0009) [2023-12-26 19:17:37,590][105692] Updated weights for policy 0, policy_version 539332 (0.0007) [2023-12-26 19:17:37,612][105620] Updated weights for policy 1, policy_version 540118 (0.0007) [2023-12-26 19:17:37,639][105692] Updated weights for policy 0, policy_version 539342 (0.0006) [2023-12-26 19:17:37,665][105620] Updated weights for policy 1, policy_version 540128 (0.0006) [2023-12-26 19:17:37,692][105692] Updated weights for policy 0, policy_version 539352 (0.0006) [2023-12-26 19:17:38,324][105620] Updated weights for policy 1, policy_version 540138 (0.0006) [2023-12-26 19:17:38,402][105620] Updated weights for policy 1, policy_version 540148 (0.0008) [2023-12-26 19:17:38,413][105692] Updated weights for policy 0, policy_version 539362 (0.0011) [2023-12-26 19:17:38,462][105620] Updated weights for policy 1, policy_version 540158 (0.0006) [2023-12-26 19:17:38,475][105692] Updated weights for policy 0, policy_version 539372 (0.0010) [2023-12-26 19:17:38,524][105620] Updated weights for policy 1, policy_version 540168 (0.0006) [2023-12-26 19:17:38,534][105692] Updated weights for policy 0, policy_version 539382 (0.0011) [2023-12-26 19:17:38,595][105692] Updated weights for policy 0, policy_version 539392 (0.0010) [2023-12-26 19:17:39,187][105620] Updated weights for policy 1, policy_version 540178 (0.0005) [2023-12-26 19:17:39,245][105620] Updated weights for policy 1, policy_version 540188 (0.0006) [2023-12-26 19:17:39,313][105620] Updated weights for policy 1, policy_version 540198 (0.0007) [2023-12-26 19:17:39,356][105692] Updated weights for policy 0, policy_version 539402 (0.0007) [2023-12-26 19:17:39,423][105692] Updated weights for policy 0, policy_version 539412 (0.0010) [2023-12-26 19:17:39,477][105692] Updated weights for policy 0, policy_version 539422 (0.0009) [2023-12-26 19:17:40,052][105620] Updated weights for policy 1, policy_version 540208 (0.0009) [2023-12-26 19:17:40,117][105620] Updated weights for policy 1, policy_version 540218 (0.0009) [2023-12-26 19:17:40,165][105692] Updated weights for policy 0, policy_version 539432 (0.0007) [2023-12-26 19:17:40,179][105620] Updated weights for policy 1, policy_version 540228 (0.0008) [2023-12-26 19:17:40,228][105692] Updated weights for policy 0, policy_version 539442 (0.0008) [2023-12-26 19:17:40,284][105692] Updated weights for policy 0, policy_version 539452 (0.0009) [2023-12-26 19:17:40,924][105692] Updated weights for policy 0, policy_version 539462 (0.0007) [2023-12-26 19:17:40,992][105692] Updated weights for policy 0, policy_version 539472 (0.0008) [2023-12-26 19:17:41,010][105620] Updated weights for policy 1, policy_version 540238 (0.0006) [2023-12-26 19:17:41,060][105692] Updated weights for policy 0, policy_version 539482 (0.0008) [2023-12-26 19:17:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 276430848. Throughput: 0: 10051.7, 1: 9642.8. Samples: 276446216. Policy #0 lag: (min: 18.0, avg: 29.6, max: 50.0) [2023-12-26 19:17:41,062][104569] Avg episode reward: [(0, '9171.610'), (1, '9175.255')] [2023-12-26 19:17:41,081][105620] Updated weights for policy 1, policy_version 540248 (0.0009) [2023-12-26 19:17:41,140][105620] Updated weights for policy 1, policy_version 540258 (0.0009) [2023-12-26 19:17:41,858][105692] Updated weights for policy 0, policy_version 539492 (0.0007) [2023-12-26 19:17:41,885][105620] Updated weights for policy 1, policy_version 540268 (0.0008) [2023-12-26 19:17:41,923][105692] Updated weights for policy 0, policy_version 539502 (0.0006) [2023-12-26 19:17:41,925][105585] KL-divergence is very high: 165.2378 [2023-12-26 19:17:41,945][105620] Updated weights for policy 1, policy_version 540278 (0.0008) [2023-12-26 19:17:41,973][105585] KL-divergence is very high: 293.1479 [2023-12-26 19:17:41,983][105692] Updated weights for policy 0, policy_version 539512 (0.0005) [2023-12-26 19:17:41,998][105620] Updated weights for policy 1, policy_version 540288 (0.0009) [2023-12-26 19:17:42,019][105585] KL-divergence is very high: 326.5959 [2023-12-26 19:17:42,661][105692] Updated weights for policy 0, policy_version 539522 (0.0007) [2023-12-26 19:17:42,716][105692] Updated weights for policy 0, policy_version 539532 (0.0009) [2023-12-26 19:17:42,767][105692] Updated weights for policy 0, policy_version 539542 (0.0009) [2023-12-26 19:17:42,774][105620] Updated weights for policy 1, policy_version 540298 (0.0008) [2023-12-26 19:17:42,817][105692] Updated weights for policy 0, policy_version 539552 (0.0006) [2023-12-26 19:17:42,831][105620] Updated weights for policy 1, policy_version 540308 (0.0007) [2023-12-26 19:17:42,888][105620] Updated weights for policy 1, policy_version 540318 (0.0008) [2023-12-26 19:17:42,951][105620] Updated weights for policy 1, policy_version 540328 (0.0009) [2023-12-26 19:17:43,573][105692] Updated weights for policy 0, policy_version 539562 (0.0010) [2023-12-26 19:17:43,621][105692] Updated weights for policy 0, policy_version 539572 (0.0010) [2023-12-26 19:17:43,679][105692] Updated weights for policy 0, policy_version 539582 (0.0010) [2023-12-26 19:17:43,709][105620] Updated weights for policy 1, policy_version 540338 (0.0010) [2023-12-26 19:17:43,767][105620] Updated weights for policy 1, policy_version 540348 (0.0008) [2023-12-26 19:17:43,821][105620] Updated weights for policy 1, policy_version 540358 (0.0008) [2023-12-26 19:17:44,301][105692] Updated weights for policy 0, policy_version 539592 (0.0010) [2023-12-26 19:17:44,346][105692] Updated weights for policy 0, policy_version 539602 (0.0010) [2023-12-26 19:17:44,393][105692] Updated weights for policy 0, policy_version 539612 (0.0010) [2023-12-26 19:17:44,642][105620] Updated weights for policy 1, policy_version 540368 (0.0008) [2023-12-26 19:17:44,699][105620] Updated weights for policy 1, policy_version 540378 (0.0008) [2023-12-26 19:17:44,748][105620] Updated weights for policy 1, policy_version 540388 (0.0006) [2023-12-26 19:17:45,156][105692] Updated weights for policy 0, policy_version 539622 (0.0010) [2023-12-26 19:17:45,216][105692] Updated weights for policy 0, policy_version 539632 (0.0010) [2023-12-26 19:17:45,283][105692] Updated weights for policy 0, policy_version 539642 (0.0010) [2023-12-26 19:17:45,489][105620] Updated weights for policy 1, policy_version 540398 (0.0009) [2023-12-26 19:17:45,554][105620] Updated weights for policy 1, policy_version 540408 (0.0009) [2023-12-26 19:17:45,608][105620] Updated weights for policy 1, policy_version 540418 (0.0008) [2023-12-26 19:17:45,963][105692] Updated weights for policy 0, policy_version 539652 (0.0010) [2023-12-26 19:17:46,014][105692] Updated weights for policy 0, policy_version 539662 (0.0009) [2023-12-26 19:17:46,061][105692] Updated weights for policy 0, policy_version 539672 (0.0009) [2023-12-26 19:17:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.9, 300 sec: 19494.2). Total num frames: 276529152. Throughput: 0: 9983.4, 1: 9665.5. Samples: 276502408. Policy #0 lag: (min: 18.0, avg: 29.6, max: 50.0) [2023-12-26 19:17:46,062][104569] Avg episode reward: [(0, '852.495'), (1, '8905.950')] [2023-12-26 19:17:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000540424_138362880.pth... [2023-12-26 19:17:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000539304_138076160.pth [2023-12-26 19:17:46,095][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000539680_138174464.pth... [2023-12-26 19:17:46,098][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000538496_137871360.pth [2023-12-26 19:17:46,362][105620] Updated weights for policy 1, policy_version 540428 (0.0007) [2023-12-26 19:17:46,425][105620] Updated weights for policy 1, policy_version 540438 (0.0005) [2023-12-26 19:17:46,486][105620] Updated weights for policy 1, policy_version 540448 (0.0005) [2023-12-26 19:17:46,811][105692] Updated weights for policy 0, policy_version 539682 (0.0009) [2023-12-26 19:17:46,858][105692] Updated weights for policy 0, policy_version 539692 (0.0008) [2023-12-26 19:17:46,905][105692] Updated weights for policy 0, policy_version 539702 (0.0008) [2023-12-26 19:17:46,952][105692] Updated weights for policy 0, policy_version 539712 (0.0009) [2023-12-26 19:17:47,089][105620] Updated weights for policy 1, policy_version 540458 (0.0006) [2023-12-26 19:17:47,143][105620] Updated weights for policy 1, policy_version 540468 (0.0009) [2023-12-26 19:17:47,196][105620] Updated weights for policy 1, policy_version 540478 (0.0008) [2023-12-26 19:17:47,242][105620] Updated weights for policy 1, policy_version 540488 (0.0008) [2023-12-26 19:17:47,702][105692] Updated weights for policy 0, policy_version 539722 (0.0006) [2023-12-26 19:17:47,748][105692] Updated weights for policy 0, policy_version 539732 (0.0008) [2023-12-26 19:17:47,798][105692] Updated weights for policy 0, policy_version 539742 (0.0009) [2023-12-26 19:17:48,046][105620] Updated weights for policy 1, policy_version 540498 (0.0009) [2023-12-26 19:17:48,107][105620] Updated weights for policy 1, policy_version 540508 (0.0009) [2023-12-26 19:17:48,177][105620] Updated weights for policy 1, policy_version 540518 (0.0009) [2023-12-26 19:17:48,516][105692] Updated weights for policy 0, policy_version 539752 (0.0009) [2023-12-26 19:17:48,568][105692] Updated weights for policy 0, policy_version 539762 (0.0009) [2023-12-26 19:17:48,622][105692] Updated weights for policy 0, policy_version 539772 (0.0009) [2023-12-26 19:17:48,935][105620] Updated weights for policy 1, policy_version 540528 (0.0008) [2023-12-26 19:17:48,999][105620] Updated weights for policy 1, policy_version 540538 (0.0009) [2023-12-26 19:17:49,059][105620] Updated weights for policy 1, policy_version 540548 (0.0008) [2023-12-26 19:17:49,397][105692] Updated weights for policy 0, policy_version 539782 (0.0010) [2023-12-26 19:17:49,453][105692] Updated weights for policy 0, policy_version 539792 (0.0006) [2023-12-26 19:17:49,499][105692] Updated weights for policy 0, policy_version 539802 (0.0005) [2023-12-26 19:17:49,852][105620] Updated weights for policy 1, policy_version 540558 (0.0010) [2023-12-26 19:17:49,913][105620] Updated weights for policy 1, policy_version 540568 (0.0008) [2023-12-26 19:17:49,974][105620] Updated weights for policy 1, policy_version 540578 (0.0008) [2023-12-26 19:17:50,173][105692] Updated weights for policy 0, policy_version 539812 (0.0006) [2023-12-26 19:17:50,222][105692] Updated weights for policy 0, policy_version 539822 (0.0008) [2023-12-26 19:17:50,282][105692] Updated weights for policy 0, policy_version 539832 (0.0008) [2023-12-26 19:17:50,699][105620] Updated weights for policy 1, policy_version 540588 (0.0010) [2023-12-26 19:17:50,751][105620] Updated weights for policy 1, policy_version 540598 (0.0008) [2023-12-26 19:17:50,800][105620] Updated weights for policy 1, policy_version 540608 (0.0010) [2023-12-26 19:17:51,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 276627456. Throughput: 0: 9976.4, 1: 9548.3. Samples: 276617212. Policy #0 lag: (min: 18.0, avg: 29.6, max: 50.0) [2023-12-26 19:17:51,063][104569] Avg episode reward: [(0, '1272.288'), (1, '8994.358')] [2023-12-26 19:17:51,110][105692] Updated weights for policy 0, policy_version 539842 (0.0008) [2023-12-26 19:17:51,169][105692] Updated weights for policy 0, policy_version 539852 (0.0008) [2023-12-26 19:17:51,225][105692] Updated weights for policy 0, policy_version 539862 (0.0008) [2023-12-26 19:17:51,284][105692] Updated weights for policy 0, policy_version 539872 (0.0008) [2023-12-26 19:17:51,462][105620] Updated weights for policy 1, policy_version 540618 (0.0010) [2023-12-26 19:17:51,526][105620] Updated weights for policy 1, policy_version 540628 (0.0008) [2023-12-26 19:17:51,589][105620] Updated weights for policy 1, policy_version 540638 (0.0007) [2023-12-26 19:17:51,655][105620] Updated weights for policy 1, policy_version 540648 (0.0008) [2023-12-26 19:17:52,070][105692] Updated weights for policy 0, policy_version 539882 (0.0005) [2023-12-26 19:17:52,116][105692] Updated weights for policy 0, policy_version 539892 (0.0005) [2023-12-26 19:17:52,164][105692] Updated weights for policy 0, policy_version 539902 (0.0005) [2023-12-26 19:17:52,390][105620] Updated weights for policy 1, policy_version 540658 (0.0008) [2023-12-26 19:17:52,460][105620] Updated weights for policy 1, policy_version 540668 (0.0007) [2023-12-26 19:17:52,522][105620] Updated weights for policy 1, policy_version 540678 (0.0007) [2023-12-26 19:17:52,764][105692] Updated weights for policy 0, policy_version 539912 (0.0007) [2023-12-26 19:17:52,819][105692] Updated weights for policy 0, policy_version 539922 (0.0009) [2023-12-26 19:17:52,884][105692] Updated weights for policy 0, policy_version 539932 (0.0009) [2023-12-26 19:17:53,303][105620] Updated weights for policy 1, policy_version 540688 (0.0007) [2023-12-26 19:17:53,363][105620] Updated weights for policy 1, policy_version 540698 (0.0005) [2023-12-26 19:17:53,418][105620] Updated weights for policy 1, policy_version 540708 (0.0008) [2023-12-26 19:17:53,501][105692] Updated weights for policy 0, policy_version 539942 (0.0006) [2023-12-26 19:17:53,558][105692] Updated weights for policy 0, policy_version 539952 (0.0006) [2023-12-26 19:17:53,612][105692] Updated weights for policy 0, policy_version 539962 (0.0005) [2023-12-26 19:17:54,043][105620] Updated weights for policy 1, policy_version 540718 (0.0010) [2023-12-26 19:17:54,102][105620] Updated weights for policy 1, policy_version 540728 (0.0008) [2023-12-26 19:17:54,161][105620] Updated weights for policy 1, policy_version 540738 (0.0008) [2023-12-26 19:17:54,259][105692] Updated weights for policy 0, policy_version 539972 (0.0007) [2023-12-26 19:17:54,322][105692] Updated weights for policy 0, policy_version 539982 (0.0010) [2023-12-26 19:17:54,384][105692] Updated weights for policy 0, policy_version 539992 (0.0010) [2023-12-26 19:17:54,805][105620] Updated weights for policy 1, policy_version 540748 (0.0008) [2023-12-26 19:17:54,853][105620] Updated weights for policy 1, policy_version 540758 (0.0006) [2023-12-26 19:17:54,903][105620] Updated weights for policy 1, policy_version 540768 (0.0005) [2023-12-26 19:17:55,086][105692] Updated weights for policy 0, policy_version 540002 (0.0011) [2023-12-26 19:17:55,135][105692] Updated weights for policy 0, policy_version 540012 (0.0011) [2023-12-26 19:17:55,187][105692] Updated weights for policy 0, policy_version 540022 (0.0010) [2023-12-26 19:17:55,240][105692] Updated weights for policy 0, policy_version 540032 (0.0011) [2023-12-26 19:17:55,627][105620] Updated weights for policy 1, policy_version 540778 (0.0008) [2023-12-26 19:17:55,675][105620] Updated weights for policy 1, policy_version 540788 (0.0008) [2023-12-26 19:17:55,731][105620] Updated weights for policy 1, policy_version 540798 (0.0008) [2023-12-26 19:17:55,780][105620] Updated weights for policy 1, policy_version 540808 (0.0008) [2023-12-26 19:17:56,003][105692] Updated weights for policy 0, policy_version 540042 (0.0005) [2023-12-26 19:17:56,055][105692] Updated weights for policy 0, policy_version 540052 (0.0005) [2023-12-26 19:17:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 276725760. Throughput: 0: 9957.7, 1: 9519.1. Samples: 276735792. Policy #0 lag: (min: 18.0, avg: 29.6, max: 50.0) [2023-12-26 19:17:56,062][104569] Avg episode reward: [(0, '5990.192'), (1, '8991.708')] [2023-12-26 19:17:56,117][105692] Updated weights for policy 0, policy_version 540062 (0.0007) [2023-12-26 19:17:56,594][105620] Updated weights for policy 1, policy_version 540818 (0.0007) [2023-12-26 19:17:56,659][105620] Updated weights for policy 1, policy_version 540828 (0.0009) [2023-12-26 19:17:56,716][105620] Updated weights for policy 1, policy_version 540838 (0.0009) [2023-12-26 19:17:56,804][105692] Updated weights for policy 0, policy_version 540072 (0.0009) [2023-12-26 19:17:56,850][105692] Updated weights for policy 0, policy_version 540082 (0.0008) [2023-12-26 19:17:56,904][105692] Updated weights for policy 0, policy_version 540092 (0.0009) [2023-12-26 19:17:57,416][105620] Updated weights for policy 1, policy_version 540848 (0.0006) [2023-12-26 19:17:57,474][105620] Updated weights for policy 1, policy_version 540858 (0.0006) [2023-12-26 19:17:57,531][105620] Updated weights for policy 1, policy_version 540868 (0.0008) [2023-12-26 19:17:57,710][105692] Updated weights for policy 0, policy_version 540102 (0.0007) [2023-12-26 19:17:57,768][105692] Updated weights for policy 0, policy_version 540112 (0.0005) [2023-12-26 19:17:57,828][105692] Updated weights for policy 0, policy_version 540122 (0.0005) [2023-12-26 19:17:58,262][105620] Updated weights for policy 1, policy_version 540878 (0.0009) [2023-12-26 19:17:58,317][105620] Updated weights for policy 1, policy_version 540888 (0.0008) [2023-12-26 19:17:58,385][105620] Updated weights for policy 1, policy_version 540898 (0.0008) [2023-12-26 19:17:58,573][105692] Updated weights for policy 0, policy_version 540132 (0.0009) [2023-12-26 19:17:58,637][105692] Updated weights for policy 0, policy_version 540142 (0.0011) [2023-12-26 19:17:58,700][105692] Updated weights for policy 0, policy_version 540152 (0.0011) [2023-12-26 19:17:59,148][105620] Updated weights for policy 1, policy_version 540908 (0.0007) [2023-12-26 19:17:59,195][105620] Updated weights for policy 1, policy_version 540918 (0.0007) [2023-12-26 19:17:59,255][105620] Updated weights for policy 1, policy_version 540928 (0.0007) [2023-12-26 19:17:59,465][105692] Updated weights for policy 0, policy_version 540162 (0.0008) [2023-12-26 19:17:59,524][105692] Updated weights for policy 0, policy_version 540172 (0.0010) [2023-12-26 19:17:59,585][105692] Updated weights for policy 0, policy_version 540182 (0.0009) [2023-12-26 19:17:59,647][105692] Updated weights for policy 0, policy_version 540192 (0.0008) [2023-12-26 19:17:59,932][105620] Updated weights for policy 1, policy_version 540938 (0.0009) [2023-12-26 19:17:59,999][105620] Updated weights for policy 1, policy_version 540948 (0.0009) [2023-12-26 19:18:00,060][105620] Updated weights for policy 1, policy_version 540958 (0.0009) [2023-12-26 19:18:00,107][105620] Updated weights for policy 1, policy_version 540968 (0.0009) [2023-12-26 19:18:00,400][105692] Updated weights for policy 0, policy_version 540202 (0.0009) [2023-12-26 19:18:00,451][105692] Updated weights for policy 0, policy_version 540212 (0.0005) [2023-12-26 19:18:00,505][105692] Updated weights for policy 0, policy_version 540222 (0.0010) [2023-12-26 19:18:00,925][105620] Updated weights for policy 1, policy_version 540978 (0.0010) [2023-12-26 19:18:00,977][105620] Updated weights for policy 1, policy_version 540989 (0.0010) [2023-12-26 19:18:01,030][105620] Updated weights for policy 1, policy_version 541000 (0.0009) [2023-12-26 19:18:01,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 276824064. Throughput: 0: 9965.7, 1: 9520.6. Samples: 276792496. Policy #0 lag: (min: 18.0, avg: 29.6, max: 50.0) [2023-12-26 19:18:01,062][104569] Avg episode reward: [(0, '7016.672'), (1, '8995.312')] [2023-12-26 19:18:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000541000_138510336.pth... [2023-12-26 19:18:01,068][105692] Updated weights for policy 0, policy_version 540232 (0.0008) [2023-12-26 19:18:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000539880_138223616.pth [2023-12-26 19:18:01,133][105692] Updated weights for policy 0, policy_version 540242 (0.0011) [2023-12-26 19:18:01,197][105692] Updated weights for policy 0, policy_version 540252 (0.0009) [2023-12-26 19:18:01,222][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000540256_138321920.pth... [2023-12-26 19:18:01,226][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000539072_138018816.pth [2023-12-26 19:18:01,797][105692] Updated weights for policy 0, policy_version 540262 (0.0010) [2023-12-26 19:18:01,871][105692] Updated weights for policy 0, policy_version 540273 (0.0011) [2023-12-26 19:18:01,899][105620] Updated weights for policy 1, policy_version 541010 (0.0008) [2023-12-26 19:18:01,929][105692] Updated weights for policy 0, policy_version 540283 (0.0010) [2023-12-26 19:18:01,956][105620] Updated weights for policy 1, policy_version 541020 (0.0009) [2023-12-26 19:18:02,014][105620] Updated weights for policy 1, policy_version 541030 (0.0009) [2023-12-26 19:18:02,554][105692] Updated weights for policy 0, policy_version 540293 (0.0011) [2023-12-26 19:18:02,593][105585] KL-divergence is very high: 162.7834 [2023-12-26 19:18:02,615][105692] Updated weights for policy 0, policy_version 540303 (0.0007) [2023-12-26 19:18:02,650][105585] KL-divergence is very high: 241.7361 [2023-12-26 19:18:02,676][105692] Updated weights for policy 0, policy_version 540313 (0.0010) [2023-12-26 19:18:02,691][105585] KL-divergence is very high: 202.9813 [2023-12-26 19:18:02,815][105620] Updated weights for policy 1, policy_version 541040 (0.0006) [2023-12-26 19:18:02,883][105620] Updated weights for policy 1, policy_version 541050 (0.0005) [2023-12-26 19:18:02,943][105620] Updated weights for policy 1, policy_version 541060 (0.0005) [2023-12-26 19:18:03,341][105692] Updated weights for policy 0, policy_version 540323 (0.0010) [2023-12-26 19:18:03,408][105692] Updated weights for policy 0, policy_version 540333 (0.0005) [2023-12-26 19:18:03,468][105692] Updated weights for policy 0, policy_version 540343 (0.0007) [2023-12-26 19:18:03,532][105620] Updated weights for policy 1, policy_version 541070 (0.0007) [2023-12-26 19:18:03,583][105620] Updated weights for policy 1, policy_version 541080 (0.0005) [2023-12-26 19:18:03,631][105620] Updated weights for policy 1, policy_version 541090 (0.0005) [2023-12-26 19:18:04,001][105692] Updated weights for policy 0, policy_version 540353 (0.0009) [2023-12-26 19:18:04,066][105692] Updated weights for policy 0, policy_version 540363 (0.0011) [2023-12-26 19:18:04,129][105692] Updated weights for policy 0, policy_version 540373 (0.0009) [2023-12-26 19:18:04,199][105692] Updated weights for policy 0, policy_version 540383 (0.0007) [2023-12-26 19:18:04,355][105620] Updated weights for policy 1, policy_version 541100 (0.0007) [2023-12-26 19:18:04,418][105620] Updated weights for policy 1, policy_version 541110 (0.0011) [2023-12-26 19:18:04,484][105620] Updated weights for policy 1, policy_version 541120 (0.0011) [2023-12-26 19:18:04,799][105692] Updated weights for policy 0, policy_version 540393 (0.0005) [2023-12-26 19:18:04,850][105692] Updated weights for policy 0, policy_version 540403 (0.0005) [2023-12-26 19:18:04,897][105692] Updated weights for policy 0, policy_version 540413 (0.0007) [2023-12-26 19:18:05,191][105620] Updated weights for policy 1, policy_version 541130 (0.0010) [2023-12-26 19:18:05,252][105620] Updated weights for policy 1, policy_version 541140 (0.0006) [2023-12-26 19:18:05,322][105620] Updated weights for policy 1, policy_version 541150 (0.0005) [2023-12-26 19:18:05,383][105620] Updated weights for policy 1, policy_version 541160 (0.0008) [2023-12-26 19:18:05,592][105692] Updated weights for policy 0, policy_version 540423 (0.0008) [2023-12-26 19:18:05,652][105692] Updated weights for policy 0, policy_version 540433 (0.0008) [2023-12-26 19:18:05,707][105692] Updated weights for policy 0, policy_version 540443 (0.0007) [2023-12-26 19:18:06,057][105620] Updated weights for policy 1, policy_version 541170 (0.0010) [2023-12-26 19:18:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 276922368. Throughput: 0: 10041.4, 1: 9546.7. Samples: 276913068. Policy #0 lag: (min: 18.0, avg: 29.6, max: 50.0) [2023-12-26 19:18:06,062][104569] Avg episode reward: [(0, '8215.447'), (1, '8903.061')] [2023-12-26 19:18:06,117][105620] Updated weights for policy 1, policy_version 541180 (0.0011) [2023-12-26 19:18:06,184][105620] Updated weights for policy 1, policy_version 541190 (0.0009) [2023-12-26 19:18:06,485][105692] Updated weights for policy 0, policy_version 540453 (0.0008) [2023-12-26 19:18:06,549][105692] Updated weights for policy 0, policy_version 540463 (0.0009) [2023-12-26 19:18:06,614][105692] Updated weights for policy 0, policy_version 540473 (0.0009) [2023-12-26 19:18:06,953][105620] Updated weights for policy 1, policy_version 541200 (0.0011) [2023-12-26 19:18:07,013][105620] Updated weights for policy 1, policy_version 541210 (0.0010) [2023-12-26 19:18:07,071][105620] Updated weights for policy 1, policy_version 541220 (0.0010) [2023-12-26 19:18:07,378][105692] Updated weights for policy 0, policy_version 540483 (0.0008) [2023-12-26 19:18:07,425][105692] Updated weights for policy 0, policy_version 540493 (0.0009) [2023-12-26 19:18:07,482][105692] Updated weights for policy 0, policy_version 540503 (0.0009) [2023-12-26 19:18:07,768][105620] Updated weights for policy 1, policy_version 541230 (0.0009) [2023-12-26 19:18:07,827][105620] Updated weights for policy 1, policy_version 541240 (0.0007) [2023-12-26 19:18:07,895][105620] Updated weights for policy 1, policy_version 541250 (0.0005) [2023-12-26 19:18:08,190][105692] Updated weights for policy 0, policy_version 540513 (0.0008) [2023-12-26 19:18:08,241][105692] Updated weights for policy 0, policy_version 540523 (0.0009) [2023-12-26 19:18:08,295][105692] Updated weights for policy 0, policy_version 540533 (0.0010) [2023-12-26 19:18:08,354][105692] Updated weights for policy 0, policy_version 540543 (0.0007) [2023-12-26 19:18:08,453][105620] Updated weights for policy 1, policy_version 541260 (0.0005) [2023-12-26 19:18:08,508][105620] Updated weights for policy 1, policy_version 541270 (0.0005) [2023-12-26 19:18:08,563][105620] Updated weights for policy 1, policy_version 541280 (0.0006) [2023-12-26 19:18:09,133][105692] Updated weights for policy 0, policy_version 540553 (0.0010) [2023-12-26 19:18:09,185][105692] Updated weights for policy 0, policy_version 540563 (0.0010) [2023-12-26 19:18:09,234][105620] Updated weights for policy 1, policy_version 541290 (0.0006) [2023-12-26 19:18:09,250][105692] Updated weights for policy 0, policy_version 540573 (0.0009) [2023-12-26 19:18:09,294][105620] Updated weights for policy 1, policy_version 541300 (0.0008) [2023-12-26 19:18:09,361][105620] Updated weights for policy 1, policy_version 541310 (0.0008) [2023-12-26 19:18:09,431][105620] Updated weights for policy 1, policy_version 541320 (0.0008) [2023-12-26 19:18:10,039][105692] Updated weights for policy 0, policy_version 540583 (0.0011) [2023-12-26 19:18:10,051][105585] KL-divergence is very high: 137.2097 [2023-12-26 19:18:10,098][105585] KL-divergence is very high: 238.9936 [2023-12-26 19:18:10,098][105692] Updated weights for policy 0, policy_version 540593 (0.0011) [2023-12-26 19:18:10,139][105585] KL-divergence is very high: 236.7814 [2023-12-26 19:18:10,150][105692] Updated weights for policy 0, policy_version 540603 (0.0011) [2023-12-26 19:18:10,197][105620] Updated weights for policy 1, policy_version 541330 (0.0007) [2023-12-26 19:18:10,260][105620] Updated weights for policy 1, policy_version 541340 (0.0008) [2023-12-26 19:18:10,326][105620] Updated weights for policy 1, policy_version 541350 (0.0009) [2023-12-26 19:18:10,858][105692] Updated weights for policy 0, policy_version 540613 (0.0011) [2023-12-26 19:18:10,932][105692] Updated weights for policy 0, policy_version 540623 (0.0011) [2023-12-26 19:18:10,995][105692] Updated weights for policy 0, policy_version 540633 (0.0009) [2023-12-26 19:18:11,029][105620] Updated weights for policy 1, policy_version 541360 (0.0008) [2023-12-26 19:18:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 277020672. Throughput: 0: 9987.8, 1: 9608.1. Samples: 277029696. Policy #0 lag: (min: 18.0, avg: 29.6, max: 50.0) [2023-12-26 19:18:11,063][104569] Avg episode reward: [(0, '8311.822'), (1, '8991.655')] [2023-12-26 19:18:11,093][105620] Updated weights for policy 1, policy_version 541370 (0.0008) [2023-12-26 19:18:11,162][105620] Updated weights for policy 1, policy_version 541380 (0.0008) [2023-12-26 19:18:11,727][105692] Updated weights for policy 0, policy_version 540643 (0.0010) [2023-12-26 19:18:11,793][105692] Updated weights for policy 0, policy_version 540653 (0.0009) [2023-12-26 19:18:11,860][105692] Updated weights for policy 0, policy_version 540663 (0.0008) [2023-12-26 19:18:11,950][105620] Updated weights for policy 1, policy_version 541390 (0.0009) [2023-12-26 19:18:12,009][105620] Updated weights for policy 1, policy_version 541400 (0.0010) [2023-12-26 19:18:12,071][105620] Updated weights for policy 1, policy_version 541410 (0.0008) [2023-12-26 19:18:12,596][105692] Updated weights for policy 0, policy_version 540673 (0.0009) [2023-12-26 19:18:12,658][105692] Updated weights for policy 0, policy_version 540683 (0.0006) [2023-12-26 19:18:12,719][105692] Updated weights for policy 0, policy_version 540693 (0.0008) [2023-12-26 19:18:12,787][105692] Updated weights for policy 0, policy_version 540703 (0.0008) [2023-12-26 19:18:12,872][105620] Updated weights for policy 1, policy_version 541420 (0.0009) [2023-12-26 19:18:12,927][105620] Updated weights for policy 1, policy_version 541430 (0.0009) [2023-12-26 19:18:12,992][105620] Updated weights for policy 1, policy_version 541440 (0.0010) [2023-12-26 19:18:13,489][105692] Updated weights for policy 0, policy_version 540713 (0.0009) [2023-12-26 19:18:13,558][105692] Updated weights for policy 0, policy_version 540723 (0.0008) [2023-12-26 19:18:13,627][105692] Updated weights for policy 0, policy_version 540733 (0.0006) [2023-12-26 19:18:13,686][105620] Updated weights for policy 1, policy_version 541450 (0.0009) [2023-12-26 19:18:13,742][105620] Updated weights for policy 1, policy_version 541460 (0.0009) [2023-12-26 19:18:13,792][105620] Updated weights for policy 1, policy_version 541470 (0.0007) [2023-12-26 19:18:13,851][105620] Updated weights for policy 1, policy_version 541480 (0.0005) [2023-12-26 19:18:14,293][105692] Updated weights for policy 0, policy_version 540743 (0.0008) [2023-12-26 19:18:14,338][105692] Updated weights for policy 0, policy_version 540753 (0.0008) [2023-12-26 19:18:14,394][105692] Updated weights for policy 0, policy_version 540763 (0.0008) [2023-12-26 19:18:14,541][105620] Updated weights for policy 1, policy_version 541490 (0.0010) [2023-12-26 19:18:14,603][105620] Updated weights for policy 1, policy_version 541500 (0.0010) [2023-12-26 19:18:14,657][105620] Updated weights for policy 1, policy_version 541510 (0.0010) [2023-12-26 19:18:15,163][105692] Updated weights for policy 0, policy_version 540773 (0.0009) [2023-12-26 19:18:15,216][105692] Updated weights for policy 0, policy_version 540783 (0.0008) [2023-12-26 19:18:15,279][105692] Updated weights for policy 0, policy_version 540793 (0.0008) [2023-12-26 19:18:15,456][105620] Updated weights for policy 1, policy_version 541520 (0.0010) [2023-12-26 19:18:15,508][105620] Updated weights for policy 1, policy_version 541530 (0.0010) [2023-12-26 19:18:15,564][105620] Updated weights for policy 1, policy_version 541540 (0.0010) [2023-12-26 19:18:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 277110784. Throughput: 0: 9876.5, 1: 9562.7. Samples: 277085164. Policy #0 lag: (min: 18.0, avg: 29.6, max: 50.0) [2023-12-26 19:18:16,062][104569] Avg episode reward: [(0, '8653.555'), (1, '9092.107')] [2023-12-26 19:18:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000541544_138649600.pth... [2023-12-26 19:18:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000540424_138362880.pth [2023-12-26 19:18:16,076][105692] Updated weights for policy 0, policy_version 540803 (0.0008) [2023-12-26 19:18:16,139][105692] Updated weights for policy 0, policy_version 540813 (0.0009) [2023-12-26 19:18:16,191][105692] Updated weights for policy 0, policy_version 540823 (0.0009) [2023-12-26 19:18:16,232][105620] Updated weights for policy 1, policy_version 541550 (0.0007) [2023-12-26 19:18:16,236][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000540832_138469376.pth... [2023-12-26 19:18:16,241][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000539680_138174464.pth [2023-12-26 19:18:16,301][105620] Updated weights for policy 1, policy_version 541560 (0.0005) [2023-12-26 19:18:16,361][105620] Updated weights for policy 1, policy_version 541570 (0.0005) [2023-12-26 19:18:16,841][105692] Updated weights for policy 0, policy_version 540833 (0.0009) [2023-12-26 19:18:16,895][105692] Updated weights for policy 0, policy_version 540843 (0.0010) [2023-12-26 19:18:16,949][105692] Updated weights for policy 0, policy_version 540853 (0.0009) [2023-12-26 19:18:16,960][105620] Updated weights for policy 1, policy_version 541580 (0.0005) [2023-12-26 19:18:17,003][105692] Updated weights for policy 0, policy_version 540863 (0.0008) [2023-12-26 19:18:17,013][105620] Updated weights for policy 1, policy_version 541590 (0.0007) [2023-12-26 19:18:17,063][105620] Updated weights for policy 1, policy_version 541600 (0.0009) [2023-12-26 19:18:17,637][105620] Updated weights for policy 1, policy_version 541610 (0.0007) [2023-12-26 19:18:17,693][105620] Updated weights for policy 1, policy_version 541620 (0.0005) [2023-12-26 19:18:17,747][105620] Updated weights for policy 1, policy_version 541630 (0.0008) [2023-12-26 19:18:17,799][105620] Updated weights for policy 1, policy_version 541640 (0.0010) [2023-12-26 19:18:17,870][105692] Updated weights for policy 0, policy_version 540873 (0.0008) [2023-12-26 19:18:17,918][105692] Updated weights for policy 0, policy_version 540883 (0.0008) [2023-12-26 19:18:17,977][105692] Updated weights for policy 0, policy_version 540893 (0.0008) [2023-12-26 19:18:18,436][105620] Updated weights for policy 1, policy_version 541650 (0.0006) [2023-12-26 19:18:18,491][105620] Updated weights for policy 1, policy_version 541660 (0.0006) [2023-12-26 19:18:18,543][105620] Updated weights for policy 1, policy_version 541670 (0.0005) [2023-12-26 19:18:18,844][105692] Updated weights for policy 0, policy_version 540903 (0.0009) [2023-12-26 19:18:18,906][105692] Updated weights for policy 0, policy_version 540913 (0.0010) [2023-12-26 19:18:18,960][105692] Updated weights for policy 0, policy_version 540923 (0.0010) [2023-12-26 19:18:19,112][105620] Updated weights for policy 1, policy_version 541680 (0.0009) [2023-12-26 19:18:19,171][105620] Updated weights for policy 1, policy_version 541690 (0.0010) [2023-12-26 19:18:19,233][105620] Updated weights for policy 1, policy_version 541700 (0.0011) [2023-12-26 19:18:19,836][105692] Updated weights for policy 0, policy_version 540934 (0.0009) [2023-12-26 19:18:19,907][105692] Updated weights for policy 0, policy_version 540944 (0.0005) [2023-12-26 19:18:19,939][105620] Updated weights for policy 1, policy_version 541710 (0.0007) [2023-12-26 19:18:19,975][105692] Updated weights for policy 0, policy_version 540954 (0.0006) [2023-12-26 19:18:20,015][105620] Updated weights for policy 1, policy_version 541720 (0.0008) [2023-12-26 19:18:20,089][105620] Updated weights for policy 1, policy_version 541730 (0.0007) [2023-12-26 19:18:20,531][105692] Updated weights for policy 0, policy_version 540964 (0.0007) [2023-12-26 19:18:20,595][105692] Updated weights for policy 0, policy_version 540974 (0.0008) [2023-12-26 19:18:20,659][105692] Updated weights for policy 0, policy_version 540984 (0.0006) [2023-12-26 19:18:20,842][105620] Updated weights for policy 1, policy_version 541740 (0.0008) [2023-12-26 19:18:20,912][105620] Updated weights for policy 1, policy_version 541750 (0.0009) [2023-12-26 19:18:20,975][105620] Updated weights for policy 1, policy_version 541760 (0.0009) [2023-12-26 19:18:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 277217280. Throughput: 0: 9757.7, 1: 9639.7. Samples: 277203104. Policy #0 lag: (min: 18.0, avg: 29.6, max: 50.0) [2023-12-26 19:18:21,062][104569] Avg episode reward: [(0, '8653.582'), (1, '8918.930')] [2023-12-26 19:18:21,352][105692] Updated weights for policy 0, policy_version 540994 (0.0006) [2023-12-26 19:18:21,414][105692] Updated weights for policy 0, policy_version 541004 (0.0010) [2023-12-26 19:18:21,468][105692] Updated weights for policy 0, policy_version 541014 (0.0007) [2023-12-26 19:18:21,529][105692] Updated weights for policy 0, policy_version 541024 (0.0007) [2023-12-26 19:18:21,767][105620] Updated weights for policy 1, policy_version 541770 (0.0008) [2023-12-26 19:18:21,837][105620] Updated weights for policy 1, policy_version 541780 (0.0009) [2023-12-26 19:18:21,901][105620] Updated weights for policy 1, policy_version 541790 (0.0009) [2023-12-26 19:18:21,975][105620] Updated weights for policy 1, policy_version 541800 (0.0010) [2023-12-26 19:18:22,169][105692] Updated weights for policy 0, policy_version 541034 (0.0006) [2023-12-26 19:18:22,223][105692] Updated weights for policy 0, policy_version 541044 (0.0006) [2023-12-26 19:18:22,287][105692] Updated weights for policy 0, policy_version 541054 (0.0008) [2023-12-26 19:18:22,707][105620] Updated weights for policy 1, policy_version 541810 (0.0006) [2023-12-26 19:18:22,773][105620] Updated weights for policy 1, policy_version 541820 (0.0006) [2023-12-26 19:18:22,829][105620] Updated weights for policy 1, policy_version 541830 (0.0008) [2023-12-26 19:18:23,031][105692] Updated weights for policy 0, policy_version 541064 (0.0008) [2023-12-26 19:18:23,103][105692] Updated weights for policy 0, policy_version 541074 (0.0009) [2023-12-26 19:18:23,172][105692] Updated weights for policy 0, policy_version 541084 (0.0008) [2023-12-26 19:18:23,455][105620] Updated weights for policy 1, policy_version 541840 (0.0008) [2023-12-26 19:18:23,501][105620] Updated weights for policy 1, policy_version 541850 (0.0008) [2023-12-26 19:18:23,547][105620] Updated weights for policy 1, policy_version 541860 (0.0008) [2023-12-26 19:18:23,759][105692] Updated weights for policy 0, policy_version 541094 (0.0006) [2023-12-26 19:18:23,804][105692] Updated weights for policy 0, policy_version 541104 (0.0005) [2023-12-26 19:18:23,850][105692] Updated weights for policy 0, policy_version 541114 (0.0005) [2023-12-26 19:18:24,289][105620] Updated weights for policy 1, policy_version 541870 (0.0006) [2023-12-26 19:18:24,339][105620] Updated weights for policy 1, policy_version 541880 (0.0006) [2023-12-26 19:18:24,388][105620] Updated weights for policy 1, policy_version 541890 (0.0005) [2023-12-26 19:18:24,566][105692] Updated weights for policy 0, policy_version 541124 (0.0010) [2023-12-26 19:18:24,623][105692] Updated weights for policy 0, policy_version 541134 (0.0010) [2023-12-26 19:18:24,684][105692] Updated weights for policy 0, policy_version 541144 (0.0010) [2023-12-26 19:18:24,989][105620] Updated weights for policy 1, policy_version 541900 (0.0005) [2023-12-26 19:18:25,049][105620] Updated weights for policy 1, policy_version 541910 (0.0007) [2023-12-26 19:18:25,093][105620] Updated weights for policy 1, policy_version 541920 (0.0008) [2023-12-26 19:18:25,411][105692] Updated weights for policy 0, policy_version 541154 (0.0010) [2023-12-26 19:18:25,471][105692] Updated weights for policy 0, policy_version 541164 (0.0008) [2023-12-26 19:18:25,532][105692] Updated weights for policy 0, policy_version 541174 (0.0007) [2023-12-26 19:18:25,585][105692] Updated weights for policy 0, policy_version 541184 (0.0005) [2023-12-26 19:18:25,774][105620] Updated weights for policy 1, policy_version 541930 (0.0008) [2023-12-26 19:18:25,824][105620] Updated weights for policy 1, policy_version 541940 (0.0005) [2023-12-26 19:18:25,879][105620] Updated weights for policy 1, policy_version 541950 (0.0006) [2023-12-26 19:18:25,937][105620] Updated weights for policy 1, policy_version 541960 (0.0007) [2023-12-26 19:18:26,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 277315584. Throughput: 0: 9772.5, 1: 9702.0. Samples: 277322568. Policy #0 lag: (min: 18.0, avg: 29.6, max: 50.0) [2023-12-26 19:18:26,062][104569] Avg episode reward: [(0, '9358.894'), (1, '8840.847')] [2023-12-26 19:18:26,246][105692] Updated weights for policy 0, policy_version 541194 (0.0010) [2023-12-26 19:18:26,298][105692] Updated weights for policy 0, policy_version 541204 (0.0010) [2023-12-26 19:18:26,363][105692] Updated weights for policy 0, policy_version 541214 (0.0006) [2023-12-26 19:18:26,710][105620] Updated weights for policy 1, policy_version 541970 (0.0007) [2023-12-26 19:18:26,771][105620] Updated weights for policy 1, policy_version 541980 (0.0009) [2023-12-26 19:18:26,826][105620] Updated weights for policy 1, policy_version 541990 (0.0006) [2023-12-26 19:18:27,096][105692] Updated weights for policy 0, policy_version 541224 (0.0008) [2023-12-26 19:18:27,151][105692] Updated weights for policy 0, policy_version 541234 (0.0010) [2023-12-26 19:18:27,206][105692] Updated weights for policy 0, policy_version 541244 (0.0011) [2023-12-26 19:18:27,405][105620] Updated weights for policy 1, policy_version 542000 (0.0008) [2023-12-26 19:18:27,458][105620] Updated weights for policy 1, policy_version 542010 (0.0010) [2023-12-26 19:18:27,525][105620] Updated weights for policy 1, policy_version 542020 (0.0010) [2023-12-26 19:18:27,870][105692] Updated weights for policy 0, policy_version 541254 (0.0007) [2023-12-26 19:18:27,928][105692] Updated weights for policy 0, policy_version 541264 (0.0005) [2023-12-26 19:18:27,992][105692] Updated weights for policy 0, policy_version 541274 (0.0007) [2023-12-26 19:18:28,360][105620] Updated weights for policy 1, policy_version 542030 (0.0007) [2023-12-26 19:18:28,420][105620] Updated weights for policy 1, policy_version 542040 (0.0009) [2023-12-26 19:18:28,493][105620] Updated weights for policy 1, policy_version 542050 (0.0010) [2023-12-26 19:18:28,614][105692] Updated weights for policy 0, policy_version 541284 (0.0008) [2023-12-26 19:18:28,682][105692] Updated weights for policy 0, policy_version 541294 (0.0006) [2023-12-26 19:18:28,735][105692] Updated weights for policy 0, policy_version 541304 (0.0009) [2023-12-26 19:18:29,314][105620] Updated weights for policy 1, policy_version 542060 (0.0010) [2023-12-26 19:18:29,336][105692] Updated weights for policy 0, policy_version 541314 (0.0010) [2023-12-26 19:18:29,376][105620] Updated weights for policy 1, policy_version 542070 (0.0007) [2023-12-26 19:18:29,395][105692] Updated weights for policy 0, policy_version 541324 (0.0007) [2023-12-26 19:18:29,435][105620] Updated weights for policy 1, policy_version 542080 (0.0007) [2023-12-26 19:18:29,454][105692] Updated weights for policy 0, policy_version 541334 (0.0007) [2023-12-26 19:18:29,527][105692] Updated weights for policy 0, policy_version 541344 (0.0009) [2023-12-26 19:18:30,168][105620] Updated weights for policy 1, policy_version 542090 (0.0006) [2023-12-26 19:18:30,220][105620] Updated weights for policy 1, policy_version 542101 (0.0009) [2023-12-26 19:18:30,247][105692] Updated weights for policy 0, policy_version 541354 (0.0005) [2023-12-26 19:18:30,282][105620] Updated weights for policy 1, policy_version 542111 (0.0008) [2023-12-26 19:18:30,292][105692] Updated weights for policy 0, policy_version 541364 (0.0005) [2023-12-26 19:18:30,339][105692] Updated weights for policy 0, policy_version 541374 (0.0005) [2023-12-26 19:18:30,961][105692] Updated weights for policy 0, policy_version 541384 (0.0009) [2023-12-26 19:18:31,024][105692] Updated weights for policy 0, policy_version 541394 (0.0009) [2023-12-26 19:18:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 277405696. Throughput: 0: 9814.8, 1: 9723.7. Samples: 277381644. Policy #0 lag: (min: 18.0, avg: 29.6, max: 50.0) [2023-12-26 19:18:31,062][104569] Avg episode reward: [(0, '9267.422'), (1, '8930.828')] [2023-12-26 19:18:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000542120_138797056.pth... [2023-12-26 19:18:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000541000_138510336.pth [2023-12-26 19:18:31,083][105692] Updated weights for policy 0, policy_version 541404 (0.0008) [2023-12-26 19:18:31,091][105620] Updated weights for policy 1, policy_version 542121 (0.0008) [2023-12-26 19:18:31,103][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000541408_138616832.pth... [2023-12-26 19:18:31,106][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000540256_138321920.pth [2023-12-26 19:18:31,156][105620] Updated weights for policy 1, policy_version 542131 (0.0008) [2023-12-26 19:18:31,219][105620] Updated weights for policy 1, policy_version 542141 (0.0005) [2023-12-26 19:18:31,283][105620] Updated weights for policy 1, policy_version 542151 (0.0006) [2023-12-26 19:18:31,812][105692] Updated weights for policy 0, policy_version 541414 (0.0006) [2023-12-26 19:18:31,864][105692] Updated weights for policy 0, policy_version 541424 (0.0010) [2023-12-26 19:18:31,925][105692] Updated weights for policy 0, policy_version 541434 (0.0011) [2023-12-26 19:18:32,013][105620] Updated weights for policy 1, policy_version 542161 (0.0008) [2023-12-26 19:18:32,076][105620] Updated weights for policy 1, policy_version 542171 (0.0008) [2023-12-26 19:18:32,135][105620] Updated weights for policy 1, policy_version 542181 (0.0008) [2023-12-26 19:18:32,614][105692] Updated weights for policy 0, policy_version 541444 (0.0011) [2023-12-26 19:18:32,662][105692] Updated weights for policy 0, policy_version 541454 (0.0010) [2023-12-26 19:18:32,713][105692] Updated weights for policy 0, policy_version 541464 (0.0010) [2023-12-26 19:18:32,881][105620] Updated weights for policy 1, policy_version 542191 (0.0006) [2023-12-26 19:18:32,935][105620] Updated weights for policy 1, policy_version 542201 (0.0006) [2023-12-26 19:18:32,981][105620] Updated weights for policy 1, policy_version 542211 (0.0007) [2023-12-26 19:18:33,378][105692] Updated weights for policy 0, policy_version 541474 (0.0008) [2023-12-26 19:18:33,433][105692] Updated weights for policy 0, policy_version 541484 (0.0008) [2023-12-26 19:18:33,492][105692] Updated weights for policy 0, policy_version 541494 (0.0008) [2023-12-26 19:18:33,547][105692] Updated weights for policy 0, policy_version 541504 (0.0008) [2023-12-26 19:18:33,709][105620] Updated weights for policy 1, policy_version 542221 (0.0008) [2023-12-26 19:18:33,760][105620] Updated weights for policy 1, policy_version 542231 (0.0006) [2023-12-26 19:18:33,809][105620] Updated weights for policy 1, policy_version 542241 (0.0009) [2023-12-26 19:18:34,243][105692] Updated weights for policy 0, policy_version 541514 (0.0009) [2023-12-26 19:18:34,302][105692] Updated weights for policy 0, policy_version 541524 (0.0008) [2023-12-26 19:18:34,356][105692] Updated weights for policy 0, policy_version 541534 (0.0008) [2023-12-26 19:18:34,583][105620] Updated weights for policy 1, policy_version 542251 (0.0009) [2023-12-26 19:18:34,648][105620] Updated weights for policy 1, policy_version 542261 (0.0009) [2023-12-26 19:18:34,705][105620] Updated weights for policy 1, policy_version 542271 (0.0009) [2023-12-26 19:18:35,106][105692] Updated weights for policy 0, policy_version 541544 (0.0009) [2023-12-26 19:18:35,158][105692] Updated weights for policy 0, policy_version 541554 (0.0009) [2023-12-26 19:18:35,214][105692] Updated weights for policy 0, policy_version 541564 (0.0009) [2023-12-26 19:18:35,460][105620] Updated weights for policy 1, policy_version 542281 (0.0010) [2023-12-26 19:18:35,510][105620] Updated weights for policy 1, policy_version 542291 (0.0009) [2023-12-26 19:18:35,558][105620] Updated weights for policy 1, policy_version 542301 (0.0008) [2023-12-26 19:18:35,605][105620] Updated weights for policy 1, policy_version 542311 (0.0009) [2023-12-26 19:18:36,007][105692] Updated weights for policy 0, policy_version 541574 (0.0009) [2023-12-26 19:18:36,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 277504000. Throughput: 0: 9858.6, 1: 9710.1. Samples: 277497800. Policy #0 lag: (min: 18.0, avg: 29.6, max: 50.0) [2023-12-26 19:18:36,062][104569] Avg episode reward: [(0, '9176.214'), (1, '9083.873')] [2023-12-26 19:18:36,072][105692] Updated weights for policy 0, policy_version 541584 (0.0009) [2023-12-26 19:18:36,141][105692] Updated weights for policy 0, policy_version 541594 (0.0008) [2023-12-26 19:18:36,311][105620] Updated weights for policy 1, policy_version 542321 (0.0008) [2023-12-26 19:18:36,365][105620] Updated weights for policy 1, policy_version 542331 (0.0008) [2023-12-26 19:18:36,430][105620] Updated weights for policy 1, policy_version 542341 (0.0008) [2023-12-26 19:18:36,922][105692] Updated weights for policy 0, policy_version 541604 (0.0009) [2023-12-26 19:18:36,970][105692] Updated weights for policy 0, policy_version 541614 (0.0009) [2023-12-26 19:18:37,021][105692] Updated weights for policy 0, policy_version 541624 (0.0009) [2023-12-26 19:18:37,140][105620] Updated weights for policy 1, policy_version 542351 (0.0009) [2023-12-26 19:18:37,207][105620] Updated weights for policy 1, policy_version 542361 (0.0009) [2023-12-26 19:18:37,271][105620] Updated weights for policy 1, policy_version 542371 (0.0007) [2023-12-26 19:18:37,804][105692] Updated weights for policy 0, policy_version 541634 (0.0009) [2023-12-26 19:18:37,862][105692] Updated weights for policy 0, policy_version 541644 (0.0005) [2023-12-26 19:18:37,914][105692] Updated weights for policy 0, policy_version 541654 (0.0005) [2023-12-26 19:18:37,952][105620] Updated weights for policy 1, policy_version 542381 (0.0007) [2023-12-26 19:18:37,970][105692] Updated weights for policy 0, policy_version 541664 (0.0005) [2023-12-26 19:18:38,008][105620] Updated weights for policy 1, policy_version 542391 (0.0006) [2023-12-26 19:18:38,063][105620] Updated weights for policy 1, policy_version 542403 (0.0010) [2023-12-26 19:18:38,538][105692] Updated weights for policy 0, policy_version 541674 (0.0010) [2023-12-26 19:18:38,592][105692] Updated weights for policy 0, policy_version 541685 (0.0009) [2023-12-26 19:18:38,652][105692] Updated weights for policy 0, policy_version 541695 (0.0010) [2023-12-26 19:18:38,739][105620] Updated weights for policy 1, policy_version 542413 (0.0009) [2023-12-26 19:18:38,797][105620] Updated weights for policy 1, policy_version 542423 (0.0009) [2023-12-26 19:18:38,853][105620] Updated weights for policy 1, policy_version 542433 (0.0008) [2023-12-26 19:18:39,511][105692] Updated weights for policy 0, policy_version 541705 (0.0009) [2023-12-26 19:18:39,560][105692] Updated weights for policy 0, policy_version 541715 (0.0009) [2023-12-26 19:18:39,579][105620] Updated weights for policy 1, policy_version 542443 (0.0008) [2023-12-26 19:18:39,607][105692] Updated weights for policy 0, policy_version 541725 (0.0007) [2023-12-26 19:18:39,640][105620] Updated weights for policy 1, policy_version 542453 (0.0007) [2023-12-26 19:18:39,700][105620] Updated weights for policy 1, policy_version 542463 (0.0009) [2023-12-26 19:18:40,402][105620] Updated weights for policy 1, policy_version 542473 (0.0009) [2023-12-26 19:18:40,410][105692] Updated weights for policy 0, policy_version 541735 (0.0009) [2023-12-26 19:18:40,468][105620] Updated weights for policy 1, policy_version 542483 (0.0006) [2023-12-26 19:18:40,470][105692] Updated weights for policy 0, policy_version 541745 (0.0009) [2023-12-26 19:18:40,529][105620] Updated weights for policy 1, policy_version 542493 (0.0007) [2023-12-26 19:18:40,531][105692] Updated weights for policy 0, policy_version 541755 (0.0006) [2023-12-26 19:18:40,589][105620] Updated weights for policy 1, policy_version 542503 (0.0009) [2023-12-26 19:18:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 277602304. Throughput: 0: 9765.3, 1: 9721.6. Samples: 277612704. Policy #0 lag: (min: 18.0, avg: 29.6, max: 50.0) [2023-12-26 19:18:41,062][104569] Avg episode reward: [(0, '9176.072'), (1, '8993.386')] [2023-12-26 19:18:41,277][105620] Updated weights for policy 1, policy_version 542513 (0.0009) [2023-12-26 19:18:41,316][105692] Updated weights for policy 0, policy_version 541765 (0.0006) [2023-12-26 19:18:41,329][105620] Updated weights for policy 1, policy_version 542523 (0.0007) [2023-12-26 19:18:41,382][105692] Updated weights for policy 0, policy_version 541775 (0.0008) [2023-12-26 19:18:41,400][105620] Updated weights for policy 1, policy_version 542533 (0.0008) [2023-12-26 19:18:41,443][105692] Updated weights for policy 0, policy_version 541785 (0.0008) [2023-12-26 19:18:42,126][105692] Updated weights for policy 0, policy_version 541795 (0.0007) [2023-12-26 19:18:42,201][105692] Updated weights for policy 0, policy_version 541805 (0.0010) [2023-12-26 19:18:42,242][105620] Updated weights for policy 1, policy_version 542543 (0.0006) [2023-12-26 19:18:42,268][105692] Updated weights for policy 0, policy_version 541815 (0.0009) [2023-12-26 19:18:42,304][105620] Updated weights for policy 1, policy_version 542553 (0.0007) [2023-12-26 19:18:42,370][105620] Updated weights for policy 1, policy_version 542563 (0.0008) [2023-12-26 19:18:42,982][105692] Updated weights for policy 0, policy_version 541825 (0.0008) [2023-12-26 19:18:43,034][105692] Updated weights for policy 0, policy_version 541835 (0.0008) [2023-12-26 19:18:43,083][105692] Updated weights for policy 0, policy_version 541845 (0.0008) [2023-12-26 19:18:43,132][105692] Updated weights for policy 0, policy_version 541855 (0.0008) [2023-12-26 19:18:43,143][105620] Updated weights for policy 1, policy_version 542573 (0.0010) [2023-12-26 19:18:43,209][105620] Updated weights for policy 1, policy_version 542583 (0.0011) [2023-12-26 19:18:43,259][105620] Updated weights for policy 1, policy_version 542593 (0.0010) [2023-12-26 19:18:43,915][105692] Updated weights for policy 0, policy_version 541865 (0.0009) [2023-12-26 19:18:43,963][105692] Updated weights for policy 0, policy_version 541875 (0.0007) [2023-12-26 19:18:44,010][105620] Updated weights for policy 1, policy_version 542603 (0.0011) [2023-12-26 19:18:44,018][105692] Updated weights for policy 0, policy_version 541885 (0.0006) [2023-12-26 19:18:44,055][105620] Updated weights for policy 1, policy_version 542613 (0.0010) [2023-12-26 19:18:44,100][105620] Updated weights for policy 1, policy_version 542623 (0.0010) [2023-12-26 19:18:44,674][105692] Updated weights for policy 0, policy_version 541895 (0.0009) [2023-12-26 19:18:44,732][105692] Updated weights for policy 0, policy_version 541905 (0.0011) [2023-12-26 19:18:44,802][105692] Updated weights for policy 0, policy_version 541915 (0.0011) [2023-12-26 19:18:44,865][105620] Updated weights for policy 1, policy_version 542633 (0.0008) [2023-12-26 19:18:44,929][105620] Updated weights for policy 1, policy_version 542643 (0.0011) [2023-12-26 19:18:45,000][105620] Updated weights for policy 1, policy_version 542653 (0.0011) [2023-12-26 19:18:45,070][105620] Updated weights for policy 1, policy_version 542663 (0.0011) [2023-12-26 19:18:45,525][105692] Updated weights for policy 0, policy_version 541925 (0.0008) [2023-12-26 19:18:45,584][105692] Updated weights for policy 0, policy_version 541935 (0.0005) [2023-12-26 19:18:45,651][105692] Updated weights for policy 0, policy_version 541945 (0.0005) [2023-12-26 19:18:45,727][105620] Updated weights for policy 1, policy_version 542673 (0.0010) [2023-12-26 19:18:45,779][105620] Updated weights for policy 1, policy_version 542683 (0.0010) [2023-12-26 19:18:45,844][105620] Updated weights for policy 1, policy_version 542693 (0.0010) [2023-12-26 19:18:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 277700608. Throughput: 0: 9754.7, 1: 9706.8. Samples: 277668268. Policy #0 lag: (min: 18.0, avg: 29.6, max: 50.0) [2023-12-26 19:18:46,062][104569] Avg episode reward: [(0, '9176.015'), (1, '9171.246')] [2023-12-26 19:18:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000541952_138756096.pth... [2023-12-26 19:18:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000542696_138944512.pth... [2023-12-26 19:18:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000540832_138469376.pth [2023-12-26 19:18:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000541544_138649600.pth [2023-12-26 19:18:46,225][105692] Updated weights for policy 0, policy_version 541955 (0.0005) [2023-12-26 19:18:46,288][105692] Updated weights for policy 0, policy_version 541965 (0.0006) [2023-12-26 19:18:46,333][105692] Updated weights for policy 0, policy_version 541975 (0.0010) [2023-12-26 19:18:46,594][105620] Updated weights for policy 1, policy_version 542703 (0.0011) [2023-12-26 19:18:46,658][105620] Updated weights for policy 1, policy_version 542713 (0.0010) [2023-12-26 19:18:46,717][105620] Updated weights for policy 1, policy_version 542723 (0.0010) [2023-12-26 19:18:46,899][105692] Updated weights for policy 0, policy_version 541985 (0.0009) [2023-12-26 19:18:46,954][105692] Updated weights for policy 0, policy_version 541995 (0.0008) [2023-12-26 19:18:47,007][105692] Updated weights for policy 0, policy_version 542005 (0.0006) [2023-12-26 19:18:47,069][105692] Updated weights for policy 0, policy_version 542015 (0.0008) [2023-12-26 19:18:47,451][105620] Updated weights for policy 1, policy_version 542733 (0.0010) [2023-12-26 19:18:47,516][105620] Updated weights for policy 1, policy_version 542743 (0.0010) [2023-12-26 19:18:47,580][105620] Updated weights for policy 1, policy_version 542753 (0.0010) [2023-12-26 19:18:47,752][105692] Updated weights for policy 0, policy_version 542025 (0.0010) [2023-12-26 19:18:47,810][105692] Updated weights for policy 0, policy_version 542035 (0.0005) [2023-12-26 19:18:47,858][105692] Updated weights for policy 0, policy_version 542045 (0.0005) [2023-12-26 19:18:48,284][105620] Updated weights for policy 1, policy_version 542763 (0.0010) [2023-12-26 19:18:48,346][105620] Updated weights for policy 1, policy_version 542773 (0.0009) [2023-12-26 19:18:48,416][105620] Updated weights for policy 1, policy_version 542783 (0.0008) [2023-12-26 19:18:48,588][105692] Updated weights for policy 0, policy_version 542055 (0.0009) [2023-12-26 19:18:48,642][105692] Updated weights for policy 0, policy_version 542065 (0.0005) [2023-12-26 19:18:48,693][105692] Updated weights for policy 0, policy_version 542075 (0.0009) [2023-12-26 19:18:49,089][105620] Updated weights for policy 1, policy_version 542793 (0.0006) [2023-12-26 19:18:49,147][105620] Updated weights for policy 1, policy_version 542803 (0.0006) [2023-12-26 19:18:49,211][105620] Updated weights for policy 1, policy_version 542813 (0.0006) [2023-12-26 19:18:49,281][105620] Updated weights for policy 1, policy_version 542824 (0.0010) [2023-12-26 19:18:49,378][105692] Updated weights for policy 0, policy_version 542085 (0.0009) [2023-12-26 19:18:49,446][105692] Updated weights for policy 0, policy_version 542095 (0.0006) [2023-12-26 19:18:49,510][105692] Updated weights for policy 0, policy_version 542105 (0.0008) [2023-12-26 19:18:50,017][105620] Updated weights for policy 1, policy_version 542834 (0.0008) [2023-12-26 19:18:50,081][105620] Updated weights for policy 1, policy_version 542844 (0.0008) [2023-12-26 19:18:50,141][105620] Updated weights for policy 1, policy_version 542854 (0.0008) [2023-12-26 19:18:50,212][105692] Updated weights for policy 0, policy_version 542115 (0.0007) [2023-12-26 19:18:50,264][105692] Updated weights for policy 0, policy_version 542125 (0.0010) [2023-12-26 19:18:50,320][105692] Updated weights for policy 0, policy_version 542135 (0.0010) [2023-12-26 19:18:50,887][105620] Updated weights for policy 1, policy_version 542864 (0.0008) [2023-12-26 19:18:50,950][105620] Updated weights for policy 1, policy_version 542874 (0.0007) [2023-12-26 19:18:51,013][105620] Updated weights for policy 1, policy_version 542884 (0.0008) [2023-12-26 19:18:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 277798912. Throughput: 0: 9749.9, 1: 9711.5. Samples: 277788836. Policy #0 lag: (min: 18.0, avg: 29.6, max: 50.0) [2023-12-26 19:18:51,063][104569] Avg episode reward: [(0, '9176.256'), (1, '9259.595')] [2023-12-26 19:18:51,076][105692] Updated weights for policy 0, policy_version 542145 (0.0010) [2023-12-26 19:18:51,145][105692] Updated weights for policy 0, policy_version 542155 (0.0010) [2023-12-26 19:18:51,196][105692] Updated weights for policy 0, policy_version 542165 (0.0009) [2023-12-26 19:18:51,259][105692] Updated weights for policy 0, policy_version 542175 (0.0008) [2023-12-26 19:18:51,712][105620] Updated weights for policy 1, policy_version 542894 (0.0009) [2023-12-26 19:18:51,782][105620] Updated weights for policy 1, policy_version 542904 (0.0009) [2023-12-26 19:18:51,836][105620] Updated weights for policy 1, policy_version 542914 (0.0009) [2023-12-26 19:18:52,037][105692] Updated weights for policy 0, policy_version 542186 (0.0010) [2023-12-26 19:18:52,104][105692] Updated weights for policy 0, policy_version 542196 (0.0010) [2023-12-26 19:18:52,166][105692] Updated weights for policy 0, policy_version 542206 (0.0009) [2023-12-26 19:18:52,517][105620] Updated weights for policy 1, policy_version 542924 (0.0009) [2023-12-26 19:18:52,584][105620] Updated weights for policy 1, policy_version 542934 (0.0010) [2023-12-26 19:18:52,643][105620] Updated weights for policy 1, policy_version 542944 (0.0005) [2023-12-26 19:18:53,025][105692] Updated weights for policy 0, policy_version 542216 (0.0008) [2023-12-26 19:18:53,092][105692] Updated weights for policy 0, policy_version 542226 (0.0008) [2023-12-26 19:18:53,151][105692] Updated weights for policy 0, policy_version 542236 (0.0008) [2023-12-26 19:18:53,311][105620] Updated weights for policy 1, policy_version 542954 (0.0006) [2023-12-26 19:18:53,373][105620] Updated weights for policy 1, policy_version 542964 (0.0010) [2023-12-26 19:18:53,438][105620] Updated weights for policy 1, policy_version 542974 (0.0010) [2023-12-26 19:18:53,496][105620] Updated weights for policy 1, policy_version 542984 (0.0010) [2023-12-26 19:18:53,919][105692] Updated weights for policy 0, policy_version 542246 (0.0008) [2023-12-26 19:18:53,984][105692] Updated weights for policy 0, policy_version 542256 (0.0008) [2023-12-26 19:18:54,051][105692] Updated weights for policy 0, policy_version 542266 (0.0008) [2023-12-26 19:18:54,232][105620] Updated weights for policy 1, policy_version 542994 (0.0010) [2023-12-26 19:18:54,294][105620] Updated weights for policy 1, policy_version 543004 (0.0010) [2023-12-26 19:18:54,353][105620] Updated weights for policy 1, policy_version 543014 (0.0011) [2023-12-26 19:18:54,631][105692] Updated weights for policy 0, policy_version 542276 (0.0009) [2023-12-26 19:18:54,688][105692] Updated weights for policy 0, policy_version 542286 (0.0009) [2023-12-26 19:18:54,746][105692] Updated weights for policy 0, policy_version 542296 (0.0006) [2023-12-26 19:18:55,087][105620] Updated weights for policy 1, policy_version 543024 (0.0010) [2023-12-26 19:18:55,138][105620] Updated weights for policy 1, policy_version 543034 (0.0010) [2023-12-26 19:18:55,183][105620] Updated weights for policy 1, policy_version 543044 (0.0010) [2023-12-26 19:18:55,388][105692] Updated weights for policy 0, policy_version 542306 (0.0006) [2023-12-26 19:18:55,439][105692] Updated weights for policy 0, policy_version 542316 (0.0006) [2023-12-26 19:18:55,489][105692] Updated weights for policy 0, policy_version 542326 (0.0006) [2023-12-26 19:18:55,546][105692] Updated weights for policy 0, policy_version 542336 (0.0006) [2023-12-26 19:18:55,949][105620] Updated weights for policy 1, policy_version 543054 (0.0010) [2023-12-26 19:18:55,993][105620] Updated weights for policy 1, policy_version 543064 (0.0010) [2023-12-26 19:18:56,048][105620] Updated weights for policy 1, policy_version 543074 (0.0010) [2023-12-26 19:18:56,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 277889024. Throughput: 0: 9752.1, 1: 9664.1. Samples: 277903428. Policy #0 lag: (min: 5.0, avg: 15.3, max: 37.0) [2023-12-26 19:18:56,063][104569] Avg episode reward: [(0, '9084.933'), (1, '9259.032')] [2023-12-26 19:18:56,263][105692] Updated weights for policy 0, policy_version 542346 (0.0008) [2023-12-26 19:18:56,318][105692] Updated weights for policy 0, policy_version 542356 (0.0005) [2023-12-26 19:18:56,377][105692] Updated weights for policy 0, policy_version 542366 (0.0006) [2023-12-26 19:18:56,798][105620] Updated weights for policy 1, policy_version 543084 (0.0010) [2023-12-26 19:18:56,850][105620] Updated weights for policy 1, policy_version 543094 (0.0010) [2023-12-26 19:18:56,902][105620] Updated weights for policy 1, policy_version 543104 (0.0010) [2023-12-26 19:18:57,060][105692] Updated weights for policy 0, policy_version 542376 (0.0010) [2023-12-26 19:18:57,113][105692] Updated weights for policy 0, policy_version 542386 (0.0011) [2023-12-26 19:18:57,158][105692] Updated weights for policy 0, policy_version 542396 (0.0010) [2023-12-26 19:18:57,652][105620] Updated weights for policy 1, policy_version 543114 (0.0009) [2023-12-26 19:18:57,704][105620] Updated weights for policy 1, policy_version 543124 (0.0010) [2023-12-26 19:18:57,749][105620] Updated weights for policy 1, policy_version 543134 (0.0010) [2023-12-26 19:18:57,803][105620] Updated weights for policy 1, policy_version 543144 (0.0010) [2023-12-26 19:18:57,836][105692] Updated weights for policy 0, policy_version 542406 (0.0010) [2023-12-26 19:18:57,894][105692] Updated weights for policy 0, policy_version 542416 (0.0010) [2023-12-26 19:18:57,941][105692] Updated weights for policy 0, policy_version 542426 (0.0010) [2023-12-26 19:18:58,629][105620] Updated weights for policy 1, policy_version 543154 (0.0011) [2023-12-26 19:18:58,692][105620] Updated weights for policy 1, policy_version 543164 (0.0012) [2023-12-26 19:18:58,737][105692] Updated weights for policy 0, policy_version 542436 (0.0010) [2023-12-26 19:18:58,762][105620] Updated weights for policy 1, policy_version 543174 (0.0010) [2023-12-26 19:18:58,803][105692] Updated weights for policy 0, policy_version 542446 (0.0009) [2023-12-26 19:18:58,866][105692] Updated weights for policy 0, policy_version 542456 (0.0008) [2023-12-26 19:18:59,452][105620] Updated weights for policy 1, policy_version 543184 (0.0007) [2023-12-26 19:18:59,510][105620] Updated weights for policy 1, policy_version 543194 (0.0008) [2023-12-26 19:18:59,565][105620] Updated weights for policy 1, policy_version 543204 (0.0009) [2023-12-26 19:18:59,598][105692] Updated weights for policy 0, policy_version 542466 (0.0008) [2023-12-26 19:18:59,654][105692] Updated weights for policy 0, policy_version 542476 (0.0009) [2023-12-26 19:18:59,713][105692] Updated weights for policy 0, policy_version 542486 (0.0008) [2023-12-26 19:18:59,769][105692] Updated weights for policy 0, policy_version 542496 (0.0011) [2023-12-26 19:19:00,364][105620] Updated weights for policy 1, policy_version 543214 (0.0009) [2023-12-26 19:19:00,423][105620] Updated weights for policy 1, policy_version 543224 (0.0008) [2023-12-26 19:19:00,461][105692] Updated weights for policy 0, policy_version 542506 (0.0006) [2023-12-26 19:19:00,484][105620] Updated weights for policy 1, policy_version 543234 (0.0007) [2023-12-26 19:19:00,511][105692] Updated weights for policy 0, policy_version 542516 (0.0010) [2023-12-26 19:19:00,566][105692] Updated weights for policy 0, policy_version 542526 (0.0011) [2023-12-26 19:19:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 277987328. Throughput: 0: 9787.5, 1: 9665.8. Samples: 277960564. Policy #0 lag: (min: 5.0, avg: 15.3, max: 37.0) [2023-12-26 19:19:01,062][104569] Avg episode reward: [(0, '9175.608'), (1, '9260.104')] [2023-12-26 19:19:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000542528_138903552.pth... [2023-12-26 19:19:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000543240_139083776.pth... [2023-12-26 19:19:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000541408_138616832.pth [2023-12-26 19:19:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000542120_138797056.pth [2023-12-26 19:19:01,251][105620] Updated weights for policy 1, policy_version 543244 (0.0008) [2023-12-26 19:19:01,309][105692] Updated weights for policy 0, policy_version 542536 (0.0011) [2023-12-26 19:19:01,314][105620] Updated weights for policy 1, policy_version 543254 (0.0011) [2023-12-26 19:19:01,372][105620] Updated weights for policy 1, policy_version 543264 (0.0011) [2023-12-26 19:19:01,377][105692] Updated weights for policy 0, policy_version 542546 (0.0009) [2023-12-26 19:19:01,437][105692] Updated weights for policy 0, policy_version 542556 (0.0006) [2023-12-26 19:19:02,028][105620] Updated weights for policy 1, policy_version 543274 (0.0011) [2023-12-26 19:19:02,082][105620] Updated weights for policy 1, policy_version 543284 (0.0010) [2023-12-26 19:19:02,084][105692] Updated weights for policy 0, policy_version 542566 (0.0009) [2023-12-26 19:19:02,141][105620] Updated weights for policy 1, policy_version 543294 (0.0011) [2023-12-26 19:19:02,146][105692] Updated weights for policy 0, policy_version 542576 (0.0011) [2023-12-26 19:19:02,201][105620] Updated weights for policy 1, policy_version 543304 (0.0010) [2023-12-26 19:19:02,210][105692] Updated weights for policy 0, policy_version 542586 (0.0008) [2023-12-26 19:19:02,867][105620] Updated weights for policy 1, policy_version 543314 (0.0010) [2023-12-26 19:19:02,901][105692] Updated weights for policy 0, policy_version 542596 (0.0007) [2023-12-26 19:19:02,929][105620] Updated weights for policy 1, policy_version 543324 (0.0010) [2023-12-26 19:19:02,955][105692] Updated weights for policy 0, policy_version 542606 (0.0010) [2023-12-26 19:19:02,977][105620] Updated weights for policy 1, policy_version 543334 (0.0010) [2023-12-26 19:19:03,006][105692] Updated weights for policy 0, policy_version 542616 (0.0005) [2023-12-26 19:19:03,527][105692] Updated weights for policy 0, policy_version 542626 (0.0005) [2023-12-26 19:19:03,574][105692] Updated weights for policy 0, policy_version 542636 (0.0009) [2023-12-26 19:19:03,621][105692] Updated weights for policy 0, policy_version 542646 (0.0010) [2023-12-26 19:19:03,674][105692] Updated weights for policy 0, policy_version 542656 (0.0008) [2023-12-26 19:19:03,716][105620] Updated weights for policy 1, policy_version 543344 (0.0010) [2023-12-26 19:19:03,767][105620] Updated weights for policy 1, policy_version 543354 (0.0010) [2023-12-26 19:19:03,821][105620] Updated weights for policy 1, policy_version 543364 (0.0010) [2023-12-26 19:19:04,311][105692] Updated weights for policy 0, policy_version 542666 (0.0007) [2023-12-26 19:19:04,370][105692] Updated weights for policy 0, policy_version 542676 (0.0007) [2023-12-26 19:19:04,427][105692] Updated weights for policy 0, policy_version 542686 (0.0006) [2023-12-26 19:19:04,598][105620] Updated weights for policy 1, policy_version 543374 (0.0010) [2023-12-26 19:19:04,653][105620] Updated weights for policy 1, policy_version 543384 (0.0010) [2023-12-26 19:19:04,701][105620] Updated weights for policy 1, policy_version 543394 (0.0010) [2023-12-26 19:19:05,118][105692] Updated weights for policy 0, policy_version 542696 (0.0006) [2023-12-26 19:19:05,182][105692] Updated weights for policy 0, policy_version 542706 (0.0006) [2023-12-26 19:19:05,247][105692] Updated weights for policy 0, policy_version 542716 (0.0006) [2023-12-26 19:19:05,368][105620] Updated weights for policy 1, policy_version 543404 (0.0009) [2023-12-26 19:19:05,426][105620] Updated weights for policy 1, policy_version 543414 (0.0009) [2023-12-26 19:19:05,473][105620] Updated weights for policy 1, policy_version 543424 (0.0009) [2023-12-26 19:19:05,936][105692] Updated weights for policy 0, policy_version 542726 (0.0008) [2023-12-26 19:19:05,988][105692] Updated weights for policy 0, policy_version 542736 (0.0007) [2023-12-26 19:19:06,051][105692] Updated weights for policy 0, policy_version 542746 (0.0005) [2023-12-26 19:19:06,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 278085632. Throughput: 0: 9942.1, 1: 9536.8. Samples: 278079656. Policy #0 lag: (min: 5.0, avg: 15.3, max: 37.0) [2023-12-26 19:19:06,062][104569] Avg episode reward: [(0, '9083.854'), (1, '9351.233')] [2023-12-26 19:19:06,075][105620] Updated weights for policy 1, policy_version 543434 (0.0009) [2023-12-26 19:19:06,140][105620] Updated weights for policy 1, policy_version 543444 (0.0008) [2023-12-26 19:19:06,194][105620] Updated weights for policy 1, policy_version 543454 (0.0008) [2023-12-26 19:19:06,256][105620] Updated weights for policy 1, policy_version 543464 (0.0006) [2023-12-26 19:19:06,760][105692] Updated weights for policy 0, policy_version 542756 (0.0007) [2023-12-26 19:19:06,818][105692] Updated weights for policy 0, policy_version 542766 (0.0009) [2023-12-26 19:19:06,877][105692] Updated weights for policy 0, policy_version 542776 (0.0009) [2023-12-26 19:19:06,964][105620] Updated weights for policy 1, policy_version 543474 (0.0007) [2023-12-26 19:19:07,026][105620] Updated weights for policy 1, policy_version 543484 (0.0007) [2023-12-26 19:19:07,091][105620] Updated weights for policy 1, policy_version 543494 (0.0009) [2023-12-26 19:19:07,518][105692] Updated weights for policy 0, policy_version 542786 (0.0008) [2023-12-26 19:19:07,573][105692] Updated weights for policy 0, policy_version 542796 (0.0005) [2023-12-26 19:19:07,637][105692] Updated weights for policy 0, policy_version 542806 (0.0005) [2023-12-26 19:19:07,687][105692] Updated weights for policy 0, policy_version 542816 (0.0008) [2023-12-26 19:19:07,871][105620] Updated weights for policy 1, policy_version 543504 (0.0007) [2023-12-26 19:19:07,929][105620] Updated weights for policy 1, policy_version 543514 (0.0006) [2023-12-26 19:19:07,986][105620] Updated weights for policy 1, policy_version 543524 (0.0005) [2023-12-26 19:19:08,387][105692] Updated weights for policy 0, policy_version 542826 (0.0011) [2023-12-26 19:19:08,440][105692] Updated weights for policy 0, policy_version 542836 (0.0010) [2023-12-26 19:19:08,492][105692] Updated weights for policy 0, policy_version 542846 (0.0010) [2023-12-26 19:19:08,647][105620] Updated weights for policy 1, policy_version 543534 (0.0006) [2023-12-26 19:19:08,705][105620] Updated weights for policy 1, policy_version 543544 (0.0007) [2023-12-26 19:19:08,758][105620] Updated weights for policy 1, policy_version 543554 (0.0008) [2023-12-26 19:19:09,259][105692] Updated weights for policy 0, policy_version 542856 (0.0007) [2023-12-26 19:19:09,318][105692] Updated weights for policy 0, policy_version 542866 (0.0009) [2023-12-26 19:19:09,382][105692] Updated weights for policy 0, policy_version 542876 (0.0011) [2023-12-26 19:19:09,440][105620] Updated weights for policy 1, policy_version 543564 (0.0009) [2023-12-26 19:19:09,499][105620] Updated weights for policy 1, policy_version 543574 (0.0010) [2023-12-26 19:19:09,553][105620] Updated weights for policy 1, policy_version 543584 (0.0011) [2023-12-26 19:19:10,031][105692] Updated weights for policy 0, policy_version 542886 (0.0011) [2023-12-26 19:19:10,097][105692] Updated weights for policy 0, policy_version 542896 (0.0011) [2023-12-26 19:19:10,150][105692] Updated weights for policy 0, policy_version 542906 (0.0011) [2023-12-26 19:19:10,334][105620] Updated weights for policy 1, policy_version 543594 (0.0011) [2023-12-26 19:19:10,401][105620] Updated weights for policy 1, policy_version 543604 (0.0011) [2023-12-26 19:19:10,460][105620] Updated weights for policy 1, policy_version 543614 (0.0010) [2023-12-26 19:19:10,522][105620] Updated weights for policy 1, policy_version 543624 (0.0011) [2023-12-26 19:19:10,905][105692] Updated weights for policy 0, policy_version 542916 (0.0011) [2023-12-26 19:19:10,971][105692] Updated weights for policy 0, policy_version 542926 (0.0011) [2023-12-26 19:19:11,029][105692] Updated weights for policy 0, policy_version 542936 (0.0011) [2023-12-26 19:19:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 278183936. Throughput: 0: 9916.6, 1: 9566.6. Samples: 278199316. Policy #0 lag: (min: 5.0, avg: 15.3, max: 37.0) [2023-12-26 19:19:11,063][104569] Avg episode reward: [(0, '9084.832'), (1, '9351.159')] [2023-12-26 19:19:11,186][105620] Updated weights for policy 1, policy_version 543634 (0.0010) [2023-12-26 19:19:11,250][105620] Updated weights for policy 1, policy_version 543644 (0.0011) [2023-12-26 19:19:11,318][105620] Updated weights for policy 1, policy_version 543654 (0.0011) [2023-12-26 19:19:11,827][105692] Updated weights for policy 0, policy_version 542946 (0.0010) [2023-12-26 19:19:11,878][105692] Updated weights for policy 0, policy_version 542956 (0.0010) [2023-12-26 19:19:11,939][105692] Updated weights for policy 0, policy_version 542967 (0.0011) [2023-12-26 19:19:11,994][105620] Updated weights for policy 1, policy_version 543664 (0.0007) [2023-12-26 19:19:12,065][105620] Updated weights for policy 1, policy_version 543674 (0.0008) [2023-12-26 19:19:12,122][105620] Updated weights for policy 1, policy_version 543684 (0.0007) [2023-12-26 19:19:12,711][105692] Updated weights for policy 0, policy_version 542977 (0.0010) [2023-12-26 19:19:12,732][105620] Updated weights for policy 1, policy_version 543694 (0.0006) [2023-12-26 19:19:12,776][105692] Updated weights for policy 0, policy_version 542987 (0.0007) [2023-12-26 19:19:12,788][105620] Updated weights for policy 1, policy_version 543704 (0.0006) [2023-12-26 19:19:12,841][105692] Updated weights for policy 0, policy_version 542997 (0.0007) [2023-12-26 19:19:12,845][105620] Updated weights for policy 1, policy_version 543714 (0.0007) [2023-12-26 19:19:12,907][105692] Updated weights for policy 0, policy_version 543007 (0.0007) [2023-12-26 19:19:13,534][105620] Updated weights for policy 1, policy_version 543724 (0.0008) [2023-12-26 19:19:13,565][105692] Updated weights for policy 0, policy_version 543017 (0.0009) [2023-12-26 19:19:13,597][105620] Updated weights for policy 1, policy_version 543734 (0.0006) [2023-12-26 19:19:13,626][105692] Updated weights for policy 0, policy_version 543027 (0.0009) [2023-12-26 19:19:13,652][105620] Updated weights for policy 1, policy_version 543744 (0.0005) [2023-12-26 19:19:13,686][105692] Updated weights for policy 0, policy_version 543037 (0.0009) [2023-12-26 19:19:14,203][105620] Updated weights for policy 1, policy_version 543754 (0.0006) [2023-12-26 19:19:14,272][105620] Updated weights for policy 1, policy_version 543764 (0.0009) [2023-12-26 19:19:14,331][105620] Updated weights for policy 1, policy_version 543774 (0.0009) [2023-12-26 19:19:14,381][105620] Updated weights for policy 1, policy_version 543784 (0.0009) [2023-12-26 19:19:14,442][105692] Updated weights for policy 0, policy_version 543047 (0.0009) [2023-12-26 19:19:14,500][105692] Updated weights for policy 0, policy_version 543057 (0.0009) [2023-12-26 19:19:14,548][105692] Updated weights for policy 0, policy_version 543067 (0.0009) [2023-12-26 19:19:15,122][105620] Updated weights for policy 1, policy_version 543794 (0.0009) [2023-12-26 19:19:15,186][105620] Updated weights for policy 1, policy_version 543804 (0.0008) [2023-12-26 19:19:15,246][105620] Updated weights for policy 1, policy_version 543814 (0.0009) [2023-12-26 19:19:15,307][105692] Updated weights for policy 0, policy_version 543077 (0.0009) [2023-12-26 19:19:15,359][105692] Updated weights for policy 0, policy_version 543087 (0.0009) [2023-12-26 19:19:15,414][105692] Updated weights for policy 0, policy_version 543097 (0.0009) [2023-12-26 19:19:15,959][105620] Updated weights for policy 1, policy_version 543824 (0.0008) [2023-12-26 19:19:16,004][105620] Updated weights for policy 1, policy_version 543834 (0.0007) [2023-12-26 19:19:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 278282240. Throughput: 0: 9850.5, 1: 9650.3. Samples: 278259180. Policy #0 lag: (min: 5.0, avg: 15.3, max: 37.0) [2023-12-26 19:19:16,063][104569] Avg episode reward: [(0, '9000.854'), (1, '9349.983')] [2023-12-26 19:19:16,067][105620] Updated weights for policy 1, policy_version 543844 (0.0009) [2023-12-26 19:19:16,070][105692] Updated weights for policy 0, policy_version 543107 (0.0007) [2023-12-26 19:19:16,084][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000543848_139239424.pth... [2023-12-26 19:19:16,088][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000542696_138944512.pth [2023-12-26 19:19:16,126][105692] Updated weights for policy 0, policy_version 543117 (0.0006) [2023-12-26 19:19:16,180][105692] Updated weights for policy 0, policy_version 543127 (0.0010) [2023-12-26 19:19:16,237][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000543136_139059200.pth... [2023-12-26 19:19:16,243][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000541952_138756096.pth [2023-12-26 19:19:16,738][105620] Updated weights for policy 1, policy_version 543854 (0.0007) [2023-12-26 19:19:16,792][105620] Updated weights for policy 1, policy_version 543864 (0.0005) [2023-12-26 19:19:16,846][105620] Updated weights for policy 1, policy_version 543874 (0.0006) [2023-12-26 19:19:16,908][105692] Updated weights for policy 0, policy_version 543137 (0.0010) [2023-12-26 19:19:16,966][105692] Updated weights for policy 0, policy_version 543147 (0.0010) [2023-12-26 19:19:17,020][105692] Updated weights for policy 0, policy_version 543157 (0.0010) [2023-12-26 19:19:17,068][105692] Updated weights for policy 0, policy_version 543167 (0.0010) [2023-12-26 19:19:17,557][105620] Updated weights for policy 1, policy_version 543884 (0.0008) [2023-12-26 19:19:17,619][105620] Updated weights for policy 1, policy_version 543894 (0.0008) [2023-12-26 19:19:17,681][105620] Updated weights for policy 1, policy_version 543904 (0.0007) [2023-12-26 19:19:17,812][105692] Updated weights for policy 0, policy_version 543177 (0.0010) [2023-12-26 19:19:17,864][105692] Updated weights for policy 0, policy_version 543187 (0.0010) [2023-12-26 19:19:17,910][105692] Updated weights for policy 0, policy_version 543197 (0.0009) [2023-12-26 19:19:18,281][105620] Updated weights for policy 1, policy_version 543914 (0.0006) [2023-12-26 19:19:18,341][105620] Updated weights for policy 1, policy_version 543924 (0.0006) [2023-12-26 19:19:18,403][105620] Updated weights for policy 1, policy_version 543934 (0.0008) [2023-12-26 19:19:18,461][105620] Updated weights for policy 1, policy_version 543944 (0.0008) [2023-12-26 19:19:18,549][105692] Updated weights for policy 0, policy_version 543207 (0.0009) [2023-12-26 19:19:18,604][105692] Updated weights for policy 0, policy_version 543217 (0.0011) [2023-12-26 19:19:18,660][105692] Updated weights for policy 0, policy_version 543227 (0.0011) [2023-12-26 19:19:19,168][105620] Updated weights for policy 1, policy_version 543954 (0.0008) [2023-12-26 19:19:19,228][105620] Updated weights for policy 1, policy_version 543964 (0.0008) [2023-12-26 19:19:19,290][105620] Updated weights for policy 1, policy_version 543974 (0.0007) [2023-12-26 19:19:19,440][105692] Updated weights for policy 0, policy_version 543237 (0.0010) [2023-12-26 19:19:19,500][105692] Updated weights for policy 0, policy_version 543247 (0.0009) [2023-12-26 19:19:19,568][105692] Updated weights for policy 0, policy_version 543257 (0.0007) [2023-12-26 19:19:20,067][105620] Updated weights for policy 1, policy_version 543984 (0.0006) [2023-12-26 19:19:20,125][105620] Updated weights for policy 1, policy_version 543994 (0.0008) [2023-12-26 19:19:20,188][105620] Updated weights for policy 1, policy_version 544004 (0.0006) [2023-12-26 19:19:20,238][105692] Updated weights for policy 0, policy_version 543267 (0.0009) [2023-12-26 19:19:20,298][105692] Updated weights for policy 0, policy_version 543278 (0.0010) [2023-12-26 19:19:20,358][105692] Updated weights for policy 0, policy_version 543288 (0.0009) [2023-12-26 19:19:20,917][105620] Updated weights for policy 1, policy_version 544014 (0.0008) [2023-12-26 19:19:20,975][105620] Updated weights for policy 1, policy_version 544024 (0.0010) [2023-12-26 19:19:21,007][105692] Updated weights for policy 0, policy_version 543298 (0.0009) [2023-12-26 19:19:21,033][105620] Updated weights for policy 1, policy_version 544034 (0.0007) [2023-12-26 19:19:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 278380544. Throughput: 0: 9788.3, 1: 9746.6. Samples: 278376872. Policy #0 lag: (min: 5.0, avg: 15.3, max: 37.0) [2023-12-26 19:19:21,063][104569] Avg episode reward: [(0, '9091.391'), (1, '9257.561')] [2023-12-26 19:19:21,075][105692] Updated weights for policy 0, policy_version 543308 (0.0008) [2023-12-26 19:19:21,130][105692] Updated weights for policy 0, policy_version 543318 (0.0009) [2023-12-26 19:19:21,197][105692] Updated weights for policy 0, policy_version 543328 (0.0007) [2023-12-26 19:19:21,809][105620] Updated weights for policy 1, policy_version 544044 (0.0009) [2023-12-26 19:19:21,874][105620] Updated weights for policy 1, policy_version 544054 (0.0008) [2023-12-26 19:19:21,925][105620] Updated weights for policy 1, policy_version 544064 (0.0008) [2023-12-26 19:19:21,988][105692] Updated weights for policy 0, policy_version 543338 (0.0008) [2023-12-26 19:19:22,040][105692] Updated weights for policy 0, policy_version 543348 (0.0009) [2023-12-26 19:19:22,096][105692] Updated weights for policy 0, policy_version 543358 (0.0009) [2023-12-26 19:19:22,709][105620] Updated weights for policy 1, policy_version 544074 (0.0007) [2023-12-26 19:19:22,764][105620] Updated weights for policy 1, policy_version 544084 (0.0009) [2023-12-26 19:19:22,822][105620] Updated weights for policy 1, policy_version 544094 (0.0009) [2023-12-26 19:19:22,869][105692] Updated weights for policy 0, policy_version 543368 (0.0006) [2023-12-26 19:19:22,876][105620] Updated weights for policy 1, policy_version 544104 (0.0009) [2023-12-26 19:19:22,926][105692] Updated weights for policy 0, policy_version 543378 (0.0005) [2023-12-26 19:19:22,987][105692] Updated weights for policy 0, policy_version 543388 (0.0006) [2023-12-26 19:19:23,600][105620] Updated weights for policy 1, policy_version 544114 (0.0009) [2023-12-26 19:19:23,661][105620] Updated weights for policy 1, policy_version 544124 (0.0009) [2023-12-26 19:19:23,704][105620] Updated weights for policy 1, policy_version 544134 (0.0008) [2023-12-26 19:19:23,709][105692] Updated weights for policy 0, policy_version 543398 (0.0009) [2023-12-26 19:19:23,759][105692] Updated weights for policy 0, policy_version 543408 (0.0009) [2023-12-26 19:19:23,818][105692] Updated weights for policy 0, policy_version 543418 (0.0009) [2023-12-26 19:19:24,377][105620] Updated weights for policy 1, policy_version 544144 (0.0009) [2023-12-26 19:19:24,430][105620] Updated weights for policy 1, policy_version 544154 (0.0009) [2023-12-26 19:19:24,479][105620] Updated weights for policy 1, policy_version 544164 (0.0009) [2023-12-26 19:19:24,545][105692] Updated weights for policy 0, policy_version 543428 (0.0010) [2023-12-26 19:19:24,599][105692] Updated weights for policy 0, policy_version 543438 (0.0009) [2023-12-26 19:19:24,654][105692] Updated weights for policy 0, policy_version 543448 (0.0009) [2023-12-26 19:19:25,170][105620] Updated weights for policy 1, policy_version 544174 (0.0009) [2023-12-26 19:19:25,217][105620] Updated weights for policy 1, policy_version 544184 (0.0009) [2023-12-26 19:19:25,263][105620] Updated weights for policy 1, policy_version 544194 (0.0008) [2023-12-26 19:19:25,437][105692] Updated weights for policy 0, policy_version 543458 (0.0008) [2023-12-26 19:19:25,496][105692] Updated weights for policy 0, policy_version 543468 (0.0005) [2023-12-26 19:19:25,557][105692] Updated weights for policy 0, policy_version 543478 (0.0007) [2023-12-26 19:19:25,614][105692] Updated weights for policy 0, policy_version 543488 (0.0009) [2023-12-26 19:19:25,894][105620] Updated weights for policy 1, policy_version 544204 (0.0008) [2023-12-26 19:19:25,940][105620] Updated weights for policy 1, policy_version 544214 (0.0008) [2023-12-26 19:19:26,005][105620] Updated weights for policy 1, policy_version 544224 (0.0010) [2023-12-26 19:19:26,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 278487040. Throughput: 0: 9837.3, 1: 9736.6. Samples: 278493528. Policy #0 lag: (min: 5.0, avg: 15.3, max: 37.0) [2023-12-26 19:19:26,062][104569] Avg episode reward: [(0, '9092.852'), (1, '9259.672')] [2023-12-26 19:19:26,306][105692] Updated weights for policy 0, policy_version 543498 (0.0005) [2023-12-26 19:19:26,362][105692] Updated weights for policy 0, policy_version 543508 (0.0005) [2023-12-26 19:19:26,413][105692] Updated weights for policy 0, policy_version 543518 (0.0005) [2023-12-26 19:19:26,733][105620] Updated weights for policy 1, policy_version 544234 (0.0009) [2023-12-26 19:19:26,804][105620] Updated weights for policy 1, policy_version 544244 (0.0009) [2023-12-26 19:19:26,856][105620] Updated weights for policy 1, policy_version 544254 (0.0005) [2023-12-26 19:19:26,913][105620] Updated weights for policy 1, policy_version 544264 (0.0005) [2023-12-26 19:19:27,024][105692] Updated weights for policy 0, policy_version 543528 (0.0006) [2023-12-26 19:19:27,077][105692] Updated weights for policy 0, policy_version 543538 (0.0006) [2023-12-26 19:19:27,140][105692] Updated weights for policy 0, policy_version 543548 (0.0005) [2023-12-26 19:19:27,604][105620] Updated weights for policy 1, policy_version 544274 (0.0011) [2023-12-26 19:19:27,664][105620] Updated weights for policy 1, policy_version 544284 (0.0010) [2023-12-26 19:19:27,728][105620] Updated weights for policy 1, policy_version 544294 (0.0009) [2023-12-26 19:19:27,759][105692] Updated weights for policy 0, policy_version 543558 (0.0007) [2023-12-26 19:19:27,821][105692] Updated weights for policy 0, policy_version 543568 (0.0008) [2023-12-26 19:19:27,881][105692] Updated weights for policy 0, policy_version 543578 (0.0009) [2023-12-26 19:19:28,338][105620] Updated weights for policy 1, policy_version 544304 (0.0007) [2023-12-26 19:19:28,403][105620] Updated weights for policy 1, policy_version 544314 (0.0008) [2023-12-26 19:19:28,460][105620] Updated weights for policy 1, policy_version 544324 (0.0008) [2023-12-26 19:19:28,553][105692] Updated weights for policy 0, policy_version 543588 (0.0009) [2023-12-26 19:19:28,612][105692] Updated weights for policy 0, policy_version 543598 (0.0008) [2023-12-26 19:19:28,659][105692] Updated weights for policy 0, policy_version 543608 (0.0007) [2023-12-26 19:19:29,186][105620] Updated weights for policy 1, policy_version 544334 (0.0009) [2023-12-26 19:19:29,251][105620] Updated weights for policy 1, policy_version 544344 (0.0011) [2023-12-26 19:19:29,320][105620] Updated weights for policy 1, policy_version 544354 (0.0011) [2023-12-26 19:19:29,403][105692] Updated weights for policy 0, policy_version 543618 (0.0008) [2023-12-26 19:19:29,460][105692] Updated weights for policy 0, policy_version 543628 (0.0010) [2023-12-26 19:19:29,519][105692] Updated weights for policy 0, policy_version 543638 (0.0008) [2023-12-26 19:19:29,581][105692] Updated weights for policy 0, policy_version 543648 (0.0010) [2023-12-26 19:19:29,953][105620] Updated weights for policy 1, policy_version 544364 (0.0010) [2023-12-26 19:19:30,015][105620] Updated weights for policy 1, policy_version 544374 (0.0008) [2023-12-26 19:19:30,078][105620] Updated weights for policy 1, policy_version 544384 (0.0008) [2023-12-26 19:19:30,329][105692] Updated weights for policy 0, policy_version 543658 (0.0009) [2023-12-26 19:19:30,389][105692] Updated weights for policy 0, policy_version 543668 (0.0010) [2023-12-26 19:19:30,443][105692] Updated weights for policy 0, policy_version 543678 (0.0010) [2023-12-26 19:19:30,680][105620] Updated weights for policy 1, policy_version 544394 (0.0007) [2023-12-26 19:19:30,723][105620] Updated weights for policy 1, policy_version 544404 (0.0005) [2023-12-26 19:19:30,771][105620] Updated weights for policy 1, policy_version 544414 (0.0005) [2023-12-26 19:19:30,822][105620] Updated weights for policy 1, policy_version 544424 (0.0005) [2023-12-26 19:19:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 278585344. Throughput: 0: 9906.5, 1: 9797.7. Samples: 278554956. Policy #0 lag: (min: 5.0, avg: 15.3, max: 37.0) [2023-12-26 19:19:31,062][104569] Avg episode reward: [(0, '8996.180'), (1, '9352.115')] [2023-12-26 19:19:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000543680_139198464.pth... [2023-12-26 19:19:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000544424_139386880.pth... [2023-12-26 19:19:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000543240_139083776.pth [2023-12-26 19:19:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000542528_138903552.pth [2023-12-26 19:19:31,336][105692] Updated weights for policy 0, policy_version 543688 (0.0009) [2023-12-26 19:19:31,403][105692] Updated weights for policy 0, policy_version 543698 (0.0008) [2023-12-26 19:19:31,457][105620] Updated weights for policy 1, policy_version 544434 (0.0006) [2023-12-26 19:19:31,463][105692] Updated weights for policy 0, policy_version 543708 (0.0008) [2023-12-26 19:19:31,506][105620] Updated weights for policy 1, policy_version 544444 (0.0005) [2023-12-26 19:19:31,553][105620] Updated weights for policy 1, policy_version 544454 (0.0007) [2023-12-26 19:19:32,172][105620] Updated weights for policy 1, policy_version 544464 (0.0009) [2023-12-26 19:19:32,231][105620] Updated weights for policy 1, policy_version 544474 (0.0010) [2023-12-26 19:19:32,278][105692] Updated weights for policy 0, policy_version 543718 (0.0007) [2023-12-26 19:19:32,295][105620] Updated weights for policy 1, policy_version 544484 (0.0011) [2023-12-26 19:19:32,331][105692] Updated weights for policy 0, policy_version 543728 (0.0007) [2023-12-26 19:19:32,395][105692] Updated weights for policy 0, policy_version 543738 (0.0008) [2023-12-26 19:19:33,009][105620] Updated weights for policy 1, policy_version 544494 (0.0007) [2023-12-26 19:19:33,065][105620] Updated weights for policy 1, policy_version 544504 (0.0007) [2023-12-26 19:19:33,126][105620] Updated weights for policy 1, policy_version 544514 (0.0008) [2023-12-26 19:19:33,163][105692] Updated weights for policy 0, policy_version 543748 (0.0008) [2023-12-26 19:19:33,231][105692] Updated weights for policy 0, policy_version 543758 (0.0006) [2023-12-26 19:19:33,296][105692] Updated weights for policy 0, policy_version 543768 (0.0005) [2023-12-26 19:19:33,679][105620] Updated weights for policy 1, policy_version 544524 (0.0007) [2023-12-26 19:19:33,739][105620] Updated weights for policy 1, policy_version 544534 (0.0005) [2023-12-26 19:19:33,793][105620] Updated weights for policy 1, policy_version 544544 (0.0007) [2023-12-26 19:19:33,901][105692] Updated weights for policy 0, policy_version 543778 (0.0006) [2023-12-26 19:19:33,954][105692] Updated weights for policy 0, policy_version 543789 (0.0010) [2023-12-26 19:19:34,010][105692] Updated weights for policy 0, policy_version 543799 (0.0009) [2023-12-26 19:19:34,442][105620] Updated weights for policy 1, policy_version 544554 (0.0009) [2023-12-26 19:19:34,495][105620] Updated weights for policy 1, policy_version 544564 (0.0008) [2023-12-26 19:19:34,553][105620] Updated weights for policy 1, policy_version 544574 (0.0009) [2023-12-26 19:19:34,607][105620] Updated weights for policy 1, policy_version 544584 (0.0009) [2023-12-26 19:19:34,799][105692] Updated weights for policy 0, policy_version 543809 (0.0009) [2023-12-26 19:19:34,853][105692] Updated weights for policy 0, policy_version 543819 (0.0009) [2023-12-26 19:19:34,905][105692] Updated weights for policy 0, policy_version 543829 (0.0007) [2023-12-26 19:19:34,962][105692] Updated weights for policy 0, policy_version 543839 (0.0005) [2023-12-26 19:19:35,378][105620] Updated weights for policy 1, policy_version 544594 (0.0009) [2023-12-26 19:19:35,425][105620] Updated weights for policy 1, policy_version 544604 (0.0009) [2023-12-26 19:19:35,472][105620] Updated weights for policy 1, policy_version 544614 (0.0009) [2023-12-26 19:19:35,669][105692] Updated weights for policy 0, policy_version 543849 (0.0009) [2023-12-26 19:19:35,731][105692] Updated weights for policy 0, policy_version 543859 (0.0009) [2023-12-26 19:19:35,792][105692] Updated weights for policy 0, policy_version 543869 (0.0009) [2023-12-26 19:19:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 278683648. Throughput: 0: 9730.5, 1: 9929.6. Samples: 278673540. Policy #0 lag: (min: 5.0, avg: 15.3, max: 37.0) [2023-12-26 19:19:36,062][104569] Avg episode reward: [(0, '8903.088'), (1, '9352.184')] [2023-12-26 19:19:36,248][105620] Updated weights for policy 1, policy_version 544624 (0.0009) [2023-12-26 19:19:36,314][105620] Updated weights for policy 1, policy_version 544634 (0.0009) [2023-12-26 19:19:36,376][105620] Updated weights for policy 1, policy_version 544644 (0.0008) [2023-12-26 19:19:36,535][105692] Updated weights for policy 0, policy_version 543879 (0.0008) [2023-12-26 19:19:36,586][105692] Updated weights for policy 0, policy_version 543889 (0.0009) [2023-12-26 19:19:36,637][105692] Updated weights for policy 0, policy_version 543899 (0.0009) [2023-12-26 19:19:37,049][105620] Updated weights for policy 1, policy_version 544654 (0.0008) [2023-12-26 19:19:37,110][105620] Updated weights for policy 1, policy_version 544664 (0.0009) [2023-12-26 19:19:37,165][105620] Updated weights for policy 1, policy_version 544674 (0.0009) [2023-12-26 19:19:37,440][105692] Updated weights for policy 0, policy_version 543909 (0.0009) [2023-12-26 19:19:37,502][105692] Updated weights for policy 0, policy_version 543919 (0.0009) [2023-12-26 19:19:37,566][105692] Updated weights for policy 0, policy_version 543929 (0.0009) [2023-12-26 19:19:37,951][105620] Updated weights for policy 1, policy_version 544684 (0.0009) [2023-12-26 19:19:38,005][105620] Updated weights for policy 1, policy_version 544695 (0.0010) [2023-12-26 19:19:38,075][105620] Updated weights for policy 1, policy_version 544705 (0.0010) [2023-12-26 19:19:38,195][105692] Updated weights for policy 0, policy_version 543939 (0.0009) [2023-12-26 19:19:38,258][105692] Updated weights for policy 0, policy_version 543949 (0.0009) [2023-12-26 19:19:38,325][105692] Updated weights for policy 0, policy_version 543959 (0.0007) [2023-12-26 19:19:38,781][105620] Updated weights for policy 1, policy_version 544715 (0.0009) [2023-12-26 19:19:38,841][105620] Updated weights for policy 1, policy_version 544725 (0.0009) [2023-12-26 19:19:38,909][105620] Updated weights for policy 1, policy_version 544735 (0.0009) [2023-12-26 19:19:39,057][105692] Updated weights for policy 0, policy_version 543969 (0.0009) [2023-12-26 19:19:39,104][105692] Updated weights for policy 0, policy_version 543979 (0.0006) [2023-12-26 19:19:39,158][105692] Updated weights for policy 0, policy_version 543989 (0.0009) [2023-12-26 19:19:39,214][105692] Updated weights for policy 0, policy_version 543999 (0.0009) [2023-12-26 19:19:39,645][105620] Updated weights for policy 1, policy_version 544745 (0.0009) [2023-12-26 19:19:39,707][105620] Updated weights for policy 1, policy_version 544755 (0.0009) [2023-12-26 19:19:39,775][105620] Updated weights for policy 1, policy_version 544765 (0.0008) [2023-12-26 19:19:39,840][105620] Updated weights for policy 1, policy_version 544775 (0.0009) [2023-12-26 19:19:40,022][105692] Updated weights for policy 0, policy_version 544009 (0.0010) [2023-12-26 19:19:40,078][105692] Updated weights for policy 0, policy_version 544019 (0.0009) [2023-12-26 19:19:40,135][105692] Updated weights for policy 0, policy_version 544029 (0.0009) [2023-12-26 19:19:40,492][105620] Updated weights for policy 1, policy_version 544785 (0.0008) [2023-12-26 19:19:40,547][105620] Updated weights for policy 1, policy_version 544795 (0.0009) [2023-12-26 19:19:40,603][105620] Updated weights for policy 1, policy_version 544805 (0.0007) [2023-12-26 19:19:40,959][105692] Updated weights for policy 0, policy_version 544039 (0.0009) [2023-12-26 19:19:41,012][105692] Updated weights for policy 0, policy_version 544049 (0.0008) [2023-12-26 19:19:41,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 278773760. Throughput: 0: 9707.5, 1: 9924.6. Samples: 278786868. Policy #0 lag: (min: 5.0, avg: 15.3, max: 37.0) [2023-12-26 19:19:41,063][104569] Avg episode reward: [(0, '9175.910'), (1, '9352.153')] [2023-12-26 19:19:41,080][105692] Updated weights for policy 0, policy_version 544059 (0.0009) [2023-12-26 19:19:41,376][105620] Updated weights for policy 1, policy_version 544815 (0.0008) [2023-12-26 19:19:41,438][105620] Updated weights for policy 1, policy_version 544825 (0.0007) [2023-12-26 19:19:41,496][105620] Updated weights for policy 1, policy_version 544835 (0.0005) [2023-12-26 19:19:41,909][105692] Updated weights for policy 0, policy_version 544069 (0.0009) [2023-12-26 19:19:41,968][105692] Updated weights for policy 0, policy_version 544079 (0.0008) [2023-12-26 19:19:42,019][105692] Updated weights for policy 0, policy_version 544089 (0.0009) [2023-12-26 19:19:42,173][105620] Updated weights for policy 1, policy_version 544845 (0.0007) [2023-12-26 19:19:42,228][105620] Updated weights for policy 1, policy_version 544855 (0.0009) [2023-12-26 19:19:42,291][105620] Updated weights for policy 1, policy_version 544865 (0.0010) [2023-12-26 19:19:42,796][105692] Updated weights for policy 0, policy_version 544099 (0.0008) [2023-12-26 19:19:42,856][105692] Updated weights for policy 0, policy_version 544109 (0.0009) [2023-12-26 19:19:42,905][105692] Updated weights for policy 0, policy_version 544119 (0.0008) [2023-12-26 19:19:43,065][105620] Updated weights for policy 1, policy_version 544875 (0.0010) [2023-12-26 19:19:43,114][105620] Updated weights for policy 1, policy_version 544885 (0.0010) [2023-12-26 19:19:43,176][105620] Updated weights for policy 1, policy_version 544895 (0.0010) [2023-12-26 19:19:43,613][105692] Updated weights for policy 0, policy_version 544129 (0.0008) [2023-12-26 19:19:43,668][105692] Updated weights for policy 0, policy_version 544139 (0.0006) [2023-12-26 19:19:43,723][105692] Updated weights for policy 0, policy_version 544149 (0.0009) [2023-12-26 19:19:43,770][105692] Updated weights for policy 0, policy_version 544159 (0.0008) [2023-12-26 19:19:43,921][105620] Updated weights for policy 1, policy_version 544905 (0.0011) [2023-12-26 19:19:43,978][105620] Updated weights for policy 1, policy_version 544915 (0.0010) [2023-12-26 19:19:44,027][105620] Updated weights for policy 1, policy_version 544925 (0.0010) [2023-12-26 19:19:44,089][105620] Updated weights for policy 1, policy_version 544935 (0.0011) [2023-12-26 19:19:44,472][105692] Updated weights for policy 0, policy_version 544169 (0.0005) [2023-12-26 19:19:44,525][105692] Updated weights for policy 0, policy_version 544179 (0.0005) [2023-12-26 19:19:44,590][105692] Updated weights for policy 0, policy_version 544189 (0.0005) [2023-12-26 19:19:44,832][105620] Updated weights for policy 1, policy_version 544945 (0.0006) [2023-12-26 19:19:44,898][105620] Updated weights for policy 1, policy_version 544955 (0.0009) [2023-12-26 19:19:44,965][105620] Updated weights for policy 1, policy_version 544965 (0.0011) [2023-12-26 19:19:45,204][105692] Updated weights for policy 0, policy_version 544199 (0.0005) [2023-12-26 19:19:45,269][105692] Updated weights for policy 0, policy_version 544209 (0.0007) [2023-12-26 19:19:45,330][105692] Updated weights for policy 0, policy_version 544219 (0.0008) [2023-12-26 19:19:45,698][105620] Updated weights for policy 1, policy_version 544975 (0.0010) [2023-12-26 19:19:45,764][105620] Updated weights for policy 1, policy_version 544985 (0.0010) [2023-12-26 19:19:45,826][105620] Updated weights for policy 1, policy_version 544995 (0.0011) [2023-12-26 19:19:45,937][105692] Updated weights for policy 0, policy_version 544229 (0.0007) [2023-12-26 19:19:45,983][105692] Updated weights for policy 0, policy_version 544239 (0.0005) [2023-12-26 19:19:46,044][105692] Updated weights for policy 0, policy_version 544249 (0.0005) [2023-12-26 19:19:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 278872064. Throughput: 0: 9666.6, 1: 9938.7. Samples: 278842800. Policy #0 lag: (min: 5.0, avg: 15.3, max: 37.0) [2023-12-26 19:19:46,062][104569] Avg episode reward: [(0, '9176.592'), (1, '9078.362')] [2023-12-26 19:19:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000545000_139534336.pth... [2023-12-26 19:19:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000543848_139239424.pth [2023-12-26 19:19:46,084][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000544256_139345920.pth... [2023-12-26 19:19:46,088][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000543136_139059200.pth [2023-12-26 19:19:46,504][105620] Updated weights for policy 1, policy_version 545005 (0.0009) [2023-12-26 19:19:46,553][105620] Updated weights for policy 1, policy_version 545015 (0.0008) [2023-12-26 19:19:46,604][105620] Updated weights for policy 1, policy_version 545025 (0.0010) [2023-12-26 19:19:46,742][105692] Updated weights for policy 0, policy_version 544259 (0.0009) [2023-12-26 19:19:46,793][105692] Updated weights for policy 0, policy_version 544269 (0.0010) [2023-12-26 19:19:46,851][105692] Updated weights for policy 0, policy_version 544279 (0.0010) [2023-12-26 19:19:47,292][105620] Updated weights for policy 1, policy_version 545035 (0.0009) [2023-12-26 19:19:47,342][105620] Updated weights for policy 1, policy_version 545045 (0.0010) [2023-12-26 19:19:47,393][105620] Updated weights for policy 1, policy_version 545055 (0.0010) [2023-12-26 19:19:47,579][105692] Updated weights for policy 0, policy_version 544289 (0.0010) [2023-12-26 19:19:47,628][105692] Updated weights for policy 0, policy_version 544299 (0.0008) [2023-12-26 19:19:47,689][105692] Updated weights for policy 0, policy_version 544309 (0.0007) [2023-12-26 19:19:47,702][105585] KL-divergence is very high: 116.7916 [2023-12-26 19:19:47,737][105585] KL-divergence is very high: 262.1860 [2023-12-26 19:19:47,756][105692] Updated weights for policy 0, policy_version 544319 (0.0006) [2023-12-26 19:19:47,758][105585] KL-divergence is very high: 377.8859 [2023-12-26 19:19:48,042][105620] Updated weights for policy 1, policy_version 545065 (0.0010) [2023-12-26 19:19:48,108][105620] Updated weights for policy 1, policy_version 545075 (0.0011) [2023-12-26 19:19:48,164][105620] Updated weights for policy 1, policy_version 545085 (0.0011) [2023-12-26 19:19:48,223][105620] Updated weights for policy 1, policy_version 545095 (0.0011) [2023-12-26 19:19:48,343][105585] KL-divergence is very high: 295.0034 [2023-12-26 19:19:48,351][105585] KL-divergence is very high: 361.3625 [2023-12-26 19:19:48,396][105585] KL-divergence is very high: 291.3331 [2023-12-26 19:19:48,401][105692] Updated weights for policy 0, policy_version 544329 (0.0007) [2023-12-26 19:19:48,403][105585] KL-divergence is very high: 353.0106 [2023-12-26 19:19:48,449][105585] KL-divergence is very high: 258.6828 [2023-12-26 19:19:48,455][105585] KL-divergence is very high: 309.3590 [2023-12-26 19:19:48,470][105692] Updated weights for policy 0, policy_version 544339 (0.0008) [2023-12-26 19:19:48,502][105585] KL-divergence is very high: 201.4828 [2023-12-26 19:19:48,509][105585] KL-divergence is very high: 241.9350 [2023-12-26 19:19:48,534][105692] Updated weights for policy 0, policy_version 544349 (0.0008) [2023-12-26 19:19:48,971][105620] Updated weights for policy 1, policy_version 545105 (0.0010) [2023-12-26 19:19:49,022][105620] Updated weights for policy 1, policy_version 545115 (0.0010) [2023-12-26 19:19:49,077][105620] Updated weights for policy 1, policy_version 545125 (0.0010) [2023-12-26 19:19:49,208][105692] Updated weights for policy 0, policy_version 544359 (0.0006) [2023-12-26 19:19:49,269][105692] Updated weights for policy 0, policy_version 544369 (0.0008) [2023-12-26 19:19:49,328][105692] Updated weights for policy 0, policy_version 544379 (0.0008) [2023-12-26 19:19:49,806][105620] Updated weights for policy 1, policy_version 545135 (0.0009) [2023-12-26 19:19:49,867][105620] Updated weights for policy 1, policy_version 545145 (0.0009) [2023-12-26 19:19:49,919][105620] Updated weights for policy 1, policy_version 545155 (0.0009) [2023-12-26 19:19:49,967][105692] Updated weights for policy 0, policy_version 544389 (0.0009) [2023-12-26 19:19:50,028][105692] Updated weights for policy 0, policy_version 544399 (0.0008) [2023-12-26 19:19:50,090][105692] Updated weights for policy 0, policy_version 544409 (0.0010) [2023-12-26 19:19:50,682][105620] Updated weights for policy 1, policy_version 545165 (0.0009) [2023-12-26 19:19:50,735][105620] Updated weights for policy 1, policy_version 545175 (0.0010) [2023-12-26 19:19:50,794][105620] Updated weights for policy 1, policy_version 545185 (0.0010) [2023-12-26 19:19:50,860][105692] Updated weights for policy 0, policy_version 544419 (0.0009) [2023-12-26 19:19:50,916][105692] Updated weights for policy 0, policy_version 544429 (0.0011) [2023-12-26 19:19:50,968][105692] Updated weights for policy 0, policy_version 544439 (0.0010) [2023-12-26 19:19:51,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 278978560. Throughput: 0: 9666.9, 1: 9956.3. Samples: 278962700. Policy #0 lag: (min: 5.0, avg: 15.3, max: 37.0) [2023-12-26 19:19:51,062][104569] Avg episode reward: [(0, '8321.419'), (1, '8988.405')] [2023-12-26 19:19:51,571][105620] Updated weights for policy 1, policy_version 545195 (0.0010) [2023-12-26 19:19:51,641][105620] Updated weights for policy 1, policy_version 545205 (0.0011) [2023-12-26 19:19:51,668][105692] Updated weights for policy 0, policy_version 544449 (0.0010) [2023-12-26 19:19:51,701][105620] Updated weights for policy 1, policy_version 545215 (0.0011) [2023-12-26 19:19:51,730][105692] Updated weights for policy 0, policy_version 544459 (0.0011) [2023-12-26 19:19:51,785][105692] Updated weights for policy 0, policy_version 544469 (0.0011) [2023-12-26 19:19:51,847][105692] Updated weights for policy 0, policy_version 544479 (0.0010) [2023-12-26 19:19:52,393][105620] Updated weights for policy 1, policy_version 545225 (0.0009) [2023-12-26 19:19:52,448][105620] Updated weights for policy 1, policy_version 545235 (0.0008) [2023-12-26 19:19:52,505][105620] Updated weights for policy 1, policy_version 545245 (0.0007) [2023-12-26 19:19:52,518][105692] Updated weights for policy 0, policy_version 544489 (0.0009) [2023-12-26 19:19:52,561][105620] Updated weights for policy 1, policy_version 545255 (0.0006) [2023-12-26 19:19:52,570][105692] Updated weights for policy 0, policy_version 544500 (0.0010) [2023-12-26 19:19:52,618][105692] Updated weights for policy 0, policy_version 544510 (0.0010) [2023-12-26 19:19:53,275][105620] Updated weights for policy 1, policy_version 545265 (0.0005) [2023-12-26 19:19:53,336][105620] Updated weights for policy 1, policy_version 545275 (0.0006) [2023-12-26 19:19:53,353][105692] Updated weights for policy 0, policy_version 544520 (0.0007) [2023-12-26 19:19:53,398][105692] Updated weights for policy 0, policy_version 544530 (0.0007) [2023-12-26 19:19:53,400][105620] Updated weights for policy 1, policy_version 545285 (0.0005) [2023-12-26 19:19:53,442][105692] Updated weights for policy 0, policy_version 544540 (0.0006) [2023-12-26 19:19:53,955][105620] Updated weights for policy 1, policy_version 545295 (0.0007) [2023-12-26 19:19:54,020][105620] Updated weights for policy 1, policy_version 545305 (0.0010) [2023-12-26 19:19:54,082][105620] Updated weights for policy 1, policy_version 545315 (0.0010) [2023-12-26 19:19:54,084][105692] Updated weights for policy 0, policy_version 544550 (0.0006) [2023-12-26 19:19:54,140][105692] Updated weights for policy 0, policy_version 544560 (0.0006) [2023-12-26 19:19:54,195][105692] Updated weights for policy 0, policy_version 544570 (0.0005) [2023-12-26 19:19:54,794][105620] Updated weights for policy 1, policy_version 545325 (0.0010) [2023-12-26 19:19:54,845][105620] Updated weights for policy 1, policy_version 545335 (0.0010) [2023-12-26 19:19:54,880][105692] Updated weights for policy 0, policy_version 544580 (0.0006) [2023-12-26 19:19:54,890][105620] Updated weights for policy 1, policy_version 545345 (0.0010) [2023-12-26 19:19:54,933][105692] Updated weights for policy 0, policy_version 544590 (0.0006) [2023-12-26 19:19:54,989][105692] Updated weights for policy 0, policy_version 544600 (0.0008) [2023-12-26 19:19:55,667][105620] Updated weights for policy 1, policy_version 545355 (0.0010) [2023-12-26 19:19:55,721][105620] Updated weights for policy 1, policy_version 545365 (0.0010) [2023-12-26 19:19:55,759][105692] Updated weights for policy 0, policy_version 544610 (0.0007) [2023-12-26 19:19:55,779][105620] Updated weights for policy 1, policy_version 545375 (0.0010) [2023-12-26 19:19:55,813][105692] Updated weights for policy 0, policy_version 544620 (0.0005) [2023-12-26 19:19:55,867][105692] Updated weights for policy 0, policy_version 544630 (0.0008) [2023-12-26 19:19:55,925][105692] Updated weights for policy 0, policy_version 544640 (0.0008) [2023-12-26 19:19:56,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 279076864. Throughput: 0: 9680.7, 1: 9923.5. Samples: 279081508. Policy #0 lag: (min: 5.0, avg: 15.3, max: 37.0) [2023-12-26 19:19:56,063][104569] Avg episode reward: [(0, '7980.021'), (1, '9171.226')] [2023-12-26 19:19:56,452][105620] Updated weights for policy 1, policy_version 545385 (0.0010) [2023-12-26 19:19:56,500][105620] Updated weights for policy 1, policy_version 545395 (0.0010) [2023-12-26 19:19:56,548][105620] Updated weights for policy 1, policy_version 545405 (0.0010) [2023-12-26 19:19:56,599][105620] Updated weights for policy 1, policy_version 545415 (0.0009) [2023-12-26 19:19:56,649][105692] Updated weights for policy 0, policy_version 544650 (0.0008) [2023-12-26 19:19:56,710][105692] Updated weights for policy 0, policy_version 544660 (0.0008) [2023-12-26 19:19:56,764][105692] Updated weights for policy 0, policy_version 544670 (0.0007) [2023-12-26 19:19:57,356][105620] Updated weights for policy 1, policy_version 545425 (0.0010) [2023-12-26 19:19:57,400][105620] Updated weights for policy 1, policy_version 545435 (0.0010) [2023-12-26 19:19:57,466][105620] Updated weights for policy 1, policy_version 545445 (0.0010) [2023-12-26 19:19:57,509][105692] Updated weights for policy 0, policy_version 544680 (0.0006) [2023-12-26 19:19:57,568][105692] Updated weights for policy 0, policy_version 544690 (0.0005) [2023-12-26 19:19:57,621][105692] Updated weights for policy 0, policy_version 544700 (0.0005) [2023-12-26 19:19:58,193][105620] Updated weights for policy 1, policy_version 545455 (0.0009) [2023-12-26 19:19:58,261][105620] Updated weights for policy 1, policy_version 545465 (0.0009) [2023-12-26 19:19:58,312][105692] Updated weights for policy 0, policy_version 544710 (0.0007) [2023-12-26 19:19:58,333][105620] Updated weights for policy 1, policy_version 545475 (0.0009) [2023-12-26 19:19:58,382][105692] Updated weights for policy 0, policy_version 544720 (0.0008) [2023-12-26 19:19:58,455][105692] Updated weights for policy 0, policy_version 544730 (0.0008) [2023-12-26 19:19:59,136][105620] Updated weights for policy 1, policy_version 545485 (0.0010) [2023-12-26 19:19:59,180][105620] Updated weights for policy 1, policy_version 545495 (0.0010) [2023-12-26 19:19:59,236][105620] Updated weights for policy 1, policy_version 545505 (0.0010) [2023-12-26 19:19:59,249][105692] Updated weights for policy 0, policy_version 544740 (0.0008) [2023-12-26 19:19:59,317][105692] Updated weights for policy 0, policy_version 544750 (0.0007) [2023-12-26 19:19:59,377][105692] Updated weights for policy 0, policy_version 544760 (0.0008) [2023-12-26 19:19:59,977][105620] Updated weights for policy 1, policy_version 545515 (0.0010) [2023-12-26 19:20:00,039][105620] Updated weights for policy 1, policy_version 545525 (0.0009) [2023-12-26 19:20:00,101][105620] Updated weights for policy 1, policy_version 545535 (0.0010) [2023-12-26 19:20:00,119][105692] Updated weights for policy 0, policy_version 544770 (0.0007) [2023-12-26 19:20:00,169][105692] Updated weights for policy 0, policy_version 544780 (0.0007) [2023-12-26 19:20:00,217][105692] Updated weights for policy 0, policy_version 544790 (0.0008) [2023-12-26 19:20:00,276][105692] Updated weights for policy 0, policy_version 544800 (0.0008) [2023-12-26 19:20:00,766][105620] Updated weights for policy 1, policy_version 545545 (0.0010) [2023-12-26 19:20:00,832][105620] Updated weights for policy 1, policy_version 545555 (0.0011) [2023-12-26 19:20:00,894][105620] Updated weights for policy 1, policy_version 545565 (0.0010) [2023-12-26 19:20:00,959][105620] Updated weights for policy 1, policy_version 545575 (0.0009) [2023-12-26 19:20:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 279166976. Throughput: 0: 9694.3, 1: 9847.4. Samples: 279138556. Policy #0 lag: (min: 5.0, avg: 15.3, max: 37.0) [2023-12-26 19:20:01,063][104569] Avg episode reward: [(0, '8393.230'), (1, '9261.680')] [2023-12-26 19:20:01,063][105692] Updated weights for policy 0, policy_version 544810 (0.0009) [2023-12-26 19:20:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000545576_139681792.pth... [2023-12-26 19:20:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000544424_139386880.pth [2023-12-26 19:20:01,115][105692] Updated weights for policy 0, policy_version 544820 (0.0006) [2023-12-26 19:20:01,182][105692] Updated weights for policy 0, policy_version 544830 (0.0009) [2023-12-26 19:20:01,190][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000544832_139493376.pth... [2023-12-26 19:20:01,194][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000543680_139198464.pth [2023-12-26 19:20:01,685][105620] Updated weights for policy 1, policy_version 545585 (0.0007) [2023-12-26 19:20:01,753][105620] Updated weights for policy 1, policy_version 545595 (0.0012) [2023-12-26 19:20:01,810][105620] Updated weights for policy 1, policy_version 545605 (0.0010) [2023-12-26 19:20:01,957][105692] Updated weights for policy 0, policy_version 544840 (0.0008) [2023-12-26 19:20:02,011][105692] Updated weights for policy 0, policy_version 544850 (0.0008) [2023-12-26 19:20:02,059][105692] Updated weights for policy 0, policy_version 544860 (0.0008) [2023-12-26 19:20:02,525][105620] Updated weights for policy 1, policy_version 545615 (0.0009) [2023-12-26 19:20:02,574][105620] Updated weights for policy 1, policy_version 545625 (0.0009) [2023-12-26 19:20:02,626][105620] Updated weights for policy 1, policy_version 545635 (0.0009) [2023-12-26 19:20:02,815][105692] Updated weights for policy 0, policy_version 544870 (0.0010) [2023-12-26 19:20:02,865][105692] Updated weights for policy 0, policy_version 544880 (0.0011) [2023-12-26 19:20:02,927][105692] Updated weights for policy 0, policy_version 544890 (0.0008) [2023-12-26 19:20:03,472][105692] Updated weights for policy 0, policy_version 544900 (0.0005) [2023-12-26 19:20:03,475][105620] Updated weights for policy 1, policy_version 545645 (0.0009) [2023-12-26 19:20:03,522][105692] Updated weights for policy 0, policy_version 544910 (0.0005) [2023-12-26 19:20:03,523][105620] Updated weights for policy 1, policy_version 545655 (0.0009) [2023-12-26 19:20:03,569][105692] Updated weights for policy 0, policy_version 544920 (0.0010) [2023-12-26 19:20:03,578][105620] Updated weights for policy 1, policy_version 545665 (0.0006) [2023-12-26 19:20:04,243][105692] Updated weights for policy 0, policy_version 544930 (0.0010) [2023-12-26 19:20:04,305][105692] Updated weights for policy 0, policy_version 544940 (0.0011) [2023-12-26 19:20:04,367][105620] Updated weights for policy 1, policy_version 545675 (0.0006) [2023-12-26 19:20:04,369][105692] Updated weights for policy 0, policy_version 544950 (0.0011) [2023-12-26 19:20:04,430][105692] Updated weights for policy 0, policy_version 544960 (0.0010) [2023-12-26 19:20:04,430][105620] Updated weights for policy 1, policy_version 545685 (0.0006) [2023-12-26 19:20:04,487][105620] Updated weights for policy 1, policy_version 545695 (0.0009) [2023-12-26 19:20:05,079][105692] Updated weights for policy 0, policy_version 544970 (0.0009) [2023-12-26 19:20:05,131][105692] Updated weights for policy 0, policy_version 544980 (0.0005) [2023-12-26 19:20:05,186][105692] Updated weights for policy 0, policy_version 544990 (0.0005) [2023-12-26 19:20:05,262][105620] Updated weights for policy 1, policy_version 545705 (0.0008) [2023-12-26 19:20:05,313][105620] Updated weights for policy 1, policy_version 545715 (0.0005) [2023-12-26 19:20:05,369][105620] Updated weights for policy 1, policy_version 545725 (0.0005) [2023-12-26 19:20:05,425][105620] Updated weights for policy 1, policy_version 545735 (0.0005) [2023-12-26 19:20:05,867][105692] Updated weights for policy 0, policy_version 545000 (0.0005) [2023-12-26 19:20:05,915][105692] Updated weights for policy 0, policy_version 545010 (0.0005) [2023-12-26 19:20:05,961][105692] Updated weights for policy 0, policy_version 545020 (0.0005) [2023-12-26 19:20:06,050][105620] Updated weights for policy 1, policy_version 545745 (0.0009) [2023-12-26 19:20:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 279265280. Throughput: 0: 9697.3, 1: 9767.5. Samples: 279252788. Policy #0 lag: (min: 5.0, avg: 15.3, max: 37.0) [2023-12-26 19:20:06,063][104569] Avg episode reward: [(0, '8752.644'), (1, '9081.639')] [2023-12-26 19:20:06,114][105620] Updated weights for policy 1, policy_version 545755 (0.0008) [2023-12-26 19:20:06,179][105620] Updated weights for policy 1, policy_version 545765 (0.0009) [2023-12-26 19:20:06,601][105692] Updated weights for policy 0, policy_version 545030 (0.0008) [2023-12-26 19:20:06,663][105692] Updated weights for policy 0, policy_version 545040 (0.0010) [2023-12-26 19:20:06,722][105692] Updated weights for policy 0, policy_version 545050 (0.0011) [2023-12-26 19:20:06,971][105620] Updated weights for policy 1, policy_version 545775 (0.0008) [2023-12-26 19:20:07,035][105620] Updated weights for policy 1, policy_version 545785 (0.0008) [2023-12-26 19:20:07,105][105620] Updated weights for policy 1, policy_version 545795 (0.0008) [2023-12-26 19:20:07,453][105692] Updated weights for policy 0, policy_version 545060 (0.0011) [2023-12-26 19:20:07,508][105692] Updated weights for policy 0, policy_version 545070 (0.0010) [2023-12-26 19:20:07,567][105692] Updated weights for policy 0, policy_version 545080 (0.0010) [2023-12-26 19:20:07,710][105620] Updated weights for policy 1, policy_version 545805 (0.0007) [2023-12-26 19:20:07,770][105620] Updated weights for policy 1, policy_version 545815 (0.0009) [2023-12-26 19:20:07,839][105620] Updated weights for policy 1, policy_version 545825 (0.0010) [2023-12-26 19:20:08,219][105692] Updated weights for policy 0, policy_version 545090 (0.0010) [2023-12-26 19:20:08,267][105692] Updated weights for policy 0, policy_version 545100 (0.0010) [2023-12-26 19:20:08,312][105692] Updated weights for policy 0, policy_version 545110 (0.0010) [2023-12-26 19:20:08,371][105692] Updated weights for policy 0, policy_version 545120 (0.0011) [2023-12-26 19:20:08,615][105620] Updated weights for policy 1, policy_version 545835 (0.0008) [2023-12-26 19:20:08,677][105620] Updated weights for policy 1, policy_version 545845 (0.0007) [2023-12-26 19:20:08,739][105620] Updated weights for policy 1, policy_version 545855 (0.0011) [2023-12-26 19:20:09,033][105692] Updated weights for policy 0, policy_version 545130 (0.0009) [2023-12-26 19:20:09,098][105692] Updated weights for policy 0, policy_version 545140 (0.0010) [2023-12-26 19:20:09,151][105692] Updated weights for policy 0, policy_version 545150 (0.0011) [2023-12-26 19:20:09,361][105620] Updated weights for policy 1, policy_version 545865 (0.0007) [2023-12-26 19:20:09,430][105620] Updated weights for policy 1, policy_version 545875 (0.0009) [2023-12-26 19:20:09,491][105620] Updated weights for policy 1, policy_version 545885 (0.0010) [2023-12-26 19:20:09,545][105620] Updated weights for policy 1, policy_version 545895 (0.0005) [2023-12-26 19:20:09,839][105692] Updated weights for policy 0, policy_version 545160 (0.0007) [2023-12-26 19:20:09,906][105692] Updated weights for policy 0, policy_version 545170 (0.0006) [2023-12-26 19:20:09,968][105692] Updated weights for policy 0, policy_version 545180 (0.0011) [2023-12-26 19:20:10,293][105620] Updated weights for policy 1, policy_version 545905 (0.0009) [2023-12-26 19:20:10,363][105620] Updated weights for policy 1, policy_version 545915 (0.0008) [2023-12-26 19:20:10,424][105620] Updated weights for policy 1, policy_version 545925 (0.0008) [2023-12-26 19:20:10,638][105692] Updated weights for policy 0, policy_version 545190 (0.0008) [2023-12-26 19:20:10,695][105692] Updated weights for policy 0, policy_version 545200 (0.0005) [2023-12-26 19:20:10,749][105692] Updated weights for policy 0, policy_version 545210 (0.0005) [2023-12-26 19:20:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 279363584. Throughput: 0: 9795.6, 1: 9744.2. Samples: 279372816. Policy #0 lag: (min: 5.0, avg: 15.3, max: 37.0) [2023-12-26 19:20:11,062][104569] Avg episode reward: [(0, '9007.606'), (1, '9172.992')] [2023-12-26 19:20:11,249][105620] Updated weights for policy 1, policy_version 545935 (0.0010) [2023-12-26 19:20:11,301][105620] Updated weights for policy 1, policy_version 545945 (0.0009) [2023-12-26 19:20:11,365][105620] Updated weights for policy 1, policy_version 545955 (0.0009) [2023-12-26 19:20:11,460][105692] Updated weights for policy 0, policy_version 545220 (0.0010) [2023-12-26 19:20:11,513][105692] Updated weights for policy 0, policy_version 545230 (0.0011) [2023-12-26 19:20:11,561][105692] Updated weights for policy 0, policy_version 545240 (0.0011) [2023-12-26 19:20:12,108][105620] Updated weights for policy 1, policy_version 545965 (0.0006) [2023-12-26 19:20:12,161][105620] Updated weights for policy 1, policy_version 545975 (0.0006) [2023-12-26 19:20:12,216][105620] Updated weights for policy 1, policy_version 545985 (0.0006) [2023-12-26 19:20:12,270][105692] Updated weights for policy 0, policy_version 545250 (0.0010) [2023-12-26 19:20:12,341][105692] Updated weights for policy 0, policy_version 545260 (0.0008) [2023-12-26 19:20:12,407][105692] Updated weights for policy 0, policy_version 545270 (0.0010) [2023-12-26 19:20:12,464][105692] Updated weights for policy 0, policy_version 545280 (0.0009) [2023-12-26 19:20:12,942][105620] Updated weights for policy 1, policy_version 545995 (0.0008) [2023-12-26 19:20:12,989][105620] Updated weights for policy 1, policy_version 546005 (0.0010) [2023-12-26 19:20:13,039][105620] Updated weights for policy 1, policy_version 546015 (0.0011) [2023-12-26 19:20:13,134][105692] Updated weights for policy 0, policy_version 545290 (0.0005) [2023-12-26 19:20:13,193][105692] Updated weights for policy 0, policy_version 545300 (0.0006) [2023-12-26 19:20:13,244][105692] Updated weights for policy 0, policy_version 545310 (0.0005) [2023-12-26 19:20:13,655][105620] Updated weights for policy 1, policy_version 546025 (0.0011) [2023-12-26 19:20:13,713][105620] Updated weights for policy 1, policy_version 546035 (0.0007) [2023-12-26 19:20:13,769][105620] Updated weights for policy 1, policy_version 546045 (0.0005) [2023-12-26 19:20:13,824][105692] Updated weights for policy 0, policy_version 545320 (0.0010) [2023-12-26 19:20:13,828][105620] Updated weights for policy 1, policy_version 546055 (0.0005) [2023-12-26 19:20:13,868][105692] Updated weights for policy 0, policy_version 545330 (0.0010) [2023-12-26 19:20:13,916][105692] Updated weights for policy 0, policy_version 545340 (0.0010) [2023-12-26 19:20:14,440][105620] Updated weights for policy 1, policy_version 546065 (0.0010) [2023-12-26 19:20:14,500][105620] Updated weights for policy 1, policy_version 546075 (0.0011) [2023-12-26 19:20:14,564][105620] Updated weights for policy 1, policy_version 546085 (0.0011) [2023-12-26 19:20:14,582][105692] Updated weights for policy 0, policy_version 545350 (0.0010) [2023-12-26 19:20:14,643][105692] Updated weights for policy 0, policy_version 545360 (0.0010) [2023-12-26 19:20:14,707][105692] Updated weights for policy 0, policy_version 545370 (0.0008) [2023-12-26 19:20:15,364][105620] Updated weights for policy 1, policy_version 546095 (0.0011) [2023-12-26 19:20:15,394][105692] Updated weights for policy 0, policy_version 545380 (0.0007) [2023-12-26 19:20:15,420][105620] Updated weights for policy 1, policy_version 546105 (0.0006) [2023-12-26 19:20:15,450][105692] Updated weights for policy 0, policy_version 545390 (0.0011) [2023-12-26 19:20:15,476][105620] Updated weights for policy 1, policy_version 546115 (0.0006) [2023-12-26 19:20:15,498][105692] Updated weights for policy 0, policy_version 545400 (0.0010) [2023-12-26 19:20:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 279461888. Throughput: 0: 9777.8, 1: 9758.0. Samples: 279434072. Policy #0 lag: (min: 5.0, avg: 15.3, max: 37.0) [2023-12-26 19:20:16,063][104569] Avg episode reward: [(0, '9264.753'), (1, '9167.080')] [2023-12-26 19:20:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000545408_139640832.pth... [2023-12-26 19:20:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000546120_139821056.pth... [2023-12-26 19:20:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000544256_139345920.pth [2023-12-26 19:20:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000545000_139534336.pth [2023-12-26 19:20:16,197][105692] Updated weights for policy 0, policy_version 545410 (0.0010) [2023-12-26 19:20:16,202][105620] Updated weights for policy 1, policy_version 546125 (0.0007) [2023-12-26 19:20:16,256][105692] Updated weights for policy 0, policy_version 545420 (0.0010) [2023-12-26 19:20:16,262][105620] Updated weights for policy 1, policy_version 546135 (0.0006) [2023-12-26 19:20:16,312][105620] Updated weights for policy 1, policy_version 546145 (0.0008) [2023-12-26 19:20:16,318][105692] Updated weights for policy 0, policy_version 545430 (0.0010) [2023-12-26 19:20:16,381][105692] Updated weights for policy 0, policy_version 545440 (0.0010) [2023-12-26 19:20:17,080][105620] Updated weights for policy 1, policy_version 546155 (0.0006) [2023-12-26 19:20:17,130][105620] Updated weights for policy 1, policy_version 546165 (0.0007) [2023-12-26 19:20:17,135][105692] Updated weights for policy 0, policy_version 545450 (0.0011) [2023-12-26 19:20:17,188][105620] Updated weights for policy 1, policy_version 546175 (0.0007) [2023-12-26 19:20:17,195][105692] Updated weights for policy 0, policy_version 545460 (0.0006) [2023-12-26 19:20:17,247][105692] Updated weights for policy 0, policy_version 545470 (0.0008) [2023-12-26 19:20:17,775][105620] Updated weights for policy 1, policy_version 546185 (0.0006) [2023-12-26 19:20:17,834][105620] Updated weights for policy 1, policy_version 546195 (0.0011) [2023-12-26 19:20:17,895][105620] Updated weights for policy 1, policy_version 546205 (0.0009) [2023-12-26 19:20:17,952][105692] Updated weights for policy 0, policy_version 545480 (0.0006) [2023-12-26 19:20:17,953][105620] Updated weights for policy 1, policy_version 546215 (0.0007) [2023-12-26 19:20:18,009][105692] Updated weights for policy 0, policy_version 545490 (0.0007) [2023-12-26 19:20:18,061][105692] Updated weights for policy 0, policy_version 545501 (0.0009) [2023-12-26 19:20:18,582][105620] Updated weights for policy 1, policy_version 546225 (0.0008) [2023-12-26 19:20:18,663][105620] Updated weights for policy 1, policy_version 546235 (0.0008) [2023-12-26 19:20:18,724][105692] Updated weights for policy 0, policy_version 545511 (0.0009) [2023-12-26 19:20:18,726][105620] Updated weights for policy 1, policy_version 546245 (0.0006) [2023-12-26 19:20:18,781][105692] Updated weights for policy 0, policy_version 545521 (0.0008) [2023-12-26 19:20:18,839][105692] Updated weights for policy 0, policy_version 545531 (0.0010) [2023-12-26 19:20:19,414][105620] Updated weights for policy 1, policy_version 546255 (0.0009) [2023-12-26 19:20:19,476][105620] Updated weights for policy 1, policy_version 546265 (0.0009) [2023-12-26 19:20:19,533][105620] Updated weights for policy 1, policy_version 546275 (0.0009) [2023-12-26 19:20:19,650][105692] Updated weights for policy 0, policy_version 545541 (0.0008) [2023-12-26 19:20:19,718][105692] Updated weights for policy 0, policy_version 545551 (0.0006) [2023-12-26 19:20:19,777][105692] Updated weights for policy 0, policy_version 545561 (0.0009) [2023-12-26 19:20:20,313][105620] Updated weights for policy 1, policy_version 546285 (0.0009) [2023-12-26 19:20:20,382][105620] Updated weights for policy 1, policy_version 546295 (0.0010) [2023-12-26 19:20:20,443][105620] Updated weights for policy 1, policy_version 546305 (0.0009) [2023-12-26 19:20:20,474][105692] Updated weights for policy 0, policy_version 545571 (0.0009) [2023-12-26 19:20:20,537][105692] Updated weights for policy 0, policy_version 545581 (0.0008) [2023-12-26 19:20:20,609][105692] Updated weights for policy 0, policy_version 545591 (0.0008) [2023-12-26 19:20:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 279560192. Throughput: 0: 9875.3, 1: 9657.0. Samples: 279552496. Policy #0 lag: (min: 5.0, avg: 15.3, max: 37.0) [2023-12-26 19:20:21,063][104569] Avg episode reward: [(0, '8815.054'), (1, '8984.151')] [2023-12-26 19:20:21,273][105620] Updated weights for policy 1, policy_version 546315 (0.0008) [2023-12-26 19:20:21,292][105692] Updated weights for policy 0, policy_version 545601 (0.0008) [2023-12-26 19:20:21,333][105620] Updated weights for policy 1, policy_version 546325 (0.0008) [2023-12-26 19:20:21,354][105692] Updated weights for policy 0, policy_version 545611 (0.0008) [2023-12-26 19:20:21,403][105620] Updated weights for policy 1, policy_version 546335 (0.0009) [2023-12-26 19:20:21,417][105692] Updated weights for policy 0, policy_version 545621 (0.0007) [2023-12-26 19:20:21,478][105692] Updated weights for policy 0, policy_version 545631 (0.0006) [2023-12-26 19:20:22,140][105620] Updated weights for policy 1, policy_version 546345 (0.0010) [2023-12-26 19:20:22,196][105620] Updated weights for policy 1, policy_version 546355 (0.0008) [2023-12-26 19:20:22,241][105692] Updated weights for policy 0, policy_version 545641 (0.0006) [2023-12-26 19:20:22,263][105620] Updated weights for policy 1, policy_version 546365 (0.0006) [2023-12-26 19:20:22,305][105692] Updated weights for policy 0, policy_version 545651 (0.0007) [2023-12-26 19:20:22,333][105620] Updated weights for policy 1, policy_version 546375 (0.0008) [2023-12-26 19:20:22,378][105692] Updated weights for policy 0, policy_version 545661 (0.0007) [2023-12-26 19:20:22,948][105620] Updated weights for policy 1, policy_version 546385 (0.0008) [2023-12-26 19:20:23,015][105620] Updated weights for policy 1, policy_version 546395 (0.0009) [2023-12-26 19:20:23,071][105620] Updated weights for policy 1, policy_version 546405 (0.0009) [2023-12-26 19:20:23,105][105692] Updated weights for policy 0, policy_version 545671 (0.0008) [2023-12-26 19:20:23,153][105692] Updated weights for policy 0, policy_version 545681 (0.0009) [2023-12-26 19:20:23,201][105692] Updated weights for policy 0, policy_version 545691 (0.0008) [2023-12-26 19:20:23,732][105620] Updated weights for policy 1, policy_version 546415 (0.0009) [2023-12-26 19:20:23,790][105620] Updated weights for policy 1, policy_version 546425 (0.0009) [2023-12-26 19:20:23,856][105620] Updated weights for policy 1, policy_version 546435 (0.0007) [2023-12-26 19:20:24,044][105692] Updated weights for policy 0, policy_version 545701 (0.0009) [2023-12-26 19:20:24,090][105692] Updated weights for policy 0, policy_version 545711 (0.0008) [2023-12-26 19:20:24,141][105692] Updated weights for policy 0, policy_version 545721 (0.0009) [2023-12-26 19:20:24,526][105620] Updated weights for policy 1, policy_version 546445 (0.0007) [2023-12-26 19:20:24,587][105620] Updated weights for policy 1, policy_version 546455 (0.0009) [2023-12-26 19:20:24,649][105620] Updated weights for policy 1, policy_version 546465 (0.0009) [2023-12-26 19:20:24,870][105692] Updated weights for policy 0, policy_version 545731 (0.0008) [2023-12-26 19:20:24,930][105692] Updated weights for policy 0, policy_version 545741 (0.0006) [2023-12-26 19:20:24,976][105692] Updated weights for policy 0, policy_version 545751 (0.0008) [2023-12-26 19:20:25,423][105620] Updated weights for policy 1, policy_version 546475 (0.0009) [2023-12-26 19:20:25,473][105620] Updated weights for policy 1, policy_version 546485 (0.0009) [2023-12-26 19:20:25,519][105620] Updated weights for policy 1, policy_version 546495 (0.0009) [2023-12-26 19:20:25,685][105692] Updated weights for policy 0, policy_version 545761 (0.0009) [2023-12-26 19:20:25,731][105692] Updated weights for policy 0, policy_version 545771 (0.0008) [2023-12-26 19:20:25,732][105585] KL-divergence is very high: 101.1453 [2023-12-26 19:20:25,771][105585] KL-divergence is very high: 138.4372 [2023-12-26 19:20:25,781][105692] Updated weights for policy 0, policy_version 545781 (0.0008) [2023-12-26 19:20:25,809][105585] KL-divergence is very high: 111.8382 [2023-12-26 19:20:25,848][105692] Updated weights for policy 0, policy_version 545791 (0.0009) [2023-12-26 19:20:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 279658496. Throughput: 0: 9885.6, 1: 9657.3. Samples: 279666296. Policy #0 lag: (min: 28.0, avg: 41.2, max: 60.0) [2023-12-26 19:20:26,062][104569] Avg episode reward: [(0, '8723.872'), (1, '9259.572')] [2023-12-26 19:20:26,261][105620] Updated weights for policy 1, policy_version 546505 (0.0009) [2023-12-26 19:20:26,306][105620] Updated weights for policy 1, policy_version 546515 (0.0007) [2023-12-26 19:20:26,354][105620] Updated weights for policy 1, policy_version 546525 (0.0008) [2023-12-26 19:20:26,402][105620] Updated weights for policy 1, policy_version 546535 (0.0008) [2023-12-26 19:20:26,607][105692] Updated weights for policy 0, policy_version 545801 (0.0006) [2023-12-26 19:20:26,655][105692] Updated weights for policy 0, policy_version 545811 (0.0005) [2023-12-26 19:20:26,718][105692] Updated weights for policy 0, policy_version 545821 (0.0007) [2023-12-26 19:20:27,018][105620] Updated weights for policy 1, policy_version 546545 (0.0006) [2023-12-26 19:20:27,065][105620] Updated weights for policy 1, policy_version 546555 (0.0005) [2023-12-26 19:20:27,120][105620] Updated weights for policy 1, policy_version 546565 (0.0005) [2023-12-26 19:20:27,334][105692] Updated weights for policy 0, policy_version 545831 (0.0006) [2023-12-26 19:20:27,391][105692] Updated weights for policy 0, policy_version 545841 (0.0005) [2023-12-26 19:20:27,448][105692] Updated weights for policy 0, policy_version 545851 (0.0007) [2023-12-26 19:20:27,648][105620] Updated weights for policy 1, policy_version 546575 (0.0005) [2023-12-26 19:20:27,710][105620] Updated weights for policy 1, policy_version 546585 (0.0006) [2023-12-26 19:20:27,772][105620] Updated weights for policy 1, policy_version 546595 (0.0008) [2023-12-26 19:20:28,066][105692] Updated weights for policy 0, policy_version 545861 (0.0008) [2023-12-26 19:20:28,127][105692] Updated weights for policy 0, policy_version 545871 (0.0005) [2023-12-26 19:20:28,177][105692] Updated weights for policy 0, policy_version 545881 (0.0010) [2023-12-26 19:20:28,479][105620] Updated weights for policy 1, policy_version 546605 (0.0009) [2023-12-26 19:20:28,535][105620] Updated weights for policy 1, policy_version 546615 (0.0011) [2023-12-26 19:20:28,588][105620] Updated weights for policy 1, policy_version 546625 (0.0010) [2023-12-26 19:20:28,894][105692] Updated weights for policy 0, policy_version 545891 (0.0010) [2023-12-26 19:20:28,957][105692] Updated weights for policy 0, policy_version 545901 (0.0011) [2023-12-26 19:20:29,015][105692] Updated weights for policy 0, policy_version 545911 (0.0010) [2023-12-26 19:20:29,328][105620] Updated weights for policy 1, policy_version 546635 (0.0010) [2023-12-26 19:20:29,388][105620] Updated weights for policy 1, policy_version 546645 (0.0011) [2023-12-26 19:20:29,435][105620] Updated weights for policy 1, policy_version 546655 (0.0010) [2023-12-26 19:20:29,745][105692] Updated weights for policy 0, policy_version 545921 (0.0010) [2023-12-26 19:20:29,793][105692] Updated weights for policy 0, policy_version 545931 (0.0008) [2023-12-26 19:20:29,852][105692] Updated weights for policy 0, policy_version 545941 (0.0008) [2023-12-26 19:20:29,910][105692] Updated weights for policy 0, policy_version 545951 (0.0006) [2023-12-26 19:20:30,209][105620] Updated weights for policy 1, policy_version 546665 (0.0010) [2023-12-26 19:20:30,267][105620] Updated weights for policy 1, policy_version 546675 (0.0010) [2023-12-26 19:20:30,327][105620] Updated weights for policy 1, policy_version 546685 (0.0010) [2023-12-26 19:20:30,390][105620] Updated weights for policy 1, policy_version 546695 (0.0010) [2023-12-26 19:20:30,614][105692] Updated weights for policy 0, policy_version 545961 (0.0009) [2023-12-26 19:20:30,672][105692] Updated weights for policy 0, policy_version 545971 (0.0008) [2023-12-26 19:20:30,724][105692] Updated weights for policy 0, policy_version 545981 (0.0010) [2023-12-26 19:20:30,991][105620] Updated weights for policy 1, policy_version 546705 (0.0006) [2023-12-26 19:20:31,055][105620] Updated weights for policy 1, policy_version 546716 (0.0007) [2023-12-26 19:20:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 279756800. Throughput: 0: 9957.2, 1: 9738.4. Samples: 279729104. Policy #0 lag: (min: 28.0, avg: 41.2, max: 60.0) [2023-12-26 19:20:31,062][104569] Avg episode reward: [(0, '8814.277'), (1, '9352.296')] [2023-12-26 19:20:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000545984_139788288.pth... [2023-12-26 19:20:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000544832_139493376.pth [2023-12-26 19:20:31,112][105620] Updated weights for policy 1, policy_version 546726 (0.0006) [2023-12-26 19:20:31,123][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000546728_139976704.pth... [2023-12-26 19:20:31,127][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000545576_139681792.pth [2023-12-26 19:20:31,588][105692] Updated weights for policy 0, policy_version 545992 (0.0009) [2023-12-26 19:20:31,651][105692] Updated weights for policy 0, policy_version 546002 (0.0009) [2023-12-26 19:20:31,713][105692] Updated weights for policy 0, policy_version 546012 (0.0009) [2023-12-26 19:20:31,828][105620] Updated weights for policy 1, policy_version 546736 (0.0007) [2023-12-26 19:20:31,879][105620] Updated weights for policy 1, policy_version 546746 (0.0008) [2023-12-26 19:20:31,930][105620] Updated weights for policy 1, policy_version 546756 (0.0008) [2023-12-26 19:20:32,453][105692] Updated weights for policy 0, policy_version 546022 (0.0009) [2023-12-26 19:20:32,502][105692] Updated weights for policy 0, policy_version 546032 (0.0008) [2023-12-26 19:20:32,554][105692] Updated weights for policy 0, policy_version 546042 (0.0008) [2023-12-26 19:20:32,645][105620] Updated weights for policy 1, policy_version 546766 (0.0009) [2023-12-26 19:20:32,704][105620] Updated weights for policy 1, policy_version 546776 (0.0009) [2023-12-26 19:20:32,757][105620] Updated weights for policy 1, policy_version 546786 (0.0009) [2023-12-26 19:20:33,263][105692] Updated weights for policy 0, policy_version 546052 (0.0007) [2023-12-26 19:20:33,313][105692] Updated weights for policy 0, policy_version 546062 (0.0009) [2023-12-26 19:20:33,360][105692] Updated weights for policy 0, policy_version 546072 (0.0009) [2023-12-26 19:20:33,444][105620] Updated weights for policy 1, policy_version 546797 (0.0009) [2023-12-26 19:20:33,489][105620] Updated weights for policy 1, policy_version 546807 (0.0008) [2023-12-26 19:20:33,539][105620] Updated weights for policy 1, policy_version 546817 (0.0005) [2023-12-26 19:20:34,106][105620] Updated weights for policy 1, policy_version 546827 (0.0006) [2023-12-26 19:20:34,139][105692] Updated weights for policy 0, policy_version 546082 (0.0008) [2023-12-26 19:20:34,169][105620] Updated weights for policy 1, policy_version 546837 (0.0008) [2023-12-26 19:20:34,201][105692] Updated weights for policy 0, policy_version 546092 (0.0007) [2023-12-26 19:20:34,229][105620] Updated weights for policy 1, policy_version 546847 (0.0008) [2023-12-26 19:20:34,265][105692] Updated weights for policy 0, policy_version 546102 (0.0006) [2023-12-26 19:20:34,328][105692] Updated weights for policy 0, policy_version 546112 (0.0008) [2023-12-26 19:20:34,965][105692] Updated weights for policy 0, policy_version 546122 (0.0009) [2023-12-26 19:20:34,971][105620] Updated weights for policy 1, policy_version 546857 (0.0008) [2023-12-26 19:20:35,017][105692] Updated weights for policy 0, policy_version 546132 (0.0008) [2023-12-26 19:20:35,020][105620] Updated weights for policy 1, policy_version 546867 (0.0005) [2023-12-26 19:20:35,063][105620] Updated weights for policy 1, policy_version 546877 (0.0005) [2023-12-26 19:20:35,065][105692] Updated weights for policy 0, policy_version 546142 (0.0008) [2023-12-26 19:20:35,112][105620] Updated weights for policy 1, policy_version 546887 (0.0005) [2023-12-26 19:20:35,755][105620] Updated weights for policy 1, policy_version 546897 (0.0009) [2023-12-26 19:20:35,812][105620] Updated weights for policy 1, policy_version 546907 (0.0007) [2023-12-26 19:20:35,863][105620] Updated weights for policy 1, policy_version 546917 (0.0005) [2023-12-26 19:20:35,905][105692] Updated weights for policy 0, policy_version 546152 (0.0008) [2023-12-26 19:20:35,952][105692] Updated weights for policy 0, policy_version 546162 (0.0009) [2023-12-26 19:20:36,005][105692] Updated weights for policy 0, policy_version 546172 (0.0009) [2023-12-26 19:20:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 279863296. Throughput: 0: 9833.2, 1: 9792.4. Samples: 279845852. Policy #0 lag: (min: 28.0, avg: 41.2, max: 60.0) [2023-12-26 19:20:36,062][104569] Avg episode reward: [(0, '8993.516'), (1, '9353.012')] [2023-12-26 19:20:36,562][105620] Updated weights for policy 1, policy_version 546927 (0.0006) [2023-12-26 19:20:36,620][105620] Updated weights for policy 1, policy_version 546937 (0.0008) [2023-12-26 19:20:36,684][105620] Updated weights for policy 1, policy_version 546947 (0.0008) [2023-12-26 19:20:36,840][105692] Updated weights for policy 0, policy_version 546182 (0.0008) [2023-12-26 19:20:36,896][105692] Updated weights for policy 0, policy_version 546192 (0.0009) [2023-12-26 19:20:36,950][105692] Updated weights for policy 0, policy_version 546202 (0.0009) [2023-12-26 19:20:37,357][105620] Updated weights for policy 1, policy_version 546957 (0.0009) [2023-12-26 19:20:37,429][105620] Updated weights for policy 1, policy_version 546967 (0.0009) [2023-12-26 19:20:37,494][105620] Updated weights for policy 1, policy_version 546977 (0.0009) [2023-12-26 19:20:37,645][105692] Updated weights for policy 0, policy_version 546212 (0.0009) [2023-12-26 19:20:37,703][105692] Updated weights for policy 0, policy_version 546222 (0.0009) [2023-12-26 19:20:37,762][105692] Updated weights for policy 0, policy_version 546232 (0.0009) [2023-12-26 19:20:38,272][105620] Updated weights for policy 1, policy_version 546987 (0.0009) [2023-12-26 19:20:38,338][105620] Updated weights for policy 1, policy_version 546997 (0.0009) [2023-12-26 19:20:38,402][105620] Updated weights for policy 1, policy_version 547007 (0.0008) [2023-12-26 19:20:38,522][105692] Updated weights for policy 0, policy_version 546242 (0.0007) [2023-12-26 19:20:38,587][105692] Updated weights for policy 0, policy_version 546252 (0.0009) [2023-12-26 19:20:38,650][105692] Updated weights for policy 0, policy_version 546262 (0.0009) [2023-12-26 19:20:38,715][105692] Updated weights for policy 0, policy_version 546272 (0.0009) [2023-12-26 19:20:39,146][105620] Updated weights for policy 1, policy_version 547017 (0.0008) [2023-12-26 19:20:39,211][105620] Updated weights for policy 1, policy_version 547027 (0.0008) [2023-12-26 19:20:39,279][105620] Updated weights for policy 1, policy_version 547037 (0.0007) [2023-12-26 19:20:39,338][105620] Updated weights for policy 1, policy_version 547047 (0.0008) [2023-12-26 19:20:39,529][105692] Updated weights for policy 0, policy_version 546282 (0.0010) [2023-12-26 19:20:39,592][105692] Updated weights for policy 0, policy_version 546292 (0.0008) [2023-12-26 19:20:39,649][105692] Updated weights for policy 0, policy_version 546302 (0.0006) [2023-12-26 19:20:40,183][105620] Updated weights for policy 1, policy_version 547057 (0.0008) [2023-12-26 19:20:40,240][105620] Updated weights for policy 1, policy_version 547067 (0.0008) [2023-12-26 19:20:40,296][105620] Updated weights for policy 1, policy_version 547077 (0.0008) [2023-12-26 19:20:40,369][105692] Updated weights for policy 0, policy_version 546312 (0.0010) [2023-12-26 19:20:40,428][105692] Updated weights for policy 0, policy_version 546322 (0.0011) [2023-12-26 19:20:40,484][105692] Updated weights for policy 0, policy_version 546332 (0.0010) [2023-12-26 19:20:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 279945216. Throughput: 0: 9735.2, 1: 9760.1. Samples: 279958792. Policy #0 lag: (min: 28.0, avg: 41.2, max: 60.0) [2023-12-26 19:20:41,062][104569] Avg episode reward: [(0, '8721.402'), (1, '9353.708')] [2023-12-26 19:20:41,075][105620] Updated weights for policy 1, policy_version 547087 (0.0008) [2023-12-26 19:20:41,139][105620] Updated weights for policy 1, policy_version 547097 (0.0009) [2023-12-26 19:20:41,196][105692] Updated weights for policy 0, policy_version 546342 (0.0009) [2023-12-26 19:20:41,205][105620] Updated weights for policy 1, policy_version 547107 (0.0009) [2023-12-26 19:20:41,261][105692] Updated weights for policy 0, policy_version 546352 (0.0009) [2023-12-26 19:20:41,322][105692] Updated weights for policy 0, policy_version 546362 (0.0009) [2023-12-26 19:20:42,015][105620] Updated weights for policy 1, policy_version 547117 (0.0009) [2023-12-26 19:20:42,068][105620] Updated weights for policy 1, policy_version 547127 (0.0008) [2023-12-26 19:20:42,114][105620] Updated weights for policy 1, policy_version 547137 (0.0007) [2023-12-26 19:20:42,153][105692] Updated weights for policy 0, policy_version 546372 (0.0010) [2023-12-26 19:20:42,212][105692] Updated weights for policy 0, policy_version 546382 (0.0011) [2023-12-26 19:20:42,275][105692] Updated weights for policy 0, policy_version 546392 (0.0011) [2023-12-26 19:20:42,873][105620] Updated weights for policy 1, policy_version 547147 (0.0006) [2023-12-26 19:20:42,923][105620] Updated weights for policy 1, policy_version 547157 (0.0008) [2023-12-26 19:20:42,980][105620] Updated weights for policy 1, policy_version 547167 (0.0007) [2023-12-26 19:20:43,057][105692] Updated weights for policy 0, policy_version 546402 (0.0011) [2023-12-26 19:20:43,120][105692] Updated weights for policy 0, policy_version 546412 (0.0011) [2023-12-26 19:20:43,184][105692] Updated weights for policy 0, policy_version 546422 (0.0008) [2023-12-26 19:20:43,244][105692] Updated weights for policy 0, policy_version 546432 (0.0005) [2023-12-26 19:20:43,584][105620] Updated weights for policy 1, policy_version 547177 (0.0008) [2023-12-26 19:20:43,641][105620] Updated weights for policy 1, policy_version 547187 (0.0005) [2023-12-26 19:20:43,699][105620] Updated weights for policy 1, policy_version 547197 (0.0005) [2023-12-26 19:20:43,760][105620] Updated weights for policy 1, policy_version 547207 (0.0006) [2023-12-26 19:20:43,896][105692] Updated weights for policy 0, policy_version 546442 (0.0005) [2023-12-26 19:20:43,968][105692] Updated weights for policy 0, policy_version 546452 (0.0008) [2023-12-26 19:20:44,032][105692] Updated weights for policy 0, policy_version 546462 (0.0010) [2023-12-26 19:20:44,299][105620] Updated weights for policy 1, policy_version 547217 (0.0006) [2023-12-26 19:20:44,358][105620] Updated weights for policy 1, policy_version 547227 (0.0006) [2023-12-26 19:20:44,423][105620] Updated weights for policy 1, policy_version 547237 (0.0009) [2023-12-26 19:20:44,567][105692] Updated weights for policy 0, policy_version 546472 (0.0005) [2023-12-26 19:20:44,621][105692] Updated weights for policy 0, policy_version 546482 (0.0010) [2023-12-26 19:20:44,679][105692] Updated weights for policy 0, policy_version 546492 (0.0010) [2023-12-26 19:20:45,139][105620] Updated weights for policy 1, policy_version 547247 (0.0009) [2023-12-26 19:20:45,194][105620] Updated weights for policy 1, policy_version 547257 (0.0008) [2023-12-26 19:20:45,255][105620] Updated weights for policy 1, policy_version 547267 (0.0008) [2023-12-26 19:20:45,323][105692] Updated weights for policy 0, policy_version 546502 (0.0008) [2023-12-26 19:20:45,387][105692] Updated weights for policy 0, policy_version 546512 (0.0010) [2023-12-26 19:20:45,447][105692] Updated weights for policy 0, policy_version 546522 (0.0011) [2023-12-26 19:20:45,969][105620] Updated weights for policy 1, policy_version 547277 (0.0007) [2023-12-26 19:20:46,030][105620] Updated weights for policy 1, policy_version 547287 (0.0006) [2023-12-26 19:20:46,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 280043520. Throughput: 0: 9713.8, 1: 9783.8. Samples: 280015944. Policy #0 lag: (min: 28.0, avg: 41.2, max: 60.0) [2023-12-26 19:20:46,062][104569] Avg episode reward: [(0, '8904.057'), (1, '9261.086')] [2023-12-26 19:20:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000546528_139927552.pth... [2023-12-26 19:20:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000545408_139640832.pth [2023-12-26 19:20:46,079][105620] Updated weights for policy 1, policy_version 547297 (0.0007) [2023-12-26 19:20:46,115][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000547304_140124160.pth... [2023-12-26 19:20:46,119][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000546120_139821056.pth [2023-12-26 19:20:46,135][105692] Updated weights for policy 0, policy_version 546532 (0.0011) [2023-12-26 19:20:46,188][105692] Updated weights for policy 0, policy_version 546542 (0.0010) [2023-12-26 19:20:46,240][105692] Updated weights for policy 0, policy_version 546552 (0.0011) [2023-12-26 19:20:46,707][105620] Updated weights for policy 1, policy_version 547307 (0.0011) [2023-12-26 19:20:46,762][105620] Updated weights for policy 1, policy_version 547317 (0.0010) [2023-12-26 19:20:46,819][105620] Updated weights for policy 1, policy_version 547327 (0.0010) [2023-12-26 19:20:46,977][105692] Updated weights for policy 0, policy_version 546562 (0.0010) [2023-12-26 19:20:47,038][105692] Updated weights for policy 0, policy_version 546572 (0.0010) [2023-12-26 19:20:47,096][105692] Updated weights for policy 0, policy_version 546582 (0.0010) [2023-12-26 19:20:47,161][105692] Updated weights for policy 0, policy_version 546592 (0.0010) [2023-12-26 19:20:47,568][105620] Updated weights for policy 1, policy_version 547337 (0.0010) [2023-12-26 19:20:47,635][105620] Updated weights for policy 1, policy_version 547347 (0.0009) [2023-12-26 19:20:47,703][105620] Updated weights for policy 1, policy_version 547357 (0.0006) [2023-12-26 19:20:47,767][105620] Updated weights for policy 1, policy_version 547367 (0.0006) [2023-12-26 19:20:47,896][105692] Updated weights for policy 0, policy_version 546602 (0.0006) [2023-12-26 19:20:47,951][105692] Updated weights for policy 0, policy_version 546612 (0.0005) [2023-12-26 19:20:48,008][105692] Updated weights for policy 0, policy_version 546622 (0.0007) [2023-12-26 19:20:48,506][105620] Updated weights for policy 1, policy_version 547377 (0.0008) [2023-12-26 19:20:48,563][105620] Updated weights for policy 1, policy_version 547387 (0.0007) [2023-12-26 19:20:48,622][105620] Updated weights for policy 1, policy_version 547397 (0.0007) [2023-12-26 19:20:48,631][105692] Updated weights for policy 0, policy_version 546632 (0.0010) [2023-12-26 19:20:48,686][105692] Updated weights for policy 0, policy_version 546642 (0.0010) [2023-12-26 19:20:48,750][105692] Updated weights for policy 0, policy_version 546652 (0.0011) [2023-12-26 19:20:49,423][105692] Updated weights for policy 0, policy_version 546662 (0.0010) [2023-12-26 19:20:49,429][105620] Updated weights for policy 1, policy_version 547407 (0.0008) [2023-12-26 19:20:49,483][105692] Updated weights for policy 0, policy_version 546672 (0.0011) [2023-12-26 19:20:49,487][105620] Updated weights for policy 1, policy_version 547417 (0.0010) [2023-12-26 19:20:49,543][105620] Updated weights for policy 1, policy_version 547427 (0.0008) [2023-12-26 19:20:49,545][105692] Updated weights for policy 0, policy_version 546682 (0.0011) [2023-12-26 19:20:50,277][105620] Updated weights for policy 1, policy_version 547437 (0.0009) [2023-12-26 19:20:50,314][105692] Updated weights for policy 0, policy_version 546692 (0.0006) [2023-12-26 19:20:50,333][105620] Updated weights for policy 1, policy_version 547447 (0.0008) [2023-12-26 19:20:50,376][105692] Updated weights for policy 0, policy_version 546702 (0.0008) [2023-12-26 19:20:50,396][105620] Updated weights for policy 1, policy_version 547457 (0.0006) [2023-12-26 19:20:50,435][105692] Updated weights for policy 0, policy_version 546712 (0.0007) [2023-12-26 19:20:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 280141824. Throughput: 0: 9794.4, 1: 9818.7. Samples: 280135376. Policy #0 lag: (min: 28.0, avg: 41.2, max: 60.0) [2023-12-26 19:20:51,063][104569] Avg episode reward: [(0, '8902.940'), (1, '9352.325')] [2023-12-26 19:20:51,180][105620] Updated weights for policy 1, policy_version 547467 (0.0007) [2023-12-26 19:20:51,192][105692] Updated weights for policy 0, policy_version 546722 (0.0009) [2023-12-26 19:20:51,237][105620] Updated weights for policy 1, policy_version 547477 (0.0008) [2023-12-26 19:20:51,256][105692] Updated weights for policy 0, policy_version 546732 (0.0007) [2023-12-26 19:20:51,296][105620] Updated weights for policy 1, policy_version 547487 (0.0009) [2023-12-26 19:20:51,306][105692] Updated weights for policy 0, policy_version 546742 (0.0006) [2023-12-26 19:20:51,382][105692] Updated weights for policy 0, policy_version 546752 (0.0008) [2023-12-26 19:20:51,983][105620] Updated weights for policy 1, policy_version 547497 (0.0007) [2023-12-26 19:20:52,043][105620] Updated weights for policy 1, policy_version 547507 (0.0008) [2023-12-26 19:20:52,107][105620] Updated weights for policy 1, policy_version 547517 (0.0008) [2023-12-26 19:20:52,157][105692] Updated weights for policy 0, policy_version 546762 (0.0011) [2023-12-26 19:20:52,172][105620] Updated weights for policy 1, policy_version 547527 (0.0007) [2023-12-26 19:20:52,220][105692] Updated weights for policy 0, policy_version 546772 (0.0011) [2023-12-26 19:20:52,275][105692] Updated weights for policy 0, policy_version 546782 (0.0010) [2023-12-26 19:20:52,936][105620] Updated weights for policy 1, policy_version 547537 (0.0009) [2023-12-26 19:20:52,984][105620] Updated weights for policy 1, policy_version 547547 (0.0008) [2023-12-26 19:20:53,015][105692] Updated weights for policy 0, policy_version 546792 (0.0011) [2023-12-26 19:20:53,037][105620] Updated weights for policy 1, policy_version 547557 (0.0005) [2023-12-26 19:20:53,066][105692] Updated weights for policy 0, policy_version 546802 (0.0011) [2023-12-26 19:20:53,114][105692] Updated weights for policy 0, policy_version 546812 (0.0010) [2023-12-26 19:20:53,812][105620] Updated weights for policy 1, policy_version 547567 (0.0007) [2023-12-26 19:20:53,866][105620] Updated weights for policy 1, policy_version 547577 (0.0008) [2023-12-26 19:20:53,881][105692] Updated weights for policy 0, policy_version 546822 (0.0010) [2023-12-26 19:20:53,922][105620] Updated weights for policy 1, policy_version 547587 (0.0006) [2023-12-26 19:20:53,945][105692] Updated weights for policy 0, policy_version 546832 (0.0011) [2023-12-26 19:20:53,997][105692] Updated weights for policy 0, policy_version 546842 (0.0010) [2023-12-26 19:20:54,665][105620] Updated weights for policy 1, policy_version 547597 (0.0009) [2023-12-26 19:20:54,717][105620] Updated weights for policy 1, policy_version 547607 (0.0008) [2023-12-26 19:20:54,746][105692] Updated weights for policy 0, policy_version 546852 (0.0010) [2023-12-26 19:20:54,768][105620] Updated weights for policy 1, policy_version 547617 (0.0007) [2023-12-26 19:20:54,797][105692] Updated weights for policy 0, policy_version 546862 (0.0010) [2023-12-26 19:20:54,859][105692] Updated weights for policy 0, policy_version 546872 (0.0010) [2023-12-26 19:20:55,535][105620] Updated weights for policy 1, policy_version 547627 (0.0007) [2023-12-26 19:20:55,582][105620] Updated weights for policy 1, policy_version 547637 (0.0008) [2023-12-26 19:20:55,602][105692] Updated weights for policy 0, policy_version 546882 (0.0010) [2023-12-26 19:20:55,634][105620] Updated weights for policy 1, policy_version 547647 (0.0009) [2023-12-26 19:20:55,654][105692] Updated weights for policy 0, policy_version 546892 (0.0010) [2023-12-26 19:20:55,708][105692] Updated weights for policy 0, policy_version 546902 (0.0010) [2023-12-26 19:20:55,762][105692] Updated weights for policy 0, policy_version 546912 (0.0010) [2023-12-26 19:20:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 280240128. Throughput: 0: 9651.3, 1: 9783.4. Samples: 280247376. Policy #0 lag: (min: 28.0, avg: 41.2, max: 60.0) [2023-12-26 19:20:56,062][104569] Avg episode reward: [(0, '8994.371'), (1, '1337.292')] [2023-12-26 19:20:56,338][105620] Updated weights for policy 1, policy_version 547657 (0.0008) [2023-12-26 19:20:56,388][105620] Updated weights for policy 1, policy_version 547667 (0.0005) [2023-12-26 19:20:56,437][105620] Updated weights for policy 1, policy_version 547677 (0.0009) [2023-12-26 19:20:56,491][105620] Updated weights for policy 1, policy_version 547687 (0.0007) [2023-12-26 19:20:56,518][105692] Updated weights for policy 0, policy_version 546922 (0.0011) [2023-12-26 19:20:56,573][105692] Updated weights for policy 0, policy_version 546932 (0.0006) [2023-12-26 19:20:56,629][105692] Updated weights for policy 0, policy_version 546942 (0.0005) [2023-12-26 19:20:57,171][105620] Updated weights for policy 1, policy_version 547697 (0.0006) [2023-12-26 19:20:57,232][105620] Updated weights for policy 1, policy_version 547707 (0.0007) [2023-12-26 19:20:57,279][105620] Updated weights for policy 1, policy_version 547717 (0.0007) [2023-12-26 19:20:57,330][105692] Updated weights for policy 0, policy_version 546952 (0.0011) [2023-12-26 19:20:57,389][105692] Updated weights for policy 0, policy_version 546962 (0.0009) [2023-12-26 19:20:57,464][105692] Updated weights for policy 0, policy_version 546972 (0.0010) [2023-12-26 19:20:57,867][105620] Updated weights for policy 1, policy_version 547727 (0.0008) [2023-12-26 19:20:57,932][105620] Updated weights for policy 1, policy_version 547737 (0.0007) [2023-12-26 19:20:57,995][105620] Updated weights for policy 1, policy_version 547747 (0.0008) [2023-12-26 19:20:58,189][105692] Updated weights for policy 0, policy_version 546982 (0.0009) [2023-12-26 19:20:58,258][105692] Updated weights for policy 0, policy_version 546992 (0.0008) [2023-12-26 19:20:58,323][105692] Updated weights for policy 0, policy_version 547002 (0.0009) [2023-12-26 19:20:58,753][105620] Updated weights for policy 1, policy_version 547757 (0.0008) [2023-12-26 19:20:58,821][105620] Updated weights for policy 1, policy_version 547767 (0.0007) [2023-12-26 19:20:58,898][105620] Updated weights for policy 1, policy_version 547777 (0.0009) [2023-12-26 19:20:59,180][105692] Updated weights for policy 0, policy_version 547012 (0.0008) [2023-12-26 19:20:59,241][105692] Updated weights for policy 0, policy_version 547022 (0.0010) [2023-12-26 19:20:59,317][105692] Updated weights for policy 0, policy_version 547032 (0.0007) [2023-12-26 19:20:59,764][105620] Updated weights for policy 1, policy_version 547787 (0.0008) [2023-12-26 19:20:59,825][105620] Updated weights for policy 1, policy_version 547797 (0.0009) [2023-12-26 19:20:59,890][105620] Updated weights for policy 1, policy_version 547807 (0.0009) [2023-12-26 19:21:00,101][105692] Updated weights for policy 0, policy_version 547042 (0.0009) [2023-12-26 19:21:00,167][105692] Updated weights for policy 0, policy_version 547052 (0.0009) [2023-12-26 19:21:00,220][105692] Updated weights for policy 0, policy_version 547062 (0.0010) [2023-12-26 19:21:00,279][105692] Updated weights for policy 0, policy_version 547072 (0.0010) [2023-12-26 19:21:00,655][105620] Updated weights for policy 1, policy_version 547817 (0.0008) [2023-12-26 19:21:00,713][105620] Updated weights for policy 1, policy_version 547827 (0.0008) [2023-12-26 19:21:00,765][105620] Updated weights for policy 1, policy_version 547837 (0.0008) [2023-12-26 19:21:00,818][105620] Updated weights for policy 1, policy_version 547847 (0.0009) [2023-12-26 19:21:00,944][105692] Updated weights for policy 0, policy_version 547082 (0.0005) [2023-12-26 19:21:00,994][105692] Updated weights for policy 0, policy_version 547092 (0.0005) [2023-12-26 19:21:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 280330240. Throughput: 0: 9590.1, 1: 9781.2. Samples: 280305784. Policy #0 lag: (min: 28.0, avg: 41.2, max: 60.0) [2023-12-26 19:21:01,063][104569] Avg episode reward: [(0, '8903.570'), (1, '2079.177')] [2023-12-26 19:21:01,064][105692] Updated weights for policy 0, policy_version 547102 (0.0007) [2023-12-26 19:21:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000547848_140263424.pth... [2023-12-26 19:21:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000546728_139976704.pth [2023-12-26 19:21:01,076][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000547104_140075008.pth... [2023-12-26 19:21:01,081][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000545984_139788288.pth [2023-12-26 19:21:01,586][105620] Updated weights for policy 1, policy_version 547857 (0.0009) [2023-12-26 19:21:01,661][105620] Updated weights for policy 1, policy_version 547867 (0.0008) [2023-12-26 19:21:01,720][105620] Updated weights for policy 1, policy_version 547877 (0.0009) [2023-12-26 19:21:01,737][105692] Updated weights for policy 0, policy_version 547112 (0.0008) [2023-12-26 19:21:01,799][105692] Updated weights for policy 0, policy_version 547122 (0.0009) [2023-12-26 19:21:01,861][105692] Updated weights for policy 0, policy_version 547132 (0.0006) [2023-12-26 19:21:02,480][105620] Updated weights for policy 1, policy_version 547887 (0.0007) [2023-12-26 19:21:02,541][105692] Updated weights for policy 0, policy_version 547142 (0.0006) [2023-12-26 19:21:02,546][105620] Updated weights for policy 1, policy_version 547897 (0.0009) [2023-12-26 19:21:02,605][105692] Updated weights for policy 0, policy_version 547152 (0.0006) [2023-12-26 19:21:02,614][105620] Updated weights for policy 1, policy_version 547907 (0.0009) [2023-12-26 19:21:02,664][105692] Updated weights for policy 0, policy_version 547162 (0.0007) [2023-12-26 19:21:03,377][105692] Updated weights for policy 0, policy_version 547172 (0.0008) [2023-12-26 19:21:03,379][105620] Updated weights for policy 1, policy_version 547917 (0.0006) [2023-12-26 19:21:03,438][105620] Updated weights for policy 1, policy_version 547927 (0.0007) [2023-12-26 19:21:03,440][105692] Updated weights for policy 0, policy_version 547182 (0.0006) [2023-12-26 19:21:03,493][105620] Updated weights for policy 1, policy_version 547937 (0.0007) [2023-12-26 19:21:03,497][105692] Updated weights for policy 0, policy_version 547192 (0.0008) [2023-12-26 19:21:04,203][105692] Updated weights for policy 0, policy_version 547202 (0.0009) [2023-12-26 19:21:04,262][105692] Updated weights for policy 0, policy_version 547212 (0.0011) [2023-12-26 19:21:04,284][105620] Updated weights for policy 1, policy_version 547947 (0.0006) [2023-12-26 19:21:04,323][105692] Updated weights for policy 0, policy_version 547222 (0.0011) [2023-12-26 19:21:04,349][105620] Updated weights for policy 1, policy_version 547957 (0.0005) [2023-12-26 19:21:04,387][105692] Updated weights for policy 0, policy_version 547232 (0.0011) [2023-12-26 19:21:04,407][105620] Updated weights for policy 1, policy_version 547967 (0.0007) [2023-12-26 19:21:05,005][105620] Updated weights for policy 1, policy_version 547977 (0.0011) [2023-12-26 19:21:05,064][105620] Updated weights for policy 1, policy_version 547987 (0.0008) [2023-12-26 19:21:05,116][105620] Updated weights for policy 1, policy_version 547997 (0.0005) [2023-12-26 19:21:05,121][105692] Updated weights for policy 0, policy_version 547242 (0.0010) [2023-12-26 19:21:05,169][105692] Updated weights for policy 0, policy_version 547252 (0.0010) [2023-12-26 19:21:05,174][105620] Updated weights for policy 1, policy_version 548007 (0.0005) [2023-12-26 19:21:05,224][105692] Updated weights for policy 0, policy_version 547262 (0.0010) [2023-12-26 19:21:05,842][105620] Updated weights for policy 1, policy_version 548017 (0.0010) [2023-12-26 19:21:05,897][105620] Updated weights for policy 1, policy_version 548027 (0.0010) [2023-12-26 19:21:05,948][105620] Updated weights for policy 1, policy_version 548037 (0.0005) [2023-12-26 19:21:05,987][105692] Updated weights for policy 0, policy_version 547272 (0.0010) [2023-12-26 19:21:06,041][105692] Updated weights for policy 0, policy_version 547282 (0.0010) [2023-12-26 19:21:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 280428544. Throughput: 0: 9539.2, 1: 9691.4. Samples: 280417872. Policy #0 lag: (min: 28.0, avg: 41.2, max: 60.0) [2023-12-26 19:21:06,062][104569] Avg episode reward: [(0, '9086.610'), (1, '5233.089')] [2023-12-26 19:21:06,092][105692] Updated weights for policy 0, policy_version 547292 (0.0010) [2023-12-26 19:21:06,626][105620] Updated weights for policy 1, policy_version 548047 (0.0009) [2023-12-26 19:21:06,690][105620] Updated weights for policy 1, policy_version 548057 (0.0008) [2023-12-26 19:21:06,756][105620] Updated weights for policy 1, policy_version 548067 (0.0006) [2023-12-26 19:21:06,849][105692] Updated weights for policy 0, policy_version 547302 (0.0011) [2023-12-26 19:21:06,902][105692] Updated weights for policy 0, policy_version 547312 (0.0006) [2023-12-26 19:21:06,948][105692] Updated weights for policy 0, policy_version 547322 (0.0005) [2023-12-26 19:21:07,387][105620] Updated weights for policy 1, policy_version 548077 (0.0010) [2023-12-26 19:21:07,450][105620] Updated weights for policy 1, policy_version 548087 (0.0009) [2023-12-26 19:21:07,509][105620] Updated weights for policy 1, policy_version 548097 (0.0008) [2023-12-26 19:21:07,522][105692] Updated weights for policy 0, policy_version 547332 (0.0007) [2023-12-26 19:21:07,581][105692] Updated weights for policy 0, policy_version 547342 (0.0010) [2023-12-26 19:21:07,636][105692] Updated weights for policy 0, policy_version 547352 (0.0010) [2023-12-26 19:21:08,299][105620] Updated weights for policy 1, policy_version 548107 (0.0009) [2023-12-26 19:21:08,351][105692] Updated weights for policy 0, policy_version 547362 (0.0010) [2023-12-26 19:21:08,362][105620] Updated weights for policy 1, policy_version 548117 (0.0007) [2023-12-26 19:21:08,410][105692] Updated weights for policy 0, policy_version 547372 (0.0011) [2023-12-26 19:21:08,412][105620] Updated weights for policy 1, policy_version 548127 (0.0007) [2023-12-26 19:21:08,469][105692] Updated weights for policy 0, policy_version 547382 (0.0011) [2023-12-26 19:21:08,531][105692] Updated weights for policy 0, policy_version 547392 (0.0010) [2023-12-26 19:21:09,210][105620] Updated weights for policy 1, policy_version 548137 (0.0007) [2023-12-26 19:21:09,226][105692] Updated weights for policy 0, policy_version 547402 (0.0008) [2023-12-26 19:21:09,280][105620] Updated weights for policy 1, policy_version 548147 (0.0007) [2023-12-26 19:21:09,286][105692] Updated weights for policy 0, policy_version 547412 (0.0008) [2023-12-26 19:21:09,347][105692] Updated weights for policy 0, policy_version 547422 (0.0007) [2023-12-26 19:21:09,353][105620] Updated weights for policy 1, policy_version 548157 (0.0008) [2023-12-26 19:21:09,425][105620] Updated weights for policy 1, policy_version 548167 (0.0009) [2023-12-26 19:21:10,049][105620] Updated weights for policy 1, policy_version 548177 (0.0007) [2023-12-26 19:21:10,064][105692] Updated weights for policy 0, policy_version 547432 (0.0008) [2023-12-26 19:21:10,105][105620] Updated weights for policy 1, policy_version 548187 (0.0005) [2023-12-26 19:21:10,131][105692] Updated weights for policy 0, policy_version 547442 (0.0008) [2023-12-26 19:21:10,166][105620] Updated weights for policy 1, policy_version 548197 (0.0006) [2023-12-26 19:21:10,197][105692] Updated weights for policy 0, policy_version 547452 (0.0008) [2023-12-26 19:21:10,789][105620] Updated weights for policy 1, policy_version 548207 (0.0005) [2023-12-26 19:21:10,843][105620] Updated weights for policy 1, policy_version 548217 (0.0005) [2023-12-26 19:21:10,894][105620] Updated weights for policy 1, policy_version 548227 (0.0008) [2023-12-26 19:21:10,921][105692] Updated weights for policy 0, policy_version 547462 (0.0009) [2023-12-26 19:21:10,976][105692] Updated weights for policy 0, policy_version 547472 (0.0005) [2023-12-26 19:21:11,027][105692] Updated weights for policy 0, policy_version 547482 (0.0006) [2023-12-26 19:21:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 280526848. Throughput: 0: 9599.7, 1: 9752.0. Samples: 280537124. Policy #0 lag: (min: 28.0, avg: 41.2, max: 60.0) [2023-12-26 19:21:11,062][104569] Avg episode reward: [(0, '9176.758'), (1, '7770.895')] [2023-12-26 19:21:11,715][105620] Updated weights for policy 1, policy_version 548237 (0.0008) [2023-12-26 19:21:11,778][105620] Updated weights for policy 1, policy_version 548247 (0.0009) [2023-12-26 19:21:11,793][105692] Updated weights for policy 0, policy_version 547492 (0.0007) [2023-12-26 19:21:11,838][105620] Updated weights for policy 1, policy_version 548257 (0.0009) [2023-12-26 19:21:11,853][105692] Updated weights for policy 0, policy_version 547502 (0.0006) [2023-12-26 19:21:11,911][105692] Updated weights for policy 0, policy_version 547512 (0.0006) [2023-12-26 19:21:12,602][105692] Updated weights for policy 0, policy_version 547522 (0.0008) [2023-12-26 19:21:12,619][105620] Updated weights for policy 1, policy_version 548267 (0.0009) [2023-12-26 19:21:12,663][105692] Updated weights for policy 0, policy_version 547532 (0.0009) [2023-12-26 19:21:12,684][105620] Updated weights for policy 1, policy_version 548277 (0.0008) [2023-12-26 19:21:12,723][105692] Updated weights for policy 0, policy_version 547542 (0.0011) [2023-12-26 19:21:12,753][105620] Updated weights for policy 1, policy_version 548287 (0.0007) [2023-12-26 19:21:12,790][105692] Updated weights for policy 0, policy_version 547552 (0.0011) [2023-12-26 19:21:13,451][105620] Updated weights for policy 1, policy_version 548297 (0.0008) [2023-12-26 19:21:13,509][105620] Updated weights for policy 1, policy_version 548307 (0.0006) [2023-12-26 19:21:13,520][105692] Updated weights for policy 0, policy_version 547562 (0.0009) [2023-12-26 19:21:13,568][105620] Updated weights for policy 1, policy_version 548317 (0.0007) [2023-12-26 19:21:13,582][105692] Updated weights for policy 0, policy_version 547572 (0.0007) [2023-12-26 19:21:13,634][105620] Updated weights for policy 1, policy_version 548327 (0.0008) [2023-12-26 19:21:13,645][105692] Updated weights for policy 0, policy_version 547582 (0.0006) [2023-12-26 19:21:14,232][105620] Updated weights for policy 1, policy_version 548337 (0.0008) [2023-12-26 19:21:14,295][105620] Updated weights for policy 1, policy_version 548347 (0.0008) [2023-12-26 19:21:14,358][105620] Updated weights for policy 1, policy_version 548357 (0.0008) [2023-12-26 19:21:14,466][105692] Updated weights for policy 0, policy_version 547593 (0.0010) [2023-12-26 19:21:14,519][105692] Updated weights for policy 0, policy_version 547603 (0.0010) [2023-12-26 19:21:14,574][105692] Updated weights for policy 0, policy_version 547614 (0.0010) [2023-12-26 19:21:14,951][105620] Updated weights for policy 1, policy_version 548367 (0.0006) [2023-12-26 19:21:15,023][105620] Updated weights for policy 1, policy_version 548377 (0.0006) [2023-12-26 19:21:15,095][105620] Updated weights for policy 1, policy_version 548387 (0.0005) [2023-12-26 19:21:15,426][105692] Updated weights for policy 0, policy_version 547624 (0.0010) [2023-12-26 19:21:15,483][105692] Updated weights for policy 0, policy_version 547634 (0.0010) [2023-12-26 19:21:15,535][105692] Updated weights for policy 0, policy_version 547644 (0.0009) [2023-12-26 19:21:15,617][105620] Updated weights for policy 1, policy_version 548397 (0.0007) [2023-12-26 19:21:15,676][105620] Updated weights for policy 1, policy_version 548407 (0.0006) [2023-12-26 19:21:15,733][105620] Updated weights for policy 1, policy_version 548417 (0.0009) [2023-12-26 19:21:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 280625152. Throughput: 0: 9545.3, 1: 9673.4. Samples: 280593948. Policy #0 lag: (min: 28.0, avg: 41.2, max: 60.0) [2023-12-26 19:21:16,063][104569] Avg episode reward: [(0, '8722.008'), (1, '9261.994')] [2023-12-26 19:21:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000547648_140214272.pth... [2023-12-26 19:21:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000548424_140410880.pth... [2023-12-26 19:21:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000546528_139927552.pth [2023-12-26 19:21:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000547304_140124160.pth [2023-12-26 19:21:16,313][105692] Updated weights for policy 0, policy_version 547654 (0.0009) [2023-12-26 19:21:16,370][105692] Updated weights for policy 0, policy_version 547664 (0.0009) [2023-12-26 19:21:16,429][105692] Updated weights for policy 0, policy_version 547674 (0.0006) [2023-12-26 19:21:16,464][105620] Updated weights for policy 1, policy_version 548427 (0.0009) [2023-12-26 19:21:16,512][105620] Updated weights for policy 1, policy_version 548437 (0.0008) [2023-12-26 19:21:16,557][105620] Updated weights for policy 1, policy_version 548447 (0.0008) [2023-12-26 19:21:17,159][105692] Updated weights for policy 0, policy_version 547684 (0.0007) [2023-12-26 19:21:17,215][105692] Updated weights for policy 0, policy_version 547694 (0.0009) [2023-12-26 19:21:17,269][105692] Updated weights for policy 0, policy_version 547705 (0.0010) [2023-12-26 19:21:17,313][105620] Updated weights for policy 1, policy_version 548457 (0.0009) [2023-12-26 19:21:17,370][105620] Updated weights for policy 1, policy_version 548467 (0.0005) [2023-12-26 19:21:17,415][105620] Updated weights for policy 1, policy_version 548477 (0.0005) [2023-12-26 19:21:17,477][105620] Updated weights for policy 1, policy_version 548487 (0.0006) [2023-12-26 19:21:18,076][105692] Updated weights for policy 0, policy_version 547716 (0.0010) [2023-12-26 19:21:18,123][105692] Updated weights for policy 0, policy_version 547726 (0.0008) [2023-12-26 19:21:18,169][105620] Updated weights for policy 1, policy_version 548497 (0.0008) [2023-12-26 19:21:18,179][105692] Updated weights for policy 0, policy_version 547736 (0.0006) [2023-12-26 19:21:18,226][105620] Updated weights for policy 1, policy_version 548507 (0.0008) [2023-12-26 19:21:18,287][105620] Updated weights for policy 1, policy_version 548517 (0.0009) [2023-12-26 19:21:18,915][105692] Updated weights for policy 0, policy_version 547746 (0.0006) [2023-12-26 19:21:18,972][105692] Updated weights for policy 0, policy_version 547756 (0.0011) [2023-12-26 19:21:19,029][105692] Updated weights for policy 0, policy_version 547766 (0.0010) [2023-12-26 19:21:19,067][105620] Updated weights for policy 1, policy_version 548527 (0.0008) [2023-12-26 19:21:19,089][105692] Updated weights for policy 0, policy_version 547776 (0.0010) [2023-12-26 19:21:19,127][105620] Updated weights for policy 1, policy_version 548537 (0.0007) [2023-12-26 19:21:19,183][105620] Updated weights for policy 1, policy_version 548547 (0.0008) [2023-12-26 19:21:19,872][105692] Updated weights for policy 0, policy_version 547786 (0.0011) [2023-12-26 19:21:19,942][105692] Updated weights for policy 0, policy_version 547796 (0.0009) [2023-12-26 19:21:19,952][105620] Updated weights for policy 1, policy_version 548557 (0.0008) [2023-12-26 19:21:20,004][105692] Updated weights for policy 0, policy_version 547806 (0.0008) [2023-12-26 19:21:20,021][105620] Updated weights for policy 1, policy_version 548567 (0.0006) [2023-12-26 19:21:20,084][105620] Updated weights for policy 1, policy_version 548577 (0.0005) [2023-12-26 19:21:20,768][105620] Updated weights for policy 1, policy_version 548587 (0.0006) [2023-12-26 19:21:20,802][105692] Updated weights for policy 0, policy_version 547816 (0.0009) [2023-12-26 19:21:20,816][105620] Updated weights for policy 1, policy_version 548597 (0.0006) [2023-12-26 19:21:20,861][105692] Updated weights for policy 0, policy_version 547826 (0.0008) [2023-12-26 19:21:20,861][105586] KL-divergence is very high: 203.5848 [2023-12-26 19:21:20,863][105620] Updated weights for policy 1, policy_version 548607 (0.0007) [2023-12-26 19:21:20,909][105586] KL-divergence is very high: 272.0161 [2023-12-26 19:21:20,926][105692] Updated weights for policy 0, policy_version 547836 (0.0008) [2023-12-26 19:21:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 280723456. Throughput: 0: 9502.2, 1: 9658.0. Samples: 280708060. Policy #0 lag: (min: 28.0, avg: 41.2, max: 60.0) [2023-12-26 19:21:21,062][104569] Avg episode reward: [(0, '8721.001'), (1, '9080.787')] [2023-12-26 19:21:21,616][105620] Updated weights for policy 1, policy_version 548617 (0.0006) [2023-12-26 19:21:21,678][105620] Updated weights for policy 1, policy_version 548627 (0.0009) [2023-12-26 19:21:21,747][105620] Updated weights for policy 1, policy_version 548637 (0.0009) [2023-12-26 19:21:21,749][105692] Updated weights for policy 0, policy_version 547846 (0.0007) [2023-12-26 19:21:21,797][105620] Updated weights for policy 1, policy_version 548647 (0.0006) [2023-12-26 19:21:21,811][105692] Updated weights for policy 0, policy_version 547856 (0.0008) [2023-12-26 19:21:21,875][105692] Updated weights for policy 0, policy_version 547866 (0.0009) [2023-12-26 19:21:22,552][105620] Updated weights for policy 1, policy_version 548657 (0.0008) [2023-12-26 19:21:22,601][105620] Updated weights for policy 1, policy_version 548667 (0.0009) [2023-12-26 19:21:22,634][105692] Updated weights for policy 0, policy_version 547876 (0.0009) [2023-12-26 19:21:22,656][105620] Updated weights for policy 1, policy_version 548677 (0.0007) [2023-12-26 19:21:22,693][105692] Updated weights for policy 0, policy_version 547886 (0.0007) [2023-12-26 19:21:22,748][105692] Updated weights for policy 0, policy_version 547896 (0.0009) [2023-12-26 19:21:23,458][105620] Updated weights for policy 1, policy_version 548687 (0.0008) [2023-12-26 19:21:23,485][105692] Updated weights for policy 0, policy_version 547906 (0.0008) [2023-12-26 19:21:23,517][105620] Updated weights for policy 1, policy_version 548697 (0.0009) [2023-12-26 19:21:23,539][105692] Updated weights for policy 0, policy_version 547916 (0.0005) [2023-12-26 19:21:23,572][105620] Updated weights for policy 1, policy_version 548707 (0.0010) [2023-12-26 19:21:23,595][105692] Updated weights for policy 0, policy_version 547926 (0.0005) [2023-12-26 19:21:23,668][105692] Updated weights for policy 0, policy_version 547936 (0.0009) [2023-12-26 19:21:24,225][105620] Updated weights for policy 1, policy_version 548717 (0.0007) [2023-12-26 19:21:24,287][105620] Updated weights for policy 1, policy_version 548727 (0.0007) [2023-12-26 19:21:24,351][105620] Updated weights for policy 1, policy_version 548737 (0.0007) [2023-12-26 19:21:24,446][105692] Updated weights for policy 0, policy_version 547946 (0.0010) [2023-12-26 19:21:24,509][105692] Updated weights for policy 0, policy_version 547956 (0.0010) [2023-12-26 19:21:24,559][105692] Updated weights for policy 0, policy_version 547966 (0.0010) [2023-12-26 19:21:25,132][105620] Updated weights for policy 1, policy_version 548747 (0.0009) [2023-12-26 19:21:25,169][105692] Updated weights for policy 0, policy_version 547976 (0.0009) [2023-12-26 19:21:25,183][105620] Updated weights for policy 1, policy_version 548757 (0.0007) [2023-12-26 19:21:25,224][105692] Updated weights for policy 0, policy_version 547986 (0.0010) [2023-12-26 19:21:25,236][105620] Updated weights for policy 1, policy_version 548767 (0.0006) [2023-12-26 19:21:25,280][105692] Updated weights for policy 0, policy_version 547996 (0.0005) [2023-12-26 19:21:25,860][105692] Updated weights for policy 0, policy_version 548006 (0.0005) [2023-12-26 19:21:25,928][105692] Updated weights for policy 0, policy_version 548016 (0.0005) [2023-12-26 19:21:25,930][105585] KL-divergence is very high: 108.5070 [2023-12-26 19:21:25,985][105620] Updated weights for policy 1, policy_version 548777 (0.0009) [2023-12-26 19:21:25,988][105692] Updated weights for policy 0, policy_version 548026 (0.0009) [2023-12-26 19:21:26,047][105620] Updated weights for policy 1, policy_version 548787 (0.0005) [2023-12-26 19:21:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 280813568. Throughput: 0: 9521.8, 1: 9646.9. Samples: 280821384. Policy #0 lag: (min: 28.0, avg: 41.2, max: 60.0) [2023-12-26 19:21:26,063][104569] Avg episode reward: [(0, '1183.042'), (1, '9171.215')] [2023-12-26 19:21:26,116][105620] Updated weights for policy 1, policy_version 548797 (0.0008) [2023-12-26 19:21:26,178][105620] Updated weights for policy 1, policy_version 548807 (0.0007) [2023-12-26 19:21:26,669][105692] Updated weights for policy 0, policy_version 548036 (0.0010) [2023-12-26 19:21:26,720][105692] Updated weights for policy 0, policy_version 548046 (0.0007) [2023-12-26 19:21:26,770][105692] Updated weights for policy 0, policy_version 548056 (0.0005) [2023-12-26 19:21:26,879][105620] Updated weights for policy 1, policy_version 548817 (0.0008) [2023-12-26 19:21:26,931][105620] Updated weights for policy 1, policy_version 548827 (0.0008) [2023-12-26 19:21:26,982][105620] Updated weights for policy 1, policy_version 548837 (0.0005) [2023-12-26 19:21:27,564][105692] Updated weights for policy 0, policy_version 548066 (0.0008) [2023-12-26 19:21:27,591][105620] Updated weights for policy 1, policy_version 548847 (0.0009) [2023-12-26 19:21:27,616][105692] Updated weights for policy 0, policy_version 548076 (0.0006) [2023-12-26 19:21:27,645][105620] Updated weights for policy 1, policy_version 548857 (0.0010) [2023-12-26 19:21:27,665][105692] Updated weights for policy 0, policy_version 548086 (0.0010) [2023-12-26 19:21:27,696][105620] Updated weights for policy 1, policy_version 548867 (0.0010) [2023-12-26 19:21:27,714][105692] Updated weights for policy 0, policy_version 548096 (0.0007) [2023-12-26 19:21:28,453][105620] Updated weights for policy 1, policy_version 548877 (0.0010) [2023-12-26 19:21:28,507][105620] Updated weights for policy 1, policy_version 548887 (0.0008) [2023-12-26 19:21:28,511][105692] Updated weights for policy 0, policy_version 548106 (0.0008) [2023-12-26 19:21:28,563][105620] Updated weights for policy 1, policy_version 548897 (0.0005) [2023-12-26 19:21:28,565][105692] Updated weights for policy 0, policy_version 548116 (0.0008) [2023-12-26 19:21:28,623][105692] Updated weights for policy 0, policy_version 548126 (0.0008) [2023-12-26 19:21:29,176][105620] Updated weights for policy 1, policy_version 548907 (0.0006) [2023-12-26 19:21:29,237][105620] Updated weights for policy 1, policy_version 548917 (0.0006) [2023-12-26 19:21:29,300][105620] Updated weights for policy 1, policy_version 548927 (0.0009) [2023-12-26 19:21:29,458][105692] Updated weights for policy 0, policy_version 548136 (0.0007) [2023-12-26 19:21:29,516][105692] Updated weights for policy 0, policy_version 548146 (0.0005) [2023-12-26 19:21:29,579][105692] Updated weights for policy 0, policy_version 548156 (0.0005) [2023-12-26 19:21:29,991][105620] Updated weights for policy 1, policy_version 548937 (0.0008) [2023-12-26 19:21:30,049][105620] Updated weights for policy 1, policy_version 548947 (0.0009) [2023-12-26 19:21:30,112][105620] Updated weights for policy 1, policy_version 548957 (0.0010) [2023-12-26 19:21:30,179][105620] Updated weights for policy 1, policy_version 548967 (0.0009) [2023-12-26 19:21:30,230][105692] Updated weights for policy 0, policy_version 548166 (0.0008) [2023-12-26 19:21:30,293][105692] Updated weights for policy 0, policy_version 548176 (0.0009) [2023-12-26 19:21:30,358][105692] Updated weights for policy 0, policy_version 548186 (0.0008) [2023-12-26 19:21:30,931][105620] Updated weights for policy 1, policy_version 548977 (0.0008) [2023-12-26 19:21:30,980][105620] Updated weights for policy 1, policy_version 548987 (0.0009) [2023-12-26 19:21:31,039][105620] Updated weights for policy 1, policy_version 548997 (0.0008) [2023-12-26 19:21:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 280911872. Throughput: 0: 9541.8, 1: 9665.2. Samples: 280880256. Policy #0 lag: (min: 28.0, avg: 41.2, max: 60.0) [2023-12-26 19:21:31,062][104569] Avg episode reward: [(0, '1204.267'), (1, '8990.782')] [2023-12-26 19:21:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000549000_140558336.pth... [2023-12-26 19:21:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000547848_140263424.pth [2023-12-26 19:21:31,092][105692] Updated weights for policy 0, policy_version 548196 (0.0007) [2023-12-26 19:21:31,158][105692] Updated weights for policy 0, policy_version 548206 (0.0007) [2023-12-26 19:21:31,217][105692] Updated weights for policy 0, policy_version 548216 (0.0009) [2023-12-26 19:21:31,271][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000548224_140361728.pth... [2023-12-26 19:21:31,276][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000547104_140075008.pth [2023-12-26 19:21:31,858][105620] Updated weights for policy 1, policy_version 549007 (0.0010) [2023-12-26 19:21:31,910][105620] Updated weights for policy 1, policy_version 549018 (0.0009) [2023-12-26 19:21:31,926][105692] Updated weights for policy 0, policy_version 548226 (0.0006) [2023-12-26 19:21:31,967][105620] Updated weights for policy 1, policy_version 549028 (0.0010) [2023-12-26 19:21:31,985][105692] Updated weights for policy 0, policy_version 548236 (0.0006) [2023-12-26 19:21:32,045][105692] Updated weights for policy 0, policy_version 548246 (0.0009) [2023-12-26 19:21:32,594][105620] Updated weights for policy 1, policy_version 549038 (0.0010) [2023-12-26 19:21:32,651][105620] Updated weights for policy 1, policy_version 549048 (0.0010) [2023-12-26 19:21:32,713][105620] Updated weights for policy 1, policy_version 549058 (0.0010) [2023-12-26 19:21:32,757][105692] Updated weights for policy 0, policy_version 548257 (0.0009) [2023-12-26 19:21:32,815][105692] Updated weights for policy 0, policy_version 548267 (0.0009) [2023-12-26 19:21:32,877][105692] Updated weights for policy 0, policy_version 548277 (0.0009) [2023-12-26 19:21:32,935][105692] Updated weights for policy 0, policy_version 548287 (0.0009) [2023-12-26 19:21:33,366][105620] Updated weights for policy 1, policy_version 549068 (0.0009) [2023-12-26 19:21:33,414][105620] Updated weights for policy 1, policy_version 549078 (0.0005) [2023-12-26 19:21:33,464][105620] Updated weights for policy 1, policy_version 549088 (0.0005) [2023-12-26 19:21:33,606][105692] Updated weights for policy 0, policy_version 548297 (0.0010) [2023-12-26 19:21:33,669][105692] Updated weights for policy 0, policy_version 548308 (0.0010) [2023-12-26 19:21:33,720][105692] Updated weights for policy 0, policy_version 548318 (0.0010) [2023-12-26 19:21:34,058][105620] Updated weights for policy 1, policy_version 549098 (0.0005) [2023-12-26 19:21:34,120][105620] Updated weights for policy 1, policy_version 549108 (0.0005) [2023-12-26 19:21:34,185][105620] Updated weights for policy 1, policy_version 549118 (0.0008) [2023-12-26 19:21:34,239][105620] Updated weights for policy 1, policy_version 549128 (0.0008) [2023-12-26 19:21:34,509][105692] Updated weights for policy 0, policy_version 548328 (0.0006) [2023-12-26 19:21:34,562][105692] Updated weights for policy 0, policy_version 548338 (0.0007) [2023-12-26 19:21:34,614][105692] Updated weights for policy 0, policy_version 548348 (0.0008) [2023-12-26 19:21:34,968][105620] Updated weights for policy 1, policy_version 549138 (0.0009) [2023-12-26 19:21:35,026][105620] Updated weights for policy 1, policy_version 549149 (0.0009) [2023-12-26 19:21:35,081][105620] Updated weights for policy 1, policy_version 549159 (0.0009) [2023-12-26 19:21:35,322][105692] Updated weights for policy 0, policy_version 548358 (0.0009) [2023-12-26 19:21:35,375][105692] Updated weights for policy 0, policy_version 548368 (0.0008) [2023-12-26 19:21:35,421][105692] Updated weights for policy 0, policy_version 548378 (0.0008) [2023-12-26 19:21:35,857][105620] Updated weights for policy 1, policy_version 549169 (0.0008) [2023-12-26 19:21:35,902][105620] Updated weights for policy 1, policy_version 549179 (0.0008) [2023-12-26 19:21:35,962][105620] Updated weights for policy 1, policy_version 549189 (0.0009) [2023-12-26 19:21:36,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19114.6, 300 sec: 19494.2). Total num frames: 281010176. Throughput: 0: 9446.5, 1: 9730.7. Samples: 280998352. Policy #0 lag: (min: 28.0, avg: 41.2, max: 60.0) [2023-12-26 19:21:36,063][104569] Avg episode reward: [(0, '6145.622'), (1, '9082.589')] [2023-12-26 19:21:36,165][105692] Updated weights for policy 0, policy_version 548388 (0.0008) [2023-12-26 19:21:36,227][105692] Updated weights for policy 0, policy_version 548398 (0.0009) [2023-12-26 19:21:36,290][105692] Updated weights for policy 0, policy_version 548408 (0.0009) [2023-12-26 19:21:36,729][105620] Updated weights for policy 1, policy_version 549199 (0.0009) [2023-12-26 19:21:36,784][105620] Updated weights for policy 1, policy_version 549209 (0.0008) [2023-12-26 19:21:36,830][105620] Updated weights for policy 1, policy_version 549219 (0.0009) [2023-12-26 19:21:37,000][105692] Updated weights for policy 0, policy_version 548418 (0.0009) [2023-12-26 19:21:37,059][105692] Updated weights for policy 0, policy_version 548428 (0.0009) [2023-12-26 19:21:37,125][105692] Updated weights for policy 0, policy_version 548438 (0.0006) [2023-12-26 19:21:37,175][105692] Updated weights for policy 0, policy_version 548448 (0.0005) [2023-12-26 19:21:37,588][105620] Updated weights for policy 1, policy_version 549229 (0.0009) [2023-12-26 19:21:37,642][105620] Updated weights for policy 1, policy_version 549239 (0.0009) [2023-12-26 19:21:37,708][105620] Updated weights for policy 1, policy_version 549249 (0.0008) [2023-12-26 19:21:37,870][105692] Updated weights for policy 0, policy_version 548458 (0.0009) [2023-12-26 19:21:37,887][105585] KL-divergence is very high: 130.3624 [2023-12-26 19:21:37,925][105692] Updated weights for policy 0, policy_version 548468 (0.0010) [2023-12-26 19:21:37,929][105585] KL-divergence is very high: 243.7886 [2023-12-26 19:21:37,983][105585] KL-divergence is very high: 256.3145 [2023-12-26 19:21:37,987][105692] Updated weights for policy 0, policy_version 548478 (0.0008) [2023-12-26 19:21:38,304][105620] Updated weights for policy 1, policy_version 549259 (0.0007) [2023-12-26 19:21:38,369][105620] Updated weights for policy 1, policy_version 549269 (0.0008) [2023-12-26 19:21:38,438][105620] Updated weights for policy 1, policy_version 549279 (0.0009) [2023-12-26 19:21:38,760][105692] Updated weights for policy 0, policy_version 548488 (0.0009) [2023-12-26 19:21:38,826][105692] Updated weights for policy 0, policy_version 548498 (0.0009) [2023-12-26 19:21:38,889][105692] Updated weights for policy 0, policy_version 548508 (0.0009) [2023-12-26 19:21:39,144][105620] Updated weights for policy 1, policy_version 549289 (0.0009) [2023-12-26 19:21:39,206][105620] Updated weights for policy 1, policy_version 549299 (0.0009) [2023-12-26 19:21:39,266][105620] Updated weights for policy 1, policy_version 549309 (0.0009) [2023-12-26 19:21:39,331][105620] Updated weights for policy 1, policy_version 549319 (0.0009) [2023-12-26 19:21:39,600][105692] Updated weights for policy 0, policy_version 548518 (0.0009) [2023-12-26 19:21:39,654][105692] Updated weights for policy 0, policy_version 548528 (0.0011) [2023-12-26 19:21:39,706][105692] Updated weights for policy 0, policy_version 548538 (0.0009) [2023-12-26 19:21:40,223][105620] Updated weights for policy 1, policy_version 549329 (0.0010) [2023-12-26 19:21:40,279][105586] KL-divergence is very high: 199.4056 [2023-12-26 19:21:40,285][105620] Updated weights for policy 1, policy_version 549339 (0.0010) [2023-12-26 19:21:40,334][105586] KL-divergence is very high: 374.7798 [2023-12-26 19:21:40,337][105692] Updated weights for policy 0, policy_version 548548 (0.0006) [2023-12-26 19:21:40,352][105620] Updated weights for policy 1, policy_version 549349 (0.0008) [2023-12-26 19:21:40,402][105692] Updated weights for policy 0, policy_version 548558 (0.0008) [2023-12-26 19:21:40,468][105692] Updated weights for policy 0, policy_version 548568 (0.0009) [2023-12-26 19:21:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 281100288. Throughput: 0: 9507.6, 1: 9730.0. Samples: 281113068. Policy #0 lag: (min: 28.0, avg: 41.2, max: 60.0) [2023-12-26 19:21:41,063][104569] Avg episode reward: [(0, '4781.381'), (1, '9081.848')] [2023-12-26 19:21:41,063][105692] Updated weights for policy 0, policy_version 548578 (0.0010) [2023-12-26 19:21:41,086][105620] Updated weights for policy 1, policy_version 549359 (0.0010) [2023-12-26 19:21:41,121][105692] Updated weights for policy 0, policy_version 548588 (0.0011) [2023-12-26 19:21:41,148][105620] Updated weights for policy 1, policy_version 549369 (0.0010) [2023-12-26 19:21:41,197][105692] Updated weights for policy 0, policy_version 548598 (0.0010) [2023-12-26 19:21:41,218][105620] Updated weights for policy 1, policy_version 549379 (0.0008) [2023-12-26 19:21:41,258][105692] Updated weights for policy 0, policy_version 548608 (0.0010) [2023-12-26 19:21:42,022][105620] Updated weights for policy 1, policy_version 549389 (0.0006) [2023-12-26 19:21:42,036][105692] Updated weights for policy 0, policy_version 548618 (0.0010) [2023-12-26 19:21:42,075][105620] Updated weights for policy 1, policy_version 549399 (0.0006) [2023-12-26 19:21:42,096][105692] Updated weights for policy 0, policy_version 548628 (0.0010) [2023-12-26 19:21:42,138][105620] Updated weights for policy 1, policy_version 549409 (0.0006) [2023-12-26 19:21:42,155][105692] Updated weights for policy 0, policy_version 548638 (0.0010) [2023-12-26 19:21:42,792][105692] Updated weights for policy 0, policy_version 548648 (0.0010) [2023-12-26 19:21:42,857][105692] Updated weights for policy 0, policy_version 548658 (0.0010) [2023-12-26 19:21:42,927][105692] Updated weights for policy 0, policy_version 548668 (0.0010) [2023-12-26 19:21:42,929][105620] Updated weights for policy 1, policy_version 549419 (0.0006) [2023-12-26 19:21:42,988][105620] Updated weights for policy 1, policy_version 549429 (0.0007) [2023-12-26 19:21:43,055][105620] Updated weights for policy 1, policy_version 549439 (0.0009) [2023-12-26 19:21:43,607][105692] Updated weights for policy 0, policy_version 548678 (0.0011) [2023-12-26 19:21:43,659][105692] Updated weights for policy 0, policy_version 548688 (0.0011) [2023-12-26 19:21:43,718][105692] Updated weights for policy 0, policy_version 548698 (0.0011) [2023-12-26 19:21:43,768][105620] Updated weights for policy 1, policy_version 549449 (0.0010) [2023-12-26 19:21:43,827][105620] Updated weights for policy 1, policy_version 549459 (0.0008) [2023-12-26 19:21:43,889][105620] Updated weights for policy 1, policy_version 549469 (0.0008) [2023-12-26 19:21:43,946][105620] Updated weights for policy 1, policy_version 549479 (0.0008) [2023-12-26 19:21:44,396][105692] Updated weights for policy 0, policy_version 548708 (0.0009) [2023-12-26 19:21:44,449][105692] Updated weights for policy 0, policy_version 548718 (0.0005) [2023-12-26 19:21:44,504][105692] Updated weights for policy 0, policy_version 548728 (0.0005) [2023-12-26 19:21:44,725][105620] Updated weights for policy 1, policy_version 549489 (0.0010) [2023-12-26 19:21:44,787][105620] Updated weights for policy 1, policy_version 549499 (0.0010) [2023-12-26 19:21:44,855][105620] Updated weights for policy 1, policy_version 549509 (0.0010) [2023-12-26 19:21:45,222][105692] Updated weights for policy 0, policy_version 548738 (0.0007) [2023-12-26 19:21:45,282][105692] Updated weights for policy 0, policy_version 548748 (0.0011) [2023-12-26 19:21:45,344][105692] Updated weights for policy 0, policy_version 548758 (0.0011) [2023-12-26 19:21:45,403][105692] Updated weights for policy 0, policy_version 548768 (0.0010) [2023-12-26 19:21:45,556][105620] Updated weights for policy 1, policy_version 549519 (0.0008) [2023-12-26 19:21:45,621][105620] Updated weights for policy 1, policy_version 549529 (0.0007) [2023-12-26 19:21:45,681][105620] Updated weights for policy 1, policy_version 549539 (0.0008) [2023-12-26 19:21:46,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 281198592. Throughput: 0: 9566.8, 1: 9651.7. Samples: 281170612. Policy #0 lag: (min: 28.0, avg: 41.2, max: 60.0) [2023-12-26 19:21:46,062][104569] Avg episode reward: [(0, '2875.349'), (1, '9085.324')] [2023-12-26 19:21:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000549544_140697600.pth... [2023-12-26 19:21:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000548424_140410880.pth [2023-12-26 19:21:46,073][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_000549544_140697600.pth [2023-12-26 19:21:46,107][105692] Updated weights for policy 0, policy_version 548778 (0.0011) [2023-12-26 19:21:46,165][105692] Updated weights for policy 0, policy_version 548788 (0.0009) [2023-12-26 19:21:46,223][105692] Updated weights for policy 0, policy_version 548798 (0.0009) [2023-12-26 19:21:46,232][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000548800_140509184.pth... [2023-12-26 19:21:46,235][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000547648_140214272.pth [2023-12-26 19:21:46,236][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_000548800_140509184.pth [2023-12-26 19:21:46,421][105620] Updated weights for policy 1, policy_version 549549 (0.0007) [2023-12-26 19:21:46,490][105620] Updated weights for policy 1, policy_version 549559 (0.0005) [2023-12-26 19:21:46,563][105620] Updated weights for policy 1, policy_version 549569 (0.0005) [2023-12-26 19:21:46,901][105692] Updated weights for policy 0, policy_version 548808 (0.0006) [2023-12-26 19:21:46,960][105692] Updated weights for policy 0, policy_version 548818 (0.0007) [2023-12-26 19:21:47,023][105692] Updated weights for policy 0, policy_version 548828 (0.0007) [2023-12-26 19:21:47,122][105620] Updated weights for policy 1, policy_version 549579 (0.0009) [2023-12-26 19:21:47,172][105620] Updated weights for policy 1, policy_version 549589 (0.0008) [2023-12-26 19:21:47,233][105620] Updated weights for policy 1, policy_version 549599 (0.0008) [2023-12-26 19:21:47,670][105692] Updated weights for policy 0, policy_version 548838 (0.0009) [2023-12-26 19:21:47,727][105692] Updated weights for policy 0, policy_version 548848 (0.0009) [2023-12-26 19:21:47,774][105692] Updated weights for policy 0, policy_version 548858 (0.0008) [2023-12-26 19:21:47,979][105620] Updated weights for policy 1, policy_version 549609 (0.0009) [2023-12-26 19:21:48,048][105620] Updated weights for policy 1, policy_version 549619 (0.0006) [2023-12-26 19:21:48,107][105620] Updated weights for policy 1, policy_version 549629 (0.0005) [2023-12-26 19:21:48,161][105620] Updated weights for policy 1, policy_version 549639 (0.0008) [2023-12-26 19:21:48,430][105692] Updated weights for policy 0, policy_version 548868 (0.0008) [2023-12-26 19:21:48,493][105692] Updated weights for policy 0, policy_version 548878 (0.0006) [2023-12-26 19:21:48,559][105692] Updated weights for policy 0, policy_version 548888 (0.0010) [2023-12-26 19:21:48,711][105620] Updated weights for policy 1, policy_version 549649 (0.0005) [2023-12-26 19:21:48,765][105620] Updated weights for policy 1, policy_version 549659 (0.0007) [2023-12-26 19:21:48,816][105620] Updated weights for policy 1, policy_version 549669 (0.0007) [2023-12-26 19:21:49,225][105692] Updated weights for policy 0, policy_version 548899 (0.0009) [2023-12-26 19:21:49,295][105585] KL-divergence is very high: 156.2164 [2023-12-26 19:21:49,296][105692] Updated weights for policy 0, policy_version 548909 (0.0007) [2023-12-26 19:21:49,303][105585] KL-divergence is very high: 134.8769 [2023-12-26 19:21:49,310][105585] KL-divergence is very high: 201.5126 [2023-12-26 19:21:49,319][105585] KL-divergence is very high: 131.7251 [2023-12-26 19:21:49,326][105585] KL-divergence is very high: 197.1714 [2023-12-26 19:21:49,333][105585] KL-divergence is very high: 160.2287 [2023-12-26 19:21:49,356][105585] KL-divergence is very high: 102.5108 [2023-12-26 19:21:49,371][105585] KL-divergence is very high: 119.7868 [2023-12-26 19:21:49,373][105692] Updated weights for policy 0, policy_version 548919 (0.0008) [2023-12-26 19:21:49,492][105620] Updated weights for policy 1, policy_version 549679 (0.0007) [2023-12-26 19:21:49,557][105620] Updated weights for policy 1, policy_version 549689 (0.0008) [2023-12-26 19:21:49,609][105620] Updated weights for policy 1, policy_version 549699 (0.0010) [2023-12-26 19:21:50,020][105692] Updated weights for policy 0, policy_version 548929 (0.0008) [2023-12-26 19:21:50,087][105692] Updated weights for policy 0, policy_version 548939 (0.0009) [2023-12-26 19:21:50,150][105692] Updated weights for policy 0, policy_version 548949 (0.0009) [2023-12-26 19:21:50,213][105692] Updated weights for policy 0, policy_version 548959 (0.0010) [2023-12-26 19:21:50,277][105620] Updated weights for policy 1, policy_version 549709 (0.0006) [2023-12-26 19:21:50,337][105620] Updated weights for policy 1, policy_version 549719 (0.0008) [2023-12-26 19:21:50,402][105620] Updated weights for policy 1, policy_version 549729 (0.0006) [2023-12-26 19:21:50,962][105692] Updated weights for policy 0, policy_version 548969 (0.0007) [2023-12-26 19:21:51,032][105692] Updated weights for policy 0, policy_version 548979 (0.0009) [2023-12-26 19:21:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 281296896. Throughput: 0: 9651.7, 1: 9771.8. Samples: 281291932. Policy #0 lag: (min: 28.0, avg: 41.2, max: 60.0) [2023-12-26 19:21:51,062][104569] Avg episode reward: [(0, '5478.500'), (1, '8997.729')] [2023-12-26 19:21:51,092][105692] Updated weights for policy 0, policy_version 548989 (0.0006) [2023-12-26 19:21:51,149][105620] Updated weights for policy 1, policy_version 549739 (0.0009) [2023-12-26 19:21:51,213][105620] Updated weights for policy 1, policy_version 549749 (0.0008) [2023-12-26 19:21:51,277][105620] Updated weights for policy 1, policy_version 549759 (0.0008) [2023-12-26 19:21:51,828][105692] Updated weights for policy 0, policy_version 548999 (0.0007) [2023-12-26 19:21:51,894][105692] Updated weights for policy 0, policy_version 549009 (0.0008) [2023-12-26 19:21:51,960][105692] Updated weights for policy 0, policy_version 549019 (0.0009) [2023-12-26 19:21:51,975][105620] Updated weights for policy 1, policy_version 549769 (0.0009) [2023-12-26 19:21:52,032][105620] Updated weights for policy 1, policy_version 549779 (0.0009) [2023-12-26 19:21:52,091][105620] Updated weights for policy 1, policy_version 549789 (0.0005) [2023-12-26 19:21:52,140][105620] Updated weights for policy 1, policy_version 549799 (0.0008) [2023-12-26 19:21:52,628][105692] Updated weights for policy 0, policy_version 549029 (0.0007) [2023-12-26 19:21:52,685][105692] Updated weights for policy 0, policy_version 549039 (0.0008) [2023-12-26 19:21:52,742][105692] Updated weights for policy 0, policy_version 549049 (0.0011) [2023-12-26 19:21:52,819][105620] Updated weights for policy 1, policy_version 549809 (0.0007) [2023-12-26 19:21:52,885][105620] Updated weights for policy 1, policy_version 549819 (0.0010) [2023-12-26 19:21:52,940][105620] Updated weights for policy 1, policy_version 549829 (0.0010) [2023-12-26 19:21:53,424][105692] Updated weights for policy 0, policy_version 549059 (0.0009) [2023-12-26 19:21:53,480][105692] Updated weights for policy 0, policy_version 549069 (0.0009) [2023-12-26 19:21:53,538][105692] Updated weights for policy 0, policy_version 549079 (0.0010) [2023-12-26 19:21:53,607][105620] Updated weights for policy 1, policy_version 549839 (0.0007) [2023-12-26 19:21:53,673][105620] Updated weights for policy 1, policy_version 549849 (0.0005) [2023-12-26 19:21:53,740][105620] Updated weights for policy 1, policy_version 549859 (0.0006) [2023-12-26 19:21:54,223][105692] Updated weights for policy 0, policy_version 549089 (0.0010) [2023-12-26 19:21:54,233][105620] Updated weights for policy 1, policy_version 549869 (0.0006) [2023-12-26 19:21:54,279][105692] Updated weights for policy 0, policy_version 549099 (0.0007) [2023-12-26 19:21:54,291][105620] Updated weights for policy 1, policy_version 549879 (0.0006) [2023-12-26 19:21:54,333][105692] Updated weights for policy 0, policy_version 549109 (0.0005) [2023-12-26 19:21:54,340][105620] Updated weights for policy 1, policy_version 549889 (0.0008) [2023-12-26 19:21:54,392][105692] Updated weights for policy 0, policy_version 549119 (0.0005) [2023-12-26 19:21:54,956][105620] Updated weights for policy 1, policy_version 549899 (0.0007) [2023-12-26 19:21:55,017][105620] Updated weights for policy 1, policy_version 549909 (0.0009) [2023-12-26 19:21:55,071][105620] Updated weights for policy 1, policy_version 549919 (0.0009) [2023-12-26 19:21:55,081][105692] Updated weights for policy 0, policy_version 549129 (0.0008) [2023-12-26 19:21:55,108][105585] KL-divergence is very high: 254.3669 [2023-12-26 19:21:55,117][105585] KL-divergence is very high: 195.9981 [2023-12-26 19:21:55,128][105692] Updated weights for policy 0, policy_version 549139 (0.0010) [2023-12-26 19:21:55,151][105585] KL-divergence is very high: 409.5067 [2023-12-26 19:21:55,162][105585] KL-divergence is very high: 234.7586 [2023-12-26 19:21:55,187][105692] Updated weights for policy 0, policy_version 549149 (0.0009) [2023-12-26 19:21:55,200][105585] KL-divergence is very high: 362.8922 [2023-12-26 19:21:55,827][105620] Updated weights for policy 1, policy_version 549929 (0.0007) [2023-12-26 19:21:55,860][105692] Updated weights for policy 0, policy_version 549159 (0.0008) [2023-12-26 19:21:55,884][105620] Updated weights for policy 1, policy_version 549939 (0.0009) [2023-12-26 19:21:55,910][105692] Updated weights for policy 0, policy_version 549169 (0.0005) [2023-12-26 19:21:55,939][105620] Updated weights for policy 1, policy_version 549949 (0.0008) [2023-12-26 19:21:55,968][105692] Updated weights for policy 0, policy_version 549179 (0.0007) [2023-12-26 19:21:55,994][105620] Updated weights for policy 1, policy_version 549959 (0.0006) [2023-12-26 19:21:56,062][104569] Fps is (10 sec: 21299.3, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 281411584. Throughput: 0: 9655.9, 1: 9811.2. Samples: 281413144. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-26 19:21:56,062][104569] Avg episode reward: [(0, '7028.249'), (1, '9174.148')] [2023-12-26 19:21:56,519][105692] Updated weights for policy 0, policy_version 549189 (0.0007) [2023-12-26 19:21:56,578][105692] Updated weights for policy 0, policy_version 549199 (0.0005) [2023-12-26 19:21:56,645][105692] Updated weights for policy 0, policy_version 549209 (0.0005) [2023-12-26 19:21:56,839][105620] Updated weights for policy 1, policy_version 549969 (0.0008) [2023-12-26 19:21:56,886][105620] Updated weights for policy 1, policy_version 549979 (0.0008) [2023-12-26 19:21:56,933][105620] Updated weights for policy 1, policy_version 549989 (0.0008) [2023-12-26 19:21:57,222][105692] Updated weights for policy 0, policy_version 549219 (0.0008) [2023-12-26 19:21:57,282][105692] Updated weights for policy 0, policy_version 549229 (0.0010) [2023-12-26 19:21:57,343][105692] Updated weights for policy 0, policy_version 549239 (0.0005) [2023-12-26 19:21:57,630][105620] Updated weights for policy 1, policy_version 549999 (0.0006) [2023-12-26 19:21:57,691][105620] Updated weights for policy 1, policy_version 550009 (0.0008) [2023-12-26 19:21:57,756][105620] Updated weights for policy 1, policy_version 550020 (0.0011) [2023-12-26 19:21:57,969][105692] Updated weights for policy 0, policy_version 549249 (0.0006) [2023-12-26 19:21:58,034][105692] Updated weights for policy 0, policy_version 549259 (0.0005) [2023-12-26 19:21:58,099][105692] Updated weights for policy 0, policy_version 549269 (0.0006) [2023-12-26 19:21:58,166][105692] Updated weights for policy 0, policy_version 549279 (0.0008) [2023-12-26 19:21:58,441][105620] Updated weights for policy 1, policy_version 550030 (0.0006) [2023-12-26 19:21:58,508][105620] Updated weights for policy 1, policy_version 550040 (0.0009) [2023-12-26 19:21:58,566][105620] Updated weights for policy 1, policy_version 550050 (0.0009) [2023-12-26 19:21:59,004][105692] Updated weights for policy 0, policy_version 549289 (0.0008) [2023-12-26 19:21:59,075][105692] Updated weights for policy 0, policy_version 549299 (0.0008) [2023-12-26 19:21:59,138][105692] Updated weights for policy 0, policy_version 549309 (0.0007) [2023-12-26 19:21:59,252][105620] Updated weights for policy 1, policy_version 550060 (0.0008) [2023-12-26 19:21:59,308][105620] Updated weights for policy 1, policy_version 550070 (0.0009) [2023-12-26 19:21:59,379][105620] Updated weights for policy 1, policy_version 550080 (0.0008) [2023-12-26 19:21:59,879][105692] Updated weights for policy 0, policy_version 549319 (0.0008) [2023-12-26 19:21:59,938][105692] Updated weights for policy 0, policy_version 549329 (0.0008) [2023-12-26 19:21:59,993][105692] Updated weights for policy 0, policy_version 549339 (0.0008) [2023-12-26 19:22:00,158][105620] Updated weights for policy 1, policy_version 550090 (0.0009) [2023-12-26 19:22:00,229][105620] Updated weights for policy 1, policy_version 550100 (0.0011) [2023-12-26 19:22:00,274][105620] Updated weights for policy 1, policy_version 550110 (0.0010) [2023-12-26 19:22:00,326][105620] Updated weights for policy 1, policy_version 550120 (0.0010) [2023-12-26 19:22:00,777][105692] Updated weights for policy 0, policy_version 549349 (0.0008) [2023-12-26 19:22:00,838][105692] Updated weights for policy 0, policy_version 549359 (0.0007) [2023-12-26 19:22:00,903][105692] Updated weights for policy 0, policy_version 549369 (0.0010) [2023-12-26 19:22:01,021][105620] Updated weights for policy 1, policy_version 550130 (0.0010) [2023-12-26 19:22:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 281501696. Throughput: 0: 9751.7, 1: 9814.0. Samples: 281474404. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-26 19:22:01,063][104569] Avg episode reward: [(0, '8526.903'), (1, '9265.161')] [2023-12-26 19:22:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000549376_140656640.pth... [2023-12-26 19:22:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000548224_140361728.pth [2023-12-26 19:22:01,089][105620] Updated weights for policy 1, policy_version 550140 (0.0011) [2023-12-26 19:22:01,155][105620] Updated weights for policy 1, policy_version 550150 (0.0011) [2023-12-26 19:22:01,165][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000550152_140853248.pth... [2023-12-26 19:22:01,170][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000549000_140558336.pth [2023-12-26 19:22:01,671][105692] Updated weights for policy 0, policy_version 549379 (0.0007) [2023-12-26 19:22:01,739][105692] Updated weights for policy 0, policy_version 549389 (0.0007) [2023-12-26 19:22:01,802][105692] Updated weights for policy 0, policy_version 549399 (0.0008) [2023-12-26 19:22:01,863][105620] Updated weights for policy 1, policy_version 550160 (0.0010) [2023-12-26 19:22:01,919][105620] Updated weights for policy 1, policy_version 550170 (0.0010) [2023-12-26 19:22:01,973][105620] Updated weights for policy 1, policy_version 550180 (0.0010) [2023-12-26 19:22:02,367][105692] Updated weights for policy 0, policy_version 549409 (0.0008) [2023-12-26 19:22:02,418][105692] Updated weights for policy 0, policy_version 549419 (0.0008) [2023-12-26 19:22:02,476][105692] Updated weights for policy 0, policy_version 549429 (0.0008) [2023-12-26 19:22:02,535][105692] Updated weights for policy 0, policy_version 549439 (0.0008) [2023-12-26 19:22:02,648][105620] Updated weights for policy 1, policy_version 550190 (0.0007) [2023-12-26 19:22:02,707][105620] Updated weights for policy 1, policy_version 550200 (0.0005) [2023-12-26 19:22:02,759][105620] Updated weights for policy 1, policy_version 550210 (0.0005) [2023-12-26 19:22:03,366][105692] Updated weights for policy 0, policy_version 549449 (0.0008) [2023-12-26 19:22:03,382][105620] Updated weights for policy 1, policy_version 550220 (0.0007) [2023-12-26 19:22:03,424][105692] Updated weights for policy 0, policy_version 549459 (0.0007) [2023-12-26 19:22:03,427][105620] Updated weights for policy 1, policy_version 550230 (0.0010) [2023-12-26 19:22:03,472][105620] Updated weights for policy 1, policy_version 550240 (0.0010) [2023-12-26 19:22:03,480][105692] Updated weights for policy 0, policy_version 549469 (0.0009) [2023-12-26 19:22:04,186][105692] Updated weights for policy 0, policy_version 549479 (0.0008) [2023-12-26 19:22:04,238][105620] Updated weights for policy 1, policy_version 550250 (0.0010) [2023-12-26 19:22:04,242][105692] Updated weights for policy 0, policy_version 549489 (0.0008) [2023-12-26 19:22:04,305][105620] Updated weights for policy 1, policy_version 550260 (0.0007) [2023-12-26 19:22:04,310][105692] Updated weights for policy 0, policy_version 549499 (0.0008) [2023-12-26 19:22:04,367][105620] Updated weights for policy 1, policy_version 550270 (0.0008) [2023-12-26 19:22:04,428][105620] Updated weights for policy 1, policy_version 550280 (0.0009) [2023-12-26 19:22:04,990][105620] Updated weights for policy 1, policy_version 550290 (0.0005) [2023-12-26 19:22:05,051][105620] Updated weights for policy 1, policy_version 550300 (0.0006) [2023-12-26 19:22:05,061][105692] Updated weights for policy 0, policy_version 549509 (0.0009) [2023-12-26 19:22:05,112][105620] Updated weights for policy 1, policy_version 550310 (0.0006) [2023-12-26 19:22:05,120][105692] Updated weights for policy 0, policy_version 549519 (0.0010) [2023-12-26 19:22:05,173][105692] Updated weights for policy 0, policy_version 549529 (0.0010) [2023-12-26 19:22:05,699][105620] Updated weights for policy 1, policy_version 550320 (0.0005) [2023-12-26 19:22:05,744][105620] Updated weights for policy 1, policy_version 550330 (0.0005) [2023-12-26 19:22:05,790][105620] Updated weights for policy 1, policy_version 550340 (0.0005) [2023-12-26 19:22:05,902][105692] Updated weights for policy 0, policy_version 549539 (0.0009) [2023-12-26 19:22:05,949][105692] Updated weights for policy 0, policy_version 549549 (0.0009) [2023-12-26 19:22:06,000][105692] Updated weights for policy 0, policy_version 549559 (0.0010) [2023-12-26 19:22:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 281608192. Throughput: 0: 9785.0, 1: 9832.4. Samples: 281590840. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-26 19:22:06,062][104569] Avg episode reward: [(0, '8723.140'), (1, '9354.164')] [2023-12-26 19:22:06,503][105620] Updated weights for policy 1, policy_version 550350 (0.0009) [2023-12-26 19:22:06,562][105620] Updated weights for policy 1, policy_version 550360 (0.0011) [2023-12-26 19:22:06,614][105620] Updated weights for policy 1, policy_version 550370 (0.0011) [2023-12-26 19:22:06,624][105692] Updated weights for policy 0, policy_version 549569 (0.0010) [2023-12-26 19:22:06,681][105692] Updated weights for policy 0, policy_version 549579 (0.0009) [2023-12-26 19:22:06,741][105692] Updated weights for policy 0, policy_version 549589 (0.0010) [2023-12-26 19:22:06,799][105692] Updated weights for policy 0, policy_version 549599 (0.0008) [2023-12-26 19:22:07,343][105620] Updated weights for policy 1, policy_version 550380 (0.0008) [2023-12-26 19:22:07,388][105620] Updated weights for policy 1, policy_version 550390 (0.0005) [2023-12-26 19:22:07,445][105620] Updated weights for policy 1, policy_version 550400 (0.0005) [2023-12-26 19:22:07,542][105692] Updated weights for policy 0, policy_version 549610 (0.0010) [2023-12-26 19:22:07,597][105692] Updated weights for policy 0, policy_version 549621 (0.0010) [2023-12-26 19:22:07,660][105692] Updated weights for policy 0, policy_version 549631 (0.0010) [2023-12-26 19:22:07,962][105620] Updated weights for policy 1, policy_version 550410 (0.0005) [2023-12-26 19:22:08,014][105620] Updated weights for policy 1, policy_version 550420 (0.0007) [2023-12-26 19:22:08,072][105620] Updated weights for policy 1, policy_version 550430 (0.0007) [2023-12-26 19:22:08,127][105620] Updated weights for policy 1, policy_version 550440 (0.0010) [2023-12-26 19:22:08,401][105692] Updated weights for policy 0, policy_version 549641 (0.0010) [2023-12-26 19:22:08,450][105692] Updated weights for policy 0, policy_version 549651 (0.0011) [2023-12-26 19:22:08,495][105692] Updated weights for policy 0, policy_version 549661 (0.0010) [2023-12-26 19:22:08,855][105620] Updated weights for policy 1, policy_version 550450 (0.0010) [2023-12-26 19:22:08,907][105620] Updated weights for policy 1, policy_version 550460 (0.0010) [2023-12-26 19:22:08,966][105620] Updated weights for policy 1, policy_version 550470 (0.0010) [2023-12-26 19:22:09,299][105692] Updated weights for policy 0, policy_version 549671 (0.0010) [2023-12-26 19:22:09,365][105692] Updated weights for policy 0, policy_version 549681 (0.0011) [2023-12-26 19:22:09,433][105692] Updated weights for policy 0, policy_version 549691 (0.0011) [2023-12-26 19:22:09,743][105620] Updated weights for policy 1, policy_version 550480 (0.0008) [2023-12-26 19:22:09,796][105620] Updated weights for policy 1, policy_version 550490 (0.0005) [2023-12-26 19:22:09,863][105620] Updated weights for policy 1, policy_version 550500 (0.0009) [2023-12-26 19:22:10,209][105692] Updated weights for policy 0, policy_version 549701 (0.0010) [2023-12-26 19:22:10,261][105692] Updated weights for policy 0, policy_version 549711 (0.0010) [2023-12-26 19:22:10,320][105692] Updated weights for policy 0, policy_version 549721 (0.0010) [2023-12-26 19:22:10,499][105620] Updated weights for policy 1, policy_version 550510 (0.0008) [2023-12-26 19:22:10,566][105620] Updated weights for policy 1, policy_version 550520 (0.0009) [2023-12-26 19:22:10,631][105620] Updated weights for policy 1, policy_version 550530 (0.0008) [2023-12-26 19:22:11,006][105692] Updated weights for policy 0, policy_version 549731 (0.0008) [2023-12-26 19:22:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 281698304. Throughput: 0: 9801.3, 1: 9958.9. Samples: 281710596. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-26 19:22:11,063][104569] Avg episode reward: [(0, '8213.532'), (1, '9354.095')] [2023-12-26 19:22:11,065][105692] Updated weights for policy 0, policy_version 549741 (0.0008) [2023-12-26 19:22:11,116][105692] Updated weights for policy 0, policy_version 549751 (0.0006) [2023-12-26 19:22:11,302][105620] Updated weights for policy 1, policy_version 550540 (0.0009) [2023-12-26 19:22:11,367][105620] Updated weights for policy 1, policy_version 550550 (0.0009) [2023-12-26 19:22:11,438][105620] Updated weights for policy 1, policy_version 550560 (0.0008) [2023-12-26 19:22:11,838][105692] Updated weights for policy 0, policy_version 549761 (0.0009) [2023-12-26 19:22:11,903][105692] Updated weights for policy 0, policy_version 549771 (0.0009) [2023-12-26 19:22:11,968][105692] Updated weights for policy 0, policy_version 549781 (0.0009) [2023-12-26 19:22:12,031][105692] Updated weights for policy 0, policy_version 549791 (0.0009) [2023-12-26 19:22:12,221][105620] Updated weights for policy 1, policy_version 550570 (0.0007) [2023-12-26 19:22:12,278][105620] Updated weights for policy 1, policy_version 550580 (0.0009) [2023-12-26 19:22:12,340][105620] Updated weights for policy 1, policy_version 550590 (0.0008) [2023-12-26 19:22:12,402][105620] Updated weights for policy 1, policy_version 550600 (0.0007) [2023-12-26 19:22:12,723][105692] Updated weights for policy 0, policy_version 549802 (0.0011) [2023-12-26 19:22:12,783][105692] Updated weights for policy 0, policy_version 549813 (0.0010) [2023-12-26 19:22:12,836][105692] Updated weights for policy 0, policy_version 549824 (0.0010) [2023-12-26 19:22:12,963][105620] Updated weights for policy 1, policy_version 550610 (0.0009) [2023-12-26 19:22:13,015][105620] Updated weights for policy 1, policy_version 550620 (0.0009) [2023-12-26 19:22:13,068][105620] Updated weights for policy 1, policy_version 550630 (0.0009) [2023-12-26 19:22:13,587][105692] Updated weights for policy 0, policy_version 549834 (0.0009) [2023-12-26 19:22:13,639][105692] Updated weights for policy 0, policy_version 549844 (0.0009) [2023-12-26 19:22:13,700][105692] Updated weights for policy 0, policy_version 549854 (0.0010) [2023-12-26 19:22:13,812][105620] Updated weights for policy 1, policy_version 550640 (0.0009) [2023-12-26 19:22:13,875][105620] Updated weights for policy 1, policy_version 550650 (0.0007) [2023-12-26 19:22:13,935][105620] Updated weights for policy 1, policy_version 550660 (0.0006) [2023-12-26 19:22:14,385][105692] Updated weights for policy 0, policy_version 549864 (0.0009) [2023-12-26 19:22:14,444][105692] Updated weights for policy 0, policy_version 549874 (0.0008) [2023-12-26 19:22:14,504][105692] Updated weights for policy 0, policy_version 549884 (0.0008) [2023-12-26 19:22:14,633][105620] Updated weights for policy 1, policy_version 550670 (0.0010) [2023-12-26 19:22:14,695][105620] Updated weights for policy 1, policy_version 550680 (0.0010) [2023-12-26 19:22:14,759][105620] Updated weights for policy 1, policy_version 550690 (0.0010) [2023-12-26 19:22:15,218][105692] Updated weights for policy 0, policy_version 549894 (0.0006) [2023-12-26 19:22:15,282][105692] Updated weights for policy 0, policy_version 549904 (0.0006) [2023-12-26 19:22:15,345][105692] Updated weights for policy 0, policy_version 549914 (0.0006) [2023-12-26 19:22:15,577][105620] Updated weights for policy 1, policy_version 550700 (0.0009) [2023-12-26 19:22:15,629][105620] Updated weights for policy 1, policy_version 550710 (0.0008) [2023-12-26 19:22:15,691][105620] Updated weights for policy 1, policy_version 550720 (0.0009) [2023-12-26 19:22:15,935][105692] Updated weights for policy 0, policy_version 549924 (0.0007) [2023-12-26 19:22:15,990][105692] Updated weights for policy 0, policy_version 549934 (0.0006) [2023-12-26 19:22:16,042][105692] Updated weights for policy 0, policy_version 549944 (0.0005) [2023-12-26 19:22:16,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 281796608. Throughput: 0: 9823.1, 1: 9936.9. Samples: 281769464. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-26 19:22:16,063][104569] Avg episode reward: [(0, '8489.681'), (1, '9354.142')] [2023-12-26 19:22:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000550728_141000704.pth... [2023-12-26 19:22:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000549544_140697600.pth [2023-12-26 19:22:16,081][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000549952_140804096.pth... [2023-12-26 19:22:16,084][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000548800_140509184.pth [2023-12-26 19:22:16,286][105620] Updated weights for policy 1, policy_version 550730 (0.0005) [2023-12-26 19:22:16,346][105620] Updated weights for policy 1, policy_version 550740 (0.0006) [2023-12-26 19:22:16,416][105620] Updated weights for policy 1, policy_version 550750 (0.0005) [2023-12-26 19:22:16,480][105620] Updated weights for policy 1, policy_version 550760 (0.0005) [2023-12-26 19:22:16,675][105692] Updated weights for policy 0, policy_version 549954 (0.0005) [2023-12-26 19:22:16,733][105692] Updated weights for policy 0, policy_version 549964 (0.0006) [2023-12-26 19:22:16,779][105692] Updated weights for policy 0, policy_version 549974 (0.0005) [2023-12-26 19:22:16,837][105692] Updated weights for policy 0, policy_version 549984 (0.0005) [2023-12-26 19:22:17,103][105620] Updated weights for policy 1, policy_version 550770 (0.0008) [2023-12-26 19:22:17,153][105620] Updated weights for policy 1, policy_version 550780 (0.0008) [2023-12-26 19:22:17,200][105620] Updated weights for policy 1, policy_version 550790 (0.0008) [2023-12-26 19:22:17,473][105692] Updated weights for policy 0, policy_version 549994 (0.0009) [2023-12-26 19:22:17,535][105692] Updated weights for policy 0, policy_version 550004 (0.0009) [2023-12-26 19:22:17,585][105692] Updated weights for policy 0, policy_version 550014 (0.0009) [2023-12-26 19:22:17,963][105620] Updated weights for policy 1, policy_version 550800 (0.0008) [2023-12-26 19:22:18,024][105620] Updated weights for policy 1, policy_version 550810 (0.0009) [2023-12-26 19:22:18,077][105620] Updated weights for policy 1, policy_version 550820 (0.0010) [2023-12-26 19:22:18,277][105692] Updated weights for policy 0, policy_version 550024 (0.0006) [2023-12-26 19:22:18,340][105692] Updated weights for policy 0, policy_version 550034 (0.0008) [2023-12-26 19:22:18,393][105692] Updated weights for policy 0, policy_version 550044 (0.0009) [2023-12-26 19:22:18,872][105620] Updated weights for policy 1, policy_version 550830 (0.0009) [2023-12-26 19:22:18,930][105620] Updated weights for policy 1, policy_version 550840 (0.0009) [2023-12-26 19:22:18,988][105620] Updated weights for policy 1, policy_version 550850 (0.0009) [2023-12-26 19:22:19,108][105692] Updated weights for policy 0, policy_version 550054 (0.0009) [2023-12-26 19:22:19,156][105692] Updated weights for policy 0, policy_version 550064 (0.0009) [2023-12-26 19:22:19,203][105692] Updated weights for policy 0, policy_version 550074 (0.0009) [2023-12-26 19:22:19,740][105620] Updated weights for policy 1, policy_version 550860 (0.0008) [2023-12-26 19:22:19,791][105620] Updated weights for policy 1, policy_version 550870 (0.0006) [2023-12-26 19:22:19,862][105620] Updated weights for policy 1, policy_version 550880 (0.0007) [2023-12-26 19:22:19,995][105692] Updated weights for policy 0, policy_version 550084 (0.0009) [2023-12-26 19:22:20,060][105692] Updated weights for policy 0, policy_version 550094 (0.0010) [2023-12-26 19:22:20,122][105692] Updated weights for policy 0, policy_version 550104 (0.0008) [2023-12-26 19:22:20,567][105620] Updated weights for policy 1, policy_version 550890 (0.0008) [2023-12-26 19:22:20,626][105620] Updated weights for policy 1, policy_version 550900 (0.0008) [2023-12-26 19:22:20,689][105620] Updated weights for policy 1, policy_version 550910 (0.0009) [2023-12-26 19:22:20,748][105620] Updated weights for policy 1, policy_version 550920 (0.0009) [2023-12-26 19:22:20,868][105692] Updated weights for policy 0, policy_version 550114 (0.0008) [2023-12-26 19:22:20,926][105692] Updated weights for policy 0, policy_version 550124 (0.0006) [2023-12-26 19:22:20,986][105692] Updated weights for policy 0, policy_version 550134 (0.0007) [2023-12-26 19:22:21,050][105692] Updated weights for policy 0, policy_version 550144 (0.0007) [2023-12-26 19:22:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 281903104. Throughput: 0: 9917.8, 1: 9868.6. Samples: 281888740. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-26 19:22:21,063][104569] Avg episode reward: [(0, '8817.460'), (1, '9262.416')] [2023-12-26 19:22:21,583][105620] Updated weights for policy 1, policy_version 550930 (0.0009) [2023-12-26 19:22:21,639][105620] Updated weights for policy 1, policy_version 550940 (0.0008) [2023-12-26 19:22:21,702][105620] Updated weights for policy 1, policy_version 550950 (0.0009) [2023-12-26 19:22:21,829][105692] Updated weights for policy 0, policy_version 550154 (0.0010) [2023-12-26 19:22:21,885][105692] Updated weights for policy 0, policy_version 550164 (0.0009) [2023-12-26 19:22:21,947][105692] Updated weights for policy 0, policy_version 550174 (0.0008) [2023-12-26 19:22:22,449][105620] Updated weights for policy 1, policy_version 550960 (0.0008) [2023-12-26 19:22:22,503][105620] Updated weights for policy 1, policy_version 550970 (0.0006) [2023-12-26 19:22:22,561][105620] Updated weights for policy 1, policy_version 550980 (0.0005) [2023-12-26 19:22:22,706][105692] Updated weights for policy 0, policy_version 550184 (0.0011) [2023-12-26 19:22:22,768][105692] Updated weights for policy 0, policy_version 550194 (0.0011) [2023-12-26 19:22:22,831][105692] Updated weights for policy 0, policy_version 550204 (0.0008) [2023-12-26 19:22:23,200][105620] Updated weights for policy 1, policy_version 550990 (0.0009) [2023-12-26 19:22:23,259][105620] Updated weights for policy 1, policy_version 551000 (0.0011) [2023-12-26 19:22:23,315][105620] Updated weights for policy 1, policy_version 551010 (0.0011) [2023-12-26 19:22:23,498][105692] Updated weights for policy 0, policy_version 550214 (0.0008) [2023-12-26 19:22:23,550][105692] Updated weights for policy 0, policy_version 550224 (0.0008) [2023-12-26 19:22:23,608][105692] Updated weights for policy 0, policy_version 550234 (0.0008) [2023-12-26 19:22:23,997][105620] Updated weights for policy 1, policy_version 551020 (0.0010) [2023-12-26 19:22:24,062][105620] Updated weights for policy 1, policy_version 551030 (0.0010) [2023-12-26 19:22:24,121][105620] Updated weights for policy 1, policy_version 551040 (0.0009) [2023-12-26 19:22:24,389][105692] Updated weights for policy 0, policy_version 550244 (0.0007) [2023-12-26 19:22:24,450][105692] Updated weights for policy 0, policy_version 550254 (0.0005) [2023-12-26 19:22:24,514][105692] Updated weights for policy 0, policy_version 550264 (0.0008) [2023-12-26 19:22:24,851][105620] Updated weights for policy 1, policy_version 551050 (0.0009) [2023-12-26 19:22:24,923][105620] Updated weights for policy 1, policy_version 551060 (0.0006) [2023-12-26 19:22:24,986][105620] Updated weights for policy 1, policy_version 551070 (0.0008) [2023-12-26 19:22:25,049][105620] Updated weights for policy 1, policy_version 551080 (0.0008) [2023-12-26 19:22:25,103][105692] Updated weights for policy 0, policy_version 550274 (0.0008) [2023-12-26 19:22:25,155][105692] Updated weights for policy 0, policy_version 550284 (0.0010) [2023-12-26 19:22:25,213][105692] Updated weights for policy 0, policy_version 550294 (0.0009) [2023-12-26 19:22:25,274][105692] Updated weights for policy 0, policy_version 550304 (0.0010) [2023-12-26 19:22:25,635][105620] Updated weights for policy 1, policy_version 551090 (0.0005) [2023-12-26 19:22:25,698][105620] Updated weights for policy 1, policy_version 551100 (0.0005) [2023-12-26 19:22:25,753][105620] Updated weights for policy 1, policy_version 551110 (0.0005) [2023-12-26 19:22:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 281993216. Throughput: 0: 9876.6, 1: 9950.4. Samples: 282005280. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-26 19:22:26,062][104569] Avg episode reward: [(0, '8810.120'), (1, '9261.085')] [2023-12-26 19:22:26,112][105692] Updated weights for policy 0, policy_version 550314 (0.0010) [2023-12-26 19:22:26,169][105692] Updated weights for policy 0, policy_version 550324 (0.0009) [2023-12-26 19:22:26,232][105692] Updated weights for policy 0, policy_version 550334 (0.0010) [2023-12-26 19:22:26,333][105620] Updated weights for policy 1, policy_version 551120 (0.0008) [2023-12-26 19:22:26,384][105620] Updated weights for policy 1, policy_version 551130 (0.0009) [2023-12-26 19:22:26,445][105620] Updated weights for policy 1, policy_version 551140 (0.0009) [2023-12-26 19:22:27,029][105692] Updated weights for policy 0, policy_version 550344 (0.0008) [2023-12-26 19:22:27,088][105692] Updated weights for policy 0, policy_version 550354 (0.0009) [2023-12-26 19:22:27,114][105620] Updated weights for policy 1, policy_version 551150 (0.0008) [2023-12-26 19:22:27,148][105692] Updated weights for policy 0, policy_version 550364 (0.0009) [2023-12-26 19:22:27,170][105620] Updated weights for policy 1, policy_version 551160 (0.0007) [2023-12-26 19:22:27,238][105620] Updated weights for policy 1, policy_version 551170 (0.0005) [2023-12-26 19:22:27,784][105620] Updated weights for policy 1, policy_version 551180 (0.0007) [2023-12-26 19:22:27,843][105620] Updated weights for policy 1, policy_version 551191 (0.0007) [2023-12-26 19:22:27,851][105692] Updated weights for policy 0, policy_version 550375 (0.0008) [2023-12-26 19:22:27,899][105692] Updated weights for policy 0, policy_version 550385 (0.0009) [2023-12-26 19:22:27,906][105620] Updated weights for policy 1, policy_version 551201 (0.0006) [2023-12-26 19:22:27,943][105692] Updated weights for policy 0, policy_version 550395 (0.0007) [2023-12-26 19:22:28,633][105692] Updated weights for policy 0, policy_version 550405 (0.0009) [2023-12-26 19:22:28,670][105620] Updated weights for policy 1, policy_version 551211 (0.0007) [2023-12-26 19:22:28,688][105692] Updated weights for policy 0, policy_version 550415 (0.0010) [2023-12-26 19:22:28,730][105620] Updated weights for policy 1, policy_version 551221 (0.0005) [2023-12-26 19:22:28,744][105692] Updated weights for policy 0, policy_version 550425 (0.0010) [2023-12-26 19:22:28,787][105620] Updated weights for policy 1, policy_version 551231 (0.0006) [2023-12-26 19:22:29,382][105692] Updated weights for policy 0, policy_version 550435 (0.0010) [2023-12-26 19:22:29,427][105692] Updated weights for policy 0, policy_version 550445 (0.0005) [2023-12-26 19:22:29,471][105692] Updated weights for policy 0, policy_version 550455 (0.0005) [2023-12-26 19:22:29,483][105620] Updated weights for policy 1, policy_version 551241 (0.0008) [2023-12-26 19:22:29,551][105620] Updated weights for policy 1, policy_version 551251 (0.0006) [2023-12-26 19:22:29,598][105620] Updated weights for policy 1, policy_version 551261 (0.0005) [2023-12-26 19:22:29,660][105620] Updated weights for policy 1, policy_version 551271 (0.0007) [2023-12-26 19:22:30,059][105692] Updated weights for policy 0, policy_version 550465 (0.0006) [2023-12-26 19:22:30,112][105692] Updated weights for policy 0, policy_version 550475 (0.0006) [2023-12-26 19:22:30,164][105692] Updated weights for policy 0, policy_version 550485 (0.0006) [2023-12-26 19:22:30,218][105692] Updated weights for policy 0, policy_version 550495 (0.0006) [2023-12-26 19:22:30,235][105620] Updated weights for policy 1, policy_version 551281 (0.0005) [2023-12-26 19:22:30,290][105620] Updated weights for policy 1, policy_version 551291 (0.0005) [2023-12-26 19:22:30,346][105620] Updated weights for policy 1, policy_version 551301 (0.0005) [2023-12-26 19:22:30,899][105620] Updated weights for policy 1, policy_version 551311 (0.0005) [2023-12-26 19:22:30,952][105620] Updated weights for policy 1, policy_version 551321 (0.0007) [2023-12-26 19:22:30,957][105692] Updated weights for policy 0, policy_version 550505 (0.0010) [2023-12-26 19:22:31,001][105692] Updated weights for policy 0, policy_version 550515 (0.0010) [2023-12-26 19:22:31,007][105620] Updated weights for policy 1, policy_version 551331 (0.0010) [2023-12-26 19:22:31,060][105692] Updated weights for policy 0, policy_version 550525 (0.0011) [2023-12-26 19:22:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 282099712. Throughput: 0: 9827.4, 1: 10041.9. Samples: 282064728. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-26 19:22:31,062][104569] Avg episode reward: [(0, '7791.119'), (1, '9352.172')] [2023-12-26 19:22:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000551336_141156352.pth... [2023-12-26 19:22:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000550152_140853248.pth [2023-12-26 19:22:31,076][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000550528_140951552.pth... [2023-12-26 19:22:31,081][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000549376_140656640.pth [2023-12-26 19:22:31,692][105620] Updated weights for policy 1, policy_version 551341 (0.0009) [2023-12-26 19:22:31,763][105620] Updated weights for policy 1, policy_version 551351 (0.0008) [2023-12-26 19:22:31,828][105620] Updated weights for policy 1, policy_version 551361 (0.0008) [2023-12-26 19:22:31,859][105692] Updated weights for policy 0, policy_version 550535 (0.0011) [2023-12-26 19:22:31,922][105692] Updated weights for policy 0, policy_version 550545 (0.0011) [2023-12-26 19:22:31,981][105692] Updated weights for policy 0, policy_version 550555 (0.0011) [2023-12-26 19:22:32,633][105620] Updated weights for policy 1, policy_version 551371 (0.0007) [2023-12-26 19:22:32,635][105692] Updated weights for policy 0, policy_version 550565 (0.0011) [2023-12-26 19:22:32,691][105620] Updated weights for policy 1, policy_version 551381 (0.0008) [2023-12-26 19:22:32,696][105692] Updated weights for policy 0, policy_version 550575 (0.0010) [2023-12-26 19:22:32,747][105620] Updated weights for policy 1, policy_version 551391 (0.0009) [2023-12-26 19:22:32,752][105692] Updated weights for policy 0, policy_version 550585 (0.0006) [2023-12-26 19:22:33,423][105692] Updated weights for policy 0, policy_version 550595 (0.0005) [2023-12-26 19:22:33,472][105692] Updated weights for policy 0, policy_version 550605 (0.0007) [2023-12-26 19:22:33,486][105620] Updated weights for policy 1, policy_version 551401 (0.0009) [2023-12-26 19:22:33,531][105692] Updated weights for policy 0, policy_version 550615 (0.0006) [2023-12-26 19:22:33,537][105620] Updated weights for policy 1, policy_version 551411 (0.0006) [2023-12-26 19:22:33,598][105620] Updated weights for policy 1, policy_version 551421 (0.0006) [2023-12-26 19:22:33,659][105620] Updated weights for policy 1, policy_version 551431 (0.0009) [2023-12-26 19:22:34,274][105692] Updated weights for policy 0, policy_version 550625 (0.0007) [2023-12-26 19:22:34,331][105692] Updated weights for policy 0, policy_version 550635 (0.0009) [2023-12-26 19:22:34,393][105692] Updated weights for policy 0, policy_version 550645 (0.0007) [2023-12-26 19:22:34,400][105620] Updated weights for policy 1, policy_version 551441 (0.0009) [2023-12-26 19:22:34,455][105692] Updated weights for policy 0, policy_version 550655 (0.0008) [2023-12-26 19:22:34,462][105620] Updated weights for policy 1, policy_version 551451 (0.0006) [2023-12-26 19:22:34,522][105620] Updated weights for policy 1, policy_version 551461 (0.0009) [2023-12-26 19:22:35,199][105692] Updated weights for policy 0, policy_version 550665 (0.0008) [2023-12-26 19:22:35,248][105692] Updated weights for policy 0, policy_version 550675 (0.0008) [2023-12-26 19:22:35,274][105620] Updated weights for policy 1, policy_version 551471 (0.0007) [2023-12-26 19:22:35,296][105692] Updated weights for policy 0, policy_version 550685 (0.0006) [2023-12-26 19:22:35,333][105620] Updated weights for policy 1, policy_version 551481 (0.0008) [2023-12-26 19:22:35,402][105620] Updated weights for policy 1, policy_version 551491 (0.0008) [2023-12-26 19:22:35,999][105692] Updated weights for policy 0, policy_version 550695 (0.0008) [2023-12-26 19:22:36,060][105692] Updated weights for policy 0, policy_version 550705 (0.0008) [2023-12-26 19:22:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.9, 300 sec: 19522.0). Total num frames: 282189824. Throughput: 0: 9814.3, 1: 10028.7. Samples: 282184868. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-26 19:22:36,062][104569] Avg episode reward: [(0, '8006.866'), (1, '9352.605')] [2023-12-26 19:22:36,110][105692] Updated weights for policy 0, policy_version 550715 (0.0008) [2023-12-26 19:22:36,175][105620] Updated weights for policy 1, policy_version 551501 (0.0009) [2023-12-26 19:22:36,233][105620] Updated weights for policy 1, policy_version 551511 (0.0008) [2023-12-26 19:22:36,294][105620] Updated weights for policy 1, policy_version 551521 (0.0009) [2023-12-26 19:22:36,884][105692] Updated weights for policy 0, policy_version 550725 (0.0009) [2023-12-26 19:22:36,933][105692] Updated weights for policy 0, policy_version 550735 (0.0010) [2023-12-26 19:22:36,996][105692] Updated weights for policy 0, policy_version 550745 (0.0010) [2023-12-26 19:22:37,082][105620] Updated weights for policy 1, policy_version 551531 (0.0009) [2023-12-26 19:22:37,138][105620] Updated weights for policy 1, policy_version 551541 (0.0008) [2023-12-26 19:22:37,197][105620] Updated weights for policy 1, policy_version 551551 (0.0008) [2023-12-26 19:22:37,732][105692] Updated weights for policy 0, policy_version 550755 (0.0010) [2023-12-26 19:22:37,781][105692] Updated weights for policy 0, policy_version 550765 (0.0010) [2023-12-26 19:22:37,840][105692] Updated weights for policy 0, policy_version 550775 (0.0010) [2023-12-26 19:22:37,978][105620] Updated weights for policy 1, policy_version 551561 (0.0008) [2023-12-26 19:22:38,037][105620] Updated weights for policy 1, policy_version 551571 (0.0008) [2023-12-26 19:22:38,099][105620] Updated weights for policy 1, policy_version 551581 (0.0007) [2023-12-26 19:22:38,168][105620] Updated weights for policy 1, policy_version 551591 (0.0005) [2023-12-26 19:22:38,599][105692] Updated weights for policy 0, policy_version 550785 (0.0010) [2023-12-26 19:22:38,658][105692] Updated weights for policy 0, policy_version 550795 (0.0011) [2023-12-26 19:22:38,710][105692] Updated weights for policy 0, policy_version 550805 (0.0011) [2023-12-26 19:22:38,762][105692] Updated weights for policy 0, policy_version 550815 (0.0010) [2023-12-26 19:22:38,834][105620] Updated weights for policy 1, policy_version 551601 (0.0010) [2023-12-26 19:22:38,893][105620] Updated weights for policy 1, policy_version 551611 (0.0011) [2023-12-26 19:22:38,951][105620] Updated weights for policy 1, policy_version 551621 (0.0010) [2023-12-26 19:22:39,505][105692] Updated weights for policy 0, policy_version 550825 (0.0011) [2023-12-26 19:22:39,572][105692] Updated weights for policy 0, policy_version 550835 (0.0011) [2023-12-26 19:22:39,631][105692] Updated weights for policy 0, policy_version 550845 (0.0010) [2023-12-26 19:22:39,644][105620] Updated weights for policy 1, policy_version 551631 (0.0007) [2023-12-26 19:22:39,698][105620] Updated weights for policy 1, policy_version 551641 (0.0006) [2023-12-26 19:22:39,760][105620] Updated weights for policy 1, policy_version 551651 (0.0005) [2023-12-26 19:22:40,362][105692] Updated weights for policy 0, policy_version 550855 (0.0009) [2023-12-26 19:22:40,419][105692] Updated weights for policy 0, policy_version 550865 (0.0007) [2023-12-26 19:22:40,426][105620] Updated weights for policy 1, policy_version 551661 (0.0008) [2023-12-26 19:22:40,490][105692] Updated weights for policy 0, policy_version 550875 (0.0007) [2023-12-26 19:22:40,495][105620] Updated weights for policy 1, policy_version 551671 (0.0008) [2023-12-26 19:22:40,558][105620] Updated weights for policy 1, policy_version 551681 (0.0007) [2023-12-26 19:22:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19797.4, 300 sec: 19522.0). Total num frames: 282288128. Throughput: 0: 9766.5, 1: 9929.3. Samples: 282299452. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-26 19:22:41,062][104569] Avg episode reward: [(0, '8176.197'), (1, '9352.734')] [2023-12-26 19:22:41,153][105692] Updated weights for policy 0, policy_version 550885 (0.0007) [2023-12-26 19:22:41,198][105620] Updated weights for policy 1, policy_version 551691 (0.0007) [2023-12-26 19:22:41,226][105692] Updated weights for policy 0, policy_version 550895 (0.0008) [2023-12-26 19:22:41,262][105620] Updated weights for policy 1, policy_version 551701 (0.0006) [2023-12-26 19:22:41,294][105692] Updated weights for policy 0, policy_version 550905 (0.0010) [2023-12-26 19:22:41,317][105620] Updated weights for policy 1, policy_version 551711 (0.0007) [2023-12-26 19:22:42,026][105620] Updated weights for policy 1, policy_version 551721 (0.0008) [2023-12-26 19:22:42,071][105692] Updated weights for policy 0, policy_version 550915 (0.0009) [2023-12-26 19:22:42,088][105620] Updated weights for policy 1, policy_version 551731 (0.0009) [2023-12-26 19:22:42,136][105692] Updated weights for policy 0, policy_version 550925 (0.0007) [2023-12-26 19:22:42,148][105620] Updated weights for policy 1, policy_version 551741 (0.0008) [2023-12-26 19:22:42,198][105692] Updated weights for policy 0, policy_version 550935 (0.0008) [2023-12-26 19:22:42,213][105620] Updated weights for policy 1, policy_version 551751 (0.0006) [2023-12-26 19:22:42,941][105620] Updated weights for policy 1, policy_version 551761 (0.0008) [2023-12-26 19:22:42,992][105620] Updated weights for policy 1, policy_version 551771 (0.0009) [2023-12-26 19:22:43,010][105692] Updated weights for policy 0, policy_version 550945 (0.0008) [2023-12-26 19:22:43,049][105620] Updated weights for policy 1, policy_version 551781 (0.0007) [2023-12-26 19:22:43,067][105692] Updated weights for policy 0, policy_version 550955 (0.0008) [2023-12-26 19:22:43,124][105692] Updated weights for policy 0, policy_version 550965 (0.0009) [2023-12-26 19:22:43,180][105692] Updated weights for policy 0, policy_version 550976 (0.0009) [2023-12-26 19:22:43,812][105692] Updated weights for policy 0, policy_version 550986 (0.0006) [2023-12-26 19:22:43,824][105620] Updated weights for policy 1, policy_version 551791 (0.0008) [2023-12-26 19:22:43,876][105692] Updated weights for policy 0, policy_version 550996 (0.0006) [2023-12-26 19:22:43,877][105620] Updated weights for policy 1, policy_version 551801 (0.0005) [2023-12-26 19:22:43,934][105620] Updated weights for policy 1, policy_version 551811 (0.0006) [2023-12-26 19:22:43,940][105692] Updated weights for policy 0, policy_version 551006 (0.0006) [2023-12-26 19:22:44,610][105692] Updated weights for policy 0, policy_version 551016 (0.0010) [2023-12-26 19:22:44,655][105692] Updated weights for policy 0, policy_version 551026 (0.0010) [2023-12-26 19:22:44,690][105620] Updated weights for policy 1, policy_version 551821 (0.0009) [2023-12-26 19:22:44,702][105692] Updated weights for policy 0, policy_version 551036 (0.0006) [2023-12-26 19:22:44,753][105620] Updated weights for policy 1, policy_version 551831 (0.0009) [2023-12-26 19:22:44,821][105620] Updated weights for policy 1, policy_version 551841 (0.0008) [2023-12-26 19:22:45,418][105692] Updated weights for policy 0, policy_version 551046 (0.0008) [2023-12-26 19:22:45,486][105692] Updated weights for policy 0, policy_version 551056 (0.0009) [2023-12-26 19:22:45,548][105692] Updated weights for policy 0, policy_version 551066 (0.0007) [2023-12-26 19:22:45,616][105620] Updated weights for policy 1, policy_version 551851 (0.0009) [2023-12-26 19:22:45,680][105620] Updated weights for policy 1, policy_version 551861 (0.0005) [2023-12-26 19:22:45,747][105620] Updated weights for policy 1, policy_version 551871 (0.0006) [2023-12-26 19:22:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19797.3, 300 sec: 19521.9). Total num frames: 282386432. Throughput: 0: 9661.4, 1: 9943.8. Samples: 282356640. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-26 19:22:46,063][104569] Avg episode reward: [(0, '9088.041'), (1, '9352.433')] [2023-12-26 19:22:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000551880_141295616.pth... [2023-12-26 19:22:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000551072_141090816.pth... [2023-12-26 19:22:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000550728_141000704.pth [2023-12-26 19:22:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000549952_140804096.pth [2023-12-26 19:22:46,148][105692] Updated weights for policy 0, policy_version 551076 (0.0008) [2023-12-26 19:22:46,206][105692] Updated weights for policy 0, policy_version 551086 (0.0010) [2023-12-26 19:22:46,247][105620] Updated weights for policy 1, policy_version 551881 (0.0005) [2023-12-26 19:22:46,268][105692] Updated weights for policy 0, policy_version 551096 (0.0008) [2023-12-26 19:22:46,301][105620] Updated weights for policy 1, policy_version 551891 (0.0007) [2023-12-26 19:22:46,366][105620] Updated weights for policy 1, policy_version 551901 (0.0008) [2023-12-26 19:22:46,431][105620] Updated weights for policy 1, policy_version 551911 (0.0006) [2023-12-26 19:22:46,960][105692] Updated weights for policy 0, policy_version 551106 (0.0007) [2023-12-26 19:22:47,018][105692] Updated weights for policy 0, policy_version 551116 (0.0006) [2023-12-26 19:22:47,080][105692] Updated weights for policy 0, policy_version 551126 (0.0006) [2023-12-26 19:22:47,115][105620] Updated weights for policy 1, policy_version 551921 (0.0007) [2023-12-26 19:22:47,139][105692] Updated weights for policy 0, policy_version 551136 (0.0009) [2023-12-26 19:22:47,176][105620] Updated weights for policy 1, policy_version 551931 (0.0008) [2023-12-26 19:22:47,228][105620] Updated weights for policy 1, policy_version 551941 (0.0009) [2023-12-26 19:22:47,751][105692] Updated weights for policy 0, policy_version 551146 (0.0008) [2023-12-26 19:22:47,813][105692] Updated weights for policy 0, policy_version 551156 (0.0008) [2023-12-26 19:22:47,874][105692] Updated weights for policy 0, policy_version 551166 (0.0009) [2023-12-26 19:22:47,967][105620] Updated weights for policy 1, policy_version 551951 (0.0010) [2023-12-26 19:22:48,020][105620] Updated weights for policy 1, policy_version 551962 (0.0010) [2023-12-26 19:22:48,074][105620] Updated weights for policy 1, policy_version 551973 (0.0010) [2023-12-26 19:22:48,597][105692] Updated weights for policy 0, policy_version 551176 (0.0009) [2023-12-26 19:22:48,660][105692] Updated weights for policy 0, policy_version 551186 (0.0009) [2023-12-26 19:22:48,714][105692] Updated weights for policy 0, policy_version 551196 (0.0009) [2023-12-26 19:22:48,753][105620] Updated weights for policy 1, policy_version 551983 (0.0008) [2023-12-26 19:22:48,807][105620] Updated weights for policy 1, policy_version 551993 (0.0009) [2023-12-26 19:22:48,871][105620] Updated weights for policy 1, policy_version 552003 (0.0009) [2023-12-26 19:22:49,354][105692] Updated weights for policy 0, policy_version 551206 (0.0007) [2023-12-26 19:22:49,417][105585] KL-divergence is very high: 222.0128 [2023-12-26 19:22:49,423][105692] Updated weights for policy 0, policy_version 551216 (0.0006) [2023-12-26 19:22:49,471][105585] KL-divergence is very high: 325.6681 [2023-12-26 19:22:49,489][105692] Updated weights for policy 0, policy_version 551226 (0.0007) [2023-12-26 19:22:49,517][105585] KL-divergence is very high: 290.7172 [2023-12-26 19:22:49,625][105620] Updated weights for policy 1, policy_version 552013 (0.0008) [2023-12-26 19:22:49,682][105620] Updated weights for policy 1, policy_version 552023 (0.0005) [2023-12-26 19:22:49,742][105620] Updated weights for policy 1, policy_version 552033 (0.0009) [2023-12-26 19:22:50,213][105585] KL-divergence is very high: 102.7407 [2023-12-26 19:22:50,230][105692] Updated weights for policy 0, policy_version 551236 (0.0008) [2023-12-26 19:22:50,276][105692] Updated weights for policy 0, policy_version 551246 (0.0005) [2023-12-26 19:22:50,322][105692] Updated weights for policy 0, policy_version 551256 (0.0005) [2023-12-26 19:22:50,425][105620] Updated weights for policy 1, policy_version 552043 (0.0010) [2023-12-26 19:22:50,474][105620] Updated weights for policy 1, policy_version 552053 (0.0010) [2023-12-26 19:22:50,518][105620] Updated weights for policy 1, policy_version 552063 (0.0010) [2023-12-26 19:22:51,014][105692] Updated weights for policy 0, policy_version 551266 (0.0006) [2023-12-26 19:22:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19521.9). Total num frames: 282484736. Throughput: 0: 9805.8, 1: 9908.7. Samples: 282477996. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-26 19:22:51,062][104569] Avg episode reward: [(0, '9088.161'), (1, '9352.237')] [2023-12-26 19:22:51,075][105692] Updated weights for policy 0, policy_version 551276 (0.0011) [2023-12-26 19:22:51,134][105692] Updated weights for policy 0, policy_version 551286 (0.0011) [2023-12-26 19:22:51,179][105692] Updated weights for policy 0, policy_version 551296 (0.0010) [2023-12-26 19:22:51,300][105620] Updated weights for policy 1, policy_version 552073 (0.0010) [2023-12-26 19:22:51,362][105620] Updated weights for policy 1, policy_version 552083 (0.0009) [2023-12-26 19:22:51,430][105620] Updated weights for policy 1, policy_version 552093 (0.0008) [2023-12-26 19:22:51,486][105620] Updated weights for policy 1, policy_version 552103 (0.0008) [2023-12-26 19:22:51,992][105692] Updated weights for policy 0, policy_version 551306 (0.0011) [2023-12-26 19:22:52,062][105692] Updated weights for policy 0, policy_version 551316 (0.0011) [2023-12-26 19:22:52,126][105692] Updated weights for policy 0, policy_version 551326 (0.0011) [2023-12-26 19:22:52,152][105620] Updated weights for policy 1, policy_version 552113 (0.0007) [2023-12-26 19:22:52,203][105620] Updated weights for policy 1, policy_version 552123 (0.0008) [2023-12-26 19:22:52,266][105620] Updated weights for policy 1, policy_version 552133 (0.0008) [2023-12-26 19:22:52,832][105692] Updated weights for policy 0, policy_version 551336 (0.0007) [2023-12-26 19:22:52,900][105692] Updated weights for policy 0, policy_version 551346 (0.0006) [2023-12-26 19:22:52,968][105692] Updated weights for policy 0, policy_version 551356 (0.0006) [2023-12-26 19:22:53,100][105620] Updated weights for policy 1, policy_version 552143 (0.0006) [2023-12-26 19:22:53,156][105620] Updated weights for policy 1, policy_version 552153 (0.0008) [2023-12-26 19:22:53,211][105620] Updated weights for policy 1, policy_version 552163 (0.0008) [2023-12-26 19:22:53,606][105692] Updated weights for policy 0, policy_version 551366 (0.0009) [2023-12-26 19:22:53,661][105692] Updated weights for policy 0, policy_version 551376 (0.0009) [2023-12-26 19:22:53,718][105692] Updated weights for policy 0, policy_version 551386 (0.0005) [2023-12-26 19:22:53,999][105620] Updated weights for policy 1, policy_version 552173 (0.0009) [2023-12-26 19:22:54,075][105620] Updated weights for policy 1, policy_version 552183 (0.0009) [2023-12-26 19:22:54,132][105620] Updated weights for policy 1, policy_version 552193 (0.0008) [2023-12-26 19:22:54,281][105692] Updated weights for policy 0, policy_version 551396 (0.0007) [2023-12-26 19:22:54,349][105692] Updated weights for policy 0, policy_version 551406 (0.0007) [2023-12-26 19:22:54,410][105692] Updated weights for policy 0, policy_version 551416 (0.0006) [2023-12-26 19:22:54,905][105620] Updated weights for policy 1, policy_version 552203 (0.0010) [2023-12-26 19:22:54,962][105620] Updated weights for policy 1, policy_version 552213 (0.0008) [2023-12-26 19:22:55,020][105620] Updated weights for policy 1, policy_version 552223 (0.0009) [2023-12-26 19:22:55,108][105692] Updated weights for policy 0, policy_version 551426 (0.0008) [2023-12-26 19:22:55,161][105692] Updated weights for policy 0, policy_version 551436 (0.0008) [2023-12-26 19:22:55,219][105692] Updated weights for policy 0, policy_version 551446 (0.0009) [2023-12-26 19:22:55,277][105692] Updated weights for policy 0, policy_version 551456 (0.0009) [2023-12-26 19:22:55,817][105620] Updated weights for policy 1, policy_version 552233 (0.0009) [2023-12-26 19:22:55,872][105620] Updated weights for policy 1, policy_version 552243 (0.0009) [2023-12-26 19:22:55,902][105692] Updated weights for policy 0, policy_version 551466 (0.0007) [2023-12-26 19:22:55,918][105620] Updated weights for policy 1, policy_version 552253 (0.0006) [2023-12-26 19:22:55,961][105692] Updated weights for policy 0, policy_version 551476 (0.0007) [2023-12-26 19:22:55,968][105620] Updated weights for policy 1, policy_version 552263 (0.0006) [2023-12-26 19:22:56,018][105692] Updated weights for policy 0, policy_version 551486 (0.0008) [2023-12-26 19:22:56,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 282591232. Throughput: 0: 9854.5, 1: 9740.2. Samples: 282592356. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-26 19:22:56,063][104569] Avg episode reward: [(0, '8997.581'), (1, '9352.140')] [2023-12-26 19:22:56,665][105620] Updated weights for policy 1, policy_version 552273 (0.0006) [2023-12-26 19:22:56,712][105620] Updated weights for policy 1, policy_version 552283 (0.0006) [2023-12-26 19:22:56,757][105620] Updated weights for policy 1, policy_version 552293 (0.0007) [2023-12-26 19:22:56,762][105692] Updated weights for policy 0, policy_version 551496 (0.0006) [2023-12-26 19:22:56,810][105692] Updated weights for policy 0, policy_version 551506 (0.0005) [2023-12-26 19:22:56,865][105692] Updated weights for policy 0, policy_version 551516 (0.0006) [2023-12-26 19:22:57,401][105620] Updated weights for policy 1, policy_version 552303 (0.0009) [2023-12-26 19:22:57,448][105620] Updated weights for policy 1, policy_version 552313 (0.0010) [2023-12-26 19:22:57,463][105692] Updated weights for policy 0, policy_version 551526 (0.0005) [2023-12-26 19:22:57,496][105620] Updated weights for policy 1, policy_version 552323 (0.0010) [2023-12-26 19:22:57,510][105692] Updated weights for policy 0, policy_version 551536 (0.0005) [2023-12-26 19:22:57,568][105692] Updated weights for policy 0, policy_version 551546 (0.0007) [2023-12-26 19:22:58,233][105620] Updated weights for policy 1, policy_version 552333 (0.0010) [2023-12-26 19:22:58,283][105620] Updated weights for policy 1, policy_version 552343 (0.0008) [2023-12-26 19:22:58,343][105692] Updated weights for policy 0, policy_version 551556 (0.0008) [2023-12-26 19:22:58,354][105620] Updated weights for policy 1, policy_version 552354 (0.0009) [2023-12-26 19:22:58,405][105692] Updated weights for policy 0, policy_version 551566 (0.0008) [2023-12-26 19:22:58,463][105692] Updated weights for policy 0, policy_version 551576 (0.0009) [2023-12-26 19:22:59,145][105620] Updated weights for policy 1, policy_version 552364 (0.0008) [2023-12-26 19:22:59,216][105620] Updated weights for policy 1, policy_version 552374 (0.0009) [2023-12-26 19:22:59,283][105620] Updated weights for policy 1, policy_version 552384 (0.0008) [2023-12-26 19:22:59,321][105692] Updated weights for policy 0, policy_version 551586 (0.0008) [2023-12-26 19:22:59,390][105692] Updated weights for policy 0, policy_version 551596 (0.0009) [2023-12-26 19:22:59,460][105692] Updated weights for policy 0, policy_version 551606 (0.0009) [2023-12-26 19:22:59,531][105692] Updated weights for policy 0, policy_version 551616 (0.0009) [2023-12-26 19:23:00,009][105620] Updated weights for policy 1, policy_version 552394 (0.0008) [2023-12-26 19:23:00,069][105620] Updated weights for policy 1, policy_version 552404 (0.0010) [2023-12-26 19:23:00,127][105620] Updated weights for policy 1, policy_version 552414 (0.0009) [2023-12-26 19:23:00,190][105620] Updated weights for policy 1, policy_version 552424 (0.0007) [2023-12-26 19:23:00,222][105692] Updated weights for policy 0, policy_version 551626 (0.0006) [2023-12-26 19:23:00,296][105692] Updated weights for policy 0, policy_version 551636 (0.0010) [2023-12-26 19:23:00,365][105692] Updated weights for policy 0, policy_version 551646 (0.0011) [2023-12-26 19:23:00,823][105620] Updated weights for policy 1, policy_version 552434 (0.0005) [2023-12-26 19:23:00,889][105620] Updated weights for policy 1, policy_version 552444 (0.0010) [2023-12-26 19:23:00,949][105620] Updated weights for policy 1, policy_version 552454 (0.0008) [2023-12-26 19:23:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 282681344. Throughput: 0: 9867.4, 1: 9758.0. Samples: 282652604. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-26 19:23:01,062][104569] Avg episode reward: [(0, '8998.400'), (1, '9260.138')] [2023-12-26 19:23:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000552456_141443072.pth... [2023-12-26 19:23:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000551336_141156352.pth [2023-12-26 19:23:01,107][105692] Updated weights for policy 0, policy_version 551656 (0.0008) [2023-12-26 19:23:01,175][105692] Updated weights for policy 0, policy_version 551666 (0.0009) [2023-12-26 19:23:01,228][105692] Updated weights for policy 0, policy_version 551676 (0.0007) [2023-12-26 19:23:01,251][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000551680_141246464.pth... [2023-12-26 19:23:01,256][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000550528_140951552.pth [2023-12-26 19:23:01,606][105620] Updated weights for policy 1, policy_version 552464 (0.0009) [2023-12-26 19:23:01,669][105620] Updated weights for policy 1, policy_version 552474 (0.0011) [2023-12-26 19:23:01,730][105620] Updated weights for policy 1, policy_version 552484 (0.0010) [2023-12-26 19:23:01,963][105692] Updated weights for policy 0, policy_version 551686 (0.0008) [2023-12-26 19:23:02,029][105692] Updated weights for policy 0, policy_version 551696 (0.0008) [2023-12-26 19:23:02,092][105692] Updated weights for policy 0, policy_version 551706 (0.0006) [2023-12-26 19:23:02,483][105620] Updated weights for policy 1, policy_version 552494 (0.0009) [2023-12-26 19:23:02,536][105620] Updated weights for policy 1, policy_version 552504 (0.0009) [2023-12-26 19:23:02,598][105620] Updated weights for policy 1, policy_version 552514 (0.0009) [2023-12-26 19:23:02,782][105692] Updated weights for policy 0, policy_version 551716 (0.0009) [2023-12-26 19:23:02,848][105692] Updated weights for policy 0, policy_version 551726 (0.0006) [2023-12-26 19:23:02,913][105692] Updated weights for policy 0, policy_version 551736 (0.0008) [2023-12-26 19:23:03,253][105620] Updated weights for policy 1, policy_version 552524 (0.0007) [2023-12-26 19:23:03,307][105620] Updated weights for policy 1, policy_version 552534 (0.0005) [2023-12-26 19:23:03,370][105620] Updated weights for policy 1, policy_version 552544 (0.0005) [2023-12-26 19:23:03,737][105692] Updated weights for policy 0, policy_version 551746 (0.0009) [2023-12-26 19:23:03,798][105692] Updated weights for policy 0, policy_version 551756 (0.0010) [2023-12-26 19:23:03,856][105692] Updated weights for policy 0, policy_version 551766 (0.0010) [2023-12-26 19:23:03,882][105620] Updated weights for policy 1, policy_version 552554 (0.0006) [2023-12-26 19:23:03,915][105692] Updated weights for policy 0, policy_version 551776 (0.0010) [2023-12-26 19:23:03,933][105620] Updated weights for policy 1, policy_version 552564 (0.0007) [2023-12-26 19:23:03,979][105620] Updated weights for policy 1, policy_version 552574 (0.0006) [2023-12-26 19:23:04,027][105620] Updated weights for policy 1, policy_version 552584 (0.0006) [2023-12-26 19:23:04,564][105692] Updated weights for policy 0, policy_version 551786 (0.0005) [2023-12-26 19:23:04,618][105692] Updated weights for policy 0, policy_version 551796 (0.0007) [2023-12-26 19:23:04,668][105692] Updated weights for policy 0, policy_version 551806 (0.0007) [2023-12-26 19:23:04,717][105620] Updated weights for policy 1, policy_version 552594 (0.0009) [2023-12-26 19:23:04,765][105620] Updated weights for policy 1, policy_version 552604 (0.0008) [2023-12-26 19:23:04,827][105620] Updated weights for policy 1, policy_version 552614 (0.0008) [2023-12-26 19:23:05,254][105692] Updated weights for policy 0, policy_version 551816 (0.0009) [2023-12-26 19:23:05,302][105692] Updated weights for policy 0, policy_version 551826 (0.0010) [2023-12-26 19:23:05,357][105692] Updated weights for policy 0, policy_version 551836 (0.0010) [2023-12-26 19:23:05,676][105620] Updated weights for policy 1, policy_version 552624 (0.0008) [2023-12-26 19:23:05,726][105620] Updated weights for policy 1, policy_version 552634 (0.0009) [2023-12-26 19:23:05,788][105620] Updated weights for policy 1, policy_version 552644 (0.0008) [2023-12-26 19:23:06,001][105692] Updated weights for policy 0, policy_version 551846 (0.0007) [2023-12-26 19:23:06,049][105692] Updated weights for policy 0, policy_version 551856 (0.0005) [2023-12-26 19:23:06,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 282779648. Throughput: 0: 9733.7, 1: 9841.8. Samples: 282769636. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-26 19:23:06,063][104569] Avg episode reward: [(0, '9082.676'), (1, '9166.862')] [2023-12-26 19:23:06,120][105692] Updated weights for policy 0, policy_version 551866 (0.0006) [2023-12-26 19:23:06,469][105620] Updated weights for policy 1, policy_version 552654 (0.0005) [2023-12-26 19:23:06,528][105620] Updated weights for policy 1, policy_version 552664 (0.0008) [2023-12-26 19:23:06,574][105586] KL-divergence is very high: 103.3842 [2023-12-26 19:23:06,587][105620] Updated weights for policy 1, policy_version 552674 (0.0008) [2023-12-26 19:23:06,819][105692] Updated weights for policy 0, policy_version 551876 (0.0007) [2023-12-26 19:23:06,872][105692] Updated weights for policy 0, policy_version 551887 (0.0010) [2023-12-26 19:23:06,920][105692] Updated weights for policy 0, policy_version 551898 (0.0010) [2023-12-26 19:23:07,198][105620] Updated weights for policy 1, policy_version 552684 (0.0008) [2023-12-26 19:23:07,266][105620] Updated weights for policy 1, policy_version 552694 (0.0010) [2023-12-26 19:23:07,328][105620] Updated weights for policy 1, policy_version 552704 (0.0011) [2023-12-26 19:23:07,719][105692] Updated weights for policy 0, policy_version 551908 (0.0010) [2023-12-26 19:23:07,764][105692] Updated weights for policy 0, policy_version 551918 (0.0010) [2023-12-26 19:23:07,812][105692] Updated weights for policy 0, policy_version 551928 (0.0010) [2023-12-26 19:23:08,046][105620] Updated weights for policy 1, policy_version 552714 (0.0010) [2023-12-26 19:23:08,107][105620] Updated weights for policy 1, policy_version 552724 (0.0010) [2023-12-26 19:23:08,159][105620] Updated weights for policy 1, policy_version 552734 (0.0010) [2023-12-26 19:23:08,207][105620] Updated weights for policy 1, policy_version 552744 (0.0010) [2023-12-26 19:23:08,592][105692] Updated weights for policy 0, policy_version 551938 (0.0010) [2023-12-26 19:23:08,656][105692] Updated weights for policy 0, policy_version 551948 (0.0008) [2023-12-26 19:23:08,719][105692] Updated weights for policy 0, policy_version 551958 (0.0009) [2023-12-26 19:23:08,773][105692] Updated weights for policy 0, policy_version 551968 (0.0008) [2023-12-26 19:23:08,944][105620] Updated weights for policy 1, policy_version 552754 (0.0009) [2023-12-26 19:23:09,006][105620] Updated weights for policy 1, policy_version 552764 (0.0009) [2023-12-26 19:23:09,060][105620] Updated weights for policy 1, policy_version 552774 (0.0010) [2023-12-26 19:23:09,555][105692] Updated weights for policy 0, policy_version 551978 (0.0009) [2023-12-26 19:23:09,616][105692] Updated weights for policy 0, policy_version 551988 (0.0008) [2023-12-26 19:23:09,672][105692] Updated weights for policy 0, policy_version 551998 (0.0008) [2023-12-26 19:23:09,808][105620] Updated weights for policy 1, policy_version 552784 (0.0010) [2023-12-26 19:23:09,884][105620] Updated weights for policy 1, policy_version 552794 (0.0011) [2023-12-26 19:23:09,952][105620] Updated weights for policy 1, policy_version 552804 (0.0010) [2023-12-26 19:23:10,467][105692] Updated weights for policy 0, policy_version 552008 (0.0007) [2023-12-26 19:23:10,533][105692] Updated weights for policy 0, policy_version 552018 (0.0007) [2023-12-26 19:23:10,594][105692] Updated weights for policy 0, policy_version 552028 (0.0012) [2023-12-26 19:23:10,627][105620] Updated weights for policy 1, policy_version 552814 (0.0006) [2023-12-26 19:23:10,689][105620] Updated weights for policy 1, policy_version 552824 (0.0010) [2023-12-26 19:23:10,754][105620] Updated weights for policy 1, policy_version 552834 (0.0010) [2023-12-26 19:23:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 282877952. Throughput: 0: 9757.1, 1: 9794.6. Samples: 282885112. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-26 19:23:11,063][104569] Avg episode reward: [(0, '8988.679'), (1, '9167.095')] [2023-12-26 19:23:11,386][105692] Updated weights for policy 0, policy_version 552038 (0.0010) [2023-12-26 19:23:11,438][105692] Updated weights for policy 0, policy_version 552048 (0.0008) [2023-12-26 19:23:11,462][105620] Updated weights for policy 1, policy_version 552844 (0.0010) [2023-12-26 19:23:11,494][105692] Updated weights for policy 0, policy_version 552058 (0.0009) [2023-12-26 19:23:11,519][105620] Updated weights for policy 1, policy_version 552854 (0.0010) [2023-12-26 19:23:11,577][105620] Updated weights for policy 1, policy_version 552864 (0.0009) [2023-12-26 19:23:12,331][105692] Updated weights for policy 0, policy_version 552068 (0.0009) [2023-12-26 19:23:12,363][105620] Updated weights for policy 1, policy_version 552874 (0.0010) [2023-12-26 19:23:12,387][105692] Updated weights for policy 0, policy_version 552078 (0.0008) [2023-12-26 19:23:12,421][105620] Updated weights for policy 1, policy_version 552884 (0.0010) [2023-12-26 19:23:12,444][105692] Updated weights for policy 0, policy_version 552088 (0.0007) [2023-12-26 19:23:12,471][105620] Updated weights for policy 1, policy_version 552894 (0.0011) [2023-12-26 19:23:12,524][105620] Updated weights for policy 1, policy_version 552904 (0.0008) [2023-12-26 19:23:13,112][105620] Updated weights for policy 1, policy_version 552914 (0.0006) [2023-12-26 19:23:13,173][105620] Updated weights for policy 1, policy_version 552924 (0.0008) [2023-12-26 19:23:13,233][105620] Updated weights for policy 1, policy_version 552934 (0.0009) [2023-12-26 19:23:13,345][105692] Updated weights for policy 0, policy_version 552098 (0.0007) [2023-12-26 19:23:13,411][105692] Updated weights for policy 0, policy_version 552108 (0.0010) [2023-12-26 19:23:13,468][105692] Updated weights for policy 0, policy_version 552118 (0.0008) [2023-12-26 19:23:13,529][105692] Updated weights for policy 0, policy_version 552128 (0.0008) [2023-12-26 19:23:13,938][105620] Updated weights for policy 1, policy_version 552944 (0.0011) [2023-12-26 19:23:13,990][105620] Updated weights for policy 1, policy_version 552954 (0.0010) [2023-12-26 19:23:14,045][105620] Updated weights for policy 1, policy_version 552964 (0.0010) [2023-12-26 19:23:14,168][105692] Updated weights for policy 0, policy_version 552138 (0.0008) [2023-12-26 19:23:14,232][105692] Updated weights for policy 0, policy_version 552148 (0.0008) [2023-12-26 19:23:14,292][105692] Updated weights for policy 0, policy_version 552158 (0.0008) [2023-12-26 19:23:14,775][105620] Updated weights for policy 1, policy_version 552974 (0.0008) [2023-12-26 19:23:14,831][105620] Updated weights for policy 1, policy_version 552984 (0.0008) [2023-12-26 19:23:14,896][105620] Updated weights for policy 1, policy_version 552994 (0.0008) [2023-12-26 19:23:15,094][105692] Updated weights for policy 0, policy_version 552168 (0.0007) [2023-12-26 19:23:15,145][105692] Updated weights for policy 0, policy_version 552178 (0.0009) [2023-12-26 19:23:15,198][105692] Updated weights for policy 0, policy_version 552188 (0.0008) [2023-12-26 19:23:15,577][105620] Updated weights for policy 1, policy_version 553004 (0.0007) [2023-12-26 19:23:15,628][105620] Updated weights for policy 1, policy_version 553014 (0.0005) [2023-12-26 19:23:15,683][105620] Updated weights for policy 1, policy_version 553024 (0.0005) [2023-12-26 19:23:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 282968064. Throughput: 0: 9686.6, 1: 9779.5. Samples: 282940704. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-26 19:23:16,062][104569] Avg episode reward: [(0, '8987.808'), (1, '9260.526')] [2023-12-26 19:23:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000553032_141590528.pth... [2023-12-26 19:23:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000551880_141295616.pth [2023-12-26 19:23:16,091][105692] Updated weights for policy 0, policy_version 552198 (0.0010) [2023-12-26 19:23:16,149][105692] Updated weights for policy 0, policy_version 552208 (0.0010) [2023-12-26 19:23:16,197][105620] Updated weights for policy 1, policy_version 553034 (0.0005) [2023-12-26 19:23:16,215][105692] Updated weights for policy 0, policy_version 552218 (0.0009) [2023-12-26 19:23:16,249][105620] Updated weights for policy 1, policy_version 553044 (0.0009) [2023-12-26 19:23:16,251][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000552224_141385728.pth... [2023-12-26 19:23:16,254][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000551072_141090816.pth [2023-12-26 19:23:16,305][105620] Updated weights for policy 1, policy_version 553054 (0.0010) [2023-12-26 19:23:16,357][105620] Updated weights for policy 1, policy_version 553064 (0.0010) [2023-12-26 19:23:16,884][105692] Updated weights for policy 0, policy_version 552228 (0.0007) [2023-12-26 19:23:16,947][105692] Updated weights for policy 0, policy_version 552238 (0.0008) [2023-12-26 19:23:16,965][105620] Updated weights for policy 1, policy_version 553074 (0.0006) [2023-12-26 19:23:17,010][105692] Updated weights for policy 0, policy_version 552248 (0.0005) [2023-12-26 19:23:17,020][105620] Updated weights for policy 1, policy_version 553084 (0.0006) [2023-12-26 19:23:17,080][105620] Updated weights for policy 1, policy_version 553095 (0.0009) [2023-12-26 19:23:17,626][105692] Updated weights for policy 0, policy_version 552258 (0.0006) [2023-12-26 19:23:17,659][105620] Updated weights for policy 1, policy_version 553105 (0.0008) [2023-12-26 19:23:17,677][105692] Updated weights for policy 0, policy_version 552268 (0.0005) [2023-12-26 19:23:17,707][105620] Updated weights for policy 1, policy_version 553115 (0.0008) [2023-12-26 19:23:17,728][105692] Updated weights for policy 0, policy_version 552278 (0.0006) [2023-12-26 19:23:17,760][105620] Updated weights for policy 1, policy_version 553125 (0.0009) [2023-12-26 19:23:17,785][105692] Updated weights for policy 0, policy_version 552288 (0.0005) [2023-12-26 19:23:18,402][105692] Updated weights for policy 0, policy_version 552298 (0.0006) [2023-12-26 19:23:18,455][105692] Updated weights for policy 0, policy_version 552308 (0.0009) [2023-12-26 19:23:18,514][105692] Updated weights for policy 0, policy_version 552318 (0.0009) [2023-12-26 19:23:18,594][105620] Updated weights for policy 1, policy_version 553135 (0.0009) [2023-12-26 19:23:18,656][105620] Updated weights for policy 1, policy_version 553145 (0.0009) [2023-12-26 19:23:18,710][105620] Updated weights for policy 1, policy_version 553155 (0.0009) [2023-12-26 19:23:19,242][105692] Updated weights for policy 0, policy_version 552328 (0.0008) [2023-12-26 19:23:19,308][105692] Updated weights for policy 0, policy_version 552338 (0.0007) [2023-12-26 19:23:19,389][105692] Updated weights for policy 0, policy_version 552348 (0.0008) [2023-12-26 19:23:19,469][105620] Updated weights for policy 1, policy_version 553165 (0.0009) [2023-12-26 19:23:19,535][105620] Updated weights for policy 1, policy_version 553175 (0.0009) [2023-12-26 19:23:19,599][105620] Updated weights for policy 1, policy_version 553185 (0.0007) [2023-12-26 19:23:20,049][105692] Updated weights for policy 0, policy_version 552358 (0.0008) [2023-12-26 19:23:20,118][105692] Updated weights for policy 0, policy_version 552368 (0.0008) [2023-12-26 19:23:20,178][105692] Updated weights for policy 0, policy_version 552378 (0.0005) [2023-12-26 19:23:20,301][105620] Updated weights for policy 1, policy_version 553195 (0.0010) [2023-12-26 19:23:20,365][105620] Updated weights for policy 1, policy_version 553205 (0.0011) [2023-12-26 19:23:20,422][105620] Updated weights for policy 1, policy_version 553215 (0.0011) [2023-12-26 19:23:20,770][105692] Updated weights for policy 0, policy_version 552388 (0.0007) [2023-12-26 19:23:20,819][105692] Updated weights for policy 0, policy_version 552398 (0.0008) [2023-12-26 19:23:20,883][105692] Updated weights for policy 0, policy_version 552408 (0.0009) [2023-12-26 19:23:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 283074560. Throughput: 0: 9671.9, 1: 9815.4. Samples: 283061800. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-26 19:23:21,063][104569] Avg episode reward: [(0, '8897.357'), (1, '9352.629')] [2023-12-26 19:23:21,115][105620] Updated weights for policy 1, policy_version 553225 (0.0010) [2023-12-26 19:23:21,174][105620] Updated weights for policy 1, policy_version 553235 (0.0008) [2023-12-26 19:23:21,233][105620] Updated weights for policy 1, policy_version 553245 (0.0008) [2023-12-26 19:23:21,292][105620] Updated weights for policy 1, policy_version 553255 (0.0008) [2023-12-26 19:23:21,622][105692] Updated weights for policy 0, policy_version 552418 (0.0009) [2023-12-26 19:23:21,682][105692] Updated weights for policy 0, policy_version 552428 (0.0009) [2023-12-26 19:23:21,700][105585] KL-divergence is very high: 131.3780 [2023-12-26 19:23:21,748][105692] Updated weights for policy 0, policy_version 552438 (0.0008) [2023-12-26 19:23:21,754][105585] KL-divergence is very high: 244.5100 [2023-12-26 19:23:21,802][105585] KL-divergence is very high: 226.4119 [2023-12-26 19:23:21,808][105692] Updated weights for policy 0, policy_version 552448 (0.0009) [2023-12-26 19:23:22,084][105620] Updated weights for policy 1, policy_version 553265 (0.0010) [2023-12-26 19:23:22,149][105620] Updated weights for policy 1, policy_version 553275 (0.0010) [2023-12-26 19:23:22,211][105620] Updated weights for policy 1, policy_version 553285 (0.0009) [2023-12-26 19:23:22,589][105692] Updated weights for policy 0, policy_version 552458 (0.0009) [2023-12-26 19:23:22,645][105692] Updated weights for policy 0, policy_version 552468 (0.0010) [2023-12-26 19:23:22,706][105692] Updated weights for policy 0, policy_version 552478 (0.0005) [2023-12-26 19:23:22,971][105620] Updated weights for policy 1, policy_version 553295 (0.0008) [2023-12-26 19:23:23,034][105620] Updated weights for policy 1, policy_version 553305 (0.0009) [2023-12-26 19:23:23,096][105620] Updated weights for policy 1, policy_version 553315 (0.0010) [2023-12-26 19:23:23,342][105692] Updated weights for policy 0, policy_version 552488 (0.0008) [2023-12-26 19:23:23,375][105585] KL-divergence is very high: 132.6791 [2023-12-26 19:23:23,406][105692] Updated weights for policy 0, policy_version 552499 (0.0010) [2023-12-26 19:23:23,414][105585] KL-divergence is very high: 674.8807 [2023-12-26 19:23:23,453][105585] KL-divergence is very high: 1270.6945 [2023-12-26 19:23:23,453][105692] Updated weights for policy 0, policy_version 552509 (0.0009) [2023-12-26 19:23:23,828][105620] Updated weights for policy 1, policy_version 553325 (0.0009) [2023-12-26 19:23:23,874][105620] Updated weights for policy 1, policy_version 553335 (0.0008) [2023-12-26 19:23:23,928][105620] Updated weights for policy 1, policy_version 553345 (0.0007) [2023-12-26 19:23:24,215][105692] Updated weights for policy 0, policy_version 552519 (0.0009) [2023-12-26 19:23:24,264][105692] Updated weights for policy 0, policy_version 552529 (0.0009) [2023-12-26 19:23:24,319][105692] Updated weights for policy 0, policy_version 552539 (0.0006) [2023-12-26 19:23:24,671][105620] Updated weights for policy 1, policy_version 553355 (0.0009) [2023-12-26 19:23:24,734][105620] Updated weights for policy 1, policy_version 553365 (0.0005) [2023-12-26 19:23:24,783][105620] Updated weights for policy 1, policy_version 553375 (0.0005) [2023-12-26 19:23:25,083][105692] Updated weights for policy 0, policy_version 552549 (0.0008) [2023-12-26 19:23:25,141][105692] Updated weights for policy 0, policy_version 552559 (0.0010) [2023-12-26 19:23:25,197][105692] Updated weights for policy 0, policy_version 552569 (0.0010) [2023-12-26 19:23:25,474][105620] Updated weights for policy 1, policy_version 553385 (0.0006) [2023-12-26 19:23:25,524][105620] Updated weights for policy 1, policy_version 553395 (0.0007) [2023-12-26 19:23:25,572][105620] Updated weights for policy 1, policy_version 553405 (0.0008) [2023-12-26 19:23:25,638][105620] Updated weights for policy 1, policy_version 553415 (0.0008) [2023-12-26 19:23:25,957][105692] Updated weights for policy 0, policy_version 552579 (0.0010) [2023-12-26 19:23:26,014][105692] Updated weights for policy 0, policy_version 552589 (0.0010) [2023-12-26 19:23:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 283164672. Throughput: 0: 9702.5, 1: 9808.6. Samples: 283177452. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-26 19:23:26,062][104569] Avg episode reward: [(0, '8987.472'), (1, '9352.944')] [2023-12-26 19:23:26,066][105692] Updated weights for policy 0, policy_version 552599 (0.0009) [2023-12-26 19:23:26,279][105620] Updated weights for policy 1, policy_version 553425 (0.0008) [2023-12-26 19:23:26,341][105620] Updated weights for policy 1, policy_version 553435 (0.0009) [2023-12-26 19:23:26,408][105620] Updated weights for policy 1, policy_version 553445 (0.0009) [2023-12-26 19:23:26,820][105692] Updated weights for policy 0, policy_version 552609 (0.0010) [2023-12-26 19:23:26,865][105692] Updated weights for policy 0, policy_version 552619 (0.0008) [2023-12-26 19:23:26,918][105692] Updated weights for policy 0, policy_version 552629 (0.0009) [2023-12-26 19:23:26,982][105692] Updated weights for policy 0, policy_version 552639 (0.0009) [2023-12-26 19:23:27,103][105620] Updated weights for policy 1, policy_version 553455 (0.0009) [2023-12-26 19:23:27,153][105620] Updated weights for policy 1, policy_version 553465 (0.0009) [2023-12-26 19:23:27,214][105620] Updated weights for policy 1, policy_version 553475 (0.0009) [2023-12-26 19:23:27,681][105692] Updated weights for policy 0, policy_version 552649 (0.0008) [2023-12-26 19:23:27,735][105692] Updated weights for policy 0, policy_version 552659 (0.0009) [2023-12-26 19:23:27,791][105692] Updated weights for policy 0, policy_version 552669 (0.0009) [2023-12-26 19:23:27,934][105620] Updated weights for policy 1, policy_version 553485 (0.0007) [2023-12-26 19:23:27,997][105620] Updated weights for policy 1, policy_version 553495 (0.0009) [2023-12-26 19:23:28,050][105620] Updated weights for policy 1, policy_version 553506 (0.0010) [2023-12-26 19:23:28,417][105692] Updated weights for policy 0, policy_version 552679 (0.0007) [2023-12-26 19:23:28,485][105692] Updated weights for policy 0, policy_version 552689 (0.0006) [2023-12-26 19:23:28,549][105692] Updated weights for policy 0, policy_version 552699 (0.0006) [2023-12-26 19:23:28,890][105620] Updated weights for policy 1, policy_version 553516 (0.0009) [2023-12-26 19:23:28,949][105620] Updated weights for policy 1, policy_version 553527 (0.0010) [2023-12-26 19:23:29,000][105620] Updated weights for policy 1, policy_version 553537 (0.0009) [2023-12-26 19:23:29,076][105692] Updated weights for policy 0, policy_version 552709 (0.0006) [2023-12-26 19:23:29,136][105692] Updated weights for policy 0, policy_version 552719 (0.0009) [2023-12-26 19:23:29,150][105585] KL-divergence is very high: 102.3439 [2023-12-26 19:23:29,190][105692] Updated weights for policy 0, policy_version 552729 (0.0009) [2023-12-26 19:23:29,780][105620] Updated weights for policy 1, policy_version 553548 (0.0011) [2023-12-26 19:23:29,844][105620] Updated weights for policy 1, policy_version 553558 (0.0009) [2023-12-26 19:23:29,912][105620] Updated weights for policy 1, policy_version 553568 (0.0006) [2023-12-26 19:23:29,985][105692] Updated weights for policy 0, policy_version 552739 (0.0008) [2023-12-26 19:23:30,037][105692] Updated weights for policy 0, policy_version 552749 (0.0008) [2023-12-26 19:23:30,103][105692] Updated weights for policy 0, policy_version 552759 (0.0009) [2023-12-26 19:23:30,488][105620] Updated weights for policy 1, policy_version 553578 (0.0009) [2023-12-26 19:23:30,543][105620] Updated weights for policy 1, policy_version 553588 (0.0010) [2023-12-26 19:23:30,605][105620] Updated weights for policy 1, policy_version 553598 (0.0010) [2023-12-26 19:23:30,669][105620] Updated weights for policy 1, policy_version 553608 (0.0010) [2023-12-26 19:23:30,910][105692] Updated weights for policy 0, policy_version 552769 (0.0009) [2023-12-26 19:23:30,970][105692] Updated weights for policy 0, policy_version 552779 (0.0009) [2023-12-26 19:23:31,023][105692] Updated weights for policy 0, policy_version 552789 (0.0010) [2023-12-26 19:23:31,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 283262976. Throughput: 0: 9749.1, 1: 9805.0. Samples: 283236572. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-26 19:23:31,062][104569] Avg episode reward: [(0, '8989.311'), (1, '9352.530')] [2023-12-26 19:23:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000553608_141737984.pth... [2023-12-26 19:23:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000552456_141443072.pth [2023-12-26 19:23:31,088][105692] Updated weights for policy 0, policy_version 552799 (0.0009) [2023-12-26 19:23:31,093][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000552800_141533184.pth... [2023-12-26 19:23:31,097][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000551680_141246464.pth [2023-12-26 19:23:31,330][105620] Updated weights for policy 1, policy_version 553618 (0.0006) [2023-12-26 19:23:31,396][105620] Updated weights for policy 1, policy_version 553628 (0.0008) [2023-12-26 19:23:31,456][105620] Updated weights for policy 1, policy_version 553638 (0.0010) [2023-12-26 19:23:31,851][105692] Updated weights for policy 0, policy_version 552809 (0.0009) [2023-12-26 19:23:31,906][105692] Updated weights for policy 0, policy_version 552819 (0.0009) [2023-12-26 19:23:31,956][105692] Updated weights for policy 0, policy_version 552829 (0.0009) [2023-12-26 19:23:32,135][105620] Updated weights for policy 1, policy_version 553648 (0.0011) [2023-12-26 19:23:32,193][105620] Updated weights for policy 1, policy_version 553658 (0.0010) [2023-12-26 19:23:32,256][105620] Updated weights for policy 1, policy_version 553668 (0.0010) [2023-12-26 19:23:32,791][105692] Updated weights for policy 0, policy_version 552839 (0.0009) [2023-12-26 19:23:32,847][105692] Updated weights for policy 0, policy_version 552849 (0.0009) [2023-12-26 19:23:32,903][105692] Updated weights for policy 0, policy_version 552859 (0.0009) [2023-12-26 19:23:32,922][105620] Updated weights for policy 1, policy_version 553678 (0.0006) [2023-12-26 19:23:32,983][105620] Updated weights for policy 1, policy_version 553688 (0.0005) [2023-12-26 19:23:33,033][105620] Updated weights for policy 1, policy_version 553698 (0.0008) [2023-12-26 19:23:33,609][105692] Updated weights for policy 0, policy_version 552869 (0.0007) [2023-12-26 19:23:33,664][105692] Updated weights for policy 0, policy_version 552880 (0.0010) [2023-12-26 19:23:33,717][105692] Updated weights for policy 0, policy_version 552891 (0.0010) [2023-12-26 19:23:33,744][105620] Updated weights for policy 1, policy_version 553708 (0.0010) [2023-12-26 19:23:33,800][105620] Updated weights for policy 1, policy_version 553718 (0.0007) [2023-12-26 19:23:33,855][105620] Updated weights for policy 1, policy_version 553728 (0.0005) [2023-12-26 19:23:34,431][105620] Updated weights for policy 1, policy_version 553738 (0.0006) [2023-12-26 19:23:34,491][105620] Updated weights for policy 1, policy_version 553748 (0.0009) [2023-12-26 19:23:34,551][105620] Updated weights for policy 1, policy_version 553758 (0.0008) [2023-12-26 19:23:34,573][105692] Updated weights for policy 0, policy_version 552901 (0.0009) [2023-12-26 19:23:34,603][105620] Updated weights for policy 1, policy_version 553768 (0.0005) [2023-12-26 19:23:34,621][105692] Updated weights for policy 0, policy_version 552911 (0.0010) [2023-12-26 19:23:34,674][105692] Updated weights for policy 0, policy_version 552921 (0.0011) [2023-12-26 19:23:35,320][105620] Updated weights for policy 1, policy_version 553778 (0.0009) [2023-12-26 19:23:35,366][105620] Updated weights for policy 1, policy_version 553788 (0.0009) [2023-12-26 19:23:35,413][105620] Updated weights for policy 1, policy_version 553798 (0.0009) [2023-12-26 19:23:35,427][105692] Updated weights for policy 0, policy_version 552931 (0.0010) [2023-12-26 19:23:35,486][105692] Updated weights for policy 0, policy_version 552941 (0.0008) [2023-12-26 19:23:35,532][105692] Updated weights for policy 0, policy_version 552951 (0.0009) [2023-12-26 19:23:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 283361280. Throughput: 0: 9601.3, 1: 9861.6. Samples: 283353824. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-26 19:23:36,062][104569] Avg episode reward: [(0, '8714.233'), (1, '9351.562')] [2023-12-26 19:23:36,166][105620] Updated weights for policy 1, policy_version 553808 (0.0009) [2023-12-26 19:23:36,226][105620] Updated weights for policy 1, policy_version 553818 (0.0008) [2023-12-26 19:23:36,287][105692] Updated weights for policy 0, policy_version 552961 (0.0009) [2023-12-26 19:23:36,289][105620] Updated weights for policy 1, policy_version 553828 (0.0008) [2023-12-26 19:23:36,345][105692] Updated weights for policy 0, policy_version 552971 (0.0008) [2023-12-26 19:23:36,382][105585] KL-divergence is very high: 316.3965 [2023-12-26 19:23:36,394][105585] KL-divergence is very high: 289.9139 [2023-12-26 19:23:36,404][105692] Updated weights for policy 0, policy_version 552981 (0.0008) [2023-12-26 19:23:36,427][105585] KL-divergence is very high: 478.1356 [2023-12-26 19:23:36,438][105585] KL-divergence is very high: 342.1354 [2023-12-26 19:23:36,459][105692] Updated weights for policy 0, policy_version 552991 (0.0008) [2023-12-26 19:23:37,064][105620] Updated weights for policy 1, policy_version 553838 (0.0008) [2023-12-26 19:23:37,123][105620] Updated weights for policy 1, policy_version 553848 (0.0010) [2023-12-26 19:23:37,180][105620] Updated weights for policy 1, policy_version 553858 (0.0009) [2023-12-26 19:23:37,207][105692] Updated weights for policy 0, policy_version 553001 (0.0009) [2023-12-26 19:23:37,273][105692] Updated weights for policy 0, policy_version 553011 (0.0011) [2023-12-26 19:23:37,339][105692] Updated weights for policy 0, policy_version 553021 (0.0011) [2023-12-26 19:23:37,763][105620] Updated weights for policy 1, policy_version 553868 (0.0008) [2023-12-26 19:23:37,812][105620] Updated weights for policy 1, policy_version 553878 (0.0008) [2023-12-26 19:23:37,866][105620] Updated weights for policy 1, policy_version 553888 (0.0005) [2023-12-26 19:23:37,921][105692] Updated weights for policy 0, policy_version 553031 (0.0011) [2023-12-26 19:23:37,972][105692] Updated weights for policy 0, policy_version 553041 (0.0010) [2023-12-26 19:23:38,028][105692] Updated weights for policy 0, policy_version 553051 (0.0009) [2023-12-26 19:23:38,475][105620] Updated weights for policy 1, policy_version 553898 (0.0006) [2023-12-26 19:23:38,537][105620] Updated weights for policy 1, policy_version 553908 (0.0008) [2023-12-26 19:23:38,595][105620] Updated weights for policy 1, policy_version 553918 (0.0008) [2023-12-26 19:23:38,651][105620] Updated weights for policy 1, policy_version 553928 (0.0008) [2023-12-26 19:23:38,783][105692] Updated weights for policy 0, policy_version 553061 (0.0010) [2023-12-26 19:23:38,838][105692] Updated weights for policy 0, policy_version 553071 (0.0011) [2023-12-26 19:23:38,897][105692] Updated weights for policy 0, policy_version 553081 (0.0011) [2023-12-26 19:23:39,298][105620] Updated weights for policy 1, policy_version 553938 (0.0007) [2023-12-26 19:23:39,367][105620] Updated weights for policy 1, policy_version 553948 (0.0008) [2023-12-26 19:23:39,437][105620] Updated weights for policy 1, policy_version 553958 (0.0008) [2023-12-26 19:23:39,681][105692] Updated weights for policy 0, policy_version 553091 (0.0010) [2023-12-26 19:23:39,740][105692] Updated weights for policy 0, policy_version 553101 (0.0010) [2023-12-26 19:23:39,796][105692] Updated weights for policy 0, policy_version 553111 (0.0009) [2023-12-26 19:23:40,145][105620] Updated weights for policy 1, policy_version 553968 (0.0008) [2023-12-26 19:23:40,212][105620] Updated weights for policy 1, policy_version 553978 (0.0009) [2023-12-26 19:23:40,274][105620] Updated weights for policy 1, policy_version 553988 (0.0009) [2023-12-26 19:23:40,566][105692] Updated weights for policy 0, policy_version 553121 (0.0009) [2023-12-26 19:23:40,628][105692] Updated weights for policy 0, policy_version 553131 (0.0010) [2023-12-26 19:23:40,683][105692] Updated weights for policy 0, policy_version 553141 (0.0010) [2023-12-26 19:23:40,734][105692] Updated weights for policy 0, policy_version 553151 (0.0010) [2023-12-26 19:23:41,020][105620] Updated weights for policy 1, policy_version 553998 (0.0007) [2023-12-26 19:23:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 283459584. Throughput: 0: 9548.5, 1: 9982.1. Samples: 283471228. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-26 19:23:41,062][104569] Avg episode reward: [(0, '8714.065'), (1, '9351.200')] [2023-12-26 19:23:41,084][105620] Updated weights for policy 1, policy_version 554008 (0.0010) [2023-12-26 19:23:41,151][105620] Updated weights for policy 1, policy_version 554018 (0.0008) [2023-12-26 19:23:41,526][105692] Updated weights for policy 0, policy_version 553161 (0.0009) [2023-12-26 19:23:41,590][105692] Updated weights for policy 0, policy_version 553171 (0.0005) [2023-12-26 19:23:41,661][105692] Updated weights for policy 0, policy_version 553181 (0.0008) [2023-12-26 19:23:41,889][105620] Updated weights for policy 1, policy_version 554028 (0.0009) [2023-12-26 19:23:41,953][105620] Updated weights for policy 1, policy_version 554038 (0.0008) [2023-12-26 19:23:42,012][105620] Updated weights for policy 1, policy_version 554048 (0.0008) [2023-12-26 19:23:42,383][105692] Updated weights for policy 0, policy_version 553191 (0.0007) [2023-12-26 19:23:42,445][105692] Updated weights for policy 0, policy_version 553201 (0.0009) [2023-12-26 19:23:42,506][105692] Updated weights for policy 0, policy_version 553211 (0.0009) [2023-12-26 19:23:42,792][105620] Updated weights for policy 1, policy_version 554058 (0.0008) [2023-12-26 19:23:42,850][105620] Updated weights for policy 1, policy_version 554068 (0.0009) [2023-12-26 19:23:42,912][105620] Updated weights for policy 1, policy_version 554078 (0.0009) [2023-12-26 19:23:42,966][105620] Updated weights for policy 1, policy_version 554088 (0.0009) [2023-12-26 19:23:43,272][105692] Updated weights for policy 0, policy_version 553221 (0.0009) [2023-12-26 19:23:43,329][105692] Updated weights for policy 0, policy_version 553231 (0.0008) [2023-12-26 19:23:43,390][105692] Updated weights for policy 0, policy_version 553241 (0.0008) [2023-12-26 19:23:43,732][105620] Updated weights for policy 1, policy_version 554098 (0.0006) [2023-12-26 19:23:43,794][105620] Updated weights for policy 1, policy_version 554108 (0.0006) [2023-12-26 19:23:43,854][105620] Updated weights for policy 1, policy_version 554118 (0.0005) [2023-12-26 19:23:43,961][105692] Updated weights for policy 0, policy_version 553251 (0.0008) [2023-12-26 19:23:44,032][105692] Updated weights for policy 0, policy_version 553261 (0.0005) [2023-12-26 19:23:44,106][105692] Updated weights for policy 0, policy_version 553271 (0.0007) [2023-12-26 19:23:44,410][105620] Updated weights for policy 1, policy_version 554128 (0.0007) [2023-12-26 19:23:44,474][105620] Updated weights for policy 1, policy_version 554138 (0.0006) [2023-12-26 19:23:44,531][105620] Updated weights for policy 1, policy_version 554148 (0.0008) [2023-12-26 19:23:44,686][105692] Updated weights for policy 0, policy_version 553281 (0.0009) [2023-12-26 19:23:44,746][105692] Updated weights for policy 0, policy_version 553291 (0.0006) [2023-12-26 19:23:44,811][105692] Updated weights for policy 0, policy_version 553301 (0.0007) [2023-12-26 19:23:44,878][105692] Updated weights for policy 0, policy_version 553311 (0.0005) [2023-12-26 19:23:45,334][105620] Updated weights for policy 1, policy_version 554158 (0.0010) [2023-12-26 19:23:45,392][105620] Updated weights for policy 1, policy_version 554169 (0.0011) [2023-12-26 19:23:45,427][105692] Updated weights for policy 0, policy_version 553321 (0.0006) [2023-12-26 19:23:45,452][105620] Updated weights for policy 1, policy_version 554179 (0.0009) [2023-12-26 19:23:45,486][105692] Updated weights for policy 0, policy_version 553331 (0.0006) [2023-12-26 19:23:45,542][105692] Updated weights for policy 0, policy_version 553341 (0.0008) [2023-12-26 19:23:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 283557888. Throughput: 0: 9495.8, 1: 9931.7. Samples: 283526844. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-26 19:23:46,062][104569] Avg episode reward: [(0, '8985.887'), (1, '9351.539')] [2023-12-26 19:23:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000554184_141885440.pth... [2023-12-26 19:23:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000553344_141672448.pth... [2023-12-26 19:23:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000553032_141590528.pth [2023-12-26 19:23:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000552224_141385728.pth [2023-12-26 19:23:46,224][105620] Updated weights for policy 1, policy_version 554189 (0.0010) [2023-12-26 19:23:46,242][105692] Updated weights for policy 0, policy_version 553351 (0.0010) [2023-12-26 19:23:46,273][105620] Updated weights for policy 1, policy_version 554199 (0.0010) [2023-12-26 19:23:46,297][105692] Updated weights for policy 0, policy_version 553361 (0.0010) [2023-12-26 19:23:46,327][105620] Updated weights for policy 1, policy_version 554209 (0.0010) [2023-12-26 19:23:46,358][105692] Updated weights for policy 0, policy_version 553371 (0.0010) [2023-12-26 19:23:47,036][105692] Updated weights for policy 0, policy_version 553381 (0.0008) [2023-12-26 19:23:47,079][105620] Updated weights for policy 1, policy_version 554219 (0.0010) [2023-12-26 19:23:47,091][105692] Updated weights for policy 0, policy_version 553391 (0.0006) [2023-12-26 19:23:47,133][105620] Updated weights for policy 1, policy_version 554229 (0.0010) [2023-12-26 19:23:47,151][105692] Updated weights for policy 0, policy_version 553401 (0.0006) [2023-12-26 19:23:47,181][105620] Updated weights for policy 1, policy_version 554239 (0.0010) [2023-12-26 19:23:47,816][105692] Updated weights for policy 0, policy_version 553411 (0.0007) [2023-12-26 19:23:47,869][105692] Updated weights for policy 0, policy_version 553421 (0.0008) [2023-12-26 19:23:47,913][105692] Updated weights for policy 0, policy_version 553431 (0.0008) [2023-12-26 19:23:47,941][105620] Updated weights for policy 1, policy_version 554249 (0.0010) [2023-12-26 19:23:47,985][105620] Updated weights for policy 1, policy_version 554259 (0.0010) [2023-12-26 19:23:48,043][105620] Updated weights for policy 1, policy_version 554269 (0.0010) [2023-12-26 19:23:48,101][105620] Updated weights for policy 1, policy_version 554279 (0.0010) [2023-12-26 19:23:48,573][105692] Updated weights for policy 0, policy_version 553441 (0.0006) [2023-12-26 19:23:48,633][105692] Updated weights for policy 0, policy_version 553451 (0.0008) [2023-12-26 19:23:48,698][105692] Updated weights for policy 0, policy_version 553461 (0.0006) [2023-12-26 19:23:48,770][105692] Updated weights for policy 0, policy_version 553471 (0.0006) [2023-12-26 19:23:48,862][105620] Updated weights for policy 1, policy_version 554289 (0.0010) [2023-12-26 19:23:48,914][105620] Updated weights for policy 1, policy_version 554299 (0.0010) [2023-12-26 19:23:48,962][105620] Updated weights for policy 1, policy_version 554309 (0.0010) [2023-12-26 19:23:49,462][105692] Updated weights for policy 0, policy_version 553481 (0.0010) [2023-12-26 19:23:49,526][105692] Updated weights for policy 0, policy_version 553491 (0.0008) [2023-12-26 19:23:49,585][105692] Updated weights for policy 0, policy_version 553501 (0.0009) [2023-12-26 19:23:49,710][105620] Updated weights for policy 1, policy_version 554319 (0.0009) [2023-12-26 19:23:49,766][105620] Updated weights for policy 1, policy_version 554329 (0.0011) [2023-12-26 19:23:49,819][105620] Updated weights for policy 1, policy_version 554339 (0.0011) [2023-12-26 19:23:50,226][105692] Updated weights for policy 0, policy_version 553511 (0.0006) [2023-12-26 19:23:50,289][105692] Updated weights for policy 0, policy_version 553521 (0.0006) [2023-12-26 19:23:50,352][105692] Updated weights for policy 0, policy_version 553531 (0.0005) [2023-12-26 19:23:50,520][105620] Updated weights for policy 1, policy_version 554349 (0.0008) [2023-12-26 19:23:50,590][105620] Updated weights for policy 1, policy_version 554359 (0.0009) [2023-12-26 19:23:50,639][105620] Updated weights for policy 1, policy_version 554369 (0.0008) [2023-12-26 19:23:50,974][105692] Updated weights for policy 0, policy_version 553541 (0.0007) [2023-12-26 19:23:51,028][105692] Updated weights for policy 0, policy_version 553551 (0.0008) [2023-12-26 19:23:51,043][105585] KL-divergence is very high: 133.5600 [2023-12-26 19:23:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 283656192. Throughput: 0: 9678.3, 1: 9825.3. Samples: 283647300. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-26 19:23:51,062][104569] Avg episode reward: [(0, '9170.493'), (1, '9351.473')] [2023-12-26 19:23:51,093][105585] KL-divergence is very high: 132.9351 [2023-12-26 19:23:51,093][105692] Updated weights for policy 0, policy_version 553561 (0.0008) [2023-12-26 19:23:51,386][105620] Updated weights for policy 1, policy_version 554379 (0.0010) [2023-12-26 19:23:51,445][105620] Updated weights for policy 1, policy_version 554389 (0.0011) [2023-12-26 19:23:51,501][105620] Updated weights for policy 1, policy_version 554399 (0.0010) [2023-12-26 19:23:51,873][105692] Updated weights for policy 0, policy_version 553571 (0.0008) [2023-12-26 19:23:51,925][105692] Updated weights for policy 0, policy_version 553581 (0.0007) [2023-12-26 19:23:51,980][105692] Updated weights for policy 0, policy_version 553591 (0.0005) [2023-12-26 19:23:52,284][105620] Updated weights for policy 1, policy_version 554409 (0.0010) [2023-12-26 19:23:52,336][105620] Updated weights for policy 1, policy_version 554419 (0.0010) [2023-12-26 19:23:52,394][105620] Updated weights for policy 1, policy_version 554429 (0.0011) [2023-12-26 19:23:52,443][105620] Updated weights for policy 1, policy_version 554439 (0.0010) [2023-12-26 19:23:52,663][105692] Updated weights for policy 0, policy_version 553601 (0.0007) [2023-12-26 19:23:52,726][105692] Updated weights for policy 0, policy_version 553611 (0.0008) [2023-12-26 19:23:52,784][105692] Updated weights for policy 0, policy_version 553621 (0.0009) [2023-12-26 19:23:52,857][105692] Updated weights for policy 0, policy_version 553631 (0.0010) [2023-12-26 19:23:53,127][105620] Updated weights for policy 1, policy_version 554449 (0.0009) [2023-12-26 19:23:53,185][105620] Updated weights for policy 1, policy_version 554459 (0.0008) [2023-12-26 19:23:53,231][105620] Updated weights for policy 1, policy_version 554469 (0.0008) [2023-12-26 19:23:53,601][105692] Updated weights for policy 0, policy_version 553641 (0.0006) [2023-12-26 19:23:53,660][105692] Updated weights for policy 0, policy_version 553651 (0.0006) [2023-12-26 19:23:53,725][105692] Updated weights for policy 0, policy_version 553661 (0.0007) [2023-12-26 19:23:53,986][105620] Updated weights for policy 1, policy_version 554479 (0.0006) [2023-12-26 19:23:54,052][105620] Updated weights for policy 1, policy_version 554489 (0.0005) [2023-12-26 19:23:54,117][105620] Updated weights for policy 1, policy_version 554499 (0.0008) [2023-12-26 19:23:54,380][105692] Updated weights for policy 0, policy_version 553671 (0.0009) [2023-12-26 19:23:54,442][105692] Updated weights for policy 0, policy_version 553681 (0.0009) [2023-12-26 19:23:54,505][105692] Updated weights for policy 0, policy_version 553691 (0.0009) [2023-12-26 19:23:54,779][105620] Updated weights for policy 1, policy_version 554509 (0.0007) [2023-12-26 19:23:54,832][105620] Updated weights for policy 1, policy_version 554519 (0.0008) [2023-12-26 19:23:54,886][105620] Updated weights for policy 1, policy_version 554529 (0.0009) [2023-12-26 19:23:55,268][105692] Updated weights for policy 0, policy_version 553701 (0.0009) [2023-12-26 19:23:55,322][105692] Updated weights for policy 0, policy_version 553711 (0.0009) [2023-12-26 19:23:55,373][105692] Updated weights for policy 0, policy_version 553721 (0.0009) [2023-12-26 19:23:55,609][105620] Updated weights for policy 1, policy_version 554539 (0.0008) [2023-12-26 19:23:55,665][105620] Updated weights for policy 1, policy_version 554549 (0.0005) [2023-12-26 19:23:55,731][105620] Updated weights for policy 1, policy_version 554559 (0.0005) [2023-12-26 19:23:56,062][104569] Fps is (10 sec: 19660.0, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 283754496. Throughput: 0: 9695.9, 1: 9839.1. Samples: 283764196. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-26 19:23:56,064][104569] Avg episode reward: [(0, '9175.395'), (1, '9260.639')] [2023-12-26 19:23:56,151][105692] Updated weights for policy 0, policy_version 553731 (0.0009) [2023-12-26 19:23:56,211][105692] Updated weights for policy 0, policy_version 553741 (0.0009) [2023-12-26 19:23:56,270][105692] Updated weights for policy 0, policy_version 553751 (0.0009) [2023-12-26 19:23:56,440][105620] Updated weights for policy 1, policy_version 554569 (0.0008) [2023-12-26 19:23:56,490][105620] Updated weights for policy 1, policy_version 554579 (0.0009) [2023-12-26 19:23:56,537][105620] Updated weights for policy 1, policy_version 554589 (0.0009) [2023-12-26 19:23:56,585][105620] Updated weights for policy 1, policy_version 554599 (0.0009) [2023-12-26 19:23:57,016][105692] Updated weights for policy 0, policy_version 553761 (0.0009) [2023-12-26 19:23:57,069][105692] Updated weights for policy 0, policy_version 553771 (0.0008) [2023-12-26 19:23:57,125][105692] Updated weights for policy 0, policy_version 553781 (0.0009) [2023-12-26 19:23:57,179][105692] Updated weights for policy 0, policy_version 553791 (0.0009) [2023-12-26 19:23:57,348][105620] Updated weights for policy 1, policy_version 554609 (0.0009) [2023-12-26 19:23:57,408][105620] Updated weights for policy 1, policy_version 554619 (0.0009) [2023-12-26 19:23:57,470][105620] Updated weights for policy 1, policy_version 554629 (0.0008) [2023-12-26 19:23:57,840][105692] Updated weights for policy 0, policy_version 553801 (0.0006) [2023-12-26 19:23:57,888][105692] Updated weights for policy 0, policy_version 553811 (0.0008) [2023-12-26 19:23:57,936][105692] Updated weights for policy 0, policy_version 553821 (0.0009) [2023-12-26 19:23:58,231][105620] Updated weights for policy 1, policy_version 554639 (0.0009) [2023-12-26 19:23:58,296][105620] Updated weights for policy 1, policy_version 554649 (0.0008) [2023-12-26 19:23:58,364][105620] Updated weights for policy 1, policy_version 554659 (0.0008) [2023-12-26 19:23:58,670][105692] Updated weights for policy 0, policy_version 553831 (0.0009) [2023-12-26 19:23:58,734][105692] Updated weights for policy 0, policy_version 553841 (0.0008) [2023-12-26 19:23:58,810][105692] Updated weights for policy 0, policy_version 553851 (0.0007) [2023-12-26 19:23:59,154][105620] Updated weights for policy 1, policy_version 554669 (0.0009) [2023-12-26 19:23:59,216][105620] Updated weights for policy 1, policy_version 554679 (0.0010) [2023-12-26 19:23:59,283][105620] Updated weights for policy 1, policy_version 554689 (0.0007) [2023-12-26 19:23:59,650][105692] Updated weights for policy 0, policy_version 553861 (0.0009) [2023-12-26 19:23:59,701][105692] Updated weights for policy 0, policy_version 553871 (0.0008) [2023-12-26 19:23:59,754][105692] Updated weights for policy 0, policy_version 553881 (0.0007) [2023-12-26 19:24:00,009][105620] Updated weights for policy 1, policy_version 554699 (0.0007) [2023-12-26 19:24:00,078][105620] Updated weights for policy 1, policy_version 554709 (0.0011) [2023-12-26 19:24:00,126][105620] Updated weights for policy 1, policy_version 554719 (0.0010) [2023-12-26 19:24:00,407][105692] Updated weights for policy 0, policy_version 553891 (0.0010) [2023-12-26 19:24:00,458][105692] Updated weights for policy 0, policy_version 553901 (0.0010) [2023-12-26 19:24:00,513][105692] Updated weights for policy 0, policy_version 553911 (0.0010) [2023-12-26 19:24:00,755][105620] Updated weights for policy 1, policy_version 554729 (0.0010) [2023-12-26 19:24:00,824][105620] Updated weights for policy 1, policy_version 554739 (0.0005) [2023-12-26 19:24:00,885][105620] Updated weights for policy 1, policy_version 554749 (0.0006) [2023-12-26 19:24:00,936][105620] Updated weights for policy 1, policy_version 554759 (0.0010) [2023-12-26 19:24:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 283852800. Throughput: 0: 9782.3, 1: 9766.3. Samples: 283820392. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-26 19:24:01,062][104569] Avg episode reward: [(0, '9095.828'), (1, '9260.317')] [2023-12-26 19:24:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000553920_141819904.pth... [2023-12-26 19:24:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000554760_142032896.pth... [2023-12-26 19:24:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000553608_141737984.pth [2023-12-26 19:24:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000552800_141533184.pth [2023-12-26 19:24:01,243][105692] Updated weights for policy 0, policy_version 553921 (0.0010) [2023-12-26 19:24:01,307][105692] Updated weights for policy 0, policy_version 553931 (0.0010) [2023-12-26 19:24:01,368][105692] Updated weights for policy 0, policy_version 553941 (0.0009) [2023-12-26 19:24:01,420][105692] Updated weights for policy 0, policy_version 553951 (0.0009) [2023-12-26 19:24:01,676][105620] Updated weights for policy 1, policy_version 554769 (0.0009) [2023-12-26 19:24:01,743][105620] Updated weights for policy 1, policy_version 554779 (0.0010) [2023-12-26 19:24:01,804][105620] Updated weights for policy 1, policy_version 554789 (0.0009) [2023-12-26 19:24:02,151][105692] Updated weights for policy 0, policy_version 553961 (0.0006) [2023-12-26 19:24:02,219][105692] Updated weights for policy 0, policy_version 553971 (0.0005) [2023-12-26 19:24:02,273][105692] Updated weights for policy 0, policy_version 553981 (0.0007) [2023-12-26 19:24:02,377][105620] Updated weights for policy 1, policy_version 554799 (0.0009) [2023-12-26 19:24:02,432][105620] Updated weights for policy 1, policy_version 554809 (0.0010) [2023-12-26 19:24:02,487][105620] Updated weights for policy 1, policy_version 554819 (0.0010) [2023-12-26 19:24:02,812][105692] Updated weights for policy 0, policy_version 553991 (0.0006) [2023-12-26 19:24:02,864][105692] Updated weights for policy 0, policy_version 554001 (0.0005) [2023-12-26 19:24:02,928][105692] Updated weights for policy 0, policy_version 554011 (0.0005) [2023-12-26 19:24:03,056][105620] Updated weights for policy 1, policy_version 554829 (0.0010) [2023-12-26 19:24:03,111][105620] Updated weights for policy 1, policy_version 554839 (0.0008) [2023-12-26 19:24:03,164][105620] Updated weights for policy 1, policy_version 554849 (0.0010) [2023-12-26 19:24:03,508][105692] Updated weights for policy 0, policy_version 554021 (0.0005) [2023-12-26 19:24:03,576][105692] Updated weights for policy 0, policy_version 554031 (0.0005) [2023-12-26 19:24:03,643][105692] Updated weights for policy 0, policy_version 554041 (0.0005) [2023-12-26 19:24:03,899][105620] Updated weights for policy 1, policy_version 554859 (0.0010) [2023-12-26 19:24:03,963][105620] Updated weights for policy 1, policy_version 554869 (0.0011) [2023-12-26 19:24:04,027][105620] Updated weights for policy 1, policy_version 554879 (0.0011) [2023-12-26 19:24:04,228][105692] Updated weights for policy 0, policy_version 554051 (0.0007) [2023-12-26 19:24:04,290][105692] Updated weights for policy 0, policy_version 554061 (0.0009) [2023-12-26 19:24:04,355][105692] Updated weights for policy 0, policy_version 554071 (0.0008) [2023-12-26 19:24:04,779][105620] Updated weights for policy 1, policy_version 554889 (0.0011) [2023-12-26 19:24:04,834][105620] Updated weights for policy 1, policy_version 554899 (0.0010) [2023-12-26 19:24:04,898][105620] Updated weights for policy 1, policy_version 554909 (0.0010) [2023-12-26 19:24:04,950][105620] Updated weights for policy 1, policy_version 554919 (0.0010) [2023-12-26 19:24:05,004][105692] Updated weights for policy 0, policy_version 554081 (0.0008) [2023-12-26 19:24:05,063][105692] Updated weights for policy 0, policy_version 554091 (0.0008) [2023-12-26 19:24:05,117][105692] Updated weights for policy 0, policy_version 554101 (0.0008) [2023-12-26 19:24:05,177][105692] Updated weights for policy 0, policy_version 554111 (0.0008) [2023-12-26 19:24:05,692][105620] Updated weights for policy 1, policy_version 554929 (0.0010) [2023-12-26 19:24:05,754][105620] Updated weights for policy 1, policy_version 554939 (0.0009) [2023-12-26 19:24:05,810][105620] Updated weights for policy 1, policy_version 554949 (0.0009) [2023-12-26 19:24:05,935][105692] Updated weights for policy 0, policy_version 554121 (0.0008) [2023-12-26 19:24:05,986][105692] Updated weights for policy 0, policy_version 554131 (0.0008) [2023-12-26 19:24:06,035][105692] Updated weights for policy 0, policy_version 554141 (0.0008) [2023-12-26 19:24:06,062][104569] Fps is (10 sec: 20481.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 283959296. Throughput: 0: 9824.1, 1: 9752.1. Samples: 283942728. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-26 19:24:06,062][104569] Avg episode reward: [(0, '9002.719'), (1, '9350.250')] [2023-12-26 19:24:06,530][105620] Updated weights for policy 1, policy_version 554959 (0.0009) [2023-12-26 19:24:06,590][105620] Updated weights for policy 1, policy_version 554969 (0.0009) [2023-12-26 19:24:06,647][105620] Updated weights for policy 1, policy_version 554979 (0.0010) [2023-12-26 19:24:06,819][105692] Updated weights for policy 0, policy_version 554151 (0.0009) [2023-12-26 19:24:06,870][105692] Updated weights for policy 0, policy_version 554161 (0.0009) [2023-12-26 19:24:06,919][105692] Updated weights for policy 0, policy_version 554171 (0.0008) [2023-12-26 19:24:07,335][105620] Updated weights for policy 1, policy_version 554989 (0.0008) [2023-12-26 19:24:07,397][105620] Updated weights for policy 1, policy_version 554999 (0.0005) [2023-12-26 19:24:07,449][105620] Updated weights for policy 1, policy_version 555009 (0.0005) [2023-12-26 19:24:07,797][105692] Updated weights for policy 0, policy_version 554181 (0.0009) [2023-12-26 19:24:07,862][105692] Updated weights for policy 0, policy_version 554191 (0.0009) [2023-12-26 19:24:07,918][105692] Updated weights for policy 0, policy_version 554201 (0.0008) [2023-12-26 19:24:07,940][105585] KL-divergence is very high: 104.0915 [2023-12-26 19:24:08,065][105620] Updated weights for policy 1, policy_version 555019 (0.0007) [2023-12-26 19:24:08,116][105620] Updated weights for policy 1, policy_version 555029 (0.0009) [2023-12-26 19:24:08,164][105620] Updated weights for policy 1, policy_version 555039 (0.0009) [2023-12-26 19:24:08,667][105692] Updated weights for policy 0, policy_version 554211 (0.0008) [2023-12-26 19:24:08,736][105692] Updated weights for policy 0, policy_version 554221 (0.0006) [2023-12-26 19:24:08,789][105692] Updated weights for policy 0, policy_version 554231 (0.0008) [2023-12-26 19:24:08,973][105620] Updated weights for policy 1, policy_version 555049 (0.0009) [2023-12-26 19:24:09,017][105620] Updated weights for policy 1, policy_version 555059 (0.0010) [2023-12-26 19:24:09,069][105620] Updated weights for policy 1, policy_version 555069 (0.0010) [2023-12-26 19:24:09,131][105620] Updated weights for policy 1, policy_version 555079 (0.0010) [2023-12-26 19:24:09,523][105692] Updated weights for policy 0, policy_version 554241 (0.0008) [2023-12-26 19:24:09,579][105692] Updated weights for policy 0, policy_version 554251 (0.0008) [2023-12-26 19:24:09,643][105692] Updated weights for policy 0, policy_version 554261 (0.0008) [2023-12-26 19:24:09,710][105692] Updated weights for policy 0, policy_version 554271 (0.0008) [2023-12-26 19:24:09,919][105620] Updated weights for policy 1, policy_version 555089 (0.0011) [2023-12-26 19:24:09,987][105620] Updated weights for policy 1, policy_version 555099 (0.0008) [2023-12-26 19:24:10,053][105620] Updated weights for policy 1, policy_version 555109 (0.0008) [2023-12-26 19:24:10,516][105692] Updated weights for policy 0, policy_version 554281 (0.0008) [2023-12-26 19:24:10,572][105692] Updated weights for policy 0, policy_version 554291 (0.0005) [2023-12-26 19:24:10,631][105692] Updated weights for policy 0, policy_version 554301 (0.0005) [2023-12-26 19:24:10,801][105620] Updated weights for policy 1, policy_version 555119 (0.0008) [2023-12-26 19:24:10,870][105620] Updated weights for policy 1, policy_version 555129 (0.0006) [2023-12-26 19:24:10,937][105620] Updated weights for policy 1, policy_version 555139 (0.0007) [2023-12-26 19:24:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 284049408. Throughput: 0: 9750.5, 1: 9768.8. Samples: 284055820. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-26 19:24:11,062][104569] Avg episode reward: [(0, '8897.734'), (1, '9349.210')] [2023-12-26 19:24:11,341][105692] Updated weights for policy 0, policy_version 554311 (0.0009) [2023-12-26 19:24:11,404][105692] Updated weights for policy 0, policy_version 554321 (0.0009) [2023-12-26 19:24:11,461][105692] Updated weights for policy 0, policy_version 554331 (0.0008) [2023-12-26 19:24:11,578][105620] Updated weights for policy 1, policy_version 555149 (0.0008) [2023-12-26 19:24:11,645][105620] Updated weights for policy 1, policy_version 555159 (0.0008) [2023-12-26 19:24:11,697][105620] Updated weights for policy 1, policy_version 555169 (0.0006) [2023-12-26 19:24:12,280][105692] Updated weights for policy 0, policy_version 554341 (0.0009) [2023-12-26 19:24:12,345][105692] Updated weights for policy 0, policy_version 554351 (0.0009) [2023-12-26 19:24:12,412][105692] Updated weights for policy 0, policy_version 554361 (0.0008) [2023-12-26 19:24:12,466][105620] Updated weights for policy 1, policy_version 555179 (0.0009) [2023-12-26 19:24:12,521][105620] Updated weights for policy 1, policy_version 555189 (0.0010) [2023-12-26 19:24:12,575][105620] Updated weights for policy 1, policy_version 555199 (0.0010) [2023-12-26 19:24:13,184][105692] Updated weights for policy 0, policy_version 554371 (0.0008) [2023-12-26 19:24:13,221][105620] Updated weights for policy 1, policy_version 555209 (0.0010) [2023-12-26 19:24:13,238][105692] Updated weights for policy 0, policy_version 554381 (0.0006) [2023-12-26 19:24:13,277][105620] Updated weights for policy 1, policy_version 555219 (0.0010) [2023-12-26 19:24:13,296][105692] Updated weights for policy 0, policy_version 554391 (0.0010) [2023-12-26 19:24:13,346][105620] Updated weights for policy 1, policy_version 555229 (0.0010) [2023-12-26 19:24:13,397][105620] Updated weights for policy 1, policy_version 555239 (0.0010) [2023-12-26 19:24:13,948][105692] Updated weights for policy 0, policy_version 554401 (0.0010) [2023-12-26 19:24:14,002][105692] Updated weights for policy 0, policy_version 554411 (0.0005) [2023-12-26 19:24:14,051][105692] Updated weights for policy 0, policy_version 554421 (0.0005) [2023-12-26 19:24:14,063][105620] Updated weights for policy 1, policy_version 555249 (0.0007) [2023-12-26 19:24:14,103][105692] Updated weights for policy 0, policy_version 554431 (0.0006) [2023-12-26 19:24:14,113][105620] Updated weights for policy 1, policy_version 555259 (0.0009) [2023-12-26 19:24:14,161][105620] Updated weights for policy 1, policy_version 555269 (0.0006) [2023-12-26 19:24:14,797][105692] Updated weights for policy 0, policy_version 554441 (0.0008) [2023-12-26 19:24:14,862][105692] Updated weights for policy 0, policy_version 554451 (0.0006) [2023-12-26 19:24:14,877][105620] Updated weights for policy 1, policy_version 555279 (0.0006) [2023-12-26 19:24:14,928][105692] Updated weights for policy 0, policy_version 554461 (0.0010) [2023-12-26 19:24:14,936][105620] Updated weights for policy 1, policy_version 555289 (0.0007) [2023-12-26 19:24:14,987][105620] Updated weights for policy 1, policy_version 555299 (0.0006) [2023-12-26 19:24:15,642][105692] Updated weights for policy 0, policy_version 554471 (0.0009) [2023-12-26 19:24:15,699][105692] Updated weights for policy 0, policy_version 554481 (0.0009) [2023-12-26 19:24:15,735][105620] Updated weights for policy 1, policy_version 555309 (0.0007) [2023-12-26 19:24:15,749][105692] Updated weights for policy 0, policy_version 554491 (0.0007) [2023-12-26 19:24:15,780][105620] Updated weights for policy 1, policy_version 555319 (0.0006) [2023-12-26 19:24:15,831][105620] Updated weights for policy 1, policy_version 555329 (0.0009) [2023-12-26 19:24:16,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 284147712. Throughput: 0: 9699.3, 1: 9789.0. Samples: 284113548. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-26 19:24:16,062][104569] Avg episode reward: [(0, '9083.972'), (1, '9258.135')] [2023-12-26 19:24:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000554496_141967360.pth... [2023-12-26 19:24:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000555336_142180352.pth... [2023-12-26 19:24:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000553344_141672448.pth [2023-12-26 19:24:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000554184_141885440.pth [2023-12-26 19:24:16,513][105692] Updated weights for policy 0, policy_version 554501 (0.0008) [2023-12-26 19:24:16,564][105620] Updated weights for policy 1, policy_version 555339 (0.0008) [2023-12-26 19:24:16,569][105692] Updated weights for policy 0, policy_version 554511 (0.0010) [2023-12-26 19:24:16,618][105620] Updated weights for policy 1, policy_version 555349 (0.0005) [2023-12-26 19:24:16,621][105692] Updated weights for policy 0, policy_version 554521 (0.0010) [2023-12-26 19:24:16,669][105620] Updated weights for policy 1, policy_version 555359 (0.0005) [2023-12-26 19:24:17,364][105692] Updated weights for policy 0, policy_version 554531 (0.0009) [2023-12-26 19:24:17,378][105620] Updated weights for policy 1, policy_version 555369 (0.0007) [2023-12-26 19:24:17,418][105692] Updated weights for policy 0, policy_version 554541 (0.0007) [2023-12-26 19:24:17,429][105620] Updated weights for policy 1, policy_version 555379 (0.0008) [2023-12-26 19:24:17,473][105692] Updated weights for policy 0, policy_version 554551 (0.0010) [2023-12-26 19:24:17,480][105620] Updated weights for policy 1, policy_version 555389 (0.0006) [2023-12-26 19:24:17,539][105620] Updated weights for policy 1, policy_version 555399 (0.0007) [2023-12-26 19:24:18,025][105692] Updated weights for policy 0, policy_version 554561 (0.0010) [2023-12-26 19:24:18,082][105692] Updated weights for policy 0, policy_version 554571 (0.0006) [2023-12-26 19:24:18,142][105692] Updated weights for policy 0, policy_version 554581 (0.0006) [2023-12-26 19:24:18,204][105692] Updated weights for policy 0, policy_version 554591 (0.0010) [2023-12-26 19:24:18,243][105620] Updated weights for policy 1, policy_version 555409 (0.0007) [2023-12-26 19:24:18,301][105620] Updated weights for policy 1, policy_version 555419 (0.0007) [2023-12-26 19:24:18,365][105620] Updated weights for policy 1, policy_version 555429 (0.0007) [2023-12-26 19:24:18,949][105620] Updated weights for policy 1, policy_version 555439 (0.0007) [2023-12-26 19:24:18,992][105620] Updated weights for policy 1, policy_version 555449 (0.0007) [2023-12-26 19:24:19,012][105692] Updated weights for policy 0, policy_version 554601 (0.0008) [2023-12-26 19:24:19,039][105620] Updated weights for policy 1, policy_version 555459 (0.0008) [2023-12-26 19:24:19,077][105692] Updated weights for policy 0, policy_version 554611 (0.0006) [2023-12-26 19:24:19,139][105692] Updated weights for policy 0, policy_version 554621 (0.0006) [2023-12-26 19:24:19,864][105620] Updated weights for policy 1, policy_version 555469 (0.0009) [2023-12-26 19:24:19,870][105692] Updated weights for policy 0, policy_version 554631 (0.0008) [2023-12-26 19:24:19,931][105620] Updated weights for policy 1, policy_version 555479 (0.0008) [2023-12-26 19:24:19,937][105692] Updated weights for policy 0, policy_version 554641 (0.0007) [2023-12-26 19:24:19,991][105620] Updated weights for policy 1, policy_version 555489 (0.0006) [2023-12-26 19:24:20,000][105692] Updated weights for policy 0, policy_version 554651 (0.0008) [2023-12-26 19:24:20,709][105692] Updated weights for policy 0, policy_version 554661 (0.0008) [2023-12-26 19:24:20,752][105620] Updated weights for policy 1, policy_version 555499 (0.0008) [2023-12-26 19:24:20,779][105692] Updated weights for policy 0, policy_version 554671 (0.0009) [2023-12-26 19:24:20,819][105620] Updated weights for policy 1, policy_version 555509 (0.0005) [2023-12-26 19:24:20,840][105692] Updated weights for policy 0, policy_version 554681 (0.0008) [2023-12-26 19:24:20,881][105620] Updated weights for policy 1, policy_version 555519 (0.0006) [2023-12-26 19:24:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 284246016. Throughput: 0: 9784.1, 1: 9732.7. Samples: 284232080. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-26 19:24:21,062][104569] Avg episode reward: [(0, '9087.066'), (1, '9166.727')] [2023-12-26 19:24:21,540][105692] Updated weights for policy 0, policy_version 554691 (0.0008) [2023-12-26 19:24:21,597][105692] Updated weights for policy 0, policy_version 554701 (0.0009) [2023-12-26 19:24:21,604][105620] Updated weights for policy 1, policy_version 555529 (0.0006) [2023-12-26 19:24:21,663][105692] Updated weights for policy 0, policy_version 554711 (0.0008) [2023-12-26 19:24:21,673][105620] Updated weights for policy 1, policy_version 555539 (0.0007) [2023-12-26 19:24:21,744][105620] Updated weights for policy 1, policy_version 555549 (0.0008) [2023-12-26 19:24:21,809][105620] Updated weights for policy 1, policy_version 555559 (0.0009) [2023-12-26 19:24:22,410][105692] Updated weights for policy 0, policy_version 554721 (0.0008) [2023-12-26 19:24:22,469][105692] Updated weights for policy 0, policy_version 554731 (0.0009) [2023-12-26 19:24:22,532][105692] Updated weights for policy 0, policy_version 554741 (0.0009) [2023-12-26 19:24:22,590][105692] Updated weights for policy 0, policy_version 554751 (0.0008) [2023-12-26 19:24:22,608][105620] Updated weights for policy 1, policy_version 555569 (0.0007) [2023-12-26 19:24:22,656][105620] Updated weights for policy 1, policy_version 555579 (0.0009) [2023-12-26 19:24:22,708][105620] Updated weights for policy 1, policy_version 555589 (0.0009) [2023-12-26 19:24:23,336][105692] Updated weights for policy 0, policy_version 554761 (0.0008) [2023-12-26 19:24:23,394][105692] Updated weights for policy 0, policy_version 554772 (0.0008) [2023-12-26 19:24:23,439][105620] Updated weights for policy 1, policy_version 555599 (0.0009) [2023-12-26 19:24:23,451][105692] Updated weights for policy 0, policy_version 554782 (0.0005) [2023-12-26 19:24:23,494][105620] Updated weights for policy 1, policy_version 555609 (0.0009) [2023-12-26 19:24:23,547][105620] Updated weights for policy 1, policy_version 555620 (0.0010) [2023-12-26 19:24:24,059][105692] Updated weights for policy 0, policy_version 554792 (0.0005) [2023-12-26 19:24:24,116][105692] Updated weights for policy 0, policy_version 554802 (0.0005) [2023-12-26 19:24:24,181][105692] Updated weights for policy 0, policy_version 554812 (0.0008) [2023-12-26 19:24:24,319][105620] Updated weights for policy 1, policy_version 555630 (0.0010) [2023-12-26 19:24:24,377][105620] Updated weights for policy 1, policy_version 555640 (0.0006) [2023-12-26 19:24:24,424][105620] Updated weights for policy 1, policy_version 555650 (0.0005) [2023-12-26 19:24:24,803][105692] Updated weights for policy 0, policy_version 554822 (0.0008) [2023-12-26 19:24:24,853][105692] Updated weights for policy 0, policy_version 554832 (0.0006) [2023-12-26 19:24:24,904][105692] Updated weights for policy 0, policy_version 554842 (0.0005) [2023-12-26 19:24:25,023][105620] Updated weights for policy 1, policy_version 555660 (0.0006) [2023-12-26 19:24:25,076][105620] Updated weights for policy 1, policy_version 555670 (0.0006) [2023-12-26 19:24:25,127][105620] Updated weights for policy 1, policy_version 555680 (0.0005) [2023-12-26 19:24:25,566][105692] Updated weights for policy 0, policy_version 554852 (0.0007) [2023-12-26 19:24:25,632][105692] Updated weights for policy 0, policy_version 554862 (0.0005) [2023-12-26 19:24:25,695][105692] Updated weights for policy 0, policy_version 554872 (0.0008) [2023-12-26 19:24:25,854][105620] Updated weights for policy 1, policy_version 555690 (0.0005) [2023-12-26 19:24:25,915][105620] Updated weights for policy 1, policy_version 555700 (0.0008) [2023-12-26 19:24:25,976][105620] Updated weights for policy 1, policy_version 555710 (0.0008) [2023-12-26 19:24:26,039][105620] Updated weights for policy 1, policy_version 555720 (0.0009) [2023-12-26 19:24:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 284344320. Throughput: 0: 9844.8, 1: 9676.4. Samples: 284349684. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-26 19:24:26,062][104569] Avg episode reward: [(0, '8904.164'), (1, '9073.243')] [2023-12-26 19:24:26,399][105692] Updated weights for policy 0, policy_version 554882 (0.0009) [2023-12-26 19:24:26,460][105692] Updated weights for policy 0, policy_version 554892 (0.0008) [2023-12-26 19:24:26,525][105692] Updated weights for policy 0, policy_version 554902 (0.0005) [2023-12-26 19:24:26,590][105692] Updated weights for policy 0, policy_version 554912 (0.0005) [2023-12-26 19:24:26,795][105620] Updated weights for policy 1, policy_version 555730 (0.0009) [2023-12-26 19:24:26,845][105620] Updated weights for policy 1, policy_version 555740 (0.0008) [2023-12-26 19:24:26,896][105620] Updated weights for policy 1, policy_version 555750 (0.0008) [2023-12-26 19:24:27,290][105692] Updated weights for policy 0, policy_version 554922 (0.0006) [2023-12-26 19:24:27,354][105692] Updated weights for policy 0, policy_version 554932 (0.0007) [2023-12-26 19:24:27,405][105692] Updated weights for policy 0, policy_version 554942 (0.0009) [2023-12-26 19:24:27,625][105620] Updated weights for policy 1, policy_version 555760 (0.0009) [2023-12-26 19:24:27,678][105620] Updated weights for policy 1, policy_version 555770 (0.0009) [2023-12-26 19:24:27,736][105620] Updated weights for policy 1, policy_version 555780 (0.0008) [2023-12-26 19:24:28,126][105692] Updated weights for policy 0, policy_version 554952 (0.0009) [2023-12-26 19:24:28,176][105692] Updated weights for policy 0, policy_version 554962 (0.0009) [2023-12-26 19:24:28,233][105692] Updated weights for policy 0, policy_version 554972 (0.0009) [2023-12-26 19:24:28,423][105620] Updated weights for policy 1, policy_version 555790 (0.0007) [2023-12-26 19:24:28,478][105620] Updated weights for policy 1, policy_version 555800 (0.0009) [2023-12-26 19:24:28,538][105620] Updated weights for policy 1, policy_version 555810 (0.0009) [2023-12-26 19:24:28,974][105692] Updated weights for policy 0, policy_version 554982 (0.0008) [2023-12-26 19:24:29,020][105692] Updated weights for policy 0, policy_version 554992 (0.0008) [2023-12-26 19:24:29,066][105692] Updated weights for policy 0, policy_version 555002 (0.0008) [2023-12-26 19:24:29,291][105620] Updated weights for policy 1, policy_version 555820 (0.0008) [2023-12-26 19:24:29,351][105620] Updated weights for policy 1, policy_version 555830 (0.0007) [2023-12-26 19:24:29,420][105620] Updated weights for policy 1, policy_version 555840 (0.0006) [2023-12-26 19:24:29,877][105692] Updated weights for policy 0, policy_version 555012 (0.0009) [2023-12-26 19:24:29,949][105692] Updated weights for policy 0, policy_version 555022 (0.0008) [2023-12-26 19:24:30,010][105692] Updated weights for policy 0, policy_version 555032 (0.0008) [2023-12-26 19:24:30,042][105620] Updated weights for policy 1, policy_version 555850 (0.0005) [2023-12-26 19:24:30,092][105620] Updated weights for policy 1, policy_version 555860 (0.0007) [2023-12-26 19:24:30,144][105620] Updated weights for policy 1, policy_version 555870 (0.0008) [2023-12-26 19:24:30,193][105620] Updated weights for policy 1, policy_version 555880 (0.0005) [2023-12-26 19:24:30,748][105692] Updated weights for policy 0, policy_version 555042 (0.0008) [2023-12-26 19:24:30,805][105692] Updated weights for policy 0, policy_version 555052 (0.0010) [2023-12-26 19:24:30,872][105692] Updated weights for policy 0, policy_version 555062 (0.0010) [2023-12-26 19:24:30,898][105620] Updated weights for policy 1, policy_version 555890 (0.0005) [2023-12-26 19:24:30,923][105692] Updated weights for policy 0, policy_version 555072 (0.0010) [2023-12-26 19:24:30,947][105620] Updated weights for policy 1, policy_version 555900 (0.0006) [2023-12-26 19:24:31,005][105620] Updated weights for policy 1, policy_version 555910 (0.0007) [2023-12-26 19:24:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 284442624. Throughput: 0: 9862.3, 1: 9694.3. Samples: 284406892. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-26 19:24:31,062][104569] Avg episode reward: [(0, '9263.572'), (1, '9165.995')] [2023-12-26 19:24:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000555072_142114816.pth... [2023-12-26 19:24:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000555912_142327808.pth... [2023-12-26 19:24:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000553920_141819904.pth [2023-12-26 19:24:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000554760_142032896.pth [2023-12-26 19:24:31,659][105692] Updated weights for policy 0, policy_version 555082 (0.0009) [2023-12-26 19:24:31,720][105692] Updated weights for policy 0, policy_version 555092 (0.0008) [2023-12-26 19:24:31,773][105620] Updated weights for policy 1, policy_version 555920 (0.0008) [2023-12-26 19:24:31,785][105692] Updated weights for policy 0, policy_version 555102 (0.0006) [2023-12-26 19:24:31,832][105620] Updated weights for policy 1, policy_version 555930 (0.0009) [2023-12-26 19:24:31,888][105620] Updated weights for policy 1, policy_version 555940 (0.0008) [2023-12-26 19:24:32,493][105692] Updated weights for policy 0, policy_version 555112 (0.0007) [2023-12-26 19:24:32,555][105692] Updated weights for policy 0, policy_version 555122 (0.0008) [2023-12-26 19:24:32,611][105692] Updated weights for policy 0, policy_version 555132 (0.0008) [2023-12-26 19:24:32,649][105620] Updated weights for policy 1, policy_version 555950 (0.0008) [2023-12-26 19:24:32,701][105620] Updated weights for policy 1, policy_version 555960 (0.0007) [2023-12-26 19:24:32,756][105620] Updated weights for policy 1, policy_version 555970 (0.0010) [2023-12-26 19:24:33,296][105692] Updated weights for policy 0, policy_version 555142 (0.0007) [2023-12-26 19:24:33,354][105692] Updated weights for policy 0, policy_version 555152 (0.0005) [2023-12-26 19:24:33,414][105692] Updated weights for policy 0, policy_version 555162 (0.0006) [2023-12-26 19:24:33,567][105620] Updated weights for policy 1, policy_version 555980 (0.0010) [2023-12-26 19:24:33,629][105620] Updated weights for policy 1, policy_version 555990 (0.0009) [2023-12-26 19:24:33,677][105620] Updated weights for policy 1, policy_version 556000 (0.0009) [2023-12-26 19:24:34,037][105692] Updated weights for policy 0, policy_version 555172 (0.0007) [2023-12-26 19:24:34,092][105692] Updated weights for policy 0, policy_version 555182 (0.0009) [2023-12-26 19:24:34,148][105692] Updated weights for policy 0, policy_version 555192 (0.0009) [2023-12-26 19:24:34,429][105620] Updated weights for policy 1, policy_version 556010 (0.0008) [2023-12-26 19:24:34,497][105620] Updated weights for policy 1, policy_version 556020 (0.0010) [2023-12-26 19:24:34,569][105620] Updated weights for policy 1, policy_version 556030 (0.0010) [2023-12-26 19:24:34,634][105620] Updated weights for policy 1, policy_version 556040 (0.0010) [2023-12-26 19:24:34,812][105692] Updated weights for policy 0, policy_version 555202 (0.0009) [2023-12-26 19:24:34,863][105692] Updated weights for policy 0, policy_version 555212 (0.0006) [2023-12-26 19:24:34,909][105692] Updated weights for policy 0, policy_version 555222 (0.0005) [2023-12-26 19:24:34,959][105692] Updated weights for policy 0, policy_version 555232 (0.0005) [2023-12-26 19:24:35,455][105620] Updated weights for policy 1, policy_version 556050 (0.0006) [2023-12-26 19:24:35,517][105620] Updated weights for policy 1, policy_version 556060 (0.0005) [2023-12-26 19:24:35,586][105620] Updated weights for policy 1, policy_version 556070 (0.0008) [2023-12-26 19:24:35,638][105692] Updated weights for policy 0, policy_version 555242 (0.0010) [2023-12-26 19:24:35,696][105692] Updated weights for policy 0, policy_version 555252 (0.0010) [2023-12-26 19:24:35,740][105692] Updated weights for policy 0, policy_version 555262 (0.0010) [2023-12-26 19:24:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 284532736. Throughput: 0: 9729.3, 1: 9688.4. Samples: 284521096. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-26 19:24:36,062][104569] Avg episode reward: [(0, '9267.777'), (1, '9352.135')] [2023-12-26 19:24:36,217][105620] Updated weights for policy 1, policy_version 556080 (0.0005) [2023-12-26 19:24:36,275][105620] Updated weights for policy 1, policy_version 556090 (0.0005) [2023-12-26 19:24:36,323][105620] Updated weights for policy 1, policy_version 556100 (0.0006) [2023-12-26 19:24:36,451][105692] Updated weights for policy 0, policy_version 555272 (0.0010) [2023-12-26 19:24:36,515][105692] Updated weights for policy 0, policy_version 555282 (0.0010) [2023-12-26 19:24:36,578][105692] Updated weights for policy 0, policy_version 555292 (0.0009) [2023-12-26 19:24:36,959][105620] Updated weights for policy 1, policy_version 556110 (0.0007) [2023-12-26 19:24:37,009][105620] Updated weights for policy 1, policy_version 556120 (0.0007) [2023-12-26 19:24:37,055][105620] Updated weights for policy 1, policy_version 556130 (0.0006) [2023-12-26 19:24:37,338][105692] Updated weights for policy 0, policy_version 555302 (0.0010) [2023-12-26 19:24:37,390][105692] Updated weights for policy 0, policy_version 555312 (0.0010) [2023-12-26 19:24:37,439][105692] Updated weights for policy 0, policy_version 555322 (0.0010) [2023-12-26 19:24:37,817][105620] Updated weights for policy 1, policy_version 556140 (0.0010) [2023-12-26 19:24:37,879][105620] Updated weights for policy 1, policy_version 556150 (0.0011) [2023-12-26 19:24:37,935][105620] Updated weights for policy 1, policy_version 556160 (0.0010) [2023-12-26 19:24:38,175][105692] Updated weights for policy 0, policy_version 555332 (0.0010) [2023-12-26 19:24:38,231][105692] Updated weights for policy 0, policy_version 555342 (0.0010) [2023-12-26 19:24:38,293][105692] Updated weights for policy 0, policy_version 555352 (0.0009) [2023-12-26 19:24:38,674][105620] Updated weights for policy 1, policy_version 556170 (0.0009) [2023-12-26 19:24:38,735][105620] Updated weights for policy 1, policy_version 556180 (0.0006) [2023-12-26 19:24:38,801][105620] Updated weights for policy 1, policy_version 556190 (0.0007) [2023-12-26 19:24:38,864][105620] Updated weights for policy 1, policy_version 556200 (0.0007) [2023-12-26 19:24:38,884][105692] Updated weights for policy 0, policy_version 555362 (0.0008) [2023-12-26 19:24:38,940][105692] Updated weights for policy 0, policy_version 555372 (0.0011) [2023-12-26 19:24:38,999][105692] Updated weights for policy 0, policy_version 555382 (0.0010) [2023-12-26 19:24:39,051][105692] Updated weights for policy 0, policy_version 555392 (0.0010) [2023-12-26 19:24:39,525][105620] Updated weights for policy 1, policy_version 556210 (0.0007) [2023-12-26 19:24:39,586][105620] Updated weights for policy 1, policy_version 556220 (0.0011) [2023-12-26 19:24:39,646][105620] Updated weights for policy 1, policy_version 556230 (0.0011) [2023-12-26 19:24:39,829][105692] Updated weights for policy 0, policy_version 555402 (0.0009) [2023-12-26 19:24:39,893][105692] Updated weights for policy 0, policy_version 555412 (0.0006) [2023-12-26 19:24:39,964][105692] Updated weights for policy 0, policy_version 555422 (0.0009) [2023-12-26 19:24:40,349][105620] Updated weights for policy 1, policy_version 556240 (0.0011) [2023-12-26 19:24:40,415][105620] Updated weights for policy 1, policy_version 556250 (0.0010) [2023-12-26 19:24:40,475][105620] Updated weights for policy 1, policy_version 556260 (0.0011) [2023-12-26 19:24:40,682][105692] Updated weights for policy 0, policy_version 555432 (0.0011) [2023-12-26 19:24:40,734][105692] Updated weights for policy 0, policy_version 555442 (0.0010) [2023-12-26 19:24:40,786][105692] Updated weights for policy 0, policy_version 555452 (0.0010) [2023-12-26 19:24:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 284631040. Throughput: 0: 9762.1, 1: 9718.1. Samples: 284640800. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-26 19:24:41,063][104569] Avg episode reward: [(0, '9083.125'), (1, '9353.684')] [2023-12-26 19:24:41,163][105620] Updated weights for policy 1, policy_version 556270 (0.0009) [2023-12-26 19:24:41,226][105620] Updated weights for policy 1, policy_version 556280 (0.0006) [2023-12-26 19:24:41,290][105620] Updated weights for policy 1, policy_version 556290 (0.0007) [2023-12-26 19:24:41,471][105692] Updated weights for policy 0, policy_version 555462 (0.0010) [2023-12-26 19:24:41,524][105692] Updated weights for policy 0, policy_version 555472 (0.0009) [2023-12-26 19:24:41,576][105692] Updated weights for policy 0, policy_version 555482 (0.0009) [2023-12-26 19:24:41,977][105620] Updated weights for policy 1, policy_version 556300 (0.0008) [2023-12-26 19:24:42,043][105620] Updated weights for policy 1, policy_version 556310 (0.0009) [2023-12-26 19:24:42,099][105620] Updated weights for policy 1, policy_version 556320 (0.0009) [2023-12-26 19:24:42,383][105692] Updated weights for policy 0, policy_version 555492 (0.0008) [2023-12-26 19:24:42,438][105692] Updated weights for policy 0, policy_version 555502 (0.0008) [2023-12-26 19:24:42,496][105692] Updated weights for policy 0, policy_version 555512 (0.0010) [2023-12-26 19:24:42,940][105620] Updated weights for policy 1, policy_version 556330 (0.0009) [2023-12-26 19:24:43,007][105620] Updated weights for policy 1, policy_version 556340 (0.0008) [2023-12-26 19:24:43,072][105620] Updated weights for policy 1, policy_version 556350 (0.0009) [2023-12-26 19:24:43,138][105620] Updated weights for policy 1, policy_version 556360 (0.0009) [2023-12-26 19:24:43,180][105692] Updated weights for policy 0, policy_version 555522 (0.0009) [2023-12-26 19:24:43,236][105692] Updated weights for policy 0, policy_version 555532 (0.0005) [2023-12-26 19:24:43,291][105692] Updated weights for policy 0, policy_version 555542 (0.0007) [2023-12-26 19:24:43,351][105692] Updated weights for policy 0, policy_version 555552 (0.0009) [2023-12-26 19:24:43,796][105620] Updated weights for policy 1, policy_version 556370 (0.0005) [2023-12-26 19:24:43,850][105620] Updated weights for policy 1, policy_version 556380 (0.0005) [2023-12-26 19:24:43,901][105620] Updated weights for policy 1, policy_version 556390 (0.0005) [2023-12-26 19:24:44,180][105692] Updated weights for policy 0, policy_version 555562 (0.0008) [2023-12-26 19:24:44,227][105692] Updated weights for policy 0, policy_version 555572 (0.0009) [2023-12-26 19:24:44,273][105692] Updated weights for policy 0, policy_version 555582 (0.0008) [2023-12-26 19:24:44,513][105620] Updated weights for policy 1, policy_version 556400 (0.0008) [2023-12-26 19:24:44,578][105620] Updated weights for policy 1, policy_version 556410 (0.0009) [2023-12-26 19:24:44,633][105620] Updated weights for policy 1, policy_version 556420 (0.0009) [2023-12-26 19:24:45,032][105692] Updated weights for policy 0, policy_version 555592 (0.0009) [2023-12-26 19:24:45,095][105692] Updated weights for policy 0, policy_version 555602 (0.0009) [2023-12-26 19:24:45,154][105692] Updated weights for policy 0, policy_version 555612 (0.0009) [2023-12-26 19:24:45,419][105620] Updated weights for policy 1, policy_version 556430 (0.0009) [2023-12-26 19:24:45,475][105620] Updated weights for policy 1, policy_version 556440 (0.0005) [2023-12-26 19:24:45,536][105620] Updated weights for policy 1, policy_version 556450 (0.0005) [2023-12-26 19:24:46,006][105692] Updated weights for policy 0, policy_version 555622 (0.0010) [2023-12-26 19:24:46,052][105692] Updated weights for policy 0, policy_version 555632 (0.0008) [2023-12-26 19:24:46,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 284721152. Throughput: 0: 9744.0, 1: 9752.5. Samples: 284697736. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-26 19:24:46,063][104569] Avg episode reward: [(0, '8993.286'), (1, '9353.904')] [2023-12-26 19:24:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000556456_142467072.pth... [2023-12-26 19:24:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000555336_142180352.pth [2023-12-26 19:24:46,106][105692] Updated weights for policy 0, policy_version 555642 (0.0009) [2023-12-26 19:24:46,140][105620] Updated weights for policy 1, policy_version 556460 (0.0007) [2023-12-26 19:24:46,141][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000555648_142262272.pth... [2023-12-26 19:24:46,145][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000554496_141967360.pth [2023-12-26 19:24:46,201][105620] Updated weights for policy 1, policy_version 556470 (0.0009) [2023-12-26 19:24:46,251][105620] Updated weights for policy 1, policy_version 556480 (0.0008) [2023-12-26 19:24:46,788][105692] Updated weights for policy 0, policy_version 555652 (0.0008) [2023-12-26 19:24:46,840][105692] Updated weights for policy 0, policy_version 555662 (0.0009) [2023-12-26 19:24:46,899][105692] Updated weights for policy 0, policy_version 555672 (0.0008) [2023-12-26 19:24:46,974][105620] Updated weights for policy 1, policy_version 556490 (0.0009) [2023-12-26 19:24:47,028][105620] Updated weights for policy 1, policy_version 556500 (0.0009) [2023-12-26 19:24:47,075][105620] Updated weights for policy 1, policy_version 556510 (0.0009) [2023-12-26 19:24:47,121][105620] Updated weights for policy 1, policy_version 556520 (0.0009) [2023-12-26 19:24:47,684][105692] Updated weights for policy 0, policy_version 555682 (0.0008) [2023-12-26 19:24:47,737][105692] Updated weights for policy 0, policy_version 555692 (0.0007) [2023-12-26 19:24:47,787][105692] Updated weights for policy 0, policy_version 555702 (0.0009) [2023-12-26 19:24:47,848][105692] Updated weights for policy 0, policy_version 555712 (0.0008) [2023-12-26 19:24:47,867][105620] Updated weights for policy 1, policy_version 556530 (0.0009) [2023-12-26 19:24:47,913][105620] Updated weights for policy 1, policy_version 556540 (0.0008) [2023-12-26 19:24:47,964][105620] Updated weights for policy 1, policy_version 556550 (0.0009) [2023-12-26 19:24:48,598][105585] KL-divergence is very high: 181.8052 [2023-12-26 19:24:48,610][105692] Updated weights for policy 0, policy_version 555722 (0.0011) [2023-12-26 19:24:48,652][105585] KL-divergence is very high: 240.1186 [2023-12-26 19:24:48,679][105692] Updated weights for policy 0, policy_version 555732 (0.0011) [2023-12-26 19:24:48,706][105585] KL-divergence is very high: 198.0874 [2023-12-26 19:24:48,708][105620] Updated weights for policy 1, policy_version 556560 (0.0010) [2023-12-26 19:24:48,743][105692] Updated weights for policy 0, policy_version 555742 (0.0011) [2023-12-26 19:24:48,772][105620] Updated weights for policy 1, policy_version 556570 (0.0011) [2023-12-26 19:24:48,835][105620] Updated weights for policy 1, policy_version 556580 (0.0011) [2023-12-26 19:24:49,459][105692] Updated weights for policy 0, policy_version 555752 (0.0009) [2023-12-26 19:24:49,519][105692] Updated weights for policy 0, policy_version 555762 (0.0009) [2023-12-26 19:24:49,538][105620] Updated weights for policy 1, policy_version 556590 (0.0010) [2023-12-26 19:24:49,581][105692] Updated weights for policy 0, policy_version 555772 (0.0006) [2023-12-26 19:24:49,600][105620] Updated weights for policy 1, policy_version 556600 (0.0010) [2023-12-26 19:24:49,666][105620] Updated weights for policy 1, policy_version 556610 (0.0008) [2023-12-26 19:24:50,366][105692] Updated weights for policy 0, policy_version 555782 (0.0008) [2023-12-26 19:24:50,422][105692] Updated weights for policy 0, policy_version 555792 (0.0008) [2023-12-26 19:24:50,422][105620] Updated weights for policy 1, policy_version 556620 (0.0010) [2023-12-26 19:24:50,483][105692] Updated weights for policy 0, policy_version 555802 (0.0010) [2023-12-26 19:24:50,486][105620] Updated weights for policy 1, policy_version 556630 (0.0011) [2023-12-26 19:24:50,543][105620] Updated weights for policy 1, policy_version 556640 (0.0011) [2023-12-26 19:24:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 284819456. Throughput: 0: 9602.1, 1: 9699.9. Samples: 284811320. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-26 19:24:51,062][104569] Avg episode reward: [(0, '8298.086'), (1, '9354.001')] [2023-12-26 19:24:51,225][105692] Updated weights for policy 0, policy_version 555812 (0.0009) [2023-12-26 19:24:51,237][105620] Updated weights for policy 1, policy_version 556650 (0.0010) [2023-12-26 19:24:51,283][105692] Updated weights for policy 0, policy_version 555822 (0.0007) [2023-12-26 19:24:51,302][105620] Updated weights for policy 1, policy_version 556660 (0.0007) [2023-12-26 19:24:51,350][105692] Updated weights for policy 0, policy_version 555832 (0.0006) [2023-12-26 19:24:51,372][105620] Updated weights for policy 1, policy_version 556671 (0.0009) [2023-12-26 19:24:51,949][105692] Updated weights for policy 0, policy_version 555842 (0.0007) [2023-12-26 19:24:52,011][105692] Updated weights for policy 0, policy_version 555852 (0.0005) [2023-12-26 19:24:52,072][105692] Updated weights for policy 0, policy_version 555862 (0.0007) [2023-12-26 19:24:52,138][105620] Updated weights for policy 1, policy_version 556681 (0.0010) [2023-12-26 19:24:52,143][105692] Updated weights for policy 0, policy_version 555872 (0.0008) [2023-12-26 19:24:52,200][105620] Updated weights for policy 1, policy_version 556691 (0.0010) [2023-12-26 19:24:52,269][105620] Updated weights for policy 1, policy_version 556701 (0.0009) [2023-12-26 19:24:52,327][105620] Updated weights for policy 1, policy_version 556711 (0.0009) [2023-12-26 19:24:52,782][105692] Updated weights for policy 0, policy_version 555882 (0.0009) [2023-12-26 19:24:52,844][105692] Updated weights for policy 0, policy_version 555892 (0.0009) [2023-12-26 19:24:52,895][105692] Updated weights for policy 0, policy_version 555902 (0.0009) [2023-12-26 19:24:53,097][105620] Updated weights for policy 1, policy_version 556721 (0.0009) [2023-12-26 19:24:53,157][105620] Updated weights for policy 1, policy_version 556731 (0.0007) [2023-12-26 19:24:53,215][105620] Updated weights for policy 1, policy_version 556741 (0.0005) [2023-12-26 19:24:53,692][105692] Updated weights for policy 0, policy_version 555912 (0.0009) [2023-12-26 19:24:53,750][105692] Updated weights for policy 0, policy_version 555922 (0.0009) [2023-12-26 19:24:53,799][105692] Updated weights for policy 0, policy_version 555932 (0.0009) [2023-12-26 19:24:53,843][105620] Updated weights for policy 1, policy_version 556751 (0.0005) [2023-12-26 19:24:53,899][105620] Updated weights for policy 1, policy_version 556761 (0.0005) [2023-12-26 19:24:53,960][105620] Updated weights for policy 1, policy_version 556771 (0.0006) [2023-12-26 19:24:54,492][105620] Updated weights for policy 1, policy_version 556781 (0.0007) [2023-12-26 19:24:54,510][105692] Updated weights for policy 0, policy_version 555942 (0.0005) [2023-12-26 19:24:54,546][105620] Updated weights for policy 1, policy_version 556791 (0.0010) [2023-12-26 19:24:54,560][105692] Updated weights for policy 0, policy_version 555952 (0.0005) [2023-12-26 19:24:54,587][105585] KL-divergence is very high: 107.7616 [2023-12-26 19:24:54,593][105585] KL-divergence is very high: 106.3523 [2023-12-26 19:24:54,599][105620] Updated weights for policy 1, policy_version 556801 (0.0008) [2023-12-26 19:24:54,620][105692] Updated weights for policy 0, policy_version 555962 (0.0007) [2023-12-26 19:24:54,637][105585] KL-divergence is very high: 146.5390 [2023-12-26 19:24:54,643][105585] KL-divergence is very high: 133.2382 [2023-12-26 19:24:55,243][105585] KL-divergence is very high: 141.8791 [2023-12-26 19:24:55,259][105692] Updated weights for policy 0, policy_version 555972 (0.0006) [2023-12-26 19:24:55,294][105585] KL-divergence is very high: 128.7259 [2023-12-26 19:24:55,323][105692] Updated weights for policy 0, policy_version 555982 (0.0006) [2023-12-26 19:24:55,338][105585] KL-divergence is very high: 110.1807 [2023-12-26 19:24:55,381][105692] Updated weights for policy 0, policy_version 555992 (0.0005) [2023-12-26 19:24:55,387][105585] KL-divergence is very high: 117.8708 [2023-12-26 19:24:55,410][105620] Updated weights for policy 1, policy_version 556811 (0.0007) [2023-12-26 19:24:55,467][105620] Updated weights for policy 1, policy_version 556821 (0.0007) [2023-12-26 19:24:55,531][105620] Updated weights for policy 1, policy_version 556831 (0.0005) [2023-12-26 19:24:56,001][105692] Updated weights for policy 0, policy_version 556002 (0.0006) [2023-12-26 19:24:56,055][105692] Updated weights for policy 0, policy_version 556012 (0.0008) [2023-12-26 19:24:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.9, 300 sec: 19494.2). Total num frames: 284917760. Throughput: 0: 9711.1, 1: 9721.3. Samples: 284930280. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-26 19:24:56,062][104569] Avg episode reward: [(0, '7733.047'), (1, '9354.307')] [2023-12-26 19:24:56,113][105692] Updated weights for policy 0, policy_version 556022 (0.0005) [2023-12-26 19:24:56,164][105692] Updated weights for policy 0, policy_version 556032 (0.0005) [2023-12-26 19:24:56,279][105620] Updated weights for policy 1, policy_version 556841 (0.0008) [2023-12-26 19:24:56,343][105620] Updated weights for policy 1, policy_version 556851 (0.0006) [2023-12-26 19:24:56,396][105620] Updated weights for policy 1, policy_version 556861 (0.0007) [2023-12-26 19:24:56,455][105620] Updated weights for policy 1, policy_version 556871 (0.0009) [2023-12-26 19:24:56,766][105692] Updated weights for policy 0, policy_version 556042 (0.0007) [2023-12-26 19:24:56,821][105692] Updated weights for policy 0, policy_version 556052 (0.0010) [2023-12-26 19:24:56,865][105692] Updated weights for policy 0, policy_version 556062 (0.0010) [2023-12-26 19:24:57,232][105620] Updated weights for policy 1, policy_version 556881 (0.0008) [2023-12-26 19:24:57,279][105620] Updated weights for policy 1, policy_version 556891 (0.0008) [2023-12-26 19:24:57,332][105620] Updated weights for policy 1, policy_version 556901 (0.0007) [2023-12-26 19:24:57,586][105692] Updated weights for policy 0, policy_version 556072 (0.0010) [2023-12-26 19:24:57,643][105692] Updated weights for policy 0, policy_version 556082 (0.0010) [2023-12-26 19:24:57,700][105692] Updated weights for policy 0, policy_version 556092 (0.0010) [2023-12-26 19:24:57,979][105620] Updated weights for policy 1, policy_version 556911 (0.0007) [2023-12-26 19:24:58,031][105620] Updated weights for policy 1, policy_version 556921 (0.0008) [2023-12-26 19:24:58,086][105620] Updated weights for policy 1, policy_version 556931 (0.0008) [2023-12-26 19:24:58,455][105692] Updated weights for policy 0, policy_version 556102 (0.0009) [2023-12-26 19:24:58,519][105692] Updated weights for policy 0, policy_version 556112 (0.0011) [2023-12-26 19:24:58,587][105692] Updated weights for policy 0, policy_version 556122 (0.0010) [2023-12-26 19:24:58,899][105620] Updated weights for policy 1, policy_version 556941 (0.0008) [2023-12-26 19:24:58,962][105620] Updated weights for policy 1, policy_version 556951 (0.0006) [2023-12-26 19:24:59,023][105620] Updated weights for policy 1, policy_version 556961 (0.0008) [2023-12-26 19:24:59,406][105692] Updated weights for policy 0, policy_version 556132 (0.0010) [2023-12-26 19:24:59,469][105692] Updated weights for policy 0, policy_version 556142 (0.0010) [2023-12-26 19:24:59,521][105692] Updated weights for policy 0, policy_version 556152 (0.0009) [2023-12-26 19:24:59,689][105620] Updated weights for policy 1, policy_version 556971 (0.0007) [2023-12-26 19:24:59,737][105620] Updated weights for policy 1, policy_version 556981 (0.0006) [2023-12-26 19:24:59,784][105620] Updated weights for policy 1, policy_version 556991 (0.0006) [2023-12-26 19:25:00,255][105692] Updated weights for policy 0, policy_version 556162 (0.0010) [2023-12-26 19:25:00,327][105692] Updated weights for policy 0, policy_version 556172 (0.0007) [2023-12-26 19:25:00,387][105692] Updated weights for policy 0, policy_version 556182 (0.0010) [2023-12-26 19:25:00,442][105692] Updated weights for policy 0, policy_version 556192 (0.0009) [2023-12-26 19:25:00,496][105620] Updated weights for policy 1, policy_version 557001 (0.0008) [2023-12-26 19:25:00,549][105620] Updated weights for policy 1, policy_version 557011 (0.0009) [2023-12-26 19:25:00,602][105620] Updated weights for policy 1, policy_version 557021 (0.0008) [2023-12-26 19:25:00,651][105620] Updated weights for policy 1, policy_version 557031 (0.0009) [2023-12-26 19:25:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 285016064. Throughput: 0: 9754.1, 1: 9679.8. Samples: 284988076. Policy #0 lag: (min: 5.0, avg: 12.5, max: 37.0) [2023-12-26 19:25:01,063][104569] Avg episode reward: [(0, '7982.874'), (1, '9354.785')] [2023-12-26 19:25:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000557032_142614528.pth... [2023-12-26 19:25:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000556192_142401536.pth... [2023-12-26 19:25:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000555912_142327808.pth [2023-12-26 19:25:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000555072_142114816.pth [2023-12-26 19:25:01,161][105692] Updated weights for policy 0, policy_version 556202 (0.0007) [2023-12-26 19:25:01,221][105692] Updated weights for policy 0, policy_version 556212 (0.0009) [2023-12-26 19:25:01,274][105692] Updated weights for policy 0, policy_version 556222 (0.0009) [2023-12-26 19:25:01,369][105620] Updated weights for policy 1, policy_version 557041 (0.0008) [2023-12-26 19:25:01,432][105620] Updated weights for policy 1, policy_version 557051 (0.0007) [2023-12-26 19:25:01,489][105620] Updated weights for policy 1, policy_version 557061 (0.0006) [2023-12-26 19:25:02,105][105620] Updated weights for policy 1, policy_version 557071 (0.0010) [2023-12-26 19:25:02,110][105692] Updated weights for policy 0, policy_version 556232 (0.0010) [2023-12-26 19:25:02,157][105620] Updated weights for policy 1, policy_version 557081 (0.0010) [2023-12-26 19:25:02,159][105692] Updated weights for policy 0, policy_version 556242 (0.0006) [2023-12-26 19:25:02,206][105692] Updated weights for policy 0, policy_version 556252 (0.0008) [2023-12-26 19:25:02,211][105620] Updated weights for policy 1, policy_version 557091 (0.0010) [2023-12-26 19:25:02,883][105620] Updated weights for policy 1, policy_version 557101 (0.0008) [2023-12-26 19:25:02,943][105620] Updated weights for policy 1, policy_version 557111 (0.0007) [2023-12-26 19:25:03,000][105620] Updated weights for policy 1, policy_version 557121 (0.0010) [2023-12-26 19:25:03,030][105692] Updated weights for policy 0, policy_version 556262 (0.0005) [2023-12-26 19:25:03,081][105692] Updated weights for policy 0, policy_version 556272 (0.0007) [2023-12-26 19:25:03,134][105692] Updated weights for policy 0, policy_version 556282 (0.0008) [2023-12-26 19:25:03,566][105620] Updated weights for policy 1, policy_version 557131 (0.0009) [2023-12-26 19:25:03,613][105620] Updated weights for policy 1, policy_version 557141 (0.0005) [2023-12-26 19:25:03,665][105620] Updated weights for policy 1, policy_version 557151 (0.0006) [2023-12-26 19:25:03,840][105692] Updated weights for policy 0, policy_version 556293 (0.0008) [2023-12-26 19:25:03,904][105692] Updated weights for policy 0, policy_version 556303 (0.0007) [2023-12-26 19:25:03,967][105692] Updated weights for policy 0, policy_version 556313 (0.0010) [2023-12-26 19:25:04,307][105620] Updated weights for policy 1, policy_version 557161 (0.0007) [2023-12-26 19:25:04,375][105620] Updated weights for policy 1, policy_version 557171 (0.0006) [2023-12-26 19:25:04,445][105620] Updated weights for policy 1, policy_version 557181 (0.0006) [2023-12-26 19:25:04,516][105620] Updated weights for policy 1, policy_version 557191 (0.0006) [2023-12-26 19:25:04,734][105692] Updated weights for policy 0, policy_version 556323 (0.0010) [2023-12-26 19:25:04,800][105692] Updated weights for policy 0, policy_version 556333 (0.0009) [2023-12-26 19:25:04,862][105692] Updated weights for policy 0, policy_version 556343 (0.0009) [2023-12-26 19:25:05,079][105620] Updated weights for policy 1, policy_version 557201 (0.0010) [2023-12-26 19:25:05,132][105620] Updated weights for policy 1, policy_version 557211 (0.0011) [2023-12-26 19:25:05,187][105620] Updated weights for policy 1, policy_version 557221 (0.0010) [2023-12-26 19:25:05,633][105692] Updated weights for policy 0, policy_version 556353 (0.0008) [2023-12-26 19:25:05,681][105692] Updated weights for policy 0, policy_version 556363 (0.0008) [2023-12-26 19:25:05,736][105692] Updated weights for policy 0, policy_version 556374 (0.0009) [2023-12-26 19:25:05,800][105692] Updated weights for policy 0, policy_version 556384 (0.0010) [2023-12-26 19:25:05,905][105620] Updated weights for policy 1, policy_version 557231 (0.0011) [2023-12-26 19:25:05,961][105620] Updated weights for policy 1, policy_version 557241 (0.0010) [2023-12-26 19:25:06,013][105620] Updated weights for policy 1, policy_version 557251 (0.0010) [2023-12-26 19:25:06,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 285122560. Throughput: 0: 9664.2, 1: 9794.4. Samples: 285107716. Policy #0 lag: (min: 5.0, avg: 12.5, max: 37.0) [2023-12-26 19:25:06,062][104569] Avg episode reward: [(0, '8136.112'), (1, '9264.656')] [2023-12-26 19:25:06,520][105692] Updated weights for policy 0, policy_version 556394 (0.0007) [2023-12-26 19:25:06,579][105692] Updated weights for policy 0, policy_version 556404 (0.0010) [2023-12-26 19:25:06,638][105692] Updated weights for policy 0, policy_version 556414 (0.0010) [2023-12-26 19:25:06,752][105620] Updated weights for policy 1, policy_version 557261 (0.0010) [2023-12-26 19:25:06,815][105620] Updated weights for policy 1, policy_version 557271 (0.0011) [2023-12-26 19:25:06,868][105620] Updated weights for policy 1, policy_version 557281 (0.0011) [2023-12-26 19:25:07,370][105692] Updated weights for policy 0, policy_version 556424 (0.0011) [2023-12-26 19:25:07,433][105692] Updated weights for policy 0, policy_version 556434 (0.0011) [2023-12-26 19:25:07,491][105692] Updated weights for policy 0, policy_version 556444 (0.0010) [2023-12-26 19:25:07,622][105620] Updated weights for policy 1, policy_version 557291 (0.0009) [2023-12-26 19:25:07,691][105620] Updated weights for policy 1, policy_version 557301 (0.0006) [2023-12-26 19:25:07,763][105620] Updated weights for policy 1, policy_version 557311 (0.0006) [2023-12-26 19:25:08,137][105692] Updated weights for policy 0, policy_version 556454 (0.0008) [2023-12-26 19:25:08,195][105692] Updated weights for policy 0, policy_version 556464 (0.0005) [2023-12-26 19:25:08,242][105692] Updated weights for policy 0, policy_version 556474 (0.0005) [2023-12-26 19:25:08,323][105620] Updated weights for policy 1, policy_version 557321 (0.0006) [2023-12-26 19:25:08,388][105620] Updated weights for policy 1, policy_version 557331 (0.0006) [2023-12-26 19:25:08,449][105620] Updated weights for policy 1, policy_version 557341 (0.0006) [2023-12-26 19:25:08,513][105620] Updated weights for policy 1, policy_version 557351 (0.0006) [2023-12-26 19:25:08,854][105692] Updated weights for policy 0, policy_version 556484 (0.0005) [2023-12-26 19:25:08,918][105692] Updated weights for policy 0, policy_version 556494 (0.0005) [2023-12-26 19:25:08,988][105692] Updated weights for policy 0, policy_version 556504 (0.0006) [2023-12-26 19:25:09,136][105620] Updated weights for policy 1, policy_version 557361 (0.0010) [2023-12-26 19:25:09,196][105620] Updated weights for policy 1, policy_version 557371 (0.0009) [2023-12-26 19:25:09,257][105620] Updated weights for policy 1, policy_version 557381 (0.0009) [2023-12-26 19:25:09,541][105692] Updated weights for policy 0, policy_version 556514 (0.0005) [2023-12-26 19:25:09,602][105692] Updated weights for policy 0, policy_version 556524 (0.0010) [2023-12-26 19:25:09,651][105585] KL-divergence is very high: 135.7150 [2023-12-26 19:25:09,665][105692] Updated weights for policy 0, policy_version 556534 (0.0010) [2023-12-26 19:25:09,705][105585] KL-divergence is very high: 126.0367 [2023-12-26 19:25:09,730][105692] Updated weights for policy 0, policy_version 556544 (0.0011) [2023-12-26 19:25:10,088][105620] Updated weights for policy 1, policy_version 557392 (0.0009) [2023-12-26 19:25:10,150][105620] Updated weights for policy 1, policy_version 557402 (0.0009) [2023-12-26 19:25:10,215][105620] Updated weights for policy 1, policy_version 557412 (0.0009) [2023-12-26 19:25:10,501][105692] Updated weights for policy 0, policy_version 556554 (0.0010) [2023-12-26 19:25:10,564][105692] Updated weights for policy 0, policy_version 556564 (0.0010) [2023-12-26 19:25:10,627][105692] Updated weights for policy 0, policy_version 556574 (0.0011) [2023-12-26 19:25:10,902][105620] Updated weights for policy 1, policy_version 557422 (0.0009) [2023-12-26 19:25:10,959][105620] Updated weights for policy 1, policy_version 557432 (0.0010) [2023-12-26 19:25:11,012][105620] Updated weights for policy 1, policy_version 557442 (0.0010) [2023-12-26 19:25:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 285220864. Throughput: 0: 9663.6, 1: 9816.8. Samples: 285226304. Policy #0 lag: (min: 5.0, avg: 12.5, max: 37.0) [2023-12-26 19:25:11,063][104569] Avg episode reward: [(0, '8725.560'), (1, '9264.877')] [2023-12-26 19:25:11,285][105692] Updated weights for policy 0, policy_version 556584 (0.0010) [2023-12-26 19:25:11,345][105692] Updated weights for policy 0, policy_version 556594 (0.0012) [2023-12-26 19:25:11,408][105692] Updated weights for policy 0, policy_version 556604 (0.0007) [2023-12-26 19:25:11,839][105620] Updated weights for policy 1, policy_version 557452 (0.0010) [2023-12-26 19:25:11,901][105620] Updated weights for policy 1, policy_version 557462 (0.0010) [2023-12-26 19:25:11,961][105620] Updated weights for policy 1, policy_version 557472 (0.0010) [2023-12-26 19:25:12,089][105692] Updated weights for policy 0, policy_version 556614 (0.0008) [2023-12-26 19:25:12,139][105692] Updated weights for policy 0, policy_version 556624 (0.0008) [2023-12-26 19:25:12,202][105692] Updated weights for policy 0, policy_version 556634 (0.0006) [2023-12-26 19:25:12,758][105620] Updated weights for policy 1, policy_version 557482 (0.0009) [2023-12-26 19:25:12,817][105620] Updated weights for policy 1, policy_version 557492 (0.0006) [2023-12-26 19:25:12,852][105692] Updated weights for policy 0, policy_version 556644 (0.0006) [2023-12-26 19:25:12,868][105620] Updated weights for policy 1, policy_version 557502 (0.0006) [2023-12-26 19:25:12,912][105692] Updated weights for policy 0, policy_version 556654 (0.0008) [2023-12-26 19:25:12,923][105620] Updated weights for policy 1, policy_version 557512 (0.0008) [2023-12-26 19:25:12,975][105692] Updated weights for policy 0, policy_version 556664 (0.0009) [2023-12-26 19:25:13,599][105620] Updated weights for policy 1, policy_version 557522 (0.0005) [2023-12-26 19:25:13,620][105692] Updated weights for policy 0, policy_version 556674 (0.0009) [2023-12-26 19:25:13,671][105692] Updated weights for policy 0, policy_version 556684 (0.0007) [2023-12-26 19:25:13,680][105620] Updated weights for policy 1, policy_version 557532 (0.0009) [2023-12-26 19:25:13,720][105692] Updated weights for policy 0, policy_version 556694 (0.0007) [2023-12-26 19:25:13,726][105620] Updated weights for policy 1, policy_version 557542 (0.0007) [2023-12-26 19:25:13,776][105692] Updated weights for policy 0, policy_version 556704 (0.0008) [2023-12-26 19:25:14,472][105620] Updated weights for policy 1, policy_version 557552 (0.0008) [2023-12-26 19:25:14,504][105692] Updated weights for policy 0, policy_version 556714 (0.0006) [2023-12-26 19:25:14,527][105620] Updated weights for policy 1, policy_version 557562 (0.0007) [2023-12-26 19:25:14,555][105692] Updated weights for policy 0, policy_version 556724 (0.0007) [2023-12-26 19:25:14,581][105620] Updated weights for policy 1, policy_version 557572 (0.0005) [2023-12-26 19:25:14,614][105692] Updated weights for policy 0, policy_version 556734 (0.0005) [2023-12-26 19:25:15,228][105692] Updated weights for policy 0, policy_version 556744 (0.0009) [2023-12-26 19:25:15,242][105620] Updated weights for policy 1, policy_version 557582 (0.0006) [2023-12-26 19:25:15,285][105692] Updated weights for policy 0, policy_version 556754 (0.0011) [2023-12-26 19:25:15,310][105620] Updated weights for policy 1, policy_version 557592 (0.0006) [2023-12-26 19:25:15,343][105692] Updated weights for policy 0, policy_version 556764 (0.0011) [2023-12-26 19:25:15,373][105620] Updated weights for policy 1, policy_version 557602 (0.0006) [2023-12-26 19:25:15,977][105620] Updated weights for policy 1, policy_version 557612 (0.0008) [2023-12-26 19:25:16,046][105620] Updated weights for policy 1, policy_version 557622 (0.0008) [2023-12-26 19:25:16,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 285310976. Throughput: 0: 9725.4, 1: 9796.7. Samples: 285285388. Policy #0 lag: (min: 5.0, avg: 12.5, max: 37.0) [2023-12-26 19:25:16,063][104569] Avg episode reward: [(0, '8993.955'), (1, '9355.523')] [2023-12-26 19:25:16,098][105620] Updated weights for policy 1, policy_version 557632 (0.0011) [2023-12-26 19:25:16,102][105692] Updated weights for policy 0, policy_version 556774 (0.0008) [2023-12-26 19:25:16,142][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000557640_142770176.pth... [2023-12-26 19:25:16,146][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000556456_142467072.pth [2023-12-26 19:25:16,162][105692] Updated weights for policy 0, policy_version 556784 (0.0009) [2023-12-26 19:25:16,220][105692] Updated weights for policy 0, policy_version 556794 (0.0010) [2023-12-26 19:25:16,257][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000556800_142557184.pth... [2023-12-26 19:25:16,262][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000555648_142262272.pth [2023-12-26 19:25:16,777][105620] Updated weights for policy 1, policy_version 557642 (0.0011) [2023-12-26 19:25:16,807][105692] Updated weights for policy 0, policy_version 556804 (0.0010) [2023-12-26 19:25:16,829][105620] Updated weights for policy 1, policy_version 557652 (0.0010) [2023-12-26 19:25:16,861][105692] Updated weights for policy 0, policy_version 556814 (0.0010) [2023-12-26 19:25:16,888][105620] Updated weights for policy 1, policy_version 557662 (0.0010) [2023-12-26 19:25:16,920][105692] Updated weights for policy 0, policy_version 556824 (0.0010) [2023-12-26 19:25:16,944][105620] Updated weights for policy 1, policy_version 557672 (0.0010) [2023-12-26 19:25:17,552][105692] Updated weights for policy 0, policy_version 556834 (0.0009) [2023-12-26 19:25:17,562][105620] Updated weights for policy 1, policy_version 557682 (0.0010) [2023-12-26 19:25:17,600][105692] Updated weights for policy 0, policy_version 556844 (0.0005) [2023-12-26 19:25:17,610][105620] Updated weights for policy 1, policy_version 557692 (0.0010) [2023-12-26 19:25:17,650][105692] Updated weights for policy 0, policy_version 556854 (0.0006) [2023-12-26 19:25:17,658][105620] Updated weights for policy 1, policy_version 557702 (0.0010) [2023-12-26 19:25:17,701][105692] Updated weights for policy 0, policy_version 556864 (0.0005) [2023-12-26 19:25:18,347][105692] Updated weights for policy 0, policy_version 556874 (0.0010) [2023-12-26 19:25:18,406][105692] Updated weights for policy 0, policy_version 556884 (0.0009) [2023-12-26 19:25:18,455][105620] Updated weights for policy 1, policy_version 557712 (0.0011) [2023-12-26 19:25:18,463][105692] Updated weights for policy 0, policy_version 556894 (0.0009) [2023-12-26 19:25:18,511][105620] Updated weights for policy 1, policy_version 557722 (0.0011) [2023-12-26 19:25:18,564][105620] Updated weights for policy 1, policy_version 557732 (0.0008) [2023-12-26 19:25:19,202][105692] Updated weights for policy 0, policy_version 556904 (0.0010) [2023-12-26 19:25:19,250][105620] Updated weights for policy 1, policy_version 557742 (0.0009) [2023-12-26 19:25:19,268][105692] Updated weights for policy 0, policy_version 556914 (0.0010) [2023-12-26 19:25:19,309][105620] Updated weights for policy 1, policy_version 557752 (0.0010) [2023-12-26 19:25:19,329][105692] Updated weights for policy 0, policy_version 556924 (0.0009) [2023-12-26 19:25:19,375][105620] Updated weights for policy 1, policy_version 557762 (0.0011) [2023-12-26 19:25:20,091][105692] Updated weights for policy 0, policy_version 556934 (0.0008) [2023-12-26 19:25:20,158][105692] Updated weights for policy 0, policy_version 556944 (0.0008) [2023-12-26 19:25:20,162][105620] Updated weights for policy 1, policy_version 557772 (0.0010) [2023-12-26 19:25:20,219][105692] Updated weights for policy 0, policy_version 556954 (0.0007) [2023-12-26 19:25:20,221][105620] Updated weights for policy 1, policy_version 557782 (0.0009) [2023-12-26 19:25:20,284][105620] Updated weights for policy 1, policy_version 557793 (0.0009) [2023-12-26 19:25:20,989][105692] Updated weights for policy 0, policy_version 556964 (0.0008) [2023-12-26 19:25:21,051][105620] Updated weights for policy 1, policy_version 557803 (0.0008) [2023-12-26 19:25:21,053][105692] Updated weights for policy 0, policy_version 556974 (0.0009) [2023-12-26 19:25:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 285409280. Throughput: 0: 9796.8, 1: 9887.9. Samples: 285406908. Policy #0 lag: (min: 5.0, avg: 12.5, max: 37.0) [2023-12-26 19:25:21,062][104569] Avg episode reward: [(0, '9078.649'), (1, '9355.505')] [2023-12-26 19:25:21,102][105692] Updated weights for policy 0, policy_version 556984 (0.0008) [2023-12-26 19:25:21,118][105620] Updated weights for policy 1, policy_version 557813 (0.0009) [2023-12-26 19:25:21,181][105620] Updated weights for policy 1, policy_version 557823 (0.0008) [2023-12-26 19:25:21,849][105692] Updated weights for policy 0, policy_version 556994 (0.0009) [2023-12-26 19:25:21,901][105692] Updated weights for policy 0, policy_version 557004 (0.0009) [2023-12-26 19:25:21,966][105692] Updated weights for policy 0, policy_version 557014 (0.0009) [2023-12-26 19:25:21,970][105620] Updated weights for policy 1, policy_version 557833 (0.0008) [2023-12-26 19:25:22,030][105692] Updated weights for policy 0, policy_version 557024 (0.0007) [2023-12-26 19:25:22,032][105620] Updated weights for policy 1, policy_version 557843 (0.0009) [2023-12-26 19:25:22,094][105620] Updated weights for policy 1, policy_version 557853 (0.0009) [2023-12-26 19:25:22,156][105620] Updated weights for policy 1, policy_version 557863 (0.0009) [2023-12-26 19:25:22,817][105585] KL-divergence is very high: 160.1197 [2023-12-26 19:25:22,826][105692] Updated weights for policy 0, policy_version 557034 (0.0007) [2023-12-26 19:25:22,852][105620] Updated weights for policy 1, policy_version 557873 (0.0009) [2023-12-26 19:25:22,858][105585] KL-divergence is very high: 272.0897 [2023-12-26 19:25:22,879][105692] Updated weights for policy 0, policy_version 557044 (0.0006) [2023-12-26 19:25:22,899][105620] Updated weights for policy 1, policy_version 557883 (0.0006) [2023-12-26 19:25:22,901][105585] KL-divergence is very high: 258.0133 [2023-12-26 19:25:22,936][105692] Updated weights for policy 0, policy_version 557054 (0.0008) [2023-12-26 19:25:22,949][105620] Updated weights for policy 1, policy_version 557893 (0.0008) [2023-12-26 19:25:23,644][105692] Updated weights for policy 0, policy_version 557064 (0.0007) [2023-12-26 19:25:23,691][105692] Updated weights for policy 0, policy_version 557074 (0.0009) [2023-12-26 19:25:23,736][105620] Updated weights for policy 1, policy_version 557903 (0.0008) [2023-12-26 19:25:23,745][105692] Updated weights for policy 0, policy_version 557084 (0.0007) [2023-12-26 19:25:23,782][105620] Updated weights for policy 1, policy_version 557913 (0.0007) [2023-12-26 19:25:23,829][105620] Updated weights for policy 1, policy_version 557923 (0.0009) [2023-12-26 19:25:24,501][105620] Updated weights for policy 1, policy_version 557933 (0.0006) [2023-12-26 19:25:24,523][105692] Updated weights for policy 0, policy_version 557094 (0.0009) [2023-12-26 19:25:24,564][105620] Updated weights for policy 1, policy_version 557943 (0.0006) [2023-12-26 19:25:24,587][105692] Updated weights for policy 0, policy_version 557104 (0.0007) [2023-12-26 19:25:24,621][105620] Updated weights for policy 1, policy_version 557953 (0.0008) [2023-12-26 19:25:24,644][105692] Updated weights for policy 0, policy_version 557114 (0.0007) [2023-12-26 19:25:25,262][105620] Updated weights for policy 1, policy_version 557963 (0.0009) [2023-12-26 19:25:25,321][105620] Updated weights for policy 1, policy_version 557973 (0.0010) [2023-12-26 19:25:25,379][105620] Updated weights for policy 1, policy_version 557983 (0.0010) [2023-12-26 19:25:25,405][105692] Updated weights for policy 0, policy_version 557124 (0.0007) [2023-12-26 19:25:25,461][105692] Updated weights for policy 0, policy_version 557134 (0.0007) [2023-12-26 19:25:25,515][105692] Updated weights for policy 0, policy_version 557144 (0.0007) [2023-12-26 19:25:26,035][105620] Updated weights for policy 1, policy_version 557993 (0.0010) [2023-12-26 19:25:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 285507584. Throughput: 0: 9695.9, 1: 9838.0. Samples: 285519828. Policy #0 lag: (min: 5.0, avg: 12.5, max: 37.0) [2023-12-26 19:25:26,062][104569] Avg episode reward: [(0, '8806.562'), (1, '9355.452')] [2023-12-26 19:25:26,094][105620] Updated weights for policy 1, policy_version 558003 (0.0005) [2023-12-26 19:25:26,153][105620] Updated weights for policy 1, policy_version 558013 (0.0005) [2023-12-26 19:25:26,207][105620] Updated weights for policy 1, policy_version 558023 (0.0005) [2023-12-26 19:25:26,354][105692] Updated weights for policy 0, policy_version 557154 (0.0008) [2023-12-26 19:25:26,416][105692] Updated weights for policy 0, policy_version 557164 (0.0009) [2023-12-26 19:25:26,472][105692] Updated weights for policy 0, policy_version 557174 (0.0009) [2023-12-26 19:25:26,521][105692] Updated weights for policy 0, policy_version 557184 (0.0008) [2023-12-26 19:25:26,861][105620] Updated weights for policy 1, policy_version 558033 (0.0007) [2023-12-26 19:25:26,914][105620] Updated weights for policy 1, policy_version 558043 (0.0009) [2023-12-26 19:25:26,970][105620] Updated weights for policy 1, policy_version 558053 (0.0008) [2023-12-26 19:25:27,272][105692] Updated weights for policy 0, policy_version 557194 (0.0009) [2023-12-26 19:25:27,324][105692] Updated weights for policy 0, policy_version 557204 (0.0007) [2023-12-26 19:25:27,369][105692] Updated weights for policy 0, policy_version 557214 (0.0006) [2023-12-26 19:25:27,758][105620] Updated weights for policy 1, policy_version 558063 (0.0008) [2023-12-26 19:25:27,815][105620] Updated weights for policy 1, policy_version 558073 (0.0008) [2023-12-26 19:25:27,885][105620] Updated weights for policy 1, policy_version 558083 (0.0009) [2023-12-26 19:25:28,034][105692] Updated weights for policy 0, policy_version 557224 (0.0007) [2023-12-26 19:25:28,054][105585] KL-divergence is very high: 206.3630 [2023-12-26 19:25:28,058][105585] KL-divergence is very high: 380.5399 [2023-12-26 19:25:28,090][105585] KL-divergence is very high: 104.2593 [2023-12-26 19:25:28,095][105692] Updated weights for policy 0, policy_version 557234 (0.0009) [2023-12-26 19:25:28,101][105585] KL-divergence is very high: 343.8027 [2023-12-26 19:25:28,107][105585] KL-divergence is very high: 587.4572 [2023-12-26 19:25:28,152][105585] KL-divergence is very high: 350.1061 [2023-12-26 19:25:28,156][105692] Updated weights for policy 0, policy_version 557244 (0.0009) [2023-12-26 19:25:28,158][105585] KL-divergence is very high: 576.2502 [2023-12-26 19:25:28,613][105620] Updated weights for policy 1, policy_version 558093 (0.0009) [2023-12-26 19:25:28,677][105620] Updated weights for policy 1, policy_version 558103 (0.0009) [2023-12-26 19:25:28,738][105620] Updated weights for policy 1, policy_version 558113 (0.0009) [2023-12-26 19:25:28,892][105585] KL-divergence is very high: 330.0836 [2023-12-26 19:25:28,900][105692] Updated weights for policy 0, policy_version 557254 (0.0009) [2023-12-26 19:25:28,939][105585] KL-divergence is very high: 282.2845 [2023-12-26 19:25:28,958][105692] Updated weights for policy 0, policy_version 557264 (0.0009) [2023-12-26 19:25:28,981][105585] KL-divergence is very high: 272.8702 [2023-12-26 19:25:29,009][105692] Updated weights for policy 0, policy_version 557274 (0.0009) [2023-12-26 19:25:29,027][105585] KL-divergence is very high: 257.9777 [2023-12-26 19:25:29,484][105620] Updated weights for policy 1, policy_version 558123 (0.0009) [2023-12-26 19:25:29,539][105620] Updated weights for policy 1, policy_version 558133 (0.0010) [2023-12-26 19:25:29,608][105620] Updated weights for policy 1, policy_version 558143 (0.0005) [2023-12-26 19:25:29,745][105692] Updated weights for policy 0, policy_version 557284 (0.0009) [2023-12-26 19:25:29,799][105692] Updated weights for policy 0, policy_version 557294 (0.0010) [2023-12-26 19:25:29,855][105692] Updated weights for policy 0, policy_version 557304 (0.0007) [2023-12-26 19:25:30,277][105620] Updated weights for policy 1, policy_version 558153 (0.0005) [2023-12-26 19:25:30,329][105620] Updated weights for policy 1, policy_version 558163 (0.0006) [2023-12-26 19:25:30,390][105620] Updated weights for policy 1, policy_version 558173 (0.0008) [2023-12-26 19:25:30,443][105620] Updated weights for policy 1, policy_version 558183 (0.0010) [2023-12-26 19:25:30,663][105692] Updated weights for policy 0, policy_version 557314 (0.0009) [2023-12-26 19:25:30,721][105692] Updated weights for policy 0, policy_version 557324 (0.0008) [2023-12-26 19:25:30,779][105692] Updated weights for policy 0, policy_version 557334 (0.0008) [2023-12-26 19:25:30,843][105692] Updated weights for policy 0, policy_version 557344 (0.0008) [2023-12-26 19:25:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 285605888. Throughput: 0: 9695.8, 1: 9845.5. Samples: 285577088. Policy #0 lag: (min: 5.0, avg: 12.5, max: 37.0) [2023-12-26 19:25:31,062][104569] Avg episode reward: [(0, '8627.563'), (1, '9355.358')] [2023-12-26 19:25:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000557344_142696448.pth... [2023-12-26 19:25:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000556192_142401536.pth [2023-12-26 19:25:31,096][105620] Updated weights for policy 1, policy_version 558193 (0.0011) [2023-12-26 19:25:31,153][105620] Updated weights for policy 1, policy_version 558203 (0.0010) [2023-12-26 19:25:31,213][105620] Updated weights for policy 1, policy_version 558213 (0.0011) [2023-12-26 19:25:31,227][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000558216_142917632.pth... [2023-12-26 19:25:31,230][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000557032_142614528.pth [2023-12-26 19:25:31,632][105692] Updated weights for policy 0, policy_version 557354 (0.0008) [2023-12-26 19:25:31,687][105692] Updated weights for policy 0, policy_version 557364 (0.0009) [2023-12-26 19:25:31,752][105692] Updated weights for policy 0, policy_version 557374 (0.0008) [2023-12-26 19:25:31,970][105620] Updated weights for policy 1, policy_version 558223 (0.0010) [2023-12-26 19:25:32,025][105620] Updated weights for policy 1, policy_version 558233 (0.0009) [2023-12-26 19:25:32,073][105620] Updated weights for policy 1, policy_version 558243 (0.0009) [2023-12-26 19:25:32,530][105692] Updated weights for policy 0, policy_version 557384 (0.0009) [2023-12-26 19:25:32,595][105692] Updated weights for policy 0, policy_version 557394 (0.0010) [2023-12-26 19:25:32,649][105692] Updated weights for policy 0, policy_version 557406 (0.0010) [2023-12-26 19:25:32,762][105620] Updated weights for policy 1, policy_version 558253 (0.0007) [2023-12-26 19:25:32,816][105620] Updated weights for policy 1, policy_version 558263 (0.0005) [2023-12-26 19:25:32,872][105620] Updated weights for policy 1, policy_version 558273 (0.0005) [2023-12-26 19:25:32,878][105586] KL-divergence is very high: 238.9088 [2023-12-26 19:25:33,446][105692] Updated weights for policy 0, policy_version 557416 (0.0008) [2023-12-26 19:25:33,492][105692] Updated weights for policy 0, policy_version 557426 (0.0005) [2023-12-26 19:25:33,537][105692] Updated weights for policy 0, policy_version 557436 (0.0005) [2023-12-26 19:25:33,574][105620] Updated weights for policy 1, policy_version 558283 (0.0006) [2023-12-26 19:25:33,642][105620] Updated weights for policy 1, policy_version 558293 (0.0007) [2023-12-26 19:25:33,692][105620] Updated weights for policy 1, policy_version 558303 (0.0005) [2023-12-26 19:25:34,282][105692] Updated weights for policy 0, policy_version 557446 (0.0009) [2023-12-26 19:25:34,343][105692] Updated weights for policy 0, policy_version 557456 (0.0009) [2023-12-26 19:25:34,402][105620] Updated weights for policy 1, policy_version 558313 (0.0005) [2023-12-26 19:25:34,411][105692] Updated weights for policy 0, policy_version 557466 (0.0009) [2023-12-26 19:25:34,465][105620] Updated weights for policy 1, policy_version 558323 (0.0007) [2023-12-26 19:25:34,524][105620] Updated weights for policy 1, policy_version 558333 (0.0009) [2023-12-26 19:25:34,586][105620] Updated weights for policy 1, policy_version 558343 (0.0008) [2023-12-26 19:25:35,134][105692] Updated weights for policy 0, policy_version 557476 (0.0007) [2023-12-26 19:25:35,196][105692] Updated weights for policy 0, policy_version 557486 (0.0009) [2023-12-26 19:25:35,260][105692] Updated weights for policy 0, policy_version 557496 (0.0009) [2023-12-26 19:25:35,356][105620] Updated weights for policy 1, policy_version 558353 (0.0010) [2023-12-26 19:25:35,407][105620] Updated weights for policy 1, policy_version 558363 (0.0009) [2023-12-26 19:25:35,454][105620] Updated weights for policy 1, policy_version 558373 (0.0008) [2023-12-26 19:25:35,946][105692] Updated weights for policy 0, policy_version 557506 (0.0006) [2023-12-26 19:25:35,993][105692] Updated weights for policy 0, policy_version 557516 (0.0005) [2023-12-26 19:25:36,042][105692] Updated weights for policy 0, policy_version 557526 (0.0008) [2023-12-26 19:25:36,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 285696000. Throughput: 0: 9694.7, 1: 9851.6. Samples: 285690900. Policy #0 lag: (min: 5.0, avg: 12.5, max: 37.0) [2023-12-26 19:25:36,062][104569] Avg episode reward: [(0, '8988.874'), (1, '9172.903')] [2023-12-26 19:25:36,092][105692] Updated weights for policy 0, policy_version 557536 (0.0005) [2023-12-26 19:25:36,282][105620] Updated weights for policy 1, policy_version 558383 (0.0010) [2023-12-26 19:25:36,347][105620] Updated weights for policy 1, policy_version 558393 (0.0011) [2023-12-26 19:25:36,410][105620] Updated weights for policy 1, policy_version 558403 (0.0009) [2023-12-26 19:25:36,843][105692] Updated weights for policy 0, policy_version 557546 (0.0010) [2023-12-26 19:25:36,895][105692] Updated weights for policy 0, policy_version 557556 (0.0008) [2023-12-26 19:25:36,952][105692] Updated weights for policy 0, policy_version 557567 (0.0010) [2023-12-26 19:25:37,045][105620] Updated weights for policy 1, policy_version 558413 (0.0008) [2023-12-26 19:25:37,096][105620] Updated weights for policy 1, policy_version 558423 (0.0005) [2023-12-26 19:25:37,157][105620] Updated weights for policy 1, policy_version 558433 (0.0005) [2023-12-26 19:25:37,753][105692] Updated weights for policy 0, policy_version 557577 (0.0009) [2023-12-26 19:25:37,824][105692] Updated weights for policy 0, policy_version 557587 (0.0009) [2023-12-26 19:25:37,861][105620] Updated weights for policy 1, policy_version 558443 (0.0008) [2023-12-26 19:25:37,890][105692] Updated weights for policy 0, policy_version 557597 (0.0009) [2023-12-26 19:25:37,923][105620] Updated weights for policy 1, policy_version 558453 (0.0005) [2023-12-26 19:25:37,986][105620] Updated weights for policy 1, policy_version 558463 (0.0006) [2023-12-26 19:25:38,578][105692] Updated weights for policy 0, policy_version 557607 (0.0009) [2023-12-26 19:25:38,630][105620] Updated weights for policy 1, policy_version 558473 (0.0008) [2023-12-26 19:25:38,634][105692] Updated weights for policy 0, policy_version 557618 (0.0011) [2023-12-26 19:25:38,686][105692] Updated weights for policy 0, policy_version 557628 (0.0008) [2023-12-26 19:25:38,688][105620] Updated weights for policy 1, policy_version 558483 (0.0007) [2023-12-26 19:25:38,740][105620] Updated weights for policy 1, policy_version 558493 (0.0005) [2023-12-26 19:25:38,786][105620] Updated weights for policy 1, policy_version 558503 (0.0005) [2023-12-26 19:25:39,471][105620] Updated weights for policy 1, policy_version 558513 (0.0009) [2023-12-26 19:25:39,503][105692] Updated weights for policy 0, policy_version 557638 (0.0007) [2023-12-26 19:25:39,530][105620] Updated weights for policy 1, policy_version 558523 (0.0008) [2023-12-26 19:25:39,571][105692] Updated weights for policy 0, policy_version 557648 (0.0007) [2023-12-26 19:25:39,583][105620] Updated weights for policy 1, policy_version 558533 (0.0008) [2023-12-26 19:25:39,587][105586] KL-divergence is very high: 114.4910 [2023-12-26 19:25:39,626][105692] Updated weights for policy 0, policy_version 557658 (0.0008) [2023-12-26 19:25:40,357][105620] Updated weights for policy 1, policy_version 558543 (0.0007) [2023-12-26 19:25:40,384][105692] Updated weights for policy 0, policy_version 557668 (0.0009) [2023-12-26 19:25:40,418][105620] Updated weights for policy 1, policy_version 558553 (0.0006) [2023-12-26 19:25:40,449][105692] Updated weights for policy 0, policy_version 557678 (0.0008) [2023-12-26 19:25:40,452][105586] KL-divergence is very high: 109.6648 [2023-12-26 19:25:40,459][105586] KL-divergence is very high: 154.3479 [2023-12-26 19:25:40,479][105586] KL-divergence is very high: 142.6262 [2023-12-26 19:25:40,484][105620] Updated weights for policy 1, policy_version 558563 (0.0006) [2023-12-26 19:25:40,485][105586] KL-divergence is very high: 151.3439 [2023-12-26 19:25:40,505][105586] KL-divergence is very high: 105.4300 [2023-12-26 19:25:40,512][105586] KL-divergence is very high: 116.2329 [2023-12-26 19:25:40,538][105692] Updated weights for policy 0, policy_version 557688 (0.0008) [2023-12-26 19:25:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 285794304. Throughput: 0: 9601.2, 1: 9853.3. Samples: 285805736. Policy #0 lag: (min: 5.0, avg: 12.5, max: 37.0) [2023-12-26 19:25:41,062][104569] Avg episode reward: [(0, '9078.700'), (1, '4097.754')] [2023-12-26 19:25:41,168][105586] KL-divergence is very high: 107.6957 [2023-12-26 19:25:41,168][105692] Updated weights for policy 0, policy_version 557698 (0.0010) [2023-12-26 19:25:41,199][105620] Updated weights for policy 1, policy_version 558573 (0.0010) [2023-12-26 19:25:41,226][105692] Updated weights for policy 0, policy_version 557708 (0.0007) [2023-12-26 19:25:41,243][105586] KL-divergence is very high: 473.4240 [2023-12-26 19:25:41,258][105620] Updated weights for policy 1, policy_version 558583 (0.0007) [2023-12-26 19:25:41,258][105586] KL-divergence is very high: 465.4225 [2023-12-26 19:25:41,272][105586] KL-divergence is very high: 113.3946 [2023-12-26 19:25:41,288][105692] Updated weights for policy 0, policy_version 557718 (0.0007) [2023-12-26 19:25:41,292][105586] KL-divergence is very high: 607.0006 [2023-12-26 19:25:41,308][105586] KL-divergence is very high: 406.3543 [2023-12-26 19:25:41,321][105620] Updated weights for policy 1, policy_version 558593 (0.0007) [2023-12-26 19:25:41,347][105586] KL-divergence is very high: 363.9667 [2023-12-26 19:25:41,349][105692] Updated weights for policy 0, policy_version 557728 (0.0008) [2023-12-26 19:25:41,363][105586] KL-divergence is very high: 175.5347 [2023-12-26 19:25:42,072][105692] Updated weights for policy 0, policy_version 557738 (0.0010) [2023-12-26 19:25:42,135][105692] Updated weights for policy 0, policy_version 557748 (0.0007) [2023-12-26 19:25:42,145][105586] KL-divergence is very high: 110.4843 [2023-12-26 19:25:42,165][105620] Updated weights for policy 1, policy_version 558603 (0.0008) [2023-12-26 19:25:42,190][105692] Updated weights for policy 0, policy_version 557758 (0.0007) [2023-12-26 19:25:42,232][105620] Updated weights for policy 1, policy_version 558613 (0.0006) [2023-12-26 19:25:42,296][105620] Updated weights for policy 1, policy_version 558623 (0.0008) [2023-12-26 19:25:42,874][105620] Updated weights for policy 1, policy_version 558633 (0.0008) [2023-12-26 19:25:42,922][105692] Updated weights for policy 0, policy_version 557768 (0.0006) [2023-12-26 19:25:42,936][105620] Updated weights for policy 1, policy_version 558643 (0.0006) [2023-12-26 19:25:42,978][105692] Updated weights for policy 0, policy_version 557778 (0.0005) [2023-12-26 19:25:43,001][105620] Updated weights for policy 1, policy_version 558653 (0.0008) [2023-12-26 19:25:43,034][105692] Updated weights for policy 0, policy_version 557788 (0.0005) [2023-12-26 19:25:43,066][105620] Updated weights for policy 1, policy_version 558663 (0.0009) [2023-12-26 19:25:43,556][105692] Updated weights for policy 0, policy_version 557798 (0.0005) [2023-12-26 19:25:43,608][105692] Updated weights for policy 0, policy_version 557808 (0.0005) [2023-12-26 19:25:43,672][105692] Updated weights for policy 0, policy_version 557818 (0.0005) [2023-12-26 19:25:43,914][105620] Updated weights for policy 1, policy_version 558673 (0.0008) [2023-12-26 19:25:43,976][105620] Updated weights for policy 1, policy_version 558683 (0.0008) [2023-12-26 19:25:44,031][105620] Updated weights for policy 1, policy_version 558693 (0.0008) [2023-12-26 19:25:44,315][105692] Updated weights for policy 0, policy_version 557828 (0.0007) [2023-12-26 19:25:44,376][105692] Updated weights for policy 0, policy_version 557838 (0.0010) [2023-12-26 19:25:44,434][105692] Updated weights for policy 0, policy_version 557848 (0.0010) [2023-12-26 19:25:44,695][105620] Updated weights for policy 1, policy_version 558703 (0.0006) [2023-12-26 19:25:44,753][105620] Updated weights for policy 1, policy_version 558713 (0.0005) [2023-12-26 19:25:44,822][105620] Updated weights for policy 1, policy_version 558723 (0.0007) [2023-12-26 19:25:45,058][105692] Updated weights for policy 0, policy_version 557858 (0.0011) [2023-12-26 19:25:45,107][105692] Updated weights for policy 0, policy_version 557868 (0.0011) [2023-12-26 19:25:45,164][105692] Updated weights for policy 0, policy_version 557878 (0.0010) [2023-12-26 19:25:45,230][105692] Updated weights for policy 0, policy_version 557888 (0.0008) [2023-12-26 19:25:45,624][105620] Updated weights for policy 1, policy_version 558733 (0.0008) [2023-12-26 19:25:45,668][105620] Updated weights for policy 1, policy_version 558743 (0.0008) [2023-12-26 19:25:45,717][105620] Updated weights for policy 1, policy_version 558753 (0.0008) [2023-12-26 19:25:45,850][105692] Updated weights for policy 0, policy_version 557898 (0.0006) [2023-12-26 19:25:45,912][105692] Updated weights for policy 0, policy_version 557908 (0.0006) [2023-12-26 19:25:45,964][105692] Updated weights for policy 0, policy_version 557918 (0.0011) [2023-12-26 19:25:46,062][104569] Fps is (10 sec: 20478.8, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 285900800. Throughput: 0: 9646.7, 1: 9841.6. Samples: 285865056. Policy #0 lag: (min: 5.0, avg: 12.5, max: 37.0) [2023-12-26 19:25:46,064][104569] Avg episode reward: [(0, '9081.271'), (1, '805.286')] [2023-12-26 19:25:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000557920_142843904.pth... [2023-12-26 19:25:46,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000558760_143056896.pth... [2023-12-26 19:25:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000557640_142770176.pth [2023-12-26 19:25:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000556800_142557184.pth [2023-12-26 19:25:46,464][105620] Updated weights for policy 1, policy_version 558763 (0.0008) [2023-12-26 19:25:46,522][105620] Updated weights for policy 1, policy_version 558773 (0.0006) [2023-12-26 19:25:46,585][105620] Updated weights for policy 1, policy_version 558783 (0.0007) [2023-12-26 19:25:46,621][105692] Updated weights for policy 0, policy_version 557928 (0.0011) [2023-12-26 19:25:46,680][105692] Updated weights for policy 0, policy_version 557938 (0.0011) [2023-12-26 19:25:46,742][105692] Updated weights for policy 0, policy_version 557948 (0.0010) [2023-12-26 19:25:47,322][105620] Updated weights for policy 1, policy_version 558793 (0.0007) [2023-12-26 19:25:47,377][105620] Updated weights for policy 1, policy_version 558803 (0.0009) [2023-12-26 19:25:47,386][105692] Updated weights for policy 0, policy_version 557958 (0.0007) [2023-12-26 19:25:47,427][105620] Updated weights for policy 1, policy_version 558813 (0.0008) [2023-12-26 19:25:47,437][105692] Updated weights for policy 0, policy_version 557968 (0.0007) [2023-12-26 19:25:47,482][105620] Updated weights for policy 1, policy_version 558823 (0.0006) [2023-12-26 19:25:47,498][105692] Updated weights for policy 0, policy_version 557978 (0.0008) [2023-12-26 19:25:48,147][105692] Updated weights for policy 0, policy_version 557988 (0.0008) [2023-12-26 19:25:48,208][105692] Updated weights for policy 0, policy_version 557998 (0.0009) [2023-12-26 19:25:48,263][105692] Updated weights for policy 0, policy_version 558008 (0.0009) [2023-12-26 19:25:48,290][105620] Updated weights for policy 1, policy_version 558833 (0.0008) [2023-12-26 19:25:48,343][105620] Updated weights for policy 1, policy_version 558843 (0.0007) [2023-12-26 19:25:48,410][105620] Updated weights for policy 1, policy_version 558853 (0.0008) [2023-12-26 19:25:49,063][105692] Updated weights for policy 0, policy_version 558018 (0.0007) [2023-12-26 19:25:49,082][105620] Updated weights for policy 1, policy_version 558863 (0.0007) [2023-12-26 19:25:49,121][105692] Updated weights for policy 0, policy_version 558028 (0.0008) [2023-12-26 19:25:49,137][105620] Updated weights for policy 1, policy_version 558873 (0.0006) [2023-12-26 19:25:49,185][105692] Updated weights for policy 0, policy_version 558038 (0.0006) [2023-12-26 19:25:49,191][105620] Updated weights for policy 1, policy_version 558883 (0.0007) [2023-12-26 19:25:49,256][105692] Updated weights for policy 0, policy_version 558048 (0.0008) [2023-12-26 19:25:49,953][105692] Updated weights for policy 0, policy_version 558058 (0.0009) [2023-12-26 19:25:49,976][105620] Updated weights for policy 1, policy_version 558893 (0.0007) [2023-12-26 19:25:50,020][105692] Updated weights for policy 0, policy_version 558068 (0.0009) [2023-12-26 19:25:50,035][105620] Updated weights for policy 1, policy_version 558903 (0.0006) [2023-12-26 19:25:50,078][105692] Updated weights for policy 0, policy_version 558078 (0.0008) [2023-12-26 19:25:50,092][105620] Updated weights for policy 1, policy_version 558913 (0.0006) [2023-12-26 19:25:50,819][105620] Updated weights for policy 1, policy_version 558923 (0.0009) [2023-12-26 19:25:50,866][105692] Updated weights for policy 0, policy_version 558088 (0.0007) [2023-12-26 19:25:50,880][105620] Updated weights for policy 1, policy_version 558933 (0.0010) [2023-12-26 19:25:50,919][105692] Updated weights for policy 0, policy_version 558098 (0.0006) [2023-12-26 19:25:50,944][105620] Updated weights for policy 1, policy_version 558943 (0.0008) [2023-12-26 19:25:50,975][105692] Updated weights for policy 0, policy_version 558108 (0.0008) [2023-12-26 19:25:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 285999104. Throughput: 0: 9778.5, 1: 9673.8. Samples: 285983068. Policy #0 lag: (min: 5.0, avg: 12.5, max: 37.0) [2023-12-26 19:25:51,062][104569] Avg episode reward: [(0, '9170.415'), (1, '678.657')] [2023-12-26 19:25:51,599][105620] Updated weights for policy 1, policy_version 558953 (0.0007) [2023-12-26 19:25:51,666][105620] Updated weights for policy 1, policy_version 558963 (0.0006) [2023-12-26 19:25:51,733][105620] Updated weights for policy 1, policy_version 558973 (0.0007) [2023-12-26 19:25:51,787][105692] Updated weights for policy 0, policy_version 558118 (0.0008) [2023-12-26 19:25:51,800][105620] Updated weights for policy 1, policy_version 558983 (0.0006) [2023-12-26 19:25:51,852][105692] Updated weights for policy 0, policy_version 558128 (0.0009) [2023-12-26 19:25:51,920][105692] Updated weights for policy 0, policy_version 558138 (0.0009) [2023-12-26 19:25:52,411][105620] Updated weights for policy 1, policy_version 558993 (0.0008) [2023-12-26 19:25:52,473][105620] Updated weights for policy 1, policy_version 559003 (0.0006) [2023-12-26 19:25:52,540][105620] Updated weights for policy 1, policy_version 559013 (0.0005) [2023-12-26 19:25:52,690][105692] Updated weights for policy 0, policy_version 558148 (0.0009) [2023-12-26 19:25:52,742][105692] Updated weights for policy 0, policy_version 558158 (0.0009) [2023-12-26 19:25:52,790][105692] Updated weights for policy 0, policy_version 558168 (0.0009) [2023-12-26 19:25:53,200][105620] Updated weights for policy 1, policy_version 559023 (0.0005) [2023-12-26 19:25:53,268][105620] Updated weights for policy 1, policy_version 559033 (0.0005) [2023-12-26 19:25:53,319][105620] Updated weights for policy 1, policy_version 559043 (0.0008) [2023-12-26 19:25:53,621][105692] Updated weights for policy 0, policy_version 558178 (0.0009) [2023-12-26 19:25:53,692][105692] Updated weights for policy 0, policy_version 558188 (0.0009) [2023-12-26 19:25:53,759][105692] Updated weights for policy 0, policy_version 558198 (0.0009) [2023-12-26 19:25:53,826][105692] Updated weights for policy 0, policy_version 558208 (0.0009) [2023-12-26 19:25:53,944][105620] Updated weights for policy 1, policy_version 559053 (0.0007) [2023-12-26 19:25:54,011][105620] Updated weights for policy 1, policy_version 559063 (0.0005) [2023-12-26 19:25:54,059][105620] Updated weights for policy 1, policy_version 559073 (0.0005) [2023-12-26 19:25:54,455][105692] Updated weights for policy 0, policy_version 558218 (0.0009) [2023-12-26 19:25:54,516][105692] Updated weights for policy 0, policy_version 558228 (0.0009) [2023-12-26 19:25:54,583][105692] Updated weights for policy 0, policy_version 558238 (0.0006) [2023-12-26 19:25:54,652][105620] Updated weights for policy 1, policy_version 559083 (0.0006) [2023-12-26 19:25:54,704][105620] Updated weights for policy 1, policy_version 559093 (0.0009) [2023-12-26 19:25:54,760][105620] Updated weights for policy 1, policy_version 559103 (0.0009) [2023-12-26 19:25:55,179][105692] Updated weights for policy 0, policy_version 558248 (0.0009) [2023-12-26 19:25:55,234][105692] Updated weights for policy 0, policy_version 558258 (0.0010) [2023-12-26 19:25:55,293][105692] Updated weights for policy 0, policy_version 558268 (0.0007) [2023-12-26 19:25:55,595][105620] Updated weights for policy 1, policy_version 559113 (0.0009) [2023-12-26 19:25:55,665][105620] Updated weights for policy 1, policy_version 559123 (0.0009) [2023-12-26 19:25:55,728][105620] Updated weights for policy 1, policy_version 559133 (0.0009) [2023-12-26 19:25:55,795][105620] Updated weights for policy 1, policy_version 559143 (0.0010) [2023-12-26 19:25:55,847][105692] Updated weights for policy 0, policy_version 558278 (0.0005) [2023-12-26 19:25:55,904][105692] Updated weights for policy 0, policy_version 558288 (0.0005) [2023-12-26 19:25:55,961][105692] Updated weights for policy 0, policy_version 558298 (0.0005) [2023-12-26 19:25:56,062][104569] Fps is (10 sec: 19661.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 286097408. Throughput: 0: 9734.4, 1: 9699.9. Samples: 286100844. Policy #0 lag: (min: 5.0, avg: 12.5, max: 37.0) [2023-12-26 19:25:56,062][104569] Avg episode reward: [(0, '8990.147'), (1, '960.839')] [2023-12-26 19:25:56,482][105620] Updated weights for policy 1, policy_version 559153 (0.0009) [2023-12-26 19:25:56,539][105620] Updated weights for policy 1, policy_version 559163 (0.0009) [2023-12-26 19:25:56,596][105620] Updated weights for policy 1, policy_version 559173 (0.0009) [2023-12-26 19:25:56,642][105692] Updated weights for policy 0, policy_version 558308 (0.0006) [2023-12-26 19:25:56,691][105692] Updated weights for policy 0, policy_version 558318 (0.0008) [2023-12-26 19:25:56,738][105692] Updated weights for policy 0, policy_version 558328 (0.0009) [2023-12-26 19:25:57,344][105620] Updated weights for policy 1, policy_version 559183 (0.0008) [2023-12-26 19:25:57,394][105620] Updated weights for policy 1, policy_version 559193 (0.0008) [2023-12-26 19:25:57,440][105692] Updated weights for policy 0, policy_version 558338 (0.0005) [2023-12-26 19:25:57,445][105620] Updated weights for policy 1, policy_version 559203 (0.0009) [2023-12-26 19:25:57,491][105692] Updated weights for policy 0, policy_version 558348 (0.0007) [2023-12-26 19:25:57,545][105692] Updated weights for policy 0, policy_version 558358 (0.0009) [2023-12-26 19:25:57,592][105692] Updated weights for policy 0, policy_version 558368 (0.0009) [2023-12-26 19:25:58,164][105620] Updated weights for policy 1, policy_version 559213 (0.0008) [2023-12-26 19:25:58,226][105620] Updated weights for policy 1, policy_version 559223 (0.0008) [2023-12-26 19:25:58,285][105620] Updated weights for policy 1, policy_version 559233 (0.0008) [2023-12-26 19:25:58,448][105692] Updated weights for policy 0, policy_version 558378 (0.0008) [2023-12-26 19:25:58,513][105692] Updated weights for policy 0, policy_version 558388 (0.0008) [2023-12-26 19:25:58,577][105692] Updated weights for policy 0, policy_version 558398 (0.0008) [2023-12-26 19:25:59,075][105620] Updated weights for policy 1, policy_version 559243 (0.0009) [2023-12-26 19:25:59,138][105620] Updated weights for policy 1, policy_version 559253 (0.0011) [2023-12-26 19:25:59,197][105620] Updated weights for policy 1, policy_version 559263 (0.0010) [2023-12-26 19:25:59,325][105692] Updated weights for policy 0, policy_version 558408 (0.0008) [2023-12-26 19:25:59,391][105692] Updated weights for policy 0, policy_version 558418 (0.0008) [2023-12-26 19:25:59,434][105692] Updated weights for policy 0, policy_version 558428 (0.0008) [2023-12-26 19:25:59,919][105620] Updated weights for policy 1, policy_version 559273 (0.0011) [2023-12-26 19:25:59,982][105620] Updated weights for policy 1, policy_version 559283 (0.0006) [2023-12-26 19:26:00,042][105620] Updated weights for policy 1, policy_version 559293 (0.0011) [2023-12-26 19:26:00,104][105620] Updated weights for policy 1, policy_version 559303 (0.0010) [2023-12-26 19:26:00,228][105692] Updated weights for policy 0, policy_version 558438 (0.0006) [2023-12-26 19:26:00,279][105692] Updated weights for policy 0, policy_version 558448 (0.0008) [2023-12-26 19:26:00,332][105692] Updated weights for policy 0, policy_version 558458 (0.0008) [2023-12-26 19:26:00,750][105620] Updated weights for policy 1, policy_version 559313 (0.0009) [2023-12-26 19:26:00,815][105620] Updated weights for policy 1, policy_version 559323 (0.0010) [2023-12-26 19:26:00,885][105620] Updated weights for policy 1, policy_version 559333 (0.0011) [2023-12-26 19:26:01,017][105692] Updated weights for policy 0, policy_version 558468 (0.0008) [2023-12-26 19:26:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 286187520. Throughput: 0: 9687.3, 1: 9732.1. Samples: 286159260. Policy #0 lag: (min: 5.0, avg: 12.5, max: 37.0) [2023-12-26 19:26:01,063][104569] Avg episode reward: [(0, '8809.315'), (1, '1216.634')] [2023-12-26 19:26:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000559336_143204352.pth... [2023-12-26 19:26:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000558216_142917632.pth [2023-12-26 19:26:01,079][105692] Updated weights for policy 0, policy_version 558478 (0.0009) [2023-12-26 19:26:01,140][105692] Updated weights for policy 0, policy_version 558488 (0.0008) [2023-12-26 19:26:01,188][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000558496_142991360.pth... [2023-12-26 19:26:01,192][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000557344_142696448.pth [2023-12-26 19:26:01,605][105620] Updated weights for policy 1, policy_version 559343 (0.0007) [2023-12-26 19:26:01,676][105620] Updated weights for policy 1, policy_version 559353 (0.0006) [2023-12-26 19:26:01,746][105620] Updated weights for policy 1, policy_version 559363 (0.0007) [2023-12-26 19:26:01,871][105692] Updated weights for policy 0, policy_version 558498 (0.0008) [2023-12-26 19:26:01,910][105585] KL-divergence is very high: 160.3058 [2023-12-26 19:26:01,928][105692] Updated weights for policy 0, policy_version 558508 (0.0009) [2023-12-26 19:26:01,950][105585] KL-divergence is very high: 210.3610 [2023-12-26 19:26:01,977][105692] Updated weights for policy 0, policy_version 558518 (0.0009) [2023-12-26 19:26:01,990][105585] KL-divergence is very high: 159.5800 [2023-12-26 19:26:02,034][105692] Updated weights for policy 0, policy_version 558528 (0.0010) [2023-12-26 19:26:02,292][105620] Updated weights for policy 1, policy_version 559373 (0.0009) [2023-12-26 19:26:02,343][105620] Updated weights for policy 1, policy_version 559383 (0.0009) [2023-12-26 19:26:02,405][105620] Updated weights for policy 1, policy_version 559393 (0.0010) [2023-12-26 19:26:02,749][105692] Updated weights for policy 0, policy_version 558538 (0.0007) [2023-12-26 19:26:02,809][105692] Updated weights for policy 0, policy_version 558548 (0.0009) [2023-12-26 19:26:02,876][105692] Updated weights for policy 0, policy_version 558558 (0.0010) [2023-12-26 19:26:03,012][105620] Updated weights for policy 1, policy_version 559404 (0.0009) [2023-12-26 19:26:03,063][105620] Updated weights for policy 1, policy_version 559414 (0.0005) [2023-12-26 19:26:03,107][105620] Updated weights for policy 1, policy_version 559424 (0.0005) [2023-12-26 19:26:03,656][105620] Updated weights for policy 1, policy_version 559434 (0.0006) [2023-12-26 19:26:03,698][105692] Updated weights for policy 0, policy_version 558568 (0.0009) [2023-12-26 19:26:03,709][105620] Updated weights for policy 1, policy_version 559444 (0.0005) [2023-12-26 19:26:03,755][105692] Updated weights for policy 0, policy_version 558578 (0.0009) [2023-12-26 19:26:03,774][105620] Updated weights for policy 1, policy_version 559454 (0.0005) [2023-12-26 19:26:03,805][105692] Updated weights for policy 0, policy_version 558588 (0.0009) [2023-12-26 19:26:03,829][105620] Updated weights for policy 1, policy_version 559464 (0.0005) [2023-12-26 19:26:04,433][105692] Updated weights for policy 0, policy_version 558598 (0.0008) [2023-12-26 19:26:04,496][105692] Updated weights for policy 0, policy_version 558608 (0.0010) [2023-12-26 19:26:04,549][105620] Updated weights for policy 1, policy_version 559474 (0.0009) [2023-12-26 19:26:04,549][105692] Updated weights for policy 0, policy_version 558618 (0.0007) [2023-12-26 19:26:04,607][105620] Updated weights for policy 1, policy_version 559484 (0.0008) [2023-12-26 19:26:04,662][105620] Updated weights for policy 1, policy_version 559494 (0.0009) [2023-12-26 19:26:05,178][105692] Updated weights for policy 0, policy_version 558628 (0.0006) [2023-12-26 19:26:05,229][105692] Updated weights for policy 0, policy_version 558638 (0.0005) [2023-12-26 19:26:05,281][105692] Updated weights for policy 0, policy_version 558648 (0.0006) [2023-12-26 19:26:05,419][105620] Updated weights for policy 1, policy_version 559504 (0.0009) [2023-12-26 19:26:05,480][105620] Updated weights for policy 1, policy_version 559514 (0.0009) [2023-12-26 19:26:05,535][105620] Updated weights for policy 1, policy_version 559524 (0.0008) [2023-12-26 19:26:05,982][105692] Updated weights for policy 0, policy_version 558658 (0.0007) [2023-12-26 19:26:06,037][105692] Updated weights for policy 0, policy_version 558668 (0.0009) [2023-12-26 19:26:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 286285824. Throughput: 0: 9601.7, 1: 9766.3. Samples: 286278468. Policy #0 lag: (min: 5.0, avg: 12.5, max: 37.0) [2023-12-26 19:26:06,063][104569] Avg episode reward: [(0, '8629.265'), (1, '1081.791')] [2023-12-26 19:26:06,093][105692] Updated weights for policy 0, policy_version 558678 (0.0009) [2023-12-26 19:26:06,159][105692] Updated weights for policy 0, policy_version 558688 (0.0008) [2023-12-26 19:26:06,251][105620] Updated weights for policy 1, policy_version 559534 (0.0008) [2023-12-26 19:26:06,317][105620] Updated weights for policy 1, policy_version 559544 (0.0008) [2023-12-26 19:26:06,384][105620] Updated weights for policy 1, policy_version 559554 (0.0008) [2023-12-26 19:26:06,947][105692] Updated weights for policy 0, policy_version 558698 (0.0005) [2023-12-26 19:26:07,002][105692] Updated weights for policy 0, policy_version 558708 (0.0006) [2023-12-26 19:26:07,060][105692] Updated weights for policy 0, policy_version 558718 (0.0009) [2023-12-26 19:26:07,121][105620] Updated weights for policy 1, policy_version 559564 (0.0008) [2023-12-26 19:26:07,190][105620] Updated weights for policy 1, policy_version 559574 (0.0006) [2023-12-26 19:26:07,253][105620] Updated weights for policy 1, policy_version 559584 (0.0009) [2023-12-26 19:26:07,791][105692] Updated weights for policy 0, policy_version 558728 (0.0009) [2023-12-26 19:26:07,853][105692] Updated weights for policy 0, policy_version 558738 (0.0008) [2023-12-26 19:26:07,911][105692] Updated weights for policy 0, policy_version 558748 (0.0009) [2023-12-26 19:26:07,959][105620] Updated weights for policy 1, policy_version 559594 (0.0009) [2023-12-26 19:26:08,022][105620] Updated weights for policy 1, policy_version 559604 (0.0010) [2023-12-26 19:26:08,101][105620] Updated weights for policy 1, policy_version 559614 (0.0009) [2023-12-26 19:26:08,156][105620] Updated weights for policy 1, policy_version 559624 (0.0009) [2023-12-26 19:26:08,615][105692] Updated weights for policy 0, policy_version 558758 (0.0010) [2023-12-26 19:26:08,673][105692] Updated weights for policy 0, policy_version 558768 (0.0009) [2023-12-26 19:26:08,734][105692] Updated weights for policy 0, policy_version 558778 (0.0008) [2023-12-26 19:26:08,931][105620] Updated weights for policy 1, policy_version 559634 (0.0008) [2023-12-26 19:26:08,994][105620] Updated weights for policy 1, policy_version 559644 (0.0009) [2023-12-26 19:26:09,057][105620] Updated weights for policy 1, policy_version 559654 (0.0009) [2023-12-26 19:26:09,495][105692] Updated weights for policy 0, policy_version 558788 (0.0008) [2023-12-26 19:26:09,557][105692] Updated weights for policy 0, policy_version 558798 (0.0010) [2023-12-26 19:26:09,629][105692] Updated weights for policy 0, policy_version 558808 (0.0010) [2023-12-26 19:26:09,794][105620] Updated weights for policy 1, policy_version 559664 (0.0007) [2023-12-26 19:26:09,857][105620] Updated weights for policy 1, policy_version 559674 (0.0009) [2023-12-26 19:26:09,922][105620] Updated weights for policy 1, policy_version 559684 (0.0009) [2023-12-26 19:26:10,377][105692] Updated weights for policy 0, policy_version 558818 (0.0010) [2023-12-26 19:26:10,434][105692] Updated weights for policy 0, policy_version 558828 (0.0009) [2023-12-26 19:26:10,489][105692] Updated weights for policy 0, policy_version 558838 (0.0009) [2023-12-26 19:26:10,544][105692] Updated weights for policy 0, policy_version 558848 (0.0009) [2023-12-26 19:26:10,702][105620] Updated weights for policy 1, policy_version 559694 (0.0010) [2023-12-26 19:26:10,763][105620] Updated weights for policy 1, policy_version 559704 (0.0009) [2023-12-26 19:26:10,823][105620] Updated weights for policy 1, policy_version 559714 (0.0009) [2023-12-26 19:26:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 286384128. Throughput: 0: 9643.8, 1: 9733.9. Samples: 286391824. Policy #0 lag: (min: 5.0, avg: 12.5, max: 37.0) [2023-12-26 19:26:11,063][104569] Avg episode reward: [(0, '8902.070'), (1, '2343.159')] [2023-12-26 19:26:11,308][105692] Updated weights for policy 0, policy_version 558858 (0.0008) [2023-12-26 19:26:11,366][105692] Updated weights for policy 0, policy_version 558868 (0.0008) [2023-12-26 19:26:11,439][105692] Updated weights for policy 0, policy_version 558878 (0.0008) [2023-12-26 19:26:11,605][105620] Updated weights for policy 1, policy_version 559724 (0.0009) [2023-12-26 19:26:11,672][105620] Updated weights for policy 1, policy_version 559734 (0.0009) [2023-12-26 19:26:11,736][105620] Updated weights for policy 1, policy_version 559744 (0.0009) [2023-12-26 19:26:12,227][105692] Updated weights for policy 0, policy_version 558888 (0.0009) [2023-12-26 19:26:12,290][105692] Updated weights for policy 0, policy_version 558898 (0.0009) [2023-12-26 19:26:12,355][105692] Updated weights for policy 0, policy_version 558908 (0.0009) [2023-12-26 19:26:12,522][105620] Updated weights for policy 1, policy_version 559754 (0.0009) [2023-12-26 19:26:12,577][105620] Updated weights for policy 1, policy_version 559764 (0.0010) [2023-12-26 19:26:12,629][105620] Updated weights for policy 1, policy_version 559774 (0.0010) [2023-12-26 19:26:12,683][105620] Updated weights for policy 1, policy_version 559784 (0.0010) [2023-12-26 19:26:13,102][105692] Updated weights for policy 0, policy_version 558918 (0.0008) [2023-12-26 19:26:13,165][105692] Updated weights for policy 0, policy_version 558928 (0.0007) [2023-12-26 19:26:13,224][105692] Updated weights for policy 0, policy_version 558938 (0.0008) [2023-12-26 19:26:13,438][105620] Updated weights for policy 1, policy_version 559794 (0.0005) [2023-12-26 19:26:13,501][105620] Updated weights for policy 1, policy_version 559804 (0.0009) [2023-12-26 19:26:13,560][105620] Updated weights for policy 1, policy_version 559814 (0.0010) [2023-12-26 19:26:13,998][105692] Updated weights for policy 0, policy_version 558948 (0.0009) [2023-12-26 19:26:14,064][105692] Updated weights for policy 0, policy_version 558958 (0.0006) [2023-12-26 19:26:14,112][105692] Updated weights for policy 0, policy_version 558968 (0.0009) [2023-12-26 19:26:14,191][105620] Updated weights for policy 1, policy_version 559824 (0.0010) [2023-12-26 19:26:14,259][105620] Updated weights for policy 1, policy_version 559834 (0.0010) [2023-12-26 19:26:14,320][105620] Updated weights for policy 1, policy_version 559844 (0.0010) [2023-12-26 19:26:14,843][105692] Updated weights for policy 0, policy_version 558978 (0.0008) [2023-12-26 19:26:14,904][105692] Updated weights for policy 0, policy_version 558988 (0.0006) [2023-12-26 19:26:14,962][105692] Updated weights for policy 0, policy_version 558998 (0.0007) [2023-12-26 19:26:15,014][105692] Updated weights for policy 0, policy_version 559008 (0.0006) [2023-12-26 19:26:15,072][105620] Updated weights for policy 1, policy_version 559854 (0.0011) [2023-12-26 19:26:15,134][105620] Updated weights for policy 1, policy_version 559864 (0.0009) [2023-12-26 19:26:15,195][105620] Updated weights for policy 1, policy_version 559874 (0.0010) [2023-12-26 19:26:15,663][105692] Updated weights for policy 0, policy_version 559018 (0.0010) [2023-12-26 19:26:15,724][105585] KL-divergence is very high: 117.9638 [2023-12-26 19:26:15,732][105692] Updated weights for policy 0, policy_version 559028 (0.0010) [2023-12-26 19:26:15,772][105585] KL-divergence is very high: 128.7108 [2023-12-26 19:26:15,793][105692] Updated weights for policy 0, policy_version 559038 (0.0008) [2023-12-26 19:26:15,880][105620] Updated weights for policy 1, policy_version 559884 (0.0011) [2023-12-26 19:26:15,939][105620] Updated weights for policy 1, policy_version 559894 (0.0011) [2023-12-26 19:26:15,999][105620] Updated weights for policy 1, policy_version 559904 (0.0011) [2023-12-26 19:26:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 286482432. Throughput: 0: 9623.6, 1: 9722.4. Samples: 286447660. Policy #0 lag: (min: 5.0, avg: 12.5, max: 37.0) [2023-12-26 19:26:16,062][104569] Avg episode reward: [(0, '8304.662'), (1, '2701.265')] [2023-12-26 19:26:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000559912_143351808.pth... [2023-12-26 19:26:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000559040_143130624.pth... [2023-12-26 19:26:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000557920_142843904.pth [2023-12-26 19:26:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000558760_143056896.pth [2023-12-26 19:26:16,464][105692] Updated weights for policy 0, policy_version 559048 (0.0008) [2023-12-26 19:26:16,515][105692] Updated weights for policy 0, policy_version 559058 (0.0008) [2023-12-26 19:26:16,560][105692] Updated weights for policy 0, policy_version 559068 (0.0008) [2023-12-26 19:26:16,752][105620] Updated weights for policy 1, policy_version 559914 (0.0010) [2023-12-26 19:26:16,814][105620] Updated weights for policy 1, policy_version 559924 (0.0009) [2023-12-26 19:26:16,862][105620] Updated weights for policy 1, policy_version 559934 (0.0009) [2023-12-26 19:26:16,909][105620] Updated weights for policy 1, policy_version 559944 (0.0009) [2023-12-26 19:26:17,239][105692] Updated weights for policy 0, policy_version 559078 (0.0007) [2023-12-26 19:26:17,291][105692] Updated weights for policy 0, policy_version 559088 (0.0009) [2023-12-26 19:26:17,347][105692] Updated weights for policy 0, policy_version 559099 (0.0009) [2023-12-26 19:26:17,723][105620] Updated weights for policy 1, policy_version 559954 (0.0008) [2023-12-26 19:26:17,778][105620] Updated weights for policy 1, policy_version 559964 (0.0005) [2023-12-26 19:26:17,828][105620] Updated weights for policy 1, policy_version 559974 (0.0007) [2023-12-26 19:26:18,034][105692] Updated weights for policy 0, policy_version 559109 (0.0006) [2023-12-26 19:26:18,091][105692] Updated weights for policy 0, policy_version 559119 (0.0005) [2023-12-26 19:26:18,141][105692] Updated weights for policy 0, policy_version 559129 (0.0005) [2023-12-26 19:26:18,649][105620] Updated weights for policy 1, policy_version 559984 (0.0008) [2023-12-26 19:26:18,713][105620] Updated weights for policy 1, policy_version 559994 (0.0008) [2023-12-26 19:26:18,769][105620] Updated weights for policy 1, policy_version 560004 (0.0005) [2023-12-26 19:26:18,780][105692] Updated weights for policy 0, policy_version 559139 (0.0006) [2023-12-26 19:26:18,836][105692] Updated weights for policy 0, policy_version 559149 (0.0009) [2023-12-26 19:26:18,883][105692] Updated weights for policy 0, policy_version 559159 (0.0009) [2023-12-26 19:26:19,518][105620] Updated weights for policy 1, policy_version 560014 (0.0008) [2023-12-26 19:26:19,573][105620] Updated weights for policy 1, policy_version 560024 (0.0009) [2023-12-26 19:26:19,628][105620] Updated weights for policy 1, policy_version 560034 (0.0008) [2023-12-26 19:26:19,661][105692] Updated weights for policy 0, policy_version 559169 (0.0009) [2023-12-26 19:26:19,727][105692] Updated weights for policy 0, policy_version 559179 (0.0011) [2023-12-26 19:26:19,786][105692] Updated weights for policy 0, policy_version 559189 (0.0011) [2023-12-26 19:26:19,851][105692] Updated weights for policy 0, policy_version 559199 (0.0011) [2023-12-26 19:26:20,464][105620] Updated weights for policy 1, policy_version 560044 (0.0008) [2023-12-26 19:26:20,466][105692] Updated weights for policy 0, policy_version 559209 (0.0010) [2023-12-26 19:26:20,519][105620] Updated weights for policy 1, policy_version 560054 (0.0006) [2023-12-26 19:26:20,526][105692] Updated weights for policy 0, policy_version 559219 (0.0011) [2023-12-26 19:26:20,573][105620] Updated weights for policy 1, policy_version 560064 (0.0008) [2023-12-26 19:26:20,593][105692] Updated weights for policy 0, policy_version 559229 (0.0009) [2023-12-26 19:26:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 286572544. Throughput: 0: 9745.1, 1: 9638.4. Samples: 286563164. Policy #0 lag: (min: 5.0, avg: 12.5, max: 37.0) [2023-12-26 19:26:21,063][104569] Avg episode reward: [(0, '7334.839'), (1, '6749.094')] [2023-12-26 19:26:21,303][105692] Updated weights for policy 0, policy_version 559239 (0.0011) [2023-12-26 19:26:21,372][105692] Updated weights for policy 0, policy_version 559249 (0.0009) [2023-12-26 19:26:21,379][105620] Updated weights for policy 1, policy_version 560074 (0.0006) [2023-12-26 19:26:21,443][105692] Updated weights for policy 0, policy_version 559259 (0.0009) [2023-12-26 19:26:21,451][105620] Updated weights for policy 1, policy_version 560084 (0.0008) [2023-12-26 19:26:21,513][105620] Updated weights for policy 1, policy_version 560094 (0.0006) [2023-12-26 19:26:21,569][105620] Updated weights for policy 1, policy_version 560104 (0.0005) [2023-12-26 19:26:22,157][105692] Updated weights for policy 0, policy_version 559269 (0.0010) [2023-12-26 19:26:22,224][105692] Updated weights for policy 0, policy_version 559279 (0.0011) [2023-12-26 19:26:22,263][105620] Updated weights for policy 1, policy_version 560114 (0.0006) [2023-12-26 19:26:22,280][105692] Updated weights for policy 0, policy_version 559289 (0.0011) [2023-12-26 19:26:22,322][105620] Updated weights for policy 1, policy_version 560124 (0.0008) [2023-12-26 19:26:22,389][105620] Updated weights for policy 1, policy_version 560134 (0.0009) [2023-12-26 19:26:23,043][105692] Updated weights for policy 0, policy_version 559299 (0.0011) [2023-12-26 19:26:23,099][105692] Updated weights for policy 0, policy_version 559309 (0.0009) [2023-12-26 19:26:23,144][105620] Updated weights for policy 1, policy_version 560144 (0.0008) [2023-12-26 19:26:23,159][105692] Updated weights for policy 0, policy_version 559319 (0.0006) [2023-12-26 19:26:23,201][105620] Updated weights for policy 1, policy_version 560154 (0.0007) [2023-12-26 19:26:23,263][105620] Updated weights for policy 1, policy_version 560164 (0.0009) [2023-12-26 19:26:23,870][105692] Updated weights for policy 0, policy_version 559329 (0.0007) [2023-12-26 19:26:23,925][105692] Updated weights for policy 0, policy_version 559339 (0.0008) [2023-12-26 19:26:23,983][105620] Updated weights for policy 1, policy_version 560174 (0.0010) [2023-12-26 19:26:23,985][105692] Updated weights for policy 0, policy_version 559349 (0.0006) [2023-12-26 19:26:24,033][105692] Updated weights for policy 0, policy_version 559359 (0.0005) [2023-12-26 19:26:24,042][105620] Updated weights for policy 1, policy_version 560184 (0.0010) [2023-12-26 19:26:24,101][105620] Updated weights for policy 1, policy_version 560194 (0.0010) [2023-12-26 19:26:24,714][105692] Updated weights for policy 0, policy_version 559369 (0.0006) [2023-12-26 19:26:24,772][105692] Updated weights for policy 0, policy_version 559379 (0.0005) [2023-12-26 19:26:24,832][105692] Updated weights for policy 0, policy_version 559389 (0.0006) [2023-12-26 19:26:24,838][105620] Updated weights for policy 1, policy_version 560204 (0.0010) [2023-12-26 19:26:24,899][105620] Updated weights for policy 1, policy_version 560214 (0.0007) [2023-12-26 19:26:24,964][105620] Updated weights for policy 1, policy_version 560224 (0.0010) [2023-12-26 19:26:25,454][105692] Updated weights for policy 0, policy_version 559399 (0.0006) [2023-12-26 19:26:25,508][105692] Updated weights for policy 0, policy_version 559409 (0.0007) [2023-12-26 19:26:25,568][105692] Updated weights for policy 0, policy_version 559419 (0.0008) [2023-12-26 19:26:25,679][105620] Updated weights for policy 1, policy_version 560234 (0.0010) [2023-12-26 19:26:25,740][105620] Updated weights for policy 1, policy_version 560244 (0.0010) [2023-12-26 19:26:25,797][105620] Updated weights for policy 1, policy_version 560254 (0.0010) [2023-12-26 19:26:25,855][105620] Updated weights for policy 1, policy_version 560264 (0.0010) [2023-12-26 19:26:26,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 286670848. Throughput: 0: 9829.6, 1: 9572.8. Samples: 286678844. Policy #0 lag: (min: 5.0, avg: 12.5, max: 37.0) [2023-12-26 19:26:26,063][104569] Avg episode reward: [(0, '6626.136'), (1, '9089.445')] [2023-12-26 19:26:26,271][105692] Updated weights for policy 0, policy_version 559429 (0.0009) [2023-12-26 19:26:26,325][105692] Updated weights for policy 0, policy_version 559439 (0.0010) [2023-12-26 19:26:26,379][105692] Updated weights for policy 0, policy_version 559451 (0.0010) [2023-12-26 19:26:26,540][105620] Updated weights for policy 1, policy_version 560274 (0.0009) [2023-12-26 19:26:26,593][105620] Updated weights for policy 1, policy_version 560285 (0.0010) [2023-12-26 19:26:26,647][105620] Updated weights for policy 1, policy_version 560296 (0.0010) [2023-12-26 19:26:27,048][105692] Updated weights for policy 0, policy_version 559462 (0.0008) [2023-12-26 19:26:27,098][105692] Updated weights for policy 0, policy_version 559472 (0.0005) [2023-12-26 19:26:27,164][105692] Updated weights for policy 0, policy_version 559482 (0.0006) [2023-12-26 19:26:27,436][105620] Updated weights for policy 1, policy_version 560306 (0.0010) [2023-12-26 19:26:27,483][105620] Updated weights for policy 1, policy_version 560316 (0.0010) [2023-12-26 19:26:27,532][105620] Updated weights for policy 1, policy_version 560326 (0.0007) [2023-12-26 19:26:27,811][105692] Updated weights for policy 0, policy_version 559492 (0.0007) [2023-12-26 19:26:27,865][105692] Updated weights for policy 0, policy_version 559502 (0.0009) [2023-12-26 19:26:27,918][105692] Updated weights for policy 0, policy_version 559512 (0.0010) [2023-12-26 19:26:28,096][105620] Updated weights for policy 1, policy_version 560336 (0.0005) [2023-12-26 19:26:28,153][105620] Updated weights for policy 1, policy_version 560346 (0.0006) [2023-12-26 19:26:28,206][105620] Updated weights for policy 1, policy_version 560356 (0.0005) [2023-12-26 19:26:28,749][105692] Updated weights for policy 0, policy_version 559522 (0.0010) [2023-12-26 19:26:28,807][105692] Updated weights for policy 0, policy_version 559532 (0.0010) [2023-12-26 19:26:28,844][105620] Updated weights for policy 1, policy_version 560366 (0.0008) [2023-12-26 19:26:28,865][105692] Updated weights for policy 0, policy_version 559542 (0.0010) [2023-12-26 19:26:28,899][105620] Updated weights for policy 1, policy_version 560376 (0.0010) [2023-12-26 19:26:28,924][105692] Updated weights for policy 0, policy_version 559552 (0.0010) [2023-12-26 19:26:28,957][105620] Updated weights for policy 1, policy_version 560386 (0.0008) [2023-12-26 19:26:29,635][105692] Updated weights for policy 0, policy_version 559562 (0.0009) [2023-12-26 19:26:29,691][105692] Updated weights for policy 0, policy_version 559572 (0.0009) [2023-12-26 19:26:29,738][105620] Updated weights for policy 1, policy_version 560396 (0.0008) [2023-12-26 19:26:29,744][105692] Updated weights for policy 0, policy_version 559582 (0.0008) [2023-12-26 19:26:29,787][105620] Updated weights for policy 1, policy_version 560406 (0.0008) [2023-12-26 19:26:29,846][105620] Updated weights for policy 1, policy_version 560416 (0.0009) [2023-12-26 19:26:30,473][105692] Updated weights for policy 0, policy_version 559592 (0.0006) [2023-12-26 19:26:30,533][105692] Updated weights for policy 0, policy_version 559602 (0.0006) [2023-12-26 19:26:30,591][105692] Updated weights for policy 0, policy_version 559612 (0.0006) [2023-12-26 19:26:30,602][105620] Updated weights for policy 1, policy_version 560426 (0.0008) [2023-12-26 19:26:30,656][105620] Updated weights for policy 1, policy_version 560436 (0.0005) [2023-12-26 19:26:30,711][105620] Updated weights for policy 1, policy_version 560446 (0.0005) [2023-12-26 19:26:30,765][105620] Updated weights for policy 1, policy_version 560456 (0.0005) [2023-12-26 19:26:31,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 286769152. Throughput: 0: 9784.7, 1: 9659.3. Samples: 286740024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:26:31,062][104569] Avg episode reward: [(0, '7597.319'), (1, '9091.559')] [2023-12-26 19:26:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000560456_143491072.pth... [2023-12-26 19:26:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000559616_143278080.pth... [2023-12-26 19:26:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000559336_143204352.pth [2023-12-26 19:26:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000558496_142991360.pth [2023-12-26 19:26:31,206][105692] Updated weights for policy 0, policy_version 559622 (0.0009) [2023-12-26 19:26:31,269][105692] Updated weights for policy 0, policy_version 559632 (0.0010) [2023-12-26 19:26:31,329][105692] Updated weights for policy 0, policy_version 559642 (0.0010) [2023-12-26 19:26:31,444][105620] Updated weights for policy 1, policy_version 560466 (0.0009) [2023-12-26 19:26:31,501][105620] Updated weights for policy 1, policy_version 560477 (0.0012) [2023-12-26 19:26:31,562][105620] Updated weights for policy 1, policy_version 560487 (0.0010) [2023-12-26 19:26:31,966][105692] Updated weights for policy 0, policy_version 559652 (0.0008) [2023-12-26 19:26:32,027][105692] Updated weights for policy 0, policy_version 559662 (0.0005) [2023-12-26 19:26:32,089][105692] Updated weights for policy 0, policy_version 559672 (0.0007) [2023-12-26 19:26:32,366][105620] Updated weights for policy 1, policy_version 560497 (0.0009) [2023-12-26 19:26:32,430][105620] Updated weights for policy 1, policy_version 560507 (0.0008) [2023-12-26 19:26:32,492][105620] Updated weights for policy 1, policy_version 560517 (0.0008) [2023-12-26 19:26:32,785][105692] Updated weights for policy 0, policy_version 559682 (0.0010) [2023-12-26 19:26:32,837][105692] Updated weights for policy 0, policy_version 559692 (0.0010) [2023-12-26 19:26:32,892][105692] Updated weights for policy 0, policy_version 559702 (0.0010) [2023-12-26 19:26:32,943][105692] Updated weights for policy 0, policy_version 559712 (0.0010) [2023-12-26 19:26:33,241][105620] Updated weights for policy 1, policy_version 560527 (0.0008) [2023-12-26 19:26:33,292][105620] Updated weights for policy 1, policy_version 560537 (0.0008) [2023-12-26 19:26:33,336][105620] Updated weights for policy 1, policy_version 560547 (0.0007) [2023-12-26 19:26:33,682][105692] Updated weights for policy 0, policy_version 559722 (0.0010) [2023-12-26 19:26:33,729][105692] Updated weights for policy 0, policy_version 559732 (0.0010) [2023-12-26 19:26:33,776][105692] Updated weights for policy 0, policy_version 559742 (0.0010) [2023-12-26 19:26:34,106][105620] Updated weights for policy 1, policy_version 560557 (0.0008) [2023-12-26 19:26:34,170][105620] Updated weights for policy 1, policy_version 560567 (0.0008) [2023-12-26 19:26:34,228][105620] Updated weights for policy 1, policy_version 560577 (0.0008) [2023-12-26 19:26:34,544][105692] Updated weights for policy 0, policy_version 559752 (0.0010) [2023-12-26 19:26:34,600][105692] Updated weights for policy 0, policy_version 559762 (0.0010) [2023-12-26 19:26:34,653][105692] Updated weights for policy 0, policy_version 559772 (0.0010) [2023-12-26 19:26:34,987][105620] Updated weights for policy 1, policy_version 560587 (0.0007) [2023-12-26 19:26:35,034][105620] Updated weights for policy 1, policy_version 560597 (0.0005) [2023-12-26 19:26:35,080][105620] Updated weights for policy 1, policy_version 560607 (0.0005) [2023-12-26 19:26:35,392][105692] Updated weights for policy 0, policy_version 559782 (0.0009) [2023-12-26 19:26:35,443][105692] Updated weights for policy 0, policy_version 559792 (0.0010) [2023-12-26 19:26:35,491][105692] Updated weights for policy 0, policy_version 559802 (0.0010) [2023-12-26 19:26:35,743][105620] Updated weights for policy 1, policy_version 560617 (0.0005) [2023-12-26 19:26:35,798][105620] Updated weights for policy 1, policy_version 560627 (0.0007) [2023-12-26 19:26:35,853][105620] Updated weights for policy 1, policy_version 560637 (0.0006) [2023-12-26 19:26:35,903][105620] Updated weights for policy 1, policy_version 560647 (0.0010) [2023-12-26 19:26:36,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 286867456. Throughput: 0: 9744.8, 1: 9639.9. Samples: 286855380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:26:36,062][104569] Avg episode reward: [(0, '8311.590'), (1, '9169.905')] [2023-12-26 19:26:36,244][105692] Updated weights for policy 0, policy_version 559812 (0.0008) [2023-12-26 19:26:36,301][105692] Updated weights for policy 0, policy_version 559822 (0.0011) [2023-12-26 19:26:36,354][105692] Updated weights for policy 0, policy_version 559832 (0.0011) [2023-12-26 19:26:36,650][105620] Updated weights for policy 1, policy_version 560657 (0.0011) [2023-12-26 19:26:36,713][105620] Updated weights for policy 1, policy_version 560667 (0.0011) [2023-12-26 19:26:36,774][105620] Updated weights for policy 1, policy_version 560677 (0.0011) [2023-12-26 19:26:37,111][105692] Updated weights for policy 0, policy_version 559842 (0.0011) [2023-12-26 19:26:37,167][105692] Updated weights for policy 0, policy_version 559852 (0.0011) [2023-12-26 19:26:37,228][105692] Updated weights for policy 0, policy_version 559862 (0.0011) [2023-12-26 19:26:37,288][105692] Updated weights for policy 0, policy_version 559872 (0.0011) [2023-12-26 19:26:37,414][105620] Updated weights for policy 1, policy_version 560687 (0.0010) [2023-12-26 19:26:37,466][105620] Updated weights for policy 1, policy_version 560697 (0.0010) [2023-12-26 19:26:37,522][105620] Updated weights for policy 1, policy_version 560707 (0.0011) [2023-12-26 19:26:38,032][105692] Updated weights for policy 0, policy_version 559882 (0.0010) [2023-12-26 19:26:38,084][105692] Updated weights for policy 0, policy_version 559892 (0.0010) [2023-12-26 19:26:38,136][105692] Updated weights for policy 0, policy_version 559902 (0.0010) [2023-12-26 19:26:38,266][105620] Updated weights for policy 1, policy_version 560717 (0.0010) [2023-12-26 19:26:38,317][105620] Updated weights for policy 1, policy_version 560727 (0.0008) [2023-12-26 19:26:38,383][105620] Updated weights for policy 1, policy_version 560737 (0.0009) [2023-12-26 19:26:38,906][105692] Updated weights for policy 0, policy_version 559912 (0.0010) [2023-12-26 19:26:38,976][105692] Updated weights for policy 0, policy_version 559922 (0.0010) [2023-12-26 19:26:39,047][105620] Updated weights for policy 1, policy_version 560747 (0.0006) [2023-12-26 19:26:39,049][105692] Updated weights for policy 0, policy_version 559932 (0.0010) [2023-12-26 19:26:39,112][105620] Updated weights for policy 1, policy_version 560757 (0.0008) [2023-12-26 19:26:39,171][105620] Updated weights for policy 1, policy_version 560767 (0.0008) [2023-12-26 19:26:39,821][105692] Updated weights for policy 0, policy_version 559942 (0.0009) [2023-12-26 19:26:39,894][105692] Updated weights for policy 0, policy_version 559952 (0.0010) [2023-12-26 19:26:39,963][105692] Updated weights for policy 0, policy_version 559962 (0.0010) [2023-12-26 19:26:39,990][105620] Updated weights for policy 1, policy_version 560777 (0.0007) [2023-12-26 19:26:40,054][105620] Updated weights for policy 1, policy_version 560787 (0.0006) [2023-12-26 19:26:40,112][105620] Updated weights for policy 1, policy_version 560797 (0.0010) [2023-12-26 19:26:40,172][105620] Updated weights for policy 1, policy_version 560807 (0.0011) [2023-12-26 19:26:40,627][105692] Updated weights for policy 0, policy_version 559972 (0.0010) [2023-12-26 19:26:40,673][105692] Updated weights for policy 0, policy_version 559982 (0.0007) [2023-12-26 19:26:40,729][105692] Updated weights for policy 0, policy_version 559992 (0.0008) [2023-12-26 19:26:40,894][105620] Updated weights for policy 1, policy_version 560817 (0.0007) [2023-12-26 19:26:40,955][105620] Updated weights for policy 1, policy_version 560827 (0.0010) [2023-12-26 19:26:41,008][105620] Updated weights for policy 1, policy_version 560837 (0.0009) [2023-12-26 19:26:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 286965760. Throughput: 0: 9703.6, 1: 9617.9. Samples: 286970316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:26:41,063][104569] Avg episode reward: [(0, '8751.276'), (1, '9260.791')] [2023-12-26 19:26:41,448][105692] Updated weights for policy 0, policy_version 560002 (0.0009) [2023-12-26 19:26:41,506][105692] Updated weights for policy 0, policy_version 560012 (0.0008) [2023-12-26 19:26:41,567][105692] Updated weights for policy 0, policy_version 560022 (0.0008) [2023-12-26 19:26:41,634][105692] Updated weights for policy 0, policy_version 560032 (0.0009) [2023-12-26 19:26:41,795][105620] Updated weights for policy 1, policy_version 560847 (0.0008) [2023-12-26 19:26:41,843][105620] Updated weights for policy 1, policy_version 560857 (0.0008) [2023-12-26 19:26:41,898][105620] Updated weights for policy 1, policy_version 560867 (0.0008) [2023-12-26 19:26:42,394][105692] Updated weights for policy 0, policy_version 560042 (0.0009) [2023-12-26 19:26:42,445][105692] Updated weights for policy 0, policy_version 560052 (0.0007) [2023-12-26 19:26:42,501][105692] Updated weights for policy 0, policy_version 560062 (0.0005) [2023-12-26 19:26:42,688][105620] Updated weights for policy 1, policy_version 560877 (0.0009) [2023-12-26 19:26:42,742][105620] Updated weights for policy 1, policy_version 560887 (0.0010) [2023-12-26 19:26:42,798][105620] Updated weights for policy 1, policy_version 560897 (0.0009) [2023-12-26 19:26:43,112][105692] Updated weights for policy 0, policy_version 560072 (0.0006) [2023-12-26 19:26:43,166][105692] Updated weights for policy 0, policy_version 560082 (0.0005) [2023-12-26 19:26:43,217][105692] Updated weights for policy 0, policy_version 560092 (0.0005) [2023-12-26 19:26:43,374][105620] Updated weights for policy 1, policy_version 560907 (0.0005) [2023-12-26 19:26:43,434][105620] Updated weights for policy 1, policy_version 560917 (0.0006) [2023-12-26 19:26:43,492][105620] Updated weights for policy 1, policy_version 560927 (0.0010) [2023-12-26 19:26:43,760][105692] Updated weights for policy 0, policy_version 560102 (0.0006) [2023-12-26 19:26:43,806][105692] Updated weights for policy 0, policy_version 560112 (0.0007) [2023-12-26 19:26:43,859][105692] Updated weights for policy 0, policy_version 560122 (0.0005) [2023-12-26 19:26:44,292][105620] Updated weights for policy 1, policy_version 560937 (0.0010) [2023-12-26 19:26:44,358][105620] Updated weights for policy 1, policy_version 560947 (0.0008) [2023-12-26 19:26:44,391][105692] Updated weights for policy 0, policy_version 560132 (0.0007) [2023-12-26 19:26:44,410][105620] Updated weights for policy 1, policy_version 560957 (0.0008) [2023-12-26 19:26:44,443][105692] Updated weights for policy 0, policy_version 560142 (0.0010) [2023-12-26 19:26:44,461][105620] Updated weights for policy 1, policy_version 560967 (0.0005) [2023-12-26 19:26:44,490][105692] Updated weights for policy 0, policy_version 560152 (0.0010) [2023-12-26 19:26:45,091][105620] Updated weights for policy 1, policy_version 560977 (0.0008) [2023-12-26 19:26:45,142][105692] Updated weights for policy 0, policy_version 560162 (0.0007) [2023-12-26 19:26:45,150][105620] Updated weights for policy 1, policy_version 560987 (0.0008) [2023-12-26 19:26:45,196][105692] Updated weights for policy 0, policy_version 560172 (0.0011) [2023-12-26 19:26:45,211][105620] Updated weights for policy 1, policy_version 560997 (0.0007) [2023-12-26 19:26:45,257][105692] Updated weights for policy 0, policy_version 560182 (0.0011) [2023-12-26 19:26:45,324][105692] Updated weights for policy 0, policy_version 560192 (0.0011) [2023-12-26 19:26:45,984][105620] Updated weights for policy 1, policy_version 561007 (0.0007) [2023-12-26 19:26:46,040][105620] Updated weights for policy 1, policy_version 561017 (0.0008) [2023-12-26 19:26:46,042][105692] Updated weights for policy 0, policy_version 560202 (0.0011) [2023-12-26 19:26:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.3, 300 sec: 19521.9). Total num frames: 287055872. Throughput: 0: 9732.7, 1: 9609.7. Samples: 287029668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:26:46,063][104569] Avg episode reward: [(0, '9093.591'), (1, '9261.253')] [2023-12-26 19:26:46,098][105692] Updated weights for policy 0, policy_version 560212 (0.0011) [2023-12-26 19:26:46,099][105620] Updated weights for policy 1, policy_version 561027 (0.0008) [2023-12-26 19:26:46,123][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000561032_143638528.pth... [2023-12-26 19:26:46,127][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000559912_143351808.pth [2023-12-26 19:26:46,154][105692] Updated weights for policy 0, policy_version 560222 (0.0010) [2023-12-26 19:26:46,163][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000560224_143433728.pth... [2023-12-26 19:26:46,166][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000559040_143130624.pth [2023-12-26 19:26:46,791][105692] Updated weights for policy 0, policy_version 560232 (0.0006) [2023-12-26 19:26:46,842][105692] Updated weights for policy 0, policy_version 560242 (0.0005) [2023-12-26 19:26:46,888][105620] Updated weights for policy 1, policy_version 561037 (0.0008) [2023-12-26 19:26:46,894][105692] Updated weights for policy 0, policy_version 560252 (0.0006) [2023-12-26 19:26:46,942][105620] Updated weights for policy 1, policy_version 561047 (0.0008) [2023-12-26 19:26:46,998][105620] Updated weights for policy 1, policy_version 561057 (0.0008) [2023-12-26 19:26:47,468][105692] Updated weights for policy 0, policy_version 560262 (0.0007) [2023-12-26 19:26:47,516][105692] Updated weights for policy 0, policy_version 560272 (0.0005) [2023-12-26 19:26:47,571][105692] Updated weights for policy 0, policy_version 560282 (0.0005) [2023-12-26 19:26:47,899][105620] Updated weights for policy 1, policy_version 561067 (0.0008) [2023-12-26 19:26:47,961][105620] Updated weights for policy 1, policy_version 561077 (0.0009) [2023-12-26 19:26:48,012][105620] Updated weights for policy 1, policy_version 561087 (0.0009) [2023-12-26 19:26:48,180][105692] Updated weights for policy 0, policy_version 560292 (0.0007) [2023-12-26 19:26:48,234][105692] Updated weights for policy 0, policy_version 560302 (0.0010) [2023-12-26 19:26:48,294][105692] Updated weights for policy 0, policy_version 560313 (0.0012) [2023-12-26 19:26:48,763][105620] Updated weights for policy 1, policy_version 561097 (0.0009) [2023-12-26 19:26:48,820][105620] Updated weights for policy 1, policy_version 561107 (0.0010) [2023-12-26 19:26:48,873][105620] Updated weights for policy 1, policy_version 561117 (0.0009) [2023-12-26 19:26:48,926][105620] Updated weights for policy 1, policy_version 561127 (0.0009) [2023-12-26 19:26:48,962][105692] Updated weights for policy 0, policy_version 560323 (0.0009) [2023-12-26 19:26:49,021][105692] Updated weights for policy 0, policy_version 560333 (0.0006) [2023-12-26 19:26:49,078][105692] Updated weights for policy 0, policy_version 560343 (0.0007) [2023-12-26 19:26:49,695][105692] Updated weights for policy 0, policy_version 560353 (0.0006) [2023-12-26 19:26:49,734][105620] Updated weights for policy 1, policy_version 561137 (0.0006) [2023-12-26 19:26:49,756][105692] Updated weights for policy 0, policy_version 560363 (0.0009) [2023-12-26 19:26:49,794][105620] Updated weights for policy 1, policy_version 561147 (0.0007) [2023-12-26 19:26:49,821][105692] Updated weights for policy 0, policy_version 560373 (0.0006) [2023-12-26 19:26:49,862][105620] Updated weights for policy 1, policy_version 561157 (0.0008) [2023-12-26 19:26:49,887][105692] Updated weights for policy 0, policy_version 560383 (0.0007) [2023-12-26 19:26:50,508][105620] Updated weights for policy 1, policy_version 561167 (0.0008) [2023-12-26 19:26:50,564][105620] Updated weights for policy 1, policy_version 561177 (0.0009) [2023-12-26 19:26:50,628][105620] Updated weights for policy 1, policy_version 561187 (0.0009) [2023-12-26 19:26:50,647][105692] Updated weights for policy 0, policy_version 560393 (0.0008) [2023-12-26 19:26:50,703][105692] Updated weights for policy 0, policy_version 560403 (0.0008) [2023-12-26 19:26:50,763][105692] Updated weights for policy 0, policy_version 560413 (0.0010) [2023-12-26 19:26:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 287162368. Throughput: 0: 9938.0, 1: 9425.6. Samples: 287149828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:26:51,062][104569] Avg episode reward: [(0, '8828.230'), (1, '9261.173')] [2023-12-26 19:26:51,406][105620] Updated weights for policy 1, policy_version 561197 (0.0008) [2023-12-26 19:26:51,460][105620] Updated weights for policy 1, policy_version 561207 (0.0006) [2023-12-26 19:26:51,518][105620] Updated weights for policy 1, policy_version 561217 (0.0008) [2023-12-26 19:26:51,524][105692] Updated weights for policy 0, policy_version 560423 (0.0008) [2023-12-26 19:26:51,579][105692] Updated weights for policy 0, policy_version 560433 (0.0006) [2023-12-26 19:26:51,637][105692] Updated weights for policy 0, policy_version 560443 (0.0009) [2023-12-26 19:26:52,282][105620] Updated weights for policy 1, policy_version 561227 (0.0009) [2023-12-26 19:26:52,344][105620] Updated weights for policy 1, policy_version 561237 (0.0009) [2023-12-26 19:26:52,403][105620] Updated weights for policy 1, policy_version 561247 (0.0009) [2023-12-26 19:26:52,419][105692] Updated weights for policy 0, policy_version 560453 (0.0008) [2023-12-26 19:26:52,474][105692] Updated weights for policy 0, policy_version 560463 (0.0008) [2023-12-26 19:26:52,537][105692] Updated weights for policy 0, policy_version 560473 (0.0009) [2023-12-26 19:26:53,121][105620] Updated weights for policy 1, policy_version 561257 (0.0007) [2023-12-26 19:26:53,171][105620] Updated weights for policy 1, policy_version 561267 (0.0009) [2023-12-26 19:26:53,232][105620] Updated weights for policy 1, policy_version 561277 (0.0009) [2023-12-26 19:26:53,280][105620] Updated weights for policy 1, policy_version 561287 (0.0008) [2023-12-26 19:26:53,289][105692] Updated weights for policy 0, policy_version 560483 (0.0010) [2023-12-26 19:26:53,346][105692] Updated weights for policy 0, policy_version 560493 (0.0009) [2023-12-26 19:26:53,405][105692] Updated weights for policy 0, policy_version 560503 (0.0005) [2023-12-26 19:26:53,935][105692] Updated weights for policy 0, policy_version 560513 (0.0005) [2023-12-26 19:26:53,987][105692] Updated weights for policy 0, policy_version 560523 (0.0007) [2023-12-26 19:26:54,034][105692] Updated weights for policy 0, policy_version 560533 (0.0009) [2023-12-26 19:26:54,076][105692] Updated weights for policy 0, policy_version 560543 (0.0007) [2023-12-26 19:26:54,147][105620] Updated weights for policy 1, policy_version 561297 (0.0005) [2023-12-26 19:26:54,208][105620] Updated weights for policy 1, policy_version 561307 (0.0009) [2023-12-26 19:26:54,266][105620] Updated weights for policy 1, policy_version 561317 (0.0010) [2023-12-26 19:26:54,811][105620] Updated weights for policy 1, policy_version 561327 (0.0010) [2023-12-26 19:26:54,860][105620] Updated weights for policy 1, policy_version 561337 (0.0010) [2023-12-26 19:26:54,862][105692] Updated weights for policy 0, policy_version 560553 (0.0007) [2023-12-26 19:26:54,911][105620] Updated weights for policy 1, policy_version 561347 (0.0010) [2023-12-26 19:26:54,913][105692] Updated weights for policy 0, policy_version 560563 (0.0005) [2023-12-26 19:26:54,966][105692] Updated weights for policy 0, policy_version 560573 (0.0008) [2023-12-26 19:26:55,685][105620] Updated weights for policy 1, policy_version 561357 (0.0009) [2023-12-26 19:26:55,702][105692] Updated weights for policy 0, policy_version 560583 (0.0006) [2023-12-26 19:26:55,742][105620] Updated weights for policy 1, policy_version 561367 (0.0010) [2023-12-26 19:26:55,762][105692] Updated weights for policy 0, policy_version 560593 (0.0007) [2023-12-26 19:26:55,799][105620] Updated weights for policy 1, policy_version 561377 (0.0008) [2023-12-26 19:26:55,825][105692] Updated weights for policy 0, policy_version 560603 (0.0007) [2023-12-26 19:26:56,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 287260672. Throughput: 0: 9944.5, 1: 9466.6. Samples: 287265320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:26:56,062][104569] Avg episode reward: [(0, '8917.731'), (1, '8874.368')] [2023-12-26 19:26:56,423][105620] Updated weights for policy 1, policy_version 561387 (0.0009) [2023-12-26 19:26:56,480][105620] Updated weights for policy 1, policy_version 561397 (0.0005) [2023-12-26 19:26:56,540][105620] Updated weights for policy 1, policy_version 561407 (0.0005) [2023-12-26 19:26:56,638][105692] Updated weights for policy 0, policy_version 560613 (0.0008) [2023-12-26 19:26:56,691][105692] Updated weights for policy 0, policy_version 560624 (0.0010) [2023-12-26 19:26:56,743][105692] Updated weights for policy 0, policy_version 560634 (0.0009) [2023-12-26 19:26:57,041][105620] Updated weights for policy 1, policy_version 561417 (0.0005) [2023-12-26 19:26:57,104][105620] Updated weights for policy 1, policy_version 561427 (0.0006) [2023-12-26 19:26:57,165][105620] Updated weights for policy 1, policy_version 561437 (0.0008) [2023-12-26 19:26:57,211][105620] Updated weights for policy 1, policy_version 561447 (0.0005) [2023-12-26 19:26:57,635][105692] Updated weights for policy 0, policy_version 560644 (0.0008) [2023-12-26 19:26:57,699][105692] Updated weights for policy 0, policy_version 560654 (0.0008) [2023-12-26 19:26:57,753][105692] Updated weights for policy 0, policy_version 560664 (0.0010) [2023-12-26 19:26:57,759][105620] Updated weights for policy 1, policy_version 561457 (0.0007) [2023-12-26 19:26:57,815][105620] Updated weights for policy 1, policy_version 561467 (0.0005) [2023-12-26 19:26:57,870][105620] Updated weights for policy 1, policy_version 561477 (0.0005) [2023-12-26 19:26:58,491][105692] Updated weights for policy 0, policy_version 560674 (0.0010) [2023-12-26 19:26:58,541][105620] Updated weights for policy 1, policy_version 561487 (0.0007) [2023-12-26 19:26:58,555][105692] Updated weights for policy 0, policy_version 560684 (0.0008) [2023-12-26 19:26:58,613][105620] Updated weights for policy 1, policy_version 561497 (0.0007) [2023-12-26 19:26:58,623][105692] Updated weights for policy 0, policy_version 560694 (0.0009) [2023-12-26 19:26:58,671][105620] Updated weights for policy 1, policy_version 561507 (0.0009) [2023-12-26 19:26:58,681][105692] Updated weights for policy 0, policy_version 560704 (0.0009) [2023-12-26 19:26:59,512][105620] Updated weights for policy 1, policy_version 561517 (0.0007) [2023-12-26 19:26:59,530][105692] Updated weights for policy 0, policy_version 560714 (0.0008) [2023-12-26 19:26:59,570][105620] Updated weights for policy 1, policy_version 561527 (0.0009) [2023-12-26 19:26:59,592][105692] Updated weights for policy 0, policy_version 560724 (0.0006) [2023-12-26 19:26:59,627][105585] KL-divergence is very high: 121.7027 [2023-12-26 19:26:59,632][105620] Updated weights for policy 1, policy_version 561537 (0.0010) [2023-12-26 19:26:59,645][105692] Updated weights for policy 0, policy_version 560734 (0.0009) [2023-12-26 19:27:00,278][105620] Updated weights for policy 1, policy_version 561547 (0.0010) [2023-12-26 19:27:00,346][105620] Updated weights for policy 1, policy_version 561557 (0.0010) [2023-12-26 19:27:00,404][105620] Updated weights for policy 1, policy_version 561567 (0.0010) [2023-12-26 19:27:00,429][105692] Updated weights for policy 0, policy_version 560744 (0.0006) [2023-12-26 19:27:00,483][105692] Updated weights for policy 0, policy_version 560754 (0.0005) [2023-12-26 19:27:00,542][105692] Updated weights for policy 0, policy_version 560764 (0.0007) [2023-12-26 19:27:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 287350784. Throughput: 0: 9927.3, 1: 9579.9. Samples: 287325484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:27:01,062][104569] Avg episode reward: [(0, '9174.408'), (1, '8883.995')] [2023-12-26 19:27:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000560768_143572992.pth... [2023-12-26 19:27:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000561576_143777792.pth... [2023-12-26 19:27:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000559616_143278080.pth [2023-12-26 19:27:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000560456_143491072.pth [2023-12-26 19:27:01,159][105692] Updated weights for policy 0, policy_version 560774 (0.0008) [2023-12-26 19:27:01,163][105620] Updated weights for policy 1, policy_version 561577 (0.0010) [2023-12-26 19:27:01,216][105692] Updated weights for policy 0, policy_version 560784 (0.0006) [2023-12-26 19:27:01,218][105620] Updated weights for policy 1, policy_version 561587 (0.0010) [2023-12-26 19:27:01,276][105620] Updated weights for policy 1, policy_version 561597 (0.0009) [2023-12-26 19:27:01,277][105692] Updated weights for policy 0, policy_version 560794 (0.0007) [2023-12-26 19:27:01,328][105620] Updated weights for policy 1, policy_version 561607 (0.0010) [2023-12-26 19:27:02,007][105692] Updated weights for policy 0, policy_version 560804 (0.0008) [2023-12-26 19:27:02,060][105692] Updated weights for policy 0, policy_version 560814 (0.0009) [2023-12-26 19:27:02,099][105620] Updated weights for policy 1, policy_version 561617 (0.0006) [2023-12-26 19:27:02,113][105692] Updated weights for policy 0, policy_version 560824 (0.0008) [2023-12-26 19:27:02,157][105620] Updated weights for policy 1, policy_version 561627 (0.0005) [2023-12-26 19:27:02,218][105620] Updated weights for policy 1, policy_version 561637 (0.0005) [2023-12-26 19:27:02,914][105620] Updated weights for policy 1, policy_version 561647 (0.0008) [2023-12-26 19:27:02,920][105692] Updated weights for policy 0, policy_version 560834 (0.0008) [2023-12-26 19:27:02,972][105692] Updated weights for policy 0, policy_version 560844 (0.0006) [2023-12-26 19:27:02,974][105620] Updated weights for policy 1, policy_version 561657 (0.0007) [2023-12-26 19:27:03,020][105692] Updated weights for policy 0, policy_version 560854 (0.0006) [2023-12-26 19:27:03,034][105620] Updated weights for policy 1, policy_version 561667 (0.0008) [2023-12-26 19:27:03,066][105692] Updated weights for policy 0, policy_version 560864 (0.0007) [2023-12-26 19:27:03,703][105620] Updated weights for policy 1, policy_version 561677 (0.0007) [2023-12-26 19:27:03,715][105692] Updated weights for policy 0, policy_version 560874 (0.0010) [2023-12-26 19:27:03,762][105620] Updated weights for policy 1, policy_version 561687 (0.0007) [2023-12-26 19:27:03,770][105692] Updated weights for policy 0, policy_version 560884 (0.0010) [2023-12-26 19:27:03,814][105620] Updated weights for policy 1, policy_version 561697 (0.0010) [2023-12-26 19:27:03,827][105692] Updated weights for policy 0, policy_version 560894 (0.0009) [2023-12-26 19:27:04,478][105692] Updated weights for policy 0, policy_version 560904 (0.0006) [2023-12-26 19:27:04,526][105620] Updated weights for policy 1, policy_version 561707 (0.0008) [2023-12-26 19:27:04,542][105692] Updated weights for policy 0, policy_version 560914 (0.0007) [2023-12-26 19:27:04,582][105620] Updated weights for policy 1, policy_version 561717 (0.0007) [2023-12-26 19:27:04,608][105692] Updated weights for policy 0, policy_version 560924 (0.0007) [2023-12-26 19:27:04,633][105620] Updated weights for policy 1, policy_version 561728 (0.0010) [2023-12-26 19:27:05,266][105692] Updated weights for policy 0, policy_version 560934 (0.0005) [2023-12-26 19:27:05,320][105620] Updated weights for policy 1, policy_version 561739 (0.0009) [2023-12-26 19:27:05,323][105692] Updated weights for policy 0, policy_version 560944 (0.0005) [2023-12-26 19:27:05,374][105620] Updated weights for policy 1, policy_version 561749 (0.0006) [2023-12-26 19:27:05,390][105692] Updated weights for policy 0, policy_version 560954 (0.0006) [2023-12-26 19:27:05,426][105620] Updated weights for policy 1, policy_version 561759 (0.0005) [2023-12-26 19:27:05,930][105692] Updated weights for policy 0, policy_version 560964 (0.0005) [2023-12-26 19:27:05,991][105692] Updated weights for policy 0, policy_version 560974 (0.0006) [2023-12-26 19:27:06,054][105692] Updated weights for policy 0, policy_version 560984 (0.0009) [2023-12-26 19:27:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 287449088. Throughput: 0: 9874.8, 1: 9636.0. Samples: 287441144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:27:06,062][104569] Avg episode reward: [(0, '9077.544'), (1, '9271.051')] [2023-12-26 19:27:06,076][105620] Updated weights for policy 1, policy_version 561769 (0.0006) [2023-12-26 19:27:06,140][105620] Updated weights for policy 1, policy_version 561779 (0.0010) [2023-12-26 19:27:06,196][105620] Updated weights for policy 1, policy_version 561789 (0.0011) [2023-12-26 19:27:06,260][105620] Updated weights for policy 1, policy_version 561799 (0.0011) [2023-12-26 19:27:06,675][105692] Updated weights for policy 0, policy_version 560994 (0.0008) [2023-12-26 19:27:06,740][105692] Updated weights for policy 0, policy_version 561004 (0.0009) [2023-12-26 19:27:06,799][105692] Updated weights for policy 0, policy_version 561014 (0.0011) [2023-12-26 19:27:06,854][105692] Updated weights for policy 0, policy_version 561024 (0.0011) [2023-12-26 19:27:06,972][105620] Updated weights for policy 1, policy_version 561809 (0.0007) [2023-12-26 19:27:07,031][105620] Updated weights for policy 1, policy_version 561819 (0.0008) [2023-12-26 19:27:07,083][105620] Updated weights for policy 1, policy_version 561829 (0.0008) [2023-12-26 19:27:07,585][105692] Updated weights for policy 0, policy_version 561034 (0.0011) [2023-12-26 19:27:07,649][105692] Updated weights for policy 0, policy_version 561044 (0.0009) [2023-12-26 19:27:07,705][105692] Updated weights for policy 0, policy_version 561054 (0.0011) [2023-12-26 19:27:07,848][105620] Updated weights for policy 1, policy_version 561839 (0.0007) [2023-12-26 19:27:07,912][105620] Updated weights for policy 1, policy_version 561849 (0.0008) [2023-12-26 19:27:07,964][105620] Updated weights for policy 1, policy_version 561859 (0.0008) [2023-12-26 19:27:08,431][105692] Updated weights for policy 0, policy_version 561064 (0.0011) [2023-12-26 19:27:08,488][105692] Updated weights for policy 0, policy_version 561074 (0.0008) [2023-12-26 19:27:08,534][105692] Updated weights for policy 0, policy_version 561084 (0.0005) [2023-12-26 19:27:08,676][105620] Updated weights for policy 1, policy_version 561869 (0.0008) [2023-12-26 19:27:08,728][105620] Updated weights for policy 1, policy_version 561879 (0.0008) [2023-12-26 19:27:08,786][105620] Updated weights for policy 1, policy_version 561889 (0.0007) [2023-12-26 19:27:09,258][105692] Updated weights for policy 0, policy_version 561094 (0.0009) [2023-12-26 19:27:09,313][105692] Updated weights for policy 0, policy_version 561104 (0.0011) [2023-12-26 19:27:09,378][105692] Updated weights for policy 0, policy_version 561114 (0.0011) [2023-12-26 19:27:09,571][105620] Updated weights for policy 1, policy_version 561899 (0.0008) [2023-12-26 19:27:09,630][105620] Updated weights for policy 1, policy_version 561909 (0.0008) [2023-12-26 19:27:09,687][105620] Updated weights for policy 1, policy_version 561919 (0.0008) [2023-12-26 19:27:10,149][105692] Updated weights for policy 0, policy_version 561124 (0.0011) [2023-12-26 19:27:10,209][105692] Updated weights for policy 0, policy_version 561134 (0.0011) [2023-12-26 19:27:10,272][105692] Updated weights for policy 0, policy_version 561144 (0.0011) [2023-12-26 19:27:10,461][105620] Updated weights for policy 1, policy_version 561929 (0.0008) [2023-12-26 19:27:10,510][105620] Updated weights for policy 1, policy_version 561939 (0.0009) [2023-12-26 19:27:10,563][105620] Updated weights for policy 1, policy_version 561949 (0.0008) [2023-12-26 19:27:10,623][105620] Updated weights for policy 1, policy_version 561959 (0.0008) [2023-12-26 19:27:11,028][105692] Updated weights for policy 0, policy_version 561154 (0.0011) [2023-12-26 19:27:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 287547392. Throughput: 0: 9876.7, 1: 9688.1. Samples: 287559260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:27:11,063][104569] Avg episode reward: [(0, '8984.403'), (1, '9262.225')] [2023-12-26 19:27:11,092][105692] Updated weights for policy 0, policy_version 561164 (0.0014) [2023-12-26 19:27:11,149][105692] Updated weights for policy 0, policy_version 561174 (0.0010) [2023-12-26 19:27:11,204][105692] Updated weights for policy 0, policy_version 561184 (0.0011) [2023-12-26 19:27:11,433][105620] Updated weights for policy 1, policy_version 561969 (0.0008) [2023-12-26 19:27:11,502][105620] Updated weights for policy 1, policy_version 561979 (0.0008) [2023-12-26 19:27:11,563][105620] Updated weights for policy 1, policy_version 561989 (0.0008) [2023-12-26 19:27:11,977][105692] Updated weights for policy 0, policy_version 561194 (0.0008) [2023-12-26 19:27:12,041][105692] Updated weights for policy 0, policy_version 561204 (0.0011) [2023-12-26 19:27:12,091][105692] Updated weights for policy 0, policy_version 561214 (0.0011) [2023-12-26 19:27:12,259][105620] Updated weights for policy 1, policy_version 561999 (0.0010) [2023-12-26 19:27:12,323][105620] Updated weights for policy 1, policy_version 562009 (0.0011) [2023-12-26 19:27:12,389][105620] Updated weights for policy 1, policy_version 562019 (0.0010) [2023-12-26 19:27:12,815][105692] Updated weights for policy 0, policy_version 561224 (0.0009) [2023-12-26 19:27:12,875][105692] Updated weights for policy 0, policy_version 561234 (0.0007) [2023-12-26 19:27:12,940][105692] Updated weights for policy 0, policy_version 561244 (0.0007) [2023-12-26 19:27:13,005][105620] Updated weights for policy 1, policy_version 562029 (0.0008) [2023-12-26 19:27:13,049][105620] Updated weights for policy 1, policy_version 562039 (0.0008) [2023-12-26 19:27:13,094][105620] Updated weights for policy 1, policy_version 562049 (0.0008) [2023-12-26 19:27:13,643][105692] Updated weights for policy 0, policy_version 561254 (0.0010) [2023-12-26 19:27:13,691][105692] Updated weights for policy 0, policy_version 561264 (0.0010) [2023-12-26 19:27:13,738][105692] Updated weights for policy 0, policy_version 561274 (0.0010) [2023-12-26 19:27:13,878][105620] Updated weights for policy 1, policy_version 562059 (0.0008) [2023-12-26 19:27:13,923][105620] Updated weights for policy 1, policy_version 562069 (0.0008) [2023-12-26 19:27:13,975][105620] Updated weights for policy 1, policy_version 562079 (0.0008) [2023-12-26 19:27:14,495][105692] Updated weights for policy 0, policy_version 561284 (0.0011) [2023-12-26 19:27:14,545][105692] Updated weights for policy 0, policy_version 561294 (0.0010) [2023-12-26 19:27:14,600][105692] Updated weights for policy 0, policy_version 561304 (0.0010) [2023-12-26 19:27:14,738][105620] Updated weights for policy 1, policy_version 562089 (0.0008) [2023-12-26 19:27:14,800][105620] Updated weights for policy 1, policy_version 562099 (0.0007) [2023-12-26 19:27:14,860][105620] Updated weights for policy 1, policy_version 562109 (0.0008) [2023-12-26 19:27:14,917][105620] Updated weights for policy 1, policy_version 562119 (0.0008) [2023-12-26 19:27:15,375][105692] Updated weights for policy 0, policy_version 561314 (0.0008) [2023-12-26 19:27:15,427][105692] Updated weights for policy 0, policy_version 561324 (0.0010) [2023-12-26 19:27:15,485][105692] Updated weights for policy 0, policy_version 561334 (0.0011) [2023-12-26 19:27:15,550][105692] Updated weights for policy 0, policy_version 561344 (0.0010) [2023-12-26 19:27:15,682][105620] Updated weights for policy 1, policy_version 562129 (0.0008) [2023-12-26 19:27:15,740][105620] Updated weights for policy 1, policy_version 562139 (0.0008) [2023-12-26 19:27:15,795][105620] Updated weights for policy 1, policy_version 562149 (0.0008) [2023-12-26 19:27:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 287645696. Throughput: 0: 9843.2, 1: 9641.2. Samples: 287616824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:27:16,062][104569] Avg episode reward: [(0, '8893.675'), (1, '9262.399')] [2023-12-26 19:27:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000562152_143925248.pth... [2023-12-26 19:27:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000561344_143720448.pth... [2023-12-26 19:27:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000561032_143638528.pth [2023-12-26 19:27:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000560224_143433728.pth [2023-12-26 19:27:16,286][105692] Updated weights for policy 0, policy_version 561354 (0.0011) [2023-12-26 19:27:16,338][105692] Updated weights for policy 0, policy_version 561364 (0.0011) [2023-12-26 19:27:16,397][105692] Updated weights for policy 0, policy_version 561374 (0.0011) [2023-12-26 19:27:16,558][105620] Updated weights for policy 1, policy_version 562159 (0.0008) [2023-12-26 19:27:16,622][105620] Updated weights for policy 1, policy_version 562169 (0.0008) [2023-12-26 19:27:16,682][105620] Updated weights for policy 1, policy_version 562179 (0.0008) [2023-12-26 19:27:17,151][105692] Updated weights for policy 0, policy_version 561384 (0.0010) [2023-12-26 19:27:17,198][105692] Updated weights for policy 0, policy_version 561394 (0.0010) [2023-12-26 19:27:17,253][105692] Updated weights for policy 0, policy_version 561404 (0.0010) [2023-12-26 19:27:17,432][105620] Updated weights for policy 1, policy_version 562189 (0.0009) [2023-12-26 19:27:17,482][105620] Updated weights for policy 1, policy_version 562199 (0.0010) [2023-12-26 19:27:17,550][105620] Updated weights for policy 1, policy_version 562209 (0.0010) [2023-12-26 19:27:17,980][105692] Updated weights for policy 0, policy_version 561414 (0.0007) [2023-12-26 19:27:18,034][105692] Updated weights for policy 0, policy_version 561424 (0.0005) [2023-12-26 19:27:18,085][105692] Updated weights for policy 0, policy_version 561434 (0.0005) [2023-12-26 19:27:18,260][105620] Updated weights for policy 1, policy_version 562219 (0.0010) [2023-12-26 19:27:18,328][105620] Updated weights for policy 1, policy_version 562229 (0.0011) [2023-12-26 19:27:18,393][105620] Updated weights for policy 1, policy_version 562239 (0.0009) [2023-12-26 19:27:18,726][105692] Updated weights for policy 0, policy_version 561444 (0.0007) [2023-12-26 19:27:18,791][105692] Updated weights for policy 0, policy_version 561454 (0.0008) [2023-12-26 19:27:18,857][105692] Updated weights for policy 0, policy_version 561464 (0.0008) [2023-12-26 19:27:19,117][105620] Updated weights for policy 1, policy_version 562249 (0.0010) [2023-12-26 19:27:19,178][105620] Updated weights for policy 1, policy_version 562259 (0.0010) [2023-12-26 19:27:19,241][105620] Updated weights for policy 1, policy_version 562269 (0.0010) [2023-12-26 19:27:19,303][105620] Updated weights for policy 1, policy_version 562279 (0.0010) [2023-12-26 19:27:19,667][105692] Updated weights for policy 0, policy_version 561474 (0.0008) [2023-12-26 19:27:19,731][105692] Updated weights for policy 0, policy_version 561484 (0.0007) [2023-12-26 19:27:19,804][105692] Updated weights for policy 0, policy_version 561494 (0.0006) [2023-12-26 19:27:19,871][105692] Updated weights for policy 0, policy_version 561504 (0.0008) [2023-12-26 19:27:19,998][105620] Updated weights for policy 1, policy_version 562289 (0.0010) [2023-12-26 19:27:20,055][105620] Updated weights for policy 1, policy_version 562299 (0.0011) [2023-12-26 19:27:20,108][105620] Updated weights for policy 1, policy_version 562309 (0.0011) [2023-12-26 19:27:20,627][105692] Updated weights for policy 0, policy_version 561514 (0.0009) [2023-12-26 19:27:20,674][105692] Updated weights for policy 0, policy_version 561524 (0.0008) [2023-12-26 19:27:20,743][105692] Updated weights for policy 0, policy_version 561534 (0.0009) [2023-12-26 19:27:20,794][105620] Updated weights for policy 1, policy_version 562319 (0.0009) [2023-12-26 19:27:20,853][105620] Updated weights for policy 1, policy_version 562329 (0.0009) [2023-12-26 19:27:20,922][105620] Updated weights for policy 1, policy_version 562339 (0.0009) [2023-12-26 19:27:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 287744000. Throughput: 0: 9797.4, 1: 9650.5. Samples: 287730536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:27:21,063][104569] Avg episode reward: [(0, '9262.818'), (1, '9262.019')] [2023-12-26 19:27:21,546][105692] Updated weights for policy 0, policy_version 561544 (0.0009) [2023-12-26 19:27:21,616][105692] Updated weights for policy 0, policy_version 561554 (0.0009) [2023-12-26 19:27:21,648][105620] Updated weights for policy 1, policy_version 562349 (0.0009) [2023-12-26 19:27:21,684][105692] Updated weights for policy 0, policy_version 561564 (0.0006) [2023-12-26 19:27:21,720][105620] Updated weights for policy 1, policy_version 562359 (0.0008) [2023-12-26 19:27:21,788][105620] Updated weights for policy 1, policy_version 562369 (0.0008) [2023-12-26 19:27:22,444][105692] Updated weights for policy 0, policy_version 561574 (0.0008) [2023-12-26 19:27:22,503][105692] Updated weights for policy 0, policy_version 561584 (0.0008) [2023-12-26 19:27:22,540][105620] Updated weights for policy 1, policy_version 562379 (0.0009) [2023-12-26 19:27:22,562][105692] Updated weights for policy 0, policy_version 561594 (0.0007) [2023-12-26 19:27:22,599][105620] Updated weights for policy 1, policy_version 562389 (0.0010) [2023-12-26 19:27:22,659][105620] Updated weights for policy 1, policy_version 562399 (0.0011) [2023-12-26 19:27:23,311][105692] Updated weights for policy 0, policy_version 561604 (0.0008) [2023-12-26 19:27:23,369][105692] Updated weights for policy 0, policy_version 561614 (0.0005) [2023-12-26 19:27:23,377][105620] Updated weights for policy 1, policy_version 562409 (0.0010) [2023-12-26 19:27:23,421][105692] Updated weights for policy 0, policy_version 561624 (0.0005) [2023-12-26 19:27:23,434][105620] Updated weights for policy 1, policy_version 562419 (0.0005) [2023-12-26 19:27:23,491][105620] Updated weights for policy 1, policy_version 562429 (0.0005) [2023-12-26 19:27:23,548][105620] Updated weights for policy 1, policy_version 562439 (0.0009) [2023-12-26 19:27:24,049][105692] Updated weights for policy 0, policy_version 561634 (0.0009) [2023-12-26 19:27:24,114][105692] Updated weights for policy 0, policy_version 561644 (0.0010) [2023-12-26 19:27:24,176][105692] Updated weights for policy 0, policy_version 561654 (0.0011) [2023-12-26 19:27:24,182][105620] Updated weights for policy 1, policy_version 562449 (0.0006) [2023-12-26 19:27:24,240][105620] Updated weights for policy 1, policy_version 562459 (0.0005) [2023-12-26 19:27:24,242][105692] Updated weights for policy 0, policy_version 561664 (0.0010) [2023-12-26 19:27:24,296][105620] Updated weights for policy 1, policy_version 562469 (0.0005) [2023-12-26 19:27:24,910][105692] Updated weights for policy 0, policy_version 561674 (0.0010) [2023-12-26 19:27:24,932][105620] Updated weights for policy 1, policy_version 562479 (0.0005) [2023-12-26 19:27:24,973][105692] Updated weights for policy 0, policy_version 561684 (0.0010) [2023-12-26 19:27:24,991][105620] Updated weights for policy 1, policy_version 562489 (0.0006) [2023-12-26 19:27:25,033][105692] Updated weights for policy 0, policy_version 561694 (0.0010) [2023-12-26 19:27:25,058][105620] Updated weights for policy 1, policy_version 562499 (0.0005) [2023-12-26 19:27:25,575][105620] Updated weights for policy 1, policy_version 562509 (0.0005) [2023-12-26 19:27:25,626][105620] Updated weights for policy 1, policy_version 562519 (0.0005) [2023-12-26 19:27:25,675][105620] Updated weights for policy 1, policy_version 562529 (0.0006) [2023-12-26 19:27:25,693][105692] Updated weights for policy 0, policy_version 561704 (0.0011) [2023-12-26 19:27:25,746][105692] Updated weights for policy 0, policy_version 561714 (0.0010) [2023-12-26 19:27:25,796][105692] Updated weights for policy 0, policy_version 561724 (0.0005) [2023-12-26 19:27:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 287842304. Throughput: 0: 9827.7, 1: 9707.0. Samples: 287849372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:27:26,062][104569] Avg episode reward: [(0, '9169.053'), (1, '9083.435')] [2023-12-26 19:27:26,386][105692] Updated weights for policy 0, policy_version 561734 (0.0008) [2023-12-26 19:27:26,430][105692] Updated weights for policy 0, policy_version 561744 (0.0010) [2023-12-26 19:27:26,455][105620] Updated weights for policy 1, policy_version 562539 (0.0006) [2023-12-26 19:27:26,469][105585] KL-divergence is very high: 103.4686 [2023-12-26 19:27:26,481][105692] Updated weights for policy 0, policy_version 561754 (0.0010) [2023-12-26 19:27:26,507][105620] Updated weights for policy 1, policy_version 562549 (0.0006) [2023-12-26 19:27:26,565][105620] Updated weights for policy 1, policy_version 562559 (0.0010) [2023-12-26 19:27:27,236][105620] Updated weights for policy 1, policy_version 562569 (0.0010) [2023-12-26 19:27:27,238][105692] Updated weights for policy 0, policy_version 561764 (0.0010) [2023-12-26 19:27:27,284][105620] Updated weights for policy 1, policy_version 562579 (0.0005) [2023-12-26 19:27:27,285][105692] Updated weights for policy 0, policy_version 561774 (0.0010) [2023-12-26 19:27:27,342][105692] Updated weights for policy 0, policy_version 561784 (0.0011) [2023-12-26 19:27:27,344][105620] Updated weights for policy 1, policy_version 562589 (0.0006) [2023-12-26 19:27:27,412][105620] Updated weights for policy 1, policy_version 562599 (0.0005) [2023-12-26 19:27:27,987][105620] Updated weights for policy 1, policy_version 562609 (0.0005) [2023-12-26 19:27:28,046][105620] Updated weights for policy 1, policy_version 562619 (0.0008) [2023-12-26 19:27:28,063][105692] Updated weights for policy 0, policy_version 561794 (0.0009) [2023-12-26 19:27:28,103][105620] Updated weights for policy 1, policy_version 562629 (0.0008) [2023-12-26 19:27:28,117][105692] Updated weights for policy 0, policy_version 561804 (0.0010) [2023-12-26 19:27:28,171][105692] Updated weights for policy 0, policy_version 561814 (0.0010) [2023-12-26 19:27:28,238][105692] Updated weights for policy 0, policy_version 561824 (0.0010) [2023-12-26 19:27:28,802][105620] Updated weights for policy 1, policy_version 562639 (0.0007) [2023-12-26 19:27:28,850][105620] Updated weights for policy 1, policy_version 562649 (0.0008) [2023-12-26 19:27:28,898][105620] Updated weights for policy 1, policy_version 562659 (0.0008) [2023-12-26 19:27:28,971][105692] Updated weights for policy 0, policy_version 561834 (0.0010) [2023-12-26 19:27:29,022][105692] Updated weights for policy 0, policy_version 561844 (0.0010) [2023-12-26 19:27:29,070][105692] Updated weights for policy 0, policy_version 561854 (0.0010) [2023-12-26 19:27:29,617][105620] Updated weights for policy 1, policy_version 562669 (0.0007) [2023-12-26 19:27:29,667][105620] Updated weights for policy 1, policy_version 562679 (0.0008) [2023-12-26 19:27:29,718][105620] Updated weights for policy 1, policy_version 562689 (0.0006) [2023-12-26 19:27:29,846][105692] Updated weights for policy 0, policy_version 561864 (0.0008) [2023-12-26 19:27:29,897][105692] Updated weights for policy 0, policy_version 561874 (0.0008) [2023-12-26 19:27:29,962][105692] Updated weights for policy 0, policy_version 561884 (0.0008) [2023-12-26 19:27:30,376][105620] Updated weights for policy 1, policy_version 562699 (0.0007) [2023-12-26 19:27:30,437][105620] Updated weights for policy 1, policy_version 562709 (0.0005) [2023-12-26 19:27:30,495][105620] Updated weights for policy 1, policy_version 562719 (0.0005) [2023-12-26 19:27:30,730][105692] Updated weights for policy 0, policy_version 561894 (0.0008) [2023-12-26 19:27:30,791][105692] Updated weights for policy 0, policy_version 561904 (0.0008) [2023-12-26 19:27:30,852][105692] Updated weights for policy 0, policy_version 561915 (0.0009) [2023-12-26 19:27:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 287940608. Throughput: 0: 9817.7, 1: 9740.5. Samples: 287909788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:27:31,063][104569] Avg episode reward: [(0, '9169.291'), (1, '9083.261')] [2023-12-26 19:27:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000561920_143867904.pth... [2023-12-26 19:27:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000562728_144072704.pth... [2023-12-26 19:27:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000560768_143572992.pth [2023-12-26 19:27:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000561576_143777792.pth [2023-12-26 19:27:31,120][105620] Updated weights for policy 1, policy_version 562729 (0.0006) [2023-12-26 19:27:31,195][105620] Updated weights for policy 1, policy_version 562739 (0.0010) [2023-12-26 19:27:31,260][105620] Updated weights for policy 1, policy_version 562749 (0.0008) [2023-12-26 19:27:31,332][105620] Updated weights for policy 1, policy_version 562759 (0.0007) [2023-12-26 19:27:31,529][105692] Updated weights for policy 0, policy_version 561925 (0.0007) [2023-12-26 19:27:31,592][105692] Updated weights for policy 0, policy_version 561935 (0.0007) [2023-12-26 19:27:31,659][105692] Updated weights for policy 0, policy_version 561945 (0.0007) [2023-12-26 19:27:31,968][105620] Updated weights for policy 1, policy_version 562769 (0.0006) [2023-12-26 19:27:32,024][105620] Updated weights for policy 1, policy_version 562779 (0.0005) [2023-12-26 19:27:32,071][105620] Updated weights for policy 1, policy_version 562789 (0.0005) [2023-12-26 19:27:32,463][105692] Updated weights for policy 0, policy_version 561955 (0.0008) [2023-12-26 19:27:32,523][105692] Updated weights for policy 0, policy_version 561965 (0.0011) [2023-12-26 19:27:32,589][105692] Updated weights for policy 0, policy_version 561975 (0.0010) [2023-12-26 19:27:32,728][105620] Updated weights for policy 1, policy_version 562799 (0.0008) [2023-12-26 19:27:32,793][105620] Updated weights for policy 1, policy_version 562809 (0.0009) [2023-12-26 19:27:32,854][105620] Updated weights for policy 1, policy_version 562819 (0.0008) [2023-12-26 19:27:33,183][105692] Updated weights for policy 0, policy_version 561985 (0.0005) [2023-12-26 19:27:33,243][105692] Updated weights for policy 0, policy_version 561995 (0.0005) [2023-12-26 19:27:33,312][105692] Updated weights for policy 0, policy_version 562005 (0.0005) [2023-12-26 19:27:33,367][105692] Updated weights for policy 0, policy_version 562015 (0.0005) [2023-12-26 19:27:33,645][105620] Updated weights for policy 1, policy_version 562829 (0.0007) [2023-12-26 19:27:33,701][105620] Updated weights for policy 1, policy_version 562839 (0.0005) [2023-12-26 19:27:33,755][105620] Updated weights for policy 1, policy_version 562849 (0.0005) [2023-12-26 19:27:33,991][105692] Updated weights for policy 0, policy_version 562025 (0.0010) [2023-12-26 19:27:34,055][105692] Updated weights for policy 0, policy_version 562035 (0.0009) [2023-12-26 19:27:34,125][105692] Updated weights for policy 0, policy_version 562045 (0.0009) [2023-12-26 19:27:34,360][105620] Updated weights for policy 1, policy_version 562859 (0.0007) [2023-12-26 19:27:34,427][105620] Updated weights for policy 1, policy_version 562869 (0.0010) [2023-12-26 19:27:34,486][105620] Updated weights for policy 1, policy_version 562879 (0.0011) [2023-12-26 19:27:34,827][105692] Updated weights for policy 0, policy_version 562055 (0.0010) [2023-12-26 19:27:34,882][105692] Updated weights for policy 0, policy_version 562065 (0.0010) [2023-12-26 19:27:34,944][105692] Updated weights for policy 0, policy_version 562075 (0.0010) [2023-12-26 19:27:35,147][105620] Updated weights for policy 1, policy_version 562889 (0.0010) [2023-12-26 19:27:35,212][105620] Updated weights for policy 1, policy_version 562899 (0.0010) [2023-12-26 19:27:35,273][105620] Updated weights for policy 1, policy_version 562909 (0.0010) [2023-12-26 19:27:35,333][105620] Updated weights for policy 1, policy_version 562919 (0.0005) [2023-12-26 19:27:35,696][105692] Updated weights for policy 0, policy_version 562085 (0.0010) [2023-12-26 19:27:35,761][105692] Updated weights for policy 0, policy_version 562095 (0.0011) [2023-12-26 19:27:35,820][105692] Updated weights for policy 0, policy_version 562105 (0.0010) [2023-12-26 19:27:35,871][105620] Updated weights for policy 1, policy_version 562929 (0.0005) [2023-12-26 19:27:35,920][105620] Updated weights for policy 1, policy_version 562939 (0.0005) [2023-12-26 19:27:35,975][105620] Updated weights for policy 1, policy_version 562949 (0.0010) [2023-12-26 19:27:36,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 288047104. Throughput: 0: 9638.8, 1: 9918.4. Samples: 288029904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:27:36,063][104569] Avg episode reward: [(0, '9353.177'), (1, '9259.911')] [2023-12-26 19:27:36,499][105692] Updated weights for policy 0, policy_version 562115 (0.0011) [2023-12-26 19:27:36,558][105692] Updated weights for policy 0, policy_version 562125 (0.0011) [2023-12-26 19:27:36,622][105692] Updated weights for policy 0, policy_version 562135 (0.0011) [2023-12-26 19:27:36,654][105620] Updated weights for policy 1, policy_version 562959 (0.0010) [2023-12-26 19:27:36,716][105620] Updated weights for policy 1, policy_version 562969 (0.0010) [2023-12-26 19:27:36,768][105620] Updated weights for policy 1, policy_version 562979 (0.0010) [2023-12-26 19:27:37,242][105692] Updated weights for policy 0, policy_version 562145 (0.0010) [2023-12-26 19:27:37,295][105692] Updated weights for policy 0, policy_version 562155 (0.0008) [2023-12-26 19:27:37,355][105692] Updated weights for policy 0, policy_version 562165 (0.0009) [2023-12-26 19:27:37,411][105692] Updated weights for policy 0, policy_version 562175 (0.0008) [2023-12-26 19:27:37,506][105620] Updated weights for policy 1, policy_version 562989 (0.0010) [2023-12-26 19:27:37,567][105620] Updated weights for policy 1, policy_version 562999 (0.0010) [2023-12-26 19:27:37,630][105620] Updated weights for policy 1, policy_version 563009 (0.0010) [2023-12-26 19:27:38,067][105692] Updated weights for policy 0, policy_version 562185 (0.0011) [2023-12-26 19:27:38,119][105692] Updated weights for policy 0, policy_version 562195 (0.0010) [2023-12-26 19:27:38,189][105692] Updated weights for policy 0, policy_version 562205 (0.0011) [2023-12-26 19:27:38,263][105620] Updated weights for policy 1, policy_version 563019 (0.0007) [2023-12-26 19:27:38,325][105620] Updated weights for policy 1, policy_version 563029 (0.0009) [2023-12-26 19:27:38,388][105620] Updated weights for policy 1, policy_version 563039 (0.0008) [2023-12-26 19:27:38,954][105692] Updated weights for policy 0, policy_version 562215 (0.0011) [2023-12-26 19:27:39,017][105692] Updated weights for policy 0, policy_version 562225 (0.0011) [2023-12-26 19:27:39,021][105620] Updated weights for policy 1, policy_version 563049 (0.0008) [2023-12-26 19:27:39,076][105692] Updated weights for policy 0, policy_version 562235 (0.0010) [2023-12-26 19:27:39,082][105620] Updated weights for policy 1, policy_version 563059 (0.0007) [2023-12-26 19:27:39,147][105620] Updated weights for policy 1, policy_version 563069 (0.0008) [2023-12-26 19:27:39,205][105620] Updated weights for policy 1, policy_version 563079 (0.0007) [2023-12-26 19:27:39,846][105692] Updated weights for policy 0, policy_version 562245 (0.0009) [2023-12-26 19:27:39,889][105620] Updated weights for policy 1, policy_version 563089 (0.0010) [2023-12-26 19:27:39,908][105692] Updated weights for policy 0, policy_version 562255 (0.0009) [2023-12-26 19:27:39,958][105620] Updated weights for policy 1, policy_version 563099 (0.0011) [2023-12-26 19:27:39,978][105692] Updated weights for policy 0, policy_version 562265 (0.0009) [2023-12-26 19:27:40,018][105620] Updated weights for policy 1, policy_version 563109 (0.0011) [2023-12-26 19:27:40,688][105692] Updated weights for policy 0, policy_version 562275 (0.0007) [2023-12-26 19:27:40,722][105620] Updated weights for policy 1, policy_version 563119 (0.0011) [2023-12-26 19:27:40,746][105692] Updated weights for policy 0, policy_version 562285 (0.0006) [2023-12-26 19:27:40,783][105620] Updated weights for policy 1, policy_version 563129 (0.0011) [2023-12-26 19:27:40,807][105692] Updated weights for policy 0, policy_version 562295 (0.0009) [2023-12-26 19:27:40,838][105620] Updated weights for policy 1, policy_version 563139 (0.0010) [2023-12-26 19:27:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 288145408. Throughput: 0: 9655.9, 1: 10009.0. Samples: 288150240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:27:41,063][104569] Avg episode reward: [(0, '9353.390'), (1, '9078.338')] [2023-12-26 19:27:41,600][105692] Updated weights for policy 0, policy_version 562305 (0.0007) [2023-12-26 19:27:41,618][105620] Updated weights for policy 1, policy_version 563149 (0.0010) [2023-12-26 19:27:41,668][105692] Updated weights for policy 0, policy_version 562315 (0.0007) [2023-12-26 19:27:41,682][105620] Updated weights for policy 1, policy_version 563159 (0.0008) [2023-12-26 19:27:41,730][105692] Updated weights for policy 0, policy_version 562325 (0.0008) [2023-12-26 19:27:41,744][105620] Updated weights for policy 1, policy_version 563169 (0.0008) [2023-12-26 19:27:41,787][105692] Updated weights for policy 0, policy_version 562335 (0.0007) [2023-12-26 19:27:42,457][105620] Updated weights for policy 1, policy_version 563179 (0.0008) [2023-12-26 19:27:42,506][105620] Updated weights for policy 1, policy_version 563189 (0.0009) [2023-12-26 19:27:42,559][105692] Updated weights for policy 0, policy_version 562345 (0.0008) [2023-12-26 19:27:42,560][105620] Updated weights for policy 1, policy_version 563199 (0.0005) [2023-12-26 19:27:42,607][105692] Updated weights for policy 0, policy_version 562355 (0.0009) [2023-12-26 19:27:42,666][105692] Updated weights for policy 0, policy_version 562366 (0.0010) [2023-12-26 19:27:43,300][105620] Updated weights for policy 1, policy_version 563209 (0.0006) [2023-12-26 19:27:43,357][105620] Updated weights for policy 1, policy_version 563219 (0.0009) [2023-12-26 19:27:43,426][105620] Updated weights for policy 1, policy_version 563229 (0.0009) [2023-12-26 19:27:43,429][105692] Updated weights for policy 0, policy_version 562376 (0.0006) [2023-12-26 19:27:43,482][105620] Updated weights for policy 1, policy_version 563239 (0.0008) [2023-12-26 19:27:43,495][105692] Updated weights for policy 0, policy_version 562386 (0.0005) [2023-12-26 19:27:43,553][105692] Updated weights for policy 0, policy_version 562396 (0.0005) [2023-12-26 19:27:44,097][105692] Updated weights for policy 0, policy_version 562406 (0.0006) [2023-12-26 19:27:44,163][105692] Updated weights for policy 0, policy_version 562416 (0.0007) [2023-12-26 19:27:44,223][105692] Updated weights for policy 0, policy_version 562426 (0.0008) [2023-12-26 19:27:44,304][105620] Updated weights for policy 1, policy_version 563249 (0.0009) [2023-12-26 19:27:44,374][105620] Updated weights for policy 1, policy_version 563259 (0.0010) [2023-12-26 19:27:44,436][105620] Updated weights for policy 1, policy_version 563269 (0.0009) [2023-12-26 19:27:44,909][105692] Updated weights for policy 0, policy_version 562436 (0.0008) [2023-12-26 19:27:44,964][105692] Updated weights for policy 0, policy_version 562446 (0.0009) [2023-12-26 19:27:45,024][105692] Updated weights for policy 0, policy_version 562456 (0.0009) [2023-12-26 19:27:45,125][105620] Updated weights for policy 1, policy_version 563279 (0.0009) [2023-12-26 19:27:45,182][105620] Updated weights for policy 1, policy_version 563289 (0.0009) [2023-12-26 19:27:45,242][105620] Updated weights for policy 1, policy_version 563299 (0.0009) [2023-12-26 19:27:45,792][105692] Updated weights for policy 0, policy_version 562466 (0.0009) [2023-12-26 19:27:45,852][105692] Updated weights for policy 0, policy_version 562476 (0.0009) [2023-12-26 19:27:45,906][105692] Updated weights for policy 0, policy_version 562486 (0.0009) [2023-12-26 19:27:45,964][105692] Updated weights for policy 0, policy_version 562496 (0.0008) [2023-12-26 19:27:46,008][105620] Updated weights for policy 1, policy_version 563309 (0.0008) [2023-12-26 19:27:46,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 288235520. Throughput: 0: 9687.2, 1: 9885.8. Samples: 288206268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:27:46,063][105620] Updated weights for policy 1, policy_version 563319 (0.0009) [2023-12-26 19:27:46,063][104569] Avg episode reward: [(0, '9172.019'), (1, '9081.218')] [2023-12-26 19:27:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000562496_144015360.pth... [2023-12-26 19:27:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000561344_143720448.pth [2023-12-26 19:27:46,124][105620] Updated weights for policy 1, policy_version 563329 (0.0009) [2023-12-26 19:27:46,167][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000563336_144228352.pth... [2023-12-26 19:27:46,172][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000562152_143925248.pth [2023-12-26 19:27:46,663][105692] Updated weights for policy 0, policy_version 562506 (0.0009) [2023-12-26 19:27:46,726][105692] Updated weights for policy 0, policy_version 562516 (0.0009) [2023-12-26 19:27:46,785][105692] Updated weights for policy 0, policy_version 562526 (0.0009) [2023-12-26 19:27:46,910][105620] Updated weights for policy 1, policy_version 563339 (0.0008) [2023-12-26 19:27:46,976][105620] Updated weights for policy 1, policy_version 563349 (0.0007) [2023-12-26 19:27:47,045][105620] Updated weights for policy 1, policy_version 563359 (0.0006) [2023-12-26 19:27:47,535][105692] Updated weights for policy 0, policy_version 562536 (0.0007) [2023-12-26 19:27:47,602][105692] Updated weights for policy 0, policy_version 562546 (0.0008) [2023-12-26 19:27:47,663][105692] Updated weights for policy 0, policy_version 562556 (0.0008) [2023-12-26 19:27:47,709][105620] Updated weights for policy 1, policy_version 563369 (0.0007) [2023-12-26 19:27:47,764][105620] Updated weights for policy 1, policy_version 563379 (0.0008) [2023-12-26 19:27:47,826][105620] Updated weights for policy 1, policy_version 563389 (0.0009) [2023-12-26 19:27:47,888][105620] Updated weights for policy 1, policy_version 563399 (0.0009) [2023-12-26 19:27:48,328][105692] Updated weights for policy 0, policy_version 562566 (0.0008) [2023-12-26 19:27:48,390][105692] Updated weights for policy 0, policy_version 562576 (0.0009) [2023-12-26 19:27:48,449][105692] Updated weights for policy 0, policy_version 562586 (0.0009) [2023-12-26 19:27:48,657][105620] Updated weights for policy 1, policy_version 563409 (0.0009) [2023-12-26 19:27:48,720][105620] Updated weights for policy 1, policy_version 563419 (0.0009) [2023-12-26 19:27:48,781][105620] Updated weights for policy 1, policy_version 563429 (0.0010) [2023-12-26 19:27:49,098][105692] Updated weights for policy 0, policy_version 562596 (0.0009) [2023-12-26 19:27:49,156][105692] Updated weights for policy 0, policy_version 562606 (0.0006) [2023-12-26 19:27:49,222][105692] Updated weights for policy 0, policy_version 562616 (0.0006) [2023-12-26 19:27:49,671][105620] Updated weights for policy 1, policy_version 563439 (0.0009) [2023-12-26 19:27:49,723][105620] Updated weights for policy 1, policy_version 563449 (0.0009) [2023-12-26 19:27:49,782][105620] Updated weights for policy 1, policy_version 563459 (0.0009) [2023-12-26 19:27:49,889][105692] Updated weights for policy 0, policy_version 562626 (0.0009) [2023-12-26 19:27:49,951][105692] Updated weights for policy 0, policy_version 562636 (0.0008) [2023-12-26 19:27:50,017][105692] Updated weights for policy 0, policy_version 562646 (0.0006) [2023-12-26 19:27:50,064][105692] Updated weights for policy 0, policy_version 562656 (0.0008) [2023-12-26 19:27:50,541][105620] Updated weights for policy 1, policy_version 563469 (0.0008) [2023-12-26 19:27:50,612][105620] Updated weights for policy 1, policy_version 563479 (0.0009) [2023-12-26 19:27:50,674][105620] Updated weights for policy 1, policy_version 563489 (0.0009) [2023-12-26 19:27:50,836][105692] Updated weights for policy 0, policy_version 562666 (0.0008) [2023-12-26 19:27:50,894][105692] Updated weights for policy 0, policy_version 562676 (0.0009) [2023-12-26 19:27:50,945][105692] Updated weights for policy 0, policy_version 562686 (0.0009) [2023-12-26 19:27:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 288333824. Throughput: 0: 9734.4, 1: 9815.6. Samples: 288320892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:27:51,062][104569] Avg episode reward: [(0, '8989.907'), (1, '9262.772')] [2023-12-26 19:27:51,375][105620] Updated weights for policy 1, policy_version 563499 (0.0008) [2023-12-26 19:27:51,437][105620] Updated weights for policy 1, policy_version 563509 (0.0009) [2023-12-26 19:27:51,492][105620] Updated weights for policy 1, policy_version 563519 (0.0009) [2023-12-26 19:27:51,766][105692] Updated weights for policy 0, policy_version 562696 (0.0009) [2023-12-26 19:27:51,832][105692] Updated weights for policy 0, policy_version 562706 (0.0010) [2023-12-26 19:27:51,885][105692] Updated weights for policy 0, policy_version 562716 (0.0010) [2023-12-26 19:27:52,205][105620] Updated weights for policy 1, policy_version 563529 (0.0009) [2023-12-26 19:27:52,268][105620] Updated weights for policy 1, policy_version 563539 (0.0007) [2023-12-26 19:27:52,327][105620] Updated weights for policy 1, policy_version 563549 (0.0007) [2023-12-26 19:27:52,385][105620] Updated weights for policy 1, policy_version 563559 (0.0011) [2023-12-26 19:27:52,672][105692] Updated weights for policy 0, policy_version 562726 (0.0010) [2023-12-26 19:27:52,721][105692] Updated weights for policy 0, policy_version 562736 (0.0010) [2023-12-26 19:27:52,774][105692] Updated weights for policy 0, policy_version 562746 (0.0010) [2023-12-26 19:27:53,115][105620] Updated weights for policy 1, policy_version 563569 (0.0006) [2023-12-26 19:27:53,179][105620] Updated weights for policy 1, policy_version 563579 (0.0005) [2023-12-26 19:27:53,240][105620] Updated weights for policy 1, policy_version 563589 (0.0005) [2023-12-26 19:27:53,511][105692] Updated weights for policy 0, policy_version 562756 (0.0011) [2023-12-26 19:27:53,574][105692] Updated weights for policy 0, policy_version 562766 (0.0011) [2023-12-26 19:27:53,636][105692] Updated weights for policy 0, policy_version 562776 (0.0010) [2023-12-26 19:27:53,804][105620] Updated weights for policy 1, policy_version 563599 (0.0006) [2023-12-26 19:27:53,850][105620] Updated weights for policy 1, policy_version 563609 (0.0007) [2023-12-26 19:27:53,917][105620] Updated weights for policy 1, policy_version 563619 (0.0007) [2023-12-26 19:27:54,358][105692] Updated weights for policy 0, policy_version 562786 (0.0010) [2023-12-26 19:27:54,405][105692] Updated weights for policy 0, policy_version 562796 (0.0010) [2023-12-26 19:27:54,460][105692] Updated weights for policy 0, policy_version 562806 (0.0010) [2023-12-26 19:27:54,508][105692] Updated weights for policy 0, policy_version 562816 (0.0010) [2023-12-26 19:27:54,534][105620] Updated weights for policy 1, policy_version 563629 (0.0007) [2023-12-26 19:27:54,582][105620] Updated weights for policy 1, policy_version 563639 (0.0008) [2023-12-26 19:27:54,630][105620] Updated weights for policy 1, policy_version 563649 (0.0008) [2023-12-26 19:27:55,242][105692] Updated weights for policy 0, policy_version 562826 (0.0010) [2023-12-26 19:27:55,293][105692] Updated weights for policy 0, policy_version 562836 (0.0010) [2023-12-26 19:27:55,340][105692] Updated weights for policy 0, policy_version 562846 (0.0009) [2023-12-26 19:27:55,429][105620] Updated weights for policy 1, policy_version 563659 (0.0007) [2023-12-26 19:27:55,478][105620] Updated weights for policy 1, policy_version 563669 (0.0005) [2023-12-26 19:27:55,545][105620] Updated weights for policy 1, policy_version 563679 (0.0005) [2023-12-26 19:27:56,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 288423936. Throughput: 0: 9630.2, 1: 9862.6. Samples: 288436432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:27:56,063][104569] Avg episode reward: [(0, '9172.248'), (1, '9262.839')] [2023-12-26 19:27:56,071][105692] Updated weights for policy 0, policy_version 562856 (0.0006) [2023-12-26 19:27:56,127][105692] Updated weights for policy 0, policy_version 562866 (0.0005) [2023-12-26 19:27:56,182][105692] Updated weights for policy 0, policy_version 562876 (0.0010) [2023-12-26 19:27:56,196][105620] Updated weights for policy 1, policy_version 563689 (0.0005) [2023-12-26 19:27:56,252][105620] Updated weights for policy 1, policy_version 563699 (0.0008) [2023-12-26 19:27:56,312][105620] Updated weights for policy 1, policy_version 563709 (0.0008) [2023-12-26 19:27:56,362][105620] Updated weights for policy 1, policy_version 563719 (0.0007) [2023-12-26 19:27:56,364][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000004 [2023-12-26 19:27:56,793][105692] Updated weights for policy 0, policy_version 562886 (0.0007) [2023-12-26 19:27:56,843][105692] Updated weights for policy 0, policy_version 562896 (0.0005) [2023-12-26 19:27:56,888][105692] Updated weights for policy 0, policy_version 562906 (0.0005) [2023-12-26 19:27:57,208][105620] Updated weights for policy 1, policy_version 563729 (0.0008) [2023-12-26 19:27:57,263][105620] Updated weights for policy 1, policy_version 563739 (0.0008) [2023-12-26 19:27:57,316][105620] Updated weights for policy 1, policy_version 563749 (0.0008) [2023-12-26 19:27:57,495][105692] Updated weights for policy 0, policy_version 562916 (0.0005) [2023-12-26 19:27:57,552][105692] Updated weights for policy 0, policy_version 562926 (0.0008) [2023-12-26 19:27:57,602][105692] Updated weights for policy 0, policy_version 562936 (0.0009) [2023-12-26 19:27:58,014][105620] Updated weights for policy 1, policy_version 563759 (0.0009) [2023-12-26 19:27:58,068][105620] Updated weights for policy 1, policy_version 563769 (0.0009) [2023-12-26 19:27:58,118][105620] Updated weights for policy 1, policy_version 563779 (0.0009) [2023-12-26 19:27:58,336][105692] Updated weights for policy 0, policy_version 562946 (0.0009) [2023-12-26 19:27:58,400][105692] Updated weights for policy 0, policy_version 562956 (0.0007) [2023-12-26 19:27:58,462][105692] Updated weights for policy 0, policy_version 562966 (0.0009) [2023-12-26 19:27:58,524][105692] Updated weights for policy 0, policy_version 562976 (0.0009) [2023-12-26 19:27:58,928][105620] Updated weights for policy 1, policy_version 563789 (0.0009) [2023-12-26 19:27:58,980][105620] Updated weights for policy 1, policy_version 563799 (0.0008) [2023-12-26 19:27:59,035][105620] Updated weights for policy 1, policy_version 563809 (0.0008) [2023-12-26 19:27:59,273][105692] Updated weights for policy 0, policy_version 562986 (0.0009) [2023-12-26 19:27:59,334][105692] Updated weights for policy 0, policy_version 562996 (0.0011) [2023-12-26 19:27:59,394][105692] Updated weights for policy 0, policy_version 563006 (0.0010) [2023-12-26 19:27:59,812][105620] Updated weights for policy 1, policy_version 563819 (0.0008) [2023-12-26 19:27:59,877][105620] Updated weights for policy 1, policy_version 563829 (0.0007) [2023-12-26 19:27:59,940][105620] Updated weights for policy 1, policy_version 563839 (0.0009) [2023-12-26 19:28:00,125][105692] Updated weights for policy 0, policy_version 563016 (0.0008) [2023-12-26 19:28:00,185][105692] Updated weights for policy 0, policy_version 563027 (0.0009) [2023-12-26 19:28:00,237][105692] Updated weights for policy 0, policy_version 563037 (0.0009) [2023-12-26 19:28:00,625][105620] Updated weights for policy 1, policy_version 563849 (0.0010) [2023-12-26 19:28:00,672][105620] Updated weights for policy 1, policy_version 563859 (0.0010) [2023-12-26 19:28:00,716][105620] Updated weights for policy 1, policy_version 563869 (0.0010) [2023-12-26 19:28:00,763][105620] Updated weights for policy 1, policy_version 563879 (0.0010) [2023-12-26 19:28:01,022][105692] Updated weights for policy 0, policy_version 563047 (0.0008) [2023-12-26 19:28:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 288522240. Throughput: 0: 9703.2, 1: 9823.4. Samples: 288495520. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 19:28:01,063][104569] Avg episode reward: [(0, '9354.629'), (1, '9353.209')] [2023-12-26 19:28:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000563880_144367616.pth... [2023-12-26 19:28:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000562728_144072704.pth [2023-12-26 19:28:01,087][105692] Updated weights for policy 0, policy_version 563057 (0.0011) [2023-12-26 19:28:01,153][105692] Updated weights for policy 0, policy_version 563067 (0.0010) [2023-12-26 19:28:01,184][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000563072_144162816.pth... [2023-12-26 19:28:01,188][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000561920_143867904.pth [2023-12-26 19:28:01,537][105620] Updated weights for policy 1, policy_version 563889 (0.0010) [2023-12-26 19:28:01,607][105620] Updated weights for policy 1, policy_version 563899 (0.0008) [2023-12-26 19:28:01,681][105620] Updated weights for policy 1, policy_version 563909 (0.0008) [2023-12-26 19:28:01,858][105692] Updated weights for policy 0, policy_version 563077 (0.0011) [2023-12-26 19:28:01,915][105692] Updated weights for policy 0, policy_version 563087 (0.0010) [2023-12-26 19:28:01,977][105692] Updated weights for policy 0, policy_version 563097 (0.0010) [2023-12-26 19:28:02,414][105620] Updated weights for policy 1, policy_version 563919 (0.0010) [2023-12-26 19:28:02,463][105620] Updated weights for policy 1, policy_version 563929 (0.0008) [2023-12-26 19:28:02,520][105620] Updated weights for policy 1, policy_version 563939 (0.0006) [2023-12-26 19:28:02,697][105692] Updated weights for policy 0, policy_version 563107 (0.0010) [2023-12-26 19:28:02,766][105692] Updated weights for policy 0, policy_version 563117 (0.0011) [2023-12-26 19:28:02,828][105692] Updated weights for policy 0, policy_version 563127 (0.0010) [2023-12-26 19:28:03,183][105620] Updated weights for policy 1, policy_version 563949 (0.0007) [2023-12-26 19:28:03,243][105620] Updated weights for policy 1, policy_version 563959 (0.0006) [2023-12-26 19:28:03,299][105620] Updated weights for policy 1, policy_version 563969 (0.0007) [2023-12-26 19:28:03,505][105692] Updated weights for policy 0, policy_version 563137 (0.0010) [2023-12-26 19:28:03,563][105692] Updated weights for policy 0, policy_version 563147 (0.0005) [2023-12-26 19:28:03,621][105692] Updated weights for policy 0, policy_version 563157 (0.0005) [2023-12-26 19:28:03,666][105692] Updated weights for policy 0, policy_version 563167 (0.0005) [2023-12-26 19:28:03,932][105620] Updated weights for policy 1, policy_version 563979 (0.0009) [2023-12-26 19:28:03,993][105620] Updated weights for policy 1, policy_version 563989 (0.0009) [2023-12-26 19:28:04,046][105620] Updated weights for policy 1, policy_version 563999 (0.0010) [2023-12-26 19:28:04,331][105692] Updated weights for policy 0, policy_version 563177 (0.0010) [2023-12-26 19:28:04,388][105692] Updated weights for policy 0, policy_version 563187 (0.0009) [2023-12-26 19:28:04,440][105692] Updated weights for policy 0, policy_version 563197 (0.0010) [2023-12-26 19:28:04,791][105620] Updated weights for policy 1, policy_version 564009 (0.0013) [2023-12-26 19:28:04,844][105620] Updated weights for policy 1, policy_version 564019 (0.0009) [2023-12-26 19:28:04,899][105620] Updated weights for policy 1, policy_version 564029 (0.0010) [2023-12-26 19:28:04,954][105620] Updated weights for policy 1, policy_version 564039 (0.0006) [2023-12-26 19:28:05,079][105692] Updated weights for policy 0, policy_version 563207 (0.0007) [2023-12-26 19:28:05,133][105692] Updated weights for policy 0, policy_version 563217 (0.0005) [2023-12-26 19:28:05,182][105692] Updated weights for policy 0, policy_version 563227 (0.0005) [2023-12-26 19:28:05,619][105620] Updated weights for policy 1, policy_version 564049 (0.0010) [2023-12-26 19:28:05,667][105620] Updated weights for policy 1, policy_version 564059 (0.0010) [2023-12-26 19:28:05,695][105692] Updated weights for policy 0, policy_version 563237 (0.0005) [2023-12-26 19:28:05,719][105620] Updated weights for policy 1, policy_version 564069 (0.0010) [2023-12-26 19:28:05,759][105692] Updated weights for policy 0, policy_version 563247 (0.0005) [2023-12-26 19:28:05,823][105692] Updated weights for policy 0, policy_version 563257 (0.0006) [2023-12-26 19:28:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 288628736. Throughput: 0: 9718.3, 1: 9856.1. Samples: 288611384. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 19:28:06,062][104569] Avg episode reward: [(0, '9353.938'), (1, '9354.043')] [2023-12-26 19:28:06,358][105620] Updated weights for policy 1, policy_version 564079 (0.0007) [2023-12-26 19:28:06,415][105620] Updated weights for policy 1, policy_version 564089 (0.0006) [2023-12-26 19:28:06,482][105620] Updated weights for policy 1, policy_version 564099 (0.0006) [2023-12-26 19:28:06,501][105692] Updated weights for policy 0, policy_version 563267 (0.0008) [2023-12-26 19:28:06,561][105692] Updated weights for policy 0, policy_version 563277 (0.0011) [2023-12-26 19:28:06,625][105692] Updated weights for policy 0, policy_version 563287 (0.0011) [2023-12-26 19:28:07,065][105620] Updated weights for policy 1, policy_version 564109 (0.0006) [2023-12-26 19:28:07,118][105620] Updated weights for policy 1, policy_version 564119 (0.0008) [2023-12-26 19:28:07,174][105620] Updated weights for policy 1, policy_version 564129 (0.0007) [2023-12-26 19:28:07,387][105692] Updated weights for policy 0, policy_version 563297 (0.0011) [2023-12-26 19:28:07,445][105692] Updated weights for policy 0, policy_version 563307 (0.0011) [2023-12-26 19:28:07,497][105692] Updated weights for policy 0, policy_version 563317 (0.0010) [2023-12-26 19:28:07,545][105692] Updated weights for policy 0, policy_version 563327 (0.0010) [2023-12-26 19:28:07,764][105620] Updated weights for policy 1, policy_version 564139 (0.0008) [2023-12-26 19:28:07,817][105620] Updated weights for policy 1, policy_version 564149 (0.0005) [2023-12-26 19:28:07,880][105620] Updated weights for policy 1, policy_version 564159 (0.0005) [2023-12-26 19:28:08,274][105692] Updated weights for policy 0, policy_version 563337 (0.0006) [2023-12-26 19:28:08,336][105692] Updated weights for policy 0, policy_version 563347 (0.0011) [2023-12-26 19:28:08,399][105692] Updated weights for policy 0, policy_version 563357 (0.0011) [2023-12-26 19:28:08,497][105620] Updated weights for policy 1, policy_version 564169 (0.0005) [2023-12-26 19:28:08,563][105620] Updated weights for policy 1, policy_version 564179 (0.0006) [2023-12-26 19:28:08,613][105620] Updated weights for policy 1, policy_version 564189 (0.0007) [2023-12-26 19:28:08,673][105620] Updated weights for policy 1, policy_version 564199 (0.0011) [2023-12-26 19:28:09,027][105692] Updated weights for policy 0, policy_version 563367 (0.0007) [2023-12-26 19:28:09,087][105692] Updated weights for policy 0, policy_version 563377 (0.0008) [2023-12-26 19:28:09,148][105692] Updated weights for policy 0, policy_version 563387 (0.0008) [2023-12-26 19:28:09,436][105620] Updated weights for policy 1, policy_version 564209 (0.0008) [2023-12-26 19:28:09,501][105620] Updated weights for policy 1, policy_version 564219 (0.0009) [2023-12-26 19:28:09,564][105620] Updated weights for policy 1, policy_version 564229 (0.0007) [2023-12-26 19:28:09,992][105692] Updated weights for policy 0, policy_version 563397 (0.0008) [2023-12-26 19:28:10,048][105692] Updated weights for policy 0, policy_version 563407 (0.0009) [2023-12-26 19:28:10,103][105692] Updated weights for policy 0, policy_version 563417 (0.0009) [2023-12-26 19:28:10,238][105620] Updated weights for policy 1, policy_version 564239 (0.0009) [2023-12-26 19:28:10,300][105620] Updated weights for policy 1, policy_version 564249 (0.0007) [2023-12-26 19:28:10,360][105620] Updated weights for policy 1, policy_version 564259 (0.0009) [2023-12-26 19:28:10,895][105692] Updated weights for policy 0, policy_version 563427 (0.0009) [2023-12-26 19:28:10,952][105692] Updated weights for policy 0, policy_version 563437 (0.0009) [2023-12-26 19:28:11,009][105692] Updated weights for policy 0, policy_version 563447 (0.0009) [2023-12-26 19:28:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 288718848. Throughput: 0: 9782.8, 1: 9894.3. Samples: 288734840. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 19:28:11,062][104569] Avg episode reward: [(0, '7826.226'), (1, '9170.688')] [2023-12-26 19:28:11,094][105620] Updated weights for policy 1, policy_version 564269 (0.0008) [2023-12-26 19:28:11,160][105620] Updated weights for policy 1, policy_version 564279 (0.0008) [2023-12-26 19:28:11,221][105620] Updated weights for policy 1, policy_version 564289 (0.0010) [2023-12-26 19:28:11,776][105692] Updated weights for policy 0, policy_version 563457 (0.0009) [2023-12-26 19:28:11,834][105692] Updated weights for policy 0, policy_version 563467 (0.0010) [2023-12-26 19:28:11,888][105692] Updated weights for policy 0, policy_version 563477 (0.0010) [2023-12-26 19:28:11,944][105692] Updated weights for policy 0, policy_version 563487 (0.0009) [2023-12-26 19:28:11,964][105620] Updated weights for policy 1, policy_version 564299 (0.0009) [2023-12-26 19:28:12,026][105620] Updated weights for policy 1, policy_version 564309 (0.0009) [2023-12-26 19:28:12,090][105620] Updated weights for policy 1, policy_version 564319 (0.0009) [2023-12-26 19:28:12,728][105692] Updated weights for policy 0, policy_version 563497 (0.0009) [2023-12-26 19:28:12,792][105692] Updated weights for policy 0, policy_version 563507 (0.0006) [2023-12-26 19:28:12,852][105692] Updated weights for policy 0, policy_version 563517 (0.0006) [2023-12-26 19:28:12,879][105620] Updated weights for policy 1, policy_version 564329 (0.0009) [2023-12-26 19:28:12,933][105620] Updated weights for policy 1, policy_version 564339 (0.0010) [2023-12-26 19:28:12,994][105620] Updated weights for policy 1, policy_version 564349 (0.0009) [2023-12-26 19:28:13,062][105620] Updated weights for policy 1, policy_version 564359 (0.0010) [2023-12-26 19:28:13,511][105692] Updated weights for policy 0, policy_version 563527 (0.0006) [2023-12-26 19:28:13,561][105692] Updated weights for policy 0, policy_version 563537 (0.0005) [2023-12-26 19:28:13,615][105692] Updated weights for policy 0, policy_version 563547 (0.0006) [2023-12-26 19:28:13,830][105620] Updated weights for policy 1, policy_version 564369 (0.0010) [2023-12-26 19:28:13,882][105620] Updated weights for policy 1, policy_version 564379 (0.0009) [2023-12-26 19:28:13,935][105620] Updated weights for policy 1, policy_version 564389 (0.0006) [2023-12-26 19:28:14,204][105692] Updated weights for policy 0, policy_version 563557 (0.0007) [2023-12-26 19:28:14,273][105692] Updated weights for policy 0, policy_version 563567 (0.0006) [2023-12-26 19:28:14,326][105692] Updated weights for policy 0, policy_version 563577 (0.0007) [2023-12-26 19:28:14,671][105620] Updated weights for policy 1, policy_version 564399 (0.0010) [2023-12-26 19:28:14,730][105620] Updated weights for policy 1, policy_version 564409 (0.0010) [2023-12-26 19:28:14,794][105620] Updated weights for policy 1, policy_version 564419 (0.0010) [2023-12-26 19:28:15,068][105692] Updated weights for policy 0, policy_version 563587 (0.0009) [2023-12-26 19:28:15,124][105692] Updated weights for policy 0, policy_version 563597 (0.0008) [2023-12-26 19:28:15,190][105692] Updated weights for policy 0, policy_version 563607 (0.0006) [2023-12-26 19:28:15,502][105620] Updated weights for policy 1, policy_version 564429 (0.0008) [2023-12-26 19:28:15,567][105620] Updated weights for policy 1, policy_version 564439 (0.0005) [2023-12-26 19:28:15,618][105620] Updated weights for policy 1, policy_version 564449 (0.0005) [2023-12-26 19:28:15,816][105692] Updated weights for policy 0, policy_version 563617 (0.0010) [2023-12-26 19:28:15,867][105692] Updated weights for policy 0, policy_version 563627 (0.0005) [2023-12-26 19:28:15,915][105692] Updated weights for policy 0, policy_version 563637 (0.0005) [2023-12-26 19:28:15,969][105692] Updated weights for policy 0, policy_version 563647 (0.0005) [2023-12-26 19:28:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 288825344. Throughput: 0: 9743.5, 1: 9817.7. Samples: 288790040. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 19:28:16,062][104569] Avg episode reward: [(0, '658.955'), (1, '9170.508')] [2023-12-26 19:28:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000563648_144310272.pth... [2023-12-26 19:28:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000564456_144515072.pth... [2023-12-26 19:28:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000562496_144015360.pth [2023-12-26 19:28:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000563336_144228352.pth [2023-12-26 19:28:16,162][105620] Updated weights for policy 1, policy_version 564459 (0.0007) [2023-12-26 19:28:16,219][105620] Updated weights for policy 1, policy_version 564469 (0.0010) [2023-12-26 19:28:16,280][105620] Updated weights for policy 1, policy_version 564479 (0.0010) [2023-12-26 19:28:16,511][105692] Updated weights for policy 0, policy_version 563657 (0.0006) [2023-12-26 19:28:16,561][105692] Updated weights for policy 0, policy_version 563667 (0.0007) [2023-12-26 19:28:16,606][105692] Updated weights for policy 0, policy_version 563677 (0.0008) [2023-12-26 19:28:17,052][105620] Updated weights for policy 1, policy_version 564489 (0.0010) [2023-12-26 19:28:17,107][105620] Updated weights for policy 1, policy_version 564499 (0.0009) [2023-12-26 19:28:17,165][105620] Updated weights for policy 1, policy_version 564509 (0.0012) [2023-12-26 19:28:17,212][105620] Updated weights for policy 1, policy_version 564519 (0.0009) [2023-12-26 19:28:17,266][105692] Updated weights for policy 0, policy_version 563687 (0.0009) [2023-12-26 19:28:17,317][105692] Updated weights for policy 0, policy_version 563697 (0.0009) [2023-12-26 19:28:17,364][105692] Updated weights for policy 0, policy_version 563707 (0.0009) [2023-12-26 19:28:17,967][105620] Updated weights for policy 1, policy_version 564529 (0.0010) [2023-12-26 19:28:18,020][105620] Updated weights for policy 1, policy_version 564539 (0.0011) [2023-12-26 19:28:18,023][105692] Updated weights for policy 0, policy_version 563717 (0.0007) [2023-12-26 19:28:18,068][105620] Updated weights for policy 1, policy_version 564549 (0.0009) [2023-12-26 19:28:18,080][105692] Updated weights for policy 0, policy_version 563727 (0.0006) [2023-12-26 19:28:18,142][105692] Updated weights for policy 0, policy_version 563737 (0.0008) [2023-12-26 19:28:18,819][105620] Updated weights for policy 1, policy_version 564559 (0.0010) [2023-12-26 19:28:18,822][105692] Updated weights for policy 0, policy_version 563747 (0.0006) [2023-12-26 19:28:18,872][105620] Updated weights for policy 1, policy_version 564569 (0.0010) [2023-12-26 19:28:18,882][105692] Updated weights for policy 0, policy_version 563757 (0.0006) [2023-12-26 19:28:18,923][105620] Updated weights for policy 1, policy_version 564579 (0.0010) [2023-12-26 19:28:18,942][105692] Updated weights for policy 0, policy_version 563767 (0.0006) [2023-12-26 19:28:19,612][105620] Updated weights for policy 1, policy_version 564589 (0.0011) [2023-12-26 19:28:19,677][105620] Updated weights for policy 1, policy_version 564599 (0.0008) [2023-12-26 19:28:19,743][105620] Updated weights for policy 1, policy_version 564609 (0.0008) [2023-12-26 19:28:19,745][105692] Updated weights for policy 0, policy_version 563777 (0.0008) [2023-12-26 19:28:19,805][105692] Updated weights for policy 0, policy_version 563787 (0.0008) [2023-12-26 19:28:19,871][105692] Updated weights for policy 0, policy_version 563797 (0.0009) [2023-12-26 19:28:19,935][105692] Updated weights for policy 0, policy_version 563807 (0.0009) [2023-12-26 19:28:20,390][105620] Updated weights for policy 1, policy_version 564619 (0.0009) [2023-12-26 19:28:20,446][105620] Updated weights for policy 1, policy_version 564629 (0.0008) [2023-12-26 19:28:20,507][105620] Updated weights for policy 1, policy_version 564639 (0.0006) [2023-12-26 19:28:20,607][105692] Updated weights for policy 0, policy_version 563817 (0.0010) [2023-12-26 19:28:20,664][105692] Updated weights for policy 0, policy_version 563827 (0.0008) [2023-12-26 19:28:20,723][105692] Updated weights for policy 0, policy_version 563837 (0.0008) [2023-12-26 19:28:21,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 288923648. Throughput: 0: 9837.5, 1: 9766.5. Samples: 288912084. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 19:28:21,063][104569] Avg episode reward: [(0, '633.160'), (1, '9079.003')] [2023-12-26 19:28:21,196][105620] Updated weights for policy 1, policy_version 564649 (0.0008) [2023-12-26 19:28:21,265][105620] Updated weights for policy 1, policy_version 564659 (0.0007) [2023-12-26 19:28:21,320][105620] Updated weights for policy 1, policy_version 564669 (0.0007) [2023-12-26 19:28:21,392][105620] Updated weights for policy 1, policy_version 564679 (0.0007) [2023-12-26 19:28:21,475][105692] Updated weights for policy 0, policy_version 563847 (0.0007) [2023-12-26 19:28:21,536][105692] Updated weights for policy 0, policy_version 563857 (0.0008) [2023-12-26 19:28:21,593][105692] Updated weights for policy 0, policy_version 563867 (0.0011) [2023-12-26 19:28:22,144][105620] Updated weights for policy 1, policy_version 564689 (0.0009) [2023-12-26 19:28:22,205][105620] Updated weights for policy 1, policy_version 564699 (0.0008) [2023-12-26 19:28:22,219][105692] Updated weights for policy 0, policy_version 563877 (0.0008) [2023-12-26 19:28:22,266][105620] Updated weights for policy 1, policy_version 564709 (0.0008) [2023-12-26 19:28:22,280][105692] Updated weights for policy 0, policy_version 563887 (0.0007) [2023-12-26 19:28:22,345][105692] Updated weights for policy 0, policy_version 563897 (0.0007) [2023-12-26 19:28:22,948][105620] Updated weights for policy 1, policy_version 564719 (0.0007) [2023-12-26 19:28:23,004][105620] Updated weights for policy 1, policy_version 564729 (0.0008) [2023-12-26 19:28:23,057][105620] Updated weights for policy 1, policy_version 564739 (0.0009) [2023-12-26 19:28:23,085][105692] Updated weights for policy 0, policy_version 563907 (0.0011) [2023-12-26 19:28:23,134][105692] Updated weights for policy 0, policy_version 563917 (0.0010) [2023-12-26 19:28:23,182][105692] Updated weights for policy 0, policy_version 563927 (0.0011) [2023-12-26 19:28:23,816][105620] Updated weights for policy 1, policy_version 564749 (0.0008) [2023-12-26 19:28:23,839][105692] Updated weights for policy 0, policy_version 563937 (0.0010) [2023-12-26 19:28:23,878][105620] Updated weights for policy 1, policy_version 564759 (0.0009) [2023-12-26 19:28:23,888][105692] Updated weights for policy 0, policy_version 563947 (0.0006) [2023-12-26 19:28:23,932][105620] Updated weights for policy 1, policy_version 564769 (0.0008) [2023-12-26 19:28:23,938][105692] Updated weights for policy 0, policy_version 563957 (0.0007) [2023-12-26 19:28:23,989][105692] Updated weights for policy 0, policy_version 563967 (0.0006) [2023-12-26 19:28:24,688][105620] Updated weights for policy 1, policy_version 564779 (0.0008) [2023-12-26 19:28:24,694][105692] Updated weights for policy 0, policy_version 563977 (0.0009) [2023-12-26 19:28:24,743][105620] Updated weights for policy 1, policy_version 564789 (0.0007) [2023-12-26 19:28:24,751][105692] Updated weights for policy 0, policy_version 563987 (0.0008) [2023-12-26 19:28:24,794][105620] Updated weights for policy 1, policy_version 564799 (0.0005) [2023-12-26 19:28:24,817][105692] Updated weights for policy 0, policy_version 563997 (0.0009) [2023-12-26 19:28:25,514][105620] Updated weights for policy 1, policy_version 564809 (0.0006) [2023-12-26 19:28:25,565][105692] Updated weights for policy 0, policy_version 564007 (0.0007) [2023-12-26 19:28:25,577][105620] Updated weights for policy 1, policy_version 564819 (0.0010) [2023-12-26 19:28:25,626][105692] Updated weights for policy 0, policy_version 564017 (0.0007) [2023-12-26 19:28:25,642][105620] Updated weights for policy 1, policy_version 564829 (0.0010) [2023-12-26 19:28:25,680][105692] Updated weights for policy 0, policy_version 564027 (0.0006) [2023-12-26 19:28:25,694][105620] Updated weights for policy 1, policy_version 564839 (0.0007) [2023-12-26 19:28:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 289021952. Throughput: 0: 9856.6, 1: 9694.8. Samples: 289030048. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 19:28:26,062][104569] Avg episode reward: [(0, '687.149'), (1, '9262.824')] [2023-12-26 19:28:26,327][105620] Updated weights for policy 1, policy_version 564849 (0.0010) [2023-12-26 19:28:26,386][105620] Updated weights for policy 1, policy_version 564859 (0.0010) [2023-12-26 19:28:26,433][105620] Updated weights for policy 1, policy_version 564869 (0.0010) [2023-12-26 19:28:26,491][105692] Updated weights for policy 0, policy_version 564037 (0.0006) [2023-12-26 19:28:26,546][105692] Updated weights for policy 0, policy_version 564047 (0.0005) [2023-12-26 19:28:26,596][105692] Updated weights for policy 0, policy_version 564057 (0.0005) [2023-12-26 19:28:27,166][105692] Updated weights for policy 0, policy_version 564067 (0.0006) [2023-12-26 19:28:27,175][105620] Updated weights for policy 1, policy_version 564879 (0.0010) [2023-12-26 19:28:27,229][105692] Updated weights for policy 0, policy_version 564077 (0.0006) [2023-12-26 19:28:27,233][105620] Updated weights for policy 1, policy_version 564889 (0.0010) [2023-12-26 19:28:27,284][105620] Updated weights for policy 1, policy_version 564899 (0.0010) [2023-12-26 19:28:27,288][105692] Updated weights for policy 0, policy_version 564087 (0.0007) [2023-12-26 19:28:27,852][105692] Updated weights for policy 0, policy_version 564097 (0.0008) [2023-12-26 19:28:27,901][105692] Updated weights for policy 0, policy_version 564107 (0.0005) [2023-12-26 19:28:27,945][105692] Updated weights for policy 0, policy_version 564117 (0.0005) [2023-12-26 19:28:27,990][105692] Updated weights for policy 0, policy_version 564127 (0.0005) [2023-12-26 19:28:28,026][105620] Updated weights for policy 1, policy_version 564909 (0.0010) [2023-12-26 19:28:28,045][105586] KL-divergence is very high: 158.7893 [2023-12-26 19:28:28,051][105586] KL-divergence is very high: 206.7987 [2023-12-26 19:28:28,062][105586] KL-divergence is very high: 257.9486 [2023-12-26 19:28:28,067][105586] KL-divergence is very high: 148.0507 [2023-12-26 19:28:28,078][105586] KL-divergence is very high: 260.8376 [2023-12-26 19:28:28,078][105620] Updated weights for policy 1, policy_version 564919 (0.0011) [2023-12-26 19:28:28,088][105586] KL-divergence is very high: 237.7541 [2023-12-26 19:28:28,094][105586] KL-divergence is very high: 124.2826 [2023-12-26 19:28:28,104][105586] KL-divergence is very high: 153.5836 [2023-12-26 19:28:28,127][105620] Updated weights for policy 1, policy_version 564929 (0.0010) [2023-12-26 19:28:28,574][105692] Updated weights for policy 0, policy_version 564137 (0.0008) [2023-12-26 19:28:28,634][105692] Updated weights for policy 0, policy_version 564147 (0.0006) [2023-12-26 19:28:28,696][105692] Updated weights for policy 0, policy_version 564157 (0.0007) [2023-12-26 19:28:28,863][105620] Updated weights for policy 1, policy_version 564939 (0.0010) [2023-12-26 19:28:28,917][105586] KL-divergence is very high: 121.7388 [2023-12-26 19:28:28,923][105620] Updated weights for policy 1, policy_version 564949 (0.0010) [2023-12-26 19:28:28,941][105586] KL-divergence is very high: 139.0557 [2023-12-26 19:28:28,949][105586] KL-divergence is very high: 158.1126 [2023-12-26 19:28:28,961][105586] KL-divergence is very high: 118.8581 [2023-12-26 19:28:28,967][105586] KL-divergence is very high: 185.9399 [2023-12-26 19:28:28,986][105620] Updated weights for policy 1, policy_version 564959 (0.0010) [2023-12-26 19:28:28,992][105586] KL-divergence is very high: 103.7911 [2023-12-26 19:28:28,998][105586] KL-divergence is very high: 101.8389 [2023-12-26 19:28:29,433][105692] Updated weights for policy 0, policy_version 564167 (0.0008) [2023-12-26 19:28:29,488][105692] Updated weights for policy 0, policy_version 564177 (0.0008) [2023-12-26 19:28:29,544][105692] Updated weights for policy 0, policy_version 564187 (0.0009) [2023-12-26 19:28:29,722][105620] Updated weights for policy 1, policy_version 564969 (0.0010) [2023-12-26 19:28:29,784][105620] Updated weights for policy 1, policy_version 564979 (0.0010) [2023-12-26 19:28:29,843][105620] Updated weights for policy 1, policy_version 564989 (0.0011) [2023-12-26 19:28:29,905][105620] Updated weights for policy 1, policy_version 564999 (0.0011) [2023-12-26 19:28:30,243][105692] Updated weights for policy 0, policy_version 564197 (0.0008) [2023-12-26 19:28:30,295][105692] Updated weights for policy 0, policy_version 564207 (0.0008) [2023-12-26 19:28:30,343][105692] Updated weights for policy 0, policy_version 564217 (0.0008) [2023-12-26 19:28:30,657][105620] Updated weights for policy 1, policy_version 565009 (0.0011) [2023-12-26 19:28:30,709][105620] Updated weights for policy 1, policy_version 565019 (0.0010) [2023-12-26 19:28:30,766][105620] Updated weights for policy 1, policy_version 565029 (0.0010) [2023-12-26 19:28:31,033][105692] Updated weights for policy 0, policy_version 564227 (0.0008) [2023-12-26 19:28:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 289120256. Throughput: 0: 9961.0, 1: 9721.1. Samples: 289091960. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 19:28:31,062][104569] Avg episode reward: [(0, '1193.541'), (1, '2351.624')] [2023-12-26 19:28:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000565032_144662528.pth... [2023-12-26 19:28:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000563880_144367616.pth [2023-12-26 19:28:31,098][105692] Updated weights for policy 0, policy_version 564237 (0.0007) [2023-12-26 19:28:31,163][105692] Updated weights for policy 0, policy_version 564247 (0.0008) [2023-12-26 19:28:31,212][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000564256_144465920.pth... [2023-12-26 19:28:31,215][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000563072_144162816.pth [2023-12-26 19:28:31,568][105620] Updated weights for policy 1, policy_version 565039 (0.0009) [2023-12-26 19:28:31,627][105586] KL-divergence is very high: 111.4494 [2023-12-26 19:28:31,634][105620] Updated weights for policy 1, policy_version 565049 (0.0010) [2023-12-26 19:28:31,686][105620] Updated weights for policy 1, policy_version 565059 (0.0011) [2023-12-26 19:28:31,913][105692] Updated weights for policy 0, policy_version 564257 (0.0007) [2023-12-26 19:28:31,965][105692] Updated weights for policy 0, policy_version 564267 (0.0008) [2023-12-26 19:28:32,021][105692] Updated weights for policy 0, policy_version 564277 (0.0008) [2023-12-26 19:28:32,069][105692] Updated weights for policy 0, policy_version 564287 (0.0007) [2023-12-26 19:28:32,373][105620] Updated weights for policy 1, policy_version 565069 (0.0009) [2023-12-26 19:28:32,387][105586] KL-divergence is very high: 120.6735 [2023-12-26 19:28:32,412][105586] KL-divergence is very high: 107.1003 [2023-12-26 19:28:32,425][105620] Updated weights for policy 1, policy_version 565079 (0.0010) [2023-12-26 19:28:32,480][105620] Updated weights for policy 1, policy_version 565089 (0.0010) [2023-12-26 19:28:32,923][105692] Updated weights for policy 0, policy_version 564298 (0.0010) [2023-12-26 19:28:32,970][105692] Updated weights for policy 0, policy_version 564308 (0.0008) [2023-12-26 19:28:33,037][105692] Updated weights for policy 0, policy_version 564318 (0.0009) [2023-12-26 19:28:33,152][105620] Updated weights for policy 1, policy_version 565099 (0.0009) [2023-12-26 19:28:33,191][105586] KL-divergence is very high: 182.0392 [2023-12-26 19:28:33,207][105620] Updated weights for policy 1, policy_version 565109 (0.0005) [2023-12-26 19:28:33,233][105586] KL-divergence is very high: 214.0184 [2023-12-26 19:28:33,260][105620] Updated weights for policy 1, policy_version 565119 (0.0005) [2023-12-26 19:28:33,275][105586] KL-divergence is very high: 166.7697 [2023-12-26 19:28:33,773][105692] Updated weights for policy 0, policy_version 564328 (0.0006) [2023-12-26 19:28:33,822][105692] Updated weights for policy 0, policy_version 564338 (0.0008) [2023-12-26 19:28:33,873][105692] Updated weights for policy 0, policy_version 564348 (0.0006) [2023-12-26 19:28:33,952][105620] Updated weights for policy 1, policy_version 565129 (0.0006) [2023-12-26 19:28:34,008][105620] Updated weights for policy 1, policy_version 565139 (0.0008) [2023-12-26 19:28:34,060][105620] Updated weights for policy 1, policy_version 565149 (0.0009) [2023-12-26 19:28:34,065][105586] KL-divergence is very high: 100.4988 [2023-12-26 19:28:34,117][105620] Updated weights for policy 1, policy_version 565159 (0.0010) [2023-12-26 19:28:34,564][105692] Updated weights for policy 0, policy_version 564358 (0.0008) [2023-12-26 19:28:34,621][105692] Updated weights for policy 0, policy_version 564368 (0.0010) [2023-12-26 19:28:34,675][105692] Updated weights for policy 0, policy_version 564378 (0.0010) [2023-12-26 19:28:34,856][105620] Updated weights for policy 1, policy_version 565169 (0.0008) [2023-12-26 19:28:34,907][105620] Updated weights for policy 1, policy_version 565179 (0.0008) [2023-12-26 19:28:34,964][105620] Updated weights for policy 1, policy_version 565189 (0.0008) [2023-12-26 19:28:35,392][105692] Updated weights for policy 0, policy_version 564388 (0.0010) [2023-12-26 19:28:35,453][105692] Updated weights for policy 0, policy_version 564398 (0.0010) [2023-12-26 19:28:35,517][105692] Updated weights for policy 0, policy_version 564408 (0.0010) [2023-12-26 19:28:35,562][105620] Updated weights for policy 1, policy_version 565199 (0.0009) [2023-12-26 19:28:35,630][105620] Updated weights for policy 1, policy_version 565209 (0.0010) [2023-12-26 19:28:35,678][105620] Updated weights for policy 1, policy_version 565219 (0.0008) [2023-12-26 19:28:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 289218560. Throughput: 0: 9905.3, 1: 9784.9. Samples: 289206952. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 19:28:36,062][104569] Avg episode reward: [(0, '1509.823'), (1, '1816.417')] [2023-12-26 19:28:36,212][105692] Updated weights for policy 0, policy_version 564418 (0.0007) [2023-12-26 19:28:36,273][105692] Updated weights for policy 0, policy_version 564428 (0.0009) [2023-12-26 19:28:36,319][105620] Updated weights for policy 1, policy_version 565229 (0.0006) [2023-12-26 19:28:36,332][105692] Updated weights for policy 0, policy_version 564438 (0.0010) [2023-12-26 19:28:36,369][105620] Updated weights for policy 1, policy_version 565239 (0.0008) [2023-12-26 19:28:36,392][105692] Updated weights for policy 0, policy_version 564448 (0.0010) [2023-12-26 19:28:36,431][105620] Updated weights for policy 1, policy_version 565249 (0.0008) [2023-12-26 19:28:37,136][105692] Updated weights for policy 0, policy_version 564458 (0.0010) [2023-12-26 19:28:37,191][105692] Updated weights for policy 0, policy_version 564468 (0.0010) [2023-12-26 19:28:37,215][105620] Updated weights for policy 1, policy_version 565259 (0.0009) [2023-12-26 19:28:37,243][105692] Updated weights for policy 0, policy_version 564478 (0.0008) [2023-12-26 19:28:37,276][105620] Updated weights for policy 1, policy_version 565269 (0.0009) [2023-12-26 19:28:37,336][105620] Updated weights for policy 1, policy_version 565279 (0.0009) [2023-12-26 19:28:37,830][105692] Updated weights for policy 0, policy_version 564488 (0.0009) [2023-12-26 19:28:37,891][105692] Updated weights for policy 0, policy_version 564498 (0.0009) [2023-12-26 19:28:37,940][105692] Updated weights for policy 0, policy_version 564508 (0.0005) [2023-12-26 19:28:38,190][105620] Updated weights for policy 1, policy_version 565289 (0.0010) [2023-12-26 19:28:38,239][105620] Updated weights for policy 1, policy_version 565299 (0.0005) [2023-12-26 19:28:38,293][105620] Updated weights for policy 1, policy_version 565309 (0.0005) [2023-12-26 19:28:38,357][105620] Updated weights for policy 1, policy_version 565319 (0.0007) [2023-12-26 19:28:38,654][105692] Updated weights for policy 0, policy_version 564518 (0.0007) [2023-12-26 19:28:38,709][105692] Updated weights for policy 0, policy_version 564528 (0.0009) [2023-12-26 19:28:38,757][105692] Updated weights for policy 0, policy_version 564538 (0.0009) [2023-12-26 19:28:39,069][105620] Updated weights for policy 1, policy_version 565329 (0.0008) [2023-12-26 19:28:39,127][105620] Updated weights for policy 1, policy_version 565339 (0.0008) [2023-12-26 19:28:39,185][105620] Updated weights for policy 1, policy_version 565349 (0.0008) [2023-12-26 19:28:39,523][105692] Updated weights for policy 0, policy_version 564548 (0.0008) [2023-12-26 19:28:39,585][105692] Updated weights for policy 0, policy_version 564558 (0.0009) [2023-12-26 19:28:39,650][105692] Updated weights for policy 0, policy_version 564568 (0.0010) [2023-12-26 19:28:39,933][105620] Updated weights for policy 1, policy_version 565359 (0.0009) [2023-12-26 19:28:39,999][105620] Updated weights for policy 1, policy_version 565369 (0.0009) [2023-12-26 19:28:40,065][105620] Updated weights for policy 1, policy_version 565379 (0.0008) [2023-12-26 19:28:40,417][105692] Updated weights for policy 0, policy_version 564578 (0.0009) [2023-12-26 19:28:40,466][105692] Updated weights for policy 0, policy_version 564588 (0.0005) [2023-12-26 19:28:40,513][105692] Updated weights for policy 0, policy_version 564598 (0.0005) [2023-12-26 19:28:40,564][105692] Updated weights for policy 0, policy_version 564608 (0.0005) [2023-12-26 19:28:40,770][105620] Updated weights for policy 1, policy_version 565389 (0.0009) [2023-12-26 19:28:40,837][105620] Updated weights for policy 1, policy_version 565399 (0.0005) [2023-12-26 19:28:40,905][105620] Updated weights for policy 1, policy_version 565409 (0.0005) [2023-12-26 19:28:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 289316864. Throughput: 0: 9983.6, 1: 9740.6. Samples: 289324024. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 19:28:41,062][104569] Avg episode reward: [(0, '6661.629'), (1, '6733.862')] [2023-12-26 19:28:41,335][105692] Updated weights for policy 0, policy_version 564618 (0.0011) [2023-12-26 19:28:41,402][105692] Updated weights for policy 0, policy_version 564628 (0.0010) [2023-12-26 19:28:41,458][105692] Updated weights for policy 0, policy_version 564638 (0.0009) [2023-12-26 19:28:41,594][105620] Updated weights for policy 1, policy_version 565419 (0.0007) [2023-12-26 19:28:41,662][105620] Updated weights for policy 1, policy_version 565429 (0.0009) [2023-12-26 19:28:41,726][105620] Updated weights for policy 1, policy_version 565439 (0.0009) [2023-12-26 19:28:42,244][105692] Updated weights for policy 0, policy_version 564648 (0.0010) [2023-12-26 19:28:42,314][105692] Updated weights for policy 0, policy_version 564658 (0.0010) [2023-12-26 19:28:42,379][105692] Updated weights for policy 0, policy_version 564668 (0.0009) [2023-12-26 19:28:42,451][105620] Updated weights for policy 1, policy_version 565449 (0.0009) [2023-12-26 19:28:42,517][105620] Updated weights for policy 1, policy_version 565459 (0.0009) [2023-12-26 19:28:42,571][105620] Updated weights for policy 1, policy_version 565469 (0.0009) [2023-12-26 19:28:42,618][105620] Updated weights for policy 1, policy_version 565479 (0.0008) [2023-12-26 19:28:43,107][105692] Updated weights for policy 0, policy_version 564678 (0.0010) [2023-12-26 19:28:43,168][105692] Updated weights for policy 0, policy_version 564688 (0.0008) [2023-12-26 19:28:43,240][105692] Updated weights for policy 0, policy_version 564698 (0.0005) [2023-12-26 19:28:43,314][105620] Updated weights for policy 1, policy_version 565489 (0.0010) [2023-12-26 19:28:43,379][105620] Updated weights for policy 1, policy_version 565499 (0.0009) [2023-12-26 19:28:43,463][105620] Updated weights for policy 1, policy_version 565509 (0.0010) [2023-12-26 19:28:43,780][105692] Updated weights for policy 0, policy_version 564708 (0.0005) [2023-12-26 19:28:43,837][105692] Updated weights for policy 0, policy_version 564718 (0.0005) [2023-12-26 19:28:43,894][105692] Updated weights for policy 0, policy_version 564728 (0.0008) [2023-12-26 19:28:44,134][105620] Updated weights for policy 1, policy_version 565520 (0.0009) [2023-12-26 19:28:44,192][105620] Updated weights for policy 1, policy_version 565530 (0.0009) [2023-12-26 19:28:44,248][105620] Updated weights for policy 1, policy_version 565540 (0.0008) [2023-12-26 19:28:44,564][105692] Updated weights for policy 0, policy_version 564738 (0.0009) [2023-12-26 19:28:44,613][105692] Updated weights for policy 0, policy_version 564748 (0.0008) [2023-12-26 19:28:44,667][105692] Updated weights for policy 0, policy_version 564758 (0.0009) [2023-12-26 19:28:44,733][105692] Updated weights for policy 0, policy_version 564768 (0.0009) [2023-12-26 19:28:44,986][105620] Updated weights for policy 1, policy_version 565550 (0.0009) [2023-12-26 19:28:45,050][105620] Updated weights for policy 1, policy_version 565560 (0.0008) [2023-12-26 19:28:45,112][105620] Updated weights for policy 1, policy_version 565570 (0.0009) [2023-12-26 19:28:45,545][105692] Updated weights for policy 0, policy_version 564778 (0.0009) [2023-12-26 19:28:45,611][105692] Updated weights for policy 0, policy_version 564788 (0.0009) [2023-12-26 19:28:45,672][105692] Updated weights for policy 0, policy_version 564798 (0.0008) [2023-12-26 19:28:45,804][105620] Updated weights for policy 1, policy_version 565580 (0.0009) [2023-12-26 19:28:45,850][105620] Updated weights for policy 1, policy_version 565590 (0.0008) [2023-12-26 19:28:45,908][105620] Updated weights for policy 1, policy_version 565600 (0.0009) [2023-12-26 19:28:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 289415168. Throughput: 0: 9916.9, 1: 9777.2. Samples: 289381756. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 19:28:46,063][104569] Avg episode reward: [(0, '9263.104'), (1, '9023.003')] [2023-12-26 19:28:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000564800_144605184.pth... [2023-12-26 19:28:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000565608_144809984.pth... [2023-12-26 19:28:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000564456_144515072.pth [2023-12-26 19:28:46,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000563648_144310272.pth [2023-12-26 19:28:46,456][105692] Updated weights for policy 0, policy_version 564808 (0.0009) [2023-12-26 19:28:46,514][105692] Updated weights for policy 0, policy_version 564818 (0.0009) [2023-12-26 19:28:46,576][105692] Updated weights for policy 0, policy_version 564828 (0.0005) [2023-12-26 19:28:46,590][105620] Updated weights for policy 1, policy_version 565610 (0.0009) [2023-12-26 19:28:46,643][105620] Updated weights for policy 1, policy_version 565620 (0.0010) [2023-12-26 19:28:46,697][105620] Updated weights for policy 1, policy_version 565630 (0.0006) [2023-12-26 19:28:46,754][105620] Updated weights for policy 1, policy_version 565640 (0.0008) [2023-12-26 19:28:47,152][105692] Updated weights for policy 0, policy_version 564838 (0.0006) [2023-12-26 19:28:47,215][105692] Updated weights for policy 0, policy_version 564848 (0.0009) [2023-12-26 19:28:47,268][105692] Updated weights for policy 0, policy_version 564858 (0.0011) [2023-12-26 19:28:47,493][105620] Updated weights for policy 1, policy_version 565650 (0.0008) [2023-12-26 19:28:47,548][105620] Updated weights for policy 1, policy_version 565660 (0.0006) [2023-12-26 19:28:47,607][105620] Updated weights for policy 1, policy_version 565670 (0.0005) [2023-12-26 19:28:47,873][105692] Updated weights for policy 0, policy_version 564868 (0.0008) [2023-12-26 19:28:47,926][105692] Updated weights for policy 0, policy_version 564878 (0.0008) [2023-12-26 19:28:47,974][105692] Updated weights for policy 0, policy_version 564888 (0.0010) [2023-12-26 19:28:48,240][105620] Updated weights for policy 1, policy_version 565680 (0.0009) [2023-12-26 19:28:48,297][105620] Updated weights for policy 1, policy_version 565690 (0.0010) [2023-12-26 19:28:48,362][105620] Updated weights for policy 1, policy_version 565701 (0.0009) [2023-12-26 19:28:48,551][105692] Updated weights for policy 0, policy_version 564898 (0.0006) [2023-12-26 19:28:48,609][105692] Updated weights for policy 0, policy_version 564908 (0.0006) [2023-12-26 19:28:48,676][105692] Updated weights for policy 0, policy_version 564918 (0.0007) [2023-12-26 19:28:48,729][105692] Updated weights for policy 0, policy_version 564928 (0.0009) [2023-12-26 19:28:49,153][105620] Updated weights for policy 1, policy_version 565711 (0.0007) [2023-12-26 19:28:49,224][105620] Updated weights for policy 1, policy_version 565721 (0.0007) [2023-12-26 19:28:49,283][105692] Updated weights for policy 0, policy_version 564938 (0.0006) [2023-12-26 19:28:49,291][105620] Updated weights for policy 1, policy_version 565731 (0.0011) [2023-12-26 19:28:49,350][105692] Updated weights for policy 0, policy_version 564948 (0.0008) [2023-12-26 19:28:49,404][105692] Updated weights for policy 0, policy_version 564958 (0.0010) [2023-12-26 19:28:49,904][105620] Updated weights for policy 1, policy_version 565741 (0.0011) [2023-12-26 19:28:49,969][105620] Updated weights for policy 1, policy_version 565751 (0.0010) [2023-12-26 19:28:50,022][105620] Updated weights for policy 1, policy_version 565761 (0.0009) [2023-12-26 19:28:50,075][105692] Updated weights for policy 0, policy_version 564968 (0.0006) [2023-12-26 19:28:50,127][105692] Updated weights for policy 0, policy_version 564978 (0.0005) [2023-12-26 19:28:50,193][105692] Updated weights for policy 0, policy_version 564988 (0.0005) [2023-12-26 19:28:50,749][105620] Updated weights for policy 1, policy_version 565771 (0.0009) [2023-12-26 19:28:50,808][105620] Updated weights for policy 1, policy_version 565781 (0.0009) [2023-12-26 19:28:50,862][105620] Updated weights for policy 1, policy_version 565791 (0.0006) [2023-12-26 19:28:50,914][105692] Updated weights for policy 0, policy_version 564998 (0.0007) [2023-12-26 19:28:50,977][105692] Updated weights for policy 0, policy_version 565008 (0.0008) [2023-12-26 19:28:51,031][105692] Updated weights for policy 0, policy_version 565018 (0.0008) [2023-12-26 19:28:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19549.8). Total num frames: 289521664. Throughput: 0: 10032.1, 1: 9820.9. Samples: 289504768. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 19:28:51,062][104569] Avg episode reward: [(0, '4901.227'), (1, '9115.243')] [2023-12-26 19:28:51,488][105620] Updated weights for policy 1, policy_version 565801 (0.0006) [2023-12-26 19:28:51,553][105620] Updated weights for policy 1, policy_version 565811 (0.0009) [2023-12-26 19:28:51,617][105620] Updated weights for policy 1, policy_version 565821 (0.0009) [2023-12-26 19:28:51,687][105620] Updated weights for policy 1, policy_version 565831 (0.0006) [2023-12-26 19:28:51,829][105692] Updated weights for policy 0, policy_version 565028 (0.0009) [2023-12-26 19:28:51,888][105692] Updated weights for policy 0, policy_version 565038 (0.0006) [2023-12-26 19:28:51,942][105692] Updated weights for policy 0, policy_version 565048 (0.0005) [2023-12-26 19:28:52,479][105620] Updated weights for policy 1, policy_version 565841 (0.0009) [2023-12-26 19:28:52,530][105620] Updated weights for policy 1, policy_version 565851 (0.0009) [2023-12-26 19:28:52,530][105692] Updated weights for policy 0, policy_version 565058 (0.0005) [2023-12-26 19:28:52,588][105620] Updated weights for policy 1, policy_version 565861 (0.0008) [2023-12-26 19:28:52,591][105692] Updated weights for policy 0, policy_version 565068 (0.0005) [2023-12-26 19:28:52,653][105692] Updated weights for policy 0, policy_version 565078 (0.0005) [2023-12-26 19:28:52,719][105692] Updated weights for policy 0, policy_version 565088 (0.0007) [2023-12-26 19:28:53,230][105620] Updated weights for policy 1, policy_version 565871 (0.0008) [2023-12-26 19:28:53,277][105620] Updated weights for policy 1, policy_version 565881 (0.0006) [2023-12-26 19:28:53,323][105620] Updated weights for policy 1, policy_version 565891 (0.0009) [2023-12-26 19:28:53,483][105692] Updated weights for policy 0, policy_version 565098 (0.0006) [2023-12-26 19:28:53,537][105692] Updated weights for policy 0, policy_version 565108 (0.0005) [2023-12-26 19:28:53,593][105692] Updated weights for policy 0, policy_version 565118 (0.0005) [2023-12-26 19:28:54,059][105620] Updated weights for policy 1, policy_version 565901 (0.0009) [2023-12-26 19:28:54,118][105620] Updated weights for policy 1, policy_version 565911 (0.0009) [2023-12-26 19:28:54,174][105620] Updated weights for policy 1, policy_version 565921 (0.0007) [2023-12-26 19:28:54,226][105692] Updated weights for policy 0, policy_version 565128 (0.0011) [2023-12-26 19:28:54,278][105692] Updated weights for policy 0, policy_version 565138 (0.0010) [2023-12-26 19:28:54,336][105692] Updated weights for policy 0, policy_version 565148 (0.0010) [2023-12-26 19:28:54,926][105620] Updated weights for policy 1, policy_version 565931 (0.0006) [2023-12-26 19:28:54,981][105620] Updated weights for policy 1, policy_version 565941 (0.0009) [2023-12-26 19:28:55,035][105620] Updated weights for policy 1, policy_version 565951 (0.0008) [2023-12-26 19:28:55,035][105692] Updated weights for policy 0, policy_version 565158 (0.0008) [2023-12-26 19:28:55,090][105692] Updated weights for policy 0, policy_version 565168 (0.0006) [2023-12-26 19:28:55,146][105692] Updated weights for policy 0, policy_version 565178 (0.0005) [2023-12-26 19:28:55,679][105692] Updated weights for policy 0, policy_version 565188 (0.0007) [2023-12-26 19:28:55,733][105692] Updated weights for policy 0, policy_version 565198 (0.0009) [2023-12-26 19:28:55,795][105692] Updated weights for policy 0, policy_version 565208 (0.0010) [2023-12-26 19:28:55,877][105620] Updated weights for policy 1, policy_version 565961 (0.0009) [2023-12-26 19:28:55,931][105620] Updated weights for policy 1, policy_version 565971 (0.0009) [2023-12-26 19:28:55,994][105620] Updated weights for policy 1, policy_version 565981 (0.0009) [2023-12-26 19:28:56,059][105620] Updated weights for policy 1, policy_version 565991 (0.0010) [2023-12-26 19:28:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 289611776. Throughput: 0: 10044.1, 1: 9687.9. Samples: 289622780. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 19:28:56,062][104569] Avg episode reward: [(0, '5950.525'), (1, '9322.202')] [2023-12-26 19:28:56,409][105692] Updated weights for policy 0, policy_version 565218 (0.0009) [2023-12-26 19:28:56,468][105692] Updated weights for policy 0, policy_version 565228 (0.0009) [2023-12-26 19:28:56,531][105692] Updated weights for policy 0, policy_version 565238 (0.0010) [2023-12-26 19:28:56,597][105692] Updated weights for policy 0, policy_version 565248 (0.0006) [2023-12-26 19:28:56,898][105620] Updated weights for policy 1, policy_version 566001 (0.0008) [2023-12-26 19:28:56,959][105620] Updated weights for policy 1, policy_version 566011 (0.0008) [2023-12-26 19:28:57,021][105620] Updated weights for policy 1, policy_version 566021 (0.0008) [2023-12-26 19:28:57,278][105692] Updated weights for policy 0, policy_version 565258 (0.0008) [2023-12-26 19:28:57,328][105692] Updated weights for policy 0, policy_version 565268 (0.0007) [2023-12-26 19:28:57,386][105692] Updated weights for policy 0, policy_version 565278 (0.0009) [2023-12-26 19:28:57,684][105620] Updated weights for policy 1, policy_version 566031 (0.0006) [2023-12-26 19:28:57,747][105620] Updated weights for policy 1, policy_version 566041 (0.0005) [2023-12-26 19:28:57,808][105620] Updated weights for policy 1, policy_version 566051 (0.0005) [2023-12-26 19:28:58,142][105692] Updated weights for policy 0, policy_version 565289 (0.0009) [2023-12-26 19:28:58,202][105692] Updated weights for policy 0, policy_version 565299 (0.0006) [2023-12-26 19:28:58,263][105692] Updated weights for policy 0, policy_version 565309 (0.0007) [2023-12-26 19:28:58,446][105620] Updated weights for policy 1, policy_version 566061 (0.0007) [2023-12-26 19:28:58,509][105620] Updated weights for policy 1, policy_version 566071 (0.0008) [2023-12-26 19:28:58,569][105620] Updated weights for policy 1, policy_version 566081 (0.0008) [2023-12-26 19:28:59,073][105692] Updated weights for policy 0, policy_version 565319 (0.0008) [2023-12-26 19:28:59,138][105692] Updated weights for policy 0, policy_version 565329 (0.0008) [2023-12-26 19:28:59,198][105692] Updated weights for policy 0, policy_version 565339 (0.0008) [2023-12-26 19:28:59,418][105620] Updated weights for policy 1, policy_version 566091 (0.0007) [2023-12-26 19:28:59,488][105620] Updated weights for policy 1, policy_version 566101 (0.0006) [2023-12-26 19:28:59,552][105620] Updated weights for policy 1, policy_version 566111 (0.0006) [2023-12-26 19:28:59,902][105692] Updated weights for policy 0, policy_version 565349 (0.0009) [2023-12-26 19:28:59,965][105692] Updated weights for policy 0, policy_version 565359 (0.0008) [2023-12-26 19:29:00,035][105692] Updated weights for policy 0, policy_version 565369 (0.0006) [2023-12-26 19:29:00,198][105620] Updated weights for policy 1, policy_version 566121 (0.0006) [2023-12-26 19:29:00,246][105620] Updated weights for policy 1, policy_version 566131 (0.0009) [2023-12-26 19:29:00,308][105620] Updated weights for policy 1, policy_version 566141 (0.0010) [2023-12-26 19:29:00,372][105620] Updated weights for policy 1, policy_version 566151 (0.0009) [2023-12-26 19:29:00,709][105692] Updated weights for policy 0, policy_version 565379 (0.0009) [2023-12-26 19:29:00,771][105692] Updated weights for policy 0, policy_version 565389 (0.0010) [2023-12-26 19:29:00,824][105692] Updated weights for policy 0, policy_version 565399 (0.0006) [2023-12-26 19:29:01,004][105620] Updated weights for policy 1, policy_version 566161 (0.0005) [2023-12-26 19:29:01,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 289710080. Throughput: 0: 10084.8, 1: 9713.6. Samples: 289680968. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 19:29:01,063][104569] Avg episode reward: [(0, '1071.895'), (1, '8989.006')] [2023-12-26 19:29:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000565408_144760832.pth... [2023-12-26 19:29:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000564256_144465920.pth [2023-12-26 19:29:01,074][105620] Updated weights for policy 1, policy_version 566171 (0.0006) [2023-12-26 19:29:01,139][105620] Updated weights for policy 1, policy_version 566181 (0.0006) [2023-12-26 19:29:01,158][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000566184_144957440.pth... [2023-12-26 19:29:01,162][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000565032_144662528.pth [2023-12-26 19:29:01,476][105692] Updated weights for policy 0, policy_version 565409 (0.0006) [2023-12-26 19:29:01,537][105692] Updated weights for policy 0, policy_version 565419 (0.0009) [2023-12-26 19:29:01,593][105692] Updated weights for policy 0, policy_version 565429 (0.0010) [2023-12-26 19:29:01,657][105692] Updated weights for policy 0, policy_version 565439 (0.0010) [2023-12-26 19:29:01,758][105620] Updated weights for policy 1, policy_version 566191 (0.0009) [2023-12-26 19:29:01,815][105620] Updated weights for policy 1, policy_version 566201 (0.0010) [2023-12-26 19:29:01,872][105620] Updated weights for policy 1, policy_version 566211 (0.0009) [2023-12-26 19:29:02,345][105692] Updated weights for policy 0, policy_version 565449 (0.0007) [2023-12-26 19:29:02,406][105692] Updated weights for policy 0, policy_version 565459 (0.0009) [2023-12-26 19:29:02,461][105692] Updated weights for policy 0, policy_version 565469 (0.0009) [2023-12-26 19:29:02,655][105620] Updated weights for policy 1, policy_version 566221 (0.0010) [2023-12-26 19:29:02,720][105620] Updated weights for policy 1, policy_version 566231 (0.0009) [2023-12-26 19:29:02,779][105620] Updated weights for policy 1, policy_version 566241 (0.0009) [2023-12-26 19:29:03,206][105692] Updated weights for policy 0, policy_version 565479 (0.0009) [2023-12-26 19:29:03,270][105692] Updated weights for policy 0, policy_version 565489 (0.0008) [2023-12-26 19:29:03,330][105692] Updated weights for policy 0, policy_version 565499 (0.0008) [2023-12-26 19:29:03,538][105620] Updated weights for policy 1, policy_version 566251 (0.0009) [2023-12-26 19:29:03,603][105620] Updated weights for policy 1, policy_version 566261 (0.0009) [2023-12-26 19:29:03,667][105620] Updated weights for policy 1, policy_version 566271 (0.0010) [2023-12-26 19:29:03,954][105692] Updated weights for policy 0, policy_version 565509 (0.0008) [2023-12-26 19:29:04,006][105692] Updated weights for policy 0, policy_version 565519 (0.0008) [2023-12-26 19:29:04,056][105692] Updated weights for policy 0, policy_version 565529 (0.0010) [2023-12-26 19:29:04,433][105620] Updated weights for policy 1, policy_version 566281 (0.0008) [2023-12-26 19:29:04,493][105620] Updated weights for policy 1, policy_version 566291 (0.0006) [2023-12-26 19:29:04,561][105620] Updated weights for policy 1, policy_version 566301 (0.0006) [2023-12-26 19:29:04,630][105620] Updated weights for policy 1, policy_version 566311 (0.0006) [2023-12-26 19:29:04,809][105692] Updated weights for policy 0, policy_version 565539 (0.0009) [2023-12-26 19:29:04,869][105692] Updated weights for policy 0, policy_version 565549 (0.0005) [2023-12-26 19:29:04,923][105692] Updated weights for policy 0, policy_version 565559 (0.0006) [2023-12-26 19:29:05,219][105620] Updated weights for policy 1, policy_version 566321 (0.0009) [2023-12-26 19:29:05,276][105620] Updated weights for policy 1, policy_version 566331 (0.0009) [2023-12-26 19:29:05,332][105620] Updated weights for policy 1, policy_version 566341 (0.0007) [2023-12-26 19:29:05,554][105692] Updated weights for policy 0, policy_version 565569 (0.0006) [2023-12-26 19:29:05,599][105692] Updated weights for policy 0, policy_version 565579 (0.0005) [2023-12-26 19:29:05,650][105692] Updated weights for policy 0, policy_version 565589 (0.0005) [2023-12-26 19:29:05,700][105692] Updated weights for policy 0, policy_version 565599 (0.0006) [2023-12-26 19:29:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 289808384. Throughput: 0: 10005.8, 1: 9703.4. Samples: 289799000. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 19:29:06,063][104569] Avg episode reward: [(0, '864.900'), (1, '9086.477')] [2023-12-26 19:29:06,138][105620] Updated weights for policy 1, policy_version 566351 (0.0009) [2023-12-26 19:29:06,197][105620] Updated weights for policy 1, policy_version 566361 (0.0007) [2023-12-26 19:29:06,257][105620] Updated weights for policy 1, policy_version 566371 (0.0008) [2023-12-26 19:29:06,425][105692] Updated weights for policy 0, policy_version 565609 (0.0010) [2023-12-26 19:29:06,481][105692] Updated weights for policy 0, policy_version 565619 (0.0010) [2023-12-26 19:29:06,540][105692] Updated weights for policy 0, policy_version 565629 (0.0010) [2023-12-26 19:29:07,022][105620] Updated weights for policy 1, policy_version 566381 (0.0008) [2023-12-26 19:29:07,080][105620] Updated weights for policy 1, policy_version 566391 (0.0007) [2023-12-26 19:29:07,134][105620] Updated weights for policy 1, policy_version 566401 (0.0005) [2023-12-26 19:29:07,295][105692] Updated weights for policy 0, policy_version 565639 (0.0010) [2023-12-26 19:29:07,347][105692] Updated weights for policy 0, policy_version 565649 (0.0010) [2023-12-26 19:29:07,395][105692] Updated weights for policy 0, policy_version 565659 (0.0010) [2023-12-26 19:29:07,860][105620] Updated weights for policy 1, policy_version 566411 (0.0007) [2023-12-26 19:29:07,926][105620] Updated weights for policy 1, policy_version 566421 (0.0008) [2023-12-26 19:29:07,979][105620] Updated weights for policy 1, policy_version 566431 (0.0008) [2023-12-26 19:29:08,138][105692] Updated weights for policy 0, policy_version 565669 (0.0008) [2023-12-26 19:29:08,203][105692] Updated weights for policy 0, policy_version 565679 (0.0005) [2023-12-26 19:29:08,250][105692] Updated weights for policy 0, policy_version 565689 (0.0005) [2023-12-26 19:29:08,744][105620] Updated weights for policy 1, policy_version 566441 (0.0008) [2023-12-26 19:29:08,805][105620] Updated weights for policy 1, policy_version 566451 (0.0007) [2023-12-26 19:29:08,875][105692] Updated weights for policy 0, policy_version 565699 (0.0007) [2023-12-26 19:29:08,877][105620] Updated weights for policy 1, policy_version 566461 (0.0007) [2023-12-26 19:29:08,934][105692] Updated weights for policy 0, policy_version 565709 (0.0011) [2023-12-26 19:29:08,944][105620] Updated weights for policy 1, policy_version 566471 (0.0005) [2023-12-26 19:29:08,997][105692] Updated weights for policy 0, policy_version 565719 (0.0011) [2023-12-26 19:29:09,648][105620] Updated weights for policy 1, policy_version 566481 (0.0008) [2023-12-26 19:29:09,713][105620] Updated weights for policy 1, policy_version 566491 (0.0008) [2023-12-26 19:29:09,774][105620] Updated weights for policy 1, policy_version 566501 (0.0008) [2023-12-26 19:29:09,777][105692] Updated weights for policy 0, policy_version 565729 (0.0010) [2023-12-26 19:29:09,847][105692] Updated weights for policy 0, policy_version 565739 (0.0010) [2023-12-26 19:29:09,912][105692] Updated weights for policy 0, policy_version 565749 (0.0009) [2023-12-26 19:29:09,975][105692] Updated weights for policy 0, policy_version 565759 (0.0010) [2023-12-26 19:29:10,587][105620] Updated weights for policy 1, policy_version 566511 (0.0007) [2023-12-26 19:29:10,644][105620] Updated weights for policy 1, policy_version 566521 (0.0008) [2023-12-26 19:29:10,695][105620] Updated weights for policy 1, policy_version 566531 (0.0008) [2023-12-26 19:29:10,747][105692] Updated weights for policy 0, policy_version 565769 (0.0009) [2023-12-26 19:29:10,808][105692] Updated weights for policy 0, policy_version 565779 (0.0008) [2023-12-26 19:29:10,859][105692] Updated weights for policy 0, policy_version 565789 (0.0008) [2023-12-26 19:29:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 289906688. Throughput: 0: 9988.5, 1: 9622.5. Samples: 289912544. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 19:29:11,062][104569] Avg episode reward: [(0, '962.603'), (1, '9162.810')] [2023-12-26 19:29:11,489][105620] Updated weights for policy 1, policy_version 566541 (0.0008) [2023-12-26 19:29:11,541][105620] Updated weights for policy 1, policy_version 566551 (0.0008) [2023-12-26 19:29:11,603][105620] Updated weights for policy 1, policy_version 566561 (0.0008) [2023-12-26 19:29:11,633][105692] Updated weights for policy 0, policy_version 565799 (0.0011) [2023-12-26 19:29:11,697][105692] Updated weights for policy 0, policy_version 565809 (0.0011) [2023-12-26 19:29:11,759][105692] Updated weights for policy 0, policy_version 565819 (0.0011) [2023-12-26 19:29:12,401][105620] Updated weights for policy 1, policy_version 566571 (0.0009) [2023-12-26 19:29:12,463][105620] Updated weights for policy 1, policy_version 566581 (0.0008) [2023-12-26 19:29:12,522][105620] Updated weights for policy 1, policy_version 566591 (0.0007) [2023-12-26 19:29:12,531][105692] Updated weights for policy 0, policy_version 565829 (0.0010) [2023-12-26 19:29:12,591][105692] Updated weights for policy 0, policy_version 565839 (0.0010) [2023-12-26 19:29:12,653][105692] Updated weights for policy 0, policy_version 565849 (0.0010) [2023-12-26 19:29:13,285][105620] Updated weights for policy 1, policy_version 566601 (0.0006) [2023-12-26 19:29:13,340][105620] Updated weights for policy 1, policy_version 566611 (0.0008) [2023-12-26 19:29:13,397][105620] Updated weights for policy 1, policy_version 566621 (0.0008) [2023-12-26 19:29:13,402][105692] Updated weights for policy 0, policy_version 565859 (0.0010) [2023-12-26 19:29:13,449][105620] Updated weights for policy 1, policy_version 566631 (0.0007) [2023-12-26 19:29:13,457][105692] Updated weights for policy 0, policy_version 565869 (0.0010) [2023-12-26 19:29:13,518][105692] Updated weights for policy 0, policy_version 565879 (0.0010) [2023-12-26 19:29:14,127][105692] Updated weights for policy 0, policy_version 565889 (0.0010) [2023-12-26 19:29:14,183][105692] Updated weights for policy 0, policy_version 565899 (0.0005) [2023-12-26 19:29:14,244][105692] Updated weights for policy 0, policy_version 565909 (0.0007) [2023-12-26 19:29:14,291][105620] Updated weights for policy 1, policy_version 566641 (0.0008) [2023-12-26 19:29:14,303][105692] Updated weights for policy 0, policy_version 565919 (0.0005) [2023-12-26 19:29:14,362][105620] Updated weights for policy 1, policy_version 566651 (0.0009) [2023-12-26 19:29:14,423][105620] Updated weights for policy 1, policy_version 566661 (0.0010) [2023-12-26 19:29:14,885][105692] Updated weights for policy 0, policy_version 565929 (0.0007) [2023-12-26 19:29:14,951][105692] Updated weights for policy 0, policy_version 565939 (0.0009) [2023-12-26 19:29:15,015][105692] Updated weights for policy 0, policy_version 565949 (0.0011) [2023-12-26 19:29:15,115][105620] Updated weights for policy 1, policy_version 566671 (0.0010) [2023-12-26 19:29:15,168][105620] Updated weights for policy 1, policy_version 566681 (0.0011) [2023-12-26 19:29:15,235][105620] Updated weights for policy 1, policy_version 566691 (0.0011) [2023-12-26 19:29:15,761][105692] Updated weights for policy 0, policy_version 565959 (0.0010) [2023-12-26 19:29:15,809][105692] Updated weights for policy 0, policy_version 565969 (0.0010) [2023-12-26 19:29:15,860][105692] Updated weights for policy 0, policy_version 565979 (0.0010) [2023-12-26 19:29:15,968][105620] Updated weights for policy 1, policy_version 566701 (0.0008) [2023-12-26 19:29:16,022][105620] Updated weights for policy 1, policy_version 566711 (0.0006) [2023-12-26 19:29:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 289996800. Throughput: 0: 9872.2, 1: 9573.1. Samples: 289966996. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 19:29:16,062][104569] Avg episode reward: [(0, '3450.195'), (1, '4754.863')] [2023-12-26 19:29:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000565984_144908288.pth... [2023-12-26 19:29:16,070][105620] Updated weights for policy 1, policy_version 566721 (0.0008) [2023-12-26 19:29:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000564800_144605184.pth [2023-12-26 19:29:16,102][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000566728_145096704.pth... [2023-12-26 19:29:16,105][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000565608_144809984.pth [2023-12-26 19:29:16,612][105620] Updated weights for policy 1, policy_version 566731 (0.0005) [2023-12-26 19:29:16,626][105692] Updated weights for policy 0, policy_version 565989 (0.0010) [2023-12-26 19:29:16,671][105620] Updated weights for policy 1, policy_version 566741 (0.0006) [2023-12-26 19:29:16,680][105692] Updated weights for policy 0, policy_version 565999 (0.0010) [2023-12-26 19:29:16,722][105620] Updated weights for policy 1, policy_version 566751 (0.0005) [2023-12-26 19:29:16,728][105692] Updated weights for policy 0, policy_version 566009 (0.0010) [2023-12-26 19:29:17,475][105620] Updated weights for policy 1, policy_version 566761 (0.0006) [2023-12-26 19:29:17,494][105692] Updated weights for policy 0, policy_version 566019 (0.0010) [2023-12-26 19:29:17,532][105620] Updated weights for policy 1, policy_version 566771 (0.0005) [2023-12-26 19:29:17,542][105692] Updated weights for policy 0, policy_version 566029 (0.0011) [2023-12-26 19:29:17,595][105620] Updated weights for policy 1, policy_version 566781 (0.0006) [2023-12-26 19:29:17,597][105692] Updated weights for policy 0, policy_version 566039 (0.0011) [2023-12-26 19:29:17,655][105620] Updated weights for policy 1, policy_version 566791 (0.0006) [2023-12-26 19:29:18,360][105692] Updated weights for policy 0, policy_version 566049 (0.0011) [2023-12-26 19:29:18,398][105620] Updated weights for policy 1, policy_version 566801 (0.0006) [2023-12-26 19:29:18,423][105692] Updated weights for policy 0, policy_version 566059 (0.0011) [2023-12-26 19:29:18,453][105620] Updated weights for policy 1, policy_version 566811 (0.0007) [2023-12-26 19:29:18,482][105692] Updated weights for policy 0, policy_version 566069 (0.0011) [2023-12-26 19:29:18,512][105620] Updated weights for policy 1, policy_version 566821 (0.0005) [2023-12-26 19:29:18,537][105692] Updated weights for policy 0, policy_version 566079 (0.0011) [2023-12-26 19:29:19,264][105620] Updated weights for policy 1, policy_version 566831 (0.0007) [2023-12-26 19:29:19,276][105692] Updated weights for policy 0, policy_version 566089 (0.0010) [2023-12-26 19:29:19,326][105620] Updated weights for policy 1, policy_version 566841 (0.0009) [2023-12-26 19:29:19,341][105692] Updated weights for policy 0, policy_version 566099 (0.0008) [2023-12-26 19:29:19,384][105620] Updated weights for policy 1, policy_version 566851 (0.0008) [2023-12-26 19:29:19,407][105692] Updated weights for policy 0, policy_version 566109 (0.0009) [2023-12-26 19:29:20,054][105620] Updated weights for policy 1, policy_version 566861 (0.0006) [2023-12-26 19:29:20,118][105620] Updated weights for policy 1, policy_version 566871 (0.0006) [2023-12-26 19:29:20,176][105692] Updated weights for policy 0, policy_version 566119 (0.0011) [2023-12-26 19:29:20,182][105620] Updated weights for policy 1, policy_version 566881 (0.0007) [2023-12-26 19:29:20,240][105692] Updated weights for policy 0, policy_version 566129 (0.0009) [2023-12-26 19:29:20,304][105692] Updated weights for policy 0, policy_version 566139 (0.0005) [2023-12-26 19:29:20,851][105620] Updated weights for policy 1, policy_version 566891 (0.0008) [2023-12-26 19:29:20,923][105620] Updated weights for policy 1, policy_version 566901 (0.0008) [2023-12-26 19:29:20,966][105692] Updated weights for policy 0, policy_version 566149 (0.0009) [2023-12-26 19:29:20,992][105620] Updated weights for policy 1, policy_version 566911 (0.0008) [2023-12-26 19:29:21,036][105692] Updated weights for policy 0, policy_version 566159 (0.0011) [2023-12-26 19:29:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 290095104. Throughput: 0: 9897.8, 1: 9606.7. Samples: 290084656. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 19:29:21,063][104569] Avg episode reward: [(0, '6386.324'), (1, '5241.108')] [2023-12-26 19:29:21,106][105692] Updated weights for policy 0, policy_version 566169 (0.0010) [2023-12-26 19:29:21,798][105620] Updated weights for policy 1, policy_version 566921 (0.0007) [2023-12-26 19:29:21,866][105620] Updated weights for policy 1, policy_version 566931 (0.0006) [2023-12-26 19:29:21,904][105692] Updated weights for policy 0, policy_version 566179 (0.0011) [2023-12-26 19:29:21,923][105620] Updated weights for policy 1, policy_version 566941 (0.0007) [2023-12-26 19:29:21,957][105692] Updated weights for policy 0, policy_version 566189 (0.0010) [2023-12-26 19:29:21,983][105620] Updated weights for policy 1, policy_version 566951 (0.0006) [2023-12-26 19:29:22,013][105692] Updated weights for policy 0, policy_version 566199 (0.0010) [2023-12-26 19:29:22,713][105620] Updated weights for policy 1, policy_version 566961 (0.0007) [2023-12-26 19:29:22,780][105620] Updated weights for policy 1, policy_version 566971 (0.0006) [2023-12-26 19:29:22,800][105692] Updated weights for policy 0, policy_version 566209 (0.0011) [2023-12-26 19:29:22,845][105620] Updated weights for policy 1, policy_version 566981 (0.0006) [2023-12-26 19:29:22,871][105692] Updated weights for policy 0, policy_version 566219 (0.0011) [2023-12-26 19:29:22,933][105692] Updated weights for policy 0, policy_version 566229 (0.0011) [2023-12-26 19:29:22,991][105692] Updated weights for policy 0, policy_version 566239 (0.0010) [2023-12-26 19:29:23,490][105620] Updated weights for policy 1, policy_version 566991 (0.0008) [2023-12-26 19:29:23,554][105620] Updated weights for policy 1, policy_version 567001 (0.0008) [2023-12-26 19:29:23,618][105620] Updated weights for policy 1, policy_version 567011 (0.0009) [2023-12-26 19:29:23,698][105692] Updated weights for policy 0, policy_version 566249 (0.0010) [2023-12-26 19:29:23,747][105692] Updated weights for policy 0, policy_version 566259 (0.0010) [2023-12-26 19:29:23,802][105692] Updated weights for policy 0, policy_version 566269 (0.0010) [2023-12-26 19:29:24,432][105620] Updated weights for policy 1, policy_version 567021 (0.0008) [2023-12-26 19:29:24,449][105692] Updated weights for policy 0, policy_version 566279 (0.0010) [2023-12-26 19:29:24,493][105692] Updated weights for policy 0, policy_version 566289 (0.0010) [2023-12-26 19:29:24,497][105620] Updated weights for policy 1, policy_version 567031 (0.0007) [2023-12-26 19:29:24,552][105692] Updated weights for policy 0, policy_version 566299 (0.0010) [2023-12-26 19:29:24,561][105620] Updated weights for policy 1, policy_version 567041 (0.0010) [2023-12-26 19:29:25,213][105692] Updated weights for policy 0, policy_version 566309 (0.0010) [2023-12-26 19:29:25,267][105692] Updated weights for policy 0, policy_version 566319 (0.0010) [2023-12-26 19:29:25,305][105620] Updated weights for policy 1, policy_version 567051 (0.0006) [2023-12-26 19:29:25,312][105692] Updated weights for policy 0, policy_version 566329 (0.0010) [2023-12-26 19:29:25,364][105620] Updated weights for policy 1, policy_version 567061 (0.0008) [2023-12-26 19:29:25,426][105620] Updated weights for policy 1, policy_version 567071 (0.0008) [2023-12-26 19:29:26,017][105692] Updated weights for policy 0, policy_version 566339 (0.0010) [2023-12-26 19:29:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 290185216. Throughput: 0: 9883.3, 1: 9564.3. Samples: 290199168. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 19:29:26,063][104569] Avg episode reward: [(0, '8907.123'), (1, '1230.736')] [2023-12-26 19:29:26,079][105620] Updated weights for policy 1, policy_version 567081 (0.0008) [2023-12-26 19:29:26,082][105692] Updated weights for policy 0, policy_version 566349 (0.0010) [2023-12-26 19:29:26,132][105620] Updated weights for policy 1, policy_version 567091 (0.0005) [2023-12-26 19:29:26,134][105692] Updated weights for policy 0, policy_version 566359 (0.0010) [2023-12-26 19:29:26,182][105620] Updated weights for policy 1, policy_version 567101 (0.0006) [2023-12-26 19:29:26,234][105620] Updated weights for policy 1, policy_version 567111 (0.0008) [2023-12-26 19:29:26,858][105692] Updated weights for policy 0, policy_version 566369 (0.0010) [2023-12-26 19:29:26,905][105692] Updated weights for policy 0, policy_version 566379 (0.0010) [2023-12-26 19:29:26,952][105692] Updated weights for policy 0, policy_version 566389 (0.0010) [2023-12-26 19:29:26,965][105620] Updated weights for policy 1, policy_version 567121 (0.0010) [2023-12-26 19:29:27,006][105692] Updated weights for policy 0, policy_version 566399 (0.0010) [2023-12-26 19:29:27,023][105620] Updated weights for policy 1, policy_version 567131 (0.0010) [2023-12-26 19:29:27,084][105620] Updated weights for policy 1, policy_version 567141 (0.0007) [2023-12-26 19:29:27,760][105692] Updated weights for policy 0, policy_version 566409 (0.0010) [2023-12-26 19:29:27,774][105620] Updated weights for policy 1, policy_version 567151 (0.0009) [2023-12-26 19:29:27,804][105692] Updated weights for policy 0, policy_version 566419 (0.0010) [2023-12-26 19:29:27,821][105620] Updated weights for policy 1, policy_version 567161 (0.0005) [2023-12-26 19:29:27,848][105692] Updated weights for policy 0, policy_version 566429 (0.0010) [2023-12-26 19:29:27,871][105620] Updated weights for policy 1, policy_version 567171 (0.0005) [2023-12-26 19:29:28,603][105692] Updated weights for policy 0, policy_version 566439 (0.0010) [2023-12-26 19:29:28,621][105620] Updated weights for policy 1, policy_version 567181 (0.0006) [2023-12-26 19:29:28,662][105692] Updated weights for policy 0, policy_version 566449 (0.0011) [2023-12-26 19:29:28,676][105620] Updated weights for policy 1, policy_version 567191 (0.0005) [2023-12-26 19:29:28,717][105692] Updated weights for policy 0, policy_version 566459 (0.0010) [2023-12-26 19:29:28,723][105620] Updated weights for policy 1, policy_version 567201 (0.0006) [2023-12-26 19:29:29,468][105692] Updated weights for policy 0, policy_version 566469 (0.0009) [2023-12-26 19:29:29,515][105620] Updated weights for policy 1, policy_version 567211 (0.0007) [2023-12-26 19:29:29,531][105692] Updated weights for policy 0, policy_version 566479 (0.0008) [2023-12-26 19:29:29,577][105620] Updated weights for policy 1, policy_version 567221 (0.0007) [2023-12-26 19:29:29,598][105692] Updated weights for policy 0, policy_version 566489 (0.0007) [2023-12-26 19:29:29,636][105620] Updated weights for policy 1, policy_version 567231 (0.0007) [2023-12-26 19:29:30,210][105692] Updated weights for policy 0, policy_version 566499 (0.0008) [2023-12-26 19:29:30,269][105692] Updated weights for policy 0, policy_version 566509 (0.0011) [2023-12-26 19:29:30,324][105692] Updated weights for policy 0, policy_version 566519 (0.0010) [2023-12-26 19:29:30,452][105620] Updated weights for policy 1, policy_version 567241 (0.0008) [2023-12-26 19:29:30,509][105620] Updated weights for policy 1, policy_version 567251 (0.0008) [2023-12-26 19:29:30,563][105620] Updated weights for policy 1, policy_version 567261 (0.0007) [2023-12-26 19:29:30,619][105620] Updated weights for policy 1, policy_version 567271 (0.0005) [2023-12-26 19:29:31,045][105692] Updated weights for policy 0, policy_version 566529 (0.0008) [2023-12-26 19:29:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 290283520. Throughput: 0: 9899.2, 1: 9571.1. Samples: 290257916. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-12-26 19:29:31,063][104569] Avg episode reward: [(0, '8994.997'), (1, '1748.565')] [2023-12-26 19:29:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000567272_145235968.pth... [2023-12-26 19:29:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000566184_144957440.pth [2023-12-26 19:29:31,103][105692] Updated weights for policy 0, policy_version 566539 (0.0006) [2023-12-26 19:29:31,161][105692] Updated weights for policy 0, policy_version 566549 (0.0008) [2023-12-26 19:29:31,220][105692] Updated weights for policy 0, policy_version 566559 (0.0010) [2023-12-26 19:29:31,223][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000566560_145055744.pth... [2023-12-26 19:29:31,226][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000565408_144760832.pth [2023-12-26 19:29:31,353][105620] Updated weights for policy 1, policy_version 567281 (0.0008) [2023-12-26 19:29:31,422][105620] Updated weights for policy 1, policy_version 567291 (0.0007) [2023-12-26 19:29:31,488][105620] Updated weights for policy 1, policy_version 567301 (0.0007) [2023-12-26 19:29:31,982][105692] Updated weights for policy 0, policy_version 566569 (0.0009) [2023-12-26 19:29:32,036][105692] Updated weights for policy 0, policy_version 566579 (0.0009) [2023-12-26 19:29:32,094][105692] Updated weights for policy 0, policy_version 566589 (0.0009) [2023-12-26 19:29:32,202][105620] Updated weights for policy 1, policy_version 567311 (0.0007) [2023-12-26 19:29:32,268][105620] Updated weights for policy 1, policy_version 567321 (0.0009) [2023-12-26 19:29:32,322][105620] Updated weights for policy 1, policy_version 567331 (0.0008) [2023-12-26 19:29:32,793][105692] Updated weights for policy 0, policy_version 566599 (0.0009) [2023-12-26 19:29:32,858][105692] Updated weights for policy 0, policy_version 566609 (0.0010) [2023-12-26 19:29:32,920][105692] Updated weights for policy 0, policy_version 566619 (0.0009) [2023-12-26 19:29:33,018][105620] Updated weights for policy 1, policy_version 567341 (0.0008) [2023-12-26 19:29:33,063][105620] Updated weights for policy 1, policy_version 567351 (0.0008) [2023-12-26 19:29:33,108][105620] Updated weights for policy 1, policy_version 567361 (0.0008) [2023-12-26 19:29:33,562][105692] Updated weights for policy 0, policy_version 566629 (0.0008) [2023-12-26 19:29:33,626][105692] Updated weights for policy 0, policy_version 566639 (0.0008) [2023-12-26 19:29:33,680][105692] Updated weights for policy 0, policy_version 566649 (0.0008) [2023-12-26 19:29:33,886][105620] Updated weights for policy 1, policy_version 567371 (0.0008) [2023-12-26 19:29:33,939][105620] Updated weights for policy 1, policy_version 567381 (0.0005) [2023-12-26 19:29:34,007][105620] Updated weights for policy 1, policy_version 567391 (0.0005) [2023-12-26 19:29:34,472][105692] Updated weights for policy 0, policy_version 566659 (0.0008) [2023-12-26 19:29:34,533][105692] Updated weights for policy 0, policy_version 566669 (0.0009) [2023-12-26 19:29:34,592][105692] Updated weights for policy 0, policy_version 566679 (0.0009) [2023-12-26 19:29:34,639][105620] Updated weights for policy 1, policy_version 567401 (0.0006) [2023-12-26 19:29:34,688][105620] Updated weights for policy 1, policy_version 567411 (0.0008) [2023-12-26 19:29:34,740][105620] Updated weights for policy 1, policy_version 567421 (0.0008) [2023-12-26 19:29:34,787][105620] Updated weights for policy 1, policy_version 567431 (0.0008) [2023-12-26 19:29:35,330][105692] Updated weights for policy 0, policy_version 566689 (0.0008) [2023-12-26 19:29:35,377][105692] Updated weights for policy 0, policy_version 566699 (0.0006) [2023-12-26 19:29:35,427][105692] Updated weights for policy 0, policy_version 566709 (0.0009) [2023-12-26 19:29:35,473][105692] Updated weights for policy 0, policy_version 566719 (0.0009) [2023-12-26 19:29:35,558][105620] Updated weights for policy 1, policy_version 567441 (0.0008) [2023-12-26 19:29:35,605][105620] Updated weights for policy 1, policy_version 567451 (0.0009) [2023-12-26 19:29:35,652][105620] Updated weights for policy 1, policy_version 567461 (0.0009) [2023-12-26 19:29:36,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 290381824. Throughput: 0: 9789.6, 1: 9511.0. Samples: 290373296. Policy #0 lag: (min: 54.0, avg: 56.0, max: 56.0) [2023-12-26 19:29:36,062][104569] Avg episode reward: [(0, '9172.060'), (1, '6599.056')] [2023-12-26 19:29:36,142][105692] Updated weights for policy 0, policy_version 566729 (0.0008) [2023-12-26 19:29:36,209][105692] Updated weights for policy 0, policy_version 566739 (0.0006) [2023-12-26 19:29:36,275][105692] Updated weights for policy 0, policy_version 566749 (0.0006) [2023-12-26 19:29:36,545][105620] Updated weights for policy 1, policy_version 567471 (0.0008) [2023-12-26 19:29:36,601][105620] Updated weights for policy 1, policy_version 567481 (0.0008) [2023-12-26 19:29:36,667][105620] Updated weights for policy 1, policy_version 567491 (0.0008) [2023-12-26 19:29:36,925][105692] Updated weights for policy 0, policy_version 566759 (0.0005) [2023-12-26 19:29:36,981][105692] Updated weights for policy 0, policy_version 566769 (0.0006) [2023-12-26 19:29:37,045][105692] Updated weights for policy 0, policy_version 566779 (0.0005) [2023-12-26 19:29:37,535][105620] Updated weights for policy 1, policy_version 567501 (0.0008) [2023-12-26 19:29:37,593][105692] Updated weights for policy 0, policy_version 566789 (0.0008) [2023-12-26 19:29:37,596][105620] Updated weights for policy 1, policy_version 567511 (0.0007) [2023-12-26 19:29:37,648][105692] Updated weights for policy 0, policy_version 566799 (0.0010) [2023-12-26 19:29:37,654][105620] Updated weights for policy 1, policy_version 567521 (0.0005) [2023-12-26 19:29:37,696][105692] Updated weights for policy 0, policy_version 566809 (0.0010) [2023-12-26 19:29:38,371][105692] Updated weights for policy 0, policy_version 566819 (0.0007) [2023-12-26 19:29:38,397][105620] Updated weights for policy 1, policy_version 567531 (0.0006) [2023-12-26 19:29:38,431][105692] Updated weights for policy 0, policy_version 566829 (0.0010) [2023-12-26 19:29:38,458][105620] Updated weights for policy 1, policy_version 567541 (0.0008) [2023-12-26 19:29:38,492][105692] Updated weights for policy 0, policy_version 566839 (0.0009) [2023-12-26 19:29:38,510][105620] Updated weights for policy 1, policy_version 567551 (0.0006) [2023-12-26 19:29:39,216][105692] Updated weights for policy 0, policy_version 566849 (0.0007) [2023-12-26 19:29:39,277][105692] Updated weights for policy 0, policy_version 566859 (0.0010) [2023-12-26 19:29:39,287][105620] Updated weights for policy 1, policy_version 567561 (0.0007) [2023-12-26 19:29:39,342][105692] Updated weights for policy 0, policy_version 566869 (0.0008) [2023-12-26 19:29:39,354][105620] Updated weights for policy 1, policy_version 567571 (0.0008) [2023-12-26 19:29:39,409][105692] Updated weights for policy 0, policy_version 566879 (0.0008) [2023-12-26 19:29:39,423][105620] Updated weights for policy 1, policy_version 567581 (0.0008) [2023-12-26 19:29:39,485][105620] Updated weights for policy 1, policy_version 567591 (0.0009) [2023-12-26 19:29:40,198][105692] Updated weights for policy 0, policy_version 566889 (0.0009) [2023-12-26 19:29:40,210][105620] Updated weights for policy 1, policy_version 567601 (0.0006) [2023-12-26 19:29:40,253][105692] Updated weights for policy 0, policy_version 566899 (0.0007) [2023-12-26 19:29:40,271][105620] Updated weights for policy 1, policy_version 567611 (0.0008) [2023-12-26 19:29:40,309][105692] Updated weights for policy 0, policy_version 566909 (0.0007) [2023-12-26 19:29:40,335][105620] Updated weights for policy 1, policy_version 567621 (0.0007) [2023-12-26 19:29:41,029][105692] Updated weights for policy 0, policy_version 566919 (0.0009) [2023-12-26 19:29:41,044][105620] Updated weights for policy 1, policy_version 567631 (0.0008) [2023-12-26 19:29:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 290471936. Throughput: 0: 9768.1, 1: 9445.5. Samples: 290487392. Policy #0 lag: (min: 54.0, avg: 56.0, max: 56.0) [2023-12-26 19:29:41,063][104569] Avg episode reward: [(0, '9172.508'), (1, '8622.986')] [2023-12-26 19:29:41,094][105692] Updated weights for policy 0, policy_version 566929 (0.0010) [2023-12-26 19:29:41,108][105620] Updated weights for policy 1, policy_version 567641 (0.0008) [2023-12-26 19:29:41,164][105692] Updated weights for policy 0, policy_version 566939 (0.0010) [2023-12-26 19:29:41,178][105620] Updated weights for policy 1, policy_version 567651 (0.0007) [2023-12-26 19:29:41,892][105692] Updated weights for policy 0, policy_version 566949 (0.0010) [2023-12-26 19:29:41,945][105692] Updated weights for policy 0, policy_version 566959 (0.0009) [2023-12-26 19:29:41,969][105620] Updated weights for policy 1, policy_version 567661 (0.0008) [2023-12-26 19:29:42,000][105692] Updated weights for policy 0, policy_version 566969 (0.0007) [2023-12-26 19:29:42,033][105620] Updated weights for policy 1, policy_version 567671 (0.0007) [2023-12-26 19:29:42,104][105620] Updated weights for policy 1, policy_version 567681 (0.0008) [2023-12-26 19:29:42,710][105692] Updated weights for policy 0, policy_version 566979 (0.0009) [2023-12-26 19:29:42,755][105620] Updated weights for policy 1, policy_version 567691 (0.0009) [2023-12-26 19:29:42,768][105692] Updated weights for policy 0, policy_version 566989 (0.0005) [2023-12-26 19:29:42,811][105620] Updated weights for policy 1, policy_version 567701 (0.0010) [2023-12-26 19:29:42,818][105692] Updated weights for policy 0, policy_version 566999 (0.0008) [2023-12-26 19:29:42,870][105620] Updated weights for policy 1, policy_version 567711 (0.0010) [2023-12-26 19:29:43,453][105692] Updated weights for policy 0, policy_version 567009 (0.0010) [2023-12-26 19:29:43,502][105692] Updated weights for policy 0, policy_version 567019 (0.0005) [2023-12-26 19:29:43,549][105692] Updated weights for policy 0, policy_version 567029 (0.0006) [2023-12-26 19:29:43,593][105692] Updated weights for policy 0, policy_version 567039 (0.0010) [2023-12-26 19:29:43,601][105620] Updated weights for policy 1, policy_version 567721 (0.0010) [2023-12-26 19:29:43,657][105620] Updated weights for policy 1, policy_version 567731 (0.0011) [2023-12-26 19:29:43,723][105620] Updated weights for policy 1, policy_version 567741 (0.0010) [2023-12-26 19:29:43,793][105620] Updated weights for policy 1, policy_version 567751 (0.0010) [2023-12-26 19:29:44,171][105692] Updated weights for policy 0, policy_version 567049 (0.0010) [2023-12-26 19:29:44,215][105692] Updated weights for policy 0, policy_version 567059 (0.0007) [2023-12-26 19:29:44,260][105692] Updated weights for policy 0, policy_version 567069 (0.0010) [2023-12-26 19:29:44,529][105620] Updated weights for policy 1, policy_version 567761 (0.0010) [2023-12-26 19:29:44,588][105620] Updated weights for policy 1, policy_version 567771 (0.0010) [2023-12-26 19:29:44,655][105620] Updated weights for policy 1, policy_version 567781 (0.0011) [2023-12-26 19:29:45,032][105692] Updated weights for policy 0, policy_version 567079 (0.0011) [2023-12-26 19:29:45,095][105692] Updated weights for policy 0, policy_version 567089 (0.0011) [2023-12-26 19:29:45,162][105692] Updated weights for policy 0, policy_version 567099 (0.0011) [2023-12-26 19:29:45,317][105620] Updated weights for policy 1, policy_version 567791 (0.0011) [2023-12-26 19:29:45,376][105620] Updated weights for policy 1, policy_version 567801 (0.0010) [2023-12-26 19:29:45,445][105620] Updated weights for policy 1, policy_version 567811 (0.0006) [2023-12-26 19:29:45,899][105692] Updated weights for policy 0, policy_version 567109 (0.0011) [2023-12-26 19:29:45,954][105692] Updated weights for policy 0, policy_version 567119 (0.0009) [2023-12-26 19:29:46,002][105692] Updated weights for policy 0, policy_version 567129 (0.0005) [2023-12-26 19:29:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.8, 300 sec: 19521.9). Total num frames: 290578432. Throughput: 0: 9770.8, 1: 9439.3. Samples: 290545424. Policy #0 lag: (min: 54.0, avg: 56.0, max: 56.0) [2023-12-26 19:29:46,062][104569] Avg episode reward: [(0, '9354.997'), (1, '9169.183')] [2023-12-26 19:29:46,064][105620] Updated weights for policy 1, policy_version 567821 (0.0007) [2023-12-26 19:29:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000567136_145203200.pth... [2023-12-26 19:29:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000565984_144908288.pth [2023-12-26 19:29:46,126][105620] Updated weights for policy 1, policy_version 567831 (0.0009) [2023-12-26 19:29:46,185][105620] Updated weights for policy 1, policy_version 567841 (0.0011) [2023-12-26 19:29:46,229][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000567848_145383424.pth... [2023-12-26 19:29:46,233][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000566728_145096704.pth [2023-12-26 19:29:46,703][105692] Updated weights for policy 0, policy_version 567139 (0.0006) [2023-12-26 19:29:46,761][105692] Updated weights for policy 0, policy_version 567149 (0.0008) [2023-12-26 19:29:46,779][105620] Updated weights for policy 1, policy_version 567851 (0.0011) [2023-12-26 19:29:46,816][105692] Updated weights for policy 0, policy_version 567159 (0.0005) [2023-12-26 19:29:46,837][105620] Updated weights for policy 1, policy_version 567861 (0.0010) [2023-12-26 19:29:46,896][105620] Updated weights for policy 1, policy_version 567871 (0.0010) [2023-12-26 19:29:47,549][105620] Updated weights for policy 1, policy_version 567881 (0.0010) [2023-12-26 19:29:47,570][105692] Updated weights for policy 0, policy_version 567169 (0.0005) [2023-12-26 19:29:47,608][105620] Updated weights for policy 1, policy_version 567891 (0.0008) [2023-12-26 19:29:47,620][105692] Updated weights for policy 0, policy_version 567179 (0.0005) [2023-12-26 19:29:47,665][105620] Updated weights for policy 1, policy_version 567901 (0.0008) [2023-12-26 19:29:47,674][105692] Updated weights for policy 0, policy_version 567189 (0.0005) [2023-12-26 19:29:47,730][105620] Updated weights for policy 1, policy_version 567911 (0.0009) [2023-12-26 19:29:47,734][105692] Updated weights for policy 0, policy_version 567199 (0.0007) [2023-12-26 19:29:48,303][105692] Updated weights for policy 0, policy_version 567209 (0.0005) [2023-12-26 19:29:48,362][105692] Updated weights for policy 0, policy_version 567219 (0.0009) [2023-12-26 19:29:48,425][105692] Updated weights for policy 0, policy_version 567229 (0.0011) [2023-12-26 19:29:48,559][105620] Updated weights for policy 1, policy_version 567921 (0.0008) [2023-12-26 19:29:48,615][105620] Updated weights for policy 1, policy_version 567931 (0.0008) [2023-12-26 19:29:48,673][105620] Updated weights for policy 1, policy_version 567941 (0.0006) [2023-12-26 19:29:49,024][105692] Updated weights for policy 0, policy_version 567239 (0.0007) [2023-12-26 19:29:49,082][105692] Updated weights for policy 0, policy_version 567249 (0.0005) [2023-12-26 19:29:49,142][105692] Updated weights for policy 0, policy_version 567259 (0.0009) [2023-12-26 19:29:49,365][105620] Updated weights for policy 1, policy_version 567951 (0.0007) [2023-12-26 19:29:49,425][105620] Updated weights for policy 1, policy_version 567961 (0.0008) [2023-12-26 19:29:49,488][105620] Updated weights for policy 1, policy_version 567971 (0.0007) [2023-12-26 19:29:49,862][105692] Updated weights for policy 0, policy_version 567269 (0.0012) [2023-12-26 19:29:49,921][105692] Updated weights for policy 0, policy_version 567279 (0.0009) [2023-12-26 19:29:49,979][105692] Updated weights for policy 0, policy_version 567289 (0.0009) [2023-12-26 19:29:50,106][105620] Updated weights for policy 1, policy_version 567981 (0.0006) [2023-12-26 19:29:50,178][105620] Updated weights for policy 1, policy_version 567991 (0.0006) [2023-12-26 19:29:50,240][105620] Updated weights for policy 1, policy_version 568001 (0.0007) [2023-12-26 19:29:50,715][105692] Updated weights for policy 0, policy_version 567299 (0.0010) [2023-12-26 19:29:50,767][105620] Updated weights for policy 1, policy_version 568011 (0.0006) [2023-12-26 19:29:50,783][105692] Updated weights for policy 0, policy_version 567309 (0.0010) [2023-12-26 19:29:50,836][105692] Updated weights for policy 0, policy_version 567319 (0.0011) [2023-12-26 19:29:50,836][105620] Updated weights for policy 1, policy_version 568021 (0.0006) [2023-12-26 19:29:50,905][105620] Updated weights for policy 1, policy_version 568031 (0.0006) [2023-12-26 19:29:51,062][104569] Fps is (10 sec: 21299.4, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 290684928. Throughput: 0: 9826.8, 1: 9487.2. Samples: 290668128. Policy #0 lag: (min: 54.0, avg: 56.0, max: 56.0) [2023-12-26 19:29:51,062][104569] Avg episode reward: [(0, '9170.432'), (1, '9262.112')] [2023-12-26 19:29:51,511][105620] Updated weights for policy 1, policy_version 568041 (0.0006) [2023-12-26 19:29:51,572][105620] Updated weights for policy 1, policy_version 568051 (0.0009) [2023-12-26 19:29:51,638][105620] Updated weights for policy 1, policy_version 568061 (0.0009) [2023-12-26 19:29:51,645][105692] Updated weights for policy 0, policy_version 567329 (0.0008) [2023-12-26 19:29:51,701][105692] Updated weights for policy 0, policy_version 567339 (0.0008) [2023-12-26 19:29:51,701][105620] Updated weights for policy 1, policy_version 568071 (0.0008) [2023-12-26 19:29:51,769][105692] Updated weights for policy 0, policy_version 567349 (0.0009) [2023-12-26 19:29:51,837][105692] Updated weights for policy 0, policy_version 567359 (0.0009) [2023-12-26 19:29:52,387][105620] Updated weights for policy 1, policy_version 568081 (0.0007) [2023-12-26 19:29:52,441][105620] Updated weights for policy 1, policy_version 568091 (0.0005) [2023-12-26 19:29:52,492][105620] Updated weights for policy 1, policy_version 568101 (0.0005) [2023-12-26 19:29:52,669][105692] Updated weights for policy 0, policy_version 567369 (0.0009) [2023-12-26 19:29:52,725][105692] Updated weights for policy 0, policy_version 567379 (0.0009) [2023-12-26 19:29:52,781][105692] Updated weights for policy 0, policy_version 567389 (0.0009) [2023-12-26 19:29:53,063][105620] Updated weights for policy 1, policy_version 568111 (0.0006) [2023-12-26 19:29:53,127][105620] Updated weights for policy 1, policy_version 568121 (0.0009) [2023-12-26 19:29:53,181][105620] Updated weights for policy 1, policy_version 568131 (0.0009) [2023-12-26 19:29:53,566][105692] Updated weights for policy 0, policy_version 567399 (0.0009) [2023-12-26 19:29:53,615][105692] Updated weights for policy 0, policy_version 567409 (0.0008) [2023-12-26 19:29:53,662][105692] Updated weights for policy 0, policy_version 567419 (0.0008) [2023-12-26 19:29:53,859][105620] Updated weights for policy 1, policy_version 568141 (0.0007) [2023-12-26 19:29:53,916][105620] Updated weights for policy 1, policy_version 568151 (0.0005) [2023-12-26 19:29:53,977][105620] Updated weights for policy 1, policy_version 568161 (0.0005) [2023-12-26 19:29:54,437][105692] Updated weights for policy 0, policy_version 567429 (0.0009) [2023-12-26 19:29:54,489][105692] Updated weights for policy 0, policy_version 567440 (0.0010) [2023-12-26 19:29:54,492][105620] Updated weights for policy 1, policy_version 568171 (0.0005) [2023-12-26 19:29:54,539][105692] Updated weights for policy 0, policy_version 567450 (0.0006) [2023-12-26 19:29:54,541][105620] Updated weights for policy 1, policy_version 568181 (0.0006) [2023-12-26 19:29:54,604][105620] Updated weights for policy 1, policy_version 568191 (0.0008) [2023-12-26 19:29:55,156][105692] Updated weights for policy 0, policy_version 567460 (0.0008) [2023-12-26 19:29:55,221][105692] Updated weights for policy 0, policy_version 567470 (0.0009) [2023-12-26 19:29:55,284][105692] Updated weights for policy 0, policy_version 567480 (0.0009) [2023-12-26 19:29:55,433][105620] Updated weights for policy 1, policy_version 568201 (0.0009) [2023-12-26 19:29:55,493][105620] Updated weights for policy 1, policy_version 568211 (0.0007) [2023-12-26 19:29:55,560][105620] Updated weights for policy 1, policy_version 568221 (0.0007) [2023-12-26 19:29:55,619][105620] Updated weights for policy 1, policy_version 568231 (0.0008) [2023-12-26 19:29:55,891][105692] Updated weights for policy 0, policy_version 567490 (0.0009) [2023-12-26 19:29:55,942][105692] Updated weights for policy 0, policy_version 567500 (0.0007) [2023-12-26 19:29:55,989][105692] Updated weights for policy 0, policy_version 567510 (0.0008) [2023-12-26 19:29:56,045][105692] Updated weights for policy 0, policy_version 567520 (0.0007) [2023-12-26 19:29:56,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 290783232. Throughput: 0: 9771.5, 1: 9671.1. Samples: 290787460. Policy #0 lag: (min: 54.0, avg: 56.0, max: 56.0) [2023-12-26 19:29:56,062][104569] Avg episode reward: [(0, '8986.973'), (1, '9168.815')] [2023-12-26 19:29:56,345][105620] Updated weights for policy 1, policy_version 568241 (0.0010) [2023-12-26 19:29:56,400][105620] Updated weights for policy 1, policy_version 568251 (0.0010) [2023-12-26 19:29:56,463][105620] Updated weights for policy 1, policy_version 568261 (0.0010) [2023-12-26 19:29:56,812][105692] Updated weights for policy 0, policy_version 567530 (0.0010) [2023-12-26 19:29:56,869][105692] Updated weights for policy 0, policy_version 567540 (0.0005) [2023-12-26 19:29:56,923][105692] Updated weights for policy 0, policy_version 567550 (0.0005) [2023-12-26 19:29:57,176][105620] Updated weights for policy 1, policy_version 568271 (0.0007) [2023-12-26 19:29:57,230][105620] Updated weights for policy 1, policy_version 568281 (0.0005) [2023-12-26 19:29:57,280][105620] Updated weights for policy 1, policy_version 568291 (0.0010) [2023-12-26 19:29:57,537][105692] Updated weights for policy 0, policy_version 567560 (0.0007) [2023-12-26 19:29:57,597][105692] Updated weights for policy 0, policy_version 567570 (0.0005) [2023-12-26 19:29:57,650][105692] Updated weights for policy 0, policy_version 567580 (0.0005) [2023-12-26 19:29:57,807][105620] Updated weights for policy 1, policy_version 568301 (0.0006) [2023-12-26 19:29:57,857][105620] Updated weights for policy 1, policy_version 568311 (0.0005) [2023-12-26 19:29:57,919][105620] Updated weights for policy 1, policy_version 568321 (0.0005) [2023-12-26 19:29:58,355][105692] Updated weights for policy 0, policy_version 567590 (0.0010) [2023-12-26 19:29:58,421][105692] Updated weights for policy 0, policy_version 567600 (0.0011) [2023-12-26 19:29:58,485][105692] Updated weights for policy 0, policy_version 567610 (0.0010) [2023-12-26 19:29:58,533][105620] Updated weights for policy 1, policy_version 568331 (0.0006) [2023-12-26 19:29:58,596][105620] Updated weights for policy 1, policy_version 568341 (0.0008) [2023-12-26 19:29:58,658][105620] Updated weights for policy 1, policy_version 568351 (0.0008) [2023-12-26 19:29:59,286][105692] Updated weights for policy 0, policy_version 567620 (0.0010) [2023-12-26 19:29:59,343][105692] Updated weights for policy 0, policy_version 567630 (0.0011) [2023-12-26 19:29:59,407][105692] Updated weights for policy 0, policy_version 567640 (0.0008) [2023-12-26 19:29:59,457][105620] Updated weights for policy 1, policy_version 568361 (0.0008) [2023-12-26 19:29:59,519][105620] Updated weights for policy 1, policy_version 568371 (0.0010) [2023-12-26 19:29:59,585][105620] Updated weights for policy 1, policy_version 568381 (0.0011) [2023-12-26 19:29:59,643][105620] Updated weights for policy 1, policy_version 568391 (0.0010) [2023-12-26 19:30:00,112][105692] Updated weights for policy 0, policy_version 567650 (0.0007) [2023-12-26 19:30:00,178][105692] Updated weights for policy 0, policy_version 567660 (0.0006) [2023-12-26 19:30:00,233][105692] Updated weights for policy 0, policy_version 567670 (0.0007) [2023-12-26 19:30:00,293][105620] Updated weights for policy 1, policy_version 568401 (0.0008) [2023-12-26 19:30:00,300][105692] Updated weights for policy 0, policy_version 567680 (0.0006) [2023-12-26 19:30:00,362][105620] Updated weights for policy 1, policy_version 568411 (0.0007) [2023-12-26 19:30:00,430][105620] Updated weights for policy 1, policy_version 568421 (0.0006) [2023-12-26 19:30:00,936][105692] Updated weights for policy 0, policy_version 567690 (0.0008) [2023-12-26 19:30:00,994][105692] Updated weights for policy 0, policy_version 567700 (0.0008) [2023-12-26 19:30:01,059][105692] Updated weights for policy 0, policy_version 567710 (0.0008) [2023-12-26 19:30:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 290873344. Throughput: 0: 9837.8, 1: 9783.0. Samples: 290849932. Policy #0 lag: (min: 54.0, avg: 56.0, max: 56.0) [2023-12-26 19:30:01,062][104569] Avg episode reward: [(0, '8987.022'), (1, '6685.288')] [2023-12-26 19:30:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000567712_145350656.pth... [2023-12-26 19:30:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000566560_145055744.pth [2023-12-26 19:30:01,122][105620] Updated weights for policy 1, policy_version 568431 (0.0006) [2023-12-26 19:30:01,178][105620] Updated weights for policy 1, policy_version 568441 (0.0008) [2023-12-26 19:30:01,226][105620] Updated weights for policy 1, policy_version 568451 (0.0010) [2023-12-26 19:30:01,254][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000568456_145539072.pth... [2023-12-26 19:30:01,257][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000567272_145235968.pth [2023-12-26 19:30:01,816][105692] Updated weights for policy 0, policy_version 567720 (0.0008) [2023-12-26 19:30:01,881][105692] Updated weights for policy 0, policy_version 567730 (0.0010) [2023-12-26 19:30:01,933][105692] Updated weights for policy 0, policy_version 567740 (0.0010) [2023-12-26 19:30:01,970][105620] Updated weights for policy 1, policy_version 568461 (0.0011) [2023-12-26 19:30:02,022][105620] Updated weights for policy 1, policy_version 568471 (0.0010) [2023-12-26 19:30:02,070][105620] Updated weights for policy 1, policy_version 568481 (0.0010) [2023-12-26 19:30:02,568][105692] Updated weights for policy 0, policy_version 567750 (0.0008) [2023-12-26 19:30:02,619][105692] Updated weights for policy 0, policy_version 567760 (0.0009) [2023-12-26 19:30:02,627][105585] KL-divergence is very high: 137.6087 [2023-12-26 19:30:02,672][105692] Updated weights for policy 0, policy_version 567770 (0.0006) [2023-12-26 19:30:02,673][105585] KL-divergence is very high: 195.9644 [2023-12-26 19:30:02,869][105620] Updated weights for policy 1, policy_version 568491 (0.0010) [2023-12-26 19:30:02,925][105620] Updated weights for policy 1, policy_version 568501 (0.0009) [2023-12-26 19:30:02,987][105620] Updated weights for policy 1, policy_version 568511 (0.0010) [2023-12-26 19:30:03,313][105692] Updated weights for policy 0, policy_version 567780 (0.0007) [2023-12-26 19:30:03,367][105692] Updated weights for policy 0, policy_version 567790 (0.0009) [2023-12-26 19:30:03,428][105692] Updated weights for policy 0, policy_version 567800 (0.0010) [2023-12-26 19:30:03,687][105620] Updated weights for policy 1, policy_version 568521 (0.0009) [2023-12-26 19:30:03,740][105620] Updated weights for policy 1, policy_version 568531 (0.0005) [2023-12-26 19:30:03,800][105620] Updated weights for policy 1, policy_version 568541 (0.0007) [2023-12-26 19:30:03,857][105620] Updated weights for policy 1, policy_version 568551 (0.0010) [2023-12-26 19:30:04,161][105692] Updated weights for policy 0, policy_version 567810 (0.0008) [2023-12-26 19:30:04,228][105692] Updated weights for policy 0, policy_version 567820 (0.0005) [2023-12-26 19:30:04,298][105692] Updated weights for policy 0, policy_version 567830 (0.0005) [2023-12-26 19:30:04,360][105692] Updated weights for policy 0, policy_version 567840 (0.0007) [2023-12-26 19:30:04,556][105620] Updated weights for policy 1, policy_version 568561 (0.0010) [2023-12-26 19:30:04,614][105620] Updated weights for policy 1, policy_version 568571 (0.0010) [2023-12-26 19:30:04,672][105620] Updated weights for policy 1, policy_version 568581 (0.0007) [2023-12-26 19:30:04,957][105692] Updated weights for policy 0, policy_version 567850 (0.0008) [2023-12-26 19:30:05,006][105692] Updated weights for policy 0, policy_version 567860 (0.0008) [2023-12-26 19:30:05,054][105692] Updated weights for policy 0, policy_version 567870 (0.0008) [2023-12-26 19:30:05,335][105620] Updated weights for policy 1, policy_version 568591 (0.0009) [2023-12-26 19:30:05,394][105620] Updated weights for policy 1, policy_version 568601 (0.0011) [2023-12-26 19:30:05,450][105620] Updated weights for policy 1, policy_version 568611 (0.0010) [2023-12-26 19:30:05,816][105692] Updated weights for policy 0, policy_version 567880 (0.0006) [2023-12-26 19:30:05,868][105692] Updated weights for policy 0, policy_version 567890 (0.0005) [2023-12-26 19:30:05,918][105692] Updated weights for policy 0, policy_version 567900 (0.0008) [2023-12-26 19:30:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 290979840. Throughput: 0: 9860.6, 1: 9787.3. Samples: 290968808. Policy #0 lag: (min: 54.0, avg: 56.0, max: 56.0) [2023-12-26 19:30:06,062][104569] Avg episode reward: [(0, '9170.539'), (1, '5475.379')] [2023-12-26 19:30:06,188][105620] Updated weights for policy 1, policy_version 568621 (0.0010) [2023-12-26 19:30:06,238][105620] Updated weights for policy 1, policy_version 568631 (0.0011) [2023-12-26 19:30:06,297][105620] Updated weights for policy 1, policy_version 568641 (0.0011) [2023-12-26 19:30:06,713][105692] Updated weights for policy 0, policy_version 567910 (0.0009) [2023-12-26 19:30:06,777][105692] Updated weights for policy 0, policy_version 567920 (0.0010) [2023-12-26 19:30:06,833][105692] Updated weights for policy 0, policy_version 567930 (0.0012) [2023-12-26 19:30:06,868][105620] Updated weights for policy 1, policy_version 568651 (0.0006) [2023-12-26 19:30:06,922][105620] Updated weights for policy 1, policy_version 568661 (0.0005) [2023-12-26 19:30:06,984][105620] Updated weights for policy 1, policy_version 568671 (0.0008) [2023-12-26 19:30:07,617][105692] Updated weights for policy 0, policy_version 567940 (0.0007) [2023-12-26 19:30:07,663][105692] Updated weights for policy 0, policy_version 567950 (0.0005) [2023-12-26 19:30:07,695][105620] Updated weights for policy 1, policy_version 568681 (0.0011) [2023-12-26 19:30:07,713][105692] Updated weights for policy 0, policy_version 567960 (0.0006) [2023-12-26 19:30:07,754][105620] Updated weights for policy 1, policy_version 568691 (0.0011) [2023-12-26 19:30:07,812][105620] Updated weights for policy 1, policy_version 568701 (0.0010) [2023-12-26 19:30:07,861][105620] Updated weights for policy 1, policy_version 568711 (0.0010) [2023-12-26 19:30:08,446][105692] Updated weights for policy 0, policy_version 567970 (0.0006) [2023-12-26 19:30:08,501][105692] Updated weights for policy 0, policy_version 567980 (0.0008) [2023-12-26 19:30:08,534][105620] Updated weights for policy 1, policy_version 568721 (0.0010) [2023-12-26 19:30:08,556][105692] Updated weights for policy 0, policy_version 567990 (0.0006) [2023-12-26 19:30:08,597][105620] Updated weights for policy 1, policy_version 568731 (0.0010) [2023-12-26 19:30:08,611][105692] Updated weights for policy 0, policy_version 568000 (0.0008) [2023-12-26 19:30:08,658][105620] Updated weights for policy 1, policy_version 568741 (0.0010) [2023-12-26 19:30:09,364][105692] Updated weights for policy 0, policy_version 568010 (0.0008) [2023-12-26 19:30:09,398][105620] Updated weights for policy 1, policy_version 568751 (0.0008) [2023-12-26 19:30:09,424][105692] Updated weights for policy 0, policy_version 568020 (0.0008) [2023-12-26 19:30:09,465][105620] Updated weights for policy 1, policy_version 568761 (0.0008) [2023-12-26 19:30:09,488][105692] Updated weights for policy 0, policy_version 568030 (0.0008) [2023-12-26 19:30:09,528][105620] Updated weights for policy 1, policy_version 568771 (0.0008) [2023-12-26 19:30:10,199][105692] Updated weights for policy 0, policy_version 568040 (0.0007) [2023-12-26 19:30:10,263][105692] Updated weights for policy 0, policy_version 568050 (0.0008) [2023-12-26 19:30:10,268][105620] Updated weights for policy 1, policy_version 568781 (0.0009) [2023-12-26 19:30:10,323][105692] Updated weights for policy 0, policy_version 568060 (0.0008) [2023-12-26 19:30:10,335][105620] Updated weights for policy 1, policy_version 568791 (0.0010) [2023-12-26 19:30:10,394][105620] Updated weights for policy 1, policy_version 568801 (0.0010) [2023-12-26 19:30:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 291069952. Throughput: 0: 9810.2, 1: 9842.7. Samples: 291083544. Policy #0 lag: (min: 54.0, avg: 56.0, max: 56.0) [2023-12-26 19:30:11,062][104569] Avg episode reward: [(0, '9354.202'), (1, '7076.516')] [2023-12-26 19:30:11,114][105692] Updated weights for policy 0, policy_version 568070 (0.0009) [2023-12-26 19:30:11,142][105620] Updated weights for policy 1, policy_version 568811 (0.0011) [2023-12-26 19:30:11,191][105692] Updated weights for policy 0, policy_version 568080 (0.0007) [2023-12-26 19:30:11,216][105620] Updated weights for policy 1, policy_version 568821 (0.0010) [2023-12-26 19:30:11,262][105692] Updated weights for policy 0, policy_version 568090 (0.0007) [2023-12-26 19:30:11,283][105620] Updated weights for policy 1, policy_version 568831 (0.0010) [2023-12-26 19:30:12,041][105620] Updated weights for policy 1, policy_version 568841 (0.0010) [2023-12-26 19:30:12,106][105620] Updated weights for policy 1, policy_version 568851 (0.0006) [2023-12-26 19:30:12,135][105692] Updated weights for policy 0, policy_version 568100 (0.0008) [2023-12-26 19:30:12,169][105620] Updated weights for policy 1, policy_version 568861 (0.0006) [2023-12-26 19:30:12,204][105692] Updated weights for policy 0, policy_version 568110 (0.0006) [2023-12-26 19:30:12,238][105620] Updated weights for policy 1, policy_version 568871 (0.0008) [2023-12-26 19:30:12,274][105692] Updated weights for policy 0, policy_version 568120 (0.0007) [2023-12-26 19:30:12,871][105620] Updated weights for policy 1, policy_version 568881 (0.0005) [2023-12-26 19:30:12,928][105620] Updated weights for policy 1, policy_version 568891 (0.0009) [2023-12-26 19:30:12,987][105620] Updated weights for policy 1, policy_version 568901 (0.0010) [2023-12-26 19:30:13,026][105692] Updated weights for policy 0, policy_version 568130 (0.0008) [2023-12-26 19:30:13,094][105692] Updated weights for policy 0, policy_version 568140 (0.0009) [2023-12-26 19:30:13,162][105692] Updated weights for policy 0, policy_version 568150 (0.0010) [2023-12-26 19:30:13,213][105692] Updated weights for policy 0, policy_version 568160 (0.0010) [2023-12-26 19:30:13,643][105620] Updated weights for policy 1, policy_version 568911 (0.0007) [2023-12-26 19:30:13,705][105620] Updated weights for policy 1, policy_version 568921 (0.0005) [2023-12-26 19:30:13,762][105620] Updated weights for policy 1, policy_version 568931 (0.0009) [2023-12-26 19:30:13,789][105692] Updated weights for policy 0, policy_version 568170 (0.0008) [2023-12-26 19:30:13,854][105692] Updated weights for policy 0, policy_version 568180 (0.0008) [2023-12-26 19:30:13,919][105692] Updated weights for policy 0, policy_version 568190 (0.0007) [2023-12-26 19:30:14,466][105620] Updated weights for policy 1, policy_version 568941 (0.0010) [2023-12-26 19:30:14,518][105620] Updated weights for policy 1, policy_version 568951 (0.0010) [2023-12-26 19:30:14,529][105692] Updated weights for policy 0, policy_version 568200 (0.0005) [2023-12-26 19:30:14,570][105620] Updated weights for policy 1, policy_version 568961 (0.0010) [2023-12-26 19:30:14,583][105692] Updated weights for policy 0, policy_version 568210 (0.0006) [2023-12-26 19:30:14,648][105692] Updated weights for policy 0, policy_version 568220 (0.0005) [2023-12-26 19:30:15,186][105692] Updated weights for policy 0, policy_version 568230 (0.0007) [2023-12-26 19:30:15,252][105692] Updated weights for policy 0, policy_version 568240 (0.0006) [2023-12-26 19:30:15,319][105692] Updated weights for policy 0, policy_version 568250 (0.0006) [2023-12-26 19:30:15,352][105620] Updated weights for policy 1, policy_version 568971 (0.0010) [2023-12-26 19:30:15,415][105620] Updated weights for policy 1, policy_version 568981 (0.0010) [2023-12-26 19:30:15,471][105620] Updated weights for policy 1, policy_version 568991 (0.0010) [2023-12-26 19:30:15,959][105692] Updated weights for policy 0, policy_version 568260 (0.0006) [2023-12-26 19:30:16,013][105692] Updated weights for policy 0, policy_version 568270 (0.0006) [2023-12-26 19:30:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 291168256. Throughput: 0: 9776.8, 1: 9840.9. Samples: 291140712. Policy #0 lag: (min: 54.0, avg: 56.0, max: 56.0) [2023-12-26 19:30:16,063][104569] Avg episode reward: [(0, '9354.767'), (1, '9262.439')] [2023-12-26 19:30:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000569000_145678336.pth... [2023-12-26 19:30:16,070][105692] Updated weights for policy 0, policy_version 568280 (0.0006) [2023-12-26 19:30:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000567848_145383424.pth [2023-12-26 19:30:16,118][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000568288_145498112.pth... [2023-12-26 19:30:16,123][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000567136_145203200.pth [2023-12-26 19:30:16,218][105620] Updated weights for policy 1, policy_version 569001 (0.0010) [2023-12-26 19:30:16,280][105620] Updated weights for policy 1, policy_version 569011 (0.0010) [2023-12-26 19:30:16,338][105620] Updated weights for policy 1, policy_version 569021 (0.0010) [2023-12-26 19:30:16,398][105620] Updated weights for policy 1, policy_version 569031 (0.0009) [2023-12-26 19:30:16,744][105692] Updated weights for policy 0, policy_version 568290 (0.0008) [2023-12-26 19:30:16,796][105692] Updated weights for policy 0, policy_version 568300 (0.0010) [2023-12-26 19:30:16,854][105692] Updated weights for policy 0, policy_version 568310 (0.0007) [2023-12-26 19:30:16,918][105692] Updated weights for policy 0, policy_version 568320 (0.0005) [2023-12-26 19:30:17,026][105620] Updated weights for policy 1, policy_version 569041 (0.0006) [2023-12-26 19:30:17,092][105620] Updated weights for policy 1, policy_version 569051 (0.0009) [2023-12-26 19:30:17,146][105620] Updated weights for policy 1, policy_version 569061 (0.0010) [2023-12-26 19:30:17,453][105692] Updated weights for policy 0, policy_version 568330 (0.0006) [2023-12-26 19:30:17,510][105692] Updated weights for policy 0, policy_version 568340 (0.0010) [2023-12-26 19:30:17,565][105692] Updated weights for policy 0, policy_version 568350 (0.0010) [2023-12-26 19:30:17,912][105620] Updated weights for policy 1, policy_version 569071 (0.0009) [2023-12-26 19:30:17,976][105620] Updated weights for policy 1, policy_version 569081 (0.0009) [2023-12-26 19:30:18,038][105620] Updated weights for policy 1, policy_version 569091 (0.0009) [2023-12-26 19:30:18,219][105692] Updated weights for policy 0, policy_version 568360 (0.0006) [2023-12-26 19:30:18,276][105692] Updated weights for policy 0, policy_version 568370 (0.0010) [2023-12-26 19:30:18,327][105692] Updated weights for policy 0, policy_version 568380 (0.0010) [2023-12-26 19:30:18,842][105620] Updated weights for policy 1, policy_version 569101 (0.0009) [2023-12-26 19:30:18,890][105620] Updated weights for policy 1, policy_version 569111 (0.0008) [2023-12-26 19:30:18,954][105620] Updated weights for policy 1, policy_version 569121 (0.0008) [2023-12-26 19:30:19,083][105692] Updated weights for policy 0, policy_version 568390 (0.0010) [2023-12-26 19:30:19,135][105692] Updated weights for policy 0, policy_version 568400 (0.0010) [2023-12-26 19:30:19,182][105692] Updated weights for policy 0, policy_version 568410 (0.0010) [2023-12-26 19:30:19,673][105620] Updated weights for policy 1, policy_version 569131 (0.0007) [2023-12-26 19:30:19,742][105620] Updated weights for policy 1, policy_version 569141 (0.0006) [2023-12-26 19:30:19,811][105620] Updated weights for policy 1, policy_version 569151 (0.0006) [2023-12-26 19:30:19,902][105692] Updated weights for policy 0, policy_version 568420 (0.0009) [2023-12-26 19:30:19,965][105692] Updated weights for policy 0, policy_version 568430 (0.0009) [2023-12-26 19:30:20,022][105692] Updated weights for policy 0, policy_version 568440 (0.0009) [2023-12-26 19:30:20,419][105620] Updated weights for policy 1, policy_version 569161 (0.0007) [2023-12-26 19:30:20,475][105620] Updated weights for policy 1, policy_version 569171 (0.0011) [2023-12-26 19:30:20,535][105620] Updated weights for policy 1, policy_version 569181 (0.0010) [2023-12-26 19:30:20,604][105620] Updated weights for policy 1, policy_version 569191 (0.0010) [2023-12-26 19:30:20,775][105692] Updated weights for policy 0, policy_version 568451 (0.0009) [2023-12-26 19:30:20,827][105692] Updated weights for policy 0, policy_version 568461 (0.0009) [2023-12-26 19:30:20,887][105692] Updated weights for policy 0, policy_version 568471 (0.0009) [2023-12-26 19:30:21,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 291274752. Throughput: 0: 9914.5, 1: 9842.1. Samples: 291262344. Policy #0 lag: (min: 54.0, avg: 56.0, max: 56.0) [2023-12-26 19:30:21,063][104569] Avg episode reward: [(0, '9354.992'), (1, '9081.434')] [2023-12-26 19:30:21,335][105620] Updated weights for policy 1, policy_version 569201 (0.0009) [2023-12-26 19:30:21,402][105620] Updated weights for policy 1, policy_version 569211 (0.0009) [2023-12-26 19:30:21,465][105620] Updated weights for policy 1, policy_version 569221 (0.0010) [2023-12-26 19:30:21,607][105692] Updated weights for policy 0, policy_version 568481 (0.0008) [2023-12-26 19:30:21,666][105692] Updated weights for policy 0, policy_version 568491 (0.0007) [2023-12-26 19:30:21,723][105692] Updated weights for policy 0, policy_version 568501 (0.0009) [2023-12-26 19:30:21,792][105692] Updated weights for policy 0, policy_version 568511 (0.0009) [2023-12-26 19:30:22,271][105620] Updated weights for policy 1, policy_version 569231 (0.0010) [2023-12-26 19:30:22,340][105586] KL-divergence is very high: 114.8526 [2023-12-26 19:30:22,342][105620] Updated weights for policy 1, policy_version 569241 (0.0010) [2023-12-26 19:30:22,392][105586] KL-divergence is very high: 115.3177 [2023-12-26 19:30:22,406][105620] Updated weights for policy 1, policy_version 569251 (0.0010) [2023-12-26 19:30:22,601][105692] Updated weights for policy 0, policy_version 568521 (0.0008) [2023-12-26 19:30:22,666][105692] Updated weights for policy 0, policy_version 568531 (0.0009) [2023-12-26 19:30:22,727][105692] Updated weights for policy 0, policy_version 568541 (0.0009) [2023-12-26 19:30:23,177][105620] Updated weights for policy 1, policy_version 569261 (0.0010) [2023-12-26 19:30:23,226][105620] Updated weights for policy 1, policy_version 569271 (0.0010) [2023-12-26 19:30:23,284][105620] Updated weights for policy 1, policy_version 569281 (0.0010) [2023-12-26 19:30:23,490][105692] Updated weights for policy 0, policy_version 568551 (0.0008) [2023-12-26 19:30:23,534][105692] Updated weights for policy 0, policy_version 568561 (0.0008) [2023-12-26 19:30:23,590][105692] Updated weights for policy 0, policy_version 568571 (0.0008) [2023-12-26 19:30:24,043][105620] Updated weights for policy 1, policy_version 569291 (0.0010) [2023-12-26 19:30:24,091][105620] Updated weights for policy 1, policy_version 569301 (0.0010) [2023-12-26 19:30:24,148][105620] Updated weights for policy 1, policy_version 569311 (0.0006) [2023-12-26 19:30:24,374][105692] Updated weights for policy 0, policy_version 568581 (0.0009) [2023-12-26 19:30:24,419][105692] Updated weights for policy 0, policy_version 568591 (0.0008) [2023-12-26 19:30:24,468][105692] Updated weights for policy 0, policy_version 568601 (0.0008) [2023-12-26 19:30:24,733][105620] Updated weights for policy 1, policy_version 569321 (0.0006) [2023-12-26 19:30:24,783][105620] Updated weights for policy 1, policy_version 569331 (0.0007) [2023-12-26 19:30:24,834][105620] Updated weights for policy 1, policy_version 569341 (0.0006) [2023-12-26 19:30:24,891][105620] Updated weights for policy 1, policy_version 569351 (0.0010) [2023-12-26 19:30:25,302][105692] Updated weights for policy 0, policy_version 568611 (0.0009) [2023-12-26 19:30:25,359][105692] Updated weights for policy 0, policy_version 568621 (0.0009) [2023-12-26 19:30:25,417][105692] Updated weights for policy 0, policy_version 568632 (0.0010) [2023-12-26 19:30:25,520][105620] Updated weights for policy 1, policy_version 569361 (0.0006) [2023-12-26 19:30:25,574][105620] Updated weights for policy 1, policy_version 569371 (0.0005) [2023-12-26 19:30:25,617][105620] Updated weights for policy 1, policy_version 569381 (0.0005) [2023-12-26 19:30:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 291364864. Throughput: 0: 9784.0, 1: 9962.0. Samples: 291375960. Policy #0 lag: (min: 54.0, avg: 56.0, max: 56.0) [2023-12-26 19:30:26,062][104569] Avg episode reward: [(0, '8915.029'), (1, '9171.259')] [2023-12-26 19:30:26,181][105620] Updated weights for policy 1, policy_version 569391 (0.0008) [2023-12-26 19:30:26,229][105620] Updated weights for policy 1, policy_version 569401 (0.0008) [2023-12-26 19:30:26,278][105692] Updated weights for policy 0, policy_version 568642 (0.0009) [2023-12-26 19:30:26,281][105620] Updated weights for policy 1, policy_version 569411 (0.0008) [2023-12-26 19:30:26,334][105692] Updated weights for policy 0, policy_version 568652 (0.0008) [2023-12-26 19:30:26,392][105692] Updated weights for policy 0, policy_version 568662 (0.0009) [2023-12-26 19:30:26,450][105692] Updated weights for policy 0, policy_version 568672 (0.0009) [2023-12-26 19:30:26,924][105620] Updated weights for policy 1, policy_version 569421 (0.0005) [2023-12-26 19:30:26,975][105620] Updated weights for policy 1, policy_version 569431 (0.0005) [2023-12-26 19:30:27,029][105620] Updated weights for policy 1, policy_version 569441 (0.0005) [2023-12-26 19:30:27,126][105692] Updated weights for policy 0, policy_version 568682 (0.0005) [2023-12-26 19:30:27,182][105692] Updated weights for policy 0, policy_version 568692 (0.0005) [2023-12-26 19:30:27,229][105692] Updated weights for policy 0, policy_version 568702 (0.0005) [2023-12-26 19:30:27,716][105620] Updated weights for policy 1, policy_version 569451 (0.0006) [2023-12-26 19:30:27,759][105620] Updated weights for policy 1, policy_version 569461 (0.0007) [2023-12-26 19:30:27,811][105620] Updated weights for policy 1, policy_version 569471 (0.0008) [2023-12-26 19:30:27,869][105692] Updated weights for policy 0, policy_version 568712 (0.0010) [2023-12-26 19:30:27,937][105692] Updated weights for policy 0, policy_version 568722 (0.0008) [2023-12-26 19:30:27,992][105692] Updated weights for policy 0, policy_version 568732 (0.0008) [2023-12-26 19:30:28,622][105620] Updated weights for policy 1, policy_version 569481 (0.0007) [2023-12-26 19:30:28,635][105692] Updated weights for policy 0, policy_version 568742 (0.0007) [2023-12-26 19:30:28,671][105620] Updated weights for policy 1, policy_version 569491 (0.0009) [2023-12-26 19:30:28,693][105692] Updated weights for policy 0, policy_version 568752 (0.0007) [2023-12-26 19:30:28,723][105620] Updated weights for policy 1, policy_version 569501 (0.0007) [2023-12-26 19:30:28,757][105692] Updated weights for policy 0, policy_version 568762 (0.0007) [2023-12-26 19:30:28,778][105620] Updated weights for policy 1, policy_version 569511 (0.0007) [2023-12-26 19:30:29,407][105692] Updated weights for policy 0, policy_version 568772 (0.0010) [2023-12-26 19:30:29,463][105692] Updated weights for policy 0, policy_version 568782 (0.0009) [2023-12-26 19:30:29,488][105585] KL-divergence is very high: 184.6288 [2023-12-26 19:30:29,522][105692] Updated weights for policy 0, policy_version 568792 (0.0009) [2023-12-26 19:30:29,536][105585] KL-divergence is very high: 275.9031 [2023-12-26 19:30:29,592][105620] Updated weights for policy 1, policy_version 569521 (0.0009) [2023-12-26 19:30:29,639][105620] Updated weights for policy 1, policy_version 569531 (0.0008) [2023-12-26 19:30:29,696][105620] Updated weights for policy 1, policy_version 569541 (0.0009) [2023-12-26 19:30:30,202][105692] Updated weights for policy 0, policy_version 568802 (0.0007) [2023-12-26 19:30:30,261][105692] Updated weights for policy 0, policy_version 568812 (0.0009) [2023-12-26 19:30:30,310][105692] Updated weights for policy 0, policy_version 568822 (0.0009) [2023-12-26 19:30:30,373][105692] Updated weights for policy 0, policy_version 568832 (0.0009) [2023-12-26 19:30:30,514][105620] Updated weights for policy 1, policy_version 569551 (0.0009) [2023-12-26 19:30:30,567][105620] Updated weights for policy 1, policy_version 569561 (0.0008) [2023-12-26 19:30:30,618][105620] Updated weights for policy 1, policy_version 569571 (0.0009) [2023-12-26 19:30:31,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 291463168. Throughput: 0: 9783.7, 1: 10014.7. Samples: 291436348. Policy #0 lag: (min: 54.0, avg: 56.0, max: 56.0) [2023-12-26 19:30:31,062][104569] Avg episode reward: [(0, '7942.227'), (1, '9079.124')] [2023-12-26 19:30:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000569576_145825792.pth... [2023-12-26 19:30:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000568456_145539072.pth [2023-12-26 19:30:31,093][105692] Updated weights for policy 0, policy_version 568842 (0.0009) [2023-12-26 19:30:31,157][105692] Updated weights for policy 0, policy_version 568852 (0.0009) [2023-12-26 19:30:31,213][105692] Updated weights for policy 0, policy_version 568862 (0.0009) [2023-12-26 19:30:31,223][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000568864_145645568.pth... [2023-12-26 19:30:31,227][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000567712_145350656.pth [2023-12-26 19:30:31,402][105620] Updated weights for policy 1, policy_version 569581 (0.0009) [2023-12-26 19:30:31,468][105620] Updated weights for policy 1, policy_version 569591 (0.0010) [2023-12-26 19:30:31,529][105620] Updated weights for policy 1, policy_version 569601 (0.0010) [2023-12-26 19:30:31,986][105692] Updated weights for policy 0, policy_version 568872 (0.0007) [2023-12-26 19:30:32,039][105692] Updated weights for policy 0, policy_version 568882 (0.0010) [2023-12-26 19:30:32,104][105692] Updated weights for policy 0, policy_version 568892 (0.0009) [2023-12-26 19:30:32,212][105620] Updated weights for policy 1, policy_version 569611 (0.0009) [2023-12-26 19:30:32,277][105620] Updated weights for policy 1, policy_version 569621 (0.0007) [2023-12-26 19:30:32,331][105620] Updated weights for policy 1, policy_version 569631 (0.0010) [2023-12-26 19:30:32,879][105692] Updated weights for policy 0, policy_version 568902 (0.0008) [2023-12-26 19:30:32,930][105692] Updated weights for policy 0, policy_version 568912 (0.0006) [2023-12-26 19:30:32,960][105620] Updated weights for policy 1, policy_version 569641 (0.0009) [2023-12-26 19:30:32,984][105692] Updated weights for policy 0, policy_version 568922 (0.0006) [2023-12-26 19:30:33,016][105620] Updated weights for policy 1, policy_version 569651 (0.0006) [2023-12-26 19:30:33,071][105620] Updated weights for policy 1, policy_version 569661 (0.0005) [2023-12-26 19:30:33,130][105620] Updated weights for policy 1, policy_version 569671 (0.0006) [2023-12-26 19:30:33,676][105692] Updated weights for policy 0, policy_version 568932 (0.0005) [2023-12-26 19:30:33,727][105692] Updated weights for policy 0, policy_version 568942 (0.0005) [2023-12-26 19:30:33,750][105620] Updated weights for policy 1, policy_version 569681 (0.0010) [2023-12-26 19:30:33,776][105692] Updated weights for policy 0, policy_version 568952 (0.0005) [2023-12-26 19:30:33,811][105620] Updated weights for policy 1, policy_version 569691 (0.0010) [2023-12-26 19:30:33,865][105620] Updated weights for policy 1, policy_version 569701 (0.0010) [2023-12-26 19:30:34,435][105692] Updated weights for policy 0, policy_version 568962 (0.0007) [2023-12-26 19:30:34,494][105692] Updated weights for policy 0, policy_version 568972 (0.0010) [2023-12-26 19:30:34,535][105620] Updated weights for policy 1, policy_version 569711 (0.0010) [2023-12-26 19:30:34,554][105692] Updated weights for policy 0, policy_version 568982 (0.0011) [2023-12-26 19:30:34,598][105620] Updated weights for policy 1, policy_version 569721 (0.0011) [2023-12-26 19:30:34,613][105692] Updated weights for policy 0, policy_version 568992 (0.0011) [2023-12-26 19:30:34,661][105620] Updated weights for policy 1, policy_version 569731 (0.0008) [2023-12-26 19:30:35,257][105692] Updated weights for policy 0, policy_version 569002 (0.0006) [2023-12-26 19:30:35,302][105620] Updated weights for policy 1, policy_version 569741 (0.0009) [2023-12-26 19:30:35,308][105692] Updated weights for policy 0, policy_version 569012 (0.0005) [2023-12-26 19:30:35,356][105692] Updated weights for policy 0, policy_version 569022 (0.0006) [2023-12-26 19:30:35,356][105620] Updated weights for policy 1, policy_version 569751 (0.0010) [2023-12-26 19:30:35,408][105620] Updated weights for policy 1, policy_version 569761 (0.0010) [2023-12-26 19:30:36,038][105692] Updated weights for policy 0, policy_version 569032 (0.0010) [2023-12-26 19:30:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 291561472. Throughput: 0: 9724.3, 1: 9975.1. Samples: 291554600. Policy #0 lag: (min: 54.0, avg: 56.0, max: 56.0) [2023-12-26 19:30:36,062][104569] Avg episode reward: [(0, '8647.625'), (1, '9083.061')] [2023-12-26 19:30:36,085][105692] Updated weights for policy 0, policy_version 569042 (0.0010) [2023-12-26 19:30:36,158][105620] Updated weights for policy 1, policy_version 569771 (0.0011) [2023-12-26 19:30:36,165][105692] Updated weights for policy 0, policy_version 569052 (0.0008) [2023-12-26 19:30:36,211][105620] Updated weights for policy 1, policy_version 569781 (0.0010) [2023-12-26 19:30:36,267][105620] Updated weights for policy 1, policy_version 569791 (0.0010) [2023-12-26 19:30:36,774][105692] Updated weights for policy 0, policy_version 569062 (0.0010) [2023-12-26 19:30:36,823][105692] Updated weights for policy 0, policy_version 569072 (0.0010) [2023-12-26 19:30:36,875][105692] Updated weights for policy 0, policy_version 569082 (0.0010) [2023-12-26 19:30:37,028][105620] Updated weights for policy 1, policy_version 569801 (0.0011) [2023-12-26 19:30:37,083][105620] Updated weights for policy 1, policy_version 569811 (0.0010) [2023-12-26 19:30:37,145][105620] Updated weights for policy 1, policy_version 569821 (0.0010) [2023-12-26 19:30:37,203][105620] Updated weights for policy 1, policy_version 569831 (0.0010) [2023-12-26 19:30:37,655][105692] Updated weights for policy 0, policy_version 569092 (0.0010) [2023-12-26 19:30:37,714][105692] Updated weights for policy 0, policy_version 569102 (0.0011) [2023-12-26 19:30:37,764][105692] Updated weights for policy 0, policy_version 569112 (0.0010) [2023-12-26 19:30:37,937][105620] Updated weights for policy 1, policy_version 569841 (0.0010) [2023-12-26 19:30:37,989][105620] Updated weights for policy 1, policy_version 569851 (0.0010) [2023-12-26 19:30:38,048][105620] Updated weights for policy 1, policy_version 569861 (0.0010) [2023-12-26 19:30:38,492][105692] Updated weights for policy 0, policy_version 569122 (0.0011) [2023-12-26 19:30:38,548][105692] Updated weights for policy 0, policy_version 569132 (0.0010) [2023-12-26 19:30:38,604][105692] Updated weights for policy 0, policy_version 569142 (0.0011) [2023-12-26 19:30:38,660][105692] Updated weights for policy 0, policy_version 569152 (0.0011) [2023-12-26 19:30:38,815][105620] Updated weights for policy 1, policy_version 569871 (0.0011) [2023-12-26 19:30:38,877][105620] Updated weights for policy 1, policy_version 569881 (0.0011) [2023-12-26 19:30:38,930][105620] Updated weights for policy 1, policy_version 569891 (0.0011) [2023-12-26 19:30:39,424][105692] Updated weights for policy 0, policy_version 569162 (0.0008) [2023-12-26 19:30:39,481][105692] Updated weights for policy 0, policy_version 569172 (0.0006) [2023-12-26 19:30:39,546][105692] Updated weights for policy 0, policy_version 569182 (0.0007) [2023-12-26 19:30:39,645][105620] Updated weights for policy 1, policy_version 569901 (0.0008) [2023-12-26 19:30:39,711][105620] Updated weights for policy 1, policy_version 569911 (0.0005) [2023-12-26 19:30:39,777][105620] Updated weights for policy 1, policy_version 569921 (0.0005) [2023-12-26 19:30:40,264][105692] Updated weights for policy 0, policy_version 569192 (0.0008) [2023-12-26 19:30:40,321][105692] Updated weights for policy 0, policy_version 569202 (0.0009) [2023-12-26 19:30:40,381][105692] Updated weights for policy 0, policy_version 569212 (0.0008) [2023-12-26 19:30:40,452][105620] Updated weights for policy 1, policy_version 569931 (0.0008) [2023-12-26 19:30:40,511][105620] Updated weights for policy 1, policy_version 569941 (0.0009) [2023-12-26 19:30:40,573][105620] Updated weights for policy 1, policy_version 569951 (0.0011) [2023-12-26 19:30:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.4, 300 sec: 19522.0). Total num frames: 291659776. Throughput: 0: 9789.4, 1: 9836.2. Samples: 291670612. Policy #0 lag: (min: 54.0, avg: 56.0, max: 56.0) [2023-12-26 19:30:41,063][104569] Avg episode reward: [(0, '9086.168'), (1, '9175.616')] [2023-12-26 19:30:41,236][105692] Updated weights for policy 0, policy_version 569222 (0.0009) [2023-12-26 19:30:41,299][105620] Updated weights for policy 1, policy_version 569961 (0.0010) [2023-12-26 19:30:41,308][105692] Updated weights for policy 0, policy_version 569232 (0.0007) [2023-12-26 19:30:41,359][105620] Updated weights for policy 1, policy_version 569971 (0.0010) [2023-12-26 19:30:41,381][105692] Updated weights for policy 0, policy_version 569242 (0.0008) [2023-12-26 19:30:41,429][105620] Updated weights for policy 1, policy_version 569981 (0.0009) [2023-12-26 19:30:41,495][105620] Updated weights for policy 1, policy_version 569991 (0.0011) [2023-12-26 19:30:42,213][105692] Updated weights for policy 0, policy_version 569252 (0.0009) [2023-12-26 19:30:42,281][105692] Updated weights for policy 0, policy_version 569262 (0.0009) [2023-12-26 19:30:42,296][105620] Updated weights for policy 1, policy_version 570001 (0.0008) [2023-12-26 19:30:42,340][105692] Updated weights for policy 0, policy_version 569272 (0.0007) [2023-12-26 19:30:42,361][105620] Updated weights for policy 1, policy_version 570011 (0.0008) [2023-12-26 19:30:42,417][105620] Updated weights for policy 1, policy_version 570021 (0.0008) [2023-12-26 19:30:43,109][105620] Updated weights for policy 1, policy_version 570031 (0.0008) [2023-12-26 19:30:43,115][105692] Updated weights for policy 0, policy_version 569282 (0.0009) [2023-12-26 19:30:43,162][105620] Updated weights for policy 1, policy_version 570041 (0.0006) [2023-12-26 19:30:43,169][105692] Updated weights for policy 0, policy_version 569292 (0.0007) [2023-12-26 19:30:43,215][105620] Updated weights for policy 1, policy_version 570051 (0.0007) [2023-12-26 19:30:43,217][105692] Updated weights for policy 0, policy_version 569302 (0.0006) [2023-12-26 19:30:43,276][105692] Updated weights for policy 0, policy_version 569312 (0.0009) [2023-12-26 19:30:43,903][105620] Updated weights for policy 1, policy_version 570061 (0.0008) [2023-12-26 19:30:43,969][105620] Updated weights for policy 1, policy_version 570071 (0.0011) [2023-12-26 19:30:44,031][105620] Updated weights for policy 1, policy_version 570081 (0.0006) [2023-12-26 19:30:44,045][105692] Updated weights for policy 0, policy_version 569322 (0.0010) [2023-12-26 19:30:44,104][105692] Updated weights for policy 0, policy_version 569332 (0.0008) [2023-12-26 19:30:44,167][105692] Updated weights for policy 0, policy_version 569342 (0.0009) [2023-12-26 19:30:44,686][105620] Updated weights for policy 1, policy_version 570091 (0.0007) [2023-12-26 19:30:44,738][105620] Updated weights for policy 1, policy_version 570101 (0.0009) [2023-12-26 19:30:44,800][105620] Updated weights for policy 1, policy_version 570111 (0.0009) [2023-12-26 19:30:44,904][105692] Updated weights for policy 0, policy_version 569352 (0.0008) [2023-12-26 19:30:44,964][105692] Updated weights for policy 0, policy_version 569362 (0.0009) [2023-12-26 19:30:45,024][105692] Updated weights for policy 0, policy_version 569372 (0.0009) [2023-12-26 19:30:45,520][105620] Updated weights for policy 1, policy_version 570121 (0.0007) [2023-12-26 19:30:45,577][105620] Updated weights for policy 1, policy_version 570131 (0.0009) [2023-12-26 19:30:45,627][105620] Updated weights for policy 1, policy_version 570141 (0.0010) [2023-12-26 19:30:45,677][105620] Updated weights for policy 1, policy_version 570151 (0.0010) [2023-12-26 19:30:45,832][105692] Updated weights for policy 0, policy_version 569382 (0.0009) [2023-12-26 19:30:45,893][105692] Updated weights for policy 0, policy_version 569392 (0.0009) [2023-12-26 19:30:45,944][105692] Updated weights for policy 0, policy_version 569402 (0.0009) [2023-12-26 19:30:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 291758080. Throughput: 0: 9689.6, 1: 9766.8. Samples: 291725472. Policy #0 lag: (min: 54.0, avg: 56.0, max: 56.0) [2023-12-26 19:30:46,063][104569] Avg episode reward: [(0, '9264.400'), (1, '9266.760')] [2023-12-26 19:30:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000569408_145784832.pth... [2023-12-26 19:30:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000570152_145973248.pth... [2023-12-26 19:30:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000568288_145498112.pth [2023-12-26 19:30:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000569000_145678336.pth [2023-12-26 19:30:46,405][105620] Updated weights for policy 1, policy_version 570161 (0.0009) [2023-12-26 19:30:46,465][105620] Updated weights for policy 1, policy_version 570172 (0.0008) [2023-12-26 19:30:46,518][105620] Updated weights for policy 1, policy_version 570182 (0.0005) [2023-12-26 19:30:46,712][105692] Updated weights for policy 0, policy_version 569412 (0.0007) [2023-12-26 19:30:46,769][105692] Updated weights for policy 0, policy_version 569422 (0.0005) [2023-12-26 19:30:46,831][105692] Updated weights for policy 0, policy_version 569432 (0.0005) [2023-12-26 19:30:47,175][105620] Updated weights for policy 1, policy_version 570192 (0.0008) [2023-12-26 19:30:47,236][105620] Updated weights for policy 1, policy_version 570202 (0.0009) [2023-12-26 19:30:47,305][105620] Updated weights for policy 1, policy_version 570212 (0.0007) [2023-12-26 19:30:47,457][105692] Updated weights for policy 0, policy_version 569442 (0.0006) [2023-12-26 19:30:47,512][105692] Updated weights for policy 0, policy_version 569452 (0.0008) [2023-12-26 19:30:47,565][105692] Updated weights for policy 0, policy_version 569462 (0.0008) [2023-12-26 19:30:47,619][105692] Updated weights for policy 0, policy_version 569472 (0.0009) [2023-12-26 19:30:47,953][105620] Updated weights for policy 1, policy_version 570222 (0.0010) [2023-12-26 19:30:48,026][105620] Updated weights for policy 1, policy_version 570232 (0.0008) [2023-12-26 19:30:48,082][105620] Updated weights for policy 1, policy_version 570242 (0.0009) [2023-12-26 19:30:48,483][105692] Updated weights for policy 0, policy_version 569482 (0.0006) [2023-12-26 19:30:48,549][105692] Updated weights for policy 0, policy_version 569492 (0.0006) [2023-12-26 19:30:48,614][105692] Updated weights for policy 0, policy_version 569502 (0.0006) [2023-12-26 19:30:48,894][105620] Updated weights for policy 1, policy_version 570252 (0.0009) [2023-12-26 19:30:48,957][105620] Updated weights for policy 1, policy_version 570262 (0.0009) [2023-12-26 19:30:49,019][105620] Updated weights for policy 1, policy_version 570272 (0.0009) [2023-12-26 19:30:49,254][105692] Updated weights for policy 0, policy_version 569512 (0.0007) [2023-12-26 19:30:49,325][105692] Updated weights for policy 0, policy_version 569522 (0.0006) [2023-12-26 19:30:49,392][105692] Updated weights for policy 0, policy_version 569532 (0.0009) [2023-12-26 19:30:49,794][105620] Updated weights for policy 1, policy_version 570282 (0.0009) [2023-12-26 19:30:49,859][105620] Updated weights for policy 1, policy_version 570292 (0.0009) [2023-12-26 19:30:49,913][105620] Updated weights for policy 1, policy_version 570302 (0.0009) [2023-12-26 19:30:49,980][105620] Updated weights for policy 1, policy_version 570312 (0.0009) [2023-12-26 19:30:50,104][105692] Updated weights for policy 0, policy_version 569542 (0.0009) [2023-12-26 19:30:50,170][105692] Updated weights for policy 0, policy_version 569552 (0.0009) [2023-12-26 19:30:50,233][105692] Updated weights for policy 0, policy_version 569562 (0.0009) [2023-12-26 19:30:50,766][105620] Updated weights for policy 1, policy_version 570322 (0.0009) [2023-12-26 19:30:50,818][105620] Updated weights for policy 1, policy_version 570332 (0.0009) [2023-12-26 19:30:50,865][105620] Updated weights for policy 1, policy_version 570342 (0.0008) [2023-12-26 19:30:51,011][105692] Updated weights for policy 0, policy_version 569572 (0.0008) [2023-12-26 19:30:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 291848192. Throughput: 0: 9606.1, 1: 9737.8. Samples: 291839288. Policy #0 lag: (min: 54.0, avg: 56.0, max: 56.0) [2023-12-26 19:30:51,063][104569] Avg episode reward: [(0, '8994.320'), (1, '9353.059')] [2023-12-26 19:30:51,081][105692] Updated weights for policy 0, policy_version 569582 (0.0010) [2023-12-26 19:30:51,136][105692] Updated weights for policy 0, policy_version 569592 (0.0009) [2023-12-26 19:30:51,652][105620] Updated weights for policy 1, policy_version 570352 (0.0009) [2023-12-26 19:30:51,715][105620] Updated weights for policy 1, policy_version 570362 (0.0008) [2023-12-26 19:30:51,781][105620] Updated weights for policy 1, policy_version 570372 (0.0008) [2023-12-26 19:30:51,942][105692] Updated weights for policy 0, policy_version 569602 (0.0009) [2023-12-26 19:30:52,002][105692] Updated weights for policy 0, policy_version 569612 (0.0008) [2023-12-26 19:30:52,065][105692] Updated weights for policy 0, policy_version 569622 (0.0008) [2023-12-26 19:30:52,129][105692] Updated weights for policy 0, policy_version 569632 (0.0008) [2023-12-26 19:30:52,518][105620] Updated weights for policy 1, policy_version 570382 (0.0008) [2023-12-26 19:30:52,572][105620] Updated weights for policy 1, policy_version 570392 (0.0009) [2023-12-26 19:30:52,633][105620] Updated weights for policy 1, policy_version 570402 (0.0009) [2023-12-26 19:30:52,882][105692] Updated weights for policy 0, policy_version 569642 (0.0008) [2023-12-26 19:30:52,937][105692] Updated weights for policy 0, policy_version 569652 (0.0006) [2023-12-26 19:30:53,001][105692] Updated weights for policy 0, policy_version 569662 (0.0007) [2023-12-26 19:30:53,452][105620] Updated weights for policy 1, policy_version 570412 (0.0009) [2023-12-26 19:30:53,518][105620] Updated weights for policy 1, policy_version 570422 (0.0011) [2023-12-26 19:30:53,587][105620] Updated weights for policy 1, policy_version 570432 (0.0011) [2023-12-26 19:30:53,621][105692] Updated weights for policy 0, policy_version 569672 (0.0007) [2023-12-26 19:30:53,683][105692] Updated weights for policy 0, policy_version 569682 (0.0010) [2023-12-26 19:30:53,738][105692] Updated weights for policy 0, policy_version 569692 (0.0010) [2023-12-26 19:30:54,240][105620] Updated weights for policy 1, policy_version 570442 (0.0010) [2023-12-26 19:30:54,287][105620] Updated weights for policy 1, policy_version 570452 (0.0005) [2023-12-26 19:30:54,348][105620] Updated weights for policy 1, policy_version 570462 (0.0007) [2023-12-26 19:30:54,417][105620] Updated weights for policy 1, policy_version 570472 (0.0006) [2023-12-26 19:30:54,465][105692] Updated weights for policy 0, policy_version 569702 (0.0010) [2023-12-26 19:30:54,524][105692] Updated weights for policy 0, policy_version 569712 (0.0010) [2023-12-26 19:30:54,572][105692] Updated weights for policy 0, policy_version 569722 (0.0010) [2023-12-26 19:30:55,082][105620] Updated weights for policy 1, policy_version 570482 (0.0008) [2023-12-26 19:30:55,137][105620] Updated weights for policy 1, policy_version 570492 (0.0008) [2023-12-26 19:30:55,182][105620] Updated weights for policy 1, policy_version 570502 (0.0008) [2023-12-26 19:30:55,323][105692] Updated weights for policy 0, policy_version 569732 (0.0010) [2023-12-26 19:30:55,383][105692] Updated weights for policy 0, policy_version 569742 (0.0010) [2023-12-26 19:30:55,448][105692] Updated weights for policy 0, policy_version 569752 (0.0009) [2023-12-26 19:30:55,909][105620] Updated weights for policy 1, policy_version 570512 (0.0010) [2023-12-26 19:30:55,976][105620] Updated weights for policy 1, policy_version 570522 (0.0007) [2023-12-26 19:30:56,031][105692] Updated weights for policy 0, policy_version 569762 (0.0009) [2023-12-26 19:30:56,035][105620] Updated weights for policy 1, policy_version 570532 (0.0007) [2023-12-26 19:30:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 291946496. Throughput: 0: 9628.5, 1: 9699.0. Samples: 291953284. Policy #0 lag: (min: 54.0, avg: 56.0, max: 56.0) [2023-12-26 19:30:56,062][104569] Avg episode reward: [(0, '8907.799'), (1, '9170.731')] [2023-12-26 19:30:56,099][105692] Updated weights for policy 0, policy_version 569772 (0.0005) [2023-12-26 19:30:56,159][105692] Updated weights for policy 0, policy_version 569782 (0.0006) [2023-12-26 19:30:56,229][105692] Updated weights for policy 0, policy_version 569792 (0.0005) [2023-12-26 19:30:56,572][105620] Updated weights for policy 1, policy_version 570542 (0.0005) [2023-12-26 19:30:56,635][105620] Updated weights for policy 1, policy_version 570552 (0.0005) [2023-12-26 19:30:56,698][105620] Updated weights for policy 1, policy_version 570562 (0.0005) [2023-12-26 19:30:56,738][105692] Updated weights for policy 0, policy_version 569802 (0.0005) [2023-12-26 19:30:56,797][105692] Updated weights for policy 0, policy_version 569812 (0.0005) [2023-12-26 19:30:56,847][105692] Updated weights for policy 0, policy_version 569822 (0.0005) [2023-12-26 19:30:57,350][105620] Updated weights for policy 1, policy_version 570572 (0.0009) [2023-12-26 19:30:57,381][105692] Updated weights for policy 0, policy_version 569832 (0.0009) [2023-12-26 19:30:57,405][105620] Updated weights for policy 1, policy_version 570582 (0.0007) [2023-12-26 19:30:57,438][105692] Updated weights for policy 0, policy_version 569842 (0.0010) [2023-12-26 19:30:57,463][105620] Updated weights for policy 1, policy_version 570592 (0.0005) [2023-12-26 19:30:57,497][105692] Updated weights for policy 0, policy_version 569852 (0.0010) [2023-12-26 19:30:58,047][105620] Updated weights for policy 1, policy_version 570602 (0.0005) [2023-12-26 19:30:58,104][105620] Updated weights for policy 1, policy_version 570612 (0.0006) [2023-12-26 19:30:58,146][105692] Updated weights for policy 0, policy_version 569862 (0.0009) [2023-12-26 19:30:58,174][105620] Updated weights for policy 1, policy_version 570622 (0.0008) [2023-12-26 19:30:58,213][105692] Updated weights for policy 0, policy_version 569872 (0.0011) [2023-12-26 19:30:58,245][105620] Updated weights for policy 1, policy_version 570632 (0.0008) [2023-12-26 19:30:58,278][105692] Updated weights for policy 0, policy_version 569882 (0.0010) [2023-12-26 19:30:58,956][105620] Updated weights for policy 1, policy_version 570642 (0.0008) [2023-12-26 19:30:59,015][105692] Updated weights for policy 0, policy_version 569892 (0.0006) [2023-12-26 19:30:59,016][105620] Updated weights for policy 1, policy_version 570652 (0.0010) [2023-12-26 19:30:59,063][105620] Updated weights for policy 1, policy_version 570662 (0.0009) [2023-12-26 19:30:59,064][105692] Updated weights for policy 0, policy_version 569902 (0.0005) [2023-12-26 19:30:59,111][105692] Updated weights for policy 0, policy_version 569912 (0.0005) [2023-12-26 19:30:59,755][105620] Updated weights for policy 1, policy_version 570672 (0.0006) [2023-12-26 19:30:59,766][105692] Updated weights for policy 0, policy_version 569922 (0.0005) [2023-12-26 19:30:59,807][105620] Updated weights for policy 1, policy_version 570682 (0.0008) [2023-12-26 19:30:59,826][105692] Updated weights for policy 0, policy_version 569932 (0.0006) [2023-12-26 19:30:59,872][105620] Updated weights for policy 1, policy_version 570692 (0.0010) [2023-12-26 19:30:59,890][105692] Updated weights for policy 0, policy_version 569942 (0.0009) [2023-12-26 19:30:59,952][105692] Updated weights for policy 0, policy_version 569952 (0.0008) [2023-12-26 19:31:00,503][105620] Updated weights for policy 1, policy_version 570702 (0.0007) [2023-12-26 19:31:00,552][105620] Updated weights for policy 1, policy_version 570712 (0.0005) [2023-12-26 19:31:00,597][105620] Updated weights for policy 1, policy_version 570722 (0.0005) [2023-12-26 19:31:00,721][105692] Updated weights for policy 0, policy_version 569962 (0.0008) [2023-12-26 19:31:00,783][105692] Updated weights for policy 0, policy_version 569972 (0.0007) [2023-12-26 19:31:00,855][105692] Updated weights for policy 0, policy_version 569982 (0.0005) [2023-12-26 19:31:01,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 292052992. Throughput: 0: 9773.5, 1: 9761.2. Samples: 292019776. Policy #0 lag: (min: 54.0, avg: 56.0, max: 56.0) [2023-12-26 19:31:01,062][104569] Avg episode reward: [(0, '8914.687'), (1, '9172.175')] [2023-12-26 19:31:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000569984_145932288.pth... [2023-12-26 19:31:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000570728_146120704.pth... [2023-12-26 19:31:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000569576_145825792.pth [2023-12-26 19:31:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000568864_145645568.pth [2023-12-26 19:31:01,147][105620] Updated weights for policy 1, policy_version 570732 (0.0007) [2023-12-26 19:31:01,205][105620] Updated weights for policy 1, policy_version 570742 (0.0006) [2023-12-26 19:31:01,274][105620] Updated weights for policy 1, policy_version 570752 (0.0007) [2023-12-26 19:31:01,417][105692] Updated weights for policy 0, policy_version 569992 (0.0006) [2023-12-26 19:31:01,474][105692] Updated weights for policy 0, policy_version 570002 (0.0005) [2023-12-26 19:31:01,538][105692] Updated weights for policy 0, policy_version 570012 (0.0005) [2023-12-26 19:31:01,999][105620] Updated weights for policy 1, policy_version 570762 (0.0008) [2023-12-26 19:31:02,065][105620] Updated weights for policy 1, policy_version 570772 (0.0006) [2023-12-26 19:31:02,114][105620] Updated weights for policy 1, policy_version 570782 (0.0009) [2023-12-26 19:31:02,167][105620] Updated weights for policy 1, policy_version 570792 (0.0009) [2023-12-26 19:31:02,181][105692] Updated weights for policy 0, policy_version 570022 (0.0006) [2023-12-26 19:31:02,235][105692] Updated weights for policy 0, policy_version 570032 (0.0009) [2023-12-26 19:31:02,296][105692] Updated weights for policy 0, policy_version 570042 (0.0008) [2023-12-26 19:31:02,797][105620] Updated weights for policy 1, policy_version 570802 (0.0005) [2023-12-26 19:31:02,851][105620] Updated weights for policy 1, policy_version 570812 (0.0005) [2023-12-26 19:31:02,902][105620] Updated weights for policy 1, policy_version 570822 (0.0006) [2023-12-26 19:31:03,106][105692] Updated weights for policy 0, policy_version 570052 (0.0007) [2023-12-26 19:31:03,166][105692] Updated weights for policy 0, policy_version 570062 (0.0010) [2023-12-26 19:31:03,220][105692] Updated weights for policy 0, policy_version 570072 (0.0010) [2023-12-26 19:31:03,443][105620] Updated weights for policy 1, policy_version 570832 (0.0005) [2023-12-26 19:31:03,496][105620] Updated weights for policy 1, policy_version 570842 (0.0006) [2023-12-26 19:31:03,553][105620] Updated weights for policy 1, policy_version 570852 (0.0006) [2023-12-26 19:31:04,094][105692] Updated weights for policy 0, policy_version 570082 (0.0009) [2023-12-26 19:31:04,156][105692] Updated weights for policy 0, policy_version 570092 (0.0009) [2023-12-26 19:31:04,197][105620] Updated weights for policy 1, policy_version 570862 (0.0006) [2023-12-26 19:31:04,207][105692] Updated weights for policy 0, policy_version 570102 (0.0007) [2023-12-26 19:31:04,258][105620] Updated weights for policy 1, policy_version 570872 (0.0007) [2023-12-26 19:31:04,263][105692] Updated weights for policy 0, policy_version 570112 (0.0006) [2023-12-26 19:31:04,321][105620] Updated weights for policy 1, policy_version 570882 (0.0009) [2023-12-26 19:31:05,005][105692] Updated weights for policy 0, policy_version 570122 (0.0008) [2023-12-26 19:31:05,060][105692] Updated weights for policy 0, policy_version 570132 (0.0009) [2023-12-26 19:31:05,064][105620] Updated weights for policy 1, policy_version 570892 (0.0008) [2023-12-26 19:31:05,113][105692] Updated weights for policy 0, policy_version 570142 (0.0010) [2023-12-26 19:31:05,115][105620] Updated weights for policy 1, policy_version 570902 (0.0005) [2023-12-26 19:31:05,166][105620] Updated weights for policy 1, policy_version 570912 (0.0005) [2023-12-26 19:31:05,797][105692] Updated weights for policy 0, policy_version 570152 (0.0006) [2023-12-26 19:31:05,846][105620] Updated weights for policy 1, policy_version 570922 (0.0006) [2023-12-26 19:31:05,848][105692] Updated weights for policy 0, policy_version 570162 (0.0005) [2023-12-26 19:31:05,894][105692] Updated weights for policy 0, policy_version 570172 (0.0005) [2023-12-26 19:31:05,904][105620] Updated weights for policy 1, policy_version 570932 (0.0005) [2023-12-26 19:31:05,954][105620] Updated weights for policy 1, policy_version 570942 (0.0005) [2023-12-26 19:31:06,008][105620] Updated weights for policy 1, policy_version 570952 (0.0005) [2023-12-26 19:31:06,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 292159488. Throughput: 0: 9623.9, 1: 9910.8. Samples: 292141404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:31:06,063][104569] Avg episode reward: [(0, '8906.235'), (1, '9172.281')] [2023-12-26 19:31:06,587][105692] Updated weights for policy 0, policy_version 570182 (0.0008) [2023-12-26 19:31:06,643][105692] Updated weights for policy 0, policy_version 570192 (0.0011) [2023-12-26 19:31:06,672][105620] Updated weights for policy 1, policy_version 570962 (0.0011) [2023-12-26 19:31:06,698][105692] Updated weights for policy 0, policy_version 570202 (0.0010) [2023-12-26 19:31:06,734][105620] Updated weights for policy 1, policy_version 570972 (0.0010) [2023-12-26 19:31:06,793][105620] Updated weights for policy 1, policy_version 570982 (0.0008) [2023-12-26 19:31:07,456][105692] Updated weights for policy 0, policy_version 570212 (0.0011) [2023-12-26 19:31:07,505][105620] Updated weights for policy 1, policy_version 570992 (0.0006) [2023-12-26 19:31:07,505][105692] Updated weights for policy 0, policy_version 570222 (0.0010) [2023-12-26 19:31:07,558][105620] Updated weights for policy 1, policy_version 571002 (0.0005) [2023-12-26 19:31:07,564][105692] Updated weights for policy 0, policy_version 570232 (0.0010) [2023-12-26 19:31:07,615][105620] Updated weights for policy 1, policy_version 571012 (0.0005) [2023-12-26 19:31:08,193][105620] Updated weights for policy 1, policy_version 571022 (0.0006) [2023-12-26 19:31:08,245][105692] Updated weights for policy 0, policy_version 570242 (0.0009) [2023-12-26 19:31:08,256][105620] Updated weights for policy 1, policy_version 571032 (0.0007) [2023-12-26 19:31:08,313][105692] Updated weights for policy 0, policy_version 570252 (0.0007) [2023-12-26 19:31:08,324][105620] Updated weights for policy 1, policy_version 571042 (0.0008) [2023-12-26 19:31:08,379][105692] Updated weights for policy 0, policy_version 570262 (0.0007) [2023-12-26 19:31:08,447][105692] Updated weights for policy 0, policy_version 570272 (0.0010) [2023-12-26 19:31:08,868][105620] Updated weights for policy 1, policy_version 571052 (0.0008) [2023-12-26 19:31:08,922][105620] Updated weights for policy 1, policy_version 571062 (0.0008) [2023-12-26 19:31:08,971][105620] Updated weights for policy 1, policy_version 571072 (0.0010) [2023-12-26 19:31:09,236][105692] Updated weights for policy 0, policy_version 570282 (0.0010) [2023-12-26 19:31:09,299][105692] Updated weights for policy 0, policy_version 570292 (0.0010) [2023-12-26 19:31:09,363][105692] Updated weights for policy 0, policy_version 570302 (0.0010) [2023-12-26 19:31:09,756][105620] Updated weights for policy 1, policy_version 571082 (0.0007) [2023-12-26 19:31:09,830][105620] Updated weights for policy 1, policy_version 571092 (0.0007) [2023-12-26 19:31:09,889][105620] Updated weights for policy 1, policy_version 571102 (0.0008) [2023-12-26 19:31:09,960][105620] Updated weights for policy 1, policy_version 571112 (0.0009) [2023-12-26 19:31:10,160][105692] Updated weights for policy 0, policy_version 570312 (0.0011) [2023-12-26 19:31:10,226][105692] Updated weights for policy 0, policy_version 570322 (0.0011) [2023-12-26 19:31:10,282][105692] Updated weights for policy 0, policy_version 570332 (0.0011) [2023-12-26 19:31:10,742][105620] Updated weights for policy 1, policy_version 571122 (0.0008) [2023-12-26 19:31:10,811][105620] Updated weights for policy 1, policy_version 571132 (0.0009) [2023-12-26 19:31:10,881][105620] Updated weights for policy 1, policy_version 571142 (0.0010) [2023-12-26 19:31:10,913][105692] Updated weights for policy 0, policy_version 570342 (0.0008) [2023-12-26 19:31:10,971][105692] Updated weights for policy 0, policy_version 570352 (0.0007) [2023-12-26 19:31:11,032][105692] Updated weights for policy 0, policy_version 570362 (0.0006) [2023-12-26 19:31:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 292249600. Throughput: 0: 9700.0, 1: 9931.8. Samples: 292259392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:31:11,062][104569] Avg episode reward: [(0, '8900.354'), (1, '9081.146')] [2023-12-26 19:31:11,715][105692] Updated weights for policy 0, policy_version 570372 (0.0009) [2023-12-26 19:31:11,753][105620] Updated weights for policy 1, policy_version 571152 (0.0007) [2023-12-26 19:31:11,783][105692] Updated weights for policy 0, policy_version 570382 (0.0010) [2023-12-26 19:31:11,802][105620] Updated weights for policy 1, policy_version 571162 (0.0008) [2023-12-26 19:31:11,849][105620] Updated weights for policy 1, policy_version 571172 (0.0008) [2023-12-26 19:31:11,849][105692] Updated weights for policy 0, policy_version 570392 (0.0009) [2023-12-26 19:31:12,629][105692] Updated weights for policy 0, policy_version 570402 (0.0009) [2023-12-26 19:31:12,661][105620] Updated weights for policy 1, policy_version 571182 (0.0008) [2023-12-26 19:31:12,687][105692] Updated weights for policy 0, policy_version 570412 (0.0007) [2023-12-26 19:31:12,715][105620] Updated weights for policy 1, policy_version 571192 (0.0006) [2023-12-26 19:31:12,750][105692] Updated weights for policy 0, policy_version 570422 (0.0006) [2023-12-26 19:31:12,779][105620] Updated weights for policy 1, policy_version 571202 (0.0007) [2023-12-26 19:31:12,812][105692] Updated weights for policy 0, policy_version 570432 (0.0008) [2023-12-26 19:31:13,352][105620] Updated weights for policy 1, policy_version 571212 (0.0006) [2023-12-26 19:31:13,410][105620] Updated weights for policy 1, policy_version 571222 (0.0007) [2023-12-26 19:31:13,461][105620] Updated weights for policy 1, policy_version 571232 (0.0007) [2023-12-26 19:31:13,487][105692] Updated weights for policy 0, policy_version 570442 (0.0011) [2023-12-26 19:31:13,532][105692] Updated weights for policy 0, policy_version 570452 (0.0010) [2023-12-26 19:31:13,582][105692] Updated weights for policy 0, policy_version 570462 (0.0010) [2023-12-26 19:31:14,047][105620] Updated weights for policy 1, policy_version 571242 (0.0006) [2023-12-26 19:31:14,101][105620] Updated weights for policy 1, policy_version 571252 (0.0010) [2023-12-26 19:31:14,162][105620] Updated weights for policy 1, policy_version 571262 (0.0008) [2023-12-26 19:31:14,220][105620] Updated weights for policy 1, policy_version 571272 (0.0009) [2023-12-26 19:31:14,265][105692] Updated weights for policy 0, policy_version 570472 (0.0006) [2023-12-26 19:31:14,331][105692] Updated weights for policy 0, policy_version 570482 (0.0010) [2023-12-26 19:31:14,394][105692] Updated weights for policy 0, policy_version 570492 (0.0011) [2023-12-26 19:31:15,035][105620] Updated weights for policy 1, policy_version 571282 (0.0007) [2023-12-26 19:31:15,094][105620] Updated weights for policy 1, policy_version 571292 (0.0009) [2023-12-26 19:31:15,120][105692] Updated weights for policy 0, policy_version 570502 (0.0010) [2023-12-26 19:31:15,157][105620] Updated weights for policy 1, policy_version 571302 (0.0006) [2023-12-26 19:31:15,182][105692] Updated weights for policy 0, policy_version 570512 (0.0007) [2023-12-26 19:31:15,249][105692] Updated weights for policy 0, policy_version 570522 (0.0006) [2023-12-26 19:31:15,815][105620] Updated weights for policy 1, policy_version 571312 (0.0008) [2023-12-26 19:31:15,845][105692] Updated weights for policy 0, policy_version 570532 (0.0006) [2023-12-26 19:31:15,863][105620] Updated weights for policy 1, policy_version 571322 (0.0008) [2023-12-26 19:31:15,890][105692] Updated weights for policy 0, policy_version 570542 (0.0006) [2023-12-26 19:31:15,909][105620] Updated weights for policy 1, policy_version 571332 (0.0006) [2023-12-26 19:31:15,949][105692] Updated weights for policy 0, policy_version 570552 (0.0007) [2023-12-26 19:31:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 292356096. Throughput: 0: 9674.3, 1: 9935.0. Samples: 292318768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:31:16,062][104569] Avg episode reward: [(0, '8736.791'), (1, '9263.825')] [2023-12-26 19:31:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000571336_146276352.pth... [2023-12-26 19:31:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000570560_146079744.pth... [2023-12-26 19:31:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000570152_145973248.pth [2023-12-26 19:31:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000569408_145784832.pth [2023-12-26 19:31:16,533][105692] Updated weights for policy 0, policy_version 570563 (0.0006) [2023-12-26 19:31:16,578][105692] Updated weights for policy 0, policy_version 570573 (0.0005) [2023-12-26 19:31:16,638][105692] Updated weights for policy 0, policy_version 570583 (0.0005) [2023-12-26 19:31:16,682][105620] Updated weights for policy 1, policy_version 571342 (0.0006) [2023-12-26 19:31:16,736][105620] Updated weights for policy 1, policy_version 571352 (0.0005) [2023-12-26 19:31:16,794][105620] Updated weights for policy 1, policy_version 571362 (0.0005) [2023-12-26 19:31:17,163][105692] Updated weights for policy 0, policy_version 570593 (0.0006) [2023-12-26 19:31:17,207][105692] Updated weights for policy 0, policy_version 570603 (0.0010) [2023-12-26 19:31:17,252][105692] Updated weights for policy 0, policy_version 570613 (0.0010) [2023-12-26 19:31:17,303][105692] Updated weights for policy 0, policy_version 570623 (0.0010) [2023-12-26 19:31:17,424][105620] Updated weights for policy 1, policy_version 571372 (0.0006) [2023-12-26 19:31:17,475][105620] Updated weights for policy 1, policy_version 571382 (0.0008) [2023-12-26 19:31:17,533][105620] Updated weights for policy 1, policy_version 571392 (0.0005) [2023-12-26 19:31:18,067][105692] Updated weights for policy 0, policy_version 570633 (0.0010) [2023-12-26 19:31:18,122][105692] Updated weights for policy 0, policy_version 570643 (0.0010) [2023-12-26 19:31:18,174][105692] Updated weights for policy 0, policy_version 570653 (0.0010) [2023-12-26 19:31:18,209][105620] Updated weights for policy 1, policy_version 571402 (0.0007) [2023-12-26 19:31:18,270][105620] Updated weights for policy 1, policy_version 571412 (0.0007) [2023-12-26 19:31:18,330][105620] Updated weights for policy 1, policy_version 571422 (0.0006) [2023-12-26 19:31:18,396][105620] Updated weights for policy 1, policy_version 571432 (0.0007) [2023-12-26 19:31:18,885][105692] Updated weights for policy 0, policy_version 570663 (0.0010) [2023-12-26 19:31:18,937][105692] Updated weights for policy 0, policy_version 570673 (0.0010) [2023-12-26 19:31:18,995][105692] Updated weights for policy 0, policy_version 570683 (0.0010) [2023-12-26 19:31:19,127][105620] Updated weights for policy 1, policy_version 571442 (0.0008) [2023-12-26 19:31:19,213][105620] Updated weights for policy 1, policy_version 571452 (0.0006) [2023-12-26 19:31:19,284][105620] Updated weights for policy 1, policy_version 571462 (0.0008) [2023-12-26 19:31:19,816][105692] Updated weights for policy 0, policy_version 570693 (0.0009) [2023-12-26 19:31:19,880][105692] Updated weights for policy 0, policy_version 570703 (0.0008) [2023-12-26 19:31:19,940][105692] Updated weights for policy 0, policy_version 570713 (0.0008) [2023-12-26 19:31:19,991][105620] Updated weights for policy 1, policy_version 571472 (0.0010) [2023-12-26 19:31:20,044][105620] Updated weights for policy 1, policy_version 571482 (0.0011) [2023-12-26 19:31:20,111][105620] Updated weights for policy 1, policy_version 571492 (0.0011) [2023-12-26 19:31:20,699][105692] Updated weights for policy 0, policy_version 570723 (0.0008) [2023-12-26 19:31:20,768][105692] Updated weights for policy 0, policy_version 570733 (0.0007) [2023-12-26 19:31:20,825][105692] Updated weights for policy 0, policy_version 570743 (0.0007) [2023-12-26 19:31:20,882][105620] Updated weights for policy 1, policy_version 571502 (0.0009) [2023-12-26 19:31:20,943][105620] Updated weights for policy 1, policy_version 571512 (0.0009) [2023-12-26 19:31:21,000][105620] Updated weights for policy 1, policy_version 571522 (0.0010) [2023-12-26 19:31:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 292454400. Throughput: 0: 9757.0, 1: 9915.1. Samples: 292439844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:31:21,062][104569] Avg episode reward: [(0, '8652.792'), (1, '9356.940')] [2023-12-26 19:31:21,547][105692] Updated weights for policy 0, policy_version 570753 (0.0007) [2023-12-26 19:31:21,609][105692] Updated weights for policy 0, policy_version 570763 (0.0008) [2023-12-26 19:31:21,676][105692] Updated weights for policy 0, policy_version 570773 (0.0008) [2023-12-26 19:31:21,745][105692] Updated weights for policy 0, policy_version 570783 (0.0008) [2023-12-26 19:31:21,774][105620] Updated weights for policy 1, policy_version 571532 (0.0009) [2023-12-26 19:31:21,838][105620] Updated weights for policy 1, policy_version 571542 (0.0008) [2023-12-26 19:31:21,899][105620] Updated weights for policy 1, policy_version 571552 (0.0009) [2023-12-26 19:31:22,558][105692] Updated weights for policy 0, policy_version 570793 (0.0010) [2023-12-26 19:31:22,621][105692] Updated weights for policy 0, policy_version 570803 (0.0010) [2023-12-26 19:31:22,659][105620] Updated weights for policy 1, policy_version 571562 (0.0009) [2023-12-26 19:31:22,673][105692] Updated weights for policy 0, policy_version 570813 (0.0008) [2023-12-26 19:31:22,720][105620] Updated weights for policy 1, policy_version 571572 (0.0009) [2023-12-26 19:31:22,788][105620] Updated weights for policy 1, policy_version 571582 (0.0010) [2023-12-26 19:31:22,844][105620] Updated weights for policy 1, policy_version 571592 (0.0010) [2023-12-26 19:31:23,342][105692] Updated weights for policy 0, policy_version 570823 (0.0009) [2023-12-26 19:31:23,393][105692] Updated weights for policy 0, policy_version 570833 (0.0010) [2023-12-26 19:31:23,444][105692] Updated weights for policy 0, policy_version 570843 (0.0009) [2023-12-26 19:31:23,514][105620] Updated weights for policy 1, policy_version 571602 (0.0009) [2023-12-26 19:31:23,571][105620] Updated weights for policy 1, policy_version 571612 (0.0007) [2023-12-26 19:31:23,629][105620] Updated weights for policy 1, policy_version 571622 (0.0005) [2023-12-26 19:31:24,070][105692] Updated weights for policy 0, policy_version 570853 (0.0007) [2023-12-26 19:31:24,127][105692] Updated weights for policy 0, policy_version 570863 (0.0008) [2023-12-26 19:31:24,187][105692] Updated weights for policy 0, policy_version 570873 (0.0007) [2023-12-26 19:31:24,382][105620] Updated weights for policy 1, policy_version 571632 (0.0008) [2023-12-26 19:31:24,441][105620] Updated weights for policy 1, policy_version 571642 (0.0009) [2023-12-26 19:31:24,493][105620] Updated weights for policy 1, policy_version 571652 (0.0005) [2023-12-26 19:31:24,803][105692] Updated weights for policy 0, policy_version 570883 (0.0010) [2023-12-26 19:31:24,854][105692] Updated weights for policy 0, policy_version 570893 (0.0010) [2023-12-26 19:31:24,898][105692] Updated weights for policy 0, policy_version 570903 (0.0010) [2023-12-26 19:31:25,275][105620] Updated weights for policy 1, policy_version 571662 (0.0007) [2023-12-26 19:31:25,327][105620] Updated weights for policy 1, policy_version 571672 (0.0007) [2023-12-26 19:31:25,379][105620] Updated weights for policy 1, policy_version 571682 (0.0008) [2023-12-26 19:31:25,654][105692] Updated weights for policy 0, policy_version 570913 (0.0007) [2023-12-26 19:31:25,718][105692] Updated weights for policy 0, policy_version 570923 (0.0009) [2023-12-26 19:31:25,778][105692] Updated weights for policy 0, policy_version 570933 (0.0010) [2023-12-26 19:31:25,831][105692] Updated weights for policy 0, policy_version 570943 (0.0010) [2023-12-26 19:31:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 292544512. Throughput: 0: 9765.3, 1: 9892.1. Samples: 292555196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:31:26,062][104569] Avg episode reward: [(0, '8833.860'), (1, '9356.551')] [2023-12-26 19:31:26,081][105620] Updated weights for policy 1, policy_version 571692 (0.0009) [2023-12-26 19:31:26,140][105620] Updated weights for policy 1, policy_version 571702 (0.0006) [2023-12-26 19:31:26,202][105620] Updated weights for policy 1, policy_version 571712 (0.0005) [2023-12-26 19:31:26,484][105692] Updated weights for policy 0, policy_version 570953 (0.0006) [2023-12-26 19:31:26,535][105692] Updated weights for policy 0, policy_version 570963 (0.0006) [2023-12-26 19:31:26,593][105692] Updated weights for policy 0, policy_version 570973 (0.0006) [2023-12-26 19:31:26,778][105620] Updated weights for policy 1, policy_version 571722 (0.0005) [2023-12-26 19:31:26,841][105620] Updated weights for policy 1, policy_version 571732 (0.0005) [2023-12-26 19:31:26,893][105620] Updated weights for policy 1, policy_version 571742 (0.0005) [2023-12-26 19:31:26,941][105620] Updated weights for policy 1, policy_version 571752 (0.0005) [2023-12-26 19:31:27,262][105692] Updated weights for policy 0, policy_version 570983 (0.0009) [2023-12-26 19:31:27,320][105692] Updated weights for policy 0, policy_version 570993 (0.0010) [2023-12-26 19:31:27,385][105692] Updated weights for policy 0, policy_version 571003 (0.0010) [2023-12-26 19:31:27,472][105620] Updated weights for policy 1, policy_version 571762 (0.0007) [2023-12-26 19:31:27,529][105620] Updated weights for policy 1, policy_version 571772 (0.0007) [2023-12-26 19:31:27,579][105620] Updated weights for policy 1, policy_version 571782 (0.0005) [2023-12-26 19:31:27,976][105692] Updated weights for policy 0, policy_version 571013 (0.0008) [2023-12-26 19:31:28,031][105692] Updated weights for policy 0, policy_version 571023 (0.0009) [2023-12-26 19:31:28,085][105692] Updated weights for policy 0, policy_version 571033 (0.0010) [2023-12-26 19:31:28,178][105620] Updated weights for policy 1, policy_version 571792 (0.0005) [2023-12-26 19:31:28,241][105620] Updated weights for policy 1, policy_version 571802 (0.0005) [2023-12-26 19:31:28,291][105620] Updated weights for policy 1, policy_version 571812 (0.0006) [2023-12-26 19:31:28,745][105692] Updated weights for policy 0, policy_version 571043 (0.0007) [2023-12-26 19:31:28,806][105692] Updated weights for policy 0, policy_version 571053 (0.0005) [2023-12-26 19:31:28,864][105692] Updated weights for policy 0, policy_version 571063 (0.0005) [2023-12-26 19:31:29,009][105620] Updated weights for policy 1, policy_version 571822 (0.0009) [2023-12-26 19:31:29,072][105620] Updated weights for policy 1, policy_version 571832 (0.0009) [2023-12-26 19:31:29,127][105620] Updated weights for policy 1, policy_version 571842 (0.0008) [2023-12-26 19:31:29,539][105692] Updated weights for policy 0, policy_version 571073 (0.0008) [2023-12-26 19:31:29,594][105692] Updated weights for policy 0, policy_version 571083 (0.0011) [2023-12-26 19:31:29,653][105692] Updated weights for policy 0, policy_version 571093 (0.0011) [2023-12-26 19:31:29,705][105692] Updated weights for policy 0, policy_version 571103 (0.0011) [2023-12-26 19:31:29,812][105620] Updated weights for policy 1, policy_version 571852 (0.0007) [2023-12-26 19:31:29,871][105620] Updated weights for policy 1, policy_version 571862 (0.0008) [2023-12-26 19:31:29,924][105620] Updated weights for policy 1, policy_version 571872 (0.0008) [2023-12-26 19:31:30,458][105692] Updated weights for policy 0, policy_version 571113 (0.0008) [2023-12-26 19:31:30,516][105692] Updated weights for policy 0, policy_version 571123 (0.0006) [2023-12-26 19:31:30,561][105692] Updated weights for policy 0, policy_version 571133 (0.0006) [2023-12-26 19:31:30,680][105620] Updated weights for policy 1, policy_version 571882 (0.0007) [2023-12-26 19:31:30,748][105620] Updated weights for policy 1, policy_version 571892 (0.0007) [2023-12-26 19:31:30,814][105620] Updated weights for policy 1, policy_version 571902 (0.0006) [2023-12-26 19:31:30,873][105620] Updated weights for policy 1, policy_version 571912 (0.0007) [2023-12-26 19:31:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 292651008. Throughput: 0: 9893.9, 1: 9990.7. Samples: 292620276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:31:31,062][104569] Avg episode reward: [(0, '6004.971'), (1, '9356.009')] [2023-12-26 19:31:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000571136_146227200.pth... [2023-12-26 19:31:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000571912_146423808.pth... [2023-12-26 19:31:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000569984_145932288.pth [2023-12-26 19:31:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000570728_146120704.pth [2023-12-26 19:31:31,156][105692] Updated weights for policy 0, policy_version 571143 (0.0007) [2023-12-26 19:31:31,222][105692] Updated weights for policy 0, policy_version 571153 (0.0008) [2023-12-26 19:31:31,281][105692] Updated weights for policy 0, policy_version 571163 (0.0008) [2023-12-26 19:31:31,548][105620] Updated weights for policy 1, policy_version 571923 (0.0008) [2023-12-26 19:31:31,614][105620] Updated weights for policy 1, policy_version 571933 (0.0008) [2023-12-26 19:31:31,679][105620] Updated weights for policy 1, policy_version 571943 (0.0010) [2023-12-26 19:31:31,967][105692] Updated weights for policy 0, policy_version 571173 (0.0008) [2023-12-26 19:31:32,029][105692] Updated weights for policy 0, policy_version 571183 (0.0008) [2023-12-26 19:31:32,087][105692] Updated weights for policy 0, policy_version 571193 (0.0008) [2023-12-26 19:31:32,399][105620] Updated weights for policy 1, policy_version 571953 (0.0010) [2023-12-26 19:31:32,461][105620] Updated weights for policy 1, policy_version 571963 (0.0011) [2023-12-26 19:31:32,523][105620] Updated weights for policy 1, policy_version 571973 (0.0007) [2023-12-26 19:31:32,829][105692] Updated weights for policy 0, policy_version 571203 (0.0008) [2023-12-26 19:31:32,879][105692] Updated weights for policy 0, policy_version 571213 (0.0009) [2023-12-26 19:31:32,933][105692] Updated weights for policy 0, policy_version 571223 (0.0009) [2023-12-26 19:31:33,173][105620] Updated weights for policy 1, policy_version 571983 (0.0006) [2023-12-26 19:31:33,227][105620] Updated weights for policy 1, policy_version 571993 (0.0005) [2023-12-26 19:31:33,295][105620] Updated weights for policy 1, policy_version 572003 (0.0005) [2023-12-26 19:31:33,742][105692] Updated weights for policy 0, policy_version 571233 (0.0009) [2023-12-26 19:31:33,800][105692] Updated weights for policy 0, policy_version 571243 (0.0010) [2023-12-26 19:31:33,856][105692] Updated weights for policy 0, policy_version 571253 (0.0009) [2023-12-26 19:31:33,876][105620] Updated weights for policy 1, policy_version 572013 (0.0005) [2023-12-26 19:31:33,908][105692] Updated weights for policy 0, policy_version 571263 (0.0009) [2023-12-26 19:31:33,927][105620] Updated weights for policy 1, policy_version 572023 (0.0005) [2023-12-26 19:31:33,992][105620] Updated weights for policy 1, policy_version 572033 (0.0005) [2023-12-26 19:31:34,596][105620] Updated weights for policy 1, policy_version 572043 (0.0005) [2023-12-26 19:31:34,662][105620] Updated weights for policy 1, policy_version 572053 (0.0006) [2023-12-26 19:31:34,720][105620] Updated weights for policy 1, policy_version 572063 (0.0007) [2023-12-26 19:31:34,813][105692] Updated weights for policy 0, policy_version 571273 (0.0009) [2023-12-26 19:31:34,878][105692] Updated weights for policy 0, policy_version 571283 (0.0009) [2023-12-26 19:31:34,931][105692] Updated weights for policy 0, policy_version 571293 (0.0009) [2023-12-26 19:31:35,334][105620] Updated weights for policy 1, policy_version 572073 (0.0006) [2023-12-26 19:31:35,388][105620] Updated weights for policy 1, policy_version 572083 (0.0006) [2023-12-26 19:31:35,436][105620] Updated weights for policy 1, policy_version 572093 (0.0009) [2023-12-26 19:31:35,486][105620] Updated weights for policy 1, policy_version 572103 (0.0008) [2023-12-26 19:31:35,664][105692] Updated weights for policy 0, policy_version 571303 (0.0009) [2023-12-26 19:31:35,712][105692] Updated weights for policy 0, policy_version 571313 (0.0008) [2023-12-26 19:31:35,767][105692] Updated weights for policy 0, policy_version 571323 (0.0008) [2023-12-26 19:31:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 292749312. Throughput: 0: 9931.3, 1: 10071.7. Samples: 292739420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:31:36,062][104569] Avg episode reward: [(0, '6996.788'), (1, '9264.468')] [2023-12-26 19:31:36,174][105620] Updated weights for policy 1, policy_version 572113 (0.0007) [2023-12-26 19:31:36,236][105620] Updated weights for policy 1, policy_version 572123 (0.0011) [2023-12-26 19:31:36,300][105620] Updated weights for policy 1, policy_version 572133 (0.0009) [2023-12-26 19:31:36,634][105692] Updated weights for policy 0, policy_version 571333 (0.0008) [2023-12-26 19:31:36,686][105692] Updated weights for policy 0, policy_version 571344 (0.0010) [2023-12-26 19:31:36,733][105692] Updated weights for policy 0, policy_version 571354 (0.0007) [2023-12-26 19:31:36,881][105620] Updated weights for policy 1, policy_version 572143 (0.0008) [2023-12-26 19:31:36,943][105620] Updated weights for policy 1, policy_version 572153 (0.0009) [2023-12-26 19:31:37,004][105620] Updated weights for policy 1, policy_version 572163 (0.0008) [2023-12-26 19:31:37,543][105692] Updated weights for policy 0, policy_version 571364 (0.0009) [2023-12-26 19:31:37,602][105692] Updated weights for policy 0, policy_version 571374 (0.0009) [2023-12-26 19:31:37,667][105692] Updated weights for policy 0, policy_version 571384 (0.0010) [2023-12-26 19:31:37,705][105620] Updated weights for policy 1, policy_version 572173 (0.0008) [2023-12-26 19:31:37,775][105620] Updated weights for policy 1, policy_version 572183 (0.0009) [2023-12-26 19:31:37,837][105620] Updated weights for policy 1, policy_version 572193 (0.0005) [2023-12-26 19:31:38,403][105620] Updated weights for policy 1, policy_version 572203 (0.0007) [2023-12-26 19:31:38,462][105620] Updated weights for policy 1, policy_version 572213 (0.0009) [2023-12-26 19:31:38,493][105692] Updated weights for policy 0, policy_version 571394 (0.0009) [2023-12-26 19:31:38,516][105620] Updated weights for policy 1, policy_version 572223 (0.0007) [2023-12-26 19:31:38,542][105692] Updated weights for policy 0, policy_version 571404 (0.0006) [2023-12-26 19:31:38,596][105692] Updated weights for policy 0, policy_version 571414 (0.0007) [2023-12-26 19:31:38,655][105692] Updated weights for policy 0, policy_version 571424 (0.0009) [2023-12-26 19:31:39,313][105692] Updated weights for policy 0, policy_version 571434 (0.0009) [2023-12-26 19:31:39,339][105620] Updated weights for policy 1, policy_version 572233 (0.0009) [2023-12-26 19:31:39,381][105692] Updated weights for policy 0, policy_version 571444 (0.0009) [2023-12-26 19:31:39,404][105620] Updated weights for policy 1, policy_version 572243 (0.0006) [2023-12-26 19:31:39,444][105692] Updated weights for policy 0, policy_version 571454 (0.0006) [2023-12-26 19:31:39,467][105620] Updated weights for policy 1, policy_version 572253 (0.0009) [2023-12-26 19:31:39,528][105620] Updated weights for policy 1, policy_version 572263 (0.0008) [2023-12-26 19:31:40,213][105692] Updated weights for policy 0, policy_version 571464 (0.0009) [2023-12-26 19:31:40,270][105692] Updated weights for policy 0, policy_version 571474 (0.0009) [2023-12-26 19:31:40,285][105620] Updated weights for policy 1, policy_version 572273 (0.0007) [2023-12-26 19:31:40,322][105692] Updated weights for policy 0, policy_version 571484 (0.0007) [2023-12-26 19:31:40,332][105620] Updated weights for policy 1, policy_version 572283 (0.0008) [2023-12-26 19:31:40,383][105620] Updated weights for policy 1, policy_version 572293 (0.0009) [2023-12-26 19:31:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 292839424. Throughput: 0: 9868.6, 1: 10130.0. Samples: 292853220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:31:41,062][104569] Avg episode reward: [(0, '8267.899'), (1, '9264.305')] [2023-12-26 19:31:41,111][105692] Updated weights for policy 0, policy_version 571494 (0.0008) [2023-12-26 19:31:41,176][105692] Updated weights for policy 0, policy_version 571504 (0.0008) [2023-12-26 19:31:41,203][105620] Updated weights for policy 1, policy_version 572303 (0.0008) [2023-12-26 19:31:41,242][105692] Updated weights for policy 0, policy_version 571514 (0.0010) [2023-12-26 19:31:41,265][105620] Updated weights for policy 1, policy_version 572313 (0.0007) [2023-12-26 19:31:41,331][105620] Updated weights for policy 1, policy_version 572323 (0.0008) [2023-12-26 19:31:42,026][105692] Updated weights for policy 0, policy_version 571524 (0.0007) [2023-12-26 19:31:42,084][105692] Updated weights for policy 0, policy_version 571534 (0.0009) [2023-12-26 19:31:42,105][105620] Updated weights for policy 1, policy_version 572333 (0.0008) [2023-12-26 19:31:42,136][105692] Updated weights for policy 0, policy_version 571545 (0.0008) [2023-12-26 19:31:42,175][105620] Updated weights for policy 1, policy_version 572343 (0.0005) [2023-12-26 19:31:42,244][105620] Updated weights for policy 1, policy_version 572353 (0.0007) [2023-12-26 19:31:42,872][105692] Updated weights for policy 0, policy_version 571555 (0.0008) [2023-12-26 19:31:42,944][105692] Updated weights for policy 0, policy_version 571565 (0.0006) [2023-12-26 19:31:42,945][105620] Updated weights for policy 1, policy_version 572363 (0.0009) [2023-12-26 19:31:43,007][105620] Updated weights for policy 1, policy_version 572373 (0.0008) [2023-12-26 19:31:43,011][105692] Updated weights for policy 0, policy_version 571575 (0.0007) [2023-12-26 19:31:43,070][105620] Updated weights for policy 1, policy_version 572383 (0.0007) [2023-12-26 19:31:43,711][105620] Updated weights for policy 1, policy_version 572393 (0.0005) [2023-12-26 19:31:43,739][105692] Updated weights for policy 0, policy_version 571585 (0.0010) [2023-12-26 19:31:43,764][105620] Updated weights for policy 1, policy_version 572403 (0.0007) [2023-12-26 19:31:43,801][105692] Updated weights for policy 0, policy_version 571595 (0.0006) [2023-12-26 19:31:43,814][105620] Updated weights for policy 1, policy_version 572413 (0.0007) [2023-12-26 19:31:43,858][105692] Updated weights for policy 0, policy_version 571605 (0.0009) [2023-12-26 19:31:43,869][105620] Updated weights for policy 1, policy_version 572423 (0.0006) [2023-12-26 19:31:43,907][105692] Updated weights for policy 0, policy_version 571615 (0.0008) [2023-12-26 19:31:44,547][105620] Updated weights for policy 1, policy_version 572433 (0.0007) [2023-12-26 19:31:44,592][105620] Updated weights for policy 1, policy_version 572443 (0.0008) [2023-12-26 19:31:44,654][105620] Updated weights for policy 1, policy_version 572453 (0.0008) [2023-12-26 19:31:44,704][105692] Updated weights for policy 0, policy_version 571625 (0.0008) [2023-12-26 19:31:44,751][105692] Updated weights for policy 0, policy_version 571635 (0.0009) [2023-12-26 19:31:44,814][105692] Updated weights for policy 0, policy_version 571645 (0.0008) [2023-12-26 19:31:45,384][105620] Updated weights for policy 1, policy_version 572463 (0.0005) [2023-12-26 19:31:45,450][105620] Updated weights for policy 1, policy_version 572473 (0.0005) [2023-12-26 19:31:45,510][105620] Updated weights for policy 1, policy_version 572483 (0.0005) [2023-12-26 19:31:45,638][105692] Updated weights for policy 0, policy_version 571655 (0.0009) [2023-12-26 19:31:45,691][105692] Updated weights for policy 0, policy_version 571667 (0.0010) [2023-12-26 19:31:45,745][105692] Updated weights for policy 0, policy_version 571678 (0.0010) [2023-12-26 19:31:46,043][105620] Updated weights for policy 1, policy_version 572493 (0.0006) [2023-12-26 19:31:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 292937728. Throughput: 0: 9720.3, 1: 10054.7. Samples: 292909648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:31:46,062][104569] Avg episode reward: [(0, '8542.654'), (1, '9354.314')] [2023-12-26 19:31:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000571680_146366464.pth... [2023-12-26 19:31:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000570560_146079744.pth [2023-12-26 19:31:46,111][105620] Updated weights for policy 1, policy_version 572503 (0.0006) [2023-12-26 19:31:46,184][105620] Updated weights for policy 1, policy_version 572513 (0.0006) [2023-12-26 19:31:46,232][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000572520_146579456.pth... [2023-12-26 19:31:46,237][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000571336_146276352.pth [2023-12-26 19:31:46,625][105692] Updated weights for policy 0, policy_version 571688 (0.0010) [2023-12-26 19:31:46,679][105692] Updated weights for policy 0, policy_version 571698 (0.0009) [2023-12-26 19:31:46,734][105692] Updated weights for policy 0, policy_version 571708 (0.0009) [2023-12-26 19:31:46,777][105620] Updated weights for policy 1, policy_version 572523 (0.0006) [2023-12-26 19:31:46,842][105620] Updated weights for policy 1, policy_version 572533 (0.0006) [2023-12-26 19:31:46,905][105620] Updated weights for policy 1, policy_version 572543 (0.0005) [2023-12-26 19:31:47,344][105692] Updated weights for policy 0, policy_version 571718 (0.0009) [2023-12-26 19:31:47,399][105692] Updated weights for policy 0, policy_version 571728 (0.0008) [2023-12-26 19:31:47,450][105692] Updated weights for policy 0, policy_version 571738 (0.0008) [2023-12-26 19:31:47,547][105620] Updated weights for policy 1, policy_version 572553 (0.0007) [2023-12-26 19:31:47,610][105620] Updated weights for policy 1, policy_version 572563 (0.0010) [2023-12-26 19:31:47,664][105620] Updated weights for policy 1, policy_version 572574 (0.0010) [2023-12-26 19:31:47,718][105620] Updated weights for policy 1, policy_version 572584 (0.0011) [2023-12-26 19:31:48,114][105692] Updated weights for policy 0, policy_version 571748 (0.0008) [2023-12-26 19:31:48,171][105692] Updated weights for policy 0, policy_version 571760 (0.0010) [2023-12-26 19:31:48,234][105692] Updated weights for policy 0, policy_version 571770 (0.0006) [2023-12-26 19:31:48,399][105620] Updated weights for policy 1, policy_version 572594 (0.0006) [2023-12-26 19:31:48,449][105620] Updated weights for policy 1, policy_version 572604 (0.0006) [2023-12-26 19:31:48,517][105620] Updated weights for policy 1, policy_version 572614 (0.0006) [2023-12-26 19:31:49,009][105692] Updated weights for policy 0, policy_version 571780 (0.0006) [2023-12-26 19:31:49,060][105692] Updated weights for policy 0, policy_version 571790 (0.0007) [2023-12-26 19:31:49,115][105692] Updated weights for policy 0, policy_version 571800 (0.0009) [2023-12-26 19:31:49,154][105620] Updated weights for policy 1, policy_version 572624 (0.0008) [2023-12-26 19:31:49,216][105586] KL-divergence is very high: 103.1253 [2023-12-26 19:31:49,217][105620] Updated weights for policy 1, policy_version 572634 (0.0008) [2023-12-26 19:31:49,223][105586] KL-divergence is very high: 136.7691 [2023-12-26 19:31:49,237][105586] KL-divergence is very high: 118.9161 [2023-12-26 19:31:49,250][105586] KL-divergence is very high: 109.0293 [2023-12-26 19:31:49,276][105620] Updated weights for policy 1, policy_version 572644 (0.0008) [2023-12-26 19:31:49,737][105692] Updated weights for policy 0, policy_version 571810 (0.0010) [2023-12-26 19:31:49,799][105692] Updated weights for policy 0, policy_version 571820 (0.0010) [2023-12-26 19:31:49,864][105692] Updated weights for policy 0, policy_version 571830 (0.0008) [2023-12-26 19:31:49,878][105620] Updated weights for policy 1, policy_version 572654 (0.0008) [2023-12-26 19:31:49,927][105692] Updated weights for policy 0, policy_version 571840 (0.0007) [2023-12-26 19:31:49,945][105620] Updated weights for policy 1, policy_version 572664 (0.0007) [2023-12-26 19:31:50,011][105620] Updated weights for policy 1, policy_version 572674 (0.0008) [2023-12-26 19:31:50,713][105620] Updated weights for policy 1, policy_version 572684 (0.0007) [2023-12-26 19:31:50,721][105692] Updated weights for policy 0, policy_version 571850 (0.0011) [2023-12-26 19:31:50,774][105620] Updated weights for policy 1, policy_version 572694 (0.0006) [2023-12-26 19:31:50,787][105692] Updated weights for policy 0, policy_version 571860 (0.0011) [2023-12-26 19:31:50,844][105620] Updated weights for policy 1, policy_version 572704 (0.0006) [2023-12-26 19:31:50,857][105692] Updated weights for policy 0, policy_version 571870 (0.0011) [2023-12-26 19:31:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 293044224. Throughput: 0: 9708.1, 1: 10059.8. Samples: 293030956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:31:51,062][104569] Avg episode reward: [(0, '9265.845'), (1, '3555.283')] [2023-12-26 19:31:51,461][105620] Updated weights for policy 1, policy_version 572714 (0.0006) [2023-12-26 19:31:51,515][105620] Updated weights for policy 1, policy_version 572725 (0.0009) [2023-12-26 19:31:51,572][105620] Updated weights for policy 1, policy_version 572735 (0.0009) [2023-12-26 19:31:51,579][105692] Updated weights for policy 0, policy_version 571880 (0.0006) [2023-12-26 19:31:51,642][105692] Updated weights for policy 0, policy_version 571890 (0.0009) [2023-12-26 19:31:51,703][105692] Updated weights for policy 0, policy_version 571900 (0.0009) [2023-12-26 19:31:52,352][105692] Updated weights for policy 0, policy_version 571910 (0.0009) [2023-12-26 19:31:52,411][105692] Updated weights for policy 0, policy_version 571920 (0.0010) [2023-12-26 19:31:52,460][105620] Updated weights for policy 1, policy_version 572745 (0.0007) [2023-12-26 19:31:52,466][105692] Updated weights for policy 0, policy_version 571930 (0.0009) [2023-12-26 19:31:52,508][105620] Updated weights for policy 1, policy_version 572755 (0.0008) [2023-12-26 19:31:52,570][105620] Updated weights for policy 1, policy_version 572765 (0.0008) [2023-12-26 19:31:52,614][105586] KL-divergence is very high: 118.6550 [2023-12-26 19:31:52,620][105586] KL-divergence is very high: 136.4424 [2023-12-26 19:31:52,630][105620] Updated weights for policy 1, policy_version 572775 (0.0008) [2023-12-26 19:31:53,157][105692] Updated weights for policy 0, policy_version 571940 (0.0007) [2023-12-26 19:31:53,206][105692] Updated weights for policy 0, policy_version 571950 (0.0010) [2023-12-26 19:31:53,267][105692] Updated weights for policy 0, policy_version 571960 (0.0010) [2023-12-26 19:31:53,424][105620] Updated weights for policy 1, policy_version 572785 (0.0010) [2023-12-26 19:31:53,483][105620] Updated weights for policy 1, policy_version 572795 (0.0010) [2023-12-26 19:31:53,545][105620] Updated weights for policy 1, policy_version 572805 (0.0009) [2023-12-26 19:31:53,842][105692] Updated weights for policy 0, policy_version 571970 (0.0010) [2023-12-26 19:31:53,897][105692] Updated weights for policy 0, policy_version 571980 (0.0005) [2023-12-26 19:31:53,953][105692] Updated weights for policy 0, policy_version 571990 (0.0005) [2023-12-26 19:31:54,012][105692] Updated weights for policy 0, policy_version 572000 (0.0005) [2023-12-26 19:31:54,369][105620] Updated weights for policy 1, policy_version 572815 (0.0007) [2023-12-26 19:31:54,427][105620] Updated weights for policy 1, policy_version 572825 (0.0005) [2023-12-26 19:31:54,476][105620] Updated weights for policy 1, policy_version 572835 (0.0008) [2023-12-26 19:31:54,565][105692] Updated weights for policy 0, policy_version 572010 (0.0010) [2023-12-26 19:31:54,569][105585] KL-divergence is very high: 164.5347 [2023-12-26 19:31:54,608][105585] KL-divergence is very high: 285.8208 [2023-12-26 19:31:54,614][105692] Updated weights for policy 0, policy_version 572020 (0.0010) [2023-12-26 19:31:54,647][105585] KL-divergence is very high: 320.2192 [2023-12-26 19:31:54,662][105692] Updated weights for policy 0, policy_version 572030 (0.0010) [2023-12-26 19:31:55,105][105620] Updated weights for policy 1, policy_version 572845 (0.0006) [2023-12-26 19:31:55,169][105620] Updated weights for policy 1, policy_version 572855 (0.0005) [2023-12-26 19:31:55,240][105620] Updated weights for policy 1, policy_version 572865 (0.0005) [2023-12-26 19:31:55,366][105692] Updated weights for policy 0, policy_version 572040 (0.0010) [2023-12-26 19:31:55,417][105692] Updated weights for policy 0, policy_version 572050 (0.0010) [2023-12-26 19:31:55,471][105692] Updated weights for policy 0, policy_version 572060 (0.0010) [2023-12-26 19:31:55,861][105620] Updated weights for policy 1, policy_version 572875 (0.0006) [2023-12-26 19:31:55,919][105620] Updated weights for policy 1, policy_version 572885 (0.0008) [2023-12-26 19:31:55,974][105620] Updated weights for policy 1, policy_version 572895 (0.0008) [2023-12-26 19:31:56,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 293142528. Throughput: 0: 9788.0, 1: 9991.5. Samples: 293149472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:31:56,063][104569] Avg episode reward: [(0, '8908.646'), (1, '2146.272')] [2023-12-26 19:31:56,183][105692] Updated weights for policy 0, policy_version 572070 (0.0010) [2023-12-26 19:31:56,241][105692] Updated weights for policy 0, policy_version 572080 (0.0010) [2023-12-26 19:31:56,306][105692] Updated weights for policy 0, policy_version 572090 (0.0011) [2023-12-26 19:31:56,756][105620] Updated weights for policy 1, policy_version 572905 (0.0008) [2023-12-26 19:31:56,808][105620] Updated weights for policy 1, policy_version 572915 (0.0008) [2023-12-26 19:31:56,871][105620] Updated weights for policy 1, policy_version 572925 (0.0007) [2023-12-26 19:31:56,936][105620] Updated weights for policy 1, policy_version 572935 (0.0008) [2023-12-26 19:31:57,032][105692] Updated weights for policy 0, policy_version 572100 (0.0009) [2023-12-26 19:31:57,090][105692] Updated weights for policy 0, policy_version 572110 (0.0005) [2023-12-26 19:31:57,135][105692] Updated weights for policy 0, policy_version 572120 (0.0005) [2023-12-26 19:31:57,663][105620] Updated weights for policy 1, policy_version 572945 (0.0008) [2023-12-26 19:31:57,718][105620] Updated weights for policy 1, policy_version 572955 (0.0008) [2023-12-26 19:31:57,732][105692] Updated weights for policy 0, policy_version 572130 (0.0006) [2023-12-26 19:31:57,776][105620] Updated weights for policy 1, policy_version 572965 (0.0006) [2023-12-26 19:31:57,789][105692] Updated weights for policy 0, policy_version 572140 (0.0010) [2023-12-26 19:31:57,843][105692] Updated weights for policy 0, policy_version 572150 (0.0010) [2023-12-26 19:31:57,907][105692] Updated weights for policy 0, policy_version 572160 (0.0010) [2023-12-26 19:31:58,460][105620] Updated weights for policy 1, policy_version 572975 (0.0007) [2023-12-26 19:31:58,531][105620] Updated weights for policy 1, policy_version 572985 (0.0008) [2023-12-26 19:31:58,600][105620] Updated weights for policy 1, policy_version 572995 (0.0009) [2023-12-26 19:31:58,682][105692] Updated weights for policy 0, policy_version 572170 (0.0011) [2023-12-26 19:31:58,745][105692] Updated weights for policy 0, policy_version 572180 (0.0011) [2023-12-26 19:31:58,801][105692] Updated weights for policy 0, policy_version 572190 (0.0011) [2023-12-26 19:31:59,290][105620] Updated weights for policy 1, policy_version 573005 (0.0009) [2023-12-26 19:31:59,347][105620] Updated weights for policy 1, policy_version 573015 (0.0011) [2023-12-26 19:31:59,408][105620] Updated weights for policy 1, policy_version 573025 (0.0008) [2023-12-26 19:31:59,518][105692] Updated weights for policy 0, policy_version 572200 (0.0009) [2023-12-26 19:31:59,581][105692] Updated weights for policy 0, policy_version 572210 (0.0008) [2023-12-26 19:31:59,632][105692] Updated weights for policy 0, policy_version 572220 (0.0010) [2023-12-26 19:32:00,077][105620] Updated weights for policy 1, policy_version 573035 (0.0008) [2023-12-26 19:32:00,140][105620] Updated weights for policy 1, policy_version 573045 (0.0008) [2023-12-26 19:32:00,194][105620] Updated weights for policy 1, policy_version 573055 (0.0007) [2023-12-26 19:32:00,387][105692] Updated weights for policy 0, policy_version 572230 (0.0010) [2023-12-26 19:32:00,447][105692] Updated weights for policy 0, policy_version 572240 (0.0010) [2023-12-26 19:32:00,508][105692] Updated weights for policy 0, policy_version 572250 (0.0010) [2023-12-26 19:32:00,760][105620] Updated weights for policy 1, policy_version 573065 (0.0005) [2023-12-26 19:32:00,810][105620] Updated weights for policy 1, policy_version 573075 (0.0007) [2023-12-26 19:32:00,875][105620] Updated weights for policy 1, policy_version 573085 (0.0005) [2023-12-26 19:32:00,927][105620] Updated weights for policy 1, policy_version 573095 (0.0007) [2023-12-26 19:32:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 293240832. Throughput: 0: 9810.4, 1: 9943.2. Samples: 293207680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:32:01,063][104569] Avg episode reward: [(0, '8473.504'), (1, '6718.092')] [2023-12-26 19:32:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000572256_146513920.pth... [2023-12-26 19:32:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000573096_146726912.pth... [2023-12-26 19:32:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000571912_146423808.pth [2023-12-26 19:32:01,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000571136_146227200.pth [2023-12-26 19:32:01,220][105692] Updated weights for policy 0, policy_version 572260 (0.0009) [2023-12-26 19:32:01,281][105692] Updated weights for policy 0, policy_version 572270 (0.0010) [2023-12-26 19:32:01,340][105692] Updated weights for policy 0, policy_version 572280 (0.0011) [2023-12-26 19:32:01,624][105620] Updated weights for policy 1, policy_version 573105 (0.0009) [2023-12-26 19:32:01,693][105620] Updated weights for policy 1, policy_version 573115 (0.0009) [2023-12-26 19:32:01,757][105620] Updated weights for policy 1, policy_version 573125 (0.0006) [2023-12-26 19:32:02,109][105692] Updated weights for policy 0, policy_version 572290 (0.0010) [2023-12-26 19:32:02,154][105585] KL-divergence is very high: 513.8170 [2023-12-26 19:32:02,177][105692] Updated weights for policy 0, policy_version 572300 (0.0010) [2023-12-26 19:32:02,197][105585] KL-divergence is very high: 1068.5510 [2023-12-26 19:32:02,228][105692] Updated weights for policy 0, policy_version 572310 (0.0010) [2023-12-26 19:32:02,240][105585] KL-divergence is very high: 1185.5402 [2023-12-26 19:32:02,291][105692] Updated weights for policy 0, policy_version 572320 (0.0010) [2023-12-26 19:32:02,423][105620] Updated weights for policy 1, policy_version 573135 (0.0007) [2023-12-26 19:32:02,480][105620] Updated weights for policy 1, policy_version 573145 (0.0007) [2023-12-26 19:32:02,535][105620] Updated weights for policy 1, policy_version 573155 (0.0008) [2023-12-26 19:32:02,949][105692] Updated weights for policy 0, policy_version 572330 (0.0005) [2023-12-26 19:32:03,005][105692] Updated weights for policy 0, policy_version 572340 (0.0010) [2023-12-26 19:32:03,068][105692] Updated weights for policy 0, policy_version 572350 (0.0010) [2023-12-26 19:32:03,302][105620] Updated weights for policy 1, policy_version 573165 (0.0009) [2023-12-26 19:32:03,355][105620] Updated weights for policy 1, policy_version 573175 (0.0010) [2023-12-26 19:32:03,403][105620] Updated weights for policy 1, policy_version 573185 (0.0010) [2023-12-26 19:32:03,792][105692] Updated weights for policy 0, policy_version 572360 (0.0011) [2023-12-26 19:32:03,861][105692] Updated weights for policy 0, policy_version 572370 (0.0011) [2023-12-26 19:32:03,920][105692] Updated weights for policy 0, policy_version 572380 (0.0010) [2023-12-26 19:32:04,066][105620] Updated weights for policy 1, policy_version 573195 (0.0007) [2023-12-26 19:32:04,118][105620] Updated weights for policy 1, policy_version 573205 (0.0010) [2023-12-26 19:32:04,177][105620] Updated weights for policy 1, policy_version 573215 (0.0010) [2023-12-26 19:32:04,624][105692] Updated weights for policy 0, policy_version 572390 (0.0011) [2023-12-26 19:32:04,676][105692] Updated weights for policy 0, policy_version 572400 (0.0010) [2023-12-26 19:32:04,730][105692] Updated weights for policy 0, policy_version 572410 (0.0010) [2023-12-26 19:32:04,889][105620] Updated weights for policy 1, policy_version 573225 (0.0010) [2023-12-26 19:32:04,950][105620] Updated weights for policy 1, policy_version 573235 (0.0005) [2023-12-26 19:32:05,008][105620] Updated weights for policy 1, policy_version 573245 (0.0005) [2023-12-26 19:32:05,062][105620] Updated weights for policy 1, policy_version 573255 (0.0005) [2023-12-26 19:32:05,493][105692] Updated weights for policy 0, policy_version 572420 (0.0010) [2023-12-26 19:32:05,558][105692] Updated weights for policy 0, policy_version 572430 (0.0010) [2023-12-26 19:32:05,613][105692] Updated weights for policy 0, policy_version 572440 (0.0010) [2023-12-26 19:32:05,618][105620] Updated weights for policy 1, policy_version 573265 (0.0005) [2023-12-26 19:32:05,676][105620] Updated weights for policy 1, policy_version 573275 (0.0005) [2023-12-26 19:32:05,729][105620] Updated weights for policy 1, policy_version 573285 (0.0005) [2023-12-26 19:32:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 293339136. Throughput: 0: 9697.1, 1: 10025.3. Samples: 293327356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:32:06,063][104569] Avg episode reward: [(0, '8383.935'), (1, '9260.602')] [2023-12-26 19:32:06,303][105620] Updated weights for policy 1, policy_version 573295 (0.0008) [2023-12-26 19:32:06,356][105692] Updated weights for policy 0, policy_version 572450 (0.0007) [2023-12-26 19:32:06,372][105620] Updated weights for policy 1, policy_version 573305 (0.0009) [2023-12-26 19:32:06,418][105692] Updated weights for policy 0, policy_version 572460 (0.0006) [2023-12-26 19:32:06,428][105620] Updated weights for policy 1, policy_version 573315 (0.0009) [2023-12-26 19:32:06,486][105692] Updated weights for policy 0, policy_version 572470 (0.0007) [2023-12-26 19:32:06,552][105692] Updated weights for policy 0, policy_version 572480 (0.0008) [2023-12-26 19:32:07,158][105692] Updated weights for policy 0, policy_version 572490 (0.0009) [2023-12-26 19:32:07,220][105692] Updated weights for policy 0, policy_version 572500 (0.0008) [2023-12-26 19:32:07,242][105620] Updated weights for policy 1, policy_version 573325 (0.0009) [2023-12-26 19:32:07,276][105692] Updated weights for policy 0, policy_version 572510 (0.0007) [2023-12-26 19:32:07,304][105620] Updated weights for policy 1, policy_version 573335 (0.0009) [2023-12-26 19:32:07,369][105620] Updated weights for policy 1, policy_version 573345 (0.0010) [2023-12-26 19:32:07,986][105620] Updated weights for policy 1, policy_version 573355 (0.0009) [2023-12-26 19:32:08,052][105620] Updated weights for policy 1, policy_version 573365 (0.0010) [2023-12-26 19:32:08,066][105692] Updated weights for policy 0, policy_version 572520 (0.0006) [2023-12-26 19:32:08,105][105620] Updated weights for policy 1, policy_version 573375 (0.0010) [2023-12-26 19:32:08,112][105692] Updated weights for policy 0, policy_version 572530 (0.0005) [2023-12-26 19:32:08,160][105692] Updated weights for policy 0, policy_version 572540 (0.0007) [2023-12-26 19:32:08,708][105620] Updated weights for policy 1, policy_version 573385 (0.0010) [2023-12-26 19:32:08,768][105620] Updated weights for policy 1, policy_version 573395 (0.0005) [2023-12-26 19:32:08,827][105620] Updated weights for policy 1, policy_version 573405 (0.0005) [2023-12-26 19:32:08,883][105620] Updated weights for policy 1, policy_version 573415 (0.0005) [2023-12-26 19:32:09,073][105692] Updated weights for policy 0, policy_version 572550 (0.0009) [2023-12-26 19:32:09,143][105692] Updated weights for policy 0, policy_version 572560 (0.0009) [2023-12-26 19:32:09,206][105692] Updated weights for policy 0, policy_version 572570 (0.0010) [2023-12-26 19:32:09,524][105620] Updated weights for policy 1, policy_version 573425 (0.0009) [2023-12-26 19:32:09,590][105620] Updated weights for policy 1, policy_version 573435 (0.0009) [2023-12-26 19:32:09,658][105620] Updated weights for policy 1, policy_version 573445 (0.0009) [2023-12-26 19:32:09,945][105692] Updated weights for policy 0, policy_version 572580 (0.0009) [2023-12-26 19:32:10,008][105692] Updated weights for policy 0, policy_version 572590 (0.0009) [2023-12-26 19:32:10,072][105692] Updated weights for policy 0, policy_version 572600 (0.0010) [2023-12-26 19:32:10,426][105620] Updated weights for policy 1, policy_version 573455 (0.0009) [2023-12-26 19:32:10,486][105620] Updated weights for policy 1, policy_version 573465 (0.0008) [2023-12-26 19:32:10,544][105620] Updated weights for policy 1, policy_version 573475 (0.0008) [2023-12-26 19:32:10,852][105692] Updated weights for policy 0, policy_version 572610 (0.0009) [2023-12-26 19:32:10,909][105692] Updated weights for policy 0, policy_version 572620 (0.0009) [2023-12-26 19:32:10,965][105692] Updated weights for policy 0, policy_version 572630 (0.0009) [2023-12-26 19:32:11,024][105692] Updated weights for policy 0, policy_version 572640 (0.0009) [2023-12-26 19:32:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 293437440. Throughput: 0: 9596.6, 1: 10148.3. Samples: 293443720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:32:11,063][104569] Avg episode reward: [(0, '8399.730'), (1, '9351.741')] [2023-12-26 19:32:11,344][105620] Updated weights for policy 1, policy_version 573485 (0.0009) [2023-12-26 19:32:11,409][105620] Updated weights for policy 1, policy_version 573495 (0.0009) [2023-12-26 19:32:11,465][105620] Updated weights for policy 1, policy_version 573505 (0.0009) [2023-12-26 19:32:11,845][105692] Updated weights for policy 0, policy_version 572650 (0.0009) [2023-12-26 19:32:11,910][105692] Updated weights for policy 0, policy_version 572660 (0.0010) [2023-12-26 19:32:11,978][105692] Updated weights for policy 0, policy_version 572670 (0.0009) [2023-12-26 19:32:12,267][105620] Updated weights for policy 1, policy_version 573515 (0.0009) [2023-12-26 19:32:12,335][105620] Updated weights for policy 1, policy_version 573525 (0.0008) [2023-12-26 19:32:12,403][105620] Updated weights for policy 1, policy_version 573535 (0.0008) [2023-12-26 19:32:12,710][105692] Updated weights for policy 0, policy_version 572680 (0.0006) [2023-12-26 19:32:12,768][105692] Updated weights for policy 0, policy_version 572690 (0.0006) [2023-12-26 19:32:12,830][105692] Updated weights for policy 0, policy_version 572700 (0.0006) [2023-12-26 19:32:13,124][105620] Updated weights for policy 1, policy_version 573545 (0.0008) [2023-12-26 19:32:13,175][105620] Updated weights for policy 1, policy_version 573555 (0.0010) [2023-12-26 19:32:13,233][105620] Updated weights for policy 1, policy_version 573565 (0.0010) [2023-12-26 19:32:13,284][105620] Updated weights for policy 1, policy_version 573575 (0.0010) [2023-12-26 19:32:13,530][105692] Updated weights for policy 0, policy_version 572710 (0.0008) [2023-12-26 19:32:13,594][105692] Updated weights for policy 0, policy_version 572720 (0.0009) [2023-12-26 19:32:13,650][105692] Updated weights for policy 0, policy_version 572730 (0.0009) [2023-12-26 19:32:14,053][105620] Updated weights for policy 1, policy_version 573585 (0.0009) [2023-12-26 19:32:14,100][105620] Updated weights for policy 1, policy_version 573595 (0.0009) [2023-12-26 19:32:14,163][105620] Updated weights for policy 1, policy_version 573605 (0.0009) [2023-12-26 19:32:14,383][105692] Updated weights for policy 0, policy_version 572740 (0.0009) [2023-12-26 19:32:14,444][105692] Updated weights for policy 0, policy_version 572750 (0.0009) [2023-12-26 19:32:14,501][105692] Updated weights for policy 0, policy_version 572760 (0.0009) [2023-12-26 19:32:14,522][105585] KL-divergence is very high: 158.5888 [2023-12-26 19:32:14,921][105620] Updated weights for policy 1, policy_version 573615 (0.0009) [2023-12-26 19:32:14,976][105620] Updated weights for policy 1, policy_version 573625 (0.0009) [2023-12-26 19:32:15,036][105620] Updated weights for policy 1, policy_version 573635 (0.0009) [2023-12-26 19:32:15,283][105692] Updated weights for policy 0, policy_version 572770 (0.0010) [2023-12-26 19:32:15,338][105692] Updated weights for policy 0, policy_version 572780 (0.0009) [2023-12-26 19:32:15,390][105692] Updated weights for policy 0, policy_version 572790 (0.0009) [2023-12-26 19:32:15,441][105692] Updated weights for policy 0, policy_version 572800 (0.0009) [2023-12-26 19:32:15,868][105620] Updated weights for policy 1, policy_version 573645 (0.0009) [2023-12-26 19:32:15,927][105620] Updated weights for policy 1, policy_version 573655 (0.0009) [2023-12-26 19:32:15,989][105620] Updated weights for policy 1, policy_version 573665 (0.0010) [2023-12-26 19:32:16,040][105692] Updated weights for policy 0, policy_version 572810 (0.0007) [2023-12-26 19:32:16,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 293527552. Throughput: 0: 9512.7, 1: 10023.4. Samples: 293499400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:32:16,062][104569] Avg episode reward: [(0, '7948.598'), (1, '9351.720')] [2023-12-26 19:32:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000573672_146874368.pth... [2023-12-26 19:32:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000572520_146579456.pth [2023-12-26 19:32:16,095][105692] Updated weights for policy 0, policy_version 572820 (0.0009) [2023-12-26 19:32:16,152][105692] Updated weights for policy 0, policy_version 572830 (0.0009) [2023-12-26 19:32:16,162][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000572832_146661376.pth... [2023-12-26 19:32:16,167][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000571680_146366464.pth [2023-12-26 19:32:16,732][105620] Updated weights for policy 1, policy_version 573675 (0.0008) [2023-12-26 19:32:16,779][105620] Updated weights for policy 1, policy_version 573685 (0.0009) [2023-12-26 19:32:16,825][105620] Updated weights for policy 1, policy_version 573695 (0.0006) [2023-12-26 19:32:16,938][105692] Updated weights for policy 0, policy_version 572840 (0.0010) [2023-12-26 19:32:17,000][105692] Updated weights for policy 0, policy_version 572850 (0.0010) [2023-12-26 19:32:17,057][105692] Updated weights for policy 0, policy_version 572860 (0.0010) [2023-12-26 19:32:17,546][105620] Updated weights for policy 1, policy_version 573705 (0.0006) [2023-12-26 19:32:17,597][105620] Updated weights for policy 1, policy_version 573715 (0.0009) [2023-12-26 19:32:17,651][105620] Updated weights for policy 1, policy_version 573725 (0.0010) [2023-12-26 19:32:17,715][105620] Updated weights for policy 1, policy_version 573735 (0.0009) [2023-12-26 19:32:17,745][105692] Updated weights for policy 0, policy_version 572870 (0.0011) [2023-12-26 19:32:17,804][105692] Updated weights for policy 0, policy_version 572880 (0.0011) [2023-12-26 19:32:17,856][105692] Updated weights for policy 0, policy_version 572890 (0.0010) [2023-12-26 19:32:18,512][105620] Updated weights for policy 1, policy_version 573745 (0.0007) [2023-12-26 19:32:18,561][105620] Updated weights for policy 1, policy_version 573755 (0.0008) [2023-12-26 19:32:18,617][105620] Updated weights for policy 1, policy_version 573765 (0.0008) [2023-12-26 19:32:18,627][105692] Updated weights for policy 0, policy_version 572900 (0.0011) [2023-12-26 19:32:18,686][105692] Updated weights for policy 0, policy_version 572910 (0.0010) [2023-12-26 19:32:18,745][105692] Updated weights for policy 0, policy_version 572920 (0.0011) [2023-12-26 19:32:19,394][105620] Updated weights for policy 1, policy_version 573775 (0.0008) [2023-12-26 19:32:19,449][105620] Updated weights for policy 1, policy_version 573785 (0.0009) [2023-12-26 19:32:19,503][105692] Updated weights for policy 0, policy_version 572930 (0.0010) [2023-12-26 19:32:19,516][105620] Updated weights for policy 1, policy_version 573795 (0.0009) [2023-12-26 19:32:19,557][105692] Updated weights for policy 0, policy_version 572940 (0.0006) [2023-12-26 19:32:19,620][105692] Updated weights for policy 0, policy_version 572950 (0.0005) [2023-12-26 19:32:19,692][105692] Updated weights for policy 0, policy_version 572960 (0.0005) [2023-12-26 19:32:20,260][105620] Updated weights for policy 1, policy_version 573805 (0.0009) [2023-12-26 19:32:20,326][105620] Updated weights for policy 1, policy_version 573815 (0.0009) [2023-12-26 19:32:20,384][105620] Updated weights for policy 1, policy_version 573825 (0.0008) [2023-12-26 19:32:20,424][105692] Updated weights for policy 0, policy_version 572970 (0.0008) [2023-12-26 19:32:20,480][105692] Updated weights for policy 0, policy_version 572980 (0.0010) [2023-12-26 19:32:20,534][105692] Updated weights for policy 0, policy_version 572990 (0.0009) [2023-12-26 19:32:21,043][105620] Updated weights for policy 1, policy_version 573835 (0.0008) [2023-12-26 19:32:21,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 293617664. Throughput: 0: 9512.5, 1: 9865.6. Samples: 293611436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:32:21,062][104569] Avg episode reward: [(0, '7947.371'), (1, '9056.925')] [2023-12-26 19:32:21,097][105620] Updated weights for policy 1, policy_version 573845 (0.0010) [2023-12-26 19:32:21,158][105620] Updated weights for policy 1, policy_version 573855 (0.0009) [2023-12-26 19:32:21,400][105692] Updated weights for policy 0, policy_version 573000 (0.0009) [2023-12-26 19:32:21,462][105692] Updated weights for policy 0, policy_version 573010 (0.0009) [2023-12-26 19:32:21,525][105692] Updated weights for policy 0, policy_version 573020 (0.0009) [2023-12-26 19:32:21,965][105620] Updated weights for policy 1, policy_version 573865 (0.0009) [2023-12-26 19:32:22,025][105620] Updated weights for policy 1, policy_version 573875 (0.0009) [2023-12-26 19:32:22,080][105620] Updated weights for policy 1, policy_version 573885 (0.0010) [2023-12-26 19:32:22,147][105620] Updated weights for policy 1, policy_version 573895 (0.0009) [2023-12-26 19:32:22,234][105692] Updated weights for policy 0, policy_version 573030 (0.0010) [2023-12-26 19:32:22,296][105692] Updated weights for policy 0, policy_version 573040 (0.0011) [2023-12-26 19:32:22,359][105692] Updated weights for policy 0, policy_version 573050 (0.0011) [2023-12-26 19:32:22,979][105620] Updated weights for policy 1, policy_version 573905 (0.0009) [2023-12-26 19:32:23,037][105620] Updated weights for policy 1, policy_version 573915 (0.0008) [2023-12-26 19:32:23,077][105692] Updated weights for policy 0, policy_version 573060 (0.0008) [2023-12-26 19:32:23,092][105620] Updated weights for policy 1, policy_version 573925 (0.0009) [2023-12-26 19:32:23,135][105692] Updated weights for policy 0, policy_version 573070 (0.0008) [2023-12-26 19:32:23,195][105692] Updated weights for policy 0, policy_version 573080 (0.0009) [2023-12-26 19:32:23,864][105620] Updated weights for policy 1, policy_version 573935 (0.0009) [2023-12-26 19:32:23,919][105620] Updated weights for policy 1, policy_version 573945 (0.0008) [2023-12-26 19:32:23,944][105692] Updated weights for policy 0, policy_version 573090 (0.0010) [2023-12-26 19:32:23,980][105620] Updated weights for policy 1, policy_version 573955 (0.0008) [2023-12-26 19:32:24,012][105692] Updated weights for policy 0, policy_version 573100 (0.0010) [2023-12-26 19:32:24,070][105692] Updated weights for policy 0, policy_version 573110 (0.0010) [2023-12-26 19:32:24,125][105692] Updated weights for policy 0, policy_version 573120 (0.0010) [2023-12-26 19:32:24,742][105620] Updated weights for policy 1, policy_version 573965 (0.0006) [2023-12-26 19:32:24,772][105692] Updated weights for policy 0, policy_version 573130 (0.0005) [2023-12-26 19:32:24,804][105620] Updated weights for policy 1, policy_version 573975 (0.0007) [2023-12-26 19:32:24,835][105692] Updated weights for policy 0, policy_version 573140 (0.0009) [2023-12-26 19:32:24,860][105620] Updated weights for policy 1, policy_version 573985 (0.0009) [2023-12-26 19:32:24,882][105692] Updated weights for policy 0, policy_version 573150 (0.0008) [2023-12-26 19:32:25,517][105620] Updated weights for policy 1, policy_version 573995 (0.0006) [2023-12-26 19:32:25,573][105620] Updated weights for policy 1, policy_version 574005 (0.0007) [2023-12-26 19:32:25,629][105620] Updated weights for policy 1, policy_version 574015 (0.0008) [2023-12-26 19:32:25,666][105692] Updated weights for policy 0, policy_version 573160 (0.0008) [2023-12-26 19:32:25,722][105692] Updated weights for policy 0, policy_version 573171 (0.0010) [2023-12-26 19:32:25,776][105692] Updated weights for policy 0, policy_version 573181 (0.0010) [2023-12-26 19:32:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 293715968. Throughput: 0: 9564.2, 1: 9813.6. Samples: 293725220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:32:26,062][104569] Avg episode reward: [(0, '8630.331'), (1, '7717.057')] [2023-12-26 19:32:26,192][105620] Updated weights for policy 1, policy_version 574025 (0.0006) [2023-12-26 19:32:26,253][105620] Updated weights for policy 1, policy_version 574035 (0.0005) [2023-12-26 19:32:26,308][105620] Updated weights for policy 1, policy_version 574045 (0.0005) [2023-12-26 19:32:26,363][105620] Updated weights for policy 1, policy_version 574055 (0.0005) [2023-12-26 19:32:26,707][105692] Updated weights for policy 0, policy_version 573192 (0.0010) [2023-12-26 19:32:26,765][105692] Updated weights for policy 0, policy_version 573203 (0.0010) [2023-12-26 19:32:26,826][105692] Updated weights for policy 0, policy_version 573214 (0.0010) [2023-12-26 19:32:26,869][105620] Updated weights for policy 1, policy_version 574065 (0.0010) [2023-12-26 19:32:26,920][105620] Updated weights for policy 1, policy_version 574075 (0.0010) [2023-12-26 19:32:26,985][105620] Updated weights for policy 1, policy_version 574085 (0.0010) [2023-12-26 19:32:27,560][105692] Updated weights for policy 0, policy_version 573224 (0.0009) [2023-12-26 19:32:27,613][105620] Updated weights for policy 1, policy_version 574095 (0.0008) [2023-12-26 19:32:27,615][105692] Updated weights for policy 0, policy_version 573235 (0.0008) [2023-12-26 19:32:27,671][105692] Updated weights for policy 0, policy_version 573245 (0.0009) [2023-12-26 19:32:27,675][105620] Updated weights for policy 1, policy_version 574105 (0.0005) [2023-12-26 19:32:27,723][105620] Updated weights for policy 1, policy_version 574115 (0.0005) [2023-12-26 19:32:28,314][105620] Updated weights for policy 1, policy_version 574125 (0.0005) [2023-12-26 19:32:28,382][105620] Updated weights for policy 1, policy_version 574135 (0.0008) [2023-12-26 19:32:28,443][105620] Updated weights for policy 1, policy_version 574145 (0.0010) [2023-12-26 19:32:28,499][105692] Updated weights for policy 0, policy_version 573255 (0.0007) [2023-12-26 19:32:28,556][105692] Updated weights for policy 0, policy_version 573265 (0.0009) [2023-12-26 19:32:28,614][105692] Updated weights for policy 0, policy_version 573275 (0.0009) [2023-12-26 19:32:29,194][105620] Updated weights for policy 1, policy_version 574155 (0.0008) [2023-12-26 19:32:29,261][105620] Updated weights for policy 1, policy_version 574165 (0.0009) [2023-12-26 19:32:29,264][105692] Updated weights for policy 0, policy_version 573285 (0.0008) [2023-12-26 19:32:29,316][105620] Updated weights for policy 1, policy_version 574175 (0.0008) [2023-12-26 19:32:29,323][105692] Updated weights for policy 0, policy_version 573295 (0.0007) [2023-12-26 19:32:29,382][105692] Updated weights for policy 0, policy_version 573305 (0.0007) [2023-12-26 19:32:30,070][105620] Updated weights for policy 1, policy_version 574185 (0.0008) [2023-12-26 19:32:30,110][105692] Updated weights for policy 0, policy_version 573315 (0.0008) [2023-12-26 19:32:30,129][105620] Updated weights for policy 1, policy_version 574195 (0.0007) [2023-12-26 19:32:30,167][105692] Updated weights for policy 0, policy_version 573325 (0.0006) [2023-12-26 19:32:30,193][105620] Updated weights for policy 1, policy_version 574205 (0.0007) [2023-12-26 19:32:30,207][105586] KL-divergence is very high: 168.1115 [2023-12-26 19:32:30,226][105692] Updated weights for policy 0, policy_version 573335 (0.0008) [2023-12-26 19:32:30,252][105586] KL-divergence is very high: 178.6763 [2023-12-26 19:32:30,254][105620] Updated weights for policy 1, policy_version 574215 (0.0008) [2023-12-26 19:32:30,921][105692] Updated weights for policy 0, policy_version 573345 (0.0008) [2023-12-26 19:32:30,970][105692] Updated weights for policy 0, policy_version 573355 (0.0008) [2023-12-26 19:32:30,971][105620] Updated weights for policy 1, policy_version 574225 (0.0005) [2023-12-26 19:32:31,026][105692] Updated weights for policy 0, policy_version 573365 (0.0005) [2023-12-26 19:32:31,028][105620] Updated weights for policy 1, policy_version 574235 (0.0008) [2023-12-26 19:32:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 293806080. Throughput: 0: 9527.7, 1: 9923.5. Samples: 293784952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:32:31,062][104569] Avg episode reward: [(0, '8730.112'), (1, '3926.283')] [2023-12-26 19:32:31,086][105692] Updated weights for policy 0, policy_version 573375 (0.0008) [2023-12-26 19:32:31,088][105620] Updated weights for policy 1, policy_version 574245 (0.0008) [2023-12-26 19:32:31,090][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000573376_146800640.pth... [2023-12-26 19:32:31,094][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000572256_146513920.pth [2023-12-26 19:32:31,104][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000574248_147021824.pth... [2023-12-26 19:32:31,108][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000573096_146726912.pth [2023-12-26 19:32:31,814][105692] Updated weights for policy 0, policy_version 573385 (0.0008) [2023-12-26 19:32:31,834][105620] Updated weights for policy 1, policy_version 574255 (0.0009) [2023-12-26 19:32:31,876][105692] Updated weights for policy 0, policy_version 573395 (0.0008) [2023-12-26 19:32:31,896][105620] Updated weights for policy 1, policy_version 574265 (0.0008) [2023-12-26 19:32:31,935][105692] Updated weights for policy 0, policy_version 573405 (0.0005) [2023-12-26 19:32:31,962][105620] Updated weights for policy 1, policy_version 574275 (0.0008) [2023-12-26 19:32:32,596][105692] Updated weights for policy 0, policy_version 573415 (0.0006) [2023-12-26 19:32:32,651][105692] Updated weights for policy 0, policy_version 573425 (0.0007) [2023-12-26 19:32:32,653][105620] Updated weights for policy 1, policy_version 574285 (0.0007) [2023-12-26 19:32:32,711][105692] Updated weights for policy 0, policy_version 573435 (0.0006) [2023-12-26 19:32:32,713][105620] Updated weights for policy 1, policy_version 574295 (0.0008) [2023-12-26 19:32:32,770][105620] Updated weights for policy 1, policy_version 574305 (0.0009) [2023-12-26 19:32:33,254][105692] Updated weights for policy 0, policy_version 573445 (0.0007) [2023-12-26 19:32:33,321][105692] Updated weights for policy 0, policy_version 573455 (0.0005) [2023-12-26 19:32:33,391][105692] Updated weights for policy 0, policy_version 573465 (0.0005) [2023-12-26 19:32:33,676][105620] Updated weights for policy 1, policy_version 574315 (0.0009) [2023-12-26 19:32:33,726][105620] Updated weights for policy 1, policy_version 574325 (0.0009) [2023-12-26 19:32:33,786][105620] Updated weights for policy 1, policy_version 574335 (0.0009) [2023-12-26 19:32:33,946][105692] Updated weights for policy 0, policy_version 573475 (0.0007) [2023-12-26 19:32:34,008][105692] Updated weights for policy 0, policy_version 573485 (0.0006) [2023-12-26 19:32:34,059][105692] Updated weights for policy 0, policy_version 573495 (0.0005) [2023-12-26 19:32:34,629][105620] Updated weights for policy 1, policy_version 574345 (0.0009) [2023-12-26 19:32:34,686][105620] Updated weights for policy 1, policy_version 574355 (0.0010) [2023-12-26 19:32:34,696][105692] Updated weights for policy 0, policy_version 573505 (0.0009) [2023-12-26 19:32:34,739][105620] Updated weights for policy 1, policy_version 574365 (0.0009) [2023-12-26 19:32:34,750][105692] Updated weights for policy 0, policy_version 573515 (0.0006) [2023-12-26 19:32:34,794][105620] Updated weights for policy 1, policy_version 574375 (0.0009) [2023-12-26 19:32:34,797][105692] Updated weights for policy 0, policy_version 573525 (0.0005) [2023-12-26 19:32:34,843][105692] Updated weights for policy 0, policy_version 573535 (0.0008) [2023-12-26 19:32:35,535][105692] Updated weights for policy 0, policy_version 573545 (0.0009) [2023-12-26 19:32:35,580][105620] Updated weights for policy 1, policy_version 574385 (0.0005) [2023-12-26 19:32:35,595][105692] Updated weights for policy 0, policy_version 573555 (0.0009) [2023-12-26 19:32:35,637][105620] Updated weights for policy 1, policy_version 574395 (0.0006) [2023-12-26 19:32:35,650][105692] Updated weights for policy 0, policy_version 573565 (0.0008) [2023-12-26 19:32:35,687][105620] Updated weights for policy 1, policy_version 574405 (0.0007) [2023-12-26 19:32:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 293912576. Throughput: 0: 9660.6, 1: 9692.4. Samples: 293901840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:32:36,062][104569] Avg episode reward: [(0, '8482.249'), (1, '6102.156')] [2023-12-26 19:32:36,365][105620] Updated weights for policy 1, policy_version 574415 (0.0009) [2023-12-26 19:32:36,422][105620] Updated weights for policy 1, policy_version 574425 (0.0008) [2023-12-26 19:32:36,455][105692] Updated weights for policy 0, policy_version 573575 (0.0009) [2023-12-26 19:32:36,489][105620] Updated weights for policy 1, policy_version 574435 (0.0007) [2023-12-26 19:32:36,516][105692] Updated weights for policy 0, policy_version 573585 (0.0008) [2023-12-26 19:32:36,569][105692] Updated weights for policy 0, policy_version 573595 (0.0008) [2023-12-26 19:32:37,245][105620] Updated weights for policy 1, policy_version 574445 (0.0008) [2023-12-26 19:32:37,301][105620] Updated weights for policy 1, policy_version 574455 (0.0009) [2023-12-26 19:32:37,325][105692] Updated weights for policy 0, policy_version 573605 (0.0008) [2023-12-26 19:32:37,360][105620] Updated weights for policy 1, policy_version 574465 (0.0007) [2023-12-26 19:32:37,378][105692] Updated weights for policy 0, policy_version 573615 (0.0008) [2023-12-26 19:32:37,434][105692] Updated weights for policy 0, policy_version 573625 (0.0008) [2023-12-26 19:32:38,116][105692] Updated weights for policy 0, policy_version 573635 (0.0009) [2023-12-26 19:32:38,160][105620] Updated weights for policy 1, policy_version 574475 (0.0006) [2023-12-26 19:32:38,162][105692] Updated weights for policy 0, policy_version 573645 (0.0009) [2023-12-26 19:32:38,208][105620] Updated weights for policy 1, policy_version 574485 (0.0006) [2023-12-26 19:32:38,221][105692] Updated weights for policy 0, policy_version 573655 (0.0008) [2023-12-26 19:32:38,259][105620] Updated weights for policy 1, policy_version 574495 (0.0006) [2023-12-26 19:32:38,978][105692] Updated weights for policy 0, policy_version 573665 (0.0009) [2023-12-26 19:32:39,039][105692] Updated weights for policy 0, policy_version 573675 (0.0008) [2023-12-26 19:32:39,047][105620] Updated weights for policy 1, policy_version 574505 (0.0007) [2023-12-26 19:32:39,094][105692] Updated weights for policy 0, policy_version 573685 (0.0008) [2023-12-26 19:32:39,105][105620] Updated weights for policy 1, policy_version 574515 (0.0007) [2023-12-26 19:32:39,153][105692] Updated weights for policy 0, policy_version 573695 (0.0008) [2023-12-26 19:32:39,156][105620] Updated weights for policy 1, policy_version 574525 (0.0006) [2023-12-26 19:32:39,216][105620] Updated weights for policy 1, policy_version 574535 (0.0009) [2023-12-26 19:32:39,826][105692] Updated weights for policy 0, policy_version 573705 (0.0009) [2023-12-26 19:32:39,887][105692] Updated weights for policy 0, policy_version 573715 (0.0010) [2023-12-26 19:32:39,958][105692] Updated weights for policy 0, policy_version 573725 (0.0010) [2023-12-26 19:32:40,038][105620] Updated weights for policy 1, policy_version 574545 (0.0008) [2023-12-26 19:32:40,101][105620] Updated weights for policy 1, policy_version 574555 (0.0009) [2023-12-26 19:32:40,163][105620] Updated weights for policy 1, policy_version 574565 (0.0009) [2023-12-26 19:32:40,724][105692] Updated weights for policy 0, policy_version 573735 (0.0009) [2023-12-26 19:32:40,783][105692] Updated weights for policy 0, policy_version 573745 (0.0009) [2023-12-26 19:32:40,844][105692] Updated weights for policy 0, policy_version 573755 (0.0010) [2023-12-26 19:32:40,915][105620] Updated weights for policy 1, policy_version 574575 (0.0009) [2023-12-26 19:32:40,972][105620] Updated weights for policy 1, policy_version 574585 (0.0009) [2023-12-26 19:32:41,027][105620] Updated weights for policy 1, policy_version 574595 (0.0010) [2023-12-26 19:32:41,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 294010880. Throughput: 0: 9589.1, 1: 9648.4. Samples: 294015156. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:32:41,062][104569] Avg episode reward: [(0, '8390.793'), (1, '7398.506')] [2023-12-26 19:32:41,600][105692] Updated weights for policy 0, policy_version 573765 (0.0008) [2023-12-26 19:32:41,666][105692] Updated weights for policy 0, policy_version 573775 (0.0010) [2023-12-26 19:32:41,737][105692] Updated weights for policy 0, policy_version 573785 (0.0008) [2023-12-26 19:32:41,844][105620] Updated weights for policy 1, policy_version 574605 (0.0009) [2023-12-26 19:32:41,892][105620] Updated weights for policy 1, policy_version 574615 (0.0008) [2023-12-26 19:32:41,937][105620] Updated weights for policy 1, policy_version 574625 (0.0008) [2023-12-26 19:32:42,501][105692] Updated weights for policy 0, policy_version 573795 (0.0007) [2023-12-26 19:32:42,558][105692] Updated weights for policy 0, policy_version 573805 (0.0005) [2023-12-26 19:32:42,613][105692] Updated weights for policy 0, policy_version 573815 (0.0005) [2023-12-26 19:32:42,641][105620] Updated weights for policy 1, policy_version 574635 (0.0007) [2023-12-26 19:32:42,699][105620] Updated weights for policy 1, policy_version 574645 (0.0007) [2023-12-26 19:32:42,764][105620] Updated weights for policy 1, policy_version 574655 (0.0008) [2023-12-26 19:32:43,269][105692] Updated weights for policy 0, policy_version 573825 (0.0010) [2023-12-26 19:32:43,318][105692] Updated weights for policy 0, policy_version 573835 (0.0005) [2023-12-26 19:32:43,347][105585] KL-divergence is very high: 100.8507 [2023-12-26 19:32:43,381][105692] Updated weights for policy 0, policy_version 573845 (0.0005) [2023-12-26 19:32:43,437][105692] Updated weights for policy 0, policy_version 573855 (0.0005) [2023-12-26 19:32:43,505][105620] Updated weights for policy 1, policy_version 574665 (0.0008) [2023-12-26 19:32:43,565][105620] Updated weights for policy 1, policy_version 574675 (0.0008) [2023-12-26 19:32:43,619][105620] Updated weights for policy 1, policy_version 574685 (0.0010) [2023-12-26 19:32:43,676][105620] Updated weights for policy 1, policy_version 574695 (0.0010) [2023-12-26 19:32:43,974][105692] Updated weights for policy 0, policy_version 573865 (0.0010) [2023-12-26 19:32:44,027][105692] Updated weights for policy 0, policy_version 573875 (0.0007) [2023-12-26 19:32:44,083][105692] Updated weights for policy 0, policy_version 573885 (0.0010) [2023-12-26 19:32:44,356][105620] Updated weights for policy 1, policy_version 574705 (0.0006) [2023-12-26 19:32:44,415][105620] Updated weights for policy 1, policy_version 574715 (0.0008) [2023-12-26 19:32:44,470][105620] Updated weights for policy 1, policy_version 574725 (0.0009) [2023-12-26 19:32:44,643][105692] Updated weights for policy 0, policy_version 573895 (0.0007) [2023-12-26 19:32:44,697][105692] Updated weights for policy 0, policy_version 573905 (0.0005) [2023-12-26 19:32:44,750][105692] Updated weights for policy 0, policy_version 573915 (0.0006) [2023-12-26 19:32:45,235][105620] Updated weights for policy 1, policy_version 574735 (0.0007) [2023-12-26 19:32:45,293][105620] Updated weights for policy 1, policy_version 574745 (0.0009) [2023-12-26 19:32:45,355][105620] Updated weights for policy 1, policy_version 574755 (0.0006) [2023-12-26 19:32:45,450][105692] Updated weights for policy 0, policy_version 573925 (0.0008) [2023-12-26 19:32:45,511][105692] Updated weights for policy 0, policy_version 573935 (0.0009) [2023-12-26 19:32:45,572][105692] Updated weights for policy 0, policy_version 573945 (0.0008) [2023-12-26 19:32:46,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19387.6, 300 sec: 19549.7). Total num frames: 294100992. Throughput: 0: 9568.7, 1: 9661.2. Samples: 294073028. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:32:46,063][104569] Avg episode reward: [(0, '8020.546'), (1, '9170.347')] [2023-12-26 19:32:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000573952_146948096.pth... [2023-12-26 19:32:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000572832_146661376.pth [2023-12-26 19:32:46,086][105620] Updated weights for policy 1, policy_version 574765 (0.0007) [2023-12-26 19:32:46,146][105620] Updated weights for policy 1, policy_version 574775 (0.0006) [2023-12-26 19:32:46,203][105620] Updated weights for policy 1, policy_version 574785 (0.0008) [2023-12-26 19:32:46,241][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000574792_147161088.pth... [2023-12-26 19:32:46,246][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000573672_146874368.pth [2023-12-26 19:32:46,306][105692] Updated weights for policy 0, policy_version 573955 (0.0008) [2023-12-26 19:32:46,361][105692] Updated weights for policy 0, policy_version 573965 (0.0005) [2023-12-26 19:32:46,432][105692] Updated weights for policy 0, policy_version 573975 (0.0005) [2023-12-26 19:32:46,790][105620] Updated weights for policy 1, policy_version 574795 (0.0007) [2023-12-26 19:32:46,841][105620] Updated weights for policy 1, policy_version 574805 (0.0010) [2023-12-26 19:32:46,885][105620] Updated weights for policy 1, policy_version 574815 (0.0010) [2023-12-26 19:32:46,972][105692] Updated weights for policy 0, policy_version 573985 (0.0005) [2023-12-26 19:32:47,015][105692] Updated weights for policy 0, policy_version 573995 (0.0005) [2023-12-26 19:32:47,062][105692] Updated weights for policy 0, policy_version 574005 (0.0005) [2023-12-26 19:32:47,111][105692] Updated weights for policy 0, policy_version 574015 (0.0005) [2023-12-26 19:32:47,533][105620] Updated weights for policy 1, policy_version 574825 (0.0010) [2023-12-26 19:32:47,591][105620] Updated weights for policy 1, policy_version 574835 (0.0005) [2023-12-26 19:32:47,636][105620] Updated weights for policy 1, policy_version 574845 (0.0005) [2023-12-26 19:32:47,661][105692] Updated weights for policy 0, policy_version 574025 (0.0010) [2023-12-26 19:32:47,687][105620] Updated weights for policy 1, policy_version 574855 (0.0007) [2023-12-26 19:32:47,719][105692] Updated weights for policy 0, policy_version 574035 (0.0010) [2023-12-26 19:32:47,784][105692] Updated weights for policy 0, policy_version 574045 (0.0010) [2023-12-26 19:32:48,321][105620] Updated weights for policy 1, policy_version 574865 (0.0006) [2023-12-26 19:32:48,389][105620] Updated weights for policy 1, policy_version 574875 (0.0008) [2023-12-26 19:32:48,452][105620] Updated weights for policy 1, policy_version 574885 (0.0007) [2023-12-26 19:32:48,484][105692] Updated weights for policy 0, policy_version 574055 (0.0010) [2023-12-26 19:32:48,546][105692] Updated weights for policy 0, policy_version 574065 (0.0010) [2023-12-26 19:32:48,607][105692] Updated weights for policy 0, policy_version 574075 (0.0010) [2023-12-26 19:32:49,176][105620] Updated weights for policy 1, policy_version 574895 (0.0011) [2023-12-26 19:32:49,218][105692] Updated weights for policy 0, policy_version 574085 (0.0010) [2023-12-26 19:32:49,237][105620] Updated weights for policy 1, policy_version 574905 (0.0009) [2023-12-26 19:32:49,276][105692] Updated weights for policy 0, policy_version 574095 (0.0008) [2023-12-26 19:32:49,292][105620] Updated weights for policy 1, policy_version 574915 (0.0008) [2023-12-26 19:32:49,341][105692] Updated weights for policy 0, policy_version 574105 (0.0008) [2023-12-26 19:32:50,072][105692] Updated weights for policy 0, policy_version 574115 (0.0008) [2023-12-26 19:32:50,089][105620] Updated weights for policy 1, policy_version 574925 (0.0008) [2023-12-26 19:32:50,129][105692] Updated weights for policy 0, policy_version 574125 (0.0005) [2023-12-26 19:32:50,154][105620] Updated weights for policy 1, policy_version 574935 (0.0010) [2023-12-26 19:32:50,195][105692] Updated weights for policy 0, policy_version 574135 (0.0005) [2023-12-26 19:32:50,220][105620] Updated weights for policy 1, policy_version 574945 (0.0009) [2023-12-26 19:32:50,925][105692] Updated weights for policy 0, policy_version 574145 (0.0006) [2023-12-26 19:32:50,974][105620] Updated weights for policy 1, policy_version 574955 (0.0008) [2023-12-26 19:32:50,981][105692] Updated weights for policy 0, policy_version 574155 (0.0008) [2023-12-26 19:32:51,036][105620] Updated weights for policy 1, policy_version 574965 (0.0008) [2023-12-26 19:32:51,040][105692] Updated weights for policy 0, policy_version 574165 (0.0009) [2023-12-26 19:32:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 294199296. Throughput: 0: 9734.9, 1: 9613.2. Samples: 294198016. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:32:51,063][104569] Avg episode reward: [(0, '7943.725'), (1, '9079.899')] [2023-12-26 19:32:51,098][105620] Updated weights for policy 1, policy_version 574975 (0.0008) [2023-12-26 19:32:51,100][105692] Updated weights for policy 0, policy_version 574175 (0.0007) [2023-12-26 19:32:51,812][105620] Updated weights for policy 1, policy_version 574985 (0.0009) [2023-12-26 19:32:51,881][105620] Updated weights for policy 1, policy_version 574995 (0.0009) [2023-12-26 19:32:51,926][105692] Updated weights for policy 0, policy_version 574185 (0.0008) [2023-12-26 19:32:51,940][105620] Updated weights for policy 1, policy_version 575005 (0.0011) [2023-12-26 19:32:51,983][105692] Updated weights for policy 0, policy_version 574195 (0.0005) [2023-12-26 19:32:52,001][105620] Updated weights for policy 1, policy_version 575015 (0.0011) [2023-12-26 19:32:52,039][105692] Updated weights for policy 0, policy_version 574205 (0.0006) [2023-12-26 19:32:52,591][105620] Updated weights for policy 1, policy_version 575025 (0.0007) [2023-12-26 19:32:52,661][105620] Updated weights for policy 1, policy_version 575035 (0.0011) [2023-12-26 19:32:52,716][105620] Updated weights for policy 1, policy_version 575045 (0.0006) [2023-12-26 19:32:52,878][105692] Updated weights for policy 0, policy_version 574215 (0.0009) [2023-12-26 19:32:52,933][105692] Updated weights for policy 0, policy_version 574225 (0.0009) [2023-12-26 19:32:52,992][105692] Updated weights for policy 0, policy_version 574235 (0.0008) [2023-12-26 19:32:53,324][105620] Updated weights for policy 1, policy_version 575055 (0.0008) [2023-12-26 19:32:53,377][105620] Updated weights for policy 1, policy_version 575065 (0.0009) [2023-12-26 19:32:53,424][105620] Updated weights for policy 1, policy_version 575075 (0.0008) [2023-12-26 19:32:53,732][105692] Updated weights for policy 0, policy_version 574245 (0.0009) [2023-12-26 19:32:53,787][105692] Updated weights for policy 0, policy_version 574255 (0.0008) [2023-12-26 19:32:53,855][105692] Updated weights for policy 0, policy_version 574265 (0.0005) [2023-12-26 19:32:54,148][105620] Updated weights for policy 1, policy_version 575085 (0.0007) [2023-12-26 19:32:54,212][105620] Updated weights for policy 1, policy_version 575095 (0.0006) [2023-12-26 19:32:54,281][105620] Updated weights for policy 1, policy_version 575105 (0.0005) [2023-12-26 19:32:54,447][105692] Updated weights for policy 0, policy_version 574275 (0.0005) [2023-12-26 19:32:54,498][105692] Updated weights for policy 0, policy_version 574285 (0.0005) [2023-12-26 19:32:54,558][105692] Updated weights for policy 0, policy_version 574295 (0.0005) [2023-12-26 19:32:54,941][105620] Updated weights for policy 1, policy_version 575115 (0.0006) [2023-12-26 19:32:55,010][105620] Updated weights for policy 1, policy_version 575125 (0.0005) [2023-12-26 19:32:55,062][105620] Updated weights for policy 1, policy_version 575135 (0.0006) [2023-12-26 19:32:55,114][105692] Updated weights for policy 0, policy_version 574305 (0.0006) [2023-12-26 19:32:55,180][105692] Updated weights for policy 0, policy_version 574315 (0.0010) [2023-12-26 19:32:55,245][105692] Updated weights for policy 0, policy_version 574325 (0.0010) [2023-12-26 19:32:55,314][105692] Updated weights for policy 0, policy_version 574335 (0.0010) [2023-12-26 19:32:55,589][105620] Updated weights for policy 1, policy_version 575145 (0.0005) [2023-12-26 19:32:55,650][105620] Updated weights for policy 1, policy_version 575155 (0.0005) [2023-12-26 19:32:55,710][105620] Updated weights for policy 1, policy_version 575165 (0.0008) [2023-12-26 19:32:55,776][105620] Updated weights for policy 1, policy_version 575175 (0.0010) [2023-12-26 19:32:55,929][105692] Updated weights for policy 0, policy_version 574345 (0.0006) [2023-12-26 19:32:55,985][105692] Updated weights for policy 0, policy_version 574355 (0.0007) [2023-12-26 19:32:56,044][105692] Updated weights for policy 0, policy_version 574365 (0.0010) [2023-12-26 19:32:56,062][104569] Fps is (10 sec: 21299.9, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 294313984. Throughput: 0: 9814.6, 1: 9606.4. Samples: 294317664. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:32:56,063][104569] Avg episode reward: [(0, '9086.887'), (1, '9080.568')] [2023-12-26 19:32:56,546][105620] Updated weights for policy 1, policy_version 575185 (0.0008) [2023-12-26 19:32:56,597][105620] Updated weights for policy 1, policy_version 575195 (0.0008) [2023-12-26 19:32:56,647][105620] Updated weights for policy 1, policy_version 575205 (0.0008) [2023-12-26 19:32:56,694][105692] Updated weights for policy 0, policy_version 574375 (0.0007) [2023-12-26 19:32:56,742][105692] Updated weights for policy 0, policy_version 574385 (0.0005) [2023-12-26 19:32:56,794][105692] Updated weights for policy 0, policy_version 574395 (0.0006) [2023-12-26 19:32:57,444][105620] Updated weights for policy 1, policy_version 575215 (0.0008) [2023-12-26 19:32:57,490][105620] Updated weights for policy 1, policy_version 575225 (0.0007) [2023-12-26 19:32:57,492][105692] Updated weights for policy 0, policy_version 574405 (0.0010) [2023-12-26 19:32:57,532][105620] Updated weights for policy 1, policy_version 575235 (0.0007) [2023-12-26 19:32:57,549][105692] Updated weights for policy 0, policy_version 574415 (0.0010) [2023-12-26 19:32:57,606][105692] Updated weights for policy 0, policy_version 574425 (0.0010) [2023-12-26 19:32:58,334][105620] Updated weights for policy 1, policy_version 575245 (0.0006) [2023-12-26 19:32:58,365][105692] Updated weights for policy 0, policy_version 574435 (0.0009) [2023-12-26 19:32:58,394][105620] Updated weights for policy 1, policy_version 575255 (0.0008) [2023-12-26 19:32:58,426][105692] Updated weights for policy 0, policy_version 574445 (0.0009) [2023-12-26 19:32:58,450][105620] Updated weights for policy 1, policy_version 575265 (0.0009) [2023-12-26 19:32:58,483][105692] Updated weights for policy 0, policy_version 574455 (0.0009) [2023-12-26 19:32:59,333][105692] Updated weights for policy 0, policy_version 574465 (0.0009) [2023-12-26 19:32:59,369][105620] Updated weights for policy 1, policy_version 575275 (0.0009) [2023-12-26 19:32:59,398][105692] Updated weights for policy 0, policy_version 574475 (0.0008) [2023-12-26 19:32:59,433][105620] Updated weights for policy 1, policy_version 575285 (0.0007) [2023-12-26 19:32:59,458][105692] Updated weights for policy 0, policy_version 574485 (0.0006) [2023-12-26 19:32:59,504][105620] Updated weights for policy 1, policy_version 575295 (0.0005) [2023-12-26 19:32:59,508][105692] Updated weights for policy 0, policy_version 574495 (0.0007) [2023-12-26 19:33:00,169][105620] Updated weights for policy 1, policy_version 575305 (0.0006) [2023-12-26 19:33:00,216][105692] Updated weights for policy 0, policy_version 574505 (0.0006) [2023-12-26 19:33:00,232][105620] Updated weights for policy 1, policy_version 575315 (0.0005) [2023-12-26 19:33:00,264][105692] Updated weights for policy 0, policy_version 574515 (0.0007) [2023-12-26 19:33:00,297][105620] Updated weights for policy 1, policy_version 575325 (0.0005) [2023-12-26 19:33:00,310][105692] Updated weights for policy 0, policy_version 574525 (0.0005) [2023-12-26 19:33:00,359][105620] Updated weights for policy 1, policy_version 575335 (0.0007) [2023-12-26 19:33:00,882][105620] Updated weights for policy 1, policy_version 575345 (0.0005) [2023-12-26 19:33:00,943][105620] Updated weights for policy 1, policy_version 575355 (0.0009) [2023-12-26 19:33:01,007][105620] Updated weights for policy 1, policy_version 575365 (0.0010) [2023-12-26 19:33:01,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 294404096. Throughput: 0: 9856.4, 1: 9581.2. Samples: 294374092. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:33:01,062][104569] Avg episode reward: [(0, '9089.711'), (1, '9080.700')] [2023-12-26 19:33:01,064][105692] Updated weights for policy 0, policy_version 574535 (0.0007) [2023-12-26 19:33:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000575368_147308544.pth... [2023-12-26 19:33:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000574248_147021824.pth [2023-12-26 19:33:01,125][105692] Updated weights for policy 0, policy_version 574545 (0.0008) [2023-12-26 19:33:01,176][105692] Updated weights for policy 0, policy_version 574555 (0.0009) [2023-12-26 19:33:01,199][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000574560_147103744.pth... [2023-12-26 19:33:01,202][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000573376_146800640.pth [2023-12-26 19:33:01,754][105620] Updated weights for policy 1, policy_version 575375 (0.0009) [2023-12-26 19:33:01,808][105620] Updated weights for policy 1, policy_version 575385 (0.0007) [2023-12-26 19:33:01,861][105692] Updated weights for policy 0, policy_version 574565 (0.0007) [2023-12-26 19:33:01,871][105620] Updated weights for policy 1, policy_version 575395 (0.0007) [2023-12-26 19:33:01,916][105692] Updated weights for policy 0, policy_version 574575 (0.0006) [2023-12-26 19:33:01,959][105692] Updated weights for policy 0, policy_version 574585 (0.0005) [2023-12-26 19:33:02,534][105692] Updated weights for policy 0, policy_version 574595 (0.0007) [2023-12-26 19:33:02,557][105620] Updated weights for policy 1, policy_version 575405 (0.0008) [2023-12-26 19:33:02,600][105692] Updated weights for policy 0, policy_version 574605 (0.0006) [2023-12-26 19:33:02,613][105620] Updated weights for policy 1, policy_version 575415 (0.0005) [2023-12-26 19:33:02,664][105692] Updated weights for policy 0, policy_version 574615 (0.0007) [2023-12-26 19:33:02,673][105620] Updated weights for policy 1, policy_version 575425 (0.0007) [2023-12-26 19:33:03,328][105620] Updated weights for policy 1, policy_version 575435 (0.0007) [2023-12-26 19:33:03,361][105692] Updated weights for policy 0, policy_version 574625 (0.0006) [2023-12-26 19:33:03,373][105620] Updated weights for policy 1, policy_version 575445 (0.0005) [2023-12-26 19:33:03,422][105692] Updated weights for policy 0, policy_version 574635 (0.0006) [2023-12-26 19:33:03,439][105620] Updated weights for policy 1, policy_version 575455 (0.0005) [2023-12-26 19:33:03,477][105692] Updated weights for policy 0, policy_version 574645 (0.0010) [2023-12-26 19:33:03,532][105692] Updated weights for policy 0, policy_version 574655 (0.0010) [2023-12-26 19:33:04,071][105620] Updated weights for policy 1, policy_version 575465 (0.0005) [2023-12-26 19:33:04,145][105620] Updated weights for policy 1, policy_version 575475 (0.0007) [2023-12-26 19:33:04,206][105620] Updated weights for policy 1, policy_version 575485 (0.0007) [2023-12-26 19:33:04,211][105692] Updated weights for policy 0, policy_version 574665 (0.0008) [2023-12-26 19:33:04,262][105692] Updated weights for policy 0, policy_version 574675 (0.0008) [2023-12-26 19:33:04,271][105620] Updated weights for policy 1, policy_version 575495 (0.0007) [2023-12-26 19:33:04,324][105692] Updated weights for policy 0, policy_version 574685 (0.0008) [2023-12-26 19:33:04,944][105620] Updated weights for policy 1, policy_version 575505 (0.0009) [2023-12-26 19:33:04,996][105620] Updated weights for policy 1, policy_version 575515 (0.0008) [2023-12-26 19:33:05,043][105620] Updated weights for policy 1, policy_version 575525 (0.0009) [2023-12-26 19:33:05,081][105692] Updated weights for policy 0, policy_version 574695 (0.0008) [2023-12-26 19:33:05,129][105692] Updated weights for policy 0, policy_version 574705 (0.0005) [2023-12-26 19:33:05,195][105692] Updated weights for policy 0, policy_version 574715 (0.0007) [2023-12-26 19:33:05,761][105620] Updated weights for policy 1, policy_version 575535 (0.0008) [2023-12-26 19:33:05,807][105620] Updated weights for policy 1, policy_version 575545 (0.0008) [2023-12-26 19:33:05,856][105620] Updated weights for policy 1, policy_version 575556 (0.0009) [2023-12-26 19:33:05,933][105692] Updated weights for policy 0, policy_version 574725 (0.0009) [2023-12-26 19:33:05,984][105692] Updated weights for policy 0, policy_version 574735 (0.0008) [2023-12-26 19:33:06,013][105585] KL-divergence is very high: 325.9983 [2023-12-26 19:33:06,032][105692] Updated weights for policy 0, policy_version 574745 (0.0008) [2023-12-26 19:33:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 294502400. Throughput: 0: 9884.6, 1: 9740.8. Samples: 294494580. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:33:06,062][104569] Avg episode reward: [(0, '8446.572'), (1, '9262.801')] [2023-12-26 19:33:06,063][105585] KL-divergence is very high: 211.9019 [2023-12-26 19:33:06,639][105620] Updated weights for policy 1, policy_version 575566 (0.0010) [2023-12-26 19:33:06,688][105620] Updated weights for policy 1, policy_version 575576 (0.0010) [2023-12-26 19:33:06,736][105620] Updated weights for policy 1, policy_version 575586 (0.0009) [2023-12-26 19:33:06,858][105692] Updated weights for policy 0, policy_version 574755 (0.0009) [2023-12-26 19:33:06,912][105692] Updated weights for policy 0, policy_version 574765 (0.0009) [2023-12-26 19:33:06,972][105692] Updated weights for policy 0, policy_version 574775 (0.0008) [2023-12-26 19:33:07,463][105620] Updated weights for policy 1, policy_version 575596 (0.0009) [2023-12-26 19:33:07,521][105620] Updated weights for policy 1, policy_version 575606 (0.0010) [2023-12-26 19:33:07,579][105620] Updated weights for policy 1, policy_version 575616 (0.0010) [2023-12-26 19:33:07,679][105692] Updated weights for policy 0, policy_version 574785 (0.0007) [2023-12-26 19:33:07,739][105692] Updated weights for policy 0, policy_version 574795 (0.0007) [2023-12-26 19:33:07,787][105692] Updated weights for policy 0, policy_version 574805 (0.0008) [2023-12-26 19:33:07,846][105692] Updated weights for policy 0, policy_version 574815 (0.0008) [2023-12-26 19:33:08,320][105620] Updated weights for policy 1, policy_version 575626 (0.0010) [2023-12-26 19:33:08,384][105620] Updated weights for policy 1, policy_version 575636 (0.0010) [2023-12-26 19:33:08,440][105620] Updated weights for policy 1, policy_version 575646 (0.0010) [2023-12-26 19:33:08,502][105620] Updated weights for policy 1, policy_version 575656 (0.0010) [2023-12-26 19:33:08,560][105692] Updated weights for policy 0, policy_version 574825 (0.0008) [2023-12-26 19:33:08,612][105692] Updated weights for policy 0, policy_version 574835 (0.0010) [2023-12-26 19:33:08,674][105692] Updated weights for policy 0, policy_version 574845 (0.0011) [2023-12-26 19:33:09,282][105620] Updated weights for policy 1, policy_version 575666 (0.0010) [2023-12-26 19:33:09,352][105692] Updated weights for policy 0, policy_version 574855 (0.0010) [2023-12-26 19:33:09,352][105620] Updated weights for policy 1, policy_version 575676 (0.0006) [2023-12-26 19:33:09,415][105692] Updated weights for policy 0, policy_version 574865 (0.0008) [2023-12-26 19:33:09,425][105620] Updated weights for policy 1, policy_version 575686 (0.0008) [2023-12-26 19:33:09,476][105692] Updated weights for policy 0, policy_version 574875 (0.0008) [2023-12-26 19:33:10,126][105692] Updated weights for policy 0, policy_version 574885 (0.0009) [2023-12-26 19:33:10,168][105620] Updated weights for policy 1, policy_version 575696 (0.0007) [2023-12-26 19:33:10,186][105692] Updated weights for policy 0, policy_version 574895 (0.0009) [2023-12-26 19:33:10,236][105620] Updated weights for policy 1, policy_version 575706 (0.0006) [2023-12-26 19:33:10,241][105692] Updated weights for policy 0, policy_version 574905 (0.0009) [2023-12-26 19:33:10,297][105620] Updated weights for policy 1, policy_version 575716 (0.0006) [2023-12-26 19:33:10,990][105620] Updated weights for policy 1, policy_version 575726 (0.0005) [2023-12-26 19:33:11,058][105620] Updated weights for policy 1, policy_version 575736 (0.0008) [2023-12-26 19:33:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 294592512. Throughput: 0: 9906.6, 1: 9720.1. Samples: 294608420. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:33:11,062][104569] Avg episode reward: [(0, '5143.561'), (1, '9352.949')] [2023-12-26 19:33:11,072][105692] Updated weights for policy 0, policy_version 574915 (0.0007) [2023-12-26 19:33:11,128][105620] Updated weights for policy 1, policy_version 575746 (0.0008) [2023-12-26 19:33:11,134][105692] Updated weights for policy 0, policy_version 574925 (0.0007) [2023-12-26 19:33:11,197][105692] Updated weights for policy 0, policy_version 574935 (0.0009) [2023-12-26 19:33:11,917][105620] Updated weights for policy 1, policy_version 575756 (0.0008) [2023-12-26 19:33:11,981][105620] Updated weights for policy 1, policy_version 575766 (0.0007) [2023-12-26 19:33:11,991][105692] Updated weights for policy 0, policy_version 574945 (0.0010) [2023-12-26 19:33:12,042][105620] Updated weights for policy 1, policy_version 575776 (0.0007) [2023-12-26 19:33:12,061][105692] Updated weights for policy 0, policy_version 574955 (0.0009) [2023-12-26 19:33:12,123][105692] Updated weights for policy 0, policy_version 574965 (0.0008) [2023-12-26 19:33:12,181][105692] Updated weights for policy 0, policy_version 574975 (0.0010) [2023-12-26 19:33:12,720][105620] Updated weights for policy 1, policy_version 575786 (0.0007) [2023-12-26 19:33:12,789][105620] Updated weights for policy 1, policy_version 575796 (0.0008) [2023-12-26 19:33:12,854][105620] Updated weights for policy 1, policy_version 575806 (0.0008) [2023-12-26 19:33:12,917][105620] Updated weights for policy 1, policy_version 575816 (0.0008) [2023-12-26 19:33:12,969][105692] Updated weights for policy 0, policy_version 574985 (0.0005) [2023-12-26 19:33:13,033][105692] Updated weights for policy 0, policy_version 574995 (0.0005) [2023-12-26 19:33:13,103][105692] Updated weights for policy 0, policy_version 575005 (0.0009) [2023-12-26 19:33:13,562][105620] Updated weights for policy 1, policy_version 575826 (0.0007) [2023-12-26 19:33:13,625][105620] Updated weights for policy 1, policy_version 575836 (0.0006) [2023-12-26 19:33:13,695][105620] Updated weights for policy 1, policy_version 575846 (0.0007) [2023-12-26 19:33:13,748][105692] Updated weights for policy 0, policy_version 575015 (0.0007) [2023-12-26 19:33:13,812][105692] Updated weights for policy 0, policy_version 575025 (0.0008) [2023-12-26 19:33:13,870][105692] Updated weights for policy 0, policy_version 575035 (0.0010) [2023-12-26 19:33:14,421][105620] Updated weights for policy 1, policy_version 575856 (0.0010) [2023-12-26 19:33:14,473][105620] Updated weights for policy 1, policy_version 575866 (0.0010) [2023-12-26 19:33:14,480][105692] Updated weights for policy 0, policy_version 575045 (0.0010) [2023-12-26 19:33:14,531][105620] Updated weights for policy 1, policy_version 575876 (0.0009) [2023-12-26 19:33:14,548][105692] Updated weights for policy 0, policy_version 575055 (0.0006) [2023-12-26 19:33:14,631][105692] Updated weights for policy 0, policy_version 575065 (0.0006) [2023-12-26 19:33:15,301][105620] Updated weights for policy 1, policy_version 575886 (0.0009) [2023-12-26 19:33:15,341][105692] Updated weights for policy 0, policy_version 575075 (0.0008) [2023-12-26 19:33:15,364][105620] Updated weights for policy 1, policy_version 575896 (0.0008) [2023-12-26 19:33:15,401][105692] Updated weights for policy 0, policy_version 575085 (0.0007) [2023-12-26 19:33:15,428][105620] Updated weights for policy 1, policy_version 575906 (0.0008) [2023-12-26 19:33:15,447][105692] Updated weights for policy 0, policy_version 575095 (0.0006) [2023-12-26 19:33:16,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19387.6, 300 sec: 19549.7). Total num frames: 294690816. Throughput: 0: 9939.4, 1: 9614.7. Samples: 294664892. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:33:16,063][104569] Avg episode reward: [(0, '6554.476'), (1, '9262.614')] [2023-12-26 19:33:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000575912_147447808.pth... [2023-12-26 19:33:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000575104_147243008.pth... [2023-12-26 19:33:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000574792_147161088.pth [2023-12-26 19:33:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000573952_146948096.pth [2023-12-26 19:33:16,097][105692] Updated weights for policy 0, policy_version 575105 (0.0008) [2023-12-26 19:33:16,128][105620] Updated weights for policy 1, policy_version 575916 (0.0008) [2023-12-26 19:33:16,159][105692] Updated weights for policy 0, policy_version 575115 (0.0010) [2023-12-26 19:33:16,188][105620] Updated weights for policy 1, policy_version 575926 (0.0006) [2023-12-26 19:33:16,214][105692] Updated weights for policy 0, policy_version 575125 (0.0009) [2023-12-26 19:33:16,249][105620] Updated weights for policy 1, policy_version 575936 (0.0006) [2023-12-26 19:33:16,278][105692] Updated weights for policy 0, policy_version 575135 (0.0009) [2023-12-26 19:33:16,838][105620] Updated weights for policy 1, policy_version 575946 (0.0008) [2023-12-26 19:33:16,905][105620] Updated weights for policy 1, policy_version 575956 (0.0009) [2023-12-26 19:33:16,971][105620] Updated weights for policy 1, policy_version 575966 (0.0007) [2023-12-26 19:33:17,029][105692] Updated weights for policy 0, policy_version 575145 (0.0008) [2023-12-26 19:33:17,090][105692] Updated weights for policy 0, policy_version 575155 (0.0010) [2023-12-26 19:33:17,151][105692] Updated weights for policy 0, policy_version 575165 (0.0010) [2023-12-26 19:33:17,687][105620] Updated weights for policy 1, policy_version 575977 (0.0009) [2023-12-26 19:33:17,745][105620] Updated weights for policy 1, policy_version 575987 (0.0009) [2023-12-26 19:33:17,805][105620] Updated weights for policy 1, policy_version 575997 (0.0010) [2023-12-26 19:33:17,827][105692] Updated weights for policy 0, policy_version 575175 (0.0007) [2023-12-26 19:33:17,860][105620] Updated weights for policy 1, policy_version 576007 (0.0008) [2023-12-26 19:33:17,875][105692] Updated weights for policy 0, policy_version 575185 (0.0005) [2023-12-26 19:33:17,921][105692] Updated weights for policy 0, policy_version 575195 (0.0005) [2023-12-26 19:33:18,462][105620] Updated weights for policy 1, policy_version 576017 (0.0008) [2023-12-26 19:33:18,516][105620] Updated weights for policy 1, policy_version 576027 (0.0009) [2023-12-26 19:33:18,573][105692] Updated weights for policy 0, policy_version 575205 (0.0006) [2023-12-26 19:33:18,575][105620] Updated weights for policy 1, policy_version 576037 (0.0009) [2023-12-26 19:33:18,636][105692] Updated weights for policy 0, policy_version 575215 (0.0008) [2023-12-26 19:33:18,692][105692] Updated weights for policy 0, policy_version 575225 (0.0009) [2023-12-26 19:33:19,344][105620] Updated weights for policy 1, policy_version 576047 (0.0008) [2023-12-26 19:33:19,415][105620] Updated weights for policy 1, policy_version 576057 (0.0006) [2023-12-26 19:33:19,459][105692] Updated weights for policy 0, policy_version 575235 (0.0009) [2023-12-26 19:33:19,475][105620] Updated weights for policy 1, policy_version 576067 (0.0010) [2023-12-26 19:33:19,517][105692] Updated weights for policy 0, policy_version 575245 (0.0009) [2023-12-26 19:33:19,574][105692] Updated weights for policy 0, policy_version 575255 (0.0008) [2023-12-26 19:33:20,155][105620] Updated weights for policy 1, policy_version 576077 (0.0006) [2023-12-26 19:33:20,213][105620] Updated weights for policy 1, policy_version 576087 (0.0006) [2023-12-26 19:33:20,273][105620] Updated weights for policy 1, policy_version 576097 (0.0008) [2023-12-26 19:33:20,460][105692] Updated weights for policy 0, policy_version 575265 (0.0009) [2023-12-26 19:33:20,525][105692] Updated weights for policy 0, policy_version 575275 (0.0008) [2023-12-26 19:33:20,589][105692] Updated weights for policy 0, policy_version 575285 (0.0008) [2023-12-26 19:33:20,647][105692] Updated weights for policy 0, policy_version 575295 (0.0010) [2023-12-26 19:33:20,954][105620] Updated weights for policy 1, policy_version 576107 (0.0009) [2023-12-26 19:33:21,018][105620] Updated weights for policy 1, policy_version 576117 (0.0007) [2023-12-26 19:33:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 294789120. Throughput: 0: 9863.3, 1: 9736.7. Samples: 294783840. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:33:21,063][104569] Avg episode reward: [(0, '8764.669'), (1, '9080.250')] [2023-12-26 19:33:21,082][105620] Updated weights for policy 1, policy_version 576127 (0.0009) [2023-12-26 19:33:21,459][105692] Updated weights for policy 0, policy_version 575305 (0.0006) [2023-12-26 19:33:21,505][105692] Updated weights for policy 0, policy_version 575315 (0.0005) [2023-12-26 19:33:21,561][105692] Updated weights for policy 0, policy_version 575325 (0.0007) [2023-12-26 19:33:21,827][105620] Updated weights for policy 1, policy_version 576137 (0.0009) [2023-12-26 19:33:21,888][105620] Updated weights for policy 1, policy_version 576147 (0.0007) [2023-12-26 19:33:21,946][105620] Updated weights for policy 1, policy_version 576157 (0.0005) [2023-12-26 19:33:22,004][105620] Updated weights for policy 1, policy_version 576167 (0.0006) [2023-12-26 19:33:22,386][105692] Updated weights for policy 0, policy_version 575335 (0.0007) [2023-12-26 19:33:22,452][105692] Updated weights for policy 0, policy_version 575345 (0.0009) [2023-12-26 19:33:22,512][105692] Updated weights for policy 0, policy_version 575355 (0.0009) [2023-12-26 19:33:22,660][105620] Updated weights for policy 1, policy_version 576177 (0.0009) [2023-12-26 19:33:22,725][105620] Updated weights for policy 1, policy_version 576187 (0.0008) [2023-12-26 19:33:22,786][105620] Updated weights for policy 1, policy_version 576197 (0.0009) [2023-12-26 19:33:23,238][105692] Updated weights for policy 0, policy_version 575365 (0.0009) [2023-12-26 19:33:23,299][105692] Updated weights for policy 0, policy_version 575375 (0.0009) [2023-12-26 19:33:23,361][105692] Updated weights for policy 0, policy_version 575385 (0.0009) [2023-12-26 19:33:23,513][105620] Updated weights for policy 1, policy_version 576207 (0.0006) [2023-12-26 19:33:23,568][105620] Updated weights for policy 1, policy_version 576217 (0.0005) [2023-12-26 19:33:23,629][105620] Updated weights for policy 1, policy_version 576227 (0.0005) [2023-12-26 19:33:24,134][105692] Updated weights for policy 0, policy_version 575395 (0.0008) [2023-12-26 19:33:24,189][105692] Updated weights for policy 0, policy_version 575405 (0.0006) [2023-12-26 19:33:24,237][105692] Updated weights for policy 0, policy_version 575415 (0.0008) [2023-12-26 19:33:24,340][105620] Updated weights for policy 1, policy_version 576237 (0.0009) [2023-12-26 19:33:24,398][105620] Updated weights for policy 1, policy_version 576247 (0.0009) [2023-12-26 19:33:24,447][105620] Updated weights for policy 1, policy_version 576257 (0.0008) [2023-12-26 19:33:25,008][105692] Updated weights for policy 0, policy_version 575425 (0.0008) [2023-12-26 19:33:25,042][105620] Updated weights for policy 1, policy_version 576267 (0.0009) [2023-12-26 19:33:25,060][105692] Updated weights for policy 0, policy_version 575435 (0.0005) [2023-12-26 19:33:25,102][105620] Updated weights for policy 1, policy_version 576277 (0.0009) [2023-12-26 19:33:25,116][105692] Updated weights for policy 0, policy_version 575445 (0.0010) [2023-12-26 19:33:25,158][105620] Updated weights for policy 1, policy_version 576287 (0.0011) [2023-12-26 19:33:25,177][105692] Updated weights for policy 0, policy_version 575455 (0.0010) [2023-12-26 19:33:25,733][105620] Updated weights for policy 1, policy_version 576297 (0.0010) [2023-12-26 19:33:25,745][105692] Updated weights for policy 0, policy_version 575465 (0.0007) [2023-12-26 19:33:25,793][105620] Updated weights for policy 1, policy_version 576307 (0.0009) [2023-12-26 19:33:25,806][105692] Updated weights for policy 0, policy_version 575475 (0.0005) [2023-12-26 19:33:25,856][105620] Updated weights for policy 1, policy_version 576317 (0.0008) [2023-12-26 19:33:25,862][105692] Updated weights for policy 0, policy_version 575485 (0.0005) [2023-12-26 19:33:25,915][105620] Updated weights for policy 1, policy_version 576327 (0.0009) [2023-12-26 19:33:26,062][104569] Fps is (10 sec: 20480.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 294895616. Throughput: 0: 9799.0, 1: 9858.9. Samples: 294899760. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:33:26,062][104569] Avg episode reward: [(0, '8605.600'), (1, '9079.367')] [2023-12-26 19:33:26,436][105692] Updated weights for policy 0, policy_version 575495 (0.0008) [2023-12-26 19:33:26,506][105692] Updated weights for policy 0, policy_version 575505 (0.0008) [2023-12-26 19:33:26,558][105692] Updated weights for policy 0, policy_version 575515 (0.0007) [2023-12-26 19:33:26,616][105620] Updated weights for policy 1, policy_version 576337 (0.0006) [2023-12-26 19:33:26,666][105620] Updated weights for policy 1, policy_version 576347 (0.0006) [2023-12-26 19:33:26,713][105620] Updated weights for policy 1, policy_version 576357 (0.0007) [2023-12-26 19:33:27,271][105620] Updated weights for policy 1, policy_version 576367 (0.0007) [2023-12-26 19:33:27,322][105692] Updated weights for policy 0, policy_version 575525 (0.0009) [2023-12-26 19:33:27,325][105620] Updated weights for policy 1, policy_version 576377 (0.0006) [2023-12-26 19:33:27,370][105620] Updated weights for policy 1, policy_version 576387 (0.0005) [2023-12-26 19:33:27,372][105692] Updated weights for policy 0, policy_version 575535 (0.0007) [2023-12-26 19:33:27,421][105692] Updated weights for policy 0, policy_version 575545 (0.0008) [2023-12-26 19:33:28,036][105620] Updated weights for policy 1, policy_version 576397 (0.0007) [2023-12-26 19:33:28,087][105620] Updated weights for policy 1, policy_version 576407 (0.0009) [2023-12-26 19:33:28,096][105692] Updated weights for policy 0, policy_version 575555 (0.0008) [2023-12-26 19:33:28,133][105620] Updated weights for policy 1, policy_version 576417 (0.0007) [2023-12-26 19:33:28,147][105692] Updated weights for policy 0, policy_version 575565 (0.0006) [2023-12-26 19:33:28,194][105692] Updated weights for policy 0, policy_version 575575 (0.0008) [2023-12-26 19:33:28,859][105620] Updated weights for policy 1, policy_version 576427 (0.0006) [2023-12-26 19:33:28,883][105692] Updated weights for policy 0, policy_version 575585 (0.0009) [2023-12-26 19:33:28,915][105620] Updated weights for policy 1, policy_version 576437 (0.0007) [2023-12-26 19:33:28,934][105692] Updated weights for policy 0, policy_version 575595 (0.0010) [2023-12-26 19:33:28,968][105620] Updated weights for policy 1, policy_version 576447 (0.0006) [2023-12-26 19:33:28,982][105692] Updated weights for policy 0, policy_version 575605 (0.0010) [2023-12-26 19:33:29,020][105585] KL-divergence is very high: 101.9880 [2023-12-26 19:33:29,030][105692] Updated weights for policy 0, policy_version 575615 (0.0010) [2023-12-26 19:33:29,628][105620] Updated weights for policy 1, policy_version 576457 (0.0009) [2023-12-26 19:33:29,697][105620] Updated weights for policy 1, policy_version 576467 (0.0006) [2023-12-26 19:33:29,753][105620] Updated weights for policy 1, policy_version 576477 (0.0005) [2023-12-26 19:33:29,789][105585] KL-divergence is very high: 264.6546 [2023-12-26 19:33:29,801][105620] Updated weights for policy 1, policy_version 576487 (0.0005) [2023-12-26 19:33:29,818][105692] Updated weights for policy 0, policy_version 575625 (0.0009) [2023-12-26 19:33:29,832][105585] KL-divergence is very high: 338.2933 [2023-12-26 19:33:29,879][105585] KL-divergence is very high: 341.0422 [2023-12-26 19:33:29,879][105692] Updated weights for policy 0, policy_version 575635 (0.0008) [2023-12-26 19:33:29,929][105585] KL-divergence is very high: 302.1843 [2023-12-26 19:33:29,942][105692] Updated weights for policy 0, policy_version 575645 (0.0009) [2023-12-26 19:33:30,399][105620] Updated weights for policy 1, policy_version 576497 (0.0006) [2023-12-26 19:33:30,454][105620] Updated weights for policy 1, policy_version 576507 (0.0010) [2023-12-26 19:33:30,505][105620] Updated weights for policy 1, policy_version 576517 (0.0010) [2023-12-26 19:33:30,702][105692] Updated weights for policy 0, policy_version 575656 (0.0006) [2023-12-26 19:33:30,756][105692] Updated weights for policy 0, policy_version 575666 (0.0009) [2023-12-26 19:33:30,805][105692] Updated weights for policy 0, policy_version 575676 (0.0009) [2023-12-26 19:33:31,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 294993920. Throughput: 0: 9847.0, 1: 9930.2. Samples: 294962996. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:33:31,062][104569] Avg episode reward: [(0, '8077.737'), (1, '9262.374')] [2023-12-26 19:33:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000575680_147390464.pth... [2023-12-26 19:33:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000574560_147103744.pth [2023-12-26 19:33:31,088][105620] Updated weights for policy 1, policy_version 576527 (0.0008) [2023-12-26 19:33:31,149][105620] Updated weights for policy 1, policy_version 576537 (0.0007) [2023-12-26 19:33:31,204][105620] Updated weights for policy 1, policy_version 576547 (0.0007) [2023-12-26 19:33:31,225][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000576552_147611648.pth... [2023-12-26 19:33:31,230][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000575368_147308544.pth [2023-12-26 19:33:31,564][105692] Updated weights for policy 0, policy_version 575686 (0.0010) [2023-12-26 19:33:31,627][105692] Updated weights for policy 0, policy_version 575696 (0.0010) [2023-12-26 19:33:31,692][105692] Updated weights for policy 0, policy_version 575706 (0.0007) [2023-12-26 19:33:31,817][105620] Updated weights for policy 1, policy_version 576557 (0.0008) [2023-12-26 19:33:31,882][105620] Updated weights for policy 1, policy_version 576567 (0.0009) [2023-12-26 19:33:31,944][105620] Updated weights for policy 1, policy_version 576577 (0.0010) [2023-12-26 19:33:32,422][105692] Updated weights for policy 0, policy_version 575716 (0.0009) [2023-12-26 19:33:32,476][105692] Updated weights for policy 0, policy_version 575726 (0.0008) [2023-12-26 19:33:32,542][105692] Updated weights for policy 0, policy_version 575736 (0.0005) [2023-12-26 19:33:32,661][105620] Updated weights for policy 1, policy_version 576587 (0.0010) [2023-12-26 19:33:32,713][105620] Updated weights for policy 1, policy_version 576597 (0.0010) [2023-12-26 19:33:32,774][105620] Updated weights for policy 1, policy_version 576607 (0.0010) [2023-12-26 19:33:33,110][105692] Updated weights for policy 0, policy_version 575746 (0.0006) [2023-12-26 19:33:33,163][105692] Updated weights for policy 0, policy_version 575756 (0.0008) [2023-12-26 19:33:33,222][105692] Updated weights for policy 0, policy_version 575766 (0.0008) [2023-12-26 19:33:33,289][105692] Updated weights for policy 0, policy_version 575776 (0.0006) [2023-12-26 19:33:33,454][105620] Updated weights for policy 1, policy_version 576617 (0.0010) [2023-12-26 19:33:33,513][105620] Updated weights for policy 1, policy_version 576627 (0.0010) [2023-12-26 19:33:33,562][105620] Updated weights for policy 1, policy_version 576637 (0.0006) [2023-12-26 19:33:33,607][105620] Updated weights for policy 1, policy_version 576647 (0.0005) [2023-12-26 19:33:33,892][105692] Updated weights for policy 0, policy_version 575786 (0.0005) [2023-12-26 19:33:33,960][105692] Updated weights for policy 0, policy_version 575796 (0.0005) [2023-12-26 19:33:34,023][105692] Updated weights for policy 0, policy_version 575806 (0.0005) [2023-12-26 19:33:34,172][105620] Updated weights for policy 1, policy_version 576657 (0.0009) [2023-12-26 19:33:34,244][105620] Updated weights for policy 1, policy_version 576667 (0.0011) [2023-12-26 19:33:34,296][105620] Updated weights for policy 1, policy_version 576677 (0.0010) [2023-12-26 19:33:34,590][105692] Updated weights for policy 0, policy_version 575816 (0.0009) [2023-12-26 19:33:34,626][105585] KL-divergence is very high: 283.6891 [2023-12-26 19:33:34,652][105692] Updated weights for policy 0, policy_version 575826 (0.0010) [2023-12-26 19:33:34,675][105585] KL-divergence is very high: 393.9981 [2023-12-26 19:33:34,712][105692] Updated weights for policy 0, policy_version 575836 (0.0010) [2023-12-26 19:33:34,724][105585] KL-divergence is very high: 304.3895 [2023-12-26 19:33:35,047][105620] Updated weights for policy 1, policy_version 576687 (0.0010) [2023-12-26 19:33:35,104][105620] Updated weights for policy 1, policy_version 576697 (0.0010) [2023-12-26 19:33:35,165][105620] Updated weights for policy 1, policy_version 576707 (0.0010) [2023-12-26 19:33:35,458][105692] Updated weights for policy 0, policy_version 575846 (0.0010) [2023-12-26 19:33:35,523][105692] Updated weights for policy 0, policy_version 575856 (0.0010) [2023-12-26 19:33:35,588][105692] Updated weights for policy 0, policy_version 575866 (0.0010) [2023-12-26 19:33:35,764][105620] Updated weights for policy 1, policy_version 576717 (0.0008) [2023-12-26 19:33:35,816][105620] Updated weights for policy 1, policy_version 576727 (0.0009) [2023-12-26 19:33:35,864][105620] Updated weights for policy 1, policy_version 576737 (0.0011) [2023-12-26 19:33:36,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19797.3, 300 sec: 19605.2). Total num frames: 295100416. Throughput: 0: 9741.4, 1: 10021.9. Samples: 295087368. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:33:36,063][104569] Avg episode reward: [(0, '8071.673'), (1, '9262.594')] [2023-12-26 19:33:36,209][105692] Updated weights for policy 0, policy_version 575876 (0.0007) [2023-12-26 19:33:36,261][105692] Updated weights for policy 0, policy_version 575886 (0.0005) [2023-12-26 19:33:36,314][105692] Updated weights for policy 0, policy_version 575896 (0.0010) [2023-12-26 19:33:36,626][105620] Updated weights for policy 1, policy_version 576747 (0.0010) [2023-12-26 19:33:36,681][105620] Updated weights for policy 1, policy_version 576757 (0.0010) [2023-12-26 19:33:36,736][105620] Updated weights for policy 1, policy_version 576767 (0.0010) [2023-12-26 19:33:36,963][105692] Updated weights for policy 0, policy_version 575906 (0.0009) [2023-12-26 19:33:37,009][105692] Updated weights for policy 0, policy_version 575916 (0.0005) [2023-12-26 19:33:37,055][105692] Updated weights for policy 0, policy_version 575926 (0.0006) [2023-12-26 19:33:37,110][105692] Updated weights for policy 0, policy_version 575936 (0.0010) [2023-12-26 19:33:37,471][105620] Updated weights for policy 1, policy_version 576777 (0.0010) [2023-12-26 19:33:37,538][105620] Updated weights for policy 1, policy_version 576787 (0.0005) [2023-12-26 19:33:37,591][105620] Updated weights for policy 1, policy_version 576797 (0.0009) [2023-12-26 19:33:37,645][105620] Updated weights for policy 1, policy_version 576807 (0.0010) [2023-12-26 19:33:37,721][105692] Updated weights for policy 0, policy_version 575946 (0.0006) [2023-12-26 19:33:37,787][105692] Updated weights for policy 0, policy_version 575956 (0.0005) [2023-12-26 19:33:37,853][105692] Updated weights for policy 0, policy_version 575966 (0.0006) [2023-12-26 19:33:38,367][105620] Updated weights for policy 1, policy_version 576817 (0.0011) [2023-12-26 19:33:38,426][105620] Updated weights for policy 1, policy_version 576827 (0.0010) [2023-12-26 19:33:38,485][105620] Updated weights for policy 1, policy_version 576837 (0.0011) [2023-12-26 19:33:38,528][105692] Updated weights for policy 0, policy_version 575976 (0.0011) [2023-12-26 19:33:38,598][105692] Updated weights for policy 0, policy_version 575986 (0.0011) [2023-12-26 19:33:38,670][105692] Updated weights for policy 0, policy_version 575996 (0.0011) [2023-12-26 19:33:39,165][105620] Updated weights for policy 1, policy_version 576847 (0.0006) [2023-12-26 19:33:39,213][105620] Updated weights for policy 1, policy_version 576857 (0.0007) [2023-12-26 19:33:39,284][105620] Updated weights for policy 1, policy_version 576867 (0.0008) [2023-12-26 19:33:39,341][105692] Updated weights for policy 0, policy_version 576006 (0.0011) [2023-12-26 19:33:39,413][105692] Updated weights for policy 0, policy_version 576016 (0.0011) [2023-12-26 19:33:39,465][105692] Updated weights for policy 0, policy_version 576026 (0.0008) [2023-12-26 19:33:40,041][105620] Updated weights for policy 1, policy_version 576877 (0.0008) [2023-12-26 19:33:40,100][105620] Updated weights for policy 1, policy_version 576887 (0.0011) [2023-12-26 19:33:40,163][105620] Updated weights for policy 1, policy_version 576897 (0.0011) [2023-12-26 19:33:40,180][105692] Updated weights for policy 0, policy_version 576036 (0.0008) [2023-12-26 19:33:40,248][105692] Updated weights for policy 0, policy_version 576046 (0.0008) [2023-12-26 19:33:40,308][105692] Updated weights for policy 0, policy_version 576056 (0.0008) [2023-12-26 19:33:40,922][105692] Updated weights for policy 0, policy_version 576066 (0.0007) [2023-12-26 19:33:40,935][105620] Updated weights for policy 1, policy_version 576907 (0.0011) [2023-12-26 19:33:40,984][105692] Updated weights for policy 0, policy_version 576076 (0.0008) [2023-12-26 19:33:40,987][105620] Updated weights for policy 1, policy_version 576917 (0.0010) [2023-12-26 19:33:41,049][105620] Updated weights for policy 1, policy_version 576927 (0.0011) [2023-12-26 19:33:41,053][105692] Updated weights for policy 0, policy_version 576086 (0.0008) [2023-12-26 19:33:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 295190528. Throughput: 0: 9794.6, 1: 9958.3. Samples: 295206544. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:33:41,063][104569] Avg episode reward: [(0, '8495.418'), (1, '9079.790')] [2023-12-26 19:33:41,125][105692] Updated weights for policy 0, policy_version 576096 (0.0009) [2023-12-26 19:33:41,829][105692] Updated weights for policy 0, policy_version 576106 (0.0008) [2023-12-26 19:33:41,840][105620] Updated weights for policy 1, policy_version 576937 (0.0011) [2023-12-26 19:33:41,890][105620] Updated weights for policy 1, policy_version 576947 (0.0011) [2023-12-26 19:33:41,892][105692] Updated weights for policy 0, policy_version 576116 (0.0008) [2023-12-26 19:33:41,939][105620] Updated weights for policy 1, policy_version 576957 (0.0010) [2023-12-26 19:33:41,951][105692] Updated weights for policy 0, policy_version 576126 (0.0006) [2023-12-26 19:33:41,992][105620] Updated weights for policy 1, policy_version 576967 (0.0010) [2023-12-26 19:33:42,624][105692] Updated weights for policy 0, policy_version 576136 (0.0006) [2023-12-26 19:33:42,688][105692] Updated weights for policy 0, policy_version 576146 (0.0005) [2023-12-26 19:33:42,751][105692] Updated weights for policy 0, policy_version 576156 (0.0005) [2023-12-26 19:33:42,786][105620] Updated weights for policy 1, policy_version 576977 (0.0011) [2023-12-26 19:33:42,855][105620] Updated weights for policy 1, policy_version 576987 (0.0010) [2023-12-26 19:33:42,914][105620] Updated weights for policy 1, policy_version 576997 (0.0011) [2023-12-26 19:33:43,360][105692] Updated weights for policy 0, policy_version 576166 (0.0007) [2023-12-26 19:33:43,408][105692] Updated weights for policy 0, policy_version 576176 (0.0008) [2023-12-26 19:33:43,452][105692] Updated weights for policy 0, policy_version 576186 (0.0008) [2023-12-26 19:33:43,647][105620] Updated weights for policy 1, policy_version 577007 (0.0011) [2023-12-26 19:33:43,705][105620] Updated weights for policy 1, policy_version 577017 (0.0010) [2023-12-26 19:33:43,770][105620] Updated weights for policy 1, policy_version 577027 (0.0010) [2023-12-26 19:33:44,142][105692] Updated weights for policy 0, policy_version 576196 (0.0007) [2023-12-26 19:33:44,188][105692] Updated weights for policy 0, policy_version 576206 (0.0005) [2023-12-26 19:33:44,243][105692] Updated weights for policy 0, policy_version 576216 (0.0005) [2023-12-26 19:33:44,440][105620] Updated weights for policy 1, policy_version 577037 (0.0008) [2023-12-26 19:33:44,505][105620] Updated weights for policy 1, policy_version 577047 (0.0006) [2023-12-26 19:33:44,556][105620] Updated weights for policy 1, policy_version 577057 (0.0010) [2023-12-26 19:33:44,958][105692] Updated weights for policy 0, policy_version 576226 (0.0009) [2023-12-26 19:33:45,020][105692] Updated weights for policy 0, policy_version 576236 (0.0010) [2023-12-26 19:33:45,078][105692] Updated weights for policy 0, policy_version 576246 (0.0008) [2023-12-26 19:33:45,128][105692] Updated weights for policy 0, policy_version 576256 (0.0008) [2023-12-26 19:33:45,300][105620] Updated weights for policy 1, policy_version 577067 (0.0010) [2023-12-26 19:33:45,364][105620] Updated weights for policy 1, policy_version 577077 (0.0009) [2023-12-26 19:33:45,428][105620] Updated weights for policy 1, policy_version 577087 (0.0010) [2023-12-26 19:33:45,841][105692] Updated weights for policy 0, policy_version 576266 (0.0005) [2023-12-26 19:33:45,898][105692] Updated weights for policy 0, policy_version 576276 (0.0006) [2023-12-26 19:33:45,962][105692] Updated weights for policy 0, policy_version 576286 (0.0005) [2023-12-26 19:33:46,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19934.0, 300 sec: 19577.5). Total num frames: 295297024. Throughput: 0: 9819.8, 1: 9971.4. Samples: 295264696. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:33:46,062][104569] Avg episode reward: [(0, '8918.214'), (1, '8989.547')] [2023-12-26 19:33:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000576288_147546112.pth... [2023-12-26 19:33:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000577096_147750912.pth... [2023-12-26 19:33:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000575104_147243008.pth [2023-12-26 19:33:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000575912_147447808.pth [2023-12-26 19:33:46,181][105620] Updated weights for policy 1, policy_version 577097 (0.0011) [2023-12-26 19:33:46,225][105620] Updated weights for policy 1, policy_version 577107 (0.0010) [2023-12-26 19:33:46,270][105620] Updated weights for policy 1, policy_version 577117 (0.0010) [2023-12-26 19:33:46,322][105620] Updated weights for policy 1, policy_version 577127 (0.0010) [2023-12-26 19:33:46,612][105692] Updated weights for policy 0, policy_version 576296 (0.0009) [2023-12-26 19:33:46,669][105692] Updated weights for policy 0, policy_version 576306 (0.0009) [2023-12-26 19:33:46,727][105692] Updated weights for policy 0, policy_version 576316 (0.0008) [2023-12-26 19:33:47,093][105620] Updated weights for policy 1, policy_version 577137 (0.0010) [2023-12-26 19:33:47,137][105620] Updated weights for policy 1, policy_version 577147 (0.0010) [2023-12-26 19:33:47,192][105620] Updated weights for policy 1, policy_version 577157 (0.0010) [2023-12-26 19:33:47,295][105692] Updated weights for policy 0, policy_version 576326 (0.0005) [2023-12-26 19:33:47,350][105692] Updated weights for policy 0, policy_version 576336 (0.0005) [2023-12-26 19:33:47,401][105692] Updated weights for policy 0, policy_version 576346 (0.0005) [2023-12-26 19:33:47,963][105620] Updated weights for policy 1, policy_version 577167 (0.0010) [2023-12-26 19:33:47,973][105692] Updated weights for policy 0, policy_version 576356 (0.0005) [2023-12-26 19:33:48,018][105620] Updated weights for policy 1, policy_version 577177 (0.0008) [2023-12-26 19:33:48,025][105692] Updated weights for policy 0, policy_version 576366 (0.0006) [2023-12-26 19:33:48,072][105620] Updated weights for policy 1, policy_version 577187 (0.0005) [2023-12-26 19:33:48,073][105692] Updated weights for policy 0, policy_version 576376 (0.0008) [2023-12-26 19:33:48,755][105620] Updated weights for policy 1, policy_version 577197 (0.0008) [2023-12-26 19:33:48,802][105692] Updated weights for policy 0, policy_version 576386 (0.0009) [2023-12-26 19:33:48,822][105620] Updated weights for policy 1, policy_version 577207 (0.0009) [2023-12-26 19:33:48,863][105692] Updated weights for policy 0, policy_version 576396 (0.0007) [2023-12-26 19:33:48,887][105620] Updated weights for policy 1, policy_version 577217 (0.0010) [2023-12-26 19:33:48,921][105692] Updated weights for policy 0, policy_version 576406 (0.0007) [2023-12-26 19:33:48,975][105692] Updated weights for policy 0, policy_version 576416 (0.0005) [2023-12-26 19:33:49,578][105620] Updated weights for policy 1, policy_version 577227 (0.0011) [2023-12-26 19:33:49,608][105692] Updated weights for policy 0, policy_version 576426 (0.0006) [2023-12-26 19:33:49,637][105620] Updated weights for policy 1, policy_version 577237 (0.0010) [2023-12-26 19:33:49,666][105692] Updated weights for policy 0, policy_version 576436 (0.0005) [2023-12-26 19:33:49,702][105620] Updated weights for policy 1, policy_version 577247 (0.0011) [2023-12-26 19:33:49,722][105692] Updated weights for policy 0, policy_version 576446 (0.0008) [2023-12-26 19:33:50,446][105620] Updated weights for policy 1, policy_version 577257 (0.0011) [2023-12-26 19:33:50,488][105692] Updated weights for policy 0, policy_version 576456 (0.0006) [2023-12-26 19:33:50,502][105620] Updated weights for policy 1, policy_version 577267 (0.0011) [2023-12-26 19:33:50,553][105692] Updated weights for policy 0, policy_version 576466 (0.0006) [2023-12-26 19:33:50,560][105620] Updated weights for policy 1, policy_version 577277 (0.0011) [2023-12-26 19:33:50,617][105692] Updated weights for policy 0, policy_version 576476 (0.0007) [2023-12-26 19:33:50,625][105620] Updated weights for policy 1, policy_version 577287 (0.0010) [2023-12-26 19:33:51,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 295395328. Throughput: 0: 9921.8, 1: 9885.4. Samples: 295385904. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:33:51,062][104569] Avg episode reward: [(0, '9089.456'), (1, '9170.500')] [2023-12-26 19:33:51,282][105692] Updated weights for policy 0, policy_version 576486 (0.0006) [2023-12-26 19:33:51,359][105692] Updated weights for policy 0, policy_version 576496 (0.0008) [2023-12-26 19:33:51,421][105620] Updated weights for policy 1, policy_version 577297 (0.0008) [2023-12-26 19:33:51,422][105692] Updated weights for policy 0, policy_version 576506 (0.0007) [2023-12-26 19:33:51,485][105620] Updated weights for policy 1, policy_version 577307 (0.0006) [2023-12-26 19:33:51,548][105620] Updated weights for policy 1, policy_version 577317 (0.0006) [2023-12-26 19:33:52,152][105692] Updated weights for policy 0, policy_version 576516 (0.0009) [2023-12-26 19:33:52,219][105692] Updated weights for policy 0, policy_version 576526 (0.0010) [2023-12-26 19:33:52,252][105620] Updated weights for policy 1, policy_version 577327 (0.0007) [2023-12-26 19:33:52,287][105692] Updated weights for policy 0, policy_version 576536 (0.0008) [2023-12-26 19:33:52,320][105620] Updated weights for policy 1, policy_version 577337 (0.0011) [2023-12-26 19:33:52,386][105620] Updated weights for policy 1, policy_version 577347 (0.0010) [2023-12-26 19:33:53,052][105692] Updated weights for policy 0, policy_version 576546 (0.0006) [2023-12-26 19:33:53,066][105620] Updated weights for policy 1, policy_version 577357 (0.0010) [2023-12-26 19:33:53,110][105692] Updated weights for policy 0, policy_version 576556 (0.0006) [2023-12-26 19:33:53,127][105620] Updated weights for policy 1, policy_version 577367 (0.0009) [2023-12-26 19:33:53,169][105692] Updated weights for policy 0, policy_version 576566 (0.0007) [2023-12-26 19:33:53,187][105620] Updated weights for policy 1, policy_version 577377 (0.0007) [2023-12-26 19:33:53,225][105692] Updated weights for policy 0, policy_version 576576 (0.0008) [2023-12-26 19:33:53,935][105620] Updated weights for policy 1, policy_version 577387 (0.0007) [2023-12-26 19:33:53,960][105692] Updated weights for policy 0, policy_version 576586 (0.0008) [2023-12-26 19:33:53,999][105620] Updated weights for policy 1, policy_version 577397 (0.0007) [2023-12-26 19:33:54,011][105692] Updated weights for policy 0, policy_version 576596 (0.0005) [2023-12-26 19:33:54,068][105620] Updated weights for policy 1, policy_version 577407 (0.0008) [2023-12-26 19:33:54,069][105692] Updated weights for policy 0, policy_version 576606 (0.0005) [2023-12-26 19:33:54,674][105692] Updated weights for policy 0, policy_version 576616 (0.0008) [2023-12-26 19:33:54,729][105692] Updated weights for policy 0, policy_version 576626 (0.0008) [2023-12-26 19:33:54,782][105692] Updated weights for policy 0, policy_version 576636 (0.0009) [2023-12-26 19:33:54,827][105620] Updated weights for policy 1, policy_version 577417 (0.0010) [2023-12-26 19:33:54,882][105620] Updated weights for policy 1, policy_version 577427 (0.0010) [2023-12-26 19:33:54,930][105620] Updated weights for policy 1, policy_version 577437 (0.0010) [2023-12-26 19:33:54,987][105620] Updated weights for policy 1, policy_version 577447 (0.0006) [2023-12-26 19:33:55,467][105692] Updated weights for policy 0, policy_version 576646 (0.0007) [2023-12-26 19:33:55,523][105692] Updated weights for policy 0, policy_version 576656 (0.0009) [2023-12-26 19:33:55,589][105692] Updated weights for policy 0, policy_version 576666 (0.0009) [2023-12-26 19:33:55,622][105620] Updated weights for policy 1, policy_version 577457 (0.0006) [2023-12-26 19:33:55,674][105620] Updated weights for policy 1, policy_version 577467 (0.0009) [2023-12-26 19:33:55,730][105620] Updated weights for policy 1, policy_version 577477 (0.0010) [2023-12-26 19:33:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 295493632. Throughput: 0: 9943.0, 1: 9906.1. Samples: 295501632. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:33:56,062][104569] Avg episode reward: [(0, '8898.969'), (1, '9352.513')] [2023-12-26 19:33:56,283][105692] Updated weights for policy 0, policy_version 576676 (0.0008) [2023-12-26 19:33:56,340][105692] Updated weights for policy 0, policy_version 576686 (0.0009) [2023-12-26 19:33:56,375][105620] Updated weights for policy 1, policy_version 577487 (0.0007) [2023-12-26 19:33:56,400][105692] Updated weights for policy 0, policy_version 576696 (0.0008) [2023-12-26 19:33:56,428][105620] Updated weights for policy 1, policy_version 577497 (0.0005) [2023-12-26 19:33:56,477][105620] Updated weights for policy 1, policy_version 577507 (0.0005) [2023-12-26 19:33:57,064][105620] Updated weights for policy 1, policy_version 577517 (0.0006) [2023-12-26 19:33:57,109][105620] Updated weights for policy 1, policy_version 577527 (0.0005) [2023-12-26 19:33:57,160][105620] Updated weights for policy 1, policy_version 577537 (0.0005) [2023-12-26 19:33:57,195][105692] Updated weights for policy 0, policy_version 576706 (0.0009) [2023-12-26 19:33:57,247][105692] Updated weights for policy 0, policy_version 576716 (0.0010) [2023-12-26 19:33:57,301][105692] Updated weights for policy 0, policy_version 576726 (0.0011) [2023-12-26 19:33:57,350][105692] Updated weights for policy 0, policy_version 576736 (0.0007) [2023-12-26 19:33:57,683][105620] Updated weights for policy 1, policy_version 577547 (0.0005) [2023-12-26 19:33:57,735][105620] Updated weights for policy 1, policy_version 577557 (0.0005) [2023-12-26 19:33:57,781][105620] Updated weights for policy 1, policy_version 577567 (0.0005) [2023-12-26 19:33:58,192][105692] Updated weights for policy 0, policy_version 576746 (0.0008) [2023-12-26 19:33:58,253][105692] Updated weights for policy 0, policy_version 576756 (0.0009) [2023-12-26 19:33:58,267][105585] KL-divergence is very high: 126.2589 [2023-12-26 19:33:58,319][105692] Updated weights for policy 0, policy_version 576766 (0.0009) [2023-12-26 19:33:58,320][105585] KL-divergence is very high: 105.1386 [2023-12-26 19:33:58,403][105620] Updated weights for policy 1, policy_version 577577 (0.0005) [2023-12-26 19:33:58,471][105620] Updated weights for policy 1, policy_version 577587 (0.0009) [2023-12-26 19:33:58,526][105620] Updated weights for policy 1, policy_version 577597 (0.0008) [2023-12-26 19:33:58,595][105620] Updated weights for policy 1, policy_version 577607 (0.0008) [2023-12-26 19:33:59,124][105692] Updated weights for policy 0, policy_version 576776 (0.0007) [2023-12-26 19:33:59,197][105692] Updated weights for policy 0, policy_version 576786 (0.0007) [2023-12-26 19:33:59,269][105692] Updated weights for policy 0, policy_version 576796 (0.0008) [2023-12-26 19:33:59,386][105620] Updated weights for policy 1, policy_version 577617 (0.0009) [2023-12-26 19:33:59,440][105620] Updated weights for policy 1, policy_version 577627 (0.0009) [2023-12-26 19:33:59,500][105620] Updated weights for policy 1, policy_version 577637 (0.0009) [2023-12-26 19:34:00,030][105692] Updated weights for policy 0, policy_version 576806 (0.0008) [2023-12-26 19:34:00,091][105692] Updated weights for policy 0, policy_version 576816 (0.0008) [2023-12-26 19:34:00,161][105692] Updated weights for policy 0, policy_version 576826 (0.0008) [2023-12-26 19:34:00,228][105620] Updated weights for policy 1, policy_version 577647 (0.0006) [2023-12-26 19:34:00,282][105620] Updated weights for policy 1, policy_version 577657 (0.0007) [2023-12-26 19:34:00,337][105620] Updated weights for policy 1, policy_version 577667 (0.0010) [2023-12-26 19:34:00,887][105692] Updated weights for policy 0, policy_version 576836 (0.0007) [2023-12-26 19:34:00,936][105692] Updated weights for policy 0, policy_version 576846 (0.0008) [2023-12-26 19:34:00,980][105692] Updated weights for policy 0, policy_version 576856 (0.0010) [2023-12-26 19:34:01,044][105620] Updated weights for policy 1, policy_version 577677 (0.0010) [2023-12-26 19:34:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 295591936. Throughput: 0: 9934.9, 1: 10008.0. Samples: 295562316. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:34:01,063][104569] Avg episode reward: [(0, '8807.430'), (1, '9079.791')] [2023-12-26 19:34:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000576864_147693568.pth... [2023-12-26 19:34:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000575680_147390464.pth [2023-12-26 19:34:01,108][105620] Updated weights for policy 1, policy_version 577687 (0.0009) [2023-12-26 19:34:01,179][105620] Updated weights for policy 1, policy_version 577697 (0.0008) [2023-12-26 19:34:01,222][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000577704_147906560.pth... [2023-12-26 19:34:01,229][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000576552_147611648.pth [2023-12-26 19:34:01,643][105692] Updated weights for policy 0, policy_version 576866 (0.0010) [2023-12-26 19:34:01,703][105692] Updated weights for policy 0, policy_version 576876 (0.0011) [2023-12-26 19:34:01,762][105692] Updated weights for policy 0, policy_version 576886 (0.0008) [2023-12-26 19:34:01,811][105692] Updated weights for policy 0, policy_version 576896 (0.0007) [2023-12-26 19:34:01,841][105620] Updated weights for policy 1, policy_version 577707 (0.0007) [2023-12-26 19:34:01,901][105620] Updated weights for policy 1, policy_version 577717 (0.0005) [2023-12-26 19:34:01,966][105620] Updated weights for policy 1, policy_version 577727 (0.0006) [2023-12-26 19:34:02,558][105620] Updated weights for policy 1, policy_version 577737 (0.0005) [2023-12-26 19:34:02,605][105692] Updated weights for policy 0, policy_version 576906 (0.0007) [2023-12-26 19:34:02,611][105620] Updated weights for policy 1, policy_version 577747 (0.0006) [2023-12-26 19:34:02,667][105620] Updated weights for policy 1, policy_version 577757 (0.0006) [2023-12-26 19:34:02,670][105692] Updated weights for policy 0, policy_version 576916 (0.0006) [2023-12-26 19:34:02,724][105620] Updated weights for policy 1, policy_version 577767 (0.0006) [2023-12-26 19:34:02,734][105692] Updated weights for policy 0, policy_version 576926 (0.0007) [2023-12-26 19:34:03,287][105620] Updated weights for policy 1, policy_version 577777 (0.0005) [2023-12-26 19:34:03,347][105620] Updated weights for policy 1, policy_version 577787 (0.0005) [2023-12-26 19:34:03,370][105692] Updated weights for policy 0, policy_version 576936 (0.0009) [2023-12-26 19:34:03,404][105620] Updated weights for policy 1, policy_version 577797 (0.0005) [2023-12-26 19:34:03,419][105692] Updated weights for policy 0, policy_version 576946 (0.0010) [2023-12-26 19:34:03,467][105692] Updated weights for policy 0, policy_version 576956 (0.0007) [2023-12-26 19:34:03,981][105620] Updated weights for policy 1, policy_version 577807 (0.0006) [2023-12-26 19:34:04,045][105620] Updated weights for policy 1, policy_version 577817 (0.0006) [2023-12-26 19:34:04,106][105620] Updated weights for policy 1, policy_version 577827 (0.0009) [2023-12-26 19:34:04,291][105692] Updated weights for policy 0, policy_version 576966 (0.0008) [2023-12-26 19:34:04,350][105692] Updated weights for policy 0, policy_version 576976 (0.0007) [2023-12-26 19:34:04,417][105692] Updated weights for policy 0, policy_version 576986 (0.0008) [2023-12-26 19:34:04,751][105620] Updated weights for policy 1, policy_version 577837 (0.0008) [2023-12-26 19:34:04,813][105620] Updated weights for policy 1, policy_version 577847 (0.0005) [2023-12-26 19:34:04,875][105620] Updated weights for policy 1, policy_version 577857 (0.0006) [2023-12-26 19:34:05,208][105692] Updated weights for policy 0, policy_version 576996 (0.0007) [2023-12-26 19:34:05,270][105692] Updated weights for policy 0, policy_version 577006 (0.0005) [2023-12-26 19:34:05,334][105692] Updated weights for policy 0, policy_version 577016 (0.0008) [2023-12-26 19:34:05,538][105620] Updated weights for policy 1, policy_version 577867 (0.0010) [2023-12-26 19:34:05,590][105620] Updated weights for policy 1, policy_version 577877 (0.0010) [2023-12-26 19:34:05,641][105620] Updated weights for policy 1, policy_version 577887 (0.0009) [2023-12-26 19:34:06,010][105692] Updated weights for policy 0, policy_version 577026 (0.0009) [2023-12-26 19:34:06,057][105692] Updated weights for policy 0, policy_version 577036 (0.0005) [2023-12-26 19:34:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 295690240. Throughput: 0: 9871.7, 1: 10111.7. Samples: 295683096. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:34:06,062][104569] Avg episode reward: [(0, '8820.357'), (1, '9079.880')] [2023-12-26 19:34:06,114][105692] Updated weights for policy 0, policy_version 577046 (0.0007) [2023-12-26 19:34:06,173][105692] Updated weights for policy 0, policy_version 577056 (0.0008) [2023-12-26 19:34:06,258][105620] Updated weights for policy 1, policy_version 577897 (0.0005) [2023-12-26 19:34:06,320][105620] Updated weights for policy 1, policy_version 577907 (0.0005) [2023-12-26 19:34:06,379][105620] Updated weights for policy 1, policy_version 577917 (0.0010) [2023-12-26 19:34:06,446][105620] Updated weights for policy 1, policy_version 577927 (0.0011) [2023-12-26 19:34:06,972][105692] Updated weights for policy 0, policy_version 577066 (0.0009) [2023-12-26 19:34:07,028][105692] Updated weights for policy 0, policy_version 577076 (0.0008) [2023-12-26 19:34:07,084][105692] Updated weights for policy 0, policy_version 577086 (0.0008) [2023-12-26 19:34:07,121][105620] Updated weights for policy 1, policy_version 577937 (0.0010) [2023-12-26 19:34:07,174][105620] Updated weights for policy 1, policy_version 577947 (0.0010) [2023-12-26 19:34:07,219][105620] Updated weights for policy 1, policy_version 577957 (0.0010) [2023-12-26 19:34:07,773][105692] Updated weights for policy 0, policy_version 577096 (0.0006) [2023-12-26 19:34:07,842][105692] Updated weights for policy 0, policy_version 577106 (0.0005) [2023-12-26 19:34:07,905][105620] Updated weights for policy 1, policy_version 577967 (0.0006) [2023-12-26 19:34:07,907][105692] Updated weights for policy 0, policy_version 577116 (0.0007) [2023-12-26 19:34:07,975][105620] Updated weights for policy 1, policy_version 577977 (0.0005) [2023-12-26 19:34:08,032][105620] Updated weights for policy 1, policy_version 577987 (0.0005) [2023-12-26 19:34:08,621][105692] Updated weights for policy 0, policy_version 577126 (0.0010) [2023-12-26 19:34:08,637][105620] Updated weights for policy 1, policy_version 577997 (0.0006) [2023-12-26 19:34:08,671][105692] Updated weights for policy 0, policy_version 577136 (0.0009) [2023-12-26 19:34:08,690][105620] Updated weights for policy 1, policy_version 578007 (0.0008) [2023-12-26 19:34:08,733][105692] Updated weights for policy 0, policy_version 577146 (0.0007) [2023-12-26 19:34:08,747][105620] Updated weights for policy 1, policy_version 578017 (0.0009) [2023-12-26 19:34:09,494][105692] Updated weights for policy 0, policy_version 577156 (0.0007) [2023-12-26 19:34:09,500][105620] Updated weights for policy 1, policy_version 578027 (0.0008) [2023-12-26 19:34:09,553][105692] Updated weights for policy 0, policy_version 577166 (0.0010) [2023-12-26 19:34:09,570][105620] Updated weights for policy 1, policy_version 578037 (0.0006) [2023-12-26 19:34:09,621][105692] Updated weights for policy 0, policy_version 577176 (0.0009) [2023-12-26 19:34:09,631][105620] Updated weights for policy 1, policy_version 578047 (0.0006) [2023-12-26 19:34:10,237][105620] Updated weights for policy 1, policy_version 578057 (0.0006) [2023-12-26 19:34:10,276][105692] Updated weights for policy 0, policy_version 577186 (0.0010) [2023-12-26 19:34:10,308][105620] Updated weights for policy 1, policy_version 578067 (0.0006) [2023-12-26 19:34:10,337][105692] Updated weights for policy 0, policy_version 577196 (0.0011) [2023-12-26 19:34:10,371][105620] Updated weights for policy 1, policy_version 578077 (0.0009) [2023-12-26 19:34:10,401][105692] Updated weights for policy 0, policy_version 577206 (0.0011) [2023-12-26 19:34:10,432][105620] Updated weights for policy 1, policy_version 578087 (0.0006) [2023-12-26 19:34:10,465][105692] Updated weights for policy 0, policy_version 577216 (0.0011) [2023-12-26 19:34:11,013][105620] Updated weights for policy 1, policy_version 578097 (0.0008) [2023-12-26 19:34:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 295788544. Throughput: 0: 9922.4, 1: 10154.1. Samples: 295803204. Policy #0 lag: (min: 19.0, avg: 30.8, max: 51.0) [2023-12-26 19:34:11,062][104569] Avg episode reward: [(0, '8133.775'), (1, '9171.267')] [2023-12-26 19:34:11,083][105620] Updated weights for policy 1, policy_version 578107 (0.0007) [2023-12-26 19:34:11,088][105692] Updated weights for policy 0, policy_version 577226 (0.0012) [2023-12-26 19:34:11,151][105620] Updated weights for policy 1, policy_version 578117 (0.0007) [2023-12-26 19:34:11,151][105692] Updated weights for policy 0, policy_version 577236 (0.0011) [2023-12-26 19:34:11,221][105692] Updated weights for policy 0, policy_version 577246 (0.0011) [2023-12-26 19:34:11,888][105620] Updated weights for policy 1, policy_version 578127 (0.0008) [2023-12-26 19:34:11,952][105620] Updated weights for policy 1, policy_version 578137 (0.0009) [2023-12-26 19:34:11,968][105692] Updated weights for policy 0, policy_version 577256 (0.0006) [2023-12-26 19:34:12,017][105620] Updated weights for policy 1, policy_version 578147 (0.0009) [2023-12-26 19:34:12,026][105692] Updated weights for policy 0, policy_version 577266 (0.0006) [2023-12-26 19:34:12,086][105692] Updated weights for policy 0, policy_version 577276 (0.0006) [2023-12-26 19:34:12,661][105692] Updated weights for policy 0, policy_version 577286 (0.0008) [2023-12-26 19:34:12,726][105692] Updated weights for policy 0, policy_version 577296 (0.0009) [2023-12-26 19:34:12,767][105585] KL-divergence is very high: 100.6265 [2023-12-26 19:34:12,789][105692] Updated weights for policy 0, policy_version 577306 (0.0009) [2023-12-26 19:34:12,812][105585] KL-divergence is very high: 103.2603 [2023-12-26 19:34:12,817][105620] Updated weights for policy 1, policy_version 578157 (0.0010) [2023-12-26 19:34:12,879][105620] Updated weights for policy 1, policy_version 578167 (0.0008) [2023-12-26 19:34:12,942][105620] Updated weights for policy 1, policy_version 578177 (0.0008) [2023-12-26 19:34:13,505][105692] Updated weights for policy 0, policy_version 577316 (0.0008) [2023-12-26 19:34:13,574][105692] Updated weights for policy 0, policy_version 577326 (0.0005) [2023-12-26 19:34:13,642][105692] Updated weights for policy 0, policy_version 577336 (0.0006) [2023-12-26 19:34:13,719][105620] Updated weights for policy 1, policy_version 578188 (0.0009) [2023-12-26 19:34:13,777][105620] Updated weights for policy 1, policy_version 578198 (0.0010) [2023-12-26 19:34:13,836][105620] Updated weights for policy 1, policy_version 578209 (0.0010) [2023-12-26 19:34:14,148][105692] Updated weights for policy 0, policy_version 577346 (0.0005) [2023-12-26 19:34:14,217][105692] Updated weights for policy 0, policy_version 577356 (0.0005) [2023-12-26 19:34:14,286][105692] Updated weights for policy 0, policy_version 577366 (0.0006) [2023-12-26 19:34:14,340][105692] Updated weights for policy 0, policy_version 577376 (0.0005) [2023-12-26 19:34:14,690][105620] Updated weights for policy 1, policy_version 578219 (0.0010) [2023-12-26 19:34:14,738][105620] Updated weights for policy 1, policy_version 578229 (0.0009) [2023-12-26 19:34:14,795][105620] Updated weights for policy 1, policy_version 578239 (0.0009) [2023-12-26 19:34:14,936][105692] Updated weights for policy 0, policy_version 577386 (0.0009) [2023-12-26 19:34:14,991][105692] Updated weights for policy 0, policy_version 577396 (0.0009) [2023-12-26 19:34:15,053][105692] Updated weights for policy 0, policy_version 577406 (0.0009) [2023-12-26 19:34:15,555][105620] Updated weights for policy 1, policy_version 578249 (0.0009) [2023-12-26 19:34:15,609][105620] Updated weights for policy 1, policy_version 578259 (0.0009) [2023-12-26 19:34:15,674][105620] Updated weights for policy 1, policy_version 578269 (0.0009) [2023-12-26 19:34:15,735][105620] Updated weights for policy 1, policy_version 578279 (0.0009) [2023-12-26 19:34:15,813][105692] Updated weights for policy 0, policy_version 577416 (0.0009) [2023-12-26 19:34:15,867][105692] Updated weights for policy 0, policy_version 577426 (0.0009) [2023-12-26 19:34:15,914][105692] Updated weights for policy 0, policy_version 577436 (0.0008) [2023-12-26 19:34:16,062][104569] Fps is (10 sec: 20479.4, 60 sec: 20070.4, 300 sec: 19660.8). Total num frames: 295895040. Throughput: 0: 9923.0, 1: 10035.9. Samples: 295861148. Policy #0 lag: (min: 19.0, avg: 30.8, max: 51.0) [2023-12-26 19:34:16,063][104569] Avg episode reward: [(0, '7928.029'), (1, '8717.271')] [2023-12-26 19:34:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000577440_147841024.pth... [2023-12-26 19:34:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000578280_148054016.pth... [2023-12-26 19:34:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000577096_147750912.pth [2023-12-26 19:34:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000576288_147546112.pth [2023-12-26 19:34:16,454][105620] Updated weights for policy 1, policy_version 578289 (0.0010) [2023-12-26 19:34:16,521][105620] Updated weights for policy 1, policy_version 578299 (0.0009) [2023-12-26 19:34:16,580][105620] Updated weights for policy 1, policy_version 578309 (0.0009) [2023-12-26 19:34:16,677][105692] Updated weights for policy 0, policy_version 577446 (0.0010) [2023-12-26 19:34:16,737][105692] Updated weights for policy 0, policy_version 577456 (0.0009) [2023-12-26 19:34:16,796][105692] Updated weights for policy 0, policy_version 577466 (0.0008) [2023-12-26 19:34:17,248][105620] Updated weights for policy 1, policy_version 578319 (0.0006) [2023-12-26 19:34:17,321][105620] Updated weights for policy 1, policy_version 578329 (0.0005) [2023-12-26 19:34:17,384][105620] Updated weights for policy 1, policy_version 578339 (0.0005) [2023-12-26 19:34:17,664][105692] Updated weights for policy 0, policy_version 577476 (0.0008) [2023-12-26 19:34:17,713][105692] Updated weights for policy 0, policy_version 577486 (0.0006) [2023-12-26 19:34:17,761][105692] Updated weights for policy 0, policy_version 577496 (0.0009) [2023-12-26 19:34:17,915][105620] Updated weights for policy 1, policy_version 578349 (0.0007) [2023-12-26 19:34:17,970][105620] Updated weights for policy 1, policy_version 578359 (0.0009) [2023-12-26 19:34:18,020][105620] Updated weights for policy 1, policy_version 578369 (0.0008) [2023-12-26 19:34:18,463][105692] Updated weights for policy 0, policy_version 577506 (0.0009) [2023-12-26 19:34:18,526][105692] Updated weights for policy 0, policy_version 577516 (0.0009) [2023-12-26 19:34:18,593][105692] Updated weights for policy 0, policy_version 577526 (0.0010) [2023-12-26 19:34:18,664][105692] Updated weights for policy 0, policy_version 577536 (0.0010) [2023-12-26 19:34:18,761][105620] Updated weights for policy 1, policy_version 578379 (0.0007) [2023-12-26 19:34:18,816][105620] Updated weights for policy 1, policy_version 578389 (0.0009) [2023-12-26 19:34:18,880][105620] Updated weights for policy 1, policy_version 578399 (0.0010) [2023-12-26 19:34:19,434][105692] Updated weights for policy 0, policy_version 577546 (0.0009) [2023-12-26 19:34:19,493][105692] Updated weights for policy 0, policy_version 577556 (0.0009) [2023-12-26 19:34:19,559][105692] Updated weights for policy 0, policy_version 577566 (0.0009) [2023-12-26 19:34:19,620][105620] Updated weights for policy 1, policy_version 578409 (0.0009) [2023-12-26 19:34:19,680][105620] Updated weights for policy 1, policy_version 578419 (0.0009) [2023-12-26 19:34:19,748][105620] Updated weights for policy 1, policy_version 578429 (0.0008) [2023-12-26 19:34:19,811][105620] Updated weights for policy 1, policy_version 578439 (0.0009) [2023-12-26 19:34:20,315][105692] Updated weights for policy 0, policy_version 577576 (0.0009) [2023-12-26 19:34:20,377][105692] Updated weights for policy 0, policy_version 577586 (0.0008) [2023-12-26 19:34:20,443][105692] Updated weights for policy 0, policy_version 577596 (0.0009) [2023-12-26 19:34:20,589][105620] Updated weights for policy 1, policy_version 578449 (0.0009) [2023-12-26 19:34:20,656][105620] Updated weights for policy 1, policy_version 578459 (0.0009) [2023-12-26 19:34:20,722][105620] Updated weights for policy 1, policy_version 578469 (0.0009) [2023-12-26 19:34:21,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19933.8, 300 sec: 19660.8). Total num frames: 295985152. Throughput: 0: 9857.2, 1: 9902.5. Samples: 295976552. Policy #0 lag: (min: 19.0, avg: 30.8, max: 51.0) [2023-12-26 19:34:21,063][104569] Avg episode reward: [(0, '8728.735'), (1, '9079.452')] [2023-12-26 19:34:21,149][105692] Updated weights for policy 0, policy_version 577606 (0.0008) [2023-12-26 19:34:21,194][105692] Updated weights for policy 0, policy_version 577616 (0.0005) [2023-12-26 19:34:21,250][105692] Updated weights for policy 0, policy_version 577626 (0.0006) [2023-12-26 19:34:21,545][105620] Updated weights for policy 1, policy_version 578479 (0.0009) [2023-12-26 19:34:21,607][105620] Updated weights for policy 1, policy_version 578489 (0.0007) [2023-12-26 19:34:21,681][105620] Updated weights for policy 1, policy_version 578499 (0.0009) [2023-12-26 19:34:21,999][105692] Updated weights for policy 0, policy_version 577636 (0.0007) [2023-12-26 19:34:22,071][105692] Updated weights for policy 0, policy_version 577646 (0.0010) [2023-12-26 19:34:22,140][105692] Updated weights for policy 0, policy_version 577656 (0.0010) [2023-12-26 19:34:22,315][105620] Updated weights for policy 1, policy_version 578509 (0.0008) [2023-12-26 19:34:22,381][105620] Updated weights for policy 1, policy_version 578519 (0.0009) [2023-12-26 19:34:22,433][105620] Updated weights for policy 1, policy_version 578529 (0.0009) [2023-12-26 19:34:22,928][105692] Updated weights for policy 0, policy_version 577666 (0.0009) [2023-12-26 19:34:22,986][105692] Updated weights for policy 0, policy_version 577676 (0.0006) [2023-12-26 19:34:23,039][105692] Updated weights for policy 0, policy_version 577686 (0.0007) [2023-12-26 19:34:23,094][105692] Updated weights for policy 0, policy_version 577696 (0.0007) [2023-12-26 19:34:23,224][105620] Updated weights for policy 1, policy_version 578539 (0.0009) [2023-12-26 19:34:23,279][105620] Updated weights for policy 1, policy_version 578549 (0.0009) [2023-12-26 19:34:23,352][105620] Updated weights for policy 1, policy_version 578559 (0.0009) [2023-12-26 19:34:23,827][105692] Updated weights for policy 0, policy_version 577706 (0.0006) [2023-12-26 19:34:23,881][105692] Updated weights for policy 0, policy_version 577716 (0.0005) [2023-12-26 19:34:23,941][105692] Updated weights for policy 0, policy_version 577726 (0.0009) [2023-12-26 19:34:24,075][105620] Updated weights for policy 1, policy_version 578569 (0.0010) [2023-12-26 19:34:24,130][105620] Updated weights for policy 1, policy_version 578579 (0.0009) [2023-12-26 19:34:24,182][105620] Updated weights for policy 1, policy_version 578589 (0.0009) [2023-12-26 19:34:24,235][105620] Updated weights for policy 1, policy_version 578599 (0.0009) [2023-12-26 19:34:24,616][105692] Updated weights for policy 0, policy_version 577736 (0.0009) [2023-12-26 19:34:24,679][105692] Updated weights for policy 0, policy_version 577746 (0.0009) [2023-12-26 19:34:24,741][105692] Updated weights for policy 0, policy_version 577756 (0.0009) [2023-12-26 19:34:25,026][105620] Updated weights for policy 1, policy_version 578609 (0.0010) [2023-12-26 19:34:25,084][105620] Updated weights for policy 1, policy_version 578619 (0.0009) [2023-12-26 19:34:25,137][105620] Updated weights for policy 1, policy_version 578629 (0.0009) [2023-12-26 19:34:25,507][105692] Updated weights for policy 0, policy_version 577766 (0.0009) [2023-12-26 19:34:25,559][105692] Updated weights for policy 0, policy_version 577776 (0.0008) [2023-12-26 19:34:25,614][105692] Updated weights for policy 0, policy_version 577786 (0.0009) [2023-12-26 19:34:25,852][105620] Updated weights for policy 1, policy_version 578639 (0.0009) [2023-12-26 19:34:25,907][105620] Updated weights for policy 1, policy_version 578649 (0.0010) [2023-12-26 19:34:25,960][105620] Updated weights for policy 1, policy_version 578659 (0.0010) [2023-12-26 19:34:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19797.2, 300 sec: 19660.8). Total num frames: 296083456. Throughput: 0: 9773.4, 1: 9842.0. Samples: 296089240. Policy #0 lag: (min: 19.0, avg: 30.8, max: 51.0) [2023-12-26 19:34:26,063][104569] Avg episode reward: [(0, '9183.994'), (1, '9261.301')] [2023-12-26 19:34:26,347][105692] Updated weights for policy 0, policy_version 577796 (0.0009) [2023-12-26 19:34:26,393][105692] Updated weights for policy 0, policy_version 577806 (0.0007) [2023-12-26 19:34:26,441][105692] Updated weights for policy 0, policy_version 577816 (0.0006) [2023-12-26 19:34:26,786][105620] Updated weights for policy 1, policy_version 578669 (0.0009) [2023-12-26 19:34:26,844][105620] Updated weights for policy 1, policy_version 578679 (0.0010) [2023-12-26 19:34:26,900][105620] Updated weights for policy 1, policy_version 578689 (0.0009) [2023-12-26 19:34:27,047][105692] Updated weights for policy 0, policy_version 577826 (0.0009) [2023-12-26 19:34:27,102][105692] Updated weights for policy 0, policy_version 577836 (0.0007) [2023-12-26 19:34:27,160][105692] Updated weights for policy 0, policy_version 577846 (0.0005) [2023-12-26 19:34:27,211][105692] Updated weights for policy 0, policy_version 577856 (0.0005) [2023-12-26 19:34:27,745][105620] Updated weights for policy 1, policy_version 578700 (0.0008) [2023-12-26 19:34:27,803][105620] Updated weights for policy 1, policy_version 578710 (0.0009) [2023-12-26 19:34:27,820][105692] Updated weights for policy 0, policy_version 577866 (0.0005) [2023-12-26 19:34:27,860][105620] Updated weights for policy 1, policy_version 578720 (0.0009) [2023-12-26 19:34:27,879][105692] Updated weights for policy 0, policy_version 577876 (0.0005) [2023-12-26 19:34:27,931][105692] Updated weights for policy 0, policy_version 577886 (0.0005) [2023-12-26 19:34:28,512][105692] Updated weights for policy 0, policy_version 577896 (0.0010) [2023-12-26 19:34:28,577][105692] Updated weights for policy 0, policy_version 577906 (0.0010) [2023-12-26 19:34:28,631][105692] Updated weights for policy 0, policy_version 577916 (0.0010) [2023-12-26 19:34:28,686][105620] Updated weights for policy 1, policy_version 578730 (0.0008) [2023-12-26 19:34:28,740][105620] Updated weights for policy 1, policy_version 578740 (0.0008) [2023-12-26 19:34:28,791][105620] Updated weights for policy 1, policy_version 578750 (0.0010) [2023-12-26 19:34:28,840][105620] Updated weights for policy 1, policy_version 578760 (0.0006) [2023-12-26 19:34:29,231][105692] Updated weights for policy 0, policy_version 577926 (0.0011) [2023-12-26 19:34:29,294][105692] Updated weights for policy 0, policy_version 577936 (0.0011) [2023-12-26 19:34:29,355][105692] Updated weights for policy 0, policy_version 577946 (0.0011) [2023-12-26 19:34:29,554][105620] Updated weights for policy 1, policy_version 578770 (0.0011) [2023-12-26 19:34:29,605][105620] Updated weights for policy 1, policy_version 578780 (0.0011) [2023-12-26 19:34:29,656][105620] Updated weights for policy 1, policy_version 578790 (0.0010) [2023-12-26 19:34:30,097][105692] Updated weights for policy 0, policy_version 577956 (0.0009) [2023-12-26 19:34:30,166][105692] Updated weights for policy 0, policy_version 577966 (0.0010) [2023-12-26 19:34:30,242][105692] Updated weights for policy 0, policy_version 577976 (0.0007) [2023-12-26 19:34:30,352][105620] Updated weights for policy 1, policy_version 578800 (0.0009) [2023-12-26 19:34:30,399][105620] Updated weights for policy 1, policy_version 578810 (0.0009) [2023-12-26 19:34:30,448][105620] Updated weights for policy 1, policy_version 578820 (0.0009) [2023-12-26 19:34:30,949][105692] Updated weights for policy 0, policy_version 577986 (0.0006) [2023-12-26 19:34:31,015][105692] Updated weights for policy 0, policy_version 577996 (0.0005) [2023-12-26 19:34:31,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 296173568. Throughput: 0: 9816.5, 1: 9824.1. Samples: 296148524. Policy #0 lag: (min: 19.0, avg: 30.8, max: 51.0) [2023-12-26 19:34:31,062][104569] Avg episode reward: [(0, '8834.302'), (1, '9352.720')] [2023-12-26 19:34:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000578824_148193280.pth... [2023-12-26 19:34:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000577704_147906560.pth [2023-12-26 19:34:31,073][105692] Updated weights for policy 0, policy_version 578006 (0.0008) [2023-12-26 19:34:31,139][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000578016_147988480.pth... [2023-12-26 19:34:31,141][105692] Updated weights for policy 0, policy_version 578016 (0.0007) [2023-12-26 19:34:31,143][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000576864_147693568.pth [2023-12-26 19:34:31,230][105620] Updated weights for policy 1, policy_version 578830 (0.0007) [2023-12-26 19:34:31,296][105620] Updated weights for policy 1, policy_version 578840 (0.0006) [2023-12-26 19:34:31,369][105620] Updated weights for policy 1, policy_version 578850 (0.0007) [2023-12-26 19:34:31,822][105692] Updated weights for policy 0, policy_version 578026 (0.0006) [2023-12-26 19:34:31,885][105692] Updated weights for policy 0, policy_version 578036 (0.0007) [2023-12-26 19:34:31,938][105692] Updated weights for policy 0, policy_version 578046 (0.0007) [2023-12-26 19:34:32,077][105620] Updated weights for policy 1, policy_version 578860 (0.0008) [2023-12-26 19:34:32,139][105620] Updated weights for policy 1, policy_version 578870 (0.0008) [2023-12-26 19:34:32,202][105620] Updated weights for policy 1, policy_version 578880 (0.0007) [2023-12-26 19:34:32,660][105692] Updated weights for policy 0, policy_version 578056 (0.0009) [2023-12-26 19:34:32,705][105585] KL-divergence is very high: 133.0077 [2023-12-26 19:34:32,706][105692] Updated weights for policy 0, policy_version 578066 (0.0009) [2023-12-26 19:34:32,742][105585] KL-divergence is very high: 182.5751 [2023-12-26 19:34:32,752][105692] Updated weights for policy 0, policy_version 578076 (0.0008) [2023-12-26 19:34:32,870][105620] Updated weights for policy 1, policy_version 578890 (0.0009) [2023-12-26 19:34:32,918][105620] Updated weights for policy 1, policy_version 578900 (0.0007) [2023-12-26 19:34:32,965][105620] Updated weights for policy 1, policy_version 578910 (0.0008) [2023-12-26 19:34:33,012][105620] Updated weights for policy 1, policy_version 578920 (0.0008) [2023-12-26 19:34:33,448][105692] Updated weights for policy 0, policy_version 578086 (0.0007) [2023-12-26 19:34:33,500][105692] Updated weights for policy 0, policy_version 578096 (0.0005) [2023-12-26 19:34:33,564][105692] Updated weights for policy 0, policy_version 578106 (0.0007) [2023-12-26 19:34:33,791][105620] Updated weights for policy 1, policy_version 578930 (0.0009) [2023-12-26 19:34:33,844][105620] Updated weights for policy 1, policy_version 578940 (0.0009) [2023-12-26 19:34:33,898][105620] Updated weights for policy 1, policy_version 578950 (0.0009) [2023-12-26 19:34:34,178][105692] Updated weights for policy 0, policy_version 578117 (0.0010) [2023-12-26 19:34:34,234][105692] Updated weights for policy 0, policy_version 578127 (0.0009) [2023-12-26 19:34:34,293][105692] Updated weights for policy 0, policy_version 578137 (0.0009) [2023-12-26 19:34:34,678][105620] Updated weights for policy 1, policy_version 578960 (0.0008) [2023-12-26 19:34:34,744][105620] Updated weights for policy 1, policy_version 578970 (0.0008) [2023-12-26 19:34:34,796][105620] Updated weights for policy 1, policy_version 578980 (0.0008) [2023-12-26 19:34:35,047][105692] Updated weights for policy 0, policy_version 578147 (0.0008) [2023-12-26 19:34:35,103][105692] Updated weights for policy 0, policy_version 578157 (0.0005) [2023-12-26 19:34:35,160][105692] Updated weights for policy 0, policy_version 578167 (0.0005) [2023-12-26 19:34:35,537][105620] Updated weights for policy 1, policy_version 578990 (0.0007) [2023-12-26 19:34:35,602][105620] Updated weights for policy 1, policy_version 579000 (0.0009) [2023-12-26 19:34:35,662][105620] Updated weights for policy 1, policy_version 579010 (0.0009) [2023-12-26 19:34:35,783][105692] Updated weights for policy 0, policy_version 578177 (0.0009) [2023-12-26 19:34:35,835][105692] Updated weights for policy 0, policy_version 578187 (0.0006) [2023-12-26 19:34:35,883][105692] Updated weights for policy 0, policy_version 578197 (0.0010) [2023-12-26 19:34:35,931][105692] Updated weights for policy 0, policy_version 578207 (0.0010) [2023-12-26 19:34:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 296280064. Throughput: 0: 9734.5, 1: 9808.9. Samples: 296265364. Policy #0 lag: (min: 19.0, avg: 30.8, max: 51.0) [2023-12-26 19:34:36,063][104569] Avg episode reward: [(0, '8827.151'), (1, '9081.420')] [2023-12-26 19:34:36,282][105620] Updated weights for policy 1, policy_version 579020 (0.0008) [2023-12-26 19:34:36,346][105620] Updated weights for policy 1, policy_version 579030 (0.0005) [2023-12-26 19:34:36,402][105620] Updated weights for policy 1, policy_version 579040 (0.0005) [2023-12-26 19:34:36,692][105692] Updated weights for policy 0, policy_version 578217 (0.0010) [2023-12-26 19:34:36,753][105692] Updated weights for policy 0, policy_version 578227 (0.0010) [2023-12-26 19:34:36,811][105692] Updated weights for policy 0, policy_version 578237 (0.0010) [2023-12-26 19:34:36,928][105620] Updated weights for policy 1, policy_version 579050 (0.0006) [2023-12-26 19:34:36,985][105620] Updated weights for policy 1, policy_version 579060 (0.0006) [2023-12-26 19:34:37,040][105620] Updated weights for policy 1, policy_version 579070 (0.0006) [2023-12-26 19:34:37,094][105620] Updated weights for policy 1, policy_version 579080 (0.0007) [2023-12-26 19:34:37,644][105692] Updated weights for policy 0, policy_version 578247 (0.0009) [2023-12-26 19:34:37,700][105692] Updated weights for policy 0, policy_version 578257 (0.0008) [2023-12-26 19:34:37,756][105692] Updated weights for policy 0, policy_version 578267 (0.0009) [2023-12-26 19:34:37,792][105620] Updated weights for policy 1, policy_version 579090 (0.0010) [2023-12-26 19:34:37,854][105620] Updated weights for policy 1, policy_version 579100 (0.0010) [2023-12-26 19:34:37,906][105620] Updated weights for policy 1, policy_version 579110 (0.0010) [2023-12-26 19:34:38,377][105692] Updated weights for policy 0, policy_version 578277 (0.0008) [2023-12-26 19:34:38,442][105692] Updated weights for policy 0, policy_version 578287 (0.0007) [2023-12-26 19:34:38,505][105692] Updated weights for policy 0, policy_version 578297 (0.0007) [2023-12-26 19:34:38,680][105620] Updated weights for policy 1, policy_version 579120 (0.0006) [2023-12-26 19:34:38,751][105620] Updated weights for policy 1, policy_version 579130 (0.0005) [2023-12-26 19:34:38,822][105620] Updated weights for policy 1, policy_version 579140 (0.0008) [2023-12-26 19:34:39,104][105692] Updated weights for policy 0, policy_version 578307 (0.0006) [2023-12-26 19:34:39,174][105692] Updated weights for policy 0, policy_version 578317 (0.0009) [2023-12-26 19:34:39,244][105692] Updated weights for policy 0, policy_version 578327 (0.0010) [2023-12-26 19:34:39,534][105620] Updated weights for policy 1, policy_version 579150 (0.0009) [2023-12-26 19:34:39,598][105620] Updated weights for policy 1, policy_version 579160 (0.0009) [2023-12-26 19:34:39,658][105620] Updated weights for policy 1, policy_version 579170 (0.0009) [2023-12-26 19:34:39,944][105692] Updated weights for policy 0, policy_version 578337 (0.0006) [2023-12-26 19:34:39,996][105692] Updated weights for policy 0, policy_version 578347 (0.0009) [2023-12-26 19:34:40,055][105692] Updated weights for policy 0, policy_version 578357 (0.0009) [2023-12-26 19:34:40,117][105692] Updated weights for policy 0, policy_version 578367 (0.0009) [2023-12-26 19:34:40,478][105620] Updated weights for policy 1, policy_version 579180 (0.0010) [2023-12-26 19:34:40,542][105620] Updated weights for policy 1, policy_version 579190 (0.0011) [2023-12-26 19:34:40,595][105620] Updated weights for policy 1, policy_version 579200 (0.0010) [2023-12-26 19:34:40,826][105692] Updated weights for policy 0, policy_version 578377 (0.0010) [2023-12-26 19:34:40,878][105692] Updated weights for policy 0, policy_version 578387 (0.0010) [2023-12-26 19:34:40,943][105692] Updated weights for policy 0, policy_version 578397 (0.0010) [2023-12-26 19:34:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 296378368. Throughput: 0: 9756.0, 1: 9858.1. Samples: 296384268. Policy #0 lag: (min: 19.0, avg: 30.8, max: 51.0) [2023-12-26 19:34:41,062][104569] Avg episode reward: [(0, '9354.796'), (1, '9172.362')] [2023-12-26 19:34:41,238][105620] Updated weights for policy 1, policy_version 579210 (0.0010) [2023-12-26 19:34:41,294][105620] Updated weights for policy 1, policy_version 579220 (0.0011) [2023-12-26 19:34:41,359][105620] Updated weights for policy 1, policy_version 579230 (0.0012) [2023-12-26 19:34:41,420][105620] Updated weights for policy 1, policy_version 579240 (0.0011) [2023-12-26 19:34:41,702][105692] Updated weights for policy 0, policy_version 578407 (0.0009) [2023-12-26 19:34:41,774][105692] Updated weights for policy 0, policy_version 578417 (0.0009) [2023-12-26 19:34:41,826][105692] Updated weights for policy 0, policy_version 578427 (0.0009) [2023-12-26 19:34:42,201][105620] Updated weights for policy 1, policy_version 579250 (0.0008) [2023-12-26 19:34:42,261][105620] Updated weights for policy 1, policy_version 579260 (0.0008) [2023-12-26 19:34:42,318][105620] Updated weights for policy 1, policy_version 579270 (0.0008) [2023-12-26 19:34:42,589][105692] Updated weights for policy 0, policy_version 578437 (0.0010) [2023-12-26 19:34:42,641][105692] Updated weights for policy 0, policy_version 578447 (0.0010) [2023-12-26 19:34:42,690][105692] Updated weights for policy 0, policy_version 578457 (0.0010) [2023-12-26 19:34:43,086][105620] Updated weights for policy 1, policy_version 579280 (0.0008) [2023-12-26 19:34:43,149][105620] Updated weights for policy 1, policy_version 579290 (0.0008) [2023-12-26 19:34:43,210][105620] Updated weights for policy 1, policy_version 579300 (0.0008) [2023-12-26 19:34:43,456][105692] Updated weights for policy 0, policy_version 578467 (0.0010) [2023-12-26 19:34:43,517][105692] Updated weights for policy 0, policy_version 578477 (0.0010) [2023-12-26 19:34:43,569][105692] Updated weights for policy 0, policy_version 578487 (0.0010) [2023-12-26 19:34:43,961][105620] Updated weights for policy 1, policy_version 579310 (0.0008) [2023-12-26 19:34:44,011][105620] Updated weights for policy 1, policy_version 579320 (0.0008) [2023-12-26 19:34:44,063][105620] Updated weights for policy 1, policy_version 579330 (0.0008) [2023-12-26 19:34:44,307][105692] Updated weights for policy 0, policy_version 578497 (0.0010) [2023-12-26 19:34:44,369][105692] Updated weights for policy 0, policy_version 578507 (0.0007) [2023-12-26 19:34:44,429][105692] Updated weights for policy 0, policy_version 578517 (0.0008) [2023-12-26 19:34:44,484][105692] Updated weights for policy 0, policy_version 578527 (0.0008) [2023-12-26 19:34:44,794][105620] Updated weights for policy 1, policy_version 579340 (0.0008) [2023-12-26 19:34:44,842][105620] Updated weights for policy 1, policy_version 579350 (0.0006) [2023-12-26 19:34:44,907][105620] Updated weights for policy 1, policy_version 579360 (0.0006) [2023-12-26 19:34:45,175][105692] Updated weights for policy 0, policy_version 578537 (0.0010) [2023-12-26 19:34:45,228][105692] Updated weights for policy 0, policy_version 578547 (0.0011) [2023-12-26 19:34:45,284][105692] Updated weights for policy 0, policy_version 578557 (0.0011) [2023-12-26 19:34:45,547][105620] Updated weights for policy 1, policy_version 579370 (0.0005) [2023-12-26 19:34:45,608][105620] Updated weights for policy 1, policy_version 579380 (0.0005) [2023-12-26 19:34:45,654][105620] Updated weights for policy 1, policy_version 579390 (0.0005) [2023-12-26 19:34:45,702][105620] Updated weights for policy 1, policy_version 579400 (0.0006) [2023-12-26 19:34:46,013][105692] Updated weights for policy 0, policy_version 578567 (0.0011) [2023-12-26 19:34:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19605.2). Total num frames: 296468480. Throughput: 0: 9776.3, 1: 9729.5. Samples: 296440084. Policy #0 lag: (min: 19.0, avg: 30.8, max: 51.0) [2023-12-26 19:34:46,063][104569] Avg episode reward: [(0, '9356.354'), (1, '9077.321')] [2023-12-26 19:34:46,064][105692] Updated weights for policy 0, policy_version 578577 (0.0010) [2023-12-26 19:34:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000579400_148340736.pth... [2023-12-26 19:34:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000578280_148054016.pth [2023-12-26 19:34:46,117][105692] Updated weights for policy 0, policy_version 578587 (0.0010) [2023-12-26 19:34:46,141][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000578592_148135936.pth... [2023-12-26 19:34:46,144][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000577440_147841024.pth [2023-12-26 19:34:46,287][105620] Updated weights for policy 1, policy_version 579410 (0.0008) [2023-12-26 19:34:46,341][105620] Updated weights for policy 1, policy_version 579420 (0.0008) [2023-12-26 19:34:46,396][105620] Updated weights for policy 1, policy_version 579430 (0.0008) [2023-12-26 19:34:46,883][105692] Updated weights for policy 0, policy_version 578597 (0.0010) [2023-12-26 19:34:46,934][105692] Updated weights for policy 0, policy_version 578607 (0.0008) [2023-12-26 19:34:46,989][105692] Updated weights for policy 0, policy_version 578617 (0.0009) [2023-12-26 19:34:47,102][105620] Updated weights for policy 1, policy_version 579440 (0.0009) [2023-12-26 19:34:47,168][105620] Updated weights for policy 1, policy_version 579450 (0.0009) [2023-12-26 19:34:47,226][105620] Updated weights for policy 1, policy_version 579460 (0.0009) [2023-12-26 19:34:47,698][105692] Updated weights for policy 0, policy_version 578627 (0.0010) [2023-12-26 19:34:47,763][105692] Updated weights for policy 0, policy_version 578637 (0.0011) [2023-12-26 19:34:47,821][105692] Updated weights for policy 0, policy_version 578647 (0.0010) [2023-12-26 19:34:48,017][105620] Updated weights for policy 1, policy_version 579470 (0.0009) [2023-12-26 19:34:48,067][105620] Updated weights for policy 1, policy_version 579480 (0.0008) [2023-12-26 19:34:48,113][105620] Updated weights for policy 1, policy_version 579490 (0.0008) [2023-12-26 19:34:48,633][105692] Updated weights for policy 0, policy_version 578657 (0.0009) [2023-12-26 19:34:48,690][105692] Updated weights for policy 0, policy_version 578667 (0.0010) [2023-12-26 19:34:48,743][105692] Updated weights for policy 0, policy_version 578677 (0.0010) [2023-12-26 19:34:48,752][105620] Updated weights for policy 1, policy_version 579500 (0.0009) [2023-12-26 19:34:48,792][105692] Updated weights for policy 0, policy_version 578687 (0.0010) [2023-12-26 19:34:48,812][105620] Updated weights for policy 1, policy_version 579510 (0.0009) [2023-12-26 19:34:48,867][105620] Updated weights for policy 1, policy_version 579520 (0.0008) [2023-12-26 19:34:49,490][105692] Updated weights for policy 0, policy_version 578697 (0.0007) [2023-12-26 19:34:49,539][105692] Updated weights for policy 0, policy_version 578707 (0.0005) [2023-12-26 19:34:49,589][105692] Updated weights for policy 0, policy_version 578717 (0.0010) [2023-12-26 19:34:49,653][105620] Updated weights for policy 1, policy_version 579530 (0.0009) [2023-12-26 19:34:49,711][105620] Updated weights for policy 1, policy_version 579540 (0.0010) [2023-12-26 19:34:49,767][105620] Updated weights for policy 1, policy_version 579550 (0.0011) [2023-12-26 19:34:49,831][105620] Updated weights for policy 1, policy_version 579560 (0.0011) [2023-12-26 19:34:50,367][105692] Updated weights for policy 0, policy_version 578727 (0.0011) [2023-12-26 19:34:50,434][105692] Updated weights for policy 0, policy_version 578737 (0.0011) [2023-12-26 19:34:50,484][105692] Updated weights for policy 0, policy_version 578747 (0.0011) [2023-12-26 19:34:50,520][105620] Updated weights for policy 1, policy_version 579570 (0.0010) [2023-12-26 19:34:50,583][105620] Updated weights for policy 1, policy_version 579580 (0.0011) [2023-12-26 19:34:50,639][105620] Updated weights for policy 1, policy_version 579590 (0.0007) [2023-12-26 19:34:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 296566784. Throughput: 0: 9784.2, 1: 9637.0. Samples: 296557048. Policy #0 lag: (min: 19.0, avg: 30.8, max: 51.0) [2023-12-26 19:34:51,063][104569] Avg episode reward: [(0, '9264.260'), (1, '9074.959')] [2023-12-26 19:34:51,175][105692] Updated weights for policy 0, policy_version 578757 (0.0010) [2023-12-26 19:34:51,227][105692] Updated weights for policy 0, policy_version 578767 (0.0009) [2023-12-26 19:34:51,283][105692] Updated weights for policy 0, policy_version 578777 (0.0008) [2023-12-26 19:34:51,304][105620] Updated weights for policy 1, policy_version 579600 (0.0007) [2023-12-26 19:34:51,370][105620] Updated weights for policy 1, policy_version 579610 (0.0009) [2023-12-26 19:34:51,432][105620] Updated weights for policy 1, policy_version 579620 (0.0008) [2023-12-26 19:34:52,078][105692] Updated weights for policy 0, policy_version 578787 (0.0007) [2023-12-26 19:34:52,129][105692] Updated weights for policy 0, policy_version 578797 (0.0009) [2023-12-26 19:34:52,179][105692] Updated weights for policy 0, policy_version 578807 (0.0009) [2023-12-26 19:34:52,203][105620] Updated weights for policy 1, policy_version 579630 (0.0008) [2023-12-26 19:34:52,260][105620] Updated weights for policy 1, policy_version 579640 (0.0007) [2023-12-26 19:34:52,318][105620] Updated weights for policy 1, policy_version 579650 (0.0009) [2023-12-26 19:34:52,971][105692] Updated weights for policy 0, policy_version 578817 (0.0007) [2023-12-26 19:34:53,028][105692] Updated weights for policy 0, policy_version 578827 (0.0009) [2023-12-26 19:34:53,048][105620] Updated weights for policy 1, policy_version 579660 (0.0008) [2023-12-26 19:34:53,078][105692] Updated weights for policy 0, policy_version 578837 (0.0007) [2023-12-26 19:34:53,096][105620] Updated weights for policy 1, policy_version 579670 (0.0007) [2023-12-26 19:34:53,126][105692] Updated weights for policy 0, policy_version 578847 (0.0007) [2023-12-26 19:34:53,151][105620] Updated weights for policy 1, policy_version 579680 (0.0005) [2023-12-26 19:34:53,848][105692] Updated weights for policy 0, policy_version 578857 (0.0005) [2023-12-26 19:34:53,865][105620] Updated weights for policy 1, policy_version 579690 (0.0006) [2023-12-26 19:34:53,900][105692] Updated weights for policy 0, policy_version 578867 (0.0006) [2023-12-26 19:34:53,922][105620] Updated weights for policy 1, policy_version 579700 (0.0006) [2023-12-26 19:34:53,953][105692] Updated weights for policy 0, policy_version 578877 (0.0006) [2023-12-26 19:34:53,974][105620] Updated weights for policy 1, policy_version 579710 (0.0005) [2023-12-26 19:34:54,040][105620] Updated weights for policy 1, policy_version 579720 (0.0008) [2023-12-26 19:34:54,507][105692] Updated weights for policy 0, policy_version 578887 (0.0006) [2023-12-26 19:34:54,560][105692] Updated weights for policy 0, policy_version 578897 (0.0005) [2023-12-26 19:34:54,610][105692] Updated weights for policy 0, policy_version 578907 (0.0006) [2023-12-26 19:34:54,769][105620] Updated weights for policy 1, policy_version 579730 (0.0009) [2023-12-26 19:34:54,826][105620] Updated weights for policy 1, policy_version 579740 (0.0008) [2023-12-26 19:34:54,879][105620] Updated weights for policy 1, policy_version 579750 (0.0008) [2023-12-26 19:34:55,213][105692] Updated weights for policy 0, policy_version 578917 (0.0005) [2023-12-26 19:34:55,266][105692] Updated weights for policy 0, policy_version 578927 (0.0005) [2023-12-26 19:34:55,319][105692] Updated weights for policy 0, policy_version 578937 (0.0005) [2023-12-26 19:34:55,794][105620] Updated weights for policy 1, policy_version 579760 (0.0008) [2023-12-26 19:34:55,841][105692] Updated weights for policy 0, policy_version 578947 (0.0007) [2023-12-26 19:34:55,848][105620] Updated weights for policy 1, policy_version 579770 (0.0007) [2023-12-26 19:34:55,900][105620] Updated weights for policy 1, policy_version 579780 (0.0009) [2023-12-26 19:34:55,903][105692] Updated weights for policy 0, policy_version 578957 (0.0010) [2023-12-26 19:34:55,953][105692] Updated weights for policy 0, policy_version 578967 (0.0008) [2023-12-26 19:34:56,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 296673280. Throughput: 0: 9870.2, 1: 9499.2. Samples: 296674828. Policy #0 lag: (min: 19.0, avg: 30.8, max: 51.0) [2023-12-26 19:34:56,063][104569] Avg episode reward: [(0, '9173.176'), (1, '9167.684')] [2023-12-26 19:34:56,553][105620] Updated weights for policy 1, policy_version 579790 (0.0006) [2023-12-26 19:34:56,601][105620] Updated weights for policy 1, policy_version 579800 (0.0005) [2023-12-26 19:34:56,622][105692] Updated weights for policy 0, policy_version 578977 (0.0010) [2023-12-26 19:34:56,647][105620] Updated weights for policy 1, policy_version 579810 (0.0005) [2023-12-26 19:34:56,686][105692] Updated weights for policy 0, policy_version 578987 (0.0007) [2023-12-26 19:34:56,750][105692] Updated weights for policy 0, policy_version 578997 (0.0008) [2023-12-26 19:34:56,811][105692] Updated weights for policy 0, policy_version 579007 (0.0009) [2023-12-26 19:34:57,366][105620] Updated weights for policy 1, policy_version 579820 (0.0007) [2023-12-26 19:34:57,417][105620] Updated weights for policy 1, policy_version 579830 (0.0008) [2023-12-26 19:34:57,472][105620] Updated weights for policy 1, policy_version 579840 (0.0008) [2023-12-26 19:34:57,519][105692] Updated weights for policy 0, policy_version 579017 (0.0010) [2023-12-26 19:34:57,577][105692] Updated weights for policy 0, policy_version 579027 (0.0010) [2023-12-26 19:34:57,624][105692] Updated weights for policy 0, policy_version 579037 (0.0010) [2023-12-26 19:34:58,249][105620] Updated weights for policy 1, policy_version 579850 (0.0007) [2023-12-26 19:34:58,313][105620] Updated weights for policy 1, policy_version 579860 (0.0008) [2023-12-26 19:34:58,378][105620] Updated weights for policy 1, policy_version 579870 (0.0008) [2023-12-26 19:34:58,404][105692] Updated weights for policy 0, policy_version 579047 (0.0009) [2023-12-26 19:34:58,442][105620] Updated weights for policy 1, policy_version 579880 (0.0008) [2023-12-26 19:34:58,462][105692] Updated weights for policy 0, policy_version 579057 (0.0008) [2023-12-26 19:34:58,526][105692] Updated weights for policy 0, policy_version 579067 (0.0008) [2023-12-26 19:34:59,114][105620] Updated weights for policy 1, policy_version 579890 (0.0006) [2023-12-26 19:34:59,162][105620] Updated weights for policy 1, policy_version 579900 (0.0010) [2023-12-26 19:34:59,214][105620] Updated weights for policy 1, policy_version 579910 (0.0010) [2023-12-26 19:34:59,265][105692] Updated weights for policy 0, policy_version 579077 (0.0009) [2023-12-26 19:34:59,317][105692] Updated weights for policy 0, policy_version 579087 (0.0010) [2023-12-26 19:34:59,383][105692] Updated weights for policy 0, policy_version 579097 (0.0008) [2023-12-26 19:34:59,838][105620] Updated weights for policy 1, policy_version 579920 (0.0008) [2023-12-26 19:34:59,912][105620] Updated weights for policy 1, policy_version 579930 (0.0007) [2023-12-26 19:34:59,976][105620] Updated weights for policy 1, policy_version 579940 (0.0009) [2023-12-26 19:35:00,131][105692] Updated weights for policy 0, policy_version 579107 (0.0010) [2023-12-26 19:35:00,190][105692] Updated weights for policy 0, policy_version 579117 (0.0006) [2023-12-26 19:35:00,247][105692] Updated weights for policy 0, policy_version 579127 (0.0005) [2023-12-26 19:35:00,623][105620] Updated weights for policy 1, policy_version 579950 (0.0008) [2023-12-26 19:35:00,677][105620] Updated weights for policy 1, policy_version 579960 (0.0010) [2023-12-26 19:35:00,735][105620] Updated weights for policy 1, policy_version 579970 (0.0010) [2023-12-26 19:35:00,960][105692] Updated weights for policy 0, policy_version 579137 (0.0008) [2023-12-26 19:35:01,015][105692] Updated weights for policy 0, policy_version 579147 (0.0008) [2023-12-26 19:35:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19605.2). Total num frames: 296763392. Throughput: 0: 9829.1, 1: 9559.9. Samples: 296733648. Policy #0 lag: (min: 19.0, avg: 30.8, max: 51.0) [2023-12-26 19:35:01,063][104569] Avg episode reward: [(0, '9175.115'), (1, '9261.157')] [2023-12-26 19:35:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000579976_148488192.pth... [2023-12-26 19:35:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000578824_148193280.pth [2023-12-26 19:35:01,090][105692] Updated weights for policy 0, policy_version 579157 (0.0008) [2023-12-26 19:35:01,158][105692] Updated weights for policy 0, policy_version 579167 (0.0008) [2023-12-26 19:35:01,162][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000579168_148283392.pth... [2023-12-26 19:35:01,165][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000578016_147988480.pth [2023-12-26 19:35:01,415][105620] Updated weights for policy 1, policy_version 579980 (0.0010) [2023-12-26 19:35:01,481][105620] Updated weights for policy 1, policy_version 579990 (0.0011) [2023-12-26 19:35:01,540][105620] Updated weights for policy 1, policy_version 580000 (0.0011) [2023-12-26 19:35:01,901][105692] Updated weights for policy 0, policy_version 579177 (0.0008) [2023-12-26 19:35:01,959][105692] Updated weights for policy 0, policy_version 579187 (0.0008) [2023-12-26 19:35:02,018][105692] Updated weights for policy 0, policy_version 579197 (0.0008) [2023-12-26 19:35:02,296][105620] Updated weights for policy 1, policy_version 580010 (0.0011) [2023-12-26 19:35:02,356][105620] Updated weights for policy 1, policy_version 580020 (0.0011) [2023-12-26 19:35:02,419][105620] Updated weights for policy 1, policy_version 580030 (0.0011) [2023-12-26 19:35:02,475][105620] Updated weights for policy 1, policy_version 580040 (0.0011) [2023-12-26 19:35:02,712][105692] Updated weights for policy 0, policy_version 579207 (0.0008) [2023-12-26 19:35:02,757][105692] Updated weights for policy 0, policy_version 579217 (0.0008) [2023-12-26 19:35:02,803][105692] Updated weights for policy 0, policy_version 579227 (0.0008) [2023-12-26 19:35:03,111][105620] Updated weights for policy 1, policy_version 580050 (0.0010) [2023-12-26 19:35:03,159][105620] Updated weights for policy 1, policy_version 580060 (0.0010) [2023-12-26 19:35:03,207][105620] Updated weights for policy 1, policy_version 580070 (0.0010) [2023-12-26 19:35:03,537][105692] Updated weights for policy 0, policy_version 579238 (0.0008) [2023-12-26 19:35:03,583][105692] Updated weights for policy 0, policy_version 579248 (0.0005) [2023-12-26 19:35:03,639][105692] Updated weights for policy 0, policy_version 579258 (0.0008) [2023-12-26 19:35:03,794][105620] Updated weights for policy 1, policy_version 580080 (0.0006) [2023-12-26 19:35:03,867][105620] Updated weights for policy 1, policy_version 580090 (0.0007) [2023-12-26 19:35:03,918][105620] Updated weights for policy 1, policy_version 580100 (0.0008) [2023-12-26 19:35:04,303][105692] Updated weights for policy 0, policy_version 579268 (0.0009) [2023-12-26 19:35:04,361][105692] Updated weights for policy 0, policy_version 579278 (0.0009) [2023-12-26 19:35:04,416][105692] Updated weights for policy 0, policy_version 579288 (0.0009) [2023-12-26 19:35:04,574][105620] Updated weights for policy 1, policy_version 580110 (0.0009) [2023-12-26 19:35:04,630][105620] Updated weights for policy 1, policy_version 580120 (0.0009) [2023-12-26 19:35:04,685][105620] Updated weights for policy 1, policy_version 580130 (0.0009) [2023-12-26 19:35:05,190][105692] Updated weights for policy 0, policy_version 579298 (0.0009) [2023-12-26 19:35:05,253][105692] Updated weights for policy 0, policy_version 579308 (0.0009) [2023-12-26 19:35:05,316][105692] Updated weights for policy 0, policy_version 579318 (0.0009) [2023-12-26 19:35:05,350][105620] Updated weights for policy 1, policy_version 580140 (0.0007) [2023-12-26 19:35:05,374][105692] Updated weights for policy 0, policy_version 579328 (0.0009) [2023-12-26 19:35:05,407][105620] Updated weights for policy 1, policy_version 580150 (0.0005) [2023-12-26 19:35:05,471][105620] Updated weights for policy 1, policy_version 580160 (0.0007) [2023-12-26 19:35:06,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 296861696. Throughput: 0: 9845.3, 1: 9674.0. Samples: 296854924. Policy #0 lag: (min: 19.0, avg: 30.8, max: 51.0) [2023-12-26 19:35:06,063][104569] Avg episode reward: [(0, '8908.017'), (1, '9170.469')] [2023-12-26 19:35:06,120][105692] Updated weights for policy 0, policy_version 579338 (0.0008) [2023-12-26 19:35:06,181][105620] Updated weights for policy 1, policy_version 580170 (0.0009) [2023-12-26 19:35:06,185][105692] Updated weights for policy 0, policy_version 579348 (0.0009) [2023-12-26 19:35:06,240][105620] Updated weights for policy 1, policy_version 580180 (0.0007) [2023-12-26 19:35:06,247][105692] Updated weights for policy 0, policy_version 579358 (0.0011) [2023-12-26 19:35:06,301][105620] Updated weights for policy 1, policy_version 580190 (0.0009) [2023-12-26 19:35:06,364][105620] Updated weights for policy 1, policy_version 580200 (0.0008) [2023-12-26 19:35:06,984][105692] Updated weights for policy 0, policy_version 579368 (0.0009) [2023-12-26 19:35:07,046][105692] Updated weights for policy 0, policy_version 579378 (0.0009) [2023-12-26 19:35:07,106][105692] Updated weights for policy 0, policy_version 579388 (0.0008) [2023-12-26 19:35:07,119][105620] Updated weights for policy 1, policy_version 580210 (0.0007) [2023-12-26 19:35:07,166][105620] Updated weights for policy 1, policy_version 580220 (0.0008) [2023-12-26 19:35:07,213][105620] Updated weights for policy 1, policy_version 580230 (0.0009) [2023-12-26 19:35:07,744][105692] Updated weights for policy 0, policy_version 579398 (0.0010) [2023-12-26 19:35:07,799][105692] Updated weights for policy 0, policy_version 579409 (0.0010) [2023-12-26 19:35:07,851][105692] Updated weights for policy 0, policy_version 579420 (0.0010) [2023-12-26 19:35:07,880][105620] Updated weights for policy 1, policy_version 580240 (0.0006) [2023-12-26 19:35:07,931][105620] Updated weights for policy 1, policy_version 580250 (0.0009) [2023-12-26 19:35:07,988][105620] Updated weights for policy 1, policy_version 580260 (0.0008) [2023-12-26 19:35:08,571][105692] Updated weights for policy 0, policy_version 579430 (0.0008) [2023-12-26 19:35:08,632][105692] Updated weights for policy 0, policy_version 579440 (0.0007) [2023-12-26 19:35:08,687][105692] Updated weights for policy 0, policy_version 579450 (0.0008) [2023-12-26 19:35:08,694][105620] Updated weights for policy 1, policy_version 580270 (0.0007) [2023-12-26 19:35:08,755][105620] Updated weights for policy 1, policy_version 580280 (0.0007) [2023-12-26 19:35:08,813][105620] Updated weights for policy 1, policy_version 580290 (0.0008) [2023-12-26 19:35:09,471][105692] Updated weights for policy 0, policy_version 579460 (0.0008) [2023-12-26 19:35:09,527][105692] Updated weights for policy 0, policy_version 579470 (0.0008) [2023-12-26 19:35:09,573][105620] Updated weights for policy 1, policy_version 580300 (0.0009) [2023-12-26 19:35:09,591][105692] Updated weights for policy 0, policy_version 579480 (0.0009) [2023-12-26 19:35:09,628][105620] Updated weights for policy 1, policy_version 580310 (0.0010) [2023-12-26 19:35:09,692][105620] Updated weights for policy 1, policy_version 580320 (0.0010) [2023-12-26 19:35:10,388][105692] Updated weights for policy 0, policy_version 579490 (0.0007) [2023-12-26 19:35:10,426][105585] KL-divergence is very high: 104.8219 [2023-12-26 19:35:10,451][105692] Updated weights for policy 0, policy_version 579500 (0.0005) [2023-12-26 19:35:10,453][105620] Updated weights for policy 1, policy_version 580330 (0.0010) [2023-12-26 19:35:10,479][105585] KL-divergence is very high: 134.9775 [2023-12-26 19:35:10,505][105620] Updated weights for policy 1, policy_version 580340 (0.0010) [2023-12-26 19:35:10,519][105692] Updated weights for policy 0, policy_version 579510 (0.0005) [2023-12-26 19:35:10,525][105585] KL-divergence is very high: 102.0234 [2023-12-26 19:35:10,531][105585] KL-divergence is very high: 101.9525 [2023-12-26 19:35:10,550][105620] Updated weights for policy 1, policy_version 580350 (0.0009) [2023-12-26 19:35:10,577][105692] Updated weights for policy 0, policy_version 579520 (0.0008) [2023-12-26 19:35:10,600][105620] Updated weights for policy 1, policy_version 580360 (0.0005) [2023-12-26 19:35:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 296960000. Throughput: 0: 9843.6, 1: 9736.8. Samples: 296970352. Policy #0 lag: (min: 19.0, avg: 30.8, max: 51.0) [2023-12-26 19:35:11,062][104569] Avg episode reward: [(0, '4917.024'), (1, '9170.503')] [2023-12-26 19:35:11,198][105620] Updated weights for policy 1, policy_version 580370 (0.0007) [2023-12-26 19:35:11,269][105620] Updated weights for policy 1, policy_version 580380 (0.0006) [2023-12-26 19:35:11,342][105620] Updated weights for policy 1, policy_version 580390 (0.0007) [2023-12-26 19:35:11,424][105585] KL-divergence is very high: 104.3294 [2023-12-26 19:35:11,435][105692] Updated weights for policy 0, policy_version 579530 (0.0007) [2023-12-26 19:35:11,475][105585] KL-divergence is very high: 112.7673 [2023-12-26 19:35:11,496][105692] Updated weights for policy 0, policy_version 579540 (0.0008) [2023-12-26 19:35:11,567][105692] Updated weights for policy 0, policy_version 579550 (0.0006) [2023-12-26 19:35:12,069][105620] Updated weights for policy 1, policy_version 580400 (0.0009) [2023-12-26 19:35:12,121][105620] Updated weights for policy 1, policy_version 580410 (0.0009) [2023-12-26 19:35:12,177][105620] Updated weights for policy 1, policy_version 580420 (0.0009) [2023-12-26 19:35:12,242][105692] Updated weights for policy 0, policy_version 579560 (0.0007) [2023-12-26 19:35:12,304][105692] Updated weights for policy 0, policy_version 579570 (0.0008) [2023-12-26 19:35:12,375][105692] Updated weights for policy 0, policy_version 579580 (0.0008) [2023-12-26 19:35:12,861][105620] Updated weights for policy 1, policy_version 580430 (0.0006) [2023-12-26 19:35:12,912][105620] Updated weights for policy 1, policy_version 580440 (0.0010) [2023-12-26 19:35:12,960][105620] Updated weights for policy 1, policy_version 580450 (0.0010) [2023-12-26 19:35:13,179][105692] Updated weights for policy 0, policy_version 579590 (0.0006) [2023-12-26 19:35:13,234][105692] Updated weights for policy 0, policy_version 579600 (0.0009) [2023-12-26 19:35:13,278][105692] Updated weights for policy 0, policy_version 579610 (0.0008) [2023-12-26 19:35:13,614][105620] Updated weights for policy 1, policy_version 580460 (0.0008) [2023-12-26 19:35:13,667][105620] Updated weights for policy 1, policy_version 580470 (0.0007) [2023-12-26 19:35:13,728][105620] Updated weights for policy 1, policy_version 580480 (0.0010) [2023-12-26 19:35:14,126][105692] Updated weights for policy 0, policy_version 579620 (0.0008) [2023-12-26 19:35:14,182][105692] Updated weights for policy 0, policy_version 579630 (0.0008) [2023-12-26 19:35:14,229][105692] Updated weights for policy 0, policy_version 579640 (0.0008) [2023-12-26 19:35:14,341][105620] Updated weights for policy 1, policy_version 580490 (0.0009) [2023-12-26 19:35:14,395][105620] Updated weights for policy 1, policy_version 580500 (0.0005) [2023-12-26 19:35:14,449][105620] Updated weights for policy 1, policy_version 580510 (0.0005) [2023-12-26 19:35:14,501][105620] Updated weights for policy 1, policy_version 580520 (0.0005) [2023-12-26 19:35:15,059][105692] Updated weights for policy 0, policy_version 579650 (0.0008) [2023-12-26 19:35:15,127][105692] Updated weights for policy 0, policy_version 579660 (0.0006) [2023-12-26 19:35:15,136][105620] Updated weights for policy 1, policy_version 580530 (0.0007) [2023-12-26 19:35:15,195][105692] Updated weights for policy 0, policy_version 579670 (0.0007) [2023-12-26 19:35:15,196][105620] Updated weights for policy 1, policy_version 580540 (0.0008) [2023-12-26 19:35:15,246][105620] Updated weights for policy 1, policy_version 580550 (0.0006) [2023-12-26 19:35:15,257][105692] Updated weights for policy 0, policy_version 579680 (0.0008) [2023-12-26 19:35:15,936][105620] Updated weights for policy 1, policy_version 580560 (0.0007) [2023-12-26 19:35:15,984][105620] Updated weights for policy 1, policy_version 580570 (0.0006) [2023-12-26 19:35:16,045][105620] Updated weights for policy 1, policy_version 580580 (0.0006) [2023-12-26 19:35:16,051][105692] Updated weights for policy 0, policy_version 579690 (0.0008) [2023-12-26 19:35:16,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 297058304. Throughput: 0: 9698.4, 1: 9835.8. Samples: 297027564. Policy #0 lag: (min: 19.0, avg: 30.8, max: 51.0) [2023-12-26 19:35:16,062][104569] Avg episode reward: [(0, '6107.739'), (1, '9080.212')] [2023-12-26 19:35:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000580584_148643840.pth... [2023-12-26 19:35:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000579400_148340736.pth [2023-12-26 19:35:16,109][105692] Updated weights for policy 0, policy_version 579700 (0.0009) [2023-12-26 19:35:16,160][105692] Updated weights for policy 0, policy_version 579710 (0.0009) [2023-12-26 19:35:16,170][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000579712_148422656.pth... [2023-12-26 19:35:16,184][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000578592_148135936.pth [2023-12-26 19:35:16,761][105620] Updated weights for policy 1, policy_version 580590 (0.0006) [2023-12-26 19:35:16,813][105620] Updated weights for policy 1, policy_version 580600 (0.0005) [2023-12-26 19:35:16,841][105692] Updated weights for policy 0, policy_version 579720 (0.0006) [2023-12-26 19:35:16,859][105620] Updated weights for policy 1, policy_version 580610 (0.0005) [2023-12-26 19:35:16,890][105692] Updated weights for policy 0, policy_version 579730 (0.0005) [2023-12-26 19:35:16,928][105585] KL-divergence is very high: 147.5977 [2023-12-26 19:35:16,940][105692] Updated weights for policy 0, policy_version 579740 (0.0005) [2023-12-26 19:35:17,596][105620] Updated weights for policy 1, policy_version 580620 (0.0010) [2023-12-26 19:35:17,602][105692] Updated weights for policy 0, policy_version 579750 (0.0006) [2023-12-26 19:35:17,649][105620] Updated weights for policy 1, policy_version 580630 (0.0011) [2023-12-26 19:35:17,655][105692] Updated weights for policy 0, policy_version 579760 (0.0006) [2023-12-26 19:35:17,685][105585] KL-divergence is very high: 205.5826 [2023-12-26 19:35:17,707][105692] Updated weights for policy 0, policy_version 579770 (0.0006) [2023-12-26 19:35:17,709][105620] Updated weights for policy 1, policy_version 580640 (0.0011) [2023-12-26 19:35:17,729][105585] KL-divergence is very high: 110.2789 [2023-12-26 19:35:18,397][105620] Updated weights for policy 1, policy_version 580650 (0.0011) [2023-12-26 19:35:18,450][105620] Updated weights for policy 1, policy_version 580660 (0.0011) [2023-12-26 19:35:18,506][105620] Updated weights for policy 1, policy_version 580670 (0.0011) [2023-12-26 19:35:18,522][105692] Updated weights for policy 0, policy_version 579780 (0.0009) [2023-12-26 19:35:18,559][105620] Updated weights for policy 1, policy_version 580680 (0.0011) [2023-12-26 19:35:18,583][105692] Updated weights for policy 0, policy_version 579790 (0.0009) [2023-12-26 19:35:18,642][105692] Updated weights for policy 0, policy_version 579800 (0.0008) [2023-12-26 19:35:19,321][105620] Updated weights for policy 1, policy_version 580690 (0.0011) [2023-12-26 19:35:19,378][105692] Updated weights for policy 0, policy_version 579810 (0.0007) [2023-12-26 19:35:19,387][105620] Updated weights for policy 1, policy_version 580700 (0.0010) [2023-12-26 19:35:19,432][105692] Updated weights for policy 0, policy_version 579820 (0.0007) [2023-12-26 19:35:19,436][105620] Updated weights for policy 1, policy_version 580710 (0.0010) [2023-12-26 19:35:19,484][105692] Updated weights for policy 0, policy_version 579830 (0.0008) [2023-12-26 19:35:19,534][105692] Updated weights for policy 0, policy_version 579840 (0.0007) [2023-12-26 19:35:20,174][105620] Updated weights for policy 1, policy_version 580720 (0.0011) [2023-12-26 19:35:20,234][105620] Updated weights for policy 1, policy_version 580730 (0.0011) [2023-12-26 19:35:20,292][105620] Updated weights for policy 1, policy_version 580740 (0.0011) [2023-12-26 19:35:20,302][105692] Updated weights for policy 0, policy_version 579850 (0.0007) [2023-12-26 19:35:20,365][105692] Updated weights for policy 0, policy_version 579860 (0.0007) [2023-12-26 19:35:20,426][105692] Updated weights for policy 0, policy_version 579870 (0.0008) [2023-12-26 19:35:21,059][105620] Updated weights for policy 1, policy_version 580750 (0.0011) [2023-12-26 19:35:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 297148416. Throughput: 0: 9592.5, 1: 9907.0. Samples: 297142836. Policy #0 lag: (min: 19.0, avg: 30.8, max: 51.0) [2023-12-26 19:35:21,062][104569] Avg episode reward: [(0, '8597.926'), (1, '9169.402')] [2023-12-26 19:35:21,133][105620] Updated weights for policy 1, policy_version 580760 (0.0010) [2023-12-26 19:35:21,200][105620] Updated weights for policy 1, policy_version 580770 (0.0010) [2023-12-26 19:35:21,211][105692] Updated weights for policy 0, policy_version 579880 (0.0007) [2023-12-26 19:35:21,273][105692] Updated weights for policy 0, policy_version 579890 (0.0007) [2023-12-26 19:35:21,329][105692] Updated weights for policy 0, policy_version 579900 (0.0006) [2023-12-26 19:35:21,932][105620] Updated weights for policy 1, policy_version 580780 (0.0011) [2023-12-26 19:35:22,002][105620] Updated weights for policy 1, policy_version 580790 (0.0011) [2023-12-26 19:35:22,057][105692] Updated weights for policy 0, policy_version 579910 (0.0006) [2023-12-26 19:35:22,068][105620] Updated weights for policy 1, policy_version 580800 (0.0010) [2023-12-26 19:35:22,110][105692] Updated weights for policy 0, policy_version 579920 (0.0006) [2023-12-26 19:35:22,163][105692] Updated weights for policy 0, policy_version 579930 (0.0008) [2023-12-26 19:35:22,731][105620] Updated weights for policy 1, policy_version 580810 (0.0010) [2023-12-26 19:35:22,798][105620] Updated weights for policy 1, policy_version 580820 (0.0011) [2023-12-26 19:35:22,863][105620] Updated weights for policy 1, policy_version 580830 (0.0012) [2023-12-26 19:35:22,933][105620] Updated weights for policy 1, policy_version 580840 (0.0010) [2023-12-26 19:35:22,940][105692] Updated weights for policy 0, policy_version 579940 (0.0009) [2023-12-26 19:35:22,985][105692] Updated weights for policy 0, policy_version 579950 (0.0010) [2023-12-26 19:35:23,048][105692] Updated weights for policy 0, policy_version 579960 (0.0010) [2023-12-26 19:35:23,646][105620] Updated weights for policy 1, policy_version 580850 (0.0005) [2023-12-26 19:35:23,702][105620] Updated weights for policy 1, policy_version 580860 (0.0010) [2023-12-26 19:35:23,753][105620] Updated weights for policy 1, policy_version 580870 (0.0010) [2023-12-26 19:35:23,806][105692] Updated weights for policy 0, policy_version 579970 (0.0011) [2023-12-26 19:35:23,864][105692] Updated weights for policy 0, policy_version 579980 (0.0010) [2023-12-26 19:35:23,924][105692] Updated weights for policy 0, policy_version 579990 (0.0010) [2023-12-26 19:35:23,985][105692] Updated weights for policy 0, policy_version 580000 (0.0005) [2023-12-26 19:35:24,412][105620] Updated weights for policy 1, policy_version 580880 (0.0006) [2023-12-26 19:35:24,465][105620] Updated weights for policy 1, policy_version 580890 (0.0009) [2023-12-26 19:35:24,524][105620] Updated weights for policy 1, policy_version 580900 (0.0011) [2023-12-26 19:35:24,552][105692] Updated weights for policy 0, policy_version 580010 (0.0006) [2023-12-26 19:35:24,598][105692] Updated weights for policy 0, policy_version 580020 (0.0005) [2023-12-26 19:35:24,656][105692] Updated weights for policy 0, policy_version 580030 (0.0005) [2023-12-26 19:35:25,202][105692] Updated weights for policy 0, policy_version 580040 (0.0005) [2023-12-26 19:35:25,244][105620] Updated weights for policy 1, policy_version 580910 (0.0010) [2023-12-26 19:35:25,247][105692] Updated weights for policy 0, policy_version 580050 (0.0005) [2023-12-26 19:35:25,292][105692] Updated weights for policy 0, policy_version 580060 (0.0005) [2023-12-26 19:35:25,303][105620] Updated weights for policy 1, policy_version 580920 (0.0010) [2023-12-26 19:35:25,362][105620] Updated weights for policy 1, policy_version 580930 (0.0011) [2023-12-26 19:35:25,929][105692] Updated weights for policy 0, policy_version 580070 (0.0008) [2023-12-26 19:35:25,990][105692] Updated weights for policy 0, policy_version 580080 (0.0007) [2023-12-26 19:35:26,052][105692] Updated weights for policy 0, policy_version 580090 (0.0005) [2023-12-26 19:35:26,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 297246720. Throughput: 0: 9612.3, 1: 9857.7. Samples: 297260420. Policy #0 lag: (min: 19.0, avg: 30.8, max: 51.0) [2023-12-26 19:35:26,063][104569] Avg episode reward: [(0, '8193.820'), (1, '9257.666')] [2023-12-26 19:35:26,107][105620] Updated weights for policy 1, policy_version 580940 (0.0011) [2023-12-26 19:35:26,165][105620] Updated weights for policy 1, policy_version 580950 (0.0011) [2023-12-26 19:35:26,228][105620] Updated weights for policy 1, policy_version 580960 (0.0011) [2023-12-26 19:35:26,641][105585] KL-divergence is very high: 147.4933 [2023-12-26 19:35:26,645][105692] Updated weights for policy 0, policy_version 580100 (0.0006) [2023-12-26 19:35:26,703][105692] Updated weights for policy 0, policy_version 580110 (0.0010) [2023-12-26 19:35:26,764][105692] Updated weights for policy 0, policy_version 580120 (0.0010) [2023-12-26 19:35:26,969][105620] Updated weights for policy 1, policy_version 580970 (0.0010) [2023-12-26 19:35:27,013][105620] Updated weights for policy 1, policy_version 580980 (0.0010) [2023-12-26 19:35:27,060][105620] Updated weights for policy 1, policy_version 580990 (0.0010) [2023-12-26 19:35:27,124][105620] Updated weights for policy 1, policy_version 581000 (0.0010) [2023-12-26 19:35:27,308][105692] Updated weights for policy 0, policy_version 580130 (0.0007) [2023-12-26 19:35:27,366][105692] Updated weights for policy 0, policy_version 580140 (0.0007) [2023-12-26 19:35:27,418][105692] Updated weights for policy 0, policy_version 580150 (0.0005) [2023-12-26 19:35:27,471][105692] Updated weights for policy 0, policy_version 580160 (0.0005) [2023-12-26 19:35:27,877][105620] Updated weights for policy 1, policy_version 581010 (0.0010) [2023-12-26 19:35:27,933][105620] Updated weights for policy 1, policy_version 581020 (0.0010) [2023-12-26 19:35:27,982][105620] Updated weights for policy 1, policy_version 581030 (0.0010) [2023-12-26 19:35:28,102][105692] Updated weights for policy 0, policy_version 580170 (0.0007) [2023-12-26 19:35:28,158][105692] Updated weights for policy 0, policy_version 580180 (0.0005) [2023-12-26 19:35:28,212][105692] Updated weights for policy 0, policy_version 580190 (0.0005) [2023-12-26 19:35:28,735][105620] Updated weights for policy 1, policy_version 581040 (0.0009) [2023-12-26 19:35:28,794][105620] Updated weights for policy 1, policy_version 581050 (0.0007) [2023-12-26 19:35:28,796][105692] Updated weights for policy 0, policy_version 580200 (0.0007) [2023-12-26 19:35:28,855][105620] Updated weights for policy 1, policy_version 581060 (0.0006) [2023-12-26 19:35:28,857][105692] Updated weights for policy 0, policy_version 580210 (0.0011) [2023-12-26 19:35:28,913][105692] Updated weights for policy 0, policy_version 580220 (0.0011) [2023-12-26 19:35:29,540][105692] Updated weights for policy 0, policy_version 580230 (0.0005) [2023-12-26 19:35:29,592][105692] Updated weights for policy 0, policy_version 580240 (0.0005) [2023-12-26 19:35:29,643][105692] Updated weights for policy 0, policy_version 580250 (0.0010) [2023-12-26 19:35:29,682][105620] Updated weights for policy 1, policy_version 581070 (0.0006) [2023-12-26 19:35:29,726][105620] Updated weights for policy 1, policy_version 581080 (0.0008) [2023-12-26 19:35:29,774][105620] Updated weights for policy 1, policy_version 581090 (0.0008) [2023-12-26 19:35:30,372][105692] Updated weights for policy 0, policy_version 580260 (0.0009) [2023-12-26 19:35:30,434][105692] Updated weights for policy 0, policy_version 580270 (0.0008) [2023-12-26 19:35:30,496][105692] Updated weights for policy 0, policy_version 580280 (0.0008) [2023-12-26 19:35:30,505][105620] Updated weights for policy 1, policy_version 581100 (0.0009) [2023-12-26 19:35:30,576][105620] Updated weights for policy 1, policy_version 581110 (0.0008) [2023-12-26 19:35:30,637][105620] Updated weights for policy 1, policy_version 581120 (0.0008) [2023-12-26 19:35:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 297353216. Throughput: 0: 9756.3, 1: 9858.1. Samples: 297322728. Policy #0 lag: (min: 19.0, avg: 30.8, max: 51.0) [2023-12-26 19:35:31,062][104569] Avg episode reward: [(0, '7475.498'), (1, '9257.463')] [2023-12-26 19:35:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000581128_148783104.pth... [2023-12-26 19:35:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000579976_148488192.pth [2023-12-26 19:35:31,084][105692] Updated weights for policy 0, policy_version 580290 (0.0007) [2023-12-26 19:35:31,149][105692] Updated weights for policy 0, policy_version 580300 (0.0010) [2023-12-26 19:35:31,201][105692] Updated weights for policy 0, policy_version 580310 (0.0010) [2023-12-26 19:35:31,258][105620] Updated weights for policy 1, policy_version 581130 (0.0008) [2023-12-26 19:35:31,261][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000580320_148578304.pth... [2023-12-26 19:35:31,261][105692] Updated weights for policy 0, policy_version 580320 (0.0009) [2023-12-26 19:35:31,264][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000579168_148283392.pth [2023-12-26 19:35:31,312][105620] Updated weights for policy 1, policy_version 581140 (0.0008) [2023-12-26 19:35:31,377][105620] Updated weights for policy 1, policy_version 581150 (0.0008) [2023-12-26 19:35:31,433][105620] Updated weights for policy 1, policy_version 581160 (0.0008) [2023-12-26 19:35:32,013][105692] Updated weights for policy 0, policy_version 580330 (0.0009) [2023-12-26 19:35:32,073][105692] Updated weights for policy 0, policy_version 580340 (0.0006) [2023-12-26 19:35:32,120][105692] Updated weights for policy 0, policy_version 580350 (0.0005) [2023-12-26 19:35:32,216][105620] Updated weights for policy 1, policy_version 581170 (0.0009) [2023-12-26 19:35:32,269][105620] Updated weights for policy 1, policy_version 581180 (0.0008) [2023-12-26 19:35:32,330][105620] Updated weights for policy 1, policy_version 581190 (0.0007) [2023-12-26 19:35:32,683][105692] Updated weights for policy 0, policy_version 580360 (0.0005) [2023-12-26 19:35:32,731][105692] Updated weights for policy 0, policy_version 580370 (0.0005) [2023-12-26 19:35:32,780][105692] Updated weights for policy 0, policy_version 580380 (0.0005) [2023-12-26 19:35:33,113][105620] Updated weights for policy 1, policy_version 581200 (0.0009) [2023-12-26 19:35:33,162][105620] Updated weights for policy 1, policy_version 581210 (0.0010) [2023-12-26 19:35:33,216][105620] Updated weights for policy 1, policy_version 581220 (0.0009) [2023-12-26 19:35:33,422][105692] Updated weights for policy 0, policy_version 580390 (0.0007) [2023-12-26 19:35:33,467][105692] Updated weights for policy 0, policy_version 580400 (0.0008) [2023-12-26 19:35:33,522][105692] Updated weights for policy 0, policy_version 580410 (0.0005) [2023-12-26 19:35:33,878][105620] Updated weights for policy 1, policy_version 581230 (0.0005) [2023-12-26 19:35:33,925][105620] Updated weights for policy 1, policy_version 581240 (0.0005) [2023-12-26 19:35:33,976][105620] Updated weights for policy 1, policy_version 581250 (0.0007) [2023-12-26 19:35:34,192][105692] Updated weights for policy 0, policy_version 580420 (0.0006) [2023-12-26 19:35:34,257][105692] Updated weights for policy 0, policy_version 580430 (0.0010) [2023-12-26 19:35:34,316][105692] Updated weights for policy 0, policy_version 580440 (0.0011) [2023-12-26 19:35:34,710][105620] Updated weights for policy 1, policy_version 581260 (0.0010) [2023-12-26 19:35:34,769][105620] Updated weights for policy 1, policy_version 581270 (0.0011) [2023-12-26 19:35:34,834][105620] Updated weights for policy 1, policy_version 581280 (0.0010) [2023-12-26 19:35:35,067][105692] Updated weights for policy 0, policy_version 580450 (0.0010) [2023-12-26 19:35:35,121][105692] Updated weights for policy 0, policy_version 580460 (0.0010) [2023-12-26 19:35:35,176][105692] Updated weights for policy 0, policy_version 580470 (0.0010) [2023-12-26 19:35:35,234][105692] Updated weights for policy 0, policy_version 580480 (0.0010) [2023-12-26 19:35:35,538][105620] Updated weights for policy 1, policy_version 581290 (0.0010) [2023-12-26 19:35:35,589][105620] Updated weights for policy 1, policy_version 581300 (0.0008) [2023-12-26 19:35:35,643][105620] Updated weights for policy 1, policy_version 581310 (0.0008) [2023-12-26 19:35:35,702][105620] Updated weights for policy 1, policy_version 581320 (0.0007) [2023-12-26 19:35:35,998][105692] Updated weights for policy 0, policy_version 580490 (0.0011) [2023-12-26 19:35:36,060][105692] Updated weights for policy 0, policy_version 580500 (0.0011) [2023-12-26 19:35:36,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 297451520. Throughput: 0: 9888.5, 1: 9819.2. Samples: 297443892. Policy #0 lag: (min: 19.0, avg: 30.8, max: 51.0) [2023-12-26 19:35:36,062][104569] Avg episode reward: [(0, '7919.826'), (1, '9349.013')] [2023-12-26 19:35:36,116][105692] Updated weights for policy 0, policy_version 580510 (0.0010) [2023-12-26 19:35:36,392][105620] Updated weights for policy 1, policy_version 581330 (0.0006) [2023-12-26 19:35:36,453][105620] Updated weights for policy 1, policy_version 581340 (0.0010) [2023-12-26 19:35:36,521][105620] Updated weights for policy 1, policy_version 581350 (0.0009) [2023-12-26 19:35:36,781][105692] Updated weights for policy 0, policy_version 580520 (0.0007) [2023-12-26 19:35:36,840][105692] Updated weights for policy 0, policy_version 580530 (0.0008) [2023-12-26 19:35:36,895][105692] Updated weights for policy 0, policy_version 580540 (0.0010) [2023-12-26 19:35:37,181][105620] Updated weights for policy 1, policy_version 581360 (0.0007) [2023-12-26 19:35:37,239][105620] Updated weights for policy 1, policy_version 581370 (0.0005) [2023-12-26 19:35:37,304][105620] Updated weights for policy 1, policy_version 581380 (0.0007) [2023-12-26 19:35:37,495][105692] Updated weights for policy 0, policy_version 580550 (0.0011) [2023-12-26 19:35:37,553][105692] Updated weights for policy 0, policy_version 580560 (0.0010) [2023-12-26 19:35:37,606][105692] Updated weights for policy 0, policy_version 580570 (0.0010) [2023-12-26 19:35:37,899][105620] Updated weights for policy 1, policy_version 581390 (0.0009) [2023-12-26 19:35:37,955][105620] Updated weights for policy 1, policy_version 581400 (0.0008) [2023-12-26 19:35:38,018][105620] Updated weights for policy 1, policy_version 581410 (0.0009) [2023-12-26 19:35:38,327][105692] Updated weights for policy 0, policy_version 580580 (0.0009) [2023-12-26 19:35:38,392][105692] Updated weights for policy 0, policy_version 580590 (0.0009) [2023-12-26 19:35:38,446][105692] Updated weights for policy 0, policy_version 580600 (0.0009) [2023-12-26 19:35:38,774][105620] Updated weights for policy 1, policy_version 581420 (0.0009) [2023-12-26 19:35:38,838][105620] Updated weights for policy 1, policy_version 581430 (0.0009) [2023-12-26 19:35:38,892][105620] Updated weights for policy 1, policy_version 581440 (0.0009) [2023-12-26 19:35:39,219][105692] Updated weights for policy 0, policy_version 580610 (0.0010) [2023-12-26 19:35:39,275][105692] Updated weights for policy 0, policy_version 580620 (0.0010) [2023-12-26 19:35:39,330][105692] Updated weights for policy 0, policy_version 580630 (0.0009) [2023-12-26 19:35:39,396][105692] Updated weights for policy 0, policy_version 580640 (0.0008) [2023-12-26 19:35:39,648][105620] Updated weights for policy 1, policy_version 581450 (0.0009) [2023-12-26 19:35:39,707][105620] Updated weights for policy 1, policy_version 581460 (0.0009) [2023-12-26 19:35:39,772][105620] Updated weights for policy 1, policy_version 581470 (0.0006) [2023-12-26 19:35:39,832][105620] Updated weights for policy 1, policy_version 581480 (0.0009) [2023-12-26 19:35:40,128][105692] Updated weights for policy 0, policy_version 580650 (0.0009) [2023-12-26 19:35:40,183][105692] Updated weights for policy 0, policy_version 580660 (0.0009) [2023-12-26 19:35:40,235][105692] Updated weights for policy 0, policy_version 580670 (0.0008) [2023-12-26 19:35:40,619][105620] Updated weights for policy 1, policy_version 581490 (0.0010) [2023-12-26 19:35:40,677][105620] Updated weights for policy 1, policy_version 581500 (0.0010) [2023-12-26 19:35:40,732][105620] Updated weights for policy 1, policy_version 581510 (0.0009) [2023-12-26 19:35:40,913][105692] Updated weights for policy 0, policy_version 580680 (0.0009) [2023-12-26 19:35:40,975][105692] Updated weights for policy 0, policy_version 580690 (0.0009) [2023-12-26 19:35:41,025][105692] Updated weights for policy 0, policy_version 580700 (0.0006) [2023-12-26 19:35:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 297558016. Throughput: 0: 9829.4, 1: 9850.3. Samples: 297560416. Policy #0 lag: (min: 1.0, avg: 15.9, max: 33.0) [2023-12-26 19:35:41,062][104569] Avg episode reward: [(0, '8465.258'), (1, '9349.867')] [2023-12-26 19:35:41,466][105620] Updated weights for policy 1, policy_version 581520 (0.0009) [2023-12-26 19:35:41,521][105620] Updated weights for policy 1, policy_version 581530 (0.0009) [2023-12-26 19:35:41,587][105620] Updated weights for policy 1, policy_version 581540 (0.0009) [2023-12-26 19:35:41,782][105692] Updated weights for policy 0, policy_version 580710 (0.0009) [2023-12-26 19:35:41,845][105692] Updated weights for policy 0, policy_version 580720 (0.0009) [2023-12-26 19:35:41,909][105692] Updated weights for policy 0, policy_version 580730 (0.0010) [2023-12-26 19:35:42,373][105620] Updated weights for policy 1, policy_version 581550 (0.0008) [2023-12-26 19:35:42,429][105620] Updated weights for policy 1, policy_version 581560 (0.0008) [2023-12-26 19:35:42,485][105620] Updated weights for policy 1, policy_version 581570 (0.0008) [2023-12-26 19:35:42,675][105692] Updated weights for policy 0, policy_version 580740 (0.0010) [2023-12-26 19:35:42,741][105692] Updated weights for policy 0, policy_version 580750 (0.0009) [2023-12-26 19:35:42,803][105692] Updated weights for policy 0, policy_version 580760 (0.0010) [2023-12-26 19:35:43,190][105620] Updated weights for policy 1, policy_version 581580 (0.0009) [2023-12-26 19:35:43,245][105620] Updated weights for policy 1, policy_version 581590 (0.0010) [2023-12-26 19:35:43,295][105620] Updated weights for policy 1, policy_version 581600 (0.0010) [2023-12-26 19:35:43,501][105692] Updated weights for policy 0, policy_version 580770 (0.0009) [2023-12-26 19:35:43,556][105692] Updated weights for policy 0, policy_version 580780 (0.0008) [2023-12-26 19:35:43,612][105692] Updated weights for policy 0, policy_version 580790 (0.0009) [2023-12-26 19:35:43,673][105692] Updated weights for policy 0, policy_version 580800 (0.0010) [2023-12-26 19:35:43,965][105620] Updated weights for policy 1, policy_version 581610 (0.0010) [2023-12-26 19:35:44,011][105620] Updated weights for policy 1, policy_version 581620 (0.0005) [2023-12-26 19:35:44,067][105620] Updated weights for policy 1, policy_version 581630 (0.0006) [2023-12-26 19:35:44,132][105620] Updated weights for policy 1, policy_version 581640 (0.0006) [2023-12-26 19:35:44,393][105692] Updated weights for policy 0, policy_version 580810 (0.0010) [2023-12-26 19:35:44,444][105692] Updated weights for policy 0, policy_version 580820 (0.0009) [2023-12-26 19:35:44,502][105692] Updated weights for policy 0, policy_version 580830 (0.0006) [2023-12-26 19:35:44,723][105620] Updated weights for policy 1, policy_version 581651 (0.0010) [2023-12-26 19:35:44,787][105620] Updated weights for policy 1, policy_version 581661 (0.0009) [2023-12-26 19:35:44,858][105620] Updated weights for policy 1, policy_version 581671 (0.0010) [2023-12-26 19:35:45,099][105692] Updated weights for policy 0, policy_version 580840 (0.0008) [2023-12-26 19:35:45,166][105692] Updated weights for policy 0, policy_version 580850 (0.0009) [2023-12-26 19:35:45,220][105692] Updated weights for policy 0, policy_version 580860 (0.0009) [2023-12-26 19:35:45,631][105620] Updated weights for policy 1, policy_version 581681 (0.0006) [2023-12-26 19:35:45,684][105620] Updated weights for policy 1, policy_version 581691 (0.0005) [2023-12-26 19:35:45,738][105620] Updated weights for policy 1, policy_version 581701 (0.0005) [2023-12-26 19:35:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.9, 300 sec: 19660.8). Total num frames: 297648128. Throughput: 0: 9807.3, 1: 9854.0. Samples: 297618404. Policy #0 lag: (min: 1.0, avg: 15.9, max: 33.0) [2023-12-26 19:35:46,063][104569] Avg episode reward: [(0, '8359.143'), (1, '9259.139')] [2023-12-26 19:35:46,063][105692] Updated weights for policy 0, policy_version 580870 (0.0010) [2023-12-26 19:35:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000581704_148930560.pth... [2023-12-26 19:35:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000580584_148643840.pth [2023-12-26 19:35:46,113][105692] Updated weights for policy 0, policy_version 580881 (0.0009) [2023-12-26 19:35:46,169][105692] Updated weights for policy 0, policy_version 580891 (0.0010) [2023-12-26 19:35:46,198][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000580896_148725760.pth... [2023-12-26 19:35:46,202][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000579712_148422656.pth [2023-12-26 19:35:46,334][105620] Updated weights for policy 1, policy_version 581711 (0.0007) [2023-12-26 19:35:46,388][105620] Updated weights for policy 1, policy_version 581721 (0.0009) [2023-12-26 19:35:46,436][105620] Updated weights for policy 1, policy_version 581731 (0.0009) [2023-12-26 19:35:46,927][105692] Updated weights for policy 0, policy_version 580901 (0.0008) [2023-12-26 19:35:46,974][105692] Updated weights for policy 0, policy_version 580911 (0.0008) [2023-12-26 19:35:47,021][105692] Updated weights for policy 0, policy_version 580921 (0.0006) [2023-12-26 19:35:47,221][105620] Updated weights for policy 1, policy_version 581741 (0.0009) [2023-12-26 19:35:47,283][105620] Updated weights for policy 1, policy_version 581751 (0.0009) [2023-12-26 19:35:47,345][105620] Updated weights for policy 1, policy_version 581761 (0.0009) [2023-12-26 19:35:47,699][105692] Updated weights for policy 0, policy_version 580931 (0.0007) [2023-12-26 19:35:47,759][105692] Updated weights for policy 0, policy_version 580941 (0.0008) [2023-12-26 19:35:47,812][105692] Updated weights for policy 0, policy_version 580951 (0.0009) [2023-12-26 19:35:48,110][105620] Updated weights for policy 1, policy_version 581771 (0.0008) [2023-12-26 19:35:48,168][105620] Updated weights for policy 1, policy_version 581781 (0.0006) [2023-12-26 19:35:48,223][105620] Updated weights for policy 1, policy_version 581791 (0.0005) [2023-12-26 19:35:48,589][105692] Updated weights for policy 0, policy_version 580961 (0.0007) [2023-12-26 19:35:48,648][105692] Updated weights for policy 0, policy_version 580971 (0.0009) [2023-12-26 19:35:48,710][105692] Updated weights for policy 0, policy_version 580981 (0.0009) [2023-12-26 19:35:48,768][105692] Updated weights for policy 0, policy_version 580991 (0.0009) [2023-12-26 19:35:48,926][105620] Updated weights for policy 1, policy_version 581801 (0.0009) [2023-12-26 19:35:48,986][105620] Updated weights for policy 1, policy_version 581811 (0.0007) [2023-12-26 19:35:49,046][105620] Updated weights for policy 1, policy_version 581821 (0.0009) [2023-12-26 19:35:49,107][105620] Updated weights for policy 1, policy_version 581831 (0.0008) [2023-12-26 19:35:49,514][105692] Updated weights for policy 0, policy_version 581001 (0.0010) [2023-12-26 19:35:49,561][105692] Updated weights for policy 0, policy_version 581011 (0.0007) [2023-12-26 19:35:49,609][105692] Updated weights for policy 0, policy_version 581021 (0.0008) [2023-12-26 19:35:49,890][105620] Updated weights for policy 1, policy_version 581841 (0.0008) [2023-12-26 19:35:49,955][105620] Updated weights for policy 1, policy_version 581851 (0.0008) [2023-12-26 19:35:50,012][105620] Updated weights for policy 1, policy_version 581861 (0.0008) [2023-12-26 19:35:50,413][105692] Updated weights for policy 0, policy_version 581031 (0.0011) [2023-12-26 19:35:50,472][105692] Updated weights for policy 0, policy_version 581041 (0.0010) [2023-12-26 19:35:50,528][105692] Updated weights for policy 0, policy_version 581051 (0.0010) [2023-12-26 19:35:50,752][105620] Updated weights for policy 1, policy_version 581871 (0.0009) [2023-12-26 19:35:50,816][105620] Updated weights for policy 1, policy_version 581881 (0.0009) [2023-12-26 19:35:50,882][105620] Updated weights for policy 1, policy_version 581891 (0.0009) [2023-12-26 19:35:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 297746432. Throughput: 0: 9790.8, 1: 9745.5. Samples: 297734052. Policy #0 lag: (min: 1.0, avg: 15.9, max: 33.0) [2023-12-26 19:35:51,062][104569] Avg episode reward: [(0, '8706.782'), (1, '9259.129')] [2023-12-26 19:35:51,339][105692] Updated weights for policy 0, policy_version 581061 (0.0009) [2023-12-26 19:35:51,405][105692] Updated weights for policy 0, policy_version 581071 (0.0007) [2023-12-26 19:35:51,461][105692] Updated weights for policy 0, policy_version 581081 (0.0005) [2023-12-26 19:35:51,556][105620] Updated weights for policy 1, policy_version 581901 (0.0007) [2023-12-26 19:35:51,631][105620] Updated weights for policy 1, policy_version 581911 (0.0006) [2023-12-26 19:35:51,698][105620] Updated weights for policy 1, policy_version 581921 (0.0009) [2023-12-26 19:35:52,195][105692] Updated weights for policy 0, policy_version 581091 (0.0007) [2023-12-26 19:35:52,257][105692] Updated weights for policy 0, policy_version 581101 (0.0009) [2023-12-26 19:35:52,313][105692] Updated weights for policy 0, policy_version 581111 (0.0008) [2023-12-26 19:35:52,348][105620] Updated weights for policy 1, policy_version 581931 (0.0008) [2023-12-26 19:35:52,411][105620] Updated weights for policy 1, policy_version 581941 (0.0009) [2023-12-26 19:35:52,477][105620] Updated weights for policy 1, policy_version 581951 (0.0011) [2023-12-26 19:35:53,079][105692] Updated weights for policy 0, policy_version 581121 (0.0007) [2023-12-26 19:35:53,127][105692] Updated weights for policy 0, policy_version 581131 (0.0008) [2023-12-26 19:35:53,185][105692] Updated weights for policy 0, policy_version 581141 (0.0008) [2023-12-26 19:35:53,234][105620] Updated weights for policy 1, policy_version 581961 (0.0011) [2023-12-26 19:35:53,243][105692] Updated weights for policy 0, policy_version 581151 (0.0008) [2023-12-26 19:35:53,292][105620] Updated weights for policy 1, policy_version 581971 (0.0010) [2023-12-26 19:35:53,349][105620] Updated weights for policy 1, policy_version 581981 (0.0010) [2023-12-26 19:35:53,407][105620] Updated weights for policy 1, policy_version 581991 (0.0010) [2023-12-26 19:35:53,906][105692] Updated weights for policy 0, policy_version 581161 (0.0005) [2023-12-26 19:35:53,965][105692] Updated weights for policy 0, policy_version 581171 (0.0010) [2023-12-26 19:35:54,023][105692] Updated weights for policy 0, policy_version 581181 (0.0010) [2023-12-26 19:35:54,152][105620] Updated weights for policy 1, policy_version 582001 (0.0008) [2023-12-26 19:35:54,210][105620] Updated weights for policy 1, policy_version 582011 (0.0010) [2023-12-26 19:35:54,268][105620] Updated weights for policy 1, policy_version 582021 (0.0010) [2023-12-26 19:35:54,649][105692] Updated weights for policy 0, policy_version 581191 (0.0011) [2023-12-26 19:35:54,717][105692] Updated weights for policy 0, policy_version 581201 (0.0010) [2023-12-26 19:35:54,778][105692] Updated weights for policy 0, policy_version 581211 (0.0010) [2023-12-26 19:35:54,963][105620] Updated weights for policy 1, policy_version 582031 (0.0008) [2023-12-26 19:35:55,029][105620] Updated weights for policy 1, policy_version 582041 (0.0006) [2023-12-26 19:35:55,086][105620] Updated weights for policy 1, policy_version 582051 (0.0005) [2023-12-26 19:35:55,501][105692] Updated weights for policy 0, policy_version 581221 (0.0010) [2023-12-26 19:35:55,550][105692] Updated weights for policy 0, policy_version 581231 (0.0010) [2023-12-26 19:35:55,590][105620] Updated weights for policy 1, policy_version 582061 (0.0006) [2023-12-26 19:35:55,602][105692] Updated weights for policy 0, policy_version 581241 (0.0010) [2023-12-26 19:35:55,643][105620] Updated weights for policy 1, policy_version 582071 (0.0008) [2023-12-26 19:35:55,702][105620] Updated weights for policy 1, policy_version 582081 (0.0010) [2023-12-26 19:35:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 297844736. Throughput: 0: 9813.3, 1: 9787.3. Samples: 297852380. Policy #0 lag: (min: 1.0, avg: 15.9, max: 33.0) [2023-12-26 19:35:56,062][104569] Avg episode reward: [(0, '6842.233'), (1, '9349.687')] [2023-12-26 19:35:56,347][105692] Updated weights for policy 0, policy_version 581251 (0.0010) [2023-12-26 19:35:56,415][105692] Updated weights for policy 0, policy_version 581261 (0.0010) [2023-12-26 19:35:56,434][105620] Updated weights for policy 1, policy_version 582091 (0.0011) [2023-12-26 19:35:56,474][105692] Updated weights for policy 0, policy_version 581271 (0.0011) [2023-12-26 19:35:56,500][105620] Updated weights for policy 1, policy_version 582101 (0.0010) [2023-12-26 19:35:56,559][105620] Updated weights for policy 1, policy_version 582111 (0.0011) [2023-12-26 19:35:57,209][105692] Updated weights for policy 0, policy_version 581281 (0.0011) [2023-12-26 19:35:57,274][105692] Updated weights for policy 0, policy_version 581291 (0.0010) [2023-12-26 19:35:57,299][105620] Updated weights for policy 1, policy_version 582121 (0.0010) [2023-12-26 19:35:57,333][105692] Updated weights for policy 0, policy_version 581301 (0.0011) [2023-12-26 19:35:57,350][105620] Updated weights for policy 1, policy_version 582131 (0.0010) [2023-12-26 19:35:57,381][105692] Updated weights for policy 0, policy_version 581311 (0.0010) [2023-12-26 19:35:57,411][105620] Updated weights for policy 1, policy_version 582141 (0.0010) [2023-12-26 19:35:57,472][105620] Updated weights for policy 1, policy_version 582151 (0.0010) [2023-12-26 19:35:58,121][105692] Updated weights for policy 0, policy_version 581321 (0.0007) [2023-12-26 19:35:58,179][105620] Updated weights for policy 1, policy_version 582161 (0.0007) [2023-12-26 19:35:58,184][105692] Updated weights for policy 0, policy_version 581331 (0.0010) [2023-12-26 19:35:58,239][105692] Updated weights for policy 0, policy_version 581341 (0.0011) [2023-12-26 19:35:58,241][105620] Updated weights for policy 1, policy_version 582171 (0.0006) [2023-12-26 19:35:58,306][105620] Updated weights for policy 1, policy_version 582181 (0.0006) [2023-12-26 19:35:59,049][105692] Updated weights for policy 0, policy_version 581351 (0.0009) [2023-12-26 19:35:59,060][105620] Updated weights for policy 1, policy_version 582191 (0.0008) [2023-12-26 19:35:59,116][105692] Updated weights for policy 0, policy_version 581361 (0.0007) [2023-12-26 19:35:59,124][105620] Updated weights for policy 1, policy_version 582201 (0.0009) [2023-12-26 19:35:59,176][105692] Updated weights for policy 0, policy_version 581371 (0.0006) [2023-12-26 19:35:59,189][105620] Updated weights for policy 1, policy_version 582211 (0.0010) [2023-12-26 19:35:59,941][105692] Updated weights for policy 0, policy_version 581381 (0.0008) [2023-12-26 19:35:59,954][105620] Updated weights for policy 1, policy_version 582221 (0.0009) [2023-12-26 19:35:59,997][105692] Updated weights for policy 0, policy_version 581391 (0.0009) [2023-12-26 19:36:00,004][105620] Updated weights for policy 1, policy_version 582231 (0.0009) [2023-12-26 19:36:00,053][105692] Updated weights for policy 0, policy_version 581401 (0.0009) [2023-12-26 19:36:00,061][105620] Updated weights for policy 1, policy_version 582241 (0.0009) [2023-12-26 19:36:00,701][105692] Updated weights for policy 0, policy_version 581411 (0.0008) [2023-12-26 19:36:00,745][105692] Updated weights for policy 0, policy_version 581421 (0.0005) [2023-12-26 19:36:00,771][105620] Updated weights for policy 1, policy_version 582251 (0.0009) [2023-12-26 19:36:00,794][105692] Updated weights for policy 0, policy_version 581431 (0.0007) [2023-12-26 19:36:00,822][105620] Updated weights for policy 1, policy_version 582261 (0.0005) [2023-12-26 19:36:00,868][105620] Updated weights for policy 1, policy_version 582271 (0.0005) [2023-12-26 19:36:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 297943040. Throughput: 0: 9847.9, 1: 9724.4. Samples: 297908320. Policy #0 lag: (min: 1.0, avg: 15.9, max: 33.0) [2023-12-26 19:36:01,062][104569] Avg episode reward: [(0, '6402.045'), (1, '9167.817')] [2023-12-26 19:36:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000581440_148865024.pth... [2023-12-26 19:36:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000582280_149078016.pth... [2023-12-26 19:36:01,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000580320_148578304.pth [2023-12-26 19:36:01,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000581128_148783104.pth [2023-12-26 19:36:01,472][105620] Updated weights for policy 1, policy_version 582281 (0.0005) [2023-12-26 19:36:01,516][105692] Updated weights for policy 0, policy_version 581441 (0.0008) [2023-12-26 19:36:01,530][105620] Updated weights for policy 1, policy_version 582291 (0.0007) [2023-12-26 19:36:01,579][105692] Updated weights for policy 0, policy_version 581451 (0.0010) [2023-12-26 19:36:01,593][105620] Updated weights for policy 1, policy_version 582301 (0.0006) [2023-12-26 19:36:01,642][105692] Updated weights for policy 0, policy_version 581461 (0.0011) [2023-12-26 19:36:01,657][105620] Updated weights for policy 1, policy_version 582311 (0.0007) [2023-12-26 19:36:01,705][105692] Updated weights for policy 0, policy_version 581471 (0.0011) [2023-12-26 19:36:02,245][105620] Updated weights for policy 1, policy_version 582321 (0.0008) [2023-12-26 19:36:02,305][105620] Updated weights for policy 1, policy_version 582331 (0.0009) [2023-12-26 19:36:02,356][105620] Updated weights for policy 1, policy_version 582341 (0.0009) [2023-12-26 19:36:02,483][105692] Updated weights for policy 0, policy_version 581481 (0.0007) [2023-12-26 19:36:02,546][105692] Updated weights for policy 0, policy_version 581492 (0.0010) [2023-12-26 19:36:02,606][105692] Updated weights for policy 0, policy_version 581502 (0.0009) [2023-12-26 19:36:03,042][105620] Updated weights for policy 1, policy_version 582351 (0.0007) [2023-12-26 19:36:03,101][105620] Updated weights for policy 1, policy_version 582361 (0.0005) [2023-12-26 19:36:03,159][105620] Updated weights for policy 1, policy_version 582371 (0.0005) [2023-12-26 19:36:03,433][105692] Updated weights for policy 0, policy_version 581512 (0.0010) [2023-12-26 19:36:03,485][105692] Updated weights for policy 0, policy_version 581522 (0.0009) [2023-12-26 19:36:03,539][105692] Updated weights for policy 0, policy_version 581533 (0.0010) [2023-12-26 19:36:03,681][105620] Updated weights for policy 1, policy_version 582381 (0.0005) [2023-12-26 19:36:03,740][105620] Updated weights for policy 1, policy_version 582391 (0.0007) [2023-12-26 19:36:03,788][105620] Updated weights for policy 1, policy_version 582401 (0.0009) [2023-12-26 19:36:04,369][105692] Updated weights for policy 0, policy_version 581543 (0.0010) [2023-12-26 19:36:04,434][105692] Updated weights for policy 0, policy_version 581553 (0.0008) [2023-12-26 19:36:04,482][105692] Updated weights for policy 0, policy_version 581563 (0.0009) [2023-12-26 19:36:04,509][105620] Updated weights for policy 1, policy_version 582411 (0.0008) [2023-12-26 19:36:04,567][105620] Updated weights for policy 1, policy_version 582421 (0.0009) [2023-12-26 19:36:04,628][105620] Updated weights for policy 1, policy_version 582431 (0.0009) [2023-12-26 19:36:05,265][105620] Updated weights for policy 1, policy_version 582441 (0.0009) [2023-12-26 19:36:05,313][105620] Updated weights for policy 1, policy_version 582451 (0.0010) [2023-12-26 19:36:05,335][105692] Updated weights for policy 0, policy_version 581573 (0.0006) [2023-12-26 19:36:05,369][105620] Updated weights for policy 1, policy_version 582461 (0.0010) [2023-12-26 19:36:05,389][105692] Updated weights for policy 0, policy_version 581583 (0.0007) [2023-12-26 19:36:05,418][105620] Updated weights for policy 1, policy_version 582471 (0.0006) [2023-12-26 19:36:05,443][105692] Updated weights for policy 0, policy_version 581593 (0.0009) [2023-12-26 19:36:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 298033152. Throughput: 0: 9842.1, 1: 9786.1. Samples: 298026104. Policy #0 lag: (min: 1.0, avg: 15.9, max: 33.0) [2023-12-26 19:36:06,062][104569] Avg episode reward: [(0, '8061.394'), (1, '9260.874')] [2023-12-26 19:36:06,066][105620] Updated weights for policy 1, policy_version 582481 (0.0010) [2023-12-26 19:36:06,129][105620] Updated weights for policy 1, policy_version 582491 (0.0011) [2023-12-26 19:36:06,187][105620] Updated weights for policy 1, policy_version 582501 (0.0011) [2023-12-26 19:36:06,237][105692] Updated weights for policy 0, policy_version 581603 (0.0009) [2023-12-26 19:36:06,301][105692] Updated weights for policy 0, policy_version 581613 (0.0008) [2023-12-26 19:36:06,369][105692] Updated weights for policy 0, policy_version 581623 (0.0008) [2023-12-26 19:36:06,938][105620] Updated weights for policy 1, policy_version 582511 (0.0011) [2023-12-26 19:36:06,990][105620] Updated weights for policy 1, policy_version 582521 (0.0011) [2023-12-26 19:36:07,049][105620] Updated weights for policy 1, policy_version 582531 (0.0011) [2023-12-26 19:36:07,119][105692] Updated weights for policy 0, policy_version 581633 (0.0008) [2023-12-26 19:36:07,171][105692] Updated weights for policy 0, policy_version 581643 (0.0009) [2023-12-26 19:36:07,219][105692] Updated weights for policy 0, policy_version 581653 (0.0007) [2023-12-26 19:36:07,272][105692] Updated weights for policy 0, policy_version 581663 (0.0008) [2023-12-26 19:36:07,742][105620] Updated weights for policy 1, policy_version 582541 (0.0011) [2023-12-26 19:36:07,791][105620] Updated weights for policy 1, policy_version 582551 (0.0010) [2023-12-26 19:36:07,843][105620] Updated weights for policy 1, policy_version 582561 (0.0009) [2023-12-26 19:36:07,976][105692] Updated weights for policy 0, policy_version 581673 (0.0010) [2023-12-26 19:36:08,029][105692] Updated weights for policy 0, policy_version 581683 (0.0010) [2023-12-26 19:36:08,083][105692] Updated weights for policy 0, policy_version 581693 (0.0010) [2023-12-26 19:36:08,532][105620] Updated weights for policy 1, policy_version 582571 (0.0009) [2023-12-26 19:36:08,577][105620] Updated weights for policy 1, policy_version 582581 (0.0010) [2023-12-26 19:36:08,626][105620] Updated weights for policy 1, policy_version 582591 (0.0011) [2023-12-26 19:36:08,770][105692] Updated weights for policy 0, policy_version 581703 (0.0007) [2023-12-26 19:36:08,826][105692] Updated weights for policy 0, policy_version 581713 (0.0008) [2023-12-26 19:36:08,878][105692] Updated weights for policy 0, policy_version 581723 (0.0008) [2023-12-26 19:36:09,414][105620] Updated weights for policy 1, policy_version 582601 (0.0011) [2023-12-26 19:36:09,474][105620] Updated weights for policy 1, policy_version 582611 (0.0011) [2023-12-26 19:36:09,538][105620] Updated weights for policy 1, policy_version 582621 (0.0011) [2023-12-26 19:36:09,602][105620] Updated weights for policy 1, policy_version 582631 (0.0011) [2023-12-26 19:36:09,678][105692] Updated weights for policy 0, policy_version 581733 (0.0008) [2023-12-26 19:36:09,737][105692] Updated weights for policy 0, policy_version 581743 (0.0007) [2023-12-26 19:36:09,740][105585] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-26 19:36:10,312][105620] Updated weights for policy 1, policy_version 582641 (0.0009) [2023-12-26 19:36:10,363][105620] Updated weights for policy 1, policy_version 582651 (0.0010) [2023-12-26 19:36:10,416][105620] Updated weights for policy 1, policy_version 582661 (0.0006) [2023-12-26 19:36:10,648][105692] Updated weights for policy 0, policy_version 581753 (0.0008) [2023-12-26 19:36:10,714][105692] Updated weights for policy 0, policy_version 581763 (0.0007) [2023-12-26 19:36:10,773][105692] Updated weights for policy 0, policy_version 581773 (0.0009) [2023-12-26 19:36:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 298131456. Throughput: 0: 9738.3, 1: 9845.7. Samples: 298141700. Policy #0 lag: (min: 1.0, avg: 15.9, max: 33.0) [2023-12-26 19:36:11,062][104569] Avg episode reward: [(0, '8700.172'), (1, '9259.636')] [2023-12-26 19:36:11,081][105620] Updated weights for policy 1, policy_version 582671 (0.0008) [2023-12-26 19:36:11,141][105620] Updated weights for policy 1, policy_version 582681 (0.0010) [2023-12-26 19:36:11,208][105620] Updated weights for policy 1, policy_version 582691 (0.0011) [2023-12-26 19:36:11,478][105692] Updated weights for policy 0, policy_version 581783 (0.0008) [2023-12-26 19:36:11,527][105692] Updated weights for policy 0, policy_version 581793 (0.0008) [2023-12-26 19:36:11,579][105692] Updated weights for policy 0, policy_version 581803 (0.0008) [2023-12-26 19:36:11,949][105620] Updated weights for policy 1, policy_version 582701 (0.0008) [2023-12-26 19:36:12,004][105620] Updated weights for policy 1, policy_version 582711 (0.0010) [2023-12-26 19:36:12,057][105620] Updated weights for policy 1, policy_version 582721 (0.0008) [2023-12-26 19:36:12,448][105692] Updated weights for policy 0, policy_version 581813 (0.0008) [2023-12-26 19:36:12,520][105692] Updated weights for policy 0, policy_version 581823 (0.0005) [2023-12-26 19:36:12,588][105692] Updated weights for policy 0, policy_version 581833 (0.0005) [2023-12-26 19:36:12,723][105620] Updated weights for policy 1, policy_version 582731 (0.0009) [2023-12-26 19:36:12,786][105620] Updated weights for policy 1, policy_version 582741 (0.0007) [2023-12-26 19:36:12,852][105620] Updated weights for policy 1, policy_version 582751 (0.0005) [2023-12-26 19:36:13,132][105692] Updated weights for policy 0, policy_version 581843 (0.0006) [2023-12-26 19:36:13,173][105585] KL-divergence is very high: 229.0323 [2023-12-26 19:36:13,177][105692] Updated weights for policy 0, policy_version 581853 (0.0005) [2023-12-26 19:36:13,212][105585] KL-divergence is very high: 336.3053 [2023-12-26 19:36:13,228][105692] Updated weights for policy 0, policy_version 581863 (0.0005) [2023-12-26 19:36:13,255][105585] KL-divergence is very high: 291.8857 [2023-12-26 19:36:13,441][105620] Updated weights for policy 1, policy_version 582761 (0.0006) [2023-12-26 19:36:13,497][105620] Updated weights for policy 1, policy_version 582771 (0.0008) [2023-12-26 19:36:13,544][105620] Updated weights for policy 1, policy_version 582781 (0.0008) [2023-12-26 19:36:13,591][105620] Updated weights for policy 1, policy_version 582791 (0.0007) [2023-12-26 19:36:13,848][105692] Updated weights for policy 0, policy_version 581873 (0.0006) [2023-12-26 19:36:13,893][105692] Updated weights for policy 0, policy_version 581883 (0.0010) [2023-12-26 19:36:13,941][105692] Updated weights for policy 0, policy_version 581893 (0.0010) [2023-12-26 19:36:13,988][105692] Updated weights for policy 0, policy_version 581903 (0.0008) [2023-12-26 19:36:14,416][105620] Updated weights for policy 1, policy_version 582801 (0.0009) [2023-12-26 19:36:14,473][105620] Updated weights for policy 1, policy_version 582811 (0.0008) [2023-12-26 19:36:14,536][105620] Updated weights for policy 1, policy_version 582821 (0.0008) [2023-12-26 19:36:14,621][105692] Updated weights for policy 0, policy_version 581913 (0.0010) [2023-12-26 19:36:14,671][105692] Updated weights for policy 0, policy_version 581923 (0.0010) [2023-12-26 19:36:14,722][105692] Updated weights for policy 0, policy_version 581933 (0.0010) [2023-12-26 19:36:15,284][105620] Updated weights for policy 1, policy_version 582831 (0.0008) [2023-12-26 19:36:15,344][105620] Updated weights for policy 1, policy_version 582841 (0.0009) [2023-12-26 19:36:15,403][105620] Updated weights for policy 1, policy_version 582851 (0.0008) [2023-12-26 19:36:15,510][105692] Updated weights for policy 0, policy_version 581943 (0.0010) [2023-12-26 19:36:15,562][105692] Updated weights for policy 0, policy_version 581953 (0.0010) [2023-12-26 19:36:15,610][105692] Updated weights for policy 0, policy_version 581963 (0.0010) [2023-12-26 19:36:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 298229760. Throughput: 0: 9638.0, 1: 9899.2. Samples: 298201904. Policy #0 lag: (min: 1.0, avg: 15.9, max: 33.0) [2023-12-26 19:36:16,063][104569] Avg episode reward: [(0, '8434.749'), (1, '9167.780')] [2023-12-26 19:36:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000581968_149004288.pth... [2023-12-26 19:36:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000582856_149225472.pth... [2023-12-26 19:36:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000580896_148725760.pth [2023-12-26 19:36:16,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000581704_148930560.pth [2023-12-26 19:36:16,120][105620] Updated weights for policy 1, policy_version 582861 (0.0008) [2023-12-26 19:36:16,173][105620] Updated weights for policy 1, policy_version 582871 (0.0009) [2023-12-26 19:36:16,219][105620] Updated weights for policy 1, policy_version 582881 (0.0009) [2023-12-26 19:36:16,355][105692] Updated weights for policy 0, policy_version 581973 (0.0010) [2023-12-26 19:36:16,414][105692] Updated weights for policy 0, policy_version 581983 (0.0010) [2023-12-26 19:36:16,474][105692] Updated weights for policy 0, policy_version 581994 (0.0010) [2023-12-26 19:36:16,828][105620] Updated weights for policy 1, policy_version 582891 (0.0008) [2023-12-26 19:36:16,897][105620] Updated weights for policy 1, policy_version 582901 (0.0006) [2023-12-26 19:36:16,966][105620] Updated weights for policy 1, policy_version 582911 (0.0005) [2023-12-26 19:36:17,151][105692] Updated weights for policy 0, policy_version 582005 (0.0008) [2023-12-26 19:36:17,204][105692] Updated weights for policy 0, policy_version 582015 (0.0005) [2023-12-26 19:36:17,257][105692] Updated weights for policy 0, policy_version 582025 (0.0005) [2023-12-26 19:36:17,643][105620] Updated weights for policy 1, policy_version 582921 (0.0006) [2023-12-26 19:36:17,694][105620] Updated weights for policy 1, policy_version 582931 (0.0010) [2023-12-26 19:36:17,741][105620] Updated weights for policy 1, policy_version 582941 (0.0010) [2023-12-26 19:36:17,772][105692] Updated weights for policy 0, policy_version 582035 (0.0006) [2023-12-26 19:36:17,790][105620] Updated weights for policy 1, policy_version 582951 (0.0005) [2023-12-26 19:36:17,825][105692] Updated weights for policy 0, policy_version 582046 (0.0008) [2023-12-26 19:36:17,880][105692] Updated weights for policy 0, policy_version 582056 (0.0009) [2023-12-26 19:36:18,467][105620] Updated weights for policy 1, policy_version 582961 (0.0007) [2023-12-26 19:36:18,541][105620] Updated weights for policy 1, policy_version 582971 (0.0008) [2023-12-26 19:36:18,544][105692] Updated weights for policy 0, policy_version 582066 (0.0008) [2023-12-26 19:36:18,603][105620] Updated weights for policy 1, policy_version 582981 (0.0008) [2023-12-26 19:36:18,610][105692] Updated weights for policy 0, policy_version 582076 (0.0006) [2023-12-26 19:36:18,678][105692] Updated weights for policy 0, policy_version 582086 (0.0006) [2023-12-26 19:36:18,739][105692] Updated weights for policy 0, policy_version 582096 (0.0006) [2023-12-26 19:36:19,363][105620] Updated weights for policy 1, policy_version 582991 (0.0010) [2023-12-26 19:36:19,427][105620] Updated weights for policy 1, policy_version 583001 (0.0007) [2023-12-26 19:36:19,446][105692] Updated weights for policy 0, policy_version 582106 (0.0008) [2023-12-26 19:36:19,489][105620] Updated weights for policy 1, policy_version 583011 (0.0007) [2023-12-26 19:36:19,502][105692] Updated weights for policy 0, policy_version 582116 (0.0008) [2023-12-26 19:36:19,571][105692] Updated weights for policy 0, policy_version 582126 (0.0008) [2023-12-26 19:36:20,251][105620] Updated weights for policy 1, policy_version 583021 (0.0007) [2023-12-26 19:36:20,310][105620] Updated weights for policy 1, policy_version 583031 (0.0005) [2023-12-26 19:36:20,370][105620] Updated weights for policy 1, policy_version 583041 (0.0006) [2023-12-26 19:36:20,393][105692] Updated weights for policy 0, policy_version 582136 (0.0009) [2023-12-26 19:36:20,443][105692] Updated weights for policy 0, policy_version 582146 (0.0009) [2023-12-26 19:36:20,495][105692] Updated weights for policy 0, policy_version 582156 (0.0006) [2023-12-26 19:36:21,033][105620] Updated weights for policy 1, policy_version 583051 (0.0009) [2023-12-26 19:36:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 298328064. Throughput: 0: 9604.5, 1: 9903.7. Samples: 298321764. Policy #0 lag: (min: 1.0, avg: 15.9, max: 33.0) [2023-12-26 19:36:21,062][104569] Avg episode reward: [(0, '7077.102'), (1, '3254.993')] [2023-12-26 19:36:21,104][105620] Updated weights for policy 1, policy_version 583061 (0.0008) [2023-12-26 19:36:21,175][105620] Updated weights for policy 1, policy_version 583071 (0.0009) [2023-12-26 19:36:21,258][105692] Updated weights for policy 0, policy_version 582166 (0.0010) [2023-12-26 19:36:21,329][105692] Updated weights for policy 0, policy_version 582176 (0.0010) [2023-12-26 19:36:21,395][105692] Updated weights for policy 0, policy_version 582186 (0.0011) [2023-12-26 19:36:21,858][105620] Updated weights for policy 1, policy_version 583081 (0.0006) [2023-12-26 19:36:21,922][105620] Updated weights for policy 1, policy_version 583091 (0.0007) [2023-12-26 19:36:21,982][105620] Updated weights for policy 1, policy_version 583101 (0.0008) [2023-12-26 19:36:22,046][105620] Updated weights for policy 1, policy_version 583111 (0.0008) [2023-12-26 19:36:22,159][105692] Updated weights for policy 0, policy_version 582196 (0.0011) [2023-12-26 19:36:22,219][105692] Updated weights for policy 0, policy_version 582206 (0.0011) [2023-12-26 19:36:22,281][105692] Updated weights for policy 0, policy_version 582216 (0.0011) [2023-12-26 19:36:22,723][105620] Updated weights for policy 1, policy_version 583121 (0.0006) [2023-12-26 19:36:22,787][105620] Updated weights for policy 1, policy_version 583131 (0.0006) [2023-12-26 19:36:22,851][105620] Updated weights for policy 1, policy_version 583141 (0.0006) [2023-12-26 19:36:23,053][105692] Updated weights for policy 0, policy_version 582226 (0.0011) [2023-12-26 19:36:23,119][105692] Updated weights for policy 0, policy_version 582236 (0.0011) [2023-12-26 19:36:23,179][105692] Updated weights for policy 0, policy_version 582246 (0.0011) [2023-12-26 19:36:23,246][105692] Updated weights for policy 0, policy_version 582256 (0.0011) [2023-12-26 19:36:23,406][105620] Updated weights for policy 1, policy_version 583151 (0.0006) [2023-12-26 19:36:23,474][105620] Updated weights for policy 1, policy_version 583161 (0.0007) [2023-12-26 19:36:23,529][105620] Updated weights for policy 1, policy_version 583171 (0.0007) [2023-12-26 19:36:24,016][105692] Updated weights for policy 0, policy_version 582266 (0.0010) [2023-12-26 19:36:24,069][105692] Updated weights for policy 0, policy_version 582277 (0.0010) [2023-12-26 19:36:24,102][105620] Updated weights for policy 1, policy_version 583181 (0.0006) [2023-12-26 19:36:24,112][105585] KL-divergence is very high: 109.3012 [2023-12-26 19:36:24,127][105692] Updated weights for policy 0, policy_version 582287 (0.0006) [2023-12-26 19:36:24,154][105620] Updated weights for policy 1, policy_version 583191 (0.0009) [2023-12-26 19:36:24,221][105620] Updated weights for policy 1, policy_version 583201 (0.0010) [2023-12-26 19:36:24,828][105692] Updated weights for policy 0, policy_version 582297 (0.0005) [2023-12-26 19:36:24,877][105692] Updated weights for policy 0, policy_version 582307 (0.0005) [2023-12-26 19:36:24,933][105585] KL-divergence is very high: 106.7931 [2023-12-26 19:36:24,944][105692] Updated weights for policy 0, policy_version 582317 (0.0005) [2023-12-26 19:36:25,022][105620] Updated weights for policy 1, policy_version 583212 (0.0010) [2023-12-26 19:36:25,073][105620] Updated weights for policy 1, policy_version 583222 (0.0008) [2023-12-26 19:36:25,128][105620] Updated weights for policy 1, policy_version 583232 (0.0008) [2023-12-26 19:36:25,600][105692] Updated weights for policy 0, policy_version 582327 (0.0009) [2023-12-26 19:36:25,643][105692] Updated weights for policy 0, policy_version 582337 (0.0007) [2023-12-26 19:36:25,691][105692] Updated weights for policy 0, policy_version 582347 (0.0005) [2023-12-26 19:36:25,938][105620] Updated weights for policy 1, policy_version 583242 (0.0008) [2023-12-26 19:36:25,991][105620] Updated weights for policy 1, policy_version 583252 (0.0010) [2023-12-26 19:36:26,048][105620] Updated weights for policy 1, policy_version 583262 (0.0010) [2023-12-26 19:36:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 298426368. Throughput: 0: 9546.2, 1: 9959.0. Samples: 298438152. Policy #0 lag: (min: 1.0, avg: 15.9, max: 33.0) [2023-12-26 19:36:26,062][104569] Avg episode reward: [(0, '7516.179'), (1, '1915.301')] [2023-12-26 19:36:26,117][105620] Updated weights for policy 1, policy_version 583272 (0.0008) [2023-12-26 19:36:26,241][105692] Updated weights for policy 0, policy_version 582357 (0.0005) [2023-12-26 19:36:26,299][105692] Updated weights for policy 0, policy_version 582367 (0.0006) [2023-12-26 19:36:26,358][105692] Updated weights for policy 0, policy_version 582377 (0.0006) [2023-12-26 19:36:26,835][105620] Updated weights for policy 1, policy_version 583282 (0.0005) [2023-12-26 19:36:26,896][105620] Updated weights for policy 1, policy_version 583292 (0.0005) [2023-12-26 19:36:26,960][105620] Updated weights for policy 1, policy_version 583302 (0.0006) [2023-12-26 19:36:26,961][105692] Updated weights for policy 0, policy_version 582387 (0.0009) [2023-12-26 19:36:27,025][105692] Updated weights for policy 0, policy_version 582397 (0.0005) [2023-12-26 19:36:27,084][105692] Updated weights for policy 0, policy_version 582407 (0.0006) [2023-12-26 19:36:27,507][105620] Updated weights for policy 1, policy_version 583312 (0.0009) [2023-12-26 19:36:27,562][105620] Updated weights for policy 1, policy_version 583322 (0.0005) [2023-12-26 19:36:27,627][105692] Updated weights for policy 0, policy_version 582417 (0.0006) [2023-12-26 19:36:27,634][105620] Updated weights for policy 1, policy_version 583332 (0.0005) [2023-12-26 19:36:27,679][105692] Updated weights for policy 0, policy_version 582427 (0.0010) [2023-12-26 19:36:27,727][105692] Updated weights for policy 0, policy_version 582437 (0.0010) [2023-12-26 19:36:27,775][105692] Updated weights for policy 0, policy_version 582447 (0.0010) [2023-12-26 19:36:28,199][105620] Updated weights for policy 1, policy_version 583342 (0.0008) [2023-12-26 19:36:28,265][105620] Updated weights for policy 1, policy_version 583352 (0.0010) [2023-12-26 19:36:28,322][105620] Updated weights for policy 1, policy_version 583362 (0.0010) [2023-12-26 19:36:28,548][105692] Updated weights for policy 0, policy_version 582457 (0.0010) [2023-12-26 19:36:28,597][105692] Updated weights for policy 0, policy_version 582467 (0.0010) [2023-12-26 19:36:28,658][105692] Updated weights for policy 0, policy_version 582477 (0.0011) [2023-12-26 19:36:29,022][105620] Updated weights for policy 1, policy_version 583372 (0.0009) [2023-12-26 19:36:29,081][105620] Updated weights for policy 1, policy_version 583382 (0.0010) [2023-12-26 19:36:29,131][105620] Updated weights for policy 1, policy_version 583392 (0.0010) [2023-12-26 19:36:29,437][105692] Updated weights for policy 0, policy_version 582487 (0.0010) [2023-12-26 19:36:29,489][105692] Updated weights for policy 0, policy_version 582497 (0.0010) [2023-12-26 19:36:29,550][105692] Updated weights for policy 0, policy_version 582507 (0.0010) [2023-12-26 19:36:29,853][105620] Updated weights for policy 1, policy_version 583402 (0.0010) [2023-12-26 19:36:29,916][105620] Updated weights for policy 1, policy_version 583412 (0.0010) [2023-12-26 19:36:29,984][105620] Updated weights for policy 1, policy_version 583422 (0.0010) [2023-12-26 19:36:30,044][105620] Updated weights for policy 1, policy_version 583432 (0.0011) [2023-12-26 19:36:30,289][105692] Updated weights for policy 0, policy_version 582517 (0.0011) [2023-12-26 19:36:30,344][105692] Updated weights for policy 0, policy_version 582527 (0.0009) [2023-12-26 19:36:30,399][105692] Updated weights for policy 0, policy_version 582537 (0.0010) [2023-12-26 19:36:30,698][105620] Updated weights for policy 1, policy_version 583442 (0.0010) [2023-12-26 19:36:30,743][105620] Updated weights for policy 1, policy_version 583452 (0.0010) [2023-12-26 19:36:30,787][105620] Updated weights for policy 1, policy_version 583462 (0.0010) [2023-12-26 19:36:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 298532864. Throughput: 0: 9651.5, 1: 10015.0. Samples: 298503392. Policy #0 lag: (min: 1.0, avg: 15.9, max: 33.0) [2023-12-26 19:36:31,062][104569] Avg episode reward: [(0, '7894.786'), (1, '6273.580')] [2023-12-26 19:36:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000583464_149381120.pth... [2023-12-26 19:36:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000582544_149151744.pth... [2023-12-26 19:36:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000582280_149078016.pth [2023-12-26 19:36:31,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000581440_148865024.pth [2023-12-26 19:36:31,112][105692] Updated weights for policy 0, policy_version 582547 (0.0009) [2023-12-26 19:36:31,180][105692] Updated weights for policy 0, policy_version 582557 (0.0009) [2023-12-26 19:36:31,237][105692] Updated weights for policy 0, policy_version 582567 (0.0007) [2023-12-26 19:36:31,547][105620] Updated weights for policy 1, policy_version 583472 (0.0011) [2023-12-26 19:36:31,611][105620] Updated weights for policy 1, policy_version 583482 (0.0011) [2023-12-26 19:36:31,674][105620] Updated weights for policy 1, policy_version 583492 (0.0011) [2023-12-26 19:36:31,919][105692] Updated weights for policy 0, policy_version 582577 (0.0008) [2023-12-26 19:36:31,981][105692] Updated weights for policy 0, policy_version 582587 (0.0008) [2023-12-26 19:36:32,033][105692] Updated weights for policy 0, policy_version 582597 (0.0008) [2023-12-26 19:36:32,090][105692] Updated weights for policy 0, policy_version 582607 (0.0007) [2023-12-26 19:36:32,453][105620] Updated weights for policy 1, policy_version 583502 (0.0008) [2023-12-26 19:36:32,510][105620] Updated weights for policy 1, policy_version 583512 (0.0010) [2023-12-26 19:36:32,568][105620] Updated weights for policy 1, policy_version 583523 (0.0010) [2023-12-26 19:36:32,699][105692] Updated weights for policy 0, policy_version 582617 (0.0005) [2023-12-26 19:36:32,743][105692] Updated weights for policy 0, policy_version 582627 (0.0005) [2023-12-26 19:36:32,788][105692] Updated weights for policy 0, policy_version 582637 (0.0006) [2023-12-26 19:36:33,267][105620] Updated weights for policy 1, policy_version 583534 (0.0010) [2023-12-26 19:36:33,321][105620] Updated weights for policy 1, policy_version 583544 (0.0009) [2023-12-26 19:36:33,369][105692] Updated weights for policy 0, policy_version 582647 (0.0009) [2023-12-26 19:36:33,375][105620] Updated weights for policy 1, policy_version 583554 (0.0008) [2023-12-26 19:36:33,422][105692] Updated weights for policy 0, policy_version 582657 (0.0007) [2023-12-26 19:36:33,472][105692] Updated weights for policy 0, policy_version 582667 (0.0009) [2023-12-26 19:36:34,009][105620] Updated weights for policy 1, policy_version 583564 (0.0007) [2023-12-26 19:36:34,059][105620] Updated weights for policy 1, policy_version 583574 (0.0009) [2023-12-26 19:36:34,109][105620] Updated weights for policy 1, policy_version 583584 (0.0009) [2023-12-26 19:36:34,277][105692] Updated weights for policy 0, policy_version 582677 (0.0007) [2023-12-26 19:36:34,334][105692] Updated weights for policy 0, policy_version 582687 (0.0009) [2023-12-26 19:36:34,388][105692] Updated weights for policy 0, policy_version 582697 (0.0010) [2023-12-26 19:36:34,821][105620] Updated weights for policy 1, policy_version 583594 (0.0008) [2023-12-26 19:36:34,872][105620] Updated weights for policy 1, policy_version 583604 (0.0005) [2023-12-26 19:36:34,929][105620] Updated weights for policy 1, policy_version 583614 (0.0005) [2023-12-26 19:36:34,989][105620] Updated weights for policy 1, policy_version 583624 (0.0008) [2023-12-26 19:36:35,157][105692] Updated weights for policy 0, policy_version 582707 (0.0009) [2023-12-26 19:36:35,205][105692] Updated weights for policy 0, policy_version 582717 (0.0009) [2023-12-26 19:36:35,263][105692] Updated weights for policy 0, policy_version 582727 (0.0009) [2023-12-26 19:36:35,653][105620] Updated weights for policy 1, policy_version 583634 (0.0010) [2023-12-26 19:36:35,711][105620] Updated weights for policy 1, policy_version 583644 (0.0010) [2023-12-26 19:36:35,778][105620] Updated weights for policy 1, policy_version 583654 (0.0008) [2023-12-26 19:36:35,961][105692] Updated weights for policy 0, policy_version 582737 (0.0006) [2023-12-26 19:36:36,023][105692] Updated weights for policy 0, policy_version 582747 (0.0009) [2023-12-26 19:36:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 298631168. Throughput: 0: 9700.9, 1: 10040.4. Samples: 298622412. Policy #0 lag: (min: 1.0, avg: 15.9, max: 33.0) [2023-12-26 19:36:36,063][104569] Avg episode reward: [(0, '8199.736'), (1, '9120.198')] [2023-12-26 19:36:36,083][105692] Updated weights for policy 0, policy_version 582757 (0.0009) [2023-12-26 19:36:36,141][105692] Updated weights for policy 0, policy_version 582767 (0.0010) [2023-12-26 19:36:36,493][105620] Updated weights for policy 1, policy_version 583664 (0.0007) [2023-12-26 19:36:36,559][105620] Updated weights for policy 1, policy_version 583674 (0.0006) [2023-12-26 19:36:36,624][105620] Updated weights for policy 1, policy_version 583684 (0.0005) [2023-12-26 19:36:36,968][105692] Updated weights for policy 0, policy_version 582777 (0.0010) [2023-12-26 19:36:37,027][105692] Updated weights for policy 0, policy_version 582787 (0.0010) [2023-12-26 19:36:37,092][105692] Updated weights for policy 0, policy_version 582797 (0.0010) [2023-12-26 19:36:37,271][105620] Updated weights for policy 1, policy_version 583694 (0.0008) [2023-12-26 19:36:37,323][105620] Updated weights for policy 1, policy_version 583704 (0.0008) [2023-12-26 19:36:37,378][105620] Updated weights for policy 1, policy_version 583714 (0.0008) [2023-12-26 19:36:37,827][105692] Updated weights for policy 0, policy_version 582807 (0.0007) [2023-12-26 19:36:37,880][105692] Updated weights for policy 0, policy_version 582817 (0.0005) [2023-12-26 19:36:37,926][105692] Updated weights for policy 0, policy_version 582827 (0.0005) [2023-12-26 19:36:38,123][105620] Updated weights for policy 1, policy_version 583724 (0.0009) [2023-12-26 19:36:38,172][105620] Updated weights for policy 1, policy_version 583734 (0.0010) [2023-12-26 19:36:38,231][105620] Updated weights for policy 1, policy_version 583744 (0.0010) [2023-12-26 19:36:38,502][105692] Updated weights for policy 0, policy_version 582837 (0.0008) [2023-12-26 19:36:38,551][105692] Updated weights for policy 0, policy_version 582847 (0.0011) [2023-12-26 19:36:38,614][105692] Updated weights for policy 0, policy_version 582857 (0.0011) [2023-12-26 19:36:38,854][105620] Updated weights for policy 1, policy_version 583754 (0.0010) [2023-12-26 19:36:38,916][105620] Updated weights for policy 1, policy_version 583764 (0.0005) [2023-12-26 19:36:38,978][105620] Updated weights for policy 1, policy_version 583774 (0.0006) [2023-12-26 19:36:39,040][105620] Updated weights for policy 1, policy_version 583784 (0.0010) [2023-12-26 19:36:39,309][105692] Updated weights for policy 0, policy_version 582867 (0.0011) [2023-12-26 19:36:39,375][105692] Updated weights for policy 0, policy_version 582877 (0.0011) [2023-12-26 19:36:39,443][105692] Updated weights for policy 0, policy_version 582887 (0.0010) [2023-12-26 19:36:39,693][105620] Updated weights for policy 1, policy_version 583794 (0.0008) [2023-12-26 19:36:39,757][105620] Updated weights for policy 1, policy_version 583804 (0.0008) [2023-12-26 19:36:39,818][105620] Updated weights for policy 1, policy_version 583814 (0.0008) [2023-12-26 19:36:40,174][105692] Updated weights for policy 0, policy_version 582897 (0.0011) [2023-12-26 19:36:40,237][105692] Updated weights for policy 0, policy_version 582907 (0.0011) [2023-12-26 19:36:40,292][105692] Updated weights for policy 0, policy_version 582917 (0.0011) [2023-12-26 19:36:40,351][105692] Updated weights for policy 0, policy_version 582927 (0.0011) [2023-12-26 19:36:40,547][105620] Updated weights for policy 1, policy_version 583824 (0.0006) [2023-12-26 19:36:40,611][105620] Updated weights for policy 1, policy_version 583834 (0.0009) [2023-12-26 19:36:40,677][105620] Updated weights for policy 1, policy_version 583844 (0.0010) [2023-12-26 19:36:41,028][105692] Updated weights for policy 0, policy_version 582937 (0.0007) [2023-12-26 19:36:41,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 298729472. Throughput: 0: 9699.5, 1: 10029.6. Samples: 298740192. Policy #0 lag: (min: 1.0, avg: 15.9, max: 33.0) [2023-12-26 19:36:41,063][104569] Avg episode reward: [(0, '8650.707'), (1, '9260.662')] [2023-12-26 19:36:41,098][105692] Updated weights for policy 0, policy_version 582948 (0.0009) [2023-12-26 19:36:41,170][105692] Updated weights for policy 0, policy_version 582958 (0.0009) [2023-12-26 19:36:41,493][105620] Updated weights for policy 1, policy_version 583854 (0.0010) [2023-12-26 19:36:41,553][105620] Updated weights for policy 1, policy_version 583864 (0.0008) [2023-12-26 19:36:41,615][105620] Updated weights for policy 1, policy_version 583874 (0.0009) [2023-12-26 19:36:41,932][105692] Updated weights for policy 0, policy_version 582968 (0.0009) [2023-12-26 19:36:41,988][105692] Updated weights for policy 0, policy_version 582978 (0.0009) [2023-12-26 19:36:42,051][105692] Updated weights for policy 0, policy_version 582988 (0.0009) [2023-12-26 19:36:42,409][105620] Updated weights for policy 1, policy_version 583884 (0.0008) [2023-12-26 19:36:42,471][105620] Updated weights for policy 1, policy_version 583894 (0.0009) [2023-12-26 19:36:42,518][105620] Updated weights for policy 1, policy_version 583904 (0.0009) [2023-12-26 19:36:42,756][105692] Updated weights for policy 0, policy_version 582998 (0.0009) [2023-12-26 19:36:42,813][105692] Updated weights for policy 0, policy_version 583008 (0.0009) [2023-12-26 19:36:42,867][105692] Updated weights for policy 0, policy_version 583018 (0.0010) [2023-12-26 19:36:43,170][105620] Updated weights for policy 1, policy_version 583914 (0.0008) [2023-12-26 19:36:43,231][105620] Updated weights for policy 1, policy_version 583924 (0.0006) [2023-12-26 19:36:43,293][105620] Updated weights for policy 1, policy_version 583934 (0.0005) [2023-12-26 19:36:43,352][105620] Updated weights for policy 1, policy_version 583944 (0.0006) [2023-12-26 19:36:43,594][105692] Updated weights for policy 0, policy_version 583028 (0.0008) [2023-12-26 19:36:43,628][105585] KL-divergence is very high: 113.1359 [2023-12-26 19:36:43,642][105692] Updated weights for policy 0, policy_version 583038 (0.0005) [2023-12-26 19:36:43,677][105585] KL-divergence is very high: 194.2768 [2023-12-26 19:36:43,702][105692] Updated weights for policy 0, policy_version 583048 (0.0005) [2023-12-26 19:36:43,717][105585] KL-divergence is very high: 216.1443 [2023-12-26 19:36:43,900][105620] Updated weights for policy 1, policy_version 583954 (0.0006) [2023-12-26 19:36:43,944][105620] Updated weights for policy 1, policy_version 583964 (0.0008) [2023-12-26 19:36:43,988][105620] Updated weights for policy 1, policy_version 583974 (0.0010) [2023-12-26 19:36:44,397][105692] Updated weights for policy 0, policy_version 583058 (0.0007) [2023-12-26 19:36:44,449][105692] Updated weights for policy 0, policy_version 583068 (0.0008) [2023-12-26 19:36:44,506][105692] Updated weights for policy 0, policy_version 583078 (0.0010) [2023-12-26 19:36:44,564][105692] Updated weights for policy 0, policy_version 583088 (0.0010) [2023-12-26 19:36:44,692][105620] Updated weights for policy 1, policy_version 583984 (0.0009) [2023-12-26 19:36:44,757][105620] Updated weights for policy 1, policy_version 583994 (0.0008) [2023-12-26 19:36:44,817][105620] Updated weights for policy 1, policy_version 584004 (0.0010) [2023-12-26 19:36:45,281][105692] Updated weights for policy 0, policy_version 583098 (0.0006) [2023-12-26 19:36:45,340][105692] Updated weights for policy 0, policy_version 583108 (0.0006) [2023-12-26 19:36:45,408][105692] Updated weights for policy 0, policy_version 583118 (0.0007) [2023-12-26 19:36:45,474][105620] Updated weights for policy 1, policy_version 584014 (0.0010) [2023-12-26 19:36:45,533][105620] Updated weights for policy 1, policy_version 584024 (0.0011) [2023-12-26 19:36:45,600][105620] Updated weights for policy 1, policy_version 584034 (0.0010) [2023-12-26 19:36:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 298827776. Throughput: 0: 9726.6, 1: 10074.2. Samples: 298799356. Policy #0 lag: (min: 1.0, avg: 15.9, max: 33.0) [2023-12-26 19:36:46,062][104569] Avg episode reward: [(0, '8825.416'), (1, '9079.955')] [2023-12-26 19:36:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000584040_149528576.pth... [2023-12-26 19:36:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000582856_149225472.pth [2023-12-26 19:36:46,116][105692] Updated weights for policy 0, policy_version 583128 (0.0010) [2023-12-26 19:36:46,174][105692] Updated weights for policy 0, policy_version 583138 (0.0010) [2023-12-26 19:36:46,210][105620] Updated weights for policy 1, policy_version 584044 (0.0010) [2023-12-26 19:36:46,225][105692] Updated weights for policy 0, policy_version 583148 (0.0010) [2023-12-26 19:36:46,241][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000583152_149307392.pth... [2023-12-26 19:36:46,244][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000581968_149004288.pth [2023-12-26 19:36:46,265][105620] Updated weights for policy 1, policy_version 584054 (0.0010) [2023-12-26 19:36:46,323][105620] Updated weights for policy 1, policy_version 584064 (0.0010) [2023-12-26 19:36:46,837][105692] Updated weights for policy 0, policy_version 583158 (0.0009) [2023-12-26 19:36:46,888][105692] Updated weights for policy 0, policy_version 583168 (0.0010) [2023-12-26 19:36:46,935][105620] Updated weights for policy 1, policy_version 584074 (0.0009) [2023-12-26 19:36:46,946][105692] Updated weights for policy 0, policy_version 583178 (0.0010) [2023-12-26 19:36:47,001][105620] Updated weights for policy 1, policy_version 584084 (0.0005) [2023-12-26 19:36:47,063][105620] Updated weights for policy 1, policy_version 584094 (0.0010) [2023-12-26 19:36:47,121][105620] Updated weights for policy 1, policy_version 584104 (0.0010) [2023-12-26 19:36:47,635][105692] Updated weights for policy 0, policy_version 583188 (0.0010) [2023-12-26 19:36:47,692][105692] Updated weights for policy 0, policy_version 583198 (0.0010) [2023-12-26 19:36:47,746][105692] Updated weights for policy 0, policy_version 583208 (0.0009) [2023-12-26 19:36:47,755][105620] Updated weights for policy 1, policy_version 584114 (0.0010) [2023-12-26 19:36:47,807][105620] Updated weights for policy 1, policy_version 584124 (0.0010) [2023-12-26 19:36:47,863][105620] Updated weights for policy 1, policy_version 584134 (0.0010) [2023-12-26 19:36:48,382][105692] Updated weights for policy 0, policy_version 583218 (0.0010) [2023-12-26 19:36:48,436][105692] Updated weights for policy 0, policy_version 583228 (0.0010) [2023-12-26 19:36:48,496][105692] Updated weights for policy 0, policy_version 583238 (0.0007) [2023-12-26 19:36:48,547][105692] Updated weights for policy 0, policy_version 583248 (0.0006) [2023-12-26 19:36:48,583][105620] Updated weights for policy 1, policy_version 584144 (0.0007) [2023-12-26 19:36:48,643][105620] Updated weights for policy 1, policy_version 584154 (0.0009) [2023-12-26 19:36:48,690][105620] Updated weights for policy 1, policy_version 584164 (0.0008) [2023-12-26 19:36:49,208][105692] Updated weights for policy 0, policy_version 583258 (0.0005) [2023-12-26 19:36:49,283][105692] Updated weights for policy 0, policy_version 583268 (0.0007) [2023-12-26 19:36:49,358][105692] Updated weights for policy 0, policy_version 583278 (0.0010) [2023-12-26 19:36:49,489][105620] Updated weights for policy 1, policy_version 584174 (0.0009) [2023-12-26 19:36:49,545][105620] Updated weights for policy 1, policy_version 584184 (0.0009) [2023-12-26 19:36:49,602][105620] Updated weights for policy 1, policy_version 584194 (0.0009) [2023-12-26 19:36:50,085][105692] Updated weights for policy 0, policy_version 583288 (0.0009) [2023-12-26 19:36:50,118][105585] KL-divergence is very high: 137.4830 [2023-12-26 19:36:50,150][105692] Updated weights for policy 0, policy_version 583298 (0.0009) [2023-12-26 19:36:50,168][105585] KL-divergence is very high: 230.2038 [2023-12-26 19:36:50,212][105692] Updated weights for policy 0, policy_version 583308 (0.0009) [2023-12-26 19:36:50,218][105585] KL-divergence is very high: 187.6764 [2023-12-26 19:36:50,341][105620] Updated weights for policy 1, policy_version 584204 (0.0009) [2023-12-26 19:36:50,399][105620] Updated weights for policy 1, policy_version 584214 (0.0008) [2023-12-26 19:36:50,456][105620] Updated weights for policy 1, policy_version 584224 (0.0007) [2023-12-26 19:36:50,934][105692] Updated weights for policy 0, policy_version 583318 (0.0008) [2023-12-26 19:36:50,990][105692] Updated weights for policy 0, policy_version 583328 (0.0011) [2023-12-26 19:36:51,052][105620] Updated weights for policy 1, policy_version 584234 (0.0006) [2023-12-26 19:36:51,054][105692] Updated weights for policy 0, policy_version 583338 (0.0011) [2023-12-26 19:36:51,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 298926080. Throughput: 0: 9868.8, 1: 10024.0. Samples: 298921280. Policy #0 lag: (min: 1.0, avg: 15.9, max: 33.0) [2023-12-26 19:36:51,062][104569] Avg episode reward: [(0, '9001.659'), (1, '8988.225')] [2023-12-26 19:36:51,118][105620] Updated weights for policy 1, policy_version 584244 (0.0008) [2023-12-26 19:36:51,181][105620] Updated weights for policy 1, policy_version 584254 (0.0007) [2023-12-26 19:36:51,250][105620] Updated weights for policy 1, policy_version 584264 (0.0009) [2023-12-26 19:36:51,848][105692] Updated weights for policy 0, policy_version 583348 (0.0009) [2023-12-26 19:36:51,909][105692] Updated weights for policy 0, policy_version 583358 (0.0009) [2023-12-26 19:36:51,967][105620] Updated weights for policy 1, policy_version 584274 (0.0006) [2023-12-26 19:36:51,976][105692] Updated weights for policy 0, policy_version 583368 (0.0009) [2023-12-26 19:36:52,020][105620] Updated weights for policy 1, policy_version 584284 (0.0008) [2023-12-26 19:36:52,081][105620] Updated weights for policy 1, policy_version 584294 (0.0006) [2023-12-26 19:36:52,712][105692] Updated weights for policy 0, policy_version 583378 (0.0007) [2023-12-26 19:36:52,760][105692] Updated weights for policy 0, policy_version 583388 (0.0008) [2023-12-26 19:36:52,814][105692] Updated weights for policy 0, policy_version 583398 (0.0009) [2023-12-26 19:36:52,845][105620] Updated weights for policy 1, policy_version 584304 (0.0007) [2023-12-26 19:36:52,864][105692] Updated weights for policy 0, policy_version 583408 (0.0006) [2023-12-26 19:36:52,901][105620] Updated weights for policy 1, policy_version 584314 (0.0007) [2023-12-26 19:36:52,956][105620] Updated weights for policy 1, policy_version 584324 (0.0008) [2023-12-26 19:36:53,650][105692] Updated weights for policy 0, policy_version 583418 (0.0008) [2023-12-26 19:36:53,666][105620] Updated weights for policy 1, policy_version 584334 (0.0010) [2023-12-26 19:36:53,700][105692] Updated weights for policy 0, policy_version 583428 (0.0007) [2023-12-26 19:36:53,714][105620] Updated weights for policy 1, policy_version 584344 (0.0010) [2023-12-26 19:36:53,753][105692] Updated weights for policy 0, policy_version 583438 (0.0006) [2023-12-26 19:36:53,763][105620] Updated weights for policy 1, policy_version 584354 (0.0006) [2023-12-26 19:36:54,384][105620] Updated weights for policy 1, policy_version 584364 (0.0007) [2023-12-26 19:36:54,443][105620] Updated weights for policy 1, policy_version 584374 (0.0009) [2023-12-26 19:36:54,505][105620] Updated weights for policy 1, policy_version 584384 (0.0009) [2023-12-26 19:36:54,586][105692] Updated weights for policy 0, policy_version 583448 (0.0008) [2023-12-26 19:36:54,633][105692] Updated weights for policy 0, policy_version 583458 (0.0009) [2023-12-26 19:36:54,681][105692] Updated weights for policy 0, policy_version 583468 (0.0009) [2023-12-26 19:36:55,189][105620] Updated weights for policy 1, policy_version 584394 (0.0009) [2023-12-26 19:36:55,235][105620] Updated weights for policy 1, policy_version 584404 (0.0008) [2023-12-26 19:36:55,281][105620] Updated weights for policy 1, policy_version 584414 (0.0008) [2023-12-26 19:36:55,327][105620] Updated weights for policy 1, policy_version 584424 (0.0009) [2023-12-26 19:36:55,480][105692] Updated weights for policy 0, policy_version 583478 (0.0009) [2023-12-26 19:36:55,529][105692] Updated weights for policy 0, policy_version 583488 (0.0009) [2023-12-26 19:36:55,578][105692] Updated weights for policy 0, policy_version 583498 (0.0009) [2023-12-26 19:36:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 299024384. Throughput: 0: 9845.5, 1: 10021.3. Samples: 299035708. Policy #0 lag: (min: 1.0, avg: 15.9, max: 33.0) [2023-12-26 19:36:56,062][104569] Avg episode reward: [(0, '9091.363'), (1, '9168.549')] [2023-12-26 19:36:56,101][105620] Updated weights for policy 1, policy_version 584434 (0.0006) [2023-12-26 19:36:56,144][105620] Updated weights for policy 1, policy_version 584444 (0.0005) [2023-12-26 19:36:56,204][105620] Updated weights for policy 1, policy_version 584454 (0.0007) [2023-12-26 19:36:56,398][105692] Updated weights for policy 0, policy_version 583508 (0.0009) [2023-12-26 19:36:56,438][105585] KL-divergence is very high: 599.7740 [2023-12-26 19:36:56,455][105692] Updated weights for policy 0, policy_version 583518 (0.0009) [2023-12-26 19:36:56,481][105585] KL-divergence is very high: 101.5450 [2023-12-26 19:36:56,488][105585] KL-divergence is very high: 863.7781 [2023-12-26 19:36:56,520][105692] Updated weights for policy 0, policy_version 583528 (0.0009) [2023-12-26 19:36:56,536][105585] KL-divergence is very high: 766.5785 [2023-12-26 19:36:56,866][105620] Updated weights for policy 1, policy_version 584464 (0.0007) [2023-12-26 19:36:56,917][105620] Updated weights for policy 1, policy_version 584474 (0.0005) [2023-12-26 19:36:56,965][105620] Updated weights for policy 1, policy_version 584484 (0.0005) [2023-12-26 19:36:57,359][105692] Updated weights for policy 0, policy_version 583538 (0.0009) [2023-12-26 19:36:57,421][105692] Updated weights for policy 0, policy_version 583548 (0.0010) [2023-12-26 19:36:57,477][105692] Updated weights for policy 0, policy_version 583558 (0.0010) [2023-12-26 19:36:57,521][105692] Updated weights for policy 0, policy_version 583568 (0.0010) [2023-12-26 19:36:57,525][105620] Updated weights for policy 1, policy_version 584494 (0.0008) [2023-12-26 19:36:57,590][105620] Updated weights for policy 1, policy_version 584504 (0.0010) [2023-12-26 19:36:57,638][105620] Updated weights for policy 1, policy_version 584514 (0.0010) [2023-12-26 19:36:58,085][105692] Updated weights for policy 0, policy_version 583578 (0.0008) [2023-12-26 19:36:58,153][105692] Updated weights for policy 0, policy_version 583588 (0.0007) [2023-12-26 19:36:58,212][105692] Updated weights for policy 0, policy_version 583598 (0.0010) [2023-12-26 19:36:58,387][105620] Updated weights for policy 1, policy_version 584524 (0.0010) [2023-12-26 19:36:58,443][105620] Updated weights for policy 1, policy_version 584534 (0.0010) [2023-12-26 19:36:58,502][105620] Updated weights for policy 1, policy_version 584544 (0.0010) [2023-12-26 19:36:59,011][105692] Updated weights for policy 0, policy_version 583608 (0.0010) [2023-12-26 19:36:59,066][105692] Updated weights for policy 0, policy_version 583618 (0.0013) [2023-12-26 19:36:59,129][105692] Updated weights for policy 0, policy_version 583628 (0.0009) [2023-12-26 19:36:59,301][105620] Updated weights for policy 1, policy_version 584554 (0.0010) [2023-12-26 19:36:59,365][105620] Updated weights for policy 1, policy_version 584564 (0.0007) [2023-12-26 19:36:59,419][105620] Updated weights for policy 1, policy_version 584574 (0.0007) [2023-12-26 19:36:59,469][105620] Updated weights for policy 1, policy_version 584584 (0.0006) [2023-12-26 19:36:59,869][105692] Updated weights for policy 0, policy_version 583638 (0.0009) [2023-12-26 19:36:59,935][105692] Updated weights for policy 0, policy_version 583648 (0.0009) [2023-12-26 19:36:59,988][105692] Updated weights for policy 0, policy_version 583658 (0.0008) [2023-12-26 19:37:00,231][105620] Updated weights for policy 1, policy_version 584594 (0.0008) [2023-12-26 19:37:00,287][105620] Updated weights for policy 1, policy_version 584604 (0.0009) [2023-12-26 19:37:00,345][105620] Updated weights for policy 1, policy_version 584614 (0.0008) [2023-12-26 19:37:00,760][105692] Updated weights for policy 0, policy_version 583668 (0.0009) [2023-12-26 19:37:00,811][105692] Updated weights for policy 0, policy_version 583678 (0.0010) [2023-12-26 19:37:00,856][105692] Updated weights for policy 0, policy_version 583688 (0.0005) [2023-12-26 19:37:00,924][105620] Updated weights for policy 1, policy_version 584624 (0.0006) [2023-12-26 19:37:00,975][105620] Updated weights for policy 1, policy_version 584634 (0.0005) [2023-12-26 19:37:01,025][105620] Updated weights for policy 1, policy_version 584644 (0.0006) [2023-12-26 19:37:01,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 299130880. Throughput: 0: 9818.2, 1: 10023.8. Samples: 299094796. Policy #0 lag: (min: 1.0, avg: 15.9, max: 33.0) [2023-12-26 19:37:01,063][104569] Avg episode reward: [(0, '9090.934'), (1, '9258.444')] [2023-12-26 19:37:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000583696_149446656.pth... [2023-12-26 19:37:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000584648_149684224.pth... [2023-12-26 19:37:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000583464_149381120.pth [2023-12-26 19:37:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000582544_149151744.pth [2023-12-26 19:37:01,539][105692] Updated weights for policy 0, policy_version 583698 (0.0005) [2023-12-26 19:37:01,589][105692] Updated weights for policy 0, policy_version 583708 (0.0005) [2023-12-26 19:37:01,647][105692] Updated weights for policy 0, policy_version 583718 (0.0006) [2023-12-26 19:37:01,700][105692] Updated weights for policy 0, policy_version 583728 (0.0007) [2023-12-26 19:37:01,803][105620] Updated weights for policy 1, policy_version 584654 (0.0008) [2023-12-26 19:37:01,858][105620] Updated weights for policy 1, policy_version 584664 (0.0007) [2023-12-26 19:37:01,909][105620] Updated weights for policy 1, policy_version 584674 (0.0008) [2023-12-26 19:37:02,381][105692] Updated weights for policy 0, policy_version 583738 (0.0010) [2023-12-26 19:37:02,436][105692] Updated weights for policy 0, policy_version 583748 (0.0009) [2023-12-26 19:37:02,487][105692] Updated weights for policy 0, policy_version 583758 (0.0009) [2023-12-26 19:37:02,704][105620] Updated weights for policy 1, policy_version 584684 (0.0008) [2023-12-26 19:37:02,766][105620] Updated weights for policy 1, policy_version 584694 (0.0008) [2023-12-26 19:37:02,825][105620] Updated weights for policy 1, policy_version 584704 (0.0008) [2023-12-26 19:37:03,221][105692] Updated weights for policy 0, policy_version 583768 (0.0010) [2023-12-26 19:37:03,279][105692] Updated weights for policy 0, policy_version 583778 (0.0010) [2023-12-26 19:37:03,337][105692] Updated weights for policy 0, policy_version 583788 (0.0010) [2023-12-26 19:37:03,495][105620] Updated weights for policy 1, policy_version 584714 (0.0008) [2023-12-26 19:37:03,554][105620] Updated weights for policy 1, policy_version 584724 (0.0008) [2023-12-26 19:37:03,598][105620] Updated weights for policy 1, policy_version 584734 (0.0008) [2023-12-26 19:37:03,650][105620] Updated weights for policy 1, policy_version 584744 (0.0008) [2023-12-26 19:37:04,058][105692] Updated weights for policy 0, policy_version 583798 (0.0010) [2023-12-26 19:37:04,107][105692] Updated weights for policy 0, policy_version 583808 (0.0010) [2023-12-26 19:37:04,170][105692] Updated weights for policy 0, policy_version 583818 (0.0011) [2023-12-26 19:37:04,267][105620] Updated weights for policy 1, policy_version 584754 (0.0008) [2023-12-26 19:37:04,325][105620] Updated weights for policy 1, policy_version 584764 (0.0007) [2023-12-26 19:37:04,385][105620] Updated weights for policy 1, policy_version 584774 (0.0008) [2023-12-26 19:37:04,878][105692] Updated weights for policy 0, policy_version 583828 (0.0008) [2023-12-26 19:37:04,929][105692] Updated weights for policy 0, policy_version 583838 (0.0005) [2023-12-26 19:37:04,982][105692] Updated weights for policy 0, policy_version 583848 (0.0005) [2023-12-26 19:37:05,187][105620] Updated weights for policy 1, policy_version 584784 (0.0008) [2023-12-26 19:37:05,245][105620] Updated weights for policy 1, policy_version 584794 (0.0007) [2023-12-26 19:37:05,297][105620] Updated weights for policy 1, policy_version 584804 (0.0007) [2023-12-26 19:37:05,673][105692] Updated weights for policy 0, policy_version 583858 (0.0009) [2023-12-26 19:37:05,737][105692] Updated weights for policy 0, policy_version 583868 (0.0010) [2023-12-26 19:37:05,785][105692] Updated weights for policy 0, policy_version 583878 (0.0010) [2023-12-26 19:37:05,846][105692] Updated weights for policy 0, policy_version 583888 (0.0010) [2023-12-26 19:37:05,852][105620] Updated weights for policy 1, policy_version 584814 (0.0005) [2023-12-26 19:37:05,910][105620] Updated weights for policy 1, policy_version 584824 (0.0008) [2023-12-26 19:37:05,961][105620] Updated weights for policy 1, policy_version 584834 (0.0008) [2023-12-26 19:37:06,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 299229184. Throughput: 0: 9737.5, 1: 10025.3. Samples: 299211092. Policy #0 lag: (min: 1.0, avg: 15.9, max: 33.0) [2023-12-26 19:37:06,063][104569] Avg episode reward: [(0, '9174.858'), (1, '9080.634')] [2023-12-26 19:37:06,598][105692] Updated weights for policy 0, policy_version 583898 (0.0011) [2023-12-26 19:37:06,654][105692] Updated weights for policy 0, policy_version 583908 (0.0011) [2023-12-26 19:37:06,689][105620] Updated weights for policy 1, policy_version 584844 (0.0007) [2023-12-26 19:37:06,711][105692] Updated weights for policy 0, policy_version 583918 (0.0011) [2023-12-26 19:37:06,743][105620] Updated weights for policy 1, policy_version 584854 (0.0006) [2023-12-26 19:37:06,803][105620] Updated weights for policy 1, policy_version 584864 (0.0008) [2023-12-26 19:37:07,457][105692] Updated weights for policy 0, policy_version 583928 (0.0008) [2023-12-26 19:37:07,515][105692] Updated weights for policy 0, policy_version 583939 (0.0010) [2023-12-26 19:37:07,550][105620] Updated weights for policy 1, policy_version 584874 (0.0008) [2023-12-26 19:37:07,574][105692] Updated weights for policy 0, policy_version 583949 (0.0007) [2023-12-26 19:37:07,604][105620] Updated weights for policy 1, policy_version 584884 (0.0005) [2023-12-26 19:37:07,661][105620] Updated weights for policy 1, policy_version 584894 (0.0005) [2023-12-26 19:37:07,714][105620] Updated weights for policy 1, policy_version 584904 (0.0005) [2023-12-26 19:37:08,260][105620] Updated weights for policy 1, policy_version 584914 (0.0005) [2023-12-26 19:37:08,267][105692] Updated weights for policy 0, policy_version 583959 (0.0010) [2023-12-26 19:37:08,324][105620] Updated weights for policy 1, policy_version 584924 (0.0006) [2023-12-26 19:37:08,325][105692] Updated weights for policy 0, policy_version 583969 (0.0007) [2023-12-26 19:37:08,386][105620] Updated weights for policy 1, policy_version 584934 (0.0011) [2023-12-26 19:37:08,386][105692] Updated weights for policy 0, policy_version 583979 (0.0008) [2023-12-26 19:37:09,002][105620] Updated weights for policy 1, policy_version 584944 (0.0009) [2023-12-26 19:37:09,067][105620] Updated weights for policy 1, policy_version 584954 (0.0009) [2023-12-26 19:37:09,119][105692] Updated weights for policy 0, policy_version 583989 (0.0008) [2023-12-26 19:37:09,125][105620] Updated weights for policy 1, policy_version 584964 (0.0010) [2023-12-26 19:37:09,174][105692] Updated weights for policy 0, policy_version 583999 (0.0005) [2023-12-26 19:37:09,230][105692] Updated weights for policy 0, policy_version 584009 (0.0011) [2023-12-26 19:37:09,940][105620] Updated weights for policy 1, policy_version 584974 (0.0008) [2023-12-26 19:37:09,946][105692] Updated weights for policy 0, policy_version 584019 (0.0011) [2023-12-26 19:37:09,995][105620] Updated weights for policy 1, policy_version 584984 (0.0008) [2023-12-26 19:37:10,005][105692] Updated weights for policy 0, policy_version 584029 (0.0011) [2023-12-26 19:37:10,061][105692] Updated weights for policy 0, policy_version 584039 (0.0010) [2023-12-26 19:37:10,063][105620] Updated weights for policy 1, policy_version 584994 (0.0006) [2023-12-26 19:37:10,713][105620] Updated weights for policy 1, policy_version 585004 (0.0006) [2023-12-26 19:37:10,776][105620] Updated weights for policy 1, policy_version 585014 (0.0006) [2023-12-26 19:37:10,821][105692] Updated weights for policy 0, policy_version 584049 (0.0010) [2023-12-26 19:37:10,837][105620] Updated weights for policy 1, policy_version 585024 (0.0005) [2023-12-26 19:37:10,880][105692] Updated weights for policy 0, policy_version 584059 (0.0011) [2023-12-26 19:37:10,941][105692] Updated weights for policy 0, policy_version 584069 (0.0010) [2023-12-26 19:37:10,990][105692] Updated weights for policy 0, policy_version 584079 (0.0010) [2023-12-26 19:37:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19933.8, 300 sec: 19660.8). Total num frames: 299327488. Throughput: 0: 9794.0, 1: 10058.6. Samples: 299331520. Policy #0 lag: (min: 1.0, avg: 15.9, max: 33.0) [2023-12-26 19:37:11,063][104569] Avg episode reward: [(0, '9081.316'), (1, '8904.079')] [2023-12-26 19:37:11,521][105620] Updated weights for policy 1, policy_version 585034 (0.0006) [2023-12-26 19:37:11,571][105620] Updated weights for policy 1, policy_version 585044 (0.0008) [2023-12-26 19:37:11,624][105620] Updated weights for policy 1, policy_version 585054 (0.0009) [2023-12-26 19:37:11,683][105620] Updated weights for policy 1, policy_version 585064 (0.0008) [2023-12-26 19:37:11,780][105692] Updated weights for policy 0, policy_version 584089 (0.0011) [2023-12-26 19:37:11,848][105692] Updated weights for policy 0, policy_version 584099 (0.0006) [2023-12-26 19:37:11,914][105692] Updated weights for policy 0, policy_version 584109 (0.0010) [2023-12-26 19:37:12,419][105620] Updated weights for policy 1, policy_version 585074 (0.0005) [2023-12-26 19:37:12,477][105620] Updated weights for policy 1, policy_version 585084 (0.0006) [2023-12-26 19:37:12,537][105620] Updated weights for policy 1, policy_version 585094 (0.0009) [2023-12-26 19:37:12,585][105692] Updated weights for policy 0, policy_version 584119 (0.0007) [2023-12-26 19:37:12,655][105692] Updated weights for policy 0, policy_version 584129 (0.0006) [2023-12-26 19:37:12,723][105692] Updated weights for policy 0, policy_version 584139 (0.0009) [2023-12-26 19:37:13,223][105620] Updated weights for policy 1, policy_version 585104 (0.0008) [2023-12-26 19:37:13,271][105620] Updated weights for policy 1, policy_version 585114 (0.0007) [2023-12-26 19:37:13,316][105620] Updated weights for policy 1, policy_version 585124 (0.0008) [2023-12-26 19:37:13,353][105692] Updated weights for policy 0, policy_version 584149 (0.0010) [2023-12-26 19:37:13,397][105692] Updated weights for policy 0, policy_version 584159 (0.0010) [2023-12-26 19:37:13,466][105692] Updated weights for policy 0, policy_version 584169 (0.0010) [2023-12-26 19:37:14,095][105692] Updated weights for policy 0, policy_version 584179 (0.0009) [2023-12-26 19:37:14,095][105620] Updated weights for policy 1, policy_version 585134 (0.0010) [2023-12-26 19:37:14,146][105692] Updated weights for policy 0, policy_version 584189 (0.0010) [2023-12-26 19:37:14,148][105620] Updated weights for policy 1, policy_version 585144 (0.0006) [2023-12-26 19:37:14,198][105692] Updated weights for policy 0, policy_version 584199 (0.0010) [2023-12-26 19:37:14,208][105620] Updated weights for policy 1, policy_version 585154 (0.0005) [2023-12-26 19:37:14,934][105692] Updated weights for policy 0, policy_version 584209 (0.0010) [2023-12-26 19:37:14,989][105620] Updated weights for policy 1, policy_version 585164 (0.0007) [2023-12-26 19:37:15,002][105692] Updated weights for policy 0, policy_version 584219 (0.0011) [2023-12-26 19:37:15,051][105620] Updated weights for policy 1, policy_version 585174 (0.0006) [2023-12-26 19:37:15,064][105692] Updated weights for policy 0, policy_version 584229 (0.0010) [2023-12-26 19:37:15,114][105620] Updated weights for policy 1, policy_version 585184 (0.0006) [2023-12-26 19:37:15,127][105692] Updated weights for policy 0, policy_version 584239 (0.0010) [2023-12-26 19:37:15,807][105620] Updated weights for policy 1, policy_version 585194 (0.0007) [2023-12-26 19:37:15,864][105620] Updated weights for policy 1, policy_version 585204 (0.0005) [2023-12-26 19:37:15,885][105692] Updated weights for policy 0, policy_version 584249 (0.0009) [2023-12-26 19:37:15,913][105620] Updated weights for policy 1, policy_version 585214 (0.0007) [2023-12-26 19:37:15,931][105692] Updated weights for policy 0, policy_version 584259 (0.0006) [2023-12-26 19:37:15,961][105620] Updated weights for policy 1, policy_version 585224 (0.0010) [2023-12-26 19:37:15,975][105692] Updated weights for policy 0, policy_version 584269 (0.0005) [2023-12-26 19:37:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19933.9, 300 sec: 19688.6). Total num frames: 299425792. Throughput: 0: 9694.7, 1: 10000.6. Samples: 299389684. Policy #0 lag: (min: 28.0, avg: 33.6, max: 56.0) [2023-12-26 19:37:16,062][104569] Avg episode reward: [(0, '8676.279'), (1, '8718.965')] [2023-12-26 19:37:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000584272_149594112.pth... [2023-12-26 19:37:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000585224_149831680.pth... [2023-12-26 19:37:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000583152_149307392.pth [2023-12-26 19:37:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000584040_149528576.pth [2023-12-26 19:37:16,589][105692] Updated weights for policy 0, policy_version 584279 (0.0006) [2023-12-26 19:37:16,646][105620] Updated weights for policy 1, policy_version 585234 (0.0007) [2023-12-26 19:37:16,650][105692] Updated weights for policy 0, policy_version 584289 (0.0006) [2023-12-26 19:37:16,701][105620] Updated weights for policy 1, policy_version 585244 (0.0009) [2023-12-26 19:37:16,702][105692] Updated weights for policy 0, policy_version 584299 (0.0006) [2023-12-26 19:37:16,752][105620] Updated weights for policy 1, policy_version 585255 (0.0009) [2023-12-26 19:37:17,244][105692] Updated weights for policy 0, policy_version 584309 (0.0006) [2023-12-26 19:37:17,301][105692] Updated weights for policy 0, policy_version 584319 (0.0005) [2023-12-26 19:37:17,361][105692] Updated weights for policy 0, policy_version 584329 (0.0005) [2023-12-26 19:37:17,491][105620] Updated weights for policy 1, policy_version 585265 (0.0006) [2023-12-26 19:37:17,541][105620] Updated weights for policy 1, policy_version 585275 (0.0005) [2023-12-26 19:37:17,597][105620] Updated weights for policy 1, policy_version 585285 (0.0005) [2023-12-26 19:37:18,053][105692] Updated weights for policy 0, policy_version 584339 (0.0007) [2023-12-26 19:37:18,114][105692] Updated weights for policy 0, policy_version 584349 (0.0010) [2023-12-26 19:37:18,169][105692] Updated weights for policy 0, policy_version 584359 (0.0010) [2023-12-26 19:37:18,175][105620] Updated weights for policy 1, policy_version 585295 (0.0005) [2023-12-26 19:37:18,223][105620] Updated weights for policy 1, policy_version 585305 (0.0006) [2023-12-26 19:37:18,277][105620] Updated weights for policy 1, policy_version 585315 (0.0008) [2023-12-26 19:37:18,834][105692] Updated weights for policy 0, policy_version 584369 (0.0010) [2023-12-26 19:37:18,890][105692] Updated weights for policy 0, policy_version 584379 (0.0010) [2023-12-26 19:37:18,944][105692] Updated weights for policy 0, policy_version 584389 (0.0010) [2023-12-26 19:37:19,008][105692] Updated weights for policy 0, policy_version 584399 (0.0011) [2023-12-26 19:37:19,049][105620] Updated weights for policy 1, policy_version 585325 (0.0009) [2023-12-26 19:37:19,108][105620] Updated weights for policy 1, policy_version 585335 (0.0008) [2023-12-26 19:37:19,164][105620] Updated weights for policy 1, policy_version 585345 (0.0008) [2023-12-26 19:37:19,778][105692] Updated weights for policy 0, policy_version 584409 (0.0010) [2023-12-26 19:37:19,844][105692] Updated weights for policy 0, policy_version 584419 (0.0011) [2023-12-26 19:37:19,912][105692] Updated weights for policy 0, policy_version 584429 (0.0008) [2023-12-26 19:37:19,930][105620] Updated weights for policy 1, policy_version 585355 (0.0008) [2023-12-26 19:37:20,000][105620] Updated weights for policy 1, policy_version 585365 (0.0007) [2023-12-26 19:37:20,061][105620] Updated weights for policy 1, policy_version 585375 (0.0007) [2023-12-26 19:37:20,674][105692] Updated weights for policy 0, policy_version 584439 (0.0011) [2023-12-26 19:37:20,731][105692] Updated weights for policy 0, policy_version 584449 (0.0011) [2023-12-26 19:37:20,781][105692] Updated weights for policy 0, policy_version 584459 (0.0011) [2023-12-26 19:37:20,825][105620] Updated weights for policy 1, policy_version 585385 (0.0008) [2023-12-26 19:37:20,886][105620] Updated weights for policy 1, policy_version 585395 (0.0009) [2023-12-26 19:37:20,935][105620] Updated weights for policy 1, policy_version 585405 (0.0010) [2023-12-26 19:37:20,992][105620] Updated weights for policy 1, policy_version 585415 (0.0011) [2023-12-26 19:37:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19933.9, 300 sec: 19688.6). Total num frames: 299524096. Throughput: 0: 9751.1, 1: 9980.5. Samples: 299510332. Policy #0 lag: (min: 28.0, avg: 33.6, max: 56.0) [2023-12-26 19:37:21,062][104569] Avg episode reward: [(0, '8585.468'), (1, '8405.621')] [2023-12-26 19:37:21,592][105692] Updated weights for policy 0, policy_version 584469 (0.0010) [2023-12-26 19:37:21,661][105692] Updated weights for policy 0, policy_version 584479 (0.0009) [2023-12-26 19:37:21,717][105620] Updated weights for policy 1, policy_version 585425 (0.0008) [2023-12-26 19:37:21,724][105692] Updated weights for policy 0, policy_version 584489 (0.0009) [2023-12-26 19:37:21,785][105620] Updated weights for policy 1, policy_version 585435 (0.0006) [2023-12-26 19:37:21,845][105620] Updated weights for policy 1, policy_version 585445 (0.0009) [2023-12-26 19:37:22,496][105692] Updated weights for policy 0, policy_version 584499 (0.0009) [2023-12-26 19:37:22,560][105692] Updated weights for policy 0, policy_version 584509 (0.0007) [2023-12-26 19:37:22,611][105620] Updated weights for policy 1, policy_version 585455 (0.0007) [2023-12-26 19:37:22,625][105692] Updated weights for policy 0, policy_version 584519 (0.0008) [2023-12-26 19:37:22,674][105620] Updated weights for policy 1, policy_version 585465 (0.0007) [2023-12-26 19:37:22,735][105620] Updated weights for policy 1, policy_version 585475 (0.0008) [2023-12-26 19:37:23,315][105692] Updated weights for policy 0, policy_version 584529 (0.0007) [2023-12-26 19:37:23,363][105692] Updated weights for policy 0, policy_version 584539 (0.0006) [2023-12-26 19:37:23,408][105692] Updated weights for policy 0, policy_version 584549 (0.0005) [2023-12-26 19:37:23,451][105692] Updated weights for policy 0, policy_version 584559 (0.0005) [2023-12-26 19:37:23,528][105620] Updated weights for policy 1, policy_version 585485 (0.0008) [2023-12-26 19:37:23,595][105620] Updated weights for policy 1, policy_version 585495 (0.0008) [2023-12-26 19:37:23,658][105620] Updated weights for policy 1, policy_version 585505 (0.0007) [2023-12-26 19:37:24,090][105692] Updated weights for policy 0, policy_version 584569 (0.0009) [2023-12-26 19:37:24,143][105692] Updated weights for policy 0, policy_version 584579 (0.0008) [2023-12-26 19:37:24,198][105692] Updated weights for policy 0, policy_version 584589 (0.0009) [2023-12-26 19:37:24,390][105620] Updated weights for policy 1, policy_version 585515 (0.0007) [2023-12-26 19:37:24,433][105620] Updated weights for policy 1, policy_version 585525 (0.0005) [2023-12-26 19:37:24,484][105620] Updated weights for policy 1, policy_version 585535 (0.0005) [2023-12-26 19:37:25,014][105692] Updated weights for policy 0, policy_version 584599 (0.0009) [2023-12-26 19:37:25,072][105692] Updated weights for policy 0, policy_version 584609 (0.0010) [2023-12-26 19:37:25,116][105620] Updated weights for policy 1, policy_version 585545 (0.0008) [2023-12-26 19:37:25,126][105692] Updated weights for policy 0, policy_version 584619 (0.0009) [2023-12-26 19:37:25,183][105620] Updated weights for policy 1, policy_version 585555 (0.0005) [2023-12-26 19:37:25,243][105620] Updated weights for policy 1, policy_version 585565 (0.0006) [2023-12-26 19:37:25,301][105620] Updated weights for policy 1, policy_version 585575 (0.0005) [2023-12-26 19:37:25,937][105692] Updated weights for policy 0, policy_version 584629 (0.0010) [2023-12-26 19:37:25,959][105620] Updated weights for policy 1, policy_version 585585 (0.0007) [2023-12-26 19:37:25,997][105692] Updated weights for policy 0, policy_version 584639 (0.0008) [2023-12-26 19:37:26,012][105620] Updated weights for policy 1, policy_version 585595 (0.0006) [2023-12-26 19:37:26,054][105692] Updated weights for policy 0, policy_version 584649 (0.0008) [2023-12-26 19:37:26,062][105620] Updated weights for policy 1, policy_version 585605 (0.0005) [2023-12-26 19:37:26,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 299606016. Throughput: 0: 9699.0, 1: 9920.9. Samples: 299623084. Policy #0 lag: (min: 28.0, avg: 33.6, max: 56.0) [2023-12-26 19:37:26,062][104569] Avg episode reward: [(0, '8746.334'), (1, '2204.327')] [2023-12-26 19:37:26,727][105692] Updated weights for policy 0, policy_version 584659 (0.0007) [2023-12-26 19:37:26,784][105692] Updated weights for policy 0, policy_version 584669 (0.0007) [2023-12-26 19:37:26,815][105620] Updated weights for policy 1, policy_version 585615 (0.0010) [2023-12-26 19:37:26,837][105692] Updated weights for policy 0, policy_version 584679 (0.0007) [2023-12-26 19:37:26,872][105620] Updated weights for policy 1, policy_version 585625 (0.0010) [2023-12-26 19:37:26,929][105620] Updated weights for policy 1, policy_version 585635 (0.0010) [2023-12-26 19:37:27,553][105692] Updated weights for policy 0, policy_version 584689 (0.0009) [2023-12-26 19:37:27,610][105692] Updated weights for policy 0, policy_version 584699 (0.0008) [2023-12-26 19:37:27,650][105620] Updated weights for policy 1, policy_version 585645 (0.0010) [2023-12-26 19:37:27,660][105692] Updated weights for policy 0, policy_version 584709 (0.0007) [2023-12-26 19:37:27,690][105620] Updated weights for policy 1, policy_version 585655 (0.0010) [2023-12-26 19:37:27,716][105692] Updated weights for policy 0, policy_version 584719 (0.0006) [2023-12-26 19:37:27,741][105620] Updated weights for policy 1, policy_version 585665 (0.0010) [2023-12-26 19:37:28,389][105692] Updated weights for policy 0, policy_version 584729 (0.0008) [2023-12-26 19:37:28,453][105692] Updated weights for policy 0, policy_version 584739 (0.0008) [2023-12-26 19:37:28,514][105692] Updated weights for policy 0, policy_version 584749 (0.0007) [2023-12-26 19:37:28,523][105620] Updated weights for policy 1, policy_version 585675 (0.0010) [2023-12-26 19:37:28,587][105620] Updated weights for policy 1, policy_version 585685 (0.0010) [2023-12-26 19:37:28,652][105620] Updated weights for policy 1, policy_version 585695 (0.0010) [2023-12-26 19:37:29,239][105692] Updated weights for policy 0, policy_version 584759 (0.0007) [2023-12-26 19:37:29,303][105692] Updated weights for policy 0, policy_version 584769 (0.0010) [2023-12-26 19:37:29,369][105692] Updated weights for policy 0, policy_version 584779 (0.0009) [2023-12-26 19:37:29,371][105620] Updated weights for policy 1, policy_version 585705 (0.0010) [2023-12-26 19:37:29,430][105620] Updated weights for policy 1, policy_version 585715 (0.0011) [2023-12-26 19:37:29,483][105620] Updated weights for policy 1, policy_version 585725 (0.0011) [2023-12-26 19:37:29,535][105620] Updated weights for policy 1, policy_version 585735 (0.0010) [2023-12-26 19:37:29,959][105692] Updated weights for policy 0, policy_version 584789 (0.0008) [2023-12-26 19:37:30,022][105692] Updated weights for policy 0, policy_version 584799 (0.0011) [2023-12-26 19:37:30,074][105692] Updated weights for policy 0, policy_version 584809 (0.0010) [2023-12-26 19:37:30,309][105620] Updated weights for policy 1, policy_version 585745 (0.0008) [2023-12-26 19:37:30,361][105620] Updated weights for policy 1, policy_version 585755 (0.0008) [2023-12-26 19:37:30,417][105620] Updated weights for policy 1, policy_version 585765 (0.0008) [2023-12-26 19:37:30,783][105692] Updated weights for policy 0, policy_version 584819 (0.0006) [2023-12-26 19:37:30,846][105692] Updated weights for policy 0, policy_version 584829 (0.0008) [2023-12-26 19:37:30,908][105692] Updated weights for policy 0, policy_version 584839 (0.0009) [2023-12-26 19:37:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 299712512. Throughput: 0: 9710.0, 1: 9883.6. Samples: 299681072. Policy #0 lag: (min: 28.0, avg: 33.6, max: 56.0) [2023-12-26 19:37:31,063][104569] Avg episode reward: [(0, '9087.199'), (1, '2034.027')] [2023-12-26 19:37:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000584848_149741568.pth... [2023-12-26 19:37:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000585768_149970944.pth... [2023-12-26 19:37:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000584648_149684224.pth [2023-12-26 19:37:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000583696_149446656.pth [2023-12-26 19:37:31,197][105620] Updated weights for policy 1, policy_version 585775 (0.0009) [2023-12-26 19:37:31,257][105620] Updated weights for policy 1, policy_version 585785 (0.0008) [2023-12-26 19:37:31,315][105620] Updated weights for policy 1, policy_version 585795 (0.0009) [2023-12-26 19:37:31,660][105692] Updated weights for policy 0, policy_version 584849 (0.0008) [2023-12-26 19:37:31,723][105692] Updated weights for policy 0, policy_version 584859 (0.0009) [2023-12-26 19:37:31,780][105692] Updated weights for policy 0, policy_version 584869 (0.0008) [2023-12-26 19:37:31,827][105692] Updated weights for policy 0, policy_version 584879 (0.0008) [2023-12-26 19:37:32,084][105620] Updated weights for policy 1, policy_version 585805 (0.0010) [2023-12-26 19:37:32,139][105620] Updated weights for policy 1, policy_version 585815 (0.0009) [2023-12-26 19:37:32,193][105620] Updated weights for policy 1, policy_version 585825 (0.0010) [2023-12-26 19:37:32,523][105692] Updated weights for policy 0, policy_version 584889 (0.0006) [2023-12-26 19:37:32,579][105692] Updated weights for policy 0, policy_version 584899 (0.0006) [2023-12-26 19:37:32,642][105692] Updated weights for policy 0, policy_version 584909 (0.0008) [2023-12-26 19:37:32,879][105620] Updated weights for policy 1, policy_version 585835 (0.0008) [2023-12-26 19:37:32,944][105620] Updated weights for policy 1, policy_version 585845 (0.0007) [2023-12-26 19:37:33,004][105620] Updated weights for policy 1, policy_version 585855 (0.0008) [2023-12-26 19:37:33,417][105692] Updated weights for policy 0, policy_version 584919 (0.0006) [2023-12-26 19:37:33,472][105692] Updated weights for policy 0, policy_version 584929 (0.0005) [2023-12-26 19:37:33,529][105692] Updated weights for policy 0, policy_version 584939 (0.0008) [2023-12-26 19:37:33,641][105620] Updated weights for policy 1, policy_version 585865 (0.0010) [2023-12-26 19:37:33,696][105620] Updated weights for policy 1, policy_version 585875 (0.0005) [2023-12-26 19:37:33,749][105620] Updated weights for policy 1, policy_version 585885 (0.0005) [2023-12-26 19:37:33,797][105620] Updated weights for policy 1, policy_version 585895 (0.0008) [2023-12-26 19:37:34,214][105692] Updated weights for policy 0, policy_version 584949 (0.0007) [2023-12-26 19:37:34,282][105692] Updated weights for policy 0, policy_version 584959 (0.0006) [2023-12-26 19:37:34,348][105692] Updated weights for policy 0, policy_version 584969 (0.0010) [2023-12-26 19:37:34,478][105620] Updated weights for policy 1, policy_version 585905 (0.0010) [2023-12-26 19:37:34,537][105620] Updated weights for policy 1, policy_version 585915 (0.0008) [2023-12-26 19:37:34,595][105620] Updated weights for policy 1, policy_version 585925 (0.0008) [2023-12-26 19:37:35,007][105692] Updated weights for policy 0, policy_version 584979 (0.0009) [2023-12-26 19:37:35,063][105692] Updated weights for policy 0, policy_version 584989 (0.0008) [2023-12-26 19:37:35,122][105692] Updated weights for policy 0, policy_version 584999 (0.0008) [2023-12-26 19:37:35,366][105620] Updated weights for policy 1, policy_version 585935 (0.0009) [2023-12-26 19:37:35,417][105620] Updated weights for policy 1, policy_version 585945 (0.0009) [2023-12-26 19:37:35,476][105620] Updated weights for policy 1, policy_version 585955 (0.0009) [2023-12-26 19:37:35,851][105692] Updated weights for policy 0, policy_version 585009 (0.0009) [2023-12-26 19:37:35,898][105692] Updated weights for policy 0, policy_version 585019 (0.0009) [2023-12-26 19:37:35,948][105692] Updated weights for policy 0, policy_version 585029 (0.0008) [2023-12-26 19:37:36,000][105692] Updated weights for policy 0, policy_version 585039 (0.0008) [2023-12-26 19:37:36,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 299810816. Throughput: 0: 9658.6, 1: 9820.8. Samples: 299797860. Policy #0 lag: (min: 28.0, avg: 33.6, max: 56.0) [2023-12-26 19:37:36,063][104569] Avg episode reward: [(0, '8512.603'), (1, '6508.417')] [2023-12-26 19:37:36,214][105620] Updated weights for policy 1, policy_version 585965 (0.0009) [2023-12-26 19:37:36,282][105620] Updated weights for policy 1, policy_version 585975 (0.0008) [2023-12-26 19:37:36,344][105620] Updated weights for policy 1, policy_version 585985 (0.0009) [2023-12-26 19:37:36,743][105692] Updated weights for policy 0, policy_version 585049 (0.0005) [2023-12-26 19:37:36,812][105692] Updated weights for policy 0, policy_version 585059 (0.0005) [2023-12-26 19:37:36,876][105692] Updated weights for policy 0, policy_version 585069 (0.0009) [2023-12-26 19:37:37,086][105620] Updated weights for policy 1, policy_version 585995 (0.0009) [2023-12-26 19:37:37,150][105620] Updated weights for policy 1, policy_version 586005 (0.0009) [2023-12-26 19:37:37,212][105620] Updated weights for policy 1, policy_version 586015 (0.0008) [2023-12-26 19:37:37,490][105692] Updated weights for policy 0, policy_version 585079 (0.0007) [2023-12-26 19:37:37,548][105692] Updated weights for policy 0, policy_version 585089 (0.0006) [2023-12-26 19:37:37,605][105692] Updated weights for policy 0, policy_version 585099 (0.0005) [2023-12-26 19:37:38,044][105620] Updated weights for policy 1, policy_version 586025 (0.0009) [2023-12-26 19:37:38,100][105620] Updated weights for policy 1, policy_version 586035 (0.0008) [2023-12-26 19:37:38,159][105620] Updated weights for policy 1, policy_version 586045 (0.0009) [2023-12-26 19:37:38,177][105692] Updated weights for policy 0, policy_version 585109 (0.0006) [2023-12-26 19:37:38,217][105620] Updated weights for policy 1, policy_version 586055 (0.0007) [2023-12-26 19:37:38,240][105692] Updated weights for policy 0, policy_version 585119 (0.0009) [2023-12-26 19:37:38,296][105692] Updated weights for policy 0, policy_version 585129 (0.0010) [2023-12-26 19:37:38,885][105692] Updated weights for policy 0, policy_version 585139 (0.0010) [2023-12-26 19:37:38,941][105692] Updated weights for policy 0, policy_version 585149 (0.0010) [2023-12-26 19:37:38,999][105620] Updated weights for policy 1, policy_version 586065 (0.0006) [2023-12-26 19:37:39,001][105692] Updated weights for policy 0, policy_version 585159 (0.0011) [2023-12-26 19:37:39,054][105620] Updated weights for policy 1, policy_version 586075 (0.0006) [2023-12-26 19:37:39,113][105620] Updated weights for policy 1, policy_version 586085 (0.0008) [2023-12-26 19:37:39,689][105692] Updated weights for policy 0, policy_version 585169 (0.0010) [2023-12-26 19:37:39,743][105692] Updated weights for policy 0, policy_version 585179 (0.0006) [2023-12-26 19:37:39,799][105692] Updated weights for policy 0, policy_version 585189 (0.0006) [2023-12-26 19:37:39,863][105692] Updated weights for policy 0, policy_version 585199 (0.0009) [2023-12-26 19:37:39,966][105620] Updated weights for policy 1, policy_version 586095 (0.0009) [2023-12-26 19:37:40,025][105620] Updated weights for policy 1, policy_version 586105 (0.0009) [2023-12-26 19:37:40,090][105620] Updated weights for policy 1, policy_version 586115 (0.0010) [2023-12-26 19:37:40,558][105692] Updated weights for policy 0, policy_version 585209 (0.0009) [2023-12-26 19:37:40,614][105692] Updated weights for policy 0, policy_version 585219 (0.0009) [2023-12-26 19:37:40,673][105692] Updated weights for policy 0, policy_version 585229 (0.0009) [2023-12-26 19:37:40,871][105620] Updated weights for policy 1, policy_version 586125 (0.0010) [2023-12-26 19:37:40,919][105620] Updated weights for policy 1, policy_version 586135 (0.0009) [2023-12-26 19:37:40,973][105620] Updated weights for policy 1, policy_version 586145 (0.0013) [2023-12-26 19:37:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.9, 300 sec: 19688.6). Total num frames: 299909120. Throughput: 0: 9835.9, 1: 9685.5. Samples: 299914172. Policy #0 lag: (min: 28.0, avg: 33.6, max: 56.0) [2023-12-26 19:37:41,062][104569] Avg episode reward: [(0, '8347.624'), (1, '8993.830')] [2023-12-26 19:37:41,466][105692] Updated weights for policy 0, policy_version 585239 (0.0009) [2023-12-26 19:37:41,518][105692] Updated weights for policy 0, policy_version 585249 (0.0009) [2023-12-26 19:37:41,573][105692] Updated weights for policy 0, policy_version 585259 (0.0009) [2023-12-26 19:37:41,786][105620] Updated weights for policy 1, policy_version 586155 (0.0010) [2023-12-26 19:37:41,846][105620] Updated weights for policy 1, policy_version 586165 (0.0009) [2023-12-26 19:37:41,900][105620] Updated weights for policy 1, policy_version 586175 (0.0009) [2023-12-26 19:37:42,380][105692] Updated weights for policy 0, policy_version 585269 (0.0008) [2023-12-26 19:37:42,438][105692] Updated weights for policy 0, policy_version 585279 (0.0010) [2023-12-26 19:37:42,498][105692] Updated weights for policy 0, policy_version 585289 (0.0009) [2023-12-26 19:37:42,595][105620] Updated weights for policy 1, policy_version 586185 (0.0005) [2023-12-26 19:37:42,661][105620] Updated weights for policy 1, policy_version 586195 (0.0006) [2023-12-26 19:37:42,723][105620] Updated weights for policy 1, policy_version 586205 (0.0007) [2023-12-26 19:37:42,778][105620] Updated weights for policy 1, policy_version 586215 (0.0009) [2023-12-26 19:37:43,315][105692] Updated weights for policy 0, policy_version 585299 (0.0008) [2023-12-26 19:37:43,359][105620] Updated weights for policy 1, policy_version 586225 (0.0008) [2023-12-26 19:37:43,366][105692] Updated weights for policy 0, policy_version 585309 (0.0005) [2023-12-26 19:37:43,411][105620] Updated weights for policy 1, policy_version 586235 (0.0007) [2023-12-26 19:37:43,417][105692] Updated weights for policy 0, policy_version 585319 (0.0007) [2023-12-26 19:37:43,461][105620] Updated weights for policy 1, policy_version 586245 (0.0006) [2023-12-26 19:37:44,005][105692] Updated weights for policy 0, policy_version 585329 (0.0007) [2023-12-26 19:37:44,054][105692] Updated weights for policy 0, policy_version 585339 (0.0005) [2023-12-26 19:37:44,110][105692] Updated weights for policy 0, policy_version 585349 (0.0005) [2023-12-26 19:37:44,146][105620] Updated weights for policy 1, policy_version 586255 (0.0007) [2023-12-26 19:37:44,173][105692] Updated weights for policy 0, policy_version 585359 (0.0005) [2023-12-26 19:37:44,208][105620] Updated weights for policy 1, policy_version 586265 (0.0005) [2023-12-26 19:37:44,275][105620] Updated weights for policy 1, policy_version 586275 (0.0006) [2023-12-26 19:37:44,794][105692] Updated weights for policy 0, policy_version 585369 (0.0009) [2023-12-26 19:37:44,852][105692] Updated weights for policy 0, policy_version 585379 (0.0008) [2023-12-26 19:37:44,910][105692] Updated weights for policy 0, policy_version 585389 (0.0007) [2023-12-26 19:37:44,920][105620] Updated weights for policy 1, policy_version 586285 (0.0009) [2023-12-26 19:37:44,976][105620] Updated weights for policy 1, policy_version 586295 (0.0011) [2023-12-26 19:37:45,040][105620] Updated weights for policy 1, policy_version 586305 (0.0011) [2023-12-26 19:37:45,665][105692] Updated weights for policy 0, policy_version 585399 (0.0007) [2023-12-26 19:37:45,709][105692] Updated weights for policy 0, policy_version 585409 (0.0007) [2023-12-26 19:37:45,753][105692] Updated weights for policy 0, policy_version 585419 (0.0008) [2023-12-26 19:37:45,807][105620] Updated weights for policy 1, policy_version 586315 (0.0011) [2023-12-26 19:37:45,862][105620] Updated weights for policy 1, policy_version 586325 (0.0010) [2023-12-26 19:37:45,920][105620] Updated weights for policy 1, policy_version 586335 (0.0010) [2023-12-26 19:37:46,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 300007424. Throughput: 0: 9789.9, 1: 9663.0. Samples: 299970172. Policy #0 lag: (min: 28.0, avg: 33.6, max: 56.0) [2023-12-26 19:37:46,062][104569] Avg episode reward: [(0, '8369.956'), (1, '8994.076')] [2023-12-26 19:37:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000585424_149889024.pth... [2023-12-26 19:37:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000586344_150118400.pth... [2023-12-26 19:37:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000584272_149594112.pth [2023-12-26 19:37:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000585224_149831680.pth [2023-12-26 19:37:46,463][105692] Updated weights for policy 0, policy_version 585429 (0.0009) [2023-12-26 19:37:46,508][105692] Updated weights for policy 0, policy_version 585439 (0.0011) [2023-12-26 19:37:46,560][105692] Updated weights for policy 0, policy_version 585449 (0.0009) [2023-12-26 19:37:46,587][105620] Updated weights for policy 1, policy_version 586345 (0.0010) [2023-12-26 19:37:46,641][105620] Updated weights for policy 1, policy_version 586355 (0.0008) [2023-12-26 19:37:46,697][105620] Updated weights for policy 1, policy_version 586365 (0.0009) [2023-12-26 19:37:46,755][105620] Updated weights for policy 1, policy_version 586375 (0.0008) [2023-12-26 19:37:47,273][105692] Updated weights for policy 0, policy_version 585459 (0.0010) [2023-12-26 19:37:47,329][105692] Updated weights for policy 0, policy_version 585469 (0.0010) [2023-12-26 19:37:47,339][105620] Updated weights for policy 1, policy_version 586385 (0.0007) [2023-12-26 19:37:47,377][105692] Updated weights for policy 0, policy_version 585479 (0.0010) [2023-12-26 19:37:47,393][105620] Updated weights for policy 1, policy_version 586395 (0.0006) [2023-12-26 19:37:47,457][105620] Updated weights for policy 1, policy_version 586405 (0.0008) [2023-12-26 19:37:48,092][105692] Updated weights for policy 0, policy_version 585489 (0.0010) [2023-12-26 19:37:48,125][105620] Updated weights for policy 1, policy_version 586415 (0.0008) [2023-12-26 19:37:48,143][105692] Updated weights for policy 0, policy_version 585499 (0.0007) [2023-12-26 19:37:48,189][105620] Updated weights for policy 1, policy_version 586425 (0.0009) [2023-12-26 19:37:48,193][105692] Updated weights for policy 0, policy_version 585509 (0.0009) [2023-12-26 19:37:48,244][105692] Updated weights for policy 0, policy_version 585519 (0.0008) [2023-12-26 19:37:48,255][105620] Updated weights for policy 1, policy_version 586435 (0.0008) [2023-12-26 19:37:48,967][105620] Updated weights for policy 1, policy_version 586445 (0.0008) [2023-12-26 19:37:48,977][105692] Updated weights for policy 0, policy_version 585529 (0.0007) [2023-12-26 19:37:49,026][105620] Updated weights for policy 1, policy_version 586455 (0.0010) [2023-12-26 19:37:49,027][105692] Updated weights for policy 0, policy_version 585539 (0.0011) [2023-12-26 19:37:49,080][105692] Updated weights for policy 0, policy_version 585549 (0.0010) [2023-12-26 19:37:49,083][105620] Updated weights for policy 1, policy_version 586465 (0.0009) [2023-12-26 19:37:49,762][105620] Updated weights for policy 1, policy_version 586475 (0.0008) [2023-12-26 19:37:49,800][105692] Updated weights for policy 0, policy_version 585559 (0.0010) [2023-12-26 19:37:49,824][105620] Updated weights for policy 1, policy_version 586485 (0.0006) [2023-12-26 19:37:49,864][105692] Updated weights for policy 0, policy_version 585569 (0.0012) [2023-12-26 19:37:49,882][105620] Updated weights for policy 1, policy_version 586495 (0.0006) [2023-12-26 19:37:49,927][105692] Updated weights for policy 0, policy_version 585579 (0.0010) [2023-12-26 19:37:50,572][105620] Updated weights for policy 1, policy_version 586505 (0.0007) [2023-12-26 19:37:50,641][105620] Updated weights for policy 1, policy_version 586515 (0.0008) [2023-12-26 19:37:50,705][105620] Updated weights for policy 1, policy_version 586525 (0.0008) [2023-12-26 19:37:50,719][105692] Updated weights for policy 0, policy_version 585589 (0.0010) [2023-12-26 19:37:50,767][105620] Updated weights for policy 1, policy_version 586535 (0.0006) [2023-12-26 19:37:50,782][105692] Updated weights for policy 0, policy_version 585599 (0.0011) [2023-12-26 19:37:50,843][105692] Updated weights for policy 0, policy_version 585609 (0.0011) [2023-12-26 19:37:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 300105728. Throughput: 0: 9860.3, 1: 9733.1. Samples: 300092792. Policy #0 lag: (min: 28.0, avg: 33.6, max: 56.0) [2023-12-26 19:37:51,063][104569] Avg episode reward: [(0, '3371.616'), (1, '9088.414')] [2023-12-26 19:37:51,542][105620] Updated weights for policy 1, policy_version 586545 (0.0008) [2023-12-26 19:37:51,566][105692] Updated weights for policy 0, policy_version 585619 (0.0009) [2023-12-26 19:37:51,602][105620] Updated weights for policy 1, policy_version 586555 (0.0008) [2023-12-26 19:37:51,631][105692] Updated weights for policy 0, policy_version 585629 (0.0007) [2023-12-26 19:37:51,667][105620] Updated weights for policy 1, policy_version 586565 (0.0008) [2023-12-26 19:37:51,695][105692] Updated weights for policy 0, policy_version 585639 (0.0009) [2023-12-26 19:37:52,393][105620] Updated weights for policy 1, policy_version 586575 (0.0009) [2023-12-26 19:37:52,449][105620] Updated weights for policy 1, policy_version 586585 (0.0009) [2023-12-26 19:37:52,475][105692] Updated weights for policy 0, policy_version 585649 (0.0009) [2023-12-26 19:37:52,500][105620] Updated weights for policy 1, policy_version 586595 (0.0008) [2023-12-26 19:37:52,535][105692] Updated weights for policy 0, policy_version 585659 (0.0008) [2023-12-26 19:37:52,591][105692] Updated weights for policy 0, policy_version 585669 (0.0009) [2023-12-26 19:37:52,647][105692] Updated weights for policy 0, policy_version 585679 (0.0009) [2023-12-26 19:37:53,277][105620] Updated weights for policy 1, policy_version 586605 (0.0008) [2023-12-26 19:37:53,332][105620] Updated weights for policy 1, policy_version 586615 (0.0010) [2023-12-26 19:37:53,381][105620] Updated weights for policy 1, policy_version 586625 (0.0010) [2023-12-26 19:37:53,409][105692] Updated weights for policy 0, policy_version 585689 (0.0010) [2023-12-26 19:37:53,472][105692] Updated weights for policy 0, policy_version 585699 (0.0011) [2023-12-26 19:37:53,528][105692] Updated weights for policy 0, policy_version 585709 (0.0011) [2023-12-26 19:37:54,097][105620] Updated weights for policy 1, policy_version 586635 (0.0009) [2023-12-26 19:37:54,150][105620] Updated weights for policy 1, policy_version 586645 (0.0005) [2023-12-26 19:37:54,214][105620] Updated weights for policy 1, policy_version 586655 (0.0008) [2023-12-26 19:37:54,221][105692] Updated weights for policy 0, policy_version 585719 (0.0007) [2023-12-26 19:37:54,285][105692] Updated weights for policy 0, policy_version 585729 (0.0007) [2023-12-26 19:37:54,343][105692] Updated weights for policy 0, policy_version 585739 (0.0009) [2023-12-26 19:37:54,941][105620] Updated weights for policy 1, policy_version 586665 (0.0007) [2023-12-26 19:37:55,003][105620] Updated weights for policy 1, policy_version 586675 (0.0010) [2023-12-26 19:37:55,046][105692] Updated weights for policy 0, policy_version 585749 (0.0005) [2023-12-26 19:37:55,065][105620] Updated weights for policy 1, policy_version 586685 (0.0008) [2023-12-26 19:37:55,105][105692] Updated weights for policy 0, policy_version 585759 (0.0005) [2023-12-26 19:37:55,117][105620] Updated weights for policy 1, policy_version 586695 (0.0009) [2023-12-26 19:37:55,165][105692] Updated weights for policy 0, policy_version 585769 (0.0006) [2023-12-26 19:37:55,829][105692] Updated weights for policy 0, policy_version 585779 (0.0008) [2023-12-26 19:37:55,876][105620] Updated weights for policy 1, policy_version 586705 (0.0008) [2023-12-26 19:37:55,882][105692] Updated weights for policy 0, policy_version 585789 (0.0006) [2023-12-26 19:37:55,923][105620] Updated weights for policy 1, policy_version 586715 (0.0007) [2023-12-26 19:37:55,927][105692] Updated weights for policy 0, policy_version 585799 (0.0007) [2023-12-26 19:37:55,987][105620] Updated weights for policy 1, policy_version 586725 (0.0007) [2023-12-26 19:37:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 300204032. Throughput: 0: 9820.1, 1: 9604.3. Samples: 300205616. Policy #0 lag: (min: 28.0, avg: 33.6, max: 56.0) [2023-12-26 19:37:56,062][104569] Avg episode reward: [(0, '1185.684'), (1, '8992.937')] [2023-12-26 19:37:56,573][105692] Updated weights for policy 0, policy_version 585809 (0.0007) [2023-12-26 19:37:56,629][105692] Updated weights for policy 0, policy_version 585819 (0.0007) [2023-12-26 19:37:56,672][105692] Updated weights for policy 0, policy_version 585829 (0.0005) [2023-12-26 19:37:56,717][105692] Updated weights for policy 0, policy_version 585839 (0.0005) [2023-12-26 19:37:56,791][105620] Updated weights for policy 1, policy_version 586735 (0.0008) [2023-12-26 19:37:56,861][105620] Updated weights for policy 1, policy_version 586745 (0.0010) [2023-12-26 19:37:56,932][105620] Updated weights for policy 1, policy_version 586755 (0.0009) [2023-12-26 19:37:57,261][105692] Updated weights for policy 0, policy_version 585849 (0.0005) [2023-12-26 19:37:57,312][105692] Updated weights for policy 0, policy_version 585859 (0.0005) [2023-12-26 19:37:57,370][105692] Updated weights for policy 0, policy_version 585869 (0.0007) [2023-12-26 19:37:57,709][105620] Updated weights for policy 1, policy_version 586765 (0.0008) [2023-12-26 19:37:57,759][105620] Updated weights for policy 1, policy_version 586775 (0.0009) [2023-12-26 19:37:57,813][105620] Updated weights for policy 1, policy_version 586785 (0.0010) [2023-12-26 19:37:57,945][105692] Updated weights for policy 0, policy_version 585879 (0.0007) [2023-12-26 19:37:58,001][105692] Updated weights for policy 0, policy_version 585889 (0.0005) [2023-12-26 19:37:58,063][105692] Updated weights for policy 0, policy_version 585899 (0.0005) [2023-12-26 19:37:58,694][105620] Updated weights for policy 1, policy_version 586796 (0.0009) [2023-12-26 19:37:58,718][105692] Updated weights for policy 0, policy_version 585909 (0.0008) [2023-12-26 19:37:58,759][105620] Updated weights for policy 1, policy_version 586806 (0.0008) [2023-12-26 19:37:58,781][105692] Updated weights for policy 0, policy_version 585919 (0.0007) [2023-12-26 19:37:58,820][105620] Updated weights for policy 1, policy_version 586816 (0.0007) [2023-12-26 19:37:58,846][105692] Updated weights for policy 0, policy_version 585929 (0.0007) [2023-12-26 19:37:59,502][105620] Updated weights for policy 1, policy_version 586826 (0.0007) [2023-12-26 19:37:59,569][105620] Updated weights for policy 1, policy_version 586836 (0.0009) [2023-12-26 19:37:59,619][105692] Updated weights for policy 0, policy_version 585939 (0.0008) [2023-12-26 19:37:59,627][105620] Updated weights for policy 1, policy_version 586846 (0.0008) [2023-12-26 19:37:59,672][105692] Updated weights for policy 0, policy_version 585949 (0.0007) [2023-12-26 19:37:59,690][105620] Updated weights for policy 1, policy_version 586856 (0.0008) [2023-12-26 19:37:59,727][105692] Updated weights for policy 0, policy_version 585959 (0.0007) [2023-12-26 19:38:00,387][105620] Updated weights for policy 1, policy_version 586866 (0.0010) [2023-12-26 19:38:00,447][105620] Updated weights for policy 1, policy_version 586876 (0.0009) [2023-12-26 19:38:00,486][105692] Updated weights for policy 0, policy_version 585969 (0.0009) [2023-12-26 19:38:00,508][105620] Updated weights for policy 1, policy_version 586886 (0.0007) [2023-12-26 19:38:00,548][105692] Updated weights for policy 0, policy_version 585979 (0.0008) [2023-12-26 19:38:00,611][105692] Updated weights for policy 0, policy_version 585989 (0.0009) [2023-12-26 19:38:00,669][105692] Updated weights for policy 0, policy_version 585999 (0.0009) [2023-12-26 19:38:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 300294144. Throughput: 0: 9933.7, 1: 9527.9. Samples: 300265456. Policy #0 lag: (min: 28.0, avg: 33.6, max: 56.0) [2023-12-26 19:38:01,062][104569] Avg episode reward: [(0, '5322.409'), (1, '8900.448')] [2023-12-26 19:38:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000586000_150036480.pth... [2023-12-26 19:38:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000586888_150257664.pth... [2023-12-26 19:38:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000584848_149741568.pth [2023-12-26 19:38:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000585768_149970944.pth [2023-12-26 19:38:01,288][105620] Updated weights for policy 1, policy_version 586896 (0.0008) [2023-12-26 19:38:01,339][105620] Updated weights for policy 1, policy_version 586906 (0.0008) [2023-12-26 19:38:01,358][105692] Updated weights for policy 0, policy_version 586009 (0.0007) [2023-12-26 19:38:01,405][105620] Updated weights for policy 1, policy_version 586916 (0.0008) [2023-12-26 19:38:01,420][105692] Updated weights for policy 0, policy_version 586019 (0.0007) [2023-12-26 19:38:01,476][105692] Updated weights for policy 0, policy_version 586029 (0.0005) [2023-12-26 19:38:02,194][105620] Updated weights for policy 1, policy_version 586926 (0.0008) [2023-12-26 19:38:02,250][105692] Updated weights for policy 0, policy_version 586039 (0.0007) [2023-12-26 19:38:02,256][105620] Updated weights for policy 1, policy_version 586936 (0.0008) [2023-12-26 19:38:02,315][105620] Updated weights for policy 1, policy_version 586946 (0.0009) [2023-12-26 19:38:02,317][105692] Updated weights for policy 0, policy_version 586049 (0.0006) [2023-12-26 19:38:02,378][105692] Updated weights for policy 0, policy_version 586059 (0.0009) [2023-12-26 19:38:03,058][105692] Updated weights for policy 0, policy_version 586069 (0.0008) [2023-12-26 19:38:03,097][105620] Updated weights for policy 1, policy_version 586956 (0.0007) [2023-12-26 19:38:03,119][105692] Updated weights for policy 0, policy_version 586079 (0.0008) [2023-12-26 19:38:03,154][105620] Updated weights for policy 1, policy_version 586966 (0.0007) [2023-12-26 19:38:03,176][105692] Updated weights for policy 0, policy_version 586089 (0.0006) [2023-12-26 19:38:03,209][105620] Updated weights for policy 1, policy_version 586976 (0.0008) [2023-12-26 19:38:03,735][105692] Updated weights for policy 0, policy_version 586099 (0.0006) [2023-12-26 19:38:03,782][105692] Updated weights for policy 0, policy_version 586109 (0.0005) [2023-12-26 19:38:03,835][105692] Updated weights for policy 0, policy_version 586119 (0.0005) [2023-12-26 19:38:04,065][105620] Updated weights for policy 1, policy_version 586986 (0.0008) [2023-12-26 19:38:04,126][105620] Updated weights for policy 1, policy_version 586996 (0.0009) [2023-12-26 19:38:04,193][105620] Updated weights for policy 1, policy_version 587006 (0.0008) [2023-12-26 19:38:04,259][105620] Updated weights for policy 1, policy_version 587016 (0.0008) [2023-12-26 19:38:04,483][105692] Updated weights for policy 0, policy_version 586129 (0.0008) [2023-12-26 19:38:04,541][105692] Updated weights for policy 0, policy_version 586139 (0.0010) [2023-12-26 19:38:04,601][105692] Updated weights for policy 0, policy_version 586149 (0.0011) [2023-12-26 19:38:04,661][105692] Updated weights for policy 0, policy_version 586159 (0.0011) [2023-12-26 19:38:05,080][105620] Updated weights for policy 1, policy_version 587026 (0.0009) [2023-12-26 19:38:05,139][105620] Updated weights for policy 1, policy_version 587036 (0.0008) [2023-12-26 19:38:05,190][105620] Updated weights for policy 1, policy_version 587046 (0.0008) [2023-12-26 19:38:05,249][105692] Updated weights for policy 0, policy_version 586169 (0.0010) [2023-12-26 19:38:05,294][105692] Updated weights for policy 0, policy_version 586179 (0.0010) [2023-12-26 19:38:05,347][105692] Updated weights for policy 0, policy_version 586189 (0.0010) [2023-12-26 19:38:05,985][105692] Updated weights for policy 0, policy_version 586199 (0.0009) [2023-12-26 19:38:06,011][105620] Updated weights for policy 1, policy_version 587056 (0.0007) [2023-12-26 19:38:06,035][105692] Updated weights for policy 0, policy_version 586209 (0.0005) [2023-12-26 19:38:06,059][105620] Updated weights for policy 1, policy_version 587066 (0.0009) [2023-12-26 19:38:06,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19251.2, 300 sec: 19633.0). Total num frames: 300384256. Throughput: 0: 9873.7, 1: 9440.2. Samples: 300379456. Policy #0 lag: (min: 28.0, avg: 33.6, max: 56.0) [2023-12-26 19:38:06,062][104569] Avg episode reward: [(0, '7351.076'), (1, '8900.470')] [2023-12-26 19:38:06,082][105692] Updated weights for policy 0, policy_version 586219 (0.0005) [2023-12-26 19:38:06,126][105620] Updated weights for policy 1, policy_version 587076 (0.0010) [2023-12-26 19:38:06,707][105692] Updated weights for policy 0, policy_version 586229 (0.0007) [2023-12-26 19:38:06,770][105692] Updated weights for policy 0, policy_version 586239 (0.0007) [2023-12-26 19:38:06,838][105692] Updated weights for policy 0, policy_version 586249 (0.0005) [2023-12-26 19:38:06,975][105620] Updated weights for policy 1, policy_version 587086 (0.0009) [2023-12-26 19:38:07,028][105620] Updated weights for policy 1, policy_version 587096 (0.0010) [2023-12-26 19:38:07,077][105620] Updated weights for policy 1, policy_version 587106 (0.0010) [2023-12-26 19:38:07,459][105692] Updated weights for policy 0, policy_version 586259 (0.0005) [2023-12-26 19:38:07,505][105692] Updated weights for policy 0, policy_version 586269 (0.0005) [2023-12-26 19:38:07,558][105692] Updated weights for policy 0, policy_version 586279 (0.0005) [2023-12-26 19:38:07,759][105620] Updated weights for policy 1, policy_version 587116 (0.0008) [2023-12-26 19:38:07,816][105620] Updated weights for policy 1, policy_version 587126 (0.0005) [2023-12-26 19:38:07,870][105620] Updated weights for policy 1, policy_version 587136 (0.0005) [2023-12-26 19:38:08,179][105692] Updated weights for policy 0, policy_version 586289 (0.0005) [2023-12-26 19:38:08,227][105692] Updated weights for policy 0, policy_version 586299 (0.0005) [2023-12-26 19:38:08,278][105692] Updated weights for policy 0, policy_version 586309 (0.0007) [2023-12-26 19:38:08,348][105692] Updated weights for policy 0, policy_version 586319 (0.0011) [2023-12-26 19:38:08,629][105620] Updated weights for policy 1, policy_version 587146 (0.0006) [2023-12-26 19:38:08,691][105620] Updated weights for policy 1, policy_version 587156 (0.0009) [2023-12-26 19:38:08,751][105620] Updated weights for policy 1, policy_version 587166 (0.0009) [2023-12-26 19:38:08,815][105620] Updated weights for policy 1, policy_version 587176 (0.0009) [2023-12-26 19:38:08,944][105692] Updated weights for policy 0, policy_version 586329 (0.0007) [2023-12-26 19:38:09,009][105692] Updated weights for policy 0, policy_version 586339 (0.0007) [2023-12-26 19:38:09,069][105692] Updated weights for policy 0, policy_version 586349 (0.0007) [2023-12-26 19:38:09,639][105620] Updated weights for policy 1, policy_version 587186 (0.0007) [2023-12-26 19:38:09,707][105620] Updated weights for policy 1, policy_version 587196 (0.0008) [2023-12-26 19:38:09,761][105692] Updated weights for policy 0, policy_version 586359 (0.0008) [2023-12-26 19:38:09,763][105620] Updated weights for policy 1, policy_version 587206 (0.0007) [2023-12-26 19:38:09,823][105692] Updated weights for policy 0, policy_version 586369 (0.0008) [2023-12-26 19:38:09,884][105692] Updated weights for policy 0, policy_version 586379 (0.0009) [2023-12-26 19:38:10,539][105620] Updated weights for policy 1, policy_version 587216 (0.0008) [2023-12-26 19:38:10,597][105620] Updated weights for policy 1, policy_version 587226 (0.0009) [2023-12-26 19:38:10,658][105620] Updated weights for policy 1, policy_version 587236 (0.0009) [2023-12-26 19:38:10,673][105692] Updated weights for policy 0, policy_version 586389 (0.0008) [2023-12-26 19:38:10,742][105692] Updated weights for policy 0, policy_version 586399 (0.0009) [2023-12-26 19:38:10,811][105692] Updated weights for policy 0, policy_version 586409 (0.0009) [2023-12-26 19:38:11,066][104569] Fps is (10 sec: 19653.5, 60 sec: 19386.5, 300 sec: 19660.6). Total num frames: 300490752. Throughput: 0: 10059.2, 1: 9370.6. Samples: 300497500. Policy #0 lag: (min: 28.0, avg: 33.6, max: 56.0) [2023-12-26 19:38:11,066][104569] Avg episode reward: [(0, '8812.969'), (1, '9170.111')] [2023-12-26 19:38:11,422][105620] Updated weights for policy 1, policy_version 587246 (0.0008) [2023-12-26 19:38:11,487][105620] Updated weights for policy 1, policy_version 587256 (0.0008) [2023-12-26 19:38:11,545][105620] Updated weights for policy 1, policy_version 587266 (0.0009) [2023-12-26 19:38:11,637][105692] Updated weights for policy 0, policy_version 586419 (0.0009) [2023-12-26 19:38:11,702][105692] Updated weights for policy 0, policy_version 586429 (0.0009) [2023-12-26 19:38:11,774][105692] Updated weights for policy 0, policy_version 586439 (0.0009) [2023-12-26 19:38:12,357][105620] Updated weights for policy 1, policy_version 587276 (0.0010) [2023-12-26 19:38:12,422][105620] Updated weights for policy 1, policy_version 587286 (0.0010) [2023-12-26 19:38:12,473][105620] Updated weights for policy 1, policy_version 587296 (0.0009) [2023-12-26 19:38:12,553][105692] Updated weights for policy 0, policy_version 586449 (0.0009) [2023-12-26 19:38:12,605][105692] Updated weights for policy 0, policy_version 586459 (0.0009) [2023-12-26 19:38:12,667][105692] Updated weights for policy 0, policy_version 586469 (0.0010) [2023-12-26 19:38:12,726][105692] Updated weights for policy 0, policy_version 586479 (0.0009) [2023-12-26 19:38:13,225][105620] Updated weights for policy 1, policy_version 587306 (0.0009) [2023-12-26 19:38:13,282][105620] Updated weights for policy 1, policy_version 587316 (0.0009) [2023-12-26 19:38:13,348][105620] Updated weights for policy 1, policy_version 587326 (0.0010) [2023-12-26 19:38:13,414][105620] Updated weights for policy 1, policy_version 587336 (0.0010) [2023-12-26 19:38:13,501][105692] Updated weights for policy 0, policy_version 586489 (0.0009) [2023-12-26 19:38:13,546][105692] Updated weights for policy 0, policy_version 586499 (0.0010) [2023-12-26 19:38:13,605][105692] Updated weights for policy 0, policy_version 586509 (0.0010) [2023-12-26 19:38:14,048][105620] Updated weights for policy 1, policy_version 587346 (0.0011) [2023-12-26 19:38:14,107][105620] Updated weights for policy 1, policy_version 587356 (0.0010) [2023-12-26 19:38:14,161][105620] Updated weights for policy 1, policy_version 587366 (0.0010) [2023-12-26 19:38:14,365][105692] Updated weights for policy 0, policy_version 586519 (0.0010) [2023-12-26 19:38:14,419][105692] Updated weights for policy 0, policy_version 586529 (0.0010) [2023-12-26 19:38:14,477][105692] Updated weights for policy 0, policy_version 586539 (0.0010) [2023-12-26 19:38:14,889][105620] Updated weights for policy 1, policy_version 587376 (0.0011) [2023-12-26 19:38:14,949][105620] Updated weights for policy 1, policy_version 587386 (0.0010) [2023-12-26 19:38:15,012][105620] Updated weights for policy 1, policy_version 587396 (0.0010) [2023-12-26 19:38:15,184][105692] Updated weights for policy 0, policy_version 586549 (0.0009) [2023-12-26 19:38:15,250][105692] Updated weights for policy 0, policy_version 586559 (0.0011) [2023-12-26 19:38:15,320][105692] Updated weights for policy 0, policy_version 586569 (0.0011) [2023-12-26 19:38:15,731][105620] Updated weights for policy 1, policy_version 587406 (0.0010) [2023-12-26 19:38:15,798][105620] Updated weights for policy 1, policy_version 587416 (0.0008) [2023-12-26 19:38:15,870][105620] Updated weights for policy 1, policy_version 587426 (0.0006) [2023-12-26 19:38:15,902][105692] Updated weights for policy 0, policy_version 586579 (0.0008) [2023-12-26 19:38:15,960][105692] Updated weights for policy 0, policy_version 586589 (0.0005) [2023-12-26 19:38:16,023][105692] Updated weights for policy 0, policy_version 586599 (0.0005) [2023-12-26 19:38:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19633.0). Total num frames: 300580864. Throughput: 0: 9982.9, 1: 9366.9. Samples: 300551816. Policy #0 lag: (min: 28.0, avg: 33.6, max: 56.0) [2023-12-26 19:38:16,063][104569] Avg episode reward: [(0, '8990.598'), (1, '9079.619')] [2023-12-26 19:38:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000587432_150396928.pth... [2023-12-26 19:38:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000586344_150118400.pth [2023-12-26 19:38:16,079][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000586608_150192128.pth... [2023-12-26 19:38:16,082][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000585424_149889024.pth [2023-12-26 19:38:16,417][105620] Updated weights for policy 1, policy_version 587436 (0.0008) [2023-12-26 19:38:16,469][105620] Updated weights for policy 1, policy_version 587446 (0.0005) [2023-12-26 19:38:16,521][105620] Updated weights for policy 1, policy_version 587456 (0.0009) [2023-12-26 19:38:16,526][105692] Updated weights for policy 0, policy_version 586609 (0.0007) [2023-12-26 19:38:16,578][105692] Updated weights for policy 0, policy_version 586619 (0.0007) [2023-12-26 19:38:16,640][105692] Updated weights for policy 0, policy_version 586629 (0.0011) [2023-12-26 19:38:16,704][105692] Updated weights for policy 0, policy_version 586639 (0.0010) [2023-12-26 19:38:17,171][105620] Updated weights for policy 1, policy_version 587466 (0.0008) [2023-12-26 19:38:17,230][105620] Updated weights for policy 1, policy_version 587476 (0.0006) [2023-12-26 19:38:17,284][105620] Updated weights for policy 1, policy_version 587486 (0.0008) [2023-12-26 19:38:17,329][105620] Updated weights for policy 1, policy_version 587496 (0.0008) [2023-12-26 19:38:17,447][105692] Updated weights for policy 0, policy_version 586649 (0.0010) [2023-12-26 19:38:17,498][105692] Updated weights for policy 0, policy_version 586659 (0.0010) [2023-12-26 19:38:17,544][105692] Updated weights for policy 0, policy_version 586669 (0.0010) [2023-12-26 19:38:18,021][105620] Updated weights for policy 1, policy_version 587506 (0.0009) [2023-12-26 19:38:18,078][105620] Updated weights for policy 1, policy_version 587516 (0.0009) [2023-12-26 19:38:18,139][105620] Updated weights for policy 1, policy_version 587526 (0.0009) [2023-12-26 19:38:18,276][105692] Updated weights for policy 0, policy_version 586679 (0.0007) [2023-12-26 19:38:18,324][105692] Updated weights for policy 0, policy_version 586689 (0.0006) [2023-12-26 19:38:18,381][105692] Updated weights for policy 0, policy_version 586699 (0.0007) [2023-12-26 19:38:18,948][105620] Updated weights for policy 1, policy_version 587536 (0.0008) [2023-12-26 19:38:19,003][105620] Updated weights for policy 1, policy_version 587546 (0.0007) [2023-12-26 19:38:19,005][105692] Updated weights for policy 0, policy_version 586709 (0.0007) [2023-12-26 19:38:19,053][105620] Updated weights for policy 1, policy_version 587556 (0.0007) [2023-12-26 19:38:19,056][105692] Updated weights for policy 0, policy_version 586719 (0.0006) [2023-12-26 19:38:19,115][105692] Updated weights for policy 0, policy_version 586729 (0.0007) [2023-12-26 19:38:19,786][105620] Updated weights for policy 1, policy_version 587566 (0.0009) [2023-12-26 19:38:19,849][105620] Updated weights for policy 1, policy_version 587576 (0.0009) [2023-12-26 19:38:19,909][105692] Updated weights for policy 0, policy_version 586739 (0.0008) [2023-12-26 19:38:19,913][105620] Updated weights for policy 1, policy_version 587586 (0.0009) [2023-12-26 19:38:19,973][105692] Updated weights for policy 0, policy_version 586749 (0.0007) [2023-12-26 19:38:20,033][105692] Updated weights for policy 0, policy_version 586759 (0.0009) [2023-12-26 19:38:20,693][105620] Updated weights for policy 1, policy_version 587596 (0.0010) [2023-12-26 19:38:20,752][105620] Updated weights for policy 1, policy_version 587606 (0.0009) [2023-12-26 19:38:20,755][105692] Updated weights for policy 0, policy_version 586769 (0.0009) [2023-12-26 19:38:20,810][105692] Updated weights for policy 0, policy_version 586779 (0.0007) [2023-12-26 19:38:20,818][105620] Updated weights for policy 1, policy_version 587616 (0.0008) [2023-12-26 19:38:20,869][105692] Updated weights for policy 0, policy_version 586789 (0.0007) [2023-12-26 19:38:20,929][105692] Updated weights for policy 0, policy_version 586799 (0.0009) [2023-12-26 19:38:21,062][104569] Fps is (10 sec: 19668.2, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 300687360. Throughput: 0: 10036.6, 1: 9399.1. Samples: 300672460. Policy #0 lag: (min: 28.0, avg: 33.6, max: 56.0) [2023-12-26 19:38:21,062][104569] Avg episode reward: [(0, '8987.778'), (1, '8901.148')] [2023-12-26 19:38:21,637][105620] Updated weights for policy 1, policy_version 587626 (0.0007) [2023-12-26 19:38:21,711][105620] Updated weights for policy 1, policy_version 587636 (0.0009) [2023-12-26 19:38:21,743][105692] Updated weights for policy 0, policy_version 586809 (0.0008) [2023-12-26 19:38:21,777][105620] Updated weights for policy 1, policy_version 587646 (0.0008) [2023-12-26 19:38:21,803][105692] Updated weights for policy 0, policy_version 586819 (0.0007) [2023-12-26 19:38:21,840][105620] Updated weights for policy 1, policy_version 587656 (0.0008) [2023-12-26 19:38:21,863][105692] Updated weights for policy 0, policy_version 586829 (0.0008) [2023-12-26 19:38:22,603][105692] Updated weights for policy 0, policy_version 586839 (0.0007) [2023-12-26 19:38:22,618][105620] Updated weights for policy 1, policy_version 587666 (0.0009) [2023-12-26 19:38:22,653][105692] Updated weights for policy 0, policy_version 586849 (0.0006) [2023-12-26 19:38:22,680][105620] Updated weights for policy 1, policy_version 587676 (0.0008) [2023-12-26 19:38:22,711][105692] Updated weights for policy 0, policy_version 586859 (0.0007) [2023-12-26 19:38:22,733][105620] Updated weights for policy 1, policy_version 587686 (0.0007) [2023-12-26 19:38:23,430][105692] Updated weights for policy 0, policy_version 586869 (0.0006) [2023-12-26 19:38:23,496][105692] Updated weights for policy 0, policy_version 586879 (0.0006) [2023-12-26 19:38:23,533][105620] Updated weights for policy 1, policy_version 587696 (0.0007) [2023-12-26 19:38:23,551][105692] Updated weights for policy 0, policy_version 586889 (0.0006) [2023-12-26 19:38:23,589][105620] Updated weights for policy 1, policy_version 587706 (0.0007) [2023-12-26 19:38:23,639][105620] Updated weights for policy 1, policy_version 587716 (0.0009) [2023-12-26 19:38:24,241][105692] Updated weights for policy 0, policy_version 586899 (0.0007) [2023-12-26 19:38:24,296][105692] Updated weights for policy 0, policy_version 586909 (0.0009) [2023-12-26 19:38:24,344][105692] Updated weights for policy 0, policy_version 586919 (0.0008) [2023-12-26 19:38:24,411][105620] Updated weights for policy 1, policy_version 587726 (0.0009) [2023-12-26 19:38:24,464][105620] Updated weights for policy 1, policy_version 587736 (0.0008) [2023-12-26 19:38:24,515][105620] Updated weights for policy 1, policy_version 587746 (0.0008) [2023-12-26 19:38:25,085][105692] Updated weights for policy 0, policy_version 586929 (0.0006) [2023-12-26 19:38:25,146][105692] Updated weights for policy 0, policy_version 586939 (0.0009) [2023-12-26 19:38:25,201][105692] Updated weights for policy 0, policy_version 586949 (0.0009) [2023-12-26 19:38:25,247][105692] Updated weights for policy 0, policy_version 586959 (0.0008) [2023-12-26 19:38:25,288][105620] Updated weights for policy 1, policy_version 587756 (0.0008) [2023-12-26 19:38:25,345][105620] Updated weights for policy 1, policy_version 587766 (0.0008) [2023-12-26 19:38:25,401][105620] Updated weights for policy 1, policy_version 587776 (0.0008) [2023-12-26 19:38:25,926][105692] Updated weights for policy 0, policy_version 586969 (0.0008) [2023-12-26 19:38:25,988][105692] Updated weights for policy 0, policy_version 586979 (0.0008) [2023-12-26 19:38:26,052][105692] Updated weights for policy 0, policy_version 586989 (0.0009) [2023-12-26 19:38:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 300769280. Throughput: 0: 9916.4, 1: 9392.1. Samples: 300783060. Policy #0 lag: (min: 28.0, avg: 33.6, max: 56.0) [2023-12-26 19:38:26,063][104569] Avg episode reward: [(0, '8622.631'), (1, '8901.583')] [2023-12-26 19:38:26,182][105620] Updated weights for policy 1, policy_version 587786 (0.0010) [2023-12-26 19:38:26,255][105620] Updated weights for policy 1, policy_version 587796 (0.0009) [2023-12-26 19:38:26,318][105620] Updated weights for policy 1, policy_version 587806 (0.0009) [2023-12-26 19:38:26,379][105620] Updated weights for policy 1, policy_version 587816 (0.0009) [2023-12-26 19:38:26,742][105692] Updated weights for policy 0, policy_version 586999 (0.0006) [2023-12-26 19:38:26,787][105692] Updated weights for policy 0, policy_version 587009 (0.0005) [2023-12-26 19:38:26,837][105692] Updated weights for policy 0, policy_version 587019 (0.0005) [2023-12-26 19:38:27,217][105620] Updated weights for policy 1, policy_version 587826 (0.0005) [2023-12-26 19:38:27,267][105620] Updated weights for policy 1, policy_version 587836 (0.0008) [2023-12-26 19:38:27,322][105620] Updated weights for policy 1, policy_version 587846 (0.0009) [2023-12-26 19:38:27,393][105692] Updated weights for policy 0, policy_version 587029 (0.0007) [2023-12-26 19:38:27,438][105692] Updated weights for policy 0, policy_version 587039 (0.0008) [2023-12-26 19:38:27,446][105585] KL-divergence is very high: 121.4546 [2023-12-26 19:38:27,488][105692] Updated weights for policy 0, policy_version 587049 (0.0009) [2023-12-26 19:38:27,489][105585] KL-divergence is very high: 102.1468 [2023-12-26 19:38:27,962][105620] Updated weights for policy 1, policy_version 587856 (0.0007) [2023-12-26 19:38:28,013][105620] Updated weights for policy 1, policy_version 587866 (0.0009) [2023-12-26 19:38:28,070][105620] Updated weights for policy 1, policy_version 587877 (0.0010) [2023-12-26 19:38:28,232][105692] Updated weights for policy 0, policy_version 587059 (0.0008) [2023-12-26 19:38:28,285][105692] Updated weights for policy 0, policy_version 587069 (0.0005) [2023-12-26 19:38:28,344][105692] Updated weights for policy 0, policy_version 587079 (0.0005) [2023-12-26 19:38:28,869][105620] Updated weights for policy 1, policy_version 587887 (0.0009) [2023-12-26 19:38:28,928][105620] Updated weights for policy 1, policy_version 587897 (0.0008) [2023-12-26 19:38:28,986][105620] Updated weights for policy 1, policy_version 587907 (0.0010) [2023-12-26 19:38:29,050][105692] Updated weights for policy 0, policy_version 587089 (0.0008) [2023-12-26 19:38:29,104][105692] Updated weights for policy 0, policy_version 587100 (0.0010) [2023-12-26 19:38:29,162][105692] Updated weights for policy 0, policy_version 587111 (0.0010) [2023-12-26 19:38:29,724][105620] Updated weights for policy 1, policy_version 587917 (0.0009) [2023-12-26 19:38:29,780][105620] Updated weights for policy 1, policy_version 587927 (0.0009) [2023-12-26 19:38:29,846][105620] Updated weights for policy 1, policy_version 587937 (0.0008) [2023-12-26 19:38:29,854][105692] Updated weights for policy 0, policy_version 587121 (0.0010) [2023-12-26 19:38:29,914][105692] Updated weights for policy 0, policy_version 587131 (0.0007) [2023-12-26 19:38:29,978][105692] Updated weights for policy 0, policy_version 587141 (0.0008) [2023-12-26 19:38:30,033][105692] Updated weights for policy 0, policy_version 587151 (0.0008) [2023-12-26 19:38:30,524][105620] Updated weights for policy 1, policy_version 587947 (0.0010) [2023-12-26 19:38:30,573][105620] Updated weights for policy 1, policy_version 587957 (0.0010) [2023-12-26 19:38:30,621][105620] Updated weights for policy 1, policy_version 587967 (0.0010) [2023-12-26 19:38:30,691][105692] Updated weights for policy 0, policy_version 587161 (0.0010) [2023-12-26 19:38:30,740][105692] Updated weights for policy 0, policy_version 587171 (0.0010) [2023-12-26 19:38:30,791][105692] Updated weights for policy 0, policy_version 587181 (0.0010) [2023-12-26 19:38:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 300875776. Throughput: 0: 10027.1, 1: 9361.2. Samples: 300842648. Policy #0 lag: (min: 28.0, avg: 33.6, max: 56.0) [2023-12-26 19:38:31,062][104569] Avg episode reward: [(0, '8625.959'), (1, '9172.816')] [2023-12-26 19:38:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000587976_150536192.pth... [2023-12-26 19:38:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000587184_150339584.pth... [2023-12-26 19:38:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000586888_150257664.pth [2023-12-26 19:38:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000586000_150036480.pth [2023-12-26 19:38:31,384][105620] Updated weights for policy 1, policy_version 587977 (0.0010) [2023-12-26 19:38:31,453][105620] Updated weights for policy 1, policy_version 587987 (0.0011) [2023-12-26 19:38:31,519][105620] Updated weights for policy 1, policy_version 587997 (0.0011) [2023-12-26 19:38:31,562][105692] Updated weights for policy 0, policy_version 587191 (0.0009) [2023-12-26 19:38:31,567][105620] Updated weights for policy 1, policy_version 588007 (0.0010) [2023-12-26 19:38:31,625][105692] Updated weights for policy 0, policy_version 587201 (0.0008) [2023-12-26 19:38:31,680][105692] Updated weights for policy 0, policy_version 587211 (0.0007) [2023-12-26 19:38:32,307][105620] Updated weights for policy 1, policy_version 588017 (0.0011) [2023-12-26 19:38:32,339][105692] Updated weights for policy 0, policy_version 587221 (0.0007) [2023-12-26 19:38:32,378][105620] Updated weights for policy 1, policy_version 588027 (0.0010) [2023-12-26 19:38:32,400][105692] Updated weights for policy 0, policy_version 587231 (0.0008) [2023-12-26 19:38:32,438][105620] Updated weights for policy 1, policy_version 588037 (0.0010) [2023-12-26 19:38:32,461][105692] Updated weights for policy 0, policy_version 587241 (0.0007) [2023-12-26 19:38:33,146][105620] Updated weights for policy 1, policy_version 588047 (0.0008) [2023-12-26 19:38:33,213][105620] Updated weights for policy 1, policy_version 588057 (0.0009) [2023-12-26 19:38:33,249][105692] Updated weights for policy 0, policy_version 587251 (0.0009) [2023-12-26 19:38:33,264][105620] Updated weights for policy 1, policy_version 588067 (0.0007) [2023-12-26 19:38:33,293][105585] KL-divergence is very high: 112.9441 [2023-12-26 19:38:33,299][105692] Updated weights for policy 0, policy_version 587261 (0.0008) [2023-12-26 19:38:33,329][105585] KL-divergence is very high: 110.7030 [2023-12-26 19:38:33,345][105692] Updated weights for policy 0, policy_version 587271 (0.0008) [2023-12-26 19:38:34,041][105620] Updated weights for policy 1, policy_version 588077 (0.0008) [2023-12-26 19:38:34,059][105692] Updated weights for policy 0, policy_version 587281 (0.0009) [2023-12-26 19:38:34,092][105620] Updated weights for policy 1, policy_version 588088 (0.0006) [2023-12-26 19:38:34,118][105692] Updated weights for policy 0, policy_version 587291 (0.0009) [2023-12-26 19:38:34,160][105620] Updated weights for policy 1, policy_version 588098 (0.0009) [2023-12-26 19:38:34,177][105692] Updated weights for policy 0, policy_version 587301 (0.0010) [2023-12-26 19:38:34,237][105692] Updated weights for policy 0, policy_version 587311 (0.0010) [2023-12-26 19:38:34,931][105692] Updated weights for policy 0, policy_version 587321 (0.0006) [2023-12-26 19:38:34,989][105692] Updated weights for policy 0, policy_version 587331 (0.0006) [2023-12-26 19:38:34,990][105620] Updated weights for policy 1, policy_version 588108 (0.0007) [2023-12-26 19:38:35,045][105620] Updated weights for policy 1, policy_version 588118 (0.0008) [2023-12-26 19:38:35,049][105692] Updated weights for policy 0, policy_version 587341 (0.0006) [2023-12-26 19:38:35,102][105620] Updated weights for policy 1, policy_version 588128 (0.0009) [2023-12-26 19:38:35,612][105692] Updated weights for policy 0, policy_version 587351 (0.0005) [2023-12-26 19:38:35,674][105692] Updated weights for policy 0, policy_version 587361 (0.0005) [2023-12-26 19:38:35,734][105692] Updated weights for policy 0, policy_version 587371 (0.0005) [2023-12-26 19:38:35,992][105620] Updated weights for policy 1, policy_version 588138 (0.0012) [2023-12-26 19:38:36,045][105620] Updated weights for policy 1, policy_version 588148 (0.0010) [2023-12-26 19:38:36,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.3, 300 sec: 19577.5). Total num frames: 300965888. Throughput: 0: 9967.0, 1: 9249.9. Samples: 300957552. Policy #0 lag: (min: 28.0, avg: 33.6, max: 56.0) [2023-12-26 19:38:36,062][104569] Avg episode reward: [(0, '8994.511'), (1, '9083.062')] [2023-12-26 19:38:36,098][105620] Updated weights for policy 1, policy_version 588158 (0.0009) [2023-12-26 19:38:36,161][105620] Updated weights for policy 1, policy_version 588168 (0.0010) [2023-12-26 19:38:36,291][105692] Updated weights for policy 0, policy_version 587381 (0.0007) [2023-12-26 19:38:36,346][105692] Updated weights for policy 0, policy_version 587391 (0.0011) [2023-12-26 19:38:36,407][105692] Updated weights for policy 0, policy_version 587401 (0.0011) [2023-12-26 19:38:36,823][105620] Updated weights for policy 1, policy_version 588178 (0.0005) [2023-12-26 19:38:36,883][105620] Updated weights for policy 1, policy_version 588188 (0.0009) [2023-12-26 19:38:36,947][105620] Updated weights for policy 1, policy_version 588198 (0.0008) [2023-12-26 19:38:37,173][105692] Updated weights for policy 0, policy_version 587411 (0.0011) [2023-12-26 19:38:37,226][105692] Updated weights for policy 0, policy_version 587421 (0.0011) [2023-12-26 19:38:37,279][105692] Updated weights for policy 0, policy_version 587431 (0.0011) [2023-12-26 19:38:37,518][105620] Updated weights for policy 1, policy_version 588208 (0.0007) [2023-12-26 19:38:37,580][105620] Updated weights for policy 1, policy_version 588218 (0.0005) [2023-12-26 19:38:37,636][105620] Updated weights for policy 1, policy_version 588228 (0.0005) [2023-12-26 19:38:38,009][105692] Updated weights for policy 0, policy_version 587441 (0.0011) [2023-12-26 19:38:38,064][105692] Updated weights for policy 0, policy_version 587451 (0.0010) [2023-12-26 19:38:38,116][105692] Updated weights for policy 0, policy_version 587461 (0.0010) [2023-12-26 19:38:38,179][105692] Updated weights for policy 0, policy_version 587471 (0.0011) [2023-12-26 19:38:38,297][105620] Updated weights for policy 1, policy_version 588238 (0.0008) [2023-12-26 19:38:38,360][105620] Updated weights for policy 1, policy_version 588248 (0.0011) [2023-12-26 19:38:38,422][105620] Updated weights for policy 1, policy_version 588258 (0.0009) [2023-12-26 19:38:38,965][105692] Updated weights for policy 0, policy_version 587481 (0.0008) [2023-12-26 19:38:39,024][105692] Updated weights for policy 0, policy_version 587491 (0.0006) [2023-12-26 19:38:39,088][105692] Updated weights for policy 0, policy_version 587501 (0.0005) [2023-12-26 19:38:39,129][105620] Updated weights for policy 1, policy_version 588268 (0.0009) [2023-12-26 19:38:39,178][105620] Updated weights for policy 1, policy_version 588278 (0.0010) [2023-12-26 19:38:39,236][105620] Updated weights for policy 1, policy_version 588288 (0.0010) [2023-12-26 19:38:39,841][105692] Updated weights for policy 0, policy_version 587511 (0.0009) [2023-12-26 19:38:39,894][105692] Updated weights for policy 0, policy_version 587521 (0.0010) [2023-12-26 19:38:39,957][105692] Updated weights for policy 0, policy_version 587531 (0.0009) [2023-12-26 19:38:39,958][105620] Updated weights for policy 1, policy_version 588298 (0.0007) [2023-12-26 19:38:40,015][105620] Updated weights for policy 1, policy_version 588308 (0.0007) [2023-12-26 19:38:40,066][105620] Updated weights for policy 1, policy_version 588318 (0.0009) [2023-12-26 19:38:40,121][105620] Updated weights for policy 1, policy_version 588328 (0.0009) [2023-12-26 19:38:40,784][105692] Updated weights for policy 0, policy_version 587541 (0.0007) [2023-12-26 19:38:40,830][105620] Updated weights for policy 1, policy_version 588338 (0.0008) [2023-12-26 19:38:40,835][105692] Updated weights for policy 0, policy_version 587551 (0.0006) [2023-12-26 19:38:40,886][105692] Updated weights for policy 0, policy_version 587561 (0.0007) [2023-12-26 19:38:40,888][105620] Updated weights for policy 1, policy_version 588348 (0.0007) [2023-12-26 19:38:40,943][105620] Updated weights for policy 1, policy_version 588358 (0.0008) [2023-12-26 19:38:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 301072384. Throughput: 0: 10027.2, 1: 9309.8. Samples: 301075780. Policy #0 lag: (min: 28.0, avg: 33.6, max: 56.0) [2023-12-26 19:38:41,062][104569] Avg episode reward: [(0, '8996.645'), (1, '8996.470')] [2023-12-26 19:38:41,647][105620] Updated weights for policy 1, policy_version 588368 (0.0008) [2023-12-26 19:38:41,706][105620] Updated weights for policy 1, policy_version 588378 (0.0008) [2023-12-26 19:38:41,722][105692] Updated weights for policy 0, policy_version 587571 (0.0006) [2023-12-26 19:38:41,772][105620] Updated weights for policy 1, policy_version 588388 (0.0007) [2023-12-26 19:38:41,787][105692] Updated weights for policy 0, policy_version 587581 (0.0008) [2023-12-26 19:38:41,845][105692] Updated weights for policy 0, policy_version 587591 (0.0008) [2023-12-26 19:38:42,454][105620] Updated weights for policy 1, policy_version 588398 (0.0009) [2023-12-26 19:38:42,521][105620] Updated weights for policy 1, policy_version 588408 (0.0011) [2023-12-26 19:38:42,577][105620] Updated weights for policy 1, policy_version 588418 (0.0009) [2023-12-26 19:38:42,686][105692] Updated weights for policy 0, policy_version 587601 (0.0010) [2023-12-26 19:38:42,740][105692] Updated weights for policy 0, policy_version 587611 (0.0010) [2023-12-26 19:38:42,791][105692] Updated weights for policy 0, policy_version 587621 (0.0009) [2023-12-26 19:38:42,842][105692] Updated weights for policy 0, policy_version 587631 (0.0009) [2023-12-26 19:38:43,180][105620] Updated weights for policy 1, policy_version 588428 (0.0007) [2023-12-26 19:38:43,225][105620] Updated weights for policy 1, policy_version 588438 (0.0009) [2023-12-26 19:38:43,275][105620] Updated weights for policy 1, policy_version 588448 (0.0006) [2023-12-26 19:38:43,628][105692] Updated weights for policy 0, policy_version 587641 (0.0006) [2023-12-26 19:38:43,678][105692] Updated weights for policy 0, policy_version 587651 (0.0008) [2023-12-26 19:38:43,736][105692] Updated weights for policy 0, policy_version 587662 (0.0010) [2023-12-26 19:38:43,932][105620] Updated weights for policy 1, policy_version 588458 (0.0009) [2023-12-26 19:38:43,976][105620] Updated weights for policy 1, policy_version 588468 (0.0005) [2023-12-26 19:38:44,024][105620] Updated weights for policy 1, policy_version 588478 (0.0005) [2023-12-26 19:38:44,082][105620] Updated weights for policy 1, policy_version 588488 (0.0005) [2023-12-26 19:38:44,601][105692] Updated weights for policy 0, policy_version 587672 (0.0009) [2023-12-26 19:38:44,647][105692] Updated weights for policy 0, policy_version 587682 (0.0006) [2023-12-26 19:38:44,693][105692] Updated weights for policy 0, policy_version 587692 (0.0005) [2023-12-26 19:38:44,704][105620] Updated weights for policy 1, policy_version 588498 (0.0010) [2023-12-26 19:38:44,761][105620] Updated weights for policy 1, policy_version 588508 (0.0010) [2023-12-26 19:38:44,823][105620] Updated weights for policy 1, policy_version 588518 (0.0009) [2023-12-26 19:38:45,477][105692] Updated weights for policy 0, policy_version 587702 (0.0008) [2023-12-26 19:38:45,537][105692] Updated weights for policy 0, policy_version 587712 (0.0008) [2023-12-26 19:38:45,553][105620] Updated weights for policy 1, policy_version 588528 (0.0008) [2023-12-26 19:38:45,589][105692] Updated weights for policy 0, policy_version 587722 (0.0007) [2023-12-26 19:38:45,608][105620] Updated weights for policy 1, policy_version 588538 (0.0006) [2023-12-26 19:38:45,662][105620] Updated weights for policy 1, policy_version 588548 (0.0008) [2023-12-26 19:38:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19251.1, 300 sec: 19549.7). Total num frames: 301162496. Throughput: 0: 9851.6, 1: 9431.1. Samples: 301133180. Policy #0 lag: (min: 28.0, avg: 33.6, max: 56.0) [2023-12-26 19:38:46,063][104569] Avg episode reward: [(0, '7648.387'), (1, '8729.377')] [2023-12-26 19:38:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000587728_150478848.pth... [2023-12-26 19:38:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000588552_150683648.pth... [2023-12-26 19:38:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000586608_150192128.pth [2023-12-26 19:38:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000587432_150396928.pth [2023-12-26 19:38:46,304][105620] Updated weights for policy 1, policy_version 588558 (0.0007) [2023-12-26 19:38:46,360][105620] Updated weights for policy 1, policy_version 588568 (0.0008) [2023-12-26 19:38:46,413][105620] Updated weights for policy 1, policy_version 588578 (0.0008) [2023-12-26 19:38:46,421][105692] Updated weights for policy 0, policy_version 587732 (0.0008) [2023-12-26 19:38:46,472][105692] Updated weights for policy 0, policy_version 587742 (0.0009) [2023-12-26 19:38:46,528][105692] Updated weights for policy 0, policy_version 587752 (0.0012) [2023-12-26 19:38:47,003][105620] Updated weights for policy 1, policy_version 588588 (0.0007) [2023-12-26 19:38:47,056][105620] Updated weights for policy 1, policy_version 588598 (0.0008) [2023-12-26 19:38:47,111][105620] Updated weights for policy 1, policy_version 588608 (0.0009) [2023-12-26 19:38:47,361][105692] Updated weights for policy 0, policy_version 587763 (0.0009) [2023-12-26 19:38:47,413][105692] Updated weights for policy 0, policy_version 587773 (0.0008) [2023-12-26 19:38:47,457][105692] Updated weights for policy 0, policy_version 587783 (0.0008) [2023-12-26 19:38:47,889][105620] Updated weights for policy 1, policy_version 588618 (0.0009) [2023-12-26 19:38:47,948][105620] Updated weights for policy 1, policy_version 588628 (0.0010) [2023-12-26 19:38:48,010][105620] Updated weights for policy 1, policy_version 588638 (0.0010) [2023-12-26 19:38:48,071][105620] Updated weights for policy 1, policy_version 588648 (0.0010) [2023-12-26 19:38:48,245][105692] Updated weights for policy 0, policy_version 587793 (0.0008) [2023-12-26 19:38:48,310][105692] Updated weights for policy 0, policy_version 587803 (0.0011) [2023-12-26 19:38:48,373][105692] Updated weights for policy 0, policy_version 587813 (0.0011) [2023-12-26 19:38:48,429][105692] Updated weights for policy 0, policy_version 587823 (0.0010) [2023-12-26 19:38:48,803][105620] Updated weights for policy 1, policy_version 588658 (0.0008) [2023-12-26 19:38:48,865][105620] Updated weights for policy 1, policy_version 588668 (0.0009) [2023-12-26 19:38:48,924][105620] Updated weights for policy 1, policy_version 588678 (0.0009) [2023-12-26 19:38:49,126][105692] Updated weights for policy 0, policy_version 587833 (0.0007) [2023-12-26 19:38:49,178][105692] Updated weights for policy 0, policy_version 587843 (0.0010) [2023-12-26 19:38:49,230][105692] Updated weights for policy 0, policy_version 587853 (0.0011) [2023-12-26 19:38:49,739][105620] Updated weights for policy 1, policy_version 588688 (0.0008) [2023-12-26 19:38:49,805][105620] Updated weights for policy 1, policy_version 588698 (0.0008) [2023-12-26 19:38:49,869][105620] Updated weights for policy 1, policy_version 588708 (0.0008) [2023-12-26 19:38:49,978][105692] Updated weights for policy 0, policy_version 587863 (0.0008) [2023-12-26 19:38:50,037][105692] Updated weights for policy 0, policy_version 587873 (0.0007) [2023-12-26 19:38:50,088][105692] Updated weights for policy 0, policy_version 587883 (0.0008) [2023-12-26 19:38:50,581][105620] Updated weights for policy 1, policy_version 588718 (0.0007) [2023-12-26 19:38:50,647][105620] Updated weights for policy 1, policy_version 588728 (0.0007) [2023-12-26 19:38:50,712][105620] Updated weights for policy 1, policy_version 588738 (0.0008) [2023-12-26 19:38:50,772][105692] Updated weights for policy 0, policy_version 587893 (0.0009) [2023-12-26 19:38:50,819][105585] KL-divergence is very high: 145.1980 [2023-12-26 19:38:50,823][105692] Updated weights for policy 0, policy_version 587903 (0.0008) [2023-12-26 19:38:50,858][105585] KL-divergence is very high: 192.3658 [2023-12-26 19:38:50,874][105692] Updated weights for policy 0, policy_version 587913 (0.0008) [2023-12-26 19:38:50,900][105585] KL-divergence is very high: 137.0878 [2023-12-26 19:38:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 301260800. Throughput: 0: 9734.9, 1: 9534.9. Samples: 301246596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:38:51,062][104569] Avg episode reward: [(0, '6895.312'), (1, '8729.836')] [2023-12-26 19:38:51,524][105620] Updated weights for policy 1, policy_version 588748 (0.0009) [2023-12-26 19:38:51,537][105692] Updated weights for policy 0, policy_version 587923 (0.0006) [2023-12-26 19:38:51,582][105620] Updated weights for policy 1, policy_version 588758 (0.0009) [2023-12-26 19:38:51,601][105692] Updated weights for policy 0, policy_version 587933 (0.0006) [2023-12-26 19:38:51,641][105620] Updated weights for policy 1, policy_version 588768 (0.0008) [2023-12-26 19:38:51,664][105692] Updated weights for policy 0, policy_version 587943 (0.0008) [2023-12-26 19:38:52,357][105620] Updated weights for policy 1, policy_version 588778 (0.0009) [2023-12-26 19:38:52,425][105620] Updated weights for policy 1, policy_version 588788 (0.0011) [2023-12-26 19:38:52,430][105692] Updated weights for policy 0, policy_version 587953 (0.0006) [2023-12-26 19:38:52,481][105692] Updated weights for policy 0, policy_version 587963 (0.0005) [2023-12-26 19:38:52,485][105620] Updated weights for policy 1, policy_version 588798 (0.0010) [2023-12-26 19:38:52,538][105692] Updated weights for policy 0, policy_version 587973 (0.0007) [2023-12-26 19:38:52,552][105620] Updated weights for policy 1, policy_version 588808 (0.0011) [2023-12-26 19:38:52,587][105692] Updated weights for policy 0, policy_version 587983 (0.0008) [2023-12-26 19:38:53,257][105692] Updated weights for policy 0, policy_version 587993 (0.0010) [2023-12-26 19:38:53,268][105620] Updated weights for policy 1, policy_version 588818 (0.0010) [2023-12-26 19:38:53,313][105692] Updated weights for policy 0, policy_version 588003 (0.0008) [2023-12-26 19:38:53,316][105620] Updated weights for policy 1, policy_version 588828 (0.0010) [2023-12-26 19:38:53,365][105620] Updated weights for policy 1, policy_version 588838 (0.0010) [2023-12-26 19:38:53,367][105692] Updated weights for policy 0, policy_version 588013 (0.0005) [2023-12-26 19:38:53,974][105692] Updated weights for policy 0, policy_version 588023 (0.0007) [2023-12-26 19:38:54,034][105692] Updated weights for policy 0, policy_version 588033 (0.0010) [2023-12-26 19:38:54,096][105692] Updated weights for policy 0, policy_version 588043 (0.0010) [2023-12-26 19:38:54,117][105620] Updated weights for policy 1, policy_version 588848 (0.0010) [2023-12-26 19:38:54,177][105620] Updated weights for policy 1, policy_version 588858 (0.0010) [2023-12-26 19:38:54,240][105620] Updated weights for policy 1, policy_version 588868 (0.0005) [2023-12-26 19:38:54,816][105692] Updated weights for policy 0, policy_version 588053 (0.0010) [2023-12-26 19:38:54,876][105692] Updated weights for policy 0, policy_version 588063 (0.0008) [2023-12-26 19:38:54,935][105692] Updated weights for policy 0, policy_version 588073 (0.0009) [2023-12-26 19:38:54,945][105620] Updated weights for policy 1, policy_version 588878 (0.0006) [2023-12-26 19:38:55,001][105620] Updated weights for policy 1, policy_version 588888 (0.0010) [2023-12-26 19:38:55,059][105620] Updated weights for policy 1, policy_version 588898 (0.0005) [2023-12-26 19:38:55,593][105692] Updated weights for policy 0, policy_version 588083 (0.0007) [2023-12-26 19:38:55,661][105692] Updated weights for policy 0, policy_version 588093 (0.0006) [2023-12-26 19:38:55,725][105692] Updated weights for policy 0, policy_version 588103 (0.0007) [2023-12-26 19:38:55,738][105620] Updated weights for policy 1, policy_version 588908 (0.0007) [2023-12-26 19:38:55,803][105620] Updated weights for policy 1, policy_version 588918 (0.0010) [2023-12-26 19:38:55,866][105620] Updated weights for policy 1, policy_version 588928 (0.0009) [2023-12-26 19:38:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.1, 300 sec: 19549.7). Total num frames: 301359104. Throughput: 0: 9667.0, 1: 9610.1. Samples: 301364900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:38:56,063][104569] Avg episode reward: [(0, '7534.408'), (1, '8821.054')] [2023-12-26 19:38:56,254][105692] Updated weights for policy 0, policy_version 588113 (0.0006) [2023-12-26 19:38:56,304][105692] Updated weights for policy 0, policy_version 588123 (0.0005) [2023-12-26 19:38:56,350][105692] Updated weights for policy 0, policy_version 588133 (0.0005) [2023-12-26 19:38:56,406][105692] Updated weights for policy 0, policy_version 588143 (0.0005) [2023-12-26 19:38:56,461][105620] Updated weights for policy 1, policy_version 588938 (0.0005) [2023-12-26 19:38:56,530][105620] Updated weights for policy 1, policy_version 588948 (0.0005) [2023-12-26 19:38:56,603][105620] Updated weights for policy 1, policy_version 588958 (0.0005) [2023-12-26 19:38:56,669][105620] Updated weights for policy 1, policy_version 588968 (0.0009) [2023-12-26 19:38:56,933][105692] Updated weights for policy 0, policy_version 588153 (0.0005) [2023-12-26 19:38:56,984][105692] Updated weights for policy 0, policy_version 588163 (0.0008) [2023-12-26 19:38:57,038][105692] Updated weights for policy 0, policy_version 588173 (0.0008) [2023-12-26 19:38:57,241][105620] Updated weights for policy 1, policy_version 588978 (0.0010) [2023-12-26 19:38:57,285][105620] Updated weights for policy 1, policy_version 588988 (0.0010) [2023-12-26 19:38:57,344][105620] Updated weights for policy 1, policy_version 588998 (0.0008) [2023-12-26 19:38:57,658][105692] Updated weights for policy 0, policy_version 588183 (0.0008) [2023-12-26 19:38:57,704][105692] Updated weights for policy 0, policy_version 588193 (0.0008) [2023-12-26 19:38:57,754][105692] Updated weights for policy 0, policy_version 588203 (0.0009) [2023-12-26 19:38:58,029][105620] Updated weights for policy 1, policy_version 589008 (0.0009) [2023-12-26 19:38:58,100][105620] Updated weights for policy 1, policy_version 589018 (0.0009) [2023-12-26 19:38:58,164][105620] Updated weights for policy 1, policy_version 589028 (0.0008) [2023-12-26 19:38:58,453][105692] Updated weights for policy 0, policy_version 588213 (0.0008) [2023-12-26 19:38:58,520][105692] Updated weights for policy 0, policy_version 588223 (0.0008) [2023-12-26 19:38:58,573][105692] Updated weights for policy 0, policy_version 588233 (0.0009) [2023-12-26 19:38:58,898][105620] Updated weights for policy 1, policy_version 589038 (0.0007) [2023-12-26 19:38:58,963][105620] Updated weights for policy 1, policy_version 589048 (0.0008) [2023-12-26 19:38:59,016][105620] Updated weights for policy 1, policy_version 589058 (0.0009) [2023-12-26 19:38:59,347][105692] Updated weights for policy 0, policy_version 588243 (0.0009) [2023-12-26 19:38:59,408][105692] Updated weights for policy 0, policy_version 588253 (0.0010) [2023-12-26 19:38:59,469][105692] Updated weights for policy 0, policy_version 588263 (0.0008) [2023-12-26 19:38:59,726][105620] Updated weights for policy 1, policy_version 589068 (0.0007) [2023-12-26 19:38:59,785][105620] Updated weights for policy 1, policy_version 589078 (0.0008) [2023-12-26 19:38:59,845][105620] Updated weights for policy 1, policy_version 589088 (0.0009) [2023-12-26 19:39:00,203][105692] Updated weights for policy 0, policy_version 588273 (0.0009) [2023-12-26 19:39:00,258][105692] Updated weights for policy 0, policy_version 588283 (0.0005) [2023-12-26 19:39:00,321][105692] Updated weights for policy 0, policy_version 588293 (0.0007) [2023-12-26 19:39:00,380][105692] Updated weights for policy 0, policy_version 588303 (0.0006) [2023-12-26 19:39:00,542][105620] Updated weights for policy 1, policy_version 589098 (0.0008) [2023-12-26 19:39:00,589][105620] Updated weights for policy 1, policy_version 589108 (0.0005) [2023-12-26 19:39:00,637][105620] Updated weights for policy 1, policy_version 589118 (0.0005) [2023-12-26 19:39:00,693][105620] Updated weights for policy 1, policy_version 589128 (0.0005) [2023-12-26 19:39:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 301457408. Throughput: 0: 9836.8, 1: 9681.4. Samples: 301430136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:39:01,063][104569] Avg episode reward: [(0, '8996.343'), (1, '8907.582')] [2023-12-26 19:39:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000588304_150626304.pth... [2023-12-26 19:39:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000589128_150831104.pth... [2023-12-26 19:39:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000587184_150339584.pth [2023-12-26 19:39:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000587976_150536192.pth [2023-12-26 19:39:01,139][105692] Updated weights for policy 0, policy_version 588313 (0.0006) [2023-12-26 19:39:01,199][105692] Updated weights for policy 0, policy_version 588323 (0.0006) [2023-12-26 19:39:01,265][105692] Updated weights for policy 0, policy_version 588333 (0.0007) [2023-12-26 19:39:01,389][105620] Updated weights for policy 1, policy_version 589138 (0.0010) [2023-12-26 19:39:01,439][105620] Updated weights for policy 1, policy_version 589148 (0.0011) [2023-12-26 19:39:01,495][105620] Updated weights for policy 1, policy_version 589158 (0.0010) [2023-12-26 19:39:01,957][105692] Updated weights for policy 0, policy_version 588343 (0.0008) [2023-12-26 19:39:02,015][105692] Updated weights for policy 0, policy_version 588353 (0.0008) [2023-12-26 19:39:02,067][105692] Updated weights for policy 0, policy_version 588363 (0.0008) [2023-12-26 19:39:02,224][105620] Updated weights for policy 1, policy_version 589168 (0.0010) [2023-12-26 19:39:02,279][105620] Updated weights for policy 1, policy_version 589178 (0.0010) [2023-12-26 19:39:02,334][105620] Updated weights for policy 1, policy_version 589188 (0.0010) [2023-12-26 19:39:02,747][105692] Updated weights for policy 0, policy_version 588373 (0.0007) [2023-12-26 19:39:02,804][105692] Updated weights for policy 0, policy_version 588383 (0.0006) [2023-12-26 19:39:02,849][105692] Updated weights for policy 0, policy_version 588393 (0.0008) [2023-12-26 19:39:03,098][105620] Updated weights for policy 1, policy_version 589198 (0.0009) [2023-12-26 19:39:03,146][105620] Updated weights for policy 1, policy_version 589208 (0.0010) [2023-12-26 19:39:03,216][105620] Updated weights for policy 1, policy_version 589218 (0.0010) [2023-12-26 19:39:03,558][105692] Updated weights for policy 0, policy_version 588403 (0.0008) [2023-12-26 19:39:03,611][105692] Updated weights for policy 0, policy_version 588413 (0.0008) [2023-12-26 19:39:03,667][105692] Updated weights for policy 0, policy_version 588423 (0.0008) [2023-12-26 19:39:03,978][105620] Updated weights for policy 1, policy_version 589228 (0.0009) [2023-12-26 19:39:04,037][105620] Updated weights for policy 1, policy_version 589238 (0.0010) [2023-12-26 19:39:04,093][105620] Updated weights for policy 1, policy_version 589248 (0.0010) [2023-12-26 19:39:04,391][105692] Updated weights for policy 0, policy_version 588433 (0.0006) [2023-12-26 19:39:04,447][105692] Updated weights for policy 0, policy_version 588443 (0.0011) [2023-12-26 19:39:04,507][105692] Updated weights for policy 0, policy_version 588453 (0.0010) [2023-12-26 19:39:04,560][105692] Updated weights for policy 0, policy_version 588463 (0.0010) [2023-12-26 19:39:04,797][105620] Updated weights for policy 1, policy_version 589258 (0.0011) [2023-12-26 19:39:04,850][105620] Updated weights for policy 1, policy_version 589268 (0.0010) [2023-12-26 19:39:04,923][105620] Updated weights for policy 1, policy_version 589278 (0.0010) [2023-12-26 19:39:04,971][105620] Updated weights for policy 1, policy_version 589288 (0.0010) [2023-12-26 19:39:05,248][105692] Updated weights for policy 0, policy_version 588473 (0.0010) [2023-12-26 19:39:05,310][105692] Updated weights for policy 0, policy_version 588483 (0.0010) [2023-12-26 19:39:05,392][105692] Updated weights for policy 0, policy_version 588493 (0.0011) [2023-12-26 19:39:05,722][105620] Updated weights for policy 1, policy_version 589298 (0.0011) [2023-12-26 19:39:05,777][105620] Updated weights for policy 1, policy_version 589308 (0.0010) [2023-12-26 19:39:05,832][105620] Updated weights for policy 1, policy_version 589318 (0.0010) [2023-12-26 19:39:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 301555712. Throughput: 0: 9757.3, 1: 9656.4. Samples: 301546080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:39:06,063][104569] Avg episode reward: [(0, '9170.910'), (1, '8999.019')] [2023-12-26 19:39:06,118][105692] Updated weights for policy 0, policy_version 588503 (0.0011) [2023-12-26 19:39:06,181][105692] Updated weights for policy 0, policy_version 588513 (0.0010) [2023-12-26 19:39:06,239][105692] Updated weights for policy 0, policy_version 588523 (0.0011) [2023-12-26 19:39:06,532][105620] Updated weights for policy 1, policy_version 589328 (0.0009) [2023-12-26 19:39:06,580][105620] Updated weights for policy 1, policy_version 589338 (0.0007) [2023-12-26 19:39:06,634][105620] Updated weights for policy 1, policy_version 589348 (0.0005) [2023-12-26 19:39:06,984][105692] Updated weights for policy 0, policy_version 588533 (0.0010) [2023-12-26 19:39:07,049][105692] Updated weights for policy 0, policy_version 588543 (0.0011) [2023-12-26 19:39:07,115][105692] Updated weights for policy 0, policy_version 588553 (0.0010) [2023-12-26 19:39:07,200][105620] Updated weights for policy 1, policy_version 589358 (0.0007) [2023-12-26 19:39:07,260][105620] Updated weights for policy 1, policy_version 589368 (0.0011) [2023-12-26 19:39:07,322][105620] Updated weights for policy 1, policy_version 589378 (0.0011) [2023-12-26 19:39:07,837][105692] Updated weights for policy 0, policy_version 588563 (0.0011) [2023-12-26 19:39:07,902][105692] Updated weights for policy 0, policy_version 588573 (0.0010) [2023-12-26 19:39:07,968][105692] Updated weights for policy 0, policy_version 588583 (0.0010) [2023-12-26 19:39:07,981][105620] Updated weights for policy 1, policy_version 589388 (0.0011) [2023-12-26 19:39:08,042][105620] Updated weights for policy 1, policy_version 589398 (0.0011) [2023-12-26 19:39:08,107][105620] Updated weights for policy 1, policy_version 589408 (0.0010) [2023-12-26 19:39:08,584][105692] Updated weights for policy 0, policy_version 588593 (0.0010) [2023-12-26 19:39:08,639][105692] Updated weights for policy 0, policy_version 588603 (0.0005) [2023-12-26 19:39:08,701][105692] Updated weights for policy 0, policy_version 588613 (0.0005) [2023-12-26 19:39:08,764][105692] Updated weights for policy 0, policy_version 588623 (0.0005) [2023-12-26 19:39:08,780][105620] Updated weights for policy 1, policy_version 589418 (0.0007) [2023-12-26 19:39:08,832][105620] Updated weights for policy 1, policy_version 589428 (0.0007) [2023-12-26 19:39:08,897][105620] Updated weights for policy 1, policy_version 589438 (0.0005) [2023-12-26 19:39:08,963][105620] Updated weights for policy 1, policy_version 589448 (0.0005) [2023-12-26 19:39:09,501][105692] Updated weights for policy 0, policy_version 588633 (0.0010) [2023-12-26 19:39:09,564][105692] Updated weights for policy 0, policy_version 588643 (0.0011) [2023-12-26 19:39:09,575][105620] Updated weights for policy 1, policy_version 589458 (0.0007) [2023-12-26 19:39:09,617][105692] Updated weights for policy 0, policy_version 588653 (0.0011) [2023-12-26 19:39:09,639][105620] Updated weights for policy 1, policy_version 589468 (0.0008) [2023-12-26 19:39:09,700][105620] Updated weights for policy 1, policy_version 589478 (0.0006) [2023-12-26 19:39:10,299][105620] Updated weights for policy 1, policy_version 589488 (0.0005) [2023-12-26 19:39:10,366][105620] Updated weights for policy 1, policy_version 589498 (0.0009) [2023-12-26 19:39:10,433][105620] Updated weights for policy 1, policy_version 589508 (0.0011) [2023-12-26 19:39:10,452][105692] Updated weights for policy 0, policy_version 588663 (0.0007) [2023-12-26 19:39:10,512][105692] Updated weights for policy 0, policy_version 588673 (0.0010) [2023-12-26 19:39:10,580][105692] Updated weights for policy 0, policy_version 588683 (0.0011) [2023-12-26 19:39:11,020][105620] Updated weights for policy 1, policy_version 589518 (0.0010) [2023-12-26 19:39:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19389.0, 300 sec: 19522.0). Total num frames: 301654016. Throughput: 0: 9758.3, 1: 9877.6. Samples: 301666672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:39:11,062][104569] Avg episode reward: [(0, '8837.211'), (1, '1349.738')] [2023-12-26 19:39:11,078][105620] Updated weights for policy 1, policy_version 589528 (0.0010) [2023-12-26 19:39:11,146][105620] Updated weights for policy 1, policy_version 589538 (0.0012) [2023-12-26 19:39:11,378][105692] Updated weights for policy 0, policy_version 588693 (0.0010) [2023-12-26 19:39:11,445][105692] Updated weights for policy 0, policy_version 588703 (0.0006) [2023-12-26 19:39:11,495][105692] Updated weights for policy 0, policy_version 588713 (0.0006) [2023-12-26 19:39:11,977][105620] Updated weights for policy 1, policy_version 589548 (0.0010) [2023-12-26 19:39:12,032][105620] Updated weights for policy 1, policy_version 589558 (0.0009) [2023-12-26 19:39:12,088][105692] Updated weights for policy 0, policy_version 588723 (0.0006) [2023-12-26 19:39:12,090][105620] Updated weights for policy 1, policy_version 589568 (0.0009) [2023-12-26 19:39:12,154][105692] Updated weights for policy 0, policy_version 588733 (0.0006) [2023-12-26 19:39:12,201][105692] Updated weights for policy 0, policy_version 588743 (0.0009) [2023-12-26 19:39:12,839][105692] Updated weights for policy 0, policy_version 588753 (0.0008) [2023-12-26 19:39:12,893][105692] Updated weights for policy 0, policy_version 588763 (0.0005) [2023-12-26 19:39:12,938][105692] Updated weights for policy 0, policy_version 588773 (0.0005) [2023-12-26 19:39:12,941][105620] Updated weights for policy 1, policy_version 589578 (0.0008) [2023-12-26 19:39:12,994][105692] Updated weights for policy 0, policy_version 588783 (0.0006) [2023-12-26 19:39:13,005][105620] Updated weights for policy 1, policy_version 589588 (0.0007) [2023-12-26 19:39:13,079][105620] Updated weights for policy 1, policy_version 589598 (0.0009) [2023-12-26 19:39:13,138][105620] Updated weights for policy 1, policy_version 589608 (0.0009) [2023-12-26 19:39:13,655][105692] Updated weights for policy 0, policy_version 588793 (0.0009) [2023-12-26 19:39:13,725][105692] Updated weights for policy 0, policy_version 588803 (0.0010) [2023-12-26 19:39:13,796][105692] Updated weights for policy 0, policy_version 588813 (0.0009) [2023-12-26 19:39:13,813][105620] Updated weights for policy 1, policy_version 589618 (0.0005) [2023-12-26 19:39:13,861][105620] Updated weights for policy 1, policy_version 589628 (0.0007) [2023-12-26 19:39:13,917][105620] Updated weights for policy 1, policy_version 589638 (0.0008) [2023-12-26 19:39:14,506][105692] Updated weights for policy 0, policy_version 588823 (0.0010) [2023-12-26 19:39:14,557][105692] Updated weights for policy 0, policy_version 588833 (0.0010) [2023-12-26 19:39:14,605][105692] Updated weights for policy 0, policy_version 588843 (0.0010) [2023-12-26 19:39:14,665][105620] Updated weights for policy 1, policy_version 589648 (0.0010) [2023-12-26 19:39:14,725][105620] Updated weights for policy 1, policy_version 589658 (0.0010) [2023-12-26 19:39:14,788][105620] Updated weights for policy 1, policy_version 589668 (0.0009) [2023-12-26 19:39:15,351][105692] Updated weights for policy 0, policy_version 588853 (0.0008) [2023-12-26 19:39:15,409][105692] Updated weights for policy 0, policy_version 588863 (0.0008) [2023-12-26 19:39:15,466][105692] Updated weights for policy 0, policy_version 588873 (0.0009) [2023-12-26 19:39:15,556][105620] Updated weights for policy 1, policy_version 589678 (0.0007) [2023-12-26 19:39:15,613][105620] Updated weights for policy 1, policy_version 589688 (0.0005) [2023-12-26 19:39:15,671][105620] Updated weights for policy 1, policy_version 589698 (0.0006) [2023-12-26 19:39:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 301752320. Throughput: 0: 9751.9, 1: 9847.0. Samples: 301724600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:39:16,063][104569] Avg episode reward: [(0, '8653.922'), (1, '4253.312')] [2023-12-26 19:39:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000589704_150978560.pth... [2023-12-26 19:39:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000588552_150683648.pth [2023-12-26 19:39:16,089][105692] Updated weights for policy 0, policy_version 588883 (0.0008) [2023-12-26 19:39:16,152][105692] Updated weights for policy 0, policy_version 588893 (0.0005) [2023-12-26 19:39:16,214][105692] Updated weights for policy 0, policy_version 588903 (0.0006) [2023-12-26 19:39:16,273][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000588912_150781952.pth... [2023-12-26 19:39:16,275][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000587728_150478848.pth [2023-12-26 19:39:16,375][105620] Updated weights for policy 1, policy_version 589708 (0.0010) [2023-12-26 19:39:16,423][105620] Updated weights for policy 1, policy_version 589718 (0.0010) [2023-12-26 19:39:16,481][105620] Updated weights for policy 1, policy_version 589728 (0.0010) [2023-12-26 19:39:16,836][105692] Updated weights for policy 0, policy_version 588913 (0.0007) [2023-12-26 19:39:16,894][105692] Updated weights for policy 0, policy_version 588923 (0.0009) [2023-12-26 19:39:16,950][105692] Updated weights for policy 0, policy_version 588933 (0.0010) [2023-12-26 19:39:16,997][105692] Updated weights for policy 0, policy_version 588943 (0.0010) [2023-12-26 19:39:17,227][105620] Updated weights for policy 1, policy_version 589738 (0.0010) [2023-12-26 19:39:17,278][105620] Updated weights for policy 1, policy_version 589748 (0.0010) [2023-12-26 19:39:17,326][105620] Updated weights for policy 1, policy_version 589758 (0.0010) [2023-12-26 19:39:17,373][105620] Updated weights for policy 1, policy_version 589768 (0.0010) [2023-12-26 19:39:17,693][105692] Updated weights for policy 0, policy_version 588953 (0.0006) [2023-12-26 19:39:17,749][105692] Updated weights for policy 0, policy_version 588963 (0.0005) [2023-12-26 19:39:17,812][105692] Updated weights for policy 0, policy_version 588973 (0.0005) [2023-12-26 19:39:18,139][105620] Updated weights for policy 1, policy_version 589778 (0.0010) [2023-12-26 19:39:18,183][105620] Updated weights for policy 1, policy_version 589788 (0.0010) [2023-12-26 19:39:18,234][105620] Updated weights for policy 1, policy_version 589798 (0.0010) [2023-12-26 19:39:18,320][105692] Updated weights for policy 0, policy_version 588983 (0.0005) [2023-12-26 19:39:18,385][105692] Updated weights for policy 0, policy_version 588993 (0.0009) [2023-12-26 19:39:18,446][105692] Updated weights for policy 0, policy_version 589003 (0.0009) [2023-12-26 19:39:18,991][105620] Updated weights for policy 1, policy_version 589808 (0.0010) [2023-12-26 19:39:19,046][105620] Updated weights for policy 1, policy_version 589818 (0.0010) [2023-12-26 19:39:19,104][105620] Updated weights for policy 1, policy_version 589828 (0.0010) [2023-12-26 19:39:19,177][105692] Updated weights for policy 0, policy_version 589013 (0.0010) [2023-12-26 19:39:19,235][105692] Updated weights for policy 0, policy_version 589023 (0.0010) [2023-12-26 19:39:19,296][105692] Updated weights for policy 0, policy_version 589033 (0.0011) [2023-12-26 19:39:19,899][105620] Updated weights for policy 1, policy_version 589838 (0.0011) [2023-12-26 19:39:19,970][105620] Updated weights for policy 1, policy_version 589848 (0.0008) [2023-12-26 19:39:20,043][105620] Updated weights for policy 1, policy_version 589858 (0.0010) [2023-12-26 19:39:20,081][105692] Updated weights for policy 0, policy_version 589043 (0.0010) [2023-12-26 19:39:20,141][105692] Updated weights for policy 0, policy_version 589053 (0.0011) [2023-12-26 19:39:20,197][105692] Updated weights for policy 0, policy_version 589063 (0.0011) [2023-12-26 19:39:20,826][105620] Updated weights for policy 1, policy_version 589868 (0.0009) [2023-12-26 19:39:20,893][105620] Updated weights for policy 1, policy_version 589878 (0.0009) [2023-12-26 19:39:20,942][105692] Updated weights for policy 0, policy_version 589073 (0.0011) [2023-12-26 19:39:20,948][105620] Updated weights for policy 1, policy_version 589888 (0.0009) [2023-12-26 19:39:20,993][105692] Updated weights for policy 0, policy_version 589083 (0.0006) [2023-12-26 19:39:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 301850624. Throughput: 0: 9805.8, 1: 9844.4. Samples: 301841816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:39:21,063][104569] Avg episode reward: [(0, '9082.715'), (1, '985.500')] [2023-12-26 19:39:21,064][105692] Updated weights for policy 0, policy_version 589093 (0.0009) [2023-12-26 19:39:21,128][105692] Updated weights for policy 0, policy_version 589103 (0.0008) [2023-12-26 19:39:21,743][105620] Updated weights for policy 1, policy_version 589898 (0.0008) [2023-12-26 19:39:21,804][105620] Updated weights for policy 1, policy_version 589908 (0.0009) [2023-12-26 19:39:21,862][105620] Updated weights for policy 1, policy_version 589918 (0.0009) [2023-12-26 19:39:21,923][105620] Updated weights for policy 1, policy_version 589928 (0.0008) [2023-12-26 19:39:21,956][105692] Updated weights for policy 0, policy_version 589113 (0.0009) [2023-12-26 19:39:22,014][105692] Updated weights for policy 0, policy_version 589123 (0.0008) [2023-12-26 19:39:22,067][105692] Updated weights for policy 0, policy_version 589133 (0.0008) [2023-12-26 19:39:22,679][105620] Updated weights for policy 1, policy_version 589938 (0.0006) [2023-12-26 19:39:22,739][105620] Updated weights for policy 1, policy_version 589948 (0.0007) [2023-12-26 19:39:22,813][105620] Updated weights for policy 1, policy_version 589958 (0.0010) [2023-12-26 19:39:22,862][105692] Updated weights for policy 0, policy_version 589143 (0.0008) [2023-12-26 19:39:22,923][105692] Updated weights for policy 0, policy_version 589153 (0.0006) [2023-12-26 19:39:22,986][105692] Updated weights for policy 0, policy_version 589163 (0.0009) [2023-12-26 19:39:23,563][105620] Updated weights for policy 1, policy_version 589968 (0.0009) [2023-12-26 19:39:23,625][105620] Updated weights for policy 1, policy_version 589978 (0.0009) [2023-12-26 19:39:23,656][105692] Updated weights for policy 0, policy_version 589173 (0.0009) [2023-12-26 19:39:23,682][105620] Updated weights for policy 1, policy_version 589988 (0.0007) [2023-12-26 19:39:23,705][105692] Updated weights for policy 0, policy_version 589183 (0.0007) [2023-12-26 19:39:23,756][105692] Updated weights for policy 0, policy_version 589193 (0.0009) [2023-12-26 19:39:24,419][105620] Updated weights for policy 1, policy_version 589998 (0.0007) [2023-12-26 19:39:24,471][105692] Updated weights for policy 0, policy_version 589203 (0.0008) [2023-12-26 19:39:24,484][105620] Updated weights for policy 1, policy_version 590008 (0.0006) [2023-12-26 19:39:24,519][105692] Updated weights for policy 0, policy_version 589213 (0.0005) [2023-12-26 19:39:24,540][105620] Updated weights for policy 1, policy_version 590018 (0.0009) [2023-12-26 19:39:24,572][105692] Updated weights for policy 0, policy_version 589223 (0.0007) [2023-12-26 19:39:25,135][105620] Updated weights for policy 1, policy_version 590028 (0.0009) [2023-12-26 19:39:25,193][105620] Updated weights for policy 1, policy_version 590038 (0.0010) [2023-12-26 19:39:25,215][105692] Updated weights for policy 0, policy_version 589233 (0.0010) [2023-12-26 19:39:25,251][105620] Updated weights for policy 1, policy_version 590048 (0.0010) [2023-12-26 19:39:25,269][105692] Updated weights for policy 0, policy_version 589243 (0.0006) [2023-12-26 19:39:25,330][105692] Updated weights for policy 0, policy_version 589253 (0.0009) [2023-12-26 19:39:25,385][105692] Updated weights for policy 0, policy_version 589263 (0.0009) [2023-12-26 19:39:25,927][105620] Updated weights for policy 1, policy_version 590058 (0.0006) [2023-12-26 19:39:25,985][105620] Updated weights for policy 1, policy_version 590068 (0.0008) [2023-12-26 19:39:26,037][105620] Updated weights for policy 1, policy_version 590078 (0.0010) [2023-12-26 19:39:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 301940736. Throughput: 0: 9770.1, 1: 9794.2. Samples: 301956176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:39:26,063][104569] Avg episode reward: [(0, '8904.572'), (1, '1513.087')] [2023-12-26 19:39:26,101][105620] Updated weights for policy 1, policy_version 590088 (0.0008) [2023-12-26 19:39:26,103][105692] Updated weights for policy 0, policy_version 589273 (0.0010) [2023-12-26 19:39:26,158][105692] Updated weights for policy 0, policy_version 589283 (0.0009) [2023-12-26 19:39:26,215][105692] Updated weights for policy 0, policy_version 589293 (0.0010) [2023-12-26 19:39:26,848][105620] Updated weights for policy 1, policy_version 590098 (0.0010) [2023-12-26 19:39:26,901][105692] Updated weights for policy 0, policy_version 589303 (0.0010) [2023-12-26 19:39:26,909][105620] Updated weights for policy 1, policy_version 590108 (0.0010) [2023-12-26 19:39:26,962][105692] Updated weights for policy 0, policy_version 589313 (0.0010) [2023-12-26 19:39:26,964][105620] Updated weights for policy 1, policy_version 590118 (0.0010) [2023-12-26 19:39:27,016][105692] Updated weights for policy 0, policy_version 589323 (0.0010) [2023-12-26 19:39:27,584][105692] Updated weights for policy 0, policy_version 589333 (0.0008) [2023-12-26 19:39:27,627][105692] Updated weights for policy 0, policy_version 589343 (0.0005) [2023-12-26 19:39:27,688][105692] Updated weights for policy 0, policy_version 589353 (0.0007) [2023-12-26 19:39:27,696][105620] Updated weights for policy 1, policy_version 590128 (0.0010) [2023-12-26 19:39:27,750][105620] Updated weights for policy 1, policy_version 590138 (0.0010) [2023-12-26 19:39:27,811][105620] Updated weights for policy 1, policy_version 590148 (0.0010) [2023-12-26 19:39:28,404][105692] Updated weights for policy 0, policy_version 589363 (0.0008) [2023-12-26 19:39:28,460][105692] Updated weights for policy 0, policy_version 589373 (0.0007) [2023-12-26 19:39:28,517][105692] Updated weights for policy 0, policy_version 589383 (0.0006) [2023-12-26 19:39:28,541][105620] Updated weights for policy 1, policy_version 590158 (0.0010) [2023-12-26 19:39:28,593][105620] Updated weights for policy 1, policy_version 590168 (0.0010) [2023-12-26 19:39:28,651][105620] Updated weights for policy 1, policy_version 590178 (0.0010) [2023-12-26 19:39:29,142][105692] Updated weights for policy 0, policy_version 589393 (0.0005) [2023-12-26 19:39:29,199][105692] Updated weights for policy 0, policy_version 589403 (0.0005) [2023-12-26 19:39:29,266][105692] Updated weights for policy 0, policy_version 589413 (0.0009) [2023-12-26 19:39:29,317][105692] Updated weights for policy 0, policy_version 589423 (0.0009) [2023-12-26 19:39:29,417][105620] Updated weights for policy 1, policy_version 590188 (0.0010) [2023-12-26 19:39:29,478][105620] Updated weights for policy 1, policy_version 590198 (0.0009) [2023-12-26 19:39:29,531][105620] Updated weights for policy 1, policy_version 590208 (0.0008) [2023-12-26 19:39:29,983][105692] Updated weights for policy 0, policy_version 589433 (0.0007) [2023-12-26 19:39:30,045][105692] Updated weights for policy 0, policy_version 589443 (0.0008) [2023-12-26 19:39:30,098][105692] Updated weights for policy 0, policy_version 589453 (0.0008) [2023-12-26 19:39:30,341][105620] Updated weights for policy 1, policy_version 590218 (0.0010) [2023-12-26 19:39:30,399][105620] Updated weights for policy 1, policy_version 590228 (0.0010) [2023-12-26 19:39:30,466][105620] Updated weights for policy 1, policy_version 590238 (0.0010) [2023-12-26 19:39:30,527][105620] Updated weights for policy 1, policy_version 590248 (0.0010) [2023-12-26 19:39:30,866][105692] Updated weights for policy 0, policy_version 589463 (0.0006) [2023-12-26 19:39:30,922][105692] Updated weights for policy 0, policy_version 589473 (0.0005) [2023-12-26 19:39:30,976][105692] Updated weights for policy 0, policy_version 589483 (0.0005) [2023-12-26 19:39:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 302047232. Throughput: 0: 9880.0, 1: 9729.4. Samples: 302015600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:39:31,062][104569] Avg episode reward: [(0, '8908.861'), (1, '5499.078')] [2023-12-26 19:39:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000589488_150929408.pth... [2023-12-26 19:39:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000590248_151117824.pth... [2023-12-26 19:39:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000589128_150831104.pth [2023-12-26 19:39:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000588304_150626304.pth [2023-12-26 19:39:31,263][105620] Updated weights for policy 1, policy_version 590258 (0.0011) [2023-12-26 19:39:31,331][105620] Updated weights for policy 1, policy_version 590268 (0.0012) [2023-12-26 19:39:31,399][105620] Updated weights for policy 1, policy_version 590279 (0.0010) [2023-12-26 19:39:31,586][105692] Updated weights for policy 0, policy_version 589493 (0.0005) [2023-12-26 19:39:31,650][105692] Updated weights for policy 0, policy_version 589503 (0.0007) [2023-12-26 19:39:31,717][105692] Updated weights for policy 0, policy_version 589513 (0.0008) [2023-12-26 19:39:32,132][105620] Updated weights for policy 1, policy_version 590289 (0.0010) [2023-12-26 19:39:32,184][105620] Updated weights for policy 1, policy_version 590299 (0.0010) [2023-12-26 19:39:32,232][105620] Updated weights for policy 1, policy_version 590309 (0.0010) [2023-12-26 19:39:32,270][105692] Updated weights for policy 0, policy_version 589523 (0.0008) [2023-12-26 19:39:32,334][105692] Updated weights for policy 0, policy_version 589533 (0.0005) [2023-12-26 19:39:32,399][105692] Updated weights for policy 0, policy_version 589543 (0.0007) [2023-12-26 19:39:32,958][105620] Updated weights for policy 1, policy_version 590319 (0.0007) [2023-12-26 19:39:33,002][105620] Updated weights for policy 1, policy_version 590329 (0.0005) [2023-12-26 19:39:33,025][105692] Updated weights for policy 0, policy_version 589553 (0.0006) [2023-12-26 19:39:33,054][105620] Updated weights for policy 1, policy_version 590339 (0.0005) [2023-12-26 19:39:33,076][105692] Updated weights for policy 0, policy_version 589563 (0.0010) [2023-12-26 19:39:33,137][105692] Updated weights for policy 0, policy_version 589573 (0.0010) [2023-12-26 19:39:33,192][105692] Updated weights for policy 0, policy_version 589583 (0.0010) [2023-12-26 19:39:33,655][105620] Updated weights for policy 1, policy_version 590349 (0.0005) [2023-12-26 19:39:33,725][105620] Updated weights for policy 1, policy_version 590359 (0.0007) [2023-12-26 19:39:33,780][105620] Updated weights for policy 1, policy_version 590369 (0.0008) [2023-12-26 19:39:33,947][105692] Updated weights for policy 0, policy_version 589593 (0.0011) [2023-12-26 19:39:34,017][105692] Updated weights for policy 0, policy_version 589603 (0.0011) [2023-12-26 19:39:34,083][105692] Updated weights for policy 0, policy_version 589613 (0.0011) [2023-12-26 19:39:34,431][105620] Updated weights for policy 1, policy_version 590379 (0.0009) [2023-12-26 19:39:34,481][105620] Updated weights for policy 1, policy_version 590389 (0.0008) [2023-12-26 19:39:34,530][105620] Updated weights for policy 1, policy_version 590399 (0.0008) [2023-12-26 19:39:34,742][105692] Updated weights for policy 0, policy_version 589623 (0.0011) [2023-12-26 19:39:34,807][105692] Updated weights for policy 0, policy_version 589633 (0.0011) [2023-12-26 19:39:34,875][105692] Updated weights for policy 0, policy_version 589643 (0.0010) [2023-12-26 19:39:35,191][105620] Updated weights for policy 1, policy_version 590409 (0.0008) [2023-12-26 19:39:35,236][105620] Updated weights for policy 1, policy_version 590419 (0.0008) [2023-12-26 19:39:35,285][105620] Updated weights for policy 1, policy_version 590429 (0.0007) [2023-12-26 19:39:35,341][105620] Updated weights for policy 1, policy_version 590439 (0.0005) [2023-12-26 19:39:35,483][105692] Updated weights for policy 0, policy_version 589653 (0.0006) [2023-12-26 19:39:35,545][105692] Updated weights for policy 0, policy_version 589663 (0.0005) [2023-12-26 19:39:35,597][105692] Updated weights for policy 0, policy_version 589673 (0.0005) [2023-12-26 19:39:35,924][105620] Updated weights for policy 1, policy_version 590449 (0.0005) [2023-12-26 19:39:35,977][105620] Updated weights for policy 1, policy_version 590459 (0.0005) [2023-12-26 19:39:36,041][105620] Updated weights for policy 1, policy_version 590469 (0.0005) [2023-12-26 19:39:36,062][104569] Fps is (10 sec: 21299.1, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 302153728. Throughput: 0: 10061.4, 1: 9708.8. Samples: 302136256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:39:36,063][104569] Avg episode reward: [(0, '8634.286'), (1, '8340.233')] [2023-12-26 19:39:36,122][105692] Updated weights for policy 0, policy_version 589683 (0.0005) [2023-12-26 19:39:36,195][105692] Updated weights for policy 0, policy_version 589693 (0.0008) [2023-12-26 19:39:36,262][105692] Updated weights for policy 0, policy_version 589703 (0.0010) [2023-12-26 19:39:36,799][105620] Updated weights for policy 1, policy_version 590479 (0.0007) [2023-12-26 19:39:36,862][105620] Updated weights for policy 1, policy_version 590489 (0.0008) [2023-12-26 19:39:36,898][105692] Updated weights for policy 0, policy_version 589713 (0.0011) [2023-12-26 19:39:36,924][105620] Updated weights for policy 1, policy_version 590499 (0.0009) [2023-12-26 19:39:36,955][105692] Updated weights for policy 0, policy_version 589723 (0.0011) [2023-12-26 19:39:37,000][105692] Updated weights for policy 0, policy_version 589733 (0.0010) [2023-12-26 19:39:37,049][105692] Updated weights for policy 0, policy_version 589743 (0.0010) [2023-12-26 19:39:37,647][105620] Updated weights for policy 1, policy_version 590509 (0.0008) [2023-12-26 19:39:37,716][105620] Updated weights for policy 1, policy_version 590519 (0.0010) [2023-12-26 19:39:37,772][105692] Updated weights for policy 0, policy_version 589753 (0.0011) [2023-12-26 19:39:37,774][105620] Updated weights for policy 1, policy_version 590529 (0.0009) [2023-12-26 19:39:37,832][105692] Updated weights for policy 0, policy_version 589763 (0.0011) [2023-12-26 19:39:37,892][105692] Updated weights for policy 0, policy_version 589773 (0.0011) [2023-12-26 19:39:38,493][105620] Updated weights for policy 1, policy_version 590539 (0.0007) [2023-12-26 19:39:38,561][105620] Updated weights for policy 1, policy_version 590549 (0.0005) [2023-12-26 19:39:38,587][105692] Updated weights for policy 0, policy_version 589783 (0.0007) [2023-12-26 19:39:38,621][105620] Updated weights for policy 1, policy_version 590559 (0.0009) [2023-12-26 19:39:38,645][105692] Updated weights for policy 0, policy_version 589793 (0.0005) [2023-12-26 19:39:38,709][105692] Updated weights for policy 0, policy_version 589803 (0.0007) [2023-12-26 19:39:39,275][105620] Updated weights for policy 1, policy_version 590569 (0.0010) [2023-12-26 19:39:39,327][105620] Updated weights for policy 1, policy_version 590579 (0.0008) [2023-12-26 19:39:39,340][105692] Updated weights for policy 0, policy_version 589813 (0.0009) [2023-12-26 19:39:39,393][105620] Updated weights for policy 1, policy_version 590589 (0.0008) [2023-12-26 19:39:39,404][105692] Updated weights for policy 0, policy_version 589823 (0.0010) [2023-12-26 19:39:39,455][105620] Updated weights for policy 1, policy_version 590599 (0.0007) [2023-12-26 19:39:39,458][105692] Updated weights for policy 0, policy_version 589833 (0.0011) [2023-12-26 19:39:40,195][105620] Updated weights for policy 1, policy_version 590609 (0.0010) [2023-12-26 19:39:40,250][105692] Updated weights for policy 0, policy_version 589843 (0.0011) [2023-12-26 19:39:40,254][105620] Updated weights for policy 1, policy_version 590619 (0.0011) [2023-12-26 19:39:40,302][105692] Updated weights for policy 0, policy_version 589853 (0.0010) [2023-12-26 19:39:40,304][105620] Updated weights for policy 1, policy_version 590629 (0.0011) [2023-12-26 19:39:40,360][105692] Updated weights for policy 0, policy_version 589863 (0.0010) [2023-12-26 19:39:41,020][105692] Updated weights for policy 0, policy_version 589873 (0.0010) [2023-12-26 19:39:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 302243840. Throughput: 0: 10076.7, 1: 9755.1. Samples: 302257328. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:39:41,063][104569] Avg episode reward: [(0, '8540.269'), (1, '9072.726')] [2023-12-26 19:39:41,064][105620] Updated weights for policy 1, policy_version 590639 (0.0009) [2023-12-26 19:39:41,085][105692] Updated weights for policy 0, policy_version 589883 (0.0008) [2023-12-26 19:39:41,130][105620] Updated weights for policy 1, policy_version 590649 (0.0007) [2023-12-26 19:39:41,146][105692] Updated weights for policy 0, policy_version 589893 (0.0009) [2023-12-26 19:39:41,193][105620] Updated weights for policy 1, policy_version 590659 (0.0007) [2023-12-26 19:39:41,207][105692] Updated weights for policy 0, policy_version 589903 (0.0007) [2023-12-26 19:39:41,937][105692] Updated weights for policy 0, policy_version 589913 (0.0009) [2023-12-26 19:39:41,996][105692] Updated weights for policy 0, policy_version 589923 (0.0008) [2023-12-26 19:39:41,997][105620] Updated weights for policy 1, policy_version 590669 (0.0010) [2023-12-26 19:39:42,055][105620] Updated weights for policy 1, policy_version 590679 (0.0009) [2023-12-26 19:39:42,059][105692] Updated weights for policy 0, policy_version 589933 (0.0007) [2023-12-26 19:39:42,113][105620] Updated weights for policy 1, policy_version 590689 (0.0009) [2023-12-26 19:39:42,806][105692] Updated weights for policy 0, policy_version 589943 (0.0008) [2023-12-26 19:39:42,860][105692] Updated weights for policy 0, policy_version 589954 (0.0009) [2023-12-26 19:39:42,861][105620] Updated weights for policy 1, policy_version 590699 (0.0008) [2023-12-26 19:39:42,909][105692] Updated weights for policy 0, policy_version 589964 (0.0006) [2023-12-26 19:39:42,911][105620] Updated weights for policy 1, policy_version 590709 (0.0007) [2023-12-26 19:39:42,966][105620] Updated weights for policy 1, policy_version 590719 (0.0009) [2023-12-26 19:39:43,623][105620] Updated weights for policy 1, policy_version 590729 (0.0008) [2023-12-26 19:39:43,687][105620] Updated weights for policy 1, policy_version 590739 (0.0005) [2023-12-26 19:39:43,696][105692] Updated weights for policy 0, policy_version 589974 (0.0008) [2023-12-26 19:39:43,742][105620] Updated weights for policy 1, policy_version 590749 (0.0006) [2023-12-26 19:39:43,757][105692] Updated weights for policy 0, policy_version 589984 (0.0008) [2023-12-26 19:39:43,809][105620] Updated weights for policy 1, policy_version 590759 (0.0006) [2023-12-26 19:39:43,812][105692] Updated weights for policy 0, policy_version 589994 (0.0008) [2023-12-26 19:39:44,383][105620] Updated weights for policy 1, policy_version 590769 (0.0007) [2023-12-26 19:39:44,447][105620] Updated weights for policy 1, policy_version 590779 (0.0009) [2023-12-26 19:39:44,481][105692] Updated weights for policy 0, policy_version 590004 (0.0008) [2023-12-26 19:39:44,499][105620] Updated weights for policy 1, policy_version 590789 (0.0007) [2023-12-26 19:39:44,535][105692] Updated weights for policy 0, policy_version 590014 (0.0008) [2023-12-26 19:39:44,591][105692] Updated weights for policy 0, policy_version 590024 (0.0009) [2023-12-26 19:39:45,288][105620] Updated weights for policy 1, policy_version 590799 (0.0008) [2023-12-26 19:39:45,298][105692] Updated weights for policy 0, policy_version 590034 (0.0009) [2023-12-26 19:39:45,342][105620] Updated weights for policy 1, policy_version 590809 (0.0007) [2023-12-26 19:39:45,348][105692] Updated weights for policy 0, policy_version 590044 (0.0006) [2023-12-26 19:39:45,398][105692] Updated weights for policy 0, policy_version 590054 (0.0006) [2023-12-26 19:39:45,404][105620] Updated weights for policy 1, policy_version 590819 (0.0008) [2023-12-26 19:39:45,450][105692] Updated weights for policy 0, policy_version 590064 (0.0006) [2023-12-26 19:39:46,055][105692] Updated weights for policy 0, policy_version 590074 (0.0009) [2023-12-26 19:39:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 302342144. Throughput: 0: 9971.4, 1: 9701.2. Samples: 302315408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:39:46,063][104569] Avg episode reward: [(0, '8990.800'), (1, '9081.981')] [2023-12-26 19:39:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000590824_151265280.pth... [2023-12-26 19:39:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000589704_150978560.pth [2023-12-26 19:39:46,103][105692] Updated weights for policy 0, policy_version 590084 (0.0009) [2023-12-26 19:39:46,154][105692] Updated weights for policy 0, policy_version 590094 (0.0009) [2023-12-26 19:39:46,163][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000590096_151085056.pth... [2023-12-26 19:39:46,166][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000588912_150781952.pth [2023-12-26 19:39:46,215][105620] Updated weights for policy 1, policy_version 590829 (0.0009) [2023-12-26 19:39:46,275][105620] Updated weights for policy 1, policy_version 590839 (0.0008) [2023-12-26 19:39:46,322][105620] Updated weights for policy 1, policy_version 590849 (0.0005) [2023-12-26 19:39:46,914][105620] Updated weights for policy 1, policy_version 590859 (0.0006) [2023-12-26 19:39:46,964][105620] Updated weights for policy 1, policy_version 590869 (0.0009) [2023-12-26 19:39:47,004][105692] Updated weights for policy 0, policy_version 590104 (0.0006) [2023-12-26 19:39:47,018][105620] Updated weights for policy 1, policy_version 590879 (0.0007) [2023-12-26 19:39:47,050][105692] Updated weights for policy 0, policy_version 590114 (0.0006) [2023-12-26 19:39:47,103][105692] Updated weights for policy 0, policy_version 590124 (0.0008) [2023-12-26 19:39:47,685][105620] Updated weights for policy 1, policy_version 590889 (0.0008) [2023-12-26 19:39:47,746][105620] Updated weights for policy 1, policy_version 590899 (0.0010) [2023-12-26 19:39:47,804][105620] Updated weights for policy 1, policy_version 590909 (0.0010) [2023-12-26 19:39:47,855][105620] Updated weights for policy 1, policy_version 590919 (0.0010) [2023-12-26 19:39:47,926][105692] Updated weights for policy 0, policy_version 590134 (0.0009) [2023-12-26 19:39:47,974][105692] Updated weights for policy 0, policy_version 590144 (0.0007) [2023-12-26 19:39:48,019][105692] Updated weights for policy 0, policy_version 590154 (0.0008) [2023-12-26 19:39:48,563][105620] Updated weights for policy 1, policy_version 590929 (0.0009) [2023-12-26 19:39:48,612][105620] Updated weights for policy 1, policy_version 590939 (0.0010) [2023-12-26 19:39:48,664][105620] Updated weights for policy 1, policy_version 590949 (0.0010) [2023-12-26 19:39:48,801][105692] Updated weights for policy 0, policy_version 590164 (0.0008) [2023-12-26 19:39:48,855][105692] Updated weights for policy 0, policy_version 590174 (0.0009) [2023-12-26 19:39:48,910][105692] Updated weights for policy 0, policy_version 590184 (0.0009) [2023-12-26 19:39:49,389][105620] Updated weights for policy 1, policy_version 590959 (0.0009) [2023-12-26 19:39:49,455][105620] Updated weights for policy 1, policy_version 590969 (0.0009) [2023-12-26 19:39:49,516][105620] Updated weights for policy 1, policy_version 590979 (0.0009) [2023-12-26 19:39:49,712][105692] Updated weights for policy 0, policy_version 590194 (0.0009) [2023-12-26 19:39:49,768][105692] Updated weights for policy 0, policy_version 590204 (0.0007) [2023-12-26 19:39:49,827][105692] Updated weights for policy 0, policy_version 590214 (0.0009) [2023-12-26 19:39:49,887][105692] Updated weights for policy 0, policy_version 590224 (0.0009) [2023-12-26 19:39:50,159][105620] Updated weights for policy 1, policy_version 590989 (0.0007) [2023-12-26 19:39:50,217][105620] Updated weights for policy 1, policy_version 590999 (0.0009) [2023-12-26 19:39:50,273][105620] Updated weights for policy 1, policy_version 591009 (0.0010) [2023-12-26 19:39:50,731][105692] Updated weights for policy 0, policy_version 590234 (0.0008) [2023-12-26 19:39:50,798][105692] Updated weights for policy 0, policy_version 590244 (0.0008) [2023-12-26 19:39:50,846][105692] Updated weights for policy 0, policy_version 590254 (0.0008) [2023-12-26 19:39:50,951][105620] Updated weights for policy 1, policy_version 591019 (0.0008) [2023-12-26 19:39:51,014][105620] Updated weights for policy 1, policy_version 591029 (0.0011) [2023-12-26 19:39:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 302440448. Throughput: 0: 9952.7, 1: 9741.8. Samples: 302432332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:39:51,062][104569] Avg episode reward: [(0, '9083.553'), (1, '8811.617')] [2023-12-26 19:39:51,081][105620] Updated weights for policy 1, policy_version 591039 (0.0011) [2023-12-26 19:39:51,644][105692] Updated weights for policy 0, policy_version 590264 (0.0007) [2023-12-26 19:39:51,709][105692] Updated weights for policy 0, policy_version 590274 (0.0007) [2023-12-26 19:39:51,771][105692] Updated weights for policy 0, policy_version 590284 (0.0007) [2023-12-26 19:39:51,841][105620] Updated weights for policy 1, policy_version 591049 (0.0010) [2023-12-26 19:39:51,893][105620] Updated weights for policy 1, policy_version 591059 (0.0010) [2023-12-26 19:39:51,948][105620] Updated weights for policy 1, policy_version 591069 (0.0010) [2023-12-26 19:39:52,007][105620] Updated weights for policy 1, policy_version 591079 (0.0010) [2023-12-26 19:39:52,524][105692] Updated weights for policy 0, policy_version 590294 (0.0007) [2023-12-26 19:39:52,587][105692] Updated weights for policy 0, policy_version 590304 (0.0008) [2023-12-26 19:39:52,647][105692] Updated weights for policy 0, policy_version 590314 (0.0008) [2023-12-26 19:39:52,651][105585] KL-divergence is very high: 110.1259 [2023-12-26 19:39:52,814][105620] Updated weights for policy 1, policy_version 591089 (0.0010) [2023-12-26 19:39:52,869][105620] Updated weights for policy 1, policy_version 591099 (0.0010) [2023-12-26 19:39:52,918][105620] Updated weights for policy 1, policy_version 591109 (0.0010) [2023-12-26 19:39:53,497][105692] Updated weights for policy 0, policy_version 590324 (0.0008) [2023-12-26 19:39:53,523][105620] Updated weights for policy 1, policy_version 591119 (0.0010) [2023-12-26 19:39:53,550][105692] Updated weights for policy 0, policy_version 590334 (0.0006) [2023-12-26 19:39:53,586][105620] Updated weights for policy 1, policy_version 591129 (0.0011) [2023-12-26 19:39:53,605][105692] Updated weights for policy 0, policy_version 590344 (0.0007) [2023-12-26 19:39:53,634][105620] Updated weights for policy 1, policy_version 591139 (0.0007) [2023-12-26 19:39:54,199][105620] Updated weights for policy 1, policy_version 591149 (0.0008) [2023-12-26 19:39:54,247][105620] Updated weights for policy 1, policy_version 591159 (0.0010) [2023-12-26 19:39:54,304][105620] Updated weights for policy 1, policy_version 591169 (0.0010) [2023-12-26 19:39:54,392][105692] Updated weights for policy 0, policy_version 590354 (0.0008) [2023-12-26 19:39:54,453][105692] Updated weights for policy 0, policy_version 590364 (0.0005) [2023-12-26 19:39:54,512][105692] Updated weights for policy 0, policy_version 590374 (0.0008) [2023-12-26 19:39:54,569][105692] Updated weights for policy 0, policy_version 590384 (0.0009) [2023-12-26 19:39:54,905][105620] Updated weights for policy 1, policy_version 591179 (0.0006) [2023-12-26 19:39:54,963][105620] Updated weights for policy 1, policy_version 591189 (0.0006) [2023-12-26 19:39:55,020][105620] Updated weights for policy 1, policy_version 591199 (0.0007) [2023-12-26 19:39:55,304][105692] Updated weights for policy 0, policy_version 590394 (0.0005) [2023-12-26 19:39:55,364][105692] Updated weights for policy 0, policy_version 590404 (0.0005) [2023-12-26 19:39:55,418][105692] Updated weights for policy 0, policy_version 590414 (0.0005) [2023-12-26 19:39:55,606][105620] Updated weights for policy 1, policy_version 591209 (0.0008) [2023-12-26 19:39:55,664][105620] Updated weights for policy 1, policy_version 591219 (0.0005) [2023-12-26 19:39:55,710][105620] Updated weights for policy 1, policy_version 591229 (0.0005) [2023-12-26 19:39:55,754][105620] Updated weights for policy 1, policy_version 591239 (0.0005) [2023-12-26 19:39:56,029][105692] Updated weights for policy 0, policy_version 590424 (0.0007) [2023-12-26 19:39:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 302538752. Throughput: 0: 9889.1, 1: 9749.1. Samples: 302550392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:39:56,062][104569] Avg episode reward: [(0, '9267.423'), (1, '8995.925')] [2023-12-26 19:39:56,098][105692] Updated weights for policy 0, policy_version 590434 (0.0005) [2023-12-26 19:39:56,158][105692] Updated weights for policy 0, policy_version 590444 (0.0008) [2023-12-26 19:39:56,367][105620] Updated weights for policy 1, policy_version 591249 (0.0005) [2023-12-26 19:39:56,420][105620] Updated weights for policy 1, policy_version 591259 (0.0005) [2023-12-26 19:39:56,474][105620] Updated weights for policy 1, policy_version 591269 (0.0006) [2023-12-26 19:39:56,909][105692] Updated weights for policy 0, policy_version 590454 (0.0008) [2023-12-26 19:39:56,963][105692] Updated weights for policy 0, policy_version 590464 (0.0009) [2023-12-26 19:39:57,018][105692] Updated weights for policy 0, policy_version 590474 (0.0008) [2023-12-26 19:39:57,054][105620] Updated weights for policy 1, policy_version 591279 (0.0006) [2023-12-26 19:39:57,099][105620] Updated weights for policy 1, policy_version 591289 (0.0006) [2023-12-26 19:39:57,145][105620] Updated weights for policy 1, policy_version 591299 (0.0007) [2023-12-26 19:39:57,808][105620] Updated weights for policy 1, policy_version 591309 (0.0008) [2023-12-26 19:39:57,821][105692] Updated weights for policy 0, policy_version 590484 (0.0009) [2023-12-26 19:39:57,866][105620] Updated weights for policy 1, policy_version 591319 (0.0010) [2023-12-26 19:39:57,866][105692] Updated weights for policy 0, policy_version 590494 (0.0006) [2023-12-26 19:39:57,914][105692] Updated weights for policy 0, policy_version 590504 (0.0009) [2023-12-26 19:39:57,918][105620] Updated weights for policy 1, policy_version 591329 (0.0010) [2023-12-26 19:39:58,683][105620] Updated weights for policy 1, policy_version 591339 (0.0010) [2023-12-26 19:39:58,750][105692] Updated weights for policy 0, policy_version 590514 (0.0006) [2023-12-26 19:39:58,752][105620] Updated weights for policy 1, policy_version 591349 (0.0010) [2023-12-26 19:39:58,815][105692] Updated weights for policy 0, policy_version 590524 (0.0007) [2023-12-26 19:39:58,820][105620] Updated weights for policy 1, policy_version 591359 (0.0008) [2023-12-26 19:39:58,877][105692] Updated weights for policy 0, policy_version 590534 (0.0006) [2023-12-26 19:39:58,937][105692] Updated weights for policy 0, policy_version 590544 (0.0008) [2023-12-26 19:39:59,554][105620] Updated weights for policy 1, policy_version 591369 (0.0010) [2023-12-26 19:39:59,620][105620] Updated weights for policy 1, policy_version 591379 (0.0006) [2023-12-26 19:39:59,666][105692] Updated weights for policy 0, policy_version 590554 (0.0009) [2023-12-26 19:39:59,679][105620] Updated weights for policy 1, policy_version 591389 (0.0006) [2023-12-26 19:39:59,714][105692] Updated weights for policy 0, policy_version 590564 (0.0009) [2023-12-26 19:39:59,729][105620] Updated weights for policy 1, policy_version 591399 (0.0008) [2023-12-26 19:39:59,770][105692] Updated weights for policy 0, policy_version 590574 (0.0008) [2023-12-26 19:40:00,487][105620] Updated weights for policy 1, policy_version 591409 (0.0009) [2023-12-26 19:40:00,509][105692] Updated weights for policy 0, policy_version 590584 (0.0007) [2023-12-26 19:40:00,546][105620] Updated weights for policy 1, policy_version 591419 (0.0008) [2023-12-26 19:40:00,565][105692] Updated weights for policy 0, policy_version 590594 (0.0005) [2023-12-26 19:40:00,606][105620] Updated weights for policy 1, policy_version 591429 (0.0007) [2023-12-26 19:40:00,625][105692] Updated weights for policy 0, policy_version 590604 (0.0006) [2023-12-26 19:40:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 302637056. Throughput: 0: 9796.1, 1: 9848.0. Samples: 302608584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:40:01,063][104569] Avg episode reward: [(0, '9085.145'), (1, '8997.704')] [2023-12-26 19:40:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000590608_151216128.pth... [2023-12-26 19:40:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000591432_151420928.pth... [2023-12-26 19:40:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000590248_151117824.pth [2023-12-26 19:40:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000589488_150929408.pth [2023-12-26 19:40:01,309][105692] Updated weights for policy 0, policy_version 590614 (0.0007) [2023-12-26 19:40:01,334][105620] Updated weights for policy 1, policy_version 591439 (0.0009) [2023-12-26 19:40:01,378][105692] Updated weights for policy 0, policy_version 590624 (0.0007) [2023-12-26 19:40:01,407][105620] Updated weights for policy 1, policy_version 591449 (0.0008) [2023-12-26 19:40:01,437][105692] Updated weights for policy 0, policy_version 590634 (0.0008) [2023-12-26 19:40:01,474][105620] Updated weights for policy 1, policy_version 591459 (0.0008) [2023-12-26 19:40:02,066][105692] Updated weights for policy 0, policy_version 590644 (0.0008) [2023-12-26 19:40:02,123][105692] Updated weights for policy 0, policy_version 590654 (0.0009) [2023-12-26 19:40:02,181][105692] Updated weights for policy 0, policy_version 590664 (0.0009) [2023-12-26 19:40:02,260][105620] Updated weights for policy 1, policy_version 591469 (0.0008) [2023-12-26 19:40:02,319][105620] Updated weights for policy 1, policy_version 591479 (0.0008) [2023-12-26 19:40:02,381][105620] Updated weights for policy 1, policy_version 591489 (0.0009) [2023-12-26 19:40:02,917][105692] Updated weights for policy 0, policy_version 590674 (0.0008) [2023-12-26 19:40:02,981][105692] Updated weights for policy 0, policy_version 590684 (0.0010) [2023-12-26 19:40:03,033][105692] Updated weights for policy 0, policy_version 590694 (0.0009) [2023-12-26 19:40:03,069][105620] Updated weights for policy 1, policy_version 591499 (0.0007) [2023-12-26 19:40:03,089][105692] Updated weights for policy 0, policy_version 590704 (0.0009) [2023-12-26 19:40:03,121][105620] Updated weights for policy 1, policy_version 591509 (0.0005) [2023-12-26 19:40:03,171][105620] Updated weights for policy 1, policy_version 591519 (0.0005) [2023-12-26 19:40:03,868][105620] Updated weights for policy 1, policy_version 591529 (0.0006) [2023-12-26 19:40:03,894][105692] Updated weights for policy 0, policy_version 590714 (0.0007) [2023-12-26 19:40:03,923][105620] Updated weights for policy 1, policy_version 591539 (0.0011) [2023-12-26 19:40:03,953][105692] Updated weights for policy 0, policy_version 590724 (0.0005) [2023-12-26 19:40:03,987][105620] Updated weights for policy 1, policy_version 591549 (0.0011) [2023-12-26 19:40:04,014][105692] Updated weights for policy 0, policy_version 590734 (0.0006) [2023-12-26 19:40:04,046][105620] Updated weights for policy 1, policy_version 591559 (0.0009) [2023-12-26 19:40:04,666][105620] Updated weights for policy 1, policy_version 591569 (0.0010) [2023-12-26 19:40:04,725][105620] Updated weights for policy 1, policy_version 591579 (0.0010) [2023-12-26 19:40:04,783][105620] Updated weights for policy 1, policy_version 591589 (0.0010) [2023-12-26 19:40:04,825][105692] Updated weights for policy 0, policy_version 590744 (0.0008) [2023-12-26 19:40:04,874][105692] Updated weights for policy 0, policy_version 590754 (0.0008) [2023-12-26 19:40:04,917][105692] Updated weights for policy 0, policy_version 590764 (0.0008) [2023-12-26 19:40:05,516][105620] Updated weights for policy 1, policy_version 591599 (0.0010) [2023-12-26 19:40:05,561][105620] Updated weights for policy 1, policy_version 591609 (0.0010) [2023-12-26 19:40:05,605][105620] Updated weights for policy 1, policy_version 591619 (0.0010) [2023-12-26 19:40:05,694][105692] Updated weights for policy 0, policy_version 590774 (0.0008) [2023-12-26 19:40:05,752][105692] Updated weights for policy 0, policy_version 590784 (0.0008) [2023-12-26 19:40:05,811][105692] Updated weights for policy 0, policy_version 590794 (0.0008) [2023-12-26 19:40:06,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 302735360. Throughput: 0: 9708.7, 1: 9887.8. Samples: 302723660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:40:06,063][104569] Avg episode reward: [(0, '8994.021'), (1, '8532.055')] [2023-12-26 19:40:06,416][105620] Updated weights for policy 1, policy_version 591629 (0.0008) [2023-12-26 19:40:06,479][105620] Updated weights for policy 1, policy_version 591639 (0.0010) [2023-12-26 19:40:06,548][105620] Updated weights for policy 1, policy_version 591649 (0.0010) [2023-12-26 19:40:06,590][105692] Updated weights for policy 0, policy_version 590804 (0.0007) [2023-12-26 19:40:06,649][105692] Updated weights for policy 0, policy_version 590814 (0.0008) [2023-12-26 19:40:06,707][105692] Updated weights for policy 0, policy_version 590824 (0.0008) [2023-12-26 19:40:07,290][105620] Updated weights for policy 1, policy_version 591659 (0.0009) [2023-12-26 19:40:07,349][105620] Updated weights for policy 1, policy_version 591669 (0.0011) [2023-12-26 19:40:07,412][105620] Updated weights for policy 1, policy_version 591679 (0.0011) [2023-12-26 19:40:07,474][105692] Updated weights for policy 0, policy_version 590834 (0.0008) [2023-12-26 19:40:07,535][105692] Updated weights for policy 0, policy_version 590844 (0.0005) [2023-12-26 19:40:07,596][105692] Updated weights for policy 0, policy_version 590854 (0.0006) [2023-12-26 19:40:07,654][105692] Updated weights for policy 0, policy_version 590864 (0.0010) [2023-12-26 19:40:08,185][105620] Updated weights for policy 1, policy_version 591689 (0.0010) [2023-12-26 19:40:08,249][105620] Updated weights for policy 1, policy_version 591699 (0.0008) [2023-12-26 19:40:08,310][105620] Updated weights for policy 1, policy_version 591709 (0.0008) [2023-12-26 19:40:08,372][105620] Updated weights for policy 1, policy_version 591719 (0.0010) [2023-12-26 19:40:08,384][105692] Updated weights for policy 0, policy_version 590874 (0.0011) [2023-12-26 19:40:08,443][105692] Updated weights for policy 0, policy_version 590884 (0.0010) [2023-12-26 19:40:08,504][105692] Updated weights for policy 0, policy_version 590894 (0.0009) [2023-12-26 19:40:09,094][105620] Updated weights for policy 1, policy_version 591729 (0.0009) [2023-12-26 19:40:09,102][105692] Updated weights for policy 0, policy_version 590904 (0.0005) [2023-12-26 19:40:09,139][105620] Updated weights for policy 1, policy_version 591739 (0.0010) [2023-12-26 19:40:09,162][105692] Updated weights for policy 0, policy_version 590914 (0.0005) [2023-12-26 19:40:09,195][105620] Updated weights for policy 1, policy_version 591749 (0.0010) [2023-12-26 19:40:09,225][105692] Updated weights for policy 0, policy_version 590924 (0.0006) [2023-12-26 19:40:09,886][105692] Updated weights for policy 0, policy_version 590934 (0.0009) [2023-12-26 19:40:09,953][105692] Updated weights for policy 0, policy_version 590944 (0.0011) [2023-12-26 19:40:09,994][105620] Updated weights for policy 1, policy_version 591759 (0.0009) [2023-12-26 19:40:10,009][105692] Updated weights for policy 0, policy_version 590954 (0.0010) [2023-12-26 19:40:10,048][105620] Updated weights for policy 1, policy_version 591769 (0.0007) [2023-12-26 19:40:10,112][105620] Updated weights for policy 1, policy_version 591779 (0.0008) [2023-12-26 19:40:10,751][105692] Updated weights for policy 0, policy_version 590964 (0.0008) [2023-12-26 19:40:10,816][105692] Updated weights for policy 0, policy_version 590974 (0.0005) [2023-12-26 19:40:10,880][105692] Updated weights for policy 0, policy_version 590984 (0.0006) [2023-12-26 19:40:10,903][105620] Updated weights for policy 1, policy_version 591789 (0.0009) [2023-12-26 19:40:10,953][105620] Updated weights for policy 1, policy_version 591799 (0.0010) [2023-12-26 19:40:11,006][105620] Updated weights for policy 1, policy_version 591809 (0.0011) [2023-12-26 19:40:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 302833664. Throughput: 0: 9711.1, 1: 9855.9. Samples: 302836692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:40:11,063][104569] Avg episode reward: [(0, '9176.262'), (1, '7801.880')] [2023-12-26 19:40:11,637][105692] Updated weights for policy 0, policy_version 590994 (0.0007) [2023-12-26 19:40:11,702][105692] Updated weights for policy 0, policy_version 591004 (0.0008) [2023-12-26 19:40:11,766][105692] Updated weights for policy 0, policy_version 591014 (0.0009) [2023-12-26 19:40:11,768][105620] Updated weights for policy 1, policy_version 591819 (0.0010) [2023-12-26 19:40:11,816][105692] Updated weights for policy 0, policy_version 591024 (0.0006) [2023-12-26 19:40:11,818][105620] Updated weights for policy 1, policy_version 591829 (0.0007) [2023-12-26 19:40:11,874][105620] Updated weights for policy 1, policy_version 591839 (0.0009) [2023-12-26 19:40:12,547][105692] Updated weights for policy 0, policy_version 591034 (0.0009) [2023-12-26 19:40:12,599][105692] Updated weights for policy 0, policy_version 591044 (0.0009) [2023-12-26 19:40:12,654][105692] Updated weights for policy 0, policy_version 591054 (0.0010) [2023-12-26 19:40:12,658][105620] Updated weights for policy 1, policy_version 591849 (0.0009) [2023-12-26 19:40:12,711][105620] Updated weights for policy 1, policy_version 591859 (0.0007) [2023-12-26 19:40:12,757][105620] Updated weights for policy 1, policy_version 591869 (0.0007) [2023-12-26 19:40:12,801][105620] Updated weights for policy 1, policy_version 591879 (0.0010) [2023-12-26 19:40:13,305][105692] Updated weights for policy 0, policy_version 591064 (0.0006) [2023-12-26 19:40:13,372][105692] Updated weights for policy 0, policy_version 591074 (0.0005) [2023-12-26 19:40:13,422][105692] Updated weights for policy 0, policy_version 591084 (0.0005) [2023-12-26 19:40:13,487][105620] Updated weights for policy 1, policy_version 591889 (0.0009) [2023-12-26 19:40:13,536][105620] Updated weights for policy 1, policy_version 591899 (0.0009) [2023-12-26 19:40:13,595][105620] Updated weights for policy 1, policy_version 591909 (0.0009) [2023-12-26 19:40:13,952][105692] Updated weights for policy 0, policy_version 591094 (0.0008) [2023-12-26 19:40:14,007][105692] Updated weights for policy 0, policy_version 591104 (0.0010) [2023-12-26 19:40:14,065][105692] Updated weights for policy 0, policy_version 591114 (0.0010) [2023-12-26 19:40:14,197][105620] Updated weights for policy 1, policy_version 591919 (0.0009) [2023-12-26 19:40:14,270][105620] Updated weights for policy 1, policy_version 591929 (0.0006) [2023-12-26 19:40:14,330][105620] Updated weights for policy 1, policy_version 591939 (0.0005) [2023-12-26 19:40:14,730][105692] Updated weights for policy 0, policy_version 591124 (0.0010) [2023-12-26 19:40:14,795][105692] Updated weights for policy 0, policy_version 591134 (0.0011) [2023-12-26 19:40:14,855][105692] Updated weights for policy 0, policy_version 591144 (0.0011) [2023-12-26 19:40:14,988][105620] Updated weights for policy 1, policy_version 591949 (0.0008) [2023-12-26 19:40:15,039][105620] Updated weights for policy 1, policy_version 591959 (0.0010) [2023-12-26 19:40:15,095][105620] Updated weights for policy 1, policy_version 591969 (0.0010) [2023-12-26 19:40:15,597][105692] Updated weights for policy 0, policy_version 591154 (0.0010) [2023-12-26 19:40:15,641][105692] Updated weights for policy 0, policy_version 591164 (0.0010) [2023-12-26 19:40:15,688][105692] Updated weights for policy 0, policy_version 591174 (0.0010) [2023-12-26 19:40:15,742][105692] Updated weights for policy 0, policy_version 591184 (0.0010) [2023-12-26 19:40:15,813][105620] Updated weights for policy 1, policy_version 591979 (0.0009) [2023-12-26 19:40:15,869][105620] Updated weights for policy 1, policy_version 591989 (0.0005) [2023-12-26 19:40:15,922][105620] Updated weights for policy 1, policy_version 591999 (0.0005) [2023-12-26 19:40:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 302931968. Throughput: 0: 9687.3, 1: 9891.1. Samples: 302896628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:40:16,063][104569] Avg episode reward: [(0, '9358.461'), (1, '8472.146')] [2023-12-26 19:40:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000591184_151363584.pth... [2023-12-26 19:40:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000592008_151568384.pth... [2023-12-26 19:40:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000590096_151085056.pth [2023-12-26 19:40:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000590824_151265280.pth [2023-12-26 19:40:16,486][105620] Updated weights for policy 1, policy_version 592009 (0.0006) [2023-12-26 19:40:16,512][105692] Updated weights for policy 0, policy_version 591194 (0.0010) [2023-12-26 19:40:16,542][105620] Updated weights for policy 1, policy_version 592019 (0.0006) [2023-12-26 19:40:16,575][105692] Updated weights for policy 0, policy_version 591204 (0.0010) [2023-12-26 19:40:16,600][105620] Updated weights for policy 1, policy_version 592029 (0.0005) [2023-12-26 19:40:16,629][105692] Updated weights for policy 0, policy_version 591214 (0.0010) [2023-12-26 19:40:16,652][105620] Updated weights for policy 1, policy_version 592039 (0.0006) [2023-12-26 19:40:17,294][105692] Updated weights for policy 0, policy_version 591224 (0.0010) [2023-12-26 19:40:17,352][105692] Updated weights for policy 0, policy_version 591234 (0.0010) [2023-12-26 19:40:17,389][105620] Updated weights for policy 1, policy_version 592049 (0.0008) [2023-12-26 19:40:17,407][105692] Updated weights for policy 0, policy_version 591244 (0.0010) [2023-12-26 19:40:17,453][105620] Updated weights for policy 1, policy_version 592059 (0.0008) [2023-12-26 19:40:17,516][105620] Updated weights for policy 1, policy_version 592069 (0.0008) [2023-12-26 19:40:18,073][105620] Updated weights for policy 1, policy_version 592079 (0.0006) [2023-12-26 19:40:18,127][105620] Updated weights for policy 1, policy_version 592089 (0.0005) [2023-12-26 19:40:18,127][105692] Updated weights for policy 0, policy_version 591254 (0.0010) [2023-12-26 19:40:18,179][105692] Updated weights for policy 0, policy_version 591264 (0.0010) [2023-12-26 19:40:18,183][105620] Updated weights for policy 1, policy_version 592099 (0.0005) [2023-12-26 19:40:18,232][105692] Updated weights for policy 0, policy_version 591274 (0.0006) [2023-12-26 19:40:18,834][105620] Updated weights for policy 1, policy_version 592109 (0.0007) [2023-12-26 19:40:18,898][105620] Updated weights for policy 1, policy_version 592119 (0.0008) [2023-12-26 19:40:18,940][105692] Updated weights for policy 0, policy_version 591284 (0.0007) [2023-12-26 19:40:18,958][105620] Updated weights for policy 1, policy_version 592129 (0.0007) [2023-12-26 19:40:18,989][105692] Updated weights for policy 0, policy_version 591294 (0.0010) [2023-12-26 19:40:19,041][105692] Updated weights for policy 0, policy_version 591304 (0.0010) [2023-12-26 19:40:19,723][105620] Updated weights for policy 1, policy_version 592139 (0.0006) [2023-12-26 19:40:19,780][105620] Updated weights for policy 1, policy_version 592149 (0.0008) [2023-12-26 19:40:19,825][105692] Updated weights for policy 0, policy_version 591314 (0.0010) [2023-12-26 19:40:19,847][105620] Updated weights for policy 1, policy_version 592159 (0.0008) [2023-12-26 19:40:19,883][105692] Updated weights for policy 0, policy_version 591324 (0.0011) [2023-12-26 19:40:19,947][105692] Updated weights for policy 0, policy_version 591334 (0.0011) [2023-12-26 19:40:20,011][105692] Updated weights for policy 0, policy_version 591344 (0.0011) [2023-12-26 19:40:20,645][105620] Updated weights for policy 1, policy_version 592169 (0.0007) [2023-12-26 19:40:20,712][105620] Updated weights for policy 1, policy_version 592179 (0.0008) [2023-12-26 19:40:20,781][105620] Updated weights for policy 1, policy_version 592189 (0.0007) [2023-12-26 19:40:20,784][105692] Updated weights for policy 0, policy_version 591354 (0.0011) [2023-12-26 19:40:20,845][105620] Updated weights for policy 1, policy_version 592199 (0.0006) [2023-12-26 19:40:20,847][105692] Updated weights for policy 0, policy_version 591364 (0.0011) [2023-12-26 19:40:20,910][105692] Updated weights for policy 0, policy_version 591374 (0.0011) [2023-12-26 19:40:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 303030272. Throughput: 0: 9627.4, 1: 9959.5. Samples: 303017664. Policy #0 lag: (min: 31.0, avg: 31.7, max: 53.0) [2023-12-26 19:40:21,063][104569] Avg episode reward: [(0, '9267.857'), (1, '8731.562')] [2023-12-26 19:40:21,633][105620] Updated weights for policy 1, policy_version 592209 (0.0008) [2023-12-26 19:40:21,655][105692] Updated weights for policy 0, policy_version 591384 (0.0009) [2023-12-26 19:40:21,702][105692] Updated weights for policy 0, policy_version 591394 (0.0006) [2023-12-26 19:40:21,705][105620] Updated weights for policy 1, policy_version 592219 (0.0007) [2023-12-26 19:40:21,764][105692] Updated weights for policy 0, policy_version 591404 (0.0008) [2023-12-26 19:40:21,768][105620] Updated weights for policy 1, policy_version 592229 (0.0008) [2023-12-26 19:40:22,431][105692] Updated weights for policy 0, policy_version 591414 (0.0009) [2023-12-26 19:40:22,457][105585] KL-divergence is very high: 146.0265 [2023-12-26 19:40:22,487][105692] Updated weights for policy 0, policy_version 591424 (0.0011) [2023-12-26 19:40:22,505][105585] KL-divergence is very high: 162.6297 [2023-12-26 19:40:22,532][105620] Updated weights for policy 1, policy_version 592239 (0.0007) [2023-12-26 19:40:22,547][105692] Updated weights for policy 0, policy_version 591434 (0.0011) [2023-12-26 19:40:22,588][105620] Updated weights for policy 1, policy_version 592249 (0.0006) [2023-12-26 19:40:22,655][105620] Updated weights for policy 1, policy_version 592259 (0.0010) [2023-12-26 19:40:23,173][105692] Updated weights for policy 0, policy_version 591444 (0.0008) [2023-12-26 19:40:23,235][105692] Updated weights for policy 0, policy_version 591454 (0.0007) [2023-12-26 19:40:23,296][105692] Updated weights for policy 0, policy_version 591464 (0.0010) [2023-12-26 19:40:23,480][105620] Updated weights for policy 1, policy_version 592269 (0.0009) [2023-12-26 19:40:23,538][105620] Updated weights for policy 1, policy_version 592279 (0.0008) [2023-12-26 19:40:23,587][105620] Updated weights for policy 1, policy_version 592289 (0.0008) [2023-12-26 19:40:23,951][105692] Updated weights for policy 0, policy_version 591474 (0.0011) [2023-12-26 19:40:24,008][105692] Updated weights for policy 0, policy_version 591484 (0.0011) [2023-12-26 19:40:24,066][105692] Updated weights for policy 0, policy_version 591494 (0.0010) [2023-12-26 19:40:24,119][105692] Updated weights for policy 0, policy_version 591504 (0.0011) [2023-12-26 19:40:24,387][105620] Updated weights for policy 1, policy_version 592299 (0.0008) [2023-12-26 19:40:24,449][105620] Updated weights for policy 1, policy_version 592309 (0.0008) [2023-12-26 19:40:24,502][105620] Updated weights for policy 1, policy_version 592319 (0.0008) [2023-12-26 19:40:24,865][105692] Updated weights for policy 0, policy_version 591514 (0.0010) [2023-12-26 19:40:24,868][105585] KL-divergence is very high: 106.8144 [2023-12-26 19:40:24,878][105585] KL-divergence is very high: 107.1384 [2023-12-26 19:40:24,909][105692] Updated weights for policy 0, policy_version 591524 (0.0010) [2023-12-26 19:40:24,960][105692] Updated weights for policy 0, policy_version 591534 (0.0010) [2023-12-26 19:40:25,231][105620] Updated weights for policy 1, policy_version 592329 (0.0008) [2023-12-26 19:40:25,289][105620] Updated weights for policy 1, policy_version 592339 (0.0008) [2023-12-26 19:40:25,343][105620] Updated weights for policy 1, policy_version 592349 (0.0006) [2023-12-26 19:40:25,394][105620] Updated weights for policy 1, policy_version 592359 (0.0007) [2023-12-26 19:40:25,705][105692] Updated weights for policy 0, policy_version 591544 (0.0010) [2023-12-26 19:40:25,760][105692] Updated weights for policy 0, policy_version 591554 (0.0010) [2023-12-26 19:40:25,810][105692] Updated weights for policy 0, policy_version 591564 (0.0010) [2023-12-26 19:40:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 303120384. Throughput: 0: 9565.9, 1: 9854.0. Samples: 303131220. Policy #0 lag: (min: 31.0, avg: 31.7, max: 53.0) [2023-12-26 19:40:26,062][104569] Avg episode reward: [(0, '8456.293'), (1, '8647.357')] [2023-12-26 19:40:26,074][105620] Updated weights for policy 1, policy_version 592369 (0.0007) [2023-12-26 19:40:26,130][105620] Updated weights for policy 1, policy_version 592379 (0.0008) [2023-12-26 19:40:26,182][105620] Updated weights for policy 1, policy_version 592389 (0.0008) [2023-12-26 19:40:26,539][105692] Updated weights for policy 0, policy_version 591574 (0.0010) [2023-12-26 19:40:26,596][105692] Updated weights for policy 0, policy_version 591584 (0.0010) [2023-12-26 19:40:26,657][105692] Updated weights for policy 0, policy_version 591594 (0.0010) [2023-12-26 19:40:26,967][105620] Updated weights for policy 1, policy_version 592399 (0.0008) [2023-12-26 19:40:27,019][105620] Updated weights for policy 1, policy_version 592409 (0.0008) [2023-12-26 19:40:27,081][105620] Updated weights for policy 1, policy_version 592419 (0.0008) [2023-12-26 19:40:27,384][105692] Updated weights for policy 0, policy_version 591604 (0.0010) [2023-12-26 19:40:27,428][105692] Updated weights for policy 0, policy_version 591614 (0.0010) [2023-12-26 19:40:27,475][105692] Updated weights for policy 0, policy_version 591624 (0.0010) [2023-12-26 19:40:27,813][105620] Updated weights for policy 1, policy_version 592429 (0.0008) [2023-12-26 19:40:27,861][105620] Updated weights for policy 1, policy_version 592439 (0.0007) [2023-12-26 19:40:27,904][105620] Updated weights for policy 1, policy_version 592449 (0.0008) [2023-12-26 19:40:28,240][105692] Updated weights for policy 0, policy_version 591634 (0.0010) [2023-12-26 19:40:28,284][105692] Updated weights for policy 0, policy_version 591644 (0.0010) [2023-12-26 19:40:28,336][105692] Updated weights for policy 0, policy_version 591654 (0.0010) [2023-12-26 19:40:28,395][105692] Updated weights for policy 0, policy_version 591664 (0.0009) [2023-12-26 19:40:28,687][105620] Updated weights for policy 1, policy_version 592459 (0.0008) [2023-12-26 19:40:28,735][105620] Updated weights for policy 1, policy_version 592469 (0.0008) [2023-12-26 19:40:28,784][105620] Updated weights for policy 1, policy_version 592479 (0.0008) [2023-12-26 19:40:29,163][105692] Updated weights for policy 0, policy_version 591674 (0.0010) [2023-12-26 19:40:29,225][105692] Updated weights for policy 0, policy_version 591684 (0.0009) [2023-12-26 19:40:29,292][105692] Updated weights for policy 0, policy_version 591694 (0.0006) [2023-12-26 19:40:29,605][105620] Updated weights for policy 1, policy_version 592489 (0.0008) [2023-12-26 19:40:29,671][105620] Updated weights for policy 1, policy_version 592499 (0.0008) [2023-12-26 19:40:29,734][105620] Updated weights for policy 1, policy_version 592509 (0.0007) [2023-12-26 19:40:29,788][105620] Updated weights for policy 1, policy_version 592519 (0.0008) [2023-12-26 19:40:30,022][105692] Updated weights for policy 0, policy_version 591704 (0.0009) [2023-12-26 19:40:30,074][105692] Updated weights for policy 0, policy_version 591714 (0.0010) [2023-12-26 19:40:30,135][105692] Updated weights for policy 0, policy_version 591724 (0.0010) [2023-12-26 19:40:30,554][105620] Updated weights for policy 1, policy_version 592529 (0.0008) [2023-12-26 19:40:30,608][105620] Updated weights for policy 1, policy_version 592539 (0.0008) [2023-12-26 19:40:30,652][105620] Updated weights for policy 1, policy_version 592549 (0.0007) [2023-12-26 19:40:30,880][105692] Updated weights for policy 0, policy_version 591734 (0.0010) [2023-12-26 19:40:30,928][105692] Updated weights for policy 0, policy_version 591744 (0.0010) [2023-12-26 19:40:30,971][105692] Updated weights for policy 0, policy_version 591754 (0.0010) [2023-12-26 19:40:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 303218688. Throughput: 0: 9550.2, 1: 9823.3. Samples: 303187212. Policy #0 lag: (min: 31.0, avg: 31.7, max: 53.0) [2023-12-26 19:40:31,062][104569] Avg episode reward: [(0, '7668.481'), (1, '6925.114')] [2023-12-26 19:40:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000591760_151511040.pth... [2023-12-26 19:40:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000592552_151707648.pth... [2023-12-26 19:40:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000591432_151420928.pth [2023-12-26 19:40:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000590608_151216128.pth [2023-12-26 19:40:31,289][105620] Updated weights for policy 1, policy_version 592559 (0.0009) [2023-12-26 19:40:31,339][105620] Updated weights for policy 1, policy_version 592569 (0.0009) [2023-12-26 19:40:31,404][105620] Updated weights for policy 1, policy_version 592579 (0.0008) [2023-12-26 19:40:31,757][105692] Updated weights for policy 0, policy_version 591764 (0.0010) [2023-12-26 19:40:31,818][105692] Updated weights for policy 0, policy_version 591774 (0.0009) [2023-12-26 19:40:31,842][105585] KL-divergence is very high: 324.7000 [2023-12-26 19:40:31,854][105585] KL-divergence is very high: 231.7686 [2023-12-26 19:40:31,879][105692] Updated weights for policy 0, policy_version 591784 (0.0009) [2023-12-26 19:40:31,890][105585] KL-divergence is very high: 449.5131 [2023-12-26 19:40:31,902][105585] KL-divergence is very high: 267.5485 [2023-12-26 19:40:32,156][105620] Updated weights for policy 1, policy_version 592589 (0.0007) [2023-12-26 19:40:32,204][105620] Updated weights for policy 1, policy_version 592599 (0.0006) [2023-12-26 19:40:32,263][105620] Updated weights for policy 1, policy_version 592609 (0.0007) [2023-12-26 19:40:32,645][105692] Updated weights for policy 0, policy_version 591794 (0.0008) [2023-12-26 19:40:32,695][105692] Updated weights for policy 0, policy_version 591804 (0.0005) [2023-12-26 19:40:32,752][105692] Updated weights for policy 0, policy_version 591814 (0.0005) [2023-12-26 19:40:32,805][105692] Updated weights for policy 0, policy_version 591824 (0.0005) [2023-12-26 19:40:32,962][105620] Updated weights for policy 1, policy_version 592619 (0.0008) [2023-12-26 19:40:33,010][105620] Updated weights for policy 1, policy_version 592629 (0.0005) [2023-12-26 19:40:33,062][105620] Updated weights for policy 1, policy_version 592639 (0.0008) [2023-12-26 19:40:33,347][105692] Updated weights for policy 0, policy_version 591834 (0.0009) [2023-12-26 19:40:33,403][105692] Updated weights for policy 0, policy_version 591845 (0.0009) [2023-12-26 19:40:33,463][105692] Updated weights for policy 0, policy_version 591855 (0.0010) [2023-12-26 19:40:33,616][105620] Updated weights for policy 1, policy_version 592649 (0.0007) [2023-12-26 19:40:33,672][105620] Updated weights for policy 1, policy_version 592659 (0.0005) [2023-12-26 19:40:33,721][105620] Updated weights for policy 1, policy_version 592669 (0.0005) [2023-12-26 19:40:33,769][105620] Updated weights for policy 1, policy_version 592679 (0.0005) [2023-12-26 19:40:34,207][105692] Updated weights for policy 0, policy_version 591865 (0.0008) [2023-12-26 19:40:34,254][105692] Updated weights for policy 0, policy_version 591875 (0.0008) [2023-12-26 19:40:34,316][105692] Updated weights for policy 0, policy_version 591885 (0.0008) [2023-12-26 19:40:34,400][105620] Updated weights for policy 1, policy_version 592689 (0.0008) [2023-12-26 19:40:34,467][105620] Updated weights for policy 1, policy_version 592699 (0.0009) [2023-12-26 19:40:34,529][105620] Updated weights for policy 1, policy_version 592709 (0.0009) [2023-12-26 19:40:35,086][105692] Updated weights for policy 0, policy_version 591895 (0.0009) [2023-12-26 19:40:35,158][105692] Updated weights for policy 0, policy_version 591905 (0.0010) [2023-12-26 19:40:35,220][105692] Updated weights for policy 0, policy_version 591915 (0.0008) [2023-12-26 19:40:35,239][105620] Updated weights for policy 1, policy_version 592719 (0.0007) [2023-12-26 19:40:35,293][105620] Updated weights for policy 1, policy_version 592729 (0.0006) [2023-12-26 19:40:35,346][105620] Updated weights for policy 1, policy_version 592739 (0.0005) [2023-12-26 19:40:35,961][105620] Updated weights for policy 1, policy_version 592749 (0.0007) [2023-12-26 19:40:36,012][105620] Updated weights for policy 1, policy_version 592759 (0.0005) [2023-12-26 19:40:36,053][105692] Updated weights for policy 0, policy_version 591925 (0.0007) [2023-12-26 19:40:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 303308800. Throughput: 0: 9570.1, 1: 9831.4. Samples: 303305396. Policy #0 lag: (min: 31.0, avg: 31.7, max: 53.0) [2023-12-26 19:40:36,063][104569] Avg episode reward: [(0, '6988.487'), (1, '6433.624')] [2023-12-26 19:40:36,063][105620] Updated weights for policy 1, policy_version 592769 (0.0010) [2023-12-26 19:40:36,104][105692] Updated weights for policy 0, policy_version 591935 (0.0005) [2023-12-26 19:40:36,155][105692] Updated weights for policy 0, policy_version 591945 (0.0008) [2023-12-26 19:40:36,778][105620] Updated weights for policy 1, policy_version 592779 (0.0010) [2023-12-26 19:40:36,836][105620] Updated weights for policy 1, policy_version 592789 (0.0009) [2023-12-26 19:40:36,873][105692] Updated weights for policy 0, policy_version 591955 (0.0007) [2023-12-26 19:40:36,901][105620] Updated weights for policy 1, policy_version 592799 (0.0008) [2023-12-26 19:40:36,929][105692] Updated weights for policy 0, policy_version 591965 (0.0006) [2023-12-26 19:40:36,990][105692] Updated weights for policy 0, policy_version 591975 (0.0006) [2023-12-26 19:40:37,584][105692] Updated weights for policy 0, policy_version 591985 (0.0006) [2023-12-26 19:40:37,607][105620] Updated weights for policy 1, policy_version 592809 (0.0008) [2023-12-26 19:40:37,635][105692] Updated weights for policy 0, policy_version 591995 (0.0009) [2023-12-26 19:40:37,656][105620] Updated weights for policy 1, policy_version 592819 (0.0005) [2023-12-26 19:40:37,683][105692] Updated weights for policy 0, policy_version 592005 (0.0009) [2023-12-26 19:40:37,711][105620] Updated weights for policy 1, policy_version 592829 (0.0006) [2023-12-26 19:40:37,749][105692] Updated weights for policy 0, policy_version 592015 (0.0006) [2023-12-26 19:40:37,760][105620] Updated weights for policy 1, policy_version 592839 (0.0009) [2023-12-26 19:40:38,403][105620] Updated weights for policy 1, policy_version 592849 (0.0006) [2023-12-26 19:40:38,417][105692] Updated weights for policy 0, policy_version 592025 (0.0008) [2023-12-26 19:40:38,465][105620] Updated weights for policy 1, policy_version 592859 (0.0006) [2023-12-26 19:40:38,477][105692] Updated weights for policy 0, policy_version 592035 (0.0009) [2023-12-26 19:40:38,527][105620] Updated weights for policy 1, policy_version 592869 (0.0006) [2023-12-26 19:40:38,542][105692] Updated weights for policy 0, policy_version 592045 (0.0009) [2023-12-26 19:40:39,137][105620] Updated weights for policy 1, policy_version 592879 (0.0008) [2023-12-26 19:40:39,192][105620] Updated weights for policy 1, policy_version 592889 (0.0009) [2023-12-26 19:40:39,260][105620] Updated weights for policy 1, policy_version 592899 (0.0007) [2023-12-26 19:40:39,384][105692] Updated weights for policy 0, policy_version 592055 (0.0008) [2023-12-26 19:40:39,447][105692] Updated weights for policy 0, policy_version 592065 (0.0010) [2023-12-26 19:40:39,503][105692] Updated weights for policy 0, policy_version 592075 (0.0009) [2023-12-26 19:40:40,042][105620] Updated weights for policy 1, policy_version 592909 (0.0006) [2023-12-26 19:40:40,107][105620] Updated weights for policy 1, policy_version 592919 (0.0006) [2023-12-26 19:40:40,171][105620] Updated weights for policy 1, policy_version 592929 (0.0009) [2023-12-26 19:40:40,285][105692] Updated weights for policy 0, policy_version 592085 (0.0008) [2023-12-26 19:40:40,343][105692] Updated weights for policy 0, policy_version 592095 (0.0005) [2023-12-26 19:40:40,403][105692] Updated weights for policy 0, policy_version 592105 (0.0007) [2023-12-26 19:40:40,931][105620] Updated weights for policy 1, policy_version 592939 (0.0009) [2023-12-26 19:40:40,992][105620] Updated weights for policy 1, policy_version 592949 (0.0010) [2023-12-26 19:40:41,046][105692] Updated weights for policy 0, policy_version 592115 (0.0008) [2023-12-26 19:40:41,054][105620] Updated weights for policy 1, policy_version 592959 (0.0008) [2023-12-26 19:40:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 303407104. Throughput: 0: 9619.3, 1: 9760.1. Samples: 303422464. Policy #0 lag: (min: 31.0, avg: 31.7, max: 53.0) [2023-12-26 19:40:41,063][104569] Avg episode reward: [(0, '4772.074'), (1, '8148.734')] [2023-12-26 19:40:41,106][105692] Updated weights for policy 0, policy_version 592125 (0.0008) [2023-12-26 19:40:41,172][105692] Updated weights for policy 0, policy_version 592135 (0.0009) [2023-12-26 19:40:41,794][105620] Updated weights for policy 1, policy_version 592969 (0.0009) [2023-12-26 19:40:41,849][105620] Updated weights for policy 1, policy_version 592979 (0.0007) [2023-12-26 19:40:41,898][105620] Updated weights for policy 1, policy_version 592989 (0.0009) [2023-12-26 19:40:41,956][105620] Updated weights for policy 1, policy_version 592999 (0.0009) [2023-12-26 19:40:42,003][105692] Updated weights for policy 0, policy_version 592145 (0.0009) [2023-12-26 19:40:42,061][105692] Updated weights for policy 0, policy_version 592155 (0.0008) [2023-12-26 19:40:42,120][105692] Updated weights for policy 0, policy_version 592165 (0.0006) [2023-12-26 19:40:42,183][105692] Updated weights for policy 0, policy_version 592175 (0.0010) [2023-12-26 19:40:42,667][105620] Updated weights for policy 1, policy_version 593009 (0.0008) [2023-12-26 19:40:42,723][105620] Updated weights for policy 1, policy_version 593019 (0.0009) [2023-12-26 19:40:42,771][105620] Updated weights for policy 1, policy_version 593029 (0.0009) [2023-12-26 19:40:42,862][105692] Updated weights for policy 0, policy_version 592185 (0.0009) [2023-12-26 19:40:42,908][105692] Updated weights for policy 0, policy_version 592195 (0.0007) [2023-12-26 19:40:42,955][105692] Updated weights for policy 0, policy_version 592205 (0.0005) [2023-12-26 19:40:43,609][105620] Updated weights for policy 1, policy_version 593039 (0.0008) [2023-12-26 19:40:43,618][105692] Updated weights for policy 0, policy_version 592215 (0.0007) [2023-12-26 19:40:43,657][105620] Updated weights for policy 1, policy_version 593049 (0.0007) [2023-12-26 19:40:43,677][105692] Updated weights for policy 0, policy_version 592225 (0.0008) [2023-12-26 19:40:43,704][105620] Updated weights for policy 1, policy_version 593059 (0.0007) [2023-12-26 19:40:43,719][105692] Updated weights for policy 0, policy_version 592235 (0.0006) [2023-12-26 19:40:44,469][105692] Updated weights for policy 0, policy_version 592245 (0.0009) [2023-12-26 19:40:44,488][105620] Updated weights for policy 1, policy_version 593069 (0.0006) [2023-12-26 19:40:44,518][105692] Updated weights for policy 0, policy_version 592255 (0.0009) [2023-12-26 19:40:44,538][105620] Updated weights for policy 1, policy_version 593079 (0.0005) [2023-12-26 19:40:44,564][105692] Updated weights for policy 0, policy_version 592265 (0.0009) [2023-12-26 19:40:44,588][105620] Updated weights for policy 1, policy_version 593089 (0.0008) [2023-12-26 19:40:45,334][105692] Updated weights for policy 0, policy_version 592275 (0.0007) [2023-12-26 19:40:45,356][105620] Updated weights for policy 1, policy_version 593099 (0.0007) [2023-12-26 19:40:45,392][105692] Updated weights for policy 0, policy_version 592285 (0.0006) [2023-12-26 19:40:45,416][105620] Updated weights for policy 1, policy_version 593109 (0.0008) [2023-12-26 19:40:45,458][105692] Updated weights for policy 0, policy_version 592295 (0.0006) [2023-12-26 19:40:45,480][105620] Updated weights for policy 1, policy_version 593119 (0.0008) [2023-12-26 19:40:46,061][105692] Updated weights for policy 0, policy_version 592305 (0.0006) [2023-12-26 19:40:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.8, 300 sec: 19521.9). Total num frames: 303505408. Throughput: 0: 9682.6, 1: 9679.8. Samples: 303479892. Policy #0 lag: (min: 31.0, avg: 31.7, max: 53.0) [2023-12-26 19:40:46,062][104569] Avg episode reward: [(0, '1211.012'), (1, '9000.876')] [2023-12-26 19:40:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000593128_151855104.pth... [2023-12-26 19:40:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000592008_151568384.pth [2023-12-26 19:40:46,120][105692] Updated weights for policy 0, policy_version 592315 (0.0009) [2023-12-26 19:40:46,171][105692] Updated weights for policy 0, policy_version 592325 (0.0009) [2023-12-26 19:40:46,171][105585] KL-divergence is very high: 129.7595 [2023-12-26 19:40:46,207][105585] KL-divergence is very high: 138.4948 [2023-12-26 19:40:46,215][105585] KL-divergence is very high: 173.8070 [2023-12-26 19:40:46,227][105692] Updated weights for policy 0, policy_version 592335 (0.0008) [2023-12-26 19:40:46,229][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000592336_151658496.pth... [2023-12-26 19:40:46,232][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000591184_151363584.pth [2023-12-26 19:40:46,239][105620] Updated weights for policy 1, policy_version 593129 (0.0009) [2023-12-26 19:40:46,293][105620] Updated weights for policy 1, policy_version 593139 (0.0009) [2023-12-26 19:40:46,347][105620] Updated weights for policy 1, policy_version 593149 (0.0008) [2023-12-26 19:40:46,403][105620] Updated weights for policy 1, policy_version 593159 (0.0008) [2023-12-26 19:40:46,962][105692] Updated weights for policy 0, policy_version 592345 (0.0009) [2023-12-26 19:40:47,013][105692] Updated weights for policy 0, policy_version 592355 (0.0009) [2023-12-26 19:40:47,070][105692] Updated weights for policy 0, policy_version 592365 (0.0009) [2023-12-26 19:40:47,130][105620] Updated weights for policy 1, policy_version 593169 (0.0009) [2023-12-26 19:40:47,178][105620] Updated weights for policy 1, policy_version 593179 (0.0008) [2023-12-26 19:40:47,235][105620] Updated weights for policy 1, policy_version 593189 (0.0009) [2023-12-26 19:40:47,795][105692] Updated weights for policy 0, policy_version 592375 (0.0006) [2023-12-26 19:40:47,846][105692] Updated weights for policy 0, policy_version 592385 (0.0005) [2023-12-26 19:40:47,897][105692] Updated weights for policy 0, policy_version 592395 (0.0005) [2023-12-26 19:40:48,070][105620] Updated weights for policy 1, policy_version 593199 (0.0010) [2023-12-26 19:40:48,121][105620] Updated weights for policy 1, policy_version 593209 (0.0009) [2023-12-26 19:40:48,168][105620] Updated weights for policy 1, policy_version 593219 (0.0009) [2023-12-26 19:40:48,521][105692] Updated weights for policy 0, policy_version 592405 (0.0007) [2023-12-26 19:40:48,582][105692] Updated weights for policy 0, policy_version 592415 (0.0009) [2023-12-26 19:40:48,640][105692] Updated weights for policy 0, policy_version 592425 (0.0009) [2023-12-26 19:40:48,947][105620] Updated weights for policy 1, policy_version 593229 (0.0009) [2023-12-26 19:40:49,005][105620] Updated weights for policy 1, policy_version 593239 (0.0010) [2023-12-26 19:40:49,060][105620] Updated weights for policy 1, policy_version 593249 (0.0009) [2023-12-26 19:40:49,355][105692] Updated weights for policy 0, policy_version 592435 (0.0009) [2023-12-26 19:40:49,412][105692] Updated weights for policy 0, policy_version 592445 (0.0009) [2023-12-26 19:40:49,469][105692] Updated weights for policy 0, policy_version 592455 (0.0008) [2023-12-26 19:40:49,890][105620] Updated weights for policy 1, policy_version 593259 (0.0009) [2023-12-26 19:40:49,949][105620] Updated weights for policy 1, policy_version 593269 (0.0009) [2023-12-26 19:40:49,994][105620] Updated weights for policy 1, policy_version 593279 (0.0008) [2023-12-26 19:40:50,160][105692] Updated weights for policy 0, policy_version 592465 (0.0009) [2023-12-26 19:40:50,206][105692] Updated weights for policy 0, policy_version 592475 (0.0008) [2023-12-26 19:40:50,254][105692] Updated weights for policy 0, policy_version 592485 (0.0009) [2023-12-26 19:40:50,306][105692] Updated weights for policy 0, policy_version 592495 (0.0009) [2023-12-26 19:40:50,755][105620] Updated weights for policy 1, policy_version 593289 (0.0008) [2023-12-26 19:40:50,802][105620] Updated weights for policy 1, policy_version 593299 (0.0009) [2023-12-26 19:40:50,858][105620] Updated weights for policy 1, policy_version 593309 (0.0009) [2023-12-26 19:40:50,929][105620] Updated weights for policy 1, policy_version 593319 (0.0010) [2023-12-26 19:40:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 303603712. Throughput: 0: 9736.8, 1: 9598.9. Samples: 303593764. Policy #0 lag: (min: 31.0, avg: 31.7, max: 53.0) [2023-12-26 19:40:51,062][104569] Avg episode reward: [(0, '2714.247'), (1, '8818.095')] [2023-12-26 19:40:51,066][105692] Updated weights for policy 0, policy_version 592505 (0.0008) [2023-12-26 19:40:51,128][105692] Updated weights for policy 0, policy_version 592515 (0.0008) [2023-12-26 19:40:51,190][105692] Updated weights for policy 0, policy_version 592525 (0.0008) [2023-12-26 19:40:51,736][105620] Updated weights for policy 1, policy_version 593329 (0.0009) [2023-12-26 19:40:51,792][105620] Updated weights for policy 1, policy_version 593339 (0.0007) [2023-12-26 19:40:51,841][105620] Updated weights for policy 1, policy_version 593349 (0.0008) [2023-12-26 19:40:51,925][105692] Updated weights for policy 0, policy_version 592535 (0.0008) [2023-12-26 19:40:51,976][105692] Updated weights for policy 0, policy_version 592545 (0.0005) [2023-12-26 19:40:52,025][105692] Updated weights for policy 0, policy_version 592555 (0.0007) [2023-12-26 19:40:52,586][105620] Updated weights for policy 1, policy_version 593359 (0.0007) [2023-12-26 19:40:52,649][105620] Updated weights for policy 1, policy_version 593369 (0.0008) [2023-12-26 19:40:52,705][105692] Updated weights for policy 0, policy_version 592565 (0.0010) [2023-12-26 19:40:52,711][105620] Updated weights for policy 1, policy_version 593379 (0.0006) [2023-12-26 19:40:52,771][105692] Updated weights for policy 0, policy_version 592575 (0.0010) [2023-12-26 19:40:52,838][105692] Updated weights for policy 0, policy_version 592585 (0.0010) [2023-12-26 19:40:53,446][105620] Updated weights for policy 1, policy_version 593389 (0.0007) [2023-12-26 19:40:53,490][105692] Updated weights for policy 0, policy_version 592595 (0.0010) [2023-12-26 19:40:53,494][105620] Updated weights for policy 1, policy_version 593399 (0.0009) [2023-12-26 19:40:53,541][105692] Updated weights for policy 0, policy_version 592605 (0.0010) [2023-12-26 19:40:53,550][105620] Updated weights for policy 1, policy_version 593409 (0.0009) [2023-12-26 19:40:53,597][105692] Updated weights for policy 0, policy_version 592615 (0.0006) [2023-12-26 19:40:54,250][105692] Updated weights for policy 0, policy_version 592625 (0.0006) [2023-12-26 19:40:54,308][105692] Updated weights for policy 0, policy_version 592635 (0.0009) [2023-12-26 19:40:54,360][105620] Updated weights for policy 1, policy_version 593419 (0.0008) [2023-12-26 19:40:54,365][105692] Updated weights for policy 0, policy_version 592645 (0.0008) [2023-12-26 19:40:54,421][105620] Updated weights for policy 1, policy_version 593429 (0.0008) [2023-12-26 19:40:54,431][105692] Updated weights for policy 0, policy_version 592655 (0.0008) [2023-12-26 19:40:54,479][105620] Updated weights for policy 1, policy_version 593439 (0.0007) [2023-12-26 19:40:55,210][105620] Updated weights for policy 1, policy_version 593449 (0.0008) [2023-12-26 19:40:55,232][105692] Updated weights for policy 0, policy_version 592665 (0.0008) [2023-12-26 19:40:55,258][105620] Updated weights for policy 1, policy_version 593459 (0.0006) [2023-12-26 19:40:55,287][105692] Updated weights for policy 0, policy_version 592675 (0.0009) [2023-12-26 19:40:55,313][105620] Updated weights for policy 1, policy_version 593469 (0.0007) [2023-12-26 19:40:55,348][105692] Updated weights for policy 0, policy_version 592685 (0.0007) [2023-12-26 19:40:55,375][105620] Updated weights for policy 1, policy_version 593479 (0.0006) [2023-12-26 19:40:56,026][105692] Updated weights for policy 0, policy_version 592695 (0.0008) [2023-12-26 19:40:56,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 303693824. Throughput: 0: 9771.5, 1: 9589.7. Samples: 303707944. Policy #0 lag: (min: 31.0, avg: 31.7, max: 53.0) [2023-12-26 19:40:56,062][104569] Avg episode reward: [(0, '6739.267'), (1, '9074.023')] [2023-12-26 19:40:56,088][105692] Updated weights for policy 0, policy_version 592705 (0.0009) [2023-12-26 19:40:56,150][105692] Updated weights for policy 0, policy_version 592715 (0.0009) [2023-12-26 19:40:56,158][105620] Updated weights for policy 1, policy_version 593489 (0.0006) [2023-12-26 19:40:56,217][105620] Updated weights for policy 1, policy_version 593499 (0.0008) [2023-12-26 19:40:56,272][105620] Updated weights for policy 1, policy_version 593509 (0.0008) [2023-12-26 19:40:56,868][105692] Updated weights for policy 0, policy_version 592725 (0.0010) [2023-12-26 19:40:56,908][105620] Updated weights for policy 1, policy_version 593519 (0.0006) [2023-12-26 19:40:56,929][105692] Updated weights for policy 0, policy_version 592735 (0.0010) [2023-12-26 19:40:56,974][105620] Updated weights for policy 1, policy_version 593529 (0.0005) [2023-12-26 19:40:56,994][105692] Updated weights for policy 0, policy_version 592745 (0.0006) [2023-12-26 19:40:57,028][105620] Updated weights for policy 1, policy_version 593539 (0.0008) [2023-12-26 19:40:57,525][105692] Updated weights for policy 0, policy_version 592755 (0.0006) [2023-12-26 19:40:57,538][105585] KL-divergence is very high: 105.7714 [2023-12-26 19:40:57,548][105585] KL-divergence is very high: 139.2844 [2023-12-26 19:40:57,552][105585] KL-divergence is very high: 146.0472 [2023-12-26 19:40:57,570][105692] Updated weights for policy 0, policy_version 592765 (0.0005) [2023-12-26 19:40:57,571][105585] KL-divergence is very high: 115.4064 [2023-12-26 19:40:57,576][105585] KL-divergence is very high: 163.5163 [2023-12-26 19:40:57,585][105585] KL-divergence is very high: 105.1435 [2023-12-26 19:40:57,616][105692] Updated weights for policy 0, policy_version 592775 (0.0005) [2023-12-26 19:40:57,785][105620] Updated weights for policy 1, policy_version 593549 (0.0009) [2023-12-26 19:40:57,838][105620] Updated weights for policy 1, policy_version 593559 (0.0011) [2023-12-26 19:40:57,890][105620] Updated weights for policy 1, policy_version 593569 (0.0009) [2023-12-26 19:40:58,180][105692] Updated weights for policy 0, policy_version 592785 (0.0006) [2023-12-26 19:40:58,243][105692] Updated weights for policy 0, policy_version 592795 (0.0008) [2023-12-26 19:40:58,308][105692] Updated weights for policy 0, policy_version 592805 (0.0007) [2023-12-26 19:40:58,379][105692] Updated weights for policy 0, policy_version 592815 (0.0007) [2023-12-26 19:40:58,782][105620] Updated weights for policy 1, policy_version 593579 (0.0010) [2023-12-26 19:40:58,845][105620] Updated weights for policy 1, policy_version 593589 (0.0009) [2023-12-26 19:40:58,913][105620] Updated weights for policy 1, policy_version 593599 (0.0008) [2023-12-26 19:40:59,143][105692] Updated weights for policy 0, policy_version 592825 (0.0008) [2023-12-26 19:40:59,209][105692] Updated weights for policy 0, policy_version 592835 (0.0009) [2023-12-26 19:40:59,274][105692] Updated weights for policy 0, policy_version 592845 (0.0009) [2023-12-26 19:40:59,698][105620] Updated weights for policy 1, policy_version 593609 (0.0007) [2023-12-26 19:40:59,750][105620] Updated weights for policy 1, policy_version 593619 (0.0009) [2023-12-26 19:40:59,801][105620] Updated weights for policy 1, policy_version 593629 (0.0009) [2023-12-26 19:40:59,863][105620] Updated weights for policy 1, policy_version 593639 (0.0008) [2023-12-26 19:40:59,983][105692] Updated weights for policy 0, policy_version 592855 (0.0008) [2023-12-26 19:41:00,039][105692] Updated weights for policy 0, policy_version 592865 (0.0008) [2023-12-26 19:41:00,105][105692] Updated weights for policy 0, policy_version 592875 (0.0008) [2023-12-26 19:41:00,665][105620] Updated weights for policy 1, policy_version 593649 (0.0010) [2023-12-26 19:41:00,716][105620] Updated weights for policy 1, policy_version 593659 (0.0010) [2023-12-26 19:41:00,764][105620] Updated weights for policy 1, policy_version 593669 (0.0010) [2023-12-26 19:41:00,849][105692] Updated weights for policy 0, policy_version 592885 (0.0007) [2023-12-26 19:41:00,903][105692] Updated weights for policy 0, policy_version 592895 (0.0005) [2023-12-26 19:41:00,952][105692] Updated weights for policy 0, policy_version 592905 (0.0005) [2023-12-26 19:41:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 303800320. Throughput: 0: 9825.3, 1: 9531.0. Samples: 303767656. Policy #0 lag: (min: 31.0, avg: 31.7, max: 53.0) [2023-12-26 19:41:01,062][104569] Avg episode reward: [(0, '6309.380'), (1, '7919.453')] [2023-12-26 19:41:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000593672_151994368.pth... [2023-12-26 19:41:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000592912_151805952.pth... [2023-12-26 19:41:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000591760_151511040.pth [2023-12-26 19:41:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000592552_151707648.pth [2023-12-26 19:41:01,542][105620] Updated weights for policy 1, policy_version 593679 (0.0010) [2023-12-26 19:41:01,597][105620] Updated weights for policy 1, policy_version 593689 (0.0010) [2023-12-26 19:41:01,615][105692] Updated weights for policy 0, policy_version 592915 (0.0006) [2023-12-26 19:41:01,659][105620] Updated weights for policy 1, policy_version 593699 (0.0008) [2023-12-26 19:41:01,680][105692] Updated weights for policy 0, policy_version 592925 (0.0009) [2023-12-26 19:41:01,748][105692] Updated weights for policy 0, policy_version 592935 (0.0008) [2023-12-26 19:41:02,334][105620] Updated weights for policy 1, policy_version 593709 (0.0008) [2023-12-26 19:41:02,395][105692] Updated weights for policy 0, policy_version 592945 (0.0009) [2023-12-26 19:41:02,401][105620] Updated weights for policy 1, policy_version 593719 (0.0009) [2023-12-26 19:41:02,447][105692] Updated weights for policy 0, policy_version 592955 (0.0010) [2023-12-26 19:41:02,457][105620] Updated weights for policy 1, policy_version 593729 (0.0005) [2023-12-26 19:41:02,496][105692] Updated weights for policy 0, policy_version 592965 (0.0010) [2023-12-26 19:41:02,544][105692] Updated weights for policy 0, policy_version 592975 (0.0010) [2023-12-26 19:41:03,019][105620] Updated weights for policy 1, policy_version 593739 (0.0006) [2023-12-26 19:41:03,075][105620] Updated weights for policy 1, policy_version 593749 (0.0005) [2023-12-26 19:41:03,136][105620] Updated weights for policy 1, policy_version 593759 (0.0006) [2023-12-26 19:41:03,287][105692] Updated weights for policy 0, policy_version 592985 (0.0006) [2023-12-26 19:41:03,335][105692] Updated weights for policy 0, policy_version 592995 (0.0008) [2023-12-26 19:41:03,381][105692] Updated weights for policy 0, policy_version 593005 (0.0008) [2023-12-26 19:41:03,804][105620] Updated weights for policy 1, policy_version 593769 (0.0008) [2023-12-26 19:41:03,871][105620] Updated weights for policy 1, policy_version 593779 (0.0011) [2023-12-26 19:41:03,937][105620] Updated weights for policy 1, policy_version 593789 (0.0010) [2023-12-26 19:41:04,004][105620] Updated weights for policy 1, policy_version 593799 (0.0010) [2023-12-26 19:41:04,111][105692] Updated weights for policy 0, policy_version 593015 (0.0009) [2023-12-26 19:41:04,176][105692] Updated weights for policy 0, policy_version 593025 (0.0006) [2023-12-26 19:41:04,245][105692] Updated weights for policy 0, policy_version 593035 (0.0006) [2023-12-26 19:41:04,668][105620] Updated weights for policy 1, policy_version 593809 (0.0006) [2023-12-26 19:41:04,720][105620] Updated weights for policy 1, policy_version 593819 (0.0005) [2023-12-26 19:41:04,772][105620] Updated weights for policy 1, policy_version 593829 (0.0008) [2023-12-26 19:41:05,024][105692] Updated weights for policy 0, policy_version 593045 (0.0008) [2023-12-26 19:41:05,084][105692] Updated weights for policy 0, policy_version 593055 (0.0008) [2023-12-26 19:41:05,143][105692] Updated weights for policy 0, policy_version 593065 (0.0008) [2023-12-26 19:41:05,473][105620] Updated weights for policy 1, policy_version 593839 (0.0010) [2023-12-26 19:41:05,539][105620] Updated weights for policy 1, policy_version 593849 (0.0010) [2023-12-26 19:41:05,591][105620] Updated weights for policy 1, policy_version 593859 (0.0010) [2023-12-26 19:41:05,780][105692] Updated weights for policy 0, policy_version 593075 (0.0007) [2023-12-26 19:41:05,847][105692] Updated weights for policy 0, policy_version 593085 (0.0006) [2023-12-26 19:41:05,909][105692] Updated weights for policy 0, policy_version 593095 (0.0005) [2023-12-26 19:41:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 303898624. Throughput: 0: 9793.2, 1: 9498.5. Samples: 303885792. Policy #0 lag: (min: 31.0, avg: 31.7, max: 53.0) [2023-12-26 19:41:06,062][104569] Avg episode reward: [(0, '7989.451'), (1, '8228.872')] [2023-12-26 19:41:06,177][105620] Updated weights for policy 1, policy_version 593869 (0.0009) [2023-12-26 19:41:06,237][105620] Updated weights for policy 1, policy_version 593879 (0.0008) [2023-12-26 19:41:06,293][105620] Updated weights for policy 1, policy_version 593889 (0.0008) [2023-12-26 19:41:06,581][105692] Updated weights for policy 0, policy_version 593105 (0.0008) [2023-12-26 19:41:06,636][105692] Updated weights for policy 0, policy_version 593115 (0.0010) [2023-12-26 19:41:06,688][105692] Updated weights for policy 0, policy_version 593125 (0.0010) [2023-12-26 19:41:06,740][105692] Updated weights for policy 0, policy_version 593135 (0.0010) [2023-12-26 19:41:07,053][105620] Updated weights for policy 1, policy_version 593899 (0.0008) [2023-12-26 19:41:07,111][105620] Updated weights for policy 1, policy_version 593909 (0.0008) [2023-12-26 19:41:07,171][105620] Updated weights for policy 1, policy_version 593919 (0.0006) [2023-12-26 19:41:07,501][105692] Updated weights for policy 0, policy_version 593145 (0.0010) [2023-12-26 19:41:07,568][105692] Updated weights for policy 0, policy_version 593155 (0.0010) [2023-12-26 19:41:07,629][105692] Updated weights for policy 0, policy_version 593165 (0.0011) [2023-12-26 19:41:07,735][105620] Updated weights for policy 1, policy_version 593929 (0.0005) [2023-12-26 19:41:07,787][105620] Updated weights for policy 1, policy_version 593939 (0.0006) [2023-12-26 19:41:07,846][105620] Updated weights for policy 1, policy_version 593949 (0.0005) [2023-12-26 19:41:07,911][105620] Updated weights for policy 1, policy_version 593959 (0.0005) [2023-12-26 19:41:08,475][105692] Updated weights for policy 0, policy_version 593175 (0.0010) [2023-12-26 19:41:08,506][105620] Updated weights for policy 1, policy_version 593969 (0.0007) [2023-12-26 19:41:08,535][105692] Updated weights for policy 0, policy_version 593185 (0.0010) [2023-12-26 19:41:08,562][105620] Updated weights for policy 1, policy_version 593979 (0.0008) [2023-12-26 19:41:08,592][105692] Updated weights for policy 0, policy_version 593195 (0.0009) [2023-12-26 19:41:08,618][105620] Updated weights for policy 1, policy_version 593989 (0.0008) [2023-12-26 19:41:09,376][105692] Updated weights for policy 0, policy_version 593205 (0.0008) [2023-12-26 19:41:09,379][105620] Updated weights for policy 1, policy_version 593999 (0.0007) [2023-12-26 19:41:09,439][105692] Updated weights for policy 0, policy_version 593215 (0.0008) [2023-12-26 19:41:09,443][105620] Updated weights for policy 1, policy_version 594009 (0.0009) [2023-12-26 19:41:09,502][105692] Updated weights for policy 0, policy_version 593225 (0.0008) [2023-12-26 19:41:09,511][105620] Updated weights for policy 1, policy_version 594019 (0.0008) [2023-12-26 19:41:10,199][105620] Updated weights for policy 1, policy_version 594029 (0.0009) [2023-12-26 19:41:10,262][105620] Updated weights for policy 1, policy_version 594039 (0.0008) [2023-12-26 19:41:10,326][105620] Updated weights for policy 1, policy_version 594049 (0.0008) [2023-12-26 19:41:10,327][105692] Updated weights for policy 0, policy_version 593235 (0.0010) [2023-12-26 19:41:10,391][105692] Updated weights for policy 0, policy_version 593245 (0.0008) [2023-12-26 19:41:10,461][105692] Updated weights for policy 0, policy_version 593255 (0.0008) [2023-12-26 19:41:11,045][105620] Updated weights for policy 1, policy_version 594059 (0.0009) [2023-12-26 19:41:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 303988736. Throughput: 0: 9720.4, 1: 9630.7. Samples: 304002020. Policy #0 lag: (min: 31.0, avg: 31.7, max: 53.0) [2023-12-26 19:41:11,062][104569] Avg episode reward: [(0, '8972.029'), (1, '9083.979')] [2023-12-26 19:41:11,105][105620] Updated weights for policy 1, policy_version 594069 (0.0008) [2023-12-26 19:41:11,166][105620] Updated weights for policy 1, policy_version 594079 (0.0010) [2023-12-26 19:41:11,251][105692] Updated weights for policy 0, policy_version 593265 (0.0009) [2023-12-26 19:41:11,317][105692] Updated weights for policy 0, policy_version 593275 (0.0009) [2023-12-26 19:41:11,378][105692] Updated weights for policy 0, policy_version 593285 (0.0009) [2023-12-26 19:41:11,433][105692] Updated weights for policy 0, policy_version 593295 (0.0010) [2023-12-26 19:41:11,915][105620] Updated weights for policy 1, policy_version 594089 (0.0009) [2023-12-26 19:41:11,976][105620] Updated weights for policy 1, policy_version 594099 (0.0008) [2023-12-26 19:41:12,043][105620] Updated weights for policy 1, policy_version 594109 (0.0008) [2023-12-26 19:41:12,109][105620] Updated weights for policy 1, policy_version 594119 (0.0009) [2023-12-26 19:41:12,211][105692] Updated weights for policy 0, policy_version 593305 (0.0006) [2023-12-26 19:41:12,279][105692] Updated weights for policy 0, policy_version 593315 (0.0008) [2023-12-26 19:41:12,343][105692] Updated weights for policy 0, policy_version 593325 (0.0008) [2023-12-26 19:41:12,902][105620] Updated weights for policy 1, policy_version 594129 (0.0006) [2023-12-26 19:41:12,948][105692] Updated weights for policy 0, policy_version 593335 (0.0010) [2023-12-26 19:41:12,958][105620] Updated weights for policy 1, policy_version 594139 (0.0006) [2023-12-26 19:41:13,006][105692] Updated weights for policy 0, policy_version 593345 (0.0010) [2023-12-26 19:41:13,009][105620] Updated weights for policy 1, policy_version 594149 (0.0006) [2023-12-26 19:41:13,065][105692] Updated weights for policy 0, policy_version 593355 (0.0010) [2023-12-26 19:41:13,731][105620] Updated weights for policy 1, policy_version 594159 (0.0008) [2023-12-26 19:41:13,779][105620] Updated weights for policy 1, policy_version 594169 (0.0007) [2023-12-26 19:41:13,803][105692] Updated weights for policy 0, policy_version 593365 (0.0010) [2023-12-26 19:41:13,825][105620] Updated weights for policy 1, policy_version 594179 (0.0008) [2023-12-26 19:41:13,861][105692] Updated weights for policy 0, policy_version 593375 (0.0010) [2023-12-26 19:41:13,922][105692] Updated weights for policy 0, policy_version 593385 (0.0010) [2023-12-26 19:41:14,621][105692] Updated weights for policy 0, policy_version 593395 (0.0010) [2023-12-26 19:41:14,642][105620] Updated weights for policy 1, policy_version 594189 (0.0008) [2023-12-26 19:41:14,671][105692] Updated weights for policy 0, policy_version 593405 (0.0009) [2023-12-26 19:41:14,699][105620] Updated weights for policy 1, policy_version 594199 (0.0006) [2023-12-26 19:41:14,723][105692] Updated weights for policy 0, policy_version 593415 (0.0007) [2023-12-26 19:41:14,763][105620] Updated weights for policy 1, policy_version 594209 (0.0007) [2023-12-26 19:41:15,419][105692] Updated weights for policy 0, policy_version 593425 (0.0007) [2023-12-26 19:41:15,496][105692] Updated weights for policy 0, policy_version 593435 (0.0008) [2023-12-26 19:41:15,559][105620] Updated weights for policy 1, policy_version 594219 (0.0008) [2023-12-26 19:41:15,564][105692] Updated weights for policy 0, policy_version 593445 (0.0009) [2023-12-26 19:41:15,622][105620] Updated weights for policy 1, policy_version 594229 (0.0008) [2023-12-26 19:41:15,632][105692] Updated weights for policy 0, policy_version 593455 (0.0008) [2023-12-26 19:41:15,683][105620] Updated weights for policy 1, policy_version 594239 (0.0005) [2023-12-26 19:41:16,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 304087040. Throughput: 0: 9721.0, 1: 9633.4. Samples: 304058164. Policy #0 lag: (min: 31.0, avg: 31.7, max: 53.0) [2023-12-26 19:41:16,063][104569] Avg episode reward: [(0, '8160.416'), (1, '8997.074')] [2023-12-26 19:41:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000594248_152141824.pth... [2023-12-26 19:41:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000593456_151945216.pth... [2023-12-26 19:41:16,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000593128_151855104.pth [2023-12-26 19:41:16,080][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000592336_151658496.pth [2023-12-26 19:41:16,221][105692] Updated weights for policy 0, policy_version 593465 (0.0006) [2023-12-26 19:41:16,272][105692] Updated weights for policy 0, policy_version 593475 (0.0005) [2023-12-26 19:41:16,326][105692] Updated weights for policy 0, policy_version 593485 (0.0005) [2023-12-26 19:41:16,473][105620] Updated weights for policy 1, policy_version 594249 (0.0006) [2023-12-26 19:41:16,536][105620] Updated weights for policy 1, policy_version 594259 (0.0010) [2023-12-26 19:41:16,590][105620] Updated weights for policy 1, policy_version 594269 (0.0010) [2023-12-26 19:41:16,648][105620] Updated weights for policy 1, policy_version 594279 (0.0010) [2023-12-26 19:41:16,992][105692] Updated weights for policy 0, policy_version 593495 (0.0005) [2023-12-26 19:41:17,046][105692] Updated weights for policy 0, policy_version 593505 (0.0006) [2023-12-26 19:41:17,090][105692] Updated weights for policy 0, policy_version 593515 (0.0007) [2023-12-26 19:41:17,287][105620] Updated weights for policy 1, policy_version 594289 (0.0010) [2023-12-26 19:41:17,335][105620] Updated weights for policy 1, policy_version 594299 (0.0010) [2023-12-26 19:41:17,385][105620] Updated weights for policy 1, policy_version 594309 (0.0010) [2023-12-26 19:41:17,826][105692] Updated weights for policy 0, policy_version 593525 (0.0009) [2023-12-26 19:41:17,881][105692] Updated weights for policy 0, policy_version 593535 (0.0009) [2023-12-26 19:41:17,938][105692] Updated weights for policy 0, policy_version 593545 (0.0009) [2023-12-26 19:41:18,099][105620] Updated weights for policy 1, policy_version 594319 (0.0007) [2023-12-26 19:41:18,167][105620] Updated weights for policy 1, policy_version 594329 (0.0006) [2023-12-26 19:41:18,236][105620] Updated weights for policy 1, policy_version 594339 (0.0007) [2023-12-26 19:41:18,692][105692] Updated weights for policy 0, policy_version 593555 (0.0009) [2023-12-26 19:41:18,751][105692] Updated weights for policy 0, policy_version 593565 (0.0009) [2023-12-26 19:41:18,813][105692] Updated weights for policy 0, policy_version 593575 (0.0006) [2023-12-26 19:41:18,967][105620] Updated weights for policy 1, policy_version 594349 (0.0008) [2023-12-26 19:41:19,031][105620] Updated weights for policy 1, policy_version 594359 (0.0009) [2023-12-26 19:41:19,092][105620] Updated weights for policy 1, policy_version 594369 (0.0009) [2023-12-26 19:41:19,567][105692] Updated weights for policy 0, policy_version 593585 (0.0007) [2023-12-26 19:41:19,626][105692] Updated weights for policy 0, policy_version 593595 (0.0010) [2023-12-26 19:41:19,688][105692] Updated weights for policy 0, policy_version 593605 (0.0009) [2023-12-26 19:41:19,740][105620] Updated weights for policy 1, policy_version 594379 (0.0009) [2023-12-26 19:41:19,751][105692] Updated weights for policy 0, policy_version 593615 (0.0007) [2023-12-26 19:41:19,801][105620] Updated weights for policy 1, policy_version 594389 (0.0008) [2023-12-26 19:41:19,870][105620] Updated weights for policy 1, policy_version 594399 (0.0009) [2023-12-26 19:41:20,558][105620] Updated weights for policy 1, policy_version 594409 (0.0007) [2023-12-26 19:41:20,596][105692] Updated weights for policy 0, policy_version 593625 (0.0008) [2023-12-26 19:41:20,618][105620] Updated weights for policy 1, policy_version 594419 (0.0008) [2023-12-26 19:41:20,656][105692] Updated weights for policy 0, policy_version 593635 (0.0009) [2023-12-26 19:41:20,684][105620] Updated weights for policy 1, policy_version 594429 (0.0008) [2023-12-26 19:41:20,720][105692] Updated weights for policy 0, policy_version 593645 (0.0011) [2023-12-26 19:41:20,748][105620] Updated weights for policy 1, policy_version 594439 (0.0008) [2023-12-26 19:41:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 304185344. Throughput: 0: 9752.4, 1: 9566.9. Samples: 304174764. Policy #0 lag: (min: 31.0, avg: 31.7, max: 53.0) [2023-12-26 19:41:21,062][104569] Avg episode reward: [(0, '8071.449'), (1, '8905.461')] [2023-12-26 19:41:21,429][105620] Updated weights for policy 1, policy_version 594449 (0.0008) [2023-12-26 19:41:21,448][105692] Updated weights for policy 0, policy_version 593655 (0.0008) [2023-12-26 19:41:21,493][105620] Updated weights for policy 1, policy_version 594459 (0.0006) [2023-12-26 19:41:21,505][105692] Updated weights for policy 0, policy_version 593665 (0.0008) [2023-12-26 19:41:21,553][105620] Updated weights for policy 1, policy_version 594469 (0.0006) [2023-12-26 19:41:21,564][105692] Updated weights for policy 0, policy_version 593675 (0.0008) [2023-12-26 19:41:22,227][105620] Updated weights for policy 1, policy_version 594479 (0.0006) [2023-12-26 19:41:22,286][105692] Updated weights for policy 0, policy_version 593685 (0.0010) [2023-12-26 19:41:22,294][105620] Updated weights for policy 1, policy_version 594489 (0.0006) [2023-12-26 19:41:22,351][105692] Updated weights for policy 0, policy_version 593695 (0.0008) [2023-12-26 19:41:22,358][105620] Updated weights for policy 1, policy_version 594499 (0.0008) [2023-12-26 19:41:22,418][105692] Updated weights for policy 0, policy_version 593705 (0.0007) [2023-12-26 19:41:23,071][105692] Updated weights for policy 0, policy_version 593715 (0.0008) [2023-12-26 19:41:23,100][105620] Updated weights for policy 1, policy_version 594509 (0.0010) [2023-12-26 19:41:23,134][105692] Updated weights for policy 0, policy_version 593725 (0.0009) [2023-12-26 19:41:23,159][105620] Updated weights for policy 1, policy_version 594519 (0.0010) [2023-12-26 19:41:23,191][105692] Updated weights for policy 0, policy_version 593735 (0.0005) [2023-12-26 19:41:23,212][105620] Updated weights for policy 1, policy_version 594529 (0.0011) [2023-12-26 19:41:23,916][105620] Updated weights for policy 1, policy_version 594539 (0.0010) [2023-12-26 19:41:23,925][105692] Updated weights for policy 0, policy_version 593745 (0.0006) [2023-12-26 19:41:23,976][105620] Updated weights for policy 1, policy_version 594549 (0.0010) [2023-12-26 19:41:23,976][105692] Updated weights for policy 0, policy_version 593755 (0.0007) [2023-12-26 19:41:24,025][105692] Updated weights for policy 0, policy_version 593765 (0.0010) [2023-12-26 19:41:24,039][105620] Updated weights for policy 1, policy_version 594559 (0.0010) [2023-12-26 19:41:24,075][105692] Updated weights for policy 0, policy_version 593775 (0.0011) [2023-12-26 19:41:24,705][105620] Updated weights for policy 1, policy_version 594569 (0.0008) [2023-12-26 19:41:24,766][105620] Updated weights for policy 1, policy_version 594579 (0.0010) [2023-12-26 19:41:24,821][105620] Updated weights for policy 1, policy_version 594589 (0.0010) [2023-12-26 19:41:24,843][105692] Updated weights for policy 0, policy_version 593785 (0.0010) [2023-12-26 19:41:24,882][105620] Updated weights for policy 1, policy_version 594599 (0.0010) [2023-12-26 19:41:24,898][105692] Updated weights for policy 0, policy_version 593795 (0.0010) [2023-12-26 19:41:24,945][105692] Updated weights for policy 0, policy_version 593805 (0.0010) [2023-12-26 19:41:25,611][105620] Updated weights for policy 1, policy_version 594609 (0.0009) [2023-12-26 19:41:25,663][105620] Updated weights for policy 1, policy_version 594619 (0.0009) [2023-12-26 19:41:25,708][105692] Updated weights for policy 0, policy_version 593815 (0.0010) [2023-12-26 19:41:25,718][105620] Updated weights for policy 1, policy_version 594629 (0.0011) [2023-12-26 19:41:25,768][105692] Updated weights for policy 0, policy_version 593825 (0.0006) [2023-12-26 19:41:25,834][105692] Updated weights for policy 0, policy_version 593835 (0.0006) [2023-12-26 19:41:26,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 304283648. Throughput: 0: 9747.2, 1: 9526.0. Samples: 304289756. Policy #0 lag: (min: 31.0, avg: 31.7, max: 53.0) [2023-12-26 19:41:26,063][104569] Avg episode reward: [(0, '8723.423'), (1, '8897.248')] [2023-12-26 19:41:26,440][105692] Updated weights for policy 0, policy_version 593845 (0.0006) [2023-12-26 19:41:26,509][105692] Updated weights for policy 0, policy_version 593855 (0.0005) [2023-12-26 19:41:26,576][105692] Updated weights for policy 0, policy_version 593865 (0.0008) [2023-12-26 19:41:26,591][105620] Updated weights for policy 1, policy_version 594639 (0.0009) [2023-12-26 19:41:26,649][105620] Updated weights for policy 1, policy_version 594649 (0.0009) [2023-12-26 19:41:26,706][105620] Updated weights for policy 1, policy_version 594659 (0.0009) [2023-12-26 19:41:27,172][105692] Updated weights for policy 0, policy_version 593875 (0.0011) [2023-12-26 19:41:27,220][105692] Updated weights for policy 0, policy_version 593885 (0.0008) [2023-12-26 19:41:27,264][105692] Updated weights for policy 0, policy_version 593895 (0.0005) [2023-12-26 19:41:27,300][105620] Updated weights for policy 1, policy_version 594669 (0.0008) [2023-12-26 19:41:27,357][105620] Updated weights for policy 1, policy_version 594679 (0.0006) [2023-12-26 19:41:27,419][105620] Updated weights for policy 1, policy_version 594689 (0.0008) [2023-12-26 19:41:27,878][105692] Updated weights for policy 0, policy_version 593905 (0.0006) [2023-12-26 19:41:27,934][105692] Updated weights for policy 0, policy_version 593915 (0.0006) [2023-12-26 19:41:27,992][105692] Updated weights for policy 0, policy_version 593925 (0.0005) [2023-12-26 19:41:28,057][105692] Updated weights for policy 0, policy_version 593935 (0.0010) [2023-12-26 19:41:28,120][105620] Updated weights for policy 1, policy_version 594699 (0.0009) [2023-12-26 19:41:28,172][105620] Updated weights for policy 1, policy_version 594709 (0.0009) [2023-12-26 19:41:28,227][105620] Updated weights for policy 1, policy_version 594719 (0.0008) [2023-12-26 19:41:28,742][105692] Updated weights for policy 0, policy_version 593945 (0.0010) [2023-12-26 19:41:28,811][105692] Updated weights for policy 0, policy_version 593955 (0.0010) [2023-12-26 19:41:28,869][105692] Updated weights for policy 0, policy_version 593965 (0.0009) [2023-12-26 19:41:29,016][105620] Updated weights for policy 1, policy_version 594729 (0.0008) [2023-12-26 19:41:29,073][105620] Updated weights for policy 1, policy_version 594739 (0.0008) [2023-12-26 19:41:29,131][105620] Updated weights for policy 1, policy_version 594749 (0.0008) [2023-12-26 19:41:29,182][105620] Updated weights for policy 1, policy_version 594759 (0.0008) [2023-12-26 19:41:29,600][105692] Updated weights for policy 0, policy_version 593975 (0.0009) [2023-12-26 19:41:29,658][105692] Updated weights for policy 0, policy_version 593985 (0.0010) [2023-12-26 19:41:29,709][105692] Updated weights for policy 0, policy_version 593995 (0.0010) [2023-12-26 19:41:29,971][105620] Updated weights for policy 1, policy_version 594769 (0.0008) [2023-12-26 19:41:30,027][105620] Updated weights for policy 1, policy_version 594779 (0.0008) [2023-12-26 19:41:30,078][105620] Updated weights for policy 1, policy_version 594789 (0.0008) [2023-12-26 19:41:30,473][105692] Updated weights for policy 0, policy_version 594005 (0.0010) [2023-12-26 19:41:30,534][105692] Updated weights for policy 0, policy_version 594015 (0.0010) [2023-12-26 19:41:30,591][105692] Updated weights for policy 0, policy_version 594025 (0.0010) [2023-12-26 19:41:30,841][105620] Updated weights for policy 1, policy_version 594799 (0.0008) [2023-12-26 19:41:30,890][105620] Updated weights for policy 1, policy_version 594809 (0.0008) [2023-12-26 19:41:30,939][105620] Updated weights for policy 1, policy_version 594819 (0.0008) [2023-12-26 19:41:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 304381952. Throughput: 0: 9800.7, 1: 9566.1. Samples: 304351392. Policy #0 lag: (min: 31.0, avg: 31.7, max: 53.0) [2023-12-26 19:41:31,062][104569] Avg episode reward: [(0, '9262.573'), (1, '8780.492')] [2023-12-26 19:41:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000594032_152092672.pth... [2023-12-26 19:41:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000594824_152289280.pth... [2023-12-26 19:41:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000593672_151994368.pth [2023-12-26 19:41:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000592912_151805952.pth [2023-12-26 19:41:31,329][105692] Updated weights for policy 0, policy_version 594035 (0.0010) [2023-12-26 19:41:31,400][105692] Updated weights for policy 0, policy_version 594045 (0.0008) [2023-12-26 19:41:31,462][105692] Updated weights for policy 0, policy_version 594055 (0.0007) [2023-12-26 19:41:31,774][105620] Updated weights for policy 1, policy_version 594829 (0.0009) [2023-12-26 19:41:31,826][105620] Updated weights for policy 1, policy_version 594839 (0.0009) [2023-12-26 19:41:31,882][105620] Updated weights for policy 1, policy_version 594849 (0.0009) [2023-12-26 19:41:32,072][105692] Updated weights for policy 0, policy_version 594065 (0.0008) [2023-12-26 19:41:32,130][105692] Updated weights for policy 0, policy_version 594075 (0.0008) [2023-12-26 19:41:32,188][105692] Updated weights for policy 0, policy_version 594085 (0.0010) [2023-12-26 19:41:32,249][105692] Updated weights for policy 0, policy_version 594095 (0.0010) [2023-12-26 19:41:32,745][105620] Updated weights for policy 1, policy_version 594859 (0.0010) [2023-12-26 19:41:32,814][105620] Updated weights for policy 1, policy_version 594869 (0.0009) [2023-12-26 19:41:32,864][105692] Updated weights for policy 0, policy_version 594105 (0.0006) [2023-12-26 19:41:32,866][105620] Updated weights for policy 1, policy_version 594879 (0.0008) [2023-12-26 19:41:32,910][105692] Updated weights for policy 0, policy_version 594115 (0.0005) [2023-12-26 19:41:32,959][105692] Updated weights for policy 0, policy_version 594125 (0.0008) [2023-12-26 19:41:33,613][105692] Updated weights for policy 0, policy_version 594135 (0.0007) [2023-12-26 19:41:33,671][105620] Updated weights for policy 1, policy_version 594889 (0.0008) [2023-12-26 19:41:33,675][105692] Updated weights for policy 0, policy_version 594145 (0.0009) [2023-12-26 19:41:33,726][105692] Updated weights for policy 0, policy_version 594155 (0.0010) [2023-12-26 19:41:33,732][105620] Updated weights for policy 1, policy_version 594899 (0.0005) [2023-12-26 19:41:33,778][105620] Updated weights for policy 1, policy_version 594909 (0.0007) [2023-12-26 19:41:33,836][105620] Updated weights for policy 1, policy_version 594919 (0.0008) [2023-12-26 19:41:34,387][105692] Updated weights for policy 0, policy_version 594165 (0.0008) [2023-12-26 19:41:34,448][105692] Updated weights for policy 0, policy_version 594175 (0.0007) [2023-12-26 19:41:34,511][105692] Updated weights for policy 0, policy_version 594185 (0.0010) [2023-12-26 19:41:34,687][105620] Updated weights for policy 1, policy_version 594929 (0.0010) [2023-12-26 19:41:34,746][105620] Updated weights for policy 1, policy_version 594940 (0.0010) [2023-12-26 19:41:34,800][105620] Updated weights for policy 1, policy_version 594950 (0.0010) [2023-12-26 19:41:35,030][105692] Updated weights for policy 0, policy_version 594195 (0.0008) [2023-12-26 19:41:35,086][105692] Updated weights for policy 0, policy_version 594205 (0.0005) [2023-12-26 19:41:35,139][105692] Updated weights for policy 0, policy_version 594215 (0.0005) [2023-12-26 19:41:35,684][105692] Updated weights for policy 0, policy_version 594225 (0.0005) [2023-12-26 19:41:35,732][105620] Updated weights for policy 1, policy_version 594960 (0.0008) [2023-12-26 19:41:35,734][105692] Updated weights for policy 0, policy_version 594235 (0.0008) [2023-12-26 19:41:35,778][105692] Updated weights for policy 0, policy_version 594245 (0.0010) [2023-12-26 19:41:35,789][105620] Updated weights for policy 1, policy_version 594970 (0.0006) [2023-12-26 19:41:35,826][105692] Updated weights for policy 0, policy_version 594255 (0.0005) [2023-12-26 19:41:35,843][105620] Updated weights for policy 1, policy_version 594980 (0.0009) [2023-12-26 19:41:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 304480256. Throughput: 0: 9834.5, 1: 9513.2. Samples: 304464412. Policy #0 lag: (min: 31.0, avg: 31.7, max: 53.0) [2023-12-26 19:41:36,063][104569] Avg episode reward: [(0, '9352.284'), (1, '8789.831')] [2023-12-26 19:41:36,413][105692] Updated weights for policy 0, policy_version 594265 (0.0006) [2023-12-26 19:41:36,481][105692] Updated weights for policy 0, policy_version 594275 (0.0006) [2023-12-26 19:41:36,544][105692] Updated weights for policy 0, policy_version 594285 (0.0010) [2023-12-26 19:41:36,635][105620] Updated weights for policy 1, policy_version 594990 (0.0008) [2023-12-26 19:41:36,693][105620] Updated weights for policy 1, policy_version 595000 (0.0009) [2023-12-26 19:41:36,763][105620] Updated weights for policy 1, policy_version 595010 (0.0010) [2023-12-26 19:41:37,217][105692] Updated weights for policy 0, policy_version 594295 (0.0010) [2023-12-26 19:41:37,268][105692] Updated weights for policy 0, policy_version 594305 (0.0009) [2023-12-26 19:41:37,312][105692] Updated weights for policy 0, policy_version 594315 (0.0010) [2023-12-26 19:41:37,543][105620] Updated weights for policy 1, policy_version 595020 (0.0009) [2023-12-26 19:41:37,599][105620] Updated weights for policy 1, policy_version 595030 (0.0009) [2023-12-26 19:41:37,662][105620] Updated weights for policy 1, policy_version 595040 (0.0008) [2023-12-26 19:41:37,957][105692] Updated weights for policy 0, policy_version 594325 (0.0010) [2023-12-26 19:41:38,010][105692] Updated weights for policy 0, policy_version 594335 (0.0010) [2023-12-26 19:41:38,073][105692] Updated weights for policy 0, policy_version 594345 (0.0010) [2023-12-26 19:41:38,469][105620] Updated weights for policy 1, policy_version 595050 (0.0008) [2023-12-26 19:41:38,539][105620] Updated weights for policy 1, policy_version 595060 (0.0009) [2023-12-26 19:41:38,598][105620] Updated weights for policy 1, policy_version 595070 (0.0008) [2023-12-26 19:41:38,661][105620] Updated weights for policy 1, policy_version 595080 (0.0008) [2023-12-26 19:41:38,828][105692] Updated weights for policy 0, policy_version 594355 (0.0011) [2023-12-26 19:41:38,884][105692] Updated weights for policy 0, policy_version 594365 (0.0010) [2023-12-26 19:41:38,946][105692] Updated weights for policy 0, policy_version 594375 (0.0011) [2023-12-26 19:41:39,275][105620] Updated weights for policy 1, policy_version 595090 (0.0006) [2023-12-26 19:41:39,339][105620] Updated weights for policy 1, policy_version 595100 (0.0007) [2023-12-26 19:41:39,404][105620] Updated weights for policy 1, policy_version 595110 (0.0007) [2023-12-26 19:41:39,689][105692] Updated weights for policy 0, policy_version 594385 (0.0010) [2023-12-26 19:41:39,751][105692] Updated weights for policy 0, policy_version 594395 (0.0006) [2023-12-26 19:41:39,804][105692] Updated weights for policy 0, policy_version 594405 (0.0005) [2023-12-26 19:41:39,867][105692] Updated weights for policy 0, policy_version 594415 (0.0011) [2023-12-26 19:41:40,176][105620] Updated weights for policy 1, policy_version 595120 (0.0008) [2023-12-26 19:41:40,233][105620] Updated weights for policy 1, policy_version 595130 (0.0008) [2023-12-26 19:41:40,286][105620] Updated weights for policy 1, policy_version 595140 (0.0008) [2023-12-26 19:41:40,581][105692] Updated weights for policy 0, policy_version 594425 (0.0006) [2023-12-26 19:41:40,654][105692] Updated weights for policy 0, policy_version 594435 (0.0005) [2023-12-26 19:41:40,698][105692] Updated weights for policy 0, policy_version 594445 (0.0005) [2023-12-26 19:41:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 304570368. Throughput: 0: 9931.0, 1: 9495.7. Samples: 304582144. Policy #0 lag: (min: 31.0, avg: 31.7, max: 53.0) [2023-12-26 19:41:41,063][104569] Avg episode reward: [(0, '9352.003'), (1, '9086.682')] [2023-12-26 19:41:41,131][105620] Updated weights for policy 1, policy_version 595150 (0.0008) [2023-12-26 19:41:41,197][105620] Updated weights for policy 1, policy_version 595160 (0.0007) [2023-12-26 19:41:41,263][105620] Updated weights for policy 1, policy_version 595170 (0.0007) [2023-12-26 19:41:41,276][105692] Updated weights for policy 0, policy_version 594455 (0.0009) [2023-12-26 19:41:41,343][105692] Updated weights for policy 0, policy_version 594465 (0.0011) [2023-12-26 19:41:41,411][105692] Updated weights for policy 0, policy_version 594475 (0.0009) [2023-12-26 19:41:42,006][105620] Updated weights for policy 1, policy_version 595180 (0.0006) [2023-12-26 19:41:42,067][105620] Updated weights for policy 1, policy_version 595190 (0.0007) [2023-12-26 19:41:42,126][105620] Updated weights for policy 1, policy_version 595200 (0.0006) [2023-12-26 19:41:42,243][105692] Updated weights for policy 0, policy_version 594485 (0.0010) [2023-12-26 19:41:42,309][105692] Updated weights for policy 0, policy_version 594495 (0.0008) [2023-12-26 19:41:42,379][105692] Updated weights for policy 0, policy_version 594505 (0.0009) [2023-12-26 19:41:42,789][105620] Updated weights for policy 1, policy_version 595210 (0.0007) [2023-12-26 19:41:42,844][105620] Updated weights for policy 1, policy_version 595220 (0.0006) [2023-12-26 19:41:42,896][105620] Updated weights for policy 1, policy_version 595230 (0.0005) [2023-12-26 19:41:42,965][105620] Updated weights for policy 1, policy_version 595240 (0.0008) [2023-12-26 19:41:43,165][105692] Updated weights for policy 0, policy_version 594515 (0.0009) [2023-12-26 19:41:43,224][105692] Updated weights for policy 0, policy_version 594525 (0.0010) [2023-12-26 19:41:43,278][105692] Updated weights for policy 0, policy_version 594535 (0.0010) [2023-12-26 19:41:43,538][105620] Updated weights for policy 1, policy_version 595250 (0.0005) [2023-12-26 19:41:43,586][105620] Updated weights for policy 1, policy_version 595260 (0.0005) [2023-12-26 19:41:43,633][105620] Updated weights for policy 1, policy_version 595270 (0.0005) [2023-12-26 19:41:43,916][105692] Updated weights for policy 0, policy_version 594545 (0.0009) [2023-12-26 19:41:43,969][105692] Updated weights for policy 0, policy_version 594555 (0.0006) [2023-12-26 19:41:44,020][105692] Updated weights for policy 0, policy_version 594565 (0.0006) [2023-12-26 19:41:44,074][105692] Updated weights for policy 0, policy_version 594575 (0.0006) [2023-12-26 19:41:44,315][105620] Updated weights for policy 1, policy_version 595280 (0.0005) [2023-12-26 19:41:44,362][105620] Updated weights for policy 1, policy_version 595290 (0.0008) [2023-12-26 19:41:44,415][105620] Updated weights for policy 1, policy_version 595300 (0.0010) [2023-12-26 19:41:44,697][105692] Updated weights for policy 0, policy_version 594585 (0.0005) [2023-12-26 19:41:44,750][105692] Updated weights for policy 0, policy_version 594595 (0.0006) [2023-12-26 19:41:44,814][105692] Updated weights for policy 0, policy_version 594605 (0.0006) [2023-12-26 19:41:45,126][105620] Updated weights for policy 1, policy_version 595310 (0.0010) [2023-12-26 19:41:45,189][105620] Updated weights for policy 1, policy_version 595320 (0.0011) [2023-12-26 19:41:45,248][105620] Updated weights for policy 1, policy_version 595330 (0.0010) [2023-12-26 19:41:45,427][105692] Updated weights for policy 0, policy_version 594615 (0.0006) [2023-12-26 19:41:45,479][105692] Updated weights for policy 0, policy_version 594625 (0.0005) [2023-12-26 19:41:45,538][105692] Updated weights for policy 0, policy_version 594635 (0.0005) [2023-12-26 19:41:45,989][105620] Updated weights for policy 1, policy_version 595340 (0.0010) [2023-12-26 19:41:46,047][105620] Updated weights for policy 1, policy_version 595350 (0.0010) [2023-12-26 19:41:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 304668672. Throughput: 0: 9834.4, 1: 9565.2. Samples: 304640640. Policy #0 lag: (min: 31.0, avg: 31.7, max: 53.0) [2023-12-26 19:41:46,063][104569] Avg episode reward: [(0, '9262.862'), (1, '9265.879')] [2023-12-26 19:41:46,074][105692] Updated weights for policy 0, policy_version 594645 (0.0005) [2023-12-26 19:41:46,106][105620] Updated weights for policy 1, policy_version 595360 (0.0010) [2023-12-26 19:41:46,123][105692] Updated weights for policy 0, policy_version 594655 (0.0007) [2023-12-26 19:41:46,158][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000595368_152428544.pth... [2023-12-26 19:41:46,162][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000594248_152141824.pth [2023-12-26 19:41:46,163][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_000595368_152428544.pth [2023-12-26 19:41:46,182][105692] Updated weights for policy 0, policy_version 594665 (0.0007) [2023-12-26 19:41:46,226][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000594672_152256512.pth... [2023-12-26 19:41:46,230][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000593456_151945216.pth [2023-12-26 19:41:46,231][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_000594672_152256512.pth [2023-12-26 19:41:46,789][105692] Updated weights for policy 0, policy_version 594675 (0.0009) [2023-12-26 19:41:46,839][105692] Updated weights for policy 0, policy_version 594685 (0.0006) [2023-12-26 19:41:46,852][105620] Updated weights for policy 1, policy_version 595370 (0.0011) [2023-12-26 19:41:46,888][105692] Updated weights for policy 0, policy_version 594695 (0.0011) [2023-12-26 19:41:46,911][105620] Updated weights for policy 1, policy_version 595380 (0.0010) [2023-12-26 19:41:46,966][105620] Updated weights for policy 1, policy_version 595390 (0.0010) [2023-12-26 19:41:47,027][105620] Updated weights for policy 1, policy_version 595400 (0.0010) [2023-12-26 19:41:47,455][105692] Updated weights for policy 0, policy_version 594705 (0.0010) [2023-12-26 19:41:47,513][105692] Updated weights for policy 0, policy_version 594715 (0.0005) [2023-12-26 19:41:47,570][105692] Updated weights for policy 0, policy_version 594725 (0.0010) [2023-12-26 19:41:47,624][105692] Updated weights for policy 0, policy_version 594735 (0.0010) [2023-12-26 19:41:47,714][105620] Updated weights for policy 1, policy_version 595410 (0.0006) [2023-12-26 19:41:47,781][105620] Updated weights for policy 1, policy_version 595420 (0.0010) [2023-12-26 19:41:47,843][105620] Updated weights for policy 1, policy_version 595430 (0.0006) [2023-12-26 19:41:48,354][105692] Updated weights for policy 0, policy_version 594745 (0.0011) [2023-12-26 19:41:48,415][105692] Updated weights for policy 0, policy_version 594755 (0.0009) [2023-12-26 19:41:48,445][105620] Updated weights for policy 1, policy_version 595440 (0.0009) [2023-12-26 19:41:48,473][105692] Updated weights for policy 0, policy_version 594765 (0.0009) [2023-12-26 19:41:48,508][105620] Updated weights for policy 1, policy_version 595450 (0.0011) [2023-12-26 19:41:48,568][105620] Updated weights for policy 1, policy_version 595460 (0.0011) [2023-12-26 19:41:49,207][105620] Updated weights for policy 1, policy_version 595470 (0.0008) [2023-12-26 19:41:49,212][105692] Updated weights for policy 0, policy_version 594775 (0.0011) [2023-12-26 19:41:49,272][105620] Updated weights for policy 1, policy_version 595480 (0.0006) [2023-12-26 19:41:49,277][105692] Updated weights for policy 0, policy_version 594785 (0.0011) [2023-12-26 19:41:49,336][105620] Updated weights for policy 1, policy_version 595490 (0.0006) [2023-12-26 19:41:49,340][105692] Updated weights for policy 0, policy_version 594795 (0.0009) [2023-12-26 19:41:49,942][105620] Updated weights for policy 1, policy_version 595500 (0.0009) [2023-12-26 19:41:49,998][105620] Updated weights for policy 1, policy_version 595510 (0.0011) [2023-12-26 19:41:50,060][105620] Updated weights for policy 1, policy_version 595520 (0.0008) [2023-12-26 19:41:50,106][105692] Updated weights for policy 0, policy_version 594805 (0.0010) [2023-12-26 19:41:50,172][105692] Updated weights for policy 0, policy_version 594815 (0.0011) [2023-12-26 19:41:50,234][105692] Updated weights for policy 0, policy_version 594825 (0.0010) [2023-12-26 19:41:50,689][105620] Updated weights for policy 1, policy_version 595530 (0.0006) [2023-12-26 19:41:50,751][105620] Updated weights for policy 1, policy_version 595540 (0.0007) [2023-12-26 19:41:50,813][105620] Updated weights for policy 1, policy_version 595550 (0.0008) [2023-12-26 19:41:50,872][105620] Updated weights for policy 1, policy_version 595560 (0.0008) [2023-12-26 19:41:50,972][105692] Updated weights for policy 0, policy_version 594835 (0.0010) [2023-12-26 19:41:51,034][105692] Updated weights for policy 0, policy_version 594845 (0.0011) [2023-12-26 19:41:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 304775168. Throughput: 0: 9986.9, 1: 9579.9. Samples: 304766300. Policy #0 lag: (min: 31.0, avg: 31.7, max: 53.0) [2023-12-26 19:41:51,062][104569] Avg episode reward: [(0, '9088.537'), (1, '9086.436')] [2023-12-26 19:41:51,103][105692] Updated weights for policy 0, policy_version 594855 (0.0011) [2023-12-26 19:41:51,605][105620] Updated weights for policy 1, policy_version 595570 (0.0007) [2023-12-26 19:41:51,673][105620] Updated weights for policy 1, policy_version 595580 (0.0008) [2023-12-26 19:41:51,738][105620] Updated weights for policy 1, policy_version 595590 (0.0008) [2023-12-26 19:41:51,848][105692] Updated weights for policy 0, policy_version 594865 (0.0009) [2023-12-26 19:41:51,914][105692] Updated weights for policy 0, policy_version 594875 (0.0006) [2023-12-26 19:41:51,971][105692] Updated weights for policy 0, policy_version 594885 (0.0006) [2023-12-26 19:41:52,036][105692] Updated weights for policy 0, policy_version 594895 (0.0007) [2023-12-26 19:41:52,419][105620] Updated weights for policy 1, policy_version 595600 (0.0008) [2023-12-26 19:41:52,482][105620] Updated weights for policy 1, policy_version 595610 (0.0008) [2023-12-26 19:41:52,544][105620] Updated weights for policy 1, policy_version 595620 (0.0008) [2023-12-26 19:41:52,739][105692] Updated weights for policy 0, policy_version 594905 (0.0007) [2023-12-26 19:41:52,790][105692] Updated weights for policy 0, policy_version 594915 (0.0006) [2023-12-26 19:41:52,855][105692] Updated weights for policy 0, policy_version 594925 (0.0005) [2023-12-26 19:41:53,216][105620] Updated weights for policy 1, policy_version 595630 (0.0009) [2023-12-26 19:41:53,277][105620] Updated weights for policy 1, policy_version 595640 (0.0009) [2023-12-26 19:41:53,335][105620] Updated weights for policy 1, policy_version 595650 (0.0009) [2023-12-26 19:41:53,523][105692] Updated weights for policy 0, policy_version 594935 (0.0008) [2023-12-26 19:41:53,570][105692] Updated weights for policy 0, policy_version 594945 (0.0010) [2023-12-26 19:41:53,618][105692] Updated weights for policy 0, policy_version 594955 (0.0010) [2023-12-26 19:41:54,094][105620] Updated weights for policy 1, policy_version 595660 (0.0008) [2023-12-26 19:41:54,153][105620] Updated weights for policy 1, policy_version 595670 (0.0008) [2023-12-26 19:41:54,211][105620] Updated weights for policy 1, policy_version 595680 (0.0007) [2023-12-26 19:41:54,365][105692] Updated weights for policy 0, policy_version 594965 (0.0011) [2023-12-26 19:41:54,431][105692] Updated weights for policy 0, policy_version 594975 (0.0010) [2023-12-26 19:41:54,493][105692] Updated weights for policy 0, policy_version 594985 (0.0010) [2023-12-26 19:41:54,828][105620] Updated weights for policy 1, policy_version 595690 (0.0006) [2023-12-26 19:41:54,884][105620] Updated weights for policy 1, policy_version 595700 (0.0005) [2023-12-26 19:41:54,945][105620] Updated weights for policy 1, policy_version 595710 (0.0006) [2023-12-26 19:41:55,006][105620] Updated weights for policy 1, policy_version 595720 (0.0006) [2023-12-26 19:41:55,197][105692] Updated weights for policy 0, policy_version 594995 (0.0009) [2023-12-26 19:41:55,247][105692] Updated weights for policy 0, policy_version 595005 (0.0005) [2023-12-26 19:41:55,302][105692] Updated weights for policy 0, policy_version 595015 (0.0005) [2023-12-26 19:41:55,521][105620] Updated weights for policy 1, policy_version 595730 (0.0005) [2023-12-26 19:41:55,574][105620] Updated weights for policy 1, policy_version 595740 (0.0005) [2023-12-26 19:41:55,618][105620] Updated weights for policy 1, policy_version 595750 (0.0005) [2023-12-26 19:41:55,990][105692] Updated weights for policy 0, policy_version 595025 (0.0007) [2023-12-26 19:41:56,048][105692] Updated weights for policy 0, policy_version 595035 (0.0007) [2023-12-26 19:41:56,062][104569] Fps is (10 sec: 20480.6, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 304873472. Throughput: 0: 10061.2, 1: 9620.5. Samples: 304887696. Policy #0 lag: (min: 19.0, avg: 43.8, max: 49.0) [2023-12-26 19:41:56,062][104569] Avg episode reward: [(0, '9004.798'), (1, '9086.388')] [2023-12-26 19:41:56,108][105692] Updated weights for policy 0, policy_version 595045 (0.0006) [2023-12-26 19:41:56,161][105692] Updated weights for policy 0, policy_version 595055 (0.0006) [2023-12-26 19:41:56,267][105620] Updated weights for policy 1, policy_version 595760 (0.0009) [2023-12-26 19:41:56,331][105620] Updated weights for policy 1, policy_version 595770 (0.0010) [2023-12-26 19:41:56,396][105620] Updated weights for policy 1, policy_version 595780 (0.0010) [2023-12-26 19:41:56,721][105692] Updated weights for policy 0, policy_version 595065 (0.0008) [2023-12-26 19:41:56,769][105692] Updated weights for policy 0, policy_version 595075 (0.0010) [2023-12-26 19:41:56,821][105692] Updated weights for policy 0, policy_version 595085 (0.0010) [2023-12-26 19:41:57,134][105620] Updated weights for policy 1, policy_version 595790 (0.0010) [2023-12-26 19:41:57,192][105620] Updated weights for policy 1, policy_version 595800 (0.0010) [2023-12-26 19:41:57,249][105620] Updated weights for policy 1, policy_version 595810 (0.0010) [2023-12-26 19:41:57,501][105692] Updated weights for policy 0, policy_version 595095 (0.0008) [2023-12-26 19:41:57,543][105692] Updated weights for policy 0, policy_version 595105 (0.0005) [2023-12-26 19:41:57,590][105692] Updated weights for policy 0, policy_version 595115 (0.0008) [2023-12-26 19:41:57,982][105620] Updated weights for policy 1, policy_version 595820 (0.0010) [2023-12-26 19:41:58,040][105620] Updated weights for policy 1, policy_version 595830 (0.0010) [2023-12-26 19:41:58,101][105620] Updated weights for policy 1, policy_version 595840 (0.0010) [2023-12-26 19:41:58,179][105692] Updated weights for policy 0, policy_version 595125 (0.0008) [2023-12-26 19:41:58,239][105692] Updated weights for policy 0, policy_version 595135 (0.0009) [2023-12-26 19:41:58,295][105692] Updated weights for policy 0, policy_version 595145 (0.0009) [2023-12-26 19:41:58,904][105620] Updated weights for policy 1, policy_version 595850 (0.0009) [2023-12-26 19:41:58,971][105620] Updated weights for policy 1, policy_version 595860 (0.0009) [2023-12-26 19:41:59,040][105620] Updated weights for policy 1, policy_version 595870 (0.0011) [2023-12-26 19:41:59,078][105692] Updated weights for policy 0, policy_version 595155 (0.0008) [2023-12-26 19:41:59,099][105620] Updated weights for policy 1, policy_version 595880 (0.0010) [2023-12-26 19:41:59,133][105692] Updated weights for policy 0, policy_version 595165 (0.0008) [2023-12-26 19:41:59,195][105692] Updated weights for policy 0, policy_version 595175 (0.0008) [2023-12-26 19:41:59,857][105620] Updated weights for policy 1, policy_version 595890 (0.0011) [2023-12-26 19:41:59,919][105620] Updated weights for policy 1, policy_version 595900 (0.0010) [2023-12-26 19:41:59,982][105620] Updated weights for policy 1, policy_version 595910 (0.0011) [2023-12-26 19:41:59,998][105692] Updated weights for policy 0, policy_version 595185 (0.0009) [2023-12-26 19:42:00,052][105692] Updated weights for policy 0, policy_version 595195 (0.0008) [2023-12-26 19:42:00,108][105692] Updated weights for policy 0, policy_version 595205 (0.0008) [2023-12-26 19:42:00,156][105692] Updated weights for policy 0, policy_version 595215 (0.0007) [2023-12-26 19:42:00,603][105620] Updated weights for policy 1, policy_version 595920 (0.0006) [2023-12-26 19:42:00,664][105620] Updated weights for policy 1, policy_version 595930 (0.0005) [2023-12-26 19:42:00,728][105620] Updated weights for policy 1, policy_version 595940 (0.0005) [2023-12-26 19:42:01,052][105692] Updated weights for policy 0, policy_version 595225 (0.0009) [2023-12-26 19:42:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 304971776. Throughput: 0: 10140.6, 1: 9622.5. Samples: 304947500. Policy #0 lag: (min: 19.0, avg: 43.8, max: 49.0) [2023-12-26 19:42:01,062][104569] Avg episode reward: [(0, '8677.577'), (1, '9174.842')] [2023-12-26 19:42:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000595944_152576000.pth... [2023-12-26 19:42:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000594824_152289280.pth [2023-12-26 19:42:01,120][105692] Updated weights for policy 0, policy_version 595235 (0.0008) [2023-12-26 19:42:01,186][105692] Updated weights for policy 0, policy_version 595245 (0.0009) [2023-12-26 19:42:01,204][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000595248_152403968.pth... [2023-12-26 19:42:01,209][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000594032_152092672.pth [2023-12-26 19:42:01,282][105620] Updated weights for policy 1, policy_version 595950 (0.0009) [2023-12-26 19:42:01,330][105620] Updated weights for policy 1, policy_version 595960 (0.0010) [2023-12-26 19:42:01,399][105620] Updated weights for policy 1, policy_version 595970 (0.0009) [2023-12-26 19:42:01,940][105692] Updated weights for policy 0, policy_version 595255 (0.0006) [2023-12-26 19:42:02,002][105692] Updated weights for policy 0, policy_version 595265 (0.0006) [2023-12-26 19:42:02,059][105692] Updated weights for policy 0, policy_version 595275 (0.0007) [2023-12-26 19:42:02,116][105620] Updated weights for policy 1, policy_version 595980 (0.0008) [2023-12-26 19:42:02,176][105620] Updated weights for policy 1, policy_version 595990 (0.0005) [2023-12-26 19:42:02,241][105620] Updated weights for policy 1, policy_version 596000 (0.0006) [2023-12-26 19:42:02,777][105692] Updated weights for policy 0, policy_version 595285 (0.0009) [2023-12-26 19:42:02,837][105692] Updated weights for policy 0, policy_version 595295 (0.0009) [2023-12-26 19:42:02,892][105620] Updated weights for policy 1, policy_version 596010 (0.0008) [2023-12-26 19:42:02,899][105692] Updated weights for policy 0, policy_version 595305 (0.0009) [2023-12-26 19:42:02,952][105620] Updated weights for policy 1, policy_version 596020 (0.0005) [2023-12-26 19:42:03,016][105620] Updated weights for policy 1, policy_version 596030 (0.0009) [2023-12-26 19:42:03,070][105620] Updated weights for policy 1, policy_version 596040 (0.0009) [2023-12-26 19:42:03,582][105692] Updated weights for policy 0, policy_version 595315 (0.0009) [2023-12-26 19:42:03,630][105692] Updated weights for policy 0, policy_version 595326 (0.0007) [2023-12-26 19:42:03,682][105692] Updated weights for policy 0, policy_version 595336 (0.0009) [2023-12-26 19:42:03,828][105620] Updated weights for policy 1, policy_version 596050 (0.0010) [2023-12-26 19:42:03,881][105620] Updated weights for policy 1, policy_version 596060 (0.0008) [2023-12-26 19:42:03,925][105620] Updated weights for policy 1, policy_version 596070 (0.0008) [2023-12-26 19:42:04,360][105692] Updated weights for policy 0, policy_version 595346 (0.0010) [2023-12-26 19:42:04,417][105692] Updated weights for policy 0, policy_version 595356 (0.0009) [2023-12-26 19:42:04,479][105692] Updated weights for policy 0, policy_version 595366 (0.0009) [2023-12-26 19:42:04,548][105692] Updated weights for policy 0, policy_version 595376 (0.0009) [2023-12-26 19:42:04,664][105620] Updated weights for policy 1, policy_version 596080 (0.0010) [2023-12-26 19:42:04,720][105620] Updated weights for policy 1, policy_version 596090 (0.0010) [2023-12-26 19:42:04,784][105620] Updated weights for policy 1, policy_version 596100 (0.0005) [2023-12-26 19:42:05,233][105692] Updated weights for policy 0, policy_version 595386 (0.0008) [2023-12-26 19:42:05,281][105692] Updated weights for policy 0, policy_version 595396 (0.0008) [2023-12-26 19:42:05,329][105692] Updated weights for policy 0, policy_version 595406 (0.0007) [2023-12-26 19:42:05,495][105620] Updated weights for policy 1, policy_version 596110 (0.0005) [2023-12-26 19:42:05,546][105620] Updated weights for policy 1, policy_version 596120 (0.0010) [2023-12-26 19:42:05,601][105620] Updated weights for policy 1, policy_version 596130 (0.0010) [2023-12-26 19:42:05,943][105692] Updated weights for policy 0, policy_version 595416 (0.0005) [2023-12-26 19:42:05,987][105692] Updated weights for policy 0, policy_version 595426 (0.0005) [2023-12-26 19:42:06,038][105692] Updated weights for policy 0, policy_version 595436 (0.0005) [2023-12-26 19:42:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 305078272. Throughput: 0: 10044.8, 1: 9684.6. Samples: 305062588. Policy #0 lag: (min: 19.0, avg: 43.8, max: 49.0) [2023-12-26 19:42:06,062][104569] Avg episode reward: [(0, '8403.629'), (1, '8828.763')] [2023-12-26 19:42:06,288][105620] Updated weights for policy 1, policy_version 596140 (0.0010) [2023-12-26 19:42:06,338][105620] Updated weights for policy 1, policy_version 596150 (0.0011) [2023-12-26 19:42:06,407][105620] Updated weights for policy 1, policy_version 596160 (0.0009) [2023-12-26 19:42:06,756][105692] Updated weights for policy 0, policy_version 595446 (0.0008) [2023-12-26 19:42:06,818][105692] Updated weights for policy 0, policy_version 595456 (0.0010) [2023-12-26 19:42:06,869][105692] Updated weights for policy 0, policy_version 595466 (0.0010) [2023-12-26 19:42:07,108][105620] Updated weights for policy 1, policy_version 596170 (0.0008) [2023-12-26 19:42:07,171][105620] Updated weights for policy 1, policy_version 596180 (0.0010) [2023-12-26 19:42:07,226][105620] Updated weights for policy 1, policy_version 596190 (0.0011) [2023-12-26 19:42:07,286][105620] Updated weights for policy 1, policy_version 596200 (0.0010) [2023-12-26 19:42:07,500][105692] Updated weights for policy 0, policy_version 595476 (0.0010) [2023-12-26 19:42:07,553][105692] Updated weights for policy 0, policy_version 595486 (0.0010) [2023-12-26 19:42:07,615][105692] Updated weights for policy 0, policy_version 595496 (0.0006) [2023-12-26 19:42:07,989][105620] Updated weights for policy 1, policy_version 596210 (0.0008) [2023-12-26 19:42:08,048][105620] Updated weights for policy 1, policy_version 596220 (0.0008) [2023-12-26 19:42:08,107][105620] Updated weights for policy 1, policy_version 596230 (0.0008) [2023-12-26 19:42:08,268][105692] Updated weights for policy 0, policy_version 595506 (0.0006) [2023-12-26 19:42:08,317][105692] Updated weights for policy 0, policy_version 595516 (0.0010) [2023-12-26 19:42:08,373][105692] Updated weights for policy 0, policy_version 595526 (0.0010) [2023-12-26 19:42:08,425][105692] Updated weights for policy 0, policy_version 595536 (0.0010) [2023-12-26 19:42:08,838][105620] Updated weights for policy 1, policy_version 596240 (0.0010) [2023-12-26 19:42:08,909][105620] Updated weights for policy 1, policy_version 596250 (0.0011) [2023-12-26 19:42:08,975][105620] Updated weights for policy 1, policy_version 596260 (0.0010) [2023-12-26 19:42:09,120][105692] Updated weights for policy 0, policy_version 595546 (0.0010) [2023-12-26 19:42:09,185][105692] Updated weights for policy 0, policy_version 595556 (0.0010) [2023-12-26 19:42:09,250][105692] Updated weights for policy 0, policy_version 595566 (0.0011) [2023-12-26 19:42:09,742][105620] Updated weights for policy 1, policy_version 596270 (0.0009) [2023-12-26 19:42:09,807][105620] Updated weights for policy 1, policy_version 596280 (0.0008) [2023-12-26 19:42:09,867][105620] Updated weights for policy 1, policy_version 596290 (0.0006) [2023-12-26 19:42:10,002][105692] Updated weights for policy 0, policy_version 595576 (0.0011) [2023-12-26 19:42:10,058][105692] Updated weights for policy 0, policy_version 595586 (0.0011) [2023-12-26 19:42:10,122][105692] Updated weights for policy 0, policy_version 595596 (0.0011) [2023-12-26 19:42:10,499][105620] Updated weights for policy 1, policy_version 596300 (0.0006) [2023-12-26 19:42:10,559][105620] Updated weights for policy 1, policy_version 596310 (0.0005) [2023-12-26 19:42:10,616][105620] Updated weights for policy 1, policy_version 596320 (0.0005) [2023-12-26 19:42:10,875][105692] Updated weights for policy 0, policy_version 595606 (0.0011) [2023-12-26 19:42:10,941][105692] Updated weights for policy 0, policy_version 595616 (0.0010) [2023-12-26 19:42:11,003][105692] Updated weights for policy 0, policy_version 595626 (0.0010) [2023-12-26 19:42:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 305176576. Throughput: 0: 10166.0, 1: 9698.9. Samples: 305183680. Policy #0 lag: (min: 19.0, avg: 43.8, max: 49.0) [2023-12-26 19:42:11,062][104569] Avg episode reward: [(0, '8904.726'), (1, '8828.494')] [2023-12-26 19:42:11,213][105620] Updated weights for policy 1, policy_version 596330 (0.0006) [2023-12-26 19:42:11,281][105620] Updated weights for policy 1, policy_version 596340 (0.0009) [2023-12-26 19:42:11,341][105620] Updated weights for policy 1, policy_version 596350 (0.0008) [2023-12-26 19:42:11,405][105620] Updated weights for policy 1, policy_version 596360 (0.0008) [2023-12-26 19:42:11,732][105692] Updated weights for policy 0, policy_version 595636 (0.0010) [2023-12-26 19:42:11,804][105692] Updated weights for policy 0, policy_version 595646 (0.0009) [2023-12-26 19:42:11,871][105692] Updated weights for policy 0, policy_version 595656 (0.0009) [2023-12-26 19:42:12,237][105620] Updated weights for policy 1, policy_version 596370 (0.0009) [2023-12-26 19:42:12,300][105620] Updated weights for policy 1, policy_version 596380 (0.0008) [2023-12-26 19:42:12,370][105620] Updated weights for policy 1, policy_version 596390 (0.0009) [2023-12-26 19:42:12,582][105692] Updated weights for policy 0, policy_version 595666 (0.0006) [2023-12-26 19:42:12,644][105692] Updated weights for policy 0, policy_version 595676 (0.0009) [2023-12-26 19:42:12,710][105692] Updated weights for policy 0, policy_version 595686 (0.0008) [2023-12-26 19:42:12,769][105692] Updated weights for policy 0, policy_version 595696 (0.0009) [2023-12-26 19:42:13,111][105620] Updated weights for policy 1, policy_version 596400 (0.0006) [2023-12-26 19:42:13,171][105620] Updated weights for policy 1, policy_version 596410 (0.0005) [2023-12-26 19:42:13,220][105620] Updated weights for policy 1, policy_version 596420 (0.0006) [2023-12-26 19:42:13,524][105692] Updated weights for policy 0, policy_version 595706 (0.0010) [2023-12-26 19:42:13,576][105692] Updated weights for policy 0, policy_version 595716 (0.0010) [2023-12-26 19:42:13,639][105692] Updated weights for policy 0, policy_version 595726 (0.0009) [2023-12-26 19:42:13,838][105620] Updated weights for policy 1, policy_version 596430 (0.0008) [2023-12-26 19:42:13,892][105620] Updated weights for policy 1, policy_version 596440 (0.0010) [2023-12-26 19:42:13,938][105620] Updated weights for policy 1, policy_version 596450 (0.0008) [2023-12-26 19:42:14,238][105692] Updated weights for policy 0, policy_version 595736 (0.0010) [2023-12-26 19:42:14,299][105692] Updated weights for policy 0, policy_version 595746 (0.0010) [2023-12-26 19:42:14,357][105692] Updated weights for policy 0, policy_version 595756 (0.0010) [2023-12-26 19:42:14,730][105620] Updated weights for policy 1, policy_version 596460 (0.0010) [2023-12-26 19:42:14,793][105620] Updated weights for policy 1, policy_version 596470 (0.0011) [2023-12-26 19:42:14,853][105620] Updated weights for policy 1, policy_version 596480 (0.0011) [2023-12-26 19:42:15,116][105692] Updated weights for policy 0, policy_version 595766 (0.0010) [2023-12-26 19:42:15,189][105692] Updated weights for policy 0, policy_version 595776 (0.0011) [2023-12-26 19:42:15,252][105692] Updated weights for policy 0, policy_version 595786 (0.0011) [2023-12-26 19:42:15,595][105620] Updated weights for policy 1, policy_version 596490 (0.0011) [2023-12-26 19:42:15,654][105620] Updated weights for policy 1, policy_version 596500 (0.0011) [2023-12-26 19:42:15,713][105620] Updated weights for policy 1, policy_version 596510 (0.0010) [2023-12-26 19:42:15,775][105620] Updated weights for policy 1, policy_version 596520 (0.0010) [2023-12-26 19:42:15,983][105692] Updated weights for policy 0, policy_version 595796 (0.0011) [2023-12-26 19:42:16,030][105692] Updated weights for policy 0, policy_version 595806 (0.0010) [2023-12-26 19:42:16,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 305266688. Throughput: 0: 10064.2, 1: 9675.7. Samples: 305239692. Policy #0 lag: (min: 19.0, avg: 43.8, max: 49.0) [2023-12-26 19:42:16,063][104569] Avg episode reward: [(0, '8998.182'), (1, '8725.171')] [2023-12-26 19:42:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000596520_152723456.pth... [2023-12-26 19:42:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000595368_152428544.pth [2023-12-26 19:42:16,079][105692] Updated weights for policy 0, policy_version 595816 (0.0010) [2023-12-26 19:42:16,120][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000595824_152551424.pth... [2023-12-26 19:42:16,123][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000594672_152256512.pth [2023-12-26 19:42:16,514][105620] Updated weights for policy 1, policy_version 596530 (0.0010) [2023-12-26 19:42:16,566][105620] Updated weights for policy 1, policy_version 596540 (0.0010) [2023-12-26 19:42:16,630][105620] Updated weights for policy 1, policy_version 596550 (0.0010) [2023-12-26 19:42:16,833][105692] Updated weights for policy 0, policy_version 595826 (0.0011) [2023-12-26 19:42:16,882][105692] Updated weights for policy 0, policy_version 595836 (0.0011) [2023-12-26 19:42:16,937][105692] Updated weights for policy 0, policy_version 595846 (0.0011) [2023-12-26 19:42:16,992][105692] Updated weights for policy 0, policy_version 595856 (0.0010) [2023-12-26 19:42:17,270][105620] Updated weights for policy 1, policy_version 596560 (0.0009) [2023-12-26 19:42:17,320][105620] Updated weights for policy 1, policy_version 596570 (0.0008) [2023-12-26 19:42:17,364][105620] Updated weights for policy 1, policy_version 596580 (0.0008) [2023-12-26 19:42:17,763][105692] Updated weights for policy 0, policy_version 595866 (0.0010) [2023-12-26 19:42:17,818][105692] Updated weights for policy 0, policy_version 595876 (0.0010) [2023-12-26 19:42:17,879][105692] Updated weights for policy 0, policy_version 595886 (0.0010) [2023-12-26 19:42:18,146][105620] Updated weights for policy 1, policy_version 596590 (0.0007) [2023-12-26 19:42:18,189][105620] Updated weights for policy 1, policy_version 596600 (0.0010) [2023-12-26 19:42:18,246][105620] Updated weights for policy 1, policy_version 596610 (0.0006) [2023-12-26 19:42:18,580][105692] Updated weights for policy 0, policy_version 595896 (0.0010) [2023-12-26 19:42:18,641][105692] Updated weights for policy 0, policy_version 595906 (0.0011) [2023-12-26 19:42:18,706][105692] Updated weights for policy 0, policy_version 595916 (0.0011) [2023-12-26 19:42:18,902][105620] Updated weights for policy 1, policy_version 596620 (0.0005) [2023-12-26 19:42:18,958][105620] Updated weights for policy 1, policy_version 596630 (0.0005) [2023-12-26 19:42:19,010][105620] Updated weights for policy 1, policy_version 596640 (0.0005) [2023-12-26 19:42:19,376][105692] Updated weights for policy 0, policy_version 595926 (0.0009) [2023-12-26 19:42:19,432][105692] Updated weights for policy 0, policy_version 595936 (0.0007) [2023-12-26 19:42:19,495][105692] Updated weights for policy 0, policy_version 595946 (0.0007) [2023-12-26 19:42:19,702][105620] Updated weights for policy 1, policy_version 596650 (0.0006) [2023-12-26 19:42:19,772][105620] Updated weights for policy 1, policy_version 596660 (0.0006) [2023-12-26 19:42:19,837][105620] Updated weights for policy 1, policy_version 596670 (0.0011) [2023-12-26 19:42:19,905][105620] Updated weights for policy 1, policy_version 596680 (0.0010) [2023-12-26 19:42:20,192][105692] Updated weights for policy 0, policy_version 595956 (0.0010) [2023-12-26 19:42:20,262][105692] Updated weights for policy 0, policy_version 595966 (0.0011) [2023-12-26 19:42:20,328][105692] Updated weights for policy 0, policy_version 595976 (0.0011) [2023-12-26 19:42:20,668][105620] Updated weights for policy 1, policy_version 596690 (0.0011) [2023-12-26 19:42:20,737][105620] Updated weights for policy 1, policy_version 596700 (0.0011) [2023-12-26 19:42:20,800][105620] Updated weights for policy 1, policy_version 596710 (0.0011) [2023-12-26 19:42:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 305364992. Throughput: 0: 10013.3, 1: 9822.7. Samples: 305357028. Policy #0 lag: (min: 19.0, avg: 43.8, max: 49.0) [2023-12-26 19:42:21,062][104569] Avg episode reward: [(0, '8907.251'), (1, '8820.593')] [2023-12-26 19:42:21,092][105692] Updated weights for policy 0, policy_version 595986 (0.0010) [2023-12-26 19:42:21,152][105692] Updated weights for policy 0, policy_version 595996 (0.0011) [2023-12-26 19:42:21,208][105692] Updated weights for policy 0, policy_version 596006 (0.0010) [2023-12-26 19:42:21,263][105692] Updated weights for policy 0, policy_version 596016 (0.0011) [2023-12-26 19:42:21,553][105620] Updated weights for policy 1, policy_version 596720 (0.0010) [2023-12-26 19:42:21,602][105620] Updated weights for policy 1, policy_version 596730 (0.0010) [2023-12-26 19:42:21,664][105620] Updated weights for policy 1, policy_version 596740 (0.0010) [2023-12-26 19:42:22,069][105692] Updated weights for policy 0, policy_version 596026 (0.0010) [2023-12-26 19:42:22,129][105692] Updated weights for policy 0, policy_version 596036 (0.0009) [2023-12-26 19:42:22,131][105585] KL-divergence is very high: 108.7641 [2023-12-26 19:42:22,174][105585] KL-divergence is very high: 113.0735 [2023-12-26 19:42:22,184][105692] Updated weights for policy 0, policy_version 596046 (0.0009) [2023-12-26 19:42:22,455][105620] Updated weights for policy 1, policy_version 596750 (0.0009) [2023-12-26 19:42:22,506][105620] Updated weights for policy 1, policy_version 596760 (0.0009) [2023-12-26 19:42:22,565][105620] Updated weights for policy 1, policy_version 596770 (0.0009) [2023-12-26 19:42:22,918][105692] Updated weights for policy 0, policy_version 596056 (0.0009) [2023-12-26 19:42:22,971][105692] Updated weights for policy 0, policy_version 596066 (0.0010) [2023-12-26 19:42:23,033][105692] Updated weights for policy 0, policy_version 596076 (0.0008) [2023-12-26 19:42:23,353][105620] Updated weights for policy 1, policy_version 596780 (0.0009) [2023-12-26 19:42:23,407][105620] Updated weights for policy 1, policy_version 596790 (0.0010) [2023-12-26 19:42:23,454][105620] Updated weights for policy 1, policy_version 596801 (0.0008) [2023-12-26 19:42:23,729][105692] Updated weights for policy 0, policy_version 596086 (0.0010) [2023-12-26 19:42:23,785][105692] Updated weights for policy 0, policy_version 596096 (0.0010) [2023-12-26 19:42:23,832][105692] Updated weights for policy 0, policy_version 596106 (0.0009) [2023-12-26 19:42:24,178][105620] Updated weights for policy 1, policy_version 596811 (0.0009) [2023-12-26 19:42:24,239][105620] Updated weights for policy 1, policy_version 596821 (0.0009) [2023-12-26 19:42:24,309][105620] Updated weights for policy 1, policy_version 596831 (0.0009) [2023-12-26 19:42:24,533][105692] Updated weights for policy 0, policy_version 596116 (0.0009) [2023-12-26 19:42:24,581][105692] Updated weights for policy 0, policy_version 596126 (0.0010) [2023-12-26 19:42:24,630][105692] Updated weights for policy 0, policy_version 596136 (0.0009) [2023-12-26 19:42:25,087][105620] Updated weights for policy 1, policy_version 596841 (0.0009) [2023-12-26 19:42:25,142][105620] Updated weights for policy 1, policy_version 596851 (0.0010) [2023-12-26 19:42:25,193][105620] Updated weights for policy 1, policy_version 596861 (0.0010) [2023-12-26 19:42:25,243][105620] Updated weights for policy 1, policy_version 596871 (0.0009) [2023-12-26 19:42:25,366][105692] Updated weights for policy 0, policy_version 596146 (0.0009) [2023-12-26 19:42:25,431][105692] Updated weights for policy 0, policy_version 596156 (0.0008) [2023-12-26 19:42:25,492][105692] Updated weights for policy 0, policy_version 596166 (0.0009) [2023-12-26 19:42:25,543][105692] Updated weights for policy 0, policy_version 596176 (0.0009) [2023-12-26 19:42:25,924][105620] Updated weights for policy 1, policy_version 596881 (0.0005) [2023-12-26 19:42:25,978][105620] Updated weights for policy 1, policy_version 596891 (0.0009) [2023-12-26 19:42:26,024][105620] Updated weights for policy 1, policy_version 596901 (0.0008) [2023-12-26 19:42:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 305463296. Throughput: 0: 9859.7, 1: 9862.2. Samples: 305469632. Policy #0 lag: (min: 19.0, avg: 43.8, max: 49.0) [2023-12-26 19:42:26,063][104569] Avg episode reward: [(0, '8814.397'), (1, '9178.150')] [2023-12-26 19:42:26,329][105692] Updated weights for policy 0, policy_version 596186 (0.0009) [2023-12-26 19:42:26,376][105692] Updated weights for policy 0, policy_version 596197 (0.0009) [2023-12-26 19:42:26,425][105692] Updated weights for policy 0, policy_version 596208 (0.0009) [2023-12-26 19:42:26,736][105620] Updated weights for policy 1, policy_version 596911 (0.0009) [2023-12-26 19:42:26,783][105620] Updated weights for policy 1, policy_version 596921 (0.0009) [2023-12-26 19:42:26,837][105620] Updated weights for policy 1, policy_version 596931 (0.0009) [2023-12-26 19:42:27,293][105692] Updated weights for policy 0, policy_version 596218 (0.0009) [2023-12-26 19:42:27,349][105692] Updated weights for policy 0, policy_version 596228 (0.0007) [2023-12-26 19:42:27,397][105692] Updated weights for policy 0, policy_version 596238 (0.0009) [2023-12-26 19:42:27,488][105620] Updated weights for policy 1, policy_version 596941 (0.0009) [2023-12-26 19:42:27,537][105620] Updated weights for policy 1, policy_version 596951 (0.0008) [2023-12-26 19:42:27,600][105620] Updated weights for policy 1, policy_version 596961 (0.0009) [2023-12-26 19:42:28,140][105692] Updated weights for policy 0, policy_version 596248 (0.0008) [2023-12-26 19:42:28,197][105692] Updated weights for policy 0, policy_version 596258 (0.0009) [2023-12-26 19:42:28,260][105692] Updated weights for policy 0, policy_version 596268 (0.0009) [2023-12-26 19:42:28,350][105620] Updated weights for policy 1, policy_version 596971 (0.0009) [2023-12-26 19:42:28,412][105620] Updated weights for policy 1, policy_version 596981 (0.0008) [2023-12-26 19:42:28,476][105620] Updated weights for policy 1, policy_version 596991 (0.0008) [2023-12-26 19:42:29,070][105692] Updated weights for policy 0, policy_version 596278 (0.0009) [2023-12-26 19:42:29,127][105692] Updated weights for policy 0, policy_version 596288 (0.0005) [2023-12-26 19:42:29,179][105620] Updated weights for policy 1, policy_version 597001 (0.0008) [2023-12-26 19:42:29,181][105692] Updated weights for policy 0, policy_version 596298 (0.0006) [2023-12-26 19:42:29,245][105620] Updated weights for policy 1, policy_version 597011 (0.0011) [2023-12-26 19:42:29,307][105620] Updated weights for policy 1, policy_version 597021 (0.0009) [2023-12-26 19:42:29,371][105620] Updated weights for policy 1, policy_version 597031 (0.0010) [2023-12-26 19:42:29,945][105692] Updated weights for policy 0, policy_version 596308 (0.0009) [2023-12-26 19:42:29,982][105620] Updated weights for policy 1, policy_version 597041 (0.0011) [2023-12-26 19:42:30,008][105692] Updated weights for policy 0, policy_version 596318 (0.0007) [2023-12-26 19:42:30,039][105620] Updated weights for policy 1, policy_version 597051 (0.0010) [2023-12-26 19:42:30,069][105692] Updated weights for policy 0, policy_version 596328 (0.0006) [2023-12-26 19:42:30,098][105620] Updated weights for policy 1, policy_version 597061 (0.0010) [2023-12-26 19:42:30,706][105620] Updated weights for policy 1, policy_version 597071 (0.0011) [2023-12-26 19:42:30,757][105620] Updated weights for policy 1, policy_version 597081 (0.0010) [2023-12-26 19:42:30,811][105620] Updated weights for policy 1, policy_version 597091 (0.0010) [2023-12-26 19:42:30,864][105692] Updated weights for policy 0, policy_version 596338 (0.0006) [2023-12-26 19:42:30,921][105692] Updated weights for policy 0, policy_version 596348 (0.0010) [2023-12-26 19:42:30,974][105692] Updated weights for policy 0, policy_version 596359 (0.0010) [2023-12-26 19:42:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 305561600. Throughput: 0: 9841.6, 1: 9842.9. Samples: 305526440. Policy #0 lag: (min: 19.0, avg: 43.8, max: 49.0) [2023-12-26 19:42:31,062][104569] Avg episode reward: [(0, '8910.718'), (1, '9266.349')] [2023-12-26 19:42:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000596368_152690688.pth... [2023-12-26 19:42:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000597096_152870912.pth... [2023-12-26 19:42:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000595248_152403968.pth [2023-12-26 19:42:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000595944_152576000.pth [2023-12-26 19:42:31,471][105620] Updated weights for policy 1, policy_version 597101 (0.0010) [2023-12-26 19:42:31,530][105620] Updated weights for policy 1, policy_version 597111 (0.0010) [2023-12-26 19:42:31,587][105620] Updated weights for policy 1, policy_version 597121 (0.0010) [2023-12-26 19:42:31,858][105692] Updated weights for policy 0, policy_version 596370 (0.0008) [2023-12-26 19:42:31,914][105692] Updated weights for policy 0, policy_version 596380 (0.0008) [2023-12-26 19:42:31,970][105692] Updated weights for policy 0, policy_version 596390 (0.0008) [2023-12-26 19:42:32,023][105692] Updated weights for policy 0, policy_version 596400 (0.0007) [2023-12-26 19:42:32,364][105620] Updated weights for policy 1, policy_version 597131 (0.0009) [2023-12-26 19:42:32,415][105620] Updated weights for policy 1, policy_version 597141 (0.0010) [2023-12-26 19:42:32,467][105620] Updated weights for policy 1, policy_version 597151 (0.0010) [2023-12-26 19:42:32,803][105692] Updated weights for policy 0, policy_version 596410 (0.0010) [2023-12-26 19:42:32,855][105692] Updated weights for policy 0, policy_version 596420 (0.0009) [2023-12-26 19:42:32,915][105692] Updated weights for policy 0, policy_version 596430 (0.0010) [2023-12-26 19:42:33,065][105620] Updated weights for policy 1, policy_version 597161 (0.0010) [2023-12-26 19:42:33,129][105620] Updated weights for policy 1, policy_version 597171 (0.0006) [2023-12-26 19:42:33,199][105620] Updated weights for policy 1, policy_version 597181 (0.0006) [2023-12-26 19:42:33,265][105620] Updated weights for policy 1, policy_version 597191 (0.0006) [2023-12-26 19:42:33,795][105692] Updated weights for policy 0, policy_version 596440 (0.0009) [2023-12-26 19:42:33,845][105692] Updated weights for policy 0, policy_version 596450 (0.0007) [2023-12-26 19:42:33,858][105620] Updated weights for policy 1, policy_version 597201 (0.0010) [2023-12-26 19:42:33,887][105692] Updated weights for policy 0, policy_version 596460 (0.0008) [2023-12-26 19:42:33,912][105620] Updated weights for policy 1, policy_version 597211 (0.0010) [2023-12-26 19:42:33,956][105620] Updated weights for policy 1, policy_version 597221 (0.0010) [2023-12-26 19:42:34,611][105692] Updated weights for policy 0, policy_version 596470 (0.0008) [2023-12-26 19:42:34,671][105692] Updated weights for policy 0, policy_version 596480 (0.0008) [2023-12-26 19:42:34,706][105620] Updated weights for policy 1, policy_version 597231 (0.0010) [2023-12-26 19:42:34,728][105692] Updated weights for policy 0, policy_version 596490 (0.0006) [2023-12-26 19:42:34,758][105620] Updated weights for policy 1, policy_version 597241 (0.0010) [2023-12-26 19:42:34,813][105620] Updated weights for policy 1, policy_version 597251 (0.0010) [2023-12-26 19:42:35,467][105620] Updated weights for policy 1, policy_version 597261 (0.0007) [2023-12-26 19:42:35,523][105620] Updated weights for policy 1, policy_version 597271 (0.0005) [2023-12-26 19:42:35,557][105692] Updated weights for policy 0, policy_version 596500 (0.0007) [2023-12-26 19:42:35,580][105620] Updated weights for policy 1, policy_version 597281 (0.0006) [2023-12-26 19:42:35,619][105692] Updated weights for policy 0, policy_version 596510 (0.0009) [2023-12-26 19:42:35,673][105692] Updated weights for policy 0, policy_version 596520 (0.0010) [2023-12-26 19:42:36,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 305651712. Throughput: 0: 9581.2, 1: 9873.8. Samples: 305641776. Policy #0 lag: (min: 19.0, avg: 43.8, max: 49.0) [2023-12-26 19:42:36,062][104569] Avg episode reward: [(0, '9270.363'), (1, '9266.379')] [2023-12-26 19:42:36,087][105620] Updated weights for policy 1, policy_version 597291 (0.0005) [2023-12-26 19:42:36,152][105620] Updated weights for policy 1, policy_version 597301 (0.0007) [2023-12-26 19:42:36,214][105620] Updated weights for policy 1, policy_version 597311 (0.0006) [2023-12-26 19:42:36,552][105692] Updated weights for policy 0, policy_version 596530 (0.0011) [2023-12-26 19:42:36,610][105692] Updated weights for policy 0, policy_version 596540 (0.0008) [2023-12-26 19:42:36,659][105692] Updated weights for policy 0, policy_version 596550 (0.0008) [2023-12-26 19:42:36,708][105692] Updated weights for policy 0, policy_version 596560 (0.0008) [2023-12-26 19:42:36,937][105620] Updated weights for policy 1, policy_version 597321 (0.0011) [2023-12-26 19:42:37,002][105620] Updated weights for policy 1, policy_version 597331 (0.0011) [2023-12-26 19:42:37,050][105620] Updated weights for policy 1, policy_version 597341 (0.0010) [2023-12-26 19:42:37,105][105620] Updated weights for policy 1, policy_version 597351 (0.0010) [2023-12-26 19:42:37,539][105692] Updated weights for policy 0, policy_version 596570 (0.0010) [2023-12-26 19:42:37,598][105692] Updated weights for policy 0, policy_version 596580 (0.0010) [2023-12-26 19:42:37,652][105692] Updated weights for policy 0, policy_version 596590 (0.0009) [2023-12-26 19:42:37,725][105620] Updated weights for policy 1, policy_version 597361 (0.0011) [2023-12-26 19:42:37,787][105620] Updated weights for policy 1, policy_version 597371 (0.0009) [2023-12-26 19:42:37,846][105620] Updated weights for policy 1, policy_version 597381 (0.0005) [2023-12-26 19:42:38,431][105692] Updated weights for policy 0, policy_version 596600 (0.0009) [2023-12-26 19:42:38,480][105692] Updated weights for policy 0, policy_version 596610 (0.0007) [2023-12-26 19:42:38,527][105692] Updated weights for policy 0, policy_version 596620 (0.0009) [2023-12-26 19:42:38,569][105620] Updated weights for policy 1, policy_version 597391 (0.0007) [2023-12-26 19:42:38,627][105620] Updated weights for policy 1, policy_version 597401 (0.0008) [2023-12-26 19:42:38,678][105620] Updated weights for policy 1, policy_version 597411 (0.0010) [2023-12-26 19:42:39,277][105692] Updated weights for policy 0, policy_version 596630 (0.0009) [2023-12-26 19:42:39,333][105692] Updated weights for policy 0, policy_version 596640 (0.0009) [2023-12-26 19:42:39,400][105692] Updated weights for policy 0, policy_version 596650 (0.0008) [2023-12-26 19:42:39,508][105620] Updated weights for policy 1, policy_version 597421 (0.0009) [2023-12-26 19:42:39,563][105620] Updated weights for policy 1, policy_version 597431 (0.0009) [2023-12-26 19:42:39,614][105620] Updated weights for policy 1, policy_version 597441 (0.0009) [2023-12-26 19:42:40,183][105692] Updated weights for policy 0, policy_version 596660 (0.0009) [2023-12-26 19:42:40,249][105692] Updated weights for policy 0, policy_version 596670 (0.0011) [2023-12-26 19:42:40,315][105692] Updated weights for policy 0, policy_version 596680 (0.0010) [2023-12-26 19:42:40,428][105620] Updated weights for policy 1, policy_version 597451 (0.0009) [2023-12-26 19:42:40,494][105620] Updated weights for policy 1, policy_version 597461 (0.0008) [2023-12-26 19:42:40,550][105620] Updated weights for policy 1, policy_version 597471 (0.0008) [2023-12-26 19:42:41,058][105692] Updated weights for policy 0, policy_version 596690 (0.0010) [2023-12-26 19:42:41,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 305741824. Throughput: 0: 9448.9, 1: 9810.9. Samples: 305754388. Policy #0 lag: (min: 19.0, avg: 43.8, max: 49.0) [2023-12-26 19:42:41,063][104569] Avg episode reward: [(0, '9357.850'), (1, '9085.216')] [2023-12-26 19:42:41,122][105692] Updated weights for policy 0, policy_version 596700 (0.0010) [2023-12-26 19:42:41,190][105692] Updated weights for policy 0, policy_version 596710 (0.0011) [2023-12-26 19:42:41,251][105692] Updated weights for policy 0, policy_version 596720 (0.0011) [2023-12-26 19:42:41,339][105620] Updated weights for policy 1, policy_version 597481 (0.0009) [2023-12-26 19:42:41,410][105620] Updated weights for policy 1, policy_version 597491 (0.0009) [2023-12-26 19:42:41,460][105620] Updated weights for policy 1, policy_version 597501 (0.0009) [2023-12-26 19:42:41,524][105620] Updated weights for policy 1, policy_version 597511 (0.0007) [2023-12-26 19:42:42,037][105692] Updated weights for policy 0, policy_version 596730 (0.0011) [2023-12-26 19:42:42,098][105692] Updated weights for policy 0, policy_version 596740 (0.0011) [2023-12-26 19:42:42,147][105692] Updated weights for policy 0, policy_version 596750 (0.0010) [2023-12-26 19:42:42,232][105620] Updated weights for policy 1, policy_version 597521 (0.0008) [2023-12-26 19:42:42,297][105620] Updated weights for policy 1, policy_version 597531 (0.0008) [2023-12-26 19:42:42,356][105620] Updated weights for policy 1, policy_version 597541 (0.0008) [2023-12-26 19:42:42,914][105692] Updated weights for policy 0, policy_version 596760 (0.0010) [2023-12-26 19:42:42,979][105692] Updated weights for policy 0, policy_version 596770 (0.0010) [2023-12-26 19:42:43,037][105692] Updated weights for policy 0, policy_version 596780 (0.0010) [2023-12-26 19:42:43,145][105620] Updated weights for policy 1, policy_version 597551 (0.0009) [2023-12-26 19:42:43,197][105620] Updated weights for policy 1, policy_version 597561 (0.0010) [2023-12-26 19:42:43,245][105620] Updated weights for policy 1, policy_version 597571 (0.0008) [2023-12-26 19:42:43,699][105692] Updated weights for policy 0, policy_version 596790 (0.0008) [2023-12-26 19:42:43,751][105692] Updated weights for policy 0, policy_version 596800 (0.0005) [2023-12-26 19:42:43,818][105692] Updated weights for policy 0, policy_version 596810 (0.0005) [2023-12-26 19:42:43,994][105620] Updated weights for policy 1, policy_version 597581 (0.0007) [2023-12-26 19:42:44,042][105620] Updated weights for policy 1, policy_version 597591 (0.0008) [2023-12-26 19:42:44,109][105620] Updated weights for policy 1, policy_version 597601 (0.0008) [2023-12-26 19:42:44,466][105692] Updated weights for policy 0, policy_version 596820 (0.0010) [2023-12-26 19:42:44,525][105692] Updated weights for policy 0, policy_version 596830 (0.0011) [2023-12-26 19:42:44,570][105692] Updated weights for policy 0, policy_version 596840 (0.0010) [2023-12-26 19:42:44,886][105620] Updated weights for policy 1, policy_version 597611 (0.0009) [2023-12-26 19:42:44,935][105620] Updated weights for policy 1, policy_version 597621 (0.0010) [2023-12-26 19:42:44,987][105620] Updated weights for policy 1, policy_version 597631 (0.0010) [2023-12-26 19:42:45,334][105692] Updated weights for policy 0, policy_version 596850 (0.0010) [2023-12-26 19:42:45,403][105692] Updated weights for policy 0, policy_version 596860 (0.0010) [2023-12-26 19:42:45,448][105692] Updated weights for policy 0, policy_version 596870 (0.0010) [2023-12-26 19:42:45,514][105692] Updated weights for policy 0, policy_version 596880 (0.0010) [2023-12-26 19:42:45,753][105620] Updated weights for policy 1, policy_version 597641 (0.0007) [2023-12-26 19:42:45,821][105620] Updated weights for policy 1, policy_version 597651 (0.0007) [2023-12-26 19:42:45,879][105620] Updated weights for policy 1, policy_version 597661 (0.0010) [2023-12-26 19:42:45,930][105620] Updated weights for policy 1, policy_version 597671 (0.0010) [2023-12-26 19:42:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 305840128. Throughput: 0: 9360.3, 1: 9821.0. Samples: 305810664. Policy #0 lag: (min: 19.0, avg: 43.8, max: 49.0) [2023-12-26 19:42:46,063][104569] Avg episode reward: [(0, '3839.123'), (1, '8993.336')] [2023-12-26 19:42:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000596880_152821760.pth... [2023-12-26 19:42:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000597672_153018368.pth... [2023-12-26 19:42:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000595824_152551424.pth [2023-12-26 19:42:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000596520_152723456.pth [2023-12-26 19:42:46,267][105692] Updated weights for policy 0, policy_version 596890 (0.0010) [2023-12-26 19:42:46,325][105692] Updated weights for policy 0, policy_version 596900 (0.0010) [2023-12-26 19:42:46,373][105692] Updated weights for policy 0, policy_version 596910 (0.0010) [2023-12-26 19:42:46,594][105620] Updated weights for policy 1, policy_version 597681 (0.0006) [2023-12-26 19:42:46,664][105620] Updated weights for policy 1, policy_version 597691 (0.0006) [2023-12-26 19:42:46,720][105620] Updated weights for policy 1, policy_version 597701 (0.0005) [2023-12-26 19:42:47,109][105692] Updated weights for policy 0, policy_version 596920 (0.0009) [2023-12-26 19:42:47,166][105692] Updated weights for policy 0, policy_version 596930 (0.0008) [2023-12-26 19:42:47,212][105692] Updated weights for policy 0, policy_version 596940 (0.0008) [2023-12-26 19:42:47,334][105620] Updated weights for policy 1, policy_version 597711 (0.0006) [2023-12-26 19:42:47,402][105620] Updated weights for policy 1, policy_version 597721 (0.0008) [2023-12-26 19:42:47,454][105620] Updated weights for policy 1, policy_version 597731 (0.0009) [2023-12-26 19:42:47,970][105692] Updated weights for policy 0, policy_version 596950 (0.0006) [2023-12-26 19:42:48,028][105692] Updated weights for policy 0, policy_version 596960 (0.0005) [2023-12-26 19:42:48,086][105692] Updated weights for policy 0, policy_version 596970 (0.0006) [2023-12-26 19:42:48,183][105620] Updated weights for policy 1, policy_version 597741 (0.0009) [2023-12-26 19:42:48,250][105620] Updated weights for policy 1, policy_version 597751 (0.0009) [2023-12-26 19:42:48,312][105620] Updated weights for policy 1, policy_version 597761 (0.0009) [2023-12-26 19:42:48,807][105692] Updated weights for policy 0, policy_version 596980 (0.0009) [2023-12-26 19:42:48,859][105692] Updated weights for policy 0, policy_version 596990 (0.0009) [2023-12-26 19:42:48,907][105692] Updated weights for policy 0, policy_version 597000 (0.0009) [2023-12-26 19:42:48,981][105620] Updated weights for policy 1, policy_version 597771 (0.0009) [2023-12-26 19:42:49,038][105620] Updated weights for policy 1, policy_version 597781 (0.0009) [2023-12-26 19:42:49,100][105620] Updated weights for policy 1, policy_version 597791 (0.0008) [2023-12-26 19:42:49,697][105692] Updated weights for policy 0, policy_version 597010 (0.0009) [2023-12-26 19:42:49,761][105692] Updated weights for policy 0, policy_version 597020 (0.0008) [2023-12-26 19:42:49,820][105692] Updated weights for policy 0, policy_version 597030 (0.0009) [2023-12-26 19:42:49,835][105620] Updated weights for policy 1, policy_version 597801 (0.0008) [2023-12-26 19:42:49,877][105692] Updated weights for policy 0, policy_version 597040 (0.0009) [2023-12-26 19:42:49,893][105620] Updated weights for policy 1, policy_version 597811 (0.0008) [2023-12-26 19:42:49,954][105620] Updated weights for policy 1, policy_version 597821 (0.0010) [2023-12-26 19:42:50,024][105620] Updated weights for policy 1, policy_version 597831 (0.0009) [2023-12-26 19:42:50,604][105692] Updated weights for policy 0, policy_version 597050 (0.0009) [2023-12-26 19:42:50,646][105620] Updated weights for policy 1, policy_version 597841 (0.0009) [2023-12-26 19:42:50,668][105692] Updated weights for policy 0, policy_version 597060 (0.0011) [2023-12-26 19:42:50,706][105620] Updated weights for policy 1, policy_version 597851 (0.0010) [2023-12-26 19:42:50,724][105692] Updated weights for policy 0, policy_version 597070 (0.0011) [2023-12-26 19:42:50,761][105620] Updated weights for policy 1, policy_version 597861 (0.0009) [2023-12-26 19:42:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 305938432. Throughput: 0: 9401.1, 1: 9782.5. Samples: 305925852. Policy #0 lag: (min: 19.0, avg: 43.8, max: 49.0) [2023-12-26 19:42:51,063][104569] Avg episode reward: [(0, '4851.418'), (1, '9173.780')] [2023-12-26 19:42:51,443][105692] Updated weights for policy 0, policy_version 597080 (0.0011) [2023-12-26 19:42:51,469][105620] Updated weights for policy 1, policy_version 597871 (0.0008) [2023-12-26 19:42:51,491][105692] Updated weights for policy 0, policy_version 597090 (0.0007) [2023-12-26 19:42:51,530][105620] Updated weights for policy 1, policy_version 597881 (0.0009) [2023-12-26 19:42:51,540][105692] Updated weights for policy 0, policy_version 597100 (0.0006) [2023-12-26 19:42:51,592][105620] Updated weights for policy 1, policy_version 597891 (0.0009) [2023-12-26 19:42:52,247][105692] Updated weights for policy 0, policy_version 597110 (0.0008) [2023-12-26 19:42:52,308][105692] Updated weights for policy 0, policy_version 597120 (0.0008) [2023-12-26 19:42:52,372][105692] Updated weights for policy 0, policy_version 597130 (0.0009) [2023-12-26 19:42:52,378][105620] Updated weights for policy 1, policy_version 597901 (0.0007) [2023-12-26 19:42:52,437][105620] Updated weights for policy 1, policy_version 597911 (0.0008) [2023-12-26 19:42:52,488][105620] Updated weights for policy 1, policy_version 597921 (0.0009) [2023-12-26 19:42:53,054][105692] Updated weights for policy 0, policy_version 597140 (0.0008) [2023-12-26 19:42:53,109][105692] Updated weights for policy 0, policy_version 597150 (0.0010) [2023-12-26 19:42:53,168][105692] Updated weights for policy 0, policy_version 597160 (0.0010) [2023-12-26 19:42:53,282][105620] Updated weights for policy 1, policy_version 597931 (0.0006) [2023-12-26 19:42:53,329][105620] Updated weights for policy 1, policy_version 597941 (0.0007) [2023-12-26 19:42:53,373][105620] Updated weights for policy 1, policy_version 597951 (0.0008) [2023-12-26 19:42:53,912][105692] Updated weights for policy 0, policy_version 597170 (0.0010) [2023-12-26 19:42:53,963][105692] Updated weights for policy 0, policy_version 597180 (0.0010) [2023-12-26 19:42:54,015][105692] Updated weights for policy 0, policy_version 597190 (0.0010) [2023-12-26 19:42:54,078][105692] Updated weights for policy 0, policy_version 597200 (0.0009) [2023-12-26 19:42:54,134][105620] Updated weights for policy 1, policy_version 597961 (0.0008) [2023-12-26 19:42:54,197][105620] Updated weights for policy 1, policy_version 597971 (0.0009) [2023-12-26 19:42:54,266][105620] Updated weights for policy 1, policy_version 597981 (0.0009) [2023-12-26 19:42:54,333][105620] Updated weights for policy 1, policy_version 597991 (0.0011) [2023-12-26 19:42:54,777][105692] Updated weights for policy 0, policy_version 597210 (0.0009) [2023-12-26 19:42:54,825][105692] Updated weights for policy 0, policy_version 597220 (0.0008) [2023-12-26 19:42:54,881][105692] Updated weights for policy 0, policy_version 597230 (0.0008) [2023-12-26 19:42:55,057][105620] Updated weights for policy 1, policy_version 598001 (0.0006) [2023-12-26 19:42:55,123][105620] Updated weights for policy 1, policy_version 598011 (0.0008) [2023-12-26 19:42:55,190][105620] Updated weights for policy 1, policy_version 598021 (0.0007) [2023-12-26 19:42:55,698][105692] Updated weights for policy 0, policy_version 597240 (0.0009) [2023-12-26 19:42:55,733][105620] Updated weights for policy 1, policy_version 598031 (0.0009) [2023-12-26 19:42:55,761][105692] Updated weights for policy 0, policy_version 597250 (0.0010) [2023-12-26 19:42:55,781][105620] Updated weights for policy 1, policy_version 598041 (0.0010) [2023-12-26 19:42:55,823][105692] Updated weights for policy 0, policy_version 597260 (0.0010) [2023-12-26 19:42:55,830][105620] Updated weights for policy 1, policy_version 598051 (0.0010) [2023-12-26 19:42:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 306036736. Throughput: 0: 9324.2, 1: 9763.6. Samples: 306042632. Policy #0 lag: (min: 19.0, avg: 43.8, max: 49.0) [2023-12-26 19:42:56,063][104569] Avg episode reward: [(0, '7211.121'), (1, '9356.289')] [2023-12-26 19:42:56,387][105692] Updated weights for policy 0, policy_version 597270 (0.0007) [2023-12-26 19:42:56,448][105692] Updated weights for policy 0, policy_version 597280 (0.0005) [2023-12-26 19:42:56,500][105692] Updated weights for policy 0, policy_version 597290 (0.0006) [2023-12-26 19:42:56,593][105620] Updated weights for policy 1, policy_version 598061 (0.0008) [2023-12-26 19:42:56,655][105620] Updated weights for policy 1, policy_version 598071 (0.0007) [2023-12-26 19:42:56,723][105620] Updated weights for policy 1, policy_version 598081 (0.0010) [2023-12-26 19:42:57,051][105692] Updated weights for policy 0, policy_version 597300 (0.0008) [2023-12-26 19:42:57,108][105692] Updated weights for policy 0, policy_version 597310 (0.0008) [2023-12-26 19:42:57,165][105692] Updated weights for policy 0, policy_version 597320 (0.0006) [2023-12-26 19:42:57,366][105620] Updated weights for policy 1, policy_version 598091 (0.0010) [2023-12-26 19:42:57,431][105620] Updated weights for policy 1, policy_version 598101 (0.0010) [2023-12-26 19:42:57,494][105620] Updated weights for policy 1, policy_version 598111 (0.0010) [2023-12-26 19:42:57,864][105692] Updated weights for policy 0, policy_version 597330 (0.0008) [2023-12-26 19:42:57,908][105692] Updated weights for policy 0, policy_version 597340 (0.0010) [2023-12-26 19:42:57,957][105692] Updated weights for policy 0, policy_version 597350 (0.0010) [2023-12-26 19:42:58,005][105692] Updated weights for policy 0, policy_version 597360 (0.0010) [2023-12-26 19:42:58,226][105620] Updated weights for policy 1, policy_version 598121 (0.0008) [2023-12-26 19:42:58,289][105620] Updated weights for policy 1, policy_version 598131 (0.0011) [2023-12-26 19:42:58,355][105620] Updated weights for policy 1, policy_version 598141 (0.0008) [2023-12-26 19:42:58,422][105620] Updated weights for policy 1, policy_version 598151 (0.0009) [2023-12-26 19:42:58,831][105692] Updated weights for policy 0, policy_version 597370 (0.0008) [2023-12-26 19:42:58,905][105692] Updated weights for policy 0, policy_version 597380 (0.0008) [2023-12-26 19:42:58,963][105692] Updated weights for policy 0, policy_version 597390 (0.0008) [2023-12-26 19:42:59,199][105620] Updated weights for policy 1, policy_version 598161 (0.0008) [2023-12-26 19:42:59,264][105620] Updated weights for policy 1, policy_version 598171 (0.0008) [2023-12-26 19:42:59,329][105620] Updated weights for policy 1, policy_version 598181 (0.0008) [2023-12-26 19:42:59,801][105692] Updated weights for policy 0, policy_version 597400 (0.0009) [2023-12-26 19:42:59,859][105692] Updated weights for policy 0, policy_version 597410 (0.0008) [2023-12-26 19:42:59,925][105692] Updated weights for policy 0, policy_version 597420 (0.0007) [2023-12-26 19:42:59,961][105620] Updated weights for policy 1, policy_version 598191 (0.0008) [2023-12-26 19:43:00,027][105620] Updated weights for policy 1, policy_version 598201 (0.0011) [2023-12-26 19:43:00,086][105620] Updated weights for policy 1, policy_version 598211 (0.0010) [2023-12-26 19:43:00,654][105692] Updated weights for policy 0, policy_version 597430 (0.0008) [2023-12-26 19:43:00,701][105692] Updated weights for policy 0, policy_version 597440 (0.0008) [2023-12-26 19:43:00,752][105692] Updated weights for policy 0, policy_version 597451 (0.0010) [2023-12-26 19:43:00,796][105620] Updated weights for policy 1, policy_version 598221 (0.0008) [2023-12-26 19:43:00,843][105620] Updated weights for policy 1, policy_version 598231 (0.0005) [2023-12-26 19:43:00,899][105620] Updated weights for policy 1, policy_version 598241 (0.0005) [2023-12-26 19:43:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 306135040. Throughput: 0: 9422.6, 1: 9768.3. Samples: 306103280. Policy #0 lag: (min: 19.0, avg: 43.8, max: 49.0) [2023-12-26 19:43:01,063][104569] Avg episode reward: [(0, '9086.965'), (1, '9036.799')] [2023-12-26 19:43:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000598248_153165824.pth... [2023-12-26 19:43:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000597456_152969216.pth... [2023-12-26 19:43:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000596368_152690688.pth [2023-12-26 19:43:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000597096_152870912.pth [2023-12-26 19:43:01,413][105692] Updated weights for policy 0, policy_version 597461 (0.0010) [2023-12-26 19:43:01,464][105692] Updated weights for policy 0, policy_version 597471 (0.0008) [2023-12-26 19:43:01,512][105692] Updated weights for policy 0, policy_version 597481 (0.0007) [2023-12-26 19:43:01,605][105620] Updated weights for policy 1, policy_version 598251 (0.0007) [2023-12-26 19:43:01,673][105620] Updated weights for policy 1, policy_version 598261 (0.0011) [2023-12-26 19:43:01,735][105620] Updated weights for policy 1, policy_version 598271 (0.0011) [2023-12-26 19:43:02,209][105692] Updated weights for policy 0, policy_version 597491 (0.0007) [2023-12-26 19:43:02,274][105692] Updated weights for policy 0, policy_version 597501 (0.0007) [2023-12-26 19:43:02,331][105692] Updated weights for policy 0, policy_version 597511 (0.0008) [2023-12-26 19:43:02,495][105620] Updated weights for policy 1, policy_version 598281 (0.0011) [2023-12-26 19:43:02,561][105620] Updated weights for policy 1, policy_version 598291 (0.0011) [2023-12-26 19:43:02,613][105620] Updated weights for policy 1, policy_version 598301 (0.0010) [2023-12-26 19:43:02,666][105620] Updated weights for policy 1, policy_version 598311 (0.0010) [2023-12-26 19:43:03,057][105692] Updated weights for policy 0, policy_version 597521 (0.0008) [2023-12-26 19:43:03,104][105692] Updated weights for policy 0, policy_version 597531 (0.0005) [2023-12-26 19:43:03,149][105692] Updated weights for policy 0, policy_version 597541 (0.0005) [2023-12-26 19:43:03,193][105692] Updated weights for policy 0, policy_version 597551 (0.0007) [2023-12-26 19:43:03,406][105620] Updated weights for policy 1, policy_version 598321 (0.0010) [2023-12-26 19:43:03,460][105620] Updated weights for policy 1, policy_version 598331 (0.0010) [2023-12-26 19:43:03,504][105620] Updated weights for policy 1, policy_version 598341 (0.0010) [2023-12-26 19:43:03,935][105692] Updated weights for policy 0, policy_version 597561 (0.0009) [2023-12-26 19:43:03,998][105692] Updated weights for policy 0, policy_version 597571 (0.0008) [2023-12-26 19:43:04,060][105692] Updated weights for policy 0, policy_version 597581 (0.0009) [2023-12-26 19:43:04,192][105620] Updated weights for policy 1, policy_version 598351 (0.0007) [2023-12-26 19:43:04,258][105620] Updated weights for policy 1, policy_version 598361 (0.0006) [2023-12-26 19:43:04,320][105620] Updated weights for policy 1, policy_version 598371 (0.0006) [2023-12-26 19:43:04,853][105620] Updated weights for policy 1, policy_version 598381 (0.0008) [2023-12-26 19:43:04,905][105692] Updated weights for policy 0, policy_version 597591 (0.0007) [2023-12-26 19:43:04,912][105620] Updated weights for policy 1, policy_version 598391 (0.0010) [2023-12-26 19:43:04,950][105692] Updated weights for policy 0, policy_version 597601 (0.0005) [2023-12-26 19:43:04,964][105620] Updated weights for policy 1, policy_version 598401 (0.0010) [2023-12-26 19:43:05,002][105692] Updated weights for policy 0, policy_version 597611 (0.0007) [2023-12-26 19:43:05,683][105692] Updated weights for policy 0, policy_version 597621 (0.0009) [2023-12-26 19:43:05,712][105620] Updated weights for policy 1, policy_version 598411 (0.0010) [2023-12-26 19:43:05,736][105692] Updated weights for policy 0, policy_version 597631 (0.0008) [2023-12-26 19:43:05,767][105620] Updated weights for policy 1, policy_version 598421 (0.0010) [2023-12-26 19:43:05,791][105692] Updated weights for policy 0, policy_version 597641 (0.0010) [2023-12-26 19:43:05,833][105620] Updated weights for policy 1, policy_version 598431 (0.0011) [2023-12-26 19:43:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19466.7). Total num frames: 306233344. Throughput: 0: 9369.9, 1: 9809.0. Samples: 306220076. Policy #0 lag: (min: 19.0, avg: 43.8, max: 49.0) [2023-12-26 19:43:06,063][104569] Avg episode reward: [(0, '9085.830'), (1, '8945.000')] [2023-12-26 19:43:06,426][105692] Updated weights for policy 0, policy_version 597651 (0.0011) [2023-12-26 19:43:06,482][105692] Updated weights for policy 0, policy_version 597661 (0.0011) [2023-12-26 19:43:06,532][105692] Updated weights for policy 0, policy_version 597671 (0.0011) [2023-12-26 19:43:06,581][105620] Updated weights for policy 1, policy_version 598441 (0.0010) [2023-12-26 19:43:06,644][105620] Updated weights for policy 1, policy_version 598451 (0.0011) [2023-12-26 19:43:06,704][105620] Updated weights for policy 1, policy_version 598461 (0.0010) [2023-12-26 19:43:06,758][105620] Updated weights for policy 1, policy_version 598471 (0.0008) [2023-12-26 19:43:07,290][105692] Updated weights for policy 0, policy_version 597681 (0.0011) [2023-12-26 19:43:07,349][105692] Updated weights for policy 0, policy_version 597691 (0.0011) [2023-12-26 19:43:07,407][105692] Updated weights for policy 0, policy_version 597701 (0.0010) [2023-12-26 19:43:07,463][105692] Updated weights for policy 0, policy_version 597711 (0.0011) [2023-12-26 19:43:07,500][105620] Updated weights for policy 1, policy_version 598481 (0.0008) [2023-12-26 19:43:07,551][105620] Updated weights for policy 1, policy_version 598491 (0.0008) [2023-12-26 19:43:07,606][105620] Updated weights for policy 1, policy_version 598501 (0.0007) [2023-12-26 19:43:08,186][105692] Updated weights for policy 0, policy_version 597721 (0.0006) [2023-12-26 19:43:08,231][105620] Updated weights for policy 1, policy_version 598511 (0.0009) [2023-12-26 19:43:08,246][105692] Updated weights for policy 0, policy_version 597731 (0.0005) [2023-12-26 19:43:08,280][105620] Updated weights for policy 1, policy_version 598521 (0.0008) [2023-12-26 19:43:08,300][105692] Updated weights for policy 0, policy_version 597741 (0.0005) [2023-12-26 19:43:08,347][105620] Updated weights for policy 1, policy_version 598531 (0.0008) [2023-12-26 19:43:09,000][105692] Updated weights for policy 0, policy_version 597751 (0.0009) [2023-12-26 19:43:09,015][105620] Updated weights for policy 1, policy_version 598541 (0.0006) [2023-12-26 19:43:09,063][105692] Updated weights for policy 0, policy_version 597761 (0.0011) [2023-12-26 19:43:09,076][105620] Updated weights for policy 1, policy_version 598551 (0.0006) [2023-12-26 19:43:09,112][105692] Updated weights for policy 0, policy_version 597771 (0.0011) [2023-12-26 19:43:09,134][105620] Updated weights for policy 1, policy_version 598561 (0.0008) [2023-12-26 19:43:09,805][105692] Updated weights for policy 0, policy_version 597781 (0.0011) [2023-12-26 19:43:09,868][105692] Updated weights for policy 0, policy_version 597791 (0.0009) [2023-12-26 19:43:09,909][105620] Updated weights for policy 1, policy_version 598571 (0.0008) [2023-12-26 19:43:09,928][105692] Updated weights for policy 0, policy_version 597801 (0.0009) [2023-12-26 19:43:09,973][105620] Updated weights for policy 1, policy_version 598581 (0.0007) [2023-12-26 19:43:10,033][105620] Updated weights for policy 1, policy_version 598591 (0.0007) [2023-12-26 19:43:10,578][105692] Updated weights for policy 0, policy_version 597811 (0.0010) [2023-12-26 19:43:10,634][105692] Updated weights for policy 0, policy_version 597821 (0.0005) [2023-12-26 19:43:10,635][105620] Updated weights for policy 1, policy_version 598601 (0.0006) [2023-12-26 19:43:10,685][105692] Updated weights for policy 0, policy_version 597831 (0.0005) [2023-12-26 19:43:10,696][105620] Updated weights for policy 1, policy_version 598611 (0.0009) [2023-12-26 19:43:10,767][105620] Updated weights for policy 1, policy_version 598621 (0.0009) [2023-12-26 19:43:10,833][105620] Updated weights for policy 1, policy_version 598631 (0.0009) [2023-12-26 19:43:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 306331648. Throughput: 0: 9446.0, 1: 9865.4. Samples: 306338648. Policy #0 lag: (min: 19.0, avg: 43.8, max: 49.0) [2023-12-26 19:43:11,062][104569] Avg episode reward: [(0, '9170.119'), (1, '8927.280')] [2023-12-26 19:43:11,331][105692] Updated weights for policy 0, policy_version 597841 (0.0006) [2023-12-26 19:43:11,401][105692] Updated weights for policy 0, policy_version 597851 (0.0008) [2023-12-26 19:43:11,467][105692] Updated weights for policy 0, policy_version 597861 (0.0008) [2023-12-26 19:43:11,530][105692] Updated weights for policy 0, policy_version 597871 (0.0008) [2023-12-26 19:43:11,551][105620] Updated weights for policy 1, policy_version 598641 (0.0009) [2023-12-26 19:43:11,613][105620] Updated weights for policy 1, policy_version 598651 (0.0007) [2023-12-26 19:43:11,673][105620] Updated weights for policy 1, policy_version 598661 (0.0007) [2023-12-26 19:43:12,320][105620] Updated weights for policy 1, policy_version 598671 (0.0007) [2023-12-26 19:43:12,328][105692] Updated weights for policy 0, policy_version 597881 (0.0008) [2023-12-26 19:43:12,393][105620] Updated weights for policy 1, policy_version 598681 (0.0009) [2023-12-26 19:43:12,402][105692] Updated weights for policy 0, policy_version 597891 (0.0008) [2023-12-26 19:43:12,455][105692] Updated weights for policy 0, policy_version 597901 (0.0007) [2023-12-26 19:43:12,457][105620] Updated weights for policy 1, policy_version 598691 (0.0007) [2023-12-26 19:43:13,123][105620] Updated weights for policy 1, policy_version 598701 (0.0008) [2023-12-26 19:43:13,171][105620] Updated weights for policy 1, policy_version 598711 (0.0009) [2023-12-26 19:43:13,231][105692] Updated weights for policy 0, policy_version 597911 (0.0006) [2023-12-26 19:43:13,237][105620] Updated weights for policy 1, policy_version 598721 (0.0008) [2023-12-26 19:43:13,288][105692] Updated weights for policy 0, policy_version 597921 (0.0007) [2023-12-26 19:43:13,348][105692] Updated weights for policy 0, policy_version 597931 (0.0009) [2023-12-26 19:43:13,816][105620] Updated weights for policy 1, policy_version 598731 (0.0006) [2023-12-26 19:43:13,878][105620] Updated weights for policy 1, policy_version 598741 (0.0005) [2023-12-26 19:43:13,938][105620] Updated weights for policy 1, policy_version 598751 (0.0005) [2023-12-26 19:43:14,149][105692] Updated weights for policy 0, policy_version 597941 (0.0010) [2023-12-26 19:43:14,204][105692] Updated weights for policy 0, policy_version 597951 (0.0009) [2023-12-26 19:43:14,260][105692] Updated weights for policy 0, policy_version 597961 (0.0009) [2023-12-26 19:43:14,586][105620] Updated weights for policy 1, policy_version 598761 (0.0005) [2023-12-26 19:43:14,649][105620] Updated weights for policy 1, policy_version 598771 (0.0008) [2023-12-26 19:43:14,711][105620] Updated weights for policy 1, policy_version 598781 (0.0006) [2023-12-26 19:43:14,776][105620] Updated weights for policy 1, policy_version 598791 (0.0006) [2023-12-26 19:43:14,913][105692] Updated weights for policy 0, policy_version 597971 (0.0006) [2023-12-26 19:43:14,974][105692] Updated weights for policy 0, policy_version 597981 (0.0009) [2023-12-26 19:43:15,034][105692] Updated weights for policy 0, policy_version 597991 (0.0010) [2023-12-26 19:43:15,487][105620] Updated weights for policy 1, policy_version 598801 (0.0009) [2023-12-26 19:43:15,551][105620] Updated weights for policy 1, policy_version 598811 (0.0009) [2023-12-26 19:43:15,610][105620] Updated weights for policy 1, policy_version 598821 (0.0009) [2023-12-26 19:43:15,798][105692] Updated weights for policy 0, policy_version 598001 (0.0009) [2023-12-26 19:43:15,849][105692] Updated weights for policy 0, policy_version 598011 (0.0009) [2023-12-26 19:43:15,906][105692] Updated weights for policy 0, policy_version 598021 (0.0009) [2023-12-26 19:43:15,969][105692] Updated weights for policy 0, policy_version 598031 (0.0008) [2023-12-26 19:43:16,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 306429952. Throughput: 0: 9444.3, 1: 9911.4. Samples: 306397452. Policy #0 lag: (min: 19.0, avg: 43.8, max: 49.0) [2023-12-26 19:43:16,063][104569] Avg episode reward: [(0, '8700.452'), (1, '8548.810')] [2023-12-26 19:43:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000598032_153116672.pth... [2023-12-26 19:43:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000598824_153313280.pth... [2023-12-26 19:43:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000596880_152821760.pth [2023-12-26 19:43:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000597672_153018368.pth [2023-12-26 19:43:16,321][105620] Updated weights for policy 1, policy_version 598831 (0.0009) [2023-12-26 19:43:16,375][105620] Updated weights for policy 1, policy_version 598841 (0.0009) [2023-12-26 19:43:16,422][105620] Updated weights for policy 1, policy_version 598851 (0.0009) [2023-12-26 19:43:16,744][105692] Updated weights for policy 0, policy_version 598041 (0.0009) [2023-12-26 19:43:16,808][105692] Updated weights for policy 0, policy_version 598051 (0.0009) [2023-12-26 19:43:16,868][105692] Updated weights for policy 0, policy_version 598061 (0.0009) [2023-12-26 19:43:17,196][105620] Updated weights for policy 1, policy_version 598861 (0.0009) [2023-12-26 19:43:17,254][105620] Updated weights for policy 1, policy_version 598871 (0.0009) [2023-12-26 19:43:17,308][105620] Updated weights for policy 1, policy_version 598881 (0.0008) [2023-12-26 19:43:17,563][105692] Updated weights for policy 0, policy_version 598071 (0.0006) [2023-12-26 19:43:17,570][105585] KL-divergence is very high: 108.3434 [2023-12-26 19:43:17,608][105585] KL-divergence is very high: 122.2246 [2023-12-26 19:43:17,613][105692] Updated weights for policy 0, policy_version 598081 (0.0005) [2023-12-26 19:43:17,663][105692] Updated weights for policy 0, policy_version 598091 (0.0005) [2023-12-26 19:43:18,011][105620] Updated weights for policy 1, policy_version 598891 (0.0008) [2023-12-26 19:43:18,074][105620] Updated weights for policy 1, policy_version 598901 (0.0005) [2023-12-26 19:43:18,132][105620] Updated weights for policy 1, policy_version 598911 (0.0006) [2023-12-26 19:43:18,235][105692] Updated weights for policy 0, policy_version 598101 (0.0008) [2023-12-26 19:43:18,258][105585] KL-divergence is very high: 130.3311 [2023-12-26 19:43:18,264][105585] KL-divergence is very high: 269.6520 [2023-12-26 19:43:18,293][105692] Updated weights for policy 0, policy_version 598111 (0.0011) [2023-12-26 19:43:18,307][105585] KL-divergence is very high: 154.1785 [2023-12-26 19:43:18,313][105585] KL-divergence is very high: 300.8562 [2023-12-26 19:43:18,361][105585] KL-divergence is very high: 251.4738 [2023-12-26 19:43:18,362][105692] Updated weights for policy 0, policy_version 598121 (0.0011) [2023-12-26 19:43:18,368][105585] KL-divergence is very high: 402.5565 [2023-12-26 19:43:18,894][105620] Updated weights for policy 1, policy_version 598921 (0.0008) [2023-12-26 19:43:18,961][105620] Updated weights for policy 1, policy_version 598931 (0.0008) [2023-12-26 19:43:18,989][105692] Updated weights for policy 0, policy_version 598131 (0.0009) [2023-12-26 19:43:19,011][105620] Updated weights for policy 1, policy_version 598941 (0.0007) [2023-12-26 19:43:19,054][105692] Updated weights for policy 0, policy_version 598141 (0.0006) [2023-12-26 19:43:19,061][105620] Updated weights for policy 1, policy_version 598951 (0.0008) [2023-12-26 19:43:19,115][105692] Updated weights for policy 0, policy_version 598151 (0.0005) [2023-12-26 19:43:19,772][105692] Updated weights for policy 0, policy_version 598161 (0.0006) [2023-12-26 19:43:19,819][105620] Updated weights for policy 1, policy_version 598961 (0.0006) [2023-12-26 19:43:19,836][105692] Updated weights for policy 0, policy_version 598171 (0.0010) [2023-12-26 19:43:19,882][105620] Updated weights for policy 1, policy_version 598971 (0.0007) [2023-12-26 19:43:19,903][105692] Updated weights for policy 0, policy_version 598181 (0.0010) [2023-12-26 19:43:19,949][105620] Updated weights for policy 1, policy_version 598981 (0.0008) [2023-12-26 19:43:19,971][105692] Updated weights for policy 0, policy_version 598191 (0.0009) [2023-12-26 19:43:20,709][105620] Updated weights for policy 1, policy_version 598991 (0.0008) [2023-12-26 19:43:20,745][105692] Updated weights for policy 0, policy_version 598201 (0.0011) [2023-12-26 19:43:20,778][105620] Updated weights for policy 1, policy_version 599001 (0.0009) [2023-12-26 19:43:20,808][105692] Updated weights for policy 0, policy_version 598211 (0.0010) [2023-12-26 19:43:20,842][105620] Updated weights for policy 1, policy_version 599011 (0.0006) [2023-12-26 19:43:20,864][105692] Updated weights for policy 0, policy_version 598221 (0.0010) [2023-12-26 19:43:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 306528256. Throughput: 0: 9610.9, 1: 9818.5. Samples: 306516100. Policy #0 lag: (min: 19.0, avg: 43.8, max: 49.0) [2023-12-26 19:43:21,062][104569] Avg episode reward: [(0, '5668.393'), (1, '8779.985')] [2023-12-26 19:43:21,599][105692] Updated weights for policy 0, policy_version 598231 (0.0009) [2023-12-26 19:43:21,611][105620] Updated weights for policy 1, policy_version 599021 (0.0007) [2023-12-26 19:43:21,662][105692] Updated weights for policy 0, policy_version 598241 (0.0008) [2023-12-26 19:43:21,678][105620] Updated weights for policy 1, policy_version 599031 (0.0008) [2023-12-26 19:43:21,730][105692] Updated weights for policy 0, policy_version 598251 (0.0008) [2023-12-26 19:43:21,747][105620] Updated weights for policy 1, policy_version 599041 (0.0007) [2023-12-26 19:43:22,285][105620] Updated weights for policy 1, policy_version 599051 (0.0006) [2023-12-26 19:43:22,344][105620] Updated weights for policy 1, policy_version 599061 (0.0006) [2023-12-26 19:43:22,409][105620] Updated weights for policy 1, policy_version 599071 (0.0009) [2023-12-26 19:43:22,522][105692] Updated weights for policy 0, policy_version 598261 (0.0008) [2023-12-26 19:43:22,588][105692] Updated weights for policy 0, policy_version 598271 (0.0005) [2023-12-26 19:43:22,660][105692] Updated weights for policy 0, policy_version 598281 (0.0008) [2023-12-26 19:43:23,172][105620] Updated weights for policy 1, policy_version 599081 (0.0008) [2023-12-26 19:43:23,226][105620] Updated weights for policy 1, policy_version 599091 (0.0007) [2023-12-26 19:43:23,288][105620] Updated weights for policy 1, policy_version 599101 (0.0006) [2023-12-26 19:43:23,306][105692] Updated weights for policy 0, policy_version 598291 (0.0007) [2023-12-26 19:43:23,340][105620] Updated weights for policy 1, policy_version 599111 (0.0006) [2023-12-26 19:43:23,361][105692] Updated weights for policy 0, policy_version 598301 (0.0010) [2023-12-26 19:43:23,416][105692] Updated weights for policy 0, policy_version 598311 (0.0010) [2023-12-26 19:43:24,085][105692] Updated weights for policy 0, policy_version 598321 (0.0010) [2023-12-26 19:43:24,115][105620] Updated weights for policy 1, policy_version 599121 (0.0008) [2023-12-26 19:43:24,140][105692] Updated weights for policy 0, policy_version 598331 (0.0005) [2023-12-26 19:43:24,182][105620] Updated weights for policy 1, policy_version 599131 (0.0008) [2023-12-26 19:43:24,193][105692] Updated weights for policy 0, policy_version 598341 (0.0005) [2023-12-26 19:43:24,237][105692] Updated weights for policy 0, policy_version 598351 (0.0005) [2023-12-26 19:43:24,243][105620] Updated weights for policy 1, policy_version 599141 (0.0009) [2023-12-26 19:43:24,892][105692] Updated weights for policy 0, policy_version 598361 (0.0008) [2023-12-26 19:43:24,939][105692] Updated weights for policy 0, policy_version 598371 (0.0009) [2023-12-26 19:43:24,996][105692] Updated weights for policy 0, policy_version 598381 (0.0009) [2023-12-26 19:43:25,022][105620] Updated weights for policy 1, policy_version 599151 (0.0008) [2023-12-26 19:43:25,080][105620] Updated weights for policy 1, policy_version 599161 (0.0010) [2023-12-26 19:43:25,142][105620] Updated weights for policy 1, policy_version 599171 (0.0009) [2023-12-26 19:43:25,595][105692] Updated weights for policy 0, policy_version 598391 (0.0006) [2023-12-26 19:43:25,653][105692] Updated weights for policy 0, policy_version 598401 (0.0005) [2023-12-26 19:43:25,713][105692] Updated weights for policy 0, policy_version 598411 (0.0005) [2023-12-26 19:43:25,878][105620] Updated weights for policy 1, policy_version 599181 (0.0008) [2023-12-26 19:43:25,928][105620] Updated weights for policy 1, policy_version 599191 (0.0008) [2023-12-26 19:43:25,986][105620] Updated weights for policy 1, policy_version 599201 (0.0010) [2023-12-26 19:43:26,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 306626560. Throughput: 0: 9757.6, 1: 9753.1. Samples: 306632368. Policy #0 lag: (min: 19.0, avg: 43.8, max: 49.0) [2023-12-26 19:43:26,062][104569] Avg episode reward: [(0, '7988.912'), (1, '9352.760')] [2023-12-26 19:43:26,351][105692] Updated weights for policy 0, policy_version 598421 (0.0007) [2023-12-26 19:43:26,406][105692] Updated weights for policy 0, policy_version 598431 (0.0008) [2023-12-26 19:43:26,466][105692] Updated weights for policy 0, policy_version 598441 (0.0008) [2023-12-26 19:43:26,702][105620] Updated weights for policy 1, policy_version 599211 (0.0009) [2023-12-26 19:43:26,759][105620] Updated weights for policy 1, policy_version 599221 (0.0007) [2023-12-26 19:43:26,812][105620] Updated weights for policy 1, policy_version 599232 (0.0010) [2023-12-26 19:43:27,128][105692] Updated weights for policy 0, policy_version 598451 (0.0008) [2023-12-26 19:43:27,176][105692] Updated weights for policy 0, policy_version 598461 (0.0008) [2023-12-26 19:43:27,222][105692] Updated weights for policy 0, policy_version 598471 (0.0009) [2023-12-26 19:43:27,536][105620] Updated weights for policy 1, policy_version 599242 (0.0007) [2023-12-26 19:43:27,589][105620] Updated weights for policy 1, policy_version 599252 (0.0010) [2023-12-26 19:43:27,640][105620] Updated weights for policy 1, policy_version 599262 (0.0010) [2023-12-26 19:43:27,690][105620] Updated weights for policy 1, policy_version 599272 (0.0010) [2023-12-26 19:43:27,956][105692] Updated weights for policy 0, policy_version 598481 (0.0009) [2023-12-26 19:43:28,015][105692] Updated weights for policy 0, policy_version 598491 (0.0008) [2023-12-26 19:43:28,069][105692] Updated weights for policy 0, policy_version 598501 (0.0008) [2023-12-26 19:43:28,127][105692] Updated weights for policy 0, policy_version 598511 (0.0008) [2023-12-26 19:43:28,448][105620] Updated weights for policy 1, policy_version 599282 (0.0010) [2023-12-26 19:43:28,507][105620] Updated weights for policy 1, policy_version 599292 (0.0010) [2023-12-26 19:43:28,575][105620] Updated weights for policy 1, policy_version 599302 (0.0009) [2023-12-26 19:43:28,923][105692] Updated weights for policy 0, policy_version 598521 (0.0009) [2023-12-26 19:43:28,978][105692] Updated weights for policy 0, policy_version 598531 (0.0009) [2023-12-26 19:43:29,033][105692] Updated weights for policy 0, policy_version 598541 (0.0009) [2023-12-26 19:43:29,271][105620] Updated weights for policy 1, policy_version 599312 (0.0007) [2023-12-26 19:43:29,317][105620] Updated weights for policy 1, policy_version 599322 (0.0008) [2023-12-26 19:43:29,382][105620] Updated weights for policy 1, policy_version 599332 (0.0010) [2023-12-26 19:43:29,757][105692] Updated weights for policy 0, policy_version 598551 (0.0008) [2023-12-26 19:43:29,808][105692] Updated weights for policy 0, policy_version 598561 (0.0009) [2023-12-26 19:43:29,872][105692] Updated weights for policy 0, policy_version 598571 (0.0008) [2023-12-26 19:43:30,131][105620] Updated weights for policy 1, policy_version 599342 (0.0008) [2023-12-26 19:43:30,192][105620] Updated weights for policy 1, policy_version 599352 (0.0006) [2023-12-26 19:43:30,252][105620] Updated weights for policy 1, policy_version 599362 (0.0006) [2023-12-26 19:43:30,702][105692] Updated weights for policy 0, policy_version 598581 (0.0009) [2023-12-26 19:43:30,750][105692] Updated weights for policy 0, policy_version 598592 (0.0007) [2023-12-26 19:43:30,803][105692] Updated weights for policy 0, policy_version 598602 (0.0005) [2023-12-26 19:43:30,841][105620] Updated weights for policy 1, policy_version 599372 (0.0007) [2023-12-26 19:43:30,897][105620] Updated weights for policy 1, policy_version 599382 (0.0008) [2023-12-26 19:43:30,954][105620] Updated weights for policy 1, policy_version 599392 (0.0009) [2023-12-26 19:43:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 306724864. Throughput: 0: 9794.9, 1: 9763.9. Samples: 306690808. Policy #0 lag: (min: 2.0, avg: 15.6, max: 34.0) [2023-12-26 19:43:31,062][104569] Avg episode reward: [(0, '8815.778'), (1, '9352.496')] [2023-12-26 19:43:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000598608_153264128.pth... [2023-12-26 19:43:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000599400_153460736.pth... [2023-12-26 19:43:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000597456_152969216.pth [2023-12-26 19:43:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000598248_153165824.pth [2023-12-26 19:43:31,463][105692] Updated weights for policy 0, policy_version 598612 (0.0007) [2023-12-26 19:43:31,514][105692] Updated weights for policy 0, policy_version 598622 (0.0010) [2023-12-26 19:43:31,572][105692] Updated weights for policy 0, policy_version 598632 (0.0010) [2023-12-26 19:43:31,666][105620] Updated weights for policy 1, policy_version 599402 (0.0010) [2023-12-26 19:43:31,731][105620] Updated weights for policy 1, policy_version 599412 (0.0009) [2023-12-26 19:43:31,792][105620] Updated weights for policy 1, policy_version 599422 (0.0008) [2023-12-26 19:43:31,852][105620] Updated weights for policy 1, policy_version 599432 (0.0009) [2023-12-26 19:43:32,248][105692] Updated weights for policy 0, policy_version 598642 (0.0008) [2023-12-26 19:43:32,300][105692] Updated weights for policy 0, policy_version 598652 (0.0010) [2023-12-26 19:43:32,361][105692] Updated weights for policy 0, policy_version 598662 (0.0010) [2023-12-26 19:43:32,410][105692] Updated weights for policy 0, policy_version 598672 (0.0010) [2023-12-26 19:43:32,584][105620] Updated weights for policy 1, policy_version 599442 (0.0005) [2023-12-26 19:43:32,633][105620] Updated weights for policy 1, policy_version 599452 (0.0005) [2023-12-26 19:43:32,686][105620] Updated weights for policy 1, policy_version 599462 (0.0005) [2023-12-26 19:43:33,148][105692] Updated weights for policy 0, policy_version 598682 (0.0010) [2023-12-26 19:43:33,192][105692] Updated weights for policy 0, policy_version 598692 (0.0010) [2023-12-26 19:43:33,229][105620] Updated weights for policy 1, policy_version 599472 (0.0007) [2023-12-26 19:43:33,239][105692] Updated weights for policy 0, policy_version 598702 (0.0010) [2023-12-26 19:43:33,279][105620] Updated weights for policy 1, policy_version 599482 (0.0007) [2023-12-26 19:43:33,337][105620] Updated weights for policy 1, policy_version 599492 (0.0010) [2023-12-26 19:43:34,000][105692] Updated weights for policy 0, policy_version 598712 (0.0010) [2023-12-26 19:43:34,052][105620] Updated weights for policy 1, policy_version 599502 (0.0010) [2023-12-26 19:43:34,058][105692] Updated weights for policy 0, policy_version 598722 (0.0008) [2023-12-26 19:43:34,110][105620] Updated weights for policy 1, policy_version 599512 (0.0011) [2023-12-26 19:43:34,123][105692] Updated weights for policy 0, policy_version 598732 (0.0005) [2023-12-26 19:43:34,172][105620] Updated weights for policy 1, policy_version 599522 (0.0010) [2023-12-26 19:43:34,862][105692] Updated weights for policy 0, policy_version 598742 (0.0007) [2023-12-26 19:43:34,911][105620] Updated weights for policy 1, policy_version 599532 (0.0010) [2023-12-26 19:43:34,924][105692] Updated weights for policy 0, policy_version 598752 (0.0007) [2023-12-26 19:43:34,963][105620] Updated weights for policy 1, policy_version 599542 (0.0010) [2023-12-26 19:43:34,973][105692] Updated weights for policy 0, policy_version 598762 (0.0005) [2023-12-26 19:43:35,021][105620] Updated weights for policy 1, policy_version 599552 (0.0010) [2023-12-26 19:43:35,733][105692] Updated weights for policy 0, policy_version 598772 (0.0008) [2023-12-26 19:43:35,772][105620] Updated weights for policy 1, policy_version 599562 (0.0009) [2023-12-26 19:43:35,795][105692] Updated weights for policy 0, policy_version 598782 (0.0005) [2023-12-26 19:43:35,838][105620] Updated weights for policy 1, policy_version 599572 (0.0005) [2023-12-26 19:43:35,845][105692] Updated weights for policy 0, policy_version 598792 (0.0005) [2023-12-26 19:43:35,902][105620] Updated weights for policy 1, policy_version 599582 (0.0005) [2023-12-26 19:43:35,959][105620] Updated weights for policy 1, policy_version 599592 (0.0008) [2023-12-26 19:43:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 306823168. Throughput: 0: 9813.4, 1: 9807.3. Samples: 306808784. Policy #0 lag: (min: 2.0, avg: 15.6, max: 34.0) [2023-12-26 19:43:36,063][104569] Avg episode reward: [(0, '8452.115'), (1, '9260.794')] [2023-12-26 19:43:36,419][105692] Updated weights for policy 0, policy_version 598802 (0.0006) [2023-12-26 19:43:36,488][105692] Updated weights for policy 0, policy_version 598812 (0.0007) [2023-12-26 19:43:36,544][105692] Updated weights for policy 0, policy_version 598822 (0.0008) [2023-12-26 19:43:36,605][105692] Updated weights for policy 0, policy_version 598832 (0.0006) [2023-12-26 19:43:36,652][105620] Updated weights for policy 1, policy_version 599602 (0.0007) [2023-12-26 19:43:36,710][105620] Updated weights for policy 1, policy_version 599612 (0.0005) [2023-12-26 19:43:36,764][105620] Updated weights for policy 1, policy_version 599622 (0.0005) [2023-12-26 19:43:37,246][105692] Updated weights for policy 0, policy_version 598842 (0.0007) [2023-12-26 19:43:37,297][105692] Updated weights for policy 0, policy_version 598852 (0.0009) [2023-12-26 19:43:37,361][105692] Updated weights for policy 0, policy_version 598862 (0.0007) [2023-12-26 19:43:37,382][105620] Updated weights for policy 1, policy_version 599632 (0.0007) [2023-12-26 19:43:37,447][105620] Updated weights for policy 1, policy_version 599642 (0.0009) [2023-12-26 19:43:37,508][105620] Updated weights for policy 1, policy_version 599652 (0.0009) [2023-12-26 19:43:38,039][105692] Updated weights for policy 0, policy_version 598872 (0.0008) [2023-12-26 19:43:38,087][105692] Updated weights for policy 0, policy_version 598882 (0.0008) [2023-12-26 19:43:38,148][105692] Updated weights for policy 0, policy_version 598892 (0.0009) [2023-12-26 19:43:38,225][105620] Updated weights for policy 1, policy_version 599662 (0.0009) [2023-12-26 19:43:38,283][105620] Updated weights for policy 1, policy_version 599672 (0.0009) [2023-12-26 19:43:38,334][105620] Updated weights for policy 1, policy_version 599682 (0.0007) [2023-12-26 19:43:38,927][105692] Updated weights for policy 0, policy_version 598902 (0.0009) [2023-12-26 19:43:38,979][105692] Updated weights for policy 0, policy_version 598912 (0.0008) [2023-12-26 19:43:39,026][105620] Updated weights for policy 1, policy_version 599692 (0.0007) [2023-12-26 19:43:39,028][105692] Updated weights for policy 0, policy_version 598922 (0.0005) [2023-12-26 19:43:39,083][105620] Updated weights for policy 1, policy_version 599702 (0.0009) [2023-12-26 19:43:39,132][105620] Updated weights for policy 1, policy_version 599712 (0.0009) [2023-12-26 19:43:39,737][105692] Updated weights for policy 0, policy_version 598932 (0.0006) [2023-12-26 19:43:39,789][105692] Updated weights for policy 0, policy_version 598942 (0.0010) [2023-12-26 19:43:39,855][105692] Updated weights for policy 0, policy_version 598952 (0.0008) [2023-12-26 19:43:39,944][105620] Updated weights for policy 1, policy_version 599722 (0.0009) [2023-12-26 19:43:40,007][105620] Updated weights for policy 1, policy_version 599732 (0.0008) [2023-12-26 19:43:40,077][105620] Updated weights for policy 1, policy_version 599742 (0.0006) [2023-12-26 19:43:40,146][105620] Updated weights for policy 1, policy_version 599752 (0.0006) [2023-12-26 19:43:40,574][105692] Updated weights for policy 0, policy_version 598962 (0.0010) [2023-12-26 19:43:40,636][105692] Updated weights for policy 0, policy_version 598972 (0.0009) [2023-12-26 19:43:40,691][105692] Updated weights for policy 0, policy_version 598982 (0.0009) [2023-12-26 19:43:40,739][105692] Updated weights for policy 0, policy_version 598992 (0.0009) [2023-12-26 19:43:40,827][105620] Updated weights for policy 1, policy_version 599762 (0.0008) [2023-12-26 19:43:40,881][105620] Updated weights for policy 1, policy_version 599773 (0.0010) [2023-12-26 19:43:40,929][105620] Updated weights for policy 1, policy_version 599783 (0.0009) [2023-12-26 19:43:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 306921472. Throughput: 0: 9845.4, 1: 9803.8. Samples: 306926844. Policy #0 lag: (min: 2.0, avg: 15.6, max: 34.0) [2023-12-26 19:43:41,063][104569] Avg episode reward: [(0, '8549.671'), (1, '9259.371')] [2023-12-26 19:43:41,500][105692] Updated weights for policy 0, policy_version 599002 (0.0009) [2023-12-26 19:43:41,551][105692] Updated weights for policy 0, policy_version 599012 (0.0008) [2023-12-26 19:43:41,621][105692] Updated weights for policy 0, policy_version 599022 (0.0009) [2023-12-26 19:43:41,691][105620] Updated weights for policy 1, policy_version 599793 (0.0010) [2023-12-26 19:43:41,757][105620] Updated weights for policy 1, policy_version 599803 (0.0009) [2023-12-26 19:43:41,806][105620] Updated weights for policy 1, policy_version 599813 (0.0009) [2023-12-26 19:43:42,297][105692] Updated weights for policy 0, policy_version 599032 (0.0008) [2023-12-26 19:43:42,361][105692] Updated weights for policy 0, policy_version 599042 (0.0009) [2023-12-26 19:43:42,424][105692] Updated weights for policy 0, policy_version 599052 (0.0009) [2023-12-26 19:43:42,632][105620] Updated weights for policy 1, policy_version 599823 (0.0010) [2023-12-26 19:43:42,685][105620] Updated weights for policy 1, policy_version 599833 (0.0008) [2023-12-26 19:43:42,738][105620] Updated weights for policy 1, policy_version 599843 (0.0010) [2023-12-26 19:43:43,178][105692] Updated weights for policy 0, policy_version 599062 (0.0009) [2023-12-26 19:43:43,236][105692] Updated weights for policy 0, policy_version 599072 (0.0010) [2023-12-26 19:43:43,292][105692] Updated weights for policy 0, policy_version 599082 (0.0010) [2023-12-26 19:43:43,447][105620] Updated weights for policy 1, policy_version 599853 (0.0008) [2023-12-26 19:43:43,510][105620] Updated weights for policy 1, policy_version 599863 (0.0005) [2023-12-26 19:43:43,570][105620] Updated weights for policy 1, policy_version 599873 (0.0008) [2023-12-26 19:43:44,034][105692] Updated weights for policy 0, policy_version 599092 (0.0010) [2023-12-26 19:43:44,082][105692] Updated weights for policy 0, policy_version 599102 (0.0010) [2023-12-26 19:43:44,134][105692] Updated weights for policy 0, policy_version 599112 (0.0010) [2023-12-26 19:43:44,174][105620] Updated weights for policy 1, policy_version 599883 (0.0010) [2023-12-26 19:43:44,235][105620] Updated weights for policy 1, policy_version 599893 (0.0006) [2023-12-26 19:43:44,284][105620] Updated weights for policy 1, policy_version 599903 (0.0005) [2023-12-26 19:43:44,867][105620] Updated weights for policy 1, policy_version 599913 (0.0006) [2023-12-26 19:43:44,867][105692] Updated weights for policy 0, policy_version 599122 (0.0010) [2023-12-26 19:43:44,931][105692] Updated weights for policy 0, policy_version 599132 (0.0011) [2023-12-26 19:43:44,933][105620] Updated weights for policy 1, policy_version 599923 (0.0008) [2023-12-26 19:43:44,990][105692] Updated weights for policy 0, policy_version 599142 (0.0008) [2023-12-26 19:43:44,992][105620] Updated weights for policy 1, policy_version 599933 (0.0008) [2023-12-26 19:43:45,057][105692] Updated weights for policy 0, policy_version 599152 (0.0011) [2023-12-26 19:43:45,061][105620] Updated weights for policy 1, policy_version 599943 (0.0006) [2023-12-26 19:43:45,669][105692] Updated weights for policy 0, policy_version 599162 (0.0007) [2023-12-26 19:43:45,721][105692] Updated weights for policy 0, policy_version 599172 (0.0010) [2023-12-26 19:43:45,775][105692] Updated weights for policy 0, policy_version 599182 (0.0010) [2023-12-26 19:43:45,786][105620] Updated weights for policy 1, policy_version 599953 (0.0005) [2023-12-26 19:43:45,834][105620] Updated weights for policy 1, policy_version 599963 (0.0008) [2023-12-26 19:43:45,878][105620] Updated weights for policy 1, policy_version 599973 (0.0008) [2023-12-26 19:43:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.9, 300 sec: 19522.0). Total num frames: 307019776. Throughput: 0: 9748.0, 1: 9804.0. Samples: 306983120. Policy #0 lag: (min: 2.0, avg: 15.6, max: 34.0) [2023-12-26 19:43:46,062][104569] Avg episode reward: [(0, '8867.330'), (1, '9350.400')] [2023-12-26 19:43:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000599976_153608192.pth... [2023-12-26 19:43:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000599184_153411584.pth... [2023-12-26 19:43:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000598824_153313280.pth [2023-12-26 19:43:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000598032_153116672.pth [2023-12-26 19:43:46,479][105692] Updated weights for policy 0, policy_version 599192 (0.0010) [2023-12-26 19:43:46,523][105692] Updated weights for policy 0, policy_version 599202 (0.0010) [2023-12-26 19:43:46,569][105692] Updated weights for policy 0, policy_version 599212 (0.0010) [2023-12-26 19:43:46,639][105620] Updated weights for policy 1, policy_version 599983 (0.0008) [2023-12-26 19:43:46,699][105620] Updated weights for policy 1, policy_version 599993 (0.0008) [2023-12-26 19:43:46,777][105620] Updated weights for policy 1, policy_version 600003 (0.0005) [2023-12-26 19:43:47,336][105692] Updated weights for policy 0, policy_version 599222 (0.0010) [2023-12-26 19:43:47,361][105620] Updated weights for policy 1, policy_version 600013 (0.0007) [2023-12-26 19:43:47,387][105692] Updated weights for policy 0, policy_version 599232 (0.0010) [2023-12-26 19:43:47,411][105620] Updated weights for policy 1, policy_version 600023 (0.0005) [2023-12-26 19:43:47,439][105692] Updated weights for policy 0, policy_version 599242 (0.0010) [2023-12-26 19:43:47,457][105620] Updated weights for policy 1, policy_version 600033 (0.0005) [2023-12-26 19:43:48,180][105692] Updated weights for policy 0, policy_version 599252 (0.0010) [2023-12-26 19:43:48,201][105620] Updated weights for policy 1, policy_version 600043 (0.0007) [2023-12-26 19:43:48,234][105692] Updated weights for policy 0, policy_version 599262 (0.0007) [2023-12-26 19:43:48,257][105620] Updated weights for policy 1, policy_version 600053 (0.0006) [2023-12-26 19:43:48,291][105692] Updated weights for policy 0, policy_version 599272 (0.0006) [2023-12-26 19:43:48,319][105620] Updated weights for policy 1, policy_version 600063 (0.0007) [2023-12-26 19:43:48,977][105620] Updated weights for policy 1, policy_version 600073 (0.0009) [2023-12-26 19:43:49,039][105692] Updated weights for policy 0, policy_version 599282 (0.0007) [2023-12-26 19:43:49,042][105620] Updated weights for policy 1, policy_version 600083 (0.0007) [2023-12-26 19:43:49,094][105620] Updated weights for policy 1, policy_version 600093 (0.0007) [2023-12-26 19:43:49,096][105692] Updated weights for policy 0, policy_version 599292 (0.0006) [2023-12-26 19:43:49,150][105620] Updated weights for policy 1, policy_version 600103 (0.0008) [2023-12-26 19:43:49,160][105692] Updated weights for policy 0, policy_version 599302 (0.0006) [2023-12-26 19:43:49,218][105692] Updated weights for policy 0, policy_version 599312 (0.0009) [2023-12-26 19:43:49,887][105692] Updated weights for policy 0, policy_version 599322 (0.0006) [2023-12-26 19:43:49,895][105620] Updated weights for policy 1, policy_version 600113 (0.0008) [2023-12-26 19:43:49,943][105692] Updated weights for policy 0, policy_version 599332 (0.0007) [2023-12-26 19:43:49,953][105620] Updated weights for policy 1, policy_version 600123 (0.0009) [2023-12-26 19:43:49,999][105692] Updated weights for policy 0, policy_version 599342 (0.0006) [2023-12-26 19:43:50,006][105620] Updated weights for policy 1, policy_version 600133 (0.0009) [2023-12-26 19:43:50,591][105692] Updated weights for policy 0, policy_version 599352 (0.0010) [2023-12-26 19:43:50,651][105692] Updated weights for policy 0, policy_version 599362 (0.0009) [2023-12-26 19:43:50,714][105692] Updated weights for policy 0, policy_version 599372 (0.0008) [2023-12-26 19:43:50,894][105620] Updated weights for policy 1, policy_version 600143 (0.0009) [2023-12-26 19:43:50,955][105620] Updated weights for policy 1, policy_version 600153 (0.0011) [2023-12-26 19:43:51,014][105620] Updated weights for policy 1, policy_version 600163 (0.0011) [2023-12-26 19:43:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 307118080. Throughput: 0: 9810.5, 1: 9810.4. Samples: 307103016. Policy #0 lag: (min: 2.0, avg: 15.6, max: 34.0) [2023-12-26 19:43:51,062][104569] Avg episode reward: [(0, '9000.047'), (1, '9259.724')] [2023-12-26 19:43:51,424][105692] Updated weights for policy 0, policy_version 599382 (0.0009) [2023-12-26 19:43:51,485][105692] Updated weights for policy 0, policy_version 599392 (0.0008) [2023-12-26 19:43:51,546][105692] Updated weights for policy 0, policy_version 599402 (0.0008) [2023-12-26 19:43:51,790][105620] Updated weights for policy 1, policy_version 600173 (0.0011) [2023-12-26 19:43:51,853][105620] Updated weights for policy 1, policy_version 600183 (0.0011) [2023-12-26 19:43:51,920][105620] Updated weights for policy 1, policy_version 600193 (0.0011) [2023-12-26 19:43:52,249][105692] Updated weights for policy 0, policy_version 599412 (0.0008) [2023-12-26 19:43:52,301][105692] Updated weights for policy 0, policy_version 599422 (0.0007) [2023-12-26 19:43:52,347][105692] Updated weights for policy 0, policy_version 599432 (0.0005) [2023-12-26 19:43:52,589][105620] Updated weights for policy 1, policy_version 600203 (0.0009) [2023-12-26 19:43:52,642][105620] Updated weights for policy 1, policy_version 600213 (0.0011) [2023-12-26 19:43:52,694][105620] Updated weights for policy 1, policy_version 600223 (0.0010) [2023-12-26 19:43:53,032][105692] Updated weights for policy 0, policy_version 599442 (0.0008) [2023-12-26 19:43:53,087][105692] Updated weights for policy 0, policy_version 599452 (0.0009) [2023-12-26 19:43:53,152][105692] Updated weights for policy 0, policy_version 599462 (0.0009) [2023-12-26 19:43:53,275][105620] Updated weights for policy 1, policy_version 600233 (0.0008) [2023-12-26 19:43:53,334][105620] Updated weights for policy 1, policy_version 600243 (0.0010) [2023-12-26 19:43:53,385][105620] Updated weights for policy 1, policy_version 600253 (0.0010) [2023-12-26 19:43:53,443][105620] Updated weights for policy 1, policy_version 600263 (0.0010) [2023-12-26 19:43:53,907][105692] Updated weights for policy 0, policy_version 599473 (0.0009) [2023-12-26 19:43:53,960][105692] Updated weights for policy 0, policy_version 599483 (0.0007) [2023-12-26 19:43:54,020][105692] Updated weights for policy 0, policy_version 599493 (0.0010) [2023-12-26 19:43:54,075][105692] Updated weights for policy 0, policy_version 599503 (0.0009) [2023-12-26 19:43:54,149][105620] Updated weights for policy 1, policy_version 600273 (0.0006) [2023-12-26 19:43:54,205][105620] Updated weights for policy 1, policy_version 600283 (0.0005) [2023-12-26 19:43:54,259][105620] Updated weights for policy 1, policy_version 600293 (0.0005) [2023-12-26 19:43:54,868][105620] Updated weights for policy 1, policy_version 600303 (0.0009) [2023-12-26 19:43:54,919][105692] Updated weights for policy 0, policy_version 599513 (0.0011) [2023-12-26 19:43:54,924][105620] Updated weights for policy 1, policy_version 600313 (0.0010) [2023-12-26 19:43:54,980][105692] Updated weights for policy 0, policy_version 599523 (0.0011) [2023-12-26 19:43:54,986][105620] Updated weights for policy 1, policy_version 600323 (0.0009) [2023-12-26 19:43:55,037][105692] Updated weights for policy 0, policy_version 599533 (0.0011) [2023-12-26 19:43:55,740][105620] Updated weights for policy 1, policy_version 600333 (0.0008) [2023-12-26 19:43:55,792][105620] Updated weights for policy 1, policy_version 600343 (0.0006) [2023-12-26 19:43:55,799][105692] Updated weights for policy 0, policy_version 599543 (0.0011) [2023-12-26 19:43:55,841][105620] Updated weights for policy 1, policy_version 600353 (0.0006) [2023-12-26 19:43:55,846][105692] Updated weights for policy 0, policy_version 599553 (0.0010) [2023-12-26 19:43:55,893][105692] Updated weights for policy 0, policy_version 599563 (0.0005) [2023-12-26 19:43:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.9, 300 sec: 19522.0). Total num frames: 307216384. Throughput: 0: 9763.8, 1: 9827.3. Samples: 307220244. Policy #0 lag: (min: 2.0, avg: 15.6, max: 34.0) [2023-12-26 19:43:56,062][104569] Avg episode reward: [(0, '9001.917'), (1, '9256.661')] [2023-12-26 19:43:56,370][105620] Updated weights for policy 1, policy_version 600363 (0.0005) [2023-12-26 19:43:56,425][105620] Updated weights for policy 1, policy_version 600373 (0.0005) [2023-12-26 19:43:56,489][105620] Updated weights for policy 1, policy_version 600383 (0.0010) [2023-12-26 19:43:56,616][105692] Updated weights for policy 0, policy_version 599573 (0.0008) [2023-12-26 19:43:56,671][105692] Updated weights for policy 0, policy_version 599583 (0.0010) [2023-12-26 19:43:56,725][105692] Updated weights for policy 0, policy_version 599593 (0.0010) [2023-12-26 19:43:57,194][105620] Updated weights for policy 1, policy_version 600393 (0.0010) [2023-12-26 19:43:57,253][105620] Updated weights for policy 1, policy_version 600403 (0.0010) [2023-12-26 19:43:57,310][105620] Updated weights for policy 1, policy_version 600413 (0.0008) [2023-12-26 19:43:57,362][105620] Updated weights for policy 1, policy_version 600423 (0.0005) [2023-12-26 19:43:57,413][105692] Updated weights for policy 0, policy_version 599603 (0.0010) [2023-12-26 19:43:57,474][105692] Updated weights for policy 0, policy_version 599613 (0.0010) [2023-12-26 19:43:57,531][105692] Updated weights for policy 0, policy_version 599623 (0.0010) [2023-12-26 19:43:57,994][105620] Updated weights for policy 1, policy_version 600433 (0.0005) [2023-12-26 19:43:58,056][105620] Updated weights for policy 1, policy_version 600443 (0.0005) [2023-12-26 19:43:58,140][105620] Updated weights for policy 1, policy_version 600453 (0.0007) [2023-12-26 19:43:58,285][105692] Updated weights for policy 0, policy_version 599633 (0.0010) [2023-12-26 19:43:58,378][105692] Updated weights for policy 0, policy_version 599643 (0.0009) [2023-12-26 19:43:58,445][105692] Updated weights for policy 0, policy_version 599653 (0.0008) [2023-12-26 19:43:58,510][105692] Updated weights for policy 0, policy_version 599663 (0.0008) [2023-12-26 19:43:58,939][105620] Updated weights for policy 1, policy_version 600463 (0.0007) [2023-12-26 19:43:59,005][105620] Updated weights for policy 1, policy_version 600473 (0.0007) [2023-12-26 19:43:59,069][105620] Updated weights for policy 1, policy_version 600483 (0.0009) [2023-12-26 19:43:59,373][105692] Updated weights for policy 0, policy_version 599673 (0.0009) [2023-12-26 19:43:59,433][105692] Updated weights for policy 0, policy_version 599684 (0.0010) [2023-12-26 19:43:59,487][105692] Updated weights for policy 0, policy_version 599694 (0.0010) [2023-12-26 19:43:59,769][105620] Updated weights for policy 1, policy_version 600493 (0.0007) [2023-12-26 19:43:59,828][105620] Updated weights for policy 1, policy_version 600503 (0.0008) [2023-12-26 19:43:59,891][105620] Updated weights for policy 1, policy_version 600513 (0.0008) [2023-12-26 19:44:00,357][105692] Updated weights for policy 0, policy_version 599705 (0.0010) [2023-12-26 19:44:00,404][105692] Updated weights for policy 0, policy_version 599715 (0.0009) [2023-12-26 19:44:00,451][105692] Updated weights for policy 0, policy_version 599725 (0.0008) [2023-12-26 19:44:00,591][105620] Updated weights for policy 1, policy_version 600523 (0.0008) [2023-12-26 19:44:00,644][105620] Updated weights for policy 1, policy_version 600533 (0.0008) [2023-12-26 19:44:00,706][105620] Updated weights for policy 1, policy_version 600543 (0.0009) [2023-12-26 19:44:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 307306496. Throughput: 0: 9797.8, 1: 9808.3. Samples: 307279720. Policy #0 lag: (min: 2.0, avg: 15.6, max: 34.0) [2023-12-26 19:44:01,062][104569] Avg episode reward: [(0, '8822.382'), (1, '9345.838')] [2023-12-26 19:44:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000599728_153550848.pth... [2023-12-26 19:44:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000600552_153755648.pth... [2023-12-26 19:44:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000598608_153264128.pth [2023-12-26 19:44:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000599400_153460736.pth [2023-12-26 19:44:01,286][105692] Updated weights for policy 0, policy_version 599735 (0.0010) [2023-12-26 19:44:01,343][105692] Updated weights for policy 0, policy_version 599745 (0.0009) [2023-12-26 19:44:01,382][105620] Updated weights for policy 1, policy_version 600553 (0.0007) [2023-12-26 19:44:01,407][105692] Updated weights for policy 0, policy_version 599755 (0.0008) [2023-12-26 19:44:01,437][105620] Updated weights for policy 1, policy_version 600563 (0.0008) [2023-12-26 19:44:01,484][105620] Updated weights for policy 1, policy_version 600573 (0.0008) [2023-12-26 19:44:01,533][105620] Updated weights for policy 1, policy_version 600583 (0.0008) [2023-12-26 19:44:02,155][105692] Updated weights for policy 0, policy_version 599765 (0.0006) [2023-12-26 19:44:02,218][105692] Updated weights for policy 0, policy_version 599775 (0.0005) [2023-12-26 19:44:02,280][105692] Updated weights for policy 0, policy_version 599785 (0.0007) [2023-12-26 19:44:02,290][105620] Updated weights for policy 1, policy_version 600593 (0.0011) [2023-12-26 19:44:02,357][105620] Updated weights for policy 1, policy_version 600603 (0.0011) [2023-12-26 19:44:02,419][105620] Updated weights for policy 1, policy_version 600613 (0.0010) [2023-12-26 19:44:02,848][105692] Updated weights for policy 0, policy_version 599795 (0.0005) [2023-12-26 19:44:02,901][105692] Updated weights for policy 0, policy_version 599805 (0.0008) [2023-12-26 19:44:02,948][105692] Updated weights for policy 0, policy_version 599815 (0.0009) [2023-12-26 19:44:03,144][105620] Updated weights for policy 1, policy_version 600623 (0.0007) [2023-12-26 19:44:03,214][105620] Updated weights for policy 1, policy_version 600633 (0.0005) [2023-12-26 19:44:03,271][105620] Updated weights for policy 1, policy_version 600643 (0.0009) [2023-12-26 19:44:03,692][105692] Updated weights for policy 0, policy_version 599825 (0.0009) [2023-12-26 19:44:03,738][105692] Updated weights for policy 0, policy_version 599835 (0.0008) [2023-12-26 19:44:03,783][105692] Updated weights for policy 0, policy_version 599845 (0.0005) [2023-12-26 19:44:03,827][105692] Updated weights for policy 0, policy_version 599855 (0.0005) [2023-12-26 19:44:03,979][105620] Updated weights for policy 1, policy_version 600653 (0.0007) [2023-12-26 19:44:04,046][105620] Updated weights for policy 1, policy_version 600663 (0.0005) [2023-12-26 19:44:04,115][105620] Updated weights for policy 1, policy_version 600673 (0.0006) [2023-12-26 19:44:04,538][105692] Updated weights for policy 0, policy_version 599865 (0.0008) [2023-12-26 19:44:04,597][105692] Updated weights for policy 0, policy_version 599875 (0.0009) [2023-12-26 19:44:04,655][105692] Updated weights for policy 0, policy_version 599885 (0.0009) [2023-12-26 19:44:04,775][105620] Updated weights for policy 1, policy_version 600683 (0.0008) [2023-12-26 19:44:04,828][105620] Updated weights for policy 1, policy_version 600693 (0.0008) [2023-12-26 19:44:04,885][105620] Updated weights for policy 1, policy_version 600703 (0.0009) [2023-12-26 19:44:05,419][105692] Updated weights for policy 0, policy_version 599895 (0.0006) [2023-12-26 19:44:05,474][105692] Updated weights for policy 0, policy_version 599905 (0.0005) [2023-12-26 19:44:05,543][105692] Updated weights for policy 0, policy_version 599915 (0.0005) [2023-12-26 19:44:05,547][105620] Updated weights for policy 1, policy_version 600713 (0.0008) [2023-12-26 19:44:05,611][105620] Updated weights for policy 1, policy_version 600723 (0.0007) [2023-12-26 19:44:05,671][105620] Updated weights for policy 1, policy_version 600733 (0.0009) [2023-12-26 19:44:05,722][105620] Updated weights for policy 1, policy_version 600743 (0.0010) [2023-12-26 19:44:06,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 307404800. Throughput: 0: 9684.2, 1: 9838.1. Samples: 307394608. Policy #0 lag: (min: 2.0, avg: 15.6, max: 34.0) [2023-12-26 19:44:06,063][104569] Avg episode reward: [(0, '9085.400'), (1, '9254.134')] [2023-12-26 19:44:06,160][105692] Updated weights for policy 0, policy_version 599925 (0.0006) [2023-12-26 19:44:06,223][105692] Updated weights for policy 0, policy_version 599935 (0.0008) [2023-12-26 19:44:06,289][105692] Updated weights for policy 0, policy_version 599945 (0.0009) [2023-12-26 19:44:06,483][105620] Updated weights for policy 1, policy_version 600753 (0.0011) [2023-12-26 19:44:06,550][105620] Updated weights for policy 1, policy_version 600763 (0.0011) [2023-12-26 19:44:06,610][105620] Updated weights for policy 1, policy_version 600773 (0.0011) [2023-12-26 19:44:06,894][105692] Updated weights for policy 0, policy_version 599955 (0.0007) [2023-12-26 19:44:06,954][105692] Updated weights for policy 0, policy_version 599965 (0.0008) [2023-12-26 19:44:07,010][105692] Updated weights for policy 0, policy_version 599975 (0.0008) [2023-12-26 19:44:07,343][105620] Updated weights for policy 1, policy_version 600783 (0.0010) [2023-12-26 19:44:07,398][105620] Updated weights for policy 1, policy_version 600793 (0.0010) [2023-12-26 19:44:07,457][105620] Updated weights for policy 1, policy_version 600803 (0.0010) [2023-12-26 19:44:07,727][105692] Updated weights for policy 0, policy_version 599985 (0.0008) [2023-12-26 19:44:07,773][105692] Updated weights for policy 0, policy_version 599995 (0.0008) [2023-12-26 19:44:07,831][105692] Updated weights for policy 0, policy_version 600005 (0.0008) [2023-12-26 19:44:07,895][105692] Updated weights for policy 0, policy_version 600015 (0.0009) [2023-12-26 19:44:08,208][105620] Updated weights for policy 1, policy_version 600813 (0.0010) [2023-12-26 19:44:08,272][105620] Updated weights for policy 1, policy_version 600823 (0.0010) [2023-12-26 19:44:08,334][105620] Updated weights for policy 1, policy_version 600833 (0.0010) [2023-12-26 19:44:08,692][105692] Updated weights for policy 0, policy_version 600025 (0.0009) [2023-12-26 19:44:08,750][105692] Updated weights for policy 0, policy_version 600035 (0.0009) [2023-12-26 19:44:08,802][105692] Updated weights for policy 0, policy_version 600045 (0.0009) [2023-12-26 19:44:09,060][105620] Updated weights for policy 1, policy_version 600843 (0.0009) [2023-12-26 19:44:09,106][105620] Updated weights for policy 1, policy_version 600853 (0.0005) [2023-12-26 19:44:09,155][105620] Updated weights for policy 1, policy_version 600863 (0.0005) [2023-12-26 19:44:09,596][105692] Updated weights for policy 0, policy_version 600055 (0.0009) [2023-12-26 19:44:09,648][105692] Updated weights for policy 0, policy_version 600065 (0.0009) [2023-12-26 19:44:09,713][105692] Updated weights for policy 0, policy_version 600075 (0.0009) [2023-12-26 19:44:09,883][105620] Updated weights for policy 1, policy_version 600873 (0.0006) [2023-12-26 19:44:09,949][105620] Updated weights for policy 1, policy_version 600883 (0.0009) [2023-12-26 19:44:10,021][105620] Updated weights for policy 1, policy_version 600893 (0.0008) [2023-12-26 19:44:10,080][105620] Updated weights for policy 1, policy_version 600903 (0.0010) [2023-12-26 19:44:10,477][105692] Updated weights for policy 0, policy_version 600085 (0.0010) [2023-12-26 19:44:10,536][105692] Updated weights for policy 0, policy_version 600095 (0.0011) [2023-12-26 19:44:10,591][105692] Updated weights for policy 0, policy_version 600105 (0.0007) [2023-12-26 19:44:10,890][105620] Updated weights for policy 1, policy_version 600913 (0.0006) [2023-12-26 19:44:10,955][105620] Updated weights for policy 1, policy_version 600923 (0.0006) [2023-12-26 19:44:11,012][105620] Updated weights for policy 1, policy_version 600933 (0.0009) [2023-12-26 19:44:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 307503104. Throughput: 0: 9637.4, 1: 9845.9. Samples: 307509120. Policy #0 lag: (min: 2.0, avg: 15.6, max: 34.0) [2023-12-26 19:44:11,062][104569] Avg episode reward: [(0, '9266.798'), (1, '9345.970')] [2023-12-26 19:44:11,300][105692] Updated weights for policy 0, policy_version 600115 (0.0006) [2023-12-26 19:44:11,356][105692] Updated weights for policy 0, policy_version 600125 (0.0010) [2023-12-26 19:44:11,425][105692] Updated weights for policy 0, policy_version 600135 (0.0007) [2023-12-26 19:44:11,732][105620] Updated weights for policy 1, policy_version 600943 (0.0009) [2023-12-26 19:44:11,795][105620] Updated weights for policy 1, policy_version 600953 (0.0008) [2023-12-26 19:44:11,860][105620] Updated weights for policy 1, policy_version 600963 (0.0009) [2023-12-26 19:44:12,213][105692] Updated weights for policy 0, policy_version 600145 (0.0007) [2023-12-26 19:44:12,272][105692] Updated weights for policy 0, policy_version 600155 (0.0010) [2023-12-26 19:44:12,336][105692] Updated weights for policy 0, policy_version 600165 (0.0010) [2023-12-26 19:44:12,394][105692] Updated weights for policy 0, policy_version 600175 (0.0009) [2023-12-26 19:44:12,618][105620] Updated weights for policy 1, policy_version 600973 (0.0009) [2023-12-26 19:44:12,682][105620] Updated weights for policy 1, policy_version 600983 (0.0008) [2023-12-26 19:44:12,744][105620] Updated weights for policy 1, policy_version 600993 (0.0008) [2023-12-26 19:44:13,034][105692] Updated weights for policy 0, policy_version 600185 (0.0009) [2023-12-26 19:44:13,082][105692] Updated weights for policy 0, policy_version 600195 (0.0010) [2023-12-26 19:44:13,130][105692] Updated weights for policy 0, policy_version 600205 (0.0010) [2023-12-26 19:44:13,442][105620] Updated weights for policy 1, policy_version 601003 (0.0008) [2023-12-26 19:44:13,491][105620] Updated weights for policy 1, policy_version 601013 (0.0007) [2023-12-26 19:44:13,538][105620] Updated weights for policy 1, policy_version 601023 (0.0009) [2023-12-26 19:44:13,888][105692] Updated weights for policy 0, policy_version 600215 (0.0006) [2023-12-26 19:44:13,936][105692] Updated weights for policy 0, policy_version 600225 (0.0005) [2023-12-26 19:44:13,990][105692] Updated weights for policy 0, policy_version 600235 (0.0006) [2023-12-26 19:44:14,337][105620] Updated weights for policy 1, policy_version 601033 (0.0008) [2023-12-26 19:44:14,390][105620] Updated weights for policy 1, policy_version 601043 (0.0009) [2023-12-26 19:44:14,446][105620] Updated weights for policy 1, policy_version 601054 (0.0010) [2023-12-26 19:44:14,493][105620] Updated weights for policy 1, policy_version 601064 (0.0009) [2023-12-26 19:44:14,617][105692] Updated weights for policy 0, policy_version 600245 (0.0008) [2023-12-26 19:44:14,687][105692] Updated weights for policy 0, policy_version 600255 (0.0010) [2023-12-26 19:44:14,741][105692] Updated weights for policy 0, policy_version 600266 (0.0010) [2023-12-26 19:44:15,220][105620] Updated weights for policy 1, policy_version 601074 (0.0008) [2023-12-26 19:44:15,267][105620] Updated weights for policy 1, policy_version 601084 (0.0009) [2023-12-26 19:44:15,315][105620] Updated weights for policy 1, policy_version 601094 (0.0009) [2023-12-26 19:44:15,529][105692] Updated weights for policy 0, policy_version 600276 (0.0007) [2023-12-26 19:44:15,577][105692] Updated weights for policy 0, policy_version 600286 (0.0009) [2023-12-26 19:44:15,624][105692] Updated weights for policy 0, policy_version 600296 (0.0008) [2023-12-26 19:44:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 307593216. Throughput: 0: 9636.0, 1: 9828.0. Samples: 307566692. Policy #0 lag: (min: 2.0, avg: 15.6, max: 34.0) [2023-12-26 19:44:16,063][104569] Avg episode reward: [(0, '9266.537'), (1, '9346.521')] [2023-12-26 19:44:16,069][105620] Updated weights for policy 1, policy_version 601104 (0.0010) [2023-12-26 19:44:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000600304_153698304.pth... [2023-12-26 19:44:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000599184_153411584.pth [2023-12-26 19:44:16,124][105620] Updated weights for policy 1, policy_version 601114 (0.0009) [2023-12-26 19:44:16,171][105620] Updated weights for policy 1, policy_version 601124 (0.0008) [2023-12-26 19:44:16,188][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000601128_153903104.pth... [2023-12-26 19:44:16,195][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000599976_153608192.pth [2023-12-26 19:44:16,411][105692] Updated weights for policy 0, policy_version 600306 (0.0008) [2023-12-26 19:44:16,470][105692] Updated weights for policy 0, policy_version 600316 (0.0007) [2023-12-26 19:44:16,534][105692] Updated weights for policy 0, policy_version 600326 (0.0008) [2023-12-26 19:44:16,594][105692] Updated weights for policy 0, policy_version 600336 (0.0008) [2023-12-26 19:44:16,848][105620] Updated weights for policy 1, policy_version 601134 (0.0007) [2023-12-26 19:44:16,901][105620] Updated weights for policy 1, policy_version 601144 (0.0006) [2023-12-26 19:44:16,970][105620] Updated weights for policy 1, policy_version 601154 (0.0008) [2023-12-26 19:44:17,335][105692] Updated weights for policy 0, policy_version 600346 (0.0008) [2023-12-26 19:44:17,388][105692] Updated weights for policy 0, policy_version 600356 (0.0009) [2023-12-26 19:44:17,452][105692] Updated weights for policy 0, policy_version 600366 (0.0008) [2023-12-26 19:44:17,615][105620] Updated weights for policy 1, policy_version 601164 (0.0008) [2023-12-26 19:44:17,667][105620] Updated weights for policy 1, policy_version 601174 (0.0005) [2023-12-26 19:44:17,726][105620] Updated weights for policy 1, policy_version 601184 (0.0005) [2023-12-26 19:44:18,257][105692] Updated weights for policy 0, policy_version 600376 (0.0009) [2023-12-26 19:44:18,351][105692] Updated weights for policy 0, policy_version 600386 (0.0009) [2023-12-26 19:44:18,382][105620] Updated weights for policy 1, policy_version 601194 (0.0005) [2023-12-26 19:44:18,409][105692] Updated weights for policy 0, policy_version 600396 (0.0008) [2023-12-26 19:44:18,441][105620] Updated weights for policy 1, policy_version 601204 (0.0006) [2023-12-26 19:44:18,499][105620] Updated weights for policy 1, policy_version 601214 (0.0009) [2023-12-26 19:44:18,558][105620] Updated weights for policy 1, policy_version 601224 (0.0009) [2023-12-26 19:44:19,095][105692] Updated weights for policy 0, policy_version 600406 (0.0010) [2023-12-26 19:44:19,156][105692] Updated weights for policy 0, policy_version 600416 (0.0009) [2023-12-26 19:44:19,221][105692] Updated weights for policy 0, policy_version 600426 (0.0009) [2023-12-26 19:44:19,349][105620] Updated weights for policy 1, policy_version 601234 (0.0009) [2023-12-26 19:44:19,408][105620] Updated weights for policy 1, policy_version 601244 (0.0010) [2023-12-26 19:44:19,472][105620] Updated weights for policy 1, policy_version 601254 (0.0009) [2023-12-26 19:44:19,970][105692] Updated weights for policy 0, policy_version 600436 (0.0009) [2023-12-26 19:44:20,025][105692] Updated weights for policy 0, policy_version 600446 (0.0009) [2023-12-26 19:44:20,080][105692] Updated weights for policy 0, policy_version 600456 (0.0008) [2023-12-26 19:44:20,232][105620] Updated weights for policy 1, policy_version 601264 (0.0009) [2023-12-26 19:44:20,283][105620] Updated weights for policy 1, policy_version 601274 (0.0009) [2023-12-26 19:44:20,338][105620] Updated weights for policy 1, policy_version 601284 (0.0009) [2023-12-26 19:44:20,845][105692] Updated weights for policy 0, policy_version 600466 (0.0009) [2023-12-26 19:44:20,907][105692] Updated weights for policy 0, policy_version 600476 (0.0008) [2023-12-26 19:44:20,966][105692] Updated weights for policy 0, policy_version 600486 (0.0009) [2023-12-26 19:44:21,029][105692] Updated weights for policy 0, policy_version 600496 (0.0009) [2023-12-26 19:44:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 307691520. Throughput: 0: 9604.2, 1: 9778.4. Samples: 307681000. Policy #0 lag: (min: 2.0, avg: 15.6, max: 34.0) [2023-12-26 19:44:21,062][104569] Avg episode reward: [(0, '9358.075'), (1, '9255.212')] [2023-12-26 19:44:21,135][105620] Updated weights for policy 1, policy_version 601294 (0.0007) [2023-12-26 19:44:21,200][105620] Updated weights for policy 1, policy_version 601304 (0.0008) [2023-12-26 19:44:21,259][105620] Updated weights for policy 1, policy_version 601314 (0.0008) [2023-12-26 19:44:21,861][105692] Updated weights for policy 0, policy_version 600506 (0.0007) [2023-12-26 19:44:21,925][105692] Updated weights for policy 0, policy_version 600516 (0.0007) [2023-12-26 19:44:21,941][105620] Updated weights for policy 1, policy_version 601324 (0.0006) [2023-12-26 19:44:21,988][105692] Updated weights for policy 0, policy_version 600526 (0.0008) [2023-12-26 19:44:22,002][105620] Updated weights for policy 1, policy_version 601334 (0.0007) [2023-12-26 19:44:22,070][105620] Updated weights for policy 1, policy_version 601344 (0.0008) [2023-12-26 19:44:22,734][105692] Updated weights for policy 0, policy_version 600536 (0.0008) [2023-12-26 19:44:22,789][105692] Updated weights for policy 0, policy_version 600546 (0.0009) [2023-12-26 19:44:22,840][105692] Updated weights for policy 0, policy_version 600556 (0.0009) [2023-12-26 19:44:22,848][105620] Updated weights for policy 1, policy_version 601354 (0.0008) [2023-12-26 19:44:22,907][105620] Updated weights for policy 1, policy_version 601364 (0.0008) [2023-12-26 19:44:22,955][105620] Updated weights for policy 1, policy_version 601374 (0.0009) [2023-12-26 19:44:23,012][105620] Updated weights for policy 1, policy_version 601384 (0.0008) [2023-12-26 19:44:23,627][105692] Updated weights for policy 0, policy_version 600566 (0.0009) [2023-12-26 19:44:23,689][105692] Updated weights for policy 0, policy_version 600576 (0.0009) [2023-12-26 19:44:23,746][105692] Updated weights for policy 0, policy_version 600586 (0.0008) [2023-12-26 19:44:23,755][105620] Updated weights for policy 1, policy_version 601394 (0.0006) [2023-12-26 19:44:23,811][105620] Updated weights for policy 1, policy_version 601404 (0.0008) [2023-12-26 19:44:23,863][105620] Updated weights for policy 1, policy_version 601414 (0.0005) [2023-12-26 19:44:24,466][105620] Updated weights for policy 1, policy_version 601424 (0.0008) [2023-12-26 19:44:24,520][105620] Updated weights for policy 1, policy_version 601434 (0.0009) [2023-12-26 19:44:24,560][105692] Updated weights for policy 0, policy_version 600596 (0.0008) [2023-12-26 19:44:24,570][105620] Updated weights for policy 1, policy_version 601444 (0.0008) [2023-12-26 19:44:24,621][105692] Updated weights for policy 0, policy_version 600606 (0.0006) [2023-12-26 19:44:24,688][105692] Updated weights for policy 0, policy_version 600616 (0.0006) [2023-12-26 19:44:25,315][105620] Updated weights for policy 1, policy_version 601454 (0.0009) [2023-12-26 19:44:25,373][105620] Updated weights for policy 1, policy_version 601464 (0.0010) [2023-12-26 19:44:25,396][105692] Updated weights for policy 0, policy_version 600626 (0.0008) [2023-12-26 19:44:25,436][105620] Updated weights for policy 1, policy_version 601474 (0.0010) [2023-12-26 19:44:25,454][105692] Updated weights for policy 0, policy_version 600636 (0.0005) [2023-12-26 19:44:25,507][105692] Updated weights for policy 0, policy_version 600646 (0.0006) [2023-12-26 19:44:25,567][105692] Updated weights for policy 0, policy_version 600656 (0.0008) [2023-12-26 19:44:26,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 307781632. Throughput: 0: 9492.6, 1: 9773.6. Samples: 307793824. Policy #0 lag: (min: 2.0, avg: 15.6, max: 34.0) [2023-12-26 19:44:26,063][104569] Avg episode reward: [(0, '9357.520'), (1, '9165.411')] [2023-12-26 19:44:26,123][105620] Updated weights for policy 1, policy_version 601484 (0.0009) [2023-12-26 19:44:26,181][105620] Updated weights for policy 1, policy_version 601494 (0.0010) [2023-12-26 19:44:26,237][105620] Updated weights for policy 1, policy_version 601504 (0.0010) [2023-12-26 19:44:26,269][105692] Updated weights for policy 0, policy_version 600666 (0.0009) [2023-12-26 19:44:26,320][105692] Updated weights for policy 0, policy_version 600676 (0.0008) [2023-12-26 19:44:26,368][105692] Updated weights for policy 0, policy_version 600686 (0.0008) [2023-12-26 19:44:26,957][105620] Updated weights for policy 1, policy_version 601514 (0.0009) [2023-12-26 19:44:26,986][105692] Updated weights for policy 0, policy_version 600696 (0.0008) [2023-12-26 19:44:27,024][105620] Updated weights for policy 1, policy_version 601524 (0.0005) [2023-12-26 19:44:27,038][105692] Updated weights for policy 0, policy_version 600706 (0.0006) [2023-12-26 19:44:27,076][105620] Updated weights for policy 1, policy_version 601534 (0.0007) [2023-12-26 19:44:27,087][105692] Updated weights for policy 0, policy_version 600716 (0.0005) [2023-12-26 19:44:27,124][105620] Updated weights for policy 1, policy_version 601544 (0.0010) [2023-12-26 19:44:27,672][105692] Updated weights for policy 0, policy_version 600726 (0.0005) [2023-12-26 19:44:27,731][105692] Updated weights for policy 0, policy_version 600736 (0.0005) [2023-12-26 19:44:27,776][105692] Updated weights for policy 0, policy_version 600746 (0.0005) [2023-12-26 19:44:27,798][105620] Updated weights for policy 1, policy_version 601554 (0.0007) [2023-12-26 19:44:27,870][105620] Updated weights for policy 1, policy_version 601564 (0.0010) [2023-12-26 19:44:27,940][105620] Updated weights for policy 1, policy_version 601574 (0.0010) [2023-12-26 19:44:28,369][105692] Updated weights for policy 0, policy_version 600756 (0.0007) [2023-12-26 19:44:28,425][105692] Updated weights for policy 0, policy_version 600766 (0.0009) [2023-12-26 19:44:28,482][105692] Updated weights for policy 0, policy_version 600776 (0.0006) [2023-12-26 19:44:28,549][105620] Updated weights for policy 1, policy_version 601584 (0.0009) [2023-12-26 19:44:28,603][105620] Updated weights for policy 1, policy_version 601594 (0.0005) [2023-12-26 19:44:28,648][105620] Updated weights for policy 1, policy_version 601604 (0.0005) [2023-12-26 19:44:29,049][105692] Updated weights for policy 0, policy_version 600786 (0.0009) [2023-12-26 19:44:29,113][105692] Updated weights for policy 0, policy_version 600796 (0.0006) [2023-12-26 19:44:29,173][105692] Updated weights for policy 0, policy_version 600806 (0.0008) [2023-12-26 19:44:29,240][105692] Updated weights for policy 0, policy_version 600816 (0.0008) [2023-12-26 19:44:29,289][105620] Updated weights for policy 1, policy_version 601614 (0.0007) [2023-12-26 19:44:29,354][105620] Updated weights for policy 1, policy_version 601624 (0.0008) [2023-12-26 19:44:29,416][105620] Updated weights for policy 1, policy_version 601634 (0.0010) [2023-12-26 19:44:29,997][105692] Updated weights for policy 0, policy_version 600826 (0.0011) [2023-12-26 19:44:30,057][105692] Updated weights for policy 0, policy_version 600836 (0.0011) [2023-12-26 19:44:30,117][105692] Updated weights for policy 0, policy_version 600846 (0.0011) [2023-12-26 19:44:30,153][105620] Updated weights for policy 1, policy_version 601644 (0.0009) [2023-12-26 19:44:30,219][105620] Updated weights for policy 1, policy_version 601654 (0.0008) [2023-12-26 19:44:30,281][105620] Updated weights for policy 1, policy_version 601664 (0.0008) [2023-12-26 19:44:30,838][105692] Updated weights for policy 0, policy_version 600856 (0.0008) [2023-12-26 19:44:30,894][105692] Updated weights for policy 0, policy_version 600866 (0.0009) [2023-12-26 19:44:30,948][105692] Updated weights for policy 0, policy_version 600878 (0.0010) [2023-12-26 19:44:30,995][105620] Updated weights for policy 1, policy_version 601674 (0.0008) [2023-12-26 19:44:31,057][105620] Updated weights for policy 1, policy_version 601684 (0.0009) [2023-12-26 19:44:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19438.7). Total num frames: 307888128. Throughput: 0: 9613.3, 1: 9837.2. Samples: 307858396. Policy #0 lag: (min: 2.0, avg: 15.6, max: 34.0) [2023-12-26 19:44:31,062][104569] Avg episode reward: [(0, '9357.272'), (1, '9256.701')] [2023-12-26 19:44:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000600880_153845760.pth... [2023-12-26 19:44:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000599728_153550848.pth [2023-12-26 19:44:31,115][105620] Updated weights for policy 1, policy_version 601694 (0.0009) [2023-12-26 19:44:31,178][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000601704_154050560.pth... [2023-12-26 19:44:31,180][105620] Updated weights for policy 1, policy_version 601704 (0.0009) [2023-12-26 19:44:31,182][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000600552_153755648.pth [2023-12-26 19:44:31,811][105692] Updated weights for policy 0, policy_version 600888 (0.0009) [2023-12-26 19:44:31,872][105692] Updated weights for policy 0, policy_version 600898 (0.0009) [2023-12-26 19:44:31,914][105620] Updated weights for policy 1, policy_version 601714 (0.0007) [2023-12-26 19:44:31,934][105692] Updated weights for policy 0, policy_version 600908 (0.0007) [2023-12-26 19:44:31,968][105620] Updated weights for policy 1, policy_version 601724 (0.0008) [2023-12-26 19:44:32,018][105620] Updated weights for policy 1, policy_version 601734 (0.0007) [2023-12-26 19:44:32,712][105692] Updated weights for policy 0, policy_version 600918 (0.0006) [2023-12-26 19:44:32,766][105692] Updated weights for policy 0, policy_version 600928 (0.0006) [2023-12-26 19:44:32,787][105620] Updated weights for policy 1, policy_version 601744 (0.0008) [2023-12-26 19:44:32,817][105692] Updated weights for policy 0, policy_version 600938 (0.0005) [2023-12-26 19:44:32,847][105620] Updated weights for policy 1, policy_version 601754 (0.0008) [2023-12-26 19:44:32,907][105620] Updated weights for policy 1, policy_version 601764 (0.0009) [2023-12-26 19:44:33,342][105692] Updated weights for policy 0, policy_version 600948 (0.0005) [2023-12-26 19:44:33,397][105692] Updated weights for policy 0, policy_version 600958 (0.0008) [2023-12-26 19:44:33,447][105692] Updated weights for policy 0, policy_version 600968 (0.0009) [2023-12-26 19:44:33,730][105620] Updated weights for policy 1, policy_version 601774 (0.0010) [2023-12-26 19:44:33,789][105620] Updated weights for policy 1, policy_version 601784 (0.0008) [2023-12-26 19:44:33,854][105620] Updated weights for policy 1, policy_version 601794 (0.0009) [2023-12-26 19:44:34,101][105692] Updated weights for policy 0, policy_version 600978 (0.0007) [2023-12-26 19:44:34,156][105692] Updated weights for policy 0, policy_version 600988 (0.0009) [2023-12-26 19:44:34,214][105692] Updated weights for policy 0, policy_version 600998 (0.0007) [2023-12-26 19:44:34,267][105692] Updated weights for policy 0, policy_version 601008 (0.0008) [2023-12-26 19:44:34,609][105620] Updated weights for policy 1, policy_version 601804 (0.0009) [2023-12-26 19:44:34,658][105620] Updated weights for policy 1, policy_version 601814 (0.0009) [2023-12-26 19:44:34,723][105620] Updated weights for policy 1, policy_version 601824 (0.0009) [2023-12-26 19:44:34,978][105692] Updated weights for policy 0, policy_version 601018 (0.0009) [2023-12-26 19:44:35,037][105692] Updated weights for policy 0, policy_version 601028 (0.0009) [2023-12-26 19:44:35,092][105692] Updated weights for policy 0, policy_version 601038 (0.0009) [2023-12-26 19:44:35,483][105620] Updated weights for policy 1, policy_version 601834 (0.0009) [2023-12-26 19:44:35,531][105620] Updated weights for policy 1, policy_version 601844 (0.0005) [2023-12-26 19:44:35,579][105620] Updated weights for policy 1, policy_version 601854 (0.0009) [2023-12-26 19:44:35,624][105620] Updated weights for policy 1, policy_version 601864 (0.0008) [2023-12-26 19:44:35,855][105692] Updated weights for policy 0, policy_version 601048 (0.0009) [2023-12-26 19:44:35,910][105692] Updated weights for policy 0, policy_version 601058 (0.0009) [2023-12-26 19:44:35,961][105692] Updated weights for policy 0, policy_version 601068 (0.0009) [2023-12-26 19:44:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 307986432. Throughput: 0: 9638.7, 1: 9704.6. Samples: 307973464. Policy #0 lag: (min: 2.0, avg: 15.6, max: 34.0) [2023-12-26 19:44:36,062][104569] Avg episode reward: [(0, '9357.150'), (1, '9255.675')] [2023-12-26 19:44:36,394][105620] Updated weights for policy 1, policy_version 601874 (0.0009) [2023-12-26 19:44:36,454][105620] Updated weights for policy 1, policy_version 601884 (0.0009) [2023-12-26 19:44:36,515][105620] Updated weights for policy 1, policy_version 601894 (0.0007) [2023-12-26 19:44:36,708][105692] Updated weights for policy 0, policy_version 601078 (0.0009) [2023-12-26 19:44:36,762][105692] Updated weights for policy 0, policy_version 601088 (0.0009) [2023-12-26 19:44:36,819][105692] Updated weights for policy 0, policy_version 601098 (0.0009) [2023-12-26 19:44:37,294][105620] Updated weights for policy 1, policy_version 601904 (0.0010) [2023-12-26 19:44:37,353][105620] Updated weights for policy 1, policy_version 601914 (0.0008) [2023-12-26 19:44:37,403][105620] Updated weights for policy 1, policy_version 601924 (0.0005) [2023-12-26 19:44:37,467][105692] Updated weights for policy 0, policy_version 601108 (0.0008) [2023-12-26 19:44:37,522][105692] Updated weights for policy 0, policy_version 601118 (0.0006) [2023-12-26 19:44:37,589][105692] Updated weights for policy 0, policy_version 601128 (0.0008) [2023-12-26 19:44:37,952][105620] Updated weights for policy 1, policy_version 601934 (0.0006) [2023-12-26 19:44:37,998][105620] Updated weights for policy 1, policy_version 601944 (0.0005) [2023-12-26 19:44:38,056][105620] Updated weights for policy 1, policy_version 601954 (0.0005) [2023-12-26 19:44:38,304][105692] Updated weights for policy 0, policy_version 601138 (0.0009) [2023-12-26 19:44:38,369][105692] Updated weights for policy 0, policy_version 601148 (0.0009) [2023-12-26 19:44:38,418][105692] Updated weights for policy 0, policy_version 601158 (0.0008) [2023-12-26 19:44:38,479][105692] Updated weights for policy 0, policy_version 601168 (0.0009) [2023-12-26 19:44:38,663][105620] Updated weights for policy 1, policy_version 601964 (0.0006) [2023-12-26 19:44:38,721][105620] Updated weights for policy 1, policy_version 601974 (0.0006) [2023-12-26 19:44:38,774][105620] Updated weights for policy 1, policy_version 601984 (0.0005) [2023-12-26 19:44:39,347][105620] Updated weights for policy 1, policy_version 601994 (0.0007) [2023-12-26 19:44:39,355][105692] Updated weights for policy 0, policy_version 601178 (0.0008) [2023-12-26 19:44:39,411][105620] Updated weights for policy 1, policy_version 602004 (0.0009) [2023-12-26 19:44:39,424][105692] Updated weights for policy 0, policy_version 601188 (0.0009) [2023-12-26 19:44:39,467][105620] Updated weights for policy 1, policy_version 602014 (0.0006) [2023-12-26 19:44:39,492][105692] Updated weights for policy 0, policy_version 601198 (0.0008) [2023-12-26 19:44:39,527][105620] Updated weights for policy 1, policy_version 602024 (0.0008) [2023-12-26 19:44:40,159][105620] Updated weights for policy 1, policy_version 602034 (0.0010) [2023-12-26 19:44:40,214][105620] Updated weights for policy 1, policy_version 602044 (0.0008) [2023-12-26 19:44:40,278][105620] Updated weights for policy 1, policy_version 602054 (0.0008) [2023-12-26 19:44:40,304][105692] Updated weights for policy 0, policy_version 601208 (0.0010) [2023-12-26 19:44:40,375][105692] Updated weights for policy 0, policy_version 601218 (0.0011) [2023-12-26 19:44:40,433][105692] Updated weights for policy 0, policy_version 601228 (0.0011) [2023-12-26 19:44:41,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19438.7). Total num frames: 308076544. Throughput: 0: 9570.8, 1: 9765.9. Samples: 308090400. Policy #0 lag: (min: 2.0, avg: 15.6, max: 34.0) [2023-12-26 19:44:41,063][104569] Avg episode reward: [(0, '9356.988'), (1, '9255.719')] [2023-12-26 19:44:41,079][105620] Updated weights for policy 1, policy_version 602064 (0.0008) [2023-12-26 19:44:41,132][105692] Updated weights for policy 0, policy_version 601238 (0.0011) [2023-12-26 19:44:41,141][105620] Updated weights for policy 1, policy_version 602074 (0.0007) [2023-12-26 19:44:41,198][105692] Updated weights for policy 0, policy_version 601248 (0.0008) [2023-12-26 19:44:41,201][105620] Updated weights for policy 1, policy_version 602084 (0.0007) [2023-12-26 19:44:41,268][105692] Updated weights for policy 0, policy_version 601258 (0.0010) [2023-12-26 19:44:41,862][105620] Updated weights for policy 1, policy_version 602094 (0.0006) [2023-12-26 19:44:41,930][105620] Updated weights for policy 1, policy_version 602104 (0.0006) [2023-12-26 19:44:41,996][105620] Updated weights for policy 1, policy_version 602114 (0.0005) [2023-12-26 19:44:42,081][105692] Updated weights for policy 0, policy_version 601268 (0.0009) [2023-12-26 19:44:42,144][105692] Updated weights for policy 0, policy_version 601278 (0.0009) [2023-12-26 19:44:42,202][105692] Updated weights for policy 0, policy_version 601288 (0.0009) [2023-12-26 19:44:42,678][105620] Updated weights for policy 1, policy_version 602124 (0.0007) [2023-12-26 19:44:42,732][105620] Updated weights for policy 1, policy_version 602134 (0.0008) [2023-12-26 19:44:42,789][105620] Updated weights for policy 1, policy_version 602144 (0.0006) [2023-12-26 19:44:42,873][105692] Updated weights for policy 0, policy_version 601298 (0.0008) [2023-12-26 19:44:42,932][105692] Updated weights for policy 0, policy_version 601308 (0.0006) [2023-12-26 19:44:42,988][105692] Updated weights for policy 0, policy_version 601318 (0.0006) [2023-12-26 19:44:43,038][105692] Updated weights for policy 0, policy_version 601328 (0.0005) [2023-12-26 19:44:43,495][105620] Updated weights for policy 1, policy_version 602154 (0.0009) [2023-12-26 19:44:43,545][105620] Updated weights for policy 1, policy_version 602164 (0.0010) [2023-12-26 19:44:43,595][105620] Updated weights for policy 1, policy_version 602174 (0.0010) [2023-12-26 19:44:43,645][105692] Updated weights for policy 0, policy_version 601338 (0.0010) [2023-12-26 19:44:43,658][105620] Updated weights for policy 1, policy_version 602184 (0.0005) [2023-12-26 19:44:43,700][105692] Updated weights for policy 0, policy_version 601348 (0.0010) [2023-12-26 19:44:43,763][105692] Updated weights for policy 0, policy_version 601358 (0.0007) [2023-12-26 19:44:44,238][105620] Updated weights for policy 1, policy_version 602194 (0.0005) [2023-12-26 19:44:44,306][105620] Updated weights for policy 1, policy_version 602204 (0.0005) [2023-12-26 19:44:44,354][105620] Updated weights for policy 1, policy_version 602214 (0.0005) [2023-12-26 19:44:44,504][105692] Updated weights for policy 0, policy_version 601368 (0.0007) [2023-12-26 19:44:44,557][105692] Updated weights for policy 0, policy_version 601378 (0.0008) [2023-12-26 19:44:44,609][105692] Updated weights for policy 0, policy_version 601388 (0.0009) [2023-12-26 19:44:44,980][105620] Updated weights for policy 1, policy_version 602224 (0.0009) [2023-12-26 19:44:45,041][105620] Updated weights for policy 1, policy_version 602234 (0.0011) [2023-12-26 19:44:45,100][105620] Updated weights for policy 1, policy_version 602244 (0.0010) [2023-12-26 19:44:45,361][105692] Updated weights for policy 0, policy_version 601398 (0.0009) [2023-12-26 19:44:45,416][105692] Updated weights for policy 0, policy_version 601408 (0.0008) [2023-12-26 19:44:45,466][105692] Updated weights for policy 0, policy_version 601418 (0.0008) [2023-12-26 19:44:45,850][105620] Updated weights for policy 1, policy_version 602254 (0.0010) [2023-12-26 19:44:45,905][105620] Updated weights for policy 1, policy_version 602264 (0.0010) [2023-12-26 19:44:45,956][105620] Updated weights for policy 1, policy_version 602274 (0.0010) [2023-12-26 19:44:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 308183040. Throughput: 0: 9596.4, 1: 9756.6. Samples: 308150608. Policy #0 lag: (min: 2.0, avg: 15.6, max: 34.0) [2023-12-26 19:44:46,062][104569] Avg episode reward: [(0, '9265.496'), (1, '9256.546')] [2023-12-26 19:44:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000601424_153985024.pth... [2023-12-26 19:44:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000602280_154198016.pth... [2023-12-26 19:44:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000600304_153698304.pth [2023-12-26 19:44:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000601128_153903104.pth [2023-12-26 19:44:46,223][105692] Updated weights for policy 0, policy_version 601428 (0.0007) [2023-12-26 19:44:46,276][105692] Updated weights for policy 0, policy_version 601438 (0.0005) [2023-12-26 19:44:46,329][105585] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000004 [2023-12-26 19:44:46,330][105692] Updated weights for policy 0, policy_version 601448 (0.0005) [2023-12-26 19:44:46,617][105620] Updated weights for policy 1, policy_version 602284 (0.0010) [2023-12-26 19:44:46,661][105620] Updated weights for policy 1, policy_version 602294 (0.0007) [2023-12-26 19:44:46,714][105620] Updated weights for policy 1, policy_version 602304 (0.0007) [2023-12-26 19:44:47,018][105692] Updated weights for policy 0, policy_version 601458 (0.0010) [2023-12-26 19:44:47,072][105692] Updated weights for policy 0, policy_version 601469 (0.0010) [2023-12-26 19:44:47,126][105692] Updated weights for policy 0, policy_version 601479 (0.0011) [2023-12-26 19:44:47,284][105620] Updated weights for policy 1, policy_version 602314 (0.0007) [2023-12-26 19:44:47,341][105620] Updated weights for policy 1, policy_version 602324 (0.0009) [2023-12-26 19:44:47,408][105620] Updated weights for policy 1, policy_version 602334 (0.0010) [2023-12-26 19:44:47,474][105620] Updated weights for policy 1, policy_version 602344 (0.0009) [2023-12-26 19:44:47,892][105692] Updated weights for policy 0, policy_version 601489 (0.0009) [2023-12-26 19:44:47,943][105692] Updated weights for policy 0, policy_version 601499 (0.0009) [2023-12-26 19:44:48,003][105692] Updated weights for policy 0, policy_version 601509 (0.0008) [2023-12-26 19:44:48,069][105620] Updated weights for policy 1, policy_version 602354 (0.0010) [2023-12-26 19:44:48,122][105620] Updated weights for policy 1, policy_version 602364 (0.0011) [2023-12-26 19:44:48,175][105620] Updated weights for policy 1, policy_version 602374 (0.0011) [2023-12-26 19:44:48,748][105692] Updated weights for policy 0, policy_version 601519 (0.0008) [2023-12-26 19:44:48,813][105692] Updated weights for policy 0, policy_version 601529 (0.0008) [2023-12-26 19:44:48,881][105692] Updated weights for policy 0, policy_version 601539 (0.0009) [2023-12-26 19:44:48,953][105620] Updated weights for policy 1, policy_version 602384 (0.0006) [2023-12-26 19:44:49,021][105620] Updated weights for policy 1, policy_version 602394 (0.0006) [2023-12-26 19:44:49,087][105620] Updated weights for policy 1, policy_version 602404 (0.0006) [2023-12-26 19:44:49,603][105692] Updated weights for policy 0, policy_version 601549 (0.0008) [2023-12-26 19:44:49,666][105692] Updated weights for policy 0, policy_version 601559 (0.0007) [2023-12-26 19:44:49,720][105692] Updated weights for policy 0, policy_version 601569 (0.0010) [2023-12-26 19:44:49,780][105620] Updated weights for policy 1, policy_version 602414 (0.0006) [2023-12-26 19:44:49,847][105620] Updated weights for policy 1, policy_version 602424 (0.0009) [2023-12-26 19:44:49,908][105620] Updated weights for policy 1, policy_version 602434 (0.0009) [2023-12-26 19:44:50,491][105620] Updated weights for policy 1, policy_version 602444 (0.0007) [2023-12-26 19:44:50,497][105692] Updated weights for policy 0, policy_version 601579 (0.0007) [2023-12-26 19:44:50,557][105620] Updated weights for policy 1, policy_version 602454 (0.0007) [2023-12-26 19:44:50,558][105692] Updated weights for policy 0, policy_version 601589 (0.0006) [2023-12-26 19:44:50,622][105692] Updated weights for policy 0, policy_version 601599 (0.0008) [2023-12-26 19:44:50,625][105620] Updated weights for policy 1, policy_version 602464 (0.0007) [2023-12-26 19:44:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 308281344. Throughput: 0: 9643.5, 1: 9821.4. Samples: 308270524. Policy #0 lag: (min: 2.0, avg: 15.6, max: 34.0) [2023-12-26 19:44:51,062][104569] Avg episode reward: [(0, '9265.438'), (1, '9346.631')] [2023-12-26 19:44:51,187][105692] Updated weights for policy 0, policy_version 601609 (0.0006) [2023-12-26 19:44:51,250][105692] Updated weights for policy 0, policy_version 601619 (0.0007) [2023-12-26 19:44:51,316][105692] Updated weights for policy 0, policy_version 601629 (0.0008) [2023-12-26 19:44:51,383][105692] Updated weights for policy 0, policy_version 601639 (0.0009) [2023-12-26 19:44:51,441][105620] Updated weights for policy 1, policy_version 602474 (0.0009) [2023-12-26 19:44:51,504][105620] Updated weights for policy 1, policy_version 602484 (0.0010) [2023-12-26 19:44:51,557][105620] Updated weights for policy 1, policy_version 602494 (0.0010) [2023-12-26 19:44:51,609][105620] Updated weights for policy 1, policy_version 602504 (0.0009) [2023-12-26 19:44:52,034][105692] Updated weights for policy 0, policy_version 601649 (0.0008) [2023-12-26 19:44:52,100][105692] Updated weights for policy 0, policy_version 601659 (0.0009) [2023-12-26 19:44:52,152][105692] Updated weights for policy 0, policy_version 601669 (0.0009) [2023-12-26 19:44:52,295][105620] Updated weights for policy 1, policy_version 602514 (0.0007) [2023-12-26 19:44:52,365][105620] Updated weights for policy 1, policy_version 602524 (0.0008) [2023-12-26 19:44:52,434][105620] Updated weights for policy 1, policy_version 602534 (0.0008) [2023-12-26 19:44:52,912][105692] Updated weights for policy 0, policy_version 601679 (0.0010) [2023-12-26 19:44:52,968][105692] Updated weights for policy 0, policy_version 601689 (0.0010) [2023-12-26 19:44:52,986][105620] Updated weights for policy 1, policy_version 602544 (0.0006) [2023-12-26 19:44:53,024][105692] Updated weights for policy 0, policy_version 601699 (0.0011) [2023-12-26 19:44:53,043][105620] Updated weights for policy 1, policy_version 602554 (0.0006) [2023-12-26 19:44:53,096][105620] Updated weights for policy 1, policy_version 602564 (0.0007) [2023-12-26 19:44:53,730][105620] Updated weights for policy 1, policy_version 602574 (0.0005) [2023-12-26 19:44:53,751][105692] Updated weights for policy 0, policy_version 601709 (0.0011) [2023-12-26 19:44:53,778][105620] Updated weights for policy 1, policy_version 602584 (0.0005) [2023-12-26 19:44:53,805][105692] Updated weights for policy 0, policy_version 601719 (0.0010) [2023-12-26 19:44:53,829][105620] Updated weights for policy 1, policy_version 602594 (0.0007) [2023-12-26 19:44:53,860][105692] Updated weights for policy 0, policy_version 601729 (0.0010) [2023-12-26 19:44:54,565][105620] Updated weights for policy 1, policy_version 602604 (0.0010) [2023-12-26 19:44:54,612][105692] Updated weights for policy 0, policy_version 601739 (0.0010) [2023-12-26 19:44:54,623][105620] Updated weights for policy 1, policy_version 602614 (0.0010) [2023-12-26 19:44:54,663][105692] Updated weights for policy 0, policy_version 601749 (0.0010) [2023-12-26 19:44:54,682][105620] Updated weights for policy 1, policy_version 602624 (0.0010) [2023-12-26 19:44:54,715][105692] Updated weights for policy 0, policy_version 601759 (0.0010) [2023-12-26 19:44:55,349][105620] Updated weights for policy 1, policy_version 602634 (0.0010) [2023-12-26 19:44:55,411][105620] Updated weights for policy 1, policy_version 602644 (0.0010) [2023-12-26 19:44:55,469][105620] Updated weights for policy 1, policy_version 602654 (0.0011) [2023-12-26 19:44:55,486][105692] Updated weights for policy 0, policy_version 601769 (0.0010) [2023-12-26 19:44:55,528][105620] Updated weights for policy 1, policy_version 602664 (0.0010) [2023-12-26 19:44:55,544][105692] Updated weights for policy 0, policy_version 601779 (0.0010) [2023-12-26 19:44:55,604][105692] Updated weights for policy 0, policy_version 601789 (0.0010) [2023-12-26 19:44:55,662][105692] Updated weights for policy 0, policy_version 601799 (0.0010) [2023-12-26 19:44:56,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.6, 300 sec: 19466.4). Total num frames: 308379648. Throughput: 0: 9662.3, 1: 9908.8. Samples: 308389820. Policy #0 lag: (min: 2.0, avg: 15.6, max: 34.0) [2023-12-26 19:44:56,063][104569] Avg episode reward: [(0, '9266.652'), (1, '9347.305')] [2023-12-26 19:44:56,264][105620] Updated weights for policy 1, policy_version 602674 (0.0008) [2023-12-26 19:44:56,312][105620] Updated weights for policy 1, policy_version 602684 (0.0008) [2023-12-26 19:44:56,363][105620] Updated weights for policy 1, policy_version 602694 (0.0009) [2023-12-26 19:44:56,404][105692] Updated weights for policy 0, policy_version 601809 (0.0010) [2023-12-26 19:44:56,461][105692] Updated weights for policy 0, policy_version 601819 (0.0010) [2023-12-26 19:44:56,523][105692] Updated weights for policy 0, policy_version 601829 (0.0010) [2023-12-26 19:44:57,125][105620] Updated weights for policy 1, policy_version 602704 (0.0008) [2023-12-26 19:44:57,184][105692] Updated weights for policy 0, policy_version 601839 (0.0007) [2023-12-26 19:44:57,189][105620] Updated weights for policy 1, policy_version 602714 (0.0008) [2023-12-26 19:44:57,243][105692] Updated weights for policy 0, policy_version 601849 (0.0005) [2023-12-26 19:44:57,251][105620] Updated weights for policy 1, policy_version 602724 (0.0007) [2023-12-26 19:44:57,306][105692] Updated weights for policy 0, policy_version 601859 (0.0006) [2023-12-26 19:44:57,821][105620] Updated weights for policy 1, policy_version 602734 (0.0008) [2023-12-26 19:44:57,893][105620] Updated weights for policy 1, policy_version 602744 (0.0008) [2023-12-26 19:44:57,910][105692] Updated weights for policy 0, policy_version 601869 (0.0005) [2023-12-26 19:44:57,956][105692] Updated weights for policy 0, policy_version 601879 (0.0005) [2023-12-26 19:44:57,962][105620] Updated weights for policy 1, policy_version 602754 (0.0009) [2023-12-26 19:44:58,004][105692] Updated weights for policy 0, policy_version 601889 (0.0005) [2023-12-26 19:44:58,613][105620] Updated weights for policy 1, policy_version 602764 (0.0008) [2023-12-26 19:44:58,678][105620] Updated weights for policy 1, policy_version 602774 (0.0009) [2023-12-26 19:44:58,743][105620] Updated weights for policy 1, policy_version 602784 (0.0008) [2023-12-26 19:44:58,759][105692] Updated weights for policy 0, policy_version 601899 (0.0006) [2023-12-26 19:44:58,824][105692] Updated weights for policy 0, policy_version 601909 (0.0007) [2023-12-26 19:44:58,896][105692] Updated weights for policy 0, policy_version 601919 (0.0007) [2023-12-26 19:44:59,505][105620] Updated weights for policy 1, policy_version 602794 (0.0007) [2023-12-26 19:44:59,562][105620] Updated weights for policy 1, policy_version 602804 (0.0007) [2023-12-26 19:44:59,619][105620] Updated weights for policy 1, policy_version 602814 (0.0006) [2023-12-26 19:44:59,667][105620] Updated weights for policy 1, policy_version 602824 (0.0009) [2023-12-26 19:44:59,710][105692] Updated weights for policy 0, policy_version 601929 (0.0009) [2023-12-26 19:44:59,769][105692] Updated weights for policy 0, policy_version 601939 (0.0010) [2023-12-26 19:44:59,828][105692] Updated weights for policy 0, policy_version 601949 (0.0008) [2023-12-26 19:44:59,884][105692] Updated weights for policy 0, policy_version 601959 (0.0006) [2023-12-26 19:45:00,361][105620] Updated weights for policy 1, policy_version 602834 (0.0009) [2023-12-26 19:45:00,424][105620] Updated weights for policy 1, policy_version 602844 (0.0009) [2023-12-26 19:45:00,480][105620] Updated weights for policy 1, policy_version 602854 (0.0007) [2023-12-26 19:45:00,678][105692] Updated weights for policy 0, policy_version 601969 (0.0008) [2023-12-26 19:45:00,731][105692] Updated weights for policy 0, policy_version 601979 (0.0008) [2023-12-26 19:45:00,788][105692] Updated weights for policy 0, policy_version 601989 (0.0009) [2023-12-26 19:45:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 308477952. Throughput: 0: 9681.6, 1: 9950.4. Samples: 308450124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:45:01,062][104569] Avg episode reward: [(0, '9266.675'), (1, '9165.385')] [2023-12-26 19:45:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000602856_154345472.pth... [2023-12-26 19:45:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000601992_154132480.pth... [2023-12-26 19:45:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000601704_154050560.pth [2023-12-26 19:45:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000600880_153845760.pth [2023-12-26 19:45:01,149][105620] Updated weights for policy 1, policy_version 602865 (0.0006) [2023-12-26 19:45:01,211][105620] Updated weights for policy 1, policy_version 602875 (0.0006) [2023-12-26 19:45:01,272][105620] Updated weights for policy 1, policy_version 602885 (0.0007) [2023-12-26 19:45:01,583][105692] Updated weights for policy 0, policy_version 601999 (0.0010) [2023-12-26 19:45:01,653][105692] Updated weights for policy 0, policy_version 602009 (0.0011) [2023-12-26 19:45:01,710][105692] Updated weights for policy 0, policy_version 602019 (0.0011) [2023-12-26 19:45:01,944][105620] Updated weights for policy 1, policy_version 602895 (0.0010) [2023-12-26 19:45:02,010][105620] Updated weights for policy 1, policy_version 602905 (0.0010) [2023-12-26 19:45:02,071][105620] Updated weights for policy 1, policy_version 602915 (0.0010) [2023-12-26 19:45:02,416][105692] Updated weights for policy 0, policy_version 602029 (0.0011) [2023-12-26 19:45:02,467][105692] Updated weights for policy 0, policy_version 602039 (0.0010) [2023-12-26 19:45:02,516][105692] Updated weights for policy 0, policy_version 602049 (0.0010) [2023-12-26 19:45:02,696][105620] Updated weights for policy 1, policy_version 602925 (0.0008) [2023-12-26 19:45:02,754][105620] Updated weights for policy 1, policy_version 602935 (0.0010) [2023-12-26 19:45:02,812][105620] Updated weights for policy 1, policy_version 602945 (0.0010) [2023-12-26 19:45:03,301][105692] Updated weights for policy 0, policy_version 602059 (0.0010) [2023-12-26 19:45:03,348][105692] Updated weights for policy 0, policy_version 602069 (0.0010) [2023-12-26 19:45:03,392][105692] Updated weights for policy 0, policy_version 602079 (0.0010) [2023-12-26 19:45:03,467][105620] Updated weights for policy 1, policy_version 602955 (0.0007) [2023-12-26 19:45:03,512][105620] Updated weights for policy 1, policy_version 602965 (0.0005) [2023-12-26 19:45:03,555][105620] Updated weights for policy 1, policy_version 602975 (0.0005) [2023-12-26 19:45:04,177][105692] Updated weights for policy 0, policy_version 602089 (0.0010) [2023-12-26 19:45:04,226][105692] Updated weights for policy 0, policy_version 602099 (0.0008) [2023-12-26 19:45:04,283][105692] Updated weights for policy 0, policy_version 602109 (0.0008) [2023-12-26 19:45:04,295][105620] Updated weights for policy 1, policy_version 602985 (0.0010) [2023-12-26 19:45:04,341][105692] Updated weights for policy 0, policy_version 602119 (0.0006) [2023-12-26 19:45:04,362][105620] Updated weights for policy 1, policy_version 602995 (0.0011) [2023-12-26 19:45:04,425][105620] Updated weights for policy 1, policy_version 603005 (0.0011) [2023-12-26 19:45:04,474][105620] Updated weights for policy 1, policy_version 603015 (0.0011) [2023-12-26 19:45:05,077][105692] Updated weights for policy 0, policy_version 602129 (0.0009) [2023-12-26 19:45:05,095][105620] Updated weights for policy 1, policy_version 603025 (0.0006) [2023-12-26 19:45:05,124][105692] Updated weights for policy 0, policy_version 602139 (0.0010) [2023-12-26 19:45:05,146][105620] Updated weights for policy 1, policy_version 603035 (0.0005) [2023-12-26 19:45:05,177][105692] Updated weights for policy 0, policy_version 602149 (0.0005) [2023-12-26 19:45:05,201][105620] Updated weights for policy 1, policy_version 603045 (0.0007) [2023-12-26 19:45:05,837][105692] Updated weights for policy 0, policy_version 602159 (0.0009) [2023-12-26 19:45:05,858][105620] Updated weights for policy 1, policy_version 603055 (0.0008) [2023-12-26 19:45:05,889][105692] Updated weights for policy 0, policy_version 602169 (0.0010) [2023-12-26 19:45:05,910][105620] Updated weights for policy 1, policy_version 603065 (0.0010) [2023-12-26 19:45:05,933][105692] Updated weights for policy 0, policy_version 602179 (0.0010) [2023-12-26 19:45:05,966][105620] Updated weights for policy 1, policy_version 603075 (0.0010) [2023-12-26 19:45:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 308584448. Throughput: 0: 9635.4, 1: 10024.1. Samples: 308565684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:45:06,063][104569] Avg episode reward: [(0, '9356.871'), (1, '9072.945')] [2023-12-26 19:45:06,683][105620] Updated weights for policy 1, policy_version 603085 (0.0011) [2023-12-26 19:45:06,702][105692] Updated weights for policy 0, policy_version 602189 (0.0011) [2023-12-26 19:45:06,739][105620] Updated weights for policy 1, policy_version 603095 (0.0011) [2023-12-26 19:45:06,758][105692] Updated weights for policy 0, policy_version 602199 (0.0011) [2023-12-26 19:45:06,792][105620] Updated weights for policy 1, policy_version 603105 (0.0011) [2023-12-26 19:45:06,820][105692] Updated weights for policy 0, policy_version 602209 (0.0010) [2023-12-26 19:45:07,480][105620] Updated weights for policy 1, policy_version 603115 (0.0007) [2023-12-26 19:45:07,540][105620] Updated weights for policy 1, policy_version 603125 (0.0010) [2023-12-26 19:45:07,570][105692] Updated weights for policy 0, policy_version 602219 (0.0010) [2023-12-26 19:45:07,598][105620] Updated weights for policy 1, policy_version 603135 (0.0010) [2023-12-26 19:45:07,622][105692] Updated weights for policy 0, policy_version 602229 (0.0010) [2023-12-26 19:45:07,679][105692] Updated weights for policy 0, policy_version 602239 (0.0010) [2023-12-26 19:45:08,255][105620] Updated weights for policy 1, policy_version 603145 (0.0010) [2023-12-26 19:45:08,319][105620] Updated weights for policy 1, policy_version 603155 (0.0006) [2023-12-26 19:45:08,386][105620] Updated weights for policy 1, policy_version 603165 (0.0011) [2023-12-26 19:45:08,433][105692] Updated weights for policy 0, policy_version 602249 (0.0010) [2023-12-26 19:45:08,450][105620] Updated weights for policy 1, policy_version 603175 (0.0011) [2023-12-26 19:45:08,502][105692] Updated weights for policy 0, policy_version 602259 (0.0009) [2023-12-26 19:45:08,565][105692] Updated weights for policy 0, policy_version 602269 (0.0010) [2023-12-26 19:45:08,625][105692] Updated weights for policy 0, policy_version 602279 (0.0011) [2023-12-26 19:45:09,154][105620] Updated weights for policy 1, policy_version 603185 (0.0009) [2023-12-26 19:45:09,211][105620] Updated weights for policy 1, policy_version 603195 (0.0008) [2023-12-26 19:45:09,277][105620] Updated weights for policy 1, policy_version 603205 (0.0007) [2023-12-26 19:45:09,378][105692] Updated weights for policy 0, policy_version 602289 (0.0007) [2023-12-26 19:45:09,451][105692] Updated weights for policy 0, policy_version 602299 (0.0009) [2023-12-26 19:45:09,517][105692] Updated weights for policy 0, policy_version 602309 (0.0006) [2023-12-26 19:45:10,009][105620] Updated weights for policy 1, policy_version 603215 (0.0009) [2023-12-26 19:45:10,071][105620] Updated weights for policy 1, policy_version 603225 (0.0008) [2023-12-26 19:45:10,140][105620] Updated weights for policy 1, policy_version 603235 (0.0008) [2023-12-26 19:45:10,190][105692] Updated weights for policy 0, policy_version 602319 (0.0009) [2023-12-26 19:45:10,245][105692] Updated weights for policy 0, policy_version 602329 (0.0010) [2023-12-26 19:45:10,301][105692] Updated weights for policy 0, policy_version 602339 (0.0011) [2023-12-26 19:45:10,831][105620] Updated weights for policy 1, policy_version 603245 (0.0008) [2023-12-26 19:45:10,890][105620] Updated weights for policy 1, policy_version 603255 (0.0007) [2023-12-26 19:45:10,956][105620] Updated weights for policy 1, policy_version 603265 (0.0007) [2023-12-26 19:45:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 308674560. Throughput: 0: 9700.7, 1: 10068.4. Samples: 308683436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:45:11,062][104569] Avg episode reward: [(0, '9265.559'), (1, '9161.652')] [2023-12-26 19:45:11,103][105692] Updated weights for policy 0, policy_version 602349 (0.0009) [2023-12-26 19:45:11,170][105692] Updated weights for policy 0, policy_version 602359 (0.0008) [2023-12-26 19:45:11,232][105692] Updated weights for policy 0, policy_version 602369 (0.0010) [2023-12-26 19:45:11,692][105620] Updated weights for policy 1, policy_version 603275 (0.0009) [2023-12-26 19:45:11,756][105620] Updated weights for policy 1, policy_version 603285 (0.0009) [2023-12-26 19:45:11,827][105620] Updated weights for policy 1, policy_version 603295 (0.0010) [2023-12-26 19:45:11,894][105692] Updated weights for policy 0, policy_version 602379 (0.0008) [2023-12-26 19:45:11,946][105692] Updated weights for policy 0, policy_version 602389 (0.0008) [2023-12-26 19:45:12,010][105692] Updated weights for policy 0, policy_version 602399 (0.0006) [2023-12-26 19:45:12,633][105620] Updated weights for policy 1, policy_version 603305 (0.0009) [2023-12-26 19:45:12,692][105620] Updated weights for policy 1, policy_version 603315 (0.0009) [2023-12-26 19:45:12,753][105620] Updated weights for policy 1, policy_version 603325 (0.0009) [2023-12-26 19:45:12,763][105692] Updated weights for policy 0, policy_version 602409 (0.0006) [2023-12-26 19:45:12,811][105620] Updated weights for policy 1, policy_version 603335 (0.0006) [2023-12-26 19:45:12,822][105692] Updated weights for policy 0, policy_version 602419 (0.0007) [2023-12-26 19:45:12,877][105692] Updated weights for policy 0, policy_version 602429 (0.0005) [2023-12-26 19:45:12,938][105692] Updated weights for policy 0, policy_version 602439 (0.0005) [2023-12-26 19:45:13,451][105692] Updated weights for policy 0, policy_version 602449 (0.0005) [2023-12-26 19:45:13,500][105620] Updated weights for policy 1, policy_version 603345 (0.0007) [2023-12-26 19:45:13,505][105692] Updated weights for policy 0, policy_version 602459 (0.0005) [2023-12-26 19:45:13,567][105620] Updated weights for policy 1, policy_version 603355 (0.0008) [2023-12-26 19:45:13,570][105692] Updated weights for policy 0, policy_version 602469 (0.0005) [2023-12-26 19:45:13,628][105620] Updated weights for policy 1, policy_version 603365 (0.0008) [2023-12-26 19:45:14,162][105620] Updated weights for policy 1, policy_version 603375 (0.0009) [2023-12-26 19:45:14,166][105692] Updated weights for policy 0, policy_version 602479 (0.0007) [2023-12-26 19:45:14,217][105620] Updated weights for policy 1, policy_version 603385 (0.0007) [2023-12-26 19:45:14,219][105692] Updated weights for policy 0, policy_version 602489 (0.0008) [2023-12-26 19:45:14,271][105692] Updated weights for policy 0, policy_version 602499 (0.0007) [2023-12-26 19:45:14,273][105620] Updated weights for policy 1, policy_version 603395 (0.0006) [2023-12-26 19:45:14,962][105692] Updated weights for policy 0, policy_version 602509 (0.0008) [2023-12-26 19:45:15,013][105620] Updated weights for policy 1, policy_version 603405 (0.0007) [2023-12-26 19:45:15,031][105692] Updated weights for policy 0, policy_version 602519 (0.0008) [2023-12-26 19:45:15,079][105620] Updated weights for policy 1, policy_version 603415 (0.0006) [2023-12-26 19:45:15,090][105692] Updated weights for policy 0, policy_version 602529 (0.0009) [2023-12-26 19:45:15,146][105620] Updated weights for policy 1, policy_version 603425 (0.0006) [2023-12-26 19:45:15,733][105692] Updated weights for policy 0, policy_version 602539 (0.0008) [2023-12-26 19:45:15,796][105692] Updated weights for policy 0, policy_version 602549 (0.0005) [2023-12-26 19:45:15,849][105620] Updated weights for policy 1, policy_version 603435 (0.0009) [2023-12-26 19:45:15,860][105692] Updated weights for policy 0, policy_version 602559 (0.0006) [2023-12-26 19:45:15,908][105620] Updated weights for policy 1, policy_version 603445 (0.0009) [2023-12-26 19:45:15,974][105620] Updated weights for policy 1, policy_version 603455 (0.0008) [2023-12-26 19:45:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 308781056. Throughput: 0: 9654.5, 1: 10017.5. Samples: 308743640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:45:16,063][104569] Avg episode reward: [(0, '9175.299'), (1, '9253.348')] [2023-12-26 19:45:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000602568_154279936.pth... [2023-12-26 19:45:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000603464_154501120.pth... [2023-12-26 19:45:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000601424_153985024.pth [2023-12-26 19:45:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000602280_154198016.pth [2023-12-26 19:45:16,533][105620] Updated weights for policy 1, policy_version 603465 (0.0008) [2023-12-26 19:45:16,552][105692] Updated weights for policy 0, policy_version 602569 (0.0007) [2023-12-26 19:45:16,586][105620] Updated weights for policy 1, policy_version 603475 (0.0005) [2023-12-26 19:45:16,611][105692] Updated weights for policy 0, policy_version 602579 (0.0007) [2023-12-26 19:45:16,640][105620] Updated weights for policy 1, policy_version 603485 (0.0005) [2023-12-26 19:45:16,667][105692] Updated weights for policy 0, policy_version 602589 (0.0007) [2023-12-26 19:45:16,695][105620] Updated weights for policy 1, policy_version 603495 (0.0005) [2023-12-26 19:45:16,718][105692] Updated weights for policy 0, policy_version 602599 (0.0009) [2023-12-26 19:45:17,294][105620] Updated weights for policy 1, policy_version 603505 (0.0006) [2023-12-26 19:45:17,301][105692] Updated weights for policy 0, policy_version 602609 (0.0005) [2023-12-26 19:45:17,345][105692] Updated weights for policy 0, policy_version 602619 (0.0005) [2023-12-26 19:45:17,352][105620] Updated weights for policy 1, policy_version 603515 (0.0006) [2023-12-26 19:45:17,395][105692] Updated weights for policy 0, policy_version 602629 (0.0007) [2023-12-26 19:45:17,403][105620] Updated weights for policy 1, policy_version 603525 (0.0006) [2023-12-26 19:45:18,016][105692] Updated weights for policy 0, policy_version 602639 (0.0006) [2023-12-26 19:45:18,020][105620] Updated weights for policy 1, policy_version 603535 (0.0008) [2023-12-26 19:45:18,065][105692] Updated weights for policy 0, policy_version 602649 (0.0005) [2023-12-26 19:45:18,079][105620] Updated weights for policy 1, policy_version 603545 (0.0008) [2023-12-26 19:45:18,123][105692] Updated weights for policy 0, policy_version 602659 (0.0006) [2023-12-26 19:45:18,133][105620] Updated weights for policy 1, policy_version 603555 (0.0008) [2023-12-26 19:45:18,772][105692] Updated weights for policy 0, policy_version 602669 (0.0008) [2023-12-26 19:45:18,828][105692] Updated weights for policy 0, policy_version 602679 (0.0010) [2023-12-26 19:45:18,880][105620] Updated weights for policy 1, policy_version 603565 (0.0007) [2023-12-26 19:45:18,882][105692] Updated weights for policy 0, policy_version 602689 (0.0009) [2023-12-26 19:45:18,929][105620] Updated weights for policy 1, policy_version 603575 (0.0006) [2023-12-26 19:45:18,983][105620] Updated weights for policy 1, policy_version 603585 (0.0005) [2023-12-26 19:45:19,652][105692] Updated weights for policy 0, policy_version 602699 (0.0006) [2023-12-26 19:45:19,655][105620] Updated weights for policy 1, policy_version 603595 (0.0006) [2023-12-26 19:45:19,713][105692] Updated weights for policy 0, policy_version 602709 (0.0007) [2023-12-26 19:45:19,724][105620] Updated weights for policy 1, policy_version 603605 (0.0007) [2023-12-26 19:45:19,779][105692] Updated weights for policy 0, policy_version 602719 (0.0008) [2023-12-26 19:45:19,798][105620] Updated weights for policy 1, policy_version 603615 (0.0007) [2023-12-26 19:45:20,450][105620] Updated weights for policy 1, policy_version 603625 (0.0007) [2023-12-26 19:45:20,510][105620] Updated weights for policy 1, policy_version 603635 (0.0006) [2023-12-26 19:45:20,557][105692] Updated weights for policy 0, policy_version 602729 (0.0008) [2023-12-26 19:45:20,577][105620] Updated weights for policy 1, policy_version 603645 (0.0007) [2023-12-26 19:45:20,621][105692] Updated weights for policy 0, policy_version 602739 (0.0008) [2023-12-26 19:45:20,645][105620] Updated weights for policy 1, policy_version 603655 (0.0008) [2023-12-26 19:45:20,683][105692] Updated weights for policy 0, policy_version 602749 (0.0008) [2023-12-26 19:45:20,745][105692] Updated weights for policy 0, policy_version 602759 (0.0009) [2023-12-26 19:45:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19521.9). Total num frames: 308879360. Throughput: 0: 9709.8, 1: 10181.8. Samples: 308868588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:45:21,063][104569] Avg episode reward: [(0, '9266.660'), (1, '9164.065')] [2023-12-26 19:45:21,379][105620] Updated weights for policy 1, policy_version 603665 (0.0009) [2023-12-26 19:45:21,444][105620] Updated weights for policy 1, policy_version 603675 (0.0008) [2023-12-26 19:45:21,507][105620] Updated weights for policy 1, policy_version 603685 (0.0006) [2023-12-26 19:45:21,539][105692] Updated weights for policy 0, policy_version 602769 (0.0008) [2023-12-26 19:45:21,606][105692] Updated weights for policy 0, policy_version 602779 (0.0008) [2023-12-26 19:45:21,634][105585] KL-divergence is very high: 119.0127 [2023-12-26 19:45:21,671][105692] Updated weights for policy 0, policy_version 602789 (0.0009) [2023-12-26 19:45:21,684][105585] KL-divergence is very high: 103.7561 [2023-12-26 19:45:22,283][105620] Updated weights for policy 1, policy_version 603695 (0.0009) [2023-12-26 19:45:22,336][105692] Updated weights for policy 0, policy_version 602799 (0.0008) [2023-12-26 19:45:22,343][105620] Updated weights for policy 1, policy_version 603705 (0.0007) [2023-12-26 19:45:22,398][105692] Updated weights for policy 0, policy_version 602809 (0.0008) [2023-12-26 19:45:22,405][105620] Updated weights for policy 1, policy_version 603715 (0.0007) [2023-12-26 19:45:22,450][105692] Updated weights for policy 0, policy_version 602819 (0.0008) [2023-12-26 19:45:23,146][105692] Updated weights for policy 0, policy_version 602829 (0.0009) [2023-12-26 19:45:23,197][105620] Updated weights for policy 1, policy_version 603725 (0.0008) [2023-12-26 19:45:23,199][105692] Updated weights for policy 0, policy_version 602839 (0.0009) [2023-12-26 19:45:23,247][105692] Updated weights for policy 0, policy_version 602849 (0.0008) [2023-12-26 19:45:23,250][105620] Updated weights for policy 1, policy_version 603735 (0.0007) [2023-12-26 19:45:23,300][105620] Updated weights for policy 1, policy_version 603745 (0.0006) [2023-12-26 19:45:24,011][105692] Updated weights for policy 0, policy_version 602859 (0.0007) [2023-12-26 19:45:24,052][105620] Updated weights for policy 1, policy_version 603755 (0.0008) [2023-12-26 19:45:24,066][105692] Updated weights for policy 0, policy_version 602869 (0.0009) [2023-12-26 19:45:24,106][105620] Updated weights for policy 1, policy_version 603765 (0.0006) [2023-12-26 19:45:24,131][105692] Updated weights for policy 0, policy_version 602879 (0.0007) [2023-12-26 19:45:24,167][105620] Updated weights for policy 1, policy_version 603775 (0.0007) [2023-12-26 19:45:24,875][105620] Updated weights for policy 1, policy_version 603785 (0.0008) [2023-12-26 19:45:24,905][105692] Updated weights for policy 0, policy_version 602889 (0.0009) [2023-12-26 19:45:24,923][105620] Updated weights for policy 1, policy_version 603795 (0.0008) [2023-12-26 19:45:24,966][105692] Updated weights for policy 0, policy_version 602899 (0.0007) [2023-12-26 19:45:24,972][105620] Updated weights for policy 1, policy_version 603805 (0.0006) [2023-12-26 19:45:25,017][105692] Updated weights for policy 0, policy_version 602909 (0.0006) [2023-12-26 19:45:25,033][105620] Updated weights for policy 1, policy_version 603815 (0.0008) [2023-12-26 19:45:25,069][105692] Updated weights for policy 0, policy_version 602919 (0.0006) [2023-12-26 19:45:25,740][105692] Updated weights for policy 0, policy_version 602929 (0.0009) [2023-12-26 19:45:25,797][105692] Updated weights for policy 0, policy_version 602939 (0.0009) [2023-12-26 19:45:25,824][105620] Updated weights for policy 1, policy_version 603825 (0.0007) [2023-12-26 19:45:25,855][105692] Updated weights for policy 0, policy_version 602949 (0.0008) [2023-12-26 19:45:25,873][105620] Updated weights for policy 1, policy_version 603835 (0.0006) [2023-12-26 19:45:25,925][105620] Updated weights for policy 1, policy_version 603845 (0.0009) [2023-12-26 19:45:26,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19933.9, 300 sec: 19522.0). Total num frames: 308977664. Throughput: 0: 9747.1, 1: 10047.1. Samples: 308981140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:45:26,062][104569] Avg episode reward: [(0, '9266.427'), (1, '8893.822')] [2023-12-26 19:45:26,612][105692] Updated weights for policy 0, policy_version 602959 (0.0009) [2023-12-26 19:45:26,661][105692] Updated weights for policy 0, policy_version 602969 (0.0008) [2023-12-26 19:45:26,683][105620] Updated weights for policy 1, policy_version 603855 (0.0007) [2023-12-26 19:45:26,720][105692] Updated weights for policy 0, policy_version 602979 (0.0007) [2023-12-26 19:45:26,734][105620] Updated weights for policy 1, policy_version 603865 (0.0006) [2023-12-26 19:45:26,782][105620] Updated weights for policy 1, policy_version 603875 (0.0008) [2023-12-26 19:45:27,444][105620] Updated weights for policy 1, policy_version 603885 (0.0009) [2023-12-26 19:45:27,500][105620] Updated weights for policy 1, policy_version 603895 (0.0008) [2023-12-26 19:45:27,516][105692] Updated weights for policy 0, policy_version 602989 (0.0009) [2023-12-26 19:45:27,561][105620] Updated weights for policy 1, policy_version 603905 (0.0007) [2023-12-26 19:45:27,575][105692] Updated weights for policy 0, policy_version 602999 (0.0006) [2023-12-26 19:45:27,635][105692] Updated weights for policy 0, policy_version 603009 (0.0006) [2023-12-26 19:45:28,219][105692] Updated weights for policy 0, policy_version 603019 (0.0006) [2023-12-26 19:45:28,278][105692] Updated weights for policy 0, policy_version 603029 (0.0008) [2023-12-26 19:45:28,331][105692] Updated weights for policy 0, policy_version 603039 (0.0006) [2023-12-26 19:45:28,358][105620] Updated weights for policy 1, policy_version 603915 (0.0007) [2023-12-26 19:45:28,419][105620] Updated weights for policy 1, policy_version 603925 (0.0008) [2023-12-26 19:45:28,483][105620] Updated weights for policy 1, policy_version 603935 (0.0009) [2023-12-26 19:45:29,049][105692] Updated weights for policy 0, policy_version 603049 (0.0008) [2023-12-26 19:45:29,105][105692] Updated weights for policy 0, policy_version 603059 (0.0005) [2023-12-26 19:45:29,161][105692] Updated weights for policy 0, policy_version 603069 (0.0007) [2023-12-26 19:45:29,208][105620] Updated weights for policy 1, policy_version 603945 (0.0009) [2023-12-26 19:45:29,212][105692] Updated weights for policy 0, policy_version 603079 (0.0008) [2023-12-26 19:45:29,274][105620] Updated weights for policy 1, policy_version 603955 (0.0010) [2023-12-26 19:45:29,338][105620] Updated weights for policy 1, policy_version 603965 (0.0010) [2023-12-26 19:45:29,402][105620] Updated weights for policy 1, policy_version 603975 (0.0008) [2023-12-26 19:45:29,917][105692] Updated weights for policy 0, policy_version 603089 (0.0009) [2023-12-26 19:45:29,969][105692] Updated weights for policy 0, policy_version 603099 (0.0008) [2023-12-26 19:45:30,021][105692] Updated weights for policy 0, policy_version 603109 (0.0008) [2023-12-26 19:45:30,029][105620] Updated weights for policy 1, policy_version 603985 (0.0007) [2023-12-26 19:45:30,090][105620] Updated weights for policy 1, policy_version 603995 (0.0009) [2023-12-26 19:45:30,152][105620] Updated weights for policy 1, policy_version 604005 (0.0008) [2023-12-26 19:45:30,807][105692] Updated weights for policy 0, policy_version 603119 (0.0007) [2023-12-26 19:45:30,809][105620] Updated weights for policy 1, policy_version 604015 (0.0009) [2023-12-26 19:45:30,864][105692] Updated weights for policy 0, policy_version 603129 (0.0007) [2023-12-26 19:45:30,866][105620] Updated weights for policy 1, policy_version 604025 (0.0006) [2023-12-26 19:45:30,915][105692] Updated weights for policy 0, policy_version 603139 (0.0008) [2023-12-26 19:45:30,920][105620] Updated weights for policy 1, policy_version 604035 (0.0005) [2023-12-26 19:45:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 309075968. Throughput: 0: 9737.2, 1: 10006.0. Samples: 309039056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:45:31,062][104569] Avg episode reward: [(0, '9266.368'), (1, '8987.579')] [2023-12-26 19:45:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000603144_154427392.pth... [2023-12-26 19:45:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000604040_154648576.pth... [2023-12-26 19:45:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000601992_154132480.pth [2023-12-26 19:45:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000602856_154345472.pth [2023-12-26 19:45:31,605][105620] Updated weights for policy 1, policy_version 604045 (0.0007) [2023-12-26 19:45:31,670][105620] Updated weights for policy 1, policy_version 604055 (0.0009) [2023-12-26 19:45:31,738][105620] Updated weights for policy 1, policy_version 604065 (0.0008) [2023-12-26 19:45:31,774][105692] Updated weights for policy 0, policy_version 603149 (0.0010) [2023-12-26 19:45:31,834][105692] Updated weights for policy 0, policy_version 603159 (0.0009) [2023-12-26 19:45:31,900][105692] Updated weights for policy 0, policy_version 603169 (0.0010) [2023-12-26 19:45:32,343][105620] Updated weights for policy 1, policy_version 604075 (0.0008) [2023-12-26 19:45:32,401][105620] Updated weights for policy 1, policy_version 604085 (0.0009) [2023-12-26 19:45:32,448][105620] Updated weights for policy 1, policy_version 604095 (0.0008) [2023-12-26 19:45:32,730][105692] Updated weights for policy 0, policy_version 603179 (0.0009) [2023-12-26 19:45:32,784][105692] Updated weights for policy 0, policy_version 603189 (0.0010) [2023-12-26 19:45:32,832][105692] Updated weights for policy 0, policy_version 603199 (0.0010) [2023-12-26 19:45:33,114][105620] Updated weights for policy 1, policy_version 604105 (0.0008) [2023-12-26 19:45:33,163][105620] Updated weights for policy 1, policy_version 604115 (0.0009) [2023-12-26 19:45:33,209][105620] Updated weights for policy 1, policy_version 604125 (0.0009) [2023-12-26 19:45:33,255][105620] Updated weights for policy 1, policy_version 604135 (0.0009) [2023-12-26 19:45:33,537][105692] Updated weights for policy 0, policy_version 603209 (0.0010) [2023-12-26 19:45:33,592][105692] Updated weights for policy 0, policy_version 603219 (0.0010) [2023-12-26 19:45:33,640][105692] Updated weights for policy 0, policy_version 603229 (0.0009) [2023-12-26 19:45:33,685][105692] Updated weights for policy 0, policy_version 603239 (0.0008) [2023-12-26 19:45:34,074][105620] Updated weights for policy 1, policy_version 604145 (0.0008) [2023-12-26 19:45:34,132][105620] Updated weights for policy 1, policy_version 604155 (0.0010) [2023-12-26 19:45:34,194][105620] Updated weights for policy 1, policy_version 604165 (0.0009) [2023-12-26 19:45:34,434][105692] Updated weights for policy 0, policy_version 603249 (0.0009) [2023-12-26 19:45:34,501][105692] Updated weights for policy 0, policy_version 603259 (0.0006) [2023-12-26 19:45:34,566][105692] Updated weights for policy 0, policy_version 603269 (0.0006) [2023-12-26 19:45:35,012][105620] Updated weights for policy 1, policy_version 604175 (0.0010) [2023-12-26 19:45:35,065][105620] Updated weights for policy 1, policy_version 604185 (0.0008) [2023-12-26 19:45:35,119][105620] Updated weights for policy 1, policy_version 604195 (0.0009) [2023-12-26 19:45:35,178][105692] Updated weights for policy 0, policy_version 603279 (0.0008) [2023-12-26 19:45:35,236][105692] Updated weights for policy 0, policy_version 603289 (0.0007) [2023-12-26 19:45:35,296][105692] Updated weights for policy 0, policy_version 603299 (0.0006) [2023-12-26 19:45:35,893][105620] Updated weights for policy 1, policy_version 604205 (0.0007) [2023-12-26 19:45:35,948][105620] Updated weights for policy 1, policy_version 604215 (0.0006) [2023-12-26 19:45:35,955][105692] Updated weights for policy 0, policy_version 603309 (0.0008) [2023-12-26 19:45:36,007][105620] Updated weights for policy 1, policy_version 604225 (0.0006) [2023-12-26 19:45:36,007][105692] Updated weights for policy 0, policy_version 603319 (0.0009) [2023-12-26 19:45:36,054][105692] Updated weights for policy 0, policy_version 603329 (0.0005) [2023-12-26 19:45:36,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 309166080. Throughput: 0: 9686.7, 1: 9963.2. Samples: 309154776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:45:36,063][104569] Avg episode reward: [(0, '9084.669'), (1, '8989.254')] [2023-12-26 19:45:36,739][105620] Updated weights for policy 1, policy_version 604235 (0.0007) [2023-12-26 19:45:36,751][105692] Updated weights for policy 0, policy_version 603339 (0.0007) [2023-12-26 19:45:36,795][105620] Updated weights for policy 1, policy_version 604245 (0.0008) [2023-12-26 19:45:36,816][105692] Updated weights for policy 0, policy_version 603349 (0.0005) [2023-12-26 19:45:36,852][105620] Updated weights for policy 1, policy_version 604255 (0.0009) [2023-12-26 19:45:36,870][105692] Updated weights for policy 0, policy_version 603359 (0.0005) [2023-12-26 19:45:37,505][105620] Updated weights for policy 1, policy_version 604265 (0.0010) [2023-12-26 19:45:37,559][105620] Updated weights for policy 1, policy_version 604275 (0.0007) [2023-12-26 19:45:37,618][105620] Updated weights for policy 1, policy_version 604285 (0.0006) [2023-12-26 19:45:37,623][105692] Updated weights for policy 0, policy_version 603369 (0.0007) [2023-12-26 19:45:37,684][105620] Updated weights for policy 1, policy_version 604295 (0.0006) [2023-12-26 19:45:37,692][105692] Updated weights for policy 0, policy_version 603379 (0.0011) [2023-12-26 19:45:37,755][105692] Updated weights for policy 0, policy_version 603389 (0.0010) [2023-12-26 19:45:37,814][105692] Updated weights for policy 0, policy_version 603399 (0.0010) [2023-12-26 19:45:38,320][105620] Updated weights for policy 1, policy_version 604305 (0.0010) [2023-12-26 19:45:38,391][105620] Updated weights for policy 1, policy_version 604315 (0.0010) [2023-12-26 19:45:38,426][105692] Updated weights for policy 0, policy_version 603409 (0.0011) [2023-12-26 19:45:38,450][105620] Updated weights for policy 1, policy_version 604325 (0.0011) [2023-12-26 19:45:38,489][105692] Updated weights for policy 0, policy_version 603419 (0.0011) [2023-12-26 19:45:38,547][105692] Updated weights for policy 0, policy_version 603429 (0.0010) [2023-12-26 19:45:39,181][105620] Updated weights for policy 1, policy_version 604335 (0.0007) [2023-12-26 19:45:39,247][105620] Updated weights for policy 1, policy_version 604345 (0.0007) [2023-12-26 19:45:39,275][105692] Updated weights for policy 0, policy_version 603439 (0.0011) [2023-12-26 19:45:39,304][105620] Updated weights for policy 1, policy_version 604355 (0.0007) [2023-12-26 19:45:39,340][105692] Updated weights for policy 0, policy_version 603449 (0.0013) [2023-12-26 19:45:39,410][105692] Updated weights for policy 0, policy_version 603459 (0.0009) [2023-12-26 19:45:39,971][105620] Updated weights for policy 1, policy_version 604365 (0.0009) [2023-12-26 19:45:40,031][105620] Updated weights for policy 1, policy_version 604375 (0.0009) [2023-12-26 19:45:40,090][105620] Updated weights for policy 1, policy_version 604385 (0.0008) [2023-12-26 19:45:40,211][105692] Updated weights for policy 0, policy_version 603469 (0.0009) [2023-12-26 19:45:40,273][105692] Updated weights for policy 0, policy_version 603479 (0.0009) [2023-12-26 19:45:40,347][105692] Updated weights for policy 0, policy_version 603489 (0.0006) [2023-12-26 19:45:40,713][105620] Updated weights for policy 1, policy_version 604395 (0.0007) [2023-12-26 19:45:40,783][105620] Updated weights for policy 1, policy_version 604405 (0.0006) [2023-12-26 19:45:40,848][105620] Updated weights for policy 1, policy_version 604415 (0.0008) [2023-12-26 19:45:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19797.4, 300 sec: 19522.0). Total num frames: 309264384. Throughput: 0: 9713.2, 1: 9931.2. Samples: 309273812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:45:41,062][104569] Avg episode reward: [(0, '9084.544'), (1, '8991.030')] [2023-12-26 19:45:41,117][105692] Updated weights for policy 0, policy_version 603499 (0.0010) [2023-12-26 19:45:41,188][105692] Updated weights for policy 0, policy_version 603509 (0.0008) [2023-12-26 19:45:41,254][105692] Updated weights for policy 0, policy_version 603519 (0.0008) [2023-12-26 19:45:41,557][105620] Updated weights for policy 1, policy_version 604425 (0.0009) [2023-12-26 19:45:41,618][105620] Updated weights for policy 1, policy_version 604435 (0.0007) [2023-12-26 19:45:41,677][105620] Updated weights for policy 1, policy_version 604445 (0.0007) [2023-12-26 19:45:41,742][105620] Updated weights for policy 1, policy_version 604455 (0.0008) [2023-12-26 19:45:42,020][105692] Updated weights for policy 0, policy_version 603529 (0.0009) [2023-12-26 19:45:42,088][105692] Updated weights for policy 0, policy_version 603539 (0.0008) [2023-12-26 19:45:42,148][105692] Updated weights for policy 0, policy_version 603549 (0.0006) [2023-12-26 19:45:42,216][105692] Updated weights for policy 0, policy_version 603559 (0.0008) [2023-12-26 19:45:42,503][105620] Updated weights for policy 1, policy_version 604465 (0.0005) [2023-12-26 19:45:42,554][105620] Updated weights for policy 1, policy_version 604475 (0.0006) [2023-12-26 19:45:42,618][105620] Updated weights for policy 1, policy_version 604485 (0.0009) [2023-12-26 19:45:42,987][105692] Updated weights for policy 0, policy_version 603569 (0.0011) [2023-12-26 19:45:43,045][105692] Updated weights for policy 0, policy_version 603579 (0.0010) [2023-12-26 19:45:43,107][105692] Updated weights for policy 0, policy_version 603589 (0.0010) [2023-12-26 19:45:43,352][105620] Updated weights for policy 1, policy_version 604495 (0.0008) [2023-12-26 19:45:43,405][105620] Updated weights for policy 1, policy_version 604505 (0.0008) [2023-12-26 19:45:43,474][105620] Updated weights for policy 1, policy_version 604515 (0.0009) [2023-12-26 19:45:43,807][105692] Updated weights for policy 0, policy_version 603599 (0.0010) [2023-12-26 19:45:43,859][105692] Updated weights for policy 0, policy_version 603609 (0.0006) [2023-12-26 19:45:43,913][105692] Updated weights for policy 0, policy_version 603619 (0.0009) [2023-12-26 19:45:44,266][105620] Updated weights for policy 1, policy_version 604525 (0.0009) [2023-12-26 19:45:44,312][105620] Updated weights for policy 1, policy_version 604535 (0.0007) [2023-12-26 19:45:44,364][105620] Updated weights for policy 1, policy_version 604545 (0.0006) [2023-12-26 19:45:44,641][105692] Updated weights for policy 0, policy_version 603629 (0.0009) [2023-12-26 19:45:44,698][105692] Updated weights for policy 0, policy_version 603639 (0.0008) [2023-12-26 19:45:44,755][105692] Updated weights for policy 0, policy_version 603649 (0.0006) [2023-12-26 19:45:45,087][105620] Updated weights for policy 1, policy_version 604555 (0.0006) [2023-12-26 19:45:45,136][105620] Updated weights for policy 1, policy_version 604565 (0.0009) [2023-12-26 19:45:45,187][105620] Updated weights for policy 1, policy_version 604575 (0.0008) [2023-12-26 19:45:45,427][105692] Updated weights for policy 0, policy_version 603659 (0.0009) [2023-12-26 19:45:45,491][105692] Updated weights for policy 0, policy_version 603669 (0.0011) [2023-12-26 19:45:45,547][105692] Updated weights for policy 0, policy_version 603679 (0.0011) [2023-12-26 19:45:45,914][105620] Updated weights for policy 1, policy_version 604585 (0.0008) [2023-12-26 19:45:45,969][105620] Updated weights for policy 1, policy_version 604595 (0.0006) [2023-12-26 19:45:46,027][105620] Updated weights for policy 1, policy_version 604605 (0.0005) [2023-12-26 19:45:46,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 309354496. Throughput: 0: 9630.8, 1: 9894.3. Samples: 309328756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:45:46,062][104569] Avg episode reward: [(0, '9265.616'), (1, '9172.423')] [2023-12-26 19:45:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000603688_154566656.pth... [2023-12-26 19:45:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000602568_154279936.pth [2023-12-26 19:45:46,086][105620] Updated weights for policy 1, policy_version 604615 (0.0006) [2023-12-26 19:45:46,091][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000604616_154796032.pth... [2023-12-26 19:45:46,096][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000603464_154501120.pth [2023-12-26 19:45:46,356][105692] Updated weights for policy 0, policy_version 603689 (0.0010) [2023-12-26 19:45:46,419][105692] Updated weights for policy 0, policy_version 603699 (0.0010) [2023-12-26 19:45:46,484][105692] Updated weights for policy 0, policy_version 603709 (0.0009) [2023-12-26 19:45:46,536][105692] Updated weights for policy 0, policy_version 603719 (0.0009) [2023-12-26 19:45:46,757][105620] Updated weights for policy 1, policy_version 604625 (0.0009) [2023-12-26 19:45:46,813][105620] Updated weights for policy 1, policy_version 604635 (0.0009) [2023-12-26 19:45:46,870][105620] Updated weights for policy 1, policy_version 604645 (0.0009) [2023-12-26 19:45:47,316][105692] Updated weights for policy 0, policy_version 603729 (0.0009) [2023-12-26 19:45:47,364][105692] Updated weights for policy 0, policy_version 603739 (0.0009) [2023-12-26 19:45:47,411][105692] Updated weights for policy 0, policy_version 603749 (0.0009) [2023-12-26 19:45:47,629][105620] Updated weights for policy 1, policy_version 604655 (0.0010) [2023-12-26 19:45:47,687][105620] Updated weights for policy 1, policy_version 604665 (0.0009) [2023-12-26 19:45:47,739][105620] Updated weights for policy 1, policy_version 604675 (0.0009) [2023-12-26 19:45:48,208][105692] Updated weights for policy 0, policy_version 603759 (0.0010) [2023-12-26 19:45:48,263][105692] Updated weights for policy 0, policy_version 603769 (0.0012) [2023-12-26 19:45:48,319][105692] Updated weights for policy 0, policy_version 603779 (0.0008) [2023-12-26 19:45:48,411][105620] Updated weights for policy 1, policy_version 604685 (0.0009) [2023-12-26 19:45:48,472][105620] Updated weights for policy 1, policy_version 604695 (0.0008) [2023-12-26 19:45:48,532][105620] Updated weights for policy 1, policy_version 604705 (0.0009) [2023-12-26 19:45:49,115][105692] Updated weights for policy 0, policy_version 603789 (0.0009) [2023-12-26 19:45:49,169][105692] Updated weights for policy 0, policy_version 603799 (0.0008) [2023-12-26 19:45:49,241][105692] Updated weights for policy 0, policy_version 603809 (0.0008) [2023-12-26 19:45:49,318][105620] Updated weights for policy 1, policy_version 604715 (0.0008) [2023-12-26 19:45:49,383][105620] Updated weights for policy 1, policy_version 604725 (0.0008) [2023-12-26 19:45:49,447][105620] Updated weights for policy 1, policy_version 604735 (0.0007) [2023-12-26 19:45:50,034][105692] Updated weights for policy 0, policy_version 603819 (0.0007) [2023-12-26 19:45:50,093][105692] Updated weights for policy 0, policy_version 603829 (0.0009) [2023-12-26 19:45:50,130][105620] Updated weights for policy 1, policy_version 604745 (0.0007) [2023-12-26 19:45:50,158][105692] Updated weights for policy 0, policy_version 603839 (0.0008) [2023-12-26 19:45:50,194][105620] Updated weights for policy 1, policy_version 604755 (0.0007) [2023-12-26 19:45:50,249][105620] Updated weights for policy 1, policy_version 604765 (0.0005) [2023-12-26 19:45:50,302][105620] Updated weights for policy 1, policy_version 604775 (0.0006) [2023-12-26 19:45:50,977][105692] Updated weights for policy 0, policy_version 603849 (0.0008) [2023-12-26 19:45:51,017][105620] Updated weights for policy 1, policy_version 604785 (0.0008) [2023-12-26 19:45:51,046][105692] Updated weights for policy 0, policy_version 603859 (0.0008) [2023-12-26 19:45:51,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 309444608. Throughput: 0: 9656.8, 1: 9818.1. Samples: 309442048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:45:51,062][104569] Avg episode reward: [(0, '9174.810'), (1, '9262.926')] [2023-12-26 19:45:51,086][105620] Updated weights for policy 1, policy_version 604795 (0.0008) [2023-12-26 19:45:51,112][105692] Updated weights for policy 0, policy_version 603869 (0.0009) [2023-12-26 19:45:51,159][105620] Updated weights for policy 1, policy_version 604805 (0.0007) [2023-12-26 19:45:51,182][105692] Updated weights for policy 0, policy_version 603879 (0.0008) [2023-12-26 19:45:51,944][105620] Updated weights for policy 1, policy_version 604815 (0.0009) [2023-12-26 19:45:51,982][105692] Updated weights for policy 0, policy_version 603889 (0.0008) [2023-12-26 19:45:52,004][105620] Updated weights for policy 1, policy_version 604825 (0.0010) [2023-12-26 19:45:52,037][105692] Updated weights for policy 0, policy_version 603899 (0.0008) [2023-12-26 19:45:52,070][105620] Updated weights for policy 1, policy_version 604835 (0.0011) [2023-12-26 19:45:52,095][105692] Updated weights for policy 0, policy_version 603909 (0.0006) [2023-12-26 19:45:52,747][105620] Updated weights for policy 1, policy_version 604845 (0.0007) [2023-12-26 19:45:52,797][105620] Updated weights for policy 1, policy_version 604855 (0.0007) [2023-12-26 19:45:52,856][105620] Updated weights for policy 1, policy_version 604865 (0.0010) [2023-12-26 19:45:52,868][105692] Updated weights for policy 0, policy_version 603919 (0.0009) [2023-12-26 19:45:52,926][105692] Updated weights for policy 0, policy_version 603929 (0.0010) [2023-12-26 19:45:52,981][105692] Updated weights for policy 0, policy_version 603939 (0.0010) [2023-12-26 19:45:53,491][105620] Updated weights for policy 1, policy_version 604875 (0.0009) [2023-12-26 19:45:53,542][105620] Updated weights for policy 1, policy_version 604885 (0.0006) [2023-12-26 19:45:53,604][105620] Updated weights for policy 1, policy_version 604895 (0.0010) [2023-12-26 19:45:53,734][105692] Updated weights for policy 0, policy_version 603949 (0.0010) [2023-12-26 19:45:53,792][105692] Updated weights for policy 0, policy_version 603959 (0.0010) [2023-12-26 19:45:53,853][105692] Updated weights for policy 0, policy_version 603969 (0.0007) [2023-12-26 19:45:54,186][105620] Updated weights for policy 1, policy_version 604905 (0.0010) [2023-12-26 19:45:54,257][105620] Updated weights for policy 1, policy_version 604915 (0.0006) [2023-12-26 19:45:54,326][105620] Updated weights for policy 1, policy_version 604925 (0.0006) [2023-12-26 19:45:54,389][105620] Updated weights for policy 1, policy_version 604935 (0.0005) [2023-12-26 19:45:54,426][105692] Updated weights for policy 0, policy_version 603979 (0.0005) [2023-12-26 19:45:54,479][105692] Updated weights for policy 0, policy_version 603989 (0.0006) [2023-12-26 19:45:54,542][105692] Updated weights for policy 0, policy_version 603999 (0.0005) [2023-12-26 19:45:54,913][105620] Updated weights for policy 1, policy_version 604945 (0.0007) [2023-12-26 19:45:54,981][105620] Updated weights for policy 1, policy_version 604955 (0.0006) [2023-12-26 19:45:55,040][105620] Updated weights for policy 1, policy_version 604965 (0.0006) [2023-12-26 19:45:55,127][105692] Updated weights for policy 0, policy_version 604009 (0.0005) [2023-12-26 19:45:55,191][105692] Updated weights for policy 0, policy_version 604019 (0.0007) [2023-12-26 19:45:55,251][105692] Updated weights for policy 0, policy_version 604029 (0.0006) [2023-12-26 19:45:55,313][105692] Updated weights for policy 0, policy_version 604039 (0.0007) [2023-12-26 19:45:55,601][105620] Updated weights for policy 1, policy_version 604975 (0.0006) [2023-12-26 19:45:55,654][105620] Updated weights for policy 1, policy_version 604985 (0.0008) [2023-12-26 19:45:55,710][105620] Updated weights for policy 1, policy_version 604995 (0.0008) [2023-12-26 19:45:56,010][105692] Updated weights for policy 0, policy_version 604049 (0.0006) [2023-12-26 19:45:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 309551104. Throughput: 0: 9654.5, 1: 9890.6. Samples: 309562964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:45:56,063][104569] Avg episode reward: [(0, '9174.747'), (1, '9355.285')] [2023-12-26 19:45:56,069][105692] Updated weights for policy 0, policy_version 604059 (0.0006) [2023-12-26 19:45:56,137][105692] Updated weights for policy 0, policy_version 604069 (0.0005) [2023-12-26 19:45:56,426][105620] Updated weights for policy 1, policy_version 605005 (0.0008) [2023-12-26 19:45:56,496][105620] Updated weights for policy 1, policy_version 605015 (0.0009) [2023-12-26 19:45:56,560][105620] Updated weights for policy 1, policy_version 605025 (0.0010) [2023-12-26 19:45:56,765][105692] Updated weights for policy 0, policy_version 604079 (0.0007) [2023-12-26 19:45:56,822][105692] Updated weights for policy 0, policy_version 604089 (0.0006) [2023-12-26 19:45:56,883][105692] Updated weights for policy 0, policy_version 604099 (0.0009) [2023-12-26 19:45:57,173][105620] Updated weights for policy 1, policy_version 605035 (0.0009) [2023-12-26 19:45:57,229][105620] Updated weights for policy 1, policy_version 605045 (0.0005) [2023-12-26 19:45:57,300][105620] Updated weights for policy 1, policy_version 605055 (0.0005) [2023-12-26 19:45:57,561][105692] Updated weights for policy 0, policy_version 604109 (0.0008) [2023-12-26 19:45:57,631][105692] Updated weights for policy 0, policy_version 604119 (0.0009) [2023-12-26 19:45:57,699][105692] Updated weights for policy 0, policy_version 604129 (0.0010) [2023-12-26 19:45:57,806][105620] Updated weights for policy 1, policy_version 605065 (0.0006) [2023-12-26 19:45:57,863][105620] Updated weights for policy 1, policy_version 605075 (0.0009) [2023-12-26 19:45:57,911][105620] Updated weights for policy 1, policy_version 605085 (0.0009) [2023-12-26 19:45:57,965][105620] Updated weights for policy 1, policy_version 605095 (0.0009) [2023-12-26 19:45:58,464][105692] Updated weights for policy 0, policy_version 604139 (0.0008) [2023-12-26 19:45:58,533][105692] Updated weights for policy 0, policy_version 604149 (0.0006) [2023-12-26 19:45:58,596][105692] Updated weights for policy 0, policy_version 604159 (0.0008) [2023-12-26 19:45:58,691][105620] Updated weights for policy 1, policy_version 605105 (0.0008) [2023-12-26 19:45:58,762][105620] Updated weights for policy 1, policy_version 605115 (0.0007) [2023-12-26 19:45:58,830][105620] Updated weights for policy 1, policy_version 605125 (0.0008) [2023-12-26 19:45:59,313][105692] Updated weights for policy 0, policy_version 604169 (0.0008) [2023-12-26 19:45:59,379][105692] Updated weights for policy 0, policy_version 604179 (0.0009) [2023-12-26 19:45:59,447][105692] Updated weights for policy 0, policy_version 604189 (0.0009) [2023-12-26 19:45:59,505][105692] Updated weights for policy 0, policy_version 604199 (0.0005) [2023-12-26 19:45:59,552][105620] Updated weights for policy 1, policy_version 605135 (0.0009) [2023-12-26 19:45:59,615][105620] Updated weights for policy 1, policy_version 605145 (0.0009) [2023-12-26 19:45:59,676][105620] Updated weights for policy 1, policy_version 605155 (0.0009) [2023-12-26 19:46:00,181][105692] Updated weights for policy 0, policy_version 604209 (0.0005) [2023-12-26 19:46:00,242][105692] Updated weights for policy 0, policy_version 604219 (0.0005) [2023-12-26 19:46:00,300][105692] Updated weights for policy 0, policy_version 604229 (0.0005) [2023-12-26 19:46:00,406][105620] Updated weights for policy 1, policy_version 605165 (0.0008) [2023-12-26 19:46:00,463][105620] Updated weights for policy 1, policy_version 605175 (0.0009) [2023-12-26 19:46:00,523][105620] Updated weights for policy 1, policy_version 605185 (0.0008) [2023-12-26 19:46:00,961][105692] Updated weights for policy 0, policy_version 604239 (0.0006) [2023-12-26 19:46:01,021][105692] Updated weights for policy 0, policy_version 604249 (0.0005) [2023-12-26 19:46:01,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 309649408. Throughput: 0: 9615.2, 1: 9936.9. Samples: 309623484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:46:01,062][104569] Avg episode reward: [(0, '9265.957'), (1, '9355.903')] [2023-12-26 19:46:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000605192_154943488.pth... [2023-12-26 19:46:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000604040_154648576.pth [2023-12-26 19:46:01,084][105692] Updated weights for policy 0, policy_version 604259 (0.0011) [2023-12-26 19:46:01,108][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000604264_154714112.pth... [2023-12-26 19:46:01,127][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000603144_154427392.pth [2023-12-26 19:46:01,283][105620] Updated weights for policy 1, policy_version 605195 (0.0008) [2023-12-26 19:46:01,340][105620] Updated weights for policy 1, policy_version 605205 (0.0009) [2023-12-26 19:46:01,408][105620] Updated weights for policy 1, policy_version 605215 (0.0008) [2023-12-26 19:46:01,785][105692] Updated weights for policy 0, policy_version 604269 (0.0008) [2023-12-26 19:46:01,852][105692] Updated weights for policy 0, policy_version 604279 (0.0009) [2023-12-26 19:46:01,915][105692] Updated weights for policy 0, policy_version 604289 (0.0011) [2023-12-26 19:46:02,191][105620] Updated weights for policy 1, policy_version 605225 (0.0008) [2023-12-26 19:46:02,254][105620] Updated weights for policy 1, policy_version 605235 (0.0008) [2023-12-26 19:46:02,324][105620] Updated weights for policy 1, policy_version 605245 (0.0008) [2023-12-26 19:46:02,324][105586] KL-divergence is very high: 152.4874 [2023-12-26 19:46:02,376][105586] KL-divergence is very high: 157.6892 [2023-12-26 19:46:02,388][105620] Updated weights for policy 1, policy_version 605255 (0.0008) [2023-12-26 19:46:02,643][105692] Updated weights for policy 0, policy_version 604299 (0.0010) [2023-12-26 19:46:02,692][105692] Updated weights for policy 0, policy_version 604309 (0.0009) [2023-12-26 19:46:02,750][105692] Updated weights for policy 0, policy_version 604319 (0.0010) [2023-12-26 19:46:03,091][105620] Updated weights for policy 1, policy_version 605265 (0.0008) [2023-12-26 19:46:03,150][105620] Updated weights for policy 1, policy_version 605275 (0.0008) [2023-12-26 19:46:03,206][105620] Updated weights for policy 1, policy_version 605285 (0.0008) [2023-12-26 19:46:03,503][105692] Updated weights for policy 0, policy_version 604329 (0.0011) [2023-12-26 19:46:03,553][105692] Updated weights for policy 0, policy_version 604339 (0.0010) [2023-12-26 19:46:03,614][105692] Updated weights for policy 0, policy_version 604349 (0.0010) [2023-12-26 19:46:03,672][105692] Updated weights for policy 0, policy_version 604359 (0.0010) [2023-12-26 19:46:03,967][105620] Updated weights for policy 1, policy_version 605295 (0.0008) [2023-12-26 19:46:04,028][105620] Updated weights for policy 1, policy_version 605305 (0.0008) [2023-12-26 19:46:04,088][105620] Updated weights for policy 1, policy_version 605315 (0.0008) [2023-12-26 19:46:04,424][105692] Updated weights for policy 0, policy_version 604369 (0.0010) [2023-12-26 19:46:04,487][105692] Updated weights for policy 0, policy_version 604379 (0.0010) [2023-12-26 19:46:04,545][105692] Updated weights for policy 0, policy_version 604389 (0.0008) [2023-12-26 19:46:04,926][105620] Updated weights for policy 1, policy_version 605325 (0.0009) [2023-12-26 19:46:04,976][105620] Updated weights for policy 1, policy_version 605335 (0.0008) [2023-12-26 19:46:05,028][105620] Updated weights for policy 1, policy_version 605345 (0.0008) [2023-12-26 19:46:05,197][105692] Updated weights for policy 0, policy_version 604399 (0.0009) [2023-12-26 19:46:05,252][105692] Updated weights for policy 0, policy_version 604409 (0.0010) [2023-12-26 19:46:05,301][105692] Updated weights for policy 0, policy_version 604419 (0.0007) [2023-12-26 19:46:05,683][105620] Updated weights for policy 1, policy_version 605355 (0.0007) [2023-12-26 19:46:05,749][105620] Updated weights for policy 1, policy_version 605365 (0.0010) [2023-12-26 19:46:05,804][105620] Updated weights for policy 1, policy_version 605375 (0.0010) [2023-12-26 19:46:06,049][105692] Updated weights for policy 0, policy_version 604429 (0.0010) [2023-12-26 19:46:06,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.8, 300 sec: 19521.9). Total num frames: 309747712. Throughput: 0: 9503.6, 1: 9780.8. Samples: 309736384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:46:06,062][104569] Avg episode reward: [(0, '9265.655'), (1, '9173.665')] [2023-12-26 19:46:06,097][105692] Updated weights for policy 0, policy_version 604439 (0.0010) [2023-12-26 19:46:06,156][105692] Updated weights for policy 0, policy_version 604449 (0.0010) [2023-12-26 19:46:06,539][105620] Updated weights for policy 1, policy_version 605385 (0.0010) [2023-12-26 19:46:06,599][105620] Updated weights for policy 1, policy_version 605395 (0.0011) [2023-12-26 19:46:06,667][105620] Updated weights for policy 1, policy_version 605405 (0.0008) [2023-12-26 19:46:06,726][105620] Updated weights for policy 1, policy_version 605415 (0.0011) [2023-12-26 19:46:06,811][105692] Updated weights for policy 0, policy_version 604459 (0.0010) [2023-12-26 19:46:06,872][105692] Updated weights for policy 0, policy_version 604469 (0.0011) [2023-12-26 19:46:06,932][105692] Updated weights for policy 0, policy_version 604479 (0.0011) [2023-12-26 19:46:07,424][105620] Updated weights for policy 1, policy_version 605425 (0.0010) [2023-12-26 19:46:07,473][105620] Updated weights for policy 1, policy_version 605435 (0.0010) [2023-12-26 19:46:07,521][105620] Updated weights for policy 1, policy_version 605445 (0.0010) [2023-12-26 19:46:07,596][105692] Updated weights for policy 0, policy_version 604489 (0.0010) [2023-12-26 19:46:07,653][105692] Updated weights for policy 0, policy_version 604499 (0.0005) [2023-12-26 19:46:07,708][105692] Updated weights for policy 0, policy_version 604509 (0.0006) [2023-12-26 19:46:07,764][105692] Updated weights for policy 0, policy_version 604519 (0.0005) [2023-12-26 19:46:08,278][105692] Updated weights for policy 0, policy_version 604529 (0.0006) [2023-12-26 19:46:08,298][105620] Updated weights for policy 1, policy_version 605455 (0.0011) [2023-12-26 19:46:08,331][105692] Updated weights for policy 0, policy_version 604539 (0.0007) [2023-12-26 19:46:08,362][105620] Updated weights for policy 1, policy_version 605465 (0.0010) [2023-12-26 19:46:08,393][105692] Updated weights for policy 0, policy_version 604549 (0.0008) [2023-12-26 19:46:08,420][105620] Updated weights for policy 1, policy_version 605475 (0.0011) [2023-12-26 19:46:09,138][105692] Updated weights for policy 0, policy_version 604559 (0.0010) [2023-12-26 19:46:09,191][105692] Updated weights for policy 0, policy_version 604569 (0.0011) [2023-12-26 19:46:09,193][105620] Updated weights for policy 1, policy_version 605485 (0.0011) [2023-12-26 19:46:09,249][105692] Updated weights for policy 0, policy_version 604579 (0.0009) [2023-12-26 19:46:09,258][105620] Updated weights for policy 1, policy_version 605495 (0.0011) [2023-12-26 19:46:09,324][105620] Updated weights for policy 1, policy_version 605505 (0.0009) [2023-12-26 19:46:10,045][105692] Updated weights for policy 0, policy_version 604589 (0.0010) [2023-12-26 19:46:10,104][105692] Updated weights for policy 0, policy_version 604599 (0.0010) [2023-12-26 19:46:10,147][105620] Updated weights for policy 1, policy_version 605515 (0.0008) [2023-12-26 19:46:10,166][105692] Updated weights for policy 0, policy_version 604609 (0.0009) [2023-12-26 19:46:10,208][105620] Updated weights for policy 1, policy_version 605525 (0.0008) [2023-12-26 19:46:10,266][105620] Updated weights for policy 1, policy_version 605535 (0.0009) [2023-12-26 19:46:10,878][105692] Updated weights for policy 0, policy_version 604619 (0.0007) [2023-12-26 19:46:10,944][105692] Updated weights for policy 0, policy_version 604629 (0.0009) [2023-12-26 19:46:11,002][105692] Updated weights for policy 0, policy_version 604639 (0.0008) [2023-12-26 19:46:11,003][105620] Updated weights for policy 1, policy_version 605545 (0.0009) [2023-12-26 19:46:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 309837824. Throughput: 0: 9594.7, 1: 9794.1. Samples: 309853640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:46:11,063][104569] Avg episode reward: [(0, '9265.587'), (1, '9081.981')] [2023-12-26 19:46:11,075][105620] Updated weights for policy 1, policy_version 605555 (0.0009) [2023-12-26 19:46:11,147][105620] Updated weights for policy 1, policy_version 605565 (0.0008) [2023-12-26 19:46:11,210][105620] Updated weights for policy 1, policy_version 605575 (0.0008) [2023-12-26 19:46:11,752][105692] Updated weights for policy 0, policy_version 604649 (0.0008) [2023-12-26 19:46:11,813][105692] Updated weights for policy 0, policy_version 604659 (0.0009) [2023-12-26 19:46:11,876][105692] Updated weights for policy 0, policy_version 604669 (0.0009) [2023-12-26 19:46:11,941][105692] Updated weights for policy 0, policy_version 604679 (0.0008) [2023-12-26 19:46:11,944][105620] Updated weights for policy 1, policy_version 605585 (0.0007) [2023-12-26 19:46:12,003][105620] Updated weights for policy 1, policy_version 605595 (0.0005) [2023-12-26 19:46:12,063][105620] Updated weights for policy 1, policy_version 605605 (0.0005) [2023-12-26 19:46:12,707][105692] Updated weights for policy 0, policy_version 604689 (0.0009) [2023-12-26 19:46:12,769][105692] Updated weights for policy 0, policy_version 604699 (0.0007) [2023-12-26 19:46:12,773][105620] Updated weights for policy 1, policy_version 605615 (0.0007) [2023-12-26 19:46:12,827][105692] Updated weights for policy 0, policy_version 604709 (0.0006) [2023-12-26 19:46:12,838][105620] Updated weights for policy 1, policy_version 605625 (0.0008) [2023-12-26 19:46:12,902][105620] Updated weights for policy 1, policy_version 605635 (0.0010) [2023-12-26 19:46:13,448][105692] Updated weights for policy 0, policy_version 604719 (0.0005) [2023-12-26 19:46:13,456][105620] Updated weights for policy 1, policy_version 605645 (0.0008) [2023-12-26 19:46:13,500][105692] Updated weights for policy 0, policy_version 604729 (0.0005) [2023-12-26 19:46:13,508][105620] Updated weights for policy 1, policy_version 605655 (0.0005) [2023-12-26 19:46:13,552][105692] Updated weights for policy 0, policy_version 604739 (0.0005) [2023-12-26 19:46:13,555][105620] Updated weights for policy 1, policy_version 605665 (0.0005) [2023-12-26 19:46:14,088][105692] Updated weights for policy 0, policy_version 604749 (0.0005) [2023-12-26 19:46:14,135][105692] Updated weights for policy 0, policy_version 604759 (0.0005) [2023-12-26 19:46:14,181][105620] Updated weights for policy 1, policy_version 605675 (0.0007) [2023-12-26 19:46:14,191][105692] Updated weights for policy 0, policy_version 604769 (0.0005) [2023-12-26 19:46:14,240][105620] Updated weights for policy 1, policy_version 605685 (0.0010) [2023-12-26 19:46:14,295][105620] Updated weights for policy 1, policy_version 605695 (0.0010) [2023-12-26 19:46:14,846][105692] Updated weights for policy 0, policy_version 604779 (0.0010) [2023-12-26 19:46:14,917][105692] Updated weights for policy 0, policy_version 604789 (0.0008) [2023-12-26 19:46:14,982][105692] Updated weights for policy 0, policy_version 604799 (0.0008) [2023-12-26 19:46:15,039][105620] Updated weights for policy 1, policy_version 605705 (0.0010) [2023-12-26 19:46:15,101][105620] Updated weights for policy 1, policy_version 605715 (0.0010) [2023-12-26 19:46:15,172][105620] Updated weights for policy 1, policy_version 605725 (0.0011) [2023-12-26 19:46:15,217][105620] Updated weights for policy 1, policy_version 605735 (0.0010) [2023-12-26 19:46:15,633][105692] Updated weights for policy 0, policy_version 604809 (0.0008) [2023-12-26 19:46:15,681][105692] Updated weights for policy 0, policy_version 604819 (0.0005) [2023-12-26 19:46:15,726][105692] Updated weights for policy 0, policy_version 604829 (0.0005) [2023-12-26 19:46:15,777][105692] Updated weights for policy 0, policy_version 604839 (0.0005) [2023-12-26 19:46:15,966][105620] Updated weights for policy 1, policy_version 605745 (0.0010) [2023-12-26 19:46:16,011][105620] Updated weights for policy 1, policy_version 605755 (0.0010) [2023-12-26 19:46:16,059][105620] Updated weights for policy 1, policy_version 605765 (0.0010) [2023-12-26 19:46:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 309944320. Throughput: 0: 9592.9, 1: 9841.5. Samples: 309913604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:46:16,062][104569] Avg episode reward: [(0, '9356.079'), (1, '9172.001')] [2023-12-26 19:46:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000604840_154861568.pth... [2023-12-26 19:46:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000603688_154566656.pth [2023-12-26 19:46:16,077][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000605768_155090944.pth... [2023-12-26 19:46:16,082][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000604616_154796032.pth [2023-12-26 19:46:16,377][105692] Updated weights for policy 0, policy_version 604849 (0.0010) [2023-12-26 19:46:16,429][105692] Updated weights for policy 0, policy_version 604859 (0.0010) [2023-12-26 19:46:16,498][105692] Updated weights for policy 0, policy_version 604869 (0.0011) [2023-12-26 19:46:16,655][105620] Updated weights for policy 1, policy_version 605775 (0.0007) [2023-12-26 19:46:16,706][105620] Updated weights for policy 1, policy_version 605785 (0.0010) [2023-12-26 19:46:16,757][105620] Updated weights for policy 1, policy_version 605795 (0.0010) [2023-12-26 19:46:17,167][105692] Updated weights for policy 0, policy_version 604879 (0.0009) [2023-12-26 19:46:17,223][105692] Updated weights for policy 0, policy_version 604889 (0.0009) [2023-12-26 19:46:17,285][105692] Updated weights for policy 0, policy_version 604899 (0.0005) [2023-12-26 19:46:17,579][105620] Updated weights for policy 1, policy_version 605805 (0.0009) [2023-12-26 19:46:17,639][105620] Updated weights for policy 1, policy_version 605815 (0.0009) [2023-12-26 19:46:17,698][105620] Updated weights for policy 1, policy_version 605825 (0.0011) [2023-12-26 19:46:17,894][105692] Updated weights for policy 0, policy_version 604909 (0.0007) [2023-12-26 19:46:17,960][105692] Updated weights for policy 0, policy_version 604919 (0.0009) [2023-12-26 19:46:18,015][105692] Updated weights for policy 0, policy_version 604929 (0.0008) [2023-12-26 19:46:18,451][105620] Updated weights for policy 1, policy_version 605835 (0.0011) [2023-12-26 19:46:18,510][105620] Updated weights for policy 1, policy_version 605845 (0.0011) [2023-12-26 19:46:18,570][105620] Updated weights for policy 1, policy_version 605855 (0.0011) [2023-12-26 19:46:18,673][105692] Updated weights for policy 0, policy_version 604939 (0.0008) [2023-12-26 19:46:18,731][105692] Updated weights for policy 0, policy_version 604949 (0.0008) [2023-12-26 19:46:18,789][105692] Updated weights for policy 0, policy_version 604959 (0.0007) [2023-12-26 19:46:19,321][105620] Updated weights for policy 1, policy_version 605865 (0.0011) [2023-12-26 19:46:19,386][105620] Updated weights for policy 1, policy_version 605875 (0.0011) [2023-12-26 19:46:19,445][105692] Updated weights for policy 0, policy_version 604969 (0.0006) [2023-12-26 19:46:19,450][105620] Updated weights for policy 1, policy_version 605885 (0.0011) [2023-12-26 19:46:19,505][105692] Updated weights for policy 0, policy_version 604979 (0.0009) [2023-12-26 19:46:19,534][105620] Updated weights for policy 1, policy_version 605895 (0.0010) [2023-12-26 19:46:19,558][105692] Updated weights for policy 0, policy_version 604989 (0.0008) [2023-12-26 19:46:19,622][105692] Updated weights for policy 0, policy_version 604999 (0.0008) [2023-12-26 19:46:20,230][105620] Updated weights for policy 1, policy_version 605905 (0.0008) [2023-12-26 19:46:20,296][105620] Updated weights for policy 1, policy_version 605915 (0.0007) [2023-12-26 19:46:20,364][105620] Updated weights for policy 1, policy_version 605925 (0.0010) [2023-12-26 19:46:20,478][105692] Updated weights for policy 0, policy_version 605009 (0.0010) [2023-12-26 19:46:20,546][105692] Updated weights for policy 0, policy_version 605019 (0.0010) [2023-12-26 19:46:20,613][105692] Updated weights for policy 0, policy_version 605029 (0.0007) [2023-12-26 19:46:20,957][105620] Updated weights for policy 1, policy_version 605935 (0.0009) [2023-12-26 19:46:21,028][105620] Updated weights for policy 1, policy_version 605945 (0.0008) [2023-12-26 19:46:21,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 310042624. Throughput: 0: 9796.4, 1: 9776.0. Samples: 310035528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:46:21,062][104569] Avg episode reward: [(0, '9265.838'), (1, '9263.856')] [2023-12-26 19:46:21,092][105620] Updated weights for policy 1, policy_version 605955 (0.0007) [2023-12-26 19:46:21,345][105692] Updated weights for policy 0, policy_version 605039 (0.0008) [2023-12-26 19:46:21,413][105692] Updated weights for policy 0, policy_version 605049 (0.0008) [2023-12-26 19:46:21,469][105692] Updated weights for policy 0, policy_version 605059 (0.0008) [2023-12-26 19:46:21,854][105620] Updated weights for policy 1, policy_version 605965 (0.0009) [2023-12-26 19:46:21,914][105620] Updated weights for policy 1, policy_version 605975 (0.0006) [2023-12-26 19:46:21,986][105620] Updated weights for policy 1, policy_version 605985 (0.0008) [2023-12-26 19:46:22,231][105692] Updated weights for policy 0, policy_version 605069 (0.0007) [2023-12-26 19:46:22,297][105692] Updated weights for policy 0, policy_version 605079 (0.0007) [2023-12-26 19:46:22,361][105692] Updated weights for policy 0, policy_version 605089 (0.0008) [2023-12-26 19:46:22,717][105620] Updated weights for policy 1, policy_version 605995 (0.0009) [2023-12-26 19:46:22,777][105620] Updated weights for policy 1, policy_version 606005 (0.0011) [2023-12-26 19:46:22,847][105620] Updated weights for policy 1, policy_version 606015 (0.0011) [2023-12-26 19:46:23,064][105692] Updated weights for policy 0, policy_version 605099 (0.0007) [2023-12-26 19:46:23,133][105692] Updated weights for policy 0, policy_version 605109 (0.0009) [2023-12-26 19:46:23,197][105692] Updated weights for policy 0, policy_version 605119 (0.0011) [2023-12-26 19:46:23,559][105620] Updated weights for policy 1, policy_version 606025 (0.0010) [2023-12-26 19:46:23,621][105620] Updated weights for policy 1, policy_version 606035 (0.0007) [2023-12-26 19:46:23,683][105620] Updated weights for policy 1, policy_version 606045 (0.0010) [2023-12-26 19:46:23,736][105620] Updated weights for policy 1, policy_version 606055 (0.0010) [2023-12-26 19:46:23,881][105692] Updated weights for policy 0, policy_version 605129 (0.0009) [2023-12-26 19:46:23,937][105692] Updated weights for policy 0, policy_version 605139 (0.0005) [2023-12-26 19:46:24,008][105692] Updated weights for policy 0, policy_version 605149 (0.0005) [2023-12-26 19:46:24,076][105692] Updated weights for policy 0, policy_version 605159 (0.0006) [2023-12-26 19:46:24,347][105620] Updated weights for policy 1, policy_version 606065 (0.0008) [2023-12-26 19:46:24,411][105620] Updated weights for policy 1, policy_version 606075 (0.0008) [2023-12-26 19:46:24,475][105620] Updated weights for policy 1, policy_version 606085 (0.0005) [2023-12-26 19:46:24,677][105692] Updated weights for policy 0, policy_version 605169 (0.0006) [2023-12-26 19:46:24,739][105692] Updated weights for policy 0, policy_version 605179 (0.0008) [2023-12-26 19:46:24,797][105692] Updated weights for policy 0, policy_version 605189 (0.0010) [2023-12-26 19:46:25,025][105620] Updated weights for policy 1, policy_version 606095 (0.0007) [2023-12-26 19:46:25,073][105620] Updated weights for policy 1, policy_version 606105 (0.0008) [2023-12-26 19:46:25,121][105620] Updated weights for policy 1, policy_version 606115 (0.0007) [2023-12-26 19:46:25,491][105692] Updated weights for policy 0, policy_version 605199 (0.0010) [2023-12-26 19:46:25,540][105692] Updated weights for policy 0, policy_version 605209 (0.0009) [2023-12-26 19:46:25,589][105692] Updated weights for policy 0, policy_version 605219 (0.0008) [2023-12-26 19:46:25,838][105620] Updated weights for policy 1, policy_version 606125 (0.0009) [2023-12-26 19:46:25,882][105620] Updated weights for policy 1, policy_version 606135 (0.0010) [2023-12-26 19:46:25,938][105620] Updated weights for policy 1, policy_version 606145 (0.0010) [2023-12-26 19:46:26,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 310149120. Throughput: 0: 9752.9, 1: 9800.3. Samples: 310153708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:46:26,062][104569] Avg episode reward: [(0, '9174.932'), (1, '9264.246')] [2023-12-26 19:46:26,288][105692] Updated weights for policy 0, policy_version 605229 (0.0007) [2023-12-26 19:46:26,344][105692] Updated weights for policy 0, policy_version 605239 (0.0005) [2023-12-26 19:46:26,399][105692] Updated weights for policy 0, policy_version 605249 (0.0006) [2023-12-26 19:46:26,746][105620] Updated weights for policy 1, policy_version 606155 (0.0010) [2023-12-26 19:46:26,819][105620] Updated weights for policy 1, policy_version 606165 (0.0009) [2023-12-26 19:46:26,891][105620] Updated weights for policy 1, policy_version 606175 (0.0009) [2023-12-26 19:46:26,946][105692] Updated weights for policy 0, policy_version 605259 (0.0006) [2023-12-26 19:46:26,995][105692] Updated weights for policy 0, policy_version 605269 (0.0005) [2023-12-26 19:46:27,054][105692] Updated weights for policy 0, policy_version 605279 (0.0005) [2023-12-26 19:46:27,580][105620] Updated weights for policy 1, policy_version 606185 (0.0009) [2023-12-26 19:46:27,631][105620] Updated weights for policy 1, policy_version 606195 (0.0005) [2023-12-26 19:46:27,681][105692] Updated weights for policy 0, policy_version 605289 (0.0007) [2023-12-26 19:46:27,681][105620] Updated weights for policy 1, policy_version 606205 (0.0005) [2023-12-26 19:46:27,725][105620] Updated weights for policy 1, policy_version 606215 (0.0005) [2023-12-26 19:46:27,729][105692] Updated weights for policy 0, policy_version 605299 (0.0008) [2023-12-26 19:46:27,784][105692] Updated weights for policy 0, policy_version 605311 (0.0010) [2023-12-26 19:46:28,261][105620] Updated weights for policy 1, policy_version 606225 (0.0005) [2023-12-26 19:46:28,312][105620] Updated weights for policy 1, policy_version 606235 (0.0005) [2023-12-26 19:46:28,388][105620] Updated weights for policy 1, policy_version 606245 (0.0008) [2023-12-26 19:46:28,615][105692] Updated weights for policy 0, policy_version 605321 (0.0010) [2023-12-26 19:46:28,672][105692] Updated weights for policy 0, policy_version 605331 (0.0008) [2023-12-26 19:46:28,732][105692] Updated weights for policy 0, policy_version 605341 (0.0007) [2023-12-26 19:46:28,785][105692] Updated weights for policy 0, policy_version 605351 (0.0008) [2023-12-26 19:46:29,082][105620] Updated weights for policy 1, policy_version 606255 (0.0009) [2023-12-26 19:46:29,141][105620] Updated weights for policy 1, policy_version 606266 (0.0010) [2023-12-26 19:46:29,190][105620] Updated weights for policy 1, policy_version 606276 (0.0008) [2023-12-26 19:46:29,503][105692] Updated weights for policy 0, policy_version 605361 (0.0006) [2023-12-26 19:46:29,570][105692] Updated weights for policy 0, policy_version 605371 (0.0007) [2023-12-26 19:46:29,618][105692] Updated weights for policy 0, policy_version 605381 (0.0006) [2023-12-26 19:46:29,890][105620] Updated weights for policy 1, policy_version 606286 (0.0007) [2023-12-26 19:46:29,947][105620] Updated weights for policy 1, policy_version 606296 (0.0007) [2023-12-26 19:46:30,011][105620] Updated weights for policy 1, policy_version 606306 (0.0009) [2023-12-26 19:46:30,410][105692] Updated weights for policy 0, policy_version 605391 (0.0007) [2023-12-26 19:46:30,483][105692] Updated weights for policy 0, policy_version 605401 (0.0006) [2023-12-26 19:46:30,542][105692] Updated weights for policy 0, policy_version 605411 (0.0006) [2023-12-26 19:46:30,720][105620] Updated weights for policy 1, policy_version 606316 (0.0008) [2023-12-26 19:46:30,785][105620] Updated weights for policy 1, policy_version 606326 (0.0009) [2023-12-26 19:46:30,853][105620] Updated weights for policy 1, policy_version 606336 (0.0008) [2023-12-26 19:46:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 310247424. Throughput: 0: 9855.8, 1: 9856.5. Samples: 310215812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:46:31,062][104569] Avg episode reward: [(0, '9081.927'), (1, '9081.172')] [2023-12-26 19:46:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000605416_155009024.pth... [2023-12-26 19:46:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000606344_155238400.pth... [2023-12-26 19:46:31,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000604264_154714112.pth [2023-12-26 19:46:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000605192_154943488.pth [2023-12-26 19:46:31,219][105692] Updated weights for policy 0, policy_version 605421 (0.0010) [2023-12-26 19:46:31,279][105692] Updated weights for policy 0, policy_version 605431 (0.0010) [2023-12-26 19:46:31,333][105692] Updated weights for policy 0, policy_version 605441 (0.0010) [2023-12-26 19:46:31,570][105620] Updated weights for policy 1, policy_version 606346 (0.0008) [2023-12-26 19:46:31,634][105620] Updated weights for policy 1, policy_version 606356 (0.0009) [2023-12-26 19:46:31,691][105620] Updated weights for policy 1, policy_version 606366 (0.0006) [2023-12-26 19:46:31,756][105620] Updated weights for policy 1, policy_version 606376 (0.0008) [2023-12-26 19:46:32,090][105692] Updated weights for policy 0, policy_version 605451 (0.0009) [2023-12-26 19:46:32,144][105692] Updated weights for policy 0, policy_version 605461 (0.0010) [2023-12-26 19:46:32,200][105692] Updated weights for policy 0, policy_version 605471 (0.0009) [2023-12-26 19:46:32,408][105620] Updated weights for policy 1, policy_version 606386 (0.0010) [2023-12-26 19:46:32,471][105620] Updated weights for policy 1, policy_version 606396 (0.0008) [2023-12-26 19:46:32,529][105620] Updated weights for policy 1, policy_version 606406 (0.0009) [2023-12-26 19:46:32,971][105692] Updated weights for policy 0, policy_version 605481 (0.0009) [2023-12-26 19:46:33,031][105692] Updated weights for policy 0, policy_version 605491 (0.0009) [2023-12-26 19:46:33,089][105692] Updated weights for policy 0, policy_version 605501 (0.0010) [2023-12-26 19:46:33,148][105692] Updated weights for policy 0, policy_version 605512 (0.0010) [2023-12-26 19:46:33,243][105620] Updated weights for policy 1, policy_version 606416 (0.0006) [2023-12-26 19:46:33,287][105620] Updated weights for policy 1, policy_version 606426 (0.0005) [2023-12-26 19:46:33,338][105620] Updated weights for policy 1, policy_version 606436 (0.0006) [2023-12-26 19:46:33,814][105692] Updated weights for policy 0, policy_version 605522 (0.0010) [2023-12-26 19:46:33,862][105692] Updated weights for policy 0, policy_version 605532 (0.0010) [2023-12-26 19:46:33,917][105692] Updated weights for policy 0, policy_version 605542 (0.0010) [2023-12-26 19:46:33,953][105620] Updated weights for policy 1, policy_version 606446 (0.0005) [2023-12-26 19:46:34,007][105620] Updated weights for policy 1, policy_version 606456 (0.0006) [2023-12-26 19:46:34,065][105620] Updated weights for policy 1, policy_version 606466 (0.0006) [2023-12-26 19:46:34,652][105692] Updated weights for policy 0, policy_version 605552 (0.0008) [2023-12-26 19:46:34,719][105692] Updated weights for policy 0, policy_version 605562 (0.0006) [2023-12-26 19:46:34,753][105620] Updated weights for policy 1, policy_version 606476 (0.0008) [2023-12-26 19:46:34,775][105692] Updated weights for policy 0, policy_version 605572 (0.0005) [2023-12-26 19:46:34,808][105620] Updated weights for policy 1, policy_version 606486 (0.0010) [2023-12-26 19:46:34,859][105620] Updated weights for policy 1, policy_version 606496 (0.0010) [2023-12-26 19:46:35,448][105692] Updated weights for policy 0, policy_version 605582 (0.0006) [2023-12-26 19:46:35,508][105692] Updated weights for policy 0, policy_version 605592 (0.0005) [2023-12-26 19:46:35,554][105692] Updated weights for policy 0, policy_version 605602 (0.0008) [2023-12-26 19:46:35,619][105620] Updated weights for policy 1, policy_version 606506 (0.0010) [2023-12-26 19:46:35,668][105620] Updated weights for policy 1, policy_version 606516 (0.0010) [2023-12-26 19:46:35,730][105620] Updated weights for policy 1, policy_version 606526 (0.0010) [2023-12-26 19:46:35,795][105620] Updated weights for policy 1, policy_version 606536 (0.0010) [2023-12-26 19:46:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 310345728. Throughput: 0: 9899.9, 1: 9910.3. Samples: 310333508. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 19:46:36,062][104569] Avg episode reward: [(0, '9081.644'), (1, '9081.203')] [2023-12-26 19:46:36,222][105692] Updated weights for policy 0, policy_version 605612 (0.0008) [2023-12-26 19:46:36,275][105692] Updated weights for policy 0, policy_version 605622 (0.0008) [2023-12-26 19:46:36,329][105692] Updated weights for policy 0, policy_version 605632 (0.0008) [2023-12-26 19:46:36,534][105620] Updated weights for policy 1, policy_version 606546 (0.0010) [2023-12-26 19:46:36,583][105620] Updated weights for policy 1, policy_version 606556 (0.0010) [2023-12-26 19:46:36,639][105620] Updated weights for policy 1, policy_version 606566 (0.0005) [2023-12-26 19:46:37,198][105692] Updated weights for policy 0, policy_version 605642 (0.0008) [2023-12-26 19:46:37,251][105692] Updated weights for policy 0, policy_version 605652 (0.0007) [2023-12-26 19:46:37,253][105620] Updated weights for policy 1, policy_version 606576 (0.0009) [2023-12-26 19:46:37,303][105692] Updated weights for policy 0, policy_version 605662 (0.0006) [2023-12-26 19:46:37,308][105620] Updated weights for policy 1, policy_version 606586 (0.0010) [2023-12-26 19:46:37,358][105692] Updated weights for policy 0, policy_version 605672 (0.0007) [2023-12-26 19:46:37,363][105620] Updated weights for policy 1, policy_version 606596 (0.0010) [2023-12-26 19:46:38,121][105620] Updated weights for policy 1, policy_version 606606 (0.0010) [2023-12-26 19:46:38,124][105692] Updated weights for policy 0, policy_version 605682 (0.0006) [2023-12-26 19:46:38,177][105692] Updated weights for policy 0, policy_version 605692 (0.0007) [2023-12-26 19:46:38,186][105620] Updated weights for policy 1, policy_version 606616 (0.0010) [2023-12-26 19:46:38,233][105692] Updated weights for policy 0, policy_version 605702 (0.0008) [2023-12-26 19:46:38,247][105620] Updated weights for policy 1, policy_version 606626 (0.0010) [2023-12-26 19:46:38,861][105620] Updated weights for policy 1, policy_version 606636 (0.0009) [2023-12-26 19:46:38,925][105620] Updated weights for policy 1, policy_version 606646 (0.0009) [2023-12-26 19:46:38,986][105620] Updated weights for policy 1, policy_version 606656 (0.0009) [2023-12-26 19:46:39,064][105692] Updated weights for policy 0, policy_version 605712 (0.0008) [2023-12-26 19:46:39,127][105692] Updated weights for policy 0, policy_version 605722 (0.0009) [2023-12-26 19:46:39,177][105692] Updated weights for policy 0, policy_version 605732 (0.0008) [2023-12-26 19:46:39,772][105620] Updated weights for policy 1, policy_version 606666 (0.0009) [2023-12-26 19:46:39,833][105620] Updated weights for policy 1, policy_version 606676 (0.0009) [2023-12-26 19:46:39,897][105620] Updated weights for policy 1, policy_version 606686 (0.0007) [2023-12-26 19:46:39,959][105692] Updated weights for policy 0, policy_version 605742 (0.0008) [2023-12-26 19:46:39,960][105620] Updated weights for policy 1, policy_version 606696 (0.0008) [2023-12-26 19:46:40,013][105692] Updated weights for policy 0, policy_version 605752 (0.0005) [2023-12-26 19:46:40,070][105692] Updated weights for policy 0, policy_version 605762 (0.0006) [2023-12-26 19:46:40,753][105620] Updated weights for policy 1, policy_version 606706 (0.0008) [2023-12-26 19:46:40,759][105692] Updated weights for policy 0, policy_version 605772 (0.0007) [2023-12-26 19:46:40,803][105692] Updated weights for policy 0, policy_version 605782 (0.0006) [2023-12-26 19:46:40,818][105620] Updated weights for policy 1, policy_version 606716 (0.0008) [2023-12-26 19:46:40,860][105692] Updated weights for policy 0, policy_version 605792 (0.0008) [2023-12-26 19:46:40,875][105620] Updated weights for policy 1, policy_version 606726 (0.0007) [2023-12-26 19:46:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 310444032. Throughput: 0: 9868.8, 1: 9779.5. Samples: 310447136. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 19:46:41,062][104569] Avg episode reward: [(0, '9264.665'), (1, '9171.312')] [2023-12-26 19:46:41,652][105692] Updated weights for policy 0, policy_version 605802 (0.0008) [2023-12-26 19:46:41,671][105620] Updated weights for policy 1, policy_version 606736 (0.0008) [2023-12-26 19:46:41,711][105692] Updated weights for policy 0, policy_version 605812 (0.0007) [2023-12-26 19:46:41,745][105620] Updated weights for policy 1, policy_version 606746 (0.0008) [2023-12-26 19:46:41,774][105692] Updated weights for policy 0, policy_version 605822 (0.0009) [2023-12-26 19:46:41,806][105620] Updated weights for policy 1, policy_version 606756 (0.0008) [2023-12-26 19:46:41,830][105692] Updated weights for policy 0, policy_version 605832 (0.0008) [2023-12-26 19:46:42,564][105620] Updated weights for policy 1, policy_version 606766 (0.0007) [2023-12-26 19:46:42,626][105620] Updated weights for policy 1, policy_version 606776 (0.0007) [2023-12-26 19:46:42,636][105585] KL-divergence is very high: 390.0963 [2023-12-26 19:46:42,648][105692] Updated weights for policy 0, policy_version 605842 (0.0008) [2023-12-26 19:46:42,683][105585] KL-divergence is very high: 604.9247 [2023-12-26 19:46:42,687][105620] Updated weights for policy 1, policy_version 606786 (0.0006) [2023-12-26 19:46:42,705][105692] Updated weights for policy 0, policy_version 605852 (0.0007) [2023-12-26 19:46:42,729][105585] KL-divergence is very high: 471.4491 [2023-12-26 19:46:42,766][105692] Updated weights for policy 0, policy_version 605862 (0.0008) [2023-12-26 19:46:43,354][105620] Updated weights for policy 1, policy_version 606796 (0.0007) [2023-12-26 19:46:43,416][105620] Updated weights for policy 1, policy_version 606806 (0.0009) [2023-12-26 19:46:43,482][105620] Updated weights for policy 1, policy_version 606816 (0.0009) [2023-12-26 19:46:43,549][105692] Updated weights for policy 0, policy_version 605872 (0.0009) [2023-12-26 19:46:43,608][105692] Updated weights for policy 0, policy_version 605883 (0.0009) [2023-12-26 19:46:43,674][105692] Updated weights for policy 0, policy_version 605893 (0.0009) [2023-12-26 19:46:44,126][105620] Updated weights for policy 1, policy_version 606826 (0.0008) [2023-12-26 19:46:44,173][105620] Updated weights for policy 1, policy_version 606836 (0.0008) [2023-12-26 19:46:44,233][105620] Updated weights for policy 1, policy_version 606846 (0.0009) [2023-12-26 19:46:44,293][105620] Updated weights for policy 1, policy_version 606856 (0.0008) [2023-12-26 19:46:44,487][105692] Updated weights for policy 0, policy_version 605903 (0.0009) [2023-12-26 19:46:44,548][105692] Updated weights for policy 0, policy_version 605913 (0.0009) [2023-12-26 19:46:44,594][105692] Updated weights for policy 0, policy_version 605923 (0.0008) [2023-12-26 19:46:45,031][105620] Updated weights for policy 1, policy_version 606866 (0.0006) [2023-12-26 19:46:45,083][105620] Updated weights for policy 1, policy_version 606876 (0.0009) [2023-12-26 19:46:45,136][105620] Updated weights for policy 1, policy_version 606886 (0.0009) [2023-12-26 19:46:45,349][105692] Updated weights for policy 0, policy_version 605933 (0.0009) [2023-12-26 19:46:45,414][105692] Updated weights for policy 0, policy_version 605943 (0.0008) [2023-12-26 19:46:45,470][105692] Updated weights for policy 0, policy_version 605953 (0.0008) [2023-12-26 19:46:45,867][105620] Updated weights for policy 1, policy_version 606896 (0.0009) [2023-12-26 19:46:45,916][105620] Updated weights for policy 1, policy_version 606907 (0.0009) [2023-12-26 19:46:45,976][105620] Updated weights for policy 1, policy_version 606917 (0.0006) [2023-12-26 19:46:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 310534144. Throughput: 0: 9809.6, 1: 9737.8. Samples: 310503120. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 19:46:46,063][104569] Avg episode reward: [(0, '9355.812'), (1, '9170.537')] [2023-12-26 19:46:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000606920_155385856.pth... [2023-12-26 19:46:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000605960_155148288.pth... [2023-12-26 19:46:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000605768_155090944.pth [2023-12-26 19:46:46,080][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000604840_154861568.pth [2023-12-26 19:46:46,153][105692] Updated weights for policy 0, policy_version 605963 (0.0008) [2023-12-26 19:46:46,207][105692] Updated weights for policy 0, policy_version 605973 (0.0009) [2023-12-26 19:46:46,264][105692] Updated weights for policy 0, policy_version 605983 (0.0008) [2023-12-26 19:46:46,674][105620] Updated weights for policy 1, policy_version 606927 (0.0005) [2023-12-26 19:46:46,730][105620] Updated weights for policy 1, policy_version 606937 (0.0006) [2023-12-26 19:46:46,785][105620] Updated weights for policy 1, policy_version 606947 (0.0008) [2023-12-26 19:46:46,931][105692] Updated weights for policy 0, policy_version 605993 (0.0006) [2023-12-26 19:46:46,988][105692] Updated weights for policy 0, policy_version 606003 (0.0006) [2023-12-26 19:46:47,041][105692] Updated weights for policy 0, policy_version 606013 (0.0007) [2023-12-26 19:46:47,086][105692] Updated weights for policy 0, policy_version 606023 (0.0010) [2023-12-26 19:46:47,420][105620] Updated weights for policy 1, policy_version 606957 (0.0007) [2023-12-26 19:46:47,471][105620] Updated weights for policy 1, policy_version 606967 (0.0005) [2023-12-26 19:46:47,532][105620] Updated weights for policy 1, policy_version 606977 (0.0005) [2023-12-26 19:46:47,766][105585] KL-divergence is very high: 185.7846 [2023-12-26 19:46:47,769][105692] Updated weights for policy 0, policy_version 606033 (0.0007) [2023-12-26 19:46:47,806][105585] KL-divergence is very high: 271.9783 [2023-12-26 19:46:47,817][105692] Updated weights for policy 0, policy_version 606043 (0.0005) [2023-12-26 19:46:47,855][105585] KL-divergence is very high: 197.9491 [2023-12-26 19:46:47,880][105692] Updated weights for policy 0, policy_version 606053 (0.0005) [2023-12-26 19:46:48,198][105620] Updated weights for policy 1, policy_version 606987 (0.0006) [2023-12-26 19:46:48,252][105620] Updated weights for policy 1, policy_version 606997 (0.0008) [2023-12-26 19:46:48,296][105620] Updated weights for policy 1, policy_version 607007 (0.0008) [2023-12-26 19:46:48,499][105692] Updated weights for policy 0, policy_version 606063 (0.0009) [2023-12-26 19:46:48,568][105692] Updated weights for policy 0, policy_version 606073 (0.0011) [2023-12-26 19:46:48,627][105692] Updated weights for policy 0, policy_version 606083 (0.0011) [2023-12-26 19:46:49,034][105620] Updated weights for policy 1, policy_version 607017 (0.0009) [2023-12-26 19:46:49,084][105620] Updated weights for policy 1, policy_version 607027 (0.0007) [2023-12-26 19:46:49,136][105620] Updated weights for policy 1, policy_version 607037 (0.0008) [2023-12-26 19:46:49,188][105620] Updated weights for policy 1, policy_version 607047 (0.0008) [2023-12-26 19:46:49,285][105692] Updated weights for policy 0, policy_version 606093 (0.0009) [2023-12-26 19:46:49,349][105692] Updated weights for policy 0, policy_version 606103 (0.0009) [2023-12-26 19:46:49,413][105692] Updated weights for policy 0, policy_version 606113 (0.0010) [2023-12-26 19:46:49,985][105620] Updated weights for policy 1, policy_version 607057 (0.0008) [2023-12-26 19:46:50,048][105620] Updated weights for policy 1, policy_version 607067 (0.0008) [2023-12-26 19:46:50,119][105620] Updated weights for policy 1, policy_version 607077 (0.0008) [2023-12-26 19:46:50,168][105692] Updated weights for policy 0, policy_version 606123 (0.0011) [2023-12-26 19:46:50,223][105692] Updated weights for policy 0, policy_version 606133 (0.0010) [2023-12-26 19:46:50,284][105692] Updated weights for policy 0, policy_version 606143 (0.0010) [2023-12-26 19:46:50,868][105620] Updated weights for policy 1, policy_version 607087 (0.0008) [2023-12-26 19:46:50,913][105620] Updated weights for policy 1, policy_version 607097 (0.0008) [2023-12-26 19:46:50,963][105620] Updated weights for policy 1, policy_version 607107 (0.0008) [2023-12-26 19:46:51,054][105692] Updated weights for policy 0, policy_version 606153 (0.0010) [2023-12-26 19:46:51,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19797.3, 300 sec: 19521.9). Total num frames: 310632448. Throughput: 0: 9846.7, 1: 9831.7. Samples: 310621916. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 19:46:51,063][104569] Avg episode reward: [(0, '9171.690'), (1, '9079.570')] [2023-12-26 19:46:51,120][105692] Updated weights for policy 0, policy_version 606163 (0.0010) [2023-12-26 19:46:51,187][105692] Updated weights for policy 0, policy_version 606173 (0.0007) [2023-12-26 19:46:51,258][105692] Updated weights for policy 0, policy_version 606183 (0.0008) [2023-12-26 19:46:51,730][105620] Updated weights for policy 1, policy_version 607117 (0.0009) [2023-12-26 19:46:51,794][105620] Updated weights for policy 1, policy_version 607127 (0.0006) [2023-12-26 19:46:51,848][105620] Updated weights for policy 1, policy_version 607137 (0.0010) [2023-12-26 19:46:51,952][105692] Updated weights for policy 0, policy_version 606193 (0.0007) [2023-12-26 19:46:52,021][105692] Updated weights for policy 0, policy_version 606203 (0.0008) [2023-12-26 19:46:52,090][105692] Updated weights for policy 0, policy_version 606213 (0.0009) [2023-12-26 19:46:52,578][105620] Updated weights for policy 1, policy_version 607147 (0.0011) [2023-12-26 19:46:52,627][105620] Updated weights for policy 1, policy_version 607157 (0.0010) [2023-12-26 19:46:52,676][105620] Updated weights for policy 1, policy_version 607167 (0.0010) [2023-12-26 19:46:52,760][105692] Updated weights for policy 0, policy_version 606223 (0.0008) [2023-12-26 19:46:52,813][105692] Updated weights for policy 0, policy_version 606233 (0.0008) [2023-12-26 19:46:52,872][105692] Updated weights for policy 0, policy_version 606243 (0.0009) [2023-12-26 19:46:53,399][105620] Updated weights for policy 1, policy_version 607177 (0.0010) [2023-12-26 19:46:53,453][105620] Updated weights for policy 1, policy_version 607187 (0.0007) [2023-12-26 19:46:53,516][105620] Updated weights for policy 1, policy_version 607197 (0.0007) [2023-12-26 19:46:53,551][105692] Updated weights for policy 0, policy_version 606253 (0.0008) [2023-12-26 19:46:53,573][105620] Updated weights for policy 1, policy_version 607207 (0.0005) [2023-12-26 19:46:53,605][105692] Updated weights for policy 0, policy_version 606263 (0.0009) [2023-12-26 19:46:53,657][105692] Updated weights for policy 0, policy_version 606273 (0.0008) [2023-12-26 19:46:54,135][105620] Updated weights for policy 1, policy_version 607217 (0.0005) [2023-12-26 19:46:54,195][105620] Updated weights for policy 1, policy_version 607227 (0.0005) [2023-12-26 19:46:54,258][105620] Updated weights for policy 1, policy_version 607237 (0.0008) [2023-12-26 19:46:54,292][105692] Updated weights for policy 0, policy_version 606283 (0.0005) [2023-12-26 19:46:54,346][105692] Updated weights for policy 0, policy_version 606293 (0.0007) [2023-12-26 19:46:54,397][105692] Updated weights for policy 0, policy_version 606303 (0.0009) [2023-12-26 19:46:54,966][105620] Updated weights for policy 1, policy_version 607247 (0.0010) [2023-12-26 19:46:55,032][105620] Updated weights for policy 1, policy_version 607257 (0.0008) [2023-12-26 19:46:55,090][105620] Updated weights for policy 1, policy_version 607267 (0.0007) [2023-12-26 19:46:55,122][105692] Updated weights for policy 0, policy_version 606313 (0.0009) [2023-12-26 19:46:55,176][105692] Updated weights for policy 0, policy_version 606323 (0.0009) [2023-12-26 19:46:55,231][105692] Updated weights for policy 0, policy_version 606333 (0.0009) [2023-12-26 19:46:55,277][105692] Updated weights for policy 0, policy_version 606343 (0.0008) [2023-12-26 19:46:55,826][105620] Updated weights for policy 1, policy_version 607277 (0.0009) [2023-12-26 19:46:55,879][105620] Updated weights for policy 1, policy_version 607287 (0.0009) [2023-12-26 19:46:55,929][105620] Updated weights for policy 1, policy_version 607297 (0.0008) [2023-12-26 19:46:56,045][105692] Updated weights for policy 0, policy_version 606353 (0.0009) [2023-12-26 19:46:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 310730752. Throughput: 0: 9795.8, 1: 9882.6. Samples: 310739168. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 19:46:56,063][104569] Avg episode reward: [(0, '9080.395'), (1, '8989.434')] [2023-12-26 19:46:56,078][105585] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000007 [2023-12-26 19:46:56,723][105620] Updated weights for policy 1, policy_version 607307 (0.0009) [2023-12-26 19:46:56,781][105620] Updated weights for policy 1, policy_version 607317 (0.0009) [2023-12-26 19:46:56,821][105692] Updated weights for policy 0, policy_version 606363 (0.0007) [2023-12-26 19:46:56,832][105620] Updated weights for policy 1, policy_version 607327 (0.0008) [2023-12-26 19:46:56,870][105692] Updated weights for policy 0, policy_version 606373 (0.0007) [2023-12-26 19:46:56,927][105692] Updated weights for policy 0, policy_version 606383 (0.0009) [2023-12-26 19:46:57,477][105620] Updated weights for policy 1, policy_version 607337 (0.0008) [2023-12-26 19:46:57,538][105620] Updated weights for policy 1, policy_version 607347 (0.0009) [2023-12-26 19:46:57,599][105620] Updated weights for policy 1, policy_version 607357 (0.0009) [2023-12-26 19:46:57,660][105620] Updated weights for policy 1, policy_version 607367 (0.0009) [2023-12-26 19:46:57,723][105692] Updated weights for policy 0, policy_version 606393 (0.0010) [2023-12-26 19:46:57,780][105692] Updated weights for policy 0, policy_version 606403 (0.0009) [2023-12-26 19:46:57,837][105692] Updated weights for policy 0, policy_version 606413 (0.0009) [2023-12-26 19:46:57,891][105692] Updated weights for policy 0, policy_version 606423 (0.0009) [2023-12-26 19:46:58,445][105620] Updated weights for policy 1, policy_version 607377 (0.0006) [2023-12-26 19:46:58,511][105620] Updated weights for policy 1, policy_version 607387 (0.0006) [2023-12-26 19:46:58,577][105620] Updated weights for policy 1, policy_version 607397 (0.0006) [2023-12-26 19:46:58,643][105692] Updated weights for policy 0, policy_version 606433 (0.0010) [2023-12-26 19:46:58,702][105692] Updated weights for policy 0, policy_version 606443 (0.0011) [2023-12-26 19:46:58,768][105692] Updated weights for policy 0, policy_version 606453 (0.0009) [2023-12-26 19:46:59,291][105620] Updated weights for policy 1, policy_version 607407 (0.0007) [2023-12-26 19:46:59,356][105620] Updated weights for policy 1, policy_version 607417 (0.0008) [2023-12-26 19:46:59,418][105620] Updated weights for policy 1, policy_version 607427 (0.0010) [2023-12-26 19:46:59,526][105692] Updated weights for policy 0, policy_version 606463 (0.0010) [2023-12-26 19:46:59,585][105692] Updated weights for policy 0, policy_version 606473 (0.0010) [2023-12-26 19:46:59,646][105692] Updated weights for policy 0, policy_version 606483 (0.0010) [2023-12-26 19:47:00,202][105620] Updated weights for policy 1, policy_version 607437 (0.0008) [2023-12-26 19:47:00,255][105620] Updated weights for policy 1, policy_version 607447 (0.0008) [2023-12-26 19:47:00,316][105620] Updated weights for policy 1, policy_version 607458 (0.0007) [2023-12-26 19:47:00,400][105692] Updated weights for policy 0, policy_version 606493 (0.0010) [2023-12-26 19:47:00,460][105692] Updated weights for policy 0, policy_version 606503 (0.0008) [2023-12-26 19:47:00,514][105692] Updated weights for policy 0, policy_version 606513 (0.0009) [2023-12-26 19:47:00,989][105620] Updated weights for policy 1, policy_version 607468 (0.0008) [2023-12-26 19:47:01,045][105620] Updated weights for policy 1, policy_version 607478 (0.0010) [2023-12-26 19:47:01,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 310820864. Throughput: 0: 9785.7, 1: 9825.5. Samples: 310796108. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 19:47:01,063][104569] Avg episode reward: [(0, '9079.980'), (1, '9082.381')] [2023-12-26 19:47:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000606520_155295744.pth... [2023-12-26 19:47:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000605416_155009024.pth [2023-12-26 19:47:01,099][105620] Updated weights for policy 1, policy_version 607488 (0.0009) [2023-12-26 19:47:01,135][105692] Updated weights for policy 0, policy_version 606523 (0.0008) [2023-12-26 19:47:01,150][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000607496_155533312.pth... [2023-12-26 19:47:01,156][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000606344_155238400.pth [2023-12-26 19:47:01,196][105692] Updated weights for policy 0, policy_version 606533 (0.0008) [2023-12-26 19:47:01,258][105692] Updated weights for policy 0, policy_version 606543 (0.0008) [2023-12-26 19:47:01,781][105620] Updated weights for policy 1, policy_version 607498 (0.0009) [2023-12-26 19:47:01,832][105620] Updated weights for policy 1, policy_version 607508 (0.0010) [2023-12-26 19:47:01,891][105620] Updated weights for policy 1, policy_version 607518 (0.0010) [2023-12-26 19:47:01,940][105620] Updated weights for policy 1, policy_version 607528 (0.0010) [2023-12-26 19:47:01,988][105692] Updated weights for policy 0, policy_version 606553 (0.0008) [2023-12-26 19:47:02,030][105692] Updated weights for policy 0, policy_version 606563 (0.0006) [2023-12-26 19:47:02,089][105692] Updated weights for policy 0, policy_version 606573 (0.0005) [2023-12-26 19:47:02,137][105692] Updated weights for policy 0, policy_version 606583 (0.0005) [2023-12-26 19:47:02,646][105620] Updated weights for policy 1, policy_version 607538 (0.0009) [2023-12-26 19:47:02,699][105620] Updated weights for policy 1, policy_version 607548 (0.0010) [2023-12-26 19:47:02,753][105620] Updated weights for policy 1, policy_version 607558 (0.0008) [2023-12-26 19:47:02,804][105692] Updated weights for policy 0, policy_version 606593 (0.0008) [2023-12-26 19:47:02,864][105692] Updated weights for policy 0, policy_version 606603 (0.0006) [2023-12-26 19:47:02,928][105692] Updated weights for policy 0, policy_version 606613 (0.0005) [2023-12-26 19:47:03,427][105620] Updated weights for policy 1, policy_version 607568 (0.0006) [2023-12-26 19:47:03,476][105620] Updated weights for policy 1, policy_version 607578 (0.0005) [2023-12-26 19:47:03,525][105620] Updated weights for policy 1, policy_version 607588 (0.0006) [2023-12-26 19:47:03,541][105692] Updated weights for policy 0, policy_version 606623 (0.0006) [2023-12-26 19:47:03,598][105692] Updated weights for policy 0, policy_version 606633 (0.0006) [2023-12-26 19:47:03,646][105692] Updated weights for policy 0, policy_version 606643 (0.0005) [2023-12-26 19:47:04,100][105620] Updated weights for policy 1, policy_version 607598 (0.0008) [2023-12-26 19:47:04,158][105620] Updated weights for policy 1, policy_version 607608 (0.0007) [2023-12-26 19:47:04,217][105620] Updated weights for policy 1, policy_version 607618 (0.0006) [2023-12-26 19:47:04,295][105692] Updated weights for policy 0, policy_version 606653 (0.0007) [2023-12-26 19:47:04,348][105692] Updated weights for policy 0, policy_version 606663 (0.0010) [2023-12-26 19:47:04,409][105692] Updated weights for policy 0, policy_version 606673 (0.0010) [2023-12-26 19:47:04,768][105620] Updated weights for policy 1, policy_version 607629 (0.0009) [2023-12-26 19:47:04,820][105620] Updated weights for policy 1, policy_version 607640 (0.0010) [2023-12-26 19:47:04,871][105620] Updated weights for policy 1, policy_version 607650 (0.0010) [2023-12-26 19:47:05,127][105692] Updated weights for policy 0, policy_version 606683 (0.0009) [2023-12-26 19:47:05,173][105692] Updated weights for policy 0, policy_version 606693 (0.0008) [2023-12-26 19:47:05,221][105692] Updated weights for policy 0, policy_version 606703 (0.0009) [2023-12-26 19:47:05,602][105620] Updated weights for policy 1, policy_version 607660 (0.0008) [2023-12-26 19:47:05,653][105620] Updated weights for policy 1, policy_version 607670 (0.0009) [2023-12-26 19:47:05,709][105620] Updated weights for policy 1, policy_version 607680 (0.0007) [2023-12-26 19:47:06,004][105692] Updated weights for policy 0, policy_version 606713 (0.0009) [2023-12-26 19:47:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 310927360. Throughput: 0: 9672.1, 1: 9926.2. Samples: 310917452. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 19:47:06,062][104569] Avg episode reward: [(0, '9170.818'), (1, '9265.010')] [2023-12-26 19:47:06,063][105692] Updated weights for policy 0, policy_version 606723 (0.0008) [2023-12-26 19:47:06,125][105692] Updated weights for policy 0, policy_version 606733 (0.0007) [2023-12-26 19:47:06,189][105692] Updated weights for policy 0, policy_version 606743 (0.0007) [2023-12-26 19:47:06,487][105620] Updated weights for policy 1, policy_version 607690 (0.0006) [2023-12-26 19:47:06,550][105620] Updated weights for policy 1, policy_version 607700 (0.0008) [2023-12-26 19:47:06,616][105620] Updated weights for policy 1, policy_version 607710 (0.0009) [2023-12-26 19:47:06,675][105620] Updated weights for policy 1, policy_version 607720 (0.0009) [2023-12-26 19:47:06,909][105692] Updated weights for policy 0, policy_version 606753 (0.0006) [2023-12-26 19:47:06,968][105692] Updated weights for policy 0, policy_version 606763 (0.0006) [2023-12-26 19:47:07,022][105692] Updated weights for policy 0, policy_version 606773 (0.0008) [2023-12-26 19:47:07,495][105620] Updated weights for policy 1, policy_version 607730 (0.0009) [2023-12-26 19:47:07,550][105620] Updated weights for policy 1, policy_version 607740 (0.0010) [2023-12-26 19:47:07,608][105620] Updated weights for policy 1, policy_version 607750 (0.0008) [2023-12-26 19:47:07,614][105692] Updated weights for policy 0, policy_version 606783 (0.0006) [2023-12-26 19:47:07,676][105692] Updated weights for policy 0, policy_version 606793 (0.0005) [2023-12-26 19:47:07,739][105692] Updated weights for policy 0, policy_version 606803 (0.0007) [2023-12-26 19:47:08,404][105620] Updated weights for policy 1, policy_version 607760 (0.0009) [2023-12-26 19:47:08,431][105692] Updated weights for policy 0, policy_version 606813 (0.0008) [2023-12-26 19:47:08,459][105620] Updated weights for policy 1, policy_version 607770 (0.0008) [2023-12-26 19:47:08,488][105692] Updated weights for policy 0, policy_version 606823 (0.0006) [2023-12-26 19:47:08,504][105620] Updated weights for policy 1, policy_version 607780 (0.0007) [2023-12-26 19:47:08,549][105692] Updated weights for policy 0, policy_version 606833 (0.0006) [2023-12-26 19:47:09,234][105692] Updated weights for policy 0, policy_version 606843 (0.0006) [2023-12-26 19:47:09,294][105692] Updated weights for policy 0, policy_version 606853 (0.0008) [2023-12-26 19:47:09,329][105620] Updated weights for policy 1, policy_version 607790 (0.0007) [2023-12-26 19:47:09,363][105692] Updated weights for policy 0, policy_version 606863 (0.0009) [2023-12-26 19:47:09,398][105620] Updated weights for policy 1, policy_version 607800 (0.0009) [2023-12-26 19:47:09,469][105620] Updated weights for policy 1, policy_version 607810 (0.0008) [2023-12-26 19:47:10,156][105692] Updated weights for policy 0, policy_version 606873 (0.0008) [2023-12-26 19:47:10,197][105620] Updated weights for policy 1, policy_version 607820 (0.0008) [2023-12-26 19:47:10,216][105692] Updated weights for policy 0, policy_version 606883 (0.0007) [2023-12-26 19:47:10,258][105620] Updated weights for policy 1, policy_version 607830 (0.0008) [2023-12-26 19:47:10,273][105692] Updated weights for policy 0, policy_version 606893 (0.0006) [2023-12-26 19:47:10,317][105620] Updated weights for policy 1, policy_version 607840 (0.0009) [2023-12-26 19:47:10,324][105692] Updated weights for policy 0, policy_version 606903 (0.0006) [2023-12-26 19:47:11,039][105620] Updated weights for policy 1, policy_version 607850 (0.0008) [2023-12-26 19:47:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 311017472. Throughput: 0: 9713.2, 1: 9801.3. Samples: 311031864. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 19:47:11,063][104569] Avg episode reward: [(0, '9171.073'), (1, '9083.347')] [2023-12-26 19:47:11,096][105692] Updated weights for policy 0, policy_version 606913 (0.0006) [2023-12-26 19:47:11,103][105620] Updated weights for policy 1, policy_version 607860 (0.0009) [2023-12-26 19:47:11,160][105692] Updated weights for policy 0, policy_version 606923 (0.0008) [2023-12-26 19:47:11,171][105620] Updated weights for policy 1, policy_version 607870 (0.0008) [2023-12-26 19:47:11,217][105692] Updated weights for policy 0, policy_version 606933 (0.0005) [2023-12-26 19:47:11,219][105620] Updated weights for policy 1, policy_version 607880 (0.0008) [2023-12-26 19:47:11,920][105692] Updated weights for policy 0, policy_version 606943 (0.0009) [2023-12-26 19:47:11,984][105692] Updated weights for policy 0, policy_version 606953 (0.0009) [2023-12-26 19:47:12,006][105620] Updated weights for policy 1, policy_version 607890 (0.0006) [2023-12-26 19:47:12,045][105692] Updated weights for policy 0, policy_version 606963 (0.0006) [2023-12-26 19:47:12,068][105620] Updated weights for policy 1, policy_version 607900 (0.0007) [2023-12-26 19:47:12,127][105620] Updated weights for policy 1, policy_version 607910 (0.0010) [2023-12-26 19:47:12,835][105620] Updated weights for policy 1, policy_version 607920 (0.0009) [2023-12-26 19:47:12,835][105692] Updated weights for policy 0, policy_version 606973 (0.0008) [2023-12-26 19:47:12,887][105692] Updated weights for policy 0, policy_version 606983 (0.0010) [2023-12-26 19:47:12,893][105620] Updated weights for policy 1, policy_version 607930 (0.0008) [2023-12-26 19:47:12,936][105692] Updated weights for policy 0, policy_version 606993 (0.0010) [2023-12-26 19:47:12,953][105620] Updated weights for policy 1, policy_version 607940 (0.0008) [2023-12-26 19:47:13,661][105620] Updated weights for policy 1, policy_version 607950 (0.0005) [2023-12-26 19:47:13,697][105692] Updated weights for policy 0, policy_version 607003 (0.0010) [2023-12-26 19:47:13,708][105620] Updated weights for policy 1, policy_version 607960 (0.0008) [2023-12-26 19:47:13,746][105692] Updated weights for policy 0, policy_version 607013 (0.0007) [2023-12-26 19:47:13,753][105620] Updated weights for policy 1, policy_version 607970 (0.0006) [2023-12-26 19:47:13,796][105692] Updated weights for policy 0, policy_version 607023 (0.0006) [2023-12-26 19:47:14,484][105620] Updated weights for policy 1, policy_version 607980 (0.0006) [2023-12-26 19:47:14,552][105620] Updated weights for policy 1, policy_version 607990 (0.0006) [2023-12-26 19:47:14,573][105692] Updated weights for policy 0, policy_version 607033 (0.0009) [2023-12-26 19:47:14,616][105620] Updated weights for policy 1, policy_version 608000 (0.0008) [2023-12-26 19:47:14,631][105692] Updated weights for policy 0, policy_version 607043 (0.0010) [2023-12-26 19:47:14,692][105692] Updated weights for policy 0, policy_version 607053 (0.0010) [2023-12-26 19:47:14,746][105692] Updated weights for policy 0, policy_version 607063 (0.0010) [2023-12-26 19:47:15,305][105620] Updated weights for policy 1, policy_version 608010 (0.0008) [2023-12-26 19:47:15,361][105620] Updated weights for policy 1, policy_version 608020 (0.0009) [2023-12-26 19:47:15,425][105620] Updated weights for policy 1, policy_version 608030 (0.0009) [2023-12-26 19:47:15,444][105692] Updated weights for policy 0, policy_version 607073 (0.0007) [2023-12-26 19:47:15,489][105620] Updated weights for policy 1, policy_version 608040 (0.0009) [2023-12-26 19:47:15,505][105692] Updated weights for policy 0, policy_version 607083 (0.0006) [2023-12-26 19:47:15,569][105692] Updated weights for policy 0, policy_version 607093 (0.0007) [2023-12-26 19:47:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 311115776. Throughput: 0: 9632.4, 1: 9758.8. Samples: 311088420. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 19:47:16,063][104569] Avg episode reward: [(0, '9172.667'), (1, '9082.987')] [2023-12-26 19:47:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000608040_155672576.pth... [2023-12-26 19:47:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000607096_155443200.pth... [2023-12-26 19:47:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000606920_155385856.pth [2023-12-26 19:47:16,080][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000605960_155148288.pth [2023-12-26 19:47:16,137][105692] Updated weights for policy 0, policy_version 607103 (0.0005) [2023-12-26 19:47:16,185][105692] Updated weights for policy 0, policy_version 607113 (0.0005) [2023-12-26 19:47:16,200][105620] Updated weights for policy 1, policy_version 608050 (0.0009) [2023-12-26 19:47:16,243][105692] Updated weights for policy 0, policy_version 607123 (0.0008) [2023-12-26 19:47:16,253][105620] Updated weights for policy 1, policy_version 608060 (0.0007) [2023-12-26 19:47:16,307][105620] Updated weights for policy 1, policy_version 608070 (0.0007) [2023-12-26 19:47:16,804][105692] Updated weights for policy 0, policy_version 607133 (0.0008) [2023-12-26 19:47:16,860][105692] Updated weights for policy 0, policy_version 607143 (0.0005) [2023-12-26 19:47:16,913][105692] Updated weights for policy 0, policy_version 607153 (0.0008) [2023-12-26 19:47:17,122][105620] Updated weights for policy 1, policy_version 608080 (0.0006) [2023-12-26 19:47:17,175][105620] Updated weights for policy 1, policy_version 608090 (0.0006) [2023-12-26 19:47:17,236][105620] Updated weights for policy 1, policy_version 608100 (0.0008) [2023-12-26 19:47:17,600][105692] Updated weights for policy 0, policy_version 607163 (0.0009) [2023-12-26 19:47:17,657][105692] Updated weights for policy 0, policy_version 607173 (0.0009) [2023-12-26 19:47:17,712][105692] Updated weights for policy 0, policy_version 607183 (0.0010) [2023-12-26 19:47:17,997][105620] Updated weights for policy 1, policy_version 608110 (0.0008) [2023-12-26 19:47:18,047][105620] Updated weights for policy 1, policy_version 608120 (0.0007) [2023-12-26 19:47:18,096][105620] Updated weights for policy 1, policy_version 608130 (0.0008) [2023-12-26 19:47:18,407][105692] Updated weights for policy 0, policy_version 607193 (0.0008) [2023-12-26 19:47:18,470][105692] Updated weights for policy 0, policy_version 607203 (0.0008) [2023-12-26 19:47:18,532][105692] Updated weights for policy 0, policy_version 607213 (0.0008) [2023-12-26 19:47:18,594][105692] Updated weights for policy 0, policy_version 607223 (0.0007) [2023-12-26 19:47:18,815][105620] Updated weights for policy 1, policy_version 608140 (0.0009) [2023-12-26 19:47:18,870][105620] Updated weights for policy 1, policy_version 608150 (0.0010) [2023-12-26 19:47:18,933][105620] Updated weights for policy 1, policy_version 608160 (0.0008) [2023-12-26 19:47:19,272][105692] Updated weights for policy 0, policy_version 607233 (0.0009) [2023-12-26 19:47:19,322][105692] Updated weights for policy 0, policy_version 607243 (0.0009) [2023-12-26 19:47:19,384][105692] Updated weights for policy 0, policy_version 607253 (0.0008) [2023-12-26 19:47:19,703][105620] Updated weights for policy 1, policy_version 608170 (0.0009) [2023-12-26 19:47:19,760][105620] Updated weights for policy 1, policy_version 608180 (0.0011) [2023-12-26 19:47:19,824][105620] Updated weights for policy 1, policy_version 608190 (0.0011) [2023-12-26 19:47:19,894][105620] Updated weights for policy 1, policy_version 608200 (0.0008) [2023-12-26 19:47:20,117][105692] Updated weights for policy 0, policy_version 607263 (0.0009) [2023-12-26 19:47:20,185][105692] Updated weights for policy 0, policy_version 607273 (0.0006) [2023-12-26 19:47:20,249][105692] Updated weights for policy 0, policy_version 607283 (0.0006) [2023-12-26 19:47:20,717][105620] Updated weights for policy 1, policy_version 608210 (0.0009) [2023-12-26 19:47:20,769][105620] Updated weights for policy 1, policy_version 608220 (0.0010) [2023-12-26 19:47:20,801][105692] Updated weights for policy 0, policy_version 607293 (0.0005) [2023-12-26 19:47:20,821][105620] Updated weights for policy 1, policy_version 608230 (0.0008) [2023-12-26 19:47:20,861][105692] Updated weights for policy 0, policy_version 607303 (0.0005) [2023-12-26 19:47:20,919][105692] Updated weights for policy 0, policy_version 607313 (0.0007) [2023-12-26 19:47:21,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 311222272. Throughput: 0: 9721.2, 1: 9669.6. Samples: 311206096. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 19:47:21,062][104569] Avg episode reward: [(0, '9172.690'), (1, '9091.239')] [2023-12-26 19:47:21,649][105620] Updated weights for policy 1, policy_version 608240 (0.0008) [2023-12-26 19:47:21,684][105692] Updated weights for policy 0, policy_version 607323 (0.0006) [2023-12-26 19:47:21,709][105620] Updated weights for policy 1, policy_version 608250 (0.0006) [2023-12-26 19:47:21,749][105692] Updated weights for policy 0, policy_version 607333 (0.0009) [2023-12-26 19:47:21,776][105620] Updated weights for policy 1, policy_version 608260 (0.0008) [2023-12-26 19:47:21,811][105692] Updated weights for policy 0, policy_version 607343 (0.0009) [2023-12-26 19:47:22,507][105692] Updated weights for policy 0, policy_version 607353 (0.0008) [2023-12-26 19:47:22,535][105620] Updated weights for policy 1, policy_version 608270 (0.0009) [2023-12-26 19:47:22,559][105692] Updated weights for policy 0, policy_version 607363 (0.0009) [2023-12-26 19:47:22,594][105620] Updated weights for policy 1, policy_version 608280 (0.0007) [2023-12-26 19:47:22,617][105692] Updated weights for policy 0, policy_version 607373 (0.0006) [2023-12-26 19:47:22,655][105620] Updated weights for policy 1, policy_version 608290 (0.0008) [2023-12-26 19:47:22,678][105692] Updated weights for policy 0, policy_version 607383 (0.0006) [2023-12-26 19:47:23,354][105620] Updated weights for policy 1, policy_version 608300 (0.0006) [2023-12-26 19:47:23,424][105620] Updated weights for policy 1, policy_version 608310 (0.0009) [2023-12-26 19:47:23,482][105620] Updated weights for policy 1, policy_version 608320 (0.0006) [2023-12-26 19:47:23,492][105692] Updated weights for policy 0, policy_version 607393 (0.0008) [2023-12-26 19:47:23,554][105692] Updated weights for policy 0, policy_version 607403 (0.0007) [2023-12-26 19:47:23,612][105692] Updated weights for policy 0, policy_version 607413 (0.0009) [2023-12-26 19:47:24,225][105620] Updated weights for policy 1, policy_version 608330 (0.0008) [2023-12-26 19:47:24,246][105692] Updated weights for policy 0, policy_version 607423 (0.0007) [2023-12-26 19:47:24,288][105620] Updated weights for policy 1, policy_version 608340 (0.0009) [2023-12-26 19:47:24,302][105692] Updated weights for policy 0, policy_version 607433 (0.0005) [2023-12-26 19:47:24,348][105620] Updated weights for policy 1, policy_version 608350 (0.0007) [2023-12-26 19:47:24,368][105692] Updated weights for policy 0, policy_version 607443 (0.0006) [2023-12-26 19:47:24,396][105620] Updated weights for policy 1, policy_version 608360 (0.0009) [2023-12-26 19:47:24,918][105692] Updated weights for policy 0, policy_version 607453 (0.0008) [2023-12-26 19:47:24,963][105692] Updated weights for policy 0, policy_version 607463 (0.0010) [2023-12-26 19:47:25,018][105692] Updated weights for policy 0, policy_version 607473 (0.0010) [2023-12-26 19:47:25,265][105620] Updated weights for policy 1, policy_version 608370 (0.0010) [2023-12-26 19:47:25,332][105620] Updated weights for policy 1, policy_version 608380 (0.0010) [2023-12-26 19:47:25,397][105620] Updated weights for policy 1, policy_version 608390 (0.0010) [2023-12-26 19:47:25,600][105692] Updated weights for policy 0, policy_version 607483 (0.0007) [2023-12-26 19:47:25,651][105692] Updated weights for policy 0, policy_version 607493 (0.0005) [2023-12-26 19:47:25,707][105692] Updated weights for policy 0, policy_version 607503 (0.0005) [2023-12-26 19:47:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 311312384. Throughput: 0: 9861.2, 1: 9568.8. Samples: 311321484. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 19:47:26,062][104569] Avg episode reward: [(0, '9263.985'), (1, '9183.363')] [2023-12-26 19:47:26,232][105620] Updated weights for policy 1, policy_version 608400 (0.0009) [2023-12-26 19:47:26,273][105692] Updated weights for policy 0, policy_version 607513 (0.0006) [2023-12-26 19:47:26,291][105620] Updated weights for policy 1, policy_version 608410 (0.0006) [2023-12-26 19:47:26,335][105692] Updated weights for policy 0, policy_version 607523 (0.0010) [2023-12-26 19:47:26,349][105620] Updated weights for policy 1, policy_version 608420 (0.0005) [2023-12-26 19:47:26,400][105692] Updated weights for policy 0, policy_version 607533 (0.0010) [2023-12-26 19:47:26,466][105692] Updated weights for policy 0, policy_version 607543 (0.0011) [2023-12-26 19:47:27,034][105620] Updated weights for policy 1, policy_version 608430 (0.0007) [2023-12-26 19:47:27,080][105620] Updated weights for policy 1, policy_version 608440 (0.0009) [2023-12-26 19:47:27,128][105620] Updated weights for policy 1, policy_version 608450 (0.0008) [2023-12-26 19:47:27,163][105692] Updated weights for policy 0, policy_version 607553 (0.0009) [2023-12-26 19:47:27,229][105692] Updated weights for policy 0, policy_version 607563 (0.0009) [2023-12-26 19:47:27,287][105692] Updated weights for policy 0, policy_version 607573 (0.0008) [2023-12-26 19:47:27,924][105620] Updated weights for policy 1, policy_version 608460 (0.0009) [2023-12-26 19:47:27,965][105692] Updated weights for policy 0, policy_version 607583 (0.0011) [2023-12-26 19:47:27,977][105620] Updated weights for policy 1, policy_version 608470 (0.0006) [2023-12-26 19:47:28,025][105692] Updated weights for policy 0, policy_version 607593 (0.0011) [2023-12-26 19:47:28,031][105620] Updated weights for policy 1, policy_version 608480 (0.0007) [2023-12-26 19:47:28,079][105692] Updated weights for policy 0, policy_version 607603 (0.0010) [2023-12-26 19:47:28,742][105692] Updated weights for policy 0, policy_version 607613 (0.0010) [2023-12-26 19:47:28,790][105692] Updated weights for policy 0, policy_version 607623 (0.0010) [2023-12-26 19:47:28,824][105620] Updated weights for policy 1, policy_version 608490 (0.0008) [2023-12-26 19:47:28,846][105692] Updated weights for policy 0, policy_version 607633 (0.0010) [2023-12-26 19:47:28,876][105620] Updated weights for policy 1, policy_version 608500 (0.0007) [2023-12-26 19:47:28,928][105620] Updated weights for policy 1, policy_version 608510 (0.0008) [2023-12-26 19:47:28,976][105620] Updated weights for policy 1, policy_version 608520 (0.0008) [2023-12-26 19:47:29,622][105692] Updated weights for policy 0, policy_version 607643 (0.0010) [2023-12-26 19:47:29,666][105692] Updated weights for policy 0, policy_version 607653 (0.0010) [2023-12-26 19:47:29,714][105692] Updated weights for policy 0, policy_version 607663 (0.0010) [2023-12-26 19:47:29,751][105620] Updated weights for policy 1, policy_version 608530 (0.0007) [2023-12-26 19:47:29,798][105620] Updated weights for policy 1, policy_version 608540 (0.0007) [2023-12-26 19:47:29,853][105620] Updated weights for policy 1, policy_version 608550 (0.0009) [2023-12-26 19:47:30,378][105692] Updated weights for policy 0, policy_version 607673 (0.0010) [2023-12-26 19:47:30,447][105692] Updated weights for policy 0, policy_version 607683 (0.0005) [2023-12-26 19:47:30,506][105692] Updated weights for policy 0, policy_version 607693 (0.0005) [2023-12-26 19:47:30,560][105692] Updated weights for policy 0, policy_version 607703 (0.0009) [2023-12-26 19:47:30,685][105620] Updated weights for policy 1, policy_version 608560 (0.0006) [2023-12-26 19:47:30,744][105620] Updated weights for policy 1, policy_version 608570 (0.0005) [2023-12-26 19:47:30,799][105620] Updated weights for policy 1, policy_version 608580 (0.0005) [2023-12-26 19:47:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 311410688. Throughput: 0: 9949.4, 1: 9530.9. Samples: 311379732. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 19:47:31,062][104569] Avg episode reward: [(0, '9354.899'), (1, '9082.569')] [2023-12-26 19:47:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000607704_155598848.pth... [2023-12-26 19:47:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000608584_155811840.pth... [2023-12-26 19:47:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000607496_155533312.pth [2023-12-26 19:47:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000606520_155295744.pth [2023-12-26 19:47:31,210][105692] Updated weights for policy 0, policy_version 607713 (0.0009) [2023-12-26 19:47:31,275][105692] Updated weights for policy 0, policy_version 607723 (0.0009) [2023-12-26 19:47:31,329][105692] Updated weights for policy 0, policy_version 607733 (0.0008) [2023-12-26 19:47:31,484][105620] Updated weights for policy 1, policy_version 608590 (0.0008) [2023-12-26 19:47:31,543][105620] Updated weights for policy 1, policy_version 608600 (0.0009) [2023-12-26 19:47:31,606][105620] Updated weights for policy 1, policy_version 608610 (0.0010) [2023-12-26 19:47:32,047][105692] Updated weights for policy 0, policy_version 607743 (0.0008) [2023-12-26 19:47:32,095][105692] Updated weights for policy 0, policy_version 607753 (0.0009) [2023-12-26 19:47:32,146][105692] Updated weights for policy 0, policy_version 607763 (0.0008) [2023-12-26 19:47:32,405][105620] Updated weights for policy 1, policy_version 608620 (0.0009) [2023-12-26 19:47:32,452][105620] Updated weights for policy 1, policy_version 608630 (0.0008) [2023-12-26 19:47:32,511][105620] Updated weights for policy 1, policy_version 608640 (0.0006) [2023-12-26 19:47:32,884][105692] Updated weights for policy 0, policy_version 607773 (0.0008) [2023-12-26 19:47:32,930][105692] Updated weights for policy 0, policy_version 607783 (0.0008) [2023-12-26 19:47:32,977][105692] Updated weights for policy 0, policy_version 607793 (0.0008) [2023-12-26 19:47:33,262][105620] Updated weights for policy 1, policy_version 608650 (0.0009) [2023-12-26 19:47:33,312][105620] Updated weights for policy 1, policy_version 608660 (0.0008) [2023-12-26 19:47:33,372][105620] Updated weights for policy 1, policy_version 608670 (0.0008) [2023-12-26 19:47:33,421][105620] Updated weights for policy 1, policy_version 608680 (0.0008) [2023-12-26 19:47:33,737][105692] Updated weights for policy 0, policy_version 607803 (0.0008) [2023-12-26 19:47:33,788][105692] Updated weights for policy 0, policy_version 607813 (0.0009) [2023-12-26 19:47:33,853][105692] Updated weights for policy 0, policy_version 607823 (0.0009) [2023-12-26 19:47:34,204][105620] Updated weights for policy 1, policy_version 608690 (0.0009) [2023-12-26 19:47:34,271][105620] Updated weights for policy 1, policy_version 608700 (0.0008) [2023-12-26 19:47:34,337][105620] Updated weights for policy 1, policy_version 608710 (0.0009) [2023-12-26 19:47:34,616][105692] Updated weights for policy 0, policy_version 607833 (0.0009) [2023-12-26 19:47:34,675][105692] Updated weights for policy 0, policy_version 607843 (0.0010) [2023-12-26 19:47:34,726][105692] Updated weights for policy 0, policy_version 607853 (0.0008) [2023-12-26 19:47:34,772][105692] Updated weights for policy 0, policy_version 607863 (0.0008) [2023-12-26 19:47:35,101][105620] Updated weights for policy 1, policy_version 608720 (0.0009) [2023-12-26 19:47:35,169][105620] Updated weights for policy 1, policy_version 608730 (0.0009) [2023-12-26 19:47:35,230][105620] Updated weights for policy 1, policy_version 608740 (0.0010) [2023-12-26 19:47:35,435][105692] Updated weights for policy 0, policy_version 607873 (0.0008) [2023-12-26 19:47:35,488][105692] Updated weights for policy 0, policy_version 607883 (0.0008) [2023-12-26 19:47:35,546][105692] Updated weights for policy 0, policy_version 607893 (0.0009) [2023-12-26 19:47:36,009][105620] Updated weights for policy 1, policy_version 608750 (0.0009) [2023-12-26 19:47:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 311500800. Throughput: 0: 9943.5, 1: 9440.0. Samples: 311494168. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 19:47:36,062][104569] Avg episode reward: [(0, '9354.804'), (1, '8991.639')] [2023-12-26 19:47:36,065][105620] Updated weights for policy 1, policy_version 608760 (0.0008) [2023-12-26 19:47:36,128][105620] Updated weights for policy 1, policy_version 608770 (0.0008) [2023-12-26 19:47:36,257][105692] Updated weights for policy 0, policy_version 607903 (0.0010) [2023-12-26 19:47:36,315][105692] Updated weights for policy 0, policy_version 607913 (0.0010) [2023-12-26 19:47:36,375][105692] Updated weights for policy 0, policy_version 607923 (0.0011) [2023-12-26 19:47:36,933][105620] Updated weights for policy 1, policy_version 608780 (0.0008) [2023-12-26 19:47:36,999][105620] Updated weights for policy 1, policy_version 608790 (0.0009) [2023-12-26 19:47:37,037][105692] Updated weights for policy 0, policy_version 607933 (0.0009) [2023-12-26 19:47:37,060][105620] Updated weights for policy 1, policy_version 608800 (0.0008) [2023-12-26 19:47:37,095][105692] Updated weights for policy 0, policy_version 607943 (0.0006) [2023-12-26 19:47:37,149][105692] Updated weights for policy 0, policy_version 607953 (0.0008) [2023-12-26 19:47:37,710][105620] Updated weights for policy 1, policy_version 608810 (0.0007) [2023-12-26 19:47:37,733][105692] Updated weights for policy 0, policy_version 607963 (0.0008) [2023-12-26 19:47:37,766][105620] Updated weights for policy 1, policy_version 608820 (0.0009) [2023-12-26 19:47:37,793][105692] Updated weights for policy 0, policy_version 607973 (0.0007) [2023-12-26 19:47:37,821][105620] Updated weights for policy 1, policy_version 608830 (0.0007) [2023-12-26 19:47:37,849][105692] Updated weights for policy 0, policy_version 607983 (0.0006) [2023-12-26 19:47:37,885][105620] Updated weights for policy 1, policy_version 608840 (0.0010) [2023-12-26 19:47:38,480][105692] Updated weights for policy 0, policy_version 607993 (0.0005) [2023-12-26 19:47:38,535][105692] Updated weights for policy 0, policy_version 608003 (0.0009) [2023-12-26 19:47:38,595][105692] Updated weights for policy 0, policy_version 608013 (0.0009) [2023-12-26 19:47:38,651][105692] Updated weights for policy 0, policy_version 608023 (0.0008) [2023-12-26 19:47:38,683][105620] Updated weights for policy 1, policy_version 608850 (0.0009) [2023-12-26 19:47:38,731][105620] Updated weights for policy 1, policy_version 608860 (0.0009) [2023-12-26 19:47:38,789][105620] Updated weights for policy 1, policy_version 608870 (0.0009) [2023-12-26 19:47:39,288][105692] Updated weights for policy 0, policy_version 608033 (0.0008) [2023-12-26 19:47:39,348][105692] Updated weights for policy 0, policy_version 608043 (0.0009) [2023-12-26 19:47:39,410][105585] KL-divergence is very high: 128.8227 [2023-12-26 19:47:39,411][105692] Updated weights for policy 0, policy_version 608053 (0.0008) [2023-12-26 19:47:39,629][105620] Updated weights for policy 1, policy_version 608880 (0.0008) [2023-12-26 19:47:39,685][105620] Updated weights for policy 1, policy_version 608890 (0.0009) [2023-12-26 19:47:39,735][105620] Updated weights for policy 1, policy_version 608900 (0.0008) [2023-12-26 19:47:40,118][105692] Updated weights for policy 0, policy_version 608063 (0.0006) [2023-12-26 19:47:40,167][105692] Updated weights for policy 0, policy_version 608073 (0.0005) [2023-12-26 19:47:40,181][105585] KL-divergence is very high: 101.0164 [2023-12-26 19:47:40,222][105692] Updated weights for policy 0, policy_version 608083 (0.0009) [2023-12-26 19:47:40,584][105620] Updated weights for policy 1, policy_version 608910 (0.0009) [2023-12-26 19:47:40,643][105620] Updated weights for policy 1, policy_version 608920 (0.0009) [2023-12-26 19:47:40,705][105620] Updated weights for policy 1, policy_version 608930 (0.0009) [2023-12-26 19:47:40,914][105692] Updated weights for policy 0, policy_version 608093 (0.0007) [2023-12-26 19:47:40,976][105692] Updated weights for policy 0, policy_version 608103 (0.0005) [2023-12-26 19:47:41,034][105692] Updated weights for policy 0, policy_version 608113 (0.0008) [2023-12-26 19:47:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 311599104. Throughput: 0: 10030.9, 1: 9318.2. Samples: 311609880. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 19:47:41,063][104569] Avg episode reward: [(0, '9173.521'), (1, '9265.350')] [2023-12-26 19:47:41,517][105620] Updated weights for policy 1, policy_version 608940 (0.0008) [2023-12-26 19:47:41,580][105620] Updated weights for policy 1, policy_version 608950 (0.0008) [2023-12-26 19:47:41,645][105620] Updated weights for policy 1, policy_version 608960 (0.0008) [2023-12-26 19:47:41,796][105692] Updated weights for policy 0, policy_version 608123 (0.0008) [2023-12-26 19:47:41,852][105692] Updated weights for policy 0, policy_version 608133 (0.0008) [2023-12-26 19:47:41,908][105692] Updated weights for policy 0, policy_version 608143 (0.0008) [2023-12-26 19:47:42,426][105620] Updated weights for policy 1, policy_version 608970 (0.0009) [2023-12-26 19:47:42,485][105620] Updated weights for policy 1, policy_version 608980 (0.0010) [2023-12-26 19:47:42,548][105620] Updated weights for policy 1, policy_version 608990 (0.0009) [2023-12-26 19:47:42,609][105620] Updated weights for policy 1, policy_version 609000 (0.0009) [2023-12-26 19:47:42,686][105692] Updated weights for policy 0, policy_version 608153 (0.0010) [2023-12-26 19:47:42,734][105692] Updated weights for policy 0, policy_version 608163 (0.0009) [2023-12-26 19:47:42,787][105692] Updated weights for policy 0, policy_version 608173 (0.0009) [2023-12-26 19:47:42,839][105692] Updated weights for policy 0, policy_version 608183 (0.0009) [2023-12-26 19:47:43,351][105620] Updated weights for policy 1, policy_version 609010 (0.0010) [2023-12-26 19:47:43,398][105620] Updated weights for policy 1, policy_version 609020 (0.0010) [2023-12-26 19:47:43,463][105620] Updated weights for policy 1, policy_version 609030 (0.0010) [2023-12-26 19:47:43,468][105692] Updated weights for policy 0, policy_version 608193 (0.0006) [2023-12-26 19:47:43,523][105692] Updated weights for policy 0, policy_version 608203 (0.0005) [2023-12-26 19:47:43,579][105692] Updated weights for policy 0, policy_version 608213 (0.0006) [2023-12-26 19:47:44,192][105692] Updated weights for policy 0, policy_version 608223 (0.0009) [2023-12-26 19:47:44,195][105620] Updated weights for policy 1, policy_version 609040 (0.0010) [2023-12-26 19:47:44,241][105692] Updated weights for policy 0, policy_version 608233 (0.0010) [2023-12-26 19:47:44,248][105620] Updated weights for policy 1, policy_version 609050 (0.0010) [2023-12-26 19:47:44,296][105620] Updated weights for policy 1, policy_version 609060 (0.0010) [2023-12-26 19:47:44,296][105692] Updated weights for policy 0, policy_version 608243 (0.0010) [2023-12-26 19:47:44,982][105692] Updated weights for policy 0, policy_version 608253 (0.0008) [2023-12-26 19:47:45,044][105692] Updated weights for policy 0, policy_version 608263 (0.0011) [2023-12-26 19:47:45,099][105620] Updated weights for policy 1, policy_version 609070 (0.0010) [2023-12-26 19:47:45,103][105692] Updated weights for policy 0, policy_version 608273 (0.0007) [2023-12-26 19:47:45,156][105620] Updated weights for policy 1, policy_version 609080 (0.0011) [2023-12-26 19:47:45,219][105620] Updated weights for policy 1, policy_version 609090 (0.0011) [2023-12-26 19:47:45,803][105692] Updated weights for policy 0, policy_version 608283 (0.0007) [2023-12-26 19:47:45,854][105692] Updated weights for policy 0, policy_version 608293 (0.0010) [2023-12-26 19:47:45,919][105692] Updated weights for policy 0, policy_version 608303 (0.0010) [2023-12-26 19:47:45,950][105620] Updated weights for policy 1, policy_version 609100 (0.0009) [2023-12-26 19:47:45,997][105620] Updated weights for policy 1, policy_version 609110 (0.0005) [2023-12-26 19:47:46,052][105620] Updated weights for policy 1, policy_version 609120 (0.0005) [2023-12-26 19:47:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 311697408. Throughput: 0: 10056.2, 1: 9303.8. Samples: 311667308. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 19:47:46,062][104569] Avg episode reward: [(0, '9172.968'), (1, '9265.628')] [2023-12-26 19:47:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000608312_155754496.pth... [2023-12-26 19:47:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000607096_155443200.pth [2023-12-26 19:47:46,102][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000609128_155951104.pth... [2023-12-26 19:47:46,107][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000608040_155672576.pth [2023-12-26 19:47:46,663][105692] Updated weights for policy 0, policy_version 608313 (0.0011) [2023-12-26 19:47:46,702][105620] Updated weights for policy 1, policy_version 609130 (0.0006) [2023-12-26 19:47:46,717][105692] Updated weights for policy 0, policy_version 608323 (0.0010) [2023-12-26 19:47:46,760][105620] Updated weights for policy 1, policy_version 609140 (0.0010) [2023-12-26 19:47:46,773][105692] Updated weights for policy 0, policy_version 608333 (0.0010) [2023-12-26 19:47:46,813][105620] Updated weights for policy 1, policy_version 609150 (0.0008) [2023-12-26 19:47:46,828][105692] Updated weights for policy 0, policy_version 608343 (0.0010) [2023-12-26 19:47:46,858][105620] Updated weights for policy 1, policy_version 609160 (0.0005) [2023-12-26 19:47:47,406][105620] Updated weights for policy 1, policy_version 609170 (0.0009) [2023-12-26 19:47:47,448][105692] Updated weights for policy 0, policy_version 608353 (0.0008) [2023-12-26 19:47:47,465][105620] Updated weights for policy 1, policy_version 609180 (0.0007) [2023-12-26 19:47:47,509][105620] Updated weights for policy 1, policy_version 609190 (0.0005) [2023-12-26 19:47:47,511][105692] Updated weights for policy 0, policy_version 608363 (0.0009) [2023-12-26 19:47:47,576][105692] Updated weights for policy 0, policy_version 608373 (0.0009) [2023-12-26 19:47:48,194][105620] Updated weights for policy 1, policy_version 609200 (0.0005) [2023-12-26 19:47:48,242][105620] Updated weights for policy 1, policy_version 609210 (0.0005) [2023-12-26 19:47:48,298][105692] Updated weights for policy 0, policy_version 608383 (0.0008) [2023-12-26 19:47:48,298][105620] Updated weights for policy 1, policy_version 609220 (0.0005) [2023-12-26 19:47:48,357][105692] Updated weights for policy 0, policy_version 608393 (0.0008) [2023-12-26 19:47:48,418][105692] Updated weights for policy 0, policy_version 608403 (0.0010) [2023-12-26 19:47:48,877][105620] Updated weights for policy 1, policy_version 609230 (0.0006) [2023-12-26 19:47:48,938][105620] Updated weights for policy 1, policy_version 609240 (0.0005) [2023-12-26 19:47:48,993][105620] Updated weights for policy 1, policy_version 609250 (0.0005) [2023-12-26 19:47:49,273][105692] Updated weights for policy 0, policy_version 608414 (0.0010) [2023-12-26 19:47:49,330][105692] Updated weights for policy 0, policy_version 608424 (0.0010) [2023-12-26 19:47:49,397][105692] Updated weights for policy 0, policy_version 608434 (0.0008) [2023-12-26 19:47:49,664][105620] Updated weights for policy 1, policy_version 609260 (0.0008) [2023-12-26 19:47:49,719][105620] Updated weights for policy 1, policy_version 609270 (0.0009) [2023-12-26 19:47:49,766][105620] Updated weights for policy 1, policy_version 609280 (0.0008) [2023-12-26 19:47:50,165][105692] Updated weights for policy 0, policy_version 608444 (0.0009) [2023-12-26 19:47:50,213][105692] Updated weights for policy 0, policy_version 608454 (0.0009) [2023-12-26 19:47:50,260][105692] Updated weights for policy 0, policy_version 608464 (0.0009) [2023-12-26 19:47:50,550][105620] Updated weights for policy 1, policy_version 609290 (0.0009) [2023-12-26 19:47:50,617][105620] Updated weights for policy 1, policy_version 609300 (0.0009) [2023-12-26 19:47:50,667][105620] Updated weights for policy 1, policy_version 609310 (0.0008) [2023-12-26 19:47:50,718][105620] Updated weights for policy 1, policy_version 609320 (0.0008) [2023-12-26 19:47:51,062][105692] Updated weights for policy 0, policy_version 608475 (0.0009) [2023-12-26 19:47:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 311795712. Throughput: 0: 10022.9, 1: 9310.8. Samples: 311787472. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 19:47:51,063][104569] Avg episode reward: [(0, '9353.680'), (1, '9265.490')] [2023-12-26 19:47:51,123][105692] Updated weights for policy 0, policy_version 608485 (0.0007) [2023-12-26 19:47:51,187][105692] Updated weights for policy 0, policy_version 608495 (0.0009) [2023-12-26 19:47:51,499][105620] Updated weights for policy 1, policy_version 609330 (0.0009) [2023-12-26 19:47:51,553][105620] Updated weights for policy 1, policy_version 609340 (0.0009) [2023-12-26 19:47:51,604][105620] Updated weights for policy 1, policy_version 609350 (0.0008) [2023-12-26 19:47:51,910][105692] Updated weights for policy 0, policy_version 608505 (0.0009) [2023-12-26 19:47:51,975][105692] Updated weights for policy 0, policy_version 608515 (0.0009) [2023-12-26 19:47:52,024][105692] Updated weights for policy 0, policy_version 608525 (0.0009) [2023-12-26 19:47:52,082][105692] Updated weights for policy 0, policy_version 608535 (0.0009) [2023-12-26 19:47:52,409][105620] Updated weights for policy 1, policy_version 609360 (0.0008) [2023-12-26 19:47:52,467][105620] Updated weights for policy 1, policy_version 609370 (0.0007) [2023-12-26 19:47:52,519][105620] Updated weights for policy 1, policy_version 609381 (0.0010) [2023-12-26 19:47:52,785][105692] Updated weights for policy 0, policy_version 608545 (0.0009) [2023-12-26 19:47:52,849][105692] Updated weights for policy 0, policy_version 608555 (0.0009) [2023-12-26 19:47:52,922][105692] Updated weights for policy 0, policy_version 608565 (0.0010) [2023-12-26 19:47:53,264][105620] Updated weights for policy 1, policy_version 609391 (0.0008) [2023-12-26 19:47:53,332][105620] Updated weights for policy 1, policy_version 609401 (0.0008) [2023-12-26 19:47:53,380][105620] Updated weights for policy 1, policy_version 609411 (0.0009) [2023-12-26 19:47:53,626][105692] Updated weights for policy 0, policy_version 608575 (0.0010) [2023-12-26 19:47:53,692][105692] Updated weights for policy 0, policy_version 608585 (0.0010) [2023-12-26 19:47:53,744][105692] Updated weights for policy 0, policy_version 608595 (0.0009) [2023-12-26 19:47:54,056][105620] Updated weights for policy 1, policy_version 609422 (0.0010) [2023-12-26 19:47:54,103][105620] Updated weights for policy 1, policy_version 609432 (0.0009) [2023-12-26 19:47:54,158][105620] Updated weights for policy 1, policy_version 609442 (0.0009) [2023-12-26 19:47:54,575][105692] Updated weights for policy 0, policy_version 608605 (0.0010) [2023-12-26 19:47:54,638][105692] Updated weights for policy 0, policy_version 608616 (0.0006) [2023-12-26 19:47:54,691][105692] Updated weights for policy 0, policy_version 608626 (0.0009) [2023-12-26 19:47:54,932][105620] Updated weights for policy 1, policy_version 609452 (0.0009) [2023-12-26 19:47:54,989][105620] Updated weights for policy 1, policy_version 609462 (0.0008) [2023-12-26 19:47:55,049][105620] Updated weights for policy 1, policy_version 609472 (0.0009) [2023-12-26 19:47:55,314][105692] Updated weights for policy 0, policy_version 608636 (0.0010) [2023-12-26 19:47:55,359][105692] Updated weights for policy 0, policy_version 608646 (0.0010) [2023-12-26 19:47:55,410][105692] Updated weights for policy 0, policy_version 608656 (0.0009) [2023-12-26 19:47:55,785][105620] Updated weights for policy 1, policy_version 609482 (0.0009) [2023-12-26 19:47:55,829][105620] Updated weights for policy 1, policy_version 609492 (0.0007) [2023-12-26 19:47:55,883][105620] Updated weights for policy 1, policy_version 609502 (0.0008) [2023-12-26 19:47:55,938][105620] Updated weights for policy 1, policy_version 609512 (0.0008) [2023-12-26 19:47:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 311894016. Throughput: 0: 9978.2, 1: 9340.8. Samples: 311901216. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 19:47:56,062][104569] Avg episode reward: [(0, '9353.811'), (1, '9266.217')] [2023-12-26 19:47:56,126][105692] Updated weights for policy 0, policy_version 608666 (0.0009) [2023-12-26 19:47:56,185][105692] Updated weights for policy 0, policy_version 608676 (0.0005) [2023-12-26 19:47:56,244][105692] Updated weights for policy 0, policy_version 608686 (0.0005) [2023-12-26 19:47:56,308][105692] Updated weights for policy 0, policy_version 608696 (0.0005) [2023-12-26 19:47:56,766][105620] Updated weights for policy 1, policy_version 609522 (0.0008) [2023-12-26 19:47:56,824][105620] Updated weights for policy 1, policy_version 609532 (0.0007) [2023-12-26 19:47:56,834][105692] Updated weights for policy 0, policy_version 608706 (0.0006) [2023-12-26 19:47:56,873][105620] Updated weights for policy 1, policy_version 609542 (0.0009) [2023-12-26 19:47:56,904][105692] Updated weights for policy 0, policy_version 608716 (0.0006) [2023-12-26 19:47:56,960][105692] Updated weights for policy 0, policy_version 608726 (0.0005) [2023-12-26 19:47:57,517][105692] Updated weights for policy 0, policy_version 608736 (0.0006) [2023-12-26 19:47:57,576][105620] Updated weights for policy 1, policy_version 609552 (0.0006) [2023-12-26 19:47:57,585][105692] Updated weights for policy 0, policy_version 608746 (0.0006) [2023-12-26 19:47:57,631][105620] Updated weights for policy 1, policy_version 609562 (0.0006) [2023-12-26 19:47:57,645][105692] Updated weights for policy 0, policy_version 608756 (0.0009) [2023-12-26 19:47:57,689][105620] Updated weights for policy 1, policy_version 609572 (0.0005) [2023-12-26 19:47:58,369][105692] Updated weights for policy 0, policy_version 608766 (0.0009) [2023-12-26 19:47:58,388][105620] Updated weights for policy 1, policy_version 609582 (0.0007) [2023-12-26 19:47:58,431][105692] Updated weights for policy 0, policy_version 608776 (0.0008) [2023-12-26 19:47:58,454][105620] Updated weights for policy 1, policy_version 609592 (0.0007) [2023-12-26 19:47:58,494][105692] Updated weights for policy 0, policy_version 608786 (0.0009) [2023-12-26 19:47:58,522][105620] Updated weights for policy 1, policy_version 609602 (0.0006) [2023-12-26 19:47:59,223][105620] Updated weights for policy 1, policy_version 609612 (0.0008) [2023-12-26 19:47:59,287][105620] Updated weights for policy 1, policy_version 609622 (0.0008) [2023-12-26 19:47:59,351][105620] Updated weights for policy 1, policy_version 609632 (0.0008) [2023-12-26 19:47:59,402][105692] Updated weights for policy 0, policy_version 608796 (0.0007) [2023-12-26 19:47:59,457][105692] Updated weights for policy 0, policy_version 608806 (0.0005) [2023-12-26 19:47:59,523][105692] Updated weights for policy 0, policy_version 608816 (0.0006) [2023-12-26 19:48:00,109][105620] Updated weights for policy 1, policy_version 609642 (0.0009) [2023-12-26 19:48:00,162][105620] Updated weights for policy 1, policy_version 609652 (0.0009) [2023-12-26 19:48:00,169][105692] Updated weights for policy 0, policy_version 608826 (0.0007) [2023-12-26 19:48:00,214][105620] Updated weights for policy 1, policy_version 609662 (0.0008) [2023-12-26 19:48:00,232][105692] Updated weights for policy 0, policy_version 608836 (0.0005) [2023-12-26 19:48:00,267][105620] Updated weights for policy 1, policy_version 609672 (0.0010) [2023-12-26 19:48:00,299][105692] Updated weights for policy 0, policy_version 608846 (0.0005) [2023-12-26 19:48:00,362][105692] Updated weights for policy 0, policy_version 608856 (0.0006) [2023-12-26 19:48:01,009][105620] Updated weights for policy 1, policy_version 609682 (0.0007) [2023-12-26 19:48:01,044][105692] Updated weights for policy 0, policy_version 608866 (0.0006) [2023-12-26 19:48:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 311984128. Throughput: 0: 10068.6, 1: 9327.2. Samples: 311961228. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 19:48:01,062][104569] Avg episode reward: [(0, '9353.903'), (1, '9265.331')] [2023-12-26 19:48:01,070][105620] Updated weights for policy 1, policy_version 609692 (0.0008) [2023-12-26 19:48:01,106][105692] Updated weights for policy 0, policy_version 608876 (0.0008) [2023-12-26 19:48:01,134][105620] Updated weights for policy 1, policy_version 609702 (0.0006) [2023-12-26 19:48:01,144][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000609704_156098560.pth... [2023-12-26 19:48:01,149][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000608584_155811840.pth [2023-12-26 19:48:01,169][105692] Updated weights for policy 0, policy_version 608886 (0.0009) [2023-12-26 19:48:01,181][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000608888_155901952.pth... [2023-12-26 19:48:01,186][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000607704_155598848.pth [2023-12-26 19:48:01,831][105620] Updated weights for policy 1, policy_version 609712 (0.0009) [2023-12-26 19:48:01,882][105620] Updated weights for policy 1, policy_version 609722 (0.0008) [2023-12-26 19:48:01,903][105692] Updated weights for policy 0, policy_version 608896 (0.0008) [2023-12-26 19:48:01,934][105585] KL-divergence is very high: 149.0299 [2023-12-26 19:48:01,936][105620] Updated weights for policy 1, policy_version 609732 (0.0009) [2023-12-26 19:48:01,959][105692] Updated weights for policy 0, policy_version 608906 (0.0007) [2023-12-26 19:48:02,013][105692] Updated weights for policy 0, policy_version 608916 (0.0009) [2023-12-26 19:48:02,019][105585] KL-divergence is very high: 148.6804 [2023-12-26 19:48:02,685][105620] Updated weights for policy 1, policy_version 609742 (0.0008) [2023-12-26 19:48:02,745][105585] KL-divergence is very high: 114.5307 [2023-12-26 19:48:02,745][105620] Updated weights for policy 1, policy_version 609752 (0.0008) [2023-12-26 19:48:02,762][105692] Updated weights for policy 0, policy_version 608926 (0.0010) [2023-12-26 19:48:02,770][105585] KL-divergence is very high: 143.4136 [2023-12-26 19:48:02,779][105585] KL-divergence is very high: 114.6588 [2023-12-26 19:48:02,801][105620] Updated weights for policy 1, policy_version 609762 (0.0007) [2023-12-26 19:48:02,815][105692] Updated weights for policy 0, policy_version 608936 (0.0008) [2023-12-26 19:48:02,815][105585] KL-divergence is very high: 122.0635 [2023-12-26 19:48:02,833][105585] KL-divergence is very high: 173.1922 [2023-12-26 19:48:02,839][105585] KL-divergence is very high: 102.3541 [2023-12-26 19:48:02,858][105585] KL-divergence is very high: 106.7511 [2023-12-26 19:48:02,867][105692] Updated weights for policy 0, policy_version 608946 (0.0005) [2023-12-26 19:48:02,868][105585] KL-divergence is very high: 139.8483 [2023-12-26 19:48:03,438][105585] KL-divergence is very high: 181.6490 [2023-12-26 19:48:03,448][105585] KL-divergence is very high: 150.9328 [2023-12-26 19:48:03,454][105585] KL-divergence is very high: 121.9964 [2023-12-26 19:48:03,459][105585] KL-divergence is very high: 133.3040 [2023-12-26 19:48:03,461][105692] Updated weights for policy 0, policy_version 608956 (0.0006) [2023-12-26 19:48:03,465][105585] KL-divergence is very high: 175.0994 [2023-12-26 19:48:03,514][105692] Updated weights for policy 0, policy_version 608966 (0.0006) [2023-12-26 19:48:03,553][105585] KL-divergence is very high: 102.9602 [2023-12-26 19:48:03,570][105692] Updated weights for policy 0, policy_version 608976 (0.0008) [2023-12-26 19:48:03,574][105585] KL-divergence is very high: 180.5341 [2023-12-26 19:48:03,580][105585] KL-divergence is very high: 114.8576 [2023-12-26 19:48:03,585][105620] Updated weights for policy 1, policy_version 609772 (0.0010) [2023-12-26 19:48:03,633][105620] Updated weights for policy 1, policy_version 609782 (0.0010) [2023-12-26 19:48:03,693][105620] Updated weights for policy 1, policy_version 609792 (0.0011) [2023-12-26 19:48:04,235][105692] Updated weights for policy 0, policy_version 608986 (0.0006) [2023-12-26 19:48:04,294][105692] Updated weights for policy 0, policy_version 608996 (0.0007) [2023-12-26 19:48:04,358][105692] Updated weights for policy 0, policy_version 609006 (0.0005) [2023-12-26 19:48:04,422][105692] Updated weights for policy 0, policy_version 609016 (0.0008) [2023-12-26 19:48:04,476][105620] Updated weights for policy 1, policy_version 609802 (0.0010) [2023-12-26 19:48:04,540][105620] Updated weights for policy 1, policy_version 609812 (0.0010) [2023-12-26 19:48:04,601][105620] Updated weights for policy 1, policy_version 609822 (0.0011) [2023-12-26 19:48:04,654][105620] Updated weights for policy 1, policy_version 609832 (0.0011) [2023-12-26 19:48:05,149][105692] Updated weights for policy 0, policy_version 609026 (0.0010) [2023-12-26 19:48:05,214][105692] Updated weights for policy 0, policy_version 609036 (0.0010) [2023-12-26 19:48:05,273][105692] Updated weights for policy 0, policy_version 609046 (0.0010) [2023-12-26 19:48:05,414][105620] Updated weights for policy 1, policy_version 609842 (0.0010) [2023-12-26 19:48:05,488][105620] Updated weights for policy 1, policy_version 609852 (0.0010) [2023-12-26 19:48:05,550][105620] Updated weights for policy 1, policy_version 609862 (0.0010) [2023-12-26 19:48:05,946][105692] Updated weights for policy 0, policy_version 609056 (0.0010) [2023-12-26 19:48:05,994][105692] Updated weights for policy 0, policy_version 609066 (0.0010) [2023-12-26 19:48:06,057][105692] Updated weights for policy 0, policy_version 609076 (0.0010) [2023-12-26 19:48:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 312082432. Throughput: 0: 10014.2, 1: 9312.6. Samples: 312075804. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 19:48:06,062][104569] Avg episode reward: [(0, '876.665'), (1, '9264.219')] [2023-12-26 19:48:06,224][105620] Updated weights for policy 1, policy_version 609872 (0.0009) [2023-12-26 19:48:06,286][105620] Updated weights for policy 1, policy_version 609882 (0.0008) [2023-12-26 19:48:06,351][105620] Updated weights for policy 1, policy_version 609892 (0.0009) [2023-12-26 19:48:06,742][105692] Updated weights for policy 0, policy_version 609086 (0.0007) [2023-12-26 19:48:06,802][105692] Updated weights for policy 0, policy_version 609096 (0.0008) [2023-12-26 19:48:06,865][105692] Updated weights for policy 0, policy_version 609106 (0.0011) [2023-12-26 19:48:07,215][105620] Updated weights for policy 1, policy_version 609902 (0.0010) [2023-12-26 19:48:07,273][105620] Updated weights for policy 1, policy_version 609912 (0.0008) [2023-12-26 19:48:07,341][105620] Updated weights for policy 1, policy_version 609922 (0.0009) [2023-12-26 19:48:07,511][105692] Updated weights for policy 0, policy_version 609116 (0.0010) [2023-12-26 19:48:07,565][105692] Updated weights for policy 0, policy_version 609126 (0.0007) [2023-12-26 19:48:07,611][105692] Updated weights for policy 0, policy_version 609136 (0.0010) [2023-12-26 19:48:08,124][105620] Updated weights for policy 1, policy_version 609932 (0.0009) [2023-12-26 19:48:08,180][105620] Updated weights for policy 1, policy_version 609942 (0.0008) [2023-12-26 19:48:08,235][105620] Updated weights for policy 1, policy_version 609952 (0.0007) [2023-12-26 19:48:08,371][105692] Updated weights for policy 0, policy_version 609146 (0.0010) [2023-12-26 19:48:08,428][105692] Updated weights for policy 0, policy_version 609156 (0.0010) [2023-12-26 19:48:08,474][105692] Updated weights for policy 0, policy_version 609166 (0.0010) [2023-12-26 19:48:08,526][105692] Updated weights for policy 0, policy_version 609176 (0.0010) [2023-12-26 19:48:09,012][105620] Updated weights for policy 1, policy_version 609962 (0.0008) [2023-12-26 19:48:09,068][105620] Updated weights for policy 1, policy_version 609972 (0.0008) [2023-12-26 19:48:09,116][105620] Updated weights for policy 1, policy_version 609982 (0.0008) [2023-12-26 19:48:09,172][105620] Updated weights for policy 1, policy_version 609992 (0.0008) [2023-12-26 19:48:09,310][105692] Updated weights for policy 0, policy_version 609186 (0.0011) [2023-12-26 19:48:09,377][105692] Updated weights for policy 0, policy_version 609196 (0.0010) [2023-12-26 19:48:09,445][105692] Updated weights for policy 0, policy_version 609206 (0.0008) [2023-12-26 19:48:09,968][105620] Updated weights for policy 1, policy_version 610002 (0.0009) [2023-12-26 19:48:10,033][105620] Updated weights for policy 1, policy_version 610012 (0.0009) [2023-12-26 19:48:10,090][105620] Updated weights for policy 1, policy_version 610022 (0.0009) [2023-12-26 19:48:10,175][105692] Updated weights for policy 0, policy_version 609216 (0.0009) [2023-12-26 19:48:10,237][105692] Updated weights for policy 0, policy_version 609226 (0.0009) [2023-12-26 19:48:10,298][105692] Updated weights for policy 0, policy_version 609236 (0.0008) [2023-12-26 19:48:10,860][105620] Updated weights for policy 1, policy_version 610032 (0.0009) [2023-12-26 19:48:10,916][105620] Updated weights for policy 1, policy_version 610042 (0.0009) [2023-12-26 19:48:10,981][105620] Updated weights for policy 1, policy_version 610052 (0.0006) [2023-12-26 19:48:11,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 312180736. Throughput: 0: 9916.1, 1: 9339.8. Samples: 312188004. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-12-26 19:48:11,063][104569] Avg episode reward: [(0, '970.776'), (1, '9264.765')] [2023-12-26 19:48:11,067][105692] Updated weights for policy 0, policy_version 609246 (0.0008) [2023-12-26 19:48:11,133][105692] Updated weights for policy 0, policy_version 609256 (0.0009) [2023-12-26 19:48:11,194][105692] Updated weights for policy 0, policy_version 609266 (0.0008) [2023-12-26 19:48:11,808][105620] Updated weights for policy 1, policy_version 610062 (0.0007) [2023-12-26 19:48:11,871][105620] Updated weights for policy 1, policy_version 610072 (0.0007) [2023-12-26 19:48:11,906][105692] Updated weights for policy 0, policy_version 609276 (0.0007) [2023-12-26 19:48:11,923][105620] Updated weights for policy 1, policy_version 610082 (0.0006) [2023-12-26 19:48:11,968][105692] Updated weights for policy 0, policy_version 609286 (0.0008) [2023-12-26 19:48:11,974][105585] KL-divergence is very high: 100.7230 [2023-12-26 19:48:12,029][105692] Updated weights for policy 0, policy_version 609296 (0.0009) [2023-12-26 19:48:12,684][105620] Updated weights for policy 1, policy_version 610092 (0.0008) [2023-12-26 19:48:12,730][105692] Updated weights for policy 0, policy_version 609306 (0.0008) [2023-12-26 19:48:12,731][105620] Updated weights for policy 1, policy_version 610102 (0.0009) [2023-12-26 19:48:12,790][105620] Updated weights for policy 1, policy_version 610112 (0.0008) [2023-12-26 19:48:12,795][105692] Updated weights for policy 0, policy_version 609316 (0.0006) [2023-12-26 19:48:12,856][105692] Updated weights for policy 0, policy_version 609326 (0.0005) [2023-12-26 19:48:12,922][105692] Updated weights for policy 0, policy_version 609336 (0.0006) [2023-12-26 19:48:13,528][105692] Updated weights for policy 0, policy_version 609346 (0.0005) [2023-12-26 19:48:13,577][105692] Updated weights for policy 0, policy_version 609356 (0.0005) [2023-12-26 19:48:13,621][105620] Updated weights for policy 1, policy_version 610122 (0.0009) [2023-12-26 19:48:13,634][105692] Updated weights for policy 0, policy_version 609366 (0.0006) [2023-12-26 19:48:13,680][105620] Updated weights for policy 1, policy_version 610132 (0.0009) [2023-12-26 19:48:13,733][105620] Updated weights for policy 1, policy_version 610143 (0.0010) [2023-12-26 19:48:14,205][105692] Updated weights for policy 0, policy_version 609376 (0.0008) [2023-12-26 19:48:14,276][105692] Updated weights for policy 0, policy_version 609386 (0.0006) [2023-12-26 19:48:14,339][105692] Updated weights for policy 0, policy_version 609396 (0.0008) [2023-12-26 19:48:14,569][105620] Updated weights for policy 1, policy_version 610153 (0.0009) [2023-12-26 19:48:14,613][105620] Updated weights for policy 1, policy_version 610163 (0.0008) [2023-12-26 19:48:14,660][105620] Updated weights for policy 1, policy_version 610173 (0.0008) [2023-12-26 19:48:14,715][105620] Updated weights for policy 1, policy_version 610183 (0.0008) [2023-12-26 19:48:15,012][105692] Updated weights for policy 0, policy_version 609406 (0.0010) [2023-12-26 19:48:15,076][105692] Updated weights for policy 0, policy_version 609416 (0.0011) [2023-12-26 19:48:15,136][105692] Updated weights for policy 0, policy_version 609426 (0.0011) [2023-12-26 19:48:15,454][105620] Updated weights for policy 1, policy_version 610193 (0.0006) [2023-12-26 19:48:15,514][105620] Updated weights for policy 1, policy_version 610203 (0.0006) [2023-12-26 19:48:15,580][105620] Updated weights for policy 1, policy_version 610213 (0.0006) [2023-12-26 19:48:15,856][105692] Updated weights for policy 0, policy_version 609436 (0.0009) [2023-12-26 19:48:15,910][105692] Updated weights for policy 0, policy_version 609446 (0.0005) [2023-12-26 19:48:15,952][105585] KL-divergence is very high: 160.0319 [2023-12-26 19:48:15,968][105692] Updated weights for policy 0, policy_version 609456 (0.0008) [2023-12-26 19:48:15,971][105585] KL-divergence is very high: 156.0040 [2023-12-26 19:48:16,001][105585] KL-divergence is very high: 140.8326 [2023-12-26 19:48:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 312279040. Throughput: 0: 9904.2, 1: 9302.5. Samples: 312244032. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-12-26 19:48:16,063][104569] Avg episode reward: [(0, '1107.896'), (1, '9263.479')] [2023-12-26 19:48:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000609464_156049408.pth... [2023-12-26 19:48:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000610216_156229632.pth... [2023-12-26 19:48:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000609128_155951104.pth [2023-12-26 19:48:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000608312_155754496.pth [2023-12-26 19:48:16,232][105620] Updated weights for policy 1, policy_version 610223 (0.0008) [2023-12-26 19:48:16,294][105620] Updated weights for policy 1, policy_version 610233 (0.0009) [2023-12-26 19:48:16,347][105620] Updated weights for policy 1, policy_version 610243 (0.0009) [2023-12-26 19:48:16,613][105692] Updated weights for policy 0, policy_version 609466 (0.0008) [2023-12-26 19:48:16,666][105692] Updated weights for policy 0, policy_version 609476 (0.0005) [2023-12-26 19:48:16,727][105692] Updated weights for policy 0, policy_version 609486 (0.0005) [2023-12-26 19:48:16,782][105692] Updated weights for policy 0, policy_version 609496 (0.0005) [2023-12-26 19:48:17,215][105620] Updated weights for policy 1, policy_version 610254 (0.0009) [2023-12-26 19:48:17,273][105620] Updated weights for policy 1, policy_version 610264 (0.0009) [2023-12-26 19:48:17,333][105620] Updated weights for policy 1, policy_version 610274 (0.0009) [2023-12-26 19:48:17,372][105692] Updated weights for policy 0, policy_version 609506 (0.0006) [2023-12-26 19:48:17,433][105692] Updated weights for policy 0, policy_version 609516 (0.0009) [2023-12-26 19:48:17,497][105692] Updated weights for policy 0, policy_version 609526 (0.0009) [2023-12-26 19:48:18,053][105620] Updated weights for policy 1, policy_version 610284 (0.0007) [2023-12-26 19:48:18,099][105620] Updated weights for policy 1, policy_version 610294 (0.0005) [2023-12-26 19:48:18,155][105620] Updated weights for policy 1, policy_version 610304 (0.0008) [2023-12-26 19:48:18,249][105692] Updated weights for policy 0, policy_version 609536 (0.0010) [2023-12-26 19:48:18,323][105692] Updated weights for policy 0, policy_version 609546 (0.0009) [2023-12-26 19:48:18,387][105692] Updated weights for policy 0, policy_version 609556 (0.0006) [2023-12-26 19:48:18,774][105620] Updated weights for policy 1, policy_version 610314 (0.0008) [2023-12-26 19:48:18,833][105620] Updated weights for policy 1, policy_version 610324 (0.0009) [2023-12-26 19:48:18,895][105620] Updated weights for policy 1, policy_version 610334 (0.0009) [2023-12-26 19:48:18,957][105620] Updated weights for policy 1, policy_version 610344 (0.0009) [2023-12-26 19:48:19,121][105692] Updated weights for policy 0, policy_version 609566 (0.0008) [2023-12-26 19:48:19,184][105692] Updated weights for policy 0, policy_version 609576 (0.0009) [2023-12-26 19:48:19,244][105692] Updated weights for policy 0, policy_version 609586 (0.0010) [2023-12-26 19:48:19,762][105620] Updated weights for policy 1, policy_version 610354 (0.0009) [2023-12-26 19:48:19,829][105620] Updated weights for policy 1, policy_version 610364 (0.0009) [2023-12-26 19:48:19,890][105620] Updated weights for policy 1, policy_version 610374 (0.0010) [2023-12-26 19:48:19,995][105692] Updated weights for policy 0, policy_version 609596 (0.0007) [2023-12-26 19:48:20,048][105692] Updated weights for policy 0, policy_version 609606 (0.0006) [2023-12-26 19:48:20,109][105692] Updated weights for policy 0, policy_version 609616 (0.0008) [2023-12-26 19:48:20,700][105620] Updated weights for policy 1, policy_version 610384 (0.0008) [2023-12-26 19:48:20,763][105620] Updated weights for policy 1, policy_version 610394 (0.0009) [2023-12-26 19:48:20,774][105692] Updated weights for policy 0, policy_version 609626 (0.0009) [2023-12-26 19:48:20,826][105620] Updated weights for policy 1, policy_version 610404 (0.0009) [2023-12-26 19:48:20,833][105692] Updated weights for policy 0, policy_version 609636 (0.0006) [2023-12-26 19:48:20,892][105692] Updated weights for policy 0, policy_version 609646 (0.0008) [2023-12-26 19:48:20,955][105692] Updated weights for policy 0, policy_version 609656 (0.0007) [2023-12-26 19:48:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 312377344. Throughput: 0: 9931.3, 1: 9343.0. Samples: 312361512. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-12-26 19:48:21,062][104569] Avg episode reward: [(0, '1523.328'), (1, '9172.047')] [2023-12-26 19:48:21,639][105620] Updated weights for policy 1, policy_version 610414 (0.0010) [2023-12-26 19:48:21,709][105620] Updated weights for policy 1, policy_version 610424 (0.0009) [2023-12-26 19:48:21,781][105620] Updated weights for policy 1, policy_version 610434 (0.0010) [2023-12-26 19:48:21,783][105692] Updated weights for policy 0, policy_version 609666 (0.0008) [2023-12-26 19:48:21,845][105692] Updated weights for policy 0, policy_version 609676 (0.0007) [2023-12-26 19:48:21,913][105692] Updated weights for policy 0, policy_version 609686 (0.0008) [2023-12-26 19:48:22,530][105620] Updated weights for policy 1, policy_version 610444 (0.0011) [2023-12-26 19:48:22,589][105620] Updated weights for policy 1, policy_version 610454 (0.0011) [2023-12-26 19:48:22,649][105620] Updated weights for policy 1, policy_version 610464 (0.0011) [2023-12-26 19:48:22,684][105692] Updated weights for policy 0, policy_version 609696 (0.0007) [2023-12-26 19:48:22,747][105692] Updated weights for policy 0, policy_version 609706 (0.0008) [2023-12-26 19:48:22,796][105692] Updated weights for policy 0, policy_version 609716 (0.0008) [2023-12-26 19:48:23,297][105620] Updated weights for policy 1, policy_version 610474 (0.0009) [2023-12-26 19:48:23,357][105620] Updated weights for policy 1, policy_version 610484 (0.0005) [2023-12-26 19:48:23,412][105620] Updated weights for policy 1, policy_version 610494 (0.0005) [2023-12-26 19:48:23,463][105620] Updated weights for policy 1, policy_version 610504 (0.0005) [2023-12-26 19:48:23,569][105692] Updated weights for policy 0, policy_version 609726 (0.0010) [2023-12-26 19:48:23,626][105692] Updated weights for policy 0, policy_version 609736 (0.0008) [2023-12-26 19:48:23,677][105692] Updated weights for policy 0, policy_version 609746 (0.0007) [2023-12-26 19:48:23,984][105620] Updated weights for policy 1, policy_version 610514 (0.0008) [2023-12-26 19:48:24,049][105620] Updated weights for policy 1, policy_version 610524 (0.0008) [2023-12-26 19:48:24,116][105620] Updated weights for policy 1, policy_version 610534 (0.0005) [2023-12-26 19:48:24,301][105692] Updated weights for policy 0, policy_version 609756 (0.0007) [2023-12-26 19:48:24,363][105692] Updated weights for policy 0, policy_version 609766 (0.0006) [2023-12-26 19:48:24,414][105692] Updated weights for policy 0, policy_version 609776 (0.0008) [2023-12-26 19:48:24,680][105620] Updated weights for policy 1, policy_version 610544 (0.0008) [2023-12-26 19:48:24,744][105620] Updated weights for policy 1, policy_version 610554 (0.0006) [2023-12-26 19:48:24,815][105620] Updated weights for policy 1, policy_version 610564 (0.0007) [2023-12-26 19:48:25,074][105692] Updated weights for policy 0, policy_version 609786 (0.0010) [2023-12-26 19:48:25,136][105692] Updated weights for policy 0, policy_version 609796 (0.0005) [2023-12-26 19:48:25,194][105692] Updated weights for policy 0, policy_version 609806 (0.0007) [2023-12-26 19:48:25,390][105620] Updated weights for policy 1, policy_version 610574 (0.0008) [2023-12-26 19:48:25,454][105620] Updated weights for policy 1, policy_version 610584 (0.0005) [2023-12-26 19:48:25,518][105620] Updated weights for policy 1, policy_version 610594 (0.0005) [2023-12-26 19:48:25,883][105692] Updated weights for policy 0, policy_version 609817 (0.0010) [2023-12-26 19:48:25,930][105692] Updated weights for policy 0, policy_version 609827 (0.0007) [2023-12-26 19:48:25,983][105692] Updated weights for policy 0, policy_version 609837 (0.0005) [2023-12-26 19:48:26,045][105692] Updated weights for policy 0, policy_version 609847 (0.0005) [2023-12-26 19:48:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 312475648. Throughput: 0: 9837.7, 1: 9538.8. Samples: 312481820. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-12-26 19:48:26,062][104569] Avg episode reward: [(0, '904.194'), (1, '9170.991')] [2023-12-26 19:48:26,108][105620] Updated weights for policy 1, policy_version 610604 (0.0005) [2023-12-26 19:48:26,177][105620] Updated weights for policy 1, policy_version 610614 (0.0005) [2023-12-26 19:48:26,250][105620] Updated weights for policy 1, policy_version 610624 (0.0005) [2023-12-26 19:48:26,564][105692] Updated weights for policy 0, policy_version 609857 (0.0005) [2023-12-26 19:48:26,623][105692] Updated weights for policy 0, policy_version 609867 (0.0007) [2023-12-26 19:48:26,680][105692] Updated weights for policy 0, policy_version 609877 (0.0010) [2023-12-26 19:48:26,769][105620] Updated weights for policy 1, policy_version 610634 (0.0005) [2023-12-26 19:48:26,829][105620] Updated weights for policy 1, policy_version 610644 (0.0005) [2023-12-26 19:48:26,882][105620] Updated weights for policy 1, policy_version 610654 (0.0005) [2023-12-26 19:48:26,930][105620] Updated weights for policy 1, policy_version 610664 (0.0005) [2023-12-26 19:48:27,325][105692] Updated weights for policy 0, policy_version 609887 (0.0007) [2023-12-26 19:48:27,389][105692] Updated weights for policy 0, policy_version 609897 (0.0008) [2023-12-26 19:48:27,453][105692] Updated weights for policy 0, policy_version 609907 (0.0008) [2023-12-26 19:48:27,548][105620] Updated weights for policy 1, policy_version 610674 (0.0006) [2023-12-26 19:48:27,611][105620] Updated weights for policy 1, policy_version 610684 (0.0006) [2023-12-26 19:48:27,671][105620] Updated weights for policy 1, policy_version 610694 (0.0006) [2023-12-26 19:48:28,178][105692] Updated weights for policy 0, policy_version 609917 (0.0010) [2023-12-26 19:48:28,238][105692] Updated weights for policy 0, policy_version 609927 (0.0010) [2023-12-26 19:48:28,292][105692] Updated weights for policy 0, policy_version 609937 (0.0007) [2023-12-26 19:48:28,301][105620] Updated weights for policy 1, policy_version 610704 (0.0008) [2023-12-26 19:48:28,366][105620] Updated weights for policy 1, policy_version 610714 (0.0009) [2023-12-26 19:48:28,433][105620] Updated weights for policy 1, policy_version 610724 (0.0011) [2023-12-26 19:48:28,881][105692] Updated weights for policy 0, policy_version 609947 (0.0008) [2023-12-26 19:48:28,934][105692] Updated weights for policy 0, policy_version 609957 (0.0011) [2023-12-26 19:48:28,998][105692] Updated weights for policy 0, policy_version 609967 (0.0011) [2023-12-26 19:48:29,145][105620] Updated weights for policy 1, policy_version 610734 (0.0009) [2023-12-26 19:48:29,203][105620] Updated weights for policy 1, policy_version 610744 (0.0005) [2023-12-26 19:48:29,262][105620] Updated weights for policy 1, policy_version 610754 (0.0008) [2023-12-26 19:48:29,779][105692] Updated weights for policy 0, policy_version 609977 (0.0010) [2023-12-26 19:48:29,837][105692] Updated weights for policy 0, policy_version 609987 (0.0008) [2023-12-26 19:48:29,894][105692] Updated weights for policy 0, policy_version 609997 (0.0009) [2023-12-26 19:48:29,924][105620] Updated weights for policy 1, policy_version 610764 (0.0008) [2023-12-26 19:48:29,957][105692] Updated weights for policy 0, policy_version 610007 (0.0008) [2023-12-26 19:48:29,984][105620] Updated weights for policy 1, policy_version 610774 (0.0008) [2023-12-26 19:48:30,042][105620] Updated weights for policy 1, policy_version 610784 (0.0009) [2023-12-26 19:48:30,680][105620] Updated weights for policy 1, policy_version 610794 (0.0010) [2023-12-26 19:48:30,733][105620] Updated weights for policy 1, policy_version 610804 (0.0010) [2023-12-26 19:48:30,789][105620] Updated weights for policy 1, policy_version 610814 (0.0011) [2023-12-26 19:48:30,806][105692] Updated weights for policy 0, policy_version 610017 (0.0007) [2023-12-26 19:48:30,848][105620] Updated weights for policy 1, policy_version 610824 (0.0010) [2023-12-26 19:48:30,854][105692] Updated weights for policy 0, policy_version 610027 (0.0007) [2023-12-26 19:48:30,916][105692] Updated weights for policy 0, policy_version 610037 (0.0006) [2023-12-26 19:48:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 312582144. Throughput: 0: 9890.1, 1: 9650.8. Samples: 312546644. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-12-26 19:48:31,062][104569] Avg episode reward: [(0, '1127.968'), (1, '9170.826')] [2023-12-26 19:48:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000610040_156196864.pth... [2023-12-26 19:48:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000610824_156385280.pth... [2023-12-26 19:48:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000608888_155901952.pth [2023-12-26 19:48:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000609704_156098560.pth [2023-12-26 19:48:31,495][105620] Updated weights for policy 1, policy_version 610834 (0.0006) [2023-12-26 19:48:31,567][105620] Updated weights for policy 1, policy_version 610844 (0.0010) [2023-12-26 19:48:31,629][105620] Updated weights for policy 1, policy_version 610854 (0.0009) [2023-12-26 19:48:31,720][105692] Updated weights for policy 0, policy_version 610047 (0.0009) [2023-12-26 19:48:31,772][105692] Updated weights for policy 0, policy_version 610057 (0.0008) [2023-12-26 19:48:31,820][105692] Updated weights for policy 0, policy_version 610067 (0.0008) [2023-12-26 19:48:32,330][105620] Updated weights for policy 1, policy_version 610864 (0.0009) [2023-12-26 19:48:32,398][105620] Updated weights for policy 1, policy_version 610874 (0.0009) [2023-12-26 19:48:32,446][105620] Updated weights for policy 1, policy_version 610884 (0.0011) [2023-12-26 19:48:32,594][105692] Updated weights for policy 0, policy_version 610077 (0.0010) [2023-12-26 19:48:32,652][105692] Updated weights for policy 0, policy_version 610087 (0.0010) [2023-12-26 19:48:32,709][105692] Updated weights for policy 0, policy_version 610097 (0.0010) [2023-12-26 19:48:33,121][105620] Updated weights for policy 1, policy_version 610894 (0.0010) [2023-12-26 19:48:33,173][105620] Updated weights for policy 1, policy_version 610905 (0.0009) [2023-12-26 19:48:33,244][105620] Updated weights for policy 1, policy_version 610916 (0.0009) [2023-12-26 19:48:33,386][105692] Updated weights for policy 0, policy_version 610107 (0.0009) [2023-12-26 19:48:33,433][105692] Updated weights for policy 0, policy_version 610117 (0.0008) [2023-12-26 19:48:33,478][105692] Updated weights for policy 0, policy_version 610127 (0.0007) [2023-12-26 19:48:33,946][105620] Updated weights for policy 1, policy_version 610926 (0.0007) [2023-12-26 19:48:33,998][105620] Updated weights for policy 1, policy_version 610936 (0.0005) [2023-12-26 19:48:34,042][105620] Updated weights for policy 1, policy_version 610946 (0.0005) [2023-12-26 19:48:34,289][105692] Updated weights for policy 0, policy_version 610137 (0.0008) [2023-12-26 19:48:34,345][105692] Updated weights for policy 0, policy_version 610147 (0.0008) [2023-12-26 19:48:34,398][105692] Updated weights for policy 0, policy_version 610157 (0.0008) [2023-12-26 19:48:34,455][105692] Updated weights for policy 0, policy_version 610167 (0.0008) [2023-12-26 19:48:34,705][105620] Updated weights for policy 1, policy_version 610956 (0.0007) [2023-12-26 19:48:34,757][105620] Updated weights for policy 1, policy_version 610966 (0.0010) [2023-12-26 19:48:34,809][105620] Updated weights for policy 1, policy_version 610976 (0.0010) [2023-12-26 19:48:35,236][105692] Updated weights for policy 0, policy_version 610177 (0.0010) [2023-12-26 19:48:35,303][105692] Updated weights for policy 0, policy_version 610187 (0.0011) [2023-12-26 19:48:35,362][105692] Updated weights for policy 0, policy_version 610197 (0.0010) [2023-12-26 19:48:35,593][105620] Updated weights for policy 1, policy_version 610986 (0.0010) [2023-12-26 19:48:35,652][105620] Updated weights for policy 1, policy_version 610996 (0.0010) [2023-12-26 19:48:35,711][105620] Updated weights for policy 1, policy_version 611006 (0.0011) [2023-12-26 19:48:35,774][105620] Updated weights for policy 1, policy_version 611016 (0.0011) [2023-12-26 19:48:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 312672256. Throughput: 0: 9822.8, 1: 9637.2. Samples: 312663172. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-12-26 19:48:36,063][104569] Avg episode reward: [(0, '1226.092'), (1, '9355.102')] [2023-12-26 19:48:36,093][105692] Updated weights for policy 0, policy_version 610207 (0.0010) [2023-12-26 19:48:36,181][105692] Updated weights for policy 0, policy_version 610217 (0.0010) [2023-12-26 19:48:36,240][105692] Updated weights for policy 0, policy_version 610227 (0.0010) [2023-12-26 19:48:36,513][105620] Updated weights for policy 1, policy_version 611026 (0.0011) [2023-12-26 19:48:36,573][105620] Updated weights for policy 1, policy_version 611036 (0.0010) [2023-12-26 19:48:36,632][105620] Updated weights for policy 1, policy_version 611046 (0.0011) [2023-12-26 19:48:36,972][105692] Updated weights for policy 0, policy_version 610237 (0.0010) [2023-12-26 19:48:37,030][105692] Updated weights for policy 0, policy_version 610247 (0.0010) [2023-12-26 19:48:37,089][105692] Updated weights for policy 0, policy_version 610257 (0.0010) [2023-12-26 19:48:37,386][105620] Updated weights for policy 1, policy_version 611057 (0.0011) [2023-12-26 19:48:37,440][105620] Updated weights for policy 1, policy_version 611068 (0.0010) [2023-12-26 19:48:37,497][105620] Updated weights for policy 1, policy_version 611078 (0.0009) [2023-12-26 19:48:37,748][105692] Updated weights for policy 0, policy_version 610267 (0.0009) [2023-12-26 19:48:37,803][105692] Updated weights for policy 0, policy_version 610277 (0.0009) [2023-12-26 19:48:37,865][105692] Updated weights for policy 0, policy_version 610287 (0.0010) [2023-12-26 19:48:38,223][105620] Updated weights for policy 1, policy_version 611088 (0.0007) [2023-12-26 19:48:38,270][105620] Updated weights for policy 1, policy_version 611098 (0.0005) [2023-12-26 19:48:38,316][105620] Updated weights for policy 1, policy_version 611108 (0.0005) [2023-12-26 19:48:38,715][105692] Updated weights for policy 0, policy_version 610297 (0.0009) [2023-12-26 19:48:38,783][105692] Updated weights for policy 0, policy_version 610307 (0.0007) [2023-12-26 19:48:38,841][105692] Updated weights for policy 0, policy_version 610317 (0.0009) [2023-12-26 19:48:38,896][105692] Updated weights for policy 0, policy_version 610327 (0.0009) [2023-12-26 19:48:38,924][105620] Updated weights for policy 1, policy_version 611118 (0.0007) [2023-12-26 19:48:38,979][105620] Updated weights for policy 1, policy_version 611128 (0.0005) [2023-12-26 19:48:39,049][105620] Updated weights for policy 1, policy_version 611138 (0.0006) [2023-12-26 19:48:39,709][105692] Updated weights for policy 0, policy_version 610337 (0.0009) [2023-12-26 19:48:39,749][105620] Updated weights for policy 1, policy_version 611148 (0.0007) [2023-12-26 19:48:39,772][105692] Updated weights for policy 0, policy_version 610347 (0.0008) [2023-12-26 19:48:39,799][105620] Updated weights for policy 1, policy_version 611158 (0.0005) [2023-12-26 19:48:39,837][105692] Updated weights for policy 0, policy_version 610357 (0.0009) [2023-12-26 19:48:39,866][105620] Updated weights for policy 1, policy_version 611168 (0.0008) [2023-12-26 19:48:40,570][105692] Updated weights for policy 0, policy_version 610367 (0.0009) [2023-12-26 19:48:40,588][105620] Updated weights for policy 1, policy_version 611178 (0.0009) [2023-12-26 19:48:40,632][105692] Updated weights for policy 0, policy_version 610377 (0.0006) [2023-12-26 19:48:40,652][105620] Updated weights for policy 1, policy_version 611188 (0.0008) [2023-12-26 19:48:40,691][105692] Updated weights for policy 0, policy_version 610387 (0.0007) [2023-12-26 19:48:40,715][105620] Updated weights for policy 1, policy_version 611198 (0.0009) [2023-12-26 19:48:40,776][105620] Updated weights for policy 1, policy_version 611208 (0.0009) [2023-12-26 19:48:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 312770560. Throughput: 0: 9782.0, 1: 9678.1. Samples: 312776920. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-12-26 19:48:41,062][104569] Avg episode reward: [(0, '6428.096'), (1, '9263.912')] [2023-12-26 19:48:41,484][105620] Updated weights for policy 1, policy_version 611218 (0.0009) [2023-12-26 19:48:41,491][105692] Updated weights for policy 0, policy_version 610397 (0.0008) [2023-12-26 19:48:41,551][105620] Updated weights for policy 1, policy_version 611228 (0.0007) [2023-12-26 19:48:41,556][105692] Updated weights for policy 0, policy_version 610407 (0.0006) [2023-12-26 19:48:41,615][105620] Updated weights for policy 1, policy_version 611238 (0.0008) [2023-12-26 19:48:41,617][105692] Updated weights for policy 0, policy_version 610417 (0.0007) [2023-12-26 19:48:42,295][105692] Updated weights for policy 0, policy_version 610427 (0.0008) [2023-12-26 19:48:42,351][105692] Updated weights for policy 0, policy_version 610437 (0.0009) [2023-12-26 19:48:42,393][105620] Updated weights for policy 1, policy_version 611248 (0.0008) [2023-12-26 19:48:42,414][105692] Updated weights for policy 0, policy_version 610447 (0.0006) [2023-12-26 19:48:42,450][105620] Updated weights for policy 1, policy_version 611258 (0.0009) [2023-12-26 19:48:42,499][105620] Updated weights for policy 1, policy_version 611268 (0.0009) [2023-12-26 19:48:43,133][105692] Updated weights for policy 0, policy_version 610457 (0.0006) [2023-12-26 19:48:43,194][105692] Updated weights for policy 0, policy_version 610467 (0.0006) [2023-12-26 19:48:43,249][105620] Updated weights for policy 1, policy_version 611278 (0.0009) [2023-12-26 19:48:43,261][105692] Updated weights for policy 0, policy_version 610477 (0.0006) [2023-12-26 19:48:43,296][105620] Updated weights for policy 1, policy_version 611288 (0.0008) [2023-12-26 19:48:43,314][105692] Updated weights for policy 0, policy_version 610487 (0.0009) [2023-12-26 19:48:43,338][105620] Updated weights for policy 1, policy_version 611298 (0.0007) [2023-12-26 19:48:43,989][105692] Updated weights for policy 0, policy_version 610497 (0.0010) [2023-12-26 19:48:44,040][105692] Updated weights for policy 0, policy_version 610507 (0.0010) [2023-12-26 19:48:44,077][105620] Updated weights for policy 1, policy_version 611308 (0.0007) [2023-12-26 19:48:44,088][105692] Updated weights for policy 0, policy_version 610517 (0.0010) [2023-12-26 19:48:44,127][105620] Updated weights for policy 1, policy_version 611318 (0.0005) [2023-12-26 19:48:44,179][105620] Updated weights for policy 1, policy_version 611328 (0.0005) [2023-12-26 19:48:44,832][105692] Updated weights for policy 0, policy_version 610527 (0.0009) [2023-12-26 19:48:44,868][105620] Updated weights for policy 1, policy_version 611338 (0.0006) [2023-12-26 19:48:44,890][105692] Updated weights for policy 0, policy_version 610537 (0.0007) [2023-12-26 19:48:44,928][105620] Updated weights for policy 1, policy_version 611348 (0.0007) [2023-12-26 19:48:44,947][105692] Updated weights for policy 0, policy_version 610547 (0.0007) [2023-12-26 19:48:44,992][105620] Updated weights for policy 1, policy_version 611358 (0.0009) [2023-12-26 19:48:45,051][105620] Updated weights for policy 1, policy_version 611368 (0.0009) [2023-12-26 19:48:45,690][105692] Updated weights for policy 0, policy_version 610557 (0.0009) [2023-12-26 19:48:45,749][105692] Updated weights for policy 0, policy_version 610567 (0.0008) [2023-12-26 19:48:45,771][105620] Updated weights for policy 1, policy_version 611378 (0.0011) [2023-12-26 19:48:45,805][105692] Updated weights for policy 0, policy_version 610577 (0.0005) [2023-12-26 19:48:45,830][105620] Updated weights for policy 1, policy_version 611388 (0.0010) [2023-12-26 19:48:45,879][105620] Updated weights for policy 1, policy_version 611398 (0.0010) [2023-12-26 19:48:46,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 312868864. Throughput: 0: 9705.2, 1: 9673.7. Samples: 312833276. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-12-26 19:48:46,062][104569] Avg episode reward: [(0, '9351.975'), (1, '9169.762')] [2023-12-26 19:48:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000611400_156532736.pth... [2023-12-26 19:48:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000610584_156336128.pth... [2023-12-26 19:48:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000610216_156229632.pth [2023-12-26 19:48:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000609464_156049408.pth [2023-12-26 19:48:46,503][105620] Updated weights for policy 1, policy_version 611408 (0.0010) [2023-12-26 19:48:46,561][105620] Updated weights for policy 1, policy_version 611418 (0.0010) [2023-12-26 19:48:46,613][105692] Updated weights for policy 0, policy_version 610587 (0.0008) [2023-12-26 19:48:46,619][105620] Updated weights for policy 1, policy_version 611428 (0.0009) [2023-12-26 19:48:46,671][105692] Updated weights for policy 0, policy_version 610597 (0.0007) [2023-12-26 19:48:46,735][105692] Updated weights for policy 0, policy_version 610607 (0.0008) [2023-12-26 19:48:47,330][105620] Updated weights for policy 1, policy_version 611438 (0.0007) [2023-12-26 19:48:47,384][105620] Updated weights for policy 1, policy_version 611448 (0.0005) [2023-12-26 19:48:47,426][105692] Updated weights for policy 0, policy_version 610617 (0.0008) [2023-12-26 19:48:47,445][105620] Updated weights for policy 1, policy_version 611458 (0.0005) [2023-12-26 19:48:47,474][105692] Updated weights for policy 0, policy_version 610627 (0.0009) [2023-12-26 19:48:47,526][105692] Updated weights for policy 0, policy_version 610637 (0.0009) [2023-12-26 19:48:47,577][105692] Updated weights for policy 0, policy_version 610647 (0.0008) [2023-12-26 19:48:48,072][105620] Updated weights for policy 1, policy_version 611468 (0.0007) [2023-12-26 19:48:48,119][105620] Updated weights for policy 1, policy_version 611478 (0.0010) [2023-12-26 19:48:48,171][105620] Updated weights for policy 1, policy_version 611488 (0.0010) [2023-12-26 19:48:48,370][105692] Updated weights for policy 0, policy_version 610657 (0.0008) [2023-12-26 19:48:48,429][105692] Updated weights for policy 0, policy_version 610667 (0.0008) [2023-12-26 19:48:48,481][105692] Updated weights for policy 0, policy_version 610677 (0.0008) [2023-12-26 19:48:48,935][105620] Updated weights for policy 1, policy_version 611498 (0.0010) [2023-12-26 19:48:48,994][105620] Updated weights for policy 1, policy_version 611508 (0.0010) [2023-12-26 19:48:49,049][105620] Updated weights for policy 1, policy_version 611518 (0.0010) [2023-12-26 19:48:49,114][105620] Updated weights for policy 1, policy_version 611528 (0.0010) [2023-12-26 19:48:49,239][105692] Updated weights for policy 0, policy_version 610687 (0.0008) [2023-12-26 19:48:49,307][105692] Updated weights for policy 0, policy_version 610697 (0.0008) [2023-12-26 19:48:49,373][105692] Updated weights for policy 0, policy_version 610707 (0.0009) [2023-12-26 19:48:49,821][105620] Updated weights for policy 1, policy_version 611538 (0.0010) [2023-12-26 19:48:49,885][105620] Updated weights for policy 1, policy_version 611548 (0.0009) [2023-12-26 19:48:49,955][105620] Updated weights for policy 1, policy_version 611558 (0.0011) [2023-12-26 19:48:50,139][105692] Updated weights for policy 0, policy_version 610717 (0.0008) [2023-12-26 19:48:50,196][105692] Updated weights for policy 0, policy_version 610727 (0.0008) [2023-12-26 19:48:50,252][105692] Updated weights for policy 0, policy_version 610737 (0.0008) [2023-12-26 19:48:50,745][105620] Updated weights for policy 1, policy_version 611568 (0.0008) [2023-12-26 19:48:50,798][105620] Updated weights for policy 1, policy_version 611578 (0.0008) [2023-12-26 19:48:50,845][105620] Updated weights for policy 1, policy_version 611588 (0.0009) [2023-12-26 19:48:50,919][105692] Updated weights for policy 0, policy_version 610747 (0.0009) [2023-12-26 19:48:50,985][105692] Updated weights for policy 0, policy_version 610757 (0.0009) [2023-12-26 19:48:51,054][105692] Updated weights for policy 0, policy_version 610767 (0.0010) [2023-12-26 19:48:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 312958976. Throughput: 0: 9642.3, 1: 9770.1. Samples: 312949364. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-12-26 19:48:51,062][104569] Avg episode reward: [(0, '9352.036'), (1, '9169.144')] [2023-12-26 19:48:51,656][105620] Updated weights for policy 1, policy_version 611598 (0.0009) [2023-12-26 19:48:51,715][105620] Updated weights for policy 1, policy_version 611608 (0.0008) [2023-12-26 19:48:51,750][105692] Updated weights for policy 0, policy_version 610777 (0.0009) [2023-12-26 19:48:51,781][105620] Updated weights for policy 1, policy_version 611618 (0.0007) [2023-12-26 19:48:51,813][105692] Updated weights for policy 0, policy_version 610787 (0.0007) [2023-12-26 19:48:51,882][105692] Updated weights for policy 0, policy_version 610797 (0.0008) [2023-12-26 19:48:51,942][105692] Updated weights for policy 0, policy_version 610807 (0.0009) [2023-12-26 19:48:52,432][105620] Updated weights for policy 1, policy_version 611628 (0.0007) [2023-12-26 19:48:52,487][105620] Updated weights for policy 1, policy_version 611638 (0.0009) [2023-12-26 19:48:52,543][105620] Updated weights for policy 1, policy_version 611648 (0.0009) [2023-12-26 19:48:52,711][105692] Updated weights for policy 0, policy_version 610817 (0.0006) [2023-12-26 19:48:52,781][105692] Updated weights for policy 0, policy_version 610827 (0.0007) [2023-12-26 19:48:52,846][105692] Updated weights for policy 0, policy_version 610837 (0.0009) [2023-12-26 19:48:53,284][105620] Updated weights for policy 1, policy_version 611658 (0.0007) [2023-12-26 19:48:53,343][105620] Updated weights for policy 1, policy_version 611668 (0.0009) [2023-12-26 19:48:53,399][105620] Updated weights for policy 1, policy_version 611678 (0.0009) [2023-12-26 19:48:53,464][105620] Updated weights for policy 1, policy_version 611688 (0.0009) [2023-12-26 19:48:53,523][105692] Updated weights for policy 0, policy_version 610847 (0.0007) [2023-12-26 19:48:53,574][105692] Updated weights for policy 0, policy_version 610857 (0.0005) [2023-12-26 19:48:53,631][105692] Updated weights for policy 0, policy_version 610867 (0.0005) [2023-12-26 19:48:54,221][105692] Updated weights for policy 0, policy_version 610877 (0.0007) [2023-12-26 19:48:54,267][105620] Updated weights for policy 1, policy_version 611698 (0.0007) [2023-12-26 19:48:54,269][105692] Updated weights for policy 0, policy_version 610887 (0.0006) [2023-12-26 19:48:54,320][105692] Updated weights for policy 0, policy_version 610897 (0.0008) [2023-12-26 19:48:54,323][105620] Updated weights for policy 1, policy_version 611708 (0.0008) [2023-12-26 19:48:54,377][105620] Updated weights for policy 1, policy_version 611718 (0.0007) [2023-12-26 19:48:55,070][105620] Updated weights for policy 1, policy_version 611728 (0.0009) [2023-12-26 19:48:55,103][105692] Updated weights for policy 0, policy_version 610907 (0.0006) [2023-12-26 19:48:55,125][105620] Updated weights for policy 1, policy_version 611738 (0.0008) [2023-12-26 19:48:55,156][105692] Updated weights for policy 0, policy_version 610917 (0.0007) [2023-12-26 19:48:55,184][105620] Updated weights for policy 1, policy_version 611748 (0.0005) [2023-12-26 19:48:55,209][105692] Updated weights for policy 0, policy_version 610927 (0.0010) [2023-12-26 19:48:55,871][105692] Updated weights for policy 0, policy_version 610938 (0.0010) [2023-12-26 19:48:55,920][105692] Updated weights for policy 0, policy_version 610948 (0.0008) [2023-12-26 19:48:55,942][105620] Updated weights for policy 1, policy_version 611758 (0.0006) [2023-12-26 19:48:55,968][105692] Updated weights for policy 0, policy_version 610958 (0.0006) [2023-12-26 19:48:55,990][105620] Updated weights for policy 1, policy_version 611768 (0.0007) [2023-12-26 19:48:56,016][105692] Updated weights for policy 0, policy_version 610968 (0.0006) [2023-12-26 19:48:56,040][105620] Updated weights for policy 1, policy_version 611778 (0.0007) [2023-12-26 19:48:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 313057280. Throughput: 0: 9670.9, 1: 9811.9. Samples: 313064728. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-12-26 19:48:56,062][104569] Avg episode reward: [(0, '9353.604'), (1, '9044.383')] [2023-12-26 19:48:56,708][105620] Updated weights for policy 1, policy_version 611788 (0.0008) [2023-12-26 19:48:56,742][105692] Updated weights for policy 0, policy_version 610978 (0.0005) [2023-12-26 19:48:56,760][105620] Updated weights for policy 1, policy_version 611798 (0.0010) [2023-12-26 19:48:56,794][105692] Updated weights for policy 0, policy_version 610988 (0.0005) [2023-12-26 19:48:56,815][105620] Updated weights for policy 1, policy_version 611808 (0.0010) [2023-12-26 19:48:56,849][105692] Updated weights for policy 0, policy_version 610998 (0.0005) [2023-12-26 19:48:57,551][105620] Updated weights for policy 1, policy_version 611818 (0.0009) [2023-12-26 19:48:57,577][105692] Updated weights for policy 0, policy_version 611008 (0.0008) [2023-12-26 19:48:57,607][105620] Updated weights for policy 1, policy_version 611828 (0.0010) [2023-12-26 19:48:57,636][105692] Updated weights for policy 0, policy_version 611018 (0.0006) [2023-12-26 19:48:57,665][105620] Updated weights for policy 1, policy_version 611838 (0.0011) [2023-12-26 19:48:57,688][105692] Updated weights for policy 0, policy_version 611028 (0.0005) [2023-12-26 19:48:57,717][105620] Updated weights for policy 1, policy_version 611848 (0.0011) [2023-12-26 19:48:58,451][105692] Updated weights for policy 0, policy_version 611038 (0.0008) [2023-12-26 19:48:58,510][105692] Updated weights for policy 0, policy_version 611048 (0.0007) [2023-12-26 19:48:58,524][105620] Updated weights for policy 1, policy_version 611858 (0.0010) [2023-12-26 19:48:58,575][105692] Updated weights for policy 0, policy_version 611058 (0.0009) [2023-12-26 19:48:58,592][105620] Updated weights for policy 1, policy_version 611868 (0.0010) [2023-12-26 19:48:58,650][105620] Updated weights for policy 1, policy_version 611878 (0.0010) [2023-12-26 19:48:59,425][105692] Updated weights for policy 0, policy_version 611068 (0.0009) [2023-12-26 19:48:59,475][105620] Updated weights for policy 1, policy_version 611888 (0.0010) [2023-12-26 19:48:59,487][105692] Updated weights for policy 0, policy_version 611078 (0.0007) [2023-12-26 19:48:59,528][105620] Updated weights for policy 1, policy_version 611898 (0.0010) [2023-12-26 19:48:59,540][105692] Updated weights for policy 0, policy_version 611088 (0.0010) [2023-12-26 19:48:59,584][105620] Updated weights for policy 1, policy_version 611908 (0.0010) [2023-12-26 19:49:00,129][105692] Updated weights for policy 0, policy_version 611098 (0.0008) [2023-12-26 19:49:00,184][105692] Updated weights for policy 0, policy_version 611108 (0.0006) [2023-12-26 19:49:00,239][105692] Updated weights for policy 0, policy_version 611118 (0.0007) [2023-12-26 19:49:00,287][105692] Updated weights for policy 0, policy_version 611128 (0.0010) [2023-12-26 19:49:00,350][105620] Updated weights for policy 1, policy_version 611918 (0.0008) [2023-12-26 19:49:00,416][105620] Updated weights for policy 1, policy_version 611928 (0.0006) [2023-12-26 19:49:00,475][105620] Updated weights for policy 1, policy_version 611938 (0.0008) [2023-12-26 19:49:00,981][105692] Updated weights for policy 0, policy_version 611138 (0.0005) [2023-12-26 19:49:01,035][105692] Updated weights for policy 0, policy_version 611148 (0.0008) [2023-12-26 19:49:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 313147392. Throughput: 0: 9648.4, 1: 9870.0. Samples: 313122356. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-12-26 19:49:01,063][104569] Avg episode reward: [(0, '9353.723'), (1, '9137.445')] [2023-12-26 19:49:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000611944_156672000.pth... [2023-12-26 19:49:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000610824_156385280.pth [2023-12-26 19:49:01,093][105692] Updated weights for policy 0, policy_version 611158 (0.0010) [2023-12-26 19:49:01,105][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000611160_156483584.pth... [2023-12-26 19:49:01,109][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000610040_156196864.pth [2023-12-26 19:49:01,185][105620] Updated weights for policy 1, policy_version 611948 (0.0008) [2023-12-26 19:49:01,246][105620] Updated weights for policy 1, policy_version 611958 (0.0007) [2023-12-26 19:49:01,301][105620] Updated weights for policy 1, policy_version 611968 (0.0007) [2023-12-26 19:49:01,904][105692] Updated weights for policy 0, policy_version 611168 (0.0009) [2023-12-26 19:49:01,962][105692] Updated weights for policy 0, policy_version 611178 (0.0009) [2023-12-26 19:49:01,967][105620] Updated weights for policy 1, policy_version 611978 (0.0008) [2023-12-26 19:49:02,022][105692] Updated weights for policy 0, policy_version 611188 (0.0006) [2023-12-26 19:49:02,027][105620] Updated weights for policy 1, policy_version 611988 (0.0009) [2023-12-26 19:49:02,083][105620] Updated weights for policy 1, policy_version 611998 (0.0009) [2023-12-26 19:49:02,638][105692] Updated weights for policy 0, policy_version 611198 (0.0006) [2023-12-26 19:49:02,686][105692] Updated weights for policy 0, policy_version 611208 (0.0005) [2023-12-26 19:49:02,739][105692] Updated weights for policy 0, policy_version 611218 (0.0005) [2023-12-26 19:49:02,948][105620] Updated weights for policy 1, policy_version 612009 (0.0010) [2023-12-26 19:49:03,001][105620] Updated weights for policy 1, policy_version 612019 (0.0009) [2023-12-26 19:49:03,048][105620] Updated weights for policy 1, policy_version 612029 (0.0009) [2023-12-26 19:49:03,093][105620] Updated weights for policy 1, policy_version 612039 (0.0008) [2023-12-26 19:49:03,339][105692] Updated weights for policy 0, policy_version 611228 (0.0005) [2023-12-26 19:49:03,383][105692] Updated weights for policy 0, policy_version 611238 (0.0005) [2023-12-26 19:49:03,433][105692] Updated weights for policy 0, policy_version 611248 (0.0005) [2023-12-26 19:49:03,744][105620] Updated weights for policy 1, policy_version 612049 (0.0009) [2023-12-26 19:49:03,807][105620] Updated weights for policy 1, policy_version 612059 (0.0008) [2023-12-26 19:49:03,875][105620] Updated weights for policy 1, policy_version 612069 (0.0010) [2023-12-26 19:49:03,992][105692] Updated weights for policy 0, policy_version 611258 (0.0006) [2023-12-26 19:49:04,041][105692] Updated weights for policy 0, policy_version 611268 (0.0008) [2023-12-26 19:49:04,098][105692] Updated weights for policy 0, policy_version 611278 (0.0006) [2023-12-26 19:49:04,156][105692] Updated weights for policy 0, policy_version 611288 (0.0008) [2023-12-26 19:49:04,660][105620] Updated weights for policy 1, policy_version 612079 (0.0008) [2023-12-26 19:49:04,720][105620] Updated weights for policy 1, policy_version 612089 (0.0008) [2023-12-26 19:49:04,780][105620] Updated weights for policy 1, policy_version 612099 (0.0009) [2023-12-26 19:49:04,894][105692] Updated weights for policy 0, policy_version 611298 (0.0007) [2023-12-26 19:49:04,950][105692] Updated weights for policy 0, policy_version 611308 (0.0005) [2023-12-26 19:49:05,021][105692] Updated weights for policy 0, policy_version 611318 (0.0006) [2023-12-26 19:49:05,389][105620] Updated weights for policy 1, policy_version 612109 (0.0009) [2023-12-26 19:49:05,443][105620] Updated weights for policy 1, policy_version 612120 (0.0010) [2023-12-26 19:49:05,490][105620] Updated weights for policy 1, policy_version 612130 (0.0011) [2023-12-26 19:49:05,585][105692] Updated weights for policy 0, policy_version 611328 (0.0006) [2023-12-26 19:49:05,642][105692] Updated weights for policy 0, policy_version 611338 (0.0005) [2023-12-26 19:49:05,707][105692] Updated weights for policy 0, policy_version 611348 (0.0005) [2023-12-26 19:49:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 313253888. Throughput: 0: 9681.9, 1: 9846.1. Samples: 313240272. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-12-26 19:49:06,062][104569] Avg episode reward: [(0, '9171.305'), (1, '9019.148')] [2023-12-26 19:49:06,296][105620] Updated weights for policy 1, policy_version 612140 (0.0010) [2023-12-26 19:49:06,365][105620] Updated weights for policy 1, policy_version 612150 (0.0010) [2023-12-26 19:49:06,375][105692] Updated weights for policy 0, policy_version 611358 (0.0006) [2023-12-26 19:49:06,434][105692] Updated weights for policy 0, policy_version 611368 (0.0009) [2023-12-26 19:49:06,434][105620] Updated weights for policy 1, policy_version 612160 (0.0006) [2023-12-26 19:49:06,489][105692] Updated weights for policy 0, policy_version 611378 (0.0008) [2023-12-26 19:49:07,121][105620] Updated weights for policy 1, policy_version 612170 (0.0009) [2023-12-26 19:49:07,179][105620] Updated weights for policy 1, policy_version 612180 (0.0010) [2023-12-26 19:49:07,225][105620] Updated weights for policy 1, policy_version 612190 (0.0005) [2023-12-26 19:49:07,271][105620] Updated weights for policy 1, policy_version 612200 (0.0005) [2023-12-26 19:49:07,313][105692] Updated weights for policy 0, policy_version 611388 (0.0009) [2023-12-26 19:49:07,372][105692] Updated weights for policy 0, policy_version 611398 (0.0010) [2023-12-26 19:49:07,430][105692] Updated weights for policy 0, policy_version 611409 (0.0010) [2023-12-26 19:49:07,859][105620] Updated weights for policy 1, policy_version 612210 (0.0005) [2023-12-26 19:49:07,923][105620] Updated weights for policy 1, policy_version 612220 (0.0006) [2023-12-26 19:49:07,969][105620] Updated weights for policy 1, policy_version 612230 (0.0005) [2023-12-26 19:49:08,325][105692] Updated weights for policy 0, policy_version 611420 (0.0010) [2023-12-26 19:49:08,387][105692] Updated weights for policy 0, policy_version 611430 (0.0008) [2023-12-26 19:49:08,436][105692] Updated weights for policy 0, policy_version 611440 (0.0008) [2023-12-26 19:49:08,610][105620] Updated weights for policy 1, policy_version 612240 (0.0008) [2023-12-26 19:49:08,678][105620] Updated weights for policy 1, policy_version 612250 (0.0008) [2023-12-26 19:49:08,743][105620] Updated weights for policy 1, policy_version 612260 (0.0008) [2023-12-26 19:49:09,254][105692] Updated weights for policy 0, policy_version 611450 (0.0009) [2023-12-26 19:49:09,323][105692] Updated weights for policy 0, policy_version 611460 (0.0009) [2023-12-26 19:49:09,389][105692] Updated weights for policy 0, policy_version 611470 (0.0009) [2023-12-26 19:49:09,444][105692] Updated weights for policy 0, policy_version 611480 (0.0009) [2023-12-26 19:49:09,448][105620] Updated weights for policy 1, policy_version 612270 (0.0007) [2023-12-26 19:49:09,505][105620] Updated weights for policy 1, policy_version 612280 (0.0005) [2023-12-26 19:49:09,558][105620] Updated weights for policy 1, policy_version 612290 (0.0007) [2023-12-26 19:49:10,201][105692] Updated weights for policy 0, policy_version 611490 (0.0008) [2023-12-26 19:49:10,261][105692] Updated weights for policy 0, policy_version 611500 (0.0008) [2023-12-26 19:49:10,311][105620] Updated weights for policy 1, policy_version 612300 (0.0009) [2023-12-26 19:49:10,323][105692] Updated weights for policy 0, policy_version 611510 (0.0008) [2023-12-26 19:49:10,378][105620] Updated weights for policy 1, policy_version 612310 (0.0011) [2023-12-26 19:49:10,444][105620] Updated weights for policy 1, policy_version 612320 (0.0006) [2023-12-26 19:49:11,030][105692] Updated weights for policy 0, policy_version 611520 (0.0009) [2023-12-26 19:49:11,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 313344000. Throughput: 0: 9615.7, 1: 9833.0. Samples: 313357008. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-12-26 19:49:11,062][104569] Avg episode reward: [(0, '9171.401'), (1, '8878.112')] [2023-12-26 19:49:11,089][105620] Updated weights for policy 1, policy_version 612330 (0.0006) [2023-12-26 19:49:11,092][105692] Updated weights for policy 0, policy_version 611530 (0.0008) [2023-12-26 19:49:11,155][105692] Updated weights for policy 0, policy_version 611540 (0.0008) [2023-12-26 19:49:11,155][105620] Updated weights for policy 1, policy_version 612340 (0.0011) [2023-12-26 19:49:11,214][105620] Updated weights for policy 1, policy_version 612350 (0.0011) [2023-12-26 19:49:11,280][105620] Updated weights for policy 1, policy_version 612360 (0.0011) [2023-12-26 19:49:11,929][105692] Updated weights for policy 0, policy_version 611550 (0.0007) [2023-12-26 19:49:11,991][105692] Updated weights for policy 0, policy_version 611560 (0.0006) [2023-12-26 19:49:12,032][105620] Updated weights for policy 1, policy_version 612370 (0.0010) [2023-12-26 19:49:12,055][105692] Updated weights for policy 0, policy_version 611570 (0.0005) [2023-12-26 19:49:12,097][105620] Updated weights for policy 1, policy_version 612380 (0.0008) [2023-12-26 19:49:12,157][105620] Updated weights for policy 1, policy_version 612390 (0.0011) [2023-12-26 19:49:12,722][105692] Updated weights for policy 0, policy_version 611580 (0.0009) [2023-12-26 19:49:12,791][105692] Updated weights for policy 0, policy_version 611590 (0.0010) [2023-12-26 19:49:12,845][105620] Updated weights for policy 1, policy_version 612400 (0.0006) [2023-12-26 19:49:12,849][105692] Updated weights for policy 0, policy_version 611600 (0.0010) [2023-12-26 19:49:12,908][105620] Updated weights for policy 1, policy_version 612410 (0.0005) [2023-12-26 19:49:12,967][105620] Updated weights for policy 1, policy_version 612420 (0.0005) [2023-12-26 19:49:13,586][105692] Updated weights for policy 0, policy_version 611610 (0.0010) [2023-12-26 19:49:13,634][105620] Updated weights for policy 1, policy_version 612430 (0.0005) [2023-12-26 19:49:13,638][105692] Updated weights for policy 0, policy_version 611620 (0.0010) [2023-12-26 19:49:13,687][105692] Updated weights for policy 0, policy_version 611630 (0.0010) [2023-12-26 19:49:13,703][105620] Updated weights for policy 1, policy_version 612440 (0.0005) [2023-12-26 19:49:13,745][105692] Updated weights for policy 0, policy_version 611640 (0.0010) [2023-12-26 19:49:13,773][105620] Updated weights for policy 1, policy_version 612450 (0.0005) [2023-12-26 19:49:14,324][105692] Updated weights for policy 0, policy_version 611650 (0.0006) [2023-12-26 19:49:14,364][105620] Updated weights for policy 1, policy_version 612460 (0.0007) [2023-12-26 19:49:14,379][105692] Updated weights for policy 0, policy_version 611660 (0.0006) [2023-12-26 19:49:14,429][105620] Updated weights for policy 1, policy_version 612470 (0.0010) [2023-12-26 19:49:14,434][105692] Updated weights for policy 0, policy_version 611670 (0.0010) [2023-12-26 19:49:14,477][105620] Updated weights for policy 1, policy_version 612480 (0.0010) [2023-12-26 19:49:15,123][105692] Updated weights for policy 0, policy_version 611680 (0.0011) [2023-12-26 19:49:15,171][105620] Updated weights for policy 1, policy_version 612490 (0.0010) [2023-12-26 19:49:15,184][105692] Updated weights for policy 0, policy_version 611690 (0.0010) [2023-12-26 19:49:15,235][105620] Updated weights for policy 1, policy_version 612500 (0.0011) [2023-12-26 19:49:15,244][105692] Updated weights for policy 0, policy_version 611700 (0.0011) [2023-12-26 19:49:15,303][105620] Updated weights for policy 1, policy_version 612510 (0.0011) [2023-12-26 19:49:15,366][105620] Updated weights for policy 1, policy_version 612520 (0.0011) [2023-12-26 19:49:15,975][105620] Updated weights for policy 1, policy_version 612530 (0.0009) [2023-12-26 19:49:15,985][105692] Updated weights for policy 0, policy_version 611710 (0.0011) [2023-12-26 19:49:16,029][105692] Updated weights for policy 0, policy_version 611720 (0.0010) [2023-12-26 19:49:16,035][105620] Updated weights for policy 1, policy_version 612540 (0.0009) [2023-12-26 19:49:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 313442304. Throughput: 0: 9541.3, 1: 9777.7. Samples: 313416000. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-12-26 19:49:16,062][104569] Avg episode reward: [(0, '9171.433'), (1, '9117.496')] [2023-12-26 19:49:16,084][105692] Updated weights for policy 0, policy_version 611730 (0.0010) [2023-12-26 19:49:16,090][105620] Updated weights for policy 1, policy_version 612550 (0.0009) [2023-12-26 19:49:16,100][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000612552_156827648.pth... [2023-12-26 19:49:16,104][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000611400_156532736.pth [2023-12-26 19:49:16,114][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000611736_156631040.pth... [2023-12-26 19:49:16,119][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000610584_156336128.pth [2023-12-26 19:49:16,712][105620] Updated weights for policy 1, policy_version 612560 (0.0010) [2023-12-26 19:49:16,760][105620] Updated weights for policy 1, policy_version 612570 (0.0010) [2023-12-26 19:49:16,776][105692] Updated weights for policy 0, policy_version 611740 (0.0008) [2023-12-26 19:49:16,808][105620] Updated weights for policy 1, policy_version 612580 (0.0010) [2023-12-26 19:49:16,829][105692] Updated weights for policy 0, policy_version 611750 (0.0005) [2023-12-26 19:49:16,876][105692] Updated weights for policy 0, policy_version 611760 (0.0005) [2023-12-26 19:49:17,526][105692] Updated weights for policy 0, policy_version 611770 (0.0006) [2023-12-26 19:49:17,581][105620] Updated weights for policy 1, policy_version 612590 (0.0010) [2023-12-26 19:49:17,591][105692] Updated weights for policy 0, policy_version 611780 (0.0006) [2023-12-26 19:49:17,637][105620] Updated weights for policy 1, policy_version 612600 (0.0010) [2023-12-26 19:49:17,643][105692] Updated weights for policy 0, policy_version 611790 (0.0006) [2023-12-26 19:49:17,689][105620] Updated weights for policy 1, policy_version 612610 (0.0008) [2023-12-26 19:49:17,693][105692] Updated weights for policy 0, policy_version 611800 (0.0007) [2023-12-26 19:49:18,418][105692] Updated weights for policy 0, policy_version 611810 (0.0008) [2023-12-26 19:49:18,437][105620] Updated weights for policy 1, policy_version 612620 (0.0008) [2023-12-26 19:49:18,477][105692] Updated weights for policy 0, policy_version 611820 (0.0008) [2023-12-26 19:49:18,496][105620] Updated weights for policy 1, policy_version 612630 (0.0010) [2023-12-26 19:49:18,535][105692] Updated weights for policy 0, policy_version 611830 (0.0008) [2023-12-26 19:49:18,548][105620] Updated weights for policy 1, policy_version 612640 (0.0010) [2023-12-26 19:49:19,233][105692] Updated weights for policy 0, policy_version 611840 (0.0008) [2023-12-26 19:49:19,289][105692] Updated weights for policy 0, policy_version 611850 (0.0006) [2023-12-26 19:49:19,305][105620] Updated weights for policy 1, policy_version 612650 (0.0010) [2023-12-26 19:49:19,353][105692] Updated weights for policy 0, policy_version 611860 (0.0007) [2023-12-26 19:49:19,371][105620] Updated weights for policy 1, policy_version 612660 (0.0011) [2023-12-26 19:49:19,434][105620] Updated weights for policy 1, policy_version 612670 (0.0011) [2023-12-26 19:49:19,498][105620] Updated weights for policy 1, policy_version 612680 (0.0009) [2023-12-26 19:49:20,139][105692] Updated weights for policy 0, policy_version 611870 (0.0007) [2023-12-26 19:49:20,188][105620] Updated weights for policy 1, policy_version 612690 (0.0011) [2023-12-26 19:49:20,196][105692] Updated weights for policy 0, policy_version 611880 (0.0008) [2023-12-26 19:49:20,247][105620] Updated weights for policy 1, policy_version 612700 (0.0011) [2023-12-26 19:49:20,253][105692] Updated weights for policy 0, policy_version 611890 (0.0006) [2023-12-26 19:49:20,306][105620] Updated weights for policy 1, policy_version 612710 (0.0011) [2023-12-26 19:49:21,040][105692] Updated weights for policy 0, policy_version 611900 (0.0006) [2023-12-26 19:49:21,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 313540608. Throughput: 0: 9673.4, 1: 9738.2. Samples: 313536696. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-12-26 19:49:21,063][104569] Avg episode reward: [(0, '9355.215'), (1, '9349.535')] [2023-12-26 19:49:21,063][105620] Updated weights for policy 1, policy_version 612720 (0.0011) [2023-12-26 19:49:21,101][105692] Updated weights for policy 0, policy_version 611910 (0.0007) [2023-12-26 19:49:21,126][105620] Updated weights for policy 1, policy_version 612730 (0.0010) [2023-12-26 19:49:21,164][105692] Updated weights for policy 0, policy_version 611920 (0.0008) [2023-12-26 19:49:21,190][105620] Updated weights for policy 1, policy_version 612740 (0.0011) [2023-12-26 19:49:21,860][105620] Updated weights for policy 1, policy_version 612750 (0.0008) [2023-12-26 19:49:21,912][105620] Updated weights for policy 1, policy_version 612760 (0.0008) [2023-12-26 19:49:21,960][105692] Updated weights for policy 0, policy_version 611930 (0.0006) [2023-12-26 19:49:21,966][105620] Updated weights for policy 1, policy_version 612770 (0.0008) [2023-12-26 19:49:22,012][105692] Updated weights for policy 0, policy_version 611940 (0.0008) [2023-12-26 19:49:22,074][105692] Updated weights for policy 0, policy_version 611950 (0.0009) [2023-12-26 19:49:22,132][105692] Updated weights for policy 0, policy_version 611960 (0.0009) [2023-12-26 19:49:22,586][105620] Updated weights for policy 1, policy_version 612780 (0.0006) [2023-12-26 19:49:22,646][105620] Updated weights for policy 1, policy_version 612790 (0.0008) [2023-12-26 19:49:22,707][105620] Updated weights for policy 1, policy_version 612800 (0.0008) [2023-12-26 19:49:22,984][105692] Updated weights for policy 0, policy_version 611970 (0.0009) [2023-12-26 19:49:23,042][105692] Updated weights for policy 0, policy_version 611980 (0.0009) [2023-12-26 19:49:23,100][105692] Updated weights for policy 0, policy_version 611990 (0.0010) [2023-12-26 19:49:23,331][105620] Updated weights for policy 1, policy_version 612810 (0.0008) [2023-12-26 19:49:23,401][105620] Updated weights for policy 1, policy_version 612820 (0.0006) [2023-12-26 19:49:23,468][105620] Updated weights for policy 1, policy_version 612830 (0.0006) [2023-12-26 19:49:23,523][105620] Updated weights for policy 1, policy_version 612840 (0.0009) [2023-12-26 19:49:23,938][105692] Updated weights for policy 0, policy_version 612000 (0.0008) [2023-12-26 19:49:24,003][105692] Updated weights for policy 0, policy_version 612010 (0.0008) [2023-12-26 19:49:24,064][105692] Updated weights for policy 0, policy_version 612020 (0.0008) [2023-12-26 19:49:24,218][105620] Updated weights for policy 1, policy_version 612850 (0.0005) [2023-12-26 19:49:24,279][105620] Updated weights for policy 1, policy_version 612860 (0.0006) [2023-12-26 19:49:24,329][105620] Updated weights for policy 1, policy_version 612870 (0.0006) [2023-12-26 19:49:24,811][105692] Updated weights for policy 0, policy_version 612030 (0.0008) [2023-12-26 19:49:24,868][105692] Updated weights for policy 0, policy_version 612040 (0.0008) [2023-12-26 19:49:24,927][105692] Updated weights for policy 0, policy_version 612050 (0.0008) [2023-12-26 19:49:24,973][105620] Updated weights for policy 1, policy_version 612880 (0.0009) [2023-12-26 19:49:25,035][105620] Updated weights for policy 1, policy_version 612890 (0.0010) [2023-12-26 19:49:25,093][105620] Updated weights for policy 1, policy_version 612900 (0.0010) [2023-12-26 19:49:25,726][105692] Updated weights for policy 0, policy_version 612060 (0.0010) [2023-12-26 19:49:25,762][105620] Updated weights for policy 1, policy_version 612910 (0.0008) [2023-12-26 19:49:25,779][105692] Updated weights for policy 0, policy_version 612070 (0.0008) [2023-12-26 19:49:25,818][105620] Updated weights for policy 1, policy_version 612920 (0.0006) [2023-12-26 19:49:25,835][105692] Updated weights for policy 0, policy_version 612080 (0.0010) [2023-12-26 19:49:25,878][105620] Updated weights for policy 1, policy_version 612930 (0.0010) [2023-12-26 19:49:26,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 313647104. Throughput: 0: 9619.7, 1: 9806.8. Samples: 313651116. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-12-26 19:49:26,062][104569] Avg episode reward: [(0, '9356.311'), (1, '9258.185')] [2023-12-26 19:49:26,600][105692] Updated weights for policy 0, policy_version 612090 (0.0007) [2023-12-26 19:49:26,600][105620] Updated weights for policy 1, policy_version 612940 (0.0009) [2023-12-26 19:49:26,648][105692] Updated weights for policy 0, policy_version 612100 (0.0007) [2023-12-26 19:49:26,668][105620] Updated weights for policy 1, policy_version 612950 (0.0008) [2023-12-26 19:49:26,708][105692] Updated weights for policy 0, policy_version 612110 (0.0007) [2023-12-26 19:49:26,729][105620] Updated weights for policy 1, policy_version 612960 (0.0008) [2023-12-26 19:49:26,762][105692] Updated weights for policy 0, policy_version 612120 (0.0006) [2023-12-26 19:49:27,322][105620] Updated weights for policy 1, policy_version 612970 (0.0007) [2023-12-26 19:49:27,389][105620] Updated weights for policy 1, policy_version 612980 (0.0006) [2023-12-26 19:49:27,459][105620] Updated weights for policy 1, policy_version 612990 (0.0007) [2023-12-26 19:49:27,520][105620] Updated weights for policy 1, policy_version 613000 (0.0008) [2023-12-26 19:49:27,562][105692] Updated weights for policy 0, policy_version 612130 (0.0007) [2023-12-26 19:49:27,627][105692] Updated weights for policy 0, policy_version 612140 (0.0009) [2023-12-26 19:49:27,685][105692] Updated weights for policy 0, policy_version 612150 (0.0009) [2023-12-26 19:49:28,144][105620] Updated weights for policy 1, policy_version 613010 (0.0009) [2023-12-26 19:49:28,206][105620] Updated weights for policy 1, policy_version 613020 (0.0009) [2023-12-26 19:49:28,270][105620] Updated weights for policy 1, policy_version 613030 (0.0008) [2023-12-26 19:49:28,447][105692] Updated weights for policy 0, policy_version 612160 (0.0010) [2023-12-26 19:49:28,505][105692] Updated weights for policy 0, policy_version 612170 (0.0009) [2023-12-26 19:49:28,557][105692] Updated weights for policy 0, policy_version 612180 (0.0009) [2023-12-26 19:49:28,993][105620] Updated weights for policy 1, policy_version 613040 (0.0009) [2023-12-26 19:49:29,050][105620] Updated weights for policy 1, policy_version 613050 (0.0009) [2023-12-26 19:49:29,096][105620] Updated weights for policy 1, policy_version 613060 (0.0009) [2023-12-26 19:49:29,324][105692] Updated weights for policy 0, policy_version 612190 (0.0008) [2023-12-26 19:49:29,386][105692] Updated weights for policy 0, policy_version 612200 (0.0006) [2023-12-26 19:49:29,452][105692] Updated weights for policy 0, policy_version 612210 (0.0005) [2023-12-26 19:49:29,819][105620] Updated weights for policy 1, policy_version 613070 (0.0009) [2023-12-26 19:49:29,881][105620] Updated weights for policy 1, policy_version 613080 (0.0008) [2023-12-26 19:49:29,940][105620] Updated weights for policy 1, policy_version 613090 (0.0007) [2023-12-26 19:49:30,039][105692] Updated weights for policy 0, policy_version 612220 (0.0006) [2023-12-26 19:49:30,095][105692] Updated weights for policy 0, policy_version 612230 (0.0008) [2023-12-26 19:49:30,150][105692] Updated weights for policy 0, policy_version 612240 (0.0008) [2023-12-26 19:49:30,728][105692] Updated weights for policy 0, policy_version 612250 (0.0008) [2023-12-26 19:49:30,775][105692] Updated weights for policy 0, policy_version 612260 (0.0008) [2023-12-26 19:49:30,789][105620] Updated weights for policy 1, policy_version 613100 (0.0008) [2023-12-26 19:49:30,837][105692] Updated weights for policy 0, policy_version 612270 (0.0007) [2023-12-26 19:49:30,847][105620] Updated weights for policy 1, policy_version 613110 (0.0007) [2023-12-26 19:49:30,895][105692] Updated weights for policy 0, policy_version 612280 (0.0007) [2023-12-26 19:49:30,901][105620] Updated weights for policy 1, policy_version 613120 (0.0006) [2023-12-26 19:49:31,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 313745408. Throughput: 0: 9587.0, 1: 9865.5. Samples: 313708640. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-12-26 19:49:31,062][104569] Avg episode reward: [(0, '9356.443'), (1, '9166.867')] [2023-12-26 19:49:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000612280_156770304.pth... [2023-12-26 19:49:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000613128_156975104.pth... [2023-12-26 19:49:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000611160_156483584.pth [2023-12-26 19:49:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000611944_156672000.pth [2023-12-26 19:49:31,677][105620] Updated weights for policy 1, policy_version 613130 (0.0008) [2023-12-26 19:49:31,679][105692] Updated weights for policy 0, policy_version 612290 (0.0008) [2023-12-26 19:49:31,741][105620] Updated weights for policy 1, policy_version 613140 (0.0008) [2023-12-26 19:49:31,743][105692] Updated weights for policy 0, policy_version 612300 (0.0008) [2023-12-26 19:49:31,799][105692] Updated weights for policy 0, policy_version 612310 (0.0008) [2023-12-26 19:49:31,804][105620] Updated weights for policy 1, policy_version 613150 (0.0009) [2023-12-26 19:49:31,864][105620] Updated weights for policy 1, policy_version 613160 (0.0009) [2023-12-26 19:49:32,533][105620] Updated weights for policy 1, policy_version 613170 (0.0007) [2023-12-26 19:49:32,599][105620] Updated weights for policy 1, policy_version 613180 (0.0006) [2023-12-26 19:49:32,608][105692] Updated weights for policy 0, policy_version 612320 (0.0006) [2023-12-26 19:49:32,662][105620] Updated weights for policy 1, policy_version 613190 (0.0005) [2023-12-26 19:49:32,670][105692] Updated weights for policy 0, policy_version 612330 (0.0005) [2023-12-26 19:49:32,732][105692] Updated weights for policy 0, policy_version 612340 (0.0006) [2023-12-26 19:49:33,178][105620] Updated weights for policy 1, policy_version 613200 (0.0008) [2023-12-26 19:49:33,227][105620] Updated weights for policy 1, policy_version 613210 (0.0008) [2023-12-26 19:49:33,278][105620] Updated weights for policy 1, policy_version 613220 (0.0005) [2023-12-26 19:49:33,463][105692] Updated weights for policy 0, policy_version 612350 (0.0005) [2023-12-26 19:49:33,510][105692] Updated weights for policy 0, policy_version 612360 (0.0010) [2023-12-26 19:49:33,566][105692] Updated weights for policy 0, policy_version 612370 (0.0005) [2023-12-26 19:49:33,975][105620] Updated weights for policy 1, policy_version 613230 (0.0005) [2023-12-26 19:49:34,023][105620] Updated weights for policy 1, policy_version 613240 (0.0005) [2023-12-26 19:49:34,069][105620] Updated weights for policy 1, policy_version 613250 (0.0005) [2023-12-26 19:49:34,214][105692] Updated weights for policy 0, policy_version 612380 (0.0007) [2023-12-26 19:49:34,270][105692] Updated weights for policy 0, policy_version 612390 (0.0011) [2023-12-26 19:49:34,326][105692] Updated weights for policy 0, policy_version 612400 (0.0011) [2023-12-26 19:49:34,797][105620] Updated weights for policy 1, policy_version 613260 (0.0008) [2023-12-26 19:49:34,844][105620] Updated weights for policy 1, policy_version 613270 (0.0005) [2023-12-26 19:49:34,892][105620] Updated weights for policy 1, policy_version 613280 (0.0005) [2023-12-26 19:49:34,926][105692] Updated weights for policy 0, policy_version 612410 (0.0010) [2023-12-26 19:49:34,978][105692] Updated weights for policy 0, policy_version 612420 (0.0005) [2023-12-26 19:49:35,040][105692] Updated weights for policy 0, policy_version 612430 (0.0005) [2023-12-26 19:49:35,101][105692] Updated weights for policy 0, policy_version 612440 (0.0005) [2023-12-26 19:49:35,527][105620] Updated weights for policy 1, policy_version 613290 (0.0005) [2023-12-26 19:49:35,583][105620] Updated weights for policy 1, policy_version 613300 (0.0005) [2023-12-26 19:49:35,609][105692] Updated weights for policy 0, policy_version 612450 (0.0011) [2023-12-26 19:49:35,647][105620] Updated weights for policy 1, policy_version 613310 (0.0006) [2023-12-26 19:49:35,654][105692] Updated weights for policy 0, policy_version 612460 (0.0010) [2023-12-26 19:49:35,696][105620] Updated weights for policy 1, policy_version 613320 (0.0005) [2023-12-26 19:49:35,701][105692] Updated weights for policy 0, policy_version 612470 (0.0010) [2023-12-26 19:49:36,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 313843712. Throughput: 0: 9668.9, 1: 9845.9. Samples: 313827536. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-12-26 19:49:36,063][104569] Avg episode reward: [(0, '9264.159'), (1, '9168.391')] [2023-12-26 19:49:36,271][105620] Updated weights for policy 1, policy_version 613330 (0.0008) [2023-12-26 19:49:36,329][105620] Updated weights for policy 1, policy_version 613340 (0.0008) [2023-12-26 19:49:36,389][105692] Updated weights for policy 0, policy_version 612480 (0.0011) [2023-12-26 19:49:36,391][105620] Updated weights for policy 1, policy_version 613350 (0.0006) [2023-12-26 19:49:36,441][105692] Updated weights for policy 0, policy_version 612490 (0.0010) [2023-12-26 19:49:36,501][105692] Updated weights for policy 0, policy_version 612500 (0.0010) [2023-12-26 19:49:37,160][105620] Updated weights for policy 1, policy_version 613360 (0.0010) [2023-12-26 19:49:37,169][105692] Updated weights for policy 0, policy_version 612510 (0.0010) [2023-12-26 19:49:37,219][105620] Updated weights for policy 1, policy_version 613370 (0.0010) [2023-12-26 19:49:37,228][105692] Updated weights for policy 0, policy_version 612520 (0.0011) [2023-12-26 19:49:37,277][105620] Updated weights for policy 1, policy_version 613380 (0.0006) [2023-12-26 19:49:37,289][105692] Updated weights for policy 0, policy_version 612530 (0.0011) [2023-12-26 19:49:37,889][105692] Updated weights for policy 0, policy_version 612540 (0.0011) [2023-12-26 19:49:37,917][105620] Updated weights for policy 1, policy_version 613390 (0.0008) [2023-12-26 19:49:37,938][105692] Updated weights for policy 0, policy_version 612550 (0.0010) [2023-12-26 19:49:37,976][105620] Updated weights for policy 1, policy_version 613400 (0.0007) [2023-12-26 19:49:37,987][105692] Updated weights for policy 0, policy_version 612560 (0.0011) [2023-12-26 19:49:38,036][105620] Updated weights for policy 1, policy_version 613410 (0.0009) [2023-12-26 19:49:38,663][105620] Updated weights for policy 1, policy_version 613420 (0.0010) [2023-12-26 19:49:38,727][105620] Updated weights for policy 1, policy_version 613430 (0.0008) [2023-12-26 19:49:38,758][105692] Updated weights for policy 0, policy_version 612570 (0.0011) [2023-12-26 19:49:38,791][105620] Updated weights for policy 1, policy_version 613440 (0.0006) [2023-12-26 19:49:38,823][105692] Updated weights for policy 0, policy_version 612580 (0.0008) [2023-12-26 19:49:38,885][105692] Updated weights for policy 0, policy_version 612590 (0.0006) [2023-12-26 19:49:38,938][105692] Updated weights for policy 0, policy_version 612600 (0.0006) [2023-12-26 19:49:39,505][105620] Updated weights for policy 1, policy_version 613450 (0.0005) [2023-12-26 19:49:39,568][105620] Updated weights for policy 1, policy_version 613460 (0.0005) [2023-12-26 19:49:39,629][105620] Updated weights for policy 1, policy_version 613470 (0.0006) [2023-12-26 19:49:39,636][105692] Updated weights for policy 0, policy_version 612610 (0.0011) [2023-12-26 19:49:39,691][105620] Updated weights for policy 1, policy_version 613480 (0.0006) [2023-12-26 19:49:39,696][105692] Updated weights for policy 0, policy_version 612620 (0.0010) [2023-12-26 19:49:39,753][105692] Updated weights for policy 0, policy_version 612630 (0.0009) [2023-12-26 19:49:40,318][105620] Updated weights for policy 1, policy_version 613490 (0.0005) [2023-12-26 19:49:40,378][105620] Updated weights for policy 1, policy_version 613500 (0.0007) [2023-12-26 19:49:40,430][105620] Updated weights for policy 1, policy_version 613510 (0.0009) [2023-12-26 19:49:40,538][105692] Updated weights for policy 0, policy_version 612640 (0.0008) [2023-12-26 19:49:40,603][105692] Updated weights for policy 0, policy_version 612650 (0.0006) [2023-12-26 19:49:40,650][105692] Updated weights for policy 0, policy_version 612660 (0.0008) [2023-12-26 19:49:41,060][105620] Updated weights for policy 1, policy_version 613520 (0.0010) [2023-12-26 19:49:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 313942016. Throughput: 0: 9740.7, 1: 9990.3. Samples: 313952624. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-12-26 19:49:41,062][104569] Avg episode reward: [(0, '9357.451'), (1, '9260.653')] [2023-12-26 19:49:41,114][105620] Updated weights for policy 1, policy_version 613530 (0.0009) [2023-12-26 19:49:41,187][105620] Updated weights for policy 1, policy_version 613540 (0.0009) [2023-12-26 19:49:41,504][105692] Updated weights for policy 0, policy_version 612670 (0.0009) [2023-12-26 19:49:41,551][105692] Updated weights for policy 0, policy_version 612680 (0.0009) [2023-12-26 19:49:41,605][105692] Updated weights for policy 0, policy_version 612690 (0.0009) [2023-12-26 19:49:41,878][105620] Updated weights for policy 1, policy_version 613550 (0.0009) [2023-12-26 19:49:41,936][105620] Updated weights for policy 1, policy_version 613560 (0.0009) [2023-12-26 19:49:41,998][105620] Updated weights for policy 1, policy_version 613570 (0.0009) [2023-12-26 19:49:42,400][105692] Updated weights for policy 0, policy_version 612700 (0.0009) [2023-12-26 19:49:42,470][105692] Updated weights for policy 0, policy_version 612710 (0.0008) [2023-12-26 19:49:42,543][105692] Updated weights for policy 0, policy_version 612720 (0.0009) [2023-12-26 19:49:42,753][105620] Updated weights for policy 1, policy_version 613580 (0.0009) [2023-12-26 19:49:42,814][105620] Updated weights for policy 1, policy_version 613590 (0.0009) [2023-12-26 19:49:42,875][105620] Updated weights for policy 1, policy_version 613600 (0.0010) [2023-12-26 19:49:43,241][105692] Updated weights for policy 0, policy_version 612730 (0.0008) [2023-12-26 19:49:43,298][105692] Updated weights for policy 0, policy_version 612740 (0.0010) [2023-12-26 19:49:43,360][105692] Updated weights for policy 0, policy_version 612751 (0.0010) [2023-12-26 19:49:43,519][105620] Updated weights for policy 1, policy_version 613610 (0.0009) [2023-12-26 19:49:43,581][105620] Updated weights for policy 1, policy_version 613620 (0.0008) [2023-12-26 19:49:43,635][105620] Updated weights for policy 1, policy_version 613630 (0.0009) [2023-12-26 19:49:43,688][105620] Updated weights for policy 1, policy_version 613640 (0.0010) [2023-12-26 19:49:44,073][105692] Updated weights for policy 0, policy_version 612761 (0.0008) [2023-12-26 19:49:44,130][105692] Updated weights for policy 0, policy_version 612771 (0.0006) [2023-12-26 19:49:44,197][105692] Updated weights for policy 0, policy_version 612781 (0.0010) [2023-12-26 19:49:44,256][105692] Updated weights for policy 0, policy_version 612791 (0.0010) [2023-12-26 19:49:44,341][105620] Updated weights for policy 1, policy_version 613650 (0.0008) [2023-12-26 19:49:44,401][105620] Updated weights for policy 1, policy_version 613660 (0.0008) [2023-12-26 19:49:44,461][105620] Updated weights for policy 1, policy_version 613670 (0.0009) [2023-12-26 19:49:44,894][105692] Updated weights for policy 0, policy_version 612801 (0.0008) [2023-12-26 19:49:44,957][105692] Updated weights for policy 0, policy_version 612811 (0.0009) [2023-12-26 19:49:45,019][105692] Updated weights for policy 0, policy_version 612821 (0.0009) [2023-12-26 19:49:45,273][105620] Updated weights for policy 1, policy_version 613680 (0.0010) [2023-12-26 19:49:45,335][105620] Updated weights for policy 1, policy_version 613690 (0.0008) [2023-12-26 19:49:45,397][105620] Updated weights for policy 1, policy_version 613700 (0.0009) [2023-12-26 19:49:45,699][105692] Updated weights for policy 0, policy_version 612831 (0.0007) [2023-12-26 19:49:45,767][105692] Updated weights for policy 0, policy_version 612841 (0.0008) [2023-12-26 19:49:45,815][105692] Updated weights for policy 0, policy_version 612851 (0.0005) [2023-12-26 19:49:46,062][104569] Fps is (10 sec: 19660.0, 60 sec: 19524.1, 300 sec: 19521.9). Total num frames: 314040320. Throughput: 0: 9684.5, 1: 10007.3. Samples: 314008496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:49:46,064][104569] Avg episode reward: [(0, '9266.909'), (1, '9353.860')] [2023-12-26 19:49:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000612856_156917760.pth... [2023-12-26 19:49:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000613704_157122560.pth... [2023-12-26 19:49:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000612552_156827648.pth [2023-12-26 19:49:46,087][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000611736_156631040.pth [2023-12-26 19:49:46,177][105620] Updated weights for policy 1, policy_version 613710 (0.0010) [2023-12-26 19:49:46,232][105620] Updated weights for policy 1, policy_version 613720 (0.0009) [2023-12-26 19:49:46,288][105620] Updated weights for policy 1, policy_version 613730 (0.0009) [2023-12-26 19:49:46,497][105692] Updated weights for policy 0, policy_version 612861 (0.0005) [2023-12-26 19:49:46,550][105692] Updated weights for policy 0, policy_version 612871 (0.0005) [2023-12-26 19:49:46,596][105692] Updated weights for policy 0, policy_version 612881 (0.0005) [2023-12-26 19:49:47,073][105620] Updated weights for policy 1, policy_version 613740 (0.0007) [2023-12-26 19:49:47,138][105620] Updated weights for policy 1, policy_version 613750 (0.0006) [2023-12-26 19:49:47,198][105620] Updated weights for policy 1, policy_version 613760 (0.0008) [2023-12-26 19:49:47,212][105692] Updated weights for policy 0, policy_version 612891 (0.0005) [2023-12-26 19:49:47,260][105692] Updated weights for policy 0, policy_version 612901 (0.0007) [2023-12-26 19:49:47,307][105692] Updated weights for policy 0, policy_version 612911 (0.0009) [2023-12-26 19:49:47,810][105620] Updated weights for policy 1, policy_version 613770 (0.0007) [2023-12-26 19:49:47,860][105692] Updated weights for policy 0, policy_version 612921 (0.0006) [2023-12-26 19:49:47,867][105620] Updated weights for policy 1, policy_version 613780 (0.0009) [2023-12-26 19:49:47,924][105620] Updated weights for policy 1, policy_version 613790 (0.0007) [2023-12-26 19:49:47,924][105692] Updated weights for policy 0, policy_version 612931 (0.0007) [2023-12-26 19:49:47,983][105692] Updated weights for policy 0, policy_version 612941 (0.0008) [2023-12-26 19:49:47,986][105620] Updated weights for policy 1, policy_version 613800 (0.0007) [2023-12-26 19:49:48,046][105692] Updated weights for policy 0, policy_version 612951 (0.0006) [2023-12-26 19:49:48,643][105692] Updated weights for policy 0, policy_version 612961 (0.0006) [2023-12-26 19:49:48,695][105692] Updated weights for policy 0, policy_version 612971 (0.0007) [2023-12-26 19:49:48,751][105692] Updated weights for policy 0, policy_version 612981 (0.0006) [2023-12-26 19:49:48,885][105620] Updated weights for policy 1, policy_version 613810 (0.0009) [2023-12-26 19:49:48,943][105620] Updated weights for policy 1, policy_version 613820 (0.0010) [2023-12-26 19:49:49,000][105620] Updated weights for policy 1, policy_version 613830 (0.0008) [2023-12-26 19:49:49,399][105692] Updated weights for policy 0, policy_version 612991 (0.0012) [2023-12-26 19:49:49,459][105692] Updated weights for policy 0, policy_version 613001 (0.0010) [2023-12-26 19:49:49,519][105692] Updated weights for policy 0, policy_version 613011 (0.0010) [2023-12-26 19:49:49,881][105620] Updated weights for policy 1, policy_version 613840 (0.0010) [2023-12-26 19:49:49,945][105620] Updated weights for policy 1, policy_version 613850 (0.0008) [2023-12-26 19:49:50,020][105620] Updated weights for policy 1, policy_version 613860 (0.0009) [2023-12-26 19:49:50,174][105692] Updated weights for policy 0, policy_version 613021 (0.0007) [2023-12-26 19:49:50,235][105692] Updated weights for policy 0, policy_version 613031 (0.0006) [2023-12-26 19:49:50,239][105585] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000003 [2023-12-26 19:49:50,782][105620] Updated weights for policy 1, policy_version 613870 (0.0007) [2023-12-26 19:49:50,845][105620] Updated weights for policy 1, policy_version 613880 (0.0008) [2023-12-26 19:49:50,912][105620] Updated weights for policy 1, policy_version 613890 (0.0007) [2023-12-26 19:49:50,991][105692] Updated weights for policy 0, policy_version 613041 (0.0009) [2023-12-26 19:49:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 314138624. Throughput: 0: 9742.8, 1: 9967.8. Samples: 314127248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:49:51,062][104569] Avg episode reward: [(0, '9174.140'), (1, '9354.318')] [2023-12-26 19:49:51,063][105692] Updated weights for policy 0, policy_version 613051 (0.0008) [2023-12-26 19:49:51,125][105692] Updated weights for policy 0, policy_version 613061 (0.0009) [2023-12-26 19:49:51,643][105620] Updated weights for policy 1, policy_version 613900 (0.0007) [2023-12-26 19:49:51,711][105620] Updated weights for policy 1, policy_version 613910 (0.0009) [2023-12-26 19:49:51,775][105620] Updated weights for policy 1, policy_version 613920 (0.0008) [2023-12-26 19:49:51,820][105692] Updated weights for policy 0, policy_version 613071 (0.0006) [2023-12-26 19:49:51,880][105692] Updated weights for policy 0, policy_version 613081 (0.0008) [2023-12-26 19:49:51,942][105692] Updated weights for policy 0, policy_version 613091 (0.0008) [2023-12-26 19:49:52,613][105692] Updated weights for policy 0, policy_version 613101 (0.0008) [2023-12-26 19:49:52,622][105620] Updated weights for policy 1, policy_version 613930 (0.0009) [2023-12-26 19:49:52,671][105692] Updated weights for policy 0, policy_version 613111 (0.0005) [2023-12-26 19:49:52,682][105620] Updated weights for policy 1, policy_version 613940 (0.0008) [2023-12-26 19:49:52,725][105692] Updated weights for policy 0, policy_version 613121 (0.0006) [2023-12-26 19:49:52,742][105620] Updated weights for policy 1, policy_version 613950 (0.0009) [2023-12-26 19:49:52,801][105620] Updated weights for policy 1, policy_version 613960 (0.0008) [2023-12-26 19:49:53,349][105692] Updated weights for policy 0, policy_version 613131 (0.0006) [2023-12-26 19:49:53,399][105692] Updated weights for policy 0, policy_version 613141 (0.0005) [2023-12-26 19:49:53,457][105692] Updated weights for policy 0, policy_version 613151 (0.0005) [2023-12-26 19:49:53,651][105620] Updated weights for policy 1, policy_version 613970 (0.0010) [2023-12-26 19:49:53,720][105620] Updated weights for policy 1, policy_version 613980 (0.0009) [2023-12-26 19:49:53,786][105620] Updated weights for policy 1, policy_version 613990 (0.0008) [2023-12-26 19:49:54,015][105692] Updated weights for policy 0, policy_version 613161 (0.0005) [2023-12-26 19:49:54,064][105692] Updated weights for policy 0, policy_version 613171 (0.0005) [2023-12-26 19:49:54,110][105692] Updated weights for policy 0, policy_version 613181 (0.0005) [2023-12-26 19:49:54,154][105692] Updated weights for policy 0, policy_version 613191 (0.0005) [2023-12-26 19:49:54,555][105620] Updated weights for policy 1, policy_version 614000 (0.0008) [2023-12-26 19:49:54,608][105620] Updated weights for policy 1, policy_version 614010 (0.0008) [2023-12-26 19:49:54,657][105620] Updated weights for policy 1, policy_version 614020 (0.0007) [2023-12-26 19:49:54,846][105692] Updated weights for policy 0, policy_version 613201 (0.0010) [2023-12-26 19:49:54,897][105692] Updated weights for policy 0, policy_version 613211 (0.0010) [2023-12-26 19:49:54,946][105692] Updated weights for policy 0, policy_version 613221 (0.0010) [2023-12-26 19:49:55,431][105620] Updated weights for policy 1, policy_version 614030 (0.0009) [2023-12-26 19:49:55,486][105620] Updated weights for policy 1, policy_version 614042 (0.0010) [2023-12-26 19:49:55,544][105620] Updated weights for policy 1, policy_version 614054 (0.0010) [2023-12-26 19:49:55,639][105692] Updated weights for policy 0, policy_version 613231 (0.0011) [2023-12-26 19:49:55,695][105692] Updated weights for policy 0, policy_version 613241 (0.0011) [2023-12-26 19:49:55,751][105692] Updated weights for policy 0, policy_version 613251 (0.0011) [2023-12-26 19:49:56,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 314236928. Throughput: 0: 9943.5, 1: 9805.8. Samples: 314245732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:49:56,063][104569] Avg episode reward: [(0, '9174.023'), (1, '9354.398')] [2023-12-26 19:49:56,242][105620] Updated weights for policy 1, policy_version 614064 (0.0008) [2023-12-26 19:49:56,295][105620] Updated weights for policy 1, policy_version 614074 (0.0008) [2023-12-26 19:49:56,354][105620] Updated weights for policy 1, policy_version 614084 (0.0008) [2023-12-26 19:49:56,486][105692] Updated weights for policy 0, policy_version 613261 (0.0011) [2023-12-26 19:49:56,538][105692] Updated weights for policy 0, policy_version 613271 (0.0010) [2023-12-26 19:49:56,589][105692] Updated weights for policy 0, policy_version 613281 (0.0010) [2023-12-26 19:49:57,138][105620] Updated weights for policy 1, policy_version 614094 (0.0008) [2023-12-26 19:49:57,186][105620] Updated weights for policy 1, policy_version 614104 (0.0008) [2023-12-26 19:49:57,232][105692] Updated weights for policy 0, policy_version 613291 (0.0009) [2023-12-26 19:49:57,242][105620] Updated weights for policy 1, policy_version 614114 (0.0007) [2023-12-26 19:49:57,295][105692] Updated weights for policy 0, policy_version 613301 (0.0011) [2023-12-26 19:49:57,343][105692] Updated weights for policy 0, policy_version 613311 (0.0010) [2023-12-26 19:49:57,931][105620] Updated weights for policy 1, policy_version 614124 (0.0007) [2023-12-26 19:49:57,997][105620] Updated weights for policy 1, policy_version 614134 (0.0008) [2023-12-26 19:49:58,055][105620] Updated weights for policy 1, policy_version 614144 (0.0008) [2023-12-26 19:49:58,094][105692] Updated weights for policy 0, policy_version 613321 (0.0010) [2023-12-26 19:49:58,164][105692] Updated weights for policy 0, policy_version 613331 (0.0008) [2023-12-26 19:49:58,228][105692] Updated weights for policy 0, policy_version 613341 (0.0008) [2023-12-26 19:49:58,295][105692] Updated weights for policy 0, policy_version 613351 (0.0010) [2023-12-26 19:49:58,827][105620] Updated weights for policy 1, policy_version 614154 (0.0007) [2023-12-26 19:49:58,894][105620] Updated weights for policy 1, policy_version 614164 (0.0010) [2023-12-26 19:49:58,959][105620] Updated weights for policy 1, policy_version 614174 (0.0011) [2023-12-26 19:49:59,019][105620] Updated weights for policy 1, policy_version 614184 (0.0010) [2023-12-26 19:49:59,127][105692] Updated weights for policy 0, policy_version 613361 (0.0008) [2023-12-26 19:49:59,182][105692] Updated weights for policy 0, policy_version 613371 (0.0008) [2023-12-26 19:49:59,248][105692] Updated weights for policy 0, policy_version 613381 (0.0008) [2023-12-26 19:49:59,791][105620] Updated weights for policy 1, policy_version 614194 (0.0011) [2023-12-26 19:49:59,858][105620] Updated weights for policy 1, policy_version 614204 (0.0011) [2023-12-26 19:49:59,914][105620] Updated weights for policy 1, policy_version 614214 (0.0011) [2023-12-26 19:49:59,964][105692] Updated weights for policy 0, policy_version 613391 (0.0008) [2023-12-26 19:50:00,025][105692] Updated weights for policy 0, policy_version 613401 (0.0007) [2023-12-26 19:50:00,075][105692] Updated weights for policy 0, policy_version 613411 (0.0008) [2023-12-26 19:50:00,540][105620] Updated weights for policy 1, policy_version 614224 (0.0009) [2023-12-26 19:50:00,594][105620] Updated weights for policy 1, policy_version 614234 (0.0009) [2023-12-26 19:50:00,641][105620] Updated weights for policy 1, policy_version 614244 (0.0009) [2023-12-26 19:50:00,845][105692] Updated weights for policy 0, policy_version 613421 (0.0010) [2023-12-26 19:50:00,897][105692] Updated weights for policy 0, policy_version 613431 (0.0009) [2023-12-26 19:50:00,956][105692] Updated weights for policy 0, policy_version 613441 (0.0009) [2023-12-26 19:50:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19494.2). Total num frames: 314335232. Throughput: 0: 9936.8, 1: 9750.8. Samples: 314301944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:50:01,062][104569] Avg episode reward: [(0, '9357.476'), (1, '9353.954')] [2023-12-26 19:50:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000613448_157073408.pth... [2023-12-26 19:50:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000614248_157261824.pth... [2023-12-26 19:50:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000612280_156770304.pth [2023-12-26 19:50:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000613128_156975104.pth [2023-12-26 19:50:01,341][105620] Updated weights for policy 1, policy_version 614254 (0.0008) [2023-12-26 19:50:01,411][105620] Updated weights for policy 1, policy_version 614264 (0.0008) [2023-12-26 19:50:01,473][105620] Updated weights for policy 1, policy_version 614274 (0.0009) [2023-12-26 19:50:01,727][105692] Updated weights for policy 0, policy_version 613451 (0.0008) [2023-12-26 19:50:01,787][105692] Updated weights for policy 0, policy_version 613461 (0.0008) [2023-12-26 19:50:01,834][105692] Updated weights for policy 0, policy_version 613471 (0.0008) [2023-12-26 19:50:02,216][105620] Updated weights for policy 1, policy_version 614284 (0.0008) [2023-12-26 19:50:02,272][105620] Updated weights for policy 1, policy_version 614294 (0.0009) [2023-12-26 19:50:02,330][105620] Updated weights for policy 1, policy_version 614304 (0.0009) [2023-12-26 19:50:02,560][105692] Updated weights for policy 0, policy_version 613481 (0.0009) [2023-12-26 19:50:02,625][105692] Updated weights for policy 0, policy_version 613491 (0.0005) [2023-12-26 19:50:02,696][105692] Updated weights for policy 0, policy_version 613501 (0.0009) [2023-12-26 19:50:02,761][105692] Updated weights for policy 0, policy_version 613511 (0.0009) [2023-12-26 19:50:02,961][105620] Updated weights for policy 1, policy_version 614314 (0.0009) [2023-12-26 19:50:03,024][105620] Updated weights for policy 1, policy_version 614324 (0.0006) [2023-12-26 19:50:03,084][105620] Updated weights for policy 1, policy_version 614334 (0.0005) [2023-12-26 19:50:03,146][105620] Updated weights for policy 1, policy_version 614344 (0.0005) [2023-12-26 19:50:03,432][105692] Updated weights for policy 0, policy_version 613521 (0.0010) [2023-12-26 19:50:03,498][105692] Updated weights for policy 0, policy_version 613531 (0.0009) [2023-12-26 19:50:03,570][105692] Updated weights for policy 0, policy_version 613541 (0.0005) [2023-12-26 19:50:03,639][105620] Updated weights for policy 1, policy_version 614354 (0.0006) [2023-12-26 19:50:03,699][105620] Updated weights for policy 1, policy_version 614364 (0.0006) [2023-12-26 19:50:03,754][105620] Updated weights for policy 1, policy_version 614374 (0.0006) [2023-12-26 19:50:04,199][105692] Updated weights for policy 0, policy_version 613551 (0.0007) [2023-12-26 19:50:04,250][105692] Updated weights for policy 0, policy_version 613561 (0.0008) [2023-12-26 19:50:04,304][105692] Updated weights for policy 0, policy_version 613571 (0.0008) [2023-12-26 19:50:04,346][105620] Updated weights for policy 1, policy_version 614384 (0.0007) [2023-12-26 19:50:04,406][105620] Updated weights for policy 1, policy_version 614394 (0.0011) [2023-12-26 19:50:04,469][105620] Updated weights for policy 1, policy_version 614404 (0.0010) [2023-12-26 19:50:04,952][105692] Updated weights for policy 0, policy_version 613581 (0.0007) [2023-12-26 19:50:05,009][105692] Updated weights for policy 0, policy_version 613591 (0.0006) [2023-12-26 19:50:05,066][105692] Updated weights for policy 0, policy_version 613601 (0.0006) [2023-12-26 19:50:05,182][105620] Updated weights for policy 1, policy_version 614414 (0.0011) [2023-12-26 19:50:05,234][105620] Updated weights for policy 1, policy_version 614424 (0.0011) [2023-12-26 19:50:05,296][105620] Updated weights for policy 1, policy_version 614434 (0.0010) [2023-12-26 19:50:05,678][105692] Updated weights for policy 0, policy_version 613611 (0.0007) [2023-12-26 19:50:05,729][105692] Updated weights for policy 0, policy_version 613621 (0.0010) [2023-12-26 19:50:05,772][105692] Updated weights for policy 0, policy_version 613631 (0.0007) [2023-12-26 19:50:06,026][105620] Updated weights for policy 1, policy_version 614444 (0.0010) [2023-12-26 19:50:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 314433536. Throughput: 0: 9851.9, 1: 9812.6. Samples: 314421600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:50:06,063][104569] Avg episode reward: [(0, '9357.012'), (1, '8837.244')] [2023-12-26 19:50:06,090][105620] Updated weights for policy 1, policy_version 614454 (0.0009) [2023-12-26 19:50:06,159][105620] Updated weights for policy 1, policy_version 614464 (0.0009) [2023-12-26 19:50:06,483][105692] Updated weights for policy 0, policy_version 613641 (0.0006) [2023-12-26 19:50:06,549][105692] Updated weights for policy 0, policy_version 613651 (0.0008) [2023-12-26 19:50:06,605][105692] Updated weights for policy 0, policy_version 613661 (0.0008) [2023-12-26 19:50:06,659][105692] Updated weights for policy 0, policy_version 613671 (0.0007) [2023-12-26 19:50:06,923][105620] Updated weights for policy 1, policy_version 614474 (0.0008) [2023-12-26 19:50:06,991][105620] Updated weights for policy 1, policy_version 614484 (0.0008) [2023-12-26 19:50:07,046][105620] Updated weights for policy 1, policy_version 614494 (0.0008) [2023-12-26 19:50:07,095][105620] Updated weights for policy 1, policy_version 614504 (0.0008) [2023-12-26 19:50:07,416][105692] Updated weights for policy 0, policy_version 613681 (0.0010) [2023-12-26 19:50:07,479][105692] Updated weights for policy 0, policy_version 613691 (0.0011) [2023-12-26 19:50:07,537][105692] Updated weights for policy 0, policy_version 613701 (0.0010) [2023-12-26 19:50:07,867][105620] Updated weights for policy 1, policy_version 614514 (0.0008) [2023-12-26 19:50:07,921][105620] Updated weights for policy 1, policy_version 614524 (0.0008) [2023-12-26 19:50:07,974][105620] Updated weights for policy 1, policy_version 614534 (0.0008) [2023-12-26 19:50:08,225][105692] Updated weights for policy 0, policy_version 613711 (0.0010) [2023-12-26 19:50:08,269][105692] Updated weights for policy 0, policy_version 613721 (0.0010) [2023-12-26 19:50:08,321][105692] Updated weights for policy 0, policy_version 613731 (0.0010) [2023-12-26 19:50:08,702][105620] Updated weights for policy 1, policy_version 614544 (0.0006) [2023-12-26 19:50:08,763][105620] Updated weights for policy 1, policy_version 614554 (0.0007) [2023-12-26 19:50:08,828][105620] Updated weights for policy 1, policy_version 614564 (0.0007) [2023-12-26 19:50:09,037][105692] Updated weights for policy 0, policy_version 613741 (0.0010) [2023-12-26 19:50:09,093][105692] Updated weights for policy 0, policy_version 613751 (0.0009) [2023-12-26 19:50:09,160][105692] Updated weights for policy 0, policy_version 613761 (0.0005) [2023-12-26 19:50:09,547][105620] Updated weights for policy 1, policy_version 614574 (0.0009) [2023-12-26 19:50:09,599][105620] Updated weights for policy 1, policy_version 614584 (0.0008) [2023-12-26 19:50:09,660][105620] Updated weights for policy 1, policy_version 614594 (0.0006) [2023-12-26 19:50:09,829][105692] Updated weights for policy 0, policy_version 613771 (0.0008) [2023-12-26 19:50:09,892][105692] Updated weights for policy 0, policy_version 613781 (0.0007) [2023-12-26 19:50:09,955][105692] Updated weights for policy 0, policy_version 613791 (0.0009) [2023-12-26 19:50:10,329][105620] Updated weights for policy 1, policy_version 614604 (0.0009) [2023-12-26 19:50:10,386][105620] Updated weights for policy 1, policy_version 614614 (0.0007) [2023-12-26 19:50:10,442][105620] Updated weights for policy 1, policy_version 614624 (0.0006) [2023-12-26 19:50:10,636][105692] Updated weights for policy 0, policy_version 613801 (0.0009) [2023-12-26 19:50:10,697][105692] Updated weights for policy 0, policy_version 613811 (0.0005) [2023-12-26 19:50:10,763][105692] Updated weights for policy 0, policy_version 613821 (0.0005) [2023-12-26 19:50:10,834][105692] Updated weights for policy 0, policy_version 613831 (0.0006) [2023-12-26 19:50:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 314531840. Throughput: 0: 10024.2, 1: 9721.8. Samples: 314539684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:50:11,063][104569] Avg episode reward: [(0, '9356.272'), (1, '9093.417')] [2023-12-26 19:50:11,226][105620] Updated weights for policy 1, policy_version 614634 (0.0009) [2023-12-26 19:50:11,288][105620] Updated weights for policy 1, policy_version 614644 (0.0009) [2023-12-26 19:50:11,357][105620] Updated weights for policy 1, policy_version 614654 (0.0009) [2023-12-26 19:50:11,424][105620] Updated weights for policy 1, policy_version 614664 (0.0007) [2023-12-26 19:50:11,456][105692] Updated weights for policy 0, policy_version 613841 (0.0009) [2023-12-26 19:50:11,523][105692] Updated weights for policy 0, policy_version 613851 (0.0009) [2023-12-26 19:50:11,585][105692] Updated weights for policy 0, policy_version 613861 (0.0009) [2023-12-26 19:50:12,159][105620] Updated weights for policy 1, policy_version 614674 (0.0011) [2023-12-26 19:50:12,219][105620] Updated weights for policy 1, policy_version 614684 (0.0011) [2023-12-26 19:50:12,279][105620] Updated weights for policy 1, policy_version 614694 (0.0011) [2023-12-26 19:50:12,398][105692] Updated weights for policy 0, policy_version 613871 (0.0011) [2023-12-26 19:50:12,447][105692] Updated weights for policy 0, policy_version 613881 (0.0010) [2023-12-26 19:50:12,499][105692] Updated weights for policy 0, policy_version 613891 (0.0010) [2023-12-26 19:50:12,925][105620] Updated weights for policy 1, policy_version 614704 (0.0007) [2023-12-26 19:50:12,986][105620] Updated weights for policy 1, policy_version 614714 (0.0007) [2023-12-26 19:50:13,045][105620] Updated weights for policy 1, policy_version 614724 (0.0008) [2023-12-26 19:50:13,253][105692] Updated weights for policy 0, policy_version 613901 (0.0009) [2023-12-26 19:50:13,322][105692] Updated weights for policy 0, policy_version 613911 (0.0007) [2023-12-26 19:50:13,390][105692] Updated weights for policy 0, policy_version 613921 (0.0007) [2023-12-26 19:50:13,684][105620] Updated weights for policy 1, policy_version 614734 (0.0008) [2023-12-26 19:50:13,748][105620] Updated weights for policy 1, policy_version 614744 (0.0009) [2023-12-26 19:50:13,817][105620] Updated weights for policy 1, policy_version 614754 (0.0009) [2023-12-26 19:50:14,022][105692] Updated weights for policy 0, policy_version 613931 (0.0005) [2023-12-26 19:50:14,079][105692] Updated weights for policy 0, policy_version 613941 (0.0005) [2023-12-26 19:50:14,142][105692] Updated weights for policy 0, policy_version 613951 (0.0007) [2023-12-26 19:50:14,475][105620] Updated weights for policy 1, policy_version 614764 (0.0007) [2023-12-26 19:50:14,543][105620] Updated weights for policy 1, policy_version 614774 (0.0005) [2023-12-26 19:50:14,605][105620] Updated weights for policy 1, policy_version 614784 (0.0006) [2023-12-26 19:50:14,767][105692] Updated weights for policy 0, policy_version 613961 (0.0009) [2023-12-26 19:50:14,826][105692] Updated weights for policy 0, policy_version 613971 (0.0007) [2023-12-26 19:50:14,889][105692] Updated weights for policy 0, policy_version 613981 (0.0008) [2023-12-26 19:50:14,954][105692] Updated weights for policy 0, policy_version 613991 (0.0009) [2023-12-26 19:50:15,211][105620] Updated weights for policy 1, policy_version 614794 (0.0007) [2023-12-26 19:50:15,277][105620] Updated weights for policy 1, policy_version 614804 (0.0008) [2023-12-26 19:50:15,352][105620] Updated weights for policy 1, policy_version 614814 (0.0007) [2023-12-26 19:50:15,410][105620] Updated weights for policy 1, policy_version 614824 (0.0009) [2023-12-26 19:50:15,797][105692] Updated weights for policy 0, policy_version 614001 (0.0008) [2023-12-26 19:50:15,846][105692] Updated weights for policy 0, policy_version 614011 (0.0008) [2023-12-26 19:50:15,908][105692] Updated weights for policy 0, policy_version 614021 (0.0006) [2023-12-26 19:50:16,054][105620] Updated weights for policy 1, policy_version 614834 (0.0010) [2023-12-26 19:50:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 314630144. Throughput: 0: 10074.8, 1: 9700.4. Samples: 314598520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:50:16,063][104569] Avg episode reward: [(0, '9356.486'), (1, '8989.361')] [2023-12-26 19:50:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000614024_157220864.pth... [2023-12-26 19:50:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000612856_156917760.pth [2023-12-26 19:50:16,115][105620] Updated weights for policy 1, policy_version 614844 (0.0010) [2023-12-26 19:50:16,169][105620] Updated weights for policy 1, policy_version 614854 (0.0006) [2023-12-26 19:50:16,179][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000614856_157417472.pth... [2023-12-26 19:50:16,183][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000613704_157122560.pth [2023-12-26 19:50:16,734][105620] Updated weights for policy 1, policy_version 614864 (0.0006) [2023-12-26 19:50:16,743][105692] Updated weights for policy 0, policy_version 614031 (0.0009) [2023-12-26 19:50:16,787][105620] Updated weights for policy 1, policy_version 614874 (0.0005) [2023-12-26 19:50:16,792][105692] Updated weights for policy 0, policy_version 614041 (0.0009) [2023-12-26 19:50:16,843][105620] Updated weights for policy 1, policy_version 614884 (0.0005) [2023-12-26 19:50:16,845][105692] Updated weights for policy 0, policy_version 614051 (0.0009) [2023-12-26 19:50:17,402][105620] Updated weights for policy 1, policy_version 614894 (0.0005) [2023-12-26 19:50:17,454][105620] Updated weights for policy 1, policy_version 614904 (0.0005) [2023-12-26 19:50:17,507][105620] Updated weights for policy 1, policy_version 614914 (0.0005) [2023-12-26 19:50:17,718][105692] Updated weights for policy 0, policy_version 614061 (0.0008) [2023-12-26 19:50:17,768][105692] Updated weights for policy 0, policy_version 614071 (0.0008) [2023-12-26 19:50:17,820][105692] Updated weights for policy 0, policy_version 614082 (0.0009) [2023-12-26 19:50:18,131][105620] Updated weights for policy 1, policy_version 614924 (0.0007) [2023-12-26 19:50:18,196][105620] Updated weights for policy 1, policy_version 614934 (0.0009) [2023-12-26 19:50:18,252][105620] Updated weights for policy 1, policy_version 614944 (0.0008) [2023-12-26 19:50:18,541][105692] Updated weights for policy 0, policy_version 614092 (0.0009) [2023-12-26 19:50:18,607][105692] Updated weights for policy 0, policy_version 614102 (0.0007) [2023-12-26 19:50:18,663][105692] Updated weights for policy 0, policy_version 614112 (0.0008) [2023-12-26 19:50:18,972][105620] Updated weights for policy 1, policy_version 614954 (0.0008) [2023-12-26 19:50:19,029][105620] Updated weights for policy 1, policy_version 614964 (0.0006) [2023-12-26 19:50:19,094][105620] Updated weights for policy 1, policy_version 614974 (0.0006) [2023-12-26 19:50:19,160][105620] Updated weights for policy 1, policy_version 614984 (0.0005) [2023-12-26 19:50:19,477][105692] Updated weights for policy 0, policy_version 614122 (0.0007) [2023-12-26 19:50:19,545][105692] Updated weights for policy 0, policy_version 614132 (0.0009) [2023-12-26 19:50:19,609][105692] Updated weights for policy 0, policy_version 614142 (0.0007) [2023-12-26 19:50:19,662][105692] Updated weights for policy 0, policy_version 614152 (0.0009) [2023-12-26 19:50:19,836][105620] Updated weights for policy 1, policy_version 614994 (0.0010) [2023-12-26 19:50:19,901][105620] Updated weights for policy 1, policy_version 615004 (0.0009) [2023-12-26 19:50:19,966][105620] Updated weights for policy 1, policy_version 615014 (0.0008) [2023-12-26 19:50:20,349][105692] Updated weights for policy 0, policy_version 614162 (0.0009) [2023-12-26 19:50:20,410][105692] Updated weights for policy 0, policy_version 614172 (0.0009) [2023-12-26 19:50:20,462][105692] Updated weights for policy 0, policy_version 614182 (0.0009) [2023-12-26 19:50:20,688][105620] Updated weights for policy 1, policy_version 615024 (0.0009) [2023-12-26 19:50:20,750][105620] Updated weights for policy 1, policy_version 615034 (0.0010) [2023-12-26 19:50:20,814][105620] Updated weights for policy 1, policy_version 615044 (0.0006) [2023-12-26 19:50:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 314728448. Throughput: 0: 9958.1, 1: 9810.5. Samples: 314717120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:50:21,063][104569] Avg episode reward: [(0, '9356.858'), (1, '8759.930')] [2023-12-26 19:50:21,276][105692] Updated weights for policy 0, policy_version 614192 (0.0008) [2023-12-26 19:50:21,327][105692] Updated weights for policy 0, policy_version 614202 (0.0006) [2023-12-26 19:50:21,403][105692] Updated weights for policy 0, policy_version 614212 (0.0008) [2023-12-26 19:50:21,489][105620] Updated weights for policy 1, policy_version 615054 (0.0009) [2023-12-26 19:50:21,548][105620] Updated weights for policy 1, policy_version 615064 (0.0010) [2023-12-26 19:50:21,616][105620] Updated weights for policy 1, policy_version 615074 (0.0009) [2023-12-26 19:50:22,098][105692] Updated weights for policy 0, policy_version 614222 (0.0006) [2023-12-26 19:50:22,160][105692] Updated weights for policy 0, policy_version 614232 (0.0006) [2023-12-26 19:50:22,224][105692] Updated weights for policy 0, policy_version 614242 (0.0008) [2023-12-26 19:50:22,344][105620] Updated weights for policy 1, policy_version 615084 (0.0010) [2023-12-26 19:50:22,406][105620] Updated weights for policy 1, policy_version 615094 (0.0010) [2023-12-26 19:50:22,469][105620] Updated weights for policy 1, policy_version 615104 (0.0011) [2023-12-26 19:50:22,957][105692] Updated weights for policy 0, policy_version 614252 (0.0008) [2023-12-26 19:50:23,009][105692] Updated weights for policy 0, policy_version 614262 (0.0008) [2023-12-26 19:50:23,058][105692] Updated weights for policy 0, policy_version 614272 (0.0008) [2023-12-26 19:50:23,227][105620] Updated weights for policy 1, policy_version 615114 (0.0010) [2023-12-26 19:50:23,275][105620] Updated weights for policy 1, policy_version 615124 (0.0010) [2023-12-26 19:50:23,335][105620] Updated weights for policy 1, policy_version 615134 (0.0010) [2023-12-26 19:50:23,389][105620] Updated weights for policy 1, policy_version 615144 (0.0010) [2023-12-26 19:50:23,841][105692] Updated weights for policy 0, policy_version 614282 (0.0008) [2023-12-26 19:50:23,898][105692] Updated weights for policy 0, policy_version 614292 (0.0010) [2023-12-26 19:50:23,955][105692] Updated weights for policy 0, policy_version 614302 (0.0010) [2023-12-26 19:50:24,013][105692] Updated weights for policy 0, policy_version 614312 (0.0011) [2023-12-26 19:50:24,079][105620] Updated weights for policy 1, policy_version 615154 (0.0010) [2023-12-26 19:50:24,127][105620] Updated weights for policy 1, policy_version 615164 (0.0010) [2023-12-26 19:50:24,178][105620] Updated weights for policy 1, policy_version 615174 (0.0010) [2023-12-26 19:50:24,636][105692] Updated weights for policy 0, policy_version 614322 (0.0010) [2023-12-26 19:50:24,687][105692] Updated weights for policy 0, policy_version 614332 (0.0010) [2023-12-26 19:50:24,743][105692] Updated weights for policy 0, policy_version 614342 (0.0010) [2023-12-26 19:50:24,953][105620] Updated weights for policy 1, policy_version 615184 (0.0010) [2023-12-26 19:50:24,998][105620] Updated weights for policy 1, policy_version 615194 (0.0010) [2023-12-26 19:50:25,050][105620] Updated weights for policy 1, policy_version 615204 (0.0010) [2023-12-26 19:50:25,361][105692] Updated weights for policy 0, policy_version 614352 (0.0005) [2023-12-26 19:50:25,414][105692] Updated weights for policy 0, policy_version 614362 (0.0006) [2023-12-26 19:50:25,475][105692] Updated weights for policy 0, policy_version 614372 (0.0009) [2023-12-26 19:50:25,774][105620] Updated weights for policy 1, policy_version 615214 (0.0007) [2023-12-26 19:50:25,821][105620] Updated weights for policy 1, policy_version 615224 (0.0005) [2023-12-26 19:50:25,871][105620] Updated weights for policy 1, policy_version 615234 (0.0005) [2023-12-26 19:50:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 314826752. Throughput: 0: 9881.0, 1: 9712.3. Samples: 314834324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:50:26,063][104569] Avg episode reward: [(0, '9265.860'), (1, '8851.805')] [2023-12-26 19:50:26,191][105692] Updated weights for policy 0, policy_version 614382 (0.0007) [2023-12-26 19:50:26,253][105692] Updated weights for policy 0, policy_version 614392 (0.0010) [2023-12-26 19:50:26,311][105692] Updated weights for policy 0, policy_version 614402 (0.0008) [2023-12-26 19:50:26,505][105620] Updated weights for policy 1, policy_version 615244 (0.0006) [2023-12-26 19:50:26,557][105620] Updated weights for policy 1, policy_version 615254 (0.0008) [2023-12-26 19:50:26,608][105620] Updated weights for policy 1, policy_version 615264 (0.0008) [2023-12-26 19:50:26,998][105692] Updated weights for policy 0, policy_version 614412 (0.0007) [2023-12-26 19:50:27,049][105692] Updated weights for policy 0, policy_version 614422 (0.0010) [2023-12-26 19:50:27,100][105692] Updated weights for policy 0, policy_version 614432 (0.0010) [2023-12-26 19:50:27,255][105620] Updated weights for policy 1, policy_version 615274 (0.0007) [2023-12-26 19:50:27,309][105620] Updated weights for policy 1, policy_version 615284 (0.0005) [2023-12-26 19:50:27,366][105620] Updated weights for policy 1, policy_version 615294 (0.0005) [2023-12-26 19:50:27,422][105620] Updated weights for policy 1, policy_version 615304 (0.0005) [2023-12-26 19:50:27,807][105692] Updated weights for policy 0, policy_version 614442 (0.0010) [2023-12-26 19:50:27,851][105692] Updated weights for policy 0, policy_version 614452 (0.0010) [2023-12-26 19:50:27,895][105692] Updated weights for policy 0, policy_version 614462 (0.0010) [2023-12-26 19:50:27,945][105692] Updated weights for policy 0, policy_version 614472 (0.0010) [2023-12-26 19:50:28,020][105620] Updated weights for policy 1, policy_version 615314 (0.0007) [2023-12-26 19:50:28,085][105620] Updated weights for policy 1, policy_version 615324 (0.0007) [2023-12-26 19:50:28,144][105620] Updated weights for policy 1, policy_version 615334 (0.0008) [2023-12-26 19:50:28,671][105692] Updated weights for policy 0, policy_version 614482 (0.0005) [2023-12-26 19:50:28,724][105692] Updated weights for policy 0, policy_version 614492 (0.0005) [2023-12-26 19:50:28,785][105692] Updated weights for policy 0, policy_version 614502 (0.0005) [2023-12-26 19:50:28,849][105620] Updated weights for policy 1, policy_version 615344 (0.0006) [2023-12-26 19:50:28,916][105620] Updated weights for policy 1, policy_version 615354 (0.0006) [2023-12-26 19:50:28,984][105620] Updated weights for policy 1, policy_version 615364 (0.0006) [2023-12-26 19:50:29,367][105692] Updated weights for policy 0, policy_version 614512 (0.0010) [2023-12-26 19:50:29,433][105692] Updated weights for policy 0, policy_version 614522 (0.0011) [2023-12-26 19:50:29,492][105692] Updated weights for policy 0, policy_version 614532 (0.0011) [2023-12-26 19:50:29,534][105620] Updated weights for policy 1, policy_version 615374 (0.0007) [2023-12-26 19:50:29,586][105620] Updated weights for policy 1, policy_version 615384 (0.0008) [2023-12-26 19:50:29,635][105620] Updated weights for policy 1, policy_version 615394 (0.0008) [2023-12-26 19:50:30,210][105692] Updated weights for policy 0, policy_version 614542 (0.0008) [2023-12-26 19:50:30,271][105692] Updated weights for policy 0, policy_version 614552 (0.0008) [2023-12-26 19:50:30,328][105692] Updated weights for policy 0, policy_version 614562 (0.0008) [2023-12-26 19:50:30,372][105620] Updated weights for policy 1, policy_version 615404 (0.0008) [2023-12-26 19:50:30,429][105620] Updated weights for policy 1, policy_version 615414 (0.0008) [2023-12-26 19:50:30,477][105620] Updated weights for policy 1, policy_version 615424 (0.0008) [2023-12-26 19:50:31,050][105692] Updated weights for policy 0, policy_version 614572 (0.0008) [2023-12-26 19:50:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 314925056. Throughput: 0: 9948.3, 1: 9775.1. Samples: 314896036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:50:31,062][104569] Avg episode reward: [(0, '9268.758'), (1, '8705.023')] [2023-12-26 19:50:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000615432_157564928.pth... [2023-12-26 19:50:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000614248_157261824.pth [2023-12-26 19:50:31,109][105692] Updated weights for policy 0, policy_version 614582 (0.0008) [2023-12-26 19:50:31,171][105692] Updated weights for policy 0, policy_version 614592 (0.0006) [2023-12-26 19:50:31,216][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000614600_157368320.pth... [2023-12-26 19:50:31,220][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000613448_157073408.pth [2023-12-26 19:50:31,234][105620] Updated weights for policy 1, policy_version 615434 (0.0008) [2023-12-26 19:50:31,291][105620] Updated weights for policy 1, policy_version 615444 (0.0009) [2023-12-26 19:50:31,350][105620] Updated weights for policy 1, policy_version 615454 (0.0009) [2023-12-26 19:50:31,418][105620] Updated weights for policy 1, policy_version 615464 (0.0009) [2023-12-26 19:50:31,861][105692] Updated weights for policy 0, policy_version 614602 (0.0006) [2023-12-26 19:50:31,911][105692] Updated weights for policy 0, policy_version 614612 (0.0009) [2023-12-26 19:50:31,960][105692] Updated weights for policy 0, policy_version 614622 (0.0008) [2023-12-26 19:50:32,023][105692] Updated weights for policy 0, policy_version 614632 (0.0005) [2023-12-26 19:50:32,093][105620] Updated weights for policy 1, policy_version 615474 (0.0010) [2023-12-26 19:50:32,146][105620] Updated weights for policy 1, policy_version 615484 (0.0010) [2023-12-26 19:50:32,205][105620] Updated weights for policy 1, policy_version 615494 (0.0008) [2023-12-26 19:50:32,704][105692] Updated weights for policy 0, policy_version 614642 (0.0009) [2023-12-26 19:50:32,756][105692] Updated weights for policy 0, policy_version 614652 (0.0009) [2023-12-26 19:50:32,807][105692] Updated weights for policy 0, policy_version 614662 (0.0009) [2023-12-26 19:50:33,010][105620] Updated weights for policy 1, policy_version 615504 (0.0008) [2023-12-26 19:50:33,068][105620] Updated weights for policy 1, policy_version 615514 (0.0009) [2023-12-26 19:50:33,128][105620] Updated weights for policy 1, policy_version 615524 (0.0009) [2023-12-26 19:50:33,529][105692] Updated weights for policy 0, policy_version 614672 (0.0009) [2023-12-26 19:50:33,582][105692] Updated weights for policy 0, policy_version 614682 (0.0010) [2023-12-26 19:50:33,635][105692] Updated weights for policy 0, policy_version 614693 (0.0010) [2023-12-26 19:50:33,800][105620] Updated weights for policy 1, policy_version 615534 (0.0009) [2023-12-26 19:50:33,845][105620] Updated weights for policy 1, policy_version 615544 (0.0008) [2023-12-26 19:50:33,895][105620] Updated weights for policy 1, policy_version 615554 (0.0009) [2023-12-26 19:50:34,424][105692] Updated weights for policy 0, policy_version 614704 (0.0010) [2023-12-26 19:50:34,486][105692] Updated weights for policy 0, policy_version 614714 (0.0010) [2023-12-26 19:50:34,552][105692] Updated weights for policy 0, policy_version 614724 (0.0009) [2023-12-26 19:50:34,563][105620] Updated weights for policy 1, policy_version 615564 (0.0007) [2023-12-26 19:50:34,627][105620] Updated weights for policy 1, policy_version 615574 (0.0006) [2023-12-26 19:50:34,693][105620] Updated weights for policy 1, policy_version 615584 (0.0007) [2023-12-26 19:50:35,271][105692] Updated weights for policy 0, policy_version 614734 (0.0007) [2023-12-26 19:50:35,326][105692] Updated weights for policy 0, policy_version 614744 (0.0008) [2023-12-26 19:50:35,341][105620] Updated weights for policy 1, policy_version 615594 (0.0008) [2023-12-26 19:50:35,384][105692] Updated weights for policy 0, policy_version 614754 (0.0008) [2023-12-26 19:50:35,399][105620] Updated weights for policy 1, policy_version 615604 (0.0006) [2023-12-26 19:50:35,452][105620] Updated weights for policy 1, policy_version 615614 (0.0005) [2023-12-26 19:50:35,501][105620] Updated weights for policy 1, policy_version 615624 (0.0009) [2023-12-26 19:50:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.9, 300 sec: 19522.0). Total num frames: 315023360. Throughput: 0: 9831.4, 1: 9899.9. Samples: 315015156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:50:36,062][104569] Avg episode reward: [(0, '9268.779'), (1, '8805.287')] [2023-12-26 19:50:36,115][105620] Updated weights for policy 1, policy_version 615634 (0.0006) [2023-12-26 19:50:36,174][105692] Updated weights for policy 0, policy_version 614764 (0.0009) [2023-12-26 19:50:36,180][105620] Updated weights for policy 1, policy_version 615644 (0.0010) [2023-12-26 19:50:36,238][105692] Updated weights for policy 0, policy_version 614774 (0.0006) [2023-12-26 19:50:36,244][105620] Updated weights for policy 1, policy_version 615654 (0.0010) [2023-12-26 19:50:36,300][105692] Updated weights for policy 0, policy_version 614784 (0.0007) [2023-12-26 19:50:36,970][105620] Updated weights for policy 1, policy_version 615664 (0.0007) [2023-12-26 19:50:37,039][105620] Updated weights for policy 1, policy_version 615674 (0.0006) [2023-12-26 19:50:37,103][105692] Updated weights for policy 0, policy_version 614794 (0.0008) [2023-12-26 19:50:37,108][105620] Updated weights for policy 1, policy_version 615684 (0.0005) [2023-12-26 19:50:37,158][105692] Updated weights for policy 0, policy_version 614804 (0.0009) [2023-12-26 19:50:37,211][105692] Updated weights for policy 0, policy_version 614814 (0.0010) [2023-12-26 19:50:37,264][105692] Updated weights for policy 0, policy_version 614824 (0.0010) [2023-12-26 19:50:37,686][105620] Updated weights for policy 1, policy_version 615694 (0.0008) [2023-12-26 19:50:37,752][105620] Updated weights for policy 1, policy_version 615704 (0.0010) [2023-12-26 19:50:37,807][105620] Updated weights for policy 1, policy_version 615714 (0.0010) [2023-12-26 19:50:38,089][105692] Updated weights for policy 0, policy_version 614834 (0.0008) [2023-12-26 19:50:38,157][105692] Updated weights for policy 0, policy_version 614844 (0.0008) [2023-12-26 19:50:38,218][105692] Updated weights for policy 0, policy_version 614854 (0.0009) [2023-12-26 19:50:38,576][105620] Updated weights for policy 1, policy_version 615724 (0.0011) [2023-12-26 19:50:38,629][105620] Updated weights for policy 1, policy_version 615734 (0.0010) [2023-12-26 19:50:38,682][105620] Updated weights for policy 1, policy_version 615744 (0.0010) [2023-12-26 19:50:38,995][105692] Updated weights for policy 0, policy_version 614864 (0.0008) [2023-12-26 19:50:39,057][105692] Updated weights for policy 0, policy_version 614874 (0.0006) [2023-12-26 19:50:39,116][105692] Updated weights for policy 0, policy_version 614884 (0.0006) [2023-12-26 19:50:39,407][105620] Updated weights for policy 1, policy_version 615754 (0.0010) [2023-12-26 19:50:39,468][105620] Updated weights for policy 1, policy_version 615764 (0.0011) [2023-12-26 19:50:39,524][105620] Updated weights for policy 1, policy_version 615774 (0.0011) [2023-12-26 19:50:39,574][105620] Updated weights for policy 1, policy_version 615784 (0.0010) [2023-12-26 19:50:39,900][105692] Updated weights for policy 0, policy_version 614894 (0.0009) [2023-12-26 19:50:39,969][105692] Updated weights for policy 0, policy_version 614904 (0.0008) [2023-12-26 19:50:40,034][105692] Updated weights for policy 0, policy_version 614914 (0.0008) [2023-12-26 19:50:40,357][105620] Updated weights for policy 1, policy_version 615794 (0.0010) [2023-12-26 19:50:40,427][105620] Updated weights for policy 1, policy_version 615804 (0.0007) [2023-12-26 19:50:40,499][105620] Updated weights for policy 1, policy_version 615814 (0.0006) [2023-12-26 19:50:40,700][105692] Updated weights for policy 0, policy_version 614924 (0.0009) [2023-12-26 19:50:40,767][105692] Updated weights for policy 0, policy_version 614934 (0.0010) [2023-12-26 19:50:40,830][105692] Updated weights for policy 0, policy_version 614944 (0.0006) [2023-12-26 19:50:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 315121664. Throughput: 0: 9626.8, 1: 10034.9. Samples: 315130504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:50:41,062][104569] Avg episode reward: [(0, '9268.563'), (1, '9095.975')] [2023-12-26 19:50:41,091][105620] Updated weights for policy 1, policy_version 615824 (0.0008) [2023-12-26 19:50:41,153][105620] Updated weights for policy 1, policy_version 615834 (0.0008) [2023-12-26 19:50:41,211][105620] Updated weights for policy 1, policy_version 615844 (0.0008) [2023-12-26 19:50:41,545][105692] Updated weights for policy 0, policy_version 614954 (0.0005) [2023-12-26 19:50:41,610][105692] Updated weights for policy 0, policy_version 614964 (0.0006) [2023-12-26 19:50:41,676][105692] Updated weights for policy 0, policy_version 614974 (0.0009) [2023-12-26 19:50:41,742][105692] Updated weights for policy 0, policy_version 614984 (0.0011) [2023-12-26 19:50:42,022][105620] Updated weights for policy 1, policy_version 615854 (0.0008) [2023-12-26 19:50:42,080][105620] Updated weights for policy 1, policy_version 615864 (0.0008) [2023-12-26 19:50:42,133][105620] Updated weights for policy 1, policy_version 615874 (0.0008) [2023-12-26 19:50:42,493][105692] Updated weights for policy 0, policy_version 614994 (0.0011) [2023-12-26 19:50:42,545][105692] Updated weights for policy 0, policy_version 615004 (0.0011) [2023-12-26 19:50:42,605][105692] Updated weights for policy 0, policy_version 615014 (0.0011) [2023-12-26 19:50:42,920][105620] Updated weights for policy 1, policy_version 615884 (0.0008) [2023-12-26 19:50:42,974][105620] Updated weights for policy 1, policy_version 615894 (0.0009) [2023-12-26 19:50:43,024][105620] Updated weights for policy 1, policy_version 615904 (0.0008) [2023-12-26 19:50:43,314][105692] Updated weights for policy 0, policy_version 615024 (0.0007) [2023-12-26 19:50:43,373][105692] Updated weights for policy 0, policy_version 615034 (0.0006) [2023-12-26 19:50:43,435][105692] Updated weights for policy 0, policy_version 615044 (0.0008) [2023-12-26 19:50:43,862][105620] Updated weights for policy 1, policy_version 615914 (0.0009) [2023-12-26 19:50:43,914][105620] Updated weights for policy 1, policy_version 615924 (0.0009) [2023-12-26 19:50:43,969][105620] Updated weights for policy 1, policy_version 615935 (0.0009) [2023-12-26 19:50:44,099][105692] Updated weights for policy 0, policy_version 615054 (0.0009) [2023-12-26 19:50:44,160][105692] Updated weights for policy 0, policy_version 615064 (0.0009) [2023-12-26 19:50:44,162][105585] KL-divergence is very high: 334.1525 [2023-12-26 19:50:44,210][105585] KL-divergence is very high: 356.5348 [2023-12-26 19:50:44,224][105692] Updated weights for policy 0, policy_version 615074 (0.0009) [2023-12-26 19:50:44,724][105620] Updated weights for policy 1, policy_version 615945 (0.0009) [2023-12-26 19:50:44,795][105620] Updated weights for policy 1, policy_version 615955 (0.0006) [2023-12-26 19:50:44,863][105620] Updated weights for policy 1, policy_version 615965 (0.0006) [2023-12-26 19:50:44,929][105620] Updated weights for policy 1, policy_version 615975 (0.0005) [2023-12-26 19:50:44,958][105692] Updated weights for policy 0, policy_version 615084 (0.0009) [2023-12-26 19:50:45,025][105692] Updated weights for policy 0, policy_version 615094 (0.0008) [2023-12-26 19:50:45,080][105692] Updated weights for policy 0, policy_version 615104 (0.0008) [2023-12-26 19:50:45,575][105620] Updated weights for policy 1, policy_version 615985 (0.0010) [2023-12-26 19:50:45,628][105620] Updated weights for policy 1, policy_version 615996 (0.0010) [2023-12-26 19:50:45,676][105620] Updated weights for policy 1, policy_version 616006 (0.0009) [2023-12-26 19:50:45,741][105692] Updated weights for policy 0, policy_version 615114 (0.0010) [2023-12-26 19:50:45,794][105692] Updated weights for policy 0, policy_version 615124 (0.0010) [2023-12-26 19:50:45,848][105692] Updated weights for policy 0, policy_version 615134 (0.0009) [2023-12-26 19:50:45,906][105692] Updated weights for policy 0, policy_version 615144 (0.0009) [2023-12-26 19:50:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19661.0, 300 sec: 19577.5). Total num frames: 315219968. Throughput: 0: 9627.6, 1: 10011.5. Samples: 315185708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:50:46,062][104569] Avg episode reward: [(0, '8991.688'), (1, '9078.847')] [2023-12-26 19:50:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000615144_157507584.pth... [2023-12-26 19:50:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000616008_157712384.pth... [2023-12-26 19:50:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000614024_157220864.pth [2023-12-26 19:50:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000614856_157417472.pth [2023-12-26 19:50:46,389][105620] Updated weights for policy 1, policy_version 616016 (0.0009) [2023-12-26 19:50:46,450][105620] Updated weights for policy 1, policy_version 616026 (0.0009) [2023-12-26 19:50:46,507][105620] Updated weights for policy 1, policy_version 616036 (0.0009) [2023-12-26 19:50:46,700][105692] Updated weights for policy 0, policy_version 615154 (0.0009) [2023-12-26 19:50:46,759][105692] Updated weights for policy 0, policy_version 615164 (0.0009) [2023-12-26 19:50:46,816][105692] Updated weights for policy 0, policy_version 615174 (0.0009) [2023-12-26 19:50:47,251][105620] Updated weights for policy 1, policy_version 616046 (0.0009) [2023-12-26 19:50:47,303][105620] Updated weights for policy 1, policy_version 616056 (0.0009) [2023-12-26 19:50:47,360][105620] Updated weights for policy 1, policy_version 616066 (0.0008) [2023-12-26 19:50:47,560][105692] Updated weights for policy 0, policy_version 615184 (0.0008) [2023-12-26 19:50:47,623][105692] Updated weights for policy 0, policy_version 615194 (0.0008) [2023-12-26 19:50:47,691][105692] Updated weights for policy 0, policy_version 615204 (0.0009) [2023-12-26 19:50:48,180][105620] Updated weights for policy 1, policy_version 616076 (0.0009) [2023-12-26 19:50:48,241][105620] Updated weights for policy 1, policy_version 616086 (0.0009) [2023-12-26 19:50:48,277][105692] Updated weights for policy 0, policy_version 615214 (0.0007) [2023-12-26 19:50:48,290][105620] Updated weights for policy 1, policy_version 616096 (0.0008) [2023-12-26 19:50:48,329][105692] Updated weights for policy 0, policy_version 615224 (0.0007) [2023-12-26 19:50:48,392][105692] Updated weights for policy 0, policy_version 615234 (0.0009) [2023-12-26 19:50:49,054][105620] Updated weights for policy 1, policy_version 616106 (0.0008) [2023-12-26 19:50:49,106][105620] Updated weights for policy 1, policy_version 616116 (0.0009) [2023-12-26 19:50:49,151][105692] Updated weights for policy 0, policy_version 615244 (0.0008) [2023-12-26 19:50:49,169][105620] Updated weights for policy 1, policy_version 616126 (0.0008) [2023-12-26 19:50:49,212][105692] Updated weights for policy 0, policy_version 615254 (0.0007) [2023-12-26 19:50:49,232][105620] Updated weights for policy 1, policy_version 616136 (0.0007) [2023-12-26 19:50:49,276][105692] Updated weights for policy 0, policy_version 615264 (0.0008) [2023-12-26 19:50:49,889][105620] Updated weights for policy 1, policy_version 616146 (0.0009) [2023-12-26 19:50:49,958][105620] Updated weights for policy 1, policy_version 616156 (0.0010) [2023-12-26 19:50:50,014][105620] Updated weights for policy 1, policy_version 616166 (0.0010) [2023-12-26 19:50:50,028][105692] Updated weights for policy 0, policy_version 615274 (0.0008) [2023-12-26 19:50:50,096][105692] Updated weights for policy 0, policy_version 615284 (0.0006) [2023-12-26 19:50:50,146][105692] Updated weights for policy 0, policy_version 615294 (0.0006) [2023-12-26 19:50:50,206][105692] Updated weights for policy 0, policy_version 615304 (0.0006) [2023-12-26 19:50:50,753][105620] Updated weights for policy 1, policy_version 616176 (0.0006) [2023-12-26 19:50:50,824][105620] Updated weights for policy 1, policy_version 616186 (0.0006) [2023-12-26 19:50:50,834][105692] Updated weights for policy 0, policy_version 615314 (0.0007) [2023-12-26 19:50:50,891][105692] Updated weights for policy 0, policy_version 615324 (0.0005) [2023-12-26 19:50:50,893][105620] Updated weights for policy 1, policy_version 616196 (0.0006) [2023-12-26 19:50:50,893][105585] KL-divergence is very high: 140.3087 [2023-12-26 19:50:50,939][105585] KL-divergence is very high: 230.9123 [2023-12-26 19:50:50,952][105692] Updated weights for policy 0, policy_version 615334 (0.0009) [2023-12-26 19:50:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 315318272. Throughput: 0: 9632.5, 1: 9904.4. Samples: 315300760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:50:51,062][104569] Avg episode reward: [(0, '9084.608'), (1, '9080.071')] [2023-12-26 19:50:51,445][105620] Updated weights for policy 1, policy_version 616206 (0.0009) [2023-12-26 19:50:51,515][105620] Updated weights for policy 1, policy_version 616216 (0.0011) [2023-12-26 19:50:51,574][105620] Updated weights for policy 1, policy_version 616226 (0.0010) [2023-12-26 19:50:51,730][105692] Updated weights for policy 0, policy_version 615344 (0.0009) [2023-12-26 19:50:51,790][105692] Updated weights for policy 0, policy_version 615354 (0.0009) [2023-12-26 19:50:51,848][105692] Updated weights for policy 0, policy_version 615364 (0.0005) [2023-12-26 19:50:52,289][105620] Updated weights for policy 1, policy_version 616236 (0.0009) [2023-12-26 19:50:52,361][105620] Updated weights for policy 1, policy_version 616246 (0.0009) [2023-12-26 19:50:52,421][105620] Updated weights for policy 1, policy_version 616256 (0.0011) [2023-12-26 19:50:52,470][105692] Updated weights for policy 0, policy_version 615374 (0.0005) [2023-12-26 19:50:52,527][105692] Updated weights for policy 0, policy_version 615384 (0.0005) [2023-12-26 19:50:52,588][105692] Updated weights for policy 0, policy_version 615394 (0.0006) [2023-12-26 19:50:53,036][105620] Updated weights for policy 1, policy_version 616266 (0.0011) [2023-12-26 19:50:53,096][105620] Updated weights for policy 1, policy_version 616277 (0.0010) [2023-12-26 19:50:53,147][105620] Updated weights for policy 1, policy_version 616287 (0.0010) [2023-12-26 19:50:53,275][105692] Updated weights for policy 0, policy_version 615404 (0.0007) [2023-12-26 19:50:53,325][105692] Updated weights for policy 0, policy_version 615414 (0.0005) [2023-12-26 19:50:53,381][105692] Updated weights for policy 0, policy_version 615424 (0.0008) [2023-12-26 19:50:53,905][105620] Updated weights for policy 1, policy_version 616297 (0.0010) [2023-12-26 19:50:53,968][105620] Updated weights for policy 1, policy_version 616307 (0.0009) [2023-12-26 19:50:54,033][105620] Updated weights for policy 1, policy_version 616317 (0.0010) [2023-12-26 19:50:54,085][105620] Updated weights for policy 1, policy_version 616327 (0.0010) [2023-12-26 19:50:54,101][105692] Updated weights for policy 0, policy_version 615434 (0.0008) [2023-12-26 19:50:54,146][105692] Updated weights for policy 0, policy_version 615444 (0.0008) [2023-12-26 19:50:54,195][105692] Updated weights for policy 0, policy_version 615454 (0.0008) [2023-12-26 19:50:54,239][105692] Updated weights for policy 0, policy_version 615464 (0.0008) [2023-12-26 19:50:54,802][105620] Updated weights for policy 1, policy_version 616337 (0.0010) [2023-12-26 19:50:54,853][105620] Updated weights for policy 1, policy_version 616347 (0.0010) [2023-12-26 19:50:54,905][105620] Updated weights for policy 1, policy_version 616357 (0.0010) [2023-12-26 19:50:55,037][105692] Updated weights for policy 0, policy_version 615474 (0.0008) [2023-12-26 19:50:55,082][105692] Updated weights for policy 0, policy_version 615484 (0.0008) [2023-12-26 19:50:55,130][105692] Updated weights for policy 0, policy_version 615494 (0.0008) [2023-12-26 19:50:55,672][105620] Updated weights for policy 1, policy_version 616367 (0.0010) [2023-12-26 19:50:55,736][105620] Updated weights for policy 1, policy_version 616377 (0.0010) [2023-12-26 19:50:55,797][105620] Updated weights for policy 1, policy_version 616387 (0.0010) [2023-12-26 19:50:55,906][105692] Updated weights for policy 0, policy_version 615504 (0.0008) [2023-12-26 19:50:55,950][105692] Updated weights for policy 0, policy_version 615514 (0.0008) [2023-12-26 19:50:55,997][105692] Updated weights for policy 0, policy_version 615524 (0.0008) [2023-12-26 19:50:56,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 315416576. Throughput: 0: 9599.3, 1: 9941.3. Samples: 315419012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:50:56,063][104569] Avg episode reward: [(0, '9175.525'), (1, '9173.303')] [2023-12-26 19:50:56,550][105620] Updated weights for policy 1, policy_version 616397 (0.0008) [2023-12-26 19:50:56,620][105620] Updated weights for policy 1, policy_version 616407 (0.0006) [2023-12-26 19:50:56,682][105620] Updated weights for policy 1, policy_version 616417 (0.0010) [2023-12-26 19:50:56,689][105692] Updated weights for policy 0, policy_version 615534 (0.0006) [2023-12-26 19:50:56,738][105692] Updated weights for policy 0, policy_version 615544 (0.0005) [2023-12-26 19:50:56,784][105692] Updated weights for policy 0, policy_version 615554 (0.0005) [2023-12-26 19:50:57,381][105692] Updated weights for policy 0, policy_version 615564 (0.0006) [2023-12-26 19:50:57,403][105585] KL-divergence is very high: 154.8647 [2023-12-26 19:50:57,404][105620] Updated weights for policy 1, policy_version 616427 (0.0006) [2023-12-26 19:50:57,436][105692] Updated weights for policy 0, policy_version 615574 (0.0008) [2023-12-26 19:50:57,449][105585] KL-divergence is very high: 156.1505 [2023-12-26 19:50:57,449][105620] Updated weights for policy 1, policy_version 616437 (0.0007) [2023-12-26 19:50:57,491][105692] Updated weights for policy 0, policy_version 615584 (0.0007) [2023-12-26 19:50:57,498][105620] Updated weights for policy 1, policy_version 616447 (0.0006) [2023-12-26 19:50:58,174][105620] Updated weights for policy 1, policy_version 616457 (0.0007) [2023-12-26 19:50:58,234][105620] Updated weights for policy 1, policy_version 616467 (0.0010) [2023-12-26 19:50:58,283][105692] Updated weights for policy 0, policy_version 615594 (0.0008) [2023-12-26 19:50:58,303][105620] Updated weights for policy 1, policy_version 616477 (0.0008) [2023-12-26 19:50:58,352][105692] Updated weights for policy 0, policy_version 615604 (0.0007) [2023-12-26 19:50:58,370][105620] Updated weights for policy 1, policy_version 616487 (0.0010) [2023-12-26 19:50:58,425][105692] Updated weights for policy 0, policy_version 615614 (0.0008) [2023-12-26 19:50:58,491][105692] Updated weights for policy 0, policy_version 615624 (0.0008) [2023-12-26 19:50:59,080][105620] Updated weights for policy 1, policy_version 616497 (0.0008) [2023-12-26 19:50:59,138][105692] Updated weights for policy 0, policy_version 615634 (0.0005) [2023-12-26 19:50:59,138][105620] Updated weights for policy 1, policy_version 616507 (0.0008) [2023-12-26 19:50:59,193][105692] Updated weights for policy 0, policy_version 615644 (0.0006) [2023-12-26 19:50:59,194][105620] Updated weights for policy 1, policy_version 616517 (0.0009) [2023-12-26 19:50:59,255][105692] Updated weights for policy 0, policy_version 615654 (0.0008) [2023-12-26 19:50:59,981][105620] Updated weights for policy 1, policy_version 616527 (0.0009) [2023-12-26 19:50:59,993][105692] Updated weights for policy 0, policy_version 615664 (0.0007) [2023-12-26 19:51:00,032][105620] Updated weights for policy 1, policy_version 616537 (0.0008) [2023-12-26 19:51:00,052][105692] Updated weights for policy 0, policy_version 615674 (0.0006) [2023-12-26 19:51:00,086][105620] Updated weights for policy 1, policy_version 616548 (0.0009) [2023-12-26 19:51:00,110][105692] Updated weights for policy 0, policy_version 615684 (0.0005) [2023-12-26 19:51:00,731][105692] Updated weights for policy 0, policy_version 615694 (0.0005) [2023-12-26 19:51:00,797][105692] Updated weights for policy 0, policy_version 615704 (0.0005) [2023-12-26 19:51:00,853][105692] Updated weights for policy 0, policy_version 615714 (0.0009) [2023-12-26 19:51:00,912][105620] Updated weights for policy 1, policy_version 616558 (0.0007) [2023-12-26 19:51:00,972][105620] Updated weights for policy 1, policy_version 616568 (0.0007) [2023-12-26 19:51:01,026][105620] Updated weights for policy 1, policy_version 616578 (0.0009) [2023-12-26 19:51:01,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 315514880. Throughput: 0: 9616.9, 1: 9923.9. Samples: 315477852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:51:01,062][104569] Avg episode reward: [(0, '9265.480'), (1, '9080.526')] [2023-12-26 19:51:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000615720_157655040.pth... [2023-12-26 19:51:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000616584_157859840.pth... [2023-12-26 19:51:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000614600_157368320.pth [2023-12-26 19:51:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000615432_157564928.pth [2023-12-26 19:51:01,575][105692] Updated weights for policy 0, policy_version 615725 (0.0009) [2023-12-26 19:51:01,640][105692] Updated weights for policy 0, policy_version 615735 (0.0008) [2023-12-26 19:51:01,709][105692] Updated weights for policy 0, policy_version 615745 (0.0009) [2023-12-26 19:51:01,781][105620] Updated weights for policy 1, policy_version 616588 (0.0009) [2023-12-26 19:51:01,839][105620] Updated weights for policy 1, policy_version 616598 (0.0010) [2023-12-26 19:51:01,902][105620] Updated weights for policy 1, policy_version 616608 (0.0010) [2023-12-26 19:51:02,385][105692] Updated weights for policy 0, policy_version 615755 (0.0006) [2023-12-26 19:51:02,446][105692] Updated weights for policy 0, policy_version 615765 (0.0005) [2023-12-26 19:51:02,507][105692] Updated weights for policy 0, policy_version 615775 (0.0005) [2023-12-26 19:51:02,630][105620] Updated weights for policy 1, policy_version 616618 (0.0009) [2023-12-26 19:51:02,687][105620] Updated weights for policy 1, policy_version 616628 (0.0010) [2023-12-26 19:51:02,739][105620] Updated weights for policy 1, policy_version 616638 (0.0009) [2023-12-26 19:51:02,800][105620] Updated weights for policy 1, policy_version 616648 (0.0010) [2023-12-26 19:51:03,044][105692] Updated weights for policy 0, policy_version 615785 (0.0006) [2023-12-26 19:51:03,098][105692] Updated weights for policy 0, policy_version 615795 (0.0007) [2023-12-26 19:51:03,147][105692] Updated weights for policy 0, policy_version 615805 (0.0009) [2023-12-26 19:51:03,201][105692] Updated weights for policy 0, policy_version 615815 (0.0010) [2023-12-26 19:51:03,584][105620] Updated weights for policy 1, policy_version 616658 (0.0010) [2023-12-26 19:51:03,628][105620] Updated weights for policy 1, policy_version 616668 (0.0010) [2023-12-26 19:51:03,696][105620] Updated weights for policy 1, policy_version 616678 (0.0006) [2023-12-26 19:51:03,870][105692] Updated weights for policy 0, policy_version 615825 (0.0009) [2023-12-26 19:51:03,935][105692] Updated weights for policy 0, policy_version 615835 (0.0008) [2023-12-26 19:51:03,999][105692] Updated weights for policy 0, policy_version 615845 (0.0006) [2023-12-26 19:51:04,406][105620] Updated weights for policy 1, policy_version 616688 (0.0010) [2023-12-26 19:51:04,473][105620] Updated weights for policy 1, policy_version 616698 (0.0009) [2023-12-26 19:51:04,539][105620] Updated weights for policy 1, policy_version 616708 (0.0008) [2023-12-26 19:51:04,693][105692] Updated weights for policy 0, policy_version 615855 (0.0005) [2023-12-26 19:51:04,745][105692] Updated weights for policy 0, policy_version 615865 (0.0007) [2023-12-26 19:51:04,809][105692] Updated weights for policy 0, policy_version 615875 (0.0007) [2023-12-26 19:51:05,295][105620] Updated weights for policy 1, policy_version 616718 (0.0007) [2023-12-26 19:51:05,357][105620] Updated weights for policy 1, policy_version 616728 (0.0006) [2023-12-26 19:51:05,423][105620] Updated weights for policy 1, policy_version 616738 (0.0005) [2023-12-26 19:51:05,431][105692] Updated weights for policy 0, policy_version 615885 (0.0008) [2023-12-26 19:51:05,499][105692] Updated weights for policy 0, policy_version 615895 (0.0010) [2023-12-26 19:51:05,560][105692] Updated weights for policy 0, policy_version 615905 (0.0010) [2023-12-26 19:51:05,970][105620] Updated weights for policy 1, policy_version 616748 (0.0005) [2023-12-26 19:51:06,028][105620] Updated weights for policy 1, policy_version 616758 (0.0006) [2023-12-26 19:51:06,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 315604992. Throughput: 0: 9801.3, 1: 9739.9. Samples: 315596472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:51:06,063][104569] Avg episode reward: [(0, '9082.601'), (1, '8989.294')] [2023-12-26 19:51:06,091][105620] Updated weights for policy 1, policy_version 616768 (0.0008) [2023-12-26 19:51:06,240][105692] Updated weights for policy 0, policy_version 615915 (0.0010) [2023-12-26 19:51:06,306][105692] Updated weights for policy 0, policy_version 615925 (0.0008) [2023-12-26 19:51:06,371][105692] Updated weights for policy 0, policy_version 615935 (0.0006) [2023-12-26 19:51:06,801][105620] Updated weights for policy 1, policy_version 616778 (0.0009) [2023-12-26 19:51:06,852][105620] Updated weights for policy 1, policy_version 616788 (0.0006) [2023-12-26 19:51:06,912][105620] Updated weights for policy 1, policy_version 616798 (0.0008) [2023-12-26 19:51:06,930][105692] Updated weights for policy 0, policy_version 615945 (0.0006) [2023-12-26 19:51:06,961][105620] Updated weights for policy 1, policy_version 616808 (0.0005) [2023-12-26 19:51:06,997][105692] Updated weights for policy 0, policy_version 615955 (0.0008) [2023-12-26 19:51:07,060][105692] Updated weights for policy 0, policy_version 615965 (0.0008) [2023-12-26 19:51:07,123][105692] Updated weights for policy 0, policy_version 615975 (0.0008) [2023-12-26 19:51:07,595][105620] Updated weights for policy 1, policy_version 616818 (0.0008) [2023-12-26 19:51:07,657][105620] Updated weights for policy 1, policy_version 616828 (0.0009) [2023-12-26 19:51:07,728][105620] Updated weights for policy 1, policy_version 616838 (0.0009) [2023-12-26 19:51:07,779][105692] Updated weights for policy 0, policy_version 615985 (0.0007) [2023-12-26 19:51:07,827][105692] Updated weights for policy 0, policy_version 615995 (0.0007) [2023-12-26 19:51:07,879][105692] Updated weights for policy 0, policy_version 616005 (0.0005) [2023-12-26 19:51:08,410][105620] Updated weights for policy 1, policy_version 616848 (0.0011) [2023-12-26 19:51:08,498][105620] Updated weights for policy 1, policy_version 616858 (0.0011) [2023-12-26 19:51:08,547][105692] Updated weights for policy 0, policy_version 616015 (0.0008) [2023-12-26 19:51:08,554][105620] Updated weights for policy 1, policy_version 616868 (0.0011) [2023-12-26 19:51:08,603][105692] Updated weights for policy 0, policy_version 616025 (0.0008) [2023-12-26 19:51:08,667][105692] Updated weights for policy 0, policy_version 616035 (0.0008) [2023-12-26 19:51:09,168][105620] Updated weights for policy 1, policy_version 616878 (0.0011) [2023-12-26 19:51:09,232][105620] Updated weights for policy 1, policy_version 616888 (0.0011) [2023-12-26 19:51:09,297][105620] Updated weights for policy 1, policy_version 616898 (0.0010) [2023-12-26 19:51:09,420][105692] Updated weights for policy 0, policy_version 616045 (0.0007) [2023-12-26 19:51:09,478][105692] Updated weights for policy 0, policy_version 616055 (0.0008) [2023-12-26 19:51:09,538][105692] Updated weights for policy 0, policy_version 616065 (0.0008) [2023-12-26 19:51:10,049][105620] Updated weights for policy 1, policy_version 616908 (0.0011) [2023-12-26 19:51:10,111][105620] Updated weights for policy 1, policy_version 616918 (0.0010) [2023-12-26 19:51:10,171][105620] Updated weights for policy 1, policy_version 616928 (0.0011) [2023-12-26 19:51:10,277][105692] Updated weights for policy 0, policy_version 616075 (0.0008) [2023-12-26 19:51:10,333][105692] Updated weights for policy 0, policy_version 616085 (0.0006) [2023-12-26 19:51:10,393][105692] Updated weights for policy 0, policy_version 616095 (0.0008) [2023-12-26 19:51:10,853][105620] Updated weights for policy 1, policy_version 616938 (0.0010) [2023-12-26 19:51:10,920][105620] Updated weights for policy 1, policy_version 616948 (0.0006) [2023-12-26 19:51:10,988][105620] Updated weights for policy 1, policy_version 616958 (0.0010) [2023-12-26 19:51:11,055][105620] Updated weights for policy 1, policy_version 616968 (0.0010) [2023-12-26 19:51:11,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 315711488. Throughput: 0: 9849.9, 1: 9798.5. Samples: 315718500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:51:11,063][104569] Avg episode reward: [(0, '9083.164'), (1, '9082.206')] [2023-12-26 19:51:11,120][105692] Updated weights for policy 0, policy_version 616105 (0.0006) [2023-12-26 19:51:11,189][105692] Updated weights for policy 0, policy_version 616115 (0.0007) [2023-12-26 19:51:11,247][105692] Updated weights for policy 0, policy_version 616125 (0.0009) [2023-12-26 19:51:11,309][105692] Updated weights for policy 0, policy_version 616135 (0.0007) [2023-12-26 19:51:11,792][105620] Updated weights for policy 1, policy_version 616978 (0.0008) [2023-12-26 19:51:11,847][105620] Updated weights for policy 1, policy_version 616988 (0.0008) [2023-12-26 19:51:11,899][105620] Updated weights for policy 1, policy_version 616998 (0.0006) [2023-12-26 19:51:12,095][105692] Updated weights for policy 0, policy_version 616145 (0.0009) [2023-12-26 19:51:12,160][105692] Updated weights for policy 0, policy_version 616155 (0.0009) [2023-12-26 19:51:12,224][105692] Updated weights for policy 0, policy_version 616165 (0.0010) [2023-12-26 19:51:12,634][105620] Updated weights for policy 1, policy_version 617008 (0.0006) [2023-12-26 19:51:12,691][105620] Updated weights for policy 1, policy_version 617018 (0.0005) [2023-12-26 19:51:12,740][105620] Updated weights for policy 1, policy_version 617028 (0.0007) [2023-12-26 19:51:13,055][105692] Updated weights for policy 0, policy_version 616175 (0.0009) [2023-12-26 19:51:13,114][105692] Updated weights for policy 0, policy_version 616185 (0.0008) [2023-12-26 19:51:13,171][105692] Updated weights for policy 0, policy_version 616195 (0.0007) [2023-12-26 19:51:13,407][105620] Updated weights for policy 1, policy_version 617038 (0.0009) [2023-12-26 19:51:13,465][105620] Updated weights for policy 1, policy_version 617048 (0.0009) [2023-12-26 19:51:13,518][105620] Updated weights for policy 1, policy_version 617058 (0.0009) [2023-12-26 19:51:13,923][105692] Updated weights for policy 0, policy_version 616205 (0.0007) [2023-12-26 19:51:13,975][105692] Updated weights for policy 0, policy_version 616215 (0.0009) [2023-12-26 19:51:14,023][105692] Updated weights for policy 0, policy_version 616225 (0.0009) [2023-12-26 19:51:14,235][105620] Updated weights for policy 1, policy_version 617068 (0.0008) [2023-12-26 19:51:14,294][105620] Updated weights for policy 1, policy_version 617078 (0.0007) [2023-12-26 19:51:14,345][105620] Updated weights for policy 1, policy_version 617088 (0.0005) [2023-12-26 19:51:14,801][105692] Updated weights for policy 0, policy_version 616236 (0.0010) [2023-12-26 19:51:14,865][105692] Updated weights for policy 0, policy_version 616246 (0.0006) [2023-12-26 19:51:14,928][105692] Updated weights for policy 0, policy_version 616256 (0.0008) [2023-12-26 19:51:14,978][105620] Updated weights for policy 1, policy_version 617098 (0.0007) [2023-12-26 19:51:15,039][105620] Updated weights for policy 1, policy_version 617108 (0.0011) [2023-12-26 19:51:15,099][105620] Updated weights for policy 1, policy_version 617118 (0.0011) [2023-12-26 19:51:15,167][105620] Updated weights for policy 1, policy_version 617128 (0.0011) [2023-12-26 19:51:15,663][105692] Updated weights for policy 0, policy_version 616266 (0.0008) [2023-12-26 19:51:15,713][105692] Updated weights for policy 0, policy_version 616276 (0.0005) [2023-12-26 19:51:15,769][105692] Updated weights for policy 0, policy_version 616286 (0.0006) [2023-12-26 19:51:15,835][105692] Updated weights for policy 0, policy_version 616296 (0.0007) [2023-12-26 19:51:15,902][105620] Updated weights for policy 1, policy_version 617138 (0.0011) [2023-12-26 19:51:15,960][105620] Updated weights for policy 1, policy_version 617148 (0.0011) [2023-12-26 19:51:16,018][105620] Updated weights for policy 1, policy_version 617158 (0.0010) [2023-12-26 19:51:16,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 315809792. Throughput: 0: 9777.6, 1: 9718.0. Samples: 315773340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:51:16,062][104569] Avg episode reward: [(0, '9266.206'), (1, '8921.904')] [2023-12-26 19:51:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000616296_157802496.pth... [2023-12-26 19:51:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000617160_158007296.pth... [2023-12-26 19:51:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000615144_157507584.pth [2023-12-26 19:51:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000616008_157712384.pth [2023-12-26 19:51:16,429][105692] Updated weights for policy 0, policy_version 616306 (0.0006) [2023-12-26 19:51:16,482][105692] Updated weights for policy 0, policy_version 616316 (0.0005) [2023-12-26 19:51:16,536][105692] Updated weights for policy 0, policy_version 616326 (0.0009) [2023-12-26 19:51:16,726][105620] Updated weights for policy 1, policy_version 617168 (0.0011) [2023-12-26 19:51:16,788][105620] Updated weights for policy 1, policy_version 617178 (0.0011) [2023-12-26 19:51:16,850][105620] Updated weights for policy 1, policy_version 617188 (0.0010) [2023-12-26 19:51:17,088][105692] Updated weights for policy 0, policy_version 616336 (0.0006) [2023-12-26 19:51:17,106][105585] KL-divergence is very high: 213.9107 [2023-12-26 19:51:17,125][105585] KL-divergence is very high: 274.2281 [2023-12-26 19:51:17,135][105692] Updated weights for policy 0, policy_version 616346 (0.0005) [2023-12-26 19:51:17,145][105585] KL-divergence is very high: 263.8802 [2023-12-26 19:51:17,163][105585] KL-divergence is very high: 223.2484 [2023-12-26 19:51:17,181][105692] Updated weights for policy 0, policy_version 616356 (0.0005) [2023-12-26 19:51:17,182][105585] KL-divergence is very high: 145.1140 [2023-12-26 19:51:17,431][105620] Updated weights for policy 1, policy_version 617198 (0.0007) [2023-12-26 19:51:17,488][105620] Updated weights for policy 1, policy_version 617209 (0.0009) [2023-12-26 19:51:17,541][105620] Updated weights for policy 1, policy_version 617219 (0.0009) [2023-12-26 19:51:17,712][105692] Updated weights for policy 0, policy_version 616366 (0.0005) [2023-12-26 19:51:17,785][105692] Updated weights for policy 0, policy_version 616376 (0.0005) [2023-12-26 19:51:17,849][105692] Updated weights for policy 0, policy_version 616386 (0.0007) [2023-12-26 19:51:18,174][105620] Updated weights for policy 1, policy_version 617229 (0.0009) [2023-12-26 19:51:18,224][105620] Updated weights for policy 1, policy_version 617239 (0.0009) [2023-12-26 19:51:18,282][105620] Updated weights for policy 1, policy_version 617249 (0.0010) [2023-12-26 19:51:18,446][105692] Updated weights for policy 0, policy_version 616396 (0.0008) [2023-12-26 19:51:18,511][105692] Updated weights for policy 0, policy_version 616406 (0.0010) [2023-12-26 19:51:18,573][105692] Updated weights for policy 0, policy_version 616416 (0.0010) [2023-12-26 19:51:19,020][105620] Updated weights for policy 1, policy_version 617259 (0.0009) [2023-12-26 19:51:19,076][105620] Updated weights for policy 1, policy_version 617269 (0.0008) [2023-12-26 19:51:19,129][105620] Updated weights for policy 1, policy_version 617279 (0.0008) [2023-12-26 19:51:19,311][105692] Updated weights for policy 0, policy_version 616426 (0.0011) [2023-12-26 19:51:19,375][105692] Updated weights for policy 0, policy_version 616436 (0.0009) [2023-12-26 19:51:19,442][105692] Updated weights for policy 0, policy_version 616446 (0.0010) [2023-12-26 19:51:19,510][105692] Updated weights for policy 0, policy_version 616456 (0.0011) [2023-12-26 19:51:19,758][105620] Updated weights for policy 1, policy_version 617289 (0.0006) [2023-12-26 19:51:19,813][105620] Updated weights for policy 1, policy_version 617299 (0.0007) [2023-12-26 19:51:19,886][105620] Updated weights for policy 1, policy_version 617309 (0.0007) [2023-12-26 19:51:19,952][105620] Updated weights for policy 1, policy_version 617319 (0.0008) [2023-12-26 19:51:20,324][105692] Updated weights for policy 0, policy_version 616466 (0.0009) [2023-12-26 19:51:20,384][105692] Updated weights for policy 0, policy_version 616476 (0.0010) [2023-12-26 19:51:20,442][105692] Updated weights for policy 0, policy_version 616486 (0.0010) [2023-12-26 19:51:20,545][105620] Updated weights for policy 1, policy_version 617329 (0.0009) [2023-12-26 19:51:20,606][105620] Updated weights for policy 1, policy_version 617339 (0.0008) [2023-12-26 19:51:20,662][105620] Updated weights for policy 1, policy_version 617349 (0.0008) [2023-12-26 19:51:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 315908096. Throughput: 0: 9853.7, 1: 9762.0. Samples: 315897864. Policy #0 lag: (min: 17.0, avg: 31.6, max: 49.0) [2023-12-26 19:51:21,063][104569] Avg episode reward: [(0, '9086.512'), (1, '8919.236')] [2023-12-26 19:51:21,189][105692] Updated weights for policy 0, policy_version 616496 (0.0009) [2023-12-26 19:51:21,252][105692] Updated weights for policy 0, policy_version 616506 (0.0009) [2023-12-26 19:51:21,313][105692] Updated weights for policy 0, policy_version 616516 (0.0008) [2023-12-26 19:51:21,422][105620] Updated weights for policy 1, policy_version 617359 (0.0009) [2023-12-26 19:51:21,478][105620] Updated weights for policy 1, policy_version 617369 (0.0009) [2023-12-26 19:51:21,548][105620] Updated weights for policy 1, policy_version 617379 (0.0006) [2023-12-26 19:51:22,140][105692] Updated weights for policy 0, policy_version 616526 (0.0008) [2023-12-26 19:51:22,189][105692] Updated weights for policy 0, policy_version 616536 (0.0006) [2023-12-26 19:51:22,208][105620] Updated weights for policy 1, policy_version 617389 (0.0008) [2023-12-26 19:51:22,247][105692] Updated weights for policy 0, policy_version 616546 (0.0006) [2023-12-26 19:51:22,265][105620] Updated weights for policy 1, policy_version 617399 (0.0008) [2023-12-26 19:51:22,323][105620] Updated weights for policy 1, policy_version 617409 (0.0009) [2023-12-26 19:51:22,930][105692] Updated weights for policy 0, policy_version 616556 (0.0007) [2023-12-26 19:51:22,988][105692] Updated weights for policy 0, policy_version 616566 (0.0005) [2023-12-26 19:51:23,047][105692] Updated weights for policy 0, policy_version 616576 (0.0009) [2023-12-26 19:51:23,152][105620] Updated weights for policy 1, policy_version 617419 (0.0009) [2023-12-26 19:51:23,206][105620] Updated weights for policy 1, policy_version 617430 (0.0010) [2023-12-26 19:51:23,256][105620] Updated weights for policy 1, policy_version 617440 (0.0009) [2023-12-26 19:51:23,641][105692] Updated weights for policy 0, policy_version 616586 (0.0006) [2023-12-26 19:51:23,693][105692] Updated weights for policy 0, policy_version 616596 (0.0005) [2023-12-26 19:51:23,747][105692] Updated weights for policy 0, policy_version 616606 (0.0006) [2023-12-26 19:51:23,795][105692] Updated weights for policy 0, policy_version 616616 (0.0010) [2023-12-26 19:51:24,154][105620] Updated weights for policy 1, policy_version 617450 (0.0008) [2023-12-26 19:51:24,223][105620] Updated weights for policy 1, policy_version 617460 (0.0009) [2023-12-26 19:51:24,279][105620] Updated weights for policy 1, policy_version 617470 (0.0009) [2023-12-26 19:51:24,329][105692] Updated weights for policy 0, policy_version 616626 (0.0006) [2023-12-26 19:51:24,341][105620] Updated weights for policy 1, policy_version 617480 (0.0008) [2023-12-26 19:51:24,386][105692] Updated weights for policy 0, policy_version 616636 (0.0006) [2023-12-26 19:51:24,450][105692] Updated weights for policy 0, policy_version 616646 (0.0006) [2023-12-26 19:51:25,041][105692] Updated weights for policy 0, policy_version 616656 (0.0006) [2023-12-26 19:51:25,047][105620] Updated weights for policy 1, policy_version 617490 (0.0006) [2023-12-26 19:51:25,104][105692] Updated weights for policy 0, policy_version 616666 (0.0006) [2023-12-26 19:51:25,106][105620] Updated weights for policy 1, policy_version 617500 (0.0008) [2023-12-26 19:51:25,164][105692] Updated weights for policy 0, policy_version 616676 (0.0007) [2023-12-26 19:51:25,170][105620] Updated weights for policy 1, policy_version 617510 (0.0007) [2023-12-26 19:51:25,731][105692] Updated weights for policy 0, policy_version 616686 (0.0007) [2023-12-26 19:51:25,783][105692] Updated weights for policy 0, policy_version 616696 (0.0005) [2023-12-26 19:51:25,830][105692] Updated weights for policy 0, policy_version 616706 (0.0005) [2023-12-26 19:51:25,933][105620] Updated weights for policy 1, policy_version 617520 (0.0007) [2023-12-26 19:51:26,000][105620] Updated weights for policy 1, policy_version 617530 (0.0008) [2023-12-26 19:51:26,056][105620] Updated weights for policy 1, policy_version 617540 (0.0008) [2023-12-26 19:51:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 316006400. Throughput: 0: 9996.5, 1: 9662.1. Samples: 316015136. Policy #0 lag: (min: 17.0, avg: 31.6, max: 49.0) [2023-12-26 19:51:26,062][104569] Avg episode reward: [(0, '9086.593'), (1, '9170.937')] [2023-12-26 19:51:26,462][105692] Updated weights for policy 0, policy_version 616716 (0.0005) [2023-12-26 19:51:26,509][105692] Updated weights for policy 0, policy_version 616726 (0.0005) [2023-12-26 19:51:26,564][105692] Updated weights for policy 0, policy_version 616736 (0.0005) [2023-12-26 19:51:26,925][105620] Updated weights for policy 1, policy_version 617550 (0.0008) [2023-12-26 19:51:26,984][105620] Updated weights for policy 1, policy_version 617561 (0.0011) [2023-12-26 19:51:27,035][105620] Updated weights for policy 1, policy_version 617571 (0.0009) [2023-12-26 19:51:27,071][105692] Updated weights for policy 0, policy_version 616746 (0.0005) [2023-12-26 19:51:27,127][105692] Updated weights for policy 0, policy_version 616756 (0.0005) [2023-12-26 19:51:27,187][105692] Updated weights for policy 0, policy_version 616766 (0.0010) [2023-12-26 19:51:27,241][105692] Updated weights for policy 0, policy_version 616776 (0.0010) [2023-12-26 19:51:27,855][105692] Updated weights for policy 0, policy_version 616786 (0.0009) [2023-12-26 19:51:27,871][105620] Updated weights for policy 1, policy_version 617581 (0.0008) [2023-12-26 19:51:27,898][105692] Updated weights for policy 0, policy_version 616796 (0.0007) [2023-12-26 19:51:27,931][105620] Updated weights for policy 1, policy_version 617591 (0.0008) [2023-12-26 19:51:27,957][105692] Updated weights for policy 0, policy_version 616806 (0.0008) [2023-12-26 19:51:27,998][105620] Updated weights for policy 1, policy_version 617601 (0.0005) [2023-12-26 19:51:28,541][105620] Updated weights for policy 1, policy_version 617611 (0.0005) [2023-12-26 19:51:28,600][105620] Updated weights for policy 1, policy_version 617621 (0.0008) [2023-12-26 19:51:28,659][105620] Updated weights for policy 1, policy_version 617631 (0.0009) [2023-12-26 19:51:28,752][105692] Updated weights for policy 0, policy_version 616816 (0.0010) [2023-12-26 19:51:28,800][105692] Updated weights for policy 0, policy_version 616826 (0.0009) [2023-12-26 19:51:28,864][105692] Updated weights for policy 0, policy_version 616836 (0.0005) [2023-12-26 19:51:29,362][105620] Updated weights for policy 1, policy_version 617641 (0.0008) [2023-12-26 19:51:29,425][105620] Updated weights for policy 1, policy_version 617651 (0.0009) [2023-12-26 19:51:29,487][105620] Updated weights for policy 1, policy_version 617661 (0.0009) [2023-12-26 19:51:29,543][105620] Updated weights for policy 1, policy_version 617671 (0.0008) [2023-12-26 19:51:29,553][105692] Updated weights for policy 0, policy_version 616846 (0.0007) [2023-12-26 19:51:29,614][105692] Updated weights for policy 0, policy_version 616856 (0.0009) [2023-12-26 19:51:29,673][105692] Updated weights for policy 0, policy_version 616866 (0.0008) [2023-12-26 19:51:30,299][105620] Updated weights for policy 1, policy_version 617681 (0.0006) [2023-12-26 19:51:30,358][105620] Updated weights for policy 1, policy_version 617691 (0.0005) [2023-12-26 19:51:30,426][105620] Updated weights for policy 1, policy_version 617701 (0.0005) [2023-12-26 19:51:30,467][105692] Updated weights for policy 0, policy_version 616876 (0.0010) [2023-12-26 19:51:30,519][105692] Updated weights for policy 0, policy_version 616886 (0.0009) [2023-12-26 19:51:30,578][105692] Updated weights for policy 0, policy_version 616896 (0.0008) [2023-12-26 19:51:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 316104704. Throughput: 0: 10095.6, 1: 9691.5. Samples: 316076124. Policy #0 lag: (min: 17.0, avg: 31.6, max: 49.0) [2023-12-26 19:51:31,062][104569] Avg episode reward: [(0, '9266.013'), (1, '9264.928')] [2023-12-26 19:51:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000616904_157958144.pth... [2023-12-26 19:51:31,068][105620] Updated weights for policy 1, policy_version 617711 (0.0007) [2023-12-26 19:51:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000615720_157655040.pth [2023-12-26 19:51:31,130][105620] Updated weights for policy 1, policy_version 617721 (0.0009) [2023-12-26 19:51:31,194][105620] Updated weights for policy 1, policy_version 617731 (0.0008) [2023-12-26 19:51:31,225][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000617736_158154752.pth... [2023-12-26 19:51:31,230][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000616584_157859840.pth [2023-12-26 19:51:31,380][105692] Updated weights for policy 0, policy_version 616906 (0.0009) [2023-12-26 19:51:31,436][105692] Updated weights for policy 0, policy_version 616916 (0.0006) [2023-12-26 19:51:31,489][105692] Updated weights for policy 0, policy_version 616926 (0.0008) [2023-12-26 19:51:31,540][105692] Updated weights for policy 0, policy_version 616936 (0.0009) [2023-12-26 19:51:31,909][105620] Updated weights for policy 1, policy_version 617741 (0.0009) [2023-12-26 19:51:31,969][105620] Updated weights for policy 1, policy_version 617751 (0.0009) [2023-12-26 19:51:32,027][105620] Updated weights for policy 1, policy_version 617761 (0.0008) [2023-12-26 19:51:32,295][105692] Updated weights for policy 0, policy_version 616946 (0.0010) [2023-12-26 19:51:32,354][105692] Updated weights for policy 0, policy_version 616956 (0.0009) [2023-12-26 19:51:32,413][105692] Updated weights for policy 0, policy_version 616966 (0.0010) [2023-12-26 19:51:32,805][105620] Updated weights for policy 1, policy_version 617771 (0.0009) [2023-12-26 19:51:32,849][105620] Updated weights for policy 1, policy_version 617781 (0.0008) [2023-12-26 19:51:32,892][105620] Updated weights for policy 1, policy_version 617791 (0.0007) [2023-12-26 19:51:33,125][105692] Updated weights for policy 0, policy_version 616976 (0.0008) [2023-12-26 19:51:33,186][105692] Updated weights for policy 0, policy_version 616986 (0.0010) [2023-12-26 19:51:33,250][105692] Updated weights for policy 0, policy_version 616996 (0.0010) [2023-12-26 19:51:33,683][105620] Updated weights for policy 1, policy_version 617801 (0.0008) [2023-12-26 19:51:33,736][105620] Updated weights for policy 1, policy_version 617811 (0.0009) [2023-12-26 19:51:33,793][105620] Updated weights for policy 1, policy_version 617821 (0.0009) [2023-12-26 19:51:33,810][105692] Updated weights for policy 0, policy_version 617006 (0.0010) [2023-12-26 19:51:33,844][105620] Updated weights for policy 1, policy_version 617831 (0.0005) [2023-12-26 19:51:33,850][105585] KL-divergence is very high: 230.3082 [2023-12-26 19:51:33,854][105692] Updated weights for policy 0, policy_version 617016 (0.0010) [2023-12-26 19:51:33,887][105585] KL-divergence is very high: 362.2557 [2023-12-26 19:51:33,904][105692] Updated weights for policy 0, policy_version 617026 (0.0010) [2023-12-26 19:51:33,931][105585] KL-divergence is very high: 272.7990 [2023-12-26 19:51:34,557][105692] Updated weights for policy 0, policy_version 617036 (0.0010) [2023-12-26 19:51:34,589][105620] Updated weights for policy 1, policy_version 617841 (0.0010) [2023-12-26 19:51:34,616][105692] Updated weights for policy 0, policy_version 617046 (0.0010) [2023-12-26 19:51:34,648][105620] Updated weights for policy 1, policy_version 617851 (0.0010) [2023-12-26 19:51:34,678][105692] Updated weights for policy 0, policy_version 617056 (0.0010) [2023-12-26 19:51:34,713][105620] Updated weights for policy 1, policy_version 617861 (0.0011) [2023-12-26 19:51:35,342][105692] Updated weights for policy 0, policy_version 617066 (0.0008) [2023-12-26 19:51:35,405][105692] Updated weights for policy 0, policy_version 617076 (0.0005) [2023-12-26 19:51:35,457][105692] Updated weights for policy 0, policy_version 617086 (0.0005) [2023-12-26 19:51:35,463][105620] Updated weights for policy 1, policy_version 617871 (0.0011) [2023-12-26 19:51:35,506][105692] Updated weights for policy 0, policy_version 617096 (0.0009) [2023-12-26 19:51:35,522][105620] Updated weights for policy 1, policy_version 617881 (0.0010) [2023-12-26 19:51:35,577][105620] Updated weights for policy 1, policy_version 617891 (0.0010) [2023-12-26 19:51:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 316203008. Throughput: 0: 10125.8, 1: 9690.9. Samples: 316192508. Policy #0 lag: (min: 17.0, avg: 31.6, max: 49.0) [2023-12-26 19:51:36,062][104569] Avg episode reward: [(0, '9172.639'), (1, '8562.203')] [2023-12-26 19:51:36,168][105692] Updated weights for policy 0, policy_version 617106 (0.0008) [2023-12-26 19:51:36,227][105692] Updated weights for policy 0, policy_version 617116 (0.0010) [2023-12-26 19:51:36,284][105692] Updated weights for policy 0, policy_version 617126 (0.0010) [2023-12-26 19:51:36,298][105620] Updated weights for policy 1, policy_version 617901 (0.0008) [2023-12-26 19:51:36,356][105620] Updated weights for policy 1, policy_version 617911 (0.0006) [2023-12-26 19:51:36,425][105620] Updated weights for policy 1, policy_version 617921 (0.0007) [2023-12-26 19:51:37,042][105620] Updated weights for policy 1, policy_version 617931 (0.0006) [2023-12-26 19:51:37,096][105692] Updated weights for policy 0, policy_version 617136 (0.0007) [2023-12-26 19:51:37,101][105620] Updated weights for policy 1, policy_version 617941 (0.0009) [2023-12-26 19:51:37,157][105620] Updated weights for policy 1, policy_version 617951 (0.0010) [2023-12-26 19:51:37,158][105692] Updated weights for policy 0, policy_version 617146 (0.0010) [2023-12-26 19:51:37,225][105692] Updated weights for policy 0, policy_version 617156 (0.0011) [2023-12-26 19:51:37,849][105620] Updated weights for policy 1, policy_version 617961 (0.0010) [2023-12-26 19:51:37,901][105620] Updated weights for policy 1, policy_version 617971 (0.0005) [2023-12-26 19:51:37,941][105692] Updated weights for policy 0, policy_version 617166 (0.0010) [2023-12-26 19:51:37,952][105620] Updated weights for policy 1, policy_version 617981 (0.0005) [2023-12-26 19:51:38,002][105692] Updated weights for policy 0, policy_version 617176 (0.0010) [2023-12-26 19:51:38,013][105620] Updated weights for policy 1, policy_version 617991 (0.0006) [2023-12-26 19:51:38,050][105692] Updated weights for policy 0, policy_version 617186 (0.0010) [2023-12-26 19:51:38,752][105692] Updated weights for policy 0, policy_version 617196 (0.0008) [2023-12-26 19:51:38,773][105620] Updated weights for policy 1, policy_version 618001 (0.0008) [2023-12-26 19:51:38,805][105692] Updated weights for policy 0, policy_version 617206 (0.0006) [2023-12-26 19:51:38,828][105620] Updated weights for policy 1, policy_version 618011 (0.0008) [2023-12-26 19:51:38,856][105692] Updated weights for policy 0, policy_version 617216 (0.0007) [2023-12-26 19:51:38,887][105620] Updated weights for policy 1, policy_version 618021 (0.0008) [2023-12-26 19:51:39,512][105620] Updated weights for policy 1, policy_version 618031 (0.0008) [2023-12-26 19:51:39,570][105620] Updated weights for policy 1, policy_version 618041 (0.0009) [2023-12-26 19:51:39,573][105692] Updated weights for policy 0, policy_version 617226 (0.0007) [2023-12-26 19:51:39,619][105620] Updated weights for policy 1, policy_version 618051 (0.0007) [2023-12-26 19:51:39,626][105692] Updated weights for policy 0, policy_version 617236 (0.0007) [2023-12-26 19:51:39,687][105692] Updated weights for policy 0, policy_version 617246 (0.0009) [2023-12-26 19:51:39,751][105692] Updated weights for policy 0, policy_version 617256 (0.0009) [2023-12-26 19:51:40,457][105620] Updated weights for policy 1, policy_version 618061 (0.0009) [2023-12-26 19:51:40,522][105620] Updated weights for policy 1, policy_version 618071 (0.0008) [2023-12-26 19:51:40,558][105692] Updated weights for policy 0, policy_version 617266 (0.0008) [2023-12-26 19:51:40,582][105620] Updated weights for policy 1, policy_version 618081 (0.0007) [2023-12-26 19:51:40,622][105692] Updated weights for policy 0, policy_version 617276 (0.0010) [2023-12-26 19:51:40,672][105692] Updated weights for policy 0, policy_version 617286 (0.0009) [2023-12-26 19:51:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 316301312. Throughput: 0: 10095.7, 1: 9687.0. Samples: 316309228. Policy #0 lag: (min: 17.0, avg: 31.6, max: 49.0) [2023-12-26 19:51:41,062][104569] Avg episode reward: [(0, '9262.626'), (1, '8127.947')] [2023-12-26 19:51:41,295][105620] Updated weights for policy 1, policy_version 618091 (0.0011) [2023-12-26 19:51:41,363][105620] Updated weights for policy 1, policy_version 618101 (0.0011) [2023-12-26 19:51:41,433][105620] Updated weights for policy 1, policy_version 618111 (0.0010) [2023-12-26 19:51:41,445][105692] Updated weights for policy 0, policy_version 617296 (0.0009) [2023-12-26 19:51:41,507][105692] Updated weights for policy 0, policy_version 617306 (0.0008) [2023-12-26 19:51:41,570][105692] Updated weights for policy 0, policy_version 617316 (0.0008) [2023-12-26 19:51:42,121][105620] Updated weights for policy 1, policy_version 618121 (0.0010) [2023-12-26 19:51:42,194][105620] Updated weights for policy 1, policy_version 618131 (0.0006) [2023-12-26 19:51:42,256][105620] Updated weights for policy 1, policy_version 618141 (0.0006) [2023-12-26 19:51:42,311][105692] Updated weights for policy 0, policy_version 617326 (0.0007) [2023-12-26 19:51:42,317][105620] Updated weights for policy 1, policy_version 618151 (0.0007) [2023-12-26 19:51:42,377][105692] Updated weights for policy 0, policy_version 617336 (0.0009) [2023-12-26 19:51:42,435][105692] Updated weights for policy 0, policy_version 617346 (0.0009) [2023-12-26 19:51:42,988][105620] Updated weights for policy 1, policy_version 618161 (0.0009) [2023-12-26 19:51:43,042][105620] Updated weights for policy 1, policy_version 618171 (0.0009) [2023-12-26 19:51:43,093][105620] Updated weights for policy 1, policy_version 618181 (0.0008) [2023-12-26 19:51:43,218][105692] Updated weights for policy 0, policy_version 617356 (0.0009) [2023-12-26 19:51:43,264][105692] Updated weights for policy 0, policy_version 617366 (0.0008) [2023-12-26 19:51:43,308][105692] Updated weights for policy 0, policy_version 617376 (0.0006) [2023-12-26 19:51:43,842][105620] Updated weights for policy 1, policy_version 618191 (0.0009) [2023-12-26 19:51:43,905][105620] Updated weights for policy 1, policy_version 618201 (0.0008) [2023-12-26 19:51:43,968][105620] Updated weights for policy 1, policy_version 618211 (0.0008) [2023-12-26 19:51:44,086][105692] Updated weights for policy 0, policy_version 617386 (0.0008) [2023-12-26 19:51:44,156][105692] Updated weights for policy 0, policy_version 617396 (0.0010) [2023-12-26 19:51:44,222][105692] Updated weights for policy 0, policy_version 617406 (0.0009) [2023-12-26 19:51:44,295][105692] Updated weights for policy 0, policy_version 617416 (0.0010) [2023-12-26 19:51:44,664][105620] Updated weights for policy 1, policy_version 618221 (0.0007) [2023-12-26 19:51:44,716][105620] Updated weights for policy 1, policy_version 618231 (0.0005) [2023-12-26 19:51:44,770][105620] Updated weights for policy 1, policy_version 618241 (0.0006) [2023-12-26 19:51:45,018][105692] Updated weights for policy 0, policy_version 617426 (0.0006) [2023-12-26 19:51:45,075][105692] Updated weights for policy 0, policy_version 617436 (0.0005) [2023-12-26 19:51:45,131][105692] Updated weights for policy 0, policy_version 617446 (0.0006) [2023-12-26 19:51:45,501][105620] Updated weights for policy 1, policy_version 618251 (0.0006) [2023-12-26 19:51:45,553][105620] Updated weights for policy 1, policy_version 618261 (0.0006) [2023-12-26 19:51:45,602][105620] Updated weights for policy 1, policy_version 618271 (0.0006) [2023-12-26 19:51:45,715][105692] Updated weights for policy 0, policy_version 617456 (0.0006) [2023-12-26 19:51:45,767][105692] Updated weights for policy 0, policy_version 617466 (0.0005) [2023-12-26 19:51:45,821][105692] Updated weights for policy 0, policy_version 617476 (0.0005) [2023-12-26 19:51:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 316399616. Throughput: 0: 10028.6, 1: 9695.9. Samples: 316365456. Policy #0 lag: (min: 17.0, avg: 31.6, max: 49.0) [2023-12-26 19:51:46,062][104569] Avg episode reward: [(0, '9355.465'), (1, '8573.324')] [2023-12-26 19:51:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000618280_158294016.pth... [2023-12-26 19:51:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000617480_158105600.pth... [2023-12-26 19:51:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000616296_157802496.pth [2023-12-26 19:51:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000617160_158007296.pth [2023-12-26 19:51:46,220][105620] Updated weights for policy 1, policy_version 618281 (0.0008) [2023-12-26 19:51:46,278][105620] Updated weights for policy 1, policy_version 618291 (0.0009) [2023-12-26 19:51:46,342][105620] Updated weights for policy 1, policy_version 618301 (0.0009) [2023-12-26 19:51:46,398][105692] Updated weights for policy 0, policy_version 617486 (0.0009) [2023-12-26 19:51:46,399][105620] Updated weights for policy 1, policy_version 618311 (0.0007) [2023-12-26 19:51:46,457][105692] Updated weights for policy 0, policy_version 617496 (0.0006) [2023-12-26 19:51:46,506][105692] Updated weights for policy 0, policy_version 617506 (0.0006) [2023-12-26 19:51:47,136][105620] Updated weights for policy 1, policy_version 618321 (0.0009) [2023-12-26 19:51:47,198][105620] Updated weights for policy 1, policy_version 618331 (0.0008) [2023-12-26 19:51:47,204][105692] Updated weights for policy 0, policy_version 617516 (0.0006) [2023-12-26 19:51:47,254][105620] Updated weights for policy 1, policy_version 618341 (0.0006) [2023-12-26 19:51:47,260][105692] Updated weights for policy 0, policy_version 617526 (0.0006) [2023-12-26 19:51:47,309][105692] Updated weights for policy 0, policy_version 617536 (0.0008) [2023-12-26 19:51:47,965][105692] Updated weights for policy 0, policy_version 617546 (0.0009) [2023-12-26 19:51:48,023][105692] Updated weights for policy 0, policy_version 617556 (0.0009) [2023-12-26 19:51:48,061][105620] Updated weights for policy 1, policy_version 618351 (0.0007) [2023-12-26 19:51:48,083][105692] Updated weights for policy 0, policy_version 617566 (0.0007) [2023-12-26 19:51:48,118][105620] Updated weights for policy 1, policy_version 618361 (0.0006) [2023-12-26 19:51:48,133][105692] Updated weights for policy 0, policy_version 617576 (0.0007) [2023-12-26 19:51:48,180][105620] Updated weights for policy 1, policy_version 618371 (0.0008) [2023-12-26 19:51:48,848][105620] Updated weights for policy 1, policy_version 618381 (0.0009) [2023-12-26 19:51:48,902][105692] Updated weights for policy 0, policy_version 617586 (0.0006) [2023-12-26 19:51:48,906][105620] Updated weights for policy 1, policy_version 618391 (0.0008) [2023-12-26 19:51:48,959][105692] Updated weights for policy 0, policy_version 617596 (0.0009) [2023-12-26 19:51:48,966][105620] Updated weights for policy 1, policy_version 618401 (0.0006) [2023-12-26 19:51:49,016][105692] Updated weights for policy 0, policy_version 617606 (0.0010) [2023-12-26 19:51:49,559][105620] Updated weights for policy 1, policy_version 618411 (0.0005) [2023-12-26 19:51:49,622][105620] Updated weights for policy 1, policy_version 618421 (0.0010) [2023-12-26 19:51:49,683][105620] Updated weights for policy 1, policy_version 618431 (0.0010) [2023-12-26 19:51:49,813][105692] Updated weights for policy 0, policy_version 617616 (0.0009) [2023-12-26 19:51:49,873][105692] Updated weights for policy 0, policy_version 617626 (0.0009) [2023-12-26 19:51:49,931][105692] Updated weights for policy 0, policy_version 617636 (0.0009) [2023-12-26 19:51:50,338][105620] Updated weights for policy 1, policy_version 618441 (0.0006) [2023-12-26 19:51:50,396][105620] Updated weights for policy 1, policy_version 618451 (0.0008) [2023-12-26 19:51:50,454][105620] Updated weights for policy 1, policy_version 618461 (0.0009) [2023-12-26 19:51:50,513][105620] Updated weights for policy 1, policy_version 618471 (0.0009) [2023-12-26 19:51:50,640][105692] Updated weights for policy 0, policy_version 617646 (0.0008) [2023-12-26 19:51:50,701][105692] Updated weights for policy 0, policy_version 617656 (0.0008) [2023-12-26 19:51:50,774][105692] Updated weights for policy 0, policy_version 617666 (0.0007) [2023-12-26 19:51:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 316497920. Throughput: 0: 9980.4, 1: 9798.1. Samples: 316486508. Policy #0 lag: (min: 17.0, avg: 31.6, max: 49.0) [2023-12-26 19:51:51,062][104569] Avg episode reward: [(0, '9266.470'), (1, '8988.713')] [2023-12-26 19:51:51,220][105620] Updated weights for policy 1, policy_version 618481 (0.0008) [2023-12-26 19:51:51,276][105620] Updated weights for policy 1, policy_version 618491 (0.0008) [2023-12-26 19:51:51,355][105620] Updated weights for policy 1, policy_version 618501 (0.0008) [2023-12-26 19:51:51,604][105692] Updated weights for policy 0, policy_version 617676 (0.0008) [2023-12-26 19:51:51,674][105692] Updated weights for policy 0, policy_version 617686 (0.0008) [2023-12-26 19:51:51,739][105692] Updated weights for policy 0, policy_version 617696 (0.0010) [2023-12-26 19:51:52,085][105620] Updated weights for policy 1, policy_version 618511 (0.0008) [2023-12-26 19:51:52,139][105620] Updated weights for policy 1, policy_version 618521 (0.0008) [2023-12-26 19:51:52,186][105620] Updated weights for policy 1, policy_version 618531 (0.0009) [2023-12-26 19:51:52,486][105692] Updated weights for policy 0, policy_version 617706 (0.0010) [2023-12-26 19:51:52,552][105692] Updated weights for policy 0, policy_version 617716 (0.0008) [2023-12-26 19:51:52,606][105692] Updated weights for policy 0, policy_version 617726 (0.0006) [2023-12-26 19:51:52,658][105692] Updated weights for policy 0, policy_version 617736 (0.0005) [2023-12-26 19:51:52,979][105620] Updated weights for policy 1, policy_version 618541 (0.0009) [2023-12-26 19:51:53,037][105620] Updated weights for policy 1, policy_version 618551 (0.0007) [2023-12-26 19:51:53,101][105620] Updated weights for policy 1, policy_version 618561 (0.0007) [2023-12-26 19:51:53,347][105692] Updated weights for policy 0, policy_version 617746 (0.0010) [2023-12-26 19:51:53,398][105692] Updated weights for policy 0, policy_version 617756 (0.0010) [2023-12-26 19:51:53,442][105692] Updated weights for policy 0, policy_version 617766 (0.0010) [2023-12-26 19:51:53,743][105620] Updated weights for policy 1, policy_version 618571 (0.0009) [2023-12-26 19:51:53,797][105620] Updated weights for policy 1, policy_version 618581 (0.0005) [2023-12-26 19:51:53,848][105620] Updated weights for policy 1, policy_version 618591 (0.0006) [2023-12-26 19:51:54,180][105692] Updated weights for policy 0, policy_version 617776 (0.0010) [2023-12-26 19:51:54,238][105692] Updated weights for policy 0, policy_version 617786 (0.0010) [2023-12-26 19:51:54,293][105692] Updated weights for policy 0, policy_version 617796 (0.0007) [2023-12-26 19:51:54,493][105620] Updated weights for policy 1, policy_version 618601 (0.0007) [2023-12-26 19:51:54,561][105620] Updated weights for policy 1, policy_version 618611 (0.0007) [2023-12-26 19:51:54,626][105620] Updated weights for policy 1, policy_version 618621 (0.0005) [2023-12-26 19:51:54,685][105620] Updated weights for policy 1, policy_version 618631 (0.0008) [2023-12-26 19:51:55,010][105692] Updated weights for policy 0, policy_version 617806 (0.0010) [2023-12-26 19:51:55,055][105692] Updated weights for policy 0, policy_version 617816 (0.0010) [2023-12-26 19:51:55,103][105692] Updated weights for policy 0, policy_version 617826 (0.0010) [2023-12-26 19:51:55,386][105620] Updated weights for policy 1, policy_version 618641 (0.0007) [2023-12-26 19:51:55,451][105620] Updated weights for policy 1, policy_version 618651 (0.0005) [2023-12-26 19:51:55,523][105620] Updated weights for policy 1, policy_version 618661 (0.0005) [2023-12-26 19:51:55,868][105692] Updated weights for policy 0, policy_version 617836 (0.0010) [2023-12-26 19:51:55,917][105692] Updated weights for policy 0, policy_version 617846 (0.0010) [2023-12-26 19:51:55,978][105692] Updated weights for policy 0, policy_version 617856 (0.0010) [2023-12-26 19:51:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 316596224. Throughput: 0: 9877.6, 1: 9765.9. Samples: 316602456. Policy #0 lag: (min: 17.0, avg: 31.6, max: 49.0) [2023-12-26 19:51:56,062][104569] Avg episode reward: [(0, '9174.597'), (1, '9080.846')] [2023-12-26 19:51:56,194][105620] Updated weights for policy 1, policy_version 618671 (0.0006) [2023-12-26 19:51:56,249][105620] Updated weights for policy 1, policy_version 618681 (0.0005) [2023-12-26 19:51:56,301][105620] Updated weights for policy 1, policy_version 618691 (0.0005) [2023-12-26 19:51:56,729][105692] Updated weights for policy 0, policy_version 617866 (0.0011) [2023-12-26 19:51:56,797][105692] Updated weights for policy 0, policy_version 617876 (0.0010) [2023-12-26 19:51:56,851][105692] Updated weights for policy 0, policy_version 617886 (0.0010) [2023-12-26 19:51:56,898][105692] Updated weights for policy 0, policy_version 617896 (0.0010) [2023-12-26 19:51:56,900][105620] Updated weights for policy 1, policy_version 618701 (0.0006) [2023-12-26 19:51:56,950][105620] Updated weights for policy 1, policy_version 618711 (0.0005) [2023-12-26 19:51:57,001][105620] Updated weights for policy 1, policy_version 618721 (0.0005) [2023-12-26 19:51:57,597][105620] Updated weights for policy 1, policy_version 618731 (0.0007) [2023-12-26 19:51:57,604][105692] Updated weights for policy 0, policy_version 617906 (0.0010) [2023-12-26 19:51:57,652][105620] Updated weights for policy 1, policy_version 618741 (0.0010) [2023-12-26 19:51:57,655][105692] Updated weights for policy 0, policy_version 617916 (0.0010) [2023-12-26 19:51:57,700][105620] Updated weights for policy 1, policy_version 618751 (0.0010) [2023-12-26 19:51:57,708][105692] Updated weights for policy 0, policy_version 617926 (0.0010) [2023-12-26 19:51:58,455][105692] Updated weights for policy 0, policy_version 617936 (0.0009) [2023-12-26 19:51:58,466][105620] Updated weights for policy 1, policy_version 618761 (0.0010) [2023-12-26 19:51:58,515][105692] Updated weights for policy 0, policy_version 617946 (0.0007) [2023-12-26 19:51:58,531][105620] Updated weights for policy 1, policy_version 618771 (0.0007) [2023-12-26 19:51:58,579][105692] Updated weights for policy 0, policy_version 617956 (0.0009) [2023-12-26 19:51:58,594][105620] Updated weights for policy 1, policy_version 618781 (0.0006) [2023-12-26 19:51:58,654][105620] Updated weights for policy 1, policy_version 618791 (0.0008) [2023-12-26 19:51:59,458][105620] Updated weights for policy 1, policy_version 618801 (0.0008) [2023-12-26 19:51:59,466][105692] Updated weights for policy 0, policy_version 617966 (0.0006) [2023-12-26 19:51:59,508][105620] Updated weights for policy 1, policy_version 618811 (0.0007) [2023-12-26 19:51:59,526][105692] Updated weights for policy 0, policy_version 617976 (0.0008) [2023-12-26 19:51:59,568][105620] Updated weights for policy 1, policy_version 618821 (0.0008) [2023-12-26 19:51:59,583][105692] Updated weights for policy 0, policy_version 617986 (0.0007) [2023-12-26 19:52:00,253][105692] Updated weights for policy 0, policy_version 617996 (0.0008) [2023-12-26 19:52:00,275][105620] Updated weights for policy 1, policy_version 618831 (0.0008) [2023-12-26 19:52:00,312][105692] Updated weights for policy 0, policy_version 618006 (0.0007) [2023-12-26 19:52:00,334][105620] Updated weights for policy 1, policy_version 618841 (0.0007) [2023-12-26 19:52:00,372][105692] Updated weights for policy 0, policy_version 618016 (0.0006) [2023-12-26 19:52:00,401][105620] Updated weights for policy 1, policy_version 618851 (0.0009) [2023-12-26 19:52:00,943][105692] Updated weights for policy 0, policy_version 618026 (0.0006) [2023-12-26 19:52:00,990][105692] Updated weights for policy 0, policy_version 618036 (0.0008) [2023-12-26 19:52:01,049][105692] Updated weights for policy 0, policy_version 618046 (0.0009) [2023-12-26 19:52:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 316686336. Throughput: 0: 9920.5, 1: 9806.2. Samples: 316661044. Policy #0 lag: (min: 17.0, avg: 31.6, max: 49.0) [2023-12-26 19:52:01,063][104569] Avg episode reward: [(0, '9263.848'), (1, '8844.213')] [2023-12-26 19:52:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000618856_158441472.pth... [2023-12-26 19:52:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000617736_158154752.pth [2023-12-26 19:52:01,119][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000618056_158253056.pth... [2023-12-26 19:52:01,120][105692] Updated weights for policy 0, policy_version 618056 (0.0007) [2023-12-26 19:52:01,123][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000616904_157958144.pth [2023-12-26 19:52:01,163][105620] Updated weights for policy 1, policy_version 618861 (0.0008) [2023-12-26 19:52:01,219][105620] Updated weights for policy 1, policy_version 618871 (0.0005) [2023-12-26 19:52:01,277][105620] Updated weights for policy 1, policy_version 618881 (0.0009) [2023-12-26 19:52:01,837][105692] Updated weights for policy 0, policy_version 618066 (0.0006) [2023-12-26 19:52:01,888][105692] Updated weights for policy 0, policy_version 618076 (0.0008) [2023-12-26 19:52:01,945][105692] Updated weights for policy 0, policy_version 618086 (0.0008) [2023-12-26 19:52:02,053][105620] Updated weights for policy 1, policy_version 618891 (0.0009) [2023-12-26 19:52:02,104][105620] Updated weights for policy 1, policy_version 618901 (0.0009) [2023-12-26 19:52:02,157][105620] Updated weights for policy 1, policy_version 618911 (0.0006) [2023-12-26 19:52:02,730][105692] Updated weights for policy 0, policy_version 618096 (0.0008) [2023-12-26 19:52:02,795][105692] Updated weights for policy 0, policy_version 618106 (0.0005) [2023-12-26 19:52:02,808][105620] Updated weights for policy 1, policy_version 618921 (0.0005) [2023-12-26 19:52:02,856][105692] Updated weights for policy 0, policy_version 618116 (0.0005) [2023-12-26 19:52:02,868][105620] Updated weights for policy 1, policy_version 618931 (0.0008) [2023-12-26 19:52:02,932][105620] Updated weights for policy 1, policy_version 618941 (0.0009) [2023-12-26 19:52:02,985][105620] Updated weights for policy 1, policy_version 618951 (0.0010) [2023-12-26 19:52:03,384][105692] Updated weights for policy 0, policy_version 618126 (0.0008) [2023-12-26 19:52:03,431][105692] Updated weights for policy 0, policy_version 618136 (0.0009) [2023-12-26 19:52:03,484][105692] Updated weights for policy 0, policy_version 618146 (0.0008) [2023-12-26 19:52:03,794][105620] Updated weights for policy 1, policy_version 618961 (0.0009) [2023-12-26 19:52:03,845][105620] Updated weights for policy 1, policy_version 618971 (0.0009) [2023-12-26 19:52:03,911][105620] Updated weights for policy 1, policy_version 618981 (0.0009) [2023-12-26 19:52:04,207][105692] Updated weights for policy 0, policy_version 618156 (0.0009) [2023-12-26 19:52:04,273][105692] Updated weights for policy 0, policy_version 618166 (0.0009) [2023-12-26 19:52:04,330][105692] Updated weights for policy 0, policy_version 618176 (0.0008) [2023-12-26 19:52:04,671][105620] Updated weights for policy 1, policy_version 618991 (0.0009) [2023-12-26 19:52:04,718][105620] Updated weights for policy 1, policy_version 619001 (0.0008) [2023-12-26 19:52:04,765][105620] Updated weights for policy 1, policy_version 619011 (0.0009) [2023-12-26 19:52:05,075][105692] Updated weights for policy 0, policy_version 618186 (0.0009) [2023-12-26 19:52:05,131][105692] Updated weights for policy 0, policy_version 618196 (0.0009) [2023-12-26 19:52:05,187][105692] Updated weights for policy 0, policy_version 618206 (0.0005) [2023-12-26 19:52:05,241][105692] Updated weights for policy 0, policy_version 618216 (0.0006) [2023-12-26 19:52:05,557][105620] Updated weights for policy 1, policy_version 619021 (0.0009) [2023-12-26 19:52:05,611][105620] Updated weights for policy 1, policy_version 619032 (0.0010) [2023-12-26 19:52:05,661][105620] Updated weights for policy 1, policy_version 619042 (0.0009) [2023-12-26 19:52:05,787][105692] Updated weights for policy 0, policy_version 618226 (0.0009) [2023-12-26 19:52:05,838][105692] Updated weights for policy 0, policy_version 618236 (0.0009) [2023-12-26 19:52:05,885][105692] Updated weights for policy 0, policy_version 618246 (0.0009) [2023-12-26 19:52:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 316792832. Throughput: 0: 9857.3, 1: 9683.6. Samples: 316777200. Policy #0 lag: (min: 17.0, avg: 31.6, max: 49.0) [2023-12-26 19:52:06,062][104569] Avg episode reward: [(0, '9355.313'), (1, '8844.568')] [2023-12-26 19:52:06,439][105620] Updated weights for policy 1, policy_version 619052 (0.0010) [2023-12-26 19:52:06,504][105620] Updated weights for policy 1, policy_version 619062 (0.0011) [2023-12-26 19:52:06,565][105620] Updated weights for policy 1, policy_version 619072 (0.0011) [2023-12-26 19:52:06,694][105692] Updated weights for policy 0, policy_version 618256 (0.0006) [2023-12-26 19:52:06,751][105692] Updated weights for policy 0, policy_version 618266 (0.0005) [2023-12-26 19:52:06,811][105692] Updated weights for policy 0, policy_version 618276 (0.0006) [2023-12-26 19:52:07,328][105620] Updated weights for policy 1, policy_version 619082 (0.0010) [2023-12-26 19:52:07,395][105620] Updated weights for policy 1, policy_version 619092 (0.0006) [2023-12-26 19:52:07,461][105620] Updated weights for policy 1, policy_version 619102 (0.0006) [2023-12-26 19:52:07,521][105620] Updated weights for policy 1, policy_version 619112 (0.0006) [2023-12-26 19:52:07,527][105692] Updated weights for policy 0, policy_version 618286 (0.0011) [2023-12-26 19:52:07,553][105585] KL-divergence is very high: 121.1546 [2023-12-26 19:52:07,584][105692] Updated weights for policy 0, policy_version 618296 (0.0010) [2023-12-26 19:52:07,646][105692] Updated weights for policy 0, policy_version 618306 (0.0011) [2023-12-26 19:52:08,026][105620] Updated weights for policy 1, policy_version 619122 (0.0009) [2023-12-26 19:52:08,075][105620] Updated weights for policy 1, policy_version 619132 (0.0010) [2023-12-26 19:52:08,131][105620] Updated weights for policy 1, policy_version 619142 (0.0011) [2023-12-26 19:52:08,251][105692] Updated weights for policy 0, policy_version 618316 (0.0006) [2023-12-26 19:52:08,309][105692] Updated weights for policy 0, policy_version 618326 (0.0005) [2023-12-26 19:52:08,375][105692] Updated weights for policy 0, policy_version 618336 (0.0008) [2023-12-26 19:52:08,774][105620] Updated weights for policy 1, policy_version 619152 (0.0007) [2023-12-26 19:52:08,822][105620] Updated weights for policy 1, policy_version 619162 (0.0005) [2023-12-26 19:52:08,872][105620] Updated weights for policy 1, policy_version 619172 (0.0005) [2023-12-26 19:52:09,025][105692] Updated weights for policy 0, policy_version 618346 (0.0008) [2023-12-26 19:52:09,080][105692] Updated weights for policy 0, policy_version 618356 (0.0005) [2023-12-26 19:52:09,144][105692] Updated weights for policy 0, policy_version 618366 (0.0005) [2023-12-26 19:52:09,197][105692] Updated weights for policy 0, policy_version 618376 (0.0005) [2023-12-26 19:52:09,552][105620] Updated weights for policy 1, policy_version 619182 (0.0007) [2023-12-26 19:52:09,613][105620] Updated weights for policy 1, policy_version 619192 (0.0008) [2023-12-26 19:52:09,672][105620] Updated weights for policy 1, policy_version 619202 (0.0007) [2023-12-26 19:52:09,899][105692] Updated weights for policy 0, policy_version 618386 (0.0007) [2023-12-26 19:52:09,971][105692] Updated weights for policy 0, policy_version 618396 (0.0008) [2023-12-26 19:52:10,037][105692] Updated weights for policy 0, policy_version 618406 (0.0008) [2023-12-26 19:52:10,414][105620] Updated weights for policy 1, policy_version 619212 (0.0008) [2023-12-26 19:52:10,481][105620] Updated weights for policy 1, policy_version 619222 (0.0008) [2023-12-26 19:52:10,541][105620] Updated weights for policy 1, policy_version 619232 (0.0008) [2023-12-26 19:52:10,751][105692] Updated weights for policy 0, policy_version 618416 (0.0009) [2023-12-26 19:52:10,813][105692] Updated weights for policy 0, policy_version 618426 (0.0008) [2023-12-26 19:52:10,873][105692] Updated weights for policy 0, policy_version 618436 (0.0009) [2023-12-26 19:52:11,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 316891136. Throughput: 0: 9851.6, 1: 9783.1. Samples: 316898700. Policy #0 lag: (min: 17.0, avg: 31.6, max: 49.0) [2023-12-26 19:52:11,062][104569] Avg episode reward: [(0, '9267.078'), (1, '9264.599')] [2023-12-26 19:52:11,313][105620] Updated weights for policy 1, policy_version 619242 (0.0007) [2023-12-26 19:52:11,387][105620] Updated weights for policy 1, policy_version 619252 (0.0008) [2023-12-26 19:52:11,451][105620] Updated weights for policy 1, policy_version 619262 (0.0006) [2023-12-26 19:52:11,523][105620] Updated weights for policy 1, policy_version 619272 (0.0005) [2023-12-26 19:52:11,659][105692] Updated weights for policy 0, policy_version 618446 (0.0007) [2023-12-26 19:52:11,721][105692] Updated weights for policy 0, policy_version 618456 (0.0010) [2023-12-26 19:52:11,784][105692] Updated weights for policy 0, policy_version 618466 (0.0009) [2023-12-26 19:52:12,183][105620] Updated weights for policy 1, policy_version 619282 (0.0006) [2023-12-26 19:52:12,242][105620] Updated weights for policy 1, policy_version 619292 (0.0009) [2023-12-26 19:52:12,308][105620] Updated weights for policy 1, policy_version 619302 (0.0008) [2023-12-26 19:52:12,599][105692] Updated weights for policy 0, policy_version 618477 (0.0007) [2023-12-26 19:52:12,664][105692] Updated weights for policy 0, policy_version 618487 (0.0010) [2023-12-26 19:52:12,721][105692] Updated weights for policy 0, policy_version 618497 (0.0010) [2023-12-26 19:52:12,941][105620] Updated weights for policy 1, policy_version 619312 (0.0006) [2023-12-26 19:52:13,004][105620] Updated weights for policy 1, policy_version 619322 (0.0010) [2023-12-26 19:52:13,059][105620] Updated weights for policy 1, policy_version 619332 (0.0010) [2023-12-26 19:52:13,465][105692] Updated weights for policy 0, policy_version 618507 (0.0008) [2023-12-26 19:52:13,523][105692] Updated weights for policy 0, policy_version 618517 (0.0009) [2023-12-26 19:52:13,577][105692] Updated weights for policy 0, policy_version 618527 (0.0009) [2023-12-26 19:52:13,772][105620] Updated weights for policy 1, policy_version 619342 (0.0010) [2023-12-26 19:52:13,833][105620] Updated weights for policy 1, policy_version 619352 (0.0010) [2023-12-26 19:52:13,888][105620] Updated weights for policy 1, policy_version 619362 (0.0010) [2023-12-26 19:52:14,254][105692] Updated weights for policy 0, policy_version 618537 (0.0007) [2023-12-26 19:52:14,318][105692] Updated weights for policy 0, policy_version 618547 (0.0010) [2023-12-26 19:52:14,377][105692] Updated weights for policy 0, policy_version 618557 (0.0011) [2023-12-26 19:52:14,436][105692] Updated weights for policy 0, policy_version 618567 (0.0010) [2023-12-26 19:52:14,592][105620] Updated weights for policy 1, policy_version 619372 (0.0008) [2023-12-26 19:52:14,656][105620] Updated weights for policy 1, policy_version 619382 (0.0009) [2023-12-26 19:52:14,718][105620] Updated weights for policy 1, policy_version 619392 (0.0010) [2023-12-26 19:52:15,140][105692] Updated weights for policy 0, policy_version 618577 (0.0006) [2023-12-26 19:52:15,193][105585] KL-divergence is very high: 108.0514 [2023-12-26 19:52:15,198][105692] Updated weights for policy 0, policy_version 618587 (0.0006) [2023-12-26 19:52:15,259][105692] Updated weights for policy 0, policy_version 618597 (0.0006) [2023-12-26 19:52:15,318][105620] Updated weights for policy 1, policy_version 619402 (0.0010) [2023-12-26 19:52:15,393][105620] Updated weights for policy 1, policy_version 619412 (0.0011) [2023-12-26 19:52:15,463][105620] Updated weights for policy 1, policy_version 619422 (0.0010) [2023-12-26 19:52:15,523][105620] Updated weights for policy 1, policy_version 619432 (0.0009) [2023-12-26 19:52:15,786][105692] Updated weights for policy 0, policy_version 618607 (0.0005) [2023-12-26 19:52:15,847][105692] Updated weights for policy 0, policy_version 618617 (0.0005) [2023-12-26 19:52:15,904][105692] Updated weights for policy 0, policy_version 618627 (0.0005) [2023-12-26 19:52:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 316989440. Throughput: 0: 9712.7, 1: 9824.3. Samples: 316955292. Policy #0 lag: (min: 17.0, avg: 31.6, max: 49.0) [2023-12-26 19:52:16,063][104569] Avg episode reward: [(0, '9175.886'), (1, '8807.037')] [2023-12-26 19:52:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000618632_158400512.pth... [2023-12-26 19:52:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000619432_158588928.pth... [2023-12-26 19:52:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000617480_158105600.pth [2023-12-26 19:52:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000618280_158294016.pth [2023-12-26 19:52:16,387][105620] Updated weights for policy 1, policy_version 619442 (0.0008) [2023-12-26 19:52:16,432][105620] Updated weights for policy 1, policy_version 619452 (0.0008) [2023-12-26 19:52:16,495][105620] Updated weights for policy 1, policy_version 619462 (0.0008) [2023-12-26 19:52:16,497][105692] Updated weights for policy 0, policy_version 618637 (0.0007) [2023-12-26 19:52:16,546][105692] Updated weights for policy 0, policy_version 618647 (0.0008) [2023-12-26 19:52:16,597][105692] Updated weights for policy 0, policy_version 618657 (0.0008) [2023-12-26 19:52:17,188][105620] Updated weights for policy 1, policy_version 619472 (0.0006) [2023-12-26 19:52:17,243][105620] Updated weights for policy 1, policy_version 619482 (0.0005) [2023-12-26 19:52:17,296][105620] Updated weights for policy 1, policy_version 619492 (0.0006) [2023-12-26 19:52:17,416][105692] Updated weights for policy 0, policy_version 618667 (0.0009) [2023-12-26 19:52:17,467][105692] Updated weights for policy 0, policy_version 618677 (0.0008) [2023-12-26 19:52:17,525][105692] Updated weights for policy 0, policy_version 618687 (0.0009) [2023-12-26 19:52:17,991][105620] Updated weights for policy 1, policy_version 619502 (0.0008) [2023-12-26 19:52:18,045][105620] Updated weights for policy 1, policy_version 619512 (0.0009) [2023-12-26 19:52:18,110][105620] Updated weights for policy 1, policy_version 619522 (0.0009) [2023-12-26 19:52:18,287][105692] Updated weights for policy 0, policy_version 618697 (0.0009) [2023-12-26 19:52:18,355][105692] Updated weights for policy 0, policy_version 618707 (0.0009) [2023-12-26 19:52:18,413][105692] Updated weights for policy 0, policy_version 618717 (0.0009) [2023-12-26 19:52:18,471][105692] Updated weights for policy 0, policy_version 618727 (0.0009) [2023-12-26 19:52:18,845][105620] Updated weights for policy 1, policy_version 619532 (0.0009) [2023-12-26 19:52:18,900][105620] Updated weights for policy 1, policy_version 619542 (0.0006) [2023-12-26 19:52:18,953][105620] Updated weights for policy 1, policy_version 619552 (0.0005) [2023-12-26 19:52:19,322][105692] Updated weights for policy 0, policy_version 618737 (0.0006) [2023-12-26 19:52:19,391][105692] Updated weights for policy 0, policy_version 618747 (0.0008) [2023-12-26 19:52:19,451][105692] Updated weights for policy 0, policy_version 618757 (0.0005) [2023-12-26 19:52:19,630][105620] Updated weights for policy 1, policy_version 619562 (0.0006) [2023-12-26 19:52:19,682][105620] Updated weights for policy 1, policy_version 619572 (0.0009) [2023-12-26 19:52:19,748][105620] Updated weights for policy 1, policy_version 619582 (0.0009) [2023-12-26 19:52:19,815][105620] Updated weights for policy 1, policy_version 619592 (0.0009) [2023-12-26 19:52:20,097][105692] Updated weights for policy 0, policy_version 618767 (0.0008) [2023-12-26 19:52:20,158][105692] Updated weights for policy 0, policy_version 618777 (0.0009) [2023-12-26 19:52:20,212][105692] Updated weights for policy 0, policy_version 618787 (0.0009) [2023-12-26 19:52:20,558][105620] Updated weights for policy 1, policy_version 619602 (0.0008) [2023-12-26 19:52:20,626][105620] Updated weights for policy 1, policy_version 619612 (0.0008) [2023-12-26 19:52:20,692][105620] Updated weights for policy 1, policy_version 619622 (0.0007) [2023-12-26 19:52:21,019][105692] Updated weights for policy 0, policy_version 618797 (0.0009) [2023-12-26 19:52:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 317079552. Throughput: 0: 9727.6, 1: 9837.5. Samples: 317072936. Policy #0 lag: (min: 17.0, avg: 31.6, max: 49.0) [2023-12-26 19:52:21,062][104569] Avg episode reward: [(0, '9175.668'), (1, '8533.860')] [2023-12-26 19:52:21,090][105585] KL-divergence is very high: 621.6952 [2023-12-26 19:52:21,110][105692] Updated weights for policy 0, policy_version 618807 (0.0009) [2023-12-26 19:52:21,144][105585] KL-divergence is very high: 649.8531 [2023-12-26 19:52:21,174][105692] Updated weights for policy 0, policy_version 618817 (0.0009) [2023-12-26 19:52:21,188][105585] KL-divergence is very high: 297.1685 [2023-12-26 19:52:21,407][105620] Updated weights for policy 1, policy_version 619632 (0.0009) [2023-12-26 19:52:21,469][105620] Updated weights for policy 1, policy_version 619642 (0.0009) [2023-12-26 19:52:21,531][105620] Updated weights for policy 1, policy_version 619652 (0.0008) [2023-12-26 19:52:21,952][105692] Updated weights for policy 0, policy_version 618827 (0.0010) [2023-12-26 19:52:22,010][105692] Updated weights for policy 0, policy_version 618837 (0.0009) [2023-12-26 19:52:22,069][105692] Updated weights for policy 0, policy_version 618847 (0.0009) [2023-12-26 19:52:22,311][105620] Updated weights for policy 1, policy_version 619662 (0.0009) [2023-12-26 19:52:22,366][105620] Updated weights for policy 1, policy_version 619672 (0.0009) [2023-12-26 19:52:22,427][105620] Updated weights for policy 1, policy_version 619682 (0.0008) [2023-12-26 19:52:22,785][105692] Updated weights for policy 0, policy_version 618857 (0.0009) [2023-12-26 19:52:22,849][105692] Updated weights for policy 0, policy_version 618867 (0.0007) [2023-12-26 19:52:22,904][105692] Updated weights for policy 0, policy_version 618877 (0.0010) [2023-12-26 19:52:22,955][105692] Updated weights for policy 0, policy_version 618887 (0.0010) [2023-12-26 19:52:23,290][105620] Updated weights for policy 1, policy_version 619692 (0.0008) [2023-12-26 19:52:23,343][105620] Updated weights for policy 1, policy_version 619702 (0.0007) [2023-12-26 19:52:23,394][105620] Updated weights for policy 1, policy_version 619712 (0.0008) [2023-12-26 19:52:23,615][105692] Updated weights for policy 0, policy_version 618897 (0.0010) [2023-12-26 19:52:23,676][105692] Updated weights for policy 0, policy_version 618907 (0.0010) [2023-12-26 19:52:23,728][105692] Updated weights for policy 0, policy_version 618917 (0.0010) [2023-12-26 19:52:23,968][105620] Updated weights for policy 1, policy_version 619722 (0.0006) [2023-12-26 19:52:24,030][105620] Updated weights for policy 1, policy_version 619732 (0.0005) [2023-12-26 19:52:24,080][105620] Updated weights for policy 1, policy_version 619742 (0.0005) [2023-12-26 19:52:24,143][105620] Updated weights for policy 1, policy_version 619752 (0.0007) [2023-12-26 19:52:24,476][105692] Updated weights for policy 0, policy_version 618927 (0.0010) [2023-12-26 19:52:24,538][105692] Updated weights for policy 0, policy_version 618937 (0.0008) [2023-12-26 19:52:24,587][105692] Updated weights for policy 0, policy_version 618947 (0.0008) [2023-12-26 19:52:24,760][105620] Updated weights for policy 1, policy_version 619762 (0.0009) [2023-12-26 19:52:24,817][105620] Updated weights for policy 1, policy_version 619772 (0.0010) [2023-12-26 19:52:24,882][105620] Updated weights for policy 1, policy_version 619782 (0.0006) [2023-12-26 19:52:25,146][105692] Updated weights for policy 0, policy_version 618957 (0.0006) [2023-12-26 19:52:25,201][105692] Updated weights for policy 0, policy_version 618967 (0.0005) [2023-12-26 19:52:25,264][105692] Updated weights for policy 0, policy_version 618977 (0.0005) [2023-12-26 19:52:25,531][105620] Updated weights for policy 1, policy_version 619792 (0.0009) [2023-12-26 19:52:25,574][105620] Updated weights for policy 1, policy_version 619802 (0.0010) [2023-12-26 19:52:25,621][105620] Updated weights for policy 1, policy_version 619812 (0.0010) [2023-12-26 19:52:25,918][105692] Updated weights for policy 0, policy_version 618987 (0.0007) [2023-12-26 19:52:25,985][105692] Updated weights for policy 0, policy_version 618997 (0.0010) [2023-12-26 19:52:26,053][105692] Updated weights for policy 0, policy_version 619007 (0.0010) [2023-12-26 19:52:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 317177856. Throughput: 0: 9747.5, 1: 9842.4. Samples: 317190780. Policy #0 lag: (min: 17.0, avg: 31.6, max: 49.0) [2023-12-26 19:52:26,063][104569] Avg episode reward: [(0, '9175.661'), (1, '8990.812')] [2023-12-26 19:52:26,260][105620] Updated weights for policy 1, policy_version 619822 (0.0007) [2023-12-26 19:52:26,305][105620] Updated weights for policy 1, policy_version 619832 (0.0005) [2023-12-26 19:52:26,353][105620] Updated weights for policy 1, policy_version 619842 (0.0005) [2023-12-26 19:52:26,730][105692] Updated weights for policy 0, policy_version 619017 (0.0010) [2023-12-26 19:52:26,781][105692] Updated weights for policy 0, policy_version 619027 (0.0009) [2023-12-26 19:52:26,833][105692] Updated weights for policy 0, policy_version 619037 (0.0010) [2023-12-26 19:52:26,886][105692] Updated weights for policy 0, policy_version 619047 (0.0010) [2023-12-26 19:52:26,924][105620] Updated weights for policy 1, policy_version 619852 (0.0005) [2023-12-26 19:52:26,982][105620] Updated weights for policy 1, policy_version 619862 (0.0005) [2023-12-26 19:52:27,039][105620] Updated weights for policy 1, policy_version 619872 (0.0005) [2023-12-26 19:52:27,611][105692] Updated weights for policy 0, policy_version 619057 (0.0010) [2023-12-26 19:52:27,639][105620] Updated weights for policy 1, policy_version 619882 (0.0007) [2023-12-26 19:52:27,663][105692] Updated weights for policy 0, policy_version 619067 (0.0006) [2023-12-26 19:52:27,696][105620] Updated weights for policy 1, policy_version 619892 (0.0008) [2023-12-26 19:52:27,714][105692] Updated weights for policy 0, policy_version 619077 (0.0006) [2023-12-26 19:52:27,747][105620] Updated weights for policy 1, policy_version 619902 (0.0010) [2023-12-26 19:52:27,807][105620] Updated weights for policy 1, policy_version 619912 (0.0009) [2023-12-26 19:52:28,249][105692] Updated weights for policy 0, policy_version 619087 (0.0005) [2023-12-26 19:52:28,308][105692] Updated weights for policy 0, policy_version 619097 (0.0005) [2023-12-26 19:52:28,375][105692] Updated weights for policy 0, policy_version 619107 (0.0007) [2023-12-26 19:52:28,523][105620] Updated weights for policy 1, policy_version 619922 (0.0006) [2023-12-26 19:52:28,590][105620] Updated weights for policy 1, policy_version 619932 (0.0005) [2023-12-26 19:52:28,648][105586] KL-divergence is very high: 118.0391 [2023-12-26 19:52:28,654][105620] Updated weights for policy 1, policy_version 619942 (0.0005) [2023-12-26 19:52:28,978][105692] Updated weights for policy 0, policy_version 619117 (0.0008) [2023-12-26 19:52:29,035][105692] Updated weights for policy 0, policy_version 619127 (0.0010) [2023-12-26 19:52:29,089][105692] Updated weights for policy 0, policy_version 619137 (0.0010) [2023-12-26 19:52:29,187][105620] Updated weights for policy 1, policy_version 619952 (0.0009) [2023-12-26 19:52:29,252][105620] Updated weights for policy 1, policy_version 619962 (0.0007) [2023-12-26 19:52:29,316][105620] Updated weights for policy 1, policy_version 619972 (0.0009) [2023-12-26 19:52:29,867][105692] Updated weights for policy 0, policy_version 619147 (0.0007) [2023-12-26 19:52:29,935][105692] Updated weights for policy 0, policy_version 619157 (0.0007) [2023-12-26 19:52:30,008][105692] Updated weights for policy 0, policy_version 619167 (0.0009) [2023-12-26 19:52:30,038][105620] Updated weights for policy 1, policy_version 619982 (0.0006) [2023-12-26 19:52:30,097][105620] Updated weights for policy 1, policy_version 619992 (0.0005) [2023-12-26 19:52:30,165][105620] Updated weights for policy 1, policy_version 620002 (0.0006) [2023-12-26 19:52:30,774][105692] Updated weights for policy 0, policy_version 619177 (0.0009) [2023-12-26 19:52:30,811][105620] Updated weights for policy 1, policy_version 620012 (0.0007) [2023-12-26 19:52:30,830][105692] Updated weights for policy 0, policy_version 619187 (0.0008) [2023-12-26 19:52:30,863][105620] Updated weights for policy 1, policy_version 620022 (0.0005) [2023-12-26 19:52:30,879][105692] Updated weights for policy 0, policy_version 619197 (0.0008) [2023-12-26 19:52:30,923][105620] Updated weights for policy 1, policy_version 620032 (0.0007) [2023-12-26 19:52:30,938][105692] Updated weights for policy 0, policy_version 619207 (0.0007) [2023-12-26 19:52:31,062][104569] Fps is (10 sec: 21299.0, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 317292544. Throughput: 0: 9854.2, 1: 9955.8. Samples: 317256908. Policy #0 lag: (min: 17.0, avg: 31.6, max: 49.0) [2023-12-26 19:52:31,063][104569] Avg episode reward: [(0, '9354.240'), (1, '9176.831')] [2023-12-26 19:52:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000619208_158547968.pth... [2023-12-26 19:52:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000620040_158744576.pth... [2023-12-26 19:52:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000618856_158441472.pth [2023-12-26 19:52:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000618056_158253056.pth [2023-12-26 19:52:31,522][105620] Updated weights for policy 1, policy_version 620042 (0.0007) [2023-12-26 19:52:31,582][105620] Updated weights for policy 1, policy_version 620052 (0.0006) [2023-12-26 19:52:31,644][105620] Updated weights for policy 1, policy_version 620062 (0.0008) [2023-12-26 19:52:31,704][105692] Updated weights for policy 0, policy_version 619217 (0.0008) [2023-12-26 19:52:31,711][105620] Updated weights for policy 1, policy_version 620072 (0.0008) [2023-12-26 19:52:31,759][105692] Updated weights for policy 0, policy_version 619227 (0.0008) [2023-12-26 19:52:31,820][105692] Updated weights for policy 0, policy_version 619237 (0.0008) [2023-12-26 19:52:32,411][105620] Updated weights for policy 1, policy_version 620082 (0.0010) [2023-12-26 19:52:32,465][105620] Updated weights for policy 1, policy_version 620092 (0.0010) [2023-12-26 19:52:32,468][105692] Updated weights for policy 0, policy_version 619247 (0.0008) [2023-12-26 19:52:32,514][105620] Updated weights for policy 1, policy_version 620102 (0.0010) [2023-12-26 19:52:32,524][105692] Updated weights for policy 0, policy_version 619257 (0.0006) [2023-12-26 19:52:32,587][105692] Updated weights for policy 0, policy_version 619267 (0.0007) [2023-12-26 19:52:33,263][105620] Updated weights for policy 1, policy_version 620112 (0.0006) [2023-12-26 19:52:33,310][105692] Updated weights for policy 0, policy_version 619277 (0.0008) [2023-12-26 19:52:33,318][105620] Updated weights for policy 1, policy_version 620122 (0.0006) [2023-12-26 19:52:33,368][105692] Updated weights for policy 0, policy_version 619287 (0.0010) [2023-12-26 19:52:33,371][105620] Updated weights for policy 1, policy_version 620132 (0.0006) [2023-12-26 19:52:33,419][105692] Updated weights for policy 0, policy_version 619297 (0.0010) [2023-12-26 19:52:34,030][105692] Updated weights for policy 0, policy_version 619307 (0.0010) [2023-12-26 19:52:34,081][105692] Updated weights for policy 0, policy_version 619317 (0.0010) [2023-12-26 19:52:34,096][105620] Updated weights for policy 1, policy_version 620142 (0.0009) [2023-12-26 19:52:34,137][105692] Updated weights for policy 0, policy_version 619327 (0.0009) [2023-12-26 19:52:34,156][105620] Updated weights for policy 1, policy_version 620152 (0.0010) [2023-12-26 19:52:34,208][105620] Updated weights for policy 1, policy_version 620162 (0.0010) [2023-12-26 19:52:34,886][105692] Updated weights for policy 0, policy_version 619337 (0.0009) [2023-12-26 19:52:34,944][105692] Updated weights for policy 0, policy_version 619347 (0.0010) [2023-12-26 19:52:34,985][105620] Updated weights for policy 1, policy_version 620172 (0.0011) [2023-12-26 19:52:35,003][105692] Updated weights for policy 0, policy_version 619357 (0.0010) [2023-12-26 19:52:35,037][105620] Updated weights for policy 1, policy_version 620182 (0.0010) [2023-12-26 19:52:35,061][105692] Updated weights for policy 0, policy_version 619367 (0.0010) [2023-12-26 19:52:35,085][105620] Updated weights for policy 1, policy_version 620192 (0.0010) [2023-12-26 19:52:35,683][105620] Updated weights for policy 1, policy_version 620202 (0.0010) [2023-12-26 19:52:35,728][105620] Updated weights for policy 1, policy_version 620212 (0.0010) [2023-12-26 19:52:35,779][105620] Updated weights for policy 1, policy_version 620222 (0.0010) [2023-12-26 19:52:35,793][105692] Updated weights for policy 0, policy_version 619377 (0.0010) [2023-12-26 19:52:35,824][105620] Updated weights for policy 1, policy_version 620232 (0.0010) [2023-12-26 19:52:35,854][105692] Updated weights for policy 0, policy_version 619387 (0.0010) [2023-12-26 19:52:35,918][105692] Updated weights for policy 0, policy_version 619397 (0.0010) [2023-12-26 19:52:36,062][104569] Fps is (10 sec: 21299.0, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 317390848. Throughput: 0: 9811.3, 1: 9930.3. Samples: 317374884. Policy #0 lag: (min: 17.0, avg: 31.6, max: 49.0) [2023-12-26 19:52:36,063][104569] Avg episode reward: [(0, '9169.834'), (1, '9268.773')] [2023-12-26 19:52:36,603][105620] Updated weights for policy 1, policy_version 620242 (0.0011) [2023-12-26 19:52:36,659][105620] Updated weights for policy 1, policy_version 620252 (0.0010) [2023-12-26 19:52:36,661][105692] Updated weights for policy 0, policy_version 619407 (0.0011) [2023-12-26 19:52:36,715][105620] Updated weights for policy 1, policy_version 620262 (0.0010) [2023-12-26 19:52:36,720][105692] Updated weights for policy 0, policy_version 619417 (0.0011) [2023-12-26 19:52:36,783][105692] Updated weights for policy 0, policy_version 619427 (0.0011) [2023-12-26 19:52:37,462][105620] Updated weights for policy 1, policy_version 620272 (0.0011) [2023-12-26 19:52:37,511][105620] Updated weights for policy 1, policy_version 620282 (0.0010) [2023-12-26 19:52:37,526][105692] Updated weights for policy 0, policy_version 619437 (0.0011) [2023-12-26 19:52:37,560][105620] Updated weights for policy 1, policy_version 620292 (0.0010) [2023-12-26 19:52:37,582][105692] Updated weights for policy 0, policy_version 619447 (0.0010) [2023-12-26 19:52:37,650][105692] Updated weights for policy 0, policy_version 619457 (0.0010) [2023-12-26 19:52:38,350][105620] Updated weights for policy 1, policy_version 620302 (0.0011) [2023-12-26 19:52:38,384][105692] Updated weights for policy 0, policy_version 619467 (0.0009) [2023-12-26 19:52:38,416][105620] Updated weights for policy 1, policy_version 620312 (0.0011) [2023-12-26 19:52:38,447][105692] Updated weights for policy 0, policy_version 619477 (0.0006) [2023-12-26 19:52:38,476][105620] Updated weights for policy 1, policy_version 620322 (0.0011) [2023-12-26 19:52:38,508][105692] Updated weights for policy 0, policy_version 619487 (0.0009) [2023-12-26 19:52:39,143][105620] Updated weights for policy 1, policy_version 620332 (0.0008) [2023-12-26 19:52:39,210][105620] Updated weights for policy 1, policy_version 620342 (0.0006) [2023-12-26 19:52:39,270][105692] Updated weights for policy 0, policy_version 619497 (0.0010) [2023-12-26 19:52:39,271][105620] Updated weights for policy 1, policy_version 620352 (0.0010) [2023-12-26 19:52:39,333][105692] Updated weights for policy 0, policy_version 619507 (0.0007) [2023-12-26 19:52:39,399][105692] Updated weights for policy 0, policy_version 619517 (0.0009) [2023-12-26 19:52:39,460][105692] Updated weights for policy 0, policy_version 619527 (0.0008) [2023-12-26 19:52:40,021][105620] Updated weights for policy 1, policy_version 620362 (0.0009) [2023-12-26 19:52:40,085][105620] Updated weights for policy 1, policy_version 620372 (0.0010) [2023-12-26 19:52:40,137][105620] Updated weights for policy 1, policy_version 620382 (0.0010) [2023-12-26 19:52:40,201][105620] Updated weights for policy 1, policy_version 620392 (0.0011) [2023-12-26 19:52:40,233][105692] Updated weights for policy 0, policy_version 619537 (0.0007) [2023-12-26 19:52:40,290][105692] Updated weights for policy 0, policy_version 619547 (0.0008) [2023-12-26 19:52:40,336][105692] Updated weights for policy 0, policy_version 619557 (0.0008) [2023-12-26 19:52:40,966][105620] Updated weights for policy 1, policy_version 620402 (0.0010) [2023-12-26 19:52:41,023][105620] Updated weights for policy 1, policy_version 620412 (0.0011) [2023-12-26 19:52:41,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 317472768. Throughput: 0: 9787.7, 1: 9893.4. Samples: 317488108. Policy #0 lag: (min: 17.0, avg: 31.6, max: 49.0) [2023-12-26 19:52:41,063][104569] Avg episode reward: [(0, '9354.325'), (1, '9082.027')] [2023-12-26 19:52:41,088][105620] Updated weights for policy 1, policy_version 620422 (0.0008) [2023-12-26 19:52:41,129][105692] Updated weights for policy 0, policy_version 619567 (0.0009) [2023-12-26 19:52:41,198][105692] Updated weights for policy 0, policy_version 619577 (0.0009) [2023-12-26 19:52:41,263][105692] Updated weights for policy 0, policy_version 619587 (0.0008) [2023-12-26 19:52:41,907][105620] Updated weights for policy 1, policy_version 620432 (0.0010) [2023-12-26 19:52:41,968][105620] Updated weights for policy 1, policy_version 620442 (0.0011) [2023-12-26 19:52:42,017][105620] Updated weights for policy 1, policy_version 620452 (0.0011) [2023-12-26 19:52:42,076][105692] Updated weights for policy 0, policy_version 619597 (0.0009) [2023-12-26 19:52:42,129][105692] Updated weights for policy 0, policy_version 619607 (0.0010) [2023-12-26 19:52:42,181][105692] Updated weights for policy 0, policy_version 619617 (0.0010) [2023-12-26 19:52:42,727][105620] Updated weights for policy 1, policy_version 620462 (0.0010) [2023-12-26 19:52:42,791][105620] Updated weights for policy 1, policy_version 620472 (0.0010) [2023-12-26 19:52:42,842][105620] Updated weights for policy 1, policy_version 620482 (0.0009) [2023-12-26 19:52:42,994][105692] Updated weights for policy 0, policy_version 619628 (0.0010) [2023-12-26 19:52:43,041][105692] Updated weights for policy 0, policy_version 619638 (0.0008) [2023-12-26 19:52:43,088][105692] Updated weights for policy 0, policy_version 619648 (0.0009) [2023-12-26 19:52:43,590][105620] Updated weights for policy 1, policy_version 620492 (0.0008) [2023-12-26 19:52:43,649][105620] Updated weights for policy 1, policy_version 620502 (0.0008) [2023-12-26 19:52:43,716][105620] Updated weights for policy 1, policy_version 620512 (0.0007) [2023-12-26 19:52:43,846][105692] Updated weights for policy 0, policy_version 619658 (0.0008) [2023-12-26 19:52:43,909][105692] Updated weights for policy 0, policy_version 619668 (0.0009) [2023-12-26 19:52:43,968][105692] Updated weights for policy 0, policy_version 619678 (0.0009) [2023-12-26 19:52:44,026][105692] Updated weights for policy 0, policy_version 619688 (0.0009) [2023-12-26 19:52:44,345][105620] Updated weights for policy 1, policy_version 620522 (0.0006) [2023-12-26 19:52:44,414][105620] Updated weights for policy 1, policy_version 620532 (0.0008) [2023-12-26 19:52:44,484][105620] Updated weights for policy 1, policy_version 620542 (0.0008) [2023-12-26 19:52:44,539][105620] Updated weights for policy 1, policy_version 620552 (0.0008) [2023-12-26 19:52:44,716][105692] Updated weights for policy 0, policy_version 619698 (0.0005) [2023-12-26 19:52:44,780][105692] Updated weights for policy 0, policy_version 619708 (0.0006) [2023-12-26 19:52:44,837][105692] Updated weights for policy 0, policy_version 619718 (0.0008) [2023-12-26 19:52:45,318][105620] Updated weights for policy 1, policy_version 620562 (0.0010) [2023-12-26 19:52:45,370][105620] Updated weights for policy 1, policy_version 620572 (0.0009) [2023-12-26 19:52:45,427][105620] Updated weights for policy 1, policy_version 620582 (0.0009) [2023-12-26 19:52:45,430][105692] Updated weights for policy 0, policy_version 619728 (0.0006) [2023-12-26 19:52:45,480][105692] Updated weights for policy 0, policy_version 619738 (0.0009) [2023-12-26 19:52:45,530][105692] Updated weights for policy 0, policy_version 619748 (0.0009) [2023-12-26 19:52:46,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 317571072. Throughput: 0: 9747.1, 1: 9855.4. Samples: 317543160. Policy #0 lag: (min: 17.0, avg: 31.6, max: 49.0) [2023-12-26 19:52:46,063][104569] Avg episode reward: [(0, '9354.365'), (1, '9081.477')] [2023-12-26 19:52:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000619752_158687232.pth... [2023-12-26 19:52:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000618632_158400512.pth [2023-12-26 19:52:46,079][105620] Updated weights for policy 1, policy_version 620592 (0.0006) [2023-12-26 19:52:46,154][105620] Updated weights for policy 1, policy_version 620602 (0.0005) [2023-12-26 19:52:46,207][105620] Updated weights for policy 1, policy_version 620612 (0.0005) [2023-12-26 19:52:46,228][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000620616_158892032.pth... [2023-12-26 19:52:46,231][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000619432_158588928.pth [2023-12-26 19:52:46,418][105692] Updated weights for policy 0, policy_version 619758 (0.0009) [2023-12-26 19:52:46,467][105692] Updated weights for policy 0, policy_version 619768 (0.0008) [2023-12-26 19:52:46,512][105692] Updated weights for policy 0, policy_version 619778 (0.0008) [2023-12-26 19:52:46,787][105620] Updated weights for policy 1, policy_version 620622 (0.0007) [2023-12-26 19:52:46,846][105620] Updated weights for policy 1, policy_version 620632 (0.0008) [2023-12-26 19:52:46,901][105620] Updated weights for policy 1, policy_version 620642 (0.0008) [2023-12-26 19:52:47,294][105692] Updated weights for policy 0, policy_version 619789 (0.0010) [2023-12-26 19:52:47,341][105692] Updated weights for policy 0, policy_version 619799 (0.0010) [2023-12-26 19:52:47,399][105692] Updated weights for policy 0, policy_version 619809 (0.0010) [2023-12-26 19:52:47,656][105620] Updated weights for policy 1, policy_version 620652 (0.0008) [2023-12-26 19:52:47,702][105620] Updated weights for policy 1, policy_version 620662 (0.0008) [2023-12-26 19:52:47,753][105620] Updated weights for policy 1, policy_version 620672 (0.0009) [2023-12-26 19:52:48,126][105692] Updated weights for policy 0, policy_version 619819 (0.0010) [2023-12-26 19:52:48,174][105692] Updated weights for policy 0, policy_version 619829 (0.0009) [2023-12-26 19:52:48,227][105692] Updated weights for policy 0, policy_version 619839 (0.0009) [2023-12-26 19:52:48,514][105620] Updated weights for policy 1, policy_version 620683 (0.0009) [2023-12-26 19:52:48,573][105620] Updated weights for policy 1, policy_version 620693 (0.0005) [2023-12-26 19:52:48,640][105620] Updated weights for policy 1, policy_version 620703 (0.0006) [2023-12-26 19:52:49,068][105692] Updated weights for policy 0, policy_version 619849 (0.0009) [2023-12-26 19:52:49,129][105692] Updated weights for policy 0, policy_version 619859 (0.0009) [2023-12-26 19:52:49,191][105692] Updated weights for policy 0, policy_version 619869 (0.0009) [2023-12-26 19:52:49,236][105620] Updated weights for policy 1, policy_version 620713 (0.0005) [2023-12-26 19:52:49,253][105692] Updated weights for policy 0, policy_version 619879 (0.0009) [2023-12-26 19:52:49,297][105620] Updated weights for policy 1, policy_version 620723 (0.0008) [2023-12-26 19:52:49,368][105620] Updated weights for policy 1, policy_version 620733 (0.0008) [2023-12-26 19:52:49,427][105620] Updated weights for policy 1, policy_version 620743 (0.0008) [2023-12-26 19:52:49,980][105692] Updated weights for policy 0, policy_version 619889 (0.0010) [2023-12-26 19:52:50,043][105692] Updated weights for policy 0, policy_version 619900 (0.0009) [2023-12-26 19:52:50,103][105620] Updated weights for policy 1, policy_version 620753 (0.0008) [2023-12-26 19:52:50,104][105692] Updated weights for policy 0, policy_version 619910 (0.0008) [2023-12-26 19:52:50,151][105620] Updated weights for policy 1, policy_version 620763 (0.0010) [2023-12-26 19:52:50,207][105620] Updated weights for policy 1, policy_version 620773 (0.0010) [2023-12-26 19:52:50,859][105620] Updated weights for policy 1, policy_version 620783 (0.0011) [2023-12-26 19:52:50,890][105692] Updated weights for policy 0, policy_version 619920 (0.0008) [2023-12-26 19:52:50,917][105620] Updated weights for policy 1, policy_version 620793 (0.0011) [2023-12-26 19:52:50,958][105692] Updated weights for policy 0, policy_version 619930 (0.0008) [2023-12-26 19:52:50,976][105620] Updated weights for policy 1, policy_version 620803 (0.0010) [2023-12-26 19:52:51,009][105692] Updated weights for policy 0, policy_version 619940 (0.0007) [2023-12-26 19:52:51,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 317677568. Throughput: 0: 9678.1, 1: 9950.7. Samples: 317660496. Policy #0 lag: (min: 17.0, avg: 31.6, max: 49.0) [2023-12-26 19:52:51,062][104569] Avg episode reward: [(0, '9354.558'), (1, '9265.575')] [2023-12-26 19:52:51,690][105620] Updated weights for policy 1, policy_version 620813 (0.0009) [2023-12-26 19:52:51,754][105620] Updated weights for policy 1, policy_version 620823 (0.0008) [2023-12-26 19:52:51,793][105692] Updated weights for policy 0, policy_version 619950 (0.0010) [2023-12-26 19:52:51,812][105620] Updated weights for policy 1, policy_version 620833 (0.0006) [2023-12-26 19:52:51,850][105692] Updated weights for policy 0, policy_version 619960 (0.0011) [2023-12-26 19:52:51,913][105692] Updated weights for policy 0, policy_version 619970 (0.0011) [2023-12-26 19:52:52,470][105620] Updated weights for policy 1, policy_version 620843 (0.0007) [2023-12-26 19:52:52,535][105620] Updated weights for policy 1, policy_version 620853 (0.0009) [2023-12-26 19:52:52,600][105620] Updated weights for policy 1, policy_version 620863 (0.0007) [2023-12-26 19:52:52,679][105692] Updated weights for policy 0, policy_version 619980 (0.0010) [2023-12-26 19:52:52,738][105692] Updated weights for policy 0, policy_version 619990 (0.0008) [2023-12-26 19:52:52,803][105692] Updated weights for policy 0, policy_version 620000 (0.0008) [2023-12-26 19:52:53,296][105620] Updated weights for policy 1, policy_version 620873 (0.0007) [2023-12-26 19:52:53,359][105620] Updated weights for policy 1, policy_version 620883 (0.0009) [2023-12-26 19:52:53,416][105620] Updated weights for policy 1, policy_version 620893 (0.0008) [2023-12-26 19:52:53,471][105620] Updated weights for policy 1, policy_version 620903 (0.0005) [2023-12-26 19:52:53,571][105692] Updated weights for policy 0, policy_version 620010 (0.0008) [2023-12-26 19:52:53,628][105692] Updated weights for policy 0, policy_version 620020 (0.0010) [2023-12-26 19:52:53,673][105692] Updated weights for policy 0, policy_version 620030 (0.0008) [2023-12-26 19:52:53,730][105692] Updated weights for policy 0, policy_version 620040 (0.0008) [2023-12-26 19:52:54,143][105620] Updated weights for policy 1, policy_version 620913 (0.0008) [2023-12-26 19:52:54,190][105620] Updated weights for policy 1, policy_version 620923 (0.0008) [2023-12-26 19:52:54,249][105620] Updated weights for policy 1, policy_version 620933 (0.0006) [2023-12-26 19:52:54,432][105692] Updated weights for policy 0, policy_version 620050 (0.0010) [2023-12-26 19:52:54,485][105692] Updated weights for policy 0, policy_version 620060 (0.0010) [2023-12-26 19:52:54,547][105692] Updated weights for policy 0, policy_version 620070 (0.0009) [2023-12-26 19:52:54,911][105620] Updated weights for policy 1, policy_version 620943 (0.0007) [2023-12-26 19:52:54,972][105620] Updated weights for policy 1, policy_version 620953 (0.0008) [2023-12-26 19:52:55,024][105620] Updated weights for policy 1, policy_version 620963 (0.0009) [2023-12-26 19:52:55,186][105692] Updated weights for policy 0, policy_version 620080 (0.0005) [2023-12-26 19:52:55,252][105692] Updated weights for policy 0, policy_version 620090 (0.0005) [2023-12-26 19:52:55,306][105692] Updated weights for policy 0, policy_version 620100 (0.0005) [2023-12-26 19:52:55,773][105620] Updated weights for policy 1, policy_version 620973 (0.0007) [2023-12-26 19:52:55,828][105692] Updated weights for policy 0, policy_version 620110 (0.0005) [2023-12-26 19:52:55,833][105620] Updated weights for policy 1, policy_version 620983 (0.0007) [2023-12-26 19:52:55,876][105692] Updated weights for policy 0, policy_version 620120 (0.0005) [2023-12-26 19:52:55,882][105620] Updated weights for policy 1, policy_version 620993 (0.0010) [2023-12-26 19:52:55,935][105692] Updated weights for policy 0, policy_version 620130 (0.0005) [2023-12-26 19:52:56,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.7, 300 sec: 19633.0). Total num frames: 317775872. Throughput: 0: 9610.7, 1: 9952.6. Samples: 317779056. Policy #0 lag: (min: 25.0, avg: 40.1, max: 57.0) [2023-12-26 19:52:56,063][104569] Avg episode reward: [(0, '9354.476'), (1, '9265.717')] [2023-12-26 19:52:56,558][105620] Updated weights for policy 1, policy_version 621003 (0.0010) [2023-12-26 19:52:56,595][105692] Updated weights for policy 0, policy_version 620140 (0.0007) [2023-12-26 19:52:56,613][105620] Updated weights for policy 1, policy_version 621013 (0.0005) [2023-12-26 19:52:56,647][105692] Updated weights for policy 0, policy_version 620150 (0.0009) [2023-12-26 19:52:56,658][105620] Updated weights for policy 1, policy_version 621023 (0.0005) [2023-12-26 19:52:56,699][105692] Updated weights for policy 0, policy_version 620160 (0.0008) [2023-12-26 19:52:57,195][105620] Updated weights for policy 1, policy_version 621033 (0.0005) [2023-12-26 19:52:57,242][105620] Updated weights for policy 1, policy_version 621043 (0.0007) [2023-12-26 19:52:57,279][105692] Updated weights for policy 0, policy_version 620170 (0.0008) [2023-12-26 19:52:57,290][105620] Updated weights for policy 1, policy_version 621053 (0.0010) [2023-12-26 19:52:57,337][105692] Updated weights for policy 0, policy_version 620180 (0.0005) [2023-12-26 19:52:57,345][105620] Updated weights for policy 1, policy_version 621063 (0.0010) [2023-12-26 19:52:57,390][105692] Updated weights for policy 0, policy_version 620190 (0.0005) [2023-12-26 19:52:57,446][105692] Updated weights for policy 0, policy_version 620200 (0.0009) [2023-12-26 19:52:57,944][105620] Updated weights for policy 1, policy_version 621073 (0.0006) [2023-12-26 19:52:57,960][105692] Updated weights for policy 0, policy_version 620210 (0.0006) [2023-12-26 19:52:58,000][105620] Updated weights for policy 1, policy_version 621083 (0.0005) [2023-12-26 19:52:58,020][105692] Updated weights for policy 0, policy_version 620220 (0.0005) [2023-12-26 19:52:58,063][105620] Updated weights for policy 1, policy_version 621093 (0.0005) [2023-12-26 19:52:58,075][105692] Updated weights for policy 0, policy_version 620230 (0.0005) [2023-12-26 19:52:58,758][105692] Updated weights for policy 0, policy_version 620240 (0.0008) [2023-12-26 19:52:58,773][105620] Updated weights for policy 1, policy_version 621103 (0.0006) [2023-12-26 19:52:58,830][105692] Updated weights for policy 0, policy_version 620250 (0.0007) [2023-12-26 19:52:58,837][105620] Updated weights for policy 1, policy_version 621113 (0.0007) [2023-12-26 19:52:58,905][105692] Updated weights for policy 0, policy_version 620260 (0.0007) [2023-12-26 19:52:58,907][105620] Updated weights for policy 1, policy_version 621123 (0.0009) [2023-12-26 19:52:59,618][105620] Updated weights for policy 1, policy_version 621133 (0.0009) [2023-12-26 19:52:59,669][105620] Updated weights for policy 1, policy_version 621143 (0.0008) [2023-12-26 19:52:59,702][105692] Updated weights for policy 0, policy_version 620270 (0.0009) [2023-12-26 19:52:59,730][105620] Updated weights for policy 1, policy_version 621153 (0.0005) [2023-12-26 19:52:59,759][105692] Updated weights for policy 0, policy_version 620281 (0.0008) [2023-12-26 19:52:59,808][105692] Updated weights for policy 0, policy_version 620291 (0.0009) [2023-12-26 19:53:00,352][105620] Updated weights for policy 1, policy_version 621163 (0.0007) [2023-12-26 19:53:00,412][105620] Updated weights for policy 1, policy_version 621173 (0.0009) [2023-12-26 19:53:00,468][105620] Updated weights for policy 1, policy_version 621183 (0.0008) [2023-12-26 19:53:00,622][105692] Updated weights for policy 0, policy_version 620301 (0.0008) [2023-12-26 19:53:00,682][105692] Updated weights for policy 0, policy_version 620311 (0.0008) [2023-12-26 19:53:00,730][105692] Updated weights for policy 0, policy_version 620321 (0.0008) [2023-12-26 19:53:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 317874176. Throughput: 0: 9773.7, 1: 10021.4. Samples: 317846072. Policy #0 lag: (min: 25.0, avg: 40.1, max: 57.0) [2023-12-26 19:53:01,062][104569] Avg episode reward: [(0, '9354.435'), (1, '9174.528')] [2023-12-26 19:53:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000621192_159039488.pth... [2023-12-26 19:53:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000620328_158834688.pth... [2023-12-26 19:53:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000620040_158744576.pth [2023-12-26 19:53:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000619208_158547968.pth [2023-12-26 19:53:01,164][105620] Updated weights for policy 1, policy_version 621193 (0.0008) [2023-12-26 19:53:01,219][105620] Updated weights for policy 1, policy_version 621203 (0.0008) [2023-12-26 19:53:01,277][105620] Updated weights for policy 1, policy_version 621213 (0.0009) [2023-12-26 19:53:01,337][105620] Updated weights for policy 1, policy_version 621223 (0.0008) [2023-12-26 19:53:01,528][105692] Updated weights for policy 0, policy_version 620331 (0.0007) [2023-12-26 19:53:01,576][105692] Updated weights for policy 0, policy_version 620341 (0.0005) [2023-12-26 19:53:01,635][105692] Updated weights for policy 0, policy_version 620351 (0.0008) [2023-12-26 19:53:02,053][105620] Updated weights for policy 1, policy_version 621233 (0.0008) [2023-12-26 19:53:02,101][105620] Updated weights for policy 1, policy_version 621243 (0.0010) [2023-12-26 19:53:02,160][105620] Updated weights for policy 1, policy_version 621253 (0.0010) [2023-12-26 19:53:02,329][105692] Updated weights for policy 0, policy_version 620361 (0.0010) [2023-12-26 19:53:02,395][105692] Updated weights for policy 0, policy_version 620371 (0.0011) [2023-12-26 19:53:02,464][105692] Updated weights for policy 0, policy_version 620381 (0.0011) [2023-12-26 19:53:02,529][105692] Updated weights for policy 0, policy_version 620391 (0.0011) [2023-12-26 19:53:02,799][105620] Updated weights for policy 1, policy_version 621263 (0.0007) [2023-12-26 19:53:02,853][105620] Updated weights for policy 1, policy_version 621273 (0.0005) [2023-12-26 19:53:02,915][105620] Updated weights for policy 1, policy_version 621283 (0.0005) [2023-12-26 19:53:03,205][105692] Updated weights for policy 0, policy_version 620401 (0.0010) [2023-12-26 19:53:03,265][105692] Updated weights for policy 0, policy_version 620411 (0.0008) [2023-12-26 19:53:03,329][105692] Updated weights for policy 0, policy_version 620421 (0.0007) [2023-12-26 19:53:03,432][105620] Updated weights for policy 1, policy_version 621293 (0.0007) [2023-12-26 19:53:03,490][105620] Updated weights for policy 1, policy_version 621303 (0.0007) [2023-12-26 19:53:03,547][105620] Updated weights for policy 1, policy_version 621313 (0.0005) [2023-12-26 19:53:04,064][105692] Updated weights for policy 0, policy_version 620431 (0.0010) [2023-12-26 19:53:04,091][105620] Updated weights for policy 1, policy_version 621323 (0.0006) [2023-12-26 19:53:04,127][105692] Updated weights for policy 0, policy_version 620441 (0.0011) [2023-12-26 19:53:04,163][105620] Updated weights for policy 1, policy_version 621333 (0.0006) [2023-12-26 19:53:04,183][105692] Updated weights for policy 0, policy_version 620451 (0.0011) [2023-12-26 19:53:04,218][105620] Updated weights for policy 1, policy_version 621343 (0.0005) [2023-12-26 19:53:04,823][105620] Updated weights for policy 1, policy_version 621353 (0.0007) [2023-12-26 19:53:04,861][105692] Updated weights for policy 0, policy_version 620461 (0.0008) [2023-12-26 19:53:04,871][105620] Updated weights for policy 1, policy_version 621363 (0.0006) [2023-12-26 19:53:04,923][105692] Updated weights for policy 0, policy_version 620471 (0.0005) [2023-12-26 19:53:04,933][105620] Updated weights for policy 1, policy_version 621373 (0.0005) [2023-12-26 19:53:04,983][105692] Updated weights for policy 0, policy_version 620481 (0.0010) [2023-12-26 19:53:04,994][105620] Updated weights for policy 1, policy_version 621383 (0.0007) [2023-12-26 19:53:05,574][105620] Updated weights for policy 1, policy_version 621393 (0.0008) [2023-12-26 19:53:05,621][105620] Updated weights for policy 1, policy_version 621403 (0.0007) [2023-12-26 19:53:05,641][105692] Updated weights for policy 0, policy_version 620491 (0.0006) [2023-12-26 19:53:05,679][105620] Updated weights for policy 1, policy_version 621413 (0.0009) [2023-12-26 19:53:05,708][105692] Updated weights for policy 0, policy_version 620501 (0.0007) [2023-12-26 19:53:05,763][105692] Updated weights for policy 0, policy_version 620511 (0.0006) [2023-12-26 19:53:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 317980672. Throughput: 0: 9681.4, 1: 10168.3. Samples: 317966176. Policy #0 lag: (min: 25.0, avg: 40.1, max: 57.0) [2023-12-26 19:53:06,063][104569] Avg episode reward: [(0, '9354.438'), (1, '9172.740')] [2023-12-26 19:53:06,418][105620] Updated weights for policy 1, policy_version 621423 (0.0008) [2023-12-26 19:53:06,451][105692] Updated weights for policy 0, policy_version 620521 (0.0008) [2023-12-26 19:53:06,485][105620] Updated weights for policy 1, policy_version 621433 (0.0007) [2023-12-26 19:53:06,508][105692] Updated weights for policy 0, policy_version 620531 (0.0008) [2023-12-26 19:53:06,549][105620] Updated weights for policy 1, policy_version 621443 (0.0006) [2023-12-26 19:53:06,564][105692] Updated weights for policy 0, policy_version 620541 (0.0008) [2023-12-26 19:53:06,623][105692] Updated weights for policy 0, policy_version 620551 (0.0009) [2023-12-26 19:53:07,251][105620] Updated weights for policy 1, policy_version 621453 (0.0007) [2023-12-26 19:53:07,306][105620] Updated weights for policy 1, policy_version 621463 (0.0009) [2023-12-26 19:53:07,360][105620] Updated weights for policy 1, policy_version 621473 (0.0008) [2023-12-26 19:53:07,402][105692] Updated weights for policy 0, policy_version 620561 (0.0008) [2023-12-26 19:53:07,457][105692] Updated weights for policy 0, policy_version 620571 (0.0009) [2023-12-26 19:53:07,523][105692] Updated weights for policy 0, policy_version 620581 (0.0009) [2023-12-26 19:53:08,142][105620] Updated weights for policy 1, policy_version 621483 (0.0007) [2023-12-26 19:53:08,193][105620] Updated weights for policy 1, policy_version 621493 (0.0009) [2023-12-26 19:53:08,238][105620] Updated weights for policy 1, policy_version 621503 (0.0007) [2023-12-26 19:53:08,271][105692] Updated weights for policy 0, policy_version 620591 (0.0009) [2023-12-26 19:53:08,340][105692] Updated weights for policy 0, policy_version 620601 (0.0006) [2023-12-26 19:53:08,400][105692] Updated weights for policy 0, policy_version 620611 (0.0009) [2023-12-26 19:53:08,956][105692] Updated weights for policy 0, policy_version 620621 (0.0007) [2023-12-26 19:53:09,012][105620] Updated weights for policy 1, policy_version 621513 (0.0008) [2023-12-26 19:53:09,013][105692] Updated weights for policy 0, policy_version 620631 (0.0007) [2023-12-26 19:53:09,076][105692] Updated weights for policy 0, policy_version 620641 (0.0007) [2023-12-26 19:53:09,078][105620] Updated weights for policy 1, policy_version 621523 (0.0005) [2023-12-26 19:53:09,144][105620] Updated weights for policy 1, policy_version 621533 (0.0006) [2023-12-26 19:53:09,197][105620] Updated weights for policy 1, policy_version 621543 (0.0006) [2023-12-26 19:53:09,798][105692] Updated weights for policy 0, policy_version 620651 (0.0007) [2023-12-26 19:53:09,865][105692] Updated weights for policy 0, policy_version 620661 (0.0009) [2023-12-26 19:53:09,879][105620] Updated weights for policy 1, policy_version 621553 (0.0007) [2023-12-26 19:53:09,927][105692] Updated weights for policy 0, policy_version 620671 (0.0006) [2023-12-26 19:53:09,943][105620] Updated weights for policy 1, policy_version 621563 (0.0008) [2023-12-26 19:53:10,016][105620] Updated weights for policy 1, policy_version 621573 (0.0009) [2023-12-26 19:53:10,706][105692] Updated weights for policy 0, policy_version 620681 (0.0009) [2023-12-26 19:53:10,736][105620] Updated weights for policy 1, policy_version 621583 (0.0007) [2023-12-26 19:53:10,762][105692] Updated weights for policy 0, policy_version 620691 (0.0011) [2023-12-26 19:53:10,792][105620] Updated weights for policy 1, policy_version 621593 (0.0006) [2023-12-26 19:53:10,824][105692] Updated weights for policy 0, policy_version 620701 (0.0010) [2023-12-26 19:53:10,841][105620] Updated weights for policy 1, policy_version 621603 (0.0008) [2023-12-26 19:53:10,882][105692] Updated weights for policy 0, policy_version 620711 (0.0010) [2023-12-26 19:53:11,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 318078976. Throughput: 0: 9688.0, 1: 10146.3. Samples: 318083320. Policy #0 lag: (min: 25.0, avg: 40.1, max: 57.0) [2023-12-26 19:53:11,062][104569] Avg episode reward: [(0, '9262.922'), (1, '8818.739')] [2023-12-26 19:53:11,610][105620] Updated weights for policy 1, policy_version 621613 (0.0008) [2023-12-26 19:53:11,662][105692] Updated weights for policy 0, policy_version 620721 (0.0009) [2023-12-26 19:53:11,674][105620] Updated weights for policy 1, policy_version 621623 (0.0009) [2023-12-26 19:53:11,740][105692] Updated weights for policy 0, policy_version 620731 (0.0009) [2023-12-26 19:53:11,754][105620] Updated weights for policy 1, policy_version 621633 (0.0008) [2023-12-26 19:53:11,824][105692] Updated weights for policy 0, policy_version 620741 (0.0008) [2023-12-26 19:53:12,539][105620] Updated weights for policy 1, policy_version 621643 (0.0013) [2023-12-26 19:53:12,597][105620] Updated weights for policy 1, policy_version 621653 (0.0008) [2023-12-26 19:53:12,649][105620] Updated weights for policy 1, policy_version 621663 (0.0008) [2023-12-26 19:53:12,655][105692] Updated weights for policy 0, policy_version 620751 (0.0008) [2023-12-26 19:53:12,712][105692] Updated weights for policy 0, policy_version 620761 (0.0009) [2023-12-26 19:53:12,771][105692] Updated weights for policy 0, policy_version 620771 (0.0009) [2023-12-26 19:53:13,408][105620] Updated weights for policy 1, policy_version 621673 (0.0006) [2023-12-26 19:53:13,462][105620] Updated weights for policy 1, policy_version 621683 (0.0009) [2023-12-26 19:53:13,486][105692] Updated weights for policy 0, policy_version 620781 (0.0008) [2023-12-26 19:53:13,508][105620] Updated weights for policy 1, policy_version 621693 (0.0006) [2023-12-26 19:53:13,542][105692] Updated weights for policy 0, policy_version 620791 (0.0008) [2023-12-26 19:53:13,560][105620] Updated weights for policy 1, policy_version 621703 (0.0006) [2023-12-26 19:53:13,605][105692] Updated weights for policy 0, policy_version 620801 (0.0008) [2023-12-26 19:53:14,199][105692] Updated weights for policy 0, policy_version 620811 (0.0009) [2023-12-26 19:53:14,254][105692] Updated weights for policy 0, policy_version 620821 (0.0007) [2023-12-26 19:53:14,313][105692] Updated weights for policy 0, policy_version 620831 (0.0005) [2023-12-26 19:53:14,432][105620] Updated weights for policy 1, policy_version 621713 (0.0009) [2023-12-26 19:53:14,493][105620] Updated weights for policy 1, policy_version 621723 (0.0007) [2023-12-26 19:53:14,554][105620] Updated weights for policy 1, policy_version 621733 (0.0009) [2023-12-26 19:53:14,964][105692] Updated weights for policy 0, policy_version 620841 (0.0006) [2023-12-26 19:53:15,041][105692] Updated weights for policy 0, policy_version 620852 (0.0010) [2023-12-26 19:53:15,099][105692] Updated weights for policy 0, policy_version 620862 (0.0008) [2023-12-26 19:53:15,152][105692] Updated weights for policy 0, policy_version 620872 (0.0010) [2023-12-26 19:53:15,286][105620] Updated weights for policy 1, policy_version 621743 (0.0007) [2023-12-26 19:53:15,339][105620] Updated weights for policy 1, policy_version 621753 (0.0008) [2023-12-26 19:53:15,394][105620] Updated weights for policy 1, policy_version 621763 (0.0010) [2023-12-26 19:53:15,862][105692] Updated weights for policy 0, policy_version 620882 (0.0005) [2023-12-26 19:53:15,915][105692] Updated weights for policy 0, policy_version 620892 (0.0005) [2023-12-26 19:53:15,974][105692] Updated weights for policy 0, policy_version 620902 (0.0005) [2023-12-26 19:53:16,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 318169088. Throughput: 0: 9579.9, 1: 9978.9. Samples: 318137052. Policy #0 lag: (min: 25.0, avg: 40.1, max: 57.0) [2023-12-26 19:53:16,062][104569] Avg episode reward: [(0, '9172.454'), (1, '8641.983')] [2023-12-26 19:53:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000621768_159186944.pth... [2023-12-26 19:53:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000620904_158982144.pth... [2023-12-26 19:53:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000620616_158892032.pth [2023-12-26 19:53:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000619752_158687232.pth [2023-12-26 19:53:16,188][105620] Updated weights for policy 1, policy_version 621773 (0.0007) [2023-12-26 19:53:16,276][105620] Updated weights for policy 1, policy_version 621783 (0.0006) [2023-12-26 19:53:16,335][105620] Updated weights for policy 1, policy_version 621793 (0.0005) [2023-12-26 19:53:16,573][105692] Updated weights for policy 0, policy_version 620912 (0.0007) [2023-12-26 19:53:16,619][105692] Updated weights for policy 0, policy_version 620922 (0.0006) [2023-12-26 19:53:16,664][105692] Updated weights for policy 0, policy_version 620932 (0.0007) [2023-12-26 19:53:16,979][105620] Updated weights for policy 1, policy_version 621803 (0.0010) [2023-12-26 19:53:17,031][105620] Updated weights for policy 1, policy_version 621813 (0.0010) [2023-12-26 19:53:17,086][105620] Updated weights for policy 1, policy_version 621823 (0.0010) [2023-12-26 19:53:17,271][105692] Updated weights for policy 0, policy_version 620942 (0.0008) [2023-12-26 19:53:17,322][105692] Updated weights for policy 0, policy_version 620952 (0.0010) [2023-12-26 19:53:17,366][105692] Updated weights for policy 0, policy_version 620962 (0.0010) [2023-12-26 19:53:17,702][105620] Updated weights for policy 1, policy_version 621833 (0.0006) [2023-12-26 19:53:17,774][105620] Updated weights for policy 1, policy_version 621843 (0.0008) [2023-12-26 19:53:17,831][105620] Updated weights for policy 1, policy_version 621853 (0.0007) [2023-12-26 19:53:17,899][105620] Updated weights for policy 1, policy_version 621863 (0.0005) [2023-12-26 19:53:18,014][105692] Updated weights for policy 0, policy_version 620972 (0.0008) [2023-12-26 19:53:18,074][105692] Updated weights for policy 0, policy_version 620982 (0.0007) [2023-12-26 19:53:18,140][105692] Updated weights for policy 0, policy_version 620992 (0.0006) [2023-12-26 19:53:18,454][105620] Updated weights for policy 1, policy_version 621873 (0.0008) [2023-12-26 19:53:18,517][105620] Updated weights for policy 1, policy_version 621883 (0.0008) [2023-12-26 19:53:18,581][105620] Updated weights for policy 1, policy_version 621893 (0.0008) [2023-12-26 19:53:18,828][105692] Updated weights for policy 0, policy_version 621002 (0.0009) [2023-12-26 19:53:18,894][105692] Updated weights for policy 0, policy_version 621012 (0.0006) [2023-12-26 19:53:18,952][105692] Updated weights for policy 0, policy_version 621022 (0.0010) [2023-12-26 19:53:18,997][105692] Updated weights for policy 0, policy_version 621032 (0.0010) [2023-12-26 19:53:19,326][105620] Updated weights for policy 1, policy_version 621903 (0.0009) [2023-12-26 19:53:19,386][105620] Updated weights for policy 1, policy_version 621913 (0.0009) [2023-12-26 19:53:19,444][105620] Updated weights for policy 1, policy_version 621923 (0.0007) [2023-12-26 19:53:19,686][105692] Updated weights for policy 0, policy_version 621042 (0.0008) [2023-12-26 19:53:19,749][105692] Updated weights for policy 0, policy_version 621052 (0.0011) [2023-12-26 19:53:19,816][105692] Updated weights for policy 0, policy_version 621062 (0.0011) [2023-12-26 19:53:20,168][105620] Updated weights for policy 1, policy_version 621933 (0.0007) [2023-12-26 19:53:20,235][105620] Updated weights for policy 1, policy_version 621943 (0.0007) [2023-12-26 19:53:20,304][105620] Updated weights for policy 1, policy_version 621953 (0.0007) [2023-12-26 19:53:20,508][105692] Updated weights for policy 0, policy_version 621072 (0.0011) [2023-12-26 19:53:20,561][105692] Updated weights for policy 0, policy_version 621082 (0.0010) [2023-12-26 19:53:20,625][105692] Updated weights for policy 0, policy_version 621092 (0.0011) [2023-12-26 19:53:21,028][105620] Updated weights for policy 1, policy_version 621963 (0.0010) [2023-12-26 19:53:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 318267392. Throughput: 0: 9708.1, 1: 9980.3. Samples: 318260856. Policy #0 lag: (min: 25.0, avg: 40.1, max: 57.0) [2023-12-26 19:53:21,062][104569] Avg episode reward: [(0, '9172.642'), (1, '8745.100')] [2023-12-26 19:53:21,091][105620] Updated weights for policy 1, policy_version 621973 (0.0008) [2023-12-26 19:53:21,156][105620] Updated weights for policy 1, policy_version 621983 (0.0008) [2023-12-26 19:53:21,403][105692] Updated weights for policy 0, policy_version 621102 (0.0011) [2023-12-26 19:53:21,470][105692] Updated weights for policy 0, policy_version 621112 (0.0011) [2023-12-26 19:53:21,539][105692] Updated weights for policy 0, policy_version 621122 (0.0011) [2023-12-26 19:53:21,925][105620] Updated weights for policy 1, policy_version 621993 (0.0009) [2023-12-26 19:53:21,986][105620] Updated weights for policy 1, policy_version 622003 (0.0008) [2023-12-26 19:53:22,053][105620] Updated weights for policy 1, policy_version 622013 (0.0009) [2023-12-26 19:53:22,111][105620] Updated weights for policy 1, policy_version 622023 (0.0009) [2023-12-26 19:53:22,214][105692] Updated weights for policy 0, policy_version 621132 (0.0011) [2023-12-26 19:53:22,277][105692] Updated weights for policy 0, policy_version 621142 (0.0008) [2023-12-26 19:53:22,330][105692] Updated weights for policy 0, policy_version 621152 (0.0008) [2023-12-26 19:53:22,947][105692] Updated weights for policy 0, policy_version 621162 (0.0007) [2023-12-26 19:53:22,947][105620] Updated weights for policy 1, policy_version 622033 (0.0010) [2023-12-26 19:53:23,006][105620] Updated weights for policy 1, policy_version 622043 (0.0007) [2023-12-26 19:53:23,008][105692] Updated weights for policy 0, policy_version 621172 (0.0010) [2023-12-26 19:53:23,067][105620] Updated weights for policy 1, policy_version 622053 (0.0006) [2023-12-26 19:53:23,071][105692] Updated weights for policy 0, policy_version 621182 (0.0011) [2023-12-26 19:53:23,134][105692] Updated weights for policy 0, policy_version 621192 (0.0006) [2023-12-26 19:53:23,701][105692] Updated weights for policy 0, policy_version 621202 (0.0005) [2023-12-26 19:53:23,747][105692] Updated weights for policy 0, policy_version 621212 (0.0005) [2023-12-26 19:53:23,792][105692] Updated weights for policy 0, policy_version 621222 (0.0008) [2023-12-26 19:53:23,868][105620] Updated weights for policy 1, policy_version 622063 (0.0005) [2023-12-26 19:53:23,933][105620] Updated weights for policy 1, policy_version 622073 (0.0005) [2023-12-26 19:53:23,993][105620] Updated weights for policy 1, policy_version 622083 (0.0007) [2023-12-26 19:53:24,394][105692] Updated weights for policy 0, policy_version 621232 (0.0008) [2023-12-26 19:53:24,458][105692] Updated weights for policy 0, policy_version 621242 (0.0010) [2023-12-26 19:53:24,518][105692] Updated weights for policy 0, policy_version 621252 (0.0009) [2023-12-26 19:53:24,660][105620] Updated weights for policy 1, policy_version 622093 (0.0006) [2023-12-26 19:53:24,718][105620] Updated weights for policy 1, policy_version 622103 (0.0008) [2023-12-26 19:53:24,777][105620] Updated weights for policy 1, policy_version 622113 (0.0008) [2023-12-26 19:53:25,147][105692] Updated weights for policy 0, policy_version 621262 (0.0010) [2023-12-26 19:53:25,193][105692] Updated weights for policy 0, policy_version 621272 (0.0009) [2023-12-26 19:53:25,241][105692] Updated weights for policy 0, policy_version 621282 (0.0008) [2023-12-26 19:53:25,574][105620] Updated weights for policy 1, policy_version 622123 (0.0008) [2023-12-26 19:53:25,638][105620] Updated weights for policy 1, policy_version 622133 (0.0009) [2023-12-26 19:53:25,703][105620] Updated weights for policy 1, policy_version 622143 (0.0009) [2023-12-26 19:53:25,841][105692] Updated weights for policy 0, policy_version 621292 (0.0008) [2023-12-26 19:53:25,891][105692] Updated weights for policy 0, policy_version 621302 (0.0009) [2023-12-26 19:53:25,944][105692] Updated weights for policy 0, policy_version 621312 (0.0009) [2023-12-26 19:53:26,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19933.8, 300 sec: 19633.0). Total num frames: 318373888. Throughput: 0: 9859.3, 1: 9899.2. Samples: 318377244. Policy #0 lag: (min: 25.0, avg: 40.1, max: 57.0) [2023-12-26 19:53:26,063][104569] Avg episode reward: [(0, '9354.801'), (1, '9007.231')] [2023-12-26 19:53:26,460][105620] Updated weights for policy 1, policy_version 622153 (0.0009) [2023-12-26 19:53:26,507][105620] Updated weights for policy 1, policy_version 622163 (0.0009) [2023-12-26 19:53:26,556][105620] Updated weights for policy 1, policy_version 622173 (0.0008) [2023-12-26 19:53:26,606][105620] Updated weights for policy 1, policy_version 622183 (0.0009) [2023-12-26 19:53:26,661][105692] Updated weights for policy 0, policy_version 621323 (0.0008) [2023-12-26 19:53:26,706][105692] Updated weights for policy 0, policy_version 621333 (0.0005) [2023-12-26 19:53:26,754][105692] Updated weights for policy 0, policy_version 621343 (0.0005) [2023-12-26 19:53:27,381][105620] Updated weights for policy 1, policy_version 622193 (0.0009) [2023-12-26 19:53:27,383][105692] Updated weights for policy 0, policy_version 621353 (0.0008) [2023-12-26 19:53:27,434][105620] Updated weights for policy 1, policy_version 622203 (0.0007) [2023-12-26 19:53:27,440][105692] Updated weights for policy 0, policy_version 621363 (0.0007) [2023-12-26 19:53:27,486][105692] Updated weights for policy 0, policy_version 621373 (0.0008) [2023-12-26 19:53:27,493][105620] Updated weights for policy 1, policy_version 622213 (0.0008) [2023-12-26 19:53:27,544][105692] Updated weights for policy 0, policy_version 621383 (0.0007) [2023-12-26 19:53:28,215][105692] Updated weights for policy 0, policy_version 621393 (0.0006) [2023-12-26 19:53:28,255][105692] Updated weights for policy 0, policy_version 621403 (0.0005) [2023-12-26 19:53:28,260][105620] Updated weights for policy 1, policy_version 622223 (0.0010) [2023-12-26 19:53:28,302][105692] Updated weights for policy 0, policy_version 621413 (0.0006) [2023-12-26 19:53:28,322][105620] Updated weights for policy 1, policy_version 622233 (0.0010) [2023-12-26 19:53:28,381][105620] Updated weights for policy 1, policy_version 622243 (0.0009) [2023-12-26 19:53:29,053][105692] Updated weights for policy 0, policy_version 621423 (0.0008) [2023-12-26 19:53:29,103][105692] Updated weights for policy 0, policy_version 621433 (0.0007) [2023-12-26 19:53:29,116][105620] Updated weights for policy 1, policy_version 622253 (0.0010) [2023-12-26 19:53:29,157][105620] Updated weights for policy 1, policy_version 622263 (0.0010) [2023-12-26 19:53:29,160][105692] Updated weights for policy 0, policy_version 621443 (0.0007) [2023-12-26 19:53:29,205][105620] Updated weights for policy 1, policy_version 622273 (0.0010) [2023-12-26 19:53:29,931][105692] Updated weights for policy 0, policy_version 621453 (0.0008) [2023-12-26 19:53:29,988][105692] Updated weights for policy 0, policy_version 621463 (0.0010) [2023-12-26 19:53:29,996][105620] Updated weights for policy 1, policy_version 622283 (0.0007) [2023-12-26 19:53:30,040][105692] Updated weights for policy 0, policy_version 621473 (0.0010) [2023-12-26 19:53:30,053][105620] Updated weights for policy 1, policy_version 622293 (0.0005) [2023-12-26 19:53:30,110][105620] Updated weights for policy 1, policy_version 622303 (0.0005) [2023-12-26 19:53:30,775][105692] Updated weights for policy 0, policy_version 621483 (0.0010) [2023-12-26 19:53:30,803][105620] Updated weights for policy 1, policy_version 622313 (0.0007) [2023-12-26 19:53:30,836][105692] Updated weights for policy 0, policy_version 621493 (0.0008) [2023-12-26 19:53:30,856][105620] Updated weights for policy 1, policy_version 622323 (0.0007) [2023-12-26 19:53:30,890][105692] Updated weights for policy 0, policy_version 621503 (0.0006) [2023-12-26 19:53:30,911][105620] Updated weights for policy 1, policy_version 622333 (0.0007) [2023-12-26 19:53:30,967][105620] Updated weights for policy 1, policy_version 622343 (0.0008) [2023-12-26 19:53:31,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 318472192. Throughput: 0: 9978.3, 1: 9893.5. Samples: 318437388. Policy #0 lag: (min: 25.0, avg: 40.1, max: 57.0) [2023-12-26 19:53:31,062][104569] Avg episode reward: [(0, '9354.803'), (1, '9179.987')] [2023-12-26 19:53:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000621512_159137792.pth... [2023-12-26 19:53:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000622344_159334400.pth... [2023-12-26 19:53:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000620328_158834688.pth [2023-12-26 19:53:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000621192_159039488.pth [2023-12-26 19:53:31,652][105692] Updated weights for policy 0, policy_version 621513 (0.0006) [2023-12-26 19:53:31,715][105620] Updated weights for policy 1, policy_version 622353 (0.0007) [2023-12-26 19:53:31,716][105692] Updated weights for policy 0, policy_version 621523 (0.0009) [2023-12-26 19:53:31,777][105620] Updated weights for policy 1, policy_version 622363 (0.0007) [2023-12-26 19:53:31,782][105692] Updated weights for policy 0, policy_version 621533 (0.0006) [2023-12-26 19:53:31,832][105620] Updated weights for policy 1, policy_version 622373 (0.0005) [2023-12-26 19:53:31,844][105692] Updated weights for policy 0, policy_version 621543 (0.0006) [2023-12-26 19:53:32,496][105620] Updated weights for policy 1, policy_version 622383 (0.0010) [2023-12-26 19:53:32,552][105620] Updated weights for policy 1, policy_version 622393 (0.0010) [2023-12-26 19:53:32,557][105692] Updated weights for policy 0, policy_version 621553 (0.0010) [2023-12-26 19:53:32,613][105620] Updated weights for policy 1, policy_version 622403 (0.0010) [2023-12-26 19:53:32,616][105692] Updated weights for policy 0, policy_version 621563 (0.0006) [2023-12-26 19:53:32,673][105692] Updated weights for policy 0, policy_version 621573 (0.0007) [2023-12-26 19:53:33,290][105692] Updated weights for policy 0, policy_version 621583 (0.0008) [2023-12-26 19:53:33,346][105692] Updated weights for policy 0, policy_version 621593 (0.0005) [2023-12-26 19:53:33,358][105620] Updated weights for policy 1, policy_version 622413 (0.0010) [2023-12-26 19:53:33,396][105692] Updated weights for policy 0, policy_version 621603 (0.0005) [2023-12-26 19:53:33,399][105620] Updated weights for policy 1, policy_version 622423 (0.0010) [2023-12-26 19:53:33,454][105620] Updated weights for policy 1, policy_version 622433 (0.0010) [2023-12-26 19:53:33,961][105692] Updated weights for policy 0, policy_version 621613 (0.0008) [2023-12-26 19:53:34,004][105692] Updated weights for policy 0, policy_version 621623 (0.0007) [2023-12-26 19:53:34,053][105692] Updated weights for policy 0, policy_version 621633 (0.0009) [2023-12-26 19:53:34,095][105620] Updated weights for policy 1, policy_version 622443 (0.0009) [2023-12-26 19:53:34,153][105620] Updated weights for policy 1, policy_version 622453 (0.0006) [2023-12-26 19:53:34,208][105620] Updated weights for policy 1, policy_version 622463 (0.0008) [2023-12-26 19:53:34,809][105692] Updated weights for policy 0, policy_version 621643 (0.0010) [2023-12-26 19:53:34,870][105692] Updated weights for policy 0, policy_version 621653 (0.0010) [2023-12-26 19:53:34,899][105620] Updated weights for policy 1, policy_version 622473 (0.0008) [2023-12-26 19:53:34,928][105692] Updated weights for policy 0, policy_version 621663 (0.0010) [2023-12-26 19:53:34,959][105620] Updated weights for policy 1, policy_version 622483 (0.0005) [2023-12-26 19:53:35,010][105620] Updated weights for policy 1, policy_version 622493 (0.0006) [2023-12-26 19:53:35,069][105620] Updated weights for policy 1, policy_version 622503 (0.0005) [2023-12-26 19:53:35,613][105620] Updated weights for policy 1, policy_version 622513 (0.0005) [2023-12-26 19:53:35,654][105692] Updated weights for policy 0, policy_version 621673 (0.0010) [2023-12-26 19:53:35,673][105620] Updated weights for policy 1, policy_version 622523 (0.0005) [2023-12-26 19:53:35,711][105692] Updated weights for policy 0, policy_version 621683 (0.0009) [2023-12-26 19:53:35,730][105620] Updated weights for policy 1, policy_version 622533 (0.0005) [2023-12-26 19:53:35,772][105692] Updated weights for policy 0, policy_version 621693 (0.0005) [2023-12-26 19:53:35,837][105692] Updated weights for policy 0, policy_version 621703 (0.0007) [2023-12-26 19:53:36,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.9, 300 sec: 19660.8). Total num frames: 318570496. Throughput: 0: 10034.5, 1: 9860.4. Samples: 318555764. Policy #0 lag: (min: 25.0, avg: 40.1, max: 57.0) [2023-12-26 19:53:36,062][104569] Avg episode reward: [(0, '9354.928'), (1, '9174.850')] [2023-12-26 19:53:36,369][105620] Updated weights for policy 1, policy_version 622543 (0.0010) [2023-12-26 19:53:36,433][105620] Updated weights for policy 1, policy_version 622553 (0.0011) [2023-12-26 19:53:36,492][105692] Updated weights for policy 0, policy_version 621713 (0.0010) [2023-12-26 19:53:36,501][105620] Updated weights for policy 1, policy_version 622563 (0.0011) [2023-12-26 19:53:36,553][105692] Updated weights for policy 0, policy_version 621723 (0.0006) [2023-12-26 19:53:36,625][105692] Updated weights for policy 0, policy_version 621733 (0.0006) [2023-12-26 19:53:37,246][105692] Updated weights for policy 0, policy_version 621743 (0.0006) [2023-12-26 19:53:37,257][105620] Updated weights for policy 1, policy_version 622573 (0.0011) [2023-12-26 19:53:37,312][105692] Updated weights for policy 0, policy_version 621753 (0.0005) [2023-12-26 19:53:37,313][105620] Updated weights for policy 1, policy_version 622583 (0.0010) [2023-12-26 19:53:37,369][105620] Updated weights for policy 1, policy_version 622593 (0.0010) [2023-12-26 19:53:37,377][105692] Updated weights for policy 0, policy_version 621763 (0.0006) [2023-12-26 19:53:38,036][105620] Updated weights for policy 1, policy_version 622603 (0.0009) [2023-12-26 19:53:38,067][105692] Updated weights for policy 0, policy_version 621773 (0.0007) [2023-12-26 19:53:38,094][105620] Updated weights for policy 1, policy_version 622613 (0.0005) [2023-12-26 19:53:38,122][105692] Updated weights for policy 0, policy_version 621783 (0.0008) [2023-12-26 19:53:38,146][105620] Updated weights for policy 1, policy_version 622623 (0.0007) [2023-12-26 19:53:38,181][105692] Updated weights for policy 0, policy_version 621793 (0.0008) [2023-12-26 19:53:38,696][105620] Updated weights for policy 1, policy_version 622633 (0.0005) [2023-12-26 19:53:38,760][105620] Updated weights for policy 1, policy_version 622643 (0.0008) [2023-12-26 19:53:38,820][105620] Updated weights for policy 1, policy_version 622653 (0.0010) [2023-12-26 19:53:38,879][105620] Updated weights for policy 1, policy_version 622663 (0.0010) [2023-12-26 19:53:38,994][105692] Updated weights for policy 0, policy_version 621803 (0.0009) [2023-12-26 19:53:39,063][105692] Updated weights for policy 0, policy_version 621813 (0.0005) [2023-12-26 19:53:39,129][105585] KL-divergence is very high: 193.3473 [2023-12-26 19:53:39,136][105692] Updated weights for policy 0, policy_version 621823 (0.0005) [2023-12-26 19:53:39,174][105585] KL-divergence is very high: 150.7015 [2023-12-26 19:53:39,513][105620] Updated weights for policy 1, policy_version 622673 (0.0008) [2023-12-26 19:53:39,577][105620] Updated weights for policy 1, policy_version 622683 (0.0011) [2023-12-26 19:53:39,644][105620] Updated weights for policy 1, policy_version 622693 (0.0011) [2023-12-26 19:53:39,776][105692] Updated weights for policy 0, policy_version 621833 (0.0010) [2023-12-26 19:53:39,830][105692] Updated weights for policy 0, policy_version 621843 (0.0011) [2023-12-26 19:53:39,900][105692] Updated weights for policy 0, policy_version 621853 (0.0011) [2023-12-26 19:53:39,966][105692] Updated weights for policy 0, policy_version 621863 (0.0011) [2023-12-26 19:53:40,357][105620] Updated weights for policy 1, policy_version 622703 (0.0008) [2023-12-26 19:53:40,425][105620] Updated weights for policy 1, policy_version 622713 (0.0006) [2023-12-26 19:53:40,489][105620] Updated weights for policy 1, policy_version 622723 (0.0006) [2023-12-26 19:53:40,671][105692] Updated weights for policy 0, policy_version 621873 (0.0007) [2023-12-26 19:53:40,735][105692] Updated weights for policy 0, policy_version 621883 (0.0005) [2023-12-26 19:53:40,791][105692] Updated weights for policy 0, policy_version 621893 (0.0010) [2023-12-26 19:53:41,057][105620] Updated weights for policy 1, policy_version 622733 (0.0008) [2023-12-26 19:53:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19933.9, 300 sec: 19660.8). Total num frames: 318668800. Throughput: 0: 10053.9, 1: 9947.7. Samples: 318679120. Policy #0 lag: (min: 25.0, avg: 40.1, max: 57.0) [2023-12-26 19:53:41,062][104569] Avg episode reward: [(0, '9267.216'), (1, '9181.026')] [2023-12-26 19:53:41,123][105620] Updated weights for policy 1, policy_version 622743 (0.0010) [2023-12-26 19:53:41,191][105620] Updated weights for policy 1, policy_version 622753 (0.0008) [2023-12-26 19:53:41,461][105692] Updated weights for policy 0, policy_version 621903 (0.0009) [2023-12-26 19:53:41,521][105692] Updated weights for policy 0, policy_version 621913 (0.0008) [2023-12-26 19:53:41,581][105692] Updated weights for policy 0, policy_version 621923 (0.0008) [2023-12-26 19:53:41,969][105620] Updated weights for policy 1, policy_version 622763 (0.0008) [2023-12-26 19:53:42,032][105620] Updated weights for policy 1, policy_version 622773 (0.0008) [2023-12-26 19:53:42,083][105620] Updated weights for policy 1, policy_version 622783 (0.0009) [2023-12-26 19:53:42,309][105692] Updated weights for policy 0, policy_version 621933 (0.0008) [2023-12-26 19:53:42,365][105692] Updated weights for policy 0, policy_version 621943 (0.0008) [2023-12-26 19:53:42,428][105692] Updated weights for policy 0, policy_version 621953 (0.0008) [2023-12-26 19:53:42,731][105620] Updated weights for policy 1, policy_version 622793 (0.0009) [2023-12-26 19:53:42,792][105620] Updated weights for policy 1, policy_version 622803 (0.0009) [2023-12-26 19:53:42,861][105620] Updated weights for policy 1, policy_version 622813 (0.0005) [2023-12-26 19:53:42,925][105620] Updated weights for policy 1, policy_version 622823 (0.0007) [2023-12-26 19:53:43,282][105692] Updated weights for policy 0, policy_version 621963 (0.0009) [2023-12-26 19:53:43,343][105692] Updated weights for policy 0, policy_version 621973 (0.0009) [2023-12-26 19:53:43,414][105692] Updated weights for policy 0, policy_version 621983 (0.0009) [2023-12-26 19:53:43,466][105620] Updated weights for policy 1, policy_version 622833 (0.0006) [2023-12-26 19:53:43,516][105620] Updated weights for policy 1, policy_version 622843 (0.0007) [2023-12-26 19:53:43,562][105620] Updated weights for policy 1, policy_version 622853 (0.0005) [2023-12-26 19:53:44,163][105620] Updated weights for policy 1, policy_version 622863 (0.0007) [2023-12-26 19:53:44,208][105620] Updated weights for policy 1, policy_version 622873 (0.0008) [2023-12-26 19:53:44,243][105692] Updated weights for policy 0, policy_version 621993 (0.0009) [2023-12-26 19:53:44,257][105620] Updated weights for policy 1, policy_version 622884 (0.0007) [2023-12-26 19:53:44,305][105692] Updated weights for policy 0, policy_version 622003 (0.0010) [2023-12-26 19:53:44,376][105692] Updated weights for policy 0, policy_version 622013 (0.0009) [2023-12-26 19:53:44,440][105692] Updated weights for policy 0, policy_version 622023 (0.0010) [2023-12-26 19:53:44,995][105620] Updated weights for policy 1, policy_version 622894 (0.0008) [2023-12-26 19:53:45,054][105620] Updated weights for policy 1, policy_version 622904 (0.0008) [2023-12-26 19:53:45,117][105620] Updated weights for policy 1, policy_version 622914 (0.0008) [2023-12-26 19:53:45,187][105692] Updated weights for policy 0, policy_version 622033 (0.0006) [2023-12-26 19:53:45,249][105692] Updated weights for policy 0, policy_version 622043 (0.0006) [2023-12-26 19:53:45,317][105692] Updated weights for policy 0, policy_version 622053 (0.0006) [2023-12-26 19:53:45,864][105692] Updated weights for policy 0, policy_version 622063 (0.0006) [2023-12-26 19:53:45,903][105620] Updated weights for policy 1, policy_version 622924 (0.0009) [2023-12-26 19:53:45,933][105692] Updated weights for policy 0, policy_version 622073 (0.0006) [2023-12-26 19:53:45,956][105620] Updated weights for policy 1, policy_version 622934 (0.0010) [2023-12-26 19:53:45,989][105692] Updated weights for policy 0, policy_version 622083 (0.0006) [2023-12-26 19:53:46,004][105620] Updated weights for policy 1, policy_version 622944 (0.0010) [2023-12-26 19:53:46,062][104569] Fps is (10 sec: 20479.8, 60 sec: 20070.4, 300 sec: 19716.3). Total num frames: 318775296. Throughput: 0: 9900.6, 1: 9909.3. Samples: 318737520. Policy #0 lag: (min: 25.0, avg: 40.1, max: 57.0) [2023-12-26 19:53:46,063][104569] Avg episode reward: [(0, '9354.815'), (1, '9356.488')] [2023-12-26 19:53:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000622088_159285248.pth... [2023-12-26 19:53:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000622952_159490048.pth... [2023-12-26 19:53:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000620904_158982144.pth [2023-12-26 19:53:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000621768_159186944.pth [2023-12-26 19:53:46,657][105620] Updated weights for policy 1, policy_version 622954 (0.0010) [2023-12-26 19:53:46,706][105692] Updated weights for policy 0, policy_version 622093 (0.0008) [2023-12-26 19:53:46,726][105620] Updated weights for policy 1, policy_version 622964 (0.0009) [2023-12-26 19:53:46,757][105692] Updated weights for policy 0, policy_version 622103 (0.0006) [2023-12-26 19:53:46,788][105620] Updated weights for policy 1, policy_version 622974 (0.0009) [2023-12-26 19:53:46,805][105692] Updated weights for policy 0, policy_version 622113 (0.0005) [2023-12-26 19:53:46,847][105620] Updated weights for policy 1, policy_version 622984 (0.0010) [2023-12-26 19:53:47,464][105692] Updated weights for policy 0, policy_version 622123 (0.0007) [2023-12-26 19:53:47,524][105692] Updated weights for policy 0, policy_version 622133 (0.0008) [2023-12-26 19:53:47,563][105620] Updated weights for policy 1, policy_version 622994 (0.0006) [2023-12-26 19:53:47,572][105692] Updated weights for policy 0, policy_version 622143 (0.0009) [2023-12-26 19:53:47,627][105620] Updated weights for policy 1, policy_version 623004 (0.0005) [2023-12-26 19:53:47,693][105620] Updated weights for policy 1, policy_version 623014 (0.0005) [2023-12-26 19:53:48,292][105620] Updated weights for policy 1, policy_version 623024 (0.0005) [2023-12-26 19:53:48,331][105692] Updated weights for policy 0, policy_version 622154 (0.0009) [2023-12-26 19:53:48,359][105620] Updated weights for policy 1, policy_version 623034 (0.0007) [2023-12-26 19:53:48,395][105692] Updated weights for policy 0, policy_version 622164 (0.0007) [2023-12-26 19:53:48,420][105620] Updated weights for policy 1, policy_version 623044 (0.0008) [2023-12-26 19:53:48,452][105692] Updated weights for policy 0, policy_version 622174 (0.0006) [2023-12-26 19:53:48,513][105692] Updated weights for policy 0, policy_version 622184 (0.0009) [2023-12-26 19:53:48,985][105620] Updated weights for policy 1, policy_version 623054 (0.0008) [2023-12-26 19:53:49,049][105620] Updated weights for policy 1, policy_version 623064 (0.0009) [2023-12-26 19:53:49,110][105620] Updated weights for policy 1, policy_version 623074 (0.0008) [2023-12-26 19:53:49,331][105692] Updated weights for policy 0, policy_version 622194 (0.0010) [2023-12-26 19:53:49,396][105692] Updated weights for policy 0, policy_version 622204 (0.0009) [2023-12-26 19:53:49,459][105692] Updated weights for policy 0, policy_version 622214 (0.0009) [2023-12-26 19:53:49,872][105620] Updated weights for policy 1, policy_version 623084 (0.0009) [2023-12-26 19:53:49,924][105620] Updated weights for policy 1, policy_version 623094 (0.0009) [2023-12-26 19:53:49,985][105620] Updated weights for policy 1, policy_version 623104 (0.0009) [2023-12-26 19:53:50,217][105692] Updated weights for policy 0, policy_version 622224 (0.0009) [2023-12-26 19:53:50,279][105692] Updated weights for policy 0, policy_version 622234 (0.0010) [2023-12-26 19:53:50,341][105692] Updated weights for policy 0, policy_version 622244 (0.0009) [2023-12-26 19:53:50,715][105620] Updated weights for policy 1, policy_version 623114 (0.0009) [2023-12-26 19:53:50,778][105620] Updated weights for policy 1, policy_version 623124 (0.0010) [2023-12-26 19:53:50,834][105620] Updated weights for policy 1, policy_version 623134 (0.0008) [2023-12-26 19:53:50,895][105620] Updated weights for policy 1, policy_version 623144 (0.0009) [2023-12-26 19:53:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 318865408. Throughput: 0: 9951.7, 1: 9817.4. Samples: 318855780. Policy #0 lag: (min: 25.0, avg: 40.1, max: 57.0) [2023-12-26 19:53:51,062][104569] Avg episode reward: [(0, '9269.220'), (1, '9096.922')] [2023-12-26 19:53:51,082][105692] Updated weights for policy 0, policy_version 622254 (0.0009) [2023-12-26 19:53:51,146][105692] Updated weights for policy 0, policy_version 622264 (0.0009) [2023-12-26 19:53:51,211][105692] Updated weights for policy 0, policy_version 622274 (0.0009) [2023-12-26 19:53:51,684][105620] Updated weights for policy 1, policy_version 623154 (0.0009) [2023-12-26 19:53:51,752][105620] Updated weights for policy 1, policy_version 623164 (0.0008) [2023-12-26 19:53:51,814][105620] Updated weights for policy 1, policy_version 623174 (0.0009) [2023-12-26 19:53:51,942][105692] Updated weights for policy 0, policy_version 622284 (0.0009) [2023-12-26 19:53:51,998][105692] Updated weights for policy 0, policy_version 622294 (0.0008) [2023-12-26 19:53:52,066][105692] Updated weights for policy 0, policy_version 622304 (0.0009) [2023-12-26 19:53:52,575][105620] Updated weights for policy 1, policy_version 623184 (0.0010) [2023-12-26 19:53:52,638][105620] Updated weights for policy 1, policy_version 623194 (0.0011) [2023-12-26 19:53:52,703][105620] Updated weights for policy 1, policy_version 623204 (0.0010) [2023-12-26 19:53:52,851][105692] Updated weights for policy 0, policy_version 622314 (0.0008) [2023-12-26 19:53:52,896][105692] Updated weights for policy 0, policy_version 622324 (0.0008) [2023-12-26 19:53:52,941][105692] Updated weights for policy 0, policy_version 622334 (0.0010) [2023-12-26 19:53:52,990][105692] Updated weights for policy 0, policy_version 622344 (0.0010) [2023-12-26 19:53:53,424][105620] Updated weights for policy 1, policy_version 623214 (0.0010) [2023-12-26 19:53:53,472][105620] Updated weights for policy 1, policy_version 623224 (0.0010) [2023-12-26 19:53:53,530][105620] Updated weights for policy 1, policy_version 623234 (0.0010) [2023-12-26 19:53:53,784][105692] Updated weights for policy 0, policy_version 622354 (0.0007) [2023-12-26 19:53:53,809][105585] KL-divergence is very high: 114.3255 [2023-12-26 19:53:53,850][105692] Updated weights for policy 0, policy_version 622364 (0.0007) [2023-12-26 19:53:53,859][105585] KL-divergence is very high: 165.9097 [2023-12-26 19:53:53,901][105585] KL-divergence is very high: 130.7699 [2023-12-26 19:53:53,902][105692] Updated weights for policy 0, policy_version 622374 (0.0008) [2023-12-26 19:53:54,273][105620] Updated weights for policy 1, policy_version 623244 (0.0011) [2023-12-26 19:53:54,323][105620] Updated weights for policy 1, policy_version 623254 (0.0010) [2023-12-26 19:53:54,381][105620] Updated weights for policy 1, policy_version 623264 (0.0011) [2023-12-26 19:53:54,621][105692] Updated weights for policy 0, policy_version 622384 (0.0008) [2023-12-26 19:53:54,677][105692] Updated weights for policy 0, policy_version 622394 (0.0007) [2023-12-26 19:53:54,739][105692] Updated weights for policy 0, policy_version 622405 (0.0009) [2023-12-26 19:53:55,027][105620] Updated weights for policy 1, policy_version 623274 (0.0010) [2023-12-26 19:53:55,089][105620] Updated weights for policy 1, policy_version 623284 (0.0011) [2023-12-26 19:53:55,154][105620] Updated weights for policy 1, policy_version 623294 (0.0010) [2023-12-26 19:53:55,205][105620] Updated weights for policy 1, policy_version 623304 (0.0010) [2023-12-26 19:53:55,541][105692] Updated weights for policy 0, policy_version 622415 (0.0008) [2023-12-26 19:53:55,600][105692] Updated weights for policy 0, policy_version 622425 (0.0008) [2023-12-26 19:53:55,660][105692] Updated weights for policy 0, policy_version 622435 (0.0008) [2023-12-26 19:53:55,921][105620] Updated weights for policy 1, policy_version 623314 (0.0009) [2023-12-26 19:53:55,978][105620] Updated weights for policy 1, policy_version 623324 (0.0008) [2023-12-26 19:53:56,039][105620] Updated weights for policy 1, policy_version 623334 (0.0008) [2023-12-26 19:53:56,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19797.4, 300 sec: 19716.4). Total num frames: 318963712. Throughput: 0: 9872.9, 1: 9792.0. Samples: 318968240. Policy #0 lag: (min: 25.0, avg: 40.1, max: 57.0) [2023-12-26 19:53:56,062][104569] Avg episode reward: [(0, '9179.912'), (1, '8822.570')] [2023-12-26 19:53:56,350][105692] Updated weights for policy 0, policy_version 622445 (0.0006) [2023-12-26 19:53:56,402][105692] Updated weights for policy 0, policy_version 622455 (0.0009) [2023-12-26 19:53:56,452][105692] Updated weights for policy 0, policy_version 622465 (0.0009) [2023-12-26 19:53:56,660][105620] Updated weights for policy 1, policy_version 623345 (0.0009) [2023-12-26 19:53:56,707][105620] Updated weights for policy 1, policy_version 623356 (0.0008) [2023-12-26 19:53:56,762][105620] Updated weights for policy 1, policy_version 623366 (0.0005) [2023-12-26 19:53:57,139][105692] Updated weights for policy 0, policy_version 622475 (0.0008) [2023-12-26 19:53:57,186][105692] Updated weights for policy 0, policy_version 622485 (0.0009) [2023-12-26 19:53:57,232][105692] Updated weights for policy 0, policy_version 622495 (0.0007) [2023-12-26 19:53:57,466][105620] Updated weights for policy 1, policy_version 623376 (0.0009) [2023-12-26 19:53:57,518][105620] Updated weights for policy 1, policy_version 623386 (0.0009) [2023-12-26 19:53:57,579][105620] Updated weights for policy 1, policy_version 623396 (0.0009) [2023-12-26 19:53:57,875][105692] Updated weights for policy 0, policy_version 622505 (0.0006) [2023-12-26 19:53:57,928][105692] Updated weights for policy 0, policy_version 622515 (0.0009) [2023-12-26 19:53:57,984][105692] Updated weights for policy 0, policy_version 622525 (0.0009) [2023-12-26 19:53:58,037][105692] Updated weights for policy 0, policy_version 622535 (0.0009) [2023-12-26 19:53:58,373][105620] Updated weights for policy 1, policy_version 623406 (0.0008) [2023-12-26 19:53:58,435][105620] Updated weights for policy 1, policy_version 623416 (0.0008) [2023-12-26 19:53:58,506][105620] Updated weights for policy 1, policy_version 623426 (0.0008) [2023-12-26 19:53:58,821][105692] Updated weights for policy 0, policy_version 622545 (0.0008) [2023-12-26 19:53:58,892][105692] Updated weights for policy 0, policy_version 622555 (0.0008) [2023-12-26 19:53:58,958][105692] Updated weights for policy 0, policy_version 622565 (0.0010) [2023-12-26 19:53:59,319][105620] Updated weights for policy 1, policy_version 623436 (0.0008) [2023-12-26 19:53:59,383][105620] Updated weights for policy 1, policy_version 623446 (0.0008) [2023-12-26 19:53:59,447][105620] Updated weights for policy 1, policy_version 623456 (0.0006) [2023-12-26 19:53:59,733][105692] Updated weights for policy 0, policy_version 622575 (0.0011) [2023-12-26 19:53:59,795][105692] Updated weights for policy 0, policy_version 622585 (0.0011) [2023-12-26 19:53:59,860][105692] Updated weights for policy 0, policy_version 622595 (0.0010) [2023-12-26 19:54:00,073][105620] Updated weights for policy 1, policy_version 623466 (0.0006) [2023-12-26 19:54:00,126][105620] Updated weights for policy 1, policy_version 623476 (0.0010) [2023-12-26 19:54:00,187][105620] Updated weights for policy 1, policy_version 623486 (0.0008) [2023-12-26 19:54:00,243][105620] Updated weights for policy 1, policy_version 623496 (0.0008) [2023-12-26 19:54:00,508][105692] Updated weights for policy 0, policy_version 622605 (0.0010) [2023-12-26 19:54:00,559][105692] Updated weights for policy 0, policy_version 622615 (0.0010) [2023-12-26 19:54:00,610][105692] Updated weights for policy 0, policy_version 622625 (0.0010) [2023-12-26 19:54:01,001][105620] Updated weights for policy 1, policy_version 623506 (0.0005) [2023-12-26 19:54:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 319053824. Throughput: 0: 9943.9, 1: 9835.9. Samples: 319027144. Policy #0 lag: (min: 25.0, avg: 40.1, max: 57.0) [2023-12-26 19:54:01,062][105620] Updated weights for policy 1, policy_version 623516 (0.0009) [2023-12-26 19:54:01,062][104569] Avg episode reward: [(0, '9265.779'), (1, '8540.906')] [2023-12-26 19:54:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000622632_159424512.pth... [2023-12-26 19:54:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000621512_159137792.pth [2023-12-26 19:54:01,124][105620] Updated weights for policy 1, policy_version 623526 (0.0006) [2023-12-26 19:54:01,136][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000623528_159637504.pth... [2023-12-26 19:54:01,140][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000622344_159334400.pth [2023-12-26 19:54:01,371][105692] Updated weights for policy 0, policy_version 622635 (0.0010) [2023-12-26 19:54:01,429][105692] Updated weights for policy 0, policy_version 622645 (0.0008) [2023-12-26 19:54:01,490][105692] Updated weights for policy 0, policy_version 622655 (0.0008) [2023-12-26 19:54:01,788][105620] Updated weights for policy 1, policy_version 623536 (0.0010) [2023-12-26 19:54:01,839][105620] Updated weights for policy 1, policy_version 623546 (0.0010) [2023-12-26 19:54:01,899][105620] Updated weights for policy 1, policy_version 623556 (0.0010) [2023-12-26 19:54:02,283][105692] Updated weights for policy 0, policy_version 622665 (0.0007) [2023-12-26 19:54:02,334][105692] Updated weights for policy 0, policy_version 622675 (0.0008) [2023-12-26 19:54:02,395][105692] Updated weights for policy 0, policy_version 622685 (0.0008) [2023-12-26 19:54:02,453][105692] Updated weights for policy 0, policy_version 622695 (0.0007) [2023-12-26 19:54:02,598][105620] Updated weights for policy 1, policy_version 623566 (0.0010) [2023-12-26 19:54:02,656][105620] Updated weights for policy 1, policy_version 623576 (0.0010) [2023-12-26 19:54:02,716][105620] Updated weights for policy 1, policy_version 623586 (0.0007) [2023-12-26 19:54:03,173][105692] Updated weights for policy 0, policy_version 622705 (0.0005) [2023-12-26 19:54:03,221][105692] Updated weights for policy 0, policy_version 622715 (0.0005) [2023-12-26 19:54:03,274][105692] Updated weights for policy 0, policy_version 622725 (0.0005) [2023-12-26 19:54:03,505][105620] Updated weights for policy 1, policy_version 623596 (0.0006) [2023-12-26 19:54:03,561][105620] Updated weights for policy 1, policy_version 623606 (0.0005) [2023-12-26 19:54:03,617][105620] Updated weights for policy 1, policy_version 623616 (0.0005) [2023-12-26 19:54:03,788][105692] Updated weights for policy 0, policy_version 622735 (0.0005) [2023-12-26 19:54:03,841][105692] Updated weights for policy 0, policy_version 622745 (0.0006) [2023-12-26 19:54:03,903][105692] Updated weights for policy 0, policy_version 622755 (0.0009) [2023-12-26 19:54:04,278][105620] Updated weights for policy 1, policy_version 623626 (0.0009) [2023-12-26 19:54:04,332][105620] Updated weights for policy 1, policy_version 623636 (0.0009) [2023-12-26 19:54:04,384][105620] Updated weights for policy 1, policy_version 623646 (0.0008) [2023-12-26 19:54:04,432][105620] Updated weights for policy 1, policy_version 623656 (0.0009) [2023-12-26 19:54:04,639][105692] Updated weights for policy 0, policy_version 622765 (0.0009) [2023-12-26 19:54:04,701][105692] Updated weights for policy 0, policy_version 622775 (0.0010) [2023-12-26 19:54:04,758][105692] Updated weights for policy 0, policy_version 622785 (0.0010) [2023-12-26 19:54:05,140][105620] Updated weights for policy 1, policy_version 623666 (0.0008) [2023-12-26 19:54:05,184][105620] Updated weights for policy 1, policy_version 623676 (0.0005) [2023-12-26 19:54:05,237][105620] Updated weights for policy 1, policy_version 623687 (0.0009) [2023-12-26 19:54:05,397][105692] Updated weights for policy 0, policy_version 622795 (0.0010) [2023-12-26 19:54:05,450][105692] Updated weights for policy 0, policy_version 622806 (0.0010) [2023-12-26 19:54:05,503][105692] Updated weights for policy 0, policy_version 622816 (0.0009) [2023-12-26 19:54:05,954][105620] Updated weights for policy 1, policy_version 623697 (0.0007) [2023-12-26 19:54:06,013][105620] Updated weights for policy 1, policy_version 623707 (0.0006) [2023-12-26 19:54:06,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 319152128. Throughput: 0: 9833.5, 1: 9810.6. Samples: 319144844. Policy #0 lag: (min: 25.0, avg: 40.1, max: 57.0) [2023-12-26 19:54:06,063][104569] Avg episode reward: [(0, '9355.295'), (1, '8989.012')] [2023-12-26 19:54:06,064][105620] Updated weights for policy 1, policy_version 623717 (0.0009) [2023-12-26 19:54:06,296][105692] Updated weights for policy 0, policy_version 622826 (0.0009) [2023-12-26 19:54:06,345][105692] Updated weights for policy 0, policy_version 622836 (0.0009) [2023-12-26 19:54:06,395][105692] Updated weights for policy 0, policy_version 622846 (0.0009) [2023-12-26 19:54:06,448][105692] Updated weights for policy 0, policy_version 622856 (0.0006) [2023-12-26 19:54:06,794][105620] Updated weights for policy 1, policy_version 623727 (0.0007) [2023-12-26 19:54:06,857][105620] Updated weights for policy 1, policy_version 623737 (0.0008) [2023-12-26 19:54:06,915][105620] Updated weights for policy 1, policy_version 623747 (0.0008) [2023-12-26 19:54:07,187][105692] Updated weights for policy 0, policy_version 622866 (0.0006) [2023-12-26 19:54:07,254][105692] Updated weights for policy 0, policy_version 622876 (0.0006) [2023-12-26 19:54:07,317][105692] Updated weights for policy 0, policy_version 622886 (0.0010) [2023-12-26 19:54:07,556][105620] Updated weights for policy 1, policy_version 623757 (0.0007) [2023-12-26 19:54:07,621][105620] Updated weights for policy 1, policy_version 623767 (0.0008) [2023-12-26 19:54:07,690][105620] Updated weights for policy 1, policy_version 623777 (0.0007) [2023-12-26 19:54:08,007][105692] Updated weights for policy 0, policy_version 622896 (0.0006) [2023-12-26 19:54:08,061][105692] Updated weights for policy 0, policy_version 622906 (0.0006) [2023-12-26 19:54:08,117][105692] Updated weights for policy 0, policy_version 622916 (0.0006) [2023-12-26 19:54:08,401][105620] Updated weights for policy 1, policy_version 623787 (0.0009) [2023-12-26 19:54:08,463][105620] Updated weights for policy 1, policy_version 623797 (0.0010) [2023-12-26 19:54:08,515][105620] Updated weights for policy 1, policy_version 623807 (0.0010) [2023-12-26 19:54:08,733][105692] Updated weights for policy 0, policy_version 622926 (0.0009) [2023-12-26 19:54:08,784][105692] Updated weights for policy 0, policy_version 622936 (0.0010) [2023-12-26 19:54:08,843][105692] Updated weights for policy 0, policy_version 622946 (0.0010) [2023-12-26 19:54:09,187][105620] Updated weights for policy 1, policy_version 623817 (0.0006) [2023-12-26 19:54:09,257][105620] Updated weights for policy 1, policy_version 623827 (0.0007) [2023-12-26 19:54:09,317][105620] Updated weights for policy 1, policy_version 623837 (0.0007) [2023-12-26 19:54:09,376][105620] Updated weights for policy 1, policy_version 623847 (0.0009) [2023-12-26 19:54:09,610][105692] Updated weights for policy 0, policy_version 622956 (0.0011) [2023-12-26 19:54:09,668][105692] Updated weights for policy 0, policy_version 622966 (0.0010) [2023-12-26 19:54:09,728][105692] Updated weights for policy 0, policy_version 622976 (0.0011) [2023-12-26 19:54:10,096][105620] Updated weights for policy 1, policy_version 623857 (0.0010) [2023-12-26 19:54:10,155][105620] Updated weights for policy 1, policy_version 623867 (0.0010) [2023-12-26 19:54:10,218][105620] Updated weights for policy 1, policy_version 623877 (0.0010) [2023-12-26 19:54:10,479][105692] Updated weights for policy 0, policy_version 622986 (0.0011) [2023-12-26 19:54:10,541][105692] Updated weights for policy 0, policy_version 622996 (0.0011) [2023-12-26 19:54:10,605][105692] Updated weights for policy 0, policy_version 623006 (0.0011) [2023-12-26 19:54:10,667][105692] Updated weights for policy 0, policy_version 623016 (0.0010) [2023-12-26 19:54:10,949][105620] Updated weights for policy 1, policy_version 623887 (0.0010) [2023-12-26 19:54:11,001][105620] Updated weights for policy 1, policy_version 623897 (0.0010) [2023-12-26 19:54:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19688.6). Total num frames: 319250432. Throughput: 0: 9743.8, 1: 9926.1. Samples: 319262384. Policy #0 lag: (min: 25.0, avg: 40.1, max: 57.0) [2023-12-26 19:54:11,062][104569] Avg episode reward: [(0, '9355.337'), (1, '9263.568')] [2023-12-26 19:54:11,063][105620] Updated weights for policy 1, policy_version 623907 (0.0010) [2023-12-26 19:54:11,421][105692] Updated weights for policy 0, policy_version 623026 (0.0011) [2023-12-26 19:54:11,480][105692] Updated weights for policy 0, policy_version 623036 (0.0011) [2023-12-26 19:54:11,536][105692] Updated weights for policy 0, policy_version 623046 (0.0010) [2023-12-26 19:54:11,833][105620] Updated weights for policy 1, policy_version 623917 (0.0007) [2023-12-26 19:54:11,899][105620] Updated weights for policy 1, policy_version 623927 (0.0005) [2023-12-26 19:54:11,965][105620] Updated weights for policy 1, policy_version 623937 (0.0009) [2023-12-26 19:54:12,261][105692] Updated weights for policy 0, policy_version 623056 (0.0008) [2023-12-26 19:54:12,320][105692] Updated weights for policy 0, policy_version 623066 (0.0009) [2023-12-26 19:54:12,391][105692] Updated weights for policy 0, policy_version 623076 (0.0009) [2023-12-26 19:54:12,807][105620] Updated weights for policy 1, policy_version 623947 (0.0009) [2023-12-26 19:54:12,874][105620] Updated weights for policy 1, policy_version 623957 (0.0011) [2023-12-26 19:54:12,944][105620] Updated weights for policy 1, policy_version 623967 (0.0011) [2023-12-26 19:54:13,092][105692] Updated weights for policy 0, policy_version 623086 (0.0007) [2023-12-26 19:54:13,146][105692] Updated weights for policy 0, policy_version 623096 (0.0009) [2023-12-26 19:54:13,214][105692] Updated weights for policy 0, policy_version 623106 (0.0010) [2023-12-26 19:54:13,664][105620] Updated weights for policy 1, policy_version 623977 (0.0010) [2023-12-26 19:54:13,714][105620] Updated weights for policy 1, policy_version 623987 (0.0007) [2023-12-26 19:54:13,766][105620] Updated weights for policy 1, policy_version 623997 (0.0005) [2023-12-26 19:54:13,828][105620] Updated weights for policy 1, policy_version 624007 (0.0006) [2023-12-26 19:54:13,967][105692] Updated weights for policy 0, policy_version 623116 (0.0010) [2023-12-26 19:54:14,016][105692] Updated weights for policy 0, policy_version 623126 (0.0010) [2023-12-26 19:54:14,061][105692] Updated weights for policy 0, policy_version 623136 (0.0010) [2023-12-26 19:54:14,511][105620] Updated weights for policy 1, policy_version 624017 (0.0008) [2023-12-26 19:54:14,558][105620] Updated weights for policy 1, policy_version 624027 (0.0007) [2023-12-26 19:54:14,610][105620] Updated weights for policy 1, policy_version 624037 (0.0006) [2023-12-26 19:54:14,837][105692] Updated weights for policy 0, policy_version 623146 (0.0009) [2023-12-26 19:54:14,903][105692] Updated weights for policy 0, policy_version 623156 (0.0006) [2023-12-26 19:54:14,974][105692] Updated weights for policy 0, policy_version 623166 (0.0006) [2023-12-26 19:54:15,038][105692] Updated weights for policy 0, policy_version 623176 (0.0006) [2023-12-26 19:54:15,342][105620] Updated weights for policy 1, policy_version 624047 (0.0009) [2023-12-26 19:54:15,404][105620] Updated weights for policy 1, policy_version 624057 (0.0007) [2023-12-26 19:54:15,464][105620] Updated weights for policy 1, policy_version 624067 (0.0008) [2023-12-26 19:54:15,612][105692] Updated weights for policy 0, policy_version 623187 (0.0010) [2023-12-26 19:54:15,671][105692] Updated weights for policy 0, policy_version 623198 (0.0010) [2023-12-26 19:54:15,734][105692] Updated weights for policy 0, policy_version 623208 (0.0010) [2023-12-26 19:54:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 319348736. Throughput: 0: 9665.2, 1: 9922.1. Samples: 319318820. Policy #0 lag: (min: 25.0, avg: 40.1, max: 57.0) [2023-12-26 19:54:16,063][104569] Avg episode reward: [(0, '9355.394'), (1, '9172.296')] [2023-12-26 19:54:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000624072_159776768.pth... [2023-12-26 19:54:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000623208_159571968.pth... [2023-12-26 19:54:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000622952_159490048.pth [2023-12-26 19:54:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000622088_159285248.pth [2023-12-26 19:54:16,120][105620] Updated weights for policy 1, policy_version 624077 (0.0008) [2023-12-26 19:54:16,168][105620] Updated weights for policy 1, policy_version 624087 (0.0008) [2023-12-26 19:54:16,223][105620] Updated weights for policy 1, policy_version 624097 (0.0008) [2023-12-26 19:54:16,522][105692] Updated weights for policy 0, policy_version 623218 (0.0011) [2023-12-26 19:54:16,575][105692] Updated weights for policy 0, policy_version 623228 (0.0010) [2023-12-26 19:54:16,620][105692] Updated weights for policy 0, policy_version 623238 (0.0010) [2023-12-26 19:54:16,998][105620] Updated weights for policy 1, policy_version 624107 (0.0008) [2023-12-26 19:54:17,046][105620] Updated weights for policy 1, policy_version 624117 (0.0010) [2023-12-26 19:54:17,103][105620] Updated weights for policy 1, policy_version 624127 (0.0008) [2023-12-26 19:54:17,367][105692] Updated weights for policy 0, policy_version 623248 (0.0010) [2023-12-26 19:54:17,423][105692] Updated weights for policy 0, policy_version 623258 (0.0007) [2023-12-26 19:54:17,468][105692] Updated weights for policy 0, policy_version 623268 (0.0005) [2023-12-26 19:54:17,795][105620] Updated weights for policy 1, policy_version 624137 (0.0006) [2023-12-26 19:54:17,852][105620] Updated weights for policy 1, policy_version 624147 (0.0005) [2023-12-26 19:54:17,903][105620] Updated weights for policy 1, policy_version 624157 (0.0005) [2023-12-26 19:54:17,953][105620] Updated weights for policy 1, policy_version 624167 (0.0005) [2023-12-26 19:54:18,102][105692] Updated weights for policy 0, policy_version 623278 (0.0005) [2023-12-26 19:54:18,155][105692] Updated weights for policy 0, policy_version 623288 (0.0005) [2023-12-26 19:54:18,215][105692] Updated weights for policy 0, policy_version 623298 (0.0005) [2023-12-26 19:54:18,597][105620] Updated weights for policy 1, policy_version 624177 (0.0005) [2023-12-26 19:54:18,662][105620] Updated weights for policy 1, policy_version 624187 (0.0010) [2023-12-26 19:54:18,724][105620] Updated weights for policy 1, policy_version 624197 (0.0009) [2023-12-26 19:54:18,864][105692] Updated weights for policy 0, policy_version 623308 (0.0005) [2023-12-26 19:54:18,928][105692] Updated weights for policy 0, policy_version 623318 (0.0006) [2023-12-26 19:54:18,989][105692] Updated weights for policy 0, policy_version 623328 (0.0009) [2023-12-26 19:54:19,435][105620] Updated weights for policy 1, policy_version 624207 (0.0007) [2023-12-26 19:54:19,498][105620] Updated weights for policy 1, policy_version 624217 (0.0007) [2023-12-26 19:54:19,564][105620] Updated weights for policy 1, policy_version 624227 (0.0006) [2023-12-26 19:54:19,730][105692] Updated weights for policy 0, policy_version 623338 (0.0009) [2023-12-26 19:54:19,802][105692] Updated weights for policy 0, policy_version 623348 (0.0010) [2023-12-26 19:54:19,870][105692] Updated weights for policy 0, policy_version 623358 (0.0008) [2023-12-26 19:54:19,942][105692] Updated weights for policy 0, policy_version 623368 (0.0007) [2023-12-26 19:54:20,235][105620] Updated weights for policy 1, policy_version 624237 (0.0009) [2023-12-26 19:54:20,298][105620] Updated weights for policy 1, policy_version 624247 (0.0010) [2023-12-26 19:54:20,364][105620] Updated weights for policy 1, policy_version 624257 (0.0009) [2023-12-26 19:54:20,535][105692] Updated weights for policy 0, policy_version 623378 (0.0008) [2023-12-26 19:54:20,604][105692] Updated weights for policy 0, policy_version 623388 (0.0009) [2023-12-26 19:54:20,667][105692] Updated weights for policy 0, policy_version 623398 (0.0009) [2023-12-26 19:54:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 319447040. Throughput: 0: 9674.9, 1: 9947.7. Samples: 319438784. Policy #0 lag: (min: 25.0, avg: 40.1, max: 57.0) [2023-12-26 19:54:21,062][104569] Avg episode reward: [(0, '9355.361'), (1, '9085.083')] [2023-12-26 19:54:21,165][105620] Updated weights for policy 1, policy_version 624267 (0.0006) [2023-12-26 19:54:21,229][105620] Updated weights for policy 1, policy_version 624277 (0.0009) [2023-12-26 19:54:21,291][105620] Updated weights for policy 1, policy_version 624287 (0.0009) [2023-12-26 19:54:21,400][105692] Updated weights for policy 0, policy_version 623408 (0.0009) [2023-12-26 19:54:21,452][105692] Updated weights for policy 0, policy_version 623418 (0.0009) [2023-12-26 19:54:21,505][105692] Updated weights for policy 0, policy_version 623428 (0.0008) [2023-12-26 19:54:22,060][105620] Updated weights for policy 1, policy_version 624297 (0.0008) [2023-12-26 19:54:22,125][105620] Updated weights for policy 1, policy_version 624307 (0.0007) [2023-12-26 19:54:22,187][105620] Updated weights for policy 1, policy_version 624317 (0.0006) [2023-12-26 19:54:22,240][105692] Updated weights for policy 0, policy_version 623438 (0.0007) [2023-12-26 19:54:22,253][105620] Updated weights for policy 1, policy_version 624327 (0.0006) [2023-12-26 19:54:22,307][105692] Updated weights for policy 0, policy_version 623448 (0.0007) [2023-12-26 19:54:22,376][105692] Updated weights for policy 0, policy_version 623458 (0.0008) [2023-12-26 19:54:22,974][105620] Updated weights for policy 1, policy_version 624337 (0.0009) [2023-12-26 19:54:23,039][105620] Updated weights for policy 1, policy_version 624347 (0.0009) [2023-12-26 19:54:23,098][105620] Updated weights for policy 1, policy_version 624357 (0.0009) [2023-12-26 19:54:23,127][105692] Updated weights for policy 0, policy_version 623468 (0.0007) [2023-12-26 19:54:23,180][105692] Updated weights for policy 0, policy_version 623478 (0.0005) [2023-12-26 19:54:23,240][105692] Updated weights for policy 0, policy_version 623488 (0.0006) [2023-12-26 19:54:23,795][105620] Updated weights for policy 1, policy_version 624367 (0.0007) [2023-12-26 19:54:23,845][105620] Updated weights for policy 1, policy_version 624377 (0.0005) [2023-12-26 19:54:23,896][105692] Updated weights for policy 0, policy_version 623498 (0.0007) [2023-12-26 19:54:23,918][105620] Updated weights for policy 1, policy_version 624387 (0.0007) [2023-12-26 19:54:23,960][105692] Updated weights for policy 0, policy_version 623508 (0.0007) [2023-12-26 19:54:24,018][105692] Updated weights for policy 0, policy_version 623518 (0.0009) [2023-12-26 19:54:24,080][105692] Updated weights for policy 0, policy_version 623528 (0.0009) [2023-12-26 19:54:24,559][105620] Updated weights for policy 1, policy_version 624397 (0.0007) [2023-12-26 19:54:24,620][105620] Updated weights for policy 1, policy_version 624407 (0.0009) [2023-12-26 19:54:24,679][105620] Updated weights for policy 1, policy_version 624417 (0.0009) [2023-12-26 19:54:24,826][105692] Updated weights for policy 0, policy_version 623538 (0.0009) [2023-12-26 19:54:24,880][105692] Updated weights for policy 0, policy_version 623548 (0.0009) [2023-12-26 19:54:24,927][105692] Updated weights for policy 0, policy_version 623558 (0.0009) [2023-12-26 19:54:25,376][105620] Updated weights for policy 1, policy_version 624427 (0.0009) [2023-12-26 19:54:25,439][105620] Updated weights for policy 1, policy_version 624437 (0.0009) [2023-12-26 19:54:25,495][105620] Updated weights for policy 1, policy_version 624447 (0.0008) [2023-12-26 19:54:25,721][105692] Updated weights for policy 0, policy_version 623568 (0.0009) [2023-12-26 19:54:25,775][105692] Updated weights for policy 0, policy_version 623578 (0.0008) [2023-12-26 19:54:25,826][105692] Updated weights for policy 0, policy_version 623588 (0.0008) [2023-12-26 19:54:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 319545344. Throughput: 0: 9647.9, 1: 9773.3. Samples: 319553076. Policy #0 lag: (min: 25.0, avg: 40.1, max: 57.0) [2023-12-26 19:54:26,063][104569] Avg episode reward: [(0, '9355.271'), (1, '9176.292')] [2023-12-26 19:54:26,193][105620] Updated weights for policy 1, policy_version 624457 (0.0009) [2023-12-26 19:54:26,244][105620] Updated weights for policy 1, policy_version 624467 (0.0005) [2023-12-26 19:54:26,301][105620] Updated weights for policy 1, policy_version 624477 (0.0005) [2023-12-26 19:54:26,368][105620] Updated weights for policy 1, policy_version 624487 (0.0005) [2023-12-26 19:54:26,601][105692] Updated weights for policy 0, policy_version 623598 (0.0006) [2023-12-26 19:54:26,646][105692] Updated weights for policy 0, policy_version 623608 (0.0005) [2023-12-26 19:54:26,689][105692] Updated weights for policy 0, policy_version 623618 (0.0005) [2023-12-26 19:54:26,901][105620] Updated weights for policy 1, policy_version 624497 (0.0010) [2023-12-26 19:54:26,948][105620] Updated weights for policy 1, policy_version 624507 (0.0010) [2023-12-26 19:54:26,999][105620] Updated weights for policy 1, policy_version 624517 (0.0008) [2023-12-26 19:54:27,283][105692] Updated weights for policy 0, policy_version 623628 (0.0005) [2023-12-26 19:54:27,344][105692] Updated weights for policy 0, policy_version 623638 (0.0006) [2023-12-26 19:54:27,395][105692] Updated weights for policy 0, policy_version 623648 (0.0010) [2023-12-26 19:54:27,697][105620] Updated weights for policy 1, policy_version 624527 (0.0005) [2023-12-26 19:54:27,751][105620] Updated weights for policy 1, policy_version 624537 (0.0006) [2023-12-26 19:54:27,798][105620] Updated weights for policy 1, policy_version 624547 (0.0010) [2023-12-26 19:54:28,097][105692] Updated weights for policy 0, policy_version 623658 (0.0009) [2023-12-26 19:54:28,155][105692] Updated weights for policy 0, policy_version 623668 (0.0005) [2023-12-26 19:54:28,208][105692] Updated weights for policy 0, policy_version 623678 (0.0005) [2023-12-26 19:54:28,260][105692] Updated weights for policy 0, policy_version 623688 (0.0010) [2023-12-26 19:54:28,515][105620] Updated weights for policy 1, policy_version 624557 (0.0009) [2023-12-26 19:54:28,574][105620] Updated weights for policy 1, policy_version 624567 (0.0008) [2023-12-26 19:54:28,629][105620] Updated weights for policy 1, policy_version 624577 (0.0008) [2023-12-26 19:54:28,946][105692] Updated weights for policy 0, policy_version 623698 (0.0007) [2023-12-26 19:54:28,999][105692] Updated weights for policy 0, policy_version 623708 (0.0009) [2023-12-26 19:54:29,052][105692] Updated weights for policy 0, policy_version 623718 (0.0009) [2023-12-26 19:54:29,420][105620] Updated weights for policy 1, policy_version 624587 (0.0009) [2023-12-26 19:54:29,484][105620] Updated weights for policy 1, policy_version 624597 (0.0008) [2023-12-26 19:54:29,547][105620] Updated weights for policy 1, policy_version 624607 (0.0010) [2023-12-26 19:54:29,757][105692] Updated weights for policy 0, policy_version 623728 (0.0007) [2023-12-26 19:54:29,805][105692] Updated weights for policy 0, policy_version 623738 (0.0006) [2023-12-26 19:54:29,865][105692] Updated weights for policy 0, policy_version 623748 (0.0008) [2023-12-26 19:54:30,236][105620] Updated weights for policy 1, policy_version 624617 (0.0009) [2023-12-26 19:54:30,304][105620] Updated weights for policy 1, policy_version 624627 (0.0008) [2023-12-26 19:54:30,371][105620] Updated weights for policy 1, policy_version 624637 (0.0010) [2023-12-26 19:54:30,433][105620] Updated weights for policy 1, policy_version 624647 (0.0010) [2023-12-26 19:54:30,591][105692] Updated weights for policy 0, policy_version 623758 (0.0007) [2023-12-26 19:54:30,651][105692] Updated weights for policy 0, policy_version 623768 (0.0005) [2023-12-26 19:54:30,709][105692] Updated weights for policy 0, policy_version 623778 (0.0007) [2023-12-26 19:54:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 319643648. Throughput: 0: 9719.1, 1: 9776.9. Samples: 319614840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:54:31,062][104569] Avg episode reward: [(0, '9355.311'), (1, '9356.155')] [2023-12-26 19:54:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000623784_159719424.pth... [2023-12-26 19:54:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000622632_159424512.pth [2023-12-26 19:54:31,102][105620] Updated weights for policy 1, policy_version 624657 (0.0010) [2023-12-26 19:54:31,165][105620] Updated weights for policy 1, policy_version 624667 (0.0009) [2023-12-26 19:54:31,226][105620] Updated weights for policy 1, policy_version 624677 (0.0009) [2023-12-26 19:54:31,246][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000624680_159932416.pth... [2023-12-26 19:54:31,250][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000623528_159637504.pth [2023-12-26 19:54:31,488][105692] Updated weights for policy 0, policy_version 623788 (0.0009) [2023-12-26 19:54:31,545][105692] Updated weights for policy 0, policy_version 623798 (0.0009) [2023-12-26 19:54:31,596][105692] Updated weights for policy 0, policy_version 623808 (0.0005) [2023-12-26 19:54:31,892][105620] Updated weights for policy 1, policy_version 624687 (0.0006) [2023-12-26 19:54:31,958][105620] Updated weights for policy 1, policy_version 624697 (0.0010) [2023-12-26 19:54:32,008][105620] Updated weights for policy 1, policy_version 624707 (0.0009) [2023-12-26 19:54:32,268][105692] Updated weights for policy 0, policy_version 623818 (0.0008) [2023-12-26 19:54:32,331][105692] Updated weights for policy 0, policy_version 623828 (0.0009) [2023-12-26 19:54:32,392][105692] Updated weights for policy 0, policy_version 623838 (0.0008) [2023-12-26 19:54:32,447][105692] Updated weights for policy 0, policy_version 623848 (0.0006) [2023-12-26 19:54:32,670][105620] Updated weights for policy 1, policy_version 624717 (0.0009) [2023-12-26 19:54:32,732][105620] Updated weights for policy 1, policy_version 624727 (0.0010) [2023-12-26 19:54:32,786][105620] Updated weights for policy 1, policy_version 624737 (0.0010) [2023-12-26 19:54:33,062][105692] Updated weights for policy 0, policy_version 623858 (0.0007) [2023-12-26 19:54:33,111][105692] Updated weights for policy 0, policy_version 623868 (0.0005) [2023-12-26 19:54:33,157][105692] Updated weights for policy 0, policy_version 623878 (0.0005) [2023-12-26 19:54:33,407][105620] Updated weights for policy 1, policy_version 624747 (0.0009) [2023-12-26 19:54:33,456][105620] Updated weights for policy 1, policy_version 624757 (0.0005) [2023-12-26 19:54:33,509][105620] Updated weights for policy 1, policy_version 624767 (0.0005) [2023-12-26 19:54:33,717][105692] Updated weights for policy 0, policy_version 623888 (0.0008) [2023-12-26 19:54:33,774][105692] Updated weights for policy 0, policy_version 623898 (0.0009) [2023-12-26 19:54:33,831][105692] Updated weights for policy 0, policy_version 623909 (0.0010) [2023-12-26 19:54:34,042][105620] Updated weights for policy 1, policy_version 624777 (0.0006) [2023-12-26 19:54:34,093][105620] Updated weights for policy 1, policy_version 624787 (0.0010) [2023-12-26 19:54:34,142][105620] Updated weights for policy 1, policy_version 624797 (0.0010) [2023-12-26 19:54:34,215][105620] Updated weights for policy 1, policy_version 624807 (0.0009) [2023-12-26 19:54:34,566][105692] Updated weights for policy 0, policy_version 623919 (0.0007) [2023-12-26 19:54:34,629][105692] Updated weights for policy 0, policy_version 623929 (0.0006) [2023-12-26 19:54:34,697][105692] Updated weights for policy 0, policy_version 623939 (0.0006) [2023-12-26 19:54:34,917][105620] Updated weights for policy 1, policy_version 624817 (0.0010) [2023-12-26 19:54:34,980][105620] Updated weights for policy 1, policy_version 624827 (0.0010) [2023-12-26 19:54:35,049][105620] Updated weights for policy 1, policy_version 624837 (0.0011) [2023-12-26 19:54:35,386][105692] Updated weights for policy 0, policy_version 623949 (0.0007) [2023-12-26 19:54:35,443][105692] Updated weights for policy 0, policy_version 623959 (0.0007) [2023-12-26 19:54:35,495][105692] Updated weights for policy 0, policy_version 623969 (0.0008) [2023-12-26 19:54:35,740][105620] Updated weights for policy 1, policy_version 624847 (0.0010) [2023-12-26 19:54:35,811][105620] Updated weights for policy 1, policy_version 624857 (0.0010) [2023-12-26 19:54:35,876][105620] Updated weights for policy 1, policy_version 624867 (0.0010) [2023-12-26 19:54:36,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 319750144. Throughput: 0: 9798.0, 1: 9814.6. Samples: 319738344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:54:36,062][104569] Avg episode reward: [(0, '9355.373'), (1, '9286.673')] [2023-12-26 19:54:36,191][105692] Updated weights for policy 0, policy_version 623979 (0.0008) [2023-12-26 19:54:36,265][105692] Updated weights for policy 0, policy_version 623989 (0.0011) [2023-12-26 19:54:36,325][105692] Updated weights for policy 0, policy_version 623999 (0.0011) [2023-12-26 19:54:36,499][105620] Updated weights for policy 1, policy_version 624877 (0.0009) [2023-12-26 19:54:36,562][105620] Updated weights for policy 1, policy_version 624887 (0.0008) [2023-12-26 19:54:36,622][105620] Updated weights for policy 1, policy_version 624897 (0.0011) [2023-12-26 19:54:36,954][105692] Updated weights for policy 0, policy_version 624009 (0.0011) [2023-12-26 19:54:37,015][105692] Updated weights for policy 0, policy_version 624019 (0.0007) [2023-12-26 19:54:37,075][105692] Updated weights for policy 0, policy_version 624029 (0.0007) [2023-12-26 19:54:37,143][105692] Updated weights for policy 0, policy_version 624039 (0.0006) [2023-12-26 19:54:37,363][105620] Updated weights for policy 1, policy_version 624907 (0.0011) [2023-12-26 19:54:37,408][105620] Updated weights for policy 1, policy_version 624917 (0.0010) [2023-12-26 19:54:37,459][105620] Updated weights for policy 1, policy_version 624927 (0.0009) [2023-12-26 19:54:37,817][105692] Updated weights for policy 0, policy_version 624049 (0.0005) [2023-12-26 19:54:37,862][105692] Updated weights for policy 0, policy_version 624059 (0.0005) [2023-12-26 19:54:37,915][105692] Updated weights for policy 0, policy_version 624069 (0.0008) [2023-12-26 19:54:38,127][105620] Updated weights for policy 1, policy_version 624937 (0.0006) [2023-12-26 19:54:38,181][105620] Updated weights for policy 1, policy_version 624947 (0.0008) [2023-12-26 19:54:38,233][105620] Updated weights for policy 1, policy_version 624957 (0.0010) [2023-12-26 19:54:38,282][105620] Updated weights for policy 1, policy_version 624967 (0.0010) [2023-12-26 19:54:38,688][105692] Updated weights for policy 0, policy_version 624079 (0.0008) [2023-12-26 19:54:38,754][105692] Updated weights for policy 0, policy_version 624089 (0.0008) [2023-12-26 19:54:38,818][105692] Updated weights for policy 0, policy_version 624099 (0.0007) [2023-12-26 19:54:39,036][105620] Updated weights for policy 1, policy_version 624977 (0.0010) [2023-12-26 19:54:39,084][105620] Updated weights for policy 1, policy_version 624987 (0.0010) [2023-12-26 19:54:39,133][105620] Updated weights for policy 1, policy_version 624997 (0.0010) [2023-12-26 19:54:39,446][105692] Updated weights for policy 0, policy_version 624109 (0.0007) [2023-12-26 19:54:39,497][105692] Updated weights for policy 0, policy_version 624119 (0.0010) [2023-12-26 19:54:39,560][105692] Updated weights for policy 0, policy_version 624129 (0.0009) [2023-12-26 19:54:39,890][105620] Updated weights for policy 1, policy_version 625007 (0.0009) [2023-12-26 19:54:39,956][105620] Updated weights for policy 1, policy_version 625017 (0.0009) [2023-12-26 19:54:40,018][105620] Updated weights for policy 1, policy_version 625027 (0.0008) [2023-12-26 19:54:40,337][105692] Updated weights for policy 0, policy_version 624139 (0.0009) [2023-12-26 19:54:40,398][105692] Updated weights for policy 0, policy_version 624149 (0.0011) [2023-12-26 19:54:40,454][105692] Updated weights for policy 0, policy_version 624159 (0.0011) [2023-12-26 19:54:40,802][105620] Updated weights for policy 1, policy_version 625037 (0.0009) [2023-12-26 19:54:40,851][105620] Updated weights for policy 1, policy_version 625047 (0.0008) [2023-12-26 19:54:40,909][105620] Updated weights for policy 1, policy_version 625057 (0.0007) [2023-12-26 19:54:41,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 319848448. Throughput: 0: 9867.8, 1: 9833.3. Samples: 319854792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:54:41,062][104569] Avg episode reward: [(0, '9355.598'), (1, '4440.637')] [2023-12-26 19:54:41,219][105692] Updated weights for policy 0, policy_version 624169 (0.0011) [2023-12-26 19:54:41,287][105692] Updated weights for policy 0, policy_version 624179 (0.0011) [2023-12-26 19:54:41,360][105692] Updated weights for policy 0, policy_version 624189 (0.0013) [2023-12-26 19:54:41,423][105692] Updated weights for policy 0, policy_version 624199 (0.0010) [2023-12-26 19:54:41,688][105620] Updated weights for policy 1, policy_version 625067 (0.0006) [2023-12-26 19:54:41,760][105620] Updated weights for policy 1, policy_version 625077 (0.0008) [2023-12-26 19:54:41,835][105620] Updated weights for policy 1, policy_version 625087 (0.0006) [2023-12-26 19:54:42,238][105692] Updated weights for policy 0, policy_version 624209 (0.0009) [2023-12-26 19:54:42,301][105692] Updated weights for policy 0, policy_version 624219 (0.0009) [2023-12-26 19:54:42,369][105692] Updated weights for policy 0, policy_version 624229 (0.0009) [2023-12-26 19:54:42,463][105620] Updated weights for policy 1, policy_version 625097 (0.0007) [2023-12-26 19:54:42,522][105620] Updated weights for policy 1, policy_version 625107 (0.0009) [2023-12-26 19:54:42,575][105620] Updated weights for policy 1, policy_version 625117 (0.0009) [2023-12-26 19:54:42,642][105620] Updated weights for policy 1, policy_version 625127 (0.0009) [2023-12-26 19:54:43,101][105692] Updated weights for policy 0, policy_version 624239 (0.0009) [2023-12-26 19:54:43,158][105692] Updated weights for policy 0, policy_version 624249 (0.0009) [2023-12-26 19:54:43,227][105692] Updated weights for policy 0, policy_version 624259 (0.0009) [2023-12-26 19:54:43,378][105620] Updated weights for policy 1, policy_version 625137 (0.0009) [2023-12-26 19:54:43,435][105620] Updated weights for policy 1, policy_version 625147 (0.0008) [2023-12-26 19:54:43,496][105620] Updated weights for policy 1, policy_version 625157 (0.0009) [2023-12-26 19:54:43,956][105692] Updated weights for policy 0, policy_version 624269 (0.0009) [2023-12-26 19:54:44,021][105692] Updated weights for policy 0, policy_version 624279 (0.0010) [2023-12-26 19:54:44,080][105692] Updated weights for policy 0, policy_version 624289 (0.0009) [2023-12-26 19:54:44,247][105620] Updated weights for policy 1, policy_version 625167 (0.0008) [2023-12-26 19:54:44,303][105620] Updated weights for policy 1, policy_version 625177 (0.0009) [2023-12-26 19:54:44,357][105620] Updated weights for policy 1, policy_version 625187 (0.0009) [2023-12-26 19:54:44,878][105692] Updated weights for policy 0, policy_version 624299 (0.0009) [2023-12-26 19:54:44,941][105692] Updated weights for policy 0, policy_version 624309 (0.0009) [2023-12-26 19:54:45,011][105692] Updated weights for policy 0, policy_version 624319 (0.0009) [2023-12-26 19:54:45,140][105620] Updated weights for policy 1, policy_version 625197 (0.0008) [2023-12-26 19:54:45,203][105620] Updated weights for policy 1, policy_version 625207 (0.0008) [2023-12-26 19:54:45,264][105620] Updated weights for policy 1, policy_version 625217 (0.0009) [2023-12-26 19:54:45,794][105692] Updated weights for policy 0, policy_version 624329 (0.0008) [2023-12-26 19:54:45,855][105692] Updated weights for policy 0, policy_version 624339 (0.0009) [2023-12-26 19:54:45,902][105692] Updated weights for policy 0, policy_version 624349 (0.0009) [2023-12-26 19:54:45,950][105692] Updated weights for policy 0, policy_version 624359 (0.0009) [2023-12-26 19:54:45,981][105620] Updated weights for policy 1, policy_version 625227 (0.0009) [2023-12-26 19:54:46,027][105620] Updated weights for policy 1, policy_version 625237 (0.0008) [2023-12-26 19:54:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.8, 300 sec: 19660.8). Total num frames: 319938560. Throughput: 0: 9796.3, 1: 9833.6. Samples: 319910488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:54:46,062][104569] Avg episode reward: [(0, '9355.600'), (1, '6163.292')] [2023-12-26 19:54:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000624360_159866880.pth... [2023-12-26 19:54:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000623208_159571968.pth [2023-12-26 19:54:46,078][105620] Updated weights for policy 1, policy_version 625247 (0.0009) [2023-12-26 19:54:46,121][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000625256_160079872.pth... [2023-12-26 19:54:46,124][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000624072_159776768.pth [2023-12-26 19:54:46,754][105692] Updated weights for policy 0, policy_version 624369 (0.0009) [2023-12-26 19:54:46,811][105692] Updated weights for policy 0, policy_version 624379 (0.0010) [2023-12-26 19:54:46,823][105620] Updated weights for policy 1, policy_version 625257 (0.0009) [2023-12-26 19:54:46,861][105692] Updated weights for policy 0, policy_version 624389 (0.0009) [2023-12-26 19:54:46,871][105620] Updated weights for policy 1, policy_version 625267 (0.0005) [2023-12-26 19:54:46,923][105620] Updated weights for policy 1, policy_version 625277 (0.0006) [2023-12-26 19:54:46,973][105620] Updated weights for policy 1, policy_version 625287 (0.0009) [2023-12-26 19:54:47,557][105620] Updated weights for policy 1, policy_version 625297 (0.0005) [2023-12-26 19:54:47,601][105620] Updated weights for policy 1, policy_version 625307 (0.0006) [2023-12-26 19:54:47,650][105620] Updated weights for policy 1, policy_version 625317 (0.0008) [2023-12-26 19:54:47,727][105692] Updated weights for policy 0, policy_version 624399 (0.0010) [2023-12-26 19:54:47,790][105692] Updated weights for policy 0, policy_version 624409 (0.0010) [2023-12-26 19:54:47,842][105692] Updated weights for policy 0, policy_version 624419 (0.0008) [2023-12-26 19:54:48,245][105620] Updated weights for policy 1, policy_version 625327 (0.0006) [2023-12-26 19:54:48,298][105620] Updated weights for policy 1, policy_version 625337 (0.0005) [2023-12-26 19:54:48,362][105620] Updated weights for policy 1, policy_version 625347 (0.0007) [2023-12-26 19:54:48,638][105692] Updated weights for policy 0, policy_version 624429 (0.0006) [2023-12-26 19:54:48,698][105692] Updated weights for policy 0, policy_version 624439 (0.0006) [2023-12-26 19:54:48,752][105692] Updated weights for policy 0, policy_version 624449 (0.0006) [2023-12-26 19:54:49,064][105620] Updated weights for policy 1, policy_version 625357 (0.0007) [2023-12-26 19:54:49,118][105620] Updated weights for policy 1, policy_version 625367 (0.0005) [2023-12-26 19:54:49,170][105620] Updated weights for policy 1, policy_version 625377 (0.0008) [2023-12-26 19:54:49,399][105692] Updated weights for policy 0, policy_version 624459 (0.0007) [2023-12-26 19:54:49,448][105692] Updated weights for policy 0, policy_version 624469 (0.0010) [2023-12-26 19:54:49,503][105692] Updated weights for policy 0, policy_version 624479 (0.0010) [2023-12-26 19:54:49,942][105620] Updated weights for policy 1, policy_version 625387 (0.0009) [2023-12-26 19:54:50,002][105620] Updated weights for policy 1, policy_version 625397 (0.0009) [2023-12-26 19:54:50,065][105620] Updated weights for policy 1, policy_version 625407 (0.0008) [2023-12-26 19:54:50,166][105692] Updated weights for policy 0, policy_version 624489 (0.0007) [2023-12-26 19:54:50,220][105692] Updated weights for policy 0, policy_version 624499 (0.0007) [2023-12-26 19:54:50,275][105692] Updated weights for policy 0, policy_version 624509 (0.0007) [2023-12-26 19:54:50,333][105692] Updated weights for policy 0, policy_version 624519 (0.0010) [2023-12-26 19:54:50,803][105620] Updated weights for policy 1, policy_version 625417 (0.0006) [2023-12-26 19:54:50,854][105620] Updated weights for policy 1, policy_version 625427 (0.0008) [2023-12-26 19:54:50,906][105620] Updated weights for policy 1, policy_version 625437 (0.0009) [2023-12-26 19:54:50,959][105620] Updated weights for policy 1, policy_version 625447 (0.0011) [2023-12-26 19:54:50,994][105692] Updated weights for policy 0, policy_version 624529 (0.0011) [2023-12-26 19:54:51,058][105692] Updated weights for policy 0, policy_version 624539 (0.0011) [2023-12-26 19:54:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 320036864. Throughput: 0: 9686.1, 1: 9884.3. Samples: 320025508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:54:51,062][104569] Avg episode reward: [(0, '9355.534'), (1, '7940.382')] [2023-12-26 19:54:51,124][105692] Updated weights for policy 0, policy_version 624549 (0.0011) [2023-12-26 19:54:51,767][105620] Updated weights for policy 1, policy_version 625457 (0.0009) [2023-12-26 19:54:51,825][105620] Updated weights for policy 1, policy_version 625467 (0.0008) [2023-12-26 19:54:51,895][105620] Updated weights for policy 1, policy_version 625477 (0.0006) [2023-12-26 19:54:51,908][105692] Updated weights for policy 0, policy_version 624559 (0.0010) [2023-12-26 19:54:51,967][105692] Updated weights for policy 0, policy_version 624569 (0.0008) [2023-12-26 19:54:52,025][105692] Updated weights for policy 0, policy_version 624579 (0.0005) [2023-12-26 19:54:52,603][105620] Updated weights for policy 1, policy_version 625487 (0.0006) [2023-12-26 19:54:52,669][105620] Updated weights for policy 1, policy_version 625497 (0.0007) [2023-12-26 19:54:52,727][105692] Updated weights for policy 0, policy_version 624589 (0.0007) [2023-12-26 19:54:52,732][105620] Updated weights for policy 1, policy_version 625507 (0.0005) [2023-12-26 19:54:52,792][105692] Updated weights for policy 0, policy_version 624599 (0.0009) [2023-12-26 19:54:52,852][105692] Updated weights for policy 0, policy_version 624609 (0.0008) [2023-12-26 19:54:53,341][105620] Updated weights for policy 1, policy_version 625517 (0.0008) [2023-12-26 19:54:53,402][105620] Updated weights for policy 1, policy_version 625527 (0.0008) [2023-12-26 19:54:53,469][105620] Updated weights for policy 1, policy_version 625537 (0.0007) [2023-12-26 19:54:53,545][105692] Updated weights for policy 0, policy_version 624619 (0.0007) [2023-12-26 19:54:53,590][105692] Updated weights for policy 0, policy_version 624629 (0.0005) [2023-12-26 19:54:53,637][105692] Updated weights for policy 0, policy_version 624639 (0.0005) [2023-12-26 19:54:54,035][105620] Updated weights for policy 1, policy_version 625547 (0.0008) [2023-12-26 19:54:54,091][105620] Updated weights for policy 1, policy_version 625558 (0.0009) [2023-12-26 19:54:54,138][105620] Updated weights for policy 1, policy_version 625568 (0.0009) [2023-12-26 19:54:54,267][105692] Updated weights for policy 0, policy_version 624649 (0.0006) [2023-12-26 19:54:54,318][105692] Updated weights for policy 0, policy_version 624659 (0.0008) [2023-12-26 19:54:54,369][105692] Updated weights for policy 0, policy_version 624669 (0.0008) [2023-12-26 19:54:54,419][105692] Updated weights for policy 0, policy_version 624679 (0.0009) [2023-12-26 19:54:54,949][105620] Updated weights for policy 1, policy_version 625578 (0.0009) [2023-12-26 19:54:55,004][105620] Updated weights for policy 1, policy_version 625588 (0.0006) [2023-12-26 19:54:55,064][105620] Updated weights for policy 1, policy_version 625598 (0.0010) [2023-12-26 19:54:55,109][105620] Updated weights for policy 1, policy_version 625608 (0.0010) [2023-12-26 19:54:55,185][105692] Updated weights for policy 0, policy_version 624689 (0.0006) [2023-12-26 19:54:55,248][105692] Updated weights for policy 0, policy_version 624699 (0.0007) [2023-12-26 19:54:55,305][105692] Updated weights for policy 0, policy_version 624709 (0.0006) [2023-12-26 19:54:55,786][105620] Updated weights for policy 1, policy_version 625618 (0.0011) [2023-12-26 19:54:55,842][105620] Updated weights for policy 1, policy_version 625628 (0.0011) [2023-12-26 19:54:55,898][105620] Updated weights for policy 1, policy_version 625638 (0.0011) [2023-12-26 19:54:56,018][105692] Updated weights for policy 0, policy_version 624719 (0.0008) [2023-12-26 19:54:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19660.8). Total num frames: 320135168. Throughput: 0: 9718.5, 1: 9878.6. Samples: 320144252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:54:56,062][104569] Avg episode reward: [(0, '8909.820'), (1, '9267.926')] [2023-12-26 19:54:56,073][105692] Updated weights for policy 0, policy_version 624729 (0.0008) [2023-12-26 19:54:56,135][105692] Updated weights for policy 0, policy_version 624739 (0.0008) [2023-12-26 19:54:56,139][105585] KL-divergence is very high: 100.0996 [2023-12-26 19:54:56,657][105620] Updated weights for policy 1, policy_version 625648 (0.0010) [2023-12-26 19:54:56,714][105620] Updated weights for policy 1, policy_version 625658 (0.0010) [2023-12-26 19:54:56,772][105620] Updated weights for policy 1, policy_version 625668 (0.0010) [2023-12-26 19:54:56,808][105585] KL-divergence is very high: 104.5277 [2023-12-26 19:54:56,863][105692] Updated weights for policy 0, policy_version 624749 (0.0008) [2023-12-26 19:54:56,911][105692] Updated weights for policy 0, policy_version 624759 (0.0008) [2023-12-26 19:54:56,958][105692] Updated weights for policy 0, policy_version 624769 (0.0008) [2023-12-26 19:54:57,497][105620] Updated weights for policy 1, policy_version 625678 (0.0007) [2023-12-26 19:54:57,554][105620] Updated weights for policy 1, policy_version 625688 (0.0009) [2023-12-26 19:54:57,611][105620] Updated weights for policy 1, policy_version 625698 (0.0005) [2023-12-26 19:54:57,730][105692] Updated weights for policy 0, policy_version 624779 (0.0008) [2023-12-26 19:54:57,780][105585] KL-divergence is very high: 282.3947 [2023-12-26 19:54:57,781][105692] Updated weights for policy 0, policy_version 624789 (0.0009) [2023-12-26 19:54:57,824][105585] KL-divergence is very high: 387.7711 [2023-12-26 19:54:57,835][105692] Updated weights for policy 0, policy_version 624799 (0.0009) [2023-12-26 19:54:57,861][105585] KL-divergence is very high: 293.1584 [2023-12-26 19:54:58,279][105620] Updated weights for policy 1, policy_version 625708 (0.0009) [2023-12-26 19:54:58,356][105620] Updated weights for policy 1, policy_version 625718 (0.0009) [2023-12-26 19:54:58,421][105620] Updated weights for policy 1, policy_version 625728 (0.0010) [2023-12-26 19:54:58,624][105692] Updated weights for policy 0, policy_version 624809 (0.0008) [2023-12-26 19:54:58,688][105692] Updated weights for policy 0, policy_version 624819 (0.0008) [2023-12-26 19:54:58,750][105692] Updated weights for policy 0, policy_version 624829 (0.0008) [2023-12-26 19:54:58,815][105692] Updated weights for policy 0, policy_version 624839 (0.0008) [2023-12-26 19:54:59,178][105620] Updated weights for policy 1, policy_version 625738 (0.0010) [2023-12-26 19:54:59,254][105620] Updated weights for policy 1, policy_version 625748 (0.0009) [2023-12-26 19:54:59,319][105620] Updated weights for policy 1, policy_version 625758 (0.0010) [2023-12-26 19:54:59,384][105620] Updated weights for policy 1, policy_version 625768 (0.0009) [2023-12-26 19:54:59,636][105692] Updated weights for policy 0, policy_version 624849 (0.0009) [2023-12-26 19:54:59,694][105692] Updated weights for policy 0, policy_version 624859 (0.0009) [2023-12-26 19:54:59,754][105692] Updated weights for policy 0, policy_version 624869 (0.0009) [2023-12-26 19:55:00,110][105620] Updated weights for policy 1, policy_version 625778 (0.0010) [2023-12-26 19:55:00,164][105620] Updated weights for policy 1, policy_version 625788 (0.0010) [2023-12-26 19:55:00,220][105620] Updated weights for policy 1, policy_version 625798 (0.0009) [2023-12-26 19:55:00,460][105692] Updated weights for policy 0, policy_version 624879 (0.0008) [2023-12-26 19:55:00,519][105692] Updated weights for policy 0, policy_version 624889 (0.0009) [2023-12-26 19:55:00,576][105692] Updated weights for policy 0, policy_version 624899 (0.0007) [2023-12-26 19:55:00,997][105620] Updated weights for policy 1, policy_version 625808 (0.0008) [2023-12-26 19:55:01,061][105620] Updated weights for policy 1, policy_version 625818 (0.0009) [2023-12-26 19:55:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 320225280. Throughput: 0: 9716.5, 1: 9882.9. Samples: 320200796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:55:01,063][104569] Avg episode reward: [(0, '8754.204'), (1, '8999.206')] [2023-12-26 19:55:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000624904_160006144.pth... [2023-12-26 19:55:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000623784_159719424.pth [2023-12-26 19:55:01,130][105620] Updated weights for policy 1, policy_version 625828 (0.0010) [2023-12-26 19:55:01,157][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000625832_160227328.pth... [2023-12-26 19:55:01,161][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000624680_159932416.pth [2023-12-26 19:55:01,265][105692] Updated weights for policy 0, policy_version 624909 (0.0007) [2023-12-26 19:55:01,325][105692] Updated weights for policy 0, policy_version 624919 (0.0010) [2023-12-26 19:55:01,395][105692] Updated weights for policy 0, policy_version 624929 (0.0009) [2023-12-26 19:55:01,793][105620] Updated weights for policy 1, policy_version 625838 (0.0008) [2023-12-26 19:55:01,856][105620] Updated weights for policy 1, policy_version 625848 (0.0006) [2023-12-26 19:55:01,918][105620] Updated weights for policy 1, policy_version 625858 (0.0007) [2023-12-26 19:55:02,050][105692] Updated weights for policy 0, policy_version 624939 (0.0010) [2023-12-26 19:55:02,105][105692] Updated weights for policy 0, policy_version 624949 (0.0010) [2023-12-26 19:55:02,164][105692] Updated weights for policy 0, policy_version 624959 (0.0010) [2023-12-26 19:55:02,626][105620] Updated weights for policy 1, policy_version 625868 (0.0008) [2023-12-26 19:55:02,685][105620] Updated weights for policy 1, policy_version 625878 (0.0008) [2023-12-26 19:55:02,749][105620] Updated weights for policy 1, policy_version 625888 (0.0009) [2023-12-26 19:55:02,801][105692] Updated weights for policy 0, policy_version 624969 (0.0010) [2023-12-26 19:55:02,854][105692] Updated weights for policy 0, policy_version 624979 (0.0008) [2023-12-26 19:55:02,912][105692] Updated weights for policy 0, policy_version 624989 (0.0010) [2023-12-26 19:55:02,969][105692] Updated weights for policy 0, policy_version 624999 (0.0010) [2023-12-26 19:55:03,419][105620] Updated weights for policy 1, policy_version 625898 (0.0008) [2023-12-26 19:55:03,483][105620] Updated weights for policy 1, policy_version 625908 (0.0005) [2023-12-26 19:55:03,536][105620] Updated weights for policy 1, policy_version 625918 (0.0005) [2023-12-26 19:55:03,590][105620] Updated weights for policy 1, policy_version 625928 (0.0006) [2023-12-26 19:55:03,671][105692] Updated weights for policy 0, policy_version 625009 (0.0009) [2023-12-26 19:55:03,733][105692] Updated weights for policy 0, policy_version 625019 (0.0010) [2023-12-26 19:55:03,795][105692] Updated weights for policy 0, policy_version 625029 (0.0010) [2023-12-26 19:55:04,140][105620] Updated weights for policy 1, policy_version 625938 (0.0006) [2023-12-26 19:55:04,194][105620] Updated weights for policy 1, policy_version 625948 (0.0006) [2023-12-26 19:55:04,255][105620] Updated weights for policy 1, policy_version 625958 (0.0006) [2023-12-26 19:55:04,618][105692] Updated weights for policy 0, policy_version 625040 (0.0007) [2023-12-26 19:55:04,663][105692] Updated weights for policy 0, policy_version 625050 (0.0008) [2023-12-26 19:55:04,716][105692] Updated weights for policy 0, policy_version 625060 (0.0008) [2023-12-26 19:55:04,964][105620] Updated weights for policy 1, policy_version 625968 (0.0010) [2023-12-26 19:55:05,018][105620] Updated weights for policy 1, policy_version 625978 (0.0010) [2023-12-26 19:55:05,069][105620] Updated weights for policy 1, policy_version 625988 (0.0010) [2023-12-26 19:55:05,496][105692] Updated weights for policy 0, policy_version 625070 (0.0009) [2023-12-26 19:55:05,550][105692] Updated weights for policy 0, policy_version 625080 (0.0010) [2023-12-26 19:55:05,609][105692] Updated weights for policy 0, policy_version 625090 (0.0010) [2023-12-26 19:55:05,816][105620] Updated weights for policy 1, policy_version 625998 (0.0010) [2023-12-26 19:55:05,878][105620] Updated weights for policy 1, policy_version 626008 (0.0010) [2023-12-26 19:55:05,934][105620] Updated weights for policy 1, policy_version 626018 (0.0009) [2023-12-26 19:55:06,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 320331776. Throughput: 0: 9680.0, 1: 9866.4. Samples: 320318380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:55:06,063][104569] Avg episode reward: [(0, '9262.896'), (1, '8821.316')] [2023-12-26 19:55:06,310][105692] Updated weights for policy 0, policy_version 625100 (0.0010) [2023-12-26 19:55:06,370][105692] Updated weights for policy 0, policy_version 625110 (0.0008) [2023-12-26 19:55:06,439][105692] Updated weights for policy 0, policy_version 625120 (0.0008) [2023-12-26 19:55:06,685][105620] Updated weights for policy 1, policy_version 626028 (0.0008) [2023-12-26 19:55:06,748][105620] Updated weights for policy 1, policy_version 626038 (0.0010) [2023-12-26 19:55:06,811][105620] Updated weights for policy 1, policy_version 626048 (0.0010) [2023-12-26 19:55:07,210][105692] Updated weights for policy 0, policy_version 625130 (0.0008) [2023-12-26 19:55:07,266][105692] Updated weights for policy 0, policy_version 625140 (0.0008) [2023-12-26 19:55:07,314][105692] Updated weights for policy 0, policy_version 625150 (0.0008) [2023-12-26 19:55:07,362][105692] Updated weights for policy 0, policy_version 625160 (0.0008) [2023-12-26 19:55:07,546][105620] Updated weights for policy 1, policy_version 626058 (0.0011) [2023-12-26 19:55:07,597][105620] Updated weights for policy 1, policy_version 626068 (0.0010) [2023-12-26 19:55:07,648][105620] Updated weights for policy 1, policy_version 626078 (0.0010) [2023-12-26 19:55:07,699][105620] Updated weights for policy 1, policy_version 626088 (0.0010) [2023-12-26 19:55:08,133][105692] Updated weights for policy 0, policy_version 625170 (0.0009) [2023-12-26 19:55:08,185][105692] Updated weights for policy 0, policy_version 625180 (0.0008) [2023-12-26 19:55:08,243][105692] Updated weights for policy 0, policy_version 625190 (0.0008) [2023-12-26 19:55:08,479][105620] Updated weights for policy 1, policy_version 626098 (0.0010) [2023-12-26 19:55:08,544][105620] Updated weights for policy 1, policy_version 626108 (0.0010) [2023-12-26 19:55:08,613][105620] Updated weights for policy 1, policy_version 626118 (0.0009) [2023-12-26 19:55:08,902][105692] Updated weights for policy 0, policy_version 625200 (0.0006) [2023-12-26 19:55:08,962][105692] Updated weights for policy 0, policy_version 625210 (0.0006) [2023-12-26 19:55:09,025][105692] Updated weights for policy 0, policy_version 625220 (0.0010) [2023-12-26 19:55:09,350][105620] Updated weights for policy 1, policy_version 626128 (0.0011) [2023-12-26 19:55:09,414][105620] Updated weights for policy 1, policy_version 626138 (0.0009) [2023-12-26 19:55:09,469][105620] Updated weights for policy 1, policy_version 626148 (0.0010) [2023-12-26 19:55:09,688][105692] Updated weights for policy 0, policy_version 625230 (0.0010) [2023-12-26 19:55:09,748][105692] Updated weights for policy 0, policy_version 625240 (0.0010) [2023-12-26 19:55:09,808][105692] Updated weights for policy 0, policy_version 625250 (0.0011) [2023-12-26 19:55:10,295][105620] Updated weights for policy 1, policy_version 626158 (0.0010) [2023-12-26 19:55:10,351][105620] Updated weights for policy 1, policy_version 626168 (0.0010) [2023-12-26 19:55:10,411][105620] Updated weights for policy 1, policy_version 626178 (0.0007) [2023-12-26 19:55:10,522][105692] Updated weights for policy 0, policy_version 625260 (0.0011) [2023-12-26 19:55:10,582][105692] Updated weights for policy 0, policy_version 625270 (0.0008) [2023-12-26 19:55:10,628][105692] Updated weights for policy 0, policy_version 625280 (0.0005) [2023-12-26 19:55:11,014][105620] Updated weights for policy 1, policy_version 626188 (0.0008) [2023-12-26 19:55:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 320421888. Throughput: 0: 9687.4, 1: 9883.5. Samples: 320433764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:55:11,062][104569] Avg episode reward: [(0, '9262.895'), (1, '8910.525')] [2023-12-26 19:55:11,079][105620] Updated weights for policy 1, policy_version 626198 (0.0010) [2023-12-26 19:55:11,144][105620] Updated weights for policy 1, policy_version 626208 (0.0009) [2023-12-26 19:55:11,249][105692] Updated weights for policy 0, policy_version 625290 (0.0006) [2023-12-26 19:55:11,323][105692] Updated weights for policy 0, policy_version 625300 (0.0007) [2023-12-26 19:55:11,395][105692] Updated weights for policy 0, policy_version 625310 (0.0008) [2023-12-26 19:55:11,459][105692] Updated weights for policy 0, policy_version 625320 (0.0005) [2023-12-26 19:55:11,864][105620] Updated weights for policy 1, policy_version 626218 (0.0006) [2023-12-26 19:55:11,923][105620] Updated weights for policy 1, policy_version 626228 (0.0008) [2023-12-26 19:55:11,977][105620] Updated weights for policy 1, policy_version 626238 (0.0009) [2023-12-26 19:55:12,037][105620] Updated weights for policy 1, policy_version 626248 (0.0008) [2023-12-26 19:55:12,143][105692] Updated weights for policy 0, policy_version 625330 (0.0011) [2023-12-26 19:55:12,203][105692] Updated weights for policy 0, policy_version 625340 (0.0011) [2023-12-26 19:55:12,273][105692] Updated weights for policy 0, policy_version 625350 (0.0011) [2023-12-26 19:55:12,816][105620] Updated weights for policy 1, policy_version 626258 (0.0008) [2023-12-26 19:55:12,861][105620] Updated weights for policy 1, policy_version 626268 (0.0008) [2023-12-26 19:55:12,909][105620] Updated weights for policy 1, policy_version 626278 (0.0008) [2023-12-26 19:55:13,025][105692] Updated weights for policy 0, policy_version 625360 (0.0010) [2023-12-26 19:55:13,073][105692] Updated weights for policy 0, policy_version 625370 (0.0010) [2023-12-26 19:55:13,121][105692] Updated weights for policy 0, policy_version 625380 (0.0010) [2023-12-26 19:55:13,688][105620] Updated weights for policy 1, policy_version 626288 (0.0008) [2023-12-26 19:55:13,736][105620] Updated weights for policy 1, policy_version 626298 (0.0008) [2023-12-26 19:55:13,784][105620] Updated weights for policy 1, policy_version 626308 (0.0008) [2023-12-26 19:55:13,889][105692] Updated weights for policy 0, policy_version 625390 (0.0010) [2023-12-26 19:55:13,960][105692] Updated weights for policy 0, policy_version 625400 (0.0010) [2023-12-26 19:55:14,025][105692] Updated weights for policy 0, policy_version 625410 (0.0010) [2023-12-26 19:55:14,410][105620] Updated weights for policy 1, policy_version 626318 (0.0007) [2023-12-26 19:55:14,475][105620] Updated weights for policy 1, policy_version 626328 (0.0007) [2023-12-26 19:55:14,536][105620] Updated weights for policy 1, policy_version 626338 (0.0008) [2023-12-26 19:55:14,737][105692] Updated weights for policy 0, policy_version 625420 (0.0011) [2023-12-26 19:55:14,803][105692] Updated weights for policy 0, policy_version 625430 (0.0009) [2023-12-26 19:55:14,870][105692] Updated weights for policy 0, policy_version 625440 (0.0010) [2023-12-26 19:55:15,250][105620] Updated weights for policy 1, policy_version 626348 (0.0007) [2023-12-26 19:55:15,321][105620] Updated weights for policy 1, policy_version 626358 (0.0006) [2023-12-26 19:55:15,385][105620] Updated weights for policy 1, policy_version 626368 (0.0009) [2023-12-26 19:55:15,588][105692] Updated weights for policy 0, policy_version 625450 (0.0009) [2023-12-26 19:55:15,652][105692] Updated weights for policy 0, policy_version 625460 (0.0011) [2023-12-26 19:55:15,719][105692] Updated weights for policy 0, policy_version 625470 (0.0011) [2023-12-26 19:55:15,782][105692] Updated weights for policy 0, policy_version 625480 (0.0010) [2023-12-26 19:55:16,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 320520192. Throughput: 0: 9663.4, 1: 9803.3. Samples: 320490840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:55:16,062][104569] Avg episode reward: [(0, '9090.323'), (1, '9173.555')] [2023-12-26 19:55:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000625480_160153600.pth... [2023-12-26 19:55:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000626376_160366592.pth... [2023-12-26 19:55:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000624360_159866880.pth [2023-12-26 19:55:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000625256_160079872.pth [2023-12-26 19:55:16,104][105620] Updated weights for policy 1, policy_version 626378 (0.0011) [2023-12-26 19:55:16,162][105620] Updated weights for policy 1, policy_version 626388 (0.0010) [2023-12-26 19:55:16,231][105620] Updated weights for policy 1, policy_version 626398 (0.0010) [2023-12-26 19:55:16,289][105620] Updated weights for policy 1, policy_version 626408 (0.0010) [2023-12-26 19:55:16,462][105692] Updated weights for policy 0, policy_version 625490 (0.0010) [2023-12-26 19:55:16,515][105692] Updated weights for policy 0, policy_version 625500 (0.0010) [2023-12-26 19:55:16,569][105692] Updated weights for policy 0, policy_version 625511 (0.0009) [2023-12-26 19:55:16,857][105620] Updated weights for policy 1, policy_version 626418 (0.0010) [2023-12-26 19:55:16,915][105620] Updated weights for policy 1, policy_version 626428 (0.0010) [2023-12-26 19:55:16,973][105620] Updated weights for policy 1, policy_version 626438 (0.0010) [2023-12-26 19:55:17,391][105692] Updated weights for policy 0, policy_version 625521 (0.0008) [2023-12-26 19:55:17,435][105692] Updated weights for policy 0, policy_version 625531 (0.0008) [2023-12-26 19:55:17,486][105692] Updated weights for policy 0, policy_version 625541 (0.0006) [2023-12-26 19:55:17,704][105620] Updated weights for policy 1, policy_version 626448 (0.0011) [2023-12-26 19:55:17,770][105620] Updated weights for policy 1, policy_version 626458 (0.0009) [2023-12-26 19:55:17,831][105620] Updated weights for policy 1, policy_version 626468 (0.0009) [2023-12-26 19:55:18,222][105692] Updated weights for policy 0, policy_version 625551 (0.0005) [2023-12-26 19:55:18,270][105692] Updated weights for policy 0, policy_version 625561 (0.0005) [2023-12-26 19:55:18,323][105692] Updated weights for policy 0, policy_version 625571 (0.0006) [2023-12-26 19:55:18,580][105620] Updated weights for policy 1, policy_version 626478 (0.0008) [2023-12-26 19:55:18,641][105620] Updated weights for policy 1, policy_version 626488 (0.0009) [2023-12-26 19:55:18,707][105620] Updated weights for policy 1, policy_version 626498 (0.0009) [2023-12-26 19:55:19,034][105692] Updated weights for policy 0, policy_version 625581 (0.0009) [2023-12-26 19:55:19,087][105692] Updated weights for policy 0, policy_version 625591 (0.0008) [2023-12-26 19:55:19,134][105692] Updated weights for policy 0, policy_version 625601 (0.0009) [2023-12-26 19:55:19,423][105620] Updated weights for policy 1, policy_version 626508 (0.0009) [2023-12-26 19:55:19,487][105620] Updated weights for policy 1, policy_version 626518 (0.0009) [2023-12-26 19:55:19,546][105620] Updated weights for policy 1, policy_version 626528 (0.0007) [2023-12-26 19:55:19,929][105692] Updated weights for policy 0, policy_version 625611 (0.0008) [2023-12-26 19:55:19,992][105692] Updated weights for policy 0, policy_version 625621 (0.0011) [2023-12-26 19:55:20,056][105692] Updated weights for policy 0, policy_version 625631 (0.0011) [2023-12-26 19:55:20,321][105620] Updated weights for policy 1, policy_version 626538 (0.0008) [2023-12-26 19:55:20,384][105620] Updated weights for policy 1, policy_version 626548 (0.0009) [2023-12-26 19:55:20,448][105620] Updated weights for policy 1, policy_version 626558 (0.0008) [2023-12-26 19:55:20,511][105620] Updated weights for policy 1, policy_version 626568 (0.0008) [2023-12-26 19:55:20,733][105692] Updated weights for policy 0, policy_version 625641 (0.0010) [2023-12-26 19:55:20,804][105692] Updated weights for policy 0, policy_version 625651 (0.0006) [2023-12-26 19:55:20,865][105692] Updated weights for policy 0, policy_version 625661 (0.0008) [2023-12-26 19:55:20,921][105692] Updated weights for policy 0, policy_version 625671 (0.0006) [2023-12-26 19:55:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 320618496. Throughput: 0: 9567.4, 1: 9737.5. Samples: 320607068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:55:21,063][104569] Avg episode reward: [(0, '9182.270'), (1, '9172.943')] [2023-12-26 19:55:21,329][105620] Updated weights for policy 1, policy_version 626578 (0.0009) [2023-12-26 19:55:21,397][105620] Updated weights for policy 1, policy_version 626588 (0.0008) [2023-12-26 19:55:21,462][105620] Updated weights for policy 1, policy_version 626598 (0.0009) [2023-12-26 19:55:21,535][105692] Updated weights for policy 0, policy_version 625681 (0.0005) [2023-12-26 19:55:21,599][105692] Updated weights for policy 0, policy_version 625691 (0.0005) [2023-12-26 19:55:21,667][105692] Updated weights for policy 0, policy_version 625701 (0.0008) [2023-12-26 19:55:22,267][105620] Updated weights for policy 1, policy_version 626608 (0.0008) [2023-12-26 19:55:22,333][105620] Updated weights for policy 1, policy_version 626618 (0.0009) [2023-12-26 19:55:22,363][105692] Updated weights for policy 0, policy_version 625711 (0.0009) [2023-12-26 19:55:22,402][105620] Updated weights for policy 1, policy_version 626628 (0.0009) [2023-12-26 19:55:22,435][105692] Updated weights for policy 0, policy_version 625721 (0.0008) [2023-12-26 19:55:22,499][105692] Updated weights for policy 0, policy_version 625731 (0.0009) [2023-12-26 19:55:23,101][105620] Updated weights for policy 1, policy_version 626638 (0.0008) [2023-12-26 19:55:23,148][105620] Updated weights for policy 1, policy_version 626648 (0.0008) [2023-12-26 19:55:23,198][105620] Updated weights for policy 1, policy_version 626658 (0.0009) [2023-12-26 19:55:23,267][105692] Updated weights for policy 0, policy_version 625741 (0.0009) [2023-12-26 19:55:23,329][105692] Updated weights for policy 0, policy_version 625751 (0.0010) [2023-12-26 19:55:23,369][105585] KL-divergence is very high: 112.5680 [2023-12-26 19:55:23,392][105692] Updated weights for policy 0, policy_version 625761 (0.0010) [2023-12-26 19:55:23,416][105585] KL-divergence is very high: 105.6139 [2023-12-26 19:55:23,944][105620] Updated weights for policy 1, policy_version 626668 (0.0009) [2023-12-26 19:55:24,013][105620] Updated weights for policy 1, policy_version 626678 (0.0009) [2023-12-26 19:55:24,039][105692] Updated weights for policy 0, policy_version 625771 (0.0009) [2023-12-26 19:55:24,067][105620] Updated weights for policy 1, policy_version 626688 (0.0010) [2023-12-26 19:55:24,104][105692] Updated weights for policy 0, policy_version 625781 (0.0005) [2023-12-26 19:55:24,171][105692] Updated weights for policy 0, policy_version 625791 (0.0006) [2023-12-26 19:55:24,761][105692] Updated weights for policy 0, policy_version 625801 (0.0006) [2023-12-26 19:55:24,815][105692] Updated weights for policy 0, policy_version 625811 (0.0005) [2023-12-26 19:55:24,861][105692] Updated weights for policy 0, policy_version 625821 (0.0005) [2023-12-26 19:55:24,885][105620] Updated weights for policy 1, policy_version 626698 (0.0009) [2023-12-26 19:55:24,907][105692] Updated weights for policy 0, policy_version 625831 (0.0005) [2023-12-26 19:55:24,942][105620] Updated weights for policy 1, policy_version 626708 (0.0008) [2023-12-26 19:55:25,001][105620] Updated weights for policy 1, policy_version 626718 (0.0008) [2023-12-26 19:55:25,057][105620] Updated weights for policy 1, policy_version 626728 (0.0008) [2023-12-26 19:55:25,574][105692] Updated weights for policy 0, policy_version 625841 (0.0010) [2023-12-26 19:55:25,627][105692] Updated weights for policy 0, policy_version 625851 (0.0007) [2023-12-26 19:55:25,677][105692] Updated weights for policy 0, policy_version 625861 (0.0005) [2023-12-26 19:55:25,841][105620] Updated weights for policy 1, policy_version 626738 (0.0008) [2023-12-26 19:55:25,900][105620] Updated weights for policy 1, policy_version 626748 (0.0008) [2023-12-26 19:55:25,968][105620] Updated weights for policy 1, policy_version 626758 (0.0008) [2023-12-26 19:55:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 320716800. Throughput: 0: 9616.6, 1: 9656.7. Samples: 320722092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:55:26,062][104569] Avg episode reward: [(0, '9172.324'), (1, '9265.051')] [2023-12-26 19:55:26,381][105692] Updated weights for policy 0, policy_version 625871 (0.0009) [2023-12-26 19:55:26,437][105692] Updated weights for policy 0, policy_version 625881 (0.0011) [2023-12-26 19:55:26,496][105692] Updated weights for policy 0, policy_version 625891 (0.0010) [2023-12-26 19:55:26,757][105620] Updated weights for policy 1, policy_version 626769 (0.0010) [2023-12-26 19:55:26,809][105620] Updated weights for policy 1, policy_version 626779 (0.0009) [2023-12-26 19:55:26,862][105620] Updated weights for policy 1, policy_version 626789 (0.0010) [2023-12-26 19:55:27,061][105692] Updated weights for policy 0, policy_version 625901 (0.0007) [2023-12-26 19:55:27,121][105692] Updated weights for policy 0, policy_version 625911 (0.0006) [2023-12-26 19:55:27,181][105692] Updated weights for policy 0, policy_version 625921 (0.0006) [2023-12-26 19:55:27,704][105620] Updated weights for policy 1, policy_version 626799 (0.0009) [2023-12-26 19:55:27,756][105620] Updated weights for policy 1, policy_version 626809 (0.0008) [2023-12-26 19:55:27,814][105620] Updated weights for policy 1, policy_version 626819 (0.0008) [2023-12-26 19:55:27,817][105692] Updated weights for policy 0, policy_version 625931 (0.0007) [2023-12-26 19:55:27,865][105692] Updated weights for policy 0, policy_version 625941 (0.0010) [2023-12-26 19:55:27,912][105692] Updated weights for policy 0, policy_version 625951 (0.0010) [2023-12-26 19:55:28,574][105620] Updated weights for policy 1, policy_version 626829 (0.0008) [2023-12-26 19:55:28,626][105620] Updated weights for policy 1, policy_version 626839 (0.0008) [2023-12-26 19:55:28,666][105692] Updated weights for policy 0, policy_version 625961 (0.0010) [2023-12-26 19:55:28,680][105620] Updated weights for policy 1, policy_version 626849 (0.0007) [2023-12-26 19:55:28,725][105692] Updated weights for policy 0, policy_version 625971 (0.0011) [2023-12-26 19:55:28,782][105692] Updated weights for policy 0, policy_version 625981 (0.0010) [2023-12-26 19:55:28,843][105692] Updated weights for policy 0, policy_version 625991 (0.0010) [2023-12-26 19:55:29,418][105620] Updated weights for policy 1, policy_version 626859 (0.0007) [2023-12-26 19:55:29,468][105620] Updated weights for policy 1, policy_version 626869 (0.0008) [2023-12-26 19:55:29,493][105692] Updated weights for policy 0, policy_version 626001 (0.0011) [2023-12-26 19:55:29,521][105620] Updated weights for policy 1, policy_version 626879 (0.0008) [2023-12-26 19:55:29,555][105692] Updated weights for policy 0, policy_version 626011 (0.0011) [2023-12-26 19:55:29,620][105692] Updated weights for policy 0, policy_version 626021 (0.0010) [2023-12-26 19:55:30,147][105620] Updated weights for policy 1, policy_version 626889 (0.0006) [2023-12-26 19:55:30,211][105620] Updated weights for policy 1, policy_version 626899 (0.0006) [2023-12-26 19:55:30,270][105620] Updated weights for policy 1, policy_version 626909 (0.0010) [2023-12-26 19:55:30,329][105620] Updated weights for policy 1, policy_version 626919 (0.0010) [2023-12-26 19:55:30,364][105692] Updated weights for policy 0, policy_version 626031 (0.0010) [2023-12-26 19:55:30,423][105692] Updated weights for policy 0, policy_version 626041 (0.0010) [2023-12-26 19:55:30,491][105692] Updated weights for policy 0, policy_version 626051 (0.0010) [2023-12-26 19:55:30,952][105620] Updated weights for policy 1, policy_version 626929 (0.0010) [2023-12-26 19:55:31,001][105620] Updated weights for policy 1, policy_version 626939 (0.0010) [2023-12-26 19:55:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 320806912. Throughput: 0: 9713.4, 1: 9613.2. Samples: 320780188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:55:31,063][104569] Avg episode reward: [(0, '9180.101'), (1, '9265.110')] [2023-12-26 19:55:31,063][105620] Updated weights for policy 1, policy_version 626949 (0.0008) [2023-12-26 19:55:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000626056_160301056.pth... [2023-12-26 19:55:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000624904_160006144.pth [2023-12-26 19:55:31,079][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000626952_160514048.pth... [2023-12-26 19:55:31,082][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000625832_160227328.pth [2023-12-26 19:55:31,194][105692] Updated weights for policy 0, policy_version 626061 (0.0010) [2023-12-26 19:55:31,247][105692] Updated weights for policy 0, policy_version 626071 (0.0010) [2023-12-26 19:55:31,309][105692] Updated weights for policy 0, policy_version 626081 (0.0011) [2023-12-26 19:55:31,752][105620] Updated weights for policy 1, policy_version 626959 (0.0007) [2023-12-26 19:55:31,805][105620] Updated weights for policy 1, policy_version 626969 (0.0008) [2023-12-26 19:55:31,862][105620] Updated weights for policy 1, policy_version 626979 (0.0008) [2023-12-26 19:55:32,074][105692] Updated weights for policy 0, policy_version 626091 (0.0010) [2023-12-26 19:55:32,122][105692] Updated weights for policy 0, policy_version 626101 (0.0010) [2023-12-26 19:55:32,174][105692] Updated weights for policy 0, policy_version 626111 (0.0010) [2023-12-26 19:55:32,490][105620] Updated weights for policy 1, policy_version 626989 (0.0010) [2023-12-26 19:55:32,546][105620] Updated weights for policy 1, policy_version 626999 (0.0007) [2023-12-26 19:55:32,605][105620] Updated weights for policy 1, policy_version 627009 (0.0006) [2023-12-26 19:55:32,972][105692] Updated weights for policy 0, policy_version 626121 (0.0010) [2023-12-26 19:55:33,021][105692] Updated weights for policy 0, policy_version 626131 (0.0008) [2023-12-26 19:55:33,080][105692] Updated weights for policy 0, policy_version 626141 (0.0008) [2023-12-26 19:55:33,135][105692] Updated weights for policy 0, policy_version 626151 (0.0008) [2023-12-26 19:55:33,281][105620] Updated weights for policy 1, policy_version 627019 (0.0005) [2023-12-26 19:55:33,331][105620] Updated weights for policy 1, policy_version 627029 (0.0009) [2023-12-26 19:55:33,389][105620] Updated weights for policy 1, policy_version 627039 (0.0010) [2023-12-26 19:55:33,953][105620] Updated weights for policy 1, policy_version 627049 (0.0008) [2023-12-26 19:55:33,962][105692] Updated weights for policy 0, policy_version 626161 (0.0008) [2023-12-26 19:55:34,000][105620] Updated weights for policy 1, policy_version 627059 (0.0010) [2023-12-26 19:55:34,007][105692] Updated weights for policy 0, policy_version 626171 (0.0005) [2023-12-26 19:55:34,046][105620] Updated weights for policy 1, policy_version 627069 (0.0007) [2023-12-26 19:55:34,069][105692] Updated weights for policy 0, policy_version 626181 (0.0006) [2023-12-26 19:55:34,099][105620] Updated weights for policy 1, policy_version 627079 (0.0005) [2023-12-26 19:55:34,724][105620] Updated weights for policy 1, policy_version 627089 (0.0005) [2023-12-26 19:55:34,785][105620] Updated weights for policy 1, policy_version 627099 (0.0008) [2023-12-26 19:55:34,843][105620] Updated weights for policy 1, policy_version 627109 (0.0010) [2023-12-26 19:55:34,908][105692] Updated weights for policy 0, policy_version 626191 (0.0007) [2023-12-26 19:55:34,956][105692] Updated weights for policy 0, policy_version 626201 (0.0008) [2023-12-26 19:55:35,009][105692] Updated weights for policy 0, policy_version 626212 (0.0009) [2023-12-26 19:55:35,416][105620] Updated weights for policy 1, policy_version 627119 (0.0007) [2023-12-26 19:55:35,484][105620] Updated weights for policy 1, policy_version 627129 (0.0005) [2023-12-26 19:55:35,540][105620] Updated weights for policy 1, policy_version 627139 (0.0005) [2023-12-26 19:55:35,649][105692] Updated weights for policy 0, policy_version 626222 (0.0007) [2023-12-26 19:55:35,705][105692] Updated weights for policy 0, policy_version 626232 (0.0005) [2023-12-26 19:55:35,751][105692] Updated weights for policy 0, policy_version 626242 (0.0006) [2023-12-26 19:55:36,041][105620] Updated weights for policy 1, policy_version 627149 (0.0006) [2023-12-26 19:55:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 320913408. Throughput: 0: 9747.3, 1: 9704.8. Samples: 320900852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:55:36,063][104569] Avg episode reward: [(0, '9088.805'), (1, '9266.043')] [2023-12-26 19:55:36,095][105620] Updated weights for policy 1, policy_version 627159 (0.0005) [2023-12-26 19:55:36,158][105620] Updated weights for policy 1, policy_version 627169 (0.0007) [2023-12-26 19:55:36,373][105692] Updated weights for policy 0, policy_version 626252 (0.0006) [2023-12-26 19:55:36,429][105692] Updated weights for policy 0, policy_version 626262 (0.0008) [2023-12-26 19:55:36,487][105692] Updated weights for policy 0, policy_version 626272 (0.0008) [2023-12-26 19:55:36,809][105620] Updated weights for policy 1, policy_version 627179 (0.0007) [2023-12-26 19:55:36,878][105620] Updated weights for policy 1, policy_version 627189 (0.0010) [2023-12-26 19:55:36,942][105620] Updated weights for policy 1, policy_version 627199 (0.0010) [2023-12-26 19:55:37,148][105692] Updated weights for policy 0, policy_version 626282 (0.0006) [2023-12-26 19:55:37,196][105692] Updated weights for policy 0, policy_version 626292 (0.0008) [2023-12-26 19:55:37,242][105692] Updated weights for policy 0, policy_version 626302 (0.0006) [2023-12-26 19:55:37,294][105692] Updated weights for policy 0, policy_version 626312 (0.0007) [2023-12-26 19:55:37,674][105620] Updated weights for policy 1, policy_version 627209 (0.0010) [2023-12-26 19:55:37,729][105620] Updated weights for policy 1, policy_version 627219 (0.0010) [2023-12-26 19:55:37,784][105620] Updated weights for policy 1, policy_version 627229 (0.0010) [2023-12-26 19:55:37,829][105620] Updated weights for policy 1, policy_version 627239 (0.0010) [2023-12-26 19:55:38,055][105692] Updated weights for policy 0, policy_version 626322 (0.0008) [2023-12-26 19:55:38,100][105692] Updated weights for policy 0, policy_version 626332 (0.0009) [2023-12-26 19:55:38,148][105692] Updated weights for policy 0, policy_version 626342 (0.0009) [2023-12-26 19:55:38,539][105620] Updated weights for policy 1, policy_version 627249 (0.0006) [2023-12-26 19:55:38,597][105620] Updated weights for policy 1, policy_version 627259 (0.0010) [2023-12-26 19:55:38,663][105620] Updated weights for policy 1, policy_version 627269 (0.0008) [2023-12-26 19:55:39,018][105692] Updated weights for policy 0, policy_version 626352 (0.0007) [2023-12-26 19:55:39,083][105692] Updated weights for policy 0, policy_version 626363 (0.0008) [2023-12-26 19:55:39,151][105692] Updated weights for policy 0, policy_version 626373 (0.0005) [2023-12-26 19:55:39,295][105620] Updated weights for policy 1, policy_version 627279 (0.0011) [2023-12-26 19:55:39,360][105620] Updated weights for policy 1, policy_version 627289 (0.0010) [2023-12-26 19:55:39,434][105620] Updated weights for policy 1, policy_version 627299 (0.0010) [2023-12-26 19:55:39,792][105692] Updated weights for policy 0, policy_version 626383 (0.0005) [2023-12-26 19:55:39,858][105692] Updated weights for policy 0, policy_version 626393 (0.0008) [2023-12-26 19:55:39,911][105692] Updated weights for policy 0, policy_version 626403 (0.0010) [2023-12-26 19:55:40,095][105620] Updated weights for policy 1, policy_version 627309 (0.0010) [2023-12-26 19:55:40,148][105620] Updated weights for policy 1, policy_version 627319 (0.0008) [2023-12-26 19:55:40,210][105620] Updated weights for policy 1, policy_version 627329 (0.0005) [2023-12-26 19:55:40,621][105692] Updated weights for policy 0, policy_version 626413 (0.0007) [2023-12-26 19:55:40,692][105692] Updated weights for policy 0, policy_version 626423 (0.0006) [2023-12-26 19:55:40,750][105692] Updated weights for policy 0, policy_version 626433 (0.0005) [2023-12-26 19:55:40,975][105620] Updated weights for policy 1, policy_version 627339 (0.0007) [2023-12-26 19:55:41,029][105620] Updated weights for policy 1, policy_version 627349 (0.0008) [2023-12-26 19:55:41,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 321011712. Throughput: 0: 9745.4, 1: 9778.5. Samples: 321022828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:55:41,062][104569] Avg episode reward: [(0, '9264.141'), (1, '8719.350')] [2023-12-26 19:55:41,090][105620] Updated weights for policy 1, policy_version 627359 (0.0009) [2023-12-26 19:55:41,452][105692] Updated weights for policy 0, policy_version 626443 (0.0006) [2023-12-26 19:55:41,518][105692] Updated weights for policy 0, policy_version 626453 (0.0009) [2023-12-26 19:55:41,577][105692] Updated weights for policy 0, policy_version 626463 (0.0009) [2023-12-26 19:55:41,872][105620] Updated weights for policy 1, policy_version 627369 (0.0009) [2023-12-26 19:55:41,942][105620] Updated weights for policy 1, policy_version 627379 (0.0008) [2023-12-26 19:55:41,996][105620] Updated weights for policy 1, policy_version 627389 (0.0007) [2023-12-26 19:55:42,043][105620] Updated weights for policy 1, policy_version 627399 (0.0009) [2023-12-26 19:55:42,337][105692] Updated weights for policy 0, policy_version 626473 (0.0010) [2023-12-26 19:55:42,402][105692] Updated weights for policy 0, policy_version 626483 (0.0011) [2023-12-26 19:55:42,455][105692] Updated weights for policy 0, policy_version 626493 (0.0011) [2023-12-26 19:55:42,509][105692] Updated weights for policy 0, policy_version 626503 (0.0007) [2023-12-26 19:55:42,694][105620] Updated weights for policy 1, policy_version 627409 (0.0007) [2023-12-26 19:55:42,754][105620] Updated weights for policy 1, policy_version 627419 (0.0008) [2023-12-26 19:55:42,820][105620] Updated weights for policy 1, policy_version 627429 (0.0009) [2023-12-26 19:55:43,169][105692] Updated weights for policy 0, policy_version 626513 (0.0006) [2023-12-26 19:55:43,215][105692] Updated weights for policy 0, policy_version 626523 (0.0005) [2023-12-26 19:55:43,263][105692] Updated weights for policy 0, policy_version 626533 (0.0005) [2023-12-26 19:55:43,457][105620] Updated weights for policy 1, policy_version 627439 (0.0009) [2023-12-26 19:55:43,515][105620] Updated weights for policy 1, policy_version 627449 (0.0009) [2023-12-26 19:55:43,566][105620] Updated weights for policy 1, policy_version 627459 (0.0008) [2023-12-26 19:55:43,817][105692] Updated weights for policy 0, policy_version 626543 (0.0005) [2023-12-26 19:55:43,865][105692] Updated weights for policy 0, policy_version 626553 (0.0006) [2023-12-26 19:55:43,926][105692] Updated weights for policy 0, policy_version 626563 (0.0005) [2023-12-26 19:55:44,207][105620] Updated weights for policy 1, policy_version 627470 (0.0009) [2023-12-26 19:55:44,252][105620] Updated weights for policy 1, policy_version 627480 (0.0008) [2023-12-26 19:55:44,300][105620] Updated weights for policy 1, policy_version 627490 (0.0008) [2023-12-26 19:55:44,470][105692] Updated weights for policy 0, policy_version 626573 (0.0005) [2023-12-26 19:55:44,525][105692] Updated weights for policy 0, policy_version 626583 (0.0005) [2023-12-26 19:55:44,581][105692] Updated weights for policy 0, policy_version 626593 (0.0009) [2023-12-26 19:55:45,106][105620] Updated weights for policy 1, policy_version 627501 (0.0010) [2023-12-26 19:55:45,169][105620] Updated weights for policy 1, policy_version 627511 (0.0011) [2023-12-26 19:55:45,221][105620] Updated weights for policy 1, policy_version 627521 (0.0010) [2023-12-26 19:55:45,265][105692] Updated weights for policy 0, policy_version 626603 (0.0010) [2023-12-26 19:55:45,328][105692] Updated weights for policy 0, policy_version 626613 (0.0011) [2023-12-26 19:55:45,354][105585] KL-divergence is very high: 122.3117 [2023-12-26 19:55:45,392][105692] Updated weights for policy 0, policy_version 626623 (0.0011) [2023-12-26 19:55:45,402][105585] KL-divergence is very high: 161.1376 [2023-12-26 19:55:46,010][105620] Updated weights for policy 1, policy_version 627531 (0.0010) [2023-12-26 19:55:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 321110016. Throughput: 0: 9793.9, 1: 9821.1. Samples: 321083472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:55:46,063][104569] Avg episode reward: [(0, '9101.960'), (1, '8720.709')] [2023-12-26 19:55:46,068][105620] Updated weights for policy 1, policy_version 627541 (0.0007) [2023-12-26 19:55:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000626632_160448512.pth... [2023-12-26 19:55:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000625480_160153600.pth [2023-12-26 19:55:46,132][105620] Updated weights for policy 1, policy_version 627551 (0.0008) [2023-12-26 19:55:46,190][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000627560_160669696.pth... [2023-12-26 19:55:46,192][105692] Updated weights for policy 0, policy_version 626633 (0.0011) [2023-12-26 19:55:46,196][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000626376_160366592.pth [2023-12-26 19:55:46,256][105692] Updated weights for policy 0, policy_version 626643 (0.0011) [2023-12-26 19:55:46,320][105692] Updated weights for policy 0, policy_version 626653 (0.0011) [2023-12-26 19:55:46,383][105692] Updated weights for policy 0, policy_version 626663 (0.0011) [2023-12-26 19:55:46,867][105620] Updated weights for policy 1, policy_version 627561 (0.0011) [2023-12-26 19:55:46,930][105620] Updated weights for policy 1, policy_version 627571 (0.0010) [2023-12-26 19:55:46,988][105620] Updated weights for policy 1, policy_version 627581 (0.0010) [2023-12-26 19:55:47,041][105620] Updated weights for policy 1, policy_version 627591 (0.0011) [2023-12-26 19:55:47,122][105692] Updated weights for policy 0, policy_version 626673 (0.0010) [2023-12-26 19:55:47,180][105692] Updated weights for policy 0, policy_version 626683 (0.0010) [2023-12-26 19:55:47,238][105692] Updated weights for policy 0, policy_version 626693 (0.0010) [2023-12-26 19:55:47,777][105620] Updated weights for policy 1, policy_version 627601 (0.0011) [2023-12-26 19:55:47,842][105620] Updated weights for policy 1, policy_version 627611 (0.0010) [2023-12-26 19:55:47,900][105620] Updated weights for policy 1, policy_version 627621 (0.0010) [2023-12-26 19:55:47,907][105692] Updated weights for policy 0, policy_version 626703 (0.0008) [2023-12-26 19:55:47,950][105692] Updated weights for policy 0, policy_version 626713 (0.0008) [2023-12-26 19:55:47,995][105692] Updated weights for policy 0, policy_version 626723 (0.0007) [2023-12-26 19:55:48,660][105620] Updated weights for policy 1, policy_version 627631 (0.0009) [2023-12-26 19:55:48,715][105692] Updated weights for policy 0, policy_version 626733 (0.0007) [2023-12-26 19:55:48,726][105620] Updated weights for policy 1, policy_version 627641 (0.0010) [2023-12-26 19:55:48,773][105692] Updated weights for policy 0, policy_version 626743 (0.0007) [2023-12-26 19:55:48,791][105620] Updated weights for policy 1, policy_version 627651 (0.0010) [2023-12-26 19:55:48,822][105692] Updated weights for policy 0, policy_version 626753 (0.0007) [2023-12-26 19:55:49,491][105620] Updated weights for policy 1, policy_version 627661 (0.0011) [2023-12-26 19:55:49,533][105692] Updated weights for policy 0, policy_version 626763 (0.0007) [2023-12-26 19:55:49,552][105620] Updated weights for policy 1, policy_version 627671 (0.0011) [2023-12-26 19:55:49,595][105692] Updated weights for policy 0, policy_version 626773 (0.0006) [2023-12-26 19:55:49,601][105620] Updated weights for policy 1, policy_version 627681 (0.0011) [2023-12-26 19:55:49,652][105692] Updated weights for policy 0, policy_version 626783 (0.0006) [2023-12-26 19:55:50,370][105620] Updated weights for policy 1, policy_version 627691 (0.0010) [2023-12-26 19:55:50,374][105692] Updated weights for policy 0, policy_version 626793 (0.0008) [2023-12-26 19:55:50,430][105620] Updated weights for policy 1, policy_version 627701 (0.0011) [2023-12-26 19:55:50,433][105692] Updated weights for policy 0, policy_version 626803 (0.0010) [2023-12-26 19:55:50,490][105620] Updated weights for policy 1, policy_version 627711 (0.0011) [2023-12-26 19:55:50,500][105692] Updated weights for policy 0, policy_version 626813 (0.0006) [2023-12-26 19:55:50,563][105692] Updated weights for policy 0, policy_version 626823 (0.0007) [2023-12-26 19:55:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 321208320. Throughput: 0: 9856.2, 1: 9748.7. Samples: 321200596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:55:51,063][104569] Avg episode reward: [(0, '8931.686'), (1, '8811.858')] [2023-12-26 19:55:51,176][105620] Updated weights for policy 1, policy_version 627721 (0.0010) [2023-12-26 19:55:51,183][105692] Updated weights for policy 0, policy_version 626833 (0.0007) [2023-12-26 19:55:51,230][105620] Updated weights for policy 1, policy_version 627731 (0.0007) [2023-12-26 19:55:51,241][105692] Updated weights for policy 0, policy_version 626843 (0.0008) [2023-12-26 19:55:51,291][105620] Updated weights for policy 1, policy_version 627741 (0.0011) [2023-12-26 19:55:51,301][105692] Updated weights for policy 0, policy_version 626853 (0.0007) [2023-12-26 19:55:51,366][105620] Updated weights for policy 1, policy_version 627751 (0.0011) [2023-12-26 19:55:52,027][105620] Updated weights for policy 1, policy_version 627761 (0.0007) [2023-12-26 19:55:52,081][105692] Updated weights for policy 0, policy_version 626863 (0.0006) [2023-12-26 19:55:52,082][105620] Updated weights for policy 1, policy_version 627771 (0.0010) [2023-12-26 19:55:52,142][105692] Updated weights for policy 0, policy_version 626873 (0.0006) [2023-12-26 19:55:52,142][105620] Updated weights for policy 1, policy_version 627781 (0.0011) [2023-12-26 19:55:52,202][105692] Updated weights for policy 0, policy_version 626883 (0.0008) [2023-12-26 19:55:52,887][105692] Updated weights for policy 0, policy_version 626893 (0.0007) [2023-12-26 19:55:52,905][105620] Updated weights for policy 1, policy_version 627791 (0.0011) [2023-12-26 19:55:52,946][105692] Updated weights for policy 0, policy_version 626903 (0.0007) [2023-12-26 19:55:52,958][105620] Updated weights for policy 1, policy_version 627801 (0.0010) [2023-12-26 19:55:53,004][105692] Updated weights for policy 0, policy_version 626913 (0.0006) [2023-12-26 19:55:53,018][105620] Updated weights for policy 1, policy_version 627811 (0.0011) [2023-12-26 19:55:53,688][105692] Updated weights for policy 0, policy_version 626923 (0.0006) [2023-12-26 19:55:53,739][105692] Updated weights for policy 0, policy_version 626933 (0.0005) [2023-12-26 19:55:53,742][105620] Updated weights for policy 1, policy_version 627821 (0.0009) [2023-12-26 19:55:53,794][105692] Updated weights for policy 0, policy_version 626943 (0.0005) [2023-12-26 19:55:53,807][105620] Updated weights for policy 1, policy_version 627831 (0.0005) [2023-12-26 19:55:53,863][105620] Updated weights for policy 1, policy_version 627841 (0.0005) [2023-12-26 19:55:54,399][105692] Updated weights for policy 0, policy_version 626953 (0.0006) [2023-12-26 19:55:54,459][105692] Updated weights for policy 0, policy_version 626963 (0.0006) [2023-12-26 19:55:54,511][105620] Updated weights for policy 1, policy_version 627851 (0.0006) [2023-12-26 19:55:54,524][105692] Updated weights for policy 0, policy_version 626973 (0.0006) [2023-12-26 19:55:54,566][105620] Updated weights for policy 1, policy_version 627861 (0.0008) [2023-12-26 19:55:54,592][105692] Updated weights for policy 0, policy_version 626983 (0.0008) [2023-12-26 19:55:54,615][105620] Updated weights for policy 1, policy_version 627871 (0.0007) [2023-12-26 19:55:55,256][105692] Updated weights for policy 0, policy_version 626993 (0.0007) [2023-12-26 19:55:55,306][105692] Updated weights for policy 0, policy_version 627003 (0.0010) [2023-12-26 19:55:55,354][105692] Updated weights for policy 0, policy_version 627013 (0.0010) [2023-12-26 19:55:55,366][105620] Updated weights for policy 1, policy_version 627881 (0.0008) [2023-12-26 19:55:55,431][105620] Updated weights for policy 1, policy_version 627891 (0.0010) [2023-12-26 19:55:55,488][105620] Updated weights for policy 1, policy_version 627901 (0.0010) [2023-12-26 19:55:55,546][105620] Updated weights for policy 1, policy_version 627911 (0.0010) [2023-12-26 19:55:55,996][105692] Updated weights for policy 0, policy_version 627023 (0.0007) [2023-12-26 19:55:56,052][105692] Updated weights for policy 0, policy_version 627033 (0.0005) [2023-12-26 19:55:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 321306624. Throughput: 0: 9895.4, 1: 9783.2. Samples: 321319300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:55:56,062][104569] Avg episode reward: [(0, '9269.934'), (1, '8908.252')] [2023-12-26 19:55:56,112][105692] Updated weights for policy 0, policy_version 627043 (0.0010) [2023-12-26 19:55:56,283][105620] Updated weights for policy 1, policy_version 627921 (0.0010) [2023-12-26 19:55:56,335][105620] Updated weights for policy 1, policy_version 627931 (0.0011) [2023-12-26 19:55:56,384][105620] Updated weights for policy 1, policy_version 627941 (0.0011) [2023-12-26 19:55:56,763][105692] Updated weights for policy 0, policy_version 627053 (0.0010) [2023-12-26 19:55:56,823][105692] Updated weights for policy 0, policy_version 627063 (0.0011) [2023-12-26 19:55:56,886][105692] Updated weights for policy 0, policy_version 627073 (0.0010) [2023-12-26 19:55:57,034][105620] Updated weights for policy 1, policy_version 627951 (0.0010) [2023-12-26 19:55:57,078][105620] Updated weights for policy 1, policy_version 627961 (0.0010) [2023-12-26 19:55:57,129][105620] Updated weights for policy 1, policy_version 627971 (0.0010) [2023-12-26 19:55:57,569][105692] Updated weights for policy 0, policy_version 627083 (0.0010) [2023-12-26 19:55:57,616][105692] Updated weights for policy 0, policy_version 627093 (0.0008) [2023-12-26 19:55:57,668][105692] Updated weights for policy 0, policy_version 627103 (0.0008) [2023-12-26 19:55:57,893][105620] Updated weights for policy 1, policy_version 627981 (0.0008) [2023-12-26 19:55:57,941][105620] Updated weights for policy 1, policy_version 627991 (0.0005) [2023-12-26 19:55:57,986][105620] Updated weights for policy 1, policy_version 628001 (0.0005) [2023-12-26 19:55:58,471][105692] Updated weights for policy 0, policy_version 627113 (0.0008) [2023-12-26 19:55:58,537][105692] Updated weights for policy 0, policy_version 627123 (0.0008) [2023-12-26 19:55:58,599][105692] Updated weights for policy 0, policy_version 627133 (0.0008) [2023-12-26 19:55:58,666][105692] Updated weights for policy 0, policy_version 627143 (0.0009) [2023-12-26 19:55:58,752][105620] Updated weights for policy 1, policy_version 628011 (0.0008) [2023-12-26 19:55:58,812][105620] Updated weights for policy 1, policy_version 628021 (0.0008) [2023-12-26 19:55:58,876][105620] Updated weights for policy 1, policy_version 628031 (0.0007) [2023-12-26 19:55:59,444][105692] Updated weights for policy 0, policy_version 627153 (0.0010) [2023-12-26 19:55:59,507][105692] Updated weights for policy 0, policy_version 627163 (0.0010) [2023-12-26 19:55:59,566][105620] Updated weights for policy 1, policy_version 628041 (0.0008) [2023-12-26 19:55:59,567][105692] Updated weights for policy 0, policy_version 627173 (0.0010) [2023-12-26 19:55:59,622][105620] Updated weights for policy 1, policy_version 628051 (0.0007) [2023-12-26 19:55:59,676][105620] Updated weights for policy 1, policy_version 628061 (0.0009) [2023-12-26 19:55:59,726][105620] Updated weights for policy 1, policy_version 628071 (0.0008) [2023-12-26 19:56:00,336][105692] Updated weights for policy 0, policy_version 627183 (0.0010) [2023-12-26 19:56:00,395][105692] Updated weights for policy 0, policy_version 627193 (0.0010) [2023-12-26 19:56:00,467][105692] Updated weights for policy 0, policy_version 627203 (0.0011) [2023-12-26 19:56:00,505][105620] Updated weights for policy 1, policy_version 628081 (0.0007) [2023-12-26 19:56:00,563][105620] Updated weights for policy 1, policy_version 628091 (0.0008) [2023-12-26 19:56:00,617][105620] Updated weights for policy 1, policy_version 628101 (0.0008) [2023-12-26 19:56:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 321404928. Throughput: 0: 9914.6, 1: 9824.2. Samples: 321379084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 19:56:01,062][104569] Avg episode reward: [(0, '8984.771'), (1, '9089.752')] [2023-12-26 19:56:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000627208_160595968.pth... [2023-12-26 19:56:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000628104_160808960.pth... [2023-12-26 19:56:01,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000626056_160301056.pth [2023-12-26 19:56:01,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000626952_160514048.pth [2023-12-26 19:56:01,191][105692] Updated weights for policy 0, policy_version 627213 (0.0010) [2023-12-26 19:56:01,250][105692] Updated weights for policy 0, policy_version 627223 (0.0010) [2023-12-26 19:56:01,302][105692] Updated weights for policy 0, policy_version 627233 (0.0011) [2023-12-26 19:56:01,399][105620] Updated weights for policy 1, policy_version 628111 (0.0008) [2023-12-26 19:56:01,467][105620] Updated weights for policy 1, policy_version 628121 (0.0008) [2023-12-26 19:56:01,531][105620] Updated weights for policy 1, policy_version 628131 (0.0009) [2023-12-26 19:56:02,085][105692] Updated weights for policy 0, policy_version 627243 (0.0011) [2023-12-26 19:56:02,133][105692] Updated weights for policy 0, policy_version 627253 (0.0010) [2023-12-26 19:56:02,185][105692] Updated weights for policy 0, policy_version 627263 (0.0010) [2023-12-26 19:56:02,297][105620] Updated weights for policy 1, policy_version 628141 (0.0008) [2023-12-26 19:56:02,362][105620] Updated weights for policy 1, policy_version 628151 (0.0007) [2023-12-26 19:56:02,423][105620] Updated weights for policy 1, policy_version 628162 (0.0008) [2023-12-26 19:56:02,899][105692] Updated weights for policy 0, policy_version 627273 (0.0010) [2023-12-26 19:56:02,957][105692] Updated weights for policy 0, policy_version 627283 (0.0006) [2023-12-26 19:56:03,013][105692] Updated weights for policy 0, policy_version 627293 (0.0006) [2023-12-26 19:56:03,064][105692] Updated weights for policy 0, policy_version 627303 (0.0006) [2023-12-26 19:56:03,126][105620] Updated weights for policy 1, policy_version 628172 (0.0009) [2023-12-26 19:56:03,180][105620] Updated weights for policy 1, policy_version 628182 (0.0005) [2023-12-26 19:56:03,243][105620] Updated weights for policy 1, policy_version 628192 (0.0005) [2023-12-26 19:56:03,601][105692] Updated weights for policy 0, policy_version 627313 (0.0005) [2023-12-26 19:56:03,657][105692] Updated weights for policy 0, policy_version 627323 (0.0005) [2023-12-26 19:56:03,717][105692] Updated weights for policy 0, policy_version 627333 (0.0008) [2023-12-26 19:56:03,847][105620] Updated weights for policy 1, policy_version 628202 (0.0006) [2023-12-26 19:56:03,908][105620] Updated weights for policy 1, policy_version 628212 (0.0009) [2023-12-26 19:56:03,963][105620] Updated weights for policy 1, policy_version 628222 (0.0009) [2023-12-26 19:56:04,019][105620] Updated weights for policy 1, policy_version 628232 (0.0010) [2023-12-26 19:56:04,359][105692] Updated weights for policy 0, policy_version 627343 (0.0008) [2023-12-26 19:56:04,420][105692] Updated weights for policy 0, policy_version 627353 (0.0009) [2023-12-26 19:56:04,482][105692] Updated weights for policy 0, policy_version 627363 (0.0009) [2023-12-26 19:56:04,859][105620] Updated weights for policy 1, policy_version 628242 (0.0008) [2023-12-26 19:56:04,914][105620] Updated weights for policy 1, policy_version 628252 (0.0009) [2023-12-26 19:56:04,964][105620] Updated weights for policy 1, policy_version 628262 (0.0009) [2023-12-26 19:56:05,160][105692] Updated weights for policy 0, policy_version 627373 (0.0007) [2023-12-26 19:56:05,207][105692] Updated weights for policy 0, policy_version 627383 (0.0006) [2023-12-26 19:56:05,268][105692] Updated weights for policy 0, policy_version 627393 (0.0009) [2023-12-26 19:56:05,773][105620] Updated weights for policy 1, policy_version 628272 (0.0009) [2023-12-26 19:56:05,824][105620] Updated weights for policy 1, policy_version 628282 (0.0009) [2023-12-26 19:56:05,874][105620] Updated weights for policy 1, policy_version 628292 (0.0009) [2023-12-26 19:56:05,955][105692] Updated weights for policy 0, policy_version 627403 (0.0009) [2023-12-26 19:56:06,015][105692] Updated weights for policy 0, policy_version 627413 (0.0008) [2023-12-26 19:56:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.4, 300 sec: 19633.0). Total num frames: 321503232. Throughput: 0: 9952.6, 1: 9760.2. Samples: 321494140. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:56:06,062][104569] Avg episode reward: [(0, '8750.862'), (1, '9176.578')] [2023-12-26 19:56:06,079][105692] Updated weights for policy 0, policy_version 627423 (0.0007) [2023-12-26 19:56:06,608][105620] Updated weights for policy 1, policy_version 628302 (0.0008) [2023-12-26 19:56:06,661][105620] Updated weights for policy 1, policy_version 628312 (0.0008) [2023-12-26 19:56:06,713][105620] Updated weights for policy 1, policy_version 628322 (0.0009) [2023-12-26 19:56:06,850][105692] Updated weights for policy 0, policy_version 627433 (0.0009) [2023-12-26 19:56:06,918][105692] Updated weights for policy 0, policy_version 627443 (0.0010) [2023-12-26 19:56:06,992][105692] Updated weights for policy 0, policy_version 627453 (0.0007) [2023-12-26 19:56:07,050][105692] Updated weights for policy 0, policy_version 627463 (0.0011) [2023-12-26 19:56:07,334][105620] Updated weights for policy 1, policy_version 628332 (0.0006) [2023-12-26 19:56:07,381][105620] Updated weights for policy 1, policy_version 628342 (0.0005) [2023-12-26 19:56:07,428][105620] Updated weights for policy 1, policy_version 628352 (0.0008) [2023-12-26 19:56:07,773][105692] Updated weights for policy 0, policy_version 627473 (0.0011) [2023-12-26 19:56:07,815][105692] Updated weights for policy 0, policy_version 627483 (0.0008) [2023-12-26 19:56:07,867][105692] Updated weights for policy 0, policy_version 627493 (0.0009) [2023-12-26 19:56:08,076][105620] Updated weights for policy 1, policy_version 628362 (0.0008) [2023-12-26 19:56:08,124][105620] Updated weights for policy 1, policy_version 628372 (0.0008) [2023-12-26 19:56:08,169][105620] Updated weights for policy 1, policy_version 628382 (0.0008) [2023-12-26 19:56:08,222][105620] Updated weights for policy 1, policy_version 628392 (0.0010) [2023-12-26 19:56:08,574][105692] Updated weights for policy 0, policy_version 627503 (0.0011) [2023-12-26 19:56:08,636][105692] Updated weights for policy 0, policy_version 627513 (0.0011) [2023-12-26 19:56:08,694][105692] Updated weights for policy 0, policy_version 627523 (0.0010) [2023-12-26 19:56:08,995][105620] Updated weights for policy 1, policy_version 628402 (0.0007) [2023-12-26 19:56:09,057][105620] Updated weights for policy 1, policy_version 628412 (0.0008) [2023-12-26 19:56:09,112][105620] Updated weights for policy 1, policy_version 628422 (0.0008) [2023-12-26 19:56:09,379][105692] Updated weights for policy 0, policy_version 627533 (0.0010) [2023-12-26 19:56:09,444][105692] Updated weights for policy 0, policy_version 627543 (0.0009) [2023-12-26 19:56:09,505][105692] Updated weights for policy 0, policy_version 627553 (0.0009) [2023-12-26 19:56:09,940][105620] Updated weights for policy 1, policy_version 628432 (0.0012) [2023-12-26 19:56:10,004][105620] Updated weights for policy 1, policy_version 628442 (0.0011) [2023-12-26 19:56:10,076][105620] Updated weights for policy 1, policy_version 628452 (0.0011) [2023-12-26 19:56:10,280][105692] Updated weights for policy 0, policy_version 627563 (0.0009) [2023-12-26 19:56:10,343][105692] Updated weights for policy 0, policy_version 627573 (0.0011) [2023-12-26 19:56:10,404][105692] Updated weights for policy 0, policy_version 627583 (0.0011) [2023-12-26 19:56:10,789][105620] Updated weights for policy 1, policy_version 628462 (0.0011) [2023-12-26 19:56:10,851][105620] Updated weights for policy 1, policy_version 628472 (0.0010) [2023-12-26 19:56:10,906][105620] Updated weights for policy 1, policy_version 628482 (0.0010) [2023-12-26 19:56:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 321601536. Throughput: 0: 9891.7, 1: 9841.1. Samples: 321610064. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:56:11,062][104569] Avg episode reward: [(0, '9000.003'), (1, '9267.636')] [2023-12-26 19:56:11,160][105692] Updated weights for policy 0, policy_version 627593 (0.0010) [2023-12-26 19:56:11,226][105692] Updated weights for policy 0, policy_version 627603 (0.0010) [2023-12-26 19:56:11,285][105692] Updated weights for policy 0, policy_version 627613 (0.0010) [2023-12-26 19:56:11,346][105692] Updated weights for policy 0, policy_version 627623 (0.0011) [2023-12-26 19:56:11,770][105620] Updated weights for policy 1, policy_version 628492 (0.0008) [2023-12-26 19:56:11,840][105620] Updated weights for policy 1, policy_version 628502 (0.0005) [2023-12-26 19:56:11,902][105620] Updated weights for policy 1, policy_version 628512 (0.0008) [2023-12-26 19:56:12,091][105692] Updated weights for policy 0, policy_version 627633 (0.0011) [2023-12-26 19:56:12,151][105692] Updated weights for policy 0, policy_version 627643 (0.0011) [2023-12-26 19:56:12,214][105692] Updated weights for policy 0, policy_version 627653 (0.0010) [2023-12-26 19:56:12,589][105620] Updated weights for policy 1, policy_version 628522 (0.0007) [2023-12-26 19:56:12,654][105620] Updated weights for policy 1, policy_version 628532 (0.0007) [2023-12-26 19:56:12,724][105620] Updated weights for policy 1, policy_version 628542 (0.0005) [2023-12-26 19:56:12,795][105620] Updated weights for policy 1, policy_version 628552 (0.0006) [2023-12-26 19:56:12,922][105692] Updated weights for policy 0, policy_version 627663 (0.0007) [2023-12-26 19:56:12,983][105692] Updated weights for policy 0, policy_version 627673 (0.0005) [2023-12-26 19:56:13,051][105692] Updated weights for policy 0, policy_version 627683 (0.0005) [2023-12-26 19:56:13,361][105620] Updated weights for policy 1, policy_version 628562 (0.0007) [2023-12-26 19:56:13,425][105620] Updated weights for policy 1, policy_version 628572 (0.0005) [2023-12-26 19:56:13,470][105620] Updated weights for policy 1, policy_version 628582 (0.0005) [2023-12-26 19:56:13,719][105692] Updated weights for policy 0, policy_version 627693 (0.0008) [2023-12-26 19:56:13,781][105692] Updated weights for policy 0, policy_version 627703 (0.0010) [2023-12-26 19:56:13,828][105692] Updated weights for policy 0, policy_version 627713 (0.0010) [2023-12-26 19:56:14,036][105620] Updated weights for policy 1, policy_version 628592 (0.0009) [2023-12-26 19:56:14,090][105620] Updated weights for policy 1, policy_version 628602 (0.0010) [2023-12-26 19:56:14,152][105620] Updated weights for policy 1, policy_version 628612 (0.0010) [2023-12-26 19:56:14,466][105692] Updated weights for policy 0, policy_version 627723 (0.0009) [2023-12-26 19:56:14,515][105692] Updated weights for policy 0, policy_version 627733 (0.0009) [2023-12-26 19:56:14,567][105692] Updated weights for policy 0, policy_version 627743 (0.0005) [2023-12-26 19:56:14,799][105620] Updated weights for policy 1, policy_version 628622 (0.0010) [2023-12-26 19:56:14,857][105620] Updated weights for policy 1, policy_version 628632 (0.0010) [2023-12-26 19:56:14,907][105620] Updated weights for policy 1, policy_version 628642 (0.0010) [2023-12-26 19:56:15,232][105692] Updated weights for policy 0, policy_version 627753 (0.0005) [2023-12-26 19:56:15,292][105692] Updated weights for policy 0, policy_version 627763 (0.0009) [2023-12-26 19:56:15,351][105692] Updated weights for policy 0, policy_version 627773 (0.0008) [2023-12-26 19:56:15,410][105692] Updated weights for policy 0, policy_version 627783 (0.0008) [2023-12-26 19:56:15,675][105620] Updated weights for policy 1, policy_version 628652 (0.0010) [2023-12-26 19:56:15,726][105620] Updated weights for policy 1, policy_version 628663 (0.0009) [2023-12-26 19:56:15,789][105620] Updated weights for policy 1, policy_version 628673 (0.0010) [2023-12-26 19:56:16,034][105692] Updated weights for policy 0, policy_version 627793 (0.0007) [2023-12-26 19:56:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19660.7, 300 sec: 19633.0). Total num frames: 321699840. Throughput: 0: 9833.9, 1: 9935.9. Samples: 321669832. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:56:16,063][104569] Avg episode reward: [(0, '6508.536'), (1, '9359.135')] [2023-12-26 19:56:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000628680_160956416.pth... [2023-12-26 19:56:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000627560_160669696.pth [2023-12-26 19:56:16,101][105692] Updated weights for policy 0, policy_version 627803 (0.0008) [2023-12-26 19:56:16,161][105692] Updated weights for policy 0, policy_version 627813 (0.0007) [2023-12-26 19:56:16,177][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000627816_160751616.pth... [2023-12-26 19:56:16,180][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000626632_160448512.pth [2023-12-26 19:56:16,483][105620] Updated weights for policy 1, policy_version 628683 (0.0008) [2023-12-26 19:56:16,547][105620] Updated weights for policy 1, policy_version 628693 (0.0011) [2023-12-26 19:56:16,614][105620] Updated weights for policy 1, policy_version 628703 (0.0011) [2023-12-26 19:56:16,734][105692] Updated weights for policy 0, policy_version 627823 (0.0009) [2023-12-26 19:56:16,798][105692] Updated weights for policy 0, policy_version 627833 (0.0011) [2023-12-26 19:56:16,815][105585] KL-divergence is very high: 108.8631 [2023-12-26 19:56:16,843][105585] KL-divergence is very high: 100.9821 [2023-12-26 19:56:16,861][105692] Updated weights for policy 0, policy_version 627843 (0.0011) [2023-12-26 19:56:17,308][105620] Updated weights for policy 1, policy_version 628713 (0.0010) [2023-12-26 19:56:17,366][105620] Updated weights for policy 1, policy_version 628723 (0.0011) [2023-12-26 19:56:17,411][105620] Updated weights for policy 1, policy_version 628733 (0.0010) [2023-12-26 19:56:17,469][105620] Updated weights for policy 1, policy_version 628743 (0.0010) [2023-12-26 19:56:17,547][105692] Updated weights for policy 0, policy_version 627853 (0.0009) [2023-12-26 19:56:17,594][105692] Updated weights for policy 0, policy_version 627863 (0.0009) [2023-12-26 19:56:17,641][105692] Updated weights for policy 0, policy_version 627873 (0.0009) [2023-12-26 19:56:18,176][105620] Updated weights for policy 1, policy_version 628753 (0.0010) [2023-12-26 19:56:18,208][105692] Updated weights for policy 0, policy_version 627883 (0.0009) [2023-12-26 19:56:18,232][105620] Updated weights for policy 1, policy_version 628763 (0.0010) [2023-12-26 19:56:18,259][105692] Updated weights for policy 0, policy_version 627893 (0.0005) [2023-12-26 19:56:18,280][105620] Updated weights for policy 1, policy_version 628773 (0.0010) [2023-12-26 19:56:18,322][105692] Updated weights for policy 0, policy_version 627903 (0.0005) [2023-12-26 19:56:19,009][105692] Updated weights for policy 0, policy_version 627913 (0.0008) [2023-12-26 19:56:19,062][105692] Updated weights for policy 0, policy_version 627923 (0.0011) [2023-12-26 19:56:19,063][105620] Updated weights for policy 1, policy_version 628783 (0.0008) [2023-12-26 19:56:19,114][105692] Updated weights for policy 0, policy_version 627933 (0.0011) [2023-12-26 19:56:19,129][105620] Updated weights for policy 1, policy_version 628793 (0.0008) [2023-12-26 19:56:19,170][105692] Updated weights for policy 0, policy_version 627943 (0.0011) [2023-12-26 19:56:19,189][105620] Updated weights for policy 1, policy_version 628803 (0.0007) [2023-12-26 19:56:19,878][105692] Updated weights for policy 0, policy_version 627953 (0.0008) [2023-12-26 19:56:19,942][105692] Updated weights for policy 0, policy_version 627963 (0.0009) [2023-12-26 19:56:19,989][105620] Updated weights for policy 1, policy_version 628813 (0.0008) [2023-12-26 19:56:20,002][105692] Updated weights for policy 0, policy_version 627973 (0.0007) [2023-12-26 19:56:20,051][105620] Updated weights for policy 1, policy_version 628823 (0.0008) [2023-12-26 19:56:20,113][105620] Updated weights for policy 1, policy_version 628833 (0.0009) [2023-12-26 19:56:20,751][105692] Updated weights for policy 0, policy_version 627983 (0.0009) [2023-12-26 19:56:20,808][105692] Updated weights for policy 0, policy_version 627993 (0.0010) [2023-12-26 19:56:20,857][105692] Updated weights for policy 0, policy_version 628003 (0.0010) [2023-12-26 19:56:20,933][105620] Updated weights for policy 1, policy_version 628843 (0.0010) [2023-12-26 19:56:20,986][105620] Updated weights for policy 1, policy_version 628853 (0.0011) [2023-12-26 19:56:21,049][105620] Updated weights for policy 1, policy_version 628863 (0.0011) [2023-12-26 19:56:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 321798144. Throughput: 0: 10007.3, 1: 9774.7. Samples: 321791040. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:56:21,062][104569] Avg episode reward: [(0, '5936.869'), (1, '9267.262')] [2023-12-26 19:56:21,595][105692] Updated weights for policy 0, policy_version 628013 (0.0010) [2023-12-26 19:56:21,666][105692] Updated weights for policy 0, policy_version 628023 (0.0008) [2023-12-26 19:56:21,736][105692] Updated weights for policy 0, policy_version 628033 (0.0009) [2023-12-26 19:56:21,753][105620] Updated weights for policy 1, policy_version 628873 (0.0010) [2023-12-26 19:56:21,813][105620] Updated weights for policy 1, policy_version 628883 (0.0007) [2023-12-26 19:56:21,881][105620] Updated weights for policy 1, policy_version 628893 (0.0008) [2023-12-26 19:56:21,933][105620] Updated weights for policy 1, policy_version 628903 (0.0006) [2023-12-26 19:56:22,554][105692] Updated weights for policy 0, policy_version 628043 (0.0008) [2023-12-26 19:56:22,580][105620] Updated weights for policy 1, policy_version 628913 (0.0008) [2023-12-26 19:56:22,614][105692] Updated weights for policy 0, policy_version 628053 (0.0007) [2023-12-26 19:56:22,626][105585] KL-divergence is very high: 478.5163 [2023-12-26 19:56:22,645][105585] KL-divergence is very high: 208.0119 [2023-12-26 19:56:22,651][105620] Updated weights for policy 1, policy_version 628923 (0.0007) [2023-12-26 19:56:22,674][105692] Updated weights for policy 0, policy_version 628063 (0.0008) [2023-12-26 19:56:22,675][105585] KL-divergence is very high: 663.0545 [2023-12-26 19:56:22,695][105585] KL-divergence is very high: 147.1276 [2023-12-26 19:56:22,708][105620] Updated weights for policy 1, policy_version 628933 (0.0005) [2023-12-26 19:56:22,727][105585] KL-divergence is very high: 591.7079 [2023-12-26 19:56:23,409][105620] Updated weights for policy 1, policy_version 628943 (0.0005) [2023-12-26 19:56:23,449][105692] Updated weights for policy 0, policy_version 628073 (0.0009) [2023-12-26 19:56:23,467][105620] Updated weights for policy 1, policy_version 628953 (0.0005) [2023-12-26 19:56:23,497][105692] Updated weights for policy 0, policy_version 628083 (0.0010) [2023-12-26 19:56:23,519][105620] Updated weights for policy 1, policy_version 628963 (0.0005) [2023-12-26 19:56:23,545][105692] Updated weights for policy 0, policy_version 628093 (0.0010) [2023-12-26 19:56:23,600][105692] Updated weights for policy 0, policy_version 628103 (0.0010) [2023-12-26 19:56:24,179][105620] Updated weights for policy 1, policy_version 628973 (0.0007) [2023-12-26 19:56:24,228][105620] Updated weights for policy 1, policy_version 628983 (0.0008) [2023-12-26 19:56:24,279][105620] Updated weights for policy 1, policy_version 628993 (0.0008) [2023-12-26 19:56:24,354][105692] Updated weights for policy 0, policy_version 628113 (0.0010) [2023-12-26 19:56:24,408][105692] Updated weights for policy 0, policy_version 628123 (0.0010) [2023-12-26 19:56:24,456][105692] Updated weights for policy 0, policy_version 628133 (0.0010) [2023-12-26 19:56:25,061][105620] Updated weights for policy 1, policy_version 629003 (0.0008) [2023-12-26 19:56:25,119][105620] Updated weights for policy 1, policy_version 629013 (0.0008) [2023-12-26 19:56:25,182][105692] Updated weights for policy 0, policy_version 628143 (0.0007) [2023-12-26 19:56:25,185][105620] Updated weights for policy 1, policy_version 629023 (0.0009) [2023-12-26 19:56:25,233][105692] Updated weights for policy 0, policy_version 628153 (0.0009) [2023-12-26 19:56:25,287][105692] Updated weights for policy 0, policy_version 628163 (0.0010) [2023-12-26 19:56:25,876][105692] Updated weights for policy 0, policy_version 628173 (0.0010) [2023-12-26 19:56:25,928][105692] Updated weights for policy 0, policy_version 628183 (0.0010) [2023-12-26 19:56:25,986][105692] Updated weights for policy 0, policy_version 628193 (0.0010) [2023-12-26 19:56:26,007][105620] Updated weights for policy 1, policy_version 629033 (0.0006) [2023-12-26 19:56:26,057][105620] Updated weights for policy 1, policy_version 629043 (0.0007) [2023-12-26 19:56:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.7, 300 sec: 19633.0). Total num frames: 321896448. Throughput: 0: 9944.2, 1: 9655.1. Samples: 321904800. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:56:26,063][104569] Avg episode reward: [(0, '7989.544'), (1, '9175.175')] [2023-12-26 19:56:26,115][105620] Updated weights for policy 1, policy_version 629053 (0.0007) [2023-12-26 19:56:26,171][105620] Updated weights for policy 1, policy_version 629063 (0.0005) [2023-12-26 19:56:26,694][105692] Updated weights for policy 0, policy_version 628203 (0.0010) [2023-12-26 19:56:26,759][105692] Updated weights for policy 0, policy_version 628213 (0.0009) [2023-12-26 19:56:26,805][105620] Updated weights for policy 1, policy_version 629073 (0.0009) [2023-12-26 19:56:26,819][105692] Updated weights for policy 0, policy_version 628223 (0.0006) [2023-12-26 19:56:26,860][105620] Updated weights for policy 1, policy_version 629083 (0.0010) [2023-12-26 19:56:26,919][105620] Updated weights for policy 1, policy_version 629093 (0.0006) [2023-12-26 19:56:27,570][105692] Updated weights for policy 0, policy_version 628233 (0.0008) [2023-12-26 19:56:27,604][105620] Updated weights for policy 1, policy_version 629103 (0.0006) [2023-12-26 19:56:27,618][105692] Updated weights for policy 0, policy_version 628243 (0.0008) [2023-12-26 19:56:27,667][105620] Updated weights for policy 1, policy_version 629113 (0.0007) [2023-12-26 19:56:27,672][105692] Updated weights for policy 0, policy_version 628253 (0.0009) [2023-12-26 19:56:27,726][105620] Updated weights for policy 1, policy_version 629123 (0.0011) [2023-12-26 19:56:27,728][105692] Updated weights for policy 0, policy_version 628263 (0.0006) [2023-12-26 19:56:28,416][105620] Updated weights for policy 1, policy_version 629133 (0.0011) [2023-12-26 19:56:28,466][105692] Updated weights for policy 0, policy_version 628273 (0.0006) [2023-12-26 19:56:28,471][105620] Updated weights for policy 1, policy_version 629143 (0.0010) [2023-12-26 19:56:28,518][105692] Updated weights for policy 0, policy_version 628283 (0.0007) [2023-12-26 19:56:28,533][105620] Updated weights for policy 1, policy_version 629153 (0.0010) [2023-12-26 19:56:28,565][105692] Updated weights for policy 0, policy_version 628293 (0.0008) [2023-12-26 19:56:29,086][105620] Updated weights for policy 1, policy_version 629163 (0.0008) [2023-12-26 19:56:29,140][105620] Updated weights for policy 1, policy_version 629173 (0.0007) [2023-12-26 19:56:29,193][105620] Updated weights for policy 1, policy_version 629183 (0.0007) [2023-12-26 19:56:29,443][105692] Updated weights for policy 0, policy_version 628303 (0.0008) [2023-12-26 19:56:29,501][105692] Updated weights for policy 0, policy_version 628313 (0.0009) [2023-12-26 19:56:29,510][105585] KL-divergence is very high: 144.8699 [2023-12-26 19:56:29,550][105585] KL-divergence is very high: 149.3434 [2023-12-26 19:56:29,552][105692] Updated weights for policy 0, policy_version 628323 (0.0009) [2023-12-26 19:56:29,875][105620] Updated weights for policy 1, policy_version 629193 (0.0007) [2023-12-26 19:56:29,943][105620] Updated weights for policy 1, policy_version 629203 (0.0008) [2023-12-26 19:56:30,005][105620] Updated weights for policy 1, policy_version 629213 (0.0008) [2023-12-26 19:56:30,067][105620] Updated weights for policy 1, policy_version 629223 (0.0007) [2023-12-26 19:56:30,391][105692] Updated weights for policy 0, policy_version 628333 (0.0009) [2023-12-26 19:56:30,444][105692] Updated weights for policy 0, policy_version 628343 (0.0010) [2023-12-26 19:56:30,495][105692] Updated weights for policy 0, policy_version 628353 (0.0007) [2023-12-26 19:56:30,652][105620] Updated weights for policy 1, policy_version 629233 (0.0009) [2023-12-26 19:56:30,707][105620] Updated weights for policy 1, policy_version 629243 (0.0012) [2023-12-26 19:56:30,764][105620] Updated weights for policy 1, policy_version 629253 (0.0009) [2023-12-26 19:56:30,775][105586] KL-divergence is very high: 149.2718 [2023-12-26 19:56:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 321994752. Throughput: 0: 9913.7, 1: 9684.7. Samples: 321965400. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:56:31,063][104569] Avg episode reward: [(0, '9000.106'), (1, '8991.885')] [2023-12-26 19:56:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000628360_160890880.pth... [2023-12-26 19:56:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000629256_161103872.pth... [2023-12-26 19:56:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000627208_160595968.pth [2023-12-26 19:56:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000628104_160808960.pth [2023-12-26 19:56:31,115][105692] Updated weights for policy 0, policy_version 628363 (0.0006) [2023-12-26 19:56:31,180][105692] Updated weights for policy 0, policy_version 628373 (0.0009) [2023-12-26 19:56:31,242][105692] Updated weights for policy 0, policy_version 628383 (0.0008) [2023-12-26 19:56:31,598][105620] Updated weights for policy 1, policy_version 629263 (0.0010) [2023-12-26 19:56:31,667][105620] Updated weights for policy 1, policy_version 629273 (0.0008) [2023-12-26 19:56:31,733][105620] Updated weights for policy 1, policy_version 629283 (0.0009) [2023-12-26 19:56:31,986][105692] Updated weights for policy 0, policy_version 628393 (0.0010) [2023-12-26 19:56:32,048][105692] Updated weights for policy 0, policy_version 628403 (0.0011) [2023-12-26 19:56:32,111][105692] Updated weights for policy 0, policy_version 628413 (0.0010) [2023-12-26 19:56:32,159][105692] Updated weights for policy 0, policy_version 628423 (0.0009) [2023-12-26 19:56:32,458][105620] Updated weights for policy 1, policy_version 629293 (0.0009) [2023-12-26 19:56:32,514][105620] Updated weights for policy 1, policy_version 629303 (0.0008) [2023-12-26 19:56:32,579][105620] Updated weights for policy 1, policy_version 629313 (0.0005) [2023-12-26 19:56:32,908][105692] Updated weights for policy 0, policy_version 628433 (0.0010) [2023-12-26 19:56:32,960][105692] Updated weights for policy 0, policy_version 628443 (0.0010) [2023-12-26 19:56:33,021][105692] Updated weights for policy 0, policy_version 628453 (0.0005) [2023-12-26 19:56:33,208][105620] Updated weights for policy 1, policy_version 629323 (0.0005) [2023-12-26 19:56:33,263][105620] Updated weights for policy 1, policy_version 629333 (0.0005) [2023-12-26 19:56:33,322][105620] Updated weights for policy 1, policy_version 629343 (0.0007) [2023-12-26 19:56:33,567][105692] Updated weights for policy 0, policy_version 628463 (0.0007) [2023-12-26 19:56:33,618][105692] Updated weights for policy 0, policy_version 628473 (0.0005) [2023-12-26 19:56:33,671][105692] Updated weights for policy 0, policy_version 628483 (0.0005) [2023-12-26 19:56:33,929][105620] Updated weights for policy 1, policy_version 629353 (0.0010) [2023-12-26 19:56:33,980][105620] Updated weights for policy 1, policy_version 629363 (0.0009) [2023-12-26 19:56:34,038][105620] Updated weights for policy 1, policy_version 629374 (0.0010) [2023-12-26 19:56:34,089][105620] Updated weights for policy 1, policy_version 629384 (0.0009) [2023-12-26 19:56:34,211][105692] Updated weights for policy 0, policy_version 628493 (0.0008) [2023-12-26 19:56:34,267][105692] Updated weights for policy 0, policy_version 628503 (0.0010) [2023-12-26 19:56:34,312][105692] Updated weights for policy 0, policy_version 628513 (0.0010) [2023-12-26 19:56:34,875][105620] Updated weights for policy 1, policy_version 629394 (0.0008) [2023-12-26 19:56:34,932][105620] Updated weights for policy 1, policy_version 629404 (0.0007) [2023-12-26 19:56:34,987][105620] Updated weights for policy 1, policy_version 629414 (0.0008) [2023-12-26 19:56:35,073][105692] Updated weights for policy 0, policy_version 628523 (0.0010) [2023-12-26 19:56:35,118][105692] Updated weights for policy 0, policy_version 628533 (0.0010) [2023-12-26 19:56:35,166][105692] Updated weights for policy 0, policy_version 628543 (0.0010) [2023-12-26 19:56:35,729][105620] Updated weights for policy 1, policy_version 629424 (0.0008) [2023-12-26 19:56:35,790][105620] Updated weights for policy 1, policy_version 629434 (0.0008) [2023-12-26 19:56:35,848][105620] Updated weights for policy 1, policy_version 629444 (0.0008) [2023-12-26 19:56:35,930][105692] Updated weights for policy 0, policy_version 628553 (0.0010) [2023-12-26 19:56:35,985][105692] Updated weights for policy 0, policy_version 628563 (0.0010) [2023-12-26 19:56:36,050][105692] Updated weights for policy 0, policy_version 628573 (0.0010) [2023-12-26 19:56:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 322093056. Throughput: 0: 9883.5, 1: 9764.2. Samples: 322084740. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:56:36,063][104569] Avg episode reward: [(0, '8917.418'), (1, '8900.234')] [2023-12-26 19:56:36,116][105692] Updated weights for policy 0, policy_version 628583 (0.0010) [2023-12-26 19:56:36,561][105620] Updated weights for policy 1, policy_version 629454 (0.0008) [2023-12-26 19:56:36,621][105620] Updated weights for policy 1, policy_version 629464 (0.0009) [2023-12-26 19:56:36,687][105620] Updated weights for policy 1, policy_version 629474 (0.0009) [2023-12-26 19:56:36,805][105692] Updated weights for policy 0, policy_version 628593 (0.0008) [2023-12-26 19:56:36,869][105692] Updated weights for policy 0, policy_version 628603 (0.0006) [2023-12-26 19:56:36,928][105692] Updated weights for policy 0, policy_version 628613 (0.0005) [2023-12-26 19:56:37,345][105620] Updated weights for policy 1, policy_version 629484 (0.0009) [2023-12-26 19:56:37,396][105620] Updated weights for policy 1, policy_version 629494 (0.0008) [2023-12-26 19:56:37,457][105620] Updated weights for policy 1, policy_version 629504 (0.0009) [2023-12-26 19:56:37,476][105692] Updated weights for policy 0, policy_version 628623 (0.0006) [2023-12-26 19:56:37,528][105692] Updated weights for policy 0, policy_version 628633 (0.0007) [2023-12-26 19:56:37,584][105692] Updated weights for policy 0, policy_version 628643 (0.0009) [2023-12-26 19:56:38,022][105620] Updated weights for policy 1, policy_version 629514 (0.0007) [2023-12-26 19:56:38,073][105620] Updated weights for policy 1, policy_version 629524 (0.0009) [2023-12-26 19:56:38,130][105620] Updated weights for policy 1, policy_version 629534 (0.0009) [2023-12-26 19:56:38,188][105620] Updated weights for policy 1, policy_version 629544 (0.0009) [2023-12-26 19:56:38,390][105692] Updated weights for policy 0, policy_version 628653 (0.0008) [2023-12-26 19:56:38,442][105692] Updated weights for policy 0, policy_version 628663 (0.0008) [2023-12-26 19:56:38,501][105692] Updated weights for policy 0, policy_version 628673 (0.0007) [2023-12-26 19:56:39,013][105620] Updated weights for policy 1, policy_version 629554 (0.0009) [2023-12-26 19:56:39,067][105620] Updated weights for policy 1, policy_version 629564 (0.0009) [2023-12-26 19:56:39,124][105620] Updated weights for policy 1, policy_version 629574 (0.0008) [2023-12-26 19:56:39,226][105692] Updated weights for policy 0, policy_version 628683 (0.0009) [2023-12-26 19:56:39,289][105692] Updated weights for policy 0, policy_version 628693 (0.0009) [2023-12-26 19:56:39,353][105692] Updated weights for policy 0, policy_version 628703 (0.0008) [2023-12-26 19:56:39,932][105620] Updated weights for policy 1, policy_version 629584 (0.0009) [2023-12-26 19:56:39,999][105620] Updated weights for policy 1, policy_version 629594 (0.0009) [2023-12-26 19:56:40,047][105692] Updated weights for policy 0, policy_version 628713 (0.0008) [2023-12-26 19:56:40,061][105620] Updated weights for policy 1, policy_version 629604 (0.0008) [2023-12-26 19:56:40,109][105692] Updated weights for policy 0, policy_version 628723 (0.0006) [2023-12-26 19:56:40,174][105692] Updated weights for policy 0, policy_version 628733 (0.0005) [2023-12-26 19:56:40,239][105692] Updated weights for policy 0, policy_version 628743 (0.0007) [2023-12-26 19:56:40,833][105620] Updated weights for policy 1, policy_version 629614 (0.0008) [2023-12-26 19:56:40,891][105692] Updated weights for policy 0, policy_version 628753 (0.0010) [2023-12-26 19:56:40,897][105620] Updated weights for policy 1, policy_version 629624 (0.0006) [2023-12-26 19:56:40,949][105692] Updated weights for policy 0, policy_version 628763 (0.0006) [2023-12-26 19:56:40,965][105620] Updated weights for policy 1, policy_version 629634 (0.0008) [2023-12-26 19:56:41,009][105692] Updated weights for policy 0, policy_version 628773 (0.0005) [2023-12-26 19:56:41,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 322199552. Throughput: 0: 9862.8, 1: 9743.3. Samples: 322201572. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:56:41,062][104569] Avg episode reward: [(0, '8826.090'), (1, '9083.771')] [2023-12-26 19:56:41,750][105692] Updated weights for policy 0, policy_version 628783 (0.0009) [2023-12-26 19:56:41,769][105620] Updated weights for policy 1, policy_version 629644 (0.0008) [2023-12-26 19:56:41,811][105692] Updated weights for policy 0, policy_version 628793 (0.0011) [2023-12-26 19:56:41,833][105620] Updated weights for policy 1, policy_version 629654 (0.0006) [2023-12-26 19:56:41,875][105692] Updated weights for policy 0, policy_version 628803 (0.0011) [2023-12-26 19:56:41,893][105620] Updated weights for policy 1, policy_version 629664 (0.0006) [2023-12-26 19:56:42,597][105692] Updated weights for policy 0, policy_version 628813 (0.0011) [2023-12-26 19:56:42,610][105620] Updated weights for policy 1, policy_version 629674 (0.0007) [2023-12-26 19:56:42,658][105692] Updated weights for policy 0, policy_version 628823 (0.0011) [2023-12-26 19:56:42,674][105620] Updated weights for policy 1, policy_version 629684 (0.0011) [2023-12-26 19:56:42,722][105692] Updated weights for policy 0, policy_version 628833 (0.0011) [2023-12-26 19:56:42,732][105620] Updated weights for policy 1, policy_version 629694 (0.0010) [2023-12-26 19:56:42,787][105620] Updated weights for policy 1, policy_version 629704 (0.0008) [2023-12-26 19:56:43,467][105692] Updated weights for policy 0, policy_version 628843 (0.0010) [2023-12-26 19:56:43,526][105620] Updated weights for policy 1, policy_version 629714 (0.0007) [2023-12-26 19:56:43,530][105692] Updated weights for policy 0, policy_version 628853 (0.0009) [2023-12-26 19:56:43,573][105620] Updated weights for policy 1, policy_version 629724 (0.0006) [2023-12-26 19:56:43,590][105692] Updated weights for policy 0, policy_version 628864 (0.0008) [2023-12-26 19:56:43,634][105620] Updated weights for policy 1, policy_version 629734 (0.0006) [2023-12-26 19:56:44,255][105620] Updated weights for policy 1, policy_version 629744 (0.0009) [2023-12-26 19:56:44,272][105692] Updated weights for policy 0, policy_version 628874 (0.0008) [2023-12-26 19:56:44,304][105620] Updated weights for policy 1, policy_version 629754 (0.0008) [2023-12-26 19:56:44,333][105692] Updated weights for policy 0, policy_version 628884 (0.0005) [2023-12-26 19:56:44,352][105620] Updated weights for policy 1, policy_version 629764 (0.0007) [2023-12-26 19:56:44,402][105692] Updated weights for policy 0, policy_version 628894 (0.0005) [2023-12-26 19:56:44,467][105692] Updated weights for policy 0, policy_version 628904 (0.0005) [2023-12-26 19:56:45,114][105620] Updated weights for policy 1, policy_version 629774 (0.0009) [2023-12-26 19:56:45,119][105692] Updated weights for policy 0, policy_version 628914 (0.0011) [2023-12-26 19:56:45,171][105620] Updated weights for policy 1, policy_version 629784 (0.0011) [2023-12-26 19:56:45,179][105692] Updated weights for policy 0, policy_version 628924 (0.0011) [2023-12-26 19:56:45,220][105620] Updated weights for policy 1, policy_version 629794 (0.0010) [2023-12-26 19:56:45,239][105692] Updated weights for policy 0, policy_version 628934 (0.0010) [2023-12-26 19:56:45,934][105620] Updated weights for policy 1, policy_version 629804 (0.0010) [2023-12-26 19:56:45,946][105692] Updated weights for policy 0, policy_version 628944 (0.0010) [2023-12-26 19:56:45,986][105620] Updated weights for policy 1, policy_version 629814 (0.0010) [2023-12-26 19:56:46,005][105692] Updated weights for policy 0, policy_version 628954 (0.0010) [2023-12-26 19:56:46,040][105620] Updated weights for policy 1, policy_version 629824 (0.0010) [2023-12-26 19:56:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 322281472. Throughput: 0: 9826.2, 1: 9732.9. Samples: 322259244. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:56:46,062][104569] Avg episode reward: [(0, '9176.481'), (1, '8903.404')] [2023-12-26 19:56:46,065][105692] Updated weights for policy 0, policy_version 628964 (0.0010) [2023-12-26 19:56:46,079][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000629832_161251328.pth... [2023-12-26 19:56:46,082][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000628680_160956416.pth [2023-12-26 19:56:46,087][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000628968_161046528.pth... [2023-12-26 19:56:46,091][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000627816_160751616.pth [2023-12-26 19:56:46,770][105620] Updated weights for policy 1, policy_version 629834 (0.0011) [2023-12-26 19:56:46,790][105692] Updated weights for policy 0, policy_version 628974 (0.0008) [2023-12-26 19:56:46,830][105620] Updated weights for policy 1, policy_version 629844 (0.0011) [2023-12-26 19:56:46,836][105692] Updated weights for policy 0, policy_version 628984 (0.0008) [2023-12-26 19:56:46,886][105692] Updated weights for policy 0, policy_version 628994 (0.0007) [2023-12-26 19:56:46,890][105620] Updated weights for policy 1, policy_version 629854 (0.0011) [2023-12-26 19:56:46,949][105620] Updated weights for policy 1, policy_version 629864 (0.0009) [2023-12-26 19:56:47,630][105692] Updated weights for policy 0, policy_version 629004 (0.0008) [2023-12-26 19:56:47,679][105692] Updated weights for policy 0, policy_version 629014 (0.0009) [2023-12-26 19:56:47,703][105620] Updated weights for policy 1, policy_version 629874 (0.0011) [2023-12-26 19:56:47,738][105692] Updated weights for policy 0, policy_version 629024 (0.0009) [2023-12-26 19:56:47,760][105620] Updated weights for policy 1, policy_version 629884 (0.0010) [2023-12-26 19:56:47,816][105620] Updated weights for policy 1, policy_version 629894 (0.0010) [2023-12-26 19:56:48,393][105692] Updated weights for policy 0, policy_version 629034 (0.0006) [2023-12-26 19:56:48,462][105692] Updated weights for policy 0, policy_version 629044 (0.0006) [2023-12-26 19:56:48,530][105692] Updated weights for policy 0, policy_version 629054 (0.0007) [2023-12-26 19:56:48,584][105692] Updated weights for policy 0, policy_version 629064 (0.0007) [2023-12-26 19:56:48,634][105620] Updated weights for policy 1, policy_version 629904 (0.0009) [2023-12-26 19:56:48,681][105620] Updated weights for policy 1, policy_version 629914 (0.0010) [2023-12-26 19:56:48,735][105620] Updated weights for policy 1, policy_version 629924 (0.0010) [2023-12-26 19:56:49,156][105692] Updated weights for policy 0, policy_version 629074 (0.0005) [2023-12-26 19:56:49,217][105692] Updated weights for policy 0, policy_version 629084 (0.0008) [2023-12-26 19:56:49,283][105692] Updated weights for policy 0, policy_version 629094 (0.0008) [2023-12-26 19:56:49,586][105620] Updated weights for policy 1, policy_version 629934 (0.0008) [2023-12-26 19:56:49,647][105620] Updated weights for policy 1, policy_version 629944 (0.0010) [2023-12-26 19:56:49,709][105620] Updated weights for policy 1, policy_version 629954 (0.0010) [2023-12-26 19:56:50,006][105692] Updated weights for policy 0, policy_version 629104 (0.0006) [2023-12-26 19:56:50,071][105692] Updated weights for policy 0, policy_version 629114 (0.0007) [2023-12-26 19:56:50,128][105692] Updated weights for policy 0, policy_version 629124 (0.0007) [2023-12-26 19:56:50,471][105620] Updated weights for policy 1, policy_version 629964 (0.0010) [2023-12-26 19:56:50,526][105620] Updated weights for policy 1, policy_version 629974 (0.0009) [2023-12-26 19:56:50,593][105620] Updated weights for policy 1, policy_version 629984 (0.0009) [2023-12-26 19:56:50,845][105692] Updated weights for policy 0, policy_version 629134 (0.0010) [2023-12-26 19:56:50,903][105692] Updated weights for policy 0, policy_version 629144 (0.0009) [2023-12-26 19:56:50,960][105692] Updated weights for policy 0, policy_version 629154 (0.0008) [2023-12-26 19:56:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 322387968. Throughput: 0: 9875.6, 1: 9703.5. Samples: 322375200. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:56:51,063][104569] Avg episode reward: [(0, '9263.912'), (1, '9177.841')] [2023-12-26 19:56:51,312][105620] Updated weights for policy 1, policy_version 629994 (0.0009) [2023-12-26 19:56:51,377][105620] Updated weights for policy 1, policy_version 630004 (0.0008) [2023-12-26 19:56:51,436][105620] Updated weights for policy 1, policy_version 630014 (0.0009) [2023-12-26 19:56:51,501][105620] Updated weights for policy 1, policy_version 630024 (0.0009) [2023-12-26 19:56:51,767][105692] Updated weights for policy 0, policy_version 629164 (0.0009) [2023-12-26 19:56:51,822][105692] Updated weights for policy 0, policy_version 629174 (0.0009) [2023-12-26 19:56:51,878][105692] Updated weights for policy 0, policy_version 629184 (0.0010) [2023-12-26 19:56:52,198][105620] Updated weights for policy 1, policy_version 630034 (0.0009) [2023-12-26 19:56:52,262][105620] Updated weights for policy 1, policy_version 630044 (0.0009) [2023-12-26 19:56:52,327][105620] Updated weights for policy 1, policy_version 630054 (0.0009) [2023-12-26 19:56:52,672][105692] Updated weights for policy 0, policy_version 629194 (0.0010) [2023-12-26 19:56:52,729][105692] Updated weights for policy 0, policy_version 629204 (0.0009) [2023-12-26 19:56:52,787][105692] Updated weights for policy 0, policy_version 629214 (0.0009) [2023-12-26 19:56:52,849][105692] Updated weights for policy 0, policy_version 629224 (0.0009) [2023-12-26 19:56:53,066][105620] Updated weights for policy 1, policy_version 630064 (0.0010) [2023-12-26 19:56:53,127][105620] Updated weights for policy 1, policy_version 630074 (0.0010) [2023-12-26 19:56:53,191][105620] Updated weights for policy 1, policy_version 630084 (0.0009) [2023-12-26 19:56:53,471][105692] Updated weights for policy 0, policy_version 629234 (0.0006) [2023-12-26 19:56:53,521][105692] Updated weights for policy 0, policy_version 629244 (0.0005) [2023-12-26 19:56:53,582][105692] Updated weights for policy 0, policy_version 629254 (0.0008) [2023-12-26 19:56:54,015][105620] Updated weights for policy 1, policy_version 630095 (0.0009) [2023-12-26 19:56:54,079][105620] Updated weights for policy 1, policy_version 630105 (0.0005) [2023-12-26 19:56:54,139][105620] Updated weights for policy 1, policy_version 630115 (0.0005) [2023-12-26 19:56:54,203][105692] Updated weights for policy 0, policy_version 629264 (0.0008) [2023-12-26 19:56:54,253][105692] Updated weights for policy 0, policy_version 629274 (0.0007) [2023-12-26 19:56:54,306][105692] Updated weights for policy 0, policy_version 629284 (0.0009) [2023-12-26 19:56:54,830][105620] Updated weights for policy 1, policy_version 630125 (0.0006) [2023-12-26 19:56:54,883][105620] Updated weights for policy 1, policy_version 630135 (0.0005) [2023-12-26 19:56:54,953][105620] Updated weights for policy 1, policy_version 630145 (0.0006) [2023-12-26 19:56:55,080][105692] Updated weights for policy 0, policy_version 629294 (0.0009) [2023-12-26 19:56:55,145][105692] Updated weights for policy 0, policy_version 629304 (0.0009) [2023-12-26 19:56:55,204][105692] Updated weights for policy 0, policy_version 629314 (0.0009) [2023-12-26 19:56:55,619][105620] Updated weights for policy 1, policy_version 630155 (0.0009) [2023-12-26 19:56:55,685][105620] Updated weights for policy 1, policy_version 630165 (0.0009) [2023-12-26 19:56:55,750][105620] Updated weights for policy 1, policy_version 630175 (0.0009) [2023-12-26 19:56:55,950][105692] Updated weights for policy 0, policy_version 629324 (0.0008) [2023-12-26 19:56:56,002][105692] Updated weights for policy 0, policy_version 629334 (0.0008) [2023-12-26 19:56:56,049][105692] Updated weights for policy 0, policy_version 629344 (0.0009) [2023-12-26 19:56:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 322478080. Throughput: 0: 9871.5, 1: 9676.4. Samples: 322489716. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:56:56,062][104569] Avg episode reward: [(0, '9263.987'), (1, '9178.358')] [2023-12-26 19:56:56,440][105620] Updated weights for policy 1, policy_version 630185 (0.0009) [2023-12-26 19:56:56,503][105620] Updated weights for policy 1, policy_version 630195 (0.0009) [2023-12-26 19:56:56,569][105620] Updated weights for policy 1, policy_version 630205 (0.0009) [2023-12-26 19:56:56,628][105620] Updated weights for policy 1, policy_version 630215 (0.0009) [2023-12-26 19:56:56,821][105692] Updated weights for policy 0, policy_version 629354 (0.0008) [2023-12-26 19:56:56,878][105692] Updated weights for policy 0, policy_version 629364 (0.0008) [2023-12-26 19:56:56,936][105692] Updated weights for policy 0, policy_version 629374 (0.0008) [2023-12-26 19:56:56,985][105692] Updated weights for policy 0, policy_version 629384 (0.0009) [2023-12-26 19:56:57,354][105620] Updated weights for policy 1, policy_version 630225 (0.0006) [2023-12-26 19:56:57,417][105620] Updated weights for policy 1, policy_version 630235 (0.0005) [2023-12-26 19:56:57,466][105620] Updated weights for policy 1, policy_version 630245 (0.0005) [2023-12-26 19:56:57,737][105692] Updated weights for policy 0, policy_version 629394 (0.0010) [2023-12-26 19:56:57,790][105692] Updated weights for policy 0, policy_version 629404 (0.0008) [2023-12-26 19:56:57,841][105692] Updated weights for policy 0, policy_version 629414 (0.0007) [2023-12-26 19:56:58,105][105620] Updated weights for policy 1, policy_version 630255 (0.0005) [2023-12-26 19:56:58,169][105620] Updated weights for policy 1, policy_version 630265 (0.0010) [2023-12-26 19:56:58,236][105620] Updated weights for policy 1, policy_version 630275 (0.0010) [2023-12-26 19:56:58,679][105692] Updated weights for policy 0, policy_version 629424 (0.0008) [2023-12-26 19:56:58,736][105692] Updated weights for policy 0, policy_version 629434 (0.0008) [2023-12-26 19:56:58,792][105692] Updated weights for policy 0, policy_version 629444 (0.0008) [2023-12-26 19:56:59,055][105620] Updated weights for policy 1, policy_version 630285 (0.0008) [2023-12-26 19:56:59,105][105620] Updated weights for policy 1, policy_version 630295 (0.0006) [2023-12-26 19:56:59,157][105620] Updated weights for policy 1, policy_version 630305 (0.0005) [2023-12-26 19:56:59,469][105692] Updated weights for policy 0, policy_version 629454 (0.0006) [2023-12-26 19:56:59,535][105692] Updated weights for policy 0, policy_version 629464 (0.0008) [2023-12-26 19:56:59,602][105692] Updated weights for policy 0, policy_version 629474 (0.0009) [2023-12-26 19:56:59,910][105620] Updated weights for policy 1, policy_version 630315 (0.0006) [2023-12-26 19:56:59,969][105620] Updated weights for policy 1, policy_version 630325 (0.0009) [2023-12-26 19:57:00,018][105620] Updated weights for policy 1, policy_version 630335 (0.0009) [2023-12-26 19:57:00,272][105692] Updated weights for policy 0, policy_version 629484 (0.0009) [2023-12-26 19:57:00,335][105692] Updated weights for policy 0, policy_version 629494 (0.0009) [2023-12-26 19:57:00,391][105692] Updated weights for policy 0, policy_version 629504 (0.0009) [2023-12-26 19:57:00,765][105620] Updated weights for policy 1, policy_version 630345 (0.0010) [2023-12-26 19:57:00,815][105620] Updated weights for policy 1, policy_version 630355 (0.0009) [2023-12-26 19:57:00,864][105620] Updated weights for policy 1, policy_version 630365 (0.0005) [2023-12-26 19:57:00,916][105620] Updated weights for policy 1, policy_version 630375 (0.0006) [2023-12-26 19:57:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 322576384. Throughput: 0: 9847.3, 1: 9636.1. Samples: 322546576. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:57:01,062][104569] Avg episode reward: [(0, '9264.025'), (1, '9179.490')] [2023-12-26 19:57:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000629512_161185792.pth... [2023-12-26 19:57:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000630376_161390592.pth... [2023-12-26 19:57:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000628360_160890880.pth [2023-12-26 19:57:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000629256_161103872.pth [2023-12-26 19:57:01,203][105692] Updated weights for policy 0, policy_version 629514 (0.0010) [2023-12-26 19:57:01,266][105692] Updated weights for policy 0, policy_version 629524 (0.0011) [2023-12-26 19:57:01,328][105692] Updated weights for policy 0, policy_version 629534 (0.0011) [2023-12-26 19:57:01,391][105692] Updated weights for policy 0, policy_version 629544 (0.0007) [2023-12-26 19:57:01,600][105620] Updated weights for policy 1, policy_version 630385 (0.0009) [2023-12-26 19:57:01,670][105620] Updated weights for policy 1, policy_version 630395 (0.0009) [2023-12-26 19:57:01,738][105620] Updated weights for policy 1, policy_version 630405 (0.0005) [2023-12-26 19:57:02,009][105692] Updated weights for policy 0, policy_version 629554 (0.0010) [2023-12-26 19:57:02,060][105692] Updated weights for policy 0, policy_version 629564 (0.0010) [2023-12-26 19:57:02,115][105692] Updated weights for policy 0, policy_version 629574 (0.0010) [2023-12-26 19:57:02,348][105620] Updated weights for policy 1, policy_version 630415 (0.0007) [2023-12-26 19:57:02,413][105620] Updated weights for policy 1, policy_version 630425 (0.0009) [2023-12-26 19:57:02,473][105620] Updated weights for policy 1, policy_version 630435 (0.0009) [2023-12-26 19:57:02,773][105692] Updated weights for policy 0, policy_version 629584 (0.0011) [2023-12-26 19:57:02,833][105692] Updated weights for policy 0, policy_version 629594 (0.0011) [2023-12-26 19:57:02,892][105692] Updated weights for policy 0, policy_version 629604 (0.0010) [2023-12-26 19:57:03,249][105620] Updated weights for policy 1, policy_version 630445 (0.0009) [2023-12-26 19:57:03,297][105620] Updated weights for policy 1, policy_version 630455 (0.0008) [2023-12-26 19:57:03,349][105620] Updated weights for policy 1, policy_version 630465 (0.0009) [2023-12-26 19:57:03,549][105692] Updated weights for policy 0, policy_version 629614 (0.0007) [2023-12-26 19:57:03,599][105692] Updated weights for policy 0, policy_version 629624 (0.0009) [2023-12-26 19:57:03,664][105692] Updated weights for policy 0, policy_version 629634 (0.0010) [2023-12-26 19:57:04,165][105620] Updated weights for policy 1, policy_version 630475 (0.0008) [2023-12-26 19:57:04,219][105620] Updated weights for policy 1, policy_version 630485 (0.0008) [2023-12-26 19:57:04,279][105620] Updated weights for policy 1, policy_version 630495 (0.0008) [2023-12-26 19:57:04,402][105692] Updated weights for policy 0, policy_version 629644 (0.0011) [2023-12-26 19:57:04,456][105692] Updated weights for policy 0, policy_version 629654 (0.0011) [2023-12-26 19:57:04,514][105692] Updated weights for policy 0, policy_version 629664 (0.0011) [2023-12-26 19:57:05,033][105620] Updated weights for policy 1, policy_version 630505 (0.0007) [2023-12-26 19:57:05,088][105620] Updated weights for policy 1, policy_version 630515 (0.0008) [2023-12-26 19:57:05,139][105620] Updated weights for policy 1, policy_version 630525 (0.0007) [2023-12-26 19:57:05,198][105620] Updated weights for policy 1, policy_version 630535 (0.0008) [2023-12-26 19:57:05,279][105692] Updated weights for policy 0, policy_version 629674 (0.0010) [2023-12-26 19:57:05,337][105692] Updated weights for policy 0, policy_version 629684 (0.0010) [2023-12-26 19:57:05,394][105692] Updated weights for policy 0, policy_version 629694 (0.0010) [2023-12-26 19:57:05,442][105692] Updated weights for policy 0, policy_version 629704 (0.0010) [2023-12-26 19:57:05,949][105620] Updated weights for policy 1, policy_version 630545 (0.0009) [2023-12-26 19:57:06,000][105620] Updated weights for policy 1, policy_version 630555 (0.0008) [2023-12-26 19:57:06,048][105620] Updated weights for policy 1, policy_version 630565 (0.0008) [2023-12-26 19:57:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 322674688. Throughput: 0: 9746.8, 1: 9637.3. Samples: 322663324. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:57:06,062][104569] Avg episode reward: [(0, '9355.331'), (1, '9271.043')] [2023-12-26 19:57:06,192][105692] Updated weights for policy 0, policy_version 629714 (0.0011) [2023-12-26 19:57:06,252][105692] Updated weights for policy 0, policy_version 629724 (0.0011) [2023-12-26 19:57:06,315][105692] Updated weights for policy 0, policy_version 629734 (0.0010) [2023-12-26 19:57:06,774][105620] Updated weights for policy 1, policy_version 630575 (0.0006) [2023-12-26 19:57:06,826][105620] Updated weights for policy 1, policy_version 630585 (0.0006) [2023-12-26 19:57:06,875][105620] Updated weights for policy 1, policy_version 630595 (0.0005) [2023-12-26 19:57:07,065][105692] Updated weights for policy 0, policy_version 629744 (0.0011) [2023-12-26 19:57:07,130][105692] Updated weights for policy 0, policy_version 629754 (0.0010) [2023-12-26 19:57:07,195][105692] Updated weights for policy 0, policy_version 629764 (0.0010) [2023-12-26 19:57:07,485][105620] Updated weights for policy 1, policy_version 630605 (0.0008) [2023-12-26 19:57:07,544][105620] Updated weights for policy 1, policy_version 630615 (0.0011) [2023-12-26 19:57:07,597][105620] Updated weights for policy 1, policy_version 630625 (0.0010) [2023-12-26 19:57:07,932][105692] Updated weights for policy 0, policy_version 629774 (0.0010) [2023-12-26 19:57:07,984][105692] Updated weights for policy 0, policy_version 629784 (0.0010) [2023-12-26 19:57:08,044][105692] Updated weights for policy 0, policy_version 629794 (0.0011) [2023-12-26 19:57:08,288][105620] Updated weights for policy 1, policy_version 630635 (0.0009) [2023-12-26 19:57:08,355][105620] Updated weights for policy 1, policy_version 630645 (0.0007) [2023-12-26 19:57:08,419][105620] Updated weights for policy 1, policy_version 630655 (0.0011) [2023-12-26 19:57:08,763][105692] Updated weights for policy 0, policy_version 629804 (0.0009) [2023-12-26 19:57:08,826][105692] Updated weights for policy 0, policy_version 629814 (0.0008) [2023-12-26 19:57:08,889][105692] Updated weights for policy 0, policy_version 629824 (0.0008) [2023-12-26 19:57:09,132][105620] Updated weights for policy 1, policy_version 630665 (0.0011) [2023-12-26 19:57:09,193][105620] Updated weights for policy 1, policy_version 630675 (0.0010) [2023-12-26 19:57:09,253][105620] Updated weights for policy 1, policy_version 630685 (0.0011) [2023-12-26 19:57:09,313][105620] Updated weights for policy 1, policy_version 630695 (0.0011) [2023-12-26 19:57:09,658][105692] Updated weights for policy 0, policy_version 629834 (0.0008) [2023-12-26 19:57:09,718][105692] Updated weights for policy 0, policy_version 629844 (0.0006) [2023-12-26 19:57:09,784][105692] Updated weights for policy 0, policy_version 629854 (0.0006) [2023-12-26 19:57:09,854][105692] Updated weights for policy 0, policy_version 629864 (0.0008) [2023-12-26 19:57:10,095][105620] Updated weights for policy 1, policy_version 630705 (0.0008) [2023-12-26 19:57:10,160][105620] Updated weights for policy 1, policy_version 630715 (0.0009) [2023-12-26 19:57:10,227][105620] Updated weights for policy 1, policy_version 630725 (0.0010) [2023-12-26 19:57:10,497][105692] Updated weights for policy 0, policy_version 629874 (0.0010) [2023-12-26 19:57:10,546][105692] Updated weights for policy 0, policy_version 629884 (0.0011) [2023-12-26 19:57:10,612][105692] Updated weights for policy 0, policy_version 629894 (0.0011) [2023-12-26 19:57:10,934][105620] Updated weights for policy 1, policy_version 630735 (0.0009) [2023-12-26 19:57:10,987][105620] Updated weights for policy 1, policy_version 630745 (0.0008) [2023-12-26 19:57:11,047][105620] Updated weights for policy 1, policy_version 630755 (0.0008) [2023-12-26 19:57:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 322764800. Throughput: 0: 9745.5, 1: 9670.9. Samples: 322778532. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:57:11,063][104569] Avg episode reward: [(0, '9007.522'), (1, '9179.690')] [2023-12-26 19:57:11,397][105692] Updated weights for policy 0, policy_version 629904 (0.0009) [2023-12-26 19:57:11,460][105692] Updated weights for policy 0, policy_version 629914 (0.0008) [2023-12-26 19:57:11,521][105692] Updated weights for policy 0, policy_version 629924 (0.0008) [2023-12-26 19:57:11,847][105620] Updated weights for policy 1, policy_version 630765 (0.0009) [2023-12-26 19:57:11,903][105620] Updated weights for policy 1, policy_version 630775 (0.0009) [2023-12-26 19:57:11,963][105620] Updated weights for policy 1, policy_version 630785 (0.0009) [2023-12-26 19:57:12,275][105692] Updated weights for policy 0, policy_version 629934 (0.0009) [2023-12-26 19:57:12,339][105692] Updated weights for policy 0, policy_version 629944 (0.0009) [2023-12-26 19:57:12,398][105692] Updated weights for policy 0, policy_version 629954 (0.0007) [2023-12-26 19:57:12,759][105620] Updated weights for policy 1, policy_version 630795 (0.0009) [2023-12-26 19:57:12,819][105620] Updated weights for policy 1, policy_version 630805 (0.0009) [2023-12-26 19:57:12,872][105620] Updated weights for policy 1, policy_version 630815 (0.0009) [2023-12-26 19:57:13,153][105692] Updated weights for policy 0, policy_version 629964 (0.0009) [2023-12-26 19:57:13,207][105692] Updated weights for policy 0, policy_version 629974 (0.0009) [2023-12-26 19:57:13,265][105692] Updated weights for policy 0, policy_version 629984 (0.0010) [2023-12-26 19:57:13,611][105620] Updated weights for policy 1, policy_version 630825 (0.0007) [2023-12-26 19:57:13,663][105620] Updated weights for policy 1, policy_version 630835 (0.0009) [2023-12-26 19:57:13,715][105620] Updated weights for policy 1, policy_version 630845 (0.0009) [2023-12-26 19:57:13,761][105620] Updated weights for policy 1, policy_version 630855 (0.0008) [2023-12-26 19:57:13,967][105692] Updated weights for policy 0, policy_version 629994 (0.0009) [2023-12-26 19:57:14,018][105692] Updated weights for policy 0, policy_version 630004 (0.0009) [2023-12-26 19:57:14,067][105692] Updated weights for policy 0, policy_version 630014 (0.0009) [2023-12-26 19:57:14,125][105692] Updated weights for policy 0, policy_version 630024 (0.0009) [2023-12-26 19:57:14,588][105620] Updated weights for policy 1, policy_version 630865 (0.0009) [2023-12-26 19:57:14,649][105620] Updated weights for policy 1, policy_version 630875 (0.0009) [2023-12-26 19:57:14,709][105620] Updated weights for policy 1, policy_version 630885 (0.0008) [2023-12-26 19:57:14,865][105692] Updated weights for policy 0, policy_version 630034 (0.0010) [2023-12-26 19:57:14,921][105692] Updated weights for policy 0, policy_version 630044 (0.0009) [2023-12-26 19:57:14,981][105692] Updated weights for policy 0, policy_version 630054 (0.0009) [2023-12-26 19:57:15,446][105620] Updated weights for policy 1, policy_version 630895 (0.0009) [2023-12-26 19:57:15,503][105620] Updated weights for policy 1, policy_version 630905 (0.0008) [2023-12-26 19:57:15,561][105620] Updated weights for policy 1, policy_version 630915 (0.0009) [2023-12-26 19:57:15,754][105692] Updated weights for policy 0, policy_version 630064 (0.0007) [2023-12-26 19:57:15,824][105692] Updated weights for policy 0, policy_version 630074 (0.0005) [2023-12-26 19:57:15,876][105692] Updated weights for policy 0, policy_version 630084 (0.0005) [2023-12-26 19:57:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 322863104. Throughput: 0: 9705.7, 1: 9569.2. Samples: 322832768. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:57:16,062][104569] Avg episode reward: [(0, '8916.985'), (1, '9179.553')] [2023-12-26 19:57:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000630088_161333248.pth... [2023-12-26 19:57:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000630920_161529856.pth... [2023-12-26 19:57:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000628968_161046528.pth [2023-12-26 19:57:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000629832_161251328.pth [2023-12-26 19:57:16,386][105692] Updated weights for policy 0, policy_version 630094 (0.0007) [2023-12-26 19:57:16,421][105620] Updated weights for policy 1, policy_version 630925 (0.0008) [2023-12-26 19:57:16,443][105692] Updated weights for policy 0, policy_version 630104 (0.0008) [2023-12-26 19:57:16,470][105620] Updated weights for policy 1, policy_version 630935 (0.0006) [2023-12-26 19:57:16,503][105692] Updated weights for policy 0, policy_version 630114 (0.0007) [2023-12-26 19:57:16,529][105620] Updated weights for policy 1, policy_version 630945 (0.0006) [2023-12-26 19:57:17,185][105620] Updated weights for policy 1, policy_version 630955 (0.0008) [2023-12-26 19:57:17,242][105620] Updated weights for policy 1, policy_version 630965 (0.0008) [2023-12-26 19:57:17,248][105692] Updated weights for policy 0, policy_version 630124 (0.0007) [2023-12-26 19:57:17,296][105620] Updated weights for policy 1, policy_version 630975 (0.0006) [2023-12-26 19:57:17,298][105692] Updated weights for policy 0, policy_version 630134 (0.0006) [2023-12-26 19:57:17,347][105692] Updated weights for policy 0, policy_version 630144 (0.0006) [2023-12-26 19:57:18,005][105620] Updated weights for policy 1, policy_version 630985 (0.0010) [2023-12-26 19:57:18,062][105620] Updated weights for policy 1, policy_version 630995 (0.0008) [2023-12-26 19:57:18,116][105692] Updated weights for policy 0, policy_version 630154 (0.0006) [2023-12-26 19:57:18,125][105620] Updated weights for policy 1, policy_version 631005 (0.0009) [2023-12-26 19:57:18,179][105692] Updated weights for policy 0, policy_version 630164 (0.0009) [2023-12-26 19:57:18,186][105620] Updated weights for policy 1, policy_version 631015 (0.0008) [2023-12-26 19:57:18,229][105692] Updated weights for policy 0, policy_version 630174 (0.0009) [2023-12-26 19:57:18,291][105692] Updated weights for policy 0, policy_version 630184 (0.0009) [2023-12-26 19:57:18,889][105620] Updated weights for policy 1, policy_version 631025 (0.0009) [2023-12-26 19:57:18,952][105620] Updated weights for policy 1, policy_version 631035 (0.0009) [2023-12-26 19:57:19,007][105620] Updated weights for policy 1, policy_version 631046 (0.0010) [2023-12-26 19:57:19,048][105692] Updated weights for policy 0, policy_version 630194 (0.0008) [2023-12-26 19:57:19,106][105692] Updated weights for policy 0, policy_version 630204 (0.0009) [2023-12-26 19:57:19,171][105692] Updated weights for policy 0, policy_version 630214 (0.0009) [2023-12-26 19:57:19,871][105692] Updated weights for policy 0, policy_version 630224 (0.0008) [2023-12-26 19:57:19,909][105620] Updated weights for policy 1, policy_version 631056 (0.0009) [2023-12-26 19:57:19,933][105692] Updated weights for policy 0, policy_version 630234 (0.0007) [2023-12-26 19:57:19,978][105620] Updated weights for policy 1, policy_version 631066 (0.0010) [2023-12-26 19:57:19,993][105692] Updated weights for policy 0, policy_version 630244 (0.0010) [2023-12-26 19:57:20,042][105620] Updated weights for policy 1, policy_version 631076 (0.0011) [2023-12-26 19:57:20,687][105692] Updated weights for policy 0, policy_version 630254 (0.0009) [2023-12-26 19:57:20,737][105692] Updated weights for policy 0, policy_version 630264 (0.0008) [2023-12-26 19:57:20,803][105692] Updated weights for policy 0, policy_version 630274 (0.0008) [2023-12-26 19:57:20,809][105620] Updated weights for policy 1, policy_version 631086 (0.0011) [2023-12-26 19:57:20,880][105620] Updated weights for policy 1, policy_version 631096 (0.0009) [2023-12-26 19:57:20,946][105620] Updated weights for policy 1, policy_version 631106 (0.0011) [2023-12-26 19:57:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 322961408. Throughput: 0: 9697.2, 1: 9473.1. Samples: 322947400. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:57:21,063][104569] Avg episode reward: [(0, '9002.649'), (1, '9083.340')] [2023-12-26 19:57:21,520][105692] Updated weights for policy 0, policy_version 630284 (0.0006) [2023-12-26 19:57:21,583][105692] Updated weights for policy 0, policy_version 630294 (0.0006) [2023-12-26 19:57:21,652][105692] Updated weights for policy 0, policy_version 630304 (0.0008) [2023-12-26 19:57:21,718][105620] Updated weights for policy 1, policy_version 631116 (0.0010) [2023-12-26 19:57:21,783][105620] Updated weights for policy 1, policy_version 631126 (0.0007) [2023-12-26 19:57:21,842][105620] Updated weights for policy 1, policy_version 631136 (0.0011) [2023-12-26 19:57:22,398][105692] Updated weights for policy 0, policy_version 630314 (0.0009) [2023-12-26 19:57:22,462][105692] Updated weights for policy 0, policy_version 630324 (0.0008) [2023-12-26 19:57:22,527][105692] Updated weights for policy 0, policy_version 630334 (0.0008) [2023-12-26 19:57:22,590][105692] Updated weights for policy 0, policy_version 630344 (0.0008) [2023-12-26 19:57:22,599][105620] Updated weights for policy 1, policy_version 631146 (0.0011) [2023-12-26 19:57:22,652][105620] Updated weights for policy 1, policy_version 631156 (0.0011) [2023-12-26 19:57:22,713][105620] Updated weights for policy 1, policy_version 631166 (0.0011) [2023-12-26 19:57:22,773][105620] Updated weights for policy 1, policy_version 631176 (0.0011) [2023-12-26 19:57:23,253][105692] Updated weights for policy 0, policy_version 630354 (0.0010) [2023-12-26 19:57:23,304][105692] Updated weights for policy 0, policy_version 630364 (0.0010) [2023-12-26 19:57:23,352][105692] Updated weights for policy 0, policy_version 630374 (0.0009) [2023-12-26 19:57:23,548][105620] Updated weights for policy 1, policy_version 631186 (0.0010) [2023-12-26 19:57:23,603][105620] Updated weights for policy 1, policy_version 631196 (0.0010) [2023-12-26 19:57:23,672][105620] Updated weights for policy 1, policy_version 631206 (0.0010) [2023-12-26 19:57:23,920][105692] Updated weights for policy 0, policy_version 630384 (0.0006) [2023-12-26 19:57:23,975][105692] Updated weights for policy 0, policy_version 630394 (0.0007) [2023-12-26 19:57:24,030][105692] Updated weights for policy 0, policy_version 630404 (0.0011) [2023-12-26 19:57:24,358][105620] Updated weights for policy 1, policy_version 631216 (0.0011) [2023-12-26 19:57:24,414][105620] Updated weights for policy 1, policy_version 631226 (0.0010) [2023-12-26 19:57:24,469][105620] Updated weights for policy 1, policy_version 631236 (0.0010) [2023-12-26 19:57:24,584][105692] Updated weights for policy 0, policy_version 630414 (0.0009) [2023-12-26 19:57:24,634][105692] Updated weights for policy 0, policy_version 630424 (0.0008) [2023-12-26 19:57:24,682][105692] Updated weights for policy 0, policy_version 630434 (0.0008) [2023-12-26 19:57:25,231][105620] Updated weights for policy 1, policy_version 631246 (0.0010) [2023-12-26 19:57:25,283][105620] Updated weights for policy 1, policy_version 631256 (0.0010) [2023-12-26 19:57:25,328][105620] Updated weights for policy 1, policy_version 631266 (0.0010) [2023-12-26 19:57:25,353][105692] Updated weights for policy 0, policy_version 630444 (0.0007) [2023-12-26 19:57:25,404][105692] Updated weights for policy 0, policy_version 630454 (0.0005) [2023-12-26 19:57:25,459][105692] Updated weights for policy 0, policy_version 630464 (0.0005) [2023-12-26 19:57:25,988][105692] Updated weights for policy 0, policy_version 630474 (0.0006) [2023-12-26 19:57:26,042][105692] Updated weights for policy 0, policy_version 630484 (0.0009) [2023-12-26 19:57:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.3, 300 sec: 19522.0). Total num frames: 323051520. Throughput: 0: 9771.1, 1: 9419.2. Samples: 323065136. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:57:26,062][104569] Avg episode reward: [(0, '9174.307'), (1, '9175.444')] [2023-12-26 19:57:26,082][105620] Updated weights for policy 1, policy_version 631276 (0.0009) [2023-12-26 19:57:26,099][105692] Updated weights for policy 0, policy_version 630494 (0.0008) [2023-12-26 19:57:26,137][105620] Updated weights for policy 1, policy_version 631286 (0.0007) [2023-12-26 19:57:26,151][105692] Updated weights for policy 0, policy_version 630504 (0.0007) [2023-12-26 19:57:26,201][105620] Updated weights for policy 1, policy_version 631296 (0.0008) [2023-12-26 19:57:26,866][105692] Updated weights for policy 0, policy_version 630514 (0.0009) [2023-12-26 19:57:26,922][105620] Updated weights for policy 1, policy_version 631306 (0.0010) [2023-12-26 19:57:26,925][105692] Updated weights for policy 0, policy_version 630524 (0.0008) [2023-12-26 19:57:26,982][105620] Updated weights for policy 1, policy_version 631316 (0.0007) [2023-12-26 19:57:26,984][105692] Updated weights for policy 0, policy_version 630534 (0.0006) [2023-12-26 19:57:27,034][105620] Updated weights for policy 1, policy_version 631326 (0.0009) [2023-12-26 19:57:27,077][105620] Updated weights for policy 1, policy_version 631336 (0.0005) [2023-12-26 19:57:27,612][105692] Updated weights for policy 0, policy_version 630544 (0.0006) [2023-12-26 19:57:27,672][105692] Updated weights for policy 0, policy_version 630554 (0.0005) [2023-12-26 19:57:27,724][105692] Updated weights for policy 0, policy_version 630564 (0.0008) [2023-12-26 19:57:27,754][105620] Updated weights for policy 1, policy_version 631346 (0.0007) [2023-12-26 19:57:27,802][105620] Updated weights for policy 1, policy_version 631356 (0.0008) [2023-12-26 19:57:27,855][105620] Updated weights for policy 1, policy_version 631366 (0.0009) [2023-12-26 19:57:28,252][105692] Updated weights for policy 0, policy_version 630574 (0.0008) [2023-12-26 19:57:28,319][105692] Updated weights for policy 0, policy_version 630584 (0.0010) [2023-12-26 19:57:28,386][105692] Updated weights for policy 0, policy_version 630594 (0.0008) [2023-12-26 19:57:28,670][105620] Updated weights for policy 1, policy_version 631376 (0.0006) [2023-12-26 19:57:28,722][105620] Updated weights for policy 1, policy_version 631386 (0.0005) [2023-12-26 19:57:28,775][105620] Updated weights for policy 1, policy_version 631396 (0.0006) [2023-12-26 19:57:29,104][105692] Updated weights for policy 0, policy_version 630604 (0.0009) [2023-12-26 19:57:29,170][105692] Updated weights for policy 0, policy_version 630614 (0.0009) [2023-12-26 19:57:29,232][105692] Updated weights for policy 0, policy_version 630624 (0.0012) [2023-12-26 19:57:29,474][105620] Updated weights for policy 1, policy_version 631406 (0.0006) [2023-12-26 19:57:29,541][105620] Updated weights for policy 1, policy_version 631416 (0.0008) [2023-12-26 19:57:29,601][105620] Updated weights for policy 1, policy_version 631426 (0.0010) [2023-12-26 19:57:30,003][105692] Updated weights for policy 0, policy_version 630634 (0.0009) [2023-12-26 19:57:30,061][105692] Updated weights for policy 0, policy_version 630644 (0.0010) [2023-12-26 19:57:30,116][105692] Updated weights for policy 0, policy_version 630654 (0.0011) [2023-12-26 19:57:30,230][105620] Updated weights for policy 1, policy_version 631436 (0.0008) [2023-12-26 19:57:30,292][105620] Updated weights for policy 1, policy_version 631446 (0.0009) [2023-12-26 19:57:30,345][105620] Updated weights for policy 1, policy_version 631456 (0.0009) [2023-12-26 19:57:30,891][105692] Updated weights for policy 0, policy_version 630665 (0.0010) [2023-12-26 19:57:30,951][105692] Updated weights for policy 0, policy_version 630675 (0.0009) [2023-12-26 19:57:31,012][105692] Updated weights for policy 0, policy_version 630685 (0.0009) [2023-12-26 19:57:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 323149824. Throughput: 0: 9878.0, 1: 9420.8. Samples: 323127692. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:57:31,062][104569] Avg episode reward: [(0, '9263.285'), (1, '9266.957')] [2023-12-26 19:57:31,075][105620] Updated weights for policy 1, policy_version 631466 (0.0009) [2023-12-26 19:57:31,102][105692] Updated weights for policy 0, policy_version 630695 (0.0008) [2023-12-26 19:57:31,107][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000630696_161488896.pth... [2023-12-26 19:57:31,112][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000629512_161185792.pth [2023-12-26 19:57:31,139][105620] Updated weights for policy 1, policy_version 631476 (0.0007) [2023-12-26 19:57:31,195][105620] Updated weights for policy 1, policy_version 631486 (0.0007) [2023-12-26 19:57:31,253][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000631496_161677312.pth... [2023-12-26 19:57:31,254][105620] Updated weights for policy 1, policy_version 631496 (0.0005) [2023-12-26 19:57:31,257][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000630376_161390592.pth [2023-12-26 19:57:31,929][105692] Updated weights for policy 0, policy_version 630705 (0.0008) [2023-12-26 19:57:31,954][105620] Updated weights for policy 1, policy_version 631506 (0.0007) [2023-12-26 19:57:31,981][105692] Updated weights for policy 0, policy_version 630715 (0.0006) [2023-12-26 19:57:32,010][105620] Updated weights for policy 1, policy_version 631516 (0.0007) [2023-12-26 19:57:32,029][105692] Updated weights for policy 0, policy_version 630725 (0.0007) [2023-12-26 19:57:32,070][105620] Updated weights for policy 1, policy_version 631526 (0.0009) [2023-12-26 19:57:32,779][105692] Updated weights for policy 0, policy_version 630735 (0.0006) [2023-12-26 19:57:32,808][105620] Updated weights for policy 1, policy_version 631536 (0.0009) [2023-12-26 19:57:32,834][105692] Updated weights for policy 0, policy_version 630745 (0.0005) [2023-12-26 19:57:32,856][105620] Updated weights for policy 1, policy_version 631546 (0.0007) [2023-12-26 19:57:32,882][105692] Updated weights for policy 0, policy_version 630755 (0.0008) [2023-12-26 19:57:32,908][105620] Updated weights for policy 1, policy_version 631556 (0.0008) [2023-12-26 19:57:33,549][105692] Updated weights for policy 0, policy_version 630765 (0.0007) [2023-12-26 19:57:33,599][105692] Updated weights for policy 0, policy_version 630775 (0.0009) [2023-12-26 19:57:33,659][105692] Updated weights for policy 0, policy_version 630785 (0.0009) [2023-12-26 19:57:33,677][105620] Updated weights for policy 1, policy_version 631566 (0.0008) [2023-12-26 19:57:33,735][105620] Updated weights for policy 1, policy_version 631576 (0.0007) [2023-12-26 19:57:33,781][105620] Updated weights for policy 1, policy_version 631586 (0.0008) [2023-12-26 19:57:34,429][105692] Updated weights for policy 0, policy_version 630795 (0.0008) [2023-12-26 19:57:34,490][105692] Updated weights for policy 0, policy_version 630805 (0.0008) [2023-12-26 19:57:34,542][105620] Updated weights for policy 1, policy_version 631596 (0.0009) [2023-12-26 19:57:34,554][105692] Updated weights for policy 0, policy_version 630815 (0.0009) [2023-12-26 19:57:34,605][105620] Updated weights for policy 1, policy_version 631606 (0.0009) [2023-12-26 19:57:34,668][105620] Updated weights for policy 1, policy_version 631616 (0.0009) [2023-12-26 19:57:35,158][105692] Updated weights for policy 0, policy_version 630825 (0.0007) [2023-12-26 19:57:35,211][105692] Updated weights for policy 0, policy_version 630835 (0.0005) [2023-12-26 19:57:35,259][105692] Updated weights for policy 0, policy_version 630845 (0.0005) [2023-12-26 19:57:35,319][105692] Updated weights for policy 0, policy_version 630855 (0.0009) [2023-12-26 19:57:35,468][105620] Updated weights for policy 1, policy_version 631627 (0.0010) [2023-12-26 19:57:35,534][105620] Updated weights for policy 1, policy_version 631637 (0.0010) [2023-12-26 19:57:35,591][105620] Updated weights for policy 1, policy_version 631647 (0.0009) [2023-12-26 19:57:35,996][105692] Updated weights for policy 0, policy_version 630865 (0.0008) [2023-12-26 19:57:36,058][105692] Updated weights for policy 0, policy_version 630875 (0.0008) [2023-12-26 19:57:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 323248128. Throughput: 0: 9766.3, 1: 9463.8. Samples: 323240556. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:57:36,063][104569] Avg episode reward: [(0, '9081.038'), (1, '9084.854')] [2023-12-26 19:57:36,115][105692] Updated weights for policy 0, policy_version 630885 (0.0008) [2023-12-26 19:57:36,265][105620] Updated weights for policy 1, policy_version 631657 (0.0009) [2023-12-26 19:57:36,328][105620] Updated weights for policy 1, policy_version 631667 (0.0009) [2023-12-26 19:57:36,387][105620] Updated weights for policy 1, policy_version 631677 (0.0010) [2023-12-26 19:57:36,454][105620] Updated weights for policy 1, policy_version 631687 (0.0007) [2023-12-26 19:57:36,900][105692] Updated weights for policy 0, policy_version 630895 (0.0009) [2023-12-26 19:57:36,954][105692] Updated weights for policy 0, policy_version 630905 (0.0010) [2023-12-26 19:57:37,012][105692] Updated weights for policy 0, policy_version 630915 (0.0010) [2023-12-26 19:57:37,084][105620] Updated weights for policy 1, policy_version 631697 (0.0010) [2023-12-26 19:57:37,139][105620] Updated weights for policy 1, policy_version 631707 (0.0010) [2023-12-26 19:57:37,188][105620] Updated weights for policy 1, policy_version 631717 (0.0010) [2023-12-26 19:57:37,783][105620] Updated weights for policy 1, policy_version 631727 (0.0007) [2023-12-26 19:57:37,842][105620] Updated weights for policy 1, policy_version 631737 (0.0010) [2023-12-26 19:57:37,885][105692] Updated weights for policy 0, policy_version 630925 (0.0008) [2023-12-26 19:57:37,899][105620] Updated weights for policy 1, policy_version 631747 (0.0011) [2023-12-26 19:57:37,945][105692] Updated weights for policy 0, policy_version 630935 (0.0007) [2023-12-26 19:57:38,005][105692] Updated weights for policy 0, policy_version 630945 (0.0008) [2023-12-26 19:57:38,571][105620] Updated weights for policy 1, policy_version 631757 (0.0010) [2023-12-26 19:57:38,623][105620] Updated weights for policy 1, policy_version 631767 (0.0010) [2023-12-26 19:57:38,671][105620] Updated weights for policy 1, policy_version 631777 (0.0010) [2023-12-26 19:57:38,721][105692] Updated weights for policy 0, policy_version 630955 (0.0007) [2023-12-26 19:57:38,776][105692] Updated weights for policy 0, policy_version 630965 (0.0008) [2023-12-26 19:57:38,835][105692] Updated weights for policy 0, policy_version 630975 (0.0008) [2023-12-26 19:57:39,461][105620] Updated weights for policy 1, policy_version 631787 (0.0010) [2023-12-26 19:57:39,521][105620] Updated weights for policy 1, policy_version 631797 (0.0011) [2023-12-26 19:57:39,523][105692] Updated weights for policy 0, policy_version 630985 (0.0005) [2023-12-26 19:57:39,574][105620] Updated weights for policy 1, policy_version 631807 (0.0011) [2023-12-26 19:57:39,584][105692] Updated weights for policy 0, policy_version 630995 (0.0007) [2023-12-26 19:57:39,650][105692] Updated weights for policy 0, policy_version 631005 (0.0006) [2023-12-26 19:57:39,710][105692] Updated weights for policy 0, policy_version 631015 (0.0007) [2023-12-26 19:57:40,379][105620] Updated weights for policy 1, policy_version 631817 (0.0011) [2023-12-26 19:57:40,443][105620] Updated weights for policy 1, policy_version 631827 (0.0009) [2023-12-26 19:57:40,474][105692] Updated weights for policy 0, policy_version 631025 (0.0006) [2023-12-26 19:57:40,497][105620] Updated weights for policy 1, policy_version 631837 (0.0006) [2023-12-26 19:57:40,528][105692] Updated weights for policy 0, policy_version 631035 (0.0006) [2023-12-26 19:57:40,544][105620] Updated weights for policy 1, policy_version 631847 (0.0006) [2023-12-26 19:57:40,579][105692] Updated weights for policy 0, policy_version 631045 (0.0008) [2023-12-26 19:57:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19114.7, 300 sec: 19577.5). Total num frames: 323346432. Throughput: 0: 9751.5, 1: 9528.4. Samples: 323357316. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 19:57:41,063][104569] Avg episode reward: [(0, '8988.628'), (1, '8994.045')] [2023-12-26 19:57:41,302][105620] Updated weights for policy 1, policy_version 631857 (0.0009) [2023-12-26 19:57:41,358][105692] Updated weights for policy 0, policy_version 631055 (0.0009) [2023-12-26 19:57:41,365][105620] Updated weights for policy 1, policy_version 631867 (0.0008) [2023-12-26 19:57:41,414][105692] Updated weights for policy 0, policy_version 631065 (0.0007) [2023-12-26 19:57:41,429][105620] Updated weights for policy 1, policy_version 631877 (0.0007) [2023-12-26 19:57:41,466][105692] Updated weights for policy 0, policy_version 631075 (0.0008) [2023-12-26 19:57:42,167][105620] Updated weights for policy 1, policy_version 631887 (0.0008) [2023-12-26 19:57:42,221][105620] Updated weights for policy 1, policy_version 631897 (0.0008) [2023-12-26 19:57:42,266][105692] Updated weights for policy 0, policy_version 631085 (0.0008) [2023-12-26 19:57:42,284][105620] Updated weights for policy 1, policy_version 631907 (0.0009) [2023-12-26 19:57:42,330][105692] Updated weights for policy 0, policy_version 631095 (0.0009) [2023-12-26 19:57:42,398][105692] Updated weights for policy 0, policy_version 631106 (0.0009) [2023-12-26 19:57:43,015][105620] Updated weights for policy 1, policy_version 631917 (0.0008) [2023-12-26 19:57:43,071][105620] Updated weights for policy 1, policy_version 631927 (0.0010) [2023-12-26 19:57:43,111][105692] Updated weights for policy 0, policy_version 631116 (0.0006) [2023-12-26 19:57:43,122][105620] Updated weights for policy 1, policy_version 631937 (0.0010) [2023-12-26 19:57:43,167][105692] Updated weights for policy 0, policy_version 631126 (0.0005) [2023-12-26 19:57:43,215][105692] Updated weights for policy 0, policy_version 631136 (0.0005) [2023-12-26 19:57:43,767][105620] Updated weights for policy 1, policy_version 631947 (0.0009) [2023-12-26 19:57:43,815][105620] Updated weights for policy 1, policy_version 631957 (0.0005) [2023-12-26 19:57:43,865][105692] Updated weights for policy 0, policy_version 631146 (0.0006) [2023-12-26 19:57:43,866][105620] Updated weights for policy 1, policy_version 631967 (0.0005) [2023-12-26 19:57:43,929][105692] Updated weights for policy 0, policy_version 631156 (0.0005) [2023-12-26 19:57:43,998][105692] Updated weights for policy 0, policy_version 631166 (0.0005) [2023-12-26 19:57:44,058][105692] Updated weights for policy 0, policy_version 631176 (0.0005) [2023-12-26 19:57:44,574][105620] Updated weights for policy 1, policy_version 631977 (0.0009) [2023-12-26 19:57:44,639][105620] Updated weights for policy 1, policy_version 631987 (0.0010) [2023-12-26 19:57:44,696][105620] Updated weights for policy 1, policy_version 631997 (0.0010) [2023-12-26 19:57:44,710][105692] Updated weights for policy 0, policy_version 631186 (0.0006) [2023-12-26 19:57:44,754][105620] Updated weights for policy 1, policy_version 632007 (0.0010) [2023-12-26 19:57:44,764][105692] Updated weights for policy 0, policy_version 631196 (0.0007) [2023-12-26 19:57:44,835][105692] Updated weights for policy 0, policy_version 631206 (0.0008) [2023-12-26 19:57:45,511][105620] Updated weights for policy 1, policy_version 632017 (0.0006) [2023-12-26 19:57:45,571][105620] Updated weights for policy 1, policy_version 632027 (0.0006) [2023-12-26 19:57:45,626][105620] Updated weights for policy 1, policy_version 632037 (0.0008) [2023-12-26 19:57:45,630][105692] Updated weights for policy 0, policy_version 631216 (0.0009) [2023-12-26 19:57:45,683][105692] Updated weights for policy 0, policy_version 631226 (0.0008) [2023-12-26 19:57:45,734][105692] Updated weights for policy 0, policy_version 631236 (0.0008) [2023-12-26 19:57:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 323444736. Throughput: 0: 9778.7, 1: 9538.8. Samples: 323415860. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-26 19:57:46,062][104569] Avg episode reward: [(0, '8994.883'), (1, '9084.837')] [2023-12-26 19:57:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000631240_161628160.pth... [2023-12-26 19:57:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000632040_161816576.pth... [2023-12-26 19:57:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000630920_161529856.pth [2023-12-26 19:57:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000630088_161333248.pth [2023-12-26 19:57:46,310][105620] Updated weights for policy 1, policy_version 632047 (0.0010) [2023-12-26 19:57:46,367][105620] Updated weights for policy 1, policy_version 632057 (0.0010) [2023-12-26 19:57:46,426][105620] Updated weights for policy 1, policy_version 632067 (0.0010) [2023-12-26 19:57:46,429][105692] Updated weights for policy 0, policy_version 631246 (0.0007) [2023-12-26 19:57:46,489][105692] Updated weights for policy 0, policy_version 631256 (0.0007) [2023-12-26 19:57:46,547][105692] Updated weights for policy 0, policy_version 631266 (0.0008) [2023-12-26 19:57:47,142][105620] Updated weights for policy 1, policy_version 632077 (0.0010) [2023-12-26 19:57:47,193][105620] Updated weights for policy 1, policy_version 632087 (0.0010) [2023-12-26 19:57:47,245][105620] Updated weights for policy 1, policy_version 632097 (0.0010) [2023-12-26 19:57:47,299][105692] Updated weights for policy 0, policy_version 631276 (0.0008) [2023-12-26 19:57:47,363][105692] Updated weights for policy 0, policy_version 631286 (0.0008) [2023-12-26 19:57:47,432][105692] Updated weights for policy 0, policy_version 631296 (0.0007) [2023-12-26 19:57:47,998][105620] Updated weights for policy 1, policy_version 632107 (0.0010) [2023-12-26 19:57:48,056][105620] Updated weights for policy 1, policy_version 632117 (0.0010) [2023-12-26 19:57:48,117][105620] Updated weights for policy 1, policy_version 632127 (0.0010) [2023-12-26 19:57:48,165][105692] Updated weights for policy 0, policy_version 631306 (0.0008) [2023-12-26 19:57:48,209][105692] Updated weights for policy 0, policy_version 631316 (0.0008) [2023-12-26 19:57:48,258][105692] Updated weights for policy 0, policy_version 631326 (0.0008) [2023-12-26 19:57:48,309][105692] Updated weights for policy 0, policy_version 631336 (0.0006) [2023-12-26 19:57:48,829][105620] Updated weights for policy 1, policy_version 632137 (0.0010) [2023-12-26 19:57:48,890][105620] Updated weights for policy 1, policy_version 632147 (0.0006) [2023-12-26 19:57:48,954][105620] Updated weights for policy 1, policy_version 632157 (0.0006) [2023-12-26 19:57:49,014][105620] Updated weights for policy 1, policy_version 632167 (0.0010) [2023-12-26 19:57:49,127][105692] Updated weights for policy 0, policy_version 631346 (0.0008) [2023-12-26 19:57:49,186][105692] Updated weights for policy 0, policy_version 631356 (0.0008) [2023-12-26 19:57:49,252][105692] Updated weights for policy 0, policy_version 631366 (0.0008) [2023-12-26 19:57:49,721][105620] Updated weights for policy 1, policy_version 632177 (0.0010) [2023-12-26 19:57:49,770][105620] Updated weights for policy 1, policy_version 632187 (0.0006) [2023-12-26 19:57:49,830][105620] Updated weights for policy 1, policy_version 632197 (0.0006) [2023-12-26 19:57:50,074][105692] Updated weights for policy 0, policy_version 631376 (0.0009) [2023-12-26 19:57:50,129][105692] Updated weights for policy 0, policy_version 631386 (0.0009) [2023-12-26 19:57:50,191][105692] Updated weights for policy 0, policy_version 631396 (0.0009) [2023-12-26 19:57:50,485][105620] Updated weights for policy 1, policy_version 632207 (0.0009) [2023-12-26 19:57:50,553][105620] Updated weights for policy 1, policy_version 632217 (0.0006) [2023-12-26 19:57:50,623][105620] Updated weights for policy 1, policy_version 632227 (0.0008) [2023-12-26 19:57:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19114.7, 300 sec: 19522.0). Total num frames: 323534848. Throughput: 0: 9697.1, 1: 9544.3. Samples: 323529188. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-26 19:57:51,062][104569] Avg episode reward: [(0, '9178.691'), (1, '9268.092')] [2023-12-26 19:57:51,068][105692] Updated weights for policy 0, policy_version 631406 (0.0009) [2023-12-26 19:57:51,131][105692] Updated weights for policy 0, policy_version 631416 (0.0009) [2023-12-26 19:57:51,199][105692] Updated weights for policy 0, policy_version 631426 (0.0008) [2023-12-26 19:57:51,218][105620] Updated weights for policy 1, policy_version 632237 (0.0008) [2023-12-26 19:57:51,279][105620] Updated weights for policy 1, policy_version 632247 (0.0009) [2023-12-26 19:57:51,345][105620] Updated weights for policy 1, policy_version 632257 (0.0009) [2023-12-26 19:57:51,939][105692] Updated weights for policy 0, policy_version 631436 (0.0007) [2023-12-26 19:57:52,002][105692] Updated weights for policy 0, policy_version 631446 (0.0009) [2023-12-26 19:57:52,062][105692] Updated weights for policy 0, policy_version 631456 (0.0009) [2023-12-26 19:57:52,187][105620] Updated weights for policy 1, policy_version 632267 (0.0008) [2023-12-26 19:57:52,245][105620] Updated weights for policy 1, policy_version 632277 (0.0009) [2023-12-26 19:57:52,308][105620] Updated weights for policy 1, policy_version 632287 (0.0007) [2023-12-26 19:57:52,718][105692] Updated weights for policy 0, policy_version 631466 (0.0009) [2023-12-26 19:57:52,777][105692] Updated weights for policy 0, policy_version 631476 (0.0010) [2023-12-26 19:57:52,830][105692] Updated weights for policy 0, policy_version 631486 (0.0010) [2023-12-26 19:57:52,884][105692] Updated weights for policy 0, policy_version 631496 (0.0009) [2023-12-26 19:57:52,986][105620] Updated weights for policy 1, policy_version 632297 (0.0007) [2023-12-26 19:57:53,048][105620] Updated weights for policy 1, policy_version 632307 (0.0009) [2023-12-26 19:57:53,098][105620] Updated weights for policy 1, policy_version 632317 (0.0008) [2023-12-26 19:57:53,150][105620] Updated weights for policy 1, policy_version 632327 (0.0008) [2023-12-26 19:57:53,693][105692] Updated weights for policy 0, policy_version 631506 (0.0009) [2023-12-26 19:57:53,748][105692] Updated weights for policy 0, policy_version 631516 (0.0008) [2023-12-26 19:57:53,799][105692] Updated weights for policy 0, policy_version 631526 (0.0009) [2023-12-26 19:57:53,875][105620] Updated weights for policy 1, policy_version 632337 (0.0009) [2023-12-26 19:57:53,933][105620] Updated weights for policy 1, policy_version 632347 (0.0008) [2023-12-26 19:57:53,991][105620] Updated weights for policy 1, policy_version 632357 (0.0009) [2023-12-26 19:57:54,555][105692] Updated weights for policy 0, policy_version 631536 (0.0009) [2023-12-26 19:57:54,613][105692] Updated weights for policy 0, policy_version 631546 (0.0009) [2023-12-26 19:57:54,673][105692] Updated weights for policy 0, policy_version 631556 (0.0008) [2023-12-26 19:57:54,752][105620] Updated weights for policy 1, policy_version 632367 (0.0009) [2023-12-26 19:57:54,805][105620] Updated weights for policy 1, policy_version 632378 (0.0010) [2023-12-26 19:57:54,855][105620] Updated weights for policy 1, policy_version 632389 (0.0010) [2023-12-26 19:57:55,346][105692] Updated weights for policy 0, policy_version 631566 (0.0008) [2023-12-26 19:57:55,408][105692] Updated weights for policy 0, policy_version 631576 (0.0006) [2023-12-26 19:57:55,456][105692] Updated weights for policy 0, policy_version 631586 (0.0005) [2023-12-26 19:57:55,623][105620] Updated weights for policy 1, policy_version 632399 (0.0009) [2023-12-26 19:57:55,672][105620] Updated weights for policy 1, policy_version 632409 (0.0008) [2023-12-26 19:57:55,721][105620] Updated weights for policy 1, policy_version 632419 (0.0008) [2023-12-26 19:57:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 323633152. Throughput: 0: 9679.8, 1: 9528.7. Samples: 323642916. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-26 19:57:56,063][104569] Avg episode reward: [(0, '9053.718'), (1, '9079.971')] [2023-12-26 19:57:56,065][105692] Updated weights for policy 0, policy_version 631596 (0.0006) [2023-12-26 19:57:56,139][105692] Updated weights for policy 0, policy_version 631606 (0.0005) [2023-12-26 19:57:56,194][105692] Updated weights for policy 0, policy_version 631616 (0.0006) [2023-12-26 19:57:56,433][105620] Updated weights for policy 1, policy_version 632429 (0.0009) [2023-12-26 19:57:56,488][105620] Updated weights for policy 1, policy_version 632439 (0.0009) [2023-12-26 19:57:56,544][105620] Updated weights for policy 1, policy_version 632449 (0.0009) [2023-12-26 19:57:56,715][105692] Updated weights for policy 0, policy_version 631626 (0.0007) [2023-12-26 19:57:56,770][105692] Updated weights for policy 0, policy_version 631636 (0.0010) [2023-12-26 19:57:56,821][105692] Updated weights for policy 0, policy_version 631646 (0.0005) [2023-12-26 19:57:56,872][105692] Updated weights for policy 0, policy_version 631656 (0.0005) [2023-12-26 19:57:57,413][105692] Updated weights for policy 0, policy_version 631666 (0.0006) [2023-12-26 19:57:57,452][105620] Updated weights for policy 1, policy_version 632459 (0.0009) [2023-12-26 19:57:57,471][105692] Updated weights for policy 0, policy_version 631676 (0.0006) [2023-12-26 19:57:57,500][105620] Updated weights for policy 1, policy_version 632469 (0.0008) [2023-12-26 19:57:57,538][105692] Updated weights for policy 0, policy_version 631686 (0.0005) [2023-12-26 19:57:57,545][105620] Updated weights for policy 1, policy_version 632479 (0.0010) [2023-12-26 19:57:58,057][105692] Updated weights for policy 0, policy_version 631696 (0.0005) [2023-12-26 19:57:58,118][105692] Updated weights for policy 0, policy_version 631706 (0.0006) [2023-12-26 19:57:58,184][105692] Updated weights for policy 0, policy_version 631716 (0.0007) [2023-12-26 19:57:58,282][105620] Updated weights for policy 1, policy_version 632489 (0.0010) [2023-12-26 19:57:58,353][105620] Updated weights for policy 1, policy_version 632499 (0.0008) [2023-12-26 19:57:58,411][105620] Updated weights for policy 1, policy_version 632509 (0.0007) [2023-12-26 19:57:58,479][105620] Updated weights for policy 1, policy_version 632519 (0.0010) [2023-12-26 19:57:58,981][105692] Updated weights for policy 0, policy_version 631727 (0.0008) [2023-12-26 19:57:58,988][105585] KL-divergence is very high: 281.0354 [2023-12-26 19:57:59,047][105585] KL-divergence is very high: 518.2013 [2023-12-26 19:57:59,054][105692] Updated weights for policy 0, policy_version 631737 (0.0007) [2023-12-26 19:57:59,097][105585] KL-divergence is very high: 541.2001 [2023-12-26 19:57:59,115][105692] Updated weights for policy 0, policy_version 631747 (0.0006) [2023-12-26 19:57:59,274][105620] Updated weights for policy 1, policy_version 632529 (0.0009) [2023-12-26 19:57:59,340][105620] Updated weights for policy 1, policy_version 632539 (0.0008) [2023-12-26 19:57:59,403][105620] Updated weights for policy 1, policy_version 632549 (0.0009) [2023-12-26 19:57:59,800][105692] Updated weights for policy 0, policy_version 631757 (0.0006) [2023-12-26 19:57:59,868][105692] Updated weights for policy 0, policy_version 631767 (0.0008) [2023-12-26 19:57:59,945][105692] Updated weights for policy 0, policy_version 631777 (0.0008) [2023-12-26 19:58:00,147][105620] Updated weights for policy 1, policy_version 632559 (0.0006) [2023-12-26 19:58:00,194][105620] Updated weights for policy 1, policy_version 632569 (0.0005) [2023-12-26 19:58:00,261][105620] Updated weights for policy 1, policy_version 632579 (0.0005) [2023-12-26 19:58:00,606][105692] Updated weights for policy 0, policy_version 631787 (0.0009) [2023-12-26 19:58:00,654][105692] Updated weights for policy 0, policy_version 631797 (0.0010) [2023-12-26 19:58:00,713][105692] Updated weights for policy 0, policy_version 631807 (0.0009) [2023-12-26 19:58:00,852][105620] Updated weights for policy 1, policy_version 632589 (0.0007) [2023-12-26 19:58:00,919][105620] Updated weights for policy 1, policy_version 632599 (0.0009) [2023-12-26 19:58:00,986][105620] Updated weights for policy 1, policy_version 632609 (0.0008) [2023-12-26 19:58:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 323739648. Throughput: 0: 9850.2, 1: 9531.6. Samples: 323704948. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-26 19:58:01,062][104569] Avg episode reward: [(0, '8835.687'), (1, '8811.843')] [2023-12-26 19:58:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000631816_161775616.pth... [2023-12-26 19:58:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000632616_161964032.pth... [2023-12-26 19:58:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000630696_161488896.pth [2023-12-26 19:58:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000631496_161677312.pth [2023-12-26 19:58:01,398][105692] Updated weights for policy 0, policy_version 631817 (0.0008) [2023-12-26 19:58:01,451][105692] Updated weights for policy 0, policy_version 631827 (0.0007) [2023-12-26 19:58:01,511][105692] Updated weights for policy 0, policy_version 631837 (0.0011) [2023-12-26 19:58:01,565][105692] Updated weights for policy 0, policy_version 631847 (0.0010) [2023-12-26 19:58:01,797][105620] Updated weights for policy 1, policy_version 632619 (0.0010) [2023-12-26 19:58:01,843][105620] Updated weights for policy 1, policy_version 632629 (0.0009) [2023-12-26 19:58:01,893][105620] Updated weights for policy 1, policy_version 632639 (0.0009) [2023-12-26 19:58:02,309][105692] Updated weights for policy 0, policy_version 631857 (0.0008) [2023-12-26 19:58:02,378][105692] Updated weights for policy 0, policy_version 631867 (0.0008) [2023-12-26 19:58:02,444][105692] Updated weights for policy 0, policy_version 631877 (0.0009) [2023-12-26 19:58:02,648][105620] Updated weights for policy 1, policy_version 632649 (0.0008) [2023-12-26 19:58:02,708][105620] Updated weights for policy 1, policy_version 632659 (0.0005) [2023-12-26 19:58:02,769][105620] Updated weights for policy 1, policy_version 632669 (0.0005) [2023-12-26 19:58:02,828][105620] Updated weights for policy 1, policy_version 632679 (0.0005) [2023-12-26 19:58:03,120][105692] Updated weights for policy 0, policy_version 631887 (0.0007) [2023-12-26 19:58:03,166][105692] Updated weights for policy 0, policy_version 631897 (0.0005) [2023-12-26 19:58:03,234][105692] Updated weights for policy 0, policy_version 631907 (0.0007) [2023-12-26 19:58:03,328][105620] Updated weights for policy 1, policy_version 632689 (0.0005) [2023-12-26 19:58:03,391][105620] Updated weights for policy 1, policy_version 632699 (0.0008) [2023-12-26 19:58:03,453][105620] Updated weights for policy 1, policy_version 632709 (0.0006) [2023-12-26 19:58:03,878][105692] Updated weights for policy 0, policy_version 631917 (0.0011) [2023-12-26 19:58:03,940][105692] Updated weights for policy 0, policy_version 631927 (0.0009) [2023-12-26 19:58:04,008][105692] Updated weights for policy 0, policy_version 631937 (0.0006) [2023-12-26 19:58:04,063][105620] Updated weights for policy 1, policy_version 632719 (0.0006) [2023-12-26 19:58:04,125][105620] Updated weights for policy 1, policy_version 632729 (0.0009) [2023-12-26 19:58:04,187][105620] Updated weights for policy 1, policy_version 632739 (0.0008) [2023-12-26 19:58:04,701][105692] Updated weights for policy 0, policy_version 631947 (0.0008) [2023-12-26 19:58:04,759][105692] Updated weights for policy 0, policy_version 631957 (0.0008) [2023-12-26 19:58:04,811][105692] Updated weights for policy 0, policy_version 631967 (0.0010) [2023-12-26 19:58:04,868][105620] Updated weights for policy 1, policy_version 632749 (0.0007) [2023-12-26 19:58:04,916][105620] Updated weights for policy 1, policy_version 632759 (0.0005) [2023-12-26 19:58:04,969][105620] Updated weights for policy 1, policy_version 632769 (0.0005) [2023-12-26 19:58:05,400][105692] Updated weights for policy 0, policy_version 631977 (0.0010) [2023-12-26 19:58:05,457][105692] Updated weights for policy 0, policy_version 631987 (0.0005) [2023-12-26 19:58:05,501][105620] Updated weights for policy 1, policy_version 632779 (0.0007) [2023-12-26 19:58:05,510][105692] Updated weights for policy 0, policy_version 631997 (0.0005) [2023-12-26 19:58:05,553][105620] Updated weights for policy 1, policy_version 632789 (0.0009) [2023-12-26 19:58:05,564][105692] Updated weights for policy 0, policy_version 632007 (0.0005) [2023-12-26 19:58:05,608][105620] Updated weights for policy 1, policy_version 632799 (0.0008) [2023-12-26 19:58:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 323837952. Throughput: 0: 9856.7, 1: 9646.8. Samples: 323825056. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-26 19:58:06,062][104569] Avg episode reward: [(0, '8925.832'), (1, '9000.043')] [2023-12-26 19:58:06,080][105692] Updated weights for policy 0, policy_version 632017 (0.0005) [2023-12-26 19:58:06,139][105692] Updated weights for policy 0, policy_version 632027 (0.0010) [2023-12-26 19:58:06,205][105692] Updated weights for policy 0, policy_version 632037 (0.0008) [2023-12-26 19:58:06,446][105620] Updated weights for policy 1, policy_version 632809 (0.0010) [2023-12-26 19:58:06,500][105620] Updated weights for policy 1, policy_version 632819 (0.0011) [2023-12-26 19:58:06,557][105620] Updated weights for policy 1, policy_version 632829 (0.0011) [2023-12-26 19:58:06,618][105620] Updated weights for policy 1, policy_version 632839 (0.0011) [2023-12-26 19:58:06,906][105692] Updated weights for policy 0, policy_version 632047 (0.0010) [2023-12-26 19:58:06,973][105692] Updated weights for policy 0, policy_version 632057 (0.0011) [2023-12-26 19:58:07,039][105692] Updated weights for policy 0, policy_version 632067 (0.0011) [2023-12-26 19:58:07,362][105620] Updated weights for policy 1, policy_version 632849 (0.0010) [2023-12-26 19:58:07,417][105620] Updated weights for policy 1, policy_version 632859 (0.0007) [2023-12-26 19:58:07,468][105620] Updated weights for policy 1, policy_version 632869 (0.0005) [2023-12-26 19:58:07,775][105692] Updated weights for policy 0, policy_version 632077 (0.0010) [2023-12-26 19:58:07,837][105692] Updated weights for policy 0, policy_version 632087 (0.0010) [2023-12-26 19:58:07,891][105692] Updated weights for policy 0, policy_version 632097 (0.0010) [2023-12-26 19:58:08,076][105620] Updated weights for policy 1, policy_version 632879 (0.0009) [2023-12-26 19:58:08,143][105620] Updated weights for policy 1, policy_version 632889 (0.0010) [2023-12-26 19:58:08,206][105620] Updated weights for policy 1, policy_version 632899 (0.0007) [2023-12-26 19:58:08,632][105692] Updated weights for policy 0, policy_version 632107 (0.0010) [2023-12-26 19:58:08,679][105692] Updated weights for policy 0, policy_version 632117 (0.0008) [2023-12-26 19:58:08,731][105692] Updated weights for policy 0, policy_version 632127 (0.0009) [2023-12-26 19:58:08,900][105620] Updated weights for policy 1, policy_version 632909 (0.0007) [2023-12-26 19:58:08,962][105620] Updated weights for policy 1, policy_version 632919 (0.0008) [2023-12-26 19:58:09,020][105620] Updated weights for policy 1, policy_version 632929 (0.0008) [2023-12-26 19:58:09,533][105692] Updated weights for policy 0, policy_version 632137 (0.0010) [2023-12-26 19:58:09,596][105692] Updated weights for policy 0, policy_version 632147 (0.0008) [2023-12-26 19:58:09,656][105692] Updated weights for policy 0, policy_version 632157 (0.0008) [2023-12-26 19:58:09,707][105692] Updated weights for policy 0, policy_version 632167 (0.0008) [2023-12-26 19:58:09,785][105620] Updated weights for policy 1, policy_version 632939 (0.0007) [2023-12-26 19:58:09,849][105620] Updated weights for policy 1, policy_version 632949 (0.0007) [2023-12-26 19:58:09,908][105620] Updated weights for policy 1, policy_version 632959 (0.0011) [2023-12-26 19:58:10,521][105620] Updated weights for policy 1, policy_version 632969 (0.0009) [2023-12-26 19:58:10,568][105692] Updated weights for policy 0, policy_version 632177 (0.0009) [2023-12-26 19:58:10,577][105620] Updated weights for policy 1, policy_version 632979 (0.0005) [2023-12-26 19:58:10,628][105692] Updated weights for policy 0, policy_version 632187 (0.0009) [2023-12-26 19:58:10,635][105620] Updated weights for policy 1, policy_version 632989 (0.0005) [2023-12-26 19:58:10,689][105692] Updated weights for policy 0, policy_version 632197 (0.0007) [2023-12-26 19:58:10,693][105620] Updated weights for policy 1, policy_version 632999 (0.0008) [2023-12-26 19:58:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 323936256. Throughput: 0: 9777.6, 1: 9764.3. Samples: 323944524. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-26 19:58:11,063][104569] Avg episode reward: [(0, '9263.829'), (1, '8991.898')] [2023-12-26 19:58:11,392][105692] Updated weights for policy 0, policy_version 632207 (0.0009) [2023-12-26 19:58:11,412][105620] Updated weights for policy 1, policy_version 633009 (0.0010) [2023-12-26 19:58:11,452][105692] Updated weights for policy 0, policy_version 632217 (0.0009) [2023-12-26 19:58:11,479][105620] Updated weights for policy 1, policy_version 633019 (0.0007) [2023-12-26 19:58:11,518][105692] Updated weights for policy 0, policy_version 632227 (0.0008) [2023-12-26 19:58:11,546][105620] Updated weights for policy 1, policy_version 633029 (0.0009) [2023-12-26 19:58:12,203][105620] Updated weights for policy 1, policy_version 633039 (0.0007) [2023-12-26 19:58:12,273][105620] Updated weights for policy 1, policy_version 633049 (0.0007) [2023-12-26 19:58:12,313][105692] Updated weights for policy 0, policy_version 632237 (0.0007) [2023-12-26 19:58:12,333][105620] Updated weights for policy 1, policy_version 633059 (0.0007) [2023-12-26 19:58:12,376][105692] Updated weights for policy 0, policy_version 632247 (0.0007) [2023-12-26 19:58:12,438][105692] Updated weights for policy 0, policy_version 632257 (0.0009) [2023-12-26 19:58:13,002][105620] Updated weights for policy 1, policy_version 633069 (0.0006) [2023-12-26 19:58:13,056][105620] Updated weights for policy 1, policy_version 633079 (0.0007) [2023-12-26 19:58:13,102][105620] Updated weights for policy 1, policy_version 633089 (0.0008) [2023-12-26 19:58:13,222][105692] Updated weights for policy 0, policy_version 632267 (0.0009) [2023-12-26 19:58:13,272][105692] Updated weights for policy 0, policy_version 632277 (0.0008) [2023-12-26 19:58:13,332][105692] Updated weights for policy 0, policy_version 632287 (0.0009) [2023-12-26 19:58:13,728][105620] Updated weights for policy 1, policy_version 633099 (0.0008) [2023-12-26 19:58:13,782][105620] Updated weights for policy 1, policy_version 633109 (0.0005) [2023-12-26 19:58:13,828][105620] Updated weights for policy 1, policy_version 633119 (0.0005) [2023-12-26 19:58:14,117][105692] Updated weights for policy 0, policy_version 632297 (0.0006) [2023-12-26 19:58:14,172][105692] Updated weights for policy 0, policy_version 632307 (0.0008) [2023-12-26 19:58:14,225][105692] Updated weights for policy 0, policy_version 632317 (0.0008) [2023-12-26 19:58:14,269][105692] Updated weights for policy 0, policy_version 632327 (0.0008) [2023-12-26 19:58:14,530][105620] Updated weights for policy 1, policy_version 633129 (0.0006) [2023-12-26 19:58:14,575][105620] Updated weights for policy 1, policy_version 633139 (0.0010) [2023-12-26 19:58:14,626][105620] Updated weights for policy 1, policy_version 633149 (0.0010) [2023-12-26 19:58:14,681][105620] Updated weights for policy 1, policy_version 633159 (0.0010) [2023-12-26 19:58:15,030][105692] Updated weights for policy 0, policy_version 632337 (0.0007) [2023-12-26 19:58:15,091][105692] Updated weights for policy 0, policy_version 632347 (0.0006) [2023-12-26 19:58:15,150][105692] Updated weights for policy 0, policy_version 632357 (0.0006) [2023-12-26 19:58:15,466][105620] Updated weights for policy 1, policy_version 633169 (0.0011) [2023-12-26 19:58:15,533][105620] Updated weights for policy 1, policy_version 633179 (0.0011) [2023-12-26 19:58:15,586][105620] Updated weights for policy 1, policy_version 633189 (0.0011) [2023-12-26 19:58:15,822][105692] Updated weights for policy 0, policy_version 632367 (0.0009) [2023-12-26 19:58:15,877][105692] Updated weights for policy 0, policy_version 632377 (0.0010) [2023-12-26 19:58:15,925][105692] Updated weights for policy 0, policy_version 632387 (0.0010) [2023-12-26 19:58:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 324034560. Throughput: 0: 9642.8, 1: 9803.4. Samples: 324002768. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-26 19:58:16,062][104569] Avg episode reward: [(0, '9175.159'), (1, '8808.704')] [2023-12-26 19:58:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000632392_161923072.pth... [2023-12-26 19:58:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000633192_162111488.pth... [2023-12-26 19:58:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000632040_161816576.pth [2023-12-26 19:58:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000631240_161628160.pth [2023-12-26 19:58:16,403][105620] Updated weights for policy 1, policy_version 633199 (0.0010) [2023-12-26 19:58:16,455][105620] Updated weights for policy 1, policy_version 633209 (0.0009) [2023-12-26 19:58:16,504][105692] Updated weights for policy 0, policy_version 632397 (0.0006) [2023-12-26 19:58:16,509][105620] Updated weights for policy 1, policy_version 633219 (0.0009) [2023-12-26 19:58:16,563][105692] Updated weights for policy 0, policy_version 632407 (0.0005) [2023-12-26 19:58:16,622][105692] Updated weights for policy 0, policy_version 632417 (0.0006) [2023-12-26 19:58:17,155][105620] Updated weights for policy 1, policy_version 633229 (0.0007) [2023-12-26 19:58:17,215][105620] Updated weights for policy 1, policy_version 633239 (0.0008) [2023-12-26 19:58:17,280][105620] Updated weights for policy 1, policy_version 633249 (0.0009) [2023-12-26 19:58:17,282][105692] Updated weights for policy 0, policy_version 632427 (0.0008) [2023-12-26 19:58:17,339][105692] Updated weights for policy 0, policy_version 632437 (0.0006) [2023-12-26 19:58:17,407][105692] Updated weights for policy 0, policy_version 632447 (0.0009) [2023-12-26 19:58:17,959][105620] Updated weights for policy 1, policy_version 633259 (0.0009) [2023-12-26 19:58:18,032][105620] Updated weights for policy 1, policy_version 633269 (0.0009) [2023-12-26 19:58:18,096][105620] Updated weights for policy 1, policy_version 633279 (0.0008) [2023-12-26 19:58:18,103][105692] Updated weights for policy 0, policy_version 632457 (0.0010) [2023-12-26 19:58:18,148][105692] Updated weights for policy 0, policy_version 632467 (0.0006) [2023-12-26 19:58:18,196][105692] Updated weights for policy 0, policy_version 632477 (0.0009) [2023-12-26 19:58:18,243][105692] Updated weights for policy 0, policy_version 632487 (0.0009) [2023-12-26 19:58:18,797][105620] Updated weights for policy 1, policy_version 633289 (0.0008) [2023-12-26 19:58:18,865][105620] Updated weights for policy 1, policy_version 633299 (0.0009) [2023-12-26 19:58:18,927][105692] Updated weights for policy 0, policy_version 632497 (0.0008) [2023-12-26 19:58:18,933][105620] Updated weights for policy 1, policy_version 633309 (0.0009) [2023-12-26 19:58:18,982][105692] Updated weights for policy 0, policy_version 632507 (0.0007) [2023-12-26 19:58:19,000][105620] Updated weights for policy 1, policy_version 633319 (0.0011) [2023-12-26 19:58:19,047][105692] Updated weights for policy 0, policy_version 632517 (0.0006) [2023-12-26 19:58:19,727][105692] Updated weights for policy 0, policy_version 632527 (0.0007) [2023-12-26 19:58:19,791][105692] Updated weights for policy 0, policy_version 632537 (0.0006) [2023-12-26 19:58:19,798][105620] Updated weights for policy 1, policy_version 633329 (0.0007) [2023-12-26 19:58:19,863][105692] Updated weights for policy 0, policy_version 632547 (0.0008) [2023-12-26 19:58:19,870][105620] Updated weights for policy 1, policy_version 633339 (0.0009) [2023-12-26 19:58:19,937][105620] Updated weights for policy 1, policy_version 633349 (0.0009) [2023-12-26 19:58:20,458][105692] Updated weights for policy 0, policy_version 632557 (0.0008) [2023-12-26 19:58:20,517][105692] Updated weights for policy 0, policy_version 632567 (0.0006) [2023-12-26 19:58:20,574][105692] Updated weights for policy 0, policy_version 632577 (0.0006) [2023-12-26 19:58:20,585][105620] Updated weights for policy 1, policy_version 633359 (0.0009) [2023-12-26 19:58:20,647][105620] Updated weights for policy 1, policy_version 633369 (0.0009) [2023-12-26 19:58:20,712][105620] Updated weights for policy 1, policy_version 633379 (0.0008) [2023-12-26 19:58:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 324132864. Throughput: 0: 9763.2, 1: 9794.0. Samples: 324120628. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-26 19:58:21,062][104569] Avg episode reward: [(0, '9175.381'), (1, '9084.040')] [2023-12-26 19:58:21,353][105692] Updated weights for policy 0, policy_version 632587 (0.0009) [2023-12-26 19:58:21,420][105692] Updated weights for policy 0, policy_version 632597 (0.0009) [2023-12-26 19:58:21,480][105692] Updated weights for policy 0, policy_version 632607 (0.0011) [2023-12-26 19:58:21,483][105620] Updated weights for policy 1, policy_version 633389 (0.0006) [2023-12-26 19:58:21,543][105620] Updated weights for policy 1, policy_version 633399 (0.0007) [2023-12-26 19:58:21,599][105620] Updated weights for policy 1, policy_version 633409 (0.0006) [2023-12-26 19:58:22,223][105692] Updated weights for policy 0, policy_version 632617 (0.0011) [2023-12-26 19:58:22,290][105692] Updated weights for policy 0, policy_version 632627 (0.0012) [2023-12-26 19:58:22,351][105692] Updated weights for policy 0, policy_version 632637 (0.0011) [2023-12-26 19:58:22,382][105620] Updated weights for policy 1, policy_version 633419 (0.0009) [2023-12-26 19:58:22,424][105692] Updated weights for policy 0, policy_version 632647 (0.0010) [2023-12-26 19:58:22,446][105620] Updated weights for policy 1, policy_version 633429 (0.0007) [2023-12-26 19:58:22,510][105620] Updated weights for policy 1, policy_version 633439 (0.0008) [2023-12-26 19:58:23,166][105692] Updated weights for policy 0, policy_version 632657 (0.0011) [2023-12-26 19:58:23,210][105620] Updated weights for policy 1, policy_version 633449 (0.0009) [2023-12-26 19:58:23,224][105692] Updated weights for policy 0, policy_version 632667 (0.0009) [2023-12-26 19:58:23,262][105620] Updated weights for policy 1, policy_version 633459 (0.0009) [2023-12-26 19:58:23,285][105692] Updated weights for policy 0, policy_version 632677 (0.0009) [2023-12-26 19:58:23,307][105620] Updated weights for policy 1, policy_version 633469 (0.0006) [2023-12-26 19:58:23,364][105620] Updated weights for policy 1, policy_version 633479 (0.0009) [2023-12-26 19:58:23,912][105692] Updated weights for policy 0, policy_version 632687 (0.0007) [2023-12-26 19:58:23,974][105692] Updated weights for policy 0, policy_version 632697 (0.0006) [2023-12-26 19:58:24,043][105692] Updated weights for policy 0, policy_version 632707 (0.0005) [2023-12-26 19:58:24,223][105620] Updated weights for policy 1, policy_version 633489 (0.0009) [2023-12-26 19:58:24,278][105620] Updated weights for policy 1, policy_version 633499 (0.0009) [2023-12-26 19:58:24,327][105620] Updated weights for policy 1, policy_version 633509 (0.0008) [2023-12-26 19:58:24,662][105692] Updated weights for policy 0, policy_version 632717 (0.0008) [2023-12-26 19:58:24,706][105692] Updated weights for policy 0, policy_version 632727 (0.0010) [2023-12-26 19:58:24,751][105692] Updated weights for policy 0, policy_version 632737 (0.0010) [2023-12-26 19:58:25,072][105620] Updated weights for policy 1, policy_version 633519 (0.0009) [2023-12-26 19:58:25,129][105620] Updated weights for policy 1, policy_version 633529 (0.0008) [2023-12-26 19:58:25,192][105620] Updated weights for policy 1, policy_version 633539 (0.0008) [2023-12-26 19:58:25,479][105692] Updated weights for policy 0, policy_version 632747 (0.0010) [2023-12-26 19:58:25,526][105692] Updated weights for policy 0, policy_version 632757 (0.0009) [2023-12-26 19:58:25,574][105692] Updated weights for policy 0, policy_version 632767 (0.0009) [2023-12-26 19:58:25,908][105620] Updated weights for policy 1, policy_version 633549 (0.0009) [2023-12-26 19:58:25,968][105620] Updated weights for policy 1, policy_version 633559 (0.0009) [2023-12-26 19:58:26,028][105620] Updated weights for policy 1, policy_version 633569 (0.0009) [2023-12-26 19:58:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 324222976. Throughput: 0: 9815.8, 1: 9716.5. Samples: 324236272. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-26 19:58:26,063][104569] Avg episode reward: [(0, '717.925'), (1, '9175.207')] [2023-12-26 19:58:26,178][105692] Updated weights for policy 0, policy_version 632777 (0.0005) [2023-12-26 19:58:26,238][105692] Updated weights for policy 0, policy_version 632787 (0.0005) [2023-12-26 19:58:26,286][105692] Updated weights for policy 0, policy_version 632797 (0.0006) [2023-12-26 19:58:26,341][105692] Updated weights for policy 0, policy_version 632807 (0.0006) [2023-12-26 19:58:26,886][105692] Updated weights for policy 0, policy_version 632817 (0.0006) [2023-12-26 19:58:26,919][105620] Updated weights for policy 1, policy_version 633579 (0.0010) [2023-12-26 19:58:26,938][105692] Updated weights for policy 0, policy_version 632827 (0.0005) [2023-12-26 19:58:26,980][105620] Updated weights for policy 1, policy_version 633589 (0.0008) [2023-12-26 19:58:26,981][105692] Updated weights for policy 0, policy_version 632837 (0.0005) [2023-12-26 19:58:27,038][105620] Updated weights for policy 1, policy_version 633599 (0.0009) [2023-12-26 19:58:27,527][105692] Updated weights for policy 0, policy_version 632847 (0.0009) [2023-12-26 19:58:27,582][105692] Updated weights for policy 0, policy_version 632857 (0.0010) [2023-12-26 19:58:27,636][105692] Updated weights for policy 0, policy_version 632867 (0.0010) [2023-12-26 19:58:27,880][105620] Updated weights for policy 1, policy_version 633609 (0.0009) [2023-12-26 19:58:27,937][105620] Updated weights for policy 1, policy_version 633619 (0.0010) [2023-12-26 19:58:27,994][105620] Updated weights for policy 1, policy_version 633630 (0.0008) [2023-12-26 19:58:28,052][105620] Updated weights for policy 1, policy_version 633640 (0.0009) [2023-12-26 19:58:28,249][105692] Updated weights for policy 0, policy_version 632877 (0.0008) [2023-12-26 19:58:28,311][105692] Updated weights for policy 0, policy_version 632887 (0.0005) [2023-12-26 19:58:28,380][105692] Updated weights for policy 0, policy_version 632897 (0.0007) [2023-12-26 19:58:28,850][105620] Updated weights for policy 1, policy_version 633650 (0.0006) [2023-12-26 19:58:28,912][105620] Updated weights for policy 1, policy_version 633660 (0.0008) [2023-12-26 19:58:28,947][105692] Updated weights for policy 0, policy_version 632907 (0.0006) [2023-12-26 19:58:28,976][105620] Updated weights for policy 1, policy_version 633670 (0.0009) [2023-12-26 19:58:29,010][105692] Updated weights for policy 0, policy_version 632917 (0.0005) [2023-12-26 19:58:29,075][105692] Updated weights for policy 0, policy_version 632927 (0.0007) [2023-12-26 19:58:29,701][105692] Updated weights for policy 0, policy_version 632937 (0.0010) [2023-12-26 19:58:29,753][105692] Updated weights for policy 0, policy_version 632947 (0.0010) [2023-12-26 19:58:29,763][105620] Updated weights for policy 1, policy_version 633680 (0.0007) [2023-12-26 19:58:29,805][105692] Updated weights for policy 0, policy_version 632957 (0.0010) [2023-12-26 19:58:29,823][105620] Updated weights for policy 1, policy_version 633690 (0.0007) [2023-12-26 19:58:29,867][105692] Updated weights for policy 0, policy_version 632967 (0.0010) [2023-12-26 19:58:29,886][105620] Updated weights for policy 1, policy_version 633700 (0.0009) [2023-12-26 19:58:30,524][105620] Updated weights for policy 1, policy_version 633710 (0.0006) [2023-12-26 19:58:30,583][105620] Updated weights for policy 1, policy_version 633720 (0.0011) [2023-12-26 19:58:30,636][105620] Updated weights for policy 1, policy_version 633730 (0.0011) [2023-12-26 19:58:30,662][105692] Updated weights for policy 0, policy_version 632977 (0.0007) [2023-12-26 19:58:30,716][105692] Updated weights for policy 0, policy_version 632987 (0.0008) [2023-12-26 19:58:30,762][105692] Updated weights for policy 0, policy_version 632997 (0.0007) [2023-12-26 19:58:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 324329472. Throughput: 0: 9978.6, 1: 9623.1. Samples: 324297940. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-26 19:58:31,063][104569] Avg episode reward: [(0, '724.006'), (1, '8992.479')] [2023-12-26 19:58:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000633736_162250752.pth... [2023-12-26 19:58:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000633000_162078720.pth... [2023-12-26 19:58:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000632616_161964032.pth [2023-12-26 19:58:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000631816_161775616.pth [2023-12-26 19:58:31,253][105620] Updated weights for policy 1, policy_version 633740 (0.0009) [2023-12-26 19:58:31,317][105620] Updated weights for policy 1, policy_version 633750 (0.0010) [2023-12-26 19:58:31,382][105620] Updated weights for policy 1, policy_version 633760 (0.0010) [2023-12-26 19:58:31,543][105692] Updated weights for policy 0, policy_version 633007 (0.0008) [2023-12-26 19:58:31,605][105692] Updated weights for policy 0, policy_version 633017 (0.0009) [2023-12-26 19:58:31,669][105692] Updated weights for policy 0, policy_version 633027 (0.0008) [2023-12-26 19:58:32,035][105620] Updated weights for policy 1, policy_version 633770 (0.0009) [2023-12-26 19:58:32,084][105620] Updated weights for policy 1, policy_version 633780 (0.0009) [2023-12-26 19:58:32,134][105620] Updated weights for policy 1, policy_version 633790 (0.0009) [2023-12-26 19:58:32,183][105620] Updated weights for policy 1, policy_version 633800 (0.0009) [2023-12-26 19:58:32,457][105692] Updated weights for policy 0, policy_version 633037 (0.0007) [2023-12-26 19:58:32,519][105692] Updated weights for policy 0, policy_version 633047 (0.0010) [2023-12-26 19:58:32,583][105692] Updated weights for policy 0, policy_version 633057 (0.0010) [2023-12-26 19:58:32,886][105620] Updated weights for policy 1, policy_version 633810 (0.0005) [2023-12-26 19:58:32,942][105620] Updated weights for policy 1, policy_version 633820 (0.0008) [2023-12-26 19:58:32,998][105620] Updated weights for policy 1, policy_version 633830 (0.0009) [2023-12-26 19:58:33,309][105692] Updated weights for policy 0, policy_version 633067 (0.0009) [2023-12-26 19:58:33,354][105692] Updated weights for policy 0, policy_version 633077 (0.0008) [2023-12-26 19:58:33,404][105692] Updated weights for policy 0, policy_version 633087 (0.0009) [2023-12-26 19:58:33,713][105620] Updated weights for policy 1, policy_version 633840 (0.0009) [2023-12-26 19:58:33,763][105620] Updated weights for policy 1, policy_version 633850 (0.0006) [2023-12-26 19:58:33,812][105620] Updated weights for policy 1, policy_version 633860 (0.0008) [2023-12-26 19:58:34,176][105692] Updated weights for policy 0, policy_version 633097 (0.0008) [2023-12-26 19:58:34,224][105692] Updated weights for policy 0, policy_version 633107 (0.0009) [2023-12-26 19:58:34,286][105692] Updated weights for policy 0, policy_version 633117 (0.0009) [2023-12-26 19:58:34,350][105692] Updated weights for policy 0, policy_version 633127 (0.0008) [2023-12-26 19:58:34,575][105620] Updated weights for policy 1, policy_version 633870 (0.0009) [2023-12-26 19:58:34,638][105620] Updated weights for policy 1, policy_version 633881 (0.0010) [2023-12-26 19:58:34,691][105620] Updated weights for policy 1, policy_version 633891 (0.0009) [2023-12-26 19:58:35,108][105692] Updated weights for policy 0, policy_version 633137 (0.0008) [2023-12-26 19:58:35,160][105692] Updated weights for policy 0, policy_version 633147 (0.0008) [2023-12-26 19:58:35,207][105692] Updated weights for policy 0, policy_version 633157 (0.0008) [2023-12-26 19:58:35,391][105620] Updated weights for policy 1, policy_version 633901 (0.0008) [2023-12-26 19:58:35,461][105620] Updated weights for policy 1, policy_version 633911 (0.0005) [2023-12-26 19:58:35,531][105620] Updated weights for policy 1, policy_version 633921 (0.0006) [2023-12-26 19:58:35,866][105692] Updated weights for policy 0, policy_version 633167 (0.0006) [2023-12-26 19:58:35,931][105692] Updated weights for policy 0, policy_version 633177 (0.0008) [2023-12-26 19:58:35,979][105692] Updated weights for policy 0, policy_version 633187 (0.0010) [2023-12-26 19:58:36,010][105620] Updated weights for policy 1, policy_version 633931 (0.0005) [2023-12-26 19:58:36,056][105620] Updated weights for policy 1, policy_version 633941 (0.0006) [2023-12-26 19:58:36,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 324427776. Throughput: 0: 10003.0, 1: 9660.9. Samples: 324414064. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-26 19:58:36,062][104569] Avg episode reward: [(0, '848.726'), (1, '8900.621')] [2023-12-26 19:58:36,108][105620] Updated weights for policy 1, policy_version 633951 (0.0006) [2023-12-26 19:58:36,622][105692] Updated weights for policy 0, policy_version 633197 (0.0008) [2023-12-26 19:58:36,687][105692] Updated weights for policy 0, policy_version 633207 (0.0007) [2023-12-26 19:58:36,743][105585] KL-divergence is very high: 575.0640 [2023-12-26 19:58:36,748][105692] Updated weights for policy 0, policy_version 633217 (0.0008) [2023-12-26 19:58:36,755][105585] KL-divergence is very high: 607.9632 [2023-12-26 19:58:36,805][105620] Updated weights for policy 1, policy_version 633961 (0.0008) [2023-12-26 19:58:36,856][105620] Updated weights for policy 1, policy_version 633971 (0.0009) [2023-12-26 19:58:36,919][105620] Updated weights for policy 1, policy_version 633981 (0.0008) [2023-12-26 19:58:36,986][105620] Updated weights for policy 1, policy_version 633991 (0.0011) [2023-12-26 19:58:37,475][105692] Updated weights for policy 0, policy_version 633227 (0.0008) [2023-12-26 19:58:37,527][105692] Updated weights for policy 0, policy_version 633237 (0.0008) [2023-12-26 19:58:37,580][105692] Updated weights for policy 0, policy_version 633247 (0.0008) [2023-12-26 19:58:37,742][105620] Updated weights for policy 1, policy_version 634001 (0.0011) [2023-12-26 19:58:37,797][105620] Updated weights for policy 1, policy_version 634011 (0.0011) [2023-12-26 19:58:37,858][105620] Updated weights for policy 1, policy_version 634021 (0.0009) [2023-12-26 19:58:38,225][105692] Updated weights for policy 0, policy_version 633257 (0.0008) [2023-12-26 19:58:38,284][105692] Updated weights for policy 0, policy_version 633267 (0.0005) [2023-12-26 19:58:38,350][105692] Updated weights for policy 0, policy_version 633277 (0.0007) [2023-12-26 19:58:38,402][105692] Updated weights for policy 0, policy_version 633287 (0.0007) [2023-12-26 19:58:38,528][105620] Updated weights for policy 1, policy_version 634031 (0.0009) [2023-12-26 19:58:38,580][105620] Updated weights for policy 1, policy_version 634041 (0.0010) [2023-12-26 19:58:38,632][105620] Updated weights for policy 1, policy_version 634051 (0.0010) [2023-12-26 19:58:39,079][105692] Updated weights for policy 0, policy_version 633297 (0.0007) [2023-12-26 19:58:39,141][105692] Updated weights for policy 0, policy_version 633307 (0.0006) [2023-12-26 19:58:39,207][105692] Updated weights for policy 0, policy_version 633317 (0.0009) [2023-12-26 19:58:39,378][105620] Updated weights for policy 1, policy_version 634061 (0.0009) [2023-12-26 19:58:39,440][105620] Updated weights for policy 1, policy_version 634071 (0.0008) [2023-12-26 19:58:39,505][105620] Updated weights for policy 1, policy_version 634081 (0.0007) [2023-12-26 19:58:39,898][105692] Updated weights for policy 0, policy_version 633327 (0.0009) [2023-12-26 19:58:39,963][105692] Updated weights for policy 0, policy_version 633337 (0.0008) [2023-12-26 19:58:40,026][105692] Updated weights for policy 0, policy_version 633347 (0.0008) [2023-12-26 19:58:40,229][105620] Updated weights for policy 1, policy_version 634091 (0.0005) [2023-12-26 19:58:40,278][105620] Updated weights for policy 1, policy_version 634101 (0.0008) [2023-12-26 19:58:40,326][105620] Updated weights for policy 1, policy_version 634111 (0.0009) [2023-12-26 19:58:40,857][105692] Updated weights for policy 0, policy_version 633357 (0.0009) [2023-12-26 19:58:40,912][105692] Updated weights for policy 0, policy_version 633367 (0.0009) [2023-12-26 19:58:40,959][105620] Updated weights for policy 1, policy_version 634121 (0.0008) [2023-12-26 19:58:40,979][105692] Updated weights for policy 0, policy_version 633377 (0.0009) [2023-12-26 19:58:41,019][105620] Updated weights for policy 1, policy_version 634131 (0.0007) [2023-12-26 19:58:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 324526080. Throughput: 0: 10087.1, 1: 9749.2. Samples: 324535548. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-26 19:58:41,062][104569] Avg episode reward: [(0, '1223.973'), (1, '8811.024')] [2023-12-26 19:58:41,091][105620] Updated weights for policy 1, policy_version 634141 (0.0008) [2023-12-26 19:58:41,167][105620] Updated weights for policy 1, policy_version 634151 (0.0008) [2023-12-26 19:58:41,700][105692] Updated weights for policy 0, policy_version 633387 (0.0010) [2023-12-26 19:58:41,771][105692] Updated weights for policy 0, policy_version 633397 (0.0008) [2023-12-26 19:58:41,835][105692] Updated weights for policy 0, policy_version 633407 (0.0009) [2023-12-26 19:58:41,975][105620] Updated weights for policy 1, policy_version 634161 (0.0007) [2023-12-26 19:58:42,034][105620] Updated weights for policy 1, policy_version 634171 (0.0011) [2023-12-26 19:58:42,099][105620] Updated weights for policy 1, policy_version 634181 (0.0009) [2023-12-26 19:58:42,606][105692] Updated weights for policy 0, policy_version 633417 (0.0009) [2023-12-26 19:58:42,658][105692] Updated weights for policy 0, policy_version 633427 (0.0006) [2023-12-26 19:58:42,707][105692] Updated weights for policy 0, policy_version 633437 (0.0005) [2023-12-26 19:58:42,773][105692] Updated weights for policy 0, policy_version 633447 (0.0006) [2023-12-26 19:58:42,812][105620] Updated weights for policy 1, policy_version 634191 (0.0007) [2023-12-26 19:58:42,867][105620] Updated weights for policy 1, policy_version 634201 (0.0005) [2023-12-26 19:58:42,921][105620] Updated weights for policy 1, policy_version 634211 (0.0005) [2023-12-26 19:58:43,415][105692] Updated weights for policy 0, policy_version 633457 (0.0006) [2023-12-26 19:58:43,478][105692] Updated weights for policy 0, policy_version 633467 (0.0006) [2023-12-26 19:58:43,537][105692] Updated weights for policy 0, policy_version 633477 (0.0007) [2023-12-26 19:58:43,600][105620] Updated weights for policy 1, policy_version 634221 (0.0007) [2023-12-26 19:58:43,655][105620] Updated weights for policy 1, policy_version 634231 (0.0005) [2023-12-26 19:58:43,713][105620] Updated weights for policy 1, policy_version 634241 (0.0005) [2023-12-26 19:58:44,188][105692] Updated weights for policy 0, policy_version 633487 (0.0010) [2023-12-26 19:58:44,237][105692] Updated weights for policy 0, policy_version 633497 (0.0010) [2023-12-26 19:58:44,247][105620] Updated weights for policy 1, policy_version 634251 (0.0007) [2023-12-26 19:58:44,285][105692] Updated weights for policy 0, policy_version 633507 (0.0010) [2023-12-26 19:58:44,299][105620] Updated weights for policy 1, policy_version 634261 (0.0010) [2023-12-26 19:58:44,362][105620] Updated weights for policy 1, policy_version 634271 (0.0008) [2023-12-26 19:58:44,939][105620] Updated weights for policy 1, policy_version 634281 (0.0006) [2023-12-26 19:58:44,999][105620] Updated weights for policy 1, policy_version 634291 (0.0011) [2023-12-26 19:58:45,030][105692] Updated weights for policy 0, policy_version 633517 (0.0008) [2023-12-26 19:58:45,067][105620] Updated weights for policy 1, policy_version 634301 (0.0011) [2023-12-26 19:58:45,096][105692] Updated weights for policy 0, policy_version 633527 (0.0006) [2023-12-26 19:58:45,123][105620] Updated weights for policy 1, policy_version 634311 (0.0011) [2023-12-26 19:58:45,165][105692] Updated weights for policy 0, policy_version 633537 (0.0006) [2023-12-26 19:58:45,787][105620] Updated weights for policy 1, policy_version 634321 (0.0007) [2023-12-26 19:58:45,813][105692] Updated weights for policy 0, policy_version 633547 (0.0006) [2023-12-26 19:58:45,842][105620] Updated weights for policy 1, policy_version 634331 (0.0007) [2023-12-26 19:58:45,859][105692] Updated weights for policy 0, policy_version 633557 (0.0005) [2023-12-26 19:58:45,907][105620] Updated weights for policy 1, policy_version 634341 (0.0006) [2023-12-26 19:58:45,923][105692] Updated weights for policy 0, policy_version 633567 (0.0006) [2023-12-26 19:58:46,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 324632576. Throughput: 0: 9951.8, 1: 9801.4. Samples: 324593844. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-26 19:58:46,063][104569] Avg episode reward: [(0, '1404.142'), (1, '8992.896')] [2023-12-26 19:58:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000633576_162226176.pth... [2023-12-26 19:58:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000634344_162406400.pth... [2023-12-26 19:58:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000633192_162111488.pth [2023-12-26 19:58:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000632392_161923072.pth [2023-12-26 19:58:46,425][105620] Updated weights for policy 1, policy_version 634351 (0.0005) [2023-12-26 19:58:46,476][105620] Updated weights for policy 1, policy_version 634361 (0.0005) [2023-12-26 19:58:46,527][105620] Updated weights for policy 1, policy_version 634371 (0.0005) [2023-12-26 19:58:46,528][105692] Updated weights for policy 0, policy_version 633577 (0.0007) [2023-12-26 19:58:46,587][105692] Updated weights for policy 0, policy_version 633587 (0.0005) [2023-12-26 19:58:46,633][105692] Updated weights for policy 0, policy_version 633597 (0.0005) [2023-12-26 19:58:46,679][105692] Updated weights for policy 0, policy_version 633607 (0.0005) [2023-12-26 19:58:47,079][105620] Updated weights for policy 1, policy_version 634381 (0.0007) [2023-12-26 19:58:47,130][105620] Updated weights for policy 1, policy_version 634391 (0.0006) [2023-12-26 19:58:47,191][105620] Updated weights for policy 1, policy_version 634401 (0.0005) [2023-12-26 19:58:47,276][105692] Updated weights for policy 0, policy_version 633617 (0.0009) [2023-12-26 19:58:47,330][105692] Updated weights for policy 0, policy_version 633628 (0.0010) [2023-12-26 19:58:47,381][105692] Updated weights for policy 0, policy_version 633638 (0.0010) [2023-12-26 19:58:47,812][105620] Updated weights for policy 1, policy_version 634411 (0.0007) [2023-12-26 19:58:47,872][105620] Updated weights for policy 1, policy_version 634421 (0.0008) [2023-12-26 19:58:47,930][105620] Updated weights for policy 1, policy_version 634431 (0.0008) [2023-12-26 19:58:48,200][105692] Updated weights for policy 0, policy_version 633648 (0.0008) [2023-12-26 19:58:48,262][105692] Updated weights for policy 0, policy_version 633658 (0.0008) [2023-12-26 19:58:48,328][105692] Updated weights for policy 0, policy_version 633668 (0.0009) [2023-12-26 19:58:48,719][105620] Updated weights for policy 1, policy_version 634441 (0.0009) [2023-12-26 19:58:48,776][105620] Updated weights for policy 1, policy_version 634451 (0.0010) [2023-12-26 19:58:48,832][105620] Updated weights for policy 1, policy_version 634462 (0.0010) [2023-12-26 19:58:48,877][105620] Updated weights for policy 1, policy_version 634472 (0.0009) [2023-12-26 19:58:49,000][105692] Updated weights for policy 0, policy_version 633678 (0.0009) [2023-12-26 19:58:49,062][105692] Updated weights for policy 0, policy_version 633688 (0.0009) [2023-12-26 19:58:49,113][105692] Updated weights for policy 0, policy_version 633698 (0.0009) [2023-12-26 19:58:49,639][105620] Updated weights for policy 1, policy_version 634482 (0.0005) [2023-12-26 19:58:49,708][105620] Updated weights for policy 1, policy_version 634492 (0.0006) [2023-12-26 19:58:49,783][105620] Updated weights for policy 1, policy_version 634502 (0.0006) [2023-12-26 19:58:49,939][105692] Updated weights for policy 0, policy_version 633708 (0.0008) [2023-12-26 19:58:49,994][105692] Updated weights for policy 0, policy_version 633718 (0.0008) [2023-12-26 19:58:50,047][105692] Updated weights for policy 0, policy_version 633728 (0.0008) [2023-12-26 19:58:50,380][105620] Updated weights for policy 1, policy_version 634512 (0.0009) [2023-12-26 19:58:50,438][105620] Updated weights for policy 1, policy_version 634522 (0.0010) [2023-12-26 19:58:50,491][105620] Updated weights for policy 1, policy_version 634532 (0.0011) [2023-12-26 19:58:50,779][105692] Updated weights for policy 0, policy_version 633738 (0.0008) [2023-12-26 19:58:50,837][105692] Updated weights for policy 0, policy_version 633748 (0.0008) [2023-12-26 19:58:50,889][105692] Updated weights for policy 0, policy_version 633758 (0.0008) [2023-12-26 19:58:50,944][105692] Updated weights for policy 0, policy_version 633768 (0.0008) [2023-12-26 19:58:51,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19933.8, 300 sec: 19549.7). Total num frames: 324730880. Throughput: 0: 9982.7, 1: 9891.9. Samples: 324719416. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-26 19:58:51,063][104569] Avg episode reward: [(0, '2630.618'), (1, '9354.771')] [2023-12-26 19:58:51,231][105620] Updated weights for policy 1, policy_version 634542 (0.0009) [2023-12-26 19:58:51,297][105620] Updated weights for policy 1, policy_version 634552 (0.0009) [2023-12-26 19:58:51,362][105620] Updated weights for policy 1, policy_version 634562 (0.0009) [2023-12-26 19:58:51,807][105692] Updated weights for policy 0, policy_version 633778 (0.0009) [2023-12-26 19:58:51,867][105692] Updated weights for policy 0, policy_version 633788 (0.0008) [2023-12-26 19:58:51,928][105692] Updated weights for policy 0, policy_version 633798 (0.0009) [2023-12-26 19:58:52,086][105620] Updated weights for policy 1, policy_version 634572 (0.0008) [2023-12-26 19:58:52,158][105620] Updated weights for policy 1, policy_version 634582 (0.0010) [2023-12-26 19:58:52,220][105620] Updated weights for policy 1, policy_version 634592 (0.0009) [2023-12-26 19:58:52,687][105692] Updated weights for policy 0, policy_version 633808 (0.0010) [2023-12-26 19:58:52,740][105692] Updated weights for policy 0, policy_version 633818 (0.0008) [2023-12-26 19:58:52,792][105692] Updated weights for policy 0, policy_version 633828 (0.0009) [2023-12-26 19:58:53,023][105620] Updated weights for policy 1, policy_version 634602 (0.0008) [2023-12-26 19:58:53,080][105620] Updated weights for policy 1, policy_version 634612 (0.0009) [2023-12-26 19:58:53,145][105620] Updated weights for policy 1, policy_version 634622 (0.0009) [2023-12-26 19:58:53,204][105620] Updated weights for policy 1, policy_version 634632 (0.0010) [2023-12-26 19:58:53,432][105692] Updated weights for policy 0, policy_version 633838 (0.0006) [2023-12-26 19:58:53,489][105692] Updated weights for policy 0, policy_version 633848 (0.0005) [2023-12-26 19:58:53,552][105692] Updated weights for policy 0, policy_version 633858 (0.0006) [2023-12-26 19:58:54,054][105620] Updated weights for policy 1, policy_version 634642 (0.0010) [2023-12-26 19:58:54,115][105620] Updated weights for policy 1, policy_version 634652 (0.0008) [2023-12-26 19:58:54,171][105692] Updated weights for policy 0, policy_version 633868 (0.0005) [2023-12-26 19:58:54,177][105620] Updated weights for policy 1, policy_version 634662 (0.0009) [2023-12-26 19:58:54,232][105692] Updated weights for policy 0, policy_version 633878 (0.0007) [2023-12-26 19:58:54,289][105692] Updated weights for policy 0, policy_version 633888 (0.0008) [2023-12-26 19:58:54,797][105620] Updated weights for policy 1, policy_version 634672 (0.0007) [2023-12-26 19:58:54,859][105620] Updated weights for policy 1, policy_version 634682 (0.0005) [2023-12-26 19:58:54,918][105620] Updated weights for policy 1, policy_version 634692 (0.0005) [2023-12-26 19:58:55,127][105692] Updated weights for policy 0, policy_version 633898 (0.0010) [2023-12-26 19:58:55,194][105692] Updated weights for policy 0, policy_version 633908 (0.0010) [2023-12-26 19:58:55,267][105692] Updated weights for policy 0, policy_version 633918 (0.0009) [2023-12-26 19:58:55,330][105692] Updated weights for policy 0, policy_version 633928 (0.0010) [2023-12-26 19:58:55,437][105620] Updated weights for policy 1, policy_version 634702 (0.0009) [2023-12-26 19:58:55,488][105620] Updated weights for policy 1, policy_version 634712 (0.0010) [2023-12-26 19:58:55,543][105620] Updated weights for policy 1, policy_version 634722 (0.0010) [2023-12-26 19:58:56,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 324820992. Throughput: 0: 9920.6, 1: 9864.5. Samples: 324834852. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-26 19:58:56,062][104569] Avg episode reward: [(0, '595.600'), (1, '9354.932')] [2023-12-26 19:58:56,115][105692] Updated weights for policy 0, policy_version 633938 (0.0005) [2023-12-26 19:58:56,167][105692] Updated weights for policy 0, policy_version 633948 (0.0006) [2023-12-26 19:58:56,218][105692] Updated weights for policy 0, policy_version 633958 (0.0006) [2023-12-26 19:58:56,218][105620] Updated weights for policy 1, policy_version 634732 (0.0007) [2023-12-26 19:58:56,266][105620] Updated weights for policy 1, policy_version 634742 (0.0010) [2023-12-26 19:58:56,316][105620] Updated weights for policy 1, policy_version 634752 (0.0008) [2023-12-26 19:58:56,838][105692] Updated weights for policy 0, policy_version 633968 (0.0010) [2023-12-26 19:58:56,896][105620] Updated weights for policy 1, policy_version 634762 (0.0006) [2023-12-26 19:58:56,907][105692] Updated weights for policy 0, policy_version 633978 (0.0008) [2023-12-26 19:58:56,959][105620] Updated weights for policy 1, policy_version 634772 (0.0007) [2023-12-26 19:58:56,969][105692] Updated weights for policy 0, policy_version 633988 (0.0005) [2023-12-26 19:58:57,023][105620] Updated weights for policy 1, policy_version 634782 (0.0007) [2023-12-26 19:58:57,077][105620] Updated weights for policy 1, policy_version 634792 (0.0010) [2023-12-26 19:58:57,530][105692] Updated weights for policy 0, policy_version 633998 (0.0008) [2023-12-26 19:58:57,584][105692] Updated weights for policy 0, policy_version 634008 (0.0010) [2023-12-26 19:58:57,633][105620] Updated weights for policy 1, policy_version 634802 (0.0008) [2023-12-26 19:58:57,639][105692] Updated weights for policy 0, policy_version 634018 (0.0010) [2023-12-26 19:58:57,698][105620] Updated weights for policy 1, policy_version 634812 (0.0010) [2023-12-26 19:58:57,769][105620] Updated weights for policy 1, policy_version 634822 (0.0010) [2023-12-26 19:58:58,239][105692] Updated weights for policy 0, policy_version 634028 (0.0006) [2023-12-26 19:58:58,286][105692] Updated weights for policy 0, policy_version 634038 (0.0007) [2023-12-26 19:58:58,363][105692] Updated weights for policy 0, policy_version 634048 (0.0008) [2023-12-26 19:58:58,485][105620] Updated weights for policy 1, policy_version 634832 (0.0010) [2023-12-26 19:58:58,549][105620] Updated weights for policy 1, policy_version 634842 (0.0011) [2023-12-26 19:58:58,607][105620] Updated weights for policy 1, policy_version 634852 (0.0010) [2023-12-26 19:58:59,187][105692] Updated weights for policy 0, policy_version 634058 (0.0008) [2023-12-26 19:58:59,247][105692] Updated weights for policy 0, policy_version 634068 (0.0008) [2023-12-26 19:58:59,320][105692] Updated weights for policy 0, policy_version 634079 (0.0008) [2023-12-26 19:58:59,380][105620] Updated weights for policy 1, policy_version 634862 (0.0010) [2023-12-26 19:58:59,440][105620] Updated weights for policy 1, policy_version 634872 (0.0012) [2023-12-26 19:58:59,495][105620] Updated weights for policy 1, policy_version 634882 (0.0010) [2023-12-26 19:59:00,073][105692] Updated weights for policy 0, policy_version 634089 (0.0007) [2023-12-26 19:59:00,127][105692] Updated weights for policy 0, policy_version 634099 (0.0005) [2023-12-26 19:59:00,182][105692] Updated weights for policy 0, policy_version 634109 (0.0006) [2023-12-26 19:59:00,234][105692] Updated weights for policy 0, policy_version 634119 (0.0008) [2023-12-26 19:59:00,244][105620] Updated weights for policy 1, policy_version 634892 (0.0010) [2023-12-26 19:59:00,302][105620] Updated weights for policy 1, policy_version 634902 (0.0010) [2023-12-26 19:59:00,368][105620] Updated weights for policy 1, policy_version 634912 (0.0009) [2023-12-26 19:59:00,797][105692] Updated weights for policy 0, policy_version 634129 (0.0005) [2023-12-26 19:59:00,844][105692] Updated weights for policy 0, policy_version 634139 (0.0010) [2023-12-26 19:59:00,896][105692] Updated weights for policy 0, policy_version 634149 (0.0010) [2023-12-26 19:59:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 324927488. Throughput: 0: 10020.0, 1: 9878.3. Samples: 324898192. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-26 19:59:01,062][104569] Avg episode reward: [(0, '3643.300'), (1, '9263.535')] [2023-12-26 19:59:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000634152_162373632.pth... [2023-12-26 19:59:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000634920_162553856.pth... [2023-12-26 19:59:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000633736_162250752.pth [2023-12-26 19:59:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000633000_162078720.pth [2023-12-26 19:59:01,113][105620] Updated weights for policy 1, policy_version 634922 (0.0009) [2023-12-26 19:59:01,183][105620] Updated weights for policy 1, policy_version 634932 (0.0011) [2023-12-26 19:59:01,237][105620] Updated weights for policy 1, policy_version 634942 (0.0007) [2023-12-26 19:59:01,294][105620] Updated weights for policy 1, policy_version 634952 (0.0009) [2023-12-26 19:59:01,576][105692] Updated weights for policy 0, policy_version 634159 (0.0009) [2023-12-26 19:59:01,641][105692] Updated weights for policy 0, policy_version 634169 (0.0008) [2023-12-26 19:59:01,701][105692] Updated weights for policy 0, policy_version 634179 (0.0008) [2023-12-26 19:59:02,024][105620] Updated weights for policy 1, policy_version 634962 (0.0005) [2023-12-26 19:59:02,097][105620] Updated weights for policy 1, policy_version 634972 (0.0007) [2023-12-26 19:59:02,147][105620] Updated weights for policy 1, policy_version 634982 (0.0009) [2023-12-26 19:59:02,443][105692] Updated weights for policy 0, policy_version 634189 (0.0009) [2023-12-26 19:59:02,491][105692] Updated weights for policy 0, policy_version 634199 (0.0009) [2023-12-26 19:59:02,543][105692] Updated weights for policy 0, policy_version 634209 (0.0009) [2023-12-26 19:59:02,855][105620] Updated weights for policy 1, policy_version 634992 (0.0009) [2023-12-26 19:59:02,908][105620] Updated weights for policy 1, policy_version 635002 (0.0008) [2023-12-26 19:59:02,965][105620] Updated weights for policy 1, policy_version 635012 (0.0005) [2023-12-26 19:59:03,305][105692] Updated weights for policy 0, policy_version 634219 (0.0010) [2023-12-26 19:59:03,357][105692] Updated weights for policy 0, policy_version 634229 (0.0010) [2023-12-26 19:59:03,418][105692] Updated weights for policy 0, policy_version 634239 (0.0010) [2023-12-26 19:59:03,497][105620] Updated weights for policy 1, policy_version 635022 (0.0005) [2023-12-26 19:59:03,555][105620] Updated weights for policy 1, policy_version 635032 (0.0007) [2023-12-26 19:59:03,615][105620] Updated weights for policy 1, policy_version 635042 (0.0007) [2023-12-26 19:59:04,174][105692] Updated weights for policy 0, policy_version 634249 (0.0010) [2023-12-26 19:59:04,199][105620] Updated weights for policy 1, policy_version 635052 (0.0005) [2023-12-26 19:59:04,235][105692] Updated weights for policy 0, policy_version 634259 (0.0011) [2023-12-26 19:59:04,261][105620] Updated weights for policy 1, policy_version 635062 (0.0009) [2023-12-26 19:59:04,303][105692] Updated weights for policy 0, policy_version 634269 (0.0011) [2023-12-26 19:59:04,324][105620] Updated weights for policy 1, policy_version 635072 (0.0008) [2023-12-26 19:59:04,357][105692] Updated weights for policy 0, policy_version 634279 (0.0011) [2023-12-26 19:59:05,022][105692] Updated weights for policy 0, policy_version 634289 (0.0009) [2023-12-26 19:59:05,059][105620] Updated weights for policy 1, policy_version 635082 (0.0011) [2023-12-26 19:59:05,070][105692] Updated weights for policy 0, policy_version 634299 (0.0010) [2023-12-26 19:59:05,107][105620] Updated weights for policy 1, policy_version 635092 (0.0010) [2023-12-26 19:59:05,119][105692] Updated weights for policy 0, policy_version 634309 (0.0010) [2023-12-26 19:59:05,163][105620] Updated weights for policy 1, policy_version 635102 (0.0010) [2023-12-26 19:59:05,220][105620] Updated weights for policy 1, policy_version 635112 (0.0010) [2023-12-26 19:59:05,838][105692] Updated weights for policy 0, policy_version 634319 (0.0007) [2023-12-26 19:59:05,884][105692] Updated weights for policy 0, policy_version 634329 (0.0006) [2023-12-26 19:59:05,937][105692] Updated weights for policy 0, policy_version 634339 (0.0005) [2023-12-26 19:59:05,975][105620] Updated weights for policy 1, policy_version 635122 (0.0010) [2023-12-26 19:59:06,022][105620] Updated weights for policy 1, policy_version 635132 (0.0009) [2023-12-26 19:59:06,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 325025792. Throughput: 0: 9953.1, 1: 9952.4. Samples: 325016380. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-26 19:59:06,063][104569] Avg episode reward: [(0, '6145.950'), (1, '8896.702')] [2023-12-26 19:59:06,075][105620] Updated weights for policy 1, policy_version 635142 (0.0005) [2023-12-26 19:59:06,604][105692] Updated weights for policy 0, policy_version 634349 (0.0007) [2023-12-26 19:59:06,661][105692] Updated weights for policy 0, policy_version 634359 (0.0009) [2023-12-26 19:59:06,726][105692] Updated weights for policy 0, policy_version 634370 (0.0010) [2023-12-26 19:59:06,774][105620] Updated weights for policy 1, policy_version 635152 (0.0010) [2023-12-26 19:59:06,832][105620] Updated weights for policy 1, policy_version 635162 (0.0006) [2023-12-26 19:59:06,899][105620] Updated weights for policy 1, policy_version 635172 (0.0008) [2023-12-26 19:59:07,525][105620] Updated weights for policy 1, policy_version 635182 (0.0009) [2023-12-26 19:59:07,544][105692] Updated weights for policy 0, policy_version 634380 (0.0010) [2023-12-26 19:59:07,587][105620] Updated weights for policy 1, policy_version 635192 (0.0005) [2023-12-26 19:59:07,600][105692] Updated weights for policy 0, policy_version 634390 (0.0010) [2023-12-26 19:59:07,645][105620] Updated weights for policy 1, policy_version 635202 (0.0010) [2023-12-26 19:59:07,655][105692] Updated weights for policy 0, policy_version 634400 (0.0008) [2023-12-26 19:59:08,332][105620] Updated weights for policy 1, policy_version 635212 (0.0009) [2023-12-26 19:59:08,378][105692] Updated weights for policy 0, policy_version 634410 (0.0009) [2023-12-26 19:59:08,392][105620] Updated weights for policy 1, policy_version 635222 (0.0006) [2023-12-26 19:59:08,434][105692] Updated weights for policy 0, policy_version 634420 (0.0009) [2023-12-26 19:59:08,447][105620] Updated weights for policy 1, policy_version 635232 (0.0006) [2023-12-26 19:59:08,493][105692] Updated weights for policy 0, policy_version 634430 (0.0008) [2023-12-26 19:59:08,548][105692] Updated weights for policy 0, policy_version 634440 (0.0010) [2023-12-26 19:59:09,058][105620] Updated weights for policy 1, policy_version 635242 (0.0006) [2023-12-26 19:59:09,109][105620] Updated weights for policy 1, policy_version 635252 (0.0009) [2023-12-26 19:59:09,157][105620] Updated weights for policy 1, policy_version 635262 (0.0009) [2023-12-26 19:59:09,214][105620] Updated weights for policy 1, policy_version 635272 (0.0008) [2023-12-26 19:59:09,374][105692] Updated weights for policy 0, policy_version 634450 (0.0009) [2023-12-26 19:59:09,441][105692] Updated weights for policy 0, policy_version 634460 (0.0009) [2023-12-26 19:59:09,498][105692] Updated weights for policy 0, policy_version 634470 (0.0006) [2023-12-26 19:59:10,014][105620] Updated weights for policy 1, policy_version 635282 (0.0006) [2023-12-26 19:59:10,076][105620] Updated weights for policy 1, policy_version 635292 (0.0005) [2023-12-26 19:59:10,136][105620] Updated weights for policy 1, policy_version 635302 (0.0007) [2023-12-26 19:59:10,231][105692] Updated weights for policy 0, policy_version 634480 (0.0007) [2023-12-26 19:59:10,283][105692] Updated weights for policy 0, policy_version 634490 (0.0009) [2023-12-26 19:59:10,332][105692] Updated weights for policy 0, policy_version 634500 (0.0009) [2023-12-26 19:59:10,804][105620] Updated weights for policy 1, policy_version 635312 (0.0009) [2023-12-26 19:59:10,857][105620] Updated weights for policy 1, policy_version 635322 (0.0009) [2023-12-26 19:59:10,905][105620] Updated weights for policy 1, policy_version 635332 (0.0009) [2023-12-26 19:59:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 325124096. Throughput: 0: 9891.7, 1: 10040.8. Samples: 325133236. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-26 19:59:11,063][104569] Avg episode reward: [(0, '6316.829'), (1, '8896.025')] [2023-12-26 19:59:11,105][105692] Updated weights for policy 0, policy_version 634510 (0.0007) [2023-12-26 19:59:11,174][105692] Updated weights for policy 0, policy_version 634520 (0.0008) [2023-12-26 19:59:11,261][105692] Updated weights for policy 0, policy_version 634530 (0.0010) [2023-12-26 19:59:11,676][105620] Updated weights for policy 1, policy_version 635342 (0.0008) [2023-12-26 19:59:11,737][105620] Updated weights for policy 1, policy_version 635352 (0.0010) [2023-12-26 19:59:11,802][105620] Updated weights for policy 1, policy_version 635362 (0.0009) [2023-12-26 19:59:11,916][105692] Updated weights for policy 0, policy_version 634540 (0.0010) [2023-12-26 19:59:11,964][105692] Updated weights for policy 0, policy_version 634550 (0.0010) [2023-12-26 19:59:12,029][105692] Updated weights for policy 0, policy_version 634560 (0.0006) [2023-12-26 19:59:12,622][105692] Updated weights for policy 0, policy_version 634570 (0.0005) [2023-12-26 19:59:12,676][105620] Updated weights for policy 1, policy_version 635372 (0.0009) [2023-12-26 19:59:12,689][105692] Updated weights for policy 0, policy_version 634580 (0.0007) [2023-12-26 19:59:12,732][105620] Updated weights for policy 1, policy_version 635382 (0.0011) [2023-12-26 19:59:12,748][105692] Updated weights for policy 0, policy_version 634590 (0.0010) [2023-12-26 19:59:12,788][105620] Updated weights for policy 1, policy_version 635392 (0.0010) [2023-12-26 19:59:12,808][105692] Updated weights for policy 0, policy_version 634600 (0.0011) [2023-12-26 19:59:13,374][105692] Updated weights for policy 0, policy_version 634610 (0.0006) [2023-12-26 19:59:13,430][105692] Updated weights for policy 0, policy_version 634620 (0.0006) [2023-12-26 19:59:13,486][105692] Updated weights for policy 0, policy_version 634630 (0.0008) [2023-12-26 19:59:13,528][105620] Updated weights for policy 1, policy_version 635402 (0.0011) [2023-12-26 19:59:13,584][105620] Updated weights for policy 1, policy_version 635412 (0.0010) [2023-12-26 19:59:13,640][105620] Updated weights for policy 1, policy_version 635422 (0.0005) [2023-12-26 19:59:13,693][105620] Updated weights for policy 1, policy_version 635432 (0.0005) [2023-12-26 19:59:14,135][105692] Updated weights for policy 0, policy_version 634640 (0.0011) [2023-12-26 19:59:14,195][105692] Updated weights for policy 0, policy_version 634650 (0.0009) [2023-12-26 19:59:14,237][105620] Updated weights for policy 1, policy_version 635442 (0.0005) [2023-12-26 19:59:14,256][105692] Updated weights for policy 0, policy_version 634660 (0.0006) [2023-12-26 19:59:14,305][105620] Updated weights for policy 1, policy_version 635452 (0.0005) [2023-12-26 19:59:14,379][105620] Updated weights for policy 1, policy_version 635462 (0.0006) [2023-12-26 19:59:14,885][105692] Updated weights for policy 0, policy_version 634670 (0.0007) [2023-12-26 19:59:14,952][105692] Updated weights for policy 0, policy_version 634680 (0.0007) [2023-12-26 19:59:15,005][105620] Updated weights for policy 1, policy_version 635472 (0.0007) [2023-12-26 19:59:15,020][105692] Updated weights for policy 0, policy_version 634690 (0.0009) [2023-12-26 19:59:15,070][105620] Updated weights for policy 1, policy_version 635482 (0.0006) [2023-12-26 19:59:15,130][105620] Updated weights for policy 1, policy_version 635492 (0.0008) [2023-12-26 19:59:15,594][105692] Updated weights for policy 0, policy_version 634700 (0.0010) [2023-12-26 19:59:15,656][105692] Updated weights for policy 0, policy_version 634710 (0.0010) [2023-12-26 19:59:15,722][105692] Updated weights for policy 0, policy_version 634720 (0.0005) [2023-12-26 19:59:15,964][105620] Updated weights for policy 1, policy_version 635502 (0.0007) [2023-12-26 19:59:16,011][105620] Updated weights for policy 1, policy_version 635512 (0.0008) [2023-12-26 19:59:16,058][105620] Updated weights for policy 1, policy_version 635522 (0.0008) [2023-12-26 19:59:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.2, 300 sec: 19577.5). Total num frames: 325222400. Throughput: 0: 9800.3, 1: 10094.8. Samples: 325193224. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-26 19:59:16,063][104569] Avg episode reward: [(0, '7658.553'), (1, '9078.962')] [2023-12-26 19:59:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000634728_162521088.pth... [2023-12-26 19:59:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000633576_162226176.pth [2023-12-26 19:59:16,086][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000635528_162709504.pth... [2023-12-26 19:59:16,091][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000634344_162406400.pth [2023-12-26 19:59:16,412][105692] Updated weights for policy 0, policy_version 634730 (0.0010) [2023-12-26 19:59:16,470][105692] Updated weights for policy 0, policy_version 634740 (0.0010) [2023-12-26 19:59:16,523][105692] Updated weights for policy 0, policy_version 634750 (0.0010) [2023-12-26 19:59:16,585][105692] Updated weights for policy 0, policy_version 634760 (0.0009) [2023-12-26 19:59:16,777][105620] Updated weights for policy 1, policy_version 635532 (0.0009) [2023-12-26 19:59:16,833][105620] Updated weights for policy 1, policy_version 635542 (0.0010) [2023-12-26 19:59:16,887][105620] Updated weights for policy 1, policy_version 635552 (0.0009) [2023-12-26 19:59:17,251][105692] Updated weights for policy 0, policy_version 634770 (0.0008) [2023-12-26 19:59:17,302][105692] Updated weights for policy 0, policy_version 634780 (0.0009) [2023-12-26 19:59:17,353][105692] Updated weights for policy 0, policy_version 634790 (0.0007) [2023-12-26 19:59:17,496][105620] Updated weights for policy 1, policy_version 635562 (0.0005) [2023-12-26 19:59:17,547][105620] Updated weights for policy 1, policy_version 635572 (0.0007) [2023-12-26 19:59:17,602][105620] Updated weights for policy 1, policy_version 635582 (0.0011) [2023-12-26 19:59:17,649][105620] Updated weights for policy 1, policy_version 635592 (0.0007) [2023-12-26 19:59:18,070][105692] Updated weights for policy 0, policy_version 634800 (0.0006) [2023-12-26 19:59:18,139][105692] Updated weights for policy 0, policy_version 634810 (0.0006) [2023-12-26 19:59:18,199][105692] Updated weights for policy 0, policy_version 634820 (0.0011) [2023-12-26 19:59:18,297][105620] Updated weights for policy 1, policy_version 635602 (0.0008) [2023-12-26 19:59:18,357][105620] Updated weights for policy 1, policy_version 635612 (0.0007) [2023-12-26 19:59:18,413][105620] Updated weights for policy 1, policy_version 635622 (0.0011) [2023-12-26 19:59:18,901][105692] Updated weights for policy 0, policy_version 634830 (0.0008) [2023-12-26 19:59:18,962][105692] Updated weights for policy 0, policy_version 634840 (0.0005) [2023-12-26 19:59:19,030][105692] Updated weights for policy 0, policy_version 634850 (0.0007) [2023-12-26 19:59:19,142][105620] Updated weights for policy 1, policy_version 635632 (0.0007) [2023-12-26 19:59:19,199][105620] Updated weights for policy 1, policy_version 635642 (0.0005) [2023-12-26 19:59:19,271][105620] Updated weights for policy 1, policy_version 635652 (0.0007) [2023-12-26 19:59:19,681][105692] Updated weights for policy 0, policy_version 634860 (0.0008) [2023-12-26 19:59:19,744][105692] Updated weights for policy 0, policy_version 634870 (0.0007) [2023-12-26 19:59:19,803][105692] Updated weights for policy 0, policy_version 634880 (0.0006) [2023-12-26 19:59:19,924][105620] Updated weights for policy 1, policy_version 635662 (0.0009) [2023-12-26 19:59:19,989][105620] Updated weights for policy 1, policy_version 635672 (0.0009) [2023-12-26 19:59:20,048][105620] Updated weights for policy 1, policy_version 635682 (0.0010) [2023-12-26 19:59:20,459][105692] Updated weights for policy 0, policy_version 634890 (0.0009) [2023-12-26 19:59:20,515][105692] Updated weights for policy 0, policy_version 634900 (0.0008) [2023-12-26 19:59:20,574][105692] Updated weights for policy 0, policy_version 634910 (0.0009) [2023-12-26 19:59:20,633][105692] Updated weights for policy 0, policy_version 634920 (0.0008) [2023-12-26 19:59:20,871][105620] Updated weights for policy 1, policy_version 635692 (0.0010) [2023-12-26 19:59:20,894][105586] KL-divergence is very high: 104.4003 [2023-12-26 19:59:20,931][105620] Updated weights for policy 1, policy_version 635702 (0.0009) [2023-12-26 19:59:20,986][105620] Updated weights for policy 1, policy_version 635712 (0.0009) [2023-12-26 19:59:21,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19933.8, 300 sec: 19605.3). Total num frames: 325328896. Throughput: 0: 9921.7, 1: 10126.5. Samples: 325316240. Policy #0 lag: (min: 1.0, avg: 22.7, max: 33.0) [2023-12-26 19:59:21,063][104569] Avg episode reward: [(0, '8906.856'), (1, '8884.282')] [2023-12-26 19:59:21,426][105692] Updated weights for policy 0, policy_version 634930 (0.0007) [2023-12-26 19:59:21,490][105692] Updated weights for policy 0, policy_version 634940 (0.0006) [2023-12-26 19:59:21,546][105692] Updated weights for policy 0, policy_version 634950 (0.0008) [2023-12-26 19:59:21,869][105620] Updated weights for policy 1, policy_version 635722 (0.0009) [2023-12-26 19:59:21,917][105620] Updated weights for policy 1, policy_version 635732 (0.0009) [2023-12-26 19:59:21,973][105620] Updated weights for policy 1, policy_version 635742 (0.0009) [2023-12-26 19:59:22,024][105620] Updated weights for policy 1, policy_version 635752 (0.0009) [2023-12-26 19:59:22,212][105692] Updated weights for policy 0, policy_version 634960 (0.0009) [2023-12-26 19:59:22,272][105692] Updated weights for policy 0, policy_version 634970 (0.0008) [2023-12-26 19:59:22,337][105692] Updated weights for policy 0, policy_version 634980 (0.0008) [2023-12-26 19:59:22,861][105620] Updated weights for policy 1, policy_version 635762 (0.0009) [2023-12-26 19:59:22,922][105620] Updated weights for policy 1, policy_version 635772 (0.0009) [2023-12-26 19:59:22,983][105620] Updated weights for policy 1, policy_version 635782 (0.0009) [2023-12-26 19:59:23,035][105692] Updated weights for policy 0, policy_version 634990 (0.0008) [2023-12-26 19:59:23,093][105692] Updated weights for policy 0, policy_version 635000 (0.0009) [2023-12-26 19:59:23,144][105692] Updated weights for policy 0, policy_version 635010 (0.0009) [2023-12-26 19:59:23,744][105620] Updated weights for policy 1, policy_version 635792 (0.0009) [2023-12-26 19:59:23,799][105620] Updated weights for policy 1, policy_version 635802 (0.0009) [2023-12-26 19:59:23,854][105620] Updated weights for policy 1, policy_version 635812 (0.0009) [2023-12-26 19:59:23,908][105692] Updated weights for policy 0, policy_version 635020 (0.0009) [2023-12-26 19:59:23,968][105692] Updated weights for policy 0, policy_version 635030 (0.0008) [2023-12-26 19:59:24,015][105692] Updated weights for policy 0, policy_version 635040 (0.0008) [2023-12-26 19:59:24,616][105620] Updated weights for policy 1, policy_version 635822 (0.0008) [2023-12-26 19:59:24,666][105620] Updated weights for policy 1, policy_version 635832 (0.0009) [2023-12-26 19:59:24,714][105692] Updated weights for policy 0, policy_version 635050 (0.0009) [2023-12-26 19:59:24,724][105620] Updated weights for policy 1, policy_version 635842 (0.0007) [2023-12-26 19:59:24,766][105692] Updated weights for policy 0, policy_version 635060 (0.0010) [2023-12-26 19:59:24,814][105692] Updated weights for policy 0, policy_version 635070 (0.0010) [2023-12-26 19:59:24,860][105692] Updated weights for policy 0, policy_version 635080 (0.0010) [2023-12-26 19:59:25,363][105620] Updated weights for policy 1, policy_version 635852 (0.0005) [2023-12-26 19:59:25,420][105620] Updated weights for policy 1, policy_version 635862 (0.0006) [2023-12-26 19:59:25,475][105620] Updated weights for policy 1, policy_version 635872 (0.0006) [2023-12-26 19:59:25,509][105692] Updated weights for policy 0, policy_version 635090 (0.0006) [2023-12-26 19:59:25,565][105692] Updated weights for policy 0, policy_version 635100 (0.0006) [2023-12-26 19:59:25,621][105692] Updated weights for policy 0, policy_version 635110 (0.0007) [2023-12-26 19:59:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19933.8, 300 sec: 19577.5). Total num frames: 325419008. Throughput: 0: 9920.9, 1: 9984.2. Samples: 325431284. Policy #0 lag: (min: 1.0, avg: 22.7, max: 33.0) [2023-12-26 19:59:26,063][104569] Avg episode reward: [(0, '8634.832'), (1, '8845.576')] [2023-12-26 19:59:26,108][105620] Updated weights for policy 1, policy_version 635882 (0.0008) [2023-12-26 19:59:26,175][105620] Updated weights for policy 1, policy_version 635892 (0.0005) [2023-12-26 19:59:26,206][105692] Updated weights for policy 0, policy_version 635120 (0.0010) [2023-12-26 19:59:26,234][105620] Updated weights for policy 1, policy_version 635902 (0.0006) [2023-12-26 19:59:26,260][105692] Updated weights for policy 0, policy_version 635130 (0.0010) [2023-12-26 19:59:26,291][105620] Updated weights for policy 1, policy_version 635912 (0.0006) [2023-12-26 19:59:26,315][105692] Updated weights for policy 0, policy_version 635140 (0.0010) [2023-12-26 19:59:26,831][105620] Updated weights for policy 1, policy_version 635922 (0.0010) [2023-12-26 19:59:26,896][105620] Updated weights for policy 1, policy_version 635932 (0.0010) [2023-12-26 19:59:26,960][105620] Updated weights for policy 1, policy_version 635942 (0.0010) [2023-12-26 19:59:26,971][105692] Updated weights for policy 0, policy_version 635150 (0.0010) [2023-12-26 19:59:27,021][105692] Updated weights for policy 0, policy_version 635160 (0.0010) [2023-12-26 19:59:27,075][105692] Updated weights for policy 0, policy_version 635170 (0.0010) [2023-12-26 19:59:27,676][105620] Updated weights for policy 1, policy_version 635952 (0.0010) [2023-12-26 19:59:27,707][105692] Updated weights for policy 0, policy_version 635180 (0.0008) [2023-12-26 19:59:27,731][105620] Updated weights for policy 1, policy_version 635962 (0.0010) [2023-12-26 19:59:27,760][105692] Updated weights for policy 0, policy_version 635190 (0.0005) [2023-12-26 19:59:27,791][105620] Updated weights for policy 1, policy_version 635972 (0.0010) [2023-12-26 19:59:27,805][105692] Updated weights for policy 0, policy_version 635200 (0.0007) [2023-12-26 19:59:28,499][105692] Updated weights for policy 0, policy_version 635210 (0.0007) [2023-12-26 19:59:28,528][105620] Updated weights for policy 1, policy_version 635982 (0.0011) [2023-12-26 19:59:28,558][105692] Updated weights for policy 0, policy_version 635220 (0.0009) [2023-12-26 19:59:28,584][105620] Updated weights for policy 1, policy_version 635992 (0.0008) [2023-12-26 19:59:28,614][105692] Updated weights for policy 0, policy_version 635230 (0.0010) [2023-12-26 19:59:28,636][105620] Updated weights for policy 1, policy_version 636002 (0.0008) [2023-12-26 19:59:28,668][105692] Updated weights for policy 0, policy_version 635240 (0.0010) [2023-12-26 19:59:29,321][105692] Updated weights for policy 0, policy_version 635250 (0.0011) [2023-12-26 19:59:29,385][105620] Updated weights for policy 1, policy_version 636012 (0.0010) [2023-12-26 19:59:29,386][105692] Updated weights for policy 0, policy_version 635260 (0.0008) [2023-12-26 19:59:29,437][105620] Updated weights for policy 1, policy_version 636022 (0.0010) [2023-12-26 19:59:29,443][105692] Updated weights for policy 0, policy_version 635270 (0.0006) [2023-12-26 19:59:29,482][105620] Updated weights for policy 1, policy_version 636032 (0.0010) [2023-12-26 19:59:30,139][105692] Updated weights for policy 0, policy_version 635280 (0.0008) [2023-12-26 19:59:30,187][105692] Updated weights for policy 0, policy_version 635290 (0.0008) [2023-12-26 19:59:30,231][105692] Updated weights for policy 0, policy_version 635300 (0.0008) [2023-12-26 19:59:30,248][105620] Updated weights for policy 1, policy_version 636042 (0.0010) [2023-12-26 19:59:30,309][105620] Updated weights for policy 1, policy_version 636052 (0.0010) [2023-12-26 19:59:30,369][105620] Updated weights for policy 1, policy_version 636062 (0.0006) [2023-12-26 19:59:30,424][105620] Updated weights for policy 1, policy_version 636072 (0.0006) [2023-12-26 19:59:30,927][105692] Updated weights for policy 0, policy_version 635310 (0.0006) [2023-12-26 19:59:30,975][105692] Updated weights for policy 0, policy_version 635320 (0.0005) [2023-12-26 19:59:31,035][105692] Updated weights for policy 0, policy_version 635330 (0.0006) [2023-12-26 19:59:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 325517312. Throughput: 0: 9997.3, 1: 10004.6. Samples: 325493928. Policy #0 lag: (min: 1.0, avg: 22.7, max: 33.0) [2023-12-26 19:59:31,063][104569] Avg episode reward: [(0, '1083.148'), (1, '8739.039')] [2023-12-26 19:59:31,075][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000635336_162676736.pth... [2023-12-26 19:59:31,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000634152_162373632.pth [2023-12-26 19:59:31,108][105620] Updated weights for policy 1, policy_version 636082 (0.0010) [2023-12-26 19:59:31,171][105620] Updated weights for policy 1, policy_version 636092 (0.0009) [2023-12-26 19:59:31,244][105620] Updated weights for policy 1, policy_version 636102 (0.0007) [2023-12-26 19:59:31,257][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000636104_162856960.pth... [2023-12-26 19:59:31,261][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000634920_162553856.pth [2023-12-26 19:59:31,644][105692] Updated weights for policy 0, policy_version 635340 (0.0006) [2023-12-26 19:59:31,706][105692] Updated weights for policy 0, policy_version 635350 (0.0009) [2023-12-26 19:59:31,771][105692] Updated weights for policy 0, policy_version 635360 (0.0008) [2023-12-26 19:59:31,961][105620] Updated weights for policy 1, policy_version 636112 (0.0006) [2023-12-26 19:59:32,026][105620] Updated weights for policy 1, policy_version 636122 (0.0008) [2023-12-26 19:59:32,082][105620] Updated weights for policy 1, policy_version 636132 (0.0009) [2023-12-26 19:59:32,541][105692] Updated weights for policy 0, policy_version 635370 (0.0009) [2023-12-26 19:59:32,605][105692] Updated weights for policy 0, policy_version 635380 (0.0008) [2023-12-26 19:59:32,660][105692] Updated weights for policy 0, policy_version 635390 (0.0008) [2023-12-26 19:59:32,717][105692] Updated weights for policy 0, policy_version 635400 (0.0006) [2023-12-26 19:59:32,787][105620] Updated weights for policy 1, policy_version 636142 (0.0008) [2023-12-26 19:59:32,850][105620] Updated weights for policy 1, policy_version 636152 (0.0010) [2023-12-26 19:59:32,914][105620] Updated weights for policy 1, policy_version 636162 (0.0008) [2023-12-26 19:59:33,487][105692] Updated weights for policy 0, policy_version 635410 (0.0009) [2023-12-26 19:59:33,554][105692] Updated weights for policy 0, policy_version 635420 (0.0010) [2023-12-26 19:59:33,566][105620] Updated weights for policy 1, policy_version 636172 (0.0007) [2023-12-26 19:59:33,619][105692] Updated weights for policy 0, policy_version 635430 (0.0010) [2023-12-26 19:59:33,619][105620] Updated weights for policy 1, policy_version 636182 (0.0010) [2023-12-26 19:59:33,677][105620] Updated weights for policy 1, policy_version 636192 (0.0010) [2023-12-26 19:59:34,298][105692] Updated weights for policy 0, policy_version 635440 (0.0009) [2023-12-26 19:59:34,350][105620] Updated weights for policy 1, policy_version 636202 (0.0008) [2023-12-26 19:59:34,358][105692] Updated weights for policy 0, policy_version 635450 (0.0009) [2023-12-26 19:59:34,416][105620] Updated weights for policy 1, policy_version 636212 (0.0008) [2023-12-26 19:59:34,418][105692] Updated weights for policy 0, policy_version 635460 (0.0009) [2023-12-26 19:59:34,478][105620] Updated weights for policy 1, policy_version 636222 (0.0011) [2023-12-26 19:59:34,541][105620] Updated weights for policy 1, policy_version 636232 (0.0008) [2023-12-26 19:59:35,052][105692] Updated weights for policy 0, policy_version 635470 (0.0009) [2023-12-26 19:59:35,121][105692] Updated weights for policy 0, policy_version 635480 (0.0005) [2023-12-26 19:59:35,176][105692] Updated weights for policy 0, policy_version 635490 (0.0006) [2023-12-26 19:59:35,239][105620] Updated weights for policy 1, policy_version 636242 (0.0007) [2023-12-26 19:59:35,302][105620] Updated weights for policy 1, policy_version 636252 (0.0008) [2023-12-26 19:59:35,359][105620] Updated weights for policy 1, policy_version 636262 (0.0009) [2023-12-26 19:59:35,839][105692] Updated weights for policy 0, policy_version 635500 (0.0009) [2023-12-26 19:59:35,899][105692] Updated weights for policy 0, policy_version 635510 (0.0011) [2023-12-26 19:59:35,914][105620] Updated weights for policy 1, policy_version 636272 (0.0008) [2023-12-26 19:59:35,955][105692] Updated weights for policy 0, policy_version 635520 (0.0011) [2023-12-26 19:59:35,971][105620] Updated weights for policy 1, policy_version 636282 (0.0009) [2023-12-26 19:59:36,028][105620] Updated weights for policy 1, policy_version 636292 (0.0008) [2023-12-26 19:59:36,062][104569] Fps is (10 sec: 21299.6, 60 sec: 20070.4, 300 sec: 19605.3). Total num frames: 325632000. Throughput: 0: 9972.2, 1: 9875.0. Samples: 325612540. Policy #0 lag: (min: 1.0, avg: 22.7, max: 33.0) [2023-12-26 19:59:36,062][104569] Avg episode reward: [(0, '1191.524'), (1, '8727.092')] [2023-12-26 19:59:36,714][105692] Updated weights for policy 0, policy_version 635530 (0.0010) [2023-12-26 19:59:36,773][105692] Updated weights for policy 0, policy_version 635540 (0.0011) [2023-12-26 19:59:36,801][105620] Updated weights for policy 1, policy_version 636302 (0.0008) [2023-12-26 19:59:36,833][105692] Updated weights for policy 0, policy_version 635550 (0.0010) [2023-12-26 19:59:36,847][105620] Updated weights for policy 1, policy_version 636312 (0.0007) [2023-12-26 19:59:36,895][105692] Updated weights for policy 0, policy_version 635560 (0.0010) [2023-12-26 19:59:36,897][105620] Updated weights for policy 1, policy_version 636322 (0.0008) [2023-12-26 19:59:37,638][105692] Updated weights for policy 0, policy_version 635570 (0.0010) [2023-12-26 19:59:37,668][105620] Updated weights for policy 1, policy_version 636332 (0.0007) [2023-12-26 19:59:37,690][105692] Updated weights for policy 0, policy_version 635580 (0.0011) [2023-12-26 19:59:37,721][105620] Updated weights for policy 1, policy_version 636342 (0.0006) [2023-12-26 19:59:37,751][105692] Updated weights for policy 0, policy_version 635590 (0.0010) [2023-12-26 19:59:37,775][105620] Updated weights for policy 1, policy_version 636352 (0.0006) [2023-12-26 19:59:38,515][105692] Updated weights for policy 0, policy_version 635600 (0.0011) [2023-12-26 19:59:38,545][105620] Updated weights for policy 1, policy_version 636362 (0.0007) [2023-12-26 19:59:38,573][105692] Updated weights for policy 0, policy_version 635610 (0.0010) [2023-12-26 19:59:38,590][105620] Updated weights for policy 1, policy_version 636372 (0.0008) [2023-12-26 19:59:38,629][105692] Updated weights for policy 0, policy_version 635620 (0.0010) [2023-12-26 19:59:38,646][105620] Updated weights for policy 1, policy_version 636382 (0.0006) [2023-12-26 19:59:38,700][105620] Updated weights for policy 1, policy_version 636392 (0.0008) [2023-12-26 19:59:39,388][105692] Updated weights for policy 0, policy_version 635630 (0.0011) [2023-12-26 19:59:39,451][105692] Updated weights for policy 0, policy_version 635640 (0.0011) [2023-12-26 19:59:39,490][105620] Updated weights for policy 1, policy_version 636402 (0.0009) [2023-12-26 19:59:39,503][105692] Updated weights for policy 0, policy_version 635650 (0.0010) [2023-12-26 19:59:39,539][105620] Updated weights for policy 1, policy_version 636412 (0.0009) [2023-12-26 19:59:39,599][105620] Updated weights for policy 1, policy_version 636422 (0.0008) [2023-12-26 19:59:40,255][105692] Updated weights for policy 0, policy_version 635660 (0.0011) [2023-12-26 19:59:40,307][105692] Updated weights for policy 0, policy_version 635670 (0.0010) [2023-12-26 19:59:40,367][105692] Updated weights for policy 0, policy_version 635680 (0.0008) [2023-12-26 19:59:40,373][105620] Updated weights for policy 1, policy_version 636432 (0.0008) [2023-12-26 19:59:40,439][105620] Updated weights for policy 1, policy_version 636442 (0.0007) [2023-12-26 19:59:40,500][105620] Updated weights for policy 1, policy_version 636452 (0.0005) [2023-12-26 19:59:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 325713920. Throughput: 0: 10000.1, 1: 9856.0. Samples: 325728376. Policy #0 lag: (min: 1.0, avg: 22.7, max: 33.0) [2023-12-26 19:59:41,063][104569] Avg episode reward: [(0, '3970.667'), (1, '8858.006')] [2023-12-26 19:59:41,108][105620] Updated weights for policy 1, policy_version 636462 (0.0010) [2023-12-26 19:59:41,147][105692] Updated weights for policy 0, policy_version 635690 (0.0010) [2023-12-26 19:59:41,172][105620] Updated weights for policy 1, policy_version 636472 (0.0007) [2023-12-26 19:59:41,203][105692] Updated weights for policy 0, policy_version 635700 (0.0009) [2023-12-26 19:59:41,234][105620] Updated weights for policy 1, policy_version 636482 (0.0008) [2023-12-26 19:59:41,273][105692] Updated weights for policy 0, policy_version 635710 (0.0009) [2023-12-26 19:59:41,326][105692] Updated weights for policy 0, policy_version 635720 (0.0010) [2023-12-26 19:59:42,003][105620] Updated weights for policy 1, policy_version 636492 (0.0009) [2023-12-26 19:59:42,063][105620] Updated weights for policy 1, policy_version 636502 (0.0009) [2023-12-26 19:59:42,110][105692] Updated weights for policy 0, policy_version 635730 (0.0010) [2023-12-26 19:59:42,117][105620] Updated weights for policy 1, policy_version 636512 (0.0006) [2023-12-26 19:59:42,159][105692] Updated weights for policy 0, policy_version 635740 (0.0010) [2023-12-26 19:59:42,214][105692] Updated weights for policy 0, policy_version 635750 (0.0010) [2023-12-26 19:59:42,899][105692] Updated weights for policy 0, policy_version 635760 (0.0006) [2023-12-26 19:59:42,932][105620] Updated weights for policy 1, policy_version 636522 (0.0005) [2023-12-26 19:59:42,966][105692] Updated weights for policy 0, policy_version 635770 (0.0007) [2023-12-26 19:59:42,993][105620] Updated weights for policy 1, policy_version 636532 (0.0005) [2023-12-26 19:59:43,020][105586] KL-divergence is very high: 119.2825 [2023-12-26 19:59:43,025][105692] Updated weights for policy 0, policy_version 635780 (0.0010) [2023-12-26 19:59:43,049][105620] Updated weights for policy 1, policy_version 636542 (0.0009) [2023-12-26 19:59:43,065][105586] KL-divergence is very high: 123.0579 [2023-12-26 19:59:43,101][105620] Updated weights for policy 1, policy_version 636552 (0.0010) [2023-12-26 19:59:43,581][105692] Updated weights for policy 0, policy_version 635790 (0.0010) [2023-12-26 19:59:43,638][105692] Updated weights for policy 0, policy_version 635800 (0.0006) [2023-12-26 19:59:43,694][105692] Updated weights for policy 0, policy_version 635810 (0.0005) [2023-12-26 19:59:43,724][105585] KL-divergence is very high: 102.2168 [2023-12-26 19:59:43,793][105620] Updated weights for policy 1, policy_version 636562 (0.0010) [2023-12-26 19:59:43,868][105620] Updated weights for policy 1, policy_version 636572 (0.0010) [2023-12-26 19:59:43,940][105620] Updated weights for policy 1, policy_version 636582 (0.0010) [2023-12-26 19:59:44,266][105585] KL-divergence is very high: 101.4291 [2023-12-26 19:59:44,273][105585] KL-divergence is very high: 100.8902 [2023-12-26 19:59:44,290][105692] Updated weights for policy 0, policy_version 635820 (0.0007) [2023-12-26 19:59:44,359][105692] Updated weights for policy 0, policy_version 635830 (0.0010) [2023-12-26 19:59:44,407][105692] Updated weights for policy 0, policy_version 635840 (0.0010) [2023-12-26 19:59:44,579][105620] Updated weights for policy 1, policy_version 636592 (0.0006) [2023-12-26 19:59:44,626][105620] Updated weights for policy 1, policy_version 636602 (0.0005) [2023-12-26 19:59:44,676][105620] Updated weights for policy 1, policy_version 636612 (0.0008) [2023-12-26 19:59:45,166][105692] Updated weights for policy 0, policy_version 635850 (0.0011) [2023-12-26 19:59:45,223][105692] Updated weights for policy 0, policy_version 635860 (0.0011) [2023-12-26 19:59:45,289][105692] Updated weights for policy 0, policy_version 635870 (0.0011) [2023-12-26 19:59:45,356][105692] Updated weights for policy 0, policy_version 635880 (0.0010) [2023-12-26 19:59:45,429][105620] Updated weights for policy 1, policy_version 636622 (0.0008) [2023-12-26 19:59:45,493][105620] Updated weights for policy 1, policy_version 636632 (0.0008) [2023-12-26 19:59:45,542][105620] Updated weights for policy 1, policy_version 636642 (0.0008) [2023-12-26 19:59:46,062][104569] Fps is (10 sec: 18022.1, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 325812224. Throughput: 0: 9955.3, 1: 9774.4. Samples: 325786032. Policy #0 lag: (min: 1.0, avg: 22.7, max: 33.0) [2023-12-26 19:59:46,062][104569] Avg episode reward: [(0, '940.463'), (1, '9091.321')] [2023-12-26 19:59:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000636648_162996224.pth... [2023-12-26 19:59:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000635528_162709504.pth [2023-12-26 19:59:46,097][105692] Updated weights for policy 0, policy_version 635890 (0.0011) [2023-12-26 19:59:46,158][105692] Updated weights for policy 0, policy_version 635900 (0.0010) [2023-12-26 19:59:46,216][105692] Updated weights for policy 0, policy_version 635910 (0.0010) [2023-12-26 19:59:46,225][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000635912_162824192.pth... [2023-12-26 19:59:46,230][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000634728_162521088.pth [2023-12-26 19:59:46,299][105620] Updated weights for policy 1, policy_version 636652 (0.0009) [2023-12-26 19:59:46,350][105620] Updated weights for policy 1, policy_version 636662 (0.0008) [2023-12-26 19:59:46,399][105620] Updated weights for policy 1, policy_version 636672 (0.0008) [2023-12-26 19:59:46,947][105692] Updated weights for policy 0, policy_version 635920 (0.0009) [2023-12-26 19:59:47,005][105692] Updated weights for policy 0, policy_version 635930 (0.0010) [2023-12-26 19:59:47,062][105692] Updated weights for policy 0, policy_version 635940 (0.0008) [2023-12-26 19:59:47,207][105620] Updated weights for policy 1, policy_version 636682 (0.0008) [2023-12-26 19:59:47,265][105620] Updated weights for policy 1, policy_version 636692 (0.0010) [2023-12-26 19:59:47,322][105620] Updated weights for policy 1, policy_version 636702 (0.0010) [2023-12-26 19:59:47,379][105620] Updated weights for policy 1, policy_version 636712 (0.0009) [2023-12-26 19:59:47,628][105692] Updated weights for policy 0, policy_version 635950 (0.0008) [2023-12-26 19:59:47,686][105692] Updated weights for policy 0, policy_version 635960 (0.0010) [2023-12-26 19:59:47,737][105692] Updated weights for policy 0, policy_version 635970 (0.0010) [2023-12-26 19:59:48,074][105620] Updated weights for policy 1, policy_version 636722 (0.0011) [2023-12-26 19:59:48,124][105620] Updated weights for policy 1, policy_version 636732 (0.0011) [2023-12-26 19:59:48,180][105620] Updated weights for policy 1, policy_version 636742 (0.0011) [2023-12-26 19:59:48,485][105692] Updated weights for policy 0, policy_version 635980 (0.0009) [2023-12-26 19:59:48,538][105692] Updated weights for policy 0, policy_version 635990 (0.0008) [2023-12-26 19:59:48,598][105692] Updated weights for policy 0, policy_version 636000 (0.0008) [2023-12-26 19:59:48,615][105585] KL-divergence is very high: 105.2122 [2023-12-26 19:59:48,958][105620] Updated weights for policy 1, policy_version 636752 (0.0011) [2023-12-26 19:59:49,020][105620] Updated weights for policy 1, policy_version 636762 (0.0010) [2023-12-26 19:59:49,068][105620] Updated weights for policy 1, policy_version 636772 (0.0010) [2023-12-26 19:59:49,366][105692] Updated weights for policy 0, policy_version 636010 (0.0007) [2023-12-26 19:59:49,433][105692] Updated weights for policy 0, policy_version 636020 (0.0007) [2023-12-26 19:59:49,499][105692] Updated weights for policy 0, policy_version 636030 (0.0006) [2023-12-26 19:59:49,560][105692] Updated weights for policy 0, policy_version 636040 (0.0007) [2023-12-26 19:59:49,867][105620] Updated weights for policy 1, policy_version 636782 (0.0008) [2023-12-26 19:59:49,931][105620] Updated weights for policy 1, policy_version 636792 (0.0007) [2023-12-26 19:59:49,985][105620] Updated weights for policy 1, policy_version 636802 (0.0007) [2023-12-26 19:59:50,263][105692] Updated weights for policy 0, policy_version 636050 (0.0008) [2023-12-26 19:59:50,312][105692] Updated weights for policy 0, policy_version 636060 (0.0005) [2023-12-26 19:59:50,377][105692] Updated weights for policy 0, policy_version 636070 (0.0005) [2023-12-26 19:59:50,764][105620] Updated weights for policy 1, policy_version 636812 (0.0009) [2023-12-26 19:59:50,820][105620] Updated weights for policy 1, policy_version 636822 (0.0007) [2023-12-26 19:59:50,887][105620] Updated weights for policy 1, policy_version 636832 (0.0008) [2023-12-26 19:59:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 325910528. Throughput: 0: 9977.9, 1: 9699.8. Samples: 325901872. Policy #0 lag: (min: 1.0, avg: 22.7, max: 33.0) [2023-12-26 19:59:51,062][104569] Avg episode reward: [(0, '1285.024'), (1, '9091.440')] [2023-12-26 19:59:51,101][105692] Updated weights for policy 0, policy_version 636080 (0.0010) [2023-12-26 19:59:51,169][105692] Updated weights for policy 0, policy_version 636090 (0.0011) [2023-12-26 19:59:51,234][105692] Updated weights for policy 0, policy_version 636100 (0.0011) [2023-12-26 19:59:51,668][105620] Updated weights for policy 1, policy_version 636842 (0.0008) [2023-12-26 19:59:51,734][105620] Updated weights for policy 1, policy_version 636852 (0.0008) [2023-12-26 19:59:51,795][105620] Updated weights for policy 1, policy_version 636862 (0.0010) [2023-12-26 19:59:51,856][105620] Updated weights for policy 1, policy_version 636872 (0.0009) [2023-12-26 19:59:51,955][105692] Updated weights for policy 0, policy_version 636110 (0.0010) [2023-12-26 19:59:52,020][105692] Updated weights for policy 0, policy_version 636120 (0.0009) [2023-12-26 19:59:52,084][105692] Updated weights for policy 0, policy_version 636130 (0.0009) [2023-12-26 19:59:52,652][105620] Updated weights for policy 1, policy_version 636882 (0.0008) [2023-12-26 19:59:52,716][105620] Updated weights for policy 1, policy_version 636892 (0.0008) [2023-12-26 19:59:52,774][105620] Updated weights for policy 1, policy_version 636902 (0.0008) [2023-12-26 19:59:52,850][105692] Updated weights for policy 0, policy_version 636140 (0.0009) [2023-12-26 19:59:52,899][105692] Updated weights for policy 0, policy_version 636150 (0.0009) [2023-12-26 19:59:52,954][105692] Updated weights for policy 0, policy_version 636160 (0.0009) [2023-12-26 19:59:53,397][105620] Updated weights for policy 1, policy_version 636912 (0.0009) [2023-12-26 19:59:53,453][105620] Updated weights for policy 1, policy_version 636922 (0.0010) [2023-12-26 19:59:53,518][105620] Updated weights for policy 1, policy_version 636932 (0.0009) [2023-12-26 19:59:53,760][105692] Updated weights for policy 0, policy_version 636170 (0.0009) [2023-12-26 19:59:53,821][105692] Updated weights for policy 0, policy_version 636180 (0.0009) [2023-12-26 19:59:53,875][105692] Updated weights for policy 0, policy_version 636190 (0.0008) [2023-12-26 19:59:53,932][105692] Updated weights for policy 0, policy_version 636200 (0.0009) [2023-12-26 19:59:54,291][105620] Updated weights for policy 1, policy_version 636942 (0.0007) [2023-12-26 19:59:54,353][105620] Updated weights for policy 1, policy_version 636952 (0.0005) [2023-12-26 19:59:54,412][105620] Updated weights for policy 1, policy_version 636962 (0.0006) [2023-12-26 19:59:54,649][105692] Updated weights for policy 0, policy_version 636210 (0.0011) [2023-12-26 19:59:54,704][105692] Updated weights for policy 0, policy_version 636220 (0.0010) [2023-12-26 19:59:54,749][105692] Updated weights for policy 0, policy_version 636230 (0.0010) [2023-12-26 19:59:55,098][105620] Updated weights for policy 1, policy_version 636972 (0.0008) [2023-12-26 19:59:55,152][105620] Updated weights for policy 1, policy_version 636982 (0.0009) [2023-12-26 19:59:55,209][105620] Updated weights for policy 1, policy_version 636992 (0.0010) [2023-12-26 19:59:55,450][105692] Updated weights for policy 0, policy_version 636240 (0.0011) [2023-12-26 19:59:55,508][105692] Updated weights for policy 0, policy_version 636250 (0.0010) [2023-12-26 19:59:55,563][105692] Updated weights for policy 0, policy_version 636260 (0.0010) [2023-12-26 19:59:55,999][105620] Updated weights for policy 1, policy_version 637002 (0.0009) [2023-12-26 19:59:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 326000640. Throughput: 0: 9979.0, 1: 9609.0. Samples: 326014696. Policy #0 lag: (min: 1.0, avg: 22.7, max: 33.0) [2023-12-26 19:59:56,063][104569] Avg episode reward: [(0, '3003.296'), (1, '9262.932')] [2023-12-26 19:59:56,065][105620] Updated weights for policy 1, policy_version 637012 (0.0008) [2023-12-26 19:59:56,124][105620] Updated weights for policy 1, policy_version 637022 (0.0008) [2023-12-26 19:59:56,191][105620] Updated weights for policy 1, policy_version 637032 (0.0008) [2023-12-26 19:59:56,293][105692] Updated weights for policy 0, policy_version 636270 (0.0007) [2023-12-26 19:59:56,350][105692] Updated weights for policy 0, policy_version 636280 (0.0005) [2023-12-26 19:59:56,405][105692] Updated weights for policy 0, policy_version 636290 (0.0005) [2023-12-26 19:59:56,911][105692] Updated weights for policy 0, policy_version 636300 (0.0005) [2023-12-26 19:59:56,973][105692] Updated weights for policy 0, policy_version 636310 (0.0005) [2023-12-26 19:59:57,035][105692] Updated weights for policy 0, policy_version 636320 (0.0006) [2023-12-26 19:59:57,058][105620] Updated weights for policy 1, policy_version 637042 (0.0009) [2023-12-26 19:59:57,118][105620] Updated weights for policy 1, policy_version 637052 (0.0008) [2023-12-26 19:59:57,187][105620] Updated weights for policy 1, policy_version 637062 (0.0006) [2023-12-26 19:59:57,720][105692] Updated weights for policy 0, policy_version 636330 (0.0009) [2023-12-26 19:59:57,737][105620] Updated weights for policy 1, policy_version 637072 (0.0005) [2023-12-26 19:59:57,775][105692] Updated weights for policy 0, policy_version 636340 (0.0010) [2023-12-26 19:59:57,793][105620] Updated weights for policy 1, policy_version 637082 (0.0006) [2023-12-26 19:59:57,832][105692] Updated weights for policy 0, policy_version 636350 (0.0010) [2023-12-26 19:59:57,850][105620] Updated weights for policy 1, policy_version 637092 (0.0006) [2023-12-26 19:59:57,885][105692] Updated weights for policy 0, policy_version 636360 (0.0010) [2023-12-26 19:59:58,593][105692] Updated weights for policy 0, policy_version 636370 (0.0010) [2023-12-26 19:59:58,620][105620] Updated weights for policy 1, policy_version 637102 (0.0008) [2023-12-26 19:59:58,658][105692] Updated weights for policy 0, policy_version 636380 (0.0010) [2023-12-26 19:59:58,688][105620] Updated weights for policy 1, policy_version 637112 (0.0008) [2023-12-26 19:59:58,729][105692] Updated weights for policy 0, policy_version 636390 (0.0009) [2023-12-26 19:59:58,753][105620] Updated weights for policy 1, policy_version 637122 (0.0009) [2023-12-26 19:59:59,444][105692] Updated weights for policy 0, policy_version 636400 (0.0006) [2023-12-26 19:59:59,463][105620] Updated weights for policy 1, policy_version 637132 (0.0008) [2023-12-26 19:59:59,508][105692] Updated weights for policy 0, policy_version 636410 (0.0005) [2023-12-26 19:59:59,514][105620] Updated weights for policy 1, policy_version 637142 (0.0007) [2023-12-26 19:59:59,565][105692] Updated weights for policy 0, policy_version 636420 (0.0006) [2023-12-26 19:59:59,567][105620] Updated weights for policy 1, policy_version 637152 (0.0008) [2023-12-26 20:00:00,215][105692] Updated weights for policy 0, policy_version 636430 (0.0007) [2023-12-26 20:00:00,276][105692] Updated weights for policy 0, policy_version 636440 (0.0005) [2023-12-26 20:00:00,336][105620] Updated weights for policy 1, policy_version 637162 (0.0009) [2023-12-26 20:00:00,337][105692] Updated weights for policy 0, policy_version 636450 (0.0007) [2023-12-26 20:00:00,392][105620] Updated weights for policy 1, policy_version 637172 (0.0009) [2023-12-26 20:00:00,446][105620] Updated weights for policy 1, policy_version 637182 (0.0010) [2023-12-26 20:00:00,500][105620] Updated weights for policy 1, policy_version 637192 (0.0010) [2023-12-26 20:00:00,957][105692] Updated weights for policy 0, policy_version 636460 (0.0008) [2023-12-26 20:00:01,003][105692] Updated weights for policy 0, policy_version 636470 (0.0006) [2023-12-26 20:00:01,056][105692] Updated weights for policy 0, policy_version 636480 (0.0006) [2023-12-26 20:00:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 326098944. Throughput: 0: 9963.9, 1: 9614.5. Samples: 326074252. Policy #0 lag: (min: 1.0, avg: 22.7, max: 33.0) [2023-12-26 20:00:01,063][104569] Avg episode reward: [(0, '5082.888'), (1, '8989.652')] [2023-12-26 20:00:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000637192_163135488.pth... [2023-12-26 20:00:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000636104_162856960.pth [2023-12-26 20:00:01,101][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000636488_162971648.pth... [2023-12-26 20:00:01,104][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000635336_162676736.pth [2023-12-26 20:00:01,286][105620] Updated weights for policy 1, policy_version 637202 (0.0009) [2023-12-26 20:00:01,345][105620] Updated weights for policy 1, policy_version 637212 (0.0009) [2023-12-26 20:00:01,404][105620] Updated weights for policy 1, policy_version 637222 (0.0009) [2023-12-26 20:00:01,773][105692] Updated weights for policy 0, policy_version 636490 (0.0007) [2023-12-26 20:00:01,824][105692] Updated weights for policy 0, policy_version 636500 (0.0009) [2023-12-26 20:00:01,871][105692] Updated weights for policy 0, policy_version 636510 (0.0009) [2023-12-26 20:00:01,921][105692] Updated weights for policy 0, policy_version 636520 (0.0008) [2023-12-26 20:00:02,146][105620] Updated weights for policy 1, policy_version 637232 (0.0010) [2023-12-26 20:00:02,203][105620] Updated weights for policy 1, policy_version 637242 (0.0010) [2023-12-26 20:00:02,261][105620] Updated weights for policy 1, policy_version 637252 (0.0010) [2023-12-26 20:00:02,779][105692] Updated weights for policy 0, policy_version 636530 (0.0009) [2023-12-26 20:00:02,848][105692] Updated weights for policy 0, policy_version 636540 (0.0010) [2023-12-26 20:00:02,916][105692] Updated weights for policy 0, policy_version 636550 (0.0009) [2023-12-26 20:00:02,934][105620] Updated weights for policy 1, policy_version 637262 (0.0008) [2023-12-26 20:00:03,001][105620] Updated weights for policy 1, policy_version 637272 (0.0007) [2023-12-26 20:00:03,052][105620] Updated weights for policy 1, policy_version 637282 (0.0005) [2023-12-26 20:00:03,658][105692] Updated weights for policy 0, policy_version 636560 (0.0009) [2023-12-26 20:00:03,707][105620] Updated weights for policy 1, policy_version 637292 (0.0007) [2023-12-26 20:00:03,720][105692] Updated weights for policy 0, policy_version 636570 (0.0009) [2023-12-26 20:00:03,758][105620] Updated weights for policy 1, policy_version 637302 (0.0005) [2023-12-26 20:00:03,772][105692] Updated weights for policy 0, policy_version 636580 (0.0007) [2023-12-26 20:00:03,810][105620] Updated weights for policy 1, policy_version 637312 (0.0007) [2023-12-26 20:00:04,532][105620] Updated weights for policy 1, policy_version 637322 (0.0008) [2023-12-26 20:00:04,557][105692] Updated weights for policy 0, policy_version 636590 (0.0005) [2023-12-26 20:00:04,591][105620] Updated weights for policy 1, policy_version 637332 (0.0005) [2023-12-26 20:00:04,621][105692] Updated weights for policy 0, policy_version 636600 (0.0007) [2023-12-26 20:00:04,646][105620] Updated weights for policy 1, policy_version 637342 (0.0006) [2023-12-26 20:00:04,674][105692] Updated weights for policy 0, policy_version 636610 (0.0011) [2023-12-26 20:00:04,699][105620] Updated weights for policy 1, policy_version 637352 (0.0009) [2023-12-26 20:00:05,328][105692] Updated weights for policy 0, policy_version 636620 (0.0008) [2023-12-26 20:00:05,386][105692] Updated weights for policy 0, policy_version 636630 (0.0007) [2023-12-26 20:00:05,436][105620] Updated weights for policy 1, policy_version 637362 (0.0007) [2023-12-26 20:00:05,446][105692] Updated weights for policy 0, policy_version 636640 (0.0006) [2023-12-26 20:00:05,489][105620] Updated weights for policy 1, policy_version 637372 (0.0005) [2023-12-26 20:00:05,539][105620] Updated weights for policy 1, policy_version 637382 (0.0007) [2023-12-26 20:00:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 326197248. Throughput: 0: 9859.6, 1: 9555.1. Samples: 326189900. Policy #0 lag: (min: 1.0, avg: 22.7, max: 33.0) [2023-12-26 20:00:06,062][104569] Avg episode reward: [(0, '6709.705'), (1, '9030.232')] [2023-12-26 20:00:06,111][105692] Updated weights for policy 0, policy_version 636650 (0.0009) [2023-12-26 20:00:06,165][105692] Updated weights for policy 0, policy_version 636660 (0.0008) [2023-12-26 20:00:06,212][105692] Updated weights for policy 0, policy_version 636670 (0.0008) [2023-12-26 20:00:06,268][105692] Updated weights for policy 0, policy_version 636680 (0.0008) [2023-12-26 20:00:06,325][105620] Updated weights for policy 1, policy_version 637392 (0.0008) [2023-12-26 20:00:06,389][105620] Updated weights for policy 1, policy_version 637402 (0.0008) [2023-12-26 20:00:06,445][105620] Updated weights for policy 1, policy_version 637412 (0.0009) [2023-12-26 20:00:06,907][105692] Updated weights for policy 0, policy_version 636690 (0.0009) [2023-12-26 20:00:06,969][105692] Updated weights for policy 0, policy_version 636700 (0.0009) [2023-12-26 20:00:07,032][105692] Updated weights for policy 0, policy_version 636710 (0.0008) [2023-12-26 20:00:07,311][105620] Updated weights for policy 1, policy_version 637422 (0.0009) [2023-12-26 20:00:07,368][105620] Updated weights for policy 1, policy_version 637432 (0.0009) [2023-12-26 20:00:07,430][105620] Updated weights for policy 1, policy_version 637442 (0.0009) [2023-12-26 20:00:07,701][105692] Updated weights for policy 0, policy_version 636720 (0.0008) [2023-12-26 20:00:07,757][105692] Updated weights for policy 0, policy_version 636730 (0.0006) [2023-12-26 20:00:07,823][105692] Updated weights for policy 0, policy_version 636740 (0.0005) [2023-12-26 20:00:08,109][105620] Updated weights for policy 1, policy_version 637452 (0.0009) [2023-12-26 20:00:08,160][105620] Updated weights for policy 1, policy_version 637462 (0.0010) [2023-12-26 20:00:08,214][105620] Updated weights for policy 1, policy_version 637472 (0.0009) [2023-12-26 20:00:08,517][105692] Updated weights for policy 0, policy_version 636750 (0.0008) [2023-12-26 20:00:08,576][105692] Updated weights for policy 0, policy_version 636760 (0.0009) [2023-12-26 20:00:08,628][105692] Updated weights for policy 0, policy_version 636770 (0.0008) [2023-12-26 20:00:08,951][105620] Updated weights for policy 1, policy_version 637482 (0.0009) [2023-12-26 20:00:09,020][105620] Updated weights for policy 1, policy_version 637492 (0.0009) [2023-12-26 20:00:09,078][105620] Updated weights for policy 1, policy_version 637502 (0.0009) [2023-12-26 20:00:09,143][105620] Updated weights for policy 1, policy_version 637512 (0.0009) [2023-12-26 20:00:09,409][105692] Updated weights for policy 0, policy_version 636780 (0.0009) [2023-12-26 20:00:09,473][105692] Updated weights for policy 0, policy_version 636790 (0.0009) [2023-12-26 20:00:09,537][105692] Updated weights for policy 0, policy_version 636800 (0.0008) [2023-12-26 20:00:09,935][105620] Updated weights for policy 1, policy_version 637522 (0.0008) [2023-12-26 20:00:09,997][105620] Updated weights for policy 1, policy_version 637532 (0.0009) [2023-12-26 20:00:10,057][105620] Updated weights for policy 1, policy_version 637542 (0.0009) [2023-12-26 20:00:10,269][105692] Updated weights for policy 0, policy_version 636810 (0.0008) [2023-12-26 20:00:10,321][105692] Updated weights for policy 0, policy_version 636820 (0.0010) [2023-12-26 20:00:10,369][105692] Updated weights for policy 0, policy_version 636830 (0.0008) [2023-12-26 20:00:10,435][105692] Updated weights for policy 0, policy_version 636840 (0.0009) [2023-12-26 20:00:10,822][105620] Updated weights for policy 1, policy_version 637552 (0.0008) [2023-12-26 20:00:10,880][105620] Updated weights for policy 1, policy_version 637562 (0.0009) [2023-12-26 20:00:10,928][105620] Updated weights for policy 1, policy_version 637572 (0.0006) [2023-12-26 20:00:11,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 326295552. Throughput: 0: 9856.2, 1: 9543.5. Samples: 326304264. Policy #0 lag: (min: 1.0, avg: 22.7, max: 33.0) [2023-12-26 20:00:11,062][104569] Avg episode reward: [(0, '8286.836'), (1, '7825.658')] [2023-12-26 20:00:11,248][105692] Updated weights for policy 0, policy_version 636850 (0.0009) [2023-12-26 20:00:11,304][105692] Updated weights for policy 0, policy_version 636860 (0.0009) [2023-12-26 20:00:11,372][105692] Updated weights for policy 0, policy_version 636870 (0.0009) [2023-12-26 20:00:11,662][105620] Updated weights for policy 1, policy_version 637582 (0.0008) [2023-12-26 20:00:11,704][105586] KL-divergence is very high: 100.9917 [2023-12-26 20:00:11,726][105620] Updated weights for policy 1, policy_version 637592 (0.0009) [2023-12-26 20:00:11,788][105620] Updated weights for policy 1, policy_version 637602 (0.0008) [2023-12-26 20:00:12,113][105692] Updated weights for policy 0, policy_version 636880 (0.0009) [2023-12-26 20:00:12,172][105692] Updated weights for policy 0, policy_version 636890 (0.0009) [2023-12-26 20:00:12,224][105692] Updated weights for policy 0, policy_version 636900 (0.0010) [2023-12-26 20:00:12,625][105620] Updated weights for policy 1, policy_version 637612 (0.0009) [2023-12-26 20:00:12,681][105620] Updated weights for policy 1, policy_version 637622 (0.0008) [2023-12-26 20:00:12,737][105620] Updated weights for policy 1, policy_version 637632 (0.0008) [2023-12-26 20:00:12,901][105692] Updated weights for policy 0, policy_version 636910 (0.0011) [2023-12-26 20:00:12,965][105692] Updated weights for policy 0, policy_version 636920 (0.0008) [2023-12-26 20:00:13,024][105692] Updated weights for policy 0, policy_version 636930 (0.0005) [2023-12-26 20:00:13,539][105620] Updated weights for policy 1, policy_version 637642 (0.0008) [2023-12-26 20:00:13,546][105692] Updated weights for policy 0, policy_version 636940 (0.0007) [2023-12-26 20:00:13,594][105620] Updated weights for policy 1, policy_version 637652 (0.0007) [2023-12-26 20:00:13,606][105692] Updated weights for policy 0, policy_version 636950 (0.0010) [2023-12-26 20:00:13,646][105620] Updated weights for policy 1, policy_version 637662 (0.0010) [2023-12-26 20:00:13,661][105692] Updated weights for policy 0, policy_version 636960 (0.0010) [2023-12-26 20:00:13,694][105620] Updated weights for policy 1, policy_version 637672 (0.0010) [2023-12-26 20:00:14,346][105692] Updated weights for policy 0, policy_version 636970 (0.0010) [2023-12-26 20:00:14,407][105692] Updated weights for policy 0, policy_version 636980 (0.0009) [2023-12-26 20:00:14,419][105620] Updated weights for policy 1, policy_version 637682 (0.0010) [2023-12-26 20:00:14,461][105692] Updated weights for policy 0, policy_version 636990 (0.0010) [2023-12-26 20:00:14,476][105620] Updated weights for policy 1, policy_version 637692 (0.0010) [2023-12-26 20:00:14,519][105692] Updated weights for policy 0, policy_version 637000 (0.0009) [2023-12-26 20:00:14,531][105620] Updated weights for policy 1, policy_version 637702 (0.0010) [2023-12-26 20:00:15,145][105692] Updated weights for policy 0, policy_version 637010 (0.0010) [2023-12-26 20:00:15,207][105692] Updated weights for policy 0, policy_version 637020 (0.0010) [2023-12-26 20:00:15,273][105692] Updated weights for policy 0, policy_version 637030 (0.0010) [2023-12-26 20:00:15,322][105620] Updated weights for policy 1, policy_version 637712 (0.0008) [2023-12-26 20:00:15,379][105620] Updated weights for policy 1, policy_version 637722 (0.0006) [2023-12-26 20:00:15,443][105620] Updated weights for policy 1, policy_version 637732 (0.0008) [2023-12-26 20:00:15,947][105692] Updated weights for policy 0, policy_version 637040 (0.0006) [2023-12-26 20:00:16,010][105692] Updated weights for policy 0, policy_version 637050 (0.0005) [2023-12-26 20:00:16,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 326385664. Throughput: 0: 9816.9, 1: 9470.3. Samples: 326361848. Policy #0 lag: (min: 1.0, avg: 22.7, max: 33.0) [2023-12-26 20:00:16,062][104569] Avg episode reward: [(0, '7468.317'), (1, '6514.063')] [2023-12-26 20:00:16,081][105692] Updated weights for policy 0, policy_version 637060 (0.0005) [2023-12-26 20:00:16,094][105620] Updated weights for policy 1, policy_version 637742 (0.0007) [2023-12-26 20:00:16,111][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000637064_163119104.pth... [2023-12-26 20:00:16,116][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000635912_162824192.pth [2023-12-26 20:00:16,145][105620] Updated weights for policy 1, policy_version 637752 (0.0006) [2023-12-26 20:00:16,209][105620] Updated weights for policy 1, policy_version 637762 (0.0007) [2023-12-26 20:00:16,239][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000637768_163282944.pth... [2023-12-26 20:00:16,244][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000636648_162996224.pth [2023-12-26 20:00:16,685][105692] Updated weights for policy 0, policy_version 637070 (0.0006) [2023-12-26 20:00:16,741][105692] Updated weights for policy 0, policy_version 637080 (0.0008) [2023-12-26 20:00:16,804][105692] Updated weights for policy 0, policy_version 637090 (0.0007) [2023-12-26 20:00:16,859][105620] Updated weights for policy 1, policy_version 637772 (0.0005) [2023-12-26 20:00:16,914][105620] Updated weights for policy 1, policy_version 637782 (0.0005) [2023-12-26 20:00:16,969][105620] Updated weights for policy 1, policy_version 637792 (0.0005) [2023-12-26 20:00:17,450][105692] Updated weights for policy 0, policy_version 637100 (0.0007) [2023-12-26 20:00:17,498][105692] Updated weights for policy 0, policy_version 637110 (0.0008) [2023-12-26 20:00:17,548][105692] Updated weights for policy 0, policy_version 637120 (0.0008) [2023-12-26 20:00:17,684][105620] Updated weights for policy 1, policy_version 637802 (0.0008) [2023-12-26 20:00:17,748][105620] Updated weights for policy 1, policy_version 637812 (0.0005) [2023-12-26 20:00:17,795][105620] Updated weights for policy 1, policy_version 637822 (0.0005) [2023-12-26 20:00:17,843][105620] Updated weights for policy 1, policy_version 637832 (0.0008) [2023-12-26 20:00:18,324][105692] Updated weights for policy 0, policy_version 637130 (0.0008) [2023-12-26 20:00:18,394][105692] Updated weights for policy 0, policy_version 637140 (0.0009) [2023-12-26 20:00:18,454][105692] Updated weights for policy 0, policy_version 637150 (0.0007) [2023-12-26 20:00:18,516][105620] Updated weights for policy 1, policy_version 637842 (0.0008) [2023-12-26 20:00:18,523][105692] Updated weights for policy 0, policy_version 637160 (0.0006) [2023-12-26 20:00:18,572][105620] Updated weights for policy 1, policy_version 637852 (0.0008) [2023-12-26 20:00:18,627][105620] Updated weights for policy 1, policy_version 637862 (0.0008) [2023-12-26 20:00:19,106][105692] Updated weights for policy 0, policy_version 637170 (0.0005) [2023-12-26 20:00:19,158][105692] Updated weights for policy 0, policy_version 637180 (0.0005) [2023-12-26 20:00:19,222][105692] Updated weights for policy 0, policy_version 637190 (0.0006) [2023-12-26 20:00:19,279][105620] Updated weights for policy 1, policy_version 637872 (0.0008) [2023-12-26 20:00:19,347][105620] Updated weights for policy 1, policy_version 637882 (0.0007) [2023-12-26 20:00:19,408][105620] Updated weights for policy 1, policy_version 637892 (0.0006) [2023-12-26 20:00:19,959][105692] Updated weights for policy 0, policy_version 637200 (0.0008) [2023-12-26 20:00:20,019][105692] Updated weights for policy 0, policy_version 637210 (0.0009) [2023-12-26 20:00:20,040][105620] Updated weights for policy 1, policy_version 637902 (0.0009) [2023-12-26 20:00:20,086][105692] Updated weights for policy 0, policy_version 637220 (0.0006) [2023-12-26 20:00:20,100][105620] Updated weights for policy 1, policy_version 637912 (0.0011) [2023-12-26 20:00:20,147][105620] Updated weights for policy 1, policy_version 637922 (0.0009) [2023-12-26 20:00:20,803][105620] Updated weights for policy 1, policy_version 637932 (0.0008) [2023-12-26 20:00:20,867][105620] Updated weights for policy 1, policy_version 637942 (0.0011) [2023-12-26 20:00:20,917][105692] Updated weights for policy 0, policy_version 637230 (0.0008) [2023-12-26 20:00:20,927][105620] Updated weights for policy 1, policy_version 637952 (0.0011) [2023-12-26 20:00:20,967][105692] Updated weights for policy 0, policy_version 637240 (0.0006) [2023-12-26 20:00:21,030][105692] Updated weights for policy 0, policy_version 637250 (0.0008) [2023-12-26 20:00:21,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 326492160. Throughput: 0: 9869.9, 1: 9519.5. Samples: 326485060. Policy #0 lag: (min: 1.0, avg: 22.7, max: 33.0) [2023-12-26 20:00:21,063][104569] Avg episode reward: [(0, '6849.134'), (1, '6952.812')] [2023-12-26 20:00:21,696][105620] Updated weights for policy 1, policy_version 637962 (0.0011) [2023-12-26 20:00:21,758][105692] Updated weights for policy 0, policy_version 637260 (0.0008) [2023-12-26 20:00:21,761][105620] Updated weights for policy 1, policy_version 637972 (0.0009) [2023-12-26 20:00:21,821][105620] Updated weights for policy 1, policy_version 637982 (0.0010) [2023-12-26 20:00:21,827][105692] Updated weights for policy 0, policy_version 637270 (0.0008) [2023-12-26 20:00:21,882][105620] Updated weights for policy 1, policy_version 637992 (0.0009) [2023-12-26 20:00:21,895][105692] Updated weights for policy 0, policy_version 637280 (0.0006) [2023-12-26 20:00:22,545][105692] Updated weights for policy 0, policy_version 637290 (0.0006) [2023-12-26 20:00:22,619][105692] Updated weights for policy 0, policy_version 637300 (0.0007) [2023-12-26 20:00:22,648][105620] Updated weights for policy 1, policy_version 638002 (0.0011) [2023-12-26 20:00:22,679][105692] Updated weights for policy 0, policy_version 637310 (0.0008) [2023-12-26 20:00:22,715][105620] Updated weights for policy 1, policy_version 638012 (0.0010) [2023-12-26 20:00:22,742][105692] Updated weights for policy 0, policy_version 637320 (0.0006) [2023-12-26 20:00:22,785][105620] Updated weights for policy 1, policy_version 638022 (0.0010) [2023-12-26 20:00:23,411][105620] Updated weights for policy 1, policy_version 638032 (0.0006) [2023-12-26 20:00:23,459][105620] Updated weights for policy 1, policy_version 638042 (0.0008) [2023-12-26 20:00:23,477][105692] Updated weights for policy 0, policy_version 637330 (0.0011) [2023-12-26 20:00:23,505][105620] Updated weights for policy 1, policy_version 638052 (0.0010) [2023-12-26 20:00:23,537][105692] Updated weights for policy 0, policy_version 637340 (0.0010) [2023-12-26 20:00:23,599][105692] Updated weights for policy 0, policy_version 637350 (0.0010) [2023-12-26 20:00:24,134][105620] Updated weights for policy 1, policy_version 638062 (0.0008) [2023-12-26 20:00:24,196][105620] Updated weights for policy 1, policy_version 638072 (0.0005) [2023-12-26 20:00:24,261][105620] Updated weights for policy 1, policy_version 638082 (0.0005) [2023-12-26 20:00:24,318][105692] Updated weights for policy 0, policy_version 637360 (0.0007) [2023-12-26 20:00:24,374][105692] Updated weights for policy 0, policy_version 637370 (0.0007) [2023-12-26 20:00:24,429][105692] Updated weights for policy 0, policy_version 637380 (0.0006) [2023-12-26 20:00:24,763][105620] Updated weights for policy 1, policy_version 638092 (0.0005) [2023-12-26 20:00:24,826][105620] Updated weights for policy 1, policy_version 638102 (0.0005) [2023-12-26 20:00:24,893][105620] Updated weights for policy 1, policy_version 638112 (0.0005) [2023-12-26 20:00:25,042][105692] Updated weights for policy 0, policy_version 637390 (0.0008) [2023-12-26 20:00:25,095][105692] Updated weights for policy 0, policy_version 637400 (0.0009) [2023-12-26 20:00:25,168][105692] Updated weights for policy 0, policy_version 637410 (0.0005) [2023-12-26 20:00:25,404][105620] Updated weights for policy 1, policy_version 638122 (0.0005) [2023-12-26 20:00:25,468][105620] Updated weights for policy 1, policy_version 638132 (0.0005) [2023-12-26 20:00:25,521][105620] Updated weights for policy 1, policy_version 638142 (0.0006) [2023-12-26 20:00:25,568][105620] Updated weights for policy 1, policy_version 638152 (0.0006) [2023-12-26 20:00:25,802][105692] Updated weights for policy 0, policy_version 637420 (0.0006) [2023-12-26 20:00:25,872][105692] Updated weights for policy 0, policy_version 637430 (0.0006) [2023-12-26 20:00:25,940][105692] Updated weights for policy 0, policy_version 637440 (0.0006) [2023-12-26 20:00:26,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19660.9, 300 sec: 19633.0). Total num frames: 326598656. Throughput: 0: 9881.3, 1: 9637.5. Samples: 326606720. Policy #0 lag: (min: 1.0, avg: 22.7, max: 33.0) [2023-12-26 20:00:26,062][104569] Avg episode reward: [(0, '8190.000'), (1, '8519.023')] [2023-12-26 20:00:26,217][105620] Updated weights for policy 1, policy_version 638162 (0.0005) [2023-12-26 20:00:26,288][105620] Updated weights for policy 1, policy_version 638172 (0.0005) [2023-12-26 20:00:26,350][105620] Updated weights for policy 1, policy_version 638182 (0.0006) [2023-12-26 20:00:26,529][105692] Updated weights for policy 0, policy_version 637450 (0.0007) [2023-12-26 20:00:26,588][105692] Updated weights for policy 0, policy_version 637460 (0.0009) [2023-12-26 20:00:26,650][105692] Updated weights for policy 0, policy_version 637470 (0.0011) [2023-12-26 20:00:26,704][105692] Updated weights for policy 0, policy_version 637480 (0.0008) [2023-12-26 20:00:27,022][105620] Updated weights for policy 1, policy_version 638192 (0.0008) [2023-12-26 20:00:27,090][105620] Updated weights for policy 1, policy_version 638202 (0.0008) [2023-12-26 20:00:27,153][105620] Updated weights for policy 1, policy_version 638212 (0.0008) [2023-12-26 20:00:27,329][105692] Updated weights for policy 0, policy_version 637490 (0.0005) [2023-12-26 20:00:27,389][105692] Updated weights for policy 0, policy_version 637500 (0.0007) [2023-12-26 20:00:27,442][105692] Updated weights for policy 0, policy_version 637510 (0.0005) [2023-12-26 20:00:27,960][105620] Updated weights for policy 1, policy_version 638222 (0.0008) [2023-12-26 20:00:28,018][105692] Updated weights for policy 0, policy_version 637520 (0.0007) [2023-12-26 20:00:28,020][105620] Updated weights for policy 1, policy_version 638232 (0.0006) [2023-12-26 20:00:28,076][105692] Updated weights for policy 0, policy_version 637530 (0.0007) [2023-12-26 20:00:28,082][105620] Updated weights for policy 1, policy_version 638242 (0.0008) [2023-12-26 20:00:28,129][105692] Updated weights for policy 0, policy_version 637540 (0.0006) [2023-12-26 20:00:28,795][105692] Updated weights for policy 0, policy_version 637550 (0.0008) [2023-12-26 20:00:28,822][105620] Updated weights for policy 1, policy_version 638252 (0.0008) [2023-12-26 20:00:28,849][105692] Updated weights for policy 0, policy_version 637560 (0.0006) [2023-12-26 20:00:28,880][105620] Updated weights for policy 1, policy_version 638262 (0.0007) [2023-12-26 20:00:28,906][105692] Updated weights for policy 0, policy_version 637570 (0.0007) [2023-12-26 20:00:28,943][105620] Updated weights for policy 1, policy_version 638272 (0.0007) [2023-12-26 20:00:29,660][105620] Updated weights for policy 1, policy_version 638282 (0.0008) [2023-12-26 20:00:29,665][105692] Updated weights for policy 0, policy_version 637580 (0.0007) [2023-12-26 20:00:29,716][105620] Updated weights for policy 1, policy_version 638292 (0.0006) [2023-12-26 20:00:29,722][105692] Updated weights for policy 0, policy_version 637590 (0.0007) [2023-12-26 20:00:29,772][105620] Updated weights for policy 1, policy_version 638302 (0.0007) [2023-12-26 20:00:29,782][105692] Updated weights for policy 0, policy_version 637600 (0.0007) [2023-12-26 20:00:29,824][105620] Updated weights for policy 1, policy_version 638312 (0.0006) [2023-12-26 20:00:30,425][105692] Updated weights for policy 0, policy_version 637610 (0.0008) [2023-12-26 20:00:30,480][105692] Updated weights for policy 0, policy_version 637620 (0.0010) [2023-12-26 20:00:30,528][105692] Updated weights for policy 0, policy_version 637630 (0.0007) [2023-12-26 20:00:30,586][105692] Updated weights for policy 0, policy_version 637640 (0.0006) [2023-12-26 20:00:30,611][105620] Updated weights for policy 1, policy_version 638322 (0.0005) [2023-12-26 20:00:30,668][105620] Updated weights for policy 1, policy_version 638332 (0.0005) [2023-12-26 20:00:30,721][105620] Updated weights for policy 1, policy_version 638342 (0.0005) [2023-12-26 20:00:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 326696960. Throughput: 0: 9946.4, 1: 9638.6. Samples: 326667352. Policy #0 lag: (min: 1.0, avg: 22.7, max: 33.0) [2023-12-26 20:00:31,062][104569] Avg episode reward: [(0, '5169.289'), (1, '9080.370')] [2023-12-26 20:00:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000637640_163266560.pth... [2023-12-26 20:00:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000638344_163430400.pth... [2023-12-26 20:00:31,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000636488_162971648.pth [2023-12-26 20:00:31,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000637192_163135488.pth [2023-12-26 20:00:31,256][105692] Updated weights for policy 0, policy_version 637650 (0.0009) [2023-12-26 20:00:31,299][105620] Updated weights for policy 1, policy_version 638352 (0.0005) [2023-12-26 20:00:31,319][105692] Updated weights for policy 0, policy_version 637660 (0.0009) [2023-12-26 20:00:31,361][105620] Updated weights for policy 1, policy_version 638362 (0.0007) [2023-12-26 20:00:31,386][105692] Updated weights for policy 0, policy_version 637670 (0.0008) [2023-12-26 20:00:31,423][105620] Updated weights for policy 1, policy_version 638372 (0.0006) [2023-12-26 20:00:32,058][105620] Updated weights for policy 1, policy_version 638382 (0.0009) [2023-12-26 20:00:32,114][105620] Updated weights for policy 1, policy_version 638392 (0.0010) [2023-12-26 20:00:32,163][105692] Updated weights for policy 0, policy_version 637680 (0.0007) [2023-12-26 20:00:32,168][105620] Updated weights for policy 1, policy_version 638402 (0.0009) [2023-12-26 20:00:32,215][105692] Updated weights for policy 0, policy_version 637690 (0.0006) [2023-12-26 20:00:32,267][105692] Updated weights for policy 0, policy_version 637700 (0.0008) [2023-12-26 20:00:32,885][105620] Updated weights for policy 1, policy_version 638412 (0.0010) [2023-12-26 20:00:32,945][105620] Updated weights for policy 1, policy_version 638422 (0.0009) [2023-12-26 20:00:33,003][105620] Updated weights for policy 1, policy_version 638432 (0.0008) [2023-12-26 20:00:33,054][105692] Updated weights for policy 0, policy_version 637710 (0.0010) [2023-12-26 20:00:33,115][105692] Updated weights for policy 0, policy_version 637720 (0.0010) [2023-12-26 20:00:33,182][105692] Updated weights for policy 0, policy_version 637730 (0.0008) [2023-12-26 20:00:33,674][105620] Updated weights for policy 1, policy_version 638442 (0.0008) [2023-12-26 20:00:33,725][105692] Updated weights for policy 0, policy_version 637740 (0.0005) [2023-12-26 20:00:33,736][105620] Updated weights for policy 1, policy_version 638452 (0.0008) [2023-12-26 20:00:33,788][105692] Updated weights for policy 0, policy_version 637750 (0.0006) [2023-12-26 20:00:33,797][105620] Updated weights for policy 1, policy_version 638462 (0.0007) [2023-12-26 20:00:33,847][105692] Updated weights for policy 0, policy_version 637760 (0.0010) [2023-12-26 20:00:33,853][105620] Updated weights for policy 1, policy_version 638472 (0.0005) [2023-12-26 20:00:34,464][105620] Updated weights for policy 1, policy_version 638482 (0.0006) [2023-12-26 20:00:34,523][105620] Updated weights for policy 1, policy_version 638492 (0.0006) [2023-12-26 20:00:34,554][105692] Updated weights for policy 0, policy_version 637770 (0.0010) [2023-12-26 20:00:34,579][105620] Updated weights for policy 1, policy_version 638502 (0.0006) [2023-12-26 20:00:34,624][105692] Updated weights for policy 0, policy_version 637780 (0.0010) [2023-12-26 20:00:34,683][105692] Updated weights for policy 0, policy_version 637790 (0.0010) [2023-12-26 20:00:34,734][105692] Updated weights for policy 0, policy_version 637800 (0.0010) [2023-12-26 20:00:35,267][105620] Updated weights for policy 1, policy_version 638512 (0.0009) [2023-12-26 20:00:35,318][105620] Updated weights for policy 1, policy_version 638522 (0.0010) [2023-12-26 20:00:35,358][105692] Updated weights for policy 0, policy_version 637810 (0.0007) [2023-12-26 20:00:35,374][105620] Updated weights for policy 1, policy_version 638532 (0.0008) [2023-12-26 20:00:35,407][105692] Updated weights for policy 0, policy_version 637820 (0.0008) [2023-12-26 20:00:35,458][105692] Updated weights for policy 0, policy_version 637830 (0.0008) [2023-12-26 20:00:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 326795264. Throughput: 0: 9966.9, 1: 9743.6. Samples: 326788848. Policy #0 lag: (min: 1.0, avg: 22.7, max: 33.0) [2023-12-26 20:00:36,062][104569] Avg episode reward: [(0, '5042.873'), (1, '9265.119')] [2023-12-26 20:00:36,108][105620] Updated weights for policy 1, policy_version 638542 (0.0009) [2023-12-26 20:00:36,177][105620] Updated weights for policy 1, policy_version 638552 (0.0009) [2023-12-26 20:00:36,221][105692] Updated weights for policy 0, policy_version 637840 (0.0009) [2023-12-26 20:00:36,248][105620] Updated weights for policy 1, policy_version 638562 (0.0009) [2023-12-26 20:00:36,280][105692] Updated weights for policy 0, policy_version 637850 (0.0007) [2023-12-26 20:00:36,334][105692] Updated weights for policy 0, policy_version 637860 (0.0008) [2023-12-26 20:00:36,971][105620] Updated weights for policy 1, policy_version 638572 (0.0009) [2023-12-26 20:00:37,025][105620] Updated weights for policy 1, policy_version 638582 (0.0010) [2023-12-26 20:00:37,047][105692] Updated weights for policy 0, policy_version 637870 (0.0008) [2023-12-26 20:00:37,076][105620] Updated weights for policy 1, policy_version 638592 (0.0010) [2023-12-26 20:00:37,097][105692] Updated weights for policy 0, policy_version 637880 (0.0009) [2023-12-26 20:00:37,152][105692] Updated weights for policy 0, policy_version 637890 (0.0009) [2023-12-26 20:00:37,635][105620] Updated weights for policy 1, policy_version 638602 (0.0007) [2023-12-26 20:00:37,681][105620] Updated weights for policy 1, policy_version 638612 (0.0005) [2023-12-26 20:00:37,744][105620] Updated weights for policy 1, policy_version 638622 (0.0007) [2023-12-26 20:00:37,810][105620] Updated weights for policy 1, policy_version 638632 (0.0008) [2023-12-26 20:00:38,053][105692] Updated weights for policy 0, policy_version 637900 (0.0008) [2023-12-26 20:00:38,123][105692] Updated weights for policy 0, policy_version 637910 (0.0005) [2023-12-26 20:00:38,185][105692] Updated weights for policy 0, policy_version 637920 (0.0010) [2023-12-26 20:00:38,354][105620] Updated weights for policy 1, policy_version 638642 (0.0008) [2023-12-26 20:00:38,424][105620] Updated weights for policy 1, policy_version 638652 (0.0009) [2023-12-26 20:00:38,495][105620] Updated weights for policy 1, policy_version 638662 (0.0005) [2023-12-26 20:00:38,879][105692] Updated weights for policy 0, policy_version 637930 (0.0009) [2023-12-26 20:00:38,926][105692] Updated weights for policy 0, policy_version 637940 (0.0005) [2023-12-26 20:00:38,976][105692] Updated weights for policy 0, policy_version 637950 (0.0005) [2023-12-26 20:00:39,027][105692] Updated weights for policy 0, policy_version 637960 (0.0009) [2023-12-26 20:00:39,152][105620] Updated weights for policy 1, policy_version 638672 (0.0009) [2023-12-26 20:00:39,216][105620] Updated weights for policy 1, policy_version 638682 (0.0010) [2023-12-26 20:00:39,278][105620] Updated weights for policy 1, policy_version 638692 (0.0010) [2023-12-26 20:00:39,762][105692] Updated weights for policy 0, policy_version 637970 (0.0007) [2023-12-26 20:00:39,831][105692] Updated weights for policy 0, policy_version 637980 (0.0008) [2023-12-26 20:00:39,898][105692] Updated weights for policy 0, policy_version 637990 (0.0010) [2023-12-26 20:00:40,038][105620] Updated weights for policy 1, policy_version 638702 (0.0010) [2023-12-26 20:00:40,102][105620] Updated weights for policy 1, policy_version 638712 (0.0011) [2023-12-26 20:00:40,166][105620] Updated weights for policy 1, policy_version 638722 (0.0011) [2023-12-26 20:00:40,615][105692] Updated weights for policy 0, policy_version 638000 (0.0008) [2023-12-26 20:00:40,674][105692] Updated weights for policy 0, policy_version 638010 (0.0008) [2023-12-26 20:00:40,731][105692] Updated weights for policy 0, policy_version 638020 (0.0008) [2023-12-26 20:00:40,752][105620] Updated weights for policy 1, policy_version 638732 (0.0008) [2023-12-26 20:00:40,810][105620] Updated weights for policy 1, policy_version 638742 (0.0010) [2023-12-26 20:00:40,869][105620] Updated weights for policy 1, policy_version 638752 (0.0011) [2023-12-26 20:00:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 326901760. Throughput: 0: 9971.9, 1: 9866.0. Samples: 326907396. Policy #0 lag: (min: 1.0, avg: 22.7, max: 33.0) [2023-12-26 20:00:41,062][104569] Avg episode reward: [(0, '7487.402'), (1, '9355.854')] [2023-12-26 20:00:41,475][105692] Updated weights for policy 0, policy_version 638030 (0.0007) [2023-12-26 20:00:41,542][105692] Updated weights for policy 0, policy_version 638040 (0.0008) [2023-12-26 20:00:41,603][105692] Updated weights for policy 0, policy_version 638050 (0.0008) [2023-12-26 20:00:41,659][105620] Updated weights for policy 1, policy_version 638762 (0.0012) [2023-12-26 20:00:41,725][105620] Updated weights for policy 1, policy_version 638772 (0.0009) [2023-12-26 20:00:41,792][105620] Updated weights for policy 1, policy_version 638782 (0.0006) [2023-12-26 20:00:41,857][105620] Updated weights for policy 1, policy_version 638792 (0.0007) [2023-12-26 20:00:42,401][105692] Updated weights for policy 0, policy_version 638060 (0.0009) [2023-12-26 20:00:42,465][105692] Updated weights for policy 0, policy_version 638070 (0.0009) [2023-12-26 20:00:42,526][105692] Updated weights for policy 0, policy_version 638080 (0.0009) [2023-12-26 20:00:42,547][105620] Updated weights for policy 1, policy_version 638802 (0.0007) [2023-12-26 20:00:42,610][105620] Updated weights for policy 1, policy_version 638812 (0.0009) [2023-12-26 20:00:42,672][105620] Updated weights for policy 1, policy_version 638822 (0.0008) [2023-12-26 20:00:43,164][105692] Updated weights for policy 0, policy_version 638090 (0.0006) [2023-12-26 20:00:43,225][105692] Updated weights for policy 0, policy_version 638100 (0.0006) [2023-12-26 20:00:43,282][105692] Updated weights for policy 0, policy_version 638110 (0.0006) [2023-12-26 20:00:43,346][105692] Updated weights for policy 0, policy_version 638120 (0.0010) [2023-12-26 20:00:43,496][105620] Updated weights for policy 1, policy_version 638832 (0.0008) [2023-12-26 20:00:43,551][105620] Updated weights for policy 1, policy_version 638842 (0.0009) [2023-12-26 20:00:43,602][105620] Updated weights for policy 1, policy_version 638852 (0.0009) [2023-12-26 20:00:43,983][105692] Updated weights for policy 0, policy_version 638130 (0.0006) [2023-12-26 20:00:44,042][105692] Updated weights for policy 0, policy_version 638140 (0.0011) [2023-12-26 20:00:44,097][105692] Updated weights for policy 0, policy_version 638150 (0.0010) [2023-12-26 20:00:44,383][105620] Updated weights for policy 1, policy_version 638862 (0.0007) [2023-12-26 20:00:44,437][105620] Updated weights for policy 1, policy_version 638872 (0.0005) [2023-12-26 20:00:44,502][105620] Updated weights for policy 1, policy_version 638882 (0.0007) [2023-12-26 20:00:44,810][105692] Updated weights for policy 0, policy_version 638160 (0.0010) [2023-12-26 20:00:44,867][105692] Updated weights for policy 0, policy_version 638170 (0.0010) [2023-12-26 20:00:44,927][105692] Updated weights for policy 0, policy_version 638180 (0.0009) [2023-12-26 20:00:45,221][105620] Updated weights for policy 1, policy_version 638892 (0.0008) [2023-12-26 20:00:45,285][105620] Updated weights for policy 1, policy_version 638902 (0.0008) [2023-12-26 20:00:45,346][105620] Updated weights for policy 1, policy_version 638912 (0.0008) [2023-12-26 20:00:45,669][105692] Updated weights for policy 0, policy_version 638190 (0.0008) [2023-12-26 20:00:45,730][105692] Updated weights for policy 0, policy_version 638200 (0.0006) [2023-12-26 20:00:45,786][105692] Updated weights for policy 0, policy_version 638210 (0.0005) [2023-12-26 20:00:46,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19660.8, 300 sec: 19605.2). Total num frames: 326991872. Throughput: 0: 9898.8, 1: 9831.2. Samples: 326962108. Policy #0 lag: (min: 1.0, avg: 22.7, max: 33.0) [2023-12-26 20:00:46,063][104569] Avg episode reward: [(0, '6595.663'), (1, '9355.534')] [2023-12-26 20:00:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000638920_163577856.pth... [2023-12-26 20:00:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000638216_163414016.pth... [2023-12-26 20:00:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000637768_163282944.pth [2023-12-26 20:00:46,080][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000637064_163119104.pth [2023-12-26 20:00:46,136][105620] Updated weights for policy 1, policy_version 638922 (0.0008) [2023-12-26 20:00:46,201][105620] Updated weights for policy 1, policy_version 638932 (0.0009) [2023-12-26 20:00:46,266][105620] Updated weights for policy 1, policy_version 638942 (0.0008) [2023-12-26 20:00:46,314][105620] Updated weights for policy 1, policy_version 638952 (0.0009) [2023-12-26 20:00:46,471][105692] Updated weights for policy 0, policy_version 638220 (0.0007) [2023-12-26 20:00:46,519][105692] Updated weights for policy 0, policy_version 638230 (0.0006) [2023-12-26 20:00:46,576][105692] Updated weights for policy 0, policy_version 638240 (0.0006) [2023-12-26 20:00:47,102][105620] Updated weights for policy 1, policy_version 638962 (0.0008) [2023-12-26 20:00:47,164][105620] Updated weights for policy 1, policy_version 638972 (0.0009) [2023-12-26 20:00:47,223][105620] Updated weights for policy 1, policy_version 638982 (0.0009) [2023-12-26 20:00:47,305][105692] Updated weights for policy 0, policy_version 638250 (0.0008) [2023-12-26 20:00:47,356][105692] Updated weights for policy 0, policy_version 638260 (0.0009) [2023-12-26 20:00:47,403][105692] Updated weights for policy 0, policy_version 638270 (0.0008) [2023-12-26 20:00:47,457][105692] Updated weights for policy 0, policy_version 638280 (0.0009) [2023-12-26 20:00:47,998][105620] Updated weights for policy 1, policy_version 638992 (0.0009) [2023-12-26 20:00:48,049][105620] Updated weights for policy 1, policy_version 639002 (0.0009) [2023-12-26 20:00:48,106][105620] Updated weights for policy 1, policy_version 639012 (0.0009) [2023-12-26 20:00:48,205][105692] Updated weights for policy 0, policy_version 638290 (0.0009) [2023-12-26 20:00:48,261][105692] Updated weights for policy 0, policy_version 638300 (0.0009) [2023-12-26 20:00:48,320][105692] Updated weights for policy 0, policy_version 638310 (0.0009) [2023-12-26 20:00:48,947][105620] Updated weights for policy 1, policy_version 639022 (0.0009) [2023-12-26 20:00:49,002][105620] Updated weights for policy 1, policy_version 639032 (0.0009) [2023-12-26 20:00:49,036][105692] Updated weights for policy 0, policy_version 638320 (0.0006) [2023-12-26 20:00:49,061][105620] Updated weights for policy 1, policy_version 639042 (0.0009) [2023-12-26 20:00:49,091][105692] Updated weights for policy 0, policy_version 638330 (0.0005) [2023-12-26 20:00:49,142][105692] Updated weights for policy 0, policy_version 638340 (0.0009) [2023-12-26 20:00:49,873][105620] Updated weights for policy 1, policy_version 639052 (0.0008) [2023-12-26 20:00:49,883][105692] Updated weights for policy 0, policy_version 638350 (0.0009) [2023-12-26 20:00:49,932][105620] Updated weights for policy 1, policy_version 639062 (0.0009) [2023-12-26 20:00:49,941][105692] Updated weights for policy 0, policy_version 638360 (0.0009) [2023-12-26 20:00:49,994][105692] Updated weights for policy 0, policy_version 638370 (0.0009) [2023-12-26 20:00:49,997][105620] Updated weights for policy 1, policy_version 639072 (0.0009) [2023-12-26 20:00:50,763][105692] Updated weights for policy 0, policy_version 638380 (0.0009) [2023-12-26 20:00:50,834][105692] Updated weights for policy 0, policy_version 638390 (0.0008) [2023-12-26 20:00:50,837][105620] Updated weights for policy 1, policy_version 639082 (0.0008) [2023-12-26 20:00:50,897][105692] Updated weights for policy 0, policy_version 638400 (0.0008) [2023-12-26 20:00:50,899][105620] Updated weights for policy 1, policy_version 639092 (0.0006) [2023-12-26 20:00:50,958][105620] Updated weights for policy 1, policy_version 639102 (0.0007) [2023-12-26 20:00:51,018][105620] Updated weights for policy 1, policy_version 639112 (0.0009) [2023-12-26 20:00:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 327090176. Throughput: 0: 9926.9, 1: 9740.8. Samples: 327074948. Policy #0 lag: (min: 1.0, avg: 22.7, max: 33.0) [2023-12-26 20:00:51,063][104569] Avg episode reward: [(0, '7007.304'), (1, '9098.918')] [2023-12-26 20:00:51,591][105692] Updated weights for policy 0, policy_version 638410 (0.0010) [2023-12-26 20:00:51,662][105692] Updated weights for policy 0, policy_version 638420 (0.0010) [2023-12-26 20:00:51,729][105585] KL-divergence is very high: 104.8792 [2023-12-26 20:00:51,729][105692] Updated weights for policy 0, policy_version 638430 (0.0011) [2023-12-26 20:00:51,750][105585] KL-divergence is very high: 120.7985 [2023-12-26 20:00:51,761][105585] KL-divergence is very high: 111.9397 [2023-12-26 20:00:51,771][105585] KL-divergence is very high: 124.1131 [2023-12-26 20:00:51,782][105692] Updated weights for policy 0, policy_version 638440 (0.0010) [2023-12-26 20:00:51,913][105620] Updated weights for policy 1, policy_version 639122 (0.0008) [2023-12-26 20:00:51,974][105620] Updated weights for policy 1, policy_version 639132 (0.0008) [2023-12-26 20:00:52,041][105620] Updated weights for policy 1, policy_version 639142 (0.0009) [2023-12-26 20:00:52,530][105692] Updated weights for policy 0, policy_version 638450 (0.0011) [2023-12-26 20:00:52,587][105692] Updated weights for policy 0, policy_version 638460 (0.0010) [2023-12-26 20:00:52,643][105692] Updated weights for policy 0, policy_version 638470 (0.0010) [2023-12-26 20:00:52,798][105620] Updated weights for policy 1, policy_version 639152 (0.0009) [2023-12-26 20:00:52,856][105620] Updated weights for policy 1, policy_version 639162 (0.0008) [2023-12-26 20:00:52,919][105620] Updated weights for policy 1, policy_version 639172 (0.0009) [2023-12-26 20:00:53,323][105692] Updated weights for policy 0, policy_version 638480 (0.0011) [2023-12-26 20:00:53,378][105692] Updated weights for policy 0, policy_version 638490 (0.0010) [2023-12-26 20:00:53,430][105692] Updated weights for policy 0, policy_version 638500 (0.0010) [2023-12-26 20:00:53,743][105620] Updated weights for policy 1, policy_version 639182 (0.0010) [2023-12-26 20:00:53,805][105620] Updated weights for policy 1, policy_version 639192 (0.0008) [2023-12-26 20:00:53,865][105620] Updated weights for policy 1, policy_version 639202 (0.0008) [2023-12-26 20:00:54,191][105692] Updated weights for policy 0, policy_version 638510 (0.0009) [2023-12-26 20:00:54,253][105692] Updated weights for policy 0, policy_version 638520 (0.0010) [2023-12-26 20:00:54,313][105692] Updated weights for policy 0, policy_version 638530 (0.0010) [2023-12-26 20:00:54,667][105620] Updated weights for policy 1, policy_version 639212 (0.0008) [2023-12-26 20:00:54,719][105620] Updated weights for policy 1, policy_version 639222 (0.0008) [2023-12-26 20:00:54,772][105620] Updated weights for policy 1, policy_version 639232 (0.0008) [2023-12-26 20:00:55,004][105692] Updated weights for policy 0, policy_version 638540 (0.0010) [2023-12-26 20:00:55,066][105692] Updated weights for policy 0, policy_version 638550 (0.0010) [2023-12-26 20:00:55,118][105692] Updated weights for policy 0, policy_version 638560 (0.0010) [2023-12-26 20:00:55,590][105620] Updated weights for policy 1, policy_version 639242 (0.0008) [2023-12-26 20:00:55,640][105620] Updated weights for policy 1, policy_version 639252 (0.0009) [2023-12-26 20:00:55,689][105620] Updated weights for policy 1, policy_version 639262 (0.0008) [2023-12-26 20:00:55,753][105620] Updated weights for policy 1, policy_version 639272 (0.0009) [2023-12-26 20:00:55,850][105692] Updated weights for policy 0, policy_version 638570 (0.0010) [2023-12-26 20:00:55,904][105692] Updated weights for policy 0, policy_version 638580 (0.0008) [2023-12-26 20:00:55,955][105692] Updated weights for policy 0, policy_version 638590 (0.0009) [2023-12-26 20:00:56,018][105692] Updated weights for policy 0, policy_version 638600 (0.0009) [2023-12-26 20:00:56,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 327180288. Throughput: 0: 9875.2, 1: 9674.0. Samples: 327183980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:00:56,063][104569] Avg episode reward: [(0, '8024.285'), (1, '9012.838')] [2023-12-26 20:00:56,527][105620] Updated weights for policy 1, policy_version 639282 (0.0009) [2023-12-26 20:00:56,585][105620] Updated weights for policy 1, policy_version 639292 (0.0009) [2023-12-26 20:00:56,645][105620] Updated weights for policy 1, policy_version 639302 (0.0009) [2023-12-26 20:00:56,766][105692] Updated weights for policy 0, policy_version 638610 (0.0007) [2023-12-26 20:00:56,825][105692] Updated weights for policy 0, policy_version 638620 (0.0005) [2023-12-26 20:00:56,891][105692] Updated weights for policy 0, policy_version 638630 (0.0005) [2023-12-26 20:00:57,351][105620] Updated weights for policy 1, policy_version 639312 (0.0010) [2023-12-26 20:00:57,410][105620] Updated weights for policy 1, policy_version 639322 (0.0010) [2023-12-26 20:00:57,454][105620] Updated weights for policy 1, policy_version 639332 (0.0006) [2023-12-26 20:00:57,497][105692] Updated weights for policy 0, policy_version 638640 (0.0008) [2023-12-26 20:00:57,550][105692] Updated weights for policy 0, policy_version 638650 (0.0008) [2023-12-26 20:00:57,595][105692] Updated weights for policy 0, policy_version 638660 (0.0008) [2023-12-26 20:00:58,050][105620] Updated weights for policy 1, policy_version 639342 (0.0005) [2023-12-26 20:00:58,098][105620] Updated weights for policy 1, policy_version 639352 (0.0005) [2023-12-26 20:00:58,167][105620] Updated weights for policy 1, policy_version 639362 (0.0007) [2023-12-26 20:00:58,317][105692] Updated weights for policy 0, policy_version 638670 (0.0009) [2023-12-26 20:00:58,382][105692] Updated weights for policy 0, policy_version 638680 (0.0009) [2023-12-26 20:00:58,447][105692] Updated weights for policy 0, policy_version 638690 (0.0008) [2023-12-26 20:00:58,945][105620] Updated weights for policy 1, policy_version 639372 (0.0008) [2023-12-26 20:00:59,000][105620] Updated weights for policy 1, policy_version 639382 (0.0008) [2023-12-26 20:00:59,055][105620] Updated weights for policy 1, policy_version 639392 (0.0008) [2023-12-26 20:00:59,245][105692] Updated weights for policy 0, policy_version 638700 (0.0010) [2023-12-26 20:00:59,295][105692] Updated weights for policy 0, policy_version 638710 (0.0009) [2023-12-26 20:00:59,360][105692] Updated weights for policy 0, policy_version 638720 (0.0011) [2023-12-26 20:00:59,734][105620] Updated weights for policy 1, policy_version 639402 (0.0006) [2023-12-26 20:00:59,792][105620] Updated weights for policy 1, policy_version 639412 (0.0006) [2023-12-26 20:00:59,851][105620] Updated weights for policy 1, policy_version 639422 (0.0007) [2023-12-26 20:00:59,916][105620] Updated weights for policy 1, policy_version 639432 (0.0005) [2023-12-26 20:01:00,095][105692] Updated weights for policy 0, policy_version 638730 (0.0010) [2023-12-26 20:01:00,153][105692] Updated weights for policy 0, policy_version 638740 (0.0009) [2023-12-26 20:01:00,211][105692] Updated weights for policy 0, policy_version 638750 (0.0009) [2023-12-26 20:01:00,272][105692] Updated weights for policy 0, policy_version 638760 (0.0010) [2023-12-26 20:01:00,605][105620] Updated weights for policy 1, policy_version 639442 (0.0010) [2023-12-26 20:01:00,657][105620] Updated weights for policy 1, policy_version 639452 (0.0010) [2023-12-26 20:01:00,710][105620] Updated weights for policy 1, policy_version 639462 (0.0011) [2023-12-26 20:01:00,918][105692] Updated weights for policy 0, policy_version 638770 (0.0005) [2023-12-26 20:01:00,973][105692] Updated weights for policy 0, policy_version 638780 (0.0008) [2023-12-26 20:01:01,037][105692] Updated weights for policy 0, policy_version 638790 (0.0009) [2023-12-26 20:01:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 327278592. Throughput: 0: 9853.2, 1: 9731.1. Samples: 327243144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:01:01,062][104569] Avg episode reward: [(0, '8270.457'), (1, '9192.770')] [2023-12-26 20:01:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000638792_163561472.pth... [2023-12-26 20:01:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000639464_163717120.pth... [2023-12-26 20:01:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000638344_163430400.pth [2023-12-26 20:01:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000637640_163266560.pth [2023-12-26 20:01:01,483][105620] Updated weights for policy 1, policy_version 639472 (0.0009) [2023-12-26 20:01:01,544][105620] Updated weights for policy 1, policy_version 639482 (0.0009) [2023-12-26 20:01:01,608][105620] Updated weights for policy 1, policy_version 639492 (0.0009) [2023-12-26 20:01:01,732][105692] Updated weights for policy 0, policy_version 638800 (0.0007) [2023-12-26 20:01:01,796][105692] Updated weights for policy 0, policy_version 638810 (0.0009) [2023-12-26 20:01:01,859][105692] Updated weights for policy 0, policy_version 638820 (0.0006) [2023-12-26 20:01:02,422][105620] Updated weights for policy 1, policy_version 639502 (0.0009) [2023-12-26 20:01:02,468][105620] Updated weights for policy 1, policy_version 639512 (0.0008) [2023-12-26 20:01:02,503][105692] Updated weights for policy 0, policy_version 638830 (0.0007) [2023-12-26 20:01:02,521][105620] Updated weights for policy 1, policy_version 639522 (0.0007) [2023-12-26 20:01:02,559][105692] Updated weights for policy 0, policy_version 638840 (0.0008) [2023-12-26 20:01:02,614][105692] Updated weights for policy 0, policy_version 638850 (0.0009) [2023-12-26 20:01:03,283][105620] Updated weights for policy 1, policy_version 639532 (0.0007) [2023-12-26 20:01:03,336][105620] Updated weights for policy 1, policy_version 639542 (0.0008) [2023-12-26 20:01:03,346][105692] Updated weights for policy 0, policy_version 638860 (0.0009) [2023-12-26 20:01:03,388][105620] Updated weights for policy 1, policy_version 639552 (0.0007) [2023-12-26 20:01:03,394][105692] Updated weights for policy 0, policy_version 638870 (0.0006) [2023-12-26 20:01:03,439][105692] Updated weights for policy 0, policy_version 638880 (0.0006) [2023-12-26 20:01:04,136][105620] Updated weights for policy 1, policy_version 639562 (0.0006) [2023-12-26 20:01:04,200][105620] Updated weights for policy 1, policy_version 639572 (0.0009) [2023-12-26 20:01:04,238][105692] Updated weights for policy 0, policy_version 638890 (0.0009) [2023-12-26 20:01:04,259][105620] Updated weights for policy 1, policy_version 639582 (0.0008) [2023-12-26 20:01:04,289][105692] Updated weights for policy 0, policy_version 638900 (0.0008) [2023-12-26 20:01:04,320][105620] Updated weights for policy 1, policy_version 639592 (0.0008) [2023-12-26 20:01:04,337][105692] Updated weights for policy 0, policy_version 638910 (0.0007) [2023-12-26 20:01:04,384][105692] Updated weights for policy 0, policy_version 638920 (0.0008) [2023-12-26 20:01:05,025][105620] Updated weights for policy 1, policy_version 639602 (0.0010) [2023-12-26 20:01:05,081][105620] Updated weights for policy 1, policy_version 639612 (0.0009) [2023-12-26 20:01:05,132][105620] Updated weights for policy 1, policy_version 639622 (0.0006) [2023-12-26 20:01:05,209][105692] Updated weights for policy 0, policy_version 638930 (0.0005) [2023-12-26 20:01:05,252][105692] Updated weights for policy 0, policy_version 638940 (0.0005) [2023-12-26 20:01:05,298][105692] Updated weights for policy 0, policy_version 638950 (0.0005) [2023-12-26 20:01:05,823][105620] Updated weights for policy 1, policy_version 639632 (0.0010) [2023-12-26 20:01:05,877][105620] Updated weights for policy 1, policy_version 639643 (0.0010) [2023-12-26 20:01:05,933][105620] Updated weights for policy 1, policy_version 639653 (0.0007) [2023-12-26 20:01:05,942][105692] Updated weights for policy 0, policy_version 638960 (0.0007) [2023-12-26 20:01:05,995][105692] Updated weights for policy 0, policy_version 638970 (0.0008) [2023-12-26 20:01:06,055][105692] Updated weights for policy 0, policy_version 638980 (0.0009) [2023-12-26 20:01:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 327368704. Throughput: 0: 9763.6, 1: 9629.4. Samples: 327357748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:01:06,063][104569] Avg episode reward: [(0, '8448.672'), (1, '9268.510')] [2023-12-26 20:01:06,740][105620] Updated weights for policy 1, policy_version 639663 (0.0006) [2023-12-26 20:01:06,776][105692] Updated weights for policy 0, policy_version 638990 (0.0009) [2023-12-26 20:01:06,807][105620] Updated weights for policy 1, policy_version 639673 (0.0006) [2023-12-26 20:01:06,838][105692] Updated weights for policy 0, policy_version 639000 (0.0007) [2023-12-26 20:01:06,880][105620] Updated weights for policy 1, policy_version 639683 (0.0006) [2023-12-26 20:01:06,900][105692] Updated weights for policy 0, policy_version 639010 (0.0006) [2023-12-26 20:01:06,922][105585] KL-divergence is very high: 108.1950 [2023-12-26 20:01:07,535][105692] Updated weights for policy 0, policy_version 639020 (0.0007) [2023-12-26 20:01:07,542][105620] Updated weights for policy 1, policy_version 639693 (0.0008) [2023-12-26 20:01:07,601][105692] Updated weights for policy 0, policy_version 639030 (0.0007) [2023-12-26 20:01:07,604][105620] Updated weights for policy 1, policy_version 639703 (0.0007) [2023-12-26 20:01:07,656][105692] Updated weights for policy 0, policy_version 639040 (0.0005) [2023-12-26 20:01:07,662][105620] Updated weights for policy 1, policy_version 639713 (0.0007) [2023-12-26 20:01:08,289][105585] KL-divergence is very high: 111.4267 [2023-12-26 20:01:08,295][105692] Updated weights for policy 0, policy_version 639050 (0.0007) [2023-12-26 20:01:08,358][105692] Updated weights for policy 0, policy_version 639060 (0.0008) [2023-12-26 20:01:08,389][105585] KL-divergence is very high: 126.9351 [2023-12-26 20:01:08,421][105692] Updated weights for policy 0, policy_version 639070 (0.0008) [2023-12-26 20:01:08,433][105620] Updated weights for policy 1, policy_version 639723 (0.0007) [2023-12-26 20:01:08,481][105692] Updated weights for policy 0, policy_version 639080 (0.0010) [2023-12-26 20:01:08,488][105620] Updated weights for policy 1, policy_version 639733 (0.0007) [2023-12-26 20:01:08,536][105620] Updated weights for policy 1, policy_version 639743 (0.0009) [2023-12-26 20:01:09,155][105692] Updated weights for policy 0, policy_version 639090 (0.0009) [2023-12-26 20:01:09,202][105692] Updated weights for policy 0, policy_version 639100 (0.0009) [2023-12-26 20:01:09,262][105692] Updated weights for policy 0, policy_version 639110 (0.0008) [2023-12-26 20:01:09,312][105620] Updated weights for policy 1, policy_version 639753 (0.0009) [2023-12-26 20:01:09,373][105620] Updated weights for policy 1, policy_version 639763 (0.0008) [2023-12-26 20:01:09,435][105620] Updated weights for policy 1, policy_version 639773 (0.0009) [2023-12-26 20:01:09,496][105620] Updated weights for policy 1, policy_version 639783 (0.0010) [2023-12-26 20:01:09,958][105692] Updated weights for policy 0, policy_version 639120 (0.0009) [2023-12-26 20:01:10,019][105692] Updated weights for policy 0, policy_version 639132 (0.0010) [2023-12-26 20:01:10,073][105692] Updated weights for policy 0, policy_version 639142 (0.0010) [2023-12-26 20:01:10,243][105620] Updated weights for policy 1, policy_version 639793 (0.0006) [2023-12-26 20:01:10,301][105620] Updated weights for policy 1, policy_version 639803 (0.0006) [2023-12-26 20:01:10,361][105620] Updated weights for policy 1, policy_version 639813 (0.0007) [2023-12-26 20:01:10,880][105692] Updated weights for policy 0, policy_version 639152 (0.0009) [2023-12-26 20:01:10,931][105692] Updated weights for policy 0, policy_version 639162 (0.0009) [2023-12-26 20:01:10,997][105692] Updated weights for policy 0, policy_version 639172 (0.0009) [2023-12-26 20:01:11,051][105620] Updated weights for policy 1, policy_version 639823 (0.0008) [2023-12-26 20:01:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 327467008. Throughput: 0: 9808.7, 1: 9477.2. Samples: 327474584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:01:11,062][104569] Avg episode reward: [(0, '639.961'), (1, '9177.304')] [2023-12-26 20:01:11,114][105620] Updated weights for policy 1, policy_version 639833 (0.0009) [2023-12-26 20:01:11,183][105620] Updated weights for policy 1, policy_version 639843 (0.0010) [2023-12-26 20:01:11,712][105692] Updated weights for policy 0, policy_version 639182 (0.0008) [2023-12-26 20:01:11,782][105692] Updated weights for policy 0, policy_version 639192 (0.0008) [2023-12-26 20:01:11,849][105692] Updated weights for policy 0, policy_version 639202 (0.0006) [2023-12-26 20:01:12,035][105620] Updated weights for policy 1, policy_version 639853 (0.0009) [2023-12-26 20:01:12,082][105620] Updated weights for policy 1, policy_version 639863 (0.0009) [2023-12-26 20:01:12,131][105620] Updated weights for policy 1, policy_version 639873 (0.0009) [2023-12-26 20:01:12,541][105692] Updated weights for policy 0, policy_version 639212 (0.0008) [2023-12-26 20:01:12,598][105692] Updated weights for policy 0, policy_version 639222 (0.0010) [2023-12-26 20:01:12,656][105692] Updated weights for policy 0, policy_version 639232 (0.0008) [2023-12-26 20:01:12,915][105620] Updated weights for policy 1, policy_version 639883 (0.0009) [2023-12-26 20:01:12,970][105620] Updated weights for policy 1, policy_version 639893 (0.0009) [2023-12-26 20:01:13,019][105620] Updated weights for policy 1, policy_version 639903 (0.0008) [2023-12-26 20:01:13,444][105692] Updated weights for policy 0, policy_version 639242 (0.0008) [2023-12-26 20:01:13,499][105692] Updated weights for policy 0, policy_version 639252 (0.0008) [2023-12-26 20:01:13,560][105692] Updated weights for policy 0, policy_version 639262 (0.0008) [2023-12-26 20:01:13,612][105692] Updated weights for policy 0, policy_version 639272 (0.0008) [2023-12-26 20:01:13,725][105620] Updated weights for policy 1, policy_version 639913 (0.0010) [2023-12-26 20:01:13,775][105620] Updated weights for policy 1, policy_version 639923 (0.0010) [2023-12-26 20:01:13,825][105620] Updated weights for policy 1, policy_version 639933 (0.0008) [2023-12-26 20:01:13,870][105620] Updated weights for policy 1, policy_version 639943 (0.0005) [2023-12-26 20:01:14,356][105692] Updated weights for policy 0, policy_version 639282 (0.0009) [2023-12-26 20:01:14,408][105692] Updated weights for policy 0, policy_version 639292 (0.0009) [2023-12-26 20:01:14,459][105692] Updated weights for policy 0, policy_version 639302 (0.0008) [2023-12-26 20:01:14,469][105620] Updated weights for policy 1, policy_version 639953 (0.0008) [2023-12-26 20:01:14,525][105620] Updated weights for policy 1, policy_version 639963 (0.0006) [2023-12-26 20:01:14,584][105620] Updated weights for policy 1, policy_version 639973 (0.0008) [2023-12-26 20:01:15,258][105692] Updated weights for policy 0, policy_version 639312 (0.0010) [2023-12-26 20:01:15,314][105620] Updated weights for policy 1, policy_version 639983 (0.0010) [2023-12-26 20:01:15,314][105692] Updated weights for policy 0, policy_version 639322 (0.0011) [2023-12-26 20:01:15,374][105620] Updated weights for policy 1, policy_version 639993 (0.0010) [2023-12-26 20:01:15,376][105692] Updated weights for policy 0, policy_version 639332 (0.0011) [2023-12-26 20:01:15,430][105620] Updated weights for policy 1, policy_version 640003 (0.0009) [2023-12-26 20:01:16,006][105620] Updated weights for policy 1, policy_version 640013 (0.0008) [2023-12-26 20:01:16,051][105620] Updated weights for policy 1, policy_version 640023 (0.0010) [2023-12-26 20:01:16,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 327557120. Throughput: 0: 9706.0, 1: 9472.1. Samples: 327530364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:01:16,063][104569] Avg episode reward: [(0, '932.927'), (1, '9172.411')] [2023-12-26 20:01:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000639336_163700736.pth... [2023-12-26 20:01:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000638216_163414016.pth [2023-12-26 20:01:16,098][105620] Updated weights for policy 1, policy_version 640033 (0.0010) [2023-12-26 20:01:16,120][105692] Updated weights for policy 0, policy_version 639342 (0.0010) [2023-12-26 20:01:16,135][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000640040_163864576.pth... [2023-12-26 20:01:16,137][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000638920_163577856.pth [2023-12-26 20:01:16,171][105692] Updated weights for policy 0, policy_version 639352 (0.0010) [2023-12-26 20:01:16,219][105692] Updated weights for policy 0, policy_version 639362 (0.0010) [2023-12-26 20:01:16,789][105620] Updated weights for policy 1, policy_version 640043 (0.0009) [2023-12-26 20:01:16,847][105620] Updated weights for policy 1, policy_version 640053 (0.0006) [2023-12-26 20:01:16,911][105620] Updated weights for policy 1, policy_version 640063 (0.0006) [2023-12-26 20:01:16,935][105692] Updated weights for policy 0, policy_version 639372 (0.0009) [2023-12-26 20:01:16,997][105692] Updated weights for policy 0, policy_version 639382 (0.0011) [2023-12-26 20:01:17,065][105692] Updated weights for policy 0, policy_version 639392 (0.0010) [2023-12-26 20:01:17,503][105620] Updated weights for policy 1, policy_version 640073 (0.0006) [2023-12-26 20:01:17,559][105620] Updated weights for policy 1, policy_version 640083 (0.0009) [2023-12-26 20:01:17,619][105620] Updated weights for policy 1, policy_version 640093 (0.0009) [2023-12-26 20:01:17,678][105620] Updated weights for policy 1, policy_version 640103 (0.0007) [2023-12-26 20:01:17,707][105692] Updated weights for policy 0, policy_version 639402 (0.0010) [2023-12-26 20:01:17,773][105692] Updated weights for policy 0, policy_version 639412 (0.0010) [2023-12-26 20:01:17,841][105692] Updated weights for policy 0, policy_version 639422 (0.0009) [2023-12-26 20:01:17,907][105692] Updated weights for policy 0, policy_version 639432 (0.0007) [2023-12-26 20:01:18,412][105620] Updated weights for policy 1, policy_version 640113 (0.0008) [2023-12-26 20:01:18,475][105620] Updated weights for policy 1, policy_version 640123 (0.0008) [2023-12-26 20:01:18,489][105692] Updated weights for policy 0, policy_version 639442 (0.0008) [2023-12-26 20:01:18,537][105620] Updated weights for policy 1, policy_version 640133 (0.0007) [2023-12-26 20:01:18,540][105692] Updated weights for policy 0, policy_version 639452 (0.0007) [2023-12-26 20:01:18,592][105692] Updated weights for policy 0, policy_version 639462 (0.0009) [2023-12-26 20:01:19,301][105692] Updated weights for policy 0, policy_version 639472 (0.0010) [2023-12-26 20:01:19,344][105620] Updated weights for policy 1, policy_version 640143 (0.0008) [2023-12-26 20:01:19,372][105692] Updated weights for policy 0, policy_version 639482 (0.0011) [2023-12-26 20:01:19,402][105620] Updated weights for policy 1, policy_version 640153 (0.0006) [2023-12-26 20:01:19,430][105692] Updated weights for policy 0, policy_version 639492 (0.0010) [2023-12-26 20:01:19,469][105620] Updated weights for policy 1, policy_version 640163 (0.0006) [2023-12-26 20:01:20,185][105692] Updated weights for policy 0, policy_version 639502 (0.0011) [2023-12-26 20:01:20,238][105620] Updated weights for policy 1, policy_version 640173 (0.0007) [2023-12-26 20:01:20,247][105692] Updated weights for policy 0, policy_version 639512 (0.0011) [2023-12-26 20:01:20,298][105620] Updated weights for policy 1, policy_version 640183 (0.0009) [2023-12-26 20:01:20,312][105692] Updated weights for policy 0, policy_version 639522 (0.0011) [2023-12-26 20:01:20,358][105620] Updated weights for policy 1, policy_version 640193 (0.0008) [2023-12-26 20:01:20,947][105692] Updated weights for policy 0, policy_version 639532 (0.0009) [2023-12-26 20:01:21,015][105692] Updated weights for policy 0, policy_version 639542 (0.0010) [2023-12-26 20:01:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 327655424. Throughput: 0: 9680.1, 1: 9460.7. Samples: 327650184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:01:21,062][104569] Avg episode reward: [(0, '1710.667'), (1, '9080.879')] [2023-12-26 20:01:21,078][105692] Updated weights for policy 0, policy_version 639552 (0.0008) [2023-12-26 20:01:21,123][105620] Updated weights for policy 1, policy_version 640203 (0.0009) [2023-12-26 20:01:21,190][105620] Updated weights for policy 1, policy_version 640213 (0.0006) [2023-12-26 20:01:21,263][105620] Updated weights for policy 1, policy_version 640223 (0.0007) [2023-12-26 20:01:21,890][105692] Updated weights for policy 0, policy_version 639562 (0.0007) [2023-12-26 20:01:21,942][105692] Updated weights for policy 0, policy_version 639572 (0.0009) [2023-12-26 20:01:21,974][105620] Updated weights for policy 1, policy_version 640233 (0.0008) [2023-12-26 20:01:21,991][105692] Updated weights for policy 0, policy_version 639582 (0.0009) [2023-12-26 20:01:22,030][105620] Updated weights for policy 1, policy_version 640243 (0.0006) [2023-12-26 20:01:22,051][105692] Updated weights for policy 0, policy_version 639592 (0.0008) [2023-12-26 20:01:22,099][105620] Updated weights for policy 1, policy_version 640254 (0.0011) [2023-12-26 20:01:22,157][105620] Updated weights for policy 1, policy_version 640264 (0.0010) [2023-12-26 20:01:22,750][105692] Updated weights for policy 0, policy_version 639602 (0.0011) [2023-12-26 20:01:22,811][105692] Updated weights for policy 0, policy_version 639612 (0.0011) [2023-12-26 20:01:22,864][105692] Updated weights for policy 0, policy_version 639622 (0.0011) [2023-12-26 20:01:22,971][105620] Updated weights for policy 1, policy_version 640274 (0.0009) [2023-12-26 20:01:23,028][105620] Updated weights for policy 1, policy_version 640284 (0.0009) [2023-12-26 20:01:23,091][105620] Updated weights for policy 1, policy_version 640294 (0.0010) [2023-12-26 20:01:23,544][105692] Updated weights for policy 0, policy_version 639632 (0.0009) [2023-12-26 20:01:23,592][105692] Updated weights for policy 0, policy_version 639642 (0.0009) [2023-12-26 20:01:23,641][105692] Updated weights for policy 0, policy_version 639652 (0.0009) [2023-12-26 20:01:23,884][105620] Updated weights for policy 1, policy_version 640304 (0.0009) [2023-12-26 20:01:23,952][105620] Updated weights for policy 1, policy_version 640314 (0.0009) [2023-12-26 20:01:24,014][105620] Updated weights for policy 1, policy_version 640324 (0.0008) [2023-12-26 20:01:24,344][105692] Updated weights for policy 0, policy_version 639662 (0.0008) [2023-12-26 20:01:24,411][105692] Updated weights for policy 0, policy_version 639672 (0.0006) [2023-12-26 20:01:24,476][105692] Updated weights for policy 0, policy_version 639682 (0.0007) [2023-12-26 20:01:24,798][105620] Updated weights for policy 1, policy_version 640334 (0.0010) [2023-12-26 20:01:24,852][105620] Updated weights for policy 1, policy_version 640344 (0.0009) [2023-12-26 20:01:24,899][105620] Updated weights for policy 1, policy_version 640354 (0.0009) [2023-12-26 20:01:25,139][105692] Updated weights for policy 0, policy_version 639692 (0.0009) [2023-12-26 20:01:25,190][105692] Updated weights for policy 0, policy_version 639702 (0.0009) [2023-12-26 20:01:25,238][105692] Updated weights for policy 0, policy_version 639712 (0.0009) [2023-12-26 20:01:25,637][105620] Updated weights for policy 1, policy_version 640364 (0.0009) [2023-12-26 20:01:25,691][105620] Updated weights for policy 1, policy_version 640374 (0.0009) [2023-12-26 20:01:25,748][105620] Updated weights for policy 1, policy_version 640384 (0.0008) [2023-12-26 20:01:26,014][105692] Updated weights for policy 0, policy_version 639722 (0.0009) [2023-12-26 20:01:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 327753728. Throughput: 0: 9715.6, 1: 9300.6. Samples: 327763124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:01:26,062][104569] Avg episode reward: [(0, '5651.740'), (1, '9262.947')] [2023-12-26 20:01:26,066][105692] Updated weights for policy 0, policy_version 639732 (0.0009) [2023-12-26 20:01:26,113][105692] Updated weights for policy 0, policy_version 639742 (0.0009) [2023-12-26 20:01:26,162][105692] Updated weights for policy 0, policy_version 639752 (0.0009) [2023-12-26 20:01:26,500][105620] Updated weights for policy 1, policy_version 640394 (0.0009) [2023-12-26 20:01:26,555][105620] Updated weights for policy 1, policy_version 640404 (0.0009) [2023-12-26 20:01:26,607][105620] Updated weights for policy 1, policy_version 640414 (0.0008) [2023-12-26 20:01:26,654][105620] Updated weights for policy 1, policy_version 640424 (0.0008) [2023-12-26 20:01:26,914][105585] KL-divergence is very high: 102.5715 [2023-12-26 20:01:26,920][105692] Updated weights for policy 0, policy_version 639762 (0.0009) [2023-12-26 20:01:26,951][105585] KL-divergence is very high: 110.3575 [2023-12-26 20:01:26,967][105692] Updated weights for policy 0, policy_version 639772 (0.0008) [2023-12-26 20:01:27,015][105692] Updated weights for policy 0, policy_version 639782 (0.0009) [2023-12-26 20:01:27,446][105620] Updated weights for policy 1, policy_version 640434 (0.0009) [2023-12-26 20:01:27,496][105620] Updated weights for policy 1, policy_version 640444 (0.0009) [2023-12-26 20:01:27,542][105620] Updated weights for policy 1, policy_version 640454 (0.0008) [2023-12-26 20:01:27,724][105692] Updated weights for policy 0, policy_version 639792 (0.0009) [2023-12-26 20:01:27,771][105692] Updated weights for policy 0, policy_version 639802 (0.0009) [2023-12-26 20:01:27,784][105585] KL-divergence is very high: 106.8783 [2023-12-26 20:01:27,818][105692] Updated weights for policy 0, policy_version 639812 (0.0009) [2023-12-26 20:01:27,822][105585] KL-divergence is very high: 120.3062 [2023-12-26 20:01:28,304][105620] Updated weights for policy 1, policy_version 640464 (0.0008) [2023-12-26 20:01:28,365][105620] Updated weights for policy 1, policy_version 640474 (0.0007) [2023-12-26 20:01:28,434][105620] Updated weights for policy 1, policy_version 640484 (0.0005) [2023-12-26 20:01:28,617][105692] Updated weights for policy 0, policy_version 639822 (0.0009) [2023-12-26 20:01:28,679][105692] Updated weights for policy 0, policy_version 639832 (0.0009) [2023-12-26 20:01:28,746][105692] Updated weights for policy 0, policy_version 639842 (0.0009) [2023-12-26 20:01:29,107][105620] Updated weights for policy 1, policy_version 640494 (0.0008) [2023-12-26 20:01:29,163][105620] Updated weights for policy 1, policy_version 640504 (0.0008) [2023-12-26 20:01:29,210][105620] Updated weights for policy 1, policy_version 640514 (0.0009) [2023-12-26 20:01:29,495][105692] Updated weights for policy 0, policy_version 639852 (0.0008) [2023-12-26 20:01:29,547][105692] Updated weights for policy 0, policy_version 639862 (0.0005) [2023-12-26 20:01:29,602][105692] Updated weights for policy 0, policy_version 639872 (0.0006) [2023-12-26 20:01:30,030][105620] Updated weights for policy 1, policy_version 640524 (0.0007) [2023-12-26 20:01:30,087][105620] Updated weights for policy 1, policy_version 640534 (0.0005) [2023-12-26 20:01:30,148][105620] Updated weights for policy 1, policy_version 640544 (0.0009) [2023-12-26 20:01:30,263][105692] Updated weights for policy 0, policy_version 639882 (0.0006) [2023-12-26 20:01:30,323][105692] Updated weights for policy 0, policy_version 639892 (0.0009) [2023-12-26 20:01:30,382][105692] Updated weights for policy 0, policy_version 639902 (0.0010) [2023-12-26 20:01:30,442][105692] Updated weights for policy 0, policy_version 639912 (0.0005) [2023-12-26 20:01:30,835][105620] Updated weights for policy 1, policy_version 640554 (0.0008) [2023-12-26 20:01:30,889][105620] Updated weights for policy 1, policy_version 640564 (0.0010) [2023-12-26 20:01:30,953][105620] Updated weights for policy 1, policy_version 640574 (0.0010) [2023-12-26 20:01:31,015][105620] Updated weights for policy 1, policy_version 640584 (0.0006) [2023-12-26 20:01:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 327852032. Throughput: 0: 9730.8, 1: 9350.7. Samples: 327820772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:01:31,063][104569] Avg episode reward: [(0, '5055.689'), (1, '9353.797')] [2023-12-26 20:01:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000639912_163848192.pth... [2023-12-26 20:01:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000640584_164003840.pth... [2023-12-26 20:01:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000639464_163717120.pth [2023-12-26 20:01:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000638792_163561472.pth [2023-12-26 20:01:31,151][105692] Updated weights for policy 0, policy_version 639922 (0.0010) [2023-12-26 20:01:31,213][105692] Updated weights for policy 0, policy_version 639932 (0.0007) [2023-12-26 20:01:31,273][105692] Updated weights for policy 0, policy_version 639942 (0.0007) [2023-12-26 20:01:31,662][105620] Updated weights for policy 1, policy_version 640594 (0.0009) [2023-12-26 20:01:31,711][105620] Updated weights for policy 1, policy_version 640604 (0.0010) [2023-12-26 20:01:31,772][105620] Updated weights for policy 1, policy_version 640614 (0.0009) [2023-12-26 20:01:31,973][105692] Updated weights for policy 0, policy_version 639952 (0.0008) [2023-12-26 20:01:32,034][105692] Updated weights for policy 0, policy_version 639962 (0.0007) [2023-12-26 20:01:32,100][105692] Updated weights for policy 0, policy_version 639972 (0.0008) [2023-12-26 20:01:32,421][105620] Updated weights for policy 1, policy_version 640624 (0.0009) [2023-12-26 20:01:32,473][105620] Updated weights for policy 1, policy_version 640634 (0.0010) [2023-12-26 20:01:32,492][105586] KL-divergence is very high: 266.0932 [2023-12-26 20:01:32,530][105620] Updated weights for policy 1, policy_version 640644 (0.0009) [2023-12-26 20:01:32,544][105586] KL-divergence is very high: 380.3165 [2023-12-26 20:01:32,873][105692] Updated weights for policy 0, policy_version 639982 (0.0009) [2023-12-26 20:01:32,920][105692] Updated weights for policy 0, policy_version 639992 (0.0010) [2023-12-26 20:01:32,962][105692] Updated weights for policy 0, policy_version 640002 (0.0010) [2023-12-26 20:01:33,260][105620] Updated weights for policy 1, policy_version 640654 (0.0010) [2023-12-26 20:01:33,319][105620] Updated weights for policy 1, policy_version 640664 (0.0009) [2023-12-26 20:01:33,370][105620] Updated weights for policy 1, policy_version 640674 (0.0008) [2023-12-26 20:01:33,663][105692] Updated weights for policy 0, policy_version 640012 (0.0008) [2023-12-26 20:01:33,714][105692] Updated weights for policy 0, policy_version 640022 (0.0005) [2023-12-26 20:01:33,767][105692] Updated weights for policy 0, policy_version 640032 (0.0005) [2023-12-26 20:01:34,191][105620] Updated weights for policy 1, policy_version 640684 (0.0008) [2023-12-26 20:01:34,261][105620] Updated weights for policy 1, policy_version 640694 (0.0008) [2023-12-26 20:01:34,326][105620] Updated weights for policy 1, policy_version 640704 (0.0009) [2023-12-26 20:01:34,460][105692] Updated weights for policy 0, policy_version 640042 (0.0006) [2023-12-26 20:01:34,517][105692] Updated weights for policy 0, policy_version 640052 (0.0009) [2023-12-26 20:01:34,573][105692] Updated weights for policy 0, policy_version 640063 (0.0008) [2023-12-26 20:01:35,134][105620] Updated weights for policy 1, policy_version 640714 (0.0008) [2023-12-26 20:01:35,155][105692] Updated weights for policy 0, policy_version 640073 (0.0005) [2023-12-26 20:01:35,184][105620] Updated weights for policy 1, policy_version 640724 (0.0007) [2023-12-26 20:01:35,211][105692] Updated weights for policy 0, policy_version 640083 (0.0006) [2023-12-26 20:01:35,240][105620] Updated weights for policy 1, policy_version 640734 (0.0006) [2023-12-26 20:01:35,261][105692] Updated weights for policy 0, policy_version 640093 (0.0006) [2023-12-26 20:01:35,307][105620] Updated weights for policy 1, policy_version 640744 (0.0007) [2023-12-26 20:01:35,309][105692] Updated weights for policy 0, policy_version 640103 (0.0006) [2023-12-26 20:01:35,859][105620] Updated weights for policy 1, policy_version 640754 (0.0005) [2023-12-26 20:01:35,908][105620] Updated weights for policy 1, policy_version 640764 (0.0005) [2023-12-26 20:01:35,961][105620] Updated weights for policy 1, policy_version 640774 (0.0006) [2023-12-26 20:01:36,025][105692] Updated weights for policy 0, policy_version 640113 (0.0011) [2023-12-26 20:01:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 327950336. Throughput: 0: 9732.8, 1: 9423.4. Samples: 327936976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:01:36,062][104569] Avg episode reward: [(0, '7315.234'), (1, '8547.369')] [2023-12-26 20:01:36,084][105692] Updated weights for policy 0, policy_version 640123 (0.0010) [2023-12-26 20:01:36,147][105692] Updated weights for policy 0, policy_version 640133 (0.0010) [2023-12-26 20:01:36,689][105620] Updated weights for policy 1, policy_version 640784 (0.0006) [2023-12-26 20:01:36,755][105620] Updated weights for policy 1, policy_version 640794 (0.0008) [2023-12-26 20:01:36,816][105620] Updated weights for policy 1, policy_version 640804 (0.0010) [2023-12-26 20:01:36,844][105692] Updated weights for policy 0, policy_version 640143 (0.0006) [2023-12-26 20:01:36,910][105692] Updated weights for policy 0, policy_version 640153 (0.0005) [2023-12-26 20:01:36,977][105692] Updated weights for policy 0, policy_version 640163 (0.0005) [2023-12-26 20:01:37,517][105620] Updated weights for policy 1, policy_version 640814 (0.0009) [2023-12-26 20:01:37,576][105620] Updated weights for policy 1, policy_version 640824 (0.0008) [2023-12-26 20:01:37,637][105620] Updated weights for policy 1, policy_version 640834 (0.0008) [2023-12-26 20:01:37,650][105692] Updated weights for policy 0, policy_version 640173 (0.0008) [2023-12-26 20:01:37,717][105692] Updated weights for policy 0, policy_version 640183 (0.0005) [2023-12-26 20:01:37,775][105692] Updated weights for policy 0, policy_version 640193 (0.0005) [2023-12-26 20:01:38,343][105692] Updated weights for policy 0, policy_version 640203 (0.0006) [2023-12-26 20:01:38,407][105692] Updated weights for policy 0, policy_version 640213 (0.0009) [2023-12-26 20:01:38,459][105620] Updated weights for policy 1, policy_version 640844 (0.0007) [2023-12-26 20:01:38,463][105692] Updated weights for policy 0, policy_version 640223 (0.0008) [2023-12-26 20:01:38,513][105620] Updated weights for policy 1, policy_version 640854 (0.0006) [2023-12-26 20:01:38,572][105620] Updated weights for policy 1, policy_version 640864 (0.0008) [2023-12-26 20:01:39,164][105692] Updated weights for policy 0, policy_version 640233 (0.0008) [2023-12-26 20:01:39,228][105692] Updated weights for policy 0, policy_version 640243 (0.0007) [2023-12-26 20:01:39,296][105692] Updated weights for policy 0, policy_version 640253 (0.0007) [2023-12-26 20:01:39,365][105692] Updated weights for policy 0, policy_version 640263 (0.0008) [2023-12-26 20:01:39,393][105620] Updated weights for policy 1, policy_version 640874 (0.0008) [2023-12-26 20:01:39,461][105620] Updated weights for policy 1, policy_version 640884 (0.0007) [2023-12-26 20:01:39,521][105620] Updated weights for policy 1, policy_version 640894 (0.0009) [2023-12-26 20:01:39,583][105620] Updated weights for policy 1, policy_version 640904 (0.0008) [2023-12-26 20:01:40,063][105692] Updated weights for policy 0, policy_version 640273 (0.0010) [2023-12-26 20:01:40,116][105692] Updated weights for policy 0, policy_version 640283 (0.0010) [2023-12-26 20:01:40,179][105692] Updated weights for policy 0, policy_version 640293 (0.0010) [2023-12-26 20:01:40,339][105620] Updated weights for policy 1, policy_version 640914 (0.0008) [2023-12-26 20:01:40,399][105620] Updated weights for policy 1, policy_version 640924 (0.0008) [2023-12-26 20:01:40,477][105620] Updated weights for policy 1, policy_version 640934 (0.0008) [2023-12-26 20:01:40,923][105692] Updated weights for policy 0, policy_version 640303 (0.0010) [2023-12-26 20:01:40,985][105692] Updated weights for policy 0, policy_version 640313 (0.0009) [2023-12-26 20:01:41,048][105692] Updated weights for policy 0, policy_version 640323 (0.0011) [2023-12-26 20:01:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 18978.1, 300 sec: 19522.0). Total num frames: 328040448. Throughput: 0: 9822.9, 1: 9533.6. Samples: 328055024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:01:41,063][104569] Avg episode reward: [(0, '8451.270'), (1, '8107.159')] [2023-12-26 20:01:41,198][105620] Updated weights for policy 1, policy_version 640944 (0.0008) [2023-12-26 20:01:41,268][105620] Updated weights for policy 1, policy_version 640954 (0.0008) [2023-12-26 20:01:41,332][105620] Updated weights for policy 1, policy_version 640964 (0.0008) [2023-12-26 20:01:41,809][105692] Updated weights for policy 0, policy_version 640333 (0.0007) [2023-12-26 20:01:41,869][105692] Updated weights for policy 0, policy_version 640343 (0.0010) [2023-12-26 20:01:41,928][105692] Updated weights for policy 0, policy_version 640353 (0.0010) [2023-12-26 20:01:42,094][105620] Updated weights for policy 1, policy_version 640974 (0.0009) [2023-12-26 20:01:42,148][105620] Updated weights for policy 1, policy_version 640984 (0.0009) [2023-12-26 20:01:42,197][105620] Updated weights for policy 1, policy_version 640994 (0.0008) [2023-12-26 20:01:42,580][105692] Updated weights for policy 0, policy_version 640363 (0.0007) [2023-12-26 20:01:42,646][105692] Updated weights for policy 0, policy_version 640373 (0.0009) [2023-12-26 20:01:42,708][105692] Updated weights for policy 0, policy_version 640383 (0.0009) [2023-12-26 20:01:42,883][105620] Updated weights for policy 1, policy_version 641004 (0.0008) [2023-12-26 20:01:42,944][105620] Updated weights for policy 1, policy_version 641014 (0.0008) [2023-12-26 20:01:43,002][105620] Updated weights for policy 1, policy_version 641024 (0.0009) [2023-12-26 20:01:43,381][105692] Updated weights for policy 0, policy_version 640393 (0.0008) [2023-12-26 20:01:43,428][105692] Updated weights for policy 0, policy_version 640403 (0.0009) [2023-12-26 20:01:43,490][105692] Updated weights for policy 0, policy_version 640413 (0.0009) [2023-12-26 20:01:43,549][105692] Updated weights for policy 0, policy_version 640423 (0.0009) [2023-12-26 20:01:43,738][105620] Updated weights for policy 1, policy_version 641034 (0.0007) [2023-12-26 20:01:43,789][105620] Updated weights for policy 1, policy_version 641044 (0.0009) [2023-12-26 20:01:43,841][105620] Updated weights for policy 1, policy_version 641054 (0.0009) [2023-12-26 20:01:43,898][105620] Updated weights for policy 1, policy_version 641064 (0.0009) [2023-12-26 20:01:44,164][105692] Updated weights for policy 0, policy_version 640433 (0.0005) [2023-12-26 20:01:44,213][105692] Updated weights for policy 0, policy_version 640443 (0.0006) [2023-12-26 20:01:44,265][105692] Updated weights for policy 0, policy_version 640453 (0.0005) [2023-12-26 20:01:44,782][105692] Updated weights for policy 0, policy_version 640463 (0.0007) [2023-12-26 20:01:44,836][105620] Updated weights for policy 1, policy_version 641074 (0.0007) [2023-12-26 20:01:44,866][105692] Updated weights for policy 0, policy_version 640473 (0.0008) [2023-12-26 20:01:44,897][105620] Updated weights for policy 1, policy_version 641084 (0.0008) [2023-12-26 20:01:44,926][105692] Updated weights for policy 0, policy_version 640483 (0.0008) [2023-12-26 20:01:44,956][105620] Updated weights for policy 1, policy_version 641094 (0.0006) [2023-12-26 20:01:45,640][105620] Updated weights for policy 1, policy_version 641104 (0.0008) [2023-12-26 20:01:45,686][105692] Updated weights for policy 0, policy_version 640493 (0.0009) [2023-12-26 20:01:45,696][105620] Updated weights for policy 1, policy_version 641114 (0.0007) [2023-12-26 20:01:45,741][105692] Updated weights for policy 0, policy_version 640503 (0.0010) [2023-12-26 20:01:45,744][105620] Updated weights for policy 1, policy_version 641124 (0.0005) [2023-12-26 20:01:45,796][105692] Updated weights for policy 0, policy_version 640513 (0.0010) [2023-12-26 20:01:46,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 328146944. Throughput: 0: 9816.1, 1: 9492.5. Samples: 328112036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:01:46,063][104569] Avg episode reward: [(0, '8996.439'), (1, '8825.380')] [2023-12-26 20:01:46,074][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000641128_164143104.pth... [2023-12-26 20:01:46,074][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000640520_164003840.pth... [2023-12-26 20:01:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000640040_163864576.pth [2023-12-26 20:01:46,078][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_000641128_164143104.pth [2023-12-26 20:01:46,082][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000639336_163700736.pth [2023-12-26 20:01:46,083][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_000640520_164003840.pth [2023-12-26 20:01:46,391][105620] Updated weights for policy 1, policy_version 641134 (0.0008) [2023-12-26 20:01:46,453][105620] Updated weights for policy 1, policy_version 641144 (0.0010) [2023-12-26 20:01:46,518][105620] Updated weights for policy 1, policy_version 641154 (0.0009) [2023-12-26 20:01:46,546][105692] Updated weights for policy 0, policy_version 640523 (0.0010) [2023-12-26 20:01:46,598][105692] Updated weights for policy 0, policy_version 640533 (0.0010) [2023-12-26 20:01:46,652][105692] Updated weights for policy 0, policy_version 640543 (0.0010) [2023-12-26 20:01:47,109][105620] Updated weights for policy 1, policy_version 641164 (0.0008) [2023-12-26 20:01:47,170][105620] Updated weights for policy 1, policy_version 641174 (0.0005) [2023-12-26 20:01:47,221][105620] Updated weights for policy 1, policy_version 641184 (0.0005) [2023-12-26 20:01:47,412][105692] Updated weights for policy 0, policy_version 640553 (0.0010) [2023-12-26 20:01:47,460][105692] Updated weights for policy 0, policy_version 640563 (0.0010) [2023-12-26 20:01:47,508][105692] Updated weights for policy 0, policy_version 640573 (0.0010) [2023-12-26 20:01:47,556][105692] Updated weights for policy 0, policy_version 640583 (0.0010) [2023-12-26 20:01:47,720][105620] Updated weights for policy 1, policy_version 641194 (0.0005) [2023-12-26 20:01:47,781][105620] Updated weights for policy 1, policy_version 641204 (0.0005) [2023-12-26 20:01:47,847][105620] Updated weights for policy 1, policy_version 641214 (0.0005) [2023-12-26 20:01:47,907][105620] Updated weights for policy 1, policy_version 641224 (0.0010) [2023-12-26 20:01:48,341][105692] Updated weights for policy 0, policy_version 640593 (0.0011) [2023-12-26 20:01:48,403][105692] Updated weights for policy 0, policy_version 640603 (0.0010) [2023-12-26 20:01:48,469][105692] Updated weights for policy 0, policy_version 640613 (0.0011) [2023-12-26 20:01:48,588][105620] Updated weights for policy 1, policy_version 641234 (0.0010) [2023-12-26 20:01:48,651][105620] Updated weights for policy 1, policy_version 641244 (0.0010) [2023-12-26 20:01:48,717][105620] Updated weights for policy 1, policy_version 641254 (0.0009) [2023-12-26 20:01:49,211][105692] Updated weights for policy 0, policy_version 640623 (0.0010) [2023-12-26 20:01:49,280][105692] Updated weights for policy 0, policy_version 640633 (0.0010) [2023-12-26 20:01:49,343][105620] Updated weights for policy 1, policy_version 641264 (0.0011) [2023-12-26 20:01:49,346][105692] Updated weights for policy 0, policy_version 640643 (0.0011) [2023-12-26 20:01:49,405][105620] Updated weights for policy 1, policy_version 641274 (0.0010) [2023-12-26 20:01:49,468][105620] Updated weights for policy 1, policy_version 641284 (0.0011) [2023-12-26 20:01:50,091][105692] Updated weights for policy 0, policy_version 640653 (0.0011) [2023-12-26 20:01:50,143][105692] Updated weights for policy 0, policy_version 640663 (0.0010) [2023-12-26 20:01:50,198][105692] Updated weights for policy 0, policy_version 640673 (0.0010) [2023-12-26 20:01:50,211][105620] Updated weights for policy 1, policy_version 641294 (0.0010) [2023-12-26 20:01:50,273][105620] Updated weights for policy 1, policy_version 641304 (0.0007) [2023-12-26 20:01:50,321][105620] Updated weights for policy 1, policy_version 641314 (0.0008) [2023-12-26 20:01:50,964][105692] Updated weights for policy 0, policy_version 640683 (0.0010) [2023-12-26 20:01:51,025][105692] Updated weights for policy 0, policy_version 640693 (0.0011) [2023-12-26 20:01:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19114.7, 300 sec: 19521.9). Total num frames: 328237056. Throughput: 0: 9850.2, 1: 9607.8. Samples: 328233360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:01:51,062][104569] Avg episode reward: [(0, '9086.373'), (1, '9172.961')] [2023-12-26 20:01:51,089][105692] Updated weights for policy 0, policy_version 640703 (0.0011) [2023-12-26 20:01:51,116][105620] Updated weights for policy 1, policy_version 641324 (0.0008) [2023-12-26 20:01:51,169][105620] Updated weights for policy 1, policy_version 641334 (0.0007) [2023-12-26 20:01:51,221][105620] Updated weights for policy 1, policy_version 641344 (0.0007) [2023-12-26 20:01:51,868][105692] Updated weights for policy 0, policy_version 640713 (0.0010) [2023-12-26 20:01:51,917][105692] Updated weights for policy 0, policy_version 640723 (0.0008) [2023-12-26 20:01:51,974][105692] Updated weights for policy 0, policy_version 640733 (0.0008) [2023-12-26 20:01:52,034][105692] Updated weights for policy 0, policy_version 640743 (0.0008) [2023-12-26 20:01:52,042][105620] Updated weights for policy 1, policy_version 641354 (0.0009) [2023-12-26 20:01:52,097][105620] Updated weights for policy 1, policy_version 641364 (0.0010) [2023-12-26 20:01:52,161][105620] Updated weights for policy 1, policy_version 641374 (0.0007) [2023-12-26 20:01:52,223][105620] Updated weights for policy 1, policy_version 641384 (0.0009) [2023-12-26 20:01:52,806][105692] Updated weights for policy 0, policy_version 640753 (0.0011) [2023-12-26 20:01:52,819][105585] KL-divergence is very high: 128.5674 [2023-12-26 20:01:52,869][105692] Updated weights for policy 0, policy_version 640763 (0.0006) [2023-12-26 20:01:52,871][105585] KL-divergence is very high: 139.3778 [2023-12-26 20:01:52,900][105620] Updated weights for policy 1, policy_version 641394 (0.0009) [2023-12-26 20:01:52,930][105692] Updated weights for policy 0, policy_version 640773 (0.0009) [2023-12-26 20:01:52,961][105620] Updated weights for policy 1, policy_version 641404 (0.0006) [2023-12-26 20:01:53,021][105620] Updated weights for policy 1, policy_version 641414 (0.0008) [2023-12-26 20:01:53,480][105692] Updated weights for policy 0, policy_version 640783 (0.0010) [2023-12-26 20:01:53,537][105692] Updated weights for policy 0, policy_version 640793 (0.0010) [2023-12-26 20:01:53,594][105692] Updated weights for policy 0, policy_version 640803 (0.0010) [2023-12-26 20:01:53,854][105620] Updated weights for policy 1, policy_version 641424 (0.0008) [2023-12-26 20:01:53,916][105620] Updated weights for policy 1, policy_version 641434 (0.0005) [2023-12-26 20:01:53,968][105620] Updated weights for policy 1, policy_version 641444 (0.0009) [2023-12-26 20:01:54,320][105692] Updated weights for policy 0, policy_version 640813 (0.0010) [2023-12-26 20:01:54,376][105692] Updated weights for policy 0, policy_version 640823 (0.0010) [2023-12-26 20:01:54,431][105692] Updated weights for policy 0, policy_version 640833 (0.0011) [2023-12-26 20:01:54,660][105620] Updated weights for policy 1, policy_version 641454 (0.0010) [2023-12-26 20:01:54,716][105620] Updated weights for policy 1, policy_version 641464 (0.0011) [2023-12-26 20:01:54,766][105620] Updated weights for policy 1, policy_version 641474 (0.0009) [2023-12-26 20:01:55,167][105692] Updated weights for policy 0, policy_version 640843 (0.0008) [2023-12-26 20:01:55,221][105692] Updated weights for policy 0, policy_version 640853 (0.0009) [2023-12-26 20:01:55,271][105692] Updated weights for policy 0, policy_version 640863 (0.0009) [2023-12-26 20:01:55,558][105620] Updated weights for policy 1, policy_version 641484 (0.0006) [2023-12-26 20:01:55,614][105620] Updated weights for policy 1, policy_version 641494 (0.0005) [2023-12-26 20:01:55,665][105620] Updated weights for policy 1, policy_version 641504 (0.0005) [2023-12-26 20:01:56,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 328335360. Throughput: 0: 9812.1, 1: 9603.3. Samples: 328348276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:01:56,062][104569] Avg episode reward: [(0, '9265.327'), (1, '9169.329')] [2023-12-26 20:01:56,068][105692] Updated weights for policy 0, policy_version 640873 (0.0009) [2023-12-26 20:01:56,119][105692] Updated weights for policy 0, policy_version 640883 (0.0008) [2023-12-26 20:01:56,164][105692] Updated weights for policy 0, policy_version 640893 (0.0008) [2023-12-26 20:01:56,208][105692] Updated weights for policy 0, policy_version 640903 (0.0008) [2023-12-26 20:01:56,272][105620] Updated weights for policy 1, policy_version 641514 (0.0006) [2023-12-26 20:01:56,330][105620] Updated weights for policy 1, policy_version 641524 (0.0010) [2023-12-26 20:01:56,398][105620] Updated weights for policy 1, policy_version 641534 (0.0010) [2023-12-26 20:01:56,450][105620] Updated weights for policy 1, policy_version 641544 (0.0010) [2023-12-26 20:01:56,983][105692] Updated weights for policy 0, policy_version 640913 (0.0010) [2023-12-26 20:01:57,047][105692] Updated weights for policy 0, policy_version 640923 (0.0010) [2023-12-26 20:01:57,077][105620] Updated weights for policy 1, policy_version 641554 (0.0008) [2023-12-26 20:01:57,105][105692] Updated weights for policy 0, policy_version 640933 (0.0010) [2023-12-26 20:01:57,135][105620] Updated weights for policy 1, policy_version 641564 (0.0010) [2023-12-26 20:01:57,189][105620] Updated weights for policy 1, policy_version 641574 (0.0010) [2023-12-26 20:01:57,745][105692] Updated weights for policy 0, policy_version 640943 (0.0007) [2023-12-26 20:01:57,802][105692] Updated weights for policy 0, policy_version 640953 (0.0005) [2023-12-26 20:01:57,803][105620] Updated weights for policy 1, policy_version 641584 (0.0010) [2023-12-26 20:01:57,848][105692] Updated weights for policy 0, policy_version 640963 (0.0005) [2023-12-26 20:01:57,858][105620] Updated weights for policy 1, policy_version 641594 (0.0006) [2023-12-26 20:01:57,921][105620] Updated weights for policy 1, policy_version 641604 (0.0006) [2023-12-26 20:01:58,559][105692] Updated weights for policy 0, policy_version 640973 (0.0007) [2023-12-26 20:01:58,567][105620] Updated weights for policy 1, policy_version 641614 (0.0007) [2023-12-26 20:01:58,629][105692] Updated weights for policy 0, policy_version 640983 (0.0008) [2023-12-26 20:01:58,631][105620] Updated weights for policy 1, policy_version 641624 (0.0008) [2023-12-26 20:01:58,695][105692] Updated weights for policy 0, policy_version 640993 (0.0010) [2023-12-26 20:01:58,697][105620] Updated weights for policy 1, policy_version 641634 (0.0008) [2023-12-26 20:01:59,391][105692] Updated weights for policy 0, policy_version 641003 (0.0007) [2023-12-26 20:01:59,434][105692] Updated weights for policy 0, policy_version 641013 (0.0005) [2023-12-26 20:01:59,483][105692] Updated weights for policy 0, policy_version 641023 (0.0005) [2023-12-26 20:01:59,503][105620] Updated weights for policy 1, policy_version 641644 (0.0008) [2023-12-26 20:01:59,555][105620] Updated weights for policy 1, policy_version 641654 (0.0009) [2023-12-26 20:01:59,608][105620] Updated weights for policy 1, policy_version 641664 (0.0010) [2023-12-26 20:02:00,113][105692] Updated weights for policy 0, policy_version 641033 (0.0006) [2023-12-26 20:02:00,166][105692] Updated weights for policy 0, policy_version 641043 (0.0005) [2023-12-26 20:02:00,220][105692] Updated weights for policy 0, policy_version 641053 (0.0009) [2023-12-26 20:02:00,291][105692] Updated weights for policy 0, policy_version 641063 (0.0010) [2023-12-26 20:02:00,366][105620] Updated weights for policy 1, policy_version 641674 (0.0010) [2023-12-26 20:02:00,428][105620] Updated weights for policy 1, policy_version 641684 (0.0010) [2023-12-26 20:02:00,476][105620] Updated weights for policy 1, policy_version 641694 (0.0010) [2023-12-26 20:02:00,524][105620] Updated weights for policy 1, policy_version 641704 (0.0010) [2023-12-26 20:02:00,879][105692] Updated weights for policy 0, policy_version 641073 (0.0006) [2023-12-26 20:02:00,942][105692] Updated weights for policy 0, policy_version 641083 (0.0005) [2023-12-26 20:02:01,005][105692] Updated weights for policy 0, policy_version 641093 (0.0005) [2023-12-26 20:02:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 328441856. Throughput: 0: 9836.2, 1: 9676.6. Samples: 328408440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:02:01,063][104569] Avg episode reward: [(0, '7518.194'), (1, '9169.701')] [2023-12-26 20:02:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000641096_164151296.pth... [2023-12-26 20:02:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000641704_164290560.pth... [2023-12-26 20:02:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000640584_164003840.pth [2023-12-26 20:02:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000639912_163848192.pth [2023-12-26 20:02:01,186][105620] Updated weights for policy 1, policy_version 641714 (0.0007) [2023-12-26 20:02:01,248][105620] Updated weights for policy 1, policy_version 641724 (0.0008) [2023-12-26 20:02:01,317][105620] Updated weights for policy 1, policy_version 641734 (0.0009) [2023-12-26 20:02:01,658][105692] Updated weights for policy 0, policy_version 641103 (0.0008) [2023-12-26 20:02:01,719][105692] Updated weights for policy 0, policy_version 641113 (0.0008) [2023-12-26 20:02:01,783][105692] Updated weights for policy 0, policy_version 641123 (0.0008) [2023-12-26 20:02:02,032][105620] Updated weights for policy 1, policy_version 641744 (0.0010) [2023-12-26 20:02:02,091][105620] Updated weights for policy 1, policy_version 641754 (0.0010) [2023-12-26 20:02:02,160][105620] Updated weights for policy 1, policy_version 641764 (0.0008) [2023-12-26 20:02:02,551][105692] Updated weights for policy 0, policy_version 641133 (0.0008) [2023-12-26 20:02:02,600][105692] Updated weights for policy 0, policy_version 641143 (0.0009) [2023-12-26 20:02:02,647][105692] Updated weights for policy 0, policy_version 641153 (0.0005) [2023-12-26 20:02:02,817][105620] Updated weights for policy 1, policy_version 641774 (0.0006) [2023-12-26 20:02:02,879][105620] Updated weights for policy 1, policy_version 641784 (0.0006) [2023-12-26 20:02:02,938][105620] Updated weights for policy 1, policy_version 641794 (0.0008) [2023-12-26 20:02:03,336][105692] Updated weights for policy 0, policy_version 641163 (0.0006) [2023-12-26 20:02:03,382][105692] Updated weights for policy 0, policy_version 641173 (0.0008) [2023-12-26 20:02:03,429][105692] Updated weights for policy 0, policy_version 641183 (0.0009) [2023-12-26 20:02:03,579][105620] Updated weights for policy 1, policy_version 641804 (0.0009) [2023-12-26 20:02:03,636][105620] Updated weights for policy 1, policy_version 641814 (0.0009) [2023-12-26 20:02:03,693][105620] Updated weights for policy 1, policy_version 641824 (0.0009) [2023-12-26 20:02:04,180][105692] Updated weights for policy 0, policy_version 641193 (0.0009) [2023-12-26 20:02:04,246][105692] Updated weights for policy 0, policy_version 641203 (0.0009) [2023-12-26 20:02:04,310][105692] Updated weights for policy 0, policy_version 641213 (0.0008) [2023-12-26 20:02:04,341][105620] Updated weights for policy 1, policy_version 641834 (0.0009) [2023-12-26 20:02:04,371][105692] Updated weights for policy 0, policy_version 641223 (0.0007) [2023-12-26 20:02:04,403][105620] Updated weights for policy 1, policy_version 641844 (0.0010) [2023-12-26 20:02:04,460][105620] Updated weights for policy 1, policy_version 641854 (0.0010) [2023-12-26 20:02:04,516][105620] Updated weights for policy 1, policy_version 641864 (0.0010) [2023-12-26 20:02:05,133][105585] KL-divergence is very high: 287.1688 [2023-12-26 20:02:05,134][105692] Updated weights for policy 0, policy_version 641233 (0.0009) [2023-12-26 20:02:05,151][105585] KL-divergence is very high: 260.0380 [2023-12-26 20:02:05,164][105585] KL-divergence is very high: 149.5713 [2023-12-26 20:02:05,181][105585] KL-divergence is very high: 416.4877 [2023-12-26 20:02:05,191][105692] Updated weights for policy 0, policy_version 641243 (0.0009) [2023-12-26 20:02:05,191][105585] KL-divergence is very high: 110.6060 [2023-12-26 20:02:05,196][105585] KL-divergence is very high: 257.9876 [2023-12-26 20:02:05,197][105620] Updated weights for policy 1, policy_version 641874 (0.0005) [2023-12-26 20:02:05,206][105585] KL-divergence is very high: 106.2874 [2023-12-26 20:02:05,220][105585] KL-divergence is very high: 297.4397 [2023-12-26 20:02:05,235][105585] KL-divergence is very high: 149.5841 [2023-12-26 20:02:05,242][105692] Updated weights for policy 0, policy_version 641253 (0.0008) [2023-12-26 20:02:05,250][105620] Updated weights for policy 1, policy_version 641884 (0.0005) [2023-12-26 20:02:05,296][105620] Updated weights for policy 1, policy_version 641894 (0.0005) [2023-12-26 20:02:05,837][105620] Updated weights for policy 1, policy_version 641904 (0.0005) [2023-12-26 20:02:05,896][105620] Updated weights for policy 1, policy_version 641914 (0.0008) [2023-12-26 20:02:05,959][105620] Updated weights for policy 1, policy_version 641924 (0.0009) [2023-12-26 20:02:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 328540160. Throughput: 0: 9868.5, 1: 9651.9. Samples: 328528604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:02:06,062][104569] Avg episode reward: [(0, '7687.104'), (1, '9260.875')] [2023-12-26 20:02:06,100][105692] Updated weights for policy 0, policy_version 641263 (0.0008) [2023-12-26 20:02:06,169][105692] Updated weights for policy 0, policy_version 641273 (0.0008) [2023-12-26 20:02:06,227][105692] Updated weights for policy 0, policy_version 641283 (0.0009) [2023-12-26 20:02:06,627][105620] Updated weights for policy 1, policy_version 641934 (0.0009) [2023-12-26 20:02:06,682][105620] Updated weights for policy 1, policy_version 641944 (0.0009) [2023-12-26 20:02:06,740][105620] Updated weights for policy 1, policy_version 641954 (0.0009) [2023-12-26 20:02:06,984][105692] Updated weights for policy 0, policy_version 641293 (0.0010) [2023-12-26 20:02:07,046][105692] Updated weights for policy 0, policy_version 641303 (0.0009) [2023-12-26 20:02:07,104][105692] Updated weights for policy 0, policy_version 641313 (0.0009) [2023-12-26 20:02:07,488][105620] Updated weights for policy 1, policy_version 641964 (0.0009) [2023-12-26 20:02:07,549][105620] Updated weights for policy 1, policy_version 641974 (0.0009) [2023-12-26 20:02:07,614][105620] Updated weights for policy 1, policy_version 641984 (0.0008) [2023-12-26 20:02:07,915][105692] Updated weights for policy 0, policy_version 641323 (0.0009) [2023-12-26 20:02:07,976][105692] Updated weights for policy 0, policy_version 641333 (0.0010) [2023-12-26 20:02:08,037][105692] Updated weights for policy 0, policy_version 641343 (0.0009) [2023-12-26 20:02:08,289][105620] Updated weights for policy 1, policy_version 641994 (0.0010) [2023-12-26 20:02:08,352][105620] Updated weights for policy 1, policy_version 642004 (0.0008) [2023-12-26 20:02:08,420][105620] Updated weights for policy 1, policy_version 642014 (0.0006) [2023-12-26 20:02:08,485][105620] Updated weights for policy 1, policy_version 642024 (0.0006) [2023-12-26 20:02:08,863][105692] Updated weights for policy 0, policy_version 641353 (0.0009) [2023-12-26 20:02:08,920][105692] Updated weights for policy 0, policy_version 641363 (0.0009) [2023-12-26 20:02:08,970][105692] Updated weights for policy 0, policy_version 641373 (0.0009) [2023-12-26 20:02:09,024][105692] Updated weights for policy 0, policy_version 641383 (0.0008) [2023-12-26 20:02:09,099][105620] Updated weights for policy 1, policy_version 642034 (0.0005) [2023-12-26 20:02:09,156][105620] Updated weights for policy 1, policy_version 642044 (0.0005) [2023-12-26 20:02:09,220][105620] Updated weights for policy 1, policy_version 642054 (0.0006) [2023-12-26 20:02:09,860][105692] Updated weights for policy 0, policy_version 641393 (0.0009) [2023-12-26 20:02:09,927][105692] Updated weights for policy 0, policy_version 641403 (0.0008) [2023-12-26 20:02:09,946][105620] Updated weights for policy 1, policy_version 642064 (0.0010) [2023-12-26 20:02:09,991][105692] Updated weights for policy 0, policy_version 641413 (0.0008) [2023-12-26 20:02:09,998][105620] Updated weights for policy 1, policy_version 642074 (0.0009) [2023-12-26 20:02:10,055][105620] Updated weights for policy 1, policy_version 642084 (0.0009) [2023-12-26 20:02:10,730][105620] Updated weights for policy 1, policy_version 642094 (0.0010) [2023-12-26 20:02:10,776][105692] Updated weights for policy 0, policy_version 641423 (0.0007) [2023-12-26 20:02:10,779][105620] Updated weights for policy 1, policy_version 642104 (0.0007) [2023-12-26 20:02:10,825][105692] Updated weights for policy 0, policy_version 641433 (0.0007) [2023-12-26 20:02:10,832][105620] Updated weights for policy 1, policy_version 642114 (0.0006) [2023-12-26 20:02:10,883][105692] Updated weights for policy 0, policy_version 641443 (0.0008) [2023-12-26 20:02:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 328638464. Throughput: 0: 9728.2, 1: 9819.0. Samples: 328642748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:02:11,062][104569] Avg episode reward: [(0, '7774.371'), (1, '9267.095')] [2023-12-26 20:02:11,586][105620] Updated weights for policy 1, policy_version 642124 (0.0007) [2023-12-26 20:02:11,654][105620] Updated weights for policy 1, policy_version 642134 (0.0007) [2023-12-26 20:02:11,662][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000010 [2023-12-26 20:02:11,683][105692] Updated weights for policy 0, policy_version 641453 (0.0006) [2023-12-26 20:02:11,755][105692] Updated weights for policy 0, policy_version 641463 (0.0007) [2023-12-26 20:02:11,815][105692] Updated weights for policy 0, policy_version 641473 (0.0008) [2023-12-26 20:02:12,474][105620] Updated weights for policy 1, policy_version 642144 (0.0008) [2023-12-26 20:02:12,536][105620] Updated weights for policy 1, policy_version 642154 (0.0008) [2023-12-26 20:02:12,568][105692] Updated weights for policy 0, policy_version 641483 (0.0008) [2023-12-26 20:02:12,590][105620] Updated weights for policy 1, policy_version 642164 (0.0008) [2023-12-26 20:02:12,627][105692] Updated weights for policy 0, policy_version 641493 (0.0006) [2023-12-26 20:02:12,684][105692] Updated weights for policy 0, policy_version 641503 (0.0006) [2023-12-26 20:02:13,318][105620] Updated weights for policy 1, policy_version 642174 (0.0010) [2023-12-26 20:02:13,346][105692] Updated weights for policy 0, policy_version 641513 (0.0005) [2023-12-26 20:02:13,380][105620] Updated weights for policy 1, policy_version 642184 (0.0010) [2023-12-26 20:02:13,406][105692] Updated weights for policy 0, policy_version 641523 (0.0005) [2023-12-26 20:02:13,431][105620] Updated weights for policy 1, policy_version 642194 (0.0010) [2023-12-26 20:02:13,461][105692] Updated weights for policy 0, policy_version 641533 (0.0006) [2023-12-26 20:02:13,513][105692] Updated weights for policy 0, policy_version 641543 (0.0008) [2023-12-26 20:02:14,180][105620] Updated weights for policy 1, policy_version 642204 (0.0010) [2023-12-26 20:02:14,239][105620] Updated weights for policy 1, policy_version 642214 (0.0010) [2023-12-26 20:02:14,282][105692] Updated weights for policy 0, policy_version 641553 (0.0006) [2023-12-26 20:02:14,299][105620] Updated weights for policy 1, policy_version 642224 (0.0010) [2023-12-26 20:02:14,339][105692] Updated weights for policy 0, policy_version 641563 (0.0007) [2023-12-26 20:02:14,404][105692] Updated weights for policy 0, policy_version 641573 (0.0008) [2023-12-26 20:02:14,953][105620] Updated weights for policy 1, policy_version 642234 (0.0007) [2023-12-26 20:02:15,017][105620] Updated weights for policy 1, policy_version 642244 (0.0008) [2023-12-26 20:02:15,073][105620] Updated weights for policy 1, policy_version 642254 (0.0009) [2023-12-26 20:02:15,124][105620] Updated weights for policy 1, policy_version 642264 (0.0009) [2023-12-26 20:02:15,184][105692] Updated weights for policy 0, policy_version 641583 (0.0008) [2023-12-26 20:02:15,251][105692] Updated weights for policy 0, policy_version 641593 (0.0009) [2023-12-26 20:02:15,318][105692] Updated weights for policy 0, policy_version 641603 (0.0009) [2023-12-26 20:02:15,730][105620] Updated weights for policy 1, policy_version 642274 (0.0006) [2023-12-26 20:02:15,787][105620] Updated weights for policy 1, policy_version 642284 (0.0009) [2023-12-26 20:02:15,844][105620] Updated weights for policy 1, policy_version 642294 (0.0005) [2023-12-26 20:02:16,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 328728576. Throughput: 0: 9706.6, 1: 9815.9. Samples: 328699288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:02:16,063][104569] Avg episode reward: [(0, '8997.715'), (1, '9181.962')] [2023-12-26 20:02:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000641608_164282368.pth... [2023-12-26 20:02:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000642296_164446208.pth... [2023-12-26 20:02:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000640520_164003840.pth [2023-12-26 20:02:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000641128_164143104.pth [2023-12-26 20:02:16,131][105692] Updated weights for policy 0, policy_version 641613 (0.0010) [2023-12-26 20:02:16,188][105692] Updated weights for policy 0, policy_version 641623 (0.0009) [2023-12-26 20:02:16,244][105692] Updated weights for policy 0, policy_version 641633 (0.0008) [2023-12-26 20:02:16,510][105620] Updated weights for policy 1, policy_version 642304 (0.0006) [2023-12-26 20:02:16,578][105620] Updated weights for policy 1, policy_version 642314 (0.0005) [2023-12-26 20:02:16,642][105620] Updated weights for policy 1, policy_version 642324 (0.0007) [2023-12-26 20:02:16,930][105692] Updated weights for policy 0, policy_version 641643 (0.0008) [2023-12-26 20:02:16,982][105692] Updated weights for policy 0, policy_version 641653 (0.0010) [2023-12-26 20:02:17,030][105692] Updated weights for policy 0, policy_version 641663 (0.0010) [2023-12-26 20:02:17,294][105620] Updated weights for policy 1, policy_version 642334 (0.0010) [2023-12-26 20:02:17,338][105620] Updated weights for policy 1, policy_version 642344 (0.0010) [2023-12-26 20:02:17,393][105620] Updated weights for policy 1, policy_version 642354 (0.0010) [2023-12-26 20:02:17,775][105692] Updated weights for policy 0, policy_version 641673 (0.0010) [2023-12-26 20:02:17,829][105692] Updated weights for policy 0, policy_version 641683 (0.0010) [2023-12-26 20:02:17,890][105692] Updated weights for policy 0, policy_version 641693 (0.0010) [2023-12-26 20:02:17,944][105692] Updated weights for policy 0, policy_version 641703 (0.0010) [2023-12-26 20:02:18,147][105620] Updated weights for policy 1, policy_version 642364 (0.0010) [2023-12-26 20:02:18,191][105620] Updated weights for policy 1, policy_version 642374 (0.0010) [2023-12-26 20:02:18,246][105620] Updated weights for policy 1, policy_version 642384 (0.0010) [2023-12-26 20:02:18,685][105692] Updated weights for policy 0, policy_version 641713 (0.0010) [2023-12-26 20:02:18,753][105692] Updated weights for policy 0, policy_version 641723 (0.0008) [2023-12-26 20:02:18,827][105692] Updated weights for policy 0, policy_version 641733 (0.0009) [2023-12-26 20:02:18,954][105620] Updated weights for policy 1, policy_version 642394 (0.0009) [2023-12-26 20:02:19,012][105620] Updated weights for policy 1, policy_version 642404 (0.0009) [2023-12-26 20:02:19,073][105620] Updated weights for policy 1, policy_version 642414 (0.0009) [2023-12-26 20:02:19,134][105620] Updated weights for policy 1, policy_version 642424 (0.0009) [2023-12-26 20:02:19,594][105692] Updated weights for policy 0, policy_version 641743 (0.0009) [2023-12-26 20:02:19,650][105692] Updated weights for policy 0, policy_version 641753 (0.0009) [2023-12-26 20:02:19,718][105692] Updated weights for policy 0, policy_version 641763 (0.0009) [2023-12-26 20:02:19,917][105620] Updated weights for policy 1, policy_version 642434 (0.0009) [2023-12-26 20:02:19,985][105620] Updated weights for policy 1, policy_version 642444 (0.0008) [2023-12-26 20:02:20,047][105620] Updated weights for policy 1, policy_version 642454 (0.0009) [2023-12-26 20:02:20,511][105692] Updated weights for policy 0, policy_version 641773 (0.0009) [2023-12-26 20:02:20,559][105692] Updated weights for policy 0, policy_version 641783 (0.0009) [2023-12-26 20:02:20,631][105692] Updated weights for policy 0, policy_version 641793 (0.0009) [2023-12-26 20:02:20,763][105620] Updated weights for policy 1, policy_version 642464 (0.0009) [2023-12-26 20:02:20,821][105620] Updated weights for policy 1, policy_version 642474 (0.0009) [2023-12-26 20:02:20,868][105586] KL-divergence is very high: 113.2001 [2023-12-26 20:02:20,880][105620] Updated weights for policy 1, policy_version 642484 (0.0009) [2023-12-26 20:02:20,886][105586] KL-divergence is very high: 154.7582 [2023-12-26 20:02:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 328826880. Throughput: 0: 9638.8, 1: 9879.6. Samples: 328815300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:02:21,062][104569] Avg episode reward: [(0, '9087.282'), (1, '8846.543')] [2023-12-26 20:02:21,467][105692] Updated weights for policy 0, policy_version 641803 (0.0009) [2023-12-26 20:02:21,520][105692] Updated weights for policy 0, policy_version 641814 (0.0010) [2023-12-26 20:02:21,544][105620] Updated weights for policy 1, policy_version 642494 (0.0007) [2023-12-26 20:02:21,582][105692] Updated weights for policy 0, policy_version 641824 (0.0008) [2023-12-26 20:02:21,608][105620] Updated weights for policy 1, policy_version 642504 (0.0007) [2023-12-26 20:02:21,671][105620] Updated weights for policy 1, policy_version 642514 (0.0009) [2023-12-26 20:02:22,387][105692] Updated weights for policy 0, policy_version 641834 (0.0008) [2023-12-26 20:02:22,424][105620] Updated weights for policy 1, policy_version 642524 (0.0009) [2023-12-26 20:02:22,454][105692] Updated weights for policy 0, policy_version 641844 (0.0009) [2023-12-26 20:02:22,481][105620] Updated weights for policy 1, policy_version 642534 (0.0009) [2023-12-26 20:02:22,518][105692] Updated weights for policy 0, policy_version 641854 (0.0010) [2023-12-26 20:02:22,540][105620] Updated weights for policy 1, policy_version 642544 (0.0008) [2023-12-26 20:02:22,570][105692] Updated weights for policy 0, policy_version 641864 (0.0009) [2023-12-26 20:02:23,301][105620] Updated weights for policy 1, policy_version 642554 (0.0007) [2023-12-26 20:02:23,308][105692] Updated weights for policy 0, policy_version 641874 (0.0009) [2023-12-26 20:02:23,352][105620] Updated weights for policy 1, policy_version 642564 (0.0006) [2023-12-26 20:02:23,366][105692] Updated weights for policy 0, policy_version 641884 (0.0008) [2023-12-26 20:02:23,409][105620] Updated weights for policy 1, policy_version 642574 (0.0007) [2023-12-26 20:02:23,427][105692] Updated weights for policy 0, policy_version 641894 (0.0006) [2023-12-26 20:02:23,468][105620] Updated weights for policy 1, policy_version 642584 (0.0008) [2023-12-26 20:02:24,053][105692] Updated weights for policy 0, policy_version 641904 (0.0008) [2023-12-26 20:02:24,115][105692] Updated weights for policy 0, policy_version 641914 (0.0010) [2023-12-26 20:02:24,173][105692] Updated weights for policy 0, policy_version 641925 (0.0010) [2023-12-26 20:02:24,222][105620] Updated weights for policy 1, policy_version 642594 (0.0006) [2023-12-26 20:02:24,276][105620] Updated weights for policy 1, policy_version 642604 (0.0009) [2023-12-26 20:02:24,330][105620] Updated weights for policy 1, policy_version 642614 (0.0009) [2023-12-26 20:02:24,884][105692] Updated weights for policy 0, policy_version 641935 (0.0006) [2023-12-26 20:02:24,936][105692] Updated weights for policy 0, policy_version 641945 (0.0008) [2023-12-26 20:02:24,990][105692] Updated weights for policy 0, policy_version 641955 (0.0011) [2023-12-26 20:02:25,126][105620] Updated weights for policy 1, policy_version 642624 (0.0009) [2023-12-26 20:02:25,187][105620] Updated weights for policy 1, policy_version 642634 (0.0008) [2023-12-26 20:02:25,243][105620] Updated weights for policy 1, policy_version 642644 (0.0008) [2023-12-26 20:02:25,727][105692] Updated weights for policy 0, policy_version 641965 (0.0010) [2023-12-26 20:02:25,796][105692] Updated weights for policy 0, policy_version 641975 (0.0010) [2023-12-26 20:02:25,851][105692] Updated weights for policy 0, policy_version 641985 (0.0010) [2023-12-26 20:02:25,993][105620] Updated weights for policy 1, policy_version 642654 (0.0008) [2023-12-26 20:02:26,056][105620] Updated weights for policy 1, policy_version 642664 (0.0008) [2023-12-26 20:02:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 328916992. Throughput: 0: 9518.3, 1: 9871.3. Samples: 328927556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:02:26,063][104569] Avg episode reward: [(0, '9086.877'), (1, '8808.561')] [2023-12-26 20:02:26,109][105620] Updated weights for policy 1, policy_version 642674 (0.0009) [2023-12-26 20:02:26,497][105692] Updated weights for policy 0, policy_version 641995 (0.0011) [2023-12-26 20:02:26,553][105692] Updated weights for policy 0, policy_version 642005 (0.0011) [2023-12-26 20:02:26,602][105692] Updated weights for policy 0, policy_version 642015 (0.0010) [2023-12-26 20:02:26,935][105620] Updated weights for policy 1, policy_version 642685 (0.0010) [2023-12-26 20:02:26,982][105620] Updated weights for policy 1, policy_version 642695 (0.0009) [2023-12-26 20:02:27,035][105620] Updated weights for policy 1, policy_version 642705 (0.0009) [2023-12-26 20:02:27,293][105692] Updated weights for policy 0, policy_version 642025 (0.0010) [2023-12-26 20:02:27,358][105692] Updated weights for policy 0, policy_version 642035 (0.0009) [2023-12-26 20:02:27,409][105692] Updated weights for policy 0, policy_version 642046 (0.0009) [2023-12-26 20:02:27,471][105692] Updated weights for policy 0, policy_version 642056 (0.0009) [2023-12-26 20:02:27,716][105620] Updated weights for policy 1, policy_version 642715 (0.0007) [2023-12-26 20:02:27,771][105620] Updated weights for policy 1, policy_version 642725 (0.0005) [2023-12-26 20:02:27,824][105620] Updated weights for policy 1, policy_version 642735 (0.0005) [2023-12-26 20:02:28,248][105692] Updated weights for policy 0, policy_version 642066 (0.0009) [2023-12-26 20:02:28,304][105692] Updated weights for policy 0, policy_version 642076 (0.0010) [2023-12-26 20:02:28,365][105692] Updated weights for policy 0, policy_version 642086 (0.0009) [2023-12-26 20:02:28,383][105620] Updated weights for policy 1, policy_version 642745 (0.0005) [2023-12-26 20:02:28,451][105620] Updated weights for policy 1, policy_version 642755 (0.0008) [2023-12-26 20:02:28,517][105620] Updated weights for policy 1, policy_version 642765 (0.0009) [2023-12-26 20:02:28,581][105620] Updated weights for policy 1, policy_version 642775 (0.0009) [2023-12-26 20:02:29,161][105692] Updated weights for policy 0, policy_version 642096 (0.0008) [2023-12-26 20:02:29,209][105692] Updated weights for policy 0, policy_version 642106 (0.0009) [2023-12-26 20:02:29,264][105620] Updated weights for policy 1, policy_version 642785 (0.0008) [2023-12-26 20:02:29,269][105692] Updated weights for policy 0, policy_version 642116 (0.0008) [2023-12-26 20:02:29,328][105620] Updated weights for policy 1, policy_version 642795 (0.0007) [2023-12-26 20:02:29,389][105620] Updated weights for policy 1, policy_version 642805 (0.0006) [2023-12-26 20:02:29,980][105620] Updated weights for policy 1, policy_version 642815 (0.0007) [2023-12-26 20:02:30,033][105620] Updated weights for policy 1, policy_version 642825 (0.0010) [2023-12-26 20:02:30,091][105620] Updated weights for policy 1, policy_version 642835 (0.0010) [2023-12-26 20:02:30,138][105692] Updated weights for policy 0, policy_version 642126 (0.0009) [2023-12-26 20:02:30,189][105692] Updated weights for policy 0, policy_version 642136 (0.0008) [2023-12-26 20:02:30,250][105692] Updated weights for policy 0, policy_version 642146 (0.0009) [2023-12-26 20:02:30,709][105620] Updated weights for policy 1, policy_version 642845 (0.0009) [2023-12-26 20:02:30,758][105620] Updated weights for policy 1, policy_version 642855 (0.0009) [2023-12-26 20:02:30,802][105620] Updated weights for policy 1, policy_version 642865 (0.0007) [2023-12-26 20:02:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 329015296. Throughput: 0: 9509.7, 1: 9924.8. Samples: 328986584. Policy #0 lag: (min: 7.0, avg: 14.9, max: 39.0) [2023-12-26 20:02:31,062][104569] Avg episode reward: [(0, '9268.832'), (1, '8756.947')] [2023-12-26 20:02:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000642152_164421632.pth... [2023-12-26 20:02:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000642872_164593664.pth... [2023-12-26 20:02:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000641096_164151296.pth [2023-12-26 20:02:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000641704_164290560.pth [2023-12-26 20:02:31,111][105692] Updated weights for policy 0, policy_version 642156 (0.0009) [2023-12-26 20:02:31,178][105692] Updated weights for policy 0, policy_version 642166 (0.0009) [2023-12-26 20:02:31,231][105692] Updated weights for policy 0, policy_version 642176 (0.0008) [2023-12-26 20:02:31,449][105620] Updated weights for policy 1, policy_version 642875 (0.0007) [2023-12-26 20:02:31,510][105620] Updated weights for policy 1, policy_version 642885 (0.0010) [2023-12-26 20:02:31,569][105620] Updated weights for policy 1, policy_version 642895 (0.0008) [2023-12-26 20:02:32,013][105692] Updated weights for policy 0, policy_version 642186 (0.0009) [2023-12-26 20:02:32,069][105692] Updated weights for policy 0, policy_version 642196 (0.0008) [2023-12-26 20:02:32,120][105692] Updated weights for policy 0, policy_version 642206 (0.0009) [2023-12-26 20:02:32,198][105620] Updated weights for policy 1, policy_version 642905 (0.0008) [2023-12-26 20:02:32,263][105620] Updated weights for policy 1, policy_version 642915 (0.0008) [2023-12-26 20:02:32,317][105620] Updated weights for policy 1, policy_version 642925 (0.0009) [2023-12-26 20:02:32,384][105620] Updated weights for policy 1, policy_version 642935 (0.0008) [2023-12-26 20:02:32,963][105620] Updated weights for policy 1, policy_version 642945 (0.0006) [2023-12-26 20:02:33,018][105692] Updated weights for policy 0, policy_version 642218 (0.0010) [2023-12-26 20:02:33,020][105620] Updated weights for policy 1, policy_version 642955 (0.0008) [2023-12-26 20:02:33,068][105692] Updated weights for policy 0, policy_version 642228 (0.0008) [2023-12-26 20:02:33,072][105620] Updated weights for policy 1, policy_version 642965 (0.0006) [2023-12-26 20:02:33,128][105692] Updated weights for policy 0, policy_version 642238 (0.0010) [2023-12-26 20:02:33,176][105692] Updated weights for policy 0, policy_version 642248 (0.0009) [2023-12-26 20:02:33,619][105620] Updated weights for policy 1, policy_version 642975 (0.0005) [2023-12-26 20:02:33,677][105620] Updated weights for policy 1, policy_version 642985 (0.0005) [2023-12-26 20:02:33,736][105620] Updated weights for policy 1, policy_version 642995 (0.0007) [2023-12-26 20:02:34,069][105692] Updated weights for policy 0, policy_version 642258 (0.0009) [2023-12-26 20:02:34,130][105692] Updated weights for policy 0, policy_version 642268 (0.0009) [2023-12-26 20:02:34,190][105692] Updated weights for policy 0, policy_version 642278 (0.0009) [2023-12-26 20:02:34,366][105620] Updated weights for policy 1, policy_version 643005 (0.0008) [2023-12-26 20:02:34,435][105620] Updated weights for policy 1, policy_version 643015 (0.0008) [2023-12-26 20:02:34,492][105620] Updated weights for policy 1, policy_version 643025 (0.0008) [2023-12-26 20:02:34,977][105692] Updated weights for policy 0, policy_version 642288 (0.0008) [2023-12-26 20:02:34,984][105585] KL-divergence is very high: 226.6126 [2023-12-26 20:02:35,030][105585] KL-divergence is very high: 365.5359 [2023-12-26 20:02:35,035][105692] Updated weights for policy 0, policy_version 642298 (0.0009) [2023-12-26 20:02:35,076][105585] KL-divergence is very high: 332.8659 [2023-12-26 20:02:35,094][105692] Updated weights for policy 0, policy_version 642308 (0.0008) [2023-12-26 20:02:35,253][105620] Updated weights for policy 1, policy_version 643035 (0.0009) [2023-12-26 20:02:35,307][105620] Updated weights for policy 1, policy_version 643045 (0.0007) [2023-12-26 20:02:35,361][105620] Updated weights for policy 1, policy_version 643055 (0.0006) [2023-12-26 20:02:35,945][105620] Updated weights for policy 1, policy_version 643065 (0.0008) [2023-12-26 20:02:35,963][105692] Updated weights for policy 0, policy_version 642318 (0.0007) [2023-12-26 20:02:35,999][105620] Updated weights for policy 1, policy_version 643075 (0.0006) [2023-12-26 20:02:36,021][105692] Updated weights for policy 0, policy_version 642328 (0.0008) [2023-12-26 20:02:36,047][105620] Updated weights for policy 1, policy_version 643085 (0.0005) [2023-12-26 20:02:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 329105408. Throughput: 0: 9320.0, 1: 10001.1. Samples: 329102808. Policy #0 lag: (min: 7.0, avg: 14.9, max: 39.0) [2023-12-26 20:02:36,063][104569] Avg episode reward: [(0, '9267.409'), (1, '9124.215')] [2023-12-26 20:02:36,074][105692] Updated weights for policy 0, policy_version 642338 (0.0007) [2023-12-26 20:02:36,105][105620] Updated weights for policy 1, policy_version 643095 (0.0007) [2023-12-26 20:02:36,727][105620] Updated weights for policy 1, policy_version 643105 (0.0009) [2023-12-26 20:02:36,790][105620] Updated weights for policy 1, policy_version 643115 (0.0009) [2023-12-26 20:02:36,852][105620] Updated weights for policy 1, policy_version 643125 (0.0009) [2023-12-26 20:02:36,901][105692] Updated weights for policy 0, policy_version 642348 (0.0009) [2023-12-26 20:02:36,960][105692] Updated weights for policy 0, policy_version 642358 (0.0009) [2023-12-26 20:02:37,021][105692] Updated weights for policy 0, policy_version 642368 (0.0009) [2023-12-26 20:02:37,489][105620] Updated weights for policy 1, policy_version 643135 (0.0009) [2023-12-26 20:02:37,539][105620] Updated weights for policy 1, policy_version 643145 (0.0008) [2023-12-26 20:02:37,599][105620] Updated weights for policy 1, policy_version 643155 (0.0006) [2023-12-26 20:02:37,867][105692] Updated weights for policy 0, policy_version 642378 (0.0009) [2023-12-26 20:02:37,917][105692] Updated weights for policy 0, policy_version 642388 (0.0009) [2023-12-26 20:02:37,968][105692] Updated weights for policy 0, policy_version 642398 (0.0009) [2023-12-26 20:02:38,014][105692] Updated weights for policy 0, policy_version 642408 (0.0008) [2023-12-26 20:02:38,214][105620] Updated weights for policy 1, policy_version 643165 (0.0008) [2023-12-26 20:02:38,264][105620] Updated weights for policy 1, policy_version 643175 (0.0010) [2023-12-26 20:02:38,309][105620] Updated weights for policy 1, policy_version 643185 (0.0010) [2023-12-26 20:02:38,812][105692] Updated weights for policy 0, policy_version 642418 (0.0008) [2023-12-26 20:02:38,868][105692] Updated weights for policy 0, policy_version 642428 (0.0008) [2023-12-26 20:02:38,925][105692] Updated weights for policy 0, policy_version 642438 (0.0008) [2023-12-26 20:02:39,071][105620] Updated weights for policy 1, policy_version 643195 (0.0009) [2023-12-26 20:02:39,117][105620] Updated weights for policy 1, policy_version 643205 (0.0008) [2023-12-26 20:02:39,182][105620] Updated weights for policy 1, policy_version 643215 (0.0010) [2023-12-26 20:02:39,743][105692] Updated weights for policy 0, policy_version 642448 (0.0010) [2023-12-26 20:02:39,802][105692] Updated weights for policy 0, policy_version 642458 (0.0011) [2023-12-26 20:02:39,868][105692] Updated weights for policy 0, policy_version 642469 (0.0009) [2023-12-26 20:02:39,966][105620] Updated weights for policy 1, policy_version 643225 (0.0010) [2023-12-26 20:02:40,026][105620] Updated weights for policy 1, policy_version 643235 (0.0011) [2023-12-26 20:02:40,086][105620] Updated weights for policy 1, policy_version 643245 (0.0011) [2023-12-26 20:02:40,146][105620] Updated weights for policy 1, policy_version 643255 (0.0011) [2023-12-26 20:02:40,557][105692] Updated weights for policy 0, policy_version 642479 (0.0009) [2023-12-26 20:02:40,613][105692] Updated weights for policy 0, policy_version 642489 (0.0011) [2023-12-26 20:02:40,675][105692] Updated weights for policy 0, policy_version 642499 (0.0010) [2023-12-26 20:02:40,920][105620] Updated weights for policy 1, policy_version 643265 (0.0010) [2023-12-26 20:02:40,973][105620] Updated weights for policy 1, policy_version 643275 (0.0009) [2023-12-26 20:02:41,025][105620] Updated weights for policy 1, policy_version 643285 (0.0007) [2023-12-26 20:02:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 329211904. Throughput: 0: 9215.8, 1: 10091.9. Samples: 329217124. Policy #0 lag: (min: 7.0, avg: 14.9, max: 39.0) [2023-12-26 20:02:41,062][104569] Avg episode reward: [(0, '8203.372'), (1, '9261.127')] [2023-12-26 20:02:41,405][105692] Updated weights for policy 0, policy_version 642509 (0.0008) [2023-12-26 20:02:41,468][105692] Updated weights for policy 0, policy_version 642519 (0.0006) [2023-12-26 20:02:41,536][105692] Updated weights for policy 0, policy_version 642529 (0.0008) [2023-12-26 20:02:41,757][105620] Updated weights for policy 1, policy_version 643295 (0.0008) [2023-12-26 20:02:41,825][105620] Updated weights for policy 1, policy_version 643305 (0.0007) [2023-12-26 20:02:41,885][105620] Updated weights for policy 1, policy_version 643315 (0.0006) [2023-12-26 20:02:42,269][105692] Updated weights for policy 0, policy_version 642539 (0.0008) [2023-12-26 20:02:42,330][105692] Updated weights for policy 0, policy_version 642549 (0.0011) [2023-12-26 20:02:42,401][105692] Updated weights for policy 0, policy_version 642559 (0.0010) [2023-12-26 20:02:42,518][105620] Updated weights for policy 1, policy_version 643325 (0.0007) [2023-12-26 20:02:42,578][105620] Updated weights for policy 1, policy_version 643335 (0.0006) [2023-12-26 20:02:42,579][105586] KL-divergence is very high: 166.5862 [2023-12-26 20:02:42,631][105586] KL-divergence is very high: 284.1122 [2023-12-26 20:02:42,644][105620] Updated weights for policy 1, policy_version 643345 (0.0007) [2023-12-26 20:02:42,684][105586] KL-divergence is very high: 307.9706 [2023-12-26 20:02:43,134][105692] Updated weights for policy 0, policy_version 642569 (0.0010) [2023-12-26 20:02:43,201][105692] Updated weights for policy 0, policy_version 642579 (0.0007) [2023-12-26 20:02:43,267][105692] Updated weights for policy 0, policy_version 642589 (0.0008) [2023-12-26 20:02:43,311][105620] Updated weights for policy 1, policy_version 643355 (0.0008) [2023-12-26 20:02:43,327][105692] Updated weights for policy 0, policy_version 642599 (0.0007) [2023-12-26 20:02:43,370][105620] Updated weights for policy 1, policy_version 643365 (0.0009) [2023-12-26 20:02:43,422][105620] Updated weights for policy 1, policy_version 643375 (0.0009) [2023-12-26 20:02:44,036][105692] Updated weights for policy 0, policy_version 642609 (0.0009) [2023-12-26 20:02:44,101][105692] Updated weights for policy 0, policy_version 642619 (0.0009) [2023-12-26 20:02:44,157][105692] Updated weights for policy 0, policy_version 642629 (0.0006) [2023-12-26 20:02:44,160][105620] Updated weights for policy 1, policy_version 643385 (0.0007) [2023-12-26 20:02:44,217][105620] Updated weights for policy 1, policy_version 643395 (0.0009) [2023-12-26 20:02:44,270][105620] Updated weights for policy 1, policy_version 643405 (0.0008) [2023-12-26 20:02:44,319][105620] Updated weights for policy 1, policy_version 643415 (0.0009) [2023-12-26 20:02:44,821][105692] Updated weights for policy 0, policy_version 642639 (0.0009) [2023-12-26 20:02:44,880][105692] Updated weights for policy 0, policy_version 642649 (0.0010) [2023-12-26 20:02:44,941][105692] Updated weights for policy 0, policy_version 642659 (0.0009) [2023-12-26 20:02:44,987][105620] Updated weights for policy 1, policy_version 643425 (0.0007) [2023-12-26 20:02:45,044][105620] Updated weights for policy 1, policy_version 643435 (0.0009) [2023-12-26 20:02:45,107][105620] Updated weights for policy 1, policy_version 643445 (0.0009) [2023-12-26 20:02:45,725][105620] Updated weights for policy 1, policy_version 643455 (0.0008) [2023-12-26 20:02:45,786][105620] Updated weights for policy 1, policy_version 643465 (0.0008) [2023-12-26 20:02:45,788][105692] Updated weights for policy 0, policy_version 642669 (0.0007) [2023-12-26 20:02:45,839][105620] Updated weights for policy 1, policy_version 643475 (0.0006) [2023-12-26 20:02:45,845][105692] Updated weights for policy 0, policy_version 642679 (0.0009) [2023-12-26 20:02:45,904][105692] Updated weights for policy 0, policy_version 642689 (0.0009) [2023-12-26 20:02:46,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 329310208. Throughput: 0: 9192.6, 1: 10073.3. Samples: 329275404. Policy #0 lag: (min: 7.0, avg: 14.9, max: 39.0) [2023-12-26 20:02:46,062][104569] Avg episode reward: [(0, '6515.311'), (1, '9082.824')] [2023-12-26 20:02:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000642696_164560896.pth... [2023-12-26 20:02:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000643480_164749312.pth... [2023-12-26 20:02:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000642296_164446208.pth [2023-12-26 20:02:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000641608_164282368.pth [2023-12-26 20:02:46,524][105620] Updated weights for policy 1, policy_version 643485 (0.0006) [2023-12-26 20:02:46,580][105620] Updated weights for policy 1, policy_version 643495 (0.0006) [2023-12-26 20:02:46,627][105620] Updated weights for policy 1, policy_version 643505 (0.0008) [2023-12-26 20:02:46,704][105692] Updated weights for policy 0, policy_version 642699 (0.0009) [2023-12-26 20:02:46,776][105692] Updated weights for policy 0, policy_version 642709 (0.0005) [2023-12-26 20:02:46,835][105692] Updated weights for policy 0, policy_version 642719 (0.0006) [2023-12-26 20:02:47,209][105620] Updated weights for policy 1, policy_version 643515 (0.0006) [2023-12-26 20:02:47,261][105620] Updated weights for policy 1, policy_version 643525 (0.0009) [2023-12-26 20:02:47,312][105620] Updated weights for policy 1, policy_version 643535 (0.0010) [2023-12-26 20:02:47,552][105692] Updated weights for policy 0, policy_version 642729 (0.0009) [2023-12-26 20:02:47,607][105692] Updated weights for policy 0, policy_version 642739 (0.0006) [2023-12-26 20:02:47,664][105692] Updated weights for policy 0, policy_version 642749 (0.0010) [2023-12-26 20:02:47,722][105692] Updated weights for policy 0, policy_version 642759 (0.0010) [2023-12-26 20:02:47,999][105620] Updated weights for policy 1, policy_version 643545 (0.0007) [2023-12-26 20:02:48,069][105620] Updated weights for policy 1, policy_version 643555 (0.0006) [2023-12-26 20:02:48,131][105620] Updated weights for policy 1, policy_version 643565 (0.0005) [2023-12-26 20:02:48,193][105620] Updated weights for policy 1, policy_version 643575 (0.0006) [2023-12-26 20:02:48,268][105692] Updated weights for policy 0, policy_version 642769 (0.0010) [2023-12-26 20:02:48,312][105692] Updated weights for policy 0, policy_version 642779 (0.0010) [2023-12-26 20:02:48,369][105692] Updated weights for policy 0, policy_version 642789 (0.0009) [2023-12-26 20:02:48,802][105620] Updated weights for policy 1, policy_version 643585 (0.0006) [2023-12-26 20:02:48,862][105620] Updated weights for policy 1, policy_version 643595 (0.0006) [2023-12-26 20:02:48,922][105620] Updated weights for policy 1, policy_version 643605 (0.0005) [2023-12-26 20:02:49,154][105692] Updated weights for policy 0, policy_version 642799 (0.0008) [2023-12-26 20:02:49,208][105692] Updated weights for policy 0, policy_version 642809 (0.0009) [2023-12-26 20:02:49,273][105692] Updated weights for policy 0, policy_version 642819 (0.0009) [2023-12-26 20:02:49,609][105620] Updated weights for policy 1, policy_version 643615 (0.0007) [2023-12-26 20:02:49,672][105620] Updated weights for policy 1, policy_version 643625 (0.0007) [2023-12-26 20:02:49,725][105620] Updated weights for policy 1, policy_version 643635 (0.0006) [2023-12-26 20:02:50,057][105692] Updated weights for policy 0, policy_version 642829 (0.0009) [2023-12-26 20:02:50,116][105692] Updated weights for policy 0, policy_version 642839 (0.0009) [2023-12-26 20:02:50,171][105692] Updated weights for policy 0, policy_version 642849 (0.0009) [2023-12-26 20:02:50,389][105620] Updated weights for policy 1, policy_version 643645 (0.0007) [2023-12-26 20:02:50,455][105620] Updated weights for policy 1, policy_version 643655 (0.0008) [2023-12-26 20:02:50,521][105620] Updated weights for policy 1, policy_version 643665 (0.0006) [2023-12-26 20:02:50,907][105692] Updated weights for policy 0, policy_version 642859 (0.0009) [2023-12-26 20:02:50,971][105692] Updated weights for policy 0, policy_version 642869 (0.0009) [2023-12-26 20:02:51,036][105692] Updated weights for policy 0, policy_version 642879 (0.0009) [2023-12-26 20:02:51,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 329400320. Throughput: 0: 9117.7, 1: 10152.7. Samples: 329395772. Policy #0 lag: (min: 7.0, avg: 14.9, max: 39.0) [2023-12-26 20:02:51,063][104569] Avg episode reward: [(0, '6491.984'), (1, '8997.199')] [2023-12-26 20:02:51,190][105620] Updated weights for policy 1, policy_version 643675 (0.0007) [2023-12-26 20:02:51,254][105620] Updated weights for policy 1, policy_version 643685 (0.0011) [2023-12-26 20:02:51,319][105620] Updated weights for policy 1, policy_version 643695 (0.0006) [2023-12-26 20:02:51,832][105692] Updated weights for policy 0, policy_version 642889 (0.0008) [2023-12-26 20:02:51,893][105692] Updated weights for policy 0, policy_version 642899 (0.0008) [2023-12-26 20:02:51,959][105692] Updated weights for policy 0, policy_version 642909 (0.0008) [2023-12-26 20:02:52,018][105692] Updated weights for policy 0, policy_version 642919 (0.0008) [2023-12-26 20:02:52,041][105620] Updated weights for policy 1, policy_version 643705 (0.0009) [2023-12-26 20:02:52,106][105620] Updated weights for policy 1, policy_version 643715 (0.0007) [2023-12-26 20:02:52,166][105620] Updated weights for policy 1, policy_version 643725 (0.0005) [2023-12-26 20:02:52,226][105620] Updated weights for policy 1, policy_version 643735 (0.0005) [2023-12-26 20:02:52,776][105620] Updated weights for policy 1, policy_version 643745 (0.0010) [2023-12-26 20:02:52,820][105692] Updated weights for policy 0, policy_version 642929 (0.0010) [2023-12-26 20:02:52,824][105620] Updated weights for policy 1, policy_version 643755 (0.0007) [2023-12-26 20:02:52,876][105692] Updated weights for policy 0, policy_version 642939 (0.0011) [2023-12-26 20:02:52,884][105620] Updated weights for policy 1, policy_version 643765 (0.0006) [2023-12-26 20:02:52,927][105692] Updated weights for policy 0, policy_version 642949 (0.0009) [2023-12-26 20:02:53,427][105620] Updated weights for policy 1, policy_version 643775 (0.0005) [2023-12-26 20:02:53,480][105620] Updated weights for policy 1, policy_version 643785 (0.0005) [2023-12-26 20:02:53,494][105692] Updated weights for policy 0, policy_version 642959 (0.0006) [2023-12-26 20:02:53,529][105620] Updated weights for policy 1, policy_version 643795 (0.0005) [2023-12-26 20:02:53,549][105692] Updated weights for policy 0, policy_version 642969 (0.0010) [2023-12-26 20:02:53,604][105692] Updated weights for policy 0, policy_version 642979 (0.0010) [2023-12-26 20:02:54,186][105620] Updated weights for policy 1, policy_version 643805 (0.0008) [2023-12-26 20:02:54,245][105620] Updated weights for policy 1, policy_version 643815 (0.0010) [2023-12-26 20:02:54,309][105620] Updated weights for policy 1, policy_version 643825 (0.0010) [2023-12-26 20:02:54,324][105692] Updated weights for policy 0, policy_version 642989 (0.0010) [2023-12-26 20:02:54,379][105692] Updated weights for policy 0, policy_version 642999 (0.0010) [2023-12-26 20:02:54,433][105692] Updated weights for policy 0, policy_version 643009 (0.0010) [2023-12-26 20:02:55,050][105620] Updated weights for policy 1, policy_version 643835 (0.0011) [2023-12-26 20:02:55,066][105692] Updated weights for policy 0, policy_version 643019 (0.0008) [2023-12-26 20:02:55,108][105620] Updated weights for policy 1, policy_version 643845 (0.0010) [2023-12-26 20:02:55,123][105692] Updated weights for policy 0, policy_version 643029 (0.0005) [2023-12-26 20:02:55,167][105620] Updated weights for policy 1, policy_version 643855 (0.0010) [2023-12-26 20:02:55,183][105692] Updated weights for policy 0, policy_version 643039 (0.0005) [2023-12-26 20:02:55,799][105692] Updated weights for policy 0, policy_version 643049 (0.0006) [2023-12-26 20:02:55,864][105692] Updated weights for policy 0, policy_version 643059 (0.0010) [2023-12-26 20:02:55,907][105620] Updated weights for policy 1, policy_version 643865 (0.0010) [2023-12-26 20:02:55,925][105692] Updated weights for policy 0, policy_version 643069 (0.0010) [2023-12-26 20:02:55,961][105620] Updated weights for policy 1, policy_version 643875 (0.0010) [2023-12-26 20:02:55,970][105692] Updated weights for policy 0, policy_version 643079 (0.0011) [2023-12-26 20:02:56,026][105620] Updated weights for policy 1, policy_version 643885 (0.0010) [2023-12-26 20:02:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 329506816. Throughput: 0: 9272.6, 1: 10158.6. Samples: 329517156. Policy #0 lag: (min: 7.0, avg: 14.9, max: 39.0) [2023-12-26 20:02:56,063][104569] Avg episode reward: [(0, '7931.963'), (1, '9176.655')] [2023-12-26 20:02:56,092][105620] Updated weights for policy 1, policy_version 643895 (0.0010) [2023-12-26 20:02:56,732][105692] Updated weights for policy 0, policy_version 643089 (0.0011) [2023-12-26 20:02:56,792][105692] Updated weights for policy 0, policy_version 643099 (0.0011) [2023-12-26 20:02:56,809][105620] Updated weights for policy 1, policy_version 643905 (0.0010) [2023-12-26 20:02:56,844][105692] Updated weights for policy 0, policy_version 643109 (0.0010) [2023-12-26 20:02:56,858][105620] Updated weights for policy 1, policy_version 643915 (0.0010) [2023-12-26 20:02:56,910][105620] Updated weights for policy 1, policy_version 643925 (0.0010) [2023-12-26 20:02:57,620][105692] Updated weights for policy 0, policy_version 643119 (0.0007) [2023-12-26 20:02:57,641][105620] Updated weights for policy 1, policy_version 643935 (0.0010) [2023-12-26 20:02:57,669][105692] Updated weights for policy 0, policy_version 643129 (0.0010) [2023-12-26 20:02:57,699][105620] Updated weights for policy 1, policy_version 643945 (0.0010) [2023-12-26 20:02:57,723][105692] Updated weights for policy 0, policy_version 643139 (0.0010) [2023-12-26 20:02:57,747][105620] Updated weights for policy 1, policy_version 643955 (0.0010) [2023-12-26 20:02:58,466][105692] Updated weights for policy 0, policy_version 643149 (0.0011) [2023-12-26 20:02:58,533][105692] Updated weights for policy 0, policy_version 643159 (0.0011) [2023-12-26 20:02:58,538][105620] Updated weights for policy 1, policy_version 643965 (0.0008) [2023-12-26 20:02:58,592][105692] Updated weights for policy 0, policy_version 643169 (0.0009) [2023-12-26 20:02:58,597][105620] Updated weights for policy 1, policy_version 643975 (0.0006) [2023-12-26 20:02:58,661][105620] Updated weights for policy 1, policy_version 643985 (0.0007) [2023-12-26 20:02:59,428][105620] Updated weights for policy 1, policy_version 643995 (0.0010) [2023-12-26 20:02:59,468][105692] Updated weights for policy 0, policy_version 643179 (0.0008) [2023-12-26 20:02:59,485][105620] Updated weights for policy 1, policy_version 644005 (0.0010) [2023-12-26 20:02:59,521][105692] Updated weights for policy 0, policy_version 643189 (0.0006) [2023-12-26 20:02:59,535][105620] Updated weights for policy 1, policy_version 644015 (0.0008) [2023-12-26 20:02:59,583][105692] Updated weights for policy 0, policy_version 643199 (0.0008) [2023-12-26 20:03:00,270][105620] Updated weights for policy 1, policy_version 644025 (0.0009) [2023-12-26 20:03:00,283][105692] Updated weights for policy 0, policy_version 643209 (0.0009) [2023-12-26 20:03:00,337][105620] Updated weights for policy 1, policy_version 644035 (0.0007) [2023-12-26 20:03:00,342][105692] Updated weights for policy 0, policy_version 643219 (0.0007) [2023-12-26 20:03:00,396][105620] Updated weights for policy 1, policy_version 644045 (0.0008) [2023-12-26 20:03:00,396][105692] Updated weights for policy 0, policy_version 643229 (0.0006) [2023-12-26 20:03:00,451][105620] Updated weights for policy 1, policy_version 644055 (0.0009) [2023-12-26 20:03:00,456][105692] Updated weights for policy 0, policy_version 643239 (0.0006) [2023-12-26 20:03:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 329596928. Throughput: 0: 9272.9, 1: 10138.8. Samples: 329572812. Policy #0 lag: (min: 7.0, avg: 14.9, max: 39.0) [2023-12-26 20:03:01,062][104569] Avg episode reward: [(0, '8554.011'), (1, '9007.777')] [2023-12-26 20:03:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000643240_164700160.pth... [2023-12-26 20:03:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000642152_164421632.pth [2023-12-26 20:03:01,076][105620] Updated weights for policy 1, policy_version 644065 (0.0009) [2023-12-26 20:03:01,130][105620] Updated weights for policy 1, policy_version 644075 (0.0009) [2023-12-26 20:03:01,161][105692] Updated weights for policy 0, policy_version 643249 (0.0009) [2023-12-26 20:03:01,193][105620] Updated weights for policy 1, policy_version 644085 (0.0009) [2023-12-26 20:03:01,207][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000644088_164904960.pth... [2023-12-26 20:03:01,210][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000642872_164593664.pth [2023-12-26 20:03:01,216][105692] Updated weights for policy 0, policy_version 643259 (0.0009) [2023-12-26 20:03:01,272][105692] Updated weights for policy 0, policy_version 643269 (0.0009) [2023-12-26 20:03:02,031][105620] Updated weights for policy 1, policy_version 644095 (0.0007) [2023-12-26 20:03:02,037][105692] Updated weights for policy 0, policy_version 643279 (0.0010) [2023-12-26 20:03:02,084][105620] Updated weights for policy 1, policy_version 644105 (0.0006) [2023-12-26 20:03:02,086][105692] Updated weights for policy 0, policy_version 643289 (0.0011) [2023-12-26 20:03:02,136][105692] Updated weights for policy 0, policy_version 643299 (0.0009) [2023-12-26 20:03:02,136][105620] Updated weights for policy 1, policy_version 644115 (0.0007) [2023-12-26 20:03:02,871][105692] Updated weights for policy 0, policy_version 643309 (0.0007) [2023-12-26 20:03:02,876][105620] Updated weights for policy 1, policy_version 644125 (0.0007) [2023-12-26 20:03:02,921][105620] Updated weights for policy 1, policy_version 644135 (0.0009) [2023-12-26 20:03:02,923][105692] Updated weights for policy 0, policy_version 643319 (0.0010) [2023-12-26 20:03:02,968][105620] Updated weights for policy 1, policy_version 644145 (0.0005) [2023-12-26 20:03:02,977][105692] Updated weights for policy 0, policy_version 643329 (0.0010) [2023-12-26 20:03:03,695][105692] Updated weights for policy 0, policy_version 643339 (0.0009) [2023-12-26 20:03:03,717][105620] Updated weights for policy 1, policy_version 644155 (0.0006) [2023-12-26 20:03:03,749][105692] Updated weights for policy 0, policy_version 643349 (0.0006) [2023-12-26 20:03:03,764][105620] Updated weights for policy 1, policy_version 644165 (0.0005) [2023-12-26 20:03:03,797][105692] Updated weights for policy 0, policy_version 643359 (0.0008) [2023-12-26 20:03:03,813][105620] Updated weights for policy 1, policy_version 644175 (0.0005) [2023-12-26 20:03:04,507][105692] Updated weights for policy 0, policy_version 643369 (0.0007) [2023-12-26 20:03:04,556][105692] Updated weights for policy 0, policy_version 643379 (0.0010) [2023-12-26 20:03:04,565][105620] Updated weights for policy 1, policy_version 644185 (0.0007) [2023-12-26 20:03:04,605][105692] Updated weights for policy 0, policy_version 643389 (0.0010) [2023-12-26 20:03:04,623][105620] Updated weights for policy 1, policy_version 644195 (0.0007) [2023-12-26 20:03:04,664][105692] Updated weights for policy 0, policy_version 643399 (0.0010) [2023-12-26 20:03:04,693][105620] Updated weights for policy 1, policy_version 644205 (0.0008) [2023-12-26 20:03:04,753][105620] Updated weights for policy 1, policy_version 644215 (0.0007) [2023-12-26 20:03:05,347][105692] Updated weights for policy 0, policy_version 643409 (0.0006) [2023-12-26 20:03:05,398][105692] Updated weights for policy 0, policy_version 643419 (0.0005) [2023-12-26 20:03:05,453][105692] Updated weights for policy 0, policy_version 643429 (0.0005) [2023-12-26 20:03:05,510][105620] Updated weights for policy 1, policy_version 644226 (0.0010) [2023-12-26 20:03:05,567][105620] Updated weights for policy 1, policy_version 644236 (0.0010) [2023-12-26 20:03:05,609][105620] Updated weights for policy 1, policy_version 644246 (0.0006) [2023-12-26 20:03:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 329695232. Throughput: 0: 9287.9, 1: 10074.9. Samples: 329686628. Policy #0 lag: (min: 7.0, avg: 14.9, max: 39.0) [2023-12-26 20:03:06,063][104569] Avg episode reward: [(0, '8372.193'), (1, '7915.163')] [2023-12-26 20:03:06,095][105692] Updated weights for policy 0, policy_version 643439 (0.0008) [2023-12-26 20:03:06,164][105692] Updated weights for policy 0, policy_version 643449 (0.0011) [2023-12-26 20:03:06,234][105692] Updated weights for policy 0, policy_version 643459 (0.0011) [2023-12-26 20:03:06,320][105620] Updated weights for policy 1, policy_version 644256 (0.0006) [2023-12-26 20:03:06,385][105620] Updated weights for policy 1, policy_version 644266 (0.0006) [2023-12-26 20:03:06,445][105620] Updated weights for policy 1, policy_version 644276 (0.0005) [2023-12-26 20:03:07,005][105692] Updated weights for policy 0, policy_version 643469 (0.0011) [2023-12-26 20:03:07,051][105620] Updated weights for policy 1, policy_version 644286 (0.0006) [2023-12-26 20:03:07,061][105692] Updated weights for policy 0, policy_version 643479 (0.0010) [2023-12-26 20:03:07,110][105620] Updated weights for policy 1, policy_version 644296 (0.0008) [2023-12-26 20:03:07,123][105692] Updated weights for policy 0, policy_version 643489 (0.0011) [2023-12-26 20:03:07,171][105620] Updated weights for policy 1, policy_version 644306 (0.0006) [2023-12-26 20:03:07,830][105620] Updated weights for policy 1, policy_version 644316 (0.0007) [2023-12-26 20:03:07,847][105692] Updated weights for policy 0, policy_version 643499 (0.0010) [2023-12-26 20:03:07,894][105620] Updated weights for policy 1, policy_version 644326 (0.0005) [2023-12-26 20:03:07,906][105692] Updated weights for policy 0, policy_version 643509 (0.0006) [2023-12-26 20:03:07,952][105620] Updated weights for policy 1, policy_version 644336 (0.0005) [2023-12-26 20:03:07,962][105692] Updated weights for policy 0, policy_version 643519 (0.0006) [2023-12-26 20:03:08,001][105585] KL-divergence is very high: 108.0366 [2023-12-26 20:03:08,550][105692] Updated weights for policy 0, policy_version 643529 (0.0006) [2023-12-26 20:03:08,607][105620] Updated weights for policy 1, policy_version 644346 (0.0008) [2023-12-26 20:03:08,612][105692] Updated weights for policy 0, policy_version 643539 (0.0008) [2023-12-26 20:03:08,661][105620] Updated weights for policy 1, policy_version 644356 (0.0007) [2023-12-26 20:03:08,664][105692] Updated weights for policy 0, policy_version 643549 (0.0005) [2023-12-26 20:03:08,717][105692] Updated weights for policy 0, policy_version 643559 (0.0008) [2023-12-26 20:03:08,721][105620] Updated weights for policy 1, policy_version 644366 (0.0006) [2023-12-26 20:03:08,785][105620] Updated weights for policy 1, policy_version 644376 (0.0009) [2023-12-26 20:03:09,478][105692] Updated weights for policy 0, policy_version 643569 (0.0008) [2023-12-26 20:03:09,511][105620] Updated weights for policy 1, policy_version 644386 (0.0006) [2023-12-26 20:03:09,535][105692] Updated weights for policy 0, policy_version 643579 (0.0009) [2023-12-26 20:03:09,571][105620] Updated weights for policy 1, policy_version 644396 (0.0005) [2023-12-26 20:03:09,592][105692] Updated weights for policy 0, policy_version 643589 (0.0009) [2023-12-26 20:03:09,635][105620] Updated weights for policy 1, policy_version 644406 (0.0010) [2023-12-26 20:03:10,362][105620] Updated weights for policy 1, policy_version 644416 (0.0011) [2023-12-26 20:03:10,386][105692] Updated weights for policy 0, policy_version 643599 (0.0010) [2023-12-26 20:03:10,418][105620] Updated weights for policy 1, policy_version 644426 (0.0011) [2023-12-26 20:03:10,445][105692] Updated weights for policy 0, policy_version 643609 (0.0007) [2023-12-26 20:03:10,474][105620] Updated weights for policy 1, policy_version 644436 (0.0010) [2023-12-26 20:03:10,509][105692] Updated weights for policy 0, policy_version 643619 (0.0006) [2023-12-26 20:03:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 329793536. Throughput: 0: 9359.8, 1: 10165.6. Samples: 329806200. Policy #0 lag: (min: 7.0, avg: 14.9, max: 39.0) [2023-12-26 20:03:11,062][104569] Avg episode reward: [(0, '8099.268'), (1, '8055.111')] [2023-12-26 20:03:11,121][105620] Updated weights for policy 1, policy_version 644446 (0.0010) [2023-12-26 20:03:11,190][105620] Updated weights for policy 1, policy_version 644456 (0.0008) [2023-12-26 20:03:11,254][105620] Updated weights for policy 1, policy_version 644466 (0.0008) [2023-12-26 20:03:11,303][105692] Updated weights for policy 0, policy_version 643629 (0.0007) [2023-12-26 20:03:11,362][105692] Updated weights for policy 0, policy_version 643639 (0.0009) [2023-12-26 20:03:11,428][105692] Updated weights for policy 0, policy_version 643649 (0.0006) [2023-12-26 20:03:12,070][105692] Updated weights for policy 0, policy_version 643659 (0.0007) [2023-12-26 20:03:12,089][105620] Updated weights for policy 1, policy_version 644476 (0.0007) [2023-12-26 20:03:12,130][105692] Updated weights for policy 0, policy_version 643669 (0.0006) [2023-12-26 20:03:12,151][105620] Updated weights for policy 1, policy_version 644486 (0.0009) [2023-12-26 20:03:12,189][105692] Updated weights for policy 0, policy_version 643679 (0.0007) [2023-12-26 20:03:12,203][105620] Updated weights for policy 1, policy_version 644496 (0.0009) [2023-12-26 20:03:12,917][105620] Updated weights for policy 1, policy_version 644506 (0.0007) [2023-12-26 20:03:12,950][105692] Updated weights for policy 0, policy_version 643689 (0.0007) [2023-12-26 20:03:12,969][105620] Updated weights for policy 1, policy_version 644516 (0.0008) [2023-12-26 20:03:12,996][105692] Updated weights for policy 0, policy_version 643699 (0.0006) [2023-12-26 20:03:13,015][105620] Updated weights for policy 1, policy_version 644526 (0.0006) [2023-12-26 20:03:13,042][105692] Updated weights for policy 0, policy_version 643709 (0.0006) [2023-12-26 20:03:13,068][105620] Updated weights for policy 1, policy_version 644536 (0.0007) [2023-12-26 20:03:13,092][105692] Updated weights for policy 0, policy_version 643719 (0.0007) [2023-12-26 20:03:13,810][105692] Updated weights for policy 0, policy_version 643729 (0.0007) [2023-12-26 20:03:13,851][105620] Updated weights for policy 1, policy_version 644546 (0.0009) [2023-12-26 20:03:13,863][105692] Updated weights for policy 0, policy_version 643739 (0.0007) [2023-12-26 20:03:13,904][105620] Updated weights for policy 1, policy_version 644556 (0.0008) [2023-12-26 20:03:13,923][105692] Updated weights for policy 0, policy_version 643749 (0.0007) [2023-12-26 20:03:13,966][105620] Updated weights for policy 1, policy_version 644566 (0.0007) [2023-12-26 20:03:14,611][105692] Updated weights for policy 0, policy_version 643759 (0.0010) [2023-12-26 20:03:14,669][105692] Updated weights for policy 0, policy_version 643769 (0.0010) [2023-12-26 20:03:14,720][105692] Updated weights for policy 0, policy_version 643779 (0.0007) [2023-12-26 20:03:14,746][105620] Updated weights for policy 1, policy_version 644576 (0.0007) [2023-12-26 20:03:14,811][105620] Updated weights for policy 1, policy_version 644586 (0.0009) [2023-12-26 20:03:14,866][105620] Updated weights for policy 1, policy_version 644596 (0.0008) [2023-12-26 20:03:15,480][105692] Updated weights for policy 0, policy_version 643789 (0.0006) [2023-12-26 20:03:15,540][105692] Updated weights for policy 0, policy_version 643799 (0.0007) [2023-12-26 20:03:15,554][105620] Updated weights for policy 1, policy_version 644606 (0.0007) [2023-12-26 20:03:15,602][105692] Updated weights for policy 0, policy_version 643809 (0.0011) [2023-12-26 20:03:15,608][105620] Updated weights for policy 1, policy_version 644616 (0.0006) [2023-12-26 20:03:15,673][105620] Updated weights for policy 1, policy_version 644626 (0.0007) [2023-12-26 20:03:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 329891840. Throughput: 0: 9343.7, 1: 10096.4. Samples: 329861392. Policy #0 lag: (min: 7.0, avg: 14.9, max: 39.0) [2023-12-26 20:03:16,063][104569] Avg episode reward: [(0, '8006.011'), (1, '8861.034')] [2023-12-26 20:03:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000643816_164847616.pth... [2023-12-26 20:03:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000644632_165044224.pth... [2023-12-26 20:03:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000642696_164560896.pth [2023-12-26 20:03:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000643480_164749312.pth [2023-12-26 20:03:16,329][105692] Updated weights for policy 0, policy_version 643819 (0.0011) [2023-12-26 20:03:16,359][105620] Updated weights for policy 1, policy_version 644636 (0.0008) [2023-12-26 20:03:16,374][105692] Updated weights for policy 0, policy_version 643829 (0.0010) [2023-12-26 20:03:16,401][105585] KL-divergence is very high: 100.0633 [2023-12-26 20:03:16,421][105620] Updated weights for policy 1, policy_version 644646 (0.0007) [2023-12-26 20:03:16,423][105692] Updated weights for policy 0, policy_version 643839 (0.0010) [2023-12-26 20:03:16,440][105585] KL-divergence is very high: 102.2873 [2023-12-26 20:03:16,474][105620] Updated weights for policy 1, policy_version 644656 (0.0006) [2023-12-26 20:03:17,183][105692] Updated weights for policy 0, policy_version 643849 (0.0010) [2023-12-26 20:03:17,184][105620] Updated weights for policy 1, policy_version 644666 (0.0007) [2023-12-26 20:03:17,241][105692] Updated weights for policy 0, policy_version 643859 (0.0010) [2023-12-26 20:03:17,249][105620] Updated weights for policy 1, policy_version 644676 (0.0005) [2023-12-26 20:03:17,296][105692] Updated weights for policy 0, policy_version 643869 (0.0010) [2023-12-26 20:03:17,299][105620] Updated weights for policy 1, policy_version 644686 (0.0006) [2023-12-26 20:03:17,347][105620] Updated weights for policy 1, policy_version 644696 (0.0009) [2023-12-26 20:03:17,354][105692] Updated weights for policy 0, policy_version 643879 (0.0010) [2023-12-26 20:03:17,990][105620] Updated weights for policy 1, policy_version 644706 (0.0007) [2023-12-26 20:03:18,049][105620] Updated weights for policy 1, policy_version 644716 (0.0007) [2023-12-26 20:03:18,101][105620] Updated weights for policy 1, policy_version 644726 (0.0005) [2023-12-26 20:03:18,108][105692] Updated weights for policy 0, policy_version 643889 (0.0010) [2023-12-26 20:03:18,171][105692] Updated weights for policy 0, policy_version 643899 (0.0010) [2023-12-26 20:03:18,225][105692] Updated weights for policy 0, policy_version 643909 (0.0010) [2023-12-26 20:03:18,750][105620] Updated weights for policy 1, policy_version 644736 (0.0009) [2023-12-26 20:03:18,816][105620] Updated weights for policy 1, policy_version 644746 (0.0010) [2023-12-26 20:03:18,882][105620] Updated weights for policy 1, policy_version 644756 (0.0010) [2023-12-26 20:03:18,904][105692] Updated weights for policy 0, policy_version 643919 (0.0010) [2023-12-26 20:03:18,962][105692] Updated weights for policy 0, policy_version 643929 (0.0010) [2023-12-26 20:03:19,017][105692] Updated weights for policy 0, policy_version 643939 (0.0010) [2023-12-26 20:03:19,561][105620] Updated weights for policy 1, policy_version 644766 (0.0008) [2023-12-26 20:03:19,612][105620] Updated weights for policy 1, policy_version 644776 (0.0008) [2023-12-26 20:03:19,667][105620] Updated weights for policy 1, policy_version 644786 (0.0008) [2023-12-26 20:03:19,696][105692] Updated weights for policy 0, policy_version 643949 (0.0010) [2023-12-26 20:03:19,754][105692] Updated weights for policy 0, policy_version 643959 (0.0010) [2023-12-26 20:03:19,819][105692] Updated weights for policy 0, policy_version 643969 (0.0009) [2023-12-26 20:03:20,396][105620] Updated weights for policy 1, policy_version 644796 (0.0007) [2023-12-26 20:03:20,445][105620] Updated weights for policy 1, policy_version 644806 (0.0005) [2023-12-26 20:03:20,492][105620] Updated weights for policy 1, policy_version 644816 (0.0005) [2023-12-26 20:03:20,592][105692] Updated weights for policy 0, policy_version 643979 (0.0009) [2023-12-26 20:03:20,662][105692] Updated weights for policy 0, policy_version 643989 (0.0008) [2023-12-26 20:03:20,721][105692] Updated weights for policy 0, policy_version 643999 (0.0008) [2023-12-26 20:03:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 329990144. Throughput: 0: 9518.4, 1: 9972.3. Samples: 329979888. Policy #0 lag: (min: 7.0, avg: 14.9, max: 39.0) [2023-12-26 20:03:21,063][104569] Avg episode reward: [(0, '7820.956'), (1, '9262.676')] [2023-12-26 20:03:21,148][105620] Updated weights for policy 1, policy_version 644826 (0.0006) [2023-12-26 20:03:21,217][105620] Updated weights for policy 1, policy_version 644836 (0.0010) [2023-12-26 20:03:21,281][105620] Updated weights for policy 1, policy_version 644846 (0.0010) [2023-12-26 20:03:21,346][105620] Updated weights for policy 1, policy_version 644856 (0.0011) [2023-12-26 20:03:21,510][105692] Updated weights for policy 0, policy_version 644009 (0.0008) [2023-12-26 20:03:21,578][105692] Updated weights for policy 0, policy_version 644019 (0.0008) [2023-12-26 20:03:21,641][105692] Updated weights for policy 0, policy_version 644029 (0.0007) [2023-12-26 20:03:21,709][105692] Updated weights for policy 0, policy_version 644039 (0.0007) [2023-12-26 20:03:21,710][105585] KL-divergence is very high: 201.1588 [2023-12-26 20:03:22,061][105620] Updated weights for policy 1, policy_version 644866 (0.0010) [2023-12-26 20:03:22,121][105620] Updated weights for policy 1, policy_version 644876 (0.0010) [2023-12-26 20:03:22,171][105620] Updated weights for policy 1, policy_version 644886 (0.0010) [2023-12-26 20:03:22,408][105692] Updated weights for policy 0, policy_version 644049 (0.0009) [2023-12-26 20:03:22,463][105692] Updated weights for policy 0, policy_version 644059 (0.0010) [2023-12-26 20:03:22,527][105692] Updated weights for policy 0, policy_version 644069 (0.0008) [2023-12-26 20:03:22,955][105620] Updated weights for policy 1, policy_version 644896 (0.0010) [2023-12-26 20:03:23,016][105620] Updated weights for policy 1, policy_version 644906 (0.0011) [2023-12-26 20:03:23,080][105620] Updated weights for policy 1, policy_version 644916 (0.0011) [2023-12-26 20:03:23,267][105692] Updated weights for policy 0, policy_version 644079 (0.0009) [2023-12-26 20:03:23,316][105692] Updated weights for policy 0, policy_version 644089 (0.0008) [2023-12-26 20:03:23,372][105692] Updated weights for policy 0, policy_version 644099 (0.0008) [2023-12-26 20:03:23,810][105620] Updated weights for policy 1, policy_version 644926 (0.0010) [2023-12-26 20:03:23,858][105620] Updated weights for policy 1, policy_version 644936 (0.0010) [2023-12-26 20:03:23,908][105620] Updated weights for policy 1, policy_version 644946 (0.0010) [2023-12-26 20:03:23,980][105692] Updated weights for policy 0, policy_version 644109 (0.0007) [2023-12-26 20:03:24,041][105692] Updated weights for policy 0, policy_version 644119 (0.0010) [2023-12-26 20:03:24,106][105692] Updated weights for policy 0, policy_version 644129 (0.0011) [2023-12-26 20:03:24,666][105620] Updated weights for policy 1, policy_version 644956 (0.0010) [2023-12-26 20:03:24,718][105620] Updated weights for policy 1, policy_version 644966 (0.0010) [2023-12-26 20:03:24,775][105620] Updated weights for policy 1, policy_version 644976 (0.0010) [2023-12-26 20:03:24,800][105692] Updated weights for policy 0, policy_version 644139 (0.0011) [2023-12-26 20:03:24,855][105692] Updated weights for policy 0, policy_version 644149 (0.0010) [2023-12-26 20:03:24,914][105692] Updated weights for policy 0, policy_version 644159 (0.0011) [2023-12-26 20:03:25,471][105620] Updated weights for policy 1, policy_version 644986 (0.0010) [2023-12-26 20:03:25,515][105620] Updated weights for policy 1, policy_version 644996 (0.0010) [2023-12-26 20:03:25,559][105620] Updated weights for policy 1, policy_version 645006 (0.0010) [2023-12-26 20:03:25,613][105620] Updated weights for policy 1, policy_version 645016 (0.0010) [2023-12-26 20:03:25,666][105692] Updated weights for policy 0, policy_version 644169 (0.0010) [2023-12-26 20:03:25,715][105692] Updated weights for policy 0, policy_version 644179 (0.0009) [2023-12-26 20:03:25,763][105692] Updated weights for policy 0, policy_version 644189 (0.0010) [2023-12-26 20:03:25,818][105692] Updated weights for policy 0, policy_version 644199 (0.0010) [2023-12-26 20:03:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 330088448. Throughput: 0: 9611.6, 1: 9913.0. Samples: 330095732. Policy #0 lag: (min: 7.0, avg: 14.9, max: 39.0) [2023-12-26 20:03:26,062][104569] Avg episode reward: [(0, '7821.925'), (1, '9263.994')] [2023-12-26 20:03:26,379][105620] Updated weights for policy 1, policy_version 645026 (0.0010) [2023-12-26 20:03:26,434][105620] Updated weights for policy 1, policy_version 645036 (0.0010) [2023-12-26 20:03:26,492][105620] Updated weights for policy 1, policy_version 645046 (0.0010) [2023-12-26 20:03:26,567][105692] Updated weights for policy 0, policy_version 644209 (0.0011) [2023-12-26 20:03:26,622][105692] Updated weights for policy 0, policy_version 644219 (0.0010) [2023-12-26 20:03:26,679][105692] Updated weights for policy 0, policy_version 644229 (0.0010) [2023-12-26 20:03:27,235][105620] Updated weights for policy 1, policy_version 645056 (0.0008) [2023-12-26 20:03:27,300][105620] Updated weights for policy 1, policy_version 645066 (0.0009) [2023-12-26 20:03:27,332][105692] Updated weights for policy 0, policy_version 644239 (0.0011) [2023-12-26 20:03:27,363][105620] Updated weights for policy 1, policy_version 645076 (0.0010) [2023-12-26 20:03:27,390][105692] Updated weights for policy 0, policy_version 644249 (0.0010) [2023-12-26 20:03:27,442][105692] Updated weights for policy 0, policy_version 644259 (0.0011) [2023-12-26 20:03:28,062][105620] Updated weights for policy 1, policy_version 645086 (0.0009) [2023-12-26 20:03:28,125][105620] Updated weights for policy 1, policy_version 645096 (0.0010) [2023-12-26 20:03:28,189][105620] Updated weights for policy 1, policy_version 645106 (0.0010) [2023-12-26 20:03:28,197][105692] Updated weights for policy 0, policy_version 644269 (0.0011) [2023-12-26 20:03:28,259][105692] Updated weights for policy 0, policy_version 644279 (0.0010) [2023-12-26 20:03:28,319][105692] Updated weights for policy 0, policy_version 644289 (0.0006) [2023-12-26 20:03:28,865][105620] Updated weights for policy 1, policy_version 645116 (0.0010) [2023-12-26 20:03:28,924][105620] Updated weights for policy 1, policy_version 645126 (0.0010) [2023-12-26 20:03:28,981][105620] Updated weights for policy 1, policy_version 645136 (0.0010) [2023-12-26 20:03:29,065][105692] Updated weights for policy 0, policy_version 644299 (0.0008) [2023-12-26 20:03:29,126][105692] Updated weights for policy 0, policy_version 644309 (0.0007) [2023-12-26 20:03:29,182][105692] Updated weights for policy 0, policy_version 644319 (0.0005) [2023-12-26 20:03:29,716][105620] Updated weights for policy 1, policy_version 645146 (0.0010) [2023-12-26 20:03:29,775][105620] Updated weights for policy 1, policy_version 645156 (0.0010) [2023-12-26 20:03:29,841][105620] Updated weights for policy 1, policy_version 645166 (0.0010) [2023-12-26 20:03:29,908][105620] Updated weights for policy 1, policy_version 645176 (0.0011) [2023-12-26 20:03:29,947][105692] Updated weights for policy 0, policy_version 644329 (0.0007) [2023-12-26 20:03:30,000][105692] Updated weights for policy 0, policy_version 644339 (0.0008) [2023-12-26 20:03:30,053][105692] Updated weights for policy 0, policy_version 644349 (0.0009) [2023-12-26 20:03:30,112][105692] Updated weights for policy 0, policy_version 644359 (0.0008) [2023-12-26 20:03:30,593][105620] Updated weights for policy 1, policy_version 645186 (0.0005) [2023-12-26 20:03:30,643][105620] Updated weights for policy 1, policy_version 645196 (0.0009) [2023-12-26 20:03:30,694][105620] Updated weights for policy 1, policy_version 645206 (0.0010) [2023-12-26 20:03:30,808][105692] Updated weights for policy 0, policy_version 644369 (0.0005) [2023-12-26 20:03:30,873][105692] Updated weights for policy 0, policy_version 644379 (0.0005) [2023-12-26 20:03:30,941][105692] Updated weights for policy 0, policy_version 644389 (0.0005) [2023-12-26 20:03:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 330186752. Throughput: 0: 9633.9, 1: 9893.0. Samples: 330154116. Policy #0 lag: (min: 7.0, avg: 14.9, max: 39.0) [2023-12-26 20:03:31,063][104569] Avg episode reward: [(0, '8277.090'), (1, '9264.010')] [2023-12-26 20:03:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000644392_164995072.pth... [2023-12-26 20:03:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000645208_165191680.pth... [2023-12-26 20:03:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000644088_164904960.pth [2023-12-26 20:03:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000643240_164700160.pth [2023-12-26 20:03:31,429][105620] Updated weights for policy 1, policy_version 645216 (0.0008) [2023-12-26 20:03:31,477][105620] Updated weights for policy 1, policy_version 645226 (0.0008) [2023-12-26 20:03:31,493][105692] Updated weights for policy 0, policy_version 644399 (0.0006) [2023-12-26 20:03:31,524][105620] Updated weights for policy 1, policy_version 645236 (0.0008) [2023-12-26 20:03:31,543][105692] Updated weights for policy 0, policy_version 644409 (0.0007) [2023-12-26 20:03:31,595][105692] Updated weights for policy 0, policy_version 644419 (0.0008) [2023-12-26 20:03:32,278][105692] Updated weights for policy 0, policy_version 644429 (0.0008) [2023-12-26 20:03:32,290][105620] Updated weights for policy 1, policy_version 645246 (0.0008) [2023-12-26 20:03:32,339][105692] Updated weights for policy 0, policy_version 644439 (0.0008) [2023-12-26 20:03:32,353][105620] Updated weights for policy 1, policy_version 645256 (0.0007) [2023-12-26 20:03:32,396][105692] Updated weights for policy 0, policy_version 644449 (0.0007) [2023-12-26 20:03:32,406][105620] Updated weights for policy 1, policy_version 645266 (0.0007) [2023-12-26 20:03:32,993][105620] Updated weights for policy 1, policy_version 645276 (0.0005) [2023-12-26 20:03:33,043][105620] Updated weights for policy 1, policy_version 645286 (0.0005) [2023-12-26 20:03:33,094][105620] Updated weights for policy 1, policy_version 645296 (0.0005) [2023-12-26 20:03:33,218][105692] Updated weights for policy 0, policy_version 644459 (0.0006) [2023-12-26 20:03:33,274][105692] Updated weights for policy 0, policy_version 644469 (0.0005) [2023-12-26 20:03:33,320][105692] Updated weights for policy 0, policy_version 644479 (0.0005) [2023-12-26 20:03:33,771][105620] Updated weights for policy 1, policy_version 645306 (0.0006) [2023-12-26 20:03:33,815][105620] Updated weights for policy 1, policy_version 645316 (0.0008) [2023-12-26 20:03:33,866][105620] Updated weights for policy 1, policy_version 645327 (0.0009) [2023-12-26 20:03:33,894][105692] Updated weights for policy 0, policy_version 644489 (0.0005) [2023-12-26 20:03:33,944][105692] Updated weights for policy 0, policy_version 644499 (0.0005) [2023-12-26 20:03:33,995][105692] Updated weights for policy 0, policy_version 644509 (0.0007) [2023-12-26 20:03:34,049][105692] Updated weights for policy 0, policy_version 644519 (0.0008) [2023-12-26 20:03:34,637][105620] Updated weights for policy 1, policy_version 645338 (0.0009) [2023-12-26 20:03:34,700][105620] Updated weights for policy 1, policy_version 645348 (0.0006) [2023-12-26 20:03:34,755][105620] Updated weights for policy 1, policy_version 645358 (0.0005) [2023-12-26 20:03:34,813][105620] Updated weights for policy 1, policy_version 645368 (0.0007) [2023-12-26 20:03:34,836][105692] Updated weights for policy 0, policy_version 644529 (0.0009) [2023-12-26 20:03:34,895][105692] Updated weights for policy 0, policy_version 644539 (0.0009) [2023-12-26 20:03:34,959][105692] Updated weights for policy 0, policy_version 644549 (0.0010) [2023-12-26 20:03:35,471][105620] Updated weights for policy 1, policy_version 645378 (0.0011) [2023-12-26 20:03:35,520][105620] Updated weights for policy 1, policy_version 645388 (0.0011) [2023-12-26 20:03:35,572][105620] Updated weights for policy 1, policy_version 645398 (0.0010) [2023-12-26 20:03:35,759][105692] Updated weights for policy 0, policy_version 644559 (0.0010) [2023-12-26 20:03:35,813][105692] Updated weights for policy 0, policy_version 644569 (0.0006) [2023-12-26 20:03:35,860][105692] Updated weights for policy 0, policy_version 644579 (0.0005) [2023-12-26 20:03:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 330285056. Throughput: 0: 9693.2, 1: 9814.5. Samples: 330273620. Policy #0 lag: (min: 7.0, avg: 14.9, max: 39.0) [2023-12-26 20:03:36,063][104569] Avg episode reward: [(0, '8455.230'), (1, '9081.896')] [2023-12-26 20:03:36,292][105620] Updated weights for policy 1, policy_version 645408 (0.0009) [2023-12-26 20:03:36,357][105620] Updated weights for policy 1, policy_version 645418 (0.0009) [2023-12-26 20:03:36,428][105620] Updated weights for policy 1, policy_version 645428 (0.0010) [2023-12-26 20:03:36,588][105692] Updated weights for policy 0, policy_version 644589 (0.0009) [2023-12-26 20:03:36,645][105692] Updated weights for policy 0, policy_version 644599 (0.0009) [2023-12-26 20:03:36,700][105692] Updated weights for policy 0, policy_version 644609 (0.0009) [2023-12-26 20:03:37,158][105620] Updated weights for policy 1, policy_version 645438 (0.0008) [2023-12-26 20:03:37,212][105620] Updated weights for policy 1, policy_version 645448 (0.0009) [2023-12-26 20:03:37,276][105620] Updated weights for policy 1, policy_version 645458 (0.0008) [2023-12-26 20:03:37,482][105692] Updated weights for policy 0, policy_version 644619 (0.0009) [2023-12-26 20:03:37,542][105692] Updated weights for policy 0, policy_version 644630 (0.0010) [2023-12-26 20:03:37,606][105692] Updated weights for policy 0, policy_version 644642 (0.0010) [2023-12-26 20:03:37,959][105620] Updated weights for policy 1, policy_version 645468 (0.0009) [2023-12-26 20:03:38,007][105620] Updated weights for policy 1, policy_version 645478 (0.0006) [2023-12-26 20:03:38,072][105620] Updated weights for policy 1, policy_version 645488 (0.0007) [2023-12-26 20:03:38,428][105692] Updated weights for policy 0, policy_version 644652 (0.0010) [2023-12-26 20:03:38,498][105692] Updated weights for policy 0, policy_version 644662 (0.0010) [2023-12-26 20:03:38,568][105692] Updated weights for policy 0, policy_version 644672 (0.0009) [2023-12-26 20:03:38,741][105620] Updated weights for policy 1, policy_version 645498 (0.0009) [2023-12-26 20:03:38,812][105620] Updated weights for policy 1, policy_version 645508 (0.0006) [2023-12-26 20:03:38,882][105620] Updated weights for policy 1, policy_version 645518 (0.0007) [2023-12-26 20:03:38,945][105620] Updated weights for policy 1, policy_version 645528 (0.0009) [2023-12-26 20:03:39,394][105692] Updated weights for policy 0, policy_version 644682 (0.0010) [2023-12-26 20:03:39,457][105692] Updated weights for policy 0, policy_version 644692 (0.0008) [2023-12-26 20:03:39,511][105692] Updated weights for policy 0, policy_version 644702 (0.0008) [2023-12-26 20:03:39,565][105692] Updated weights for policy 0, policy_version 644712 (0.0008) [2023-12-26 20:03:39,643][105620] Updated weights for policy 1, policy_version 645538 (0.0011) [2023-12-26 20:03:39,692][105620] Updated weights for policy 1, policy_version 645548 (0.0010) [2023-12-26 20:03:39,748][105620] Updated weights for policy 1, policy_version 645558 (0.0011) [2023-12-26 20:03:40,378][105692] Updated weights for policy 0, policy_version 644722 (0.0008) [2023-12-26 20:03:40,439][105692] Updated weights for policy 0, policy_version 644732 (0.0008) [2023-12-26 20:03:40,494][105692] Updated weights for policy 0, policy_version 644742 (0.0009) [2023-12-26 20:03:40,520][105620] Updated weights for policy 1, policy_version 645568 (0.0008) [2023-12-26 20:03:40,580][105620] Updated weights for policy 1, policy_version 645578 (0.0010) [2023-12-26 20:03:40,637][105620] Updated weights for policy 1, policy_version 645588 (0.0009) [2023-12-26 20:03:41,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.6, 300 sec: 19466.4). Total num frames: 330375168. Throughput: 0: 9540.8, 1: 9739.9. Samples: 330384788. Policy #0 lag: (min: 7.0, avg: 14.9, max: 39.0) [2023-12-26 20:03:41,063][104569] Avg episode reward: [(0, '8905.575'), (1, '9082.869')] [2023-12-26 20:03:41,286][105620] Updated weights for policy 1, policy_version 645598 (0.0011) [2023-12-26 20:03:41,358][105620] Updated weights for policy 1, policy_version 645608 (0.0011) [2023-12-26 20:03:41,391][105692] Updated weights for policy 0, policy_version 644752 (0.0007) [2023-12-26 20:03:41,425][105620] Updated weights for policy 1, policy_version 645618 (0.0008) [2023-12-26 20:03:41,450][105692] Updated weights for policy 0, policy_version 644762 (0.0007) [2023-12-26 20:03:41,500][105692] Updated weights for policy 0, policy_version 644772 (0.0009) [2023-12-26 20:03:42,145][105620] Updated weights for policy 1, policy_version 645628 (0.0009) [2023-12-26 20:03:42,202][105620] Updated weights for policy 1, policy_version 645638 (0.0010) [2023-12-26 20:03:42,261][105620] Updated weights for policy 1, policy_version 645648 (0.0010) [2023-12-26 20:03:42,303][105692] Updated weights for policy 0, policy_version 644782 (0.0009) [2023-12-26 20:03:42,364][105692] Updated weights for policy 0, policy_version 644792 (0.0008) [2023-12-26 20:03:42,430][105692] Updated weights for policy 0, policy_version 644802 (0.0006) [2023-12-26 20:03:42,974][105620] Updated weights for policy 1, policy_version 645658 (0.0011) [2023-12-26 20:03:43,038][105620] Updated weights for policy 1, policy_version 645668 (0.0011) [2023-12-26 20:03:43,101][105620] Updated weights for policy 1, policy_version 645678 (0.0011) [2023-12-26 20:03:43,158][105692] Updated weights for policy 0, policy_version 644812 (0.0009) [2023-12-26 20:03:43,159][105620] Updated weights for policy 1, policy_version 645688 (0.0010) [2023-12-26 20:03:43,218][105692] Updated weights for policy 0, policy_version 644822 (0.0010) [2023-12-26 20:03:43,276][105692] Updated weights for policy 0, policy_version 644832 (0.0010) [2023-12-26 20:03:43,886][105620] Updated weights for policy 1, policy_version 645698 (0.0005) [2023-12-26 20:03:43,945][105620] Updated weights for policy 1, policy_version 645708 (0.0005) [2023-12-26 20:03:44,006][105620] Updated weights for policy 1, policy_version 645718 (0.0005) [2023-12-26 20:03:44,013][105692] Updated weights for policy 0, policy_version 644842 (0.0010) [2023-12-26 20:03:44,071][105692] Updated weights for policy 0, policy_version 644852 (0.0010) [2023-12-26 20:03:44,134][105692] Updated weights for policy 0, policy_version 644862 (0.0011) [2023-12-26 20:03:44,193][105692] Updated weights for policy 0, policy_version 644872 (0.0009) [2023-12-26 20:03:44,528][105620] Updated weights for policy 1, policy_version 645728 (0.0006) [2023-12-26 20:03:44,583][105620] Updated weights for policy 1, policy_version 645738 (0.0005) [2023-12-26 20:03:44,636][105620] Updated weights for policy 1, policy_version 645748 (0.0005) [2023-12-26 20:03:44,850][105692] Updated weights for policy 0, policy_version 644882 (0.0009) [2023-12-26 20:03:44,915][105692] Updated weights for policy 0, policy_version 644892 (0.0009) [2023-12-26 20:03:44,973][105692] Updated weights for policy 0, policy_version 644902 (0.0009) [2023-12-26 20:03:45,249][105620] Updated weights for policy 1, policy_version 645758 (0.0009) [2023-12-26 20:03:45,308][105620] Updated weights for policy 1, policy_version 645768 (0.0005) [2023-12-26 20:03:45,371][105620] Updated weights for policy 1, policy_version 645778 (0.0006) [2023-12-26 20:03:45,790][105692] Updated weights for policy 0, policy_version 644912 (0.0008) [2023-12-26 20:03:45,848][105692] Updated weights for policy 0, policy_version 644922 (0.0008) [2023-12-26 20:03:45,906][105692] Updated weights for policy 0, policy_version 644932 (0.0007) [2023-12-26 20:03:46,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 330473472. Throughput: 0: 9521.5, 1: 9773.3. Samples: 330441080. Policy #0 lag: (min: 7.0, avg: 14.9, max: 39.0) [2023-12-26 20:03:46,062][104569] Avg episode reward: [(0, '8365.064'), (1, '9265.527')] [2023-12-26 20:03:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000644936_165134336.pth... [2023-12-26 20:03:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000643816_164847616.pth [2023-12-26 20:03:46,079][105620] Updated weights for policy 1, policy_version 645788 (0.0011) [2023-12-26 20:03:46,141][105620] Updated weights for policy 1, policy_version 645798 (0.0010) [2023-12-26 20:03:46,206][105620] Updated weights for policy 1, policy_version 645808 (0.0010) [2023-12-26 20:03:46,256][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000645816_165347328.pth... [2023-12-26 20:03:46,261][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000644632_165044224.pth [2023-12-26 20:03:46,524][105692] Updated weights for policy 0, policy_version 644942 (0.0005) [2023-12-26 20:03:46,579][105692] Updated weights for policy 0, policy_version 644952 (0.0009) [2023-12-26 20:03:46,634][105692] Updated weights for policy 0, policy_version 644962 (0.0005) [2023-12-26 20:03:46,822][105620] Updated weights for policy 1, policy_version 645818 (0.0010) [2023-12-26 20:03:46,885][105620] Updated weights for policy 1, policy_version 645828 (0.0006) [2023-12-26 20:03:46,942][105620] Updated weights for policy 1, policy_version 645838 (0.0005) [2023-12-26 20:03:47,010][105620] Updated weights for policy 1, policy_version 645848 (0.0006) [2023-12-26 20:03:47,218][105692] Updated weights for policy 0, policy_version 644972 (0.0005) [2023-12-26 20:03:47,272][105585] KL-divergence is very high: 205.5660 [2023-12-26 20:03:47,280][105692] Updated weights for policy 0, policy_version 644982 (0.0008) [2023-12-26 20:03:47,316][105585] KL-divergence is very high: 299.4276 [2023-12-26 20:03:47,334][105692] Updated weights for policy 0, policy_version 644992 (0.0009) [2023-12-26 20:03:47,367][105585] KL-divergence is very high: 240.6756 [2023-12-26 20:03:47,664][105620] Updated weights for policy 1, policy_version 645858 (0.0010) [2023-12-26 20:03:47,715][105620] Updated weights for policy 1, policy_version 645868 (0.0010) [2023-12-26 20:03:47,766][105620] Updated weights for policy 1, policy_version 645878 (0.0010) [2023-12-26 20:03:48,041][105692] Updated weights for policy 0, policy_version 645002 (0.0010) [2023-12-26 20:03:48,104][105692] Updated weights for policy 0, policy_version 645012 (0.0006) [2023-12-26 20:03:48,161][105692] Updated weights for policy 0, policy_version 645022 (0.0006) [2023-12-26 20:03:48,216][105692] Updated weights for policy 0, policy_version 645032 (0.0006) [2023-12-26 20:03:48,517][105620] Updated weights for policy 1, policy_version 645888 (0.0010) [2023-12-26 20:03:48,572][105620] Updated weights for policy 1, policy_version 645898 (0.0010) [2023-12-26 20:03:48,642][105620] Updated weights for policy 1, policy_version 645908 (0.0010) [2023-12-26 20:03:48,876][105692] Updated weights for policy 0, policy_version 645042 (0.0010) [2023-12-26 20:03:48,930][105692] Updated weights for policy 0, policy_version 645052 (0.0010) [2023-12-26 20:03:48,988][105692] Updated weights for policy 0, policy_version 645062 (0.0010) [2023-12-26 20:03:49,410][105620] Updated weights for policy 1, policy_version 645918 (0.0009) [2023-12-26 20:03:49,477][105620] Updated weights for policy 1, policy_version 645928 (0.0010) [2023-12-26 20:03:49,535][105620] Updated weights for policy 1, policy_version 645938 (0.0009) [2023-12-26 20:03:49,814][105692] Updated weights for policy 0, policy_version 645072 (0.0009) [2023-12-26 20:03:49,816][105585] KL-divergence is very high: 323.7229 [2023-12-26 20:03:49,830][105585] KL-divergence is very high: 178.9377 [2023-12-26 20:03:49,869][105585] KL-divergence is very high: 384.8412 [2023-12-26 20:03:49,881][105692] Updated weights for policy 0, policy_version 645082 (0.0008) [2023-12-26 20:03:49,882][105585] KL-divergence is very high: 136.8282 [2023-12-26 20:03:49,914][105585] KL-divergence is very high: 264.2356 [2023-12-26 20:03:49,941][105692] Updated weights for policy 0, policy_version 645092 (0.0007) [2023-12-26 20:03:50,173][105620] Updated weights for policy 1, policy_version 645948 (0.0009) [2023-12-26 20:03:50,231][105620] Updated weights for policy 1, policy_version 645958 (0.0006) [2023-12-26 20:03:50,295][105620] Updated weights for policy 1, policy_version 645968 (0.0006) [2023-12-26 20:03:50,761][105692] Updated weights for policy 0, policy_version 645102 (0.0009) [2023-12-26 20:03:50,816][105692] Updated weights for policy 0, policy_version 645112 (0.0009) [2023-12-26 20:03:50,870][105692] Updated weights for policy 0, policy_version 645122 (0.0009) [2023-12-26 20:03:50,925][105620] Updated weights for policy 1, policy_version 645978 (0.0006) [2023-12-26 20:03:50,975][105620] Updated weights for policy 1, policy_version 645988 (0.0005) [2023-12-26 20:03:51,039][105620] Updated weights for policy 1, policy_version 645998 (0.0007) [2023-12-26 20:03:51,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 330571776. Throughput: 0: 9601.9, 1: 9873.7. Samples: 330563028. Policy #0 lag: (min: 7.0, avg: 14.9, max: 39.0) [2023-12-26 20:03:51,062][104569] Avg episode reward: [(0, '7736.147'), (1, '9265.473')] [2023-12-26 20:03:51,102][105620] Updated weights for policy 1, policy_version 646008 (0.0009) [2023-12-26 20:03:51,601][105692] Updated weights for policy 0, policy_version 645132 (0.0010) [2023-12-26 20:03:51,663][105692] Updated weights for policy 0, policy_version 645142 (0.0008) [2023-12-26 20:03:51,728][105692] Updated weights for policy 0, policy_version 645152 (0.0010) [2023-12-26 20:03:51,796][105620] Updated weights for policy 1, policy_version 646018 (0.0007) [2023-12-26 20:03:51,856][105620] Updated weights for policy 1, policy_version 646028 (0.0005) [2023-12-26 20:03:51,912][105620] Updated weights for policy 1, policy_version 646038 (0.0005) [2023-12-26 20:03:52,541][105620] Updated weights for policy 1, policy_version 646048 (0.0008) [2023-12-26 20:03:52,555][105692] Updated weights for policy 0, policy_version 645162 (0.0009) [2023-12-26 20:03:52,587][105620] Updated weights for policy 1, policy_version 646058 (0.0007) [2023-12-26 20:03:52,601][105692] Updated weights for policy 0, policy_version 645172 (0.0007) [2023-12-26 20:03:52,636][105620] Updated weights for policy 1, policy_version 646068 (0.0007) [2023-12-26 20:03:52,651][105692] Updated weights for policy 0, policy_version 645182 (0.0006) [2023-12-26 20:03:52,712][105692] Updated weights for policy 0, policy_version 645192 (0.0009) [2023-12-26 20:03:53,234][105620] Updated weights for policy 1, policy_version 646078 (0.0006) [2023-12-26 20:03:53,288][105620] Updated weights for policy 1, policy_version 646088 (0.0005) [2023-12-26 20:03:53,342][105620] Updated weights for policy 1, policy_version 646098 (0.0005) [2023-12-26 20:03:53,616][105692] Updated weights for policy 0, policy_version 645202 (0.0009) [2023-12-26 20:03:53,668][105692] Updated weights for policy 0, policy_version 645212 (0.0010) [2023-12-26 20:03:53,721][105692] Updated weights for policy 0, policy_version 645222 (0.0010) [2023-12-26 20:03:53,963][105620] Updated weights for policy 1, policy_version 646108 (0.0006) [2023-12-26 20:03:54,010][105620] Updated weights for policy 1, policy_version 646118 (0.0008) [2023-12-26 20:03:54,062][105620] Updated weights for policy 1, policy_version 646128 (0.0008) [2023-12-26 20:03:54,482][105692] Updated weights for policy 0, policy_version 645232 (0.0011) [2023-12-26 20:03:54,527][105692] Updated weights for policy 0, policy_version 645242 (0.0010) [2023-12-26 20:03:54,576][105692] Updated weights for policy 0, policy_version 645252 (0.0010) [2023-12-26 20:03:54,789][105620] Updated weights for policy 1, policy_version 646138 (0.0009) [2023-12-26 20:03:54,850][105620] Updated weights for policy 1, policy_version 646148 (0.0008) [2023-12-26 20:03:54,913][105620] Updated weights for policy 1, policy_version 646158 (0.0006) [2023-12-26 20:03:54,975][105620] Updated weights for policy 1, policy_version 646168 (0.0008) [2023-12-26 20:03:55,270][105692] Updated weights for policy 0, policy_version 645262 (0.0007) [2023-12-26 20:03:55,297][105585] KL-divergence is very high: 118.8168 [2023-12-26 20:03:55,302][105585] KL-divergence is very high: 138.7365 [2023-12-26 20:03:55,309][105585] KL-divergence is very high: 137.3372 [2023-12-26 20:03:55,314][105585] KL-divergence is very high: 149.2129 [2023-12-26 20:03:55,322][105585] KL-divergence is very high: 133.0666 [2023-12-26 20:03:55,328][105692] Updated weights for policy 0, policy_version 645272 (0.0009) [2023-12-26 20:03:55,383][105692] Updated weights for policy 0, policy_version 645282 (0.0010) [2023-12-26 20:03:55,633][105620] Updated weights for policy 1, policy_version 646178 (0.0010) [2023-12-26 20:03:55,687][105620] Updated weights for policy 1, policy_version 646188 (0.0010) [2023-12-26 20:03:55,745][105620] Updated weights for policy 1, policy_version 646198 (0.0010) [2023-12-26 20:03:56,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 330670080. Throughput: 0: 9484.1, 1: 9928.7. Samples: 330679780. Policy #0 lag: (min: 7.0, avg: 14.9, max: 39.0) [2023-12-26 20:03:56,063][104569] Avg episode reward: [(0, '6483.992'), (1, '9265.404')] [2023-12-26 20:03:56,099][105692] Updated weights for policy 0, policy_version 645292 (0.0009) [2023-12-26 20:03:56,158][105692] Updated weights for policy 0, policy_version 645302 (0.0010) [2023-12-26 20:03:56,217][105692] Updated weights for policy 0, policy_version 645312 (0.0011) [2023-12-26 20:03:56,479][105620] Updated weights for policy 1, policy_version 646208 (0.0006) [2023-12-26 20:03:56,531][105620] Updated weights for policy 1, policy_version 646218 (0.0006) [2023-12-26 20:03:56,582][105620] Updated weights for policy 1, policy_version 646228 (0.0010) [2023-12-26 20:03:56,962][105692] Updated weights for policy 0, policy_version 645322 (0.0010) [2023-12-26 20:03:57,020][105692] Updated weights for policy 0, policy_version 645332 (0.0010) [2023-12-26 20:03:57,080][105692] Updated weights for policy 0, policy_version 645342 (0.0007) [2023-12-26 20:03:57,127][105692] Updated weights for policy 0, policy_version 645352 (0.0008) [2023-12-26 20:03:57,259][105620] Updated weights for policy 1, policy_version 646238 (0.0010) [2023-12-26 20:03:57,307][105620] Updated weights for policy 1, policy_version 646248 (0.0010) [2023-12-26 20:03:57,365][105620] Updated weights for policy 1, policy_version 646258 (0.0010) [2023-12-26 20:03:57,826][105692] Updated weights for policy 0, policy_version 645362 (0.0008) [2023-12-26 20:03:57,871][105692] Updated weights for policy 0, policy_version 645372 (0.0008) [2023-12-26 20:03:57,928][105692] Updated weights for policy 0, policy_version 645382 (0.0008) [2023-12-26 20:03:58,125][105620] Updated weights for policy 1, policy_version 646268 (0.0010) [2023-12-26 20:03:58,189][105620] Updated weights for policy 1, policy_version 646278 (0.0010) [2023-12-26 20:03:58,255][105620] Updated weights for policy 1, policy_version 646288 (0.0010) [2023-12-26 20:03:58,777][105692] Updated weights for policy 0, policy_version 645392 (0.0007) [2023-12-26 20:03:58,841][105692] Updated weights for policy 0, policy_version 645402 (0.0007) [2023-12-26 20:03:58,913][105692] Updated weights for policy 0, policy_version 645412 (0.0008) [2023-12-26 20:03:58,993][105620] Updated weights for policy 1, policy_version 646298 (0.0010) [2023-12-26 20:03:59,056][105620] Updated weights for policy 1, policy_version 646308 (0.0006) [2023-12-26 20:03:59,114][105620] Updated weights for policy 1, policy_version 646318 (0.0006) [2023-12-26 20:03:59,160][105620] Updated weights for policy 1, policy_version 646328 (0.0007) [2023-12-26 20:03:59,555][105692] Updated weights for policy 0, policy_version 645422 (0.0009) [2023-12-26 20:03:59,621][105692] Updated weights for policy 0, policy_version 645432 (0.0011) [2023-12-26 20:03:59,684][105692] Updated weights for policy 0, policy_version 645442 (0.0007) [2023-12-26 20:03:59,881][105620] Updated weights for policy 1, policy_version 646338 (0.0011) [2023-12-26 20:03:59,942][105620] Updated weights for policy 1, policy_version 646348 (0.0009) [2023-12-26 20:03:59,994][105620] Updated weights for policy 1, policy_version 646358 (0.0008) [2023-12-26 20:04:00,361][105692] Updated weights for policy 0, policy_version 645452 (0.0007) [2023-12-26 20:04:00,424][105692] Updated weights for policy 0, policy_version 645462 (0.0009) [2023-12-26 20:04:00,482][105692] Updated weights for policy 0, policy_version 645472 (0.0009) [2023-12-26 20:04:00,685][105620] Updated weights for policy 1, policy_version 646368 (0.0007) [2023-12-26 20:04:00,758][105620] Updated weights for policy 1, policy_version 646378 (0.0008) [2023-12-26 20:04:00,829][105620] Updated weights for policy 1, policy_version 646388 (0.0009) [2023-12-26 20:04:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 330768384. Throughput: 0: 9485.3, 1: 9970.6. Samples: 330736908. Policy #0 lag: (min: 7.0, avg: 14.9, max: 39.0) [2023-12-26 20:04:01,062][104569] Avg episode reward: [(0, '6756.335'), (1, '9356.169')] [2023-12-26 20:04:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000645480_165273600.pth... [2023-12-26 20:04:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000646392_165494784.pth... [2023-12-26 20:04:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000645208_165191680.pth [2023-12-26 20:04:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000644392_164995072.pth [2023-12-26 20:04:01,194][105692] Updated weights for policy 0, policy_version 645482 (0.0009) [2023-12-26 20:04:01,251][105692] Updated weights for policy 0, policy_version 645492 (0.0010) [2023-12-26 20:04:01,317][105692] Updated weights for policy 0, policy_version 645502 (0.0010) [2023-12-26 20:04:01,393][105692] Updated weights for policy 0, policy_version 645512 (0.0010) [2023-12-26 20:04:01,511][105620] Updated weights for policy 1, policy_version 646398 (0.0009) [2023-12-26 20:04:01,569][105620] Updated weights for policy 1, policy_version 646408 (0.0008) [2023-12-26 20:04:01,631][105620] Updated weights for policy 1, policy_version 646418 (0.0008) [2023-12-26 20:04:02,120][105692] Updated weights for policy 0, policy_version 645522 (0.0010) [2023-12-26 20:04:02,172][105692] Updated weights for policy 0, policy_version 645532 (0.0010) [2023-12-26 20:04:02,216][105692] Updated weights for policy 0, policy_version 645542 (0.0010) [2023-12-26 20:04:02,355][105620] Updated weights for policy 1, policy_version 646428 (0.0008) [2023-12-26 20:04:02,414][105620] Updated weights for policy 1, policy_version 646438 (0.0008) [2023-12-26 20:04:02,466][105620] Updated weights for policy 1, policy_version 646448 (0.0008) [2023-12-26 20:04:02,970][105692] Updated weights for policy 0, policy_version 645552 (0.0008) [2023-12-26 20:04:03,032][105692] Updated weights for policy 0, policy_version 645562 (0.0008) [2023-12-26 20:04:03,087][105692] Updated weights for policy 0, policy_version 645572 (0.0006) [2023-12-26 20:04:03,204][105620] Updated weights for policy 1, policy_version 646458 (0.0008) [2023-12-26 20:04:03,252][105620] Updated weights for policy 1, policy_version 646468 (0.0008) [2023-12-26 20:04:03,299][105620] Updated weights for policy 1, policy_version 646478 (0.0007) [2023-12-26 20:04:03,350][105620] Updated weights for policy 1, policy_version 646488 (0.0008) [2023-12-26 20:04:03,754][105692] Updated weights for policy 0, policy_version 645582 (0.0009) [2023-12-26 20:04:03,808][105692] Updated weights for policy 0, policy_version 645592 (0.0010) [2023-12-26 20:04:03,870][105692] Updated weights for policy 0, policy_version 645602 (0.0011) [2023-12-26 20:04:04,140][105620] Updated weights for policy 1, policy_version 646498 (0.0008) [2023-12-26 20:04:04,200][105620] Updated weights for policy 1, policy_version 646508 (0.0008) [2023-12-26 20:04:04,256][105620] Updated weights for policy 1, policy_version 646518 (0.0008) [2023-12-26 20:04:04,624][105692] Updated weights for policy 0, policy_version 645612 (0.0010) [2023-12-26 20:04:04,674][105692] Updated weights for policy 0, policy_version 645622 (0.0006) [2023-12-26 20:04:04,734][105692] Updated weights for policy 0, policy_version 645632 (0.0005) [2023-12-26 20:04:05,056][105620] Updated weights for policy 1, policy_version 646528 (0.0006) [2023-12-26 20:04:05,101][105620] Updated weights for policy 1, policy_version 646538 (0.0008) [2023-12-26 20:04:05,153][105620] Updated weights for policy 1, policy_version 646548 (0.0008) [2023-12-26 20:04:05,406][105692] Updated weights for policy 0, policy_version 645642 (0.0006) [2023-12-26 20:04:05,458][105692] Updated weights for policy 0, policy_version 645652 (0.0008) [2023-12-26 20:04:05,513][105692] Updated weights for policy 0, policy_version 645662 (0.0008) [2023-12-26 20:04:05,568][105692] Updated weights for policy 0, policy_version 645672 (0.0008) [2023-12-26 20:04:05,912][105586] KL-divergence is very high: 110.6457 [2023-12-26 20:04:05,916][105620] Updated weights for policy 1, policy_version 646558 (0.0007) [2023-12-26 20:04:05,935][105586] KL-divergence is very high: 141.3588 [2023-12-26 20:04:05,941][105586] KL-divergence is very high: 197.6772 [2023-12-26 20:04:05,947][105586] KL-divergence is very high: 139.1920 [2023-12-26 20:04:05,960][105586] KL-divergence is very high: 130.7857 [2023-12-26 20:04:05,979][105620] Updated weights for policy 1, policy_version 646568 (0.0006) [2023-12-26 20:04:05,990][105586] KL-divergence is very high: 100.0619 [2023-12-26 20:04:06,034][105620] Updated weights for policy 1, policy_version 646578 (0.0007) [2023-12-26 20:04:06,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.8, 300 sec: 19438.7). Total num frames: 330858496. Throughput: 0: 9474.3, 1: 9915.4. Samples: 330852420. Policy #0 lag: (min: 7.0, avg: 14.9, max: 39.0) [2023-12-26 20:04:06,062][104569] Avg episode reward: [(0, '8004.790'), (1, '6017.527')] [2023-12-26 20:04:06,250][105692] Updated weights for policy 0, policy_version 645682 (0.0011) [2023-12-26 20:04:06,318][105692] Updated weights for policy 0, policy_version 645692 (0.0008) [2023-12-26 20:04:06,381][105692] Updated weights for policy 0, policy_version 645702 (0.0008) [2023-12-26 20:04:06,738][105620] Updated weights for policy 1, policy_version 646588 (0.0007) [2023-12-26 20:04:06,787][105620] Updated weights for policy 1, policy_version 646598 (0.0008) [2023-12-26 20:04:06,839][105620] Updated weights for policy 1, policy_version 646608 (0.0007) [2023-12-26 20:04:07,042][105692] Updated weights for policy 0, policy_version 645712 (0.0010) [2023-12-26 20:04:07,096][105692] Updated weights for policy 0, policy_version 645722 (0.0011) [2023-12-26 20:04:07,160][105692] Updated weights for policy 0, policy_version 645732 (0.0011) [2023-12-26 20:04:07,445][105620] Updated weights for policy 1, policy_version 646618 (0.0008) [2023-12-26 20:04:07,495][105620] Updated weights for policy 1, policy_version 646628 (0.0005) [2023-12-26 20:04:07,565][105620] Updated weights for policy 1, policy_version 646638 (0.0005) [2023-12-26 20:04:07,625][105620] Updated weights for policy 1, policy_version 646648 (0.0005) [2023-12-26 20:04:07,887][105692] Updated weights for policy 0, policy_version 645742 (0.0011) [2023-12-26 20:04:07,956][105692] Updated weights for policy 0, policy_version 645752 (0.0011) [2023-12-26 20:04:08,015][105692] Updated weights for policy 0, policy_version 645762 (0.0011) [2023-12-26 20:04:08,145][105620] Updated weights for policy 1, policy_version 646658 (0.0008) [2023-12-26 20:04:08,205][105620] Updated weights for policy 1, policy_version 646668 (0.0007) [2023-12-26 20:04:08,252][105620] Updated weights for policy 1, policy_version 646678 (0.0010) [2023-12-26 20:04:08,657][105692] Updated weights for policy 0, policy_version 645772 (0.0008) [2023-12-26 20:04:08,735][105692] Updated weights for policy 0, policy_version 645782 (0.0007) [2023-12-26 20:04:08,796][105692] Updated weights for policy 0, policy_version 645792 (0.0009) [2023-12-26 20:04:08,989][105620] Updated weights for policy 1, policy_version 646688 (0.0011) [2023-12-26 20:04:09,046][105620] Updated weights for policy 1, policy_version 646698 (0.0011) [2023-12-26 20:04:09,101][105620] Updated weights for policy 1, policy_version 646708 (0.0010) [2023-12-26 20:04:09,442][105692] Updated weights for policy 0, policy_version 645802 (0.0010) [2023-12-26 20:04:09,494][105692] Updated weights for policy 0, policy_version 645812 (0.0011) [2023-12-26 20:04:09,547][105692] Updated weights for policy 0, policy_version 645822 (0.0010) [2023-12-26 20:04:09,596][105692] Updated weights for policy 0, policy_version 645832 (0.0009) [2023-12-26 20:04:09,816][105620] Updated weights for policy 1, policy_version 646718 (0.0010) [2023-12-26 20:04:09,882][105620] Updated weights for policy 1, policy_version 646728 (0.0012) [2023-12-26 20:04:09,951][105620] Updated weights for policy 1, policy_version 646738 (0.0011) [2023-12-26 20:04:10,407][105692] Updated weights for policy 0, policy_version 645842 (0.0011) [2023-12-26 20:04:10,471][105692] Updated weights for policy 0, policy_version 645852 (0.0011) [2023-12-26 20:04:10,531][105692] Updated weights for policy 0, policy_version 645862 (0.0011) [2023-12-26 20:04:10,685][105620] Updated weights for policy 1, policy_version 646748 (0.0010) [2023-12-26 20:04:10,751][105620] Updated weights for policy 1, policy_version 646758 (0.0009) [2023-12-26 20:04:10,817][105620] Updated weights for policy 1, policy_version 646768 (0.0010) [2023-12-26 20:04:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 330964992. Throughput: 0: 9519.1, 1: 9959.9. Samples: 330972288. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-26 20:04:11,063][104569] Avg episode reward: [(0, '8268.942'), (1, '5817.350')] [2023-12-26 20:04:11,198][105692] Updated weights for policy 0, policy_version 645872 (0.0009) [2023-12-26 20:04:11,260][105692] Updated weights for policy 0, policy_version 645882 (0.0007) [2023-12-26 20:04:11,313][105692] Updated weights for policy 0, policy_version 645892 (0.0008) [2023-12-26 20:04:11,579][105620] Updated weights for policy 1, policy_version 646778 (0.0009) [2023-12-26 20:04:11,642][105620] Updated weights for policy 1, policy_version 646788 (0.0008) [2023-12-26 20:04:11,712][105620] Updated weights for policy 1, policy_version 646798 (0.0008) [2023-12-26 20:04:11,781][105620] Updated weights for policy 1, policy_version 646808 (0.0008) [2023-12-26 20:04:12,018][105692] Updated weights for policy 0, policy_version 645902 (0.0007) [2023-12-26 20:04:12,079][105692] Updated weights for policy 0, policy_version 645912 (0.0007) [2023-12-26 20:04:12,134][105692] Updated weights for policy 0, policy_version 645922 (0.0005) [2023-12-26 20:04:12,543][105620] Updated weights for policy 1, policy_version 646818 (0.0009) [2023-12-26 20:04:12,600][105620] Updated weights for policy 1, policy_version 646828 (0.0009) [2023-12-26 20:04:12,655][105620] Updated weights for policy 1, policy_version 646838 (0.0009) [2023-12-26 20:04:12,809][105692] Updated weights for policy 0, policy_version 645932 (0.0007) [2023-12-26 20:04:12,852][105692] Updated weights for policy 0, policy_version 645942 (0.0006) [2023-12-26 20:04:12,900][105692] Updated weights for policy 0, policy_version 645952 (0.0008) [2023-12-26 20:04:13,430][105620] Updated weights for policy 1, policy_version 646848 (0.0011) [2023-12-26 20:04:13,502][105620] Updated weights for policy 1, policy_version 646858 (0.0010) [2023-12-26 20:04:13,564][105620] Updated weights for policy 1, policy_version 646868 (0.0010) [2023-12-26 20:04:13,617][105692] Updated weights for policy 0, policy_version 645962 (0.0008) [2023-12-26 20:04:13,672][105692] Updated weights for policy 0, policy_version 645972 (0.0005) [2023-12-26 20:04:13,731][105692] Updated weights for policy 0, policy_version 645982 (0.0006) [2023-12-26 20:04:14,279][105692] Updated weights for policy 0, policy_version 645993 (0.0011) [2023-12-26 20:04:14,292][105620] Updated weights for policy 1, policy_version 646878 (0.0011) [2023-12-26 20:04:14,317][105585] KL-divergence is very high: 230.0579 [2023-12-26 20:04:14,330][105692] Updated weights for policy 0, policy_version 646003 (0.0009) [2023-12-26 20:04:14,348][105620] Updated weights for policy 1, policy_version 646888 (0.0010) [2023-12-26 20:04:14,359][105585] KL-divergence is very high: 344.8360 [2023-12-26 20:04:14,383][105692] Updated weights for policy 0, policy_version 646013 (0.0008) [2023-12-26 20:04:14,392][105620] Updated weights for policy 1, policy_version 646898 (0.0010) [2023-12-26 20:04:14,400][105585] KL-divergence is very high: 265.4619 [2023-12-26 20:04:14,441][105692] Updated weights for policy 0, policy_version 646023 (0.0010) [2023-12-26 20:04:15,036][105620] Updated weights for policy 1, policy_version 646908 (0.0009) [2023-12-26 20:04:15,106][105620] Updated weights for policy 1, policy_version 646918 (0.0011) [2023-12-26 20:04:15,172][105620] Updated weights for policy 1, policy_version 646928 (0.0011) [2023-12-26 20:04:15,200][105692] Updated weights for policy 0, policy_version 646033 (0.0010) [2023-12-26 20:04:15,266][105692] Updated weights for policy 0, policy_version 646043 (0.0010) [2023-12-26 20:04:15,326][105692] Updated weights for policy 0, policy_version 646053 (0.0011) [2023-12-26 20:04:15,846][105620] Updated weights for policy 1, policy_version 646938 (0.0010) [2023-12-26 20:04:15,900][105620] Updated weights for policy 1, policy_version 646948 (0.0006) [2023-12-26 20:04:15,949][105620] Updated weights for policy 1, policy_version 646958 (0.0008) [2023-12-26 20:04:16,004][105620] Updated weights for policy 1, policy_version 646968 (0.0008) [2023-12-26 20:04:16,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 331063296. Throughput: 0: 9542.6, 1: 9928.0. Samples: 331030292. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-26 20:04:16,062][104569] Avg episode reward: [(0, '8723.876'), (1, '7822.376')] [2023-12-26 20:04:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000646968_165642240.pth... [2023-12-26 20:04:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000645816_165347328.pth [2023-12-26 20:04:16,083][105692] Updated weights for policy 0, policy_version 646063 (0.0011) [2023-12-26 20:04:16,145][105692] Updated weights for policy 0, policy_version 646073 (0.0010) [2023-12-26 20:04:16,206][105692] Updated weights for policy 0, policy_version 646083 (0.0010) [2023-12-26 20:04:16,232][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000646088_165429248.pth... [2023-12-26 20:04:16,236][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000644936_165134336.pth [2023-12-26 20:04:16,768][105620] Updated weights for policy 1, policy_version 646978 (0.0007) [2023-12-26 20:04:16,820][105620] Updated weights for policy 1, policy_version 646988 (0.0006) [2023-12-26 20:04:16,879][105620] Updated weights for policy 1, policy_version 646998 (0.0005) [2023-12-26 20:04:16,935][105692] Updated weights for policy 0, policy_version 646093 (0.0010) [2023-12-26 20:04:16,983][105692] Updated weights for policy 0, policy_version 646103 (0.0010) [2023-12-26 20:04:17,031][105692] Updated weights for policy 0, policy_version 646113 (0.0009) [2023-12-26 20:04:17,596][105620] Updated weights for policy 1, policy_version 647008 (0.0008) [2023-12-26 20:04:17,653][105620] Updated weights for policy 1, policy_version 647018 (0.0008) [2023-12-26 20:04:17,712][105620] Updated weights for policy 1, policy_version 647028 (0.0008) [2023-12-26 20:04:17,782][105692] Updated weights for policy 0, policy_version 646123 (0.0009) [2023-12-26 20:04:17,831][105692] Updated weights for policy 0, policy_version 646133 (0.0007) [2023-12-26 20:04:17,883][105692] Updated weights for policy 0, policy_version 646143 (0.0010) [2023-12-26 20:04:18,497][105620] Updated weights for policy 1, policy_version 647038 (0.0009) [2023-12-26 20:04:18,567][105620] Updated weights for policy 1, policy_version 647048 (0.0009) [2023-12-26 20:04:18,582][105692] Updated weights for policy 0, policy_version 646153 (0.0010) [2023-12-26 20:04:18,628][105620] Updated weights for policy 1, policy_version 647058 (0.0009) [2023-12-26 20:04:18,639][105692] Updated weights for policy 0, policy_version 646163 (0.0006) [2023-12-26 20:04:18,698][105692] Updated weights for policy 0, policy_version 646173 (0.0007) [2023-12-26 20:04:18,760][105692] Updated weights for policy 0, policy_version 646183 (0.0009) [2023-12-26 20:04:19,381][105620] Updated weights for policy 1, policy_version 647068 (0.0008) [2023-12-26 20:04:19,434][105620] Updated weights for policy 1, policy_version 647078 (0.0006) [2023-12-26 20:04:19,494][105620] Updated weights for policy 1, policy_version 647088 (0.0008) [2023-12-26 20:04:19,534][105692] Updated weights for policy 0, policy_version 646193 (0.0007) [2023-12-26 20:04:19,597][105692] Updated weights for policy 0, policy_version 646203 (0.0008) [2023-12-26 20:04:19,659][105692] Updated weights for policy 0, policy_version 646213 (0.0009) [2023-12-26 20:04:20,278][105620] Updated weights for policy 1, policy_version 647098 (0.0008) [2023-12-26 20:04:20,342][105620] Updated weights for policy 1, policy_version 647108 (0.0008) [2023-12-26 20:04:20,356][105692] Updated weights for policy 0, policy_version 646223 (0.0008) [2023-12-26 20:04:20,404][105620] Updated weights for policy 1, policy_version 647118 (0.0007) [2023-12-26 20:04:20,415][105692] Updated weights for policy 0, policy_version 646233 (0.0007) [2023-12-26 20:04:20,469][105620] Updated weights for policy 1, policy_version 647128 (0.0009) [2023-12-26 20:04:20,471][105692] Updated weights for policy 0, policy_version 646243 (0.0006) [2023-12-26 20:04:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19438.7). Total num frames: 331153408. Throughput: 0: 9518.8, 1: 9875.6. Samples: 331146364. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-26 20:04:21,063][104569] Avg episode reward: [(0, '8815.353'), (1, '9002.050')] [2023-12-26 20:04:21,212][105692] Updated weights for policy 0, policy_version 646253 (0.0008) [2023-12-26 20:04:21,270][105620] Updated weights for policy 1, policy_version 647138 (0.0009) [2023-12-26 20:04:21,281][105692] Updated weights for policy 0, policy_version 646263 (0.0008) [2023-12-26 20:04:21,337][105620] Updated weights for policy 1, policy_version 647148 (0.0007) [2023-12-26 20:04:21,344][105692] Updated weights for policy 0, policy_version 646273 (0.0007) [2023-12-26 20:04:21,409][105620] Updated weights for policy 1, policy_version 647158 (0.0008) [2023-12-26 20:04:22,033][105692] Updated weights for policy 0, policy_version 646283 (0.0008) [2023-12-26 20:04:22,095][105692] Updated weights for policy 0, policy_version 646293 (0.0010) [2023-12-26 20:04:22,120][105585] KL-divergence is very high: 145.3196 [2023-12-26 20:04:22,156][105692] Updated weights for policy 0, policy_version 646303 (0.0009) [2023-12-26 20:04:22,169][105585] KL-divergence is very high: 104.3019 [2023-12-26 20:04:22,186][105620] Updated weights for policy 1, policy_version 647168 (0.0006) [2023-12-26 20:04:22,245][105620] Updated weights for policy 1, policy_version 647178 (0.0006) [2023-12-26 20:04:22,314][105620] Updated weights for policy 1, policy_version 647188 (0.0009) [2023-12-26 20:04:22,946][105692] Updated weights for policy 0, policy_version 646313 (0.0010) [2023-12-26 20:04:22,999][105692] Updated weights for policy 0, policy_version 646323 (0.0008) [2023-12-26 20:04:23,014][105620] Updated weights for policy 1, policy_version 647198 (0.0007) [2023-12-26 20:04:23,056][105692] Updated weights for policy 0, policy_version 646333 (0.0007) [2023-12-26 20:04:23,065][105620] Updated weights for policy 1, policy_version 647208 (0.0006) [2023-12-26 20:04:23,111][105692] Updated weights for policy 0, policy_version 646343 (0.0006) [2023-12-26 20:04:23,124][105620] Updated weights for policy 1, policy_version 647218 (0.0008) [2023-12-26 20:04:23,754][105692] Updated weights for policy 0, policy_version 646353 (0.0006) [2023-12-26 20:04:23,801][105692] Updated weights for policy 0, policy_version 646363 (0.0009) [2023-12-26 20:04:23,850][105692] Updated weights for policy 0, policy_version 646373 (0.0007) [2023-12-26 20:04:23,968][105620] Updated weights for policy 1, policy_version 647228 (0.0009) [2023-12-26 20:04:24,022][105620] Updated weights for policy 1, policy_version 647238 (0.0010) [2023-12-26 20:04:24,076][105620] Updated weights for policy 1, policy_version 647249 (0.0010) [2023-12-26 20:04:24,389][105692] Updated weights for policy 0, policy_version 646383 (0.0006) [2023-12-26 20:04:24,449][105692] Updated weights for policy 0, policy_version 646393 (0.0005) [2023-12-26 20:04:24,503][105692] Updated weights for policy 0, policy_version 646403 (0.0006) [2023-12-26 20:04:24,996][105620] Updated weights for policy 1, policy_version 647260 (0.0008) [2023-12-26 20:04:25,043][105692] Updated weights for policy 0, policy_version 646413 (0.0006) [2023-12-26 20:04:25,050][105620] Updated weights for policy 1, policy_version 647270 (0.0008) [2023-12-26 20:04:25,100][105620] Updated weights for policy 1, policy_version 647280 (0.0007) [2023-12-26 20:04:25,125][105692] Updated weights for policy 0, policy_version 646423 (0.0011) [2023-12-26 20:04:25,188][105692] Updated weights for policy 0, policy_version 646433 (0.0011) [2023-12-26 20:04:25,862][105620] Updated weights for policy 1, policy_version 647290 (0.0007) [2023-12-26 20:04:25,886][105692] Updated weights for policy 0, policy_version 646443 (0.0010) [2023-12-26 20:04:25,918][105620] Updated weights for policy 1, policy_version 647300 (0.0009) [2023-12-26 20:04:25,944][105692] Updated weights for policy 0, policy_version 646453 (0.0010) [2023-12-26 20:04:25,974][105620] Updated weights for policy 1, policy_version 647310 (0.0006) [2023-12-26 20:04:26,002][105692] Updated weights for policy 0, policy_version 646463 (0.0010) [2023-12-26 20:04:26,025][105620] Updated weights for policy 1, policy_version 647320 (0.0006) [2023-12-26 20:04:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 331259904. Throughput: 0: 9727.7, 1: 9743.7. Samples: 331260996. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-26 20:04:26,063][104569] Avg episode reward: [(0, '8723.895'), (1, '9095.348')] [2023-12-26 20:04:26,699][105692] Updated weights for policy 0, policy_version 646473 (0.0010) [2023-12-26 20:04:26,743][105692] Updated weights for policy 0, policy_version 646483 (0.0010) [2023-12-26 20:04:26,794][105692] Updated weights for policy 0, policy_version 646493 (0.0010) [2023-12-26 20:04:26,808][105620] Updated weights for policy 1, policy_version 647330 (0.0005) [2023-12-26 20:04:26,849][105692] Updated weights for policy 0, policy_version 646503 (0.0010) [2023-12-26 20:04:26,859][105620] Updated weights for policy 1, policy_version 647340 (0.0005) [2023-12-26 20:04:26,906][105620] Updated weights for policy 1, policy_version 647350 (0.0008) [2023-12-26 20:04:27,494][105692] Updated weights for policy 0, policy_version 646513 (0.0009) [2023-12-26 20:04:27,539][105692] Updated weights for policy 0, policy_version 646523 (0.0006) [2023-12-26 20:04:27,558][105585] KL-divergence is very high: 101.5131 [2023-12-26 20:04:27,587][105692] Updated weights for policy 0, policy_version 646533 (0.0005) [2023-12-26 20:04:27,737][105620] Updated weights for policy 1, policy_version 647360 (0.0010) [2023-12-26 20:04:27,790][105620] Updated weights for policy 1, policy_version 647370 (0.0010) [2023-12-26 20:04:27,850][105620] Updated weights for policy 1, policy_version 647381 (0.0009) [2023-12-26 20:04:28,155][105692] Updated weights for policy 0, policy_version 646543 (0.0008) [2023-12-26 20:04:28,217][105692] Updated weights for policy 0, policy_version 646553 (0.0009) [2023-12-26 20:04:28,274][105585] KL-divergence is very high: 827.1431 [2023-12-26 20:04:28,282][105692] Updated weights for policy 0, policy_version 646563 (0.0009) [2023-12-26 20:04:28,656][105620] Updated weights for policy 1, policy_version 647391 (0.0009) [2023-12-26 20:04:28,713][105620] Updated weights for policy 1, policy_version 647401 (0.0008) [2023-12-26 20:04:28,776][105620] Updated weights for policy 1, policy_version 647411 (0.0009) [2023-12-26 20:04:29,009][105585] KL-divergence is very high: 673.3192 [2023-12-26 20:04:29,028][105692] Updated weights for policy 0, policy_version 646573 (0.0009) [2023-12-26 20:04:29,047][105585] KL-divergence is very high: 461.6587 [2023-12-26 20:04:29,080][105692] Updated weights for policy 0, policy_version 646583 (0.0008) [2023-12-26 20:04:29,092][105585] KL-divergence is very high: 270.9841 [2023-12-26 20:04:29,135][105585] KL-divergence is very high: 163.6690 [2023-12-26 20:04:29,136][105692] Updated weights for policy 0, policy_version 646593 (0.0008) [2023-12-26 20:04:29,525][105620] Updated weights for policy 1, policy_version 647421 (0.0007) [2023-12-26 20:04:29,575][105620] Updated weights for policy 1, policy_version 647431 (0.0008) [2023-12-26 20:04:29,625][105620] Updated weights for policy 1, policy_version 647441 (0.0007) [2023-12-26 20:04:29,762][105692] Updated weights for policy 0, policy_version 646603 (0.0008) [2023-12-26 20:04:29,830][105692] Updated weights for policy 0, policy_version 646613 (0.0009) [2023-12-26 20:04:29,890][105692] Updated weights for policy 0, policy_version 646623 (0.0007) [2023-12-26 20:04:30,316][105620] Updated weights for policy 1, policy_version 647451 (0.0007) [2023-12-26 20:04:30,371][105620] Updated weights for policy 1, policy_version 647461 (0.0009) [2023-12-26 20:04:30,418][105620] Updated weights for policy 1, policy_version 647471 (0.0005) [2023-12-26 20:04:30,625][105692] Updated weights for policy 0, policy_version 646633 (0.0007) [2023-12-26 20:04:30,689][105692] Updated weights for policy 0, policy_version 646643 (0.0005) [2023-12-26 20:04:30,745][105692] Updated weights for policy 0, policy_version 646653 (0.0007) [2023-12-26 20:04:30,798][105692] Updated weights for policy 0, policy_version 646663 (0.0010) [2023-12-26 20:04:30,987][105620] Updated weights for policy 1, policy_version 647481 (0.0006) [2023-12-26 20:04:31,051][105620] Updated weights for policy 1, policy_version 647491 (0.0010) [2023-12-26 20:04:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 331350016. Throughput: 0: 9824.9, 1: 9678.1. Samples: 331318716. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-26 20:04:31,062][104569] Avg episode reward: [(0, '8544.816'), (1, '9002.983')] [2023-12-26 20:04:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000646664_165576704.pth... [2023-12-26 20:04:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000645480_165273600.pth [2023-12-26 20:04:31,112][105620] Updated weights for policy 1, policy_version 647501 (0.0011) [2023-12-26 20:04:31,173][105620] Updated weights for policy 1, policy_version 647511 (0.0011) [2023-12-26 20:04:31,178][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000647512_165781504.pth... [2023-12-26 20:04:31,183][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000646392_165494784.pth [2023-12-26 20:04:31,507][105692] Updated weights for policy 0, policy_version 646673 (0.0009) [2023-12-26 20:04:31,569][105692] Updated weights for policy 0, policy_version 646683 (0.0009) [2023-12-26 20:04:31,634][105692] Updated weights for policy 0, policy_version 646693 (0.0008) [2023-12-26 20:04:31,946][105620] Updated weights for policy 1, policy_version 647521 (0.0010) [2023-12-26 20:04:32,008][105620] Updated weights for policy 1, policy_version 647531 (0.0010) [2023-12-26 20:04:32,066][105620] Updated weights for policy 1, policy_version 647541 (0.0010) [2023-12-26 20:04:32,407][105692] Updated weights for policy 0, policy_version 646703 (0.0008) [2023-12-26 20:04:32,462][105692] Updated weights for policy 0, policy_version 646713 (0.0008) [2023-12-26 20:04:32,511][105692] Updated weights for policy 0, policy_version 646723 (0.0008) [2023-12-26 20:04:32,786][105620] Updated weights for policy 1, policy_version 647551 (0.0007) [2023-12-26 20:04:32,842][105620] Updated weights for policy 1, policy_version 647561 (0.0006) [2023-12-26 20:04:32,850][105586] KL-divergence is very high: 104.7444 [2023-12-26 20:04:32,894][105586] KL-divergence is very high: 186.8483 [2023-12-26 20:04:32,902][105620] Updated weights for policy 1, policy_version 647571 (0.0006) [2023-12-26 20:04:33,188][105692] Updated weights for policy 0, policy_version 646733 (0.0008) [2023-12-26 20:04:33,236][105692] Updated weights for policy 0, policy_version 646743 (0.0006) [2023-12-26 20:04:33,287][105692] Updated weights for policy 0, policy_version 646753 (0.0008) [2023-12-26 20:04:33,596][105620] Updated weights for policy 1, policy_version 647581 (0.0008) [2023-12-26 20:04:33,645][105620] Updated weights for policy 1, policy_version 647591 (0.0005) [2023-12-26 20:04:33,691][105620] Updated weights for policy 1, policy_version 647601 (0.0005) [2023-12-26 20:04:33,881][105692] Updated weights for policy 0, policy_version 646763 (0.0007) [2023-12-26 20:04:33,935][105585] KL-divergence is very high: 102.8669 [2023-12-26 20:04:33,938][105692] Updated weights for policy 0, policy_version 646773 (0.0006) [2023-12-26 20:04:33,974][105585] KL-divergence is very high: 176.9118 [2023-12-26 20:04:33,997][105692] Updated weights for policy 0, policy_version 646784 (0.0010) [2023-12-26 20:04:34,020][105585] KL-divergence is very high: 191.7785 [2023-12-26 20:04:34,251][105620] Updated weights for policy 1, policy_version 647611 (0.0006) [2023-12-26 20:04:34,313][105620] Updated weights for policy 1, policy_version 647621 (0.0008) [2023-12-26 20:04:34,377][105620] Updated weights for policy 1, policy_version 647631 (0.0008) [2023-12-26 20:04:34,674][105692] Updated weights for policy 0, policy_version 646795 (0.0010) [2023-12-26 20:04:34,735][105692] Updated weights for policy 0, policy_version 646805 (0.0008) [2023-12-26 20:04:34,798][105692] Updated weights for policy 0, policy_version 646815 (0.0008) [2023-12-26 20:04:35,116][105620] Updated weights for policy 1, policy_version 647641 (0.0007) [2023-12-26 20:04:35,170][105620] Updated weights for policy 1, policy_version 647651 (0.0010) [2023-12-26 20:04:35,222][105620] Updated weights for policy 1, policy_version 647662 (0.0009) [2023-12-26 20:04:35,484][105692] Updated weights for policy 0, policy_version 646825 (0.0009) [2023-12-26 20:04:35,536][105692] Updated weights for policy 0, policy_version 646835 (0.0006) [2023-12-26 20:04:35,593][105692] Updated weights for policy 0, policy_version 646845 (0.0008) [2023-12-26 20:04:35,643][105692] Updated weights for policy 0, policy_version 646856 (0.0009) [2023-12-26 20:04:35,958][105620] Updated weights for policy 1, policy_version 647673 (0.0010) [2023-12-26 20:04:36,011][105620] Updated weights for policy 1, policy_version 647683 (0.0010) [2023-12-26 20:04:36,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 331448320. Throughput: 0: 9837.0, 1: 9658.2. Samples: 331440312. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-26 20:04:36,062][104569] Avg episode reward: [(0, '8458.702'), (1, '8651.079')] [2023-12-26 20:04:36,073][105620] Updated weights for policy 1, policy_version 647694 (0.0010) [2023-12-26 20:04:36,122][105620] Updated weights for policy 1, policy_version 647704 (0.0009) [2023-12-26 20:04:36,321][105692] Updated weights for policy 0, policy_version 646866 (0.0005) [2023-12-26 20:04:36,382][105692] Updated weights for policy 0, policy_version 646876 (0.0008) [2023-12-26 20:04:36,441][105692] Updated weights for policy 0, policy_version 646886 (0.0009) [2023-12-26 20:04:36,979][105620] Updated weights for policy 1, policy_version 647714 (0.0010) [2023-12-26 20:04:37,018][105692] Updated weights for policy 0, policy_version 646896 (0.0008) [2023-12-26 20:04:37,033][105620] Updated weights for policy 1, policy_version 647724 (0.0005) [2023-12-26 20:04:37,079][105692] Updated weights for policy 0, policy_version 646906 (0.0009) [2023-12-26 20:04:37,089][105620] Updated weights for policy 1, policy_version 647734 (0.0006) [2023-12-26 20:04:37,143][105692] Updated weights for policy 0, policy_version 646916 (0.0008) [2023-12-26 20:04:37,846][105692] Updated weights for policy 0, policy_version 646926 (0.0007) [2023-12-26 20:04:37,895][105692] Updated weights for policy 0, policy_version 646936 (0.0005) [2023-12-26 20:04:37,903][105620] Updated weights for policy 1, policy_version 647744 (0.0008) [2023-12-26 20:04:37,946][105692] Updated weights for policy 0, policy_version 646946 (0.0007) [2023-12-26 20:04:37,960][105620] Updated weights for policy 1, policy_version 647754 (0.0006) [2023-12-26 20:04:38,025][105620] Updated weights for policy 1, policy_version 647764 (0.0005) [2023-12-26 20:04:38,600][105692] Updated weights for policy 0, policy_version 646956 (0.0010) [2023-12-26 20:04:38,651][105692] Updated weights for policy 0, policy_version 646966 (0.0009) [2023-12-26 20:04:38,707][105620] Updated weights for policy 1, policy_version 647774 (0.0007) [2023-12-26 20:04:38,713][105692] Updated weights for policy 0, policy_version 646976 (0.0008) [2023-12-26 20:04:38,763][105620] Updated weights for policy 1, policy_version 647784 (0.0008) [2023-12-26 20:04:38,810][105620] Updated weights for policy 1, policy_version 647794 (0.0008) [2023-12-26 20:04:39,438][105692] Updated weights for policy 0, policy_version 646986 (0.0007) [2023-12-26 20:04:39,491][105692] Updated weights for policy 0, policy_version 646996 (0.0010) [2023-12-26 20:04:39,552][105692] Updated weights for policy 0, policy_version 647006 (0.0009) [2023-12-26 20:04:39,604][105620] Updated weights for policy 1, policy_version 647804 (0.0007) [2023-12-26 20:04:39,608][105692] Updated weights for policy 0, policy_version 647016 (0.0008) [2023-12-26 20:04:39,658][105620] Updated weights for policy 1, policy_version 647814 (0.0009) [2023-12-26 20:04:39,713][105620] Updated weights for policy 1, policy_version 647824 (0.0010) [2023-12-26 20:04:40,359][105692] Updated weights for policy 0, policy_version 647026 (0.0009) [2023-12-26 20:04:40,414][105692] Updated weights for policy 0, policy_version 647036 (0.0009) [2023-12-26 20:04:40,476][105692] Updated weights for policy 0, policy_version 647046 (0.0009) [2023-12-26 20:04:40,554][105620] Updated weights for policy 1, policy_version 647834 (0.0010) [2023-12-26 20:04:40,612][105620] Updated weights for policy 1, policy_version 647844 (0.0009) [2023-12-26 20:04:40,674][105620] Updated weights for policy 1, policy_version 647854 (0.0009) [2023-12-26 20:04:40,734][105620] Updated weights for policy 1, policy_version 647864 (0.0006) [2023-12-26 20:04:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 331546624. Throughput: 0: 9987.4, 1: 9478.7. Samples: 331555752. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-26 20:04:41,063][104569] Avg episode reward: [(0, '8454.768'), (1, '8379.266')] [2023-12-26 20:04:41,222][105692] Updated weights for policy 0, policy_version 647056 (0.0010) [2023-12-26 20:04:41,285][105692] Updated weights for policy 0, policy_version 647066 (0.0009) [2023-12-26 20:04:41,346][105692] Updated weights for policy 0, policy_version 647076 (0.0009) [2023-12-26 20:04:41,443][105620] Updated weights for policy 1, policy_version 647874 (0.0009) [2023-12-26 20:04:41,510][105620] Updated weights for policy 1, policy_version 647884 (0.0008) [2023-12-26 20:04:41,577][105620] Updated weights for policy 1, policy_version 647894 (0.0009) [2023-12-26 20:04:42,158][105692] Updated weights for policy 0, policy_version 647086 (0.0007) [2023-12-26 20:04:42,225][105692] Updated weights for policy 0, policy_version 647096 (0.0005) [2023-12-26 20:04:42,295][105692] Updated weights for policy 0, policy_version 647106 (0.0008) [2023-12-26 20:04:42,327][105620] Updated weights for policy 1, policy_version 647904 (0.0006) [2023-12-26 20:04:42,397][105620] Updated weights for policy 1, policy_version 647914 (0.0008) [2023-12-26 20:04:42,456][105620] Updated weights for policy 1, policy_version 647924 (0.0008) [2023-12-26 20:04:42,930][105692] Updated weights for policy 0, policy_version 647116 (0.0009) [2023-12-26 20:04:42,980][105692] Updated weights for policy 0, policy_version 647126 (0.0006) [2023-12-26 20:04:43,024][105692] Updated weights for policy 0, policy_version 647136 (0.0006) [2023-12-26 20:04:43,169][105620] Updated weights for policy 1, policy_version 647934 (0.0010) [2023-12-26 20:04:43,214][105620] Updated weights for policy 1, policy_version 647944 (0.0010) [2023-12-26 20:04:43,272][105620] Updated weights for policy 1, policy_version 647954 (0.0010) [2023-12-26 20:04:43,618][105692] Updated weights for policy 0, policy_version 647146 (0.0008) [2023-12-26 20:04:43,671][105692] Updated weights for policy 0, policy_version 647157 (0.0010) [2023-12-26 20:04:43,729][105692] Updated weights for policy 0, policy_version 647168 (0.0009) [2023-12-26 20:04:43,856][105620] Updated weights for policy 1, policy_version 647964 (0.0009) [2023-12-26 20:04:43,918][105620] Updated weights for policy 1, policy_version 647974 (0.0005) [2023-12-26 20:04:43,986][105620] Updated weights for policy 1, policy_version 647984 (0.0005) [2023-12-26 20:04:44,352][105692] Updated weights for policy 0, policy_version 647178 (0.0005) [2023-12-26 20:04:44,407][105692] Updated weights for policy 0, policy_version 647188 (0.0006) [2023-12-26 20:04:44,458][105692] Updated weights for policy 0, policy_version 647198 (0.0005) [2023-12-26 20:04:44,512][105692] Updated weights for policy 0, policy_version 647208 (0.0005) [2023-12-26 20:04:44,660][105620] Updated weights for policy 1, policy_version 647994 (0.0006) [2023-12-26 20:04:44,731][105620] Updated weights for policy 1, policy_version 648004 (0.0010) [2023-12-26 20:04:44,798][105620] Updated weights for policy 1, policy_version 648014 (0.0008) [2023-12-26 20:04:44,853][105620] Updated weights for policy 1, policy_version 648024 (0.0010) [2023-12-26 20:04:45,095][105692] Updated weights for policy 0, policy_version 647218 (0.0007) [2023-12-26 20:04:45,155][105692] Updated weights for policy 0, policy_version 647228 (0.0009) [2023-12-26 20:04:45,218][105692] Updated weights for policy 0, policy_version 647238 (0.0008) [2023-12-26 20:04:45,613][105620] Updated weights for policy 1, policy_version 648034 (0.0009) [2023-12-26 20:04:45,674][105620] Updated weights for policy 1, policy_version 648044 (0.0009) [2023-12-26 20:04:45,724][105620] Updated weights for policy 1, policy_version 648054 (0.0009) [2023-12-26 20:04:45,898][105692] Updated weights for policy 0, policy_version 647248 (0.0010) [2023-12-26 20:04:45,956][105692] Updated weights for policy 0, policy_version 647258 (0.0010) [2023-12-26 20:04:46,021][105692] Updated weights for policy 0, policy_version 647268 (0.0010) [2023-12-26 20:04:46,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 331653120. Throughput: 0: 10037.7, 1: 9507.7. Samples: 331616452. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-26 20:04:46,062][104569] Avg episode reward: [(0, '8904.733'), (1, '8561.216')] [2023-12-26 20:04:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000647272_165732352.pth... [2023-12-26 20:04:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000648056_165920768.pth... [2023-12-26 20:04:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000646088_165429248.pth [2023-12-26 20:04:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000646968_165642240.pth [2023-12-26 20:04:46,448][105620] Updated weights for policy 1, policy_version 648064 (0.0010) [2023-12-26 20:04:46,507][105620] Updated weights for policy 1, policy_version 648074 (0.0010) [2023-12-26 20:04:46,565][105620] Updated weights for policy 1, policy_version 648084 (0.0011) [2023-12-26 20:04:46,636][105692] Updated weights for policy 0, policy_version 647278 (0.0008) [2023-12-26 20:04:46,702][105692] Updated weights for policy 0, policy_version 647288 (0.0007) [2023-12-26 20:04:46,754][105692] Updated weights for policy 0, policy_version 647298 (0.0005) [2023-12-26 20:04:47,145][105620] Updated weights for policy 1, policy_version 648094 (0.0010) [2023-12-26 20:04:47,201][105620] Updated weights for policy 1, policy_version 648104 (0.0008) [2023-12-26 20:04:47,265][105620] Updated weights for policy 1, policy_version 648114 (0.0005) [2023-12-26 20:04:47,283][105692] Updated weights for policy 0, policy_version 647308 (0.0006) [2023-12-26 20:04:47,341][105692] Updated weights for policy 0, policy_version 647318 (0.0005) [2023-12-26 20:04:47,407][105692] Updated weights for policy 0, policy_version 647328 (0.0005) [2023-12-26 20:04:47,908][105620] Updated weights for policy 1, policy_version 648124 (0.0007) [2023-12-26 20:04:47,956][105620] Updated weights for policy 1, policy_version 648134 (0.0010) [2023-12-26 20:04:47,973][105692] Updated weights for policy 0, policy_version 647338 (0.0006) [2023-12-26 20:04:48,011][105620] Updated weights for policy 1, policy_version 648144 (0.0010) [2023-12-26 20:04:48,022][105692] Updated weights for policy 0, policy_version 647348 (0.0009) [2023-12-26 20:04:48,069][105692] Updated weights for policy 0, policy_version 647358 (0.0009) [2023-12-26 20:04:48,113][105692] Updated weights for policy 0, policy_version 647368 (0.0007) [2023-12-26 20:04:48,710][105620] Updated weights for policy 1, policy_version 648154 (0.0010) [2023-12-26 20:04:48,776][105620] Updated weights for policy 1, policy_version 648164 (0.0010) [2023-12-26 20:04:48,843][105620] Updated weights for policy 1, policy_version 648174 (0.0011) [2023-12-26 20:04:48,896][105620] Updated weights for policy 1, policy_version 648184 (0.0011) [2023-12-26 20:04:48,902][105692] Updated weights for policy 0, policy_version 647378 (0.0005) [2023-12-26 20:04:48,964][105692] Updated weights for policy 0, policy_version 647388 (0.0007) [2023-12-26 20:04:49,019][105692] Updated weights for policy 0, policy_version 647398 (0.0005) [2023-12-26 20:04:49,645][105620] Updated weights for policy 1, policy_version 648194 (0.0011) [2023-12-26 20:04:49,669][105692] Updated weights for policy 0, policy_version 647408 (0.0006) [2023-12-26 20:04:49,697][105620] Updated weights for policy 1, policy_version 648204 (0.0010) [2023-12-26 20:04:49,731][105692] Updated weights for policy 0, policy_version 647418 (0.0006) [2023-12-26 20:04:49,754][105620] Updated weights for policy 1, policy_version 648214 (0.0011) [2023-12-26 20:04:49,789][105692] Updated weights for policy 0, policy_version 647428 (0.0008) [2023-12-26 20:04:50,372][105692] Updated weights for policy 0, policy_version 647438 (0.0007) [2023-12-26 20:04:50,428][105692] Updated weights for policy 0, policy_version 647448 (0.0008) [2023-12-26 20:04:50,488][105692] Updated weights for policy 0, policy_version 647458 (0.0008) [2023-12-26 20:04:50,534][105620] Updated weights for policy 1, policy_version 648224 (0.0010) [2023-12-26 20:04:50,608][105620] Updated weights for policy 1, policy_version 648234 (0.0009) [2023-12-26 20:04:50,665][105620] Updated weights for policy 1, policy_version 648244 (0.0010) [2023-12-26 20:04:51,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 331751424. Throughput: 0: 10196.2, 1: 9548.9. Samples: 331740956. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-26 20:04:51,063][104569] Avg episode reward: [(0, '9087.479'), (1, '8827.080')] [2023-12-26 20:04:51,276][105692] Updated weights for policy 0, policy_version 647468 (0.0006) [2023-12-26 20:04:51,336][105692] Updated weights for policy 0, policy_version 647478 (0.0008) [2023-12-26 20:04:51,368][105620] Updated weights for policy 1, policy_version 648254 (0.0010) [2023-12-26 20:04:51,403][105692] Updated weights for policy 0, policy_version 647488 (0.0007) [2023-12-26 20:04:51,436][105620] Updated weights for policy 1, policy_version 648264 (0.0008) [2023-12-26 20:04:51,488][105620] Updated weights for policy 1, policy_version 648274 (0.0006) [2023-12-26 20:04:52,151][105620] Updated weights for policy 1, policy_version 648284 (0.0008) [2023-12-26 20:04:52,213][105620] Updated weights for policy 1, policy_version 648294 (0.0009) [2023-12-26 20:04:52,228][105692] Updated weights for policy 0, policy_version 647498 (0.0008) [2023-12-26 20:04:52,275][105620] Updated weights for policy 1, policy_version 648304 (0.0007) [2023-12-26 20:04:52,287][105692] Updated weights for policy 0, policy_version 647508 (0.0007) [2023-12-26 20:04:52,349][105692] Updated weights for policy 0, policy_version 647518 (0.0008) [2023-12-26 20:04:52,416][105692] Updated weights for policy 0, policy_version 647528 (0.0010) [2023-12-26 20:04:52,906][105620] Updated weights for policy 1, policy_version 648314 (0.0010) [2023-12-26 20:04:52,968][105620] Updated weights for policy 1, policy_version 648324 (0.0006) [2023-12-26 20:04:53,021][105620] Updated weights for policy 1, policy_version 648334 (0.0005) [2023-12-26 20:04:53,081][105620] Updated weights for policy 1, policy_version 648344 (0.0009) [2023-12-26 20:04:53,244][105585] KL-divergence is very high: 621.5231 [2023-12-26 20:04:53,250][105692] Updated weights for policy 0, policy_version 647538 (0.0009) [2023-12-26 20:04:53,292][105585] KL-divergence is very high: 963.5701 [2023-12-26 20:04:53,309][105692] Updated weights for policy 0, policy_version 647548 (0.0009) [2023-12-26 20:04:53,340][105585] KL-divergence is very high: 916.8574 [2023-12-26 20:04:53,371][105692] Updated weights for policy 0, policy_version 647558 (0.0008) [2023-12-26 20:04:53,789][105620] Updated weights for policy 1, policy_version 648354 (0.0007) [2023-12-26 20:04:53,839][105620] Updated weights for policy 1, policy_version 648364 (0.0009) [2023-12-26 20:04:53,890][105620] Updated weights for policy 1, policy_version 648374 (0.0009) [2023-12-26 20:04:54,166][105692] Updated weights for policy 0, policy_version 647568 (0.0009) [2023-12-26 20:04:54,224][105692] Updated weights for policy 0, policy_version 647578 (0.0008) [2023-12-26 20:04:54,283][105692] Updated weights for policy 0, policy_version 647588 (0.0008) [2023-12-26 20:04:54,574][105620] Updated weights for policy 1, policy_version 648384 (0.0006) [2023-12-26 20:04:54,629][105620] Updated weights for policy 1, policy_version 648394 (0.0010) [2023-12-26 20:04:54,682][105620] Updated weights for policy 1, policy_version 648404 (0.0007) [2023-12-26 20:04:55,081][105692] Updated weights for policy 0, policy_version 647598 (0.0008) [2023-12-26 20:04:55,137][105692] Updated weights for policy 0, policy_version 647608 (0.0008) [2023-12-26 20:04:55,197][105692] Updated weights for policy 0, policy_version 647618 (0.0008) [2023-12-26 20:04:55,406][105620] Updated weights for policy 1, policy_version 648414 (0.0010) [2023-12-26 20:04:55,458][105620] Updated weights for policy 1, policy_version 648424 (0.0010) [2023-12-26 20:04:55,508][105620] Updated weights for policy 1, policy_version 648434 (0.0010) [2023-12-26 20:04:55,973][105692] Updated weights for policy 0, policy_version 647628 (0.0008) [2023-12-26 20:04:56,029][105692] Updated weights for policy 0, policy_version 647638 (0.0008) [2023-12-26 20:04:56,046][105585] KL-divergence is very high: 134.4405 [2023-12-26 20:04:56,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 331841536. Throughput: 0: 10076.9, 1: 9538.0. Samples: 331854960. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-26 20:04:56,063][104569] Avg episode reward: [(0, '9084.991'), (1, '9182.566')] [2023-12-26 20:04:56,090][105692] Updated weights for policy 0, policy_version 647648 (0.0009) [2023-12-26 20:04:56,096][105585] KL-divergence is very high: 109.6174 [2023-12-26 20:04:56,202][105620] Updated weights for policy 1, policy_version 648444 (0.0008) [2023-12-26 20:04:56,257][105620] Updated weights for policy 1, policy_version 648454 (0.0005) [2023-12-26 20:04:56,306][105620] Updated weights for policy 1, policy_version 648464 (0.0009) [2023-12-26 20:04:56,836][105620] Updated weights for policy 1, policy_version 648474 (0.0009) [2023-12-26 20:04:56,883][105620] Updated weights for policy 1, policy_version 648484 (0.0010) [2023-12-26 20:04:56,940][105620] Updated weights for policy 1, policy_version 648494 (0.0010) [2023-12-26 20:04:56,973][105692] Updated weights for policy 0, policy_version 647658 (0.0010) [2023-12-26 20:04:57,003][105620] Updated weights for policy 1, policy_version 648504 (0.0006) [2023-12-26 20:04:57,024][105692] Updated weights for policy 0, policy_version 647670 (0.0010) [2023-12-26 20:04:57,082][105692] Updated weights for policy 0, policy_version 647680 (0.0010) [2023-12-26 20:04:57,581][105620] Updated weights for policy 1, policy_version 648514 (0.0005) [2023-12-26 20:04:57,634][105620] Updated weights for policy 1, policy_version 648524 (0.0005) [2023-12-26 20:04:57,680][105620] Updated weights for policy 1, policy_version 648534 (0.0005) [2023-12-26 20:04:57,974][105692] Updated weights for policy 0, policy_version 647690 (0.0009) [2023-12-26 20:04:58,038][105692] Updated weights for policy 0, policy_version 647700 (0.0006) [2023-12-26 20:04:58,097][105692] Updated weights for policy 0, policy_version 647710 (0.0008) [2023-12-26 20:04:58,165][105692] Updated weights for policy 0, policy_version 647720 (0.0008) [2023-12-26 20:04:58,364][105620] Updated weights for policy 1, policy_version 648544 (0.0007) [2023-12-26 20:04:58,424][105620] Updated weights for policy 1, policy_version 648554 (0.0008) [2023-12-26 20:04:58,495][105620] Updated weights for policy 1, policy_version 648564 (0.0008) [2023-12-26 20:04:58,896][105692] Updated weights for policy 0, policy_version 647730 (0.0008) [2023-12-26 20:04:58,949][105692] Updated weights for policy 0, policy_version 647740 (0.0009) [2023-12-26 20:04:59,002][105692] Updated weights for policy 0, policy_version 647750 (0.0008) [2023-12-26 20:04:59,009][105585] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000001 [2023-12-26 20:04:59,222][105620] Updated weights for policy 1, policy_version 648574 (0.0007) [2023-12-26 20:04:59,284][105620] Updated weights for policy 1, policy_version 648584 (0.0008) [2023-12-26 20:04:59,349][105620] Updated weights for policy 1, policy_version 648594 (0.0008) [2023-12-26 20:04:59,872][105692] Updated weights for policy 0, policy_version 647760 (0.0008) [2023-12-26 20:04:59,935][105692] Updated weights for policy 0, policy_version 647770 (0.0007) [2023-12-26 20:04:59,996][105692] Updated weights for policy 0, policy_version 647780 (0.0009) [2023-12-26 20:05:00,116][105620] Updated weights for policy 1, policy_version 648604 (0.0009) [2023-12-26 20:05:00,176][105620] Updated weights for policy 1, policy_version 648614 (0.0009) [2023-12-26 20:05:00,234][105620] Updated weights for policy 1, policy_version 648625 (0.0009) [2023-12-26 20:05:00,697][105692] Updated weights for policy 0, policy_version 647790 (0.0009) [2023-12-26 20:05:00,752][105692] Updated weights for policy 0, policy_version 647800 (0.0010) [2023-12-26 20:05:00,811][105692] Updated weights for policy 0, policy_version 647810 (0.0010) [2023-12-26 20:05:00,958][105620] Updated weights for policy 1, policy_version 648635 (0.0007) [2023-12-26 20:05:01,007][105620] Updated weights for policy 1, policy_version 648645 (0.0006) [2023-12-26 20:05:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 331939840. Throughput: 0: 9971.7, 1: 9654.8. Samples: 331913488. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-26 20:05:01,063][104569] Avg episode reward: [(0, '8995.820'), (1, '9265.891')] [2023-12-26 20:05:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000647816_165871616.pth... [2023-12-26 20:05:01,073][105620] Updated weights for policy 1, policy_version 648655 (0.0006) [2023-12-26 20:05:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000646664_165576704.pth [2023-12-26 20:05:01,126][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000648664_166076416.pth... [2023-12-26 20:05:01,131][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000647512_165781504.pth [2023-12-26 20:05:01,636][105692] Updated weights for policy 0, policy_version 647821 (0.0010) [2023-12-26 20:05:01,690][105692] Updated weights for policy 0, policy_version 647831 (0.0009) [2023-12-26 20:05:01,749][105692] Updated weights for policy 0, policy_version 647841 (0.0010) [2023-12-26 20:05:01,760][105620] Updated weights for policy 1, policy_version 648665 (0.0010) [2023-12-26 20:05:01,818][105620] Updated weights for policy 1, policy_version 648675 (0.0008) [2023-12-26 20:05:01,883][105620] Updated weights for policy 1, policy_version 648685 (0.0009) [2023-12-26 20:05:01,949][105620] Updated weights for policy 1, policy_version 648695 (0.0010) [2023-12-26 20:05:02,394][105692] Updated weights for policy 0, policy_version 647851 (0.0006) [2023-12-26 20:05:02,462][105692] Updated weights for policy 0, policy_version 647861 (0.0005) [2023-12-26 20:05:02,531][105692] Updated weights for policy 0, policy_version 647871 (0.0006) [2023-12-26 20:05:02,760][105620] Updated weights for policy 1, policy_version 648705 (0.0006) [2023-12-26 20:05:02,809][105620] Updated weights for policy 1, policy_version 648715 (0.0005) [2023-12-26 20:05:02,859][105620] Updated weights for policy 1, policy_version 648725 (0.0007) [2023-12-26 20:05:03,174][105692] Updated weights for policy 0, policy_version 647881 (0.0006) [2023-12-26 20:05:03,226][105692] Updated weights for policy 0, policy_version 647891 (0.0005) [2023-12-26 20:05:03,273][105692] Updated weights for policy 0, policy_version 647901 (0.0005) [2023-12-26 20:05:03,321][105692] Updated weights for policy 0, policy_version 647911 (0.0005) [2023-12-26 20:05:03,440][105620] Updated weights for policy 1, policy_version 648736 (0.0009) [2023-12-26 20:05:03,498][105620] Updated weights for policy 1, policy_version 648746 (0.0010) [2023-12-26 20:05:03,550][105620] Updated weights for policy 1, policy_version 648756 (0.0009) [2023-12-26 20:05:03,912][105692] Updated weights for policy 0, policy_version 647921 (0.0009) [2023-12-26 20:05:03,966][105692] Updated weights for policy 0, policy_version 647931 (0.0010) [2023-12-26 20:05:04,018][105692] Updated weights for policy 0, policy_version 647941 (0.0009) [2023-12-26 20:05:04,293][105620] Updated weights for policy 1, policy_version 648766 (0.0009) [2023-12-26 20:05:04,348][105620] Updated weights for policy 1, policy_version 648776 (0.0009) [2023-12-26 20:05:04,408][105620] Updated weights for policy 1, policy_version 648786 (0.0009) [2023-12-26 20:05:04,774][105692] Updated weights for policy 0, policy_version 647952 (0.0007) [2023-12-26 20:05:04,832][105692] Updated weights for policy 0, policy_version 647962 (0.0009) [2023-12-26 20:05:04,892][105692] Updated weights for policy 0, policy_version 647972 (0.0010) [2023-12-26 20:05:05,194][105620] Updated weights for policy 1, policy_version 648796 (0.0009) [2023-12-26 20:05:05,248][105620] Updated weights for policy 1, policy_version 648806 (0.0008) [2023-12-26 20:05:05,305][105620] Updated weights for policy 1, policy_version 648816 (0.0010) [2023-12-26 20:05:05,630][105692] Updated weights for policy 0, policy_version 647982 (0.0010) [2023-12-26 20:05:05,688][105692] Updated weights for policy 0, policy_version 647992 (0.0010) [2023-12-26 20:05:05,755][105692] Updated weights for policy 0, policy_version 648002 (0.0010) [2023-12-26 20:05:06,018][105620] Updated weights for policy 1, policy_version 648826 (0.0008) [2023-12-26 20:05:06,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 332038144. Throughput: 0: 9970.9, 1: 9655.3. Samples: 332029544. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-26 20:05:06,062][104569] Avg episode reward: [(0, '9088.129'), (1, '9174.490')] [2023-12-26 20:05:06,077][105620] Updated weights for policy 1, policy_version 648836 (0.0008) [2023-12-26 20:05:06,139][105620] Updated weights for policy 1, policy_version 648846 (0.0009) [2023-12-26 20:05:06,191][105620] Updated weights for policy 1, policy_version 648856 (0.0009) [2023-12-26 20:05:06,542][105692] Updated weights for policy 0, policy_version 648012 (0.0009) [2023-12-26 20:05:06,597][105692] Updated weights for policy 0, policy_version 648022 (0.0008) [2023-12-26 20:05:06,656][105692] Updated weights for policy 0, policy_version 648032 (0.0009) [2023-12-26 20:05:06,929][105620] Updated weights for policy 1, policy_version 648866 (0.0008) [2023-12-26 20:05:06,991][105620] Updated weights for policy 1, policy_version 648876 (0.0008) [2023-12-26 20:05:07,051][105620] Updated weights for policy 1, policy_version 648886 (0.0008) [2023-12-26 20:05:07,429][105692] Updated weights for policy 0, policy_version 648042 (0.0010) [2023-12-26 20:05:07,488][105692] Updated weights for policy 0, policy_version 648052 (0.0008) [2023-12-26 20:05:07,550][105692] Updated weights for policy 0, policy_version 648062 (0.0008) [2023-12-26 20:05:07,594][105692] Updated weights for policy 0, policy_version 648072 (0.0005) [2023-12-26 20:05:07,786][105620] Updated weights for policy 1, policy_version 648896 (0.0006) [2023-12-26 20:05:07,846][105620] Updated weights for policy 1, policy_version 648906 (0.0005) [2023-12-26 20:05:07,905][105620] Updated weights for policy 1, policy_version 648916 (0.0005) [2023-12-26 20:05:08,163][105692] Updated weights for policy 0, policy_version 648083 (0.0010) [2023-12-26 20:05:08,215][105692] Updated weights for policy 0, policy_version 648093 (0.0009) [2023-12-26 20:05:08,266][105692] Updated weights for policy 0, policy_version 648103 (0.0008) [2023-12-26 20:05:08,517][105620] Updated weights for policy 1, policy_version 648926 (0.0008) [2023-12-26 20:05:08,575][105620] Updated weights for policy 1, policy_version 648936 (0.0009) [2023-12-26 20:05:08,626][105620] Updated weights for policy 1, policy_version 648946 (0.0009) [2023-12-26 20:05:09,037][105692] Updated weights for policy 0, policy_version 648113 (0.0008) [2023-12-26 20:05:09,085][105692] Updated weights for policy 0, policy_version 648123 (0.0009) [2023-12-26 20:05:09,153][105692] Updated weights for policy 0, policy_version 648133 (0.0008) [2023-12-26 20:05:09,415][105620] Updated weights for policy 1, policy_version 648956 (0.0009) [2023-12-26 20:05:09,485][105620] Updated weights for policy 1, policy_version 648966 (0.0008) [2023-12-26 20:05:09,546][105620] Updated weights for policy 1, policy_version 648976 (0.0008) [2023-12-26 20:05:09,891][105692] Updated weights for policy 0, policy_version 648143 (0.0008) [2023-12-26 20:05:09,956][105692] Updated weights for policy 0, policy_version 648153 (0.0010) [2023-12-26 20:05:10,017][105692] Updated weights for policy 0, policy_version 648163 (0.0008) [2023-12-26 20:05:10,355][105620] Updated weights for policy 1, policy_version 648986 (0.0009) [2023-12-26 20:05:10,425][105620] Updated weights for policy 1, policy_version 648996 (0.0007) [2023-12-26 20:05:10,482][105620] Updated weights for policy 1, policy_version 649006 (0.0008) [2023-12-26 20:05:10,540][105620] Updated weights for policy 1, policy_version 649016 (0.0009) [2023-12-26 20:05:10,702][105692] Updated weights for policy 0, policy_version 648173 (0.0008) [2023-12-26 20:05:10,756][105692] Updated weights for policy 0, policy_version 648183 (0.0009) [2023-12-26 20:05:10,804][105692] Updated weights for policy 0, policy_version 648193 (0.0009) [2023-12-26 20:05:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 332136448. Throughput: 0: 9887.8, 1: 9746.5. Samples: 332144540. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-26 20:05:11,063][104569] Avg episode reward: [(0, '9266.642'), (1, '9265.100')] [2023-12-26 20:05:11,298][105620] Updated weights for policy 1, policy_version 649026 (0.0009) [2023-12-26 20:05:11,361][105620] Updated weights for policy 1, policy_version 649036 (0.0011) [2023-12-26 20:05:11,430][105620] Updated weights for policy 1, policy_version 649046 (0.0012) [2023-12-26 20:05:11,510][105692] Updated weights for policy 0, policy_version 648203 (0.0008) [2023-12-26 20:05:11,563][105692] Updated weights for policy 0, policy_version 648213 (0.0011) [2023-12-26 20:05:11,615][105692] Updated weights for policy 0, policy_version 648223 (0.0010) [2023-12-26 20:05:12,184][105620] Updated weights for policy 1, policy_version 649056 (0.0009) [2023-12-26 20:05:12,241][105620] Updated weights for policy 1, policy_version 649066 (0.0008) [2023-12-26 20:05:12,300][105620] Updated weights for policy 1, policy_version 649076 (0.0008) [2023-12-26 20:05:12,418][105692] Updated weights for policy 0, policy_version 648233 (0.0010) [2023-12-26 20:05:12,483][105692] Updated weights for policy 0, policy_version 648243 (0.0009) [2023-12-26 20:05:12,538][105692] Updated weights for policy 0, policy_version 648253 (0.0008) [2023-12-26 20:05:12,592][105692] Updated weights for policy 0, policy_version 648263 (0.0008) [2023-12-26 20:05:13,097][105620] Updated weights for policy 1, policy_version 649086 (0.0008) [2023-12-26 20:05:13,144][105620] Updated weights for policy 1, policy_version 649096 (0.0008) [2023-12-26 20:05:13,199][105620] Updated weights for policy 1, policy_version 649106 (0.0009) [2023-12-26 20:05:13,313][105692] Updated weights for policy 0, policy_version 648273 (0.0006) [2023-12-26 20:05:13,381][105692] Updated weights for policy 0, policy_version 648283 (0.0008) [2023-12-26 20:05:13,445][105692] Updated weights for policy 0, policy_version 648293 (0.0006) [2023-12-26 20:05:13,968][105692] Updated weights for policy 0, policy_version 648303 (0.0005) [2023-12-26 20:05:14,005][105620] Updated weights for policy 1, policy_version 649116 (0.0009) [2023-12-26 20:05:14,020][105585] KL-divergence is very high: 107.9039 [2023-12-26 20:05:14,025][105692] Updated weights for policy 0, policy_version 648313 (0.0006) [2023-12-26 20:05:14,063][105620] Updated weights for policy 1, policy_version 649126 (0.0008) [2023-12-26 20:05:14,081][105692] Updated weights for policy 0, policy_version 648323 (0.0006) [2023-12-26 20:05:14,116][105620] Updated weights for policy 1, policy_version 649136 (0.0006) [2023-12-26 20:05:14,779][105692] Updated weights for policy 0, policy_version 648333 (0.0007) [2023-12-26 20:05:14,843][105692] Updated weights for policy 0, policy_version 648343 (0.0008) [2023-12-26 20:05:14,868][105620] Updated weights for policy 1, policy_version 649146 (0.0008) [2023-12-26 20:05:14,906][105692] Updated weights for policy 0, policy_version 648353 (0.0009) [2023-12-26 20:05:14,923][105620] Updated weights for policy 1, policy_version 649156 (0.0007) [2023-12-26 20:05:14,979][105620] Updated weights for policy 1, policy_version 649166 (0.0010) [2023-12-26 20:05:15,043][105620] Updated weights for policy 1, policy_version 649176 (0.0008) [2023-12-26 20:05:15,510][105692] Updated weights for policy 0, policy_version 648363 (0.0006) [2023-12-26 20:05:15,567][105692] Updated weights for policy 0, policy_version 648373 (0.0008) [2023-12-26 20:05:15,627][105692] Updated weights for policy 0, policy_version 648383 (0.0008) [2023-12-26 20:05:15,802][105620] Updated weights for policy 1, policy_version 649186 (0.0009) [2023-12-26 20:05:15,872][105620] Updated weights for policy 1, policy_version 649196 (0.0005) [2023-12-26 20:05:15,937][105620] Updated weights for policy 1, policy_version 649206 (0.0009) [2023-12-26 20:05:16,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 332234752. Throughput: 0: 9834.5, 1: 9764.3. Samples: 332200668. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-26 20:05:16,063][104569] Avg episode reward: [(0, '9178.472'), (1, '9265.230')] [2023-12-26 20:05:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000648392_166019072.pth... [2023-12-26 20:05:16,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000649208_166215680.pth... [2023-12-26 20:05:16,080][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000647272_165732352.pth [2023-12-26 20:05:16,081][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000648056_165920768.pth [2023-12-26 20:05:16,408][105692] Updated weights for policy 0, policy_version 648393 (0.0008) [2023-12-26 20:05:16,456][105692] Updated weights for policy 0, policy_version 648403 (0.0009) [2023-12-26 20:05:16,518][105692] Updated weights for policy 0, policy_version 648413 (0.0009) [2023-12-26 20:05:16,540][105620] Updated weights for policy 1, policy_version 649216 (0.0010) [2023-12-26 20:05:16,567][105692] Updated weights for policy 0, policy_version 648423 (0.0006) [2023-12-26 20:05:16,592][105620] Updated weights for policy 1, policy_version 649226 (0.0010) [2023-12-26 20:05:16,640][105620] Updated weights for policy 1, policy_version 649236 (0.0010) [2023-12-26 20:05:17,238][105692] Updated weights for policy 0, policy_version 648433 (0.0008) [2023-12-26 20:05:17,290][105692] Updated weights for policy 0, policy_version 648443 (0.0010) [2023-12-26 20:05:17,314][105620] Updated weights for policy 1, policy_version 649246 (0.0007) [2023-12-26 20:05:17,349][105692] Updated weights for policy 0, policy_version 648453 (0.0009) [2023-12-26 20:05:17,371][105620] Updated weights for policy 1, policy_version 649256 (0.0009) [2023-12-26 20:05:17,424][105620] Updated weights for policy 1, policy_version 649266 (0.0005) [2023-12-26 20:05:18,064][105692] Updated weights for policy 0, policy_version 648463 (0.0009) [2023-12-26 20:05:18,091][105620] Updated weights for policy 1, policy_version 649276 (0.0008) [2023-12-26 20:05:18,113][105692] Updated weights for policy 0, policy_version 648473 (0.0006) [2023-12-26 20:05:18,144][105620] Updated weights for policy 1, policy_version 649286 (0.0007) [2023-12-26 20:05:18,160][105692] Updated weights for policy 0, policy_version 648483 (0.0008) [2023-12-26 20:05:18,195][105620] Updated weights for policy 1, policy_version 649296 (0.0009) [2023-12-26 20:05:18,935][105620] Updated weights for policy 1, policy_version 649306 (0.0010) [2023-12-26 20:05:18,941][105692] Updated weights for policy 0, policy_version 648493 (0.0007) [2023-12-26 20:05:18,990][105620] Updated weights for policy 1, policy_version 649316 (0.0011) [2023-12-26 20:05:18,996][105692] Updated weights for policy 0, policy_version 648503 (0.0005) [2023-12-26 20:05:19,042][105620] Updated weights for policy 1, policy_version 649326 (0.0010) [2023-12-26 20:05:19,052][105692] Updated weights for policy 0, policy_version 648513 (0.0006) [2023-12-26 20:05:19,098][105620] Updated weights for policy 1, policy_version 649336 (0.0010) [2023-12-26 20:05:19,836][105620] Updated weights for policy 1, policy_version 649346 (0.0009) [2023-12-26 20:05:19,869][105692] Updated weights for policy 0, policy_version 648523 (0.0006) [2023-12-26 20:05:19,894][105620] Updated weights for policy 1, policy_version 649356 (0.0008) [2023-12-26 20:05:19,924][105692] Updated weights for policy 0, policy_version 648533 (0.0010) [2023-12-26 20:05:19,957][105620] Updated weights for policy 1, policy_version 649366 (0.0008) [2023-12-26 20:05:19,989][105692] Updated weights for policy 0, policy_version 648543 (0.0011) [2023-12-26 20:05:20,745][105620] Updated weights for policy 1, policy_version 649376 (0.0008) [2023-12-26 20:05:20,760][105692] Updated weights for policy 0, policy_version 648553 (0.0011) [2023-12-26 20:05:20,803][105620] Updated weights for policy 1, policy_version 649386 (0.0007) [2023-12-26 20:05:20,814][105692] Updated weights for policy 0, policy_version 648563 (0.0010) [2023-12-26 20:05:20,864][105620] Updated weights for policy 1, policy_version 649396 (0.0007) [2023-12-26 20:05:20,875][105692] Updated weights for policy 0, policy_version 648573 (0.0009) [2023-12-26 20:05:20,934][105692] Updated weights for policy 0, policy_version 648583 (0.0009) [2023-12-26 20:05:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 332333056. Throughput: 0: 9808.4, 1: 9708.9. Samples: 332318588. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-26 20:05:21,062][104569] Avg episode reward: [(0, '9178.699'), (1, '9173.919')] [2023-12-26 20:05:21,686][105620] Updated weights for policy 1, policy_version 649406 (0.0008) [2023-12-26 20:05:21,747][105692] Updated weights for policy 0, policy_version 648593 (0.0010) [2023-12-26 20:05:21,752][105620] Updated weights for policy 1, policy_version 649416 (0.0007) [2023-12-26 20:05:21,805][105620] Updated weights for policy 1, policy_version 649426 (0.0007) [2023-12-26 20:05:21,812][105692] Updated weights for policy 0, policy_version 648603 (0.0006) [2023-12-26 20:05:21,879][105692] Updated weights for policy 0, policy_version 648613 (0.0005) [2023-12-26 20:05:22,532][105620] Updated weights for policy 1, policy_version 649436 (0.0009) [2023-12-26 20:05:22,590][105620] Updated weights for policy 1, policy_version 649446 (0.0006) [2023-12-26 20:05:22,591][105692] Updated weights for policy 0, policy_version 648623 (0.0009) [2023-12-26 20:05:22,642][105620] Updated weights for policy 1, policy_version 649456 (0.0006) [2023-12-26 20:05:22,643][105692] Updated weights for policy 0, policy_version 648633 (0.0010) [2023-12-26 20:05:22,703][105692] Updated weights for policy 0, policy_version 648643 (0.0010) [2023-12-26 20:05:23,310][105620] Updated weights for policy 1, policy_version 649466 (0.0009) [2023-12-26 20:05:23,366][105620] Updated weights for policy 1, policy_version 649476 (0.0008) [2023-12-26 20:05:23,412][105620] Updated weights for policy 1, policy_version 649486 (0.0008) [2023-12-26 20:05:23,468][105620] Updated weights for policy 1, policy_version 649496 (0.0005) [2023-12-26 20:05:23,470][105692] Updated weights for policy 0, policy_version 648653 (0.0011) [2023-12-26 20:05:23,534][105692] Updated weights for policy 0, policy_version 648663 (0.0010) [2023-12-26 20:05:23,591][105692] Updated weights for policy 0, policy_version 648673 (0.0010) [2023-12-26 20:05:24,222][105620] Updated weights for policy 1, policy_version 649506 (0.0010) [2023-12-26 20:05:24,278][105620] Updated weights for policy 1, policy_version 649516 (0.0010) [2023-12-26 20:05:24,302][105692] Updated weights for policy 0, policy_version 648683 (0.0009) [2023-12-26 20:05:24,338][105620] Updated weights for policy 1, policy_version 649526 (0.0007) [2023-12-26 20:05:24,352][105692] Updated weights for policy 0, policy_version 648693 (0.0006) [2023-12-26 20:05:24,398][105692] Updated weights for policy 0, policy_version 648703 (0.0008) [2023-12-26 20:05:25,068][105620] Updated weights for policy 1, policy_version 649536 (0.0010) [2023-12-26 20:05:25,110][105692] Updated weights for policy 0, policy_version 648713 (0.0007) [2023-12-26 20:05:25,126][105620] Updated weights for policy 1, policy_version 649546 (0.0010) [2023-12-26 20:05:25,162][105692] Updated weights for policy 0, policy_version 648723 (0.0010) [2023-12-26 20:05:25,175][105620] Updated weights for policy 1, policy_version 649556 (0.0010) [2023-12-26 20:05:25,220][105692] Updated weights for policy 0, policy_version 648733 (0.0010) [2023-12-26 20:05:25,284][105692] Updated weights for policy 0, policy_version 648743 (0.0010) [2023-12-26 20:05:25,894][105620] Updated weights for policy 1, policy_version 649566 (0.0010) [2023-12-26 20:05:25,945][105620] Updated weights for policy 1, policy_version 649576 (0.0010) [2023-12-26 20:05:25,993][105620] Updated weights for policy 1, policy_version 649586 (0.0010) [2023-12-26 20:05:26,017][105692] Updated weights for policy 0, policy_version 648753 (0.0011) [2023-12-26 20:05:26,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 332423168. Throughput: 0: 9710.2, 1: 9756.0. Samples: 332431732. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-26 20:05:26,062][104569] Avg episode reward: [(0, '9266.546'), (1, '9082.355')] [2023-12-26 20:05:26,068][105692] Updated weights for policy 0, policy_version 648763 (0.0010) [2023-12-26 20:05:26,090][105585] KL-divergence is very high: 103.9054 [2023-12-26 20:05:26,119][105692] Updated weights for policy 0, policy_version 648773 (0.0010) [2023-12-26 20:05:26,691][105620] Updated weights for policy 1, policy_version 649596 (0.0010) [2023-12-26 20:05:26,745][105620] Updated weights for policy 1, policy_version 649607 (0.0010) [2023-12-26 20:05:26,794][105620] Updated weights for policy 1, policy_version 649617 (0.0009) [2023-12-26 20:05:26,809][105692] Updated weights for policy 0, policy_version 648783 (0.0008) [2023-12-26 20:05:26,860][105692] Updated weights for policy 0, policy_version 648793 (0.0010) [2023-12-26 20:05:26,903][105692] Updated weights for policy 0, policy_version 648803 (0.0010) [2023-12-26 20:05:27,559][105692] Updated weights for policy 0, policy_version 648813 (0.0008) [2023-12-26 20:05:27,566][105620] Updated weights for policy 1, policy_version 649627 (0.0006) [2023-12-26 20:05:27,613][105692] Updated weights for policy 0, policy_version 648823 (0.0005) [2023-12-26 20:05:27,618][105620] Updated weights for policy 1, policy_version 649637 (0.0009) [2023-12-26 20:05:27,666][105620] Updated weights for policy 1, policy_version 649647 (0.0009) [2023-12-26 20:05:27,670][105692] Updated weights for policy 0, policy_version 648833 (0.0005) [2023-12-26 20:05:28,216][105692] Updated weights for policy 0, policy_version 648843 (0.0005) [2023-12-26 20:05:28,270][105692] Updated weights for policy 0, policy_version 648853 (0.0006) [2023-12-26 20:05:28,329][105692] Updated weights for policy 0, policy_version 648863 (0.0007) [2023-12-26 20:05:28,371][105620] Updated weights for policy 1, policy_version 649657 (0.0009) [2023-12-26 20:05:28,424][105620] Updated weights for policy 1, policy_version 649667 (0.0009) [2023-12-26 20:05:28,477][105620] Updated weights for policy 1, policy_version 649677 (0.0008) [2023-12-26 20:05:28,538][105620] Updated weights for policy 1, policy_version 649687 (0.0009) [2023-12-26 20:05:29,029][105692] Updated weights for policy 0, policy_version 648873 (0.0008) [2023-12-26 20:05:29,087][105692] Updated weights for policy 0, policy_version 648883 (0.0006) [2023-12-26 20:05:29,152][105692] Updated weights for policy 0, policy_version 648893 (0.0006) [2023-12-26 20:05:29,207][105692] Updated weights for policy 0, policy_version 648903 (0.0006) [2023-12-26 20:05:29,310][105620] Updated weights for policy 1, policy_version 649697 (0.0010) [2023-12-26 20:05:29,375][105620] Updated weights for policy 1, policy_version 649707 (0.0009) [2023-12-26 20:05:29,428][105620] Updated weights for policy 1, policy_version 649717 (0.0009) [2023-12-26 20:05:29,820][105692] Updated weights for policy 0, policy_version 648913 (0.0009) [2023-12-26 20:05:29,887][105692] Updated weights for policy 0, policy_version 648923 (0.0010) [2023-12-26 20:05:29,948][105692] Updated weights for policy 0, policy_version 648933 (0.0009) [2023-12-26 20:05:30,284][105620] Updated weights for policy 1, policy_version 649727 (0.0009) [2023-12-26 20:05:30,339][105620] Updated weights for policy 1, policy_version 649737 (0.0009) [2023-12-26 20:05:30,407][105620] Updated weights for policy 1, policy_version 649747 (0.0009) [2023-12-26 20:05:30,605][105692] Updated weights for policy 0, policy_version 648943 (0.0007) [2023-12-26 20:05:30,671][105692] Updated weights for policy 0, policy_version 648953 (0.0005) [2023-12-26 20:05:30,737][105692] Updated weights for policy 0, policy_version 648963 (0.0005) [2023-12-26 20:05:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 332521472. Throughput: 0: 9749.9, 1: 9730.5. Samples: 332493076. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-26 20:05:31,063][104569] Avg episode reward: [(0, '9266.675'), (1, '9171.735')] [2023-12-26 20:05:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000648968_166166528.pth... [2023-12-26 20:05:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000649752_166354944.pth... [2023-12-26 20:05:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000648664_166076416.pth [2023-12-26 20:05:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000647816_165871616.pth [2023-12-26 20:05:31,283][105620] Updated weights for policy 1, policy_version 649757 (0.0011) [2023-12-26 20:05:31,303][105692] Updated weights for policy 0, policy_version 648973 (0.0007) [2023-12-26 20:05:31,341][105620] Updated weights for policy 1, policy_version 649767 (0.0008) [2023-12-26 20:05:31,370][105692] Updated weights for policy 0, policy_version 648983 (0.0008) [2023-12-26 20:05:31,407][105620] Updated weights for policy 1, policy_version 649777 (0.0006) [2023-12-26 20:05:31,436][105692] Updated weights for policy 0, policy_version 648993 (0.0007) [2023-12-26 20:05:32,006][105620] Updated weights for policy 1, policy_version 649787 (0.0006) [2023-12-26 20:05:32,059][105620] Updated weights for policy 1, policy_version 649797 (0.0005) [2023-12-26 20:05:32,121][105620] Updated weights for policy 1, policy_version 649807 (0.0006) [2023-12-26 20:05:32,204][105692] Updated weights for policy 0, policy_version 649003 (0.0007) [2023-12-26 20:05:32,258][105692] Updated weights for policy 0, policy_version 649013 (0.0008) [2023-12-26 20:05:32,317][105692] Updated weights for policy 0, policy_version 649023 (0.0010) [2023-12-26 20:05:32,733][105620] Updated weights for policy 1, policy_version 649817 (0.0005) [2023-12-26 20:05:32,786][105620] Updated weights for policy 1, policy_version 649827 (0.0005) [2023-12-26 20:05:32,831][105620] Updated weights for policy 1, policy_version 649837 (0.0007) [2023-12-26 20:05:32,883][105620] Updated weights for policy 1, policy_version 649847 (0.0005) [2023-12-26 20:05:33,086][105692] Updated weights for policy 0, policy_version 649033 (0.0009) [2023-12-26 20:05:33,146][105692] Updated weights for policy 0, policy_version 649043 (0.0011) [2023-12-26 20:05:33,198][105692] Updated weights for policy 0, policy_version 649053 (0.0010) [2023-12-26 20:05:33,256][105692] Updated weights for policy 0, policy_version 649063 (0.0010) [2023-12-26 20:05:33,523][105620] Updated weights for policy 1, policy_version 649857 (0.0007) [2023-12-26 20:05:33,578][105620] Updated weights for policy 1, policy_version 649867 (0.0008) [2023-12-26 20:05:33,628][105620] Updated weights for policy 1, policy_version 649877 (0.0007) [2023-12-26 20:05:33,952][105692] Updated weights for policy 0, policy_version 649073 (0.0008) [2023-12-26 20:05:34,012][105692] Updated weights for policy 0, policy_version 649083 (0.0005) [2023-12-26 20:05:34,071][105692] Updated weights for policy 0, policy_version 649093 (0.0005) [2023-12-26 20:05:34,396][105620] Updated weights for policy 1, policy_version 649887 (0.0010) [2023-12-26 20:05:34,459][105620] Updated weights for policy 1, policy_version 649897 (0.0011) [2023-12-26 20:05:34,519][105620] Updated weights for policy 1, policy_version 649907 (0.0011) [2023-12-26 20:05:34,706][105692] Updated weights for policy 0, policy_version 649103 (0.0009) [2023-12-26 20:05:34,768][105692] Updated weights for policy 0, policy_version 649113 (0.0011) [2023-12-26 20:05:34,830][105692] Updated weights for policy 0, policy_version 649123 (0.0010) [2023-12-26 20:05:35,197][105620] Updated weights for policy 1, policy_version 649917 (0.0008) [2023-12-26 20:05:35,255][105620] Updated weights for policy 1, policy_version 649927 (0.0005) [2023-12-26 20:05:35,303][105620] Updated weights for policy 1, policy_version 649937 (0.0005) [2023-12-26 20:05:35,533][105692] Updated weights for policy 0, policy_version 649133 (0.0008) [2023-12-26 20:05:35,594][105692] Updated weights for policy 0, policy_version 649143 (0.0005) [2023-12-26 20:05:35,643][105692] Updated weights for policy 0, policy_version 649153 (0.0005) [2023-12-26 20:05:35,908][105620] Updated weights for policy 1, policy_version 649947 (0.0007) [2023-12-26 20:05:35,956][105620] Updated weights for policy 1, policy_version 649957 (0.0010) [2023-12-26 20:05:36,021][105620] Updated weights for policy 1, policy_version 649967 (0.0011) [2023-12-26 20:05:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 332619776. Throughput: 0: 9647.8, 1: 9699.1. Samples: 332611564. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-26 20:05:36,062][104569] Avg episode reward: [(0, '9268.116'), (1, '9351.607')] [2023-12-26 20:05:36,307][105692] Updated weights for policy 0, policy_version 649163 (0.0006) [2023-12-26 20:05:36,361][105692] Updated weights for policy 0, policy_version 649173 (0.0008) [2023-12-26 20:05:36,411][105692] Updated weights for policy 0, policy_version 649183 (0.0008) [2023-12-26 20:05:36,789][105620] Updated weights for policy 1, policy_version 649977 (0.0010) [2023-12-26 20:05:36,840][105620] Updated weights for policy 1, policy_version 649987 (0.0010) [2023-12-26 20:05:36,889][105620] Updated weights for policy 1, policy_version 649997 (0.0010) [2023-12-26 20:05:36,949][105620] Updated weights for policy 1, policy_version 650007 (0.0008) [2023-12-26 20:05:37,207][105692] Updated weights for policy 0, policy_version 649193 (0.0008) [2023-12-26 20:05:37,265][105692] Updated weights for policy 0, policy_version 649203 (0.0011) [2023-12-26 20:05:37,318][105692] Updated weights for policy 0, policy_version 649213 (0.0010) [2023-12-26 20:05:37,381][105692] Updated weights for policy 0, policy_version 649223 (0.0008) [2023-12-26 20:05:37,640][105620] Updated weights for policy 1, policy_version 650017 (0.0006) [2023-12-26 20:05:37,701][105620] Updated weights for policy 1, policy_version 650027 (0.0006) [2023-12-26 20:05:37,752][105620] Updated weights for policy 1, policy_version 650037 (0.0005) [2023-12-26 20:05:38,181][105692] Updated weights for policy 0, policy_version 649233 (0.0010) [2023-12-26 20:05:38,229][105692] Updated weights for policy 0, policy_version 649243 (0.0010) [2023-12-26 20:05:38,287][105692] Updated weights for policy 0, policy_version 649253 (0.0010) [2023-12-26 20:05:38,440][105620] Updated weights for policy 1, policy_version 650047 (0.0007) [2023-12-26 20:05:38,500][105620] Updated weights for policy 1, policy_version 650057 (0.0008) [2023-12-26 20:05:38,545][105620] Updated weights for policy 1, policy_version 650067 (0.0008) [2023-12-26 20:05:39,040][105692] Updated weights for policy 0, policy_version 649263 (0.0011) [2023-12-26 20:05:39,106][105692] Updated weights for policy 0, policy_version 649273 (0.0010) [2023-12-26 20:05:39,165][105692] Updated weights for policy 0, policy_version 649283 (0.0010) [2023-12-26 20:05:39,344][105620] Updated weights for policy 1, policy_version 650077 (0.0009) [2023-12-26 20:05:39,412][105620] Updated weights for policy 1, policy_version 650087 (0.0008) [2023-12-26 20:05:39,476][105620] Updated weights for policy 1, policy_version 650097 (0.0008) [2023-12-26 20:05:39,956][105692] Updated weights for policy 0, policy_version 649293 (0.0011) [2023-12-26 20:05:40,030][105692] Updated weights for policy 0, policy_version 649303 (0.0011) [2023-12-26 20:05:40,097][105692] Updated weights for policy 0, policy_version 649313 (0.0011) [2023-12-26 20:05:40,230][105620] Updated weights for policy 1, policy_version 650107 (0.0009) [2023-12-26 20:05:40,294][105620] Updated weights for policy 1, policy_version 650117 (0.0008) [2023-12-26 20:05:40,362][105620] Updated weights for policy 1, policy_version 650127 (0.0008) [2023-12-26 20:05:40,833][105692] Updated weights for policy 0, policy_version 649323 (0.0011) [2023-12-26 20:05:40,882][105692] Updated weights for policy 0, policy_version 649333 (0.0011) [2023-12-26 20:05:40,942][105692] Updated weights for policy 0, policy_version 649343 (0.0011) [2023-12-26 20:05:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 332718080. Throughput: 0: 9684.0, 1: 9676.1. Samples: 332726156. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-26 20:05:41,062][104569] Avg episode reward: [(0, '9268.057'), (1, '9352.016')] [2023-12-26 20:05:41,123][105620] Updated weights for policy 1, policy_version 650137 (0.0008) [2023-12-26 20:05:41,193][105620] Updated weights for policy 1, policy_version 650147 (0.0008) [2023-12-26 20:05:41,259][105620] Updated weights for policy 1, policy_version 650157 (0.0009) [2023-12-26 20:05:41,326][105620] Updated weights for policy 1, policy_version 650167 (0.0007) [2023-12-26 20:05:41,711][105692] Updated weights for policy 0, policy_version 649353 (0.0010) [2023-12-26 20:05:41,783][105692] Updated weights for policy 0, policy_version 649363 (0.0008) [2023-12-26 20:05:41,836][105692] Updated weights for policy 0, policy_version 649373 (0.0008) [2023-12-26 20:05:41,896][105692] Updated weights for policy 0, policy_version 649383 (0.0008) [2023-12-26 20:05:42,080][105620] Updated weights for policy 1, policy_version 650177 (0.0010) [2023-12-26 20:05:42,144][105620] Updated weights for policy 1, policy_version 650187 (0.0011) [2023-12-26 20:05:42,193][105620] Updated weights for policy 1, policy_version 650197 (0.0010) [2023-12-26 20:05:42,676][105692] Updated weights for policy 0, policy_version 649393 (0.0009) [2023-12-26 20:05:42,746][105692] Updated weights for policy 0, policy_version 649403 (0.0007) [2023-12-26 20:05:42,811][105692] Updated weights for policy 0, policy_version 649413 (0.0008) [2023-12-26 20:05:42,926][105620] Updated weights for policy 1, policy_version 650207 (0.0008) [2023-12-26 20:05:42,982][105620] Updated weights for policy 1, policy_version 650217 (0.0009) [2023-12-26 20:05:43,042][105620] Updated weights for policy 1, policy_version 650227 (0.0008) [2023-12-26 20:05:43,407][105692] Updated weights for policy 0, policy_version 649423 (0.0008) [2023-12-26 20:05:43,469][105692] Updated weights for policy 0, policy_version 649433 (0.0009) [2023-12-26 20:05:43,529][105692] Updated weights for policy 0, policy_version 649443 (0.0009) [2023-12-26 20:05:43,831][105620] Updated weights for policy 1, policy_version 650237 (0.0009) [2023-12-26 20:05:43,885][105620] Updated weights for policy 1, policy_version 650247 (0.0009) [2023-12-26 20:05:43,937][105620] Updated weights for policy 1, policy_version 650257 (0.0008) [2023-12-26 20:05:44,177][105692] Updated weights for policy 0, policy_version 649453 (0.0008) [2023-12-26 20:05:44,224][105692] Updated weights for policy 0, policy_version 649463 (0.0009) [2023-12-26 20:05:44,271][105692] Updated weights for policy 0, policy_version 649473 (0.0009) [2023-12-26 20:05:44,657][105620] Updated weights for policy 1, policy_version 650268 (0.0010) [2023-12-26 20:05:44,712][105620] Updated weights for policy 1, policy_version 650278 (0.0009) [2023-12-26 20:05:44,775][105620] Updated weights for policy 1, policy_version 650288 (0.0008) [2023-12-26 20:05:44,966][105692] Updated weights for policy 0, policy_version 649483 (0.0009) [2023-12-26 20:05:45,027][105692] Updated weights for policy 0, policy_version 649493 (0.0009) [2023-12-26 20:05:45,087][105692] Updated weights for policy 0, policy_version 649503 (0.0006) [2023-12-26 20:05:45,630][105620] Updated weights for policy 1, policy_version 650298 (0.0008) [2023-12-26 20:05:45,672][105692] Updated weights for policy 0, policy_version 649513 (0.0006) [2023-12-26 20:05:45,678][105620] Updated weights for policy 1, policy_version 650308 (0.0008) [2023-12-26 20:05:45,734][105692] Updated weights for policy 0, policy_version 649523 (0.0011) [2023-12-26 20:05:45,741][105620] Updated weights for policy 1, policy_version 650318 (0.0006) [2023-12-26 20:05:45,790][105692] Updated weights for policy 0, policy_version 649533 (0.0011) [2023-12-26 20:05:45,794][105620] Updated weights for policy 1, policy_version 650328 (0.0009) [2023-12-26 20:05:45,845][105692] Updated weights for policy 0, policy_version 649543 (0.0010) [2023-12-26 20:05:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 332816384. Throughput: 0: 9743.8, 1: 9543.5. Samples: 332781412. Policy #0 lag: (min: 8.0, avg: 36.0, max: 40.0) [2023-12-26 20:05:46,062][104569] Avg episode reward: [(0, '9266.130'), (1, '9352.060')] [2023-12-26 20:05:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000649544_166313984.pth... [2023-12-26 20:05:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000650328_166502400.pth... [2023-12-26 20:05:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000648392_166019072.pth [2023-12-26 20:05:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000649208_166215680.pth [2023-12-26 20:05:46,423][105585] KL-divergence is very high: 307.5471 [2023-12-26 20:05:46,424][105692] Updated weights for policy 0, policy_version 649553 (0.0006) [2023-12-26 20:05:46,461][105585] KL-divergence is very high: 471.1320 [2023-12-26 20:05:46,471][105692] Updated weights for policy 0, policy_version 649563 (0.0007) [2023-12-26 20:05:46,499][105585] KL-divergence is very high: 374.5931 [2023-12-26 20:05:46,518][105692] Updated weights for policy 0, policy_version 649573 (0.0009) [2023-12-26 20:05:46,656][105620] Updated weights for policy 1, policy_version 650338 (0.0008) [2023-12-26 20:05:46,707][105620] Updated weights for policy 1, policy_version 650348 (0.0008) [2023-12-26 20:05:46,759][105620] Updated weights for policy 1, policy_version 650358 (0.0008) [2023-12-26 20:05:47,232][105692] Updated weights for policy 0, policy_version 649583 (0.0010) [2023-12-26 20:05:47,294][105692] Updated weights for policy 0, policy_version 649593 (0.0011) [2023-12-26 20:05:47,359][105692] Updated weights for policy 0, policy_version 649603 (0.0011) [2023-12-26 20:05:47,534][105620] Updated weights for policy 1, policy_version 650368 (0.0009) [2023-12-26 20:05:47,587][105620] Updated weights for policy 1, policy_version 650378 (0.0008) [2023-12-26 20:05:47,644][105620] Updated weights for policy 1, policy_version 650388 (0.0009) [2023-12-26 20:05:48,041][105692] Updated weights for policy 0, policy_version 649613 (0.0011) [2023-12-26 20:05:48,104][105692] Updated weights for policy 0, policy_version 649623 (0.0009) [2023-12-26 20:05:48,159][105692] Updated weights for policy 0, policy_version 649633 (0.0010) [2023-12-26 20:05:48,358][105620] Updated weights for policy 1, policy_version 650398 (0.0008) [2023-12-26 20:05:48,425][105620] Updated weights for policy 1, policy_version 650408 (0.0006) [2023-12-26 20:05:48,496][105620] Updated weights for policy 1, policy_version 650418 (0.0006) [2023-12-26 20:05:48,850][105692] Updated weights for policy 0, policy_version 649643 (0.0008) [2023-12-26 20:05:48,912][105692] Updated weights for policy 0, policy_version 649653 (0.0010) [2023-12-26 20:05:48,977][105692] Updated weights for policy 0, policy_version 649663 (0.0010) [2023-12-26 20:05:49,116][105620] Updated weights for policy 1, policy_version 650428 (0.0007) [2023-12-26 20:05:49,167][105620] Updated weights for policy 1, policy_version 650438 (0.0007) [2023-12-26 20:05:49,234][105620] Updated weights for policy 1, policy_version 650448 (0.0008) [2023-12-26 20:05:49,728][105692] Updated weights for policy 0, policy_version 649673 (0.0011) [2023-12-26 20:05:49,793][105692] Updated weights for policy 0, policy_version 649683 (0.0011) [2023-12-26 20:05:49,867][105692] Updated weights for policy 0, policy_version 649693 (0.0008) [2023-12-26 20:05:49,918][105692] Updated weights for policy 0, policy_version 649703 (0.0005) [2023-12-26 20:05:49,944][105620] Updated weights for policy 1, policy_version 650458 (0.0008) [2023-12-26 20:05:50,010][105620] Updated weights for policy 1, policy_version 650468 (0.0008) [2023-12-26 20:05:50,069][105620] Updated weights for policy 1, policy_version 650478 (0.0008) [2023-12-26 20:05:50,129][105620] Updated weights for policy 1, policy_version 650488 (0.0008) [2023-12-26 20:05:50,632][105692] Updated weights for policy 0, policy_version 649713 (0.0007) [2023-12-26 20:05:50,691][105692] Updated weights for policy 0, policy_version 649723 (0.0010) [2023-12-26 20:05:50,755][105692] Updated weights for policy 0, policy_version 649733 (0.0011) [2023-12-26 20:05:50,809][105620] Updated weights for policy 1, policy_version 650498 (0.0007) [2023-12-26 20:05:50,871][105620] Updated weights for policy 1, policy_version 650508 (0.0008) [2023-12-26 20:05:50,929][105620] Updated weights for policy 1, policy_version 650518 (0.0010) [2023-12-26 20:05:51,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 332914688. Throughput: 0: 9829.6, 1: 9516.6. Samples: 332900124. Policy #0 lag: (min: 8.0, avg: 36.0, max: 40.0) [2023-12-26 20:05:51,063][104569] Avg episode reward: [(0, '9265.955'), (1, '9352.106')] [2023-12-26 20:05:51,423][105692] Updated weights for policy 0, policy_version 649743 (0.0007) [2023-12-26 20:05:51,487][105692] Updated weights for policy 0, policy_version 649753 (0.0009) [2023-12-26 20:05:51,548][105692] Updated weights for policy 0, policy_version 649763 (0.0009) [2023-12-26 20:05:51,771][105620] Updated weights for policy 1, policy_version 650528 (0.0008) [2023-12-26 20:05:51,830][105620] Updated weights for policy 1, policy_version 650538 (0.0008) [2023-12-26 20:05:51,903][105620] Updated weights for policy 1, policy_version 650548 (0.0005) [2023-12-26 20:05:52,290][105692] Updated weights for policy 0, policy_version 649773 (0.0009) [2023-12-26 20:05:52,349][105692] Updated weights for policy 0, policy_version 649783 (0.0011) [2023-12-26 20:05:52,406][105692] Updated weights for policy 0, policy_version 649793 (0.0010) [2023-12-26 20:05:52,622][105620] Updated weights for policy 1, policy_version 650558 (0.0008) [2023-12-26 20:05:52,675][105620] Updated weights for policy 1, policy_version 650568 (0.0005) [2023-12-26 20:05:52,729][105620] Updated weights for policy 1, policy_version 650578 (0.0005) [2023-12-26 20:05:53,178][105692] Updated weights for policy 0, policy_version 649803 (0.0011) [2023-12-26 20:05:53,230][105692] Updated weights for policy 0, policy_version 649813 (0.0010) [2023-12-26 20:05:53,278][105692] Updated weights for policy 0, policy_version 649823 (0.0010) [2023-12-26 20:05:53,394][105620] Updated weights for policy 1, policy_version 650588 (0.0008) [2023-12-26 20:05:53,453][105620] Updated weights for policy 1, policy_version 650598 (0.0009) [2023-12-26 20:05:53,507][105620] Updated weights for policy 1, policy_version 650608 (0.0005) [2023-12-26 20:05:53,970][105692] Updated weights for policy 0, policy_version 649833 (0.0010) [2023-12-26 20:05:54,015][105692] Updated weights for policy 0, policy_version 649843 (0.0010) [2023-12-26 20:05:54,059][105620] Updated weights for policy 1, policy_version 650618 (0.0005) [2023-12-26 20:05:54,060][105692] Updated weights for policy 0, policy_version 649853 (0.0010) [2023-12-26 20:05:54,108][105692] Updated weights for policy 0, policy_version 649863 (0.0010) [2023-12-26 20:05:54,119][105620] Updated weights for policy 1, policy_version 650628 (0.0006) [2023-12-26 20:05:54,190][105620] Updated weights for policy 1, policy_version 650638 (0.0008) [2023-12-26 20:05:54,256][105620] Updated weights for policy 1, policy_version 650648 (0.0008) [2023-12-26 20:05:54,868][105692] Updated weights for policy 0, policy_version 649873 (0.0010) [2023-12-26 20:05:54,923][105692] Updated weights for policy 0, policy_version 649883 (0.0010) [2023-12-26 20:05:54,931][105620] Updated weights for policy 1, policy_version 650658 (0.0006) [2023-12-26 20:05:54,980][105692] Updated weights for policy 0, policy_version 649893 (0.0008) [2023-12-26 20:05:54,988][105620] Updated weights for policy 1, policy_version 650668 (0.0011) [2023-12-26 20:05:55,040][105620] Updated weights for policy 1, policy_version 650678 (0.0010) [2023-12-26 20:05:55,604][105692] Updated weights for policy 0, policy_version 649903 (0.0005) [2023-12-26 20:05:55,659][105692] Updated weights for policy 0, policy_version 649913 (0.0005) [2023-12-26 20:05:55,669][105620] Updated weights for policy 1, policy_version 650688 (0.0008) [2023-12-26 20:05:55,713][105692] Updated weights for policy 0, policy_version 649923 (0.0005) [2023-12-26 20:05:55,719][105620] Updated weights for policy 1, policy_version 650698 (0.0005) [2023-12-26 20:05:55,773][105620] Updated weights for policy 1, policy_version 650708 (0.0005) [2023-12-26 20:05:56,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 333012992. Throughput: 0: 9850.6, 1: 9606.3. Samples: 333020104. Policy #0 lag: (min: 8.0, avg: 36.0, max: 40.0) [2023-12-26 20:05:56,063][104569] Avg episode reward: [(0, '9175.056'), (1, '9261.127')] [2023-12-26 20:05:56,260][105692] Updated weights for policy 0, policy_version 649933 (0.0006) [2023-12-26 20:05:56,319][105692] Updated weights for policy 0, policy_version 649943 (0.0005) [2023-12-26 20:05:56,341][105620] Updated weights for policy 1, policy_version 650718 (0.0006) [2023-12-26 20:05:56,372][105692] Updated weights for policy 0, policy_version 649953 (0.0008) [2023-12-26 20:05:56,401][105620] Updated weights for policy 1, policy_version 650728 (0.0006) [2023-12-26 20:05:56,458][105620] Updated weights for policy 1, policy_version 650738 (0.0009) [2023-12-26 20:05:57,051][105692] Updated weights for policy 0, policy_version 649963 (0.0008) [2023-12-26 20:05:57,115][105692] Updated weights for policy 0, policy_version 649973 (0.0009) [2023-12-26 20:05:57,147][105620] Updated weights for policy 1, policy_version 650748 (0.0008) [2023-12-26 20:05:57,173][105692] Updated weights for policy 0, policy_version 649983 (0.0006) [2023-12-26 20:05:57,216][105620] Updated weights for policy 1, policy_version 650758 (0.0005) [2023-12-26 20:05:57,283][105620] Updated weights for policy 1, policy_version 650768 (0.0005) [2023-12-26 20:05:57,794][105692] Updated weights for policy 0, policy_version 649993 (0.0006) [2023-12-26 20:05:57,852][105692] Updated weights for policy 0, policy_version 650003 (0.0010) [2023-12-26 20:05:57,876][105620] Updated weights for policy 1, policy_version 650778 (0.0006) [2023-12-26 20:05:57,906][105692] Updated weights for policy 0, policy_version 650013 (0.0007) [2023-12-26 20:05:57,925][105620] Updated weights for policy 1, policy_version 650788 (0.0005) [2023-12-26 20:05:57,959][105692] Updated weights for policy 0, policy_version 650023 (0.0005) [2023-12-26 20:05:57,978][105620] Updated weights for policy 1, policy_version 650798 (0.0008) [2023-12-26 20:05:58,027][105620] Updated weights for policy 1, policy_version 650808 (0.0010) [2023-12-26 20:05:58,702][105692] Updated weights for policy 0, policy_version 650033 (0.0008) [2023-12-26 20:05:58,766][105692] Updated weights for policy 0, policy_version 650043 (0.0008) [2023-12-26 20:05:58,812][105620] Updated weights for policy 1, policy_version 650818 (0.0009) [2023-12-26 20:05:58,832][105692] Updated weights for policy 0, policy_version 650053 (0.0006) [2023-12-26 20:05:58,882][105620] Updated weights for policy 1, policy_version 650828 (0.0006) [2023-12-26 20:05:58,945][105620] Updated weights for policy 1, policy_version 650838 (0.0006) [2023-12-26 20:05:59,544][105620] Updated weights for policy 1, policy_version 650848 (0.0006) [2023-12-26 20:05:59,598][105620] Updated weights for policy 1, policy_version 650858 (0.0008) [2023-12-26 20:05:59,645][105692] Updated weights for policy 0, policy_version 650063 (0.0006) [2023-12-26 20:05:59,650][105620] Updated weights for policy 1, policy_version 650868 (0.0009) [2023-12-26 20:05:59,651][105585] KL-divergence is very high: 104.8781 [2023-12-26 20:05:59,679][105585] KL-divergence is very high: 151.0297 [2023-12-26 20:05:59,700][105585] KL-divergence is very high: 105.5395 [2023-12-26 20:05:59,707][105692] Updated weights for policy 0, policy_version 650073 (0.0005) [2023-12-26 20:05:59,763][105692] Updated weights for policy 0, policy_version 650083 (0.0005) [2023-12-26 20:06:00,391][105692] Updated weights for policy 0, policy_version 650093 (0.0008) [2023-12-26 20:06:00,425][105620] Updated weights for policy 1, policy_version 650878 (0.0009) [2023-12-26 20:06:00,436][105692] Updated weights for policy 0, policy_version 650103 (0.0010) [2023-12-26 20:06:00,482][105620] Updated weights for policy 1, policy_version 650888 (0.0006) [2023-12-26 20:06:00,489][105692] Updated weights for policy 0, policy_version 650113 (0.0010) [2023-12-26 20:06:00,491][105585] KL-divergence is very high: 131.4066 [2023-12-26 20:06:00,532][105620] Updated weights for policy 1, policy_version 650898 (0.0006) [2023-12-26 20:06:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 333111296. Throughput: 0: 9908.2, 1: 9691.3. Samples: 333082644. Policy #0 lag: (min: 8.0, avg: 36.0, max: 40.0) [2023-12-26 20:06:01,063][104569] Avg episode reward: [(0, '9176.088'), (1, '9352.987')] [2023-12-26 20:06:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000650904_166649856.pth... [2023-12-26 20:06:01,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000650120_166461440.pth... [2023-12-26 20:06:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000649752_166354944.pth [2023-12-26 20:06:01,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000648968_166166528.pth [2023-12-26 20:06:01,217][105692] Updated weights for policy 0, policy_version 650123 (0.0009) [2023-12-26 20:06:01,228][105620] Updated weights for policy 1, policy_version 650908 (0.0009) [2023-12-26 20:06:01,278][105692] Updated weights for policy 0, policy_version 650133 (0.0007) [2023-12-26 20:06:01,292][105620] Updated weights for policy 1, policy_version 650918 (0.0011) [2023-12-26 20:06:01,335][105692] Updated weights for policy 0, policy_version 650143 (0.0006) [2023-12-26 20:06:01,356][105620] Updated weights for policy 1, policy_version 650928 (0.0010) [2023-12-26 20:06:02,054][105620] Updated weights for policy 1, policy_version 650938 (0.0009) [2023-12-26 20:06:02,093][105692] Updated weights for policy 0, policy_version 650153 (0.0008) [2023-12-26 20:06:02,114][105620] Updated weights for policy 1, policy_version 650948 (0.0010) [2023-12-26 20:06:02,145][105692] Updated weights for policy 0, policy_version 650163 (0.0007) [2023-12-26 20:06:02,173][105620] Updated weights for policy 1, policy_version 650958 (0.0010) [2023-12-26 20:06:02,197][105692] Updated weights for policy 0, policy_version 650173 (0.0009) [2023-12-26 20:06:02,235][105620] Updated weights for policy 1, policy_version 650968 (0.0007) [2023-12-26 20:06:02,260][105692] Updated weights for policy 0, policy_version 650183 (0.0008) [2023-12-26 20:06:02,894][105620] Updated weights for policy 1, policy_version 650978 (0.0010) [2023-12-26 20:06:02,956][105620] Updated weights for policy 1, policy_version 650988 (0.0008) [2023-12-26 20:06:03,017][105620] Updated weights for policy 1, policy_version 650998 (0.0006) [2023-12-26 20:06:03,019][105692] Updated weights for policy 0, policy_version 650193 (0.0009) [2023-12-26 20:06:03,073][105692] Updated weights for policy 0, policy_version 650203 (0.0008) [2023-12-26 20:06:03,128][105692] Updated weights for policy 0, policy_version 650213 (0.0009) [2023-12-26 20:06:03,724][105620] Updated weights for policy 1, policy_version 651008 (0.0006) [2023-12-26 20:06:03,774][105620] Updated weights for policy 1, policy_version 651018 (0.0007) [2023-12-26 20:06:03,824][105692] Updated weights for policy 0, policy_version 650223 (0.0008) [2023-12-26 20:06:03,830][105620] Updated weights for policy 1, policy_version 651028 (0.0009) [2023-12-26 20:06:03,893][105692] Updated weights for policy 0, policy_version 650233 (0.0007) [2023-12-26 20:06:03,951][105692] Updated weights for policy 0, policy_version 650243 (0.0006) [2023-12-26 20:06:04,525][105620] Updated weights for policy 1, policy_version 651038 (0.0007) [2023-12-26 20:06:04,574][105692] Updated weights for policy 0, policy_version 650253 (0.0006) [2023-12-26 20:06:04,583][105620] Updated weights for policy 1, policy_version 651048 (0.0006) [2023-12-26 20:06:04,638][105692] Updated weights for policy 0, policy_version 650263 (0.0006) [2023-12-26 20:06:04,647][105620] Updated weights for policy 1, policy_version 651058 (0.0008) [2023-12-26 20:06:04,705][105692] Updated weights for policy 0, policy_version 650273 (0.0008) [2023-12-26 20:06:05,224][105620] Updated weights for policy 1, policy_version 651068 (0.0008) [2023-12-26 20:06:05,288][105620] Updated weights for policy 1, policy_version 651078 (0.0005) [2023-12-26 20:06:05,339][105620] Updated weights for policy 1, policy_version 651088 (0.0005) [2023-12-26 20:06:05,441][105692] Updated weights for policy 0, policy_version 650283 (0.0009) [2023-12-26 20:06:05,498][105692] Updated weights for policy 0, policy_version 650293 (0.0009) [2023-12-26 20:06:05,555][105692] Updated weights for policy 0, policy_version 650303 (0.0009) [2023-12-26 20:06:05,920][105620] Updated weights for policy 1, policy_version 651098 (0.0006) [2023-12-26 20:06:05,968][105620] Updated weights for policy 1, policy_version 651108 (0.0010) [2023-12-26 20:06:06,012][105620] Updated weights for policy 1, policy_version 651118 (0.0010) [2023-12-26 20:06:06,056][105620] Updated weights for policy 1, policy_version 651128 (0.0010) [2023-12-26 20:06:06,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 333217792. Throughput: 0: 9880.5, 1: 9737.1. Samples: 333201376. Policy #0 lag: (min: 8.0, avg: 36.0, max: 40.0) [2023-12-26 20:06:06,062][104569] Avg episode reward: [(0, '9267.080'), (1, '9261.324')] [2023-12-26 20:06:06,280][105692] Updated weights for policy 0, policy_version 650314 (0.0011) [2023-12-26 20:06:06,343][105692] Updated weights for policy 0, policy_version 650324 (0.0011) [2023-12-26 20:06:06,406][105692] Updated weights for policy 0, policy_version 650334 (0.0011) [2023-12-26 20:06:06,469][105692] Updated weights for policy 0, policy_version 650344 (0.0011) [2023-12-26 20:06:06,765][105620] Updated weights for policy 1, policy_version 651138 (0.0011) [2023-12-26 20:06:06,822][105620] Updated weights for policy 1, policy_version 651148 (0.0006) [2023-12-26 20:06:06,886][105620] Updated weights for policy 1, policy_version 651158 (0.0009) [2023-12-26 20:06:07,210][105692] Updated weights for policy 0, policy_version 650354 (0.0011) [2023-12-26 20:06:07,264][105692] Updated weights for policy 0, policy_version 650364 (0.0009) [2023-12-26 20:06:07,313][105692] Updated weights for policy 0, policy_version 650374 (0.0010) [2023-12-26 20:06:07,537][105620] Updated weights for policy 1, policy_version 651168 (0.0006) [2023-12-26 20:06:07,591][105620] Updated weights for policy 1, policy_version 651178 (0.0005) [2023-12-26 20:06:07,653][105620] Updated weights for policy 1, policy_version 651188 (0.0005) [2023-12-26 20:06:08,023][105692] Updated weights for policy 0, policy_version 650384 (0.0008) [2023-12-26 20:06:08,078][105692] Updated weights for policy 0, policy_version 650394 (0.0008) [2023-12-26 20:06:08,130][105692] Updated weights for policy 0, policy_version 650404 (0.0008) [2023-12-26 20:06:08,215][105620] Updated weights for policy 1, policy_version 651198 (0.0006) [2023-12-26 20:06:08,262][105620] Updated weights for policy 1, policy_version 651208 (0.0005) [2023-12-26 20:06:08,314][105620] Updated weights for policy 1, policy_version 651218 (0.0005) [2023-12-26 20:06:08,814][105692] Updated weights for policy 0, policy_version 650414 (0.0007) [2023-12-26 20:06:08,884][105692] Updated weights for policy 0, policy_version 650424 (0.0005) [2023-12-26 20:06:08,941][105692] Updated weights for policy 0, policy_version 650434 (0.0005) [2023-12-26 20:06:09,011][105620] Updated weights for policy 1, policy_version 651228 (0.0010) [2023-12-26 20:06:09,059][105620] Updated weights for policy 1, policy_version 651238 (0.0010) [2023-12-26 20:06:09,107][105620] Updated weights for policy 1, policy_version 651248 (0.0010) [2023-12-26 20:06:09,507][105692] Updated weights for policy 0, policy_version 650444 (0.0006) [2023-12-26 20:06:09,569][105692] Updated weights for policy 0, policy_version 650454 (0.0008) [2023-12-26 20:06:09,628][105692] Updated weights for policy 0, policy_version 650464 (0.0008) [2023-12-26 20:06:09,797][105620] Updated weights for policy 1, policy_version 651258 (0.0010) [2023-12-26 20:06:09,868][105620] Updated weights for policy 1, policy_version 651268 (0.0010) [2023-12-26 20:06:09,940][105620] Updated weights for policy 1, policy_version 651278 (0.0011) [2023-12-26 20:06:09,999][105620] Updated weights for policy 1, policy_version 651288 (0.0010) [2023-12-26 20:06:10,356][105692] Updated weights for policy 0, policy_version 650474 (0.0009) [2023-12-26 20:06:10,418][105692] Updated weights for policy 0, policy_version 650484 (0.0011) [2023-12-26 20:06:10,467][105692] Updated weights for policy 0, policy_version 650494 (0.0009) [2023-12-26 20:06:10,511][105692] Updated weights for policy 0, policy_version 650504 (0.0006) [2023-12-26 20:06:10,739][105620] Updated weights for policy 1, policy_version 651298 (0.0007) [2023-12-26 20:06:10,805][105620] Updated weights for policy 1, policy_version 651308 (0.0006) [2023-12-26 20:06:10,862][105620] Updated weights for policy 1, policy_version 651318 (0.0009) [2023-12-26 20:06:11,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 333316096. Throughput: 0: 9943.9, 1: 9880.5. Samples: 333323832. Policy #0 lag: (min: 8.0, avg: 36.0, max: 40.0) [2023-12-26 20:06:11,062][104569] Avg episode reward: [(0, '9266.854'), (1, '9261.053')] [2023-12-26 20:06:11,197][105692] Updated weights for policy 0, policy_version 650514 (0.0009) [2023-12-26 20:06:11,261][105692] Updated weights for policy 0, policy_version 650524 (0.0006) [2023-12-26 20:06:11,322][105692] Updated weights for policy 0, policy_version 650534 (0.0006) [2023-12-26 20:06:11,546][105620] Updated weights for policy 1, policy_version 651328 (0.0006) [2023-12-26 20:06:11,618][105620] Updated weights for policy 1, policy_version 651338 (0.0006) [2023-12-26 20:06:11,685][105620] Updated weights for policy 1, policy_version 651348 (0.0007) [2023-12-26 20:06:12,075][105692] Updated weights for policy 0, policy_version 650544 (0.0008) [2023-12-26 20:06:12,136][105692] Updated weights for policy 0, policy_version 650554 (0.0008) [2023-12-26 20:06:12,197][105692] Updated weights for policy 0, policy_version 650564 (0.0008) [2023-12-26 20:06:12,424][105620] Updated weights for policy 1, policy_version 651358 (0.0010) [2023-12-26 20:06:12,487][105620] Updated weights for policy 1, policy_version 651368 (0.0010) [2023-12-26 20:06:12,553][105620] Updated weights for policy 1, policy_version 651378 (0.0006) [2023-12-26 20:06:13,035][105692] Updated weights for policy 0, policy_version 650574 (0.0009) [2023-12-26 20:06:13,107][105692] Updated weights for policy 0, policy_version 650584 (0.0009) [2023-12-26 20:06:13,139][105620] Updated weights for policy 1, policy_version 651388 (0.0005) [2023-12-26 20:06:13,176][105692] Updated weights for policy 0, policy_version 650594 (0.0008) [2023-12-26 20:06:13,195][105620] Updated weights for policy 1, policy_version 651398 (0.0005) [2023-12-26 20:06:13,259][105620] Updated weights for policy 1, policy_version 651408 (0.0005) [2023-12-26 20:06:13,908][105692] Updated weights for policy 0, policy_version 650604 (0.0008) [2023-12-26 20:06:13,967][105620] Updated weights for policy 1, policy_version 651418 (0.0006) [2023-12-26 20:06:13,969][105692] Updated weights for policy 0, policy_version 650614 (0.0007) [2023-12-26 20:06:14,019][105620] Updated weights for policy 1, policy_version 651428 (0.0010) [2023-12-26 20:06:14,029][105692] Updated weights for policy 0, policy_version 650624 (0.0007) [2023-12-26 20:06:14,071][105620] Updated weights for policy 1, policy_version 651438 (0.0010) [2023-12-26 20:06:14,127][105620] Updated weights for policy 1, policy_version 651448 (0.0010) [2023-12-26 20:06:14,721][105692] Updated weights for policy 0, policy_version 650634 (0.0006) [2023-12-26 20:06:14,788][105692] Updated weights for policy 0, policy_version 650644 (0.0008) [2023-12-26 20:06:14,846][105692] Updated weights for policy 0, policy_version 650654 (0.0008) [2023-12-26 20:06:14,885][105620] Updated weights for policy 1, policy_version 651458 (0.0010) [2023-12-26 20:06:14,910][105692] Updated weights for policy 0, policy_version 650664 (0.0008) [2023-12-26 20:06:14,947][105620] Updated weights for policy 1, policy_version 651468 (0.0010) [2023-12-26 20:06:15,011][105620] Updated weights for policy 1, policy_version 651478 (0.0009) [2023-12-26 20:06:15,629][105692] Updated weights for policy 0, policy_version 650674 (0.0006) [2023-12-26 20:06:15,689][105692] Updated weights for policy 0, policy_version 650684 (0.0008) [2023-12-26 20:06:15,740][105692] Updated weights for policy 0, policy_version 650694 (0.0008) [2023-12-26 20:06:15,759][105620] Updated weights for policy 1, policy_version 651488 (0.0010) [2023-12-26 20:06:15,820][105620] Updated weights for policy 1, policy_version 651498 (0.0010) [2023-12-26 20:06:15,874][105620] Updated weights for policy 1, policy_version 651508 (0.0010) [2023-12-26 20:06:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 333414400. Throughput: 0: 9842.8, 1: 9895.0. Samples: 333381280. Policy #0 lag: (min: 8.0, avg: 36.0, max: 40.0) [2023-12-26 20:06:16,063][104569] Avg episode reward: [(0, '9358.469'), (1, '9260.900')] [2023-12-26 20:06:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000651512_166805504.pth... [2023-12-26 20:06:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000650696_166608896.pth... [2023-12-26 20:06:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000650328_166502400.pth [2023-12-26 20:06:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000649544_166313984.pth [2023-12-26 20:06:16,509][105692] Updated weights for policy 0, policy_version 650704 (0.0009) [2023-12-26 20:06:16,536][105620] Updated weights for policy 1, policy_version 651518 (0.0007) [2023-12-26 20:06:16,566][105692] Updated weights for policy 0, policy_version 650714 (0.0008) [2023-12-26 20:06:16,580][105620] Updated weights for policy 1, policy_version 651528 (0.0007) [2023-12-26 20:06:16,623][105692] Updated weights for policy 0, policy_version 650724 (0.0009) [2023-12-26 20:06:16,629][105620] Updated weights for policy 1, policy_version 651538 (0.0006) [2023-12-26 20:06:17,314][105692] Updated weights for policy 0, policy_version 650734 (0.0008) [2023-12-26 20:06:17,365][105692] Updated weights for policy 0, policy_version 650744 (0.0009) [2023-12-26 20:06:17,412][105692] Updated weights for policy 0, policy_version 650754 (0.0009) [2023-12-26 20:06:17,441][105620] Updated weights for policy 1, policy_version 651548 (0.0010) [2023-12-26 20:06:17,498][105620] Updated weights for policy 1, policy_version 651558 (0.0008) [2023-12-26 20:06:17,552][105620] Updated weights for policy 1, policy_version 651568 (0.0009) [2023-12-26 20:06:18,217][105692] Updated weights for policy 0, policy_version 650764 (0.0007) [2023-12-26 20:06:18,246][105620] Updated weights for policy 1, policy_version 651578 (0.0009) [2023-12-26 20:06:18,264][105692] Updated weights for policy 0, policy_version 650774 (0.0007) [2023-12-26 20:06:18,299][105620] Updated weights for policy 1, policy_version 651588 (0.0008) [2023-12-26 20:06:18,316][105692] Updated weights for policy 0, policy_version 650784 (0.0008) [2023-12-26 20:06:18,362][105620] Updated weights for policy 1, policy_version 651598 (0.0007) [2023-12-26 20:06:18,421][105620] Updated weights for policy 1, policy_version 651608 (0.0008) [2023-12-26 20:06:19,135][105620] Updated weights for policy 1, policy_version 651618 (0.0005) [2023-12-26 20:06:19,141][105692] Updated weights for policy 0, policy_version 650794 (0.0007) [2023-12-26 20:06:19,191][105692] Updated weights for policy 0, policy_version 650804 (0.0008) [2023-12-26 20:06:19,203][105620] Updated weights for policy 1, policy_version 651628 (0.0005) [2023-12-26 20:06:19,259][105692] Updated weights for policy 0, policy_version 650814 (0.0007) [2023-12-26 20:06:19,265][105620] Updated weights for policy 1, policy_version 651638 (0.0008) [2023-12-26 20:06:19,321][105692] Updated weights for policy 0, policy_version 650824 (0.0009) [2023-12-26 20:06:20,025][105620] Updated weights for policy 1, policy_version 651648 (0.0009) [2023-12-26 20:06:20,034][105692] Updated weights for policy 0, policy_version 650834 (0.0006) [2023-12-26 20:06:20,080][105620] Updated weights for policy 1, policy_version 651658 (0.0008) [2023-12-26 20:06:20,094][105692] Updated weights for policy 0, policy_version 650844 (0.0006) [2023-12-26 20:06:20,134][105620] Updated weights for policy 1, policy_version 651668 (0.0008) [2023-12-26 20:06:20,154][105692] Updated weights for policy 0, policy_version 650854 (0.0006) [2023-12-26 20:06:20,866][105692] Updated weights for policy 0, policy_version 650864 (0.0008) [2023-12-26 20:06:20,930][105692] Updated weights for policy 0, policy_version 650874 (0.0008) [2023-12-26 20:06:20,944][105620] Updated weights for policy 1, policy_version 651678 (0.0009) [2023-12-26 20:06:20,985][105692] Updated weights for policy 0, policy_version 650884 (0.0008) [2023-12-26 20:06:21,011][105620] Updated weights for policy 1, policy_version 651688 (0.0009) [2023-12-26 20:06:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 333504512. Throughput: 0: 9733.0, 1: 9882.8. Samples: 333494276. Policy #0 lag: (min: 8.0, avg: 36.0, max: 40.0) [2023-12-26 20:06:21,063][104569] Avg episode reward: [(0, '9357.679'), (1, '9261.003')] [2023-12-26 20:06:21,082][105620] Updated weights for policy 1, policy_version 651698 (0.0009) [2023-12-26 20:06:21,773][105692] Updated weights for policy 0, policy_version 650894 (0.0008) [2023-12-26 20:06:21,833][105692] Updated weights for policy 0, policy_version 650904 (0.0009) [2023-12-26 20:06:21,882][105620] Updated weights for policy 1, policy_version 651708 (0.0008) [2023-12-26 20:06:21,896][105692] Updated weights for policy 0, policy_version 650914 (0.0009) [2023-12-26 20:06:21,945][105620] Updated weights for policy 1, policy_version 651718 (0.0007) [2023-12-26 20:06:22,003][105620] Updated weights for policy 1, policy_version 651728 (0.0009) [2023-12-26 20:06:22,685][105692] Updated weights for policy 0, policy_version 650924 (0.0007) [2023-12-26 20:06:22,748][105692] Updated weights for policy 0, policy_version 650934 (0.0006) [2023-12-26 20:06:22,785][105620] Updated weights for policy 1, policy_version 651738 (0.0008) [2023-12-26 20:06:22,805][105692] Updated weights for policy 0, policy_version 650944 (0.0006) [2023-12-26 20:06:22,851][105620] Updated weights for policy 1, policy_version 651748 (0.0008) [2023-12-26 20:06:22,911][105620] Updated weights for policy 1, policy_version 651758 (0.0009) [2023-12-26 20:06:22,970][105620] Updated weights for policy 1, policy_version 651768 (0.0009) [2023-12-26 20:06:23,486][105692] Updated weights for policy 0, policy_version 650954 (0.0007) [2023-12-26 20:06:23,545][105692] Updated weights for policy 0, policy_version 650964 (0.0007) [2023-12-26 20:06:23,600][105692] Updated weights for policy 0, policy_version 650974 (0.0009) [2023-12-26 20:06:23,659][105692] Updated weights for policy 0, policy_version 650984 (0.0009) [2023-12-26 20:06:23,759][105620] Updated weights for policy 1, policy_version 651778 (0.0009) [2023-12-26 20:06:23,806][105620] Updated weights for policy 1, policy_version 651788 (0.0009) [2023-12-26 20:06:23,851][105620] Updated weights for policy 1, policy_version 651798 (0.0008) [2023-12-26 20:06:24,403][105692] Updated weights for policy 0, policy_version 650994 (0.0010) [2023-12-26 20:06:24,452][105692] Updated weights for policy 0, policy_version 651004 (0.0010) [2023-12-26 20:06:24,506][105692] Updated weights for policy 0, policy_version 651014 (0.0010) [2023-12-26 20:06:24,592][105620] Updated weights for policy 1, policy_version 651808 (0.0007) [2023-12-26 20:06:24,655][105620] Updated weights for policy 1, policy_version 651818 (0.0008) [2023-12-26 20:06:24,722][105620] Updated weights for policy 1, policy_version 651828 (0.0009) [2023-12-26 20:06:25,142][105692] Updated weights for policy 0, policy_version 651024 (0.0010) [2023-12-26 20:06:25,193][105692] Updated weights for policy 0, policy_version 651034 (0.0010) [2023-12-26 20:06:25,251][105692] Updated weights for policy 0, policy_version 651044 (0.0010) [2023-12-26 20:06:25,456][105620] Updated weights for policy 1, policy_version 651838 (0.0009) [2023-12-26 20:06:25,522][105620] Updated weights for policy 1, policy_version 651848 (0.0009) [2023-12-26 20:06:25,568][105620] Updated weights for policy 1, policy_version 651858 (0.0009) [2023-12-26 20:06:25,948][105692] Updated weights for policy 0, policy_version 651054 (0.0008) [2023-12-26 20:06:26,006][105692] Updated weights for policy 0, policy_version 651064 (0.0010) [2023-12-26 20:06:26,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 333594624. Throughput: 0: 9793.4, 1: 9796.2. Samples: 333607692. Policy #0 lag: (min: 8.0, avg: 36.0, max: 40.0) [2023-12-26 20:06:26,062][104569] Avg episode reward: [(0, '9357.374'), (1, '9169.559')] [2023-12-26 20:06:26,065][105692] Updated weights for policy 0, policy_version 651074 (0.0010) [2023-12-26 20:06:26,231][105620] Updated weights for policy 1, policy_version 651868 (0.0009) [2023-12-26 20:06:26,284][105620] Updated weights for policy 1, policy_version 651878 (0.0010) [2023-12-26 20:06:26,347][105620] Updated weights for policy 1, policy_version 651888 (0.0010) [2023-12-26 20:06:26,372][105586] KL-divergence is very high: 122.3717 [2023-12-26 20:06:26,623][105692] Updated weights for policy 0, policy_version 651084 (0.0008) [2023-12-26 20:06:26,674][105692] Updated weights for policy 0, policy_version 651094 (0.0007) [2023-12-26 20:06:26,737][105692] Updated weights for policy 0, policy_version 651104 (0.0006) [2023-12-26 20:06:27,129][105620] Updated weights for policy 1, policy_version 651898 (0.0010) [2023-12-26 20:06:27,182][105620] Updated weights for policy 1, policy_version 651908 (0.0009) [2023-12-26 20:06:27,235][105620] Updated weights for policy 1, policy_version 651918 (0.0010) [2023-12-26 20:06:27,292][105620] Updated weights for policy 1, policy_version 651928 (0.0010) [2023-12-26 20:06:27,299][105692] Updated weights for policy 0, policy_version 651114 (0.0005) [2023-12-26 20:06:27,361][105692] Updated weights for policy 0, policy_version 651124 (0.0005) [2023-12-26 20:06:27,433][105692] Updated weights for policy 0, policy_version 651134 (0.0005) [2023-12-26 20:06:27,488][105692] Updated weights for policy 0, policy_version 651144 (0.0009) [2023-12-26 20:06:27,975][105620] Updated weights for policy 1, policy_version 651938 (0.0010) [2023-12-26 20:06:28,026][105620] Updated weights for policy 1, policy_version 651950 (0.0010) [2023-12-26 20:06:28,081][105620] Updated weights for policy 1, policy_version 651960 (0.0010) [2023-12-26 20:06:28,118][105692] Updated weights for policy 0, policy_version 651154 (0.0005) [2023-12-26 20:06:28,176][105692] Updated weights for policy 0, policy_version 651164 (0.0005) [2023-12-26 20:06:28,229][105692] Updated weights for policy 0, policy_version 651174 (0.0005) [2023-12-26 20:06:28,857][105692] Updated weights for policy 0, policy_version 651184 (0.0009) [2023-12-26 20:06:28,910][105692] Updated weights for policy 0, policy_version 651194 (0.0007) [2023-12-26 20:06:28,923][105620] Updated weights for policy 1, policy_version 651970 (0.0009) [2023-12-26 20:06:28,963][105692] Updated weights for policy 0, policy_version 651204 (0.0005) [2023-12-26 20:06:28,968][105620] Updated weights for policy 1, policy_version 651980 (0.0010) [2023-12-26 20:06:29,019][105620] Updated weights for policy 1, policy_version 651990 (0.0010) [2023-12-26 20:06:29,597][105692] Updated weights for policy 0, policy_version 651214 (0.0007) [2023-12-26 20:06:29,643][105692] Updated weights for policy 0, policy_version 651224 (0.0008) [2023-12-26 20:06:29,679][105620] Updated weights for policy 1, policy_version 652000 (0.0009) [2023-12-26 20:06:29,693][105692] Updated weights for policy 0, policy_version 651234 (0.0008) [2023-12-26 20:06:29,734][105620] Updated weights for policy 1, policy_version 652010 (0.0005) [2023-12-26 20:06:29,794][105620] Updated weights for policy 1, policy_version 652020 (0.0005) [2023-12-26 20:06:30,389][105692] Updated weights for policy 0, policy_version 651244 (0.0008) [2023-12-26 20:06:30,444][105692] Updated weights for policy 0, policy_version 651254 (0.0009) [2023-12-26 20:06:30,510][105692] Updated weights for policy 0, policy_version 651264 (0.0010) [2023-12-26 20:06:30,547][105620] Updated weights for policy 1, policy_version 652030 (0.0007) [2023-12-26 20:06:30,598][105620] Updated weights for policy 1, policy_version 652040 (0.0008) [2023-12-26 20:06:30,651][105620] Updated weights for policy 1, policy_version 652050 (0.0005) [2023-12-26 20:06:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 333701120. Throughput: 0: 9917.0, 1: 9830.5. Samples: 333670056. Policy #0 lag: (min: 8.0, avg: 36.0, max: 40.0) [2023-12-26 20:06:31,063][104569] Avg episode reward: [(0, '9356.876'), (1, '9260.473')] [2023-12-26 20:06:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000651272_166756352.pth... [2023-12-26 20:06:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000652056_166944768.pth... [2023-12-26 20:06:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000650904_166649856.pth [2023-12-26 20:06:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000650120_166461440.pth [2023-12-26 20:06:31,245][105692] Updated weights for policy 0, policy_version 651274 (0.0010) [2023-12-26 20:06:31,313][105692] Updated weights for policy 0, policy_version 651284 (0.0008) [2023-12-26 20:06:31,374][105620] Updated weights for policy 1, policy_version 652060 (0.0006) [2023-12-26 20:06:31,376][105692] Updated weights for policy 0, policy_version 651294 (0.0008) [2023-12-26 20:06:31,431][105620] Updated weights for policy 1, policy_version 652070 (0.0006) [2023-12-26 20:06:31,434][105692] Updated weights for policy 0, policy_version 651304 (0.0008) [2023-12-26 20:06:31,480][105620] Updated weights for policy 1, policy_version 652080 (0.0006) [2023-12-26 20:06:32,155][105620] Updated weights for policy 1, policy_version 652090 (0.0007) [2023-12-26 20:06:32,205][105620] Updated weights for policy 1, policy_version 652100 (0.0007) [2023-12-26 20:06:32,222][105692] Updated weights for policy 0, policy_version 651314 (0.0008) [2023-12-26 20:06:32,263][105620] Updated weights for policy 1, policy_version 652110 (0.0006) [2023-12-26 20:06:32,294][105692] Updated weights for policy 0, policy_version 651324 (0.0007) [2023-12-26 20:06:32,320][105620] Updated weights for policy 1, policy_version 652120 (0.0010) [2023-12-26 20:06:32,356][105692] Updated weights for policy 0, policy_version 651334 (0.0007) [2023-12-26 20:06:32,942][105692] Updated weights for policy 0, policy_version 651344 (0.0008) [2023-12-26 20:06:32,984][105620] Updated weights for policy 1, policy_version 652130 (0.0010) [2023-12-26 20:06:32,990][105692] Updated weights for policy 0, policy_version 651354 (0.0005) [2023-12-26 20:06:33,031][105620] Updated weights for policy 1, policy_version 652140 (0.0010) [2023-12-26 20:06:33,034][105692] Updated weights for policy 0, policy_version 651364 (0.0005) [2023-12-26 20:06:33,092][105620] Updated weights for policy 1, policy_version 652150 (0.0010) [2023-12-26 20:06:33,767][105692] Updated weights for policy 0, policy_version 651374 (0.0006) [2023-12-26 20:06:33,818][105692] Updated weights for policy 0, policy_version 651384 (0.0007) [2023-12-26 20:06:33,841][105620] Updated weights for policy 1, policy_version 652160 (0.0010) [2023-12-26 20:06:33,880][105692] Updated weights for policy 0, policy_version 651394 (0.0009) [2023-12-26 20:06:33,899][105620] Updated weights for policy 1, policy_version 652170 (0.0010) [2023-12-26 20:06:33,957][105620] Updated weights for policy 1, policy_version 652180 (0.0010) [2023-12-26 20:06:34,466][105692] Updated weights for policy 0, policy_version 651404 (0.0005) [2023-12-26 20:06:34,526][105692] Updated weights for policy 0, policy_version 651414 (0.0007) [2023-12-26 20:06:34,583][105692] Updated weights for policy 0, policy_version 651424 (0.0008) [2023-12-26 20:06:34,733][105620] Updated weights for policy 1, policy_version 652190 (0.0010) [2023-12-26 20:06:34,792][105620] Updated weights for policy 1, policy_version 652200 (0.0010) [2023-12-26 20:06:34,858][105620] Updated weights for policy 1, policy_version 652210 (0.0010) [2023-12-26 20:06:35,170][105692] Updated weights for policy 0, policy_version 651434 (0.0007) [2023-12-26 20:06:35,235][105692] Updated weights for policy 0, policy_version 651444 (0.0006) [2023-12-26 20:06:35,301][105692] Updated weights for policy 0, policy_version 651454 (0.0010) [2023-12-26 20:06:35,359][105692] Updated weights for policy 0, policy_version 651464 (0.0010) [2023-12-26 20:06:35,568][105620] Updated weights for policy 1, policy_version 652220 (0.0008) [2023-12-26 20:06:35,619][105620] Updated weights for policy 1, policy_version 652230 (0.0005) [2023-12-26 20:06:35,670][105620] Updated weights for policy 1, policy_version 652240 (0.0005) [2023-12-26 20:06:35,949][105692] Updated weights for policy 0, policy_version 651474 (0.0005) [2023-12-26 20:06:35,993][105692] Updated weights for policy 0, policy_version 651484 (0.0005) [2023-12-26 20:06:36,040][105692] Updated weights for policy 0, policy_version 651494 (0.0005) [2023-12-26 20:06:36,062][104569] Fps is (10 sec: 21299.3, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 333807616. Throughput: 0: 9881.2, 1: 9877.2. Samples: 333789248. Policy #0 lag: (min: 8.0, avg: 36.0, max: 40.0) [2023-12-26 20:06:36,062][104569] Avg episode reward: [(0, '9356.636'), (1, '9168.898')] [2023-12-26 20:06:36,270][105620] Updated weights for policy 1, policy_version 652250 (0.0006) [2023-12-26 20:06:36,330][105620] Updated weights for policy 1, policy_version 652260 (0.0010) [2023-12-26 20:06:36,397][105620] Updated weights for policy 1, policy_version 652270 (0.0011) [2023-12-26 20:06:36,461][105620] Updated weights for policy 1, policy_version 652280 (0.0008) [2023-12-26 20:06:36,742][105692] Updated weights for policy 0, policy_version 651504 (0.0010) [2023-12-26 20:06:36,794][105692] Updated weights for policy 0, policy_version 651514 (0.0011) [2023-12-26 20:06:36,843][105692] Updated weights for policy 0, policy_version 651524 (0.0011) [2023-12-26 20:06:37,191][105620] Updated weights for policy 1, policy_version 652290 (0.0010) [2023-12-26 20:06:37,251][105620] Updated weights for policy 1, policy_version 652300 (0.0010) [2023-12-26 20:06:37,307][105620] Updated weights for policy 1, policy_version 652310 (0.0010) [2023-12-26 20:06:37,601][105692] Updated weights for policy 0, policy_version 651534 (0.0010) [2023-12-26 20:06:37,666][105692] Updated weights for policy 0, policy_version 651544 (0.0010) [2023-12-26 20:06:37,738][105692] Updated weights for policy 0, policy_version 651554 (0.0010) [2023-12-26 20:06:37,964][105620] Updated weights for policy 1, policy_version 652320 (0.0011) [2023-12-26 20:06:38,035][105620] Updated weights for policy 1, policy_version 652330 (0.0010) [2023-12-26 20:06:38,103][105620] Updated weights for policy 1, policy_version 652340 (0.0008) [2023-12-26 20:06:38,315][105692] Updated weights for policy 0, policy_version 651564 (0.0008) [2023-12-26 20:06:38,376][105692] Updated weights for policy 0, policy_version 651574 (0.0011) [2023-12-26 20:06:38,439][105692] Updated weights for policy 0, policy_version 651584 (0.0011) [2023-12-26 20:06:38,768][105620] Updated weights for policy 1, policy_version 652350 (0.0009) [2023-12-26 20:06:38,826][105620] Updated weights for policy 1, policy_version 652360 (0.0009) [2023-12-26 20:06:38,885][105620] Updated weights for policy 1, policy_version 652370 (0.0009) [2023-12-26 20:06:39,139][105692] Updated weights for policy 0, policy_version 651594 (0.0010) [2023-12-26 20:06:39,193][105692] Updated weights for policy 0, policy_version 651604 (0.0006) [2023-12-26 20:06:39,253][105692] Updated weights for policy 0, policy_version 651614 (0.0009) [2023-12-26 20:06:39,310][105692] Updated weights for policy 0, policy_version 651624 (0.0007) [2023-12-26 20:06:39,612][105620] Updated weights for policy 1, policy_version 652380 (0.0009) [2023-12-26 20:06:39,668][105620] Updated weights for policy 1, policy_version 652390 (0.0009) [2023-12-26 20:06:39,721][105620] Updated weights for policy 1, policy_version 652400 (0.0009) [2023-12-26 20:06:40,014][105692] Updated weights for policy 0, policy_version 651634 (0.0008) [2023-12-26 20:06:40,069][105692] Updated weights for policy 0, policy_version 651644 (0.0008) [2023-12-26 20:06:40,135][105692] Updated weights for policy 0, policy_version 651654 (0.0008) [2023-12-26 20:06:40,591][105620] Updated weights for policy 1, policy_version 652410 (0.0010) [2023-12-26 20:06:40,653][105620] Updated weights for policy 1, policy_version 652420 (0.0008) [2023-12-26 20:06:40,724][105620] Updated weights for policy 1, policy_version 652430 (0.0008) [2023-12-26 20:06:40,794][105620] Updated weights for policy 1, policy_version 652440 (0.0005) [2023-12-26 20:06:40,813][105692] Updated weights for policy 0, policy_version 651664 (0.0007) [2023-12-26 20:06:40,861][105692] Updated weights for policy 0, policy_version 651674 (0.0005) [2023-12-26 20:06:40,914][105692] Updated weights for policy 0, policy_version 651684 (0.0007) [2023-12-26 20:06:41,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 333905920. Throughput: 0: 9954.1, 1: 9835.4. Samples: 333910628. Policy #0 lag: (min: 8.0, avg: 36.0, max: 40.0) [2023-12-26 20:06:41,062][104569] Avg episode reward: [(0, '9356.281'), (1, '9168.345')] [2023-12-26 20:06:41,424][105620] Updated weights for policy 1, policy_version 652450 (0.0008) [2023-12-26 20:06:41,489][105620] Updated weights for policy 1, policy_version 652460 (0.0008) [2023-12-26 20:06:41,555][105620] Updated weights for policy 1, policy_version 652470 (0.0009) [2023-12-26 20:06:41,654][105692] Updated weights for policy 0, policy_version 651694 (0.0010) [2023-12-26 20:06:41,719][105692] Updated weights for policy 0, policy_version 651704 (0.0010) [2023-12-26 20:06:41,786][105692] Updated weights for policy 0, policy_version 651714 (0.0011) [2023-12-26 20:06:42,229][105620] Updated weights for policy 1, policy_version 652480 (0.0010) [2023-12-26 20:06:42,289][105620] Updated weights for policy 1, policy_version 652490 (0.0006) [2023-12-26 20:06:42,350][105620] Updated weights for policy 1, policy_version 652500 (0.0006) [2023-12-26 20:06:42,529][105692] Updated weights for policy 0, policy_version 651724 (0.0009) [2023-12-26 20:06:42,591][105692] Updated weights for policy 0, policy_version 651734 (0.0009) [2023-12-26 20:06:42,651][105692] Updated weights for policy 0, policy_version 651744 (0.0010) [2023-12-26 20:06:43,052][105620] Updated weights for policy 1, policy_version 652510 (0.0008) [2023-12-26 20:06:43,107][105620] Updated weights for policy 1, policy_version 652520 (0.0009) [2023-12-26 20:06:43,168][105620] Updated weights for policy 1, policy_version 652530 (0.0010) [2023-12-26 20:06:43,316][105692] Updated weights for policy 0, policy_version 651754 (0.0009) [2023-12-26 20:06:43,364][105692] Updated weights for policy 0, policy_version 651764 (0.0009) [2023-12-26 20:06:43,415][105692] Updated weights for policy 0, policy_version 651774 (0.0009) [2023-12-26 20:06:43,466][105692] Updated weights for policy 0, policy_version 651784 (0.0009) [2023-12-26 20:06:43,947][105620] Updated weights for policy 1, policy_version 652540 (0.0009) [2023-12-26 20:06:44,005][105620] Updated weights for policy 1, policy_version 652550 (0.0009) [2023-12-26 20:06:44,057][105620] Updated weights for policy 1, policy_version 652560 (0.0009) [2023-12-26 20:06:44,252][105692] Updated weights for policy 0, policy_version 651794 (0.0009) [2023-12-26 20:06:44,299][105692] Updated weights for policy 0, policy_version 651804 (0.0008) [2023-12-26 20:06:44,349][105692] Updated weights for policy 0, policy_version 651814 (0.0008) [2023-12-26 20:06:44,864][105620] Updated weights for policy 1, policy_version 652570 (0.0009) [2023-12-26 20:06:44,927][105620] Updated weights for policy 1, policy_version 652580 (0.0008) [2023-12-26 20:06:44,988][105620] Updated weights for policy 1, policy_version 652590 (0.0005) [2023-12-26 20:06:45,052][105620] Updated weights for policy 1, policy_version 652600 (0.0008) [2023-12-26 20:06:45,062][105692] Updated weights for policy 0, policy_version 651824 (0.0007) [2023-12-26 20:06:45,126][105692] Updated weights for policy 0, policy_version 651834 (0.0006) [2023-12-26 20:06:45,192][105692] Updated weights for policy 0, policy_version 651844 (0.0005) [2023-12-26 20:06:45,712][105620] Updated weights for policy 1, policy_version 652610 (0.0010) [2023-12-26 20:06:45,757][105620] Updated weights for policy 1, policy_version 652620 (0.0010) [2023-12-26 20:06:45,806][105620] Updated weights for policy 1, policy_version 652630 (0.0010) [2023-12-26 20:06:45,919][105692] Updated weights for policy 0, policy_version 651854 (0.0008) [2023-12-26 20:06:45,964][105692] Updated weights for policy 0, policy_version 651864 (0.0008) [2023-12-26 20:06:46,011][105692] Updated weights for policy 0, policy_version 651874 (0.0008) [2023-12-26 20:06:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 334004224. Throughput: 0: 9887.0, 1: 9789.0. Samples: 333968068. Policy #0 lag: (min: 8.0, avg: 36.0, max: 40.0) [2023-12-26 20:06:46,063][104569] Avg episode reward: [(0, '9355.799'), (1, '9168.564')] [2023-12-26 20:06:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000651880_166912000.pth... [2023-12-26 20:06:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000652632_167092224.pth... [2023-12-26 20:06:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000651512_166805504.pth [2023-12-26 20:06:46,093][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000650696_166608896.pth [2023-12-26 20:06:46,553][105620] Updated weights for policy 1, policy_version 652640 (0.0006) [2023-12-26 20:06:46,609][105620] Updated weights for policy 1, policy_version 652650 (0.0005) [2023-12-26 20:06:46,643][105692] Updated weights for policy 0, policy_version 651884 (0.0008) [2023-12-26 20:06:46,668][105620] Updated weights for policy 1, policy_version 652660 (0.0005) [2023-12-26 20:06:46,691][105692] Updated weights for policy 0, policy_version 651894 (0.0009) [2023-12-26 20:06:46,744][105692] Updated weights for policy 0, policy_version 651904 (0.0010) [2023-12-26 20:06:47,262][105620] Updated weights for policy 1, policy_version 652670 (0.0005) [2023-12-26 20:06:47,327][105620] Updated weights for policy 1, policy_version 652680 (0.0005) [2023-12-26 20:06:47,399][105620] Updated weights for policy 1, policy_version 652690 (0.0008) [2023-12-26 20:06:47,428][105692] Updated weights for policy 0, policy_version 651914 (0.0009) [2023-12-26 20:06:47,479][105692] Updated weights for policy 0, policy_version 651924 (0.0008) [2023-12-26 20:06:47,538][105692] Updated weights for policy 0, policy_version 651934 (0.0010) [2023-12-26 20:06:47,594][105692] Updated weights for policy 0, policy_version 651944 (0.0008) [2023-12-26 20:06:47,911][105620] Updated weights for policy 1, policy_version 652700 (0.0008) [2023-12-26 20:06:47,964][105620] Updated weights for policy 1, policy_version 652710 (0.0005) [2023-12-26 20:06:48,012][105620] Updated weights for policy 1, policy_version 652720 (0.0008) [2023-12-26 20:06:48,224][105692] Updated weights for policy 0, policy_version 651954 (0.0008) [2023-12-26 20:06:48,275][105692] Updated weights for policy 0, policy_version 651964 (0.0009) [2023-12-26 20:06:48,331][105692] Updated weights for policy 0, policy_version 651975 (0.0010) [2023-12-26 20:06:48,690][105620] Updated weights for policy 1, policy_version 652730 (0.0010) [2023-12-26 20:06:48,753][105620] Updated weights for policy 1, policy_version 652740 (0.0008) [2023-12-26 20:06:48,800][105620] Updated weights for policy 1, policy_version 652750 (0.0009) [2023-12-26 20:06:48,850][105620] Updated weights for policy 1, policy_version 652760 (0.0008) [2023-12-26 20:06:49,132][105692] Updated weights for policy 0, policy_version 651985 (0.0008) [2023-12-26 20:06:49,189][105692] Updated weights for policy 0, policy_version 651995 (0.0009) [2023-12-26 20:06:49,264][105692] Updated weights for policy 0, policy_version 652005 (0.0010) [2023-12-26 20:06:49,670][105620] Updated weights for policy 1, policy_version 652770 (0.0009) [2023-12-26 20:06:49,738][105620] Updated weights for policy 1, policy_version 652780 (0.0009) [2023-12-26 20:06:49,803][105620] Updated weights for policy 1, policy_version 652790 (0.0009) [2023-12-26 20:06:49,986][105692] Updated weights for policy 0, policy_version 652015 (0.0008) [2023-12-26 20:06:50,046][105692] Updated weights for policy 0, policy_version 652025 (0.0008) [2023-12-26 20:06:50,105][105692] Updated weights for policy 0, policy_version 652035 (0.0008) [2023-12-26 20:06:50,595][105620] Updated weights for policy 1, policy_version 652800 (0.0008) [2023-12-26 20:06:50,688][105620] Updated weights for policy 1, policy_version 652810 (0.0009) [2023-12-26 20:06:50,753][105620] Updated weights for policy 1, policy_version 652820 (0.0009) [2023-12-26 20:06:50,863][105692] Updated weights for policy 0, policy_version 652045 (0.0008) [2023-12-26 20:06:50,920][105692] Updated weights for policy 0, policy_version 652055 (0.0011) [2023-12-26 20:06:50,973][105692] Updated weights for policy 0, policy_version 652065 (0.0011) [2023-12-26 20:06:51,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 334102528. Throughput: 0: 9908.8, 1: 9773.0. Samples: 334087060. Policy #0 lag: (min: 8.0, avg: 36.0, max: 40.0) [2023-12-26 20:06:51,063][104569] Avg episode reward: [(0, '9271.022'), (1, '9168.721')] [2023-12-26 20:06:51,340][105620] Updated weights for policy 1, policy_version 652830 (0.0006) [2023-12-26 20:06:51,407][105620] Updated weights for policy 1, policy_version 652840 (0.0008) [2023-12-26 20:06:51,477][105620] Updated weights for policy 1, policy_version 652850 (0.0008) [2023-12-26 20:06:51,733][105692] Updated weights for policy 0, policy_version 652075 (0.0007) [2023-12-26 20:06:51,790][105692] Updated weights for policy 0, policy_version 652085 (0.0010) [2023-12-26 20:06:51,851][105692] Updated weights for policy 0, policy_version 652095 (0.0009) [2023-12-26 20:06:52,155][105620] Updated weights for policy 1, policy_version 652860 (0.0008) [2023-12-26 20:06:52,221][105620] Updated weights for policy 1, policy_version 652870 (0.0009) [2023-12-26 20:06:52,281][105620] Updated weights for policy 1, policy_version 652880 (0.0009) [2023-12-26 20:06:52,629][105692] Updated weights for policy 0, policy_version 652105 (0.0009) [2023-12-26 20:06:52,684][105692] Updated weights for policy 0, policy_version 652115 (0.0010) [2023-12-26 20:06:52,745][105692] Updated weights for policy 0, policy_version 652125 (0.0010) [2023-12-26 20:06:52,804][105692] Updated weights for policy 0, policy_version 652135 (0.0010) [2023-12-26 20:06:53,069][105620] Updated weights for policy 1, policy_version 652890 (0.0009) [2023-12-26 20:06:53,128][105620] Updated weights for policy 1, policy_version 652900 (0.0008) [2023-12-26 20:06:53,204][105620] Updated weights for policy 1, policy_version 652910 (0.0009) [2023-12-26 20:06:53,266][105620] Updated weights for policy 1, policy_version 652920 (0.0009) [2023-12-26 20:06:53,483][105692] Updated weights for policy 0, policy_version 652145 (0.0009) [2023-12-26 20:06:53,532][105692] Updated weights for policy 0, policy_version 652155 (0.0009) [2023-12-26 20:06:53,579][105692] Updated weights for policy 0, policy_version 652165 (0.0009) [2023-12-26 20:06:54,015][105620] Updated weights for policy 1, policy_version 652930 (0.0007) [2023-12-26 20:06:54,073][105620] Updated weights for policy 1, policy_version 652940 (0.0005) [2023-12-26 20:06:54,140][105620] Updated weights for policy 1, policy_version 652950 (0.0006) [2023-12-26 20:06:54,327][105692] Updated weights for policy 0, policy_version 652175 (0.0009) [2023-12-26 20:06:54,374][105692] Updated weights for policy 0, policy_version 652185 (0.0009) [2023-12-26 20:06:54,423][105692] Updated weights for policy 0, policy_version 652195 (0.0009) [2023-12-26 20:06:54,834][105620] Updated weights for policy 1, policy_version 652960 (0.0008) [2023-12-26 20:06:54,894][105620] Updated weights for policy 1, policy_version 652970 (0.0008) [2023-12-26 20:06:54,959][105620] Updated weights for policy 1, policy_version 652980 (0.0007) [2023-12-26 20:06:55,232][105692] Updated weights for policy 0, policy_version 652205 (0.0010) [2023-12-26 20:06:55,289][105692] Updated weights for policy 0, policy_version 652215 (0.0010) [2023-12-26 20:06:55,352][105692] Updated weights for policy 0, policy_version 652226 (0.0010) [2023-12-26 20:06:55,571][105620] Updated weights for policy 1, policy_version 652990 (0.0009) [2023-12-26 20:06:55,618][105620] Updated weights for policy 1, policy_version 653000 (0.0009) [2023-12-26 20:06:55,667][105620] Updated weights for policy 1, policy_version 653010 (0.0008) [2023-12-26 20:06:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 334192640. Throughput: 0: 9843.6, 1: 9674.3. Samples: 334202140. Policy #0 lag: (min: 8.0, avg: 36.0, max: 40.0) [2023-12-26 20:06:56,062][104569] Avg episode reward: [(0, '9270.817'), (1, '9169.069')] [2023-12-26 20:06:56,164][105692] Updated weights for policy 0, policy_version 652236 (0.0009) [2023-12-26 20:06:56,219][105692] Updated weights for policy 0, policy_version 652246 (0.0008) [2023-12-26 20:06:56,274][105692] Updated weights for policy 0, policy_version 652256 (0.0008) [2023-12-26 20:06:56,285][105620] Updated weights for policy 1, policy_version 653020 (0.0009) [2023-12-26 20:06:56,342][105620] Updated weights for policy 1, policy_version 653030 (0.0010) [2023-12-26 20:06:56,400][105620] Updated weights for policy 1, policy_version 653040 (0.0010) [2023-12-26 20:06:56,963][105620] Updated weights for policy 1, policy_version 653050 (0.0009) [2023-12-26 20:06:57,028][105620] Updated weights for policy 1, policy_version 653060 (0.0005) [2023-12-26 20:06:57,078][105620] Updated weights for policy 1, policy_version 653070 (0.0005) [2023-12-26 20:06:57,138][105620] Updated weights for policy 1, policy_version 653080 (0.0005) [2023-12-26 20:06:57,146][105692] Updated weights for policy 0, policy_version 652266 (0.0006) [2023-12-26 20:06:57,204][105692] Updated weights for policy 0, policy_version 652276 (0.0010) [2023-12-26 20:06:57,257][105692] Updated weights for policy 0, policy_version 652288 (0.0010) [2023-12-26 20:06:57,630][105620] Updated weights for policy 1, policy_version 653090 (0.0005) [2023-12-26 20:06:57,680][105620] Updated weights for policy 1, policy_version 653100 (0.0005) [2023-12-26 20:06:57,736][105620] Updated weights for policy 1, policy_version 653110 (0.0006) [2023-12-26 20:06:58,121][105692] Updated weights for policy 0, policy_version 652298 (0.0009) [2023-12-26 20:06:58,178][105692] Updated weights for policy 0, policy_version 652308 (0.0006) [2023-12-26 20:06:58,235][105692] Updated weights for policy 0, policy_version 652318 (0.0006) [2023-12-26 20:06:58,286][105692] Updated weights for policy 0, policy_version 652328 (0.0009) [2023-12-26 20:06:58,453][105620] Updated weights for policy 1, policy_version 653120 (0.0008) [2023-12-26 20:06:58,516][105620] Updated weights for policy 1, policy_version 653130 (0.0011) [2023-12-26 20:06:58,577][105620] Updated weights for policy 1, policy_version 653140 (0.0009) [2023-12-26 20:06:59,131][105692] Updated weights for policy 0, policy_version 652338 (0.0009) [2023-12-26 20:06:59,184][105692] Updated weights for policy 0, policy_version 652348 (0.0009) [2023-12-26 20:06:59,238][105692] Updated weights for policy 0, policy_version 652358 (0.0012) [2023-12-26 20:06:59,314][105620] Updated weights for policy 1, policy_version 653150 (0.0010) [2023-12-26 20:06:59,382][105620] Updated weights for policy 1, policy_version 653160 (0.0009) [2023-12-26 20:06:59,443][105620] Updated weights for policy 1, policy_version 653170 (0.0009) [2023-12-26 20:06:59,996][105692] Updated weights for policy 0, policy_version 652368 (0.0009) [2023-12-26 20:07:00,057][105692] Updated weights for policy 0, policy_version 652378 (0.0006) [2023-12-26 20:07:00,116][105692] Updated weights for policy 0, policy_version 652388 (0.0006) [2023-12-26 20:07:00,207][105620] Updated weights for policy 1, policy_version 653180 (0.0009) [2023-12-26 20:07:00,263][105620] Updated weights for policy 1, policy_version 653190 (0.0008) [2023-12-26 20:07:00,312][105620] Updated weights for policy 1, policy_version 653200 (0.0009) [2023-12-26 20:07:00,831][105692] Updated weights for policy 0, policy_version 652398 (0.0009) [2023-12-26 20:07:00,892][105692] Updated weights for policy 0, policy_version 652408 (0.0009) [2023-12-26 20:07:00,957][105692] Updated weights for policy 0, policy_version 652418 (0.0010) [2023-12-26 20:07:01,000][105620] Updated weights for policy 1, policy_version 653210 (0.0009) [2023-12-26 20:07:01,059][105620] Updated weights for policy 1, policy_version 653220 (0.0010) [2023-12-26 20:07:01,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 334290944. Throughput: 0: 9794.4, 1: 9749.0. Samples: 334260732. Policy #0 lag: (min: 8.0, avg: 36.0, max: 40.0) [2023-12-26 20:07:01,062][104569] Avg episode reward: [(0, '9354.680'), (1, '9351.796')] [2023-12-26 20:07:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000652424_167051264.pth... [2023-12-26 20:07:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000651272_166756352.pth [2023-12-26 20:07:01,122][105620] Updated weights for policy 1, policy_version 653230 (0.0011) [2023-12-26 20:07:01,188][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000653240_167247872.pth... [2023-12-26 20:07:01,189][105620] Updated weights for policy 1, policy_version 653240 (0.0011) [2023-12-26 20:07:01,193][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000652056_166944768.pth [2023-12-26 20:07:01,729][105692] Updated weights for policy 0, policy_version 652428 (0.0009) [2023-12-26 20:07:01,784][105692] Updated weights for policy 0, policy_version 652438 (0.0008) [2023-12-26 20:07:01,835][105692] Updated weights for policy 0, policy_version 652448 (0.0008) [2023-12-26 20:07:01,931][105620] Updated weights for policy 1, policy_version 653250 (0.0010) [2023-12-26 20:07:01,979][105620] Updated weights for policy 1, policy_version 653260 (0.0010) [2023-12-26 20:07:02,032][105620] Updated weights for policy 1, policy_version 653270 (0.0010) [2023-12-26 20:07:02,615][105692] Updated weights for policy 0, policy_version 652458 (0.0009) [2023-12-26 20:07:02,672][105692] Updated weights for policy 0, policy_version 652468 (0.0010) [2023-12-26 20:07:02,729][105692] Updated weights for policy 0, policy_version 652478 (0.0009) [2023-12-26 20:07:02,771][105620] Updated weights for policy 1, policy_version 653280 (0.0007) [2023-12-26 20:07:02,784][105692] Updated weights for policy 0, policy_version 652488 (0.0008) [2023-12-26 20:07:02,834][105620] Updated weights for policy 1, policy_version 653290 (0.0005) [2023-12-26 20:07:02,888][105620] Updated weights for policy 1, policy_version 653300 (0.0005) [2023-12-26 20:07:03,409][105620] Updated weights for policy 1, policy_version 653310 (0.0006) [2023-12-26 20:07:03,454][105620] Updated weights for policy 1, policy_version 653320 (0.0005) [2023-12-26 20:07:03,503][105620] Updated weights for policy 1, policy_version 653330 (0.0005) [2023-12-26 20:07:03,515][105692] Updated weights for policy 0, policy_version 652498 (0.0005) [2023-12-26 20:07:03,568][105692] Updated weights for policy 0, policy_version 652508 (0.0009) [2023-12-26 20:07:03,622][105692] Updated weights for policy 0, policy_version 652518 (0.0009) [2023-12-26 20:07:04,156][105620] Updated weights for policy 1, policy_version 653340 (0.0006) [2023-12-26 20:07:04,207][105620] Updated weights for policy 1, policy_version 653350 (0.0006) [2023-12-26 20:07:04,281][105620] Updated weights for policy 1, policy_version 653360 (0.0006) [2023-12-26 20:07:04,411][105692] Updated weights for policy 0, policy_version 652528 (0.0011) [2023-12-26 20:07:04,466][105692] Updated weights for policy 0, policy_version 652538 (0.0011) [2023-12-26 20:07:04,525][105692] Updated weights for policy 0, policy_version 652548 (0.0011) [2023-12-26 20:07:04,990][105620] Updated weights for policy 1, policy_version 653370 (0.0008) [2023-12-26 20:07:05,054][105620] Updated weights for policy 1, policy_version 653380 (0.0008) [2023-12-26 20:07:05,113][105620] Updated weights for policy 1, policy_version 653390 (0.0006) [2023-12-26 20:07:05,167][105692] Updated weights for policy 0, policy_version 652558 (0.0009) [2023-12-26 20:07:05,176][105620] Updated weights for policy 1, policy_version 653400 (0.0007) [2023-12-26 20:07:05,233][105692] Updated weights for policy 0, policy_version 652568 (0.0008) [2023-12-26 20:07:05,295][105692] Updated weights for policy 0, policy_version 652578 (0.0010) [2023-12-26 20:07:05,734][105620] Updated weights for policy 1, policy_version 653410 (0.0007) [2023-12-26 20:07:05,796][105620] Updated weights for policy 1, policy_version 653420 (0.0005) [2023-12-26 20:07:05,852][105620] Updated weights for policy 1, policy_version 653430 (0.0006) [2023-12-26 20:07:05,982][105692] Updated weights for policy 0, policy_version 652588 (0.0008) [2023-12-26 20:07:06,036][105692] Updated weights for policy 0, policy_version 652598 (0.0010) [2023-12-26 20:07:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 334389248. Throughput: 0: 9789.8, 1: 9831.1. Samples: 334377216. Policy #0 lag: (min: 8.0, avg: 36.0, max: 40.0) [2023-12-26 20:07:06,063][104569] Avg episode reward: [(0, '9354.813'), (1, '9170.164')] [2023-12-26 20:07:06,100][105692] Updated weights for policy 0, policy_version 652608 (0.0010) [2023-12-26 20:07:06,449][105620] Updated weights for policy 1, policy_version 653440 (0.0006) [2023-12-26 20:07:06,516][105620] Updated weights for policy 1, policy_version 653450 (0.0005) [2023-12-26 20:07:06,582][105620] Updated weights for policy 1, policy_version 653460 (0.0007) [2023-12-26 20:07:06,855][105692] Updated weights for policy 0, policy_version 652618 (0.0010) [2023-12-26 20:07:06,915][105692] Updated weights for policy 0, policy_version 652628 (0.0011) [2023-12-26 20:07:06,974][105692] Updated weights for policy 0, policy_version 652638 (0.0011) [2023-12-26 20:07:07,034][105692] Updated weights for policy 0, policy_version 652648 (0.0011) [2023-12-26 20:07:07,196][105620] Updated weights for policy 1, policy_version 653470 (0.0007) [2023-12-26 20:07:07,253][105620] Updated weights for policy 1, policy_version 653480 (0.0005) [2023-12-26 20:07:07,321][105620] Updated weights for policy 1, policy_version 653490 (0.0007) [2023-12-26 20:07:07,750][105692] Updated weights for policy 0, policy_version 652658 (0.0010) [2023-12-26 20:07:07,801][105692] Updated weights for policy 0, policy_version 652668 (0.0008) [2023-12-26 20:07:07,861][105692] Updated weights for policy 0, policy_version 652678 (0.0005) [2023-12-26 20:07:07,916][105620] Updated weights for policy 1, policy_version 653500 (0.0007) [2023-12-26 20:07:07,975][105620] Updated weights for policy 1, policy_version 653510 (0.0005) [2023-12-26 20:07:08,028][105620] Updated weights for policy 1, policy_version 653520 (0.0006) [2023-12-26 20:07:08,592][105692] Updated weights for policy 0, policy_version 652688 (0.0008) [2023-12-26 20:07:08,612][105620] Updated weights for policy 1, policy_version 653530 (0.0006) [2023-12-26 20:07:08,655][105692] Updated weights for policy 0, policy_version 652698 (0.0007) [2023-12-26 20:07:08,675][105620] Updated weights for policy 1, policy_version 653540 (0.0010) [2023-12-26 20:07:08,709][105692] Updated weights for policy 0, policy_version 652708 (0.0009) [2023-12-26 20:07:08,734][105620] Updated weights for policy 1, policy_version 653550 (0.0010) [2023-12-26 20:07:08,793][105620] Updated weights for policy 1, policy_version 653560 (0.0010) [2023-12-26 20:07:09,484][105692] Updated weights for policy 0, policy_version 652718 (0.0009) [2023-12-26 20:07:09,512][105620] Updated weights for policy 1, policy_version 653570 (0.0008) [2023-12-26 20:07:09,549][105692] Updated weights for policy 0, policy_version 652728 (0.0009) [2023-12-26 20:07:09,564][105620] Updated weights for policy 1, policy_version 653580 (0.0007) [2023-12-26 20:07:09,609][105692] Updated weights for policy 0, policy_version 652738 (0.0009) [2023-12-26 20:07:09,619][105620] Updated weights for policy 1, policy_version 653590 (0.0006) [2023-12-26 20:07:10,348][105620] Updated weights for policy 1, policy_version 653600 (0.0010) [2023-12-26 20:07:10,410][105620] Updated weights for policy 1, policy_version 653610 (0.0007) [2023-12-26 20:07:10,420][105692] Updated weights for policy 0, policy_version 652748 (0.0009) [2023-12-26 20:07:10,471][105620] Updated weights for policy 1, policy_version 653620 (0.0011) [2023-12-26 20:07:10,485][105692] Updated weights for policy 0, policy_version 652758 (0.0006) [2023-12-26 20:07:10,533][105692] Updated weights for policy 0, policy_version 652768 (0.0008) [2023-12-26 20:07:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 334487552. Throughput: 0: 9749.4, 1: 10029.3. Samples: 334497736. Policy #0 lag: (min: 8.0, avg: 36.0, max: 40.0) [2023-12-26 20:07:11,062][104569] Avg episode reward: [(0, '9354.890'), (1, '8988.253')] [2023-12-26 20:07:11,239][105620] Updated weights for policy 1, policy_version 653630 (0.0011) [2023-12-26 20:07:11,256][105692] Updated weights for policy 0, policy_version 652778 (0.0008) [2023-12-26 20:07:11,306][105620] Updated weights for policy 1, policy_version 653640 (0.0010) [2023-12-26 20:07:11,312][105692] Updated weights for policy 0, policy_version 652788 (0.0008) [2023-12-26 20:07:11,375][105620] Updated weights for policy 1, policy_version 653650 (0.0010) [2023-12-26 20:07:11,390][105692] Updated weights for policy 0, policy_version 652798 (0.0007) [2023-12-26 20:07:11,449][105692] Updated weights for policy 0, policy_version 652808 (0.0006) [2023-12-26 20:07:12,061][105620] Updated weights for policy 1, policy_version 653660 (0.0010) [2023-12-26 20:07:12,099][105586] KL-divergence is very high: 102.8117 [2023-12-26 20:07:12,120][105620] Updated weights for policy 1, policy_version 653670 (0.0011) [2023-12-26 20:07:12,141][105586] KL-divergence is very high: 115.4634 [2023-12-26 20:07:12,177][105620] Updated weights for policy 1, policy_version 653680 (0.0010) [2023-12-26 20:07:12,199][105692] Updated weights for policy 0, policy_version 652818 (0.0006) [2023-12-26 20:07:12,257][105692] Updated weights for policy 0, policy_version 652828 (0.0007) [2023-12-26 20:07:12,322][105692] Updated weights for policy 0, policy_version 652838 (0.0009) [2023-12-26 20:07:12,888][105620] Updated weights for policy 1, policy_version 653690 (0.0011) [2023-12-26 20:07:12,941][105620] Updated weights for policy 1, policy_version 653700 (0.0010) [2023-12-26 20:07:12,996][105620] Updated weights for policy 1, policy_version 653710 (0.0010) [2023-12-26 20:07:13,054][105620] Updated weights for policy 1, policy_version 653720 (0.0009) [2023-12-26 20:07:13,114][105692] Updated weights for policy 0, policy_version 652848 (0.0008) [2023-12-26 20:07:13,179][105692] Updated weights for policy 0, policy_version 652858 (0.0009) [2023-12-26 20:07:13,245][105692] Updated weights for policy 0, policy_version 652868 (0.0008) [2023-12-26 20:07:13,789][105620] Updated weights for policy 1, policy_version 653730 (0.0005) [2023-12-26 20:07:13,848][105620] Updated weights for policy 1, policy_version 653740 (0.0005) [2023-12-26 20:07:13,904][105620] Updated weights for policy 1, policy_version 653750 (0.0007) [2023-12-26 20:07:13,954][105692] Updated weights for policy 0, policy_version 652878 (0.0007) [2023-12-26 20:07:14,016][105692] Updated weights for policy 0, policy_version 652888 (0.0007) [2023-12-26 20:07:14,080][105692] Updated weights for policy 0, policy_version 652898 (0.0010) [2023-12-26 20:07:14,485][105620] Updated weights for policy 1, policy_version 653760 (0.0008) [2023-12-26 20:07:14,532][105620] Updated weights for policy 1, policy_version 653770 (0.0009) [2023-12-26 20:07:14,584][105620] Updated weights for policy 1, policy_version 653780 (0.0008) [2023-12-26 20:07:14,870][105692] Updated weights for policy 0, policy_version 652908 (0.0008) [2023-12-26 20:07:14,927][105692] Updated weights for policy 0, policy_version 652918 (0.0006) [2023-12-26 20:07:14,981][105692] Updated weights for policy 0, policy_version 652928 (0.0007) [2023-12-26 20:07:15,401][105620] Updated weights for policy 1, policy_version 653790 (0.0008) [2023-12-26 20:07:15,453][105620] Updated weights for policy 1, policy_version 653800 (0.0008) [2023-12-26 20:07:15,503][105620] Updated weights for policy 1, policy_version 653810 (0.0005) [2023-12-26 20:07:15,609][105692] Updated weights for policy 0, policy_version 652938 (0.0006) [2023-12-26 20:07:15,669][105692] Updated weights for policy 0, policy_version 652948 (0.0010) [2023-12-26 20:07:15,733][105692] Updated weights for policy 0, policy_version 652958 (0.0009) [2023-12-26 20:07:15,793][105692] Updated weights for policy 0, policy_version 652968 (0.0009) [2023-12-26 20:07:16,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 334585856. Throughput: 0: 9612.8, 1: 10046.4. Samples: 334554720. Policy #0 lag: (min: 8.0, avg: 36.0, max: 40.0) [2023-12-26 20:07:16,062][104569] Avg episode reward: [(0, '9354.879'), (1, '8713.202')] [2023-12-26 20:07:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000652968_167190528.pth... [2023-12-26 20:07:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000653816_167395328.pth... [2023-12-26 20:07:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000651880_166912000.pth [2023-12-26 20:07:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000652632_167092224.pth [2023-12-26 20:07:16,215][105620] Updated weights for policy 1, policy_version 653820 (0.0008) [2023-12-26 20:07:16,270][105620] Updated weights for policy 1, policy_version 653830 (0.0010) [2023-12-26 20:07:16,335][105620] Updated weights for policy 1, policy_version 653840 (0.0010) [2023-12-26 20:07:16,575][105692] Updated weights for policy 0, policy_version 652978 (0.0008) [2023-12-26 20:07:16,620][105692] Updated weights for policy 0, policy_version 652988 (0.0008) [2023-12-26 20:07:16,672][105692] Updated weights for policy 0, policy_version 652998 (0.0008) [2023-12-26 20:07:16,999][105620] Updated weights for policy 1, policy_version 653850 (0.0009) [2023-12-26 20:07:17,051][105620] Updated weights for policy 1, policy_version 653860 (0.0007) [2023-12-26 20:07:17,106][105620] Updated weights for policy 1, policy_version 653870 (0.0006) [2023-12-26 20:07:17,165][105620] Updated weights for policy 1, policy_version 653880 (0.0005) [2023-12-26 20:07:17,579][105692] Updated weights for policy 0, policy_version 653008 (0.0008) [2023-12-26 20:07:17,631][105692] Updated weights for policy 0, policy_version 653018 (0.0008) [2023-12-26 20:07:17,689][105692] Updated weights for policy 0, policy_version 653028 (0.0008) [2023-12-26 20:07:17,730][105620] Updated weights for policy 1, policy_version 653890 (0.0008) [2023-12-26 20:07:17,793][105620] Updated weights for policy 1, policy_version 653900 (0.0009) [2023-12-26 20:07:17,854][105620] Updated weights for policy 1, policy_version 653910 (0.0010) [2023-12-26 20:07:18,451][105692] Updated weights for policy 0, policy_version 653038 (0.0010) [2023-12-26 20:07:18,503][105692] Updated weights for policy 0, policy_version 653048 (0.0009) [2023-12-26 20:07:18,535][105620] Updated weights for policy 1, policy_version 653920 (0.0007) [2023-12-26 20:07:18,553][105692] Updated weights for policy 0, policy_version 653058 (0.0007) [2023-12-26 20:07:18,591][105620] Updated weights for policy 1, policy_version 653930 (0.0008) [2023-12-26 20:07:18,643][105620] Updated weights for policy 1, policy_version 653940 (0.0009) [2023-12-26 20:07:19,390][105620] Updated weights for policy 1, policy_version 653950 (0.0008) [2023-12-26 20:07:19,404][105692] Updated weights for policy 0, policy_version 653068 (0.0008) [2023-12-26 20:07:19,421][105586] KL-divergence is very high: 144.0593 [2023-12-26 20:07:19,427][105586] KL-divergence is very high: 174.5316 [2023-12-26 20:07:19,450][105620] Updated weights for policy 1, policy_version 653960 (0.0007) [2023-12-26 20:07:19,465][105692] Updated weights for policy 0, policy_version 653078 (0.0007) [2023-12-26 20:07:19,516][105620] Updated weights for policy 1, policy_version 653970 (0.0007) [2023-12-26 20:07:19,528][105692] Updated weights for policy 0, policy_version 653088 (0.0008) [2023-12-26 20:07:20,235][105692] Updated weights for policy 0, policy_version 653098 (0.0006) [2023-12-26 20:07:20,286][105692] Updated weights for policy 0, policy_version 653108 (0.0007) [2023-12-26 20:07:20,297][105620] Updated weights for policy 1, policy_version 653980 (0.0008) [2023-12-26 20:07:20,351][105692] Updated weights for policy 0, policy_version 653118 (0.0011) [2023-12-26 20:07:20,351][105620] Updated weights for policy 1, policy_version 653990 (0.0008) [2023-12-26 20:07:20,410][105620] Updated weights for policy 1, policy_version 654000 (0.0007) [2023-12-26 20:07:20,411][105692] Updated weights for policy 0, policy_version 653128 (0.0011) [2023-12-26 20:07:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 334675968. Throughput: 0: 9476.7, 1: 10084.0. Samples: 334669480. Policy #0 lag: (min: 8.0, avg: 36.0, max: 40.0) [2023-12-26 20:07:21,063][104569] Avg episode reward: [(0, '3000.955'), (1, '8529.758')] [2023-12-26 20:07:21,113][105692] Updated weights for policy 0, policy_version 653138 (0.0011) [2023-12-26 20:07:21,176][105692] Updated weights for policy 0, policy_version 653148 (0.0010) [2023-12-26 20:07:21,232][105620] Updated weights for policy 1, policy_version 654010 (0.0008) [2023-12-26 20:07:21,241][105692] Updated weights for policy 0, policy_version 653158 (0.0010) [2023-12-26 20:07:21,285][105620] Updated weights for policy 1, policy_version 654020 (0.0008) [2023-12-26 20:07:21,340][105620] Updated weights for policy 1, policy_version 654030 (0.0008) [2023-12-26 20:07:21,407][105620] Updated weights for policy 1, policy_version 654040 (0.0009) [2023-12-26 20:07:21,942][105692] Updated weights for policy 0, policy_version 653168 (0.0011) [2023-12-26 20:07:21,942][105585] KL-divergence is very high: 132.9596 [2023-12-26 20:07:21,949][105585] KL-divergence is very high: 129.2927 [2023-12-26 20:07:21,993][105585] KL-divergence is very high: 117.0538 [2023-12-26 20:07:22,005][105692] Updated weights for policy 0, policy_version 653178 (0.0010) [2023-12-26 20:07:22,072][105620] Updated weights for policy 1, policy_version 654050 (0.0006) [2023-12-26 20:07:22,073][105692] Updated weights for policy 0, policy_version 653188 (0.0011) [2023-12-26 20:07:22,133][105620] Updated weights for policy 1, policy_version 654060 (0.0006) [2023-12-26 20:07:22,200][105620] Updated weights for policy 1, policy_version 654070 (0.0006) [2023-12-26 20:07:22,810][105692] Updated weights for policy 0, policy_version 653198 (0.0010) [2023-12-26 20:07:22,838][105620] Updated weights for policy 1, policy_version 654080 (0.0007) [2023-12-26 20:07:22,866][105692] Updated weights for policy 0, policy_version 653208 (0.0007) [2023-12-26 20:07:22,902][105620] Updated weights for policy 1, policy_version 654090 (0.0007) [2023-12-26 20:07:22,928][105692] Updated weights for policy 0, policy_version 653218 (0.0007) [2023-12-26 20:07:22,943][105585] KL-divergence is very high: 110.6012 [2023-12-26 20:07:22,960][105620] Updated weights for policy 1, policy_version 654100 (0.0008) [2023-12-26 20:07:23,540][105692] Updated weights for policy 0, policy_version 653228 (0.0006) [2023-12-26 20:07:23,596][105692] Updated weights for policy 0, policy_version 653238 (0.0005) [2023-12-26 20:07:23,652][105692] Updated weights for policy 0, policy_version 653248 (0.0005) [2023-12-26 20:07:23,732][105620] Updated weights for policy 1, policy_version 654110 (0.0009) [2023-12-26 20:07:23,794][105620] Updated weights for policy 1, policy_version 654120 (0.0009) [2023-12-26 20:07:23,851][105620] Updated weights for policy 1, policy_version 654130 (0.0009) [2023-12-26 20:07:24,333][105692] Updated weights for policy 0, policy_version 653258 (0.0007) [2023-12-26 20:07:24,387][105692] Updated weights for policy 0, policy_version 653268 (0.0006) [2023-12-26 20:07:24,393][105585] KL-divergence is very high: 199.9616 [2023-12-26 20:07:24,443][105585] KL-divergence is very high: 169.2729 [2023-12-26 20:07:24,448][105692] Updated weights for policy 0, policy_version 653278 (0.0005) [2023-12-26 20:07:24,500][105620] Updated weights for policy 1, policy_version 654140 (0.0008) [2023-12-26 20:07:24,511][105692] Updated weights for policy 0, policy_version 653288 (0.0007) [2023-12-26 20:07:24,557][105620] Updated weights for policy 1, policy_version 654150 (0.0006) [2023-12-26 20:07:24,606][105620] Updated weights for policy 1, policy_version 654160 (0.0005) [2023-12-26 20:07:25,207][105692] Updated weights for policy 0, policy_version 653298 (0.0009) [2023-12-26 20:07:25,242][105620] Updated weights for policy 1, policy_version 654170 (0.0006) [2023-12-26 20:07:25,268][105692] Updated weights for policy 0, policy_version 653308 (0.0008) [2023-12-26 20:07:25,298][105620] Updated weights for policy 1, policy_version 654180 (0.0007) [2023-12-26 20:07:25,321][105692] Updated weights for policy 0, policy_version 653318 (0.0006) [2023-12-26 20:07:25,352][105620] Updated weights for policy 1, policy_version 654190 (0.0009) [2023-12-26 20:07:25,410][105620] Updated weights for policy 1, policy_version 654200 (0.0009) [2023-12-26 20:07:26,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 334774272. Throughput: 0: 9404.3, 1: 10095.3. Samples: 334788116. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 20:07:26,063][104569] Avg episode reward: [(0, '4975.378'), (1, '8895.692')] [2023-12-26 20:07:26,066][105620] Updated weights for policy 1, policy_version 654210 (0.0005) [2023-12-26 20:07:26,115][105620] Updated weights for policy 1, policy_version 654220 (0.0006) [2023-12-26 20:07:26,127][105692] Updated weights for policy 0, policy_version 653328 (0.0008) [2023-12-26 20:07:26,166][105620] Updated weights for policy 1, policy_version 654230 (0.0008) [2023-12-26 20:07:26,173][105585] KL-divergence is very high: 175.1241 [2023-12-26 20:07:26,185][105692] Updated weights for policy 0, policy_version 653338 (0.0007) [2023-12-26 20:07:26,243][105692] Updated weights for policy 0, policy_version 653348 (0.0009) [2023-12-26 20:07:26,868][105620] Updated weights for policy 1, policy_version 654240 (0.0008) [2023-12-26 20:07:26,915][105620] Updated weights for policy 1, policy_version 654250 (0.0009) [2023-12-26 20:07:26,963][105620] Updated weights for policy 1, policy_version 654260 (0.0007) [2023-12-26 20:07:26,977][105692] Updated weights for policy 0, policy_version 653358 (0.0008) [2023-12-26 20:07:27,022][105692] Updated weights for policy 0, policy_version 653368 (0.0008) [2023-12-26 20:07:27,072][105692] Updated weights for policy 0, policy_version 653378 (0.0009) [2023-12-26 20:07:27,744][105620] Updated weights for policy 1, policy_version 654270 (0.0008) [2023-12-26 20:07:27,789][105692] Updated weights for policy 0, policy_version 653388 (0.0008) [2023-12-26 20:07:27,795][105620] Updated weights for policy 1, policy_version 654280 (0.0008) [2023-12-26 20:07:27,845][105692] Updated weights for policy 0, policy_version 653398 (0.0007) [2023-12-26 20:07:27,854][105620] Updated weights for policy 1, policy_version 654290 (0.0006) [2023-12-26 20:07:27,890][105692] Updated weights for policy 0, policy_version 653408 (0.0006) [2023-12-26 20:07:28,607][105620] Updated weights for policy 1, policy_version 654300 (0.0008) [2023-12-26 20:07:28,654][105692] Updated weights for policy 0, policy_version 653418 (0.0009) [2023-12-26 20:07:28,666][105620] Updated weights for policy 1, policy_version 654310 (0.0008) [2023-12-26 20:07:28,715][105692] Updated weights for policy 0, policy_version 653428 (0.0008) [2023-12-26 20:07:28,729][105620] Updated weights for policy 1, policy_version 654320 (0.0009) [2023-12-26 20:07:28,773][105692] Updated weights for policy 0, policy_version 653438 (0.0007) [2023-12-26 20:07:28,827][105692] Updated weights for policy 0, policy_version 653448 (0.0009) [2023-12-26 20:07:29,511][105620] Updated weights for policy 1, policy_version 654330 (0.0008) [2023-12-26 20:07:29,543][105692] Updated weights for policy 0, policy_version 653458 (0.0008) [2023-12-26 20:07:29,570][105620] Updated weights for policy 1, policy_version 654340 (0.0008) [2023-12-26 20:07:29,591][105692] Updated weights for policy 0, policy_version 653468 (0.0005) [2023-12-26 20:07:29,617][105620] Updated weights for policy 1, policy_version 654350 (0.0009) [2023-12-26 20:07:29,648][105692] Updated weights for policy 0, policy_version 653478 (0.0005) [2023-12-26 20:07:29,686][105620] Updated weights for policy 1, policy_version 654360 (0.0008) [2023-12-26 20:07:30,351][105692] Updated weights for policy 0, policy_version 653488 (0.0008) [2023-12-26 20:07:30,409][105692] Updated weights for policy 0, policy_version 653498 (0.0008) [2023-12-26 20:07:30,457][105692] Updated weights for policy 0, policy_version 653508 (0.0008) [2023-12-26 20:07:30,476][105620] Updated weights for policy 1, policy_version 654370 (0.0007) [2023-12-26 20:07:30,523][105620] Updated weights for policy 1, policy_version 654380 (0.0008) [2023-12-26 20:07:30,570][105620] Updated weights for policy 1, policy_version 654390 (0.0009) [2023-12-26 20:07:31,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 334872576. Throughput: 0: 9405.2, 1: 10091.2. Samples: 334845404. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 20:07:31,062][104569] Avg episode reward: [(0, '7121.370'), (1, '8896.228')] [2023-12-26 20:07:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000654392_167542784.pth... [2023-12-26 20:07:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000653512_167329792.pth... [2023-12-26 20:07:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000653240_167247872.pth [2023-12-26 20:07:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000652424_167051264.pth [2023-12-26 20:07:31,189][105692] Updated weights for policy 0, policy_version 653518 (0.0006) [2023-12-26 20:07:31,246][105692] Updated weights for policy 0, policy_version 653528 (0.0006) [2023-12-26 20:07:31,303][105692] Updated weights for policy 0, policy_version 653538 (0.0007) [2023-12-26 20:07:31,345][105620] Updated weights for policy 1, policy_version 654400 (0.0009) [2023-12-26 20:07:31,417][105620] Updated weights for policy 1, policy_version 654410 (0.0009) [2023-12-26 20:07:31,479][105620] Updated weights for policy 1, policy_version 654420 (0.0008) [2023-12-26 20:07:31,936][105692] Updated weights for policy 0, policy_version 653548 (0.0007) [2023-12-26 20:07:31,993][105692] Updated weights for policy 0, policy_version 653558 (0.0008) [2023-12-26 20:07:32,046][105692] Updated weights for policy 0, policy_version 653568 (0.0010) [2023-12-26 20:07:32,315][105620] Updated weights for policy 1, policy_version 654430 (0.0009) [2023-12-26 20:07:32,375][105620] Updated weights for policy 1, policy_version 654440 (0.0009) [2023-12-26 20:07:32,432][105620] Updated weights for policy 1, policy_version 654450 (0.0009) [2023-12-26 20:07:32,677][105692] Updated weights for policy 0, policy_version 653578 (0.0009) [2023-12-26 20:07:32,737][105692] Updated weights for policy 0, policy_version 653588 (0.0009) [2023-12-26 20:07:32,797][105692] Updated weights for policy 0, policy_version 653598 (0.0010) [2023-12-26 20:07:32,854][105692] Updated weights for policy 0, policy_version 653608 (0.0010) [2023-12-26 20:07:33,170][105620] Updated weights for policy 1, policy_version 654460 (0.0010) [2023-12-26 20:07:33,223][105620] Updated weights for policy 1, policy_version 654471 (0.0009) [2023-12-26 20:07:33,285][105620] Updated weights for policy 1, policy_version 654481 (0.0009) [2023-12-26 20:07:33,443][105692] Updated weights for policy 0, policy_version 653618 (0.0005) [2023-12-26 20:07:33,496][105692] Updated weights for policy 0, policy_version 653628 (0.0007) [2023-12-26 20:07:33,547][105692] Updated weights for policy 0, policy_version 653638 (0.0010) [2023-12-26 20:07:34,078][105620] Updated weights for policy 1, policy_version 654491 (0.0009) [2023-12-26 20:07:34,133][105620] Updated weights for policy 1, policy_version 654501 (0.0008) [2023-12-26 20:07:34,200][105620] Updated weights for policy 1, policy_version 654511 (0.0008) [2023-12-26 20:07:34,268][105692] Updated weights for policy 0, policy_version 653648 (0.0010) [2023-12-26 20:07:34,327][105692] Updated weights for policy 0, policy_version 653658 (0.0011) [2023-12-26 20:07:34,383][105692] Updated weights for policy 0, policy_version 653668 (0.0010) [2023-12-26 20:07:34,890][105620] Updated weights for policy 1, policy_version 654521 (0.0007) [2023-12-26 20:07:34,950][105620] Updated weights for policy 1, policy_version 654531 (0.0008) [2023-12-26 20:07:35,014][105620] Updated weights for policy 1, policy_version 654541 (0.0008) [2023-12-26 20:07:35,082][105620] Updated weights for policy 1, policy_version 654551 (0.0008) [2023-12-26 20:07:35,140][105692] Updated weights for policy 0, policy_version 653678 (0.0011) [2023-12-26 20:07:35,202][105692] Updated weights for policy 0, policy_version 653688 (0.0011) [2023-12-26 20:07:35,253][105692] Updated weights for policy 0, policy_version 653698 (0.0010) [2023-12-26 20:07:35,817][105620] Updated weights for policy 1, policy_version 654561 (0.0008) [2023-12-26 20:07:35,870][105620] Updated weights for policy 1, policy_version 654571 (0.0008) [2023-12-26 20:07:35,922][105620] Updated weights for policy 1, policy_version 654581 (0.0008) [2023-12-26 20:07:35,968][105692] Updated weights for policy 0, policy_version 653708 (0.0008) [2023-12-26 20:07:36,018][105692] Updated weights for policy 0, policy_version 653718 (0.0005) [2023-12-26 20:07:36,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19387.8, 300 sec: 19521.9). Total num frames: 334970880. Throughput: 0: 9448.8, 1: 9982.6. Samples: 334961472. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 20:07:36,062][104569] Avg episode reward: [(0, '7982.141'), (1, '8895.956')] [2023-12-26 20:07:36,066][105692] Updated weights for policy 0, policy_version 653728 (0.0011) [2023-12-26 20:07:36,718][105620] Updated weights for policy 1, policy_version 654591 (0.0009) [2023-12-26 20:07:36,775][105620] Updated weights for policy 1, policy_version 654601 (0.0008) [2023-12-26 20:07:36,820][105692] Updated weights for policy 0, policy_version 653738 (0.0011) [2023-12-26 20:07:36,830][105620] Updated weights for policy 1, policy_version 654611 (0.0007) [2023-12-26 20:07:36,870][105692] Updated weights for policy 0, policy_version 653748 (0.0011) [2023-12-26 20:07:36,926][105692] Updated weights for policy 0, policy_version 653758 (0.0010) [2023-12-26 20:07:36,995][105692] Updated weights for policy 0, policy_version 653768 (0.0008) [2023-12-26 20:07:37,566][105620] Updated weights for policy 1, policy_version 654621 (0.0006) [2023-12-26 20:07:37,622][105620] Updated weights for policy 1, policy_version 654631 (0.0008) [2023-12-26 20:07:37,676][105620] Updated weights for policy 1, policy_version 654641 (0.0008) [2023-12-26 20:07:37,716][105692] Updated weights for policy 0, policy_version 653778 (0.0011) [2023-12-26 20:07:37,774][105692] Updated weights for policy 0, policy_version 653788 (0.0010) [2023-12-26 20:07:37,836][105692] Updated weights for policy 0, policy_version 653798 (0.0010) [2023-12-26 20:07:38,468][105620] Updated weights for policy 1, policy_version 654651 (0.0007) [2023-12-26 20:07:38,536][105620] Updated weights for policy 1, policy_version 654661 (0.0010) [2023-12-26 20:07:38,549][105692] Updated weights for policy 0, policy_version 653808 (0.0007) [2023-12-26 20:07:38,596][105620] Updated weights for policy 1, policy_version 654671 (0.0009) [2023-12-26 20:07:38,597][105692] Updated weights for policy 0, policy_version 653818 (0.0005) [2023-12-26 20:07:38,646][105692] Updated weights for policy 0, policy_version 653828 (0.0006) [2023-12-26 20:07:39,282][105692] Updated weights for policy 0, policy_version 653838 (0.0008) [2023-12-26 20:07:39,340][105692] Updated weights for policy 0, policy_version 653848 (0.0008) [2023-12-26 20:07:39,406][105692] Updated weights for policy 0, policy_version 653858 (0.0008) [2023-12-26 20:07:39,407][105620] Updated weights for policy 1, policy_version 654681 (0.0009) [2023-12-26 20:07:39,467][105620] Updated weights for policy 1, policy_version 654691 (0.0009) [2023-12-26 20:07:39,529][105620] Updated weights for policy 1, policy_version 654701 (0.0009) [2023-12-26 20:07:39,592][105620] Updated weights for policy 1, policy_version 654711 (0.0009) [2023-12-26 20:07:40,052][105692] Updated weights for policy 0, policy_version 653868 (0.0008) [2023-12-26 20:07:40,114][105692] Updated weights for policy 0, policy_version 653878 (0.0009) [2023-12-26 20:07:40,178][105692] Updated weights for policy 0, policy_version 653888 (0.0009) [2023-12-26 20:07:40,393][105620] Updated weights for policy 1, policy_version 654721 (0.0009) [2023-12-26 20:07:40,446][105620] Updated weights for policy 1, policy_version 654731 (0.0009) [2023-12-26 20:07:40,502][105620] Updated weights for policy 1, policy_version 654741 (0.0006) [2023-12-26 20:07:40,989][105692] Updated weights for policy 0, policy_version 653898 (0.0009) [2023-12-26 20:07:41,061][105692] Updated weights for policy 0, policy_version 653908 (0.0009) [2023-12-26 20:07:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 335060992. Throughput: 0: 9509.5, 1: 9877.6. Samples: 335074560. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 20:07:41,062][104569] Avg episode reward: [(0, '8140.218'), (1, '8987.665')] [2023-12-26 20:07:41,123][105692] Updated weights for policy 0, policy_version 653918 (0.0009) [2023-12-26 20:07:41,188][105692] Updated weights for policy 0, policy_version 653928 (0.0008) [2023-12-26 20:07:41,256][105620] Updated weights for policy 1, policy_version 654751 (0.0007) [2023-12-26 20:07:41,321][105620] Updated weights for policy 1, policy_version 654761 (0.0008) [2023-12-26 20:07:41,387][105620] Updated weights for policy 1, policy_version 654771 (0.0009) [2023-12-26 20:07:41,928][105692] Updated weights for policy 0, policy_version 653938 (0.0009) [2023-12-26 20:07:41,987][105692] Updated weights for policy 0, policy_version 653948 (0.0009) [2023-12-26 20:07:42,049][105692] Updated weights for policy 0, policy_version 653958 (0.0009) [2023-12-26 20:07:42,164][105620] Updated weights for policy 1, policy_version 654781 (0.0008) [2023-12-26 20:07:42,230][105620] Updated weights for policy 1, policy_version 654791 (0.0009) [2023-12-26 20:07:42,296][105620] Updated weights for policy 1, policy_version 654801 (0.0009) [2023-12-26 20:07:42,801][105692] Updated weights for policy 0, policy_version 653968 (0.0007) [2023-12-26 20:07:42,860][105692] Updated weights for policy 0, policy_version 653978 (0.0007) [2023-12-26 20:07:42,925][105692] Updated weights for policy 0, policy_version 653988 (0.0005) [2023-12-26 20:07:43,117][105620] Updated weights for policy 1, policy_version 654811 (0.0009) [2023-12-26 20:07:43,171][105620] Updated weights for policy 1, policy_version 654821 (0.0010) [2023-12-26 20:07:43,234][105620] Updated weights for policy 1, policy_version 654831 (0.0010) [2023-12-26 20:07:43,474][105692] Updated weights for policy 0, policy_version 653998 (0.0008) [2023-12-26 20:07:43,534][105692] Updated weights for policy 0, policy_version 654008 (0.0006) [2023-12-26 20:07:43,591][105692] Updated weights for policy 0, policy_version 654018 (0.0005) [2023-12-26 20:07:44,030][105620] Updated weights for policy 1, policy_version 654841 (0.0010) [2023-12-26 20:07:44,095][105620] Updated weights for policy 1, policy_version 654851 (0.0006) [2023-12-26 20:07:44,131][105692] Updated weights for policy 0, policy_version 654028 (0.0005) [2023-12-26 20:07:44,162][105620] Updated weights for policy 1, policy_version 654861 (0.0006) [2023-12-26 20:07:44,193][105692] Updated weights for policy 0, policy_version 654038 (0.0006) [2023-12-26 20:07:44,231][105620] Updated weights for policy 1, policy_version 654871 (0.0006) [2023-12-26 20:07:44,259][105692] Updated weights for policy 0, policy_version 654048 (0.0007) [2023-12-26 20:07:44,797][105620] Updated weights for policy 1, policy_version 654881 (0.0007) [2023-12-26 20:07:44,856][105620] Updated weights for policy 1, policy_version 654891 (0.0006) [2023-12-26 20:07:44,890][105692] Updated weights for policy 0, policy_version 654058 (0.0006) [2023-12-26 20:07:44,905][105620] Updated weights for policy 1, policy_version 654901 (0.0005) [2023-12-26 20:07:44,937][105692] Updated weights for policy 0, policy_version 654068 (0.0008) [2023-12-26 20:07:44,990][105692] Updated weights for policy 0, policy_version 654078 (0.0008) [2023-12-26 20:07:45,586][105620] Updated weights for policy 1, policy_version 654911 (0.0006) [2023-12-26 20:07:45,645][105620] Updated weights for policy 1, policy_version 654921 (0.0007) [2023-12-26 20:07:45,700][105620] Updated weights for policy 1, policy_version 654931 (0.0009) [2023-12-26 20:07:45,866][105692] Updated weights for policy 0, policy_version 654089 (0.0008) [2023-12-26 20:07:45,935][105692] Updated weights for policy 0, policy_version 654099 (0.0009) [2023-12-26 20:07:46,002][105692] Updated weights for policy 0, policy_version 654109 (0.0010) [2023-12-26 20:07:46,060][105692] Updated weights for policy 0, policy_version 654119 (0.0009) [2023-12-26 20:07:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 335159296. Throughput: 0: 9597.3, 1: 9726.8. Samples: 335130320. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 20:07:46,062][104569] Avg episode reward: [(0, '9088.157'), (1, '9171.450')] [2023-12-26 20:07:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000654120_167485440.pth... [2023-12-26 20:07:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000654936_167682048.pth... [2023-12-26 20:07:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000653816_167395328.pth [2023-12-26 20:07:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000652968_167190528.pth [2023-12-26 20:07:46,306][105620] Updated weights for policy 1, policy_version 654941 (0.0007) [2023-12-26 20:07:46,352][105620] Updated weights for policy 1, policy_version 654951 (0.0005) [2023-12-26 20:07:46,398][105620] Updated weights for policy 1, policy_version 654961 (0.0005) [2023-12-26 20:07:46,859][105692] Updated weights for policy 0, policy_version 654129 (0.0009) [2023-12-26 20:07:46,905][105692] Updated weights for policy 0, policy_version 654139 (0.0009) [2023-12-26 20:07:46,963][105692] Updated weights for policy 0, policy_version 654149 (0.0010) [2023-12-26 20:07:47,065][105620] Updated weights for policy 1, policy_version 654971 (0.0007) [2023-12-26 20:07:47,125][105620] Updated weights for policy 1, policy_version 654981 (0.0007) [2023-12-26 20:07:47,181][105620] Updated weights for policy 1, policy_version 654991 (0.0010) [2023-12-26 20:07:47,758][105692] Updated weights for policy 0, policy_version 654159 (0.0009) [2023-12-26 20:07:47,812][105692] Updated weights for policy 0, policy_version 654170 (0.0009) [2023-12-26 20:07:47,824][105620] Updated weights for policy 1, policy_version 655001 (0.0010) [2023-12-26 20:07:47,858][105692] Updated weights for policy 0, policy_version 654180 (0.0007) [2023-12-26 20:07:47,882][105620] Updated weights for policy 1, policy_version 655011 (0.0010) [2023-12-26 20:07:47,946][105620] Updated weights for policy 1, policy_version 655021 (0.0010) [2023-12-26 20:07:48,014][105620] Updated weights for policy 1, policy_version 655031 (0.0010) [2023-12-26 20:07:48,660][105692] Updated weights for policy 0, policy_version 654190 (0.0009) [2023-12-26 20:07:48,678][105620] Updated weights for policy 1, policy_version 655041 (0.0006) [2023-12-26 20:07:48,723][105692] Updated weights for policy 0, policy_version 654200 (0.0009) [2023-12-26 20:07:48,736][105620] Updated weights for policy 1, policy_version 655051 (0.0007) [2023-12-26 20:07:48,778][105692] Updated weights for policy 0, policy_version 654210 (0.0008) [2023-12-26 20:07:48,792][105620] Updated weights for policy 1, policy_version 655061 (0.0005) [2023-12-26 20:07:49,476][105620] Updated weights for policy 1, policy_version 655071 (0.0008) [2023-12-26 20:07:49,532][105620] Updated weights for policy 1, policy_version 655081 (0.0009) [2023-12-26 20:07:49,563][105692] Updated weights for policy 0, policy_version 654220 (0.0007) [2023-12-26 20:07:49,592][105620] Updated weights for policy 1, policy_version 655091 (0.0008) [2023-12-26 20:07:49,616][105692] Updated weights for policy 0, policy_version 654230 (0.0006) [2023-12-26 20:07:49,673][105692] Updated weights for policy 0, policy_version 654240 (0.0005) [2023-12-26 20:07:50,322][105692] Updated weights for policy 0, policy_version 654250 (0.0006) [2023-12-26 20:07:50,386][105692] Updated weights for policy 0, policy_version 654260 (0.0006) [2023-12-26 20:07:50,435][105620] Updated weights for policy 1, policy_version 655101 (0.0009) [2023-12-26 20:07:50,446][105692] Updated weights for policy 0, policy_version 654270 (0.0005) [2023-12-26 20:07:50,496][105620] Updated weights for policy 1, policy_version 655111 (0.0009) [2023-12-26 20:07:50,500][105692] Updated weights for policy 0, policy_version 654280 (0.0005) [2023-12-26 20:07:50,550][105620] Updated weights for policy 1, policy_version 655121 (0.0010) [2023-12-26 20:07:51,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 335257600. Throughput: 0: 9611.3, 1: 9765.0. Samples: 335249148. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 20:07:51,062][104569] Avg episode reward: [(0, '9172.504'), (1, '9080.628')] [2023-12-26 20:07:51,200][105692] Updated weights for policy 0, policy_version 654290 (0.0005) [2023-12-26 20:07:51,260][105692] Updated weights for policy 0, policy_version 654300 (0.0005) [2023-12-26 20:07:51,321][105692] Updated weights for policy 0, policy_version 654310 (0.0006) [2023-12-26 20:07:51,327][105620] Updated weights for policy 1, policy_version 655131 (0.0010) [2023-12-26 20:07:51,396][105620] Updated weights for policy 1, policy_version 655141 (0.0008) [2023-12-26 20:07:51,466][105620] Updated weights for policy 1, policy_version 655151 (0.0009) [2023-12-26 20:07:51,975][105692] Updated weights for policy 0, policy_version 654320 (0.0009) [2023-12-26 20:07:52,040][105692] Updated weights for policy 0, policy_version 654330 (0.0010) [2023-12-26 20:07:52,112][105692] Updated weights for policy 0, policy_version 654340 (0.0010) [2023-12-26 20:07:52,154][105620] Updated weights for policy 1, policy_version 655161 (0.0009) [2023-12-26 20:07:52,221][105620] Updated weights for policy 1, policy_version 655171 (0.0007) [2023-12-26 20:07:52,285][105620] Updated weights for policy 1, policy_version 655181 (0.0008) [2023-12-26 20:07:52,336][105620] Updated weights for policy 1, policy_version 655191 (0.0008) [2023-12-26 20:07:52,918][105692] Updated weights for policy 0, policy_version 654350 (0.0008) [2023-12-26 20:07:52,973][105692] Updated weights for policy 0, policy_version 654360 (0.0009) [2023-12-26 20:07:53,032][105692] Updated weights for policy 0, policy_version 654370 (0.0008) [2023-12-26 20:07:53,054][105620] Updated weights for policy 1, policy_version 655201 (0.0007) [2023-12-26 20:07:53,114][105620] Updated weights for policy 1, policy_version 655211 (0.0009) [2023-12-26 20:07:53,172][105620] Updated weights for policy 1, policy_version 655222 (0.0010) [2023-12-26 20:07:53,729][105692] Updated weights for policy 0, policy_version 654380 (0.0007) [2023-12-26 20:07:53,792][105692] Updated weights for policy 0, policy_version 654390 (0.0009) [2023-12-26 20:07:53,832][105620] Updated weights for policy 1, policy_version 655232 (0.0007) [2023-12-26 20:07:53,842][105692] Updated weights for policy 0, policy_version 654400 (0.0007) [2023-12-26 20:07:53,899][105620] Updated weights for policy 1, policy_version 655242 (0.0009) [2023-12-26 20:07:53,962][105620] Updated weights for policy 1, policy_version 655252 (0.0010) [2023-12-26 20:07:54,438][105692] Updated weights for policy 0, policy_version 654410 (0.0006) [2023-12-26 20:07:54,495][105692] Updated weights for policy 0, policy_version 654420 (0.0009) [2023-12-26 20:07:54,546][105692] Updated weights for policy 0, policy_version 654430 (0.0009) [2023-12-26 20:07:54,598][105692] Updated weights for policy 0, policy_version 654440 (0.0008) [2023-12-26 20:07:54,737][105620] Updated weights for policy 1, policy_version 655262 (0.0009) [2023-12-26 20:07:54,791][105620] Updated weights for policy 1, policy_version 655272 (0.0009) [2023-12-26 20:07:54,852][105620] Updated weights for policy 1, policy_version 655282 (0.0009) [2023-12-26 20:07:55,370][105692] Updated weights for policy 0, policy_version 654450 (0.0009) [2023-12-26 20:07:55,424][105692] Updated weights for policy 0, policy_version 654461 (0.0010) [2023-12-26 20:07:55,477][105692] Updated weights for policy 0, policy_version 654471 (0.0009) [2023-12-26 20:07:55,532][105620] Updated weights for policy 1, policy_version 655292 (0.0008) [2023-12-26 20:07:55,593][105620] Updated weights for policy 1, policy_version 655302 (0.0006) [2023-12-26 20:07:55,658][105620] Updated weights for policy 1, policy_version 655312 (0.0005) [2023-12-26 20:07:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 335355904. Throughput: 0: 9657.1, 1: 9633.1. Samples: 335365792. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 20:07:56,063][104569] Avg episode reward: [(0, '9263.146'), (1, '8989.410')] [2023-12-26 20:07:56,248][105692] Updated weights for policy 0, policy_version 654481 (0.0006) [2023-12-26 20:07:56,282][105620] Updated weights for policy 1, policy_version 655322 (0.0007) [2023-12-26 20:07:56,311][105692] Updated weights for policy 0, policy_version 654491 (0.0005) [2023-12-26 20:07:56,342][105620] Updated weights for policy 1, policy_version 655332 (0.0009) [2023-12-26 20:07:56,370][105692] Updated weights for policy 0, policy_version 654501 (0.0005) [2023-12-26 20:07:56,393][105620] Updated weights for policy 1, policy_version 655343 (0.0009) [2023-12-26 20:07:56,917][105692] Updated weights for policy 0, policy_version 654511 (0.0009) [2023-12-26 20:07:56,960][105692] Updated weights for policy 0, policy_version 654521 (0.0010) [2023-12-26 20:07:57,003][105692] Updated weights for policy 0, policy_version 654531 (0.0010) [2023-12-26 20:07:57,220][105620] Updated weights for policy 1, policy_version 655353 (0.0009) [2023-12-26 20:07:57,272][105620] Updated weights for policy 1, policy_version 655363 (0.0008) [2023-12-26 20:07:57,331][105620] Updated weights for policy 1, policy_version 655373 (0.0007) [2023-12-26 20:07:57,379][105620] Updated weights for policy 1, policy_version 655383 (0.0008) [2023-12-26 20:07:57,685][105692] Updated weights for policy 0, policy_version 654541 (0.0010) [2023-12-26 20:07:57,739][105692] Updated weights for policy 0, policy_version 654551 (0.0006) [2023-12-26 20:07:57,788][105692] Updated weights for policy 0, policy_version 654561 (0.0009) [2023-12-26 20:07:58,193][105620] Updated weights for policy 1, policy_version 655393 (0.0008) [2023-12-26 20:07:58,252][105620] Updated weights for policy 1, policy_version 655403 (0.0008) [2023-12-26 20:07:58,313][105620] Updated weights for policy 1, policy_version 655413 (0.0008) [2023-12-26 20:07:58,465][105692] Updated weights for policy 0, policy_version 654571 (0.0009) [2023-12-26 20:07:58,533][105692] Updated weights for policy 0, policy_version 654581 (0.0009) [2023-12-26 20:07:58,590][105692] Updated weights for policy 0, policy_version 654591 (0.0009) [2023-12-26 20:07:59,037][105620] Updated weights for policy 1, policy_version 655423 (0.0008) [2023-12-26 20:07:59,085][105620] Updated weights for policy 1, policy_version 655433 (0.0007) [2023-12-26 20:07:59,136][105620] Updated weights for policy 1, policy_version 655443 (0.0007) [2023-12-26 20:07:59,404][105692] Updated weights for policy 0, policy_version 654601 (0.0009) [2023-12-26 20:07:59,454][105692] Updated weights for policy 0, policy_version 654611 (0.0009) [2023-12-26 20:07:59,508][105692] Updated weights for policy 0, policy_version 654621 (0.0010) [2023-12-26 20:07:59,566][105692] Updated weights for policy 0, policy_version 654631 (0.0009) [2023-12-26 20:07:59,773][105620] Updated weights for policy 1, policy_version 655453 (0.0006) [2023-12-26 20:07:59,825][105620] Updated weights for policy 1, policy_version 655463 (0.0006) [2023-12-26 20:07:59,890][105620] Updated weights for policy 1, policy_version 655473 (0.0006) [2023-12-26 20:08:00,386][105692] Updated weights for policy 0, policy_version 654641 (0.0009) [2023-12-26 20:08:00,445][105692] Updated weights for policy 0, policy_version 654651 (0.0009) [2023-12-26 20:08:00,494][105692] Updated weights for policy 0, policy_version 654661 (0.0009) [2023-12-26 20:08:00,531][105620] Updated weights for policy 1, policy_version 655483 (0.0007) [2023-12-26 20:08:00,602][105620] Updated weights for policy 1, policy_version 655493 (0.0007) [2023-12-26 20:08:00,658][105620] Updated weights for policy 1, policy_version 655503 (0.0006) [2023-12-26 20:08:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 335454208. Throughput: 0: 9748.0, 1: 9584.0. Samples: 335424660. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 20:08:01,062][104569] Avg episode reward: [(0, '9354.104'), (1, '9080.371')] [2023-12-26 20:08:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000654664_167624704.pth... [2023-12-26 20:08:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000655512_167829504.pth... [2023-12-26 20:08:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000654392_167542784.pth [2023-12-26 20:08:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000653512_167329792.pth [2023-12-26 20:08:01,228][105692] Updated weights for policy 0, policy_version 654671 (0.0009) [2023-12-26 20:08:01,292][105692] Updated weights for policy 0, policy_version 654681 (0.0009) [2023-12-26 20:08:01,341][105620] Updated weights for policy 1, policy_version 655513 (0.0006) [2023-12-26 20:08:01,362][105692] Updated weights for policy 0, policy_version 654691 (0.0009) [2023-12-26 20:08:01,404][105620] Updated weights for policy 1, policy_version 655523 (0.0008) [2023-12-26 20:08:01,461][105620] Updated weights for policy 1, policy_version 655533 (0.0008) [2023-12-26 20:08:01,517][105620] Updated weights for policy 1, policy_version 655543 (0.0010) [2023-12-26 20:08:02,114][105692] Updated weights for policy 0, policy_version 654701 (0.0009) [2023-12-26 20:08:02,140][105620] Updated weights for policy 1, policy_version 655553 (0.0006) [2023-12-26 20:08:02,164][105692] Updated weights for policy 0, policy_version 654711 (0.0010) [2023-12-26 20:08:02,189][105620] Updated weights for policy 1, policy_version 655563 (0.0009) [2023-12-26 20:08:02,220][105692] Updated weights for policy 0, policy_version 654721 (0.0008) [2023-12-26 20:08:02,240][105620] Updated weights for policy 1, policy_version 655573 (0.0007) [2023-12-26 20:08:02,958][105692] Updated weights for policy 0, policy_version 654731 (0.0008) [2023-12-26 20:08:02,983][105620] Updated weights for policy 1, policy_version 655583 (0.0008) [2023-12-26 20:08:03,007][105692] Updated weights for policy 0, policy_version 654741 (0.0006) [2023-12-26 20:08:03,041][105620] Updated weights for policy 1, policy_version 655593 (0.0007) [2023-12-26 20:08:03,063][105692] Updated weights for policy 0, policy_version 654751 (0.0008) [2023-12-26 20:08:03,099][105620] Updated weights for policy 1, policy_version 655603 (0.0009) [2023-12-26 20:08:03,818][105692] Updated weights for policy 0, policy_version 654761 (0.0006) [2023-12-26 20:08:03,819][105620] Updated weights for policy 1, policy_version 655613 (0.0009) [2023-12-26 20:08:03,877][105620] Updated weights for policy 1, policy_version 655623 (0.0008) [2023-12-26 20:08:03,877][105692] Updated weights for policy 0, policy_version 654771 (0.0007) [2023-12-26 20:08:03,924][105620] Updated weights for policy 1, policy_version 655633 (0.0007) [2023-12-26 20:08:03,942][105692] Updated weights for policy 0, policy_version 654781 (0.0009) [2023-12-26 20:08:04,005][105692] Updated weights for policy 0, policy_version 654791 (0.0007) [2023-12-26 20:08:04,643][105620] Updated weights for policy 1, policy_version 655643 (0.0007) [2023-12-26 20:08:04,699][105620] Updated weights for policy 1, policy_version 655653 (0.0009) [2023-12-26 20:08:04,742][105692] Updated weights for policy 0, policy_version 654801 (0.0007) [2023-12-26 20:08:04,756][105620] Updated weights for policy 1, policy_version 655663 (0.0007) [2023-12-26 20:08:04,796][105692] Updated weights for policy 0, policy_version 654811 (0.0006) [2023-12-26 20:08:04,840][105692] Updated weights for policy 0, policy_version 654821 (0.0008) [2023-12-26 20:08:05,394][105620] Updated weights for policy 1, policy_version 655673 (0.0008) [2023-12-26 20:08:05,431][105692] Updated weights for policy 0, policy_version 654831 (0.0006) [2023-12-26 20:08:05,454][105620] Updated weights for policy 1, policy_version 655683 (0.0006) [2023-12-26 20:08:05,496][105692] Updated weights for policy 0, policy_version 654841 (0.0005) [2023-12-26 20:08:05,515][105620] Updated weights for policy 1, policy_version 655693 (0.0006) [2023-12-26 20:08:05,563][105692] Updated weights for policy 0, policy_version 654851 (0.0005) [2023-12-26 20:08:05,579][105620] Updated weights for policy 1, policy_version 655703 (0.0010) [2023-12-26 20:08:06,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 335552512. Throughput: 0: 9760.7, 1: 9591.4. Samples: 335540320. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 20:08:06,062][104569] Avg episode reward: [(0, '9354.373'), (1, '9353.721')] [2023-12-26 20:08:06,080][105692] Updated weights for policy 0, policy_version 654861 (0.0006) [2023-12-26 20:08:06,134][105692] Updated weights for policy 0, policy_version 654871 (0.0008) [2023-12-26 20:08:06,187][105692] Updated weights for policy 0, policy_version 654881 (0.0005) [2023-12-26 20:08:06,273][105620] Updated weights for policy 1, policy_version 655713 (0.0006) [2023-12-26 20:08:06,332][105620] Updated weights for policy 1, policy_version 655723 (0.0006) [2023-12-26 20:08:06,394][105620] Updated weights for policy 1, policy_version 655733 (0.0010) [2023-12-26 20:08:06,876][105692] Updated weights for policy 0, policy_version 654891 (0.0007) [2023-12-26 20:08:06,931][105692] Updated weights for policy 0, policy_version 654901 (0.0006) [2023-12-26 20:08:06,985][105692] Updated weights for policy 0, policy_version 654911 (0.0008) [2023-12-26 20:08:07,094][105620] Updated weights for policy 1, policy_version 655743 (0.0011) [2023-12-26 20:08:07,163][105620] Updated weights for policy 1, policy_version 655753 (0.0010) [2023-12-26 20:08:07,234][105620] Updated weights for policy 1, policy_version 655763 (0.0009) [2023-12-26 20:08:07,711][105692] Updated weights for policy 0, policy_version 654921 (0.0008) [2023-12-26 20:08:07,767][105692] Updated weights for policy 0, policy_version 654931 (0.0006) [2023-12-26 20:08:07,826][105692] Updated weights for policy 0, policy_version 654941 (0.0008) [2023-12-26 20:08:07,882][105692] Updated weights for policy 0, policy_version 654951 (0.0007) [2023-12-26 20:08:07,892][105620] Updated weights for policy 1, policy_version 655773 (0.0008) [2023-12-26 20:08:07,947][105620] Updated weights for policy 1, policy_version 655783 (0.0010) [2023-12-26 20:08:08,000][105620] Updated weights for policy 1, policy_version 655793 (0.0008) [2023-12-26 20:08:08,643][105692] Updated weights for policy 0, policy_version 654961 (0.0009) [2023-12-26 20:08:08,706][105692] Updated weights for policy 0, policy_version 654971 (0.0008) [2023-12-26 20:08:08,759][105620] Updated weights for policy 1, policy_version 655803 (0.0010) [2023-12-26 20:08:08,765][105692] Updated weights for policy 0, policy_version 654981 (0.0007) [2023-12-26 20:08:08,821][105620] Updated weights for policy 1, policy_version 655813 (0.0011) [2023-12-26 20:08:08,884][105620] Updated weights for policy 1, policy_version 655823 (0.0011) [2023-12-26 20:08:09,530][105692] Updated weights for policy 0, policy_version 654991 (0.0008) [2023-12-26 20:08:09,583][105692] Updated weights for policy 0, policy_version 655001 (0.0008) [2023-12-26 20:08:09,630][105620] Updated weights for policy 1, policy_version 655833 (0.0011) [2023-12-26 20:08:09,640][105692] Updated weights for policy 0, policy_version 655011 (0.0009) [2023-12-26 20:08:09,686][105620] Updated weights for policy 1, policy_version 655843 (0.0011) [2023-12-26 20:08:09,743][105620] Updated weights for policy 1, policy_version 655853 (0.0011) [2023-12-26 20:08:09,806][105620] Updated weights for policy 1, policy_version 655863 (0.0011) [2023-12-26 20:08:10,432][105692] Updated weights for policy 0, policy_version 655021 (0.0005) [2023-12-26 20:08:10,501][105692] Updated weights for policy 0, policy_version 655031 (0.0005) [2023-12-26 20:08:10,564][105692] Updated weights for policy 0, policy_version 655041 (0.0007) [2023-12-26 20:08:10,578][105620] Updated weights for policy 1, policy_version 655873 (0.0009) [2023-12-26 20:08:10,633][105620] Updated weights for policy 1, policy_version 655883 (0.0007) [2023-12-26 20:08:10,695][105620] Updated weights for policy 1, policy_version 655893 (0.0009) [2023-12-26 20:08:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 335650816. Throughput: 0: 9787.2, 1: 9556.5. Samples: 335658576. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 20:08:11,063][104569] Avg episode reward: [(0, '9263.176'), (1, '8988.414')] [2023-12-26 20:08:11,175][105692] Updated weights for policy 0, policy_version 655051 (0.0008) [2023-12-26 20:08:11,241][105692] Updated weights for policy 0, policy_version 655061 (0.0008) [2023-12-26 20:08:11,306][105692] Updated weights for policy 0, policy_version 655071 (0.0008) [2023-12-26 20:08:11,493][105620] Updated weights for policy 1, policy_version 655903 (0.0007) [2023-12-26 20:08:11,548][105620] Updated weights for policy 1, policy_version 655913 (0.0006) [2023-12-26 20:08:11,610][105620] Updated weights for policy 1, policy_version 655923 (0.0008) [2023-12-26 20:08:12,082][105692] Updated weights for policy 0, policy_version 655081 (0.0008) [2023-12-26 20:08:12,143][105692] Updated weights for policy 0, policy_version 655091 (0.0009) [2023-12-26 20:08:12,194][105585] KL-divergence is very high: 128.8660 [2023-12-26 20:08:12,205][105692] Updated weights for policy 0, policy_version 655101 (0.0009) [2023-12-26 20:08:12,249][105585] KL-divergence is very high: 128.4573 [2023-12-26 20:08:12,274][105692] Updated weights for policy 0, policy_version 655111 (0.0009) [2023-12-26 20:08:12,380][105620] Updated weights for policy 1, policy_version 655933 (0.0007) [2023-12-26 20:08:12,448][105620] Updated weights for policy 1, policy_version 655943 (0.0011) [2023-12-26 20:08:12,511][105620] Updated weights for policy 1, policy_version 655953 (0.0009) [2023-12-26 20:08:12,955][105692] Updated weights for policy 0, policy_version 655121 (0.0009) [2023-12-26 20:08:13,013][105692] Updated weights for policy 0, policy_version 655131 (0.0010) [2023-12-26 20:08:13,072][105692] Updated weights for policy 0, policy_version 655141 (0.0011) [2023-12-26 20:08:13,216][105620] Updated weights for policy 1, policy_version 655964 (0.0009) [2023-12-26 20:08:13,272][105620] Updated weights for policy 1, policy_version 655974 (0.0006) [2023-12-26 20:08:13,322][105620] Updated weights for policy 1, policy_version 655984 (0.0008) [2023-12-26 20:08:13,779][105692] Updated weights for policy 0, policy_version 655151 (0.0007) [2023-12-26 20:08:13,841][105692] Updated weights for policy 0, policy_version 655161 (0.0005) [2023-12-26 20:08:13,903][105692] Updated weights for policy 0, policy_version 655171 (0.0006) [2023-12-26 20:08:14,076][105620] Updated weights for policy 1, policy_version 655994 (0.0008) [2023-12-26 20:08:14,132][105620] Updated weights for policy 1, policy_version 656004 (0.0005) [2023-12-26 20:08:14,198][105620] Updated weights for policy 1, policy_version 656014 (0.0005) [2023-12-26 20:08:14,261][105620] Updated weights for policy 1, policy_version 656024 (0.0006) [2023-12-26 20:08:14,480][105692] Updated weights for policy 0, policy_version 655181 (0.0009) [2023-12-26 20:08:14,540][105692] Updated weights for policy 0, policy_version 655191 (0.0011) [2023-12-26 20:08:14,606][105692] Updated weights for policy 0, policy_version 655201 (0.0011) [2023-12-26 20:08:14,955][105620] Updated weights for policy 1, policy_version 656034 (0.0009) [2023-12-26 20:08:15,016][105620] Updated weights for policy 1, policy_version 656044 (0.0008) [2023-12-26 20:08:15,075][105620] Updated weights for policy 1, policy_version 656054 (0.0008) [2023-12-26 20:08:15,374][105692] Updated weights for policy 0, policy_version 655211 (0.0011) [2023-12-26 20:08:15,438][105692] Updated weights for policy 0, policy_version 655221 (0.0011) [2023-12-26 20:08:15,500][105692] Updated weights for policy 0, policy_version 655231 (0.0010) [2023-12-26 20:08:15,777][105620] Updated weights for policy 1, policy_version 656064 (0.0006) [2023-12-26 20:08:15,825][105620] Updated weights for policy 1, policy_version 656074 (0.0005) [2023-12-26 20:08:15,876][105620] Updated weights for policy 1, policy_version 656084 (0.0005) [2023-12-26 20:08:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 335749120. Throughput: 0: 9792.1, 1: 9546.5. Samples: 335715648. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 20:08:16,063][104569] Avg episode reward: [(0, '9170.525'), (1, '8804.290')] [2023-12-26 20:08:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000655240_167772160.pth... [2023-12-26 20:08:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000656088_167976960.pth... [2023-12-26 20:08:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000654120_167485440.pth [2023-12-26 20:08:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000654936_167682048.pth [2023-12-26 20:08:16,154][105692] Updated weights for policy 0, policy_version 655241 (0.0010) [2023-12-26 20:08:16,207][105692] Updated weights for policy 0, policy_version 655251 (0.0005) [2023-12-26 20:08:16,258][105692] Updated weights for policy 0, policy_version 655261 (0.0010) [2023-12-26 20:08:16,306][105692] Updated weights for policy 0, policy_version 655271 (0.0010) [2023-12-26 20:08:16,612][105620] Updated weights for policy 1, policy_version 656094 (0.0008) [2023-12-26 20:08:16,672][105620] Updated weights for policy 1, policy_version 656104 (0.0010) [2023-12-26 20:08:16,731][105620] Updated weights for policy 1, policy_version 656114 (0.0010) [2023-12-26 20:08:16,869][105692] Updated weights for policy 0, policy_version 655281 (0.0009) [2023-12-26 20:08:16,916][105692] Updated weights for policy 0, policy_version 655291 (0.0010) [2023-12-26 20:08:16,967][105692] Updated weights for policy 0, policy_version 655301 (0.0007) [2023-12-26 20:08:17,391][105620] Updated weights for policy 1, policy_version 656124 (0.0010) [2023-12-26 20:08:17,459][105620] Updated weights for policy 1, policy_version 656134 (0.0010) [2023-12-26 20:08:17,525][105620] Updated weights for policy 1, policy_version 656144 (0.0010) [2023-12-26 20:08:17,677][105692] Updated weights for policy 0, policy_version 655311 (0.0009) [2023-12-26 20:08:17,743][105692] Updated weights for policy 0, policy_version 655321 (0.0010) [2023-12-26 20:08:17,805][105692] Updated weights for policy 0, policy_version 655331 (0.0010) [2023-12-26 20:08:18,098][105620] Updated weights for policy 1, policy_version 656154 (0.0009) [2023-12-26 20:08:18,155][105620] Updated weights for policy 1, policy_version 656164 (0.0006) [2023-12-26 20:08:18,214][105620] Updated weights for policy 1, policy_version 656174 (0.0005) [2023-12-26 20:08:18,263][105620] Updated weights for policy 1, policy_version 656184 (0.0005) [2023-12-26 20:08:18,524][105692] Updated weights for policy 0, policy_version 655341 (0.0010) [2023-12-26 20:08:18,580][105585] KL-divergence is very high: 117.9290 [2023-12-26 20:08:18,586][105692] Updated weights for policy 0, policy_version 655351 (0.0010) [2023-12-26 20:08:18,608][105585] KL-divergence is very high: 108.3526 [2023-12-26 20:08:18,614][105585] KL-divergence is very high: 150.0675 [2023-12-26 20:08:18,623][105585] KL-divergence is very high: 140.9621 [2023-12-26 20:08:18,638][105692] Updated weights for policy 0, policy_version 655361 (0.0010) [2023-12-26 20:08:18,650][105585] KL-divergence is very high: 112.5216 [2023-12-26 20:08:18,897][105620] Updated weights for policy 1, policy_version 656194 (0.0005) [2023-12-26 20:08:18,943][105620] Updated weights for policy 1, policy_version 656204 (0.0005) [2023-12-26 20:08:19,006][105620] Updated weights for policy 1, policy_version 656214 (0.0006) [2023-12-26 20:08:19,407][105692] Updated weights for policy 0, policy_version 655371 (0.0010) [2023-12-26 20:08:19,472][105692] Updated weights for policy 0, policy_version 655381 (0.0010) [2023-12-26 20:08:19,537][105692] Updated weights for policy 0, policy_version 655391 (0.0010) [2023-12-26 20:08:19,653][105620] Updated weights for policy 1, policy_version 656224 (0.0007) [2023-12-26 20:08:19,710][105620] Updated weights for policy 1, policy_version 656234 (0.0007) [2023-12-26 20:08:19,774][105620] Updated weights for policy 1, policy_version 656244 (0.0008) [2023-12-26 20:08:20,304][105692] Updated weights for policy 0, policy_version 655401 (0.0010) [2023-12-26 20:08:20,378][105692] Updated weights for policy 0, policy_version 655411 (0.0006) [2023-12-26 20:08:20,447][105692] Updated weights for policy 0, policy_version 655421 (0.0009) [2023-12-26 20:08:20,448][105620] Updated weights for policy 1, policy_version 656254 (0.0007) [2023-12-26 20:08:20,509][105620] Updated weights for policy 1, policy_version 656264 (0.0008) [2023-12-26 20:08:20,510][105692] Updated weights for policy 0, policy_version 655431 (0.0008) [2023-12-26 20:08:20,581][105620] Updated weights for policy 1, policy_version 656274 (0.0009) [2023-12-26 20:08:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 335847424. Throughput: 0: 9790.4, 1: 9684.3. Samples: 335837832. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 20:08:21,063][104569] Avg episode reward: [(0, '8986.988'), (1, '8896.501')] [2023-12-26 20:08:21,119][105692] Updated weights for policy 0, policy_version 655441 (0.0010) [2023-12-26 20:08:21,180][105692] Updated weights for policy 0, policy_version 655451 (0.0007) [2023-12-26 20:08:21,241][105692] Updated weights for policy 0, policy_version 655461 (0.0006) [2023-12-26 20:08:21,398][105620] Updated weights for policy 1, policy_version 656284 (0.0008) [2023-12-26 20:08:21,463][105620] Updated weights for policy 1, policy_version 656294 (0.0010) [2023-12-26 20:08:21,530][105620] Updated weights for policy 1, policy_version 656304 (0.0010) [2023-12-26 20:08:21,920][105692] Updated weights for policy 0, policy_version 655471 (0.0010) [2023-12-26 20:08:21,974][105692] Updated weights for policy 0, policy_version 655481 (0.0011) [2023-12-26 20:08:22,040][105692] Updated weights for policy 0, policy_version 655491 (0.0011) [2023-12-26 20:08:22,287][105620] Updated weights for policy 1, policy_version 656314 (0.0008) [2023-12-26 20:08:22,357][105620] Updated weights for policy 1, policy_version 656324 (0.0007) [2023-12-26 20:08:22,420][105620] Updated weights for policy 1, policy_version 656334 (0.0009) [2023-12-26 20:08:22,472][105620] Updated weights for policy 1, policy_version 656344 (0.0008) [2023-12-26 20:08:22,828][105692] Updated weights for policy 0, policy_version 655501 (0.0011) [2023-12-26 20:08:22,894][105692] Updated weights for policy 0, policy_version 655511 (0.0010) [2023-12-26 20:08:22,956][105692] Updated weights for policy 0, policy_version 655521 (0.0010) [2023-12-26 20:08:23,158][105620] Updated weights for policy 1, policy_version 656354 (0.0008) [2023-12-26 20:08:23,211][105620] Updated weights for policy 1, policy_version 656364 (0.0008) [2023-12-26 20:08:23,272][105620] Updated weights for policy 1, policy_version 656374 (0.0008) [2023-12-26 20:08:23,685][105692] Updated weights for policy 0, policy_version 655531 (0.0010) [2023-12-26 20:08:23,736][105692] Updated weights for policy 0, policy_version 655541 (0.0010) [2023-12-26 20:08:23,784][105692] Updated weights for policy 0, policy_version 655551 (0.0010) [2023-12-26 20:08:24,044][105620] Updated weights for policy 1, policy_version 656384 (0.0008) [2023-12-26 20:08:24,114][105620] Updated weights for policy 1, policy_version 656394 (0.0009) [2023-12-26 20:08:24,165][105620] Updated weights for policy 1, policy_version 656404 (0.0005) [2023-12-26 20:08:24,530][105692] Updated weights for policy 0, policy_version 655561 (0.0010) [2023-12-26 20:08:24,584][105692] Updated weights for policy 0, policy_version 655571 (0.0005) [2023-12-26 20:08:24,637][105692] Updated weights for policy 0, policy_version 655581 (0.0005) [2023-12-26 20:08:24,705][105692] Updated weights for policy 0, policy_version 655591 (0.0005) [2023-12-26 20:08:24,940][105620] Updated weights for policy 1, policy_version 656414 (0.0007) [2023-12-26 20:08:24,998][105620] Updated weights for policy 1, policy_version 656424 (0.0010) [2023-12-26 20:08:25,057][105620] Updated weights for policy 1, policy_version 656435 (0.0010) [2023-12-26 20:08:25,283][105692] Updated weights for policy 0, policy_version 655601 (0.0008) [2023-12-26 20:08:25,330][105692] Updated weights for policy 0, policy_version 655611 (0.0008) [2023-12-26 20:08:25,376][105692] Updated weights for policy 0, policy_version 655621 (0.0008) [2023-12-26 20:08:25,827][105620] Updated weights for policy 1, policy_version 656445 (0.0010) [2023-12-26 20:08:25,885][105620] Updated weights for policy 1, policy_version 656455 (0.0009) [2023-12-26 20:08:25,943][105620] Updated weights for policy 1, policy_version 656465 (0.0010) [2023-12-26 20:08:26,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19524.4, 300 sec: 19522.0). Total num frames: 335945728. Throughput: 0: 9785.2, 1: 9704.6. Samples: 335951600. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 20:08:26,062][104569] Avg episode reward: [(0, '9172.308'), (1, '9173.329')] [2023-12-26 20:08:26,095][105692] Updated weights for policy 0, policy_version 655631 (0.0009) [2023-12-26 20:08:26,157][105692] Updated weights for policy 0, policy_version 655641 (0.0009) [2023-12-26 20:08:26,221][105692] Updated weights for policy 0, policy_version 655651 (0.0009) [2023-12-26 20:08:26,645][105620] Updated weights for policy 1, policy_version 656475 (0.0009) [2023-12-26 20:08:26,696][105620] Updated weights for policy 1, policy_version 656485 (0.0009) [2023-12-26 20:08:26,747][105620] Updated weights for policy 1, policy_version 656495 (0.0010) [2023-12-26 20:08:26,995][105692] Updated weights for policy 0, policy_version 655661 (0.0009) [2023-12-26 20:08:27,058][105692] Updated weights for policy 0, policy_version 655671 (0.0010) [2023-12-26 20:08:27,115][105692] Updated weights for policy 0, policy_version 655681 (0.0010) [2023-12-26 20:08:27,380][105620] Updated weights for policy 1, policy_version 656505 (0.0010) [2023-12-26 20:08:27,451][105620] Updated weights for policy 1, policy_version 656515 (0.0006) [2023-12-26 20:08:27,512][105620] Updated weights for policy 1, policy_version 656525 (0.0005) [2023-12-26 20:08:27,560][105620] Updated weights for policy 1, policy_version 656535 (0.0005) [2023-12-26 20:08:27,746][105692] Updated weights for policy 0, policy_version 655691 (0.0008) [2023-12-26 20:08:27,794][105692] Updated weights for policy 0, policy_version 655701 (0.0005) [2023-12-26 20:08:27,847][105692] Updated weights for policy 0, policy_version 655711 (0.0005) [2023-12-26 20:08:28,050][105620] Updated weights for policy 1, policy_version 656545 (0.0006) [2023-12-26 20:08:28,095][105620] Updated weights for policy 1, policy_version 656555 (0.0005) [2023-12-26 20:08:28,148][105620] Updated weights for policy 1, policy_version 656565 (0.0007) [2023-12-26 20:08:28,585][105692] Updated weights for policy 0, policy_version 655722 (0.0010) [2023-12-26 20:08:28,642][105692] Updated weights for policy 0, policy_version 655732 (0.0009) [2023-12-26 20:08:28,703][105692] Updated weights for policy 0, policy_version 655742 (0.0009) [2023-12-26 20:08:28,760][105692] Updated weights for policy 0, policy_version 655752 (0.0009) [2023-12-26 20:08:28,877][105620] Updated weights for policy 1, policy_version 656575 (0.0009) [2023-12-26 20:08:28,935][105620] Updated weights for policy 1, policy_version 656585 (0.0009) [2023-12-26 20:08:28,997][105620] Updated weights for policy 1, policy_version 656596 (0.0009) [2023-12-26 20:08:29,508][105692] Updated weights for policy 0, policy_version 655762 (0.0009) [2023-12-26 20:08:29,562][105692] Updated weights for policy 0, policy_version 655772 (0.0009) [2023-12-26 20:08:29,624][105692] Updated weights for policy 0, policy_version 655782 (0.0009) [2023-12-26 20:08:29,759][105620] Updated weights for policy 1, policy_version 656606 (0.0009) [2023-12-26 20:08:29,810][105620] Updated weights for policy 1, policy_version 656616 (0.0007) [2023-12-26 20:08:29,874][105620] Updated weights for policy 1, policy_version 656626 (0.0008) [2023-12-26 20:08:30,418][105692] Updated weights for policy 0, policy_version 655792 (0.0009) [2023-12-26 20:08:30,468][105692] Updated weights for policy 0, policy_version 655802 (0.0009) [2023-12-26 20:08:30,522][105692] Updated weights for policy 0, policy_version 655812 (0.0009) [2023-12-26 20:08:30,572][105620] Updated weights for policy 1, policy_version 656636 (0.0007) [2023-12-26 20:08:30,634][105620] Updated weights for policy 1, policy_version 656646 (0.0008) [2023-12-26 20:08:30,678][105620] Updated weights for policy 1, policy_version 656656 (0.0008) [2023-12-26 20:08:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 336044032. Throughput: 0: 9790.8, 1: 9846.6. Samples: 336014000. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 20:08:31,062][104569] Avg episode reward: [(0, '9354.120'), (1, '9082.438')] [2023-12-26 20:08:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000656664_168124416.pth... [2023-12-26 20:08:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000655816_167919616.pth... [2023-12-26 20:08:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000655512_167829504.pth [2023-12-26 20:08:31,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000654664_167624704.pth [2023-12-26 20:08:31,267][105692] Updated weights for policy 0, policy_version 655822 (0.0010) [2023-12-26 20:08:31,333][105692] Updated weights for policy 0, policy_version 655832 (0.0009) [2023-12-26 20:08:31,398][105692] Updated weights for policy 0, policy_version 655842 (0.0007) [2023-12-26 20:08:31,483][105620] Updated weights for policy 1, policy_version 656666 (0.0007) [2023-12-26 20:08:31,548][105620] Updated weights for policy 1, policy_version 656676 (0.0005) [2023-12-26 20:08:31,615][105620] Updated weights for policy 1, policy_version 656686 (0.0006) [2023-12-26 20:08:31,675][105620] Updated weights for policy 1, policy_version 656696 (0.0007) [2023-12-26 20:08:32,089][105692] Updated weights for policy 0, policy_version 655852 (0.0008) [2023-12-26 20:08:32,141][105692] Updated weights for policy 0, policy_version 655862 (0.0005) [2023-12-26 20:08:32,193][105692] Updated weights for policy 0, policy_version 655872 (0.0005) [2023-12-26 20:08:32,352][105620] Updated weights for policy 1, policy_version 656706 (0.0011) [2023-12-26 20:08:32,421][105620] Updated weights for policy 1, policy_version 656716 (0.0011) [2023-12-26 20:08:32,473][105620] Updated weights for policy 1, policy_version 656726 (0.0010) [2023-12-26 20:08:32,770][105692] Updated weights for policy 0, policy_version 655882 (0.0005) [2023-12-26 20:08:32,823][105692] Updated weights for policy 0, policy_version 655892 (0.0006) [2023-12-26 20:08:32,867][105692] Updated weights for policy 0, policy_version 655902 (0.0005) [2023-12-26 20:08:32,921][105692] Updated weights for policy 0, policy_version 655912 (0.0007) [2023-12-26 20:08:33,242][105620] Updated weights for policy 1, policy_version 656736 (0.0009) [2023-12-26 20:08:33,295][105620] Updated weights for policy 1, policy_version 656746 (0.0006) [2023-12-26 20:08:33,359][105620] Updated weights for policy 1, policy_version 656756 (0.0005) [2023-12-26 20:08:33,555][105692] Updated weights for policy 0, policy_version 655922 (0.0009) [2023-12-26 20:08:33,597][105692] Updated weights for policy 0, policy_version 655932 (0.0006) [2023-12-26 20:08:33,652][105692] Updated weights for policy 0, policy_version 655942 (0.0005) [2023-12-26 20:08:33,986][105620] Updated weights for policy 1, policy_version 656766 (0.0007) [2023-12-26 20:08:34,044][105620] Updated weights for policy 1, policy_version 656777 (0.0014) [2023-12-26 20:08:34,114][105620] Updated weights for policy 1, policy_version 656787 (0.0005) [2023-12-26 20:08:34,285][105692] Updated weights for policy 0, policy_version 655952 (0.0007) [2023-12-26 20:08:34,351][105692] Updated weights for policy 0, policy_version 655962 (0.0006) [2023-12-26 20:08:34,413][105692] Updated weights for policy 0, policy_version 655972 (0.0006) [2023-12-26 20:08:34,871][105620] Updated weights for policy 1, policy_version 656797 (0.0009) [2023-12-26 20:08:34,934][105620] Updated weights for policy 1, policy_version 656807 (0.0008) [2023-12-26 20:08:35,000][105620] Updated weights for policy 1, policy_version 656817 (0.0009) [2023-12-26 20:08:35,052][105692] Updated weights for policy 0, policy_version 655982 (0.0006) [2023-12-26 20:08:35,113][105692] Updated weights for policy 0, policy_version 655992 (0.0008) [2023-12-26 20:08:35,161][105692] Updated weights for policy 0, policy_version 656002 (0.0010) [2023-12-26 20:08:35,655][105620] Updated weights for policy 1, policy_version 656827 (0.0008) [2023-12-26 20:08:35,710][105620] Updated weights for policy 1, policy_version 656837 (0.0005) [2023-12-26 20:08:35,768][105620] Updated weights for policy 1, policy_version 656847 (0.0007) [2023-12-26 20:08:35,914][105692] Updated weights for policy 0, policy_version 656012 (0.0009) [2023-12-26 20:08:35,970][105692] Updated weights for policy 0, policy_version 656022 (0.0009) [2023-12-26 20:08:36,021][105692] Updated weights for policy 0, policy_version 656032 (0.0009) [2023-12-26 20:08:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 336150528. Throughput: 0: 9901.3, 1: 9736.4. Samples: 336132840. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 20:08:36,063][104569] Avg episode reward: [(0, '9262.628'), (1, '8626.912')] [2023-12-26 20:08:36,408][105620] Updated weights for policy 1, policy_version 656857 (0.0008) [2023-12-26 20:08:36,457][105620] Updated weights for policy 1, policy_version 656867 (0.0010) [2023-12-26 20:08:36,513][105620] Updated weights for policy 1, policy_version 656877 (0.0010) [2023-12-26 20:08:36,569][105620] Updated weights for policy 1, policy_version 656887 (0.0008) [2023-12-26 20:08:36,743][105692] Updated weights for policy 0, policy_version 656042 (0.0010) [2023-12-26 20:08:36,795][105692] Updated weights for policy 0, policy_version 656052 (0.0010) [2023-12-26 20:08:36,842][105692] Updated weights for policy 0, policy_version 656062 (0.0007) [2023-12-26 20:08:36,902][105692] Updated weights for policy 0, policy_version 656072 (0.0006) [2023-12-26 20:08:37,279][105620] Updated weights for policy 1, policy_version 656897 (0.0006) [2023-12-26 20:08:37,344][105620] Updated weights for policy 1, policy_version 656907 (0.0005) [2023-12-26 20:08:37,408][105620] Updated weights for policy 1, policy_version 656917 (0.0005) [2023-12-26 20:08:37,502][105692] Updated weights for policy 0, policy_version 656082 (0.0011) [2023-12-26 20:08:37,569][105692] Updated weights for policy 0, policy_version 656092 (0.0010) [2023-12-26 20:08:37,632][105692] Updated weights for policy 0, policy_version 656102 (0.0010) [2023-12-26 20:08:38,060][105620] Updated weights for policy 1, policy_version 656927 (0.0011) [2023-12-26 20:08:38,122][105620] Updated weights for policy 1, policy_version 656937 (0.0011) [2023-12-26 20:08:38,185][105620] Updated weights for policy 1, policy_version 656947 (0.0010) [2023-12-26 20:08:38,328][105692] Updated weights for policy 0, policy_version 656112 (0.0007) [2023-12-26 20:08:38,391][105692] Updated weights for policy 0, policy_version 656122 (0.0009) [2023-12-26 20:08:38,458][105692] Updated weights for policy 0, policy_version 656132 (0.0011) [2023-12-26 20:08:38,899][105620] Updated weights for policy 1, policy_version 656957 (0.0010) [2023-12-26 20:08:38,972][105620] Updated weights for policy 1, policy_version 656967 (0.0010) [2023-12-26 20:08:39,037][105620] Updated weights for policy 1, policy_version 656977 (0.0009) [2023-12-26 20:08:39,153][105692] Updated weights for policy 0, policy_version 656142 (0.0010) [2023-12-26 20:08:39,210][105692] Updated weights for policy 0, policy_version 656152 (0.0009) [2023-12-26 20:08:39,276][105692] Updated weights for policy 0, policy_version 656162 (0.0007) [2023-12-26 20:08:39,678][105620] Updated weights for policy 1, policy_version 656987 (0.0008) [2023-12-26 20:08:39,741][105620] Updated weights for policy 1, policy_version 656997 (0.0008) [2023-12-26 20:08:39,806][105620] Updated weights for policy 1, policy_version 657007 (0.0007) [2023-12-26 20:08:40,101][105692] Updated weights for policy 0, policy_version 656172 (0.0007) [2023-12-26 20:08:40,162][105692] Updated weights for policy 0, policy_version 656182 (0.0009) [2023-12-26 20:08:40,225][105692] Updated weights for policy 0, policy_version 656192 (0.0009) [2023-12-26 20:08:40,457][105620] Updated weights for policy 1, policy_version 657017 (0.0007) [2023-12-26 20:08:40,516][105620] Updated weights for policy 1, policy_version 657027 (0.0005) [2023-12-26 20:08:40,575][105620] Updated weights for policy 1, policy_version 657037 (0.0005) [2023-12-26 20:08:40,642][105620] Updated weights for policy 1, policy_version 657047 (0.0007) [2023-12-26 20:08:41,037][105692] Updated weights for policy 0, policy_version 656202 (0.0009) [2023-12-26 20:08:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 336240640. Throughput: 0: 9877.2, 1: 9819.7. Samples: 336252152. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 20:08:41,063][104569] Avg episode reward: [(0, '9262.767'), (1, '8535.445')] [2023-12-26 20:08:41,112][105692] Updated weights for policy 0, policy_version 656212 (0.0006) [2023-12-26 20:08:41,172][105692] Updated weights for policy 0, policy_version 656222 (0.0009) [2023-12-26 20:08:41,233][105692] Updated weights for policy 0, policy_version 656232 (0.0010) [2023-12-26 20:08:41,263][105620] Updated weights for policy 1, policy_version 657057 (0.0006) [2023-12-26 20:08:41,324][105620] Updated weights for policy 1, policy_version 657067 (0.0007) [2023-12-26 20:08:41,400][105620] Updated weights for policy 1, policy_version 657077 (0.0009) [2023-12-26 20:08:41,978][105692] Updated weights for policy 0, policy_version 656242 (0.0009) [2023-12-26 20:08:42,040][105692] Updated weights for policy 0, policy_version 656252 (0.0009) [2023-12-26 20:08:42,098][105692] Updated weights for policy 0, policy_version 656262 (0.0009) [2023-12-26 20:08:42,111][105620] Updated weights for policy 1, policy_version 657087 (0.0006) [2023-12-26 20:08:42,179][105620] Updated weights for policy 1, policy_version 657097 (0.0006) [2023-12-26 20:08:42,248][105620] Updated weights for policy 1, policy_version 657107 (0.0006) [2023-12-26 20:08:42,908][105692] Updated weights for policy 0, policy_version 656272 (0.0008) [2023-12-26 20:08:42,927][105620] Updated weights for policy 1, policy_version 657117 (0.0008) [2023-12-26 20:08:42,958][105692] Updated weights for policy 0, policy_version 656282 (0.0005) [2023-12-26 20:08:42,980][105620] Updated weights for policy 1, policy_version 657127 (0.0008) [2023-12-26 20:08:43,018][105692] Updated weights for policy 0, policy_version 656292 (0.0007) [2023-12-26 20:08:43,037][105620] Updated weights for policy 1, policy_version 657137 (0.0006) [2023-12-26 20:08:43,718][105620] Updated weights for policy 1, policy_version 657147 (0.0009) [2023-12-26 20:08:43,780][105620] Updated weights for policy 1, policy_version 657157 (0.0008) [2023-12-26 20:08:43,795][105692] Updated weights for policy 0, policy_version 656302 (0.0006) [2023-12-26 20:08:43,843][105620] Updated weights for policy 1, policy_version 657167 (0.0009) [2023-12-26 20:08:43,853][105692] Updated weights for policy 0, policy_version 656312 (0.0006) [2023-12-26 20:08:43,902][105692] Updated weights for policy 0, policy_version 656322 (0.0006) [2023-12-26 20:08:44,595][105692] Updated weights for policy 0, policy_version 656332 (0.0009) [2023-12-26 20:08:44,615][105620] Updated weights for policy 1, policy_version 657177 (0.0006) [2023-12-26 20:08:44,646][105692] Updated weights for policy 0, policy_version 656342 (0.0007) [2023-12-26 20:08:44,668][105620] Updated weights for policy 1, policy_version 657187 (0.0007) [2023-12-26 20:08:44,699][105692] Updated weights for policy 0, policy_version 656352 (0.0007) [2023-12-26 20:08:44,717][105620] Updated weights for policy 1, policy_version 657197 (0.0006) [2023-12-26 20:08:44,773][105620] Updated weights for policy 1, policy_version 657207 (0.0006) [2023-12-26 20:08:45,391][105620] Updated weights for policy 1, policy_version 657217 (0.0010) [2023-12-26 20:08:45,447][105620] Updated weights for policy 1, policy_version 657227 (0.0009) [2023-12-26 20:08:45,502][105620] Updated weights for policy 1, policy_version 657237 (0.0009) [2023-12-26 20:08:45,586][105692] Updated weights for policy 0, policy_version 656362 (0.0008) [2023-12-26 20:08:45,642][105692] Updated weights for policy 0, policy_version 656372 (0.0005) [2023-12-26 20:08:45,707][105692] Updated weights for policy 0, policy_version 656382 (0.0006) [2023-12-26 20:08:45,760][105692] Updated weights for policy 0, policy_version 656392 (0.0005) [2023-12-26 20:08:46,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 336338944. Throughput: 0: 9763.7, 1: 9865.2. Samples: 336307968. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 20:08:46,063][104569] Avg episode reward: [(0, '9353.921'), (1, '8534.563')] [2023-12-26 20:08:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000656392_168067072.pth... [2023-12-26 20:08:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000657240_168271872.pth... [2023-12-26 20:08:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000656088_167976960.pth [2023-12-26 20:08:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000655240_167772160.pth [2023-12-26 20:08:46,289][105692] Updated weights for policy 0, policy_version 656402 (0.0011) [2023-12-26 20:08:46,338][105692] Updated weights for policy 0, policy_version 656412 (0.0010) [2023-12-26 20:08:46,347][105620] Updated weights for policy 1, policy_version 657247 (0.0007) [2023-12-26 20:08:46,393][105692] Updated weights for policy 0, policy_version 656422 (0.0010) [2023-12-26 20:08:46,395][105620] Updated weights for policy 1, policy_version 657257 (0.0005) [2023-12-26 20:08:46,449][105620] Updated weights for policy 1, policy_version 657267 (0.0008) [2023-12-26 20:08:47,140][105692] Updated weights for policy 0, policy_version 656432 (0.0010) [2023-12-26 20:08:47,198][105692] Updated weights for policy 0, policy_version 656442 (0.0010) [2023-12-26 20:08:47,217][105620] Updated weights for policy 1, policy_version 657277 (0.0007) [2023-12-26 20:08:47,260][105692] Updated weights for policy 0, policy_version 656452 (0.0010) [2023-12-26 20:08:47,266][105620] Updated weights for policy 1, policy_version 657287 (0.0008) [2023-12-26 20:08:47,314][105620] Updated weights for policy 1, policy_version 657297 (0.0007) [2023-12-26 20:08:47,989][105692] Updated weights for policy 0, policy_version 656462 (0.0007) [2023-12-26 20:08:48,038][105692] Updated weights for policy 0, policy_version 656472 (0.0005) [2023-12-26 20:08:48,085][105620] Updated weights for policy 1, policy_version 657307 (0.0007) [2023-12-26 20:08:48,098][105692] Updated weights for policy 0, policy_version 656482 (0.0009) [2023-12-26 20:08:48,140][105620] Updated weights for policy 1, policy_version 657317 (0.0006) [2023-12-26 20:08:48,192][105620] Updated weights for policy 1, policy_version 657327 (0.0008) [2023-12-26 20:08:48,824][105692] Updated weights for policy 0, policy_version 656492 (0.0011) [2023-12-26 20:08:48,890][105692] Updated weights for policy 0, policy_version 656502 (0.0011) [2023-12-26 20:08:48,948][105692] Updated weights for policy 0, policy_version 656512 (0.0006) [2023-12-26 20:08:48,951][105620] Updated weights for policy 1, policy_version 657337 (0.0007) [2023-12-26 20:08:48,998][105620] Updated weights for policy 1, policy_version 657347 (0.0006) [2023-12-26 20:08:49,043][105620] Updated weights for policy 1, policy_version 657357 (0.0005) [2023-12-26 20:08:49,095][105620] Updated weights for policy 1, policy_version 657367 (0.0006) [2023-12-26 20:08:49,657][105692] Updated weights for policy 0, policy_version 656522 (0.0007) [2023-12-26 20:08:49,723][105692] Updated weights for policy 0, policy_version 656532 (0.0009) [2023-12-26 20:08:49,735][105620] Updated weights for policy 1, policy_version 657377 (0.0006) [2023-12-26 20:08:49,786][105692] Updated weights for policy 0, policy_version 656542 (0.0008) [2023-12-26 20:08:49,800][105620] Updated weights for policy 1, policy_version 657387 (0.0006) [2023-12-26 20:08:49,845][105692] Updated weights for policy 0, policy_version 656552 (0.0008) [2023-12-26 20:08:49,864][105620] Updated weights for policy 1, policy_version 657397 (0.0008) [2023-12-26 20:08:50,530][105620] Updated weights for policy 1, policy_version 657407 (0.0006) [2023-12-26 20:08:50,556][105692] Updated weights for policy 0, policy_version 656562 (0.0006) [2023-12-26 20:08:50,594][105620] Updated weights for policy 1, policy_version 657417 (0.0009) [2023-12-26 20:08:50,614][105692] Updated weights for policy 0, policy_version 656572 (0.0008) [2023-12-26 20:08:50,652][105620] Updated weights for policy 1, policy_version 657427 (0.0008) [2023-12-26 20:08:50,672][105692] Updated weights for policy 0, policy_version 656582 (0.0006) [2023-12-26 20:08:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 336437248. Throughput: 0: 9832.2, 1: 9813.0. Samples: 336424352. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 20:08:51,063][104569] Avg episode reward: [(0, '9353.439'), (1, '8725.076')] [2023-12-26 20:08:51,405][105620] Updated weights for policy 1, policy_version 657437 (0.0007) [2023-12-26 20:08:51,423][105692] Updated weights for policy 0, policy_version 656592 (0.0010) [2023-12-26 20:08:51,465][105620] Updated weights for policy 1, policy_version 657447 (0.0007) [2023-12-26 20:08:51,476][105692] Updated weights for policy 0, policy_version 656602 (0.0010) [2023-12-26 20:08:51,528][105620] Updated weights for policy 1, policy_version 657457 (0.0008) [2023-12-26 20:08:51,533][105692] Updated weights for policy 0, policy_version 656612 (0.0011) [2023-12-26 20:08:52,158][105692] Updated weights for policy 0, policy_version 656622 (0.0008) [2023-12-26 20:08:52,226][105692] Updated weights for policy 0, policy_version 656632 (0.0006) [2023-12-26 20:08:52,287][105692] Updated weights for policy 0, policy_version 656642 (0.0009) [2023-12-26 20:08:52,309][105620] Updated weights for policy 1, policy_version 657467 (0.0007) [2023-12-26 20:08:52,369][105620] Updated weights for policy 1, policy_version 657477 (0.0008) [2023-12-26 20:08:52,433][105620] Updated weights for policy 1, policy_version 657487 (0.0007) [2023-12-26 20:08:53,030][105692] Updated weights for policy 0, policy_version 656652 (0.0009) [2023-12-26 20:08:53,078][105692] Updated weights for policy 0, policy_version 656662 (0.0010) [2023-12-26 20:08:53,095][105620] Updated weights for policy 1, policy_version 657497 (0.0007) [2023-12-26 20:08:53,126][105692] Updated weights for policy 0, policy_version 656672 (0.0010) [2023-12-26 20:08:53,148][105620] Updated weights for policy 1, policy_version 657507 (0.0009) [2023-12-26 20:08:53,206][105620] Updated weights for policy 1, policy_version 657517 (0.0010) [2023-12-26 20:08:53,275][105620] Updated weights for policy 1, policy_version 657527 (0.0010) [2023-12-26 20:08:53,896][105692] Updated weights for policy 0, policy_version 656682 (0.0011) [2023-12-26 20:08:53,955][105692] Updated weights for policy 0, policy_version 656692 (0.0010) [2023-12-26 20:08:54,024][105692] Updated weights for policy 0, policy_version 656702 (0.0009) [2023-12-26 20:08:54,035][105620] Updated weights for policy 1, policy_version 657537 (0.0006) [2023-12-26 20:08:54,082][105692] Updated weights for policy 0, policy_version 656712 (0.0007) [2023-12-26 20:08:54,101][105620] Updated weights for policy 1, policy_version 657547 (0.0005) [2023-12-26 20:08:54,165][105620] Updated weights for policy 1, policy_version 657557 (0.0010) [2023-12-26 20:08:54,652][105692] Updated weights for policy 0, policy_version 656722 (0.0011) [2023-12-26 20:08:54,707][105692] Updated weights for policy 0, policy_version 656732 (0.0010) [2023-12-26 20:08:54,764][105692] Updated weights for policy 0, policy_version 656742 (0.0010) [2023-12-26 20:08:54,921][105620] Updated weights for policy 1, policy_version 657567 (0.0009) [2023-12-26 20:08:54,981][105620] Updated weights for policy 1, policy_version 657577 (0.0008) [2023-12-26 20:08:55,037][105620] Updated weights for policy 1, policy_version 657587 (0.0008) [2023-12-26 20:08:55,455][105692] Updated weights for policy 0, policy_version 656752 (0.0007) [2023-12-26 20:08:55,512][105692] Updated weights for policy 0, policy_version 656762 (0.0010) [2023-12-26 20:08:55,563][105692] Updated weights for policy 0, policy_version 656772 (0.0006) [2023-12-26 20:08:55,835][105620] Updated weights for policy 1, policy_version 657597 (0.0008) [2023-12-26 20:08:55,889][105620] Updated weights for policy 1, policy_version 657607 (0.0009) [2023-12-26 20:08:55,947][105620] Updated weights for policy 1, policy_version 657617 (0.0009) [2023-12-26 20:08:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 336535552. Throughput: 0: 9832.4, 1: 9770.8. Samples: 336540724. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-26 20:08:56,063][104569] Avg episode reward: [(0, '9261.939'), (1, '8814.855')] [2023-12-26 20:08:56,233][105692] Updated weights for policy 0, policy_version 656782 (0.0009) [2023-12-26 20:08:56,297][105692] Updated weights for policy 0, policy_version 656792 (0.0008) [2023-12-26 20:08:56,348][105692] Updated weights for policy 0, policy_version 656802 (0.0006) [2023-12-26 20:08:56,723][105620] Updated weights for policy 1, policy_version 657627 (0.0009) [2023-12-26 20:08:56,781][105620] Updated weights for policy 1, policy_version 657638 (0.0010) [2023-12-26 20:08:56,838][105620] Updated weights for policy 1, policy_version 657648 (0.0010) [2023-12-26 20:08:56,949][105692] Updated weights for policy 0, policy_version 656812 (0.0009) [2023-12-26 20:08:57,000][105692] Updated weights for policy 0, policy_version 656822 (0.0009) [2023-12-26 20:08:57,056][105692] Updated weights for policy 0, policy_version 656832 (0.0008) [2023-12-26 20:08:57,599][105620] Updated weights for policy 1, policy_version 657658 (0.0010) [2023-12-26 20:08:57,648][105620] Updated weights for policy 1, policy_version 657668 (0.0009) [2023-12-26 20:08:57,699][105620] Updated weights for policy 1, policy_version 657678 (0.0009) [2023-12-26 20:08:57,757][105620] Updated weights for policy 1, policy_version 657688 (0.0009) [2023-12-26 20:08:57,799][105692] Updated weights for policy 0, policy_version 656842 (0.0009) [2023-12-26 20:08:57,849][105692] Updated weights for policy 0, policy_version 656852 (0.0008) [2023-12-26 20:08:57,899][105692] Updated weights for policy 0, policy_version 656862 (0.0009) [2023-12-26 20:08:57,955][105692] Updated weights for policy 0, policy_version 656872 (0.0006) [2023-12-26 20:08:58,612][105620] Updated weights for policy 1, policy_version 657698 (0.0008) [2023-12-26 20:08:58,677][105620] Updated weights for policy 1, policy_version 657708 (0.0010) [2023-12-26 20:08:58,684][105692] Updated weights for policy 0, policy_version 656882 (0.0009) [2023-12-26 20:08:58,746][105620] Updated weights for policy 1, policy_version 657718 (0.0009) [2023-12-26 20:08:58,751][105692] Updated weights for policy 0, policy_version 656892 (0.0009) [2023-12-26 20:08:58,817][105692] Updated weights for policy 0, policy_version 656902 (0.0009) [2023-12-26 20:08:59,544][105692] Updated weights for policy 0, policy_version 656912 (0.0007) [2023-12-26 20:08:59,596][105620] Updated weights for policy 1, policy_version 657728 (0.0008) [2023-12-26 20:08:59,607][105692] Updated weights for policy 0, policy_version 656922 (0.0006) [2023-12-26 20:08:59,653][105620] Updated weights for policy 1, policy_version 657738 (0.0007) [2023-12-26 20:08:59,663][105692] Updated weights for policy 0, policy_version 656932 (0.0006) [2023-12-26 20:08:59,707][105620] Updated weights for policy 1, policy_version 657748 (0.0006) [2023-12-26 20:09:00,335][105692] Updated weights for policy 0, policy_version 656942 (0.0008) [2023-12-26 20:09:00,399][105620] Updated weights for policy 1, policy_version 657758 (0.0007) [2023-12-26 20:09:00,401][105692] Updated weights for policy 0, policy_version 656952 (0.0008) [2023-12-26 20:09:00,452][105692] Updated weights for policy 0, policy_version 656962 (0.0005) [2023-12-26 20:09:00,458][105620] Updated weights for policy 1, policy_version 657768 (0.0008) [2023-12-26 20:09:00,521][105620] Updated weights for policy 1, policy_version 657778 (0.0009) [2023-12-26 20:09:00,994][105692] Updated weights for policy 0, policy_version 656972 (0.0007) [2023-12-26 20:09:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 336625664. Throughput: 0: 9873.7, 1: 9731.5. Samples: 336597880. Policy #0 lag: (min: 13.0, avg: 17.1, max: 45.0) [2023-12-26 20:09:01,063][104569] Avg episode reward: [(0, '9079.445'), (1, '8898.161')] [2023-12-26 20:09:01,065][105692] Updated weights for policy 0, policy_version 656982 (0.0011) [2023-12-26 20:09:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000657784_168411136.pth... [2023-12-26 20:09:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000656664_168124416.pth [2023-12-26 20:09:01,120][105692] Updated weights for policy 0, policy_version 656992 (0.0010) [2023-12-26 20:09:01,172][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000657000_168222720.pth... [2023-12-26 20:09:01,200][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000655816_167919616.pth [2023-12-26 20:09:01,352][105620] Updated weights for policy 1, policy_version 657788 (0.0009) [2023-12-26 20:09:01,414][105620] Updated weights for policy 1, policy_version 657798 (0.0007) [2023-12-26 20:09:01,471][105620] Updated weights for policy 1, policy_version 657808 (0.0006) [2023-12-26 20:09:01,865][105692] Updated weights for policy 0, policy_version 657002 (0.0011) [2023-12-26 20:09:01,916][105692] Updated weights for policy 0, policy_version 657012 (0.0009) [2023-12-26 20:09:01,962][105692] Updated weights for policy 0, policy_version 657022 (0.0008) [2023-12-26 20:09:02,014][105692] Updated weights for policy 0, policy_version 657032 (0.0010) [2023-12-26 20:09:02,125][105620] Updated weights for policy 1, policy_version 657818 (0.0006) [2023-12-26 20:09:02,177][105620] Updated weights for policy 1, policy_version 657828 (0.0009) [2023-12-26 20:09:02,237][105620] Updated weights for policy 1, policy_version 657838 (0.0009) [2023-12-26 20:09:02,297][105620] Updated weights for policy 1, policy_version 657848 (0.0009) [2023-12-26 20:09:02,792][105692] Updated weights for policy 0, policy_version 657042 (0.0009) [2023-12-26 20:09:02,843][105692] Updated weights for policy 0, policy_version 657052 (0.0008) [2023-12-26 20:09:02,889][105692] Updated weights for policy 0, policy_version 657062 (0.0009) [2023-12-26 20:09:03,079][105620] Updated weights for policy 1, policy_version 657858 (0.0008) [2023-12-26 20:09:03,125][105620] Updated weights for policy 1, policy_version 657868 (0.0008) [2023-12-26 20:09:03,180][105620] Updated weights for policy 1, policy_version 657878 (0.0009) [2023-12-26 20:09:03,595][105692] Updated weights for policy 0, policy_version 657072 (0.0006) [2023-12-26 20:09:03,655][105692] Updated weights for policy 0, policy_version 657082 (0.0006) [2023-12-26 20:09:03,716][105692] Updated weights for policy 0, policy_version 657092 (0.0005) [2023-12-26 20:09:04,070][105620] Updated weights for policy 1, policy_version 657888 (0.0010) [2023-12-26 20:09:04,132][105620] Updated weights for policy 1, policy_version 657898 (0.0009) [2023-12-26 20:09:04,192][105620] Updated weights for policy 1, policy_version 657908 (0.0009) [2023-12-26 20:09:04,281][105692] Updated weights for policy 0, policy_version 657102 (0.0008) [2023-12-26 20:09:04,341][105692] Updated weights for policy 0, policy_version 657112 (0.0005) [2023-12-26 20:09:04,405][105692] Updated weights for policy 0, policy_version 657122 (0.0007) [2023-12-26 20:09:04,978][105620] Updated weights for policy 1, policy_version 657918 (0.0010) [2023-12-26 20:09:05,035][105620] Updated weights for policy 1, policy_version 657928 (0.0011) [2023-12-26 20:09:05,040][105692] Updated weights for policy 0, policy_version 657132 (0.0007) [2023-12-26 20:09:05,087][105620] Updated weights for policy 1, policy_version 657938 (0.0010) [2023-12-26 20:09:05,093][105692] Updated weights for policy 0, policy_version 657142 (0.0005) [2023-12-26 20:09:05,155][105692] Updated weights for policy 0, policy_version 657152 (0.0007) [2023-12-26 20:09:05,849][105692] Updated weights for policy 0, policy_version 657162 (0.0008) [2023-12-26 20:09:05,882][105620] Updated weights for policy 1, policy_version 657948 (0.0009) [2023-12-26 20:09:05,912][105692] Updated weights for policy 0, policy_version 657172 (0.0008) [2023-12-26 20:09:05,939][105620] Updated weights for policy 1, policy_version 657958 (0.0005) [2023-12-26 20:09:05,969][105692] Updated weights for policy 0, policy_version 657182 (0.0005) [2023-12-26 20:09:05,986][105620] Updated weights for policy 1, policy_version 657968 (0.0009) [2023-12-26 20:09:06,020][105692] Updated weights for policy 0, policy_version 657192 (0.0005) [2023-12-26 20:09:06,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 336732160. Throughput: 0: 9875.0, 1: 9587.2. Samples: 336713632. Policy #0 lag: (min: 13.0, avg: 17.1, max: 45.0) [2023-12-26 20:09:06,063][104569] Avg episode reward: [(0, '9086.583'), (1, '8806.895')] [2023-12-26 20:09:06,610][105692] Updated weights for policy 0, policy_version 657202 (0.0008) [2023-12-26 20:09:06,672][105692] Updated weights for policy 0, policy_version 657212 (0.0008) [2023-12-26 20:09:06,706][105620] Updated weights for policy 1, policy_version 657978 (0.0010) [2023-12-26 20:09:06,737][105692] Updated weights for policy 0, policy_version 657222 (0.0009) [2023-12-26 20:09:06,766][105620] Updated weights for policy 1, policy_version 657988 (0.0011) [2023-12-26 20:09:06,832][105620] Updated weights for policy 1, policy_version 657998 (0.0010) [2023-12-26 20:09:06,905][105620] Updated weights for policy 1, policy_version 658008 (0.0011) [2023-12-26 20:09:07,418][105692] Updated weights for policy 0, policy_version 657232 (0.0008) [2023-12-26 20:09:07,480][105692] Updated weights for policy 0, policy_version 657242 (0.0008) [2023-12-26 20:09:07,531][105692] Updated weights for policy 0, policy_version 657252 (0.0006) [2023-12-26 20:09:07,647][105620] Updated weights for policy 1, policy_version 658018 (0.0007) [2023-12-26 20:09:07,717][105620] Updated weights for policy 1, policy_version 658028 (0.0007) [2023-12-26 20:09:07,784][105620] Updated weights for policy 1, policy_version 658038 (0.0006) [2023-12-26 20:09:08,287][105692] Updated weights for policy 0, policy_version 657262 (0.0007) [2023-12-26 20:09:08,340][105692] Updated weights for policy 0, policy_version 657272 (0.0008) [2023-12-26 20:09:08,387][105620] Updated weights for policy 1, policy_version 658048 (0.0009) [2023-12-26 20:09:08,404][105692] Updated weights for policy 0, policy_version 657282 (0.0007) [2023-12-26 20:09:08,452][105620] Updated weights for policy 1, policy_version 658058 (0.0010) [2023-12-26 20:09:08,515][105620] Updated weights for policy 1, policy_version 658068 (0.0010) [2023-12-26 20:09:09,084][105692] Updated weights for policy 0, policy_version 657292 (0.0005) [2023-12-26 20:09:09,134][105620] Updated weights for policy 1, policy_version 658078 (0.0007) [2023-12-26 20:09:09,144][105692] Updated weights for policy 0, policy_version 657302 (0.0006) [2023-12-26 20:09:09,197][105620] Updated weights for policy 1, policy_version 658088 (0.0005) [2023-12-26 20:09:09,213][105692] Updated weights for policy 0, policy_version 657312 (0.0007) [2023-12-26 20:09:09,267][105620] Updated weights for policy 1, policy_version 658098 (0.0008) [2023-12-26 20:09:09,920][105620] Updated weights for policy 1, policy_version 658108 (0.0007) [2023-12-26 20:09:09,961][105692] Updated weights for policy 0, policy_version 657322 (0.0010) [2023-12-26 20:09:09,983][105620] Updated weights for policy 1, policy_version 658118 (0.0010) [2023-12-26 20:09:10,018][105692] Updated weights for policy 0, policy_version 657332 (0.0007) [2023-12-26 20:09:10,045][105620] Updated weights for policy 1, policy_version 658128 (0.0010) [2023-12-26 20:09:10,072][105692] Updated weights for policy 0, policy_version 657342 (0.0006) [2023-12-26 20:09:10,131][105692] Updated weights for policy 0, policy_version 657352 (0.0009) [2023-12-26 20:09:10,794][105620] Updated weights for policy 1, policy_version 658138 (0.0011) [2023-12-26 20:09:10,853][105620] Updated weights for policy 1, policy_version 658148 (0.0010) [2023-12-26 20:09:10,899][105692] Updated weights for policy 0, policy_version 657362 (0.0010) [2023-12-26 20:09:10,912][105620] Updated weights for policy 1, policy_version 658158 (0.0010) [2023-12-26 20:09:10,957][105692] Updated weights for policy 0, policy_version 657372 (0.0010) [2023-12-26 20:09:10,963][105620] Updated weights for policy 1, policy_version 658168 (0.0010) [2023-12-26 20:09:11,003][105585] KL-divergence is very high: 103.7927 [2023-12-26 20:09:11,015][105692] Updated weights for policy 0, policy_version 657382 (0.0010) [2023-12-26 20:09:11,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 336830464. Throughput: 0: 9897.0, 1: 9661.1. Samples: 336831712. Policy #0 lag: (min: 13.0, avg: 17.1, max: 45.0) [2023-12-26 20:09:11,062][104569] Avg episode reward: [(0, '8815.740'), (1, '8901.824')] [2023-12-26 20:09:11,681][105620] Updated weights for policy 1, policy_version 658178 (0.0009) [2023-12-26 20:09:11,753][105620] Updated weights for policy 1, policy_version 658188 (0.0009) [2023-12-26 20:09:11,821][105620] Updated weights for policy 1, policy_version 658198 (0.0007) [2023-12-26 20:09:11,823][105692] Updated weights for policy 0, policy_version 657392 (0.0008) [2023-12-26 20:09:11,879][105692] Updated weights for policy 0, policy_version 657402 (0.0008) [2023-12-26 20:09:11,932][105692] Updated weights for policy 0, policy_version 657412 (0.0008) [2023-12-26 20:09:12,539][105620] Updated weights for policy 1, policy_version 658208 (0.0010) [2023-12-26 20:09:12,601][105620] Updated weights for policy 1, policy_version 658218 (0.0011) [2023-12-26 20:09:12,640][105692] Updated weights for policy 0, policy_version 657422 (0.0006) [2023-12-26 20:09:12,663][105620] Updated weights for policy 1, policy_version 658228 (0.0011) [2023-12-26 20:09:12,697][105692] Updated weights for policy 0, policy_version 657432 (0.0007) [2023-12-26 20:09:12,751][105692] Updated weights for policy 0, policy_version 657442 (0.0009) [2023-12-26 20:09:13,198][105620] Updated weights for policy 1, policy_version 658238 (0.0007) [2023-12-26 20:09:13,269][105620] Updated weights for policy 1, policy_version 658248 (0.0010) [2023-12-26 20:09:13,326][105620] Updated weights for policy 1, policy_version 658258 (0.0010) [2023-12-26 20:09:13,608][105692] Updated weights for policy 0, policy_version 657452 (0.0009) [2023-12-26 20:09:13,661][105692] Updated weights for policy 0, policy_version 657462 (0.0010) [2023-12-26 20:09:13,719][105692] Updated weights for policy 0, policy_version 657472 (0.0010) [2023-12-26 20:09:13,924][105620] Updated weights for policy 1, policy_version 658268 (0.0009) [2023-12-26 20:09:13,981][105620] Updated weights for policy 1, policy_version 658278 (0.0008) [2023-12-26 20:09:14,044][105620] Updated weights for policy 1, policy_version 658288 (0.0008) [2023-12-26 20:09:14,481][105692] Updated weights for policy 0, policy_version 657482 (0.0009) [2023-12-26 20:09:14,535][105692] Updated weights for policy 0, policy_version 657492 (0.0010) [2023-12-26 20:09:14,588][105692] Updated weights for policy 0, policy_version 657502 (0.0009) [2023-12-26 20:09:14,643][105692] Updated weights for policy 0, policy_version 657512 (0.0006) [2023-12-26 20:09:14,771][105620] Updated weights for policy 1, policy_version 658298 (0.0008) [2023-12-26 20:09:14,833][105620] Updated weights for policy 1, policy_version 658308 (0.0006) [2023-12-26 20:09:14,892][105620] Updated weights for policy 1, policy_version 658318 (0.0008) [2023-12-26 20:09:14,944][105620] Updated weights for policy 1, policy_version 658328 (0.0008) [2023-12-26 20:09:15,400][105692] Updated weights for policy 0, policy_version 657522 (0.0010) [2023-12-26 20:09:15,451][105692] Updated weights for policy 0, policy_version 657532 (0.0009) [2023-12-26 20:09:15,502][105692] Updated weights for policy 0, policy_version 657542 (0.0008) [2023-12-26 20:09:15,545][105620] Updated weights for policy 1, policy_version 658338 (0.0005) [2023-12-26 20:09:15,608][105620] Updated weights for policy 1, policy_version 658348 (0.0005) [2023-12-26 20:09:15,659][105620] Updated weights for policy 1, policy_version 658358 (0.0005) [2023-12-26 20:09:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 336920576. Throughput: 0: 9836.7, 1: 9642.2. Samples: 336890556. Policy #0 lag: (min: 13.0, avg: 17.1, max: 45.0) [2023-12-26 20:09:16,063][104569] Avg episode reward: [(0, '8898.650'), (1, '8714.343')] [2023-12-26 20:09:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000657544_168361984.pth... [2023-12-26 20:09:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000658360_168558592.pth... [2023-12-26 20:09:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000656392_168067072.pth [2023-12-26 20:09:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000657240_168271872.pth [2023-12-26 20:09:16,170][105620] Updated weights for policy 1, policy_version 658368 (0.0008) [2023-12-26 20:09:16,230][105620] Updated weights for policy 1, policy_version 658378 (0.0009) [2023-12-26 20:09:16,286][105692] Updated weights for policy 0, policy_version 657552 (0.0007) [2023-12-26 20:09:16,292][105620] Updated weights for policy 1, policy_version 658388 (0.0010) [2023-12-26 20:09:16,337][105692] Updated weights for policy 0, policy_version 657562 (0.0007) [2023-12-26 20:09:16,391][105692] Updated weights for policy 0, policy_version 657572 (0.0009) [2023-12-26 20:09:17,069][105620] Updated weights for policy 1, policy_version 658398 (0.0008) [2023-12-26 20:09:17,081][105692] Updated weights for policy 0, policy_version 657582 (0.0009) [2023-12-26 20:09:17,124][105620] Updated weights for policy 1, policy_version 658408 (0.0008) [2023-12-26 20:09:17,140][105692] Updated weights for policy 0, policy_version 657592 (0.0008) [2023-12-26 20:09:17,177][105620] Updated weights for policy 1, policy_version 658418 (0.0009) [2023-12-26 20:09:17,203][105692] Updated weights for policy 0, policy_version 657602 (0.0008) [2023-12-26 20:09:17,761][105620] Updated weights for policy 1, policy_version 658428 (0.0006) [2023-12-26 20:09:17,814][105620] Updated weights for policy 1, policy_version 658438 (0.0007) [2023-12-26 20:09:17,842][105692] Updated weights for policy 0, policy_version 657612 (0.0007) [2023-12-26 20:09:17,873][105620] Updated weights for policy 1, policy_version 658448 (0.0008) [2023-12-26 20:09:17,896][105692] Updated weights for policy 0, policy_version 657622 (0.0009) [2023-12-26 20:09:17,952][105692] Updated weights for policy 0, policy_version 657632 (0.0009) [2023-12-26 20:09:18,470][105620] Updated weights for policy 1, policy_version 658458 (0.0006) [2023-12-26 20:09:18,528][105620] Updated weights for policy 1, policy_version 658468 (0.0008) [2023-12-26 20:09:18,575][105620] Updated weights for policy 1, policy_version 658478 (0.0008) [2023-12-26 20:09:18,634][105620] Updated weights for policy 1, policy_version 658488 (0.0006) [2023-12-26 20:09:18,795][105692] Updated weights for policy 0, policy_version 657642 (0.0009) [2023-12-26 20:09:18,866][105692] Updated weights for policy 0, policy_version 657652 (0.0006) [2023-12-26 20:09:18,935][105692] Updated weights for policy 0, policy_version 657662 (0.0009) [2023-12-26 20:09:18,998][105692] Updated weights for policy 0, policy_version 657672 (0.0010) [2023-12-26 20:09:19,289][105620] Updated weights for policy 1, policy_version 658498 (0.0008) [2023-12-26 20:09:19,351][105620] Updated weights for policy 1, policy_version 658508 (0.0008) [2023-12-26 20:09:19,419][105620] Updated weights for policy 1, policy_version 658518 (0.0008) [2023-12-26 20:09:19,672][105692] Updated weights for policy 0, policy_version 657682 (0.0008) [2023-12-26 20:09:19,743][105692] Updated weights for policy 0, policy_version 657692 (0.0009) [2023-12-26 20:09:19,807][105692] Updated weights for policy 0, policy_version 657702 (0.0009) [2023-12-26 20:09:20,135][105620] Updated weights for policy 1, policy_version 658528 (0.0008) [2023-12-26 20:09:20,203][105620] Updated weights for policy 1, policy_version 658538 (0.0008) [2023-12-26 20:09:20,263][105620] Updated weights for policy 1, policy_version 658548 (0.0011) [2023-12-26 20:09:20,528][105692] Updated weights for policy 0, policy_version 657712 (0.0007) [2023-12-26 20:09:20,596][105692] Updated weights for policy 0, policy_version 657722 (0.0011) [2023-12-26 20:09:20,655][105692] Updated weights for policy 0, policy_version 657732 (0.0010) [2023-12-26 20:09:21,009][105620] Updated weights for policy 1, policy_version 658558 (0.0009) [2023-12-26 20:09:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 337018880. Throughput: 0: 9742.3, 1: 9780.6. Samples: 337011372. Policy #0 lag: (min: 13.0, avg: 17.1, max: 45.0) [2023-12-26 20:09:21,062][104569] Avg episode reward: [(0, '8991.606'), (1, '8803.580')] [2023-12-26 20:09:21,089][105620] Updated weights for policy 1, policy_version 658568 (0.0009) [2023-12-26 20:09:21,157][105620] Updated weights for policy 1, policy_version 658578 (0.0009) [2023-12-26 20:09:21,432][105692] Updated weights for policy 0, policy_version 657742 (0.0011) [2023-12-26 20:09:21,499][105692] Updated weights for policy 0, policy_version 657752 (0.0011) [2023-12-26 20:09:21,561][105692] Updated weights for policy 0, policy_version 657762 (0.0011) [2023-12-26 20:09:21,949][105620] Updated weights for policy 1, policy_version 658588 (0.0008) [2023-12-26 20:09:22,013][105620] Updated weights for policy 1, policy_version 658598 (0.0009) [2023-12-26 20:09:22,066][105620] Updated weights for policy 1, policy_version 658608 (0.0008) [2023-12-26 20:09:22,328][105692] Updated weights for policy 0, policy_version 657772 (0.0010) [2023-12-26 20:09:22,393][105692] Updated weights for policy 0, policy_version 657782 (0.0011) [2023-12-26 20:09:22,460][105692] Updated weights for policy 0, policy_version 657792 (0.0011) [2023-12-26 20:09:22,837][105620] Updated weights for policy 1, policy_version 658618 (0.0008) [2023-12-26 20:09:22,896][105620] Updated weights for policy 1, policy_version 658628 (0.0008) [2023-12-26 20:09:22,965][105620] Updated weights for policy 1, policy_version 658638 (0.0009) [2023-12-26 20:09:23,030][105620] Updated weights for policy 1, policy_version 658648 (0.0008) [2023-12-26 20:09:23,230][105692] Updated weights for policy 0, policy_version 657802 (0.0011) [2023-12-26 20:09:23,282][105692] Updated weights for policy 0, policy_version 657812 (0.0011) [2023-12-26 20:09:23,334][105692] Updated weights for policy 0, policy_version 657822 (0.0011) [2023-12-26 20:09:23,383][105692] Updated weights for policy 0, policy_version 657832 (0.0010) [2023-12-26 20:09:23,771][105620] Updated weights for policy 1, policy_version 658658 (0.0008) [2023-12-26 20:09:23,820][105620] Updated weights for policy 1, policy_version 658668 (0.0008) [2023-12-26 20:09:23,872][105620] Updated weights for policy 1, policy_version 658678 (0.0008) [2023-12-26 20:09:24,143][105692] Updated weights for policy 0, policy_version 657842 (0.0010) [2023-12-26 20:09:24,192][105692] Updated weights for policy 0, policy_version 657852 (0.0010) [2023-12-26 20:09:24,247][105692] Updated weights for policy 0, policy_version 657862 (0.0010) [2023-12-26 20:09:24,606][105620] Updated weights for policy 1, policy_version 658688 (0.0008) [2023-12-26 20:09:24,672][105620] Updated weights for policy 1, policy_version 658698 (0.0008) [2023-12-26 20:09:24,730][105620] Updated weights for policy 1, policy_version 658708 (0.0009) [2023-12-26 20:09:24,942][105692] Updated weights for policy 0, policy_version 657872 (0.0008) [2023-12-26 20:09:25,000][105692] Updated weights for policy 0, policy_version 657882 (0.0008) [2023-12-26 20:09:25,060][105692] Updated weights for policy 0, policy_version 657892 (0.0008) [2023-12-26 20:09:25,438][105620] Updated weights for policy 1, policy_version 658718 (0.0007) [2023-12-26 20:09:25,484][105620] Updated weights for policy 1, policy_version 658728 (0.0005) [2023-12-26 20:09:25,535][105620] Updated weights for policy 1, policy_version 658738 (0.0005) [2023-12-26 20:09:25,847][105692] Updated weights for policy 0, policy_version 657902 (0.0008) [2023-12-26 20:09:25,899][105692] Updated weights for policy 0, policy_version 657912 (0.0007) [2023-12-26 20:09:25,955][105692] Updated weights for policy 0, policy_version 657922 (0.0006) [2023-12-26 20:09:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 337117184. Throughput: 0: 9706.9, 1: 9674.5. Samples: 337124312. Policy #0 lag: (min: 13.0, avg: 17.1, max: 45.0) [2023-12-26 20:09:26,063][104569] Avg episode reward: [(0, '9082.078'), (1, '8895.044')] [2023-12-26 20:09:26,154][105620] Updated weights for policy 1, policy_version 658748 (0.0007) [2023-12-26 20:09:26,203][105620] Updated weights for policy 1, policy_version 658758 (0.0010) [2023-12-26 20:09:26,252][105620] Updated weights for policy 1, policy_version 658768 (0.0010) [2023-12-26 20:09:26,562][105692] Updated weights for policy 0, policy_version 657932 (0.0005) [2023-12-26 20:09:26,622][105692] Updated weights for policy 0, policy_version 657942 (0.0008) [2023-12-26 20:09:26,674][105692] Updated weights for policy 0, policy_version 657953 (0.0010) [2023-12-26 20:09:26,826][105620] Updated weights for policy 1, policy_version 658778 (0.0009) [2023-12-26 20:09:26,876][105620] Updated weights for policy 1, policy_version 658788 (0.0007) [2023-12-26 20:09:26,927][105620] Updated weights for policy 1, policy_version 658798 (0.0007) [2023-12-26 20:09:26,972][105620] Updated weights for policy 1, policy_version 658808 (0.0005) [2023-12-26 20:09:27,394][105692] Updated weights for policy 0, policy_version 657963 (0.0009) [2023-12-26 20:09:27,455][105692] Updated weights for policy 0, policy_version 657974 (0.0010) [2023-12-26 20:09:27,508][105692] Updated weights for policy 0, policy_version 657985 (0.0010) [2023-12-26 20:09:27,556][105620] Updated weights for policy 1, policy_version 658818 (0.0005) [2023-12-26 20:09:27,626][105620] Updated weights for policy 1, policy_version 658828 (0.0006) [2023-12-26 20:09:27,674][105620] Updated weights for policy 1, policy_version 658838 (0.0010) [2023-12-26 20:09:28,174][105692] Updated weights for policy 0, policy_version 657995 (0.0007) [2023-12-26 20:09:28,226][105692] Updated weights for policy 0, policy_version 658005 (0.0008) [2023-12-26 20:09:28,277][105692] Updated weights for policy 0, policy_version 658015 (0.0008) [2023-12-26 20:09:28,389][105620] Updated weights for policy 1, policy_version 658848 (0.0010) [2023-12-26 20:09:28,457][105620] Updated weights for policy 1, policy_version 658858 (0.0011) [2023-12-26 20:09:28,522][105620] Updated weights for policy 1, policy_version 658868 (0.0010) [2023-12-26 20:09:28,992][105692] Updated weights for policy 0, policy_version 658025 (0.0008) [2023-12-26 20:09:29,050][105692] Updated weights for policy 0, policy_version 658035 (0.0008) [2023-12-26 20:09:29,109][105692] Updated weights for policy 0, policy_version 658045 (0.0008) [2023-12-26 20:09:29,176][105692] Updated weights for policy 0, policy_version 658055 (0.0010) [2023-12-26 20:09:29,209][105620] Updated weights for policy 1, policy_version 658878 (0.0007) [2023-12-26 20:09:29,277][105620] Updated weights for policy 1, policy_version 658888 (0.0009) [2023-12-26 20:09:29,343][105620] Updated weights for policy 1, policy_version 658898 (0.0010) [2023-12-26 20:09:29,901][105620] Updated weights for policy 1, policy_version 658908 (0.0009) [2023-12-26 20:09:29,964][105620] Updated weights for policy 1, policy_version 658918 (0.0009) [2023-12-26 20:09:30,002][105692] Updated weights for policy 0, policy_version 658065 (0.0007) [2023-12-26 20:09:30,026][105620] Updated weights for policy 1, policy_version 658928 (0.0010) [2023-12-26 20:09:30,064][105692] Updated weights for policy 0, policy_version 658075 (0.0009) [2023-12-26 20:09:30,121][105692] Updated weights for policy 0, policy_version 658085 (0.0008) [2023-12-26 20:09:30,675][105620] Updated weights for policy 1, policy_version 658938 (0.0010) [2023-12-26 20:09:30,732][105620] Updated weights for policy 1, policy_version 658948 (0.0008) [2023-12-26 20:09:30,798][105620] Updated weights for policy 1, policy_version 658958 (0.0008) [2023-12-26 20:09:30,851][105620] Updated weights for policy 1, policy_version 658968 (0.0006) [2023-12-26 20:09:30,853][105692] Updated weights for policy 0, policy_version 658095 (0.0008) [2023-12-26 20:09:30,899][105692] Updated weights for policy 0, policy_version 658105 (0.0009) [2023-12-26 20:09:30,948][105692] Updated weights for policy 0, policy_version 658115 (0.0009) [2023-12-26 20:09:31,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 337223680. Throughput: 0: 9802.9, 1: 9736.8. Samples: 337187252. Policy #0 lag: (min: 13.0, avg: 17.1, max: 45.0) [2023-12-26 20:09:31,063][104569] Avg episode reward: [(0, '9082.449'), (1, '8710.575')] [2023-12-26 20:09:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000658120_168509440.pth... [2023-12-26 20:09:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000658968_168714240.pth... [2023-12-26 20:09:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000657784_168411136.pth [2023-12-26 20:09:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000657000_168222720.pth [2023-12-26 20:09:31,591][105692] Updated weights for policy 0, policy_version 658125 (0.0010) [2023-12-26 20:09:31,634][105620] Updated weights for policy 1, policy_version 658978 (0.0007) [2023-12-26 20:09:31,649][105692] Updated weights for policy 0, policy_version 658135 (0.0009) [2023-12-26 20:09:31,697][105620] Updated weights for policy 1, policy_version 658988 (0.0007) [2023-12-26 20:09:31,704][105692] Updated weights for policy 0, policy_version 658145 (0.0006) [2023-12-26 20:09:31,758][105620] Updated weights for policy 1, policy_version 658998 (0.0008) [2023-12-26 20:09:32,358][105692] Updated weights for policy 0, policy_version 658155 (0.0007) [2023-12-26 20:09:32,422][105692] Updated weights for policy 0, policy_version 658165 (0.0007) [2023-12-26 20:09:32,473][105692] Updated weights for policy 0, policy_version 658175 (0.0009) [2023-12-26 20:09:32,572][105620] Updated weights for policy 1, policy_version 659008 (0.0009) [2023-12-26 20:09:32,636][105620] Updated weights for policy 1, policy_version 659018 (0.0010) [2023-12-26 20:09:32,696][105620] Updated weights for policy 1, policy_version 659028 (0.0009) [2023-12-26 20:09:33,113][105692] Updated weights for policy 0, policy_version 658185 (0.0008) [2023-12-26 20:09:33,164][105692] Updated weights for policy 0, policy_version 658195 (0.0005) [2023-12-26 20:09:33,212][105692] Updated weights for policy 0, policy_version 658205 (0.0005) [2023-12-26 20:09:33,274][105692] Updated weights for policy 0, policy_version 658215 (0.0005) [2023-12-26 20:09:33,536][105620] Updated weights for policy 1, policy_version 659038 (0.0009) [2023-12-26 20:09:33,593][105620] Updated weights for policy 1, policy_version 659048 (0.0009) [2023-12-26 20:09:33,657][105620] Updated weights for policy 1, policy_version 659058 (0.0008) [2023-12-26 20:09:33,919][105692] Updated weights for policy 0, policy_version 658225 (0.0009) [2023-12-26 20:09:33,976][105692] Updated weights for policy 0, policy_version 658235 (0.0009) [2023-12-26 20:09:34,036][105692] Updated weights for policy 0, policy_version 658245 (0.0008) [2023-12-26 20:09:34,257][105620] Updated weights for policy 1, policy_version 659068 (0.0008) [2023-12-26 20:09:34,318][105620] Updated weights for policy 1, policy_version 659078 (0.0006) [2023-12-26 20:09:34,381][105620] Updated weights for policy 1, policy_version 659088 (0.0005) [2023-12-26 20:09:34,908][105692] Updated weights for policy 0, policy_version 658255 (0.0009) [2023-12-26 20:09:34,962][105620] Updated weights for policy 1, policy_version 659098 (0.0005) [2023-12-26 20:09:34,964][105692] Updated weights for policy 0, policy_version 658265 (0.0010) [2023-12-26 20:09:35,014][105620] Updated weights for policy 1, policy_version 659108 (0.0006) [2023-12-26 20:09:35,015][105692] Updated weights for policy 0, policy_version 658275 (0.0008) [2023-12-26 20:09:35,076][105620] Updated weights for policy 1, policy_version 659118 (0.0005) [2023-12-26 20:09:35,142][105620] Updated weights for policy 1, policy_version 659128 (0.0006) [2023-12-26 20:09:35,680][105692] Updated weights for policy 0, policy_version 658285 (0.0009) [2023-12-26 20:09:35,686][105620] Updated weights for policy 1, policy_version 659138 (0.0006) [2023-12-26 20:09:35,741][105620] Updated weights for policy 1, policy_version 659148 (0.0005) [2023-12-26 20:09:35,742][105692] Updated weights for policy 0, policy_version 658295 (0.0009) [2023-12-26 20:09:35,798][105620] Updated weights for policy 1, policy_version 659158 (0.0005) [2023-12-26 20:09:35,803][105692] Updated weights for policy 0, policy_version 658305 (0.0009) [2023-12-26 20:09:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 337321984. Throughput: 0: 9808.7, 1: 9772.9. Samples: 337305524. Policy #0 lag: (min: 13.0, avg: 17.1, max: 45.0) [2023-12-26 20:09:36,062][104569] Avg episode reward: [(0, '9169.210'), (1, '8985.053')] [2023-12-26 20:09:36,479][105620] Updated weights for policy 1, policy_version 659168 (0.0010) [2023-12-26 20:09:36,539][105620] Updated weights for policy 1, policy_version 659178 (0.0011) [2023-12-26 20:09:36,589][105692] Updated weights for policy 0, policy_version 658315 (0.0009) [2023-12-26 20:09:36,604][105620] Updated weights for policy 1, policy_version 659188 (0.0011) [2023-12-26 20:09:36,642][105692] Updated weights for policy 0, policy_version 658325 (0.0011) [2023-12-26 20:09:36,694][105692] Updated weights for policy 0, policy_version 658335 (0.0011) [2023-12-26 20:09:37,297][105620] Updated weights for policy 1, policy_version 659198 (0.0007) [2023-12-26 20:09:37,356][105620] Updated weights for policy 1, policy_version 659208 (0.0005) [2023-12-26 20:09:37,361][105692] Updated weights for policy 0, policy_version 658345 (0.0007) [2023-12-26 20:09:37,405][105692] Updated weights for policy 0, policy_version 658355 (0.0005) [2023-12-26 20:09:37,422][105620] Updated weights for policy 1, policy_version 659218 (0.0005) [2023-12-26 20:09:37,458][105692] Updated weights for policy 0, policy_version 658365 (0.0006) [2023-12-26 20:09:37,505][105692] Updated weights for policy 0, policy_version 658375 (0.0006) [2023-12-26 20:09:37,970][105620] Updated weights for policy 1, policy_version 659228 (0.0009) [2023-12-26 20:09:38,029][105620] Updated weights for policy 1, policy_version 659238 (0.0007) [2023-12-26 20:09:38,092][105620] Updated weights for policy 1, policy_version 659248 (0.0006) [2023-12-26 20:09:38,260][105692] Updated weights for policy 0, policy_version 658385 (0.0010) [2023-12-26 20:09:38,324][105692] Updated weights for policy 0, policy_version 658395 (0.0009) [2023-12-26 20:09:38,387][105692] Updated weights for policy 0, policy_version 658405 (0.0009) [2023-12-26 20:09:38,853][105620] Updated weights for policy 1, policy_version 659258 (0.0007) [2023-12-26 20:09:38,919][105620] Updated weights for policy 1, policy_version 659268 (0.0006) [2023-12-26 20:09:38,975][105620] Updated weights for policy 1, policy_version 659278 (0.0006) [2023-12-26 20:09:39,036][105620] Updated weights for policy 1, policy_version 659288 (0.0005) [2023-12-26 20:09:39,108][105692] Updated weights for policy 0, policy_version 658415 (0.0010) [2023-12-26 20:09:39,159][105692] Updated weights for policy 0, policy_version 658425 (0.0010) [2023-12-26 20:09:39,215][105692] Updated weights for policy 0, policy_version 658435 (0.0010) [2023-12-26 20:09:39,721][105620] Updated weights for policy 1, policy_version 659298 (0.0009) [2023-12-26 20:09:39,783][105620] Updated weights for policy 1, policy_version 659308 (0.0008) [2023-12-26 20:09:39,854][105620] Updated weights for policy 1, policy_version 659318 (0.0008) [2023-12-26 20:09:39,989][105692] Updated weights for policy 0, policy_version 658445 (0.0008) [2023-12-26 20:09:40,058][105692] Updated weights for policy 0, policy_version 658455 (0.0010) [2023-12-26 20:09:40,127][105692] Updated weights for policy 0, policy_version 658465 (0.0009) [2023-12-26 20:09:40,525][105620] Updated weights for policy 1, policy_version 659328 (0.0008) [2023-12-26 20:09:40,575][105620] Updated weights for policy 1, policy_version 659338 (0.0007) [2023-12-26 20:09:40,627][105620] Updated weights for policy 1, policy_version 659348 (0.0008) [2023-12-26 20:09:40,845][105692] Updated weights for policy 0, policy_version 658475 (0.0008) [2023-12-26 20:09:40,893][105692] Updated weights for policy 0, policy_version 658485 (0.0010) [2023-12-26 20:09:40,944][105692] Updated weights for policy 0, policy_version 658495 (0.0010) [2023-12-26 20:09:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 337420288. Throughput: 0: 9747.0, 1: 9905.3. Samples: 337425072. Policy #0 lag: (min: 13.0, avg: 17.1, max: 45.0) [2023-12-26 20:09:41,063][104569] Avg episode reward: [(0, '7984.049'), (1, '9078.378')] [2023-12-26 20:09:41,464][105620] Updated weights for policy 1, policy_version 659358 (0.0007) [2023-12-26 20:09:41,524][105620] Updated weights for policy 1, policy_version 659368 (0.0008) [2023-12-26 20:09:41,582][105620] Updated weights for policy 1, policy_version 659378 (0.0006) [2023-12-26 20:09:41,720][105692] Updated weights for policy 0, policy_version 658505 (0.0010) [2023-12-26 20:09:41,780][105692] Updated weights for policy 0, policy_version 658515 (0.0008) [2023-12-26 20:09:41,843][105692] Updated weights for policy 0, policy_version 658525 (0.0007) [2023-12-26 20:09:41,901][105692] Updated weights for policy 0, policy_version 658535 (0.0006) [2023-12-26 20:09:42,406][105620] Updated weights for policy 1, policy_version 659388 (0.0007) [2023-12-26 20:09:42,463][105620] Updated weights for policy 1, policy_version 659398 (0.0008) [2023-12-26 20:09:42,535][105620] Updated weights for policy 1, policy_version 659408 (0.0009) [2023-12-26 20:09:42,646][105692] Updated weights for policy 0, policy_version 658545 (0.0009) [2023-12-26 20:09:42,704][105692] Updated weights for policy 0, policy_version 658555 (0.0009) [2023-12-26 20:09:42,771][105692] Updated weights for policy 0, policy_version 658565 (0.0009) [2023-12-26 20:09:43,343][105620] Updated weights for policy 1, policy_version 659418 (0.0009) [2023-12-26 20:09:43,401][105620] Updated weights for policy 1, policy_version 659428 (0.0010) [2023-12-26 20:09:43,409][105692] Updated weights for policy 0, policy_version 658575 (0.0007) [2023-12-26 20:09:43,460][105620] Updated weights for policy 1, policy_version 659438 (0.0008) [2023-12-26 20:09:43,465][105692] Updated weights for policy 0, policy_version 658585 (0.0005) [2023-12-26 20:09:43,516][105692] Updated weights for policy 0, policy_version 658595 (0.0006) [2023-12-26 20:09:43,517][105620] Updated weights for policy 1, policy_version 659448 (0.0009) [2023-12-26 20:09:44,211][105692] Updated weights for policy 0, policy_version 658605 (0.0009) [2023-12-26 20:09:44,271][105692] Updated weights for policy 0, policy_version 658615 (0.0009) [2023-12-26 20:09:44,291][105620] Updated weights for policy 1, policy_version 659458 (0.0006) [2023-12-26 20:09:44,322][105692] Updated weights for policy 0, policy_version 658625 (0.0007) [2023-12-26 20:09:44,352][105620] Updated weights for policy 1, policy_version 659468 (0.0007) [2023-12-26 20:09:44,399][105620] Updated weights for policy 1, policy_version 659478 (0.0008) [2023-12-26 20:09:44,928][105692] Updated weights for policy 0, policy_version 658635 (0.0007) [2023-12-26 20:09:44,992][105692] Updated weights for policy 0, policy_version 658645 (0.0009) [2023-12-26 20:09:45,051][105692] Updated weights for policy 0, policy_version 658655 (0.0008) [2023-12-26 20:09:45,276][105620] Updated weights for policy 1, policy_version 659488 (0.0008) [2023-12-26 20:09:45,341][105620] Updated weights for policy 1, policy_version 659498 (0.0010) [2023-12-26 20:09:45,407][105620] Updated weights for policy 1, policy_version 659508 (0.0010) [2023-12-26 20:09:45,750][105692] Updated weights for policy 0, policy_version 658665 (0.0009) [2023-12-26 20:09:45,808][105692] Updated weights for policy 0, policy_version 658675 (0.0010) [2023-12-26 20:09:45,859][105692] Updated weights for policy 0, policy_version 658685 (0.0008) [2023-12-26 20:09:45,911][105692] Updated weights for policy 0, policy_version 658695 (0.0009) [2023-12-26 20:09:46,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 337510400. Throughput: 0: 9711.9, 1: 9907.5. Samples: 337480756. Policy #0 lag: (min: 13.0, avg: 17.1, max: 45.0) [2023-12-26 20:09:46,063][104569] Avg episode reward: [(0, '8166.789'), (1, '9078.023')] [2023-12-26 20:09:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000658696_168656896.pth... [2023-12-26 20:09:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000657544_168361984.pth [2023-12-26 20:09:46,098][105620] Updated weights for policy 1, policy_version 659518 (0.0009) [2023-12-26 20:09:46,162][105620] Updated weights for policy 1, policy_version 659528 (0.0009) [2023-12-26 20:09:46,221][105620] Updated weights for policy 1, policy_version 659538 (0.0006) [2023-12-26 20:09:46,252][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000659544_168861696.pth... [2023-12-26 20:09:46,256][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000658360_168558592.pth [2023-12-26 20:09:46,632][105692] Updated weights for policy 0, policy_version 658705 (0.0009) [2023-12-26 20:09:46,700][105692] Updated weights for policy 0, policy_version 658715 (0.0008) [2023-12-26 20:09:46,769][105692] Updated weights for policy 0, policy_version 658725 (0.0009) [2023-12-26 20:09:46,872][105620] Updated weights for policy 1, policy_version 659548 (0.0007) [2023-12-26 20:09:46,936][105620] Updated weights for policy 1, policy_version 659558 (0.0010) [2023-12-26 20:09:46,995][105620] Updated weights for policy 1, policy_version 659568 (0.0009) [2023-12-26 20:09:47,440][105692] Updated weights for policy 0, policy_version 658735 (0.0010) [2023-12-26 20:09:47,498][105692] Updated weights for policy 0, policy_version 658745 (0.0009) [2023-12-26 20:09:47,550][105692] Updated weights for policy 0, policy_version 658755 (0.0010) [2023-12-26 20:09:47,650][105620] Updated weights for policy 1, policy_version 659578 (0.0009) [2023-12-26 20:09:47,701][105620] Updated weights for policy 1, policy_version 659588 (0.0009) [2023-12-26 20:09:47,758][105620] Updated weights for policy 1, policy_version 659598 (0.0009) [2023-12-26 20:09:47,820][105620] Updated weights for policy 1, policy_version 659608 (0.0009) [2023-12-26 20:09:48,357][105692] Updated weights for policy 0, policy_version 658765 (0.0009) [2023-12-26 20:09:48,418][105692] Updated weights for policy 0, policy_version 658775 (0.0008) [2023-12-26 20:09:48,489][105692] Updated weights for policy 0, policy_version 658785 (0.0006) [2023-12-26 20:09:48,545][105620] Updated weights for policy 1, policy_version 659618 (0.0008) [2023-12-26 20:09:48,605][105620] Updated weights for policy 1, policy_version 659628 (0.0008) [2023-12-26 20:09:48,672][105620] Updated weights for policy 1, policy_version 659638 (0.0008) [2023-12-26 20:09:49,167][105692] Updated weights for policy 0, policy_version 658795 (0.0008) [2023-12-26 20:09:49,229][105692] Updated weights for policy 0, policy_version 658805 (0.0009) [2023-12-26 20:09:49,284][105692] Updated weights for policy 0, policy_version 658815 (0.0008) [2023-12-26 20:09:49,387][105620] Updated weights for policy 1, policy_version 659648 (0.0007) [2023-12-26 20:09:49,452][105620] Updated weights for policy 1, policy_version 659658 (0.0009) [2023-12-26 20:09:49,506][105620] Updated weights for policy 1, policy_version 659668 (0.0009) [2023-12-26 20:09:50,035][105692] Updated weights for policy 0, policy_version 658825 (0.0009) [2023-12-26 20:09:50,085][105692] Updated weights for policy 0, policy_version 658835 (0.0009) [2023-12-26 20:09:50,136][105692] Updated weights for policy 0, policy_version 658845 (0.0009) [2023-12-26 20:09:50,193][105692] Updated weights for policy 0, policy_version 658855 (0.0009) [2023-12-26 20:09:50,256][105620] Updated weights for policy 1, policy_version 659678 (0.0008) [2023-12-26 20:09:50,317][105620] Updated weights for policy 1, policy_version 659688 (0.0009) [2023-12-26 20:09:50,384][105620] Updated weights for policy 1, policy_version 659698 (0.0009) [2023-12-26 20:09:50,967][105692] Updated weights for policy 0, policy_version 658865 (0.0009) [2023-12-26 20:09:51,026][105692] Updated weights for policy 0, policy_version 658875 (0.0009) [2023-12-26 20:09:51,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 337600512. Throughput: 0: 9668.6, 1: 9968.0. Samples: 337597276. Policy #0 lag: (min: 13.0, avg: 17.1, max: 45.0) [2023-12-26 20:09:51,063][104569] Avg episode reward: [(0, '9261.027'), (1, '8985.176')] [2023-12-26 20:09:51,090][105692] Updated weights for policy 0, policy_version 658885 (0.0010) [2023-12-26 20:09:51,142][105620] Updated weights for policy 1, policy_version 659708 (0.0009) [2023-12-26 20:09:51,203][105620] Updated weights for policy 1, policy_version 659718 (0.0007) [2023-12-26 20:09:51,270][105620] Updated weights for policy 1, policy_version 659728 (0.0006) [2023-12-26 20:09:51,874][105620] Updated weights for policy 1, policy_version 659738 (0.0006) [2023-12-26 20:09:51,929][105620] Updated weights for policy 1, policy_version 659748 (0.0005) [2023-12-26 20:09:51,936][105586] KL-divergence is very high: 288.2197 [2023-12-26 20:09:51,956][105692] Updated weights for policy 0, policy_version 658895 (0.0009) [2023-12-26 20:09:51,986][105586] KL-divergence is very high: 289.5409 [2023-12-26 20:09:51,992][105620] Updated weights for policy 1, policy_version 659758 (0.0005) [2023-12-26 20:09:52,021][105692] Updated weights for policy 0, policy_version 658905 (0.0008) [2023-12-26 20:09:52,037][105586] KL-divergence is very high: 114.3043 [2023-12-26 20:09:52,056][105620] Updated weights for policy 1, policy_version 659768 (0.0008) [2023-12-26 20:09:52,076][105692] Updated weights for policy 0, policy_version 658915 (0.0007) [2023-12-26 20:09:52,706][105620] Updated weights for policy 1, policy_version 659778 (0.0009) [2023-12-26 20:09:52,765][105620] Updated weights for policy 1, policy_version 659788 (0.0007) [2023-12-26 20:09:52,830][105620] Updated weights for policy 1, policy_version 659798 (0.0006) [2023-12-26 20:09:52,934][105692] Updated weights for policy 0, policy_version 658925 (0.0009) [2023-12-26 20:09:52,991][105692] Updated weights for policy 0, policy_version 658935 (0.0010) [2023-12-26 20:09:53,044][105692] Updated weights for policy 0, policy_version 658945 (0.0009) [2023-12-26 20:09:53,480][105620] Updated weights for policy 1, policy_version 659808 (0.0005) [2023-12-26 20:09:53,545][105620] Updated weights for policy 1, policy_version 659818 (0.0006) [2023-12-26 20:09:53,598][105620] Updated weights for policy 1, policy_version 659828 (0.0005) [2023-12-26 20:09:53,747][105692] Updated weights for policy 0, policy_version 658955 (0.0008) [2023-12-26 20:09:53,802][105692] Updated weights for policy 0, policy_version 658965 (0.0005) [2023-12-26 20:09:53,848][105692] Updated weights for policy 0, policy_version 658975 (0.0005) [2023-12-26 20:09:54,329][105620] Updated weights for policy 1, policy_version 659838 (0.0007) [2023-12-26 20:09:54,387][105620] Updated weights for policy 1, policy_version 659848 (0.0010) [2023-12-26 20:09:54,447][105620] Updated weights for policy 1, policy_version 659858 (0.0008) [2023-12-26 20:09:54,477][105692] Updated weights for policy 0, policy_version 658985 (0.0005) [2023-12-26 20:09:54,546][105692] Updated weights for policy 0, policy_version 658995 (0.0009) [2023-12-26 20:09:54,600][105692] Updated weights for policy 0, policy_version 659005 (0.0007) [2023-12-26 20:09:54,660][105692] Updated weights for policy 0, policy_version 659015 (0.0009) [2023-12-26 20:09:55,194][105620] Updated weights for policy 1, policy_version 659868 (0.0008) [2023-12-26 20:09:55,243][105620] Updated weights for policy 1, policy_version 659878 (0.0007) [2023-12-26 20:09:55,287][105620] Updated weights for policy 1, policy_version 659888 (0.0005) [2023-12-26 20:09:55,382][105692] Updated weights for policy 0, policy_version 659025 (0.0009) [2023-12-26 20:09:55,440][105692] Updated weights for policy 0, policy_version 659035 (0.0009) [2023-12-26 20:09:55,490][105692] Updated weights for policy 0, policy_version 659045 (0.0008) [2023-12-26 20:09:55,930][105620] Updated weights for policy 1, policy_version 659898 (0.0006) [2023-12-26 20:09:55,982][105620] Updated weights for policy 1, policy_version 659908 (0.0009) [2023-12-26 20:09:56,037][105620] Updated weights for policy 1, policy_version 659918 (0.0009) [2023-12-26 20:09:56,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 337698816. Throughput: 0: 9578.1, 1: 10009.0. Samples: 337713136. Policy #0 lag: (min: 13.0, avg: 17.1, max: 45.0) [2023-12-26 20:09:56,063][104569] Avg episode reward: [(0, '9260.873'), (1, '9074.951')] [2023-12-26 20:09:56,094][105620] Updated weights for policy 1, policy_version 659928 (0.0009) [2023-12-26 20:09:56,246][105692] Updated weights for policy 0, policy_version 659055 (0.0007) [2023-12-26 20:09:56,310][105692] Updated weights for policy 0, policy_version 659065 (0.0007) [2023-12-26 20:09:56,365][105692] Updated weights for policy 0, policy_version 659075 (0.0007) [2023-12-26 20:09:56,831][105620] Updated weights for policy 1, policy_version 659938 (0.0010) [2023-12-26 20:09:56,880][105620] Updated weights for policy 1, policy_version 659948 (0.0010) [2023-12-26 20:09:56,908][105692] Updated weights for policy 0, policy_version 659085 (0.0006) [2023-12-26 20:09:56,924][105620] Updated weights for policy 1, policy_version 659958 (0.0010) [2023-12-26 20:09:56,951][105692] Updated weights for policy 0, policy_version 659095 (0.0005) [2023-12-26 20:09:57,000][105692] Updated weights for policy 0, policy_version 659105 (0.0005) [2023-12-26 20:09:57,585][105692] Updated weights for policy 0, policy_version 659115 (0.0006) [2023-12-26 20:09:57,642][105692] Updated weights for policy 0, policy_version 659125 (0.0005) [2023-12-26 20:09:57,687][105692] Updated weights for policy 0, policy_version 659135 (0.0005) [2023-12-26 20:09:57,695][105620] Updated weights for policy 1, policy_version 659968 (0.0010) [2023-12-26 20:09:57,747][105620] Updated weights for policy 1, policy_version 659978 (0.0010) [2023-12-26 20:09:57,799][105620] Updated weights for policy 1, policy_version 659988 (0.0010) [2023-12-26 20:09:58,274][105692] Updated weights for policy 0, policy_version 659145 (0.0006) [2023-12-26 20:09:58,339][105692] Updated weights for policy 0, policy_version 659156 (0.0007) [2023-12-26 20:09:58,407][105692] Updated weights for policy 0, policy_version 659166 (0.0008) [2023-12-26 20:09:58,471][105692] Updated weights for policy 0, policy_version 659176 (0.0007) [2023-12-26 20:09:58,586][105620] Updated weights for policy 1, policy_version 659998 (0.0011) [2023-12-26 20:09:58,651][105620] Updated weights for policy 1, policy_version 660008 (0.0009) [2023-12-26 20:09:58,712][105620] Updated weights for policy 1, policy_version 660018 (0.0009) [2023-12-26 20:09:59,426][105692] Updated weights for policy 0, policy_version 659186 (0.0008) [2023-12-26 20:09:59,488][105692] Updated weights for policy 0, policy_version 659196 (0.0008) [2023-12-26 20:09:59,549][105692] Updated weights for policy 0, policy_version 659206 (0.0008) [2023-12-26 20:09:59,621][105620] Updated weights for policy 1, policy_version 660028 (0.0010) [2023-12-26 20:09:59,680][105620] Updated weights for policy 1, policy_version 660038 (0.0010) [2023-12-26 20:09:59,741][105620] Updated weights for policy 1, policy_version 660048 (0.0008) [2023-12-26 20:10:00,244][105692] Updated weights for policy 0, policy_version 659216 (0.0006) [2023-12-26 20:10:00,311][105692] Updated weights for policy 0, policy_version 659226 (0.0006) [2023-12-26 20:10:00,371][105692] Updated weights for policy 0, policy_version 659236 (0.0008) [2023-12-26 20:10:00,472][105620] Updated weights for policy 1, policy_version 660058 (0.0009) [2023-12-26 20:10:00,530][105620] Updated weights for policy 1, policy_version 660068 (0.0005) [2023-12-26 20:10:00,584][105620] Updated weights for policy 1, policy_version 660078 (0.0005) [2023-12-26 20:10:00,633][105620] Updated weights for policy 1, policy_version 660088 (0.0005) [2023-12-26 20:10:01,059][105692] Updated weights for policy 0, policy_version 659246 (0.0006) [2023-12-26 20:10:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 337797120. Throughput: 0: 9720.8, 1: 9903.6. Samples: 337773648. Policy #0 lag: (min: 13.0, avg: 17.1, max: 45.0) [2023-12-26 20:10:01,062][104569] Avg episode reward: [(0, '9171.188'), (1, '9076.546')] [2023-12-26 20:10:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000660088_169000960.pth... [2023-12-26 20:10:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000658968_168714240.pth [2023-12-26 20:10:01,125][105692] Updated weights for policy 0, policy_version 659257 (0.0010) [2023-12-26 20:10:01,187][105692] Updated weights for policy 0, policy_version 659267 (0.0009) [2023-12-26 20:10:01,209][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000659272_168804352.pth... [2023-12-26 20:10:01,214][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000658120_168509440.pth [2023-12-26 20:10:01,269][105620] Updated weights for policy 1, policy_version 660098 (0.0009) [2023-12-26 20:10:01,327][105620] Updated weights for policy 1, policy_version 660108 (0.0009) [2023-12-26 20:10:01,392][105620] Updated weights for policy 1, policy_version 660118 (0.0007) [2023-12-26 20:10:02,001][105692] Updated weights for policy 0, policy_version 659277 (0.0010) [2023-12-26 20:10:02,055][105692] Updated weights for policy 0, policy_version 659287 (0.0007) [2023-12-26 20:10:02,073][105620] Updated weights for policy 1, policy_version 660128 (0.0009) [2023-12-26 20:10:02,115][105692] Updated weights for policy 0, policy_version 659297 (0.0008) [2023-12-26 20:10:02,130][105620] Updated weights for policy 1, policy_version 660138 (0.0006) [2023-12-26 20:10:02,182][105620] Updated weights for policy 1, policy_version 660148 (0.0007) [2023-12-26 20:10:02,835][105692] Updated weights for policy 0, policy_version 659307 (0.0007) [2023-12-26 20:10:02,887][105692] Updated weights for policy 0, policy_version 659318 (0.0010) [2023-12-26 20:10:02,911][105620] Updated weights for policy 1, policy_version 660158 (0.0007) [2023-12-26 20:10:02,937][105692] Updated weights for policy 0, policy_version 659328 (0.0006) [2023-12-26 20:10:02,959][105620] Updated weights for policy 1, policy_version 660168 (0.0010) [2023-12-26 20:10:03,006][105620] Updated weights for policy 1, policy_version 660178 (0.0010) [2023-12-26 20:10:03,628][105692] Updated weights for policy 0, policy_version 659338 (0.0005) [2023-12-26 20:10:03,636][105620] Updated weights for policy 1, policy_version 660188 (0.0007) [2023-12-26 20:10:03,680][105692] Updated weights for policy 0, policy_version 659348 (0.0005) [2023-12-26 20:10:03,697][105620] Updated weights for policy 1, policy_version 660198 (0.0006) [2023-12-26 20:10:03,734][105692] Updated weights for policy 0, policy_version 659358 (0.0006) [2023-12-26 20:10:03,755][105620] Updated weights for policy 1, policy_version 660208 (0.0009) [2023-12-26 20:10:03,790][105692] Updated weights for policy 0, policy_version 659368 (0.0005) [2023-12-26 20:10:04,478][105620] Updated weights for policy 1, policy_version 660218 (0.0008) [2023-12-26 20:10:04,490][105692] Updated weights for policy 0, policy_version 659378 (0.0008) [2023-12-26 20:10:04,530][105620] Updated weights for policy 1, policy_version 660228 (0.0009) [2023-12-26 20:10:04,536][105692] Updated weights for policy 0, policy_version 659388 (0.0009) [2023-12-26 20:10:04,585][105620] Updated weights for policy 1, policy_version 660238 (0.0008) [2023-12-26 20:10:04,590][105692] Updated weights for policy 0, policy_version 659398 (0.0009) [2023-12-26 20:10:04,645][105620] Updated weights for policy 1, policy_version 660248 (0.0009) [2023-12-26 20:10:05,331][105692] Updated weights for policy 0, policy_version 659408 (0.0009) [2023-12-26 20:10:05,386][105620] Updated weights for policy 1, policy_version 660258 (0.0006) [2023-12-26 20:10:05,387][105692] Updated weights for policy 0, policy_version 659418 (0.0008) [2023-12-26 20:10:05,433][105692] Updated weights for policy 0, policy_version 659428 (0.0006) [2023-12-26 20:10:05,438][105620] Updated weights for policy 1, policy_version 660268 (0.0007) [2023-12-26 20:10:05,495][105620] Updated weights for policy 1, policy_version 660278 (0.0009) [2023-12-26 20:10:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 337895424. Throughput: 0: 9705.9, 1: 9800.0. Samples: 337889136. Policy #0 lag: (min: 13.0, avg: 17.1, max: 45.0) [2023-12-26 20:10:06,062][104569] Avg episode reward: [(0, '8899.047'), (1, '8985.606')] [2023-12-26 20:10:06,212][105692] Updated weights for policy 0, policy_version 659438 (0.0006) [2023-12-26 20:10:06,263][105620] Updated weights for policy 1, policy_version 660288 (0.0008) [2023-12-26 20:10:06,276][105692] Updated weights for policy 0, policy_version 659448 (0.0007) [2023-12-26 20:10:06,333][105620] Updated weights for policy 1, policy_version 660298 (0.0006) [2023-12-26 20:10:06,340][105692] Updated weights for policy 0, policy_version 659458 (0.0008) [2023-12-26 20:10:06,401][105620] Updated weights for policy 1, policy_version 660308 (0.0009) [2023-12-26 20:10:07,079][105692] Updated weights for policy 0, policy_version 659468 (0.0008) [2023-12-26 20:10:07,128][105620] Updated weights for policy 1, policy_version 660318 (0.0008) [2023-12-26 20:10:07,130][105692] Updated weights for policy 0, policy_version 659478 (0.0007) [2023-12-26 20:10:07,184][105692] Updated weights for policy 0, policy_version 659488 (0.0006) [2023-12-26 20:10:07,186][105620] Updated weights for policy 1, policy_version 660328 (0.0006) [2023-12-26 20:10:07,236][105620] Updated weights for policy 1, policy_version 660338 (0.0005) [2023-12-26 20:10:07,790][105692] Updated weights for policy 0, policy_version 659498 (0.0007) [2023-12-26 20:10:07,860][105692] Updated weights for policy 0, policy_version 659508 (0.0005) [2023-12-26 20:10:07,910][105692] Updated weights for policy 0, policy_version 659518 (0.0005) [2023-12-26 20:10:07,978][105692] Updated weights for policy 0, policy_version 659528 (0.0005) [2023-12-26 20:10:08,132][105620] Updated weights for policy 1, policy_version 660348 (0.0010) [2023-12-26 20:10:08,182][105620] Updated weights for policy 1, policy_version 660358 (0.0009) [2023-12-26 20:10:08,241][105620] Updated weights for policy 1, policy_version 660368 (0.0009) [2023-12-26 20:10:08,581][105692] Updated weights for policy 0, policy_version 659538 (0.0009) [2023-12-26 20:10:08,645][105692] Updated weights for policy 0, policy_version 659548 (0.0008) [2023-12-26 20:10:08,708][105692] Updated weights for policy 0, policy_version 659558 (0.0009) [2023-12-26 20:10:09,014][105620] Updated weights for policy 1, policy_version 660378 (0.0009) [2023-12-26 20:10:09,076][105620] Updated weights for policy 1, policy_version 660388 (0.0009) [2023-12-26 20:10:09,130][105620] Updated weights for policy 1, policy_version 660398 (0.0009) [2023-12-26 20:10:09,177][105620] Updated weights for policy 1, policy_version 660408 (0.0008) [2023-12-26 20:10:09,493][105692] Updated weights for policy 0, policy_version 659568 (0.0009) [2023-12-26 20:10:09,560][105692] Updated weights for policy 0, policy_version 659578 (0.0009) [2023-12-26 20:10:09,621][105692] Updated weights for policy 0, policy_version 659588 (0.0009) [2023-12-26 20:10:09,964][105620] Updated weights for policy 1, policy_version 660418 (0.0008) [2023-12-26 20:10:10,024][105620] Updated weights for policy 1, policy_version 660428 (0.0009) [2023-12-26 20:10:10,083][105620] Updated weights for policy 1, policy_version 660438 (0.0009) [2023-12-26 20:10:10,451][105692] Updated weights for policy 0, policy_version 659598 (0.0009) [2023-12-26 20:10:10,520][105692] Updated weights for policy 0, policy_version 659608 (0.0008) [2023-12-26 20:10:10,585][105692] Updated weights for policy 0, policy_version 659618 (0.0008) [2023-12-26 20:10:10,726][105620] Updated weights for policy 1, policy_version 660448 (0.0009) [2023-12-26 20:10:10,779][105620] Updated weights for policy 1, policy_version 660458 (0.0009) [2023-12-26 20:10:10,836][105620] Updated weights for policy 1, policy_version 660468 (0.0008) [2023-12-26 20:10:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 337993728. Throughput: 0: 9739.2, 1: 9769.2. Samples: 338002188. Policy #0 lag: (min: 13.0, avg: 17.1, max: 45.0) [2023-12-26 20:10:11,062][104569] Avg episode reward: [(0, '9081.752'), (1, '8895.239')] [2023-12-26 20:10:11,371][105692] Updated weights for policy 0, policy_version 659628 (0.0010) [2023-12-26 20:10:11,436][105692] Updated weights for policy 0, policy_version 659638 (0.0009) [2023-12-26 20:10:11,495][105692] Updated weights for policy 0, policy_version 659648 (0.0010) [2023-12-26 20:10:11,536][105620] Updated weights for policy 1, policy_version 660478 (0.0006) [2023-12-26 20:10:11,602][105620] Updated weights for policy 1, policy_version 660488 (0.0009) [2023-12-26 20:10:11,664][105620] Updated weights for policy 1, policy_version 660498 (0.0007) [2023-12-26 20:10:12,204][105692] Updated weights for policy 0, policy_version 659658 (0.0008) [2023-12-26 20:10:12,253][105692] Updated weights for policy 0, policy_version 659668 (0.0009) [2023-12-26 20:10:12,320][105692] Updated weights for policy 0, policy_version 659678 (0.0008) [2023-12-26 20:10:12,387][105692] Updated weights for policy 0, policy_version 659688 (0.0007) [2023-12-26 20:10:12,407][105620] Updated weights for policy 1, policy_version 660508 (0.0008) [2023-12-26 20:10:12,467][105620] Updated weights for policy 1, policy_version 660518 (0.0009) [2023-12-26 20:10:12,526][105620] Updated weights for policy 1, policy_version 660528 (0.0009) [2023-12-26 20:10:13,090][105692] Updated weights for policy 0, policy_version 659698 (0.0005) [2023-12-26 20:10:13,137][105692] Updated weights for policy 0, policy_version 659708 (0.0006) [2023-12-26 20:10:13,151][105620] Updated weights for policy 1, policy_version 660538 (0.0006) [2023-12-26 20:10:13,196][105692] Updated weights for policy 0, policy_version 659718 (0.0008) [2023-12-26 20:10:13,214][105620] Updated weights for policy 1, policy_version 660548 (0.0011) [2023-12-26 20:10:13,279][105620] Updated weights for policy 1, policy_version 660558 (0.0011) [2023-12-26 20:10:13,344][105620] Updated weights for policy 1, policy_version 660568 (0.0011) [2023-12-26 20:10:13,756][105692] Updated weights for policy 0, policy_version 659728 (0.0009) [2023-12-26 20:10:13,801][105692] Updated weights for policy 0, policy_version 659738 (0.0008) [2023-12-26 20:10:13,849][105692] Updated weights for policy 0, policy_version 659748 (0.0007) [2023-12-26 20:10:14,072][105620] Updated weights for policy 1, policy_version 660578 (0.0010) [2023-12-26 20:10:14,131][105620] Updated weights for policy 1, policy_version 660588 (0.0010) [2023-12-26 20:10:14,179][105620] Updated weights for policy 1, policy_version 660598 (0.0010) [2023-12-26 20:10:14,484][105692] Updated weights for policy 0, policy_version 659758 (0.0006) [2023-12-26 20:10:14,540][105692] Updated weights for policy 0, policy_version 659768 (0.0005) [2023-12-26 20:10:14,606][105692] Updated weights for policy 0, policy_version 659778 (0.0006) [2023-12-26 20:10:14,951][105620] Updated weights for policy 1, policy_version 660608 (0.0009) [2023-12-26 20:10:15,015][105620] Updated weights for policy 1, policy_version 660618 (0.0008) [2023-12-26 20:10:15,072][105620] Updated weights for policy 1, policy_version 660628 (0.0008) [2023-12-26 20:10:15,239][105692] Updated weights for policy 0, policy_version 659788 (0.0008) [2023-12-26 20:10:15,302][105692] Updated weights for policy 0, policy_version 659798 (0.0011) [2023-12-26 20:10:15,373][105692] Updated weights for policy 0, policy_version 659808 (0.0011) [2023-12-26 20:10:15,776][105620] Updated weights for policy 1, policy_version 660638 (0.0006) [2023-12-26 20:10:15,837][105620] Updated weights for policy 1, policy_version 660648 (0.0005) [2023-12-26 20:10:15,907][105620] Updated weights for policy 1, policy_version 660658 (0.0005) [2023-12-26 20:10:16,038][105692] Updated weights for policy 0, policy_version 659818 (0.0009) [2023-12-26 20:10:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.4, 300 sec: 19522.0). Total num frames: 338092032. Throughput: 0: 9705.1, 1: 9715.1. Samples: 338061156. Policy #0 lag: (min: 13.0, avg: 17.1, max: 45.0) [2023-12-26 20:10:16,062][104569] Avg episode reward: [(0, '8991.237'), (1, '8803.864')] [2023-12-26 20:10:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000660664_169148416.pth... [2023-12-26 20:10:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000659544_168861696.pth [2023-12-26 20:10:16,093][105692] Updated weights for policy 0, policy_version 659828 (0.0005) [2023-12-26 20:10:16,154][105692] Updated weights for policy 0, policy_version 659838 (0.0010) [2023-12-26 20:10:16,214][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000659848_168951808.pth... [2023-12-26 20:10:16,216][105692] Updated weights for policy 0, policy_version 659848 (0.0011) [2023-12-26 20:10:16,218][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000658696_168656896.pth [2023-12-26 20:10:16,474][105620] Updated weights for policy 1, policy_version 660668 (0.0005) [2023-12-26 20:10:16,540][105620] Updated weights for policy 1, policy_version 660678 (0.0006) [2023-12-26 20:10:16,594][105620] Updated weights for policy 1, policy_version 660688 (0.0007) [2023-12-26 20:10:16,877][105692] Updated weights for policy 0, policy_version 659858 (0.0011) [2023-12-26 20:10:16,928][105692] Updated weights for policy 0, policy_version 659868 (0.0010) [2023-12-26 20:10:16,996][105692] Updated weights for policy 0, policy_version 659878 (0.0011) [2023-12-26 20:10:17,238][105620] Updated weights for policy 1, policy_version 660698 (0.0008) [2023-12-26 20:10:17,290][105620] Updated weights for policy 1, policy_version 660708 (0.0005) [2023-12-26 20:10:17,348][105620] Updated weights for policy 1, policy_version 660718 (0.0006) [2023-12-26 20:10:17,409][105620] Updated weights for policy 1, policy_version 660728 (0.0006) [2023-12-26 20:10:17,751][105692] Updated weights for policy 0, policy_version 659888 (0.0010) [2023-12-26 20:10:17,806][105692] Updated weights for policy 0, policy_version 659898 (0.0011) [2023-12-26 20:10:17,854][105692] Updated weights for policy 0, policy_version 659908 (0.0010) [2023-12-26 20:10:18,065][105620] Updated weights for policy 1, policy_version 660738 (0.0008) [2023-12-26 20:10:18,115][105620] Updated weights for policy 1, policy_version 660748 (0.0007) [2023-12-26 20:10:18,166][105620] Updated weights for policy 1, policy_version 660758 (0.0005) [2023-12-26 20:10:18,598][105692] Updated weights for policy 0, policy_version 659918 (0.0010) [2023-12-26 20:10:18,660][105692] Updated weights for policy 0, policy_version 659928 (0.0010) [2023-12-26 20:10:18,715][105692] Updated weights for policy 0, policy_version 659938 (0.0010) [2023-12-26 20:10:18,811][105620] Updated weights for policy 1, policy_version 660768 (0.0008) [2023-12-26 20:10:18,875][105620] Updated weights for policy 1, policy_version 660778 (0.0008) [2023-12-26 20:10:18,930][105620] Updated weights for policy 1, policy_version 660788 (0.0008) [2023-12-26 20:10:19,440][105692] Updated weights for policy 0, policy_version 659948 (0.0010) [2023-12-26 20:10:19,506][105692] Updated weights for policy 0, policy_version 659958 (0.0011) [2023-12-26 20:10:19,563][105692] Updated weights for policy 0, policy_version 659968 (0.0010) [2023-12-26 20:10:19,732][105620] Updated weights for policy 1, policy_version 660798 (0.0009) [2023-12-26 20:10:19,781][105620] Updated weights for policy 1, policy_version 660808 (0.0008) [2023-12-26 20:10:19,841][105620] Updated weights for policy 1, policy_version 660818 (0.0008) [2023-12-26 20:10:20,281][105692] Updated weights for policy 0, policy_version 659978 (0.0008) [2023-12-26 20:10:20,343][105692] Updated weights for policy 0, policy_version 659988 (0.0010) [2023-12-26 20:10:20,400][105692] Updated weights for policy 0, policy_version 659998 (0.0011) [2023-12-26 20:10:20,464][105692] Updated weights for policy 0, policy_version 660008 (0.0011) [2023-12-26 20:10:20,666][105620] Updated weights for policy 1, policy_version 660828 (0.0008) [2023-12-26 20:10:20,723][105620] Updated weights for policy 1, policy_version 660838 (0.0008) [2023-12-26 20:10:20,780][105620] Updated weights for policy 1, policy_version 660848 (0.0008) [2023-12-26 20:10:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 338190336. Throughput: 0: 9747.4, 1: 9717.9. Samples: 338181468. Policy #0 lag: (min: 13.0, avg: 17.1, max: 45.0) [2023-12-26 20:10:21,063][104569] Avg episode reward: [(0, '8990.402'), (1, '8622.485')] [2023-12-26 20:10:21,227][105692] Updated weights for policy 0, policy_version 660018 (0.0009) [2023-12-26 20:10:21,286][105692] Updated weights for policy 0, policy_version 660028 (0.0010) [2023-12-26 20:10:21,346][105692] Updated weights for policy 0, policy_version 660038 (0.0009) [2023-12-26 20:10:21,532][105620] Updated weights for policy 1, policy_version 660858 (0.0007) [2023-12-26 20:10:21,585][105620] Updated weights for policy 1, policy_version 660868 (0.0005) [2023-12-26 20:10:21,648][105620] Updated weights for policy 1, policy_version 660878 (0.0008) [2023-12-26 20:10:21,714][105620] Updated weights for policy 1, policy_version 660888 (0.0007) [2023-12-26 20:10:22,098][105692] Updated weights for policy 0, policy_version 660048 (0.0009) [2023-12-26 20:10:22,163][105692] Updated weights for policy 0, policy_version 660058 (0.0008) [2023-12-26 20:10:22,221][105692] Updated weights for policy 0, policy_version 660068 (0.0010) [2023-12-26 20:10:22,339][105620] Updated weights for policy 1, policy_version 660898 (0.0009) [2023-12-26 20:10:22,383][105586] KL-divergence is very high: 105.5780 [2023-12-26 20:10:22,411][105620] Updated weights for policy 1, policy_version 660908 (0.0010) [2023-12-26 20:10:22,440][105586] KL-divergence is very high: 106.0314 [2023-12-26 20:10:22,478][105620] Updated weights for policy 1, policy_version 660918 (0.0009) [2023-12-26 20:10:22,979][105692] Updated weights for policy 0, policy_version 660078 (0.0009) [2023-12-26 20:10:23,027][105692] Updated weights for policy 0, policy_version 660088 (0.0010) [2023-12-26 20:10:23,076][105692] Updated weights for policy 0, policy_version 660098 (0.0008) [2023-12-26 20:10:23,183][105620] Updated weights for policy 1, policy_version 660928 (0.0006) [2023-12-26 20:10:23,243][105620] Updated weights for policy 1, policy_version 660938 (0.0006) [2023-12-26 20:10:23,304][105620] Updated weights for policy 1, policy_version 660948 (0.0005) [2023-12-26 20:10:23,840][105692] Updated weights for policy 0, policy_version 660108 (0.0010) [2023-12-26 20:10:23,889][105692] Updated weights for policy 0, policy_version 660118 (0.0008) [2023-12-26 20:10:23,945][105692] Updated weights for policy 0, policy_version 660128 (0.0008) [2023-12-26 20:10:23,992][105620] Updated weights for policy 1, policy_version 660958 (0.0009) [2023-12-26 20:10:24,044][105620] Updated weights for policy 1, policy_version 660968 (0.0010) [2023-12-26 20:10:24,091][105620] Updated weights for policy 1, policy_version 660978 (0.0010) [2023-12-26 20:10:24,682][105692] Updated weights for policy 0, policy_version 660138 (0.0007) [2023-12-26 20:10:24,733][105692] Updated weights for policy 0, policy_version 660148 (0.0007) [2023-12-26 20:10:24,771][105620] Updated weights for policy 1, policy_version 660988 (0.0008) [2023-12-26 20:10:24,793][105692] Updated weights for policy 0, policy_version 660158 (0.0005) [2023-12-26 20:10:24,825][105620] Updated weights for policy 1, policy_version 660998 (0.0005) [2023-12-26 20:10:24,851][105692] Updated weights for policy 0, policy_version 660168 (0.0005) [2023-12-26 20:10:24,884][105620] Updated weights for policy 1, policy_version 661008 (0.0005) [2023-12-26 20:10:25,413][105620] Updated weights for policy 1, policy_version 661018 (0.0007) [2023-12-26 20:10:25,474][105620] Updated weights for policy 1, policy_version 661028 (0.0008) [2023-12-26 20:10:25,538][105620] Updated weights for policy 1, policy_version 661038 (0.0008) [2023-12-26 20:10:25,599][105620] Updated weights for policy 1, policy_version 661048 (0.0007) [2023-12-26 20:10:25,608][105692] Updated weights for policy 0, policy_version 660178 (0.0007) [2023-12-26 20:10:25,665][105692] Updated weights for policy 0, policy_version 660188 (0.0010) [2023-12-26 20:10:25,718][105692] Updated weights for policy 0, policy_version 660199 (0.0009) [2023-12-26 20:10:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 338288640. Throughput: 0: 9711.7, 1: 9711.1. Samples: 338299096. Policy #0 lag: (min: 13.0, avg: 17.1, max: 45.0) [2023-12-26 20:10:26,062][104569] Avg episode reward: [(0, '8632.388'), (1, '7900.362')] [2023-12-26 20:10:26,132][105620] Updated weights for policy 1, policy_version 661058 (0.0005) [2023-12-26 20:10:26,185][105620] Updated weights for policy 1, policy_version 661068 (0.0010) [2023-12-26 20:10:26,233][105620] Updated weights for policy 1, policy_version 661078 (0.0010) [2023-12-26 20:10:26,561][105692] Updated weights for policy 0, policy_version 660209 (0.0008) [2023-12-26 20:10:26,613][105692] Updated weights for policy 0, policy_version 660219 (0.0009) [2023-12-26 20:10:26,675][105692] Updated weights for policy 0, policy_version 660230 (0.0010) [2023-12-26 20:10:26,890][105620] Updated weights for policy 1, policy_version 661088 (0.0007) [2023-12-26 20:10:26,942][105586] KL-divergence is very high: 173.3169 [2023-12-26 20:10:26,947][105620] Updated weights for policy 1, policy_version 661098 (0.0005) [2023-12-26 20:10:26,959][105586] KL-divergence is very high: 120.1959 [2023-12-26 20:10:26,983][105586] KL-divergence is very high: 318.4134 [2023-12-26 20:10:26,997][105586] KL-divergence is very high: 136.7935 [2023-12-26 20:10:26,997][105620] Updated weights for policy 1, policy_version 661108 (0.0005) [2023-12-26 20:10:27,402][105692] Updated weights for policy 0, policy_version 660240 (0.0006) [2023-12-26 20:10:27,456][105692] Updated weights for policy 0, policy_version 660250 (0.0005) [2023-12-26 20:10:27,510][105692] Updated weights for policy 0, policy_version 660260 (0.0005) [2023-12-26 20:10:27,602][105620] Updated weights for policy 1, policy_version 661118 (0.0005) [2023-12-26 20:10:27,659][105620] Updated weights for policy 1, policy_version 661128 (0.0005) [2023-12-26 20:10:27,713][105620] Updated weights for policy 1, policy_version 661138 (0.0005) [2023-12-26 20:10:28,220][105620] Updated weights for policy 1, policy_version 661148 (0.0005) [2023-12-26 20:10:28,241][105692] Updated weights for policy 0, policy_version 660270 (0.0007) [2023-12-26 20:10:28,269][105620] Updated weights for policy 1, policy_version 661158 (0.0005) [2023-12-26 20:10:28,298][105692] Updated weights for policy 0, policy_version 660280 (0.0009) [2023-12-26 20:10:28,324][105620] Updated weights for policy 1, policy_version 661168 (0.0006) [2023-12-26 20:10:28,360][105692] Updated weights for policy 0, policy_version 660290 (0.0008) [2023-12-26 20:10:28,978][105620] Updated weights for policy 1, policy_version 661178 (0.0009) [2023-12-26 20:10:29,026][105620] Updated weights for policy 1, policy_version 661188 (0.0005) [2023-12-26 20:10:29,074][105620] Updated weights for policy 1, policy_version 661198 (0.0005) [2023-12-26 20:10:29,114][105692] Updated weights for policy 0, policy_version 660300 (0.0009) [2023-12-26 20:10:29,119][105620] Updated weights for policy 1, policy_version 661208 (0.0005) [2023-12-26 20:10:29,171][105692] Updated weights for policy 0, policy_version 660310 (0.0009) [2023-12-26 20:10:29,223][105692] Updated weights for policy 0, policy_version 660320 (0.0010) [2023-12-26 20:10:29,783][105620] Updated weights for policy 1, policy_version 661218 (0.0005) [2023-12-26 20:10:29,843][105620] Updated weights for policy 1, policy_version 661228 (0.0009) [2023-12-26 20:10:29,895][105620] Updated weights for policy 1, policy_version 661238 (0.0007) [2023-12-26 20:10:30,023][105692] Updated weights for policy 0, policy_version 660330 (0.0008) [2023-12-26 20:10:30,080][105692] Updated weights for policy 0, policy_version 660340 (0.0006) [2023-12-26 20:10:30,133][105692] Updated weights for policy 0, policy_version 660350 (0.0006) [2023-12-26 20:10:30,190][105692] Updated weights for policy 0, policy_version 660360 (0.0005) [2023-12-26 20:10:30,589][105620] Updated weights for policy 1, policy_version 661248 (0.0006) [2023-12-26 20:10:30,652][105620] Updated weights for policy 1, policy_version 661258 (0.0008) [2023-12-26 20:10:30,711][105620] Updated weights for policy 1, policy_version 661268 (0.0006) [2023-12-26 20:10:30,787][105692] Updated weights for policy 0, policy_version 660370 (0.0007) [2023-12-26 20:10:30,850][105692] Updated weights for policy 0, policy_version 660380 (0.0008) [2023-12-26 20:10:30,895][105692] Updated weights for policy 0, policy_version 660390 (0.0009) [2023-12-26 20:10:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 338395136. Throughput: 0: 9675.2, 1: 9884.1. Samples: 338360920. Policy #0 lag: (min: 13.0, avg: 17.1, max: 45.0) [2023-12-26 20:10:31,062][104569] Avg episode reward: [(0, '8455.949'), (1, '8348.774')] [2023-12-26 20:10:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000660392_169091072.pth... [2023-12-26 20:10:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000661272_169304064.pth... [2023-12-26 20:10:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000659272_168804352.pth [2023-12-26 20:10:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000660088_169000960.pth [2023-12-26 20:10:31,407][105620] Updated weights for policy 1, policy_version 661278 (0.0007) [2023-12-26 20:10:31,456][105620] Updated weights for policy 1, policy_version 661288 (0.0008) [2023-12-26 20:10:31,511][105620] Updated weights for policy 1, policy_version 661298 (0.0008) [2023-12-26 20:10:31,676][105692] Updated weights for policy 0, policy_version 660400 (0.0010) [2023-12-26 20:10:31,739][105692] Updated weights for policy 0, policy_version 660410 (0.0010) [2023-12-26 20:10:31,790][105692] Updated weights for policy 0, policy_version 660420 (0.0008) [2023-12-26 20:10:32,248][105620] Updated weights for policy 1, policy_version 661308 (0.0008) [2023-12-26 20:10:32,305][105620] Updated weights for policy 1, policy_version 661318 (0.0010) [2023-12-26 20:10:32,365][105620] Updated weights for policy 1, policy_version 661328 (0.0009) [2023-12-26 20:10:32,536][105692] Updated weights for policy 0, policy_version 660430 (0.0007) [2023-12-26 20:10:32,583][105692] Updated weights for policy 0, policy_version 660440 (0.0009) [2023-12-26 20:10:32,641][105692] Updated weights for policy 0, policy_version 660450 (0.0009) [2023-12-26 20:10:32,994][105620] Updated weights for policy 1, policy_version 661338 (0.0009) [2023-12-26 20:10:33,043][105620] Updated weights for policy 1, policy_version 661348 (0.0005) [2023-12-26 20:10:33,088][105620] Updated weights for policy 1, policy_version 661358 (0.0005) [2023-12-26 20:10:33,142][105620] Updated weights for policy 1, policy_version 661368 (0.0007) [2023-12-26 20:10:33,398][105692] Updated weights for policy 0, policy_version 660460 (0.0010) [2023-12-26 20:10:33,459][105692] Updated weights for policy 0, policy_version 660470 (0.0010) [2023-12-26 20:10:33,513][105692] Updated weights for policy 0, policy_version 660480 (0.0010) [2023-12-26 20:10:33,876][105620] Updated weights for policy 1, policy_version 661378 (0.0009) [2023-12-26 20:10:33,933][105620] Updated weights for policy 1, policy_version 661388 (0.0009) [2023-12-26 20:10:34,003][105620] Updated weights for policy 1, policy_version 661398 (0.0009) [2023-12-26 20:10:34,112][105692] Updated weights for policy 0, policy_version 660490 (0.0009) [2023-12-26 20:10:34,167][105692] Updated weights for policy 0, policy_version 660500 (0.0007) [2023-12-26 20:10:34,224][105692] Updated weights for policy 0, policy_version 660510 (0.0007) [2023-12-26 20:10:34,285][105692] Updated weights for policy 0, policy_version 660520 (0.0007) [2023-12-26 20:10:34,798][105620] Updated weights for policy 1, policy_version 661408 (0.0008) [2023-12-26 20:10:34,859][105620] Updated weights for policy 1, policy_version 661418 (0.0009) [2023-12-26 20:10:34,908][105620] Updated weights for policy 1, policy_version 661428 (0.0008) [2023-12-26 20:10:35,001][105692] Updated weights for policy 0, policy_version 660530 (0.0010) [2023-12-26 20:10:35,050][105692] Updated weights for policy 0, policy_version 660540 (0.0011) [2023-12-26 20:10:35,105][105692] Updated weights for policy 0, policy_version 660550 (0.0010) [2023-12-26 20:10:35,683][105620] Updated weights for policy 1, policy_version 661438 (0.0006) [2023-12-26 20:10:35,734][105620] Updated weights for policy 1, policy_version 661448 (0.0005) [2023-12-26 20:10:35,791][105620] Updated weights for policy 1, policy_version 661458 (0.0007) [2023-12-26 20:10:35,852][105692] Updated weights for policy 0, policy_version 660560 (0.0010) [2023-12-26 20:10:35,906][105692] Updated weights for policy 0, policy_version 660570 (0.0010) [2023-12-26 20:10:35,965][105692] Updated weights for policy 0, policy_version 660580 (0.0010) [2023-12-26 20:10:36,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 338493440. Throughput: 0: 9667.7, 1: 9923.5. Samples: 338478884. Policy #0 lag: (min: 13.0, avg: 17.1, max: 45.0) [2023-12-26 20:10:36,063][104569] Avg episode reward: [(0, '8724.562'), (1, '9169.119')] [2023-12-26 20:10:36,422][105620] Updated weights for policy 1, policy_version 661468 (0.0009) [2023-12-26 20:10:36,482][105620] Updated weights for policy 1, policy_version 661478 (0.0011) [2023-12-26 20:10:36,546][105620] Updated weights for policy 1, policy_version 661488 (0.0011) [2023-12-26 20:10:36,753][105692] Updated weights for policy 0, policy_version 660590 (0.0009) [2023-12-26 20:10:36,809][105692] Updated weights for policy 0, policy_version 660600 (0.0008) [2023-12-26 20:10:36,873][105692] Updated weights for policy 0, policy_version 660610 (0.0008) [2023-12-26 20:10:37,313][105620] Updated weights for policy 1, policy_version 661498 (0.0011) [2023-12-26 20:10:37,380][105620] Updated weights for policy 1, policy_version 661508 (0.0011) [2023-12-26 20:10:37,441][105620] Updated weights for policy 1, policy_version 661518 (0.0011) [2023-12-26 20:10:37,498][105620] Updated weights for policy 1, policy_version 661528 (0.0011) [2023-12-26 20:10:37,638][105692] Updated weights for policy 0, policy_version 660620 (0.0009) [2023-12-26 20:10:37,697][105692] Updated weights for policy 0, policy_version 660630 (0.0007) [2023-12-26 20:10:37,760][105692] Updated weights for policy 0, policy_version 660640 (0.0006) [2023-12-26 20:10:38,181][105620] Updated weights for policy 1, policy_version 661538 (0.0005) [2023-12-26 20:10:38,246][105620] Updated weights for policy 1, policy_version 661548 (0.0005) [2023-12-26 20:10:38,319][105620] Updated weights for policy 1, policy_version 661558 (0.0005) [2023-12-26 20:10:38,444][105692] Updated weights for policy 0, policy_version 660650 (0.0008) [2023-12-26 20:10:38,501][105692] Updated weights for policy 0, policy_version 660660 (0.0010) [2023-12-26 20:10:38,567][105692] Updated weights for policy 0, policy_version 660670 (0.0008) [2023-12-26 20:10:38,628][105692] Updated weights for policy 0, policy_version 660680 (0.0008) [2023-12-26 20:10:38,946][105620] Updated weights for policy 1, policy_version 661568 (0.0010) [2023-12-26 20:10:39,009][105620] Updated weights for policy 1, policy_version 661578 (0.0010) [2023-12-26 20:10:39,067][105620] Updated weights for policy 1, policy_version 661588 (0.0010) [2023-12-26 20:10:39,372][105692] Updated weights for policy 0, policy_version 660690 (0.0007) [2023-12-26 20:10:39,435][105692] Updated weights for policy 0, policy_version 660700 (0.0007) [2023-12-26 20:10:39,491][105692] Updated weights for policy 0, policy_version 660710 (0.0008) [2023-12-26 20:10:39,827][105620] Updated weights for policy 1, policy_version 661598 (0.0010) [2023-12-26 20:10:39,890][105620] Updated weights for policy 1, policy_version 661608 (0.0008) [2023-12-26 20:10:39,958][105620] Updated weights for policy 1, policy_version 661618 (0.0009) [2023-12-26 20:10:40,276][105692] Updated weights for policy 0, policy_version 660720 (0.0008) [2023-12-26 20:10:40,326][105692] Updated weights for policy 0, policy_version 660730 (0.0008) [2023-12-26 20:10:40,376][105692] Updated weights for policy 0, policy_version 660740 (0.0008) [2023-12-26 20:10:40,745][105620] Updated weights for policy 1, policy_version 661628 (0.0011) [2023-12-26 20:10:40,806][105620] Updated weights for policy 1, policy_version 661638 (0.0010) [2023-12-26 20:10:40,865][105620] Updated weights for policy 1, policy_version 661648 (0.0009) [2023-12-26 20:10:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 338583552. Throughput: 0: 9666.4, 1: 9866.3. Samples: 338592108. Policy #0 lag: (min: 8.0, avg: 36.5, max: 40.0) [2023-12-26 20:10:41,063][104569] Avg episode reward: [(0, '9082.604'), (1, '9078.509')] [2023-12-26 20:10:41,205][105692] Updated weights for policy 0, policy_version 660750 (0.0008) [2023-12-26 20:10:41,266][105692] Updated weights for policy 0, policy_version 660760 (0.0008) [2023-12-26 20:10:41,328][105692] Updated weights for policy 0, policy_version 660770 (0.0008) [2023-12-26 20:10:41,567][105620] Updated weights for policy 1, policy_version 661658 (0.0010) [2023-12-26 20:10:41,632][105620] Updated weights for policy 1, policy_version 661668 (0.0010) [2023-12-26 20:10:41,693][105620] Updated weights for policy 1, policy_version 661678 (0.0009) [2023-12-26 20:10:41,753][105620] Updated weights for policy 1, policy_version 661688 (0.0009) [2023-12-26 20:10:42,133][105692] Updated weights for policy 0, policy_version 660780 (0.0009) [2023-12-26 20:10:42,182][105692] Updated weights for policy 0, policy_version 660790 (0.0008) [2023-12-26 20:10:42,234][105692] Updated weights for policy 0, policy_version 660800 (0.0009) [2023-12-26 20:10:42,512][105620] Updated weights for policy 1, policy_version 661698 (0.0009) [2023-12-26 20:10:42,558][105620] Updated weights for policy 1, policy_version 661708 (0.0010) [2023-12-26 20:10:42,607][105620] Updated weights for policy 1, policy_version 661718 (0.0011) [2023-12-26 20:10:43,017][105692] Updated weights for policy 0, policy_version 660810 (0.0009) [2023-12-26 20:10:43,082][105692] Updated weights for policy 0, policy_version 660820 (0.0006) [2023-12-26 20:10:43,141][105692] Updated weights for policy 0, policy_version 660830 (0.0005) [2023-12-26 20:10:43,188][105585] KL-divergence is very high: 130.0966 [2023-12-26 20:10:43,193][105692] Updated weights for policy 0, policy_version 660840 (0.0008) [2023-12-26 20:10:43,311][105620] Updated weights for policy 1, policy_version 661728 (0.0009) [2023-12-26 20:10:43,374][105620] Updated weights for policy 1, policy_version 661738 (0.0008) [2023-12-26 20:10:43,432][105620] Updated weights for policy 1, policy_version 661748 (0.0009) [2023-12-26 20:10:43,887][105692] Updated weights for policy 0, policy_version 660850 (0.0009) [2023-12-26 20:10:43,944][105692] Updated weights for policy 0, policy_version 660860 (0.0009) [2023-12-26 20:10:44,000][105692] Updated weights for policy 0, policy_version 660870 (0.0009) [2023-12-26 20:10:44,038][105620] Updated weights for policy 1, policy_version 661758 (0.0007) [2023-12-26 20:10:44,092][105620] Updated weights for policy 1, policy_version 661768 (0.0005) [2023-12-26 20:10:44,140][105620] Updated weights for policy 1, policy_version 661778 (0.0010) [2023-12-26 20:10:44,641][105692] Updated weights for policy 0, policy_version 660880 (0.0009) [2023-12-26 20:10:44,689][105692] Updated weights for policy 0, policy_version 660890 (0.0008) [2023-12-26 20:10:44,741][105692] Updated weights for policy 0, policy_version 660900 (0.0008) [2023-12-26 20:10:44,862][105620] Updated weights for policy 1, policy_version 661788 (0.0010) [2023-12-26 20:10:44,925][105620] Updated weights for policy 1, policy_version 661798 (0.0011) [2023-12-26 20:10:44,988][105620] Updated weights for policy 1, policy_version 661808 (0.0010) [2023-12-26 20:10:45,545][105692] Updated weights for policy 0, policy_version 660910 (0.0009) [2023-12-26 20:10:45,592][105692] Updated weights for policy 0, policy_version 660920 (0.0010) [2023-12-26 20:10:45,640][105692] Updated weights for policy 0, policy_version 660930 (0.0010) [2023-12-26 20:10:45,711][105620] Updated weights for policy 1, policy_version 661818 (0.0009) [2023-12-26 20:10:45,766][105620] Updated weights for policy 1, policy_version 661828 (0.0007) [2023-12-26 20:10:45,821][105620] Updated weights for policy 1, policy_version 661838 (0.0010) [2023-12-26 20:10:45,865][105620] Updated weights for policy 1, policy_version 661848 (0.0007) [2023-12-26 20:10:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 338681856. Throughput: 0: 9535.3, 1: 9923.1. Samples: 338649280. Policy #0 lag: (min: 8.0, avg: 36.5, max: 40.0) [2023-12-26 20:10:46,063][104569] Avg episode reward: [(0, '8214.958'), (1, '8988.049')] [2023-12-26 20:10:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000660936_169230336.pth... [2023-12-26 20:10:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000661848_169451520.pth... [2023-12-26 20:10:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000659848_168951808.pth [2023-12-26 20:10:46,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000660664_169148416.pth [2023-12-26 20:10:46,276][105692] Updated weights for policy 0, policy_version 660940 (0.0008) [2023-12-26 20:10:46,337][105692] Updated weights for policy 0, policy_version 660950 (0.0005) [2023-12-26 20:10:46,391][105692] Updated weights for policy 0, policy_version 660960 (0.0005) [2023-12-26 20:10:46,517][105620] Updated weights for policy 1, policy_version 661858 (0.0010) [2023-12-26 20:10:46,575][105620] Updated weights for policy 1, policy_version 661868 (0.0010) [2023-12-26 20:10:46,630][105620] Updated weights for policy 1, policy_version 661878 (0.0010) [2023-12-26 20:10:46,997][105692] Updated weights for policy 0, policy_version 660970 (0.0005) [2023-12-26 20:10:47,057][105692] Updated weights for policy 0, policy_version 660980 (0.0005) [2023-12-26 20:10:47,111][105692] Updated weights for policy 0, policy_version 660990 (0.0009) [2023-12-26 20:10:47,163][105692] Updated weights for policy 0, policy_version 661000 (0.0010) [2023-12-26 20:10:47,392][105620] Updated weights for policy 1, policy_version 661888 (0.0011) [2023-12-26 20:10:47,457][105620] Updated weights for policy 1, policy_version 661898 (0.0010) [2023-12-26 20:10:47,519][105620] Updated weights for policy 1, policy_version 661908 (0.0010) [2023-12-26 20:10:47,839][105692] Updated weights for policy 0, policy_version 661010 (0.0008) [2023-12-26 20:10:47,899][105692] Updated weights for policy 0, policy_version 661020 (0.0008) [2023-12-26 20:10:47,954][105692] Updated weights for policy 0, policy_version 661030 (0.0007) [2023-12-26 20:10:48,174][105620] Updated weights for policy 1, policy_version 661918 (0.0011) [2023-12-26 20:10:48,223][105620] Updated weights for policy 1, policy_version 661928 (0.0010) [2023-12-26 20:10:48,281][105620] Updated weights for policy 1, policy_version 661938 (0.0010) [2023-12-26 20:10:48,725][105692] Updated weights for policy 0, policy_version 661040 (0.0008) [2023-12-26 20:10:48,782][105692] Updated weights for policy 0, policy_version 661050 (0.0006) [2023-12-26 20:10:48,843][105692] Updated weights for policy 0, policy_version 661060 (0.0006) [2023-12-26 20:10:49,052][105620] Updated weights for policy 1, policy_version 661948 (0.0010) [2023-12-26 20:10:49,117][105620] Updated weights for policy 1, policy_version 661958 (0.0011) [2023-12-26 20:10:49,176][105620] Updated weights for policy 1, policy_version 661968 (0.0010) [2023-12-26 20:10:49,502][105692] Updated weights for policy 0, policy_version 661070 (0.0009) [2023-12-26 20:10:49,554][105692] Updated weights for policy 0, policy_version 661081 (0.0009) [2023-12-26 20:10:49,607][105692] Updated weights for policy 0, policy_version 661091 (0.0008) [2023-12-26 20:10:49,831][105620] Updated weights for policy 1, policy_version 661978 (0.0011) [2023-12-26 20:10:49,897][105620] Updated weights for policy 1, policy_version 661988 (0.0011) [2023-12-26 20:10:49,957][105620] Updated weights for policy 1, policy_version 661998 (0.0008) [2023-12-26 20:10:50,025][105620] Updated weights for policy 1, policy_version 662008 (0.0007) [2023-12-26 20:10:50,421][105692] Updated weights for policy 0, policy_version 661101 (0.0009) [2023-12-26 20:10:50,478][105692] Updated weights for policy 0, policy_version 661111 (0.0008) [2023-12-26 20:10:50,527][105692] Updated weights for policy 0, policy_version 661121 (0.0008) [2023-12-26 20:10:50,715][105620] Updated weights for policy 1, policy_version 662018 (0.0011) [2023-12-26 20:10:50,770][105620] Updated weights for policy 1, policy_version 662028 (0.0011) [2023-12-26 20:10:50,840][105620] Updated weights for policy 1, policy_version 662038 (0.0011) [2023-12-26 20:10:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 338780160. Throughput: 0: 9629.4, 1: 9926.3. Samples: 338769144. Policy #0 lag: (min: 8.0, avg: 36.5, max: 40.0) [2023-12-26 20:10:51,063][104569] Avg episode reward: [(0, '8216.600'), (1, '8899.778')] [2023-12-26 20:10:51,187][105692] Updated weights for policy 0, policy_version 661131 (0.0008) [2023-12-26 20:10:51,249][105692] Updated weights for policy 0, policy_version 661141 (0.0006) [2023-12-26 20:10:51,316][105692] Updated weights for policy 0, policy_version 661151 (0.0006) [2023-12-26 20:10:51,619][105620] Updated weights for policy 1, policy_version 662048 (0.0011) [2023-12-26 20:10:51,675][105620] Updated weights for policy 1, policy_version 662058 (0.0011) [2023-12-26 20:10:51,735][105620] Updated weights for policy 1, policy_version 662068 (0.0011) [2023-12-26 20:10:51,982][105692] Updated weights for policy 0, policy_version 661161 (0.0009) [2023-12-26 20:10:52,036][105692] Updated weights for policy 0, policy_version 661171 (0.0010) [2023-12-26 20:10:52,096][105692] Updated weights for policy 0, policy_version 661181 (0.0010) [2023-12-26 20:10:52,151][105692] Updated weights for policy 0, policy_version 661191 (0.0010) [2023-12-26 20:10:52,495][105620] Updated weights for policy 1, policy_version 662078 (0.0010) [2023-12-26 20:10:52,555][105620] Updated weights for policy 1, policy_version 662088 (0.0010) [2023-12-26 20:10:52,618][105620] Updated weights for policy 1, policy_version 662098 (0.0011) [2023-12-26 20:10:52,856][105692] Updated weights for policy 0, policy_version 661201 (0.0009) [2023-12-26 20:10:52,922][105692] Updated weights for policy 0, policy_version 661211 (0.0008) [2023-12-26 20:10:52,984][105692] Updated weights for policy 0, policy_version 661221 (0.0008) [2023-12-26 20:10:53,362][105620] Updated weights for policy 1, policy_version 662108 (0.0011) [2023-12-26 20:10:53,421][105620] Updated weights for policy 1, policy_version 662118 (0.0010) [2023-12-26 20:10:53,473][105620] Updated weights for policy 1, policy_version 662128 (0.0010) [2023-12-26 20:10:53,568][105692] Updated weights for policy 0, policy_version 661231 (0.0006) [2023-12-26 20:10:53,625][105692] Updated weights for policy 0, policy_version 661241 (0.0005) [2023-12-26 20:10:53,682][105692] Updated weights for policy 0, policy_version 661251 (0.0006) [2023-12-26 20:10:54,254][105620] Updated weights for policy 1, policy_version 662138 (0.0010) [2023-12-26 20:10:54,313][105620] Updated weights for policy 1, policy_version 662148 (0.0010) [2023-12-26 20:10:54,375][105620] Updated weights for policy 1, policy_version 662158 (0.0010) [2023-12-26 20:10:54,402][105692] Updated weights for policy 0, policy_version 661261 (0.0007) [2023-12-26 20:10:54,437][105620] Updated weights for policy 1, policy_version 662168 (0.0009) [2023-12-26 20:10:54,463][105692] Updated weights for policy 0, policy_version 661271 (0.0008) [2023-12-26 20:10:54,515][105692] Updated weights for policy 0, policy_version 661281 (0.0007) [2023-12-26 20:10:55,170][105620] Updated weights for policy 1, policy_version 662178 (0.0011) [2023-12-26 20:10:55,219][105620] Updated weights for policy 1, policy_version 662188 (0.0010) [2023-12-26 20:10:55,268][105620] Updated weights for policy 1, policy_version 662198 (0.0010) [2023-12-26 20:10:55,287][105692] Updated weights for policy 0, policy_version 661291 (0.0008) [2023-12-26 20:10:55,335][105692] Updated weights for policy 0, policy_version 661301 (0.0008) [2023-12-26 20:10:55,378][105692] Updated weights for policy 0, policy_version 661311 (0.0007) [2023-12-26 20:10:56,026][105692] Updated weights for policy 0, policy_version 661321 (0.0007) [2023-12-26 20:10:56,044][105620] Updated weights for policy 1, policy_version 662208 (0.0010) [2023-12-26 20:10:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 338870272. Throughput: 0: 9667.2, 1: 9941.9. Samples: 338884600. Policy #0 lag: (min: 8.0, avg: 36.5, max: 40.0) [2023-12-26 20:10:56,063][104569] Avg episode reward: [(0, '8400.983'), (1, '8900.197')] [2023-12-26 20:10:56,078][105692] Updated weights for policy 0, policy_version 661331 (0.0005) [2023-12-26 20:10:56,099][105620] Updated weights for policy 1, policy_version 662218 (0.0010) [2023-12-26 20:10:56,137][105692] Updated weights for policy 0, policy_version 661341 (0.0005) [2023-12-26 20:10:56,147][105620] Updated weights for policy 1, policy_version 662228 (0.0010) [2023-12-26 20:10:56,186][105692] Updated weights for policy 0, policy_version 661351 (0.0006) [2023-12-26 20:10:56,887][105620] Updated weights for policy 1, policy_version 662238 (0.0010) [2023-12-26 20:10:56,938][105620] Updated weights for policy 1, policy_version 662248 (0.0010) [2023-12-26 20:10:56,960][105692] Updated weights for policy 0, policy_version 661361 (0.0008) [2023-12-26 20:10:56,989][105620] Updated weights for policy 1, policy_version 662258 (0.0010) [2023-12-26 20:10:57,018][105692] Updated weights for policy 0, policy_version 661371 (0.0005) [2023-12-26 20:10:57,068][105692] Updated weights for policy 0, policy_version 661381 (0.0008) [2023-12-26 20:10:57,638][105620] Updated weights for policy 1, policy_version 662268 (0.0008) [2023-12-26 20:10:57,694][105620] Updated weights for policy 1, policy_version 662278 (0.0005) [2023-12-26 20:10:57,742][105620] Updated weights for policy 1, policy_version 662288 (0.0009) [2023-12-26 20:10:57,907][105692] Updated weights for policy 0, policy_version 661391 (0.0009) [2023-12-26 20:10:57,963][105692] Updated weights for policy 0, policy_version 661401 (0.0008) [2023-12-26 20:10:58,018][105692] Updated weights for policy 0, policy_version 661411 (0.0008) [2023-12-26 20:10:58,374][105620] Updated weights for policy 1, policy_version 662298 (0.0006) [2023-12-26 20:10:58,442][105620] Updated weights for policy 1, policy_version 662308 (0.0009) [2023-12-26 20:10:58,501][105620] Updated weights for policy 1, policy_version 662318 (0.0011) [2023-12-26 20:10:58,569][105620] Updated weights for policy 1, policy_version 662328 (0.0010) [2023-12-26 20:10:58,904][105692] Updated weights for policy 0, policy_version 661421 (0.0010) [2023-12-26 20:10:58,967][105692] Updated weights for policy 0, policy_version 661431 (0.0009) [2023-12-26 20:10:58,987][105585] KL-divergence is very high: 172.7087 [2023-12-26 20:10:59,030][105692] Updated weights for policy 0, policy_version 661441 (0.0008) [2023-12-26 20:10:59,036][105585] KL-divergence is very high: 334.1462 [2023-12-26 20:10:59,348][105620] Updated weights for policy 1, policy_version 662338 (0.0011) [2023-12-26 20:10:59,414][105620] Updated weights for policy 1, policy_version 662348 (0.0009) [2023-12-26 20:10:59,459][105620] Updated weights for policy 1, policy_version 662358 (0.0010) [2023-12-26 20:10:59,794][105692] Updated weights for policy 0, policy_version 661451 (0.0008) [2023-12-26 20:10:59,853][105692] Updated weights for policy 0, policy_version 661461 (0.0008) [2023-12-26 20:10:59,910][105692] Updated weights for policy 0, policy_version 661472 (0.0010) [2023-12-26 20:11:00,167][105620] Updated weights for policy 1, policy_version 662368 (0.0007) [2023-12-26 20:11:00,237][105620] Updated weights for policy 1, policy_version 662378 (0.0005) [2023-12-26 20:11:00,307][105620] Updated weights for policy 1, policy_version 662388 (0.0005) [2023-12-26 20:11:00,816][105692] Updated weights for policy 0, policy_version 661482 (0.0009) [2023-12-26 20:11:00,825][105620] Updated weights for policy 1, policy_version 662398 (0.0005) [2023-12-26 20:11:00,870][105692] Updated weights for policy 0, policy_version 661492 (0.0009) [2023-12-26 20:11:00,888][105620] Updated weights for policy 1, policy_version 662408 (0.0005) [2023-12-26 20:11:00,918][105692] Updated weights for policy 0, policy_version 661502 (0.0009) [2023-12-26 20:11:00,949][105620] Updated weights for policy 1, policy_version 662418 (0.0005) [2023-12-26 20:11:00,966][105692] Updated weights for policy 0, policy_version 661512 (0.0009) [2023-12-26 20:11:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 338976768. Throughput: 0: 9635.6, 1: 9950.0. Samples: 338942512. Policy #0 lag: (min: 8.0, avg: 36.5, max: 40.0) [2023-12-26 20:11:01,062][104569] Avg episode reward: [(0, '7960.741'), (1, '8988.514')] [2023-12-26 20:11:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000661512_169377792.pth... [2023-12-26 20:11:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000662424_169598976.pth... [2023-12-26 20:11:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000660392_169091072.pth [2023-12-26 20:11:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000661272_169304064.pth [2023-12-26 20:11:01,618][105620] Updated weights for policy 1, policy_version 662428 (0.0007) [2023-12-26 20:11:01,678][105620] Updated weights for policy 1, policy_version 662438 (0.0011) [2023-12-26 20:11:01,740][105620] Updated weights for policy 1, policy_version 662448 (0.0010) [2023-12-26 20:11:01,779][105692] Updated weights for policy 0, policy_version 661522 (0.0008) [2023-12-26 20:11:01,839][105692] Updated weights for policy 0, policy_version 661532 (0.0010) [2023-12-26 20:11:01,887][105692] Updated weights for policy 0, policy_version 661542 (0.0009) [2023-12-26 20:11:02,372][105620] Updated weights for policy 1, policy_version 662458 (0.0007) [2023-12-26 20:11:02,436][105620] Updated weights for policy 1, policy_version 662468 (0.0007) [2023-12-26 20:11:02,501][105620] Updated weights for policy 1, policy_version 662478 (0.0008) [2023-12-26 20:11:02,553][105620] Updated weights for policy 1, policy_version 662488 (0.0008) [2023-12-26 20:11:02,686][105692] Updated weights for policy 0, policy_version 661552 (0.0009) [2023-12-26 20:11:02,750][105692] Updated weights for policy 0, policy_version 661562 (0.0008) [2023-12-26 20:11:02,817][105692] Updated weights for policy 0, policy_version 661572 (0.0008) [2023-12-26 20:11:03,256][105620] Updated weights for policy 1, policy_version 662498 (0.0010) [2023-12-26 20:11:03,300][105620] Updated weights for policy 1, policy_version 662508 (0.0010) [2023-12-26 20:11:03,348][105620] Updated weights for policy 1, policy_version 662518 (0.0010) [2023-12-26 20:11:03,596][105692] Updated weights for policy 0, policy_version 661582 (0.0009) [2023-12-26 20:11:03,656][105692] Updated weights for policy 0, policy_version 661592 (0.0008) [2023-12-26 20:11:03,718][105692] Updated weights for policy 0, policy_version 661602 (0.0008) [2023-12-26 20:11:04,035][105620] Updated weights for policy 1, policy_version 662528 (0.0007) [2023-12-26 20:11:04,091][105620] Updated weights for policy 1, policy_version 662538 (0.0008) [2023-12-26 20:11:04,162][105620] Updated weights for policy 1, policy_version 662548 (0.0006) [2023-12-26 20:11:04,538][105692] Updated weights for policy 0, policy_version 661612 (0.0008) [2023-12-26 20:11:04,602][105692] Updated weights for policy 0, policy_version 661622 (0.0006) [2023-12-26 20:11:04,669][105692] Updated weights for policy 0, policy_version 661632 (0.0006) [2023-12-26 20:11:04,863][105620] Updated weights for policy 1, policy_version 662558 (0.0010) [2023-12-26 20:11:04,913][105620] Updated weights for policy 1, policy_version 662568 (0.0009) [2023-12-26 20:11:04,968][105620] Updated weights for policy 1, policy_version 662578 (0.0010) [2023-12-26 20:11:05,256][105692] Updated weights for policy 0, policy_version 661642 (0.0006) [2023-12-26 20:11:05,324][105692] Updated weights for policy 0, policy_version 661652 (0.0005) [2023-12-26 20:11:05,392][105692] Updated weights for policy 0, policy_version 661662 (0.0006) [2023-12-26 20:11:05,444][105692] Updated weights for policy 0, policy_version 661672 (0.0008) [2023-12-26 20:11:05,708][105620] Updated weights for policy 1, policy_version 662588 (0.0008) [2023-12-26 20:11:05,757][105620] Updated weights for policy 1, policy_version 662598 (0.0005) [2023-12-26 20:11:05,814][105620] Updated weights for policy 1, policy_version 662608 (0.0005) [2023-12-26 20:11:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 339066880. Throughput: 0: 9442.0, 1: 9985.1. Samples: 339055688. Policy #0 lag: (min: 8.0, avg: 36.5, max: 40.0) [2023-12-26 20:11:06,063][104569] Avg episode reward: [(0, '8550.855'), (1, '9078.506')] [2023-12-26 20:11:06,207][105692] Updated weights for policy 0, policy_version 661682 (0.0009) [2023-12-26 20:11:06,264][105692] Updated weights for policy 0, policy_version 661692 (0.0008) [2023-12-26 20:11:06,320][105692] Updated weights for policy 0, policy_version 661702 (0.0008) [2023-12-26 20:11:06,424][105620] Updated weights for policy 1, policy_version 662618 (0.0007) [2023-12-26 20:11:06,490][105620] Updated weights for policy 1, policy_version 662628 (0.0011) [2023-12-26 20:11:06,553][105620] Updated weights for policy 1, policy_version 662638 (0.0010) [2023-12-26 20:11:06,612][105620] Updated weights for policy 1, policy_version 662648 (0.0010) [2023-12-26 20:11:07,061][105692] Updated weights for policy 0, policy_version 661712 (0.0009) [2023-12-26 20:11:07,121][105692] Updated weights for policy 0, policy_version 661722 (0.0008) [2023-12-26 20:11:07,181][105692] Updated weights for policy 0, policy_version 661732 (0.0008) [2023-12-26 20:11:07,336][105620] Updated weights for policy 1, policy_version 662658 (0.0011) [2023-12-26 20:11:07,395][105620] Updated weights for policy 1, policy_version 662668 (0.0010) [2023-12-26 20:11:07,462][105620] Updated weights for policy 1, policy_version 662678 (0.0011) [2023-12-26 20:11:07,997][105692] Updated weights for policy 0, policy_version 661742 (0.0008) [2023-12-26 20:11:08,057][105692] Updated weights for policy 0, policy_version 661752 (0.0008) [2023-12-26 20:11:08,121][105692] Updated weights for policy 0, policy_version 661762 (0.0009) [2023-12-26 20:11:08,128][105620] Updated weights for policy 1, policy_version 662688 (0.0011) [2023-12-26 20:11:08,186][105620] Updated weights for policy 1, policy_version 662698 (0.0010) [2023-12-26 20:11:08,233][105620] Updated weights for policy 1, policy_version 662708 (0.0005) [2023-12-26 20:11:08,880][105692] Updated weights for policy 0, policy_version 661772 (0.0006) [2023-12-26 20:11:08,943][105692] Updated weights for policy 0, policy_version 661782 (0.0008) [2023-12-26 20:11:08,997][105620] Updated weights for policy 1, policy_version 662718 (0.0007) [2023-12-26 20:11:08,999][105692] Updated weights for policy 0, policy_version 661792 (0.0006) [2023-12-26 20:11:09,055][105620] Updated weights for policy 1, policy_version 662728 (0.0008) [2023-12-26 20:11:09,116][105620] Updated weights for policy 1, policy_version 662738 (0.0006) [2023-12-26 20:11:09,763][105692] Updated weights for policy 0, policy_version 661802 (0.0007) [2023-12-26 20:11:09,823][105692] Updated weights for policy 0, policy_version 661812 (0.0006) [2023-12-26 20:11:09,889][105620] Updated weights for policy 1, policy_version 662748 (0.0009) [2023-12-26 20:11:09,900][105692] Updated weights for policy 0, policy_version 661822 (0.0008) [2023-12-26 20:11:09,956][105620] Updated weights for policy 1, policy_version 662758 (0.0008) [2023-12-26 20:11:09,971][105692] Updated weights for policy 0, policy_version 661832 (0.0008) [2023-12-26 20:11:10,027][105620] Updated weights for policy 1, policy_version 662768 (0.0010) [2023-12-26 20:11:10,668][105620] Updated weights for policy 1, policy_version 662778 (0.0008) [2023-12-26 20:11:10,691][105692] Updated weights for policy 0, policy_version 661842 (0.0008) [2023-12-26 20:11:10,729][105620] Updated weights for policy 1, policy_version 662788 (0.0008) [2023-12-26 20:11:10,754][105692] Updated weights for policy 0, policy_version 661852 (0.0008) [2023-12-26 20:11:10,784][105620] Updated weights for policy 1, policy_version 662798 (0.0005) [2023-12-26 20:11:10,815][105692] Updated weights for policy 0, policy_version 661862 (0.0009) [2023-12-26 20:11:10,850][105620] Updated weights for policy 1, policy_version 662808 (0.0005) [2023-12-26 20:11:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 339165184. Throughput: 0: 9445.5, 1: 9937.7. Samples: 339171340. Policy #0 lag: (min: 8.0, avg: 36.5, max: 40.0) [2023-12-26 20:11:11,062][104569] Avg episode reward: [(0, '7395.454'), (1, '9078.395')] [2023-12-26 20:11:11,582][105692] Updated weights for policy 0, policy_version 661872 (0.0008) [2023-12-26 20:11:11,588][105620] Updated weights for policy 1, policy_version 662818 (0.0007) [2023-12-26 20:11:11,641][105692] Updated weights for policy 0, policy_version 661882 (0.0008) [2023-12-26 20:11:11,651][105620] Updated weights for policy 1, policy_version 662828 (0.0007) [2023-12-26 20:11:11,701][105692] Updated weights for policy 0, policy_version 661892 (0.0008) [2023-12-26 20:11:11,715][105620] Updated weights for policy 1, policy_version 662838 (0.0008) [2023-12-26 20:11:12,479][105620] Updated weights for policy 1, policy_version 662848 (0.0008) [2023-12-26 20:11:12,485][105692] Updated weights for policy 0, policy_version 661902 (0.0008) [2023-12-26 20:11:12,538][105692] Updated weights for policy 0, policy_version 661912 (0.0006) [2023-12-26 20:11:12,540][105620] Updated weights for policy 1, policy_version 662858 (0.0008) [2023-12-26 20:11:12,587][105692] Updated weights for policy 0, policy_version 661922 (0.0006) [2023-12-26 20:11:12,597][105620] Updated weights for policy 1, policy_version 662868 (0.0007) [2023-12-26 20:11:13,303][105692] Updated weights for policy 0, policy_version 661932 (0.0008) [2023-12-26 20:11:13,363][105692] Updated weights for policy 0, policy_version 661942 (0.0011) [2023-12-26 20:11:13,396][105620] Updated weights for policy 1, policy_version 662878 (0.0009) [2023-12-26 20:11:13,430][105692] Updated weights for policy 0, policy_version 661952 (0.0011) [2023-12-26 20:11:13,464][105620] Updated weights for policy 1, policy_version 662888 (0.0006) [2023-12-26 20:11:13,518][105620] Updated weights for policy 1, policy_version 662898 (0.0009) [2023-12-26 20:11:14,001][105692] Updated weights for policy 0, policy_version 661962 (0.0009) [2023-12-26 20:11:14,059][105692] Updated weights for policy 0, policy_version 661972 (0.0005) [2023-12-26 20:11:14,113][105692] Updated weights for policy 0, policy_version 661982 (0.0006) [2023-12-26 20:11:14,165][105692] Updated weights for policy 0, policy_version 661992 (0.0010) [2023-12-26 20:11:14,369][105620] Updated weights for policy 1, policy_version 662908 (0.0009) [2023-12-26 20:11:14,425][105620] Updated weights for policy 1, policy_version 662918 (0.0008) [2023-12-26 20:11:14,473][105620] Updated weights for policy 1, policy_version 662928 (0.0008) [2023-12-26 20:11:14,887][105692] Updated weights for policy 0, policy_version 662002 (0.0010) [2023-12-26 20:11:14,946][105692] Updated weights for policy 0, policy_version 662012 (0.0010) [2023-12-26 20:11:15,016][105692] Updated weights for policy 0, policy_version 662022 (0.0011) [2023-12-26 20:11:15,248][105620] Updated weights for policy 1, policy_version 662938 (0.0008) [2023-12-26 20:11:15,302][105620] Updated weights for policy 1, policy_version 662948 (0.0011) [2023-12-26 20:11:15,365][105620] Updated weights for policy 1, policy_version 662958 (0.0011) [2023-12-26 20:11:15,426][105620] Updated weights for policy 1, policy_version 662968 (0.0011) [2023-12-26 20:11:15,675][105692] Updated weights for policy 0, policy_version 662032 (0.0010) [2023-12-26 20:11:15,726][105692] Updated weights for policy 0, policy_version 662042 (0.0010) [2023-12-26 20:11:15,784][105692] Updated weights for policy 0, policy_version 662052 (0.0010) [2023-12-26 20:11:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 339255296. Throughput: 0: 9455.3, 1: 9759.1. Samples: 339225568. Policy #0 lag: (min: 8.0, avg: 36.5, max: 40.0) [2023-12-26 20:11:16,063][104569] Avg episode reward: [(0, '7554.774'), (1, '8985.762')] [2023-12-26 20:11:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000662056_169517056.pth... [2023-12-26 20:11:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000660936_169230336.pth [2023-12-26 20:11:16,102][105620] Updated weights for policy 1, policy_version 662978 (0.0005) [2023-12-26 20:11:16,149][105620] Updated weights for policy 1, policy_version 662988 (0.0005) [2023-12-26 20:11:16,206][105620] Updated weights for policy 1, policy_version 662998 (0.0007) [2023-12-26 20:11:16,214][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000663000_169746432.pth... [2023-12-26 20:11:16,217][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000661848_169451520.pth [2023-12-26 20:11:16,484][105692] Updated weights for policy 0, policy_version 662062 (0.0008) [2023-12-26 20:11:16,535][105692] Updated weights for policy 0, policy_version 662072 (0.0010) [2023-12-26 20:11:16,594][105692] Updated weights for policy 0, policy_version 662082 (0.0010) [2023-12-26 20:11:16,850][105620] Updated weights for policy 1, policy_version 663008 (0.0010) [2023-12-26 20:11:16,911][105620] Updated weights for policy 1, policy_version 663018 (0.0009) [2023-12-26 20:11:16,968][105620] Updated weights for policy 1, policy_version 663028 (0.0008) [2023-12-26 20:11:17,341][105692] Updated weights for policy 0, policy_version 662092 (0.0010) [2023-12-26 20:11:17,399][105692] Updated weights for policy 0, policy_version 662102 (0.0010) [2023-12-26 20:11:17,458][105692] Updated weights for policy 0, policy_version 662112 (0.0010) [2023-12-26 20:11:17,673][105620] Updated weights for policy 1, policy_version 663038 (0.0008) [2023-12-26 20:11:17,728][105620] Updated weights for policy 1, policy_version 663048 (0.0010) [2023-12-26 20:11:17,779][105620] Updated weights for policy 1, policy_version 663058 (0.0010) [2023-12-26 20:11:17,816][105586] KL-divergence is very high: 110.3579 [2023-12-26 20:11:18,122][105692] Updated weights for policy 0, policy_version 662122 (0.0010) [2023-12-26 20:11:18,183][105692] Updated weights for policy 0, policy_version 662132 (0.0009) [2023-12-26 20:11:18,256][105692] Updated weights for policy 0, policy_version 662142 (0.0010) [2023-12-26 20:11:18,314][105692] Updated weights for policy 0, policy_version 662152 (0.0008) [2023-12-26 20:11:18,494][105620] Updated weights for policy 1, policy_version 663068 (0.0011) [2023-12-26 20:11:18,516][105586] KL-divergence is very high: 107.5262 [2023-12-26 20:11:18,555][105620] Updated weights for policy 1, policy_version 663078 (0.0011) [2023-12-26 20:11:18,569][105586] KL-divergence is very high: 115.0238 [2023-12-26 20:11:18,619][105586] KL-divergence is very high: 103.6007 [2023-12-26 20:11:18,619][105620] Updated weights for policy 1, policy_version 663088 (0.0011) [2023-12-26 20:11:19,050][105692] Updated weights for policy 0, policy_version 662162 (0.0006) [2023-12-26 20:11:19,103][105692] Updated weights for policy 0, policy_version 662172 (0.0006) [2023-12-26 20:11:19,163][105692] Updated weights for policy 0, policy_version 662182 (0.0007) [2023-12-26 20:11:19,372][105620] Updated weights for policy 1, policy_version 663098 (0.0011) [2023-12-26 20:11:19,431][105620] Updated weights for policy 1, policy_version 663108 (0.0011) [2023-12-26 20:11:19,490][105620] Updated weights for policy 1, policy_version 663118 (0.0010) [2023-12-26 20:11:19,553][105620] Updated weights for policy 1, policy_version 663128 (0.0009) [2023-12-26 20:11:19,889][105692] Updated weights for policy 0, policy_version 662192 (0.0011) [2023-12-26 20:11:19,955][105692] Updated weights for policy 0, policy_version 662202 (0.0011) [2023-12-26 20:11:20,015][105692] Updated weights for policy 0, policy_version 662212 (0.0011) [2023-12-26 20:11:20,345][105620] Updated weights for policy 1, policy_version 663138 (0.0011) [2023-12-26 20:11:20,407][105620] Updated weights for policy 1, policy_version 663148 (0.0011) [2023-12-26 20:11:20,474][105620] Updated weights for policy 1, policy_version 663158 (0.0011) [2023-12-26 20:11:20,723][105692] Updated weights for policy 0, policy_version 662222 (0.0009) [2023-12-26 20:11:20,793][105692] Updated weights for policy 0, policy_version 662232 (0.0008) [2023-12-26 20:11:20,858][105692] Updated weights for policy 0, policy_version 662242 (0.0008) [2023-12-26 20:11:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 339353600. Throughput: 0: 9488.8, 1: 9722.5. Samples: 339343392. Policy #0 lag: (min: 8.0, avg: 36.5, max: 40.0) [2023-12-26 20:11:21,062][104569] Avg episode reward: [(0, '8467.873'), (1, '8805.044')] [2023-12-26 20:11:21,165][105620] Updated weights for policy 1, policy_version 663168 (0.0007) [2023-12-26 20:11:21,233][105620] Updated weights for policy 1, policy_version 663178 (0.0006) [2023-12-26 20:11:21,300][105620] Updated weights for policy 1, policy_version 663188 (0.0008) [2023-12-26 20:11:21,642][105692] Updated weights for policy 0, policy_version 662252 (0.0008) [2023-12-26 20:11:21,708][105692] Updated weights for policy 0, policy_version 662262 (0.0009) [2023-12-26 20:11:21,770][105692] Updated weights for policy 0, policy_version 662272 (0.0009) [2023-12-26 20:11:21,946][105620] Updated weights for policy 1, policy_version 663198 (0.0008) [2023-12-26 20:11:22,012][105620] Updated weights for policy 1, policy_version 663208 (0.0010) [2023-12-26 20:11:22,086][105620] Updated weights for policy 1, policy_version 663218 (0.0011) [2023-12-26 20:11:22,594][105692] Updated weights for policy 0, policy_version 662282 (0.0009) [2023-12-26 20:11:22,652][105692] Updated weights for policy 0, policy_version 662292 (0.0009) [2023-12-26 20:11:22,724][105692] Updated weights for policy 0, policy_version 662302 (0.0009) [2023-12-26 20:11:22,734][105620] Updated weights for policy 1, policy_version 663228 (0.0009) [2023-12-26 20:11:22,788][105692] Updated weights for policy 0, policy_version 662312 (0.0007) [2023-12-26 20:11:22,798][105620] Updated weights for policy 1, policy_version 663238 (0.0009) [2023-12-26 20:11:22,860][105620] Updated weights for policy 1, policy_version 663248 (0.0010) [2023-12-26 20:11:23,491][105620] Updated weights for policy 1, policy_version 663258 (0.0009) [2023-12-26 20:11:23,549][105620] Updated weights for policy 1, policy_version 663268 (0.0006) [2023-12-26 20:11:23,603][105620] Updated weights for policy 1, policy_version 663278 (0.0008) [2023-12-26 20:11:23,614][105692] Updated weights for policy 0, policy_version 662322 (0.0008) [2023-12-26 20:11:23,652][105620] Updated weights for policy 1, policy_version 663288 (0.0005) [2023-12-26 20:11:23,668][105692] Updated weights for policy 0, policy_version 662332 (0.0010) [2023-12-26 20:11:23,721][105692] Updated weights for policy 0, policy_version 662344 (0.0010) [2023-12-26 20:11:24,200][105620] Updated weights for policy 1, policy_version 663298 (0.0010) [2023-12-26 20:11:24,252][105620] Updated weights for policy 1, policy_version 663308 (0.0010) [2023-12-26 20:11:24,308][105620] Updated weights for policy 1, policy_version 663318 (0.0010) [2023-12-26 20:11:24,515][105692] Updated weights for policy 0, policy_version 662354 (0.0010) [2023-12-26 20:11:24,572][105692] Updated weights for policy 0, policy_version 662365 (0.0010) [2023-12-26 20:11:24,624][105692] Updated weights for policy 0, policy_version 662375 (0.0010) [2023-12-26 20:11:25,008][105620] Updated weights for policy 1, policy_version 663328 (0.0010) [2023-12-26 20:11:25,070][105620] Updated weights for policy 1, policy_version 663338 (0.0010) [2023-12-26 20:11:25,123][105620] Updated weights for policy 1, policy_version 663348 (0.0006) [2023-12-26 20:11:25,496][105692] Updated weights for policy 0, policy_version 662385 (0.0009) [2023-12-26 20:11:25,555][105692] Updated weights for policy 0, policy_version 662395 (0.0010) [2023-12-26 20:11:25,626][105692] Updated weights for policy 0, policy_version 662405 (0.0010) [2023-12-26 20:11:25,669][105620] Updated weights for policy 1, policy_version 663358 (0.0005) [2023-12-26 20:11:25,718][105620] Updated weights for policy 1, policy_version 663368 (0.0005) [2023-12-26 20:11:25,784][105620] Updated weights for policy 1, policy_version 663378 (0.0006) [2023-12-26 20:11:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 339451904. Throughput: 0: 9431.2, 1: 9850.8. Samples: 339459800. Policy #0 lag: (min: 8.0, avg: 36.5, max: 40.0) [2023-12-26 20:11:26,063][104569] Avg episode reward: [(0, '1695.215'), (1, '8986.777')] [2023-12-26 20:11:26,424][105692] Updated weights for policy 0, policy_version 662415 (0.0008) [2023-12-26 20:11:26,451][105620] Updated weights for policy 1, policy_version 663388 (0.0010) [2023-12-26 20:11:26,478][105692] Updated weights for policy 0, policy_version 662425 (0.0006) [2023-12-26 20:11:26,497][105620] Updated weights for policy 1, policy_version 663398 (0.0009) [2023-12-26 20:11:26,540][105692] Updated weights for policy 0, policy_version 662435 (0.0008) [2023-12-26 20:11:26,549][105620] Updated weights for policy 1, policy_version 663408 (0.0005) [2023-12-26 20:11:27,079][105620] Updated weights for policy 1, policy_version 663418 (0.0006) [2023-12-26 20:11:27,126][105620] Updated weights for policy 1, policy_version 663428 (0.0005) [2023-12-26 20:11:27,169][105620] Updated weights for policy 1, policy_version 663438 (0.0005) [2023-12-26 20:11:27,219][105620] Updated weights for policy 1, policy_version 663448 (0.0005) [2023-12-26 20:11:27,325][105692] Updated weights for policy 0, policy_version 662445 (0.0010) [2023-12-26 20:11:27,379][105692] Updated weights for policy 0, policy_version 662456 (0.0010) [2023-12-26 20:11:27,433][105692] Updated weights for policy 0, policy_version 662466 (0.0010) [2023-12-26 20:11:27,816][105620] Updated weights for policy 1, policy_version 663458 (0.0010) [2023-12-26 20:11:27,864][105620] Updated weights for policy 1, policy_version 663468 (0.0010) [2023-12-26 20:11:27,914][105620] Updated weights for policy 1, policy_version 663478 (0.0010) [2023-12-26 20:11:28,260][105692] Updated weights for policy 0, policy_version 662476 (0.0010) [2023-12-26 20:11:28,314][105692] Updated weights for policy 0, policy_version 662489 (0.0010) [2023-12-26 20:11:28,372][105692] Updated weights for policy 0, policy_version 662499 (0.0008) [2023-12-26 20:11:28,551][105620] Updated weights for policy 1, policy_version 663488 (0.0010) [2023-12-26 20:11:28,619][105620] Updated weights for policy 1, policy_version 663498 (0.0010) [2023-12-26 20:11:28,680][105620] Updated weights for policy 1, policy_version 663508 (0.0010) [2023-12-26 20:11:29,255][105692] Updated weights for policy 0, policy_version 662509 (0.0008) [2023-12-26 20:11:29,263][105620] Updated weights for policy 1, policy_version 663518 (0.0010) [2023-12-26 20:11:29,315][105620] Updated weights for policy 1, policy_version 663528 (0.0010) [2023-12-26 20:11:29,317][105692] Updated weights for policy 0, policy_version 662519 (0.0006) [2023-12-26 20:11:29,384][105620] Updated weights for policy 1, policy_version 663538 (0.0011) [2023-12-26 20:11:29,386][105692] Updated weights for policy 0, policy_version 662529 (0.0007) [2023-12-26 20:11:30,116][105692] Updated weights for policy 0, policy_version 662539 (0.0007) [2023-12-26 20:11:30,142][105620] Updated weights for policy 1, policy_version 663548 (0.0009) [2023-12-26 20:11:30,180][105692] Updated weights for policy 0, policy_version 662549 (0.0007) [2023-12-26 20:11:30,195][105620] Updated weights for policy 1, policy_version 663558 (0.0006) [2023-12-26 20:11:30,237][105692] Updated weights for policy 0, policy_version 662559 (0.0007) [2023-12-26 20:11:30,243][105620] Updated weights for policy 1, policy_version 663568 (0.0006) [2023-12-26 20:11:30,920][105620] Updated weights for policy 1, policy_version 663578 (0.0006) [2023-12-26 20:11:30,975][105620] Updated weights for policy 1, policy_version 663588 (0.0005) [2023-12-26 20:11:31,038][105620] Updated weights for policy 1, policy_version 663598 (0.0006) [2023-12-26 20:11:31,050][105692] Updated weights for policy 0, policy_version 662569 (0.0007) [2023-12-26 20:11:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19114.7, 300 sec: 19438.6). Total num frames: 339542016. Throughput: 0: 9405.4, 1: 9946.6. Samples: 339520120. Policy #0 lag: (min: 8.0, avg: 36.5, max: 40.0) [2023-12-26 20:11:31,063][104569] Avg episode reward: [(0, '2055.299'), (1, '9075.874')] [2023-12-26 20:11:31,101][105620] Updated weights for policy 1, policy_version 663608 (0.0007) [2023-12-26 20:11:31,102][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000663608_169902080.pth... [2023-12-26 20:11:31,105][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000662424_169598976.pth [2023-12-26 20:11:31,108][105692] Updated weights for policy 0, policy_version 662579 (0.0008) [2023-12-26 20:11:31,163][105692] Updated weights for policy 0, policy_version 662589 (0.0009) [2023-12-26 20:11:31,209][105692] Updated weights for policy 0, policy_version 662599 (0.0008) [2023-12-26 20:11:31,216][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000662600_169656320.pth... [2023-12-26 20:11:31,220][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000661512_169377792.pth [2023-12-26 20:11:31,786][105620] Updated weights for policy 1, policy_version 663618 (0.0008) [2023-12-26 20:11:31,848][105620] Updated weights for policy 1, policy_version 663628 (0.0008) [2023-12-26 20:11:31,907][105620] Updated weights for policy 1, policy_version 663639 (0.0010) [2023-12-26 20:11:31,985][105692] Updated weights for policy 0, policy_version 662609 (0.0006) [2023-12-26 20:11:32,047][105692] Updated weights for policy 0, policy_version 662619 (0.0007) [2023-12-26 20:11:32,106][105692] Updated weights for policy 0, policy_version 662629 (0.0006) [2023-12-26 20:11:32,631][105620] Updated weights for policy 1, policy_version 663649 (0.0009) [2023-12-26 20:11:32,678][105620] Updated weights for policy 1, policy_version 663659 (0.0009) [2023-12-26 20:11:32,712][105692] Updated weights for policy 0, policy_version 662639 (0.0008) [2023-12-26 20:11:32,723][105620] Updated weights for policy 1, policy_version 663669 (0.0006) [2023-12-26 20:11:32,769][105692] Updated weights for policy 0, policy_version 662649 (0.0009) [2023-12-26 20:11:32,823][105692] Updated weights for policy 0, policy_version 662659 (0.0009) [2023-12-26 20:11:33,507][105620] Updated weights for policy 1, policy_version 663679 (0.0009) [2023-12-26 20:11:33,530][105692] Updated weights for policy 0, policy_version 662669 (0.0008) [2023-12-26 20:11:33,560][105620] Updated weights for policy 1, policy_version 663689 (0.0009) [2023-12-26 20:11:33,575][105692] Updated weights for policy 0, policy_version 662679 (0.0006) [2023-12-26 20:11:33,614][105620] Updated weights for policy 1, policy_version 663699 (0.0007) [2023-12-26 20:11:33,618][105692] Updated weights for policy 0, policy_version 662689 (0.0008) [2023-12-26 20:11:34,341][105620] Updated weights for policy 1, policy_version 663709 (0.0008) [2023-12-26 20:11:34,396][105692] Updated weights for policy 0, policy_version 662699 (0.0007) [2023-12-26 20:11:34,404][105620] Updated weights for policy 1, policy_version 663719 (0.0009) [2023-12-26 20:11:34,456][105692] Updated weights for policy 0, policy_version 662709 (0.0006) [2023-12-26 20:11:34,463][105620] Updated weights for policy 1, policy_version 663729 (0.0009) [2023-12-26 20:11:34,520][105692] Updated weights for policy 0, policy_version 662719 (0.0008) [2023-12-26 20:11:35,220][105620] Updated weights for policy 1, policy_version 663739 (0.0009) [2023-12-26 20:11:35,252][105692] Updated weights for policy 0, policy_version 662729 (0.0009) [2023-12-26 20:11:35,279][105620] Updated weights for policy 1, policy_version 663749 (0.0008) [2023-12-26 20:11:35,306][105692] Updated weights for policy 0, policy_version 662739 (0.0006) [2023-12-26 20:11:35,332][105620] Updated weights for policy 1, policy_version 663759 (0.0006) [2023-12-26 20:11:35,362][105692] Updated weights for policy 0, policy_version 662749 (0.0006) [2023-12-26 20:11:35,418][105692] Updated weights for policy 0, policy_version 662759 (0.0006) [2023-12-26 20:11:36,034][105692] Updated weights for policy 0, policy_version 662769 (0.0006) [2023-12-26 20:11:36,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19114.7, 300 sec: 19438.6). Total num frames: 339640320. Throughput: 0: 9310.7, 1: 9926.9. Samples: 339634836. Policy #0 lag: (min: 8.0, avg: 36.5, max: 40.0) [2023-12-26 20:11:36,062][104569] Avg episode reward: [(0, '6578.216'), (1, '8985.943')] [2023-12-26 20:11:36,086][105692] Updated weights for policy 0, policy_version 662779 (0.0006) [2023-12-26 20:11:36,149][105620] Updated weights for policy 1, policy_version 663769 (0.0007) [2023-12-26 20:11:36,158][105692] Updated weights for policy 0, policy_version 662789 (0.0010) [2023-12-26 20:11:36,214][105620] Updated weights for policy 1, policy_version 663779 (0.0009) [2023-12-26 20:11:36,280][105620] Updated weights for policy 1, policy_version 663789 (0.0008) [2023-12-26 20:11:36,338][105620] Updated weights for policy 1, policy_version 663799 (0.0009) [2023-12-26 20:11:36,901][105692] Updated weights for policy 0, policy_version 662799 (0.0008) [2023-12-26 20:11:36,959][105692] Updated weights for policy 0, policy_version 662809 (0.0009) [2023-12-26 20:11:37,019][105692] Updated weights for policy 0, policy_version 662819 (0.0007) [2023-12-26 20:11:37,028][105620] Updated weights for policy 1, policy_version 663809 (0.0008) [2023-12-26 20:11:37,089][105620] Updated weights for policy 1, policy_version 663819 (0.0009) [2023-12-26 20:11:37,160][105620] Updated weights for policy 1, policy_version 663829 (0.0009) [2023-12-26 20:11:37,735][105692] Updated weights for policy 0, policy_version 662829 (0.0006) [2023-12-26 20:11:37,795][105692] Updated weights for policy 0, policy_version 662839 (0.0005) [2023-12-26 20:11:37,841][105692] Updated weights for policy 0, policy_version 662849 (0.0005) [2023-12-26 20:11:37,939][105620] Updated weights for policy 1, policy_version 663839 (0.0008) [2023-12-26 20:11:37,999][105620] Updated weights for policy 1, policy_version 663849 (0.0008) [2023-12-26 20:11:38,050][105620] Updated weights for policy 1, policy_version 663859 (0.0007) [2023-12-26 20:11:38,530][105692] Updated weights for policy 0, policy_version 662859 (0.0005) [2023-12-26 20:11:38,591][105692] Updated weights for policy 0, policy_version 662869 (0.0011) [2023-12-26 20:11:38,647][105692] Updated weights for policy 0, policy_version 662879 (0.0011) [2023-12-26 20:11:38,829][105620] Updated weights for policy 1, policy_version 663869 (0.0008) [2023-12-26 20:11:38,891][105620] Updated weights for policy 1, policy_version 663879 (0.0008) [2023-12-26 20:11:38,956][105620] Updated weights for policy 1, policy_version 663889 (0.0008) [2023-12-26 20:11:39,361][105692] Updated weights for policy 0, policy_version 662889 (0.0011) [2023-12-26 20:11:39,435][105692] Updated weights for policy 0, policy_version 662900 (0.0009) [2023-12-26 20:11:39,493][105692] Updated weights for policy 0, policy_version 662910 (0.0007) [2023-12-26 20:11:39,553][105692] Updated weights for policy 0, policy_version 662920 (0.0005) [2023-12-26 20:11:39,785][105620] Updated weights for policy 1, policy_version 663899 (0.0008) [2023-12-26 20:11:39,840][105620] Updated weights for policy 1, policy_version 663909 (0.0007) [2023-12-26 20:11:39,906][105620] Updated weights for policy 1, policy_version 663919 (0.0009) [2023-12-26 20:11:40,229][105692] Updated weights for policy 0, policy_version 662930 (0.0009) [2023-12-26 20:11:40,286][105692] Updated weights for policy 0, policy_version 662940 (0.0009) [2023-12-26 20:11:40,333][105692] Updated weights for policy 0, policy_version 662950 (0.0008) [2023-12-26 20:11:40,663][105620] Updated weights for policy 1, policy_version 663929 (0.0009) [2023-12-26 20:11:40,727][105620] Updated weights for policy 1, policy_version 663939 (0.0008) [2023-12-26 20:11:40,795][105620] Updated weights for policy 1, policy_version 663949 (0.0008) [2023-12-26 20:11:40,855][105620] Updated weights for policy 1, policy_version 663959 (0.0008) [2023-12-26 20:11:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19438.7). Total num frames: 339738624. Throughput: 0: 9301.3, 1: 9887.2. Samples: 339748080. Policy #0 lag: (min: 8.0, avg: 36.5, max: 40.0) [2023-12-26 20:11:41,063][104569] Avg episode reward: [(0, '8808.322'), (1, '8986.426')] [2023-12-26 20:11:41,159][105692] Updated weights for policy 0, policy_version 662960 (0.0008) [2023-12-26 20:11:41,224][105692] Updated weights for policy 0, policy_version 662970 (0.0009) [2023-12-26 20:11:41,295][105692] Updated weights for policy 0, policy_version 662980 (0.0009) [2023-12-26 20:11:41,595][105620] Updated weights for policy 1, policy_version 663969 (0.0010) [2023-12-26 20:11:41,659][105620] Updated weights for policy 1, policy_version 663979 (0.0009) [2023-12-26 20:11:41,721][105620] Updated weights for policy 1, policy_version 663989 (0.0009) [2023-12-26 20:11:42,055][105692] Updated weights for policy 0, policy_version 662990 (0.0008) [2023-12-26 20:11:42,110][105692] Updated weights for policy 0, policy_version 663000 (0.0009) [2023-12-26 20:11:42,158][105692] Updated weights for policy 0, policy_version 663010 (0.0008) [2023-12-26 20:11:42,506][105620] Updated weights for policy 1, policy_version 663999 (0.0009) [2023-12-26 20:11:42,583][105620] Updated weights for policy 1, policy_version 664009 (0.0010) [2023-12-26 20:11:42,642][105620] Updated weights for policy 1, policy_version 664019 (0.0009) [2023-12-26 20:11:42,829][105692] Updated weights for policy 0, policy_version 663020 (0.0009) [2023-12-26 20:11:42,891][105692] Updated weights for policy 0, policy_version 663030 (0.0009) [2023-12-26 20:11:42,940][105692] Updated weights for policy 0, policy_version 663040 (0.0007) [2023-12-26 20:11:43,389][105620] Updated weights for policy 1, policy_version 664029 (0.0009) [2023-12-26 20:11:43,451][105620] Updated weights for policy 1, policy_version 664039 (0.0007) [2023-12-26 20:11:43,507][105620] Updated weights for policy 1, policy_version 664049 (0.0008) [2023-12-26 20:11:43,543][105692] Updated weights for policy 0, policy_version 663050 (0.0006) [2023-12-26 20:11:43,602][105692] Updated weights for policy 0, policy_version 663060 (0.0010) [2023-12-26 20:11:43,659][105692] Updated weights for policy 0, policy_version 663070 (0.0010) [2023-12-26 20:11:43,710][105692] Updated weights for policy 0, policy_version 663080 (0.0010) [2023-12-26 20:11:44,307][105620] Updated weights for policy 1, policy_version 664059 (0.0008) [2023-12-26 20:11:44,341][105692] Updated weights for policy 0, policy_version 663090 (0.0007) [2023-12-26 20:11:44,375][105620] Updated weights for policy 1, policy_version 664069 (0.0008) [2023-12-26 20:11:44,390][105692] Updated weights for policy 0, policy_version 663100 (0.0005) [2023-12-26 20:11:44,435][105620] Updated weights for policy 1, policy_version 664079 (0.0009) [2023-12-26 20:11:44,441][105692] Updated weights for policy 0, policy_version 663110 (0.0005) [2023-12-26 20:11:45,118][105692] Updated weights for policy 0, policy_version 663120 (0.0010) [2023-12-26 20:11:45,132][105620] Updated weights for policy 1, policy_version 664089 (0.0009) [2023-12-26 20:11:45,186][105692] Updated weights for policy 0, policy_version 663130 (0.0011) [2023-12-26 20:11:45,200][105620] Updated weights for policy 1, policy_version 664099 (0.0006) [2023-12-26 20:11:45,250][105692] Updated weights for policy 0, policy_version 663140 (0.0011) [2023-12-26 20:11:45,264][105620] Updated weights for policy 1, policy_version 664109 (0.0005) [2023-12-26 20:11:45,329][105620] Updated weights for policy 1, policy_version 664119 (0.0007) [2023-12-26 20:11:45,985][105692] Updated weights for policy 0, policy_version 663150 (0.0011) [2023-12-26 20:11:46,019][105620] Updated weights for policy 1, policy_version 664129 (0.0005) [2023-12-26 20:11:46,033][105692] Updated weights for policy 0, policy_version 663160 (0.0010) [2023-12-26 20:11:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19410.9). Total num frames: 339828736. Throughput: 0: 9336.5, 1: 9831.1. Samples: 339805056. Policy #0 lag: (min: 8.0, avg: 36.5, max: 40.0) [2023-12-26 20:11:46,062][104569] Avg episode reward: [(0, '8805.528'), (1, '8908.901')] [2023-12-26 20:11:46,063][105620] Updated weights for policy 1, policy_version 664139 (0.0005) [2023-12-26 20:11:46,080][105692] Updated weights for policy 0, policy_version 663170 (0.0010) [2023-12-26 20:11:46,114][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000663176_169803776.pth... [2023-12-26 20:11:46,118][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000662056_169517056.pth [2023-12-26 20:11:46,122][105620] Updated weights for policy 1, policy_version 664149 (0.0006) [2023-12-26 20:11:46,140][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000664152_170041344.pth... [2023-12-26 20:11:46,144][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000663000_169746432.pth [2023-12-26 20:11:46,839][105620] Updated weights for policy 1, policy_version 664159 (0.0008) [2023-12-26 20:11:46,847][105692] Updated weights for policy 0, policy_version 663180 (0.0010) [2023-12-26 20:11:46,900][105620] Updated weights for policy 1, policy_version 664169 (0.0007) [2023-12-26 20:11:46,902][105692] Updated weights for policy 0, policy_version 663190 (0.0010) [2023-12-26 20:11:46,950][105620] Updated weights for policy 1, policy_version 664179 (0.0007) [2023-12-26 20:11:46,961][105692] Updated weights for policy 0, policy_version 663200 (0.0010) [2023-12-26 20:11:47,695][105692] Updated weights for policy 0, policy_version 663210 (0.0010) [2023-12-26 20:11:47,713][105620] Updated weights for policy 1, policy_version 664189 (0.0007) [2023-12-26 20:11:47,750][105692] Updated weights for policy 0, policy_version 663220 (0.0010) [2023-12-26 20:11:47,757][105620] Updated weights for policy 1, policy_version 664199 (0.0006) [2023-12-26 20:11:47,803][105620] Updated weights for policy 1, policy_version 664209 (0.0008) [2023-12-26 20:11:47,808][105692] Updated weights for policy 0, policy_version 663230 (0.0010) [2023-12-26 20:11:47,861][105692] Updated weights for policy 0, policy_version 663240 (0.0010) [2023-12-26 20:11:48,577][105620] Updated weights for policy 1, policy_version 664219 (0.0006) [2023-12-26 20:11:48,614][105692] Updated weights for policy 0, policy_version 663250 (0.0011) [2023-12-26 20:11:48,630][105620] Updated weights for policy 1, policy_version 664229 (0.0008) [2023-12-26 20:11:48,670][105692] Updated weights for policy 0, policy_version 663260 (0.0011) [2023-12-26 20:11:48,681][105620] Updated weights for policy 1, policy_version 664239 (0.0007) [2023-12-26 20:11:48,722][105692] Updated weights for policy 0, policy_version 663270 (0.0011) [2023-12-26 20:11:49,492][105620] Updated weights for policy 1, policy_version 664249 (0.0006) [2023-12-26 20:11:49,514][105692] Updated weights for policy 0, policy_version 663280 (0.0010) [2023-12-26 20:11:49,548][105620] Updated weights for policy 1, policy_version 664259 (0.0006) [2023-12-26 20:11:49,563][105692] Updated weights for policy 0, policy_version 663290 (0.0010) [2023-12-26 20:11:49,607][105620] Updated weights for policy 1, policy_version 664269 (0.0006) [2023-12-26 20:11:49,624][105692] Updated weights for policy 0, policy_version 663300 (0.0011) [2023-12-26 20:11:49,668][105620] Updated weights for policy 1, policy_version 664279 (0.0007) [2023-12-26 20:11:50,394][105692] Updated weights for policy 0, policy_version 663310 (0.0011) [2023-12-26 20:11:50,425][105620] Updated weights for policy 1, policy_version 664289 (0.0007) [2023-12-26 20:11:50,450][105692] Updated weights for policy 0, policy_version 663320 (0.0011) [2023-12-26 20:11:50,474][105620] Updated weights for policy 1, policy_version 664299 (0.0007) [2023-12-26 20:11:50,506][105692] Updated weights for policy 0, policy_version 663330 (0.0011) [2023-12-26 20:11:50,528][105620] Updated weights for policy 1, policy_version 664309 (0.0007) [2023-12-26 20:11:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19438.7). Total num frames: 339927040. Throughput: 0: 9470.9, 1: 9716.2. Samples: 339919104. Policy #0 lag: (min: 8.0, avg: 36.5, max: 40.0) [2023-12-26 20:11:51,062][104569] Avg episode reward: [(0, '8716.472'), (1, '8729.888')] [2023-12-26 20:11:51,286][105620] Updated weights for policy 1, policy_version 664319 (0.0008) [2023-12-26 20:11:51,318][105692] Updated weights for policy 0, policy_version 663340 (0.0009) [2023-12-26 20:11:51,346][105620] Updated weights for policy 1, policy_version 664329 (0.0008) [2023-12-26 20:11:51,385][105692] Updated weights for policy 0, policy_version 663350 (0.0009) [2023-12-26 20:11:51,412][105620] Updated weights for policy 1, policy_version 664339 (0.0007) [2023-12-26 20:11:51,439][105692] Updated weights for policy 0, policy_version 663360 (0.0006) [2023-12-26 20:11:52,139][105620] Updated weights for policy 1, policy_version 664349 (0.0009) [2023-12-26 20:11:52,195][105620] Updated weights for policy 1, policy_version 664359 (0.0008) [2023-12-26 20:11:52,234][105692] Updated weights for policy 0, policy_version 663370 (0.0010) [2023-12-26 20:11:52,252][105620] Updated weights for policy 1, policy_version 664369 (0.0007) [2023-12-26 20:11:52,289][105692] Updated weights for policy 0, policy_version 663380 (0.0010) [2023-12-26 20:11:52,354][105692] Updated weights for policy 0, policy_version 663390 (0.0010) [2023-12-26 20:11:52,417][105692] Updated weights for policy 0, policy_version 663400 (0.0011) [2023-12-26 20:11:53,041][105620] Updated weights for policy 1, policy_version 664379 (0.0007) [2023-12-26 20:11:53,090][105620] Updated weights for policy 1, policy_version 664389 (0.0008) [2023-12-26 20:11:53,137][105620] Updated weights for policy 1, policy_version 664399 (0.0007) [2023-12-26 20:11:53,154][105692] Updated weights for policy 0, policy_version 663410 (0.0011) [2023-12-26 20:11:53,202][105692] Updated weights for policy 0, policy_version 663420 (0.0010) [2023-12-26 20:11:53,256][105692] Updated weights for policy 0, policy_version 663430 (0.0010) [2023-12-26 20:11:53,840][105620] Updated weights for policy 1, policy_version 664409 (0.0006) [2023-12-26 20:11:53,903][105620] Updated weights for policy 1, policy_version 664419 (0.0005) [2023-12-26 20:11:53,933][105692] Updated weights for policy 0, policy_version 663440 (0.0009) [2023-12-26 20:11:53,965][105620] Updated weights for policy 1, policy_version 664429 (0.0007) [2023-12-26 20:11:53,997][105692] Updated weights for policy 0, policy_version 663450 (0.0010) [2023-12-26 20:11:54,019][105620] Updated weights for policy 1, policy_version 664439 (0.0008) [2023-12-26 20:11:54,045][105692] Updated weights for policy 0, policy_version 663460 (0.0010) [2023-12-26 20:11:54,676][105620] Updated weights for policy 1, policy_version 664449 (0.0008) [2023-12-26 20:11:54,744][105620] Updated weights for policy 1, policy_version 664459 (0.0005) [2023-12-26 20:11:54,771][105692] Updated weights for policy 0, policy_version 663470 (0.0010) [2023-12-26 20:11:54,808][105620] Updated weights for policy 1, policy_version 664469 (0.0005) [2023-12-26 20:11:54,833][105692] Updated weights for policy 0, policy_version 663480 (0.0010) [2023-12-26 20:11:54,893][105692] Updated weights for policy 0, policy_version 663490 (0.0005) [2023-12-26 20:11:55,436][105692] Updated weights for policy 0, policy_version 663500 (0.0008) [2023-12-26 20:11:55,497][105692] Updated weights for policy 0, policy_version 663510 (0.0010) [2023-12-26 20:11:55,554][105692] Updated weights for policy 0, policy_version 663520 (0.0010) [2023-12-26 20:11:55,561][105620] Updated weights for policy 1, policy_version 664479 (0.0006) [2023-12-26 20:11:55,620][105620] Updated weights for policy 1, policy_version 664489 (0.0007) [2023-12-26 20:11:55,674][105620] Updated weights for policy 1, policy_version 664499 (0.0010) [2023-12-26 20:11:56,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 340025344. Throughput: 0: 9515.1, 1: 9655.1. Samples: 340034008. Policy #0 lag: (min: 8.0, avg: 36.5, max: 40.0) [2023-12-26 20:11:56,063][104569] Avg episode reward: [(0, '8990.992'), (1, '8626.415')] [2023-12-26 20:11:56,187][105692] Updated weights for policy 0, policy_version 663530 (0.0008) [2023-12-26 20:11:56,252][105692] Updated weights for policy 0, policy_version 663540 (0.0010) [2023-12-26 20:11:56,314][105692] Updated weights for policy 0, policy_version 663550 (0.0011) [2023-12-26 20:11:56,375][105692] Updated weights for policy 0, policy_version 663560 (0.0010) [2023-12-26 20:11:56,443][105620] Updated weights for policy 1, policy_version 664509 (0.0009) [2023-12-26 20:11:56,495][105620] Updated weights for policy 1, policy_version 664519 (0.0008) [2023-12-26 20:11:56,542][105620] Updated weights for policy 1, policy_version 664529 (0.0008) [2023-12-26 20:11:57,088][105692] Updated weights for policy 0, policy_version 663570 (0.0010) [2023-12-26 20:11:57,152][105692] Updated weights for policy 0, policy_version 663580 (0.0010) [2023-12-26 20:11:57,206][105692] Updated weights for policy 0, policy_version 663590 (0.0010) [2023-12-26 20:11:57,313][105620] Updated weights for policy 1, policy_version 664539 (0.0008) [2023-12-26 20:11:57,368][105620] Updated weights for policy 1, policy_version 664549 (0.0007) [2023-12-26 20:11:57,436][105620] Updated weights for policy 1, policy_version 664559 (0.0010) [2023-12-26 20:11:57,934][105692] Updated weights for policy 0, policy_version 663600 (0.0010) [2023-12-26 20:11:57,992][105692] Updated weights for policy 0, policy_version 663610 (0.0008) [2023-12-26 20:11:58,035][105620] Updated weights for policy 1, policy_version 664569 (0.0006) [2023-12-26 20:11:58,054][105692] Updated weights for policy 0, policy_version 663620 (0.0005) [2023-12-26 20:11:58,101][105620] Updated weights for policy 1, policy_version 664579 (0.0006) [2023-12-26 20:11:58,178][105620] Updated weights for policy 1, policy_version 664589 (0.0009) [2023-12-26 20:11:58,240][105620] Updated weights for policy 1, policy_version 664599 (0.0011) [2023-12-26 20:11:58,755][105692] Updated weights for policy 0, policy_version 663630 (0.0008) [2023-12-26 20:11:58,817][105692] Updated weights for policy 0, policy_version 663640 (0.0011) [2023-12-26 20:11:58,879][105692] Updated weights for policy 0, policy_version 663650 (0.0011) [2023-12-26 20:11:58,956][105620] Updated weights for policy 1, policy_version 664609 (0.0008) [2023-12-26 20:11:59,009][105620] Updated weights for policy 1, policy_version 664619 (0.0008) [2023-12-26 20:11:59,067][105620] Updated weights for policy 1, policy_version 664629 (0.0005) [2023-12-26 20:11:59,514][105692] Updated weights for policy 0, policy_version 663660 (0.0008) [2023-12-26 20:11:59,576][105692] Updated weights for policy 0, policy_version 663670 (0.0011) [2023-12-26 20:11:59,630][105692] Updated weights for policy 0, policy_version 663680 (0.0010) [2023-12-26 20:11:59,716][105620] Updated weights for policy 1, policy_version 664639 (0.0009) [2023-12-26 20:11:59,776][105620] Updated weights for policy 1, policy_version 664649 (0.0005) [2023-12-26 20:11:59,842][105620] Updated weights for policy 1, policy_version 664659 (0.0007) [2023-12-26 20:12:00,383][105692] Updated weights for policy 0, policy_version 663690 (0.0009) [2023-12-26 20:12:00,445][105692] Updated weights for policy 0, policy_version 663700 (0.0010) [2023-12-26 20:12:00,503][105692] Updated weights for policy 0, policy_version 663710 (0.0010) [2023-12-26 20:12:00,561][105692] Updated weights for policy 0, policy_version 663720 (0.0010) [2023-12-26 20:12:00,598][105620] Updated weights for policy 1, policy_version 664669 (0.0009) [2023-12-26 20:12:00,655][105620] Updated weights for policy 1, policy_version 664679 (0.0010) [2023-12-26 20:12:00,732][105620] Updated weights for policy 1, policy_version 664689 (0.0010) [2023-12-26 20:12:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19114.7, 300 sec: 19438.7). Total num frames: 340123648. Throughput: 0: 9569.5, 1: 9724.4. Samples: 340093792. Policy #0 lag: (min: 8.0, avg: 36.5, max: 40.0) [2023-12-26 20:12:01,062][104569] Avg episode reward: [(0, '9080.401'), (1, '8899.170')] [2023-12-26 20:12:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000663720_169943040.pth... [2023-12-26 20:12:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000664696_170180608.pth... [2023-12-26 20:12:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000662600_169656320.pth [2023-12-26 20:12:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000663608_169902080.pth [2023-12-26 20:12:01,311][105692] Updated weights for policy 0, policy_version 663730 (0.0009) [2023-12-26 20:12:01,364][105692] Updated weights for policy 0, policy_version 663740 (0.0009) [2023-12-26 20:12:01,427][105692] Updated weights for policy 0, policy_version 663750 (0.0008) [2023-12-26 20:12:01,429][105620] Updated weights for policy 1, policy_version 664699 (0.0007) [2023-12-26 20:12:01,492][105620] Updated weights for policy 1, policy_version 664709 (0.0008) [2023-12-26 20:12:01,540][105620] Updated weights for policy 1, policy_version 664719 (0.0009) [2023-12-26 20:12:02,201][105692] Updated weights for policy 0, policy_version 663760 (0.0009) [2023-12-26 20:12:02,258][105620] Updated weights for policy 1, policy_version 664729 (0.0008) [2023-12-26 20:12:02,264][105692] Updated weights for policy 0, policy_version 663770 (0.0009) [2023-12-26 20:12:02,317][105692] Updated weights for policy 0, policy_version 663780 (0.0006) [2023-12-26 20:12:02,321][105620] Updated weights for policy 1, policy_version 664739 (0.0008) [2023-12-26 20:12:02,386][105620] Updated weights for policy 1, policy_version 664749 (0.0009) [2023-12-26 20:12:02,444][105620] Updated weights for policy 1, policy_version 664759 (0.0010) [2023-12-26 20:12:03,014][105692] Updated weights for policy 0, policy_version 663790 (0.0008) [2023-12-26 20:12:03,061][105692] Updated weights for policy 0, policy_version 663800 (0.0008) [2023-12-26 20:12:03,107][105692] Updated weights for policy 0, policy_version 663810 (0.0008) [2023-12-26 20:12:03,139][105620] Updated weights for policy 1, policy_version 664769 (0.0009) [2023-12-26 20:12:03,186][105620] Updated weights for policy 1, policy_version 664779 (0.0008) [2023-12-26 20:12:03,243][105620] Updated weights for policy 1, policy_version 664790 (0.0010) [2023-12-26 20:12:03,691][105692] Updated weights for policy 0, policy_version 663820 (0.0008) [2023-12-26 20:12:03,744][105692] Updated weights for policy 0, policy_version 663830 (0.0005) [2023-12-26 20:12:03,801][105692] Updated weights for policy 0, policy_version 663840 (0.0010) [2023-12-26 20:12:04,054][105620] Updated weights for policy 1, policy_version 664800 (0.0009) [2023-12-26 20:12:04,113][105620] Updated weights for policy 1, policy_version 664810 (0.0009) [2023-12-26 20:12:04,171][105620] Updated weights for policy 1, policy_version 664820 (0.0009) [2023-12-26 20:12:04,463][105692] Updated weights for policy 0, policy_version 663850 (0.0010) [2023-12-26 20:12:04,532][105692] Updated weights for policy 0, policy_version 663860 (0.0010) [2023-12-26 20:12:04,595][105692] Updated weights for policy 0, policy_version 663870 (0.0009) [2023-12-26 20:12:04,658][105692] Updated weights for policy 0, policy_version 663880 (0.0010) [2023-12-26 20:12:04,932][105620] Updated weights for policy 1, policy_version 664830 (0.0008) [2023-12-26 20:12:05,005][105620] Updated weights for policy 1, policy_version 664840 (0.0008) [2023-12-26 20:12:05,057][105620] Updated weights for policy 1, policy_version 664850 (0.0008) [2023-12-26 20:12:05,392][105692] Updated weights for policy 0, policy_version 663890 (0.0011) [2023-12-26 20:12:05,439][105692] Updated weights for policy 0, policy_version 663900 (0.0010) [2023-12-26 20:12:05,497][105692] Updated weights for policy 0, policy_version 663910 (0.0010) [2023-12-26 20:12:05,814][105620] Updated weights for policy 1, policy_version 664860 (0.0008) [2023-12-26 20:12:05,875][105620] Updated weights for policy 1, policy_version 664870 (0.0008) [2023-12-26 20:12:05,930][105620] Updated weights for policy 1, policy_version 664880 (0.0008) [2023-12-26 20:12:06,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 340221952. Throughput: 0: 9554.2, 1: 9729.0. Samples: 340211136. Policy #0 lag: (min: 8.0, avg: 36.5, max: 40.0) [2023-12-26 20:12:06,063][104569] Avg episode reward: [(0, '8896.732'), (1, '8905.110')] [2023-12-26 20:12:06,241][105692] Updated weights for policy 0, policy_version 663920 (0.0011) [2023-12-26 20:12:06,297][105692] Updated weights for policy 0, policy_version 663930 (0.0010) [2023-12-26 20:12:06,362][105692] Updated weights for policy 0, policy_version 663940 (0.0011) [2023-12-26 20:12:06,696][105620] Updated weights for policy 1, policy_version 664890 (0.0008) [2023-12-26 20:12:06,760][105620] Updated weights for policy 1, policy_version 664900 (0.0008) [2023-12-26 20:12:06,823][105620] Updated weights for policy 1, policy_version 664910 (0.0008) [2023-12-26 20:12:06,883][105620] Updated weights for policy 1, policy_version 664920 (0.0007) [2023-12-26 20:12:07,114][105692] Updated weights for policy 0, policy_version 663950 (0.0011) [2023-12-26 20:12:07,179][105692] Updated weights for policy 0, policy_version 663960 (0.0008) [2023-12-26 20:12:07,245][105692] Updated weights for policy 0, policy_version 663970 (0.0008) [2023-12-26 20:12:07,485][105620] Updated weights for policy 1, policy_version 664930 (0.0008) [2023-12-26 20:12:07,535][105620] Updated weights for policy 1, policy_version 664940 (0.0008) [2023-12-26 20:12:07,583][105620] Updated weights for policy 1, policy_version 664950 (0.0008) [2023-12-26 20:12:07,915][105692] Updated weights for policy 0, policy_version 663980 (0.0009) [2023-12-26 20:12:07,979][105692] Updated weights for policy 0, policy_version 663990 (0.0008) [2023-12-26 20:12:08,049][105692] Updated weights for policy 0, policy_version 664000 (0.0009) [2023-12-26 20:12:08,266][105620] Updated weights for policy 1, policy_version 664960 (0.0009) [2023-12-26 20:12:08,328][105620] Updated weights for policy 1, policy_version 664970 (0.0008) [2023-12-26 20:12:08,391][105620] Updated weights for policy 1, policy_version 664980 (0.0006) [2023-12-26 20:12:08,692][105692] Updated weights for policy 0, policy_version 664010 (0.0011) [2023-12-26 20:12:08,755][105692] Updated weights for policy 0, policy_version 664020 (0.0008) [2023-12-26 20:12:08,816][105692] Updated weights for policy 0, policy_version 664030 (0.0010) [2023-12-26 20:12:08,875][105692] Updated weights for policy 0, policy_version 664040 (0.0010) [2023-12-26 20:12:09,048][105620] Updated weights for policy 1, policy_version 664990 (0.0009) [2023-12-26 20:12:09,110][105620] Updated weights for policy 1, policy_version 665000 (0.0010) [2023-12-26 20:12:09,163][105620] Updated weights for policy 1, policy_version 665010 (0.0009) [2023-12-26 20:12:09,664][105692] Updated weights for policy 0, policy_version 664050 (0.0008) [2023-12-26 20:12:09,737][105692] Updated weights for policy 0, policy_version 664060 (0.0007) [2023-12-26 20:12:09,808][105692] Updated weights for policy 0, policy_version 664070 (0.0009) [2023-12-26 20:12:09,957][105620] Updated weights for policy 1, policy_version 665020 (0.0009) [2023-12-26 20:12:10,022][105620] Updated weights for policy 1, policy_version 665030 (0.0009) [2023-12-26 20:12:10,091][105620] Updated weights for policy 1, policy_version 665040 (0.0008) [2023-12-26 20:12:10,534][105692] Updated weights for policy 0, policy_version 664080 (0.0010) [2023-12-26 20:12:10,594][105692] Updated weights for policy 0, policy_version 664090 (0.0010) [2023-12-26 20:12:10,649][105692] Updated weights for policy 0, policy_version 664100 (0.0011) [2023-12-26 20:12:10,811][105620] Updated weights for policy 1, policy_version 665050 (0.0007) [2023-12-26 20:12:10,856][105620] Updated weights for policy 1, policy_version 665060 (0.0008) [2023-12-26 20:12:10,909][105620] Updated weights for policy 1, policy_version 665070 (0.0008) [2023-12-26 20:12:10,959][105620] Updated weights for policy 1, policy_version 665080 (0.0008) [2023-12-26 20:12:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 340320256. Throughput: 0: 9647.0, 1: 9597.9. Samples: 340325820. Policy #0 lag: (min: 8.0, avg: 36.5, max: 40.0) [2023-12-26 20:12:11,062][104569] Avg episode reward: [(0, '8463.785'), (1, '8371.693')] [2023-12-26 20:12:11,384][105692] Updated weights for policy 0, policy_version 664110 (0.0010) [2023-12-26 20:12:11,446][105692] Updated weights for policy 0, policy_version 664120 (0.0011) [2023-12-26 20:12:11,510][105692] Updated weights for policy 0, policy_version 664130 (0.0011) [2023-12-26 20:12:11,811][105620] Updated weights for policy 1, policy_version 665090 (0.0006) [2023-12-26 20:12:11,875][105620] Updated weights for policy 1, policy_version 665100 (0.0006) [2023-12-26 20:12:11,937][105620] Updated weights for policy 1, policy_version 665110 (0.0005) [2023-12-26 20:12:12,279][105692] Updated weights for policy 0, policy_version 664140 (0.0010) [2023-12-26 20:12:12,341][105692] Updated weights for policy 0, policy_version 664150 (0.0009) [2023-12-26 20:12:12,415][105692] Updated weights for policy 0, policy_version 664160 (0.0009) [2023-12-26 20:12:12,652][105620] Updated weights for policy 1, policy_version 665120 (0.0008) [2023-12-26 20:12:12,705][105620] Updated weights for policy 1, policy_version 665130 (0.0005) [2023-12-26 20:12:12,761][105620] Updated weights for policy 1, policy_version 665140 (0.0005) [2023-12-26 20:12:13,129][105692] Updated weights for policy 0, policy_version 664170 (0.0006) [2023-12-26 20:12:13,188][105692] Updated weights for policy 0, policy_version 664180 (0.0010) [2023-12-26 20:12:13,246][105692] Updated weights for policy 0, policy_version 664190 (0.0010) [2023-12-26 20:12:13,301][105692] Updated weights for policy 0, policy_version 664200 (0.0009) [2023-12-26 20:12:13,314][105620] Updated weights for policy 1, policy_version 665150 (0.0009) [2023-12-26 20:12:13,369][105620] Updated weights for policy 1, policy_version 665160 (0.0006) [2023-12-26 20:12:13,417][105620] Updated weights for policy 1, policy_version 665170 (0.0008) [2023-12-26 20:12:13,933][105692] Updated weights for policy 0, policy_version 664210 (0.0009) [2023-12-26 20:12:13,993][105692] Updated weights for policy 0, policy_version 664220 (0.0009) [2023-12-26 20:12:14,055][105692] Updated weights for policy 0, policy_version 664230 (0.0009) [2023-12-26 20:12:14,146][105620] Updated weights for policy 1, policy_version 665180 (0.0009) [2023-12-26 20:12:14,203][105620] Updated weights for policy 1, policy_version 665190 (0.0009) [2023-12-26 20:12:14,263][105620] Updated weights for policy 1, policy_version 665200 (0.0008) [2023-12-26 20:12:14,817][105692] Updated weights for policy 0, policy_version 664240 (0.0006) [2023-12-26 20:12:14,877][105692] Updated weights for policy 0, policy_version 664250 (0.0009) [2023-12-26 20:12:14,932][105692] Updated weights for policy 0, policy_version 664260 (0.0008) [2023-12-26 20:12:15,025][105620] Updated weights for policy 1, policy_version 665210 (0.0009) [2023-12-26 20:12:15,074][105620] Updated weights for policy 1, policy_version 665220 (0.0005) [2023-12-26 20:12:15,121][105620] Updated weights for policy 1, policy_version 665230 (0.0005) [2023-12-26 20:12:15,169][105620] Updated weights for policy 1, policy_version 665240 (0.0006) [2023-12-26 20:12:15,727][105692] Updated weights for policy 0, policy_version 664270 (0.0010) [2023-12-26 20:12:15,791][105692] Updated weights for policy 0, policy_version 664280 (0.0010) [2023-12-26 20:12:15,853][105692] Updated weights for policy 0, policy_version 664290 (0.0011) [2023-12-26 20:12:15,865][105620] Updated weights for policy 1, policy_version 665250 (0.0006) [2023-12-26 20:12:15,915][105620] Updated weights for policy 1, policy_version 665260 (0.0008) [2023-12-26 20:12:15,975][105620] Updated weights for policy 1, policy_version 665270 (0.0008) [2023-12-26 20:12:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 340418560. Throughput: 0: 9677.1, 1: 9494.7. Samples: 340382856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:12:16,063][104569] Avg episode reward: [(0, '8556.523'), (1, '8455.225')] [2023-12-26 20:12:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000664296_170090496.pth... [2023-12-26 20:12:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000665272_170328064.pth... [2023-12-26 20:12:16,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000663176_169803776.pth [2023-12-26 20:12:16,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000664152_170041344.pth [2023-12-26 20:12:16,577][105692] Updated weights for policy 0, policy_version 664300 (0.0010) [2023-12-26 20:12:16,635][105692] Updated weights for policy 0, policy_version 664310 (0.0010) [2023-12-26 20:12:16,692][105692] Updated weights for policy 0, policy_version 664320 (0.0010) [2023-12-26 20:12:16,745][105620] Updated weights for policy 1, policy_version 665280 (0.0006) [2023-12-26 20:12:16,802][105620] Updated weights for policy 1, policy_version 665290 (0.0008) [2023-12-26 20:12:16,861][105620] Updated weights for policy 1, policy_version 665300 (0.0008) [2023-12-26 20:12:17,451][105692] Updated weights for policy 0, policy_version 664330 (0.0010) [2023-12-26 20:12:17,515][105692] Updated weights for policy 0, policy_version 664340 (0.0010) [2023-12-26 20:12:17,573][105692] Updated weights for policy 0, policy_version 664350 (0.0010) [2023-12-26 20:12:17,622][105620] Updated weights for policy 1, policy_version 665310 (0.0006) [2023-12-26 20:12:17,631][105692] Updated weights for policy 0, policy_version 664360 (0.0010) [2023-12-26 20:12:17,673][105620] Updated weights for policy 1, policy_version 665320 (0.0008) [2023-12-26 20:12:17,720][105620] Updated weights for policy 1, policy_version 665330 (0.0008) [2023-12-26 20:12:18,369][105692] Updated weights for policy 0, policy_version 664370 (0.0011) [2023-12-26 20:12:18,432][105692] Updated weights for policy 0, policy_version 664380 (0.0010) [2023-12-26 20:12:18,491][105692] Updated weights for policy 0, policy_version 664390 (0.0010) [2023-12-26 20:12:18,506][105620] Updated weights for policy 1, policy_version 665340 (0.0008) [2023-12-26 20:12:18,566][105620] Updated weights for policy 1, policy_version 665350 (0.0008) [2023-12-26 20:12:18,629][105620] Updated weights for policy 1, policy_version 665360 (0.0008) [2023-12-26 20:12:19,264][105692] Updated weights for policy 0, policy_version 664400 (0.0009) [2023-12-26 20:12:19,332][105692] Updated weights for policy 0, policy_version 664410 (0.0008) [2023-12-26 20:12:19,391][105620] Updated weights for policy 1, policy_version 665370 (0.0008) [2023-12-26 20:12:19,401][105692] Updated weights for policy 0, policy_version 664420 (0.0009) [2023-12-26 20:12:19,455][105620] Updated weights for policy 1, policy_version 665380 (0.0008) [2023-12-26 20:12:19,526][105620] Updated weights for policy 1, policy_version 665390 (0.0009) [2023-12-26 20:12:19,583][105620] Updated weights for policy 1, policy_version 665400 (0.0009) [2023-12-26 20:12:20,109][105692] Updated weights for policy 0, policy_version 664430 (0.0007) [2023-12-26 20:12:20,168][105692] Updated weights for policy 0, policy_version 664440 (0.0008) [2023-12-26 20:12:20,231][105692] Updated weights for policy 0, policy_version 664450 (0.0008) [2023-12-26 20:12:20,373][105620] Updated weights for policy 1, policy_version 665410 (0.0009) [2023-12-26 20:12:20,439][105620] Updated weights for policy 1, policy_version 665420 (0.0009) [2023-12-26 20:12:20,506][105620] Updated weights for policy 1, policy_version 665430 (0.0009) [2023-12-26 20:12:20,971][105692] Updated weights for policy 0, policy_version 664460 (0.0008) [2023-12-26 20:12:21,035][105692] Updated weights for policy 0, policy_version 664471 (0.0008) [2023-12-26 20:12:21,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19114.7, 300 sec: 19410.9). Total num frames: 340500480. Throughput: 0: 9678.3, 1: 9440.2. Samples: 340495168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:12:21,062][104569] Avg episode reward: [(0, '9263.017'), (1, '8804.854')] [2023-12-26 20:12:21,097][105692] Updated weights for policy 0, policy_version 664481 (0.0009) [2023-12-26 20:12:21,306][105620] Updated weights for policy 1, policy_version 665440 (0.0008) [2023-12-26 20:12:21,367][105620] Updated weights for policy 1, policy_version 665450 (0.0009) [2023-12-26 20:12:21,433][105620] Updated weights for policy 1, policy_version 665460 (0.0008) [2023-12-26 20:12:21,915][105692] Updated weights for policy 0, policy_version 664491 (0.0007) [2023-12-26 20:12:21,983][105692] Updated weights for policy 0, policy_version 664501 (0.0005) [2023-12-26 20:12:22,052][105692] Updated weights for policy 0, policy_version 664511 (0.0008) [2023-12-26 20:12:22,334][105620] Updated weights for policy 1, policy_version 665470 (0.0009) [2023-12-26 20:12:22,402][105620] Updated weights for policy 1, policy_version 665480 (0.0008) [2023-12-26 20:12:22,460][105620] Updated weights for policy 1, policy_version 665490 (0.0009) [2023-12-26 20:12:22,691][105692] Updated weights for policy 0, policy_version 664521 (0.0009) [2023-12-26 20:12:22,757][105692] Updated weights for policy 0, policy_version 664531 (0.0006) [2023-12-26 20:12:22,824][105692] Updated weights for policy 0, policy_version 664541 (0.0006) [2023-12-26 20:12:22,878][105692] Updated weights for policy 0, policy_version 664551 (0.0008) [2023-12-26 20:12:23,275][105620] Updated weights for policy 1, policy_version 665500 (0.0009) [2023-12-26 20:12:23,340][105620] Updated weights for policy 1, policy_version 665510 (0.0010) [2023-12-26 20:12:23,402][105620] Updated weights for policy 1, policy_version 665520 (0.0010) [2023-12-26 20:12:23,453][105692] Updated weights for policy 0, policy_version 664561 (0.0006) [2023-12-26 20:12:23,506][105692] Updated weights for policy 0, policy_version 664571 (0.0005) [2023-12-26 20:12:23,568][105692] Updated weights for policy 0, policy_version 664581 (0.0007) [2023-12-26 20:12:24,132][105620] Updated weights for policy 1, policy_version 665530 (0.0010) [2023-12-26 20:12:24,185][105692] Updated weights for policy 0, policy_version 664591 (0.0006) [2023-12-26 20:12:24,199][105620] Updated weights for policy 1, policy_version 665540 (0.0011) [2023-12-26 20:12:24,234][105692] Updated weights for policy 0, policy_version 664601 (0.0006) [2023-12-26 20:12:24,267][105620] Updated weights for policy 1, policy_version 665550 (0.0010) [2023-12-26 20:12:24,291][105692] Updated weights for policy 0, policy_version 664611 (0.0006) [2023-12-26 20:12:24,329][105620] Updated weights for policy 1, policy_version 665560 (0.0010) [2023-12-26 20:12:24,934][105692] Updated weights for policy 0, policy_version 664621 (0.0006) [2023-12-26 20:12:25,005][105692] Updated weights for policy 0, policy_version 664631 (0.0010) [2023-12-26 20:12:25,032][105620] Updated weights for policy 1, policy_version 665570 (0.0011) [2023-12-26 20:12:25,052][105692] Updated weights for policy 0, policy_version 664641 (0.0009) [2023-12-26 20:12:25,085][105620] Updated weights for policy 1, policy_version 665580 (0.0006) [2023-12-26 20:12:25,146][105620] Updated weights for policy 1, policy_version 665590 (0.0008) [2023-12-26 20:12:25,635][105692] Updated weights for policy 0, policy_version 664651 (0.0008) [2023-12-26 20:12:25,698][105692] Updated weights for policy 0, policy_version 664661 (0.0006) [2023-12-26 20:12:25,754][105692] Updated weights for policy 0, policy_version 664671 (0.0008) [2023-12-26 20:12:25,790][105620] Updated weights for policy 1, policy_version 665600 (0.0010) [2023-12-26 20:12:25,838][105620] Updated weights for policy 1, policy_version 665610 (0.0010) [2023-12-26 20:12:25,903][105620] Updated weights for policy 1, policy_version 665620 (0.0010) [2023-12-26 20:12:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 340606976. Throughput: 0: 9745.0, 1: 9454.0. Samples: 340612040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:12:26,063][104569] Avg episode reward: [(0, '9261.977'), (1, '8582.321')] [2023-12-26 20:12:26,307][105692] Updated weights for policy 0, policy_version 664681 (0.0005) [2023-12-26 20:12:26,353][105692] Updated weights for policy 0, policy_version 664691 (0.0005) [2023-12-26 20:12:26,411][105692] Updated weights for policy 0, policy_version 664701 (0.0005) [2023-12-26 20:12:26,474][105692] Updated weights for policy 0, policy_version 664711 (0.0007) [2023-12-26 20:12:26,665][105620] Updated weights for policy 1, policy_version 665630 (0.0010) [2023-12-26 20:12:26,719][105620] Updated weights for policy 1, policy_version 665640 (0.0008) [2023-12-26 20:12:26,768][105620] Updated weights for policy 1, policy_version 665650 (0.0005) [2023-12-26 20:12:27,158][105692] Updated weights for policy 0, policy_version 664722 (0.0009) [2023-12-26 20:12:27,208][105692] Updated weights for policy 0, policy_version 664732 (0.0005) [2023-12-26 20:12:27,263][105692] Updated weights for policy 0, policy_version 664742 (0.0005) [2023-12-26 20:12:27,303][105620] Updated weights for policy 1, policy_version 665660 (0.0007) [2023-12-26 20:12:27,366][105620] Updated weights for policy 1, policy_version 665670 (0.0011) [2023-12-26 20:12:27,415][105620] Updated weights for policy 1, policy_version 665680 (0.0010) [2023-12-26 20:12:27,954][105692] Updated weights for policy 0, policy_version 664752 (0.0009) [2023-12-26 20:12:28,001][105692] Updated weights for policy 0, policy_version 664762 (0.0009) [2023-12-26 20:12:28,052][105620] Updated weights for policy 1, policy_version 665690 (0.0009) [2023-12-26 20:12:28,062][105692] Updated weights for policy 0, policy_version 664772 (0.0009) [2023-12-26 20:12:28,103][105620] Updated weights for policy 1, policy_version 665700 (0.0005) [2023-12-26 20:12:28,155][105620] Updated weights for policy 1, policy_version 665710 (0.0006) [2023-12-26 20:12:28,209][105620] Updated weights for policy 1, policy_version 665720 (0.0005) [2023-12-26 20:12:28,766][105692] Updated weights for policy 0, policy_version 664782 (0.0008) [2023-12-26 20:12:28,771][105620] Updated weights for policy 1, policy_version 665730 (0.0010) [2023-12-26 20:12:28,821][105620] Updated weights for policy 1, policy_version 665740 (0.0011) [2023-12-26 20:12:28,826][105692] Updated weights for policy 0, policy_version 664792 (0.0007) [2023-12-26 20:12:28,866][105620] Updated weights for policy 1, policy_version 665750 (0.0011) [2023-12-26 20:12:28,884][105692] Updated weights for policy 0, policy_version 664802 (0.0008) [2023-12-26 20:12:29,530][105620] Updated weights for policy 1, policy_version 665760 (0.0010) [2023-12-26 20:12:29,561][105692] Updated weights for policy 0, policy_version 664812 (0.0006) [2023-12-26 20:12:29,579][105620] Updated weights for policy 1, policy_version 665770 (0.0008) [2023-12-26 20:12:29,609][105692] Updated weights for policy 0, policy_version 664822 (0.0006) [2023-12-26 20:12:29,630][105620] Updated weights for policy 1, policy_version 665780 (0.0008) [2023-12-26 20:12:29,647][105586] KL-divergence is very high: 112.8896 [2023-12-26 20:12:29,663][105692] Updated weights for policy 0, policy_version 664832 (0.0005) [2023-12-26 20:12:30,249][105692] Updated weights for policy 0, policy_version 664842 (0.0005) [2023-12-26 20:12:30,313][105692] Updated weights for policy 0, policy_version 664852 (0.0005) [2023-12-26 20:12:30,368][105692] Updated weights for policy 0, policy_version 664862 (0.0005) [2023-12-26 20:12:30,393][105620] Updated weights for policy 1, policy_version 665790 (0.0008) [2023-12-26 20:12:30,419][105692] Updated weights for policy 0, policy_version 664872 (0.0007) [2023-12-26 20:12:30,421][105586] KL-divergence is very high: 113.4341 [2023-12-26 20:12:30,447][105620] Updated weights for policy 1, policy_version 665800 (0.0007) [2023-12-26 20:12:30,507][105620] Updated weights for policy 1, policy_version 665810 (0.0008) [2023-12-26 20:12:30,949][105692] Updated weights for policy 0, policy_version 664882 (0.0005) [2023-12-26 20:12:31,002][105692] Updated weights for policy 0, policy_version 664892 (0.0005) [2023-12-26 20:12:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 340705280. Throughput: 0: 9788.8, 1: 9576.0. Samples: 340676472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:12:31,062][104569] Avg episode reward: [(0, '8989.971'), (1, '5517.616')] [2023-12-26 20:12:31,063][105692] Updated weights for policy 0, policy_version 664902 (0.0009) [2023-12-26 20:12:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000665816_170467328.pth... [2023-12-26 20:12:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000664696_170180608.pth [2023-12-26 20:12:31,074][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000664904_170246144.pth... [2023-12-26 20:12:31,087][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000663720_169943040.pth [2023-12-26 20:12:31,397][105620] Updated weights for policy 1, policy_version 665820 (0.0009) [2023-12-26 20:12:31,455][105620] Updated weights for policy 1, policy_version 665830 (0.0009) [2023-12-26 20:12:31,505][105620] Updated weights for policy 1, policy_version 665840 (0.0010) [2023-12-26 20:12:31,803][105692] Updated weights for policy 0, policy_version 664912 (0.0009) [2023-12-26 20:12:31,855][105692] Updated weights for policy 0, policy_version 664922 (0.0009) [2023-12-26 20:12:31,905][105692] Updated weights for policy 0, policy_version 664932 (0.0008) [2023-12-26 20:12:32,257][105620] Updated weights for policy 1, policy_version 665850 (0.0008) [2023-12-26 20:12:32,308][105620] Updated weights for policy 1, policy_version 665860 (0.0005) [2023-12-26 20:12:32,370][105620] Updated weights for policy 1, policy_version 665870 (0.0006) [2023-12-26 20:12:32,429][105620] Updated weights for policy 1, policy_version 665880 (0.0009) [2023-12-26 20:12:32,613][105692] Updated weights for policy 0, policy_version 664942 (0.0007) [2023-12-26 20:12:32,678][105692] Updated weights for policy 0, policy_version 664952 (0.0008) [2023-12-26 20:12:32,732][105692] Updated weights for policy 0, policy_version 664962 (0.0009) [2023-12-26 20:12:33,106][105620] Updated weights for policy 1, policy_version 665890 (0.0008) [2023-12-26 20:12:33,161][105620] Updated weights for policy 1, policy_version 665900 (0.0008) [2023-12-26 20:12:33,211][105620] Updated weights for policy 1, policy_version 665910 (0.0009) [2023-12-26 20:12:33,433][105692] Updated weights for policy 0, policy_version 664972 (0.0009) [2023-12-26 20:12:33,481][105692] Updated weights for policy 0, policy_version 664982 (0.0010) [2023-12-26 20:12:33,530][105692] Updated weights for policy 0, policy_version 664992 (0.0010) [2023-12-26 20:12:33,988][105620] Updated weights for policy 1, policy_version 665920 (0.0008) [2023-12-26 20:12:34,039][105620] Updated weights for policy 1, policy_version 665930 (0.0009) [2023-12-26 20:12:34,094][105620] Updated weights for policy 1, policy_version 665942 (0.0010) [2023-12-26 20:12:34,229][105692] Updated weights for policy 0, policy_version 665002 (0.0010) [2023-12-26 20:12:34,297][105692] Updated weights for policy 0, policy_version 665012 (0.0005) [2023-12-26 20:12:34,361][105692] Updated weights for policy 0, policy_version 665022 (0.0005) [2023-12-26 20:12:34,419][105692] Updated weights for policy 0, policy_version 665032 (0.0010) [2023-12-26 20:12:34,761][105620] Updated weights for policy 1, policy_version 665952 (0.0009) [2023-12-26 20:12:34,819][105620] Updated weights for policy 1, policy_version 665962 (0.0009) [2023-12-26 20:12:34,878][105620] Updated weights for policy 1, policy_version 665972 (0.0009) [2023-12-26 20:12:35,103][105692] Updated weights for policy 0, policy_version 665042 (0.0010) [2023-12-26 20:12:35,153][105692] Updated weights for policy 0, policy_version 665052 (0.0010) [2023-12-26 20:12:35,208][105692] Updated weights for policy 0, policy_version 665062 (0.0010) [2023-12-26 20:12:35,704][105620] Updated weights for policy 1, policy_version 665982 (0.0009) [2023-12-26 20:12:35,761][105620] Updated weights for policy 1, policy_version 665992 (0.0010) [2023-12-26 20:12:35,818][105620] Updated weights for policy 1, policy_version 666002 (0.0009) [2023-12-26 20:12:35,824][105692] Updated weights for policy 0, policy_version 665072 (0.0007) [2023-12-26 20:12:35,873][105692] Updated weights for policy 0, policy_version 665082 (0.0010) [2023-12-26 20:12:35,918][105692] Updated weights for policy 0, policy_version 665092 (0.0010) [2023-12-26 20:12:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 340811776. Throughput: 0: 9895.2, 1: 9611.4. Samples: 340796908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:12:36,063][104569] Avg episode reward: [(0, '8898.288'), (1, '4050.119')] [2023-12-26 20:12:36,511][105620] Updated weights for policy 1, policy_version 666012 (0.0006) [2023-12-26 20:12:36,568][105620] Updated weights for policy 1, policy_version 666022 (0.0006) [2023-12-26 20:12:36,626][105620] Updated weights for policy 1, policy_version 666032 (0.0006) [2023-12-26 20:12:36,654][105692] Updated weights for policy 0, policy_version 665102 (0.0010) [2023-12-26 20:12:36,718][105692] Updated weights for policy 0, policy_version 665112 (0.0011) [2023-12-26 20:12:36,774][105692] Updated weights for policy 0, policy_version 665122 (0.0011) [2023-12-26 20:12:37,184][105620] Updated weights for policy 1, policy_version 666042 (0.0006) [2023-12-26 20:12:37,247][105620] Updated weights for policy 1, policy_version 666052 (0.0005) [2023-12-26 20:12:37,304][105620] Updated weights for policy 1, policy_version 666062 (0.0005) [2023-12-26 20:12:37,363][105620] Updated weights for policy 1, policy_version 666072 (0.0008) [2023-12-26 20:12:37,375][105692] Updated weights for policy 0, policy_version 665132 (0.0008) [2023-12-26 20:12:37,433][105692] Updated weights for policy 0, policy_version 665142 (0.0005) [2023-12-26 20:12:37,486][105692] Updated weights for policy 0, policy_version 665152 (0.0005) [2023-12-26 20:12:37,912][105620] Updated weights for policy 1, policy_version 666082 (0.0006) [2023-12-26 20:12:37,978][105620] Updated weights for policy 1, policy_version 666092 (0.0006) [2023-12-26 20:12:38,044][105620] Updated weights for policy 1, policy_version 666102 (0.0006) [2023-12-26 20:12:38,143][105692] Updated weights for policy 0, policy_version 665162 (0.0005) [2023-12-26 20:12:38,210][105692] Updated weights for policy 0, policy_version 665172 (0.0007) [2023-12-26 20:12:38,258][105692] Updated weights for policy 0, policy_version 665182 (0.0011) [2023-12-26 20:12:38,322][105692] Updated weights for policy 0, policy_version 665192 (0.0011) [2023-12-26 20:12:38,641][105620] Updated weights for policy 1, policy_version 666112 (0.0010) [2023-12-26 20:12:38,697][105620] Updated weights for policy 1, policy_version 666122 (0.0011) [2023-12-26 20:12:38,742][105620] Updated weights for policy 1, policy_version 666132 (0.0010) [2023-12-26 20:12:39,068][105692] Updated weights for policy 0, policy_version 665202 (0.0011) [2023-12-26 20:12:39,120][105692] Updated weights for policy 0, policy_version 665212 (0.0010) [2023-12-26 20:12:39,177][105692] Updated weights for policy 0, policy_version 665222 (0.0008) [2023-12-26 20:12:39,408][105620] Updated weights for policy 1, policy_version 666142 (0.0009) [2023-12-26 20:12:39,474][105620] Updated weights for policy 1, policy_version 666152 (0.0008) [2023-12-26 20:12:39,531][105620] Updated weights for policy 1, policy_version 666162 (0.0011) [2023-12-26 20:12:39,974][105692] Updated weights for policy 0, policy_version 665232 (0.0008) [2023-12-26 20:12:40,041][105692] Updated weights for policy 0, policy_version 665242 (0.0009) [2023-12-26 20:12:40,105][105692] Updated weights for policy 0, policy_version 665252 (0.0006) [2023-12-26 20:12:40,312][105620] Updated weights for policy 1, policy_version 666172 (0.0010) [2023-12-26 20:12:40,378][105620] Updated weights for policy 1, policy_version 666182 (0.0008) [2023-12-26 20:12:40,440][105620] Updated weights for policy 1, policy_version 666192 (0.0008) [2023-12-26 20:12:40,771][105692] Updated weights for policy 0, policy_version 665262 (0.0006) [2023-12-26 20:12:40,842][105692] Updated weights for policy 0, policy_version 665272 (0.0008) [2023-12-26 20:12:40,901][105692] Updated weights for policy 0, policy_version 665282 (0.0009) [2023-12-26 20:12:41,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 340910080. Throughput: 0: 9949.6, 1: 9702.7. Samples: 340918360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:12:41,062][104569] Avg episode reward: [(0, '9169.677'), (1, '7075.055')] [2023-12-26 20:12:41,239][105620] Updated weights for policy 1, policy_version 666202 (0.0007) [2023-12-26 20:12:41,302][105620] Updated weights for policy 1, policy_version 666212 (0.0009) [2023-12-26 20:12:41,364][105620] Updated weights for policy 1, policy_version 666222 (0.0009) [2023-12-26 20:12:41,424][105620] Updated weights for policy 1, policy_version 666232 (0.0008) [2023-12-26 20:12:41,624][105692] Updated weights for policy 0, policy_version 665292 (0.0007) [2023-12-26 20:12:41,689][105692] Updated weights for policy 0, policy_version 665302 (0.0009) [2023-12-26 20:12:41,754][105692] Updated weights for policy 0, policy_version 665312 (0.0010) [2023-12-26 20:12:42,108][105620] Updated weights for policy 1, policy_version 666242 (0.0008) [2023-12-26 20:12:42,172][105620] Updated weights for policy 1, policy_version 666252 (0.0010) [2023-12-26 20:12:42,228][105620] Updated weights for policy 1, policy_version 666262 (0.0009) [2023-12-26 20:12:42,511][105692] Updated weights for policy 0, policy_version 665322 (0.0008) [2023-12-26 20:12:42,577][105692] Updated weights for policy 0, policy_version 665332 (0.0009) [2023-12-26 20:12:42,640][105692] Updated weights for policy 0, policy_version 665342 (0.0009) [2023-12-26 20:12:42,704][105692] Updated weights for policy 0, policy_version 665352 (0.0009) [2023-12-26 20:12:42,943][105620] Updated weights for policy 1, policy_version 666272 (0.0009) [2023-12-26 20:12:43,006][105620] Updated weights for policy 1, policy_version 666282 (0.0009) [2023-12-26 20:12:43,065][105620] Updated weights for policy 1, policy_version 666292 (0.0007) [2023-12-26 20:12:43,466][105692] Updated weights for policy 0, policy_version 665362 (0.0005) [2023-12-26 20:12:43,507][105692] Updated weights for policy 0, policy_version 665372 (0.0005) [2023-12-26 20:12:43,555][105692] Updated weights for policy 0, policy_version 665382 (0.0005) [2023-12-26 20:12:43,767][105620] Updated weights for policy 1, policy_version 666302 (0.0008) [2023-12-26 20:12:43,822][105620] Updated weights for policy 1, policy_version 666312 (0.0009) [2023-12-26 20:12:43,880][105620] Updated weights for policy 1, policy_version 666322 (0.0013) [2023-12-26 20:12:44,158][105692] Updated weights for policy 0, policy_version 665392 (0.0005) [2023-12-26 20:12:44,209][105692] Updated weights for policy 0, policy_version 665402 (0.0005) [2023-12-26 20:12:44,256][105692] Updated weights for policy 0, policy_version 665412 (0.0006) [2023-12-26 20:12:44,597][105620] Updated weights for policy 1, policy_version 666332 (0.0009) [2023-12-26 20:12:44,642][105620] Updated weights for policy 1, policy_version 666342 (0.0010) [2023-12-26 20:12:44,687][105620] Updated weights for policy 1, policy_version 666352 (0.0010) [2023-12-26 20:12:44,922][105692] Updated weights for policy 0, policy_version 665422 (0.0008) [2023-12-26 20:12:44,973][105692] Updated weights for policy 0, policy_version 665432 (0.0005) [2023-12-26 20:12:45,029][105692] Updated weights for policy 0, policy_version 665442 (0.0006) [2023-12-26 20:12:45,468][105620] Updated weights for policy 1, policy_version 666362 (0.0009) [2023-12-26 20:12:45,530][105620] Updated weights for policy 1, policy_version 666372 (0.0005) [2023-12-26 20:12:45,580][105620] Updated weights for policy 1, policy_version 666382 (0.0006) [2023-12-26 20:12:45,639][105620] Updated weights for policy 1, policy_version 666392 (0.0006) [2023-12-26 20:12:45,640][105692] Updated weights for policy 0, policy_version 665452 (0.0007) [2023-12-26 20:12:45,695][105692] Updated weights for policy 0, policy_version 665462 (0.0009) [2023-12-26 20:12:45,753][105692] Updated weights for policy 0, policy_version 665472 (0.0005) [2023-12-26 20:12:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.7, 300 sec: 19494.2). Total num frames: 341008384. Throughput: 0: 9907.5, 1: 9676.3. Samples: 340975068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:12:46,063][104569] Avg episode reward: [(0, '8804.884'), (1, '8164.330')] [2023-12-26 20:12:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000665480_170393600.pth... [2023-12-26 20:12:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000666392_170614784.pth... [2023-12-26 20:12:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000664296_170090496.pth [2023-12-26 20:12:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000665272_170328064.pth [2023-12-26 20:12:46,286][105620] Updated weights for policy 1, policy_version 666402 (0.0010) [2023-12-26 20:12:46,336][105692] Updated weights for policy 0, policy_version 665482 (0.0005) [2023-12-26 20:12:46,353][105620] Updated weights for policy 1, policy_version 666412 (0.0006) [2023-12-26 20:12:46,380][105692] Updated weights for policy 0, policy_version 665492 (0.0005) [2023-12-26 20:12:46,423][105620] Updated weights for policy 1, policy_version 666422 (0.0006) [2023-12-26 20:12:46,427][105692] Updated weights for policy 0, policy_version 665502 (0.0006) [2023-12-26 20:12:46,478][105692] Updated weights for policy 0, policy_version 665512 (0.0009) [2023-12-26 20:12:47,058][105620] Updated weights for policy 1, policy_version 666432 (0.0008) [2023-12-26 20:12:47,123][105620] Updated weights for policy 1, policy_version 666442 (0.0009) [2023-12-26 20:12:47,176][105692] Updated weights for policy 0, policy_version 665522 (0.0008) [2023-12-26 20:12:47,179][105620] Updated weights for policy 1, policy_version 666452 (0.0007) [2023-12-26 20:12:47,231][105692] Updated weights for policy 0, policy_version 665532 (0.0009) [2023-12-26 20:12:47,284][105692] Updated weights for policy 0, policy_version 665543 (0.0009) [2023-12-26 20:12:47,848][105620] Updated weights for policy 1, policy_version 666462 (0.0008) [2023-12-26 20:12:47,896][105620] Updated weights for policy 1, policy_version 666472 (0.0009) [2023-12-26 20:12:47,950][105620] Updated weights for policy 1, policy_version 666482 (0.0009) [2023-12-26 20:12:48,072][105692] Updated weights for policy 0, policy_version 665553 (0.0010) [2023-12-26 20:12:48,126][105692] Updated weights for policy 0, policy_version 665563 (0.0009) [2023-12-26 20:12:48,183][105692] Updated weights for policy 0, policy_version 665573 (0.0008) [2023-12-26 20:12:48,756][105620] Updated weights for policy 1, policy_version 666492 (0.0009) [2023-12-26 20:12:48,819][105620] Updated weights for policy 1, policy_version 666502 (0.0009) [2023-12-26 20:12:48,882][105620] Updated weights for policy 1, policy_version 666512 (0.0009) [2023-12-26 20:12:48,956][105692] Updated weights for policy 0, policy_version 665583 (0.0008) [2023-12-26 20:12:49,017][105692] Updated weights for policy 0, policy_version 665593 (0.0009) [2023-12-26 20:12:49,074][105692] Updated weights for policy 0, policy_version 665603 (0.0009) [2023-12-26 20:12:49,566][105620] Updated weights for policy 1, policy_version 666522 (0.0009) [2023-12-26 20:12:49,620][105620] Updated weights for policy 1, policy_version 666532 (0.0005) [2023-12-26 20:12:49,672][105620] Updated weights for policy 1, policy_version 666542 (0.0006) [2023-12-26 20:12:49,720][105620] Updated weights for policy 1, policy_version 666552 (0.0009) [2023-12-26 20:12:49,867][105692] Updated weights for policy 0, policy_version 665613 (0.0009) [2023-12-26 20:12:49,921][105692] Updated weights for policy 0, policy_version 665623 (0.0006) [2023-12-26 20:12:49,986][105692] Updated weights for policy 0, policy_version 665633 (0.0008) [2023-12-26 20:12:50,480][105620] Updated weights for policy 1, policy_version 666562 (0.0009) [2023-12-26 20:12:50,528][105620] Updated weights for policy 1, policy_version 666572 (0.0010) [2023-12-26 20:12:50,585][105620] Updated weights for policy 1, policy_version 666582 (0.0010) [2023-12-26 20:12:50,748][105692] Updated weights for policy 0, policy_version 665643 (0.0009) [2023-12-26 20:12:50,809][105692] Updated weights for policy 0, policy_version 665653 (0.0010) [2023-12-26 20:12:50,870][105692] Updated weights for policy 0, policy_version 665663 (0.0010) [2023-12-26 20:12:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 341106688. Throughput: 0: 9938.6, 1: 9721.8. Samples: 341095852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:12:51,062][104569] Avg episode reward: [(0, '8622.850'), (1, '8986.907')] [2023-12-26 20:12:51,241][105620] Updated weights for policy 1, policy_version 666592 (0.0011) [2023-12-26 20:12:51,304][105620] Updated weights for policy 1, policy_version 666602 (0.0009) [2023-12-26 20:12:51,367][105620] Updated weights for policy 1, policy_version 666612 (0.0010) [2023-12-26 20:12:51,670][105692] Updated weights for policy 0, policy_version 665673 (0.0009) [2023-12-26 20:12:51,741][105692] Updated weights for policy 0, policy_version 665683 (0.0007) [2023-12-26 20:12:51,802][105692] Updated weights for policy 0, policy_version 665693 (0.0006) [2023-12-26 20:12:51,871][105692] Updated weights for policy 0, policy_version 665703 (0.0006) [2023-12-26 20:12:52,142][105620] Updated weights for policy 1, policy_version 666622 (0.0010) [2023-12-26 20:12:52,201][105620] Updated weights for policy 1, policy_version 666632 (0.0011) [2023-12-26 20:12:52,268][105620] Updated weights for policy 1, policy_version 666642 (0.0010) [2023-12-26 20:12:52,573][105692] Updated weights for policy 0, policy_version 665713 (0.0009) [2023-12-26 20:12:52,639][105692] Updated weights for policy 0, policy_version 665723 (0.0010) [2023-12-26 20:12:52,696][105692] Updated weights for policy 0, policy_version 665733 (0.0008) [2023-12-26 20:12:52,981][105620] Updated weights for policy 1, policy_version 666652 (0.0010) [2023-12-26 20:12:53,039][105620] Updated weights for policy 1, policy_version 666662 (0.0009) [2023-12-26 20:12:53,102][105620] Updated weights for policy 1, policy_version 666672 (0.0010) [2023-12-26 20:12:53,317][105692] Updated weights for policy 0, policy_version 665743 (0.0009) [2023-12-26 20:12:53,367][105692] Updated weights for policy 0, policy_version 665753 (0.0008) [2023-12-26 20:12:53,418][105692] Updated weights for policy 0, policy_version 665763 (0.0009) [2023-12-26 20:12:53,843][105620] Updated weights for policy 1, policy_version 666682 (0.0009) [2023-12-26 20:12:53,902][105620] Updated weights for policy 1, policy_version 666692 (0.0006) [2023-12-26 20:12:53,963][105620] Updated weights for policy 1, policy_version 666702 (0.0010) [2023-12-26 20:12:54,018][105620] Updated weights for policy 1, policy_version 666712 (0.0010) [2023-12-26 20:12:54,197][105692] Updated weights for policy 0, policy_version 665773 (0.0009) [2023-12-26 20:12:54,209][105585] KL-divergence is very high: 144.6770 [2023-12-26 20:12:54,248][105585] KL-divergence is very high: 173.7196 [2023-12-26 20:12:54,250][105692] Updated weights for policy 0, policy_version 665784 (0.0010) [2023-12-26 20:12:54,298][105692] Updated weights for policy 0, policy_version 665794 (0.0007) [2023-12-26 20:12:54,587][105620] Updated weights for policy 1, policy_version 666722 (0.0011) [2023-12-26 20:12:54,641][105620] Updated weights for policy 1, policy_version 666732 (0.0010) [2023-12-26 20:12:54,693][105620] Updated weights for policy 1, policy_version 666742 (0.0010) [2023-12-26 20:12:55,048][105692] Updated weights for policy 0, policy_version 665804 (0.0005) [2023-12-26 20:12:55,107][105692] Updated weights for policy 0, policy_version 665814 (0.0006) [2023-12-26 20:12:55,161][105692] Updated weights for policy 0, policy_version 665824 (0.0008) [2023-12-26 20:12:55,405][105620] Updated weights for policy 1, policy_version 666752 (0.0010) [2023-12-26 20:12:55,471][105620] Updated weights for policy 1, policy_version 666762 (0.0009) [2023-12-26 20:12:55,533][105620] Updated weights for policy 1, policy_version 666772 (0.0010) [2023-12-26 20:12:55,752][105692] Updated weights for policy 0, policy_version 665834 (0.0005) [2023-12-26 20:12:55,816][105692] Updated weights for policy 0, policy_version 665844 (0.0006) [2023-12-26 20:12:55,879][105692] Updated weights for policy 0, policy_version 665854 (0.0008) [2023-12-26 20:12:55,947][105692] Updated weights for policy 0, policy_version 665864 (0.0005) [2023-12-26 20:12:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 341204992. Throughput: 0: 9953.8, 1: 9743.9. Samples: 341212220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:12:56,063][104569] Avg episode reward: [(0, '8715.385'), (1, '9172.332')] [2023-12-26 20:12:56,300][105620] Updated weights for policy 1, policy_version 666782 (0.0009) [2023-12-26 20:12:56,348][105620] Updated weights for policy 1, policy_version 666792 (0.0010) [2023-12-26 20:12:56,395][105620] Updated weights for policy 1, policy_version 666802 (0.0010) [2023-12-26 20:12:56,521][105692] Updated weights for policy 0, policy_version 665874 (0.0008) [2023-12-26 20:12:56,573][105692] Updated weights for policy 0, policy_version 665884 (0.0009) [2023-12-26 20:12:56,623][105692] Updated weights for policy 0, policy_version 665894 (0.0007) [2023-12-26 20:12:57,044][105620] Updated weights for policy 1, policy_version 666812 (0.0008) [2023-12-26 20:12:57,103][105620] Updated weights for policy 1, policy_version 666822 (0.0006) [2023-12-26 20:12:57,155][105620] Updated weights for policy 1, policy_version 666832 (0.0007) [2023-12-26 20:12:57,256][105692] Updated weights for policy 0, policy_version 665904 (0.0005) [2023-12-26 20:12:57,303][105692] Updated weights for policy 0, policy_version 665914 (0.0005) [2023-12-26 20:12:57,362][105692] Updated weights for policy 0, policy_version 665924 (0.0008) [2023-12-26 20:12:57,748][105620] Updated weights for policy 1, policy_version 666842 (0.0006) [2023-12-26 20:12:57,801][105620] Updated weights for policy 1, policy_version 666852 (0.0007) [2023-12-26 20:12:57,853][105620] Updated weights for policy 1, policy_version 666862 (0.0009) [2023-12-26 20:12:57,897][105620] Updated weights for policy 1, policy_version 666872 (0.0010) [2023-12-26 20:12:57,961][105692] Updated weights for policy 0, policy_version 665934 (0.0007) [2023-12-26 20:12:58,034][105692] Updated weights for policy 0, policy_version 665944 (0.0005) [2023-12-26 20:12:58,089][105692] Updated weights for policy 0, policy_version 665954 (0.0007) [2023-12-26 20:12:58,540][105620] Updated weights for policy 1, policy_version 666882 (0.0011) [2023-12-26 20:12:58,605][105620] Updated weights for policy 1, policy_version 666892 (0.0008) [2023-12-26 20:12:58,664][105620] Updated weights for policy 1, policy_version 666902 (0.0011) [2023-12-26 20:12:58,821][105692] Updated weights for policy 0, policy_version 665964 (0.0009) [2023-12-26 20:12:58,892][105692] Updated weights for policy 0, policy_version 665974 (0.0008) [2023-12-26 20:12:58,961][105692] Updated weights for policy 0, policy_version 665984 (0.0008) [2023-12-26 20:12:59,532][105620] Updated weights for policy 1, policy_version 666912 (0.0010) [2023-12-26 20:12:59,589][105620] Updated weights for policy 1, policy_version 666923 (0.0009) [2023-12-26 20:12:59,642][105620] Updated weights for policy 1, policy_version 666933 (0.0009) [2023-12-26 20:12:59,673][105692] Updated weights for policy 0, policy_version 665994 (0.0008) [2023-12-26 20:12:59,732][105692] Updated weights for policy 0, policy_version 666004 (0.0005) [2023-12-26 20:12:59,796][105692] Updated weights for policy 0, policy_version 666014 (0.0007) [2023-12-26 20:12:59,868][105692] Updated weights for policy 0, policy_version 666024 (0.0009) [2023-12-26 20:13:00,426][105620] Updated weights for policy 1, policy_version 666943 (0.0007) [2023-12-26 20:13:00,483][105620] Updated weights for policy 1, policy_version 666953 (0.0008) [2023-12-26 20:13:00,552][105620] Updated weights for policy 1, policy_version 666963 (0.0007) [2023-12-26 20:13:00,580][105692] Updated weights for policy 0, policy_version 666034 (0.0007) [2023-12-26 20:13:00,637][105692] Updated weights for policy 0, policy_version 666044 (0.0009) [2023-12-26 20:13:00,685][105692] Updated weights for policy 0, policy_version 666054 (0.0009) [2023-12-26 20:13:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 341303296. Throughput: 0: 10064.1, 1: 9792.2. Samples: 341276388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:13:01,062][104569] Avg episode reward: [(0, '8807.294'), (1, '8898.628')] [2023-12-26 20:13:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000666056_170541056.pth... [2023-12-26 20:13:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000666968_170762240.pth... [2023-12-26 20:13:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000665816_170467328.pth [2023-12-26 20:13:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000664904_170246144.pth [2023-12-26 20:13:01,170][105620] Updated weights for policy 1, policy_version 666973 (0.0009) [2023-12-26 20:13:01,228][105620] Updated weights for policy 1, policy_version 666983 (0.0009) [2023-12-26 20:13:01,291][105620] Updated weights for policy 1, policy_version 666993 (0.0009) [2023-12-26 20:13:01,470][105692] Updated weights for policy 0, policy_version 666064 (0.0006) [2023-12-26 20:13:01,516][105692] Updated weights for policy 0, policy_version 666074 (0.0005) [2023-12-26 20:13:01,565][105692] Updated weights for policy 0, policy_version 666084 (0.0005) [2023-12-26 20:13:02,081][105620] Updated weights for policy 1, policy_version 667003 (0.0009) [2023-12-26 20:13:02,135][105620] Updated weights for policy 1, policy_version 667013 (0.0009) [2023-12-26 20:13:02,192][105620] Updated weights for policy 1, policy_version 667023 (0.0006) [2023-12-26 20:13:02,291][105692] Updated weights for policy 0, policy_version 666094 (0.0007) [2023-12-26 20:13:02,356][105692] Updated weights for policy 0, policy_version 666104 (0.0009) [2023-12-26 20:13:02,416][105692] Updated weights for policy 0, policy_version 666114 (0.0009) [2023-12-26 20:13:02,948][105620] Updated weights for policy 1, policy_version 667033 (0.0008) [2023-12-26 20:13:03,006][105620] Updated weights for policy 1, policy_version 667043 (0.0010) [2023-12-26 20:13:03,065][105620] Updated weights for policy 1, policy_version 667053 (0.0010) [2023-12-26 20:13:03,120][105620] Updated weights for policy 1, policy_version 667063 (0.0010) [2023-12-26 20:13:03,129][105692] Updated weights for policy 0, policy_version 666124 (0.0008) [2023-12-26 20:13:03,176][105692] Updated weights for policy 0, policy_version 666134 (0.0008) [2023-12-26 20:13:03,228][105692] Updated weights for policy 0, policy_version 666144 (0.0008) [2023-12-26 20:13:03,717][105620] Updated weights for policy 1, policy_version 667073 (0.0008) [2023-12-26 20:13:03,775][105620] Updated weights for policy 1, policy_version 667083 (0.0007) [2023-12-26 20:13:03,835][105620] Updated weights for policy 1, policy_version 667093 (0.0008) [2023-12-26 20:13:04,062][105692] Updated weights for policy 0, policy_version 666154 (0.0008) [2023-12-26 20:13:04,118][105692] Updated weights for policy 0, policy_version 666164 (0.0010) [2023-12-26 20:13:04,178][105692] Updated weights for policy 0, policy_version 666174 (0.0009) [2023-12-26 20:13:04,234][105692] Updated weights for policy 0, policy_version 666184 (0.0009) [2023-12-26 20:13:04,523][105620] Updated weights for policy 1, policy_version 667103 (0.0009) [2023-12-26 20:13:04,581][105620] Updated weights for policy 1, policy_version 667113 (0.0008) [2023-12-26 20:13:04,649][105620] Updated weights for policy 1, policy_version 667123 (0.0006) [2023-12-26 20:13:05,005][105692] Updated weights for policy 0, policy_version 666194 (0.0009) [2023-12-26 20:13:05,060][105692] Updated weights for policy 0, policy_version 666204 (0.0008) [2023-12-26 20:13:05,115][105692] Updated weights for policy 0, policy_version 666214 (0.0008) [2023-12-26 20:13:05,344][105620] Updated weights for policy 1, policy_version 667133 (0.0005) [2023-12-26 20:13:05,401][105620] Updated weights for policy 1, policy_version 667143 (0.0007) [2023-12-26 20:13:05,455][105620] Updated weights for policy 1, policy_version 667153 (0.0010) [2023-12-26 20:13:05,821][105692] Updated weights for policy 0, policy_version 666224 (0.0006) [2023-12-26 20:13:05,884][105692] Updated weights for policy 0, policy_version 666234 (0.0006) [2023-12-26 20:13:05,942][105692] Updated weights for policy 0, policy_version 666244 (0.0008) [2023-12-26 20:13:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 341401600. Throughput: 0: 10050.7, 1: 9837.5. Samples: 341390140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:13:06,063][104569] Avg episode reward: [(0, '8722.200'), (1, '8529.149')] [2023-12-26 20:13:06,174][105620] Updated weights for policy 1, policy_version 667163 (0.0009) [2023-12-26 20:13:06,238][105620] Updated weights for policy 1, policy_version 667173 (0.0007) [2023-12-26 20:13:06,299][105620] Updated weights for policy 1, policy_version 667183 (0.0009) [2023-12-26 20:13:06,685][105692] Updated weights for policy 0, policy_version 666254 (0.0009) [2023-12-26 20:13:06,740][105692] Updated weights for policy 0, policy_version 666264 (0.0010) [2023-12-26 20:13:06,795][105692] Updated weights for policy 0, policy_version 666274 (0.0008) [2023-12-26 20:13:06,860][105620] Updated weights for policy 1, policy_version 667193 (0.0010) [2023-12-26 20:13:06,929][105620] Updated weights for policy 1, policy_version 667203 (0.0009) [2023-12-26 20:13:06,987][105620] Updated weights for policy 1, policy_version 667213 (0.0009) [2023-12-26 20:13:07,049][105620] Updated weights for policy 1, policy_version 667223 (0.0009) [2023-12-26 20:13:07,607][105620] Updated weights for policy 1, policy_version 667233 (0.0006) [2023-12-26 20:13:07,670][105620] Updated weights for policy 1, policy_version 667243 (0.0008) [2023-12-26 20:13:07,692][105692] Updated weights for policy 0, policy_version 666284 (0.0008) [2023-12-26 20:13:07,732][105620] Updated weights for policy 1, policy_version 667253 (0.0010) [2023-12-26 20:13:07,747][105692] Updated weights for policy 0, policy_version 666294 (0.0005) [2023-12-26 20:13:07,800][105692] Updated weights for policy 0, policy_version 666304 (0.0005) [2023-12-26 20:13:08,343][105692] Updated weights for policy 0, policy_version 666314 (0.0006) [2023-12-26 20:13:08,404][105692] Updated weights for policy 0, policy_version 666324 (0.0011) [2023-12-26 20:13:08,405][105620] Updated weights for policy 1, policy_version 667263 (0.0010) [2023-12-26 20:13:08,465][105620] Updated weights for policy 1, policy_version 667273 (0.0011) [2023-12-26 20:13:08,467][105692] Updated weights for policy 0, policy_version 666334 (0.0011) [2023-12-26 20:13:08,521][105620] Updated weights for policy 1, policy_version 667283 (0.0010) [2023-12-26 20:13:08,523][105692] Updated weights for policy 0, policy_version 666344 (0.0011) [2023-12-26 20:13:09,220][105620] Updated weights for policy 1, policy_version 667293 (0.0008) [2023-12-26 20:13:09,263][105692] Updated weights for policy 0, policy_version 666354 (0.0010) [2023-12-26 20:13:09,281][105620] Updated weights for policy 1, policy_version 667303 (0.0007) [2023-12-26 20:13:09,319][105692] Updated weights for policy 0, policy_version 666364 (0.0010) [2023-12-26 20:13:09,342][105620] Updated weights for policy 1, policy_version 667313 (0.0007) [2023-12-26 20:13:09,380][105692] Updated weights for policy 0, policy_version 666374 (0.0011) [2023-12-26 20:13:09,959][105620] Updated weights for policy 1, policy_version 667323 (0.0008) [2023-12-26 20:13:10,023][105620] Updated weights for policy 1, policy_version 667333 (0.0008) [2023-12-26 20:13:10,087][105620] Updated weights for policy 1, policy_version 667343 (0.0008) [2023-12-26 20:13:10,146][105692] Updated weights for policy 0, policy_version 666384 (0.0010) [2023-12-26 20:13:10,196][105692] Updated weights for policy 0, policy_version 666394 (0.0009) [2023-12-26 20:13:10,264][105692] Updated weights for policy 0, policy_version 666404 (0.0010) [2023-12-26 20:13:10,852][105620] Updated weights for policy 1, policy_version 667353 (0.0009) [2023-12-26 20:13:10,907][105620] Updated weights for policy 1, policy_version 667363 (0.0008) [2023-12-26 20:13:10,965][105692] Updated weights for policy 0, policy_version 666414 (0.0011) [2023-12-26 20:13:10,970][105620] Updated weights for policy 1, policy_version 667373 (0.0007) [2023-12-26 20:13:11,027][105692] Updated weights for policy 0, policy_version 666424 (0.0011) [2023-12-26 20:13:11,039][105620] Updated weights for policy 1, policy_version 667383 (0.0006) [2023-12-26 20:13:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 341499904. Throughput: 0: 9942.5, 1: 9986.5. Samples: 341508844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:13:11,063][104569] Avg episode reward: [(0, '8994.932'), (1, '8626.379')] [2023-12-26 20:13:11,087][105692] Updated weights for policy 0, policy_version 666434 (0.0011) [2023-12-26 20:13:11,831][105692] Updated weights for policy 0, policy_version 666444 (0.0009) [2023-12-26 20:13:11,854][105620] Updated weights for policy 1, policy_version 667393 (0.0008) [2023-12-26 20:13:11,890][105692] Updated weights for policy 0, policy_version 666454 (0.0007) [2023-12-26 20:13:11,911][105620] Updated weights for policy 1, policy_version 667403 (0.0007) [2023-12-26 20:13:11,946][105692] Updated weights for policy 0, policy_version 666464 (0.0008) [2023-12-26 20:13:11,965][105620] Updated weights for policy 1, policy_version 667413 (0.0007) [2023-12-26 20:13:12,664][105692] Updated weights for policy 0, policy_version 666474 (0.0006) [2023-12-26 20:13:12,721][105692] Updated weights for policy 0, policy_version 666484 (0.0009) [2023-12-26 20:13:12,744][105620] Updated weights for policy 1, policy_version 667423 (0.0006) [2023-12-26 20:13:12,779][105692] Updated weights for policy 0, policy_version 666494 (0.0008) [2023-12-26 20:13:12,797][105620] Updated weights for policy 1, policy_version 667433 (0.0007) [2023-12-26 20:13:12,832][105692] Updated weights for policy 0, policy_version 666504 (0.0009) [2023-12-26 20:13:12,847][105620] Updated weights for policy 1, policy_version 667443 (0.0005) [2023-12-26 20:13:13,518][105620] Updated weights for policy 1, policy_version 667453 (0.0008) [2023-12-26 20:13:13,526][105692] Updated weights for policy 0, policy_version 666514 (0.0006) [2023-12-26 20:13:13,566][105620] Updated weights for policy 1, policy_version 667463 (0.0010) [2023-12-26 20:13:13,573][105692] Updated weights for policy 0, policy_version 666524 (0.0005) [2023-12-26 20:13:13,611][105620] Updated weights for policy 1, policy_version 667473 (0.0010) [2023-12-26 20:13:13,626][105692] Updated weights for policy 0, policy_version 666534 (0.0006) [2023-12-26 20:13:14,288][105692] Updated weights for policy 0, policy_version 666544 (0.0005) [2023-12-26 20:13:14,353][105692] Updated weights for policy 0, policy_version 666554 (0.0005) [2023-12-26 20:13:14,359][105620] Updated weights for policy 1, policy_version 667483 (0.0010) [2023-12-26 20:13:14,401][105692] Updated weights for policy 0, policy_version 666564 (0.0010) [2023-12-26 20:13:14,403][105620] Updated weights for policy 1, policy_version 667493 (0.0010) [2023-12-26 20:13:14,461][105620] Updated weights for policy 1, policy_version 667503 (0.0010) [2023-12-26 20:13:14,979][105692] Updated weights for policy 0, policy_version 666574 (0.0011) [2023-12-26 20:13:15,037][105692] Updated weights for policy 0, policy_version 666584 (0.0009) [2023-12-26 20:13:15,093][105692] Updated weights for policy 0, policy_version 666594 (0.0010) [2023-12-26 20:13:15,222][105620] Updated weights for policy 1, policy_version 667513 (0.0007) [2023-12-26 20:13:15,280][105620] Updated weights for policy 1, policy_version 667523 (0.0006) [2023-12-26 20:13:15,341][105620] Updated weights for policy 1, policy_version 667533 (0.0006) [2023-12-26 20:13:15,401][105620] Updated weights for policy 1, policy_version 667543 (0.0008) [2023-12-26 20:13:15,820][105692] Updated weights for policy 0, policy_version 666604 (0.0011) [2023-12-26 20:13:15,878][105692] Updated weights for policy 0, policy_version 666614 (0.0010) [2023-12-26 20:13:15,936][105692] Updated weights for policy 0, policy_version 666624 (0.0010) [2023-12-26 20:13:16,046][105620] Updated weights for policy 1, policy_version 667553 (0.0010) [2023-12-26 20:13:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 341598208. Throughput: 0: 9899.0, 1: 9880.6. Samples: 341566564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:13:16,063][104569] Avg episode reward: [(0, '9079.527'), (1, '8815.172')] [2023-12-26 20:13:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000666632_170688512.pth... [2023-12-26 20:13:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000665480_170393600.pth [2023-12-26 20:13:16,111][105620] Updated weights for policy 1, policy_version 667563 (0.0010) [2023-12-26 20:13:16,176][105620] Updated weights for policy 1, policy_version 667573 (0.0010) [2023-12-26 20:13:16,196][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000667576_170917888.pth... [2023-12-26 20:13:16,200][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000666392_170614784.pth [2023-12-26 20:13:16,688][105692] Updated weights for policy 0, policy_version 666634 (0.0010) [2023-12-26 20:13:16,743][105692] Updated weights for policy 0, policy_version 666644 (0.0010) [2023-12-26 20:13:16,797][105692] Updated weights for policy 0, policy_version 666654 (0.0010) [2023-12-26 20:13:16,804][105620] Updated weights for policy 1, policy_version 667583 (0.0010) [2023-12-26 20:13:16,841][105692] Updated weights for policy 0, policy_version 666664 (0.0010) [2023-12-26 20:13:16,865][105620] Updated weights for policy 1, policy_version 667593 (0.0007) [2023-12-26 20:13:16,912][105620] Updated weights for policy 1, policy_version 667603 (0.0005) [2023-12-26 20:13:17,529][105692] Updated weights for policy 0, policy_version 666674 (0.0010) [2023-12-26 20:13:17,547][105620] Updated weights for policy 1, policy_version 667613 (0.0005) [2023-12-26 20:13:17,580][105692] Updated weights for policy 0, policy_version 666684 (0.0010) [2023-12-26 20:13:17,591][105620] Updated weights for policy 1, policy_version 667623 (0.0007) [2023-12-26 20:13:17,628][105692] Updated weights for policy 0, policy_version 666694 (0.0010) [2023-12-26 20:13:17,637][105620] Updated weights for policy 1, policy_version 667633 (0.0009) [2023-12-26 20:13:18,264][105620] Updated weights for policy 1, policy_version 667643 (0.0007) [2023-12-26 20:13:18,315][105620] Updated weights for policy 1, policy_version 667653 (0.0005) [2023-12-26 20:13:18,381][105620] Updated weights for policy 1, policy_version 667663 (0.0007) [2023-12-26 20:13:18,390][105692] Updated weights for policy 0, policy_version 666704 (0.0009) [2023-12-26 20:13:18,449][105692] Updated weights for policy 0, policy_version 666714 (0.0010) [2023-12-26 20:13:18,508][105692] Updated weights for policy 0, policy_version 666724 (0.0010) [2023-12-26 20:13:19,029][105620] Updated weights for policy 1, policy_version 667673 (0.0007) [2023-12-26 20:13:19,078][105620] Updated weights for policy 1, policy_version 667683 (0.0008) [2023-12-26 20:13:19,130][105620] Updated weights for policy 1, policy_version 667693 (0.0009) [2023-12-26 20:13:19,182][105620] Updated weights for policy 1, policy_version 667703 (0.0008) [2023-12-26 20:13:19,263][105692] Updated weights for policy 0, policy_version 666734 (0.0009) [2023-12-26 20:13:19,330][105692] Updated weights for policy 0, policy_version 666744 (0.0009) [2023-12-26 20:13:19,394][105692] Updated weights for policy 0, policy_version 666754 (0.0011) [2023-12-26 20:13:19,987][105620] Updated weights for policy 1, policy_version 667713 (0.0007) [2023-12-26 20:13:20,055][105620] Updated weights for policy 1, policy_version 667723 (0.0008) [2023-12-26 20:13:20,118][105620] Updated weights for policy 1, policy_version 667733 (0.0007) [2023-12-26 20:13:20,159][105692] Updated weights for policy 0, policy_version 666764 (0.0010) [2023-12-26 20:13:20,223][105692] Updated weights for policy 0, policy_version 666774 (0.0011) [2023-12-26 20:13:20,286][105692] Updated weights for policy 0, policy_version 666784 (0.0011) [2023-12-26 20:13:20,824][105620] Updated weights for policy 1, policy_version 667743 (0.0009) [2023-12-26 20:13:20,887][105620] Updated weights for policy 1, policy_version 667753 (0.0007) [2023-12-26 20:13:20,941][105620] Updated weights for policy 1, policy_version 667763 (0.0008) [2023-12-26 20:13:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19933.9, 300 sec: 19494.2). Total num frames: 341696512. Throughput: 0: 9821.6, 1: 9963.6. Samples: 341687240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:13:21,062][104569] Avg episode reward: [(0, '9079.345'), (1, '8539.620')] [2023-12-26 20:13:21,082][105692] Updated weights for policy 0, policy_version 666794 (0.0010) [2023-12-26 20:13:21,142][105692] Updated weights for policy 0, policy_version 666804 (0.0011) [2023-12-26 20:13:21,195][105692] Updated weights for policy 0, policy_version 666814 (0.0010) [2023-12-26 20:13:21,255][105692] Updated weights for policy 0, policy_version 666824 (0.0011) [2023-12-26 20:13:21,738][105620] Updated weights for policy 1, policy_version 667773 (0.0008) [2023-12-26 20:13:21,794][105620] Updated weights for policy 1, policy_version 667783 (0.0008) [2023-12-26 20:13:21,854][105620] Updated weights for policy 1, policy_version 667793 (0.0007) [2023-12-26 20:13:22,013][105692] Updated weights for policy 0, policy_version 666834 (0.0009) [2023-12-26 20:13:22,068][105692] Updated weights for policy 0, policy_version 666844 (0.0009) [2023-12-26 20:13:22,123][105692] Updated weights for policy 0, policy_version 666854 (0.0009) [2023-12-26 20:13:22,615][105620] Updated weights for policy 1, policy_version 667803 (0.0009) [2023-12-26 20:13:22,673][105620] Updated weights for policy 1, policy_version 667813 (0.0009) [2023-12-26 20:13:22,736][105620] Updated weights for policy 1, policy_version 667823 (0.0009) [2023-12-26 20:13:22,901][105692] Updated weights for policy 0, policy_version 666864 (0.0009) [2023-12-26 20:13:22,952][105692] Updated weights for policy 0, policy_version 666874 (0.0009) [2023-12-26 20:13:23,009][105692] Updated weights for policy 0, policy_version 666884 (0.0010) [2023-12-26 20:13:23,497][105620] Updated weights for policy 1, policy_version 667833 (0.0009) [2023-12-26 20:13:23,563][105620] Updated weights for policy 1, policy_version 667843 (0.0008) [2023-12-26 20:13:23,625][105620] Updated weights for policy 1, policy_version 667853 (0.0009) [2023-12-26 20:13:23,690][105620] Updated weights for policy 1, policy_version 667863 (0.0010) [2023-12-26 20:13:23,762][105692] Updated weights for policy 0, policy_version 666894 (0.0007) [2023-12-26 20:13:23,832][105692] Updated weights for policy 0, policy_version 666904 (0.0005) [2023-12-26 20:13:23,898][105692] Updated weights for policy 0, policy_version 666914 (0.0005) [2023-12-26 20:13:24,451][105692] Updated weights for policy 0, policy_version 666924 (0.0007) [2023-12-26 20:13:24,517][105692] Updated weights for policy 0, policy_version 666934 (0.0008) [2023-12-26 20:13:24,520][105620] Updated weights for policy 1, policy_version 667873 (0.0007) [2023-12-26 20:13:24,580][105620] Updated weights for policy 1, policy_version 667883 (0.0008) [2023-12-26 20:13:24,580][105692] Updated weights for policy 0, policy_version 666944 (0.0007) [2023-12-26 20:13:24,633][105620] Updated weights for policy 1, policy_version 667893 (0.0007) [2023-12-26 20:13:25,223][105692] Updated weights for policy 0, policy_version 666954 (0.0006) [2023-12-26 20:13:25,239][105620] Updated weights for policy 1, policy_version 667903 (0.0007) [2023-12-26 20:13:25,282][105692] Updated weights for policy 0, policy_version 666964 (0.0008) [2023-12-26 20:13:25,301][105620] Updated weights for policy 1, policy_version 667913 (0.0007) [2023-12-26 20:13:25,343][105692] Updated weights for policy 0, policy_version 666974 (0.0005) [2023-12-26 20:13:25,367][105620] Updated weights for policy 1, policy_version 667923 (0.0009) [2023-12-26 20:13:25,395][105692] Updated weights for policy 0, policy_version 666984 (0.0006) [2023-12-26 20:13:26,049][105692] Updated weights for policy 0, policy_version 666994 (0.0005) [2023-12-26 20:13:26,058][105620] Updated weights for policy 1, policy_version 667933 (0.0007) [2023-12-26 20:13:26,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 341786624. Throughput: 0: 9751.8, 1: 9854.2. Samples: 341800632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:13:26,062][104569] Avg episode reward: [(0, '8572.191'), (1, '8255.382')] [2023-12-26 20:13:26,102][105692] Updated weights for policy 0, policy_version 667004 (0.0007) [2023-12-26 20:13:26,110][105620] Updated weights for policy 1, policy_version 667943 (0.0005) [2023-12-26 20:13:26,159][105692] Updated weights for policy 0, policy_version 667014 (0.0007) [2023-12-26 20:13:26,165][105620] Updated weights for policy 1, policy_version 667953 (0.0006) [2023-12-26 20:13:26,862][105620] Updated weights for policy 1, policy_version 667963 (0.0007) [2023-12-26 20:13:26,876][105692] Updated weights for policy 0, policy_version 667024 (0.0007) [2023-12-26 20:13:26,917][105620] Updated weights for policy 1, policy_version 667973 (0.0009) [2023-12-26 20:13:26,930][105692] Updated weights for policy 0, policy_version 667034 (0.0008) [2023-12-26 20:13:26,968][105620] Updated weights for policy 1, policy_version 667983 (0.0006) [2023-12-26 20:13:26,982][105692] Updated weights for policy 0, policy_version 667044 (0.0006) [2023-12-26 20:13:27,700][105620] Updated weights for policy 1, policy_version 667993 (0.0007) [2023-12-26 20:13:27,759][105620] Updated weights for policy 1, policy_version 668003 (0.0006) [2023-12-26 20:13:27,784][105692] Updated weights for policy 0, policy_version 667054 (0.0006) [2023-12-26 20:13:27,829][105620] Updated weights for policy 1, policy_version 668013 (0.0008) [2023-12-26 20:13:27,846][105692] Updated weights for policy 0, policy_version 667064 (0.0005) [2023-12-26 20:13:27,893][105620] Updated weights for policy 1, policy_version 668023 (0.0009) [2023-12-26 20:13:27,902][105692] Updated weights for policy 0, policy_version 667074 (0.0005) [2023-12-26 20:13:28,537][105692] Updated weights for policy 0, policy_version 667084 (0.0007) [2023-12-26 20:13:28,595][105692] Updated weights for policy 0, policy_version 667094 (0.0007) [2023-12-26 20:13:28,613][105620] Updated weights for policy 1, policy_version 668033 (0.0007) [2023-12-26 20:13:28,655][105692] Updated weights for policy 0, policy_version 667104 (0.0007) [2023-12-26 20:13:28,669][105620] Updated weights for policy 1, policy_version 668043 (0.0006) [2023-12-26 20:13:28,723][105620] Updated weights for policy 1, policy_version 668053 (0.0007) [2023-12-26 20:13:29,405][105692] Updated weights for policy 0, policy_version 667114 (0.0007) [2023-12-26 20:13:29,466][105692] Updated weights for policy 0, policy_version 667124 (0.0010) [2023-12-26 20:13:29,496][105620] Updated weights for policy 1, policy_version 668063 (0.0006) [2023-12-26 20:13:29,525][105692] Updated weights for policy 0, policy_version 667134 (0.0011) [2023-12-26 20:13:29,551][105620] Updated weights for policy 1, policy_version 668073 (0.0005) [2023-12-26 20:13:29,580][105692] Updated weights for policy 0, policy_version 667144 (0.0010) [2023-12-26 20:13:29,608][105620] Updated weights for policy 1, policy_version 668083 (0.0007) [2023-12-26 20:13:30,267][105692] Updated weights for policy 0, policy_version 667154 (0.0010) [2023-12-26 20:13:30,322][105692] Updated weights for policy 0, policy_version 667164 (0.0010) [2023-12-26 20:13:30,371][105692] Updated weights for policy 0, policy_version 667174 (0.0010) [2023-12-26 20:13:30,393][105620] Updated weights for policy 1, policy_version 668093 (0.0007) [2023-12-26 20:13:30,457][105620] Updated weights for policy 1, policy_version 668103 (0.0010) [2023-12-26 20:13:30,509][105620] Updated weights for policy 1, policy_version 668113 (0.0009) [2023-12-26 20:13:31,042][105692] Updated weights for policy 0, policy_version 667184 (0.0010) [2023-12-26 20:13:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 341884928. Throughput: 0: 9786.0, 1: 9872.4. Samples: 341859692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:13:31,062][104569] Avg episode reward: [(0, '8572.662'), (1, '8348.082')] [2023-12-26 20:13:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000668120_171057152.pth... [2023-12-26 20:13:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000666968_170762240.pth [2023-12-26 20:13:31,111][105692] Updated weights for policy 0, policy_version 667194 (0.0008) [2023-12-26 20:13:31,184][105692] Updated weights for policy 0, policy_version 667204 (0.0010) [2023-12-26 20:13:31,206][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000667208_170835968.pth... [2023-12-26 20:13:31,211][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000666056_170541056.pth [2023-12-26 20:13:31,304][105620] Updated weights for policy 1, policy_version 668123 (0.0008) [2023-12-26 20:13:31,378][105620] Updated weights for policy 1, policy_version 668133 (0.0007) [2023-12-26 20:13:31,437][105620] Updated weights for policy 1, policy_version 668143 (0.0008) [2023-12-26 20:13:31,926][105692] Updated weights for policy 0, policy_version 667214 (0.0007) [2023-12-26 20:13:31,983][105692] Updated weights for policy 0, policy_version 667224 (0.0009) [2023-12-26 20:13:32,044][105692] Updated weights for policy 0, policy_version 667234 (0.0008) [2023-12-26 20:13:32,169][105620] Updated weights for policy 1, policy_version 668153 (0.0009) [2023-12-26 20:13:32,227][105620] Updated weights for policy 1, policy_version 668163 (0.0009) [2023-12-26 20:13:32,281][105620] Updated weights for policy 1, policy_version 668173 (0.0008) [2023-12-26 20:13:32,327][105620] Updated weights for policy 1, policy_version 668183 (0.0005) [2023-12-26 20:13:32,782][105692] Updated weights for policy 0, policy_version 667244 (0.0009) [2023-12-26 20:13:32,830][105692] Updated weights for policy 0, policy_version 667254 (0.0009) [2023-12-26 20:13:32,876][105692] Updated weights for policy 0, policy_version 667264 (0.0009) [2023-12-26 20:13:33,069][105620] Updated weights for policy 1, policy_version 668193 (0.0008) [2023-12-26 20:13:33,119][105620] Updated weights for policy 1, policy_version 668203 (0.0009) [2023-12-26 20:13:33,173][105620] Updated weights for policy 1, policy_version 668213 (0.0009) [2023-12-26 20:13:33,654][105692] Updated weights for policy 0, policy_version 667274 (0.0008) [2023-12-26 20:13:33,707][105692] Updated weights for policy 0, policy_version 667284 (0.0005) [2023-12-26 20:13:33,764][105692] Updated weights for policy 0, policy_version 667294 (0.0008) [2023-12-26 20:13:33,818][105692] Updated weights for policy 0, policy_version 667304 (0.0009) [2023-12-26 20:13:33,855][105620] Updated weights for policy 1, policy_version 668223 (0.0009) [2023-12-26 20:13:33,915][105620] Updated weights for policy 1, policy_version 668233 (0.0009) [2023-12-26 20:13:33,962][105620] Updated weights for policy 1, policy_version 668243 (0.0009) [2023-12-26 20:13:34,481][105692] Updated weights for policy 0, policy_version 667314 (0.0008) [2023-12-26 20:13:34,537][105692] Updated weights for policy 0, policy_version 667324 (0.0009) [2023-12-26 20:13:34,599][105692] Updated weights for policy 0, policy_version 667334 (0.0010) [2023-12-26 20:13:34,734][105620] Updated weights for policy 1, policy_version 668253 (0.0009) [2023-12-26 20:13:34,796][105620] Updated weights for policy 1, policy_version 668263 (0.0009) [2023-12-26 20:13:34,857][105620] Updated weights for policy 1, policy_version 668273 (0.0009) [2023-12-26 20:13:35,311][105692] Updated weights for policy 0, policy_version 667344 (0.0006) [2023-12-26 20:13:35,357][105692] Updated weights for policy 0, policy_version 667354 (0.0005) [2023-12-26 20:13:35,411][105692] Updated weights for policy 0, policy_version 667364 (0.0005) [2023-12-26 20:13:35,623][105620] Updated weights for policy 1, policy_version 668283 (0.0009) [2023-12-26 20:13:35,687][105620] Updated weights for policy 1, policy_version 668293 (0.0008) [2023-12-26 20:13:35,745][105620] Updated weights for policy 1, policy_version 668303 (0.0009) [2023-12-26 20:13:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 341983232. Throughput: 0: 9724.8, 1: 9779.0. Samples: 341973524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:13:36,062][104569] Avg episode reward: [(0, '8234.522'), (1, '8804.109')] [2023-12-26 20:13:36,097][105692] Updated weights for policy 0, policy_version 667374 (0.0006) [2023-12-26 20:13:36,151][105692] Updated weights for policy 0, policy_version 667384 (0.0008) [2023-12-26 20:13:36,210][105692] Updated weights for policy 0, policy_version 667394 (0.0008) [2023-12-26 20:13:36,557][105620] Updated weights for policy 1, policy_version 668313 (0.0009) [2023-12-26 20:13:36,620][105620] Updated weights for policy 1, policy_version 668323 (0.0007) [2023-12-26 20:13:36,686][105620] Updated weights for policy 1, policy_version 668333 (0.0009) [2023-12-26 20:13:36,751][105620] Updated weights for policy 1, policy_version 668343 (0.0009) [2023-12-26 20:13:36,857][105692] Updated weights for policy 0, policy_version 667404 (0.0008) [2023-12-26 20:13:36,912][105692] Updated weights for policy 0, policy_version 667414 (0.0009) [2023-12-26 20:13:36,966][105692] Updated weights for policy 0, policy_version 667424 (0.0009) [2023-12-26 20:13:37,499][105620] Updated weights for policy 1, policy_version 668353 (0.0009) [2023-12-26 20:13:37,567][105620] Updated weights for policy 1, policy_version 668363 (0.0009) [2023-12-26 20:13:37,625][105620] Updated weights for policy 1, policy_version 668373 (0.0009) [2023-12-26 20:13:37,701][105692] Updated weights for policy 0, policy_version 667434 (0.0008) [2023-12-26 20:13:37,770][105692] Updated weights for policy 0, policy_version 667444 (0.0008) [2023-12-26 20:13:37,836][105692] Updated weights for policy 0, policy_version 667454 (0.0009) [2023-12-26 20:13:37,896][105692] Updated weights for policy 0, policy_version 667464 (0.0009) [2023-12-26 20:13:38,459][105620] Updated weights for policy 1, policy_version 668383 (0.0009) [2023-12-26 20:13:38,515][105620] Updated weights for policy 1, policy_version 668393 (0.0008) [2023-12-26 20:13:38,564][105692] Updated weights for policy 0, policy_version 667474 (0.0008) [2023-12-26 20:13:38,575][105620] Updated weights for policy 1, policy_version 668403 (0.0006) [2023-12-26 20:13:38,617][105692] Updated weights for policy 0, policy_version 667484 (0.0005) [2023-12-26 20:13:38,679][105692] Updated weights for policy 0, policy_version 667494 (0.0005) [2023-12-26 20:13:39,297][105692] Updated weights for policy 0, policy_version 667504 (0.0007) [2023-12-26 20:13:39,360][105692] Updated weights for policy 0, policy_version 667514 (0.0009) [2023-12-26 20:13:39,401][105620] Updated weights for policy 1, policy_version 668413 (0.0008) [2023-12-26 20:13:39,432][105692] Updated weights for policy 0, policy_version 667524 (0.0008) [2023-12-26 20:13:39,460][105620] Updated weights for policy 1, policy_version 668423 (0.0008) [2023-12-26 20:13:39,519][105620] Updated weights for policy 1, policy_version 668433 (0.0008) [2023-12-26 20:13:40,232][105620] Updated weights for policy 1, policy_version 668443 (0.0009) [2023-12-26 20:13:40,239][105692] Updated weights for policy 0, policy_version 667534 (0.0007) [2023-12-26 20:13:40,294][105620] Updated weights for policy 1, policy_version 668453 (0.0007) [2023-12-26 20:13:40,301][105692] Updated weights for policy 0, policy_version 667544 (0.0006) [2023-12-26 20:13:40,356][105620] Updated weights for policy 1, policy_version 668463 (0.0007) [2023-12-26 20:13:40,366][105692] Updated weights for policy 0, policy_version 667554 (0.0007) [2023-12-26 20:13:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19438.7). Total num frames: 342073344. Throughput: 0: 9763.4, 1: 9682.8. Samples: 342087296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:13:41,062][104569] Avg episode reward: [(0, '8325.120'), (1, '8987.138')] [2023-12-26 20:13:41,082][105620] Updated weights for policy 1, policy_version 668473 (0.0008) [2023-12-26 20:13:41,155][105620] Updated weights for policy 1, policy_version 668483 (0.0009) [2023-12-26 20:13:41,169][105692] Updated weights for policy 0, policy_version 667564 (0.0007) [2023-12-26 20:13:41,216][105620] Updated weights for policy 1, policy_version 668493 (0.0008) [2023-12-26 20:13:41,233][105692] Updated weights for policy 0, policy_version 667574 (0.0008) [2023-12-26 20:13:41,283][105620] Updated weights for policy 1, policy_version 668503 (0.0010) [2023-12-26 20:13:41,298][105692] Updated weights for policy 0, policy_version 667584 (0.0011) [2023-12-26 20:13:42,070][105620] Updated weights for policy 1, policy_version 668513 (0.0006) [2023-12-26 20:13:42,122][105692] Updated weights for policy 0, policy_version 667594 (0.0010) [2023-12-26 20:13:42,128][105620] Updated weights for policy 1, policy_version 668523 (0.0006) [2023-12-26 20:13:42,173][105692] Updated weights for policy 0, policy_version 667604 (0.0007) [2023-12-26 20:13:42,176][105620] Updated weights for policy 1, policy_version 668533 (0.0007) [2023-12-26 20:13:42,222][105692] Updated weights for policy 0, policy_version 667614 (0.0007) [2023-12-26 20:13:42,279][105692] Updated weights for policy 0, policy_version 667624 (0.0008) [2023-12-26 20:13:42,944][105620] Updated weights for policy 1, policy_version 668543 (0.0007) [2023-12-26 20:13:42,971][105692] Updated weights for policy 0, policy_version 667634 (0.0006) [2023-12-26 20:13:42,991][105620] Updated weights for policy 1, policy_version 668553 (0.0006) [2023-12-26 20:13:43,024][105692] Updated weights for policy 0, policy_version 667644 (0.0006) [2023-12-26 20:13:43,051][105620] Updated weights for policy 1, policy_version 668563 (0.0009) [2023-12-26 20:13:43,069][105692] Updated weights for policy 0, policy_version 667654 (0.0005) [2023-12-26 20:13:43,724][105692] Updated weights for policy 0, policy_version 667664 (0.0005) [2023-12-26 20:13:43,780][105692] Updated weights for policy 0, policy_version 667674 (0.0005) [2023-12-26 20:13:43,838][105692] Updated weights for policy 0, policy_version 667684 (0.0009) [2023-12-26 20:13:43,855][105620] Updated weights for policy 1, policy_version 668573 (0.0009) [2023-12-26 20:13:43,923][105620] Updated weights for policy 1, policy_version 668583 (0.0008) [2023-12-26 20:13:43,991][105620] Updated weights for policy 1, policy_version 668593 (0.0008) [2023-12-26 20:13:44,498][105692] Updated weights for policy 0, policy_version 667694 (0.0006) [2023-12-26 20:13:44,565][105692] Updated weights for policy 0, policy_version 667704 (0.0005) [2023-12-26 20:13:44,622][105692] Updated weights for policy 0, policy_version 667714 (0.0009) [2023-12-26 20:13:44,694][105620] Updated weights for policy 1, policy_version 668603 (0.0007) [2023-12-26 20:13:44,748][105620] Updated weights for policy 1, policy_version 668613 (0.0005) [2023-12-26 20:13:44,815][105620] Updated weights for policy 1, policy_version 668623 (0.0008) [2023-12-26 20:13:45,352][105692] Updated weights for policy 0, policy_version 667724 (0.0010) [2023-12-26 20:13:45,374][105620] Updated weights for policy 1, policy_version 668633 (0.0007) [2023-12-26 20:13:45,412][105692] Updated weights for policy 0, policy_version 667734 (0.0009) [2023-12-26 20:13:45,426][105620] Updated weights for policy 1, policy_version 668643 (0.0007) [2023-12-26 20:13:45,474][105692] Updated weights for policy 0, policy_version 667744 (0.0008) [2023-12-26 20:13:45,476][105620] Updated weights for policy 1, policy_version 668653 (0.0006) [2023-12-26 20:13:45,531][105620] Updated weights for policy 1, policy_version 668663 (0.0007) [2023-12-26 20:13:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 342171648. Throughput: 0: 9672.1, 1: 9582.3. Samples: 342142836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:13:46,062][104569] Avg episode reward: [(0, '9080.362'), (1, '8897.450')] [2023-12-26 20:13:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000667752_170975232.pth... [2023-12-26 20:13:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000666632_170688512.pth [2023-12-26 20:13:46,105][105620] Updated weights for policy 1, policy_version 668673 (0.0005) [2023-12-26 20:13:46,145][105692] Updated weights for policy 0, policy_version 667754 (0.0008) [2023-12-26 20:13:46,157][105620] Updated weights for policy 1, policy_version 668683 (0.0005) [2023-12-26 20:13:46,194][105692] Updated weights for policy 0, policy_version 667764 (0.0008) [2023-12-26 20:13:46,203][105620] Updated weights for policy 1, policy_version 668693 (0.0005) [2023-12-26 20:13:46,219][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000668696_171204608.pth... [2023-12-26 20:13:46,223][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000667576_170917888.pth [2023-12-26 20:13:46,248][105692] Updated weights for policy 0, policy_version 667774 (0.0010) [2023-12-26 20:13:46,831][105620] Updated weights for policy 1, policy_version 668703 (0.0008) [2023-12-26 20:13:46,887][105620] Updated weights for policy 1, policy_version 668713 (0.0010) [2023-12-26 20:13:46,940][105620] Updated weights for policy 1, policy_version 668724 (0.0010) [2023-12-26 20:13:47,019][105692] Updated weights for policy 0, policy_version 667786 (0.0010) [2023-12-26 20:13:47,074][105692] Updated weights for policy 0, policy_version 667796 (0.0009) [2023-12-26 20:13:47,128][105692] Updated weights for policy 0, policy_version 667806 (0.0009) [2023-12-26 20:13:47,183][105692] Updated weights for policy 0, policy_version 667816 (0.0008) [2023-12-26 20:13:47,579][105620] Updated weights for policy 1, policy_version 668734 (0.0008) [2023-12-26 20:13:47,631][105620] Updated weights for policy 1, policy_version 668744 (0.0010) [2023-12-26 20:13:47,682][105620] Updated weights for policy 1, policy_version 668754 (0.0010) [2023-12-26 20:13:48,007][105692] Updated weights for policy 0, policy_version 667826 (0.0006) [2023-12-26 20:13:48,064][105692] Updated weights for policy 0, policy_version 667836 (0.0007) [2023-12-26 20:13:48,115][105692] Updated weights for policy 0, policy_version 667846 (0.0010) [2023-12-26 20:13:48,394][105620] Updated weights for policy 1, policy_version 668764 (0.0011) [2023-12-26 20:13:48,459][105620] Updated weights for policy 1, policy_version 668774 (0.0009) [2023-12-26 20:13:48,532][105620] Updated weights for policy 1, policy_version 668784 (0.0006) [2023-12-26 20:13:48,756][105692] Updated weights for policy 0, policy_version 667856 (0.0007) [2023-12-26 20:13:48,813][105692] Updated weights for policy 0, policy_version 667866 (0.0007) [2023-12-26 20:13:48,871][105692] Updated weights for policy 0, policy_version 667876 (0.0009) [2023-12-26 20:13:49,158][105620] Updated weights for policy 1, policy_version 668794 (0.0007) [2023-12-26 20:13:49,224][105620] Updated weights for policy 1, policy_version 668804 (0.0011) [2023-12-26 20:13:49,288][105620] Updated weights for policy 1, policy_version 668814 (0.0008) [2023-12-26 20:13:49,343][105620] Updated weights for policy 1, policy_version 668824 (0.0008) [2023-12-26 20:13:49,628][105692] Updated weights for policy 0, policy_version 667886 (0.0007) [2023-12-26 20:13:49,678][105692] Updated weights for policy 0, policy_version 667897 (0.0009) [2023-12-26 20:13:49,737][105692] Updated weights for policy 0, policy_version 667907 (0.0008) [2023-12-26 20:13:49,993][105620] Updated weights for policy 1, policy_version 668834 (0.0010) [2023-12-26 20:13:50,054][105620] Updated weights for policy 1, policy_version 668844 (0.0008) [2023-12-26 20:13:50,118][105620] Updated weights for policy 1, policy_version 668854 (0.0010) [2023-12-26 20:13:50,446][105692] Updated weights for policy 0, policy_version 667917 (0.0008) [2023-12-26 20:13:50,518][105692] Updated weights for policy 0, policy_version 667927 (0.0006) [2023-12-26 20:13:50,579][105692] Updated weights for policy 0, policy_version 667937 (0.0006) [2023-12-26 20:13:50,826][105620] Updated weights for policy 1, policy_version 668864 (0.0011) [2023-12-26 20:13:50,885][105620] Updated weights for policy 1, policy_version 668874 (0.0011) [2023-12-26 20:13:50,946][105620] Updated weights for policy 1, policy_version 668884 (0.0010) [2023-12-26 20:13:51,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 342278144. Throughput: 0: 9729.1, 1: 9722.9. Samples: 342265476. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:13:51,063][104569] Avg episode reward: [(0, '9079.548'), (1, '8986.807')] [2023-12-26 20:13:51,260][105692] Updated weights for policy 0, policy_version 667947 (0.0008) [2023-12-26 20:13:51,315][105692] Updated weights for policy 0, policy_version 667957 (0.0008) [2023-12-26 20:13:51,402][105692] Updated weights for policy 0, policy_version 667967 (0.0008) [2023-12-26 20:13:51,726][105620] Updated weights for policy 1, policy_version 668894 (0.0010) [2023-12-26 20:13:51,788][105620] Updated weights for policy 1, policy_version 668904 (0.0008) [2023-12-26 20:13:51,854][105620] Updated weights for policy 1, policy_version 668914 (0.0010) [2023-12-26 20:13:52,088][105692] Updated weights for policy 0, policy_version 667977 (0.0008) [2023-12-26 20:13:52,137][105692] Updated weights for policy 0, policy_version 667987 (0.0008) [2023-12-26 20:13:52,180][105692] Updated weights for policy 0, policy_version 667997 (0.0005) [2023-12-26 20:13:52,245][105692] Updated weights for policy 0, policy_version 668007 (0.0006) [2023-12-26 20:13:52,612][105620] Updated weights for policy 1, policy_version 668924 (0.0011) [2023-12-26 20:13:52,677][105620] Updated weights for policy 1, policy_version 668934 (0.0010) [2023-12-26 20:13:52,744][105620] Updated weights for policy 1, policy_version 668944 (0.0010) [2023-12-26 20:13:52,990][105692] Updated weights for policy 0, policy_version 668017 (0.0008) [2023-12-26 20:13:53,057][105692] Updated weights for policy 0, policy_version 668027 (0.0007) [2023-12-26 20:13:53,122][105692] Updated weights for policy 0, policy_version 668037 (0.0008) [2023-12-26 20:13:53,447][105620] Updated weights for policy 1, policy_version 668954 (0.0010) [2023-12-26 20:13:53,492][105586] KL-divergence is very high: 285.8654 [2023-12-26 20:13:53,515][105620] Updated weights for policy 1, policy_version 668964 (0.0010) [2023-12-26 20:13:53,545][105586] KL-divergence is very high: 558.7094 [2023-12-26 20:13:53,577][105620] Updated weights for policy 1, policy_version 668974 (0.0010) [2023-12-26 20:13:53,595][105586] KL-divergence is very high: 609.4048 [2023-12-26 20:13:53,643][105620] Updated weights for policy 1, policy_version 668984 (0.0010) [2023-12-26 20:13:53,678][105692] Updated weights for policy 0, policy_version 668047 (0.0010) [2023-12-26 20:13:53,737][105692] Updated weights for policy 0, policy_version 668057 (0.0010) [2023-12-26 20:13:53,796][105692] Updated weights for policy 0, policy_version 668067 (0.0010) [2023-12-26 20:13:54,369][105620] Updated weights for policy 1, policy_version 668994 (0.0008) [2023-12-26 20:13:54,429][105620] Updated weights for policy 1, policy_version 669004 (0.0008) [2023-12-26 20:13:54,477][105620] Updated weights for policy 1, policy_version 669014 (0.0008) [2023-12-26 20:13:54,534][105692] Updated weights for policy 0, policy_version 668077 (0.0010) [2023-12-26 20:13:54,594][105692] Updated weights for policy 0, policy_version 668087 (0.0010) [2023-12-26 20:13:54,656][105692] Updated weights for policy 0, policy_version 668097 (0.0010) [2023-12-26 20:13:55,194][105620] Updated weights for policy 1, policy_version 669024 (0.0008) [2023-12-26 20:13:55,265][105620] Updated weights for policy 1, policy_version 669034 (0.0006) [2023-12-26 20:13:55,323][105620] Updated weights for policy 1, policy_version 669044 (0.0008) [2023-12-26 20:13:55,393][105692] Updated weights for policy 0, policy_version 668107 (0.0010) [2023-12-26 20:13:55,441][105692] Updated weights for policy 0, policy_version 668117 (0.0010) [2023-12-26 20:13:55,489][105692] Updated weights for policy 0, policy_version 668127 (0.0010) [2023-12-26 20:13:55,989][105620] Updated weights for policy 1, policy_version 669054 (0.0009) [2023-12-26 20:13:56,037][105620] Updated weights for policy 1, policy_version 669064 (0.0010) [2023-12-26 20:13:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 342368256. Throughput: 0: 9777.7, 1: 9620.1. Samples: 342381744. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 20:13:56,062][104569] Avg episode reward: [(0, '8990.854'), (1, '8893.234')] [2023-12-26 20:13:56,089][105620] Updated weights for policy 1, policy_version 669074 (0.0010) [2023-12-26 20:13:56,256][105692] Updated weights for policy 0, policy_version 668137 (0.0010) [2023-12-26 20:13:56,310][105692] Updated weights for policy 0, policy_version 668147 (0.0010) [2023-12-26 20:13:56,355][105692] Updated weights for policy 0, policy_version 668157 (0.0010) [2023-12-26 20:13:56,406][105692] Updated weights for policy 0, policy_version 668167 (0.0010) [2023-12-26 20:13:56,851][105620] Updated weights for policy 1, policy_version 669084 (0.0010) [2023-12-26 20:13:56,904][105620] Updated weights for policy 1, policy_version 669094 (0.0010) [2023-12-26 20:13:56,955][105620] Updated weights for policy 1, policy_version 669104 (0.0010) [2023-12-26 20:13:57,135][105692] Updated weights for policy 0, policy_version 668177 (0.0009) [2023-12-26 20:13:57,199][105692] Updated weights for policy 0, policy_version 668187 (0.0010) [2023-12-26 20:13:57,253][105692] Updated weights for policy 0, policy_version 668197 (0.0010) [2023-12-26 20:13:57,702][105620] Updated weights for policy 1, policy_version 669114 (0.0010) [2023-12-26 20:13:57,753][105620] Updated weights for policy 1, policy_version 669124 (0.0010) [2023-12-26 20:13:57,810][105620] Updated weights for policy 1, policy_version 669134 (0.0010) [2023-12-26 20:13:57,857][105620] Updated weights for policy 1, policy_version 669144 (0.0010) [2023-12-26 20:13:57,960][105692] Updated weights for policy 0, policy_version 668207 (0.0010) [2023-12-26 20:13:58,008][105692] Updated weights for policy 0, policy_version 668217 (0.0010) [2023-12-26 20:13:58,055][105692] Updated weights for policy 0, policy_version 668227 (0.0010) [2023-12-26 20:13:58,633][105620] Updated weights for policy 1, policy_version 669154 (0.0010) [2023-12-26 20:13:58,701][105620] Updated weights for policy 1, policy_version 669164 (0.0009) [2023-12-26 20:13:58,762][105620] Updated weights for policy 1, policy_version 669174 (0.0009) [2023-12-26 20:13:58,892][105692] Updated weights for policy 0, policy_version 668237 (0.0009) [2023-12-26 20:13:58,962][105692] Updated weights for policy 0, policy_version 668247 (0.0008) [2023-12-26 20:13:59,024][105692] Updated weights for policy 0, policy_version 668257 (0.0008) [2023-12-26 20:13:59,513][105620] Updated weights for policy 1, policy_version 669184 (0.0006) [2023-12-26 20:13:59,574][105620] Updated weights for policy 1, policy_version 669194 (0.0006) [2023-12-26 20:13:59,638][105620] Updated weights for policy 1, policy_version 669204 (0.0006) [2023-12-26 20:13:59,865][105692] Updated weights for policy 0, policy_version 668267 (0.0008) [2023-12-26 20:13:59,924][105692] Updated weights for policy 0, policy_version 668277 (0.0009) [2023-12-26 20:13:59,986][105692] Updated weights for policy 0, policy_version 668288 (0.0010) [2023-12-26 20:14:00,189][105620] Updated weights for policy 1, policy_version 669214 (0.0007) [2023-12-26 20:14:00,249][105620] Updated weights for policy 1, policy_version 669224 (0.0008) [2023-12-26 20:14:00,306][105620] Updated weights for policy 1, policy_version 669234 (0.0007) [2023-12-26 20:14:00,840][105692] Updated weights for policy 0, policy_version 668298 (0.0009) [2023-12-26 20:14:00,894][105692] Updated weights for policy 0, policy_version 668308 (0.0008) [2023-12-26 20:14:00,946][105692] Updated weights for policy 0, policy_version 668318 (0.0008) [2023-12-26 20:14:00,980][105620] Updated weights for policy 1, policy_version 669244 (0.0007) [2023-12-26 20:14:00,998][105692] Updated weights for policy 0, policy_version 668328 (0.0008) [2023-12-26 20:14:01,044][105620] Updated weights for policy 1, policy_version 669254 (0.0008) [2023-12-26 20:14:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19438.7). Total num frames: 342466560. Throughput: 0: 9760.1, 1: 9605.0. Samples: 342437988. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 20:14:01,062][104569] Avg episode reward: [(0, '8558.621'), (1, '8434.618')] [2023-12-26 20:14:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000668328_171122688.pth... [2023-12-26 20:14:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000667208_170835968.pth [2023-12-26 20:14:01,106][105620] Updated weights for policy 1, policy_version 669264 (0.0009) [2023-12-26 20:14:01,161][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000669272_171352064.pth... [2023-12-26 20:14:01,165][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000668120_171057152.pth [2023-12-26 20:14:01,785][105692] Updated weights for policy 0, policy_version 668338 (0.0008) [2023-12-26 20:14:01,852][105692] Updated weights for policy 0, policy_version 668348 (0.0008) [2023-12-26 20:14:01,870][105620] Updated weights for policy 1, policy_version 669274 (0.0009) [2023-12-26 20:14:01,918][105692] Updated weights for policy 0, policy_version 668358 (0.0007) [2023-12-26 20:14:01,936][105620] Updated weights for policy 1, policy_version 669284 (0.0007) [2023-12-26 20:14:02,004][105620] Updated weights for policy 1, policy_version 669294 (0.0006) [2023-12-26 20:14:02,070][105620] Updated weights for policy 1, policy_version 669304 (0.0008) [2023-12-26 20:14:02,674][105692] Updated weights for policy 0, policy_version 668368 (0.0009) [2023-12-26 20:14:02,724][105692] Updated weights for policy 0, policy_version 668378 (0.0008) [2023-12-26 20:14:02,729][105620] Updated weights for policy 1, policy_version 669314 (0.0007) [2023-12-26 20:14:02,775][105692] Updated weights for policy 0, policy_version 668389 (0.0009) [2023-12-26 20:14:02,793][105620] Updated weights for policy 1, policy_version 669324 (0.0007) [2023-12-26 20:14:02,857][105620] Updated weights for policy 1, policy_version 669334 (0.0005) [2023-12-26 20:14:03,425][105692] Updated weights for policy 0, policy_version 668399 (0.0006) [2023-12-26 20:14:03,478][105585] KL-divergence is very high: 171.2011 [2023-12-26 20:14:03,478][105692] Updated weights for policy 0, policy_version 668409 (0.0009) [2023-12-26 20:14:03,526][105585] KL-divergence is very high: 256.6068 [2023-12-26 20:14:03,539][105692] Updated weights for policy 0, policy_version 668419 (0.0010) [2023-12-26 20:14:03,560][105620] Updated weights for policy 1, policy_version 669344 (0.0006) [2023-12-26 20:14:03,607][105620] Updated weights for policy 1, policy_version 669354 (0.0005) [2023-12-26 20:14:03,649][105620] Updated weights for policy 1, policy_version 669364 (0.0005) [2023-12-26 20:14:04,215][105692] Updated weights for policy 0, policy_version 668429 (0.0010) [2023-12-26 20:14:04,267][105620] Updated weights for policy 1, policy_version 669374 (0.0009) [2023-12-26 20:14:04,278][105692] Updated weights for policy 0, policy_version 668439 (0.0007) [2023-12-26 20:14:04,331][105620] Updated weights for policy 1, policy_version 669384 (0.0011) [2023-12-26 20:14:04,342][105692] Updated weights for policy 0, policy_version 668449 (0.0007) [2023-12-26 20:14:04,387][105620] Updated weights for policy 1, policy_version 669394 (0.0010) [2023-12-26 20:14:05,066][105620] Updated weights for policy 1, policy_version 669404 (0.0008) [2023-12-26 20:14:05,117][105692] Updated weights for policy 0, policy_version 668459 (0.0009) [2023-12-26 20:14:05,124][105620] Updated weights for policy 1, policy_version 669414 (0.0008) [2023-12-26 20:14:05,173][105692] Updated weights for policy 0, policy_version 668469 (0.0008) [2023-12-26 20:14:05,180][105620] Updated weights for policy 1, policy_version 669424 (0.0007) [2023-12-26 20:14:05,222][105692] Updated weights for policy 0, policy_version 668479 (0.0009) [2023-12-26 20:14:05,784][105620] Updated weights for policy 1, policy_version 669434 (0.0006) [2023-12-26 20:14:05,834][105620] Updated weights for policy 1, policy_version 669444 (0.0005) [2023-12-26 20:14:05,836][105692] Updated weights for policy 0, policy_version 668489 (0.0009) [2023-12-26 20:14:05,888][105692] Updated weights for policy 0, policy_version 668499 (0.0009) [2023-12-26 20:14:05,889][105620] Updated weights for policy 1, policy_version 669454 (0.0006) [2023-12-26 20:14:05,940][105692] Updated weights for policy 0, policy_version 668509 (0.0007) [2023-12-26 20:14:05,944][105620] Updated weights for policy 1, policy_version 669464 (0.0007) [2023-12-26 20:14:05,993][105692] Updated weights for policy 0, policy_version 668519 (0.0010) [2023-12-26 20:14:06,062][104569] Fps is (10 sec: 20479.2, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 342573056. Throughput: 0: 9662.2, 1: 9606.7. Samples: 342554348. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 20:14:06,063][104569] Avg episode reward: [(0, '8725.984'), (1, '8617.980')] [2023-12-26 20:14:06,619][105620] Updated weights for policy 1, policy_version 669474 (0.0011) [2023-12-26 20:14:06,682][105620] Updated weights for policy 1, policy_version 669484 (0.0006) [2023-12-26 20:14:06,739][105620] Updated weights for policy 1, policy_version 669494 (0.0009) [2023-12-26 20:14:06,815][105692] Updated weights for policy 0, policy_version 668529 (0.0007) [2023-12-26 20:14:06,870][105692] Updated weights for policy 0, policy_version 668539 (0.0008) [2023-12-26 20:14:06,927][105692] Updated weights for policy 0, policy_version 668549 (0.0007) [2023-12-26 20:14:07,455][105620] Updated weights for policy 1, policy_version 669504 (0.0010) [2023-12-26 20:14:07,499][105620] Updated weights for policy 1, policy_version 669514 (0.0010) [2023-12-26 20:14:07,528][105692] Updated weights for policy 0, policy_version 668559 (0.0005) [2023-12-26 20:14:07,544][105620] Updated weights for policy 1, policy_version 669524 (0.0010) [2023-12-26 20:14:07,589][105692] Updated weights for policy 0, policy_version 668569 (0.0007) [2023-12-26 20:14:07,645][105692] Updated weights for policy 0, policy_version 668579 (0.0008) [2023-12-26 20:14:08,249][105692] Updated weights for policy 0, policy_version 668589 (0.0009) [2023-12-26 20:14:08,270][105620] Updated weights for policy 1, policy_version 669534 (0.0009) [2023-12-26 20:14:08,303][105692] Updated weights for policy 0, policy_version 668599 (0.0010) [2023-12-26 20:14:08,331][105620] Updated weights for policy 1, policy_version 669544 (0.0007) [2023-12-26 20:14:08,367][105692] Updated weights for policy 0, policy_version 668609 (0.0010) [2023-12-26 20:14:08,395][105620] Updated weights for policy 1, policy_version 669554 (0.0008) [2023-12-26 20:14:09,112][105692] Updated weights for policy 0, policy_version 668619 (0.0010) [2023-12-26 20:14:09,159][105620] Updated weights for policy 1, policy_version 669564 (0.0008) [2023-12-26 20:14:09,169][105692] Updated weights for policy 0, policy_version 668629 (0.0011) [2023-12-26 20:14:09,212][105620] Updated weights for policy 1, policy_version 669574 (0.0006) [2023-12-26 20:14:09,223][105692] Updated weights for policy 0, policy_version 668639 (0.0010) [2023-12-26 20:14:09,280][105620] Updated weights for policy 1, policy_version 669584 (0.0009) [2023-12-26 20:14:09,931][105692] Updated weights for policy 0, policy_version 668649 (0.0011) [2023-12-26 20:14:09,995][105692] Updated weights for policy 0, policy_version 668659 (0.0010) [2023-12-26 20:14:10,052][105692] Updated weights for policy 0, policy_version 668669 (0.0011) [2023-12-26 20:14:10,099][105620] Updated weights for policy 1, policy_version 669594 (0.0008) [2023-12-26 20:14:10,118][105692] Updated weights for policy 0, policy_version 668679 (0.0009) [2023-12-26 20:14:10,151][105620] Updated weights for policy 1, policy_version 669604 (0.0008) [2023-12-26 20:14:10,203][105620] Updated weights for policy 1, policy_version 669614 (0.0008) [2023-12-26 20:14:10,255][105620] Updated weights for policy 1, policy_version 669624 (0.0008) [2023-12-26 20:14:10,838][105692] Updated weights for policy 0, policy_version 668689 (0.0009) [2023-12-26 20:14:10,900][105692] Updated weights for policy 0, policy_version 668699 (0.0009) [2023-12-26 20:14:10,958][105692] Updated weights for policy 0, policy_version 668709 (0.0007) [2023-12-26 20:14:10,981][105620] Updated weights for policy 1, policy_version 669634 (0.0005) [2023-12-26 20:14:11,026][105620] Updated weights for policy 1, policy_version 669644 (0.0006) [2023-12-26 20:14:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 342663168. Throughput: 0: 9719.8, 1: 9672.4. Samples: 342673280. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 20:14:11,063][104569] Avg episode reward: [(0, '8322.709'), (1, '8623.365')] [2023-12-26 20:14:11,085][105620] Updated weights for policy 1, policy_version 669654 (0.0009) [2023-12-26 20:14:11,622][105692] Updated weights for policy 0, policy_version 668719 (0.0007) [2023-12-26 20:14:11,689][105692] Updated weights for policy 0, policy_version 668729 (0.0009) [2023-12-26 20:14:11,754][105692] Updated weights for policy 0, policy_version 668739 (0.0008) [2023-12-26 20:14:11,827][105620] Updated weights for policy 1, policy_version 669664 (0.0008) [2023-12-26 20:14:11,892][105620] Updated weights for policy 1, policy_version 669674 (0.0009) [2023-12-26 20:14:11,959][105620] Updated weights for policy 1, policy_version 669684 (0.0009) [2023-12-26 20:14:12,547][105692] Updated weights for policy 0, policy_version 668749 (0.0009) [2023-12-26 20:14:12,594][105692] Updated weights for policy 0, policy_version 668759 (0.0009) [2023-12-26 20:14:12,625][105620] Updated weights for policy 1, policy_version 669694 (0.0008) [2023-12-26 20:14:12,650][105692] Updated weights for policy 0, policy_version 668769 (0.0007) [2023-12-26 20:14:12,675][105620] Updated weights for policy 1, policy_version 669704 (0.0008) [2023-12-26 20:14:12,720][105620] Updated weights for policy 1, policy_version 669714 (0.0008) [2023-12-26 20:14:13,288][105692] Updated weights for policy 0, policy_version 668779 (0.0007) [2023-12-26 20:14:13,346][105692] Updated weights for policy 0, policy_version 668789 (0.0005) [2023-12-26 20:14:13,402][105692] Updated weights for policy 0, policy_version 668799 (0.0006) [2023-12-26 20:14:13,543][105620] Updated weights for policy 1, policy_version 669724 (0.0009) [2023-12-26 20:14:13,596][105620] Updated weights for policy 1, policy_version 669734 (0.0010) [2023-12-26 20:14:13,644][105620] Updated weights for policy 1, policy_version 669744 (0.0007) [2023-12-26 20:14:14,040][105692] Updated weights for policy 0, policy_version 668809 (0.0009) [2023-12-26 20:14:14,095][105692] Updated weights for policy 0, policy_version 668819 (0.0005) [2023-12-26 20:14:14,150][105692] Updated weights for policy 0, policy_version 668829 (0.0009) [2023-12-26 20:14:14,205][105692] Updated weights for policy 0, policy_version 668839 (0.0008) [2023-12-26 20:14:14,315][105620] Updated weights for policy 1, policy_version 669754 (0.0006) [2023-12-26 20:14:14,369][105620] Updated weights for policy 1, policy_version 669764 (0.0009) [2023-12-26 20:14:14,429][105620] Updated weights for policy 1, policy_version 669774 (0.0006) [2023-12-26 20:14:14,492][105620] Updated weights for policy 1, policy_version 669784 (0.0010) [2023-12-26 20:14:14,791][105692] Updated weights for policy 0, policy_version 668849 (0.0007) [2023-12-26 20:14:14,852][105692] Updated weights for policy 0, policy_version 668859 (0.0009) [2023-12-26 20:14:14,910][105692] Updated weights for policy 0, policy_version 668869 (0.0007) [2023-12-26 20:14:15,241][105620] Updated weights for policy 1, policy_version 669794 (0.0011) [2023-12-26 20:14:15,301][105620] Updated weights for policy 1, policy_version 669804 (0.0011) [2023-12-26 20:14:15,350][105620] Updated weights for policy 1, policy_version 669814 (0.0010) [2023-12-26 20:14:15,582][105692] Updated weights for policy 0, policy_version 668879 (0.0007) [2023-12-26 20:14:15,647][105692] Updated weights for policy 0, policy_version 668889 (0.0009) [2023-12-26 20:14:15,702][105692] Updated weights for policy 0, policy_version 668899 (0.0010) [2023-12-26 20:14:16,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 342761472. Throughput: 0: 9717.6, 1: 9665.5. Samples: 342731932. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 20:14:16,063][104569] Avg episode reward: [(0, '7623.979'), (1, '8804.509')] [2023-12-26 20:14:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000668904_171270144.pth... [2023-12-26 20:14:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000667752_170975232.pth [2023-12-26 20:14:16,109][105620] Updated weights for policy 1, policy_version 669824 (0.0006) [2023-12-26 20:14:16,171][105620] Updated weights for policy 1, policy_version 669834 (0.0006) [2023-12-26 20:14:16,229][105620] Updated weights for policy 1, policy_version 669844 (0.0005) [2023-12-26 20:14:16,246][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000669848_171499520.pth... [2023-12-26 20:14:16,249][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000668696_171204608.pth [2023-12-26 20:14:16,412][105692] Updated weights for policy 0, policy_version 668909 (0.0008) [2023-12-26 20:14:16,462][105692] Updated weights for policy 0, policy_version 668919 (0.0005) [2023-12-26 20:14:16,512][105692] Updated weights for policy 0, policy_version 668929 (0.0005) [2023-12-26 20:14:16,741][105620] Updated weights for policy 1, policy_version 669854 (0.0008) [2023-12-26 20:14:16,789][105620] Updated weights for policy 1, policy_version 669864 (0.0011) [2023-12-26 20:14:16,854][105620] Updated weights for policy 1, policy_version 669874 (0.0009) [2023-12-26 20:14:17,089][105692] Updated weights for policy 0, policy_version 668939 (0.0007) [2023-12-26 20:14:17,145][105692] Updated weights for policy 0, policy_version 668949 (0.0010) [2023-12-26 20:14:17,201][105692] Updated weights for policy 0, policy_version 668959 (0.0009) [2023-12-26 20:14:17,560][105620] Updated weights for policy 1, policy_version 669884 (0.0009) [2023-12-26 20:14:17,604][105620] Updated weights for policy 1, policy_version 669894 (0.0010) [2023-12-26 20:14:17,651][105620] Updated weights for policy 1, policy_version 669904 (0.0010) [2023-12-26 20:14:17,949][105692] Updated weights for policy 0, policy_version 668969 (0.0010) [2023-12-26 20:14:18,013][105692] Updated weights for policy 0, policy_version 668979 (0.0010) [2023-12-26 20:14:18,075][105692] Updated weights for policy 0, policy_version 668989 (0.0010) [2023-12-26 20:14:18,139][105692] Updated weights for policy 0, policy_version 668999 (0.0009) [2023-12-26 20:14:18,314][105620] Updated weights for policy 1, policy_version 669914 (0.0010) [2023-12-26 20:14:18,376][105620] Updated weights for policy 1, policy_version 669924 (0.0008) [2023-12-26 20:14:18,440][105620] Updated weights for policy 1, policy_version 669934 (0.0007) [2023-12-26 20:14:18,502][105620] Updated weights for policy 1, policy_version 669944 (0.0010) [2023-12-26 20:14:18,862][105692] Updated weights for policy 0, policy_version 669009 (0.0007) [2023-12-26 20:14:18,913][105692] Updated weights for policy 0, policy_version 669019 (0.0010) [2023-12-26 20:14:18,966][105692] Updated weights for policy 0, policy_version 669029 (0.0009) [2023-12-26 20:14:19,208][105620] Updated weights for policy 1, policy_version 669954 (0.0009) [2023-12-26 20:14:19,269][105620] Updated weights for policy 1, policy_version 669964 (0.0008) [2023-12-26 20:14:19,336][105620] Updated weights for policy 1, policy_version 669974 (0.0008) [2023-12-26 20:14:19,601][105692] Updated weights for policy 0, policy_version 669039 (0.0006) [2023-12-26 20:14:19,654][105692] Updated weights for policy 0, policy_version 669049 (0.0006) [2023-12-26 20:14:19,713][105692] Updated weights for policy 0, policy_version 669059 (0.0006) [2023-12-26 20:14:20,155][105620] Updated weights for policy 1, policy_version 669984 (0.0008) [2023-12-26 20:14:20,225][105620] Updated weights for policy 1, policy_version 669994 (0.0010) [2023-12-26 20:14:20,298][105620] Updated weights for policy 1, policy_version 670004 (0.0009) [2023-12-26 20:14:20,317][105692] Updated weights for policy 0, policy_version 669069 (0.0007) [2023-12-26 20:14:20,372][105692] Updated weights for policy 0, policy_version 669079 (0.0009) [2023-12-26 20:14:20,428][105692] Updated weights for policy 0, policy_version 669089 (0.0009) [2023-12-26 20:14:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 342859776. Throughput: 0: 9809.3, 1: 9763.4. Samples: 342854296. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 20:14:21,062][104569] Avg episode reward: [(0, '8403.847'), (1, '8442.857')] [2023-12-26 20:14:21,064][105620] Updated weights for policy 1, policy_version 670014 (0.0008) [2023-12-26 20:14:21,125][105620] Updated weights for policy 1, policy_version 670024 (0.0008) [2023-12-26 20:14:21,186][105620] Updated weights for policy 1, policy_version 670034 (0.0007) [2023-12-26 20:14:21,236][105692] Updated weights for policy 0, policy_version 669099 (0.0009) [2023-12-26 20:14:21,303][105692] Updated weights for policy 0, policy_version 669109 (0.0009) [2023-12-26 20:14:21,385][105692] Updated weights for policy 0, policy_version 669119 (0.0009) [2023-12-26 20:14:21,941][105620] Updated weights for policy 1, policy_version 670044 (0.0008) [2023-12-26 20:14:21,995][105620] Updated weights for policy 1, policy_version 670054 (0.0011) [2023-12-26 20:14:22,052][105620] Updated weights for policy 1, policy_version 670064 (0.0011) [2023-12-26 20:14:22,135][105692] Updated weights for policy 0, policy_version 669129 (0.0011) [2023-12-26 20:14:22,194][105692] Updated weights for policy 0, policy_version 669139 (0.0011) [2023-12-26 20:14:22,239][105692] Updated weights for policy 0, policy_version 669149 (0.0010) [2023-12-26 20:14:22,303][105692] Updated weights for policy 0, policy_version 669159 (0.0011) [2023-12-26 20:14:22,769][105620] Updated weights for policy 1, policy_version 670074 (0.0010) [2023-12-26 20:14:22,828][105620] Updated weights for policy 1, policy_version 670084 (0.0009) [2023-12-26 20:14:22,883][105620] Updated weights for policy 1, policy_version 670094 (0.0009) [2023-12-26 20:14:22,939][105620] Updated weights for policy 1, policy_version 670104 (0.0009) [2023-12-26 20:14:23,095][105692] Updated weights for policy 0, policy_version 669170 (0.0010) [2023-12-26 20:14:23,148][105692] Updated weights for policy 0, policy_version 669180 (0.0009) [2023-12-26 20:14:23,204][105692] Updated weights for policy 0, policy_version 669190 (0.0009) [2023-12-26 20:14:23,691][105620] Updated weights for policy 1, policy_version 670114 (0.0009) [2023-12-26 20:14:23,738][105620] Updated weights for policy 1, policy_version 670124 (0.0008) [2023-12-26 20:14:23,792][105620] Updated weights for policy 1, policy_version 670134 (0.0009) [2023-12-26 20:14:23,987][105692] Updated weights for policy 0, policy_version 669200 (0.0009) [2023-12-26 20:14:24,041][105692] Updated weights for policy 0, policy_version 669210 (0.0009) [2023-12-26 20:14:24,094][105692] Updated weights for policy 0, policy_version 669220 (0.0010) [2023-12-26 20:14:24,488][105620] Updated weights for policy 1, policy_version 670144 (0.0007) [2023-12-26 20:14:24,542][105620] Updated weights for policy 1, policy_version 670154 (0.0005) [2023-12-26 20:14:24,591][105620] Updated weights for policy 1, policy_version 670164 (0.0005) [2023-12-26 20:14:24,922][105692] Updated weights for policy 0, policy_version 669230 (0.0010) [2023-12-26 20:14:24,969][105692] Updated weights for policy 0, policy_version 669240 (0.0008) [2023-12-26 20:14:25,017][105692] Updated weights for policy 0, policy_version 669250 (0.0009) [2023-12-26 20:14:25,270][105620] Updated weights for policy 1, policy_version 670174 (0.0008) [2023-12-26 20:14:25,327][105620] Updated weights for policy 1, policy_version 670184 (0.0009) [2023-12-26 20:14:25,388][105620] Updated weights for policy 1, policy_version 670194 (0.0009) [2023-12-26 20:14:25,757][105692] Updated weights for policy 0, policy_version 669260 (0.0009) [2023-12-26 20:14:25,805][105692] Updated weights for policy 0, policy_version 669270 (0.0009) [2023-12-26 20:14:25,856][105692] Updated weights for policy 0, policy_version 669280 (0.0009) [2023-12-26 20:14:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 342958080. Throughput: 0: 9720.0, 1: 9808.6. Samples: 342966088. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 20:14:26,063][104569] Avg episode reward: [(0, '8115.435'), (1, '8614.842')] [2023-12-26 20:14:26,140][105620] Updated weights for policy 1, policy_version 670204 (0.0008) [2023-12-26 20:14:26,201][105620] Updated weights for policy 1, policy_version 670214 (0.0007) [2023-12-26 20:14:26,260][105620] Updated weights for policy 1, policy_version 670224 (0.0007) [2023-12-26 20:14:26,578][105692] Updated weights for policy 0, policy_version 669290 (0.0009) [2023-12-26 20:14:26,628][105692] Updated weights for policy 0, policy_version 669301 (0.0009) [2023-12-26 20:14:26,677][105692] Updated weights for policy 0, policy_version 669311 (0.0010) [2023-12-26 20:14:26,866][105620] Updated weights for policy 1, policy_version 670234 (0.0006) [2023-12-26 20:14:26,912][105620] Updated weights for policy 1, policy_version 670244 (0.0009) [2023-12-26 20:14:26,956][105620] Updated weights for policy 1, policy_version 670254 (0.0010) [2023-12-26 20:14:27,003][105620] Updated weights for policy 1, policy_version 670264 (0.0010) [2023-12-26 20:14:27,330][105692] Updated weights for policy 0, policy_version 669321 (0.0010) [2023-12-26 20:14:27,387][105692] Updated weights for policy 0, policy_version 669331 (0.0006) [2023-12-26 20:14:27,438][105692] Updated weights for policy 0, policy_version 669341 (0.0006) [2023-12-26 20:14:27,485][105692] Updated weights for policy 0, policy_version 669351 (0.0008) [2023-12-26 20:14:27,684][105620] Updated weights for policy 1, policy_version 670274 (0.0005) [2023-12-26 20:14:27,742][105620] Updated weights for policy 1, policy_version 670284 (0.0005) [2023-12-26 20:14:27,811][105620] Updated weights for policy 1, policy_version 670294 (0.0005) [2023-12-26 20:14:28,030][105692] Updated weights for policy 0, policy_version 669361 (0.0005) [2023-12-26 20:14:28,074][105692] Updated weights for policy 0, policy_version 669371 (0.0005) [2023-12-26 20:14:28,130][105692] Updated weights for policy 0, policy_version 669381 (0.0005) [2023-12-26 20:14:28,302][105620] Updated weights for policy 1, policy_version 670304 (0.0005) [2023-12-26 20:14:28,362][105620] Updated weights for policy 1, policy_version 670314 (0.0010) [2023-12-26 20:14:28,421][105620] Updated weights for policy 1, policy_version 670324 (0.0011) [2023-12-26 20:14:28,768][105692] Updated weights for policy 0, policy_version 669391 (0.0007) [2023-12-26 20:14:28,816][105692] Updated weights for policy 0, policy_version 669401 (0.0008) [2023-12-26 20:14:28,868][105692] Updated weights for policy 0, policy_version 669411 (0.0008) [2023-12-26 20:14:29,097][105620] Updated weights for policy 1, policy_version 670334 (0.0007) [2023-12-26 20:14:29,153][105620] Updated weights for policy 1, policy_version 670344 (0.0005) [2023-12-26 20:14:29,216][105620] Updated weights for policy 1, policy_version 670354 (0.0006) [2023-12-26 20:14:29,725][105692] Updated weights for policy 0, policy_version 669421 (0.0008) [2023-12-26 20:14:29,777][105692] Updated weights for policy 0, policy_version 669431 (0.0008) [2023-12-26 20:14:29,839][105692] Updated weights for policy 0, policy_version 669441 (0.0008) [2023-12-26 20:14:29,861][105620] Updated weights for policy 1, policy_version 670364 (0.0009) [2023-12-26 20:14:29,909][105620] Updated weights for policy 1, policy_version 670374 (0.0010) [2023-12-26 20:14:29,969][105620] Updated weights for policy 1, policy_version 670384 (0.0008) [2023-12-26 20:14:30,555][105620] Updated weights for policy 1, policy_version 670394 (0.0006) [2023-12-26 20:14:30,607][105620] Updated weights for policy 1, policy_version 670404 (0.0005) [2023-12-26 20:14:30,657][105620] Updated weights for policy 1, policy_version 670414 (0.0005) [2023-12-26 20:14:30,710][105692] Updated weights for policy 0, policy_version 669451 (0.0007) [2023-12-26 20:14:30,713][105620] Updated weights for policy 1, policy_version 670424 (0.0005) [2023-12-26 20:14:30,761][105692] Updated weights for policy 0, policy_version 669461 (0.0009) [2023-12-26 20:14:30,810][105692] Updated weights for policy 0, policy_version 669471 (0.0005) [2023-12-26 20:14:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 343064576. Throughput: 0: 9813.9, 1: 9946.1. Samples: 343032036. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 20:14:31,063][104569] Avg episode reward: [(0, '8713.730'), (1, '8713.361')] [2023-12-26 20:14:31,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000669480_171417600.pth... [2023-12-26 20:14:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000670424_171646976.pth... [2023-12-26 20:14:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000668328_171122688.pth [2023-12-26 20:14:31,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000669272_171352064.pth [2023-12-26 20:14:31,303][105620] Updated weights for policy 1, policy_version 670434 (0.0006) [2023-12-26 20:14:31,361][105620] Updated weights for policy 1, policy_version 670444 (0.0012) [2023-12-26 20:14:31,414][105620] Updated weights for policy 1, policy_version 670454 (0.0008) [2023-12-26 20:14:31,490][105692] Updated weights for policy 0, policy_version 669481 (0.0005) [2023-12-26 20:14:31,556][105692] Updated weights for policy 0, policy_version 669491 (0.0005) [2023-12-26 20:14:31,624][105692] Updated weights for policy 0, policy_version 669501 (0.0007) [2023-12-26 20:14:31,680][105692] Updated weights for policy 0, policy_version 669511 (0.0010) [2023-12-26 20:14:32,161][105620] Updated weights for policy 1, policy_version 670464 (0.0009) [2023-12-26 20:14:32,216][105620] Updated weights for policy 1, policy_version 670474 (0.0008) [2023-12-26 20:14:32,265][105620] Updated weights for policy 1, policy_version 670484 (0.0008) [2023-12-26 20:14:32,279][105586] KL-divergence is very high: 125.8697 [2023-12-26 20:14:32,389][105692] Updated weights for policy 0, policy_version 669521 (0.0007) [2023-12-26 20:14:32,446][105692] Updated weights for policy 0, policy_version 669531 (0.0006) [2023-12-26 20:14:32,501][105692] Updated weights for policy 0, policy_version 669541 (0.0008) [2023-12-26 20:14:33,074][105620] Updated weights for policy 1, policy_version 670494 (0.0009) [2023-12-26 20:14:33,125][105620] Updated weights for policy 1, policy_version 670504 (0.0009) [2023-12-26 20:14:33,165][105692] Updated weights for policy 0, policy_version 669551 (0.0008) [2023-12-26 20:14:33,178][105620] Updated weights for policy 1, policy_version 670514 (0.0007) [2023-12-26 20:14:33,211][105692] Updated weights for policy 0, policy_version 669561 (0.0005) [2023-12-26 20:14:33,269][105692] Updated weights for policy 0, policy_version 669571 (0.0007) [2023-12-26 20:14:33,905][105620] Updated weights for policy 1, policy_version 670524 (0.0007) [2023-12-26 20:14:33,964][105620] Updated weights for policy 1, policy_version 670534 (0.0005) [2023-12-26 20:14:34,014][105620] Updated weights for policy 1, policy_version 670544 (0.0005) [2023-12-26 20:14:34,067][105692] Updated weights for policy 0, policy_version 669581 (0.0009) [2023-12-26 20:14:34,116][105692] Updated weights for policy 0, policy_version 669591 (0.0010) [2023-12-26 20:14:34,197][105692] Updated weights for policy 0, policy_version 669601 (0.0010) [2023-12-26 20:14:34,613][105620] Updated weights for policy 1, policy_version 670554 (0.0006) [2023-12-26 20:14:34,673][105620] Updated weights for policy 1, policy_version 670564 (0.0009) [2023-12-26 20:14:34,731][105620] Updated weights for policy 1, policy_version 670574 (0.0005) [2023-12-26 20:14:34,794][105620] Updated weights for policy 1, policy_version 670584 (0.0005) [2023-12-26 20:14:34,952][105692] Updated weights for policy 0, policy_version 669611 (0.0011) [2023-12-26 20:14:35,011][105692] Updated weights for policy 0, policy_version 669621 (0.0010) [2023-12-26 20:14:35,067][105692] Updated weights for policy 0, policy_version 669631 (0.0010) [2023-12-26 20:14:35,333][105620] Updated weights for policy 1, policy_version 670594 (0.0005) [2023-12-26 20:14:35,396][105620] Updated weights for policy 1, policy_version 670604 (0.0005) [2023-12-26 20:14:35,453][105620] Updated weights for policy 1, policy_version 670614 (0.0006) [2023-12-26 20:14:35,787][105692] Updated weights for policy 0, policy_version 669641 (0.0010) [2023-12-26 20:14:35,853][105692] Updated weights for policy 0, policy_version 669651 (0.0005) [2023-12-26 20:14:35,916][105692] Updated weights for policy 0, policy_version 669661 (0.0009) [2023-12-26 20:14:35,961][105692] Updated weights for policy 0, policy_version 669671 (0.0010) [2023-12-26 20:14:36,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 343162880. Throughput: 0: 9750.8, 1: 9907.6. Samples: 343150100. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 20:14:36,062][104569] Avg episode reward: [(0, '8837.895'), (1, '8620.347')] [2023-12-26 20:14:36,133][105620] Updated weights for policy 1, policy_version 670626 (0.0022) [2023-12-26 20:14:36,211][105620] Updated weights for policy 1, policy_version 670636 (0.0008) [2023-12-26 20:14:36,257][105620] Updated weights for policy 1, policy_version 670646 (0.0008) [2023-12-26 20:14:36,665][105692] Updated weights for policy 0, policy_version 669681 (0.0011) [2023-12-26 20:14:36,724][105692] Updated weights for policy 0, policy_version 669691 (0.0010) [2023-12-26 20:14:36,783][105692] Updated weights for policy 0, policy_version 669701 (0.0011) [2023-12-26 20:14:36,992][105620] Updated weights for policy 1, policy_version 670656 (0.0008) [2023-12-26 20:14:37,051][105620] Updated weights for policy 1, policy_version 670666 (0.0008) [2023-12-26 20:14:37,106][105620] Updated weights for policy 1, policy_version 670676 (0.0008) [2023-12-26 20:14:37,528][105692] Updated weights for policy 0, policy_version 669711 (0.0007) [2023-12-26 20:14:37,599][105692] Updated weights for policy 0, policy_version 669721 (0.0006) [2023-12-26 20:14:37,658][105692] Updated weights for policy 0, policy_version 669731 (0.0006) [2023-12-26 20:14:37,776][105620] Updated weights for policy 1, policy_version 670686 (0.0006) [2023-12-26 20:14:37,840][105620] Updated weights for policy 1, policy_version 670696 (0.0005) [2023-12-26 20:14:37,907][105620] Updated weights for policy 1, policy_version 670706 (0.0006) [2023-12-26 20:14:38,172][105692] Updated weights for policy 0, policy_version 669741 (0.0006) [2023-12-26 20:14:38,230][105692] Updated weights for policy 0, policy_version 669751 (0.0005) [2023-12-26 20:14:38,296][105692] Updated weights for policy 0, policy_version 669761 (0.0006) [2023-12-26 20:14:38,639][105620] Updated weights for policy 1, policy_version 670717 (0.0010) [2023-12-26 20:14:38,691][105620] Updated weights for policy 1, policy_version 670727 (0.0010) [2023-12-26 20:14:38,741][105620] Updated weights for policy 1, policy_version 670737 (0.0010) [2023-12-26 20:14:38,879][105692] Updated weights for policy 0, policy_version 669771 (0.0007) [2023-12-26 20:14:38,940][105692] Updated weights for policy 0, policy_version 669781 (0.0008) [2023-12-26 20:14:38,996][105692] Updated weights for policy 0, policy_version 669791 (0.0006) [2023-12-26 20:14:39,503][105620] Updated weights for policy 1, policy_version 670747 (0.0010) [2023-12-26 20:14:39,563][105620] Updated weights for policy 1, policy_version 670757 (0.0011) [2023-12-26 20:14:39,631][105620] Updated weights for policy 1, policy_version 670767 (0.0011) [2023-12-26 20:14:39,661][105692] Updated weights for policy 0, policy_version 669801 (0.0005) [2023-12-26 20:14:39,723][105692] Updated weights for policy 0, policy_version 669811 (0.0007) [2023-12-26 20:14:39,780][105692] Updated weights for policy 0, policy_version 669821 (0.0007) [2023-12-26 20:14:39,844][105692] Updated weights for policy 0, policy_version 669831 (0.0009) [2023-12-26 20:14:40,349][105620] Updated weights for policy 1, policy_version 670777 (0.0010) [2023-12-26 20:14:40,427][105620] Updated weights for policy 1, policy_version 670787 (0.0006) [2023-12-26 20:14:40,494][105620] Updated weights for policy 1, policy_version 670797 (0.0010) [2023-12-26 20:14:40,565][105620] Updated weights for policy 1, policy_version 670807 (0.0008) [2023-12-26 20:14:40,615][105692] Updated weights for policy 0, policy_version 669841 (0.0008) [2023-12-26 20:14:40,687][105692] Updated weights for policy 0, policy_version 669851 (0.0007) [2023-12-26 20:14:40,742][105692] Updated weights for policy 0, policy_version 669861 (0.0006) [2023-12-26 20:14:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 343261184. Throughput: 0: 9786.7, 1: 9949.2. Samples: 343269860. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 20:14:41,062][104569] Avg episode reward: [(0, '8815.229'), (1, '8619.030')] [2023-12-26 20:14:41,325][105620] Updated weights for policy 1, policy_version 670817 (0.0009) [2023-12-26 20:14:41,391][105620] Updated weights for policy 1, policy_version 670827 (0.0010) [2023-12-26 20:14:41,423][105692] Updated weights for policy 0, policy_version 669871 (0.0006) [2023-12-26 20:14:41,446][105620] Updated weights for policy 1, policy_version 670837 (0.0009) [2023-12-26 20:14:41,483][105692] Updated weights for policy 0, policy_version 669881 (0.0007) [2023-12-26 20:14:41,554][105692] Updated weights for policy 0, policy_version 669891 (0.0009) [2023-12-26 20:14:42,233][105620] Updated weights for policy 1, policy_version 670847 (0.0010) [2023-12-26 20:14:42,299][105620] Updated weights for policy 1, policy_version 670857 (0.0011) [2023-12-26 20:14:42,326][105692] Updated weights for policy 0, policy_version 669901 (0.0009) [2023-12-26 20:14:42,370][105620] Updated weights for policy 1, policy_version 670868 (0.0009) [2023-12-26 20:14:42,399][105692] Updated weights for policy 0, policy_version 669911 (0.0009) [2023-12-26 20:14:42,465][105692] Updated weights for policy 0, policy_version 669921 (0.0007) [2023-12-26 20:14:43,058][105620] Updated weights for policy 1, policy_version 670878 (0.0009) [2023-12-26 20:14:43,059][105692] Updated weights for policy 0, policy_version 669931 (0.0008) [2023-12-26 20:14:43,107][105692] Updated weights for policy 0, policy_version 669941 (0.0006) [2023-12-26 20:14:43,109][105620] Updated weights for policy 1, policy_version 670888 (0.0010) [2023-12-26 20:14:43,152][105692] Updated weights for policy 0, policy_version 669951 (0.0008) [2023-12-26 20:14:43,164][105620] Updated weights for policy 1, policy_version 670898 (0.0010) [2023-12-26 20:14:43,743][105620] Updated weights for policy 1, policy_version 670908 (0.0008) [2023-12-26 20:14:43,805][105620] Updated weights for policy 1, policy_version 670918 (0.0006) [2023-12-26 20:14:43,810][105692] Updated weights for policy 0, policy_version 669961 (0.0006) [2023-12-26 20:14:43,863][105620] Updated weights for policy 1, policy_version 670928 (0.0008) [2023-12-26 20:14:43,876][105692] Updated weights for policy 0, policy_version 669971 (0.0010) [2023-12-26 20:14:43,946][105692] Updated weights for policy 0, policy_version 669981 (0.0011) [2023-12-26 20:14:44,013][105692] Updated weights for policy 0, policy_version 669991 (0.0010) [2023-12-26 20:14:44,558][105620] Updated weights for policy 1, policy_version 670938 (0.0007) [2023-12-26 20:14:44,614][105620] Updated weights for policy 1, policy_version 670948 (0.0008) [2023-12-26 20:14:44,669][105620] Updated weights for policy 1, policy_version 670958 (0.0007) [2023-12-26 20:14:44,693][105692] Updated weights for policy 0, policy_version 670001 (0.0007) [2023-12-26 20:14:44,730][105620] Updated weights for policy 1, policy_version 670968 (0.0008) [2023-12-26 20:14:44,740][105692] Updated weights for policy 0, policy_version 670011 (0.0007) [2023-12-26 20:14:44,801][105692] Updated weights for policy 0, policy_version 670021 (0.0008) [2023-12-26 20:14:45,475][105692] Updated weights for policy 0, policy_version 670031 (0.0006) [2023-12-26 20:14:45,477][105620] Updated weights for policy 1, policy_version 670978 (0.0009) [2023-12-26 20:14:45,527][105620] Updated weights for policy 1, policy_version 670988 (0.0008) [2023-12-26 20:14:45,542][105692] Updated weights for policy 0, policy_version 670041 (0.0006) [2023-12-26 20:14:45,580][105620] Updated weights for policy 1, policy_version 670998 (0.0008) [2023-12-26 20:14:45,610][105692] Updated weights for policy 0, policy_version 670051 (0.0006) [2023-12-26 20:14:46,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19797.2, 300 sec: 19521.9). Total num frames: 343359488. Throughput: 0: 9823.0, 1: 9993.7. Samples: 343329744. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 20:14:46,063][104569] Avg episode reward: [(0, '8649.309'), (1, '8622.855')] [2023-12-26 20:14:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000671000_171794432.pth... [2023-12-26 20:14:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000670056_171565056.pth... [2023-12-26 20:14:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000669848_171499520.pth [2023-12-26 20:14:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000668904_171270144.pth [2023-12-26 20:14:46,202][105692] Updated weights for policy 0, policy_version 670061 (0.0008) [2023-12-26 20:14:46,266][105692] Updated weights for policy 0, policy_version 670071 (0.0005) [2023-12-26 20:14:46,322][105692] Updated weights for policy 0, policy_version 670081 (0.0007) [2023-12-26 20:14:46,430][105620] Updated weights for policy 1, policy_version 671008 (0.0008) [2023-12-26 20:14:46,490][105620] Updated weights for policy 1, policy_version 671018 (0.0009) [2023-12-26 20:14:46,547][105620] Updated weights for policy 1, policy_version 671028 (0.0010) [2023-12-26 20:14:46,895][105692] Updated weights for policy 0, policy_version 670091 (0.0006) [2023-12-26 20:14:46,944][105692] Updated weights for policy 0, policy_version 670101 (0.0006) [2023-12-26 20:14:47,000][105692] Updated weights for policy 0, policy_version 670111 (0.0006) [2023-12-26 20:14:47,370][105620] Updated weights for policy 1, policy_version 671038 (0.0008) [2023-12-26 20:14:47,428][105620] Updated weights for policy 1, policy_version 671048 (0.0009) [2023-12-26 20:14:47,482][105620] Updated weights for policy 1, policy_version 671058 (0.0008) [2023-12-26 20:14:47,647][105692] Updated weights for policy 0, policy_version 670121 (0.0008) [2023-12-26 20:14:47,693][105692] Updated weights for policy 0, policy_version 670131 (0.0005) [2023-12-26 20:14:47,741][105692] Updated weights for policy 0, policy_version 670141 (0.0008) [2023-12-26 20:14:47,801][105692] Updated weights for policy 0, policy_version 670151 (0.0005) [2023-12-26 20:14:48,364][105692] Updated weights for policy 0, policy_version 670161 (0.0008) [2023-12-26 20:14:48,377][105620] Updated weights for policy 1, policy_version 671068 (0.0009) [2023-12-26 20:14:48,415][105692] Updated weights for policy 0, policy_version 670171 (0.0006) [2023-12-26 20:14:48,436][105620] Updated weights for policy 1, policy_version 671078 (0.0009) [2023-12-26 20:14:48,464][105692] Updated weights for policy 0, policy_version 670181 (0.0007) [2023-12-26 20:14:48,501][105620] Updated weights for policy 1, policy_version 671088 (0.0008) [2023-12-26 20:14:49,086][105692] Updated weights for policy 0, policy_version 670191 (0.0009) [2023-12-26 20:14:49,130][105692] Updated weights for policy 0, policy_version 670201 (0.0005) [2023-12-26 20:14:49,177][105692] Updated weights for policy 0, policy_version 670211 (0.0005) [2023-12-26 20:14:49,330][105620] Updated weights for policy 1, policy_version 671098 (0.0009) [2023-12-26 20:14:49,397][105620] Updated weights for policy 1, policy_version 671108 (0.0009) [2023-12-26 20:14:49,462][105620] Updated weights for policy 1, policy_version 671118 (0.0009) [2023-12-26 20:14:49,856][105692] Updated weights for policy 0, policy_version 670221 (0.0007) [2023-12-26 20:14:49,912][105692] Updated weights for policy 0, policy_version 670231 (0.0009) [2023-12-26 20:14:49,976][105692] Updated weights for policy 0, policy_version 670241 (0.0009) [2023-12-26 20:14:50,276][105620] Updated weights for policy 1, policy_version 671129 (0.0010) [2023-12-26 20:14:50,328][105620] Updated weights for policy 1, policy_version 671139 (0.0009) [2023-12-26 20:14:50,394][105620] Updated weights for policy 1, policy_version 671149 (0.0009) [2023-12-26 20:14:50,454][105620] Updated weights for policy 1, policy_version 671159 (0.0009) [2023-12-26 20:14:50,756][105692] Updated weights for policy 0, policy_version 670251 (0.0010) [2023-12-26 20:14:50,804][105692] Updated weights for policy 0, policy_version 670261 (0.0009) [2023-12-26 20:14:50,852][105692] Updated weights for policy 0, policy_version 670271 (0.0009) [2023-12-26 20:14:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 343457792. Throughput: 0: 10051.1, 1: 9784.4. Samples: 343446940. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 20:14:51,063][104569] Avg episode reward: [(0, '8636.133'), (1, '8806.024')] [2023-12-26 20:14:51,188][105620] Updated weights for policy 1, policy_version 671169 (0.0009) [2023-12-26 20:14:51,246][105620] Updated weights for policy 1, policy_version 671179 (0.0009) [2023-12-26 20:14:51,320][105620] Updated weights for policy 1, policy_version 671189 (0.0010) [2023-12-26 20:14:51,594][105692] Updated weights for policy 0, policy_version 670281 (0.0009) [2023-12-26 20:14:51,660][105692] Updated weights for policy 0, policy_version 670291 (0.0009) [2023-12-26 20:14:51,727][105692] Updated weights for policy 0, policy_version 670301 (0.0008) [2023-12-26 20:14:51,792][105692] Updated weights for policy 0, policy_version 670311 (0.0006) [2023-12-26 20:14:52,012][105620] Updated weights for policy 1, policy_version 671199 (0.0007) [2023-12-26 20:14:52,070][105620] Updated weights for policy 1, policy_version 671209 (0.0006) [2023-12-26 20:14:52,118][105620] Updated weights for policy 1, policy_version 671219 (0.0008) [2023-12-26 20:14:52,441][105692] Updated weights for policy 0, policy_version 670321 (0.0008) [2023-12-26 20:14:52,490][105692] Updated weights for policy 0, policy_version 670331 (0.0008) [2023-12-26 20:14:52,555][105692] Updated weights for policy 0, policy_version 670341 (0.0008) [2023-12-26 20:14:52,869][105620] Updated weights for policy 1, policy_version 671229 (0.0010) [2023-12-26 20:14:52,932][105620] Updated weights for policy 1, policy_version 671239 (0.0010) [2023-12-26 20:14:52,990][105620] Updated weights for policy 1, policy_version 671249 (0.0010) [2023-12-26 20:14:53,344][105692] Updated weights for policy 0, policy_version 670351 (0.0009) [2023-12-26 20:14:53,402][105692] Updated weights for policy 0, policy_version 670361 (0.0010) [2023-12-26 20:14:53,465][105692] Updated weights for policy 0, policy_version 670371 (0.0010) [2023-12-26 20:14:53,642][105620] Updated weights for policy 1, policy_version 671259 (0.0010) [2023-12-26 20:14:53,697][105620] Updated weights for policy 1, policy_version 671269 (0.0007) [2023-12-26 20:14:53,754][105620] Updated weights for policy 1, policy_version 671279 (0.0005) [2023-12-26 20:14:54,244][105692] Updated weights for policy 0, policy_version 670381 (0.0009) [2023-12-26 20:14:54,297][105692] Updated weights for policy 0, policy_version 670391 (0.0007) [2023-12-26 20:14:54,345][105692] Updated weights for policy 0, policy_version 670401 (0.0010) [2023-12-26 20:14:54,390][105620] Updated weights for policy 1, policy_version 671289 (0.0006) [2023-12-26 20:14:54,447][105620] Updated weights for policy 1, policy_version 671299 (0.0008) [2023-12-26 20:14:54,498][105620] Updated weights for policy 1, policy_version 671309 (0.0009) [2023-12-26 20:14:54,556][105620] Updated weights for policy 1, policy_version 671319 (0.0008) [2023-12-26 20:14:55,036][105692] Updated weights for policy 0, policy_version 670411 (0.0010) [2023-12-26 20:14:55,084][105692] Updated weights for policy 0, policy_version 670421 (0.0007) [2023-12-26 20:14:55,138][105692] Updated weights for policy 0, policy_version 670431 (0.0007) [2023-12-26 20:14:55,296][105620] Updated weights for policy 1, policy_version 671329 (0.0007) [2023-12-26 20:14:55,348][105620] Updated weights for policy 1, policy_version 671339 (0.0007) [2023-12-26 20:14:55,407][105620] Updated weights for policy 1, policy_version 671349 (0.0008) [2023-12-26 20:14:55,907][105692] Updated weights for policy 0, policy_version 670441 (0.0010) [2023-12-26 20:14:55,965][105692] Updated weights for policy 0, policy_version 670451 (0.0011) [2023-12-26 20:14:56,025][105692] Updated weights for policy 0, policy_version 670461 (0.0010) [2023-12-26 20:14:56,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 343547904. Throughput: 0: 9986.3, 1: 9788.0. Samples: 343563124. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 20:14:56,063][104569] Avg episode reward: [(0, '7403.378'), (1, '8892.062')] [2023-12-26 20:14:56,083][105620] Updated weights for policy 1, policy_version 671359 (0.0007) [2023-12-26 20:14:56,091][105692] Updated weights for policy 0, policy_version 670471 (0.0011) [2023-12-26 20:14:56,135][105620] Updated weights for policy 1, policy_version 671369 (0.0008) [2023-12-26 20:14:56,184][105620] Updated weights for policy 1, policy_version 671379 (0.0008) [2023-12-26 20:14:56,673][105692] Updated weights for policy 0, policy_version 670481 (0.0006) [2023-12-26 20:14:56,723][105692] Updated weights for policy 0, policy_version 670491 (0.0005) [2023-12-26 20:14:56,772][105692] Updated weights for policy 0, policy_version 670501 (0.0005) [2023-12-26 20:14:56,916][105620] Updated weights for policy 1, policy_version 671389 (0.0007) [2023-12-26 20:14:56,976][105620] Updated weights for policy 1, policy_version 671399 (0.0009) [2023-12-26 20:14:57,020][105620] Updated weights for policy 1, policy_version 671409 (0.0010) [2023-12-26 20:14:57,301][105692] Updated weights for policy 0, policy_version 670511 (0.0005) [2023-12-26 20:14:57,359][105692] Updated weights for policy 0, policy_version 670521 (0.0005) [2023-12-26 20:14:57,415][105692] Updated weights for policy 0, policy_version 670531 (0.0005) [2023-12-26 20:14:57,659][105620] Updated weights for policy 1, policy_version 671419 (0.0010) [2023-12-26 20:14:57,717][105620] Updated weights for policy 1, policy_version 671429 (0.0010) [2023-12-26 20:14:57,775][105620] Updated weights for policy 1, policy_version 671439 (0.0005) [2023-12-26 20:14:57,917][105692] Updated weights for policy 0, policy_version 670541 (0.0005) [2023-12-26 20:14:57,980][105692] Updated weights for policy 0, policy_version 670551 (0.0005) [2023-12-26 20:14:58,038][105692] Updated weights for policy 0, policy_version 670561 (0.0005) [2023-12-26 20:14:58,538][105620] Updated weights for policy 1, policy_version 671449 (0.0006) [2023-12-26 20:14:58,603][105620] Updated weights for policy 1, policy_version 671459 (0.0006) [2023-12-26 20:14:58,669][105620] Updated weights for policy 1, policy_version 671469 (0.0008) [2023-12-26 20:14:58,720][105692] Updated weights for policy 0, policy_version 670571 (0.0007) [2023-12-26 20:14:58,734][105620] Updated weights for policy 1, policy_version 671479 (0.0008) [2023-12-26 20:14:58,789][105692] Updated weights for policy 0, policy_version 670581 (0.0010) [2023-12-26 20:14:58,861][105692] Updated weights for policy 0, policy_version 670591 (0.0010) [2023-12-26 20:14:59,527][105620] Updated weights for policy 1, policy_version 671489 (0.0010) [2023-12-26 20:14:59,576][105692] Updated weights for policy 0, policy_version 670601 (0.0008) [2023-12-26 20:14:59,590][105620] Updated weights for policy 1, policy_version 671499 (0.0011) [2023-12-26 20:14:59,628][105692] Updated weights for policy 0, policy_version 670611 (0.0005) [2023-12-26 20:14:59,649][105620] Updated weights for policy 1, policy_version 671509 (0.0009) [2023-12-26 20:14:59,680][105692] Updated weights for policy 0, policy_version 670621 (0.0008) [2023-12-26 20:14:59,734][105692] Updated weights for policy 0, policy_version 670631 (0.0010) [2023-12-26 20:15:00,322][105620] Updated weights for policy 1, policy_version 671519 (0.0009) [2023-12-26 20:15:00,373][105620] Updated weights for policy 1, policy_version 671529 (0.0010) [2023-12-26 20:15:00,425][105620] Updated weights for policy 1, policy_version 671539 (0.0010) [2023-12-26 20:15:00,435][105692] Updated weights for policy 0, policy_version 670641 (0.0007) [2023-12-26 20:15:00,490][105692] Updated weights for policy 0, policy_version 670651 (0.0008) [2023-12-26 20:15:00,543][105692] Updated weights for policy 0, policy_version 670661 (0.0008) [2023-12-26 20:15:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19521.9). Total num frames: 343654400. Throughput: 0: 10090.8, 1: 9797.6. Samples: 343626908. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 20:15:01,063][104569] Avg episode reward: [(0, '7993.298'), (1, '9075.328')] [2023-12-26 20:15:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000670664_171720704.pth... [2023-12-26 20:15:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000671544_171933696.pth... [2023-12-26 20:15:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000670424_171646976.pth [2023-12-26 20:15:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000669480_171417600.pth [2023-12-26 20:15:01,202][105620] Updated weights for policy 1, policy_version 671549 (0.0008) [2023-12-26 20:15:01,204][105692] Updated weights for policy 0, policy_version 670671 (0.0008) [2023-12-26 20:15:01,260][105620] Updated weights for policy 1, policy_version 671559 (0.0006) [2023-12-26 20:15:01,267][105692] Updated weights for policy 0, policy_version 670681 (0.0007) [2023-12-26 20:15:01,316][105620] Updated weights for policy 1, policy_version 671569 (0.0007) [2023-12-26 20:15:01,330][105692] Updated weights for policy 0, policy_version 670691 (0.0007) [2023-12-26 20:15:02,052][105620] Updated weights for policy 1, policy_version 671579 (0.0007) [2023-12-26 20:15:02,073][105692] Updated weights for policy 0, policy_version 670701 (0.0009) [2023-12-26 20:15:02,100][105620] Updated weights for policy 1, policy_version 671589 (0.0006) [2023-12-26 20:15:02,124][105692] Updated weights for policy 0, policy_version 670711 (0.0010) [2023-12-26 20:15:02,162][105620] Updated weights for policy 1, policy_version 671599 (0.0006) [2023-12-26 20:15:02,177][105692] Updated weights for policy 0, policy_version 670721 (0.0010) [2023-12-26 20:15:02,794][105692] Updated weights for policy 0, policy_version 670731 (0.0009) [2023-12-26 20:15:02,846][105692] Updated weights for policy 0, policy_version 670741 (0.0006) [2023-12-26 20:15:02,898][105692] Updated weights for policy 0, policy_version 670751 (0.0010) [2023-12-26 20:15:02,934][105620] Updated weights for policy 1, policy_version 671609 (0.0008) [2023-12-26 20:15:02,984][105620] Updated weights for policy 1, policy_version 671619 (0.0005) [2023-12-26 20:15:03,045][105620] Updated weights for policy 1, policy_version 671629 (0.0007) [2023-12-26 20:15:03,104][105620] Updated weights for policy 1, policy_version 671639 (0.0009) [2023-12-26 20:15:03,539][105692] Updated weights for policy 0, policy_version 670761 (0.0010) [2023-12-26 20:15:03,591][105692] Updated weights for policy 0, policy_version 670771 (0.0005) [2023-12-26 20:15:03,643][105692] Updated weights for policy 0, policy_version 670781 (0.0005) [2023-12-26 20:15:03,707][105692] Updated weights for policy 0, policy_version 670791 (0.0005) [2023-12-26 20:15:03,906][105620] Updated weights for policy 1, policy_version 671649 (0.0009) [2023-12-26 20:15:03,960][105620] Updated weights for policy 1, policy_version 671659 (0.0009) [2023-12-26 20:15:04,012][105620] Updated weights for policy 1, policy_version 671669 (0.0009) [2023-12-26 20:15:04,316][105692] Updated weights for policy 0, policy_version 670801 (0.0009) [2023-12-26 20:15:04,376][105692] Updated weights for policy 0, policy_version 670811 (0.0008) [2023-12-26 20:15:04,435][105692] Updated weights for policy 0, policy_version 670821 (0.0006) [2023-12-26 20:15:04,843][105620] Updated weights for policy 1, policy_version 671679 (0.0009) [2023-12-26 20:15:04,899][105620] Updated weights for policy 1, policy_version 671689 (0.0010) [2023-12-26 20:15:04,952][105620] Updated weights for policy 1, policy_version 671699 (0.0010) [2023-12-26 20:15:05,170][105692] Updated weights for policy 0, policy_version 670831 (0.0010) [2023-12-26 20:15:05,232][105692] Updated weights for policy 0, policy_version 670841 (0.0010) [2023-12-26 20:15:05,310][105692] Updated weights for policy 0, policy_version 670851 (0.0010) [2023-12-26 20:15:05,721][105620] Updated weights for policy 1, policy_version 671709 (0.0009) [2023-12-26 20:15:05,774][105620] Updated weights for policy 1, policy_version 671719 (0.0008) [2023-12-26 20:15:05,829][105620] Updated weights for policy 1, policy_version 671729 (0.0008) [2023-12-26 20:15:06,030][105692] Updated weights for policy 0, policy_version 670861 (0.0010) [2023-12-26 20:15:06,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19661.0, 300 sec: 19522.0). Total num frames: 343752704. Throughput: 0: 10053.6, 1: 9692.2. Samples: 343742856. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 20:15:06,062][104569] Avg episode reward: [(0, '8196.740'), (1, '9167.469')] [2023-12-26 20:15:06,082][105692] Updated weights for policy 0, policy_version 670871 (0.0010) [2023-12-26 20:15:06,147][105692] Updated weights for policy 0, policy_version 670881 (0.0009) [2023-12-26 20:15:06,618][105620] Updated weights for policy 1, policy_version 671739 (0.0008) [2023-12-26 20:15:06,683][105620] Updated weights for policy 1, policy_version 671749 (0.0008) [2023-12-26 20:15:06,748][105620] Updated weights for policy 1, policy_version 671759 (0.0008) [2023-12-26 20:15:06,903][105692] Updated weights for policy 0, policy_version 670891 (0.0010) [2023-12-26 20:15:06,951][105692] Updated weights for policy 0, policy_version 670901 (0.0009) [2023-12-26 20:15:07,013][105692] Updated weights for policy 0, policy_version 670911 (0.0010) [2023-12-26 20:15:07,518][105620] Updated weights for policy 1, policy_version 671769 (0.0008) [2023-12-26 20:15:07,580][105620] Updated weights for policy 1, policy_version 671779 (0.0008) [2023-12-26 20:15:07,640][105620] Updated weights for policy 1, policy_version 671789 (0.0008) [2023-12-26 20:15:07,702][105620] Updated weights for policy 1, policy_version 671800 (0.0009) [2023-12-26 20:15:07,739][105692] Updated weights for policy 0, policy_version 670921 (0.0010) [2023-12-26 20:15:07,794][105692] Updated weights for policy 0, policy_version 670931 (0.0005) [2023-12-26 20:15:07,856][105692] Updated weights for policy 0, policy_version 670941 (0.0005) [2023-12-26 20:15:07,912][105692] Updated weights for policy 0, policy_version 670951 (0.0009) [2023-12-26 20:15:08,536][105620] Updated weights for policy 1, policy_version 671810 (0.0009) [2023-12-26 20:15:08,570][105692] Updated weights for policy 0, policy_version 670961 (0.0009) [2023-12-26 20:15:08,592][105620] Updated weights for policy 1, policy_version 671820 (0.0006) [2023-12-26 20:15:08,626][105692] Updated weights for policy 0, policy_version 670971 (0.0011) [2023-12-26 20:15:08,648][105620] Updated weights for policy 1, policy_version 671830 (0.0005) [2023-12-26 20:15:08,681][105692] Updated weights for policy 0, policy_version 670981 (0.0010) [2023-12-26 20:15:09,374][105692] Updated weights for policy 0, policy_version 670991 (0.0008) [2023-12-26 20:15:09,437][105620] Updated weights for policy 1, policy_version 671840 (0.0008) [2023-12-26 20:15:09,438][105692] Updated weights for policy 0, policy_version 671001 (0.0007) [2023-12-26 20:15:09,490][105692] Updated weights for policy 0, policy_version 671011 (0.0006) [2023-12-26 20:15:09,494][105620] Updated weights for policy 1, policy_version 671850 (0.0009) [2023-12-26 20:15:09,562][105620] Updated weights for policy 1, policy_version 671860 (0.0009) [2023-12-26 20:15:10,104][105692] Updated weights for policy 0, policy_version 671021 (0.0007) [2023-12-26 20:15:10,160][105692] Updated weights for policy 0, policy_version 671031 (0.0010) [2023-12-26 20:15:10,216][105692] Updated weights for policy 0, policy_version 671041 (0.0011) [2023-12-26 20:15:10,361][105620] Updated weights for policy 1, policy_version 671870 (0.0009) [2023-12-26 20:15:10,411][105620] Updated weights for policy 1, policy_version 671880 (0.0008) [2023-12-26 20:15:10,460][105620] Updated weights for policy 1, policy_version 671890 (0.0008) [2023-12-26 20:15:10,954][105692] Updated weights for policy 0, policy_version 671051 (0.0008) [2023-12-26 20:15:11,018][105692] Updated weights for policy 0, policy_version 671061 (0.0009) [2023-12-26 20:15:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 343842816. Throughput: 0: 10135.6, 1: 9630.1. Samples: 343855540. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 20:15:11,062][104569] Avg episode reward: [(0, '8268.956'), (1, '9075.820')] [2023-12-26 20:15:11,090][105692] Updated weights for policy 0, policy_version 671071 (0.0009) [2023-12-26 20:15:11,275][105620] Updated weights for policy 1, policy_version 671900 (0.0007) [2023-12-26 20:15:11,342][105620] Updated weights for policy 1, policy_version 671910 (0.0008) [2023-12-26 20:15:11,410][105620] Updated weights for policy 1, policy_version 671920 (0.0007) [2023-12-26 20:15:11,878][105692] Updated weights for policy 0, policy_version 671081 (0.0009) [2023-12-26 20:15:11,938][105692] Updated weights for policy 0, policy_version 671091 (0.0009) [2023-12-26 20:15:11,995][105692] Updated weights for policy 0, policy_version 671101 (0.0009) [2023-12-26 20:15:12,051][105692] Updated weights for policy 0, policy_version 671111 (0.0009) [2023-12-26 20:15:12,129][105620] Updated weights for policy 1, policy_version 671930 (0.0008) [2023-12-26 20:15:12,194][105620] Updated weights for policy 1, policy_version 671940 (0.0008) [2023-12-26 20:15:12,256][105620] Updated weights for policy 1, policy_version 671950 (0.0009) [2023-12-26 20:15:12,313][105620] Updated weights for policy 1, policy_version 671960 (0.0008) [2023-12-26 20:15:12,777][105692] Updated weights for policy 0, policy_version 671121 (0.0011) [2023-12-26 20:15:12,833][105692] Updated weights for policy 0, policy_version 671131 (0.0011) [2023-12-26 20:15:12,893][105692] Updated weights for policy 0, policy_version 671141 (0.0011) [2023-12-26 20:15:13,130][105620] Updated weights for policy 1, policy_version 671970 (0.0006) [2023-12-26 20:15:13,193][105620] Updated weights for policy 1, policy_version 671980 (0.0005) [2023-12-26 20:15:13,259][105620] Updated weights for policy 1, policy_version 671990 (0.0007) [2023-12-26 20:15:13,557][105692] Updated weights for policy 0, policy_version 671151 (0.0010) [2023-12-26 20:15:13,602][105692] Updated weights for policy 0, policy_version 671161 (0.0010) [2023-12-26 20:15:13,646][105692] Updated weights for policy 0, policy_version 671171 (0.0010) [2023-12-26 20:15:13,993][105620] Updated weights for policy 1, policy_version 672000 (0.0009) [2023-12-26 20:15:14,046][105620] Updated weights for policy 1, policy_version 672010 (0.0008) [2023-12-26 20:15:14,095][105620] Updated weights for policy 1, policy_version 672020 (0.0008) [2023-12-26 20:15:14,431][105692] Updated weights for policy 0, policy_version 671181 (0.0010) [2023-12-26 20:15:14,482][105692] Updated weights for policy 0, policy_version 671191 (0.0010) [2023-12-26 20:15:14,528][105692] Updated weights for policy 0, policy_version 671201 (0.0010) [2023-12-26 20:15:14,842][105620] Updated weights for policy 1, policy_version 672030 (0.0008) [2023-12-26 20:15:14,899][105620] Updated weights for policy 1, policy_version 672040 (0.0008) [2023-12-26 20:15:14,960][105620] Updated weights for policy 1, policy_version 672050 (0.0008) [2023-12-26 20:15:15,320][105692] Updated weights for policy 0, policy_version 671211 (0.0010) [2023-12-26 20:15:15,366][105692] Updated weights for policy 0, policy_version 671221 (0.0011) [2023-12-26 20:15:15,423][105692] Updated weights for policy 0, policy_version 671231 (0.0011) [2023-12-26 20:15:15,648][105620] Updated weights for policy 1, policy_version 672060 (0.0007) [2023-12-26 20:15:15,709][105620] Updated weights for policy 1, policy_version 672070 (0.0008) [2023-12-26 20:15:15,754][105620] Updated weights for policy 1, policy_version 672080 (0.0008) [2023-12-26 20:15:16,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19660.7, 300 sec: 19494.2). Total num frames: 343941120. Throughput: 0: 10051.0, 1: 9499.9. Samples: 343911828. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 20:15:16,063][104569] Avg episode reward: [(0, '8626.181'), (1, '9036.962')] [2023-12-26 20:15:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000671240_171868160.pth... [2023-12-26 20:15:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000672088_172072960.pth... [2023-12-26 20:15:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000670056_171565056.pth [2023-12-26 20:15:16,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000671000_171794432.pth [2023-12-26 20:15:16,187][105692] Updated weights for policy 0, policy_version 671241 (0.0010) [2023-12-26 20:15:16,241][105692] Updated weights for policy 0, policy_version 671251 (0.0008) [2023-12-26 20:15:16,292][105692] Updated weights for policy 0, policy_version 671261 (0.0010) [2023-12-26 20:15:16,343][105692] Updated weights for policy 0, policy_version 671271 (0.0010) [2023-12-26 20:15:16,506][105620] Updated weights for policy 1, policy_version 672090 (0.0008) [2023-12-26 20:15:16,561][105620] Updated weights for policy 1, policy_version 672100 (0.0009) [2023-12-26 20:15:16,608][105620] Updated weights for policy 1, policy_version 672110 (0.0009) [2023-12-26 20:15:16,658][105620] Updated weights for policy 1, policy_version 672120 (0.0009) [2023-12-26 20:15:16,980][105692] Updated weights for policy 0, policy_version 671281 (0.0008) [2023-12-26 20:15:17,028][105692] Updated weights for policy 0, policy_version 671291 (0.0009) [2023-12-26 20:15:17,082][105692] Updated weights for policy 0, policy_version 671301 (0.0009) [2023-12-26 20:15:17,477][105620] Updated weights for policy 1, policy_version 672130 (0.0008) [2023-12-26 20:15:17,521][105620] Updated weights for policy 1, policy_version 672140 (0.0007) [2023-12-26 20:15:17,566][105620] Updated weights for policy 1, policy_version 672150 (0.0008) [2023-12-26 20:15:17,804][105692] Updated weights for policy 0, policy_version 671311 (0.0007) [2023-12-26 20:15:17,853][105692] Updated weights for policy 0, policy_version 671321 (0.0005) [2023-12-26 20:15:17,902][105692] Updated weights for policy 0, policy_version 671331 (0.0005) [2023-12-26 20:15:18,154][105620] Updated weights for policy 1, policy_version 672160 (0.0006) [2023-12-26 20:15:18,203][105620] Updated weights for policy 1, policy_version 672170 (0.0005) [2023-12-26 20:15:18,264][105620] Updated weights for policy 1, policy_version 672180 (0.0006) [2023-12-26 20:15:18,609][105692] Updated weights for policy 0, policy_version 671341 (0.0008) [2023-12-26 20:15:18,675][105692] Updated weights for policy 0, policy_version 671351 (0.0010) [2023-12-26 20:15:18,740][105692] Updated weights for policy 0, policy_version 671361 (0.0011) [2023-12-26 20:15:19,021][105620] Updated weights for policy 1, policy_version 672190 (0.0009) [2023-12-26 20:15:19,086][105620] Updated weights for policy 1, policy_version 672200 (0.0008) [2023-12-26 20:15:19,146][105620] Updated weights for policy 1, policy_version 672210 (0.0005) [2023-12-26 20:15:19,292][105692] Updated weights for policy 0, policy_version 671371 (0.0007) [2023-12-26 20:15:19,363][105692] Updated weights for policy 0, policy_version 671381 (0.0008) [2023-12-26 20:15:19,411][105692] Updated weights for policy 0, policy_version 671391 (0.0007) [2023-12-26 20:15:19,848][105620] Updated weights for policy 1, policy_version 672220 (0.0008) [2023-12-26 20:15:19,908][105620] Updated weights for policy 1, policy_version 672230 (0.0011) [2023-12-26 20:15:19,967][105620] Updated weights for policy 1, policy_version 672240 (0.0008) [2023-12-26 20:15:20,105][105692] Updated weights for policy 0, policy_version 671401 (0.0011) [2023-12-26 20:15:20,165][105692] Updated weights for policy 0, policy_version 671411 (0.0011) [2023-12-26 20:15:20,233][105692] Updated weights for policy 0, policy_version 671421 (0.0006) [2023-12-26 20:15:20,293][105692] Updated weights for policy 0, policy_version 671431 (0.0006) [2023-12-26 20:15:20,667][105620] Updated weights for policy 1, policy_version 672250 (0.0006) [2023-12-26 20:15:20,733][105620] Updated weights for policy 1, policy_version 672260 (0.0007) [2023-12-26 20:15:20,806][105620] Updated weights for policy 1, policy_version 672270 (0.0007) [2023-12-26 20:15:20,876][105620] Updated weights for policy 1, policy_version 672280 (0.0007) [2023-12-26 20:15:20,886][105692] Updated weights for policy 0, policy_version 671441 (0.0009) [2023-12-26 20:15:20,948][105692] Updated weights for policy 0, policy_version 671452 (0.0009) [2023-12-26 20:15:20,996][105692] Updated weights for policy 0, policy_version 671462 (0.0009) [2023-12-26 20:15:21,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19521.9). Total num frames: 344047616. Throughput: 0: 10146.1, 1: 9398.3. Samples: 344029600. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 20:15:21,063][104569] Avg episode reward: [(0, '8898.833'), (1, '9036.603')] [2023-12-26 20:15:21,504][105620] Updated weights for policy 1, policy_version 672290 (0.0006) [2023-12-26 20:15:21,575][105620] Updated weights for policy 1, policy_version 672300 (0.0006) [2023-12-26 20:15:21,641][105620] Updated weights for policy 1, policy_version 672310 (0.0008) [2023-12-26 20:15:21,871][105692] Updated weights for policy 0, policy_version 671472 (0.0009) [2023-12-26 20:15:21,925][105692] Updated weights for policy 0, policy_version 671482 (0.0009) [2023-12-26 20:15:21,984][105692] Updated weights for policy 0, policy_version 671492 (0.0009) [2023-12-26 20:15:22,356][105620] Updated weights for policy 1, policy_version 672320 (0.0008) [2023-12-26 20:15:22,414][105620] Updated weights for policy 1, policy_version 672330 (0.0007) [2023-12-26 20:15:22,481][105620] Updated weights for policy 1, policy_version 672340 (0.0007) [2023-12-26 20:15:22,842][105692] Updated weights for policy 0, policy_version 671502 (0.0010) [2023-12-26 20:15:22,898][105692] Updated weights for policy 0, policy_version 671512 (0.0006) [2023-12-26 20:15:22,956][105692] Updated weights for policy 0, policy_version 671522 (0.0005) [2023-12-26 20:15:23,137][105620] Updated weights for policy 1, policy_version 672350 (0.0007) [2023-12-26 20:15:23,195][105620] Updated weights for policy 1, policy_version 672360 (0.0009) [2023-12-26 20:15:23,251][105620] Updated weights for policy 1, policy_version 672370 (0.0007) [2023-12-26 20:15:23,615][105692] Updated weights for policy 0, policy_version 671532 (0.0006) [2023-12-26 20:15:23,666][105692] Updated weights for policy 0, policy_version 671542 (0.0005) [2023-12-26 20:15:23,723][105692] Updated weights for policy 0, policy_version 671552 (0.0005) [2023-12-26 20:15:24,057][105620] Updated weights for policy 1, policy_version 672380 (0.0007) [2023-12-26 20:15:24,111][105620] Updated weights for policy 1, policy_version 672390 (0.0009) [2023-12-26 20:15:24,165][105620] Updated weights for policy 1, policy_version 672401 (0.0009) [2023-12-26 20:15:24,259][105692] Updated weights for policy 0, policy_version 671562 (0.0006) [2023-12-26 20:15:24,315][105692] Updated weights for policy 0, policy_version 671572 (0.0009) [2023-12-26 20:15:24,363][105692] Updated weights for policy 0, policy_version 671582 (0.0009) [2023-12-26 20:15:24,414][105692] Updated weights for policy 0, policy_version 671592 (0.0009) [2023-12-26 20:15:24,969][105620] Updated weights for policy 1, policy_version 672411 (0.0008) [2023-12-26 20:15:25,025][105620] Updated weights for policy 1, policy_version 672421 (0.0009) [2023-12-26 20:15:25,082][105620] Updated weights for policy 1, policy_version 672431 (0.0009) [2023-12-26 20:15:25,171][105692] Updated weights for policy 0, policy_version 671602 (0.0009) [2023-12-26 20:15:25,218][105692] Updated weights for policy 0, policy_version 671612 (0.0009) [2023-12-26 20:15:25,270][105692] Updated weights for policy 0, policy_version 671622 (0.0009) [2023-12-26 20:15:25,738][105620] Updated weights for policy 1, policy_version 672441 (0.0008) [2023-12-26 20:15:25,802][105620] Updated weights for policy 1, policy_version 672451 (0.0006) [2023-12-26 20:15:25,868][105620] Updated weights for policy 1, policy_version 672461 (0.0005) [2023-12-26 20:15:25,933][105620] Updated weights for policy 1, policy_version 672471 (0.0009) [2023-12-26 20:15:25,977][105692] Updated weights for policy 0, policy_version 671632 (0.0007) [2023-12-26 20:15:26,050][105692] Updated weights for policy 0, policy_version 671642 (0.0009) [2023-12-26 20:15:26,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.9, 300 sec: 19466.4). Total num frames: 344137728. Throughput: 0: 10102.0, 1: 9390.2. Samples: 344147008. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 20:15:26,062][104569] Avg episode reward: [(0, '9081.242'), (1, '8983.886')] [2023-12-26 20:15:26,112][105692] Updated weights for policy 0, policy_version 671652 (0.0006) [2023-12-26 20:15:26,650][105620] Updated weights for policy 1, policy_version 672481 (0.0010) [2023-12-26 20:15:26,708][105620] Updated weights for policy 1, policy_version 672491 (0.0008) [2023-12-26 20:15:26,725][105692] Updated weights for policy 0, policy_version 671662 (0.0008) [2023-12-26 20:15:26,764][105620] Updated weights for policy 1, policy_version 672501 (0.0007) [2023-12-26 20:15:26,779][105692] Updated weights for policy 0, policy_version 671672 (0.0008) [2023-12-26 20:15:26,836][105692] Updated weights for policy 0, policy_version 671682 (0.0007) [2023-12-26 20:15:27,374][105620] Updated weights for policy 1, policy_version 672511 (0.0005) [2023-12-26 20:15:27,423][105620] Updated weights for policy 1, policy_version 672521 (0.0005) [2023-12-26 20:15:27,433][105692] Updated weights for policy 0, policy_version 671692 (0.0005) [2023-12-26 20:15:27,470][105620] Updated weights for policy 1, policy_version 672531 (0.0007) [2023-12-26 20:15:27,484][105692] Updated weights for policy 0, policy_version 671702 (0.0005) [2023-12-26 20:15:27,532][105692] Updated weights for policy 0, policy_version 671712 (0.0005) [2023-12-26 20:15:28,151][105620] Updated weights for policy 1, policy_version 672542 (0.0008) [2023-12-26 20:15:28,201][105620] Updated weights for policy 1, policy_version 672552 (0.0009) [2023-12-26 20:15:28,247][105620] Updated weights for policy 1, policy_version 672562 (0.0007) [2023-12-26 20:15:28,266][105692] Updated weights for policy 0, policy_version 671722 (0.0009) [2023-12-26 20:15:28,329][105692] Updated weights for policy 0, policy_version 671732 (0.0009) [2023-12-26 20:15:28,391][105692] Updated weights for policy 0, policy_version 671742 (0.0007) [2023-12-26 20:15:28,448][105692] Updated weights for policy 0, policy_version 671752 (0.0009) [2023-12-26 20:15:28,982][105620] Updated weights for policy 1, policy_version 672572 (0.0005) [2023-12-26 20:15:29,037][105620] Updated weights for policy 1, policy_version 672582 (0.0005) [2023-12-26 20:15:29,082][105692] Updated weights for policy 0, policy_version 671762 (0.0008) [2023-12-26 20:15:29,094][105620] Updated weights for policy 1, policy_version 672592 (0.0005) [2023-12-26 20:15:29,134][105692] Updated weights for policy 0, policy_version 671772 (0.0007) [2023-12-26 20:15:29,190][105692] Updated weights for policy 0, policy_version 671782 (0.0008) [2023-12-26 20:15:29,728][105620] Updated weights for policy 1, policy_version 672602 (0.0006) [2023-12-26 20:15:29,787][105620] Updated weights for policy 1, policy_version 672612 (0.0005) [2023-12-26 20:15:29,845][105692] Updated weights for policy 0, policy_version 671792 (0.0009) [2023-12-26 20:15:29,850][105620] Updated weights for policy 1, policy_version 672622 (0.0007) [2023-12-26 20:15:29,901][105692] Updated weights for policy 0, policy_version 671802 (0.0008) [2023-12-26 20:15:29,903][105620] Updated weights for policy 1, policy_version 672632 (0.0005) [2023-12-26 20:15:29,956][105692] Updated weights for policy 0, policy_version 671812 (0.0007) [2023-12-26 20:15:30,629][105620] Updated weights for policy 1, policy_version 672642 (0.0008) [2023-12-26 20:15:30,644][105692] Updated weights for policy 0, policy_version 671822 (0.0006) [2023-12-26 20:15:30,690][105620] Updated weights for policy 1, policy_version 672652 (0.0008) [2023-12-26 20:15:30,694][105692] Updated weights for policy 0, policy_version 671832 (0.0006) [2023-12-26 20:15:30,742][105692] Updated weights for policy 0, policy_version 671842 (0.0006) [2023-12-26 20:15:30,748][105620] Updated weights for policy 1, policy_version 672662 (0.0008) [2023-12-26 20:15:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 344244224. Throughput: 0: 10110.9, 1: 9416.2. Samples: 344208456. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-26 20:15:31,062][104569] Avg episode reward: [(0, '9171.828'), (1, '8712.045')] [2023-12-26 20:15:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000671848_172023808.pth... [2023-12-26 20:15:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000672664_172220416.pth... [2023-12-26 20:15:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000670664_171720704.pth [2023-12-26 20:15:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000671544_171933696.pth [2023-12-26 20:15:31,435][105692] Updated weights for policy 0, policy_version 671852 (0.0006) [2023-12-26 20:15:31,490][105692] Updated weights for policy 0, policy_version 671862 (0.0005) [2023-12-26 20:15:31,536][105620] Updated weights for policy 1, policy_version 672672 (0.0008) [2023-12-26 20:15:31,547][105692] Updated weights for policy 0, policy_version 671872 (0.0007) [2023-12-26 20:15:31,598][105620] Updated weights for policy 1, policy_version 672682 (0.0009) [2023-12-26 20:15:31,656][105620] Updated weights for policy 1, policy_version 672692 (0.0007) [2023-12-26 20:15:32,255][105692] Updated weights for policy 0, policy_version 671882 (0.0007) [2023-12-26 20:15:32,308][105692] Updated weights for policy 0, policy_version 671892 (0.0008) [2023-12-26 20:15:32,353][105620] Updated weights for policy 1, policy_version 672702 (0.0009) [2023-12-26 20:15:32,366][105692] Updated weights for policy 0, policy_version 671902 (0.0008) [2023-12-26 20:15:32,415][105620] Updated weights for policy 1, policy_version 672712 (0.0006) [2023-12-26 20:15:32,423][105692] Updated weights for policy 0, policy_version 671912 (0.0008) [2023-12-26 20:15:32,480][105620] Updated weights for policy 1, policy_version 672722 (0.0006) [2023-12-26 20:15:33,147][105692] Updated weights for policy 0, policy_version 671922 (0.0009) [2023-12-26 20:15:33,148][105620] Updated weights for policy 1, policy_version 672732 (0.0005) [2023-12-26 20:15:33,198][105692] Updated weights for policy 0, policy_version 671932 (0.0008) [2023-12-26 20:15:33,206][105620] Updated weights for policy 1, policy_version 672742 (0.0005) [2023-12-26 20:15:33,251][105692] Updated weights for policy 0, policy_version 671942 (0.0006) [2023-12-26 20:15:33,272][105620] Updated weights for policy 1, policy_version 672752 (0.0005) [2023-12-26 20:15:33,939][105620] Updated weights for policy 1, policy_version 672762 (0.0007) [2023-12-26 20:15:33,982][105692] Updated weights for policy 0, policy_version 671952 (0.0007) [2023-12-26 20:15:33,985][105620] Updated weights for policy 1, policy_version 672772 (0.0007) [2023-12-26 20:15:34,036][105620] Updated weights for policy 1, policy_version 672782 (0.0006) [2023-12-26 20:15:34,037][105692] Updated weights for policy 0, policy_version 671962 (0.0008) [2023-12-26 20:15:34,089][105692] Updated weights for policy 0, policy_version 671972 (0.0006) [2023-12-26 20:15:34,091][105620] Updated weights for policy 1, policy_version 672792 (0.0007) [2023-12-26 20:15:34,835][105692] Updated weights for policy 0, policy_version 671982 (0.0009) [2023-12-26 20:15:34,861][105620] Updated weights for policy 1, policy_version 672802 (0.0007) [2023-12-26 20:15:34,884][105692] Updated weights for policy 0, policy_version 671992 (0.0006) [2023-12-26 20:15:34,925][105620] Updated weights for policy 1, policy_version 672812 (0.0009) [2023-12-26 20:15:34,935][105692] Updated weights for policy 0, policy_version 672002 (0.0006) [2023-12-26 20:15:34,974][105620] Updated weights for policy 1, policy_version 672822 (0.0006) [2023-12-26 20:15:35,606][105692] Updated weights for policy 0, policy_version 672012 (0.0006) [2023-12-26 20:15:35,650][105692] Updated weights for policy 0, policy_version 672022 (0.0005) [2023-12-26 20:15:35,707][105692] Updated weights for policy 0, policy_version 672032 (0.0006) [2023-12-26 20:15:35,776][105620] Updated weights for policy 1, policy_version 672832 (0.0007) [2023-12-26 20:15:35,848][105620] Updated weights for policy 1, policy_version 672842 (0.0008) [2023-12-26 20:15:35,905][105620] Updated weights for policy 1, policy_version 672853 (0.0010) [2023-12-26 20:15:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 344342528. Throughput: 0: 10015.8, 1: 9561.3. Samples: 344327908. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 20:15:36,063][104569] Avg episode reward: [(0, '8996.035'), (1, '8894.815')] [2023-12-26 20:15:36,347][105692] Updated weights for policy 0, policy_version 672042 (0.0010) [2023-12-26 20:15:36,403][105692] Updated weights for policy 0, policy_version 672052 (0.0009) [2023-12-26 20:15:36,459][105692] Updated weights for policy 0, policy_version 672062 (0.0008) [2023-12-26 20:15:36,525][105692] Updated weights for policy 0, policy_version 672072 (0.0008) [2023-12-26 20:15:36,728][105620] Updated weights for policy 1, policy_version 672864 (0.0009) [2023-12-26 20:15:36,788][105620] Updated weights for policy 1, policy_version 672874 (0.0009) [2023-12-26 20:15:36,843][105620] Updated weights for policy 1, policy_version 672884 (0.0009) [2023-12-26 20:15:37,230][105692] Updated weights for policy 0, policy_version 672082 (0.0008) [2023-12-26 20:15:37,286][105692] Updated weights for policy 0, policy_version 672092 (0.0008) [2023-12-26 20:15:37,341][105692] Updated weights for policy 0, policy_version 672102 (0.0008) [2023-12-26 20:15:37,601][105620] Updated weights for policy 1, policy_version 672894 (0.0010) [2023-12-26 20:15:37,666][105620] Updated weights for policy 1, policy_version 672904 (0.0011) [2023-12-26 20:15:37,732][105620] Updated weights for policy 1, policy_version 672914 (0.0009) [2023-12-26 20:15:38,153][105692] Updated weights for policy 0, policy_version 672112 (0.0009) [2023-12-26 20:15:38,204][105692] Updated weights for policy 0, policy_version 672122 (0.0009) [2023-12-26 20:15:38,258][105692] Updated weights for policy 0, policy_version 672132 (0.0009) [2023-12-26 20:15:38,392][105620] Updated weights for policy 1, policy_version 672924 (0.0010) [2023-12-26 20:15:38,447][105620] Updated weights for policy 1, policy_version 672934 (0.0010) [2023-12-26 20:15:38,507][105620] Updated weights for policy 1, policy_version 672944 (0.0010) [2023-12-26 20:15:39,094][105620] Updated weights for policy 1, policy_version 672954 (0.0009) [2023-12-26 20:15:39,098][105692] Updated weights for policy 0, policy_version 672142 (0.0010) [2023-12-26 20:15:39,159][105620] Updated weights for policy 1, policy_version 672964 (0.0005) [2023-12-26 20:15:39,160][105692] Updated weights for policy 0, policy_version 672152 (0.0009) [2023-12-26 20:15:39,213][105620] Updated weights for policy 1, policy_version 672974 (0.0007) [2023-12-26 20:15:39,219][105692] Updated weights for policy 0, policy_version 672162 (0.0008) [2023-12-26 20:15:39,286][105620] Updated weights for policy 1, policy_version 672984 (0.0010) [2023-12-26 20:15:40,007][105620] Updated weights for policy 1, policy_version 672994 (0.0008) [2023-12-26 20:15:40,034][105692] Updated weights for policy 0, policy_version 672172 (0.0009) [2023-12-26 20:15:40,063][105620] Updated weights for policy 1, policy_version 673004 (0.0008) [2023-12-26 20:15:40,088][105692] Updated weights for policy 0, policy_version 672182 (0.0008) [2023-12-26 20:15:40,121][105620] Updated weights for policy 1, policy_version 673014 (0.0008) [2023-12-26 20:15:40,152][105692] Updated weights for policy 0, policy_version 672192 (0.0007) [2023-12-26 20:15:40,784][105620] Updated weights for policy 1, policy_version 673024 (0.0008) [2023-12-26 20:15:40,848][105620] Updated weights for policy 1, policy_version 673034 (0.0007) [2023-12-26 20:15:40,907][105620] Updated weights for policy 1, policy_version 673044 (0.0009) [2023-12-26 20:15:40,908][105692] Updated weights for policy 0, policy_version 672202 (0.0007) [2023-12-26 20:15:40,966][105692] Updated weights for policy 0, policy_version 672212 (0.0009) [2023-12-26 20:15:41,019][105692] Updated weights for policy 0, policy_version 672222 (0.0009) [2023-12-26 20:15:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 344432640. Throughput: 0: 10000.9, 1: 9535.6. Samples: 344442264. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 20:15:41,062][104569] Avg episode reward: [(0, '8815.924'), (1, '9075.034')] [2023-12-26 20:15:41,080][105692] Updated weights for policy 0, policy_version 672232 (0.0008) [2023-12-26 20:15:41,590][105620] Updated weights for policy 1, policy_version 673054 (0.0008) [2023-12-26 20:15:41,655][105620] Updated weights for policy 1, policy_version 673064 (0.0008) [2023-12-26 20:15:41,722][105620] Updated weights for policy 1, policy_version 673074 (0.0009) [2023-12-26 20:15:41,931][105692] Updated weights for policy 0, policy_version 672242 (0.0009) [2023-12-26 20:15:41,992][105692] Updated weights for policy 0, policy_version 672252 (0.0009) [2023-12-26 20:15:42,051][105692] Updated weights for policy 0, policy_version 672262 (0.0009) [2023-12-26 20:15:42,513][105620] Updated weights for policy 1, policy_version 673084 (0.0009) [2023-12-26 20:15:42,572][105620] Updated weights for policy 1, policy_version 673094 (0.0009) [2023-12-26 20:15:42,583][105586] KL-divergence is very high: 166.3822 [2023-12-26 20:15:42,634][105620] Updated weights for policy 1, policy_version 673104 (0.0008) [2023-12-26 20:15:42,636][105586] KL-divergence is very high: 207.9925 [2023-12-26 20:15:42,760][105692] Updated weights for policy 0, policy_version 672272 (0.0008) [2023-12-26 20:15:42,824][105692] Updated weights for policy 0, policy_version 672282 (0.0009) [2023-12-26 20:15:42,885][105692] Updated weights for policy 0, policy_version 672292 (0.0008) [2023-12-26 20:15:43,432][105620] Updated weights for policy 1, policy_version 673114 (0.0009) [2023-12-26 20:15:43,490][105620] Updated weights for policy 1, policy_version 673124 (0.0008) [2023-12-26 20:15:43,544][105620] Updated weights for policy 1, policy_version 673134 (0.0006) [2023-12-26 20:15:43,611][105620] Updated weights for policy 1, policy_version 673144 (0.0006) [2023-12-26 20:15:43,618][105692] Updated weights for policy 0, policy_version 672302 (0.0007) [2023-12-26 20:15:43,679][105692] Updated weights for policy 0, policy_version 672312 (0.0009) [2023-12-26 20:15:43,742][105692] Updated weights for policy 0, policy_version 672322 (0.0009) [2023-12-26 20:15:44,374][105620] Updated weights for policy 1, policy_version 673154 (0.0010) [2023-12-26 20:15:44,427][105620] Updated weights for policy 1, policy_version 673164 (0.0010) [2023-12-26 20:15:44,479][105620] Updated weights for policy 1, policy_version 673174 (0.0008) [2023-12-26 20:15:44,481][105692] Updated weights for policy 0, policy_version 672332 (0.0008) [2023-12-26 20:15:44,542][105692] Updated weights for policy 0, policy_version 672342 (0.0009) [2023-12-26 20:15:44,603][105692] Updated weights for policy 0, policy_version 672352 (0.0009) [2023-12-26 20:15:45,308][105620] Updated weights for policy 1, policy_version 673184 (0.0008) [2023-12-26 20:15:45,363][105620] Updated weights for policy 1, policy_version 673194 (0.0008) [2023-12-26 20:15:45,364][105692] Updated weights for policy 0, policy_version 672362 (0.0009) [2023-12-26 20:15:45,425][105620] Updated weights for policy 1, policy_version 673204 (0.0008) [2023-12-26 20:15:45,427][105692] Updated weights for policy 0, policy_version 672372 (0.0006) [2023-12-26 20:15:45,487][105692] Updated weights for policy 0, policy_version 672382 (0.0008) [2023-12-26 20:15:45,545][105692] Updated weights for policy 0, policy_version 672392 (0.0008) [2023-12-26 20:15:46,062][104569] Fps is (10 sec: 18021.9, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 344522752. Throughput: 0: 9846.5, 1: 9496.9. Samples: 344497368. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 20:15:46,063][104569] Avg episode reward: [(0, '8997.939'), (1, '8713.747')] [2023-12-26 20:15:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000673208_172359680.pth... [2023-12-26 20:15:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000672392_172163072.pth... [2023-12-26 20:15:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000672088_172072960.pth [2023-12-26 20:15:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000671240_171868160.pth [2023-12-26 20:15:46,229][105620] Updated weights for policy 1, policy_version 673214 (0.0009) [2023-12-26 20:15:46,285][105620] Updated weights for policy 1, policy_version 673224 (0.0010) [2023-12-26 20:15:46,298][105692] Updated weights for policy 0, policy_version 672402 (0.0008) [2023-12-26 20:15:46,335][105620] Updated weights for policy 1, policy_version 673234 (0.0007) [2023-12-26 20:15:46,356][105692] Updated weights for policy 0, policy_version 672412 (0.0010) [2023-12-26 20:15:46,408][105692] Updated weights for policy 0, policy_version 672422 (0.0010) [2023-12-26 20:15:47,026][105620] Updated weights for policy 1, policy_version 673244 (0.0009) [2023-12-26 20:15:47,034][105692] Updated weights for policy 0, policy_version 672432 (0.0010) [2023-12-26 20:15:47,082][105692] Updated weights for policy 0, policy_version 672442 (0.0011) [2023-12-26 20:15:47,088][105620] Updated weights for policy 1, policy_version 673254 (0.0008) [2023-12-26 20:15:47,130][105692] Updated weights for policy 0, policy_version 672452 (0.0010) [2023-12-26 20:15:47,146][105620] Updated weights for policy 1, policy_version 673264 (0.0010) [2023-12-26 20:15:47,793][105620] Updated weights for policy 1, policy_version 673274 (0.0006) [2023-12-26 20:15:47,856][105620] Updated weights for policy 1, policy_version 673284 (0.0006) [2023-12-26 20:15:47,880][105692] Updated weights for policy 0, policy_version 672462 (0.0007) [2023-12-26 20:15:47,921][105620] Updated weights for policy 1, policy_version 673294 (0.0007) [2023-12-26 20:15:47,946][105692] Updated weights for policy 0, policy_version 672472 (0.0007) [2023-12-26 20:15:47,984][105620] Updated weights for policy 1, policy_version 673304 (0.0006) [2023-12-26 20:15:48,004][105692] Updated weights for policy 0, policy_version 672482 (0.0008) [2023-12-26 20:15:48,627][105620] Updated weights for policy 1, policy_version 673314 (0.0011) [2023-12-26 20:15:48,692][105620] Updated weights for policy 1, policy_version 673324 (0.0011) [2023-12-26 20:15:48,712][105692] Updated weights for policy 0, policy_version 672492 (0.0009) [2023-12-26 20:15:48,753][105620] Updated weights for policy 1, policy_version 673334 (0.0011) [2023-12-26 20:15:48,775][105692] Updated weights for policy 0, policy_version 672502 (0.0006) [2023-12-26 20:15:48,830][105692] Updated weights for policy 0, policy_version 672512 (0.0009) [2023-12-26 20:15:49,446][105620] Updated weights for policy 1, policy_version 673344 (0.0008) [2023-12-26 20:15:49,510][105620] Updated weights for policy 1, policy_version 673354 (0.0010) [2023-12-26 20:15:49,574][105620] Updated weights for policy 1, policy_version 673364 (0.0007) [2023-12-26 20:15:49,626][105692] Updated weights for policy 0, policy_version 672522 (0.0009) [2023-12-26 20:15:49,693][105692] Updated weights for policy 0, policy_version 672532 (0.0009) [2023-12-26 20:15:49,746][105692] Updated weights for policy 0, policy_version 672542 (0.0009) [2023-12-26 20:15:49,793][105692] Updated weights for policy 0, policy_version 672552 (0.0008) [2023-12-26 20:15:50,294][105620] Updated weights for policy 1, policy_version 673374 (0.0008) [2023-12-26 20:15:50,360][105620] Updated weights for policy 1, policy_version 673384 (0.0009) [2023-12-26 20:15:50,425][105620] Updated weights for policy 1, policy_version 673394 (0.0006) [2023-12-26 20:15:50,551][105692] Updated weights for policy 0, policy_version 672562 (0.0007) [2023-12-26 20:15:50,614][105692] Updated weights for policy 0, policy_version 672572 (0.0011) [2023-12-26 20:15:50,682][105692] Updated weights for policy 0, policy_version 672582 (0.0011) [2023-12-26 20:15:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 344621056. Throughput: 0: 9761.2, 1: 9554.1. Samples: 344612048. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 20:15:51,063][104569] Avg episode reward: [(0, '9178.729'), (1, '8274.740')] [2023-12-26 20:15:51,142][105620] Updated weights for policy 1, policy_version 673404 (0.0007) [2023-12-26 20:15:51,195][105586] KL-divergence is very high: 197.0258 [2023-12-26 20:15:51,199][105620] Updated weights for policy 1, policy_version 673414 (0.0008) [2023-12-26 20:15:51,244][105586] KL-divergence is very high: 227.3612 [2023-12-26 20:15:51,261][105620] Updated weights for policy 1, policy_version 673424 (0.0009) [2023-12-26 20:15:51,292][105586] KL-divergence is very high: 128.5970 [2023-12-26 20:15:51,436][105692] Updated weights for policy 0, policy_version 672592 (0.0009) [2023-12-26 20:15:51,500][105692] Updated weights for policy 0, policy_version 672602 (0.0008) [2023-12-26 20:15:51,567][105692] Updated weights for policy 0, policy_version 672612 (0.0008) [2023-12-26 20:15:52,059][105620] Updated weights for policy 1, policy_version 673434 (0.0008) [2023-12-26 20:15:52,121][105620] Updated weights for policy 1, policy_version 673444 (0.0010) [2023-12-26 20:15:52,190][105620] Updated weights for policy 1, policy_version 673454 (0.0011) [2023-12-26 20:15:52,259][105620] Updated weights for policy 1, policy_version 673464 (0.0011) [2023-12-26 20:15:52,280][105692] Updated weights for policy 0, policy_version 672622 (0.0008) [2023-12-26 20:15:52,339][105692] Updated weights for policy 0, policy_version 672632 (0.0008) [2023-12-26 20:15:52,406][105692] Updated weights for policy 0, policy_version 672642 (0.0008) [2023-12-26 20:15:52,917][105620] Updated weights for policy 1, policy_version 673474 (0.0006) [2023-12-26 20:15:52,969][105620] Updated weights for policy 1, policy_version 673484 (0.0005) [2023-12-26 20:15:53,029][105620] Updated weights for policy 1, policy_version 673494 (0.0005) [2023-12-26 20:15:53,125][105692] Updated weights for policy 0, policy_version 672652 (0.0009) [2023-12-26 20:15:53,196][105692] Updated weights for policy 0, policy_version 672662 (0.0005) [2023-12-26 20:15:53,257][105692] Updated weights for policy 0, policy_version 672672 (0.0005) [2023-12-26 20:15:53,651][105620] Updated weights for policy 1, policy_version 673504 (0.0008) [2023-12-26 20:15:53,707][105620] Updated weights for policy 1, policy_version 673514 (0.0008) [2023-12-26 20:15:53,759][105620] Updated weights for policy 1, policy_version 673524 (0.0005) [2023-12-26 20:15:53,949][105692] Updated weights for policy 0, policy_version 672682 (0.0005) [2023-12-26 20:15:53,997][105692] Updated weights for policy 0, policy_version 672692 (0.0005) [2023-12-26 20:15:54,042][105692] Updated weights for policy 0, policy_version 672702 (0.0005) [2023-12-26 20:15:54,096][105692] Updated weights for policy 0, policy_version 672712 (0.0005) [2023-12-26 20:15:54,332][105620] Updated weights for policy 1, policy_version 673534 (0.0009) [2023-12-26 20:15:54,385][105620] Updated weights for policy 1, policy_version 673544 (0.0009) [2023-12-26 20:15:54,436][105620] Updated weights for policy 1, policy_version 673554 (0.0005) [2023-12-26 20:15:54,828][105692] Updated weights for policy 0, policy_version 672722 (0.0009) [2023-12-26 20:15:54,883][105692] Updated weights for policy 0, policy_version 672734 (0.0011) [2023-12-26 20:15:54,935][105692] Updated weights for policy 0, policy_version 672744 (0.0009) [2023-12-26 20:15:55,088][105620] Updated weights for policy 1, policy_version 673564 (0.0005) [2023-12-26 20:15:55,151][105620] Updated weights for policy 1, policy_version 673574 (0.0007) [2023-12-26 20:15:55,222][105620] Updated weights for policy 1, policy_version 673584 (0.0010) [2023-12-26 20:15:55,724][105692] Updated weights for policy 0, policy_version 672754 (0.0008) [2023-12-26 20:15:55,777][105692] Updated weights for policy 0, policy_version 672764 (0.0009) [2023-12-26 20:15:55,826][105692] Updated weights for policy 0, policy_version 672774 (0.0008) [2023-12-26 20:15:55,913][105620] Updated weights for policy 1, policy_version 673594 (0.0010) [2023-12-26 20:15:55,978][105620] Updated weights for policy 1, policy_version 673604 (0.0011) [2023-12-26 20:15:56,029][105620] Updated weights for policy 1, policy_version 673614 (0.0010) [2023-12-26 20:15:56,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 344719360. Throughput: 0: 9707.7, 1: 9710.0. Samples: 344729336. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 20:15:56,062][104569] Avg episode reward: [(0, '9353.582'), (1, '7604.177')] [2023-12-26 20:15:56,081][105620] Updated weights for policy 1, policy_version 673624 (0.0010) [2023-12-26 20:15:56,609][105692] Updated weights for policy 0, policy_version 672784 (0.0009) [2023-12-26 20:15:56,670][105692] Updated weights for policy 0, policy_version 672794 (0.0005) [2023-12-26 20:15:56,735][105692] Updated weights for policy 0, policy_version 672804 (0.0005) [2023-12-26 20:15:56,860][105620] Updated weights for policy 1, policy_version 673634 (0.0010) [2023-12-26 20:15:56,918][105620] Updated weights for policy 1, policy_version 673644 (0.0010) [2023-12-26 20:15:56,968][105620] Updated weights for policy 1, policy_version 673654 (0.0010) [2023-12-26 20:15:57,399][105692] Updated weights for policy 0, policy_version 672814 (0.0007) [2023-12-26 20:15:57,449][105692] Updated weights for policy 0, policy_version 672824 (0.0008) [2023-12-26 20:15:57,496][105692] Updated weights for policy 0, policy_version 672834 (0.0008) [2023-12-26 20:15:57,708][105620] Updated weights for policy 1, policy_version 673664 (0.0010) [2023-12-26 20:15:57,766][105620] Updated weights for policy 1, policy_version 673674 (0.0011) [2023-12-26 20:15:57,814][105620] Updated weights for policy 1, policy_version 673684 (0.0010) [2023-12-26 20:15:58,271][105692] Updated weights for policy 0, policy_version 672844 (0.0008) [2023-12-26 20:15:58,336][105692] Updated weights for policy 0, policy_version 672854 (0.0008) [2023-12-26 20:15:58,402][105692] Updated weights for policy 0, policy_version 672864 (0.0009) [2023-12-26 20:15:58,616][105620] Updated weights for policy 1, policy_version 673694 (0.0010) [2023-12-26 20:15:58,679][105620] Updated weights for policy 1, policy_version 673704 (0.0008) [2023-12-26 20:15:58,742][105620] Updated weights for policy 1, policy_version 673714 (0.0009) [2023-12-26 20:15:59,213][105692] Updated weights for policy 0, policy_version 672874 (0.0009) [2023-12-26 20:15:59,272][105692] Updated weights for policy 0, policy_version 672884 (0.0009) [2023-12-26 20:15:59,331][105692] Updated weights for policy 0, policy_version 672894 (0.0008) [2023-12-26 20:15:59,400][105692] Updated weights for policy 0, policy_version 672904 (0.0009) [2023-12-26 20:15:59,450][105620] Updated weights for policy 1, policy_version 673724 (0.0007) [2023-12-26 20:15:59,507][105620] Updated weights for policy 1, policy_version 673734 (0.0006) [2023-12-26 20:15:59,555][105620] Updated weights for policy 1, policy_version 673744 (0.0005) [2023-12-26 20:16:00,165][105692] Updated weights for policy 0, policy_version 672914 (0.0010) [2023-12-26 20:16:00,221][105620] Updated weights for policy 1, policy_version 673754 (0.0007) [2023-12-26 20:16:00,226][105692] Updated weights for policy 0, policy_version 672924 (0.0010) [2023-12-26 20:16:00,271][105620] Updated weights for policy 1, policy_version 673764 (0.0006) [2023-12-26 20:16:00,286][105692] Updated weights for policy 0, policy_version 672934 (0.0009) [2023-12-26 20:16:00,328][105620] Updated weights for policy 1, policy_version 673774 (0.0007) [2023-12-26 20:16:00,378][105620] Updated weights for policy 1, policy_version 673784 (0.0008) [2023-12-26 20:16:01,016][105692] Updated weights for policy 0, policy_version 672944 (0.0006) [2023-12-26 20:16:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 344809472. Throughput: 0: 9683.9, 1: 9714.3. Samples: 344784740. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 20:16:01,062][104569] Avg episode reward: [(0, '9265.305'), (1, '6500.306')] [2023-12-26 20:16:01,084][105692] Updated weights for policy 0, policy_version 672954 (0.0008) [2023-12-26 20:16:01,123][105620] Updated weights for policy 1, policy_version 673794 (0.0008) [2023-12-26 20:16:01,144][105692] Updated weights for policy 0, policy_version 672964 (0.0008) [2023-12-26 20:16:01,169][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000672968_172310528.pth... [2023-12-26 20:16:01,174][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000671848_172023808.pth [2023-12-26 20:16:01,190][105620] Updated weights for policy 1, policy_version 673804 (0.0007) [2023-12-26 20:16:01,248][105620] Updated weights for policy 1, policy_version 673814 (0.0010) [2023-12-26 20:16:01,260][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000673816_172515328.pth... [2023-12-26 20:16:01,264][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000672664_172220416.pth [2023-12-26 20:16:01,756][105692] Updated weights for policy 0, policy_version 672974 (0.0007) [2023-12-26 20:16:01,807][105692] Updated weights for policy 0, policy_version 672984 (0.0008) [2023-12-26 20:16:01,854][105692] Updated weights for policy 0, policy_version 672994 (0.0008) [2023-12-26 20:16:02,054][105620] Updated weights for policy 1, policy_version 673824 (0.0009) [2023-12-26 20:16:02,116][105620] Updated weights for policy 1, policy_version 673834 (0.0009) [2023-12-26 20:16:02,178][105620] Updated weights for policy 1, policy_version 673844 (0.0009) [2023-12-26 20:16:02,511][105692] Updated weights for policy 0, policy_version 673004 (0.0007) [2023-12-26 20:16:02,569][105692] Updated weights for policy 0, policy_version 673014 (0.0006) [2023-12-26 20:16:02,627][105692] Updated weights for policy 0, policy_version 673024 (0.0009) [2023-12-26 20:16:02,934][105620] Updated weights for policy 1, policy_version 673854 (0.0008) [2023-12-26 20:16:02,986][105620] Updated weights for policy 1, policy_version 673864 (0.0008) [2023-12-26 20:16:03,030][105620] Updated weights for policy 1, policy_version 673874 (0.0008) [2023-12-26 20:16:03,337][105692] Updated weights for policy 0, policy_version 673034 (0.0011) [2023-12-26 20:16:03,388][105692] Updated weights for policy 0, policy_version 673044 (0.0010) [2023-12-26 20:16:03,439][105692] Updated weights for policy 0, policy_version 673054 (0.0010) [2023-12-26 20:16:03,487][105692] Updated weights for policy 0, policy_version 673064 (0.0010) [2023-12-26 20:16:03,799][105620] Updated weights for policy 1, policy_version 673884 (0.0008) [2023-12-26 20:16:03,857][105620] Updated weights for policy 1, policy_version 673894 (0.0008) [2023-12-26 20:16:03,910][105620] Updated weights for policy 1, policy_version 673904 (0.0008) [2023-12-26 20:16:04,264][105692] Updated weights for policy 0, policy_version 673074 (0.0011) [2023-12-26 20:16:04,321][105692] Updated weights for policy 0, policy_version 673084 (0.0011) [2023-12-26 20:16:04,377][105692] Updated weights for policy 0, policy_version 673094 (0.0010) [2023-12-26 20:16:04,695][105620] Updated weights for policy 1, policy_version 673914 (0.0008) [2023-12-26 20:16:04,754][105620] Updated weights for policy 1, policy_version 673924 (0.0008) [2023-12-26 20:16:04,810][105620] Updated weights for policy 1, policy_version 673934 (0.0008) [2023-12-26 20:16:04,868][105620] Updated weights for policy 1, policy_version 673944 (0.0008) [2023-12-26 20:16:05,109][105692] Updated weights for policy 0, policy_version 673104 (0.0010) [2023-12-26 20:16:05,174][105692] Updated weights for policy 0, policy_version 673114 (0.0010) [2023-12-26 20:16:05,202][105585] KL-divergence is very high: 127.2548 [2023-12-26 20:16:05,231][105692] Updated weights for policy 0, policy_version 673124 (0.0010) [2023-12-26 20:16:05,249][105585] KL-divergence is very high: 135.1798 [2023-12-26 20:16:05,440][105620] Updated weights for policy 1, policy_version 673954 (0.0005) [2023-12-26 20:16:05,504][105620] Updated weights for policy 1, policy_version 673964 (0.0005) [2023-12-26 20:16:05,557][105620] Updated weights for policy 1, policy_version 673974 (0.0005) [2023-12-26 20:16:05,962][105692] Updated weights for policy 0, policy_version 673134 (0.0010) [2023-12-26 20:16:06,013][105692] Updated weights for policy 0, policy_version 673144 (0.0010) [2023-12-26 20:16:06,057][105692] Updated weights for policy 0, policy_version 673154 (0.0010) [2023-12-26 20:16:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 344907776. Throughput: 0: 9645.0, 1: 9678.8. Samples: 344899172. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 20:16:06,063][104569] Avg episode reward: [(0, '9265.343'), (1, '7174.437')] [2023-12-26 20:16:06,191][105620] Updated weights for policy 1, policy_version 673984 (0.0008) [2023-12-26 20:16:06,253][105620] Updated weights for policy 1, policy_version 673994 (0.0011) [2023-12-26 20:16:06,312][105620] Updated weights for policy 1, policy_version 674004 (0.0010) [2023-12-26 20:16:06,823][105692] Updated weights for policy 0, policy_version 673164 (0.0010) [2023-12-26 20:16:06,881][105692] Updated weights for policy 0, policy_version 673174 (0.0010) [2023-12-26 20:16:06,915][105620] Updated weights for policy 1, policy_version 674014 (0.0011) [2023-12-26 20:16:06,940][105692] Updated weights for policy 0, policy_version 673184 (0.0010) [2023-12-26 20:16:06,973][105620] Updated weights for policy 1, policy_version 674024 (0.0011) [2023-12-26 20:16:07,030][105620] Updated weights for policy 1, policy_version 674034 (0.0011) [2023-12-26 20:16:07,595][105692] Updated weights for policy 0, policy_version 673194 (0.0010) [2023-12-26 20:16:07,655][105692] Updated weights for policy 0, policy_version 673204 (0.0005) [2023-12-26 20:16:07,720][105692] Updated weights for policy 0, policy_version 673214 (0.0006) [2023-12-26 20:16:07,739][105620] Updated weights for policy 1, policy_version 674044 (0.0008) [2023-12-26 20:16:07,779][105692] Updated weights for policy 0, policy_version 673224 (0.0010) [2023-12-26 20:16:07,798][105620] Updated weights for policy 1, policy_version 674054 (0.0005) [2023-12-26 20:16:07,854][105620] Updated weights for policy 1, policy_version 674064 (0.0005) [2023-12-26 20:16:08,474][105692] Updated weights for policy 0, policy_version 673234 (0.0010) [2023-12-26 20:16:08,487][105620] Updated weights for policy 1, policy_version 674074 (0.0008) [2023-12-26 20:16:08,527][105692] Updated weights for policy 0, policy_version 673244 (0.0011) [2023-12-26 20:16:08,542][105620] Updated weights for policy 1, policy_version 674084 (0.0011) [2023-12-26 20:16:08,583][105692] Updated weights for policy 0, policy_version 673254 (0.0010) [2023-12-26 20:16:08,602][105620] Updated weights for policy 1, policy_version 674094 (0.0011) [2023-12-26 20:16:08,668][105620] Updated weights for policy 1, policy_version 674104 (0.0011) [2023-12-26 20:16:09,306][105692] Updated weights for policy 0, policy_version 673264 (0.0011) [2023-12-26 20:16:09,373][105692] Updated weights for policy 0, policy_version 673274 (0.0011) [2023-12-26 20:16:09,445][105692] Updated weights for policy 0, policy_version 673284 (0.0011) [2023-12-26 20:16:09,485][105620] Updated weights for policy 1, policy_version 674114 (0.0007) [2023-12-26 20:16:09,535][105620] Updated weights for policy 1, policy_version 674124 (0.0006) [2023-12-26 20:16:09,589][105620] Updated weights for policy 1, policy_version 674134 (0.0005) [2023-12-26 20:16:10,080][105692] Updated weights for policy 0, policy_version 673294 (0.0007) [2023-12-26 20:16:10,139][105692] Updated weights for policy 0, policy_version 673304 (0.0009) [2023-12-26 20:16:10,189][105692] Updated weights for policy 0, policy_version 673314 (0.0006) [2023-12-26 20:16:10,337][105620] Updated weights for policy 1, policy_version 674144 (0.0008) [2023-12-26 20:16:10,395][105620] Updated weights for policy 1, policy_version 674154 (0.0010) [2023-12-26 20:16:10,449][105620] Updated weights for policy 1, policy_version 674164 (0.0010) [2023-12-26 20:16:10,803][105692] Updated weights for policy 0, policy_version 673324 (0.0006) [2023-12-26 20:16:10,855][105692] Updated weights for policy 0, policy_version 673334 (0.0005) [2023-12-26 20:16:10,911][105692] Updated weights for policy 0, policy_version 673344 (0.0005) [2023-12-26 20:16:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 345014272. Throughput: 0: 9670.3, 1: 9717.1. Samples: 345019440. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 20:16:11,063][104569] Avg episode reward: [(0, '8994.606'), (1, '8118.804')] [2023-12-26 20:16:11,304][105620] Updated weights for policy 1, policy_version 674174 (0.0009) [2023-12-26 20:16:11,369][105620] Updated weights for policy 1, policy_version 674184 (0.0007) [2023-12-26 20:16:11,431][105620] Updated weights for policy 1, policy_version 674194 (0.0007) [2023-12-26 20:16:11,562][105692] Updated weights for policy 0, policy_version 673354 (0.0005) [2023-12-26 20:16:11,652][105692] Updated weights for policy 0, policy_version 673364 (0.0007) [2023-12-26 20:16:11,725][105692] Updated weights for policy 0, policy_version 673374 (0.0008) [2023-12-26 20:16:11,790][105692] Updated weights for policy 0, policy_version 673384 (0.0010) [2023-12-26 20:16:12,111][105620] Updated weights for policy 1, policy_version 674204 (0.0008) [2023-12-26 20:16:12,175][105620] Updated weights for policy 1, policy_version 674214 (0.0008) [2023-12-26 20:16:12,244][105620] Updated weights for policy 1, policy_version 674224 (0.0008) [2023-12-26 20:16:12,495][105692] Updated weights for policy 0, policy_version 673394 (0.0010) [2023-12-26 20:16:12,554][105692] Updated weights for policy 0, policy_version 673404 (0.0010) [2023-12-26 20:16:12,602][105692] Updated weights for policy 0, policy_version 673414 (0.0010) [2023-12-26 20:16:12,966][105620] Updated weights for policy 1, policy_version 674234 (0.0007) [2023-12-26 20:16:13,017][105620] Updated weights for policy 1, policy_version 674244 (0.0006) [2023-12-26 20:16:13,068][105620] Updated weights for policy 1, policy_version 674254 (0.0008) [2023-12-26 20:16:13,138][105620] Updated weights for policy 1, policy_version 674264 (0.0008) [2023-12-26 20:16:13,322][105692] Updated weights for policy 0, policy_version 673424 (0.0010) [2023-12-26 20:16:13,371][105692] Updated weights for policy 0, policy_version 673434 (0.0010) [2023-12-26 20:16:13,423][105692] Updated weights for policy 0, policy_version 673444 (0.0010) [2023-12-26 20:16:13,793][105620] Updated weights for policy 1, policy_version 674274 (0.0005) [2023-12-26 20:16:13,839][105620] Updated weights for policy 1, policy_version 674284 (0.0005) [2023-12-26 20:16:13,887][105620] Updated weights for policy 1, policy_version 674294 (0.0005) [2023-12-26 20:16:14,196][105692] Updated weights for policy 0, policy_version 673454 (0.0010) [2023-12-26 20:16:14,266][105692] Updated weights for policy 0, policy_version 673464 (0.0011) [2023-12-26 20:16:14,318][105692] Updated weights for policy 0, policy_version 673474 (0.0011) [2023-12-26 20:16:14,456][105620] Updated weights for policy 1, policy_version 674304 (0.0008) [2023-12-26 20:16:14,513][105620] Updated weights for policy 1, policy_version 674314 (0.0008) [2023-12-26 20:16:14,578][105620] Updated weights for policy 1, policy_version 674324 (0.0008) [2023-12-26 20:16:15,035][105692] Updated weights for policy 0, policy_version 673484 (0.0010) [2023-12-26 20:16:15,087][105692] Updated weights for policy 0, policy_version 673494 (0.0010) [2023-12-26 20:16:15,140][105692] Updated weights for policy 0, policy_version 673504 (0.0010) [2023-12-26 20:16:15,384][105620] Updated weights for policy 1, policy_version 674334 (0.0009) [2023-12-26 20:16:15,439][105620] Updated weights for policy 1, policy_version 674344 (0.0008) [2023-12-26 20:16:15,488][105620] Updated weights for policy 1, policy_version 674354 (0.0008) [2023-12-26 20:16:15,905][105692] Updated weights for policy 0, policy_version 673514 (0.0010) [2023-12-26 20:16:15,964][105692] Updated weights for policy 0, policy_version 673524 (0.0010) [2023-12-26 20:16:16,029][105692] Updated weights for policy 0, policy_version 673534 (0.0010) [2023-12-26 20:16:16,062][104569] Fps is (10 sec: 19659.8, 60 sec: 19387.6, 300 sec: 19494.1). Total num frames: 345104384. Throughput: 0: 9636.9, 1: 9698.6. Samples: 345078564. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 20:16:16,064][104569] Avg episode reward: [(0, '8991.559'), (1, '8632.518')] [2023-12-26 20:16:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000674360_172654592.pth... [2023-12-26 20:16:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000673208_172359680.pth [2023-12-26 20:16:16,088][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000673544_172457984.pth... [2023-12-26 20:16:16,089][105692] Updated weights for policy 0, policy_version 673544 (0.0010) [2023-12-26 20:16:16,091][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000672392_172163072.pth [2023-12-26 20:16:16,251][105620] Updated weights for policy 1, policy_version 674364 (0.0008) [2023-12-26 20:16:16,295][105620] Updated weights for policy 1, policy_version 674374 (0.0007) [2023-12-26 20:16:16,347][105620] Updated weights for policy 1, policy_version 674384 (0.0008) [2023-12-26 20:16:16,837][105692] Updated weights for policy 0, policy_version 673554 (0.0008) [2023-12-26 20:16:16,901][105692] Updated weights for policy 0, policy_version 673564 (0.0005) [2023-12-26 20:16:16,958][105692] Updated weights for policy 0, policy_version 673574 (0.0005) [2023-12-26 20:16:17,038][105620] Updated weights for policy 1, policy_version 674394 (0.0007) [2023-12-26 20:16:17,088][105620] Updated weights for policy 1, policy_version 674404 (0.0006) [2023-12-26 20:16:17,134][105620] Updated weights for policy 1, policy_version 674414 (0.0005) [2023-12-26 20:16:17,186][105620] Updated weights for policy 1, policy_version 674424 (0.0006) [2023-12-26 20:16:17,513][105692] Updated weights for policy 0, policy_version 673584 (0.0005) [2023-12-26 20:16:17,564][105692] Updated weights for policy 0, policy_version 673594 (0.0005) [2023-12-26 20:16:17,612][105692] Updated weights for policy 0, policy_version 673604 (0.0005) [2023-12-26 20:16:18,012][105620] Updated weights for policy 1, policy_version 674434 (0.0010) [2023-12-26 20:16:18,074][105620] Updated weights for policy 1, policy_version 674444 (0.0010) [2023-12-26 20:16:18,130][105620] Updated weights for policy 1, policy_version 674454 (0.0009) [2023-12-26 20:16:18,135][105692] Updated weights for policy 0, policy_version 673614 (0.0005) [2023-12-26 20:16:18,186][105692] Updated weights for policy 0, policy_version 673624 (0.0005) [2023-12-26 20:16:18,229][105692] Updated weights for policy 0, policy_version 673634 (0.0005) [2023-12-26 20:16:18,793][105692] Updated weights for policy 0, policy_version 673644 (0.0005) [2023-12-26 20:16:18,808][105620] Updated weights for policy 1, policy_version 674464 (0.0006) [2023-12-26 20:16:18,853][105692] Updated weights for policy 0, policy_version 673654 (0.0008) [2023-12-26 20:16:18,858][105620] Updated weights for policy 1, policy_version 674474 (0.0005) [2023-12-26 20:16:18,913][105620] Updated weights for policy 1, policy_version 674484 (0.0005) [2023-12-26 20:16:18,918][105692] Updated weights for policy 0, policy_version 673664 (0.0005) [2023-12-26 20:16:19,542][105692] Updated weights for policy 0, policy_version 673674 (0.0007) [2023-12-26 20:16:19,551][105620] Updated weights for policy 1, policy_version 674494 (0.0007) [2023-12-26 20:16:19,602][105692] Updated weights for policy 0, policy_version 673684 (0.0011) [2023-12-26 20:16:19,608][105620] Updated weights for policy 1, policy_version 674504 (0.0005) [2023-12-26 20:16:19,662][105692] Updated weights for policy 0, policy_version 673694 (0.0011) [2023-12-26 20:16:19,672][105620] Updated weights for policy 1, policy_version 674514 (0.0005) [2023-12-26 20:16:19,719][105692] Updated weights for policy 0, policy_version 673704 (0.0011) [2023-12-26 20:16:20,423][105620] Updated weights for policy 1, policy_version 674524 (0.0007) [2023-12-26 20:16:20,479][105620] Updated weights for policy 1, policy_version 674534 (0.0007) [2023-12-26 20:16:20,488][105692] Updated weights for policy 0, policy_version 673714 (0.0011) [2023-12-26 20:16:20,530][105620] Updated weights for policy 1, policy_version 674544 (0.0007) [2023-12-26 20:16:20,547][105692] Updated weights for policy 0, policy_version 673724 (0.0011) [2023-12-26 20:16:20,630][105692] Updated weights for policy 0, policy_version 673734 (0.0010) [2023-12-26 20:16:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 345210880. Throughput: 0: 9694.4, 1: 9700.9. Samples: 345200696. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 20:16:21,062][104569] Avg episode reward: [(0, '9084.304'), (1, '8806.445')] [2023-12-26 20:16:21,347][105620] Updated weights for policy 1, policy_version 674554 (0.0007) [2023-12-26 20:16:21,397][105692] Updated weights for policy 0, policy_version 673744 (0.0010) [2023-12-26 20:16:21,416][105620] Updated weights for policy 1, policy_version 674564 (0.0006) [2023-12-26 20:16:21,454][105692] Updated weights for policy 0, policy_version 673754 (0.0011) [2023-12-26 20:16:21,476][105620] Updated weights for policy 1, policy_version 674574 (0.0006) [2023-12-26 20:16:21,514][105692] Updated weights for policy 0, policy_version 673764 (0.0011) [2023-12-26 20:16:21,543][105620] Updated weights for policy 1, policy_version 674584 (0.0006) [2023-12-26 20:16:22,264][105692] Updated weights for policy 0, policy_version 673774 (0.0010) [2023-12-26 20:16:22,290][105620] Updated weights for policy 1, policy_version 674594 (0.0007) [2023-12-26 20:16:22,325][105692] Updated weights for policy 0, policy_version 673784 (0.0007) [2023-12-26 20:16:22,351][105620] Updated weights for policy 1, policy_version 674604 (0.0008) [2023-12-26 20:16:22,387][105692] Updated weights for policy 0, policy_version 673794 (0.0007) [2023-12-26 20:16:22,408][105620] Updated weights for policy 1, policy_version 674614 (0.0008) [2023-12-26 20:16:23,112][105692] Updated weights for policy 0, policy_version 673804 (0.0008) [2023-12-26 20:16:23,161][105692] Updated weights for policy 0, policy_version 673814 (0.0009) [2023-12-26 20:16:23,200][105620] Updated weights for policy 1, policy_version 674624 (0.0006) [2023-12-26 20:16:23,214][105692] Updated weights for policy 0, policy_version 673824 (0.0007) [2023-12-26 20:16:23,259][105620] Updated weights for policy 1, policy_version 674634 (0.0006) [2023-12-26 20:16:23,313][105620] Updated weights for policy 1, policy_version 674644 (0.0009) [2023-12-26 20:16:23,955][105620] Updated weights for policy 1, policy_version 674654 (0.0008) [2023-12-26 20:16:23,968][105692] Updated weights for policy 0, policy_version 673834 (0.0009) [2023-12-26 20:16:24,002][105620] Updated weights for policy 1, policy_version 674664 (0.0006) [2023-12-26 20:16:24,028][105692] Updated weights for policy 0, policy_version 673844 (0.0011) [2023-12-26 20:16:24,054][105620] Updated weights for policy 1, policy_version 674674 (0.0007) [2023-12-26 20:16:24,084][105692] Updated weights for policy 0, policy_version 673854 (0.0010) [2023-12-26 20:16:24,140][105692] Updated weights for policy 0, policy_version 673864 (0.0011) [2023-12-26 20:16:24,852][105620] Updated weights for policy 1, policy_version 674684 (0.0006) [2023-12-26 20:16:24,885][105692] Updated weights for policy 0, policy_version 673874 (0.0011) [2023-12-26 20:16:24,902][105620] Updated weights for policy 1, policy_version 674694 (0.0005) [2023-12-26 20:16:24,933][105692] Updated weights for policy 0, policy_version 673884 (0.0010) [2023-12-26 20:16:24,945][105620] Updated weights for policy 1, policy_version 674704 (0.0005) [2023-12-26 20:16:24,988][105692] Updated weights for policy 0, policy_version 673894 (0.0010) [2023-12-26 20:16:25,644][105620] Updated weights for policy 1, policy_version 674714 (0.0005) [2023-12-26 20:16:25,701][105620] Updated weights for policy 1, policy_version 674724 (0.0007) [2023-12-26 20:16:25,764][105620] Updated weights for policy 1, policy_version 674734 (0.0005) [2023-12-26 20:16:25,766][105692] Updated weights for policy 0, policy_version 673904 (0.0006) [2023-12-26 20:16:25,826][105620] Updated weights for policy 1, policy_version 674744 (0.0005) [2023-12-26 20:16:25,832][105692] Updated weights for policy 0, policy_version 673914 (0.0005) [2023-12-26 20:16:25,892][105692] Updated weights for policy 0, policy_version 673924 (0.0009) [2023-12-26 20:16:26,062][104569] Fps is (10 sec: 20480.6, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 345309184. Throughput: 0: 9681.2, 1: 9680.6. Samples: 345313548. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 20:16:26,063][104569] Avg episode reward: [(0, '9084.905'), (1, '8630.597')] [2023-12-26 20:16:26,464][105692] Updated weights for policy 0, policy_version 673934 (0.0010) [2023-12-26 20:16:26,520][105692] Updated weights for policy 0, policy_version 673944 (0.0006) [2023-12-26 20:16:26,525][105620] Updated weights for policy 1, policy_version 674754 (0.0008) [2023-12-26 20:16:26,572][105692] Updated weights for policy 0, policy_version 673954 (0.0005) [2023-12-26 20:16:26,585][105620] Updated weights for policy 1, policy_version 674764 (0.0008) [2023-12-26 20:16:26,651][105620] Updated weights for policy 1, policy_version 674774 (0.0007) [2023-12-26 20:16:27,219][105692] Updated weights for policy 0, policy_version 673964 (0.0008) [2023-12-26 20:16:27,266][105692] Updated weights for policy 0, policy_version 673974 (0.0005) [2023-12-26 20:16:27,319][105692] Updated weights for policy 0, policy_version 673984 (0.0009) [2023-12-26 20:16:27,438][105620] Updated weights for policy 1, policy_version 674784 (0.0006) [2023-12-26 20:16:27,482][105620] Updated weights for policy 1, policy_version 674794 (0.0007) [2023-12-26 20:16:27,544][105620] Updated weights for policy 1, policy_version 674804 (0.0009) [2023-12-26 20:16:27,959][105692] Updated weights for policy 0, policy_version 673994 (0.0010) [2023-12-26 20:16:28,005][105692] Updated weights for policy 0, policy_version 674004 (0.0008) [2023-12-26 20:16:28,051][105692] Updated weights for policy 0, policy_version 674014 (0.0008) [2023-12-26 20:16:28,099][105692] Updated weights for policy 0, policy_version 674024 (0.0009) [2023-12-26 20:16:28,329][105620] Updated weights for policy 1, policy_version 674814 (0.0009) [2023-12-26 20:16:28,390][105620] Updated weights for policy 1, policy_version 674824 (0.0009) [2023-12-26 20:16:28,441][105620] Updated weights for policy 1, policy_version 674834 (0.0009) [2023-12-26 20:16:28,845][105692] Updated weights for policy 0, policy_version 674034 (0.0008) [2023-12-26 20:16:28,893][105692] Updated weights for policy 0, policy_version 674044 (0.0009) [2023-12-26 20:16:28,939][105692] Updated weights for policy 0, policy_version 674054 (0.0008) [2023-12-26 20:16:29,225][105620] Updated weights for policy 1, policy_version 674844 (0.0008) [2023-12-26 20:16:29,288][105620] Updated weights for policy 1, policy_version 674854 (0.0009) [2023-12-26 20:16:29,354][105620] Updated weights for policy 1, policy_version 674864 (0.0009) [2023-12-26 20:16:29,714][105692] Updated weights for policy 0, policy_version 674064 (0.0009) [2023-12-26 20:16:29,776][105692] Updated weights for policy 0, policy_version 674074 (0.0009) [2023-12-26 20:16:29,843][105692] Updated weights for policy 0, policy_version 674084 (0.0008) [2023-12-26 20:16:30,112][105620] Updated weights for policy 1, policy_version 674874 (0.0009) [2023-12-26 20:16:30,177][105620] Updated weights for policy 1, policy_version 674884 (0.0009) [2023-12-26 20:16:30,226][105586] KL-divergence is very high: 121.4068 [2023-12-26 20:16:30,238][105620] Updated weights for policy 1, policy_version 674894 (0.0008) [2023-12-26 20:16:30,275][105586] KL-divergence is very high: 111.3406 [2023-12-26 20:16:30,299][105620] Updated weights for policy 1, policy_version 674904 (0.0009) [2023-12-26 20:16:30,572][105692] Updated weights for policy 0, policy_version 674094 (0.0009) [2023-12-26 20:16:30,626][105692] Updated weights for policy 0, policy_version 674104 (0.0008) [2023-12-26 20:16:30,687][105692] Updated weights for policy 0, policy_version 674114 (0.0008) [2023-12-26 20:16:31,046][105620] Updated weights for policy 1, policy_version 674914 (0.0009) [2023-12-26 20:16:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 345399296. Throughput: 0: 9773.4, 1: 9667.0. Samples: 345372180. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 20:16:31,062][104569] Avg episode reward: [(0, '8812.652'), (1, '8262.534')] [2023-12-26 20:16:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000674120_172605440.pth... [2023-12-26 20:16:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000672968_172310528.pth [2023-12-26 20:16:31,106][105620] Updated weights for policy 1, policy_version 674924 (0.0009) [2023-12-26 20:16:31,170][105620] Updated weights for policy 1, policy_version 674934 (0.0009) [2023-12-26 20:16:31,177][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000674936_172802048.pth... [2023-12-26 20:16:31,180][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000673816_172515328.pth [2023-12-26 20:16:31,453][105692] Updated weights for policy 0, policy_version 674124 (0.0008) [2023-12-26 20:16:31,506][105692] Updated weights for policy 0, policy_version 674134 (0.0005) [2023-12-26 20:16:31,569][105692] Updated weights for policy 0, policy_version 674144 (0.0008) [2023-12-26 20:16:31,921][105620] Updated weights for policy 1, policy_version 674944 (0.0009) [2023-12-26 20:16:31,989][105620] Updated weights for policy 1, policy_version 674954 (0.0009) [2023-12-26 20:16:32,052][105620] Updated weights for policy 1, policy_version 674964 (0.0009) [2023-12-26 20:16:32,292][105692] Updated weights for policy 0, policy_version 674154 (0.0009) [2023-12-26 20:16:32,353][105692] Updated weights for policy 0, policy_version 674164 (0.0009) [2023-12-26 20:16:32,409][105692] Updated weights for policy 0, policy_version 674174 (0.0009) [2023-12-26 20:16:32,465][105692] Updated weights for policy 0, policy_version 674184 (0.0008) [2023-12-26 20:16:32,814][105620] Updated weights for policy 1, policy_version 674974 (0.0009) [2023-12-26 20:16:32,868][105620] Updated weights for policy 1, policy_version 674984 (0.0009) [2023-12-26 20:16:32,919][105620] Updated weights for policy 1, policy_version 674994 (0.0009) [2023-12-26 20:16:33,208][105692] Updated weights for policy 0, policy_version 674194 (0.0009) [2023-12-26 20:16:33,262][105692] Updated weights for policy 0, policy_version 674204 (0.0009) [2023-12-26 20:16:33,319][105692] Updated weights for policy 0, policy_version 674214 (0.0008) [2023-12-26 20:16:33,702][105620] Updated weights for policy 1, policy_version 675004 (0.0010) [2023-12-26 20:16:33,757][105620] Updated weights for policy 1, policy_version 675014 (0.0010) [2023-12-26 20:16:33,805][105620] Updated weights for policy 1, policy_version 675024 (0.0008) [2023-12-26 20:16:34,065][105692] Updated weights for policy 0, policy_version 674224 (0.0009) [2023-12-26 20:16:34,114][105692] Updated weights for policy 0, policy_version 674234 (0.0008) [2023-12-26 20:16:34,171][105692] Updated weights for policy 0, policy_version 674244 (0.0008) [2023-12-26 20:16:34,414][105620] Updated weights for policy 1, policy_version 675034 (0.0010) [2023-12-26 20:16:34,481][105620] Updated weights for policy 1, policy_version 675044 (0.0011) [2023-12-26 20:16:34,540][105620] Updated weights for policy 1, policy_version 675054 (0.0010) [2023-12-26 20:16:34,603][105620] Updated weights for policy 1, policy_version 675064 (0.0009) [2023-12-26 20:16:34,977][105692] Updated weights for policy 0, policy_version 674254 (0.0008) [2023-12-26 20:16:35,030][105692] Updated weights for policy 0, policy_version 674264 (0.0008) [2023-12-26 20:16:35,081][105692] Updated weights for policy 0, policy_version 674274 (0.0008) [2023-12-26 20:16:35,328][105620] Updated weights for policy 1, policy_version 675074 (0.0010) [2023-12-26 20:16:35,394][105620] Updated weights for policy 1, policy_version 675084 (0.0010) [2023-12-26 20:16:35,445][105620] Updated weights for policy 1, policy_version 675094 (0.0010) [2023-12-26 20:16:35,846][105692] Updated weights for policy 0, policy_version 674284 (0.0008) [2023-12-26 20:16:35,900][105692] Updated weights for policy 0, policy_version 674294 (0.0008) [2023-12-26 20:16:35,955][105692] Updated weights for policy 0, policy_version 674304 (0.0007) [2023-12-26 20:16:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 345497600. Throughput: 0: 9761.2, 1: 9645.4. Samples: 345485348. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 20:16:36,063][104569] Avg episode reward: [(0, '8716.666'), (1, '8535.593')] [2023-12-26 20:16:36,185][105620] Updated weights for policy 1, policy_version 675104 (0.0011) [2023-12-26 20:16:36,248][105620] Updated weights for policy 1, policy_version 675114 (0.0011) [2023-12-26 20:16:36,300][105620] Updated weights for policy 1, policy_version 675124 (0.0010) [2023-12-26 20:16:36,727][105692] Updated weights for policy 0, policy_version 674314 (0.0008) [2023-12-26 20:16:36,782][105692] Updated weights for policy 0, policy_version 674324 (0.0008) [2023-12-26 20:16:36,834][105692] Updated weights for policy 0, policy_version 674334 (0.0006) [2023-12-26 20:16:36,888][105692] Updated weights for policy 0, policy_version 674344 (0.0006) [2023-12-26 20:16:37,056][105620] Updated weights for policy 1, policy_version 675134 (0.0010) [2023-12-26 20:16:37,105][105620] Updated weights for policy 1, policy_version 675144 (0.0010) [2023-12-26 20:16:37,153][105620] Updated weights for policy 1, policy_version 675154 (0.0010) [2023-12-26 20:16:37,589][105692] Updated weights for policy 0, policy_version 674354 (0.0010) [2023-12-26 20:16:37,648][105692] Updated weights for policy 0, policy_version 674364 (0.0010) [2023-12-26 20:16:37,710][105692] Updated weights for policy 0, policy_version 674374 (0.0010) [2023-12-26 20:16:37,829][105620] Updated weights for policy 1, policy_version 675164 (0.0009) [2023-12-26 20:16:37,894][105620] Updated weights for policy 1, policy_version 675174 (0.0010) [2023-12-26 20:16:37,949][105620] Updated weights for policy 1, policy_version 675184 (0.0010) [2023-12-26 20:16:38,534][105692] Updated weights for policy 0, policy_version 674384 (0.0009) [2023-12-26 20:16:38,590][105692] Updated weights for policy 0, policy_version 674394 (0.0008) [2023-12-26 20:16:38,602][105585] KL-divergence is very high: 158.3874 [2023-12-26 20:16:38,638][105620] Updated weights for policy 1, policy_version 675194 (0.0010) [2023-12-26 20:16:38,651][105585] KL-divergence is very high: 163.4979 [2023-12-26 20:16:38,652][105692] Updated weights for policy 0, policy_version 674404 (0.0007) [2023-12-26 20:16:38,697][105620] Updated weights for policy 1, policy_version 675204 (0.0010) [2023-12-26 20:16:38,764][105620] Updated weights for policy 1, policy_version 675214 (0.0011) [2023-12-26 20:16:38,824][105620] Updated weights for policy 1, policy_version 675224 (0.0009) [2023-12-26 20:16:39,369][105692] Updated weights for policy 0, policy_version 674414 (0.0008) [2023-12-26 20:16:39,434][105692] Updated weights for policy 0, policy_version 674424 (0.0009) [2023-12-26 20:16:39,496][105692] Updated weights for policy 0, policy_version 674434 (0.0009) [2023-12-26 20:16:39,574][105620] Updated weights for policy 1, policy_version 675234 (0.0007) [2023-12-26 20:16:39,639][105620] Updated weights for policy 1, policy_version 675244 (0.0009) [2023-12-26 20:16:39,706][105620] Updated weights for policy 1, policy_version 675254 (0.0010) [2023-12-26 20:16:40,151][105692] Updated weights for policy 0, policy_version 674444 (0.0007) [2023-12-26 20:16:40,219][105692] Updated weights for policy 0, policy_version 674454 (0.0006) [2023-12-26 20:16:40,288][105692] Updated weights for policy 0, policy_version 674464 (0.0009) [2023-12-26 20:16:40,483][105620] Updated weights for policy 1, policy_version 675264 (0.0009) [2023-12-26 20:16:40,542][105620] Updated weights for policy 1, policy_version 675274 (0.0009) [2023-12-26 20:16:40,596][105620] Updated weights for policy 1, policy_version 675284 (0.0008) [2023-12-26 20:16:40,971][105692] Updated weights for policy 0, policy_version 674474 (0.0009) [2023-12-26 20:16:41,025][105692] Updated weights for policy 0, policy_version 674484 (0.0008) [2023-12-26 20:16:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 345587712. Throughput: 0: 9763.8, 1: 9559.7. Samples: 345598896. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 20:16:41,063][104569] Avg episode reward: [(0, '8805.863'), (1, '8719.403')] [2023-12-26 20:16:41,086][105692] Updated weights for policy 0, policy_version 674494 (0.0009) [2023-12-26 20:16:41,161][105692] Updated weights for policy 0, policy_version 674504 (0.0009) [2023-12-26 20:16:41,326][105620] Updated weights for policy 1, policy_version 675294 (0.0008) [2023-12-26 20:16:41,399][105620] Updated weights for policy 1, policy_version 675304 (0.0008) [2023-12-26 20:16:41,468][105620] Updated weights for policy 1, policy_version 675314 (0.0006) [2023-12-26 20:16:42,000][105692] Updated weights for policy 0, policy_version 674514 (0.0007) [2023-12-26 20:16:42,064][105692] Updated weights for policy 0, policy_version 674524 (0.0006) [2023-12-26 20:16:42,124][105620] Updated weights for policy 1, policy_version 675324 (0.0007) [2023-12-26 20:16:42,125][105692] Updated weights for policy 0, policy_version 674534 (0.0006) [2023-12-26 20:16:42,190][105620] Updated weights for policy 1, policy_version 675334 (0.0008) [2023-12-26 20:16:42,255][105620] Updated weights for policy 1, policy_version 675344 (0.0006) [2023-12-26 20:16:42,837][105692] Updated weights for policy 0, policy_version 674544 (0.0005) [2023-12-26 20:16:42,902][105692] Updated weights for policy 0, policy_version 674554 (0.0005) [2023-12-26 20:16:42,952][105620] Updated weights for policy 1, policy_version 675354 (0.0007) [2023-12-26 20:16:42,964][105692] Updated weights for policy 0, policy_version 674564 (0.0005) [2023-12-26 20:16:43,022][105620] Updated weights for policy 1, policy_version 675364 (0.0005) [2023-12-26 20:16:43,081][105620] Updated weights for policy 1, policy_version 675374 (0.0005) [2023-12-26 20:16:43,138][105620] Updated weights for policy 1, policy_version 675384 (0.0005) [2023-12-26 20:16:43,557][105692] Updated weights for policy 0, policy_version 674574 (0.0006) [2023-12-26 20:16:43,606][105692] Updated weights for policy 0, policy_version 674584 (0.0006) [2023-12-26 20:16:43,659][105692] Updated weights for policy 0, policy_version 674594 (0.0006) [2023-12-26 20:16:43,691][105620] Updated weights for policy 1, policy_version 675394 (0.0005) [2023-12-26 20:16:43,750][105620] Updated weights for policy 1, policy_version 675404 (0.0005) [2023-12-26 20:16:43,812][105620] Updated weights for policy 1, policy_version 675414 (0.0005) [2023-12-26 20:16:44,280][105692] Updated weights for policy 0, policy_version 674604 (0.0007) [2023-12-26 20:16:44,329][105692] Updated weights for policy 0, policy_version 674614 (0.0005) [2023-12-26 20:16:44,378][105692] Updated weights for policy 0, policy_version 674624 (0.0008) [2023-12-26 20:16:44,467][105620] Updated weights for policy 1, policy_version 675424 (0.0010) [2023-12-26 20:16:44,524][105620] Updated weights for policy 1, policy_version 675434 (0.0011) [2023-12-26 20:16:44,573][105620] Updated weights for policy 1, policy_version 675444 (0.0010) [2023-12-26 20:16:45,015][105692] Updated weights for policy 0, policy_version 674634 (0.0009) [2023-12-26 20:16:45,076][105692] Updated weights for policy 0, policy_version 674644 (0.0008) [2023-12-26 20:16:45,133][105692] Updated weights for policy 0, policy_version 674654 (0.0008) [2023-12-26 20:16:45,195][105692] Updated weights for policy 0, policy_version 674664 (0.0006) [2023-12-26 20:16:45,367][105620] Updated weights for policy 1, policy_version 675454 (0.0011) [2023-12-26 20:16:45,435][105620] Updated weights for policy 1, policy_version 675464 (0.0009) [2023-12-26 20:16:45,499][105620] Updated weights for policy 1, policy_version 675474 (0.0006) [2023-12-26 20:16:45,910][105692] Updated weights for policy 0, policy_version 674674 (0.0007) [2023-12-26 20:16:45,962][105692] Updated weights for policy 0, policy_version 674684 (0.0005) [2023-12-26 20:16:46,013][105692] Updated weights for policy 0, policy_version 674694 (0.0005) [2023-12-26 20:16:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 345694208. Throughput: 0: 9794.4, 1: 9662.9. Samples: 345660320. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 20:16:46,063][104569] Avg episode reward: [(0, '8807.009'), (1, '9264.392')] [2023-12-26 20:16:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000675480_172941312.pth... [2023-12-26 20:16:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000674696_172752896.pth... [2023-12-26 20:16:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000673544_172457984.pth [2023-12-26 20:16:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000674360_172654592.pth [2023-12-26 20:16:46,212][105620] Updated weights for policy 1, policy_version 675484 (0.0011) [2023-12-26 20:16:46,272][105620] Updated weights for policy 1, policy_version 675494 (0.0011) [2023-12-26 20:16:46,331][105620] Updated weights for policy 1, policy_version 675504 (0.0011) [2023-12-26 20:16:46,683][105692] Updated weights for policy 0, policy_version 674704 (0.0005) [2023-12-26 20:16:46,736][105692] Updated weights for policy 0, policy_version 674714 (0.0005) [2023-12-26 20:16:46,790][105692] Updated weights for policy 0, policy_version 674724 (0.0005) [2023-12-26 20:16:47,000][105620] Updated weights for policy 1, policy_version 675514 (0.0010) [2023-12-26 20:16:47,065][105620] Updated weights for policy 1, policy_version 675524 (0.0005) [2023-12-26 20:16:47,130][105620] Updated weights for policy 1, policy_version 675534 (0.0005) [2023-12-26 20:16:47,194][105620] Updated weights for policy 1, policy_version 675544 (0.0006) [2023-12-26 20:16:47,369][105692] Updated weights for policy 0, policy_version 674734 (0.0006) [2023-12-26 20:16:47,413][105692] Updated weights for policy 0, policy_version 674744 (0.0005) [2023-12-26 20:16:47,460][105692] Updated weights for policy 0, policy_version 674754 (0.0005) [2023-12-26 20:16:47,769][105620] Updated weights for policy 1, policy_version 675554 (0.0009) [2023-12-26 20:16:47,833][105620] Updated weights for policy 1, policy_version 675564 (0.0008) [2023-12-26 20:16:47,888][105620] Updated weights for policy 1, policy_version 675574 (0.0010) [2023-12-26 20:16:48,013][105692] Updated weights for policy 0, policy_version 674764 (0.0005) [2023-12-26 20:16:48,074][105692] Updated weights for policy 0, policy_version 674774 (0.0005) [2023-12-26 20:16:48,127][105692] Updated weights for policy 0, policy_version 674784 (0.0005) [2023-12-26 20:16:48,634][105620] Updated weights for policy 1, policy_version 675584 (0.0006) [2023-12-26 20:16:48,695][105620] Updated weights for policy 1, policy_version 675594 (0.0006) [2023-12-26 20:16:48,756][105692] Updated weights for policy 0, policy_version 674794 (0.0006) [2023-12-26 20:16:48,757][105620] Updated weights for policy 1, policy_version 675604 (0.0008) [2023-12-26 20:16:48,802][105692] Updated weights for policy 0, policy_version 674804 (0.0010) [2023-12-26 20:16:48,857][105692] Updated weights for policy 0, policy_version 674814 (0.0010) [2023-12-26 20:16:48,906][105692] Updated weights for policy 0, policy_version 674824 (0.0010) [2023-12-26 20:16:49,395][105620] Updated weights for policy 1, policy_version 675614 (0.0008) [2023-12-26 20:16:49,460][105620] Updated weights for policy 1, policy_version 675624 (0.0009) [2023-12-26 20:16:49,518][105620] Updated weights for policy 1, policy_version 675634 (0.0009) [2023-12-26 20:16:49,651][105692] Updated weights for policy 0, policy_version 674834 (0.0009) [2023-12-26 20:16:49,710][105692] Updated weights for policy 0, policy_version 674844 (0.0008) [2023-12-26 20:16:49,763][105692] Updated weights for policy 0, policy_version 674854 (0.0008) [2023-12-26 20:16:50,248][105620] Updated weights for policy 1, policy_version 675644 (0.0010) [2023-12-26 20:16:50,310][105620] Updated weights for policy 1, policy_version 675654 (0.0010) [2023-12-26 20:16:50,375][105620] Updated weights for policy 1, policy_version 675664 (0.0010) [2023-12-26 20:16:50,480][105692] Updated weights for policy 0, policy_version 674864 (0.0008) [2023-12-26 20:16:50,526][105692] Updated weights for policy 0, policy_version 674874 (0.0008) [2023-12-26 20:16:50,583][105692] Updated weights for policy 0, policy_version 674885 (0.0009) [2023-12-26 20:16:51,059][105620] Updated weights for policy 1, policy_version 675674 (0.0007) [2023-12-26 20:16:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 345792512. Throughput: 0: 9936.5, 1: 9721.5. Samples: 345783780. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 20:16:51,062][104569] Avg episode reward: [(0, '8492.833'), (1, '9080.046')] [2023-12-26 20:16:51,125][105620] Updated weights for policy 1, policy_version 675684 (0.0010) [2023-12-26 20:16:51,190][105620] Updated weights for policy 1, policy_version 675694 (0.0009) [2023-12-26 20:16:51,241][105620] Updated weights for policy 1, policy_version 675704 (0.0009) [2023-12-26 20:16:51,359][105692] Updated weights for policy 0, policy_version 674895 (0.0008) [2023-12-26 20:16:51,415][105692] Updated weights for policy 0, policy_version 674905 (0.0008) [2023-12-26 20:16:51,476][105692] Updated weights for policy 0, policy_version 674915 (0.0009) [2023-12-26 20:16:52,036][105620] Updated weights for policy 1, policy_version 675714 (0.0006) [2023-12-26 20:16:52,098][105620] Updated weights for policy 1, policy_version 675724 (0.0009) [2023-12-26 20:16:52,148][105620] Updated weights for policy 1, policy_version 675734 (0.0009) [2023-12-26 20:16:52,163][105692] Updated weights for policy 0, policy_version 674925 (0.0006) [2023-12-26 20:16:52,225][105692] Updated weights for policy 0, policy_version 674935 (0.0009) [2023-12-26 20:16:52,282][105692] Updated weights for policy 0, policy_version 674945 (0.0009) [2023-12-26 20:16:52,820][105620] Updated weights for policy 1, policy_version 675744 (0.0006) [2023-12-26 20:16:52,880][105620] Updated weights for policy 1, policy_version 675754 (0.0005) [2023-12-26 20:16:52,944][105620] Updated weights for policy 1, policy_version 675764 (0.0006) [2023-12-26 20:16:53,048][105692] Updated weights for policy 0, policy_version 674955 (0.0008) [2023-12-26 20:16:53,114][105692] Updated weights for policy 0, policy_version 674965 (0.0008) [2023-12-26 20:16:53,167][105692] Updated weights for policy 0, policy_version 674975 (0.0010) [2023-12-26 20:16:53,510][105620] Updated weights for policy 1, policy_version 675774 (0.0009) [2023-12-26 20:16:53,567][105620] Updated weights for policy 1, policy_version 675784 (0.0009) [2023-12-26 20:16:53,625][105620] Updated weights for policy 1, policy_version 675794 (0.0006) [2023-12-26 20:16:53,836][105692] Updated weights for policy 0, policy_version 674985 (0.0009) [2023-12-26 20:16:53,894][105692] Updated weights for policy 0, policy_version 674995 (0.0005) [2023-12-26 20:16:53,954][105692] Updated weights for policy 0, policy_version 675005 (0.0007) [2023-12-26 20:16:54,002][105692] Updated weights for policy 0, policy_version 675015 (0.0010) [2023-12-26 20:16:54,353][105620] Updated weights for policy 1, policy_version 675804 (0.0007) [2023-12-26 20:16:54,404][105620] Updated weights for policy 1, policy_version 675814 (0.0008) [2023-12-26 20:16:54,460][105620] Updated weights for policy 1, policy_version 675824 (0.0008) [2023-12-26 20:16:54,724][105692] Updated weights for policy 0, policy_version 675025 (0.0010) [2023-12-26 20:16:54,776][105692] Updated weights for policy 0, policy_version 675035 (0.0007) [2023-12-26 20:16:54,830][105692] Updated weights for policy 0, policy_version 675045 (0.0005) [2023-12-26 20:16:55,160][105620] Updated weights for policy 1, policy_version 675834 (0.0008) [2023-12-26 20:16:55,220][105620] Updated weights for policy 1, policy_version 675844 (0.0008) [2023-12-26 20:16:55,276][105620] Updated weights for policy 1, policy_version 675854 (0.0008) [2023-12-26 20:16:55,333][105620] Updated weights for policy 1, policy_version 675864 (0.0008) [2023-12-26 20:16:55,561][105692] Updated weights for policy 0, policy_version 675055 (0.0009) [2023-12-26 20:16:55,623][105692] Updated weights for policy 0, policy_version 675065 (0.0010) [2023-12-26 20:16:55,678][105692] Updated weights for policy 0, policy_version 675075 (0.0011) [2023-12-26 20:16:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 345890816. Throughput: 0: 9899.0, 1: 9703.5. Samples: 345901552. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 20:16:56,063][104569] Avg episode reward: [(0, '8583.282'), (1, '8908.754')] [2023-12-26 20:16:56,081][105620] Updated weights for policy 1, policy_version 675874 (0.0008) [2023-12-26 20:16:56,139][105620] Updated weights for policy 1, policy_version 675884 (0.0008) [2023-12-26 20:16:56,190][105620] Updated weights for policy 1, policy_version 675894 (0.0007) [2023-12-26 20:16:56,374][105692] Updated weights for policy 0, policy_version 675085 (0.0011) [2023-12-26 20:16:56,429][105692] Updated weights for policy 0, policy_version 675095 (0.0011) [2023-12-26 20:16:56,488][105692] Updated weights for policy 0, policy_version 675105 (0.0011) [2023-12-26 20:16:56,910][105620] Updated weights for policy 1, policy_version 675904 (0.0009) [2023-12-26 20:16:56,956][105620] Updated weights for policy 1, policy_version 675914 (0.0008) [2023-12-26 20:16:57,011][105620] Updated weights for policy 1, policy_version 675924 (0.0009) [2023-12-26 20:16:57,272][105692] Updated weights for policy 0, policy_version 675115 (0.0010) [2023-12-26 20:16:57,328][105692] Updated weights for policy 0, policy_version 675125 (0.0009) [2023-12-26 20:16:57,381][105692] Updated weights for policy 0, policy_version 675135 (0.0010) [2023-12-26 20:16:57,690][105620] Updated weights for policy 1, policy_version 675934 (0.0008) [2023-12-26 20:16:57,745][105586] KL-divergence is very high: 102.0274 [2023-12-26 20:16:57,750][105620] Updated weights for policy 1, policy_version 675944 (0.0009) [2023-12-26 20:16:57,763][105586] KL-divergence is very high: 109.5649 [2023-12-26 20:16:57,814][105620] Updated weights for policy 1, policy_version 675954 (0.0009) [2023-12-26 20:16:58,018][105692] Updated weights for policy 0, policy_version 675145 (0.0010) [2023-12-26 20:16:58,075][105692] Updated weights for policy 0, policy_version 675155 (0.0005) [2023-12-26 20:16:58,147][105692] Updated weights for policy 0, policy_version 675165 (0.0006) [2023-12-26 20:16:58,207][105692] Updated weights for policy 0, policy_version 675175 (0.0007) [2023-12-26 20:16:58,638][105620] Updated weights for policy 1, policy_version 675964 (0.0009) [2023-12-26 20:16:58,702][105620] Updated weights for policy 1, policy_version 675974 (0.0009) [2023-12-26 20:16:58,767][105620] Updated weights for policy 1, policy_version 675984 (0.0008) [2023-12-26 20:16:58,910][105692] Updated weights for policy 0, policy_version 675185 (0.0007) [2023-12-26 20:16:58,976][105692] Updated weights for policy 0, policy_version 675195 (0.0008) [2023-12-26 20:16:59,029][105692] Updated weights for policy 0, policy_version 675205 (0.0009) [2023-12-26 20:16:59,558][105620] Updated weights for policy 1, policy_version 675994 (0.0010) [2023-12-26 20:16:59,609][105620] Updated weights for policy 1, policy_version 676005 (0.0009) [2023-12-26 20:16:59,654][105620] Updated weights for policy 1, policy_version 676015 (0.0008) [2023-12-26 20:16:59,797][105692] Updated weights for policy 0, policy_version 675215 (0.0010) [2023-12-26 20:16:59,853][105692] Updated weights for policy 0, policy_version 675225 (0.0009) [2023-12-26 20:16:59,916][105692] Updated weights for policy 0, policy_version 675235 (0.0007) [2023-12-26 20:17:00,475][105620] Updated weights for policy 1, policy_version 676025 (0.0007) [2023-12-26 20:17:00,538][105620] Updated weights for policy 1, policy_version 676035 (0.0008) [2023-12-26 20:17:00,593][105620] Updated weights for policy 1, policy_version 676045 (0.0009) [2023-12-26 20:17:00,638][105692] Updated weights for policy 0, policy_version 675245 (0.0010) [2023-12-26 20:17:00,640][105620] Updated weights for policy 1, policy_version 676055 (0.0008) [2023-12-26 20:17:00,703][105692] Updated weights for policy 0, policy_version 675255 (0.0005) [2023-12-26 20:17:00,767][105692] Updated weights for policy 0, policy_version 675265 (0.0005) [2023-12-26 20:17:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 345989120. Throughput: 0: 9903.7, 1: 9648.6. Samples: 345958404. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 20:17:01,062][104569] Avg episode reward: [(0, '8987.863'), (1, '7782.272')] [2023-12-26 20:17:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000675272_172900352.pth... [2023-12-26 20:17:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000676056_173088768.pth... [2023-12-26 20:17:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000674120_172605440.pth [2023-12-26 20:17:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000674936_172802048.pth [2023-12-26 20:17:01,332][105620] Updated weights for policy 1, policy_version 676065 (0.0006) [2023-12-26 20:17:01,401][105620] Updated weights for policy 1, policy_version 676075 (0.0009) [2023-12-26 20:17:01,454][105620] Updated weights for policy 1, policy_version 676085 (0.0011) [2023-12-26 20:17:01,493][105692] Updated weights for policy 0, policy_version 675275 (0.0007) [2023-12-26 20:17:01,544][105692] Updated weights for policy 0, policy_version 675285 (0.0008) [2023-12-26 20:17:01,601][105692] Updated weights for policy 0, policy_version 675295 (0.0007) [2023-12-26 20:17:02,132][105620] Updated weights for policy 1, policy_version 676095 (0.0009) [2023-12-26 20:17:02,194][105620] Updated weights for policy 1, policy_version 676105 (0.0009) [2023-12-26 20:17:02,251][105620] Updated weights for policy 1, policy_version 676115 (0.0010) [2023-12-26 20:17:02,380][105692] Updated weights for policy 0, policy_version 675305 (0.0008) [2023-12-26 20:17:02,431][105692] Updated weights for policy 0, policy_version 675315 (0.0009) [2023-12-26 20:17:02,486][105692] Updated weights for policy 0, policy_version 675325 (0.0009) [2023-12-26 20:17:02,552][105692] Updated weights for policy 0, policy_version 675335 (0.0009) [2023-12-26 20:17:03,017][105620] Updated weights for policy 1, policy_version 676125 (0.0010) [2023-12-26 20:17:03,062][105620] Updated weights for policy 1, policy_version 676135 (0.0010) [2023-12-26 20:17:03,109][105620] Updated weights for policy 1, policy_version 676145 (0.0009) [2023-12-26 20:17:03,353][105692] Updated weights for policy 0, policy_version 675345 (0.0009) [2023-12-26 20:17:03,414][105692] Updated weights for policy 0, policy_version 675355 (0.0010) [2023-12-26 20:17:03,485][105692] Updated weights for policy 0, policy_version 675365 (0.0010) [2023-12-26 20:17:03,716][105620] Updated weights for policy 1, policy_version 676155 (0.0007) [2023-12-26 20:17:03,770][105620] Updated weights for policy 1, policy_version 676165 (0.0010) [2023-12-26 20:17:03,814][105620] Updated weights for policy 1, policy_version 676175 (0.0010) [2023-12-26 20:17:04,319][105692] Updated weights for policy 0, policy_version 675375 (0.0009) [2023-12-26 20:17:04,382][105692] Updated weights for policy 0, policy_version 675385 (0.0008) [2023-12-26 20:17:04,442][105692] Updated weights for policy 0, policy_version 675395 (0.0008) [2023-12-26 20:17:04,508][105620] Updated weights for policy 1, policy_version 676185 (0.0010) [2023-12-26 20:17:04,566][105620] Updated weights for policy 1, policy_version 676195 (0.0010) [2023-12-26 20:17:04,624][105620] Updated weights for policy 1, policy_version 676205 (0.0010) [2023-12-26 20:17:04,672][105620] Updated weights for policy 1, policy_version 676215 (0.0010) [2023-12-26 20:17:05,243][105692] Updated weights for policy 0, policy_version 675405 (0.0008) [2023-12-26 20:17:05,295][105692] Updated weights for policy 0, policy_version 675415 (0.0008) [2023-12-26 20:17:05,354][105692] Updated weights for policy 0, policy_version 675425 (0.0008) [2023-12-26 20:17:05,426][105620] Updated weights for policy 1, policy_version 676225 (0.0010) [2023-12-26 20:17:05,474][105620] Updated weights for policy 1, policy_version 676235 (0.0010) [2023-12-26 20:17:05,522][105620] Updated weights for policy 1, policy_version 676245 (0.0010) [2023-12-26 20:17:06,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 346079232. Throughput: 0: 9708.9, 1: 9661.1. Samples: 346072352. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 20:17:06,063][104569] Avg episode reward: [(0, '9261.020'), (1, '8021.119')] [2023-12-26 20:17:06,167][105692] Updated weights for policy 0, policy_version 675435 (0.0007) [2023-12-26 20:17:06,174][105620] Updated weights for policy 1, policy_version 676255 (0.0009) [2023-12-26 20:17:06,222][105692] Updated weights for policy 0, policy_version 675445 (0.0009) [2023-12-26 20:17:06,225][105620] Updated weights for policy 1, policy_version 676265 (0.0005) [2023-12-26 20:17:06,280][105620] Updated weights for policy 1, policy_version 676275 (0.0006) [2023-12-26 20:17:06,286][105692] Updated weights for policy 0, policy_version 675455 (0.0008) [2023-12-26 20:17:06,937][105620] Updated weights for policy 1, policy_version 676285 (0.0008) [2023-12-26 20:17:06,996][105620] Updated weights for policy 1, policy_version 676295 (0.0010) [2023-12-26 20:17:07,051][105620] Updated weights for policy 1, policy_version 676305 (0.0010) [2023-12-26 20:17:07,104][105692] Updated weights for policy 0, policy_version 675465 (0.0009) [2023-12-26 20:17:07,158][105692] Updated weights for policy 0, policy_version 675475 (0.0010) [2023-12-26 20:17:07,213][105692] Updated weights for policy 0, policy_version 675485 (0.0010) [2023-12-26 20:17:07,273][105692] Updated weights for policy 0, policy_version 675495 (0.0010) [2023-12-26 20:17:07,616][105620] Updated weights for policy 1, policy_version 676315 (0.0009) [2023-12-26 20:17:07,664][105620] Updated weights for policy 1, policy_version 676325 (0.0010) [2023-12-26 20:17:07,712][105620] Updated weights for policy 1, policy_version 676335 (0.0010) [2023-12-26 20:17:08,108][105692] Updated weights for policy 0, policy_version 675505 (0.0009) [2023-12-26 20:17:08,170][105692] Updated weights for policy 0, policy_version 675515 (0.0009) [2023-12-26 20:17:08,229][105692] Updated weights for policy 0, policy_version 675525 (0.0010) [2023-12-26 20:17:08,307][105620] Updated weights for policy 1, policy_version 676345 (0.0010) [2023-12-26 20:17:08,377][105620] Updated weights for policy 1, policy_version 676355 (0.0008) [2023-12-26 20:17:08,439][105620] Updated weights for policy 1, policy_version 676365 (0.0006) [2023-12-26 20:17:08,490][105620] Updated weights for policy 1, policy_version 676375 (0.0005) [2023-12-26 20:17:09,048][105620] Updated weights for policy 1, policy_version 676385 (0.0006) [2023-12-26 20:17:09,109][105620] Updated weights for policy 1, policy_version 676395 (0.0005) [2023-12-26 20:17:09,143][105692] Updated weights for policy 0, policy_version 675536 (0.0010) [2023-12-26 20:17:09,165][105620] Updated weights for policy 1, policy_version 676405 (0.0005) [2023-12-26 20:17:09,197][105692] Updated weights for policy 0, policy_version 675546 (0.0009) [2023-12-26 20:17:09,260][105692] Updated weights for policy 0, policy_version 675556 (0.0009) [2023-12-26 20:17:09,874][105620] Updated weights for policy 1, policy_version 676415 (0.0008) [2023-12-26 20:17:09,939][105620] Updated weights for policy 1, policy_version 676425 (0.0009) [2023-12-26 20:17:10,005][105620] Updated weights for policy 1, policy_version 676435 (0.0007) [2023-12-26 20:17:10,027][105692] Updated weights for policy 0, policy_version 675566 (0.0008) [2023-12-26 20:17:10,075][105692] Updated weights for policy 0, policy_version 675576 (0.0009) [2023-12-26 20:17:10,138][105692] Updated weights for policy 0, policy_version 675586 (0.0008) [2023-12-26 20:17:10,692][105620] Updated weights for policy 1, policy_version 676445 (0.0009) [2023-12-26 20:17:10,752][105620] Updated weights for policy 1, policy_version 676455 (0.0009) [2023-12-26 20:17:10,818][105620] Updated weights for policy 1, policy_version 676465 (0.0008) [2023-12-26 20:17:10,933][105692] Updated weights for policy 0, policy_version 675596 (0.0009) [2023-12-26 20:17:10,995][105692] Updated weights for policy 0, policy_version 675606 (0.0009) [2023-12-26 20:17:11,062][105692] Updated weights for policy 0, policy_version 675616 (0.0009) [2023-12-26 20:17:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 346177536. Throughput: 0: 9611.9, 1: 9812.3. Samples: 346187632. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-26 20:17:11,062][104569] Avg episode reward: [(0, '9078.754'), (1, '9077.873')] [2023-12-26 20:17:11,614][105620] Updated weights for policy 1, policy_version 676475 (0.0009) [2023-12-26 20:17:11,674][105620] Updated weights for policy 1, policy_version 676485 (0.0009) [2023-12-26 20:17:11,736][105620] Updated weights for policy 1, policy_version 676495 (0.0008) [2023-12-26 20:17:11,858][105692] Updated weights for policy 0, policy_version 675626 (0.0008) [2023-12-26 20:17:11,911][105692] Updated weights for policy 0, policy_version 675636 (0.0009) [2023-12-26 20:17:11,960][105692] Updated weights for policy 0, policy_version 675646 (0.0009) [2023-12-26 20:17:12,022][105692] Updated weights for policy 0, policy_version 675656 (0.0009) [2023-12-26 20:17:12,471][105620] Updated weights for policy 1, policy_version 676505 (0.0009) [2023-12-26 20:17:12,540][105620] Updated weights for policy 1, policy_version 676515 (0.0007) [2023-12-26 20:17:12,602][105620] Updated weights for policy 1, policy_version 676525 (0.0009) [2023-12-26 20:17:12,664][105620] Updated weights for policy 1, policy_version 676535 (0.0007) [2023-12-26 20:17:12,876][105692] Updated weights for policy 0, policy_version 675666 (0.0008) [2023-12-26 20:17:12,941][105692] Updated weights for policy 0, policy_version 675676 (0.0008) [2023-12-26 20:17:13,002][105692] Updated weights for policy 0, policy_version 675686 (0.0008) [2023-12-26 20:17:13,406][105620] Updated weights for policy 1, policy_version 676545 (0.0009) [2023-12-26 20:17:13,454][105620] Updated weights for policy 1, policy_version 676555 (0.0008) [2023-12-26 20:17:13,503][105620] Updated weights for policy 1, policy_version 676565 (0.0009) [2023-12-26 20:17:13,573][105692] Updated weights for policy 0, policy_version 675697 (0.0007) [2023-12-26 20:17:13,627][105692] Updated weights for policy 0, policy_version 675707 (0.0008) [2023-12-26 20:17:13,675][105692] Updated weights for policy 0, policy_version 675717 (0.0009) [2023-12-26 20:17:14,208][105620] Updated weights for policy 1, policy_version 676575 (0.0008) [2023-12-26 20:17:14,255][105620] Updated weights for policy 1, policy_version 676585 (0.0009) [2023-12-26 20:17:14,302][105620] Updated weights for policy 1, policy_version 676595 (0.0009) [2023-12-26 20:17:14,479][105692] Updated weights for policy 0, policy_version 675727 (0.0009) [2023-12-26 20:17:14,527][105692] Updated weights for policy 0, policy_version 675737 (0.0009) [2023-12-26 20:17:14,582][105692] Updated weights for policy 0, policy_version 675747 (0.0009) [2023-12-26 20:17:15,089][105620] Updated weights for policy 1, policy_version 676605 (0.0007) [2023-12-26 20:17:15,149][105620] Updated weights for policy 1, policy_version 676615 (0.0006) [2023-12-26 20:17:15,213][105620] Updated weights for policy 1, policy_version 676625 (0.0006) [2023-12-26 20:17:15,437][105692] Updated weights for policy 0, policy_version 675757 (0.0009) [2023-12-26 20:17:15,495][105692] Updated weights for policy 0, policy_version 675767 (0.0009) [2023-12-26 20:17:15,562][105692] Updated weights for policy 0, policy_version 675777 (0.0007) [2023-12-26 20:17:15,891][105620] Updated weights for policy 1, policy_version 676635 (0.0005) [2023-12-26 20:17:15,953][105620] Updated weights for policy 1, policy_version 676645 (0.0005) [2023-12-26 20:17:16,016][105620] Updated weights for policy 1, policy_version 676655 (0.0005) [2023-12-26 20:17:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.9, 300 sec: 19549.7). Total num frames: 346267648. Throughput: 0: 9543.2, 1: 9840.8. Samples: 346244460. Policy #0 lag: (min: 31.0, avg: 31.5, max: 49.0) [2023-12-26 20:17:16,063][104569] Avg episode reward: [(0, '8986.741'), (1, '8896.626')] [2023-12-26 20:17:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000675784_173031424.pth... [2023-12-26 20:17:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000676664_173244416.pth... [2023-12-26 20:17:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000674696_172752896.pth [2023-12-26 20:17:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000675480_172941312.pth [2023-12-26 20:17:16,337][105692] Updated weights for policy 0, policy_version 675787 (0.0008) [2023-12-26 20:17:16,399][105692] Updated weights for policy 0, policy_version 675797 (0.0009) [2023-12-26 20:17:16,464][105692] Updated weights for policy 0, policy_version 675807 (0.0009) [2023-12-26 20:17:16,626][105620] Updated weights for policy 1, policy_version 676665 (0.0006) [2023-12-26 20:17:16,686][105620] Updated weights for policy 1, policy_version 676675 (0.0008) [2023-12-26 20:17:16,742][105620] Updated weights for policy 1, policy_version 676685 (0.0009) [2023-12-26 20:17:16,800][105620] Updated weights for policy 1, policy_version 676695 (0.0010) [2023-12-26 20:17:17,166][105692] Updated weights for policy 0, policy_version 675817 (0.0010) [2023-12-26 20:17:17,222][105692] Updated weights for policy 0, policy_version 675827 (0.0009) [2023-12-26 20:17:17,281][105692] Updated weights for policy 0, policy_version 675837 (0.0009) [2023-12-26 20:17:17,336][105692] Updated weights for policy 0, policy_version 675847 (0.0009) [2023-12-26 20:17:17,514][105620] Updated weights for policy 1, policy_version 676705 (0.0006) [2023-12-26 20:17:17,565][105620] Updated weights for policy 1, policy_version 676715 (0.0005) [2023-12-26 20:17:17,624][105620] Updated weights for policy 1, policy_version 676725 (0.0006) [2023-12-26 20:17:18,167][105620] Updated weights for policy 1, policy_version 676735 (0.0005) [2023-12-26 20:17:18,180][105692] Updated weights for policy 0, policy_version 675857 (0.0009) [2023-12-26 20:17:18,224][105620] Updated weights for policy 1, policy_version 676745 (0.0007) [2023-12-26 20:17:18,227][105692] Updated weights for policy 0, policy_version 675867 (0.0006) [2023-12-26 20:17:18,277][105620] Updated weights for policy 1, policy_version 676755 (0.0006) [2023-12-26 20:17:18,290][105692] Updated weights for policy 0, policy_version 675877 (0.0008) [2023-12-26 20:17:19,037][105620] Updated weights for policy 1, policy_version 676765 (0.0010) [2023-12-26 20:17:19,068][105692] Updated weights for policy 0, policy_version 675887 (0.0008) [2023-12-26 20:17:19,102][105620] Updated weights for policy 1, policy_version 676775 (0.0007) [2023-12-26 20:17:19,123][105692] Updated weights for policy 0, policy_version 675897 (0.0007) [2023-12-26 20:17:19,163][105620] Updated weights for policy 1, policy_version 676785 (0.0007) [2023-12-26 20:17:19,175][105692] Updated weights for policy 0, policy_version 675907 (0.0008) [2023-12-26 20:17:19,846][105692] Updated weights for policy 0, policy_version 675917 (0.0009) [2023-12-26 20:17:19,903][105692] Updated weights for policy 0, policy_version 675927 (0.0011) [2023-12-26 20:17:19,970][105692] Updated weights for policy 0, policy_version 675937 (0.0009) [2023-12-26 20:17:19,974][105620] Updated weights for policy 1, policy_version 676795 (0.0006) [2023-12-26 20:17:20,039][105620] Updated weights for policy 1, policy_version 676805 (0.0010) [2023-12-26 20:17:20,106][105620] Updated weights for policy 1, policy_version 676815 (0.0008) [2023-12-26 20:17:20,682][105692] Updated weights for policy 0, policy_version 675947 (0.0009) [2023-12-26 20:17:20,743][105692] Updated weights for policy 0, policy_version 675957 (0.0006) [2023-12-26 20:17:20,744][105585] KL-divergence is very high: 141.9022 [2023-12-26 20:17:20,795][105585] KL-divergence is very high: 228.9145 [2023-12-26 20:17:20,806][105692] Updated weights for policy 0, policy_version 675967 (0.0006) [2023-12-26 20:17:20,845][105585] KL-divergence is very high: 232.3142 [2023-12-26 20:17:20,937][105620] Updated weights for policy 1, policy_version 676825 (0.0008) [2023-12-26 20:17:21,004][105620] Updated weights for policy 1, policy_version 676835 (0.0009) [2023-12-26 20:17:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 346365952. Throughput: 0: 9486.4, 1: 9914.0. Samples: 346358364. Policy #0 lag: (min: 31.0, avg: 31.5, max: 49.0) [2023-12-26 20:17:21,062][104569] Avg episode reward: [(0, '8622.972'), (1, '8896.301')] [2023-12-26 20:17:21,069][105620] Updated weights for policy 1, policy_version 676845 (0.0008) [2023-12-26 20:17:21,138][105620] Updated weights for policy 1, policy_version 676855 (0.0011) [2023-12-26 20:17:21,497][105692] Updated weights for policy 0, policy_version 675977 (0.0006) [2023-12-26 20:17:21,556][105692] Updated weights for policy 0, policy_version 675987 (0.0008) [2023-12-26 20:17:21,605][105692] Updated weights for policy 0, policy_version 675997 (0.0008) [2023-12-26 20:17:21,671][105692] Updated weights for policy 0, policy_version 676007 (0.0008) [2023-12-26 20:17:21,905][105620] Updated weights for policy 1, policy_version 676865 (0.0011) [2023-12-26 20:17:21,971][105620] Updated weights for policy 1, policy_version 676875 (0.0011) [2023-12-26 20:17:22,031][105620] Updated weights for policy 1, policy_version 676885 (0.0010) [2023-12-26 20:17:22,453][105692] Updated weights for policy 0, policy_version 676017 (0.0008) [2023-12-26 20:17:22,509][105692] Updated weights for policy 0, policy_version 676027 (0.0008) [2023-12-26 20:17:22,573][105692] Updated weights for policy 0, policy_version 676037 (0.0008) [2023-12-26 20:17:22,779][105620] Updated weights for policy 1, policy_version 676895 (0.0010) [2023-12-26 20:17:22,845][105620] Updated weights for policy 1, policy_version 676905 (0.0010) [2023-12-26 20:17:22,912][105620] Updated weights for policy 1, policy_version 676915 (0.0010) [2023-12-26 20:17:23,345][105692] Updated weights for policy 0, policy_version 676047 (0.0008) [2023-12-26 20:17:23,393][105692] Updated weights for policy 0, policy_version 676057 (0.0008) [2023-12-26 20:17:23,447][105692] Updated weights for policy 0, policy_version 676067 (0.0007) [2023-12-26 20:17:23,657][105620] Updated weights for policy 1, policy_version 676925 (0.0010) [2023-12-26 20:17:23,705][105620] Updated weights for policy 1, policy_version 676935 (0.0010) [2023-12-26 20:17:23,752][105620] Updated weights for policy 1, policy_version 676945 (0.0010) [2023-12-26 20:17:24,210][105692] Updated weights for policy 0, policy_version 676077 (0.0008) [2023-12-26 20:17:24,265][105692] Updated weights for policy 0, policy_version 676087 (0.0008) [2023-12-26 20:17:24,309][105692] Updated weights for policy 0, policy_version 676097 (0.0008) [2023-12-26 20:17:24,488][105620] Updated weights for policy 1, policy_version 676955 (0.0009) [2023-12-26 20:17:24,538][105620] Updated weights for policy 1, policy_version 676965 (0.0006) [2023-12-26 20:17:24,582][105620] Updated weights for policy 1, policy_version 676975 (0.0007) [2023-12-26 20:17:25,122][105692] Updated weights for policy 0, policy_version 676107 (0.0008) [2023-12-26 20:17:25,168][105692] Updated weights for policy 0, policy_version 676117 (0.0009) [2023-12-26 20:17:25,219][105692] Updated weights for policy 0, policy_version 676127 (0.0009) [2023-12-26 20:17:25,271][105620] Updated weights for policy 1, policy_version 676985 (0.0006) [2023-12-26 20:17:25,315][105620] Updated weights for policy 1, policy_version 676995 (0.0010) [2023-12-26 20:17:25,366][105620] Updated weights for policy 1, policy_version 677005 (0.0010) [2023-12-26 20:17:25,427][105620] Updated weights for policy 1, policy_version 677015 (0.0009) [2023-12-26 20:17:25,968][105692] Updated weights for policy 0, policy_version 676137 (0.0009) [2023-12-26 20:17:26,030][105692] Updated weights for policy 0, policy_version 676147 (0.0010) [2023-12-26 20:17:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19494.2). Total num frames: 346456064. Throughput: 0: 9476.6, 1: 9882.1. Samples: 346470040. Policy #0 lag: (min: 31.0, avg: 31.5, max: 49.0) [2023-12-26 20:17:26,063][104569] Avg episode reward: [(0, '8546.517'), (1, '8985.626')] [2023-12-26 20:17:26,092][105692] Updated weights for policy 0, policy_version 676157 (0.0009) [2023-12-26 20:17:26,138][105692] Updated weights for policy 0, policy_version 676167 (0.0010) [2023-12-26 20:17:26,185][105620] Updated weights for policy 1, policy_version 677025 (0.0010) [2023-12-26 20:17:26,233][105620] Updated weights for policy 1, policy_version 677035 (0.0010) [2023-12-26 20:17:26,279][105620] Updated weights for policy 1, policy_version 677045 (0.0010) [2023-12-26 20:17:26,817][105692] Updated weights for policy 0, policy_version 676177 (0.0011) [2023-12-26 20:17:26,875][105692] Updated weights for policy 0, policy_version 676187 (0.0010) [2023-12-26 20:17:26,934][105692] Updated weights for policy 0, policy_version 676197 (0.0010) [2023-12-26 20:17:27,051][105620] Updated weights for policy 1, policy_version 677055 (0.0011) [2023-12-26 20:17:27,113][105620] Updated weights for policy 1, policy_version 677065 (0.0011) [2023-12-26 20:17:27,165][105620] Updated weights for policy 1, policy_version 677075 (0.0010) [2023-12-26 20:17:27,635][105692] Updated weights for policy 0, policy_version 676207 (0.0010) [2023-12-26 20:17:27,682][105692] Updated weights for policy 0, policy_version 676217 (0.0010) [2023-12-26 20:17:27,726][105692] Updated weights for policy 0, policy_version 676227 (0.0010) [2023-12-26 20:17:27,792][105620] Updated weights for policy 1, policy_version 677085 (0.0008) [2023-12-26 20:17:27,852][105620] Updated weights for policy 1, policy_version 677095 (0.0005) [2023-12-26 20:17:27,899][105620] Updated weights for policy 1, policy_version 677105 (0.0005) [2023-12-26 20:17:28,402][105692] Updated weights for policy 0, policy_version 676237 (0.0009) [2023-12-26 20:17:28,455][105692] Updated weights for policy 0, policy_version 676247 (0.0011) [2023-12-26 20:17:28,497][105620] Updated weights for policy 1, policy_version 677115 (0.0006) [2023-12-26 20:17:28,521][105692] Updated weights for policy 0, policy_version 676257 (0.0011) [2023-12-26 20:17:28,551][105620] Updated weights for policy 1, policy_version 677125 (0.0007) [2023-12-26 20:17:28,613][105620] Updated weights for policy 1, policy_version 677135 (0.0011) [2023-12-26 20:17:29,117][105692] Updated weights for policy 0, policy_version 676267 (0.0011) [2023-12-26 20:17:29,164][105692] Updated weights for policy 0, policy_version 676277 (0.0010) [2023-12-26 20:17:29,209][105692] Updated weights for policy 0, policy_version 676287 (0.0010) [2023-12-26 20:17:29,259][105620] Updated weights for policy 1, policy_version 677145 (0.0010) [2023-12-26 20:17:29,317][105620] Updated weights for policy 1, policy_version 677155 (0.0011) [2023-12-26 20:17:29,378][105620] Updated weights for policy 1, policy_version 677165 (0.0010) [2023-12-26 20:17:29,440][105620] Updated weights for policy 1, policy_version 677175 (0.0005) [2023-12-26 20:17:29,864][105692] Updated weights for policy 0, policy_version 676297 (0.0011) [2023-12-26 20:17:29,916][105692] Updated weights for policy 0, policy_version 676307 (0.0008) [2023-12-26 20:17:29,976][105692] Updated weights for policy 0, policy_version 676317 (0.0008) [2023-12-26 20:17:30,022][105692] Updated weights for policy 0, policy_version 676327 (0.0008) [2023-12-26 20:17:30,137][105620] Updated weights for policy 1, policy_version 677185 (0.0007) [2023-12-26 20:17:30,198][105620] Updated weights for policy 1, policy_version 677195 (0.0005) [2023-12-26 20:17:30,269][105620] Updated weights for policy 1, policy_version 677205 (0.0005) [2023-12-26 20:17:30,799][105620] Updated weights for policy 1, policy_version 677215 (0.0009) [2023-12-26 20:17:30,859][105620] Updated weights for policy 1, policy_version 677225 (0.0010) [2023-12-26 20:17:30,877][105692] Updated weights for policy 0, policy_version 676337 (0.0006) [2023-12-26 20:17:30,917][105620] Updated weights for policy 1, policy_version 677235 (0.0010) [2023-12-26 20:17:30,923][105692] Updated weights for policy 0, policy_version 676347 (0.0007) [2023-12-26 20:17:30,977][105692] Updated weights for policy 0, policy_version 676357 (0.0007) [2023-12-26 20:17:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 346570752. Throughput: 0: 9497.8, 1: 9854.1. Samples: 346531152. Policy #0 lag: (min: 31.0, avg: 31.5, max: 49.0) [2023-12-26 20:17:31,062][104569] Avg episode reward: [(0, '8728.909'), (1, '9002.148')] [2023-12-26 20:17:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000676360_173178880.pth... [2023-12-26 20:17:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000677240_173391872.pth... [2023-12-26 20:17:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000675272_172900352.pth [2023-12-26 20:17:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000676056_173088768.pth [2023-12-26 20:17:31,650][105620] Updated weights for policy 1, policy_version 677245 (0.0009) [2023-12-26 20:17:31,707][105620] Updated weights for policy 1, policy_version 677255 (0.0011) [2023-12-26 20:17:31,766][105620] Updated weights for policy 1, policy_version 677265 (0.0012) [2023-12-26 20:17:31,782][105692] Updated weights for policy 0, policy_version 676367 (0.0007) [2023-12-26 20:17:31,828][105692] Updated weights for policy 0, policy_version 676377 (0.0005) [2023-12-26 20:17:31,881][105692] Updated weights for policy 0, policy_version 676387 (0.0005) [2023-12-26 20:17:32,517][105620] Updated weights for policy 1, policy_version 677275 (0.0011) [2023-12-26 20:17:32,526][105692] Updated weights for policy 0, policy_version 676397 (0.0007) [2023-12-26 20:17:32,572][105692] Updated weights for policy 0, policy_version 676407 (0.0006) [2023-12-26 20:17:32,573][105620] Updated weights for policy 1, policy_version 677285 (0.0010) [2023-12-26 20:17:32,619][105692] Updated weights for policy 0, policy_version 676417 (0.0006) [2023-12-26 20:17:32,621][105620] Updated weights for policy 1, policy_version 677295 (0.0010) [2023-12-26 20:17:33,253][105692] Updated weights for policy 0, policy_version 676427 (0.0005) [2023-12-26 20:17:33,301][105692] Updated weights for policy 0, policy_version 676437 (0.0005) [2023-12-26 20:17:33,349][105692] Updated weights for policy 0, policy_version 676447 (0.0005) [2023-12-26 20:17:33,374][105620] Updated weights for policy 1, policy_version 677305 (0.0010) [2023-12-26 20:17:33,418][105620] Updated weights for policy 1, policy_version 677315 (0.0010) [2023-12-26 20:17:33,469][105620] Updated weights for policy 1, policy_version 677325 (0.0010) [2023-12-26 20:17:33,524][105620] Updated weights for policy 1, policy_version 677335 (0.0010) [2023-12-26 20:17:34,028][105692] Updated weights for policy 0, policy_version 676457 (0.0006) [2023-12-26 20:17:34,076][105692] Updated weights for policy 0, policy_version 676467 (0.0010) [2023-12-26 20:17:34,120][105692] Updated weights for policy 0, policy_version 676477 (0.0010) [2023-12-26 20:17:34,186][105692] Updated weights for policy 0, policy_version 676487 (0.0011) [2023-12-26 20:17:34,211][105620] Updated weights for policy 1, policy_version 677345 (0.0006) [2023-12-26 20:17:34,282][105620] Updated weights for policy 1, policy_version 677355 (0.0006) [2023-12-26 20:17:34,350][105620] Updated weights for policy 1, policy_version 677365 (0.0006) [2023-12-26 20:17:34,871][105692] Updated weights for policy 0, policy_version 676497 (0.0008) [2023-12-26 20:17:34,918][105692] Updated weights for policy 0, policy_version 676507 (0.0007) [2023-12-26 20:17:34,969][105692] Updated weights for policy 0, policy_version 676517 (0.0008) [2023-12-26 20:17:35,037][105620] Updated weights for policy 1, policy_version 677375 (0.0006) [2023-12-26 20:17:35,103][105620] Updated weights for policy 1, policy_version 677385 (0.0005) [2023-12-26 20:17:35,171][105620] Updated weights for policy 1, policy_version 677395 (0.0005) [2023-12-26 20:17:35,657][105692] Updated weights for policy 0, policy_version 676527 (0.0010) [2023-12-26 20:17:35,715][105692] Updated weights for policy 0, policy_version 676537 (0.0010) [2023-12-26 20:17:35,765][105620] Updated weights for policy 1, policy_version 677405 (0.0008) [2023-12-26 20:17:35,766][105692] Updated weights for policy 0, policy_version 676547 (0.0010) [2023-12-26 20:17:35,821][105620] Updated weights for policy 1, policy_version 677415 (0.0010) [2023-12-26 20:17:35,879][105620] Updated weights for policy 1, policy_version 677425 (0.0010) [2023-12-26 20:17:36,062][104569] Fps is (10 sec: 21298.9, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 346669056. Throughput: 0: 9419.1, 1: 9880.8. Samples: 346652276. Policy #0 lag: (min: 31.0, avg: 31.5, max: 49.0) [2023-12-26 20:17:36,063][104569] Avg episode reward: [(0, '8817.554'), (1, '8910.264')] [2023-12-26 20:17:36,449][105692] Updated weights for policy 0, policy_version 676557 (0.0008) [2023-12-26 20:17:36,519][105692] Updated weights for policy 0, policy_version 676567 (0.0006) [2023-12-26 20:17:36,520][105620] Updated weights for policy 1, policy_version 677435 (0.0010) [2023-12-26 20:17:36,578][105692] Updated weights for policy 0, policy_version 676577 (0.0006) [2023-12-26 20:17:36,581][105620] Updated weights for policy 1, policy_version 677445 (0.0011) [2023-12-26 20:17:36,639][105620] Updated weights for policy 1, policy_version 677455 (0.0011) [2023-12-26 20:17:37,232][105620] Updated weights for policy 1, policy_version 677465 (0.0010) [2023-12-26 20:17:37,235][105692] Updated weights for policy 0, policy_version 676587 (0.0008) [2023-12-26 20:17:37,282][105620] Updated weights for policy 1, policy_version 677475 (0.0005) [2023-12-26 20:17:37,283][105692] Updated weights for policy 0, policy_version 676597 (0.0010) [2023-12-26 20:17:37,330][105692] Updated weights for policy 0, policy_version 676607 (0.0010) [2023-12-26 20:17:37,336][105620] Updated weights for policy 1, policy_version 677485 (0.0006) [2023-12-26 20:17:37,396][105620] Updated weights for policy 1, policy_version 677495 (0.0008) [2023-12-26 20:17:38,001][105620] Updated weights for policy 1, policy_version 677505 (0.0006) [2023-12-26 20:17:38,056][105620] Updated weights for policy 1, policy_version 677515 (0.0005) [2023-12-26 20:17:38,099][105692] Updated weights for policy 0, policy_version 676617 (0.0010) [2023-12-26 20:17:38,114][105620] Updated weights for policy 1, policy_version 677525 (0.0006) [2023-12-26 20:17:38,156][105692] Updated weights for policy 0, policy_version 676627 (0.0009) [2023-12-26 20:17:38,215][105692] Updated weights for policy 0, policy_version 676637 (0.0010) [2023-12-26 20:17:38,271][105692] Updated weights for policy 0, policy_version 676647 (0.0011) [2023-12-26 20:17:38,797][105620] Updated weights for policy 1, policy_version 677535 (0.0009) [2023-12-26 20:17:38,853][105620] Updated weights for policy 1, policy_version 677545 (0.0010) [2023-12-26 20:17:38,905][105620] Updated weights for policy 1, policy_version 677555 (0.0010) [2023-12-26 20:17:38,911][105692] Updated weights for policy 0, policy_version 676657 (0.0006) [2023-12-26 20:17:38,970][105692] Updated weights for policy 0, policy_version 676667 (0.0008) [2023-12-26 20:17:39,036][105692] Updated weights for policy 0, policy_version 676677 (0.0009) [2023-12-26 20:17:39,593][105620] Updated weights for policy 1, policy_version 677565 (0.0010) [2023-12-26 20:17:39,645][105620] Updated weights for policy 1, policy_version 677575 (0.0010) [2023-12-26 20:17:39,690][105620] Updated weights for policy 1, policy_version 677585 (0.0010) [2023-12-26 20:17:39,883][105692] Updated weights for policy 0, policy_version 676687 (0.0009) [2023-12-26 20:17:39,951][105692] Updated weights for policy 0, policy_version 676697 (0.0009) [2023-12-26 20:17:40,016][105692] Updated weights for policy 0, policy_version 676707 (0.0009) [2023-12-26 20:17:40,453][105620] Updated weights for policy 1, policy_version 677595 (0.0010) [2023-12-26 20:17:40,515][105620] Updated weights for policy 1, policy_version 677605 (0.0008) [2023-12-26 20:17:40,579][105620] Updated weights for policy 1, policy_version 677615 (0.0008) [2023-12-26 20:17:40,771][105692] Updated weights for policy 0, policy_version 676717 (0.0009) [2023-12-26 20:17:40,817][105692] Updated weights for policy 0, policy_version 676727 (0.0010) [2023-12-26 20:17:40,862][105692] Updated weights for policy 0, policy_version 676737 (0.0010) [2023-12-26 20:17:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 346767360. Throughput: 0: 9429.4, 1: 9949.3. Samples: 346773596. Policy #0 lag: (min: 31.0, avg: 31.5, max: 49.0) [2023-12-26 20:17:41,062][104569] Avg episode reward: [(0, '8550.569'), (1, '9077.080')] [2023-12-26 20:17:41,353][105620] Updated weights for policy 1, policy_version 677625 (0.0008) [2023-12-26 20:17:41,422][105620] Updated weights for policy 1, policy_version 677635 (0.0007) [2023-12-26 20:17:41,484][105620] Updated weights for policy 1, policy_version 677645 (0.0008) [2023-12-26 20:17:41,538][105620] Updated weights for policy 1, policy_version 677655 (0.0009) [2023-12-26 20:17:41,604][105692] Updated weights for policy 0, policy_version 676747 (0.0008) [2023-12-26 20:17:41,671][105692] Updated weights for policy 0, policy_version 676757 (0.0009) [2023-12-26 20:17:41,734][105692] Updated weights for policy 0, policy_version 676767 (0.0008) [2023-12-26 20:17:42,398][105620] Updated weights for policy 1, policy_version 677665 (0.0009) [2023-12-26 20:17:42,459][105620] Updated weights for policy 1, policy_version 677675 (0.0008) [2023-12-26 20:17:42,477][105692] Updated weights for policy 0, policy_version 676777 (0.0009) [2023-12-26 20:17:42,513][105620] Updated weights for policy 1, policy_version 677685 (0.0008) [2023-12-26 20:17:42,536][105692] Updated weights for policy 0, policy_version 676787 (0.0009) [2023-12-26 20:17:42,619][105692] Updated weights for policy 0, policy_version 676797 (0.0009) [2023-12-26 20:17:42,685][105692] Updated weights for policy 0, policy_version 676807 (0.0009) [2023-12-26 20:17:43,186][105620] Updated weights for policy 1, policy_version 677695 (0.0006) [2023-12-26 20:17:43,254][105620] Updated weights for policy 1, policy_version 677705 (0.0010) [2023-12-26 20:17:43,315][105620] Updated weights for policy 1, policy_version 677715 (0.0010) [2023-12-26 20:17:43,440][105692] Updated weights for policy 0, policy_version 676817 (0.0006) [2023-12-26 20:17:43,493][105692] Updated weights for policy 0, policy_version 676827 (0.0005) [2023-12-26 20:17:43,548][105692] Updated weights for policy 0, policy_version 676837 (0.0005) [2023-12-26 20:17:43,982][105620] Updated weights for policy 1, policy_version 677725 (0.0010) [2023-12-26 20:17:44,029][105620] Updated weights for policy 1, policy_version 677735 (0.0009) [2023-12-26 20:17:44,081][105620] Updated weights for policy 1, policy_version 677745 (0.0008) [2023-12-26 20:17:44,130][105692] Updated weights for policy 0, policy_version 676847 (0.0006) [2023-12-26 20:17:44,195][105692] Updated weights for policy 0, policy_version 676857 (0.0009) [2023-12-26 20:17:44,255][105692] Updated weights for policy 0, policy_version 676867 (0.0009) [2023-12-26 20:17:44,888][105620] Updated weights for policy 1, policy_version 677755 (0.0007) [2023-12-26 20:17:44,900][105692] Updated weights for policy 0, policy_version 676877 (0.0009) [2023-12-26 20:17:44,934][105620] Updated weights for policy 1, policy_version 677765 (0.0006) [2023-12-26 20:17:44,949][105692] Updated weights for policy 0, policy_version 676887 (0.0007) [2023-12-26 20:17:44,985][105620] Updated weights for policy 1, policy_version 677775 (0.0007) [2023-12-26 20:17:45,000][105692] Updated weights for policy 0, policy_version 676897 (0.0006) [2023-12-26 20:17:45,729][105620] Updated weights for policy 1, policy_version 677785 (0.0008) [2023-12-26 20:17:45,783][105620] Updated weights for policy 1, policy_version 677795 (0.0009) [2023-12-26 20:17:45,790][105692] Updated weights for policy 0, policy_version 676907 (0.0007) [2023-12-26 20:17:45,832][105620] Updated weights for policy 1, policy_version 677805 (0.0007) [2023-12-26 20:17:45,846][105692] Updated weights for policy 0, policy_version 676917 (0.0006) [2023-12-26 20:17:45,895][105620] Updated weights for policy 1, policy_version 677815 (0.0008) [2023-12-26 20:17:45,895][105692] Updated weights for policy 0, policy_version 676927 (0.0006) [2023-12-26 20:17:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 346865664. Throughput: 0: 9402.5, 1: 9982.5. Samples: 346830728. Policy #0 lag: (min: 31.0, avg: 31.5, max: 49.0) [2023-12-26 20:17:46,063][104569] Avg episode reward: [(0, '8538.902'), (1, '9170.012')] [2023-12-26 20:17:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000676936_173326336.pth... [2023-12-26 20:17:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000677816_173539328.pth... [2023-12-26 20:17:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000675784_173031424.pth [2023-12-26 20:17:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000676664_173244416.pth [2023-12-26 20:17:46,594][105620] Updated weights for policy 1, policy_version 677825 (0.0006) [2023-12-26 20:17:46,632][105692] Updated weights for policy 0, policy_version 676937 (0.0009) [2023-12-26 20:17:46,647][105620] Updated weights for policy 1, policy_version 677835 (0.0007) [2023-12-26 20:17:46,691][105692] Updated weights for policy 0, policy_version 676947 (0.0006) [2023-12-26 20:17:46,697][105620] Updated weights for policy 1, policy_version 677845 (0.0007) [2023-12-26 20:17:46,746][105692] Updated weights for policy 0, policy_version 676957 (0.0008) [2023-12-26 20:17:46,798][105692] Updated weights for policy 0, policy_version 676967 (0.0009) [2023-12-26 20:17:47,367][105620] Updated weights for policy 1, policy_version 677855 (0.0005) [2023-12-26 20:17:47,412][105620] Updated weights for policy 1, policy_version 677865 (0.0005) [2023-12-26 20:17:47,457][105620] Updated weights for policy 1, policy_version 677875 (0.0005) [2023-12-26 20:17:47,632][105692] Updated weights for policy 0, policy_version 676977 (0.0009) [2023-12-26 20:17:47,694][105692] Updated weights for policy 0, policy_version 676987 (0.0009) [2023-12-26 20:17:47,703][105585] KL-divergence is very high: 157.8305 [2023-12-26 20:17:47,753][105585] KL-divergence is very high: 212.8731 [2023-12-26 20:17:47,758][105692] Updated weights for policy 0, policy_version 676997 (0.0009) [2023-12-26 20:17:48,117][105620] Updated weights for policy 1, policy_version 677885 (0.0007) [2023-12-26 20:17:48,171][105620] Updated weights for policy 1, policy_version 677895 (0.0009) [2023-12-26 20:17:48,231][105620] Updated weights for policy 1, policy_version 677905 (0.0006) [2023-12-26 20:17:48,588][105692] Updated weights for policy 0, policy_version 677007 (0.0010) [2023-12-26 20:17:48,654][105692] Updated weights for policy 0, policy_version 677017 (0.0009) [2023-12-26 20:17:48,708][105692] Updated weights for policy 0, policy_version 677027 (0.0010) [2023-12-26 20:17:48,823][105620] Updated weights for policy 1, policy_version 677915 (0.0007) [2023-12-26 20:17:48,887][105620] Updated weights for policy 1, policy_version 677925 (0.0009) [2023-12-26 20:17:48,945][105620] Updated weights for policy 1, policy_version 677935 (0.0009) [2023-12-26 20:17:49,506][105692] Updated weights for policy 0, policy_version 677037 (0.0009) [2023-12-26 20:17:49,561][105692] Updated weights for policy 0, policy_version 677047 (0.0009) [2023-12-26 20:17:49,618][105692] Updated weights for policy 0, policy_version 677057 (0.0010) [2023-12-26 20:17:49,635][105620] Updated weights for policy 1, policy_version 677945 (0.0009) [2023-12-26 20:17:49,699][105620] Updated weights for policy 1, policy_version 677955 (0.0006) [2023-12-26 20:17:49,758][105620] Updated weights for policy 1, policy_version 677965 (0.0009) [2023-12-26 20:17:49,805][105620] Updated weights for policy 1, policy_version 677975 (0.0008) [2023-12-26 20:17:50,412][105692] Updated weights for policy 0, policy_version 677067 (0.0009) [2023-12-26 20:17:50,460][105692] Updated weights for policy 0, policy_version 677077 (0.0009) [2023-12-26 20:17:50,511][105692] Updated weights for policy 0, policy_version 677087 (0.0006) [2023-12-26 20:17:50,549][105620] Updated weights for policy 1, policy_version 677985 (0.0008) [2023-12-26 20:17:50,612][105620] Updated weights for policy 1, policy_version 677995 (0.0008) [2023-12-26 20:17:50,667][105620] Updated weights for policy 1, policy_version 678006 (0.0010) [2023-12-26 20:17:51,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19387.6, 300 sec: 19494.2). Total num frames: 346955776. Throughput: 0: 9426.9, 1: 10003.5. Samples: 346946720. Policy #0 lag: (min: 31.0, avg: 31.5, max: 49.0) [2023-12-26 20:17:51,063][104569] Avg episode reward: [(0, '8806.742'), (1, '8988.466')] [2023-12-26 20:17:51,293][105692] Updated weights for policy 0, policy_version 677097 (0.0006) [2023-12-26 20:17:51,364][105692] Updated weights for policy 0, policy_version 677107 (0.0009) [2023-12-26 20:17:51,428][105692] Updated weights for policy 0, policy_version 677117 (0.0008) [2023-12-26 20:17:51,461][105620] Updated weights for policy 1, policy_version 678016 (0.0009) [2023-12-26 20:17:51,486][105692] Updated weights for policy 0, policy_version 677127 (0.0006) [2023-12-26 20:17:51,526][105620] Updated weights for policy 1, policy_version 678026 (0.0007) [2023-12-26 20:17:51,592][105620] Updated weights for policy 1, policy_version 678036 (0.0009) [2023-12-26 20:17:52,178][105692] Updated weights for policy 0, policy_version 677137 (0.0008) [2023-12-26 20:17:52,238][105692] Updated weights for policy 0, policy_version 677147 (0.0008) [2023-12-26 20:17:52,306][105692] Updated weights for policy 0, policy_version 677157 (0.0009) [2023-12-26 20:17:52,339][105620] Updated weights for policy 1, policy_version 678046 (0.0010) [2023-12-26 20:17:52,409][105620] Updated weights for policy 1, policy_version 678056 (0.0011) [2023-12-26 20:17:52,466][105620] Updated weights for policy 1, policy_version 678066 (0.0011) [2023-12-26 20:17:53,095][105692] Updated weights for policy 0, policy_version 677167 (0.0008) [2023-12-26 20:17:53,142][105692] Updated weights for policy 0, policy_version 677177 (0.0008) [2023-12-26 20:17:53,193][105692] Updated weights for policy 0, policy_version 677187 (0.0008) [2023-12-26 20:17:53,224][105620] Updated weights for policy 1, policy_version 678076 (0.0011) [2023-12-26 20:17:53,282][105620] Updated weights for policy 1, policy_version 678086 (0.0010) [2023-12-26 20:17:53,347][105620] Updated weights for policy 1, policy_version 678096 (0.0010) [2023-12-26 20:17:53,958][105692] Updated weights for policy 0, policy_version 677197 (0.0007) [2023-12-26 20:17:54,020][105692] Updated weights for policy 0, policy_version 677207 (0.0008) [2023-12-26 20:17:54,078][105692] Updated weights for policy 0, policy_version 677217 (0.0008) [2023-12-26 20:17:54,081][105620] Updated weights for policy 1, policy_version 678106 (0.0011) [2023-12-26 20:17:54,133][105620] Updated weights for policy 1, policy_version 678116 (0.0010) [2023-12-26 20:17:54,192][105620] Updated weights for policy 1, policy_version 678126 (0.0011) [2023-12-26 20:17:54,248][105620] Updated weights for policy 1, policy_version 678136 (0.0010) [2023-12-26 20:17:54,833][105692] Updated weights for policy 0, policy_version 677227 (0.0008) [2023-12-26 20:17:54,895][105692] Updated weights for policy 0, policy_version 677237 (0.0009) [2023-12-26 20:17:54,904][105620] Updated weights for policy 1, policy_version 678146 (0.0006) [2023-12-26 20:17:54,952][105692] Updated weights for policy 0, policy_version 677247 (0.0011) [2023-12-26 20:17:54,960][105620] Updated weights for policy 1, policy_version 678156 (0.0007) [2023-12-26 20:17:55,016][105620] Updated weights for policy 1, policy_version 678166 (0.0008) [2023-12-26 20:17:55,615][105692] Updated weights for policy 0, policy_version 677257 (0.0010) [2023-12-26 20:17:55,671][105692] Updated weights for policy 0, policy_version 677267 (0.0005) [2023-12-26 20:17:55,722][105692] Updated weights for policy 0, policy_version 677277 (0.0005) [2023-12-26 20:17:55,729][105620] Updated weights for policy 1, policy_version 678176 (0.0006) [2023-12-26 20:17:55,776][105692] Updated weights for policy 0, policy_version 677287 (0.0007) [2023-12-26 20:17:55,780][105620] Updated weights for policy 1, policy_version 678186 (0.0005) [2023-12-26 20:17:55,828][105620] Updated weights for policy 1, policy_version 678196 (0.0007) [2023-12-26 20:17:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 347054080. Throughput: 0: 9539.3, 1: 9862.7. Samples: 347060720. Policy #0 lag: (min: 31.0, avg: 31.5, max: 49.0) [2023-12-26 20:17:56,063][104569] Avg episode reward: [(0, '9169.325'), (1, '9078.774')] [2023-12-26 20:17:56,409][105620] Updated weights for policy 1, policy_version 678206 (0.0006) [2023-12-26 20:17:56,442][105692] Updated weights for policy 0, policy_version 677297 (0.0011) [2023-12-26 20:17:56,477][105620] Updated weights for policy 1, policy_version 678216 (0.0005) [2023-12-26 20:17:56,508][105692] Updated weights for policy 0, policy_version 677307 (0.0005) [2023-12-26 20:17:56,536][105620] Updated weights for policy 1, policy_version 678226 (0.0005) [2023-12-26 20:17:56,564][105692] Updated weights for policy 0, policy_version 677317 (0.0005) [2023-12-26 20:17:57,114][105620] Updated weights for policy 1, policy_version 678236 (0.0007) [2023-12-26 20:17:57,118][105692] Updated weights for policy 0, policy_version 677327 (0.0005) [2023-12-26 20:17:57,170][105692] Updated weights for policy 0, policy_version 677337 (0.0006) [2023-12-26 20:17:57,175][105620] Updated weights for policy 1, policy_version 678246 (0.0010) [2023-12-26 20:17:57,232][105692] Updated weights for policy 0, policy_version 677347 (0.0006) [2023-12-26 20:17:57,241][105620] Updated weights for policy 1, policy_version 678256 (0.0008) [2023-12-26 20:17:57,795][105692] Updated weights for policy 0, policy_version 677357 (0.0007) [2023-12-26 20:17:57,845][105692] Updated weights for policy 0, policy_version 677367 (0.0007) [2023-12-26 20:17:57,890][105692] Updated weights for policy 0, policy_version 677377 (0.0005) [2023-12-26 20:17:57,934][105620] Updated weights for policy 1, policy_version 678266 (0.0009) [2023-12-26 20:17:57,992][105620] Updated weights for policy 1, policy_version 678276 (0.0010) [2023-12-26 20:17:58,050][105620] Updated weights for policy 1, policy_version 678286 (0.0010) [2023-12-26 20:17:58,108][105620] Updated weights for policy 1, policy_version 678296 (0.0010) [2023-12-26 20:17:58,570][105692] Updated weights for policy 0, policy_version 677387 (0.0007) [2023-12-26 20:17:58,633][105692] Updated weights for policy 0, policy_version 677397 (0.0011) [2023-12-26 20:17:58,708][105692] Updated weights for policy 0, policy_version 677407 (0.0012) [2023-12-26 20:17:58,851][105620] Updated weights for policy 1, policy_version 678306 (0.0007) [2023-12-26 20:17:58,922][105620] Updated weights for policy 1, policy_version 678316 (0.0008) [2023-12-26 20:17:58,970][105620] Updated weights for policy 1, policy_version 678326 (0.0009) [2023-12-26 20:17:59,568][105692] Updated weights for policy 0, policy_version 677417 (0.0011) [2023-12-26 20:17:59,630][105692] Updated weights for policy 0, policy_version 677427 (0.0009) [2023-12-26 20:17:59,680][105692] Updated weights for policy 0, policy_version 677437 (0.0009) [2023-12-26 20:17:59,732][105692] Updated weights for policy 0, policy_version 677447 (0.0010) [2023-12-26 20:17:59,767][105620] Updated weights for policy 1, policy_version 678336 (0.0008) [2023-12-26 20:17:59,818][105620] Updated weights for policy 1, policy_version 678346 (0.0008) [2023-12-26 20:17:59,877][105620] Updated weights for policy 1, policy_version 678356 (0.0009) [2023-12-26 20:18:00,378][105692] Updated weights for policy 0, policy_version 677457 (0.0008) [2023-12-26 20:18:00,437][105692] Updated weights for policy 0, policy_version 677467 (0.0011) [2023-12-26 20:18:00,485][105692] Updated weights for policy 0, policy_version 677477 (0.0010) [2023-12-26 20:18:00,739][105620] Updated weights for policy 1, policy_version 678366 (0.0008) [2023-12-26 20:18:00,795][105620] Updated weights for policy 1, policy_version 678376 (0.0009) [2023-12-26 20:18:00,848][105620] Updated weights for policy 1, policy_version 678386 (0.0009) [2023-12-26 20:18:01,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 347152384. Throughput: 0: 9642.8, 1: 9924.3. Samples: 347124976. Policy #0 lag: (min: 31.0, avg: 31.5, max: 49.0) [2023-12-26 20:18:01,062][104569] Avg episode reward: [(0, '9260.952'), (1, '9259.175')] [2023-12-26 20:18:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000678392_173686784.pth... [2023-12-26 20:18:01,067][105692] Updated weights for policy 0, policy_version 677487 (0.0009) [2023-12-26 20:18:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000677240_173391872.pth [2023-12-26 20:18:01,132][105692] Updated weights for policy 0, policy_version 677497 (0.0008) [2023-12-26 20:18:01,190][105692] Updated weights for policy 0, policy_version 677507 (0.0008) [2023-12-26 20:18:01,222][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000677512_173473792.pth... [2023-12-26 20:18:01,227][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000676360_173178880.pth [2023-12-26 20:18:01,648][105620] Updated weights for policy 1, policy_version 678396 (0.0009) [2023-12-26 20:18:01,712][105620] Updated weights for policy 1, policy_version 678406 (0.0008) [2023-12-26 20:18:01,778][105620] Updated weights for policy 1, policy_version 678416 (0.0007) [2023-12-26 20:18:01,919][105692] Updated weights for policy 0, policy_version 677517 (0.0007) [2023-12-26 20:18:01,984][105692] Updated weights for policy 0, policy_version 677527 (0.0006) [2023-12-26 20:18:02,039][105692] Updated weights for policy 0, policy_version 677537 (0.0006) [2023-12-26 20:18:02,554][105620] Updated weights for policy 1, policy_version 678426 (0.0009) [2023-12-26 20:18:02,618][105620] Updated weights for policy 1, policy_version 678436 (0.0009) [2023-12-26 20:18:02,641][105692] Updated weights for policy 0, policy_version 677547 (0.0009) [2023-12-26 20:18:02,671][105620] Updated weights for policy 1, policy_version 678446 (0.0006) [2023-12-26 20:18:02,696][105692] Updated weights for policy 0, policy_version 677557 (0.0009) [2023-12-26 20:18:02,720][105620] Updated weights for policy 1, policy_version 678456 (0.0010) [2023-12-26 20:18:02,742][105692] Updated weights for policy 0, policy_version 677567 (0.0009) [2023-12-26 20:18:03,348][105692] Updated weights for policy 0, policy_version 677577 (0.0006) [2023-12-26 20:18:03,408][105692] Updated weights for policy 0, policy_version 677587 (0.0010) [2023-12-26 20:18:03,465][105692] Updated weights for policy 0, policy_version 677597 (0.0010) [2023-12-26 20:18:03,523][105692] Updated weights for policy 0, policy_version 677607 (0.0010) [2023-12-26 20:18:03,557][105620] Updated weights for policy 1, policy_version 678466 (0.0007) [2023-12-26 20:18:03,601][105620] Updated weights for policy 1, policy_version 678476 (0.0008) [2023-12-26 20:18:03,653][105620] Updated weights for policy 1, policy_version 678486 (0.0007) [2023-12-26 20:18:04,168][105692] Updated weights for policy 0, policy_version 677617 (0.0006) [2023-12-26 20:18:04,227][105692] Updated weights for policy 0, policy_version 677627 (0.0007) [2023-12-26 20:18:04,279][105692] Updated weights for policy 0, policy_version 677637 (0.0009) [2023-12-26 20:18:04,484][105620] Updated weights for policy 1, policy_version 678496 (0.0006) [2023-12-26 20:18:04,550][105620] Updated weights for policy 1, policy_version 678506 (0.0006) [2023-12-26 20:18:04,611][105620] Updated weights for policy 1, policy_version 678516 (0.0007) [2023-12-26 20:18:05,011][105692] Updated weights for policy 0, policy_version 677647 (0.0007) [2023-12-26 20:18:05,064][105692] Updated weights for policy 0, policy_version 677657 (0.0005) [2023-12-26 20:18:05,111][105692] Updated weights for policy 0, policy_version 677667 (0.0005) [2023-12-26 20:18:05,174][105620] Updated weights for policy 1, policy_version 678526 (0.0005) [2023-12-26 20:18:05,228][105620] Updated weights for policy 1, policy_version 678536 (0.0008) [2023-12-26 20:18:05,287][105620] Updated weights for policy 1, policy_version 678546 (0.0009) [2023-12-26 20:18:05,693][105692] Updated weights for policy 0, policy_version 677677 (0.0007) [2023-12-26 20:18:05,739][105692] Updated weights for policy 0, policy_version 677687 (0.0005) [2023-12-26 20:18:05,802][105692] Updated weights for policy 0, policy_version 677697 (0.0005) [2023-12-26 20:18:05,965][105620] Updated weights for policy 1, policy_version 678556 (0.0007) [2023-12-26 20:18:06,014][105620] Updated weights for policy 1, policy_version 678566 (0.0005) [2023-12-26 20:18:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 347250688. Throughput: 0: 9810.9, 1: 9775.8. Samples: 347239768. Policy #0 lag: (min: 31.0, avg: 31.5, max: 49.0) [2023-12-26 20:18:06,063][104569] Avg episode reward: [(0, '8987.575'), (1, '9166.724')] [2023-12-26 20:18:06,071][105620] Updated weights for policy 1, policy_version 678576 (0.0005) [2023-12-26 20:18:06,407][105692] Updated weights for policy 0, policy_version 677707 (0.0007) [2023-12-26 20:18:06,463][105692] Updated weights for policy 0, policy_version 677717 (0.0010) [2023-12-26 20:18:06,515][105692] Updated weights for policy 0, policy_version 677727 (0.0010) [2023-12-26 20:18:06,843][105620] Updated weights for policy 1, policy_version 678586 (0.0007) [2023-12-26 20:18:06,910][105620] Updated weights for policy 1, policy_version 678596 (0.0008) [2023-12-26 20:18:06,966][105620] Updated weights for policy 1, policy_version 678606 (0.0008) [2023-12-26 20:18:07,014][105620] Updated weights for policy 1, policy_version 678616 (0.0008) [2023-12-26 20:18:07,160][105692] Updated weights for policy 0, policy_version 677737 (0.0006) [2023-12-26 20:18:07,222][105692] Updated weights for policy 0, policy_version 677747 (0.0008) [2023-12-26 20:18:07,285][105692] Updated weights for policy 0, policy_version 677757 (0.0008) [2023-12-26 20:18:07,334][105692] Updated weights for policy 0, policy_version 677767 (0.0005) [2023-12-26 20:18:07,716][105620] Updated weights for policy 1, policy_version 678626 (0.0008) [2023-12-26 20:18:07,769][105620] Updated weights for policy 1, policy_version 678636 (0.0008) [2023-12-26 20:18:07,814][105620] Updated weights for policy 1, policy_version 678646 (0.0008) [2023-12-26 20:18:07,990][105692] Updated weights for policy 0, policy_version 677777 (0.0010) [2023-12-26 20:18:08,041][105692] Updated weights for policy 0, policy_version 677787 (0.0010) [2023-12-26 20:18:08,089][105692] Updated weights for policy 0, policy_version 677797 (0.0010) [2023-12-26 20:18:08,403][105620] Updated weights for policy 1, policy_version 678656 (0.0008) [2023-12-26 20:18:08,466][105620] Updated weights for policy 1, policy_version 678666 (0.0008) [2023-12-26 20:18:08,525][105620] Updated weights for policy 1, policy_version 678676 (0.0008) [2023-12-26 20:18:08,792][105692] Updated weights for policy 0, policy_version 677807 (0.0010) [2023-12-26 20:18:08,854][105692] Updated weights for policy 0, policy_version 677817 (0.0010) [2023-12-26 20:18:08,913][105692] Updated weights for policy 0, policy_version 677827 (0.0010) [2023-12-26 20:18:09,189][105620] Updated weights for policy 1, policy_version 678686 (0.0008) [2023-12-26 20:18:09,255][105620] Updated weights for policy 1, policy_version 678696 (0.0008) [2023-12-26 20:18:09,307][105620] Updated weights for policy 1, policy_version 678706 (0.0008) [2023-12-26 20:18:09,630][105692] Updated weights for policy 0, policy_version 677837 (0.0010) [2023-12-26 20:18:09,682][105692] Updated weights for policy 0, policy_version 677847 (0.0011) [2023-12-26 20:18:09,745][105692] Updated weights for policy 0, policy_version 677857 (0.0010) [2023-12-26 20:18:10,033][105620] Updated weights for policy 1, policy_version 678716 (0.0008) [2023-12-26 20:18:10,088][105620] Updated weights for policy 1, policy_version 678726 (0.0008) [2023-12-26 20:18:10,148][105620] Updated weights for policy 1, policy_version 678736 (0.0007) [2023-12-26 20:18:10,501][105692] Updated weights for policy 0, policy_version 677867 (0.0010) [2023-12-26 20:18:10,566][105692] Updated weights for policy 0, policy_version 677877 (0.0009) [2023-12-26 20:18:10,625][105692] Updated weights for policy 0, policy_version 677887 (0.0006) [2023-12-26 20:18:10,943][105620] Updated weights for policy 1, policy_version 678746 (0.0009) [2023-12-26 20:18:11,004][105620] Updated weights for policy 1, policy_version 678756 (0.0008) [2023-12-26 20:18:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 347348992. Throughput: 0: 9936.8, 1: 9874.7. Samples: 347361556. Policy #0 lag: (min: 31.0, avg: 31.5, max: 49.0) [2023-12-26 20:18:11,062][105620] Updated weights for policy 1, policy_version 678766 (0.0009) [2023-12-26 20:18:11,062][104569] Avg episode reward: [(0, '8988.228'), (1, '9170.204')] [2023-12-26 20:18:11,118][105620] Updated weights for policy 1, policy_version 678776 (0.0008) [2023-12-26 20:18:11,253][105692] Updated weights for policy 0, policy_version 677897 (0.0006) [2023-12-26 20:18:11,308][105692] Updated weights for policy 0, policy_version 677907 (0.0009) [2023-12-26 20:18:11,370][105692] Updated weights for policy 0, policy_version 677917 (0.0010) [2023-12-26 20:18:11,441][105692] Updated weights for policy 0, policy_version 677927 (0.0010) [2023-12-26 20:18:11,875][105620] Updated weights for policy 1, policy_version 678786 (0.0008) [2023-12-26 20:18:11,939][105620] Updated weights for policy 1, policy_version 678796 (0.0008) [2023-12-26 20:18:12,000][105620] Updated weights for policy 1, policy_version 678806 (0.0008) [2023-12-26 20:18:12,185][105692] Updated weights for policy 0, policy_version 677937 (0.0011) [2023-12-26 20:18:12,244][105692] Updated weights for policy 0, policy_version 677947 (0.0010) [2023-12-26 20:18:12,300][105692] Updated weights for policy 0, policy_version 677957 (0.0011) [2023-12-26 20:18:12,772][105620] Updated weights for policy 1, policy_version 678816 (0.0010) [2023-12-26 20:18:12,825][105620] Updated weights for policy 1, policy_version 678826 (0.0010) [2023-12-26 20:18:12,883][105620] Updated weights for policy 1, policy_version 678836 (0.0010) [2023-12-26 20:18:12,976][105692] Updated weights for policy 0, policy_version 677967 (0.0007) [2023-12-26 20:18:13,035][105692] Updated weights for policy 0, policy_version 677977 (0.0008) [2023-12-26 20:18:13,102][105692] Updated weights for policy 0, policy_version 677987 (0.0011) [2023-12-26 20:18:13,700][105692] Updated weights for policy 0, policy_version 677997 (0.0008) [2023-12-26 20:18:13,738][105620] Updated weights for policy 1, policy_version 678846 (0.0009) [2023-12-26 20:18:13,762][105692] Updated weights for policy 0, policy_version 678007 (0.0008) [2023-12-26 20:18:13,799][105620] Updated weights for policy 1, policy_version 678856 (0.0009) [2023-12-26 20:18:13,822][105692] Updated weights for policy 0, policy_version 678017 (0.0007) [2023-12-26 20:18:13,859][105620] Updated weights for policy 1, policy_version 678866 (0.0007) [2023-12-26 20:18:14,399][105692] Updated weights for policy 0, policy_version 678027 (0.0008) [2023-12-26 20:18:14,457][105692] Updated weights for policy 0, policy_version 678037 (0.0010) [2023-12-26 20:18:14,518][105692] Updated weights for policy 0, policy_version 678047 (0.0010) [2023-12-26 20:18:14,673][105620] Updated weights for policy 1, policy_version 678876 (0.0008) [2023-12-26 20:18:14,728][105620] Updated weights for policy 1, policy_version 678886 (0.0008) [2023-12-26 20:18:14,791][105620] Updated weights for policy 1, policy_version 678896 (0.0008) [2023-12-26 20:18:15,169][105692] Updated weights for policy 0, policy_version 678057 (0.0010) [2023-12-26 20:18:15,233][105692] Updated weights for policy 0, policy_version 678067 (0.0005) [2023-12-26 20:18:15,290][105692] Updated weights for policy 0, policy_version 678077 (0.0005) [2023-12-26 20:18:15,339][105692] Updated weights for policy 0, policy_version 678087 (0.0005) [2023-12-26 20:18:15,700][105620] Updated weights for policy 1, policy_version 678906 (0.0008) [2023-12-26 20:18:15,757][105620] Updated weights for policy 1, policy_version 678916 (0.0008) [2023-12-26 20:18:15,815][105620] Updated weights for policy 1, policy_version 678926 (0.0008) [2023-12-26 20:18:15,869][105620] Updated weights for policy 1, policy_version 678936 (0.0007) [2023-12-26 20:18:15,878][105692] Updated weights for policy 0, policy_version 678097 (0.0010) [2023-12-26 20:18:15,939][105692] Updated weights for policy 0, policy_version 678107 (0.0010) [2023-12-26 20:18:15,998][105692] Updated weights for policy 0, policy_version 678117 (0.0011) [2023-12-26 20:18:16,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.3, 300 sec: 19521.9). Total num frames: 347455488. Throughput: 0: 9933.8, 1: 9769.0. Samples: 347417784. Policy #0 lag: (min: 31.0, avg: 31.5, max: 49.0) [2023-12-26 20:18:16,063][104569] Avg episode reward: [(0, '7376.228'), (1, '9262.093')] [2023-12-26 20:18:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000678120_173629440.pth... [2023-12-26 20:18:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000678936_173826048.pth... [2023-12-26 20:18:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000676936_173326336.pth [2023-12-26 20:18:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000677816_173539328.pth [2023-12-26 20:18:16,616][105692] Updated weights for policy 0, policy_version 678127 (0.0008) [2023-12-26 20:18:16,672][105692] Updated weights for policy 0, policy_version 678137 (0.0011) [2023-12-26 20:18:16,687][105620] Updated weights for policy 1, policy_version 678946 (0.0006) [2023-12-26 20:18:16,731][105692] Updated weights for policy 0, policy_version 678147 (0.0011) [2023-12-26 20:18:16,750][105620] Updated weights for policy 1, policy_version 678956 (0.0005) [2023-12-26 20:18:16,808][105620] Updated weights for policy 1, policy_version 678966 (0.0007) [2023-12-26 20:18:17,370][105692] Updated weights for policy 0, policy_version 678157 (0.0011) [2023-12-26 20:18:17,428][105692] Updated weights for policy 0, policy_version 678167 (0.0011) [2023-12-26 20:18:17,476][105692] Updated weights for policy 0, policy_version 678177 (0.0010) [2023-12-26 20:18:17,621][105620] Updated weights for policy 1, policy_version 678976 (0.0008) [2023-12-26 20:18:17,657][105586] KL-divergence is very high: 149.5173 [2023-12-26 20:18:17,668][105620] Updated weights for policy 1, policy_version 678986 (0.0008) [2023-12-26 20:18:17,696][105586] KL-divergence is very high: 161.8149 [2023-12-26 20:18:17,716][105620] Updated weights for policy 1, policy_version 678996 (0.0008) [2023-12-26 20:18:18,226][105692] Updated weights for policy 0, policy_version 678187 (0.0011) [2023-12-26 20:18:18,291][105692] Updated weights for policy 0, policy_version 678197 (0.0010) [2023-12-26 20:18:18,356][105692] Updated weights for policy 0, policy_version 678207 (0.0009) [2023-12-26 20:18:18,519][105620] Updated weights for policy 1, policy_version 679006 (0.0010) [2023-12-26 20:18:18,588][105620] Updated weights for policy 1, policy_version 679016 (0.0010) [2023-12-26 20:18:18,654][105620] Updated weights for policy 1, policy_version 679026 (0.0011) [2023-12-26 20:18:19,064][105692] Updated weights for policy 0, policy_version 678217 (0.0010) [2023-12-26 20:18:19,122][105692] Updated weights for policy 0, policy_version 678227 (0.0005) [2023-12-26 20:18:19,180][105692] Updated weights for policy 0, policy_version 678237 (0.0010) [2023-12-26 20:18:19,237][105692] Updated weights for policy 0, policy_version 678247 (0.0010) [2023-12-26 20:18:19,387][105620] Updated weights for policy 1, policy_version 679036 (0.0010) [2023-12-26 20:18:19,435][105620] Updated weights for policy 1, policy_version 679046 (0.0007) [2023-12-26 20:18:19,498][105620] Updated weights for policy 1, policy_version 679056 (0.0006) [2023-12-26 20:18:19,949][105692] Updated weights for policy 0, policy_version 678258 (0.0009) [2023-12-26 20:18:20,007][105692] Updated weights for policy 0, policy_version 678268 (0.0009) [2023-12-26 20:18:20,057][105692] Updated weights for policy 0, policy_version 678278 (0.0005) [2023-12-26 20:18:20,376][105620] Updated weights for policy 1, policy_version 679066 (0.0007) [2023-12-26 20:18:20,441][105620] Updated weights for policy 1, policy_version 679076 (0.0008) [2023-12-26 20:18:20,502][105620] Updated weights for policy 1, policy_version 679086 (0.0008) [2023-12-26 20:18:20,576][105620] Updated weights for policy 1, policy_version 679096 (0.0008) [2023-12-26 20:18:20,690][105692] Updated weights for policy 0, policy_version 678288 (0.0010) [2023-12-26 20:18:20,758][105692] Updated weights for policy 0, policy_version 678298 (0.0011) [2023-12-26 20:18:20,818][105692] Updated weights for policy 0, policy_version 678308 (0.0011) [2023-12-26 20:18:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 347545600. Throughput: 0: 10001.5, 1: 9602.4. Samples: 347534452. Policy #0 lag: (min: 31.0, avg: 31.5, max: 49.0) [2023-12-26 20:18:21,063][104569] Avg episode reward: [(0, '7443.435'), (1, '8804.285')] [2023-12-26 20:18:21,337][105620] Updated weights for policy 1, policy_version 679106 (0.0008) [2023-12-26 20:18:21,405][105620] Updated weights for policy 1, policy_version 679116 (0.0009) [2023-12-26 20:18:21,472][105620] Updated weights for policy 1, policy_version 679126 (0.0008) [2023-12-26 20:18:21,585][105692] Updated weights for policy 0, policy_version 678318 (0.0011) [2023-12-26 20:18:21,654][105692] Updated weights for policy 0, policy_version 678328 (0.0011) [2023-12-26 20:18:21,714][105692] Updated weights for policy 0, policy_version 678338 (0.0011) [2023-12-26 20:18:22,259][105620] Updated weights for policy 1, policy_version 679136 (0.0009) [2023-12-26 20:18:22,312][105620] Updated weights for policy 1, policy_version 679146 (0.0008) [2023-12-26 20:18:22,371][105620] Updated weights for policy 1, policy_version 679156 (0.0009) [2023-12-26 20:18:22,499][105692] Updated weights for policy 0, policy_version 678348 (0.0010) [2023-12-26 20:18:22,556][105692] Updated weights for policy 0, policy_version 678358 (0.0010) [2023-12-26 20:18:22,612][105692] Updated weights for policy 0, policy_version 678368 (0.0010) [2023-12-26 20:18:23,154][105620] Updated weights for policy 1, policy_version 679166 (0.0008) [2023-12-26 20:18:23,206][105620] Updated weights for policy 1, policy_version 679176 (0.0008) [2023-12-26 20:18:23,254][105620] Updated weights for policy 1, policy_version 679186 (0.0008) [2023-12-26 20:18:23,369][105692] Updated weights for policy 0, policy_version 678378 (0.0010) [2023-12-26 20:18:23,425][105692] Updated weights for policy 0, policy_version 678388 (0.0010) [2023-12-26 20:18:23,483][105692] Updated weights for policy 0, policy_version 678398 (0.0010) [2023-12-26 20:18:23,532][105692] Updated weights for policy 0, policy_version 678408 (0.0010) [2023-12-26 20:18:23,924][105620] Updated weights for policy 1, policy_version 679196 (0.0008) [2023-12-26 20:18:23,972][105620] Updated weights for policy 1, policy_version 679206 (0.0008) [2023-12-26 20:18:24,020][105620] Updated weights for policy 1, policy_version 679216 (0.0008) [2023-12-26 20:18:24,289][105692] Updated weights for policy 0, policy_version 678418 (0.0010) [2023-12-26 20:18:24,351][105692] Updated weights for policy 0, policy_version 678428 (0.0010) [2023-12-26 20:18:24,411][105692] Updated weights for policy 0, policy_version 678438 (0.0010) [2023-12-26 20:18:24,824][105620] Updated weights for policy 1, policy_version 679226 (0.0009) [2023-12-26 20:18:24,887][105620] Updated weights for policy 1, policy_version 679236 (0.0009) [2023-12-26 20:18:24,953][105620] Updated weights for policy 1, policy_version 679246 (0.0008) [2023-12-26 20:18:25,013][105620] Updated weights for policy 1, policy_version 679256 (0.0008) [2023-12-26 20:18:25,064][105692] Updated weights for policy 0, policy_version 678448 (0.0011) [2023-12-26 20:18:25,129][105692] Updated weights for policy 0, policy_version 678458 (0.0011) [2023-12-26 20:18:25,190][105692] Updated weights for policy 0, policy_version 678468 (0.0010) [2023-12-26 20:18:25,735][105620] Updated weights for policy 1, policy_version 679266 (0.0008) [2023-12-26 20:18:25,781][105620] Updated weights for policy 1, policy_version 679276 (0.0006) [2023-12-26 20:18:25,827][105620] Updated weights for policy 1, policy_version 679286 (0.0005) [2023-12-26 20:18:25,926][105692] Updated weights for policy 0, policy_version 678478 (0.0011) [2023-12-26 20:18:25,980][105692] Updated weights for policy 0, policy_version 678488 (0.0010) [2023-12-26 20:18:26,028][105692] Updated weights for policy 0, policy_version 678498 (0.0010) [2023-12-26 20:18:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19797.4, 300 sec: 19522.0). Total num frames: 347643904. Throughput: 0: 9988.5, 1: 9434.8. Samples: 347647640. Policy #0 lag: (min: 31.0, avg: 31.5, max: 49.0) [2023-12-26 20:18:26,062][104569] Avg episode reward: [(0, '8805.624'), (1, '8539.441')] [2023-12-26 20:18:26,544][105620] Updated weights for policy 1, policy_version 679296 (0.0007) [2023-12-26 20:18:26,597][105620] Updated weights for policy 1, policy_version 679306 (0.0008) [2023-12-26 20:18:26,661][105620] Updated weights for policy 1, policy_version 679316 (0.0009) [2023-12-26 20:18:26,783][105692] Updated weights for policy 0, policy_version 678508 (0.0010) [2023-12-26 20:18:26,835][105692] Updated weights for policy 0, policy_version 678518 (0.0010) [2023-12-26 20:18:26,897][105692] Updated weights for policy 0, policy_version 678528 (0.0010) [2023-12-26 20:18:27,367][105620] Updated weights for policy 1, policy_version 679326 (0.0007) [2023-12-26 20:18:27,428][105620] Updated weights for policy 1, policy_version 679336 (0.0008) [2023-12-26 20:18:27,489][105620] Updated weights for policy 1, policy_version 679346 (0.0008) [2023-12-26 20:18:27,631][105692] Updated weights for policy 0, policy_version 678538 (0.0006) [2023-12-26 20:18:27,692][105692] Updated weights for policy 0, policy_version 678548 (0.0010) [2023-12-26 20:18:27,747][105692] Updated weights for policy 0, policy_version 678558 (0.0010) [2023-12-26 20:18:27,805][105692] Updated weights for policy 0, policy_version 678568 (0.0010) [2023-12-26 20:18:28,196][105620] Updated weights for policy 1, policy_version 679356 (0.0007) [2023-12-26 20:18:28,261][105620] Updated weights for policy 1, policy_version 679366 (0.0008) [2023-12-26 20:18:28,305][105620] Updated weights for policy 1, policy_version 679376 (0.0008) [2023-12-26 20:18:28,547][105692] Updated weights for policy 0, policy_version 678578 (0.0010) [2023-12-26 20:18:28,609][105692] Updated weights for policy 0, policy_version 678588 (0.0010) [2023-12-26 20:18:28,664][105692] Updated weights for policy 0, policy_version 678598 (0.0010) [2023-12-26 20:18:29,066][105620] Updated weights for policy 1, policy_version 679386 (0.0008) [2023-12-26 20:18:29,128][105620] Updated weights for policy 1, policy_version 679397 (0.0010) [2023-12-26 20:18:29,194][105620] Updated weights for policy 1, policy_version 679407 (0.0009) [2023-12-26 20:18:29,308][105692] Updated weights for policy 0, policy_version 678608 (0.0008) [2023-12-26 20:18:29,374][105692] Updated weights for policy 0, policy_version 678618 (0.0010) [2023-12-26 20:18:29,443][105692] Updated weights for policy 0, policy_version 678628 (0.0011) [2023-12-26 20:18:29,846][105620] Updated weights for policy 1, policy_version 679417 (0.0009) [2023-12-26 20:18:29,897][105620] Updated weights for policy 1, policy_version 679427 (0.0009) [2023-12-26 20:18:29,956][105620] Updated weights for policy 1, policy_version 679437 (0.0008) [2023-12-26 20:18:30,012][105620] Updated weights for policy 1, policy_version 679447 (0.0009) [2023-12-26 20:18:30,217][105692] Updated weights for policy 0, policy_version 678638 (0.0008) [2023-12-26 20:18:30,285][105692] Updated weights for policy 0, policy_version 678648 (0.0009) [2023-12-26 20:18:30,344][105692] Updated weights for policy 0, policy_version 678658 (0.0008) [2023-12-26 20:18:30,642][105620] Updated weights for policy 1, policy_version 679457 (0.0006) [2023-12-26 20:18:30,689][105620] Updated weights for policy 1, policy_version 679467 (0.0005) [2023-12-26 20:18:30,756][105620] Updated weights for policy 1, policy_version 679477 (0.0005) [2023-12-26 20:18:31,006][105692] Updated weights for policy 0, policy_version 678668 (0.0007) [2023-12-26 20:18:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 347734016. Throughput: 0: 9990.7, 1: 9445.7. Samples: 347705364. Policy #0 lag: (min: 31.0, avg: 31.5, max: 49.0) [2023-12-26 20:18:31,063][104569] Avg episode reward: [(0, '9080.040'), (1, '8633.265')] [2023-12-26 20:18:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000679480_173965312.pth... [2023-12-26 20:18:31,067][105692] Updated weights for policy 0, policy_version 678678 (0.0008) [2023-12-26 20:18:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000678392_173686784.pth [2023-12-26 20:18:31,136][105692] Updated weights for policy 0, policy_version 678688 (0.0009) [2023-12-26 20:18:31,185][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000678696_173776896.pth... [2023-12-26 20:18:31,190][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000677512_173473792.pth [2023-12-26 20:18:31,411][105620] Updated weights for policy 1, policy_version 679487 (0.0007) [2023-12-26 20:18:31,463][105620] Updated weights for policy 1, policy_version 679497 (0.0006) [2023-12-26 20:18:31,522][105620] Updated weights for policy 1, policy_version 679507 (0.0005) [2023-12-26 20:18:31,797][105692] Updated weights for policy 0, policy_version 678698 (0.0006) [2023-12-26 20:18:31,857][105692] Updated weights for policy 0, policy_version 678708 (0.0008) [2023-12-26 20:18:31,915][105692] Updated weights for policy 0, policy_version 678718 (0.0009) [2023-12-26 20:18:31,972][105692] Updated weights for policy 0, policy_version 678728 (0.0009) [2023-12-26 20:18:32,264][105620] Updated weights for policy 1, policy_version 679517 (0.0009) [2023-12-26 20:18:32,353][105620] Updated weights for policy 1, policy_version 679527 (0.0009) [2023-12-26 20:18:32,416][105620] Updated weights for policy 1, policy_version 679537 (0.0007) [2023-12-26 20:18:32,651][105692] Updated weights for policy 0, policy_version 678738 (0.0009) [2023-12-26 20:18:32,701][105692] Updated weights for policy 0, policy_version 678748 (0.0009) [2023-12-26 20:18:32,763][105692] Updated weights for policy 0, policy_version 678758 (0.0009) [2023-12-26 20:18:33,111][105620] Updated weights for policy 1, policy_version 679547 (0.0007) [2023-12-26 20:18:33,161][105620] Updated weights for policy 1, policy_version 679557 (0.0009) [2023-12-26 20:18:33,208][105620] Updated weights for policy 1, policy_version 679567 (0.0009) [2023-12-26 20:18:33,532][105692] Updated weights for policy 0, policy_version 678768 (0.0009) [2023-12-26 20:18:33,579][105692] Updated weights for policy 0, policy_version 678778 (0.0009) [2023-12-26 20:18:33,625][105692] Updated weights for policy 0, policy_version 678788 (0.0009) [2023-12-26 20:18:33,857][105620] Updated weights for policy 1, policy_version 679577 (0.0008) [2023-12-26 20:18:33,912][105620] Updated weights for policy 1, policy_version 679587 (0.0009) [2023-12-26 20:18:33,972][105620] Updated weights for policy 1, policy_version 679597 (0.0008) [2023-12-26 20:18:34,034][105620] Updated weights for policy 1, policy_version 679607 (0.0007) [2023-12-26 20:18:34,327][105692] Updated weights for policy 0, policy_version 678798 (0.0008) [2023-12-26 20:18:34,386][105692] Updated weights for policy 0, policy_version 678808 (0.0010) [2023-12-26 20:18:34,448][105692] Updated weights for policy 0, policy_version 678818 (0.0007) [2023-12-26 20:18:34,718][105620] Updated weights for policy 1, policy_version 679617 (0.0009) [2023-12-26 20:18:34,776][105620] Updated weights for policy 1, policy_version 679627 (0.0010) [2023-12-26 20:18:34,840][105620] Updated weights for policy 1, policy_version 679637 (0.0011) [2023-12-26 20:18:35,148][105692] Updated weights for policy 0, policy_version 678828 (0.0005) [2023-12-26 20:18:35,197][105692] Updated weights for policy 0, policy_version 678838 (0.0005) [2023-12-26 20:18:35,246][105692] Updated weights for policy 0, policy_version 678848 (0.0005) [2023-12-26 20:18:35,556][105620] Updated weights for policy 1, policy_version 679647 (0.0010) [2023-12-26 20:18:35,624][105620] Updated weights for policy 1, policy_version 679657 (0.0010) [2023-12-26 20:18:35,678][105620] Updated weights for policy 1, policy_version 679667 (0.0010) [2023-12-26 20:18:35,892][105692] Updated weights for policy 0, policy_version 678858 (0.0006) [2023-12-26 20:18:35,951][105692] Updated weights for policy 0, policy_version 678868 (0.0010) [2023-12-26 20:18:36,015][105692] Updated weights for policy 0, policy_version 678878 (0.0010) [2023-12-26 20:18:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 347832320. Throughput: 0: 10072.9, 1: 9444.2. Samples: 347824980. Policy #0 lag: (min: 31.0, avg: 31.5, max: 49.0) [2023-12-26 20:18:36,062][104569] Avg episode reward: [(0, '9171.642'), (1, '8630.354')] [2023-12-26 20:18:36,080][105692] Updated weights for policy 0, policy_version 678888 (0.0010) [2023-12-26 20:18:36,391][105620] Updated weights for policy 1, policy_version 679677 (0.0008) [2023-12-26 20:18:36,456][105620] Updated weights for policy 1, policy_version 679687 (0.0005) [2023-12-26 20:18:36,523][105620] Updated weights for policy 1, policy_version 679697 (0.0006) [2023-12-26 20:18:36,804][105692] Updated weights for policy 0, policy_version 678898 (0.0011) [2023-12-26 20:18:36,859][105692] Updated weights for policy 0, policy_version 678908 (0.0010) [2023-12-26 20:18:36,914][105692] Updated weights for policy 0, policy_version 678918 (0.0006) [2023-12-26 20:18:37,164][105620] Updated weights for policy 1, policy_version 679707 (0.0007) [2023-12-26 20:18:37,213][105620] Updated weights for policy 1, policy_version 679717 (0.0010) [2023-12-26 20:18:37,273][105620] Updated weights for policy 1, policy_version 679727 (0.0011) [2023-12-26 20:18:37,465][105692] Updated weights for policy 0, policy_version 678928 (0.0005) [2023-12-26 20:18:37,511][105692] Updated weights for policy 0, policy_version 678938 (0.0005) [2023-12-26 20:18:37,564][105692] Updated weights for policy 0, policy_version 678948 (0.0005) [2023-12-26 20:18:37,983][105620] Updated weights for policy 1, policy_version 679737 (0.0010) [2023-12-26 20:18:38,046][105620] Updated weights for policy 1, policy_version 679747 (0.0011) [2023-12-26 20:18:38,107][105620] Updated weights for policy 1, policy_version 679757 (0.0010) [2023-12-26 20:18:38,155][105620] Updated weights for policy 1, policy_version 679767 (0.0010) [2023-12-26 20:18:38,194][105692] Updated weights for policy 0, policy_version 678958 (0.0007) [2023-12-26 20:18:38,245][105692] Updated weights for policy 0, policy_version 678968 (0.0008) [2023-12-26 20:18:38,297][105692] Updated weights for policy 0, policy_version 678978 (0.0008) [2023-12-26 20:18:38,892][105620] Updated weights for policy 1, policy_version 679777 (0.0010) [2023-12-26 20:18:38,941][105620] Updated weights for policy 1, policy_version 679787 (0.0010) [2023-12-26 20:18:38,999][105620] Updated weights for policy 1, policy_version 679797 (0.0010) [2023-12-26 20:18:39,058][105692] Updated weights for policy 0, policy_version 678988 (0.0007) [2023-12-26 20:18:39,104][105692] Updated weights for policy 0, policy_version 678998 (0.0005) [2023-12-26 20:18:39,148][105692] Updated weights for policy 0, policy_version 679008 (0.0005) [2023-12-26 20:18:39,805][105620] Updated weights for policy 1, policy_version 679807 (0.0011) [2023-12-26 20:18:39,862][105692] Updated weights for policy 0, policy_version 679018 (0.0006) [2023-12-26 20:18:39,867][105620] Updated weights for policy 1, policy_version 679817 (0.0012) [2023-12-26 20:18:39,925][105692] Updated weights for policy 0, policy_version 679028 (0.0009) [2023-12-26 20:18:39,931][105620] Updated weights for policy 1, policy_version 679827 (0.0010) [2023-12-26 20:18:39,985][105692] Updated weights for policy 0, policy_version 679038 (0.0007) [2023-12-26 20:18:40,049][105692] Updated weights for policy 0, policy_version 679048 (0.0009) [2023-12-26 20:18:40,634][105620] Updated weights for policy 1, policy_version 679837 (0.0008) [2023-12-26 20:18:40,686][105620] Updated weights for policy 1, policy_version 679847 (0.0005) [2023-12-26 20:18:40,746][105620] Updated weights for policy 1, policy_version 679857 (0.0006) [2023-12-26 20:18:40,777][105692] Updated weights for policy 0, policy_version 679058 (0.0010) [2023-12-26 20:18:40,834][105692] Updated weights for policy 0, policy_version 679068 (0.0010) [2023-12-26 20:18:40,882][105692] Updated weights for policy 0, policy_version 679078 (0.0010) [2023-12-26 20:18:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 347938816. Throughput: 0: 10177.3, 1: 9470.1. Samples: 347944856. Policy #0 lag: (min: 31.0, avg: 31.5, max: 49.0) [2023-12-26 20:18:41,063][104569] Avg episode reward: [(0, '9080.872'), (1, '8899.894')] [2023-12-26 20:18:41,402][105620] Updated weights for policy 1, policy_version 679867 (0.0008) [2023-12-26 20:18:41,461][105620] Updated weights for policy 1, policy_version 679877 (0.0010) [2023-12-26 20:18:41,520][105620] Updated weights for policy 1, policy_version 679887 (0.0010) [2023-12-26 20:18:41,650][105692] Updated weights for policy 0, policy_version 679088 (0.0010) [2023-12-26 20:18:41,714][105692] Updated weights for policy 0, policy_version 679098 (0.0010) [2023-12-26 20:18:41,778][105692] Updated weights for policy 0, policy_version 679108 (0.0008) [2023-12-26 20:18:42,280][105620] Updated weights for policy 1, policy_version 679897 (0.0011) [2023-12-26 20:18:42,342][105620] Updated weights for policy 1, policy_version 679907 (0.0007) [2023-12-26 20:18:42,419][105620] Updated weights for policy 1, policy_version 679917 (0.0008) [2023-12-26 20:18:42,484][105620] Updated weights for policy 1, policy_version 679927 (0.0007) [2023-12-26 20:18:42,560][105692] Updated weights for policy 0, policy_version 679118 (0.0009) [2023-12-26 20:18:42,617][105692] Updated weights for policy 0, policy_version 679128 (0.0008) [2023-12-26 20:18:42,676][105692] Updated weights for policy 0, policy_version 679138 (0.0009) [2023-12-26 20:18:43,193][105620] Updated weights for policy 1, policy_version 679937 (0.0008) [2023-12-26 20:18:43,252][105620] Updated weights for policy 1, policy_version 679947 (0.0009) [2023-12-26 20:18:43,307][105620] Updated weights for policy 1, policy_version 679957 (0.0009) [2023-12-26 20:18:43,469][105692] Updated weights for policy 0, policy_version 679148 (0.0008) [2023-12-26 20:18:43,527][105692] Updated weights for policy 0, policy_version 679158 (0.0009) [2023-12-26 20:18:43,583][105692] Updated weights for policy 0, policy_version 679168 (0.0009) [2023-12-26 20:18:44,018][105620] Updated weights for policy 1, policy_version 679968 (0.0009) [2023-12-26 20:18:44,077][105620] Updated weights for policy 1, policy_version 679978 (0.0005) [2023-12-26 20:18:44,136][105620] Updated weights for policy 1, policy_version 679988 (0.0008) [2023-12-26 20:18:44,330][105692] Updated weights for policy 0, policy_version 679178 (0.0008) [2023-12-26 20:18:44,383][105692] Updated weights for policy 0, policy_version 679188 (0.0007) [2023-12-26 20:18:44,443][105692] Updated weights for policy 0, policy_version 679198 (0.0006) [2023-12-26 20:18:44,501][105692] Updated weights for policy 0, policy_version 679208 (0.0009) [2023-12-26 20:18:44,852][105620] Updated weights for policy 1, policy_version 679998 (0.0009) [2023-12-26 20:18:44,911][105620] Updated weights for policy 1, policy_version 680008 (0.0009) [2023-12-26 20:18:44,976][105620] Updated weights for policy 1, policy_version 680018 (0.0009) [2023-12-26 20:18:45,269][105692] Updated weights for policy 0, policy_version 679218 (0.0010) [2023-12-26 20:18:45,330][105692] Updated weights for policy 0, policy_version 679228 (0.0009) [2023-12-26 20:18:45,394][105692] Updated weights for policy 0, policy_version 679238 (0.0007) [2023-12-26 20:18:45,686][105620] Updated weights for policy 1, policy_version 680028 (0.0009) [2023-12-26 20:18:45,738][105620] Updated weights for policy 1, policy_version 680039 (0.0010) [2023-12-26 20:18:45,787][105620] Updated weights for policy 1, policy_version 680050 (0.0009) [2023-12-26 20:18:46,039][105692] Updated weights for policy 0, policy_version 679249 (0.0009) [2023-12-26 20:18:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 348028928. Throughput: 0: 10029.8, 1: 9420.6. Samples: 348000244. Policy #0 lag: (min: 31.0, avg: 31.5, max: 49.0) [2023-12-26 20:18:46,062][104569] Avg episode reward: [(0, '9171.902'), (1, '8714.150')] [2023-12-26 20:18:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000680056_174112768.pth... [2023-12-26 20:18:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000678936_173826048.pth [2023-12-26 20:18:46,091][105692] Updated weights for policy 0, policy_version 679259 (0.0009) [2023-12-26 20:18:46,149][105692] Updated weights for policy 0, policy_version 679269 (0.0009) [2023-12-26 20:18:46,160][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000679272_173924352.pth... [2023-12-26 20:18:46,164][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000678120_173629440.pth [2023-12-26 20:18:46,601][105620] Updated weights for policy 1, policy_version 680060 (0.0008) [2023-12-26 20:18:46,648][105620] Updated weights for policy 1, policy_version 680070 (0.0008) [2023-12-26 20:18:46,698][105620] Updated weights for policy 1, policy_version 680080 (0.0008) [2023-12-26 20:18:46,826][105692] Updated weights for policy 0, policy_version 679279 (0.0008) [2023-12-26 20:18:46,885][105692] Updated weights for policy 0, policy_version 679289 (0.0009) [2023-12-26 20:18:46,949][105692] Updated weights for policy 0, policy_version 679299 (0.0009) [2023-12-26 20:18:47,471][105620] Updated weights for policy 1, policy_version 680090 (0.0006) [2023-12-26 20:18:47,521][105620] Updated weights for policy 1, policy_version 680100 (0.0008) [2023-12-26 20:18:47,560][105692] Updated weights for policy 0, policy_version 679309 (0.0008) [2023-12-26 20:18:47,570][105620] Updated weights for policy 1, policy_version 680110 (0.0008) [2023-12-26 20:18:47,618][105620] Updated weights for policy 1, policy_version 680120 (0.0008) [2023-12-26 20:18:47,620][105692] Updated weights for policy 0, policy_version 679319 (0.0007) [2023-12-26 20:18:47,682][105692] Updated weights for policy 0, policy_version 679329 (0.0009) [2023-12-26 20:18:48,326][105692] Updated weights for policy 0, policy_version 679339 (0.0008) [2023-12-26 20:18:48,379][105692] Updated weights for policy 0, policy_version 679349 (0.0007) [2023-12-26 20:18:48,444][105692] Updated weights for policy 0, policy_version 679359 (0.0007) [2023-12-26 20:18:48,479][105620] Updated weights for policy 1, policy_version 680130 (0.0007) [2023-12-26 20:18:48,539][105620] Updated weights for policy 1, policy_version 680140 (0.0008) [2023-12-26 20:18:48,605][105620] Updated weights for policy 1, policy_version 680150 (0.0009) [2023-12-26 20:18:49,105][105692] Updated weights for policy 0, policy_version 679369 (0.0006) [2023-12-26 20:18:49,160][105692] Updated weights for policy 0, policy_version 679379 (0.0005) [2023-12-26 20:18:49,219][105692] Updated weights for policy 0, policy_version 679389 (0.0006) [2023-12-26 20:18:49,280][105692] Updated weights for policy 0, policy_version 679399 (0.0007) [2023-12-26 20:18:49,416][105620] Updated weights for policy 1, policy_version 680160 (0.0009) [2023-12-26 20:18:49,474][105620] Updated weights for policy 1, policy_version 680170 (0.0010) [2023-12-26 20:18:49,537][105620] Updated weights for policy 1, policy_version 680180 (0.0010) [2023-12-26 20:18:49,845][105692] Updated weights for policy 0, policy_version 679409 (0.0009) [2023-12-26 20:18:49,901][105692] Updated weights for policy 0, policy_version 679419 (0.0009) [2023-12-26 20:18:49,968][105692] Updated weights for policy 0, policy_version 679429 (0.0007) [2023-12-26 20:18:50,437][105620] Updated weights for policy 1, policy_version 680190 (0.0010) [2023-12-26 20:18:50,509][105620] Updated weights for policy 1, policy_version 680200 (0.0009) [2023-12-26 20:18:50,561][105620] Updated weights for policy 1, policy_version 680210 (0.0008) [2023-12-26 20:18:50,574][105692] Updated weights for policy 0, policy_version 679439 (0.0008) [2023-12-26 20:18:50,634][105692] Updated weights for policy 0, policy_version 679449 (0.0007) [2023-12-26 20:18:50,696][105692] Updated weights for policy 0, policy_version 679459 (0.0009) [2023-12-26 20:18:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 348127232. Throughput: 0: 10036.8, 1: 9434.3. Samples: 348115960. Policy #0 lag: (min: 31.0, avg: 31.5, max: 49.0) [2023-12-26 20:18:51,063][104569] Avg episode reward: [(0, '8937.424'), (1, '8529.557')] [2023-12-26 20:18:51,364][105692] Updated weights for policy 0, policy_version 679469 (0.0009) [2023-12-26 20:18:51,399][105620] Updated weights for policy 1, policy_version 680220 (0.0008) [2023-12-26 20:18:51,428][105692] Updated weights for policy 0, policy_version 679479 (0.0007) [2023-12-26 20:18:51,463][105620] Updated weights for policy 1, policy_version 680230 (0.0007) [2023-12-26 20:18:51,482][105692] Updated weights for policy 0, policy_version 679489 (0.0007) [2023-12-26 20:18:51,514][105620] Updated weights for policy 1, policy_version 680240 (0.0007) [2023-12-26 20:18:52,198][105692] Updated weights for policy 0, policy_version 679499 (0.0007) [2023-12-26 20:18:52,256][105692] Updated weights for policy 0, policy_version 679509 (0.0009) [2023-12-26 20:18:52,320][105692] Updated weights for policy 0, policy_version 679519 (0.0009) [2023-12-26 20:18:52,332][105620] Updated weights for policy 1, policy_version 680250 (0.0008) [2023-12-26 20:18:52,393][105620] Updated weights for policy 1, policy_version 680260 (0.0008) [2023-12-26 20:18:52,451][105620] Updated weights for policy 1, policy_version 680270 (0.0009) [2023-12-26 20:18:52,506][105620] Updated weights for policy 1, policy_version 680280 (0.0009) [2023-12-26 20:18:53,034][105692] Updated weights for policy 0, policy_version 679529 (0.0010) [2023-12-26 20:18:53,092][105692] Updated weights for policy 0, policy_version 679539 (0.0008) [2023-12-26 20:18:53,146][105692] Updated weights for policy 0, policy_version 679549 (0.0009) [2023-12-26 20:18:53,201][105692] Updated weights for policy 0, policy_version 679559 (0.0009) [2023-12-26 20:18:53,295][105620] Updated weights for policy 1, policy_version 680290 (0.0007) [2023-12-26 20:18:53,361][105620] Updated weights for policy 1, policy_version 680300 (0.0009) [2023-12-26 20:18:53,420][105620] Updated weights for policy 1, policy_version 680310 (0.0009) [2023-12-26 20:18:53,959][105692] Updated weights for policy 0, policy_version 679569 (0.0009) [2023-12-26 20:18:54,011][105692] Updated weights for policy 0, policy_version 679579 (0.0008) [2023-12-26 20:18:54,066][105692] Updated weights for policy 0, policy_version 679589 (0.0009) [2023-12-26 20:18:54,135][105620] Updated weights for policy 1, policy_version 680320 (0.0006) [2023-12-26 20:18:54,189][105620] Updated weights for policy 1, policy_version 680330 (0.0010) [2023-12-26 20:18:54,240][105620] Updated weights for policy 1, policy_version 680340 (0.0010) [2023-12-26 20:18:54,778][105692] Updated weights for policy 0, policy_version 679599 (0.0007) [2023-12-26 20:18:54,831][105692] Updated weights for policy 0, policy_version 679609 (0.0005) [2023-12-26 20:18:54,877][105692] Updated weights for policy 0, policy_version 679619 (0.0005) [2023-12-26 20:18:54,958][105620] Updated weights for policy 1, policy_version 680350 (0.0007) [2023-12-26 20:18:55,011][105620] Updated weights for policy 1, policy_version 680360 (0.0006) [2023-12-26 20:18:55,069][105620] Updated weights for policy 1, policy_version 680370 (0.0005) [2023-12-26 20:18:55,471][105692] Updated weights for policy 0, policy_version 679629 (0.0006) [2023-12-26 20:18:55,528][105692] Updated weights for policy 0, policy_version 679639 (0.0007) [2023-12-26 20:18:55,587][105692] Updated weights for policy 0, policy_version 679649 (0.0008) [2023-12-26 20:18:55,749][105620] Updated weights for policy 1, policy_version 680380 (0.0007) [2023-12-26 20:18:55,803][105620] Updated weights for policy 1, policy_version 680390 (0.0010) [2023-12-26 20:18:55,857][105620] Updated weights for policy 1, policy_version 680400 (0.0010) [2023-12-26 20:18:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 348225536. Throughput: 0: 10022.1, 1: 9329.4. Samples: 348232372. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 20:18:56,062][104569] Avg episode reward: [(0, '4362.850'), (1, '8801.973')] [2023-12-26 20:18:56,163][105692] Updated weights for policy 0, policy_version 679659 (0.0008) [2023-12-26 20:18:56,220][105692] Updated weights for policy 0, policy_version 679669 (0.0009) [2023-12-26 20:18:56,275][105692] Updated weights for policy 0, policy_version 679679 (0.0010) [2023-12-26 20:18:56,541][105620] Updated weights for policy 1, policy_version 680410 (0.0009) [2023-12-26 20:18:56,600][105620] Updated weights for policy 1, policy_version 680420 (0.0005) [2023-12-26 20:18:56,635][105586] KL-divergence is very high: 115.4806 [2023-12-26 20:18:56,652][105620] Updated weights for policy 1, policy_version 680430 (0.0005) [2023-12-26 20:18:56,657][105586] KL-divergence is very high: 152.5378 [2023-12-26 20:18:56,673][105586] KL-divergence is very high: 141.7409 [2023-12-26 20:18:56,701][105586] KL-divergence is very high: 153.4218 [2023-12-26 20:18:56,704][105620] Updated weights for policy 1, policy_version 680440 (0.0005) [2023-12-26 20:18:56,996][105692] Updated weights for policy 0, policy_version 679689 (0.0010) [2023-12-26 20:18:57,053][105692] Updated weights for policy 0, policy_version 679699 (0.0010) [2023-12-26 20:18:57,111][105692] Updated weights for policy 0, policy_version 679709 (0.0010) [2023-12-26 20:18:57,168][105692] Updated weights for policy 0, policy_version 679719 (0.0010) [2023-12-26 20:18:57,211][105620] Updated weights for policy 1, policy_version 680450 (0.0007) [2023-12-26 20:18:57,268][105620] Updated weights for policy 1, policy_version 680460 (0.0010) [2023-12-26 20:18:57,326][105620] Updated weights for policy 1, policy_version 680470 (0.0008) [2023-12-26 20:18:57,920][105692] Updated weights for policy 0, policy_version 679729 (0.0007) [2023-12-26 20:18:57,974][105692] Updated weights for policy 0, policy_version 679740 (0.0010) [2023-12-26 20:18:58,019][105620] Updated weights for policy 1, policy_version 680480 (0.0006) [2023-12-26 20:18:58,034][105692] Updated weights for policy 0, policy_version 679750 (0.0008) [2023-12-26 20:18:58,081][105620] Updated weights for policy 1, policy_version 680490 (0.0005) [2023-12-26 20:18:58,157][105620] Updated weights for policy 1, policy_version 680500 (0.0007) [2023-12-26 20:18:58,808][105692] Updated weights for policy 0, policy_version 679760 (0.0008) [2023-12-26 20:18:58,865][105692] Updated weights for policy 0, policy_version 679770 (0.0009) [2023-12-26 20:18:58,875][105620] Updated weights for policy 1, policy_version 680511 (0.0008) [2023-12-26 20:18:58,931][105692] Updated weights for policy 0, policy_version 679780 (0.0008) [2023-12-26 20:18:58,945][105620] Updated weights for policy 1, policy_version 680522 (0.0010) [2023-12-26 20:18:59,000][105620] Updated weights for policy 1, policy_version 680532 (0.0009) [2023-12-26 20:18:59,686][105692] Updated weights for policy 0, policy_version 679790 (0.0009) [2023-12-26 20:18:59,752][105692] Updated weights for policy 0, policy_version 679800 (0.0009) [2023-12-26 20:18:59,769][105620] Updated weights for policy 1, policy_version 680542 (0.0008) [2023-12-26 20:18:59,817][105692] Updated weights for policy 0, policy_version 679810 (0.0006) [2023-12-26 20:18:59,835][105620] Updated weights for policy 1, policy_version 680552 (0.0009) [2023-12-26 20:18:59,892][105620] Updated weights for policy 1, policy_version 680562 (0.0008) [2023-12-26 20:19:00,512][105620] Updated weights for policy 1, policy_version 680572 (0.0008) [2023-12-26 20:19:00,568][105620] Updated weights for policy 1, policy_version 680582 (0.0005) [2023-12-26 20:19:00,626][105620] Updated weights for policy 1, policy_version 680592 (0.0005) [2023-12-26 20:19:00,639][105692] Updated weights for policy 0, policy_version 679820 (0.0006) [2023-12-26 20:19:00,700][105692] Updated weights for policy 0, policy_version 679830 (0.0005) [2023-12-26 20:19:00,761][105692] Updated weights for policy 0, policy_version 679840 (0.0008) [2023-12-26 20:19:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 348323840. Throughput: 0: 10018.4, 1: 9443.0. Samples: 348293548. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 20:19:01,062][104569] Avg episode reward: [(0, '6713.737'), (1, '8912.863')] [2023-12-26 20:19:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000679848_174071808.pth... [2023-12-26 20:19:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000680600_174252032.pth... [2023-12-26 20:19:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000678696_173776896.pth [2023-12-26 20:19:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000679480_173965312.pth [2023-12-26 20:19:01,144][105620] Updated weights for policy 1, policy_version 680602 (0.0005) [2023-12-26 20:19:01,209][105620] Updated weights for policy 1, policy_version 680612 (0.0007) [2023-12-26 20:19:01,272][105620] Updated weights for policy 1, policy_version 680622 (0.0008) [2023-12-26 20:19:01,334][105620] Updated weights for policy 1, policy_version 680632 (0.0008) [2023-12-26 20:19:01,452][105692] Updated weights for policy 0, policy_version 679850 (0.0010) [2023-12-26 20:19:01,507][105692] Updated weights for policy 0, policy_version 679860 (0.0009) [2023-12-26 20:19:01,564][105692] Updated weights for policy 0, policy_version 679870 (0.0010) [2023-12-26 20:19:01,622][105692] Updated weights for policy 0, policy_version 679880 (0.0009) [2023-12-26 20:19:01,988][105620] Updated weights for policy 1, policy_version 680642 (0.0008) [2023-12-26 20:19:02,049][105620] Updated weights for policy 1, policy_version 680652 (0.0009) [2023-12-26 20:19:02,112][105620] Updated weights for policy 1, policy_version 680662 (0.0007) [2023-12-26 20:19:02,417][105692] Updated weights for policy 0, policy_version 679890 (0.0009) [2023-12-26 20:19:02,477][105692] Updated weights for policy 0, policy_version 679900 (0.0009) [2023-12-26 20:19:02,528][105692] Updated weights for policy 0, policy_version 679910 (0.0009) [2023-12-26 20:19:02,796][105620] Updated weights for policy 1, policy_version 680672 (0.0009) [2023-12-26 20:19:02,853][105620] Updated weights for policy 1, policy_version 680682 (0.0006) [2023-12-26 20:19:02,920][105620] Updated weights for policy 1, policy_version 680692 (0.0005) [2023-12-26 20:19:03,319][105692] Updated weights for policy 0, policy_version 679920 (0.0006) [2023-12-26 20:19:03,387][105692] Updated weights for policy 0, policy_version 679930 (0.0006) [2023-12-26 20:19:03,450][105692] Updated weights for policy 0, policy_version 679940 (0.0009) [2023-12-26 20:19:03,472][105620] Updated weights for policy 1, policy_version 680702 (0.0005) [2023-12-26 20:19:03,531][105620] Updated weights for policy 1, policy_version 680712 (0.0007) [2023-12-26 20:19:03,583][105620] Updated weights for policy 1, policy_version 680722 (0.0008) [2023-12-26 20:19:04,158][105620] Updated weights for policy 1, policy_version 680732 (0.0009) [2023-12-26 20:19:04,207][105692] Updated weights for policy 0, policy_version 679950 (0.0009) [2023-12-26 20:19:04,214][105620] Updated weights for policy 1, policy_version 680742 (0.0007) [2023-12-26 20:19:04,269][105620] Updated weights for policy 1, policy_version 680752 (0.0008) [2023-12-26 20:19:04,273][105692] Updated weights for policy 0, policy_version 679960 (0.0007) [2023-12-26 20:19:04,336][105692] Updated weights for policy 0, policy_version 679970 (0.0008) [2023-12-26 20:19:04,996][105620] Updated weights for policy 1, policy_version 680762 (0.0009) [2023-12-26 20:19:05,052][105620] Updated weights for policy 1, policy_version 680772 (0.0009) [2023-12-26 20:19:05,078][105692] Updated weights for policy 0, policy_version 679980 (0.0009) [2023-12-26 20:19:05,097][105620] Updated weights for policy 1, policy_version 680782 (0.0006) [2023-12-26 20:19:05,142][105692] Updated weights for policy 0, policy_version 679990 (0.0010) [2023-12-26 20:19:05,148][105620] Updated weights for policy 1, policy_version 680792 (0.0006) [2023-12-26 20:19:05,202][105692] Updated weights for policy 0, policy_version 680000 (0.0009) [2023-12-26 20:19:05,903][105620] Updated weights for policy 1, policy_version 680802 (0.0008) [2023-12-26 20:19:05,956][105692] Updated weights for policy 0, policy_version 680010 (0.0009) [2023-12-26 20:19:05,960][105620] Updated weights for policy 1, policy_version 680812 (0.0006) [2023-12-26 20:19:06,004][105692] Updated weights for policy 0, policy_version 680020 (0.0009) [2023-12-26 20:19:06,014][105620] Updated weights for policy 1, policy_version 680822 (0.0005) [2023-12-26 20:19:06,060][105692] Updated weights for policy 0, policy_version 680030 (0.0010) [2023-12-26 20:19:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 348422144. Throughput: 0: 9805.1, 1: 9701.9. Samples: 348412264. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 20:19:06,062][104569] Avg episode reward: [(0, '7678.496'), (1, '8894.840')] [2023-12-26 20:19:06,120][105692] Updated weights for policy 0, policy_version 680040 (0.0009) [2023-12-26 20:19:06,553][105620] Updated weights for policy 1, policy_version 680832 (0.0005) [2023-12-26 20:19:06,604][105620] Updated weights for policy 1, policy_version 680842 (0.0005) [2023-12-26 20:19:06,656][105620] Updated weights for policy 1, policy_version 680852 (0.0005) [2023-12-26 20:19:06,930][105692] Updated weights for policy 0, policy_version 680050 (0.0005) [2023-12-26 20:19:06,985][105692] Updated weights for policy 0, policy_version 680060 (0.0007) [2023-12-26 20:19:07,041][105692] Updated weights for policy 0, policy_version 680070 (0.0008) [2023-12-26 20:19:07,261][105620] Updated weights for policy 1, policy_version 680862 (0.0005) [2023-12-26 20:19:07,316][105620] Updated weights for policy 1, policy_version 680872 (0.0006) [2023-12-26 20:19:07,383][105620] Updated weights for policy 1, policy_version 680882 (0.0010) [2023-12-26 20:19:07,783][105692] Updated weights for policy 0, policy_version 680080 (0.0006) [2023-12-26 20:19:07,840][105692] Updated weights for policy 0, policy_version 680090 (0.0005) [2023-12-26 20:19:07,887][105692] Updated weights for policy 0, policy_version 680100 (0.0005) [2023-12-26 20:19:08,104][105620] Updated weights for policy 1, policy_version 680892 (0.0010) [2023-12-26 20:19:08,159][105620] Updated weights for policy 1, policy_version 680902 (0.0010) [2023-12-26 20:19:08,211][105620] Updated weights for policy 1, policy_version 680912 (0.0007) [2023-12-26 20:19:08,468][105692] Updated weights for policy 0, policy_version 680110 (0.0005) [2023-12-26 20:19:08,524][105692] Updated weights for policy 0, policy_version 680120 (0.0008) [2023-12-26 20:19:08,581][105692] Updated weights for policy 0, policy_version 680130 (0.0009) [2023-12-26 20:19:08,844][105620] Updated weights for policy 1, policy_version 680922 (0.0006) [2023-12-26 20:19:08,905][105620] Updated weights for policy 1, policy_version 680932 (0.0008) [2023-12-26 20:19:08,964][105620] Updated weights for policy 1, policy_version 680942 (0.0010) [2023-12-26 20:19:09,019][105620] Updated weights for policy 1, policy_version 680952 (0.0008) [2023-12-26 20:19:09,183][105692] Updated weights for policy 0, policy_version 680140 (0.0008) [2023-12-26 20:19:09,248][105692] Updated weights for policy 0, policy_version 680150 (0.0010) [2023-12-26 20:19:09,302][105692] Updated weights for policy 0, policy_version 680160 (0.0010) [2023-12-26 20:19:09,777][105620] Updated weights for policy 1, policy_version 680962 (0.0008) [2023-12-26 20:19:09,844][105620] Updated weights for policy 1, policy_version 680972 (0.0009) [2023-12-26 20:19:09,920][105620] Updated weights for policy 1, policy_version 680982 (0.0009) [2023-12-26 20:19:10,054][105692] Updated weights for policy 0, policy_version 680170 (0.0009) [2023-12-26 20:19:10,111][105692] Updated weights for policy 0, policy_version 680180 (0.0010) [2023-12-26 20:19:10,170][105692] Updated weights for policy 0, policy_version 680190 (0.0010) [2023-12-26 20:19:10,232][105692] Updated weights for policy 0, policy_version 680200 (0.0010) [2023-12-26 20:19:10,586][105620] Updated weights for policy 1, policy_version 680992 (0.0007) [2023-12-26 20:19:10,633][105620] Updated weights for policy 1, policy_version 681002 (0.0005) [2023-12-26 20:19:10,691][105620] Updated weights for policy 1, policy_version 681012 (0.0009) [2023-12-26 20:19:10,883][105692] Updated weights for policy 0, policy_version 680210 (0.0007) [2023-12-26 20:19:10,935][105692] Updated weights for policy 0, policy_version 680220 (0.0010) [2023-12-26 20:19:10,998][105692] Updated weights for policy 0, policy_version 680230 (0.0010) [2023-12-26 20:19:11,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 348528640. Throughput: 0: 9812.7, 1: 9839.6. Samples: 348531996. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 20:19:11,062][104569] Avg episode reward: [(0, '8808.282'), (1, '8622.678')] [2023-12-26 20:19:11,398][105620] Updated weights for policy 1, policy_version 681022 (0.0008) [2023-12-26 20:19:11,454][105620] Updated weights for policy 1, policy_version 681032 (0.0006) [2023-12-26 20:19:11,514][105620] Updated weights for policy 1, policy_version 681042 (0.0006) [2023-12-26 20:19:11,868][105692] Updated weights for policy 0, policy_version 680240 (0.0006) [2023-12-26 20:19:11,933][105692] Updated weights for policy 0, policy_version 680250 (0.0005) [2023-12-26 20:19:11,994][105692] Updated weights for policy 0, policy_version 680260 (0.0006) [2023-12-26 20:19:12,225][105620] Updated weights for policy 1, policy_version 681052 (0.0007) [2023-12-26 20:19:12,286][105620] Updated weights for policy 1, policy_version 681062 (0.0009) [2023-12-26 20:19:12,347][105620] Updated weights for policy 1, policy_version 681072 (0.0008) [2023-12-26 20:19:12,546][105692] Updated weights for policy 0, policy_version 680270 (0.0006) [2023-12-26 20:19:12,609][105692] Updated weights for policy 0, policy_version 680280 (0.0005) [2023-12-26 20:19:12,670][105692] Updated weights for policy 0, policy_version 680290 (0.0006) [2023-12-26 20:19:13,140][105620] Updated weights for policy 1, policy_version 681082 (0.0007) [2023-12-26 20:19:13,200][105620] Updated weights for policy 1, policy_version 681092 (0.0006) [2023-12-26 20:19:13,245][105692] Updated weights for policy 0, policy_version 680300 (0.0005) [2023-12-26 20:19:13,259][105620] Updated weights for policy 1, policy_version 681102 (0.0006) [2023-12-26 20:19:13,309][105692] Updated weights for policy 0, policy_version 680310 (0.0005) [2023-12-26 20:19:13,316][105620] Updated weights for policy 1, policy_version 681112 (0.0005) [2023-12-26 20:19:13,356][105692] Updated weights for policy 0, policy_version 680320 (0.0005) [2023-12-26 20:19:13,834][105620] Updated weights for policy 1, policy_version 681122 (0.0009) [2023-12-26 20:19:13,885][105620] Updated weights for policy 1, policy_version 681132 (0.0009) [2023-12-26 20:19:13,936][105620] Updated weights for policy 1, policy_version 681142 (0.0010) [2023-12-26 20:19:14,037][105692] Updated weights for policy 0, policy_version 680330 (0.0007) [2023-12-26 20:19:14,113][105692] Updated weights for policy 0, policy_version 680340 (0.0006) [2023-12-26 20:19:14,181][105692] Updated weights for policy 0, policy_version 680350 (0.0009) [2023-12-26 20:19:14,239][105692] Updated weights for policy 0, policy_version 680360 (0.0008) [2023-12-26 20:19:14,549][105620] Updated weights for policy 1, policy_version 681152 (0.0010) [2023-12-26 20:19:14,603][105620] Updated weights for policy 1, policy_version 681162 (0.0010) [2023-12-26 20:19:14,661][105620] Updated weights for policy 1, policy_version 681172 (0.0010) [2023-12-26 20:19:14,774][105692] Updated weights for policy 0, policy_version 680370 (0.0006) [2023-12-26 20:19:14,834][105692] Updated weights for policy 0, policy_version 680380 (0.0007) [2023-12-26 20:19:14,898][105692] Updated weights for policy 0, policy_version 680390 (0.0010) [2023-12-26 20:19:15,383][105620] Updated weights for policy 1, policy_version 681182 (0.0008) [2023-12-26 20:19:15,456][105620] Updated weights for policy 1, policy_version 681192 (0.0007) [2023-12-26 20:19:15,521][105620] Updated weights for policy 1, policy_version 681202 (0.0005) [2023-12-26 20:19:15,558][105692] Updated weights for policy 0, policy_version 680400 (0.0011) [2023-12-26 20:19:15,625][105692] Updated weights for policy 0, policy_version 680410 (0.0011) [2023-12-26 20:19:15,700][105692] Updated weights for policy 0, policy_version 680420 (0.0007) [2023-12-26 20:19:16,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 348626944. Throughput: 0: 9871.2, 1: 9872.0. Samples: 348593808. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 20:19:16,062][104569] Avg episode reward: [(0, '8897.891'), (1, '8630.546')] [2023-12-26 20:19:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000680424_174219264.pth... [2023-12-26 20:19:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000681208_174407680.pth... [2023-12-26 20:19:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000680056_174112768.pth [2023-12-26 20:19:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000679272_173924352.pth [2023-12-26 20:19:16,136][105620] Updated weights for policy 1, policy_version 681212 (0.0005) [2023-12-26 20:19:16,191][105620] Updated weights for policy 1, policy_version 681222 (0.0005) [2023-12-26 20:19:16,246][105620] Updated weights for policy 1, policy_version 681232 (0.0005) [2023-12-26 20:19:16,360][105692] Updated weights for policy 0, policy_version 680430 (0.0010) [2023-12-26 20:19:16,408][105692] Updated weights for policy 0, policy_version 680440 (0.0010) [2023-12-26 20:19:16,457][105692] Updated weights for policy 0, policy_version 680450 (0.0010) [2023-12-26 20:19:16,787][105620] Updated weights for policy 1, policy_version 681242 (0.0006) [2023-12-26 20:19:16,850][105620] Updated weights for policy 1, policy_version 681252 (0.0010) [2023-12-26 20:19:16,902][105620] Updated weights for policy 1, policy_version 681262 (0.0010) [2023-12-26 20:19:16,952][105620] Updated weights for policy 1, policy_version 681272 (0.0009) [2023-12-26 20:19:17,189][105692] Updated weights for policy 0, policy_version 680460 (0.0010) [2023-12-26 20:19:17,237][105692] Updated weights for policy 0, policy_version 680470 (0.0010) [2023-12-26 20:19:17,285][105692] Updated weights for policy 0, policy_version 680480 (0.0010) [2023-12-26 20:19:17,593][105620] Updated weights for policy 1, policy_version 681282 (0.0005) [2023-12-26 20:19:17,639][105620] Updated weights for policy 1, policy_version 681292 (0.0005) [2023-12-26 20:19:17,694][105620] Updated weights for policy 1, policy_version 681302 (0.0005) [2023-12-26 20:19:18,046][105692] Updated weights for policy 0, policy_version 680490 (0.0010) [2023-12-26 20:19:18,102][105692] Updated weights for policy 0, policy_version 680500 (0.0011) [2023-12-26 20:19:18,164][105692] Updated weights for policy 0, policy_version 680510 (0.0011) [2023-12-26 20:19:18,227][105692] Updated weights for policy 0, policy_version 680520 (0.0010) [2023-12-26 20:19:18,328][105620] Updated weights for policy 1, policy_version 681312 (0.0008) [2023-12-26 20:19:18,386][105620] Updated weights for policy 1, policy_version 681322 (0.0009) [2023-12-26 20:19:18,456][105620] Updated weights for policy 1, policy_version 681332 (0.0009) [2023-12-26 20:19:18,992][105692] Updated weights for policy 0, policy_version 680530 (0.0008) [2023-12-26 20:19:19,050][105692] Updated weights for policy 0, policy_version 680540 (0.0005) [2023-12-26 20:19:19,107][105692] Updated weights for policy 0, policy_version 680550 (0.0008) [2023-12-26 20:19:19,178][105620] Updated weights for policy 1, policy_version 681342 (0.0007) [2023-12-26 20:19:19,247][105620] Updated weights for policy 1, policy_version 681352 (0.0009) [2023-12-26 20:19:19,302][105620] Updated weights for policy 1, policy_version 681362 (0.0009) [2023-12-26 20:19:19,847][105692] Updated weights for policy 0, policy_version 680560 (0.0008) [2023-12-26 20:19:19,919][105692] Updated weights for policy 0, policy_version 680570 (0.0008) [2023-12-26 20:19:19,979][105692] Updated weights for policy 0, policy_version 680580 (0.0009) [2023-12-26 20:19:20,107][105620] Updated weights for policy 1, policy_version 681372 (0.0010) [2023-12-26 20:19:20,163][105620] Updated weights for policy 1, policy_version 681382 (0.0009) [2023-12-26 20:19:20,223][105620] Updated weights for policy 1, policy_version 681392 (0.0007) [2023-12-26 20:19:20,805][105692] Updated weights for policy 0, policy_version 680590 (0.0008) [2023-12-26 20:19:20,854][105692] Updated weights for policy 0, policy_version 680600 (0.0009) [2023-12-26 20:19:20,896][105620] Updated weights for policy 1, policy_version 681402 (0.0008) [2023-12-26 20:19:20,907][105692] Updated weights for policy 0, policy_version 680610 (0.0009) [2023-12-26 20:19:20,951][105620] Updated weights for policy 1, policy_version 681412 (0.0008) [2023-12-26 20:19:21,006][105620] Updated weights for policy 1, policy_version 681422 (0.0008) [2023-12-26 20:19:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 348725248. Throughput: 0: 9885.9, 1: 9927.8. Samples: 348716600. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 20:19:21,063][104569] Avg episode reward: [(0, '6587.998'), (1, '8450.890')] [2023-12-26 20:19:21,075][105620] Updated weights for policy 1, policy_version 681432 (0.0008) [2023-12-26 20:19:21,642][105692] Updated weights for policy 0, policy_version 680620 (0.0006) [2023-12-26 20:19:21,703][105692] Updated weights for policy 0, policy_version 680630 (0.0006) [2023-12-26 20:19:21,764][105692] Updated weights for policy 0, policy_version 680640 (0.0008) [2023-12-26 20:19:21,880][105620] Updated weights for policy 1, policy_version 681442 (0.0006) [2023-12-26 20:19:21,946][105620] Updated weights for policy 1, policy_version 681452 (0.0006) [2023-12-26 20:19:22,009][105620] Updated weights for policy 1, policy_version 681462 (0.0006) [2023-12-26 20:19:22,453][105692] Updated weights for policy 0, policy_version 680650 (0.0007) [2023-12-26 20:19:22,515][105692] Updated weights for policy 0, policy_version 680660 (0.0010) [2023-12-26 20:19:22,574][105692] Updated weights for policy 0, policy_version 680670 (0.0010) [2023-12-26 20:19:22,599][105620] Updated weights for policy 1, policy_version 681472 (0.0006) [2023-12-26 20:19:22,636][105692] Updated weights for policy 0, policy_version 680680 (0.0007) [2023-12-26 20:19:22,653][105620] Updated weights for policy 1, policy_version 681482 (0.0007) [2023-12-26 20:19:22,700][105620] Updated weights for policy 1, policy_version 681492 (0.0008) [2023-12-26 20:19:23,322][105620] Updated weights for policy 1, policy_version 681502 (0.0007) [2023-12-26 20:19:23,377][105620] Updated weights for policy 1, policy_version 681512 (0.0006) [2023-12-26 20:19:23,428][105620] Updated weights for policy 1, policy_version 681522 (0.0008) [2023-12-26 20:19:23,451][105692] Updated weights for policy 0, policy_version 680690 (0.0009) [2023-12-26 20:19:23,520][105692] Updated weights for policy 0, policy_version 680700 (0.0006) [2023-12-26 20:19:23,581][105692] Updated weights for policy 0, policy_version 680710 (0.0009) [2023-12-26 20:19:24,008][105620] Updated weights for policy 1, policy_version 681532 (0.0007) [2023-12-26 20:19:24,060][105620] Updated weights for policy 1, policy_version 681542 (0.0005) [2023-12-26 20:19:24,114][105620] Updated weights for policy 1, policy_version 681552 (0.0005) [2023-12-26 20:19:24,405][105692] Updated weights for policy 0, policy_version 680720 (0.0009) [2023-12-26 20:19:24,463][105692] Updated weights for policy 0, policy_version 680730 (0.0010) [2023-12-26 20:19:24,516][105692] Updated weights for policy 0, policy_version 680740 (0.0009) [2023-12-26 20:19:24,720][105620] Updated weights for policy 1, policy_version 681562 (0.0006) [2023-12-26 20:19:24,782][105620] Updated weights for policy 1, policy_version 681572 (0.0009) [2023-12-26 20:19:24,846][105620] Updated weights for policy 1, policy_version 681582 (0.0008) [2023-12-26 20:19:24,906][105620] Updated weights for policy 1, policy_version 681592 (0.0005) [2023-12-26 20:19:25,328][105692] Updated weights for policy 0, policy_version 680750 (0.0009) [2023-12-26 20:19:25,383][105692] Updated weights for policy 0, policy_version 680760 (0.0009) [2023-12-26 20:19:25,430][105692] Updated weights for policy 0, policy_version 680770 (0.0008) [2023-12-26 20:19:25,573][105620] Updated weights for policy 1, policy_version 681602 (0.0010) [2023-12-26 20:19:25,626][105620] Updated weights for policy 1, policy_version 681612 (0.0008) [2023-12-26 20:19:25,683][105620] Updated weights for policy 1, policy_version 681622 (0.0009) [2023-12-26 20:19:26,059][105692] Updated weights for policy 0, policy_version 680780 (0.0007) [2023-12-26 20:19:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 348823552. Throughput: 0: 9712.9, 1: 10004.4. Samples: 348832132. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 20:19:26,062][104569] Avg episode reward: [(0, '6111.209'), (1, '8622.062')] [2023-12-26 20:19:26,106][105692] Updated weights for policy 0, policy_version 680790 (0.0009) [2023-12-26 20:19:26,150][105692] Updated weights for policy 0, policy_version 680800 (0.0010) [2023-12-26 20:19:26,521][105620] Updated weights for policy 1, policy_version 681632 (0.0009) [2023-12-26 20:19:26,575][105620] Updated weights for policy 1, policy_version 681642 (0.0010) [2023-12-26 20:19:26,621][105620] Updated weights for policy 1, policy_version 681652 (0.0006) [2023-12-26 20:19:26,792][105692] Updated weights for policy 0, policy_version 680810 (0.0007) [2023-12-26 20:19:26,839][105692] Updated weights for policy 0, policy_version 680820 (0.0009) [2023-12-26 20:19:26,887][105692] Updated weights for policy 0, policy_version 680830 (0.0010) [2023-12-26 20:19:26,934][105692] Updated weights for policy 0, policy_version 680840 (0.0007) [2023-12-26 20:19:27,372][105620] Updated weights for policy 1, policy_version 681662 (0.0007) [2023-12-26 20:19:27,420][105620] Updated weights for policy 1, policy_version 681672 (0.0010) [2023-12-26 20:19:27,470][105620] Updated weights for policy 1, policy_version 681682 (0.0007) [2023-12-26 20:19:27,585][105692] Updated weights for policy 0, policy_version 680850 (0.0005) [2023-12-26 20:19:27,631][105692] Updated weights for policy 0, policy_version 680860 (0.0007) [2023-12-26 20:19:27,689][105692] Updated weights for policy 0, policy_version 680870 (0.0010) [2023-12-26 20:19:28,071][105620] Updated weights for policy 1, policy_version 681692 (0.0005) [2023-12-26 20:19:28,122][105620] Updated weights for policy 1, policy_version 681702 (0.0005) [2023-12-26 20:19:28,170][105620] Updated weights for policy 1, policy_version 681712 (0.0007) [2023-12-26 20:19:28,364][105692] Updated weights for policy 0, policy_version 680880 (0.0010) [2023-12-26 20:19:28,410][105692] Updated weights for policy 0, policy_version 680890 (0.0010) [2023-12-26 20:19:28,461][105692] Updated weights for policy 0, policy_version 680900 (0.0011) [2023-12-26 20:19:28,825][105620] Updated weights for policy 1, policy_version 681722 (0.0009) [2023-12-26 20:19:28,883][105620] Updated weights for policy 1, policy_version 681732 (0.0010) [2023-12-26 20:19:28,932][105586] KL-divergence is very high: 117.9367 [2023-12-26 20:19:28,941][105620] Updated weights for policy 1, policy_version 681742 (0.0005) [2023-12-26 20:19:28,994][105620] Updated weights for policy 1, policy_version 681752 (0.0005) [2023-12-26 20:19:29,242][105692] Updated weights for policy 0, policy_version 680910 (0.0010) [2023-12-26 20:19:29,309][105692] Updated weights for policy 0, policy_version 680920 (0.0009) [2023-12-26 20:19:29,378][105692] Updated weights for policy 0, policy_version 680930 (0.0010) [2023-12-26 20:19:29,584][105620] Updated weights for policy 1, policy_version 681762 (0.0010) [2023-12-26 20:19:29,644][105620] Updated weights for policy 1, policy_version 681772 (0.0010) [2023-12-26 20:19:29,701][105620] Updated weights for policy 1, policy_version 681782 (0.0010) [2023-12-26 20:19:30,121][105692] Updated weights for policy 0, policy_version 680940 (0.0009) [2023-12-26 20:19:30,180][105692] Updated weights for policy 0, policy_version 680950 (0.0005) [2023-12-26 20:19:30,239][105692] Updated weights for policy 0, policy_version 680960 (0.0005) [2023-12-26 20:19:30,441][105620] Updated weights for policy 1, policy_version 681792 (0.0011) [2023-12-26 20:19:30,503][105620] Updated weights for policy 1, policy_version 681802 (0.0005) [2023-12-26 20:19:30,571][105620] Updated weights for policy 1, policy_version 681812 (0.0009) [2023-12-26 20:19:30,846][105692] Updated weights for policy 0, policy_version 680970 (0.0008) [2023-12-26 20:19:30,900][105692] Updated weights for policy 0, policy_version 680980 (0.0005) [2023-12-26 20:19:30,950][105692] Updated weights for policy 0, policy_version 680990 (0.0009) [2023-12-26 20:19:31,001][105692] Updated weights for policy 0, policy_version 681000 (0.0010) [2023-12-26 20:19:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19933.9, 300 sec: 19549.7). Total num frames: 348930048. Throughput: 0: 9841.0, 1: 10045.7. Samples: 348895144. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 20:19:31,063][104569] Avg episode reward: [(0, '7190.853'), (1, '8893.803')] [2023-12-26 20:19:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000681000_174366720.pth... [2023-12-26 20:19:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000681816_174563328.pth... [2023-12-26 20:19:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000679848_174071808.pth [2023-12-26 20:19:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000680600_174252032.pth [2023-12-26 20:19:31,221][105620] Updated weights for policy 1, policy_version 681822 (0.0011) [2023-12-26 20:19:31,283][105620] Updated weights for policy 1, policy_version 681832 (0.0011) [2023-12-26 20:19:31,342][105620] Updated weights for policy 1, policy_version 681842 (0.0010) [2023-12-26 20:19:31,699][105692] Updated weights for policy 0, policy_version 681010 (0.0006) [2023-12-26 20:19:31,764][105692] Updated weights for policy 0, policy_version 681020 (0.0010) [2023-12-26 20:19:31,826][105692] Updated weights for policy 0, policy_version 681030 (0.0010) [2023-12-26 20:19:32,068][105620] Updated weights for policy 1, policy_version 681852 (0.0009) [2023-12-26 20:19:32,123][105620] Updated weights for policy 1, policy_version 681862 (0.0008) [2023-12-26 20:19:32,153][105586] KL-divergence is very high: 670.8311 [2023-12-26 20:19:32,175][105620] Updated weights for policy 1, policy_version 681872 (0.0008) [2023-12-26 20:19:32,194][105586] KL-divergence is very high: 962.0426 [2023-12-26 20:19:32,505][105692] Updated weights for policy 0, policy_version 681040 (0.0006) [2023-12-26 20:19:32,571][105692] Updated weights for policy 0, policy_version 681050 (0.0006) [2023-12-26 20:19:32,636][105692] Updated weights for policy 0, policy_version 681060 (0.0006) [2023-12-26 20:19:32,952][105620] Updated weights for policy 1, policy_version 681882 (0.0009) [2023-12-26 20:19:33,009][105620] Updated weights for policy 1, policy_version 681892 (0.0010) [2023-12-26 20:19:33,065][105620] Updated weights for policy 1, policy_version 681902 (0.0009) [2023-12-26 20:19:33,126][105620] Updated weights for policy 1, policy_version 681912 (0.0005) [2023-12-26 20:19:33,182][105692] Updated weights for policy 0, policy_version 681070 (0.0006) [2023-12-26 20:19:33,251][105692] Updated weights for policy 0, policy_version 681080 (0.0005) [2023-12-26 20:19:33,321][105692] Updated weights for policy 0, policy_version 681090 (0.0005) [2023-12-26 20:19:33,728][105620] Updated weights for policy 1, policy_version 681922 (0.0005) [2023-12-26 20:19:33,783][105620] Updated weights for policy 1, policy_version 681932 (0.0005) [2023-12-26 20:19:33,832][105620] Updated weights for policy 1, policy_version 681942 (0.0005) [2023-12-26 20:19:33,948][105692] Updated weights for policy 0, policy_version 681100 (0.0008) [2023-12-26 20:19:34,005][105692] Updated weights for policy 0, policy_version 681110 (0.0010) [2023-12-26 20:19:34,062][105692] Updated weights for policy 0, policy_version 681120 (0.0010) [2023-12-26 20:19:34,441][105620] Updated weights for policy 1, policy_version 681952 (0.0010) [2023-12-26 20:19:34,494][105620] Updated weights for policy 1, policy_version 681962 (0.0011) [2023-12-26 20:19:34,546][105620] Updated weights for policy 1, policy_version 681972 (0.0011) [2023-12-26 20:19:34,821][105692] Updated weights for policy 0, policy_version 681130 (0.0010) [2023-12-26 20:19:34,893][105692] Updated weights for policy 0, policy_version 681140 (0.0010) [2023-12-26 20:19:34,954][105692] Updated weights for policy 0, policy_version 681150 (0.0009) [2023-12-26 20:19:35,020][105692] Updated weights for policy 0, policy_version 681160 (0.0010) [2023-12-26 20:19:35,193][105620] Updated weights for policy 1, policy_version 681982 (0.0010) [2023-12-26 20:19:35,249][105620] Updated weights for policy 1, policy_version 681992 (0.0010) [2023-12-26 20:19:35,300][105620] Updated weights for policy 1, policy_version 682002 (0.0010) [2023-12-26 20:19:35,787][105692] Updated weights for policy 0, policy_version 681170 (0.0006) [2023-12-26 20:19:35,845][105692] Updated weights for policy 0, policy_version 681180 (0.0006) [2023-12-26 20:19:35,900][105692] Updated weights for policy 0, policy_version 681190 (0.0006) [2023-12-26 20:19:35,946][105620] Updated weights for policy 1, policy_version 682012 (0.0010) [2023-12-26 20:19:35,997][105620] Updated weights for policy 1, policy_version 682022 (0.0010) [2023-12-26 20:19:36,052][105620] Updated weights for policy 1, policy_version 682032 (0.0010) [2023-12-26 20:19:36,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19933.8, 300 sec: 19549.7). Total num frames: 349028352. Throughput: 0: 9832.8, 1: 10211.8. Samples: 349017968. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 20:19:36,063][104569] Avg episode reward: [(0, '9081.012'), (1, '8987.526')] [2023-12-26 20:19:36,496][105692] Updated weights for policy 0, policy_version 681200 (0.0010) [2023-12-26 20:19:36,542][105692] Updated weights for policy 0, policy_version 681210 (0.0011) [2023-12-26 20:19:36,595][105692] Updated weights for policy 0, policy_version 681220 (0.0011) [2023-12-26 20:19:36,777][105620] Updated weights for policy 1, policy_version 682042 (0.0008) [2023-12-26 20:19:36,826][105620] Updated weights for policy 1, policy_version 682052 (0.0010) [2023-12-26 20:19:36,882][105620] Updated weights for policy 1, policy_version 682062 (0.0007) [2023-12-26 20:19:36,943][105620] Updated weights for policy 1, policy_version 682072 (0.0005) [2023-12-26 20:19:37,230][105692] Updated weights for policy 0, policy_version 681230 (0.0011) [2023-12-26 20:19:37,278][105692] Updated weights for policy 0, policy_version 681240 (0.0011) [2023-12-26 20:19:37,330][105692] Updated weights for policy 0, policy_version 681250 (0.0011) [2023-12-26 20:19:37,629][105620] Updated weights for policy 1, policy_version 682082 (0.0011) [2023-12-26 20:19:37,687][105620] Updated weights for policy 1, policy_version 682092 (0.0010) [2023-12-26 20:19:37,746][105620] Updated weights for policy 1, policy_version 682102 (0.0010) [2023-12-26 20:19:38,063][105692] Updated weights for policy 0, policy_version 681260 (0.0009) [2023-12-26 20:19:38,112][105692] Updated weights for policy 0, policy_version 681270 (0.0011) [2023-12-26 20:19:38,158][105692] Updated weights for policy 0, policy_version 681280 (0.0011) [2023-12-26 20:19:38,382][105620] Updated weights for policy 1, policy_version 682112 (0.0010) [2023-12-26 20:19:38,448][105620] Updated weights for policy 1, policy_version 682122 (0.0010) [2023-12-26 20:19:38,496][105620] Updated weights for policy 1, policy_version 682132 (0.0010) [2023-12-26 20:19:38,876][105692] Updated weights for policy 0, policy_version 681290 (0.0010) [2023-12-26 20:19:38,931][105692] Updated weights for policy 0, policy_version 681300 (0.0010) [2023-12-26 20:19:38,974][105692] Updated weights for policy 0, policy_version 681310 (0.0007) [2023-12-26 20:19:39,043][105692] Updated weights for policy 0, policy_version 681320 (0.0005) [2023-12-26 20:19:39,139][105620] Updated weights for policy 1, policy_version 682142 (0.0007) [2023-12-26 20:19:39,203][105620] Updated weights for policy 1, policy_version 682152 (0.0008) [2023-12-26 20:19:39,266][105620] Updated weights for policy 1, policy_version 682162 (0.0010) [2023-12-26 20:19:39,747][105692] Updated weights for policy 0, policy_version 681330 (0.0010) [2023-12-26 20:19:39,812][105692] Updated weights for policy 0, policy_version 681340 (0.0009) [2023-12-26 20:19:39,878][105692] Updated weights for policy 0, policy_version 681350 (0.0008) [2023-12-26 20:19:40,032][105620] Updated weights for policy 1, policy_version 682172 (0.0010) [2023-12-26 20:19:40,087][105620] Updated weights for policy 1, policy_version 682182 (0.0009) [2023-12-26 20:19:40,148][105620] Updated weights for policy 1, policy_version 682192 (0.0009) [2023-12-26 20:19:40,511][105692] Updated weights for policy 0, policy_version 681360 (0.0006) [2023-12-26 20:19:40,574][105692] Updated weights for policy 0, policy_version 681370 (0.0006) [2023-12-26 20:19:40,629][105692] Updated weights for policy 0, policy_version 681380 (0.0008) [2023-12-26 20:19:40,911][105620] Updated weights for policy 1, policy_version 682202 (0.0009) [2023-12-26 20:19:40,974][105620] Updated weights for policy 1, policy_version 682212 (0.0006) [2023-12-26 20:19:41,034][105620] Updated weights for policy 1, policy_version 682222 (0.0008) [2023-12-26 20:19:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 349126656. Throughput: 0: 9815.4, 1: 10340.9. Samples: 349139404. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 20:19:41,062][104569] Avg episode reward: [(0, '8405.847'), (1, '8989.025')] [2023-12-26 20:19:41,094][105620] Updated weights for policy 1, policy_version 682232 (0.0008) [2023-12-26 20:19:41,360][105692] Updated weights for policy 0, policy_version 681390 (0.0011) [2023-12-26 20:19:41,426][105692] Updated weights for policy 0, policy_version 681400 (0.0011) [2023-12-26 20:19:41,482][105692] Updated weights for policy 0, policy_version 681410 (0.0011) [2023-12-26 20:19:41,898][105620] Updated weights for policy 1, policy_version 682242 (0.0008) [2023-12-26 20:19:41,962][105620] Updated weights for policy 1, policy_version 682252 (0.0008) [2023-12-26 20:19:42,033][105620] Updated weights for policy 1, policy_version 682262 (0.0008) [2023-12-26 20:19:42,218][105692] Updated weights for policy 0, policy_version 681420 (0.0011) [2023-12-26 20:19:42,285][105692] Updated weights for policy 0, policy_version 681430 (0.0011) [2023-12-26 20:19:42,354][105692] Updated weights for policy 0, policy_version 681440 (0.0011) [2023-12-26 20:19:42,801][105620] Updated weights for policy 1, policy_version 682272 (0.0008) [2023-12-26 20:19:42,864][105620] Updated weights for policy 1, policy_version 682282 (0.0007) [2023-12-26 20:19:42,924][105620] Updated weights for policy 1, policy_version 682292 (0.0008) [2023-12-26 20:19:43,037][105692] Updated weights for policy 0, policy_version 681450 (0.0008) [2023-12-26 20:19:43,103][105692] Updated weights for policy 0, policy_version 681460 (0.0009) [2023-12-26 20:19:43,168][105692] Updated weights for policy 0, policy_version 681470 (0.0011) [2023-12-26 20:19:43,240][105692] Updated weights for policy 0, policy_version 681480 (0.0006) [2023-12-26 20:19:43,543][105620] Updated weights for policy 1, policy_version 682303 (0.0009) [2023-12-26 20:19:43,597][105620] Updated weights for policy 1, policy_version 682313 (0.0006) [2023-12-26 20:19:43,655][105620] Updated weights for policy 1, policy_version 682323 (0.0009) [2023-12-26 20:19:43,803][105692] Updated weights for policy 0, policy_version 681490 (0.0006) [2023-12-26 20:19:43,862][105692] Updated weights for policy 0, policy_version 681500 (0.0005) [2023-12-26 20:19:43,935][105692] Updated weights for policy 0, policy_version 681510 (0.0005) [2023-12-26 20:19:44,484][105620] Updated weights for policy 1, policy_version 682333 (0.0008) [2023-12-26 20:19:44,521][105692] Updated weights for policy 0, policy_version 681521 (0.0009) [2023-12-26 20:19:44,543][105620] Updated weights for policy 1, policy_version 682343 (0.0007) [2023-12-26 20:19:44,581][105692] Updated weights for policy 0, policy_version 681531 (0.0008) [2023-12-26 20:19:44,592][105620] Updated weights for policy 1, policy_version 682353 (0.0007) [2023-12-26 20:19:44,634][105692] Updated weights for policy 0, policy_version 681541 (0.0009) [2023-12-26 20:19:45,310][105620] Updated weights for policy 1, policy_version 682363 (0.0009) [2023-12-26 20:19:45,361][105620] Updated weights for policy 1, policy_version 682373 (0.0007) [2023-12-26 20:19:45,386][105692] Updated weights for policy 0, policy_version 681551 (0.0010) [2023-12-26 20:19:45,413][105620] Updated weights for policy 1, policy_version 682383 (0.0005) [2023-12-26 20:19:45,438][105692] Updated weights for policy 0, policy_version 681561 (0.0011) [2023-12-26 20:19:45,499][105692] Updated weights for policy 0, policy_version 681571 (0.0010) [2023-12-26 20:19:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19933.8, 300 sec: 19549.7). Total num frames: 349224960. Throughput: 0: 9805.6, 1: 10264.1. Samples: 349196692. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 20:19:46,063][104569] Avg episode reward: [(0, '8407.981'), (1, '8898.876')] [2023-12-26 20:19:46,073][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000681576_174514176.pth... [2023-12-26 20:19:46,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000682392_174710784.pth... [2023-12-26 20:19:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000681208_174407680.pth [2023-12-26 20:19:46,080][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000680424_174219264.pth [2023-12-26 20:19:46,182][105620] Updated weights for policy 1, policy_version 682393 (0.0005) [2023-12-26 20:19:46,199][105692] Updated weights for policy 0, policy_version 681581 (0.0010) [2023-12-26 20:19:46,233][105620] Updated weights for policy 1, policy_version 682403 (0.0006) [2023-12-26 20:19:46,255][105692] Updated weights for policy 0, policy_version 681591 (0.0010) [2023-12-26 20:19:46,293][105620] Updated weights for policy 1, policy_version 682413 (0.0006) [2023-12-26 20:19:46,307][105692] Updated weights for policy 0, policy_version 681601 (0.0010) [2023-12-26 20:19:46,352][105620] Updated weights for policy 1, policy_version 682423 (0.0005) [2023-12-26 20:19:47,060][105692] Updated weights for policy 0, policy_version 681611 (0.0010) [2023-12-26 20:19:47,116][105692] Updated weights for policy 0, policy_version 681621 (0.0010) [2023-12-26 20:19:47,126][105620] Updated weights for policy 1, policy_version 682433 (0.0006) [2023-12-26 20:19:47,165][105692] Updated weights for policy 0, policy_version 681631 (0.0010) [2023-12-26 20:19:47,186][105620] Updated weights for policy 1, policy_version 682443 (0.0006) [2023-12-26 20:19:47,248][105620] Updated weights for policy 1, policy_version 682453 (0.0006) [2023-12-26 20:19:47,883][105692] Updated weights for policy 0, policy_version 681641 (0.0010) [2023-12-26 20:19:47,911][105620] Updated weights for policy 1, policy_version 682463 (0.0009) [2023-12-26 20:19:47,939][105692] Updated weights for policy 0, policy_version 681651 (0.0010) [2023-12-26 20:19:47,963][105620] Updated weights for policy 1, policy_version 682473 (0.0010) [2023-12-26 20:19:47,994][105692] Updated weights for policy 0, policy_version 681661 (0.0010) [2023-12-26 20:19:48,015][105620] Updated weights for policy 1, policy_version 682483 (0.0010) [2023-12-26 20:19:48,047][105692] Updated weights for policy 0, policy_version 681671 (0.0010) [2023-12-26 20:19:48,773][105620] Updated weights for policy 1, policy_version 682493 (0.0010) [2023-12-26 20:19:48,791][105692] Updated weights for policy 0, policy_version 681681 (0.0010) [2023-12-26 20:19:48,832][105620] Updated weights for policy 1, policy_version 682503 (0.0010) [2023-12-26 20:19:48,840][105692] Updated weights for policy 0, policy_version 681691 (0.0010) [2023-12-26 20:19:48,892][105620] Updated weights for policy 1, policy_version 682513 (0.0010) [2023-12-26 20:19:48,901][105692] Updated weights for policy 0, policy_version 681701 (0.0011) [2023-12-26 20:19:49,637][105620] Updated weights for policy 1, policy_version 682523 (0.0010) [2023-12-26 20:19:49,691][105692] Updated weights for policy 0, policy_version 681711 (0.0010) [2023-12-26 20:19:49,695][105620] Updated weights for policy 1, policy_version 682533 (0.0010) [2023-12-26 20:19:49,749][105692] Updated weights for policy 0, policy_version 681721 (0.0010) [2023-12-26 20:19:49,756][105620] Updated weights for policy 1, policy_version 682543 (0.0010) [2023-12-26 20:19:49,814][105692] Updated weights for policy 0, policy_version 681731 (0.0008) [2023-12-26 20:19:50,458][105692] Updated weights for policy 0, policy_version 681741 (0.0010) [2023-12-26 20:19:50,510][105620] Updated weights for policy 1, policy_version 682553 (0.0010) [2023-12-26 20:19:50,522][105692] Updated weights for policy 0, policy_version 681751 (0.0011) [2023-12-26 20:19:50,571][105620] Updated weights for policy 1, policy_version 682563 (0.0011) [2023-12-26 20:19:50,590][105692] Updated weights for policy 0, policy_version 681761 (0.0011) [2023-12-26 20:19:50,643][105620] Updated weights for policy 1, policy_version 682573 (0.0006) [2023-12-26 20:19:50,714][105620] Updated weights for policy 1, policy_version 682583 (0.0006) [2023-12-26 20:19:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 349323264. Throughput: 0: 9910.6, 1: 10085.9. Samples: 349312108. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 20:19:51,063][104569] Avg episode reward: [(0, '8291.309'), (1, '8896.744')] [2023-12-26 20:19:51,328][105692] Updated weights for policy 0, policy_version 681771 (0.0011) [2023-12-26 20:19:51,396][105692] Updated weights for policy 0, policy_version 681781 (0.0011) [2023-12-26 20:19:51,411][105620] Updated weights for policy 1, policy_version 682593 (0.0008) [2023-12-26 20:19:51,461][105692] Updated weights for policy 0, policy_version 681791 (0.0008) [2023-12-26 20:19:51,471][105620] Updated weights for policy 1, policy_version 682603 (0.0008) [2023-12-26 20:19:51,531][105620] Updated weights for policy 1, policy_version 682613 (0.0011) [2023-12-26 20:19:52,105][105692] Updated weights for policy 0, policy_version 681801 (0.0006) [2023-12-26 20:19:52,176][105692] Updated weights for policy 0, policy_version 681811 (0.0007) [2023-12-26 20:19:52,241][105620] Updated weights for policy 1, policy_version 682623 (0.0010) [2023-12-26 20:19:52,242][105692] Updated weights for policy 0, policy_version 681821 (0.0011) [2023-12-26 20:19:52,302][105620] Updated weights for policy 1, policy_version 682633 (0.0011) [2023-12-26 20:19:52,302][105692] Updated weights for policy 0, policy_version 681831 (0.0011) [2023-12-26 20:19:52,362][105620] Updated weights for policy 1, policy_version 682643 (0.0011) [2023-12-26 20:19:53,027][105692] Updated weights for policy 0, policy_version 681841 (0.0010) [2023-12-26 20:19:53,077][105692] Updated weights for policy 0, policy_version 681851 (0.0009) [2023-12-26 20:19:53,138][105692] Updated weights for policy 0, policy_version 681861 (0.0005) [2023-12-26 20:19:53,156][105620] Updated weights for policy 1, policy_version 682653 (0.0010) [2023-12-26 20:19:53,208][105620] Updated weights for policy 1, policy_version 682663 (0.0009) [2023-12-26 20:19:53,262][105620] Updated weights for policy 1, policy_version 682674 (0.0010) [2023-12-26 20:19:53,769][105692] Updated weights for policy 0, policy_version 681871 (0.0009) [2023-12-26 20:19:53,834][105692] Updated weights for policy 0, policy_version 681881 (0.0011) [2023-12-26 20:19:53,889][105692] Updated weights for policy 0, policy_version 681891 (0.0010) [2023-12-26 20:19:53,935][105620] Updated weights for policy 1, policy_version 682685 (0.0008) [2023-12-26 20:19:53,993][105620] Updated weights for policy 1, policy_version 682695 (0.0005) [2023-12-26 20:19:54,044][105620] Updated weights for policy 1, policy_version 682705 (0.0005) [2023-12-26 20:19:54,526][105692] Updated weights for policy 0, policy_version 681901 (0.0010) [2023-12-26 20:19:54,579][105692] Updated weights for policy 0, policy_version 681912 (0.0010) [2023-12-26 20:19:54,610][105620] Updated weights for policy 1, policy_version 682715 (0.0005) [2023-12-26 20:19:54,642][105692] Updated weights for policy 0, policy_version 681922 (0.0009) [2023-12-26 20:19:54,661][105620] Updated weights for policy 1, policy_version 682725 (0.0005) [2023-12-26 20:19:54,712][105620] Updated weights for policy 1, policy_version 682735 (0.0008) [2023-12-26 20:19:55,334][105620] Updated weights for policy 1, policy_version 682745 (0.0006) [2023-12-26 20:19:55,335][105692] Updated weights for policy 0, policy_version 681932 (0.0008) [2023-12-26 20:19:55,381][105692] Updated weights for policy 0, policy_version 681942 (0.0005) [2023-12-26 20:19:55,382][105620] Updated weights for policy 1, policy_version 682755 (0.0005) [2023-12-26 20:19:55,437][105692] Updated weights for policy 0, policy_version 681952 (0.0005) [2023-12-26 20:19:55,437][105620] Updated weights for policy 1, policy_version 682765 (0.0008) [2023-12-26 20:19:55,489][105620] Updated weights for policy 1, policy_version 682775 (0.0010) [2023-12-26 20:19:55,963][105692] Updated weights for policy 0, policy_version 681962 (0.0005) [2023-12-26 20:19:56,017][105692] Updated weights for policy 0, policy_version 681972 (0.0005) [2023-12-26 20:19:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19933.8, 300 sec: 19549.7). Total num frames: 349421568. Throughput: 0: 9977.1, 1: 10086.5. Samples: 349434856. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 20:19:56,063][104569] Avg episode reward: [(0, '8716.059'), (1, '9078.241')] [2023-12-26 20:19:56,063][105692] Updated weights for policy 0, policy_version 681982 (0.0005) [2023-12-26 20:19:56,117][105692] Updated weights for policy 0, policy_version 681992 (0.0005) [2023-12-26 20:19:56,185][105620] Updated weights for policy 1, policy_version 682785 (0.0010) [2023-12-26 20:19:56,232][105620] Updated weights for policy 1, policy_version 682795 (0.0010) [2023-12-26 20:19:56,280][105620] Updated weights for policy 1, policy_version 682805 (0.0010) [2023-12-26 20:19:56,750][105692] Updated weights for policy 0, policy_version 682002 (0.0010) [2023-12-26 20:19:56,794][105692] Updated weights for policy 0, policy_version 682012 (0.0010) [2023-12-26 20:19:56,838][105692] Updated weights for policy 0, policy_version 682022 (0.0007) [2023-12-26 20:19:57,003][105620] Updated weights for policy 1, policy_version 682815 (0.0007) [2023-12-26 20:19:57,047][105620] Updated weights for policy 1, policy_version 682825 (0.0010) [2023-12-26 20:19:57,101][105620] Updated weights for policy 1, policy_version 682835 (0.0010) [2023-12-26 20:19:57,443][105692] Updated weights for policy 0, policy_version 682032 (0.0008) [2023-12-26 20:19:57,500][105692] Updated weights for policy 0, policy_version 682042 (0.0008) [2023-12-26 20:19:57,554][105692] Updated weights for policy 0, policy_version 682052 (0.0005) [2023-12-26 20:19:57,768][105620] Updated weights for policy 1, policy_version 682845 (0.0010) [2023-12-26 20:19:57,812][105620] Updated weights for policy 1, policy_version 682855 (0.0010) [2023-12-26 20:19:57,859][105620] Updated weights for policy 1, policy_version 682865 (0.0010) [2023-12-26 20:19:58,163][105692] Updated weights for policy 0, policy_version 682062 (0.0006) [2023-12-26 20:19:58,232][105692] Updated weights for policy 0, policy_version 682072 (0.0007) [2023-12-26 20:19:58,297][105692] Updated weights for policy 0, policy_version 682082 (0.0008) [2023-12-26 20:19:58,596][105620] Updated weights for policy 1, policy_version 682875 (0.0010) [2023-12-26 20:19:58,657][105620] Updated weights for policy 1, policy_version 682885 (0.0008) [2023-12-26 20:19:58,722][105620] Updated weights for policy 1, policy_version 682895 (0.0006) [2023-12-26 20:19:59,040][105692] Updated weights for policy 0, policy_version 682092 (0.0008) [2023-12-26 20:19:59,098][105692] Updated weights for policy 0, policy_version 682102 (0.0008) [2023-12-26 20:19:59,158][105692] Updated weights for policy 0, policy_version 682112 (0.0009) [2023-12-26 20:19:59,433][105620] Updated weights for policy 1, policy_version 682905 (0.0009) [2023-12-26 20:19:59,481][105620] Updated weights for policy 1, policy_version 682915 (0.0009) [2023-12-26 20:19:59,531][105620] Updated weights for policy 1, policy_version 682925 (0.0010) [2023-12-26 20:19:59,585][105620] Updated weights for policy 1, policy_version 682935 (0.0009) [2023-12-26 20:19:59,941][105692] Updated weights for policy 0, policy_version 682122 (0.0009) [2023-12-26 20:20:00,000][105692] Updated weights for policy 0, policy_version 682132 (0.0010) [2023-12-26 20:20:00,049][105692] Updated weights for policy 0, policy_version 682142 (0.0010) [2023-12-26 20:20:00,103][105692] Updated weights for policy 0, policy_version 682152 (0.0010) [2023-12-26 20:20:00,360][105620] Updated weights for policy 1, policy_version 682945 (0.0010) [2023-12-26 20:20:00,418][105620] Updated weights for policy 1, policy_version 682955 (0.0010) [2023-12-26 20:20:00,478][105620] Updated weights for policy 1, policy_version 682965 (0.0011) [2023-12-26 20:20:00,857][105692] Updated weights for policy 0, policy_version 682162 (0.0010) [2023-12-26 20:20:00,901][105692] Updated weights for policy 0, policy_version 682172 (0.0010) [2023-12-26 20:20:00,955][105692] Updated weights for policy 0, policy_version 682182 (0.0008) [2023-12-26 20:20:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 20070.4, 300 sec: 19577.5). Total num frames: 349528064. Throughput: 0: 10035.1, 1: 10052.9. Samples: 349497768. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 20:20:01,063][104569] Avg episode reward: [(0, '8923.296'), (1, '8987.112')] [2023-12-26 20:20:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000682184_174669824.pth... [2023-12-26 20:20:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000682968_174858240.pth... [2023-12-26 20:20:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000681000_174366720.pth [2023-12-26 20:20:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000681816_174563328.pth [2023-12-26 20:20:01,212][105620] Updated weights for policy 1, policy_version 682975 (0.0011) [2023-12-26 20:20:01,271][105620] Updated weights for policy 1, policy_version 682985 (0.0010) [2023-12-26 20:20:01,332][105620] Updated weights for policy 1, policy_version 682995 (0.0010) [2023-12-26 20:20:01,737][105692] Updated weights for policy 0, policy_version 682192 (0.0008) [2023-12-26 20:20:01,800][105692] Updated weights for policy 0, policy_version 682202 (0.0009) [2023-12-26 20:20:01,859][105692] Updated weights for policy 0, policy_version 682212 (0.0009) [2023-12-26 20:20:02,092][105620] Updated weights for policy 1, policy_version 683005 (0.0008) [2023-12-26 20:20:02,151][105620] Updated weights for policy 1, policy_version 683015 (0.0010) [2023-12-26 20:20:02,212][105620] Updated weights for policy 1, policy_version 683025 (0.0010) [2023-12-26 20:20:02,494][105692] Updated weights for policy 0, policy_version 682222 (0.0008) [2023-12-26 20:20:02,555][105692] Updated weights for policy 0, policy_version 682232 (0.0008) [2023-12-26 20:20:02,614][105692] Updated weights for policy 0, policy_version 682242 (0.0011) [2023-12-26 20:20:02,955][105620] Updated weights for policy 1, policy_version 683035 (0.0010) [2023-12-26 20:20:03,003][105620] Updated weights for policy 1, policy_version 683045 (0.0010) [2023-12-26 20:20:03,053][105620] Updated weights for policy 1, policy_version 683055 (0.0010) [2023-12-26 20:20:03,304][105692] Updated weights for policy 0, policy_version 682252 (0.0009) [2023-12-26 20:20:03,348][105692] Updated weights for policy 0, policy_version 682262 (0.0008) [2023-12-26 20:20:03,396][105692] Updated weights for policy 0, policy_version 682272 (0.0008) [2023-12-26 20:20:03,759][105620] Updated weights for policy 1, policy_version 683065 (0.0010) [2023-12-26 20:20:03,811][105620] Updated weights for policy 1, policy_version 683075 (0.0008) [2023-12-26 20:20:03,869][105620] Updated weights for policy 1, policy_version 683085 (0.0009) [2023-12-26 20:20:03,918][105620] Updated weights for policy 1, policy_version 683095 (0.0008) [2023-12-26 20:20:04,154][105692] Updated weights for policy 0, policy_version 682282 (0.0007) [2023-12-26 20:20:04,227][105692] Updated weights for policy 0, policy_version 682292 (0.0006) [2023-12-26 20:20:04,292][105692] Updated weights for policy 0, policy_version 682302 (0.0006) [2023-12-26 20:20:04,348][105692] Updated weights for policy 0, policy_version 682312 (0.0006) [2023-12-26 20:20:04,606][105620] Updated weights for policy 1, policy_version 683105 (0.0008) [2023-12-26 20:20:04,659][105620] Updated weights for policy 1, policy_version 683115 (0.0011) [2023-12-26 20:20:04,711][105620] Updated weights for policy 1, policy_version 683125 (0.0010) [2023-12-26 20:20:04,985][105692] Updated weights for policy 0, policy_version 682322 (0.0008) [2023-12-26 20:20:05,050][105692] Updated weights for policy 0, policy_version 682332 (0.0010) [2023-12-26 20:20:05,105][105692] Updated weights for policy 0, policy_version 682342 (0.0010) [2023-12-26 20:20:05,475][105620] Updated weights for policy 1, policy_version 683135 (0.0010) [2023-12-26 20:20:05,530][105620] Updated weights for policy 1, policy_version 683145 (0.0010) [2023-12-26 20:20:05,582][105620] Updated weights for policy 1, policy_version 683155 (0.0010) [2023-12-26 20:20:05,813][105692] Updated weights for policy 0, policy_version 682352 (0.0006) [2023-12-26 20:20:05,864][105692] Updated weights for policy 0, policy_version 682362 (0.0005) [2023-12-26 20:20:05,912][105692] Updated weights for policy 0, policy_version 682372 (0.0005) [2023-12-26 20:20:06,062][104569] Fps is (10 sec: 20479.9, 60 sec: 20070.4, 300 sec: 19605.3). Total num frames: 349626368. Throughput: 0: 9966.2, 1: 9943.7. Samples: 349612548. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 20:20:06,063][104569] Avg episode reward: [(0, '7458.170'), (1, '8894.491')] [2023-12-26 20:20:06,324][105620] Updated weights for policy 1, policy_version 683165 (0.0011) [2023-12-26 20:20:06,380][105620] Updated weights for policy 1, policy_version 683175 (0.0011) [2023-12-26 20:20:06,437][105620] Updated weights for policy 1, policy_version 683185 (0.0011) [2023-12-26 20:20:06,509][105692] Updated weights for policy 0, policy_version 682382 (0.0007) [2023-12-26 20:20:06,573][105692] Updated weights for policy 0, policy_version 682392 (0.0009) [2023-12-26 20:20:06,641][105692] Updated weights for policy 0, policy_version 682402 (0.0008) [2023-12-26 20:20:07,196][105620] Updated weights for policy 1, policy_version 683195 (0.0011) [2023-12-26 20:20:07,258][105620] Updated weights for policy 1, policy_version 683205 (0.0010) [2023-12-26 20:20:07,297][105692] Updated weights for policy 0, policy_version 682412 (0.0010) [2023-12-26 20:20:07,320][105620] Updated weights for policy 1, policy_version 683215 (0.0010) [2023-12-26 20:20:07,355][105692] Updated weights for policy 0, policy_version 682422 (0.0010) [2023-12-26 20:20:07,418][105692] Updated weights for policy 0, policy_version 682432 (0.0011) [2023-12-26 20:20:08,069][105620] Updated weights for policy 1, policy_version 683225 (0.0010) [2023-12-26 20:20:08,128][105620] Updated weights for policy 1, policy_version 683235 (0.0010) [2023-12-26 20:20:08,163][105692] Updated weights for policy 0, policy_version 682442 (0.0010) [2023-12-26 20:20:08,187][105620] Updated weights for policy 1, policy_version 683245 (0.0010) [2023-12-26 20:20:08,217][105692] Updated weights for policy 0, policy_version 682452 (0.0006) [2023-12-26 20:20:08,249][105620] Updated weights for policy 1, policy_version 683255 (0.0010) [2023-12-26 20:20:08,274][105692] Updated weights for policy 0, policy_version 682462 (0.0009) [2023-12-26 20:20:08,332][105692] Updated weights for policy 0, policy_version 682472 (0.0008) [2023-12-26 20:20:08,996][105620] Updated weights for policy 1, policy_version 683265 (0.0010) [2023-12-26 20:20:09,047][105620] Updated weights for policy 1, policy_version 683275 (0.0010) [2023-12-26 20:20:09,105][105620] Updated weights for policy 1, policy_version 683285 (0.0010) [2023-12-26 20:20:09,120][105692] Updated weights for policy 0, policy_version 682482 (0.0006) [2023-12-26 20:20:09,168][105692] Updated weights for policy 0, policy_version 682492 (0.0008) [2023-12-26 20:20:09,228][105692] Updated weights for policy 0, policy_version 682502 (0.0008) [2023-12-26 20:20:09,879][105620] Updated weights for policy 1, policy_version 683295 (0.0010) [2023-12-26 20:20:09,955][105620] Updated weights for policy 1, policy_version 683305 (0.0010) [2023-12-26 20:20:10,019][105620] Updated weights for policy 1, policy_version 683315 (0.0010) [2023-12-26 20:20:10,040][105692] Updated weights for policy 0, policy_version 682512 (0.0010) [2023-12-26 20:20:10,100][105692] Updated weights for policy 0, policy_version 682522 (0.0008) [2023-12-26 20:20:10,157][105692] Updated weights for policy 0, policy_version 682532 (0.0009) [2023-12-26 20:20:10,756][105620] Updated weights for policy 1, policy_version 683325 (0.0010) [2023-12-26 20:20:10,811][105620] Updated weights for policy 1, policy_version 683335 (0.0010) [2023-12-26 20:20:10,877][105620] Updated weights for policy 1, policy_version 683345 (0.0010) [2023-12-26 20:20:10,927][105692] Updated weights for policy 0, policy_version 682542 (0.0006) [2023-12-26 20:20:10,979][105692] Updated weights for policy 0, policy_version 682552 (0.0008) [2023-12-26 20:20:11,046][105692] Updated weights for policy 0, policy_version 682562 (0.0008) [2023-12-26 20:20:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 349716480. Throughput: 0: 10064.8, 1: 9800.5. Samples: 349726072. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 20:20:11,062][104569] Avg episode reward: [(0, '8033.584'), (1, '9076.329')] [2023-12-26 20:20:11,655][105620] Updated weights for policy 1, policy_version 683355 (0.0010) [2023-12-26 20:20:11,715][105620] Updated weights for policy 1, policy_version 683365 (0.0010) [2023-12-26 20:20:11,781][105620] Updated weights for policy 1, policy_version 683375 (0.0011) [2023-12-26 20:20:11,848][105692] Updated weights for policy 0, policy_version 682572 (0.0007) [2023-12-26 20:20:11,900][105692] Updated weights for policy 0, policy_version 682582 (0.0005) [2023-12-26 20:20:11,960][105692] Updated weights for policy 0, policy_version 682592 (0.0006) [2023-12-26 20:20:12,440][105620] Updated weights for policy 1, policy_version 683385 (0.0010) [2023-12-26 20:20:12,515][105620] Updated weights for policy 1, policy_version 683395 (0.0005) [2023-12-26 20:20:12,587][105620] Updated weights for policy 1, policy_version 683405 (0.0005) [2023-12-26 20:20:12,612][105692] Updated weights for policy 0, policy_version 682602 (0.0006) [2023-12-26 20:20:12,653][105620] Updated weights for policy 1, policy_version 683415 (0.0010) [2023-12-26 20:20:12,673][105692] Updated weights for policy 0, policy_version 682612 (0.0006) [2023-12-26 20:20:12,726][105692] Updated weights for policy 0, policy_version 682622 (0.0008) [2023-12-26 20:20:12,778][105692] Updated weights for policy 0, policy_version 682632 (0.0006) [2023-12-26 20:20:13,282][105620] Updated weights for policy 1, policy_version 683425 (0.0006) [2023-12-26 20:20:13,334][105620] Updated weights for policy 1, policy_version 683435 (0.0010) [2023-12-26 20:20:13,400][105620] Updated weights for policy 1, policy_version 683445 (0.0009) [2023-12-26 20:20:13,478][105692] Updated weights for policy 0, policy_version 682642 (0.0008) [2023-12-26 20:20:13,537][105692] Updated weights for policy 0, policy_version 682652 (0.0008) [2023-12-26 20:20:13,582][105692] Updated weights for policy 0, policy_version 682662 (0.0008) [2023-12-26 20:20:14,112][105620] Updated weights for policy 1, policy_version 683455 (0.0010) [2023-12-26 20:20:14,170][105620] Updated weights for policy 1, policy_version 683465 (0.0009) [2023-12-26 20:20:14,223][105620] Updated weights for policy 1, policy_version 683475 (0.0010) [2023-12-26 20:20:14,366][105692] Updated weights for policy 0, policy_version 682672 (0.0008) [2023-12-26 20:20:14,411][105692] Updated weights for policy 0, policy_version 682682 (0.0008) [2023-12-26 20:20:14,469][105692] Updated weights for policy 0, policy_version 682692 (0.0007) [2023-12-26 20:20:14,917][105620] Updated weights for policy 1, policy_version 683485 (0.0010) [2023-12-26 20:20:14,986][105620] Updated weights for policy 1, policy_version 683495 (0.0006) [2023-12-26 20:20:15,055][105620] Updated weights for policy 1, policy_version 683505 (0.0007) [2023-12-26 20:20:15,090][105692] Updated weights for policy 0, policy_version 682702 (0.0007) [2023-12-26 20:20:15,148][105692] Updated weights for policy 0, policy_version 682712 (0.0010) [2023-12-26 20:20:15,208][105692] Updated weights for policy 0, policy_version 682722 (0.0008) [2023-12-26 20:20:15,636][105620] Updated weights for policy 1, policy_version 683515 (0.0008) [2023-12-26 20:20:15,694][105620] Updated weights for policy 1, policy_version 683525 (0.0010) [2023-12-26 20:20:15,747][105620] Updated weights for policy 1, policy_version 683535 (0.0008) [2023-12-26 20:20:16,021][105692] Updated weights for policy 0, policy_version 682732 (0.0007) [2023-12-26 20:20:16,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 349814784. Throughput: 0: 9986.7, 1: 9778.2. Samples: 349784564. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 20:20:16,062][104569] Avg episode reward: [(0, '1801.671'), (1, '8985.023')] [2023-12-26 20:20:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000683544_175005696.pth... [2023-12-26 20:20:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000682392_174710784.pth [2023-12-26 20:20:16,106][105692] Updated weights for policy 0, policy_version 682742 (0.0010) [2023-12-26 20:20:16,159][105692] Updated weights for policy 0, policy_version 682753 (0.0010) [2023-12-26 20:20:16,194][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000682760_174817280.pth... [2023-12-26 20:20:16,197][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000681576_174514176.pth [2023-12-26 20:20:16,343][105620] Updated weights for policy 1, policy_version 683545 (0.0006) [2023-12-26 20:20:16,398][105620] Updated weights for policy 1, policy_version 683555 (0.0006) [2023-12-26 20:20:16,456][105620] Updated weights for policy 1, policy_version 683565 (0.0006) [2023-12-26 20:20:16,511][105620] Updated weights for policy 1, policy_version 683575 (0.0006) [2023-12-26 20:20:16,893][105692] Updated weights for policy 0, policy_version 682763 (0.0009) [2023-12-26 20:20:16,944][105692] Updated weights for policy 0, policy_version 682773 (0.0009) [2023-12-26 20:20:17,002][105692] Updated weights for policy 0, policy_version 682783 (0.0009) [2023-12-26 20:20:17,224][105620] Updated weights for policy 1, policy_version 683585 (0.0005) [2023-12-26 20:20:17,284][105620] Updated weights for policy 1, policy_version 683595 (0.0005) [2023-12-26 20:20:17,332][105620] Updated weights for policy 1, policy_version 683605 (0.0006) [2023-12-26 20:20:17,814][105692] Updated weights for policy 0, policy_version 682793 (0.0009) [2023-12-26 20:20:17,870][105692] Updated weights for policy 0, policy_version 682803 (0.0009) [2023-12-26 20:20:17,933][105692] Updated weights for policy 0, policy_version 682813 (0.0010) [2023-12-26 20:20:17,975][105620] Updated weights for policy 1, policy_version 683615 (0.0007) [2023-12-26 20:20:17,993][105692] Updated weights for policy 0, policy_version 682823 (0.0006) [2023-12-26 20:20:18,045][105620] Updated weights for policy 1, policy_version 683625 (0.0008) [2023-12-26 20:20:18,109][105620] Updated weights for policy 1, policy_version 683635 (0.0006) [2023-12-26 20:20:18,780][105692] Updated weights for policy 0, policy_version 682833 (0.0008) [2023-12-26 20:20:18,829][105620] Updated weights for policy 1, policy_version 683645 (0.0007) [2023-12-26 20:20:18,835][105692] Updated weights for policy 0, policy_version 682843 (0.0008) [2023-12-26 20:20:18,891][105692] Updated weights for policy 0, policy_version 682853 (0.0006) [2023-12-26 20:20:18,892][105620] Updated weights for policy 1, policy_version 683655 (0.0010) [2023-12-26 20:20:18,954][105620] Updated weights for policy 1, policy_version 683665 (0.0010) [2023-12-26 20:20:19,638][105620] Updated weights for policy 1, policy_version 683675 (0.0010) [2023-12-26 20:20:19,679][105586] KL-divergence is very high: 107.1888 [2023-12-26 20:20:19,705][105620] Updated weights for policy 1, policy_version 683685 (0.0011) [2023-12-26 20:20:19,733][105586] KL-divergence is very high: 135.7467 [2023-12-26 20:20:19,763][105692] Updated weights for policy 0, policy_version 682863 (0.0008) [2023-12-26 20:20:19,772][105620] Updated weights for policy 1, policy_version 683695 (0.0011) [2023-12-26 20:20:19,827][105692] Updated weights for policy 0, policy_version 682873 (0.0007) [2023-12-26 20:20:19,892][105692] Updated weights for policy 0, policy_version 682883 (0.0008) [2023-12-26 20:20:20,548][105620] Updated weights for policy 1, policy_version 683705 (0.0011) [2023-12-26 20:20:20,612][105620] Updated weights for policy 1, policy_version 683715 (0.0010) [2023-12-26 20:20:20,613][105692] Updated weights for policy 0, policy_version 682893 (0.0008) [2023-12-26 20:20:20,673][105692] Updated weights for policy 0, policy_version 682903 (0.0006) [2023-12-26 20:20:20,674][105620] Updated weights for policy 1, policy_version 683725 (0.0010) [2023-12-26 20:20:20,737][105620] Updated weights for policy 1, policy_version 683735 (0.0010) [2023-12-26 20:20:20,739][105692] Updated weights for policy 0, policy_version 682913 (0.0007) [2023-12-26 20:20:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 349913088. Throughput: 0: 9844.6, 1: 9772.2. Samples: 349900720. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 20:20:21,063][104569] Avg episode reward: [(0, '1798.274'), (1, '8986.676')] [2023-12-26 20:20:21,433][105692] Updated weights for policy 0, policy_version 682923 (0.0007) [2023-12-26 20:20:21,482][105692] Updated weights for policy 0, policy_version 682933 (0.0008) [2023-12-26 20:20:21,521][105620] Updated weights for policy 1, policy_version 683745 (0.0010) [2023-12-26 20:20:21,532][105692] Updated weights for policy 0, policy_version 682943 (0.0007) [2023-12-26 20:20:21,577][105620] Updated weights for policy 1, policy_version 683755 (0.0010) [2023-12-26 20:20:21,632][105620] Updated weights for policy 1, policy_version 683765 (0.0010) [2023-12-26 20:20:22,249][105692] Updated weights for policy 0, policy_version 682953 (0.0007) [2023-12-26 20:20:22,313][105692] Updated weights for policy 0, policy_version 682963 (0.0008) [2023-12-26 20:20:22,383][105692] Updated weights for policy 0, policy_version 682973 (0.0009) [2023-12-26 20:20:22,438][105620] Updated weights for policy 1, policy_version 683775 (0.0010) [2023-12-26 20:20:22,440][105692] Updated weights for policy 0, policy_version 682983 (0.0011) [2023-12-26 20:20:22,499][105620] Updated weights for policy 1, policy_version 683785 (0.0011) [2023-12-26 20:20:22,554][105620] Updated weights for policy 1, policy_version 683795 (0.0006) [2023-12-26 20:20:23,213][105692] Updated weights for policy 0, policy_version 682993 (0.0007) [2023-12-26 20:20:23,239][105620] Updated weights for policy 1, policy_version 683805 (0.0006) [2023-12-26 20:20:23,271][105692] Updated weights for policy 0, policy_version 683003 (0.0007) [2023-12-26 20:20:23,297][105620] Updated weights for policy 1, policy_version 683815 (0.0008) [2023-12-26 20:20:23,321][105692] Updated weights for policy 0, policy_version 683013 (0.0006) [2023-12-26 20:20:23,344][105586] KL-divergence is very high: 118.5793 [2023-12-26 20:20:23,357][105620] Updated weights for policy 1, policy_version 683825 (0.0009) [2023-12-26 20:20:23,395][105586] KL-divergence is very high: 100.8585 [2023-12-26 20:20:24,034][105692] Updated weights for policy 0, policy_version 683023 (0.0008) [2023-12-26 20:20:24,066][105620] Updated weights for policy 1, policy_version 683835 (0.0010) [2023-12-26 20:20:24,092][105692] Updated weights for policy 0, policy_version 683033 (0.0007) [2023-12-26 20:20:24,123][105620] Updated weights for policy 1, policy_version 683845 (0.0009) [2023-12-26 20:20:24,145][105692] Updated weights for policy 0, policy_version 683043 (0.0005) [2023-12-26 20:20:24,173][105620] Updated weights for policy 1, policy_version 683855 (0.0008) [2023-12-26 20:20:24,791][105620] Updated weights for policy 1, policy_version 683865 (0.0008) [2023-12-26 20:20:24,844][105692] Updated weights for policy 0, policy_version 683053 (0.0009) [2023-12-26 20:20:24,854][105620] Updated weights for policy 1, policy_version 683875 (0.0008) [2023-12-26 20:20:24,902][105692] Updated weights for policy 0, policy_version 683063 (0.0010) [2023-12-26 20:20:24,909][105620] Updated weights for policy 1, policy_version 683885 (0.0007) [2023-12-26 20:20:24,960][105692] Updated weights for policy 0, policy_version 683073 (0.0010) [2023-12-26 20:20:24,962][105620] Updated weights for policy 1, policy_version 683895 (0.0007) [2023-12-26 20:20:25,638][105692] Updated weights for policy 0, policy_version 683083 (0.0010) [2023-12-26 20:20:25,664][105620] Updated weights for policy 1, policy_version 683905 (0.0009) [2023-12-26 20:20:25,693][105692] Updated weights for policy 0, policy_version 683093 (0.0010) [2023-12-26 20:20:25,724][105620] Updated weights for policy 1, policy_version 683915 (0.0006) [2023-12-26 20:20:25,744][105692] Updated weights for policy 0, policy_version 683103 (0.0010) [2023-12-26 20:20:25,776][105620] Updated weights for policy 1, policy_version 683925 (0.0010) [2023-12-26 20:20:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 350011392. Throughput: 0: 9798.4, 1: 9711.3. Samples: 350017348. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 20:20:26,063][104569] Avg episode reward: [(0, '6628.033'), (1, '8806.206')] [2023-12-26 20:20:26,449][105620] Updated weights for policy 1, policy_version 683935 (0.0008) [2023-12-26 20:20:26,481][105692] Updated weights for policy 0, policy_version 683113 (0.0010) [2023-12-26 20:20:26,502][105620] Updated weights for policy 1, policy_version 683945 (0.0008) [2023-12-26 20:20:26,543][105692] Updated weights for policy 0, policy_version 683123 (0.0008) [2023-12-26 20:20:26,552][105620] Updated weights for policy 1, policy_version 683955 (0.0006) [2023-12-26 20:20:26,606][105692] Updated weights for policy 0, policy_version 683133 (0.0009) [2023-12-26 20:20:26,667][105692] Updated weights for policy 0, policy_version 683143 (0.0009) [2023-12-26 20:20:27,174][105620] Updated weights for policy 1, policy_version 683965 (0.0007) [2023-12-26 20:20:27,234][105620] Updated weights for policy 1, policy_version 683975 (0.0008) [2023-12-26 20:20:27,301][105620] Updated weights for policy 1, policy_version 683985 (0.0008) [2023-12-26 20:20:27,469][105692] Updated weights for policy 0, policy_version 683153 (0.0009) [2023-12-26 20:20:27,526][105692] Updated weights for policy 0, policy_version 683163 (0.0009) [2023-12-26 20:20:27,573][105692] Updated weights for policy 0, policy_version 683173 (0.0009) [2023-12-26 20:20:27,972][105620] Updated weights for policy 1, policy_version 683995 (0.0008) [2023-12-26 20:20:28,029][105620] Updated weights for policy 1, policy_version 684005 (0.0008) [2023-12-26 20:20:28,089][105620] Updated weights for policy 1, policy_version 684015 (0.0008) [2023-12-26 20:20:28,367][105692] Updated weights for policy 0, policy_version 683183 (0.0009) [2023-12-26 20:20:28,417][105692] Updated weights for policy 0, policy_version 683193 (0.0009) [2023-12-26 20:20:28,472][105692] Updated weights for policy 0, policy_version 683203 (0.0009) [2023-12-26 20:20:28,741][105620] Updated weights for policy 1, policy_version 684025 (0.0008) [2023-12-26 20:20:28,799][105620] Updated weights for policy 1, policy_version 684035 (0.0005) [2023-12-26 20:20:28,860][105620] Updated weights for policy 1, policy_version 684045 (0.0005) [2023-12-26 20:20:28,926][105620] Updated weights for policy 1, policy_version 684055 (0.0006) [2023-12-26 20:20:29,371][105692] Updated weights for policy 0, policy_version 683214 (0.0010) [2023-12-26 20:20:29,418][105692] Updated weights for policy 0, policy_version 683224 (0.0009) [2023-12-26 20:20:29,472][105692] Updated weights for policy 0, policy_version 683234 (0.0008) [2023-12-26 20:20:29,510][105620] Updated weights for policy 1, policy_version 684065 (0.0008) [2023-12-26 20:20:29,581][105620] Updated weights for policy 1, policy_version 684075 (0.0009) [2023-12-26 20:20:29,644][105620] Updated weights for policy 1, policy_version 684085 (0.0009) [2023-12-26 20:20:30,190][105692] Updated weights for policy 0, policy_version 683244 (0.0006) [2023-12-26 20:20:30,261][105692] Updated weights for policy 0, policy_version 683254 (0.0005) [2023-12-26 20:20:30,318][105692] Updated weights for policy 0, policy_version 683264 (0.0005) [2023-12-26 20:20:30,371][105620] Updated weights for policy 1, policy_version 684095 (0.0010) [2023-12-26 20:20:30,429][105620] Updated weights for policy 1, policy_version 684105 (0.0010) [2023-12-26 20:20:30,480][105620] Updated weights for policy 1, policy_version 684115 (0.0010) [2023-12-26 20:20:30,942][105692] Updated weights for policy 0, policy_version 683274 (0.0006) [2023-12-26 20:20:30,990][105692] Updated weights for policy 0, policy_version 683284 (0.0008) [2023-12-26 20:20:31,044][105692] Updated weights for policy 0, policy_version 683294 (0.0008) [2023-12-26 20:20:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 350101504. Throughput: 0: 9742.5, 1: 9799.9. Samples: 350076092. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-26 20:20:31,063][104569] Avg episode reward: [(0, '8893.460'), (1, '8444.928')] [2023-12-26 20:20:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000684120_175153152.pth... [2023-12-26 20:20:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000682968_174858240.pth [2023-12-26 20:20:31,112][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000683304_174956544.pth... [2023-12-26 20:20:31,114][105692] Updated weights for policy 0, policy_version 683304 (0.0007) [2023-12-26 20:20:31,120][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000682184_174669824.pth [2023-12-26 20:20:31,223][105620] Updated weights for policy 1, policy_version 684125 (0.0008) [2023-12-26 20:20:31,281][105620] Updated weights for policy 1, policy_version 684135 (0.0009) [2023-12-26 20:20:31,343][105620] Updated weights for policy 1, policy_version 684145 (0.0010) [2023-12-26 20:20:31,834][105692] Updated weights for policy 0, policy_version 683314 (0.0008) [2023-12-26 20:20:31,894][105692] Updated weights for policy 0, policy_version 683324 (0.0008) [2023-12-26 20:20:31,953][105692] Updated weights for policy 0, policy_version 683334 (0.0008) [2023-12-26 20:20:32,045][105620] Updated weights for policy 1, policy_version 684155 (0.0008) [2023-12-26 20:20:32,092][105620] Updated weights for policy 1, policy_version 684165 (0.0010) [2023-12-26 20:20:32,144][105620] Updated weights for policy 1, policy_version 684175 (0.0010) [2023-12-26 20:20:32,711][105692] Updated weights for policy 0, policy_version 683344 (0.0008) [2023-12-26 20:20:32,766][105692] Updated weights for policy 0, policy_version 683354 (0.0006) [2023-12-26 20:20:32,825][105692] Updated weights for policy 0, policy_version 683364 (0.0007) [2023-12-26 20:20:32,898][105620] Updated weights for policy 1, policy_version 684185 (0.0010) [2023-12-26 20:20:32,965][105620] Updated weights for policy 1, policy_version 684195 (0.0005) [2023-12-26 20:20:33,017][105620] Updated weights for policy 1, policy_version 684205 (0.0005) [2023-12-26 20:20:33,063][105620] Updated weights for policy 1, policy_version 684215 (0.0005) [2023-12-26 20:20:33,396][105692] Updated weights for policy 0, policy_version 683374 (0.0009) [2023-12-26 20:20:33,446][105692] Updated weights for policy 0, policy_version 683384 (0.0008) [2023-12-26 20:20:33,503][105692] Updated weights for policy 0, policy_version 683394 (0.0006) [2023-12-26 20:20:33,690][105620] Updated weights for policy 1, policy_version 684225 (0.0007) [2023-12-26 20:20:33,742][105620] Updated weights for policy 1, policy_version 684235 (0.0006) [2023-12-26 20:20:33,789][105620] Updated weights for policy 1, policy_version 684245 (0.0008) [2023-12-26 20:20:34,218][105692] Updated weights for policy 0, policy_version 683404 (0.0010) [2023-12-26 20:20:34,277][105692] Updated weights for policy 0, policy_version 683414 (0.0010) [2023-12-26 20:20:34,336][105692] Updated weights for policy 0, policy_version 683424 (0.0010) [2023-12-26 20:20:34,486][105620] Updated weights for policy 1, policy_version 684255 (0.0006) [2023-12-26 20:20:34,547][105620] Updated weights for policy 1, policy_version 684265 (0.0006) [2023-12-26 20:20:34,604][105620] Updated weights for policy 1, policy_version 684275 (0.0010) [2023-12-26 20:20:35,093][105692] Updated weights for policy 0, policy_version 683434 (0.0010) [2023-12-26 20:20:35,148][105692] Updated weights for policy 0, policy_version 683444 (0.0008) [2023-12-26 20:20:35,201][105692] Updated weights for policy 0, policy_version 683454 (0.0008) [2023-12-26 20:20:35,257][105692] Updated weights for policy 0, policy_version 683464 (0.0009) [2023-12-26 20:20:35,288][105620] Updated weights for policy 1, policy_version 684285 (0.0010) [2023-12-26 20:20:35,349][105620] Updated weights for policy 1, policy_version 684295 (0.0009) [2023-12-26 20:20:35,407][105620] Updated weights for policy 1, policy_version 684305 (0.0010) [2023-12-26 20:20:35,908][105692] Updated weights for policy 0, policy_version 683474 (0.0006) [2023-12-26 20:20:35,966][105692] Updated weights for policy 0, policy_version 683484 (0.0008) [2023-12-26 20:20:36,015][105692] Updated weights for policy 0, policy_version 683494 (0.0010) [2023-12-26 20:20:36,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 350208000. Throughput: 0: 9736.0, 1: 9876.7. Samples: 350194676. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 20:20:36,062][104569] Avg episode reward: [(0, '8722.296'), (1, '8986.645')] [2023-12-26 20:20:36,133][105620] Updated weights for policy 1, policy_version 684315 (0.0010) [2023-12-26 20:20:36,185][105620] Updated weights for policy 1, policy_version 684325 (0.0007) [2023-12-26 20:20:36,245][105620] Updated weights for policy 1, policy_version 684335 (0.0010) [2023-12-26 20:20:36,747][105692] Updated weights for policy 0, policy_version 683504 (0.0011) [2023-12-26 20:20:36,805][105692] Updated weights for policy 0, policy_version 683514 (0.0011) [2023-12-26 20:20:36,861][105692] Updated weights for policy 0, policy_version 683524 (0.0010) [2023-12-26 20:20:36,963][105620] Updated weights for policy 1, policy_version 684345 (0.0010) [2023-12-26 20:20:37,019][105620] Updated weights for policy 1, policy_version 684355 (0.0005) [2023-12-26 20:20:37,079][105620] Updated weights for policy 1, policy_version 684365 (0.0005) [2023-12-26 20:20:37,142][105620] Updated weights for policy 1, policy_version 684375 (0.0006) [2023-12-26 20:20:37,565][105692] Updated weights for policy 0, policy_version 683534 (0.0009) [2023-12-26 20:20:37,617][105692] Updated weights for policy 0, policy_version 683544 (0.0010) [2023-12-26 20:20:37,665][105692] Updated weights for policy 0, policy_version 683554 (0.0010) [2023-12-26 20:20:37,708][105620] Updated weights for policy 1, policy_version 684385 (0.0010) [2023-12-26 20:20:37,775][105620] Updated weights for policy 1, policy_version 684395 (0.0011) [2023-12-26 20:20:37,835][105620] Updated weights for policy 1, policy_version 684405 (0.0011) [2023-12-26 20:20:38,455][105692] Updated weights for policy 0, policy_version 683564 (0.0011) [2023-12-26 20:20:38,510][105692] Updated weights for policy 0, policy_version 683574 (0.0010) [2023-12-26 20:20:38,530][105620] Updated weights for policy 1, policy_version 684415 (0.0011) [2023-12-26 20:20:38,570][105692] Updated weights for policy 0, policy_version 683584 (0.0011) [2023-12-26 20:20:38,587][105620] Updated weights for policy 1, policy_version 684425 (0.0011) [2023-12-26 20:20:38,654][105620] Updated weights for policy 1, policy_version 684435 (0.0011) [2023-12-26 20:20:39,222][105692] Updated weights for policy 0, policy_version 683594 (0.0011) [2023-12-26 20:20:39,286][105692] Updated weights for policy 0, policy_version 683604 (0.0011) [2023-12-26 20:20:39,341][105692] Updated weights for policy 0, policy_version 683614 (0.0009) [2023-12-26 20:20:39,407][105692] Updated weights for policy 0, policy_version 683624 (0.0011) [2023-12-26 20:20:39,424][105620] Updated weights for policy 1, policy_version 684445 (0.0010) [2023-12-26 20:20:39,486][105620] Updated weights for policy 1, policy_version 684455 (0.0008) [2023-12-26 20:20:39,548][105620] Updated weights for policy 1, policy_version 684465 (0.0008) [2023-12-26 20:20:40,105][105692] Updated weights for policy 0, policy_version 683634 (0.0011) [2023-12-26 20:20:40,154][105692] Updated weights for policy 0, policy_version 683644 (0.0011) [2023-12-26 20:20:40,210][105692] Updated weights for policy 0, policy_version 683654 (0.0009) [2023-12-26 20:20:40,304][105620] Updated weights for policy 1, policy_version 684475 (0.0009) [2023-12-26 20:20:40,366][105620] Updated weights for policy 1, policy_version 684485 (0.0010) [2023-12-26 20:20:40,425][105620] Updated weights for policy 1, policy_version 684495 (0.0010) [2023-12-26 20:20:40,964][105692] Updated weights for policy 0, policy_version 683664 (0.0007) [2023-12-26 20:20:41,019][105692] Updated weights for policy 0, policy_version 683674 (0.0006) [2023-12-26 20:20:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 350298112. Throughput: 0: 9681.5, 1: 9819.9. Samples: 350312420. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 20:20:41,062][104569] Avg episode reward: [(0, '8731.801'), (1, '9259.997')] [2023-12-26 20:20:41,078][105692] Updated weights for policy 0, policy_version 683684 (0.0009) [2023-12-26 20:20:41,174][105620] Updated weights for policy 1, policy_version 684505 (0.0011) [2023-12-26 20:20:41,238][105620] Updated weights for policy 1, policy_version 684515 (0.0011) [2023-12-26 20:20:41,302][105620] Updated weights for policy 1, policy_version 684525 (0.0008) [2023-12-26 20:20:41,361][105620] Updated weights for policy 1, policy_version 684535 (0.0010) [2023-12-26 20:20:41,796][105692] Updated weights for policy 0, policy_version 683694 (0.0009) [2023-12-26 20:20:41,856][105692] Updated weights for policy 0, policy_version 683704 (0.0009) [2023-12-26 20:20:41,920][105692] Updated weights for policy 0, policy_version 683714 (0.0008) [2023-12-26 20:20:42,150][105620] Updated weights for policy 1, policy_version 684545 (0.0011) [2023-12-26 20:20:42,210][105620] Updated weights for policy 1, policy_version 684555 (0.0011) [2023-12-26 20:20:42,282][105620] Updated weights for policy 1, policy_version 684565 (0.0011) [2023-12-26 20:20:42,657][105692] Updated weights for policy 0, policy_version 683724 (0.0008) [2023-12-26 20:20:42,716][105692] Updated weights for policy 0, policy_version 683734 (0.0005) [2023-12-26 20:20:42,769][105692] Updated weights for policy 0, policy_version 683744 (0.0005) [2023-12-26 20:20:42,987][105620] Updated weights for policy 1, policy_version 684575 (0.0007) [2023-12-26 20:20:43,041][105620] Updated weights for policy 1, policy_version 684585 (0.0005) [2023-12-26 20:20:43,090][105620] Updated weights for policy 1, policy_version 684595 (0.0005) [2023-12-26 20:20:43,329][105692] Updated weights for policy 0, policy_version 683754 (0.0008) [2023-12-26 20:20:43,393][105692] Updated weights for policy 0, policy_version 683764 (0.0007) [2023-12-26 20:20:43,441][105692] Updated weights for policy 0, policy_version 683774 (0.0005) [2023-12-26 20:20:43,508][105692] Updated weights for policy 0, policy_version 683784 (0.0005) [2023-12-26 20:20:43,733][105620] Updated weights for policy 1, policy_version 684605 (0.0007) [2023-12-26 20:20:43,793][105620] Updated weights for policy 1, policy_version 684615 (0.0008) [2023-12-26 20:20:43,858][105620] Updated weights for policy 1, policy_version 684625 (0.0008) [2023-12-26 20:20:44,182][105692] Updated weights for policy 0, policy_version 683794 (0.0009) [2023-12-26 20:20:44,242][105692] Updated weights for policy 0, policy_version 683804 (0.0009) [2023-12-26 20:20:44,297][105692] Updated weights for policy 0, policy_version 683814 (0.0010) [2023-12-26 20:20:44,494][105620] Updated weights for policy 1, policy_version 684635 (0.0008) [2023-12-26 20:20:44,540][105620] Updated weights for policy 1, policy_version 684645 (0.0005) [2023-12-26 20:20:44,592][105620] Updated weights for policy 1, policy_version 684655 (0.0006) [2023-12-26 20:20:45,151][105692] Updated weights for policy 0, policy_version 683824 (0.0006) [2023-12-26 20:20:45,216][105692] Updated weights for policy 0, policy_version 683834 (0.0006) [2023-12-26 20:20:45,244][105620] Updated weights for policy 1, policy_version 684665 (0.0006) [2023-12-26 20:20:45,278][105692] Updated weights for policy 0, policy_version 683844 (0.0006) [2023-12-26 20:20:45,307][105620] Updated weights for policy 1, policy_version 684675 (0.0011) [2023-12-26 20:20:45,360][105620] Updated weights for policy 1, policy_version 684685 (0.0011) [2023-12-26 20:20:45,417][105620] Updated weights for policy 1, policy_version 684695 (0.0011) [2023-12-26 20:20:45,842][105692] Updated weights for policy 0, policy_version 683854 (0.0006) [2023-12-26 20:20:45,897][105692] Updated weights for policy 0, policy_version 683864 (0.0006) [2023-12-26 20:20:45,954][105692] Updated weights for policy 0, policy_version 683874 (0.0007) [2023-12-26 20:20:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 350404608. Throughput: 0: 9618.2, 1: 9804.3. Samples: 350371780. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 20:20:46,063][104569] Avg episode reward: [(0, '8818.729'), (1, '9170.282')] [2023-12-26 20:20:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000683880_175104000.pth... [2023-12-26 20:20:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000684696_175300608.pth... [2023-12-26 20:20:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000683544_175005696.pth [2023-12-26 20:20:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000682760_174817280.pth [2023-12-26 20:20:46,165][105620] Updated weights for policy 1, policy_version 684705 (0.0006) [2023-12-26 20:20:46,229][105620] Updated weights for policy 1, policy_version 684715 (0.0005) [2023-12-26 20:20:46,280][105620] Updated weights for policy 1, policy_version 684725 (0.0005) [2023-12-26 20:20:46,778][105692] Updated weights for policy 0, policy_version 683884 (0.0008) [2023-12-26 20:20:46,804][105620] Updated weights for policy 1, policy_version 684735 (0.0006) [2023-12-26 20:20:46,838][105692] Updated weights for policy 0, policy_version 683894 (0.0008) [2023-12-26 20:20:46,859][105620] Updated weights for policy 1, policy_version 684745 (0.0005) [2023-12-26 20:20:46,896][105692] Updated weights for policy 0, policy_version 683904 (0.0009) [2023-12-26 20:20:46,916][105620] Updated weights for policy 1, policy_version 684755 (0.0009) [2023-12-26 20:20:47,630][105692] Updated weights for policy 0, policy_version 683914 (0.0007) [2023-12-26 20:20:47,633][105620] Updated weights for policy 1, policy_version 684765 (0.0010) [2023-12-26 20:20:47,674][105692] Updated weights for policy 0, policy_version 683924 (0.0005) [2023-12-26 20:20:47,678][105620] Updated weights for policy 1, policy_version 684775 (0.0010) [2023-12-26 20:20:47,726][105692] Updated weights for policy 0, policy_version 683934 (0.0006) [2023-12-26 20:20:47,734][105620] Updated weights for policy 1, policy_version 684785 (0.0011) [2023-12-26 20:20:47,783][105692] Updated weights for policy 0, policy_version 683944 (0.0005) [2023-12-26 20:20:48,462][105692] Updated weights for policy 0, policy_version 683954 (0.0008) [2023-12-26 20:20:48,500][105620] Updated weights for policy 1, policy_version 684795 (0.0010) [2023-12-26 20:20:48,524][105692] Updated weights for policy 0, policy_version 683964 (0.0009) [2023-12-26 20:20:48,563][105620] Updated weights for policy 1, policy_version 684805 (0.0006) [2023-12-26 20:20:48,582][105692] Updated weights for policy 0, policy_version 683974 (0.0009) [2023-12-26 20:20:48,627][105620] Updated weights for policy 1, policy_version 684815 (0.0006) [2023-12-26 20:20:49,266][105620] Updated weights for policy 1, policy_version 684825 (0.0006) [2023-12-26 20:20:49,326][105620] Updated weights for policy 1, policy_version 684835 (0.0008) [2023-12-26 20:20:49,390][105620] Updated weights for policy 1, policy_version 684845 (0.0008) [2023-12-26 20:20:49,410][105692] Updated weights for policy 0, policy_version 683984 (0.0008) [2023-12-26 20:20:49,439][105620] Updated weights for policy 1, policy_version 684855 (0.0006) [2023-12-26 20:20:49,468][105692] Updated weights for policy 0, policy_version 683994 (0.0009) [2023-12-26 20:20:49,527][105692] Updated weights for policy 0, policy_version 684004 (0.0010) [2023-12-26 20:20:50,135][105620] Updated weights for policy 1, policy_version 684865 (0.0009) [2023-12-26 20:20:50,208][105620] Updated weights for policy 1, policy_version 684875 (0.0007) [2023-12-26 20:20:50,279][105620] Updated weights for policy 1, policy_version 684885 (0.0008) [2023-12-26 20:20:50,345][105692] Updated weights for policy 0, policy_version 684014 (0.0010) [2023-12-26 20:20:50,404][105692] Updated weights for policy 0, policy_version 684024 (0.0011) [2023-12-26 20:20:50,463][105692] Updated weights for policy 0, policy_version 684034 (0.0008) [2023-12-26 20:20:50,900][105620] Updated weights for policy 1, policy_version 684895 (0.0007) [2023-12-26 20:20:50,971][105620] Updated weights for policy 1, policy_version 684905 (0.0006) [2023-12-26 20:20:51,032][105620] Updated weights for policy 1, policy_version 684915 (0.0008) [2023-12-26 20:20:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 350494720. Throughput: 0: 9591.4, 1: 9906.8. Samples: 350489964. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 20:20:51,062][104569] Avg episode reward: [(0, '8391.910'), (1, '9261.130')] [2023-12-26 20:20:51,078][105692] Updated weights for policy 0, policy_version 684044 (0.0007) [2023-12-26 20:20:51,139][105692] Updated weights for policy 0, policy_version 684054 (0.0011) [2023-12-26 20:20:51,195][105692] Updated weights for policy 0, policy_version 684064 (0.0011) [2023-12-26 20:20:51,758][105620] Updated weights for policy 1, policy_version 684925 (0.0009) [2023-12-26 20:20:51,821][105620] Updated weights for policy 1, policy_version 684935 (0.0009) [2023-12-26 20:20:51,880][105620] Updated weights for policy 1, policy_version 684945 (0.0009) [2023-12-26 20:20:51,973][105692] Updated weights for policy 0, policy_version 684074 (0.0010) [2023-12-26 20:20:52,036][105692] Updated weights for policy 0, policy_version 684084 (0.0007) [2023-12-26 20:20:52,099][105692] Updated weights for policy 0, policy_version 684094 (0.0007) [2023-12-26 20:20:52,158][105692] Updated weights for policy 0, policy_version 684104 (0.0010) [2023-12-26 20:20:52,559][105620] Updated weights for policy 1, policy_version 684955 (0.0008) [2023-12-26 20:20:52,619][105620] Updated weights for policy 1, policy_version 684965 (0.0005) [2023-12-26 20:20:52,686][105620] Updated weights for policy 1, policy_version 684975 (0.0007) [2023-12-26 20:20:52,941][105692] Updated weights for policy 0, policy_version 684114 (0.0007) [2023-12-26 20:20:53,004][105692] Updated weights for policy 0, policy_version 684124 (0.0009) [2023-12-26 20:20:53,062][105692] Updated weights for policy 0, policy_version 684134 (0.0009) [2023-12-26 20:20:53,410][105620] Updated weights for policy 1, policy_version 684985 (0.0009) [2023-12-26 20:20:53,468][105620] Updated weights for policy 1, policy_version 684995 (0.0006) [2023-12-26 20:20:53,520][105620] Updated weights for policy 1, policy_version 685005 (0.0005) [2023-12-26 20:20:53,581][105620] Updated weights for policy 1, policy_version 685015 (0.0006) [2023-12-26 20:20:53,832][105692] Updated weights for policy 0, policy_version 684144 (0.0009) [2023-12-26 20:20:53,883][105692] Updated weights for policy 0, policy_version 684154 (0.0009) [2023-12-26 20:20:53,939][105692] Updated weights for policy 0, policy_version 684164 (0.0009) [2023-12-26 20:20:54,197][105620] Updated weights for policy 1, policy_version 685025 (0.0008) [2023-12-26 20:20:54,250][105620] Updated weights for policy 1, policy_version 685035 (0.0008) [2023-12-26 20:20:54,299][105620] Updated weights for policy 1, policy_version 685045 (0.0009) [2023-12-26 20:20:54,703][105692] Updated weights for policy 0, policy_version 684174 (0.0009) [2023-12-26 20:20:54,754][105692] Updated weights for policy 0, policy_version 684184 (0.0008) [2023-12-26 20:20:54,818][105692] Updated weights for policy 0, policy_version 684194 (0.0009) [2023-12-26 20:20:55,054][105620] Updated weights for policy 1, policy_version 685056 (0.0010) [2023-12-26 20:20:55,114][105620] Updated weights for policy 1, policy_version 685067 (0.0010) [2023-12-26 20:20:55,170][105620] Updated weights for policy 1, policy_version 685077 (0.0009) [2023-12-26 20:20:55,519][105692] Updated weights for policy 0, policy_version 684204 (0.0010) [2023-12-26 20:20:55,563][105692] Updated weights for policy 0, policy_version 684214 (0.0010) [2023-12-26 20:20:55,621][105692] Updated weights for policy 0, policy_version 684224 (0.0010) [2023-12-26 20:20:55,967][105620] Updated weights for policy 1, policy_version 685087 (0.0008) [2023-12-26 20:20:56,029][105620] Updated weights for policy 1, policy_version 685097 (0.0008) [2023-12-26 20:20:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19605.2). Total num frames: 350593024. Throughput: 0: 9568.9, 1: 9969.0. Samples: 350605284. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 20:20:56,063][104569] Avg episode reward: [(0, '7295.950'), (1, '9168.740')] [2023-12-26 20:20:56,084][105620] Updated weights for policy 1, policy_version 685107 (0.0008) [2023-12-26 20:20:56,375][105692] Updated weights for policy 0, policy_version 684234 (0.0009) [2023-12-26 20:20:56,425][105692] Updated weights for policy 0, policy_version 684244 (0.0010) [2023-12-26 20:20:56,469][105692] Updated weights for policy 0, policy_version 684254 (0.0010) [2023-12-26 20:20:56,516][105692] Updated weights for policy 0, policy_version 684264 (0.0010) [2023-12-26 20:20:56,853][105620] Updated weights for policy 1, policy_version 685117 (0.0008) [2023-12-26 20:20:56,903][105620] Updated weights for policy 1, policy_version 685127 (0.0009) [2023-12-26 20:20:56,950][105620] Updated weights for policy 1, policy_version 685137 (0.0009) [2023-12-26 20:20:57,203][105692] Updated weights for policy 0, policy_version 684274 (0.0005) [2023-12-26 20:20:57,258][105692] Updated weights for policy 0, policy_version 684284 (0.0007) [2023-12-26 20:20:57,312][105692] Updated weights for policy 0, policy_version 684294 (0.0008) [2023-12-26 20:20:57,661][105620] Updated weights for policy 1, policy_version 685147 (0.0008) [2023-12-26 20:20:57,721][105620] Updated weights for policy 1, policy_version 685157 (0.0006) [2023-12-26 20:20:57,775][105620] Updated weights for policy 1, policy_version 685167 (0.0006) [2023-12-26 20:20:57,843][105692] Updated weights for policy 0, policy_version 684304 (0.0009) [2023-12-26 20:20:57,900][105692] Updated weights for policy 0, policy_version 684314 (0.0010) [2023-12-26 20:20:57,963][105692] Updated weights for policy 0, policy_version 684324 (0.0010) [2023-12-26 20:20:58,438][105620] Updated weights for policy 1, policy_version 685177 (0.0007) [2023-12-26 20:20:58,504][105620] Updated weights for policy 1, policy_version 685187 (0.0008) [2023-12-26 20:20:58,566][105620] Updated weights for policy 1, policy_version 685197 (0.0008) [2023-12-26 20:20:58,630][105620] Updated weights for policy 1, policy_version 685207 (0.0008) [2023-12-26 20:20:58,720][105692] Updated weights for policy 0, policy_version 684334 (0.0009) [2023-12-26 20:20:58,782][105692] Updated weights for policy 0, policy_version 684344 (0.0008) [2023-12-26 20:20:58,850][105692] Updated weights for policy 0, policy_version 684354 (0.0009) [2023-12-26 20:20:59,358][105620] Updated weights for policy 1, policy_version 685217 (0.0008) [2023-12-26 20:20:59,421][105620] Updated weights for policy 1, policy_version 685227 (0.0007) [2023-12-26 20:20:59,485][105620] Updated weights for policy 1, policy_version 685237 (0.0005) [2023-12-26 20:20:59,590][105692] Updated weights for policy 0, policy_version 684364 (0.0009) [2023-12-26 20:20:59,654][105692] Updated weights for policy 0, policy_version 684374 (0.0009) [2023-12-26 20:20:59,716][105692] Updated weights for policy 0, policy_version 684384 (0.0009) [2023-12-26 20:21:00,093][105620] Updated weights for policy 1, policy_version 685247 (0.0006) [2023-12-26 20:21:00,150][105620] Updated weights for policy 1, policy_version 685257 (0.0007) [2023-12-26 20:21:00,201][105620] Updated weights for policy 1, policy_version 685267 (0.0009) [2023-12-26 20:21:00,552][105692] Updated weights for policy 0, policy_version 684394 (0.0009) [2023-12-26 20:21:00,602][105692] Updated weights for policy 0, policy_version 684404 (0.0009) [2023-12-26 20:21:00,664][105692] Updated weights for policy 0, policy_version 684414 (0.0009) [2023-12-26 20:21:00,716][105692] Updated weights for policy 0, policy_version 684424 (0.0008) [2023-12-26 20:21:00,870][105620] Updated weights for policy 1, policy_version 685277 (0.0007) [2023-12-26 20:21:00,925][105620] Updated weights for policy 1, policy_version 685287 (0.0005) [2023-12-26 20:21:00,978][105620] Updated weights for policy 1, policy_version 685297 (0.0005) [2023-12-26 20:21:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 350699520. Throughput: 0: 9614.8, 1: 9944.8. Samples: 350664744. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 20:21:01,062][104569] Avg episode reward: [(0, '7901.456'), (1, '8712.576')] [2023-12-26 20:21:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000684424_175243264.pth... [2023-12-26 20:21:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000685304_175456256.pth... [2023-12-26 20:21:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000683304_174956544.pth [2023-12-26 20:21:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000684120_175153152.pth [2023-12-26 20:21:01,389][105692] Updated weights for policy 0, policy_version 684434 (0.0009) [2023-12-26 20:21:01,455][105692] Updated weights for policy 0, policy_version 684444 (0.0009) [2023-12-26 20:21:01,516][105692] Updated weights for policy 0, policy_version 684454 (0.0009) [2023-12-26 20:21:01,719][105620] Updated weights for policy 1, policy_version 685307 (0.0010) [2023-12-26 20:21:01,776][105620] Updated weights for policy 1, policy_version 685317 (0.0007) [2023-12-26 20:21:01,825][105620] Updated weights for policy 1, policy_version 685327 (0.0005) [2023-12-26 20:21:02,246][105692] Updated weights for policy 0, policy_version 684464 (0.0006) [2023-12-26 20:21:02,298][105692] Updated weights for policy 0, policy_version 684474 (0.0010) [2023-12-26 20:21:02,366][105692] Updated weights for policy 0, policy_version 684484 (0.0009) [2023-12-26 20:21:02,463][105620] Updated weights for policy 1, policy_version 685337 (0.0006) [2023-12-26 20:21:02,513][105620] Updated weights for policy 1, policy_version 685347 (0.0005) [2023-12-26 20:21:02,582][105620] Updated weights for policy 1, policy_version 685357 (0.0008) [2023-12-26 20:21:02,651][105620] Updated weights for policy 1, policy_version 685367 (0.0008) [2023-12-26 20:21:03,084][105692] Updated weights for policy 0, policy_version 684494 (0.0007) [2023-12-26 20:21:03,136][105692] Updated weights for policy 0, policy_version 684504 (0.0008) [2023-12-26 20:21:03,193][105692] Updated weights for policy 0, policy_version 684514 (0.0006) [2023-12-26 20:21:03,211][105620] Updated weights for policy 1, policy_version 685377 (0.0010) [2023-12-26 20:21:03,260][105620] Updated weights for policy 1, policy_version 685387 (0.0010) [2023-12-26 20:21:03,319][105620] Updated weights for policy 1, policy_version 685397 (0.0011) [2023-12-26 20:21:03,974][105692] Updated weights for policy 0, policy_version 684524 (0.0006) [2023-12-26 20:21:04,035][105692] Updated weights for policy 0, policy_version 684534 (0.0008) [2023-12-26 20:21:04,088][105620] Updated weights for policy 1, policy_version 685407 (0.0011) [2023-12-26 20:21:04,098][105692] Updated weights for policy 0, policy_version 684544 (0.0010) [2023-12-26 20:21:04,153][105620] Updated weights for policy 1, policy_version 685417 (0.0011) [2023-12-26 20:21:04,221][105620] Updated weights for policy 1, policy_version 685427 (0.0011) [2023-12-26 20:21:04,773][105692] Updated weights for policy 0, policy_version 684554 (0.0010) [2023-12-26 20:21:04,839][105692] Updated weights for policy 0, policy_version 684564 (0.0005) [2023-12-26 20:21:04,907][105692] Updated weights for policy 0, policy_version 684574 (0.0007) [2023-12-26 20:21:04,948][105620] Updated weights for policy 1, policy_version 685437 (0.0009) [2023-12-26 20:21:04,966][105692] Updated weights for policy 0, policy_version 684584 (0.0008) [2023-12-26 20:21:05,007][105620] Updated weights for policy 1, policy_version 685447 (0.0010) [2023-12-26 20:21:05,060][105620] Updated weights for policy 1, policy_version 685457 (0.0010) [2023-12-26 20:21:05,564][105692] Updated weights for policy 0, policy_version 684594 (0.0008) [2023-12-26 20:21:05,615][105692] Updated weights for policy 0, policy_version 684604 (0.0009) [2023-12-26 20:21:05,671][105692] Updated weights for policy 0, policy_version 684614 (0.0008) [2023-12-26 20:21:05,746][105620] Updated weights for policy 1, policy_version 685467 (0.0010) [2023-12-26 20:21:05,804][105620] Updated weights for policy 1, policy_version 685477 (0.0009) [2023-12-26 20:21:05,862][105620] Updated weights for policy 1, policy_version 685487 (0.0009) [2023-12-26 20:21:06,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 350797824. Throughput: 0: 9640.5, 1: 9948.8. Samples: 350782236. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 20:21:06,062][104569] Avg episode reward: [(0, '8904.635'), (1, '8804.720')] [2023-12-26 20:21:06,413][105692] Updated weights for policy 0, policy_version 684624 (0.0010) [2023-12-26 20:21:06,472][105692] Updated weights for policy 0, policy_version 684634 (0.0010) [2023-12-26 20:21:06,535][105692] Updated weights for policy 0, policy_version 684644 (0.0010) [2023-12-26 20:21:06,638][105620] Updated weights for policy 1, policy_version 685497 (0.0009) [2023-12-26 20:21:06,704][105620] Updated weights for policy 1, policy_version 685507 (0.0007) [2023-12-26 20:21:06,769][105620] Updated weights for policy 1, policy_version 685517 (0.0008) [2023-12-26 20:21:06,832][105620] Updated weights for policy 1, policy_version 685527 (0.0006) [2023-12-26 20:21:07,176][105692] Updated weights for policy 0, policy_version 684654 (0.0008) [2023-12-26 20:21:07,244][105692] Updated weights for policy 0, policy_version 684664 (0.0005) [2023-12-26 20:21:07,304][105692] Updated weights for policy 0, policy_version 684674 (0.0005) [2023-12-26 20:21:07,445][105620] Updated weights for policy 1, policy_version 685537 (0.0008) [2023-12-26 20:21:07,508][105620] Updated weights for policy 1, policy_version 685547 (0.0009) [2023-12-26 20:21:07,567][105620] Updated weights for policy 1, policy_version 685557 (0.0006) [2023-12-26 20:21:07,954][105692] Updated weights for policy 0, policy_version 684684 (0.0007) [2023-12-26 20:21:08,018][105692] Updated weights for policy 0, policy_version 684694 (0.0008) [2023-12-26 20:21:08,079][105692] Updated weights for policy 0, policy_version 684704 (0.0010) [2023-12-26 20:21:08,264][105620] Updated weights for policy 1, policy_version 685567 (0.0008) [2023-12-26 20:21:08,321][105620] Updated weights for policy 1, policy_version 685577 (0.0010) [2023-12-26 20:21:08,385][105620] Updated weights for policy 1, policy_version 685587 (0.0009) [2023-12-26 20:21:08,764][105692] Updated weights for policy 0, policy_version 684714 (0.0010) [2023-12-26 20:21:08,826][105692] Updated weights for policy 0, policy_version 684724 (0.0010) [2023-12-26 20:21:08,885][105692] Updated weights for policy 0, policy_version 684734 (0.0011) [2023-12-26 20:21:08,948][105692] Updated weights for policy 0, policy_version 684744 (0.0010) [2023-12-26 20:21:09,171][105620] Updated weights for policy 1, policy_version 685597 (0.0009) [2023-12-26 20:21:09,233][105620] Updated weights for policy 1, policy_version 685607 (0.0010) [2023-12-26 20:21:09,292][105620] Updated weights for policy 1, policy_version 685617 (0.0011) [2023-12-26 20:21:09,685][105692] Updated weights for policy 0, policy_version 684754 (0.0008) [2023-12-26 20:21:09,749][105692] Updated weights for policy 0, policy_version 684764 (0.0008) [2023-12-26 20:21:09,812][105692] Updated weights for policy 0, policy_version 684774 (0.0008) [2023-12-26 20:21:10,077][105620] Updated weights for policy 1, policy_version 685627 (0.0009) [2023-12-26 20:21:10,143][105620] Updated weights for policy 1, policy_version 685637 (0.0011) [2023-12-26 20:21:10,211][105620] Updated weights for policy 1, policy_version 685647 (0.0010) [2023-12-26 20:21:10,619][105692] Updated weights for policy 0, policy_version 684784 (0.0006) [2023-12-26 20:21:10,684][105692] Updated weights for policy 0, policy_version 684794 (0.0006) [2023-12-26 20:21:10,743][105692] Updated weights for policy 0, policy_version 684804 (0.0008) [2023-12-26 20:21:10,920][105620] Updated weights for policy 1, policy_version 685657 (0.0010) [2023-12-26 20:21:10,982][105620] Updated weights for policy 1, policy_version 685667 (0.0010) [2023-12-26 20:21:11,037][105620] Updated weights for policy 1, policy_version 685677 (0.0011) [2023-12-26 20:21:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 350887936. Throughput: 0: 9668.3, 1: 9928.0. Samples: 350899180. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 20:21:11,063][104569] Avg episode reward: [(0, '8988.376'), (1, '8534.684')] [2023-12-26 20:21:11,103][105620] Updated weights for policy 1, policy_version 685687 (0.0011) [2023-12-26 20:21:11,463][105692] Updated weights for policy 0, policy_version 684814 (0.0006) [2023-12-26 20:21:11,515][105692] Updated weights for policy 0, policy_version 684824 (0.0005) [2023-12-26 20:21:11,574][105692] Updated weights for policy 0, policy_version 684834 (0.0006) [2023-12-26 20:21:11,896][105620] Updated weights for policy 1, policy_version 685697 (0.0008) [2023-12-26 20:21:11,958][105620] Updated weights for policy 1, policy_version 685707 (0.0009) [2023-12-26 20:21:12,027][105620] Updated weights for policy 1, policy_version 685717 (0.0006) [2023-12-26 20:21:12,347][105692] Updated weights for policy 0, policy_version 684844 (0.0008) [2023-12-26 20:21:12,413][105692] Updated weights for policy 0, policy_version 684854 (0.0009) [2023-12-26 20:21:12,473][105692] Updated weights for policy 0, policy_version 684864 (0.0009) [2023-12-26 20:21:12,678][105620] Updated weights for policy 1, policy_version 685727 (0.0008) [2023-12-26 20:21:12,737][105620] Updated weights for policy 1, policy_version 685737 (0.0009) [2023-12-26 20:21:12,799][105620] Updated weights for policy 1, policy_version 685747 (0.0009) [2023-12-26 20:21:13,168][105692] Updated weights for policy 0, policy_version 684874 (0.0008) [2023-12-26 20:21:13,231][105692] Updated weights for policy 0, policy_version 684884 (0.0007) [2023-12-26 20:21:13,282][105692] Updated weights for policy 0, policy_version 684894 (0.0009) [2023-12-26 20:21:13,338][105692] Updated weights for policy 0, policy_version 684904 (0.0009) [2023-12-26 20:21:13,646][105620] Updated weights for policy 1, policy_version 685757 (0.0009) [2023-12-26 20:21:13,703][105620] Updated weights for policy 1, policy_version 685767 (0.0008) [2023-12-26 20:21:13,761][105620] Updated weights for policy 1, policy_version 685777 (0.0009) [2023-12-26 20:21:13,935][105692] Updated weights for policy 0, policy_version 684914 (0.0009) [2023-12-26 20:21:14,000][105692] Updated weights for policy 0, policy_version 684924 (0.0009) [2023-12-26 20:21:14,055][105692] Updated weights for policy 0, policy_version 684934 (0.0010) [2023-12-26 20:21:14,608][105620] Updated weights for policy 1, policy_version 685787 (0.0009) [2023-12-26 20:21:14,666][105620] Updated weights for policy 1, policy_version 685797 (0.0009) [2023-12-26 20:21:14,733][105620] Updated weights for policy 1, policy_version 685807 (0.0009) [2023-12-26 20:21:14,759][105692] Updated weights for policy 0, policy_version 684944 (0.0007) [2023-12-26 20:21:14,816][105692] Updated weights for policy 0, policy_version 684954 (0.0009) [2023-12-26 20:21:14,868][105692] Updated weights for policy 0, policy_version 684964 (0.0009) [2023-12-26 20:21:15,509][105620] Updated weights for policy 1, policy_version 685817 (0.0008) [2023-12-26 20:21:15,556][105620] Updated weights for policy 1, policy_version 685827 (0.0009) [2023-12-26 20:21:15,615][105692] Updated weights for policy 0, policy_version 684974 (0.0008) [2023-12-26 20:21:15,621][105620] Updated weights for policy 1, policy_version 685837 (0.0007) [2023-12-26 20:21:15,668][105692] Updated weights for policy 0, policy_version 684984 (0.0007) [2023-12-26 20:21:15,673][105620] Updated weights for policy 1, policy_version 685847 (0.0006) [2023-12-26 20:21:15,728][105692] Updated weights for policy 0, policy_version 684994 (0.0008) [2023-12-26 20:21:16,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 350986240. Throughput: 0: 9717.2, 1: 9818.4. Samples: 350955196. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 20:21:16,063][104569] Avg episode reward: [(0, '8985.368'), (1, '8170.799')] [2023-12-26 20:21:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000685000_175390720.pth... [2023-12-26 20:21:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000685848_175595520.pth... [2023-12-26 20:21:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000683880_175104000.pth [2023-12-26 20:21:16,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000684696_175300608.pth [2023-12-26 20:21:16,331][105692] Updated weights for policy 0, policy_version 685004 (0.0009) [2023-12-26 20:21:16,392][105692] Updated weights for policy 0, policy_version 685014 (0.0009) [2023-12-26 20:21:16,447][105692] Updated weights for policy 0, policy_version 685024 (0.0008) [2023-12-26 20:21:16,527][105620] Updated weights for policy 1, policy_version 685857 (0.0008) [2023-12-26 20:21:16,583][105620] Updated weights for policy 1, policy_version 685867 (0.0008) [2023-12-26 20:21:16,649][105620] Updated weights for policy 1, policy_version 685877 (0.0009) [2023-12-26 20:21:17,060][105692] Updated weights for policy 0, policy_version 685034 (0.0008) [2023-12-26 20:21:17,114][105692] Updated weights for policy 0, policy_version 685044 (0.0009) [2023-12-26 20:21:17,165][105692] Updated weights for policy 0, policy_version 685054 (0.0009) [2023-12-26 20:21:17,219][105692] Updated weights for policy 0, policy_version 685064 (0.0009) [2023-12-26 20:21:17,489][105620] Updated weights for policy 1, policy_version 685887 (0.0008) [2023-12-26 20:21:17,540][105620] Updated weights for policy 1, policy_version 685897 (0.0009) [2023-12-26 20:21:17,588][105620] Updated weights for policy 1, policy_version 685907 (0.0009) [2023-12-26 20:21:17,880][105692] Updated weights for policy 0, policy_version 685074 (0.0005) [2023-12-26 20:21:17,935][105692] Updated weights for policy 0, policy_version 685084 (0.0005) [2023-12-26 20:21:17,996][105692] Updated weights for policy 0, policy_version 685094 (0.0008) [2023-12-26 20:21:18,336][105620] Updated weights for policy 1, policy_version 685918 (0.0008) [2023-12-26 20:21:18,399][105620] Updated weights for policy 1, policy_version 685928 (0.0009) [2023-12-26 20:21:18,461][105620] Updated weights for policy 1, policy_version 685938 (0.0009) [2023-12-26 20:21:18,795][105692] Updated weights for policy 0, policy_version 685104 (0.0009) [2023-12-26 20:21:18,857][105692] Updated weights for policy 0, policy_version 685114 (0.0009) [2023-12-26 20:21:18,924][105692] Updated weights for policy 0, policy_version 685124 (0.0010) [2023-12-26 20:21:19,103][105620] Updated weights for policy 1, policy_version 685948 (0.0008) [2023-12-26 20:21:19,158][105620] Updated weights for policy 1, policy_version 685958 (0.0005) [2023-12-26 20:21:19,213][105620] Updated weights for policy 1, policy_version 685968 (0.0006) [2023-12-26 20:21:19,757][105692] Updated weights for policy 0, policy_version 685134 (0.0008) [2023-12-26 20:21:19,821][105692] Updated weights for policy 0, policy_version 685144 (0.0006) [2023-12-26 20:21:19,885][105692] Updated weights for policy 0, policy_version 685154 (0.0010) [2023-12-26 20:21:19,969][105620] Updated weights for policy 1, policy_version 685978 (0.0009) [2023-12-26 20:21:20,028][105620] Updated weights for policy 1, policy_version 685988 (0.0008) [2023-12-26 20:21:20,089][105620] Updated weights for policy 1, policy_version 685998 (0.0009) [2023-12-26 20:21:20,143][105620] Updated weights for policy 1, policy_version 686008 (0.0010) [2023-12-26 20:21:20,543][105692] Updated weights for policy 0, policy_version 685164 (0.0009) [2023-12-26 20:21:20,629][105692] Updated weights for policy 0, policy_version 685174 (0.0010) [2023-12-26 20:21:20,696][105692] Updated weights for policy 0, policy_version 685184 (0.0010) [2023-12-26 20:21:20,983][105620] Updated weights for policy 1, policy_version 686018 (0.0009) [2023-12-26 20:21:21,046][105620] Updated weights for policy 1, policy_version 686028 (0.0009) [2023-12-26 20:21:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 351076352. Throughput: 0: 9738.9, 1: 9719.6. Samples: 351070312. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 20:21:21,063][104569] Avg episode reward: [(0, '8895.873'), (1, '8806.534')] [2023-12-26 20:21:21,112][105620] Updated weights for policy 1, policy_version 686038 (0.0008) [2023-12-26 20:21:21,483][105692] Updated weights for policy 0, policy_version 685194 (0.0009) [2023-12-26 20:21:21,552][105692] Updated weights for policy 0, policy_version 685204 (0.0008) [2023-12-26 20:21:21,621][105692] Updated weights for policy 0, policy_version 685214 (0.0006) [2023-12-26 20:21:21,686][105692] Updated weights for policy 0, policy_version 685224 (0.0008) [2023-12-26 20:21:21,886][105620] Updated weights for policy 1, policy_version 686048 (0.0007) [2023-12-26 20:21:21,950][105620] Updated weights for policy 1, policy_version 686058 (0.0007) [2023-12-26 20:21:22,011][105620] Updated weights for policy 1, policy_version 686068 (0.0008) [2023-12-26 20:21:22,494][105692] Updated weights for policy 0, policy_version 685234 (0.0009) [2023-12-26 20:21:22,549][105692] Updated weights for policy 0, policy_version 685244 (0.0009) [2023-12-26 20:21:22,607][105692] Updated weights for policy 0, policy_version 685254 (0.0009) [2023-12-26 20:21:22,668][105620] Updated weights for policy 1, policy_version 686078 (0.0007) [2023-12-26 20:21:22,732][105620] Updated weights for policy 1, policy_version 686088 (0.0007) [2023-12-26 20:21:22,797][105620] Updated weights for policy 1, policy_version 686098 (0.0009) [2023-12-26 20:21:23,384][105692] Updated weights for policy 0, policy_version 685264 (0.0010) [2023-12-26 20:21:23,447][105692] Updated weights for policy 0, policy_version 685274 (0.0010) [2023-12-26 20:21:23,500][105692] Updated weights for policy 0, policy_version 685284 (0.0010) [2023-12-26 20:21:23,540][105620] Updated weights for policy 1, policy_version 686108 (0.0008) [2023-12-26 20:21:23,603][105620] Updated weights for policy 1, policy_version 686118 (0.0006) [2023-12-26 20:21:23,666][105620] Updated weights for policy 1, policy_version 686128 (0.0009) [2023-12-26 20:21:24,181][105692] Updated weights for policy 0, policy_version 685294 (0.0008) [2023-12-26 20:21:24,243][105692] Updated weights for policy 0, policy_version 685304 (0.0005) [2023-12-26 20:21:24,303][105692] Updated weights for policy 0, policy_version 685314 (0.0005) [2023-12-26 20:21:24,372][105620] Updated weights for policy 1, policy_version 686138 (0.0009) [2023-12-26 20:21:24,433][105620] Updated weights for policy 1, policy_version 686148 (0.0008) [2023-12-26 20:21:24,501][105620] Updated weights for policy 1, policy_version 686158 (0.0010) [2023-12-26 20:21:24,572][105620] Updated weights for policy 1, policy_version 686168 (0.0010) [2023-12-26 20:21:24,915][105692] Updated weights for policy 0, policy_version 685324 (0.0007) [2023-12-26 20:21:24,966][105692] Updated weights for policy 0, policy_version 685334 (0.0010) [2023-12-26 20:21:25,021][105692] Updated weights for policy 0, policy_version 685344 (0.0010) [2023-12-26 20:21:25,332][105620] Updated weights for policy 1, policy_version 686178 (0.0005) [2023-12-26 20:21:25,399][105620] Updated weights for policy 1, policy_version 686188 (0.0005) [2023-12-26 20:21:25,461][105620] Updated weights for policy 1, policy_version 686198 (0.0009) [2023-12-26 20:21:25,732][105692] Updated weights for policy 0, policy_version 685354 (0.0010) [2023-12-26 20:21:25,783][105692] Updated weights for policy 0, policy_version 685364 (0.0009) [2023-12-26 20:21:25,842][105692] Updated weights for policy 0, policy_version 685374 (0.0010) [2023-12-26 20:21:25,900][105692] Updated weights for policy 0, policy_version 685384 (0.0010) [2023-12-26 20:21:26,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 351174656. Throughput: 0: 9692.6, 1: 9668.4. Samples: 351183668. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 20:21:26,062][104569] Avg episode reward: [(0, '8984.437'), (1, '9080.171')] [2023-12-26 20:21:26,120][105620] Updated weights for policy 1, policy_version 686208 (0.0007) [2023-12-26 20:21:26,185][105620] Updated weights for policy 1, policy_version 686218 (0.0011) [2023-12-26 20:21:26,237][105620] Updated weights for policy 1, policy_version 686228 (0.0010) [2023-12-26 20:21:26,512][105692] Updated weights for policy 0, policy_version 685394 (0.0011) [2023-12-26 20:21:26,567][105692] Updated weights for policy 0, policy_version 685404 (0.0010) [2023-12-26 20:21:26,628][105692] Updated weights for policy 0, policy_version 685414 (0.0010) [2023-12-26 20:21:26,951][105620] Updated weights for policy 1, policy_version 686238 (0.0010) [2023-12-26 20:21:27,003][105620] Updated weights for policy 1, policy_version 686248 (0.0010) [2023-12-26 20:21:27,063][105620] Updated weights for policy 1, policy_version 686258 (0.0010) [2023-12-26 20:21:27,171][105692] Updated weights for policy 0, policy_version 685424 (0.0005) [2023-12-26 20:21:27,228][105692] Updated weights for policy 0, policy_version 685434 (0.0005) [2023-12-26 20:21:27,286][105692] Updated weights for policy 0, policy_version 685444 (0.0010) [2023-12-26 20:21:27,817][105692] Updated weights for policy 0, policy_version 685454 (0.0008) [2023-12-26 20:21:27,818][105620] Updated weights for policy 1, policy_version 686268 (0.0010) [2023-12-26 20:21:27,861][105692] Updated weights for policy 0, policy_version 685464 (0.0005) [2023-12-26 20:21:27,880][105620] Updated weights for policy 1, policy_version 686278 (0.0010) [2023-12-26 20:21:27,904][105692] Updated weights for policy 0, policy_version 685474 (0.0005) [2023-12-26 20:21:27,937][105620] Updated weights for policy 1, policy_version 686288 (0.0010) [2023-12-26 20:21:28,538][105620] Updated weights for policy 1, policy_version 686298 (0.0009) [2023-12-26 20:21:28,595][105620] Updated weights for policy 1, policy_version 686308 (0.0007) [2023-12-26 20:21:28,660][105620] Updated weights for policy 1, policy_version 686318 (0.0010) [2023-12-26 20:21:28,705][105692] Updated weights for policy 0, policy_version 685484 (0.0006) [2023-12-26 20:21:28,717][105620] Updated weights for policy 1, policy_version 686328 (0.0008) [2023-12-26 20:21:28,765][105692] Updated weights for policy 0, policy_version 685494 (0.0006) [2023-12-26 20:21:28,819][105692] Updated weights for policy 0, policy_version 685504 (0.0009) [2023-12-26 20:21:29,351][105620] Updated weights for policy 1, policy_version 686338 (0.0009) [2023-12-26 20:21:29,418][105620] Updated weights for policy 1, policy_version 686348 (0.0010) [2023-12-26 20:21:29,460][105692] Updated weights for policy 0, policy_version 685514 (0.0007) [2023-12-26 20:21:29,480][105620] Updated weights for policy 1, policy_version 686358 (0.0011) [2023-12-26 20:21:29,512][105692] Updated weights for policy 0, policy_version 685524 (0.0010) [2023-12-26 20:21:29,560][105692] Updated weights for policy 0, policy_version 685534 (0.0010) [2023-12-26 20:21:29,619][105692] Updated weights for policy 0, policy_version 685544 (0.0010) [2023-12-26 20:21:30,271][105620] Updated weights for policy 1, policy_version 686368 (0.0008) [2023-12-26 20:21:30,322][105620] Updated weights for policy 1, policy_version 686378 (0.0008) [2023-12-26 20:21:30,372][105692] Updated weights for policy 0, policy_version 685554 (0.0011) [2023-12-26 20:21:30,378][105620] Updated weights for policy 1, policy_version 686388 (0.0007) [2023-12-26 20:21:30,425][105692] Updated weights for policy 0, policy_version 685564 (0.0010) [2023-12-26 20:21:30,473][105692] Updated weights for policy 0, policy_version 685574 (0.0010) [2023-12-26 20:21:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 351272960. Throughput: 0: 9743.2, 1: 9704.6. Samples: 351246928. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 20:21:31,063][104569] Avg episode reward: [(0, '8442.150'), (1, '8712.441')] [2023-12-26 20:21:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000686392_175734784.pth... [2023-12-26 20:21:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000685576_175538176.pth... [2023-12-26 20:21:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000685304_175456256.pth [2023-12-26 20:21:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000684424_175243264.pth [2023-12-26 20:21:31,146][105620] Updated weights for policy 1, policy_version 686398 (0.0008) [2023-12-26 20:21:31,208][105620] Updated weights for policy 1, policy_version 686408 (0.0008) [2023-12-26 20:21:31,238][105586] KL-divergence is very high: 144.6268 [2023-12-26 20:21:31,246][105692] Updated weights for policy 0, policy_version 685584 (0.0011) [2023-12-26 20:21:31,274][105620] Updated weights for policy 1, policy_version 686418 (0.0007) [2023-12-26 20:21:31,289][105586] KL-divergence is very high: 245.3044 [2023-12-26 20:21:31,310][105692] Updated weights for policy 0, policy_version 685594 (0.0011) [2023-12-26 20:21:31,370][105692] Updated weights for policy 0, policy_version 685604 (0.0011) [2023-12-26 20:21:32,009][105620] Updated weights for policy 1, policy_version 686428 (0.0010) [2023-12-26 20:21:32,066][105620] Updated weights for policy 1, policy_version 686438 (0.0007) [2023-12-26 20:21:32,068][105692] Updated weights for policy 0, policy_version 685614 (0.0011) [2023-12-26 20:21:32,121][105620] Updated weights for policy 1, policy_version 686448 (0.0006) [2023-12-26 20:21:32,126][105692] Updated weights for policy 0, policy_version 685624 (0.0011) [2023-12-26 20:21:32,184][105692] Updated weights for policy 0, policy_version 685634 (0.0010) [2023-12-26 20:21:32,835][105620] Updated weights for policy 1, policy_version 686458 (0.0007) [2023-12-26 20:21:32,896][105620] Updated weights for policy 1, policy_version 686468 (0.0008) [2023-12-26 20:21:32,931][105692] Updated weights for policy 0, policy_version 685644 (0.0010) [2023-12-26 20:21:32,953][105620] Updated weights for policy 1, policy_version 686478 (0.0007) [2023-12-26 20:21:32,986][105692] Updated weights for policy 0, policy_version 685654 (0.0010) [2023-12-26 20:21:33,012][105620] Updated weights for policy 1, policy_version 686488 (0.0005) [2023-12-26 20:21:33,044][105692] Updated weights for policy 0, policy_version 685664 (0.0010) [2023-12-26 20:21:33,658][105692] Updated weights for policy 0, policy_version 685674 (0.0010) [2023-12-26 20:21:33,705][105692] Updated weights for policy 0, policy_version 685684 (0.0010) [2023-12-26 20:21:33,753][105692] Updated weights for policy 0, policy_version 685694 (0.0009) [2023-12-26 20:21:33,775][105620] Updated weights for policy 1, policy_version 686498 (0.0008) [2023-12-26 20:21:33,798][105692] Updated weights for policy 0, policy_version 685704 (0.0007) [2023-12-26 20:21:33,829][105620] Updated weights for policy 1, policy_version 686508 (0.0009) [2023-12-26 20:21:33,881][105620] Updated weights for policy 1, policy_version 686518 (0.0009) [2023-12-26 20:21:34,572][105692] Updated weights for policy 0, policy_version 685714 (0.0008) [2023-12-26 20:21:34,607][105620] Updated weights for policy 1, policy_version 686528 (0.0006) [2023-12-26 20:21:34,630][105692] Updated weights for policy 0, policy_version 685724 (0.0006) [2023-12-26 20:21:34,667][105620] Updated weights for policy 1, policy_version 686538 (0.0006) [2023-12-26 20:21:34,692][105692] Updated weights for policy 0, policy_version 685734 (0.0006) [2023-12-26 20:21:34,737][105620] Updated weights for policy 1, policy_version 686548 (0.0010) [2023-12-26 20:21:35,403][105620] Updated weights for policy 1, policy_version 686558 (0.0008) [2023-12-26 20:21:35,439][105692] Updated weights for policy 0, policy_version 685744 (0.0008) [2023-12-26 20:21:35,466][105620] Updated weights for policy 1, policy_version 686568 (0.0007) [2023-12-26 20:21:35,491][105692] Updated weights for policy 0, policy_version 685754 (0.0009) [2023-12-26 20:21:35,515][105620] Updated weights for policy 1, policy_version 686578 (0.0007) [2023-12-26 20:21:35,537][105692] Updated weights for policy 0, policy_version 685764 (0.0008) [2023-12-26 20:21:36,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 351371264. Throughput: 0: 9808.8, 1: 9589.6. Samples: 351362896. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 20:21:36,063][104569] Avg episode reward: [(0, '8623.756'), (1, '7778.014')] [2023-12-26 20:21:36,185][105620] Updated weights for policy 1, policy_version 686588 (0.0007) [2023-12-26 20:21:36,244][105620] Updated weights for policy 1, policy_version 686598 (0.0008) [2023-12-26 20:21:36,302][105620] Updated weights for policy 1, policy_version 686608 (0.0009) [2023-12-26 20:21:36,320][105692] Updated weights for policy 0, policy_version 685774 (0.0008) [2023-12-26 20:21:36,379][105692] Updated weights for policy 0, policy_version 685784 (0.0008) [2023-12-26 20:21:36,441][105692] Updated weights for policy 0, policy_version 685794 (0.0008) [2023-12-26 20:21:37,039][105620] Updated weights for policy 1, policy_version 686618 (0.0006) [2023-12-26 20:21:37,086][105620] Updated weights for policy 1, policy_version 686628 (0.0009) [2023-12-26 20:21:37,133][105620] Updated weights for policy 1, policy_version 686638 (0.0009) [2023-12-26 20:21:37,180][105620] Updated weights for policy 1, policy_version 686648 (0.0008) [2023-12-26 20:21:37,204][105692] Updated weights for policy 0, policy_version 685804 (0.0009) [2023-12-26 20:21:37,261][105692] Updated weights for policy 0, policy_version 685814 (0.0009) [2023-12-26 20:21:37,323][105692] Updated weights for policy 0, policy_version 685824 (0.0009) [2023-12-26 20:21:37,884][105620] Updated weights for policy 1, policy_version 686658 (0.0008) [2023-12-26 20:21:37,949][105620] Updated weights for policy 1, policy_version 686668 (0.0007) [2023-12-26 20:21:37,983][105692] Updated weights for policy 0, policy_version 685834 (0.0009) [2023-12-26 20:21:38,018][105620] Updated weights for policy 1, policy_version 686678 (0.0006) [2023-12-26 20:21:38,036][105692] Updated weights for policy 0, policy_version 685844 (0.0007) [2023-12-26 20:21:38,095][105692] Updated weights for policy 0, policy_version 685854 (0.0009) [2023-12-26 20:21:38,153][105692] Updated weights for policy 0, policy_version 685864 (0.0009) [2023-12-26 20:21:38,608][105620] Updated weights for policy 1, policy_version 686688 (0.0006) [2023-12-26 20:21:38,654][105620] Updated weights for policy 1, policy_version 686698 (0.0005) [2023-12-26 20:21:38,700][105620] Updated weights for policy 1, policy_version 686708 (0.0007) [2023-12-26 20:21:38,933][105692] Updated weights for policy 0, policy_version 685874 (0.0009) [2023-12-26 20:21:38,981][105692] Updated weights for policy 0, policy_version 685884 (0.0008) [2023-12-26 20:21:39,034][105692] Updated weights for policy 0, policy_version 685894 (0.0010) [2023-12-26 20:21:39,398][105620] Updated weights for policy 1, policy_version 686718 (0.0008) [2023-12-26 20:21:39,461][105620] Updated weights for policy 1, policy_version 686728 (0.0009) [2023-12-26 20:21:39,526][105620] Updated weights for policy 1, policy_version 686738 (0.0010) [2023-12-26 20:21:39,780][105692] Updated weights for policy 0, policy_version 685904 (0.0009) [2023-12-26 20:21:39,843][105692] Updated weights for policy 0, policy_version 685914 (0.0010) [2023-12-26 20:21:39,911][105692] Updated weights for policy 0, policy_version 685924 (0.0009) [2023-12-26 20:21:40,340][105620] Updated weights for policy 1, policy_version 686748 (0.0008) [2023-12-26 20:21:40,392][105620] Updated weights for policy 1, policy_version 686758 (0.0005) [2023-12-26 20:21:40,447][105620] Updated weights for policy 1, policy_version 686768 (0.0008) [2023-12-26 20:21:40,673][105692] Updated weights for policy 0, policy_version 685934 (0.0009) [2023-12-26 20:21:40,732][105692] Updated weights for policy 0, policy_version 685944 (0.0006) [2023-12-26 20:21:40,788][105692] Updated weights for policy 0, policy_version 685954 (0.0005) [2023-12-26 20:21:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 351469568. Throughput: 0: 9801.9, 1: 9606.4. Samples: 351478656. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 20:21:41,062][104569] Avg episode reward: [(0, '8630.941'), (1, '7655.619')] [2023-12-26 20:21:41,238][105620] Updated weights for policy 1, policy_version 686778 (0.0008) [2023-12-26 20:21:41,287][105620] Updated weights for policy 1, policy_version 686788 (0.0009) [2023-12-26 20:21:41,337][105620] Updated weights for policy 1, policy_version 686798 (0.0006) [2023-12-26 20:21:41,416][105620] Updated weights for policy 1, policy_version 686808 (0.0008) [2023-12-26 20:21:41,431][105692] Updated weights for policy 0, policy_version 685964 (0.0006) [2023-12-26 20:21:41,491][105692] Updated weights for policy 0, policy_version 685974 (0.0009) [2023-12-26 20:21:41,557][105692] Updated weights for policy 0, policy_version 685984 (0.0009) [2023-12-26 20:21:42,173][105620] Updated weights for policy 1, policy_version 686818 (0.0009) [2023-12-26 20:21:42,235][105620] Updated weights for policy 1, policy_version 686828 (0.0007) [2023-12-26 20:21:42,297][105620] Updated weights for policy 1, policy_version 686838 (0.0007) [2023-12-26 20:21:42,338][105692] Updated weights for policy 0, policy_version 685994 (0.0009) [2023-12-26 20:21:42,404][105692] Updated weights for policy 0, policy_version 686004 (0.0008) [2023-12-26 20:21:42,461][105692] Updated weights for policy 0, policy_version 686014 (0.0006) [2023-12-26 20:21:42,516][105692] Updated weights for policy 0, policy_version 686024 (0.0008) [2023-12-26 20:21:43,060][105620] Updated weights for policy 1, policy_version 686848 (0.0009) [2023-12-26 20:21:43,082][105586] KL-divergence is very high: 120.0627 [2023-12-26 20:21:43,117][105620] Updated weights for policy 1, policy_version 686858 (0.0009) [2023-12-26 20:21:43,121][105586] KL-divergence is very high: 138.5458 [2023-12-26 20:21:43,128][105586] KL-divergence is very high: 161.1192 [2023-12-26 20:21:43,171][105620] Updated weights for policy 1, policy_version 686868 (0.0009) [2023-12-26 20:21:43,171][105586] KL-divergence is very high: 118.4936 [2023-12-26 20:21:43,220][105692] Updated weights for policy 0, policy_version 686034 (0.0008) [2023-12-26 20:21:43,265][105692] Updated weights for policy 0, policy_version 686044 (0.0007) [2023-12-26 20:21:43,319][105692] Updated weights for policy 0, policy_version 686054 (0.0005) [2023-12-26 20:21:43,894][105692] Updated weights for policy 0, policy_version 686064 (0.0005) [2023-12-26 20:21:43,962][105692] Updated weights for policy 0, policy_version 686074 (0.0007) [2023-12-26 20:21:44,019][105692] Updated weights for policy 0, policy_version 686084 (0.0008) [2023-12-26 20:21:44,041][105620] Updated weights for policy 1, policy_version 686878 (0.0008) [2023-12-26 20:21:44,102][105620] Updated weights for policy 1, policy_version 686888 (0.0009) [2023-12-26 20:21:44,174][105620] Updated weights for policy 1, policy_version 686898 (0.0009) [2023-12-26 20:21:44,666][105692] Updated weights for policy 0, policy_version 686094 (0.0008) [2023-12-26 20:21:44,712][105692] Updated weights for policy 0, policy_version 686104 (0.0008) [2023-12-26 20:21:44,768][105692] Updated weights for policy 0, policy_version 686114 (0.0009) [2023-12-26 20:21:44,956][105620] Updated weights for policy 1, policy_version 686908 (0.0009) [2023-12-26 20:21:45,022][105620] Updated weights for policy 1, policy_version 686918 (0.0009) [2023-12-26 20:21:45,077][105620] Updated weights for policy 1, policy_version 686928 (0.0009) [2023-12-26 20:21:45,554][105692] Updated weights for policy 0, policy_version 686124 (0.0009) [2023-12-26 20:21:45,613][105692] Updated weights for policy 0, policy_version 686134 (0.0007) [2023-12-26 20:21:45,683][105692] Updated weights for policy 0, policy_version 686144 (0.0005) [2023-12-26 20:21:45,780][105620] Updated weights for policy 1, policy_version 686938 (0.0009) [2023-12-26 20:21:45,835][105620] Updated weights for policy 1, policy_version 686948 (0.0009) [2023-12-26 20:21:45,894][105620] Updated weights for policy 1, policy_version 686958 (0.0007) [2023-12-26 20:21:45,952][105620] Updated weights for policy 1, policy_version 686968 (0.0006) [2023-12-26 20:21:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 351567872. Throughput: 0: 9782.5, 1: 9558.1. Samples: 351535080. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 20:21:46,063][104569] Avg episode reward: [(0, '8185.598'), (1, '8427.759')] [2023-12-26 20:21:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000686968_175882240.pth... [2023-12-26 20:21:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000686152_175685632.pth... [2023-12-26 20:21:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000685848_175595520.pth [2023-12-26 20:21:46,075][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_000686968_175882240.pth [2023-12-26 20:21:46,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000685000_175390720.pth [2023-12-26 20:21:46,079][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_000686152_175685632.pth [2023-12-26 20:21:46,316][105692] Updated weights for policy 0, policy_version 686154 (0.0007) [2023-12-26 20:21:46,372][105692] Updated weights for policy 0, policy_version 686164 (0.0005) [2023-12-26 20:21:46,421][105692] Updated weights for policy 0, policy_version 686174 (0.0005) [2023-12-26 20:21:46,483][105692] Updated weights for policy 0, policy_version 686184 (0.0005) [2023-12-26 20:21:46,607][105620] Updated weights for policy 1, policy_version 686978 (0.0010) [2023-12-26 20:21:46,660][105620] Updated weights for policy 1, policy_version 686988 (0.0010) [2023-12-26 20:21:46,726][105620] Updated weights for policy 1, policy_version 686998 (0.0010) [2023-12-26 20:21:47,156][105692] Updated weights for policy 0, policy_version 686194 (0.0006) [2023-12-26 20:21:47,208][105692] Updated weights for policy 0, policy_version 686204 (0.0006) [2023-12-26 20:21:47,263][105692] Updated weights for policy 0, policy_version 686214 (0.0007) [2023-12-26 20:21:47,459][105620] Updated weights for policy 1, policy_version 687008 (0.0010) [2023-12-26 20:21:47,506][105620] Updated weights for policy 1, policy_version 687018 (0.0010) [2023-12-26 20:21:47,557][105620] Updated weights for policy 1, policy_version 687028 (0.0010) [2023-12-26 20:21:47,975][105692] Updated weights for policy 0, policy_version 686224 (0.0010) [2023-12-26 20:21:48,032][105692] Updated weights for policy 0, policy_version 686234 (0.0009) [2023-12-26 20:21:48,088][105692] Updated weights for policy 0, policy_version 686244 (0.0009) [2023-12-26 20:21:48,142][105620] Updated weights for policy 1, policy_version 687038 (0.0007) [2023-12-26 20:21:48,212][105620] Updated weights for policy 1, policy_version 687048 (0.0005) [2023-12-26 20:21:48,276][105620] Updated weights for policy 1, policy_version 687058 (0.0005) [2023-12-26 20:21:48,880][105620] Updated weights for policy 1, policy_version 687068 (0.0007) [2023-12-26 20:21:48,917][105692] Updated weights for policy 0, policy_version 686254 (0.0008) [2023-12-26 20:21:48,943][105620] Updated weights for policy 1, policy_version 687078 (0.0007) [2023-12-26 20:21:48,977][105692] Updated weights for policy 0, policy_version 686264 (0.0008) [2023-12-26 20:21:49,008][105620] Updated weights for policy 1, policy_version 687088 (0.0007) [2023-12-26 20:21:49,027][105692] Updated weights for policy 0, policy_version 686274 (0.0007) [2023-12-26 20:21:49,676][105620] Updated weights for policy 1, policy_version 687098 (0.0010) [2023-12-26 20:21:49,746][105620] Updated weights for policy 1, policy_version 687108 (0.0007) [2023-12-26 20:21:49,755][105692] Updated weights for policy 0, policy_version 686284 (0.0007) [2023-12-26 20:21:49,807][105692] Updated weights for policy 0, policy_version 686294 (0.0005) [2023-12-26 20:21:49,808][105620] Updated weights for policy 1, policy_version 687118 (0.0009) [2023-12-26 20:21:49,869][105620] Updated weights for policy 1, policy_version 687128 (0.0010) [2023-12-26 20:21:49,874][105692] Updated weights for policy 0, policy_version 686304 (0.0008) [2023-12-26 20:21:50,487][105620] Updated weights for policy 1, policy_version 687138 (0.0011) [2023-12-26 20:21:50,532][105620] Updated weights for policy 1, policy_version 687148 (0.0010) [2023-12-26 20:21:50,580][105620] Updated weights for policy 1, policy_version 687158 (0.0011) [2023-12-26 20:21:50,601][105692] Updated weights for policy 0, policy_version 686314 (0.0008) [2023-12-26 20:21:50,650][105692] Updated weights for policy 0, policy_version 686324 (0.0007) [2023-12-26 20:21:50,706][105692] Updated weights for policy 0, policy_version 686334 (0.0007) [2023-12-26 20:21:50,770][105692] Updated weights for policy 0, policy_version 686344 (0.0008) [2023-12-26 20:21:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 351666176. Throughput: 0: 9849.4, 1: 9540.3. Samples: 351654776. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 20:21:51,063][104569] Avg episode reward: [(0, '8452.479'), (1, '8624.460')] [2023-12-26 20:21:51,394][105620] Updated weights for policy 1, policy_version 687168 (0.0008) [2023-12-26 20:21:51,452][105620] Updated weights for policy 1, policy_version 687178 (0.0007) [2023-12-26 20:21:51,512][105692] Updated weights for policy 0, policy_version 686354 (0.0006) [2023-12-26 20:21:51,519][105620] Updated weights for policy 1, policy_version 687188 (0.0006) [2023-12-26 20:21:51,583][105692] Updated weights for policy 0, policy_version 686364 (0.0009) [2023-12-26 20:21:51,650][105692] Updated weights for policy 0, policy_version 686374 (0.0010) [2023-12-26 20:21:52,216][105620] Updated weights for policy 1, policy_version 687198 (0.0006) [2023-12-26 20:21:52,286][105620] Updated weights for policy 1, policy_version 687208 (0.0007) [2023-12-26 20:21:52,326][105692] Updated weights for policy 0, policy_version 686384 (0.0011) [2023-12-26 20:21:52,356][105620] Updated weights for policy 1, policy_version 687218 (0.0008) [2023-12-26 20:21:52,386][105692] Updated weights for policy 0, policy_version 686394 (0.0011) [2023-12-26 20:21:52,447][105692] Updated weights for policy 0, policy_version 686404 (0.0011) [2023-12-26 20:21:53,061][105620] Updated weights for policy 1, policy_version 687228 (0.0009) [2023-12-26 20:21:53,125][105620] Updated weights for policy 1, policy_version 687238 (0.0009) [2023-12-26 20:21:53,187][105620] Updated weights for policy 1, policy_version 687248 (0.0007) [2023-12-26 20:21:53,197][105692] Updated weights for policy 0, policy_version 686414 (0.0009) [2023-12-26 20:21:53,246][105692] Updated weights for policy 0, policy_version 686424 (0.0006) [2023-12-26 20:21:53,305][105692] Updated weights for policy 0, policy_version 686434 (0.0005) [2023-12-26 20:21:53,753][105620] Updated weights for policy 1, policy_version 687258 (0.0007) [2023-12-26 20:21:53,810][105620] Updated weights for policy 1, policy_version 687268 (0.0006) [2023-12-26 20:21:53,861][105620] Updated weights for policy 1, policy_version 687278 (0.0005) [2023-12-26 20:21:53,908][105620] Updated weights for policy 1, policy_version 687288 (0.0008) [2023-12-26 20:21:54,059][105692] Updated weights for policy 0, policy_version 686444 (0.0006) [2023-12-26 20:21:54,108][105692] Updated weights for policy 0, policy_version 686454 (0.0008) [2023-12-26 20:21:54,157][105692] Updated weights for policy 0, policy_version 686464 (0.0008) [2023-12-26 20:21:54,606][105620] Updated weights for policy 1, policy_version 687298 (0.0010) [2023-12-26 20:21:54,668][105620] Updated weights for policy 1, policy_version 687308 (0.0010) [2023-12-26 20:21:54,716][105620] Updated weights for policy 1, policy_version 687318 (0.0010) [2023-12-26 20:21:54,804][105692] Updated weights for policy 0, policy_version 686474 (0.0006) [2023-12-26 20:21:54,865][105692] Updated weights for policy 0, policy_version 686484 (0.0008) [2023-12-26 20:21:54,913][105692] Updated weights for policy 0, policy_version 686494 (0.0008) [2023-12-26 20:21:54,979][105692] Updated weights for policy 0, policy_version 686504 (0.0008) [2023-12-26 20:21:55,386][105620] Updated weights for policy 1, policy_version 687328 (0.0007) [2023-12-26 20:21:55,438][105620] Updated weights for policy 1, policy_version 687338 (0.0005) [2023-12-26 20:21:55,482][105620] Updated weights for policy 1, policy_version 687348 (0.0005) [2023-12-26 20:21:55,749][105692] Updated weights for policy 0, policy_version 686514 (0.0008) [2023-12-26 20:21:55,814][105692] Updated weights for policy 0, policy_version 686524 (0.0009) [2023-12-26 20:21:55,877][105692] Updated weights for policy 0, policy_version 686534 (0.0009) [2023-12-26 20:21:56,062][104569] Fps is (10 sec: 19661.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 351764480. Throughput: 0: 9794.7, 1: 9647.8. Samples: 351774092. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 20:21:56,062][104569] Avg episode reward: [(0, '8622.104'), (1, '9082.193')] [2023-12-26 20:21:56,116][105620] Updated weights for policy 1, policy_version 687358 (0.0006) [2023-12-26 20:21:56,172][105620] Updated weights for policy 1, policy_version 687368 (0.0005) [2023-12-26 20:21:56,233][105620] Updated weights for policy 1, policy_version 687378 (0.0009) [2023-12-26 20:21:56,517][105692] Updated weights for policy 0, policy_version 686544 (0.0010) [2023-12-26 20:21:56,581][105692] Updated weights for policy 0, policy_version 686554 (0.0010) [2023-12-26 20:21:56,645][105692] Updated weights for policy 0, policy_version 686564 (0.0010) [2023-12-26 20:21:56,777][105620] Updated weights for policy 1, policy_version 687388 (0.0007) [2023-12-26 20:21:56,838][105620] Updated weights for policy 1, policy_version 687398 (0.0010) [2023-12-26 20:21:56,895][105620] Updated weights for policy 1, policy_version 687408 (0.0010) [2023-12-26 20:21:57,329][105692] Updated weights for policy 0, policy_version 686574 (0.0010) [2023-12-26 20:21:57,373][105692] Updated weights for policy 0, policy_version 686584 (0.0010) [2023-12-26 20:21:57,420][105692] Updated weights for policy 0, policy_version 686594 (0.0010) [2023-12-26 20:21:57,530][105620] Updated weights for policy 1, policy_version 687418 (0.0010) [2023-12-26 20:21:57,580][105620] Updated weights for policy 1, policy_version 687428 (0.0010) [2023-12-26 20:21:57,640][105620] Updated weights for policy 1, policy_version 687438 (0.0008) [2023-12-26 20:21:57,695][105620] Updated weights for policy 1, policy_version 687448 (0.0010) [2023-12-26 20:21:58,045][105692] Updated weights for policy 0, policy_version 686604 (0.0008) [2023-12-26 20:21:58,105][105692] Updated weights for policy 0, policy_version 686614 (0.0005) [2023-12-26 20:21:58,162][105692] Updated weights for policy 0, policy_version 686624 (0.0007) [2023-12-26 20:21:58,433][105620] Updated weights for policy 1, policy_version 687458 (0.0007) [2023-12-26 20:21:58,494][105620] Updated weights for policy 1, policy_version 687468 (0.0008) [2023-12-26 20:21:58,558][105620] Updated weights for policy 1, policy_version 687478 (0.0007) [2023-12-26 20:21:58,801][105692] Updated weights for policy 0, policy_version 686634 (0.0006) [2023-12-26 20:21:58,870][105692] Updated weights for policy 0, policy_version 686644 (0.0007) [2023-12-26 20:21:58,939][105692] Updated weights for policy 0, policy_version 686654 (0.0007) [2023-12-26 20:21:59,003][105692] Updated weights for policy 0, policy_version 686664 (0.0006) [2023-12-26 20:21:59,390][105620] Updated weights for policy 1, policy_version 687488 (0.0008) [2023-12-26 20:21:59,451][105620] Updated weights for policy 1, policy_version 687498 (0.0010) [2023-12-26 20:21:59,512][105620] Updated weights for policy 1, policy_version 687508 (0.0009) [2023-12-26 20:21:59,689][105692] Updated weights for policy 0, policy_version 686674 (0.0008) [2023-12-26 20:21:59,758][105692] Updated weights for policy 0, policy_version 686684 (0.0008) [2023-12-26 20:21:59,826][105692] Updated weights for policy 0, policy_version 686694 (0.0009) [2023-12-26 20:22:00,172][105620] Updated weights for policy 1, policy_version 687518 (0.0007) [2023-12-26 20:22:00,221][105620] Updated weights for policy 1, policy_version 687528 (0.0009) [2023-12-26 20:22:00,271][105620] Updated weights for policy 1, policy_version 687538 (0.0009) [2023-12-26 20:22:00,569][105692] Updated weights for policy 0, policy_version 686704 (0.0006) [2023-12-26 20:22:00,627][105692] Updated weights for policy 0, policy_version 686714 (0.0005) [2023-12-26 20:22:00,684][105692] Updated weights for policy 0, policy_version 686724 (0.0005) [2023-12-26 20:22:01,011][105620] Updated weights for policy 1, policy_version 687548 (0.0007) [2023-12-26 20:22:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 351862784. Throughput: 0: 9849.5, 1: 9730.9. Samples: 351836312. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 20:22:01,063][104569] Avg episode reward: [(0, '7833.922'), (1, '9171.571')] [2023-12-26 20:22:01,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000686728_175833088.pth... [2023-12-26 20:22:01,072][105620] Updated weights for policy 1, policy_version 687558 (0.0008) [2023-12-26 20:22:01,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000685576_175538176.pth [2023-12-26 20:22:01,137][105620] Updated weights for policy 1, policy_version 687568 (0.0009) [2023-12-26 20:22:01,181][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000687576_176037888.pth... [2023-12-26 20:22:01,184][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000686392_175734784.pth [2023-12-26 20:22:01,284][105692] Updated weights for policy 0, policy_version 686734 (0.0007) [2023-12-26 20:22:01,335][105692] Updated weights for policy 0, policy_version 686744 (0.0009) [2023-12-26 20:22:01,403][105692] Updated weights for policy 0, policy_version 686754 (0.0008) [2023-12-26 20:22:01,857][105620] Updated weights for policy 1, policy_version 687578 (0.0009) [2023-12-26 20:22:01,908][105620] Updated weights for policy 1, policy_version 687588 (0.0009) [2023-12-26 20:22:01,968][105620] Updated weights for policy 1, policy_version 687598 (0.0009) [2023-12-26 20:22:02,028][105620] Updated weights for policy 1, policy_version 687608 (0.0008) [2023-12-26 20:22:02,205][105692] Updated weights for policy 0, policy_version 686764 (0.0010) [2023-12-26 20:22:02,250][105692] Updated weights for policy 0, policy_version 686774 (0.0008) [2023-12-26 20:22:02,309][105692] Updated weights for policy 0, policy_version 686784 (0.0009) [2023-12-26 20:22:02,779][105620] Updated weights for policy 1, policy_version 687618 (0.0010) [2023-12-26 20:22:02,831][105620] Updated weights for policy 1, policy_version 687628 (0.0010) [2023-12-26 20:22:02,893][105620] Updated weights for policy 1, policy_version 687638 (0.0010) [2023-12-26 20:22:03,062][105692] Updated weights for policy 0, policy_version 686794 (0.0009) [2023-12-26 20:22:03,110][105692] Updated weights for policy 0, policy_version 686804 (0.0008) [2023-12-26 20:22:03,163][105692] Updated weights for policy 0, policy_version 686814 (0.0008) [2023-12-26 20:22:03,211][105692] Updated weights for policy 0, policy_version 686824 (0.0009) [2023-12-26 20:22:03,626][105620] Updated weights for policy 1, policy_version 687648 (0.0009) [2023-12-26 20:22:03,681][105620] Updated weights for policy 1, policy_version 687658 (0.0008) [2023-12-26 20:22:03,728][105620] Updated weights for policy 1, policy_version 687668 (0.0008) [2023-12-26 20:22:03,877][105692] Updated weights for policy 0, policy_version 686834 (0.0007) [2023-12-26 20:22:03,943][105692] Updated weights for policy 0, policy_version 686844 (0.0008) [2023-12-26 20:22:04,001][105692] Updated weights for policy 0, policy_version 686854 (0.0011) [2023-12-26 20:22:04,493][105620] Updated weights for policy 1, policy_version 687678 (0.0009) [2023-12-26 20:22:04,543][105620] Updated weights for policy 1, policy_version 687688 (0.0008) [2023-12-26 20:22:04,599][105620] Updated weights for policy 1, policy_version 687698 (0.0008) [2023-12-26 20:22:04,763][105692] Updated weights for policy 0, policy_version 686864 (0.0010) [2023-12-26 20:22:04,814][105692] Updated weights for policy 0, policy_version 686874 (0.0010) [2023-12-26 20:22:04,870][105692] Updated weights for policy 0, policy_version 686884 (0.0005) [2023-12-26 20:22:05,192][105620] Updated weights for policy 1, policy_version 687708 (0.0008) [2023-12-26 20:22:05,240][105620] Updated weights for policy 1, policy_version 687718 (0.0008) [2023-12-26 20:22:05,295][105620] Updated weights for policy 1, policy_version 687728 (0.0008) [2023-12-26 20:22:05,614][105692] Updated weights for policy 0, policy_version 686894 (0.0010) [2023-12-26 20:22:05,668][105692] Updated weights for policy 0, policy_version 686904 (0.0010) [2023-12-26 20:22:05,723][105692] Updated weights for policy 0, policy_version 686914 (0.0010) [2023-12-26 20:22:05,948][105620] Updated weights for policy 1, policy_version 687738 (0.0007) [2023-12-26 20:22:05,996][105620] Updated weights for policy 1, policy_version 687748 (0.0005) [2023-12-26 20:22:06,049][105620] Updated weights for policy 1, policy_version 687758 (0.0006) [2023-12-26 20:22:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 351961088. Throughput: 0: 9821.2, 1: 9786.8. Samples: 351952668. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 20:22:06,062][104569] Avg episode reward: [(0, '8017.060'), (1, '9080.262')] [2023-12-26 20:22:06,100][105620] Updated weights for policy 1, policy_version 687768 (0.0009) [2023-12-26 20:22:06,435][105692] Updated weights for policy 0, policy_version 686924 (0.0009) [2023-12-26 20:22:06,497][105692] Updated weights for policy 0, policy_version 686934 (0.0009) [2023-12-26 20:22:06,559][105692] Updated weights for policy 0, policy_version 686944 (0.0006) [2023-12-26 20:22:06,851][105620] Updated weights for policy 1, policy_version 687778 (0.0009) [2023-12-26 20:22:06,902][105620] Updated weights for policy 1, policy_version 687788 (0.0009) [2023-12-26 20:22:06,962][105620] Updated weights for policy 1, policy_version 687798 (0.0008) [2023-12-26 20:22:07,253][105692] Updated weights for policy 0, policy_version 686954 (0.0010) [2023-12-26 20:22:07,305][105692] Updated weights for policy 0, policy_version 686964 (0.0008) [2023-12-26 20:22:07,361][105692] Updated weights for policy 0, policy_version 686974 (0.0007) [2023-12-26 20:22:07,415][105692] Updated weights for policy 0, policy_version 686984 (0.0007) [2023-12-26 20:22:07,609][105620] Updated weights for policy 1, policy_version 687808 (0.0009) [2023-12-26 20:22:07,665][105620] Updated weights for policy 1, policy_version 687818 (0.0009) [2023-12-26 20:22:07,717][105620] Updated weights for policy 1, policy_version 687828 (0.0005) [2023-12-26 20:22:08,018][105692] Updated weights for policy 0, policy_version 686994 (0.0009) [2023-12-26 20:22:08,070][105692] Updated weights for policy 0, policy_version 687005 (0.0009) [2023-12-26 20:22:08,123][105692] Updated weights for policy 0, policy_version 687016 (0.0010) [2023-12-26 20:22:08,308][105620] Updated weights for policy 1, policy_version 687838 (0.0005) [2023-12-26 20:22:08,370][105620] Updated weights for policy 1, policy_version 687848 (0.0007) [2023-12-26 20:22:08,432][105620] Updated weights for policy 1, policy_version 687858 (0.0008) [2023-12-26 20:22:08,889][105692] Updated weights for policy 0, policy_version 687026 (0.0005) [2023-12-26 20:22:08,945][105692] Updated weights for policy 0, policy_version 687036 (0.0005) [2023-12-26 20:22:09,001][105692] Updated weights for policy 0, policy_version 687046 (0.0005) [2023-12-26 20:22:09,169][105620] Updated weights for policy 1, policy_version 687868 (0.0007) [2023-12-26 20:22:09,237][105620] Updated weights for policy 1, policy_version 687878 (0.0006) [2023-12-26 20:22:09,301][105620] Updated weights for policy 1, policy_version 687888 (0.0006) [2023-12-26 20:22:09,692][105692] Updated weights for policy 0, policy_version 687056 (0.0009) [2023-12-26 20:22:09,755][105692] Updated weights for policy 0, policy_version 687066 (0.0010) [2023-12-26 20:22:09,818][105692] Updated weights for policy 0, policy_version 687076 (0.0009) [2023-12-26 20:22:09,893][105620] Updated weights for policy 1, policy_version 687898 (0.0007) [2023-12-26 20:22:09,951][105620] Updated weights for policy 1, policy_version 687908 (0.0007) [2023-12-26 20:22:10,007][105620] Updated weights for policy 1, policy_version 687918 (0.0008) [2023-12-26 20:22:10,074][105620] Updated weights for policy 1, policy_version 687928 (0.0006) [2023-12-26 20:22:10,522][105692] Updated weights for policy 0, policy_version 687086 (0.0010) [2023-12-26 20:22:10,578][105692] Updated weights for policy 0, policy_version 687096 (0.0010) [2023-12-26 20:22:10,627][105692] Updated weights for policy 0, policy_version 687106 (0.0010) [2023-12-26 20:22:10,687][105620] Updated weights for policy 1, policy_version 687938 (0.0005) [2023-12-26 20:22:10,739][105620] Updated weights for policy 1, policy_version 687948 (0.0005) [2023-12-26 20:22:10,795][105620] Updated weights for policy 1, policy_version 687958 (0.0005) [2023-12-26 20:22:11,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 352067584. Throughput: 0: 9873.0, 1: 9952.0. Samples: 352075792. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) [2023-12-26 20:22:11,062][104569] Avg episode reward: [(0, '8715.428'), (1, '9169.671')] [2023-12-26 20:22:11,378][105692] Updated weights for policy 0, policy_version 687116 (0.0011) [2023-12-26 20:22:11,442][105692] Updated weights for policy 0, policy_version 687126 (0.0010) [2023-12-26 20:22:11,486][105620] Updated weights for policy 1, policy_version 687968 (0.0006) [2023-12-26 20:22:11,506][105692] Updated weights for policy 0, policy_version 687136 (0.0011) [2023-12-26 20:22:11,544][105620] Updated weights for policy 1, policy_version 687978 (0.0008) [2023-12-26 20:22:11,603][105620] Updated weights for policy 1, policy_version 687988 (0.0008) [2023-12-26 20:22:12,274][105692] Updated weights for policy 0, policy_version 687146 (0.0011) [2023-12-26 20:22:12,315][105620] Updated weights for policy 1, policy_version 687998 (0.0008) [2023-12-26 20:22:12,334][105692] Updated weights for policy 0, policy_version 687156 (0.0011) [2023-12-26 20:22:12,386][105620] Updated weights for policy 1, policy_version 688008 (0.0009) [2023-12-26 20:22:12,398][105692] Updated weights for policy 0, policy_version 687166 (0.0010) [2023-12-26 20:22:12,445][105620] Updated weights for policy 1, policy_version 688018 (0.0011) [2023-12-26 20:22:12,461][105692] Updated weights for policy 0, policy_version 687176 (0.0010) [2023-12-26 20:22:13,185][105620] Updated weights for policy 1, policy_version 688028 (0.0009) [2023-12-26 20:22:13,209][105692] Updated weights for policy 0, policy_version 687186 (0.0010) [2023-12-26 20:22:13,239][105620] Updated weights for policy 1, policy_version 688038 (0.0005) [2023-12-26 20:22:13,261][105692] Updated weights for policy 0, policy_version 687196 (0.0010) [2023-12-26 20:22:13,288][105620] Updated weights for policy 1, policy_version 688048 (0.0005) [2023-12-26 20:22:13,309][105692] Updated weights for policy 0, policy_version 687206 (0.0010) [2023-12-26 20:22:14,047][105692] Updated weights for policy 0, policy_version 687216 (0.0010) [2023-12-26 20:22:14,058][105620] Updated weights for policy 1, policy_version 688058 (0.0006) [2023-12-26 20:22:14,101][105692] Updated weights for policy 0, policy_version 687226 (0.0010) [2023-12-26 20:22:14,104][105620] Updated weights for policy 1, policy_version 688068 (0.0007) [2023-12-26 20:22:14,132][105586] KL-divergence is very high: 114.1946 [2023-12-26 20:22:14,151][105620] Updated weights for policy 1, policy_version 688078 (0.0006) [2023-12-26 20:22:14,156][105692] Updated weights for policy 0, policy_version 687236 (0.0010) [2023-12-26 20:22:14,204][105620] Updated weights for policy 1, policy_version 688088 (0.0008) [2023-12-26 20:22:14,852][105692] Updated weights for policy 0, policy_version 687246 (0.0008) [2023-12-26 20:22:14,921][105692] Updated weights for policy 0, policy_version 687256 (0.0006) [2023-12-26 20:22:14,979][105692] Updated weights for policy 0, policy_version 687266 (0.0005) [2023-12-26 20:22:15,011][105620] Updated weights for policy 1, policy_version 688098 (0.0007) [2023-12-26 20:22:15,077][105620] Updated weights for policy 1, policy_version 688108 (0.0008) [2023-12-26 20:22:15,150][105620] Updated weights for policy 1, policy_version 688118 (0.0007) [2023-12-26 20:22:15,606][105692] Updated weights for policy 0, policy_version 687276 (0.0010) [2023-12-26 20:22:15,660][105692] Updated weights for policy 0, policy_version 687286 (0.0010) [2023-12-26 20:22:15,713][105692] Updated weights for policy 0, policy_version 687296 (0.0005) [2023-12-26 20:22:15,775][105620] Updated weights for policy 1, policy_version 688128 (0.0008) [2023-12-26 20:22:15,845][105620] Updated weights for policy 1, policy_version 688138 (0.0010) [2023-12-26 20:22:15,914][105620] Updated weights for policy 1, policy_version 688148 (0.0010) [2023-12-26 20:22:16,062][104569] Fps is (10 sec: 20479.4, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 352165888. Throughput: 0: 9752.9, 1: 9913.7. Samples: 352131932. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) [2023-12-26 20:22:16,063][104569] Avg episode reward: [(0, '8442.757'), (1, '9077.677')] [2023-12-26 20:22:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000687304_175980544.pth... [2023-12-26 20:22:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000688152_176185344.pth... [2023-12-26 20:22:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000686152_175685632.pth [2023-12-26 20:22:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000686968_175882240.pth [2023-12-26 20:22:16,250][105692] Updated weights for policy 0, policy_version 687306 (0.0006) [2023-12-26 20:22:16,298][105692] Updated weights for policy 0, policy_version 687316 (0.0010) [2023-12-26 20:22:16,350][105692] Updated weights for policy 0, policy_version 687326 (0.0010) [2023-12-26 20:22:16,408][105692] Updated weights for policy 0, policy_version 687336 (0.0010) [2023-12-26 20:22:16,552][105620] Updated weights for policy 1, policy_version 688158 (0.0008) [2023-12-26 20:22:16,611][105620] Updated weights for policy 1, policy_version 688168 (0.0008) [2023-12-26 20:22:16,662][105620] Updated weights for policy 1, policy_version 688178 (0.0008) [2023-12-26 20:22:17,166][105692] Updated weights for policy 0, policy_version 687346 (0.0010) [2023-12-26 20:22:17,224][105692] Updated weights for policy 0, policy_version 687356 (0.0010) [2023-12-26 20:22:17,288][105692] Updated weights for policy 0, policy_version 687366 (0.0010) [2023-12-26 20:22:17,343][105620] Updated weights for policy 1, policy_version 688188 (0.0008) [2023-12-26 20:22:17,391][105620] Updated weights for policy 1, policy_version 688198 (0.0008) [2023-12-26 20:22:17,440][105620] Updated weights for policy 1, policy_version 688208 (0.0006) [2023-12-26 20:22:18,022][105692] Updated weights for policy 0, policy_version 687376 (0.0010) [2023-12-26 20:22:18,074][105692] Updated weights for policy 0, policy_version 687386 (0.0006) [2023-12-26 20:22:18,129][105620] Updated weights for policy 1, policy_version 688218 (0.0007) [2023-12-26 20:22:18,140][105692] Updated weights for policy 0, policy_version 687396 (0.0006) [2023-12-26 20:22:18,181][105620] Updated weights for policy 1, policy_version 688228 (0.0010) [2023-12-26 20:22:18,226][105620] Updated weights for policy 1, policy_version 688238 (0.0010) [2023-12-26 20:22:18,277][105620] Updated weights for policy 1, policy_version 688248 (0.0010) [2023-12-26 20:22:18,849][105692] Updated weights for policy 0, policy_version 687406 (0.0010) [2023-12-26 20:22:18,908][105692] Updated weights for policy 0, policy_version 687416 (0.0010) [2023-12-26 20:22:18,918][105620] Updated weights for policy 1, policy_version 688258 (0.0011) [2023-12-26 20:22:18,965][105692] Updated weights for policy 0, policy_version 687426 (0.0006) [2023-12-26 20:22:18,975][105620] Updated weights for policy 1, policy_version 688268 (0.0011) [2023-12-26 20:22:19,031][105620] Updated weights for policy 1, policy_version 688278 (0.0009) [2023-12-26 20:22:19,593][105692] Updated weights for policy 0, policy_version 687436 (0.0005) [2023-12-26 20:22:19,655][105692] Updated weights for policy 0, policy_version 687446 (0.0006) [2023-12-26 20:22:19,720][105692] Updated weights for policy 0, policy_version 687456 (0.0005) [2023-12-26 20:22:19,876][105620] Updated weights for policy 1, policy_version 688288 (0.0009) [2023-12-26 20:22:19,937][105620] Updated weights for policy 1, policy_version 688298 (0.0007) [2023-12-26 20:22:19,994][105620] Updated weights for policy 1, policy_version 688308 (0.0006) [2023-12-26 20:22:20,436][105692] Updated weights for policy 0, policy_version 687466 (0.0006) [2023-12-26 20:22:20,500][105692] Updated weights for policy 0, policy_version 687476 (0.0009) [2023-12-26 20:22:20,554][105692] Updated weights for policy 0, policy_version 687486 (0.0009) [2023-12-26 20:22:20,621][105692] Updated weights for policy 0, policy_version 687496 (0.0009) [2023-12-26 20:22:20,660][105620] Updated weights for policy 1, policy_version 688318 (0.0008) [2023-12-26 20:22:20,723][105620] Updated weights for policy 1, policy_version 688328 (0.0009) [2023-12-26 20:22:20,785][105620] Updated weights for policy 1, policy_version 688338 (0.0009) [2023-12-26 20:22:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19688.6). Total num frames: 352264192. Throughput: 0: 9820.2, 1: 9956.1. Samples: 352252828. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) [2023-12-26 20:22:21,063][104569] Avg episode reward: [(0, '8545.841'), (1, '9078.201')] [2023-12-26 20:22:21,420][105692] Updated weights for policy 0, policy_version 687506 (0.0010) [2023-12-26 20:22:21,476][105692] Updated weights for policy 0, policy_version 687516 (0.0009) [2023-12-26 20:22:21,532][105692] Updated weights for policy 0, policy_version 687526 (0.0008) [2023-12-26 20:22:21,574][105620] Updated weights for policy 1, policy_version 688348 (0.0008) [2023-12-26 20:22:21,637][105620] Updated weights for policy 1, policy_version 688358 (0.0007) [2023-12-26 20:22:21,703][105620] Updated weights for policy 1, policy_version 688368 (0.0008) [2023-12-26 20:22:22,322][105692] Updated weights for policy 0, policy_version 687536 (0.0008) [2023-12-26 20:22:22,388][105692] Updated weights for policy 0, policy_version 687546 (0.0008) [2023-12-26 20:22:22,415][105620] Updated weights for policy 1, policy_version 688378 (0.0008) [2023-12-26 20:22:22,455][105692] Updated weights for policy 0, policy_version 687556 (0.0008) [2023-12-26 20:22:22,476][105620] Updated weights for policy 1, policy_version 688388 (0.0007) [2023-12-26 20:22:22,535][105620] Updated weights for policy 1, policy_version 688398 (0.0008) [2023-12-26 20:22:22,594][105620] Updated weights for policy 1, policy_version 688408 (0.0007) [2023-12-26 20:22:23,124][105692] Updated weights for policy 0, policy_version 687566 (0.0008) [2023-12-26 20:22:23,179][105692] Updated weights for policy 0, policy_version 687576 (0.0009) [2023-12-26 20:22:23,241][105692] Updated weights for policy 0, policy_version 687586 (0.0009) [2023-12-26 20:22:23,324][105620] Updated weights for policy 1, policy_version 688418 (0.0007) [2023-12-26 20:22:23,388][105620] Updated weights for policy 1, policy_version 688428 (0.0006) [2023-12-26 20:22:23,440][105620] Updated weights for policy 1, policy_version 688438 (0.0005) [2023-12-26 20:22:23,915][105692] Updated weights for policy 0, policy_version 687596 (0.0008) [2023-12-26 20:22:23,966][105692] Updated weights for policy 0, policy_version 687606 (0.0011) [2023-12-26 20:22:24,010][105692] Updated weights for policy 0, policy_version 687616 (0.0010) [2023-12-26 20:22:24,037][105620] Updated weights for policy 1, policy_version 688448 (0.0008) [2023-12-26 20:22:24,091][105620] Updated weights for policy 1, policy_version 688458 (0.0006) [2023-12-26 20:22:24,153][105620] Updated weights for policy 1, policy_version 688468 (0.0006) [2023-12-26 20:22:24,613][105692] Updated weights for policy 0, policy_version 687626 (0.0010) [2023-12-26 20:22:24,664][105692] Updated weights for policy 0, policy_version 687636 (0.0005) [2023-12-26 20:22:24,718][105692] Updated weights for policy 0, policy_version 687646 (0.0006) [2023-12-26 20:22:24,755][105620] Updated weights for policy 1, policy_version 688478 (0.0006) [2023-12-26 20:22:24,780][105692] Updated weights for policy 0, policy_version 687656 (0.0010) [2023-12-26 20:22:24,810][105620] Updated weights for policy 1, policy_version 688488 (0.0006) [2023-12-26 20:22:24,864][105620] Updated weights for policy 1, policy_version 688498 (0.0005) [2023-12-26 20:22:25,336][105692] Updated weights for policy 0, policy_version 687666 (0.0008) [2023-12-26 20:22:25,405][105692] Updated weights for policy 0, policy_version 687676 (0.0009) [2023-12-26 20:22:25,464][105692] Updated weights for policy 0, policy_version 687686 (0.0009) [2023-12-26 20:22:25,470][105620] Updated weights for policy 1, policy_version 688508 (0.0005) [2023-12-26 20:22:25,523][105620] Updated weights for policy 1, policy_version 688518 (0.0005) [2023-12-26 20:22:25,582][105620] Updated weights for policy 1, policy_version 688528 (0.0005) [2023-12-26 20:22:26,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 352362496. Throughput: 0: 9885.1, 1: 10036.5. Samples: 352375128. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) [2023-12-26 20:22:26,063][104569] Avg episode reward: [(0, '7390.453'), (1, '9260.678')] [2023-12-26 20:22:26,097][105620] Updated weights for policy 1, policy_version 688538 (0.0006) [2023-12-26 20:22:26,145][105620] Updated weights for policy 1, policy_version 688548 (0.0010) [2023-12-26 20:22:26,200][105620] Updated weights for policy 1, policy_version 688558 (0.0010) [2023-12-26 20:22:26,245][105692] Updated weights for policy 0, policy_version 687696 (0.0009) [2023-12-26 20:22:26,261][105620] Updated weights for policy 1, policy_version 688568 (0.0011) [2023-12-26 20:22:26,302][105692] Updated weights for policy 0, policy_version 687706 (0.0007) [2023-12-26 20:22:26,360][105692] Updated weights for policy 0, policy_version 687716 (0.0008) [2023-12-26 20:22:26,992][105620] Updated weights for policy 1, policy_version 688578 (0.0010) [2023-12-26 20:22:27,010][105692] Updated weights for policy 0, policy_version 687726 (0.0006) [2023-12-26 20:22:27,040][105620] Updated weights for policy 1, policy_version 688588 (0.0010) [2023-12-26 20:22:27,066][105692] Updated weights for policy 0, policy_version 687736 (0.0005) [2023-12-26 20:22:27,098][105620] Updated weights for policy 1, policy_version 688598 (0.0010) [2023-12-26 20:22:27,125][105692] Updated weights for policy 0, policy_version 687746 (0.0006) [2023-12-26 20:22:27,824][105692] Updated weights for policy 0, policy_version 687756 (0.0007) [2023-12-26 20:22:27,841][105620] Updated weights for policy 1, policy_version 688608 (0.0010) [2023-12-26 20:22:27,875][105692] Updated weights for policy 0, policy_version 687766 (0.0005) [2023-12-26 20:22:27,888][105620] Updated weights for policy 1, policy_version 688618 (0.0010) [2023-12-26 20:22:27,924][105692] Updated weights for policy 0, policy_version 687776 (0.0006) [2023-12-26 20:22:27,936][105620] Updated weights for policy 1, policy_version 688628 (0.0010) [2023-12-26 20:22:28,563][105692] Updated weights for policy 0, policy_version 687786 (0.0006) [2023-12-26 20:22:28,611][105692] Updated weights for policy 0, policy_version 687796 (0.0008) [2023-12-26 20:22:28,670][105692] Updated weights for policy 0, policy_version 687806 (0.0008) [2023-12-26 20:22:28,702][105620] Updated weights for policy 1, policy_version 688638 (0.0010) [2023-12-26 20:22:28,727][105692] Updated weights for policy 0, policy_version 687816 (0.0007) [2023-12-26 20:22:28,763][105620] Updated weights for policy 1, policy_version 688648 (0.0010) [2023-12-26 20:22:28,826][105620] Updated weights for policy 1, policy_version 688658 (0.0010) [2023-12-26 20:22:29,498][105620] Updated weights for policy 1, policy_version 688668 (0.0006) [2023-12-26 20:22:29,518][105692] Updated weights for policy 0, policy_version 687826 (0.0008) [2023-12-26 20:22:29,560][105620] Updated weights for policy 1, policy_version 688678 (0.0008) [2023-12-26 20:22:29,578][105692] Updated weights for policy 0, policy_version 687836 (0.0007) [2023-12-26 20:22:29,626][105620] Updated weights for policy 1, policy_version 688688 (0.0007) [2023-12-26 20:22:29,641][105692] Updated weights for policy 0, policy_version 687846 (0.0007) [2023-12-26 20:22:30,315][105692] Updated weights for policy 0, policy_version 687856 (0.0009) [2023-12-26 20:22:30,356][105620] Updated weights for policy 1, policy_version 688698 (0.0010) [2023-12-26 20:22:30,376][105692] Updated weights for policy 0, policy_version 687866 (0.0008) [2023-12-26 20:22:30,408][105620] Updated weights for policy 1, policy_version 688708 (0.0010) [2023-12-26 20:22:30,434][105692] Updated weights for policy 0, policy_version 687876 (0.0008) [2023-12-26 20:22:30,470][105620] Updated weights for policy 1, policy_version 688718 (0.0010) [2023-12-26 20:22:30,535][105620] Updated weights for policy 1, policy_version 688728 (0.0010) [2023-12-26 20:22:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 352460800. Throughput: 0: 9898.0, 1: 10102.3. Samples: 352435088. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) [2023-12-26 20:22:31,063][104569] Avg episode reward: [(0, '7584.522'), (1, '8992.378')] [2023-12-26 20:22:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000687880_176128000.pth... [2023-12-26 20:22:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000688728_176332800.pth... [2023-12-26 20:22:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000686728_175833088.pth [2023-12-26 20:22:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000687576_176037888.pth [2023-12-26 20:22:31,153][105692] Updated weights for policy 0, policy_version 687886 (0.0009) [2023-12-26 20:22:31,203][105692] Updated weights for policy 0, policy_version 687896 (0.0009) [2023-12-26 20:22:31,262][105692] Updated weights for policy 0, policy_version 687906 (0.0007) [2023-12-26 20:22:31,263][105620] Updated weights for policy 1, policy_version 688738 (0.0006) [2023-12-26 20:22:31,323][105620] Updated weights for policy 1, policy_version 688748 (0.0007) [2023-12-26 20:22:31,389][105620] Updated weights for policy 1, policy_version 688758 (0.0006) [2023-12-26 20:22:32,019][105620] Updated weights for policy 1, policy_version 688768 (0.0009) [2023-12-26 20:22:32,079][105692] Updated weights for policy 0, policy_version 687916 (0.0007) [2023-12-26 20:22:32,080][105586] KL-divergence is very high: 143.3651 [2023-12-26 20:22:32,085][105620] Updated weights for policy 1, policy_version 688778 (0.0008) [2023-12-26 20:22:32,142][105620] Updated weights for policy 1, policy_version 688788 (0.0006) [2023-12-26 20:22:32,143][105692] Updated weights for policy 0, policy_version 687926 (0.0009) [2023-12-26 20:22:32,205][105692] Updated weights for policy 0, policy_version 687936 (0.0009) [2023-12-26 20:22:32,841][105620] Updated weights for policy 1, policy_version 688798 (0.0008) [2023-12-26 20:22:32,906][105620] Updated weights for policy 1, policy_version 688808 (0.0009) [2023-12-26 20:22:32,960][105620] Updated weights for policy 1, policy_version 688818 (0.0009) [2023-12-26 20:22:32,965][105692] Updated weights for policy 0, policy_version 687946 (0.0008) [2023-12-26 20:22:33,016][105692] Updated weights for policy 0, policy_version 687956 (0.0008) [2023-12-26 20:22:33,068][105692] Updated weights for policy 0, policy_version 687966 (0.0009) [2023-12-26 20:22:33,131][105692] Updated weights for policy 0, policy_version 687976 (0.0005) [2023-12-26 20:22:33,692][105692] Updated weights for policy 0, policy_version 687986 (0.0006) [2023-12-26 20:22:33,718][105620] Updated weights for policy 1, policy_version 688828 (0.0006) [2023-12-26 20:22:33,748][105692] Updated weights for policy 0, policy_version 687996 (0.0005) [2023-12-26 20:22:33,776][105620] Updated weights for policy 1, policy_version 688838 (0.0007) [2023-12-26 20:22:33,801][105692] Updated weights for policy 0, policy_version 688006 (0.0005) [2023-12-26 20:22:33,829][105620] Updated weights for policy 1, policy_version 688848 (0.0009) [2023-12-26 20:22:34,389][105692] Updated weights for policy 0, policy_version 688016 (0.0006) [2023-12-26 20:22:34,445][105620] Updated weights for policy 1, policy_version 688858 (0.0008) [2023-12-26 20:22:34,450][105692] Updated weights for policy 0, policy_version 688026 (0.0009) [2023-12-26 20:22:34,510][105620] Updated weights for policy 1, policy_version 688868 (0.0006) [2023-12-26 20:22:34,511][105692] Updated weights for policy 0, policy_version 688036 (0.0011) [2023-12-26 20:22:34,580][105620] Updated weights for policy 1, policy_version 688878 (0.0007) [2023-12-26 20:22:34,636][105620] Updated weights for policy 1, policy_version 688888 (0.0008) [2023-12-26 20:22:35,166][105692] Updated weights for policy 0, policy_version 688046 (0.0008) [2023-12-26 20:22:35,227][105692] Updated weights for policy 0, policy_version 688056 (0.0006) [2023-12-26 20:22:35,288][105692] Updated weights for policy 0, policy_version 688066 (0.0007) [2023-12-26 20:22:35,399][105620] Updated weights for policy 1, policy_version 688898 (0.0008) [2023-12-26 20:22:35,447][105620] Updated weights for policy 1, policy_version 688908 (0.0005) [2023-12-26 20:22:35,498][105620] Updated weights for policy 1, policy_version 688918 (0.0005) [2023-12-26 20:22:36,006][105692] Updated weights for policy 0, policy_version 688076 (0.0007) [2023-12-26 20:22:36,052][105692] Updated weights for policy 0, policy_version 688086 (0.0006) [2023-12-26 20:22:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 352559104. Throughput: 0: 9903.8, 1: 10065.5. Samples: 352553396. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) [2023-12-26 20:22:36,062][104569] Avg episode reward: [(0, '8264.211'), (1, '8901.675')] [2023-12-26 20:22:36,096][105620] Updated weights for policy 1, policy_version 688928 (0.0008) [2023-12-26 20:22:36,114][105692] Updated weights for policy 0, policy_version 688096 (0.0007) [2023-12-26 20:22:36,164][105620] Updated weights for policy 1, policy_version 688938 (0.0008) [2023-12-26 20:22:36,227][105620] Updated weights for policy 1, policy_version 688948 (0.0008) [2023-12-26 20:22:36,770][105692] Updated weights for policy 0, policy_version 688106 (0.0008) [2023-12-26 20:22:36,833][105692] Updated weights for policy 0, policy_version 688116 (0.0009) [2023-12-26 20:22:36,900][105692] Updated weights for policy 0, policy_version 688126 (0.0009) [2023-12-26 20:22:36,962][105692] Updated weights for policy 0, policy_version 688136 (0.0009) [2023-12-26 20:22:36,985][105620] Updated weights for policy 1, policy_version 688958 (0.0007) [2023-12-26 20:22:37,054][105620] Updated weights for policy 1, policy_version 688968 (0.0009) [2023-12-26 20:22:37,110][105620] Updated weights for policy 1, policy_version 688978 (0.0008) [2023-12-26 20:22:37,677][105692] Updated weights for policy 0, policy_version 688146 (0.0009) [2023-12-26 20:22:37,729][105692] Updated weights for policy 0, policy_version 688156 (0.0009) [2023-12-26 20:22:37,792][105692] Updated weights for policy 0, policy_version 688166 (0.0009) [2023-12-26 20:22:37,836][105620] Updated weights for policy 1, policy_version 688988 (0.0008) [2023-12-26 20:22:37,885][105620] Updated weights for policy 1, policy_version 688998 (0.0009) [2023-12-26 20:22:37,946][105620] Updated weights for policy 1, policy_version 689009 (0.0010) [2023-12-26 20:22:38,549][105692] Updated weights for policy 0, policy_version 688176 (0.0009) [2023-12-26 20:22:38,601][105692] Updated weights for policy 0, policy_version 688186 (0.0009) [2023-12-26 20:22:38,656][105692] Updated weights for policy 0, policy_version 688196 (0.0009) [2023-12-26 20:22:38,734][105620] Updated weights for policy 1, policy_version 689019 (0.0009) [2023-12-26 20:22:38,787][105620] Updated weights for policy 1, policy_version 689029 (0.0010) [2023-12-26 20:22:38,852][105620] Updated weights for policy 1, policy_version 689039 (0.0010) [2023-12-26 20:22:39,293][105692] Updated weights for policy 0, policy_version 688206 (0.0007) [2023-12-26 20:22:39,370][105692] Updated weights for policy 0, policy_version 688216 (0.0009) [2023-12-26 20:22:39,430][105692] Updated weights for policy 0, policy_version 688226 (0.0007) [2023-12-26 20:22:39,678][105620] Updated weights for policy 1, policy_version 689049 (0.0009) [2023-12-26 20:22:39,742][105620] Updated weights for policy 1, policy_version 689059 (0.0009) [2023-12-26 20:22:39,808][105620] Updated weights for policy 1, policy_version 689069 (0.0009) [2023-12-26 20:22:39,875][105620] Updated weights for policy 1, policy_version 689079 (0.0009) [2023-12-26 20:22:40,208][105692] Updated weights for policy 0, policy_version 688236 (0.0009) [2023-12-26 20:22:40,271][105692] Updated weights for policy 0, policy_version 688246 (0.0009) [2023-12-26 20:22:40,326][105692] Updated weights for policy 0, policy_version 688256 (0.0009) [2023-12-26 20:22:40,611][105620] Updated weights for policy 1, policy_version 689089 (0.0008) [2023-12-26 20:22:40,672][105620] Updated weights for policy 1, policy_version 689099 (0.0007) [2023-12-26 20:22:40,731][105620] Updated weights for policy 1, policy_version 689109 (0.0008) [2023-12-26 20:22:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 352657408. Throughput: 0: 9923.6, 1: 9949.6. Samples: 352668388. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) [2023-12-26 20:22:41,062][104569] Avg episode reward: [(0, '8534.894'), (1, '9351.955')] [2023-12-26 20:22:41,117][105692] Updated weights for policy 0, policy_version 688266 (0.0009) [2023-12-26 20:22:41,175][105692] Updated weights for policy 0, policy_version 688276 (0.0009) [2023-12-26 20:22:41,230][105692] Updated weights for policy 0, policy_version 688287 (0.0010) [2023-12-26 20:22:41,418][105620] Updated weights for policy 1, policy_version 689119 (0.0008) [2023-12-26 20:22:41,467][105620] Updated weights for policy 1, policy_version 689129 (0.0008) [2023-12-26 20:22:41,515][105620] Updated weights for policy 1, policy_version 689139 (0.0008) [2023-12-26 20:22:42,070][105692] Updated weights for policy 0, policy_version 688297 (0.0008) [2023-12-26 20:22:42,126][105692] Updated weights for policy 0, policy_version 688307 (0.0006) [2023-12-26 20:22:42,179][105692] Updated weights for policy 0, policy_version 688317 (0.0005) [2023-12-26 20:22:42,243][105692] Updated weights for policy 0, policy_version 688327 (0.0006) [2023-12-26 20:22:42,286][105620] Updated weights for policy 1, policy_version 689149 (0.0007) [2023-12-26 20:22:42,354][105620] Updated weights for policy 1, policy_version 689159 (0.0009) [2023-12-26 20:22:42,419][105620] Updated weights for policy 1, policy_version 689169 (0.0009) [2023-12-26 20:22:42,938][105692] Updated weights for policy 0, policy_version 688337 (0.0009) [2023-12-26 20:22:42,993][105692] Updated weights for policy 0, policy_version 688348 (0.0010) [2023-12-26 20:22:43,048][105692] Updated weights for policy 0, policy_version 688358 (0.0009) [2023-12-26 20:22:43,109][105620] Updated weights for policy 1, policy_version 689179 (0.0008) [2023-12-26 20:22:43,164][105620] Updated weights for policy 1, policy_version 689189 (0.0005) [2023-12-26 20:22:43,215][105620] Updated weights for policy 1, policy_version 689199 (0.0005) [2023-12-26 20:22:43,776][105692] Updated weights for policy 0, policy_version 688368 (0.0006) [2023-12-26 20:22:43,793][105620] Updated weights for policy 1, policy_version 689209 (0.0007) [2023-12-26 20:22:43,845][105692] Updated weights for policy 0, policy_version 688378 (0.0006) [2023-12-26 20:22:43,849][105620] Updated weights for policy 1, policy_version 689219 (0.0008) [2023-12-26 20:22:43,904][105692] Updated weights for policy 0, policy_version 688388 (0.0006) [2023-12-26 20:22:43,906][105620] Updated weights for policy 1, policy_version 689229 (0.0008) [2023-12-26 20:22:43,963][105620] Updated weights for policy 1, policy_version 689239 (0.0008) [2023-12-26 20:22:44,617][105620] Updated weights for policy 1, policy_version 689249 (0.0006) [2023-12-26 20:22:44,623][105692] Updated weights for policy 0, policy_version 688398 (0.0008) [2023-12-26 20:22:44,675][105620] Updated weights for policy 1, policy_version 689259 (0.0005) [2023-12-26 20:22:44,682][105692] Updated weights for policy 0, policy_version 688408 (0.0008) [2023-12-26 20:22:44,737][105620] Updated weights for policy 1, policy_version 689269 (0.0008) [2023-12-26 20:22:44,746][105692] Updated weights for policy 0, policy_version 688418 (0.0009) [2023-12-26 20:22:45,490][105692] Updated weights for policy 0, policy_version 688428 (0.0009) [2023-12-26 20:22:45,496][105620] Updated weights for policy 1, policy_version 689279 (0.0008) [2023-12-26 20:22:45,549][105692] Updated weights for policy 0, policy_version 688438 (0.0006) [2023-12-26 20:22:45,551][105620] Updated weights for policy 1, policy_version 689289 (0.0008) [2023-12-26 20:22:45,609][105692] Updated weights for policy 0, policy_version 688448 (0.0006) [2023-12-26 20:22:45,611][105620] Updated weights for policy 1, policy_version 689299 (0.0008) [2023-12-26 20:22:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 352755712. Throughput: 0: 9832.8, 1: 9938.3. Samples: 352726012. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) [2023-12-26 20:22:46,062][104569] Avg episode reward: [(0, '9349.338'), (1, '9259.199')] [2023-12-26 20:22:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000688456_176275456.pth... [2023-12-26 20:22:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000689304_176480256.pth... [2023-12-26 20:22:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000688152_176185344.pth [2023-12-26 20:22:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000687304_175980544.pth [2023-12-26 20:22:46,161][105692] Updated weights for policy 0, policy_version 688458 (0.0006) [2023-12-26 20:22:46,212][105692] Updated weights for policy 0, policy_version 688468 (0.0005) [2023-12-26 20:22:46,263][105692] Updated weights for policy 0, policy_version 688478 (0.0005) [2023-12-26 20:22:46,317][105692] Updated weights for policy 0, policy_version 688488 (0.0005) [2023-12-26 20:22:46,508][105620] Updated weights for policy 1, policy_version 689309 (0.0006) [2023-12-26 20:22:46,559][105620] Updated weights for policy 1, policy_version 689319 (0.0005) [2023-12-26 20:22:46,615][105620] Updated weights for policy 1, policy_version 689329 (0.0005) [2023-12-26 20:22:46,927][105692] Updated weights for policy 0, policy_version 688498 (0.0010) [2023-12-26 20:22:46,977][105692] Updated weights for policy 0, policy_version 688508 (0.0005) [2023-12-26 20:22:47,040][105692] Updated weights for policy 0, policy_version 688518 (0.0005) [2023-12-26 20:22:47,179][105620] Updated weights for policy 1, policy_version 689339 (0.0006) [2023-12-26 20:22:47,238][105620] Updated weights for policy 1, policy_version 689349 (0.0007) [2023-12-26 20:22:47,294][105620] Updated weights for policy 1, policy_version 689359 (0.0006) [2023-12-26 20:22:47,629][105692] Updated weights for policy 0, policy_version 688528 (0.0005) [2023-12-26 20:22:47,675][105692] Updated weights for policy 0, policy_version 688538 (0.0005) [2023-12-26 20:22:47,739][105692] Updated weights for policy 0, policy_version 688548 (0.0005) [2023-12-26 20:22:47,947][105620] Updated weights for policy 1, policy_version 689369 (0.0009) [2023-12-26 20:22:48,005][105620] Updated weights for policy 1, policy_version 689379 (0.0009) [2023-12-26 20:22:48,065][105620] Updated weights for policy 1, policy_version 689389 (0.0008) [2023-12-26 20:22:48,127][105620] Updated weights for policy 1, policy_version 689399 (0.0008) [2023-12-26 20:22:48,361][105692] Updated weights for policy 0, policy_version 688558 (0.0009) [2023-12-26 20:22:48,426][105692] Updated weights for policy 0, policy_version 688568 (0.0010) [2023-12-26 20:22:48,488][105692] Updated weights for policy 0, policy_version 688578 (0.0008) [2023-12-26 20:22:48,774][105620] Updated weights for policy 1, policy_version 689409 (0.0005) [2023-12-26 20:22:48,828][105620] Updated weights for policy 1, policy_version 689419 (0.0005) [2023-12-26 20:22:48,888][105620] Updated weights for policy 1, policy_version 689429 (0.0005) [2023-12-26 20:22:49,335][105692] Updated weights for policy 0, policy_version 688588 (0.0008) [2023-12-26 20:22:49,398][105692] Updated weights for policy 0, policy_version 688598 (0.0008) [2023-12-26 20:22:49,454][105692] Updated weights for policy 0, policy_version 688608 (0.0007) [2023-12-26 20:22:49,504][105620] Updated weights for policy 1, policy_version 689439 (0.0008) [2023-12-26 20:22:49,565][105620] Updated weights for policy 1, policy_version 689449 (0.0009) [2023-12-26 20:22:49,622][105620] Updated weights for policy 1, policy_version 689459 (0.0010) [2023-12-26 20:22:50,057][105692] Updated weights for policy 0, policy_version 688618 (0.0006) [2023-12-26 20:22:50,109][105692] Updated weights for policy 0, policy_version 688628 (0.0011) [2023-12-26 20:22:50,168][105692] Updated weights for policy 0, policy_version 688638 (0.0010) [2023-12-26 20:22:50,227][105692] Updated weights for policy 0, policy_version 688648 (0.0005) [2023-12-26 20:22:50,435][105620] Updated weights for policy 1, policy_version 689469 (0.0009) [2023-12-26 20:22:50,484][105620] Updated weights for policy 1, policy_version 689479 (0.0008) [2023-12-26 20:22:50,541][105620] Updated weights for policy 1, policy_version 689489 (0.0008) [2023-12-26 20:22:50,894][105692] Updated weights for policy 0, policy_version 688658 (0.0011) [2023-12-26 20:22:50,943][105692] Updated weights for policy 0, policy_version 688668 (0.0011) [2023-12-26 20:22:51,025][105692] Updated weights for policy 0, policy_version 688678 (0.0010) [2023-12-26 20:22:51,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19933.9, 300 sec: 19688.6). Total num frames: 352862208. Throughput: 0: 9914.9, 1: 9980.9. Samples: 352847980. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) [2023-12-26 20:22:51,063][104569] Avg episode reward: [(0, '8985.427'), (1, '9077.868')] [2023-12-26 20:22:51,416][105620] Updated weights for policy 1, policy_version 689499 (0.0008) [2023-12-26 20:22:51,481][105620] Updated weights for policy 1, policy_version 689509 (0.0010) [2023-12-26 20:22:51,544][105620] Updated weights for policy 1, policy_version 689519 (0.0010) [2023-12-26 20:22:51,658][105692] Updated weights for policy 0, policy_version 688688 (0.0008) [2023-12-26 20:22:51,725][105692] Updated weights for policy 0, policy_version 688698 (0.0010) [2023-12-26 20:22:51,782][105692] Updated weights for policy 0, policy_version 688708 (0.0011) [2023-12-26 20:22:52,251][105620] Updated weights for policy 1, policy_version 689529 (0.0008) [2023-12-26 20:22:52,316][105620] Updated weights for policy 1, policy_version 689539 (0.0009) [2023-12-26 20:22:52,381][105620] Updated weights for policy 1, policy_version 689549 (0.0007) [2023-12-26 20:22:52,451][105620] Updated weights for policy 1, policy_version 689559 (0.0006) [2023-12-26 20:22:52,565][105692] Updated weights for policy 0, policy_version 688718 (0.0010) [2023-12-26 20:22:52,630][105692] Updated weights for policy 0, policy_version 688728 (0.0009) [2023-12-26 20:22:52,693][105692] Updated weights for policy 0, policy_version 688738 (0.0009) [2023-12-26 20:22:53,115][105620] Updated weights for policy 1, policy_version 689569 (0.0008) [2023-12-26 20:22:53,170][105620] Updated weights for policy 1, policy_version 689579 (0.0008) [2023-12-26 20:22:53,223][105620] Updated weights for policy 1, policy_version 689589 (0.0008) [2023-12-26 20:22:53,451][105692] Updated weights for policy 0, policy_version 688748 (0.0010) [2023-12-26 20:22:53,500][105692] Updated weights for policy 0, policy_version 688758 (0.0009) [2023-12-26 20:22:53,547][105692] Updated weights for policy 0, policy_version 688768 (0.0009) [2023-12-26 20:22:53,910][105620] Updated weights for policy 1, policy_version 689599 (0.0010) [2023-12-26 20:22:53,972][105620] Updated weights for policy 1, policy_version 689609 (0.0009) [2023-12-26 20:22:54,041][105620] Updated weights for policy 1, policy_version 689619 (0.0009) [2023-12-26 20:22:54,351][105692] Updated weights for policy 0, policy_version 688778 (0.0009) [2023-12-26 20:22:54,418][105692] Updated weights for policy 0, policy_version 688788 (0.0007) [2023-12-26 20:22:54,475][105692] Updated weights for policy 0, policy_version 688798 (0.0009) [2023-12-26 20:22:54,526][105692] Updated weights for policy 0, policy_version 688808 (0.0009) [2023-12-26 20:22:54,783][105620] Updated weights for policy 1, policy_version 689629 (0.0008) [2023-12-26 20:22:54,847][105620] Updated weights for policy 1, policy_version 689639 (0.0009) [2023-12-26 20:22:54,910][105620] Updated weights for policy 1, policy_version 689649 (0.0010) [2023-12-26 20:22:55,265][105692] Updated weights for policy 0, policy_version 688818 (0.0009) [2023-12-26 20:22:55,318][105692] Updated weights for policy 0, policy_version 688828 (0.0008) [2023-12-26 20:22:55,372][105692] Updated weights for policy 0, policy_version 688838 (0.0009) [2023-12-26 20:22:55,660][105620] Updated weights for policy 1, policy_version 689659 (0.0009) [2023-12-26 20:22:55,712][105620] Updated weights for policy 1, policy_version 689669 (0.0009) [2023-12-26 20:22:55,764][105620] Updated weights for policy 1, policy_version 689679 (0.0009) [2023-12-26 20:22:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 352952320. Throughput: 0: 9879.7, 1: 9808.5. Samples: 352961756. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) [2023-12-26 20:22:56,062][104569] Avg episode reward: [(0, '8712.416'), (1, '8987.504')] [2023-12-26 20:22:56,122][105692] Updated weights for policy 0, policy_version 688848 (0.0008) [2023-12-26 20:22:56,172][105692] Updated weights for policy 0, policy_version 688858 (0.0008) [2023-12-26 20:22:56,239][105692] Updated weights for policy 0, policy_version 688868 (0.0007) [2023-12-26 20:22:56,511][105620] Updated weights for policy 1, policy_version 689689 (0.0008) [2023-12-26 20:22:56,571][105620] Updated weights for policy 1, policy_version 689699 (0.0005) [2023-12-26 20:22:56,634][105620] Updated weights for policy 1, policy_version 689709 (0.0005) [2023-12-26 20:22:56,689][105620] Updated weights for policy 1, policy_version 689719 (0.0005) [2023-12-26 20:22:57,088][105692] Updated weights for policy 0, policy_version 688878 (0.0008) [2023-12-26 20:22:57,135][105692] Updated weights for policy 0, policy_version 688888 (0.0008) [2023-12-26 20:22:57,188][105692] Updated weights for policy 0, policy_version 688898 (0.0007) [2023-12-26 20:22:57,210][105620] Updated weights for policy 1, policy_version 689729 (0.0010) [2023-12-26 20:22:57,260][105620] Updated weights for policy 1, policy_version 689739 (0.0010) [2023-12-26 20:22:57,315][105620] Updated weights for policy 1, policy_version 689749 (0.0009) [2023-12-26 20:22:57,980][105692] Updated weights for policy 0, policy_version 688908 (0.0007) [2023-12-26 20:22:58,032][105620] Updated weights for policy 1, policy_version 689759 (0.0010) [2023-12-26 20:22:58,037][105692] Updated weights for policy 0, policy_version 688918 (0.0006) [2023-12-26 20:22:58,087][105620] Updated weights for policy 1, policy_version 689769 (0.0010) [2023-12-26 20:22:58,093][105692] Updated weights for policy 0, policy_version 688928 (0.0005) [2023-12-26 20:22:58,149][105620] Updated weights for policy 1, policy_version 689779 (0.0010) [2023-12-26 20:22:58,921][105620] Updated weights for policy 1, policy_version 689789 (0.0010) [2023-12-26 20:22:58,968][105692] Updated weights for policy 0, policy_version 688938 (0.0007) [2023-12-26 20:22:58,985][105620] Updated weights for policy 1, policy_version 689799 (0.0010) [2023-12-26 20:22:59,027][105692] Updated weights for policy 0, policy_version 688948 (0.0011) [2023-12-26 20:22:59,048][105620] Updated weights for policy 1, policy_version 689809 (0.0007) [2023-12-26 20:22:59,086][105692] Updated weights for policy 0, policy_version 688958 (0.0011) [2023-12-26 20:22:59,142][105692] Updated weights for policy 0, policy_version 688968 (0.0011) [2023-12-26 20:22:59,766][105620] Updated weights for policy 1, policy_version 689819 (0.0010) [2023-12-26 20:22:59,830][105620] Updated weights for policy 1, policy_version 689829 (0.0008) [2023-12-26 20:22:59,876][105692] Updated weights for policy 0, policy_version 688978 (0.0008) [2023-12-26 20:22:59,890][105620] Updated weights for policy 1, policy_version 689839 (0.0007) [2023-12-26 20:22:59,944][105692] Updated weights for policy 0, policy_version 688988 (0.0008) [2023-12-26 20:23:00,000][105692] Updated weights for policy 0, policy_version 688998 (0.0008) [2023-12-26 20:23:00,476][105620] Updated weights for policy 1, policy_version 689849 (0.0008) [2023-12-26 20:23:00,533][105620] Updated weights for policy 1, policy_version 689859 (0.0009) [2023-12-26 20:23:00,580][105620] Updated weights for policy 1, policy_version 689869 (0.0009) [2023-12-26 20:23:00,629][105620] Updated weights for policy 1, policy_version 689879 (0.0008) [2023-12-26 20:23:00,808][105692] Updated weights for policy 0, policy_version 689008 (0.0007) [2023-12-26 20:23:00,870][105692] Updated weights for policy 0, policy_version 689018 (0.0005) [2023-12-26 20:23:00,926][105692] Updated weights for policy 0, policy_version 689028 (0.0005) [2023-12-26 20:23:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 353050624. Throughput: 0: 9858.7, 1: 9862.5. Samples: 353019384. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) [2023-12-26 20:23:01,063][104569] Avg episode reward: [(0, '8814.388'), (1, '8986.350')] [2023-12-26 20:23:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000689032_176422912.pth... [2023-12-26 20:23:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000689880_176627712.pth... [2023-12-26 20:23:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000688728_176332800.pth [2023-12-26 20:23:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000687880_176128000.pth [2023-12-26 20:23:01,442][105620] Updated weights for policy 1, policy_version 689889 (0.0006) [2023-12-26 20:23:01,488][105586] KL-divergence is very high: 121.8554 [2023-12-26 20:23:01,506][105620] Updated weights for policy 1, policy_version 689899 (0.0006) [2023-12-26 20:23:01,534][105586] KL-divergence is very high: 100.7854 [2023-12-26 20:23:01,565][105620] Updated weights for policy 1, policy_version 689909 (0.0006) [2023-12-26 20:23:01,595][105692] Updated weights for policy 0, policy_version 689038 (0.0007) [2023-12-26 20:23:01,665][105692] Updated weights for policy 0, policy_version 689048 (0.0008) [2023-12-26 20:23:01,737][105692] Updated weights for policy 0, policy_version 689058 (0.0008) [2023-12-26 20:23:02,276][105620] Updated weights for policy 1, policy_version 689919 (0.0007) [2023-12-26 20:23:02,344][105620] Updated weights for policy 1, policy_version 689929 (0.0009) [2023-12-26 20:23:02,391][105692] Updated weights for policy 0, policy_version 689068 (0.0009) [2023-12-26 20:23:02,406][105620] Updated weights for policy 1, policy_version 689939 (0.0007) [2023-12-26 20:23:02,454][105692] Updated weights for policy 0, policy_version 689078 (0.0011) [2023-12-26 20:23:02,524][105692] Updated weights for policy 0, policy_version 689088 (0.0011) [2023-12-26 20:23:02,942][105620] Updated weights for policy 1, policy_version 689949 (0.0007) [2023-12-26 20:23:02,999][105620] Updated weights for policy 1, policy_version 689959 (0.0009) [2023-12-26 20:23:03,045][105620] Updated weights for policy 1, policy_version 689969 (0.0009) [2023-12-26 20:23:03,259][105692] Updated weights for policy 0, policy_version 689098 (0.0010) [2023-12-26 20:23:03,310][105692] Updated weights for policy 0, policy_version 689108 (0.0007) [2023-12-26 20:23:03,370][105692] Updated weights for policy 0, policy_version 689118 (0.0005) [2023-12-26 20:23:03,418][105692] Updated weights for policy 0, policy_version 689128 (0.0005) [2023-12-26 20:23:03,823][105620] Updated weights for policy 1, policy_version 689979 (0.0009) [2023-12-26 20:23:03,875][105620] Updated weights for policy 1, policy_version 689989 (0.0009) [2023-12-26 20:23:03,932][105620] Updated weights for policy 1, policy_version 689999 (0.0008) [2023-12-26 20:23:04,080][105692] Updated weights for policy 0, policy_version 689138 (0.0009) [2023-12-26 20:23:04,142][105692] Updated weights for policy 0, policy_version 689148 (0.0009) [2023-12-26 20:23:04,201][105692] Updated weights for policy 0, policy_version 689158 (0.0009) [2023-12-26 20:23:04,712][105620] Updated weights for policy 1, policy_version 690009 (0.0009) [2023-12-26 20:23:04,782][105620] Updated weights for policy 1, policy_version 690019 (0.0010) [2023-12-26 20:23:04,848][105620] Updated weights for policy 1, policy_version 690029 (0.0010) [2023-12-26 20:23:04,911][105620] Updated weights for policy 1, policy_version 690039 (0.0007) [2023-12-26 20:23:04,915][105692] Updated weights for policy 0, policy_version 689168 (0.0007) [2023-12-26 20:23:04,974][105692] Updated weights for policy 0, policy_version 689178 (0.0006) [2023-12-26 20:23:05,023][105692] Updated weights for policy 0, policy_version 689188 (0.0005) [2023-12-26 20:23:05,640][105692] Updated weights for policy 0, policy_version 689198 (0.0008) [2023-12-26 20:23:05,695][105692] Updated weights for policy 0, policy_version 689208 (0.0008) [2023-12-26 20:23:05,728][105620] Updated weights for policy 1, policy_version 690049 (0.0008) [2023-12-26 20:23:05,754][105692] Updated weights for policy 0, policy_version 689218 (0.0007) [2023-12-26 20:23:05,776][105620] Updated weights for policy 1, policy_version 690059 (0.0006) [2023-12-26 20:23:05,821][105620] Updated weights for policy 1, policy_version 690069 (0.0008) [2023-12-26 20:23:06,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 353148928. Throughput: 0: 9763.1, 1: 9851.2. Samples: 353135472. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) [2023-12-26 20:23:06,063][104569] Avg episode reward: [(0, '8433.156'), (1, '9168.576')] [2023-12-26 20:23:06,519][105692] Updated weights for policy 0, policy_version 689228 (0.0008) [2023-12-26 20:23:06,576][105620] Updated weights for policy 1, policy_version 690079 (0.0007) [2023-12-26 20:23:06,579][105692] Updated weights for policy 0, policy_version 689238 (0.0007) [2023-12-26 20:23:06,630][105692] Updated weights for policy 0, policy_version 689248 (0.0008) [2023-12-26 20:23:06,636][105620] Updated weights for policy 1, policy_version 690089 (0.0007) [2023-12-26 20:23:06,694][105620] Updated weights for policy 1, policy_version 690099 (0.0007) [2023-12-26 20:23:07,378][105692] Updated weights for policy 0, policy_version 689258 (0.0007) [2023-12-26 20:23:07,437][105692] Updated weights for policy 0, policy_version 689268 (0.0009) [2023-12-26 20:23:07,465][105620] Updated weights for policy 1, policy_version 690109 (0.0007) [2023-12-26 20:23:07,487][105692] Updated weights for policy 0, policy_version 689278 (0.0009) [2023-12-26 20:23:07,517][105620] Updated weights for policy 1, policy_version 690119 (0.0005) [2023-12-26 20:23:07,535][105692] Updated weights for policy 0, policy_version 689288 (0.0009) [2023-12-26 20:23:07,585][105620] Updated weights for policy 1, policy_version 690129 (0.0005) [2023-12-26 20:23:08,121][105620] Updated weights for policy 1, policy_version 690139 (0.0006) [2023-12-26 20:23:08,182][105620] Updated weights for policy 1, policy_version 690150 (0.0008) [2023-12-26 20:23:08,230][105620] Updated weights for policy 1, policy_version 690160 (0.0005) [2023-12-26 20:23:08,409][105692] Updated weights for policy 0, policy_version 689298 (0.0008) [2023-12-26 20:23:08,471][105692] Updated weights for policy 0, policy_version 689308 (0.0008) [2023-12-26 20:23:08,528][105692] Updated weights for policy 0, policy_version 689318 (0.0010) [2023-12-26 20:23:08,843][105620] Updated weights for policy 1, policy_version 690170 (0.0007) [2023-12-26 20:23:08,899][105620] Updated weights for policy 1, policy_version 690180 (0.0011) [2023-12-26 20:23:08,957][105620] Updated weights for policy 1, policy_version 690190 (0.0010) [2023-12-26 20:23:09,015][105620] Updated weights for policy 1, policy_version 690200 (0.0010) [2023-12-26 20:23:09,279][105692] Updated weights for policy 0, policy_version 689328 (0.0009) [2023-12-26 20:23:09,336][105692] Updated weights for policy 0, policy_version 689338 (0.0009) [2023-12-26 20:23:09,389][105692] Updated weights for policy 0, policy_version 689348 (0.0009) [2023-12-26 20:23:09,766][105620] Updated weights for policy 1, policy_version 690210 (0.0011) [2023-12-26 20:23:09,823][105620] Updated weights for policy 1, policy_version 690220 (0.0011) [2023-12-26 20:23:09,888][105620] Updated weights for policy 1, policy_version 690230 (0.0008) [2023-12-26 20:23:10,123][105692] Updated weights for policy 0, policy_version 689358 (0.0007) [2023-12-26 20:23:10,172][105692] Updated weights for policy 0, policy_version 689368 (0.0008) [2023-12-26 20:23:10,233][105692] Updated weights for policy 0, policy_version 689378 (0.0009) [2023-12-26 20:23:10,606][105620] Updated weights for policy 1, policy_version 690240 (0.0010) [2023-12-26 20:23:10,655][105620] Updated weights for policy 1, policy_version 690250 (0.0011) [2023-12-26 20:23:10,703][105620] Updated weights for policy 1, policy_version 690260 (0.0010) [2023-12-26 20:23:11,038][105692] Updated weights for policy 0, policy_version 689388 (0.0009) [2023-12-26 20:23:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 353239040. Throughput: 0: 9695.9, 1: 9775.0. Samples: 353251320. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) [2023-12-26 20:23:11,063][104569] Avg episode reward: [(0, '8606.249'), (1, '9168.751')] [2023-12-26 20:23:11,101][105692] Updated weights for policy 0, policy_version 689398 (0.0007) [2023-12-26 20:23:11,173][105692] Updated weights for policy 0, policy_version 689408 (0.0007) [2023-12-26 20:23:11,420][105620] Updated weights for policy 1, policy_version 690270 (0.0009) [2023-12-26 20:23:11,485][105620] Updated weights for policy 1, policy_version 690280 (0.0011) [2023-12-26 20:23:11,545][105620] Updated weights for policy 1, policy_version 690290 (0.0011) [2023-12-26 20:23:11,877][105692] Updated weights for policy 0, policy_version 689418 (0.0009) [2023-12-26 20:23:11,934][105692] Updated weights for policy 0, policy_version 689428 (0.0008) [2023-12-26 20:23:11,996][105692] Updated weights for policy 0, policy_version 689438 (0.0008) [2023-12-26 20:23:12,058][105692] Updated weights for policy 0, policy_version 689448 (0.0009) [2023-12-26 20:23:12,295][105620] Updated weights for policy 1, policy_version 690300 (0.0010) [2023-12-26 20:23:12,367][105620] Updated weights for policy 1, policy_version 690310 (0.0009) [2023-12-26 20:23:12,419][105620] Updated weights for policy 1, policy_version 690320 (0.0010) [2023-12-26 20:23:12,813][105692] Updated weights for policy 0, policy_version 689458 (0.0009) [2023-12-26 20:23:12,881][105692] Updated weights for policy 0, policy_version 689468 (0.0009) [2023-12-26 20:23:12,934][105692] Updated weights for policy 0, policy_version 689478 (0.0008) [2023-12-26 20:23:13,145][105620] Updated weights for policy 1, policy_version 690330 (0.0010) [2023-12-26 20:23:13,203][105620] Updated weights for policy 1, policy_version 690340 (0.0010) [2023-12-26 20:23:13,251][105620] Updated weights for policy 1, policy_version 690350 (0.0009) [2023-12-26 20:23:13,303][105620] Updated weights for policy 1, policy_version 690360 (0.0010) [2023-12-26 20:23:13,655][105692] Updated weights for policy 0, policy_version 689488 (0.0006) [2023-12-26 20:23:13,709][105692] Updated weights for policy 0, policy_version 689498 (0.0005) [2023-12-26 20:23:13,759][105692] Updated weights for policy 0, policy_version 689508 (0.0005) [2023-12-26 20:23:14,037][105620] Updated weights for policy 1, policy_version 690370 (0.0006) [2023-12-26 20:23:14,098][105620] Updated weights for policy 1, policy_version 690380 (0.0010) [2023-12-26 20:23:14,150][105620] Updated weights for policy 1, policy_version 690390 (0.0010) [2023-12-26 20:23:14,313][105692] Updated weights for policy 0, policy_version 689518 (0.0005) [2023-12-26 20:23:14,365][105692] Updated weights for policy 0, policy_version 689528 (0.0005) [2023-12-26 20:23:14,422][105692] Updated weights for policy 0, policy_version 689538 (0.0005) [2023-12-26 20:23:14,799][105620] Updated weights for policy 1, policy_version 690400 (0.0010) [2023-12-26 20:23:14,851][105620] Updated weights for policy 1, policy_version 690410 (0.0010) [2023-12-26 20:23:14,908][105620] Updated weights for policy 1, policy_version 690420 (0.0010) [2023-12-26 20:23:15,056][105692] Updated weights for policy 0, policy_version 689548 (0.0008) [2023-12-26 20:23:15,113][105692] Updated weights for policy 0, policy_version 689558 (0.0008) [2023-12-26 20:23:15,169][105692] Updated weights for policy 0, policy_version 689568 (0.0009) [2023-12-26 20:23:15,547][105620] Updated weights for policy 1, policy_version 690430 (0.0007) [2023-12-26 20:23:15,593][105620] Updated weights for policy 1, policy_version 690440 (0.0005) [2023-12-26 20:23:15,643][105620] Updated weights for policy 1, policy_version 690450 (0.0005) [2023-12-26 20:23:15,933][105692] Updated weights for policy 0, policy_version 689578 (0.0009) [2023-12-26 20:23:15,990][105692] Updated weights for policy 0, policy_version 689588 (0.0005) [2023-12-26 20:23:16,047][105692] Updated weights for policy 0, policy_version 689598 (0.0005) [2023-12-26 20:23:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 353337344. Throughput: 0: 9641.9, 1: 9744.9. Samples: 353307496. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) [2023-12-26 20:23:16,063][104569] Avg episode reward: [(0, '8928.983'), (1, '9261.060')] [2023-12-26 20:23:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000690456_176775168.pth... [2023-12-26 20:23:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000689304_176480256.pth [2023-12-26 20:23:16,102][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000689608_176570368.pth... [2023-12-26 20:23:16,103][105692] Updated weights for policy 0, policy_version 689608 (0.0005) [2023-12-26 20:23:16,106][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000688456_176275456.pth [2023-12-26 20:23:16,265][105620] Updated weights for policy 1, policy_version 690460 (0.0006) [2023-12-26 20:23:16,323][105620] Updated weights for policy 1, policy_version 690470 (0.0007) [2023-12-26 20:23:16,378][105620] Updated weights for policy 1, policy_version 690480 (0.0010) [2023-12-26 20:23:16,713][105692] Updated weights for policy 0, policy_version 689618 (0.0008) [2023-12-26 20:23:16,756][105692] Updated weights for policy 0, policy_version 689628 (0.0008) [2023-12-26 20:23:16,811][105692] Updated weights for policy 0, policy_version 689638 (0.0008) [2023-12-26 20:23:17,116][105620] Updated weights for policy 1, policy_version 690490 (0.0007) [2023-12-26 20:23:17,175][105620] Updated weights for policy 1, policy_version 690500 (0.0010) [2023-12-26 20:23:17,227][105620] Updated weights for policy 1, policy_version 690510 (0.0010) [2023-12-26 20:23:17,277][105620] Updated weights for policy 1, policy_version 690520 (0.0010) [2023-12-26 20:23:17,528][105692] Updated weights for policy 0, policy_version 689648 (0.0008) [2023-12-26 20:23:17,584][105692] Updated weights for policy 0, policy_version 689658 (0.0008) [2023-12-26 20:23:17,636][105692] Updated weights for policy 0, policy_version 689668 (0.0008) [2023-12-26 20:23:17,970][105620] Updated weights for policy 1, policy_version 690530 (0.0005) [2023-12-26 20:23:18,025][105620] Updated weights for policy 1, policy_version 690540 (0.0005) [2023-12-26 20:23:18,081][105620] Updated weights for policy 1, policy_version 690550 (0.0008) [2023-12-26 20:23:18,490][105692] Updated weights for policy 0, policy_version 689678 (0.0008) [2023-12-26 20:23:18,551][105692] Updated weights for policy 0, policy_version 689688 (0.0007) [2023-12-26 20:23:18,618][105692] Updated weights for policy 0, policy_version 689698 (0.0008) [2023-12-26 20:23:18,712][105620] Updated weights for policy 1, policy_version 690560 (0.0010) [2023-12-26 20:23:18,760][105620] Updated weights for policy 1, policy_version 690570 (0.0010) [2023-12-26 20:23:18,808][105620] Updated weights for policy 1, policy_version 690580 (0.0010) [2023-12-26 20:23:19,395][105692] Updated weights for policy 0, policy_version 689708 (0.0009) [2023-12-26 20:23:19,456][105692] Updated weights for policy 0, policy_version 689718 (0.0008) [2023-12-26 20:23:19,520][105692] Updated weights for policy 0, policy_version 689728 (0.0008) [2023-12-26 20:23:19,568][105620] Updated weights for policy 1, policy_version 690590 (0.0009) [2023-12-26 20:23:19,634][105620] Updated weights for policy 1, policy_version 690600 (0.0008) [2023-12-26 20:23:19,696][105620] Updated weights for policy 1, policy_version 690610 (0.0008) [2023-12-26 20:23:20,283][105692] Updated weights for policy 0, policy_version 689738 (0.0007) [2023-12-26 20:23:20,350][105692] Updated weights for policy 0, policy_version 689748 (0.0009) [2023-12-26 20:23:20,408][105692] Updated weights for policy 0, policy_version 689758 (0.0009) [2023-12-26 20:23:20,449][105620] Updated weights for policy 1, policy_version 690620 (0.0008) [2023-12-26 20:23:20,467][105692] Updated weights for policy 0, policy_version 689768 (0.0009) [2023-12-26 20:23:20,508][105620] Updated weights for policy 1, policy_version 690630 (0.0008) [2023-12-26 20:23:20,579][105620] Updated weights for policy 1, policy_version 690640 (0.0009) [2023-12-26 20:23:21,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 353435648. Throughput: 0: 9641.4, 1: 9792.4. Samples: 353427916. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) [2023-12-26 20:23:21,062][104569] Avg episode reward: [(0, '9047.523'), (1, '9080.941')] [2023-12-26 20:23:21,209][105692] Updated weights for policy 0, policy_version 689778 (0.0009) [2023-12-26 20:23:21,268][105692] Updated weights for policy 0, policy_version 689788 (0.0007) [2023-12-26 20:23:21,329][105692] Updated weights for policy 0, policy_version 689798 (0.0006) [2023-12-26 20:23:21,346][105620] Updated weights for policy 1, policy_version 690650 (0.0009) [2023-12-26 20:23:21,416][105620] Updated weights for policy 1, policy_version 690660 (0.0011) [2023-12-26 20:23:21,479][105620] Updated weights for policy 1, policy_version 690670 (0.0009) [2023-12-26 20:23:21,536][105620] Updated weights for policy 1, policy_version 690680 (0.0006) [2023-12-26 20:23:22,077][105692] Updated weights for policy 0, policy_version 689808 (0.0009) [2023-12-26 20:23:22,137][105692] Updated weights for policy 0, policy_version 689818 (0.0008) [2023-12-26 20:23:22,209][105692] Updated weights for policy 0, policy_version 689828 (0.0007) [2023-12-26 20:23:22,269][105620] Updated weights for policy 1, policy_version 690690 (0.0007) [2023-12-26 20:23:22,335][105620] Updated weights for policy 1, policy_version 690700 (0.0007) [2023-12-26 20:23:22,402][105620] Updated weights for policy 1, policy_version 690710 (0.0008) [2023-12-26 20:23:23,014][105692] Updated weights for policy 0, policy_version 689838 (0.0009) [2023-12-26 20:23:23,056][105620] Updated weights for policy 1, policy_version 690720 (0.0006) [2023-12-26 20:23:23,062][105692] Updated weights for policy 0, policy_version 689848 (0.0008) [2023-12-26 20:23:23,091][105586] KL-divergence is very high: 118.0049 [2023-12-26 20:23:23,108][105620] Updated weights for policy 1, policy_version 690730 (0.0006) [2023-12-26 20:23:23,117][105692] Updated weights for policy 0, policy_version 689858 (0.0009) [2023-12-26 20:23:23,142][105586] KL-divergence is very high: 168.3436 [2023-12-26 20:23:23,176][105620] Updated weights for policy 1, policy_version 690740 (0.0006) [2023-12-26 20:23:23,197][105586] KL-divergence is very high: 123.5309 [2023-12-26 20:23:23,819][105620] Updated weights for policy 1, policy_version 690750 (0.0008) [2023-12-26 20:23:23,874][105620] Updated weights for policy 1, policy_version 690760 (0.0009) [2023-12-26 20:23:23,930][105620] Updated weights for policy 1, policy_version 690770 (0.0008) [2023-12-26 20:23:23,938][105692] Updated weights for policy 0, policy_version 689868 (0.0008) [2023-12-26 20:23:23,998][105692] Updated weights for policy 0, policy_version 689878 (0.0008) [2023-12-26 20:23:24,058][105692] Updated weights for policy 0, policy_version 689888 (0.0009) [2023-12-26 20:23:24,691][105620] Updated weights for policy 1, policy_version 690780 (0.0006) [2023-12-26 20:23:24,745][105620] Updated weights for policy 1, policy_version 690790 (0.0005) [2023-12-26 20:23:24,770][105692] Updated weights for policy 0, policy_version 689898 (0.0008) [2023-12-26 20:23:24,799][105620] Updated weights for policy 1, policy_version 690800 (0.0005) [2023-12-26 20:23:24,822][105692] Updated weights for policy 0, policy_version 689908 (0.0009) [2023-12-26 20:23:24,876][105692] Updated weights for policy 0, policy_version 689918 (0.0009) [2023-12-26 20:23:24,934][105692] Updated weights for policy 0, policy_version 689928 (0.0009) [2023-12-26 20:23:25,426][105620] Updated weights for policy 1, policy_version 690810 (0.0005) [2023-12-26 20:23:25,493][105620] Updated weights for policy 1, policy_version 690820 (0.0006) [2023-12-26 20:23:25,555][105620] Updated weights for policy 1, policy_version 690830 (0.0005) [2023-12-26 20:23:25,600][105692] Updated weights for policy 0, policy_version 689938 (0.0007) [2023-12-26 20:23:25,606][105620] Updated weights for policy 1, policy_version 690840 (0.0005) [2023-12-26 20:23:25,648][105692] Updated weights for policy 0, policy_version 689948 (0.0010) [2023-12-26 20:23:25,713][105692] Updated weights for policy 0, policy_version 689958 (0.0005) [2023-12-26 20:23:26,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19660.8). Total num frames: 353533952. Throughput: 0: 9592.9, 1: 9858.7. Samples: 353543716. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) [2023-12-26 20:23:26,063][104569] Avg episode reward: [(0, '8808.088'), (1, '8988.264')] [2023-12-26 20:23:26,231][105620] Updated weights for policy 1, policy_version 690850 (0.0005) [2023-12-26 20:23:26,289][105620] Updated weights for policy 1, policy_version 690860 (0.0005) [2023-12-26 20:23:26,332][105692] Updated weights for policy 0, policy_version 689968 (0.0010) [2023-12-26 20:23:26,341][105620] Updated weights for policy 1, policy_version 690870 (0.0007) [2023-12-26 20:23:26,381][105692] Updated weights for policy 0, policy_version 689978 (0.0007) [2023-12-26 20:23:26,438][105692] Updated weights for policy 0, policy_version 689988 (0.0007) [2023-12-26 20:23:27,011][105620] Updated weights for policy 1, policy_version 690880 (0.0008) [2023-12-26 20:23:27,065][105620] Updated weights for policy 1, policy_version 690890 (0.0008) [2023-12-26 20:23:27,113][105620] Updated weights for policy 1, policy_version 690900 (0.0008) [2023-12-26 20:23:27,150][105692] Updated weights for policy 0, policy_version 689998 (0.0010) [2023-12-26 20:23:27,198][105692] Updated weights for policy 0, policy_version 690008 (0.0010) [2023-12-26 20:23:27,245][105692] Updated weights for policy 0, policy_version 690018 (0.0010) [2023-12-26 20:23:27,748][105620] Updated weights for policy 1, policy_version 690910 (0.0006) [2023-12-26 20:23:27,799][105620] Updated weights for policy 1, policy_version 690920 (0.0005) [2023-12-26 20:23:27,842][105692] Updated weights for policy 0, policy_version 690028 (0.0008) [2023-12-26 20:23:27,852][105620] Updated weights for policy 1, policy_version 690930 (0.0006) [2023-12-26 20:23:27,895][105692] Updated weights for policy 0, policy_version 690038 (0.0005) [2023-12-26 20:23:27,955][105692] Updated weights for policy 0, policy_version 690048 (0.0005) [2023-12-26 20:23:28,526][105692] Updated weights for policy 0, policy_version 690058 (0.0006) [2023-12-26 20:23:28,585][105692] Updated weights for policy 0, policy_version 690068 (0.0010) [2023-12-26 20:23:28,635][105620] Updated weights for policy 1, policy_version 690940 (0.0008) [2023-12-26 20:23:28,647][105692] Updated weights for policy 0, policy_version 690078 (0.0010) [2023-12-26 20:23:28,682][105620] Updated weights for policy 1, policy_version 690950 (0.0009) [2023-12-26 20:23:28,706][105692] Updated weights for policy 0, policy_version 690088 (0.0010) [2023-12-26 20:23:28,734][105620] Updated weights for policy 1, policy_version 690960 (0.0006) [2023-12-26 20:23:29,406][105692] Updated weights for policy 0, policy_version 690098 (0.0010) [2023-12-26 20:23:29,471][105620] Updated weights for policy 1, policy_version 690970 (0.0005) [2023-12-26 20:23:29,474][105692] Updated weights for policy 0, policy_version 690108 (0.0010) [2023-12-26 20:23:29,520][105620] Updated weights for policy 1, policy_version 690980 (0.0007) [2023-12-26 20:23:29,533][105692] Updated weights for policy 0, policy_version 690118 (0.0010) [2023-12-26 20:23:29,582][105620] Updated weights for policy 1, policy_version 690990 (0.0009) [2023-12-26 20:23:29,637][105620] Updated weights for policy 1, policy_version 691000 (0.0008) [2023-12-26 20:23:30,203][105692] Updated weights for policy 0, policy_version 690128 (0.0006) [2023-12-26 20:23:30,257][105692] Updated weights for policy 0, policy_version 690138 (0.0005) [2023-12-26 20:23:30,313][105692] Updated weights for policy 0, policy_version 690148 (0.0005) [2023-12-26 20:23:30,391][105620] Updated weights for policy 1, policy_version 691010 (0.0008) [2023-12-26 20:23:30,400][105586] KL-divergence is very high: 444.4330 [2023-12-26 20:23:30,443][105620] Updated weights for policy 1, policy_version 691020 (0.0008) [2023-12-26 20:23:30,444][105586] KL-divergence is very high: 656.2839 [2023-12-26 20:23:30,490][105586] KL-divergence is very high: 587.4965 [2023-12-26 20:23:30,499][105620] Updated weights for policy 1, policy_version 691030 (0.0008) [2023-12-26 20:23:31,010][105692] Updated weights for policy 0, policy_version 690158 (0.0009) [2023-12-26 20:23:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19660.8). Total num frames: 353632256. Throughput: 0: 9714.8, 1: 9878.1. Samples: 353607688. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) [2023-12-26 20:23:31,063][104569] Avg episode reward: [(0, '8811.816'), (1, '8985.352')] [2023-12-26 20:23:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000691032_176922624.pth... [2023-12-26 20:23:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000689880_176627712.pth [2023-12-26 20:23:31,074][105692] Updated weights for policy 0, policy_version 690168 (0.0007) [2023-12-26 20:23:31,128][105692] Updated weights for policy 0, policy_version 690178 (0.0008) [2023-12-26 20:23:31,136][105620] Updated weights for policy 1, policy_version 691040 (0.0008) [2023-12-26 20:23:31,170][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000690184_176717824.pth... [2023-12-26 20:23:31,175][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000689032_176422912.pth [2023-12-26 20:23:31,195][105620] Updated weights for policy 1, policy_version 691050 (0.0008) [2023-12-26 20:23:31,261][105620] Updated weights for policy 1, policy_version 691060 (0.0008) [2023-12-26 20:23:31,913][105692] Updated weights for policy 0, policy_version 690188 (0.0011) [2023-12-26 20:23:31,969][105692] Updated weights for policy 0, policy_version 690198 (0.0010) [2023-12-26 20:23:31,972][105620] Updated weights for policy 1, policy_version 691070 (0.0007) [2023-12-26 20:23:32,022][105692] Updated weights for policy 0, policy_version 690208 (0.0008) [2023-12-26 20:23:32,036][105620] Updated weights for policy 1, policy_version 691080 (0.0006) [2023-12-26 20:23:32,088][105620] Updated weights for policy 1, policy_version 691090 (0.0006) [2023-12-26 20:23:32,650][105620] Updated weights for policy 1, policy_version 691100 (0.0008) [2023-12-26 20:23:32,697][105692] Updated weights for policy 0, policy_version 690218 (0.0006) [2023-12-26 20:23:32,706][105620] Updated weights for policy 1, policy_version 691110 (0.0008) [2023-12-26 20:23:32,759][105692] Updated weights for policy 0, policy_version 690228 (0.0007) [2023-12-26 20:23:32,764][105620] Updated weights for policy 1, policy_version 691120 (0.0009) [2023-12-26 20:23:32,818][105692] Updated weights for policy 0, policy_version 690238 (0.0006) [2023-12-26 20:23:32,886][105692] Updated weights for policy 0, policy_version 690248 (0.0009) [2023-12-26 20:23:33,510][105692] Updated weights for policy 0, policy_version 690258 (0.0010) [2023-12-26 20:23:33,538][105620] Updated weights for policy 1, policy_version 691130 (0.0010) [2023-12-26 20:23:33,557][105692] Updated weights for policy 0, policy_version 690268 (0.0010) [2023-12-26 20:23:33,598][105620] Updated weights for policy 1, policy_version 691140 (0.0006) [2023-12-26 20:23:33,611][105692] Updated weights for policy 0, policy_version 690278 (0.0010) [2023-12-26 20:23:33,654][105620] Updated weights for policy 1, policy_version 691150 (0.0007) [2023-12-26 20:23:33,707][105620] Updated weights for policy 1, policy_version 691160 (0.0008) [2023-12-26 20:23:34,273][105692] Updated weights for policy 0, policy_version 690288 (0.0011) [2023-12-26 20:23:34,329][105692] Updated weights for policy 0, policy_version 690298 (0.0010) [2023-12-26 20:23:34,331][105620] Updated weights for policy 1, policy_version 691170 (0.0006) [2023-12-26 20:23:34,390][105620] Updated weights for policy 1, policy_version 691180 (0.0005) [2023-12-26 20:23:34,396][105692] Updated weights for policy 0, policy_version 690308 (0.0011) [2023-12-26 20:23:34,445][105620] Updated weights for policy 1, policy_version 691190 (0.0006) [2023-12-26 20:23:35,108][105692] Updated weights for policy 0, policy_version 690318 (0.0010) [2023-12-26 20:23:35,160][105692] Updated weights for policy 0, policy_version 690328 (0.0010) [2023-12-26 20:23:35,178][105620] Updated weights for policy 1, policy_version 691200 (0.0006) [2023-12-26 20:23:35,213][105692] Updated weights for policy 0, policy_version 690338 (0.0010) [2023-12-26 20:23:35,224][105620] Updated weights for policy 1, policy_version 691210 (0.0005) [2023-12-26 20:23:35,269][105620] Updated weights for policy 1, policy_version 691220 (0.0005) [2023-12-26 20:23:35,949][105692] Updated weights for policy 0, policy_version 690348 (0.0008) [2023-12-26 20:23:35,964][105620] Updated weights for policy 1, policy_version 691230 (0.0006) [2023-12-26 20:23:36,009][105692] Updated weights for policy 0, policy_version 690358 (0.0007) [2023-12-26 20:23:36,029][105620] Updated weights for policy 1, policy_version 691240 (0.0005) [2023-12-26 20:23:36,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 353730560. Throughput: 0: 9665.8, 1: 9877.9. Samples: 353727448. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) [2023-12-26 20:23:36,062][105692] Updated weights for policy 0, policy_version 690368 (0.0007) [2023-12-26 20:23:36,063][104569] Avg episode reward: [(0, '8989.865'), (1, '8984.774')] [2023-12-26 20:23:36,092][105620] Updated weights for policy 1, policy_version 691250 (0.0006) [2023-12-26 20:23:36,599][105620] Updated weights for policy 1, policy_version 691260 (0.0007) [2023-12-26 20:23:36,660][105620] Updated weights for policy 1, policy_version 691270 (0.0006) [2023-12-26 20:23:36,718][105620] Updated weights for policy 1, policy_version 691280 (0.0007) [2023-12-26 20:23:36,786][105692] Updated weights for policy 0, policy_version 690378 (0.0008) [2023-12-26 20:23:36,835][105692] Updated weights for policy 0, policy_version 690388 (0.0011) [2023-12-26 20:23:36,893][105692] Updated weights for policy 0, policy_version 690398 (0.0006) [2023-12-26 20:23:36,939][105692] Updated weights for policy 0, policy_version 690408 (0.0005) [2023-12-26 20:23:37,369][105620] Updated weights for policy 1, policy_version 691290 (0.0006) [2023-12-26 20:23:37,428][105620] Updated weights for policy 1, policy_version 691300 (0.0008) [2023-12-26 20:23:37,491][105620] Updated weights for policy 1, policy_version 691310 (0.0008) [2023-12-26 20:23:37,552][105620] Updated weights for policy 1, policy_version 691320 (0.0008) [2023-12-26 20:23:37,596][105692] Updated weights for policy 0, policy_version 690418 (0.0010) [2023-12-26 20:23:37,645][105692] Updated weights for policy 0, policy_version 690428 (0.0010) [2023-12-26 20:23:37,697][105692] Updated weights for policy 0, policy_version 690438 (0.0010) [2023-12-26 20:23:38,227][105620] Updated weights for policy 1, policy_version 691330 (0.0008) [2023-12-26 20:23:38,276][105620] Updated weights for policy 1, policy_version 691340 (0.0008) [2023-12-26 20:23:38,343][105620] Updated weights for policy 1, policy_version 691350 (0.0011) [2023-12-26 20:23:38,494][105692] Updated weights for policy 0, policy_version 690448 (0.0011) [2023-12-26 20:23:38,556][105692] Updated weights for policy 0, policy_version 690458 (0.0011) [2023-12-26 20:23:38,622][105692] Updated weights for policy 0, policy_version 690468 (0.0011) [2023-12-26 20:23:39,096][105620] Updated weights for policy 1, policy_version 691360 (0.0010) [2023-12-26 20:23:39,144][105620] Updated weights for policy 1, policy_version 691370 (0.0010) [2023-12-26 20:23:39,197][105620] Updated weights for policy 1, policy_version 691380 (0.0010) [2023-12-26 20:23:39,336][105692] Updated weights for policy 0, policy_version 690478 (0.0010) [2023-12-26 20:23:39,404][105692] Updated weights for policy 0, policy_version 690488 (0.0011) [2023-12-26 20:23:39,468][105692] Updated weights for policy 0, policy_version 690498 (0.0011) [2023-12-26 20:23:40,029][105620] Updated weights for policy 1, policy_version 691390 (0.0009) [2023-12-26 20:23:40,118][105620] Updated weights for policy 1, policy_version 691400 (0.0009) [2023-12-26 20:23:40,177][105620] Updated weights for policy 1, policy_version 691410 (0.0008) [2023-12-26 20:23:40,201][105692] Updated weights for policy 0, policy_version 690508 (0.0011) [2023-12-26 20:23:40,261][105692] Updated weights for policy 0, policy_version 690518 (0.0011) [2023-12-26 20:23:40,314][105692] Updated weights for policy 0, policy_version 690528 (0.0011) [2023-12-26 20:23:40,857][105620] Updated weights for policy 1, policy_version 691420 (0.0008) [2023-12-26 20:23:40,924][105620] Updated weights for policy 1, policy_version 691430 (0.0005) [2023-12-26 20:23:40,981][105620] Updated weights for policy 1, policy_version 691440 (0.0008) [2023-12-26 20:23:41,029][105692] Updated weights for policy 0, policy_version 690538 (0.0010) [2023-12-26 20:23:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 353837056. Throughput: 0: 9666.6, 1: 9979.8. Samples: 353845848. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) [2023-12-26 20:23:41,063][104569] Avg episode reward: [(0, '9083.626'), (1, '9260.839')] [2023-12-26 20:23:41,085][105692] Updated weights for policy 0, policy_version 690548 (0.0010) [2023-12-26 20:23:41,148][105692] Updated weights for policy 0, policy_version 690558 (0.0010) [2023-12-26 20:23:41,217][105692] Updated weights for policy 0, policy_version 690568 (0.0010) [2023-12-26 20:23:41,630][105620] Updated weights for policy 1, policy_version 691450 (0.0007) [2023-12-26 20:23:41,694][105620] Updated weights for policy 1, policy_version 691460 (0.0009) [2023-12-26 20:23:41,766][105620] Updated weights for policy 1, policy_version 691470 (0.0008) [2023-12-26 20:23:41,827][105620] Updated weights for policy 1, policy_version 691480 (0.0009) [2023-12-26 20:23:41,967][105692] Updated weights for policy 0, policy_version 690578 (0.0011) [2023-12-26 20:23:42,013][105692] Updated weights for policy 0, policy_version 690588 (0.0011) [2023-12-26 20:23:42,068][105692] Updated weights for policy 0, policy_version 690598 (0.0010) [2023-12-26 20:23:42,558][105620] Updated weights for policy 1, policy_version 691490 (0.0009) [2023-12-26 20:23:42,626][105620] Updated weights for policy 1, policy_version 691500 (0.0009) [2023-12-26 20:23:42,689][105620] Updated weights for policy 1, policy_version 691510 (0.0009) [2023-12-26 20:23:42,774][105692] Updated weights for policy 0, policy_version 690608 (0.0009) [2023-12-26 20:23:42,839][105692] Updated weights for policy 0, policy_version 690618 (0.0009) [2023-12-26 20:23:42,900][105692] Updated weights for policy 0, policy_version 690628 (0.0010) [2023-12-26 20:23:43,369][105620] Updated weights for policy 1, policy_version 691520 (0.0006) [2023-12-26 20:23:43,430][105620] Updated weights for policy 1, policy_version 691530 (0.0006) [2023-12-26 20:23:43,492][105620] Updated weights for policy 1, policy_version 691540 (0.0006) [2023-12-26 20:23:43,639][105692] Updated weights for policy 0, policy_version 690638 (0.0010) [2023-12-26 20:23:43,694][105692] Updated weights for policy 0, policy_version 690648 (0.0010) [2023-12-26 20:23:43,741][105692] Updated weights for policy 0, policy_version 690658 (0.0010) [2023-12-26 20:23:44,211][105620] Updated weights for policy 1, policy_version 691550 (0.0007) [2023-12-26 20:23:44,258][105620] Updated weights for policy 1, policy_version 691560 (0.0005) [2023-12-26 20:23:44,302][105620] Updated weights for policy 1, policy_version 691570 (0.0005) [2023-12-26 20:23:44,394][105692] Updated weights for policy 0, policy_version 690668 (0.0010) [2023-12-26 20:23:44,444][105692] Updated weights for policy 0, policy_version 690678 (0.0009) [2023-12-26 20:23:44,500][105692] Updated weights for policy 0, policy_version 690688 (0.0010) [2023-12-26 20:23:44,991][105620] Updated weights for policy 1, policy_version 691580 (0.0005) [2023-12-26 20:23:45,050][105620] Updated weights for policy 1, policy_version 691590 (0.0010) [2023-12-26 20:23:45,105][105620] Updated weights for policy 1, policy_version 691601 (0.0010) [2023-12-26 20:23:45,213][105692] Updated weights for policy 0, policy_version 690698 (0.0009) [2023-12-26 20:23:45,273][105692] Updated weights for policy 0, policy_version 690708 (0.0009) [2023-12-26 20:23:45,333][105692] Updated weights for policy 0, policy_version 690718 (0.0007) [2023-12-26 20:23:45,394][105692] Updated weights for policy 0, policy_version 690728 (0.0007) [2023-12-26 20:23:45,922][105620] Updated weights for policy 1, policy_version 691611 (0.0009) [2023-12-26 20:23:45,981][105620] Updated weights for policy 1, policy_version 691621 (0.0009) [2023-12-26 20:23:46,044][105620] Updated weights for policy 1, policy_version 691631 (0.0009) [2023-12-26 20:23:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 353927168. Throughput: 0: 9705.2, 1: 9942.3. Samples: 353903516. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) [2023-12-26 20:23:46,062][104569] Avg episode reward: [(0, '8992.819'), (1, '9171.098')] [2023-12-26 20:23:46,082][105692] Updated weights for policy 0, policy_version 690738 (0.0006) [2023-12-26 20:23:46,095][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000691640_177078272.pth... [2023-12-26 20:23:46,100][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000690456_176775168.pth [2023-12-26 20:23:46,137][105692] Updated weights for policy 0, policy_version 690748 (0.0007) [2023-12-26 20:23:46,192][105692] Updated weights for policy 0, policy_version 690758 (0.0009) [2023-12-26 20:23:46,202][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000690760_176865280.pth... [2023-12-26 20:23:46,206][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000689608_176570368.pth [2023-12-26 20:23:46,776][105692] Updated weights for policy 0, policy_version 690768 (0.0006) [2023-12-26 20:23:46,828][105692] Updated weights for policy 0, policy_version 690778 (0.0005) [2023-12-26 20:23:46,852][105620] Updated weights for policy 1, policy_version 691641 (0.0007) [2023-12-26 20:23:46,878][105692] Updated weights for policy 0, policy_version 690788 (0.0006) [2023-12-26 20:23:46,906][105620] Updated weights for policy 1, policy_version 691651 (0.0009) [2023-12-26 20:23:46,967][105620] Updated weights for policy 1, policy_version 691661 (0.0007) [2023-12-26 20:23:47,015][105620] Updated weights for policy 1, policy_version 691671 (0.0010) [2023-12-26 20:23:47,550][105692] Updated weights for policy 0, policy_version 690798 (0.0008) [2023-12-26 20:23:47,606][105692] Updated weights for policy 0, policy_version 690808 (0.0008) [2023-12-26 20:23:47,652][105692] Updated weights for policy 0, policy_version 690818 (0.0006) [2023-12-26 20:23:47,748][105620] Updated weights for policy 1, policy_version 691681 (0.0010) [2023-12-26 20:23:47,812][105620] Updated weights for policy 1, policy_version 691691 (0.0010) [2023-12-26 20:23:47,873][105620] Updated weights for policy 1, policy_version 691701 (0.0010) [2023-12-26 20:23:48,317][105692] Updated weights for policy 0, policy_version 690828 (0.0005) [2023-12-26 20:23:48,382][105692] Updated weights for policy 0, policy_version 690838 (0.0008) [2023-12-26 20:23:48,441][105692] Updated weights for policy 0, policy_version 690848 (0.0007) [2023-12-26 20:23:48,579][105620] Updated weights for policy 1, policy_version 691711 (0.0007) [2023-12-26 20:23:48,645][105620] Updated weights for policy 1, policy_version 691721 (0.0010) [2023-12-26 20:23:48,711][105620] Updated weights for policy 1, policy_version 691731 (0.0010) [2023-12-26 20:23:49,100][105692] Updated weights for policy 0, policy_version 690858 (0.0006) [2023-12-26 20:23:49,164][105692] Updated weights for policy 0, policy_version 690868 (0.0006) [2023-12-26 20:23:49,224][105692] Updated weights for policy 0, policy_version 690878 (0.0006) [2023-12-26 20:23:49,285][105692] Updated weights for policy 0, policy_version 690888 (0.0009) [2023-12-26 20:23:49,412][105620] Updated weights for policy 1, policy_version 691741 (0.0009) [2023-12-26 20:23:49,475][105620] Updated weights for policy 1, policy_version 691751 (0.0009) [2023-12-26 20:23:49,547][105620] Updated weights for policy 1, policy_version 691761 (0.0009) [2023-12-26 20:23:49,962][105692] Updated weights for policy 0, policy_version 690898 (0.0009) [2023-12-26 20:23:50,028][105692] Updated weights for policy 0, policy_version 690908 (0.0009) [2023-12-26 20:23:50,086][105692] Updated weights for policy 0, policy_version 690918 (0.0009) [2023-12-26 20:23:50,205][105620] Updated weights for policy 1, policy_version 691771 (0.0005) [2023-12-26 20:23:50,271][105620] Updated weights for policy 1, policy_version 691781 (0.0005) [2023-12-26 20:23:50,332][105620] Updated weights for policy 1, policy_version 691791 (0.0005) [2023-12-26 20:23:50,798][105692] Updated weights for policy 0, policy_version 690928 (0.0007) [2023-12-26 20:23:50,864][105692] Updated weights for policy 0, policy_version 690938 (0.0009) [2023-12-26 20:23:50,930][105692] Updated weights for policy 0, policy_version 690948 (0.0008) [2023-12-26 20:23:50,958][105620] Updated weights for policy 1, policy_version 691801 (0.0005) [2023-12-26 20:23:51,012][105620] Updated weights for policy 1, policy_version 691811 (0.0009) [2023-12-26 20:23:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 354033664. Throughput: 0: 9822.6, 1: 9893.3. Samples: 354022684. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 20:23:51,062][104569] Avg episode reward: [(0, '8058.337'), (1, '8986.689')] [2023-12-26 20:23:51,080][105620] Updated weights for policy 1, policy_version 691821 (0.0009) [2023-12-26 20:23:51,141][105620] Updated weights for policy 1, policy_version 691831 (0.0009) [2023-12-26 20:23:51,713][105692] Updated weights for policy 0, policy_version 690958 (0.0008) [2023-12-26 20:23:51,779][105692] Updated weights for policy 0, policy_version 690968 (0.0008) [2023-12-26 20:23:51,836][105692] Updated weights for policy 0, policy_version 690978 (0.0007) [2023-12-26 20:23:51,886][105620] Updated weights for policy 1, policy_version 691841 (0.0007) [2023-12-26 20:23:51,948][105620] Updated weights for policy 1, policy_version 691851 (0.0009) [2023-12-26 20:23:52,003][105620] Updated weights for policy 1, policy_version 691861 (0.0005) [2023-12-26 20:23:52,611][105692] Updated weights for policy 0, policy_version 690988 (0.0008) [2023-12-26 20:23:52,658][105692] Updated weights for policy 0, policy_version 690998 (0.0007) [2023-12-26 20:23:52,668][105620] Updated weights for policy 1, policy_version 691871 (0.0007) [2023-12-26 20:23:52,705][105692] Updated weights for policy 0, policy_version 691008 (0.0008) [2023-12-26 20:23:52,725][105620] Updated weights for policy 1, policy_version 691881 (0.0009) [2023-12-26 20:23:52,787][105620] Updated weights for policy 1, policy_version 691891 (0.0008) [2023-12-26 20:23:53,480][105692] Updated weights for policy 0, policy_version 691018 (0.0008) [2023-12-26 20:23:53,513][105620] Updated weights for policy 1, policy_version 691901 (0.0009) [2023-12-26 20:23:53,548][105692] Updated weights for policy 0, policy_version 691028 (0.0007) [2023-12-26 20:23:53,566][105620] Updated weights for policy 1, policy_version 691911 (0.0007) [2023-12-26 20:23:53,601][105692] Updated weights for policy 0, policy_version 691038 (0.0006) [2023-12-26 20:23:53,621][105620] Updated weights for policy 1, policy_version 691921 (0.0007) [2023-12-26 20:23:53,650][105692] Updated weights for policy 0, policy_version 691048 (0.0008) [2023-12-26 20:23:54,247][105620] Updated weights for policy 1, policy_version 691931 (0.0007) [2023-12-26 20:23:54,297][105620] Updated weights for policy 1, policy_version 691941 (0.0009) [2023-12-26 20:23:54,344][105620] Updated weights for policy 1, policy_version 691951 (0.0008) [2023-12-26 20:23:54,446][105692] Updated weights for policy 0, policy_version 691058 (0.0009) [2023-12-26 20:23:54,490][105585] KL-divergence is very high: 150.5436 [2023-12-26 20:23:54,496][105692] Updated weights for policy 0, policy_version 691068 (0.0008) [2023-12-26 20:23:54,517][105585] KL-divergence is very high: 122.8559 [2023-12-26 20:23:54,535][105585] KL-divergence is very high: 167.3461 [2023-12-26 20:23:54,555][105692] Updated weights for policy 0, policy_version 691078 (0.0009) [2023-12-26 20:23:55,104][105620] Updated weights for policy 1, policy_version 691961 (0.0009) [2023-12-26 20:23:55,163][105620] Updated weights for policy 1, policy_version 691971 (0.0009) [2023-12-26 20:23:55,229][105620] Updated weights for policy 1, policy_version 691981 (0.0009) [2023-12-26 20:23:55,289][105620] Updated weights for policy 1, policy_version 691991 (0.0007) [2023-12-26 20:23:55,318][105692] Updated weights for policy 0, policy_version 691089 (0.0010) [2023-12-26 20:23:55,371][105692] Updated weights for policy 0, policy_version 691100 (0.0010) [2023-12-26 20:23:55,425][105692] Updated weights for policy 0, policy_version 691110 (0.0010) [2023-12-26 20:23:55,873][105620] Updated weights for policy 1, policy_version 692001 (0.0009) [2023-12-26 20:23:55,926][105620] Updated weights for policy 1, policy_version 692011 (0.0009) [2023-12-26 20:23:55,975][105620] Updated weights for policy 1, policy_version 692021 (0.0008) [2023-12-26 20:23:56,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.7, 300 sec: 19688.6). Total num frames: 354131968. Throughput: 0: 9789.5, 1: 9929.5. Samples: 354138676. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 20:23:56,063][104569] Avg episode reward: [(0, '8242.475'), (1, '9076.982')] [2023-12-26 20:23:56,064][105692] Updated weights for policy 0, policy_version 691120 (0.0009) [2023-12-26 20:23:56,115][105692] Updated weights for policy 0, policy_version 691130 (0.0009) [2023-12-26 20:23:56,169][105692] Updated weights for policy 0, policy_version 691140 (0.0009) [2023-12-26 20:23:56,658][105620] Updated weights for policy 1, policy_version 692031 (0.0009) [2023-12-26 20:23:56,703][105620] Updated weights for policy 1, policy_version 692041 (0.0009) [2023-12-26 20:23:56,756][105620] Updated weights for policy 1, policy_version 692051 (0.0010) [2023-12-26 20:23:57,006][105692] Updated weights for policy 0, policy_version 691150 (0.0009) [2023-12-26 20:23:57,053][105692] Updated weights for policy 0, policy_version 691160 (0.0009) [2023-12-26 20:23:57,103][105692] Updated weights for policy 0, policy_version 691170 (0.0006) [2023-12-26 20:23:57,445][105620] Updated weights for policy 1, policy_version 692061 (0.0009) [2023-12-26 20:23:57,506][105620] Updated weights for policy 1, policy_version 692071 (0.0008) [2023-12-26 20:23:57,560][105620] Updated weights for policy 1, policy_version 692081 (0.0009) [2023-12-26 20:23:57,841][105692] Updated weights for policy 0, policy_version 691180 (0.0007) [2023-12-26 20:23:57,904][105692] Updated weights for policy 0, policy_version 691190 (0.0007) [2023-12-26 20:23:57,966][105692] Updated weights for policy 0, policy_version 691200 (0.0009) [2023-12-26 20:23:58,355][105620] Updated weights for policy 1, policy_version 692091 (0.0009) [2023-12-26 20:23:58,416][105620] Updated weights for policy 1, policy_version 692101 (0.0008) [2023-12-26 20:23:58,476][105620] Updated weights for policy 1, policy_version 692111 (0.0008) [2023-12-26 20:23:58,705][105692] Updated weights for policy 0, policy_version 691210 (0.0008) [2023-12-26 20:23:58,766][105692] Updated weights for policy 0, policy_version 691220 (0.0009) [2023-12-26 20:23:58,830][105692] Updated weights for policy 0, policy_version 691230 (0.0008) [2023-12-26 20:23:58,899][105692] Updated weights for policy 0, policy_version 691240 (0.0008) [2023-12-26 20:23:59,331][105620] Updated weights for policy 1, policy_version 692121 (0.0008) [2023-12-26 20:23:59,401][105620] Updated weights for policy 1, policy_version 692131 (0.0009) [2023-12-26 20:23:59,459][105620] Updated weights for policy 1, policy_version 692141 (0.0009) [2023-12-26 20:23:59,520][105620] Updated weights for policy 1, policy_version 692151 (0.0009) [2023-12-26 20:23:59,660][105692] Updated weights for policy 0, policy_version 691250 (0.0009) [2023-12-26 20:23:59,714][105692] Updated weights for policy 0, policy_version 691260 (0.0009) [2023-12-26 20:23:59,774][105692] Updated weights for policy 0, policy_version 691270 (0.0009) [2023-12-26 20:24:00,262][105620] Updated weights for policy 1, policy_version 692161 (0.0008) [2023-12-26 20:24:00,308][105620] Updated weights for policy 1, policy_version 692171 (0.0007) [2023-12-26 20:24:00,367][105620] Updated weights for policy 1, policy_version 692181 (0.0009) [2023-12-26 20:24:00,580][105692] Updated weights for policy 0, policy_version 691280 (0.0006) [2023-12-26 20:24:00,635][105692] Updated weights for policy 0, policy_version 691290 (0.0006) [2023-12-26 20:24:00,694][105692] Updated weights for policy 0, policy_version 691300 (0.0006) [2023-12-26 20:24:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 354222080. Throughput: 0: 9808.9, 1: 9945.3. Samples: 354196432. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 20:24:01,062][104569] Avg episode reward: [(0, '8920.098'), (1, '9259.887')] [2023-12-26 20:24:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000691304_177004544.pth... [2023-12-26 20:24:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000690184_176717824.pth [2023-12-26 20:24:01,090][105620] Updated weights for policy 1, policy_version 692191 (0.0008) [2023-12-26 20:24:01,148][105620] Updated weights for policy 1, policy_version 692201 (0.0009) [2023-12-26 20:24:01,209][105620] Updated weights for policy 1, policy_version 692211 (0.0008) [2023-12-26 20:24:01,236][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000692216_177225728.pth... [2023-12-26 20:24:01,240][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000691032_176922624.pth [2023-12-26 20:24:01,305][105692] Updated weights for policy 0, policy_version 691310 (0.0008) [2023-12-26 20:24:01,369][105692] Updated weights for policy 0, policy_version 691320 (0.0008) [2023-12-26 20:24:01,426][105692] Updated weights for policy 0, policy_version 691330 (0.0006) [2023-12-26 20:24:01,970][105620] Updated weights for policy 1, policy_version 692221 (0.0007) [2023-12-26 20:24:02,035][105620] Updated weights for policy 1, policy_version 692231 (0.0008) [2023-12-26 20:24:02,098][105620] Updated weights for policy 1, policy_version 692241 (0.0008) [2023-12-26 20:24:02,173][105692] Updated weights for policy 0, policy_version 691340 (0.0009) [2023-12-26 20:24:02,220][105692] Updated weights for policy 0, policy_version 691350 (0.0009) [2023-12-26 20:24:02,271][105692] Updated weights for policy 0, policy_version 691360 (0.0009) [2023-12-26 20:24:02,798][105620] Updated weights for policy 1, policy_version 692251 (0.0009) [2023-12-26 20:24:02,852][105620] Updated weights for policy 1, policy_version 692261 (0.0010) [2023-12-26 20:24:02,896][105620] Updated weights for policy 1, policy_version 692271 (0.0010) [2023-12-26 20:24:02,992][105692] Updated weights for policy 0, policy_version 691370 (0.0009) [2023-12-26 20:24:03,037][105692] Updated weights for policy 0, policy_version 691380 (0.0008) [2023-12-26 20:24:03,088][105692] Updated weights for policy 0, policy_version 691390 (0.0008) [2023-12-26 20:24:03,134][105692] Updated weights for policy 0, policy_version 691400 (0.0009) [2023-12-26 20:24:03,602][105620] Updated weights for policy 1, policy_version 692281 (0.0010) [2023-12-26 20:24:03,664][105620] Updated weights for policy 1, policy_version 692291 (0.0008) [2023-12-26 20:24:03,717][105620] Updated weights for policy 1, policy_version 692302 (0.0010) [2023-12-26 20:24:03,766][105620] Updated weights for policy 1, policy_version 692312 (0.0005) [2023-12-26 20:24:03,797][105692] Updated weights for policy 0, policy_version 691410 (0.0011) [2023-12-26 20:24:03,852][105692] Updated weights for policy 0, policy_version 691420 (0.0010) [2023-12-26 20:24:03,918][105692] Updated weights for policy 0, policy_version 691430 (0.0011) [2023-12-26 20:24:04,450][105620] Updated weights for policy 1, policy_version 692322 (0.0009) [2023-12-26 20:24:04,511][105620] Updated weights for policy 1, policy_version 692332 (0.0011) [2023-12-26 20:24:04,559][105692] Updated weights for policy 0, policy_version 691440 (0.0007) [2023-12-26 20:24:04,565][105620] Updated weights for policy 1, policy_version 692342 (0.0011) [2023-12-26 20:24:04,624][105692] Updated weights for policy 0, policy_version 691450 (0.0008) [2023-12-26 20:24:04,685][105692] Updated weights for policy 0, policy_version 691460 (0.0008) [2023-12-26 20:24:05,331][105620] Updated weights for policy 1, policy_version 692352 (0.0010) [2023-12-26 20:24:05,379][105620] Updated weights for policy 1, policy_version 692362 (0.0010) [2023-12-26 20:24:05,405][105692] Updated weights for policy 0, policy_version 691470 (0.0006) [2023-12-26 20:24:05,427][105620] Updated weights for policy 1, policy_version 692372 (0.0010) [2023-12-26 20:24:05,450][105692] Updated weights for policy 0, policy_version 691480 (0.0005) [2023-12-26 20:24:05,501][105692] Updated weights for policy 0, policy_version 691490 (0.0008) [2023-12-26 20:24:06,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 354320384. Throughput: 0: 9796.5, 1: 9864.4. Samples: 354312660. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 20:24:06,062][104569] Avg episode reward: [(0, '8838.384'), (1, '9258.270')] [2023-12-26 20:24:06,201][105620] Updated weights for policy 1, policy_version 692382 (0.0010) [2023-12-26 20:24:06,254][105620] Updated weights for policy 1, policy_version 692392 (0.0010) [2023-12-26 20:24:06,275][105692] Updated weights for policy 0, policy_version 691500 (0.0007) [2023-12-26 20:24:06,315][105620] Updated weights for policy 1, policy_version 692402 (0.0009) [2023-12-26 20:24:06,330][105692] Updated weights for policy 0, policy_version 691510 (0.0006) [2023-12-26 20:24:06,391][105692] Updated weights for policy 0, policy_version 691520 (0.0007) [2023-12-26 20:24:06,978][105692] Updated weights for policy 0, policy_version 691530 (0.0006) [2023-12-26 20:24:07,025][105692] Updated weights for policy 0, policy_version 691540 (0.0009) [2023-12-26 20:24:07,075][105692] Updated weights for policy 0, policy_version 691550 (0.0008) [2023-12-26 20:24:07,129][105692] Updated weights for policy 0, policy_version 691560 (0.0009) [2023-12-26 20:24:07,156][105620] Updated weights for policy 1, policy_version 692412 (0.0009) [2023-12-26 20:24:07,211][105620] Updated weights for policy 1, policy_version 692422 (0.0009) [2023-12-26 20:24:07,259][105620] Updated weights for policy 1, policy_version 692432 (0.0009) [2023-12-26 20:24:07,820][105692] Updated weights for policy 0, policy_version 691570 (0.0009) [2023-12-26 20:24:07,878][105692] Updated weights for policy 0, policy_version 691580 (0.0009) [2023-12-26 20:24:07,932][105692] Updated weights for policy 0, policy_version 691590 (0.0008) [2023-12-26 20:24:08,076][105620] Updated weights for policy 1, policy_version 692442 (0.0009) [2023-12-26 20:24:08,128][105620] Updated weights for policy 1, policy_version 692452 (0.0009) [2023-12-26 20:24:08,185][105620] Updated weights for policy 1, policy_version 692462 (0.0010) [2023-12-26 20:24:08,237][105620] Updated weights for policy 1, policy_version 692472 (0.0009) [2023-12-26 20:24:08,687][105692] Updated weights for policy 0, policy_version 691600 (0.0009) [2023-12-26 20:24:08,748][105692] Updated weights for policy 0, policy_version 691610 (0.0008) [2023-12-26 20:24:08,810][105692] Updated weights for policy 0, policy_version 691620 (0.0009) [2023-12-26 20:24:08,957][105620] Updated weights for policy 1, policy_version 692482 (0.0008) [2023-12-26 20:24:09,010][105620] Updated weights for policy 1, policy_version 692492 (0.0008) [2023-12-26 20:24:09,065][105620] Updated weights for policy 1, policy_version 692502 (0.0009) [2023-12-26 20:24:09,577][105692] Updated weights for policy 0, policy_version 691630 (0.0009) [2023-12-26 20:24:09,633][105692] Updated weights for policy 0, policy_version 691640 (0.0010) [2023-12-26 20:24:09,693][105692] Updated weights for policy 0, policy_version 691650 (0.0009) [2023-12-26 20:24:09,766][105620] Updated weights for policy 1, policy_version 692512 (0.0009) [2023-12-26 20:24:09,825][105620] Updated weights for policy 1, policy_version 692522 (0.0008) [2023-12-26 20:24:09,887][105620] Updated weights for policy 1, policy_version 692532 (0.0008) [2023-12-26 20:24:10,515][105692] Updated weights for policy 0, policy_version 691660 (0.0009) [2023-12-26 20:24:10,574][105692] Updated weights for policy 0, policy_version 691670 (0.0009) [2023-12-26 20:24:10,626][105692] Updated weights for policy 0, policy_version 691680 (0.0009) [2023-12-26 20:24:10,675][105620] Updated weights for policy 1, policy_version 692542 (0.0007) [2023-12-26 20:24:10,740][105620] Updated weights for policy 1, policy_version 692552 (0.0009) [2023-12-26 20:24:10,794][105620] Updated weights for policy 1, policy_version 692563 (0.0010) [2023-12-26 20:24:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 354418688. Throughput: 0: 9834.7, 1: 9753.4. Samples: 354425176. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 20:24:11,063][104569] Avg episode reward: [(0, '8864.437'), (1, '9258.079')] [2023-12-26 20:24:11,346][105692] Updated weights for policy 0, policy_version 691690 (0.0009) [2023-12-26 20:24:11,404][105692] Updated weights for policy 0, policy_version 691700 (0.0009) [2023-12-26 20:24:11,465][105692] Updated weights for policy 0, policy_version 691710 (0.0009) [2023-12-26 20:24:11,523][105692] Updated weights for policy 0, policy_version 691720 (0.0008) [2023-12-26 20:24:11,577][105620] Updated weights for policy 1, policy_version 692574 (0.0009) [2023-12-26 20:24:11,646][105620] Updated weights for policy 1, policy_version 692584 (0.0010) [2023-12-26 20:24:11,722][105620] Updated weights for policy 1, policy_version 692594 (0.0009) [2023-12-26 20:24:12,256][105692] Updated weights for policy 0, policy_version 691730 (0.0006) [2023-12-26 20:24:12,316][105692] Updated weights for policy 0, policy_version 691740 (0.0009) [2023-12-26 20:24:12,385][105692] Updated weights for policy 0, policy_version 691750 (0.0008) [2023-12-26 20:24:12,532][105620] Updated weights for policy 1, policy_version 692604 (0.0009) [2023-12-26 20:24:12,594][105620] Updated weights for policy 1, policy_version 692614 (0.0009) [2023-12-26 20:24:12,652][105620] Updated weights for policy 1, policy_version 692624 (0.0009) [2023-12-26 20:24:13,079][105692] Updated weights for policy 0, policy_version 691760 (0.0009) [2023-12-26 20:24:13,143][105692] Updated weights for policy 0, policy_version 691770 (0.0009) [2023-12-26 20:24:13,198][105692] Updated weights for policy 0, policy_version 691780 (0.0009) [2023-12-26 20:24:13,429][105620] Updated weights for policy 1, policy_version 692634 (0.0010) [2023-12-26 20:24:13,490][105620] Updated weights for policy 1, policy_version 692644 (0.0009) [2023-12-26 20:24:13,543][105620] Updated weights for policy 1, policy_version 692654 (0.0009) [2023-12-26 20:24:13,608][105620] Updated weights for policy 1, policy_version 692664 (0.0009) [2023-12-26 20:24:13,909][105692] Updated weights for policy 0, policy_version 691790 (0.0007) [2023-12-26 20:24:13,976][105692] Updated weights for policy 0, policy_version 691800 (0.0005) [2023-12-26 20:24:14,045][105692] Updated weights for policy 0, policy_version 691810 (0.0005) [2023-12-26 20:24:14,468][105620] Updated weights for policy 1, policy_version 692674 (0.0010) [2023-12-26 20:24:14,532][105620] Updated weights for policy 1, policy_version 692684 (0.0008) [2023-12-26 20:24:14,539][105586] KL-divergence is very high: 130.8703 [2023-12-26 20:24:14,541][105692] Updated weights for policy 0, policy_version 691820 (0.0006) [2023-12-26 20:24:14,584][105586] KL-divergence is very high: 109.9224 [2023-12-26 20:24:14,591][105620] Updated weights for policy 1, policy_version 692694 (0.0009) [2023-12-26 20:24:14,593][105692] Updated weights for policy 0, policy_version 691830 (0.0006) [2023-12-26 20:24:14,643][105692] Updated weights for policy 0, policy_version 691840 (0.0008) [2023-12-26 20:24:15,378][105620] Updated weights for policy 1, policy_version 692704 (0.0009) [2023-12-26 20:24:15,421][105692] Updated weights for policy 0, policy_version 691850 (0.0008) [2023-12-26 20:24:15,438][105620] Updated weights for policy 1, policy_version 692714 (0.0009) [2023-12-26 20:24:15,477][105692] Updated weights for policy 0, policy_version 691860 (0.0009) [2023-12-26 20:24:15,501][105620] Updated weights for policy 1, policy_version 692724 (0.0006) [2023-12-26 20:24:15,541][105692] Updated weights for policy 0, policy_version 691870 (0.0008) [2023-12-26 20:24:15,603][105692] Updated weights for policy 0, policy_version 691880 (0.0008) [2023-12-26 20:24:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 354508800. Throughput: 0: 9738.1, 1: 9659.7. Samples: 354480584. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 20:24:16,062][104569] Avg episode reward: [(0, '8869.295'), (1, '9074.437')] [2023-12-26 20:24:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000692728_177356800.pth... [2023-12-26 20:24:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000691880_177152000.pth... [2023-12-26 20:24:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000691640_177078272.pth [2023-12-26 20:24:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000690760_176865280.pth [2023-12-26 20:24:16,251][105620] Updated weights for policy 1, policy_version 692734 (0.0006) [2023-12-26 20:24:16,312][105620] Updated weights for policy 1, policy_version 692744 (0.0005) [2023-12-26 20:24:16,335][105692] Updated weights for policy 0, policy_version 691890 (0.0008) [2023-12-26 20:24:16,365][105620] Updated weights for policy 1, policy_version 692754 (0.0007) [2023-12-26 20:24:16,387][105692] Updated weights for policy 0, policy_version 691900 (0.0006) [2023-12-26 20:24:16,440][105692] Updated weights for policy 0, policy_version 691910 (0.0008) [2023-12-26 20:24:17,088][105620] Updated weights for policy 1, policy_version 692764 (0.0007) [2023-12-26 20:24:17,138][105620] Updated weights for policy 1, policy_version 692774 (0.0009) [2023-12-26 20:24:17,188][105692] Updated weights for policy 0, policy_version 691920 (0.0007) [2023-12-26 20:24:17,203][105620] Updated weights for policy 1, policy_version 692784 (0.0007) [2023-12-26 20:24:17,249][105692] Updated weights for policy 0, policy_version 691930 (0.0007) [2023-12-26 20:24:17,311][105692] Updated weights for policy 0, policy_version 691940 (0.0005) [2023-12-26 20:24:17,834][105692] Updated weights for policy 0, policy_version 691950 (0.0005) [2023-12-26 20:24:17,892][105692] Updated weights for policy 0, policy_version 691960 (0.0007) [2023-12-26 20:24:17,946][105692] Updated weights for policy 0, policy_version 691971 (0.0009) [2023-12-26 20:24:17,963][105620] Updated weights for policy 1, policy_version 692794 (0.0008) [2023-12-26 20:24:18,016][105620] Updated weights for policy 1, policy_version 692804 (0.0008) [2023-12-26 20:24:18,069][105620] Updated weights for policy 1, policy_version 692814 (0.0006) [2023-12-26 20:24:18,123][105620] Updated weights for policy 1, policy_version 692824 (0.0009) [2023-12-26 20:24:18,646][105692] Updated weights for policy 0, policy_version 691981 (0.0007) [2023-12-26 20:24:18,712][105692] Updated weights for policy 0, policy_version 691991 (0.0008) [2023-12-26 20:24:18,774][105692] Updated weights for policy 0, policy_version 692001 (0.0009) [2023-12-26 20:24:18,911][105620] Updated weights for policy 1, policy_version 692834 (0.0005) [2023-12-26 20:24:18,977][105620] Updated weights for policy 1, policy_version 692844 (0.0008) [2023-12-26 20:24:19,042][105620] Updated weights for policy 1, policy_version 692854 (0.0009) [2023-12-26 20:24:19,506][105692] Updated weights for policy 0, policy_version 692011 (0.0008) [2023-12-26 20:24:19,565][105692] Updated weights for policy 0, policy_version 692021 (0.0005) [2023-12-26 20:24:19,567][105585] KL-divergence is very high: 109.9753 [2023-12-26 20:24:19,622][105585] KL-divergence is very high: 211.5482 [2023-12-26 20:24:19,634][105692] Updated weights for policy 0, policy_version 692031 (0.0006) [2023-12-26 20:24:19,671][105585] KL-divergence is very high: 232.7599 [2023-12-26 20:24:19,724][105620] Updated weights for policy 1, policy_version 692864 (0.0009) [2023-12-26 20:24:19,787][105620] Updated weights for policy 1, policy_version 692874 (0.0010) [2023-12-26 20:24:19,859][105620] Updated weights for policy 1, policy_version 692884 (0.0009) [2023-12-26 20:24:20,333][105692] Updated weights for policy 0, policy_version 692041 (0.0005) [2023-12-26 20:24:20,395][105692] Updated weights for policy 0, policy_version 692051 (0.0009) [2023-12-26 20:24:20,463][105692] Updated weights for policy 0, policy_version 692061 (0.0008) [2023-12-26 20:24:20,513][105620] Updated weights for policy 1, policy_version 692894 (0.0009) [2023-12-26 20:24:20,519][105692] Updated weights for policy 0, policy_version 692071 (0.0010) [2023-12-26 20:24:20,573][105620] Updated weights for policy 1, policy_version 692904 (0.0008) [2023-12-26 20:24:20,633][105620] Updated weights for policy 1, policy_version 692914 (0.0008) [2023-12-26 20:24:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 354607104. Throughput: 0: 9775.5, 1: 9555.6. Samples: 354597348. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 20:24:21,063][104569] Avg episode reward: [(0, '8782.883'), (1, '9168.235')] [2023-12-26 20:24:21,317][105692] Updated weights for policy 0, policy_version 692081 (0.0010) [2023-12-26 20:24:21,388][105692] Updated weights for policy 0, policy_version 692091 (0.0010) [2023-12-26 20:24:21,457][105692] Updated weights for policy 0, policy_version 692101 (0.0010) [2023-12-26 20:24:21,467][105620] Updated weights for policy 1, policy_version 692924 (0.0007) [2023-12-26 20:24:21,530][105620] Updated weights for policy 1, policy_version 692934 (0.0008) [2023-12-26 20:24:21,591][105620] Updated weights for policy 1, policy_version 692944 (0.0008) [2023-12-26 20:24:22,168][105692] Updated weights for policy 0, policy_version 692111 (0.0007) [2023-12-26 20:24:22,228][105692] Updated weights for policy 0, policy_version 692121 (0.0005) [2023-12-26 20:24:22,292][105692] Updated weights for policy 0, policy_version 692131 (0.0007) [2023-12-26 20:24:22,395][105620] Updated weights for policy 1, policy_version 692954 (0.0008) [2023-12-26 20:24:22,447][105620] Updated weights for policy 1, policy_version 692964 (0.0008) [2023-12-26 20:24:22,504][105620] Updated weights for policy 1, policy_version 692974 (0.0008) [2023-12-26 20:24:22,560][105620] Updated weights for policy 1, policy_version 692984 (0.0008) [2023-12-26 20:24:22,900][105692] Updated weights for policy 0, policy_version 692141 (0.0008) [2023-12-26 20:24:22,956][105692] Updated weights for policy 0, policy_version 692151 (0.0009) [2023-12-26 20:24:23,008][105692] Updated weights for policy 0, policy_version 692161 (0.0010) [2023-12-26 20:24:23,326][105620] Updated weights for policy 1, policy_version 692994 (0.0008) [2023-12-26 20:24:23,381][105620] Updated weights for policy 1, policy_version 693004 (0.0006) [2023-12-26 20:24:23,448][105620] Updated weights for policy 1, policy_version 693014 (0.0008) [2023-12-26 20:24:23,694][105692] Updated weights for policy 0, policy_version 692171 (0.0009) [2023-12-26 20:24:23,744][105692] Updated weights for policy 0, policy_version 692181 (0.0006) [2023-12-26 20:24:23,806][105692] Updated weights for policy 0, policy_version 692191 (0.0011) [2023-12-26 20:24:24,209][105620] Updated weights for policy 1, policy_version 693024 (0.0008) [2023-12-26 20:24:24,275][105620] Updated weights for policy 1, policy_version 693034 (0.0007) [2023-12-26 20:24:24,329][105620] Updated weights for policy 1, policy_version 693044 (0.0007) [2023-12-26 20:24:24,514][105692] Updated weights for policy 0, policy_version 692201 (0.0011) [2023-12-26 20:24:24,562][105692] Updated weights for policy 0, policy_version 692211 (0.0011) [2023-12-26 20:24:24,620][105692] Updated weights for policy 0, policy_version 692221 (0.0010) [2023-12-26 20:24:24,676][105692] Updated weights for policy 0, policy_version 692231 (0.0010) [2023-12-26 20:24:25,080][105620] Updated weights for policy 1, policy_version 693054 (0.0008) [2023-12-26 20:24:25,127][105620] Updated weights for policy 1, policy_version 693064 (0.0008) [2023-12-26 20:24:25,175][105620] Updated weights for policy 1, policy_version 693074 (0.0008) [2023-12-26 20:24:25,446][105692] Updated weights for policy 0, policy_version 692241 (0.0010) [2023-12-26 20:24:25,507][105692] Updated weights for policy 0, policy_version 692251 (0.0010) [2023-12-26 20:24:25,564][105692] Updated weights for policy 0, policy_version 692261 (0.0010) [2023-12-26 20:24:25,935][105620] Updated weights for policy 1, policy_version 693084 (0.0007) [2023-12-26 20:24:25,995][105620] Updated weights for policy 1, policy_version 693094 (0.0008) [2023-12-26 20:24:26,053][105620] Updated weights for policy 1, policy_version 693104 (0.0009) [2023-12-26 20:24:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 354697216. Throughput: 0: 9784.2, 1: 9451.5. Samples: 354711452. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 20:24:26,062][104569] Avg episode reward: [(0, '7911.589'), (1, '9352.729')] [2023-12-26 20:24:26,295][105692] Updated weights for policy 0, policy_version 692271 (0.0009) [2023-12-26 20:24:26,342][105692] Updated weights for policy 0, policy_version 692281 (0.0008) [2023-12-26 20:24:26,396][105692] Updated weights for policy 0, policy_version 692291 (0.0009) [2023-12-26 20:24:26,834][105620] Updated weights for policy 1, policy_version 693114 (0.0009) [2023-12-26 20:24:26,884][105620] Updated weights for policy 1, policy_version 693124 (0.0008) [2023-12-26 20:24:26,942][105620] Updated weights for policy 1, policy_version 693134 (0.0008) [2023-12-26 20:24:26,998][105620] Updated weights for policy 1, policy_version 693144 (0.0008) [2023-12-26 20:24:27,053][105692] Updated weights for policy 0, policy_version 692301 (0.0007) [2023-12-26 20:24:27,107][105692] Updated weights for policy 0, policy_version 692311 (0.0005) [2023-12-26 20:24:27,158][105692] Updated weights for policy 0, policy_version 692321 (0.0006) [2023-12-26 20:24:27,737][105692] Updated weights for policy 0, policy_version 692331 (0.0008) [2023-12-26 20:24:27,783][105692] Updated weights for policy 0, policy_version 692341 (0.0008) [2023-12-26 20:24:27,829][105692] Updated weights for policy 0, policy_version 692351 (0.0007) [2023-12-26 20:24:27,833][105620] Updated weights for policy 1, policy_version 693154 (0.0008) [2023-12-26 20:24:27,892][105620] Updated weights for policy 1, policy_version 693164 (0.0008) [2023-12-26 20:24:27,944][105620] Updated weights for policy 1, policy_version 693174 (0.0008) [2023-12-26 20:24:28,610][105692] Updated weights for policy 0, policy_version 692361 (0.0008) [2023-12-26 20:24:28,670][105692] Updated weights for policy 0, policy_version 692371 (0.0008) [2023-12-26 20:24:28,702][105620] Updated weights for policy 1, policy_version 693184 (0.0007) [2023-12-26 20:24:28,726][105692] Updated weights for policy 0, policy_version 692381 (0.0008) [2023-12-26 20:24:28,763][105620] Updated weights for policy 1, policy_version 693194 (0.0006) [2023-12-26 20:24:28,782][105692] Updated weights for policy 0, policy_version 692391 (0.0008) [2023-12-26 20:24:28,817][105620] Updated weights for policy 1, policy_version 693204 (0.0007) [2023-12-26 20:24:29,462][105620] Updated weights for policy 1, policy_version 693214 (0.0006) [2023-12-26 20:24:29,529][105620] Updated weights for policy 1, policy_version 693224 (0.0006) [2023-12-26 20:24:29,581][105620] Updated weights for policy 1, policy_version 693234 (0.0006) [2023-12-26 20:24:29,617][105692] Updated weights for policy 0, policy_version 692401 (0.0008) [2023-12-26 20:24:29,670][105692] Updated weights for policy 0, policy_version 692411 (0.0009) [2023-12-26 20:24:29,724][105692] Updated weights for policy 0, policy_version 692421 (0.0010) [2023-12-26 20:24:30,194][105620] Updated weights for policy 1, policy_version 693244 (0.0007) [2023-12-26 20:24:30,245][105620] Updated weights for policy 1, policy_version 693254 (0.0008) [2023-12-26 20:24:30,308][105620] Updated weights for policy 1, policy_version 693264 (0.0005) [2023-12-26 20:24:30,548][105692] Updated weights for policy 0, policy_version 692431 (0.0009) [2023-12-26 20:24:30,603][105692] Updated weights for policy 0, policy_version 692441 (0.0009) [2023-12-26 20:24:30,661][105692] Updated weights for policy 0, policy_version 692451 (0.0009) [2023-12-26 20:24:31,010][105620] Updated weights for policy 1, policy_version 693274 (0.0006) [2023-12-26 20:24:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 354795520. Throughput: 0: 9839.1, 1: 9395.7. Samples: 354769084. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 20:24:31,063][104569] Avg episode reward: [(0, '8005.516'), (1, '9352.655')] [2023-12-26 20:24:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000692456_177299456.pth... [2023-12-26 20:24:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000691304_177004544.pth [2023-12-26 20:24:31,077][105620] Updated weights for policy 1, policy_version 693284 (0.0009) [2023-12-26 20:24:31,127][105620] Updated weights for policy 1, policy_version 693294 (0.0009) [2023-12-26 20:24:31,183][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000693304_177504256.pth... [2023-12-26 20:24:31,186][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000692216_177225728.pth [2023-12-26 20:24:31,186][105620] Updated weights for policy 1, policy_version 693304 (0.0009) [2023-12-26 20:24:31,462][105692] Updated weights for policy 0, policy_version 692461 (0.0009) [2023-12-26 20:24:31,513][105692] Updated weights for policy 0, policy_version 692471 (0.0006) [2023-12-26 20:24:31,571][105692] Updated weights for policy 0, policy_version 692481 (0.0005) [2023-12-26 20:24:31,983][105620] Updated weights for policy 1, policy_version 693314 (0.0008) [2023-12-26 20:24:32,044][105620] Updated weights for policy 1, policy_version 693324 (0.0010) [2023-12-26 20:24:32,099][105620] Updated weights for policy 1, policy_version 693334 (0.0006) [2023-12-26 20:24:32,208][105692] Updated weights for policy 0, policy_version 692491 (0.0005) [2023-12-26 20:24:32,263][105692] Updated weights for policy 0, policy_version 692501 (0.0005) [2023-12-26 20:24:32,314][105692] Updated weights for policy 0, policy_version 692511 (0.0005) [2023-12-26 20:24:32,875][105692] Updated weights for policy 0, policy_version 692521 (0.0008) [2023-12-26 20:24:32,906][105620] Updated weights for policy 1, policy_version 693344 (0.0008) [2023-12-26 20:24:32,934][105692] Updated weights for policy 0, policy_version 692531 (0.0005) [2023-12-26 20:24:32,955][105620] Updated weights for policy 1, policy_version 693354 (0.0009) [2023-12-26 20:24:32,985][105692] Updated weights for policy 0, policy_version 692541 (0.0007) [2023-12-26 20:24:33,009][105620] Updated weights for policy 1, policy_version 693364 (0.0007) [2023-12-26 20:24:33,032][105692] Updated weights for policy 0, policy_version 692551 (0.0008) [2023-12-26 20:24:33,715][105620] Updated weights for policy 1, policy_version 693374 (0.0006) [2023-12-26 20:24:33,760][105620] Updated weights for policy 1, policy_version 693384 (0.0005) [2023-12-26 20:24:33,804][105692] Updated weights for policy 0, policy_version 692561 (0.0009) [2023-12-26 20:24:33,814][105620] Updated weights for policy 1, policy_version 693394 (0.0006) [2023-12-26 20:24:33,862][105692] Updated weights for policy 0, policy_version 692571 (0.0007) [2023-12-26 20:24:33,915][105692] Updated weights for policy 0, policy_version 692581 (0.0009) [2023-12-26 20:24:34,522][105620] Updated weights for policy 1, policy_version 693404 (0.0007) [2023-12-26 20:24:34,580][105620] Updated weights for policy 1, policy_version 693414 (0.0009) [2023-12-26 20:24:34,639][105620] Updated weights for policy 1, policy_version 693424 (0.0009) [2023-12-26 20:24:34,704][105692] Updated weights for policy 0, policy_version 692591 (0.0007) [2023-12-26 20:24:34,767][105692] Updated weights for policy 0, policy_version 692601 (0.0006) [2023-12-26 20:24:34,831][105692] Updated weights for policy 0, policy_version 692611 (0.0009) [2023-12-26 20:24:35,423][105620] Updated weights for policy 1, policy_version 693434 (0.0009) [2023-12-26 20:24:35,470][105620] Updated weights for policy 1, policy_version 693444 (0.0007) [2023-12-26 20:24:35,498][105692] Updated weights for policy 0, policy_version 692621 (0.0007) [2023-12-26 20:24:35,519][105620] Updated weights for policy 1, policy_version 693454 (0.0008) [2023-12-26 20:24:35,555][105692] Updated weights for policy 0, policy_version 692631 (0.0006) [2023-12-26 20:24:35,578][105620] Updated weights for policy 1, policy_version 693464 (0.0007) [2023-12-26 20:24:35,601][105692] Updated weights for policy 0, policy_version 692641 (0.0007) [2023-12-26 20:24:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 354893824. Throughput: 0: 9699.8, 1: 9440.1. Samples: 354883980. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 20:24:36,062][104569] Avg episode reward: [(0, '8870.804'), (1, '9352.546')] [2023-12-26 20:24:36,256][105620] Updated weights for policy 1, policy_version 693474 (0.0011) [2023-12-26 20:24:36,326][105620] Updated weights for policy 1, policy_version 693484 (0.0011) [2023-12-26 20:24:36,379][105620] Updated weights for policy 1, policy_version 693494 (0.0010) [2023-12-26 20:24:36,385][105692] Updated weights for policy 0, policy_version 692652 (0.0009) [2023-12-26 20:24:36,446][105692] Updated weights for policy 0, policy_version 692662 (0.0008) [2023-12-26 20:24:36,507][105692] Updated weights for policy 0, policy_version 692672 (0.0008) [2023-12-26 20:24:37,145][105620] Updated weights for policy 1, policy_version 693504 (0.0010) [2023-12-26 20:24:37,201][105620] Updated weights for policy 1, policy_version 693514 (0.0011) [2023-12-26 20:24:37,260][105620] Updated weights for policy 1, policy_version 693524 (0.0010) [2023-12-26 20:24:37,300][105692] Updated weights for policy 0, policy_version 692682 (0.0008) [2023-12-26 20:24:37,354][105692] Updated weights for policy 0, policy_version 692692 (0.0005) [2023-12-26 20:24:37,406][105692] Updated weights for policy 0, policy_version 692702 (0.0005) [2023-12-26 20:24:37,459][105692] Updated weights for policy 0, policy_version 692712 (0.0007) [2023-12-26 20:24:38,027][105620] Updated weights for policy 1, policy_version 693534 (0.0010) [2023-12-26 20:24:38,083][105620] Updated weights for policy 1, policy_version 693544 (0.0008) [2023-12-26 20:24:38,143][105620] Updated weights for policy 1, policy_version 693554 (0.0006) [2023-12-26 20:24:38,153][105692] Updated weights for policy 0, policy_version 692722 (0.0010) [2023-12-26 20:24:38,197][105692] Updated weights for policy 0, policy_version 692732 (0.0010) [2023-12-26 20:24:38,245][105692] Updated weights for policy 0, policy_version 692742 (0.0010) [2023-12-26 20:24:38,899][105692] Updated weights for policy 0, policy_version 692752 (0.0006) [2023-12-26 20:24:38,928][105620] Updated weights for policy 1, policy_version 693564 (0.0006) [2023-12-26 20:24:38,961][105692] Updated weights for policy 0, policy_version 692762 (0.0008) [2023-12-26 20:24:38,987][105620] Updated weights for policy 1, policy_version 693574 (0.0010) [2023-12-26 20:24:39,013][105692] Updated weights for policy 0, policy_version 692772 (0.0010) [2023-12-26 20:24:39,039][105620] Updated weights for policy 1, policy_version 693584 (0.0005) [2023-12-26 20:24:39,732][105692] Updated weights for policy 0, policy_version 692782 (0.0011) [2023-12-26 20:24:39,798][105692] Updated weights for policy 0, policy_version 692792 (0.0010) [2023-12-26 20:24:39,828][105620] Updated weights for policy 1, policy_version 693594 (0.0008) [2023-12-26 20:24:39,863][105692] Updated weights for policy 0, policy_version 692802 (0.0010) [2023-12-26 20:24:39,892][105620] Updated weights for policy 1, policy_version 693604 (0.0006) [2023-12-26 20:24:39,954][105620] Updated weights for policy 1, policy_version 693614 (0.0008) [2023-12-26 20:24:40,012][105620] Updated weights for policy 1, policy_version 693624 (0.0008) [2023-12-26 20:24:40,610][105692] Updated weights for policy 0, policy_version 692812 (0.0010) [2023-12-26 20:24:40,655][105692] Updated weights for policy 0, policy_version 692822 (0.0010) [2023-12-26 20:24:40,703][105692] Updated weights for policy 0, policy_version 692832 (0.0010) [2023-12-26 20:24:40,779][105620] Updated weights for policy 1, policy_version 693634 (0.0009) [2023-12-26 20:24:40,833][105620] Updated weights for policy 1, policy_version 693644 (0.0009) [2023-12-26 20:24:40,887][105620] Updated weights for policy 1, policy_version 693654 (0.0010) [2023-12-26 20:24:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 354992128. Throughput: 0: 9765.9, 1: 9305.5. Samples: 354996888. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 20:24:41,063][104569] Avg episode reward: [(0, '9146.590'), (1, '8714.200')] [2023-12-26 20:24:41,420][105692] Updated weights for policy 0, policy_version 692842 (0.0011) [2023-12-26 20:24:41,484][105692] Updated weights for policy 0, policy_version 692852 (0.0011) [2023-12-26 20:24:41,546][105692] Updated weights for policy 0, policy_version 692862 (0.0006) [2023-12-26 20:24:41,618][105692] Updated weights for policy 0, policy_version 692872 (0.0010) [2023-12-26 20:24:41,746][105620] Updated weights for policy 1, policy_version 693664 (0.0009) [2023-12-26 20:24:41,810][105620] Updated weights for policy 1, policy_version 693674 (0.0008) [2023-12-26 20:24:41,878][105620] Updated weights for policy 1, policy_version 693684 (0.0007) [2023-12-26 20:24:42,360][105692] Updated weights for policy 0, policy_version 692882 (0.0009) [2023-12-26 20:24:42,432][105692] Updated weights for policy 0, policy_version 692892 (0.0008) [2023-12-26 20:24:42,497][105692] Updated weights for policy 0, policy_version 692902 (0.0008) [2023-12-26 20:24:42,610][105620] Updated weights for policy 1, policy_version 693694 (0.0006) [2023-12-26 20:24:42,671][105620] Updated weights for policy 1, policy_version 693704 (0.0006) [2023-12-26 20:24:42,731][105620] Updated weights for policy 1, policy_version 693714 (0.0007) [2023-12-26 20:24:43,243][105692] Updated weights for policy 0, policy_version 692912 (0.0010) [2023-12-26 20:24:43,298][105692] Updated weights for policy 0, policy_version 692922 (0.0010) [2023-12-26 20:24:43,345][105692] Updated weights for policy 0, policy_version 692932 (0.0010) [2023-12-26 20:24:43,447][105620] Updated weights for policy 1, policy_version 693724 (0.0008) [2023-12-26 20:24:43,499][105620] Updated weights for policy 1, policy_version 693734 (0.0008) [2023-12-26 20:24:43,551][105620] Updated weights for policy 1, policy_version 693744 (0.0008) [2023-12-26 20:24:44,018][105692] Updated weights for policy 0, policy_version 692942 (0.0010) [2023-12-26 20:24:44,066][105692] Updated weights for policy 0, policy_version 692952 (0.0010) [2023-12-26 20:24:44,120][105692] Updated weights for policy 0, policy_version 692962 (0.0010) [2023-12-26 20:24:44,364][105620] Updated weights for policy 1, policy_version 693754 (0.0008) [2023-12-26 20:24:44,416][105620] Updated weights for policy 1, policy_version 693764 (0.0008) [2023-12-26 20:24:44,460][105620] Updated weights for policy 1, policy_version 693774 (0.0008) [2023-12-26 20:24:44,512][105620] Updated weights for policy 1, policy_version 693784 (0.0008) [2023-12-26 20:24:44,897][105692] Updated weights for policy 0, policy_version 692972 (0.0010) [2023-12-26 20:24:44,953][105692] Updated weights for policy 0, policy_version 692982 (0.0011) [2023-12-26 20:24:45,013][105692] Updated weights for policy 0, policy_version 692992 (0.0011) [2023-12-26 20:24:45,310][105620] Updated weights for policy 1, policy_version 693794 (0.0009) [2023-12-26 20:24:45,369][105620] Updated weights for policy 1, policy_version 693804 (0.0008) [2023-12-26 20:24:45,431][105620] Updated weights for policy 1, policy_version 693814 (0.0008) [2023-12-26 20:24:45,768][105692] Updated weights for policy 0, policy_version 693002 (0.0011) [2023-12-26 20:24:45,834][105692] Updated weights for policy 0, policy_version 693012 (0.0011) [2023-12-26 20:24:45,887][105692] Updated weights for policy 0, policy_version 693022 (0.0010) [2023-12-26 20:24:45,942][105692] Updated weights for policy 0, policy_version 693032 (0.0007) [2023-12-26 20:24:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 355082240. Throughput: 0: 9744.3, 1: 9277.2. Samples: 355052396. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 20:24:46,062][104569] Avg episode reward: [(0, '9170.427'), (1, '8530.791')] [2023-12-26 20:24:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000693032_177446912.pth... [2023-12-26 20:24:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000693816_177635328.pth... [2023-12-26 20:24:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000691880_177152000.pth [2023-12-26 20:24:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000692728_177356800.pth [2023-12-26 20:24:46,203][105620] Updated weights for policy 1, policy_version 693824 (0.0008) [2023-12-26 20:24:46,269][105620] Updated weights for policy 1, policy_version 693834 (0.0006) [2023-12-26 20:24:46,336][105620] Updated weights for policy 1, policy_version 693844 (0.0005) [2023-12-26 20:24:46,650][105692] Updated weights for policy 0, policy_version 693042 (0.0007) [2023-12-26 20:24:46,707][105692] Updated weights for policy 0, policy_version 693052 (0.0009) [2023-12-26 20:24:46,766][105692] Updated weights for policy 0, policy_version 693062 (0.0009) [2023-12-26 20:24:46,877][105620] Updated weights for policy 1, policy_version 693854 (0.0006) [2023-12-26 20:24:46,926][105620] Updated weights for policy 1, policy_version 693864 (0.0009) [2023-12-26 20:24:46,986][105620] Updated weights for policy 1, policy_version 693874 (0.0008) [2023-12-26 20:24:47,417][105692] Updated weights for policy 0, policy_version 693072 (0.0006) [2023-12-26 20:24:47,463][105692] Updated weights for policy 0, policy_version 693082 (0.0005) [2023-12-26 20:24:47,517][105692] Updated weights for policy 0, policy_version 693092 (0.0006) [2023-12-26 20:24:47,538][105620] Updated weights for policy 1, policy_version 693884 (0.0005) [2023-12-26 20:24:47,587][105620] Updated weights for policy 1, policy_version 693894 (0.0005) [2023-12-26 20:24:47,638][105620] Updated weights for policy 1, policy_version 693904 (0.0005) [2023-12-26 20:24:48,225][105692] Updated weights for policy 0, policy_version 693102 (0.0005) [2023-12-26 20:24:48,283][105692] Updated weights for policy 0, policy_version 693112 (0.0005) [2023-12-26 20:24:48,327][105620] Updated weights for policy 1, policy_version 693914 (0.0005) [2023-12-26 20:24:48,345][105692] Updated weights for policy 0, policy_version 693122 (0.0007) [2023-12-26 20:24:48,390][105620] Updated weights for policy 1, policy_version 693924 (0.0009) [2023-12-26 20:24:48,447][105620] Updated weights for policy 1, policy_version 693934 (0.0009) [2023-12-26 20:24:48,509][105620] Updated weights for policy 1, policy_version 693944 (0.0009) [2023-12-26 20:24:49,039][105692] Updated weights for policy 0, policy_version 693132 (0.0008) [2023-12-26 20:24:49,098][105692] Updated weights for policy 0, policy_version 693142 (0.0008) [2023-12-26 20:24:49,157][105692] Updated weights for policy 0, policy_version 693152 (0.0008) [2023-12-26 20:24:49,287][105620] Updated weights for policy 1, policy_version 693954 (0.0011) [2023-12-26 20:24:49,348][105620] Updated weights for policy 1, policy_version 693964 (0.0010) [2023-12-26 20:24:49,417][105620] Updated weights for policy 1, policy_version 693974 (0.0009) [2023-12-26 20:24:49,846][105692] Updated weights for policy 0, policy_version 693162 (0.0007) [2023-12-26 20:24:49,909][105692] Updated weights for policy 0, policy_version 693172 (0.0009) [2023-12-26 20:24:49,976][105692] Updated weights for policy 0, policy_version 693182 (0.0008) [2023-12-26 20:24:50,035][105692] Updated weights for policy 0, policy_version 693192 (0.0009) [2023-12-26 20:24:50,171][105620] Updated weights for policy 1, policy_version 693984 (0.0009) [2023-12-26 20:24:50,226][105620] Updated weights for policy 1, policy_version 693994 (0.0009) [2023-12-26 20:24:50,279][105620] Updated weights for policy 1, policy_version 694004 (0.0008) [2023-12-26 20:24:50,814][105692] Updated weights for policy 0, policy_version 693202 (0.0009) [2023-12-26 20:24:50,873][105692] Updated weights for policy 0, policy_version 693212 (0.0010) [2023-12-26 20:24:50,930][105692] Updated weights for policy 0, policy_version 693222 (0.0009) [2023-12-26 20:24:50,968][105620] Updated weights for policy 1, policy_version 694014 (0.0008) [2023-12-26 20:24:51,033][105620] Updated weights for policy 1, policy_version 694024 (0.0009) [2023-12-26 20:24:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19522.0). Total num frames: 355180544. Throughput: 0: 9766.8, 1: 9318.8. Samples: 355171508. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 20:24:51,062][104569] Avg episode reward: [(0, '9079.512'), (1, '8895.484')] [2023-12-26 20:24:51,099][105620] Updated weights for policy 1, policy_version 694034 (0.0009) [2023-12-26 20:24:51,643][105692] Updated weights for policy 0, policy_version 693232 (0.0007) [2023-12-26 20:24:51,702][105692] Updated weights for policy 0, policy_version 693242 (0.0006) [2023-12-26 20:24:51,770][105692] Updated weights for policy 0, policy_version 693252 (0.0007) [2023-12-26 20:24:51,944][105620] Updated weights for policy 1, policy_version 694044 (0.0009) [2023-12-26 20:24:52,012][105620] Updated weights for policy 1, policy_version 694054 (0.0009) [2023-12-26 20:24:52,071][105620] Updated weights for policy 1, policy_version 694064 (0.0009) [2023-12-26 20:24:52,455][105692] Updated weights for policy 0, policy_version 693262 (0.0008) [2023-12-26 20:24:52,513][105692] Updated weights for policy 0, policy_version 693272 (0.0009) [2023-12-26 20:24:52,569][105692] Updated weights for policy 0, policy_version 693282 (0.0008) [2023-12-26 20:24:52,827][105620] Updated weights for policy 1, policy_version 694074 (0.0009) [2023-12-26 20:24:52,888][105620] Updated weights for policy 1, policy_version 694084 (0.0009) [2023-12-26 20:24:52,943][105620] Updated weights for policy 1, policy_version 694094 (0.0009) [2023-12-26 20:24:52,997][105620] Updated weights for policy 1, policy_version 694104 (0.0009) [2023-12-26 20:24:53,364][105692] Updated weights for policy 0, policy_version 693292 (0.0009) [2023-12-26 20:24:53,410][105692] Updated weights for policy 0, policy_version 693302 (0.0008) [2023-12-26 20:24:53,457][105692] Updated weights for policy 0, policy_version 693312 (0.0009) [2023-12-26 20:24:53,708][105620] Updated weights for policy 1, policy_version 694114 (0.0009) [2023-12-26 20:24:53,760][105620] Updated weights for policy 1, policy_version 694124 (0.0009) [2023-12-26 20:24:53,817][105620] Updated weights for policy 1, policy_version 694134 (0.0008) [2023-12-26 20:24:54,229][105692] Updated weights for policy 0, policy_version 693322 (0.0009) [2023-12-26 20:24:54,281][105692] Updated weights for policy 0, policy_version 693332 (0.0008) [2023-12-26 20:24:54,334][105692] Updated weights for policy 0, policy_version 693342 (0.0007) [2023-12-26 20:24:54,397][105692] Updated weights for policy 0, policy_version 693352 (0.0005) [2023-12-26 20:24:54,607][105620] Updated weights for policy 1, policy_version 694144 (0.0009) [2023-12-26 20:24:54,668][105620] Updated weights for policy 1, policy_version 694154 (0.0009) [2023-12-26 20:24:54,723][105620] Updated weights for policy 1, policy_version 694164 (0.0009) [2023-12-26 20:24:55,076][105692] Updated weights for policy 0, policy_version 693362 (0.0009) [2023-12-26 20:24:55,132][105692] Updated weights for policy 0, policy_version 693372 (0.0009) [2023-12-26 20:24:55,181][105692] Updated weights for policy 0, policy_version 693382 (0.0009) [2023-12-26 20:24:55,436][105620] Updated weights for policy 1, policy_version 694174 (0.0008) [2023-12-26 20:24:55,498][105620] Updated weights for policy 1, policy_version 694184 (0.0009) [2023-12-26 20:24:55,558][105620] Updated weights for policy 1, policy_version 694194 (0.0009) [2023-12-26 20:24:55,888][105692] Updated weights for policy 0, policy_version 693392 (0.0006) [2023-12-26 20:24:55,956][105692] Updated weights for policy 0, policy_version 693402 (0.0005) [2023-12-26 20:24:56,025][105692] Updated weights for policy 0, policy_version 693412 (0.0005) [2023-12-26 20:24:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19114.7, 300 sec: 19494.2). Total num frames: 355278848. Throughput: 0: 9735.3, 1: 9331.3. Samples: 355283172. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 20:24:56,062][104569] Avg episode reward: [(0, '9261.679'), (1, '9170.900')] [2023-12-26 20:24:56,425][105620] Updated weights for policy 1, policy_version 694204 (0.0009) [2023-12-26 20:24:56,478][105620] Updated weights for policy 1, policy_version 694214 (0.0009) [2023-12-26 20:24:56,514][105692] Updated weights for policy 0, policy_version 693422 (0.0007) [2023-12-26 20:24:56,529][105620] Updated weights for policy 1, policy_version 694224 (0.0007) [2023-12-26 20:24:56,578][105692] Updated weights for policy 0, policy_version 693432 (0.0008) [2023-12-26 20:24:56,633][105692] Updated weights for policy 0, policy_version 693442 (0.0009) [2023-12-26 20:24:57,323][105620] Updated weights for policy 1, policy_version 694234 (0.0006) [2023-12-26 20:24:57,365][105692] Updated weights for policy 0, policy_version 693452 (0.0008) [2023-12-26 20:24:57,378][105620] Updated weights for policy 1, policy_version 694244 (0.0008) [2023-12-26 20:24:57,424][105692] Updated weights for policy 0, policy_version 693462 (0.0007) [2023-12-26 20:24:57,430][105620] Updated weights for policy 1, policy_version 694254 (0.0006) [2023-12-26 20:24:57,481][105620] Updated weights for policy 1, policy_version 694264 (0.0006) [2023-12-26 20:24:57,483][105692] Updated weights for policy 0, policy_version 693472 (0.0008) [2023-12-26 20:24:58,225][105620] Updated weights for policy 1, policy_version 694274 (0.0009) [2023-12-26 20:24:58,229][105692] Updated weights for policy 0, policy_version 693482 (0.0009) [2023-12-26 20:24:58,285][105620] Updated weights for policy 1, policy_version 694284 (0.0010) [2023-12-26 20:24:58,291][105692] Updated weights for policy 0, policy_version 693492 (0.0008) [2023-12-26 20:24:58,344][105620] Updated weights for policy 1, policy_version 694294 (0.0007) [2023-12-26 20:24:58,356][105692] Updated weights for policy 0, policy_version 693502 (0.0008) [2023-12-26 20:24:58,421][105692] Updated weights for policy 0, policy_version 693512 (0.0007) [2023-12-26 20:24:59,169][105620] Updated weights for policy 1, policy_version 694304 (0.0007) [2023-12-26 20:24:59,229][105620] Updated weights for policy 1, policy_version 694314 (0.0007) [2023-12-26 20:24:59,245][105692] Updated weights for policy 0, policy_version 693522 (0.0010) [2023-12-26 20:24:59,293][105620] Updated weights for policy 1, policy_version 694324 (0.0007) [2023-12-26 20:24:59,306][105692] Updated weights for policy 0, policy_version 693532 (0.0008) [2023-12-26 20:24:59,373][105692] Updated weights for policy 0, policy_version 693542 (0.0008) [2023-12-26 20:25:00,030][105620] Updated weights for policy 1, policy_version 694334 (0.0010) [2023-12-26 20:25:00,082][105620] Updated weights for policy 1, policy_version 694344 (0.0010) [2023-12-26 20:25:00,134][105620] Updated weights for policy 1, policy_version 694354 (0.0009) [2023-12-26 20:25:00,249][105692] Updated weights for policy 0, policy_version 693552 (0.0008) [2023-12-26 20:25:00,312][105692] Updated weights for policy 0, policy_version 693562 (0.0008) [2023-12-26 20:25:00,372][105692] Updated weights for policy 0, policy_version 693572 (0.0008) [2023-12-26 20:25:00,901][105620] Updated weights for policy 1, policy_version 694364 (0.0010) [2023-12-26 20:25:00,963][105620] Updated weights for policy 1, policy_version 694374 (0.0010) [2023-12-26 20:25:01,028][105620] Updated weights for policy 1, policy_version 694384 (0.0010) [2023-12-26 20:25:01,062][104569] Fps is (10 sec: 18022.3, 60 sec: 18978.1, 300 sec: 19438.6). Total num frames: 355360768. Throughput: 0: 9790.2, 1: 9327.0. Samples: 355340860. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 20:25:01,063][104569] Avg episode reward: [(0, '9171.448'), (1, '9171.993')] [2023-12-26 20:25:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000693576_177586176.pth... [2023-12-26 20:25:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000692456_177299456.pth [2023-12-26 20:25:01,076][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000694392_177782784.pth... [2023-12-26 20:25:01,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000693304_177504256.pth [2023-12-26 20:25:01,132][105692] Updated weights for policy 0, policy_version 693582 (0.0008) [2023-12-26 20:25:01,188][105692] Updated weights for policy 0, policy_version 693592 (0.0008) [2023-12-26 20:25:01,253][105692] Updated weights for policy 0, policy_version 693602 (0.0007) [2023-12-26 20:25:01,682][105620] Updated weights for policy 1, policy_version 694394 (0.0009) [2023-12-26 20:25:01,754][105620] Updated weights for policy 1, policy_version 694404 (0.0012) [2023-12-26 20:25:01,805][105620] Updated weights for policy 1, policy_version 694414 (0.0008) [2023-12-26 20:25:01,863][105620] Updated weights for policy 1, policy_version 694424 (0.0010) [2023-12-26 20:25:02,047][105692] Updated weights for policy 0, policy_version 693612 (0.0007) [2023-12-26 20:25:02,111][105692] Updated weights for policy 0, policy_version 693622 (0.0008) [2023-12-26 20:25:02,171][105692] Updated weights for policy 0, policy_version 693632 (0.0008) [2023-12-26 20:25:02,511][105620] Updated weights for policy 1, policy_version 694434 (0.0009) [2023-12-26 20:25:02,559][105620] Updated weights for policy 1, policy_version 694444 (0.0010) [2023-12-26 20:25:02,608][105620] Updated weights for policy 1, policy_version 694454 (0.0010) [2023-12-26 20:25:02,809][105692] Updated weights for policy 0, policy_version 693642 (0.0005) [2023-12-26 20:25:02,859][105692] Updated weights for policy 0, policy_version 693652 (0.0005) [2023-12-26 20:25:02,906][105692] Updated weights for policy 0, policy_version 693662 (0.0005) [2023-12-26 20:25:02,952][105692] Updated weights for policy 0, policy_version 693672 (0.0005) [2023-12-26 20:25:03,229][105620] Updated weights for policy 1, policy_version 694464 (0.0007) [2023-12-26 20:25:03,287][105620] Updated weights for policy 1, policy_version 694474 (0.0005) [2023-12-26 20:25:03,340][105620] Updated weights for policy 1, policy_version 694484 (0.0005) [2023-12-26 20:25:03,493][105692] Updated weights for policy 0, policy_version 693682 (0.0005) [2023-12-26 20:25:03,543][105692] Updated weights for policy 0, policy_version 693692 (0.0005) [2023-12-26 20:25:03,601][105692] Updated weights for policy 0, policy_version 693702 (0.0005) [2023-12-26 20:25:03,876][105620] Updated weights for policy 1, policy_version 694494 (0.0007) [2023-12-26 20:25:03,937][105620] Updated weights for policy 1, policy_version 694504 (0.0009) [2023-12-26 20:25:03,994][105620] Updated weights for policy 1, policy_version 694514 (0.0008) [2023-12-26 20:25:04,259][105692] Updated weights for policy 0, policy_version 693712 (0.0008) [2023-12-26 20:25:04,306][105692] Updated weights for policy 0, policy_version 693722 (0.0008) [2023-12-26 20:25:04,369][105692] Updated weights for policy 0, policy_version 693732 (0.0009) [2023-12-26 20:25:04,653][105620] Updated weights for policy 1, policy_version 694524 (0.0009) [2023-12-26 20:25:04,719][105620] Updated weights for policy 1, policy_version 694534 (0.0009) [2023-12-26 20:25:04,782][105620] Updated weights for policy 1, policy_version 694544 (0.0010) [2023-12-26 20:25:05,119][105692] Updated weights for policy 0, policy_version 693742 (0.0007) [2023-12-26 20:25:05,176][105692] Updated weights for policy 0, policy_version 693752 (0.0006) [2023-12-26 20:25:05,231][105692] Updated weights for policy 0, policy_version 693762 (0.0005) [2023-12-26 20:25:05,593][105620] Updated weights for policy 1, policy_version 694554 (0.0009) [2023-12-26 20:25:05,643][105620] Updated weights for policy 1, policy_version 694564 (0.0008) [2023-12-26 20:25:05,687][105620] Updated weights for policy 1, policy_version 694574 (0.0008) [2023-12-26 20:25:05,742][105620] Updated weights for policy 1, policy_version 694584 (0.0008) [2023-12-26 20:25:05,810][105692] Updated weights for policy 0, policy_version 693772 (0.0007) [2023-12-26 20:25:05,858][105692] Updated weights for policy 0, policy_version 693782 (0.0010) [2023-12-26 20:25:05,905][105692] Updated weights for policy 0, policy_version 693792 (0.0010) [2023-12-26 20:25:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 355475456. Throughput: 0: 9690.0, 1: 9476.0. Samples: 355459816. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 20:25:06,062][104569] Avg episode reward: [(0, '7139.058'), (1, '8657.951')] [2023-12-26 20:25:06,497][105620] Updated weights for policy 1, policy_version 694594 (0.0008) [2023-12-26 20:25:06,562][105620] Updated weights for policy 1, policy_version 694604 (0.0005) [2023-12-26 20:25:06,629][105620] Updated weights for policy 1, policy_version 694614 (0.0008) [2023-12-26 20:25:06,683][105692] Updated weights for policy 0, policy_version 693802 (0.0010) [2023-12-26 20:25:06,749][105692] Updated weights for policy 0, policy_version 693812 (0.0011) [2023-12-26 20:25:06,800][105692] Updated weights for policy 0, policy_version 693822 (0.0010) [2023-12-26 20:25:06,848][105692] Updated weights for policy 0, policy_version 693832 (0.0010) [2023-12-26 20:25:07,237][105620] Updated weights for policy 1, policy_version 694624 (0.0007) [2023-12-26 20:25:07,286][105620] Updated weights for policy 1, policy_version 694634 (0.0005) [2023-12-26 20:25:07,332][105620] Updated weights for policy 1, policy_version 694644 (0.0005) [2023-12-26 20:25:07,558][105692] Updated weights for policy 0, policy_version 693842 (0.0011) [2023-12-26 20:25:07,614][105692] Updated weights for policy 0, policy_version 693852 (0.0010) [2023-12-26 20:25:07,671][105692] Updated weights for policy 0, policy_version 693862 (0.0011) [2023-12-26 20:25:07,882][105620] Updated weights for policy 1, policy_version 694654 (0.0006) [2023-12-26 20:25:07,940][105620] Updated weights for policy 1, policy_version 694664 (0.0009) [2023-12-26 20:25:07,994][105620] Updated weights for policy 1, policy_version 694674 (0.0008) [2023-12-26 20:25:08,312][105692] Updated weights for policy 0, policy_version 693872 (0.0006) [2023-12-26 20:25:08,375][105692] Updated weights for policy 0, policy_version 693882 (0.0008) [2023-12-26 20:25:08,441][105692] Updated weights for policy 0, policy_version 693892 (0.0006) [2023-12-26 20:25:08,808][105620] Updated weights for policy 1, policy_version 694684 (0.0010) [2023-12-26 20:25:08,870][105620] Updated weights for policy 1, policy_version 694694 (0.0010) [2023-12-26 20:25:08,929][105620] Updated weights for policy 1, policy_version 694704 (0.0011) [2023-12-26 20:25:09,102][105692] Updated weights for policy 0, policy_version 693902 (0.0009) [2023-12-26 20:25:09,160][105692] Updated weights for policy 0, policy_version 693912 (0.0011) [2023-12-26 20:25:09,225][105692] Updated weights for policy 0, policy_version 693922 (0.0010) [2023-12-26 20:25:09,651][105620] Updated weights for policy 1, policy_version 694714 (0.0010) [2023-12-26 20:25:09,704][105620] Updated weights for policy 1, policy_version 694724 (0.0010) [2023-12-26 20:25:09,763][105620] Updated weights for policy 1, policy_version 694734 (0.0011) [2023-12-26 20:25:09,820][105620] Updated weights for policy 1, policy_version 694744 (0.0011) [2023-12-26 20:25:09,923][105692] Updated weights for policy 0, policy_version 693932 (0.0008) [2023-12-26 20:25:09,989][105692] Updated weights for policy 0, policy_version 693942 (0.0008) [2023-12-26 20:25:10,043][105692] Updated weights for policy 0, policy_version 693952 (0.0008) [2023-12-26 20:25:10,624][105620] Updated weights for policy 1, policy_version 694754 (0.0009) [2023-12-26 20:25:10,673][105620] Updated weights for policy 1, policy_version 694764 (0.0008) [2023-12-26 20:25:10,721][105620] Updated weights for policy 1, policy_version 694774 (0.0009) [2023-12-26 20:25:10,770][105692] Updated weights for policy 0, policy_version 693962 (0.0009) [2023-12-26 20:25:10,832][105692] Updated weights for policy 0, policy_version 693972 (0.0010) [2023-12-26 20:25:10,884][105692] Updated weights for policy 0, policy_version 693982 (0.0010) [2023-12-26 20:25:10,941][105692] Updated weights for policy 0, policy_version 693992 (0.0007) [2023-12-26 20:25:11,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 355573760. Throughput: 0: 9755.4, 1: 9519.6. Samples: 355578828. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 20:25:11,063][104569] Avg episode reward: [(0, '1124.252'), (1, '8393.168')] [2023-12-26 20:25:11,519][105620] Updated weights for policy 1, policy_version 694784 (0.0008) [2023-12-26 20:25:11,574][105620] Updated weights for policy 1, policy_version 694794 (0.0008) [2023-12-26 20:25:11,634][105620] Updated weights for policy 1, policy_version 694804 (0.0008) [2023-12-26 20:25:11,640][105692] Updated weights for policy 0, policy_version 694002 (0.0007) [2023-12-26 20:25:11,701][105692] Updated weights for policy 0, policy_version 694012 (0.0006) [2023-12-26 20:25:11,769][105692] Updated weights for policy 0, policy_version 694022 (0.0009) [2023-12-26 20:25:12,433][105620] Updated weights for policy 1, policy_version 694814 (0.0008) [2023-12-26 20:25:12,480][105620] Updated weights for policy 1, policy_version 694824 (0.0008) [2023-12-26 20:25:12,507][105692] Updated weights for policy 0, policy_version 694032 (0.0007) [2023-12-26 20:25:12,539][105620] Updated weights for policy 1, policy_version 694834 (0.0007) [2023-12-26 20:25:12,561][105692] Updated weights for policy 0, policy_version 694042 (0.0008) [2023-12-26 20:25:12,621][105692] Updated weights for policy 0, policy_version 694052 (0.0008) [2023-12-26 20:25:13,303][105620] Updated weights for policy 1, policy_version 694844 (0.0008) [2023-12-26 20:25:13,350][105620] Updated weights for policy 1, policy_version 694854 (0.0009) [2023-12-26 20:25:13,383][105692] Updated weights for policy 0, policy_version 694062 (0.0009) [2023-12-26 20:25:13,402][105620] Updated weights for policy 1, policy_version 694864 (0.0006) [2023-12-26 20:25:13,439][105692] Updated weights for policy 0, policy_version 694072 (0.0007) [2023-12-26 20:25:13,492][105692] Updated weights for policy 0, policy_version 694082 (0.0009) [2023-12-26 20:25:14,145][105692] Updated weights for policy 0, policy_version 694092 (0.0009) [2023-12-26 20:25:14,200][105692] Updated weights for policy 0, policy_version 694102 (0.0010) [2023-12-26 20:25:14,210][105620] Updated weights for policy 1, policy_version 694874 (0.0005) [2023-12-26 20:25:14,254][105692] Updated weights for policy 0, policy_version 694112 (0.0010) [2023-12-26 20:25:14,262][105620] Updated weights for policy 1, policy_version 694884 (0.0007) [2023-12-26 20:25:14,315][105620] Updated weights for policy 1, policy_version 694894 (0.0009) [2023-12-26 20:25:14,373][105620] Updated weights for policy 1, policy_version 694904 (0.0008) [2023-12-26 20:25:14,977][105692] Updated weights for policy 0, policy_version 694122 (0.0011) [2023-12-26 20:25:15,035][105692] Updated weights for policy 0, policy_version 694132 (0.0011) [2023-12-26 20:25:15,084][105692] Updated weights for policy 0, policy_version 694142 (0.0011) [2023-12-26 20:25:15,098][105620] Updated weights for policy 1, policy_version 694914 (0.0006) [2023-12-26 20:25:15,148][105692] Updated weights for policy 0, policy_version 694152 (0.0011) [2023-12-26 20:25:15,162][105620] Updated weights for policy 1, policy_version 694924 (0.0006) [2023-12-26 20:25:15,218][105620] Updated weights for policy 1, policy_version 694934 (0.0008) [2023-12-26 20:25:15,872][105692] Updated weights for policy 0, policy_version 694162 (0.0010) [2023-12-26 20:25:15,916][105692] Updated weights for policy 0, policy_version 694172 (0.0010) [2023-12-26 20:25:15,961][105692] Updated weights for policy 0, policy_version 694182 (0.0008) [2023-12-26 20:25:15,973][105620] Updated weights for policy 1, policy_version 694944 (0.0008) [2023-12-26 20:25:16,032][105620] Updated weights for policy 1, policy_version 694954 (0.0008) [2023-12-26 20:25:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 355663872. Throughput: 0: 9686.3, 1: 9525.1. Samples: 355633592. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 20:25:16,062][104569] Avg episode reward: [(0, '1137.098'), (1, '8619.844')] [2023-12-26 20:25:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000694184_177741824.pth... [2023-12-26 20:25:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000693032_177446912.pth [2023-12-26 20:25:16,089][105620] Updated weights for policy 1, policy_version 694964 (0.0009) [2023-12-26 20:25:16,114][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000694968_177930240.pth... [2023-12-26 20:25:16,119][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000693816_177635328.pth [2023-12-26 20:25:16,710][105692] Updated weights for policy 0, policy_version 694192 (0.0009) [2023-12-26 20:25:16,754][105692] Updated weights for policy 0, policy_version 694202 (0.0010) [2023-12-26 20:25:16,818][105692] Updated weights for policy 0, policy_version 694212 (0.0010) [2023-12-26 20:25:16,873][105620] Updated weights for policy 1, policy_version 694974 (0.0009) [2023-12-26 20:25:16,938][105620] Updated weights for policy 1, policy_version 694984 (0.0010) [2023-12-26 20:25:16,992][105620] Updated weights for policy 1, policy_version 694994 (0.0010) [2023-12-26 20:25:17,575][105692] Updated weights for policy 0, policy_version 694222 (0.0010) [2023-12-26 20:25:17,636][105692] Updated weights for policy 0, policy_version 694232 (0.0010) [2023-12-26 20:25:17,697][105620] Updated weights for policy 1, policy_version 695004 (0.0008) [2023-12-26 20:25:17,698][105692] Updated weights for policy 0, policy_version 694242 (0.0009) [2023-12-26 20:25:17,752][105620] Updated weights for policy 1, policy_version 695014 (0.0006) [2023-12-26 20:25:17,813][105620] Updated weights for policy 1, policy_version 695024 (0.0005) [2023-12-26 20:25:18,393][105692] Updated weights for policy 0, policy_version 694252 (0.0009) [2023-12-26 20:25:18,442][105620] Updated weights for policy 1, policy_version 695034 (0.0006) [2023-12-26 20:25:18,449][105692] Updated weights for policy 0, policy_version 694262 (0.0007) [2023-12-26 20:25:18,503][105692] Updated weights for policy 0, policy_version 694272 (0.0007) [2023-12-26 20:25:18,505][105620] Updated weights for policy 1, policy_version 695044 (0.0008) [2023-12-26 20:25:18,567][105620] Updated weights for policy 1, policy_version 695054 (0.0007) [2023-12-26 20:25:18,626][105620] Updated weights for policy 1, policy_version 695064 (0.0006) [2023-12-26 20:25:19,202][105620] Updated weights for policy 1, policy_version 695074 (0.0005) [2023-12-26 20:25:19,271][105620] Updated weights for policy 1, policy_version 695084 (0.0007) [2023-12-26 20:25:19,339][105620] Updated weights for policy 1, policy_version 695094 (0.0007) [2023-12-26 20:25:19,376][105692] Updated weights for policy 0, policy_version 694282 (0.0006) [2023-12-26 20:25:19,438][105692] Updated weights for policy 0, policy_version 694292 (0.0008) [2023-12-26 20:25:19,499][105692] Updated weights for policy 0, policy_version 694302 (0.0008) [2023-12-26 20:25:19,560][105692] Updated weights for policy 0, policy_version 694312 (0.0006) [2023-12-26 20:25:20,048][105620] Updated weights for policy 1, policy_version 695104 (0.0009) [2023-12-26 20:25:20,114][105620] Updated weights for policy 1, policy_version 695114 (0.0009) [2023-12-26 20:25:20,177][105620] Updated weights for policy 1, policy_version 695124 (0.0007) [2023-12-26 20:25:20,202][105692] Updated weights for policy 0, policy_version 694322 (0.0007) [2023-12-26 20:25:20,251][105692] Updated weights for policy 0, policy_version 694332 (0.0006) [2023-12-26 20:25:20,304][105692] Updated weights for policy 0, policy_version 694342 (0.0009) [2023-12-26 20:25:20,928][105620] Updated weights for policy 1, policy_version 695134 (0.0008) [2023-12-26 20:25:20,946][105586] KL-divergence is very high: 157.4885 [2023-12-26 20:25:20,994][105620] Updated weights for policy 1, policy_version 695144 (0.0009) [2023-12-26 20:25:20,999][105586] KL-divergence is very high: 207.5024 [2023-12-26 20:25:21,057][105586] KL-divergence is very high: 139.0203 [2023-12-26 20:25:21,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 355753984. Throughput: 0: 9702.1, 1: 9545.9. Samples: 355750140. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 20:25:21,063][104569] Avg episode reward: [(0, '6431.331'), (1, '8804.600')] [2023-12-26 20:25:21,063][105620] Updated weights for policy 1, policy_version 695154 (0.0008) [2023-12-26 20:25:21,127][105692] Updated weights for policy 0, policy_version 694352 (0.0008) [2023-12-26 20:25:21,193][105692] Updated weights for policy 0, policy_version 694362 (0.0008) [2023-12-26 20:25:21,259][105692] Updated weights for policy 0, policy_version 694372 (0.0008) [2023-12-26 20:25:21,775][105620] Updated weights for policy 1, policy_version 695164 (0.0007) [2023-12-26 20:25:21,848][105620] Updated weights for policy 1, policy_version 695174 (0.0006) [2023-12-26 20:25:21,919][105620] Updated weights for policy 1, policy_version 695184 (0.0008) [2023-12-26 20:25:22,084][105692] Updated weights for policy 0, policy_version 694382 (0.0009) [2023-12-26 20:25:22,143][105692] Updated weights for policy 0, policy_version 694392 (0.0009) [2023-12-26 20:25:22,192][105692] Updated weights for policy 0, policy_version 694402 (0.0009) [2023-12-26 20:25:22,550][105620] Updated weights for policy 1, policy_version 695194 (0.0008) [2023-12-26 20:25:22,605][105620] Updated weights for policy 1, policy_version 695204 (0.0008) [2023-12-26 20:25:22,661][105620] Updated weights for policy 1, policy_version 695214 (0.0009) [2023-12-26 20:25:22,720][105620] Updated weights for policy 1, policy_version 695224 (0.0009) [2023-12-26 20:25:22,990][105692] Updated weights for policy 0, policy_version 694412 (0.0008) [2023-12-26 20:25:23,044][105692] Updated weights for policy 0, policy_version 694422 (0.0008) [2023-12-26 20:25:23,102][105692] Updated weights for policy 0, policy_version 694432 (0.0009) [2023-12-26 20:25:23,437][105620] Updated weights for policy 1, policy_version 695235 (0.0009) [2023-12-26 20:25:23,491][105620] Updated weights for policy 1, policy_version 695245 (0.0010) [2023-12-26 20:25:23,547][105620] Updated weights for policy 1, policy_version 695256 (0.0010) [2023-12-26 20:25:23,753][105692] Updated weights for policy 0, policy_version 694442 (0.0008) [2023-12-26 20:25:23,813][105692] Updated weights for policy 0, policy_version 694452 (0.0005) [2023-12-26 20:25:23,867][105692] Updated weights for policy 0, policy_version 694462 (0.0005) [2023-12-26 20:25:23,928][105692] Updated weights for policy 0, policy_version 694472 (0.0005) [2023-12-26 20:25:24,352][105620] Updated weights for policy 1, policy_version 695266 (0.0009) [2023-12-26 20:25:24,415][105620] Updated weights for policy 1, policy_version 695276 (0.0008) [2023-12-26 20:25:24,487][105620] Updated weights for policy 1, policy_version 695286 (0.0008) [2023-12-26 20:25:24,573][105692] Updated weights for policy 0, policy_version 694482 (0.0009) [2023-12-26 20:25:24,640][105692] Updated weights for policy 0, policy_version 694492 (0.0007) [2023-12-26 20:25:24,702][105692] Updated weights for policy 0, policy_version 694502 (0.0006) [2023-12-26 20:25:25,241][105692] Updated weights for policy 0, policy_version 694512 (0.0006) [2023-12-26 20:25:25,301][105692] Updated weights for policy 0, policy_version 694522 (0.0007) [2023-12-26 20:25:25,328][105620] Updated weights for policy 1, policy_version 695296 (0.0007) [2023-12-26 20:25:25,358][105692] Updated weights for policy 0, policy_version 694532 (0.0008) [2023-12-26 20:25:25,381][105620] Updated weights for policy 1, policy_version 695306 (0.0006) [2023-12-26 20:25:25,434][105620] Updated weights for policy 1, policy_version 695316 (0.0008) [2023-12-26 20:25:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 355852288. Throughput: 0: 9744.4, 1: 9562.0. Samples: 355865672. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 20:25:26,062][104569] Avg episode reward: [(0, '9260.823'), (1, '8988.678')] [2023-12-26 20:25:26,077][105692] Updated weights for policy 0, policy_version 694542 (0.0008) [2023-12-26 20:25:26,128][105692] Updated weights for policy 0, policy_version 694552 (0.0009) [2023-12-26 20:25:26,151][105620] Updated weights for policy 1, policy_version 695326 (0.0008) [2023-12-26 20:25:26,174][105692] Updated weights for policy 0, policy_version 694562 (0.0006) [2023-12-26 20:25:26,204][105620] Updated weights for policy 1, policy_version 695336 (0.0007) [2023-12-26 20:25:26,266][105620] Updated weights for policy 1, policy_version 695346 (0.0008) [2023-12-26 20:25:26,830][105692] Updated weights for policy 0, policy_version 694572 (0.0005) [2023-12-26 20:25:26,893][105692] Updated weights for policy 0, policy_version 694582 (0.0005) [2023-12-26 20:25:26,954][105692] Updated weights for policy 0, policy_version 694592 (0.0005) [2023-12-26 20:25:27,109][105620] Updated weights for policy 1, policy_version 695356 (0.0008) [2023-12-26 20:25:27,170][105620] Updated weights for policy 1, policy_version 695366 (0.0009) [2023-12-26 20:25:27,216][105620] Updated weights for policy 1, policy_version 695376 (0.0009) [2023-12-26 20:25:27,612][105692] Updated weights for policy 0, policy_version 694602 (0.0008) [2023-12-26 20:25:27,675][105692] Updated weights for policy 0, policy_version 694612 (0.0009) [2023-12-26 20:25:27,734][105692] Updated weights for policy 0, policy_version 694622 (0.0009) [2023-12-26 20:25:27,815][105692] Updated weights for policy 0, policy_version 694632 (0.0009) [2023-12-26 20:25:27,956][105620] Updated weights for policy 1, policy_version 695386 (0.0009) [2023-12-26 20:25:28,017][105620] Updated weights for policy 1, policy_version 695396 (0.0009) [2023-12-26 20:25:28,064][105620] Updated weights for policy 1, policy_version 695406 (0.0009) [2023-12-26 20:25:28,111][105620] Updated weights for policy 1, policy_version 695416 (0.0009) [2023-12-26 20:25:28,525][105692] Updated weights for policy 0, policy_version 694642 (0.0009) [2023-12-26 20:25:28,576][105692] Updated weights for policy 0, policy_version 694652 (0.0008) [2023-12-26 20:25:28,626][105692] Updated weights for policy 0, policy_version 694662 (0.0008) [2023-12-26 20:25:28,871][105620] Updated weights for policy 1, policy_version 695426 (0.0008) [2023-12-26 20:25:28,925][105620] Updated weights for policy 1, policy_version 695436 (0.0008) [2023-12-26 20:25:28,976][105620] Updated weights for policy 1, policy_version 695446 (0.0008) [2023-12-26 20:25:29,412][105692] Updated weights for policy 0, policy_version 694672 (0.0010) [2023-12-26 20:25:29,469][105692] Updated weights for policy 0, policy_version 694683 (0.0008) [2023-12-26 20:25:29,524][105692] Updated weights for policy 0, policy_version 694693 (0.0008) [2023-12-26 20:25:29,701][105620] Updated weights for policy 1, policy_version 695456 (0.0006) [2023-12-26 20:25:29,766][105620] Updated weights for policy 1, policy_version 695466 (0.0008) [2023-12-26 20:25:29,827][105620] Updated weights for policy 1, policy_version 695476 (0.0008) [2023-12-26 20:25:30,373][105620] Updated weights for policy 1, policy_version 695486 (0.0006) [2023-12-26 20:25:30,416][105692] Updated weights for policy 0, policy_version 694703 (0.0008) [2023-12-26 20:25:30,429][105620] Updated weights for policy 1, policy_version 695496 (0.0005) [2023-12-26 20:25:30,479][105692] Updated weights for policy 0, policy_version 694713 (0.0008) [2023-12-26 20:25:30,484][105620] Updated weights for policy 1, policy_version 695506 (0.0006) [2023-12-26 20:25:30,534][105692] Updated weights for policy 0, policy_version 694723 (0.0006) [2023-12-26 20:25:31,058][105620] Updated weights for policy 1, policy_version 695516 (0.0009) [2023-12-26 20:25:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 355950592. Throughput: 0: 9780.1, 1: 9564.6. Samples: 355922908. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-26 20:25:31,062][104569] Avg episode reward: [(0, '9179.107'), (1, '8987.153')] [2023-12-26 20:25:31,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000694728_177881088.pth... [2023-12-26 20:25:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000693576_177586176.pth [2023-12-26 20:25:31,128][105620] Updated weights for policy 1, policy_version 695526 (0.0006) [2023-12-26 20:25:31,223][105620] Updated weights for policy 1, policy_version 695536 (0.0009) [2023-12-26 20:25:31,276][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000695544_178077696.pth... [2023-12-26 20:25:31,281][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000694392_177782784.pth [2023-12-26 20:25:31,426][105692] Updated weights for policy 0, policy_version 694733 (0.0008) [2023-12-26 20:25:31,485][105692] Updated weights for policy 0, policy_version 694743 (0.0010) [2023-12-26 20:25:31,553][105692] Updated weights for policy 0, policy_version 694753 (0.0010) [2023-12-26 20:25:31,845][105620] Updated weights for policy 1, policy_version 695546 (0.0009) [2023-12-26 20:25:31,897][105620] Updated weights for policy 1, policy_version 695556 (0.0006) [2023-12-26 20:25:31,947][105620] Updated weights for policy 1, policy_version 695566 (0.0008) [2023-12-26 20:25:31,996][105620] Updated weights for policy 1, policy_version 695576 (0.0010) [2023-12-26 20:25:32,359][105692] Updated weights for policy 0, policy_version 694763 (0.0009) [2023-12-26 20:25:32,422][105692] Updated weights for policy 0, policy_version 694773 (0.0009) [2023-12-26 20:25:32,482][105692] Updated weights for policy 0, policy_version 694783 (0.0009) [2023-12-26 20:25:32,658][105620] Updated weights for policy 1, policy_version 695586 (0.0010) [2023-12-26 20:25:32,726][105620] Updated weights for policy 1, policy_version 695596 (0.0010) [2023-12-26 20:25:32,793][105620] Updated weights for policy 1, policy_version 695606 (0.0006) [2023-12-26 20:25:33,128][105692] Updated weights for policy 0, policy_version 694793 (0.0009) [2023-12-26 20:25:33,198][105692] Updated weights for policy 0, policy_version 694803 (0.0009) [2023-12-26 20:25:33,249][105692] Updated weights for policy 0, policy_version 694813 (0.0010) [2023-12-26 20:25:33,302][105692] Updated weights for policy 0, policy_version 694823 (0.0011) [2023-12-26 20:25:33,311][105620] Updated weights for policy 1, policy_version 695616 (0.0005) [2023-12-26 20:25:33,370][105620] Updated weights for policy 1, policy_version 695626 (0.0005) [2023-12-26 20:25:33,431][105620] Updated weights for policy 1, policy_version 695636 (0.0005) [2023-12-26 20:25:33,993][105692] Updated weights for policy 0, policy_version 694833 (0.0006) [2023-12-26 20:25:34,054][105692] Updated weights for policy 0, policy_version 694843 (0.0008) [2023-12-26 20:25:34,056][105620] Updated weights for policy 1, policy_version 695646 (0.0008) [2023-12-26 20:25:34,111][105620] Updated weights for policy 1, policy_version 695656 (0.0010) [2023-12-26 20:25:34,113][105692] Updated weights for policy 0, policy_version 694853 (0.0006) [2023-12-26 20:25:34,177][105620] Updated weights for policy 1, policy_version 695666 (0.0008) [2023-12-26 20:25:34,834][105620] Updated weights for policy 1, policy_version 695676 (0.0006) [2023-12-26 20:25:34,873][105692] Updated weights for policy 0, policy_version 694863 (0.0008) [2023-12-26 20:25:34,900][105620] Updated weights for policy 1, policy_version 695686 (0.0005) [2023-12-26 20:25:34,917][105692] Updated weights for policy 0, policy_version 694873 (0.0005) [2023-12-26 20:25:34,959][105620] Updated weights for policy 1, policy_version 695696 (0.0005) [2023-12-26 20:25:34,984][105692] Updated weights for policy 0, policy_version 694883 (0.0006) [2023-12-26 20:25:35,577][105620] Updated weights for policy 1, policy_version 695706 (0.0006) [2023-12-26 20:25:35,632][105620] Updated weights for policy 1, policy_version 695716 (0.0009) [2023-12-26 20:25:35,633][105692] Updated weights for policy 0, policy_version 694893 (0.0009) [2023-12-26 20:25:35,677][105620] Updated weights for policy 1, policy_version 695726 (0.0010) [2023-12-26 20:25:35,683][105692] Updated weights for policy 0, policy_version 694903 (0.0006) [2023-12-26 20:25:35,726][105620] Updated weights for policy 1, policy_version 695736 (0.0009) [2023-12-26 20:25:35,737][105692] Updated weights for policy 0, policy_version 694913 (0.0007) [2023-12-26 20:25:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 356057088. Throughput: 0: 9660.3, 1: 9718.8. Samples: 356043572. Policy #0 lag: (min: 27.0, avg: 40.4, max: 59.0) [2023-12-26 20:25:36,063][104569] Avg episode reward: [(0, '8376.302'), (1, '8987.779')] [2023-12-26 20:25:36,436][105692] Updated weights for policy 0, policy_version 694923 (0.0008) [2023-12-26 20:25:36,484][105620] Updated weights for policy 1, policy_version 695746 (0.0008) [2023-12-26 20:25:36,502][105692] Updated weights for policy 0, policy_version 694933 (0.0006) [2023-12-26 20:25:36,540][105620] Updated weights for policy 1, policy_version 695756 (0.0010) [2023-12-26 20:25:36,565][105692] Updated weights for policy 0, policy_version 694943 (0.0006) [2023-12-26 20:25:36,597][105620] Updated weights for policy 1, policy_version 695766 (0.0011) [2023-12-26 20:25:37,155][105692] Updated weights for policy 0, policy_version 694953 (0.0005) [2023-12-26 20:25:37,205][105692] Updated weights for policy 0, policy_version 694963 (0.0005) [2023-12-26 20:25:37,255][105692] Updated weights for policy 0, policy_version 694973 (0.0005) [2023-12-26 20:25:37,300][105620] Updated weights for policy 1, policy_version 695776 (0.0008) [2023-12-26 20:25:37,308][105692] Updated weights for policy 0, policy_version 694983 (0.0005) [2023-12-26 20:25:37,357][105620] Updated weights for policy 1, policy_version 695786 (0.0009) [2023-12-26 20:25:37,416][105620] Updated weights for policy 1, policy_version 695796 (0.0008) [2023-12-26 20:25:38,071][105692] Updated weights for policy 0, policy_version 694993 (0.0008) [2023-12-26 20:25:38,077][105620] Updated weights for policy 1, policy_version 695806 (0.0006) [2023-12-26 20:25:38,130][105692] Updated weights for policy 0, policy_version 695003 (0.0011) [2023-12-26 20:25:38,140][105620] Updated weights for policy 1, policy_version 695816 (0.0006) [2023-12-26 20:25:38,187][105692] Updated weights for policy 0, policy_version 695013 (0.0011) [2023-12-26 20:25:38,201][105620] Updated weights for policy 1, policy_version 695826 (0.0006) [2023-12-26 20:25:38,914][105692] Updated weights for policy 0, policy_version 695023 (0.0010) [2023-12-26 20:25:38,947][105620] Updated weights for policy 1, policy_version 695836 (0.0006) [2023-12-26 20:25:38,976][105692] Updated weights for policy 0, policy_version 695033 (0.0010) [2023-12-26 20:25:39,006][105620] Updated weights for policy 1, policy_version 695846 (0.0008) [2023-12-26 20:25:39,032][105692] Updated weights for policy 0, policy_version 695043 (0.0010) [2023-12-26 20:25:39,066][105620] Updated weights for policy 1, policy_version 695856 (0.0006) [2023-12-26 20:25:39,771][105692] Updated weights for policy 0, policy_version 695053 (0.0008) [2023-12-26 20:25:39,833][105620] Updated weights for policy 1, policy_version 695866 (0.0008) [2023-12-26 20:25:39,836][105692] Updated weights for policy 0, policy_version 695063 (0.0007) [2023-12-26 20:25:39,898][105692] Updated weights for policy 0, policy_version 695073 (0.0006) [2023-12-26 20:25:39,901][105620] Updated weights for policy 1, policy_version 695876 (0.0007) [2023-12-26 20:25:39,970][105620] Updated weights for policy 1, policy_version 695886 (0.0008) [2023-12-26 20:25:40,568][105692] Updated weights for policy 0, policy_version 695083 (0.0008) [2023-12-26 20:25:40,628][105692] Updated weights for policy 0, policy_version 695093 (0.0009) [2023-12-26 20:25:40,695][105692] Updated weights for policy 0, policy_version 695103 (0.0011) [2023-12-26 20:25:40,756][105620] Updated weights for policy 1, policy_version 695897 (0.0011) [2023-12-26 20:25:40,813][105620] Updated weights for policy 1, policy_version 695907 (0.0008) [2023-12-26 20:25:40,870][105620] Updated weights for policy 1, policy_version 695917 (0.0008) [2023-12-26 20:25:40,929][105620] Updated weights for policy 1, policy_version 695927 (0.0008) [2023-12-26 20:25:41,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 356155392. Throughput: 0: 9729.6, 1: 9761.0. Samples: 356160248. Policy #0 lag: (min: 27.0, avg: 40.4, max: 59.0) [2023-12-26 20:25:41,063][104569] Avg episode reward: [(0, '8151.169'), (1, '8714.701')] [2023-12-26 20:25:41,415][105692] Updated weights for policy 0, policy_version 695113 (0.0011) [2023-12-26 20:25:41,481][105692] Updated weights for policy 0, policy_version 695123 (0.0009) [2023-12-26 20:25:41,539][105692] Updated weights for policy 0, policy_version 695133 (0.0006) [2023-12-26 20:25:41,595][105692] Updated weights for policy 0, policy_version 695143 (0.0007) [2023-12-26 20:25:41,753][105620] Updated weights for policy 1, policy_version 695937 (0.0008) [2023-12-26 20:25:41,806][105620] Updated weights for policy 1, policy_version 695947 (0.0009) [2023-12-26 20:25:41,860][105620] Updated weights for policy 1, policy_version 695957 (0.0010) [2023-12-26 20:25:42,299][105692] Updated weights for policy 0, policy_version 695153 (0.0011) [2023-12-26 20:25:42,362][105692] Updated weights for policy 0, policy_version 695163 (0.0008) [2023-12-26 20:25:42,429][105692] Updated weights for policy 0, policy_version 695173 (0.0009) [2023-12-26 20:25:42,587][105620] Updated weights for policy 1, policy_version 695967 (0.0010) [2023-12-26 20:25:42,646][105620] Updated weights for policy 1, policy_version 695977 (0.0009) [2023-12-26 20:25:42,705][105620] Updated weights for policy 1, policy_version 695987 (0.0006) [2023-12-26 20:25:43,199][105692] Updated weights for policy 0, policy_version 695183 (0.0006) [2023-12-26 20:25:43,255][105692] Updated weights for policy 0, policy_version 695193 (0.0007) [2023-12-26 20:25:43,311][105692] Updated weights for policy 0, policy_version 695203 (0.0005) [2023-12-26 20:25:43,317][105620] Updated weights for policy 1, policy_version 695997 (0.0008) [2023-12-26 20:25:43,386][105620] Updated weights for policy 1, policy_version 696007 (0.0006) [2023-12-26 20:25:43,448][105620] Updated weights for policy 1, policy_version 696017 (0.0007) [2023-12-26 20:25:43,949][105692] Updated weights for policy 0, policy_version 695213 (0.0007) [2023-12-26 20:25:44,009][105692] Updated weights for policy 0, policy_version 695223 (0.0008) [2023-12-26 20:25:44,074][105692] Updated weights for policy 0, policy_version 695233 (0.0009) [2023-12-26 20:25:44,175][105620] Updated weights for policy 1, policy_version 696027 (0.0009) [2023-12-26 20:25:44,233][105620] Updated weights for policy 1, policy_version 696037 (0.0009) [2023-12-26 20:25:44,288][105620] Updated weights for policy 1, policy_version 696047 (0.0005) [2023-12-26 20:25:44,734][105692] Updated weights for policy 0, policy_version 695243 (0.0009) [2023-12-26 20:25:44,797][105692] Updated weights for policy 0, policy_version 695253 (0.0009) [2023-12-26 20:25:44,861][105692] Updated weights for policy 0, policy_version 695263 (0.0008) [2023-12-26 20:25:45,092][105620] Updated weights for policy 1, policy_version 696057 (0.0008) [2023-12-26 20:25:45,160][105620] Updated weights for policy 1, policy_version 696067 (0.0009) [2023-12-26 20:25:45,224][105620] Updated weights for policy 1, policy_version 696077 (0.0008) [2023-12-26 20:25:45,290][105620] Updated weights for policy 1, policy_version 696087 (0.0009) [2023-12-26 20:25:45,606][105692] Updated weights for policy 0, policy_version 695273 (0.0009) [2023-12-26 20:25:45,666][105692] Updated weights for policy 0, policy_version 695283 (0.0009) [2023-12-26 20:25:45,725][105692] Updated weights for policy 0, policy_version 695293 (0.0009) [2023-12-26 20:25:45,781][105692] Updated weights for policy 0, policy_version 695303 (0.0009) [2023-12-26 20:25:46,019][105620] Updated weights for policy 1, policy_version 696097 (0.0010) [2023-12-26 20:25:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 356245504. Throughput: 0: 9676.9, 1: 9813.1. Samples: 356217908. Policy #0 lag: (min: 27.0, avg: 40.4, max: 59.0) [2023-12-26 20:25:46,062][104569] Avg episode reward: [(0, '5498.501'), (1, '8714.626')] [2023-12-26 20:25:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000695304_178028544.pth... [2023-12-26 20:25:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000694184_177741824.pth [2023-12-26 20:25:46,071][105620] Updated weights for policy 1, policy_version 696107 (0.0009) [2023-12-26 20:25:46,121][105620] Updated weights for policy 1, policy_version 696117 (0.0009) [2023-12-26 20:25:46,131][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000696120_178225152.pth... [2023-12-26 20:25:46,134][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000694968_177930240.pth [2023-12-26 20:25:46,480][105692] Updated weights for policy 0, policy_version 695313 (0.0008) [2023-12-26 20:25:46,538][105692] Updated weights for policy 0, policy_version 695323 (0.0009) [2023-12-26 20:25:46,591][105692] Updated weights for policy 0, policy_version 695333 (0.0010) [2023-12-26 20:25:46,823][105620] Updated weights for policy 1, policy_version 696128 (0.0009) [2023-12-26 20:25:46,880][105620] Updated weights for policy 1, policy_version 696138 (0.0007) [2023-12-26 20:25:46,933][105620] Updated weights for policy 1, policy_version 696148 (0.0006) [2023-12-26 20:25:47,344][105692] Updated weights for policy 0, policy_version 695343 (0.0010) [2023-12-26 20:25:47,398][105692] Updated weights for policy 0, policy_version 695353 (0.0010) [2023-12-26 20:25:47,458][105692] Updated weights for policy 0, policy_version 695363 (0.0010) [2023-12-26 20:25:47,598][105620] Updated weights for policy 1, policy_version 696158 (0.0005) [2023-12-26 20:25:47,661][105620] Updated weights for policy 1, policy_version 696168 (0.0010) [2023-12-26 20:25:47,713][105620] Updated weights for policy 1, policy_version 696178 (0.0010) [2023-12-26 20:25:48,075][105692] Updated weights for policy 0, policy_version 695373 (0.0008) [2023-12-26 20:25:48,135][105692] Updated weights for policy 0, policy_version 695383 (0.0006) [2023-12-26 20:25:48,201][105692] Updated weights for policy 0, policy_version 695393 (0.0007) [2023-12-26 20:25:48,373][105620] Updated weights for policy 1, policy_version 696188 (0.0010) [2023-12-26 20:25:48,432][105620] Updated weights for policy 1, policy_version 696198 (0.0005) [2023-12-26 20:25:48,481][105620] Updated weights for policy 1, policy_version 696208 (0.0005) [2023-12-26 20:25:49,015][105692] Updated weights for policy 0, policy_version 695403 (0.0009) [2023-12-26 20:25:49,066][105692] Updated weights for policy 0, policy_version 695413 (0.0009) [2023-12-26 20:25:49,072][105620] Updated weights for policy 1, policy_version 696218 (0.0006) [2023-12-26 20:25:49,116][105692] Updated weights for policy 0, policy_version 695423 (0.0007) [2023-12-26 20:25:49,122][105620] Updated weights for policy 1, policy_version 696228 (0.0007) [2023-12-26 20:25:49,176][105620] Updated weights for policy 1, policy_version 696238 (0.0005) [2023-12-26 20:25:49,253][105620] Updated weights for policy 1, policy_version 696248 (0.0007) [2023-12-26 20:25:49,870][105692] Updated weights for policy 0, policy_version 695433 (0.0008) [2023-12-26 20:25:49,931][105692] Updated weights for policy 0, policy_version 695443 (0.0009) [2023-12-26 20:25:49,991][105692] Updated weights for policy 0, policy_version 695453 (0.0010) [2023-12-26 20:25:50,047][105692] Updated weights for policy 0, policy_version 695463 (0.0011) [2023-12-26 20:25:50,061][105620] Updated weights for policy 1, policy_version 696258 (0.0006) [2023-12-26 20:25:50,122][105620] Updated weights for policy 1, policy_version 696268 (0.0006) [2023-12-26 20:25:50,178][105620] Updated weights for policy 1, policy_version 696278 (0.0008) [2023-12-26 20:25:50,762][105692] Updated weights for policy 0, policy_version 695473 (0.0009) [2023-12-26 20:25:50,829][105692] Updated weights for policy 0, policy_version 695483 (0.0009) [2023-12-26 20:25:50,890][105692] Updated weights for policy 0, policy_version 695493 (0.0008) [2023-12-26 20:25:50,929][105620] Updated weights for policy 1, policy_version 696288 (0.0008) [2023-12-26 20:25:50,987][105620] Updated weights for policy 1, policy_version 696298 (0.0009) [2023-12-26 20:25:51,051][105620] Updated weights for policy 1, policy_version 696308 (0.0009) [2023-12-26 20:25:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 356343808. Throughput: 0: 9697.1, 1: 9759.2. Samples: 356335352. Policy #0 lag: (min: 27.0, avg: 40.4, max: 59.0) [2023-12-26 20:25:51,062][104569] Avg episode reward: [(0, '5660.194'), (1, '9079.535')] [2023-12-26 20:25:51,609][105692] Updated weights for policy 0, policy_version 695503 (0.0007) [2023-12-26 20:25:51,677][105692] Updated weights for policy 0, policy_version 695513 (0.0008) [2023-12-26 20:25:51,745][105692] Updated weights for policy 0, policy_version 695523 (0.0008) [2023-12-26 20:25:51,898][105620] Updated weights for policy 1, policy_version 696318 (0.0010) [2023-12-26 20:25:51,969][105620] Updated weights for policy 1, policy_version 696328 (0.0010) [2023-12-26 20:25:52,033][105620] Updated weights for policy 1, policy_version 696338 (0.0009) [2023-12-26 20:25:52,317][105692] Updated weights for policy 0, policy_version 695533 (0.0009) [2023-12-26 20:25:52,383][105692] Updated weights for policy 0, policy_version 695543 (0.0007) [2023-12-26 20:25:52,441][105692] Updated weights for policy 0, policy_version 695553 (0.0008) [2023-12-26 20:25:52,722][105620] Updated weights for policy 1, policy_version 696348 (0.0007) [2023-12-26 20:25:52,786][105620] Updated weights for policy 1, policy_version 696358 (0.0005) [2023-12-26 20:25:52,843][105620] Updated weights for policy 1, policy_version 696368 (0.0005) [2023-12-26 20:25:53,113][105692] Updated weights for policy 0, policy_version 695563 (0.0007) [2023-12-26 20:25:53,167][105692] Updated weights for policy 0, policy_version 695573 (0.0009) [2023-12-26 20:25:53,221][105692] Updated weights for policy 0, policy_version 695583 (0.0009) [2023-12-26 20:25:53,436][105620] Updated weights for policy 1, policy_version 696378 (0.0006) [2023-12-26 20:25:53,490][105620] Updated weights for policy 1, policy_version 696388 (0.0009) [2023-12-26 20:25:53,535][105620] Updated weights for policy 1, policy_version 696398 (0.0008) [2023-12-26 20:25:53,586][105620] Updated weights for policy 1, policy_version 696408 (0.0009) [2023-12-26 20:25:53,896][105692] Updated weights for policy 0, policy_version 695593 (0.0009) [2023-12-26 20:25:53,945][105692] Updated weights for policy 0, policy_version 695603 (0.0007) [2023-12-26 20:25:53,999][105692] Updated weights for policy 0, policy_version 695613 (0.0007) [2023-12-26 20:25:54,047][105692] Updated weights for policy 0, policy_version 695623 (0.0009) [2023-12-26 20:25:54,423][105620] Updated weights for policy 1, policy_version 696418 (0.0009) [2023-12-26 20:25:54,469][105620] Updated weights for policy 1, policy_version 696428 (0.0008) [2023-12-26 20:25:54,517][105620] Updated weights for policy 1, policy_version 696438 (0.0009) [2023-12-26 20:25:54,760][105692] Updated weights for policy 0, policy_version 695633 (0.0009) [2023-12-26 20:25:54,810][105692] Updated weights for policy 0, policy_version 695643 (0.0008) [2023-12-26 20:25:54,861][105692] Updated weights for policy 0, policy_version 695653 (0.0006) [2023-12-26 20:25:55,302][105620] Updated weights for policy 1, policy_version 696448 (0.0009) [2023-12-26 20:25:55,352][105620] Updated weights for policy 1, policy_version 696458 (0.0008) [2023-12-26 20:25:55,399][105620] Updated weights for policy 1, policy_version 696468 (0.0008) [2023-12-26 20:25:55,559][105692] Updated weights for policy 0, policy_version 695663 (0.0005) [2023-12-26 20:25:55,605][105692] Updated weights for policy 0, policy_version 695673 (0.0005) [2023-12-26 20:25:55,657][105692] Updated weights for policy 0, policy_version 695683 (0.0009) [2023-12-26 20:25:56,034][105620] Updated weights for policy 1, policy_version 696478 (0.0009) [2023-12-26 20:25:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 356442112. Throughput: 0: 9696.3, 1: 9747.4. Samples: 356453792. Policy #0 lag: (min: 27.0, avg: 40.4, max: 59.0) [2023-12-26 20:25:56,063][104569] Avg episode reward: [(0, '6726.244'), (1, '9078.117')] [2023-12-26 20:25:56,093][105620] Updated weights for policy 1, policy_version 696488 (0.0008) [2023-12-26 20:25:56,156][105620] Updated weights for policy 1, policy_version 696498 (0.0009) [2023-12-26 20:25:56,360][105692] Updated weights for policy 0, policy_version 695694 (0.0009) [2023-12-26 20:25:56,410][105692] Updated weights for policy 0, policy_version 695704 (0.0009) [2023-12-26 20:25:56,472][105692] Updated weights for policy 0, policy_version 695714 (0.0009) [2023-12-26 20:25:56,925][105620] Updated weights for policy 1, policy_version 696508 (0.0008) [2023-12-26 20:25:56,991][105620] Updated weights for policy 1, policy_version 696518 (0.0005) [2023-12-26 20:25:57,045][105620] Updated weights for policy 1, policy_version 696528 (0.0007) [2023-12-26 20:25:57,156][105692] Updated weights for policy 0, policy_version 695724 (0.0008) [2023-12-26 20:25:57,225][105692] Updated weights for policy 0, policy_version 695734 (0.0005) [2023-12-26 20:25:57,289][105692] Updated weights for policy 0, policy_version 695744 (0.0005) [2023-12-26 20:25:57,651][105620] Updated weights for policy 1, policy_version 696538 (0.0010) [2023-12-26 20:25:57,699][105620] Updated weights for policy 1, policy_version 696548 (0.0010) [2023-12-26 20:25:57,751][105620] Updated weights for policy 1, policy_version 696558 (0.0006) [2023-12-26 20:25:57,797][105620] Updated weights for policy 1, policy_version 696568 (0.0006) [2023-12-26 20:25:57,831][105692] Updated weights for policy 0, policy_version 695754 (0.0006) [2023-12-26 20:25:57,891][105692] Updated weights for policy 0, policy_version 695764 (0.0008) [2023-12-26 20:25:57,938][105692] Updated weights for policy 0, policy_version 695774 (0.0008) [2023-12-26 20:25:57,981][105692] Updated weights for policy 0, policy_version 695784 (0.0007) [2023-12-26 20:25:58,504][105620] Updated weights for policy 1, policy_version 696578 (0.0010) [2023-12-26 20:25:58,570][105620] Updated weights for policy 1, policy_version 696588 (0.0008) [2023-12-26 20:25:58,637][105620] Updated weights for policy 1, policy_version 696598 (0.0011) [2023-12-26 20:25:58,779][105692] Updated weights for policy 0, policy_version 695794 (0.0008) [2023-12-26 20:25:58,847][105692] Updated weights for policy 0, policy_version 695804 (0.0009) [2023-12-26 20:25:58,928][105692] Updated weights for policy 0, policy_version 695814 (0.0008) [2023-12-26 20:25:59,459][105620] Updated weights for policy 1, policy_version 696608 (0.0010) [2023-12-26 20:25:59,520][105620] Updated weights for policy 1, policy_version 696618 (0.0011) [2023-12-26 20:25:59,583][105620] Updated weights for policy 1, policy_version 696628 (0.0011) [2023-12-26 20:25:59,690][105692] Updated weights for policy 0, policy_version 695824 (0.0006) [2023-12-26 20:25:59,736][105692] Updated weights for policy 0, policy_version 695834 (0.0005) [2023-12-26 20:25:59,785][105692] Updated weights for policy 0, policy_version 695844 (0.0005) [2023-12-26 20:26:00,338][105620] Updated weights for policy 1, policy_version 696638 (0.0011) [2023-12-26 20:26:00,393][105620] Updated weights for policy 1, policy_version 696648 (0.0011) [2023-12-26 20:26:00,436][105692] Updated weights for policy 0, policy_version 695854 (0.0006) [2023-12-26 20:26:00,446][105620] Updated weights for policy 1, policy_version 696658 (0.0011) [2023-12-26 20:26:00,495][105692] Updated weights for policy 0, policy_version 695864 (0.0006) [2023-12-26 20:26:00,559][105692] Updated weights for policy 0, policy_version 695874 (0.0008) [2023-12-26 20:26:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 356540416. Throughput: 0: 9757.3, 1: 9790.2. Samples: 356513232. Policy #0 lag: (min: 27.0, avg: 40.4, max: 59.0) [2023-12-26 20:26:01,062][104569] Avg episode reward: [(0, '8148.893'), (1, '9078.765')] [2023-12-26 20:26:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000695880_178176000.pth... [2023-12-26 20:26:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000696664_178364416.pth... [2023-12-26 20:26:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000694728_177881088.pth [2023-12-26 20:26:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000695544_178077696.pth [2023-12-26 20:26:01,150][105692] Updated weights for policy 0, policy_version 695884 (0.0007) [2023-12-26 20:26:01,207][105692] Updated weights for policy 0, policy_version 695894 (0.0008) [2023-12-26 20:26:01,226][105620] Updated weights for policy 1, policy_version 696668 (0.0010) [2023-12-26 20:26:01,266][105692] Updated weights for policy 0, policy_version 695904 (0.0008) [2023-12-26 20:26:01,285][105620] Updated weights for policy 1, policy_version 696678 (0.0011) [2023-12-26 20:26:01,340][105620] Updated weights for policy 1, policy_version 696688 (0.0011) [2023-12-26 20:26:02,013][105692] Updated weights for policy 0, policy_version 695914 (0.0008) [2023-12-26 20:26:02,065][105692] Updated weights for policy 0, policy_version 695924 (0.0010) [2023-12-26 20:26:02,078][105620] Updated weights for policy 1, policy_version 696698 (0.0009) [2023-12-26 20:26:02,125][105692] Updated weights for policy 0, policy_version 695934 (0.0006) [2023-12-26 20:26:02,138][105620] Updated weights for policy 1, policy_version 696708 (0.0011) [2023-12-26 20:26:02,188][105692] Updated weights for policy 0, policy_version 695944 (0.0006) [2023-12-26 20:26:02,208][105620] Updated weights for policy 1, policy_version 696718 (0.0010) [2023-12-26 20:26:02,268][105620] Updated weights for policy 1, policy_version 696728 (0.0010) [2023-12-26 20:26:02,925][105692] Updated weights for policy 0, policy_version 695954 (0.0008) [2023-12-26 20:26:02,978][105692] Updated weights for policy 0, policy_version 695964 (0.0007) [2023-12-26 20:26:02,991][105620] Updated weights for policy 1, policy_version 696738 (0.0010) [2023-12-26 20:26:03,034][105692] Updated weights for policy 0, policy_version 695974 (0.0006) [2023-12-26 20:26:03,048][105620] Updated weights for policy 1, policy_version 696748 (0.0010) [2023-12-26 20:26:03,095][105620] Updated weights for policy 1, policy_version 696758 (0.0010) [2023-12-26 20:26:03,670][105620] Updated weights for policy 1, policy_version 696768 (0.0006) [2023-12-26 20:26:03,724][105620] Updated weights for policy 1, policy_version 696778 (0.0005) [2023-12-26 20:26:03,774][105620] Updated weights for policy 1, policy_version 696788 (0.0005) [2023-12-26 20:26:03,906][105692] Updated weights for policy 0, policy_version 695984 (0.0008) [2023-12-26 20:26:03,965][105692] Updated weights for policy 0, policy_version 695994 (0.0009) [2023-12-26 20:26:04,018][105692] Updated weights for policy 0, policy_version 696004 (0.0010) [2023-12-26 20:26:04,382][105620] Updated weights for policy 1, policy_version 696798 (0.0009) [2023-12-26 20:26:04,445][105620] Updated weights for policy 1, policy_version 696808 (0.0009) [2023-12-26 20:26:04,517][105620] Updated weights for policy 1, policy_version 696818 (0.0007) [2023-12-26 20:26:04,827][105692] Updated weights for policy 0, policy_version 696014 (0.0009) [2023-12-26 20:26:04,883][105692] Updated weights for policy 0, policy_version 696024 (0.0009) [2023-12-26 20:26:04,933][105692] Updated weights for policy 0, policy_version 696034 (0.0009) [2023-12-26 20:26:05,262][105620] Updated weights for policy 1, policy_version 696828 (0.0008) [2023-12-26 20:26:05,308][105620] Updated weights for policy 1, policy_version 696838 (0.0005) [2023-12-26 20:26:05,354][105620] Updated weights for policy 1, policy_version 696848 (0.0005) [2023-12-26 20:26:05,608][105692] Updated weights for policy 0, policy_version 696044 (0.0008) [2023-12-26 20:26:05,662][105692] Updated weights for policy 0, policy_version 696054 (0.0009) [2023-12-26 20:26:05,726][105692] Updated weights for policy 0, policy_version 696064 (0.0005) [2023-12-26 20:26:05,907][105620] Updated weights for policy 1, policy_version 696858 (0.0006) [2023-12-26 20:26:05,961][105620] Updated weights for policy 1, policy_version 696868 (0.0009) [2023-12-26 20:26:06,019][105620] Updated weights for policy 1, policy_version 696878 (0.0010) [2023-12-26 20:26:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 356638720. Throughput: 0: 9741.2, 1: 9777.4. Samples: 356628476. Policy #0 lag: (min: 27.0, avg: 40.4, max: 59.0) [2023-12-26 20:26:06,062][104569] Avg episode reward: [(0, '8790.127'), (1, '9169.238')] [2023-12-26 20:26:06,080][105620] Updated weights for policy 1, policy_version 696888 (0.0008) [2023-12-26 20:26:06,308][105692] Updated weights for policy 0, policy_version 696074 (0.0006) [2023-12-26 20:26:06,368][105692] Updated weights for policy 0, policy_version 696084 (0.0006) [2023-12-26 20:26:06,430][105692] Updated weights for policy 0, policy_version 696094 (0.0005) [2023-12-26 20:26:06,495][105692] Updated weights for policy 0, policy_version 696104 (0.0007) [2023-12-26 20:26:06,909][105620] Updated weights for policy 1, policy_version 696898 (0.0008) [2023-12-26 20:26:06,971][105620] Updated weights for policy 1, policy_version 696908 (0.0006) [2023-12-26 20:26:07,037][105620] Updated weights for policy 1, policy_version 696918 (0.0005) [2023-12-26 20:26:07,131][105692] Updated weights for policy 0, policy_version 696114 (0.0008) [2023-12-26 20:26:07,197][105692] Updated weights for policy 0, policy_version 696124 (0.0005) [2023-12-26 20:26:07,267][105692] Updated weights for policy 0, policy_version 696134 (0.0005) [2023-12-26 20:26:07,665][105620] Updated weights for policy 1, policy_version 696928 (0.0005) [2023-12-26 20:26:07,721][105620] Updated weights for policy 1, policy_version 696938 (0.0005) [2023-12-26 20:26:07,774][105620] Updated weights for policy 1, policy_version 696948 (0.0006) [2023-12-26 20:26:07,825][105692] Updated weights for policy 0, policy_version 696144 (0.0009) [2023-12-26 20:26:07,879][105692] Updated weights for policy 0, policy_version 696154 (0.0010) [2023-12-26 20:26:07,937][105692] Updated weights for policy 0, policy_version 696164 (0.0010) [2023-12-26 20:26:08,475][105620] Updated weights for policy 1, policy_version 696958 (0.0007) [2023-12-26 20:26:08,527][105620] Updated weights for policy 1, policy_version 696968 (0.0008) [2023-12-26 20:26:08,586][105620] Updated weights for policy 1, policy_version 696978 (0.0008) [2023-12-26 20:26:08,705][105692] Updated weights for policy 0, policy_version 696174 (0.0010) [2023-12-26 20:26:08,771][105692] Updated weights for policy 0, policy_version 696184 (0.0011) [2023-12-26 20:26:08,824][105692] Updated weights for policy 0, policy_version 696194 (0.0011) [2023-12-26 20:26:09,297][105620] Updated weights for policy 1, policy_version 696988 (0.0006) [2023-12-26 20:26:09,355][105620] Updated weights for policy 1, policy_version 696998 (0.0009) [2023-12-26 20:26:09,417][105620] Updated weights for policy 1, policy_version 697008 (0.0008) [2023-12-26 20:26:09,533][105692] Updated weights for policy 0, policy_version 696204 (0.0005) [2023-12-26 20:26:09,586][105692] Updated weights for policy 0, policy_version 696214 (0.0007) [2023-12-26 20:26:09,634][105692] Updated weights for policy 0, policy_version 696224 (0.0009) [2023-12-26 20:26:10,197][105620] Updated weights for policy 1, policy_version 697018 (0.0009) [2023-12-26 20:26:10,237][105586] KL-divergence is very high: 352.1156 [2023-12-26 20:26:10,256][105620] Updated weights for policy 1, policy_version 697028 (0.0007) [2023-12-26 20:26:10,287][105586] KL-divergence is very high: 567.7975 [2023-12-26 20:26:10,314][105620] Updated weights for policy 1, policy_version 697038 (0.0008) [2023-12-26 20:26:10,332][105586] KL-divergence is very high: 475.2708 [2023-12-26 20:26:10,377][105620] Updated weights for policy 1, policy_version 697048 (0.0007) [2023-12-26 20:26:10,413][105692] Updated weights for policy 0, policy_version 696234 (0.0009) [2023-12-26 20:26:10,478][105692] Updated weights for policy 0, policy_version 696244 (0.0010) [2023-12-26 20:26:10,538][105692] Updated weights for policy 0, policy_version 696254 (0.0008) [2023-12-26 20:26:10,593][105692] Updated weights for policy 0, policy_version 696264 (0.0010) [2023-12-26 20:26:11,036][105620] Updated weights for policy 1, policy_version 697058 (0.0008) [2023-12-26 20:26:11,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 356737024. Throughput: 0: 9769.7, 1: 9868.0. Samples: 356749372. Policy #0 lag: (min: 27.0, avg: 40.4, max: 59.0) [2023-12-26 20:26:11,063][104569] Avg episode reward: [(0, '9257.863'), (1, '9168.466')] [2023-12-26 20:26:11,104][105620] Updated weights for policy 1, policy_version 697068 (0.0007) [2023-12-26 20:26:11,175][105620] Updated weights for policy 1, policy_version 697078 (0.0007) [2023-12-26 20:26:11,349][105692] Updated weights for policy 0, policy_version 696274 (0.0009) [2023-12-26 20:26:11,416][105692] Updated weights for policy 0, policy_version 696284 (0.0009) [2023-12-26 20:26:11,472][105692] Updated weights for policy 0, policy_version 696294 (0.0008) [2023-12-26 20:26:11,981][105620] Updated weights for policy 1, policy_version 697088 (0.0009) [2023-12-26 20:26:12,028][105620] Updated weights for policy 1, policy_version 697098 (0.0009) [2023-12-26 20:26:12,078][105620] Updated weights for policy 1, policy_version 697108 (0.0005) [2023-12-26 20:26:12,204][105692] Updated weights for policy 0, policy_version 696304 (0.0010) [2023-12-26 20:26:12,258][105692] Updated weights for policy 0, policy_version 696314 (0.0009) [2023-12-26 20:26:12,315][105692] Updated weights for policy 0, policy_version 696324 (0.0009) [2023-12-26 20:26:12,839][105620] Updated weights for policy 1, policy_version 697118 (0.0010) [2023-12-26 20:26:12,895][105620] Updated weights for policy 1, policy_version 697128 (0.0010) [2023-12-26 20:26:12,950][105620] Updated weights for policy 1, policy_version 697138 (0.0010) [2023-12-26 20:26:13,055][105692] Updated weights for policy 0, policy_version 696334 (0.0008) [2023-12-26 20:26:13,110][105692] Updated weights for policy 0, policy_version 696344 (0.0008) [2023-12-26 20:26:13,153][105692] Updated weights for policy 0, policy_version 696354 (0.0007) [2023-12-26 20:26:13,667][105620] Updated weights for policy 1, policy_version 697148 (0.0010) [2023-12-26 20:26:13,712][105620] Updated weights for policy 1, policy_version 697158 (0.0010) [2023-12-26 20:26:13,754][105692] Updated weights for policy 0, policy_version 696364 (0.0005) [2023-12-26 20:26:13,757][105620] Updated weights for policy 1, policy_version 697168 (0.0006) [2023-12-26 20:26:13,807][105692] Updated weights for policy 0, policy_version 696374 (0.0007) [2023-12-26 20:26:13,858][105692] Updated weights for policy 0, policy_version 696384 (0.0007) [2023-12-26 20:26:14,513][105620] Updated weights for policy 1, policy_version 697178 (0.0009) [2023-12-26 20:26:14,514][105692] Updated weights for policy 0, policy_version 696394 (0.0007) [2023-12-26 20:26:14,565][105620] Updated weights for policy 1, policy_version 697188 (0.0010) [2023-12-26 20:26:14,567][105692] Updated weights for policy 0, policy_version 696404 (0.0006) [2023-12-26 20:26:14,613][105620] Updated weights for policy 1, policy_version 697198 (0.0010) [2023-12-26 20:26:14,615][105692] Updated weights for policy 0, policy_version 696414 (0.0005) [2023-12-26 20:26:14,673][105620] Updated weights for policy 1, policy_version 697208 (0.0011) [2023-12-26 20:26:14,675][105692] Updated weights for policy 0, policy_version 696424 (0.0007) [2023-12-26 20:26:15,310][105692] Updated weights for policy 0, policy_version 696434 (0.0010) [2023-12-26 20:26:15,370][105692] Updated weights for policy 0, policy_version 696444 (0.0006) [2023-12-26 20:26:15,383][105620] Updated weights for policy 1, policy_version 697218 (0.0009) [2023-12-26 20:26:15,429][105692] Updated weights for policy 0, policy_version 696454 (0.0010) [2023-12-26 20:26:15,441][105620] Updated weights for policy 1, policy_version 697228 (0.0010) [2023-12-26 20:26:15,510][105620] Updated weights for policy 1, policy_version 697238 (0.0010) [2023-12-26 20:26:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 356835328. Throughput: 0: 9776.1, 1: 9878.8. Samples: 356807376. Policy #0 lag: (min: 27.0, avg: 40.4, max: 59.0) [2023-12-26 20:26:16,062][104569] Avg episode reward: [(0, '9257.955'), (1, '9169.497')] [2023-12-26 20:26:16,106][105692] Updated weights for policy 0, policy_version 696464 (0.0010) [2023-12-26 20:26:16,109][105620] Updated weights for policy 1, policy_version 697248 (0.0010) [2023-12-26 20:26:16,168][105620] Updated weights for policy 1, policy_version 697258 (0.0007) [2023-12-26 20:26:16,169][105692] Updated weights for policy 0, policy_version 696474 (0.0011) [2023-12-26 20:26:16,217][105692] Updated weights for policy 0, policy_version 696484 (0.0010) [2023-12-26 20:26:16,224][105620] Updated weights for policy 1, policy_version 697268 (0.0006) [2023-12-26 20:26:16,233][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000696488_178331648.pth... [2023-12-26 20:26:16,236][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000695304_178028544.pth [2023-12-26 20:26:16,242][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000697272_178520064.pth... [2023-12-26 20:26:16,245][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000696120_178225152.pth [2023-12-26 20:26:16,942][105692] Updated weights for policy 0, policy_version 696494 (0.0010) [2023-12-26 20:26:16,971][105620] Updated weights for policy 1, policy_version 697278 (0.0008) [2023-12-26 20:26:17,007][105692] Updated weights for policy 0, policy_version 696504 (0.0010) [2023-12-26 20:26:17,025][105620] Updated weights for policy 1, policy_version 697288 (0.0008) [2023-12-26 20:26:17,073][105692] Updated weights for policy 0, policy_version 696514 (0.0011) [2023-12-26 20:26:17,083][105620] Updated weights for policy 1, policy_version 697298 (0.0009) [2023-12-26 20:26:17,736][105692] Updated weights for policy 0, policy_version 696524 (0.0010) [2023-12-26 20:26:17,774][105620] Updated weights for policy 1, policy_version 697308 (0.0006) [2023-12-26 20:26:17,791][105692] Updated weights for policy 0, policy_version 696534 (0.0010) [2023-12-26 20:26:17,822][105620] Updated weights for policy 1, policy_version 697318 (0.0005) [2023-12-26 20:26:17,840][105692] Updated weights for policy 0, policy_version 696544 (0.0010) [2023-12-26 20:26:17,878][105620] Updated weights for policy 1, policy_version 697328 (0.0005) [2023-12-26 20:26:18,466][105692] Updated weights for policy 0, policy_version 696554 (0.0010) [2023-12-26 20:26:18,522][105692] Updated weights for policy 0, policy_version 696564 (0.0011) [2023-12-26 20:26:18,581][105692] Updated weights for policy 0, policy_version 696574 (0.0010) [2023-12-26 20:26:18,646][105692] Updated weights for policy 0, policy_version 696584 (0.0011) [2023-12-26 20:26:18,701][105620] Updated weights for policy 1, policy_version 697338 (0.0008) [2023-12-26 20:26:18,753][105620] Updated weights for policy 1, policy_version 697348 (0.0008) [2023-12-26 20:26:18,797][105620] Updated weights for policy 1, policy_version 697358 (0.0008) [2023-12-26 20:26:18,853][105620] Updated weights for policy 1, policy_version 697368 (0.0008) [2023-12-26 20:26:19,403][105692] Updated weights for policy 0, policy_version 696594 (0.0011) [2023-12-26 20:26:19,455][105692] Updated weights for policy 0, policy_version 696604 (0.0010) [2023-12-26 20:26:19,518][105692] Updated weights for policy 0, policy_version 696614 (0.0011) [2023-12-26 20:26:19,643][105620] Updated weights for policy 1, policy_version 697378 (0.0008) [2023-12-26 20:26:19,703][105620] Updated weights for policy 1, policy_version 697388 (0.0008) [2023-12-26 20:26:19,762][105620] Updated weights for policy 1, policy_version 697398 (0.0008) [2023-12-26 20:26:20,290][105692] Updated weights for policy 0, policy_version 696624 (0.0010) [2023-12-26 20:26:20,354][105692] Updated weights for policy 0, policy_version 696634 (0.0009) [2023-12-26 20:26:20,410][105692] Updated weights for policy 0, policy_version 696644 (0.0009) [2023-12-26 20:26:20,539][105620] Updated weights for policy 1, policy_version 697408 (0.0006) [2023-12-26 20:26:20,602][105620] Updated weights for policy 1, policy_version 697418 (0.0006) [2023-12-26 20:26:20,669][105620] Updated weights for policy 1, policy_version 697428 (0.0008) [2023-12-26 20:26:21,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 356933632. Throughput: 0: 9911.7, 1: 9682.7. Samples: 356925316. Policy #0 lag: (min: 27.0, avg: 40.4, max: 59.0) [2023-12-26 20:26:21,062][104569] Avg episode reward: [(0, '9349.447'), (1, '9171.204')] [2023-12-26 20:26:21,145][105692] Updated weights for policy 0, policy_version 696654 (0.0008) [2023-12-26 20:26:21,208][105692] Updated weights for policy 0, policy_version 696664 (0.0010) [2023-12-26 20:26:21,276][105692] Updated weights for policy 0, policy_version 696674 (0.0011) [2023-12-26 20:26:21,375][105620] Updated weights for policy 1, policy_version 697438 (0.0008) [2023-12-26 20:26:21,438][105620] Updated weights for policy 1, policy_version 697448 (0.0006) [2023-12-26 20:26:21,500][105620] Updated weights for policy 1, policy_version 697458 (0.0009) [2023-12-26 20:26:22,069][105692] Updated weights for policy 0, policy_version 696684 (0.0010) [2023-12-26 20:26:22,131][105692] Updated weights for policy 0, policy_version 696694 (0.0009) [2023-12-26 20:26:22,191][105692] Updated weights for policy 0, policy_version 696704 (0.0010) [2023-12-26 20:26:22,207][105620] Updated weights for policy 1, policy_version 697468 (0.0008) [2023-12-26 20:26:22,260][105620] Updated weights for policy 1, policy_version 697478 (0.0008) [2023-12-26 20:26:22,307][105620] Updated weights for policy 1, policy_version 697488 (0.0006) [2023-12-26 20:26:22,967][105620] Updated weights for policy 1, policy_version 697498 (0.0010) [2023-12-26 20:26:22,974][105692] Updated weights for policy 0, policy_version 696714 (0.0008) [2023-12-26 20:26:23,026][105620] Updated weights for policy 1, policy_version 697508 (0.0010) [2023-12-26 20:26:23,034][105692] Updated weights for policy 0, policy_version 696724 (0.0007) [2023-12-26 20:26:23,086][105620] Updated weights for policy 1, policy_version 697518 (0.0011) [2023-12-26 20:26:23,088][105692] Updated weights for policy 0, policy_version 696734 (0.0007) [2023-12-26 20:26:23,141][105620] Updated weights for policy 1, policy_version 697528 (0.0011) [2023-12-26 20:26:23,149][105692] Updated weights for policy 0, policy_version 696744 (0.0006) [2023-12-26 20:26:23,701][105692] Updated weights for policy 0, policy_version 696754 (0.0005) [2023-12-26 20:26:23,760][105692] Updated weights for policy 0, policy_version 696764 (0.0005) [2023-12-26 20:26:23,805][105620] Updated weights for policy 1, policy_version 697538 (0.0006) [2023-12-26 20:26:23,816][105692] Updated weights for policy 0, policy_version 696774 (0.0006) [2023-12-26 20:26:23,866][105620] Updated weights for policy 1, policy_version 697548 (0.0005) [2023-12-26 20:26:23,914][105620] Updated weights for policy 1, policy_version 697558 (0.0005) [2023-12-26 20:26:24,422][105692] Updated weights for policy 0, policy_version 696784 (0.0006) [2023-12-26 20:26:24,477][105692] Updated weights for policy 0, policy_version 696794 (0.0005) [2023-12-26 20:26:24,516][105620] Updated weights for policy 1, policy_version 697568 (0.0007) [2023-12-26 20:26:24,525][105692] Updated weights for policy 0, policy_version 696804 (0.0005) [2023-12-26 20:26:24,570][105620] Updated weights for policy 1, policy_version 697578 (0.0008) [2023-12-26 20:26:24,621][105620] Updated weights for policy 1, policy_version 697588 (0.0009) [2023-12-26 20:26:25,056][105692] Updated weights for policy 0, policy_version 696814 (0.0005) [2023-12-26 20:26:25,103][105692] Updated weights for policy 0, policy_version 696824 (0.0005) [2023-12-26 20:26:25,146][105692] Updated weights for policy 0, policy_version 696834 (0.0005) [2023-12-26 20:26:25,301][105620] Updated weights for policy 1, policy_version 697598 (0.0010) [2023-12-26 20:26:25,353][105620] Updated weights for policy 1, policy_version 697608 (0.0010) [2023-12-26 20:26:25,411][105620] Updated weights for policy 1, policy_version 697618 (0.0010) [2023-12-26 20:26:25,809][105692] Updated weights for policy 0, policy_version 696844 (0.0007) [2023-12-26 20:26:25,866][105692] Updated weights for policy 0, policy_version 696855 (0.0010) [2023-12-26 20:26:25,919][105692] Updated weights for policy 0, policy_version 696867 (0.0010) [2023-12-26 20:26:26,053][105620] Updated weights for policy 1, policy_version 697628 (0.0010) [2023-12-26 20:26:26,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 357040128. Throughput: 0: 9969.7, 1: 9778.5. Samples: 357048916. Policy #0 lag: (min: 27.0, avg: 40.4, max: 59.0) [2023-12-26 20:26:26,063][104569] Avg episode reward: [(0, '9167.434'), (1, '9173.123')] [2023-12-26 20:26:26,108][105620] Updated weights for policy 1, policy_version 697638 (0.0009) [2023-12-26 20:26:26,156][105620] Updated weights for policy 1, policy_version 697648 (0.0009) [2023-12-26 20:26:26,714][105692] Updated weights for policy 0, policy_version 696877 (0.0010) [2023-12-26 20:26:26,772][105692] Updated weights for policy 0, policy_version 696887 (0.0010) [2023-12-26 20:26:26,827][105692] Updated weights for policy 0, policy_version 696897 (0.0010) [2023-12-26 20:26:26,904][105620] Updated weights for policy 1, policy_version 697658 (0.0009) [2023-12-26 20:26:26,948][105620] Updated weights for policy 1, policy_version 697668 (0.0007) [2023-12-26 20:26:27,001][105620] Updated weights for policy 1, policy_version 697678 (0.0006) [2023-12-26 20:26:27,051][105620] Updated weights for policy 1, policy_version 697688 (0.0009) [2023-12-26 20:26:27,515][105692] Updated weights for policy 0, policy_version 696907 (0.0010) [2023-12-26 20:26:27,559][105692] Updated weights for policy 0, policy_version 696917 (0.0010) [2023-12-26 20:26:27,607][105692] Updated weights for policy 0, policy_version 696927 (0.0010) [2023-12-26 20:26:27,707][105620] Updated weights for policy 1, policy_version 697698 (0.0007) [2023-12-26 20:26:27,765][105620] Updated weights for policy 1, policy_version 697708 (0.0006) [2023-12-26 20:26:27,813][105620] Updated weights for policy 1, policy_version 697718 (0.0005) [2023-12-26 20:26:28,333][105692] Updated weights for policy 0, policy_version 696937 (0.0010) [2023-12-26 20:26:28,393][105692] Updated weights for policy 0, policy_version 696947 (0.0008) [2023-12-26 20:26:28,442][105692] Updated weights for policy 0, policy_version 696957 (0.0008) [2023-12-26 20:26:28,489][105692] Updated weights for policy 0, policy_version 696967 (0.0005) [2023-12-26 20:26:28,555][105620] Updated weights for policy 1, policy_version 697728 (0.0008) [2023-12-26 20:26:28,612][105620] Updated weights for policy 1, policy_version 697738 (0.0010) [2023-12-26 20:26:28,674][105620] Updated weights for policy 1, policy_version 697748 (0.0010) [2023-12-26 20:26:29,026][105692] Updated weights for policy 0, policy_version 696977 (0.0010) [2023-12-26 20:26:29,070][105692] Updated weights for policy 0, policy_version 696987 (0.0010) [2023-12-26 20:26:29,120][105692] Updated weights for policy 0, policy_version 696997 (0.0010) [2023-12-26 20:26:29,522][105620] Updated weights for policy 1, policy_version 697758 (0.0008) [2023-12-26 20:26:29,566][105620] Updated weights for policy 1, policy_version 697768 (0.0008) [2023-12-26 20:26:29,615][105620] Updated weights for policy 1, policy_version 697778 (0.0008) [2023-12-26 20:26:29,818][105692] Updated weights for policy 0, policy_version 697007 (0.0008) [2023-12-26 20:26:29,884][105692] Updated weights for policy 0, policy_version 697017 (0.0011) [2023-12-26 20:26:29,952][105692] Updated weights for policy 0, policy_version 697027 (0.0010) [2023-12-26 20:26:30,338][105620] Updated weights for policy 1, policy_version 697788 (0.0007) [2023-12-26 20:26:30,394][105620] Updated weights for policy 1, policy_version 697798 (0.0007) [2023-12-26 20:26:30,444][105620] Updated weights for policy 1, policy_version 697808 (0.0009) [2023-12-26 20:26:30,645][105692] Updated weights for policy 0, policy_version 697037 (0.0007) [2023-12-26 20:26:30,696][105692] Updated weights for policy 0, policy_version 697047 (0.0005) [2023-12-26 20:26:30,743][105692] Updated weights for policy 0, policy_version 697057 (0.0005) [2023-12-26 20:26:31,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 357138432. Throughput: 0: 9978.6, 1: 9773.4. Samples: 357106752. Policy #0 lag: (min: 27.0, avg: 40.4, max: 59.0) [2023-12-26 20:26:31,063][104569] Avg episode reward: [(0, '6241.559'), (1, '9173.178')] [2023-12-26 20:26:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000697064_178479104.pth... [2023-12-26 20:26:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000695880_178176000.pth [2023-12-26 20:26:31,075][105620] Updated weights for policy 1, policy_version 697818 (0.0006) [2023-12-26 20:26:31,144][105620] Updated weights for policy 1, policy_version 697828 (0.0011) [2023-12-26 20:26:31,212][105620] Updated weights for policy 1, policy_version 697838 (0.0006) [2023-12-26 20:26:31,275][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000697848_178667520.pth... [2023-12-26 20:26:31,276][105620] Updated weights for policy 1, policy_version 697848 (0.0006) [2023-12-26 20:26:31,280][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000696664_178364416.pth [2023-12-26 20:26:31,315][105692] Updated weights for policy 0, policy_version 697067 (0.0005) [2023-12-26 20:26:31,378][105692] Updated weights for policy 0, policy_version 697077 (0.0007) [2023-12-26 20:26:31,439][105692] Updated weights for policy 0, policy_version 697087 (0.0006) [2023-12-26 20:26:31,964][105620] Updated weights for policy 1, policy_version 697858 (0.0008) [2023-12-26 20:26:32,018][105620] Updated weights for policy 1, policy_version 697868 (0.0010) [2023-12-26 20:26:32,071][105620] Updated weights for policy 1, policy_version 697878 (0.0009) [2023-12-26 20:26:32,126][105692] Updated weights for policy 0, policy_version 697097 (0.0008) [2023-12-26 20:26:32,191][105692] Updated weights for policy 0, policy_version 697107 (0.0009) [2023-12-26 20:26:32,251][105692] Updated weights for policy 0, policy_version 697117 (0.0008) [2023-12-26 20:26:32,309][105692] Updated weights for policy 0, policy_version 697127 (0.0009) [2023-12-26 20:26:32,751][105620] Updated weights for policy 1, policy_version 697888 (0.0007) [2023-12-26 20:26:32,799][105620] Updated weights for policy 1, policy_version 697898 (0.0009) [2023-12-26 20:26:32,846][105620] Updated weights for policy 1, policy_version 697908 (0.0009) [2023-12-26 20:26:33,125][105692] Updated weights for policy 0, policy_version 697137 (0.0009) [2023-12-26 20:26:33,179][105692] Updated weights for policy 0, policy_version 697147 (0.0008) [2023-12-26 20:26:33,237][105692] Updated weights for policy 0, policy_version 697157 (0.0007) [2023-12-26 20:26:33,523][105620] Updated weights for policy 1, policy_version 697919 (0.0010) [2023-12-26 20:26:33,594][105620] Updated weights for policy 1, policy_version 697929 (0.0006) [2023-12-26 20:26:33,660][105620] Updated weights for policy 1, policy_version 697939 (0.0005) [2023-12-26 20:26:33,949][105692] Updated weights for policy 0, policy_version 697167 (0.0008) [2023-12-26 20:26:34,002][105692] Updated weights for policy 0, policy_version 697177 (0.0010) [2023-12-26 20:26:34,052][105692] Updated weights for policy 0, policy_version 697187 (0.0009) [2023-12-26 20:26:34,205][105620] Updated weights for policy 1, policy_version 697949 (0.0007) [2023-12-26 20:26:34,263][105620] Updated weights for policy 1, policy_version 697959 (0.0009) [2023-12-26 20:26:34,321][105620] Updated weights for policy 1, policy_version 697969 (0.0006) [2023-12-26 20:26:34,788][105692] Updated weights for policy 0, policy_version 697197 (0.0005) [2023-12-26 20:26:34,858][105692] Updated weights for policy 0, policy_version 697207 (0.0006) [2023-12-26 20:26:34,922][105692] Updated weights for policy 0, policy_version 697217 (0.0008) [2023-12-26 20:26:35,105][105620] Updated weights for policy 1, policy_version 697979 (0.0008) [2023-12-26 20:26:35,168][105620] Updated weights for policy 1, policy_version 697989 (0.0009) [2023-12-26 20:26:35,234][105620] Updated weights for policy 1, policy_version 697999 (0.0008) [2023-12-26 20:26:35,601][105692] Updated weights for policy 0, policy_version 697227 (0.0008) [2023-12-26 20:26:35,648][105692] Updated weights for policy 0, policy_version 697237 (0.0008) [2023-12-26 20:26:35,699][105692] Updated weights for policy 0, policy_version 697247 (0.0009) [2023-12-26 20:26:35,923][105620] Updated weights for policy 1, policy_version 698009 (0.0006) [2023-12-26 20:26:35,992][105620] Updated weights for policy 1, policy_version 698019 (0.0005) [2023-12-26 20:26:36,050][105620] Updated weights for policy 1, policy_version 698029 (0.0005) [2023-12-26 20:26:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 357236736. Throughput: 0: 10049.5, 1: 9806.0. Samples: 357228852. Policy #0 lag: (min: 27.0, avg: 40.4, max: 59.0) [2023-12-26 20:26:36,063][104569] Avg episode reward: [(0, '1341.380'), (1, '9173.157')] [2023-12-26 20:26:36,116][105620] Updated weights for policy 1, policy_version 698039 (0.0006) [2023-12-26 20:26:36,598][105692] Updated weights for policy 0, policy_version 697257 (0.0009) [2023-12-26 20:26:36,617][105620] Updated weights for policy 1, policy_version 698049 (0.0006) [2023-12-26 20:26:36,656][105692] Updated weights for policy 0, policy_version 697267 (0.0009) [2023-12-26 20:26:36,683][105620] Updated weights for policy 1, policy_version 698059 (0.0007) [2023-12-26 20:26:36,713][105692] Updated weights for policy 0, policy_version 697277 (0.0009) [2023-12-26 20:26:36,740][105620] Updated weights for policy 1, policy_version 698069 (0.0007) [2023-12-26 20:26:36,768][105692] Updated weights for policy 0, policy_version 697287 (0.0008) [2023-12-26 20:26:37,322][105620] Updated weights for policy 1, policy_version 698079 (0.0008) [2023-12-26 20:26:37,371][105620] Updated weights for policy 1, policy_version 698089 (0.0009) [2023-12-26 20:26:37,435][105620] Updated weights for policy 1, policy_version 698099 (0.0008) [2023-12-26 20:26:37,599][105692] Updated weights for policy 0, policy_version 697297 (0.0009) [2023-12-26 20:26:37,662][105692] Updated weights for policy 0, policy_version 697307 (0.0009) [2023-12-26 20:26:37,722][105692] Updated weights for policy 0, policy_version 697317 (0.0009) [2023-12-26 20:26:38,210][105620] Updated weights for policy 1, policy_version 698109 (0.0009) [2023-12-26 20:26:38,268][105620] Updated weights for policy 1, policy_version 698119 (0.0009) [2023-12-26 20:26:38,332][105620] Updated weights for policy 1, policy_version 698129 (0.0009) [2023-12-26 20:26:38,476][105692] Updated weights for policy 0, policy_version 697327 (0.0007) [2023-12-26 20:26:38,535][105692] Updated weights for policy 0, policy_version 697337 (0.0009) [2023-12-26 20:26:38,605][105692] Updated weights for policy 0, policy_version 697347 (0.0009) [2023-12-26 20:26:39,119][105620] Updated weights for policy 1, policy_version 698139 (0.0009) [2023-12-26 20:26:39,169][105620] Updated weights for policy 1, policy_version 698150 (0.0010) [2023-12-26 20:26:39,219][105620] Updated weights for policy 1, policy_version 698160 (0.0009) [2023-12-26 20:26:39,240][105692] Updated weights for policy 0, policy_version 697357 (0.0008) [2023-12-26 20:26:39,309][105692] Updated weights for policy 0, policy_version 697367 (0.0008) [2023-12-26 20:26:39,374][105692] Updated weights for policy 0, policy_version 697377 (0.0009) [2023-12-26 20:26:40,041][105620] Updated weights for policy 1, policy_version 698170 (0.0007) [2023-12-26 20:26:40,092][105620] Updated weights for policy 1, policy_version 698180 (0.0008) [2023-12-26 20:26:40,140][105620] Updated weights for policy 1, policy_version 698190 (0.0009) [2023-12-26 20:26:40,143][105692] Updated weights for policy 0, policy_version 697387 (0.0009) [2023-12-26 20:26:40,190][105620] Updated weights for policy 1, policy_version 698200 (0.0006) [2023-12-26 20:26:40,208][105692] Updated weights for policy 0, policy_version 697397 (0.0009) [2023-12-26 20:26:40,270][105692] Updated weights for policy 0, policy_version 697407 (0.0009) [2023-12-26 20:26:40,964][105620] Updated weights for policy 1, policy_version 698210 (0.0008) [2023-12-26 20:26:41,001][105692] Updated weights for policy 0, policy_version 697417 (0.0009) [2023-12-26 20:26:41,021][105620] Updated weights for policy 1, policy_version 698220 (0.0008) [2023-12-26 20:26:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 357326848. Throughput: 0: 9911.4, 1: 9841.6. Samples: 357342676. Policy #0 lag: (min: 27.0, avg: 40.4, max: 59.0) [2023-12-26 20:26:41,063][104569] Avg episode reward: [(0, '2114.877'), (1, '9263.143')] [2023-12-26 20:26:41,063][105692] Updated weights for policy 0, policy_version 697427 (0.0008) [2023-12-26 20:26:41,085][105620] Updated weights for policy 1, policy_version 698230 (0.0007) [2023-12-26 20:26:41,126][105692] Updated weights for policy 0, policy_version 697437 (0.0010) [2023-12-26 20:26:41,185][105692] Updated weights for policy 0, policy_version 697447 (0.0010) [2023-12-26 20:26:41,848][105620] Updated weights for policy 1, policy_version 698240 (0.0008) [2023-12-26 20:26:41,910][105620] Updated weights for policy 1, policy_version 698250 (0.0008) [2023-12-26 20:26:41,974][105620] Updated weights for policy 1, policy_version 698260 (0.0008) [2023-12-26 20:26:41,979][105692] Updated weights for policy 0, policy_version 697457 (0.0008) [2023-12-26 20:26:42,040][105692] Updated weights for policy 0, policy_version 697467 (0.0009) [2023-12-26 20:26:42,100][105692] Updated weights for policy 0, policy_version 697477 (0.0008) [2023-12-26 20:26:42,741][105620] Updated weights for policy 1, policy_version 698270 (0.0006) [2023-12-26 20:26:42,802][105620] Updated weights for policy 1, policy_version 698280 (0.0005) [2023-12-26 20:26:42,853][105620] Updated weights for policy 1, policy_version 698290 (0.0006) [2023-12-26 20:26:42,862][105692] Updated weights for policy 0, policy_version 697487 (0.0008) [2023-12-26 20:26:42,923][105692] Updated weights for policy 0, policy_version 697497 (0.0008) [2023-12-26 20:26:42,981][105692] Updated weights for policy 0, policy_version 697507 (0.0009) [2023-12-26 20:26:43,557][105620] Updated weights for policy 1, policy_version 698300 (0.0008) [2023-12-26 20:26:43,606][105620] Updated weights for policy 1, policy_version 698310 (0.0009) [2023-12-26 20:26:43,650][105620] Updated weights for policy 1, policy_version 698320 (0.0005) [2023-12-26 20:26:43,744][105692] Updated weights for policy 0, policy_version 697517 (0.0009) [2023-12-26 20:26:43,806][105692] Updated weights for policy 0, policy_version 697527 (0.0009) [2023-12-26 20:26:43,854][105692] Updated weights for policy 0, policy_version 697537 (0.0009) [2023-12-26 20:26:44,384][105620] Updated weights for policy 1, policy_version 698330 (0.0007) [2023-12-26 20:26:44,438][105620] Updated weights for policy 1, policy_version 698340 (0.0009) [2023-12-26 20:26:44,489][105620] Updated weights for policy 1, policy_version 698350 (0.0009) [2023-12-26 20:26:44,535][105620] Updated weights for policy 1, policy_version 698360 (0.0009) [2023-12-26 20:26:44,555][105692] Updated weights for policy 0, policy_version 697547 (0.0009) [2023-12-26 20:26:44,606][105692] Updated weights for policy 0, policy_version 697557 (0.0008) [2023-12-26 20:26:44,656][105692] Updated weights for policy 0, policy_version 697567 (0.0008) [2023-12-26 20:26:45,260][105620] Updated weights for policy 1, policy_version 698370 (0.0008) [2023-12-26 20:26:45,309][105620] Updated weights for policy 1, policy_version 698380 (0.0009) [2023-12-26 20:26:45,364][105620] Updated weights for policy 1, policy_version 698390 (0.0009) [2023-12-26 20:26:45,449][105692] Updated weights for policy 0, policy_version 697577 (0.0009) [2023-12-26 20:26:45,500][105692] Updated weights for policy 0, policy_version 697587 (0.0009) [2023-12-26 20:26:45,548][105692] Updated weights for policy 0, policy_version 697597 (0.0009) [2023-12-26 20:26:45,604][105692] Updated weights for policy 0, policy_version 697607 (0.0009) [2023-12-26 20:26:46,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 357425152. Throughput: 0: 9839.9, 1: 9834.7. Samples: 357398588. Policy #0 lag: (min: 27.0, avg: 40.4, max: 59.0) [2023-12-26 20:26:46,062][104569] Avg episode reward: [(0, '6234.408'), (1, '9352.149')] [2023-12-26 20:26:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000697608_178618368.pth... [2023-12-26 20:26:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000698392_178806784.pth... [2023-12-26 20:26:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000697272_178520064.pth [2023-12-26 20:26:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000696488_178331648.pth [2023-12-26 20:26:46,190][105620] Updated weights for policy 1, policy_version 698400 (0.0007) [2023-12-26 20:26:46,260][105620] Updated weights for policy 1, policy_version 698410 (0.0006) [2023-12-26 20:26:46,276][105692] Updated weights for policy 0, policy_version 697617 (0.0006) [2023-12-26 20:26:46,317][105620] Updated weights for policy 1, policy_version 698420 (0.0009) [2023-12-26 20:26:46,336][105692] Updated weights for policy 0, policy_version 697627 (0.0006) [2023-12-26 20:26:46,393][105692] Updated weights for policy 0, policy_version 697637 (0.0005) [2023-12-26 20:26:46,996][105692] Updated weights for policy 0, policy_version 697647 (0.0007) [2023-12-26 20:26:47,054][105620] Updated weights for policy 1, policy_version 698430 (0.0009) [2023-12-26 20:26:47,061][105692] Updated weights for policy 0, policy_version 697657 (0.0009) [2023-12-26 20:26:47,101][105620] Updated weights for policy 1, policy_version 698440 (0.0006) [2023-12-26 20:26:47,118][105692] Updated weights for policy 0, policy_version 697667 (0.0008) [2023-12-26 20:26:47,158][105620] Updated weights for policy 1, policy_version 698450 (0.0007) [2023-12-26 20:26:47,833][105692] Updated weights for policy 0, policy_version 697677 (0.0006) [2023-12-26 20:26:47,894][105692] Updated weights for policy 0, policy_version 697687 (0.0005) [2023-12-26 20:26:47,946][105692] Updated weights for policy 0, policy_version 697697 (0.0005) [2023-12-26 20:26:47,953][105620] Updated weights for policy 1, policy_version 698460 (0.0009) [2023-12-26 20:26:48,003][105620] Updated weights for policy 1, policy_version 698470 (0.0009) [2023-12-26 20:26:48,056][105620] Updated weights for policy 1, policy_version 698481 (0.0010) [2023-12-26 20:26:48,621][105692] Updated weights for policy 0, policy_version 697707 (0.0007) [2023-12-26 20:26:48,674][105692] Updated weights for policy 0, policy_version 697717 (0.0010) [2023-12-26 20:26:48,723][105692] Updated weights for policy 0, policy_version 697727 (0.0010) [2023-12-26 20:26:48,764][105620] Updated weights for policy 1, policy_version 698491 (0.0008) [2023-12-26 20:26:48,821][105620] Updated weights for policy 1, policy_version 698501 (0.0010) [2023-12-26 20:26:48,877][105620] Updated weights for policy 1, policy_version 698511 (0.0009) [2023-12-26 20:26:49,395][105692] Updated weights for policy 0, policy_version 697737 (0.0007) [2023-12-26 20:26:49,460][105692] Updated weights for policy 0, policy_version 697747 (0.0010) [2023-12-26 20:26:49,518][105692] Updated weights for policy 0, policy_version 697757 (0.0010) [2023-12-26 20:26:49,584][105692] Updated weights for policy 0, policy_version 697767 (0.0011) [2023-12-26 20:26:49,665][105620] Updated weights for policy 1, policy_version 698521 (0.0010) [2023-12-26 20:26:49,713][105620] Updated weights for policy 1, policy_version 698531 (0.0008) [2023-12-26 20:26:49,776][105620] Updated weights for policy 1, policy_version 698541 (0.0008) [2023-12-26 20:26:49,825][105620] Updated weights for policy 1, policy_version 698551 (0.0008) [2023-12-26 20:26:50,279][105692] Updated weights for policy 0, policy_version 697777 (0.0009) [2023-12-26 20:26:50,342][105692] Updated weights for policy 0, policy_version 697787 (0.0008) [2023-12-26 20:26:50,405][105692] Updated weights for policy 0, policy_version 697797 (0.0009) [2023-12-26 20:26:50,662][105620] Updated weights for policy 1, policy_version 698561 (0.0010) [2023-12-26 20:26:50,728][105620] Updated weights for policy 1, policy_version 698571 (0.0010) [2023-12-26 20:26:50,793][105620] Updated weights for policy 1, policy_version 698581 (0.0006) [2023-12-26 20:26:51,054][105692] Updated weights for policy 0, policy_version 697807 (0.0007) [2023-12-26 20:26:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 357523456. Throughput: 0: 9921.8, 1: 9760.1. Samples: 357514164. Policy #0 lag: (min: 27.0, avg: 40.4, max: 59.0) [2023-12-26 20:26:51,063][104569] Avg episode reward: [(0, '1145.294'), (1, '9171.666')] [2023-12-26 20:26:51,116][105692] Updated weights for policy 0, policy_version 697817 (0.0006) [2023-12-26 20:26:51,176][105692] Updated weights for policy 0, policy_version 697827 (0.0007) [2023-12-26 20:26:51,558][105620] Updated weights for policy 1, policy_version 698591 (0.0008) [2023-12-26 20:26:51,628][105620] Updated weights for policy 1, policy_version 698601 (0.0007) [2023-12-26 20:26:51,687][105620] Updated weights for policy 1, policy_version 698611 (0.0007) [2023-12-26 20:26:51,831][105692] Updated weights for policy 0, policy_version 697837 (0.0006) [2023-12-26 20:26:51,893][105692] Updated weights for policy 0, policy_version 697847 (0.0006) [2023-12-26 20:26:51,958][105692] Updated weights for policy 0, policy_version 697857 (0.0006) [2023-12-26 20:26:52,497][105620] Updated weights for policy 1, policy_version 698621 (0.0010) [2023-12-26 20:26:52,562][105620] Updated weights for policy 1, policy_version 698631 (0.0010) [2023-12-26 20:26:52,564][105692] Updated weights for policy 0, policy_version 697867 (0.0007) [2023-12-26 20:26:52,620][105620] Updated weights for policy 1, policy_version 698641 (0.0007) [2023-12-26 20:26:52,629][105692] Updated weights for policy 0, policy_version 697877 (0.0005) [2023-12-26 20:26:52,698][105692] Updated weights for policy 0, policy_version 697887 (0.0006) [2023-12-26 20:26:53,183][105620] Updated weights for policy 1, policy_version 698651 (0.0006) [2023-12-26 20:26:53,247][105620] Updated weights for policy 1, policy_version 698661 (0.0005) [2023-12-26 20:26:53,304][105620] Updated weights for policy 1, policy_version 698671 (0.0006) [2023-12-26 20:26:53,452][105692] Updated weights for policy 0, policy_version 697897 (0.0006) [2023-12-26 20:26:53,510][105692] Updated weights for policy 0, policy_version 697907 (0.0010) [2023-12-26 20:26:53,572][105692] Updated weights for policy 0, policy_version 697917 (0.0010) [2023-12-26 20:26:53,624][105692] Updated weights for policy 0, policy_version 697927 (0.0010) [2023-12-26 20:26:53,979][105620] Updated weights for policy 1, policy_version 698681 (0.0006) [2023-12-26 20:26:54,037][105620] Updated weights for policy 1, policy_version 698691 (0.0005) [2023-12-26 20:26:54,092][105620] Updated weights for policy 1, policy_version 698701 (0.0006) [2023-12-26 20:26:54,149][105620] Updated weights for policy 1, policy_version 698711 (0.0005) [2023-12-26 20:26:54,286][105692] Updated weights for policy 0, policy_version 697937 (0.0011) [2023-12-26 20:26:54,346][105692] Updated weights for policy 0, policy_version 697947 (0.0011) [2023-12-26 20:26:54,405][105692] Updated weights for policy 0, policy_version 697957 (0.0010) [2023-12-26 20:26:54,732][105620] Updated weights for policy 1, policy_version 698721 (0.0010) [2023-12-26 20:26:54,793][105620] Updated weights for policy 1, policy_version 698731 (0.0011) [2023-12-26 20:26:54,844][105620] Updated weights for policy 1, policy_version 698741 (0.0010) [2023-12-26 20:26:55,017][105692] Updated weights for policy 0, policy_version 697967 (0.0010) [2023-12-26 20:26:55,079][105692] Updated weights for policy 0, policy_version 697977 (0.0009) [2023-12-26 20:26:55,141][105692] Updated weights for policy 0, policy_version 697987 (0.0009) [2023-12-26 20:26:55,501][105620] Updated weights for policy 1, policy_version 698751 (0.0011) [2023-12-26 20:26:55,563][105620] Updated weights for policy 1, policy_version 698761 (0.0010) [2023-12-26 20:26:55,627][105620] Updated weights for policy 1, policy_version 698771 (0.0010) [2023-12-26 20:26:55,929][105692] Updated weights for policy 0, policy_version 697997 (0.0009) [2023-12-26 20:26:55,980][105692] Updated weights for policy 0, policy_version 698007 (0.0008) [2023-12-26 20:26:56,039][105692] Updated weights for policy 0, policy_version 698017 (0.0008) [2023-12-26 20:26:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 357621760. Throughput: 0: 9921.3, 1: 9760.0. Samples: 357635028. Policy #0 lag: (min: 27.0, avg: 40.4, max: 59.0) [2023-12-26 20:26:56,062][104569] Avg episode reward: [(0, '1548.544'), (1, '9172.846')] [2023-12-26 20:26:56,328][105620] Updated weights for policy 1, policy_version 698781 (0.0008) [2023-12-26 20:26:56,387][105620] Updated weights for policy 1, policy_version 698791 (0.0005) [2023-12-26 20:26:56,445][105620] Updated weights for policy 1, policy_version 698801 (0.0005) [2023-12-26 20:26:56,781][105692] Updated weights for policy 0, policy_version 698027 (0.0007) [2023-12-26 20:26:56,843][105692] Updated weights for policy 0, policy_version 698037 (0.0005) [2023-12-26 20:26:56,910][105692] Updated weights for policy 0, policy_version 698047 (0.0006) [2023-12-26 20:26:57,015][105620] Updated weights for policy 1, policy_version 698811 (0.0007) [2023-12-26 20:26:57,071][105620] Updated weights for policy 1, policy_version 698821 (0.0010) [2023-12-26 20:26:57,128][105620] Updated weights for policy 1, policy_version 698831 (0.0010) [2023-12-26 20:26:57,490][105692] Updated weights for policy 0, policy_version 698057 (0.0008) [2023-12-26 20:26:57,559][105692] Updated weights for policy 0, policy_version 698067 (0.0009) [2023-12-26 20:26:57,619][105692] Updated weights for policy 0, policy_version 698077 (0.0008) [2023-12-26 20:26:57,667][105692] Updated weights for policy 0, policy_version 698087 (0.0008) [2023-12-26 20:26:57,877][105620] Updated weights for policy 1, policy_version 698841 (0.0010) [2023-12-26 20:26:57,938][105620] Updated weights for policy 1, policy_version 698851 (0.0010) [2023-12-26 20:26:57,992][105620] Updated weights for policy 1, policy_version 698861 (0.0010) [2023-12-26 20:26:58,047][105620] Updated weights for policy 1, policy_version 698871 (0.0006) [2023-12-26 20:26:58,452][105692] Updated weights for policy 0, policy_version 698097 (0.0009) [2023-12-26 20:26:58,506][105692] Updated weights for policy 0, policy_version 698107 (0.0009) [2023-12-26 20:26:58,572][105692] Updated weights for policy 0, policy_version 698117 (0.0007) [2023-12-26 20:26:58,778][105620] Updated weights for policy 1, policy_version 698881 (0.0009) [2023-12-26 20:26:58,856][105620] Updated weights for policy 1, policy_version 698891 (0.0012) [2023-12-26 20:26:58,928][105620] Updated weights for policy 1, policy_version 698901 (0.0009) [2023-12-26 20:26:59,410][105692] Updated weights for policy 0, policy_version 698127 (0.0009) [2023-12-26 20:26:59,470][105692] Updated weights for policy 0, policy_version 698137 (0.0009) [2023-12-26 20:26:59,525][105692] Updated weights for policy 0, policy_version 698147 (0.0008) [2023-12-26 20:26:59,732][105620] Updated weights for policy 1, policy_version 698911 (0.0008) [2023-12-26 20:26:59,787][105620] Updated weights for policy 1, policy_version 698921 (0.0006) [2023-12-26 20:26:59,850][105620] Updated weights for policy 1, policy_version 698931 (0.0008) [2023-12-26 20:27:00,259][105692] Updated weights for policy 0, policy_version 698157 (0.0007) [2023-12-26 20:27:00,326][105692] Updated weights for policy 0, policy_version 698167 (0.0005) [2023-12-26 20:27:00,384][105692] Updated weights for policy 0, policy_version 698177 (0.0009) [2023-12-26 20:27:00,545][105620] Updated weights for policy 1, policy_version 698941 (0.0007) [2023-12-26 20:27:00,602][105620] Updated weights for policy 1, policy_version 698951 (0.0009) [2023-12-26 20:27:00,650][105620] Updated weights for policy 1, policy_version 698961 (0.0009) [2023-12-26 20:27:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 357720064. Throughput: 0: 9905.2, 1: 9798.0. Samples: 357694016. Policy #0 lag: (min: 27.0, avg: 40.4, max: 59.0) [2023-12-26 20:27:01,062][104569] Avg episode reward: [(0, '6513.245'), (1, '9174.142')] [2023-12-26 20:27:01,063][105692] Updated weights for policy 0, policy_version 698187 (0.0009) [2023-12-26 20:27:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000698968_178954240.pth... [2023-12-26 20:27:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000697848_178667520.pth [2023-12-26 20:27:01,128][105692] Updated weights for policy 0, policy_version 698197 (0.0008) [2023-12-26 20:27:01,186][105692] Updated weights for policy 0, policy_version 698207 (0.0009) [2023-12-26 20:27:01,236][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000698216_178774016.pth... [2023-12-26 20:27:01,240][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000697064_178479104.pth [2023-12-26 20:27:01,241][105620] Updated weights for policy 1, policy_version 698971 (0.0006) [2023-12-26 20:27:01,310][105620] Updated weights for policy 1, policy_version 698981 (0.0009) [2023-12-26 20:27:01,377][105620] Updated weights for policy 1, policy_version 698991 (0.0007) [2023-12-26 20:27:01,975][105692] Updated weights for policy 0, policy_version 698217 (0.0009) [2023-12-26 20:27:02,028][105692] Updated weights for policy 0, policy_version 698227 (0.0009) [2023-12-26 20:27:02,076][105620] Updated weights for policy 1, policy_version 699001 (0.0007) [2023-12-26 20:27:02,087][105692] Updated weights for policy 0, policy_version 698237 (0.0008) [2023-12-26 20:27:02,131][105620] Updated weights for policy 1, policy_version 699011 (0.0005) [2023-12-26 20:27:02,136][105692] Updated weights for policy 0, policy_version 698247 (0.0009) [2023-12-26 20:27:02,190][105620] Updated weights for policy 1, policy_version 699021 (0.0005) [2023-12-26 20:27:02,241][105620] Updated weights for policy 1, policy_version 699031 (0.0005) [2023-12-26 20:27:02,888][105620] Updated weights for policy 1, policy_version 699041 (0.0006) [2023-12-26 20:27:02,956][105620] Updated weights for policy 1, policy_version 699051 (0.0005) [2023-12-26 20:27:02,974][105692] Updated weights for policy 0, policy_version 698257 (0.0009) [2023-12-26 20:27:03,015][105620] Updated weights for policy 1, policy_version 699061 (0.0007) [2023-12-26 20:27:03,033][105692] Updated weights for policy 0, policy_version 698267 (0.0009) [2023-12-26 20:27:03,093][105692] Updated weights for policy 0, policy_version 698277 (0.0006) [2023-12-26 20:27:03,602][105692] Updated weights for policy 0, policy_version 698287 (0.0005) [2023-12-26 20:27:03,650][105692] Updated weights for policy 0, policy_version 698297 (0.0005) [2023-12-26 20:27:03,696][105620] Updated weights for policy 1, policy_version 699071 (0.0008) [2023-12-26 20:27:03,701][105692] Updated weights for policy 0, policy_version 698307 (0.0005) [2023-12-26 20:27:03,745][105620] Updated weights for policy 1, policy_version 699081 (0.0008) [2023-12-26 20:27:03,795][105620] Updated weights for policy 1, policy_version 699091 (0.0009) [2023-12-26 20:27:04,432][105692] Updated weights for policy 0, policy_version 698317 (0.0007) [2023-12-26 20:27:04,493][105692] Updated weights for policy 0, policy_version 698327 (0.0010) [2023-12-26 20:27:04,555][105692] Updated weights for policy 0, policy_version 698337 (0.0009) [2023-12-26 20:27:04,572][105620] Updated weights for policy 1, policy_version 699101 (0.0009) [2023-12-26 20:27:04,631][105620] Updated weights for policy 1, policy_version 699111 (0.0007) [2023-12-26 20:27:04,687][105620] Updated weights for policy 1, policy_version 699121 (0.0009) [2023-12-26 20:27:05,330][105692] Updated weights for policy 0, policy_version 698347 (0.0009) [2023-12-26 20:27:05,378][105692] Updated weights for policy 0, policy_version 698357 (0.0009) [2023-12-26 20:27:05,393][105620] Updated weights for policy 1, policy_version 699131 (0.0008) [2023-12-26 20:27:05,424][105692] Updated weights for policy 0, policy_version 698367 (0.0006) [2023-12-26 20:27:05,451][105620] Updated weights for policy 1, policy_version 699141 (0.0007) [2023-12-26 20:27:05,538][105620] Updated weights for policy 1, policy_version 699151 (0.0008) [2023-12-26 20:27:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 357818368. Throughput: 0: 9830.4, 1: 9849.2. Samples: 357810896. Policy #0 lag: (min: 27.0, avg: 40.4, max: 59.0) [2023-12-26 20:27:06,062][104569] Avg episode reward: [(0, '8455.137'), (1, '9083.152')] [2023-12-26 20:27:06,217][105620] Updated weights for policy 1, policy_version 699161 (0.0009) [2023-12-26 20:27:06,240][105692] Updated weights for policy 0, policy_version 698377 (0.0008) [2023-12-26 20:27:06,272][105620] Updated weights for policy 1, policy_version 699171 (0.0007) [2023-12-26 20:27:06,302][105692] Updated weights for policy 0, policy_version 698387 (0.0008) [2023-12-26 20:27:06,325][105620] Updated weights for policy 1, policy_version 699181 (0.0006) [2023-12-26 20:27:06,363][105692] Updated weights for policy 0, policy_version 698397 (0.0007) [2023-12-26 20:27:06,378][105620] Updated weights for policy 1, policy_version 699191 (0.0006) [2023-12-26 20:27:06,428][105692] Updated weights for policy 0, policy_version 698407 (0.0008) [2023-12-26 20:27:07,047][105620] Updated weights for policy 1, policy_version 699201 (0.0009) [2023-12-26 20:27:07,103][105620] Updated weights for policy 1, policy_version 699211 (0.0009) [2023-12-26 20:27:07,155][105620] Updated weights for policy 1, policy_version 699221 (0.0009) [2023-12-26 20:27:07,224][105692] Updated weights for policy 0, policy_version 698417 (0.0009) [2023-12-26 20:27:07,275][105692] Updated weights for policy 0, policy_version 698427 (0.0009) [2023-12-26 20:27:07,327][105692] Updated weights for policy 0, policy_version 698437 (0.0009) [2023-12-26 20:27:07,936][105620] Updated weights for policy 1, policy_version 699231 (0.0007) [2023-12-26 20:27:08,004][105620] Updated weights for policy 1, policy_version 699241 (0.0006) [2023-12-26 20:27:08,056][105692] Updated weights for policy 0, policy_version 698447 (0.0008) [2023-12-26 20:27:08,064][105620] Updated weights for policy 1, policy_version 699251 (0.0010) [2023-12-26 20:27:08,122][105692] Updated weights for policy 0, policy_version 698457 (0.0008) [2023-12-26 20:27:08,188][105692] Updated weights for policy 0, policy_version 698467 (0.0008) [2023-12-26 20:27:08,622][105620] Updated weights for policy 1, policy_version 699261 (0.0005) [2023-12-26 20:27:08,679][105620] Updated weights for policy 1, policy_version 699271 (0.0005) [2023-12-26 20:27:08,739][105620] Updated weights for policy 1, policy_version 699281 (0.0010) [2023-12-26 20:27:09,017][105692] Updated weights for policy 0, policy_version 698477 (0.0008) [2023-12-26 20:27:09,085][105692] Updated weights for policy 0, policy_version 698487 (0.0010) [2023-12-26 20:27:09,138][105692] Updated weights for policy 0, policy_version 698497 (0.0010) [2023-12-26 20:27:09,299][105620] Updated weights for policy 1, policy_version 699291 (0.0007) [2023-12-26 20:27:09,370][105620] Updated weights for policy 1, policy_version 699301 (0.0010) [2023-12-26 20:27:09,439][105620] Updated weights for policy 1, policy_version 699311 (0.0013) [2023-12-26 20:27:09,953][105692] Updated weights for policy 0, policy_version 698507 (0.0010) [2023-12-26 20:27:10,019][105692] Updated weights for policy 0, policy_version 698517 (0.0008) [2023-12-26 20:27:10,089][105692] Updated weights for policy 0, policy_version 698527 (0.0009) [2023-12-26 20:27:10,152][105620] Updated weights for policy 1, policy_version 699321 (0.0009) [2023-12-26 20:27:10,216][105620] Updated weights for policy 1, policy_version 699331 (0.0009) [2023-12-26 20:27:10,284][105620] Updated weights for policy 1, policy_version 699341 (0.0008) [2023-12-26 20:27:10,353][105620] Updated weights for policy 1, policy_version 699351 (0.0007) [2023-12-26 20:27:10,714][105692] Updated weights for policy 0, policy_version 698537 (0.0007) [2023-12-26 20:27:10,762][105692] Updated weights for policy 0, policy_version 698547 (0.0010) [2023-12-26 20:27:10,810][105692] Updated weights for policy 0, policy_version 698557 (0.0005) [2023-12-26 20:27:10,866][105692] Updated weights for policy 0, policy_version 698567 (0.0006) [2023-12-26 20:27:10,929][105620] Updated weights for policy 1, policy_version 699361 (0.0006) [2023-12-26 20:27:10,979][105620] Updated weights for policy 1, policy_version 699371 (0.0005) [2023-12-26 20:27:11,034][105620] Updated weights for policy 1, policy_version 699381 (0.0007) [2023-12-26 20:27:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.4, 300 sec: 19522.0). Total num frames: 357924864. Throughput: 0: 9673.4, 1: 9867.4. Samples: 357928252. Policy #0 lag: (min: 27.0, avg: 40.4, max: 59.0) [2023-12-26 20:27:11,063][104569] Avg episode reward: [(0, '7700.330'), (1, '8991.437')] [2023-12-26 20:27:11,665][105692] Updated weights for policy 0, policy_version 698577 (0.0008) [2023-12-26 20:27:11,727][105692] Updated weights for policy 0, policy_version 698587 (0.0008) [2023-12-26 20:27:11,771][105620] Updated weights for policy 1, policy_version 699391 (0.0010) [2023-12-26 20:27:11,790][105692] Updated weights for policy 0, policy_version 698597 (0.0007) [2023-12-26 20:27:11,824][105620] Updated weights for policy 1, policy_version 699401 (0.0010) [2023-12-26 20:27:11,883][105620] Updated weights for policy 1, policy_version 699411 (0.0010) [2023-12-26 20:27:12,538][105692] Updated weights for policy 0, policy_version 698607 (0.0006) [2023-12-26 20:27:12,608][105692] Updated weights for policy 0, policy_version 698617 (0.0007) [2023-12-26 20:27:12,614][105620] Updated weights for policy 1, policy_version 699421 (0.0010) [2023-12-26 20:27:12,657][105692] Updated weights for policy 0, policy_version 698627 (0.0010) [2023-12-26 20:27:12,678][105620] Updated weights for policy 1, policy_version 699431 (0.0009) [2023-12-26 20:27:12,742][105620] Updated weights for policy 1, policy_version 699441 (0.0008) [2023-12-26 20:27:13,237][105692] Updated weights for policy 0, policy_version 698637 (0.0009) [2023-12-26 20:27:13,285][105692] Updated weights for policy 0, policy_version 698647 (0.0010) [2023-12-26 20:27:13,327][105620] Updated weights for policy 1, policy_version 699451 (0.0007) [2023-12-26 20:27:13,340][105692] Updated weights for policy 0, policy_version 698657 (0.0010) [2023-12-26 20:27:13,385][105620] Updated weights for policy 1, policy_version 699461 (0.0006) [2023-12-26 20:27:13,447][105620] Updated weights for policy 1, policy_version 699471 (0.0005) [2023-12-26 20:27:13,957][105692] Updated weights for policy 0, policy_version 698667 (0.0009) [2023-12-26 20:27:13,987][105620] Updated weights for policy 1, policy_version 699481 (0.0006) [2023-12-26 20:27:14,015][105692] Updated weights for policy 0, policy_version 698677 (0.0008) [2023-12-26 20:27:14,045][105620] Updated weights for policy 1, policy_version 699491 (0.0005) [2023-12-26 20:27:14,074][105692] Updated weights for policy 0, policy_version 698687 (0.0011) [2023-12-26 20:27:14,105][105620] Updated weights for policy 1, policy_version 699501 (0.0005) [2023-12-26 20:27:14,166][105620] Updated weights for policy 1, policy_version 699511 (0.0006) [2023-12-26 20:27:14,718][105620] Updated weights for policy 1, policy_version 699521 (0.0005) [2023-12-26 20:27:14,784][105692] Updated weights for policy 0, policy_version 698697 (0.0010) [2023-12-26 20:27:14,784][105620] Updated weights for policy 1, policy_version 699531 (0.0008) [2023-12-26 20:27:14,847][105692] Updated weights for policy 0, policy_version 698707 (0.0010) [2023-12-26 20:27:14,848][105620] Updated weights for policy 1, policy_version 699541 (0.0007) [2023-12-26 20:27:14,903][105692] Updated weights for policy 0, policy_version 698717 (0.0011) [2023-12-26 20:27:14,956][105692] Updated weights for policy 0, policy_version 698727 (0.0010) [2023-12-26 20:27:15,581][105620] Updated weights for policy 1, policy_version 699551 (0.0008) [2023-12-26 20:27:15,634][105620] Updated weights for policy 1, policy_version 699561 (0.0008) [2023-12-26 20:27:15,675][105692] Updated weights for policy 0, policy_version 698737 (0.0006) [2023-12-26 20:27:15,693][105620] Updated weights for policy 1, policy_version 699571 (0.0009) [2023-12-26 20:27:15,745][105692] Updated weights for policy 0, policy_version 698747 (0.0005) [2023-12-26 20:27:15,815][105692] Updated weights for policy 0, policy_version 698757 (0.0005) [2023-12-26 20:27:16,062][104569] Fps is (10 sec: 20479.3, 60 sec: 19797.2, 300 sec: 19521.9). Total num frames: 358023168. Throughput: 0: 9674.6, 1: 9931.1. Samples: 357989012. Policy #0 lag: (min: 14.0, avg: 31.0, max: 32.0) [2023-12-26 20:27:16,063][104569] Avg episode reward: [(0, '7970.324'), (1, '8813.398')] [2023-12-26 20:27:16,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000699576_179109888.pth... [2023-12-26 20:27:16,073][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000698760_178913280.pth... [2023-12-26 20:27:16,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000698392_178806784.pth [2023-12-26 20:27:16,081][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000697608_178618368.pth [2023-12-26 20:27:16,277][105692] Updated weights for policy 0, policy_version 698767 (0.0009) [2023-12-26 20:27:16,325][105692] Updated weights for policy 0, policy_version 698777 (0.0010) [2023-12-26 20:27:16,373][105692] Updated weights for policy 0, policy_version 698787 (0.0010) [2023-12-26 20:27:16,531][105620] Updated weights for policy 1, policy_version 699581 (0.0007) [2023-12-26 20:27:16,579][105620] Updated weights for policy 1, policy_version 699591 (0.0005) [2023-12-26 20:27:16,627][105620] Updated weights for policy 1, policy_version 699601 (0.0005) [2023-12-26 20:27:17,054][105692] Updated weights for policy 0, policy_version 698797 (0.0008) [2023-12-26 20:27:17,119][105692] Updated weights for policy 0, policy_version 698807 (0.0007) [2023-12-26 20:27:17,179][105692] Updated weights for policy 0, policy_version 698817 (0.0008) [2023-12-26 20:27:17,222][105620] Updated weights for policy 1, policy_version 699611 (0.0006) [2023-12-26 20:27:17,284][105620] Updated weights for policy 1, policy_version 699621 (0.0008) [2023-12-26 20:27:17,339][105620] Updated weights for policy 1, policy_version 699631 (0.0006) [2023-12-26 20:27:17,763][105692] Updated weights for policy 0, policy_version 698827 (0.0008) [2023-12-26 20:27:17,808][105692] Updated weights for policy 0, policy_version 698837 (0.0005) [2023-12-26 20:27:17,862][105692] Updated weights for policy 0, policy_version 698847 (0.0005) [2023-12-26 20:27:17,932][105620] Updated weights for policy 1, policy_version 699641 (0.0007) [2023-12-26 20:27:17,984][105620] Updated weights for policy 1, policy_version 699651 (0.0005) [2023-12-26 20:27:18,041][105620] Updated weights for policy 1, policy_version 699661 (0.0005) [2023-12-26 20:27:18,093][105620] Updated weights for policy 1, policy_version 699671 (0.0005) [2023-12-26 20:27:18,495][105692] Updated weights for policy 0, policy_version 698857 (0.0006) [2023-12-26 20:27:18,558][105692] Updated weights for policy 0, policy_version 698867 (0.0011) [2023-12-26 20:27:18,631][105692] Updated weights for policy 0, policy_version 698877 (0.0011) [2023-12-26 20:27:18,698][105692] Updated weights for policy 0, policy_version 698887 (0.0011) [2023-12-26 20:27:18,704][105620] Updated weights for policy 1, policy_version 699681 (0.0007) [2023-12-26 20:27:18,751][105620] Updated weights for policy 1, policy_version 699691 (0.0006) [2023-12-26 20:27:18,802][105620] Updated weights for policy 1, policy_version 699701 (0.0006) [2023-12-26 20:27:19,323][105692] Updated weights for policy 0, policy_version 698897 (0.0008) [2023-12-26 20:27:19,392][105692] Updated weights for policy 0, policy_version 698907 (0.0008) [2023-12-26 20:27:19,424][105620] Updated weights for policy 1, policy_version 699711 (0.0007) [2023-12-26 20:27:19,454][105692] Updated weights for policy 0, policy_version 698917 (0.0007) [2023-12-26 20:27:19,482][105620] Updated weights for policy 1, policy_version 699721 (0.0007) [2023-12-26 20:27:19,544][105620] Updated weights for policy 1, policy_version 699731 (0.0008) [2023-12-26 20:27:20,251][105620] Updated weights for policy 1, policy_version 699741 (0.0009) [2023-12-26 20:27:20,257][105692] Updated weights for policy 0, policy_version 698927 (0.0006) [2023-12-26 20:27:20,306][105620] Updated weights for policy 1, policy_version 699751 (0.0007) [2023-12-26 20:27:20,317][105692] Updated weights for policy 0, policy_version 698937 (0.0007) [2023-12-26 20:27:20,368][105620] Updated weights for policy 1, policy_version 699761 (0.0007) [2023-12-26 20:27:20,374][105692] Updated weights for policy 0, policy_version 698947 (0.0006) [2023-12-26 20:27:21,027][105620] Updated weights for policy 1, policy_version 699771 (0.0007) [2023-12-26 20:27:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19521.9). Total num frames: 358121472. Throughput: 0: 9755.7, 1: 9994.9. Samples: 358117624. Policy #0 lag: (min: 14.0, avg: 31.0, max: 32.0) [2023-12-26 20:27:21,063][104569] Avg episode reward: [(0, '8816.986'), (1, '8724.822')] [2023-12-26 20:27:21,087][105620] Updated weights for policy 1, policy_version 699781 (0.0008) [2023-12-26 20:27:21,133][105692] Updated weights for policy 0, policy_version 698957 (0.0007) [2023-12-26 20:27:21,154][105620] Updated weights for policy 1, policy_version 699791 (0.0008) [2023-12-26 20:27:21,194][105692] Updated weights for policy 0, policy_version 698967 (0.0007) [2023-12-26 20:27:21,257][105692] Updated weights for policy 0, policy_version 698977 (0.0007) [2023-12-26 20:27:21,848][105620] Updated weights for policy 1, policy_version 699801 (0.0009) [2023-12-26 20:27:21,900][105620] Updated weights for policy 1, policy_version 699811 (0.0009) [2023-12-26 20:27:21,963][105620] Updated weights for policy 1, policy_version 699821 (0.0009) [2023-12-26 20:27:21,973][105692] Updated weights for policy 0, policy_version 698987 (0.0009) [2023-12-26 20:27:22,017][105620] Updated weights for policy 1, policy_version 699831 (0.0007) [2023-12-26 20:27:22,031][105692] Updated weights for policy 0, policy_version 698997 (0.0008) [2023-12-26 20:27:22,082][105692] Updated weights for policy 0, policy_version 699007 (0.0009) [2023-12-26 20:27:22,825][105692] Updated weights for policy 0, policy_version 699017 (0.0009) [2023-12-26 20:27:22,842][105620] Updated weights for policy 1, policy_version 699841 (0.0009) [2023-12-26 20:27:22,888][105692] Updated weights for policy 0, policy_version 699027 (0.0008) [2023-12-26 20:27:22,904][105620] Updated weights for policy 1, policy_version 699851 (0.0006) [2023-12-26 20:27:22,946][105692] Updated weights for policy 0, policy_version 699037 (0.0006) [2023-12-26 20:27:22,960][105620] Updated weights for policy 1, policy_version 699861 (0.0007) [2023-12-26 20:27:22,998][105692] Updated weights for policy 0, policy_version 699047 (0.0007) [2023-12-26 20:27:23,652][105620] Updated weights for policy 1, policy_version 699871 (0.0008) [2023-12-26 20:27:23,716][105620] Updated weights for policy 1, policy_version 699881 (0.0008) [2023-12-26 20:27:23,780][105620] Updated weights for policy 1, policy_version 699891 (0.0009) [2023-12-26 20:27:23,791][105692] Updated weights for policy 0, policy_version 699057 (0.0007) [2023-12-26 20:27:23,848][105692] Updated weights for policy 0, policy_version 699067 (0.0006) [2023-12-26 20:27:23,896][105692] Updated weights for policy 0, policy_version 699077 (0.0005) [2023-12-26 20:27:24,505][105620] Updated weights for policy 1, policy_version 699901 (0.0009) [2023-12-26 20:27:24,522][105692] Updated weights for policy 0, policy_version 699087 (0.0006) [2023-12-26 20:27:24,568][105620] Updated weights for policy 1, policy_version 699911 (0.0006) [2023-12-26 20:27:24,574][105692] Updated weights for policy 0, policy_version 699097 (0.0006) [2023-12-26 20:27:24,624][105620] Updated weights for policy 1, policy_version 699921 (0.0005) [2023-12-26 20:27:24,635][105692] Updated weights for policy 0, policy_version 699107 (0.0007) [2023-12-26 20:27:25,178][105692] Updated weights for policy 0, policy_version 699117 (0.0009) [2023-12-26 20:27:25,206][105620] Updated weights for policy 1, policy_version 699931 (0.0007) [2023-12-26 20:27:25,218][105585] KL-divergence is very high: 108.6640 [2023-12-26 20:27:25,236][105585] KL-divergence is very high: 109.4679 [2023-12-26 20:27:25,238][105692] Updated weights for policy 0, policy_version 699127 (0.0011) [2023-12-26 20:27:25,261][105620] Updated weights for policy 1, policy_version 699941 (0.0007) [2023-12-26 20:27:25,270][105585] KL-divergence is very high: 158.6028 [2023-12-26 20:27:25,290][105585] KL-divergence is very high: 114.3692 [2023-12-26 20:27:25,303][105692] Updated weights for policy 0, policy_version 699137 (0.0011) [2023-12-26 20:27:25,318][105620] Updated weights for policy 1, policy_version 699951 (0.0005) [2023-12-26 20:27:25,322][105585] KL-divergence is very high: 126.2564 [2023-12-26 20:27:25,911][105585] KL-divergence is very high: 102.2330 [2023-12-26 20:27:25,928][105692] Updated weights for policy 0, policy_version 699147 (0.0009) [2023-12-26 20:27:25,989][105692] Updated weights for policy 0, policy_version 699157 (0.0005) [2023-12-26 20:27:26,012][105620] Updated weights for policy 1, policy_version 699961 (0.0007) [2023-12-26 20:27:26,048][105692] Updated weights for policy 0, policy_version 699167 (0.0009) [2023-12-26 20:27:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19660.9, 300 sec: 19522.0). Total num frames: 358219776. Throughput: 0: 9827.9, 1: 9999.8. Samples: 358234924. Policy #0 lag: (min: 14.0, avg: 31.0, max: 32.0) [2023-12-26 20:27:26,062][104569] Avg episode reward: [(0, '8897.629'), (1, '8904.270')] [2023-12-26 20:27:26,074][105620] Updated weights for policy 1, policy_version 699971 (0.0008) [2023-12-26 20:27:26,128][105620] Updated weights for policy 1, policy_version 699981 (0.0006) [2023-12-26 20:27:26,183][105620] Updated weights for policy 1, policy_version 699991 (0.0005) [2023-12-26 20:27:26,603][105692] Updated weights for policy 0, policy_version 699177 (0.0006) [2023-12-26 20:27:26,658][105692] Updated weights for policy 0, policy_version 699187 (0.0005) [2023-12-26 20:27:26,702][105692] Updated weights for policy 0, policy_version 699197 (0.0005) [2023-12-26 20:27:26,746][105692] Updated weights for policy 0, policy_version 699207 (0.0005) [2023-12-26 20:27:26,770][105620] Updated weights for policy 1, policy_version 700001 (0.0010) [2023-12-26 20:27:26,815][105620] Updated weights for policy 1, policy_version 700011 (0.0010) [2023-12-26 20:27:26,873][105620] Updated weights for policy 1, policy_version 700021 (0.0010) [2023-12-26 20:27:27,282][105692] Updated weights for policy 0, policy_version 699217 (0.0005) [2023-12-26 20:27:27,339][105692] Updated weights for policy 0, policy_version 699227 (0.0005) [2023-12-26 20:27:27,401][105692] Updated weights for policy 0, policy_version 699237 (0.0005) [2023-12-26 20:27:27,546][105620] Updated weights for policy 1, policy_version 700031 (0.0007) [2023-12-26 20:27:27,604][105620] Updated weights for policy 1, policy_version 700041 (0.0010) [2023-12-26 20:27:27,652][105620] Updated weights for policy 1, policy_version 700051 (0.0010) [2023-12-26 20:27:28,015][105692] Updated weights for policy 0, policy_version 699247 (0.0006) [2023-12-26 20:27:28,072][105692] Updated weights for policy 0, policy_version 699257 (0.0006) [2023-12-26 20:27:28,123][105692] Updated weights for policy 0, policy_version 699267 (0.0010) [2023-12-26 20:27:28,275][105620] Updated weights for policy 1, policy_version 700061 (0.0010) [2023-12-26 20:27:28,327][105620] Updated weights for policy 1, policy_version 700071 (0.0010) [2023-12-26 20:27:28,375][105620] Updated weights for policy 1, policy_version 700081 (0.0008) [2023-12-26 20:27:28,751][105692] Updated weights for policy 0, policy_version 699277 (0.0011) [2023-12-26 20:27:28,799][105692] Updated weights for policy 0, policy_version 699287 (0.0010) [2023-12-26 20:27:28,860][105692] Updated weights for policy 0, policy_version 699297 (0.0008) [2023-12-26 20:27:29,134][105620] Updated weights for policy 1, policy_version 700091 (0.0010) [2023-12-26 20:27:29,178][105620] Updated weights for policy 1, policy_version 700101 (0.0010) [2023-12-26 20:27:29,237][105620] Updated weights for policy 1, policy_version 700111 (0.0009) [2023-12-26 20:27:29,546][105692] Updated weights for policy 0, policy_version 699307 (0.0008) [2023-12-26 20:27:29,597][105692] Updated weights for policy 0, policy_version 699317 (0.0010) [2023-12-26 20:27:29,656][105692] Updated weights for policy 0, policy_version 699327 (0.0010) [2023-12-26 20:27:30,001][105620] Updated weights for policy 1, policy_version 700121 (0.0007) [2023-12-26 20:27:30,071][105620] Updated weights for policy 1, policy_version 700131 (0.0011) [2023-12-26 20:27:30,133][105620] Updated weights for policy 1, policy_version 700141 (0.0010) [2023-12-26 20:27:30,202][105620] Updated weights for policy 1, policy_version 700151 (0.0010) [2023-12-26 20:27:30,353][105692] Updated weights for policy 0, policy_version 699337 (0.0008) [2023-12-26 20:27:30,419][105692] Updated weights for policy 0, policy_version 699347 (0.0010) [2023-12-26 20:27:30,481][105692] Updated weights for policy 0, policy_version 699357 (0.0011) [2023-12-26 20:27:30,543][105692] Updated weights for policy 0, policy_version 699367 (0.0010) [2023-12-26 20:27:30,847][105620] Updated weights for policy 1, policy_version 700161 (0.0010) [2023-12-26 20:27:30,912][105620] Updated weights for policy 1, policy_version 700171 (0.0010) [2023-12-26 20:27:30,970][105620] Updated weights for policy 1, policy_version 700181 (0.0010) [2023-12-26 20:27:31,062][104569] Fps is (10 sec: 21299.4, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 358334464. Throughput: 0: 10014.5, 1: 10068.5. Samples: 358302320. Policy #0 lag: (min: 14.0, avg: 31.0, max: 32.0) [2023-12-26 20:27:31,063][104569] Avg episode reward: [(0, '8986.621'), (1, '9176.712')] [2023-12-26 20:27:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000699368_179068928.pth... [2023-12-26 20:27:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000700184_179265536.pth... [2023-12-26 20:27:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000698216_178774016.pth [2023-12-26 20:27:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000698968_178954240.pth [2023-12-26 20:27:31,275][105692] Updated weights for policy 0, policy_version 699377 (0.0011) [2023-12-26 20:27:31,326][105692] Updated weights for policy 0, policy_version 699387 (0.0011) [2023-12-26 20:27:31,392][105692] Updated weights for policy 0, policy_version 699397 (0.0012) [2023-12-26 20:27:31,628][105620] Updated weights for policy 1, policy_version 700191 (0.0009) [2023-12-26 20:27:31,689][105620] Updated weights for policy 1, policy_version 700201 (0.0008) [2023-12-26 20:27:31,752][105620] Updated weights for policy 1, policy_version 700211 (0.0009) [2023-12-26 20:27:32,143][105692] Updated weights for policy 0, policy_version 699407 (0.0007) [2023-12-26 20:27:32,213][105692] Updated weights for policy 0, policy_version 699417 (0.0006) [2023-12-26 20:27:32,280][105692] Updated weights for policy 0, policy_version 699427 (0.0010) [2023-12-26 20:27:32,513][105620] Updated weights for policy 1, policy_version 700221 (0.0007) [2023-12-26 20:27:32,589][105620] Updated weights for policy 1, policy_version 700231 (0.0007) [2023-12-26 20:27:32,649][105620] Updated weights for policy 1, policy_version 700241 (0.0009) [2023-12-26 20:27:32,845][105692] Updated weights for policy 0, policy_version 699437 (0.0010) [2023-12-26 20:27:32,900][105692] Updated weights for policy 0, policy_version 699447 (0.0006) [2023-12-26 20:27:32,949][105692] Updated weights for policy 0, policy_version 699457 (0.0005) [2023-12-26 20:27:33,200][105620] Updated weights for policy 1, policy_version 700251 (0.0009) [2023-12-26 20:27:33,250][105620] Updated weights for policy 1, policy_version 700261 (0.0010) [2023-12-26 20:27:33,298][105620] Updated weights for policy 1, policy_version 700271 (0.0010) [2023-12-26 20:27:33,505][105692] Updated weights for policy 0, policy_version 699467 (0.0006) [2023-12-26 20:27:33,562][105692] Updated weights for policy 0, policy_version 699477 (0.0006) [2023-12-26 20:27:33,611][105692] Updated weights for policy 0, policy_version 699487 (0.0006) [2023-12-26 20:27:34,037][105620] Updated weights for policy 1, policy_version 700281 (0.0010) [2023-12-26 20:27:34,081][105620] Updated weights for policy 1, policy_version 700291 (0.0010) [2023-12-26 20:27:34,142][105620] Updated weights for policy 1, policy_version 700301 (0.0010) [2023-12-26 20:27:34,186][105692] Updated weights for policy 0, policy_version 699497 (0.0005) [2023-12-26 20:27:34,202][105620] Updated weights for policy 1, policy_version 700311 (0.0010) [2023-12-26 20:27:34,237][105692] Updated weights for policy 0, policy_version 699507 (0.0008) [2023-12-26 20:27:34,299][105692] Updated weights for policy 0, policy_version 699517 (0.0008) [2023-12-26 20:27:34,361][105692] Updated weights for policy 0, policy_version 699527 (0.0008) [2023-12-26 20:27:34,932][105620] Updated weights for policy 1, policy_version 700321 (0.0008) [2023-12-26 20:27:34,973][105692] Updated weights for policy 0, policy_version 699537 (0.0006) [2023-12-26 20:27:34,991][105620] Updated weights for policy 1, policy_version 700331 (0.0005) [2023-12-26 20:27:35,030][105692] Updated weights for policy 0, policy_version 699547 (0.0009) [2023-12-26 20:27:35,053][105620] Updated weights for policy 1, policy_version 700341 (0.0007) [2023-12-26 20:27:35,084][105692] Updated weights for policy 0, policy_version 699557 (0.0008) [2023-12-26 20:27:35,587][105620] Updated weights for policy 1, policy_version 700351 (0.0006) [2023-12-26 20:27:35,647][105620] Updated weights for policy 1, policy_version 700361 (0.0005) [2023-12-26 20:27:35,705][105620] Updated weights for policy 1, policy_version 700371 (0.0007) [2023-12-26 20:27:35,899][105692] Updated weights for policy 0, policy_version 699567 (0.0007) [2023-12-26 20:27:35,966][105692] Updated weights for policy 0, policy_version 699577 (0.0009) [2023-12-26 20:27:36,032][105692] Updated weights for policy 0, policy_version 699587 (0.0009) [2023-12-26 20:27:36,062][104569] Fps is (10 sec: 22118.5, 60 sec: 20070.5, 300 sec: 19605.3). Total num frames: 358440960. Throughput: 0: 10086.1, 1: 10161.4. Samples: 358425300. Policy #0 lag: (min: 14.0, avg: 31.0, max: 32.0) [2023-12-26 20:27:36,062][104569] Avg episode reward: [(0, '9077.271'), (1, '9085.972')] [2023-12-26 20:27:36,317][105620] Updated weights for policy 1, policy_version 700381 (0.0007) [2023-12-26 20:27:36,382][105620] Updated weights for policy 1, policy_version 700391 (0.0007) [2023-12-26 20:27:36,440][105620] Updated weights for policy 1, policy_version 700401 (0.0006) [2023-12-26 20:27:36,835][105692] Updated weights for policy 0, policy_version 699597 (0.0007) [2023-12-26 20:27:36,898][105692] Updated weights for policy 0, policy_version 699607 (0.0005) [2023-12-26 20:27:36,954][105692] Updated weights for policy 0, policy_version 699617 (0.0005) [2023-12-26 20:27:37,091][105620] Updated weights for policy 1, policy_version 700411 (0.0006) [2023-12-26 20:27:37,151][105620] Updated weights for policy 1, policy_version 700421 (0.0009) [2023-12-26 20:27:37,204][105620] Updated weights for policy 1, policy_version 700431 (0.0009) [2023-12-26 20:27:37,628][105692] Updated weights for policy 0, policy_version 699627 (0.0007) [2023-12-26 20:27:37,690][105692] Updated weights for policy 0, policy_version 699637 (0.0009) [2023-12-26 20:27:37,753][105692] Updated weights for policy 0, policy_version 699647 (0.0009) [2023-12-26 20:27:37,957][105620] Updated weights for policy 1, policy_version 700441 (0.0009) [2023-12-26 20:27:38,015][105620] Updated weights for policy 1, policy_version 700451 (0.0009) [2023-12-26 20:27:38,045][105586] KL-divergence is very high: 180.7046 [2023-12-26 20:27:38,074][105620] Updated weights for policy 1, policy_version 700461 (0.0009) [2023-12-26 20:27:38,094][105586] KL-divergence is very high: 221.3139 [2023-12-26 20:27:38,136][105620] Updated weights for policy 1, policy_version 700471 (0.0009) [2023-12-26 20:27:38,535][105692] Updated weights for policy 0, policy_version 699657 (0.0009) [2023-12-26 20:27:38,596][105692] Updated weights for policy 0, policy_version 699667 (0.0009) [2023-12-26 20:27:38,662][105692] Updated weights for policy 0, policy_version 699678 (0.0010) [2023-12-26 20:27:38,719][105692] Updated weights for policy 0, policy_version 699688 (0.0010) [2023-12-26 20:27:38,782][105620] Updated weights for policy 1, policy_version 700481 (0.0008) [2023-12-26 20:27:38,844][105620] Updated weights for policy 1, policy_version 700491 (0.0009) [2023-12-26 20:27:38,904][105620] Updated weights for policy 1, policy_version 700501 (0.0008) [2023-12-26 20:27:39,528][105692] Updated weights for policy 0, policy_version 699698 (0.0006) [2023-12-26 20:27:39,583][105692] Updated weights for policy 0, policy_version 699708 (0.0009) [2023-12-26 20:27:39,601][105620] Updated weights for policy 1, policy_version 700511 (0.0007) [2023-12-26 20:27:39,644][105692] Updated weights for policy 0, policy_version 699718 (0.0007) [2023-12-26 20:27:39,660][105620] Updated weights for policy 1, policy_version 700521 (0.0007) [2023-12-26 20:27:39,722][105620] Updated weights for policy 1, policy_version 700531 (0.0005) [2023-12-26 20:27:40,422][105620] Updated weights for policy 1, policy_version 700541 (0.0008) [2023-12-26 20:27:40,437][105692] Updated weights for policy 0, policy_version 699728 (0.0007) [2023-12-26 20:27:40,477][105620] Updated weights for policy 1, policy_version 700551 (0.0008) [2023-12-26 20:27:40,492][105692] Updated weights for policy 0, policy_version 699738 (0.0006) [2023-12-26 20:27:40,543][105620] Updated weights for policy 1, policy_version 700561 (0.0006) [2023-12-26 20:27:40,554][105692] Updated weights for policy 0, policy_version 699748 (0.0010) [2023-12-26 20:27:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 20070.4, 300 sec: 19577.5). Total num frames: 358531072. Throughput: 0: 9944.0, 1: 10215.1. Samples: 358542188. Policy #0 lag: (min: 14.0, avg: 31.0, max: 32.0) [2023-12-26 20:27:41,062][104569] Avg episode reward: [(0, '8622.966'), (1, '8629.761')] [2023-12-26 20:27:41,286][105620] Updated weights for policy 1, policy_version 700571 (0.0008) [2023-12-26 20:27:41,316][105586] KL-divergence is very high: 140.2290 [2023-12-26 20:27:41,347][105620] Updated weights for policy 1, policy_version 700581 (0.0009) [2023-12-26 20:27:41,371][105586] KL-divergence is very high: 195.3117 [2023-12-26 20:27:41,389][105692] Updated weights for policy 0, policy_version 699758 (0.0009) [2023-12-26 20:27:41,413][105620] Updated weights for policy 1, policy_version 700591 (0.0007) [2023-12-26 20:27:41,419][105586] KL-divergence is very high: 138.1212 [2023-12-26 20:27:41,448][105692] Updated weights for policy 0, policy_version 699768 (0.0007) [2023-12-26 20:27:41,506][105692] Updated weights for policy 0, policy_version 699778 (0.0008) [2023-12-26 20:27:42,141][105620] Updated weights for policy 1, policy_version 700601 (0.0008) [2023-12-26 20:27:42,198][105620] Updated weights for policy 1, policy_version 700611 (0.0008) [2023-12-26 20:27:42,216][105692] Updated weights for policy 0, policy_version 699788 (0.0010) [2023-12-26 20:27:42,255][105620] Updated weights for policy 1, policy_version 700621 (0.0008) [2023-12-26 20:27:42,284][105692] Updated weights for policy 0, policy_version 699798 (0.0008) [2023-12-26 20:27:42,320][105620] Updated weights for policy 1, policy_version 700631 (0.0008) [2023-12-26 20:27:42,351][105692] Updated weights for policy 0, policy_version 699808 (0.0008) [2023-12-26 20:27:43,045][105620] Updated weights for policy 1, policy_version 700641 (0.0009) [2023-12-26 20:27:43,093][105620] Updated weights for policy 1, policy_version 700651 (0.0009) [2023-12-26 20:27:43,109][105692] Updated weights for policy 0, policy_version 699818 (0.0008) [2023-12-26 20:27:43,143][105620] Updated weights for policy 1, policy_version 700661 (0.0007) [2023-12-26 20:27:43,158][105692] Updated weights for policy 0, policy_version 699828 (0.0006) [2023-12-26 20:27:43,204][105692] Updated weights for policy 0, policy_version 699838 (0.0008) [2023-12-26 20:27:43,262][105692] Updated weights for policy 0, policy_version 699848 (0.0008) [2023-12-26 20:27:43,874][105620] Updated weights for policy 1, policy_version 700671 (0.0006) [2023-12-26 20:27:43,927][105620] Updated weights for policy 1, policy_version 700681 (0.0007) [2023-12-26 20:27:43,979][105620] Updated weights for policy 1, policy_version 700691 (0.0009) [2023-12-26 20:27:44,054][105692] Updated weights for policy 0, policy_version 699858 (0.0009) [2023-12-26 20:27:44,106][105692] Updated weights for policy 0, policy_version 699868 (0.0008) [2023-12-26 20:27:44,157][105692] Updated weights for policy 0, policy_version 699878 (0.0007) [2023-12-26 20:27:44,677][105620] Updated weights for policy 1, policy_version 700701 (0.0008) [2023-12-26 20:27:44,725][105620] Updated weights for policy 1, policy_version 700711 (0.0010) [2023-12-26 20:27:44,781][105620] Updated weights for policy 1, policy_version 700721 (0.0009) [2023-12-26 20:27:44,952][105692] Updated weights for policy 0, policy_version 699888 (0.0009) [2023-12-26 20:27:45,016][105692] Updated weights for policy 0, policy_version 699898 (0.0008) [2023-12-26 20:27:45,083][105692] Updated weights for policy 0, policy_version 699908 (0.0008) [2023-12-26 20:27:45,556][105620] Updated weights for policy 1, policy_version 700731 (0.0010) [2023-12-26 20:27:45,617][105620] Updated weights for policy 1, policy_version 700741 (0.0009) [2023-12-26 20:27:45,677][105620] Updated weights for policy 1, policy_version 700751 (0.0005) [2023-12-26 20:27:45,854][105692] Updated weights for policy 0, policy_version 699918 (0.0009) [2023-12-26 20:27:45,914][105692] Updated weights for policy 0, policy_version 699928 (0.0010) [2023-12-26 20:27:45,966][105692] Updated weights for policy 0, policy_version 699938 (0.0009) [2023-12-26 20:27:46,062][104569] Fps is (10 sec: 18841.0, 60 sec: 20070.3, 300 sec: 19549.7). Total num frames: 358629376. Throughput: 0: 9903.8, 1: 10193.2. Samples: 358598384. Policy #0 lag: (min: 14.0, avg: 31.0, max: 32.0) [2023-12-26 20:27:46,063][104569] Avg episode reward: [(0, '8550.016'), (1, '8632.617')] [2023-12-26 20:27:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000699944_179216384.pth... [2023-12-26 20:27:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000700760_179412992.pth... [2023-12-26 20:27:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000698760_178913280.pth [2023-12-26 20:27:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000699576_179109888.pth [2023-12-26 20:27:46,282][105620] Updated weights for policy 1, policy_version 700761 (0.0006) [2023-12-26 20:27:46,339][105620] Updated weights for policy 1, policy_version 700771 (0.0009) [2023-12-26 20:27:46,390][105620] Updated weights for policy 1, policy_version 700781 (0.0009) [2023-12-26 20:27:46,446][105620] Updated weights for policy 1, policy_version 700791 (0.0005) [2023-12-26 20:27:46,618][105692] Updated weights for policy 0, policy_version 699948 (0.0007) [2023-12-26 20:27:46,673][105692] Updated weights for policy 0, policy_version 699958 (0.0008) [2023-12-26 20:27:46,731][105692] Updated weights for policy 0, policy_version 699968 (0.0008) [2023-12-26 20:27:47,163][105620] Updated weights for policy 1, policy_version 700801 (0.0010) [2023-12-26 20:27:47,224][105620] Updated weights for policy 1, policy_version 700811 (0.0010) [2023-12-26 20:27:47,287][105620] Updated weights for policy 1, policy_version 700821 (0.0011) [2023-12-26 20:27:47,512][105692] Updated weights for policy 0, policy_version 699978 (0.0008) [2023-12-26 20:27:47,572][105692] Updated weights for policy 0, policy_version 699988 (0.0008) [2023-12-26 20:27:47,632][105692] Updated weights for policy 0, policy_version 699998 (0.0008) [2023-12-26 20:27:47,691][105692] Updated weights for policy 0, policy_version 700008 (0.0008) [2023-12-26 20:27:48,034][105620] Updated weights for policy 1, policy_version 700831 (0.0010) [2023-12-26 20:27:48,088][105620] Updated weights for policy 1, policy_version 700841 (0.0010) [2023-12-26 20:27:48,136][105620] Updated weights for policy 1, policy_version 700851 (0.0010) [2023-12-26 20:27:48,460][105692] Updated weights for policy 0, policy_version 700018 (0.0006) [2023-12-26 20:27:48,525][105692] Updated weights for policy 0, policy_version 700028 (0.0005) [2023-12-26 20:27:48,592][105692] Updated weights for policy 0, policy_version 700038 (0.0006) [2023-12-26 20:27:48,906][105620] Updated weights for policy 1, policy_version 700861 (0.0010) [2023-12-26 20:27:48,972][105620] Updated weights for policy 1, policy_version 700871 (0.0010) [2023-12-26 20:27:49,045][105620] Updated weights for policy 1, policy_version 700881 (0.0010) [2023-12-26 20:27:49,128][105692] Updated weights for policy 0, policy_version 700048 (0.0005) [2023-12-26 20:27:49,194][105692] Updated weights for policy 0, policy_version 700058 (0.0007) [2023-12-26 20:27:49,256][105692] Updated weights for policy 0, policy_version 700068 (0.0009) [2023-12-26 20:27:49,645][105620] Updated weights for policy 1, policy_version 700891 (0.0007) [2023-12-26 20:27:49,699][105620] Updated weights for policy 1, policy_version 700901 (0.0005) [2023-12-26 20:27:49,749][105620] Updated weights for policy 1, policy_version 700911 (0.0008) [2023-12-26 20:27:49,872][105692] Updated weights for policy 0, policy_version 700078 (0.0009) [2023-12-26 20:27:49,935][105692] Updated weights for policy 0, policy_version 700088 (0.0008) [2023-12-26 20:27:49,998][105692] Updated weights for policy 0, policy_version 700098 (0.0007) [2023-12-26 20:27:50,416][105620] Updated weights for policy 1, policy_version 700921 (0.0010) [2023-12-26 20:27:50,471][105620] Updated weights for policy 1, policy_version 700931 (0.0008) [2023-12-26 20:27:50,498][105586] KL-divergence is very high: 165.1180 [2023-12-26 20:27:50,526][105620] Updated weights for policy 1, policy_version 700941 (0.0005) [2023-12-26 20:27:50,543][105586] KL-divergence is very high: 287.7288 [2023-12-26 20:27:50,593][105620] Updated weights for policy 1, policy_version 700951 (0.0006) [2023-12-26 20:27:50,672][105692] Updated weights for policy 0, policy_version 700108 (0.0009) [2023-12-26 20:27:50,742][105692] Updated weights for policy 0, policy_version 700118 (0.0008) [2023-12-26 20:27:50,808][105692] Updated weights for policy 0, policy_version 700128 (0.0006) [2023-12-26 20:27:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 20070.4, 300 sec: 19577.5). Total num frames: 358727680. Throughput: 0: 9921.4, 1: 10182.8. Samples: 358715584. Policy #0 lag: (min: 14.0, avg: 31.0, max: 32.0) [2023-12-26 20:27:51,062][104569] Avg episode reward: [(0, '9080.985'), (1, '8996.953')] [2023-12-26 20:27:51,297][105620] Updated weights for policy 1, policy_version 700961 (0.0008) [2023-12-26 20:27:51,356][105620] Updated weights for policy 1, policy_version 700971 (0.0009) [2023-12-26 20:27:51,419][105620] Updated weights for policy 1, policy_version 700981 (0.0009) [2023-12-26 20:27:51,484][105692] Updated weights for policy 0, policy_version 700138 (0.0006) [2023-12-26 20:27:51,541][105692] Updated weights for policy 0, policy_version 700148 (0.0010) [2023-12-26 20:27:51,597][105692] Updated weights for policy 0, policy_version 700158 (0.0008) [2023-12-26 20:27:51,655][105692] Updated weights for policy 0, policy_version 700168 (0.0009) [2023-12-26 20:27:52,171][105620] Updated weights for policy 1, policy_version 700991 (0.0007) [2023-12-26 20:27:52,223][105620] Updated weights for policy 1, policy_version 701001 (0.0006) [2023-12-26 20:27:52,291][105620] Updated weights for policy 1, policy_version 701011 (0.0007) [2023-12-26 20:27:52,442][105692] Updated weights for policy 0, policy_version 700178 (0.0006) [2023-12-26 20:27:52,491][105692] Updated weights for policy 0, policy_version 700188 (0.0005) [2023-12-26 20:27:52,546][105692] Updated weights for policy 0, policy_version 700198 (0.0006) [2023-12-26 20:27:53,068][105620] Updated weights for policy 1, policy_version 701021 (0.0008) [2023-12-26 20:27:53,118][105692] Updated weights for policy 0, policy_version 700208 (0.0005) [2023-12-26 20:27:53,130][105620] Updated weights for policy 1, policy_version 701031 (0.0009) [2023-12-26 20:27:53,173][105692] Updated weights for policy 0, policy_version 700218 (0.0006) [2023-12-26 20:27:53,183][105620] Updated weights for policy 1, policy_version 701041 (0.0009) [2023-12-26 20:27:53,230][105692] Updated weights for policy 0, policy_version 700228 (0.0005) [2023-12-26 20:27:53,763][105692] Updated weights for policy 0, policy_version 700238 (0.0005) [2023-12-26 20:27:53,791][105620] Updated weights for policy 1, policy_version 701051 (0.0010) [2023-12-26 20:27:53,826][105692] Updated weights for policy 0, policy_version 700248 (0.0009) [2023-12-26 20:27:53,840][105620] Updated weights for policy 1, policy_version 701061 (0.0010) [2023-12-26 20:27:53,875][105692] Updated weights for policy 0, policy_version 700258 (0.0011) [2023-12-26 20:27:53,888][105620] Updated weights for policy 1, policy_version 701071 (0.0010) [2023-12-26 20:27:54,569][105692] Updated weights for policy 0, policy_version 700268 (0.0010) [2023-12-26 20:27:54,624][105692] Updated weights for policy 0, policy_version 700278 (0.0009) [2023-12-26 20:27:54,653][105620] Updated weights for policy 1, policy_version 701081 (0.0010) [2023-12-26 20:27:54,686][105692] Updated weights for policy 0, policy_version 700288 (0.0010) [2023-12-26 20:27:54,704][105620] Updated weights for policy 1, policy_version 701091 (0.0010) [2023-12-26 20:27:54,759][105620] Updated weights for policy 1, policy_version 701101 (0.0010) [2023-12-26 20:27:54,821][105620] Updated weights for policy 1, policy_version 701111 (0.0010) [2023-12-26 20:27:55,280][105692] Updated weights for policy 0, policy_version 700298 (0.0009) [2023-12-26 20:27:55,344][105692] Updated weights for policy 0, policy_version 700308 (0.0006) [2023-12-26 20:27:55,414][105692] Updated weights for policy 0, policy_version 700318 (0.0010) [2023-12-26 20:27:55,479][105692] Updated weights for policy 0, policy_version 700328 (0.0007) [2023-12-26 20:27:55,566][105620] Updated weights for policy 1, policy_version 701121 (0.0010) [2023-12-26 20:27:55,626][105620] Updated weights for policy 1, policy_version 701131 (0.0010) [2023-12-26 20:27:55,682][105620] Updated weights for policy 1, policy_version 701141 (0.0005) [2023-12-26 20:27:56,039][105692] Updated weights for policy 0, policy_version 700338 (0.0005) [2023-12-26 20:27:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 20070.4, 300 sec: 19577.5). Total num frames: 358825984. Throughput: 0: 10111.3, 1: 10104.4. Samples: 358837960. Policy #0 lag: (min: 14.0, avg: 31.0, max: 32.0) [2023-12-26 20:27:56,062][104569] Avg episode reward: [(0, '9080.598'), (1, '8900.878')] [2023-12-26 20:27:56,098][105692] Updated weights for policy 0, policy_version 700348 (0.0005) [2023-12-26 20:27:56,154][105692] Updated weights for policy 0, policy_version 700358 (0.0005) [2023-12-26 20:27:56,375][105620] Updated weights for policy 1, policy_version 701151 (0.0009) [2023-12-26 20:27:56,419][105620] Updated weights for policy 1, policy_version 701161 (0.0010) [2023-12-26 20:27:56,466][105620] Updated weights for policy 1, policy_version 701171 (0.0010) [2023-12-26 20:27:56,650][105692] Updated weights for policy 0, policy_version 700368 (0.0008) [2023-12-26 20:27:56,697][105692] Updated weights for policy 0, policy_version 700378 (0.0010) [2023-12-26 20:27:56,748][105692] Updated weights for policy 0, policy_version 700388 (0.0010) [2023-12-26 20:27:57,183][105620] Updated weights for policy 1, policy_version 701181 (0.0010) [2023-12-26 20:27:57,243][105620] Updated weights for policy 1, policy_version 701191 (0.0010) [2023-12-26 20:27:57,304][105620] Updated weights for policy 1, policy_version 701201 (0.0010) [2023-12-26 20:27:57,517][105692] Updated weights for policy 0, policy_version 700398 (0.0009) [2023-12-26 20:27:57,568][105692] Updated weights for policy 0, policy_version 700408 (0.0010) [2023-12-26 20:27:57,615][105692] Updated weights for policy 0, policy_version 700418 (0.0010) [2023-12-26 20:27:57,907][105620] Updated weights for policy 1, policy_version 701211 (0.0008) [2023-12-26 20:27:57,952][105620] Updated weights for policy 1, policy_version 701221 (0.0010) [2023-12-26 20:27:57,996][105620] Updated weights for policy 1, policy_version 701231 (0.0010) [2023-12-26 20:27:58,364][105692] Updated weights for policy 0, policy_version 700428 (0.0007) [2023-12-26 20:27:58,430][105692] Updated weights for policy 0, policy_version 700438 (0.0009) [2023-12-26 20:27:58,492][105692] Updated weights for policy 0, policy_version 700448 (0.0009) [2023-12-26 20:27:58,872][105620] Updated weights for policy 1, policy_version 701241 (0.0006) [2023-12-26 20:27:58,944][105620] Updated weights for policy 1, policy_version 701251 (0.0009) [2023-12-26 20:27:59,001][105620] Updated weights for policy 1, policy_version 701261 (0.0009) [2023-12-26 20:27:59,053][105620] Updated weights for policy 1, policy_version 701271 (0.0009) [2023-12-26 20:27:59,403][105692] Updated weights for policy 0, policy_version 700458 (0.0009) [2023-12-26 20:27:59,454][105692] Updated weights for policy 0, policy_version 700468 (0.0007) [2023-12-26 20:27:59,508][105692] Updated weights for policy 0, policy_version 700478 (0.0009) [2023-12-26 20:27:59,564][105692] Updated weights for policy 0, policy_version 700488 (0.0010) [2023-12-26 20:27:59,857][105620] Updated weights for policy 1, policy_version 701281 (0.0008) [2023-12-26 20:27:59,919][105620] Updated weights for policy 1, policy_version 701291 (0.0006) [2023-12-26 20:27:59,984][105620] Updated weights for policy 1, policy_version 701301 (0.0009) [2023-12-26 20:28:00,426][105692] Updated weights for policy 0, policy_version 700498 (0.0009) [2023-12-26 20:28:00,495][105692] Updated weights for policy 0, policy_version 700508 (0.0010) [2023-12-26 20:28:00,559][105692] Updated weights for policy 0, policy_version 700518 (0.0010) [2023-12-26 20:28:00,579][105620] Updated weights for policy 1, policy_version 701311 (0.0006) [2023-12-26 20:28:00,623][105620] Updated weights for policy 1, policy_version 701321 (0.0005) [2023-12-26 20:28:00,676][105620] Updated weights for policy 1, policy_version 701331 (0.0009) [2023-12-26 20:28:01,062][104569] Fps is (10 sec: 19660.4, 60 sec: 20070.3, 300 sec: 19577.5). Total num frames: 358924288. Throughput: 0: 10152.3, 1: 10047.1. Samples: 358897984. Policy #0 lag: (min: 14.0, avg: 31.0, max: 32.0) [2023-12-26 20:28:01,063][104569] Avg episode reward: [(0, '9168.896'), (1, '8992.467')] [2023-12-26 20:28:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000700520_179363840.pth... [2023-12-26 20:28:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000701336_179560448.pth... [2023-12-26 20:28:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000699368_179068928.pth [2023-12-26 20:28:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000700184_179265536.pth [2023-12-26 20:28:01,258][105692] Updated weights for policy 0, policy_version 700528 (0.0008) [2023-12-26 20:28:01,317][105692] Updated weights for policy 0, policy_version 700538 (0.0010) [2023-12-26 20:28:01,355][105620] Updated weights for policy 1, policy_version 701341 (0.0010) [2023-12-26 20:28:01,379][105692] Updated weights for policy 0, policy_version 700548 (0.0010) [2023-12-26 20:28:01,426][105620] Updated weights for policy 1, policy_version 701351 (0.0011) [2023-12-26 20:28:01,485][105620] Updated weights for policy 1, policy_version 701361 (0.0010) [2023-12-26 20:28:02,010][105692] Updated weights for policy 0, policy_version 700558 (0.0007) [2023-12-26 20:28:02,068][105692] Updated weights for policy 0, policy_version 700568 (0.0009) [2023-12-26 20:28:02,163][105692] Updated weights for policy 0, policy_version 700578 (0.0007) [2023-12-26 20:28:02,173][105620] Updated weights for policy 1, policy_version 701371 (0.0010) [2023-12-26 20:28:02,229][105620] Updated weights for policy 1, policy_version 701381 (0.0010) [2023-12-26 20:28:02,292][105620] Updated weights for policy 1, policy_version 701391 (0.0011) [2023-12-26 20:28:02,690][105692] Updated weights for policy 0, policy_version 700588 (0.0011) [2023-12-26 20:28:02,754][105692] Updated weights for policy 0, policy_version 700598 (0.0010) [2023-12-26 20:28:02,813][105692] Updated weights for policy 0, policy_version 700608 (0.0009) [2023-12-26 20:28:02,975][105620] Updated weights for policy 1, policy_version 701401 (0.0011) [2023-12-26 20:28:03,024][105620] Updated weights for policy 1, policy_version 701411 (0.0009) [2023-12-26 20:28:03,079][105620] Updated weights for policy 1, policy_version 701421 (0.0009) [2023-12-26 20:28:03,126][105620] Updated weights for policy 1, policy_version 701431 (0.0005) [2023-12-26 20:28:03,414][105692] Updated weights for policy 0, policy_version 700618 (0.0006) [2023-12-26 20:28:03,465][105692] Updated weights for policy 0, policy_version 700628 (0.0010) [2023-12-26 20:28:03,516][105692] Updated weights for policy 0, policy_version 700638 (0.0010) [2023-12-26 20:28:03,726][105620] Updated weights for policy 1, policy_version 701441 (0.0005) [2023-12-26 20:28:03,780][105620] Updated weights for policy 1, policy_version 701451 (0.0005) [2023-12-26 20:28:03,840][105620] Updated weights for policy 1, policy_version 701461 (0.0009) [2023-12-26 20:28:04,189][105692] Updated weights for policy 0, policy_version 700649 (0.0010) [2023-12-26 20:28:04,252][105692] Updated weights for policy 0, policy_version 700659 (0.0010) [2023-12-26 20:28:04,308][105692] Updated weights for policy 0, policy_version 700669 (0.0010) [2023-12-26 20:28:04,361][105692] Updated weights for policy 0, policy_version 700679 (0.0010) [2023-12-26 20:28:04,603][105620] Updated weights for policy 1, policy_version 701471 (0.0008) [2023-12-26 20:28:04,650][105620] Updated weights for policy 1, policy_version 701481 (0.0008) [2023-12-26 20:28:04,702][105620] Updated weights for policy 1, policy_version 701492 (0.0010) [2023-12-26 20:28:05,002][105692] Updated weights for policy 0, policy_version 700689 (0.0006) [2023-12-26 20:28:05,058][105692] Updated weights for policy 0, policy_version 700699 (0.0006) [2023-12-26 20:28:05,117][105692] Updated weights for policy 0, policy_version 700709 (0.0005) [2023-12-26 20:28:05,455][105620] Updated weights for policy 1, policy_version 701502 (0.0007) [2023-12-26 20:28:05,504][105620] Updated weights for policy 1, policy_version 701512 (0.0005) [2023-12-26 20:28:05,550][105620] Updated weights for policy 1, policy_version 701522 (0.0005) [2023-12-26 20:28:05,639][105692] Updated weights for policy 0, policy_version 700719 (0.0005) [2023-12-26 20:28:05,692][105692] Updated weights for policy 0, policy_version 700729 (0.0005) [2023-12-26 20:28:05,741][105692] Updated weights for policy 0, policy_version 700739 (0.0005) [2023-12-26 20:28:06,062][104569] Fps is (10 sec: 20480.4, 60 sec: 20207.0, 300 sec: 19633.0). Total num frames: 359030784. Throughput: 0: 10027.6, 1: 9960.1. Samples: 359017068. Policy #0 lag: (min: 14.0, avg: 31.0, max: 32.0) [2023-12-26 20:28:06,062][104569] Avg episode reward: [(0, '8987.265'), (1, '9082.015')] [2023-12-26 20:28:06,191][105620] Updated weights for policy 1, policy_version 701532 (0.0006) [2023-12-26 20:28:06,249][105620] Updated weights for policy 1, policy_version 701542 (0.0008) [2023-12-26 20:28:06,293][105620] Updated weights for policy 1, policy_version 701552 (0.0008) [2023-12-26 20:28:06,423][105692] Updated weights for policy 0, policy_version 700749 (0.0005) [2023-12-26 20:28:06,491][105692] Updated weights for policy 0, policy_version 700759 (0.0005) [2023-12-26 20:28:06,548][105692] Updated weights for policy 0, policy_version 700769 (0.0007) [2023-12-26 20:28:07,076][105620] Updated weights for policy 1, policy_version 701562 (0.0008) [2023-12-26 20:28:07,136][105620] Updated weights for policy 1, policy_version 701572 (0.0008) [2023-12-26 20:28:07,186][105620] Updated weights for policy 1, policy_version 701582 (0.0008) [2023-12-26 20:28:07,220][105692] Updated weights for policy 0, policy_version 700779 (0.0010) [2023-12-26 20:28:07,243][105620] Updated weights for policy 1, policy_version 701592 (0.0009) [2023-12-26 20:28:07,279][105692] Updated weights for policy 0, policy_version 700789 (0.0010) [2023-12-26 20:28:07,332][105692] Updated weights for policy 0, policy_version 700799 (0.0008) [2023-12-26 20:28:07,961][105692] Updated weights for policy 0, policy_version 700809 (0.0010) [2023-12-26 20:28:08,019][105692] Updated weights for policy 0, policy_version 700819 (0.0008) [2023-12-26 20:28:08,034][105620] Updated weights for policy 1, policy_version 701602 (0.0010) [2023-12-26 20:28:08,078][105692] Updated weights for policy 0, policy_version 700829 (0.0009) [2023-12-26 20:28:08,093][105620] Updated weights for policy 1, policy_version 701612 (0.0006) [2023-12-26 20:28:08,126][105692] Updated weights for policy 0, policy_version 700839 (0.0009) [2023-12-26 20:28:08,160][105620] Updated weights for policy 1, policy_version 701622 (0.0007) [2023-12-26 20:28:08,813][105692] Updated weights for policy 0, policy_version 700849 (0.0009) [2023-12-26 20:28:08,872][105620] Updated weights for policy 1, policy_version 701632 (0.0007) [2023-12-26 20:28:08,874][105692] Updated weights for policy 0, policy_version 700859 (0.0011) [2023-12-26 20:28:08,933][105692] Updated weights for policy 0, policy_version 700869 (0.0011) [2023-12-26 20:28:08,939][105620] Updated weights for policy 1, policy_version 701642 (0.0008) [2023-12-26 20:28:08,995][105620] Updated weights for policy 1, policy_version 701652 (0.0010) [2023-12-26 20:28:09,646][105692] Updated weights for policy 0, policy_version 700879 (0.0010) [2023-12-26 20:28:09,707][105620] Updated weights for policy 1, policy_version 701662 (0.0011) [2023-12-26 20:28:09,709][105692] Updated weights for policy 0, policy_version 700889 (0.0010) [2023-12-26 20:28:09,766][105620] Updated weights for policy 1, policy_version 701672 (0.0011) [2023-12-26 20:28:09,768][105692] Updated weights for policy 0, policy_version 700899 (0.0010) [2023-12-26 20:28:09,822][105620] Updated weights for policy 1, policy_version 701682 (0.0010) [2023-12-26 20:28:10,501][105692] Updated weights for policy 0, policy_version 700909 (0.0010) [2023-12-26 20:28:10,557][105692] Updated weights for policy 0, policy_version 700919 (0.0010) [2023-12-26 20:28:10,589][105620] Updated weights for policy 1, policy_version 701692 (0.0010) [2023-12-26 20:28:10,610][105692] Updated weights for policy 0, policy_version 700929 (0.0011) [2023-12-26 20:28:10,652][105620] Updated weights for policy 1, policy_version 701702 (0.0010) [2023-12-26 20:28:10,713][105620] Updated weights for policy 1, policy_version 701712 (0.0010) [2023-12-26 20:28:11,062][104569] Fps is (10 sec: 20480.3, 60 sec: 20070.4, 300 sec: 19633.0). Total num frames: 359129088. Throughput: 0: 10130.6, 1: 9931.5. Samples: 359137720. Policy #0 lag: (min: 14.0, avg: 31.0, max: 32.0) [2023-12-26 20:28:11,062][104569] Avg episode reward: [(0, '8670.107'), (1, '8989.940')] [2023-12-26 20:28:11,377][105620] Updated weights for policy 1, policy_version 701722 (0.0009) [2023-12-26 20:28:11,379][105692] Updated weights for policy 0, policy_version 700939 (0.0010) [2023-12-26 20:28:11,439][105620] Updated weights for policy 1, policy_version 701732 (0.0011) [2023-12-26 20:28:11,441][105692] Updated weights for policy 0, policy_version 700949 (0.0006) [2023-12-26 20:28:11,502][105692] Updated weights for policy 0, policy_version 700959 (0.0007) [2023-12-26 20:28:11,503][105620] Updated weights for policy 1, policy_version 701742 (0.0010) [2023-12-26 20:28:11,566][105620] Updated weights for policy 1, policy_version 701752 (0.0006) [2023-12-26 20:28:12,240][105692] Updated weights for policy 0, policy_version 700969 (0.0011) [2023-12-26 20:28:12,264][105620] Updated weights for policy 1, policy_version 701762 (0.0010) [2023-12-26 20:28:12,305][105692] Updated weights for policy 0, policy_version 700979 (0.0011) [2023-12-26 20:28:12,331][105620] Updated weights for policy 1, policy_version 701772 (0.0010) [2023-12-26 20:28:12,372][105692] Updated weights for policy 0, policy_version 700989 (0.0010) [2023-12-26 20:28:12,394][105620] Updated weights for policy 1, policy_version 701782 (0.0013) [2023-12-26 20:28:12,432][105692] Updated weights for policy 0, policy_version 700999 (0.0011) [2023-12-26 20:28:13,112][105620] Updated weights for policy 1, policy_version 701792 (0.0007) [2023-12-26 20:28:13,178][105620] Updated weights for policy 1, policy_version 701802 (0.0005) [2023-12-26 20:28:13,184][105692] Updated weights for policy 0, policy_version 701009 (0.0011) [2023-12-26 20:28:13,234][105620] Updated weights for policy 1, policy_version 701812 (0.0006) [2023-12-26 20:28:13,236][105692] Updated weights for policy 0, policy_version 701019 (0.0010) [2023-12-26 20:28:13,290][105692] Updated weights for policy 0, policy_version 701029 (0.0010) [2023-12-26 20:28:13,879][105692] Updated weights for policy 0, policy_version 701039 (0.0007) [2023-12-26 20:28:13,935][105692] Updated weights for policy 0, policy_version 701049 (0.0005) [2023-12-26 20:28:13,936][105620] Updated weights for policy 1, policy_version 701822 (0.0010) [2023-12-26 20:28:13,999][105692] Updated weights for policy 0, policy_version 701059 (0.0005) [2023-12-26 20:28:14,002][105620] Updated weights for policy 1, policy_version 701832 (0.0010) [2023-12-26 20:28:14,067][105620] Updated weights for policy 1, policy_version 701842 (0.0010) [2023-12-26 20:28:14,514][105692] Updated weights for policy 0, policy_version 701069 (0.0008) [2023-12-26 20:28:14,570][105692] Updated weights for policy 0, policy_version 701079 (0.0011) [2023-12-26 20:28:14,614][105692] Updated weights for policy 0, policy_version 701089 (0.0010) [2023-12-26 20:28:14,775][105620] Updated weights for policy 1, policy_version 701852 (0.0010) [2023-12-26 20:28:14,832][105620] Updated weights for policy 1, policy_version 701862 (0.0010) [2023-12-26 20:28:14,899][105620] Updated weights for policy 1, policy_version 701872 (0.0011) [2023-12-26 20:28:15,263][105692] Updated weights for policy 0, policy_version 701099 (0.0009) [2023-12-26 20:28:15,318][105692] Updated weights for policy 0, policy_version 701109 (0.0011) [2023-12-26 20:28:15,379][105692] Updated weights for policy 0, policy_version 701119 (0.0011) [2023-12-26 20:28:15,643][105620] Updated weights for policy 1, policy_version 701882 (0.0010) [2023-12-26 20:28:15,691][105620] Updated weights for policy 1, policy_version 701892 (0.0010) [2023-12-26 20:28:15,743][105620] Updated weights for policy 1, policy_version 701902 (0.0010) [2023-12-26 20:28:15,788][105620] Updated weights for policy 1, policy_version 701912 (0.0010) [2023-12-26 20:28:15,941][105692] Updated weights for policy 0, policy_version 701129 (0.0010) [2023-12-26 20:28:15,987][105692] Updated weights for policy 0, policy_version 701139 (0.0005) [2023-12-26 20:28:16,038][105692] Updated weights for policy 0, policy_version 701149 (0.0005) [2023-12-26 20:28:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 20070.5, 300 sec: 19633.0). Total num frames: 359227392. Throughput: 0: 9955.3, 1: 9875.4. Samples: 359194700. Policy #0 lag: (min: 14.0, avg: 31.0, max: 32.0) [2023-12-26 20:28:16,062][104569] Avg episode reward: [(0, '8708.108'), (1, '9262.847')] [2023-12-26 20:28:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000701912_179707904.pth... [2023-12-26 20:28:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000700760_179412992.pth [2023-12-26 20:28:16,096][105692] Updated weights for policy 0, policy_version 701159 (0.0010) [2023-12-26 20:28:16,098][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000701160_179527680.pth... [2023-12-26 20:28:16,101][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000699944_179216384.pth [2023-12-26 20:28:16,476][105620] Updated weights for policy 1, policy_version 701922 (0.0008) [2023-12-26 20:28:16,547][105620] Updated weights for policy 1, policy_version 701932 (0.0009) [2023-12-26 20:28:16,616][105620] Updated weights for policy 1, policy_version 701942 (0.0009) [2023-12-26 20:28:16,750][105692] Updated weights for policy 0, policy_version 701169 (0.0006) [2023-12-26 20:28:16,808][105692] Updated weights for policy 0, policy_version 701179 (0.0005) [2023-12-26 20:28:16,866][105692] Updated weights for policy 0, policy_version 701189 (0.0005) [2023-12-26 20:28:17,342][105620] Updated weights for policy 1, policy_version 701952 (0.0008) [2023-12-26 20:28:17,409][105620] Updated weights for policy 1, policy_version 701962 (0.0005) [2023-12-26 20:28:17,463][105692] Updated weights for policy 0, policy_version 701199 (0.0009) [2023-12-26 20:28:17,476][105620] Updated weights for policy 1, policy_version 701972 (0.0007) [2023-12-26 20:28:17,515][105692] Updated weights for policy 0, policy_version 701209 (0.0011) [2023-12-26 20:28:17,581][105692] Updated weights for policy 0, policy_version 701219 (0.0011) [2023-12-26 20:28:18,077][105620] Updated weights for policy 1, policy_version 701982 (0.0006) [2023-12-26 20:28:18,137][105620] Updated weights for policy 1, policy_version 701992 (0.0006) [2023-12-26 20:28:18,196][105620] Updated weights for policy 1, policy_version 702002 (0.0005) [2023-12-26 20:28:18,368][105692] Updated weights for policy 0, policy_version 701229 (0.0010) [2023-12-26 20:28:18,423][105692] Updated weights for policy 0, policy_version 701239 (0.0008) [2023-12-26 20:28:18,474][105692] Updated weights for policy 0, policy_version 701249 (0.0009) [2023-12-26 20:28:18,865][105620] Updated weights for policy 1, policy_version 702012 (0.0007) [2023-12-26 20:28:18,927][105620] Updated weights for policy 1, policy_version 702022 (0.0009) [2023-12-26 20:28:18,994][105620] Updated weights for policy 1, policy_version 702032 (0.0008) [2023-12-26 20:28:19,192][105692] Updated weights for policy 0, policy_version 701259 (0.0009) [2023-12-26 20:28:19,251][105692] Updated weights for policy 0, policy_version 701269 (0.0009) [2023-12-26 20:28:19,310][105692] Updated weights for policy 0, policy_version 701279 (0.0009) [2023-12-26 20:28:19,702][105620] Updated weights for policy 1, policy_version 702042 (0.0009) [2023-12-26 20:28:19,757][105620] Updated weights for policy 1, policy_version 702052 (0.0008) [2023-12-26 20:28:19,809][105620] Updated weights for policy 1, policy_version 702062 (0.0009) [2023-12-26 20:28:19,864][105620] Updated weights for policy 1, policy_version 702072 (0.0009) [2023-12-26 20:28:20,156][105692] Updated weights for policy 0, policy_version 701289 (0.0008) [2023-12-26 20:28:20,216][105692] Updated weights for policy 0, policy_version 701299 (0.0010) [2023-12-26 20:28:20,277][105692] Updated weights for policy 0, policy_version 701309 (0.0010) [2023-12-26 20:28:20,335][105692] Updated weights for policy 0, policy_version 701319 (0.0010) [2023-12-26 20:28:20,605][105620] Updated weights for policy 1, policy_version 702082 (0.0009) [2023-12-26 20:28:20,662][105620] Updated weights for policy 1, policy_version 702092 (0.0008) [2023-12-26 20:28:20,716][105620] Updated weights for policy 1, policy_version 702102 (0.0008) [2023-12-26 20:28:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 20070.4, 300 sec: 19633.0). Total num frames: 359325696. Throughput: 0: 9979.5, 1: 9850.9. Samples: 359317672. Policy #0 lag: (min: 14.0, avg: 31.0, max: 32.0) [2023-12-26 20:28:21,062][104569] Avg episode reward: [(0, '8605.148'), (1, '9263.796')] [2023-12-26 20:28:21,123][105692] Updated weights for policy 0, policy_version 701329 (0.0009) [2023-12-26 20:28:21,183][105692] Updated weights for policy 0, policy_version 701339 (0.0009) [2023-12-26 20:28:21,244][105692] Updated weights for policy 0, policy_version 701349 (0.0008) [2023-12-26 20:28:21,476][105620] Updated weights for policy 1, policy_version 702112 (0.0006) [2023-12-26 20:28:21,547][105620] Updated weights for policy 1, policy_version 702122 (0.0006) [2023-12-26 20:28:21,616][105620] Updated weights for policy 1, policy_version 702132 (0.0006) [2023-12-26 20:28:21,981][105692] Updated weights for policy 0, policy_version 701359 (0.0006) [2023-12-26 20:28:22,050][105692] Updated weights for policy 0, policy_version 701369 (0.0006) [2023-12-26 20:28:22,108][105692] Updated weights for policy 0, policy_version 701379 (0.0008) [2023-12-26 20:28:22,354][105620] Updated weights for policy 1, policy_version 702142 (0.0010) [2023-12-26 20:28:22,417][105620] Updated weights for policy 1, policy_version 702152 (0.0008) [2023-12-26 20:28:22,480][105620] Updated weights for policy 1, policy_version 702162 (0.0008) [2023-12-26 20:28:22,728][105692] Updated weights for policy 0, policy_version 701389 (0.0009) [2023-12-26 20:28:22,787][105692] Updated weights for policy 0, policy_version 701399 (0.0006) [2023-12-26 20:28:22,854][105692] Updated weights for policy 0, policy_version 701409 (0.0005) [2023-12-26 20:28:23,090][105620] Updated weights for policy 1, policy_version 702172 (0.0008) [2023-12-26 20:28:23,141][105620] Updated weights for policy 1, policy_version 702182 (0.0007) [2023-12-26 20:28:23,203][105620] Updated weights for policy 1, policy_version 702192 (0.0009) [2023-12-26 20:28:23,546][105692] Updated weights for policy 0, policy_version 701419 (0.0007) [2023-12-26 20:28:23,594][105692] Updated weights for policy 0, policy_version 701429 (0.0008) [2023-12-26 20:28:23,648][105692] Updated weights for policy 0, policy_version 701439 (0.0010) [2023-12-26 20:28:23,879][105620] Updated weights for policy 1, policy_version 702202 (0.0008) [2023-12-26 20:28:23,937][105620] Updated weights for policy 1, policy_version 702212 (0.0006) [2023-12-26 20:28:23,996][105620] Updated weights for policy 1, policy_version 702222 (0.0009) [2023-12-26 20:28:24,057][105620] Updated weights for policy 1, policy_version 702232 (0.0005) [2023-12-26 20:28:24,481][105692] Updated weights for policy 0, policy_version 701449 (0.0009) [2023-12-26 20:28:24,536][105692] Updated weights for policy 0, policy_version 701459 (0.0008) [2023-12-26 20:28:24,598][105692] Updated weights for policy 0, policy_version 701469 (0.0009) [2023-12-26 20:28:24,667][105692] Updated weights for policy 0, policy_version 701479 (0.0009) [2023-12-26 20:28:24,697][105620] Updated weights for policy 1, policy_version 702242 (0.0007) [2023-12-26 20:28:24,745][105620] Updated weights for policy 1, policy_version 702252 (0.0010) [2023-12-26 20:28:24,803][105620] Updated weights for policy 1, policy_version 702262 (0.0010) [2023-12-26 20:28:25,348][105692] Updated weights for policy 0, policy_version 701489 (0.0006) [2023-12-26 20:28:25,398][105692] Updated weights for policy 0, policy_version 701499 (0.0006) [2023-12-26 20:28:25,453][105692] Updated weights for policy 0, policy_version 701509 (0.0008) [2023-12-26 20:28:25,551][105620] Updated weights for policy 1, policy_version 702272 (0.0010) [2023-12-26 20:28:25,606][105620] Updated weights for policy 1, policy_version 702282 (0.0010) [2023-12-26 20:28:25,671][105620] Updated weights for policy 1, policy_version 702292 (0.0010) [2023-12-26 20:28:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 20070.4, 300 sec: 19633.0). Total num frames: 359424000. Throughput: 0: 10037.8, 1: 9784.3. Samples: 359434188. Policy #0 lag: (min: 14.0, avg: 31.0, max: 32.0) [2023-12-26 20:28:26,062][104569] Avg episode reward: [(0, '9167.036'), (1, '9263.557')] [2023-12-26 20:28:26,174][105692] Updated weights for policy 0, policy_version 701519 (0.0008) [2023-12-26 20:28:26,229][105692] Updated weights for policy 0, policy_version 701529 (0.0008) [2023-12-26 20:28:26,280][105692] Updated weights for policy 0, policy_version 701539 (0.0008) [2023-12-26 20:28:26,361][105620] Updated weights for policy 1, policy_version 702302 (0.0011) [2023-12-26 20:28:26,410][105620] Updated weights for policy 1, policy_version 702312 (0.0010) [2023-12-26 20:28:26,462][105620] Updated weights for policy 1, policy_version 702322 (0.0010) [2023-12-26 20:28:27,003][105692] Updated weights for policy 0, policy_version 701549 (0.0009) [2023-12-26 20:28:27,061][105692] Updated weights for policy 0, policy_version 701559 (0.0010) [2023-12-26 20:28:27,110][105620] Updated weights for policy 1, policy_version 702332 (0.0008) [2023-12-26 20:28:27,118][105692] Updated weights for policy 0, policy_version 701569 (0.0010) [2023-12-26 20:28:27,156][105620] Updated weights for policy 1, policy_version 702342 (0.0005) [2023-12-26 20:28:27,208][105620] Updated weights for policy 1, policy_version 702352 (0.0005) [2023-12-26 20:28:27,764][105692] Updated weights for policy 0, policy_version 701579 (0.0010) [2023-12-26 20:28:27,818][105692] Updated weights for policy 0, policy_version 701590 (0.0008) [2023-12-26 20:28:27,862][105692] Updated weights for policy 0, policy_version 701600 (0.0007) [2023-12-26 20:28:27,889][105620] Updated weights for policy 1, policy_version 702362 (0.0007) [2023-12-26 20:28:27,947][105620] Updated weights for policy 1, policy_version 702372 (0.0010) [2023-12-26 20:28:28,007][105620] Updated weights for policy 1, policy_version 702382 (0.0011) [2023-12-26 20:28:28,058][105620] Updated weights for policy 1, policy_version 702392 (0.0010) [2023-12-26 20:28:28,459][105692] Updated weights for policy 0, policy_version 701610 (0.0007) [2023-12-26 20:28:28,517][105692] Updated weights for policy 0, policy_version 701620 (0.0010) [2023-12-26 20:28:28,581][105692] Updated weights for policy 0, policy_version 701630 (0.0009) [2023-12-26 20:28:28,635][105692] Updated weights for policy 0, policy_version 701640 (0.0010) [2023-12-26 20:28:28,743][105620] Updated weights for policy 1, policy_version 702402 (0.0010) [2023-12-26 20:28:28,813][105620] Updated weights for policy 1, policy_version 702412 (0.0011) [2023-12-26 20:28:28,883][105620] Updated weights for policy 1, policy_version 702422 (0.0011) [2023-12-26 20:28:29,432][105692] Updated weights for policy 0, policy_version 701650 (0.0008) [2023-12-26 20:28:29,499][105692] Updated weights for policy 0, policy_version 701660 (0.0008) [2023-12-26 20:28:29,563][105692] Updated weights for policy 0, policy_version 701670 (0.0007) [2023-12-26 20:28:29,588][105620] Updated weights for policy 1, policy_version 702432 (0.0010) [2023-12-26 20:28:29,643][105620] Updated weights for policy 1, policy_version 702442 (0.0010) [2023-12-26 20:28:29,694][105620] Updated weights for policy 1, policy_version 702452 (0.0009) [2023-12-26 20:28:30,365][105620] Updated weights for policy 1, policy_version 702462 (0.0009) [2023-12-26 20:28:30,370][105692] Updated weights for policy 0, policy_version 701680 (0.0008) [2023-12-26 20:28:30,422][105692] Updated weights for policy 0, policy_version 701690 (0.0010) [2023-12-26 20:28:30,425][105620] Updated weights for policy 1, policy_version 702472 (0.0006) [2023-12-26 20:28:30,479][105692] Updated weights for policy 0, policy_version 701700 (0.0010) [2023-12-26 20:28:30,486][105620] Updated weights for policy 1, policy_version 702482 (0.0008) [2023-12-26 20:28:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 359522304. Throughput: 0: 10110.5, 1: 9839.5. Samples: 359496128. Policy #0 lag: (min: 14.0, avg: 31.0, max: 32.0) [2023-12-26 20:28:31,062][104569] Avg episode reward: [(0, '9350.965'), (1, '9264.400')] [2023-12-26 20:28:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000701704_179666944.pth... [2023-12-26 20:28:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000702488_179855360.pth... [2023-12-26 20:28:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000700520_179363840.pth [2023-12-26 20:28:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000701336_179560448.pth [2023-12-26 20:28:31,121][105620] Updated weights for policy 1, policy_version 702492 (0.0008) [2023-12-26 20:28:31,179][105620] Updated weights for policy 1, policy_version 702502 (0.0008) [2023-12-26 20:28:31,201][105692] Updated weights for policy 0, policy_version 701710 (0.0009) [2023-12-26 20:28:31,231][105620] Updated weights for policy 1, policy_version 702512 (0.0006) [2023-12-26 20:28:31,258][105692] Updated weights for policy 0, policy_version 701720 (0.0010) [2023-12-26 20:28:31,311][105692] Updated weights for policy 0, policy_version 701730 (0.0010) [2023-12-26 20:28:31,884][105620] Updated weights for policy 1, policy_version 702522 (0.0007) [2023-12-26 20:28:31,936][105620] Updated weights for policy 1, policy_version 702532 (0.0008) [2023-12-26 20:28:32,001][105620] Updated weights for policy 1, policy_version 702542 (0.0008) [2023-12-26 20:28:32,020][105692] Updated weights for policy 0, policy_version 701740 (0.0011) [2023-12-26 20:28:32,059][105620] Updated weights for policy 1, policy_version 702552 (0.0009) [2023-12-26 20:28:32,072][105692] Updated weights for policy 0, policy_version 701750 (0.0010) [2023-12-26 20:28:32,133][105692] Updated weights for policy 0, policy_version 701760 (0.0007) [2023-12-26 20:28:32,744][105620] Updated weights for policy 1, policy_version 702562 (0.0006) [2023-12-26 20:28:32,799][105620] Updated weights for policy 1, policy_version 702572 (0.0005) [2023-12-26 20:28:32,855][105620] Updated weights for policy 1, policy_version 702582 (0.0005) [2023-12-26 20:28:32,869][105692] Updated weights for policy 0, policy_version 701770 (0.0006) [2023-12-26 20:28:32,919][105692] Updated weights for policy 0, policy_version 701780 (0.0009) [2023-12-26 20:28:32,977][105692] Updated weights for policy 0, policy_version 701792 (0.0011) [2023-12-26 20:28:33,371][105620] Updated weights for policy 1, policy_version 702592 (0.0005) [2023-12-26 20:28:33,416][105620] Updated weights for policy 1, policy_version 702602 (0.0005) [2023-12-26 20:28:33,459][105620] Updated weights for policy 1, policy_version 702612 (0.0005) [2023-12-26 20:28:33,678][105692] Updated weights for policy 0, policy_version 701802 (0.0009) [2023-12-26 20:28:33,729][105692] Updated weights for policy 0, policy_version 701812 (0.0009) [2023-12-26 20:28:33,792][105692] Updated weights for policy 0, policy_version 701822 (0.0011) [2023-12-26 20:28:33,844][105692] Updated weights for policy 0, policy_version 701832 (0.0010) [2023-12-26 20:28:34,109][105620] Updated weights for policy 1, policy_version 702622 (0.0008) [2023-12-26 20:28:34,172][105620] Updated weights for policy 1, policy_version 702632 (0.0010) [2023-12-26 20:28:34,233][105620] Updated weights for policy 1, policy_version 702642 (0.0009) [2023-12-26 20:28:34,579][105692] Updated weights for policy 0, policy_version 701842 (0.0009) [2023-12-26 20:28:34,644][105692] Updated weights for policy 0, policy_version 701852 (0.0010) [2023-12-26 20:28:34,703][105692] Updated weights for policy 0, policy_version 701862 (0.0010) [2023-12-26 20:28:34,967][105620] Updated weights for policy 1, policy_version 702652 (0.0010) [2023-12-26 20:28:35,026][105620] Updated weights for policy 1, policy_version 702662 (0.0010) [2023-12-26 20:28:35,087][105620] Updated weights for policy 1, policy_version 702672 (0.0010) [2023-12-26 20:28:35,400][105692] Updated weights for policy 0, policy_version 701872 (0.0011) [2023-12-26 20:28:35,456][105692] Updated weights for policy 0, policy_version 701882 (0.0010) [2023-12-26 20:28:35,502][105692] Updated weights for policy 0, policy_version 701892 (0.0008) [2023-12-26 20:28:35,754][105620] Updated weights for policy 1, policy_version 702682 (0.0008) [2023-12-26 20:28:35,805][105620] Updated weights for policy 1, policy_version 702692 (0.0005) [2023-12-26 20:28:35,859][105620] Updated weights for policy 1, policy_version 702702 (0.0008) [2023-12-26 20:28:35,917][105620] Updated weights for policy 1, policy_version 702712 (0.0010) [2023-12-26 20:28:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 359628800. Throughput: 0: 10093.4, 1: 9925.0. Samples: 359616416. Policy #0 lag: (min: 14.0, avg: 31.0, max: 32.0) [2023-12-26 20:28:36,062][104569] Avg episode reward: [(0, '9167.454'), (1, '9082.211')] [2023-12-26 20:28:36,075][105692] Updated weights for policy 0, policy_version 701902 (0.0006) [2023-12-26 20:28:36,141][105692] Updated weights for policy 0, policy_version 701912 (0.0007) [2023-12-26 20:28:36,205][105692] Updated weights for policy 0, policy_version 701922 (0.0006) [2023-12-26 20:28:36,647][105620] Updated weights for policy 1, policy_version 702722 (0.0008) [2023-12-26 20:28:36,718][105620] Updated weights for policy 1, policy_version 702732 (0.0008) [2023-12-26 20:28:36,765][105692] Updated weights for policy 0, policy_version 701932 (0.0007) [2023-12-26 20:28:36,773][105620] Updated weights for policy 1, policy_version 702742 (0.0010) [2023-12-26 20:28:36,823][105692] Updated weights for policy 0, policy_version 701942 (0.0007) [2023-12-26 20:28:36,885][105692] Updated weights for policy 0, policy_version 701952 (0.0006) [2023-12-26 20:28:37,426][105620] Updated weights for policy 1, policy_version 702752 (0.0008) [2023-12-26 20:28:37,477][105692] Updated weights for policy 0, policy_version 701962 (0.0008) [2023-12-26 20:28:37,489][105620] Updated weights for policy 1, policy_version 702762 (0.0005) [2023-12-26 20:28:37,546][105692] Updated weights for policy 0, policy_version 701972 (0.0007) [2023-12-26 20:28:37,552][105620] Updated weights for policy 1, policy_version 702772 (0.0005) [2023-12-26 20:28:37,618][105692] Updated weights for policy 0, policy_version 701982 (0.0006) [2023-12-26 20:28:37,683][105692] Updated weights for policy 0, policy_version 701992 (0.0005) [2023-12-26 20:28:38,143][105620] Updated weights for policy 1, policy_version 702782 (0.0006) [2023-12-26 20:28:38,194][105620] Updated weights for policy 1, policy_version 702792 (0.0007) [2023-12-26 20:28:38,235][105692] Updated weights for policy 0, policy_version 702002 (0.0010) [2023-12-26 20:28:38,260][105620] Updated weights for policy 1, policy_version 702802 (0.0005) [2023-12-26 20:28:38,291][105692] Updated weights for policy 0, policy_version 702012 (0.0010) [2023-12-26 20:28:38,355][105692] Updated weights for policy 0, policy_version 702022 (0.0011) [2023-12-26 20:28:38,882][105620] Updated weights for policy 1, policy_version 702812 (0.0007) [2023-12-26 20:28:38,939][105620] Updated weights for policy 1, policy_version 702822 (0.0008) [2023-12-26 20:28:38,994][105620] Updated weights for policy 1, policy_version 702832 (0.0007) [2023-12-26 20:28:39,113][105692] Updated weights for policy 0, policy_version 702032 (0.0010) [2023-12-26 20:28:39,173][105692] Updated weights for policy 0, policy_version 702042 (0.0010) [2023-12-26 20:28:39,242][105692] Updated weights for policy 0, policy_version 702052 (0.0011) [2023-12-26 20:28:39,712][105620] Updated weights for policy 1, policy_version 702842 (0.0008) [2023-12-26 20:28:39,780][105620] Updated weights for policy 1, policy_version 702852 (0.0006) [2023-12-26 20:28:39,849][105620] Updated weights for policy 1, policy_version 702862 (0.0009) [2023-12-26 20:28:39,903][105620] Updated weights for policy 1, policy_version 702872 (0.0008) [2023-12-26 20:28:40,050][105692] Updated weights for policy 0, policy_version 702062 (0.0010) [2023-12-26 20:28:40,109][105692] Updated weights for policy 0, policy_version 702072 (0.0011) [2023-12-26 20:28:40,170][105692] Updated weights for policy 0, policy_version 702082 (0.0011) [2023-12-26 20:28:40,595][105620] Updated weights for policy 1, policy_version 702882 (0.0008) [2023-12-26 20:28:40,648][105620] Updated weights for policy 1, policy_version 702892 (0.0005) [2023-12-26 20:28:40,704][105620] Updated weights for policy 1, policy_version 702902 (0.0006) [2023-12-26 20:28:40,889][105692] Updated weights for policy 0, policy_version 702092 (0.0009) [2023-12-26 20:28:40,950][105692] Updated weights for policy 0, policy_version 702102 (0.0005) [2023-12-26 20:28:41,007][105692] Updated weights for policy 0, policy_version 702112 (0.0005) [2023-12-26 20:28:41,062][104569] Fps is (10 sec: 21299.5, 60 sec: 20070.4, 300 sec: 19688.6). Total num frames: 359735296. Throughput: 0: 10054.8, 1: 9986.2. Samples: 359739800. Policy #0 lag: (min: 14.0, avg: 31.0, max: 32.0) [2023-12-26 20:28:41,062][104569] Avg episode reward: [(0, '9076.117'), (1, '8900.529')] [2023-12-26 20:28:41,396][105620] Updated weights for policy 1, policy_version 702912 (0.0008) [2023-12-26 20:28:41,461][105620] Updated weights for policy 1, policy_version 702922 (0.0009) [2023-12-26 20:28:41,509][105620] Updated weights for policy 1, policy_version 702932 (0.0008) [2023-12-26 20:28:41,712][105692] Updated weights for policy 0, policy_version 702122 (0.0008) [2023-12-26 20:28:41,781][105692] Updated weights for policy 0, policy_version 702132 (0.0007) [2023-12-26 20:28:41,846][105692] Updated weights for policy 0, policy_version 702142 (0.0006) [2023-12-26 20:28:41,918][105692] Updated weights for policy 0, policy_version 702152 (0.0005) [2023-12-26 20:28:42,232][105620] Updated weights for policy 1, policy_version 702942 (0.0009) [2023-12-26 20:28:42,300][105620] Updated weights for policy 1, policy_version 702952 (0.0009) [2023-12-26 20:28:42,375][105620] Updated weights for policy 1, policy_version 702962 (0.0009) [2023-12-26 20:28:42,515][105692] Updated weights for policy 0, policy_version 702162 (0.0009) [2023-12-26 20:28:42,575][105692] Updated weights for policy 0, policy_version 702172 (0.0009) [2023-12-26 20:28:42,628][105692] Updated weights for policy 0, policy_version 702182 (0.0009) [2023-12-26 20:28:43,095][105620] Updated weights for policy 1, policy_version 702972 (0.0008) [2023-12-26 20:28:43,145][105620] Updated weights for policy 1, policy_version 702982 (0.0009) [2023-12-26 20:28:43,199][105620] Updated weights for policy 1, policy_version 702992 (0.0009) [2023-12-26 20:28:43,389][105692] Updated weights for policy 0, policy_version 702192 (0.0007) [2023-12-26 20:28:43,437][105692] Updated weights for policy 0, policy_version 702202 (0.0009) [2023-12-26 20:28:43,484][105692] Updated weights for policy 0, policy_version 702212 (0.0009) [2023-12-26 20:28:43,903][105620] Updated weights for policy 1, policy_version 703002 (0.0008) [2023-12-26 20:28:43,974][105620] Updated weights for policy 1, policy_version 703012 (0.0005) [2023-12-26 20:28:44,020][105620] Updated weights for policy 1, policy_version 703022 (0.0005) [2023-12-26 20:28:44,066][105620] Updated weights for policy 1, policy_version 703032 (0.0005) [2023-12-26 20:28:44,247][105692] Updated weights for policy 0, policy_version 702222 (0.0008) [2023-12-26 20:28:44,302][105692] Updated weights for policy 0, policy_version 702232 (0.0008) [2023-12-26 20:28:44,353][105692] Updated weights for policy 0, policy_version 702242 (0.0008) [2023-12-26 20:28:44,701][105620] Updated weights for policy 1, policy_version 703042 (0.0009) [2023-12-26 20:28:44,757][105620] Updated weights for policy 1, policy_version 703052 (0.0009) [2023-12-26 20:28:44,824][105620] Updated weights for policy 1, policy_version 703062 (0.0010) [2023-12-26 20:28:45,086][105692] Updated weights for policy 0, policy_version 702252 (0.0008) [2023-12-26 20:28:45,152][105692] Updated weights for policy 0, policy_version 702262 (0.0009) [2023-12-26 20:28:45,212][105692] Updated weights for policy 0, policy_version 702272 (0.0009) [2023-12-26 20:28:45,585][105620] Updated weights for policy 1, policy_version 703072 (0.0008) [2023-12-26 20:28:45,650][105620] Updated weights for policy 1, policy_version 703082 (0.0009) [2023-12-26 20:28:45,708][105620] Updated weights for policy 1, policy_version 703092 (0.0009) [2023-12-26 20:28:45,923][105692] Updated weights for policy 0, policy_version 702282 (0.0008) [2023-12-26 20:28:45,984][105692] Updated weights for policy 0, policy_version 702292 (0.0005) [2023-12-26 20:28:46,048][105692] Updated weights for policy 0, policy_version 702302 (0.0005) [2023-12-26 20:28:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19934.0, 300 sec: 19633.0). Total num frames: 359825408. Throughput: 0: 10028.7, 1: 9994.9. Samples: 359799044. Policy #0 lag: (min: 14.0, avg: 31.0, max: 32.0) [2023-12-26 20:28:46,062][104569] Avg episode reward: [(0, '8986.600'), (1, '8815.807')] [2023-12-26 20:28:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000703096_180011008.pth... [2023-12-26 20:28:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000701912_179707904.pth [2023-12-26 20:28:46,106][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000702312_179822592.pth... [2023-12-26 20:28:46,108][105692] Updated weights for policy 0, policy_version 702312 (0.0008) [2023-12-26 20:28:46,110][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000701160_179527680.pth [2023-12-26 20:28:46,497][105620] Updated weights for policy 1, policy_version 703102 (0.0009) [2023-12-26 20:28:46,540][105620] Updated weights for policy 1, policy_version 703112 (0.0010) [2023-12-26 20:28:46,593][105620] Updated weights for policy 1, policy_version 703122 (0.0009) [2023-12-26 20:28:46,794][105692] Updated weights for policy 0, policy_version 702322 (0.0010) [2023-12-26 20:28:46,850][105692] Updated weights for policy 0, policy_version 702332 (0.0009) [2023-12-26 20:28:46,913][105692] Updated weights for policy 0, policy_version 702342 (0.0008) [2023-12-26 20:28:47,236][105620] Updated weights for policy 1, policy_version 703132 (0.0007) [2023-12-26 20:28:47,284][105620] Updated weights for policy 1, policy_version 703142 (0.0007) [2023-12-26 20:28:47,338][105620] Updated weights for policy 1, policy_version 703152 (0.0010) [2023-12-26 20:28:47,725][105692] Updated weights for policy 0, policy_version 702352 (0.0008) [2023-12-26 20:28:47,788][105692] Updated weights for policy 0, policy_version 702362 (0.0008) [2023-12-26 20:28:47,843][105692] Updated weights for policy 0, policy_version 702372 (0.0008) [2023-12-26 20:28:48,039][105620] Updated weights for policy 1, policy_version 703162 (0.0009) [2023-12-26 20:28:48,095][105620] Updated weights for policy 1, policy_version 703172 (0.0005) [2023-12-26 20:28:48,144][105620] Updated weights for policy 1, policy_version 703182 (0.0005) [2023-12-26 20:28:48,191][105620] Updated weights for policy 1, policy_version 703192 (0.0005) [2023-12-26 20:28:48,643][105692] Updated weights for policy 0, policy_version 702382 (0.0008) [2023-12-26 20:28:48,711][105692] Updated weights for policy 0, policy_version 702392 (0.0008) [2023-12-26 20:28:48,775][105692] Updated weights for policy 0, policy_version 702402 (0.0008) [2023-12-26 20:28:48,850][105620] Updated weights for policy 1, policy_version 703202 (0.0010) [2023-12-26 20:28:48,909][105620] Updated weights for policy 1, policy_version 703212 (0.0010) [2023-12-26 20:28:48,967][105620] Updated weights for policy 1, policy_version 703222 (0.0010) [2023-12-26 20:28:49,529][105692] Updated weights for policy 0, policy_version 702412 (0.0008) [2023-12-26 20:28:49,595][105692] Updated weights for policy 0, policy_version 702422 (0.0008) [2023-12-26 20:28:49,654][105692] Updated weights for policy 0, policy_version 702432 (0.0006) [2023-12-26 20:28:49,733][105620] Updated weights for policy 1, policy_version 703232 (0.0011) [2023-12-26 20:28:49,800][105620] Updated weights for policy 1, policy_version 703242 (0.0010) [2023-12-26 20:28:49,865][105620] Updated weights for policy 1, policy_version 703252 (0.0010) [2023-12-26 20:28:50,360][105692] Updated weights for policy 0, policy_version 702442 (0.0006) [2023-12-26 20:28:50,424][105692] Updated weights for policy 0, policy_version 702452 (0.0011) [2023-12-26 20:28:50,476][105692] Updated weights for policy 0, policy_version 702462 (0.0010) [2023-12-26 20:28:50,532][105692] Updated weights for policy 0, policy_version 702472 (0.0010) [2023-12-26 20:28:50,589][105620] Updated weights for policy 1, policy_version 703262 (0.0009) [2023-12-26 20:28:50,658][105620] Updated weights for policy 1, policy_version 703272 (0.0008) [2023-12-26 20:28:50,718][105620] Updated weights for policy 1, policy_version 703282 (0.0011) [2023-12-26 20:28:51,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19933.8, 300 sec: 19633.0). Total num frames: 359923712. Throughput: 0: 9947.4, 1: 9967.6. Samples: 359913248. Policy #0 lag: (min: 14.0, avg: 31.0, max: 32.0) [2023-12-26 20:28:51,063][104569] Avg episode reward: [(0, '8988.416'), (1, '8751.079')] [2023-12-26 20:28:51,300][105692] Updated weights for policy 0, policy_version 702482 (0.0007) [2023-12-26 20:28:51,353][105692] Updated weights for policy 0, policy_version 702492 (0.0007) [2023-12-26 20:28:51,429][105692] Updated weights for policy 0, policy_version 702502 (0.0008) [2023-12-26 20:28:51,523][105620] Updated weights for policy 1, policy_version 703292 (0.0010) [2023-12-26 20:28:51,581][105620] Updated weights for policy 1, policy_version 703302 (0.0008) [2023-12-26 20:28:51,637][105620] Updated weights for policy 1, policy_version 703312 (0.0009) [2023-12-26 20:28:52,130][105692] Updated weights for policy 0, policy_version 702512 (0.0009) [2023-12-26 20:28:52,193][105692] Updated weights for policy 0, policy_version 702522 (0.0008) [2023-12-26 20:28:52,239][105692] Updated weights for policy 0, policy_version 702532 (0.0008) [2023-12-26 20:28:52,366][105620] Updated weights for policy 1, policy_version 703322 (0.0008) [2023-12-26 20:28:52,432][105620] Updated weights for policy 1, policy_version 703332 (0.0010) [2023-12-26 20:28:52,495][105620] Updated weights for policy 1, policy_version 703342 (0.0010) [2023-12-26 20:28:52,560][105620] Updated weights for policy 1, policy_version 703352 (0.0010) [2023-12-26 20:28:53,107][105692] Updated weights for policy 0, policy_version 702542 (0.0008) [2023-12-26 20:28:53,140][105620] Updated weights for policy 1, policy_version 703362 (0.0010) [2023-12-26 20:28:53,151][105692] Updated weights for policy 0, policy_version 702552 (0.0006) [2023-12-26 20:28:53,191][105620] Updated weights for policy 1, policy_version 703372 (0.0010) [2023-12-26 20:28:53,197][105692] Updated weights for policy 0, policy_version 702562 (0.0009) [2023-12-26 20:28:53,239][105620] Updated weights for policy 1, policy_version 703382 (0.0010) [2023-12-26 20:28:53,855][105620] Updated weights for policy 1, policy_version 703392 (0.0008) [2023-12-26 20:28:53,903][105620] Updated weights for policy 1, policy_version 703402 (0.0010) [2023-12-26 20:28:53,968][105620] Updated weights for policy 1, policy_version 703412 (0.0010) [2023-12-26 20:28:54,056][105692] Updated weights for policy 0, policy_version 702572 (0.0007) [2023-12-26 20:28:54,123][105692] Updated weights for policy 0, policy_version 702582 (0.0008) [2023-12-26 20:28:54,171][105692] Updated weights for policy 0, policy_version 702592 (0.0007) [2023-12-26 20:28:54,613][105620] Updated weights for policy 1, policy_version 703422 (0.0007) [2023-12-26 20:28:54,666][105620] Updated weights for policy 1, policy_version 703432 (0.0006) [2023-12-26 20:28:54,718][105620] Updated weights for policy 1, policy_version 703442 (0.0010) [2023-12-26 20:28:54,999][105692] Updated weights for policy 0, policy_version 702602 (0.0007) [2023-12-26 20:28:55,056][105692] Updated weights for policy 0, policy_version 702612 (0.0008) [2023-12-26 20:28:55,116][105692] Updated weights for policy 0, policy_version 702622 (0.0008) [2023-12-26 20:28:55,173][105692] Updated weights for policy 0, policy_version 702632 (0.0008) [2023-12-26 20:28:55,378][105620] Updated weights for policy 1, policy_version 703452 (0.0008) [2023-12-26 20:28:55,437][105620] Updated weights for policy 1, policy_version 703462 (0.0009) [2023-12-26 20:28:55,486][105620] Updated weights for policy 1, policy_version 703472 (0.0010) [2023-12-26 20:28:55,769][105692] Updated weights for policy 0, policy_version 702642 (0.0005) [2023-12-26 20:28:55,834][105692] Updated weights for policy 0, policy_version 702652 (0.0007) [2023-12-26 20:28:55,894][105692] Updated weights for policy 0, policy_version 702662 (0.0008) [2023-12-26 20:28:56,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19933.8, 300 sec: 19660.8). Total num frames: 360022016. Throughput: 0: 9774.7, 1: 10038.0. Samples: 360029296. Policy #0 lag: (min: 2.0, avg: 24.6, max: 34.0) [2023-12-26 20:28:56,063][104569] Avg episode reward: [(0, '9083.980'), (1, '9018.574')] [2023-12-26 20:28:56,150][105620] Updated weights for policy 1, policy_version 703482 (0.0009) [2023-12-26 20:28:56,201][105620] Updated weights for policy 1, policy_version 703492 (0.0005) [2023-12-26 20:28:56,247][105620] Updated weights for policy 1, policy_version 703502 (0.0005) [2023-12-26 20:28:56,293][105620] Updated weights for policy 1, policy_version 703512 (0.0005) [2023-12-26 20:28:56,604][105692] Updated weights for policy 0, policy_version 702672 (0.0010) [2023-12-26 20:28:56,655][105692] Updated weights for policy 0, policy_version 702682 (0.0010) [2023-12-26 20:28:56,708][105692] Updated weights for policy 0, policy_version 702692 (0.0010) [2023-12-26 20:28:56,971][105620] Updated weights for policy 1, policy_version 703522 (0.0010) [2023-12-26 20:28:57,022][105620] Updated weights for policy 1, policy_version 703532 (0.0010) [2023-12-26 20:28:57,073][105620] Updated weights for policy 1, policy_version 703542 (0.0010) [2023-12-26 20:28:57,290][105692] Updated weights for policy 0, policy_version 702702 (0.0010) [2023-12-26 20:28:57,342][105692] Updated weights for policy 0, policy_version 702712 (0.0010) [2023-12-26 20:28:57,399][105692] Updated weights for policy 0, policy_version 702722 (0.0010) [2023-12-26 20:28:57,776][105620] Updated weights for policy 1, policy_version 703552 (0.0010) [2023-12-26 20:28:57,824][105620] Updated weights for policy 1, policy_version 703562 (0.0010) [2023-12-26 20:28:57,874][105620] Updated weights for policy 1, policy_version 703572 (0.0010) [2023-12-26 20:28:58,104][105692] Updated weights for policy 0, policy_version 702732 (0.0010) [2023-12-26 20:28:58,163][105692] Updated weights for policy 0, policy_version 702742 (0.0011) [2023-12-26 20:28:58,225][105692] Updated weights for policy 0, policy_version 702752 (0.0010) [2023-12-26 20:28:58,660][105620] Updated weights for policy 1, policy_version 703582 (0.0010) [2023-12-26 20:28:58,729][105620] Updated weights for policy 1, policy_version 703592 (0.0011) [2023-12-26 20:28:58,794][105620] Updated weights for policy 1, policy_version 703602 (0.0011) [2023-12-26 20:28:59,072][105692] Updated weights for policy 0, policy_version 702762 (0.0010) [2023-12-26 20:28:59,131][105692] Updated weights for policy 0, policy_version 702772 (0.0011) [2023-12-26 20:28:59,193][105692] Updated weights for policy 0, policy_version 702782 (0.0009) [2023-12-26 20:28:59,259][105692] Updated weights for policy 0, policy_version 702792 (0.0010) [2023-12-26 20:28:59,630][105620] Updated weights for policy 1, policy_version 703612 (0.0010) [2023-12-26 20:28:59,687][105620] Updated weights for policy 1, policy_version 703622 (0.0011) [2023-12-26 20:28:59,743][105620] Updated weights for policy 1, policy_version 703632 (0.0011) [2023-12-26 20:28:59,978][105692] Updated weights for policy 0, policy_version 702802 (0.0011) [2023-12-26 20:29:00,039][105692] Updated weights for policy 0, policy_version 702812 (0.0010) [2023-12-26 20:29:00,103][105692] Updated weights for policy 0, policy_version 702822 (0.0011) [2023-12-26 20:29:00,471][105620] Updated weights for policy 1, policy_version 703642 (0.0009) [2023-12-26 20:29:00,532][105620] Updated weights for policy 1, policy_version 703652 (0.0010) [2023-12-26 20:29:00,593][105620] Updated weights for policy 1, policy_version 703662 (0.0010) [2023-12-26 20:29:00,659][105620] Updated weights for policy 1, policy_version 703672 (0.0008) [2023-12-26 20:29:00,689][105692] Updated weights for policy 0, policy_version 702832 (0.0007) [2023-12-26 20:29:00,736][105692] Updated weights for policy 0, policy_version 702842 (0.0006) [2023-12-26 20:29:00,788][105692] Updated weights for policy 0, policy_version 702852 (0.0011) [2023-12-26 20:29:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19933.9, 300 sec: 19660.8). Total num frames: 360120320. Throughput: 0: 9836.6, 1: 10047.9. Samples: 360089504. Policy #0 lag: (min: 2.0, avg: 24.6, max: 34.0) [2023-12-26 20:29:01,062][104569] Avg episode reward: [(0, '9174.697'), (1, '9355.890')] [2023-12-26 20:29:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000702856_179961856.pth... [2023-12-26 20:29:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000703672_180158464.pth... [2023-12-26 20:29:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000702488_179855360.pth [2023-12-26 20:29:01,082][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000701704_179666944.pth [2023-12-26 20:29:01,432][105620] Updated weights for policy 1, policy_version 703682 (0.0007) [2023-12-26 20:29:01,499][105620] Updated weights for policy 1, policy_version 703692 (0.0006) [2023-12-26 20:29:01,568][105620] Updated weights for policy 1, policy_version 703702 (0.0006) [2023-12-26 20:29:01,590][105692] Updated weights for policy 0, policy_version 702862 (0.0008) [2023-12-26 20:29:01,648][105692] Updated weights for policy 0, policy_version 702872 (0.0008) [2023-12-26 20:29:01,707][105692] Updated weights for policy 0, policy_version 702882 (0.0009) [2023-12-26 20:29:02,198][105620] Updated weights for policy 1, policy_version 703712 (0.0007) [2023-12-26 20:29:02,249][105620] Updated weights for policy 1, policy_version 703722 (0.0007) [2023-12-26 20:29:02,306][105620] Updated weights for policy 1, policy_version 703732 (0.0006) [2023-12-26 20:29:02,383][105692] Updated weights for policy 0, policy_version 702892 (0.0009) [2023-12-26 20:29:02,440][105692] Updated weights for policy 0, policy_version 702902 (0.0010) [2023-12-26 20:29:02,506][105692] Updated weights for policy 0, policy_version 702912 (0.0010) [2023-12-26 20:29:02,951][105620] Updated weights for policy 1, policy_version 703742 (0.0009) [2023-12-26 20:29:03,005][105620] Updated weights for policy 1, policy_version 703752 (0.0008) [2023-12-26 20:29:03,061][105620] Updated weights for policy 1, policy_version 703762 (0.0005) [2023-12-26 20:29:03,173][105692] Updated weights for policy 0, policy_version 702922 (0.0010) [2023-12-26 20:29:03,226][105692] Updated weights for policy 0, policy_version 702932 (0.0006) [2023-12-26 20:29:03,290][105692] Updated weights for policy 0, policy_version 702942 (0.0008) [2023-12-26 20:29:03,345][105692] Updated weights for policy 0, policy_version 702952 (0.0010) [2023-12-26 20:29:03,827][105620] Updated weights for policy 1, policy_version 703772 (0.0007) [2023-12-26 20:29:03,884][105620] Updated weights for policy 1, policy_version 703782 (0.0008) [2023-12-26 20:29:03,939][105620] Updated weights for policy 1, policy_version 703792 (0.0007) [2023-12-26 20:29:03,983][105692] Updated weights for policy 0, policy_version 702962 (0.0008) [2023-12-26 20:29:04,041][105692] Updated weights for policy 0, policy_version 702972 (0.0007) [2023-12-26 20:29:04,114][105692] Updated weights for policy 0, policy_version 702982 (0.0010) [2023-12-26 20:29:04,642][105620] Updated weights for policy 1, policy_version 703802 (0.0008) [2023-12-26 20:29:04,702][105620] Updated weights for policy 1, policy_version 703812 (0.0009) [2023-12-26 20:29:04,749][105620] Updated weights for policy 1, policy_version 703822 (0.0008) [2023-12-26 20:29:04,799][105620] Updated weights for policy 1, policy_version 703832 (0.0008) [2023-12-26 20:29:04,875][105692] Updated weights for policy 0, policy_version 702992 (0.0009) [2023-12-26 20:29:04,933][105692] Updated weights for policy 0, policy_version 703002 (0.0009) [2023-12-26 20:29:04,988][105692] Updated weights for policy 0, policy_version 703012 (0.0009) [2023-12-26 20:29:05,509][105620] Updated weights for policy 1, policy_version 703842 (0.0006) [2023-12-26 20:29:05,561][105620] Updated weights for policy 1, policy_version 703852 (0.0009) [2023-12-26 20:29:05,616][105620] Updated weights for policy 1, policy_version 703862 (0.0009) [2023-12-26 20:29:05,745][105692] Updated weights for policy 0, policy_version 703023 (0.0010) [2023-12-26 20:29:05,796][105692] Updated weights for policy 0, policy_version 703033 (0.0010) [2023-12-26 20:29:05,843][105692] Updated weights for policy 0, policy_version 703043 (0.0010) [2023-12-26 20:29:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.2, 300 sec: 19660.8). Total num frames: 360218624. Throughput: 0: 9712.4, 1: 10029.7. Samples: 360206068. Policy #0 lag: (min: 2.0, avg: 24.6, max: 34.0) [2023-12-26 20:29:06,063][104569] Avg episode reward: [(0, '9266.262'), (1, '9356.594')] [2023-12-26 20:29:06,390][105620] Updated weights for policy 1, policy_version 703872 (0.0008) [2023-12-26 20:29:06,455][105620] Updated weights for policy 1, policy_version 703882 (0.0008) [2023-12-26 20:29:06,490][105692] Updated weights for policy 0, policy_version 703053 (0.0010) [2023-12-26 20:29:06,516][105620] Updated weights for policy 1, policy_version 703892 (0.0007) [2023-12-26 20:29:06,539][105692] Updated weights for policy 0, policy_version 703063 (0.0011) [2023-12-26 20:29:06,592][105692] Updated weights for policy 0, policy_version 703073 (0.0010) [2023-12-26 20:29:07,271][105620] Updated weights for policy 1, policy_version 703902 (0.0006) [2023-12-26 20:29:07,323][105620] Updated weights for policy 1, policy_version 703912 (0.0007) [2023-12-26 20:29:07,327][105692] Updated weights for policy 0, policy_version 703083 (0.0011) [2023-12-26 20:29:07,389][105692] Updated weights for policy 0, policy_version 703093 (0.0011) [2023-12-26 20:29:07,392][105620] Updated weights for policy 1, policy_version 703922 (0.0006) [2023-12-26 20:29:07,445][105692] Updated weights for policy 0, policy_version 703103 (0.0010) [2023-12-26 20:29:07,925][105620] Updated weights for policy 1, policy_version 703932 (0.0006) [2023-12-26 20:29:07,975][105620] Updated weights for policy 1, policy_version 703942 (0.0009) [2023-12-26 20:29:08,034][105620] Updated weights for policy 1, policy_version 703952 (0.0007) [2023-12-26 20:29:08,171][105692] Updated weights for policy 0, policy_version 703113 (0.0009) [2023-12-26 20:29:08,225][105692] Updated weights for policy 0, policy_version 703123 (0.0010) [2023-12-26 20:29:08,286][105692] Updated weights for policy 0, policy_version 703133 (0.0005) [2023-12-26 20:29:08,355][105692] Updated weights for policy 0, policy_version 703143 (0.0012) [2023-12-26 20:29:08,724][105620] Updated weights for policy 1, policy_version 703962 (0.0007) [2023-12-26 20:29:08,781][105620] Updated weights for policy 1, policy_version 703972 (0.0008) [2023-12-26 20:29:08,831][105620] Updated weights for policy 1, policy_version 703982 (0.0009) [2023-12-26 20:29:08,892][105620] Updated weights for policy 1, policy_version 703992 (0.0009) [2023-12-26 20:29:09,083][105692] Updated weights for policy 0, policy_version 703153 (0.0010) [2023-12-26 20:29:09,149][105692] Updated weights for policy 0, policy_version 703163 (0.0011) [2023-12-26 20:29:09,212][105692] Updated weights for policy 0, policy_version 703173 (0.0010) [2023-12-26 20:29:09,655][105620] Updated weights for policy 1, policy_version 704002 (0.0011) [2023-12-26 20:29:09,713][105620] Updated weights for policy 1, policy_version 704012 (0.0011) [2023-12-26 20:29:09,774][105620] Updated weights for policy 1, policy_version 704022 (0.0011) [2023-12-26 20:29:09,958][105692] Updated weights for policy 0, policy_version 703183 (0.0010) [2023-12-26 20:29:10,028][105692] Updated weights for policy 0, policy_version 703193 (0.0011) [2023-12-26 20:29:10,092][105692] Updated weights for policy 0, policy_version 703203 (0.0011) [2023-12-26 20:29:10,561][105620] Updated weights for policy 1, policy_version 704032 (0.0010) [2023-12-26 20:29:10,603][105586] KL-divergence is very high: 114.8526 [2023-12-26 20:29:10,631][105620] Updated weights for policy 1, policy_version 704042 (0.0011) [2023-12-26 20:29:10,654][105586] KL-divergence is very high: 149.7964 [2023-12-26 20:29:10,695][105620] Updated weights for policy 1, policy_version 704052 (0.0008) [2023-12-26 20:29:10,753][105692] Updated weights for policy 0, policy_version 703213 (0.0010) [2023-12-26 20:29:10,815][105692] Updated weights for policy 0, policy_version 703223 (0.0010) [2023-12-26 20:29:10,882][105692] Updated weights for policy 0, policy_version 703233 (0.0011) [2023-12-26 20:29:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 360316928. Throughput: 0: 9729.0, 1: 10009.8. Samples: 360322432. Policy #0 lag: (min: 2.0, avg: 24.6, max: 34.0) [2023-12-26 20:29:11,063][104569] Avg episode reward: [(0, '8992.964'), (1, '9094.596')] [2023-12-26 20:29:11,520][105620] Updated weights for policy 1, policy_version 704062 (0.0009) [2023-12-26 20:29:11,569][105620] Updated weights for policy 1, policy_version 704072 (0.0010) [2023-12-26 20:29:11,625][105620] Updated weights for policy 1, policy_version 704082 (0.0011) [2023-12-26 20:29:11,670][105692] Updated weights for policy 0, policy_version 703243 (0.0011) [2023-12-26 20:29:11,741][105692] Updated weights for policy 0, policy_version 703253 (0.0012) [2023-12-26 20:29:11,798][105692] Updated weights for policy 0, policy_version 703263 (0.0010) [2023-12-26 20:29:12,355][105620] Updated weights for policy 1, policy_version 704092 (0.0010) [2023-12-26 20:29:12,412][105620] Updated weights for policy 1, policy_version 704102 (0.0008) [2023-12-26 20:29:12,466][105620] Updated weights for policy 1, policy_version 704112 (0.0006) [2023-12-26 20:29:12,561][105692] Updated weights for policy 0, policy_version 703273 (0.0011) [2023-12-26 20:29:12,626][105692] Updated weights for policy 0, policy_version 703283 (0.0011) [2023-12-26 20:29:12,689][105692] Updated weights for policy 0, policy_version 703293 (0.0011) [2023-12-26 20:29:12,753][105692] Updated weights for policy 0, policy_version 703303 (0.0011) [2023-12-26 20:29:13,044][105620] Updated weights for policy 1, policy_version 704122 (0.0006) [2023-12-26 20:29:13,100][105620] Updated weights for policy 1, policy_version 704132 (0.0006) [2023-12-26 20:29:13,157][105620] Updated weights for policy 1, policy_version 704142 (0.0006) [2023-12-26 20:29:13,216][105620] Updated weights for policy 1, policy_version 704152 (0.0005) [2023-12-26 20:29:13,470][105692] Updated weights for policy 0, policy_version 703313 (0.0006) [2023-12-26 20:29:13,516][105692] Updated weights for policy 0, policy_version 703323 (0.0005) [2023-12-26 20:29:13,567][105692] Updated weights for policy 0, policy_version 703333 (0.0008) [2023-12-26 20:29:13,856][105620] Updated weights for policy 1, policy_version 704162 (0.0010) [2023-12-26 20:29:13,912][105620] Updated weights for policy 1, policy_version 704172 (0.0010) [2023-12-26 20:29:13,973][105620] Updated weights for policy 1, policy_version 704182 (0.0010) [2023-12-26 20:29:14,283][105692] Updated weights for policy 0, policy_version 703343 (0.0006) [2023-12-26 20:29:14,329][105692] Updated weights for policy 0, policy_version 703353 (0.0005) [2023-12-26 20:29:14,378][105692] Updated weights for policy 0, policy_version 703363 (0.0010) [2023-12-26 20:29:14,678][105620] Updated weights for policy 1, policy_version 704192 (0.0009) [2023-12-26 20:29:14,727][105620] Updated weights for policy 1, policy_version 704202 (0.0009) [2023-12-26 20:29:14,782][105620] Updated weights for policy 1, policy_version 704212 (0.0008) [2023-12-26 20:29:15,118][105692] Updated weights for policy 0, policy_version 703373 (0.0009) [2023-12-26 20:29:15,178][105692] Updated weights for policy 0, policy_version 703383 (0.0010) [2023-12-26 20:29:15,234][105692] Updated weights for policy 0, policy_version 703393 (0.0005) [2023-12-26 20:29:15,565][105620] Updated weights for policy 1, policy_version 704222 (0.0009) [2023-12-26 20:29:15,627][105620] Updated weights for policy 1, policy_version 704232 (0.0010) [2023-12-26 20:29:15,681][105620] Updated weights for policy 1, policy_version 704243 (0.0010) [2023-12-26 20:29:15,791][105692] Updated weights for policy 0, policy_version 703403 (0.0005) [2023-12-26 20:29:15,839][105692] Updated weights for policy 0, policy_version 703413 (0.0010) [2023-12-26 20:29:15,895][105692] Updated weights for policy 0, policy_version 703423 (0.0010) [2023-12-26 20:29:16,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 360415232. Throughput: 0: 9663.3, 1: 9994.0. Samples: 360380704. Policy #0 lag: (min: 2.0, avg: 24.6, max: 34.0) [2023-12-26 20:29:16,062][104569] Avg episode reward: [(0, '8902.099'), (1, '9004.084')] [2023-12-26 20:29:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000704248_180305920.pth... [2023-12-26 20:29:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000703432_180109312.pth... [2023-12-26 20:29:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000703096_180011008.pth [2023-12-26 20:29:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000702312_179822592.pth [2023-12-26 20:29:16,535][105692] Updated weights for policy 0, policy_version 703433 (0.0009) [2023-12-26 20:29:16,537][105620] Updated weights for policy 1, policy_version 704253 (0.0010) [2023-12-26 20:29:16,593][105620] Updated weights for policy 1, policy_version 704263 (0.0008) [2023-12-26 20:29:16,594][105692] Updated weights for policy 0, policy_version 703443 (0.0005) [2023-12-26 20:29:16,649][105620] Updated weights for policy 1, policy_version 704273 (0.0009) [2023-12-26 20:29:16,659][105692] Updated weights for policy 0, policy_version 703453 (0.0005) [2023-12-26 20:29:16,713][105692] Updated weights for policy 0, policy_version 703463 (0.0006) [2023-12-26 20:29:17,382][105620] Updated weights for policy 1, policy_version 704283 (0.0009) [2023-12-26 20:29:17,402][105692] Updated weights for policy 0, policy_version 703473 (0.0008) [2023-12-26 20:29:17,437][105620] Updated weights for policy 1, policy_version 704293 (0.0007) [2023-12-26 20:29:17,454][105692] Updated weights for policy 0, policy_version 703483 (0.0007) [2023-12-26 20:29:17,489][105620] Updated weights for policy 1, policy_version 704303 (0.0006) [2023-12-26 20:29:17,511][105692] Updated weights for policy 0, policy_version 703493 (0.0008) [2023-12-26 20:29:18,179][105620] Updated weights for policy 1, policy_version 704313 (0.0006) [2023-12-26 20:29:18,242][105620] Updated weights for policy 1, policy_version 704323 (0.0009) [2023-12-26 20:29:18,300][105692] Updated weights for policy 0, policy_version 703503 (0.0009) [2023-12-26 20:29:18,301][105620] Updated weights for policy 1, policy_version 704333 (0.0009) [2023-12-26 20:29:18,364][105692] Updated weights for policy 0, policy_version 703513 (0.0007) [2023-12-26 20:29:18,369][105620] Updated weights for policy 1, policy_version 704343 (0.0008) [2023-12-26 20:29:18,418][105692] Updated weights for policy 0, policy_version 703523 (0.0009) [2023-12-26 20:29:19,121][105620] Updated weights for policy 1, policy_version 704353 (0.0005) [2023-12-26 20:29:19,128][105692] Updated weights for policy 0, policy_version 703533 (0.0010) [2023-12-26 20:29:19,173][105620] Updated weights for policy 1, policy_version 704363 (0.0006) [2023-12-26 20:29:19,184][105692] Updated weights for policy 0, policy_version 703543 (0.0010) [2023-12-26 20:29:19,228][105620] Updated weights for policy 1, policy_version 704373 (0.0007) [2023-12-26 20:29:19,245][105692] Updated weights for policy 0, policy_version 703553 (0.0009) [2023-12-26 20:29:19,854][105620] Updated weights for policy 1, policy_version 704383 (0.0008) [2023-12-26 20:29:19,907][105620] Updated weights for policy 1, policy_version 704393 (0.0008) [2023-12-26 20:29:19,976][105620] Updated weights for policy 1, policy_version 704403 (0.0008) [2023-12-26 20:29:20,036][105692] Updated weights for policy 0, policy_version 703563 (0.0011) [2023-12-26 20:29:20,092][105692] Updated weights for policy 0, policy_version 703573 (0.0006) [2023-12-26 20:29:20,160][105692] Updated weights for policy 0, policy_version 703583 (0.0006) [2023-12-26 20:29:20,704][105620] Updated weights for policy 1, policy_version 704413 (0.0008) [2023-12-26 20:29:20,770][105620] Updated weights for policy 1, policy_version 704423 (0.0008) [2023-12-26 20:29:20,785][105586] KL-divergence is very high: 147.6862 [2023-12-26 20:29:20,800][105692] Updated weights for policy 0, policy_version 703593 (0.0006) [2023-12-26 20:29:20,842][105620] Updated weights for policy 1, policy_version 704433 (0.0006) [2023-12-26 20:29:20,842][105586] KL-divergence is very high: 252.6755 [2023-12-26 20:29:20,866][105692] Updated weights for policy 0, policy_version 703603 (0.0009) [2023-12-26 20:29:20,938][105692] Updated weights for policy 0, policy_version 703613 (0.0009) [2023-12-26 20:29:21,004][105692] Updated weights for policy 0, policy_version 703623 (0.0008) [2023-12-26 20:29:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19716.3). Total num frames: 360513536. Throughput: 0: 9725.5, 1: 9857.1. Samples: 360497636. Policy #0 lag: (min: 2.0, avg: 24.6, max: 34.0) [2023-12-26 20:29:21,063][104569] Avg episode reward: [(0, '8996.929'), (1, '9083.439')] [2023-12-26 20:29:21,577][105620] Updated weights for policy 1, policy_version 704443 (0.0006) [2023-12-26 20:29:21,640][105620] Updated weights for policy 1, policy_version 704453 (0.0008) [2023-12-26 20:29:21,694][105620] Updated weights for policy 1, policy_version 704463 (0.0009) [2023-12-26 20:29:21,766][105692] Updated weights for policy 0, policy_version 703633 (0.0008) [2023-12-26 20:29:21,830][105692] Updated weights for policy 0, policy_version 703643 (0.0009) [2023-12-26 20:29:21,895][105692] Updated weights for policy 0, policy_version 703653 (0.0009) [2023-12-26 20:29:22,472][105620] Updated weights for policy 1, policy_version 704473 (0.0007) [2023-12-26 20:29:22,531][105620] Updated weights for policy 1, policy_version 704483 (0.0009) [2023-12-26 20:29:22,584][105620] Updated weights for policy 1, policy_version 704493 (0.0008) [2023-12-26 20:29:22,603][105692] Updated weights for policy 0, policy_version 703663 (0.0009) [2023-12-26 20:29:22,642][105620] Updated weights for policy 1, policy_version 704503 (0.0008) [2023-12-26 20:29:22,645][105692] Updated weights for policy 0, policy_version 703673 (0.0007) [2023-12-26 20:29:22,695][105692] Updated weights for policy 0, policy_version 703683 (0.0005) [2023-12-26 20:29:23,329][105692] Updated weights for policy 0, policy_version 703693 (0.0009) [2023-12-26 20:29:23,381][105692] Updated weights for policy 0, policy_version 703703 (0.0009) [2023-12-26 20:29:23,438][105692] Updated weights for policy 0, policy_version 703713 (0.0006) [2023-12-26 20:29:23,444][105620] Updated weights for policy 1, policy_version 704513 (0.0008) [2023-12-26 20:29:23,498][105620] Updated weights for policy 1, policy_version 704523 (0.0008) [2023-12-26 20:29:23,548][105620] Updated weights for policy 1, policy_version 704533 (0.0008) [2023-12-26 20:29:24,134][105692] Updated weights for policy 0, policy_version 703723 (0.0007) [2023-12-26 20:29:24,197][105692] Updated weights for policy 0, policy_version 703733 (0.0007) [2023-12-26 20:29:24,250][105692] Updated weights for policy 0, policy_version 703743 (0.0006) [2023-12-26 20:29:24,344][105620] Updated weights for policy 1, policy_version 704543 (0.0008) [2023-12-26 20:29:24,403][105620] Updated weights for policy 1, policy_version 704553 (0.0008) [2023-12-26 20:29:24,463][105620] Updated weights for policy 1, policy_version 704563 (0.0008) [2023-12-26 20:29:24,850][105692] Updated weights for policy 0, policy_version 703753 (0.0010) [2023-12-26 20:29:24,903][105692] Updated weights for policy 0, policy_version 703763 (0.0005) [2023-12-26 20:29:24,961][105692] Updated weights for policy 0, policy_version 703773 (0.0005) [2023-12-26 20:29:25,024][105692] Updated weights for policy 0, policy_version 703783 (0.0005) [2023-12-26 20:29:25,352][105620] Updated weights for policy 1, policy_version 704573 (0.0009) [2023-12-26 20:29:25,403][105620] Updated weights for policy 1, policy_version 704583 (0.0009) [2023-12-26 20:29:25,456][105620] Updated weights for policy 1, policy_version 704593 (0.0009) [2023-12-26 20:29:25,517][105692] Updated weights for policy 0, policy_version 703793 (0.0010) [2023-12-26 20:29:25,575][105692] Updated weights for policy 0, policy_version 703803 (0.0010) [2023-12-26 20:29:25,642][105692] Updated weights for policy 0, policy_version 703813 (0.0010) [2023-12-26 20:29:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 360603648. Throughput: 0: 9746.2, 1: 9675.4. Samples: 360613776. Policy #0 lag: (min: 2.0, avg: 24.6, max: 34.0) [2023-12-26 20:29:26,062][104569] Avg episode reward: [(0, '8902.666'), (1, '9083.789')] [2023-12-26 20:29:26,235][105620] Updated weights for policy 1, policy_version 704603 (0.0006) [2023-12-26 20:29:26,294][105620] Updated weights for policy 1, policy_version 704613 (0.0006) [2023-12-26 20:29:26,348][105620] Updated weights for policy 1, policy_version 704623 (0.0007) [2023-12-26 20:29:26,360][105692] Updated weights for policy 0, policy_version 703823 (0.0010) [2023-12-26 20:29:26,422][105692] Updated weights for policy 0, policy_version 703833 (0.0010) [2023-12-26 20:29:26,489][105692] Updated weights for policy 0, policy_version 703843 (0.0011) [2023-12-26 20:29:27,063][105620] Updated weights for policy 1, policy_version 704633 (0.0007) [2023-12-26 20:29:27,114][105620] Updated weights for policy 1, policy_version 704643 (0.0006) [2023-12-26 20:29:27,162][105692] Updated weights for policy 0, policy_version 703853 (0.0008) [2023-12-26 20:29:27,169][105620] Updated weights for policy 1, policy_version 704653 (0.0006) [2023-12-26 20:29:27,221][105692] Updated weights for policy 0, policy_version 703863 (0.0005) [2023-12-26 20:29:27,223][105620] Updated weights for policy 1, policy_version 704663 (0.0005) [2023-12-26 20:29:27,278][105692] Updated weights for policy 0, policy_version 703873 (0.0008) [2023-12-26 20:29:27,918][105620] Updated weights for policy 1, policy_version 704673 (0.0010) [2023-12-26 20:29:27,965][105692] Updated weights for policy 0, policy_version 703883 (0.0009) [2023-12-26 20:29:27,980][105620] Updated weights for policy 1, policy_version 704683 (0.0010) [2023-12-26 20:29:28,021][105692] Updated weights for policy 0, policy_version 703893 (0.0005) [2023-12-26 20:29:28,042][105620] Updated weights for policy 1, policy_version 704693 (0.0010) [2023-12-26 20:29:28,076][105692] Updated weights for policy 0, policy_version 703903 (0.0005) [2023-12-26 20:29:28,657][105692] Updated weights for policy 0, policy_version 703913 (0.0006) [2023-12-26 20:29:28,689][105620] Updated weights for policy 1, policy_version 704703 (0.0008) [2023-12-26 20:29:28,711][105692] Updated weights for policy 0, policy_version 703923 (0.0009) [2023-12-26 20:29:28,725][105585] KL-divergence is very high: 113.2596 [2023-12-26 20:29:28,746][105620] Updated weights for policy 1, policy_version 704713 (0.0008) [2023-12-26 20:29:28,774][105585] KL-divergence is very high: 106.9307 [2023-12-26 20:29:28,775][105692] Updated weights for policy 0, policy_version 703933 (0.0011) [2023-12-26 20:29:28,797][105620] Updated weights for policy 1, policy_version 704723 (0.0006) [2023-12-26 20:29:28,837][105692] Updated weights for policy 0, policy_version 703943 (0.0007) [2023-12-26 20:29:29,412][105692] Updated weights for policy 0, policy_version 703953 (0.0006) [2023-12-26 20:29:29,478][105692] Updated weights for policy 0, policy_version 703963 (0.0005) [2023-12-26 20:29:29,496][105620] Updated weights for policy 1, policy_version 704733 (0.0008) [2023-12-26 20:29:29,533][105692] Updated weights for policy 0, policy_version 703973 (0.0009) [2023-12-26 20:29:29,556][105620] Updated weights for policy 1, policy_version 704743 (0.0006) [2023-12-26 20:29:29,616][105620] Updated weights for policy 1, policy_version 704753 (0.0008) [2023-12-26 20:29:30,185][105692] Updated weights for policy 0, policy_version 703983 (0.0006) [2023-12-26 20:29:30,242][105692] Updated weights for policy 0, policy_version 703993 (0.0006) [2023-12-26 20:29:30,299][105692] Updated weights for policy 0, policy_version 704003 (0.0010) [2023-12-26 20:29:30,325][105620] Updated weights for policy 1, policy_version 704763 (0.0006) [2023-12-26 20:29:30,390][105620] Updated weights for policy 1, policy_version 704773 (0.0009) [2023-12-26 20:29:30,457][105620] Updated weights for policy 1, policy_version 704783 (0.0009) [2023-12-26 20:29:31,002][105692] Updated weights for policy 0, policy_version 704013 (0.0011) [2023-12-26 20:29:31,062][105692] Updated weights for policy 0, policy_version 704023 (0.0010) [2023-12-26 20:29:31,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 360701952. Throughput: 0: 9770.2, 1: 9685.8. Samples: 360674564. Policy #0 lag: (min: 2.0, avg: 24.6, max: 34.0) [2023-12-26 20:29:31,062][104569] Avg episode reward: [(0, '8996.265'), (1, '8992.711')] [2023-12-26 20:29:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000704792_180445184.pth... [2023-12-26 20:29:31,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000703672_180158464.pth [2023-12-26 20:29:31,105][105620] Updated weights for policy 1, policy_version 704793 (0.0008) [2023-12-26 20:29:31,122][105692] Updated weights for policy 0, policy_version 704033 (0.0008) [2023-12-26 20:29:31,164][105620] Updated weights for policy 1, policy_version 704803 (0.0007) [2023-12-26 20:29:31,171][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000704040_180264960.pth... [2023-12-26 20:29:31,176][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000702856_179961856.pth [2023-12-26 20:29:31,232][105620] Updated weights for policy 1, policy_version 704813 (0.0007) [2023-12-26 20:29:31,293][105620] Updated weights for policy 1, policy_version 704823 (0.0008) [2023-12-26 20:29:31,940][105620] Updated weights for policy 1, policy_version 704833 (0.0007) [2023-12-26 20:29:31,976][105692] Updated weights for policy 0, policy_version 704043 (0.0007) [2023-12-26 20:29:31,997][105620] Updated weights for policy 1, policy_version 704843 (0.0005) [2023-12-26 20:29:32,035][105692] Updated weights for policy 0, policy_version 704053 (0.0008) [2023-12-26 20:29:32,056][105620] Updated weights for policy 1, policy_version 704853 (0.0006) [2023-12-26 20:29:32,091][105692] Updated weights for policy 0, policy_version 704063 (0.0009) [2023-12-26 20:29:32,758][105620] Updated weights for policy 1, policy_version 704863 (0.0008) [2023-12-26 20:29:32,808][105692] Updated weights for policy 0, policy_version 704073 (0.0007) [2023-12-26 20:29:32,810][105620] Updated weights for policy 1, policy_version 704873 (0.0008) [2023-12-26 20:29:32,859][105692] Updated weights for policy 0, policy_version 704083 (0.0007) [2023-12-26 20:29:32,861][105620] Updated weights for policy 1, policy_version 704883 (0.0007) [2023-12-26 20:29:32,904][105692] Updated weights for policy 0, policy_version 704093 (0.0006) [2023-12-26 20:29:32,958][105692] Updated weights for policy 0, policy_version 704103 (0.0006) [2023-12-26 20:29:33,556][105692] Updated weights for policy 0, policy_version 704113 (0.0009) [2023-12-26 20:29:33,609][105692] Updated weights for policy 0, policy_version 704123 (0.0009) [2023-12-26 20:29:33,662][105692] Updated weights for policy 0, policy_version 704133 (0.0005) [2023-12-26 20:29:33,722][105620] Updated weights for policy 1, policy_version 704893 (0.0010) [2023-12-26 20:29:33,782][105620] Updated weights for policy 1, policy_version 704903 (0.0010) [2023-12-26 20:29:33,834][105620] Updated weights for policy 1, policy_version 704913 (0.0006) [2023-12-26 20:29:34,286][105692] Updated weights for policy 0, policy_version 704143 (0.0006) [2023-12-26 20:29:34,333][105692] Updated weights for policy 0, policy_version 704153 (0.0005) [2023-12-26 20:29:34,379][105692] Updated weights for policy 0, policy_version 704163 (0.0008) [2023-12-26 20:29:34,480][105620] Updated weights for policy 1, policy_version 704923 (0.0006) [2023-12-26 20:29:34,532][105620] Updated weights for policy 1, policy_version 704933 (0.0010) [2023-12-26 20:29:34,586][105620] Updated weights for policy 1, policy_version 704943 (0.0010) [2023-12-26 20:29:35,083][105692] Updated weights for policy 0, policy_version 704173 (0.0008) [2023-12-26 20:29:35,143][105692] Updated weights for policy 0, policy_version 704183 (0.0007) [2023-12-26 20:29:35,201][105692] Updated weights for policy 0, policy_version 704193 (0.0011) [2023-12-26 20:29:35,396][105620] Updated weights for policy 1, policy_version 704953 (0.0010) [2023-12-26 20:29:35,449][105620] Updated weights for policy 1, policy_version 704963 (0.0010) [2023-12-26 20:29:35,501][105620] Updated weights for policy 1, policy_version 704973 (0.0008) [2023-12-26 20:29:35,551][105620] Updated weights for policy 1, policy_version 704983 (0.0009) [2023-12-26 20:29:35,880][105692] Updated weights for policy 0, policy_version 704203 (0.0009) [2023-12-26 20:29:35,930][105692] Updated weights for policy 0, policy_version 704213 (0.0009) [2023-12-26 20:29:35,984][105692] Updated weights for policy 0, policy_version 704223 (0.0009) [2023-12-26 20:29:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19716.3). Total num frames: 360808448. Throughput: 0: 9897.1, 1: 9695.5. Samples: 360794912. Policy #0 lag: (min: 2.0, avg: 24.6, max: 34.0) [2023-12-26 20:29:36,062][104569] Avg episode reward: [(0, '8813.997'), (1, '8907.154')] [2023-12-26 20:29:36,314][105620] Updated weights for policy 1, policy_version 704993 (0.0009) [2023-12-26 20:29:36,366][105620] Updated weights for policy 1, policy_version 705003 (0.0009) [2023-12-26 20:29:36,421][105620] Updated weights for policy 1, policy_version 705013 (0.0009) [2023-12-26 20:29:36,766][105692] Updated weights for policy 0, policy_version 704233 (0.0008) [2023-12-26 20:29:36,824][105692] Updated weights for policy 0, policy_version 704243 (0.0006) [2023-12-26 20:29:36,882][105692] Updated weights for policy 0, policy_version 704253 (0.0005) [2023-12-26 20:29:36,930][105692] Updated weights for policy 0, policy_version 704263 (0.0005) [2023-12-26 20:29:37,262][105620] Updated weights for policy 1, policy_version 705023 (0.0010) [2023-12-26 20:29:37,327][105620] Updated weights for policy 1, policy_version 705033 (0.0010) [2023-12-26 20:29:37,399][105620] Updated weights for policy 1, policy_version 705043 (0.0010) [2023-12-26 20:29:37,520][105692] Updated weights for policy 0, policy_version 704273 (0.0009) [2023-12-26 20:29:37,592][105692] Updated weights for policy 0, policy_version 704283 (0.0010) [2023-12-26 20:29:37,663][105692] Updated weights for policy 0, policy_version 704293 (0.0010) [2023-12-26 20:29:37,981][105620] Updated weights for policy 1, policy_version 705053 (0.0010) [2023-12-26 20:29:38,037][105620] Updated weights for policy 1, policy_version 705063 (0.0010) [2023-12-26 20:29:38,089][105620] Updated weights for policy 1, policy_version 705073 (0.0010) [2023-12-26 20:29:38,441][105692] Updated weights for policy 0, policy_version 704303 (0.0009) [2023-12-26 20:29:38,501][105692] Updated weights for policy 0, policy_version 704313 (0.0008) [2023-12-26 20:29:38,569][105692] Updated weights for policy 0, policy_version 704323 (0.0009) [2023-12-26 20:29:38,855][105620] Updated weights for policy 1, policy_version 705083 (0.0010) [2023-12-26 20:29:38,915][105620] Updated weights for policy 1, policy_version 705093 (0.0009) [2023-12-26 20:29:38,982][105620] Updated weights for policy 1, policy_version 705103 (0.0007) [2023-12-26 20:29:39,179][105692] Updated weights for policy 0, policy_version 704333 (0.0009) [2023-12-26 20:29:39,235][105692] Updated weights for policy 0, policy_version 704343 (0.0008) [2023-12-26 20:29:39,293][105692] Updated weights for policy 0, policy_version 704353 (0.0006) [2023-12-26 20:29:39,759][105620] Updated weights for policy 1, policy_version 705113 (0.0011) [2023-12-26 20:29:39,824][105620] Updated weights for policy 1, policy_version 705123 (0.0011) [2023-12-26 20:29:39,899][105620] Updated weights for policy 1, policy_version 705133 (0.0011) [2023-12-26 20:29:39,961][105692] Updated weights for policy 0, policy_version 704363 (0.0008) [2023-12-26 20:29:39,967][105620] Updated weights for policy 1, policy_version 705143 (0.0011) [2023-12-26 20:29:40,021][105692] Updated weights for policy 0, policy_version 704373 (0.0008) [2023-12-26 20:29:40,077][105692] Updated weights for policy 0, policy_version 704383 (0.0008) [2023-12-26 20:29:40,701][105620] Updated weights for policy 1, policy_version 705153 (0.0010) [2023-12-26 20:29:40,759][105620] Updated weights for policy 1, policy_version 705163 (0.0010) [2023-12-26 20:29:40,824][105620] Updated weights for policy 1, policy_version 705173 (0.0010) [2023-12-26 20:29:40,841][105692] Updated weights for policy 0, policy_version 704393 (0.0008) [2023-12-26 20:29:40,903][105692] Updated weights for policy 0, policy_version 704403 (0.0008) [2023-12-26 20:29:40,966][105692] Updated weights for policy 0, policy_version 704413 (0.0007) [2023-12-26 20:29:41,022][105692] Updated weights for policy 0, policy_version 704423 (0.0008) [2023-12-26 20:29:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.2, 300 sec: 19744.1). Total num frames: 360906752. Throughput: 0: 10001.3, 1: 9580.8. Samples: 360910488. Policy #0 lag: (min: 2.0, avg: 24.6, max: 34.0) [2023-12-26 20:29:41,062][104569] Avg episode reward: [(0, '8985.965'), (1, '9002.858')] [2023-12-26 20:29:41,592][105620] Updated weights for policy 1, policy_version 705183 (0.0011) [2023-12-26 20:29:41,657][105620] Updated weights for policy 1, policy_version 705193 (0.0010) [2023-12-26 20:29:41,721][105620] Updated weights for policy 1, policy_version 705203 (0.0008) [2023-12-26 20:29:41,779][105692] Updated weights for policy 0, policy_version 704433 (0.0008) [2023-12-26 20:29:41,847][105692] Updated weights for policy 0, policy_version 704443 (0.0008) [2023-12-26 20:29:41,917][105692] Updated weights for policy 0, policy_version 704453 (0.0008) [2023-12-26 20:29:42,459][105620] Updated weights for policy 1, policy_version 705213 (0.0008) [2023-12-26 20:29:42,533][105620] Updated weights for policy 1, policy_version 705223 (0.0009) [2023-12-26 20:29:42,593][105620] Updated weights for policy 1, policy_version 705233 (0.0009) [2023-12-26 20:29:42,673][105692] Updated weights for policy 0, policy_version 704463 (0.0007) [2023-12-26 20:29:42,724][105692] Updated weights for policy 0, policy_version 704473 (0.0009) [2023-12-26 20:29:42,772][105692] Updated weights for policy 0, policy_version 704483 (0.0009) [2023-12-26 20:29:43,227][105620] Updated weights for policy 1, policy_version 705243 (0.0009) [2023-12-26 20:29:43,286][105620] Updated weights for policy 1, policy_version 705253 (0.0009) [2023-12-26 20:29:43,337][105620] Updated weights for policy 1, policy_version 705263 (0.0009) [2023-12-26 20:29:43,451][105692] Updated weights for policy 0, policy_version 704493 (0.0009) [2023-12-26 20:29:43,506][105692] Updated weights for policy 0, policy_version 704503 (0.0008) [2023-12-26 20:29:43,555][105692] Updated weights for policy 0, policy_version 704514 (0.0009) [2023-12-26 20:29:44,105][105620] Updated weights for policy 1, policy_version 705273 (0.0010) [2023-12-26 20:29:44,151][105620] Updated weights for policy 1, policy_version 705283 (0.0009) [2023-12-26 20:29:44,199][105620] Updated weights for policy 1, policy_version 705293 (0.0008) [2023-12-26 20:29:44,249][105620] Updated weights for policy 1, policy_version 705303 (0.0009) [2023-12-26 20:29:44,309][105692] Updated weights for policy 0, policy_version 704524 (0.0008) [2023-12-26 20:29:44,367][105692] Updated weights for policy 0, policy_version 704534 (0.0009) [2023-12-26 20:29:44,425][105692] Updated weights for policy 0, policy_version 704544 (0.0008) [2023-12-26 20:29:45,011][105620] Updated weights for policy 1, policy_version 705313 (0.0010) [2023-12-26 20:29:45,081][105620] Updated weights for policy 1, policy_version 705323 (0.0009) [2023-12-26 20:29:45,143][105620] Updated weights for policy 1, policy_version 705333 (0.0009) [2023-12-26 20:29:45,170][105692] Updated weights for policy 0, policy_version 704554 (0.0009) [2023-12-26 20:29:45,227][105692] Updated weights for policy 0, policy_version 704564 (0.0009) [2023-12-26 20:29:45,286][105692] Updated weights for policy 0, policy_version 704574 (0.0009) [2023-12-26 20:29:45,342][105692] Updated weights for policy 0, policy_version 704584 (0.0009) [2023-12-26 20:29:45,839][105620] Updated weights for policy 1, policy_version 705343 (0.0006) [2023-12-26 20:29:45,892][105620] Updated weights for policy 1, policy_version 705353 (0.0005) [2023-12-26 20:29:45,949][105620] Updated weights for policy 1, policy_version 705363 (0.0005) [2023-12-26 20:29:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19716.3). Total num frames: 360996864. Throughput: 0: 9939.9, 1: 9551.4. Samples: 360966612. Policy #0 lag: (min: 2.0, avg: 24.6, max: 34.0) [2023-12-26 20:29:46,063][104569] Avg episode reward: [(0, '8894.875'), (1, '9088.822')] [2023-12-26 20:29:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000704584_180404224.pth... [2023-12-26 20:29:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000705368_180592640.pth... [2023-12-26 20:29:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000703432_180109312.pth [2023-12-26 20:29:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000704248_180305920.pth [2023-12-26 20:29:46,178][105692] Updated weights for policy 0, policy_version 704594 (0.0009) [2023-12-26 20:29:46,233][105692] Updated weights for policy 0, policy_version 704604 (0.0006) [2023-12-26 20:29:46,293][105692] Updated weights for policy 0, policy_version 704614 (0.0006) [2023-12-26 20:29:46,704][105620] Updated weights for policy 1, policy_version 705373 (0.0007) [2023-12-26 20:29:46,752][105620] Updated weights for policy 1, policy_version 705383 (0.0008) [2023-12-26 20:29:46,800][105620] Updated weights for policy 1, policy_version 705394 (0.0008) [2023-12-26 20:29:46,851][105692] Updated weights for policy 0, policy_version 704624 (0.0005) [2023-12-26 20:29:46,906][105692] Updated weights for policy 0, policy_version 704634 (0.0006) [2023-12-26 20:29:46,967][105692] Updated weights for policy 0, policy_version 704644 (0.0005) [2023-12-26 20:29:47,487][105692] Updated weights for policy 0, policy_version 704654 (0.0005) [2023-12-26 20:29:47,550][105692] Updated weights for policy 0, policy_version 704664 (0.0005) [2023-12-26 20:29:47,618][105692] Updated weights for policy 0, policy_version 704674 (0.0009) [2023-12-26 20:29:47,700][105620] Updated weights for policy 1, policy_version 705404 (0.0009) [2023-12-26 20:29:47,762][105620] Updated weights for policy 1, policy_version 705414 (0.0009) [2023-12-26 20:29:47,814][105620] Updated weights for policy 1, policy_version 705424 (0.0008) [2023-12-26 20:29:48,249][105692] Updated weights for policy 0, policy_version 704684 (0.0008) [2023-12-26 20:29:48,312][105692] Updated weights for policy 0, policy_version 704694 (0.0006) [2023-12-26 20:29:48,377][105692] Updated weights for policy 0, policy_version 704704 (0.0008) [2023-12-26 20:29:48,572][105620] Updated weights for policy 1, policy_version 705434 (0.0007) [2023-12-26 20:29:48,630][105620] Updated weights for policy 1, policy_version 705444 (0.0005) [2023-12-26 20:29:48,682][105620] Updated weights for policy 1, policy_version 705454 (0.0005) [2023-12-26 20:29:48,740][105620] Updated weights for policy 1, policy_version 705464 (0.0005) [2023-12-26 20:29:49,054][105692] Updated weights for policy 0, policy_version 704714 (0.0008) [2023-12-26 20:29:49,104][105692] Updated weights for policy 0, policy_version 704724 (0.0005) [2023-12-26 20:29:49,165][105692] Updated weights for policy 0, policy_version 704734 (0.0009) [2023-12-26 20:29:49,234][105692] Updated weights for policy 0, policy_version 704744 (0.0010) [2023-12-26 20:29:49,263][105620] Updated weights for policy 1, policy_version 705474 (0.0008) [2023-12-26 20:29:49,311][105620] Updated weights for policy 1, policy_version 705484 (0.0008) [2023-12-26 20:29:49,374][105620] Updated weights for policy 1, policy_version 705494 (0.0009) [2023-12-26 20:29:49,965][105692] Updated weights for policy 0, policy_version 704754 (0.0007) [2023-12-26 20:29:50,024][105692] Updated weights for policy 0, policy_version 704764 (0.0006) [2023-12-26 20:29:50,084][105692] Updated weights for policy 0, policy_version 704774 (0.0007) [2023-12-26 20:29:50,120][105620] Updated weights for policy 1, policy_version 705504 (0.0008) [2023-12-26 20:29:50,185][105620] Updated weights for policy 1, policy_version 705514 (0.0010) [2023-12-26 20:29:50,245][105620] Updated weights for policy 1, policy_version 705524 (0.0009) [2023-12-26 20:29:50,631][105692] Updated weights for policy 0, policy_version 704784 (0.0006) [2023-12-26 20:29:50,696][105692] Updated weights for policy 0, policy_version 704794 (0.0005) [2023-12-26 20:29:50,753][105692] Updated weights for policy 0, policy_version 704804 (0.0006) [2023-12-26 20:29:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19716.3). Total num frames: 361095168. Throughput: 0: 9985.0, 1: 9545.8. Samples: 361084952. Policy #0 lag: (min: 2.0, avg: 24.6, max: 34.0) [2023-12-26 20:29:51,062][104569] Avg episode reward: [(0, '8986.898'), (1, '9174.776')] [2023-12-26 20:29:51,110][105620] Updated weights for policy 1, policy_version 705534 (0.0009) [2023-12-26 20:29:51,172][105620] Updated weights for policy 1, policy_version 705544 (0.0008) [2023-12-26 20:29:51,237][105620] Updated weights for policy 1, policy_version 705554 (0.0008) [2023-12-26 20:29:51,461][105692] Updated weights for policy 0, policy_version 704814 (0.0009) [2023-12-26 20:29:51,523][105692] Updated weights for policy 0, policy_version 704824 (0.0009) [2023-12-26 20:29:51,584][105692] Updated weights for policy 0, policy_version 704834 (0.0009) [2023-12-26 20:29:51,996][105620] Updated weights for policy 1, policy_version 705564 (0.0009) [2023-12-26 20:29:52,046][105620] Updated weights for policy 1, policy_version 705574 (0.0009) [2023-12-26 20:29:52,100][105620] Updated weights for policy 1, policy_version 705584 (0.0008) [2023-12-26 20:29:52,370][105692] Updated weights for policy 0, policy_version 704844 (0.0010) [2023-12-26 20:29:52,434][105692] Updated weights for policy 0, policy_version 704854 (0.0009) [2023-12-26 20:29:52,497][105692] Updated weights for policy 0, policy_version 704864 (0.0008) [2023-12-26 20:29:52,802][105620] Updated weights for policy 1, policy_version 705594 (0.0008) [2023-12-26 20:29:52,861][105620] Updated weights for policy 1, policy_version 705604 (0.0009) [2023-12-26 20:29:52,926][105620] Updated weights for policy 1, policy_version 705614 (0.0009) [2023-12-26 20:29:52,999][105620] Updated weights for policy 1, policy_version 705624 (0.0005) [2023-12-26 20:29:53,234][105692] Updated weights for policy 0, policy_version 704874 (0.0008) [2023-12-26 20:29:53,284][105692] Updated weights for policy 0, policy_version 704884 (0.0009) [2023-12-26 20:29:53,336][105692] Updated weights for policy 0, policy_version 704894 (0.0006) [2023-12-26 20:29:53,389][105692] Updated weights for policy 0, policy_version 704904 (0.0005) [2023-12-26 20:29:53,603][105620] Updated weights for policy 1, policy_version 705634 (0.0005) [2023-12-26 20:29:53,659][105620] Updated weights for policy 1, policy_version 705644 (0.0005) [2023-12-26 20:29:53,721][105620] Updated weights for policy 1, policy_version 705654 (0.0005) [2023-12-26 20:29:53,995][105692] Updated weights for policy 0, policy_version 704914 (0.0010) [2023-12-26 20:29:54,050][105692] Updated weights for policy 0, policy_version 704926 (0.0010) [2023-12-26 20:29:54,221][105620] Updated weights for policy 1, policy_version 705664 (0.0009) [2023-12-26 20:29:54,266][105620] Updated weights for policy 1, policy_version 705674 (0.0008) [2023-12-26 20:29:54,319][105620] Updated weights for policy 1, policy_version 705684 (0.0008) [2023-12-26 20:29:54,782][105692] Updated weights for policy 0, policy_version 704937 (0.0011) [2023-12-26 20:29:54,830][105692] Updated weights for policy 0, policy_version 704947 (0.0010) [2023-12-26 20:29:54,874][105692] Updated weights for policy 0, policy_version 704957 (0.0010) [2023-12-26 20:29:54,917][105692] Updated weights for policy 0, policy_version 704967 (0.0007) [2023-12-26 20:29:55,045][105620] Updated weights for policy 1, policy_version 705694 (0.0009) [2023-12-26 20:29:55,090][105620] Updated weights for policy 1, policy_version 705704 (0.0010) [2023-12-26 20:29:55,137][105620] Updated weights for policy 1, policy_version 705714 (0.0009) [2023-12-26 20:29:55,602][105692] Updated weights for policy 0, policy_version 704977 (0.0007) [2023-12-26 20:29:55,668][105692] Updated weights for policy 0, policy_version 704987 (0.0005) [2023-12-26 20:29:55,732][105692] Updated weights for policy 0, policy_version 704997 (0.0005) [2023-12-26 20:29:55,917][105620] Updated weights for policy 1, policy_version 705724 (0.0008) [2023-12-26 20:29:55,974][105620] Updated weights for policy 1, policy_version 705734 (0.0009) [2023-12-26 20:29:56,023][105620] Updated weights for policy 1, policy_version 705744 (0.0010) [2023-12-26 20:29:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19771.9). Total num frames: 361193472. Throughput: 0: 10058.8, 1: 9560.3. Samples: 361205292. Policy #0 lag: (min: 2.0, avg: 24.6, max: 34.0) [2023-12-26 20:29:56,063][104569] Avg episode reward: [(0, '8823.611'), (1, '9087.957')] [2023-12-26 20:29:56,267][105692] Updated weights for policy 0, policy_version 705007 (0.0007) [2023-12-26 20:29:56,315][105692] Updated weights for policy 0, policy_version 705017 (0.0007) [2023-12-26 20:29:56,373][105692] Updated weights for policy 0, policy_version 705027 (0.0008) [2023-12-26 20:29:56,696][105620] Updated weights for policy 1, policy_version 705754 (0.0009) [2023-12-26 20:29:56,754][105620] Updated weights for policy 1, policy_version 705764 (0.0005) [2023-12-26 20:29:56,809][105620] Updated weights for policy 1, policy_version 705774 (0.0010) [2023-12-26 20:29:56,868][105620] Updated weights for policy 1, policy_version 705784 (0.0008) [2023-12-26 20:29:57,069][105692] Updated weights for policy 0, policy_version 705037 (0.0008) [2023-12-26 20:29:57,116][105692] Updated weights for policy 0, policy_version 705047 (0.0008) [2023-12-26 20:29:57,163][105692] Updated weights for policy 0, policy_version 705057 (0.0007) [2023-12-26 20:29:57,572][105620] Updated weights for policy 1, policy_version 705794 (0.0006) [2023-12-26 20:29:57,625][105620] Updated weights for policy 1, policy_version 705804 (0.0010) [2023-12-26 20:29:57,681][105620] Updated weights for policy 1, policy_version 705815 (0.0010) [2023-12-26 20:29:57,742][105692] Updated weights for policy 0, policy_version 705067 (0.0008) [2023-12-26 20:29:57,799][105692] Updated weights for policy 0, policy_version 705077 (0.0009) [2023-12-26 20:29:57,851][105692] Updated weights for policy 0, policy_version 705087 (0.0009) [2023-12-26 20:29:58,273][105620] Updated weights for policy 1, policy_version 705825 (0.0008) [2023-12-26 20:29:58,334][105620] Updated weights for policy 1, policy_version 705835 (0.0009) [2023-12-26 20:29:58,391][105620] Updated weights for policy 1, policy_version 705845 (0.0009) [2023-12-26 20:29:58,705][105692] Updated weights for policy 0, policy_version 705098 (0.0010) [2023-12-26 20:29:58,776][105692] Updated weights for policy 0, policy_version 705108 (0.0009) [2023-12-26 20:29:58,848][105692] Updated weights for policy 0, policy_version 705118 (0.0009) [2023-12-26 20:29:58,917][105692] Updated weights for policy 0, policy_version 705128 (0.0010) [2023-12-26 20:29:59,192][105620] Updated weights for policy 1, policy_version 705855 (0.0007) [2023-12-26 20:29:59,261][105620] Updated weights for policy 1, policy_version 705865 (0.0008) [2023-12-26 20:29:59,320][105620] Updated weights for policy 1, policy_version 705875 (0.0008) [2023-12-26 20:29:59,752][105692] Updated weights for policy 0, policy_version 705138 (0.0007) [2023-12-26 20:29:59,798][105692] Updated weights for policy 0, policy_version 705148 (0.0009) [2023-12-26 20:29:59,855][105692] Updated weights for policy 0, policy_version 705158 (0.0008) [2023-12-26 20:30:00,040][105620] Updated weights for policy 1, policy_version 705885 (0.0009) [2023-12-26 20:30:00,081][105586] KL-divergence is very high: 150.1691 [2023-12-26 20:30:00,090][105620] Updated weights for policy 1, policy_version 705896 (0.0009) [2023-12-26 20:30:00,116][105586] KL-divergence is very high: 208.5430 [2023-12-26 20:30:00,135][105620] Updated weights for policy 1, policy_version 705906 (0.0007) [2023-12-26 20:30:00,154][105586] KL-divergence is very high: 149.6165 [2023-12-26 20:30:00,561][105692] Updated weights for policy 0, policy_version 705168 (0.0009) [2023-12-26 20:30:00,631][105692] Updated weights for policy 0, policy_version 705178 (0.0010) [2023-12-26 20:30:00,695][105692] Updated weights for policy 0, policy_version 705188 (0.0010) [2023-12-26 20:30:00,949][105620] Updated weights for policy 1, policy_version 705916 (0.0008) [2023-12-26 20:30:01,013][105620] Updated weights for policy 1, policy_version 705926 (0.0009) [2023-12-26 20:30:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19716.3). Total num frames: 361291776. Throughput: 0: 10133.8, 1: 9554.2. Samples: 361266668. Policy #0 lag: (min: 2.0, avg: 24.6, max: 34.0) [2023-12-26 20:30:01,063][104569] Avg episode reward: [(0, '8736.796'), (1, '8910.760')] [2023-12-26 20:30:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000705192_180559872.pth... [2023-12-26 20:30:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000704040_180264960.pth [2023-12-26 20:30:01,076][105620] Updated weights for policy 1, policy_version 705936 (0.0009) [2023-12-26 20:30:01,120][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000705944_180740096.pth... [2023-12-26 20:30:01,123][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000704792_180445184.pth [2023-12-26 20:30:01,383][105692] Updated weights for policy 0, policy_version 705198 (0.0010) [2023-12-26 20:30:01,429][105692] Updated weights for policy 0, policy_version 705208 (0.0008) [2023-12-26 20:30:01,492][105692] Updated weights for policy 0, policy_version 705218 (0.0010) [2023-12-26 20:30:01,783][105620] Updated weights for policy 1, policy_version 705946 (0.0010) [2023-12-26 20:30:01,840][105620] Updated weights for policy 1, policy_version 705956 (0.0008) [2023-12-26 20:30:01,901][105620] Updated weights for policy 1, policy_version 705966 (0.0007) [2023-12-26 20:30:01,955][105620] Updated weights for policy 1, policy_version 705976 (0.0009) [2023-12-26 20:30:02,159][105692] Updated weights for policy 0, policy_version 705228 (0.0010) [2023-12-26 20:30:02,218][105692] Updated weights for policy 0, policy_version 705238 (0.0009) [2023-12-26 20:30:02,282][105692] Updated weights for policy 0, policy_version 705248 (0.0008) [2023-12-26 20:30:02,763][105620] Updated weights for policy 1, policy_version 705986 (0.0009) [2023-12-26 20:30:02,816][105620] Updated weights for policy 1, policy_version 705997 (0.0010) [2023-12-26 20:30:02,865][105620] Updated weights for policy 1, policy_version 706008 (0.0009) [2023-12-26 20:30:02,949][105692] Updated weights for policy 0, policy_version 705258 (0.0009) [2023-12-26 20:30:03,000][105692] Updated weights for policy 0, policy_version 705268 (0.0008) [2023-12-26 20:30:03,052][105692] Updated weights for policy 0, policy_version 705278 (0.0008) [2023-12-26 20:30:03,108][105692] Updated weights for policy 0, policy_version 705288 (0.0009) [2023-12-26 20:30:03,625][105620] Updated weights for policy 1, policy_version 706018 (0.0009) [2023-12-26 20:30:03,678][105620] Updated weights for policy 1, policy_version 706028 (0.0010) [2023-12-26 20:30:03,727][105620] Updated weights for policy 1, policy_version 706038 (0.0008) [2023-12-26 20:30:03,742][105692] Updated weights for policy 0, policy_version 705298 (0.0007) [2023-12-26 20:30:03,790][105692] Updated weights for policy 0, policy_version 705308 (0.0010) [2023-12-26 20:30:03,838][105692] Updated weights for policy 0, policy_version 705318 (0.0009) [2023-12-26 20:30:04,529][105620] Updated weights for policy 1, policy_version 706048 (0.0010) [2023-12-26 20:30:04,562][105692] Updated weights for policy 0, policy_version 705328 (0.0006) [2023-12-26 20:30:04,583][105620] Updated weights for policy 1, policy_version 706058 (0.0010) [2023-12-26 20:30:04,613][105692] Updated weights for policy 0, policy_version 705338 (0.0006) [2023-12-26 20:30:04,639][105620] Updated weights for policy 1, policy_version 706068 (0.0010) [2023-12-26 20:30:04,664][105692] Updated weights for policy 0, policy_version 705348 (0.0005) [2023-12-26 20:30:05,269][105692] Updated weights for policy 0, policy_version 705358 (0.0008) [2023-12-26 20:30:05,321][105692] Updated weights for policy 0, policy_version 705368 (0.0010) [2023-12-26 20:30:05,354][105620] Updated weights for policy 1, policy_version 706078 (0.0007) [2023-12-26 20:30:05,373][105692] Updated weights for policy 0, policy_version 705378 (0.0010) [2023-12-26 20:30:05,407][105620] Updated weights for policy 1, policy_version 706088 (0.0010) [2023-12-26 20:30:05,472][105620] Updated weights for policy 1, policy_version 706098 (0.0010) [2023-12-26 20:30:06,038][105620] Updated weights for policy 1, policy_version 706108 (0.0008) [2023-12-26 20:30:06,039][105692] Updated weights for policy 0, policy_version 705388 (0.0008) [2023-12-26 20:30:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19716.3). Total num frames: 361390080. Throughput: 0: 10099.4, 1: 9540.2. Samples: 361381424. Policy #0 lag: (min: 2.0, avg: 24.6, max: 34.0) [2023-12-26 20:30:06,063][104569] Avg episode reward: [(0, '8990.668'), (1, '9086.861')] [2023-12-26 20:30:06,096][105620] Updated weights for policy 1, policy_version 706118 (0.0006) [2023-12-26 20:30:06,097][105692] Updated weights for policy 0, policy_version 705398 (0.0006) [2023-12-26 20:30:06,157][105620] Updated weights for policy 1, policy_version 706128 (0.0009) [2023-12-26 20:30:06,169][105692] Updated weights for policy 0, policy_version 705408 (0.0008) [2023-12-26 20:30:06,849][105620] Updated weights for policy 1, policy_version 706138 (0.0009) [2023-12-26 20:30:06,903][105692] Updated weights for policy 0, policy_version 705418 (0.0008) [2023-12-26 20:30:06,910][105620] Updated weights for policy 1, policy_version 706148 (0.0005) [2023-12-26 20:30:06,958][105692] Updated weights for policy 0, policy_version 705428 (0.0010) [2023-12-26 20:30:06,990][105620] Updated weights for policy 1, policy_version 706158 (0.0007) [2023-12-26 20:30:07,005][105692] Updated weights for policy 0, policy_version 705438 (0.0007) [2023-12-26 20:30:07,055][105620] Updated weights for policy 1, policy_version 706168 (0.0007) [2023-12-26 20:30:07,055][105692] Updated weights for policy 0, policy_version 705448 (0.0008) [2023-12-26 20:30:07,569][105620] Updated weights for policy 1, policy_version 706178 (0.0005) [2023-12-26 20:30:07,633][105620] Updated weights for policy 1, policy_version 706188 (0.0005) [2023-12-26 20:30:07,700][105620] Updated weights for policy 1, policy_version 706198 (0.0006) [2023-12-26 20:30:07,924][105692] Updated weights for policy 0, policy_version 705458 (0.0006) [2023-12-26 20:30:07,987][105692] Updated weights for policy 0, policy_version 705468 (0.0010) [2023-12-26 20:30:08,040][105692] Updated weights for policy 0, policy_version 705478 (0.0008) [2023-12-26 20:30:08,300][105620] Updated weights for policy 1, policy_version 706208 (0.0009) [2023-12-26 20:30:08,359][105620] Updated weights for policy 1, policy_version 706218 (0.0009) [2023-12-26 20:30:08,421][105620] Updated weights for policy 1, policy_version 706228 (0.0009) [2023-12-26 20:30:08,798][105692] Updated weights for policy 0, policy_version 705488 (0.0009) [2023-12-26 20:30:08,861][105692] Updated weights for policy 0, policy_version 705498 (0.0009) [2023-12-26 20:30:08,912][105692] Updated weights for policy 0, policy_version 705508 (0.0008) [2023-12-26 20:30:09,174][105620] Updated weights for policy 1, policy_version 706238 (0.0009) [2023-12-26 20:30:09,227][105620] Updated weights for policy 1, policy_version 706248 (0.0008) [2023-12-26 20:30:09,290][105620] Updated weights for policy 1, policy_version 706258 (0.0008) [2023-12-26 20:30:09,675][105692] Updated weights for policy 0, policy_version 705518 (0.0009) [2023-12-26 20:30:09,741][105692] Updated weights for policy 0, policy_version 705528 (0.0009) [2023-12-26 20:30:09,792][105692] Updated weights for policy 0, policy_version 705538 (0.0009) [2023-12-26 20:30:10,066][105620] Updated weights for policy 1, policy_version 706268 (0.0009) [2023-12-26 20:30:10,127][105620] Updated weights for policy 1, policy_version 706278 (0.0008) [2023-12-26 20:30:10,194][105620] Updated weights for policy 1, policy_version 706288 (0.0009) [2023-12-26 20:30:10,537][105692] Updated weights for policy 0, policy_version 705548 (0.0009) [2023-12-26 20:30:10,591][105692] Updated weights for policy 0, policy_version 705558 (0.0009) [2023-12-26 20:30:10,645][105692] Updated weights for policy 0, policy_version 705568 (0.0007) [2023-12-26 20:30:11,005][105620] Updated weights for policy 1, policy_version 706298 (0.0010) [2023-12-26 20:30:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19744.1). Total num frames: 361488384. Throughput: 0: 9976.5, 1: 9702.2. Samples: 361499320. Policy #0 lag: (min: 2.0, avg: 24.6, max: 34.0) [2023-12-26 20:30:11,063][104569] Avg episode reward: [(0, '8813.284'), (1, '8995.467')] [2023-12-26 20:30:11,086][105620] Updated weights for policy 1, policy_version 706308 (0.0009) [2023-12-26 20:30:11,153][105620] Updated weights for policy 1, policy_version 706318 (0.0008) [2023-12-26 20:30:11,201][105620] Updated weights for policy 1, policy_version 706328 (0.0008) [2023-12-26 20:30:11,252][105692] Updated weights for policy 0, policy_version 705578 (0.0006) [2023-12-26 20:30:11,318][105692] Updated weights for policy 0, policy_version 705588 (0.0007) [2023-12-26 20:30:11,387][105692] Updated weights for policy 0, policy_version 705598 (0.0008) [2023-12-26 20:30:11,448][105692] Updated weights for policy 0, policy_version 705608 (0.0008) [2023-12-26 20:30:12,011][105620] Updated weights for policy 1, policy_version 706338 (0.0006) [2023-12-26 20:30:12,069][105620] Updated weights for policy 1, policy_version 706348 (0.0008) [2023-12-26 20:30:12,132][105620] Updated weights for policy 1, policy_version 706358 (0.0009) [2023-12-26 20:30:12,227][105692] Updated weights for policy 0, policy_version 705618 (0.0009) [2023-12-26 20:30:12,284][105692] Updated weights for policy 0, policy_version 705628 (0.0009) [2023-12-26 20:30:12,341][105692] Updated weights for policy 0, policy_version 705638 (0.0009) [2023-12-26 20:30:12,859][105620] Updated weights for policy 1, policy_version 706368 (0.0009) [2023-12-26 20:30:12,918][105620] Updated weights for policy 1, policy_version 706378 (0.0009) [2023-12-26 20:30:12,980][105620] Updated weights for policy 1, policy_version 706388 (0.0009) [2023-12-26 20:30:13,109][105692] Updated weights for policy 0, policy_version 705648 (0.0010) [2023-12-26 20:30:13,162][105692] Updated weights for policy 0, policy_version 705658 (0.0011) [2023-12-26 20:30:13,211][105692] Updated weights for policy 0, policy_version 705668 (0.0011) [2023-12-26 20:30:13,807][105620] Updated weights for policy 1, policy_version 706398 (0.0010) [2023-12-26 20:30:13,851][105692] Updated weights for policy 0, policy_version 705678 (0.0009) [2023-12-26 20:30:13,862][105620] Updated weights for policy 1, policy_version 706408 (0.0007) [2023-12-26 20:30:13,916][105692] Updated weights for policy 0, policy_version 705688 (0.0008) [2023-12-26 20:30:13,923][105620] Updated weights for policy 1, policy_version 706418 (0.0009) [2023-12-26 20:30:13,982][105692] Updated weights for policy 0, policy_version 705698 (0.0008) [2023-12-26 20:30:14,635][105692] Updated weights for policy 0, policy_version 705708 (0.0007) [2023-12-26 20:30:14,689][105692] Updated weights for policy 0, policy_version 705718 (0.0009) [2023-12-26 20:30:14,732][105620] Updated weights for policy 1, policy_version 706428 (0.0007) [2023-12-26 20:30:14,747][105692] Updated weights for policy 0, policy_version 705728 (0.0007) [2023-12-26 20:30:14,792][105620] Updated weights for policy 1, policy_version 706438 (0.0008) [2023-12-26 20:30:14,842][105620] Updated weights for policy 1, policy_version 706448 (0.0008) [2023-12-26 20:30:15,518][105692] Updated weights for policy 0, policy_version 705738 (0.0009) [2023-12-26 20:30:15,572][105692] Updated weights for policy 0, policy_version 705748 (0.0009) [2023-12-26 20:30:15,597][105620] Updated weights for policy 1, policy_version 706458 (0.0009) [2023-12-26 20:30:15,619][105692] Updated weights for policy 0, policy_version 705758 (0.0007) [2023-12-26 20:30:15,646][105620] Updated weights for policy 1, policy_version 706468 (0.0006) [2023-12-26 20:30:15,682][105692] Updated weights for policy 0, policy_version 705768 (0.0006) [2023-12-26 20:30:15,708][105620] Updated weights for policy 1, policy_version 706478 (0.0006) [2023-12-26 20:30:15,765][105620] Updated weights for policy 1, policy_version 706488 (0.0007) [2023-12-26 20:30:16,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19771.9). Total num frames: 361586688. Throughput: 0: 9940.1, 1: 9626.0. Samples: 361555040. Policy #0 lag: (min: 2.0, avg: 24.6, max: 34.0) [2023-12-26 20:30:16,062][104569] Avg episode reward: [(0, '8898.788'), (1, '8998.263')] [2023-12-26 20:30:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000705768_180707328.pth... [2023-12-26 20:30:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000706488_180879360.pth... [2023-12-26 20:30:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000704584_180404224.pth [2023-12-26 20:30:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000705368_180592640.pth [2023-12-26 20:30:16,438][105692] Updated weights for policy 0, policy_version 705778 (0.0008) [2023-12-26 20:30:16,494][105692] Updated weights for policy 0, policy_version 705788 (0.0007) [2023-12-26 20:30:16,496][105620] Updated weights for policy 1, policy_version 706498 (0.0007) [2023-12-26 20:30:16,543][105692] Updated weights for policy 0, policy_version 705798 (0.0007) [2023-12-26 20:30:16,545][105620] Updated weights for policy 1, policy_version 706508 (0.0006) [2023-12-26 20:30:16,597][105620] Updated weights for policy 1, policy_version 706518 (0.0008) [2023-12-26 20:30:17,159][105692] Updated weights for policy 0, policy_version 705808 (0.0008) [2023-12-26 20:30:17,216][105692] Updated weights for policy 0, policy_version 705818 (0.0009) [2023-12-26 20:30:17,277][105692] Updated weights for policy 0, policy_version 705828 (0.0009) [2023-12-26 20:30:17,423][105620] Updated weights for policy 1, policy_version 706528 (0.0009) [2023-12-26 20:30:17,474][105620] Updated weights for policy 1, policy_version 706538 (0.0010) [2023-12-26 20:30:17,528][105620] Updated weights for policy 1, policy_version 706550 (0.0010) [2023-12-26 20:30:17,867][105692] Updated weights for policy 0, policy_version 705838 (0.0009) [2023-12-26 20:30:17,913][105692] Updated weights for policy 0, policy_version 705848 (0.0008) [2023-12-26 20:30:17,958][105692] Updated weights for policy 0, policy_version 705858 (0.0008) [2023-12-26 20:30:18,372][105620] Updated weights for policy 1, policy_version 706560 (0.0009) [2023-12-26 20:30:18,432][105620] Updated weights for policy 1, policy_version 706570 (0.0009) [2023-12-26 20:30:18,492][105620] Updated weights for policy 1, policy_version 706580 (0.0008) [2023-12-26 20:30:18,744][105692] Updated weights for policy 0, policy_version 705868 (0.0009) [2023-12-26 20:30:18,791][105692] Updated weights for policy 0, policy_version 705878 (0.0009) [2023-12-26 20:30:18,839][105692] Updated weights for policy 0, policy_version 705888 (0.0008) [2023-12-26 20:30:19,243][105620] Updated weights for policy 1, policy_version 706590 (0.0008) [2023-12-26 20:30:19,297][105620] Updated weights for policy 1, policy_version 706600 (0.0006) [2023-12-26 20:30:19,360][105620] Updated weights for policy 1, policy_version 706610 (0.0009) [2023-12-26 20:30:19,633][105692] Updated weights for policy 0, policy_version 705898 (0.0010) [2023-12-26 20:30:19,694][105692] Updated weights for policy 0, policy_version 705908 (0.0009) [2023-12-26 20:30:19,754][105692] Updated weights for policy 0, policy_version 705918 (0.0009) [2023-12-26 20:30:19,812][105692] Updated weights for policy 0, policy_version 705928 (0.0008) [2023-12-26 20:30:20,066][105620] Updated weights for policy 1, policy_version 706620 (0.0009) [2023-12-26 20:30:20,117][105620] Updated weights for policy 1, policy_version 706630 (0.0010) [2023-12-26 20:30:20,176][105620] Updated weights for policy 1, policy_version 706640 (0.0010) [2023-12-26 20:30:20,556][105692] Updated weights for policy 0, policy_version 705938 (0.0006) [2023-12-26 20:30:20,626][105692] Updated weights for policy 0, policy_version 705948 (0.0007) [2023-12-26 20:30:20,691][105692] Updated weights for policy 0, policy_version 705958 (0.0008) [2023-12-26 20:30:20,934][105620] Updated weights for policy 1, policy_version 706650 (0.0010) [2023-12-26 20:30:20,988][105620] Updated weights for policy 1, policy_version 706660 (0.0010) [2023-12-26 20:30:21,056][105620] Updated weights for policy 1, policy_version 706670 (0.0009) [2023-12-26 20:30:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19744.1). Total num frames: 361676800. Throughput: 0: 9910.9, 1: 9535.2. Samples: 361669984. Policy #0 lag: (min: 2.0, avg: 24.6, max: 34.0) [2023-12-26 20:30:21,063][104569] Avg episode reward: [(0, '8989.053'), (1, '8912.030')] [2023-12-26 20:30:21,104][105620] Updated weights for policy 1, policy_version 706680 (0.0006) [2023-12-26 20:30:21,433][105692] Updated weights for policy 0, policy_version 705968 (0.0007) [2023-12-26 20:30:21,481][105692] Updated weights for policy 0, policy_version 705978 (0.0006) [2023-12-26 20:30:21,533][105692] Updated weights for policy 0, policy_version 705988 (0.0006) [2023-12-26 20:30:21,870][105620] Updated weights for policy 1, policy_version 706690 (0.0006) [2023-12-26 20:30:21,925][105620] Updated weights for policy 1, policy_version 706700 (0.0005) [2023-12-26 20:30:21,979][105620] Updated weights for policy 1, policy_version 706710 (0.0005) [2023-12-26 20:30:22,331][105692] Updated weights for policy 0, policy_version 705998 (0.0007) [2023-12-26 20:30:22,398][105692] Updated weights for policy 0, policy_version 706008 (0.0007) [2023-12-26 20:30:22,465][105692] Updated weights for policy 0, policy_version 706018 (0.0005) [2023-12-26 20:30:22,598][105620] Updated weights for policy 1, policy_version 706720 (0.0009) [2023-12-26 20:30:22,653][105620] Updated weights for policy 1, policy_version 706730 (0.0010) [2023-12-26 20:30:22,715][105620] Updated weights for policy 1, policy_version 706740 (0.0010) [2023-12-26 20:30:23,115][105692] Updated weights for policy 0, policy_version 706028 (0.0007) [2023-12-26 20:30:23,184][105692] Updated weights for policy 0, policy_version 706038 (0.0006) [2023-12-26 20:30:23,253][105692] Updated weights for policy 0, policy_version 706048 (0.0008) [2023-12-26 20:30:23,341][105620] Updated weights for policy 1, policy_version 706750 (0.0010) [2023-12-26 20:30:23,398][105620] Updated weights for policy 1, policy_version 706760 (0.0010) [2023-12-26 20:30:23,438][105586] KL-divergence is very high: 148.5950 [2023-12-26 20:30:23,465][105586] KL-divergence is very high: 193.6379 [2023-12-26 20:30:23,466][105620] Updated weights for policy 1, policy_version 706770 (0.0010) [2023-12-26 20:30:23,490][105586] KL-divergence is very high: 249.1707 [2023-12-26 20:30:24,032][105692] Updated weights for policy 0, policy_version 706058 (0.0009) [2023-12-26 20:30:24,077][105692] Updated weights for policy 0, policy_version 706068 (0.0009) [2023-12-26 20:30:24,079][105620] Updated weights for policy 1, policy_version 706780 (0.0009) [2023-12-26 20:30:24,124][105692] Updated weights for policy 0, policy_version 706078 (0.0005) [2023-12-26 20:30:24,125][105620] Updated weights for policy 1, policy_version 706790 (0.0006) [2023-12-26 20:30:24,167][105620] Updated weights for policy 1, policy_version 706800 (0.0006) [2023-12-26 20:30:24,169][105692] Updated weights for policy 0, policy_version 706088 (0.0006) [2023-12-26 20:30:24,917][105692] Updated weights for policy 0, policy_version 706098 (0.0008) [2023-12-26 20:30:24,931][105620] Updated weights for policy 1, policy_version 706810 (0.0009) [2023-12-26 20:30:24,970][105692] Updated weights for policy 0, policy_version 706108 (0.0006) [2023-12-26 20:30:24,988][105620] Updated weights for policy 1, policy_version 706820 (0.0007) [2023-12-26 20:30:25,027][105692] Updated weights for policy 0, policy_version 706118 (0.0007) [2023-12-26 20:30:25,049][105620] Updated weights for policy 1, policy_version 706830 (0.0007) [2023-12-26 20:30:25,108][105620] Updated weights for policy 1, policy_version 706840 (0.0006) [2023-12-26 20:30:25,652][105620] Updated weights for policy 1, policy_version 706850 (0.0005) [2023-12-26 20:30:25,708][105620] Updated weights for policy 1, policy_version 706860 (0.0006) [2023-12-26 20:30:25,756][105620] Updated weights for policy 1, policy_version 706870 (0.0005) [2023-12-26 20:30:25,888][105692] Updated weights for policy 0, policy_version 706128 (0.0009) [2023-12-26 20:30:25,946][105692] Updated weights for policy 0, policy_version 706138 (0.0010) [2023-12-26 20:30:26,012][105692] Updated weights for policy 0, policy_version 706148 (0.0009) [2023-12-26 20:30:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.7, 300 sec: 19771.9). Total num frames: 361783296. Throughput: 0: 9819.2, 1: 9680.6. Samples: 361787984. Policy #0 lag: (min: 2.0, avg: 24.6, max: 34.0) [2023-12-26 20:30:26,063][104569] Avg episode reward: [(0, '8898.619'), (1, '8373.793')] [2023-12-26 20:30:26,283][105620] Updated weights for policy 1, policy_version 706880 (0.0005) [2023-12-26 20:30:26,329][105620] Updated weights for policy 1, policy_version 706890 (0.0005) [2023-12-26 20:30:26,377][105620] Updated weights for policy 1, policy_version 706900 (0.0005) [2023-12-26 20:30:26,931][105620] Updated weights for policy 1, policy_version 706910 (0.0007) [2023-12-26 20:30:26,933][105692] Updated weights for policy 0, policy_version 706158 (0.0008) [2023-12-26 20:30:26,982][105692] Updated weights for policy 0, policy_version 706168 (0.0006) [2023-12-26 20:30:26,987][105620] Updated weights for policy 1, policy_version 706920 (0.0008) [2023-12-26 20:30:27,041][105692] Updated weights for policy 0, policy_version 706178 (0.0007) [2023-12-26 20:30:27,046][105620] Updated weights for policy 1, policy_version 706930 (0.0006) [2023-12-26 20:30:27,781][105620] Updated weights for policy 1, policy_version 706940 (0.0009) [2023-12-26 20:30:27,799][105692] Updated weights for policy 0, policy_version 706188 (0.0007) [2023-12-26 20:30:27,830][105620] Updated weights for policy 1, policy_version 706950 (0.0009) [2023-12-26 20:30:27,848][105692] Updated weights for policy 0, policy_version 706198 (0.0006) [2023-12-26 20:30:27,883][105620] Updated weights for policy 1, policy_version 706960 (0.0006) [2023-12-26 20:30:27,905][105692] Updated weights for policy 0, policy_version 706208 (0.0008) [2023-12-26 20:30:28,485][105620] Updated weights for policy 1, policy_version 706970 (0.0006) [2023-12-26 20:30:28,535][105620] Updated weights for policy 1, policy_version 706980 (0.0008) [2023-12-26 20:30:28,586][105620] Updated weights for policy 1, policy_version 706990 (0.0009) [2023-12-26 20:30:28,633][105620] Updated weights for policy 1, policy_version 707000 (0.0008) [2023-12-26 20:30:28,717][105692] Updated weights for policy 0, policy_version 706218 (0.0008) [2023-12-26 20:30:28,768][105692] Updated weights for policy 0, policy_version 706228 (0.0009) [2023-12-26 20:30:28,822][105692] Updated weights for policy 0, policy_version 706238 (0.0009) [2023-12-26 20:30:28,877][105692] Updated weights for policy 0, policy_version 706248 (0.0009) [2023-12-26 20:30:29,436][105620] Updated weights for policy 1, policy_version 707010 (0.0009) [2023-12-26 20:30:29,496][105620] Updated weights for policy 1, policy_version 707020 (0.0009) [2023-12-26 20:30:29,556][105620] Updated weights for policy 1, policy_version 707030 (0.0008) [2023-12-26 20:30:29,657][105692] Updated weights for policy 0, policy_version 706258 (0.0009) [2023-12-26 20:30:29,704][105692] Updated weights for policy 0, policy_version 706268 (0.0009) [2023-12-26 20:30:29,758][105692] Updated weights for policy 0, policy_version 706278 (0.0009) [2023-12-26 20:30:30,294][105620] Updated weights for policy 1, policy_version 707040 (0.0008) [2023-12-26 20:30:30,347][105620] Updated weights for policy 1, policy_version 707050 (0.0008) [2023-12-26 20:30:30,405][105620] Updated weights for policy 1, policy_version 707060 (0.0009) [2023-12-26 20:30:30,547][105692] Updated weights for policy 0, policy_version 706288 (0.0009) [2023-12-26 20:30:30,598][105692] Updated weights for policy 0, policy_version 706298 (0.0009) [2023-12-26 20:30:30,645][105692] Updated weights for policy 0, policy_version 706308 (0.0009) [2023-12-26 20:30:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19716.3). Total num frames: 361873408. Throughput: 0: 9768.8, 1: 9783.8. Samples: 361846476. Policy #0 lag: (min: 2.0, avg: 24.6, max: 34.0) [2023-12-26 20:30:31,062][104569] Avg episode reward: [(0, '8680.431'), (1, '8727.155')] [2023-12-26 20:30:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000706312_180846592.pth... [2023-12-26 20:30:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000707064_181026816.pth... [2023-12-26 20:30:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000705192_180559872.pth [2023-12-26 20:30:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000705944_180740096.pth [2023-12-26 20:30:31,167][105620] Updated weights for policy 1, policy_version 707070 (0.0009) [2023-12-26 20:30:31,229][105620] Updated weights for policy 1, policy_version 707080 (0.0009) [2023-12-26 20:30:31,295][105620] Updated weights for policy 1, policy_version 707090 (0.0009) [2023-12-26 20:30:31,426][105692] Updated weights for policy 0, policy_version 706318 (0.0009) [2023-12-26 20:30:31,478][105692] Updated weights for policy 0, policy_version 706328 (0.0009) [2023-12-26 20:30:31,531][105692] Updated weights for policy 0, policy_version 706338 (0.0009) [2023-12-26 20:30:32,098][105620] Updated weights for policy 1, policy_version 707100 (0.0009) [2023-12-26 20:30:32,145][105620] Updated weights for policy 1, policy_version 707110 (0.0009) [2023-12-26 20:30:32,206][105620] Updated weights for policy 1, policy_version 707120 (0.0009) [2023-12-26 20:30:32,267][105692] Updated weights for policy 0, policy_version 706348 (0.0009) [2023-12-26 20:30:32,325][105692] Updated weights for policy 0, policy_version 706358 (0.0007) [2023-12-26 20:30:32,386][105692] Updated weights for policy 0, policy_version 706368 (0.0007) [2023-12-26 20:30:33,004][105620] Updated weights for policy 1, policy_version 707130 (0.0009) [2023-12-26 20:30:33,053][105620] Updated weights for policy 1, policy_version 707140 (0.0008) [2023-12-26 20:30:33,053][105692] Updated weights for policy 0, policy_version 706378 (0.0009) [2023-12-26 20:30:33,101][105620] Updated weights for policy 1, policy_version 707150 (0.0008) [2023-12-26 20:30:33,111][105692] Updated weights for policy 0, policy_version 706388 (0.0008) [2023-12-26 20:30:33,152][105620] Updated weights for policy 1, policy_version 707160 (0.0006) [2023-12-26 20:30:33,167][105692] Updated weights for policy 0, policy_version 706398 (0.0007) [2023-12-26 20:30:33,222][105692] Updated weights for policy 0, policy_version 706408 (0.0009) [2023-12-26 20:30:33,919][105620] Updated weights for policy 1, policy_version 707170 (0.0008) [2023-12-26 20:30:33,969][105692] Updated weights for policy 0, policy_version 706418 (0.0007) [2023-12-26 20:30:33,971][105620] Updated weights for policy 1, policy_version 707180 (0.0007) [2023-12-26 20:30:34,018][105692] Updated weights for policy 0, policy_version 706428 (0.0006) [2023-12-26 20:30:34,024][105620] Updated weights for policy 1, policy_version 707190 (0.0007) [2023-12-26 20:30:34,070][105692] Updated weights for policy 0, policy_version 706438 (0.0008) [2023-12-26 20:30:34,661][105620] Updated weights for policy 1, policy_version 707200 (0.0007) [2023-12-26 20:30:34,716][105620] Updated weights for policy 1, policy_version 707210 (0.0005) [2023-12-26 20:30:34,764][105620] Updated weights for policy 1, policy_version 707220 (0.0005) [2023-12-26 20:30:34,914][105692] Updated weights for policy 0, policy_version 706448 (0.0008) [2023-12-26 20:30:34,967][105692] Updated weights for policy 0, policy_version 706458 (0.0006) [2023-12-26 20:30:35,021][105692] Updated weights for policy 0, policy_version 706468 (0.0007) [2023-12-26 20:30:35,463][105620] Updated weights for policy 1, policy_version 707230 (0.0007) [2023-12-26 20:30:35,527][105620] Updated weights for policy 1, policy_version 707240 (0.0009) [2023-12-26 20:30:35,583][105620] Updated weights for policy 1, policy_version 707250 (0.0009) [2023-12-26 20:30:35,649][105692] Updated weights for policy 0, policy_version 706478 (0.0008) [2023-12-26 20:30:35,707][105692] Updated weights for policy 0, policy_version 706488 (0.0010) [2023-12-26 20:30:35,760][105692] Updated weights for policy 0, policy_version 706498 (0.0010) [2023-12-26 20:30:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19716.3). Total num frames: 361971712. Throughput: 0: 9653.5, 1: 9776.9. Samples: 361959324. Policy #0 lag: (min: 11.0, avg: 30.6, max: 32.0) [2023-12-26 20:30:36,063][104569] Avg episode reward: [(0, '6784.071'), (1, '9081.902')] [2023-12-26 20:30:36,319][105620] Updated weights for policy 1, policy_version 707260 (0.0008) [2023-12-26 20:30:36,386][105620] Updated weights for policy 1, policy_version 707270 (0.0010) [2023-12-26 20:30:36,450][105620] Updated weights for policy 1, policy_version 707280 (0.0010) [2023-12-26 20:30:36,525][105692] Updated weights for policy 0, policy_version 706508 (0.0008) [2023-12-26 20:30:36,581][105692] Updated weights for policy 0, policy_version 706518 (0.0008) [2023-12-26 20:30:36,638][105692] Updated weights for policy 0, policy_version 706528 (0.0008) [2023-12-26 20:30:37,143][105620] Updated weights for policy 1, policy_version 707290 (0.0010) [2023-12-26 20:30:37,201][105620] Updated weights for policy 1, policy_version 707300 (0.0009) [2023-12-26 20:30:37,260][105620] Updated weights for policy 1, policy_version 707310 (0.0009) [2023-12-26 20:30:37,315][105620] Updated weights for policy 1, policy_version 707320 (0.0009) [2023-12-26 20:30:37,397][105692] Updated weights for policy 0, policy_version 706538 (0.0007) [2023-12-26 20:30:37,461][105692] Updated weights for policy 0, policy_version 706548 (0.0005) [2023-12-26 20:30:37,515][105692] Updated weights for policy 0, policy_version 706558 (0.0005) [2023-12-26 20:30:37,584][105692] Updated weights for policy 0, policy_version 706568 (0.0006) [2023-12-26 20:30:38,148][105620] Updated weights for policy 1, policy_version 707330 (0.0008) [2023-12-26 20:30:38,186][105692] Updated weights for policy 0, policy_version 706578 (0.0007) [2023-12-26 20:30:38,207][105620] Updated weights for policy 1, policy_version 707340 (0.0008) [2023-12-26 20:30:38,238][105692] Updated weights for policy 0, policy_version 706588 (0.0007) [2023-12-26 20:30:38,260][105620] Updated weights for policy 1, policy_version 707350 (0.0007) [2023-12-26 20:30:38,288][105692] Updated weights for policy 0, policy_version 706598 (0.0007) [2023-12-26 20:30:39,025][105620] Updated weights for policy 1, policy_version 707360 (0.0008) [2023-12-26 20:30:39,075][105620] Updated weights for policy 1, policy_version 707370 (0.0009) [2023-12-26 20:30:39,081][105692] Updated weights for policy 0, policy_version 706608 (0.0006) [2023-12-26 20:30:39,124][105620] Updated weights for policy 1, policy_version 707380 (0.0009) [2023-12-26 20:30:39,150][105692] Updated weights for policy 0, policy_version 706618 (0.0006) [2023-12-26 20:30:39,212][105692] Updated weights for policy 0, policy_version 706628 (0.0010) [2023-12-26 20:30:39,883][105620] Updated weights for policy 1, policy_version 707390 (0.0009) [2023-12-26 20:30:39,916][105692] Updated weights for policy 0, policy_version 706638 (0.0010) [2023-12-26 20:30:39,948][105620] Updated weights for policy 1, policy_version 707400 (0.0010) [2023-12-26 20:30:39,977][105692] Updated weights for policy 0, policy_version 706648 (0.0007) [2023-12-26 20:30:39,994][105586] KL-divergence is very high: 137.8416 [2023-12-26 20:30:40,014][105620] Updated weights for policy 1, policy_version 707410 (0.0007) [2023-12-26 20:30:40,031][105692] Updated weights for policy 0, policy_version 706658 (0.0007) [2023-12-26 20:30:40,049][105586] KL-divergence is very high: 119.8169 [2023-12-26 20:30:40,669][105692] Updated weights for policy 0, policy_version 706668 (0.0006) [2023-12-26 20:30:40,727][105692] Updated weights for policy 0, policy_version 706678 (0.0005) [2023-12-26 20:30:40,742][105620] Updated weights for policy 1, policy_version 707420 (0.0007) [2023-12-26 20:30:40,789][105692] Updated weights for policy 0, policy_version 706688 (0.0005) [2023-12-26 20:30:40,799][105620] Updated weights for policy 1, policy_version 707430 (0.0009) [2023-12-26 20:30:40,854][105620] Updated weights for policy 1, policy_version 707440 (0.0009) [2023-12-26 20:30:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19744.1). Total num frames: 362070016. Throughput: 0: 9611.2, 1: 9692.1. Samples: 362073940. Policy #0 lag: (min: 11.0, avg: 30.6, max: 32.0) [2023-12-26 20:30:41,062][104569] Avg episode reward: [(0, '7386.910'), (1, '8995.215')] [2023-12-26 20:30:41,417][105692] Updated weights for policy 0, policy_version 706698 (0.0006) [2023-12-26 20:30:41,477][105692] Updated weights for policy 0, policy_version 706708 (0.0011) [2023-12-26 20:30:41,536][105692] Updated weights for policy 0, policy_version 706718 (0.0011) [2023-12-26 20:30:41,592][105692] Updated weights for policy 0, policy_version 706728 (0.0011) [2023-12-26 20:30:41,696][105620] Updated weights for policy 1, policy_version 707450 (0.0009) [2023-12-26 20:30:41,759][105620] Updated weights for policy 1, policy_version 707460 (0.0009) [2023-12-26 20:30:41,822][105620] Updated weights for policy 1, policy_version 707470 (0.0009) [2023-12-26 20:30:41,886][105620] Updated weights for policy 1, policy_version 707480 (0.0008) [2023-12-26 20:30:42,412][105692] Updated weights for policy 0, policy_version 706738 (0.0010) [2023-12-26 20:30:42,480][105692] Updated weights for policy 0, policy_version 706748 (0.0010) [2023-12-26 20:30:42,550][105692] Updated weights for policy 0, policy_version 706758 (0.0009) [2023-12-26 20:30:42,590][105620] Updated weights for policy 1, policy_version 707490 (0.0007) [2023-12-26 20:30:42,658][105620] Updated weights for policy 1, policy_version 707500 (0.0009) [2023-12-26 20:30:42,712][105620] Updated weights for policy 1, policy_version 707510 (0.0010) [2023-12-26 20:30:43,232][105692] Updated weights for policy 0, policy_version 706768 (0.0010) [2023-12-26 20:30:43,290][105692] Updated weights for policy 0, policy_version 706778 (0.0010) [2023-12-26 20:30:43,356][105692] Updated weights for policy 0, policy_version 706788 (0.0010) [2023-12-26 20:30:43,407][105620] Updated weights for policy 1, policy_version 707520 (0.0010) [2023-12-26 20:30:43,457][105620] Updated weights for policy 1, policy_version 707530 (0.0009) [2023-12-26 20:30:43,506][105620] Updated weights for policy 1, policy_version 707540 (0.0010) [2023-12-26 20:30:44,122][105692] Updated weights for policy 0, policy_version 706798 (0.0010) [2023-12-26 20:30:44,146][105620] Updated weights for policy 1, policy_version 707550 (0.0009) [2023-12-26 20:30:44,182][105692] Updated weights for policy 0, policy_version 706808 (0.0011) [2023-12-26 20:30:44,207][105620] Updated weights for policy 1, policy_version 707560 (0.0011) [2023-12-26 20:30:44,246][105692] Updated weights for policy 0, policy_version 706818 (0.0011) [2023-12-26 20:30:44,271][105620] Updated weights for policy 1, policy_version 707570 (0.0009) [2023-12-26 20:30:45,017][105692] Updated weights for policy 0, policy_version 706828 (0.0010) [2023-12-26 20:30:45,019][105620] Updated weights for policy 1, policy_version 707580 (0.0010) [2023-12-26 20:30:45,076][105692] Updated weights for policy 0, policy_version 706838 (0.0009) [2023-12-26 20:30:45,082][105620] Updated weights for policy 1, policy_version 707590 (0.0010) [2023-12-26 20:30:45,138][105692] Updated weights for policy 0, policy_version 706848 (0.0006) [2023-12-26 20:30:45,143][105620] Updated weights for policy 1, policy_version 707600 (0.0011) [2023-12-26 20:30:45,884][105692] Updated weights for policy 0, policy_version 706858 (0.0006) [2023-12-26 20:30:45,922][105620] Updated weights for policy 1, policy_version 707610 (0.0011) [2023-12-26 20:30:45,950][105692] Updated weights for policy 0, policy_version 706868 (0.0008) [2023-12-26 20:30:45,986][105620] Updated weights for policy 1, policy_version 707620 (0.0011) [2023-12-26 20:30:46,010][105692] Updated weights for policy 0, policy_version 706878 (0.0008) [2023-12-26 20:30:46,048][105620] Updated weights for policy 1, policy_version 707630 (0.0006) [2023-12-26 20:30:46,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19251.2, 300 sec: 19688.6). Total num frames: 362151936. Throughput: 0: 9559.6, 1: 9676.7. Samples: 362132300. Policy #0 lag: (min: 11.0, avg: 30.6, max: 32.0) [2023-12-26 20:30:46,063][104569] Avg episode reward: [(0, '8502.501'), (1, '8813.029')] [2023-12-26 20:30:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000706888_180994048.pth... [2023-12-26 20:30:46,073][105692] Updated weights for policy 0, policy_version 706888 (0.0006) [2023-12-26 20:30:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000705768_180707328.pth [2023-12-26 20:30:46,101][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000707640_181174272.pth... [2023-12-26 20:30:46,101][105620] Updated weights for policy 1, policy_version 707640 (0.0007) [2023-12-26 20:30:46,104][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000706488_180879360.pth [2023-12-26 20:30:46,745][105620] Updated weights for policy 1, policy_version 707650 (0.0006) [2023-12-26 20:30:46,791][105692] Updated weights for policy 0, policy_version 706898 (0.0009) [2023-12-26 20:30:46,806][105620] Updated weights for policy 1, policy_version 707660 (0.0005) [2023-12-26 20:30:46,852][105692] Updated weights for policy 0, policy_version 706908 (0.0009) [2023-12-26 20:30:46,864][105620] Updated weights for policy 1, policy_version 707670 (0.0005) [2023-12-26 20:30:46,917][105692] Updated weights for policy 0, policy_version 706918 (0.0008) [2023-12-26 20:30:47,501][105620] Updated weights for policy 1, policy_version 707680 (0.0009) [2023-12-26 20:30:47,549][105620] Updated weights for policy 1, policy_version 707690 (0.0010) [2023-12-26 20:30:47,596][105620] Updated weights for policy 1, policy_version 707700 (0.0010) [2023-12-26 20:30:47,713][105692] Updated weights for policy 0, policy_version 706928 (0.0008) [2023-12-26 20:30:47,770][105692] Updated weights for policy 0, policy_version 706938 (0.0008) [2023-12-26 20:30:47,819][105692] Updated weights for policy 0, policy_version 706948 (0.0008) [2023-12-26 20:30:48,343][105620] Updated weights for policy 1, policy_version 707710 (0.0009) [2023-12-26 20:30:48,404][105620] Updated weights for policy 1, policy_version 707720 (0.0012) [2023-12-26 20:30:48,467][105620] Updated weights for policy 1, policy_version 707730 (0.0011) [2023-12-26 20:30:48,509][105692] Updated weights for policy 0, policy_version 706958 (0.0007) [2023-12-26 20:30:48,571][105692] Updated weights for policy 0, policy_version 706968 (0.0008) [2023-12-26 20:30:48,629][105692] Updated weights for policy 0, policy_version 706978 (0.0009) [2023-12-26 20:30:49,210][105620] Updated weights for policy 1, policy_version 707740 (0.0011) [2023-12-26 20:30:49,273][105620] Updated weights for policy 1, policy_version 707750 (0.0010) [2023-12-26 20:30:49,338][105620] Updated weights for policy 1, policy_version 707760 (0.0011) [2023-12-26 20:30:49,433][105692] Updated weights for policy 0, policy_version 706988 (0.0008) [2023-12-26 20:30:49,485][105692] Updated weights for policy 0, policy_version 706998 (0.0008) [2023-12-26 20:30:49,539][105692] Updated weights for policy 0, policy_version 707008 (0.0007) [2023-12-26 20:30:50,103][105620] Updated weights for policy 1, policy_version 707770 (0.0011) [2023-12-26 20:30:50,167][105620] Updated weights for policy 1, policy_version 707780 (0.0010) [2023-12-26 20:30:50,230][105620] Updated weights for policy 1, policy_version 707790 (0.0011) [2023-12-26 20:30:50,279][105692] Updated weights for policy 0, policy_version 707018 (0.0007) [2023-12-26 20:30:50,297][105620] Updated weights for policy 1, policy_version 707800 (0.0011) [2023-12-26 20:30:50,337][105692] Updated weights for policy 0, policy_version 707028 (0.0008) [2023-12-26 20:30:50,405][105692] Updated weights for policy 0, policy_version 707038 (0.0008) [2023-12-26 20:30:50,470][105692] Updated weights for policy 0, policy_version 707048 (0.0008) [2023-12-26 20:30:51,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19251.2, 300 sec: 19688.6). Total num frames: 362250240. Throughput: 0: 9496.3, 1: 9694.8. Samples: 362245020. Policy #0 lag: (min: 11.0, avg: 30.6, max: 32.0) [2023-12-26 20:30:51,063][104569] Avg episode reward: [(0, '9168.790'), (1, '8635.687')] [2023-12-26 20:30:51,064][105620] Updated weights for policy 1, policy_version 707810 (0.0008) [2023-12-26 20:30:51,135][105620] Updated weights for policy 1, policy_version 707820 (0.0009) [2023-12-26 20:30:51,200][105620] Updated weights for policy 1, policy_version 707830 (0.0009) [2023-12-26 20:30:51,254][105692] Updated weights for policy 0, policy_version 707058 (0.0008) [2023-12-26 20:30:51,314][105692] Updated weights for policy 0, policy_version 707068 (0.0010) [2023-12-26 20:30:51,382][105692] Updated weights for policy 0, policy_version 707078 (0.0009) [2023-12-26 20:30:51,969][105620] Updated weights for policy 1, policy_version 707840 (0.0009) [2023-12-26 20:30:52,032][105620] Updated weights for policy 1, policy_version 707850 (0.0009) [2023-12-26 20:30:52,095][105620] Updated weights for policy 1, policy_version 707860 (0.0009) [2023-12-26 20:30:52,141][105692] Updated weights for policy 0, policy_version 707088 (0.0008) [2023-12-26 20:30:52,211][105692] Updated weights for policy 0, policy_version 707098 (0.0010) [2023-12-26 20:30:52,278][105692] Updated weights for policy 0, policy_version 707108 (0.0009) [2023-12-26 20:30:52,862][105620] Updated weights for policy 1, policy_version 707870 (0.0006) [2023-12-26 20:30:52,916][105620] Updated weights for policy 1, policy_version 707880 (0.0007) [2023-12-26 20:30:52,972][105620] Updated weights for policy 1, policy_version 707891 (0.0008) [2023-12-26 20:30:53,064][105692] Updated weights for policy 0, policy_version 707118 (0.0009) [2023-12-26 20:30:53,119][105692] Updated weights for policy 0, policy_version 707128 (0.0008) [2023-12-26 20:30:53,167][105692] Updated weights for policy 0, policy_version 707138 (0.0008) [2023-12-26 20:30:53,633][105620] Updated weights for policy 1, policy_version 707901 (0.0010) [2023-12-26 20:30:53,661][105586] KL-divergence is very high: 106.9805 [2023-12-26 20:30:53,694][105620] Updated weights for policy 1, policy_version 707911 (0.0010) [2023-12-26 20:30:53,712][105586] KL-divergence is very high: 163.2732 [2023-12-26 20:30:53,762][105620] Updated weights for policy 1, policy_version 707921 (0.0010) [2023-12-26 20:30:53,765][105586] KL-divergence is very high: 137.7468 [2023-12-26 20:30:53,969][105692] Updated weights for policy 0, policy_version 707149 (0.0009) [2023-12-26 20:30:54,028][105692] Updated weights for policy 0, policy_version 707159 (0.0008) [2023-12-26 20:30:54,100][105692] Updated weights for policy 0, policy_version 707169 (0.0008) [2023-12-26 20:30:54,505][105620] Updated weights for policy 1, policy_version 707931 (0.0010) [2023-12-26 20:30:54,565][105620] Updated weights for policy 1, policy_version 707941 (0.0008) [2023-12-26 20:30:54,616][105620] Updated weights for policy 1, policy_version 707951 (0.0010) [2023-12-26 20:30:54,825][105692] Updated weights for policy 0, policy_version 707179 (0.0007) [2023-12-26 20:30:54,892][105692] Updated weights for policy 0, policy_version 707189 (0.0007) [2023-12-26 20:30:54,948][105692] Updated weights for policy 0, policy_version 707199 (0.0008) [2023-12-26 20:30:55,269][105620] Updated weights for policy 1, policy_version 707961 (0.0010) [2023-12-26 20:30:55,326][105620] Updated weights for policy 1, policy_version 707971 (0.0010) [2023-12-26 20:30:55,388][105620] Updated weights for policy 1, policy_version 707981 (0.0010) [2023-12-26 20:30:55,453][105620] Updated weights for policy 1, policy_version 707991 (0.0010) [2023-12-26 20:30:55,694][105692] Updated weights for policy 0, policy_version 707209 (0.0008) [2023-12-26 20:30:55,743][105692] Updated weights for policy 0, policy_version 707219 (0.0008) [2023-12-26 20:30:55,794][105692] Updated weights for policy 0, policy_version 707229 (0.0008) [2023-12-26 20:30:55,840][105692] Updated weights for policy 0, policy_version 707239 (0.0008) [2023-12-26 20:30:56,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19688.6). Total num frames: 362348544. Throughput: 0: 9447.0, 1: 9618.2. Samples: 362357256. Policy #0 lag: (min: 11.0, avg: 30.6, max: 32.0) [2023-12-26 20:30:56,063][104569] Avg episode reward: [(0, '9351.414'), (1, '8550.468')] [2023-12-26 20:30:56,190][105620] Updated weights for policy 1, policy_version 708001 (0.0010) [2023-12-26 20:30:56,245][105620] Updated weights for policy 1, policy_version 708011 (0.0010) [2023-12-26 20:30:56,299][105620] Updated weights for policy 1, policy_version 708021 (0.0010) [2023-12-26 20:30:56,616][105692] Updated weights for policy 0, policy_version 707249 (0.0008) [2023-12-26 20:30:56,663][105692] Updated weights for policy 0, policy_version 707259 (0.0007) [2023-12-26 20:30:56,719][105692] Updated weights for policy 0, policy_version 707269 (0.0008) [2023-12-26 20:30:57,050][105620] Updated weights for policy 1, policy_version 708031 (0.0010) [2023-12-26 20:30:57,107][105620] Updated weights for policy 1, policy_version 708041 (0.0010) [2023-12-26 20:30:57,161][105620] Updated weights for policy 1, policy_version 708051 (0.0010) [2023-12-26 20:30:57,469][105692] Updated weights for policy 0, policy_version 707279 (0.0008) [2023-12-26 20:30:57,518][105692] Updated weights for policy 0, policy_version 707289 (0.0008) [2023-12-26 20:30:57,566][105692] Updated weights for policy 0, policy_version 707299 (0.0008) [2023-12-26 20:30:57,876][105620] Updated weights for policy 1, policy_version 708061 (0.0009) [2023-12-26 20:30:57,939][105620] Updated weights for policy 1, policy_version 708071 (0.0010) [2023-12-26 20:30:57,997][105620] Updated weights for policy 1, policy_version 708081 (0.0010) [2023-12-26 20:30:58,298][105692] Updated weights for policy 0, policy_version 707309 (0.0008) [2023-12-26 20:30:58,370][105692] Updated weights for policy 0, policy_version 707319 (0.0008) [2023-12-26 20:30:58,436][105692] Updated weights for policy 0, policy_version 707329 (0.0008) [2023-12-26 20:30:58,781][105620] Updated weights for policy 1, policy_version 708091 (0.0009) [2023-12-26 20:30:58,847][105620] Updated weights for policy 1, policy_version 708101 (0.0008) [2023-12-26 20:30:58,911][105620] Updated weights for policy 1, policy_version 708111 (0.0008) [2023-12-26 20:30:59,162][105692] Updated weights for policy 0, policy_version 707339 (0.0008) [2023-12-26 20:30:59,209][105692] Updated weights for policy 0, policy_version 707349 (0.0006) [2023-12-26 20:30:59,272][105692] Updated weights for policy 0, policy_version 707359 (0.0009) [2023-12-26 20:30:59,608][105620] Updated weights for policy 1, policy_version 708121 (0.0007) [2023-12-26 20:30:59,667][105620] Updated weights for policy 1, policy_version 708131 (0.0011) [2023-12-26 20:30:59,719][105620] Updated weights for policy 1, policy_version 708141 (0.0010) [2023-12-26 20:30:59,771][105620] Updated weights for policy 1, policy_version 708151 (0.0010) [2023-12-26 20:30:59,948][105692] Updated weights for policy 0, policy_version 707369 (0.0008) [2023-12-26 20:31:00,009][105692] Updated weights for policy 0, policy_version 707379 (0.0008) [2023-12-26 20:31:00,064][105692] Updated weights for policy 0, policy_version 707389 (0.0009) [2023-12-26 20:31:00,130][105692] Updated weights for policy 0, policy_version 707399 (0.0009) [2023-12-26 20:31:00,416][105620] Updated weights for policy 1, policy_version 708161 (0.0006) [2023-12-26 20:31:00,471][105620] Updated weights for policy 1, policy_version 708171 (0.0005) [2023-12-26 20:31:00,517][105620] Updated weights for policy 1, policy_version 708181 (0.0005) [2023-12-26 20:31:00,946][105692] Updated weights for policy 0, policy_version 707409 (0.0006) [2023-12-26 20:31:00,999][105692] Updated weights for policy 0, policy_version 707419 (0.0006) [2023-12-26 20:31:01,057][105620] Updated weights for policy 1, policy_version 708191 (0.0007) [2023-12-26 20:31:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19660.8). Total num frames: 362438656. Throughput: 0: 9419.3, 1: 9656.0. Samples: 362413428. Policy #0 lag: (min: 11.0, avg: 30.6, max: 32.0) [2023-12-26 20:31:01,062][104569] Avg episode reward: [(0, '8345.788'), (1, '8725.576')] [2023-12-26 20:31:01,067][105692] Updated weights for policy 0, policy_version 707429 (0.0006) [2023-12-26 20:31:01,083][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000707432_181133312.pth... [2023-12-26 20:31:01,086][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000706312_180846592.pth [2023-12-26 20:31:01,124][105620] Updated weights for policy 1, policy_version 708201 (0.0006) [2023-12-26 20:31:01,190][105620] Updated weights for policy 1, policy_version 708211 (0.0008) [2023-12-26 20:31:01,216][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000708216_181321728.pth... [2023-12-26 20:31:01,220][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000707064_181026816.pth [2023-12-26 20:31:01,807][105620] Updated weights for policy 1, policy_version 708221 (0.0008) [2023-12-26 20:31:01,809][105692] Updated weights for policy 0, policy_version 707439 (0.0007) [2023-12-26 20:31:01,866][105692] Updated weights for policy 0, policy_version 707449 (0.0007) [2023-12-26 20:31:01,867][105620] Updated weights for policy 1, policy_version 708231 (0.0008) [2023-12-26 20:31:01,919][105692] Updated weights for policy 0, policy_version 707459 (0.0007) [2023-12-26 20:31:01,924][105620] Updated weights for policy 1, policy_version 708241 (0.0008) [2023-12-26 20:31:02,687][105620] Updated weights for policy 1, policy_version 708251 (0.0009) [2023-12-26 20:31:02,698][105692] Updated weights for policy 0, policy_version 707469 (0.0006) [2023-12-26 20:31:02,746][105620] Updated weights for policy 1, policy_version 708261 (0.0011) [2023-12-26 20:31:02,757][105692] Updated weights for policy 0, policy_version 707479 (0.0007) [2023-12-26 20:31:02,805][105620] Updated weights for policy 1, policy_version 708271 (0.0011) [2023-12-26 20:31:02,812][105692] Updated weights for policy 0, policy_version 707489 (0.0006) [2023-12-26 20:31:03,520][105692] Updated weights for policy 0, policy_version 707499 (0.0007) [2023-12-26 20:31:03,531][105620] Updated weights for policy 1, policy_version 708281 (0.0010) [2023-12-26 20:31:03,574][105692] Updated weights for policy 0, policy_version 707509 (0.0006) [2023-12-26 20:31:03,584][105620] Updated weights for policy 1, policy_version 708291 (0.0011) [2023-12-26 20:31:03,631][105692] Updated weights for policy 0, policy_version 707519 (0.0005) [2023-12-26 20:31:03,633][105620] Updated weights for policy 1, policy_version 708301 (0.0010) [2023-12-26 20:31:03,694][105620] Updated weights for policy 1, policy_version 708311 (0.0007) [2023-12-26 20:31:04,331][105692] Updated weights for policy 0, policy_version 707529 (0.0008) [2023-12-26 20:31:04,383][105692] Updated weights for policy 0, policy_version 707539 (0.0008) [2023-12-26 20:31:04,436][105692] Updated weights for policy 0, policy_version 707549 (0.0008) [2023-12-26 20:31:04,492][105620] Updated weights for policy 1, policy_version 708321 (0.0008) [2023-12-26 20:31:04,495][105692] Updated weights for policy 0, policy_version 707559 (0.0006) [2023-12-26 20:31:04,551][105620] Updated weights for policy 1, policy_version 708331 (0.0008) [2023-12-26 20:31:04,603][105620] Updated weights for policy 1, policy_version 708341 (0.0008) [2023-12-26 20:31:05,243][105692] Updated weights for policy 0, policy_version 707569 (0.0008) [2023-12-26 20:31:05,301][105692] Updated weights for policy 0, policy_version 707579 (0.0008) [2023-12-26 20:31:05,352][105620] Updated weights for policy 1, policy_version 708351 (0.0010) [2023-12-26 20:31:05,358][105692] Updated weights for policy 0, policy_version 707589 (0.0007) [2023-12-26 20:31:05,404][105620] Updated weights for policy 1, policy_version 708361 (0.0010) [2023-12-26 20:31:05,465][105620] Updated weights for policy 1, policy_version 708371 (0.0008) [2023-12-26 20:31:06,009][105620] Updated weights for policy 1, policy_version 708381 (0.0005) [2023-12-26 20:31:06,060][105620] Updated weights for policy 1, policy_version 708391 (0.0005) [2023-12-26 20:31:06,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19114.7, 300 sec: 19660.8). Total num frames: 362536960. Throughput: 0: 9351.2, 1: 9780.0. Samples: 362530888. Policy #0 lag: (min: 11.0, avg: 30.6, max: 32.0) [2023-12-26 20:31:06,062][104569] Avg episode reward: [(0, '6850.183'), (1, '8661.709')] [2023-12-26 20:31:06,112][105620] Updated weights for policy 1, policy_version 708401 (0.0009) [2023-12-26 20:31:06,173][105692] Updated weights for policy 0, policy_version 707599 (0.0007) [2023-12-26 20:31:06,237][105692] Updated weights for policy 0, policy_version 707609 (0.0008) [2023-12-26 20:31:06,303][105692] Updated weights for policy 0, policy_version 707619 (0.0006) [2023-12-26 20:31:06,837][105620] Updated weights for policy 1, policy_version 708411 (0.0011) [2023-12-26 20:31:06,897][105620] Updated weights for policy 1, policy_version 708421 (0.0011) [2023-12-26 20:31:06,950][105692] Updated weights for policy 0, policy_version 707629 (0.0007) [2023-12-26 20:31:06,957][105620] Updated weights for policy 1, policy_version 708431 (0.0011) [2023-12-26 20:31:06,958][105586] KL-divergence is very high: 102.3978 [2023-12-26 20:31:07,013][105692] Updated weights for policy 0, policy_version 707639 (0.0007) [2023-12-26 20:31:07,069][105692] Updated weights for policy 0, policy_version 707649 (0.0005) [2023-12-26 20:31:07,678][105692] Updated weights for policy 0, policy_version 707659 (0.0007) [2023-12-26 20:31:07,689][105620] Updated weights for policy 1, policy_version 708441 (0.0010) [2023-12-26 20:31:07,743][105692] Updated weights for policy 0, policy_version 707669 (0.0009) [2023-12-26 20:31:07,749][105620] Updated weights for policy 1, policy_version 708451 (0.0005) [2023-12-26 20:31:07,788][105692] Updated weights for policy 0, policy_version 707679 (0.0010) [2023-12-26 20:31:07,805][105620] Updated weights for policy 1, policy_version 708461 (0.0007) [2023-12-26 20:31:07,864][105620] Updated weights for policy 1, policy_version 708471 (0.0010) [2023-12-26 20:31:08,517][105692] Updated weights for policy 0, policy_version 707689 (0.0010) [2023-12-26 20:31:08,571][105620] Updated weights for policy 1, policy_version 708481 (0.0011) [2023-12-26 20:31:08,573][105692] Updated weights for policy 0, policy_version 707699 (0.0006) [2023-12-26 20:31:08,627][105620] Updated weights for policy 1, policy_version 708491 (0.0011) [2023-12-26 20:31:08,634][105692] Updated weights for policy 0, policy_version 707709 (0.0006) [2023-12-26 20:31:08,684][105620] Updated weights for policy 1, policy_version 708501 (0.0010) [2023-12-26 20:31:08,700][105692] Updated weights for policy 0, policy_version 707719 (0.0005) [2023-12-26 20:31:09,293][105692] Updated weights for policy 0, policy_version 707729 (0.0008) [2023-12-26 20:31:09,363][105692] Updated weights for policy 0, policy_version 707739 (0.0010) [2023-12-26 20:31:09,429][105692] Updated weights for policy 0, policy_version 707749 (0.0007) [2023-12-26 20:31:09,478][105620] Updated weights for policy 1, policy_version 708511 (0.0009) [2023-12-26 20:31:09,548][105620] Updated weights for policy 1, policy_version 708521 (0.0008) [2023-12-26 20:31:09,615][105620] Updated weights for policy 1, policy_version 708531 (0.0009) [2023-12-26 20:31:10,142][105692] Updated weights for policy 0, policy_version 707759 (0.0008) [2023-12-26 20:31:10,204][105692] Updated weights for policy 0, policy_version 707769 (0.0008) [2023-12-26 20:31:10,264][105692] Updated weights for policy 0, policy_version 707779 (0.0009) [2023-12-26 20:31:10,355][105620] Updated weights for policy 1, policy_version 708541 (0.0009) [2023-12-26 20:31:10,411][105620] Updated weights for policy 1, policy_version 708551 (0.0008) [2023-12-26 20:31:10,461][105620] Updated weights for policy 1, policy_version 708561 (0.0008) [2023-12-26 20:31:11,043][105692] Updated weights for policy 0, policy_version 707789 (0.0009) [2023-12-26 20:31:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19114.7, 300 sec: 19660.8). Total num frames: 362635264. Throughput: 0: 9432.9, 1: 9694.1. Samples: 362648696. Policy #0 lag: (min: 11.0, avg: 30.6, max: 32.0) [2023-12-26 20:31:11,063][104569] Avg episode reward: [(0, '7665.399'), (1, '8263.281')] [2023-12-26 20:31:11,110][105692] Updated weights for policy 0, policy_version 707799 (0.0010) [2023-12-26 20:31:11,176][105692] Updated weights for policy 0, policy_version 707809 (0.0008) [2023-12-26 20:31:11,204][105620] Updated weights for policy 1, policy_version 708571 (0.0006) [2023-12-26 20:31:11,263][105620] Updated weights for policy 1, policy_version 708581 (0.0008) [2023-12-26 20:31:11,320][105620] Updated weights for policy 1, policy_version 708591 (0.0009) [2023-12-26 20:31:11,878][105692] Updated weights for policy 0, policy_version 707819 (0.0007) [2023-12-26 20:31:11,940][105692] Updated weights for policy 0, policy_version 707829 (0.0009) [2023-12-26 20:31:11,996][105692] Updated weights for policy 0, policy_version 707839 (0.0009) [2023-12-26 20:31:12,108][105620] Updated weights for policy 1, policy_version 708601 (0.0009) [2023-12-26 20:31:12,170][105620] Updated weights for policy 1, policy_version 708611 (0.0009) [2023-12-26 20:31:12,226][105620] Updated weights for policy 1, policy_version 708621 (0.0009) [2023-12-26 20:31:12,292][105620] Updated weights for policy 1, policy_version 708631 (0.0008) [2023-12-26 20:31:12,794][105692] Updated weights for policy 0, policy_version 707849 (0.0009) [2023-12-26 20:31:12,857][105692] Updated weights for policy 0, policy_version 707859 (0.0009) [2023-12-26 20:31:12,911][105692] Updated weights for policy 0, policy_version 707869 (0.0009) [2023-12-26 20:31:12,967][105692] Updated weights for policy 0, policy_version 707879 (0.0009) [2023-12-26 20:31:13,046][105620] Updated weights for policy 1, policy_version 708641 (0.0009) [2023-12-26 20:31:13,106][105620] Updated weights for policy 1, policy_version 708651 (0.0009) [2023-12-26 20:31:13,164][105620] Updated weights for policy 1, policy_version 708662 (0.0007) [2023-12-26 20:31:13,743][105620] Updated weights for policy 1, policy_version 708672 (0.0007) [2023-12-26 20:31:13,799][105620] Updated weights for policy 1, policy_version 708682 (0.0009) [2023-12-26 20:31:13,807][105692] Updated weights for policy 0, policy_version 707889 (0.0007) [2023-12-26 20:31:13,861][105620] Updated weights for policy 1, policy_version 708692 (0.0007) [2023-12-26 20:31:13,868][105692] Updated weights for policy 0, policy_version 707899 (0.0007) [2023-12-26 20:31:13,925][105692] Updated weights for policy 0, policy_version 707909 (0.0009) [2023-12-26 20:31:14,498][105620] Updated weights for policy 1, policy_version 708702 (0.0006) [2023-12-26 20:31:14,556][105620] Updated weights for policy 1, policy_version 708712 (0.0005) [2023-12-26 20:31:14,613][105620] Updated weights for policy 1, policy_version 708722 (0.0006) [2023-12-26 20:31:14,624][105692] Updated weights for policy 0, policy_version 707919 (0.0008) [2023-12-26 20:31:14,676][105692] Updated weights for policy 0, policy_version 707929 (0.0008) [2023-12-26 20:31:14,739][105692] Updated weights for policy 0, policy_version 707939 (0.0008) [2023-12-26 20:31:15,320][105620] Updated weights for policy 1, policy_version 708732 (0.0009) [2023-12-26 20:31:15,381][105692] Updated weights for policy 0, policy_version 707949 (0.0007) [2023-12-26 20:31:15,387][105620] Updated weights for policy 1, policy_version 708742 (0.0009) [2023-12-26 20:31:15,431][105692] Updated weights for policy 0, policy_version 707959 (0.0006) [2023-12-26 20:31:15,449][105620] Updated weights for policy 1, policy_version 708752 (0.0009) [2023-12-26 20:31:15,478][105692] Updated weights for policy 0, policy_version 707969 (0.0007) [2023-12-26 20:31:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19114.6, 300 sec: 19660.8). Total num frames: 362733568. Throughput: 0: 9464.1, 1: 9604.1. Samples: 362704544. Policy #0 lag: (min: 11.0, avg: 30.6, max: 32.0) [2023-12-26 20:31:16,063][104569] Avg episode reward: [(0, '8882.536'), (1, '8330.128')] [2023-12-26 20:31:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000707976_181272576.pth... [2023-12-26 20:31:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000708760_181460992.pth... [2023-12-26 20:31:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000707640_181174272.pth [2023-12-26 20:31:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000706888_180994048.pth [2023-12-26 20:31:16,129][105692] Updated weights for policy 0, policy_version 707979 (0.0006) [2023-12-26 20:31:16,177][105620] Updated weights for policy 1, policy_version 708762 (0.0008) [2023-12-26 20:31:16,179][105692] Updated weights for policy 0, policy_version 707989 (0.0007) [2023-12-26 20:31:16,234][105692] Updated weights for policy 0, policy_version 707999 (0.0006) [2023-12-26 20:31:16,235][105620] Updated weights for policy 1, policy_version 708772 (0.0010) [2023-12-26 20:31:16,298][105620] Updated weights for policy 1, policy_version 708782 (0.0009) [2023-12-26 20:31:16,359][105620] Updated weights for policy 1, policy_version 708792 (0.0010) [2023-12-26 20:31:16,907][105692] Updated weights for policy 0, policy_version 708009 (0.0007) [2023-12-26 20:31:16,965][105692] Updated weights for policy 0, policy_version 708019 (0.0007) [2023-12-26 20:31:17,015][105692] Updated weights for policy 0, policy_version 708029 (0.0006) [2023-12-26 20:31:17,025][105620] Updated weights for policy 1, policy_version 708802 (0.0011) [2023-12-26 20:31:17,063][105692] Updated weights for policy 0, policy_version 708039 (0.0006) [2023-12-26 20:31:17,073][105620] Updated weights for policy 1, policy_version 708812 (0.0010) [2023-12-26 20:31:17,130][105620] Updated weights for policy 1, policy_version 708822 (0.0010) [2023-12-26 20:31:17,831][105692] Updated weights for policy 0, policy_version 708049 (0.0008) [2023-12-26 20:31:17,880][105692] Updated weights for policy 0, policy_version 708059 (0.0008) [2023-12-26 20:31:17,887][105620] Updated weights for policy 1, policy_version 708832 (0.0011) [2023-12-26 20:31:17,929][105692] Updated weights for policy 0, policy_version 708069 (0.0007) [2023-12-26 20:31:17,946][105620] Updated weights for policy 1, policy_version 708842 (0.0011) [2023-12-26 20:31:18,001][105620] Updated weights for policy 1, policy_version 708852 (0.0010) [2023-12-26 20:31:18,637][105620] Updated weights for policy 1, policy_version 708862 (0.0009) [2023-12-26 20:31:18,696][105692] Updated weights for policy 0, policy_version 708079 (0.0007) [2023-12-26 20:31:18,704][105620] Updated weights for policy 1, policy_version 708872 (0.0011) [2023-12-26 20:31:18,761][105692] Updated weights for policy 0, policy_version 708089 (0.0007) [2023-12-26 20:31:18,768][105620] Updated weights for policy 1, policy_version 708882 (0.0010) [2023-12-26 20:31:18,823][105692] Updated weights for policy 0, policy_version 708099 (0.0007) [2023-12-26 20:31:19,507][105620] Updated weights for policy 1, policy_version 708892 (0.0010) [2023-12-26 20:31:19,555][105620] Updated weights for policy 1, policy_version 708902 (0.0009) [2023-12-26 20:31:19,608][105692] Updated weights for policy 0, policy_version 708109 (0.0009) [2023-12-26 20:31:19,614][105620] Updated weights for policy 1, policy_version 708912 (0.0006) [2023-12-26 20:31:19,669][105692] Updated weights for policy 0, policy_version 708119 (0.0009) [2023-12-26 20:31:19,727][105692] Updated weights for policy 0, policy_version 708129 (0.0009) [2023-12-26 20:31:20,299][105620] Updated weights for policy 1, policy_version 708922 (0.0006) [2023-12-26 20:31:20,358][105620] Updated weights for policy 1, policy_version 708932 (0.0008) [2023-12-26 20:31:20,420][105620] Updated weights for policy 1, policy_version 708942 (0.0007) [2023-12-26 20:31:20,480][105620] Updated weights for policy 1, policy_version 708952 (0.0006) [2023-12-26 20:31:20,488][105692] Updated weights for policy 0, policy_version 708139 (0.0010) [2023-12-26 20:31:20,548][105692] Updated weights for policy 0, policy_version 708149 (0.0011) [2023-12-26 20:31:20,616][105692] Updated weights for policy 0, policy_version 708159 (0.0009) [2023-12-26 20:31:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19633.0). Total num frames: 362831872. Throughput: 0: 9539.8, 1: 9666.1. Samples: 362823588. Policy #0 lag: (min: 11.0, avg: 30.6, max: 32.0) [2023-12-26 20:31:21,063][104569] Avg episode reward: [(0, '8900.979'), (1, '8993.841')] [2023-12-26 20:31:21,095][105620] Updated weights for policy 1, policy_version 708962 (0.0011) [2023-12-26 20:31:21,154][105620] Updated weights for policy 1, policy_version 708972 (0.0010) [2023-12-26 20:31:21,208][105620] Updated weights for policy 1, policy_version 708982 (0.0009) [2023-12-26 20:31:21,402][105692] Updated weights for policy 0, policy_version 708169 (0.0009) [2023-12-26 20:31:21,459][105692] Updated weights for policy 0, policy_version 708179 (0.0009) [2023-12-26 20:31:21,519][105692] Updated weights for policy 0, policy_version 708189 (0.0007) [2023-12-26 20:31:21,575][105692] Updated weights for policy 0, policy_version 708199 (0.0006) [2023-12-26 20:31:21,922][105620] Updated weights for policy 1, policy_version 708992 (0.0009) [2023-12-26 20:31:21,978][105620] Updated weights for policy 1, policy_version 709002 (0.0008) [2023-12-26 20:31:22,033][105620] Updated weights for policy 1, policy_version 709012 (0.0008) [2023-12-26 20:31:22,267][105692] Updated weights for policy 0, policy_version 708209 (0.0008) [2023-12-26 20:31:22,328][105692] Updated weights for policy 0, policy_version 708219 (0.0011) [2023-12-26 20:31:22,390][105692] Updated weights for policy 0, policy_version 708229 (0.0010) [2023-12-26 20:31:22,776][105620] Updated weights for policy 1, policy_version 709022 (0.0010) [2023-12-26 20:31:22,841][105620] Updated weights for policy 1, policy_version 709032 (0.0011) [2023-12-26 20:31:22,905][105620] Updated weights for policy 1, policy_version 709042 (0.0011) [2023-12-26 20:31:23,031][105692] Updated weights for policy 0, policy_version 708239 (0.0007) [2023-12-26 20:31:23,093][105692] Updated weights for policy 0, policy_version 708249 (0.0007) [2023-12-26 20:31:23,146][105692] Updated weights for policy 0, policy_version 708259 (0.0009) [2023-12-26 20:31:23,647][105620] Updated weights for policy 1, policy_version 709052 (0.0011) [2023-12-26 20:31:23,695][105620] Updated weights for policy 1, policy_version 709062 (0.0010) [2023-12-26 20:31:23,754][105620] Updated weights for policy 1, policy_version 709072 (0.0009) [2023-12-26 20:31:23,763][105692] Updated weights for policy 0, policy_version 708269 (0.0010) [2023-12-26 20:31:23,821][105692] Updated weights for policy 0, policy_version 708279 (0.0008) [2023-12-26 20:31:23,874][105692] Updated weights for policy 0, policy_version 708289 (0.0009) [2023-12-26 20:31:24,455][105620] Updated weights for policy 1, policy_version 709082 (0.0006) [2023-12-26 20:31:24,509][105620] Updated weights for policy 1, policy_version 709092 (0.0010) [2023-12-26 20:31:24,558][105620] Updated weights for policy 1, policy_version 709102 (0.0010) [2023-12-26 20:31:24,610][105620] Updated weights for policy 1, policy_version 709112 (0.0010) [2023-12-26 20:31:24,619][105692] Updated weights for policy 0, policy_version 708299 (0.0010) [2023-12-26 20:31:24,674][105692] Updated weights for policy 0, policy_version 708309 (0.0007) [2023-12-26 20:31:24,721][105692] Updated weights for policy 0, policy_version 708319 (0.0005) [2023-12-26 20:31:25,214][105620] Updated weights for policy 1, policy_version 709122 (0.0005) [2023-12-26 20:31:25,260][105620] Updated weights for policy 1, policy_version 709132 (0.0005) [2023-12-26 20:31:25,305][105620] Updated weights for policy 1, policy_version 709142 (0.0010) [2023-12-26 20:31:25,311][105692] Updated weights for policy 0, policy_version 708329 (0.0006) [2023-12-26 20:31:25,362][105692] Updated weights for policy 0, policy_version 708339 (0.0010) [2023-12-26 20:31:25,424][105692] Updated weights for policy 0, policy_version 708349 (0.0010) [2023-12-26 20:31:25,487][105692] Updated weights for policy 0, policy_version 708359 (0.0010) [2023-12-26 20:31:26,041][105692] Updated weights for policy 0, policy_version 708369 (0.0010) [2023-12-26 20:31:26,053][105620] Updated weights for policy 1, policy_version 709152 (0.0008) [2023-12-26 20:31:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19114.7, 300 sec: 19633.0). Total num frames: 362930176. Throughput: 0: 9544.8, 1: 9780.4. Samples: 362943576. Policy #0 lag: (min: 11.0, avg: 30.6, max: 32.0) [2023-12-26 20:31:26,062][104569] Avg episode reward: [(0, '8896.720'), (1, '9082.441')] [2023-12-26 20:31:26,089][105692] Updated weights for policy 0, policy_version 708379 (0.0010) [2023-12-26 20:31:26,115][105620] Updated weights for policy 1, policy_version 709162 (0.0009) [2023-12-26 20:31:26,147][105692] Updated weights for policy 0, policy_version 708389 (0.0010) [2023-12-26 20:31:26,175][105620] Updated weights for policy 1, policy_version 709172 (0.0010) [2023-12-26 20:31:26,769][105692] Updated weights for policy 0, policy_version 708399 (0.0007) [2023-12-26 20:31:26,787][105620] Updated weights for policy 1, policy_version 709182 (0.0008) [2023-12-26 20:31:26,825][105692] Updated weights for policy 0, policy_version 708409 (0.0005) [2023-12-26 20:31:26,841][105620] Updated weights for policy 1, policy_version 709192 (0.0005) [2023-12-26 20:31:26,885][105692] Updated weights for policy 0, policy_version 708419 (0.0008) [2023-12-26 20:31:26,900][105620] Updated weights for policy 1, policy_version 709202 (0.0006) [2023-12-26 20:31:27,409][105692] Updated weights for policy 0, policy_version 708429 (0.0008) [2023-12-26 20:31:27,455][105692] Updated weights for policy 0, policy_version 708439 (0.0005) [2023-12-26 20:31:27,500][105692] Updated weights for policy 0, policy_version 708449 (0.0005) [2023-12-26 20:31:27,556][105620] Updated weights for policy 1, policy_version 709212 (0.0006) [2023-12-26 20:31:27,616][105620] Updated weights for policy 1, policy_version 709222 (0.0007) [2023-12-26 20:31:27,674][105620] Updated weights for policy 1, policy_version 709232 (0.0008) [2023-12-26 20:31:28,184][105692] Updated weights for policy 0, policy_version 708459 (0.0007) [2023-12-26 20:31:28,242][105692] Updated weights for policy 0, policy_version 708469 (0.0010) [2023-12-26 20:31:28,297][105692] Updated weights for policy 0, policy_version 708479 (0.0010) [2023-12-26 20:31:28,335][105620] Updated weights for policy 1, policy_version 709242 (0.0008) [2023-12-26 20:31:28,402][105620] Updated weights for policy 1, policy_version 709252 (0.0008) [2023-12-26 20:31:28,460][105620] Updated weights for policy 1, policy_version 709262 (0.0007) [2023-12-26 20:31:28,515][105620] Updated weights for policy 1, policy_version 709272 (0.0005) [2023-12-26 20:31:29,013][105692] Updated weights for policy 0, policy_version 708489 (0.0010) [2023-12-26 20:31:29,061][105692] Updated weights for policy 0, policy_version 708499 (0.0010) [2023-12-26 20:31:29,112][105692] Updated weights for policy 0, policy_version 708509 (0.0010) [2023-12-26 20:31:29,157][105692] Updated weights for policy 0, policy_version 708519 (0.0010) [2023-12-26 20:31:29,239][105620] Updated weights for policy 1, policy_version 709282 (0.0008) [2023-12-26 20:31:29,291][105620] Updated weights for policy 1, policy_version 709292 (0.0008) [2023-12-26 20:31:29,358][105620] Updated weights for policy 1, policy_version 709302 (0.0009) [2023-12-26 20:31:29,820][105692] Updated weights for policy 0, policy_version 708529 (0.0007) [2023-12-26 20:31:29,877][105692] Updated weights for policy 0, policy_version 708539 (0.0009) [2023-12-26 20:31:29,924][105692] Updated weights for policy 0, policy_version 708549 (0.0009) [2023-12-26 20:31:30,155][105620] Updated weights for policy 1, policy_version 709312 (0.0009) [2023-12-26 20:31:30,203][105620] Updated weights for policy 1, policy_version 709322 (0.0007) [2023-12-26 20:31:30,252][105620] Updated weights for policy 1, policy_version 709332 (0.0007) [2023-12-26 20:31:30,669][105692] Updated weights for policy 0, policy_version 708559 (0.0009) [2023-12-26 20:31:30,716][105692] Updated weights for policy 0, policy_version 708569 (0.0009) [2023-12-26 20:31:30,768][105692] Updated weights for policy 0, policy_version 708579 (0.0008) [2023-12-26 20:31:30,948][105620] Updated weights for policy 1, policy_version 709342 (0.0008) [2023-12-26 20:31:31,004][105620] Updated weights for policy 1, policy_version 709352 (0.0008) [2023-12-26 20:31:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 363036672. Throughput: 0: 9657.8, 1: 9815.3. Samples: 363008588. Policy #0 lag: (min: 11.0, avg: 30.6, max: 32.0) [2023-12-26 20:31:31,063][104569] Avg episode reward: [(0, '8895.133'), (1, '9085.173')] [2023-12-26 20:31:31,063][105620] Updated weights for policy 1, policy_version 709362 (0.0008) [2023-12-26 20:31:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000708584_181428224.pth... [2023-12-26 20:31:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000707432_181133312.pth [2023-12-26 20:31:31,099][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000709368_181616640.pth... [2023-12-26 20:31:31,103][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000708216_181321728.pth [2023-12-26 20:31:31,603][105692] Updated weights for policy 0, policy_version 708589 (0.0010) [2023-12-26 20:31:31,671][105692] Updated weights for policy 0, policy_version 708599 (0.0009) [2023-12-26 20:31:31,732][105692] Updated weights for policy 0, policy_version 708609 (0.0010) [2023-12-26 20:31:31,891][105620] Updated weights for policy 1, policy_version 709372 (0.0009) [2023-12-26 20:31:31,945][105620] Updated weights for policy 1, policy_version 709382 (0.0007) [2023-12-26 20:31:31,994][105620] Updated weights for policy 1, policy_version 709392 (0.0005) [2023-12-26 20:31:32,522][105692] Updated weights for policy 0, policy_version 708619 (0.0008) [2023-12-26 20:31:32,577][105692] Updated weights for policy 0, policy_version 708629 (0.0008) [2023-12-26 20:31:32,621][105620] Updated weights for policy 1, policy_version 709402 (0.0006) [2023-12-26 20:31:32,627][105692] Updated weights for policy 0, policy_version 708639 (0.0010) [2023-12-26 20:31:32,677][105620] Updated weights for policy 1, policy_version 709412 (0.0010) [2023-12-26 20:31:32,736][105620] Updated weights for policy 1, policy_version 709422 (0.0010) [2023-12-26 20:31:32,795][105620] Updated weights for policy 1, policy_version 709432 (0.0009) [2023-12-26 20:31:33,387][105692] Updated weights for policy 0, policy_version 708649 (0.0010) [2023-12-26 20:31:33,445][105692] Updated weights for policy 0, policy_version 708659 (0.0011) [2023-12-26 20:31:33,503][105692] Updated weights for policy 0, policy_version 708669 (0.0010) [2023-12-26 20:31:33,544][105620] Updated weights for policy 1, policy_version 709442 (0.0010) [2023-12-26 20:31:33,554][105692] Updated weights for policy 0, policy_version 708679 (0.0010) [2023-12-26 20:31:33,605][105620] Updated weights for policy 1, policy_version 709452 (0.0010) [2023-12-26 20:31:33,669][105620] Updated weights for policy 1, policy_version 709462 (0.0010) [2023-12-26 20:31:34,282][105692] Updated weights for policy 0, policy_version 708689 (0.0011) [2023-12-26 20:31:34,347][105692] Updated weights for policy 0, policy_version 708699 (0.0010) [2023-12-26 20:31:34,377][105620] Updated weights for policy 1, policy_version 709472 (0.0010) [2023-12-26 20:31:34,409][105692] Updated weights for policy 0, policy_version 708709 (0.0011) [2023-12-26 20:31:34,440][105620] Updated weights for policy 1, policy_version 709482 (0.0010) [2023-12-26 20:31:34,493][105620] Updated weights for policy 1, policy_version 709492 (0.0008) [2023-12-26 20:31:35,138][105692] Updated weights for policy 0, policy_version 708719 (0.0010) [2023-12-26 20:31:35,168][105620] Updated weights for policy 1, policy_version 709502 (0.0009) [2023-12-26 20:31:35,183][105692] Updated weights for policy 0, policy_version 708729 (0.0010) [2023-12-26 20:31:35,227][105620] Updated weights for policy 1, policy_version 709512 (0.0010) [2023-12-26 20:31:35,234][105692] Updated weights for policy 0, policy_version 708739 (0.0010) [2023-12-26 20:31:35,289][105620] Updated weights for policy 1, policy_version 709522 (0.0010) [2023-12-26 20:31:35,911][105692] Updated weights for policy 0, policy_version 708749 (0.0008) [2023-12-26 20:31:35,980][105692] Updated weights for policy 0, policy_version 708759 (0.0007) [2023-12-26 20:31:36,018][105620] Updated weights for policy 1, policy_version 709532 (0.0010) [2023-12-26 20:31:36,044][105692] Updated weights for policy 0, policy_version 708769 (0.0007) [2023-12-26 20:31:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.3, 300 sec: 19660.8). Total num frames: 363126784. Throughput: 0: 9688.8, 1: 9838.9. Samples: 363123764. Policy #0 lag: (min: 11.0, avg: 30.6, max: 32.0) [2023-12-26 20:31:36,063][104569] Avg episode reward: [(0, '8843.307'), (1, '9085.124')] [2023-12-26 20:31:36,070][105620] Updated weights for policy 1, policy_version 709542 (0.0010) [2023-12-26 20:31:36,128][105620] Updated weights for policy 1, policy_version 709552 (0.0010) [2023-12-26 20:31:36,668][105692] Updated weights for policy 0, policy_version 708779 (0.0006) [2023-12-26 20:31:36,719][105692] Updated weights for policy 0, policy_version 708789 (0.0008) [2023-12-26 20:31:36,770][105692] Updated weights for policy 0, policy_version 708799 (0.0008) [2023-12-26 20:31:36,898][105620] Updated weights for policy 1, policy_version 709562 (0.0011) [2023-12-26 20:31:36,953][105620] Updated weights for policy 1, policy_version 709572 (0.0010) [2023-12-26 20:31:37,005][105620] Updated weights for policy 1, policy_version 709582 (0.0010) [2023-12-26 20:31:37,053][105620] Updated weights for policy 1, policy_version 709592 (0.0010) [2023-12-26 20:31:37,478][105692] Updated weights for policy 0, policy_version 708809 (0.0008) [2023-12-26 20:31:37,536][105692] Updated weights for policy 0, policy_version 708819 (0.0005) [2023-12-26 20:31:37,603][105692] Updated weights for policy 0, policy_version 708829 (0.0005) [2023-12-26 20:31:37,654][105692] Updated weights for policy 0, policy_version 708839 (0.0005) [2023-12-26 20:31:37,841][105620] Updated weights for policy 1, policy_version 709602 (0.0010) [2023-12-26 20:31:37,885][105620] Updated weights for policy 1, policy_version 709612 (0.0010) [2023-12-26 20:31:37,945][105620] Updated weights for policy 1, policy_version 709622 (0.0010) [2023-12-26 20:31:38,246][105692] Updated weights for policy 0, policy_version 708849 (0.0005) [2023-12-26 20:31:38,306][105692] Updated weights for policy 0, policy_version 708859 (0.0005) [2023-12-26 20:31:38,374][105692] Updated weights for policy 0, policy_version 708869 (0.0008) [2023-12-26 20:31:38,714][105620] Updated weights for policy 1, policy_version 709632 (0.0011) [2023-12-26 20:31:38,780][105620] Updated weights for policy 1, policy_version 709642 (0.0011) [2023-12-26 20:31:38,843][105620] Updated weights for policy 1, policy_version 709652 (0.0008) [2023-12-26 20:31:39,030][105692] Updated weights for policy 0, policy_version 708879 (0.0007) [2023-12-26 20:31:39,092][105692] Updated weights for policy 0, policy_version 708889 (0.0005) [2023-12-26 20:31:39,137][105692] Updated weights for policy 0, policy_version 708899 (0.0007) [2023-12-26 20:31:39,576][105620] Updated weights for policy 1, policy_version 709662 (0.0009) [2023-12-26 20:31:39,635][105620] Updated weights for policy 1, policy_version 709672 (0.0011) [2023-12-26 20:31:39,702][105620] Updated weights for policy 1, policy_version 709682 (0.0011) [2023-12-26 20:31:39,785][105692] Updated weights for policy 0, policy_version 708909 (0.0007) [2023-12-26 20:31:39,847][105692] Updated weights for policy 0, policy_version 708919 (0.0008) [2023-12-26 20:31:39,918][105692] Updated weights for policy 0, policy_version 708929 (0.0007) [2023-12-26 20:31:40,495][105620] Updated weights for policy 1, policy_version 709692 (0.0011) [2023-12-26 20:31:40,559][105620] Updated weights for policy 1, policy_version 709702 (0.0011) [2023-12-26 20:31:40,617][105620] Updated weights for policy 1, policy_version 709712 (0.0008) [2023-12-26 20:31:40,620][105692] Updated weights for policy 0, policy_version 708939 (0.0007) [2023-12-26 20:31:40,680][105692] Updated weights for policy 0, policy_version 708949 (0.0008) [2023-12-26 20:31:40,747][105692] Updated weights for policy 0, policy_version 708959 (0.0008) [2023-12-26 20:31:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19688.6). Total num frames: 363233280. Throughput: 0: 9843.4, 1: 9799.7. Samples: 363241192. Policy #0 lag: (min: 11.0, avg: 30.6, max: 32.0) [2023-12-26 20:31:41,063][104569] Avg episode reward: [(0, '8935.315'), (1, '8904.060')] [2023-12-26 20:31:41,403][105620] Updated weights for policy 1, policy_version 709722 (0.0010) [2023-12-26 20:31:41,442][105692] Updated weights for policy 0, policy_version 708969 (0.0008) [2023-12-26 20:31:41,469][105620] Updated weights for policy 1, policy_version 709732 (0.0008) [2023-12-26 20:31:41,506][105692] Updated weights for policy 0, policy_version 708979 (0.0009) [2023-12-26 20:31:41,529][105620] Updated weights for policy 1, policy_version 709742 (0.0005) [2023-12-26 20:31:41,563][105692] Updated weights for policy 0, policy_version 708989 (0.0009) [2023-12-26 20:31:41,587][105620] Updated weights for policy 1, policy_version 709752 (0.0006) [2023-12-26 20:31:41,632][105692] Updated weights for policy 0, policy_version 708999 (0.0009) [2023-12-26 20:31:42,233][105620] Updated weights for policy 1, policy_version 709762 (0.0005) [2023-12-26 20:31:42,309][105620] Updated weights for policy 1, policy_version 709772 (0.0010) [2023-12-26 20:31:42,374][105620] Updated weights for policy 1, policy_version 709782 (0.0011) [2023-12-26 20:31:42,437][105692] Updated weights for policy 0, policy_version 709009 (0.0008) [2023-12-26 20:31:42,494][105692] Updated weights for policy 0, policy_version 709019 (0.0008) [2023-12-26 20:31:42,559][105692] Updated weights for policy 0, policy_version 709029 (0.0008) [2023-12-26 20:31:42,963][105620] Updated weights for policy 1, policy_version 709792 (0.0006) [2023-12-26 20:31:43,030][105620] Updated weights for policy 1, policy_version 709802 (0.0006) [2023-12-26 20:31:43,092][105620] Updated weights for policy 1, policy_version 709812 (0.0005) [2023-12-26 20:31:43,253][105692] Updated weights for policy 0, policy_version 709039 (0.0006) [2023-12-26 20:31:43,317][105692] Updated weights for policy 0, policy_version 709049 (0.0008) [2023-12-26 20:31:43,379][105692] Updated weights for policy 0, policy_version 709059 (0.0008) [2023-12-26 20:31:43,713][105620] Updated weights for policy 1, policy_version 709822 (0.0008) [2023-12-26 20:31:43,768][105620] Updated weights for policy 1, policy_version 709832 (0.0010) [2023-12-26 20:31:43,831][105620] Updated weights for policy 1, policy_version 709842 (0.0010) [2023-12-26 20:31:44,049][105692] Updated weights for policy 0, policy_version 709069 (0.0009) [2023-12-26 20:31:44,107][105692] Updated weights for policy 0, policy_version 709079 (0.0011) [2023-12-26 20:31:44,166][105692] Updated weights for policy 0, policy_version 709089 (0.0010) [2023-12-26 20:31:44,575][105620] Updated weights for policy 1, policy_version 709852 (0.0009) [2023-12-26 20:31:44,631][105620] Updated weights for policy 1, policy_version 709862 (0.0007) [2023-12-26 20:31:44,689][105620] Updated weights for policy 1, policy_version 709872 (0.0010) [2023-12-26 20:31:44,865][105692] Updated weights for policy 0, policy_version 709099 (0.0010) [2023-12-26 20:31:44,928][105692] Updated weights for policy 0, policy_version 709109 (0.0011) [2023-12-26 20:31:44,988][105692] Updated weights for policy 0, policy_version 709119 (0.0011) [2023-12-26 20:31:45,362][105620] Updated weights for policy 1, policy_version 709882 (0.0010) [2023-12-26 20:31:45,422][105620] Updated weights for policy 1, policy_version 709892 (0.0011) [2023-12-26 20:31:45,474][105620] Updated weights for policy 1, policy_version 709902 (0.0010) [2023-12-26 20:31:45,526][105620] Updated weights for policy 1, policy_version 709912 (0.0010) [2023-12-26 20:31:45,763][105692] Updated weights for policy 0, policy_version 709129 (0.0011) [2023-12-26 20:31:45,817][105692] Updated weights for policy 0, policy_version 709139 (0.0010) [2023-12-26 20:31:45,875][105692] Updated weights for policy 0, policy_version 709149 (0.0010) [2023-12-26 20:31:45,933][105692] Updated weights for policy 0, policy_version 709159 (0.0011) [2023-12-26 20:31:46,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 363331584. Throughput: 0: 9852.9, 1: 9862.3. Samples: 363300612. Policy #0 lag: (min: 11.0, avg: 30.6, max: 32.0) [2023-12-26 20:31:46,063][104569] Avg episode reward: [(0, '9170.844'), (1, '8631.373')] [2023-12-26 20:31:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000709912_181755904.pth... [2023-12-26 20:31:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000709160_181575680.pth... [2023-12-26 20:31:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000707976_181272576.pth [2023-12-26 20:31:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000708760_181460992.pth [2023-12-26 20:31:46,202][105620] Updated weights for policy 1, policy_version 709922 (0.0006) [2023-12-26 20:31:46,254][105620] Updated weights for policy 1, policy_version 709932 (0.0008) [2023-12-26 20:31:46,307][105620] Updated weights for policy 1, policy_version 709942 (0.0005) [2023-12-26 20:31:46,634][105692] Updated weights for policy 0, policy_version 709169 (0.0008) [2023-12-26 20:31:46,682][105692] Updated weights for policy 0, policy_version 709179 (0.0008) [2023-12-26 20:31:46,739][105692] Updated weights for policy 0, policy_version 709189 (0.0009) [2023-12-26 20:31:46,988][105620] Updated weights for policy 1, policy_version 709952 (0.0008) [2023-12-26 20:31:47,040][105620] Updated weights for policy 1, policy_version 709962 (0.0010) [2023-12-26 20:31:47,102][105620] Updated weights for policy 1, policy_version 709972 (0.0010) [2023-12-26 20:31:47,344][105692] Updated weights for policy 0, policy_version 709199 (0.0007) [2023-12-26 20:31:47,405][105692] Updated weights for policy 0, policy_version 709209 (0.0006) [2023-12-26 20:31:47,463][105692] Updated weights for policy 0, policy_version 709219 (0.0007) [2023-12-26 20:31:47,831][105620] Updated weights for policy 1, policy_version 709982 (0.0010) [2023-12-26 20:31:47,893][105620] Updated weights for policy 1, policy_version 709992 (0.0011) [2023-12-26 20:31:47,943][105620] Updated weights for policy 1, policy_version 710002 (0.0010) [2023-12-26 20:31:47,968][105586] KL-divergence is very high: 215.7064 [2023-12-26 20:31:48,160][105692] Updated weights for policy 0, policy_version 709229 (0.0005) [2023-12-26 20:31:48,216][105692] Updated weights for policy 0, policy_version 709239 (0.0005) [2023-12-26 20:31:48,273][105692] Updated weights for policy 0, policy_version 709249 (0.0005) [2023-12-26 20:31:48,618][105620] Updated weights for policy 1, policy_version 710012 (0.0008) [2023-12-26 20:31:48,667][105620] Updated weights for policy 1, policy_version 710022 (0.0010) [2023-12-26 20:31:48,723][105620] Updated weights for policy 1, policy_version 710032 (0.0006) [2023-12-26 20:31:48,993][105692] Updated weights for policy 0, policy_version 709259 (0.0006) [2023-12-26 20:31:49,041][105692] Updated weights for policy 0, policy_version 709269 (0.0006) [2023-12-26 20:31:49,104][105692] Updated weights for policy 0, policy_version 709279 (0.0006) [2023-12-26 20:31:49,376][105620] Updated weights for policy 1, policy_version 710042 (0.0006) [2023-12-26 20:31:49,445][105620] Updated weights for policy 1, policy_version 710052 (0.0010) [2023-12-26 20:31:49,511][105620] Updated weights for policy 1, policy_version 710062 (0.0010) [2023-12-26 20:31:49,575][105620] Updated weights for policy 1, policy_version 710072 (0.0008) [2023-12-26 20:31:49,802][105692] Updated weights for policy 0, policy_version 709289 (0.0006) [2023-12-26 20:31:49,869][105692] Updated weights for policy 0, policy_version 709299 (0.0008) [2023-12-26 20:31:49,938][105692] Updated weights for policy 0, policy_version 709309 (0.0009) [2023-12-26 20:31:49,998][105692] Updated weights for policy 0, policy_version 709319 (0.0006) [2023-12-26 20:31:50,321][105620] Updated weights for policy 1, policy_version 710082 (0.0010) [2023-12-26 20:31:50,376][105620] Updated weights for policy 1, policy_version 710092 (0.0010) [2023-12-26 20:31:50,434][105620] Updated weights for policy 1, policy_version 710102 (0.0006) [2023-12-26 20:31:50,615][105692] Updated weights for policy 0, policy_version 709329 (0.0008) [2023-12-26 20:31:50,683][105692] Updated weights for policy 0, policy_version 709339 (0.0008) [2023-12-26 20:31:50,740][105692] Updated weights for policy 0, policy_version 709349 (0.0008) [2023-12-26 20:31:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 363429888. Throughput: 0: 9916.8, 1: 9852.9. Samples: 363420524. Policy #0 lag: (min: 11.0, avg: 30.6, max: 32.0) [2023-12-26 20:31:51,063][104569] Avg episode reward: [(0, '9260.090'), (1, '8995.994')] [2023-12-26 20:31:51,164][105620] Updated weights for policy 1, policy_version 710112 (0.0007) [2023-12-26 20:31:51,231][105620] Updated weights for policy 1, policy_version 710122 (0.0008) [2023-12-26 20:31:51,287][105620] Updated weights for policy 1, policy_version 710132 (0.0010) [2023-12-26 20:31:51,441][105692] Updated weights for policy 0, policy_version 709359 (0.0009) [2023-12-26 20:31:51,494][105692] Updated weights for policy 0, policy_version 709369 (0.0009) [2023-12-26 20:31:51,548][105692] Updated weights for policy 0, policy_version 709379 (0.0011) [2023-12-26 20:31:51,971][105620] Updated weights for policy 1, policy_version 710142 (0.0010) [2023-12-26 20:31:52,030][105620] Updated weights for policy 1, policy_version 710152 (0.0011) [2023-12-26 20:31:52,086][105620] Updated weights for policy 1, policy_version 710162 (0.0011) [2023-12-26 20:31:52,319][105692] Updated weights for policy 0, policy_version 709389 (0.0008) [2023-12-26 20:31:52,387][105692] Updated weights for policy 0, policy_version 709399 (0.0009) [2023-12-26 20:31:52,440][105692] Updated weights for policy 0, policy_version 709409 (0.0011) [2023-12-26 20:31:52,720][105620] Updated weights for policy 1, policy_version 710172 (0.0011) [2023-12-26 20:31:52,769][105620] Updated weights for policy 1, policy_version 710182 (0.0010) [2023-12-26 20:31:52,827][105620] Updated weights for policy 1, policy_version 710192 (0.0010) [2023-12-26 20:31:53,105][105692] Updated weights for policy 0, policy_version 709419 (0.0006) [2023-12-26 20:31:53,165][105692] Updated weights for policy 0, policy_version 709429 (0.0005) [2023-12-26 20:31:53,234][105692] Updated weights for policy 0, policy_version 709439 (0.0005) [2023-12-26 20:31:53,640][105620] Updated weights for policy 1, policy_version 710202 (0.0008) [2023-12-26 20:31:53,700][105620] Updated weights for policy 1, policy_version 710212 (0.0005) [2023-12-26 20:31:53,761][105620] Updated weights for policy 1, policy_version 710222 (0.0006) [2023-12-26 20:31:53,808][105692] Updated weights for policy 0, policy_version 709449 (0.0006) [2023-12-26 20:31:53,813][105620] Updated weights for policy 1, policy_version 710232 (0.0008) [2023-12-26 20:31:53,864][105692] Updated weights for policy 0, policy_version 709459 (0.0009) [2023-12-26 20:31:53,929][105692] Updated weights for policy 0, policy_version 709469 (0.0006) [2023-12-26 20:31:53,990][105692] Updated weights for policy 0, policy_version 709479 (0.0006) [2023-12-26 20:31:54,416][105620] Updated weights for policy 1, policy_version 710242 (0.0008) [2023-12-26 20:31:54,465][105620] Updated weights for policy 1, policy_version 710252 (0.0009) [2023-12-26 20:31:54,519][105620] Updated weights for policy 1, policy_version 710262 (0.0009) [2023-12-26 20:31:54,616][105692] Updated weights for policy 0, policy_version 709489 (0.0009) [2023-12-26 20:31:54,678][105692] Updated weights for policy 0, policy_version 709499 (0.0009) [2023-12-26 20:31:54,735][105692] Updated weights for policy 0, policy_version 709509 (0.0009) [2023-12-26 20:31:55,206][105620] Updated weights for policy 1, policy_version 710272 (0.0006) [2023-12-26 20:31:55,262][105620] Updated weights for policy 1, policy_version 710282 (0.0006) [2023-12-26 20:31:55,320][105620] Updated weights for policy 1, policy_version 710292 (0.0006) [2023-12-26 20:31:55,410][105692] Updated weights for policy 0, policy_version 709519 (0.0009) [2023-12-26 20:31:55,473][105692] Updated weights for policy 0, policy_version 709529 (0.0008) [2023-12-26 20:31:55,534][105692] Updated weights for policy 0, policy_version 709539 (0.0009) [2023-12-26 20:31:55,991][105620] Updated weights for policy 1, policy_version 710302 (0.0008) [2023-12-26 20:31:56,043][105620] Updated weights for policy 1, policy_version 710312 (0.0010) [2023-12-26 20:31:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.9, 300 sec: 19688.6). Total num frames: 363528192. Throughput: 0: 9952.6, 1: 9884.0. Samples: 363541344. Policy #0 lag: (min: 11.0, avg: 30.6, max: 32.0) [2023-12-26 20:31:56,062][104569] Avg episode reward: [(0, '9081.709'), (1, '9179.021')] [2023-12-26 20:31:56,090][105620] Updated weights for policy 1, policy_version 710322 (0.0009) [2023-12-26 20:31:56,276][105692] Updated weights for policy 0, policy_version 709549 (0.0008) [2023-12-26 20:31:56,328][105692] Updated weights for policy 0, policy_version 709559 (0.0006) [2023-12-26 20:31:56,378][105692] Updated weights for policy 0, policy_version 709569 (0.0006) [2023-12-26 20:31:56,901][105620] Updated weights for policy 1, policy_version 710332 (0.0008) [2023-12-26 20:31:56,961][105620] Updated weights for policy 1, policy_version 710342 (0.0007) [2023-12-26 20:31:57,011][105620] Updated weights for policy 1, policy_version 710352 (0.0009) [2023-12-26 20:31:57,079][105692] Updated weights for policy 0, policy_version 709579 (0.0009) [2023-12-26 20:31:57,129][105692] Updated weights for policy 0, policy_version 709589 (0.0009) [2023-12-26 20:31:57,183][105692] Updated weights for policy 0, policy_version 709599 (0.0009) [2023-12-26 20:31:57,634][105620] Updated weights for policy 1, policy_version 710362 (0.0009) [2023-12-26 20:31:57,687][105620] Updated weights for policy 1, policy_version 710373 (0.0010) [2023-12-26 20:31:57,742][105620] Updated weights for policy 1, policy_version 710384 (0.0007) [2023-12-26 20:31:57,994][105692] Updated weights for policy 0, policy_version 709609 (0.0008) [2023-12-26 20:31:58,044][105692] Updated weights for policy 0, policy_version 709619 (0.0008) [2023-12-26 20:31:58,093][105692] Updated weights for policy 0, policy_version 709629 (0.0009) [2023-12-26 20:31:58,153][105692] Updated weights for policy 0, policy_version 709639 (0.0009) [2023-12-26 20:31:58,393][105620] Updated weights for policy 1, policy_version 710394 (0.0006) [2023-12-26 20:31:58,458][105620] Updated weights for policy 1, policy_version 710404 (0.0008) [2023-12-26 20:31:58,521][105620] Updated weights for policy 1, policy_version 710414 (0.0007) [2023-12-26 20:31:58,584][105620] Updated weights for policy 1, policy_version 710424 (0.0006) [2023-12-26 20:31:58,960][105692] Updated weights for policy 0, policy_version 709649 (0.0009) [2023-12-26 20:31:59,021][105692] Updated weights for policy 0, policy_version 709659 (0.0009) [2023-12-26 20:31:59,072][105692] Updated weights for policy 0, policy_version 709669 (0.0010) [2023-12-26 20:31:59,203][105620] Updated weights for policy 1, policy_version 710434 (0.0009) [2023-12-26 20:31:59,265][105620] Updated weights for policy 1, policy_version 710444 (0.0009) [2023-12-26 20:31:59,320][105620] Updated weights for policy 1, policy_version 710454 (0.0009) [2023-12-26 20:31:59,900][105692] Updated weights for policy 0, policy_version 709679 (0.0009) [2023-12-26 20:31:59,965][105692] Updated weights for policy 0, policy_version 709689 (0.0008) [2023-12-26 20:32:00,015][105620] Updated weights for policy 1, policy_version 710464 (0.0008) [2023-12-26 20:32:00,022][105692] Updated weights for policy 0, policy_version 709699 (0.0007) [2023-12-26 20:32:00,062][105620] Updated weights for policy 1, policy_version 710474 (0.0008) [2023-12-26 20:32:00,116][105620] Updated weights for policy 1, policy_version 710484 (0.0008) [2023-12-26 20:32:00,685][105692] Updated weights for policy 0, policy_version 709709 (0.0008) [2023-12-26 20:32:00,729][105692] Updated weights for policy 0, policy_version 709719 (0.0005) [2023-12-26 20:32:00,775][105692] Updated weights for policy 0, policy_version 709729 (0.0005) [2023-12-26 20:32:00,857][105620] Updated weights for policy 1, policy_version 710494 (0.0006) [2023-12-26 20:32:00,901][105620] Updated weights for policy 1, policy_version 710504 (0.0005) [2023-12-26 20:32:00,961][105620] Updated weights for policy 1, policy_version 710514 (0.0005) [2023-12-26 20:32:01,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19933.9, 300 sec: 19716.3). Total num frames: 363634688. Throughput: 0: 9979.0, 1: 9910.9. Samples: 363599588. Policy #0 lag: (min: 11.0, avg: 30.6, max: 32.0) [2023-12-26 20:32:01,063][104569] Avg episode reward: [(0, '8727.812'), (1, '9266.496')] [2023-12-26 20:32:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000709736_181723136.pth... [2023-12-26 20:32:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000710520_181911552.pth... [2023-12-26 20:32:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000708584_181428224.pth [2023-12-26 20:32:01,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000709368_181616640.pth [2023-12-26 20:32:01,519][105692] Updated weights for policy 0, policy_version 709739 (0.0006) [2023-12-26 20:32:01,575][105692] Updated weights for policy 0, policy_version 709749 (0.0008) [2023-12-26 20:32:01,626][105620] Updated weights for policy 1, policy_version 710524 (0.0007) [2023-12-26 20:32:01,643][105692] Updated weights for policy 0, policy_version 709759 (0.0007) [2023-12-26 20:32:01,685][105620] Updated weights for policy 1, policy_version 710534 (0.0009) [2023-12-26 20:32:01,748][105620] Updated weights for policy 1, policy_version 710544 (0.0010) [2023-12-26 20:32:02,404][105620] Updated weights for policy 1, policy_version 710554 (0.0008) [2023-12-26 20:32:02,449][105692] Updated weights for policy 0, policy_version 709769 (0.0005) [2023-12-26 20:32:02,459][105620] Updated weights for policy 1, policy_version 710564 (0.0010) [2023-12-26 20:32:02,501][105692] Updated weights for policy 0, policy_version 709779 (0.0005) [2023-12-26 20:32:02,519][105620] Updated weights for policy 1, policy_version 710574 (0.0010) [2023-12-26 20:32:02,561][105692] Updated weights for policy 0, policy_version 709789 (0.0005) [2023-12-26 20:32:02,578][105620] Updated weights for policy 1, policy_version 710584 (0.0010) [2023-12-26 20:32:02,626][105692] Updated weights for policy 0, policy_version 709799 (0.0009) [2023-12-26 20:32:03,226][105692] Updated weights for policy 0, policy_version 709809 (0.0005) [2023-12-26 20:32:03,269][105692] Updated weights for policy 0, policy_version 709819 (0.0006) [2023-12-26 20:32:03,326][105620] Updated weights for policy 1, policy_version 710594 (0.0011) [2023-12-26 20:32:03,329][105692] Updated weights for policy 0, policy_version 709829 (0.0011) [2023-12-26 20:32:03,392][105620] Updated weights for policy 1, policy_version 710604 (0.0011) [2023-12-26 20:32:03,457][105620] Updated weights for policy 1, policy_version 710614 (0.0010) [2023-12-26 20:32:03,963][105692] Updated weights for policy 0, policy_version 709839 (0.0011) [2023-12-26 20:32:04,014][105692] Updated weights for policy 0, policy_version 709849 (0.0011) [2023-12-26 20:32:04,072][105620] Updated weights for policy 1, policy_version 710624 (0.0006) [2023-12-26 20:32:04,078][105692] Updated weights for policy 0, policy_version 709859 (0.0011) [2023-12-26 20:32:04,131][105620] Updated weights for policy 1, policy_version 710634 (0.0005) [2023-12-26 20:32:04,191][105620] Updated weights for policy 1, policy_version 710644 (0.0006) [2023-12-26 20:32:04,771][105692] Updated weights for policy 0, policy_version 709869 (0.0008) [2023-12-26 20:32:04,839][105692] Updated weights for policy 0, policy_version 709879 (0.0007) [2023-12-26 20:32:04,867][105620] Updated weights for policy 1, policy_version 710654 (0.0009) [2023-12-26 20:32:04,891][105692] Updated weights for policy 0, policy_version 709889 (0.0008) [2023-12-26 20:32:04,921][105620] Updated weights for policy 1, policy_version 710664 (0.0010) [2023-12-26 20:32:04,971][105620] Updated weights for policy 1, policy_version 710674 (0.0011) [2023-12-26 20:32:05,561][105692] Updated weights for policy 0, policy_version 709899 (0.0007) [2023-12-26 20:32:05,610][105692] Updated weights for policy 0, policy_version 709909 (0.0009) [2023-12-26 20:32:05,664][105692] Updated weights for policy 0, policy_version 709919 (0.0009) [2023-12-26 20:32:05,699][105620] Updated weights for policy 1, policy_version 710684 (0.0010) [2023-12-26 20:32:05,750][105620] Updated weights for policy 1, policy_version 710694 (0.0007) [2023-12-26 20:32:05,801][105620] Updated weights for policy 1, policy_version 710704 (0.0009) [2023-12-26 20:32:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19933.9, 300 sec: 19688.6). Total num frames: 363732992. Throughput: 0: 9959.6, 1: 9926.7. Samples: 363718468. Policy #0 lag: (min: 11.0, avg: 30.6, max: 32.0) [2023-12-26 20:32:06,062][104569] Avg episode reward: [(0, '6290.473'), (1, '8905.007')] [2023-12-26 20:32:06,447][105692] Updated weights for policy 0, policy_version 709929 (0.0007) [2023-12-26 20:32:06,509][105692] Updated weights for policy 0, policy_version 709939 (0.0005) [2023-12-26 20:32:06,572][105692] Updated weights for policy 0, policy_version 709949 (0.0005) [2023-12-26 20:32:06,622][105620] Updated weights for policy 1, policy_version 710714 (0.0009) [2023-12-26 20:32:06,632][105692] Updated weights for policy 0, policy_version 709959 (0.0005) [2023-12-26 20:32:06,679][105620] Updated weights for policy 1, policy_version 710724 (0.0010) [2023-12-26 20:32:06,735][105620] Updated weights for policy 1, policy_version 710734 (0.0009) [2023-12-26 20:32:06,769][105586] KL-divergence is very high: 124.1873 [2023-12-26 20:32:06,801][105620] Updated weights for policy 1, policy_version 710744 (0.0010) [2023-12-26 20:32:07,165][105692] Updated weights for policy 0, policy_version 709969 (0.0008) [2023-12-26 20:32:07,221][105692] Updated weights for policy 0, policy_version 709979 (0.0009) [2023-12-26 20:32:07,279][105692] Updated weights for policy 0, policy_version 709989 (0.0009) [2023-12-26 20:32:07,659][105620] Updated weights for policy 1, policy_version 710754 (0.0009) [2023-12-26 20:32:07,713][105620] Updated weights for policy 1, policy_version 710764 (0.0009) [2023-12-26 20:32:07,766][105620] Updated weights for policy 1, policy_version 710774 (0.0010) [2023-12-26 20:32:07,943][105692] Updated weights for policy 0, policy_version 709999 (0.0009) [2023-12-26 20:32:08,001][105692] Updated weights for policy 0, policy_version 710010 (0.0009) [2023-12-26 20:32:08,058][105692] Updated weights for policy 0, policy_version 710020 (0.0010) [2023-12-26 20:32:08,498][105620] Updated weights for policy 1, policy_version 710784 (0.0009) [2023-12-26 20:32:08,549][105620] Updated weights for policy 1, policy_version 710794 (0.0008) [2023-12-26 20:32:08,602][105620] Updated weights for policy 1, policy_version 710804 (0.0008) [2023-12-26 20:32:08,840][105692] Updated weights for policy 0, policy_version 710030 (0.0010) [2023-12-26 20:32:08,904][105692] Updated weights for policy 0, policy_version 710040 (0.0009) [2023-12-26 20:32:08,962][105692] Updated weights for policy 0, policy_version 710050 (0.0010) [2023-12-26 20:32:09,385][105620] Updated weights for policy 1, policy_version 710814 (0.0009) [2023-12-26 20:32:09,448][105620] Updated weights for policy 1, policy_version 710824 (0.0008) [2023-12-26 20:32:09,509][105620] Updated weights for policy 1, policy_version 710834 (0.0008) [2023-12-26 20:32:09,723][105692] Updated weights for policy 0, policy_version 710060 (0.0010) [2023-12-26 20:32:09,772][105692] Updated weights for policy 0, policy_version 710070 (0.0010) [2023-12-26 20:32:09,826][105692] Updated weights for policy 0, policy_version 710080 (0.0010) [2023-12-26 20:32:10,292][105620] Updated weights for policy 1, policy_version 710844 (0.0008) [2023-12-26 20:32:10,357][105620] Updated weights for policy 1, policy_version 710854 (0.0008) [2023-12-26 20:32:10,412][105620] Updated weights for policy 1, policy_version 710864 (0.0008) [2023-12-26 20:32:10,595][105692] Updated weights for policy 0, policy_version 710090 (0.0011) [2023-12-26 20:32:10,644][105692] Updated weights for policy 0, policy_version 710100 (0.0010) [2023-12-26 20:32:10,692][105692] Updated weights for policy 0, policy_version 710110 (0.0010) [2023-12-26 20:32:10,744][105692] Updated weights for policy 0, policy_version 710120 (0.0010) [2023-12-26 20:32:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 363823104. Throughput: 0: 9943.9, 1: 9790.1. Samples: 363831608. Policy #0 lag: (min: 11.0, avg: 30.6, max: 32.0) [2023-12-26 20:32:11,062][104569] Avg episode reward: [(0, '4098.120'), (1, '8906.521')] [2023-12-26 20:32:11,215][105620] Updated weights for policy 1, policy_version 710874 (0.0009) [2023-12-26 20:32:11,281][105620] Updated weights for policy 1, policy_version 710884 (0.0008) [2023-12-26 20:32:11,334][105620] Updated weights for policy 1, policy_version 710894 (0.0008) [2023-12-26 20:32:11,398][105620] Updated weights for policy 1, policy_version 710904 (0.0009) [2023-12-26 20:32:11,537][105692] Updated weights for policy 0, policy_version 710130 (0.0011) [2023-12-26 20:32:11,588][105692] Updated weights for policy 0, policy_version 710140 (0.0010) [2023-12-26 20:32:11,658][105692] Updated weights for policy 0, policy_version 710151 (0.0010) [2023-12-26 20:32:12,178][105620] Updated weights for policy 1, policy_version 710914 (0.0005) [2023-12-26 20:32:12,232][105620] Updated weights for policy 1, policy_version 710924 (0.0005) [2023-12-26 20:32:12,295][105620] Updated weights for policy 1, policy_version 710934 (0.0008) [2023-12-26 20:32:12,489][105692] Updated weights for policy 0, policy_version 710161 (0.0008) [2023-12-26 20:32:12,546][105692] Updated weights for policy 0, policy_version 710171 (0.0008) [2023-12-26 20:32:12,607][105692] Updated weights for policy 0, policy_version 710181 (0.0009) [2023-12-26 20:32:13,032][105620] Updated weights for policy 1, policy_version 710944 (0.0008) [2023-12-26 20:32:13,083][105620] Updated weights for policy 1, policy_version 710954 (0.0008) [2023-12-26 20:32:13,143][105620] Updated weights for policy 1, policy_version 710964 (0.0008) [2023-12-26 20:32:13,294][105692] Updated weights for policy 0, policy_version 710191 (0.0007) [2023-12-26 20:32:13,338][105692] Updated weights for policy 0, policy_version 710201 (0.0006) [2023-12-26 20:32:13,384][105692] Updated weights for policy 0, policy_version 710211 (0.0005) [2023-12-26 20:32:13,843][105620] Updated weights for policy 1, policy_version 710974 (0.0005) [2023-12-26 20:32:13,909][105620] Updated weights for policy 1, policy_version 710984 (0.0006) [2023-12-26 20:32:13,919][105692] Updated weights for policy 0, policy_version 710221 (0.0005) [2023-12-26 20:32:13,963][105620] Updated weights for policy 1, policy_version 710994 (0.0005) [2023-12-26 20:32:13,981][105692] Updated weights for policy 0, policy_version 710231 (0.0005) [2023-12-26 20:32:14,045][105692] Updated weights for policy 0, policy_version 710241 (0.0005) [2023-12-26 20:32:14,638][105620] Updated weights for policy 1, policy_version 711004 (0.0010) [2023-12-26 20:32:14,700][105620] Updated weights for policy 1, policy_version 711014 (0.0010) [2023-12-26 20:32:14,701][105692] Updated weights for policy 0, policy_version 710251 (0.0005) [2023-12-26 20:32:14,759][105692] Updated weights for policy 0, policy_version 710261 (0.0006) [2023-12-26 20:32:14,765][105620] Updated weights for policy 1, policy_version 711024 (0.0010) [2023-12-26 20:32:14,823][105692] Updated weights for policy 0, policy_version 710271 (0.0011) [2023-12-26 20:32:15,506][105620] Updated weights for policy 1, policy_version 711034 (0.0010) [2023-12-26 20:32:15,539][105692] Updated weights for policy 0, policy_version 710281 (0.0011) [2023-12-26 20:32:15,571][105620] Updated weights for policy 1, policy_version 711044 (0.0008) [2023-12-26 20:32:15,603][105692] Updated weights for policy 0, policy_version 710291 (0.0011) [2023-12-26 20:32:15,629][105620] Updated weights for policy 1, policy_version 711054 (0.0005) [2023-12-26 20:32:15,658][105692] Updated weights for policy 0, policy_version 710301 (0.0011) [2023-12-26 20:32:15,690][105620] Updated weights for policy 1, policy_version 711064 (0.0005) [2023-12-26 20:32:15,713][105692] Updated weights for policy 0, policy_version 710311 (0.0010) [2023-12-26 20:32:16,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 363921408. Throughput: 0: 9821.9, 1: 9736.4. Samples: 363888716. Policy #0 lag: (min: 14.0, avg: 21.6, max: 46.0) [2023-12-26 20:32:16,063][104569] Avg episode reward: [(0, '7414.548'), (1, '9266.310')] [2023-12-26 20:32:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000710312_181870592.pth... [2023-12-26 20:32:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000711064_182050816.pth... [2023-12-26 20:32:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000709912_181755904.pth [2023-12-26 20:32:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000709160_181575680.pth [2023-12-26 20:32:16,328][105620] Updated weights for policy 1, policy_version 711074 (0.0010) [2023-12-26 20:32:16,337][105692] Updated weights for policy 0, policy_version 710321 (0.0010) [2023-12-26 20:32:16,387][105620] Updated weights for policy 1, policy_version 711084 (0.0006) [2023-12-26 20:32:16,396][105692] Updated weights for policy 0, policy_version 710331 (0.0010) [2023-12-26 20:32:16,444][105692] Updated weights for policy 0, policy_version 710341 (0.0010) [2023-12-26 20:32:16,445][105620] Updated weights for policy 1, policy_version 711094 (0.0006) [2023-12-26 20:32:17,051][105620] Updated weights for policy 1, policy_version 711104 (0.0006) [2023-12-26 20:32:17,094][105692] Updated weights for policy 0, policy_version 710351 (0.0007) [2023-12-26 20:32:17,103][105620] Updated weights for policy 1, policy_version 711114 (0.0008) [2023-12-26 20:32:17,147][105692] Updated weights for policy 0, policy_version 710361 (0.0005) [2023-12-26 20:32:17,155][105620] Updated weights for policy 1, policy_version 711124 (0.0006) [2023-12-26 20:32:17,207][105692] Updated weights for policy 0, policy_version 710371 (0.0008) [2023-12-26 20:32:17,789][105620] Updated weights for policy 1, policy_version 711134 (0.0007) [2023-12-26 20:32:17,822][105692] Updated weights for policy 0, policy_version 710381 (0.0008) [2023-12-26 20:32:17,846][105620] Updated weights for policy 1, policy_version 711144 (0.0009) [2023-12-26 20:32:17,889][105692] Updated weights for policy 0, policy_version 710391 (0.0005) [2023-12-26 20:32:17,902][105620] Updated weights for policy 1, policy_version 711154 (0.0009) [2023-12-26 20:32:17,957][105692] Updated weights for policy 0, policy_version 710401 (0.0005) [2023-12-26 20:32:18,544][105692] Updated weights for policy 0, policy_version 710411 (0.0007) [2023-12-26 20:32:18,605][105692] Updated weights for policy 0, policy_version 710421 (0.0009) [2023-12-26 20:32:18,673][105692] Updated weights for policy 0, policy_version 710431 (0.0005) [2023-12-26 20:32:18,747][105620] Updated weights for policy 1, policy_version 711164 (0.0008) [2023-12-26 20:32:18,804][105620] Updated weights for policy 1, policy_version 711174 (0.0008) [2023-12-26 20:32:18,868][105620] Updated weights for policy 1, policy_version 711184 (0.0008) [2023-12-26 20:32:19,419][105692] Updated weights for policy 0, policy_version 710441 (0.0008) [2023-12-26 20:32:19,490][105692] Updated weights for policy 0, policy_version 710451 (0.0009) [2023-12-26 20:32:19,555][105692] Updated weights for policy 0, policy_version 710461 (0.0008) [2023-12-26 20:32:19,558][105620] Updated weights for policy 1, policy_version 711194 (0.0008) [2023-12-26 20:32:19,613][105692] Updated weights for policy 0, policy_version 710471 (0.0006) [2023-12-26 20:32:19,622][105620] Updated weights for policy 1, policy_version 711204 (0.0011) [2023-12-26 20:32:19,674][105620] Updated weights for policy 1, policy_version 711214 (0.0010) [2023-12-26 20:32:19,726][105620] Updated weights for policy 1, policy_version 711224 (0.0010) [2023-12-26 20:32:20,303][105692] Updated weights for policy 0, policy_version 710481 (0.0006) [2023-12-26 20:32:20,362][105692] Updated weights for policy 0, policy_version 710491 (0.0008) [2023-12-26 20:32:20,424][105692] Updated weights for policy 0, policy_version 710501 (0.0011) [2023-12-26 20:32:20,453][105620] Updated weights for policy 1, policy_version 711234 (0.0007) [2023-12-26 20:32:20,505][105620] Updated weights for policy 1, policy_version 711244 (0.0010) [2023-12-26 20:32:20,557][105620] Updated weights for policy 1, policy_version 711254 (0.0010) [2023-12-26 20:32:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 364019712. Throughput: 0: 9954.5, 1: 9759.2. Samples: 364010880. Policy #0 lag: (min: 14.0, avg: 21.6, max: 46.0) [2023-12-26 20:32:21,062][104569] Avg episode reward: [(0, '9167.867'), (1, '9174.455')] [2023-12-26 20:32:21,174][105692] Updated weights for policy 0, policy_version 710511 (0.0011) [2023-12-26 20:32:21,239][105692] Updated weights for policy 0, policy_version 710521 (0.0010) [2023-12-26 20:32:21,304][105692] Updated weights for policy 0, policy_version 710531 (0.0011) [2023-12-26 20:32:21,347][105620] Updated weights for policy 1, policy_version 711264 (0.0008) [2023-12-26 20:32:21,416][105620] Updated weights for policy 1, policy_version 711274 (0.0008) [2023-12-26 20:32:21,484][105620] Updated weights for policy 1, policy_version 711284 (0.0008) [2023-12-26 20:32:22,001][105692] Updated weights for policy 0, policy_version 710541 (0.0009) [2023-12-26 20:32:22,059][105692] Updated weights for policy 0, policy_version 710551 (0.0009) [2023-12-26 20:32:22,122][105692] Updated weights for policy 0, policy_version 710561 (0.0011) [2023-12-26 20:32:22,317][105620] Updated weights for policy 1, policy_version 711294 (0.0008) [2023-12-26 20:32:22,382][105620] Updated weights for policy 1, policy_version 711304 (0.0008) [2023-12-26 20:32:22,443][105620] Updated weights for policy 1, policy_version 711314 (0.0008) [2023-12-26 20:32:22,877][105692] Updated weights for policy 0, policy_version 710571 (0.0009) [2023-12-26 20:32:22,931][105692] Updated weights for policy 0, policy_version 710581 (0.0005) [2023-12-26 20:32:22,994][105692] Updated weights for policy 0, policy_version 710591 (0.0005) [2023-12-26 20:32:23,107][105620] Updated weights for policy 1, policy_version 711324 (0.0009) [2023-12-26 20:32:23,169][105620] Updated weights for policy 1, policy_version 711334 (0.0011) [2023-12-26 20:32:23,235][105620] Updated weights for policy 1, policy_version 711344 (0.0011) [2023-12-26 20:32:23,608][105692] Updated weights for policy 0, policy_version 710601 (0.0006) [2023-12-26 20:32:23,659][105692] Updated weights for policy 0, policy_version 710611 (0.0010) [2023-12-26 20:32:23,722][105692] Updated weights for policy 0, policy_version 710621 (0.0010) [2023-12-26 20:32:23,786][105692] Updated weights for policy 0, policy_version 710631 (0.0006) [2023-12-26 20:32:23,933][105620] Updated weights for policy 1, policy_version 711354 (0.0010) [2023-12-26 20:32:24,004][105620] Updated weights for policy 1, policy_version 711364 (0.0009) [2023-12-26 20:32:24,076][105620] Updated weights for policy 1, policy_version 711374 (0.0008) [2023-12-26 20:32:24,150][105620] Updated weights for policy 1, policy_version 711384 (0.0010) [2023-12-26 20:32:24,377][105692] Updated weights for policy 0, policy_version 710641 (0.0006) [2023-12-26 20:32:24,430][105692] Updated weights for policy 0, policy_version 710651 (0.0005) [2023-12-26 20:32:24,486][105692] Updated weights for policy 0, policy_version 710661 (0.0005) [2023-12-26 20:32:24,918][105620] Updated weights for policy 1, policy_version 711394 (0.0008) [2023-12-26 20:32:24,978][105620] Updated weights for policy 1, policy_version 711404 (0.0009) [2023-12-26 20:32:25,031][105620] Updated weights for policy 1, policy_version 711414 (0.0009) [2023-12-26 20:32:25,100][105692] Updated weights for policy 0, policy_version 710671 (0.0009) [2023-12-26 20:32:25,157][105692] Updated weights for policy 0, policy_version 710681 (0.0007) [2023-12-26 20:32:25,208][105692] Updated weights for policy 0, policy_version 710691 (0.0005) [2023-12-26 20:32:25,802][105620] Updated weights for policy 1, policy_version 711424 (0.0009) [2023-12-26 20:32:25,841][105692] Updated weights for policy 0, policy_version 710701 (0.0008) [2023-12-26 20:32:25,852][105620] Updated weights for policy 1, policy_version 711434 (0.0007) [2023-12-26 20:32:25,901][105620] Updated weights for policy 1, policy_version 711444 (0.0009) [2023-12-26 20:32:25,901][105692] Updated weights for policy 0, policy_version 710711 (0.0005) [2023-12-26 20:32:25,959][105692] Updated weights for policy 0, policy_version 710721 (0.0005) [2023-12-26 20:32:26,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19933.8, 300 sec: 19633.0). Total num frames: 364126208. Throughput: 0: 9972.2, 1: 9761.8. Samples: 364129224. Policy #0 lag: (min: 14.0, avg: 21.6, max: 46.0) [2023-12-26 20:32:26,063][104569] Avg episode reward: [(0, '9259.523'), (1, '8697.305')] [2023-12-26 20:32:26,554][105692] Updated weights for policy 0, policy_version 710731 (0.0007) [2023-12-26 20:32:26,602][105692] Updated weights for policy 0, policy_version 710741 (0.0010) [2023-12-26 20:32:26,627][105620] Updated weights for policy 1, policy_version 711454 (0.0007) [2023-12-26 20:32:26,649][105692] Updated weights for policy 0, policy_version 710751 (0.0010) [2023-12-26 20:32:26,672][105620] Updated weights for policy 1, policy_version 711464 (0.0005) [2023-12-26 20:32:26,718][105620] Updated weights for policy 1, policy_version 711474 (0.0006) [2023-12-26 20:32:27,359][105692] Updated weights for policy 0, policy_version 710761 (0.0010) [2023-12-26 20:32:27,406][105620] Updated weights for policy 1, policy_version 711484 (0.0008) [2023-12-26 20:32:27,412][105692] Updated weights for policy 0, policy_version 710771 (0.0005) [2023-12-26 20:32:27,464][105620] Updated weights for policy 1, policy_version 711494 (0.0008) [2023-12-26 20:32:27,468][105692] Updated weights for policy 0, policy_version 710781 (0.0005) [2023-12-26 20:32:27,512][105620] Updated weights for policy 1, policy_version 711504 (0.0005) [2023-12-26 20:32:27,519][105692] Updated weights for policy 0, policy_version 710791 (0.0005) [2023-12-26 20:32:28,038][105692] Updated weights for policy 0, policy_version 710801 (0.0005) [2023-12-26 20:32:28,090][105692] Updated weights for policy 0, policy_version 710811 (0.0005) [2023-12-26 20:32:28,147][105692] Updated weights for policy 0, policy_version 710821 (0.0005) [2023-12-26 20:32:28,393][105620] Updated weights for policy 1, policy_version 711514 (0.0009) [2023-12-26 20:32:28,452][105620] Updated weights for policy 1, policy_version 711524 (0.0009) [2023-12-26 20:32:28,505][105620] Updated weights for policy 1, policy_version 711535 (0.0009) [2023-12-26 20:32:28,680][105692] Updated weights for policy 0, policy_version 710831 (0.0005) [2023-12-26 20:32:28,731][105692] Updated weights for policy 0, policy_version 710841 (0.0005) [2023-12-26 20:32:28,786][105692] Updated weights for policy 0, policy_version 710851 (0.0010) [2023-12-26 20:32:29,233][105620] Updated weights for policy 1, policy_version 711545 (0.0010) [2023-12-26 20:32:29,289][105620] Updated weights for policy 1, policy_version 711555 (0.0010) [2023-12-26 20:32:29,345][105620] Updated weights for policy 1, policy_version 711565 (0.0010) [2023-12-26 20:32:29,408][105620] Updated weights for policy 1, policy_version 711575 (0.0008) [2023-12-26 20:32:29,497][105692] Updated weights for policy 0, policy_version 710861 (0.0009) [2023-12-26 20:32:29,540][105692] Updated weights for policy 0, policy_version 710871 (0.0008) [2023-12-26 20:32:29,604][105692] Updated weights for policy 0, policy_version 710881 (0.0008) [2023-12-26 20:32:30,064][105620] Updated weights for policy 1, policy_version 711585 (0.0006) [2023-12-26 20:32:30,131][105620] Updated weights for policy 1, policy_version 711595 (0.0006) [2023-12-26 20:32:30,185][105586] KL-divergence is very high: 110.8594 [2023-12-26 20:32:30,197][105620] Updated weights for policy 1, policy_version 711605 (0.0006) [2023-12-26 20:32:30,327][105692] Updated weights for policy 0, policy_version 710891 (0.0011) [2023-12-26 20:32:30,384][105692] Updated weights for policy 0, policy_version 710901 (0.0010) [2023-12-26 20:32:30,438][105692] Updated weights for policy 0, policy_version 710911 (0.0010) [2023-12-26 20:32:30,763][105620] Updated weights for policy 1, policy_version 711615 (0.0007) [2023-12-26 20:32:30,823][105620] Updated weights for policy 1, policy_version 711625 (0.0009) [2023-12-26 20:32:30,886][105620] Updated weights for policy 1, policy_version 711635 (0.0007) [2023-12-26 20:32:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 364224512. Throughput: 0: 10098.6, 1: 9691.8. Samples: 364191180. Policy #0 lag: (min: 14.0, avg: 21.6, max: 46.0) [2023-12-26 20:32:31,062][104569] Avg episode reward: [(0, '9351.365'), (1, '8210.398')] [2023-12-26 20:32:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000711640_182198272.pth... [2023-12-26 20:32:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000710920_182026240.pth... [2023-12-26 20:32:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000710520_181911552.pth [2023-12-26 20:32:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000709736_181723136.pth [2023-12-26 20:32:31,131][105692] Updated weights for policy 0, policy_version 710921 (0.0010) [2023-12-26 20:32:31,199][105692] Updated weights for policy 0, policy_version 710931 (0.0010) [2023-12-26 20:32:31,257][105692] Updated weights for policy 0, policy_version 710941 (0.0009) [2023-12-26 20:32:31,309][105692] Updated weights for policy 0, policy_version 710951 (0.0009) [2023-12-26 20:32:31,548][105620] Updated weights for policy 1, policy_version 711645 (0.0008) [2023-12-26 20:32:31,599][105620] Updated weights for policy 1, policy_version 711655 (0.0010) [2023-12-26 20:32:31,659][105620] Updated weights for policy 1, policy_version 711665 (0.0008) [2023-12-26 20:32:32,081][105692] Updated weights for policy 0, policy_version 710961 (0.0010) [2023-12-26 20:32:32,139][105692] Updated weights for policy 0, policy_version 710971 (0.0010) [2023-12-26 20:32:32,187][105692] Updated weights for policy 0, policy_version 710981 (0.0010) [2023-12-26 20:32:32,295][105620] Updated weights for policy 1, policy_version 711675 (0.0008) [2023-12-26 20:32:32,354][105620] Updated weights for policy 1, policy_version 711685 (0.0010) [2023-12-26 20:32:32,422][105620] Updated weights for policy 1, policy_version 711695 (0.0007) [2023-12-26 20:32:32,850][105692] Updated weights for policy 0, policy_version 710991 (0.0007) [2023-12-26 20:32:32,907][105692] Updated weights for policy 0, policy_version 711001 (0.0007) [2023-12-26 20:32:32,959][105692] Updated weights for policy 0, policy_version 711011 (0.0008) [2023-12-26 20:32:33,082][105620] Updated weights for policy 1, policy_version 711705 (0.0006) [2023-12-26 20:32:33,130][105620] Updated weights for policy 1, policy_version 711715 (0.0010) [2023-12-26 20:32:33,187][105620] Updated weights for policy 1, policy_version 711725 (0.0008) [2023-12-26 20:32:33,244][105620] Updated weights for policy 1, policy_version 711735 (0.0010) [2023-12-26 20:32:33,734][105692] Updated weights for policy 0, policy_version 711021 (0.0009) [2023-12-26 20:32:33,787][105692] Updated weights for policy 0, policy_version 711032 (0.0009) [2023-12-26 20:32:33,836][105692] Updated weights for policy 0, policy_version 711043 (0.0008) [2023-12-26 20:32:33,845][105620] Updated weights for policy 1, policy_version 711745 (0.0007) [2023-12-26 20:32:33,897][105620] Updated weights for policy 1, policy_version 711755 (0.0010) [2023-12-26 20:32:33,949][105620] Updated weights for policy 1, policy_version 711765 (0.0010) [2023-12-26 20:32:34,634][105692] Updated weights for policy 0, policy_version 711053 (0.0010) [2023-12-26 20:32:34,642][105620] Updated weights for policy 1, policy_version 711775 (0.0009) [2023-12-26 20:32:34,690][105692] Updated weights for policy 0, policy_version 711063 (0.0008) [2023-12-26 20:32:34,705][105620] Updated weights for policy 1, policy_version 711785 (0.0008) [2023-12-26 20:32:34,749][105692] Updated weights for policy 0, policy_version 711073 (0.0007) [2023-12-26 20:32:34,767][105620] Updated weights for policy 1, policy_version 711795 (0.0007) [2023-12-26 20:32:35,422][105620] Updated weights for policy 1, policy_version 711805 (0.0009) [2023-12-26 20:32:35,485][105620] Updated weights for policy 1, policy_version 711815 (0.0009) [2023-12-26 20:32:35,500][105692] Updated weights for policy 0, policy_version 711083 (0.0008) [2023-12-26 20:32:35,542][105620] Updated weights for policy 1, policy_version 711825 (0.0006) [2023-12-26 20:32:35,559][105692] Updated weights for policy 0, policy_version 711093 (0.0007) [2023-12-26 20:32:35,617][105692] Updated weights for policy 0, policy_version 711103 (0.0007) [2023-12-26 20:32:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.8, 300 sec: 19633.0). Total num frames: 364322816. Throughput: 0: 10064.0, 1: 9768.0. Samples: 364312964. Policy #0 lag: (min: 14.0, avg: 21.6, max: 46.0) [2023-12-26 20:32:36,063][104569] Avg episode reward: [(0, '9261.376'), (1, '8374.253')] [2023-12-26 20:32:36,183][105620] Updated weights for policy 1, policy_version 711835 (0.0008) [2023-12-26 20:32:36,255][105620] Updated weights for policy 1, policy_version 711845 (0.0009) [2023-12-26 20:32:36,315][105620] Updated weights for policy 1, policy_version 711855 (0.0008) [2023-12-26 20:32:36,452][105692] Updated weights for policy 0, policy_version 711113 (0.0009) [2023-12-26 20:32:36,508][105692] Updated weights for policy 0, policy_version 711123 (0.0009) [2023-12-26 20:32:36,564][105692] Updated weights for policy 0, policy_version 711133 (0.0009) [2023-12-26 20:32:36,622][105692] Updated weights for policy 0, policy_version 711143 (0.0008) [2023-12-26 20:32:37,079][105620] Updated weights for policy 1, policy_version 711865 (0.0009) [2023-12-26 20:32:37,141][105620] Updated weights for policy 1, policy_version 711875 (0.0010) [2023-12-26 20:32:37,196][105620] Updated weights for policy 1, policy_version 711885 (0.0010) [2023-12-26 20:32:37,258][105620] Updated weights for policy 1, policy_version 711895 (0.0006) [2023-12-26 20:32:37,328][105692] Updated weights for policy 0, policy_version 711153 (0.0010) [2023-12-26 20:32:37,390][105692] Updated weights for policy 0, policy_version 711163 (0.0011) [2023-12-26 20:32:37,459][105692] Updated weights for policy 0, policy_version 711173 (0.0011) [2023-12-26 20:32:37,894][105620] Updated weights for policy 1, policy_version 711905 (0.0010) [2023-12-26 20:32:37,946][105620] Updated weights for policy 1, policy_version 711915 (0.0010) [2023-12-26 20:32:38,001][105620] Updated weights for policy 1, policy_version 711925 (0.0011) [2023-12-26 20:32:38,141][105692] Updated weights for policy 0, policy_version 711183 (0.0007) [2023-12-26 20:32:38,194][105692] Updated weights for policy 0, policy_version 711193 (0.0009) [2023-12-26 20:32:38,239][105692] Updated weights for policy 0, policy_version 711203 (0.0008) [2023-12-26 20:32:38,683][105620] Updated weights for policy 1, policy_version 711935 (0.0010) [2023-12-26 20:32:38,736][105620] Updated weights for policy 1, policy_version 711945 (0.0009) [2023-12-26 20:32:38,784][105620] Updated weights for policy 1, policy_version 711955 (0.0008) [2023-12-26 20:32:38,963][105692] Updated weights for policy 0, policy_version 711213 (0.0008) [2023-12-26 20:32:39,011][105692] Updated weights for policy 0, policy_version 711223 (0.0008) [2023-12-26 20:32:39,059][105692] Updated weights for policy 0, policy_version 711233 (0.0008) [2023-12-26 20:32:39,577][105620] Updated weights for policy 1, policy_version 711965 (0.0009) [2023-12-26 20:32:39,629][105620] Updated weights for policy 1, policy_version 711975 (0.0009) [2023-12-26 20:32:39,685][105620] Updated weights for policy 1, policy_version 711985 (0.0009) [2023-12-26 20:32:39,797][105692] Updated weights for policy 0, policy_version 711243 (0.0009) [2023-12-26 20:32:39,856][105692] Updated weights for policy 0, policy_version 711253 (0.0008) [2023-12-26 20:32:39,915][105692] Updated weights for policy 0, policy_version 711263 (0.0010) [2023-12-26 20:32:40,500][105620] Updated weights for policy 1, policy_version 711995 (0.0010) [2023-12-26 20:32:40,556][105620] Updated weights for policy 1, policy_version 712005 (0.0008) [2023-12-26 20:32:40,616][105620] Updated weights for policy 1, policy_version 712015 (0.0008) [2023-12-26 20:32:40,655][105692] Updated weights for policy 0, policy_version 711273 (0.0008) [2023-12-26 20:32:40,715][105692] Updated weights for policy 0, policy_version 711283 (0.0010) [2023-12-26 20:32:40,773][105692] Updated weights for policy 0, policy_version 711293 (0.0010) [2023-12-26 20:32:40,829][105692] Updated weights for policy 0, policy_version 711303 (0.0008) [2023-12-26 20:32:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 364421120. Throughput: 0: 9969.3, 1: 9733.8. Samples: 364427984. Policy #0 lag: (min: 14.0, avg: 21.6, max: 46.0) [2023-12-26 20:32:41,063][104569] Avg episode reward: [(0, '8809.744'), (1, '8645.552')] [2023-12-26 20:32:41,460][105620] Updated weights for policy 1, policy_version 712025 (0.0008) [2023-12-26 20:32:41,506][105692] Updated weights for policy 0, policy_version 711313 (0.0006) [2023-12-26 20:32:41,518][105620] Updated weights for policy 1, policy_version 712035 (0.0009) [2023-12-26 20:32:41,563][105692] Updated weights for policy 0, policy_version 711323 (0.0009) [2023-12-26 20:32:41,575][105620] Updated weights for policy 1, policy_version 712045 (0.0010) [2023-12-26 20:32:41,621][105692] Updated weights for policy 0, policy_version 711333 (0.0010) [2023-12-26 20:32:41,639][105620] Updated weights for policy 1, policy_version 712055 (0.0008) [2023-12-26 20:32:42,387][105692] Updated weights for policy 0, policy_version 711343 (0.0010) [2023-12-26 20:32:42,401][105620] Updated weights for policy 1, policy_version 712065 (0.0007) [2023-12-26 20:32:42,440][105692] Updated weights for policy 0, policy_version 711353 (0.0011) [2023-12-26 20:32:42,454][105620] Updated weights for policy 1, policy_version 712075 (0.0005) [2023-12-26 20:32:42,492][105692] Updated weights for policy 0, policy_version 711363 (0.0010) [2023-12-26 20:32:42,510][105620] Updated weights for policy 1, policy_version 712085 (0.0005) [2023-12-26 20:32:43,229][105692] Updated weights for policy 0, policy_version 711373 (0.0010) [2023-12-26 20:32:43,278][105620] Updated weights for policy 1, policy_version 712095 (0.0007) [2023-12-26 20:32:43,285][105692] Updated weights for policy 0, policy_version 711383 (0.0009) [2023-12-26 20:32:43,336][105620] Updated weights for policy 1, policy_version 712105 (0.0009) [2023-12-26 20:32:43,338][105692] Updated weights for policy 0, policy_version 711393 (0.0008) [2023-12-26 20:32:43,387][105620] Updated weights for policy 1, policy_version 712115 (0.0005) [2023-12-26 20:32:44,050][105620] Updated weights for policy 1, policy_version 712125 (0.0008) [2023-12-26 20:32:44,094][105620] Updated weights for policy 1, policy_version 712135 (0.0010) [2023-12-26 20:32:44,114][105692] Updated weights for policy 0, policy_version 711403 (0.0009) [2023-12-26 20:32:44,156][105620] Updated weights for policy 1, policy_version 712145 (0.0010) [2023-12-26 20:32:44,166][105692] Updated weights for policy 0, policy_version 711413 (0.0008) [2023-12-26 20:32:44,214][105692] Updated weights for policy 0, policy_version 711423 (0.0011) [2023-12-26 20:32:44,856][105620] Updated weights for policy 1, policy_version 712155 (0.0010) [2023-12-26 20:32:44,920][105620] Updated weights for policy 1, policy_version 712165 (0.0007) [2023-12-26 20:32:44,981][105620] Updated weights for policy 1, policy_version 712175 (0.0007) [2023-12-26 20:32:44,991][105692] Updated weights for policy 0, policy_version 711433 (0.0010) [2023-12-26 20:32:45,057][105692] Updated weights for policy 0, policy_version 711443 (0.0009) [2023-12-26 20:32:45,123][105692] Updated weights for policy 0, policy_version 711453 (0.0011) [2023-12-26 20:32:45,191][105692] Updated weights for policy 0, policy_version 711463 (0.0011) [2023-12-26 20:32:45,606][105620] Updated weights for policy 1, policy_version 712185 (0.0007) [2023-12-26 20:32:45,672][105620] Updated weights for policy 1, policy_version 712195 (0.0007) [2023-12-26 20:32:45,733][105620] Updated weights for policy 1, policy_version 712205 (0.0009) [2023-12-26 20:32:45,788][105620] Updated weights for policy 1, policy_version 712215 (0.0005) [2023-12-26 20:32:45,848][105692] Updated weights for policy 0, policy_version 711473 (0.0010) [2023-12-26 20:32:45,912][105692] Updated weights for policy 0, policy_version 711483 (0.0006) [2023-12-26 20:32:45,980][105692] Updated weights for policy 0, policy_version 711493 (0.0005) [2023-12-26 20:32:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 364519424. Throughput: 0: 9976.3, 1: 9689.3. Samples: 364484544. Policy #0 lag: (min: 14.0, avg: 21.6, max: 46.0) [2023-12-26 20:32:46,062][104569] Avg episode reward: [(0, '8810.953'), (1, '8453.959')] [2023-12-26 20:32:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000711496_182173696.pth... [2023-12-26 20:32:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000712216_182345728.pth... [2023-12-26 20:32:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000711064_182050816.pth [2023-12-26 20:32:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000710312_181870592.pth [2023-12-26 20:32:46,336][105620] Updated weights for policy 1, policy_version 712225 (0.0009) [2023-12-26 20:32:46,394][105620] Updated weights for policy 1, policy_version 712235 (0.0005) [2023-12-26 20:32:46,448][105620] Updated weights for policy 1, policy_version 712245 (0.0005) [2023-12-26 20:32:46,815][105692] Updated weights for policy 0, policy_version 711503 (0.0008) [2023-12-26 20:32:46,875][105692] Updated weights for policy 0, policy_version 711513 (0.0008) [2023-12-26 20:32:46,936][105692] Updated weights for policy 0, policy_version 711523 (0.0008) [2023-12-26 20:32:46,993][105620] Updated weights for policy 1, policy_version 712255 (0.0009) [2023-12-26 20:32:47,048][105620] Updated weights for policy 1, policy_version 712265 (0.0010) [2023-12-26 20:32:47,107][105620] Updated weights for policy 1, policy_version 712275 (0.0010) [2023-12-26 20:32:47,625][105692] Updated weights for policy 0, policy_version 711533 (0.0006) [2023-12-26 20:32:47,680][105692] Updated weights for policy 0, policy_version 711543 (0.0005) [2023-12-26 20:32:47,732][105692] Updated weights for policy 0, policy_version 711553 (0.0005) [2023-12-26 20:32:47,836][105620] Updated weights for policy 1, policy_version 712285 (0.0008) [2023-12-26 20:32:47,883][105620] Updated weights for policy 1, policy_version 712295 (0.0006) [2023-12-26 20:32:47,942][105620] Updated weights for policy 1, policy_version 712305 (0.0011) [2023-12-26 20:32:48,377][105692] Updated weights for policy 0, policy_version 711563 (0.0008) [2023-12-26 20:32:48,427][105692] Updated weights for policy 0, policy_version 711573 (0.0010) [2023-12-26 20:32:48,478][105692] Updated weights for policy 0, policy_version 711583 (0.0008) [2023-12-26 20:32:48,532][105620] Updated weights for policy 1, policy_version 712315 (0.0009) [2023-12-26 20:32:48,596][105620] Updated weights for policy 1, policy_version 712325 (0.0005) [2023-12-26 20:32:48,655][105620] Updated weights for policy 1, policy_version 712335 (0.0008) [2023-12-26 20:32:49,208][105692] Updated weights for policy 0, policy_version 711593 (0.0009) [2023-12-26 20:32:49,273][105692] Updated weights for policy 0, policy_version 711603 (0.0010) [2023-12-26 20:32:49,300][105620] Updated weights for policy 1, policy_version 712345 (0.0007) [2023-12-26 20:32:49,334][105692] Updated weights for policy 0, policy_version 711613 (0.0010) [2023-12-26 20:32:49,362][105620] Updated weights for policy 1, policy_version 712355 (0.0011) [2023-12-26 20:32:49,405][105692] Updated weights for policy 0, policy_version 711623 (0.0010) [2023-12-26 20:32:49,428][105620] Updated weights for policy 1, policy_version 712365 (0.0011) [2023-12-26 20:32:49,491][105620] Updated weights for policy 1, policy_version 712375 (0.0010) [2023-12-26 20:32:50,055][105692] Updated weights for policy 0, policy_version 711633 (0.0008) [2023-12-26 20:32:50,108][105692] Updated weights for policy 0, policy_version 711643 (0.0008) [2023-12-26 20:32:50,152][105692] Updated weights for policy 0, policy_version 711653 (0.0008) [2023-12-26 20:32:50,253][105620] Updated weights for policy 1, policy_version 712385 (0.0010) [2023-12-26 20:32:50,265][105586] KL-divergence is very high: 108.2682 [2023-12-26 20:32:50,314][105586] KL-divergence is very high: 142.1171 [2023-12-26 20:32:50,315][105620] Updated weights for policy 1, policy_version 712395 (0.0010) [2023-12-26 20:32:50,381][105620] Updated weights for policy 1, policy_version 712405 (0.0011) [2023-12-26 20:32:50,935][105620] Updated weights for policy 1, policy_version 712415 (0.0005) [2023-12-26 20:32:50,964][105692] Updated weights for policy 0, policy_version 711663 (0.0007) [2023-12-26 20:32:50,987][105620] Updated weights for policy 1, policy_version 712425 (0.0010) [2023-12-26 20:32:51,025][105692] Updated weights for policy 0, policy_version 711673 (0.0006) [2023-12-26 20:32:51,049][105620] Updated weights for policy 1, policy_version 712435 (0.0010) [2023-12-26 20:32:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 364609536. Throughput: 0: 9955.5, 1: 9756.9. Samples: 364605524. Policy #0 lag: (min: 14.0, avg: 21.6, max: 46.0) [2023-12-26 20:32:51,062][104569] Avg episode reward: [(0, '5732.281'), (1, '8723.025')] [2023-12-26 20:32:51,084][105692] Updated weights for policy 0, policy_version 711683 (0.0007) [2023-12-26 20:32:51,798][105620] Updated weights for policy 1, policy_version 712445 (0.0009) [2023-12-26 20:32:51,813][105692] Updated weights for policy 0, policy_version 711693 (0.0008) [2023-12-26 20:32:51,857][105620] Updated weights for policy 1, policy_version 712455 (0.0008) [2023-12-26 20:32:51,860][105692] Updated weights for policy 0, policy_version 711703 (0.0005) [2023-12-26 20:32:51,912][105620] Updated weights for policy 1, policy_version 712465 (0.0009) [2023-12-26 20:32:51,916][105692] Updated weights for policy 0, policy_version 711713 (0.0005) [2023-12-26 20:32:52,574][105692] Updated weights for policy 0, policy_version 711723 (0.0007) [2023-12-26 20:32:52,637][105692] Updated weights for policy 0, policy_version 711733 (0.0006) [2023-12-26 20:32:52,699][105692] Updated weights for policy 0, policy_version 711743 (0.0007) [2023-12-26 20:32:52,777][105620] Updated weights for policy 1, policy_version 712475 (0.0009) [2023-12-26 20:32:52,830][105620] Updated weights for policy 1, policy_version 712485 (0.0010) [2023-12-26 20:32:52,903][105620] Updated weights for policy 1, policy_version 712495 (0.0009) [2023-12-26 20:32:53,306][105692] Updated weights for policy 0, policy_version 711753 (0.0009) [2023-12-26 20:32:53,365][105692] Updated weights for policy 0, policy_version 711763 (0.0005) [2023-12-26 20:32:53,410][105692] Updated weights for policy 0, policy_version 711773 (0.0005) [2023-12-26 20:32:53,453][105692] Updated weights for policy 0, policy_version 711783 (0.0005) [2023-12-26 20:32:53,753][105620] Updated weights for policy 1, policy_version 712505 (0.0009) [2023-12-26 20:32:53,814][105620] Updated weights for policy 1, policy_version 712515 (0.0009) [2023-12-26 20:32:53,878][105620] Updated weights for policy 1, policy_version 712525 (0.0009) [2023-12-26 20:32:53,936][105620] Updated weights for policy 1, policy_version 712535 (0.0009) [2023-12-26 20:32:54,051][105692] Updated weights for policy 0, policy_version 711793 (0.0005) [2023-12-26 20:32:54,111][105692] Updated weights for policy 0, policy_version 711803 (0.0005) [2023-12-26 20:32:54,166][105692] Updated weights for policy 0, policy_version 711813 (0.0008) [2023-12-26 20:32:54,670][105620] Updated weights for policy 1, policy_version 712545 (0.0006) [2023-12-26 20:32:54,724][105620] Updated weights for policy 1, policy_version 712555 (0.0005) [2023-12-26 20:32:54,769][105620] Updated weights for policy 1, policy_version 712565 (0.0005) [2023-12-26 20:32:54,904][105692] Updated weights for policy 0, policy_version 711823 (0.0010) [2023-12-26 20:32:54,958][105692] Updated weights for policy 0, policy_version 711833 (0.0011) [2023-12-26 20:32:55,019][105692] Updated weights for policy 0, policy_version 711843 (0.0009) [2023-12-26 20:32:55,367][105620] Updated weights for policy 1, policy_version 712575 (0.0005) [2023-12-26 20:32:55,416][105620] Updated weights for policy 1, policy_version 712585 (0.0009) [2023-12-26 20:32:55,463][105620] Updated weights for policy 1, policy_version 712595 (0.0010) [2023-12-26 20:32:55,725][105692] Updated weights for policy 0, policy_version 711853 (0.0008) [2023-12-26 20:32:55,782][105692] Updated weights for policy 0, policy_version 711863 (0.0010) [2023-12-26 20:32:55,840][105692] Updated weights for policy 0, policy_version 711873 (0.0010) [2023-12-26 20:32:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 364716032. Throughput: 0: 10012.6, 1: 9838.6. Samples: 364724912. Policy #0 lag: (min: 14.0, avg: 21.6, max: 46.0) [2023-12-26 20:32:56,062][104569] Avg episode reward: [(0, '4620.186'), (1, '8812.809')] [2023-12-26 20:32:56,194][105620] Updated weights for policy 1, policy_version 712605 (0.0010) [2023-12-26 20:32:56,250][105620] Updated weights for policy 1, policy_version 712615 (0.0009) [2023-12-26 20:32:56,311][105620] Updated weights for policy 1, policy_version 712625 (0.0006) [2023-12-26 20:32:56,517][105692] Updated weights for policy 0, policy_version 711883 (0.0009) [2023-12-26 20:32:56,582][105692] Updated weights for policy 0, policy_version 711893 (0.0005) [2023-12-26 20:32:56,637][105692] Updated weights for policy 0, policy_version 711903 (0.0007) [2023-12-26 20:32:56,987][105620] Updated weights for policy 1, policy_version 712635 (0.0007) [2023-12-26 20:32:57,038][105620] Updated weights for policy 1, policy_version 712645 (0.0010) [2023-12-26 20:32:57,101][105620] Updated weights for policy 1, policy_version 712655 (0.0010) [2023-12-26 20:32:57,259][105692] Updated weights for policy 0, policy_version 711913 (0.0008) [2023-12-26 20:32:57,314][105692] Updated weights for policy 0, policy_version 711923 (0.0005) [2023-12-26 20:32:57,376][105692] Updated weights for policy 0, policy_version 711934 (0.0009) [2023-12-26 20:32:57,427][105692] Updated weights for policy 0, policy_version 711944 (0.0007) [2023-12-26 20:32:57,830][105620] Updated weights for policy 1, policy_version 712665 (0.0010) [2023-12-26 20:32:57,885][105620] Updated weights for policy 1, policy_version 712675 (0.0010) [2023-12-26 20:32:57,929][105620] Updated weights for policy 1, policy_version 712685 (0.0010) [2023-12-26 20:32:57,973][105620] Updated weights for policy 1, policy_version 712695 (0.0010) [2023-12-26 20:32:58,144][105692] Updated weights for policy 0, policy_version 711954 (0.0006) [2023-12-26 20:32:58,204][105692] Updated weights for policy 0, policy_version 711964 (0.0009) [2023-12-26 20:32:58,270][105692] Updated weights for policy 0, policy_version 711974 (0.0007) [2023-12-26 20:32:58,884][105620] Updated weights for policy 1, policy_version 712705 (0.0010) [2023-12-26 20:32:58,950][105620] Updated weights for policy 1, policy_version 712715 (0.0010) [2023-12-26 20:32:59,003][105692] Updated weights for policy 0, policy_version 711984 (0.0006) [2023-12-26 20:32:59,008][105620] Updated weights for policy 1, policy_version 712725 (0.0008) [2023-12-26 20:32:59,057][105692] Updated weights for policy 0, policy_version 711994 (0.0007) [2023-12-26 20:32:59,105][105692] Updated weights for policy 0, policy_version 712004 (0.0005) [2023-12-26 20:32:59,727][105692] Updated weights for policy 0, policy_version 712014 (0.0008) [2023-12-26 20:32:59,789][105692] Updated weights for policy 0, policy_version 712024 (0.0009) [2023-12-26 20:32:59,839][105620] Updated weights for policy 1, policy_version 712735 (0.0010) [2023-12-26 20:32:59,853][105692] Updated weights for policy 0, policy_version 712034 (0.0008) [2023-12-26 20:32:59,892][105620] Updated weights for policy 1, policy_version 712745 (0.0008) [2023-12-26 20:32:59,950][105620] Updated weights for policy 1, policy_version 712756 (0.0009) [2023-12-26 20:33:00,662][105692] Updated weights for policy 0, policy_version 712044 (0.0008) [2023-12-26 20:33:00,713][105692] Updated weights for policy 0, policy_version 712054 (0.0008) [2023-12-26 20:33:00,760][105692] Updated weights for policy 0, policy_version 712064 (0.0008) [2023-12-26 20:33:00,778][105620] Updated weights for policy 1, policy_version 712766 (0.0010) [2023-12-26 20:33:00,823][105620] Updated weights for policy 1, policy_version 712776 (0.0010) [2023-12-26 20:33:00,871][105620] Updated weights for policy 1, policy_version 712786 (0.0010) [2023-12-26 20:33:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 364814336. Throughput: 0: 10047.8, 1: 9819.7. Samples: 364782748. Policy #0 lag: (min: 14.0, avg: 21.6, max: 46.0) [2023-12-26 20:33:01,062][104569] Avg episode reward: [(0, '4585.416'), (1, '8813.677')] [2023-12-26 20:33:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000712792_182493184.pth... [2023-12-26 20:33:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000712072_182321152.pth... [2023-12-26 20:33:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000711640_182198272.pth [2023-12-26 20:33:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000710920_182026240.pth [2023-12-26 20:33:01,535][105692] Updated weights for policy 0, policy_version 712074 (0.0006) [2023-12-26 20:33:01,584][105692] Updated weights for policy 0, policy_version 712084 (0.0008) [2023-12-26 20:33:01,635][105692] Updated weights for policy 0, policy_version 712094 (0.0008) [2023-12-26 20:33:01,644][105620] Updated weights for policy 1, policy_version 712796 (0.0010) [2023-12-26 20:33:01,688][105692] Updated weights for policy 0, policy_version 712104 (0.0008) [2023-12-26 20:33:01,705][105620] Updated weights for policy 1, policy_version 712806 (0.0010) [2023-12-26 20:33:01,759][105620] Updated weights for policy 1, policy_version 712816 (0.0010) [2023-12-26 20:33:02,455][105692] Updated weights for policy 0, policy_version 712114 (0.0008) [2023-12-26 20:33:02,514][105692] Updated weights for policy 0, policy_version 712124 (0.0009) [2023-12-26 20:33:02,532][105620] Updated weights for policy 1, policy_version 712826 (0.0009) [2023-12-26 20:33:02,575][105692] Updated weights for policy 0, policy_version 712134 (0.0009) [2023-12-26 20:33:02,589][105620] Updated weights for policy 1, policy_version 712836 (0.0005) [2023-12-26 20:33:02,656][105620] Updated weights for policy 1, policy_version 712846 (0.0006) [2023-12-26 20:33:02,718][105620] Updated weights for policy 1, policy_version 712856 (0.0011) [2023-12-26 20:33:03,257][105620] Updated weights for policy 1, policy_version 712866 (0.0005) [2023-12-26 20:33:03,322][105620] Updated weights for policy 1, policy_version 712876 (0.0005) [2023-12-26 20:33:03,384][105620] Updated weights for policy 1, policy_version 712886 (0.0007) [2023-12-26 20:33:03,441][105692] Updated weights for policy 0, policy_version 712144 (0.0008) [2023-12-26 20:33:03,495][105692] Updated weights for policy 0, policy_version 712154 (0.0008) [2023-12-26 20:33:03,548][105692] Updated weights for policy 0, policy_version 712164 (0.0009) [2023-12-26 20:33:03,910][105620] Updated weights for policy 1, policy_version 712896 (0.0006) [2023-12-26 20:33:03,975][105620] Updated weights for policy 1, policy_version 712906 (0.0009) [2023-12-26 20:33:04,046][105620] Updated weights for policy 1, policy_version 712916 (0.0009) [2023-12-26 20:33:04,434][105692] Updated weights for policy 0, policy_version 712174 (0.0008) [2023-12-26 20:33:04,496][105692] Updated weights for policy 0, policy_version 712184 (0.0009) [2023-12-26 20:33:04,553][105692] Updated weights for policy 0, policy_version 712194 (0.0008) [2023-12-26 20:33:04,732][105620] Updated weights for policy 1, policy_version 712926 (0.0011) [2023-12-26 20:33:04,793][105620] Updated weights for policy 1, policy_version 712936 (0.0011) [2023-12-26 20:33:04,850][105620] Updated weights for policy 1, policy_version 712946 (0.0010) [2023-12-26 20:33:05,335][105692] Updated weights for policy 0, policy_version 712204 (0.0008) [2023-12-26 20:33:05,400][105692] Updated weights for policy 0, policy_version 712214 (0.0008) [2023-12-26 20:33:05,461][105692] Updated weights for policy 0, policy_version 712224 (0.0008) [2023-12-26 20:33:05,590][105620] Updated weights for policy 1, policy_version 712956 (0.0010) [2023-12-26 20:33:05,655][105620] Updated weights for policy 1, policy_version 712966 (0.0010) [2023-12-26 20:33:05,719][105620] Updated weights for policy 1, policy_version 712976 (0.0010) [2023-12-26 20:33:06,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 364904448. Throughput: 0: 9852.1, 1: 9803.0. Samples: 364895364. Policy #0 lag: (min: 14.0, avg: 21.6, max: 46.0) [2023-12-26 20:33:06,063][104569] Avg episode reward: [(0, '6461.272'), (1, '8634.666')] [2023-12-26 20:33:06,140][105692] Updated weights for policy 0, policy_version 712234 (0.0008) [2023-12-26 20:33:06,204][105692] Updated weights for policy 0, policy_version 712244 (0.0008) [2023-12-26 20:33:06,271][105692] Updated weights for policy 0, policy_version 712254 (0.0009) [2023-12-26 20:33:06,338][105692] Updated weights for policy 0, policy_version 712264 (0.0010) [2023-12-26 20:33:06,404][105620] Updated weights for policy 1, policy_version 712986 (0.0010) [2023-12-26 20:33:06,464][105620] Updated weights for policy 1, policy_version 712996 (0.0007) [2023-12-26 20:33:06,522][105620] Updated weights for policy 1, policy_version 713006 (0.0005) [2023-12-26 20:33:06,582][105620] Updated weights for policy 1, policy_version 713016 (0.0007) [2023-12-26 20:33:06,997][105692] Updated weights for policy 0, policy_version 712274 (0.0005) [2023-12-26 20:33:07,044][105692] Updated weights for policy 0, policy_version 712284 (0.0005) [2023-12-26 20:33:07,092][105692] Updated weights for policy 0, policy_version 712294 (0.0005) [2023-12-26 20:33:07,275][105620] Updated weights for policy 1, policy_version 713026 (0.0005) [2023-12-26 20:33:07,327][105620] Updated weights for policy 1, policy_version 713036 (0.0005) [2023-12-26 20:33:07,381][105620] Updated weights for policy 1, policy_version 713046 (0.0005) [2023-12-26 20:33:07,782][105692] Updated weights for policy 0, policy_version 712304 (0.0006) [2023-12-26 20:33:07,838][105692] Updated weights for policy 0, policy_version 712314 (0.0005) [2023-12-26 20:33:07,898][105692] Updated weights for policy 0, policy_version 712324 (0.0005) [2023-12-26 20:33:08,045][105620] Updated weights for policy 1, policy_version 713056 (0.0008) [2023-12-26 20:33:08,089][105620] Updated weights for policy 1, policy_version 713066 (0.0010) [2023-12-26 20:33:08,136][105620] Updated weights for policy 1, policy_version 713076 (0.0005) [2023-12-26 20:33:08,436][105692] Updated weights for policy 0, policy_version 712334 (0.0005) [2023-12-26 20:33:08,494][105692] Updated weights for policy 0, policy_version 712344 (0.0010) [2023-12-26 20:33:08,563][105692] Updated weights for policy 0, policy_version 712354 (0.0005) [2023-12-26 20:33:08,812][105620] Updated weights for policy 1, policy_version 713086 (0.0006) [2023-12-26 20:33:08,880][105620] Updated weights for policy 1, policy_version 713096 (0.0008) [2023-12-26 20:33:08,940][105620] Updated weights for policy 1, policy_version 713106 (0.0011) [2023-12-26 20:33:09,160][105692] Updated weights for policy 0, policy_version 712364 (0.0006) [2023-12-26 20:33:09,214][105692] Updated weights for policy 0, policy_version 712374 (0.0007) [2023-12-26 20:33:09,278][105692] Updated weights for policy 0, policy_version 712384 (0.0008) [2023-12-26 20:33:09,698][105620] Updated weights for policy 1, policy_version 713116 (0.0010) [2023-12-26 20:33:09,754][105620] Updated weights for policy 1, policy_version 713126 (0.0009) [2023-12-26 20:33:09,813][105620] Updated weights for policy 1, policy_version 713136 (0.0010) [2023-12-26 20:33:10,004][105692] Updated weights for policy 0, policy_version 712394 (0.0009) [2023-12-26 20:33:10,067][105692] Updated weights for policy 0, policy_version 712404 (0.0011) [2023-12-26 20:33:10,126][105692] Updated weights for policy 0, policy_version 712414 (0.0011) [2023-12-26 20:33:10,185][105692] Updated weights for policy 0, policy_version 712424 (0.0011) [2023-12-26 20:33:10,635][105620] Updated weights for policy 1, policy_version 713146 (0.0009) [2023-12-26 20:33:10,694][105620] Updated weights for policy 1, policy_version 713156 (0.0010) [2023-12-26 20:33:10,752][105620] Updated weights for policy 1, policy_version 713166 (0.0009) [2023-12-26 20:33:10,818][105620] Updated weights for policy 1, policy_version 713176 (0.0008) [2023-12-26 20:33:10,827][105692] Updated weights for policy 0, policy_version 712434 (0.0006) [2023-12-26 20:33:10,886][105692] Updated weights for policy 0, policy_version 712444 (0.0008) [2023-12-26 20:33:10,937][105692] Updated weights for policy 0, policy_version 712454 (0.0009) [2023-12-26 20:33:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 365010944. Throughput: 0: 9824.3, 1: 9842.0. Samples: 365014208. Policy #0 lag: (min: 14.0, avg: 21.6, max: 46.0) [2023-12-26 20:33:11,063][104569] Avg episode reward: [(0, '5524.734'), (1, '2604.124')] [2023-12-26 20:33:11,591][105692] Updated weights for policy 0, policy_version 712464 (0.0006) [2023-12-26 20:33:11,661][105692] Updated weights for policy 0, policy_version 712474 (0.0007) [2023-12-26 20:33:11,723][105620] Updated weights for policy 1, policy_version 713186 (0.0009) [2023-12-26 20:33:11,734][105692] Updated weights for policy 0, policy_version 712484 (0.0008) [2023-12-26 20:33:11,786][105620] Updated weights for policy 1, policy_version 713196 (0.0010) [2023-12-26 20:33:11,841][105620] Updated weights for policy 1, policy_version 713206 (0.0009) [2023-12-26 20:33:12,408][105692] Updated weights for policy 0, policy_version 712494 (0.0007) [2023-12-26 20:33:12,464][105692] Updated weights for policy 0, policy_version 712504 (0.0007) [2023-12-26 20:33:12,519][105692] Updated weights for policy 0, policy_version 712514 (0.0006) [2023-12-26 20:33:12,585][105620] Updated weights for policy 1, policy_version 713216 (0.0009) [2023-12-26 20:33:12,646][105620] Updated weights for policy 1, policy_version 713226 (0.0007) [2023-12-26 20:33:12,712][105620] Updated weights for policy 1, policy_version 713236 (0.0009) [2023-12-26 20:33:13,262][105692] Updated weights for policy 0, policy_version 712524 (0.0008) [2023-12-26 20:33:13,325][105692] Updated weights for policy 0, policy_version 712534 (0.0006) [2023-12-26 20:33:13,360][105620] Updated weights for policy 1, policy_version 713246 (0.0007) [2023-12-26 20:33:13,385][105692] Updated weights for policy 0, policy_version 712544 (0.0006) [2023-12-26 20:33:13,409][105620] Updated weights for policy 1, policy_version 713256 (0.0005) [2023-12-26 20:33:13,472][105620] Updated weights for policy 1, policy_version 713266 (0.0005) [2023-12-26 20:33:14,021][105620] Updated weights for policy 1, policy_version 713276 (0.0005) [2023-12-26 20:33:14,082][105620] Updated weights for policy 1, policy_version 713286 (0.0005) [2023-12-26 20:33:14,139][105620] Updated weights for policy 1, policy_version 713296 (0.0006) [2023-12-26 20:33:14,151][105692] Updated weights for policy 0, policy_version 712554 (0.0009) [2023-12-26 20:33:14,215][105692] Updated weights for policy 0, policy_version 712564 (0.0009) [2023-12-26 20:33:14,276][105692] Updated weights for policy 0, policy_version 712574 (0.0010) [2023-12-26 20:33:14,331][105692] Updated weights for policy 0, policy_version 712584 (0.0011) [2023-12-26 20:33:14,769][105620] Updated weights for policy 1, policy_version 713306 (0.0006) [2023-12-26 20:33:14,832][105620] Updated weights for policy 1, policy_version 713316 (0.0008) [2023-12-26 20:33:14,895][105620] Updated weights for policy 1, policy_version 713326 (0.0008) [2023-12-26 20:33:14,947][105620] Updated weights for policy 1, policy_version 713336 (0.0008) [2023-12-26 20:33:15,087][105692] Updated weights for policy 0, policy_version 712594 (0.0007) [2023-12-26 20:33:15,143][105692] Updated weights for policy 0, policy_version 712604 (0.0009) [2023-12-26 20:33:15,210][105692] Updated weights for policy 0, policy_version 712614 (0.0009) [2023-12-26 20:33:15,701][105620] Updated weights for policy 1, policy_version 713346 (0.0009) [2023-12-26 20:33:15,759][105620] Updated weights for policy 1, policy_version 713356 (0.0009) [2023-12-26 20:33:15,821][105620] Updated weights for policy 1, policy_version 713366 (0.0009) [2023-12-26 20:33:15,946][105692] Updated weights for policy 0, policy_version 712624 (0.0006) [2023-12-26 20:33:16,004][105692] Updated weights for policy 0, policy_version 712634 (0.0006) [2023-12-26 20:33:16,058][105692] Updated weights for policy 0, policy_version 712644 (0.0010) [2023-12-26 20:33:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 365101056. Throughput: 0: 9720.8, 1: 9881.5. Samples: 365073284. Policy #0 lag: (min: 14.0, avg: 21.6, max: 46.0) [2023-12-26 20:33:16,063][104569] Avg episode reward: [(0, '7212.541'), (1, '4031.772')] [2023-12-26 20:33:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000713368_182640640.pth... [2023-12-26 20:33:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000712216_182345728.pth [2023-12-26 20:33:16,079][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000712648_182468608.pth... [2023-12-26 20:33:16,082][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000711496_182173696.pth [2023-12-26 20:33:16,531][105620] Updated weights for policy 1, policy_version 713376 (0.0008) [2023-12-26 20:33:16,587][105620] Updated weights for policy 1, policy_version 713386 (0.0008) [2023-12-26 20:33:16,636][105620] Updated weights for policy 1, policy_version 713396 (0.0008) [2023-12-26 20:33:16,764][105692] Updated weights for policy 0, policy_version 712654 (0.0009) [2023-12-26 20:33:16,824][105692] Updated weights for policy 0, policy_version 712664 (0.0009) [2023-12-26 20:33:16,876][105692] Updated weights for policy 0, policy_version 712674 (0.0009) [2023-12-26 20:33:17,320][105620] Updated weights for policy 1, policy_version 713406 (0.0006) [2023-12-26 20:33:17,382][105620] Updated weights for policy 1, policy_version 713416 (0.0008) [2023-12-26 20:33:17,433][105620] Updated weights for policy 1, policy_version 713426 (0.0010) [2023-12-26 20:33:17,646][105692] Updated weights for policy 0, policy_version 712684 (0.0008) [2023-12-26 20:33:17,695][105692] Updated weights for policy 0, policy_version 712694 (0.0005) [2023-12-26 20:33:17,749][105692] Updated weights for policy 0, policy_version 712704 (0.0005) [2023-12-26 20:33:18,146][105620] Updated weights for policy 1, policy_version 713436 (0.0010) [2023-12-26 20:33:18,203][105620] Updated weights for policy 1, policy_version 713446 (0.0009) [2023-12-26 20:33:18,258][105620] Updated weights for policy 1, policy_version 713456 (0.0009) [2023-12-26 20:33:18,398][105692] Updated weights for policy 0, policy_version 712714 (0.0006) [2023-12-26 20:33:18,464][105692] Updated weights for policy 0, policy_version 712724 (0.0008) [2023-12-26 20:33:18,533][105692] Updated weights for policy 0, policy_version 712734 (0.0008) [2023-12-26 20:33:18,594][105692] Updated weights for policy 0, policy_version 712744 (0.0009) [2023-12-26 20:33:19,003][105620] Updated weights for policy 1, policy_version 713466 (0.0008) [2023-12-26 20:33:19,051][105620] Updated weights for policy 1, policy_version 713476 (0.0005) [2023-12-26 20:33:19,114][105620] Updated weights for policy 1, policy_version 713486 (0.0005) [2023-12-26 20:33:19,185][105620] Updated weights for policy 1, policy_version 713496 (0.0005) [2023-12-26 20:33:19,294][105692] Updated weights for policy 0, policy_version 712754 (0.0008) [2023-12-26 20:33:19,360][105692] Updated weights for policy 0, policy_version 712764 (0.0009) [2023-12-26 20:33:19,417][105692] Updated weights for policy 0, policy_version 712774 (0.0010) [2023-12-26 20:33:19,831][105620] Updated weights for policy 1, policy_version 713506 (0.0009) [2023-12-26 20:33:19,895][105620] Updated weights for policy 1, policy_version 713516 (0.0009) [2023-12-26 20:33:19,965][105620] Updated weights for policy 1, policy_version 713526 (0.0009) [2023-12-26 20:33:20,286][105692] Updated weights for policy 0, policy_version 712784 (0.0009) [2023-12-26 20:33:20,343][105692] Updated weights for policy 0, policy_version 712794 (0.0009) [2023-12-26 20:33:20,403][105692] Updated weights for policy 0, policy_version 712804 (0.0009) [2023-12-26 20:33:20,752][105620] Updated weights for policy 1, policy_version 713536 (0.0008) [2023-12-26 20:33:20,822][105620] Updated weights for policy 1, policy_version 713546 (0.0010) [2023-12-26 20:33:20,884][105620] Updated weights for policy 1, policy_version 713556 (0.0009) [2023-12-26 20:33:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 365199360. Throughput: 0: 9693.8, 1: 9799.5. Samples: 365190164. Policy #0 lag: (min: 14.0, avg: 21.6, max: 46.0) [2023-12-26 20:33:21,063][104569] Avg episode reward: [(0, '9174.993'), (1, '7321.969')] [2023-12-26 20:33:21,145][105692] Updated weights for policy 0, policy_version 712814 (0.0009) [2023-12-26 20:33:21,206][105692] Updated weights for policy 0, policy_version 712824 (0.0009) [2023-12-26 20:33:21,267][105692] Updated weights for policy 0, policy_version 712834 (0.0010) [2023-12-26 20:33:21,619][105620] Updated weights for policy 1, policy_version 713566 (0.0008) [2023-12-26 20:33:21,685][105620] Updated weights for policy 1, policy_version 713576 (0.0008) [2023-12-26 20:33:21,761][105620] Updated weights for policy 1, policy_version 713586 (0.0008) [2023-12-26 20:33:22,103][105692] Updated weights for policy 0, policy_version 712844 (0.0008) [2023-12-26 20:33:22,163][105692] Updated weights for policy 0, policy_version 712854 (0.0006) [2023-12-26 20:33:22,217][105692] Updated weights for policy 0, policy_version 712864 (0.0006) [2023-12-26 20:33:22,501][105620] Updated weights for policy 1, policy_version 713596 (0.0008) [2023-12-26 20:33:22,559][105620] Updated weights for policy 1, policy_version 713606 (0.0009) [2023-12-26 20:33:22,592][105586] KL-divergence is very high: 106.9323 [2023-12-26 20:33:22,618][105620] Updated weights for policy 1, policy_version 713616 (0.0010) [2023-12-26 20:33:22,843][105692] Updated weights for policy 0, policy_version 712874 (0.0006) [2023-12-26 20:33:22,894][105692] Updated weights for policy 0, policy_version 712884 (0.0008) [2023-12-26 20:33:22,941][105692] Updated weights for policy 0, policy_version 712894 (0.0009) [2023-12-26 20:33:22,991][105692] Updated weights for policy 0, policy_version 712904 (0.0005) [2023-12-26 20:33:23,316][105620] Updated weights for policy 1, policy_version 713626 (0.0009) [2023-12-26 20:33:23,363][105620] Updated weights for policy 1, policy_version 713636 (0.0010) [2023-12-26 20:33:23,414][105620] Updated weights for policy 1, policy_version 713646 (0.0010) [2023-12-26 20:33:23,472][105620] Updated weights for policy 1, policy_version 713656 (0.0010) [2023-12-26 20:33:23,752][105692] Updated weights for policy 0, policy_version 712914 (0.0008) [2023-12-26 20:33:23,800][105692] Updated weights for policy 0, policy_version 712924 (0.0008) [2023-12-26 20:33:23,864][105692] Updated weights for policy 0, policy_version 712934 (0.0009) [2023-12-26 20:33:24,220][105620] Updated weights for policy 1, policy_version 713666 (0.0006) [2023-12-26 20:33:24,280][105620] Updated weights for policy 1, policy_version 713676 (0.0008) [2023-12-26 20:33:24,342][105620] Updated weights for policy 1, policy_version 713686 (0.0010) [2023-12-26 20:33:24,616][105692] Updated weights for policy 0, policy_version 712944 (0.0008) [2023-12-26 20:33:24,663][105692] Updated weights for policy 0, policy_version 712954 (0.0008) [2023-12-26 20:33:24,712][105692] Updated weights for policy 0, policy_version 712964 (0.0008) [2023-12-26 20:33:24,963][105620] Updated weights for policy 1, policy_version 713696 (0.0006) [2023-12-26 20:33:25,020][105620] Updated weights for policy 1, policy_version 713706 (0.0005) [2023-12-26 20:33:25,071][105620] Updated weights for policy 1, policy_version 713716 (0.0005) [2023-12-26 20:33:25,400][105692] Updated weights for policy 0, policy_version 712974 (0.0006) [2023-12-26 20:33:25,457][105692] Updated weights for policy 0, policy_version 712984 (0.0005) [2023-12-26 20:33:25,511][105692] Updated weights for policy 0, policy_version 712994 (0.0006) [2023-12-26 20:33:25,631][105620] Updated weights for policy 1, policy_version 713726 (0.0008) [2023-12-26 20:33:25,689][105620] Updated weights for policy 1, policy_version 713736 (0.0010) [2023-12-26 20:33:25,747][105620] Updated weights for policy 1, policy_version 713746 (0.0010) [2023-12-26 20:33:26,051][105692] Updated weights for policy 0, policy_version 713004 (0.0008) [2023-12-26 20:33:26,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 365297664. Throughput: 0: 9709.5, 1: 9822.8. Samples: 365306944. Policy #0 lag: (min: 14.0, avg: 21.6, max: 46.0) [2023-12-26 20:33:26,063][104569] Avg episode reward: [(0, '9169.184'), (1, '7859.898')] [2023-12-26 20:33:26,107][105692] Updated weights for policy 0, policy_version 713014 (0.0005) [2023-12-26 20:33:26,163][105692] Updated weights for policy 0, policy_version 713024 (0.0006) [2023-12-26 20:33:26,335][105620] Updated weights for policy 1, policy_version 713756 (0.0005) [2023-12-26 20:33:26,387][105620] Updated weights for policy 1, policy_version 713766 (0.0005) [2023-12-26 20:33:26,434][105620] Updated weights for policy 1, policy_version 713776 (0.0005) [2023-12-26 20:33:26,689][105692] Updated weights for policy 0, policy_version 713034 (0.0005) [2023-12-26 20:33:26,737][105692] Updated weights for policy 0, policy_version 713044 (0.0005) [2023-12-26 20:33:26,781][105692] Updated weights for policy 0, policy_version 713054 (0.0006) [2023-12-26 20:33:26,825][105692] Updated weights for policy 0, policy_version 713064 (0.0010) [2023-12-26 20:33:26,999][105620] Updated weights for policy 1, policy_version 713786 (0.0007) [2023-12-26 20:33:27,060][105620] Updated weights for policy 1, policy_version 713796 (0.0010) [2023-12-26 20:33:27,107][105620] Updated weights for policy 1, policy_version 713806 (0.0010) [2023-12-26 20:33:27,161][105620] Updated weights for policy 1, policy_version 713816 (0.0010) [2023-12-26 20:33:27,484][105692] Updated weights for policy 0, policy_version 713074 (0.0010) [2023-12-26 20:33:27,541][105692] Updated weights for policy 0, policy_version 713084 (0.0010) [2023-12-26 20:33:27,582][105692] Updated weights for policy 0, policy_version 713094 (0.0010) [2023-12-26 20:33:27,861][105620] Updated weights for policy 1, policy_version 713826 (0.0005) [2023-12-26 20:33:27,914][105620] Updated weights for policy 1, policy_version 713836 (0.0005) [2023-12-26 20:33:27,967][105620] Updated weights for policy 1, policy_version 713846 (0.0005) [2023-12-26 20:33:28,336][105692] Updated weights for policy 0, policy_version 713104 (0.0010) [2023-12-26 20:33:28,387][105692] Updated weights for policy 0, policy_version 713114 (0.0010) [2023-12-26 20:33:28,449][105692] Updated weights for policy 0, policy_version 713124 (0.0011) [2023-12-26 20:33:28,538][105620] Updated weights for policy 1, policy_version 713856 (0.0008) [2023-12-26 20:33:28,597][105620] Updated weights for policy 1, policy_version 713866 (0.0008) [2023-12-26 20:33:28,644][105620] Updated weights for policy 1, policy_version 713876 (0.0008) [2023-12-26 20:33:29,183][105692] Updated weights for policy 0, policy_version 713134 (0.0010) [2023-12-26 20:33:29,234][105692] Updated weights for policy 0, policy_version 713144 (0.0009) [2023-12-26 20:33:29,295][105692] Updated weights for policy 0, policy_version 713154 (0.0007) [2023-12-26 20:33:29,407][105620] Updated weights for policy 1, policy_version 713886 (0.0008) [2023-12-26 20:33:29,455][105620] Updated weights for policy 1, policy_version 713896 (0.0008) [2023-12-26 20:33:29,513][105620] Updated weights for policy 1, policy_version 713906 (0.0008) [2023-12-26 20:33:30,076][105692] Updated weights for policy 0, policy_version 713164 (0.0010) [2023-12-26 20:33:30,138][105692] Updated weights for policy 0, policy_version 713174 (0.0011) [2023-12-26 20:33:30,196][105692] Updated weights for policy 0, policy_version 713184 (0.0011) [2023-12-26 20:33:30,287][105620] Updated weights for policy 1, policy_version 713916 (0.0009) [2023-12-26 20:33:30,345][105620] Updated weights for policy 1, policy_version 713926 (0.0011) [2023-12-26 20:33:30,404][105620] Updated weights for policy 1, policy_version 713936 (0.0010) [2023-12-26 20:33:30,852][105692] Updated weights for policy 0, policy_version 713194 (0.0009) [2023-12-26 20:33:30,898][105692] Updated weights for policy 0, policy_version 713204 (0.0005) [2023-12-26 20:33:30,946][105692] Updated weights for policy 0, policy_version 713214 (0.0008) [2023-12-26 20:33:30,990][105692] Updated weights for policy 0, policy_version 713224 (0.0010) [2023-12-26 20:33:31,043][105620] Updated weights for policy 1, policy_version 713946 (0.0010) [2023-12-26 20:33:31,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 365404160. Throughput: 0: 9807.8, 1: 9931.2. Samples: 365372800. Policy #0 lag: (min: 14.0, avg: 21.6, max: 46.0) [2023-12-26 20:33:31,062][104569] Avg episode reward: [(0, '9168.867'), (1, '8543.595')] [2023-12-26 20:33:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000713224_182616064.pth... [2023-12-26 20:33:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000712072_182321152.pth [2023-12-26 20:33:31,103][105620] Updated weights for policy 1, policy_version 713956 (0.0011) [2023-12-26 20:33:31,165][105620] Updated weights for policy 1, policy_version 713966 (0.0011) [2023-12-26 20:33:31,226][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000713976_182796288.pth... [2023-12-26 20:33:31,228][105620] Updated weights for policy 1, policy_version 713976 (0.0011) [2023-12-26 20:33:31,230][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000712792_182493184.pth [2023-12-26 20:33:31,626][105692] Updated weights for policy 0, policy_version 713234 (0.0007) [2023-12-26 20:33:31,686][105692] Updated weights for policy 0, policy_version 713244 (0.0006) [2023-12-26 20:33:31,752][105692] Updated weights for policy 0, policy_version 713254 (0.0009) [2023-12-26 20:33:31,990][105620] Updated weights for policy 1, policy_version 713986 (0.0011) [2023-12-26 20:33:32,041][105620] Updated weights for policy 1, policy_version 713996 (0.0010) [2023-12-26 20:33:32,100][105620] Updated weights for policy 1, policy_version 714006 (0.0010) [2023-12-26 20:33:32,337][105692] Updated weights for policy 0, policy_version 713264 (0.0010) [2023-12-26 20:33:32,397][105692] Updated weights for policy 0, policy_version 713274 (0.0010) [2023-12-26 20:33:32,458][105692] Updated weights for policy 0, policy_version 713284 (0.0010) [2023-12-26 20:33:32,921][105620] Updated weights for policy 1, policy_version 714016 (0.0009) [2023-12-26 20:33:32,974][105620] Updated weights for policy 1, policy_version 714026 (0.0009) [2023-12-26 20:33:33,025][105620] Updated weights for policy 1, policy_version 714036 (0.0009) [2023-12-26 20:33:33,054][105692] Updated weights for policy 0, policy_version 713294 (0.0009) [2023-12-26 20:33:33,117][105692] Updated weights for policy 0, policy_version 713304 (0.0009) [2023-12-26 20:33:33,166][105692] Updated weights for policy 0, policy_version 713314 (0.0005) [2023-12-26 20:33:33,594][105620] Updated weights for policy 1, policy_version 714046 (0.0005) [2023-12-26 20:33:33,647][105620] Updated weights for policy 1, policy_version 714056 (0.0006) [2023-12-26 20:33:33,706][105620] Updated weights for policy 1, policy_version 714066 (0.0005) [2023-12-26 20:33:33,709][105692] Updated weights for policy 0, policy_version 713324 (0.0006) [2023-12-26 20:33:33,756][105692] Updated weights for policy 0, policy_version 713334 (0.0010) [2023-12-26 20:33:33,820][105692] Updated weights for policy 0, policy_version 713344 (0.0010) [2023-12-26 20:33:34,383][105620] Updated weights for policy 1, policy_version 714076 (0.0006) [2023-12-26 20:33:34,436][105620] Updated weights for policy 1, policy_version 714086 (0.0008) [2023-12-26 20:33:34,492][105620] Updated weights for policy 1, policy_version 714096 (0.0008) [2023-12-26 20:33:34,588][105692] Updated weights for policy 0, policy_version 713354 (0.0010) [2023-12-26 20:33:34,646][105692] Updated weights for policy 0, policy_version 713364 (0.0011) [2023-12-26 20:33:34,695][105692] Updated weights for policy 0, policy_version 713374 (0.0010) [2023-12-26 20:33:34,754][105692] Updated weights for policy 0, policy_version 713384 (0.0011) [2023-12-26 20:33:35,307][105620] Updated weights for policy 1, policy_version 714106 (0.0008) [2023-12-26 20:33:35,373][105620] Updated weights for policy 1, policy_version 714116 (0.0009) [2023-12-26 20:33:35,440][105620] Updated weights for policy 1, policy_version 714126 (0.0008) [2023-12-26 20:33:35,470][105692] Updated weights for policy 0, policy_version 713394 (0.0008) [2023-12-26 20:33:35,501][105620] Updated weights for policy 1, policy_version 714136 (0.0007) [2023-12-26 20:33:35,526][105692] Updated weights for policy 0, policy_version 713404 (0.0007) [2023-12-26 20:33:35,582][105692] Updated weights for policy 0, policy_version 713414 (0.0009) [2023-12-26 20:33:36,062][104569] Fps is (10 sec: 20480.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 365502464. Throughput: 0: 9931.0, 1: 9820.4. Samples: 365494340. Policy #0 lag: (min: 14.0, avg: 21.6, max: 46.0) [2023-12-26 20:33:36,062][104569] Avg episode reward: [(0, '9260.987'), (1, '8634.895')] [2023-12-26 20:33:36,254][105620] Updated weights for policy 1, policy_version 714146 (0.0009) [2023-12-26 20:33:36,314][105620] Updated weights for policy 1, policy_version 714156 (0.0008) [2023-12-26 20:33:36,340][105692] Updated weights for policy 0, policy_version 713424 (0.0008) [2023-12-26 20:33:36,375][105620] Updated weights for policy 1, policy_version 714166 (0.0007) [2023-12-26 20:33:36,394][105692] Updated weights for policy 0, policy_version 713434 (0.0006) [2023-12-26 20:33:36,448][105692] Updated weights for policy 0, policy_version 713444 (0.0007) [2023-12-26 20:33:37,138][105620] Updated weights for policy 1, policy_version 714176 (0.0009) [2023-12-26 20:33:37,195][105692] Updated weights for policy 0, policy_version 713454 (0.0006) [2023-12-26 20:33:37,201][105620] Updated weights for policy 1, policy_version 714186 (0.0009) [2023-12-26 20:33:37,246][105692] Updated weights for policy 0, policy_version 713464 (0.0005) [2023-12-26 20:33:37,257][105620] Updated weights for policy 1, policy_version 714196 (0.0008) [2023-12-26 20:33:37,298][105692] Updated weights for policy 0, policy_version 713474 (0.0006) [2023-12-26 20:33:37,896][105692] Updated weights for policy 0, policy_version 713484 (0.0008) [2023-12-26 20:33:37,947][105692] Updated weights for policy 0, policy_version 713494 (0.0009) [2023-12-26 20:33:37,998][105692] Updated weights for policy 0, policy_version 713504 (0.0009) [2023-12-26 20:33:38,072][105620] Updated weights for policy 1, policy_version 714206 (0.0009) [2023-12-26 20:33:38,121][105620] Updated weights for policy 1, policy_version 714216 (0.0008) [2023-12-26 20:33:38,189][105620] Updated weights for policy 1, policy_version 714226 (0.0009) [2023-12-26 20:33:38,728][105692] Updated weights for policy 0, policy_version 713514 (0.0008) [2023-12-26 20:33:38,779][105692] Updated weights for policy 0, policy_version 713524 (0.0009) [2023-12-26 20:33:38,833][105692] Updated weights for policy 0, policy_version 713534 (0.0009) [2023-12-26 20:33:38,883][105692] Updated weights for policy 0, policy_version 713544 (0.0008) [2023-12-26 20:33:38,986][105620] Updated weights for policy 1, policy_version 714236 (0.0010) [2023-12-26 20:33:39,033][105620] Updated weights for policy 1, policy_version 714246 (0.0008) [2023-12-26 20:33:39,095][105620] Updated weights for policy 1, policy_version 714256 (0.0009) [2023-12-26 20:33:39,670][105692] Updated weights for policy 0, policy_version 713554 (0.0008) [2023-12-26 20:33:39,732][105692] Updated weights for policy 0, policy_version 713564 (0.0008) [2023-12-26 20:33:39,797][105692] Updated weights for policy 0, policy_version 713574 (0.0009) [2023-12-26 20:33:39,912][105620] Updated weights for policy 1, policy_version 714266 (0.0009) [2023-12-26 20:33:39,983][105620] Updated weights for policy 1, policy_version 714276 (0.0011) [2023-12-26 20:33:40,050][105620] Updated weights for policy 1, policy_version 714286 (0.0007) [2023-12-26 20:33:40,115][105620] Updated weights for policy 1, policy_version 714296 (0.0009) [2023-12-26 20:33:40,582][105692] Updated weights for policy 0, policy_version 713584 (0.0009) [2023-12-26 20:33:40,638][105692] Updated weights for policy 0, policy_version 713594 (0.0007) [2023-12-26 20:33:40,702][105692] Updated weights for policy 0, policy_version 713604 (0.0006) [2023-12-26 20:33:40,841][105620] Updated weights for policy 1, policy_version 714306 (0.0011) [2023-12-26 20:33:40,897][105620] Updated weights for policy 1, policy_version 714316 (0.0010) [2023-12-26 20:33:40,951][105620] Updated weights for policy 1, policy_version 714326 (0.0010) [2023-12-26 20:33:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 365600768. Throughput: 0: 9835.0, 1: 9725.0. Samples: 365605112. Policy #0 lag: (min: 14.0, avg: 21.6, max: 46.0) [2023-12-26 20:33:41,063][104569] Avg episode reward: [(0, '7326.549'), (1, '8457.488')] [2023-12-26 20:33:41,450][105692] Updated weights for policy 0, policy_version 713614 (0.0010) [2023-12-26 20:33:41,517][105692] Updated weights for policy 0, policy_version 713624 (0.0011) [2023-12-26 20:33:41,582][105692] Updated weights for policy 0, policy_version 713634 (0.0011) [2023-12-26 20:33:41,781][105620] Updated weights for policy 1, policy_version 714336 (0.0009) [2023-12-26 20:33:41,841][105620] Updated weights for policy 1, policy_version 714346 (0.0009) [2023-12-26 20:33:41,905][105620] Updated weights for policy 1, policy_version 714356 (0.0006) [2023-12-26 20:33:42,275][105692] Updated weights for policy 0, policy_version 713644 (0.0007) [2023-12-26 20:33:42,343][105692] Updated weights for policy 0, policy_version 713654 (0.0006) [2023-12-26 20:33:42,412][105692] Updated weights for policy 0, policy_version 713664 (0.0007) [2023-12-26 20:33:42,591][105620] Updated weights for policy 1, policy_version 714366 (0.0005) [2023-12-26 20:33:42,655][105620] Updated weights for policy 1, policy_version 714376 (0.0005) [2023-12-26 20:33:42,710][105620] Updated weights for policy 1, policy_version 714386 (0.0005) [2023-12-26 20:33:42,961][105692] Updated weights for policy 0, policy_version 713674 (0.0006) [2023-12-26 20:33:43,016][105692] Updated weights for policy 0, policy_version 713684 (0.0006) [2023-12-26 20:33:43,062][105692] Updated weights for policy 0, policy_version 713694 (0.0006) [2023-12-26 20:33:43,111][105692] Updated weights for policy 0, policy_version 713704 (0.0010) [2023-12-26 20:33:43,360][105620] Updated weights for policy 1, policy_version 714396 (0.0005) [2023-12-26 20:33:43,386][105586] KL-divergence is very high: 180.3040 [2023-12-26 20:33:43,412][105620] Updated weights for policy 1, policy_version 714406 (0.0005) [2023-12-26 20:33:43,432][105586] KL-divergence is very high: 273.7272 [2023-12-26 20:33:43,476][105620] Updated weights for policy 1, policy_version 714416 (0.0006) [2023-12-26 20:33:43,485][105586] KL-divergence is very high: 231.0037 [2023-12-26 20:33:43,792][105692] Updated weights for policy 0, policy_version 713714 (0.0005) [2023-12-26 20:33:43,855][105692] Updated weights for policy 0, policy_version 713724 (0.0009) [2023-12-26 20:33:43,906][105692] Updated weights for policy 0, policy_version 713734 (0.0010) [2023-12-26 20:33:44,075][105620] Updated weights for policy 1, policy_version 714426 (0.0007) [2023-12-26 20:33:44,133][105620] Updated weights for policy 1, policy_version 714436 (0.0007) [2023-12-26 20:33:44,194][105620] Updated weights for policy 1, policy_version 714446 (0.0010) [2023-12-26 20:33:44,252][105620] Updated weights for policy 1, policy_version 714456 (0.0010) [2023-12-26 20:33:44,631][105692] Updated weights for policy 0, policy_version 713744 (0.0008) [2023-12-26 20:33:44,679][105692] Updated weights for policy 0, policy_version 713754 (0.0008) [2023-12-26 20:33:44,730][105692] Updated weights for policy 0, policy_version 713764 (0.0008) [2023-12-26 20:33:44,949][105620] Updated weights for policy 1, policy_version 714466 (0.0011) [2023-12-26 20:33:45,009][105620] Updated weights for policy 1, policy_version 714476 (0.0011) [2023-12-26 20:33:45,072][105620] Updated weights for policy 1, policy_version 714486 (0.0010) [2023-12-26 20:33:45,548][105692] Updated weights for policy 0, policy_version 713774 (0.0008) [2023-12-26 20:33:45,593][105692] Updated weights for policy 0, policy_version 713784 (0.0008) [2023-12-26 20:33:45,643][105692] Updated weights for policy 0, policy_version 713794 (0.0008) [2023-12-26 20:33:45,830][105620] Updated weights for policy 1, policy_version 714496 (0.0007) [2023-12-26 20:33:45,893][105620] Updated weights for policy 1, policy_version 714506 (0.0005) [2023-12-26 20:33:45,962][105620] Updated weights for policy 1, policy_version 714516 (0.0006) [2023-12-26 20:33:46,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 365699072. Throughput: 0: 9840.6, 1: 9778.6. Samples: 365665620. Policy #0 lag: (min: 14.0, avg: 21.6, max: 46.0) [2023-12-26 20:33:46,063][104569] Avg episode reward: [(0, '5879.879'), (1, '8908.294')] [2023-12-26 20:33:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000714520_182935552.pth... [2023-12-26 20:33:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000713800_182763520.pth... [2023-12-26 20:33:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000713368_182640640.pth [2023-12-26 20:33:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000712648_182468608.pth [2023-12-26 20:33:46,401][105692] Updated weights for policy 0, policy_version 713804 (0.0009) [2023-12-26 20:33:46,454][105692] Updated weights for policy 0, policy_version 713814 (0.0010) [2023-12-26 20:33:46,503][105692] Updated weights for policy 0, policy_version 713824 (0.0009) [2023-12-26 20:33:46,504][105620] Updated weights for policy 1, policy_version 714526 (0.0005) [2023-12-26 20:33:46,550][105620] Updated weights for policy 1, policy_version 714536 (0.0005) [2023-12-26 20:33:46,607][105620] Updated weights for policy 1, policy_version 714546 (0.0005) [2023-12-26 20:33:47,286][105620] Updated weights for policy 1, policy_version 714556 (0.0006) [2023-12-26 20:33:47,329][105692] Updated weights for policy 0, policy_version 713834 (0.0008) [2023-12-26 20:33:47,345][105620] Updated weights for policy 1, policy_version 714566 (0.0007) [2023-12-26 20:33:47,399][105692] Updated weights for policy 0, policy_version 713844 (0.0008) [2023-12-26 20:33:47,401][105620] Updated weights for policy 1, policy_version 714576 (0.0007) [2023-12-26 20:33:47,463][105692] Updated weights for policy 0, policy_version 713854 (0.0009) [2023-12-26 20:33:47,532][105692] Updated weights for policy 0, policy_version 713864 (0.0006) [2023-12-26 20:33:47,995][105620] Updated weights for policy 1, policy_version 714586 (0.0007) [2023-12-26 20:33:48,046][105620] Updated weights for policy 1, policy_version 714596 (0.0005) [2023-12-26 20:33:48,083][105692] Updated weights for policy 0, policy_version 713874 (0.0008) [2023-12-26 20:33:48,100][105620] Updated weights for policy 1, policy_version 714606 (0.0005) [2023-12-26 20:33:48,132][105692] Updated weights for policy 0, policy_version 713884 (0.0009) [2023-12-26 20:33:48,162][105620] Updated weights for policy 1, policy_version 714616 (0.0007) [2023-12-26 20:33:48,192][105692] Updated weights for policy 0, policy_version 713894 (0.0005) [2023-12-26 20:33:48,861][105620] Updated weights for policy 1, policy_version 714626 (0.0011) [2023-12-26 20:33:48,882][105692] Updated weights for policy 0, policy_version 713904 (0.0009) [2023-12-26 20:33:48,924][105620] Updated weights for policy 1, policy_version 714636 (0.0011) [2023-12-26 20:33:48,930][105692] Updated weights for policy 0, policy_version 713914 (0.0010) [2023-12-26 20:33:48,979][105620] Updated weights for policy 1, policy_version 714646 (0.0010) [2023-12-26 20:33:48,979][105692] Updated weights for policy 0, policy_version 713924 (0.0010) [2023-12-26 20:33:49,721][105620] Updated weights for policy 1, policy_version 714656 (0.0010) [2023-12-26 20:33:49,773][105620] Updated weights for policy 1, policy_version 714666 (0.0010) [2023-12-26 20:33:49,826][105692] Updated weights for policy 0, policy_version 713934 (0.0010) [2023-12-26 20:33:49,842][105620] Updated weights for policy 1, policy_version 714676 (0.0011) [2023-12-26 20:33:49,892][105692] Updated weights for policy 0, policy_version 713944 (0.0009) [2023-12-26 20:33:49,957][105692] Updated weights for policy 0, policy_version 713954 (0.0010) [2023-12-26 20:33:50,614][105620] Updated weights for policy 1, policy_version 714686 (0.0009) [2023-12-26 20:33:50,675][105620] Updated weights for policy 1, policy_version 714696 (0.0011) [2023-12-26 20:33:50,690][105692] Updated weights for policy 0, policy_version 713964 (0.0009) [2023-12-26 20:33:50,742][105620] Updated weights for policy 1, policy_version 714706 (0.0011) [2023-12-26 20:33:50,754][105692] Updated weights for policy 0, policy_version 713974 (0.0006) [2023-12-26 20:33:50,821][105692] Updated weights for policy 0, policy_version 713984 (0.0006) [2023-12-26 20:33:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 365797376. Throughput: 0: 9918.7, 1: 9829.4. Samples: 365784024. Policy #0 lag: (min: 14.0, avg: 21.6, max: 46.0) [2023-12-26 20:33:51,062][104569] Avg episode reward: [(0, '7431.096'), (1, '8806.928')] [2023-12-26 20:33:51,498][105620] Updated weights for policy 1, policy_version 714716 (0.0009) [2023-12-26 20:33:51,535][105692] Updated weights for policy 0, policy_version 713994 (0.0007) [2023-12-26 20:33:51,563][105620] Updated weights for policy 1, policy_version 714726 (0.0009) [2023-12-26 20:33:51,601][105692] Updated weights for policy 0, policy_version 714004 (0.0009) [2023-12-26 20:33:51,612][105620] Updated weights for policy 1, policy_version 714736 (0.0010) [2023-12-26 20:33:51,660][105692] Updated weights for policy 0, policy_version 714014 (0.0008) [2023-12-26 20:33:51,730][105692] Updated weights for policy 0, policy_version 714024 (0.0009) [2023-12-26 20:33:52,357][105620] Updated weights for policy 1, policy_version 714746 (0.0010) [2023-12-26 20:33:52,377][105692] Updated weights for policy 0, policy_version 714034 (0.0008) [2023-12-26 20:33:52,417][105620] Updated weights for policy 1, policy_version 714756 (0.0008) [2023-12-26 20:33:52,436][105692] Updated weights for policy 0, policy_version 714044 (0.0008) [2023-12-26 20:33:52,478][105620] Updated weights for policy 1, policy_version 714766 (0.0010) [2023-12-26 20:33:52,493][105692] Updated weights for policy 0, policy_version 714054 (0.0006) [2023-12-26 20:33:52,536][105620] Updated weights for policy 1, policy_version 714776 (0.0009) [2023-12-26 20:33:53,218][105692] Updated weights for policy 0, policy_version 714064 (0.0006) [2023-12-26 20:33:53,244][105620] Updated weights for policy 1, policy_version 714786 (0.0010) [2023-12-26 20:33:53,277][105692] Updated weights for policy 0, policy_version 714074 (0.0005) [2023-12-26 20:33:53,301][105620] Updated weights for policy 1, policy_version 714796 (0.0011) [2023-12-26 20:33:53,343][105692] Updated weights for policy 0, policy_version 714084 (0.0006) [2023-12-26 20:33:53,360][105620] Updated weights for policy 1, policy_version 714806 (0.0010) [2023-12-26 20:33:53,955][105620] Updated weights for policy 1, policy_version 714816 (0.0006) [2023-12-26 20:33:53,990][105692] Updated weights for policy 0, policy_version 714094 (0.0006) [2023-12-26 20:33:54,006][105620] Updated weights for policy 1, policy_version 714826 (0.0005) [2023-12-26 20:33:54,037][105692] Updated weights for policy 0, policy_version 714104 (0.0005) [2023-12-26 20:33:54,053][105620] Updated weights for policy 1, policy_version 714836 (0.0005) [2023-12-26 20:33:54,088][105692] Updated weights for policy 0, policy_version 714114 (0.0006) [2023-12-26 20:33:54,660][105692] Updated weights for policy 0, policy_version 714124 (0.0008) [2023-12-26 20:33:54,698][105620] Updated weights for policy 1, policy_version 714846 (0.0006) [2023-12-26 20:33:54,725][105692] Updated weights for policy 0, policy_version 714134 (0.0006) [2023-12-26 20:33:54,760][105620] Updated weights for policy 1, policy_version 714856 (0.0008) [2023-12-26 20:33:54,776][105692] Updated weights for policy 0, policy_version 714144 (0.0005) [2023-12-26 20:33:54,826][105620] Updated weights for policy 1, policy_version 714866 (0.0006) [2023-12-26 20:33:55,312][105692] Updated weights for policy 0, policy_version 714154 (0.0007) [2023-12-26 20:33:55,360][105692] Updated weights for policy 0, policy_version 714164 (0.0010) [2023-12-26 20:33:55,418][105692] Updated weights for policy 0, policy_version 714174 (0.0010) [2023-12-26 20:33:55,462][105692] Updated weights for policy 0, policy_version 714184 (0.0010) [2023-12-26 20:33:55,500][105620] Updated weights for policy 1, policy_version 714876 (0.0007) [2023-12-26 20:33:55,557][105620] Updated weights for policy 1, policy_version 714886 (0.0007) [2023-12-26 20:33:55,617][105620] Updated weights for policy 1, policy_version 714896 (0.0009) [2023-12-26 20:33:56,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 365895680. Throughput: 0: 9917.4, 1: 9869.8. Samples: 365904628. Policy #0 lag: (min: 9.0, avg: 20.3, max: 41.0) [2023-12-26 20:33:56,063][104569] Avg episode reward: [(0, '8980.231'), (1, '8912.021')] [2023-12-26 20:33:56,082][105692] Updated weights for policy 0, policy_version 714194 (0.0005) [2023-12-26 20:33:56,144][105692] Updated weights for policy 0, policy_version 714204 (0.0005) [2023-12-26 20:33:56,198][105692] Updated weights for policy 0, policy_version 714214 (0.0005) [2023-12-26 20:33:56,355][105620] Updated weights for policy 1, policy_version 714906 (0.0009) [2023-12-26 20:33:56,414][105620] Updated weights for policy 1, policy_version 714916 (0.0006) [2023-12-26 20:33:56,462][105620] Updated weights for policy 1, policy_version 714926 (0.0006) [2023-12-26 20:33:56,513][105620] Updated weights for policy 1, policy_version 714936 (0.0005) [2023-12-26 20:33:56,864][105692] Updated weights for policy 0, policy_version 714224 (0.0005) [2023-12-26 20:33:56,916][105692] Updated weights for policy 0, policy_version 714234 (0.0006) [2023-12-26 20:33:56,962][105692] Updated weights for policy 0, policy_version 714244 (0.0007) [2023-12-26 20:33:57,116][105620] Updated weights for policy 1, policy_version 714946 (0.0007) [2023-12-26 20:33:57,160][105620] Updated weights for policy 1, policy_version 714956 (0.0010) [2023-12-26 20:33:57,215][105620] Updated weights for policy 1, policy_version 714966 (0.0007) [2023-12-26 20:33:57,603][105692] Updated weights for policy 0, policy_version 714254 (0.0008) [2023-12-26 20:33:57,660][105692] Updated weights for policy 0, policy_version 714264 (0.0010) [2023-12-26 20:33:57,711][105692] Updated weights for policy 0, policy_version 714274 (0.0010) [2023-12-26 20:33:57,788][105620] Updated weights for policy 1, policy_version 714976 (0.0010) [2023-12-26 20:33:57,835][105620] Updated weights for policy 1, policy_version 714986 (0.0007) [2023-12-26 20:33:57,887][105620] Updated weights for policy 1, policy_version 714996 (0.0010) [2023-12-26 20:33:58,454][105692] Updated weights for policy 0, policy_version 714284 (0.0010) [2023-12-26 20:33:58,519][105692] Updated weights for policy 0, policy_version 714294 (0.0009) [2023-12-26 20:33:58,578][105692] Updated weights for policy 0, policy_version 714304 (0.0006) [2023-12-26 20:33:58,677][105620] Updated weights for policy 1, policy_version 715006 (0.0009) [2023-12-26 20:33:58,740][105620] Updated weights for policy 1, policy_version 715016 (0.0008) [2023-12-26 20:33:58,813][105620] Updated weights for policy 1, policy_version 715026 (0.0010) [2023-12-26 20:33:59,294][105692] Updated weights for policy 0, policy_version 714314 (0.0008) [2023-12-26 20:33:59,356][105692] Updated weights for policy 0, policy_version 714324 (0.0010) [2023-12-26 20:33:59,417][105692] Updated weights for policy 0, policy_version 714334 (0.0009) [2023-12-26 20:33:59,466][105692] Updated weights for policy 0, policy_version 714344 (0.0009) [2023-12-26 20:33:59,495][105620] Updated weights for policy 1, policy_version 715036 (0.0011) [2023-12-26 20:33:59,547][105620] Updated weights for policy 1, policy_version 715046 (0.0009) [2023-12-26 20:33:59,587][105586] KL-divergence is very high: 165.9013 [2023-12-26 20:33:59,604][105620] Updated weights for policy 1, policy_version 715056 (0.0009) [2023-12-26 20:33:59,633][105586] KL-divergence is very high: 171.8839 [2023-12-26 20:34:00,274][105692] Updated weights for policy 0, policy_version 714354 (0.0006) [2023-12-26 20:34:00,343][105692] Updated weights for policy 0, policy_version 714364 (0.0005) [2023-12-26 20:34:00,359][105620] Updated weights for policy 1, policy_version 715066 (0.0008) [2023-12-26 20:34:00,406][105692] Updated weights for policy 0, policy_version 714374 (0.0009) [2023-12-26 20:34:00,421][105620] Updated weights for policy 1, policy_version 715076 (0.0006) [2023-12-26 20:34:00,485][105620] Updated weights for policy 1, policy_version 715086 (0.0005) [2023-12-26 20:34:00,538][105620] Updated weights for policy 1, policy_version 715096 (0.0009) [2023-12-26 20:34:01,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 365993984. Throughput: 0: 9965.7, 1: 9912.5. Samples: 365967804. Policy #0 lag: (min: 9.0, avg: 20.3, max: 41.0) [2023-12-26 20:34:01,063][104569] Avg episode reward: [(0, '8926.233'), (1, '8368.795')] [2023-12-26 20:34:01,121][105692] Updated weights for policy 0, policy_version 714384 (0.0008) [2023-12-26 20:34:01,141][105620] Updated weights for policy 1, policy_version 715106 (0.0007) [2023-12-26 20:34:01,183][105692] Updated weights for policy 0, policy_version 714394 (0.0007) [2023-12-26 20:34:01,204][105620] Updated weights for policy 1, policy_version 715116 (0.0009) [2023-12-26 20:34:01,247][105692] Updated weights for policy 0, policy_version 714404 (0.0006) [2023-12-26 20:34:01,264][105620] Updated weights for policy 1, policy_version 715126 (0.0007) [2023-12-26 20:34:01,269][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000714408_182919168.pth... [2023-12-26 20:34:01,273][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000713224_182616064.pth [2023-12-26 20:34:01,275][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000715128_183091200.pth... [2023-12-26 20:34:01,278][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000713976_182796288.pth [2023-12-26 20:34:01,948][105620] Updated weights for policy 1, policy_version 715136 (0.0006) [2023-12-26 20:34:02,001][105620] Updated weights for policy 1, policy_version 715146 (0.0008) [2023-12-26 20:34:02,059][105620] Updated weights for policy 1, policy_version 715156 (0.0005) [2023-12-26 20:34:02,078][105692] Updated weights for policy 0, policy_version 714414 (0.0008) [2023-12-26 20:34:02,133][105692] Updated weights for policy 0, policy_version 714424 (0.0009) [2023-12-26 20:34:02,189][105692] Updated weights for policy 0, policy_version 714434 (0.0010) [2023-12-26 20:34:02,756][105620] Updated weights for policy 1, policy_version 715166 (0.0008) [2023-12-26 20:34:02,815][105620] Updated weights for policy 1, policy_version 715176 (0.0008) [2023-12-26 20:34:02,821][105692] Updated weights for policy 0, policy_version 714444 (0.0009) [2023-12-26 20:34:02,874][105692] Updated weights for policy 0, policy_version 714454 (0.0009) [2023-12-26 20:34:02,878][105620] Updated weights for policy 1, policy_version 715186 (0.0006) [2023-12-26 20:34:02,932][105692] Updated weights for policy 0, policy_version 714464 (0.0008) [2023-12-26 20:34:03,423][105620] Updated weights for policy 1, policy_version 715196 (0.0005) [2023-12-26 20:34:03,479][105620] Updated weights for policy 1, policy_version 715206 (0.0005) [2023-12-26 20:34:03,492][105692] Updated weights for policy 0, policy_version 714474 (0.0006) [2023-12-26 20:34:03,529][105620] Updated weights for policy 1, policy_version 715216 (0.0005) [2023-12-26 20:34:03,541][105692] Updated weights for policy 0, policy_version 714484 (0.0005) [2023-12-26 20:34:03,584][105692] Updated weights for policy 0, policy_version 714494 (0.0005) [2023-12-26 20:34:03,638][105692] Updated weights for policy 0, policy_version 714504 (0.0009) [2023-12-26 20:34:04,061][105620] Updated weights for policy 1, policy_version 715226 (0.0005) [2023-12-26 20:34:04,109][105620] Updated weights for policy 1, policy_version 715236 (0.0005) [2023-12-26 20:34:04,162][105620] Updated weights for policy 1, policy_version 715246 (0.0006) [2023-12-26 20:34:04,226][105620] Updated weights for policy 1, policy_version 715256 (0.0005) [2023-12-26 20:34:04,345][105692] Updated weights for policy 0, policy_version 714514 (0.0011) [2023-12-26 20:34:04,407][105692] Updated weights for policy 0, policy_version 714524 (0.0011) [2023-12-26 20:34:04,477][105692] Updated weights for policy 0, policy_version 714534 (0.0011) [2023-12-26 20:34:04,779][105620] Updated weights for policy 1, policy_version 715266 (0.0007) [2023-12-26 20:34:04,826][105620] Updated weights for policy 1, policy_version 715276 (0.0006) [2023-12-26 20:34:04,876][105620] Updated weights for policy 1, policy_version 715286 (0.0008) [2023-12-26 20:34:05,214][105692] Updated weights for policy 0, policy_version 714544 (0.0010) [2023-12-26 20:34:05,272][105692] Updated weights for policy 0, policy_version 714554 (0.0010) [2023-12-26 20:34:05,327][105692] Updated weights for policy 0, policy_version 714564 (0.0010) [2023-12-26 20:34:05,581][105620] Updated weights for policy 1, policy_version 715296 (0.0009) [2023-12-26 20:34:05,635][105620] Updated weights for policy 1, policy_version 715306 (0.0009) [2023-12-26 20:34:05,688][105620] Updated weights for policy 1, policy_version 715316 (0.0010) [2023-12-26 20:34:05,967][105692] Updated weights for policy 0, policy_version 714574 (0.0007) [2023-12-26 20:34:06,040][105692] Updated weights for policy 0, policy_version 714584 (0.0005) [2023-12-26 20:34:06,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 366100480. Throughput: 0: 10011.7, 1: 10031.8. Samples: 366092124. Policy #0 lag: (min: 9.0, avg: 20.3, max: 41.0) [2023-12-26 20:34:06,063][104569] Avg episode reward: [(0, '7148.364'), (1, '8268.127')] [2023-12-26 20:34:06,110][105692] Updated weights for policy 0, policy_version 714594 (0.0007) [2023-12-26 20:34:06,538][105620] Updated weights for policy 1, policy_version 715326 (0.0007) [2023-12-26 20:34:06,602][105620] Updated weights for policy 1, policy_version 715336 (0.0006) [2023-12-26 20:34:06,664][105620] Updated weights for policy 1, policy_version 715346 (0.0009) [2023-12-26 20:34:06,706][105692] Updated weights for policy 0, policy_version 714604 (0.0008) [2023-12-26 20:34:06,755][105692] Updated weights for policy 0, policy_version 714614 (0.0005) [2023-12-26 20:34:06,826][105692] Updated weights for policy 0, policy_version 714624 (0.0006) [2023-12-26 20:34:07,356][105692] Updated weights for policy 0, policy_version 714634 (0.0007) [2023-12-26 20:34:07,418][105692] Updated weights for policy 0, policy_version 714644 (0.0010) [2023-12-26 20:34:07,453][105620] Updated weights for policy 1, policy_version 715356 (0.0009) [2023-12-26 20:34:07,475][105692] Updated weights for policy 0, policy_version 714654 (0.0007) [2023-12-26 20:34:07,509][105620] Updated weights for policy 1, policy_version 715366 (0.0008) [2023-12-26 20:34:07,533][105692] Updated weights for policy 0, policy_version 714664 (0.0008) [2023-12-26 20:34:07,556][105620] Updated weights for policy 1, policy_version 715376 (0.0006) [2023-12-26 20:34:08,158][105692] Updated weights for policy 0, policy_version 714674 (0.0011) [2023-12-26 20:34:08,172][105620] Updated weights for policy 1, policy_version 715386 (0.0008) [2023-12-26 20:34:08,211][105692] Updated weights for policy 0, policy_version 714684 (0.0007) [2023-12-26 20:34:08,233][105620] Updated weights for policy 1, policy_version 715396 (0.0008) [2023-12-26 20:34:08,260][105692] Updated weights for policy 0, policy_version 714694 (0.0005) [2023-12-26 20:34:08,285][105620] Updated weights for policy 1, policy_version 715406 (0.0008) [2023-12-26 20:34:08,357][105620] Updated weights for policy 1, policy_version 715416 (0.0008) [2023-12-26 20:34:09,006][105692] Updated weights for policy 0, policy_version 714704 (0.0006) [2023-12-26 20:34:09,019][105620] Updated weights for policy 1, policy_version 715426 (0.0008) [2023-12-26 20:34:09,060][105692] Updated weights for policy 0, policy_version 714714 (0.0005) [2023-12-26 20:34:09,075][105620] Updated weights for policy 1, policy_version 715436 (0.0009) [2023-12-26 20:34:09,125][105692] Updated weights for policy 0, policy_version 714724 (0.0005) [2023-12-26 20:34:09,134][105620] Updated weights for policy 1, policy_version 715446 (0.0010) [2023-12-26 20:34:09,875][105692] Updated weights for policy 0, policy_version 714734 (0.0006) [2023-12-26 20:34:09,881][105620] Updated weights for policy 1, policy_version 715456 (0.0010) [2023-12-26 20:34:09,941][105692] Updated weights for policy 0, policy_version 714744 (0.0007) [2023-12-26 20:34:09,948][105620] Updated weights for policy 1, policy_version 715466 (0.0009) [2023-12-26 20:34:10,005][105620] Updated weights for policy 1, policy_version 715476 (0.0007) [2023-12-26 20:34:10,036][105692] Updated weights for policy 0, policy_version 714754 (0.0009) [2023-12-26 20:34:10,737][105692] Updated weights for policy 0, policy_version 714764 (0.0009) [2023-12-26 20:34:10,769][105620] Updated weights for policy 1, policy_version 715486 (0.0007) [2023-12-26 20:34:10,788][105692] Updated weights for policy 0, policy_version 714774 (0.0006) [2023-12-26 20:34:10,823][105586] KL-divergence is very high: 129.8990 [2023-12-26 20:34:10,824][105620] Updated weights for policy 1, policy_version 715496 (0.0008) [2023-12-26 20:34:10,844][105692] Updated weights for policy 0, policy_version 714784 (0.0009) [2023-12-26 20:34:10,875][105586] KL-divergence is very high: 131.3958 [2023-12-26 20:34:10,887][105620] Updated weights for policy 1, policy_version 715506 (0.0006) [2023-12-26 20:34:11,062][104569] Fps is (10 sec: 21299.6, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 366206976. Throughput: 0: 10110.0, 1: 9977.1. Samples: 366210856. Policy #0 lag: (min: 9.0, avg: 20.3, max: 41.0) [2023-12-26 20:34:11,062][104569] Avg episode reward: [(0, '7403.836'), (1, '8452.275')] [2023-12-26 20:34:11,583][105620] Updated weights for policy 1, policy_version 715516 (0.0007) [2023-12-26 20:34:11,640][105620] Updated weights for policy 1, policy_version 715526 (0.0009) [2023-12-26 20:34:11,705][105620] Updated weights for policy 1, policy_version 715536 (0.0009) [2023-12-26 20:34:11,713][105692] Updated weights for policy 0, policy_version 714794 (0.0008) [2023-12-26 20:34:11,786][105692] Updated weights for policy 0, policy_version 714804 (0.0007) [2023-12-26 20:34:11,843][105692] Updated weights for policy 0, policy_version 714814 (0.0006) [2023-12-26 20:34:11,900][105692] Updated weights for policy 0, policy_version 714824 (0.0006) [2023-12-26 20:34:12,536][105620] Updated weights for policy 1, policy_version 715546 (0.0008) [2023-12-26 20:34:12,575][105692] Updated weights for policy 0, policy_version 714834 (0.0007) [2023-12-26 20:34:12,594][105620] Updated weights for policy 1, policy_version 715556 (0.0007) [2023-12-26 20:34:12,634][105692] Updated weights for policy 0, policy_version 714844 (0.0006) [2023-12-26 20:34:12,656][105620] Updated weights for policy 1, policy_version 715566 (0.0005) [2023-12-26 20:34:12,692][105692] Updated weights for policy 0, policy_version 714854 (0.0006) [2023-12-26 20:34:12,722][105620] Updated weights for policy 1, policy_version 715576 (0.0005) [2023-12-26 20:34:13,296][105692] Updated weights for policy 0, policy_version 714864 (0.0008) [2023-12-26 20:34:13,360][105692] Updated weights for policy 0, policy_version 714874 (0.0009) [2023-12-26 20:34:13,408][105620] Updated weights for policy 1, policy_version 715586 (0.0007) [2023-12-26 20:34:13,415][105692] Updated weights for policy 0, policy_version 714884 (0.0006) [2023-12-26 20:34:13,458][105620] Updated weights for policy 1, policy_version 715596 (0.0009) [2023-12-26 20:34:13,510][105620] Updated weights for policy 1, policy_version 715606 (0.0009) [2023-12-26 20:34:14,020][105692] Updated weights for policy 0, policy_version 714894 (0.0005) [2023-12-26 20:34:14,081][105692] Updated weights for policy 0, policy_version 714904 (0.0005) [2023-12-26 20:34:14,140][105692] Updated weights for policy 0, policy_version 714914 (0.0006) [2023-12-26 20:34:14,341][105620] Updated weights for policy 1, policy_version 715616 (0.0010) [2023-12-26 20:34:14,399][105620] Updated weights for policy 1, policy_version 715626 (0.0009) [2023-12-26 20:34:14,452][105620] Updated weights for policy 1, policy_version 715636 (0.0008) [2023-12-26 20:34:14,825][105692] Updated weights for policy 0, policy_version 714924 (0.0008) [2023-12-26 20:34:14,884][105692] Updated weights for policy 0, policy_version 714934 (0.0009) [2023-12-26 20:34:14,944][105692] Updated weights for policy 0, policy_version 714944 (0.0010) [2023-12-26 20:34:15,224][105620] Updated weights for policy 1, policy_version 715646 (0.0010) [2023-12-26 20:34:15,288][105620] Updated weights for policy 1, policy_version 715656 (0.0009) [2023-12-26 20:34:15,350][105620] Updated weights for policy 1, policy_version 715666 (0.0009) [2023-12-26 20:34:15,692][105692] Updated weights for policy 0, policy_version 714954 (0.0009) [2023-12-26 20:34:15,751][105692] Updated weights for policy 0, policy_version 714964 (0.0009) [2023-12-26 20:34:15,800][105692] Updated weights for policy 0, policy_version 714974 (0.0009) [2023-12-26 20:34:15,857][105692] Updated weights for policy 0, policy_version 714984 (0.0008) [2023-12-26 20:34:16,059][105620] Updated weights for policy 1, policy_version 715676 (0.0009) [2023-12-26 20:34:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 366297088. Throughput: 0: 10031.4, 1: 9854.4. Samples: 366267660. Policy #0 lag: (min: 9.0, avg: 20.3, max: 41.0) [2023-12-26 20:34:16,062][104569] Avg episode reward: [(0, '9351.233'), (1, '8454.470')] [2023-12-26 20:34:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000714984_183066624.pth... [2023-12-26 20:34:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000713800_182763520.pth [2023-12-26 20:34:16,121][105620] Updated weights for policy 1, policy_version 715686 (0.0009) [2023-12-26 20:34:16,190][105620] Updated weights for policy 1, policy_version 715696 (0.0009) [2023-12-26 20:34:16,243][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000715704_183238656.pth... [2023-12-26 20:34:16,247][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000714520_182935552.pth [2023-12-26 20:34:16,492][105692] Updated weights for policy 0, policy_version 714994 (0.0006) [2023-12-26 20:34:16,564][105692] Updated weights for policy 0, policy_version 715004 (0.0005) [2023-12-26 20:34:16,626][105692] Updated weights for policy 0, policy_version 715014 (0.0005) [2023-12-26 20:34:17,075][105620] Updated weights for policy 1, policy_version 715706 (0.0009) [2023-12-26 20:34:17,131][105620] Updated weights for policy 1, policy_version 715716 (0.0009) [2023-12-26 20:34:17,167][105692] Updated weights for policy 0, policy_version 715024 (0.0005) [2023-12-26 20:34:17,186][105620] Updated weights for policy 1, policy_version 715726 (0.0009) [2023-12-26 20:34:17,218][105692] Updated weights for policy 0, policy_version 715034 (0.0006) [2023-12-26 20:34:17,248][105620] Updated weights for policy 1, policy_version 715736 (0.0006) [2023-12-26 20:34:17,272][105692] Updated weights for policy 0, policy_version 715044 (0.0008) [2023-12-26 20:34:17,929][105692] Updated weights for policy 0, policy_version 715054 (0.0007) [2023-12-26 20:34:17,995][105692] Updated weights for policy 0, policy_version 715064 (0.0005) [2023-12-26 20:34:18,042][105620] Updated weights for policy 1, policy_version 715746 (0.0008) [2023-12-26 20:34:18,055][105692] Updated weights for policy 0, policy_version 715074 (0.0006) [2023-12-26 20:34:18,107][105620] Updated weights for policy 1, policy_version 715756 (0.0005) [2023-12-26 20:34:18,173][105620] Updated weights for policy 1, policy_version 715766 (0.0005) [2023-12-26 20:34:18,610][105692] Updated weights for policy 0, policy_version 715084 (0.0005) [2023-12-26 20:34:18,672][105692] Updated weights for policy 0, policy_version 715094 (0.0005) [2023-12-26 20:34:18,738][105692] Updated weights for policy 0, policy_version 715104 (0.0005) [2023-12-26 20:34:18,935][105620] Updated weights for policy 1, policy_version 715776 (0.0009) [2023-12-26 20:34:18,996][105620] Updated weights for policy 1, policy_version 715786 (0.0009) [2023-12-26 20:34:19,055][105620] Updated weights for policy 1, policy_version 715796 (0.0009) [2023-12-26 20:34:19,267][105692] Updated weights for policy 0, policy_version 715114 (0.0005) [2023-12-26 20:34:19,322][105692] Updated weights for policy 0, policy_version 715124 (0.0005) [2023-12-26 20:34:19,387][105692] Updated weights for policy 0, policy_version 715134 (0.0008) [2023-12-26 20:34:19,448][105692] Updated weights for policy 0, policy_version 715144 (0.0009) [2023-12-26 20:34:19,898][105620] Updated weights for policy 1, policy_version 715806 (0.0009) [2023-12-26 20:34:19,965][105620] Updated weights for policy 1, policy_version 715816 (0.0010) [2023-12-26 20:34:20,025][105620] Updated weights for policy 1, policy_version 715826 (0.0009) [2023-12-26 20:34:20,162][105692] Updated weights for policy 0, policy_version 715154 (0.0010) [2023-12-26 20:34:20,223][105692] Updated weights for policy 0, policy_version 715164 (0.0011) [2023-12-26 20:34:20,280][105692] Updated weights for policy 0, policy_version 715174 (0.0011) [2023-12-26 20:34:20,837][105620] Updated weights for policy 1, policy_version 715836 (0.0010) [2023-12-26 20:34:20,905][105620] Updated weights for policy 1, policy_version 715846 (0.0009) [2023-12-26 20:34:20,963][105620] Updated weights for policy 1, policy_version 715856 (0.0009) [2023-12-26 20:34:21,022][105692] Updated weights for policy 0, policy_version 715184 (0.0009) [2023-12-26 20:34:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 366395392. Throughput: 0: 10088.2, 1: 9735.1. Samples: 366386392. Policy #0 lag: (min: 9.0, avg: 20.3, max: 41.0) [2023-12-26 20:34:21,063][104569] Avg episode reward: [(0, '9351.082'), (1, '8727.109')] [2023-12-26 20:34:21,091][105692] Updated weights for policy 0, policy_version 715194 (0.0006) [2023-12-26 20:34:21,158][105692] Updated weights for policy 0, policy_version 715204 (0.0006) [2023-12-26 20:34:21,816][105620] Updated weights for policy 1, policy_version 715866 (0.0007) [2023-12-26 20:34:21,864][105692] Updated weights for policy 0, policy_version 715214 (0.0007) [2023-12-26 20:34:21,875][105620] Updated weights for policy 1, policy_version 715876 (0.0007) [2023-12-26 20:34:21,921][105692] Updated weights for policy 0, policy_version 715224 (0.0009) [2023-12-26 20:34:21,944][105620] Updated weights for policy 1, policy_version 715886 (0.0006) [2023-12-26 20:34:21,970][105692] Updated weights for policy 0, policy_version 715234 (0.0009) [2023-12-26 20:34:22,011][105620] Updated weights for policy 1, policy_version 715896 (0.0006) [2023-12-26 20:34:22,657][105620] Updated weights for policy 1, policy_version 715906 (0.0009) [2023-12-26 20:34:22,716][105620] Updated weights for policy 1, policy_version 715916 (0.0008) [2023-12-26 20:34:22,778][105620] Updated weights for policy 1, policy_version 715926 (0.0009) [2023-12-26 20:34:22,798][105692] Updated weights for policy 0, policy_version 715244 (0.0009) [2023-12-26 20:34:22,865][105692] Updated weights for policy 0, policy_version 715254 (0.0005) [2023-12-26 20:34:22,921][105692] Updated weights for policy 0, policy_version 715264 (0.0009) [2023-12-26 20:34:23,504][105620] Updated weights for policy 1, policy_version 715936 (0.0006) [2023-12-26 20:34:23,564][105620] Updated weights for policy 1, policy_version 715946 (0.0008) [2023-12-26 20:34:23,611][105692] Updated weights for policy 0, policy_version 715274 (0.0011) [2023-12-26 20:34:23,618][105620] Updated weights for policy 1, policy_version 715956 (0.0008) [2023-12-26 20:34:23,665][105692] Updated weights for policy 0, policy_version 715284 (0.0009) [2023-12-26 20:34:23,720][105692] Updated weights for policy 0, policy_version 715294 (0.0010) [2023-12-26 20:34:23,771][105692] Updated weights for policy 0, policy_version 715304 (0.0009) [2023-12-26 20:34:24,329][105620] Updated weights for policy 1, policy_version 715966 (0.0009) [2023-12-26 20:34:24,390][105620] Updated weights for policy 1, policy_version 715976 (0.0010) [2023-12-26 20:34:24,444][105620] Updated weights for policy 1, policy_version 715986 (0.0009) [2023-12-26 20:34:24,455][105692] Updated weights for policy 0, policy_version 715314 (0.0006) [2023-12-26 20:34:24,518][105692] Updated weights for policy 0, policy_version 715324 (0.0011) [2023-12-26 20:34:24,574][105692] Updated weights for policy 0, policy_version 715334 (0.0011) [2023-12-26 20:34:25,153][105620] Updated weights for policy 1, policy_version 715996 (0.0006) [2023-12-26 20:34:25,203][105620] Updated weights for policy 1, policy_version 716006 (0.0005) [2023-12-26 20:34:25,255][105620] Updated weights for policy 1, policy_version 716016 (0.0005) [2023-12-26 20:34:25,260][105692] Updated weights for policy 0, policy_version 715344 (0.0010) [2023-12-26 20:34:25,315][105692] Updated weights for policy 0, policy_version 715354 (0.0010) [2023-12-26 20:34:25,363][105692] Updated weights for policy 0, policy_version 715364 (0.0010) [2023-12-26 20:34:25,935][105620] Updated weights for policy 1, policy_version 716026 (0.0006) [2023-12-26 20:34:25,990][105620] Updated weights for policy 1, policy_version 716037 (0.0011) [2023-12-26 20:34:26,040][105620] Updated weights for policy 1, policy_version 716048 (0.0008) [2023-12-26 20:34:26,051][105692] Updated weights for policy 0, policy_version 715374 (0.0007) [2023-12-26 20:34:26,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 366485504. Throughput: 0: 10104.3, 1: 9823.7. Samples: 366501876. Policy #0 lag: (min: 9.0, avg: 20.3, max: 41.0) [2023-12-26 20:34:26,063][104569] Avg episode reward: [(0, '9351.457'), (1, '8917.896')] [2023-12-26 20:34:26,104][105692] Updated weights for policy 0, policy_version 715384 (0.0005) [2023-12-26 20:34:26,150][105692] Updated weights for policy 0, policy_version 715394 (0.0005) [2023-12-26 20:34:26,681][105692] Updated weights for policy 0, policy_version 715404 (0.0005) [2023-12-26 20:34:26,741][105692] Updated weights for policy 0, policy_version 715414 (0.0005) [2023-12-26 20:34:26,793][105692] Updated weights for policy 0, policy_version 715424 (0.0005) [2023-12-26 20:34:26,942][105620] Updated weights for policy 1, policy_version 716058 (0.0008) [2023-12-26 20:34:27,000][105620] Updated weights for policy 1, policy_version 716068 (0.0008) [2023-12-26 20:34:27,053][105620] Updated weights for policy 1, policy_version 716078 (0.0009) [2023-12-26 20:34:27,105][105620] Updated weights for policy 1, policy_version 716088 (0.0009) [2023-12-26 20:34:27,371][105692] Updated weights for policy 0, policy_version 715434 (0.0006) [2023-12-26 20:34:27,424][105692] Updated weights for policy 0, policy_version 715444 (0.0011) [2023-12-26 20:34:27,483][105692] Updated weights for policy 0, policy_version 715454 (0.0011) [2023-12-26 20:34:27,539][105692] Updated weights for policy 0, policy_version 715464 (0.0011) [2023-12-26 20:34:27,890][105620] Updated weights for policy 1, policy_version 716098 (0.0008) [2023-12-26 20:34:27,951][105620] Updated weights for policy 1, policy_version 716108 (0.0009) [2023-12-26 20:34:28,016][105620] Updated weights for policy 1, policy_version 716118 (0.0009) [2023-12-26 20:34:28,209][105692] Updated weights for policy 0, policy_version 715474 (0.0007) [2023-12-26 20:34:28,262][105692] Updated weights for policy 0, policy_version 715484 (0.0005) [2023-12-26 20:34:28,319][105692] Updated weights for policy 0, policy_version 715494 (0.0008) [2023-12-26 20:34:28,832][105620] Updated weights for policy 1, policy_version 716128 (0.0008) [2023-12-26 20:34:28,880][105620] Updated weights for policy 1, policy_version 716138 (0.0008) [2023-12-26 20:34:28,933][105620] Updated weights for policy 1, policy_version 716148 (0.0008) [2023-12-26 20:34:28,987][105692] Updated weights for policy 0, policy_version 715504 (0.0010) [2023-12-26 20:34:29,049][105692] Updated weights for policy 0, policy_version 715514 (0.0011) [2023-12-26 20:34:29,110][105692] Updated weights for policy 0, policy_version 715524 (0.0011) [2023-12-26 20:34:29,736][105620] Updated weights for policy 1, policy_version 716158 (0.0009) [2023-12-26 20:34:29,783][105620] Updated weights for policy 1, policy_version 716168 (0.0007) [2023-12-26 20:34:29,785][105692] Updated weights for policy 0, policy_version 715534 (0.0008) [2023-12-26 20:34:29,848][105620] Updated weights for policy 1, policy_version 716178 (0.0009) [2023-12-26 20:34:29,848][105692] Updated weights for policy 0, policy_version 715544 (0.0008) [2023-12-26 20:34:29,908][105692] Updated weights for policy 0, policy_version 715554 (0.0008) [2023-12-26 20:34:30,610][105692] Updated weights for policy 0, policy_version 715564 (0.0008) [2023-12-26 20:34:30,655][105620] Updated weights for policy 1, policy_version 716188 (0.0008) [2023-12-26 20:34:30,658][105692] Updated weights for policy 0, policy_version 715574 (0.0007) [2023-12-26 20:34:30,706][105692] Updated weights for policy 0, policy_version 715584 (0.0007) [2023-12-26 20:34:30,713][105620] Updated weights for policy 1, policy_version 716198 (0.0009) [2023-12-26 20:34:30,768][105620] Updated weights for policy 1, policy_version 716208 (0.0007) [2023-12-26 20:34:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 366592000. Throughput: 0: 10182.4, 1: 9721.8. Samples: 366561300. Policy #0 lag: (min: 9.0, avg: 20.3, max: 41.0) [2023-12-26 20:34:31,062][104569] Avg episode reward: [(0, '9262.459'), (1, '8652.206')] [2023-12-26 20:34:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000716216_183369728.pth... [2023-12-26 20:34:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000715592_183222272.pth... [2023-12-26 20:34:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000715128_183091200.pth [2023-12-26 20:34:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000714408_182919168.pth [2023-12-26 20:34:31,361][105692] Updated weights for policy 0, policy_version 715594 (0.0007) [2023-12-26 20:34:31,433][105692] Updated weights for policy 0, policy_version 715604 (0.0007) [2023-12-26 20:34:31,500][105692] Updated weights for policy 0, policy_version 715614 (0.0008) [2023-12-26 20:34:31,558][105620] Updated weights for policy 1, policy_version 716218 (0.0008) [2023-12-26 20:34:31,563][105692] Updated weights for policy 0, policy_version 715624 (0.0010) [2023-12-26 20:34:31,615][105620] Updated weights for policy 1, policy_version 716228 (0.0008) [2023-12-26 20:34:31,683][105620] Updated weights for policy 1, policy_version 716238 (0.0007) [2023-12-26 20:34:31,741][105620] Updated weights for policy 1, policy_version 716248 (0.0008) [2023-12-26 20:34:32,247][105692] Updated weights for policy 0, policy_version 715634 (0.0006) [2023-12-26 20:34:32,309][105692] Updated weights for policy 0, policy_version 715644 (0.0006) [2023-12-26 20:34:32,378][105692] Updated weights for policy 0, policy_version 715654 (0.0006) [2023-12-26 20:34:32,483][105620] Updated weights for policy 1, policy_version 716258 (0.0006) [2023-12-26 20:34:32,534][105620] Updated weights for policy 1, policy_version 716268 (0.0008) [2023-12-26 20:34:32,586][105620] Updated weights for policy 1, policy_version 716278 (0.0008) [2023-12-26 20:34:33,082][105692] Updated weights for policy 0, policy_version 715664 (0.0010) [2023-12-26 20:34:33,143][105692] Updated weights for policy 0, policy_version 715674 (0.0009) [2023-12-26 20:34:33,149][105620] Updated weights for policy 1, policy_version 716288 (0.0007) [2023-12-26 20:34:33,203][105620] Updated weights for policy 1, policy_version 716298 (0.0005) [2023-12-26 20:34:33,206][105692] Updated weights for policy 0, policy_version 715684 (0.0010) [2023-12-26 20:34:33,256][105620] Updated weights for policy 1, policy_version 716308 (0.0005) [2023-12-26 20:34:33,821][105692] Updated weights for policy 0, policy_version 715694 (0.0007) [2023-12-26 20:34:33,876][105692] Updated weights for policy 0, policy_version 715704 (0.0005) [2023-12-26 20:34:33,941][105692] Updated weights for policy 0, policy_version 715714 (0.0005) [2023-12-26 20:34:34,044][105620] Updated weights for policy 1, policy_version 716318 (0.0008) [2023-12-26 20:34:34,105][105620] Updated weights for policy 1, policy_version 716328 (0.0007) [2023-12-26 20:34:34,178][105620] Updated weights for policy 1, policy_version 716338 (0.0006) [2023-12-26 20:34:34,546][105692] Updated weights for policy 0, policy_version 715724 (0.0006) [2023-12-26 20:34:34,599][105692] Updated weights for policy 0, policy_version 715734 (0.0009) [2023-12-26 20:34:34,656][105692] Updated weights for policy 0, policy_version 715744 (0.0010) [2023-12-26 20:34:34,853][105620] Updated weights for policy 1, policy_version 716348 (0.0006) [2023-12-26 20:34:34,909][105620] Updated weights for policy 1, policy_version 716358 (0.0010) [2023-12-26 20:34:34,956][105620] Updated weights for policy 1, policy_version 716368 (0.0007) [2023-12-26 20:34:35,441][105692] Updated weights for policy 0, policy_version 715754 (0.0008) [2023-12-26 20:34:35,500][105692] Updated weights for policy 0, policy_version 715764 (0.0005) [2023-12-26 20:34:35,561][105692] Updated weights for policy 0, policy_version 715774 (0.0005) [2023-12-26 20:34:35,611][105620] Updated weights for policy 1, policy_version 716378 (0.0006) [2023-12-26 20:34:35,623][105692] Updated weights for policy 0, policy_version 715784 (0.0008) [2023-12-26 20:34:35,662][105620] Updated weights for policy 1, policy_version 716388 (0.0010) [2023-12-26 20:34:35,726][105620] Updated weights for policy 1, policy_version 716398 (0.0009) [2023-12-26 20:34:35,794][105620] Updated weights for policy 1, policy_version 716408 (0.0008) [2023-12-26 20:34:36,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 366690304. Throughput: 0: 10264.9, 1: 9649.6. Samples: 366680176. Policy #0 lag: (min: 9.0, avg: 20.3, max: 41.0) [2023-12-26 20:34:36,062][104569] Avg episode reward: [(0, '9261.913'), (1, '8729.161')] [2023-12-26 20:34:36,252][105692] Updated weights for policy 0, policy_version 715794 (0.0009) [2023-12-26 20:34:36,314][105692] Updated weights for policy 0, policy_version 715804 (0.0009) [2023-12-26 20:34:36,383][105692] Updated weights for policy 0, policy_version 715814 (0.0008) [2023-12-26 20:34:36,539][105620] Updated weights for policy 1, policy_version 716418 (0.0011) [2023-12-26 20:34:36,596][105620] Updated weights for policy 1, policy_version 716428 (0.0008) [2023-12-26 20:34:36,665][105620] Updated weights for policy 1, policy_version 716438 (0.0010) [2023-12-26 20:34:37,092][105692] Updated weights for policy 0, policy_version 715824 (0.0008) [2023-12-26 20:34:37,154][105692] Updated weights for policy 0, policy_version 715834 (0.0006) [2023-12-26 20:34:37,205][105692] Updated weights for policy 0, policy_version 715844 (0.0009) [2023-12-26 20:34:37,352][105620] Updated weights for policy 1, policy_version 716448 (0.0006) [2023-12-26 20:34:37,408][105620] Updated weights for policy 1, policy_version 716458 (0.0007) [2023-12-26 20:34:37,456][105620] Updated weights for policy 1, policy_version 716468 (0.0010) [2023-12-26 20:34:37,928][105692] Updated weights for policy 0, policy_version 715854 (0.0011) [2023-12-26 20:34:37,993][105692] Updated weights for policy 0, policy_version 715864 (0.0011) [2023-12-26 20:34:38,061][105692] Updated weights for policy 0, policy_version 715874 (0.0006) [2023-12-26 20:34:38,164][105620] Updated weights for policy 1, policy_version 716478 (0.0011) [2023-12-26 20:34:38,223][105620] Updated weights for policy 1, policy_version 716488 (0.0010) [2023-12-26 20:34:38,271][105620] Updated weights for policy 1, policy_version 716498 (0.0010) [2023-12-26 20:34:38,773][105692] Updated weights for policy 0, policy_version 715884 (0.0010) [2023-12-26 20:34:38,825][105692] Updated weights for policy 0, policy_version 715894 (0.0010) [2023-12-26 20:34:38,873][105692] Updated weights for policy 0, policy_version 715904 (0.0010) [2023-12-26 20:34:38,971][105620] Updated weights for policy 1, policy_version 716508 (0.0009) [2023-12-26 20:34:39,038][105620] Updated weights for policy 1, policy_version 716518 (0.0006) [2023-12-26 20:34:39,104][105620] Updated weights for policy 1, policy_version 716528 (0.0006) [2023-12-26 20:34:39,701][105620] Updated weights for policy 1, policy_version 716538 (0.0006) [2023-12-26 20:34:39,702][105692] Updated weights for policy 0, policy_version 715914 (0.0011) [2023-12-26 20:34:39,761][105692] Updated weights for policy 0, policy_version 715924 (0.0011) [2023-12-26 20:34:39,764][105620] Updated weights for policy 1, policy_version 716548 (0.0010) [2023-12-26 20:34:39,825][105692] Updated weights for policy 0, policy_version 715934 (0.0011) [2023-12-26 20:34:39,828][105620] Updated weights for policy 1, policy_version 716558 (0.0011) [2023-12-26 20:34:39,879][105692] Updated weights for policy 0, policy_version 715944 (0.0010) [2023-12-26 20:34:39,886][105620] Updated weights for policy 1, policy_version 716568 (0.0011) [2023-12-26 20:34:40,567][105692] Updated weights for policy 0, policy_version 715954 (0.0006) [2023-12-26 20:34:40,625][105692] Updated weights for policy 0, policy_version 715964 (0.0006) [2023-12-26 20:34:40,628][105620] Updated weights for policy 1, policy_version 716578 (0.0011) [2023-12-26 20:34:40,678][105692] Updated weights for policy 0, policy_version 715974 (0.0006) [2023-12-26 20:34:40,685][105620] Updated weights for policy 1, policy_version 716588 (0.0011) [2023-12-26 20:34:40,739][105620] Updated weights for policy 1, policy_version 716598 (0.0010) [2023-12-26 20:34:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 366788608. Throughput: 0: 10185.2, 1: 9649.8. Samples: 366797204. Policy #0 lag: (min: 9.0, avg: 20.3, max: 41.0) [2023-12-26 20:34:41,063][104569] Avg episode reward: [(0, '9259.255'), (1, '9091.378')] [2023-12-26 20:34:41,328][105692] Updated weights for policy 0, policy_version 715984 (0.0006) [2023-12-26 20:34:41,397][105692] Updated weights for policy 0, policy_version 715994 (0.0008) [2023-12-26 20:34:41,464][105692] Updated weights for policy 0, policy_version 716004 (0.0009) [2023-12-26 20:34:41,549][105620] Updated weights for policy 1, policy_version 716608 (0.0010) [2023-12-26 20:34:41,620][105620] Updated weights for policy 1, policy_version 716618 (0.0007) [2023-12-26 20:34:41,688][105620] Updated weights for policy 1, policy_version 716628 (0.0008) [2023-12-26 20:34:42,179][105692] Updated weights for policy 0, policy_version 716014 (0.0008) [2023-12-26 20:34:42,246][105692] Updated weights for policy 0, policy_version 716024 (0.0009) [2023-12-26 20:34:42,310][105620] Updated weights for policy 1, policy_version 716638 (0.0010) [2023-12-26 20:34:42,312][105692] Updated weights for policy 0, policy_version 716034 (0.0007) [2023-12-26 20:34:42,376][105620] Updated weights for policy 1, policy_version 716648 (0.0010) [2023-12-26 20:34:42,437][105620] Updated weights for policy 1, policy_version 716658 (0.0010) [2023-12-26 20:34:42,976][105692] Updated weights for policy 0, policy_version 716044 (0.0009) [2023-12-26 20:34:43,027][105692] Updated weights for policy 0, policy_version 716054 (0.0009) [2023-12-26 20:34:43,079][105692] Updated weights for policy 0, policy_version 716064 (0.0010) [2023-12-26 20:34:43,115][105620] Updated weights for policy 1, policy_version 716668 (0.0006) [2023-12-26 20:34:43,163][105620] Updated weights for policy 1, policy_version 716678 (0.0005) [2023-12-26 20:34:43,229][105620] Updated weights for policy 1, policy_version 716688 (0.0006) [2023-12-26 20:34:43,753][105620] Updated weights for policy 1, policy_version 716698 (0.0006) [2023-12-26 20:34:43,807][105620] Updated weights for policy 1, policy_version 716708 (0.0007) [2023-12-26 20:34:43,822][105692] Updated weights for policy 0, policy_version 716074 (0.0009) [2023-12-26 20:34:43,860][105620] Updated weights for policy 1, policy_version 716718 (0.0010) [2023-12-26 20:34:43,878][105692] Updated weights for policy 0, policy_version 716084 (0.0010) [2023-12-26 20:34:43,909][105620] Updated weights for policy 1, policy_version 716728 (0.0010) [2023-12-26 20:34:43,933][105692] Updated weights for policy 0, policy_version 716094 (0.0010) [2023-12-26 20:34:43,987][105692] Updated weights for policy 0, policy_version 716104 (0.0010) [2023-12-26 20:34:44,617][105620] Updated weights for policy 1, policy_version 716738 (0.0005) [2023-12-26 20:34:44,678][105620] Updated weights for policy 1, policy_version 716748 (0.0005) [2023-12-26 20:34:44,711][105692] Updated weights for policy 0, policy_version 716114 (0.0007) [2023-12-26 20:34:44,733][105620] Updated weights for policy 1, policy_version 716758 (0.0007) [2023-12-26 20:34:44,776][105692] Updated weights for policy 0, policy_version 716124 (0.0009) [2023-12-26 20:34:44,838][105692] Updated weights for policy 0, policy_version 716134 (0.0008) [2023-12-26 20:34:45,392][105620] Updated weights for policy 1, policy_version 716768 (0.0010) [2023-12-26 20:34:45,446][105620] Updated weights for policy 1, policy_version 716778 (0.0009) [2023-12-26 20:34:45,507][105620] Updated weights for policy 1, policy_version 716788 (0.0005) [2023-12-26 20:34:45,530][105692] Updated weights for policy 0, policy_version 716144 (0.0008) [2023-12-26 20:34:45,580][105692] Updated weights for policy 0, policy_version 716154 (0.0008) [2023-12-26 20:34:45,634][105692] Updated weights for policy 0, policy_version 716164 (0.0010) [2023-12-26 20:34:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 366886912. Throughput: 0: 10142.3, 1: 9645.4. Samples: 366858248. Policy #0 lag: (min: 9.0, avg: 20.3, max: 41.0) [2023-12-26 20:34:46,062][104569] Avg episode reward: [(0, '9259.895'), (1, '8722.214')] [2023-12-26 20:34:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000716168_183369728.pth... [2023-12-26 20:34:46,069][105620] Updated weights for policy 1, policy_version 716798 (0.0008) [2023-12-26 20:34:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000714984_183066624.pth [2023-12-26 20:34:46,128][105620] Updated weights for policy 1, policy_version 716808 (0.0008) [2023-12-26 20:34:46,183][105620] Updated weights for policy 1, policy_version 716818 (0.0009) [2023-12-26 20:34:46,208][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000716824_183525376.pth... [2023-12-26 20:34:46,211][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000715704_183238656.pth [2023-12-26 20:34:46,517][105692] Updated weights for policy 0, policy_version 716175 (0.0009) [2023-12-26 20:34:46,577][105692] Updated weights for policy 0, policy_version 716186 (0.0008) [2023-12-26 20:34:46,641][105692] Updated weights for policy 0, policy_version 716196 (0.0005) [2023-12-26 20:34:46,767][105620] Updated weights for policy 1, policy_version 716828 (0.0008) [2023-12-26 20:34:46,815][105620] Updated weights for policy 1, policy_version 716838 (0.0005) [2023-12-26 20:34:46,877][105620] Updated weights for policy 1, policy_version 716848 (0.0005) [2023-12-26 20:34:47,331][105692] Updated weights for policy 0, policy_version 716206 (0.0008) [2023-12-26 20:34:47,379][105692] Updated weights for policy 0, policy_version 716216 (0.0009) [2023-12-26 20:34:47,445][105692] Updated weights for policy 0, policy_version 716226 (0.0009) [2023-12-26 20:34:47,531][105620] Updated weights for policy 1, policy_version 716858 (0.0005) [2023-12-26 20:34:47,577][105620] Updated weights for policy 1, policy_version 716868 (0.0009) [2023-12-26 20:34:47,623][105620] Updated weights for policy 1, policy_version 716878 (0.0009) [2023-12-26 20:34:47,669][105620] Updated weights for policy 1, policy_version 716888 (0.0008) [2023-12-26 20:34:48,230][105692] Updated weights for policy 0, policy_version 716236 (0.0009) [2023-12-26 20:34:48,288][105692] Updated weights for policy 0, policy_version 716246 (0.0009) [2023-12-26 20:34:48,351][105692] Updated weights for policy 0, policy_version 716256 (0.0008) [2023-12-26 20:34:48,404][105620] Updated weights for policy 1, policy_version 716898 (0.0007) [2023-12-26 20:34:48,465][105620] Updated weights for policy 1, policy_version 716908 (0.0009) [2023-12-26 20:34:48,520][105620] Updated weights for policy 1, policy_version 716918 (0.0009) [2023-12-26 20:34:49,147][105692] Updated weights for policy 0, policy_version 716266 (0.0007) [2023-12-26 20:34:49,194][105692] Updated weights for policy 0, policy_version 716276 (0.0009) [2023-12-26 20:34:49,260][105692] Updated weights for policy 0, policy_version 716286 (0.0010) [2023-12-26 20:34:49,311][105620] Updated weights for policy 1, policy_version 716928 (0.0007) [2023-12-26 20:34:49,319][105692] Updated weights for policy 0, policy_version 716296 (0.0008) [2023-12-26 20:34:49,375][105620] Updated weights for policy 1, policy_version 716938 (0.0007) [2023-12-26 20:34:49,435][105620] Updated weights for policy 1, policy_version 716948 (0.0006) [2023-12-26 20:34:49,966][105692] Updated weights for policy 0, policy_version 716306 (0.0008) [2023-12-26 20:34:50,031][105692] Updated weights for policy 0, policy_version 716316 (0.0008) [2023-12-26 20:34:50,095][105692] Updated weights for policy 0, policy_version 716326 (0.0006) [2023-12-26 20:34:50,139][105620] Updated weights for policy 1, policy_version 716958 (0.0007) [2023-12-26 20:34:50,203][105620] Updated weights for policy 1, policy_version 716968 (0.0009) [2023-12-26 20:34:50,268][105620] Updated weights for policy 1, policy_version 716978 (0.0009) [2023-12-26 20:34:50,670][105692] Updated weights for policy 0, policy_version 716336 (0.0009) [2023-12-26 20:34:50,730][105692] Updated weights for policy 0, policy_version 716346 (0.0008) [2023-12-26 20:34:50,792][105692] Updated weights for policy 0, policy_version 716356 (0.0006) [2023-12-26 20:34:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 366985216. Throughput: 0: 10068.2, 1: 9578.9. Samples: 366976244. Policy #0 lag: (min: 9.0, avg: 20.3, max: 41.0) [2023-12-26 20:34:51,063][104569] Avg episode reward: [(0, '9351.628'), (1, '8544.940')] [2023-12-26 20:34:51,107][105620] Updated weights for policy 1, policy_version 716988 (0.0009) [2023-12-26 20:34:51,167][105620] Updated weights for policy 1, policy_version 716998 (0.0008) [2023-12-26 20:34:51,220][105620] Updated weights for policy 1, policy_version 717008 (0.0008) [2023-12-26 20:34:51,533][105692] Updated weights for policy 0, policy_version 716366 (0.0007) [2023-12-26 20:34:51,597][105692] Updated weights for policy 0, policy_version 716376 (0.0010) [2023-12-26 20:34:51,666][105692] Updated weights for policy 0, policy_version 716386 (0.0011) [2023-12-26 20:34:52,022][105620] Updated weights for policy 1, policy_version 717018 (0.0009) [2023-12-26 20:34:52,074][105620] Updated weights for policy 1, policy_version 717028 (0.0008) [2023-12-26 20:34:52,132][105620] Updated weights for policy 1, policy_version 717038 (0.0009) [2023-12-26 20:34:52,358][105692] Updated weights for policy 0, policy_version 716396 (0.0011) [2023-12-26 20:34:52,412][105692] Updated weights for policy 0, policy_version 716406 (0.0010) [2023-12-26 20:34:52,471][105692] Updated weights for policy 0, policy_version 716416 (0.0010) [2023-12-26 20:34:52,990][105620] Updated weights for policy 1, policy_version 717049 (0.0011) [2023-12-26 20:34:53,054][105620] Updated weights for policy 1, policy_version 717059 (0.0008) [2023-12-26 20:34:53,114][105620] Updated weights for policy 1, policy_version 717069 (0.0008) [2023-12-26 20:34:53,168][105620] Updated weights for policy 1, policy_version 717079 (0.0007) [2023-12-26 20:34:53,178][105692] Updated weights for policy 0, policy_version 716426 (0.0010) [2023-12-26 20:34:53,243][105692] Updated weights for policy 0, policy_version 716436 (0.0010) [2023-12-26 20:34:53,308][105692] Updated weights for policy 0, policy_version 716446 (0.0010) [2023-12-26 20:34:53,374][105692] Updated weights for policy 0, policy_version 716456 (0.0010) [2023-12-26 20:34:53,829][105620] Updated weights for policy 1, policy_version 717089 (0.0006) [2023-12-26 20:34:53,878][105620] Updated weights for policy 1, policy_version 717099 (0.0008) [2023-12-26 20:34:53,926][105620] Updated weights for policy 1, policy_version 717109 (0.0008) [2023-12-26 20:34:54,093][105692] Updated weights for policy 0, policy_version 716466 (0.0010) [2023-12-26 20:34:54,154][105692] Updated weights for policy 0, policy_version 716476 (0.0010) [2023-12-26 20:34:54,215][105692] Updated weights for policy 0, policy_version 716486 (0.0010) [2023-12-26 20:34:54,694][105620] Updated weights for policy 1, policy_version 717119 (0.0006) [2023-12-26 20:34:54,757][105620] Updated weights for policy 1, policy_version 717129 (0.0005) [2023-12-26 20:34:54,831][105620] Updated weights for policy 1, policy_version 717139 (0.0005) [2023-12-26 20:34:54,872][105692] Updated weights for policy 0, policy_version 716496 (0.0009) [2023-12-26 20:34:54,926][105692] Updated weights for policy 0, policy_version 716506 (0.0009) [2023-12-26 20:34:54,979][105692] Updated weights for policy 0, policy_version 716516 (0.0009) [2023-12-26 20:34:55,478][105620] Updated weights for policy 1, policy_version 717149 (0.0007) [2023-12-26 20:34:55,536][105620] Updated weights for policy 1, policy_version 717159 (0.0009) [2023-12-26 20:34:55,598][105620] Updated weights for policy 1, policy_version 717169 (0.0009) [2023-12-26 20:34:55,808][105692] Updated weights for policy 0, policy_version 716526 (0.0009) [2023-12-26 20:34:55,859][105692] Updated weights for policy 0, policy_version 716536 (0.0007) [2023-12-26 20:34:55,912][105692] Updated weights for policy 0, policy_version 716546 (0.0007) [2023-12-26 20:34:56,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 367083520. Throughput: 0: 10006.4, 1: 9529.5. Samples: 367089980. Policy #0 lag: (min: 9.0, avg: 20.3, max: 41.0) [2023-12-26 20:34:56,063][104569] Avg episode reward: [(0, '9351.978'), (1, '8826.477')] [2023-12-26 20:34:56,314][105620] Updated weights for policy 1, policy_version 717179 (0.0009) [2023-12-26 20:34:56,367][105620] Updated weights for policy 1, policy_version 717189 (0.0009) [2023-12-26 20:34:56,422][105620] Updated weights for policy 1, policy_version 717199 (0.0009) [2023-12-26 20:34:56,666][105692] Updated weights for policy 0, policy_version 716556 (0.0009) [2023-12-26 20:34:56,712][105692] Updated weights for policy 0, policy_version 716566 (0.0008) [2023-12-26 20:34:56,758][105692] Updated weights for policy 0, policy_version 716576 (0.0009) [2023-12-26 20:34:57,130][105620] Updated weights for policy 1, policy_version 717209 (0.0009) [2023-12-26 20:34:57,180][105620] Updated weights for policy 1, policy_version 717219 (0.0009) [2023-12-26 20:34:57,227][105620] Updated weights for policy 1, policy_version 717229 (0.0009) [2023-12-26 20:34:57,274][105620] Updated weights for policy 1, policy_version 717239 (0.0009) [2023-12-26 20:34:57,537][105692] Updated weights for policy 0, policy_version 716586 (0.0008) [2023-12-26 20:34:57,583][105692] Updated weights for policy 0, policy_version 716596 (0.0008) [2023-12-26 20:34:57,637][105692] Updated weights for policy 0, policy_version 716606 (0.0009) [2023-12-26 20:34:57,693][105692] Updated weights for policy 0, policy_version 716616 (0.0009) [2023-12-26 20:34:58,060][105620] Updated weights for policy 1, policy_version 717249 (0.0006) [2023-12-26 20:34:58,112][105620] Updated weights for policy 1, policy_version 717259 (0.0005) [2023-12-26 20:34:58,172][105620] Updated weights for policy 1, policy_version 717269 (0.0007) [2023-12-26 20:34:58,487][105692] Updated weights for policy 0, policy_version 716626 (0.0009) [2023-12-26 20:34:58,548][105692] Updated weights for policy 0, policy_version 716636 (0.0008) [2023-12-26 20:34:58,602][105692] Updated weights for policy 0, policy_version 716646 (0.0008) [2023-12-26 20:34:58,939][105620] Updated weights for policy 1, policy_version 717279 (0.0008) [2023-12-26 20:34:59,001][105620] Updated weights for policy 1, policy_version 717289 (0.0008) [2023-12-26 20:34:59,067][105620] Updated weights for policy 1, policy_version 717299 (0.0008) [2023-12-26 20:34:59,380][105692] Updated weights for policy 0, policy_version 716656 (0.0008) [2023-12-26 20:34:59,435][105692] Updated weights for policy 0, policy_version 716666 (0.0008) [2023-12-26 20:34:59,489][105692] Updated weights for policy 0, policy_version 716676 (0.0008) [2023-12-26 20:34:59,889][105620] Updated weights for policy 1, policy_version 717309 (0.0008) [2023-12-26 20:34:59,955][105620] Updated weights for policy 1, policy_version 717319 (0.0009) [2023-12-26 20:35:00,008][105620] Updated weights for policy 1, policy_version 717329 (0.0009) [2023-12-26 20:35:00,185][105692] Updated weights for policy 0, policy_version 716686 (0.0009) [2023-12-26 20:35:00,240][105692] Updated weights for policy 0, policy_version 716696 (0.0009) [2023-12-26 20:35:00,288][105692] Updated weights for policy 0, policy_version 716706 (0.0008) [2023-12-26 20:35:00,769][105620] Updated weights for policy 1, policy_version 717339 (0.0008) [2023-12-26 20:35:00,834][105620] Updated weights for policy 1, policy_version 717349 (0.0005) [2023-12-26 20:35:00,851][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000004 [2023-12-26 20:35:01,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 367173632. Throughput: 0: 9968.3, 1: 9559.5. Samples: 367146412. Policy #0 lag: (min: 9.0, avg: 20.3, max: 41.0) [2023-12-26 20:35:01,062][104569] Avg episode reward: [(0, '1506.811'), (1, '8825.452')] [2023-12-26 20:35:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000717352_183664640.pth... [2023-12-26 20:35:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000716216_183369728.pth [2023-12-26 20:35:01,094][105692] Updated weights for policy 0, policy_version 716716 (0.0009) [2023-12-26 20:35:01,160][105692] Updated weights for policy 0, policy_version 716726 (0.0009) [2023-12-26 20:35:01,218][105692] Updated weights for policy 0, policy_version 716736 (0.0010) [2023-12-26 20:35:01,270][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000716744_183517184.pth... [2023-12-26 20:35:01,274][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000715592_183222272.pth [2023-12-26 20:35:01,553][105620] Updated weights for policy 1, policy_version 717359 (0.0009) [2023-12-26 20:35:01,615][105620] Updated weights for policy 1, policy_version 717369 (0.0010) [2023-12-26 20:35:01,684][105620] Updated weights for policy 1, policy_version 717379 (0.0010) [2023-12-26 20:35:01,972][105692] Updated weights for policy 0, policy_version 716746 (0.0009) [2023-12-26 20:35:02,028][105692] Updated weights for policy 0, policy_version 716756 (0.0008) [2023-12-26 20:35:02,082][105692] Updated weights for policy 0, policy_version 716766 (0.0007) [2023-12-26 20:35:02,146][105692] Updated weights for policy 0, policy_version 716776 (0.0010) [2023-12-26 20:35:02,367][105620] Updated weights for policy 1, policy_version 717389 (0.0009) [2023-12-26 20:35:02,422][105620] Updated weights for policy 1, policy_version 717399 (0.0010) [2023-12-26 20:35:02,484][105620] Updated weights for policy 1, policy_version 717409 (0.0010) [2023-12-26 20:35:02,831][105692] Updated weights for policy 0, policy_version 716786 (0.0006) [2023-12-26 20:35:02,895][105692] Updated weights for policy 0, policy_version 716796 (0.0005) [2023-12-26 20:35:02,955][105692] Updated weights for policy 0, policy_version 716806 (0.0008) [2023-12-26 20:35:03,228][105620] Updated weights for policy 1, policy_version 717419 (0.0010) [2023-12-26 20:35:03,276][105620] Updated weights for policy 1, policy_version 717429 (0.0010) [2023-12-26 20:35:03,320][105620] Updated weights for policy 1, policy_version 717439 (0.0010) [2023-12-26 20:35:03,601][105692] Updated weights for policy 0, policy_version 716816 (0.0008) [2023-12-26 20:35:03,649][105692] Updated weights for policy 0, policy_version 716826 (0.0008) [2023-12-26 20:35:03,700][105692] Updated weights for policy 0, policy_version 716836 (0.0008) [2023-12-26 20:35:04,070][105620] Updated weights for policy 1, policy_version 717449 (0.0010) [2023-12-26 20:35:04,137][105620] Updated weights for policy 1, policy_version 717459 (0.0010) [2023-12-26 20:35:04,202][105620] Updated weights for policy 1, policy_version 717469 (0.0008) [2023-12-26 20:35:04,262][105620] Updated weights for policy 1, policy_version 717479 (0.0008) [2023-12-26 20:35:04,497][105692] Updated weights for policy 0, policy_version 716846 (0.0009) [2023-12-26 20:35:04,560][105692] Updated weights for policy 0, policy_version 716856 (0.0008) [2023-12-26 20:35:04,614][105692] Updated weights for policy 0, policy_version 716866 (0.0006) [2023-12-26 20:35:05,029][105620] Updated weights for policy 1, policy_version 717489 (0.0006) [2023-12-26 20:35:05,087][105620] Updated weights for policy 1, policy_version 717499 (0.0005) [2023-12-26 20:35:05,137][105620] Updated weights for policy 1, policy_version 717509 (0.0010) [2023-12-26 20:35:05,338][105692] Updated weights for policy 0, policy_version 716876 (0.0005) [2023-12-26 20:35:05,401][105692] Updated weights for policy 0, policy_version 716886 (0.0007) [2023-12-26 20:35:05,451][105692] Updated weights for policy 0, policy_version 716896 (0.0008) [2023-12-26 20:35:05,723][105620] Updated weights for policy 1, policy_version 717519 (0.0007) [2023-12-26 20:35:05,767][105620] Updated weights for policy 1, policy_version 717529 (0.0005) [2023-12-26 20:35:05,814][105620] Updated weights for policy 1, policy_version 717539 (0.0005) [2023-12-26 20:35:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 367271936. Throughput: 0: 9770.4, 1: 9636.4. Samples: 367259700. Policy #0 lag: (min: 9.0, avg: 20.3, max: 41.0) [2023-12-26 20:35:06,063][104569] Avg episode reward: [(0, '814.546'), (1, '8727.576')] [2023-12-26 20:35:06,293][105692] Updated weights for policy 0, policy_version 716906 (0.0008) [2023-12-26 20:35:06,342][105692] Updated weights for policy 0, policy_version 716916 (0.0009) [2023-12-26 20:35:06,389][105692] Updated weights for policy 0, policy_version 716926 (0.0009) [2023-12-26 20:35:06,431][105620] Updated weights for policy 1, policy_version 717549 (0.0006) [2023-12-26 20:35:06,441][105692] Updated weights for policy 0, policy_version 716936 (0.0008) [2023-12-26 20:35:06,485][105620] Updated weights for policy 1, policy_version 717559 (0.0008) [2023-12-26 20:35:06,541][105620] Updated weights for policy 1, policy_version 717569 (0.0009) [2023-12-26 20:35:07,125][105692] Updated weights for policy 0, policy_version 716946 (0.0009) [2023-12-26 20:35:07,186][105692] Updated weights for policy 0, policy_version 716956 (0.0009) [2023-12-26 20:35:07,249][105692] Updated weights for policy 0, policy_version 716966 (0.0009) [2023-12-26 20:35:07,374][105620] Updated weights for policy 1, policy_version 717579 (0.0009) [2023-12-26 20:35:07,435][105620] Updated weights for policy 1, policy_version 717589 (0.0009) [2023-12-26 20:35:07,488][105620] Updated weights for policy 1, policy_version 717599 (0.0009) [2023-12-26 20:35:07,909][105692] Updated weights for policy 0, policy_version 716976 (0.0006) [2023-12-26 20:35:07,958][105692] Updated weights for policy 0, policy_version 716986 (0.0008) [2023-12-26 20:35:08,021][105692] Updated weights for policy 0, policy_version 716996 (0.0009) [2023-12-26 20:35:08,280][105620] Updated weights for policy 1, policy_version 717609 (0.0009) [2023-12-26 20:35:08,347][105620] Updated weights for policy 1, policy_version 717619 (0.0008) [2023-12-26 20:35:08,407][105620] Updated weights for policy 1, policy_version 717629 (0.0006) [2023-12-26 20:35:08,469][105620] Updated weights for policy 1, policy_version 717639 (0.0007) [2023-12-26 20:35:08,771][105692] Updated weights for policy 0, policy_version 717007 (0.0009) [2023-12-26 20:35:08,838][105692] Updated weights for policy 0, policy_version 717017 (0.0010) [2023-12-26 20:35:08,902][105692] Updated weights for policy 0, policy_version 717027 (0.0010) [2023-12-26 20:35:09,144][105620] Updated weights for policy 1, policy_version 717649 (0.0006) [2023-12-26 20:35:09,206][105620] Updated weights for policy 1, policy_version 717659 (0.0005) [2023-12-26 20:35:09,275][105620] Updated weights for policy 1, policy_version 717669 (0.0009) [2023-12-26 20:35:09,739][105692] Updated weights for policy 0, policy_version 717037 (0.0007) [2023-12-26 20:35:09,800][105692] Updated weights for policy 0, policy_version 717047 (0.0006) [2023-12-26 20:35:09,869][105692] Updated weights for policy 0, policy_version 717057 (0.0008) [2023-12-26 20:35:09,981][105620] Updated weights for policy 1, policy_version 717679 (0.0008) [2023-12-26 20:35:10,040][105620] Updated weights for policy 1, policy_version 717689 (0.0008) [2023-12-26 20:35:10,100][105620] Updated weights for policy 1, policy_version 717699 (0.0008) [2023-12-26 20:35:10,620][105692] Updated weights for policy 0, policy_version 717067 (0.0009) [2023-12-26 20:35:10,679][105692] Updated weights for policy 0, policy_version 717077 (0.0009) [2023-12-26 20:35:10,741][105692] Updated weights for policy 0, policy_version 717087 (0.0009) [2023-12-26 20:35:10,827][105620] Updated weights for policy 1, policy_version 717709 (0.0008) [2023-12-26 20:35:10,887][105620] Updated weights for policy 1, policy_version 717719 (0.0009) [2023-12-26 20:35:10,941][105620] Updated weights for policy 1, policy_version 717729 (0.0008) [2023-12-26 20:35:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 367370240. Throughput: 0: 9742.1, 1: 9653.4. Samples: 367374668. Policy #0 lag: (min: 9.0, avg: 20.3, max: 41.0) [2023-12-26 20:35:11,063][104569] Avg episode reward: [(0, '1191.945'), (1, '9090.308')] [2023-12-26 20:35:11,546][105692] Updated weights for policy 0, policy_version 717097 (0.0009) [2023-12-26 20:35:11,605][105692] Updated weights for policy 0, policy_version 717107 (0.0009) [2023-12-26 20:35:11,669][105692] Updated weights for policy 0, policy_version 717117 (0.0009) [2023-12-26 20:35:11,733][105692] Updated weights for policy 0, policy_version 717127 (0.0009) [2023-12-26 20:35:11,782][105620] Updated weights for policy 1, policy_version 717739 (0.0009) [2023-12-26 20:35:11,845][105620] Updated weights for policy 1, policy_version 717749 (0.0009) [2023-12-26 20:35:11,900][105620] Updated weights for policy 1, policy_version 717759 (0.0009) [2023-12-26 20:35:12,544][105692] Updated weights for policy 0, policy_version 717137 (0.0009) [2023-12-26 20:35:12,602][105692] Updated weights for policy 0, policy_version 717147 (0.0009) [2023-12-26 20:35:12,621][105620] Updated weights for policy 1, policy_version 717769 (0.0009) [2023-12-26 20:35:12,656][105692] Updated weights for policy 0, policy_version 717157 (0.0008) [2023-12-26 20:35:12,672][105620] Updated weights for policy 1, policy_version 717779 (0.0006) [2023-12-26 20:35:12,727][105620] Updated weights for policy 1, policy_version 717789 (0.0009) [2023-12-26 20:35:12,786][105620] Updated weights for policy 1, policy_version 717799 (0.0009) [2023-12-26 20:35:13,402][105692] Updated weights for policy 0, policy_version 717167 (0.0008) [2023-12-26 20:35:13,462][105692] Updated weights for policy 0, policy_version 717177 (0.0009) [2023-12-26 20:35:13,516][105692] Updated weights for policy 0, policy_version 717187 (0.0007) [2023-12-26 20:35:13,548][105620] Updated weights for policy 1, policy_version 717809 (0.0008) [2023-12-26 20:35:13,601][105620] Updated weights for policy 1, policy_version 717819 (0.0008) [2023-12-26 20:35:13,650][105620] Updated weights for policy 1, policy_version 717829 (0.0009) [2023-12-26 20:35:14,285][105692] Updated weights for policy 0, policy_version 717197 (0.0008) [2023-12-26 20:35:14,349][105692] Updated weights for policy 0, policy_version 717207 (0.0008) [2023-12-26 20:35:14,371][105620] Updated weights for policy 1, policy_version 717839 (0.0008) [2023-12-26 20:35:14,401][105692] Updated weights for policy 0, policy_version 717217 (0.0007) [2023-12-26 20:35:14,439][105620] Updated weights for policy 1, policy_version 717849 (0.0008) [2023-12-26 20:35:14,511][105620] Updated weights for policy 1, policy_version 717859 (0.0005) [2023-12-26 20:35:15,118][105620] Updated weights for policy 1, policy_version 717869 (0.0010) [2023-12-26 20:35:15,144][105692] Updated weights for policy 0, policy_version 717227 (0.0009) [2023-12-26 20:35:15,180][105620] Updated weights for policy 1, policy_version 717879 (0.0010) [2023-12-26 20:35:15,211][105692] Updated weights for policy 0, policy_version 717237 (0.0011) [2023-12-26 20:35:15,248][105620] Updated weights for policy 1, policy_version 717889 (0.0008) [2023-12-26 20:35:15,271][105692] Updated weights for policy 0, policy_version 717247 (0.0011) [2023-12-26 20:35:15,921][105620] Updated weights for policy 1, policy_version 717899 (0.0008) [2023-12-26 20:35:15,979][105620] Updated weights for policy 1, policy_version 717909 (0.0010) [2023-12-26 20:35:16,003][105692] Updated weights for policy 0, policy_version 717257 (0.0011) [2023-12-26 20:35:16,035][105620] Updated weights for policy 1, policy_version 717919 (0.0010) [2023-12-26 20:35:16,061][105692] Updated weights for policy 0, policy_version 717267 (0.0010) [2023-12-26 20:35:16,062][104569] Fps is (10 sec: 18022.8, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 367452160. Throughput: 0: 9581.1, 1: 9702.8. Samples: 367429072. Policy #0 lag: (min: 9.0, avg: 20.3, max: 41.0) [2023-12-26 20:35:16,062][104569] Avg episode reward: [(0, '6460.462'), (1, '9006.432')] [2023-12-26 20:35:16,076][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000717928_183812096.pth... [2023-12-26 20:35:16,081][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000716824_183525376.pth [2023-12-26 20:35:16,119][105692] Updated weights for policy 0, policy_version 717277 (0.0010) [2023-12-26 20:35:16,180][105692] Updated weights for policy 0, policy_version 717287 (0.0010) [2023-12-26 20:35:16,185][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000717288_183656448.pth... [2023-12-26 20:35:16,189][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000716168_183369728.pth [2023-12-26 20:35:16,735][105620] Updated weights for policy 1, policy_version 717929 (0.0010) [2023-12-26 20:35:16,741][105692] Updated weights for policy 0, policy_version 717297 (0.0011) [2023-12-26 20:35:16,787][105620] Updated weights for policy 1, policy_version 717939 (0.0010) [2023-12-26 20:35:16,790][105692] Updated weights for policy 0, policy_version 717307 (0.0011) [2023-12-26 20:35:16,835][105620] Updated weights for policy 1, policy_version 717949 (0.0010) [2023-12-26 20:35:16,852][105692] Updated weights for policy 0, policy_version 717317 (0.0010) [2023-12-26 20:35:16,883][105620] Updated weights for policy 1, policy_version 717959 (0.0010) [2023-12-26 20:35:17,552][105692] Updated weights for policy 0, policy_version 717327 (0.0007) [2023-12-26 20:35:17,560][105620] Updated weights for policy 1, policy_version 717969 (0.0009) [2023-12-26 20:35:17,608][105692] Updated weights for policy 0, policy_version 717337 (0.0005) [2023-12-26 20:35:17,622][105620] Updated weights for policy 1, policy_version 717979 (0.0010) [2023-12-26 20:35:17,659][105692] Updated weights for policy 0, policy_version 717347 (0.0005) [2023-12-26 20:35:17,681][105620] Updated weights for policy 1, policy_version 717989 (0.0010) [2023-12-26 20:35:18,276][105692] Updated weights for policy 0, policy_version 717357 (0.0008) [2023-12-26 20:35:18,299][105620] Updated weights for policy 1, policy_version 717999 (0.0008) [2023-12-26 20:35:18,340][105692] Updated weights for policy 0, policy_version 717367 (0.0010) [2023-12-26 20:35:18,358][105620] Updated weights for policy 1, policy_version 718009 (0.0007) [2023-12-26 20:35:18,409][105692] Updated weights for policy 0, policy_version 717377 (0.0009) [2023-12-26 20:35:18,423][105620] Updated weights for policy 1, policy_version 718019 (0.0008) [2023-12-26 20:35:19,042][105620] Updated weights for policy 1, policy_version 718029 (0.0010) [2023-12-26 20:35:19,120][105620] Updated weights for policy 1, policy_version 718039 (0.0010) [2023-12-26 20:35:19,147][105692] Updated weights for policy 0, policy_version 717387 (0.0011) [2023-12-26 20:35:19,182][105620] Updated weights for policy 1, policy_version 718049 (0.0010) [2023-12-26 20:35:19,205][105692] Updated weights for policy 0, policy_version 717397 (0.0011) [2023-12-26 20:35:19,270][105692] Updated weights for policy 0, policy_version 717407 (0.0008) [2023-12-26 20:35:19,874][105620] Updated weights for policy 1, policy_version 718059 (0.0011) [2023-12-26 20:35:19,944][105620] Updated weights for policy 1, policy_version 718069 (0.0011) [2023-12-26 20:35:19,974][105692] Updated weights for policy 0, policy_version 717417 (0.0008) [2023-12-26 20:35:20,000][105620] Updated weights for policy 1, policy_version 718079 (0.0009) [2023-12-26 20:35:20,036][105692] Updated weights for policy 0, policy_version 717427 (0.0009) [2023-12-26 20:35:20,101][105692] Updated weights for policy 0, policy_version 717437 (0.0008) [2023-12-26 20:35:20,164][105692] Updated weights for policy 0, policy_version 717447 (0.0007) [2023-12-26 20:35:20,674][105620] Updated weights for policy 1, policy_version 718089 (0.0010) [2023-12-26 20:35:20,733][105620] Updated weights for policy 1, policy_version 718099 (0.0009) [2023-12-26 20:35:20,797][105620] Updated weights for policy 1, policy_version 718109 (0.0011) [2023-12-26 20:35:20,865][105620] Updated weights for policy 1, policy_version 718119 (0.0008) [2023-12-26 20:35:20,896][105692] Updated weights for policy 0, policy_version 717457 (0.0010) [2023-12-26 20:35:20,955][105692] Updated weights for policy 0, policy_version 717467 (0.0009) [2023-12-26 20:35:21,019][105692] Updated weights for policy 0, policy_version 717477 (0.0006) [2023-12-26 20:35:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 367566848. Throughput: 0: 9544.0, 1: 9806.6. Samples: 367550952. Policy #0 lag: (min: 9.0, avg: 20.3, max: 41.0) [2023-12-26 20:35:21,062][104569] Avg episode reward: [(0, '9171.523'), (1, '8743.810')] [2023-12-26 20:35:21,545][105620] Updated weights for policy 1, policy_version 718129 (0.0007) [2023-12-26 20:35:21,614][105620] Updated weights for policy 1, policy_version 718139 (0.0007) [2023-12-26 20:35:21,679][105620] Updated weights for policy 1, policy_version 718149 (0.0010) [2023-12-26 20:35:21,760][105692] Updated weights for policy 0, policy_version 717487 (0.0007) [2023-12-26 20:35:21,819][105692] Updated weights for policy 0, policy_version 717497 (0.0006) [2023-12-26 20:35:21,878][105692] Updated weights for policy 0, policy_version 717507 (0.0011) [2023-12-26 20:35:22,390][105620] Updated weights for policy 1, policy_version 718159 (0.0010) [2023-12-26 20:35:22,443][105620] Updated weights for policy 1, policy_version 718169 (0.0011) [2023-12-26 20:35:22,496][105620] Updated weights for policy 1, policy_version 718179 (0.0010) [2023-12-26 20:35:22,563][105692] Updated weights for policy 0, policy_version 717517 (0.0011) [2023-12-26 20:35:22,625][105692] Updated weights for policy 0, policy_version 717527 (0.0010) [2023-12-26 20:35:22,684][105692] Updated weights for policy 0, policy_version 717537 (0.0010) [2023-12-26 20:35:23,309][105620] Updated weights for policy 1, policy_version 718189 (0.0010) [2023-12-26 20:35:23,318][105692] Updated weights for policy 0, policy_version 717547 (0.0009) [2023-12-26 20:35:23,368][105620] Updated weights for policy 1, policy_version 718199 (0.0008) [2023-12-26 20:35:23,372][105692] Updated weights for policy 0, policy_version 717557 (0.0007) [2023-12-26 20:35:23,429][105620] Updated weights for policy 1, policy_version 718209 (0.0008) [2023-12-26 20:35:23,430][105692] Updated weights for policy 0, policy_version 717567 (0.0010) [2023-12-26 20:35:24,057][105692] Updated weights for policy 0, policy_version 717577 (0.0010) [2023-12-26 20:35:24,121][105692] Updated weights for policy 0, policy_version 717587 (0.0005) [2023-12-26 20:35:24,187][105692] Updated weights for policy 0, policy_version 717597 (0.0006) [2023-12-26 20:35:24,200][105620] Updated weights for policy 1, policy_version 718219 (0.0007) [2023-12-26 20:35:24,257][105692] Updated weights for policy 0, policy_version 717607 (0.0006) [2023-12-26 20:35:24,257][105620] Updated weights for policy 1, policy_version 718229 (0.0008) [2023-12-26 20:35:24,310][105620] Updated weights for policy 1, policy_version 718240 (0.0010) [2023-12-26 20:35:24,847][105692] Updated weights for policy 0, policy_version 717617 (0.0006) [2023-12-26 20:35:24,893][105692] Updated weights for policy 0, policy_version 717627 (0.0008) [2023-12-26 20:35:24,939][105692] Updated weights for policy 0, policy_version 717637 (0.0008) [2023-12-26 20:35:25,158][105620] Updated weights for policy 1, policy_version 718250 (0.0009) [2023-12-26 20:35:25,219][105620] Updated weights for policy 1, policy_version 718260 (0.0009) [2023-12-26 20:35:25,280][105620] Updated weights for policy 1, policy_version 718270 (0.0009) [2023-12-26 20:35:25,343][105620] Updated weights for policy 1, policy_version 718280 (0.0009) [2023-12-26 20:35:25,587][105692] Updated weights for policy 0, policy_version 717647 (0.0008) [2023-12-26 20:35:25,642][105692] Updated weights for policy 0, policy_version 717657 (0.0009) [2023-12-26 20:35:25,696][105692] Updated weights for policy 0, policy_version 717667 (0.0009) [2023-12-26 20:35:26,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 367656960. Throughput: 0: 9620.6, 1: 9718.2. Samples: 367667448. Policy #0 lag: (min: 9.0, avg: 20.3, max: 41.0) [2023-12-26 20:35:26,062][104569] Avg episode reward: [(0, '8717.255'), (1, '8762.584')] [2023-12-26 20:35:26,110][105620] Updated weights for policy 1, policy_version 718290 (0.0010) [2023-12-26 20:35:26,179][105620] Updated weights for policy 1, policy_version 718300 (0.0010) [2023-12-26 20:35:26,240][105620] Updated weights for policy 1, policy_version 718310 (0.0009) [2023-12-26 20:35:26,408][105692] Updated weights for policy 0, policy_version 717677 (0.0009) [2023-12-26 20:35:26,464][105692] Updated weights for policy 0, policy_version 717687 (0.0009) [2023-12-26 20:35:26,526][105692] Updated weights for policy 0, policy_version 717697 (0.0009) [2023-12-26 20:35:26,995][105620] Updated weights for policy 1, policy_version 718320 (0.0009) [2023-12-26 20:35:27,055][105620] Updated weights for policy 1, policy_version 718330 (0.0009) [2023-12-26 20:35:27,108][105620] Updated weights for policy 1, policy_version 718340 (0.0009) [2023-12-26 20:35:27,171][105692] Updated weights for policy 0, policy_version 717707 (0.0008) [2023-12-26 20:35:27,219][105692] Updated weights for policy 0, policy_version 717717 (0.0009) [2023-12-26 20:35:27,269][105692] Updated weights for policy 0, policy_version 717727 (0.0010) [2023-12-26 20:35:27,905][105692] Updated weights for policy 0, policy_version 717737 (0.0010) [2023-12-26 20:35:27,947][105620] Updated weights for policy 1, policy_version 718350 (0.0008) [2023-12-26 20:35:27,964][105692] Updated weights for policy 0, policy_version 717747 (0.0007) [2023-12-26 20:35:27,994][105620] Updated weights for policy 1, policy_version 718360 (0.0009) [2023-12-26 20:35:28,019][105692] Updated weights for policy 0, policy_version 717757 (0.0006) [2023-12-26 20:35:28,061][105620] Updated weights for policy 1, policy_version 718370 (0.0006) [2023-12-26 20:35:28,083][105692] Updated weights for policy 0, policy_version 717767 (0.0005) [2023-12-26 20:35:28,641][105692] Updated weights for policy 0, policy_version 717777 (0.0005) [2023-12-26 20:35:28,702][105692] Updated weights for policy 0, policy_version 717787 (0.0008) [2023-12-26 20:35:28,757][105692] Updated weights for policy 0, policy_version 717797 (0.0009) [2023-12-26 20:35:28,879][105620] Updated weights for policy 1, policy_version 718380 (0.0008) [2023-12-26 20:35:28,938][105620] Updated weights for policy 1, policy_version 718390 (0.0009) [2023-12-26 20:35:28,997][105620] Updated weights for policy 1, policy_version 718400 (0.0009) [2023-12-26 20:35:29,401][105692] Updated weights for policy 0, policy_version 717807 (0.0008) [2023-12-26 20:35:29,458][105692] Updated weights for policy 0, policy_version 717817 (0.0007) [2023-12-26 20:35:29,524][105692] Updated weights for policy 0, policy_version 717827 (0.0008) [2023-12-26 20:35:29,813][105620] Updated weights for policy 1, policy_version 718410 (0.0008) [2023-12-26 20:35:29,873][105620] Updated weights for policy 1, policy_version 718420 (0.0008) [2023-12-26 20:35:29,936][105620] Updated weights for policy 1, policy_version 718430 (0.0008) [2023-12-26 20:35:29,999][105620] Updated weights for policy 1, policy_version 718440 (0.0008) [2023-12-26 20:35:30,214][105692] Updated weights for policy 0, policy_version 717837 (0.0009) [2023-12-26 20:35:30,276][105692] Updated weights for policy 0, policy_version 717847 (0.0010) [2023-12-26 20:35:30,330][105692] Updated weights for policy 0, policy_version 717857 (0.0010) [2023-12-26 20:35:30,738][105620] Updated weights for policy 1, policy_version 718450 (0.0008) [2023-12-26 20:35:30,789][105620] Updated weights for policy 1, policy_version 718460 (0.0008) [2023-12-26 20:35:30,837][105620] Updated weights for policy 1, policy_version 718470 (0.0008) [2023-12-26 20:35:31,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 367755264. Throughput: 0: 9676.9, 1: 9607.1. Samples: 367726032. Policy #0 lag: (min: 9.0, avg: 20.3, max: 41.0) [2023-12-26 20:35:31,063][104569] Avg episode reward: [(0, '8895.895'), (1, '9109.419')] [2023-12-26 20:35:31,067][105692] Updated weights for policy 0, policy_version 717867 (0.0010) [2023-12-26 20:35:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000718472_183951360.pth... [2023-12-26 20:35:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000717352_183664640.pth [2023-12-26 20:35:31,138][105692] Updated weights for policy 0, policy_version 717877 (0.0007) [2023-12-26 20:35:31,193][105692] Updated weights for policy 0, policy_version 717887 (0.0006) [2023-12-26 20:35:31,240][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000717896_183812096.pth... [2023-12-26 20:35:31,244][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000716744_183517184.pth [2023-12-26 20:35:31,625][105620] Updated weights for policy 1, policy_version 718480 (0.0009) [2023-12-26 20:35:31,681][105620] Updated weights for policy 1, policy_version 718490 (0.0008) [2023-12-26 20:35:31,745][105620] Updated weights for policy 1, policy_version 718500 (0.0009) [2023-12-26 20:35:31,825][105692] Updated weights for policy 0, policy_version 717897 (0.0006) [2023-12-26 20:35:31,879][105692] Updated weights for policy 0, policy_version 717907 (0.0005) [2023-12-26 20:35:31,946][105692] Updated weights for policy 0, policy_version 717917 (0.0005) [2023-12-26 20:35:32,012][105692] Updated weights for policy 0, policy_version 717927 (0.0007) [2023-12-26 20:35:32,548][105692] Updated weights for policy 0, policy_version 717937 (0.0010) [2023-12-26 20:35:32,609][105692] Updated weights for policy 0, policy_version 717947 (0.0009) [2023-12-26 20:35:32,632][105620] Updated weights for policy 1, policy_version 718510 (0.0007) [2023-12-26 20:35:32,667][105692] Updated weights for policy 0, policy_version 717957 (0.0007) [2023-12-26 20:35:32,697][105620] Updated weights for policy 1, policy_version 718520 (0.0006) [2023-12-26 20:35:32,752][105620] Updated weights for policy 1, policy_version 718530 (0.0008) [2023-12-26 20:35:33,225][105692] Updated weights for policy 0, policy_version 717967 (0.0010) [2023-12-26 20:35:33,269][105692] Updated weights for policy 0, policy_version 717977 (0.0010) [2023-12-26 20:35:33,323][105692] Updated weights for policy 0, policy_version 717987 (0.0010) [2023-12-26 20:35:33,480][105620] Updated weights for policy 1, policy_version 718540 (0.0009) [2023-12-26 20:35:33,532][105620] Updated weights for policy 1, policy_version 718550 (0.0008) [2023-12-26 20:35:33,580][105620] Updated weights for policy 1, policy_version 718560 (0.0006) [2023-12-26 20:35:34,072][105692] Updated weights for policy 0, policy_version 717997 (0.0008) [2023-12-26 20:35:34,121][105692] Updated weights for policy 0, policy_version 718007 (0.0005) [2023-12-26 20:35:34,180][105692] Updated weights for policy 0, policy_version 718017 (0.0007) [2023-12-26 20:35:34,365][105620] Updated weights for policy 1, policy_version 718570 (0.0007) [2023-12-26 20:35:34,427][105620] Updated weights for policy 1, policy_version 718580 (0.0006) [2023-12-26 20:35:34,491][105620] Updated weights for policy 1, policy_version 718590 (0.0006) [2023-12-26 20:35:34,544][105620] Updated weights for policy 1, policy_version 718600 (0.0008) [2023-12-26 20:35:34,940][105692] Updated weights for policy 0, policy_version 718027 (0.0007) [2023-12-26 20:35:35,009][105692] Updated weights for policy 0, policy_version 718037 (0.0005) [2023-12-26 20:35:35,075][105692] Updated weights for policy 0, policy_version 718047 (0.0005) [2023-12-26 20:35:35,207][105620] Updated weights for policy 1, policy_version 718610 (0.0008) [2023-12-26 20:35:35,256][105620] Updated weights for policy 1, policy_version 718620 (0.0008) [2023-12-26 20:35:35,314][105620] Updated weights for policy 1, policy_version 718630 (0.0005) [2023-12-26 20:35:35,629][105692] Updated weights for policy 0, policy_version 718057 (0.0010) [2023-12-26 20:35:35,682][105692] Updated weights for policy 0, policy_version 718067 (0.0005) [2023-12-26 20:35:35,728][105692] Updated weights for policy 0, policy_version 718077 (0.0010) [2023-12-26 20:35:35,776][105692] Updated weights for policy 0, policy_version 718087 (0.0010) [2023-12-26 20:35:35,901][105620] Updated weights for policy 1, policy_version 718640 (0.0007) [2023-12-26 20:35:35,960][105620] Updated weights for policy 1, policy_version 718650 (0.0008) [2023-12-26 20:35:36,021][105620] Updated weights for policy 1, policy_version 718660 (0.0008) [2023-12-26 20:35:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 367861760. Throughput: 0: 9818.3, 1: 9435.8. Samples: 367842680. Policy #0 lag: (min: 9.0, avg: 20.3, max: 41.0) [2023-12-26 20:35:36,062][104569] Avg episode reward: [(0, '9260.145'), (1, '9264.503')] [2023-12-26 20:35:36,552][105692] Updated weights for policy 0, policy_version 718097 (0.0011) [2023-12-26 20:35:36,611][105692] Updated weights for policy 0, policy_version 718107 (0.0011) [2023-12-26 20:35:36,674][105692] Updated weights for policy 0, policy_version 718117 (0.0011) [2023-12-26 20:35:36,817][105620] Updated weights for policy 1, policy_version 718670 (0.0009) [2023-12-26 20:35:36,875][105620] Updated weights for policy 1, policy_version 718680 (0.0008) [2023-12-26 20:35:36,931][105620] Updated weights for policy 1, policy_version 718690 (0.0008) [2023-12-26 20:35:37,404][105692] Updated weights for policy 0, policy_version 718127 (0.0010) [2023-12-26 20:35:37,469][105692] Updated weights for policy 0, policy_version 718137 (0.0010) [2023-12-26 20:35:37,536][105692] Updated weights for policy 0, policy_version 718147 (0.0011) [2023-12-26 20:35:37,701][105620] Updated weights for policy 1, policy_version 718700 (0.0007) [2023-12-26 20:35:37,764][105620] Updated weights for policy 1, policy_version 718710 (0.0008) [2023-12-26 20:35:37,828][105620] Updated weights for policy 1, policy_version 718720 (0.0008) [2023-12-26 20:35:38,258][105692] Updated weights for policy 0, policy_version 718157 (0.0010) [2023-12-26 20:35:38,310][105692] Updated weights for policy 0, policy_version 718167 (0.0009) [2023-12-26 20:35:38,373][105692] Updated weights for policy 0, policy_version 718177 (0.0007) [2023-12-26 20:35:38,596][105620] Updated weights for policy 1, policy_version 718730 (0.0008) [2023-12-26 20:35:38,651][105620] Updated weights for policy 1, policy_version 718740 (0.0008) [2023-12-26 20:35:38,706][105620] Updated weights for policy 1, policy_version 718750 (0.0007) [2023-12-26 20:35:38,763][105620] Updated weights for policy 1, policy_version 718760 (0.0009) [2023-12-26 20:35:38,965][105692] Updated weights for policy 0, policy_version 718187 (0.0005) [2023-12-26 20:35:39,028][105692] Updated weights for policy 0, policy_version 718197 (0.0009) [2023-12-26 20:35:39,094][105692] Updated weights for policy 0, policy_version 718207 (0.0009) [2023-12-26 20:35:39,542][105620] Updated weights for policy 1, policy_version 718770 (0.0008) [2023-12-26 20:35:39,610][105620] Updated weights for policy 1, policy_version 718780 (0.0008) [2023-12-26 20:35:39,669][105620] Updated weights for policy 1, policy_version 718790 (0.0009) [2023-12-26 20:35:39,847][105692] Updated weights for policy 0, policy_version 718217 (0.0010) [2023-12-26 20:35:39,909][105692] Updated weights for policy 0, policy_version 718227 (0.0009) [2023-12-26 20:35:39,980][105692] Updated weights for policy 0, policy_version 718237 (0.0009) [2023-12-26 20:35:40,041][105692] Updated weights for policy 0, policy_version 718247 (0.0009) [2023-12-26 20:35:40,465][105620] Updated weights for policy 1, policy_version 718800 (0.0009) [2023-12-26 20:35:40,515][105620] Updated weights for policy 1, policy_version 718810 (0.0009) [2023-12-26 20:35:40,576][105620] Updated weights for policy 1, policy_version 718820 (0.0009) [2023-12-26 20:35:40,680][105692] Updated weights for policy 0, policy_version 718257 (0.0006) [2023-12-26 20:35:40,735][105692] Updated weights for policy 0, policy_version 718267 (0.0006) [2023-12-26 20:35:40,787][105692] Updated weights for policy 0, policy_version 718277 (0.0009) [2023-12-26 20:35:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.8, 300 sec: 19660.8). Total num frames: 367951872. Throughput: 0: 9844.0, 1: 9466.7. Samples: 367958960. Policy #0 lag: (min: 21.0, avg: 25.9, max: 53.0) [2023-12-26 20:35:41,062][104569] Avg episode reward: [(0, '9070.324'), (1, '9083.820')] [2023-12-26 20:35:41,286][105620] Updated weights for policy 1, policy_version 718830 (0.0010) [2023-12-26 20:35:41,354][105620] Updated weights for policy 1, policy_version 718840 (0.0010) [2023-12-26 20:35:41,422][105620] Updated weights for policy 1, policy_version 718850 (0.0010) [2023-12-26 20:35:41,540][105692] Updated weights for policy 0, policy_version 718287 (0.0009) [2023-12-26 20:35:41,593][105692] Updated weights for policy 0, policy_version 718297 (0.0008) [2023-12-26 20:35:41,658][105692] Updated weights for policy 0, policy_version 718307 (0.0008) [2023-12-26 20:35:42,121][105620] Updated weights for policy 1, policy_version 718860 (0.0009) [2023-12-26 20:35:42,195][105620] Updated weights for policy 1, policy_version 718870 (0.0006) [2023-12-26 20:35:42,262][105620] Updated weights for policy 1, policy_version 718880 (0.0011) [2023-12-26 20:35:42,424][105692] Updated weights for policy 0, policy_version 718317 (0.0008) [2023-12-26 20:35:42,492][105692] Updated weights for policy 0, policy_version 718327 (0.0008) [2023-12-26 20:35:42,551][105692] Updated weights for policy 0, policy_version 718337 (0.0009) [2023-12-26 20:35:42,924][105620] Updated weights for policy 1, policy_version 718890 (0.0011) [2023-12-26 20:35:42,972][105620] Updated weights for policy 1, policy_version 718900 (0.0010) [2023-12-26 20:35:43,016][105620] Updated weights for policy 1, policy_version 718910 (0.0010) [2023-12-26 20:35:43,060][105620] Updated weights for policy 1, policy_version 718920 (0.0010) [2023-12-26 20:35:43,305][105692] Updated weights for policy 0, policy_version 718347 (0.0008) [2023-12-26 20:35:43,350][105692] Updated weights for policy 0, policy_version 718357 (0.0008) [2023-12-26 20:35:43,401][105692] Updated weights for policy 0, policy_version 718368 (0.0009) [2023-12-26 20:35:43,815][105620] Updated weights for policy 1, policy_version 718930 (0.0011) [2023-12-26 20:35:43,821][105586] KL-divergence is very high: 177.9468 [2023-12-26 20:35:43,874][105586] KL-divergence is very high: 217.4931 [2023-12-26 20:35:43,883][105620] Updated weights for policy 1, policy_version 718940 (0.0010) [2023-12-26 20:35:43,924][105586] KL-divergence is very high: 150.4424 [2023-12-26 20:35:43,941][105620] Updated weights for policy 1, policy_version 718950 (0.0010) [2023-12-26 20:35:44,144][105692] Updated weights for policy 0, policy_version 718379 (0.0010) [2023-12-26 20:35:44,201][105692] Updated weights for policy 0, policy_version 718390 (0.0010) [2023-12-26 20:35:44,258][105692] Updated weights for policy 0, policy_version 718400 (0.0010) [2023-12-26 20:35:44,571][105620] Updated weights for policy 1, policy_version 718960 (0.0010) [2023-12-26 20:35:44,618][105620] Updated weights for policy 1, policy_version 718970 (0.0010) [2023-12-26 20:35:44,670][105620] Updated weights for policy 1, policy_version 718980 (0.0010) [2023-12-26 20:35:45,071][105692] Updated weights for policy 0, policy_version 718410 (0.0008) [2023-12-26 20:35:45,126][105692] Updated weights for policy 0, policy_version 718420 (0.0008) [2023-12-26 20:35:45,190][105692] Updated weights for policy 0, policy_version 718430 (0.0008) [2023-12-26 20:35:45,247][105692] Updated weights for policy 0, policy_version 718440 (0.0008) [2023-12-26 20:35:45,401][105620] Updated weights for policy 1, policy_version 718990 (0.0010) [2023-12-26 20:35:45,452][105620] Updated weights for policy 1, policy_version 719000 (0.0010) [2023-12-26 20:35:45,501][105620] Updated weights for policy 1, policy_version 719010 (0.0009) [2023-12-26 20:35:45,967][105692] Updated weights for policy 0, policy_version 718450 (0.0008) [2023-12-26 20:35:46,032][105692] Updated weights for policy 0, policy_version 718460 (0.0008) [2023-12-26 20:35:46,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19251.2, 300 sec: 19633.0). Total num frames: 368041984. Throughput: 0: 9848.2, 1: 9486.2. Samples: 368016460. Policy #0 lag: (min: 21.0, avg: 25.9, max: 53.0) [2023-12-26 20:35:46,062][104569] Avg episode reward: [(0, '9159.508'), (1, '9083.669')] [2023-12-26 20:35:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000719016_184090624.pth... [2023-12-26 20:35:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000717928_183812096.pth [2023-12-26 20:35:46,087][105692] Updated weights for policy 0, policy_version 718470 (0.0008) [2023-12-26 20:35:46,097][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000718472_183959552.pth... [2023-12-26 20:35:46,101][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000717288_183656448.pth [2023-12-26 20:35:46,227][105620] Updated weights for policy 1, policy_version 719020 (0.0007) [2023-12-26 20:35:46,285][105620] Updated weights for policy 1, policy_version 719030 (0.0011) [2023-12-26 20:35:46,332][105620] Updated weights for policy 1, policy_version 719040 (0.0010) [2023-12-26 20:35:46,851][105692] Updated weights for policy 0, policy_version 718480 (0.0008) [2023-12-26 20:35:46,899][105692] Updated weights for policy 0, policy_version 718490 (0.0008) [2023-12-26 20:35:46,949][105692] Updated weights for policy 0, policy_version 718500 (0.0008) [2023-12-26 20:35:47,076][105620] Updated weights for policy 1, policy_version 719050 (0.0010) [2023-12-26 20:35:47,124][105620] Updated weights for policy 1, policy_version 719060 (0.0010) [2023-12-26 20:35:47,178][105620] Updated weights for policy 1, policy_version 719070 (0.0010) [2023-12-26 20:35:47,241][105620] Updated weights for policy 1, policy_version 719080 (0.0010) [2023-12-26 20:35:47,732][105692] Updated weights for policy 0, policy_version 718510 (0.0008) [2023-12-26 20:35:47,782][105692] Updated weights for policy 0, policy_version 718520 (0.0008) [2023-12-26 20:35:47,838][105692] Updated weights for policy 0, policy_version 718530 (0.0010) [2023-12-26 20:35:47,878][105620] Updated weights for policy 1, policy_version 719090 (0.0007) [2023-12-26 20:35:47,933][105620] Updated weights for policy 1, policy_version 719100 (0.0007) [2023-12-26 20:35:47,987][105620] Updated weights for policy 1, policy_version 719110 (0.0005) [2023-12-26 20:35:48,650][105692] Updated weights for policy 0, policy_version 718540 (0.0009) [2023-12-26 20:35:48,696][105620] Updated weights for policy 1, policy_version 719120 (0.0006) [2023-12-26 20:35:48,709][105692] Updated weights for policy 0, policy_version 718550 (0.0008) [2023-12-26 20:35:48,756][105620] Updated weights for policy 1, policy_version 719130 (0.0008) [2023-12-26 20:35:48,772][105692] Updated weights for policy 0, policy_version 718560 (0.0008) [2023-12-26 20:35:48,808][105620] Updated weights for policy 1, policy_version 719140 (0.0007) [2023-12-26 20:35:49,525][105620] Updated weights for policy 1, policy_version 719150 (0.0007) [2023-12-26 20:35:49,564][105692] Updated weights for policy 0, policy_version 718570 (0.0008) [2023-12-26 20:35:49,591][105620] Updated weights for policy 1, policy_version 719160 (0.0007) [2023-12-26 20:35:49,626][105692] Updated weights for policy 0, policy_version 718580 (0.0009) [2023-12-26 20:35:49,650][105620] Updated weights for policy 1, policy_version 719170 (0.0006) [2023-12-26 20:35:49,679][105692] Updated weights for policy 0, policy_version 718590 (0.0007) [2023-12-26 20:35:49,728][105692] Updated weights for policy 0, policy_version 718600 (0.0008) [2023-12-26 20:35:50,343][105620] Updated weights for policy 1, policy_version 719180 (0.0007) [2023-12-26 20:35:50,392][105620] Updated weights for policy 1, policy_version 719190 (0.0009) [2023-12-26 20:35:50,453][105620] Updated weights for policy 1, policy_version 719200 (0.0009) [2023-12-26 20:35:50,484][105692] Updated weights for policy 0, policy_version 718610 (0.0009) [2023-12-26 20:35:50,542][105692] Updated weights for policy 0, policy_version 718620 (0.0007) [2023-12-26 20:35:50,603][105692] Updated weights for policy 0, policy_version 718630 (0.0009) [2023-12-26 20:35:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19633.0). Total num frames: 368140288. Throughput: 0: 9823.7, 1: 9561.4. Samples: 368132024. Policy #0 lag: (min: 21.0, avg: 25.9, max: 53.0) [2023-12-26 20:35:51,062][104569] Avg episode reward: [(0, '9166.966'), (1, '9176.988')] [2023-12-26 20:35:51,256][105620] Updated weights for policy 1, policy_version 719210 (0.0008) [2023-12-26 20:35:51,318][105620] Updated weights for policy 1, policy_version 719220 (0.0008) [2023-12-26 20:35:51,385][105620] Updated weights for policy 1, policy_version 719230 (0.0008) [2023-12-26 20:35:51,397][105692] Updated weights for policy 0, policy_version 718640 (0.0012) [2023-12-26 20:35:51,449][105620] Updated weights for policy 1, policy_version 719240 (0.0007) [2023-12-26 20:35:51,461][105692] Updated weights for policy 0, policy_version 718650 (0.0008) [2023-12-26 20:35:51,520][105692] Updated weights for policy 0, policy_version 718660 (0.0010) [2023-12-26 20:35:52,146][105620] Updated weights for policy 1, policy_version 719250 (0.0010) [2023-12-26 20:35:52,201][105620] Updated weights for policy 1, policy_version 719260 (0.0009) [2023-12-26 20:35:52,264][105620] Updated weights for policy 1, policy_version 719270 (0.0010) [2023-12-26 20:35:52,310][105692] Updated weights for policy 0, policy_version 718670 (0.0007) [2023-12-26 20:35:52,374][105692] Updated weights for policy 0, policy_version 718680 (0.0006) [2023-12-26 20:35:52,443][105692] Updated weights for policy 0, policy_version 718690 (0.0006) [2023-12-26 20:35:53,032][105620] Updated weights for policy 1, policy_version 719280 (0.0006) [2023-12-26 20:35:53,078][105620] Updated weights for policy 1, policy_version 719290 (0.0005) [2023-12-26 20:35:53,104][105692] Updated weights for policy 0, policy_version 718700 (0.0006) [2023-12-26 20:35:53,128][105620] Updated weights for policy 1, policy_version 719300 (0.0005) [2023-12-26 20:35:53,153][105692] Updated weights for policy 0, policy_version 718710 (0.0008) [2023-12-26 20:35:53,203][105692] Updated weights for policy 0, policy_version 718720 (0.0006) [2023-12-26 20:35:53,760][105620] Updated weights for policy 1, policy_version 719310 (0.0005) [2023-12-26 20:35:53,814][105620] Updated weights for policy 1, policy_version 719320 (0.0005) [2023-12-26 20:35:53,869][105620] Updated weights for policy 1, policy_version 719330 (0.0005) [2023-12-26 20:35:53,869][105692] Updated weights for policy 0, policy_version 718730 (0.0006) [2023-12-26 20:35:53,919][105692] Updated weights for policy 0, policy_version 718740 (0.0009) [2023-12-26 20:35:53,986][105692] Updated weights for policy 0, policy_version 718750 (0.0005) [2023-12-26 20:35:54,047][105692] Updated weights for policy 0, policy_version 718760 (0.0005) [2023-12-26 20:35:54,483][105620] Updated weights for policy 1, policy_version 719340 (0.0006) [2023-12-26 20:35:54,545][105620] Updated weights for policy 1, policy_version 719350 (0.0006) [2023-12-26 20:35:54,614][105620] Updated weights for policy 1, policy_version 719360 (0.0005) [2023-12-26 20:35:54,636][105692] Updated weights for policy 0, policy_version 718770 (0.0007) [2023-12-26 20:35:54,700][105692] Updated weights for policy 0, policy_version 718780 (0.0006) [2023-12-26 20:35:54,753][105692] Updated weights for policy 0, policy_version 718790 (0.0007) [2023-12-26 20:35:55,166][105620] Updated weights for policy 1, policy_version 719370 (0.0006) [2023-12-26 20:35:55,229][105620] Updated weights for policy 1, policy_version 719380 (0.0005) [2023-12-26 20:35:55,279][105620] Updated weights for policy 1, policy_version 719390 (0.0005) [2023-12-26 20:35:55,459][105692] Updated weights for policy 0, policy_version 718800 (0.0006) [2023-12-26 20:35:55,516][105692] Updated weights for policy 0, policy_version 718810 (0.0005) [2023-12-26 20:35:55,600][105692] Updated weights for policy 0, policy_version 718820 (0.0006) [2023-12-26 20:35:56,001][105620] Updated weights for policy 1, policy_version 719401 (0.0010) [2023-12-26 20:35:56,057][105620] Updated weights for policy 1, policy_version 719411 (0.0008) [2023-12-26 20:35:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.3, 300 sec: 19660.8). Total num frames: 368238592. Throughput: 0: 9872.2, 1: 9612.5. Samples: 368251480. Policy #0 lag: (min: 21.0, avg: 25.9, max: 53.0) [2023-12-26 20:35:56,062][104569] Avg episode reward: [(0, '9349.339'), (1, '9269.597')] [2023-12-26 20:35:56,107][105620] Updated weights for policy 1, policy_version 719421 (0.0008) [2023-12-26 20:35:56,161][105620] Updated weights for policy 1, policy_version 719431 (0.0009) [2023-12-26 20:35:56,248][105692] Updated weights for policy 0, policy_version 718830 (0.0005) [2023-12-26 20:35:56,300][105692] Updated weights for policy 0, policy_version 718840 (0.0005) [2023-12-26 20:35:56,351][105692] Updated weights for policy 0, policy_version 718850 (0.0005) [2023-12-26 20:35:56,971][105692] Updated weights for policy 0, policy_version 718860 (0.0005) [2023-12-26 20:35:56,975][105620] Updated weights for policy 1, policy_version 719441 (0.0009) [2023-12-26 20:35:57,022][105620] Updated weights for policy 1, policy_version 719451 (0.0008) [2023-12-26 20:35:57,025][105692] Updated weights for policy 0, policy_version 718870 (0.0006) [2023-12-26 20:35:57,068][105692] Updated weights for policy 0, policy_version 718880 (0.0006) [2023-12-26 20:35:57,075][105620] Updated weights for policy 1, policy_version 719461 (0.0008) [2023-12-26 20:35:57,661][105692] Updated weights for policy 0, policy_version 718890 (0.0006) [2023-12-26 20:35:57,722][105692] Updated weights for policy 0, policy_version 718900 (0.0010) [2023-12-26 20:35:57,781][105692] Updated weights for policy 0, policy_version 718910 (0.0010) [2023-12-26 20:35:57,837][105692] Updated weights for policy 0, policy_version 718920 (0.0007) [2023-12-26 20:35:57,918][105620] Updated weights for policy 1, policy_version 719471 (0.0006) [2023-12-26 20:35:57,981][105620] Updated weights for policy 1, policy_version 719481 (0.0006) [2023-12-26 20:35:58,042][105620] Updated weights for policy 1, policy_version 719491 (0.0006) [2023-12-26 20:35:58,639][105692] Updated weights for policy 0, policy_version 718930 (0.0008) [2023-12-26 20:35:58,699][105692] Updated weights for policy 0, policy_version 718940 (0.0008) [2023-12-26 20:35:58,769][105692] Updated weights for policy 0, policy_version 718950 (0.0008) [2023-12-26 20:35:58,795][105620] Updated weights for policy 1, policy_version 719501 (0.0008) [2023-12-26 20:35:58,862][105620] Updated weights for policy 1, policy_version 719511 (0.0011) [2023-12-26 20:35:58,919][105620] Updated weights for policy 1, policy_version 719521 (0.0010) [2023-12-26 20:35:59,424][105692] Updated weights for policy 0, policy_version 718960 (0.0009) [2023-12-26 20:35:59,477][105692] Updated weights for policy 0, policy_version 718970 (0.0009) [2023-12-26 20:35:59,537][105692] Updated weights for policy 0, policy_version 718980 (0.0007) [2023-12-26 20:35:59,579][105620] Updated weights for policy 1, policy_version 719531 (0.0010) [2023-12-26 20:35:59,644][105620] Updated weights for policy 1, policy_version 719541 (0.0009) [2023-12-26 20:35:59,694][105620] Updated weights for policy 1, policy_version 719551 (0.0009) [2023-12-26 20:36:00,316][105692] Updated weights for policy 0, policy_version 718991 (0.0008) [2023-12-26 20:36:00,374][105692] Updated weights for policy 0, policy_version 719001 (0.0010) [2023-12-26 20:36:00,403][105620] Updated weights for policy 1, policy_version 719561 (0.0008) [2023-12-26 20:36:00,434][105692] Updated weights for policy 0, policy_version 719011 (0.0009) [2023-12-26 20:36:00,454][105620] Updated weights for policy 1, policy_version 719571 (0.0006) [2023-12-26 20:36:00,510][105620] Updated weights for policy 1, policy_version 719581 (0.0006) [2023-12-26 20:36:00,556][105620] Updated weights for policy 1, policy_version 719591 (0.0008) [2023-12-26 20:36:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 368336896. Throughput: 0: 9969.0, 1: 9587.4. Samples: 368309112. Policy #0 lag: (min: 21.0, avg: 25.9, max: 53.0) [2023-12-26 20:36:01,063][104569] Avg episode reward: [(0, '9267.541'), (1, '9269.425')] [2023-12-26 20:36:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000719016_184098816.pth... [2023-12-26 20:36:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000719592_184238080.pth... [2023-12-26 20:36:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000717896_183812096.pth [2023-12-26 20:36:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000718472_183951360.pth [2023-12-26 20:36:01,196][105620] Updated weights for policy 1, policy_version 719601 (0.0009) [2023-12-26 20:36:01,207][105692] Updated weights for policy 0, policy_version 719021 (0.0008) [2023-12-26 20:36:01,254][105620] Updated weights for policy 1, policy_version 719611 (0.0007) [2023-12-26 20:36:01,267][105692] Updated weights for policy 0, policy_version 719031 (0.0007) [2023-12-26 20:36:01,313][105620] Updated weights for policy 1, policy_version 719621 (0.0008) [2023-12-26 20:36:01,318][105692] Updated weights for policy 0, policy_version 719041 (0.0008) [2023-12-26 20:36:02,081][105692] Updated weights for policy 0, policy_version 719051 (0.0008) [2023-12-26 20:36:02,091][105620] Updated weights for policy 1, policy_version 719631 (0.0008) [2023-12-26 20:36:02,135][105692] Updated weights for policy 0, policy_version 719061 (0.0008) [2023-12-26 20:36:02,142][105620] Updated weights for policy 1, policy_version 719641 (0.0008) [2023-12-26 20:36:02,190][105620] Updated weights for policy 1, policy_version 719651 (0.0009) [2023-12-26 20:36:02,191][105692] Updated weights for policy 0, policy_version 719071 (0.0006) [2023-12-26 20:36:02,852][105620] Updated weights for policy 1, policy_version 719661 (0.0010) [2023-12-26 20:36:02,906][105620] Updated weights for policy 1, policy_version 719671 (0.0008) [2023-12-26 20:36:02,971][105620] Updated weights for policy 1, policy_version 719681 (0.0009) [2023-12-26 20:36:03,002][105692] Updated weights for policy 0, policy_version 719081 (0.0007) [2023-12-26 20:36:03,062][105692] Updated weights for policy 0, policy_version 719091 (0.0009) [2023-12-26 20:36:03,135][105692] Updated weights for policy 0, policy_version 719101 (0.0010) [2023-12-26 20:36:03,194][105692] Updated weights for policy 0, policy_version 719111 (0.0009) [2023-12-26 20:36:03,689][105620] Updated weights for policy 1, policy_version 719691 (0.0008) [2023-12-26 20:36:03,749][105620] Updated weights for policy 1, policy_version 719701 (0.0009) [2023-12-26 20:36:03,814][105620] Updated weights for policy 1, policy_version 719711 (0.0009) [2023-12-26 20:36:03,927][105692] Updated weights for policy 0, policy_version 719121 (0.0009) [2023-12-26 20:36:03,982][105692] Updated weights for policy 0, policy_version 719131 (0.0009) [2023-12-26 20:36:04,043][105692] Updated weights for policy 0, policy_version 719141 (0.0009) [2023-12-26 20:36:04,460][105620] Updated weights for policy 1, policy_version 719721 (0.0008) [2023-12-26 20:36:04,522][105620] Updated weights for policy 1, policy_version 719731 (0.0006) [2023-12-26 20:36:04,584][105620] Updated weights for policy 1, policy_version 719741 (0.0006) [2023-12-26 20:36:04,645][105620] Updated weights for policy 1, policy_version 719751 (0.0006) [2023-12-26 20:36:04,924][105692] Updated weights for policy 0, policy_version 719151 (0.0010) [2023-12-26 20:36:04,978][105692] Updated weights for policy 0, policy_version 719161 (0.0008) [2023-12-26 20:36:05,028][105692] Updated weights for policy 0, policy_version 719171 (0.0008) [2023-12-26 20:36:05,235][105620] Updated weights for policy 1, policy_version 719761 (0.0006) [2023-12-26 20:36:05,289][105620] Updated weights for policy 1, policy_version 719771 (0.0006) [2023-12-26 20:36:05,344][105620] Updated weights for policy 1, policy_version 719781 (0.0006) [2023-12-26 20:36:05,885][105620] Updated weights for policy 1, policy_version 719791 (0.0006) [2023-12-26 20:36:05,902][105692] Updated weights for policy 0, policy_version 719182 (0.0009) [2023-12-26 20:36:05,946][105620] Updated weights for policy 1, policy_version 719801 (0.0006) [2023-12-26 20:36:05,956][105692] Updated weights for policy 0, policy_version 719192 (0.0008) [2023-12-26 20:36:05,998][105620] Updated weights for policy 1, policy_version 719811 (0.0005) [2023-12-26 20:36:06,019][105692] Updated weights for policy 0, policy_version 719202 (0.0008) [2023-12-26 20:36:06,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 368443392. Throughput: 0: 9861.9, 1: 9577.3. Samples: 368425720. Policy #0 lag: (min: 21.0, avg: 25.9, max: 53.0) [2023-12-26 20:36:06,063][104569] Avg episode reward: [(0, '9191.299'), (1, '9267.377')] [2023-12-26 20:36:06,715][105620] Updated weights for policy 1, policy_version 719821 (0.0009) [2023-12-26 20:36:06,778][105620] Updated weights for policy 1, policy_version 719831 (0.0009) [2023-12-26 20:36:06,807][105692] Updated weights for policy 0, policy_version 719212 (0.0008) [2023-12-26 20:36:06,829][105620] Updated weights for policy 1, policy_version 719841 (0.0008) [2023-12-26 20:36:06,866][105692] Updated weights for policy 0, policy_version 719222 (0.0007) [2023-12-26 20:36:06,924][105692] Updated weights for policy 0, policy_version 719232 (0.0009) [2023-12-26 20:36:07,526][105620] Updated weights for policy 1, policy_version 719851 (0.0007) [2023-12-26 20:36:07,573][105620] Updated weights for policy 1, policy_version 719861 (0.0008) [2023-12-26 20:36:07,624][105620] Updated weights for policy 1, policy_version 719871 (0.0007) [2023-12-26 20:36:07,625][105692] Updated weights for policy 0, policy_version 719242 (0.0009) [2023-12-26 20:36:07,680][105692] Updated weights for policy 0, policy_version 719252 (0.0010) [2023-12-26 20:36:07,727][105692] Updated weights for policy 0, policy_version 719262 (0.0011) [2023-12-26 20:36:07,776][105692] Updated weights for policy 0, policy_version 719272 (0.0010) [2023-12-26 20:36:08,424][105620] Updated weights for policy 1, policy_version 719881 (0.0006) [2023-12-26 20:36:08,441][105692] Updated weights for policy 0, policy_version 719282 (0.0007) [2023-12-26 20:36:08,483][105620] Updated weights for policy 1, policy_version 719891 (0.0006) [2023-12-26 20:36:08,503][105692] Updated weights for policy 0, policy_version 719292 (0.0009) [2023-12-26 20:36:08,542][105620] Updated weights for policy 1, policy_version 719901 (0.0010) [2023-12-26 20:36:08,561][105692] Updated weights for policy 0, policy_version 719302 (0.0007) [2023-12-26 20:36:08,598][105620] Updated weights for policy 1, policy_version 719911 (0.0007) [2023-12-26 20:36:09,152][105620] Updated weights for policy 1, policy_version 719921 (0.0005) [2023-12-26 20:36:09,203][105620] Updated weights for policy 1, policy_version 719931 (0.0005) [2023-12-26 20:36:09,235][105692] Updated weights for policy 0, policy_version 719312 (0.0010) [2023-12-26 20:36:09,269][105620] Updated weights for policy 1, policy_version 719941 (0.0009) [2023-12-26 20:36:09,300][105692] Updated weights for policy 0, policy_version 719322 (0.0011) [2023-12-26 20:36:09,364][105692] Updated weights for policy 0, policy_version 719332 (0.0011) [2023-12-26 20:36:09,879][105620] Updated weights for policy 1, policy_version 719951 (0.0011) [2023-12-26 20:36:09,937][105620] Updated weights for policy 1, policy_version 719961 (0.0011) [2023-12-26 20:36:09,999][105620] Updated weights for policy 1, policy_version 719971 (0.0011) [2023-12-26 20:36:10,120][105692] Updated weights for policy 0, policy_version 719342 (0.0011) [2023-12-26 20:36:10,169][105692] Updated weights for policy 0, policy_version 719352 (0.0010) [2023-12-26 20:36:10,226][105692] Updated weights for policy 0, policy_version 719362 (0.0006) [2023-12-26 20:36:10,768][105620] Updated weights for policy 1, policy_version 719981 (0.0011) [2023-12-26 20:36:10,803][105692] Updated weights for policy 0, policy_version 719372 (0.0007) [2023-12-26 20:36:10,831][105620] Updated weights for policy 1, policy_version 719991 (0.0011) [2023-12-26 20:36:10,856][105692] Updated weights for policy 0, policy_version 719382 (0.0010) [2023-12-26 20:36:10,883][105620] Updated weights for policy 1, policy_version 720001 (0.0011) [2023-12-26 20:36:10,913][105692] Updated weights for policy 0, policy_version 719392 (0.0009) [2023-12-26 20:36:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 368541696. Throughput: 0: 9767.6, 1: 9718.1. Samples: 368544308. Policy #0 lag: (min: 21.0, avg: 25.9, max: 53.0) [2023-12-26 20:36:11,063][104569] Avg episode reward: [(0, '9193.532'), (1, '9267.198')] [2023-12-26 20:36:11,622][105620] Updated weights for policy 1, policy_version 720011 (0.0010) [2023-12-26 20:36:11,688][105620] Updated weights for policy 1, policy_version 720021 (0.0011) [2023-12-26 20:36:11,699][105692] Updated weights for policy 0, policy_version 719402 (0.0010) [2023-12-26 20:36:11,757][105620] Updated weights for policy 1, policy_version 720031 (0.0011) [2023-12-26 20:36:11,764][105692] Updated weights for policy 0, policy_version 719412 (0.0006) [2023-12-26 20:36:11,820][105692] Updated weights for policy 0, policy_version 719422 (0.0006) [2023-12-26 20:36:11,876][105692] Updated weights for policy 0, policy_version 719432 (0.0006) [2023-12-26 20:36:12,390][105620] Updated weights for policy 1, policy_version 720041 (0.0011) [2023-12-26 20:36:12,454][105620] Updated weights for policy 1, policy_version 720051 (0.0009) [2023-12-26 20:36:12,514][105620] Updated weights for policy 1, policy_version 720061 (0.0009) [2023-12-26 20:36:12,526][105692] Updated weights for policy 0, policy_version 719442 (0.0007) [2023-12-26 20:36:12,572][105620] Updated weights for policy 1, policy_version 720071 (0.0009) [2023-12-26 20:36:12,574][105692] Updated weights for policy 0, policy_version 719452 (0.0005) [2023-12-26 20:36:12,621][105692] Updated weights for policy 0, policy_version 719462 (0.0007) [2023-12-26 20:36:13,273][105620] Updated weights for policy 1, policy_version 720081 (0.0008) [2023-12-26 20:36:13,327][105620] Updated weights for policy 1, policy_version 720091 (0.0006) [2023-12-26 20:36:13,358][105692] Updated weights for policy 0, policy_version 719472 (0.0008) [2023-12-26 20:36:13,384][105620] Updated weights for policy 1, policy_version 720101 (0.0006) [2023-12-26 20:36:13,411][105692] Updated weights for policy 0, policy_version 719482 (0.0009) [2023-12-26 20:36:13,464][105692] Updated weights for policy 0, policy_version 719493 (0.0011) [2023-12-26 20:36:14,050][105620] Updated weights for policy 1, policy_version 720111 (0.0005) [2023-12-26 20:36:14,110][105620] Updated weights for policy 1, policy_version 720121 (0.0005) [2023-12-26 20:36:14,164][105620] Updated weights for policy 1, policy_version 720131 (0.0006) [2023-12-26 20:36:14,167][105692] Updated weights for policy 0, policy_version 719503 (0.0009) [2023-12-26 20:36:14,220][105692] Updated weights for policy 0, policy_version 719513 (0.0005) [2023-12-26 20:36:14,288][105692] Updated weights for policy 0, policy_version 719523 (0.0005) [2023-12-26 20:36:14,809][105620] Updated weights for policy 1, policy_version 720141 (0.0007) [2023-12-26 20:36:14,843][105692] Updated weights for policy 0, policy_version 719533 (0.0006) [2023-12-26 20:36:14,871][105620] Updated weights for policy 1, policy_version 720151 (0.0005) [2023-12-26 20:36:14,908][105692] Updated weights for policy 0, policy_version 719543 (0.0007) [2023-12-26 20:36:14,942][105620] Updated weights for policy 1, policy_version 720161 (0.0007) [2023-12-26 20:36:14,969][105692] Updated weights for policy 0, policy_version 719553 (0.0005) [2023-12-26 20:36:15,539][105692] Updated weights for policy 0, policy_version 719563 (0.0006) [2023-12-26 20:36:15,591][105692] Updated weights for policy 0, policy_version 719573 (0.0005) [2023-12-26 20:36:15,643][105692] Updated weights for policy 0, policy_version 719583 (0.0006) [2023-12-26 20:36:15,674][105620] Updated weights for policy 1, policy_version 720171 (0.0009) [2023-12-26 20:36:15,732][105620] Updated weights for policy 1, policy_version 720181 (0.0009) [2023-12-26 20:36:15,798][105620] Updated weights for policy 1, policy_version 720191 (0.0007) [2023-12-26 20:36:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 368640000. Throughput: 0: 9694.7, 1: 9808.3. Samples: 368603664. Policy #0 lag: (min: 21.0, avg: 25.9, max: 53.0) [2023-12-26 20:36:16,063][104569] Avg episode reward: [(0, '885.149'), (1, '9265.933')] [2023-12-26 20:36:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000720200_184393728.pth... [2023-12-26 20:36:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000719592_184246272.pth... [2023-12-26 20:36:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000719016_184090624.pth [2023-12-26 20:36:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000718472_183959552.pth [2023-12-26 20:36:16,367][105620] Updated weights for policy 1, policy_version 720201 (0.0006) [2023-12-26 20:36:16,398][105692] Updated weights for policy 0, policy_version 719593 (0.0006) [2023-12-26 20:36:16,421][105620] Updated weights for policy 1, policy_version 720211 (0.0008) [2023-12-26 20:36:16,449][105692] Updated weights for policy 0, policy_version 719603 (0.0008) [2023-12-26 20:36:16,468][105620] Updated weights for policy 1, policy_version 720221 (0.0008) [2023-12-26 20:36:16,510][105692] Updated weights for policy 0, policy_version 719613 (0.0007) [2023-12-26 20:36:16,529][105620] Updated weights for policy 1, policy_version 720231 (0.0008) [2023-12-26 20:36:16,565][105692] Updated weights for policy 0, policy_version 719623 (0.0006) [2023-12-26 20:36:17,196][105692] Updated weights for policy 0, policy_version 719633 (0.0005) [2023-12-26 20:36:17,256][105692] Updated weights for policy 0, policy_version 719643 (0.0007) [2023-12-26 20:36:17,314][105692] Updated weights for policy 0, policy_version 719653 (0.0009) [2023-12-26 20:36:17,365][105620] Updated weights for policy 1, policy_version 720241 (0.0005) [2023-12-26 20:36:17,414][105620] Updated weights for policy 1, policy_version 720251 (0.0006) [2023-12-26 20:36:17,464][105620] Updated weights for policy 1, policy_version 720261 (0.0008) [2023-12-26 20:36:17,960][105692] Updated weights for policy 0, policy_version 719663 (0.0007) [2023-12-26 20:36:18,012][105692] Updated weights for policy 0, policy_version 719673 (0.0009) [2023-12-26 20:36:18,065][105692] Updated weights for policy 0, policy_version 719684 (0.0009) [2023-12-26 20:36:18,177][105620] Updated weights for policy 1, policy_version 720271 (0.0009) [2023-12-26 20:36:18,230][105620] Updated weights for policy 1, policy_version 720281 (0.0009) [2023-12-26 20:36:18,287][105620] Updated weights for policy 1, policy_version 720291 (0.0009) [2023-12-26 20:36:18,918][105692] Updated weights for policy 0, policy_version 719694 (0.0008) [2023-12-26 20:36:18,919][105620] Updated weights for policy 1, policy_version 720301 (0.0010) [2023-12-26 20:36:18,978][105620] Updated weights for policy 1, policy_version 720311 (0.0011) [2023-12-26 20:36:18,980][105692] Updated weights for policy 0, policy_version 719704 (0.0006) [2023-12-26 20:36:19,038][105620] Updated weights for policy 1, policy_version 720321 (0.0011) [2023-12-26 20:36:19,044][105692] Updated weights for policy 0, policy_version 719714 (0.0005) [2023-12-26 20:36:19,684][105620] Updated weights for policy 1, policy_version 720331 (0.0009) [2023-12-26 20:36:19,749][105620] Updated weights for policy 1, policy_version 720341 (0.0008) [2023-12-26 20:36:19,809][105620] Updated weights for policy 1, policy_version 720351 (0.0011) [2023-12-26 20:36:19,815][105692] Updated weights for policy 0, policy_version 719724 (0.0006) [2023-12-26 20:36:19,880][105692] Updated weights for policy 0, policy_version 719734 (0.0008) [2023-12-26 20:36:19,948][105692] Updated weights for policy 0, policy_version 719744 (0.0008) [2023-12-26 20:36:20,481][105620] Updated weights for policy 1, policy_version 720361 (0.0011) [2023-12-26 20:36:20,530][105620] Updated weights for policy 1, policy_version 720371 (0.0010) [2023-12-26 20:36:20,586][105620] Updated weights for policy 1, policy_version 720381 (0.0010) [2023-12-26 20:36:20,658][105620] Updated weights for policy 1, policy_version 720391 (0.0011) [2023-12-26 20:36:20,734][105692] Updated weights for policy 0, policy_version 719754 (0.0008) [2023-12-26 20:36:20,798][105692] Updated weights for policy 0, policy_version 719764 (0.0007) [2023-12-26 20:36:20,851][105692] Updated weights for policy 0, policy_version 719774 (0.0007) [2023-12-26 20:36:20,920][105692] Updated weights for policy 0, policy_version 719784 (0.0008) [2023-12-26 20:36:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 368738304. Throughput: 0: 9685.6, 1: 9939.3. Samples: 368725800. Policy #0 lag: (min: 21.0, avg: 25.9, max: 53.0) [2023-12-26 20:36:21,062][104569] Avg episode reward: [(0, '525.388'), (1, '9083.108')] [2023-12-26 20:36:21,440][105620] Updated weights for policy 1, policy_version 720401 (0.0008) [2023-12-26 20:36:21,502][105620] Updated weights for policy 1, policy_version 720411 (0.0008) [2023-12-26 20:36:21,567][105620] Updated weights for policy 1, policy_version 720421 (0.0009) [2023-12-26 20:36:21,651][105692] Updated weights for policy 0, policy_version 719794 (0.0009) [2023-12-26 20:36:21,716][105692] Updated weights for policy 0, policy_version 719804 (0.0008) [2023-12-26 20:36:21,778][105692] Updated weights for policy 0, policy_version 719814 (0.0008) [2023-12-26 20:36:22,314][105620] Updated weights for policy 1, policy_version 720431 (0.0010) [2023-12-26 20:36:22,378][105620] Updated weights for policy 1, policy_version 720441 (0.0010) [2023-12-26 20:36:22,445][105620] Updated weights for policy 1, policy_version 720451 (0.0011) [2023-12-26 20:36:22,493][105692] Updated weights for policy 0, policy_version 719824 (0.0007) [2023-12-26 20:36:22,543][105692] Updated weights for policy 0, policy_version 719834 (0.0008) [2023-12-26 20:36:22,590][105692] Updated weights for policy 0, policy_version 719844 (0.0008) [2023-12-26 20:36:23,186][105620] Updated weights for policy 1, policy_version 720461 (0.0008) [2023-12-26 20:36:23,243][105620] Updated weights for policy 1, policy_version 720471 (0.0005) [2023-12-26 20:36:23,296][105620] Updated weights for policy 1, policy_version 720481 (0.0006) [2023-12-26 20:36:23,396][105692] Updated weights for policy 0, policy_version 719854 (0.0010) [2023-12-26 20:36:23,446][105692] Updated weights for policy 0, policy_version 719864 (0.0006) [2023-12-26 20:36:23,493][105692] Updated weights for policy 0, policy_version 719874 (0.0005) [2023-12-26 20:36:23,922][105620] Updated weights for policy 1, policy_version 720491 (0.0007) [2023-12-26 20:36:23,990][105620] Updated weights for policy 1, policy_version 720501 (0.0010) [2023-12-26 20:36:24,055][105620] Updated weights for policy 1, policy_version 720511 (0.0010) [2023-12-26 20:36:24,067][105692] Updated weights for policy 0, policy_version 719884 (0.0005) [2023-12-26 20:36:24,126][105692] Updated weights for policy 0, policy_version 719894 (0.0005) [2023-12-26 20:36:24,186][105692] Updated weights for policy 0, policy_version 719904 (0.0005) [2023-12-26 20:36:24,781][105620] Updated weights for policy 1, policy_version 720521 (0.0010) [2023-12-26 20:36:24,825][105692] Updated weights for policy 0, policy_version 719914 (0.0008) [2023-12-26 20:36:24,840][105620] Updated weights for policy 1, policy_version 720531 (0.0010) [2023-12-26 20:36:24,880][105692] Updated weights for policy 0, policy_version 719924 (0.0010) [2023-12-26 20:36:24,898][105620] Updated weights for policy 1, policy_version 720541 (0.0010) [2023-12-26 20:36:24,936][105692] Updated weights for policy 0, policy_version 719934 (0.0010) [2023-12-26 20:36:24,946][105620] Updated weights for policy 1, policy_version 720551 (0.0010) [2023-12-26 20:36:24,988][105692] Updated weights for policy 0, policy_version 719944 (0.0010) [2023-12-26 20:36:25,666][105620] Updated weights for policy 1, policy_version 720561 (0.0010) [2023-12-26 20:36:25,689][105692] Updated weights for policy 0, policy_version 719954 (0.0009) [2023-12-26 20:36:25,714][105620] Updated weights for policy 1, policy_version 720571 (0.0010) [2023-12-26 20:36:25,733][105692] Updated weights for policy 0, policy_version 719964 (0.0008) [2023-12-26 20:36:25,761][105620] Updated weights for policy 1, policy_version 720581 (0.0010) [2023-12-26 20:36:25,784][105692] Updated weights for policy 0, policy_version 719974 (0.0007) [2023-12-26 20:36:26,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 368836608. Throughput: 0: 9659.5, 1: 9965.3. Samples: 368842072. Policy #0 lag: (min: 21.0, avg: 25.9, max: 53.0) [2023-12-26 20:36:26,062][104569] Avg episode reward: [(0, '717.985'), (1, '8901.791')] [2023-12-26 20:36:26,524][105620] Updated weights for policy 1, policy_version 720591 (0.0010) [2023-12-26 20:36:26,536][105692] Updated weights for policy 0, policy_version 719984 (0.0006) [2023-12-26 20:36:26,586][105620] Updated weights for policy 1, policy_version 720601 (0.0010) [2023-12-26 20:36:26,592][105692] Updated weights for policy 0, policy_version 719994 (0.0005) [2023-12-26 20:36:26,647][105692] Updated weights for policy 0, policy_version 720004 (0.0006) [2023-12-26 20:36:26,652][105620] Updated weights for policy 1, policy_version 720611 (0.0010) [2023-12-26 20:36:27,370][105620] Updated weights for policy 1, policy_version 720621 (0.0010) [2023-12-26 20:36:27,389][105692] Updated weights for policy 0, policy_version 720014 (0.0007) [2023-12-26 20:36:27,418][105620] Updated weights for policy 1, policy_version 720631 (0.0010) [2023-12-26 20:36:27,444][105692] Updated weights for policy 0, policy_version 720024 (0.0006) [2023-12-26 20:36:27,466][105620] Updated weights for policy 1, policy_version 720641 (0.0010) [2023-12-26 20:36:27,498][105692] Updated weights for policy 0, policy_version 720034 (0.0005) [2023-12-26 20:36:28,119][105692] Updated weights for policy 0, policy_version 720044 (0.0009) [2023-12-26 20:36:28,163][105692] Updated weights for policy 0, policy_version 720054 (0.0010) [2023-12-26 20:36:28,210][105692] Updated weights for policy 0, policy_version 720064 (0.0010) [2023-12-26 20:36:28,222][105620] Updated weights for policy 1, policy_version 720651 (0.0010) [2023-12-26 20:36:28,277][105620] Updated weights for policy 1, policy_version 720661 (0.0010) [2023-12-26 20:36:28,334][105620] Updated weights for policy 1, policy_version 720671 (0.0010) [2023-12-26 20:36:28,858][105692] Updated weights for policy 0, policy_version 720074 (0.0009) [2023-12-26 20:36:28,919][105692] Updated weights for policy 0, policy_version 720084 (0.0008) [2023-12-26 20:36:28,979][105692] Updated weights for policy 0, policy_version 720094 (0.0010) [2023-12-26 20:36:29,039][105692] Updated weights for policy 0, policy_version 720104 (0.0011) [2023-12-26 20:36:29,084][105620] Updated weights for policy 1, policy_version 720681 (0.0008) [2023-12-26 20:36:29,138][105620] Updated weights for policy 1, policy_version 720691 (0.0008) [2023-12-26 20:36:29,196][105620] Updated weights for policy 1, policy_version 720701 (0.0008) [2023-12-26 20:36:29,255][105620] Updated weights for policy 1, policy_version 720711 (0.0009) [2023-12-26 20:36:29,755][105692] Updated weights for policy 0, policy_version 720114 (0.0009) [2023-12-26 20:36:29,821][105692] Updated weights for policy 0, policy_version 720124 (0.0009) [2023-12-26 20:36:29,884][105692] Updated weights for policy 0, policy_version 720134 (0.0008) [2023-12-26 20:36:30,055][105620] Updated weights for policy 1, policy_version 720721 (0.0008) [2023-12-26 20:36:30,110][105620] Updated weights for policy 1, policy_version 720731 (0.0009) [2023-12-26 20:36:30,170][105620] Updated weights for policy 1, policy_version 720741 (0.0009) [2023-12-26 20:36:30,544][105692] Updated weights for policy 0, policy_version 720144 (0.0010) [2023-12-26 20:36:30,601][105692] Updated weights for policy 0, policy_version 720154 (0.0009) [2023-12-26 20:36:30,658][105692] Updated weights for policy 0, policy_version 720164 (0.0011) [2023-12-26 20:36:30,753][105620] Updated weights for policy 1, policy_version 720751 (0.0006) [2023-12-26 20:36:30,814][105620] Updated weights for policy 1, policy_version 720761 (0.0005) [2023-12-26 20:36:30,873][105620] Updated weights for policy 1, policy_version 720771 (0.0007) [2023-12-26 20:36:31,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 368934912. Throughput: 0: 9711.9, 1: 9941.1. Samples: 368900848. Policy #0 lag: (min: 21.0, avg: 25.9, max: 53.0) [2023-12-26 20:36:31,063][104569] Avg episode reward: [(0, '1164.530'), (1, '9175.100')] [2023-12-26 20:36:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000720776_184541184.pth... [2023-12-26 20:36:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000720168_184393728.pth... [2023-12-26 20:36:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000719592_184238080.pth [2023-12-26 20:36:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000719016_184098816.pth [2023-12-26 20:36:31,461][105692] Updated weights for policy 0, policy_version 720174 (0.0007) [2023-12-26 20:36:31,489][105620] Updated weights for policy 1, policy_version 720781 (0.0008) [2023-12-26 20:36:31,528][105692] Updated weights for policy 0, policy_version 720184 (0.0007) [2023-12-26 20:36:31,538][105620] Updated weights for policy 1, policy_version 720791 (0.0010) [2023-12-26 20:36:31,588][105692] Updated weights for policy 0, policy_version 720194 (0.0008) [2023-12-26 20:36:31,590][105620] Updated weights for policy 1, policy_version 720801 (0.0010) [2023-12-26 20:36:32,239][105692] Updated weights for policy 0, policy_version 720204 (0.0009) [2023-12-26 20:36:32,243][105620] Updated weights for policy 1, policy_version 720811 (0.0008) [2023-12-26 20:36:32,298][105692] Updated weights for policy 0, policy_version 720214 (0.0009) [2023-12-26 20:36:32,300][105620] Updated weights for policy 1, policy_version 720821 (0.0007) [2023-12-26 20:36:32,354][105620] Updated weights for policy 1, policy_version 720831 (0.0006) [2023-12-26 20:36:32,361][105692] Updated weights for policy 0, policy_version 720224 (0.0010) [2023-12-26 20:36:32,955][105692] Updated weights for policy 0, policy_version 720234 (0.0010) [2023-12-26 20:36:32,978][105620] Updated weights for policy 1, policy_version 720841 (0.0006) [2023-12-26 20:36:33,017][105692] Updated weights for policy 0, policy_version 720244 (0.0011) [2023-12-26 20:36:33,034][105620] Updated weights for policy 1, policy_version 720851 (0.0010) [2023-12-26 20:36:33,066][105692] Updated weights for policy 0, policy_version 720254 (0.0010) [2023-12-26 20:36:33,089][105620] Updated weights for policy 1, policy_version 720861 (0.0010) [2023-12-26 20:36:33,121][105692] Updated weights for policy 0, policy_version 720264 (0.0010) [2023-12-26 20:36:33,137][105620] Updated weights for policy 1, policy_version 720871 (0.0010) [2023-12-26 20:36:33,822][105692] Updated weights for policy 0, policy_version 720274 (0.0010) [2023-12-26 20:36:33,873][105620] Updated weights for policy 1, policy_version 720881 (0.0010) [2023-12-26 20:36:33,880][105692] Updated weights for policy 0, policy_version 720284 (0.0010) [2023-12-26 20:36:33,922][105620] Updated weights for policy 1, policy_version 720891 (0.0005) [2023-12-26 20:36:33,941][105692] Updated weights for policy 0, policy_version 720294 (0.0010) [2023-12-26 20:36:33,976][105620] Updated weights for policy 1, policy_version 720901 (0.0007) [2023-12-26 20:36:34,646][105620] Updated weights for policy 1, policy_version 720911 (0.0005) [2023-12-26 20:36:34,709][105620] Updated weights for policy 1, policy_version 720921 (0.0006) [2023-12-26 20:36:34,732][105692] Updated weights for policy 0, policy_version 720304 (0.0008) [2023-12-26 20:36:34,769][105620] Updated weights for policy 1, policy_version 720931 (0.0010) [2023-12-26 20:36:34,783][105692] Updated weights for policy 0, policy_version 720314 (0.0006) [2023-12-26 20:36:34,841][105692] Updated weights for policy 0, policy_version 720324 (0.0008) [2023-12-26 20:36:35,351][105620] Updated weights for policy 1, policy_version 720941 (0.0010) [2023-12-26 20:36:35,402][105620] Updated weights for policy 1, policy_version 720951 (0.0010) [2023-12-26 20:36:35,459][105620] Updated weights for policy 1, policy_version 720961 (0.0010) [2023-12-26 20:36:35,664][105692] Updated weights for policy 0, policy_version 720334 (0.0010) [2023-12-26 20:36:35,717][105692] Updated weights for policy 0, policy_version 720344 (0.0009) [2023-12-26 20:36:35,771][105692] Updated weights for policy 0, policy_version 720354 (0.0009) [2023-12-26 20:36:36,014][105620] Updated weights for policy 1, policy_version 720971 (0.0006) [2023-12-26 20:36:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 369033216. Throughput: 0: 9807.8, 1: 9988.3. Samples: 369022848. Policy #0 lag: (min: 21.0, avg: 25.9, max: 53.0) [2023-12-26 20:36:36,062][104569] Avg episode reward: [(0, '6409.902'), (1, '9177.949')] [2023-12-26 20:36:36,077][105620] Updated weights for policy 1, policy_version 720981 (0.0006) [2023-12-26 20:36:36,142][105620] Updated weights for policy 1, policy_version 720991 (0.0011) [2023-12-26 20:36:36,568][105692] Updated weights for policy 0, policy_version 720364 (0.0009) [2023-12-26 20:36:36,618][105692] Updated weights for policy 0, policy_version 720374 (0.0008) [2023-12-26 20:36:36,683][105692] Updated weights for policy 0, policy_version 720384 (0.0008) [2023-12-26 20:36:36,866][105620] Updated weights for policy 1, policy_version 721001 (0.0011) [2023-12-26 20:36:36,928][105620] Updated weights for policy 1, policy_version 721011 (0.0010) [2023-12-26 20:36:36,998][105620] Updated weights for policy 1, policy_version 721021 (0.0008) [2023-12-26 20:36:37,054][105620] Updated weights for policy 1, policy_version 721031 (0.0006) [2023-12-26 20:36:37,508][105692] Updated weights for policy 0, policy_version 720394 (0.0008) [2023-12-26 20:36:37,568][105692] Updated weights for policy 0, policy_version 720404 (0.0006) [2023-12-26 20:36:37,634][105692] Updated weights for policy 0, policy_version 720414 (0.0008) [2023-12-26 20:36:37,660][105620] Updated weights for policy 1, policy_version 721041 (0.0010) [2023-12-26 20:36:37,690][105692] Updated weights for policy 0, policy_version 720424 (0.0005) [2023-12-26 20:36:37,727][105620] Updated weights for policy 1, policy_version 721051 (0.0011) [2023-12-26 20:36:37,791][105620] Updated weights for policy 1, policy_version 721061 (0.0010) [2023-12-26 20:36:38,432][105620] Updated weights for policy 1, policy_version 721071 (0.0008) [2023-12-26 20:36:38,463][105692] Updated weights for policy 0, policy_version 720434 (0.0008) [2023-12-26 20:36:38,484][105620] Updated weights for policy 1, policy_version 721081 (0.0005) [2023-12-26 20:36:38,528][105692] Updated weights for policy 0, policy_version 720444 (0.0008) [2023-12-26 20:36:38,540][105620] Updated weights for policy 1, policy_version 721091 (0.0006) [2023-12-26 20:36:38,589][105692] Updated weights for policy 0, policy_version 720454 (0.0008) [2023-12-26 20:36:39,269][105620] Updated weights for policy 1, policy_version 721101 (0.0008) [2023-12-26 20:36:39,337][105620] Updated weights for policy 1, policy_version 721111 (0.0009) [2023-12-26 20:36:39,397][105692] Updated weights for policy 0, policy_version 720464 (0.0009) [2023-12-26 20:36:39,408][105620] Updated weights for policy 1, policy_version 721121 (0.0009) [2023-12-26 20:36:39,459][105692] Updated weights for policy 0, policy_version 720474 (0.0007) [2023-12-26 20:36:39,517][105692] Updated weights for policy 0, policy_version 720484 (0.0008) [2023-12-26 20:36:40,155][105620] Updated weights for policy 1, policy_version 721131 (0.0008) [2023-12-26 20:36:40,205][105620] Updated weights for policy 1, policy_version 721141 (0.0008) [2023-12-26 20:36:40,257][105620] Updated weights for policy 1, policy_version 721151 (0.0009) [2023-12-26 20:36:40,293][105692] Updated weights for policy 0, policy_version 720494 (0.0006) [2023-12-26 20:36:40,358][105692] Updated weights for policy 0, policy_version 720504 (0.0008) [2023-12-26 20:36:40,425][105692] Updated weights for policy 0, policy_version 720514 (0.0009) [2023-12-26 20:36:40,923][105620] Updated weights for policy 1, policy_version 721161 (0.0008) [2023-12-26 20:36:40,988][105620] Updated weights for policy 1, policy_version 721171 (0.0005) [2023-12-26 20:36:41,057][105620] Updated weights for policy 1, policy_version 721181 (0.0006) [2023-12-26 20:36:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 369123328. Throughput: 0: 9660.5, 1: 10025.9. Samples: 369137368. Policy #0 lag: (min: 21.0, avg: 25.9, max: 53.0) [2023-12-26 20:36:41,063][104569] Avg episode reward: [(0, '9046.619'), (1, '9084.840')] [2023-12-26 20:36:41,116][105620] Updated weights for policy 1, policy_version 721191 (0.0006) [2023-12-26 20:36:41,260][105692] Updated weights for policy 0, policy_version 720524 (0.0008) [2023-12-26 20:36:41,314][105692] Updated weights for policy 0, policy_version 720534 (0.0009) [2023-12-26 20:36:41,372][105692] Updated weights for policy 0, policy_version 720544 (0.0008) [2023-12-26 20:36:41,782][105620] Updated weights for policy 1, policy_version 721201 (0.0008) [2023-12-26 20:36:41,845][105620] Updated weights for policy 1, policy_version 721211 (0.0007) [2023-12-26 20:36:41,902][105620] Updated weights for policy 1, policy_version 721221 (0.0010) [2023-12-26 20:36:42,184][105692] Updated weights for policy 0, policy_version 720554 (0.0008) [2023-12-26 20:36:42,242][105692] Updated weights for policy 0, policy_version 720564 (0.0010) [2023-12-26 20:36:42,302][105692] Updated weights for policy 0, policy_version 720574 (0.0009) [2023-12-26 20:36:42,365][105692] Updated weights for policy 0, policy_version 720584 (0.0009) [2023-12-26 20:36:42,629][105620] Updated weights for policy 1, policy_version 721231 (0.0007) [2023-12-26 20:36:42,684][105620] Updated weights for policy 1, policy_version 721241 (0.0005) [2023-12-26 20:36:42,739][105620] Updated weights for policy 1, policy_version 721251 (0.0005) [2023-12-26 20:36:43,219][105692] Updated weights for policy 0, policy_version 720594 (0.0008) [2023-12-26 20:36:43,275][105692] Updated weights for policy 0, policy_version 720605 (0.0009) [2023-12-26 20:36:43,289][105620] Updated weights for policy 1, policy_version 721261 (0.0005) [2023-12-26 20:36:43,327][105692] Updated weights for policy 0, policy_version 720615 (0.0009) [2023-12-26 20:36:43,343][105620] Updated weights for policy 1, policy_version 721271 (0.0005) [2023-12-26 20:36:43,398][105620] Updated weights for policy 1, policy_version 721281 (0.0005) [2023-12-26 20:36:44,047][105620] Updated weights for policy 1, policy_version 721291 (0.0006) [2023-12-26 20:36:44,094][105620] Updated weights for policy 1, policy_version 721301 (0.0008) [2023-12-26 20:36:44,152][105620] Updated weights for policy 1, policy_version 721311 (0.0008) [2023-12-26 20:36:44,154][105692] Updated weights for policy 0, policy_version 720625 (0.0011) [2023-12-26 20:36:44,212][105692] Updated weights for policy 0, policy_version 720635 (0.0010) [2023-12-26 20:36:44,263][105692] Updated weights for policy 0, policy_version 720645 (0.0010) [2023-12-26 20:36:44,864][105620] Updated weights for policy 1, policy_version 721321 (0.0008) [2023-12-26 20:36:44,938][105620] Updated weights for policy 1, policy_version 721331 (0.0010) [2023-12-26 20:36:44,947][105692] Updated weights for policy 0, policy_version 720655 (0.0009) [2023-12-26 20:36:44,994][105620] Updated weights for policy 1, policy_version 721341 (0.0009) [2023-12-26 20:36:45,013][105692] Updated weights for policy 0, policy_version 720665 (0.0008) [2023-12-26 20:36:45,048][105620] Updated weights for policy 1, policy_version 721351 (0.0008) [2023-12-26 20:36:45,079][105692] Updated weights for policy 0, policy_version 720675 (0.0008) [2023-12-26 20:36:45,756][105620] Updated weights for policy 1, policy_version 721361 (0.0006) [2023-12-26 20:36:45,813][105620] Updated weights for policy 1, policy_version 721371 (0.0005) [2023-12-26 20:36:45,840][105692] Updated weights for policy 0, policy_version 720685 (0.0007) [2023-12-26 20:36:45,874][105620] Updated weights for policy 1, policy_version 721381 (0.0008) [2023-12-26 20:36:45,898][105692] Updated weights for policy 0, policy_version 720695 (0.0005) [2023-12-26 20:36:45,952][105692] Updated weights for policy 0, policy_version 720705 (0.0005) [2023-12-26 20:36:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 369229824. Throughput: 0: 9534.1, 1: 10135.7. Samples: 369194252. Policy #0 lag: (min: 21.0, avg: 25.9, max: 53.0) [2023-12-26 20:36:46,062][104569] Avg episode reward: [(0, '8048.058'), (1, '9084.666')] [2023-12-26 20:36:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000721384_184696832.pth... [2023-12-26 20:36:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000720712_184532992.pth... [2023-12-26 20:36:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000720200_184393728.pth [2023-12-26 20:36:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000719592_184246272.pth [2023-12-26 20:36:46,534][105692] Updated weights for policy 0, policy_version 720715 (0.0007) [2023-12-26 20:36:46,555][105620] Updated weights for policy 1, policy_version 721391 (0.0011) [2023-12-26 20:36:46,585][105692] Updated weights for policy 0, policy_version 720725 (0.0005) [2023-12-26 20:36:46,607][105620] Updated weights for policy 1, policy_version 721401 (0.0010) [2023-12-26 20:36:46,631][105692] Updated weights for policy 0, policy_version 720735 (0.0005) [2023-12-26 20:36:46,652][105620] Updated weights for policy 1, policy_version 721411 (0.0010) [2023-12-26 20:36:47,297][105620] Updated weights for policy 1, policy_version 721421 (0.0010) [2023-12-26 20:36:47,304][105692] Updated weights for policy 0, policy_version 720745 (0.0006) [2023-12-26 20:36:47,347][105692] Updated weights for policy 0, policy_version 720755 (0.0008) [2023-12-26 20:36:47,352][105620] Updated weights for policy 1, policy_version 721431 (0.0010) [2023-12-26 20:36:47,392][105692] Updated weights for policy 0, policy_version 720765 (0.0006) [2023-12-26 20:36:47,403][105620] Updated weights for policy 1, policy_version 721441 (0.0009) [2023-12-26 20:36:47,448][105692] Updated weights for policy 0, policy_version 720775 (0.0005) [2023-12-26 20:36:48,072][105692] Updated weights for policy 0, policy_version 720785 (0.0010) [2023-12-26 20:36:48,108][105620] Updated weights for policy 1, policy_version 721451 (0.0009) [2023-12-26 20:36:48,137][105692] Updated weights for policy 0, policy_version 720795 (0.0010) [2023-12-26 20:36:48,159][105620] Updated weights for policy 1, policy_version 721461 (0.0005) [2023-12-26 20:36:48,195][105692] Updated weights for policy 0, policy_version 720805 (0.0010) [2023-12-26 20:36:48,213][105620] Updated weights for policy 1, policy_version 721471 (0.0005) [2023-12-26 20:36:48,789][105692] Updated weights for policy 0, policy_version 720815 (0.0006) [2023-12-26 20:36:48,857][105692] Updated weights for policy 0, policy_version 720825 (0.0005) [2023-12-26 20:36:48,920][105692] Updated weights for policy 0, policy_version 720835 (0.0005) [2023-12-26 20:36:49,013][105620] Updated weights for policy 1, policy_version 721481 (0.0007) [2023-12-26 20:36:49,074][105620] Updated weights for policy 1, policy_version 721491 (0.0009) [2023-12-26 20:36:49,134][105620] Updated weights for policy 1, policy_version 721501 (0.0009) [2023-12-26 20:36:49,191][105620] Updated weights for policy 1, policy_version 721511 (0.0013) [2023-12-26 20:36:49,455][105692] Updated weights for policy 0, policy_version 720845 (0.0005) [2023-12-26 20:36:49,518][105692] Updated weights for policy 0, policy_version 720855 (0.0005) [2023-12-26 20:36:49,582][105692] Updated weights for policy 0, policy_version 720865 (0.0006) [2023-12-26 20:36:50,014][105620] Updated weights for policy 1, policy_version 721521 (0.0008) [2023-12-26 20:36:50,077][105620] Updated weights for policy 1, policy_version 721531 (0.0007) [2023-12-26 20:36:50,141][105620] Updated weights for policy 1, policy_version 721541 (0.0006) [2023-12-26 20:36:50,311][105692] Updated weights for policy 0, policy_version 720875 (0.0010) [2023-12-26 20:36:50,371][105692] Updated weights for policy 0, policy_version 720885 (0.0009) [2023-12-26 20:36:50,432][105692] Updated weights for policy 0, policy_version 720895 (0.0009) [2023-12-26 20:36:50,909][105620] Updated weights for policy 1, policy_version 721551 (0.0009) [2023-12-26 20:36:50,969][105620] Updated weights for policy 1, policy_version 721561 (0.0008) [2023-12-26 20:36:51,033][105620] Updated weights for policy 1, policy_version 721571 (0.0008) [2023-12-26 20:36:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 369319936. Throughput: 0: 9735.8, 1: 10043.8. Samples: 369315800. Policy #0 lag: (min: 21.0, avg: 25.9, max: 53.0) [2023-12-26 20:36:51,062][104569] Avg episode reward: [(0, '7841.120'), (1, '9265.970')] [2023-12-26 20:36:51,230][105692] Updated weights for policy 0, policy_version 720905 (0.0009) [2023-12-26 20:36:51,294][105692] Updated weights for policy 0, policy_version 720915 (0.0010) [2023-12-26 20:36:51,357][105692] Updated weights for policy 0, policy_version 720925 (0.0010) [2023-12-26 20:36:51,429][105692] Updated weights for policy 0, policy_version 720935 (0.0011) [2023-12-26 20:36:51,793][105620] Updated weights for policy 1, policy_version 721581 (0.0009) [2023-12-26 20:36:51,853][105620] Updated weights for policy 1, policy_version 721591 (0.0008) [2023-12-26 20:36:51,913][105620] Updated weights for policy 1, policy_version 721601 (0.0008) [2023-12-26 20:36:52,200][105692] Updated weights for policy 0, policy_version 720945 (0.0010) [2023-12-26 20:36:52,255][105692] Updated weights for policy 0, policy_version 720955 (0.0010) [2023-12-26 20:36:52,313][105692] Updated weights for policy 0, policy_version 720965 (0.0008) [2023-12-26 20:36:52,651][105620] Updated weights for policy 1, policy_version 721611 (0.0009) [2023-12-26 20:36:52,709][105620] Updated weights for policy 1, policy_version 721621 (0.0009) [2023-12-26 20:36:52,770][105620] Updated weights for policy 1, policy_version 721631 (0.0008) [2023-12-26 20:36:53,025][105692] Updated weights for policy 0, policy_version 720975 (0.0009) [2023-12-26 20:36:53,080][105692] Updated weights for policy 0, policy_version 720985 (0.0009) [2023-12-26 20:36:53,138][105692] Updated weights for policy 0, policy_version 720995 (0.0009) [2023-12-26 20:36:53,543][105620] Updated weights for policy 1, policy_version 721641 (0.0009) [2023-12-26 20:36:53,611][105620] Updated weights for policy 1, policy_version 721651 (0.0009) [2023-12-26 20:36:53,661][105620] Updated weights for policy 1, policy_version 721661 (0.0009) [2023-12-26 20:36:53,708][105620] Updated weights for policy 1, policy_version 721671 (0.0009) [2023-12-26 20:36:53,884][105692] Updated weights for policy 0, policy_version 721005 (0.0010) [2023-12-26 20:36:53,932][105692] Updated weights for policy 0, policy_version 721015 (0.0010) [2023-12-26 20:36:53,980][105692] Updated weights for policy 0, policy_version 721025 (0.0010) [2023-12-26 20:36:54,494][105620] Updated weights for policy 1, policy_version 721681 (0.0009) [2023-12-26 20:36:54,550][105620] Updated weights for policy 1, policy_version 721691 (0.0009) [2023-12-26 20:36:54,604][105620] Updated weights for policy 1, policy_version 721701 (0.0010) [2023-12-26 20:36:54,631][105692] Updated weights for policy 0, policy_version 721035 (0.0009) [2023-12-26 20:36:54,689][105692] Updated weights for policy 0, policy_version 721045 (0.0006) [2023-12-26 20:36:54,745][105692] Updated weights for policy 0, policy_version 721055 (0.0006) [2023-12-26 20:36:55,316][105692] Updated weights for policy 0, policy_version 721065 (0.0006) [2023-12-26 20:36:55,373][105692] Updated weights for policy 0, policy_version 721075 (0.0010) [2023-12-26 20:36:55,425][105692] Updated weights for policy 0, policy_version 721085 (0.0010) [2023-12-26 20:36:55,470][105620] Updated weights for policy 1, policy_version 721711 (0.0008) [2023-12-26 20:36:55,473][105692] Updated weights for policy 0, policy_version 721095 (0.0010) [2023-12-26 20:36:55,513][105620] Updated weights for policy 1, policy_version 721721 (0.0007) [2023-12-26 20:36:55,565][105620] Updated weights for policy 1, policy_version 721731 (0.0008) [2023-12-26 20:36:56,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 369418240. Throughput: 0: 9763.8, 1: 9871.6. Samples: 369427896. Policy #0 lag: (min: 21.0, avg: 25.9, max: 53.0) [2023-12-26 20:36:56,062][104569] Avg episode reward: [(0, '8907.479'), (1, '9026.292')] [2023-12-26 20:36:56,150][105692] Updated weights for policy 0, policy_version 721105 (0.0007) [2023-12-26 20:36:56,215][105692] Updated weights for policy 0, policy_version 721115 (0.0005) [2023-12-26 20:36:56,281][105692] Updated weights for policy 0, policy_version 721125 (0.0006) [2023-12-26 20:36:56,397][105620] Updated weights for policy 1, policy_version 721741 (0.0008) [2023-12-26 20:36:56,443][105620] Updated weights for policy 1, policy_version 721751 (0.0008) [2023-12-26 20:36:56,496][105620] Updated weights for policy 1, policy_version 721761 (0.0008) [2023-12-26 20:36:56,870][105692] Updated weights for policy 0, policy_version 721135 (0.0009) [2023-12-26 20:36:56,925][105692] Updated weights for policy 0, policy_version 721145 (0.0006) [2023-12-26 20:36:56,975][105692] Updated weights for policy 0, policy_version 721155 (0.0005) [2023-12-26 20:36:57,322][105620] Updated weights for policy 1, policy_version 721771 (0.0009) [2023-12-26 20:36:57,375][105620] Updated weights for policy 1, policy_version 721781 (0.0009) [2023-12-26 20:36:57,433][105620] Updated weights for policy 1, policy_version 721792 (0.0010) [2023-12-26 20:36:57,525][105692] Updated weights for policy 0, policy_version 721165 (0.0005) [2023-12-26 20:36:57,578][105692] Updated weights for policy 0, policy_version 721175 (0.0005) [2023-12-26 20:36:57,637][105692] Updated weights for policy 0, policy_version 721185 (0.0005) [2023-12-26 20:36:58,233][105692] Updated weights for policy 0, policy_version 721195 (0.0007) [2023-12-26 20:36:58,265][105620] Updated weights for policy 1, policy_version 721802 (0.0009) [2023-12-26 20:36:58,306][105692] Updated weights for policy 0, policy_version 721205 (0.0009) [2023-12-26 20:36:58,334][105620] Updated weights for policy 1, policy_version 721812 (0.0007) [2023-12-26 20:36:58,376][105692] Updated weights for policy 0, policy_version 721215 (0.0008) [2023-12-26 20:36:58,398][105620] Updated weights for policy 1, policy_version 721822 (0.0008) [2023-12-26 20:36:58,461][105620] Updated weights for policy 1, policy_version 721832 (0.0008) [2023-12-26 20:36:59,217][105692] Updated weights for policy 0, policy_version 721225 (0.0007) [2023-12-26 20:36:59,232][105620] Updated weights for policy 1, policy_version 721842 (0.0008) [2023-12-26 20:36:59,287][105692] Updated weights for policy 0, policy_version 721235 (0.0009) [2023-12-26 20:36:59,300][105620] Updated weights for policy 1, policy_version 721852 (0.0007) [2023-12-26 20:36:59,350][105692] Updated weights for policy 0, policy_version 721245 (0.0009) [2023-12-26 20:36:59,374][105620] Updated weights for policy 1, policy_version 721862 (0.0007) [2023-12-26 20:36:59,417][105692] Updated weights for policy 0, policy_version 721255 (0.0008) [2023-12-26 20:37:00,068][105620] Updated weights for policy 1, policy_version 721872 (0.0006) [2023-12-26 20:37:00,119][105692] Updated weights for policy 0, policy_version 721265 (0.0009) [2023-12-26 20:37:00,128][105620] Updated weights for policy 1, policy_version 721882 (0.0006) [2023-12-26 20:37:00,169][105692] Updated weights for policy 0, policy_version 721276 (0.0008) [2023-12-26 20:37:00,190][105620] Updated weights for policy 1, policy_version 721892 (0.0009) [2023-12-26 20:37:00,215][105692] Updated weights for policy 0, policy_version 721286 (0.0006) [2023-12-26 20:37:00,875][105692] Updated weights for policy 0, policy_version 721296 (0.0007) [2023-12-26 20:37:00,893][105620] Updated weights for policy 1, policy_version 721902 (0.0009) [2023-12-26 20:37:00,930][105692] Updated weights for policy 0, policy_version 721306 (0.0006) [2023-12-26 20:37:00,940][105620] Updated weights for policy 1, policy_version 721912 (0.0010) [2023-12-26 20:37:00,993][105692] Updated weights for policy 0, policy_version 721316 (0.0005) [2023-12-26 20:37:00,998][105620] Updated weights for policy 1, policy_version 721922 (0.0010) [2023-12-26 20:37:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 369524736. Throughput: 0: 9860.9, 1: 9777.1. Samples: 369487372. Policy #0 lag: (min: 21.0, avg: 25.9, max: 53.0) [2023-12-26 20:37:01,063][104569] Avg episode reward: [(0, '9174.894'), (1, '9025.964')] [2023-12-26 20:37:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000721320_184688640.pth... [2023-12-26 20:37:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000721928_184836096.pth... [2023-12-26 20:37:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000720168_184393728.pth [2023-12-26 20:37:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000720776_184541184.pth [2023-12-26 20:37:01,753][105620] Updated weights for policy 1, policy_version 721932 (0.0010) [2023-12-26 20:37:01,768][105692] Updated weights for policy 0, policy_version 721326 (0.0007) [2023-12-26 20:37:01,812][105620] Updated weights for policy 1, policy_version 721942 (0.0007) [2023-12-26 20:37:01,822][105692] Updated weights for policy 0, policy_version 721336 (0.0008) [2023-12-26 20:37:01,869][105620] Updated weights for policy 1, policy_version 721952 (0.0008) [2023-12-26 20:37:01,879][105692] Updated weights for policy 0, policy_version 721346 (0.0005) [2023-12-26 20:37:02,541][105692] Updated weights for policy 0, policy_version 721356 (0.0005) [2023-12-26 20:37:02,592][105692] Updated weights for policy 0, policy_version 721366 (0.0005) [2023-12-26 20:37:02,638][105692] Updated weights for policy 0, policy_version 721376 (0.0005) [2023-12-26 20:37:02,652][105620] Updated weights for policy 1, policy_version 721962 (0.0008) [2023-12-26 20:37:02,721][105620] Updated weights for policy 1, policy_version 721972 (0.0005) [2023-12-26 20:37:02,792][105620] Updated weights for policy 1, policy_version 721982 (0.0005) [2023-12-26 20:37:02,864][105620] Updated weights for policy 1, policy_version 721992 (0.0007) [2023-12-26 20:37:03,167][105692] Updated weights for policy 0, policy_version 721386 (0.0006) [2023-12-26 20:37:03,223][105692] Updated weights for policy 0, policy_version 721396 (0.0008) [2023-12-26 20:37:03,270][105692] Updated weights for policy 0, policy_version 721406 (0.0005) [2023-12-26 20:37:03,315][105692] Updated weights for policy 0, policy_version 721416 (0.0005) [2023-12-26 20:37:03,379][105620] Updated weights for policy 1, policy_version 722002 (0.0005) [2023-12-26 20:37:03,441][105620] Updated weights for policy 1, policy_version 722012 (0.0008) [2023-12-26 20:37:03,492][105620] Updated weights for policy 1, policy_version 722022 (0.0010) [2023-12-26 20:37:03,865][105692] Updated weights for policy 0, policy_version 721426 (0.0007) [2023-12-26 20:37:03,922][105692] Updated weights for policy 0, policy_version 721436 (0.0009) [2023-12-26 20:37:03,975][105692] Updated weights for policy 0, policy_version 721446 (0.0010) [2023-12-26 20:37:04,248][105620] Updated weights for policy 1, policy_version 722032 (0.0010) [2023-12-26 20:37:04,316][105620] Updated weights for policy 1, policy_version 722042 (0.0007) [2023-12-26 20:37:04,379][105620] Updated weights for policy 1, policy_version 722052 (0.0008) [2023-12-26 20:37:04,700][105692] Updated weights for policy 0, policy_version 721456 (0.0006) [2023-12-26 20:37:04,763][105692] Updated weights for policy 0, policy_version 721466 (0.0005) [2023-12-26 20:37:04,823][105692] Updated weights for policy 0, policy_version 721476 (0.0008) [2023-12-26 20:37:05,082][105620] Updated weights for policy 1, policy_version 722062 (0.0009) [2023-12-26 20:37:05,134][105620] Updated weights for policy 1, policy_version 722072 (0.0010) [2023-12-26 20:37:05,177][105620] Updated weights for policy 1, policy_version 722082 (0.0010) [2023-12-26 20:37:05,420][105692] Updated weights for policy 0, policy_version 721486 (0.0007) [2023-12-26 20:37:05,474][105692] Updated weights for policy 0, policy_version 721496 (0.0007) [2023-12-26 20:37:05,527][105692] Updated weights for policy 0, policy_version 721506 (0.0010) [2023-12-26 20:37:05,772][105620] Updated weights for policy 1, policy_version 722092 (0.0008) [2023-12-26 20:37:05,822][105620] Updated weights for policy 1, policy_version 722102 (0.0008) [2023-12-26 20:37:05,868][105620] Updated weights for policy 1, policy_version 722112 (0.0010) [2023-12-26 20:37:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 369623040. Throughput: 0: 9850.3, 1: 9730.3. Samples: 369606928. Policy #0 lag: (min: 21.0, avg: 25.9, max: 53.0) [2023-12-26 20:37:06,062][104569] Avg episode reward: [(0, '9264.890'), (1, '9110.650')] [2023-12-26 20:37:06,242][105692] Updated weights for policy 0, policy_version 721516 (0.0008) [2023-12-26 20:37:06,299][105692] Updated weights for policy 0, policy_version 721526 (0.0008) [2023-12-26 20:37:06,348][105692] Updated weights for policy 0, policy_version 721536 (0.0008) [2023-12-26 20:37:06,630][105620] Updated weights for policy 1, policy_version 722122 (0.0010) [2023-12-26 20:37:06,686][105620] Updated weights for policy 1, policy_version 722132 (0.0011) [2023-12-26 20:37:06,740][105620] Updated weights for policy 1, policy_version 722142 (0.0010) [2023-12-26 20:37:06,789][105620] Updated weights for policy 1, policy_version 722152 (0.0006) [2023-12-26 20:37:07,137][105692] Updated weights for policy 0, policy_version 721546 (0.0009) [2023-12-26 20:37:07,197][105692] Updated weights for policy 0, policy_version 721556 (0.0008) [2023-12-26 20:37:07,256][105692] Updated weights for policy 0, policy_version 721566 (0.0008) [2023-12-26 20:37:07,324][105692] Updated weights for policy 0, policy_version 721576 (0.0008) [2023-12-26 20:37:07,512][105620] Updated weights for policy 1, policy_version 722162 (0.0005) [2023-12-26 20:37:07,578][105620] Updated weights for policy 1, policy_version 722172 (0.0006) [2023-12-26 20:37:07,626][105620] Updated weights for policy 1, policy_version 722182 (0.0007) [2023-12-26 20:37:08,079][105692] Updated weights for policy 0, policy_version 721586 (0.0006) [2023-12-26 20:37:08,136][105692] Updated weights for policy 0, policy_version 721596 (0.0005) [2023-12-26 20:37:08,196][105692] Updated weights for policy 0, policy_version 721606 (0.0005) [2023-12-26 20:37:08,318][105620] Updated weights for policy 1, policy_version 722192 (0.0007) [2023-12-26 20:37:08,382][105620] Updated weights for policy 1, policy_version 722202 (0.0007) [2023-12-26 20:37:08,451][105620] Updated weights for policy 1, policy_version 722212 (0.0006) [2023-12-26 20:37:08,804][105692] Updated weights for policy 0, policy_version 721616 (0.0007) [2023-12-26 20:37:08,856][105692] Updated weights for policy 0, policy_version 721626 (0.0008) [2023-12-26 20:37:08,916][105692] Updated weights for policy 0, policy_version 721636 (0.0008) [2023-12-26 20:37:09,158][105620] Updated weights for policy 1, policy_version 722222 (0.0011) [2023-12-26 20:37:09,235][105620] Updated weights for policy 1, policy_version 722232 (0.0010) [2023-12-26 20:37:09,292][105620] Updated weights for policy 1, policy_version 722242 (0.0008) [2023-12-26 20:37:09,605][105692] Updated weights for policy 0, policy_version 721646 (0.0008) [2023-12-26 20:37:09,672][105692] Updated weights for policy 0, policy_version 721656 (0.0009) [2023-12-26 20:37:09,736][105692] Updated weights for policy 0, policy_version 721666 (0.0008) [2023-12-26 20:37:10,059][105620] Updated weights for policy 1, policy_version 722252 (0.0010) [2023-12-26 20:37:10,128][105620] Updated weights for policy 1, policy_version 722262 (0.0009) [2023-12-26 20:37:10,191][105620] Updated weights for policy 1, policy_version 722272 (0.0009) [2023-12-26 20:37:10,431][105692] Updated weights for policy 0, policy_version 721676 (0.0007) [2023-12-26 20:37:10,482][105692] Updated weights for policy 0, policy_version 721686 (0.0009) [2023-12-26 20:37:10,528][105692] Updated weights for policy 0, policy_version 721696 (0.0009) [2023-12-26 20:37:10,976][105620] Updated weights for policy 1, policy_version 722282 (0.0009) [2023-12-26 20:37:11,053][105620] Updated weights for policy 1, policy_version 722292 (0.0008) [2023-12-26 20:37:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 369713152. Throughput: 0: 9868.8, 1: 9749.0. Samples: 369724876. Policy #0 lag: (min: 21.0, avg: 25.9, max: 53.0) [2023-12-26 20:37:11,063][104569] Avg episode reward: [(0, '9352.574'), (1, '9265.658')] [2023-12-26 20:37:11,119][105620] Updated weights for policy 1, policy_version 722302 (0.0009) [2023-12-26 20:37:11,194][105620] Updated weights for policy 1, policy_version 722312 (0.0009) [2023-12-26 20:37:11,336][105692] Updated weights for policy 0, policy_version 721706 (0.0009) [2023-12-26 20:37:11,407][105692] Updated weights for policy 0, policy_version 721716 (0.0008) [2023-12-26 20:37:11,467][105692] Updated weights for policy 0, policy_version 721726 (0.0009) [2023-12-26 20:37:11,523][105692] Updated weights for policy 0, policy_version 721736 (0.0010) [2023-12-26 20:37:11,927][105620] Updated weights for policy 1, policy_version 722322 (0.0008) [2023-12-26 20:37:11,991][105620] Updated weights for policy 1, policy_version 722332 (0.0008) [2023-12-26 20:37:12,060][105620] Updated weights for policy 1, policy_version 722342 (0.0009) [2023-12-26 20:37:12,268][105692] Updated weights for policy 0, policy_version 721746 (0.0011) [2023-12-26 20:37:12,329][105692] Updated weights for policy 0, policy_version 721756 (0.0011) [2023-12-26 20:37:12,393][105692] Updated weights for policy 0, policy_version 721766 (0.0007) [2023-12-26 20:37:12,793][105620] Updated weights for policy 1, policy_version 722352 (0.0007) [2023-12-26 20:37:12,853][105620] Updated weights for policy 1, policy_version 722362 (0.0007) [2023-12-26 20:37:12,908][105620] Updated weights for policy 1, policy_version 722372 (0.0007) [2023-12-26 20:37:13,093][105692] Updated weights for policy 0, policy_version 721776 (0.0007) [2023-12-26 20:37:13,151][105692] Updated weights for policy 0, policy_version 721786 (0.0006) [2023-12-26 20:37:13,217][105692] Updated weights for policy 0, policy_version 721796 (0.0006) [2023-12-26 20:37:13,639][105620] Updated weights for policy 1, policy_version 722382 (0.0008) [2023-12-26 20:37:13,693][105620] Updated weights for policy 1, policy_version 722392 (0.0007) [2023-12-26 20:37:13,749][105620] Updated weights for policy 1, policy_version 722402 (0.0006) [2023-12-26 20:37:13,769][105692] Updated weights for policy 0, policy_version 721806 (0.0006) [2023-12-26 20:37:13,834][105692] Updated weights for policy 0, policy_version 721816 (0.0005) [2023-12-26 20:37:13,905][105692] Updated weights for policy 0, policy_version 721826 (0.0005) [2023-12-26 20:37:14,370][105620] Updated weights for policy 1, policy_version 722412 (0.0007) [2023-12-26 20:37:14,434][105620] Updated weights for policy 1, policy_version 722422 (0.0006) [2023-12-26 20:37:14,490][105692] Updated weights for policy 0, policy_version 721836 (0.0007) [2023-12-26 20:37:14,497][105620] Updated weights for policy 1, policy_version 722432 (0.0005) [2023-12-26 20:37:14,549][105692] Updated weights for policy 0, policy_version 721846 (0.0010) [2023-12-26 20:37:14,593][105692] Updated weights for policy 0, policy_version 721856 (0.0010) [2023-12-26 20:37:15,103][105620] Updated weights for policy 1, policy_version 722442 (0.0006) [2023-12-26 20:37:15,170][105620] Updated weights for policy 1, policy_version 722452 (0.0010) [2023-12-26 20:37:15,230][105620] Updated weights for policy 1, policy_version 722462 (0.0011) [2023-12-26 20:37:15,290][105620] Updated weights for policy 1, policy_version 722472 (0.0011) [2023-12-26 20:37:15,296][105692] Updated weights for policy 0, policy_version 721866 (0.0010) [2023-12-26 20:37:15,359][105692] Updated weights for policy 0, policy_version 721876 (0.0011) [2023-12-26 20:37:15,416][105692] Updated weights for policy 0, policy_version 721886 (0.0007) [2023-12-26 20:37:15,465][105692] Updated weights for policy 0, policy_version 721896 (0.0005) [2023-12-26 20:37:16,052][105620] Updated weights for policy 1, policy_version 722482 (0.0010) [2023-12-26 20:37:16,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 369811456. Throughput: 0: 9858.9, 1: 9728.9. Samples: 369782300. Policy #0 lag: (min: 21.0, avg: 25.9, max: 53.0) [2023-12-26 20:37:16,063][104569] Avg episode reward: [(0, '9353.363'), (1, '9262.029')] [2023-12-26 20:37:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000721896_184836096.pth... [2023-12-26 20:37:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000720712_184532992.pth [2023-12-26 20:37:16,114][105620] Updated weights for policy 1, policy_version 722492 (0.0011) [2023-12-26 20:37:16,143][105692] Updated weights for policy 0, policy_version 721906 (0.0010) [2023-12-26 20:37:16,162][105620] Updated weights for policy 1, policy_version 722502 (0.0010) [2023-12-26 20:37:16,169][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000722504_184983552.pth... [2023-12-26 20:37:16,172][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000721384_184696832.pth [2023-12-26 20:37:16,196][105692] Updated weights for policy 0, policy_version 721916 (0.0008) [2023-12-26 20:37:16,251][105692] Updated weights for policy 0, policy_version 721926 (0.0009) [2023-12-26 20:37:16,909][105620] Updated weights for policy 1, policy_version 722512 (0.0010) [2023-12-26 20:37:16,915][105692] Updated weights for policy 0, policy_version 721936 (0.0007) [2023-12-26 20:37:16,954][105620] Updated weights for policy 1, policy_version 722522 (0.0010) [2023-12-26 20:37:16,973][105692] Updated weights for policy 0, policy_version 721946 (0.0006) [2023-12-26 20:37:17,003][105620] Updated weights for policy 1, policy_version 722532 (0.0010) [2023-12-26 20:37:17,025][105692] Updated weights for policy 0, policy_version 721956 (0.0005) [2023-12-26 20:37:17,598][105692] Updated weights for policy 0, policy_version 721966 (0.0005) [2023-12-26 20:37:17,653][105692] Updated weights for policy 0, policy_version 721976 (0.0005) [2023-12-26 20:37:17,662][105620] Updated weights for policy 1, policy_version 722542 (0.0007) [2023-12-26 20:37:17,711][105692] Updated weights for policy 0, policy_version 721986 (0.0006) [2023-12-26 20:37:17,715][105620] Updated weights for policy 1, policy_version 722552 (0.0006) [2023-12-26 20:37:17,765][105620] Updated weights for policy 1, policy_version 722562 (0.0007) [2023-12-26 20:37:18,316][105692] Updated weights for policy 0, policy_version 721996 (0.0011) [2023-12-26 20:37:18,320][105620] Updated weights for policy 1, policy_version 722572 (0.0006) [2023-12-26 20:37:18,381][105620] Updated weights for policy 1, policy_version 722582 (0.0008) [2023-12-26 20:37:18,384][105692] Updated weights for policy 0, policy_version 722006 (0.0010) [2023-12-26 20:37:18,441][105692] Updated weights for policy 0, policy_version 722016 (0.0011) [2023-12-26 20:37:18,444][105620] Updated weights for policy 1, policy_version 722592 (0.0007) [2023-12-26 20:37:19,130][105620] Updated weights for policy 1, policy_version 722602 (0.0006) [2023-12-26 20:37:19,130][105692] Updated weights for policy 0, policy_version 722026 (0.0009) [2023-12-26 20:37:19,177][105620] Updated weights for policy 1, policy_version 722612 (0.0009) [2023-12-26 20:37:19,182][105692] Updated weights for policy 0, policy_version 722036 (0.0011) [2023-12-26 20:37:19,231][105620] Updated weights for policy 1, policy_version 722622 (0.0008) [2023-12-26 20:37:19,245][105692] Updated weights for policy 0, policy_version 722046 (0.0009) [2023-12-26 20:37:19,294][105620] Updated weights for policy 1, policy_version 722632 (0.0007) [2023-12-26 20:37:19,309][105692] Updated weights for policy 0, policy_version 722056 (0.0008) [2023-12-26 20:37:20,061][105692] Updated weights for policy 0, policy_version 722066 (0.0008) [2023-12-26 20:37:20,083][105620] Updated weights for policy 1, policy_version 722642 (0.0008) [2023-12-26 20:37:20,124][105692] Updated weights for policy 0, policy_version 722076 (0.0008) [2023-12-26 20:37:20,135][105620] Updated weights for policy 1, policy_version 722652 (0.0008) [2023-12-26 20:37:20,185][105692] Updated weights for policy 0, policy_version 722086 (0.0007) [2023-12-26 20:37:20,186][105620] Updated weights for policy 1, policy_version 722662 (0.0008) [2023-12-26 20:37:20,867][105620] Updated weights for policy 1, policy_version 722672 (0.0010) [2023-12-26 20:37:20,930][105620] Updated weights for policy 1, policy_version 722682 (0.0008) [2023-12-26 20:37:20,994][105620] Updated weights for policy 1, policy_version 722692 (0.0008) [2023-12-26 20:37:20,996][105692] Updated weights for policy 0, policy_version 722096 (0.0008) [2023-12-26 20:37:21,058][105692] Updated weights for policy 0, policy_version 722106 (0.0008) [2023-12-26 20:37:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 369917952. Throughput: 0: 9936.3, 1: 9709.2. Samples: 369906900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:37:21,063][104569] Avg episode reward: [(0, '9354.018'), (1, '9260.806')] [2023-12-26 20:37:21,128][105692] Updated weights for policy 0, policy_version 722116 (0.0007) [2023-12-26 20:37:21,841][105620] Updated weights for policy 1, policy_version 722702 (0.0008) [2023-12-26 20:37:21,867][105692] Updated weights for policy 0, policy_version 722126 (0.0006) [2023-12-26 20:37:21,898][105620] Updated weights for policy 1, policy_version 722712 (0.0009) [2023-12-26 20:37:21,922][105692] Updated weights for policy 0, policy_version 722136 (0.0007) [2023-12-26 20:37:21,959][105620] Updated weights for policy 1, policy_version 722722 (0.0009) [2023-12-26 20:37:21,976][105692] Updated weights for policy 0, policy_version 722146 (0.0007) [2023-12-26 20:37:22,715][105692] Updated weights for policy 0, policy_version 722156 (0.0008) [2023-12-26 20:37:22,724][105620] Updated weights for policy 1, policy_version 722732 (0.0010) [2023-12-26 20:37:22,765][105692] Updated weights for policy 0, policy_version 722166 (0.0007) [2023-12-26 20:37:22,772][105620] Updated weights for policy 1, policy_version 722742 (0.0007) [2023-12-26 20:37:22,815][105692] Updated weights for policy 0, policy_version 722176 (0.0006) [2023-12-26 20:37:22,822][105620] Updated weights for policy 1, policy_version 722752 (0.0006) [2023-12-26 20:37:23,568][105620] Updated weights for policy 1, policy_version 722762 (0.0008) [2023-12-26 20:37:23,596][105692] Updated weights for policy 0, policy_version 722186 (0.0007) [2023-12-26 20:37:23,627][105620] Updated weights for policy 1, policy_version 722772 (0.0008) [2023-12-26 20:37:23,645][105692] Updated weights for policy 0, policy_version 722196 (0.0006) [2023-12-26 20:37:23,682][105620] Updated weights for policy 1, policy_version 722782 (0.0007) [2023-12-26 20:37:23,695][105692] Updated weights for policy 0, policy_version 722206 (0.0005) [2023-12-26 20:37:23,730][105620] Updated weights for policy 1, policy_version 722792 (0.0005) [2023-12-26 20:37:23,749][105692] Updated weights for policy 0, policy_version 722216 (0.0006) [2023-12-26 20:37:24,345][105620] Updated weights for policy 1, policy_version 722802 (0.0006) [2023-12-26 20:37:24,405][105692] Updated weights for policy 0, policy_version 722226 (0.0005) [2023-12-26 20:37:24,412][105620] Updated weights for policy 1, policy_version 722812 (0.0008) [2023-12-26 20:37:24,455][105692] Updated weights for policy 0, policy_version 722236 (0.0006) [2023-12-26 20:37:24,461][105620] Updated weights for policy 1, policy_version 722822 (0.0010) [2023-12-26 20:37:24,508][105692] Updated weights for policy 0, policy_version 722246 (0.0007) [2023-12-26 20:37:25,103][105620] Updated weights for policy 1, policy_version 722832 (0.0010) [2023-12-26 20:37:25,151][105620] Updated weights for policy 1, policy_version 722842 (0.0010) [2023-12-26 20:37:25,198][105620] Updated weights for policy 1, policy_version 722852 (0.0010) [2023-12-26 20:37:25,230][105692] Updated weights for policy 0, policy_version 722256 (0.0007) [2023-12-26 20:37:25,274][105692] Updated weights for policy 0, policy_version 722266 (0.0008) [2023-12-26 20:37:25,321][105692] Updated weights for policy 0, policy_version 722276 (0.0007) [2023-12-26 20:37:25,975][105620] Updated weights for policy 1, policy_version 722862 (0.0010) [2023-12-26 20:37:26,037][105620] Updated weights for policy 1, policy_version 722872 (0.0010) [2023-12-26 20:37:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.1, 300 sec: 19605.2). Total num frames: 370008064. Throughput: 0: 10030.1, 1: 9628.6. Samples: 370022016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:37:26,064][104569] Avg episode reward: [(0, '9354.392'), (1, '9171.211')] [2023-12-26 20:37:26,088][105620] Updated weights for policy 1, policy_version 722882 (0.0009) [2023-12-26 20:37:26,111][105692] Updated weights for policy 0, policy_version 722286 (0.0007) [2023-12-26 20:37:26,161][105692] Updated weights for policy 0, policy_version 722296 (0.0008) [2023-12-26 20:37:26,209][105692] Updated weights for policy 0, policy_version 722306 (0.0008) [2023-12-26 20:37:26,826][105620] Updated weights for policy 1, policy_version 722892 (0.0010) [2023-12-26 20:37:26,883][105620] Updated weights for policy 1, policy_version 722902 (0.0010) [2023-12-26 20:37:26,917][105692] Updated weights for policy 0, policy_version 722316 (0.0007) [2023-12-26 20:37:26,937][105620] Updated weights for policy 1, policy_version 722912 (0.0010) [2023-12-26 20:37:26,971][105692] Updated weights for policy 0, policy_version 722326 (0.0005) [2023-12-26 20:37:27,025][105692] Updated weights for policy 0, policy_version 722336 (0.0005) [2023-12-26 20:37:27,632][105692] Updated weights for policy 0, policy_version 722346 (0.0006) [2023-12-26 20:37:27,683][105620] Updated weights for policy 1, policy_version 722922 (0.0010) [2023-12-26 20:37:27,692][105692] Updated weights for policy 0, policy_version 722356 (0.0007) [2023-12-26 20:37:27,737][105620] Updated weights for policy 1, policy_version 722932 (0.0010) [2023-12-26 20:37:27,754][105692] Updated weights for policy 0, policy_version 722366 (0.0008) [2023-12-26 20:37:27,791][105620] Updated weights for policy 1, policy_version 722942 (0.0010) [2023-12-26 20:37:27,810][105692] Updated weights for policy 0, policy_version 722376 (0.0006) [2023-12-26 20:37:27,839][105620] Updated weights for policy 1, policy_version 722952 (0.0010) [2023-12-26 20:37:28,511][105692] Updated weights for policy 0, policy_version 722386 (0.0008) [2023-12-26 20:37:28,568][105692] Updated weights for policy 0, policy_version 722396 (0.0006) [2023-12-26 20:37:28,585][105620] Updated weights for policy 1, policy_version 722962 (0.0011) [2023-12-26 20:37:28,624][105692] Updated weights for policy 0, policy_version 722406 (0.0005) [2023-12-26 20:37:28,637][105620] Updated weights for policy 1, policy_version 722972 (0.0010) [2023-12-26 20:37:28,695][105620] Updated weights for policy 1, policy_version 722982 (0.0010) [2023-12-26 20:37:29,318][105692] Updated weights for policy 0, policy_version 722416 (0.0009) [2023-12-26 20:37:29,382][105692] Updated weights for policy 0, policy_version 722426 (0.0011) [2023-12-26 20:37:29,404][105620] Updated weights for policy 1, policy_version 722992 (0.0010) [2023-12-26 20:37:29,434][105692] Updated weights for policy 0, policy_version 722436 (0.0010) [2023-12-26 20:37:29,462][105620] Updated weights for policy 1, policy_version 723002 (0.0010) [2023-12-26 20:37:29,530][105620] Updated weights for policy 1, policy_version 723012 (0.0010) [2023-12-26 20:37:30,040][105692] Updated weights for policy 0, policy_version 722446 (0.0008) [2023-12-26 20:37:30,110][105692] Updated weights for policy 0, policy_version 722456 (0.0007) [2023-12-26 20:37:30,176][105692] Updated weights for policy 0, policy_version 722466 (0.0007) [2023-12-26 20:37:30,205][105620] Updated weights for policy 1, policy_version 723022 (0.0007) [2023-12-26 20:37:30,273][105620] Updated weights for policy 1, policy_version 723032 (0.0005) [2023-12-26 20:37:30,340][105620] Updated weights for policy 1, policy_version 723042 (0.0006) [2023-12-26 20:37:30,795][105692] Updated weights for policy 0, policy_version 722476 (0.0008) [2023-12-26 20:37:30,842][105692] Updated weights for policy 0, policy_version 722486 (0.0010) [2023-12-26 20:37:30,890][105692] Updated weights for policy 0, policy_version 722496 (0.0010) [2023-12-26 20:37:30,895][105620] Updated weights for policy 1, policy_version 723052 (0.0008) [2023-12-26 20:37:30,942][105620] Updated weights for policy 1, policy_version 723062 (0.0007) [2023-12-26 20:37:30,993][105620] Updated weights for policy 1, policy_version 723072 (0.0008) [2023-12-26 20:37:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 370122752. Throughput: 0: 10142.9, 1: 9551.3. Samples: 370080492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:37:31,062][104569] Avg episode reward: [(0, '9354.441'), (1, '9263.467')] [2023-12-26 20:37:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000722504_184991744.pth... [2023-12-26 20:37:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000723080_185131008.pth... [2023-12-26 20:37:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000721320_184688640.pth [2023-12-26 20:37:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000721928_184836096.pth [2023-12-26 20:37:31,694][105692] Updated weights for policy 0, policy_version 722506 (0.0010) [2023-12-26 20:37:31,760][105692] Updated weights for policy 0, policy_version 722516 (0.0008) [2023-12-26 20:37:31,788][105620] Updated weights for policy 1, policy_version 723082 (0.0009) [2023-12-26 20:37:31,828][105692] Updated weights for policy 0, policy_version 722526 (0.0008) [2023-12-26 20:37:31,846][105620] Updated weights for policy 1, policy_version 723092 (0.0006) [2023-12-26 20:37:31,891][105692] Updated weights for policy 0, policy_version 722536 (0.0010) [2023-12-26 20:37:31,908][105620] Updated weights for policy 1, policy_version 723102 (0.0009) [2023-12-26 20:37:31,974][105620] Updated weights for policy 1, policy_version 723112 (0.0008) [2023-12-26 20:37:32,586][105692] Updated weights for policy 0, policy_version 722546 (0.0010) [2023-12-26 20:37:32,640][105692] Updated weights for policy 0, policy_version 722556 (0.0010) [2023-12-26 20:37:32,693][105692] Updated weights for policy 0, policy_version 722566 (0.0009) [2023-12-26 20:37:32,701][105620] Updated weights for policy 1, policy_version 723122 (0.0005) [2023-12-26 20:37:32,753][105620] Updated weights for policy 1, policy_version 723132 (0.0008) [2023-12-26 20:37:32,803][105620] Updated weights for policy 1, policy_version 723142 (0.0008) [2023-12-26 20:37:33,467][105692] Updated weights for policy 0, policy_version 722576 (0.0007) [2023-12-26 20:37:33,510][105620] Updated weights for policy 1, policy_version 723152 (0.0006) [2023-12-26 20:37:33,520][105692] Updated weights for policy 0, policy_version 722586 (0.0009) [2023-12-26 20:37:33,558][105620] Updated weights for policy 1, policy_version 723162 (0.0005) [2023-12-26 20:37:33,573][105692] Updated weights for policy 0, policy_version 722596 (0.0009) [2023-12-26 20:37:33,610][105620] Updated weights for policy 1, policy_version 723172 (0.0007) [2023-12-26 20:37:34,168][105692] Updated weights for policy 0, policy_version 722606 (0.0010) [2023-12-26 20:37:34,232][105692] Updated weights for policy 0, policy_version 722616 (0.0007) [2023-12-26 20:37:34,299][105692] Updated weights for policy 0, policy_version 722626 (0.0006) [2023-12-26 20:37:34,431][105620] Updated weights for policy 1, policy_version 723182 (0.0008) [2023-12-26 20:37:34,488][105620] Updated weights for policy 1, policy_version 723192 (0.0005) [2023-12-26 20:37:34,542][105620] Updated weights for policy 1, policy_version 723202 (0.0009) [2023-12-26 20:37:34,896][105692] Updated weights for policy 0, policy_version 722636 (0.0007) [2023-12-26 20:37:34,943][105692] Updated weights for policy 0, policy_version 722646 (0.0009) [2023-12-26 20:37:34,991][105692] Updated weights for policy 0, policy_version 722657 (0.0009) [2023-12-26 20:37:35,264][105620] Updated weights for policy 1, policy_version 723212 (0.0008) [2023-12-26 20:37:35,311][105620] Updated weights for policy 1, policy_version 723222 (0.0009) [2023-12-26 20:37:35,363][105620] Updated weights for policy 1, policy_version 723232 (0.0009) [2023-12-26 20:37:35,663][105692] Updated weights for policy 0, policy_version 722668 (0.0008) [2023-12-26 20:37:35,721][105692] Updated weights for policy 0, policy_version 722678 (0.0005) [2023-12-26 20:37:35,777][105692] Updated weights for policy 0, policy_version 722688 (0.0006) [2023-12-26 20:37:36,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19660.7, 300 sec: 19633.0). Total num frames: 370212864. Throughput: 0: 10081.4, 1: 9582.5. Samples: 370200680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:37:36,063][104569] Avg episode reward: [(0, '9262.509'), (1, '9267.352')] [2023-12-26 20:37:36,134][105620] Updated weights for policy 1, policy_version 723242 (0.0010) [2023-12-26 20:37:36,188][105620] Updated weights for policy 1, policy_version 723252 (0.0008) [2023-12-26 20:37:36,245][105620] Updated weights for policy 1, policy_version 723262 (0.0008) [2023-12-26 20:37:36,306][105620] Updated weights for policy 1, policy_version 723272 (0.0009) [2023-12-26 20:37:36,493][105692] Updated weights for policy 0, policy_version 722698 (0.0009) [2023-12-26 20:37:36,546][105692] Updated weights for policy 0, policy_version 722708 (0.0011) [2023-12-26 20:37:36,603][105692] Updated weights for policy 0, policy_version 722718 (0.0008) [2023-12-26 20:37:36,659][105692] Updated weights for policy 0, policy_version 722728 (0.0010) [2023-12-26 20:37:37,114][105620] Updated weights for policy 1, policy_version 723282 (0.0009) [2023-12-26 20:37:37,182][105620] Updated weights for policy 1, policy_version 723292 (0.0010) [2023-12-26 20:37:37,244][105620] Updated weights for policy 1, policy_version 723302 (0.0009) [2023-12-26 20:37:37,330][105692] Updated weights for policy 0, policy_version 722738 (0.0009) [2023-12-26 20:37:37,394][105692] Updated weights for policy 0, policy_version 722748 (0.0009) [2023-12-26 20:37:37,447][105692] Updated weights for policy 0, policy_version 722758 (0.0011) [2023-12-26 20:37:37,912][105620] Updated weights for policy 1, policy_version 723312 (0.0006) [2023-12-26 20:37:37,970][105620] Updated weights for policy 1, policy_version 723322 (0.0006) [2023-12-26 20:37:38,021][105620] Updated weights for policy 1, policy_version 723332 (0.0006) [2023-12-26 20:37:38,208][105692] Updated weights for policy 0, policy_version 722768 (0.0006) [2023-12-26 20:37:38,257][105692] Updated weights for policy 0, policy_version 722778 (0.0005) [2023-12-26 20:37:38,303][105692] Updated weights for policy 0, policy_version 722788 (0.0006) [2023-12-26 20:37:38,720][105620] Updated weights for policy 1, policy_version 723342 (0.0007) [2023-12-26 20:37:38,776][105620] Updated weights for policy 1, policy_version 723352 (0.0008) [2023-12-26 20:37:38,835][105620] Updated weights for policy 1, policy_version 723362 (0.0008) [2023-12-26 20:37:38,959][105692] Updated weights for policy 0, policy_version 722798 (0.0009) [2023-12-26 20:37:39,018][105692] Updated weights for policy 0, policy_version 722808 (0.0011) [2023-12-26 20:37:39,066][105692] Updated weights for policy 0, policy_version 722818 (0.0010) [2023-12-26 20:37:39,589][105620] Updated weights for policy 1, policy_version 723372 (0.0009) [2023-12-26 20:37:39,657][105620] Updated weights for policy 1, policy_version 723382 (0.0011) [2023-12-26 20:37:39,716][105620] Updated weights for policy 1, policy_version 723392 (0.0009) [2023-12-26 20:37:39,875][105692] Updated weights for policy 0, policy_version 722828 (0.0010) [2023-12-26 20:37:39,942][105692] Updated weights for policy 0, policy_version 722838 (0.0008) [2023-12-26 20:37:40,013][105692] Updated weights for policy 0, policy_version 722848 (0.0009) [2023-12-26 20:37:40,373][105620] Updated weights for policy 1, policy_version 723402 (0.0006) [2023-12-26 20:37:40,433][105620] Updated weights for policy 1, policy_version 723412 (0.0008) [2023-12-26 20:37:40,496][105620] Updated weights for policy 1, policy_version 723422 (0.0009) [2023-12-26 20:37:40,553][105620] Updated weights for policy 1, policy_version 723432 (0.0010) [2023-12-26 20:37:40,800][105692] Updated weights for policy 0, policy_version 722858 (0.0009) [2023-12-26 20:37:40,863][105692] Updated weights for policy 0, policy_version 722868 (0.0009) [2023-12-26 20:37:40,929][105692] Updated weights for policy 0, policy_version 722878 (0.0009) [2023-12-26 20:37:40,989][105692] Updated weights for policy 0, policy_version 722888 (0.0010) [2023-12-26 20:37:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 370311168. Throughput: 0: 10101.3, 1: 9673.5. Samples: 370317764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:37:41,062][104569] Avg episode reward: [(0, '9354.082'), (1, '9087.645')] [2023-12-26 20:37:41,307][105620] Updated weights for policy 1, policy_version 723442 (0.0009) [2023-12-26 20:37:41,378][105620] Updated weights for policy 1, policy_version 723452 (0.0009) [2023-12-26 20:37:41,441][105620] Updated weights for policy 1, policy_version 723462 (0.0009) [2023-12-26 20:37:41,826][105692] Updated weights for policy 0, policy_version 722898 (0.0009) [2023-12-26 20:37:41,884][105692] Updated weights for policy 0, policy_version 722908 (0.0010) [2023-12-26 20:37:41,948][105692] Updated weights for policy 0, policy_version 722918 (0.0009) [2023-12-26 20:37:42,139][105620] Updated weights for policy 1, policy_version 723472 (0.0007) [2023-12-26 20:37:42,207][105620] Updated weights for policy 1, policy_version 723482 (0.0007) [2023-12-26 20:37:42,266][105620] Updated weights for policy 1, policy_version 723492 (0.0009) [2023-12-26 20:37:42,747][105692] Updated weights for policy 0, policy_version 722928 (0.0007) [2023-12-26 20:37:42,809][105692] Updated weights for policy 0, policy_version 722938 (0.0005) [2023-12-26 20:37:42,869][105692] Updated weights for policy 0, policy_version 722948 (0.0006) [2023-12-26 20:37:43,026][105620] Updated weights for policy 1, policy_version 723502 (0.0008) [2023-12-26 20:37:43,082][105620] Updated weights for policy 1, policy_version 723512 (0.0008) [2023-12-26 20:37:43,143][105620] Updated weights for policy 1, policy_version 723522 (0.0009) [2023-12-26 20:37:43,541][105692] Updated weights for policy 0, policy_version 722958 (0.0008) [2023-12-26 20:37:43,589][105692] Updated weights for policy 0, policy_version 722968 (0.0009) [2023-12-26 20:37:43,636][105692] Updated weights for policy 0, policy_version 722978 (0.0008) [2023-12-26 20:37:43,890][105620] Updated weights for policy 1, policy_version 723532 (0.0010) [2023-12-26 20:37:43,952][105620] Updated weights for policy 1, policy_version 723543 (0.0011) [2023-12-26 20:37:44,009][105620] Updated weights for policy 1, policy_version 723553 (0.0009) [2023-12-26 20:37:44,309][105692] Updated weights for policy 0, policy_version 722988 (0.0008) [2023-12-26 20:37:44,376][105692] Updated weights for policy 0, policy_version 722998 (0.0008) [2023-12-26 20:37:44,438][105692] Updated weights for policy 0, policy_version 723008 (0.0010) [2023-12-26 20:37:44,786][105620] Updated weights for policy 1, policy_version 723563 (0.0009) [2023-12-26 20:37:44,846][105620] Updated weights for policy 1, policy_version 723573 (0.0008) [2023-12-26 20:37:44,910][105620] Updated weights for policy 1, policy_version 723583 (0.0008) [2023-12-26 20:37:45,164][105692] Updated weights for policy 0, policy_version 723018 (0.0009) [2023-12-26 20:37:45,216][105692] Updated weights for policy 0, policy_version 723028 (0.0010) [2023-12-26 20:37:45,276][105692] Updated weights for policy 0, policy_version 723038 (0.0010) [2023-12-26 20:37:45,338][105692] Updated weights for policy 0, policy_version 723048 (0.0011) [2023-12-26 20:37:45,696][105620] Updated weights for policy 1, policy_version 723593 (0.0008) [2023-12-26 20:37:45,740][105620] Updated weights for policy 1, policy_version 723603 (0.0008) [2023-12-26 20:37:45,783][105620] Updated weights for policy 1, policy_version 723613 (0.0008) [2023-12-26 20:37:45,831][105620] Updated weights for policy 1, policy_version 723623 (0.0008) [2023-12-26 20:37:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 370401280. Throughput: 0: 9960.9, 1: 9724.9. Samples: 370373240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:37:46,063][104569] Avg episode reward: [(0, '9263.322'), (1, '9174.435')] [2023-12-26 20:37:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000723624_185270272.pth... [2023-12-26 20:37:46,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000722504_184983552.pth [2023-12-26 20:37:46,098][105692] Updated weights for policy 0, policy_version 723058 (0.0010) [2023-12-26 20:37:46,153][105692] Updated weights for policy 0, policy_version 723068 (0.0010) [2023-12-26 20:37:46,207][105692] Updated weights for policy 0, policy_version 723078 (0.0010) [2023-12-26 20:37:46,213][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000723080_185139200.pth... [2023-12-26 20:37:46,216][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000721896_184836096.pth [2023-12-26 20:37:46,593][105620] Updated weights for policy 1, policy_version 723633 (0.0011) [2023-12-26 20:37:46,655][105620] Updated weights for policy 1, policy_version 723643 (0.0007) [2023-12-26 20:37:46,718][105620] Updated weights for policy 1, policy_version 723653 (0.0008) [2023-12-26 20:37:46,818][105692] Updated weights for policy 0, policy_version 723088 (0.0009) [2023-12-26 20:37:46,862][105692] Updated weights for policy 0, policy_version 723098 (0.0010) [2023-12-26 20:37:46,913][105692] Updated weights for policy 0, policy_version 723108 (0.0010) [2023-12-26 20:37:47,316][105620] Updated weights for policy 1, policy_version 723663 (0.0007) [2023-12-26 20:37:47,377][105620] Updated weights for policy 1, policy_version 723673 (0.0006) [2023-12-26 20:37:47,427][105620] Updated weights for policy 1, policy_version 723683 (0.0005) [2023-12-26 20:37:47,615][105692] Updated weights for policy 0, policy_version 723118 (0.0010) [2023-12-26 20:37:47,665][105692] Updated weights for policy 0, policy_version 723128 (0.0009) [2023-12-26 20:37:47,711][105692] Updated weights for policy 0, policy_version 723138 (0.0005) [2023-12-26 20:37:48,066][105620] Updated weights for policy 1, policy_version 723693 (0.0008) [2023-12-26 20:37:48,121][105620] Updated weights for policy 1, policy_version 723703 (0.0007) [2023-12-26 20:37:48,180][105620] Updated weights for policy 1, policy_version 723713 (0.0008) [2023-12-26 20:37:48,360][105692] Updated weights for policy 0, policy_version 723148 (0.0007) [2023-12-26 20:37:48,419][105692] Updated weights for policy 0, policy_version 723158 (0.0010) [2023-12-26 20:37:48,478][105692] Updated weights for policy 0, policy_version 723168 (0.0010) [2023-12-26 20:37:48,800][105620] Updated weights for policy 1, policy_version 723723 (0.0007) [2023-12-26 20:37:48,859][105620] Updated weights for policy 1, policy_version 723733 (0.0005) [2023-12-26 20:37:48,918][105620] Updated weights for policy 1, policy_version 723743 (0.0005) [2023-12-26 20:37:49,254][105692] Updated weights for policy 0, policy_version 723178 (0.0011) [2023-12-26 20:37:49,309][105692] Updated weights for policy 0, policy_version 723188 (0.0011) [2023-12-26 20:37:49,382][105692] Updated weights for policy 0, policy_version 723198 (0.0011) [2023-12-26 20:37:49,437][105692] Updated weights for policy 0, policy_version 723208 (0.0011) [2023-12-26 20:37:49,519][105620] Updated weights for policy 1, policy_version 723753 (0.0005) [2023-12-26 20:37:49,585][105620] Updated weights for policy 1, policy_version 723763 (0.0008) [2023-12-26 20:37:49,644][105620] Updated weights for policy 1, policy_version 723773 (0.0005) [2023-12-26 20:37:49,706][105620] Updated weights for policy 1, policy_version 723783 (0.0010) [2023-12-26 20:37:50,230][105692] Updated weights for policy 0, policy_version 723218 (0.0010) [2023-12-26 20:37:50,292][105692] Updated weights for policy 0, policy_version 723228 (0.0010) [2023-12-26 20:37:50,341][105620] Updated weights for policy 1, policy_version 723793 (0.0007) [2023-12-26 20:37:50,347][105692] Updated weights for policy 0, policy_version 723238 (0.0010) [2023-12-26 20:37:50,401][105620] Updated weights for policy 1, policy_version 723803 (0.0006) [2023-12-26 20:37:50,460][105620] Updated weights for policy 1, policy_version 723813 (0.0007) [2023-12-26 20:37:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 370499584. Throughput: 0: 9925.6, 1: 9782.0. Samples: 370493772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:37:51,063][104569] Avg episode reward: [(0, '9263.581'), (1, '8783.433')] [2023-12-26 20:37:51,073][105692] Updated weights for policy 0, policy_version 723248 (0.0012) [2023-12-26 20:37:51,143][105692] Updated weights for policy 0, policy_version 723258 (0.0010) [2023-12-26 20:37:51,150][105620] Updated weights for policy 1, policy_version 723823 (0.0009) [2023-12-26 20:37:51,202][105692] Updated weights for policy 0, policy_version 723268 (0.0008) [2023-12-26 20:37:51,208][105620] Updated weights for policy 1, policy_version 723833 (0.0011) [2023-12-26 20:37:51,270][105620] Updated weights for policy 1, policy_version 723843 (0.0011) [2023-12-26 20:37:51,960][105692] Updated weights for policy 0, policy_version 723278 (0.0007) [2023-12-26 20:37:52,006][105692] Updated weights for policy 0, policy_version 723288 (0.0009) [2023-12-26 20:37:52,010][105620] Updated weights for policy 1, policy_version 723853 (0.0009) [2023-12-26 20:37:52,059][105692] Updated weights for policy 0, policy_version 723298 (0.0009) [2023-12-26 20:37:52,072][105620] Updated weights for policy 1, policy_version 723863 (0.0007) [2023-12-26 20:37:52,138][105620] Updated weights for policy 1, policy_version 723873 (0.0008) [2023-12-26 20:37:52,755][105620] Updated weights for policy 1, policy_version 723883 (0.0006) [2023-12-26 20:37:52,816][105620] Updated weights for policy 1, policy_version 723893 (0.0005) [2023-12-26 20:37:52,876][105620] Updated weights for policy 1, policy_version 723903 (0.0005) [2023-12-26 20:37:52,944][105692] Updated weights for policy 0, policy_version 723308 (0.0008) [2023-12-26 20:37:53,006][105692] Updated weights for policy 0, policy_version 723318 (0.0009) [2023-12-26 20:37:53,069][105692] Updated weights for policy 0, policy_version 723328 (0.0009) [2023-12-26 20:37:53,429][105620] Updated weights for policy 1, policy_version 723913 (0.0009) [2023-12-26 20:37:53,489][105620] Updated weights for policy 1, policy_version 723923 (0.0009) [2023-12-26 20:37:53,552][105620] Updated weights for policy 1, policy_version 723933 (0.0010) [2023-12-26 20:37:53,606][105620] Updated weights for policy 1, policy_version 723943 (0.0009) [2023-12-26 20:37:53,843][105692] Updated weights for policy 0, policy_version 723338 (0.0009) [2023-12-26 20:37:53,903][105692] Updated weights for policy 0, policy_version 723348 (0.0009) [2023-12-26 20:37:53,961][105692] Updated weights for policy 0, policy_version 723358 (0.0009) [2023-12-26 20:37:54,021][105692] Updated weights for policy 0, policy_version 723368 (0.0008) [2023-12-26 20:37:54,384][105620] Updated weights for policy 1, policy_version 723953 (0.0010) [2023-12-26 20:37:54,434][105620] Updated weights for policy 1, policy_version 723963 (0.0010) [2023-12-26 20:37:54,491][105620] Updated weights for policy 1, policy_version 723973 (0.0010) [2023-12-26 20:37:54,764][105692] Updated weights for policy 0, policy_version 723378 (0.0005) [2023-12-26 20:37:54,812][105692] Updated weights for policy 0, policy_version 723388 (0.0005) [2023-12-26 20:37:54,858][105692] Updated weights for policy 0, policy_version 723398 (0.0005) [2023-12-26 20:37:55,125][105620] Updated weights for policy 1, policy_version 723983 (0.0007) [2023-12-26 20:37:55,172][105620] Updated weights for policy 1, policy_version 723993 (0.0005) [2023-12-26 20:37:55,227][105620] Updated weights for policy 1, policy_version 724003 (0.0006) [2023-12-26 20:37:55,483][105692] Updated weights for policy 0, policy_version 723408 (0.0008) [2023-12-26 20:37:55,538][105692] Updated weights for policy 0, policy_version 723418 (0.0010) [2023-12-26 20:37:55,595][105692] Updated weights for policy 0, policy_version 723428 (0.0010) [2023-12-26 20:37:55,827][105620] Updated weights for policy 1, policy_version 724013 (0.0008) [2023-12-26 20:37:55,878][105620] Updated weights for policy 1, policy_version 724023 (0.0010) [2023-12-26 20:37:55,925][105620] Updated weights for policy 1, policy_version 724033 (0.0010) [2023-12-26 20:37:56,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 370606080. Throughput: 0: 9838.0, 1: 9877.7. Samples: 370612084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:37:56,063][104569] Avg episode reward: [(0, '9355.765'), (1, '8874.938')] [2023-12-26 20:37:56,206][105692] Updated weights for policy 0, policy_version 723438 (0.0009) [2023-12-26 20:37:56,260][105692] Updated weights for policy 0, policy_version 723448 (0.0010) [2023-12-26 20:37:56,321][105692] Updated weights for policy 0, policy_version 723458 (0.0005) [2023-12-26 20:37:56,598][105620] Updated weights for policy 1, policy_version 724043 (0.0008) [2023-12-26 20:37:56,657][105620] Updated weights for policy 1, policy_version 724053 (0.0007) [2023-12-26 20:37:56,712][105620] Updated weights for policy 1, policy_version 724063 (0.0009) [2023-12-26 20:37:56,869][105692] Updated weights for policy 0, policy_version 723468 (0.0005) [2023-12-26 20:37:56,922][105692] Updated weights for policy 0, policy_version 723478 (0.0005) [2023-12-26 20:37:56,968][105692] Updated weights for policy 0, policy_version 723488 (0.0005) [2023-12-26 20:37:57,402][105620] Updated weights for policy 1, policy_version 724073 (0.0006) [2023-12-26 20:37:57,454][105620] Updated weights for policy 1, policy_version 724083 (0.0009) [2023-12-26 20:37:57,507][105620] Updated weights for policy 1, policy_version 724093 (0.0009) [2023-12-26 20:37:57,533][105692] Updated weights for policy 0, policy_version 723498 (0.0006) [2023-12-26 20:37:57,560][105620] Updated weights for policy 1, policy_version 724103 (0.0008) [2023-12-26 20:37:57,586][105692] Updated weights for policy 0, policy_version 723508 (0.0008) [2023-12-26 20:37:57,640][105692] Updated weights for policy 0, policy_version 723518 (0.0009) [2023-12-26 20:37:57,702][105692] Updated weights for policy 0, policy_version 723528 (0.0009) [2023-12-26 20:37:58,241][105620] Updated weights for policy 1, policy_version 724113 (0.0008) [2023-12-26 20:37:58,296][105620] Updated weights for policy 1, policy_version 724123 (0.0009) [2023-12-26 20:37:58,361][105620] Updated weights for policy 1, policy_version 724133 (0.0009) [2023-12-26 20:37:58,403][105692] Updated weights for policy 0, policy_version 723538 (0.0009) [2023-12-26 20:37:58,467][105692] Updated weights for policy 0, policy_version 723548 (0.0009) [2023-12-26 20:37:58,526][105692] Updated weights for policy 0, policy_version 723558 (0.0007) [2023-12-26 20:37:59,163][105620] Updated weights for policy 1, policy_version 724143 (0.0008) [2023-12-26 20:37:59,222][105620] Updated weights for policy 1, policy_version 724153 (0.0008) [2023-12-26 20:37:59,288][105620] Updated weights for policy 1, policy_version 724163 (0.0011) [2023-12-26 20:37:59,368][105692] Updated weights for policy 0, policy_version 723568 (0.0009) [2023-12-26 20:37:59,428][105692] Updated weights for policy 0, policy_version 723578 (0.0006) [2023-12-26 20:37:59,492][105692] Updated weights for policy 0, policy_version 723588 (0.0009) [2023-12-26 20:38:00,044][105620] Updated weights for policy 1, policy_version 724173 (0.0010) [2023-12-26 20:38:00,089][105620] Updated weights for policy 1, policy_version 724183 (0.0010) [2023-12-26 20:38:00,148][105620] Updated weights for policy 1, policy_version 724193 (0.0010) [2023-12-26 20:38:00,230][105692] Updated weights for policy 0, policy_version 723598 (0.0009) [2023-12-26 20:38:00,277][105692] Updated weights for policy 0, policy_version 723608 (0.0010) [2023-12-26 20:38:00,328][105692] Updated weights for policy 0, policy_version 723618 (0.0009) [2023-12-26 20:38:00,912][105620] Updated weights for policy 1, policy_version 724203 (0.0010) [2023-12-26 20:38:00,929][105692] Updated weights for policy 0, policy_version 723628 (0.0008) [2023-12-26 20:38:00,960][105620] Updated weights for policy 1, policy_version 724213 (0.0010) [2023-12-26 20:38:00,980][105692] Updated weights for policy 0, policy_version 723638 (0.0005) [2023-12-26 20:38:01,016][105620] Updated weights for policy 1, policy_version 724223 (0.0009) [2023-12-26 20:38:01,041][105692] Updated weights for policy 0, policy_version 723648 (0.0007) [2023-12-26 20:38:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 370696192. Throughput: 0: 9916.1, 1: 9918.5. Samples: 370674856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:38:01,063][104569] Avg episode reward: [(0, '9355.950'), (1, '9356.334')] [2023-12-26 20:38:01,074][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000724232_185425920.pth... [2023-12-26 20:38:01,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000723080_185131008.pth [2023-12-26 20:38:01,084][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000723656_185286656.pth... [2023-12-26 20:38:01,087][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000722504_184991744.pth [2023-12-26 20:38:01,785][105620] Updated weights for policy 1, policy_version 724233 (0.0006) [2023-12-26 20:38:01,842][105620] Updated weights for policy 1, policy_version 724243 (0.0005) [2023-12-26 20:38:01,843][105692] Updated weights for policy 0, policy_version 723658 (0.0009) [2023-12-26 20:38:01,900][105692] Updated weights for policy 0, policy_version 723668 (0.0006) [2023-12-26 20:38:01,911][105620] Updated weights for policy 1, policy_version 724253 (0.0009) [2023-12-26 20:38:01,959][105692] Updated weights for policy 0, policy_version 723678 (0.0005) [2023-12-26 20:38:01,967][105620] Updated weights for policy 1, policy_version 724263 (0.0010) [2023-12-26 20:38:02,016][105692] Updated weights for policy 0, policy_version 723688 (0.0005) [2023-12-26 20:38:02,598][105620] Updated weights for policy 1, policy_version 724273 (0.0009) [2023-12-26 20:38:02,621][105692] Updated weights for policy 0, policy_version 723698 (0.0006) [2023-12-26 20:38:02,658][105620] Updated weights for policy 1, policy_version 724283 (0.0011) [2023-12-26 20:38:02,673][105692] Updated weights for policy 0, policy_version 723708 (0.0006) [2023-12-26 20:38:02,722][105620] Updated weights for policy 1, policy_version 724293 (0.0011) [2023-12-26 20:38:02,729][105692] Updated weights for policy 0, policy_version 723718 (0.0008) [2023-12-26 20:38:03,403][105620] Updated weights for policy 1, policy_version 724303 (0.0008) [2023-12-26 20:38:03,471][105620] Updated weights for policy 1, policy_version 724313 (0.0008) [2023-12-26 20:38:03,478][105692] Updated weights for policy 0, policy_version 723728 (0.0008) [2023-12-26 20:38:03,531][105692] Updated weights for policy 0, policy_version 723738 (0.0009) [2023-12-26 20:38:03,541][105620] Updated weights for policy 1, policy_version 724323 (0.0009) [2023-12-26 20:38:03,577][105692] Updated weights for policy 0, policy_version 723748 (0.0008) [2023-12-26 20:38:04,213][105620] Updated weights for policy 1, policy_version 724333 (0.0008) [2023-12-26 20:38:04,280][105620] Updated weights for policy 1, policy_version 724343 (0.0008) [2023-12-26 20:38:04,302][105692] Updated weights for policy 0, policy_version 723758 (0.0008) [2023-12-26 20:38:04,341][105620] Updated weights for policy 1, policy_version 724353 (0.0009) [2023-12-26 20:38:04,360][105692] Updated weights for policy 0, policy_version 723768 (0.0006) [2023-12-26 20:38:04,420][105692] Updated weights for policy 0, policy_version 723778 (0.0008) [2023-12-26 20:38:04,903][105620] Updated weights for policy 1, policy_version 724363 (0.0011) [2023-12-26 20:38:04,955][105620] Updated weights for policy 1, policy_version 724373 (0.0010) [2023-12-26 20:38:05,015][105620] Updated weights for policy 1, policy_version 724383 (0.0011) [2023-12-26 20:38:05,234][105692] Updated weights for policy 0, policy_version 723788 (0.0009) [2023-12-26 20:38:05,285][105692] Updated weights for policy 0, policy_version 723798 (0.0007) [2023-12-26 20:38:05,337][105692] Updated weights for policy 0, policy_version 723808 (0.0007) [2023-12-26 20:38:05,708][105620] Updated weights for policy 1, policy_version 724393 (0.0010) [2023-12-26 20:38:05,758][105620] Updated weights for policy 1, policy_version 724403 (0.0005) [2023-12-26 20:38:05,826][105620] Updated weights for policy 1, policy_version 724413 (0.0005) [2023-12-26 20:38:05,894][105620] Updated weights for policy 1, policy_version 724423 (0.0005) [2023-12-26 20:38:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.7, 300 sec: 19633.0). Total num frames: 370802688. Throughput: 0: 9795.9, 1: 9893.2. Samples: 370792912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:38:06,063][104569] Avg episode reward: [(0, '9007.688'), (1, '9266.376')] [2023-12-26 20:38:06,074][105692] Updated weights for policy 0, policy_version 723818 (0.0008) [2023-12-26 20:38:06,139][105692] Updated weights for policy 0, policy_version 723828 (0.0009) [2023-12-26 20:38:06,199][105692] Updated weights for policy 0, policy_version 723838 (0.0008) [2023-12-26 20:38:06,260][105692] Updated weights for policy 0, policy_version 723848 (0.0009) [2023-12-26 20:38:06,484][105620] Updated weights for policy 1, policy_version 724433 (0.0010) [2023-12-26 20:38:06,536][105620] Updated weights for policy 1, policy_version 724443 (0.0010) [2023-12-26 20:38:06,594][105620] Updated weights for policy 1, policy_version 724453 (0.0008) [2023-12-26 20:38:07,008][105692] Updated weights for policy 0, policy_version 723858 (0.0011) [2023-12-26 20:38:07,067][105692] Updated weights for policy 0, policy_version 723868 (0.0010) [2023-12-26 20:38:07,128][105692] Updated weights for policy 0, policy_version 723878 (0.0005) [2023-12-26 20:38:07,289][105620] Updated weights for policy 1, policy_version 724463 (0.0007) [2023-12-26 20:38:07,353][105620] Updated weights for policy 1, policy_version 724473 (0.0005) [2023-12-26 20:38:07,414][105620] Updated weights for policy 1, policy_version 724483 (0.0008) [2023-12-26 20:38:07,838][105692] Updated weights for policy 0, policy_version 723888 (0.0009) [2023-12-26 20:38:07,896][105692] Updated weights for policy 0, policy_version 723898 (0.0010) [2023-12-26 20:38:07,952][105692] Updated weights for policy 0, policy_version 723908 (0.0010) [2023-12-26 20:38:08,022][105620] Updated weights for policy 1, policy_version 724493 (0.0007) [2023-12-26 20:38:08,078][105620] Updated weights for policy 1, policy_version 724503 (0.0008) [2023-12-26 20:38:08,126][105620] Updated weights for policy 1, policy_version 724513 (0.0006) [2023-12-26 20:38:08,653][105692] Updated weights for policy 0, policy_version 723918 (0.0007) [2023-12-26 20:38:08,674][105620] Updated weights for policy 1, policy_version 724523 (0.0005) [2023-12-26 20:38:08,724][105692] Updated weights for policy 0, policy_version 723928 (0.0006) [2023-12-26 20:38:08,740][105620] Updated weights for policy 1, policy_version 724533 (0.0006) [2023-12-26 20:38:08,787][105692] Updated weights for policy 0, policy_version 723938 (0.0006) [2023-12-26 20:38:08,801][105620] Updated weights for policy 1, policy_version 724543 (0.0007) [2023-12-26 20:38:09,429][105692] Updated weights for policy 0, policy_version 723948 (0.0011) [2023-12-26 20:38:09,494][105692] Updated weights for policy 0, policy_version 723958 (0.0011) [2023-12-26 20:38:09,535][105620] Updated weights for policy 1, policy_version 724553 (0.0007) [2023-12-26 20:38:09,554][105692] Updated weights for policy 0, policy_version 723968 (0.0011) [2023-12-26 20:38:09,590][105620] Updated weights for policy 1, policy_version 724563 (0.0008) [2023-12-26 20:38:09,646][105620] Updated weights for policy 1, policy_version 724573 (0.0008) [2023-12-26 20:38:09,705][105620] Updated weights for policy 1, policy_version 724583 (0.0008) [2023-12-26 20:38:10,301][105692] Updated weights for policy 0, policy_version 723978 (0.0010) [2023-12-26 20:38:10,348][105692] Updated weights for policy 0, policy_version 723988 (0.0009) [2023-12-26 20:38:10,396][105692] Updated weights for policy 0, policy_version 723998 (0.0009) [2023-12-26 20:38:10,458][105692] Updated weights for policy 0, policy_version 724008 (0.0009) [2023-12-26 20:38:10,489][105620] Updated weights for policy 1, policy_version 724593 (0.0007) [2023-12-26 20:38:10,543][105620] Updated weights for policy 1, policy_version 724603 (0.0006) [2023-12-26 20:38:10,608][105620] Updated weights for policy 1, policy_version 724613 (0.0006) [2023-12-26 20:38:11,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 370900992. Throughput: 0: 9811.6, 1: 9968.7. Samples: 370912120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:38:11,062][104569] Avg episode reward: [(0, '9007.808'), (1, '9178.248')] [2023-12-26 20:38:11,252][105620] Updated weights for policy 1, policy_version 724623 (0.0007) [2023-12-26 20:38:11,319][105620] Updated weights for policy 1, policy_version 724633 (0.0007) [2023-12-26 20:38:11,392][105620] Updated weights for policy 1, policy_version 724643 (0.0010) [2023-12-26 20:38:11,405][105692] Updated weights for policy 0, policy_version 724018 (0.0010) [2023-12-26 20:38:11,468][105692] Updated weights for policy 0, policy_version 724028 (0.0008) [2023-12-26 20:38:11,541][105692] Updated weights for policy 0, policy_version 724038 (0.0008) [2023-12-26 20:38:12,029][105620] Updated weights for policy 1, policy_version 724653 (0.0008) [2023-12-26 20:38:12,087][105620] Updated weights for policy 1, policy_version 724663 (0.0006) [2023-12-26 20:38:12,159][105620] Updated weights for policy 1, policy_version 724673 (0.0010) [2023-12-26 20:38:12,323][105692] Updated weights for policy 0, policy_version 724048 (0.0011) [2023-12-26 20:38:12,391][105692] Updated weights for policy 0, policy_version 724058 (0.0009) [2023-12-26 20:38:12,447][105692] Updated weights for policy 0, policy_version 724068 (0.0008) [2023-12-26 20:38:12,849][105620] Updated weights for policy 1, policy_version 724683 (0.0010) [2023-12-26 20:38:12,910][105620] Updated weights for policy 1, policy_version 724693 (0.0010) [2023-12-26 20:38:12,964][105620] Updated weights for policy 1, policy_version 724703 (0.0010) [2023-12-26 20:38:13,243][105692] Updated weights for policy 0, policy_version 724078 (0.0009) [2023-12-26 20:38:13,293][105692] Updated weights for policy 0, policy_version 724088 (0.0008) [2023-12-26 20:38:13,341][105692] Updated weights for policy 0, policy_version 724098 (0.0007) [2023-12-26 20:38:13,689][105620] Updated weights for policy 1, policy_version 724713 (0.0010) [2023-12-26 20:38:13,738][105620] Updated weights for policy 1, policy_version 724723 (0.0005) [2023-12-26 20:38:13,785][105620] Updated weights for policy 1, policy_version 724733 (0.0005) [2023-12-26 20:38:13,839][105620] Updated weights for policy 1, policy_version 724743 (0.0006) [2023-12-26 20:38:13,951][105692] Updated weights for policy 0, policy_version 724108 (0.0009) [2023-12-26 20:38:14,006][105692] Updated weights for policy 0, policy_version 724118 (0.0009) [2023-12-26 20:38:14,051][105692] Updated weights for policy 0, policy_version 724128 (0.0010) [2023-12-26 20:38:14,517][105620] Updated weights for policy 1, policy_version 724753 (0.0009) [2023-12-26 20:38:14,572][105620] Updated weights for policy 1, policy_version 724763 (0.0008) [2023-12-26 20:38:14,629][105620] Updated weights for policy 1, policy_version 724773 (0.0008) [2023-12-26 20:38:14,840][105692] Updated weights for policy 0, policy_version 724138 (0.0009) [2023-12-26 20:38:14,905][105692] Updated weights for policy 0, policy_version 724148 (0.0007) [2023-12-26 20:38:14,967][105692] Updated weights for policy 0, policy_version 724158 (0.0008) [2023-12-26 20:38:15,031][105692] Updated weights for policy 0, policy_version 724168 (0.0008) [2023-12-26 20:38:15,411][105620] Updated weights for policy 1, policy_version 724783 (0.0009) [2023-12-26 20:38:15,482][105620] Updated weights for policy 1, policy_version 724793 (0.0010) [2023-12-26 20:38:15,542][105620] Updated weights for policy 1, policy_version 724803 (0.0009) [2023-12-26 20:38:15,701][105692] Updated weights for policy 0, policy_version 724178 (0.0010) [2023-12-26 20:38:15,749][105692] Updated weights for policy 0, policy_version 724188 (0.0010) [2023-12-26 20:38:15,797][105692] Updated weights for policy 0, policy_version 724198 (0.0010) [2023-12-26 20:38:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 370999296. Throughput: 0: 9727.3, 1: 10004.6. Samples: 370968424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:38:16,062][104569] Avg episode reward: [(0, '9004.746'), (1, '9177.115')] [2023-12-26 20:38:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000724808_185573376.pth... [2023-12-26 20:38:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000724200_185425920.pth... [2023-12-26 20:38:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000723624_185270272.pth [2023-12-26 20:38:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000723080_185139200.pth [2023-12-26 20:38:16,176][105620] Updated weights for policy 1, policy_version 724813 (0.0008) [2023-12-26 20:38:16,234][105620] Updated weights for policy 1, policy_version 724823 (0.0009) [2023-12-26 20:38:16,288][105620] Updated weights for policy 1, policy_version 724833 (0.0010) [2023-12-26 20:38:16,498][105692] Updated weights for policy 0, policy_version 724208 (0.0011) [2023-12-26 20:38:16,557][105692] Updated weights for policy 0, policy_version 724218 (0.0010) [2023-12-26 20:38:16,612][105692] Updated weights for policy 0, policy_version 724228 (0.0010) [2023-12-26 20:38:16,990][105620] Updated weights for policy 1, policy_version 724843 (0.0008) [2023-12-26 20:38:17,037][105620] Updated weights for policy 1, policy_version 724853 (0.0007) [2023-12-26 20:38:17,087][105620] Updated weights for policy 1, policy_version 724863 (0.0006) [2023-12-26 20:38:17,367][105692] Updated weights for policy 0, policy_version 724238 (0.0007) [2023-12-26 20:38:17,430][105692] Updated weights for policy 0, policy_version 724248 (0.0005) [2023-12-26 20:38:17,479][105692] Updated weights for policy 0, policy_version 724258 (0.0005) [2023-12-26 20:38:17,716][105620] Updated weights for policy 1, policy_version 724873 (0.0008) [2023-12-26 20:38:17,775][105620] Updated weights for policy 1, policy_version 724883 (0.0009) [2023-12-26 20:38:17,833][105620] Updated weights for policy 1, policy_version 724894 (0.0010) [2023-12-26 20:38:17,891][105620] Updated weights for policy 1, policy_version 724904 (0.0010) [2023-12-26 20:38:18,002][105692] Updated weights for policy 0, policy_version 724268 (0.0007) [2023-12-26 20:38:18,059][105692] Updated weights for policy 0, policy_version 724278 (0.0009) [2023-12-26 20:38:18,117][105692] Updated weights for policy 0, policy_version 724288 (0.0005) [2023-12-26 20:38:18,649][105620] Updated weights for policy 1, policy_version 724914 (0.0009) [2023-12-26 20:38:18,711][105620] Updated weights for policy 1, policy_version 724924 (0.0006) [2023-12-26 20:38:18,774][105620] Updated weights for policy 1, policy_version 724934 (0.0009) [2023-12-26 20:38:18,876][105692] Updated weights for policy 0, policy_version 724298 (0.0006) [2023-12-26 20:38:18,930][105692] Updated weights for policy 0, policy_version 724308 (0.0009) [2023-12-26 20:38:18,983][105692] Updated weights for policy 0, policy_version 724318 (0.0008) [2023-12-26 20:38:19,039][105692] Updated weights for policy 0, policy_version 724328 (0.0007) [2023-12-26 20:38:19,542][105620] Updated weights for policy 1, policy_version 724944 (0.0009) [2023-12-26 20:38:19,609][105620] Updated weights for policy 1, policy_version 724954 (0.0006) [2023-12-26 20:38:19,669][105620] Updated weights for policy 1, policy_version 724964 (0.0008) [2023-12-26 20:38:19,717][105692] Updated weights for policy 0, policy_version 724338 (0.0009) [2023-12-26 20:38:19,772][105692] Updated weights for policy 0, policy_version 724348 (0.0008) [2023-12-26 20:38:19,835][105692] Updated weights for policy 0, policy_version 724358 (0.0007) [2023-12-26 20:38:20,498][105620] Updated weights for policy 1, policy_version 724974 (0.0008) [2023-12-26 20:38:20,556][105620] Updated weights for policy 1, policy_version 724984 (0.0008) [2023-12-26 20:38:20,567][105692] Updated weights for policy 0, policy_version 724368 (0.0007) [2023-12-26 20:38:20,616][105620] Updated weights for policy 1, policy_version 724994 (0.0007) [2023-12-26 20:38:20,632][105692] Updated weights for policy 0, policy_version 724378 (0.0008) [2023-12-26 20:38:20,690][105692] Updated weights for policy 0, policy_version 724388 (0.0008) [2023-12-26 20:38:21,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 371097600. Throughput: 0: 9724.7, 1: 9998.5. Samples: 371088224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:38:21,063][104569] Avg episode reward: [(0, '9080.472'), (1, '9173.246')] [2023-12-26 20:38:21,422][105620] Updated weights for policy 1, policy_version 725004 (0.0007) [2023-12-26 20:38:21,464][105692] Updated weights for policy 0, policy_version 724398 (0.0009) [2023-12-26 20:38:21,477][105620] Updated weights for policy 1, policy_version 725014 (0.0006) [2023-12-26 20:38:21,529][105692] Updated weights for policy 0, policy_version 724408 (0.0008) [2023-12-26 20:38:21,532][105620] Updated weights for policy 1, policy_version 725024 (0.0006) [2023-12-26 20:38:21,599][105692] Updated weights for policy 0, policy_version 724418 (0.0009) [2023-12-26 20:38:22,300][105620] Updated weights for policy 1, policy_version 725034 (0.0006) [2023-12-26 20:38:22,307][105692] Updated weights for policy 0, policy_version 724428 (0.0008) [2023-12-26 20:38:22,367][105692] Updated weights for policy 0, policy_version 724438 (0.0007) [2023-12-26 20:38:22,368][105620] Updated weights for policy 1, policy_version 725044 (0.0008) [2023-12-26 20:38:22,432][105620] Updated weights for policy 1, policy_version 725054 (0.0006) [2023-12-26 20:38:22,434][105692] Updated weights for policy 0, policy_version 724448 (0.0008) [2023-12-26 20:38:22,492][105620] Updated weights for policy 1, policy_version 725064 (0.0006) [2023-12-26 20:38:23,182][105692] Updated weights for policy 0, policy_version 724458 (0.0009) [2023-12-26 20:38:23,229][105620] Updated weights for policy 1, policy_version 725074 (0.0008) [2023-12-26 20:38:23,239][105692] Updated weights for policy 0, policy_version 724468 (0.0006) [2023-12-26 20:38:23,285][105620] Updated weights for policy 1, policy_version 725084 (0.0006) [2023-12-26 20:38:23,299][105692] Updated weights for policy 0, policy_version 724478 (0.0007) [2023-12-26 20:38:23,342][105620] Updated weights for policy 1, policy_version 725094 (0.0008) [2023-12-26 20:38:23,352][105692] Updated weights for policy 0, policy_version 724488 (0.0006) [2023-12-26 20:38:24,077][105692] Updated weights for policy 0, policy_version 724498 (0.0008) [2023-12-26 20:38:24,111][105620] Updated weights for policy 1, policy_version 725104 (0.0008) [2023-12-26 20:38:24,130][105692] Updated weights for policy 0, policy_version 724508 (0.0006) [2023-12-26 20:38:24,170][105620] Updated weights for policy 1, policy_version 725114 (0.0007) [2023-12-26 20:38:24,180][105692] Updated weights for policy 0, policy_version 724518 (0.0007) [2023-12-26 20:38:24,236][105620] Updated weights for policy 1, policy_version 725124 (0.0006) [2023-12-26 20:38:24,954][105620] Updated weights for policy 1, policy_version 725134 (0.0008) [2023-12-26 20:38:24,956][105692] Updated weights for policy 0, policy_version 724528 (0.0008) [2023-12-26 20:38:25,009][105692] Updated weights for policy 0, policy_version 724538 (0.0006) [2023-12-26 20:38:25,015][105620] Updated weights for policy 1, policy_version 725144 (0.0009) [2023-12-26 20:38:25,068][105620] Updated weights for policy 1, policy_version 725154 (0.0006) [2023-12-26 20:38:25,074][105692] Updated weights for policy 0, policy_version 724548 (0.0007) [2023-12-26 20:38:25,708][105620] Updated weights for policy 1, policy_version 725164 (0.0007) [2023-12-26 20:38:25,758][105620] Updated weights for policy 1, policy_version 725174 (0.0007) [2023-12-26 20:38:25,803][105620] Updated weights for policy 1, policy_version 725184 (0.0008) [2023-12-26 20:38:25,891][105692] Updated weights for policy 0, policy_version 724558 (0.0009) [2023-12-26 20:38:25,939][105692] Updated weights for policy 0, policy_version 724568 (0.0010) [2023-12-26 20:38:25,993][105692] Updated weights for policy 0, policy_version 724578 (0.0010) [2023-12-26 20:38:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 371195904. Throughput: 0: 9637.0, 1: 9953.9. Samples: 371199356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:38:26,062][104569] Avg episode reward: [(0, '9079.680'), (1, '9174.394')] [2023-12-26 20:38:26,618][105620] Updated weights for policy 1, policy_version 725194 (0.0009) [2023-12-26 20:38:26,662][105692] Updated weights for policy 0, policy_version 724588 (0.0008) [2023-12-26 20:38:26,669][105620] Updated weights for policy 1, policy_version 725204 (0.0008) [2023-12-26 20:38:26,721][105620] Updated weights for policy 1, policy_version 725214 (0.0008) [2023-12-26 20:38:26,725][105692] Updated weights for policy 0, policy_version 724598 (0.0006) [2023-12-26 20:38:26,773][105620] Updated weights for policy 1, policy_version 725224 (0.0008) [2023-12-26 20:38:26,782][105692] Updated weights for policy 0, policy_version 724608 (0.0005) [2023-12-26 20:38:27,277][105692] Updated weights for policy 0, policy_version 724618 (0.0005) [2023-12-26 20:38:27,328][105692] Updated weights for policy 0, policy_version 724628 (0.0005) [2023-12-26 20:38:27,385][105692] Updated weights for policy 0, policy_version 724638 (0.0008) [2023-12-26 20:38:27,441][105692] Updated weights for policy 0, policy_version 724648 (0.0010) [2023-12-26 20:38:27,636][105620] Updated weights for policy 1, policy_version 725234 (0.0008) [2023-12-26 20:38:27,686][105620] Updated weights for policy 1, policy_version 725244 (0.0007) [2023-12-26 20:38:27,738][105620] Updated weights for policy 1, policy_version 725254 (0.0008) [2023-12-26 20:38:28,115][105692] Updated weights for policy 0, policy_version 724658 (0.0009) [2023-12-26 20:38:28,172][105692] Updated weights for policy 0, policy_version 724668 (0.0010) [2023-12-26 20:38:28,236][105692] Updated weights for policy 0, policy_version 724678 (0.0010) [2023-12-26 20:38:28,438][105620] Updated weights for policy 1, policy_version 725264 (0.0006) [2023-12-26 20:38:28,495][105620] Updated weights for policy 1, policy_version 725274 (0.0005) [2023-12-26 20:38:28,563][105620] Updated weights for policy 1, policy_version 725284 (0.0005) [2023-12-26 20:38:28,861][105692] Updated weights for policy 0, policy_version 724688 (0.0008) [2023-12-26 20:38:28,920][105692] Updated weights for policy 0, policy_version 724698 (0.0005) [2023-12-26 20:38:28,971][105692] Updated weights for policy 0, policy_version 724708 (0.0010) [2023-12-26 20:38:29,294][105620] Updated weights for policy 1, policy_version 725294 (0.0005) [2023-12-26 20:38:29,360][105620] Updated weights for policy 1, policy_version 725304 (0.0007) [2023-12-26 20:38:29,415][105620] Updated weights for policy 1, policy_version 725314 (0.0008) [2023-12-26 20:38:29,649][105692] Updated weights for policy 0, policy_version 724718 (0.0010) [2023-12-26 20:38:29,707][105692] Updated weights for policy 0, policy_version 724728 (0.0010) [2023-12-26 20:38:29,773][105692] Updated weights for policy 0, policy_version 724738 (0.0007) [2023-12-26 20:38:30,164][105620] Updated weights for policy 1, policy_version 725324 (0.0009) [2023-12-26 20:38:30,216][105620] Updated weights for policy 1, policy_version 725334 (0.0009) [2023-12-26 20:38:30,274][105620] Updated weights for policy 1, policy_version 725344 (0.0005) [2023-12-26 20:38:30,450][105692] Updated weights for policy 0, policy_version 724748 (0.0006) [2023-12-26 20:38:30,542][105692] Updated weights for policy 0, policy_version 724758 (0.0009) [2023-12-26 20:38:30,613][105692] Updated weights for policy 0, policy_version 724768 (0.0008) [2023-12-26 20:38:30,891][105620] Updated weights for policy 1, policy_version 725354 (0.0005) [2023-12-26 20:38:30,947][105620] Updated weights for policy 1, policy_version 725364 (0.0005) [2023-12-26 20:38:31,002][105620] Updated weights for policy 1, policy_version 725374 (0.0005) [2023-12-26 20:38:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 371286016. Throughput: 0: 9750.8, 1: 9937.1. Samples: 371259192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:38:31,063][104569] Avg episode reward: [(0, '9262.932'), (1, '9265.917')] [2023-12-26 20:38:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000724776_185573376.pth... [2023-12-26 20:38:31,068][105620] Updated weights for policy 1, policy_version 725384 (0.0008) [2023-12-26 20:38:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000725384_185720832.pth... [2023-12-26 20:38:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000723656_185286656.pth [2023-12-26 20:38:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000724232_185425920.pth [2023-12-26 20:38:31,293][105692] Updated weights for policy 0, policy_version 724778 (0.0009) [2023-12-26 20:38:31,370][105692] Updated weights for policy 0, policy_version 724788 (0.0009) [2023-12-26 20:38:31,433][105692] Updated weights for policy 0, policy_version 724798 (0.0006) [2023-12-26 20:38:31,498][105692] Updated weights for policy 0, policy_version 724808 (0.0010) [2023-12-26 20:38:31,771][105620] Updated weights for policy 1, policy_version 725394 (0.0008) [2023-12-26 20:38:31,836][105620] Updated weights for policy 1, policy_version 725404 (0.0008) [2023-12-26 20:38:31,895][105620] Updated weights for policy 1, policy_version 725414 (0.0007) [2023-12-26 20:38:32,178][105692] Updated weights for policy 0, policy_version 724818 (0.0006) [2023-12-26 20:38:32,229][105692] Updated weights for policy 0, policy_version 724828 (0.0009) [2023-12-26 20:38:32,285][105692] Updated weights for policy 0, policy_version 724838 (0.0010) [2023-12-26 20:38:32,570][105620] Updated weights for policy 1, policy_version 725424 (0.0008) [2023-12-26 20:38:32,638][105620] Updated weights for policy 1, policy_version 725434 (0.0007) [2023-12-26 20:38:32,699][105620] Updated weights for policy 1, policy_version 725444 (0.0010) [2023-12-26 20:38:32,869][105692] Updated weights for policy 0, policy_version 724848 (0.0010) [2023-12-26 20:38:32,924][105692] Updated weights for policy 0, policy_version 724858 (0.0010) [2023-12-26 20:38:32,978][105692] Updated weights for policy 0, policy_version 724868 (0.0010) [2023-12-26 20:38:33,346][105620] Updated weights for policy 1, policy_version 725454 (0.0008) [2023-12-26 20:38:33,393][105620] Updated weights for policy 1, policy_version 725464 (0.0010) [2023-12-26 20:38:33,440][105620] Updated weights for policy 1, policy_version 725474 (0.0010) [2023-12-26 20:38:33,602][105692] Updated weights for policy 0, policy_version 724878 (0.0007) [2023-12-26 20:38:33,651][105692] Updated weights for policy 0, policy_version 724888 (0.0008) [2023-12-26 20:38:33,698][105692] Updated weights for policy 0, policy_version 724898 (0.0010) [2023-12-26 20:38:34,117][105620] Updated weights for policy 1, policy_version 725484 (0.0010) [2023-12-26 20:38:34,180][105620] Updated weights for policy 1, policy_version 725494 (0.0009) [2023-12-26 20:38:34,244][105620] Updated weights for policy 1, policy_version 725504 (0.0009) [2023-12-26 20:38:34,279][105692] Updated weights for policy 0, policy_version 724908 (0.0006) [2023-12-26 20:38:34,335][105692] Updated weights for policy 0, policy_version 724918 (0.0006) [2023-12-26 20:38:34,396][105692] Updated weights for policy 0, policy_version 724928 (0.0005) [2023-12-26 20:38:34,998][105620] Updated weights for policy 1, policy_version 725514 (0.0008) [2023-12-26 20:38:35,009][105692] Updated weights for policy 0, policy_version 724938 (0.0006) [2023-12-26 20:38:35,058][105620] Updated weights for policy 1, policy_version 725524 (0.0009) [2023-12-26 20:38:35,071][105692] Updated weights for policy 0, policy_version 724948 (0.0005) [2023-12-26 20:38:35,116][105620] Updated weights for policy 1, policy_version 725534 (0.0007) [2023-12-26 20:38:35,130][105692] Updated weights for policy 0, policy_version 724958 (0.0008) [2023-12-26 20:38:35,175][105620] Updated weights for policy 1, policy_version 725544 (0.0006) [2023-12-26 20:38:35,195][105692] Updated weights for policy 0, policy_version 724968 (0.0009) [2023-12-26 20:38:35,729][105620] Updated weights for policy 1, policy_version 725554 (0.0006) [2023-12-26 20:38:35,793][105620] Updated weights for policy 1, policy_version 725564 (0.0006) [2023-12-26 20:38:35,846][105620] Updated weights for policy 1, policy_version 725574 (0.0009) [2023-12-26 20:38:35,973][105692] Updated weights for policy 0, policy_version 724978 (0.0011) [2023-12-26 20:38:36,031][105692] Updated weights for policy 0, policy_version 724988 (0.0010) [2023-12-26 20:38:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 371392512. Throughput: 0: 9855.3, 1: 9915.3. Samples: 371383452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:38:36,063][104569] Avg episode reward: [(0, '9091.531'), (1, '9268.168')] [2023-12-26 20:38:36,092][105692] Updated weights for policy 0, policy_version 724998 (0.0010) [2023-12-26 20:38:36,580][105620] Updated weights for policy 1, policy_version 725584 (0.0008) [2023-12-26 20:38:36,633][105620] Updated weights for policy 1, policy_version 725594 (0.0008) [2023-12-26 20:38:36,691][105620] Updated weights for policy 1, policy_version 725604 (0.0009) [2023-12-26 20:38:36,765][105692] Updated weights for policy 0, policy_version 725008 (0.0007) [2023-12-26 20:38:36,825][105692] Updated weights for policy 0, policy_version 725018 (0.0007) [2023-12-26 20:38:36,882][105692] Updated weights for policy 0, policy_version 725028 (0.0006) [2023-12-26 20:38:37,494][105620] Updated weights for policy 1, policy_version 725614 (0.0007) [2023-12-26 20:38:37,521][105692] Updated weights for policy 0, policy_version 725038 (0.0008) [2023-12-26 20:38:37,563][105620] Updated weights for policy 1, policy_version 725624 (0.0006) [2023-12-26 20:38:37,581][105692] Updated weights for policy 0, policy_version 725048 (0.0010) [2023-12-26 20:38:37,623][105620] Updated weights for policy 1, policy_version 725634 (0.0006) [2023-12-26 20:38:37,633][105692] Updated weights for policy 0, policy_version 725058 (0.0010) [2023-12-26 20:38:38,333][105620] Updated weights for policy 1, policy_version 725644 (0.0008) [2023-12-26 20:38:38,366][105692] Updated weights for policy 0, policy_version 725068 (0.0009) [2023-12-26 20:38:38,401][105620] Updated weights for policy 1, policy_version 725654 (0.0008) [2023-12-26 20:38:38,421][105692] Updated weights for policy 0, policy_version 725078 (0.0007) [2023-12-26 20:38:38,467][105620] Updated weights for policy 1, policy_version 725664 (0.0008) [2023-12-26 20:38:38,481][105692] Updated weights for policy 0, policy_version 725088 (0.0007) [2023-12-26 20:38:39,029][105620] Updated weights for policy 1, policy_version 725674 (0.0007) [2023-12-26 20:38:39,084][105620] Updated weights for policy 1, policy_version 725684 (0.0005) [2023-12-26 20:38:39,142][105620] Updated weights for policy 1, policy_version 725694 (0.0009) [2023-12-26 20:38:39,203][105620] Updated weights for policy 1, policy_version 725704 (0.0008) [2023-12-26 20:38:39,337][105692] Updated weights for policy 0, policy_version 725098 (0.0009) [2023-12-26 20:38:39,403][105692] Updated weights for policy 0, policy_version 725108 (0.0009) [2023-12-26 20:38:39,459][105692] Updated weights for policy 0, policy_version 725118 (0.0009) [2023-12-26 20:38:39,514][105692] Updated weights for policy 0, policy_version 725128 (0.0007) [2023-12-26 20:38:39,949][105620] Updated weights for policy 1, policy_version 725714 (0.0009) [2023-12-26 20:38:40,001][105620] Updated weights for policy 1, policy_version 725724 (0.0007) [2023-12-26 20:38:40,060][105620] Updated weights for policy 1, policy_version 725734 (0.0006) [2023-12-26 20:38:40,295][105692] Updated weights for policy 0, policy_version 725138 (0.0010) [2023-12-26 20:38:40,351][105692] Updated weights for policy 0, policy_version 725148 (0.0009) [2023-12-26 20:38:40,414][105692] Updated weights for policy 0, policy_version 725158 (0.0009) [2023-12-26 20:38:40,708][105620] Updated weights for policy 1, policy_version 725744 (0.0006) [2023-12-26 20:38:40,774][105620] Updated weights for policy 1, policy_version 725754 (0.0005) [2023-12-26 20:38:40,827][105620] Updated weights for policy 1, policy_version 725764 (0.0005) [2023-12-26 20:38:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 371490816. Throughput: 0: 9869.3, 1: 9884.6. Samples: 371501008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:38:41,063][104569] Avg episode reward: [(0, '9091.546'), (1, '9086.148')] [2023-12-26 20:38:41,156][105692] Updated weights for policy 0, policy_version 725168 (0.0008) [2023-12-26 20:38:41,229][105692] Updated weights for policy 0, policy_version 725178 (0.0008) [2023-12-26 20:38:41,295][105692] Updated weights for policy 0, policy_version 725188 (0.0008) [2023-12-26 20:38:41,539][105620] Updated weights for policy 1, policy_version 725774 (0.0009) [2023-12-26 20:38:41,601][105620] Updated weights for policy 1, policy_version 725784 (0.0011) [2023-12-26 20:38:41,666][105620] Updated weights for policy 1, policy_version 725794 (0.0010) [2023-12-26 20:38:42,085][105692] Updated weights for policy 0, policy_version 725198 (0.0008) [2023-12-26 20:38:42,132][105692] Updated weights for policy 0, policy_version 725208 (0.0009) [2023-12-26 20:38:42,188][105692] Updated weights for policy 0, policy_version 725218 (0.0009) [2023-12-26 20:38:42,415][105620] Updated weights for policy 1, policy_version 725804 (0.0008) [2023-12-26 20:38:42,465][105620] Updated weights for policy 1, policy_version 725814 (0.0009) [2023-12-26 20:38:42,524][105620] Updated weights for policy 1, policy_version 725824 (0.0007) [2023-12-26 20:38:42,974][105692] Updated weights for policy 0, policy_version 725228 (0.0009) [2023-12-26 20:38:43,033][105692] Updated weights for policy 0, policy_version 725238 (0.0009) [2023-12-26 20:38:43,086][105692] Updated weights for policy 0, policy_version 725248 (0.0010) [2023-12-26 20:38:43,188][105620] Updated weights for policy 1, policy_version 725834 (0.0008) [2023-12-26 20:38:43,237][105620] Updated weights for policy 1, policy_version 725844 (0.0010) [2023-12-26 20:38:43,289][105620] Updated weights for policy 1, policy_version 725854 (0.0010) [2023-12-26 20:38:43,343][105620] Updated weights for policy 1, policy_version 725864 (0.0010) [2023-12-26 20:38:43,828][105692] Updated weights for policy 0, policy_version 725258 (0.0009) [2023-12-26 20:38:43,893][105692] Updated weights for policy 0, policy_version 725268 (0.0008) [2023-12-26 20:38:43,940][105620] Updated weights for policy 1, policy_version 725874 (0.0006) [2023-12-26 20:38:43,956][105692] Updated weights for policy 0, policy_version 725278 (0.0009) [2023-12-26 20:38:43,997][105620] Updated weights for policy 1, policy_version 725884 (0.0007) [2023-12-26 20:38:44,018][105692] Updated weights for policy 0, policy_version 725288 (0.0011) [2023-12-26 20:38:44,053][105620] Updated weights for policy 1, policy_version 725894 (0.0010) [2023-12-26 20:38:44,672][105692] Updated weights for policy 0, policy_version 725298 (0.0009) [2023-12-26 20:38:44,702][105620] Updated weights for policy 1, policy_version 725904 (0.0006) [2023-12-26 20:38:44,722][105692] Updated weights for policy 0, policy_version 725308 (0.0008) [2023-12-26 20:38:44,755][105620] Updated weights for policy 1, policy_version 725914 (0.0005) [2023-12-26 20:38:44,783][105692] Updated weights for policy 0, policy_version 725318 (0.0007) [2023-12-26 20:38:44,837][105620] Updated weights for policy 1, policy_version 725924 (0.0009) [2023-12-26 20:38:45,430][105620] Updated weights for policy 1, policy_version 725934 (0.0009) [2023-12-26 20:38:45,485][105692] Updated weights for policy 0, policy_version 725328 (0.0010) [2023-12-26 20:38:45,487][105585] KL-divergence is very high: 205.4761 [2023-12-26 20:38:45,487][105620] Updated weights for policy 1, policy_version 725944 (0.0011) [2023-12-26 20:38:45,493][105585] KL-divergence is very high: 135.3509 [2023-12-26 20:38:45,505][105585] KL-divergence is very high: 102.8510 [2023-12-26 20:38:45,511][105585] KL-divergence is very high: 249.1417 [2023-12-26 20:38:45,536][105585] KL-divergence is very high: 204.5001 [2023-12-26 20:38:45,541][105585] KL-divergence is very high: 109.0940 [2023-12-26 20:38:45,546][105620] Updated weights for policy 1, policy_version 725954 (0.0010) [2023-12-26 20:38:45,547][105692] Updated weights for policy 0, policy_version 725338 (0.0009) [2023-12-26 20:38:45,560][105585] KL-divergence is very high: 144.0274 [2023-12-26 20:38:45,606][105692] Updated weights for policy 0, policy_version 725348 (0.0010) [2023-12-26 20:38:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 371589120. Throughput: 0: 9737.1, 1: 9907.6. Samples: 371558872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:38:46,063][104569] Avg episode reward: [(0, '8877.680'), (1, '9174.987')] [2023-12-26 20:38:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000725960_185868288.pth... [2023-12-26 20:38:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000725352_185720832.pth... [2023-12-26 20:38:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000724808_185573376.pth [2023-12-26 20:38:46,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000724200_185425920.pth [2023-12-26 20:38:46,156][105620] Updated weights for policy 1, policy_version 725964 (0.0010) [2023-12-26 20:38:46,213][105692] Updated weights for policy 0, policy_version 725358 (0.0007) [2023-12-26 20:38:46,218][105620] Updated weights for policy 1, policy_version 725974 (0.0009) [2023-12-26 20:38:46,271][105692] Updated weights for policy 0, policy_version 725368 (0.0006) [2023-12-26 20:38:46,286][105620] Updated weights for policy 1, policy_version 725984 (0.0009) [2023-12-26 20:38:46,323][105692] Updated weights for policy 0, policy_version 725378 (0.0008) [2023-12-26 20:38:46,961][105620] Updated weights for policy 1, policy_version 725994 (0.0010) [2023-12-26 20:38:46,991][105692] Updated weights for policy 0, policy_version 725388 (0.0005) [2023-12-26 20:38:47,022][105620] Updated weights for policy 1, policy_version 726004 (0.0007) [2023-12-26 20:38:47,056][105692] Updated weights for policy 0, policy_version 725398 (0.0005) [2023-12-26 20:38:47,084][105620] Updated weights for policy 1, policy_version 726014 (0.0009) [2023-12-26 20:38:47,114][105692] Updated weights for policy 0, policy_version 725408 (0.0005) [2023-12-26 20:38:47,148][105620] Updated weights for policy 1, policy_version 726024 (0.0009) [2023-12-26 20:38:47,730][105692] Updated weights for policy 0, policy_version 725418 (0.0005) [2023-12-26 20:38:47,787][105692] Updated weights for policy 0, policy_version 725428 (0.0005) [2023-12-26 20:38:47,843][105692] Updated weights for policy 0, policy_version 725438 (0.0007) [2023-12-26 20:38:47,904][105692] Updated weights for policy 0, policy_version 725448 (0.0008) [2023-12-26 20:38:47,938][105620] Updated weights for policy 1, policy_version 726034 (0.0010) [2023-12-26 20:38:47,991][105620] Updated weights for policy 1, policy_version 726044 (0.0010) [2023-12-26 20:38:48,061][105620] Updated weights for policy 1, policy_version 726054 (0.0010) [2023-12-26 20:38:48,485][105692] Updated weights for policy 0, policy_version 725458 (0.0008) [2023-12-26 20:38:48,540][105692] Updated weights for policy 0, policy_version 725468 (0.0009) [2023-12-26 20:38:48,594][105692] Updated weights for policy 0, policy_version 725478 (0.0009) [2023-12-26 20:38:48,882][105620] Updated weights for policy 1, policy_version 726064 (0.0009) [2023-12-26 20:38:48,939][105620] Updated weights for policy 1, policy_version 726074 (0.0009) [2023-12-26 20:38:48,993][105620] Updated weights for policy 1, policy_version 726084 (0.0007) [2023-12-26 20:38:49,378][105692] Updated weights for policy 0, policy_version 725488 (0.0008) [2023-12-26 20:38:49,433][105692] Updated weights for policy 0, policy_version 725498 (0.0008) [2023-12-26 20:38:49,494][105692] Updated weights for policy 0, policy_version 725508 (0.0008) [2023-12-26 20:38:49,719][105620] Updated weights for policy 1, policy_version 726095 (0.0009) [2023-12-26 20:38:49,773][105620] Updated weights for policy 1, policy_version 726105 (0.0009) [2023-12-26 20:38:49,836][105620] Updated weights for policy 1, policy_version 726115 (0.0008) [2023-12-26 20:38:50,186][105692] Updated weights for policy 0, policy_version 725518 (0.0008) [2023-12-26 20:38:50,249][105692] Updated weights for policy 0, policy_version 725528 (0.0009) [2023-12-26 20:38:50,309][105692] Updated weights for policy 0, policy_version 725538 (0.0009) [2023-12-26 20:38:50,646][105620] Updated weights for policy 1, policy_version 726125 (0.0009) [2023-12-26 20:38:50,706][105620] Updated weights for policy 1, policy_version 726135 (0.0009) [2023-12-26 20:38:50,760][105620] Updated weights for policy 1, policy_version 726145 (0.0010) [2023-12-26 20:38:50,968][105692] Updated weights for policy 0, policy_version 725548 (0.0009) [2023-12-26 20:38:51,032][105692] Updated weights for policy 0, policy_version 725558 (0.0007) [2023-12-26 20:38:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 371687424. Throughput: 0: 9831.5, 1: 9867.2. Samples: 371679352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:38:51,063][104569] Avg episode reward: [(0, '8730.613'), (1, '9175.117')] [2023-12-26 20:38:51,099][105692] Updated weights for policy 0, policy_version 725568 (0.0006) [2023-12-26 20:38:51,653][105620] Updated weights for policy 1, policy_version 726155 (0.0009) [2023-12-26 20:38:51,717][105620] Updated weights for policy 1, policy_version 726165 (0.0008) [2023-12-26 20:38:51,748][105692] Updated weights for policy 0, policy_version 725578 (0.0008) [2023-12-26 20:38:51,787][105620] Updated weights for policy 1, policy_version 726175 (0.0007) [2023-12-26 20:38:51,805][105692] Updated weights for policy 0, policy_version 725588 (0.0011) [2023-12-26 20:38:51,870][105692] Updated weights for policy 0, policy_version 725598 (0.0011) [2023-12-26 20:38:51,933][105692] Updated weights for policy 0, policy_version 725608 (0.0011) [2023-12-26 20:38:52,575][105620] Updated weights for policy 1, policy_version 726185 (0.0006) [2023-12-26 20:38:52,614][105692] Updated weights for policy 0, policy_version 725618 (0.0008) [2023-12-26 20:38:52,636][105620] Updated weights for policy 1, policy_version 726195 (0.0007) [2023-12-26 20:38:52,670][105692] Updated weights for policy 0, policy_version 725628 (0.0011) [2023-12-26 20:38:52,696][105620] Updated weights for policy 1, policy_version 726205 (0.0005) [2023-12-26 20:38:52,732][105692] Updated weights for policy 0, policy_version 725639 (0.0011) [2023-12-26 20:38:52,754][105620] Updated weights for policy 1, policy_version 726215 (0.0008) [2023-12-26 20:38:53,453][105620] Updated weights for policy 1, policy_version 726225 (0.0008) [2023-12-26 20:38:53,466][105692] Updated weights for policy 0, policy_version 725649 (0.0006) [2023-12-26 20:38:53,510][105620] Updated weights for policy 1, policy_version 726235 (0.0005) [2023-12-26 20:38:53,535][105692] Updated weights for policy 0, policy_version 725659 (0.0006) [2023-12-26 20:38:53,566][105620] Updated weights for policy 1, policy_version 726245 (0.0005) [2023-12-26 20:38:53,594][105692] Updated weights for policy 0, policy_version 725669 (0.0006) [2023-12-26 20:38:54,110][105620] Updated weights for policy 1, policy_version 726255 (0.0005) [2023-12-26 20:38:54,141][105692] Updated weights for policy 0, policy_version 725679 (0.0007) [2023-12-26 20:38:54,174][105620] Updated weights for policy 1, policy_version 726265 (0.0008) [2023-12-26 20:38:54,204][105692] Updated weights for policy 0, policy_version 725689 (0.0006) [2023-12-26 20:38:54,239][105620] Updated weights for policy 1, policy_version 726275 (0.0009) [2023-12-26 20:38:54,260][105692] Updated weights for policy 0, policy_version 725699 (0.0005) [2023-12-26 20:38:54,905][105692] Updated weights for policy 0, policy_version 725709 (0.0009) [2023-12-26 20:38:54,960][105620] Updated weights for policy 1, policy_version 726285 (0.0008) [2023-12-26 20:38:54,961][105692] Updated weights for policy 0, policy_version 725719 (0.0010) [2023-12-26 20:38:55,021][105692] Updated weights for policy 0, policy_version 725729 (0.0011) [2023-12-26 20:38:55,024][105620] Updated weights for policy 1, policy_version 726295 (0.0006) [2023-12-26 20:38:55,086][105620] Updated weights for policy 1, policy_version 726305 (0.0006) [2023-12-26 20:38:55,661][105620] Updated weights for policy 1, policy_version 726315 (0.0005) [2023-12-26 20:38:55,717][105620] Updated weights for policy 1, policy_version 726325 (0.0005) [2023-12-26 20:38:55,730][105692] Updated weights for policy 0, policy_version 725739 (0.0010) [2023-12-26 20:38:55,769][105620] Updated weights for policy 1, policy_version 726335 (0.0006) [2023-12-26 20:38:55,791][105692] Updated weights for policy 0, policy_version 725749 (0.0009) [2023-12-26 20:38:55,849][105692] Updated weights for policy 0, policy_version 725759 (0.0007) [2023-12-26 20:38:56,062][104569] Fps is (10 sec: 20480.7, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 371793920. Throughput: 0: 9939.3, 1: 9785.6. Samples: 371799740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:38:56,062][104569] Avg episode reward: [(0, '8730.162'), (1, '9089.153')] [2023-12-26 20:38:56,456][105620] Updated weights for policy 1, policy_version 726345 (0.0009) [2023-12-26 20:38:56,512][105620] Updated weights for policy 1, policy_version 726355 (0.0005) [2023-12-26 20:38:56,568][105620] Updated weights for policy 1, policy_version 726365 (0.0008) [2023-12-26 20:38:56,573][105692] Updated weights for policy 0, policy_version 725769 (0.0008) [2023-12-26 20:38:56,623][105620] Updated weights for policy 1, policy_version 726375 (0.0008) [2023-12-26 20:38:56,635][105692] Updated weights for policy 0, policy_version 725779 (0.0006) [2023-12-26 20:38:56,700][105692] Updated weights for policy 0, policy_version 725789 (0.0005) [2023-12-26 20:38:56,760][105692] Updated weights for policy 0, policy_version 725799 (0.0006) [2023-12-26 20:38:57,299][105692] Updated weights for policy 0, policy_version 725809 (0.0005) [2023-12-26 20:38:57,351][105692] Updated weights for policy 0, policy_version 725819 (0.0006) [2023-12-26 20:38:57,360][105620] Updated weights for policy 1, policy_version 726385 (0.0009) [2023-12-26 20:38:57,398][105692] Updated weights for policy 0, policy_version 725829 (0.0006) [2023-12-26 20:38:57,419][105620] Updated weights for policy 1, policy_version 726395 (0.0008) [2023-12-26 20:38:57,479][105620] Updated weights for policy 1, policy_version 726405 (0.0009) [2023-12-26 20:38:58,028][105692] Updated weights for policy 0, policy_version 725839 (0.0007) [2023-12-26 20:38:58,045][105620] Updated weights for policy 1, policy_version 726415 (0.0007) [2023-12-26 20:38:58,075][105692] Updated weights for policy 0, policy_version 725849 (0.0008) [2023-12-26 20:38:58,095][105620] Updated weights for policy 1, policy_version 726425 (0.0006) [2023-12-26 20:38:58,123][105692] Updated weights for policy 0, policy_version 725859 (0.0007) [2023-12-26 20:38:58,153][105620] Updated weights for policy 1, policy_version 726435 (0.0007) [2023-12-26 20:38:58,826][105620] Updated weights for policy 1, policy_version 726445 (0.0007) [2023-12-26 20:38:58,897][105620] Updated weights for policy 1, policy_version 726455 (0.0009) [2023-12-26 20:38:58,917][105586] KL-divergence is very high: 120.0089 [2023-12-26 20:38:58,928][105692] Updated weights for policy 0, policy_version 725869 (0.0008) [2023-12-26 20:38:58,952][105620] Updated weights for policy 1, policy_version 726465 (0.0006) [2023-12-26 20:38:58,960][105586] KL-divergence is very high: 130.5959 [2023-12-26 20:38:58,990][105692] Updated weights for policy 0, policy_version 725879 (0.0008) [2023-12-26 20:38:59,053][105692] Updated weights for policy 0, policy_version 725889 (0.0008) [2023-12-26 20:38:59,687][105620] Updated weights for policy 1, policy_version 726475 (0.0008) [2023-12-26 20:38:59,738][105620] Updated weights for policy 1, policy_version 726485 (0.0010) [2023-12-26 20:38:59,795][105620] Updated weights for policy 1, policy_version 726495 (0.0010) [2023-12-26 20:38:59,836][105692] Updated weights for policy 0, policy_version 725899 (0.0007) [2023-12-26 20:38:59,884][105692] Updated weights for policy 0, policy_version 725909 (0.0006) [2023-12-26 20:38:59,949][105692] Updated weights for policy 0, policy_version 725919 (0.0007) [2023-12-26 20:39:00,553][105620] Updated weights for policy 1, policy_version 726505 (0.0009) [2023-12-26 20:39:00,602][105620] Updated weights for policy 1, policy_version 726515 (0.0010) [2023-12-26 20:39:00,650][105692] Updated weights for policy 0, policy_version 725929 (0.0008) [2023-12-26 20:39:00,657][105620] Updated weights for policy 1, policy_version 726525 (0.0010) [2023-12-26 20:39:00,711][105692] Updated weights for policy 0, policy_version 725939 (0.0005) [2023-12-26 20:39:00,711][105620] Updated weights for policy 1, policy_version 726535 (0.0010) [2023-12-26 20:39:00,776][105692] Updated weights for policy 0, policy_version 725949 (0.0005) [2023-12-26 20:39:00,828][105692] Updated weights for policy 0, policy_version 725959 (0.0005) [2023-12-26 20:39:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 371892224. Throughput: 0: 10045.1, 1: 9809.9. Samples: 371861900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:39:01,062][104569] Avg episode reward: [(0, '9084.074'), (1, '9096.930')] [2023-12-26 20:39:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000725960_185876480.pth... [2023-12-26 20:39:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000726536_186015744.pth... [2023-12-26 20:39:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000724776_185573376.pth [2023-12-26 20:39:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000725384_185720832.pth [2023-12-26 20:39:01,390][105620] Updated weights for policy 1, policy_version 726545 (0.0010) [2023-12-26 20:39:01,456][105620] Updated weights for policy 1, policy_version 726555 (0.0006) [2023-12-26 20:39:01,488][105692] Updated weights for policy 0, policy_version 725969 (0.0005) [2023-12-26 20:39:01,519][105620] Updated weights for policy 1, policy_version 726565 (0.0008) [2023-12-26 20:39:01,546][105692] Updated weights for policy 0, policy_version 725979 (0.0005) [2023-12-26 20:39:01,603][105692] Updated weights for policy 0, policy_version 725989 (0.0005) [2023-12-26 20:39:02,205][105620] Updated weights for policy 1, policy_version 726575 (0.0009) [2023-12-26 20:39:02,266][105620] Updated weights for policy 1, policy_version 726585 (0.0009) [2023-12-26 20:39:02,316][105692] Updated weights for policy 0, policy_version 725999 (0.0008) [2023-12-26 20:39:02,318][105620] Updated weights for policy 1, policy_version 726595 (0.0007) [2023-12-26 20:39:02,382][105692] Updated weights for policy 0, policy_version 726009 (0.0010) [2023-12-26 20:39:02,444][105692] Updated weights for policy 0, policy_version 726019 (0.0009) [2023-12-26 20:39:03,076][105620] Updated weights for policy 1, policy_version 726605 (0.0007) [2023-12-26 20:39:03,086][105692] Updated weights for policy 0, policy_version 726029 (0.0007) [2023-12-26 20:39:03,136][105692] Updated weights for policy 0, policy_version 726039 (0.0005) [2023-12-26 20:39:03,143][105620] Updated weights for policy 1, policy_version 726615 (0.0009) [2023-12-26 20:39:03,183][105692] Updated weights for policy 0, policy_version 726049 (0.0005) [2023-12-26 20:39:03,201][105620] Updated weights for policy 1, policy_version 726625 (0.0005) [2023-12-26 20:39:03,704][105692] Updated weights for policy 0, policy_version 726059 (0.0005) [2023-12-26 20:39:03,751][105692] Updated weights for policy 0, policy_version 726069 (0.0005) [2023-12-26 20:39:03,775][105620] Updated weights for policy 1, policy_version 726635 (0.0006) [2023-12-26 20:39:03,809][105692] Updated weights for policy 0, policy_version 726079 (0.0009) [2023-12-26 20:39:03,823][105620] Updated weights for policy 1, policy_version 726645 (0.0009) [2023-12-26 20:39:03,886][105620] Updated weights for policy 1, policy_version 726655 (0.0010) [2023-12-26 20:39:04,536][105692] Updated weights for policy 0, policy_version 726089 (0.0009) [2023-12-26 20:39:04,597][105620] Updated weights for policy 1, policy_version 726665 (0.0010) [2023-12-26 20:39:04,602][105692] Updated weights for policy 0, policy_version 726099 (0.0010) [2023-12-26 20:39:04,659][105692] Updated weights for policy 0, policy_version 726109 (0.0008) [2023-12-26 20:39:04,664][105620] Updated weights for policy 1, policy_version 726675 (0.0006) [2023-12-26 20:39:04,704][105692] Updated weights for policy 0, policy_version 726119 (0.0007) [2023-12-26 20:39:04,732][105620] Updated weights for policy 1, policy_version 726685 (0.0005) [2023-12-26 20:39:04,803][105620] Updated weights for policy 1, policy_version 726695 (0.0005) [2023-12-26 20:39:05,305][105620] Updated weights for policy 1, policy_version 726705 (0.0010) [2023-12-26 20:39:05,333][105692] Updated weights for policy 0, policy_version 726129 (0.0010) [2023-12-26 20:39:05,363][105620] Updated weights for policy 1, policy_version 726715 (0.0010) [2023-12-26 20:39:05,396][105692] Updated weights for policy 0, policy_version 726139 (0.0008) [2023-12-26 20:39:05,413][105620] Updated weights for policy 1, policy_version 726725 (0.0008) [2023-12-26 20:39:05,446][105692] Updated weights for policy 0, policy_version 726149 (0.0010) [2023-12-26 20:39:05,966][105620] Updated weights for policy 1, policy_version 726735 (0.0005) [2023-12-26 20:39:06,018][105620] Updated weights for policy 1, policy_version 726745 (0.0005) [2023-12-26 20:39:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 371990528. Throughput: 0: 10031.6, 1: 9838.3. Samples: 371982364. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 20:39:06,062][104569] Avg episode reward: [(0, '9082.425'), (1, '9181.605')] [2023-12-26 20:39:06,066][105620] Updated weights for policy 1, policy_version 726755 (0.0005) [2023-12-26 20:39:06,138][105692] Updated weights for policy 0, policy_version 726159 (0.0014) [2023-12-26 20:39:06,195][105692] Updated weights for policy 0, policy_version 726169 (0.0007) [2023-12-26 20:39:06,240][105692] Updated weights for policy 0, policy_version 726179 (0.0010) [2023-12-26 20:39:06,776][105620] Updated weights for policy 1, policy_version 726765 (0.0008) [2023-12-26 20:39:06,831][105620] Updated weights for policy 1, policy_version 726775 (0.0010) [2023-12-26 20:39:06,883][105620] Updated weights for policy 1, policy_version 726785 (0.0010) [2023-12-26 20:39:07,008][105692] Updated weights for policy 0, policy_version 726189 (0.0010) [2023-12-26 20:39:07,066][105692] Updated weights for policy 0, policy_version 726199 (0.0010) [2023-12-26 20:39:07,120][105692] Updated weights for policy 0, policy_version 726209 (0.0010) [2023-12-26 20:39:07,446][105620] Updated weights for policy 1, policy_version 726795 (0.0007) [2023-12-26 20:39:07,511][105620] Updated weights for policy 1, policy_version 726805 (0.0010) [2023-12-26 20:39:07,574][105620] Updated weights for policy 1, policy_version 726815 (0.0010) [2023-12-26 20:39:08,004][105692] Updated weights for policy 0, policy_version 726220 (0.0010) [2023-12-26 20:39:08,069][105692] Updated weights for policy 0, policy_version 726230 (0.0008) [2023-12-26 20:39:08,127][105692] Updated weights for policy 0, policy_version 726240 (0.0008) [2023-12-26 20:39:08,309][105620] Updated weights for policy 1, policy_version 726825 (0.0011) [2023-12-26 20:39:08,379][105620] Updated weights for policy 1, policy_version 726835 (0.0011) [2023-12-26 20:39:08,445][105620] Updated weights for policy 1, policy_version 726845 (0.0010) [2023-12-26 20:39:08,500][105620] Updated weights for policy 1, policy_version 726855 (0.0010) [2023-12-26 20:39:08,895][105692] Updated weights for policy 0, policy_version 726250 (0.0009) [2023-12-26 20:39:08,962][105692] Updated weights for policy 0, policy_version 726260 (0.0008) [2023-12-26 20:39:09,026][105692] Updated weights for policy 0, policy_version 726270 (0.0008) [2023-12-26 20:39:09,089][105692] Updated weights for policy 0, policy_version 726280 (0.0008) [2023-12-26 20:39:09,268][105620] Updated weights for policy 1, policy_version 726865 (0.0008) [2023-12-26 20:39:09,322][105620] Updated weights for policy 1, policy_version 726875 (0.0006) [2023-12-26 20:39:09,385][105620] Updated weights for policy 1, policy_version 726885 (0.0010) [2023-12-26 20:39:09,868][105692] Updated weights for policy 0, policy_version 726290 (0.0009) [2023-12-26 20:39:09,940][105692] Updated weights for policy 0, policy_version 726300 (0.0008) [2023-12-26 20:39:09,999][105692] Updated weights for policy 0, policy_version 726310 (0.0009) [2023-12-26 20:39:10,117][105620] Updated weights for policy 1, policy_version 726895 (0.0010) [2023-12-26 20:39:10,177][105620] Updated weights for policy 1, policy_version 726905 (0.0010) [2023-12-26 20:39:10,246][105620] Updated weights for policy 1, policy_version 726915 (0.0010) [2023-12-26 20:39:10,771][105692] Updated weights for policy 0, policy_version 726320 (0.0008) [2023-12-26 20:39:10,830][105692] Updated weights for policy 0, policy_version 726330 (0.0008) [2023-12-26 20:39:10,889][105692] Updated weights for policy 0, policy_version 726340 (0.0008) [2023-12-26 20:39:10,980][105620] Updated weights for policy 1, policy_version 726925 (0.0010) [2023-12-26 20:39:11,042][105620] Updated weights for policy 1, policy_version 726935 (0.0010) [2023-12-26 20:39:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 372088832. Throughput: 0: 10035.1, 1: 9958.1. Samples: 372099048. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 20:39:11,065][104569] Avg episode reward: [(0, '9173.702'), (1, '9264.135')] [2023-12-26 20:39:11,107][105620] Updated weights for policy 1, policy_version 726945 (0.0008) [2023-12-26 20:39:11,760][105692] Updated weights for policy 0, policy_version 726350 (0.0008) [2023-12-26 20:39:11,824][105692] Updated weights for policy 0, policy_version 726360 (0.0008) [2023-12-26 20:39:11,855][105620] Updated weights for policy 1, policy_version 726955 (0.0008) [2023-12-26 20:39:11,885][105692] Updated weights for policy 0, policy_version 726370 (0.0008) [2023-12-26 20:39:11,916][105620] Updated weights for policy 1, policy_version 726965 (0.0008) [2023-12-26 20:39:11,977][105620] Updated weights for policy 1, policy_version 726975 (0.0008) [2023-12-26 20:39:12,553][105692] Updated weights for policy 0, policy_version 726380 (0.0007) [2023-12-26 20:39:12,618][105692] Updated weights for policy 0, policy_version 726390 (0.0008) [2023-12-26 20:39:12,683][105692] Updated weights for policy 0, policy_version 726400 (0.0011) [2023-12-26 20:39:12,685][105620] Updated weights for policy 1, policy_version 726985 (0.0008) [2023-12-26 20:39:12,746][105620] Updated weights for policy 1, policy_version 726995 (0.0007) [2023-12-26 20:39:12,807][105620] Updated weights for policy 1, policy_version 727005 (0.0008) [2023-12-26 20:39:12,861][105620] Updated weights for policy 1, policy_version 727015 (0.0006) [2023-12-26 20:39:13,396][105692] Updated weights for policy 0, policy_version 726410 (0.0010) [2023-12-26 20:39:13,448][105692] Updated weights for policy 0, policy_version 726420 (0.0006) [2023-12-26 20:39:13,502][105692] Updated weights for policy 0, policy_version 726430 (0.0006) [2023-12-26 20:39:13,558][105692] Updated weights for policy 0, policy_version 726440 (0.0006) [2023-12-26 20:39:13,586][105620] Updated weights for policy 1, policy_version 727025 (0.0008) [2023-12-26 20:39:13,640][105620] Updated weights for policy 1, policy_version 727035 (0.0008) [2023-12-26 20:39:13,712][105620] Updated weights for policy 1, policy_version 727045 (0.0008) [2023-12-26 20:39:14,151][105692] Updated weights for policy 0, policy_version 726450 (0.0005) [2023-12-26 20:39:14,209][105692] Updated weights for policy 0, policy_version 726460 (0.0005) [2023-12-26 20:39:14,261][105692] Updated weights for policy 0, policy_version 726470 (0.0005) [2023-12-26 20:39:14,473][105620] Updated weights for policy 1, policy_version 727055 (0.0008) [2023-12-26 20:39:14,520][105620] Updated weights for policy 1, policy_version 727065 (0.0009) [2023-12-26 20:39:14,568][105620] Updated weights for policy 1, policy_version 727075 (0.0009) [2023-12-26 20:39:14,884][105692] Updated weights for policy 0, policy_version 726480 (0.0006) [2023-12-26 20:39:14,949][105692] Updated weights for policy 0, policy_version 726490 (0.0006) [2023-12-26 20:39:15,019][105692] Updated weights for policy 0, policy_version 726500 (0.0006) [2023-12-26 20:39:15,222][105620] Updated weights for policy 1, policy_version 727085 (0.0005) [2023-12-26 20:39:15,280][105620] Updated weights for policy 1, policy_version 727095 (0.0006) [2023-12-26 20:39:15,342][105620] Updated weights for policy 1, policy_version 727105 (0.0009) [2023-12-26 20:39:15,547][105692] Updated weights for policy 0, policy_version 726510 (0.0008) [2023-12-26 20:39:15,609][105692] Updated weights for policy 0, policy_version 726520 (0.0006) [2023-12-26 20:39:15,675][105692] Updated weights for policy 0, policy_version 726530 (0.0006) [2023-12-26 20:39:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 372187136. Throughput: 0: 9945.9, 1: 9979.4. Samples: 372155828. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 20:39:16,062][104569] Avg episode reward: [(0, '9263.457'), (1, '9355.796')] [2023-12-26 20:39:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000726536_186023936.pth... [2023-12-26 20:39:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000727112_186163200.pth... [2023-12-26 20:39:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000725352_185720832.pth [2023-12-26 20:39:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000725960_185868288.pth [2023-12-26 20:39:16,155][105620] Updated weights for policy 1, policy_version 727115 (0.0009) [2023-12-26 20:39:16,212][105620] Updated weights for policy 1, policy_version 727125 (0.0009) [2023-12-26 20:39:16,259][105692] Updated weights for policy 0, policy_version 726540 (0.0006) [2023-12-26 20:39:16,261][105620] Updated weights for policy 1, policy_version 727135 (0.0007) [2023-12-26 20:39:16,317][105692] Updated weights for policy 0, policy_version 726550 (0.0008) [2023-12-26 20:39:16,378][105692] Updated weights for policy 0, policy_version 726560 (0.0009) [2023-12-26 20:39:16,963][105620] Updated weights for policy 1, policy_version 727145 (0.0006) [2023-12-26 20:39:17,033][105620] Updated weights for policy 1, policy_version 727155 (0.0008) [2023-12-26 20:39:17,102][105620] Updated weights for policy 1, policy_version 727165 (0.0009) [2023-12-26 20:39:17,109][105692] Updated weights for policy 0, policy_version 726570 (0.0008) [2023-12-26 20:39:17,161][105620] Updated weights for policy 1, policy_version 727175 (0.0006) [2023-12-26 20:39:17,169][105692] Updated weights for policy 0, policy_version 726580 (0.0006) [2023-12-26 20:39:17,215][105692] Updated weights for policy 0, policy_version 726590 (0.0005) [2023-12-26 20:39:17,260][105692] Updated weights for policy 0, policy_version 726600 (0.0005) [2023-12-26 20:39:17,787][105620] Updated weights for policy 1, policy_version 727185 (0.0010) [2023-12-26 20:39:17,832][105620] Updated weights for policy 1, policy_version 727195 (0.0010) [2023-12-26 20:39:17,881][105620] Updated weights for policy 1, policy_version 727205 (0.0010) [2023-12-26 20:39:17,886][105692] Updated weights for policy 0, policy_version 726610 (0.0006) [2023-12-26 20:39:17,944][105692] Updated weights for policy 0, policy_version 726620 (0.0007) [2023-12-26 20:39:17,993][105692] Updated weights for policy 0, policy_version 726630 (0.0008) [2023-12-26 20:39:18,657][105620] Updated weights for policy 1, policy_version 727215 (0.0008) [2023-12-26 20:39:18,714][105620] Updated weights for policy 1, policy_version 727225 (0.0008) [2023-12-26 20:39:18,756][105692] Updated weights for policy 0, policy_version 726640 (0.0010) [2023-12-26 20:39:18,770][105620] Updated weights for policy 1, policy_version 727235 (0.0006) [2023-12-26 20:39:18,815][105692] Updated weights for policy 0, policy_version 726650 (0.0010) [2023-12-26 20:39:18,888][105692] Updated weights for policy 0, policy_version 726660 (0.0010) [2023-12-26 20:39:19,451][105620] Updated weights for policy 1, policy_version 727245 (0.0006) [2023-12-26 20:39:19,521][105620] Updated weights for policy 1, policy_version 727255 (0.0007) [2023-12-26 20:39:19,575][105620] Updated weights for policy 1, policy_version 727265 (0.0006) [2023-12-26 20:39:19,630][105692] Updated weights for policy 0, policy_version 726670 (0.0009) [2023-12-26 20:39:19,701][105692] Updated weights for policy 0, policy_version 726680 (0.0010) [2023-12-26 20:39:19,759][105692] Updated weights for policy 0, policy_version 726690 (0.0010) [2023-12-26 20:39:20,239][105620] Updated weights for policy 1, policy_version 727275 (0.0006) [2023-12-26 20:39:20,314][105620] Updated weights for policy 1, policy_version 727285 (0.0008) [2023-12-26 20:39:20,376][105620] Updated weights for policy 1, policy_version 727295 (0.0006) [2023-12-26 20:39:20,479][105692] Updated weights for policy 0, policy_version 726700 (0.0011) [2023-12-26 20:39:20,530][105692] Updated weights for policy 0, policy_version 726710 (0.0010) [2023-12-26 20:39:20,591][105692] Updated weights for policy 0, policy_version 726720 (0.0010) [2023-12-26 20:39:20,999][105620] Updated weights for policy 1, policy_version 727305 (0.0007) [2023-12-26 20:39:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 372285440. Throughput: 0: 9919.3, 1: 9960.4. Samples: 372278036. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 20:39:21,062][105620] Updated weights for policy 1, policy_version 727315 (0.0008) [2023-12-26 20:39:21,063][104569] Avg episode reward: [(0, '9199.468'), (1, '9083.608')] [2023-12-26 20:39:21,129][105620] Updated weights for policy 1, policy_version 727325 (0.0008) [2023-12-26 20:39:21,198][105620] Updated weights for policy 1, policy_version 727335 (0.0006) [2023-12-26 20:39:21,419][105692] Updated weights for policy 0, policy_version 726730 (0.0011) [2023-12-26 20:39:21,482][105692] Updated weights for policy 0, policy_version 726740 (0.0011) [2023-12-26 20:39:21,538][105692] Updated weights for policy 0, policy_version 726750 (0.0010) [2023-12-26 20:39:21,586][105692] Updated weights for policy 0, policy_version 726760 (0.0010) [2023-12-26 20:39:21,840][105620] Updated weights for policy 1, policy_version 727345 (0.0008) [2023-12-26 20:39:21,888][105620] Updated weights for policy 1, policy_version 727355 (0.0008) [2023-12-26 20:39:21,937][105620] Updated weights for policy 1, policy_version 727365 (0.0008) [2023-12-26 20:39:22,381][105692] Updated weights for policy 0, policy_version 726770 (0.0010) [2023-12-26 20:39:22,447][105692] Updated weights for policy 0, policy_version 726780 (0.0010) [2023-12-26 20:39:22,513][105692] Updated weights for policy 0, policy_version 726790 (0.0011) [2023-12-26 20:39:22,732][105620] Updated weights for policy 1, policy_version 727375 (0.0008) [2023-12-26 20:39:22,793][105620] Updated weights for policy 1, policy_version 727385 (0.0008) [2023-12-26 20:39:22,865][105620] Updated weights for policy 1, policy_version 727395 (0.0008) [2023-12-26 20:39:23,250][105692] Updated weights for policy 0, policy_version 726800 (0.0007) [2023-12-26 20:39:23,301][105692] Updated weights for policy 0, policy_version 726810 (0.0005) [2023-12-26 20:39:23,351][105692] Updated weights for policy 0, policy_version 726820 (0.0009) [2023-12-26 20:39:23,664][105620] Updated weights for policy 1, policy_version 727405 (0.0008) [2023-12-26 20:39:23,718][105620] Updated weights for policy 1, policy_version 727415 (0.0009) [2023-12-26 20:39:23,774][105620] Updated weights for policy 1, policy_version 727425 (0.0009) [2023-12-26 20:39:24,036][105692] Updated weights for policy 0, policy_version 726830 (0.0010) [2023-12-26 20:39:24,096][105692] Updated weights for policy 0, policy_version 726840 (0.0009) [2023-12-26 20:39:24,158][105692] Updated weights for policy 0, policy_version 726850 (0.0009) [2023-12-26 20:39:24,544][105620] Updated weights for policy 1, policy_version 727436 (0.0008) [2023-12-26 20:39:24,598][105620] Updated weights for policy 1, policy_version 727446 (0.0009) [2023-12-26 20:39:24,652][105620] Updated weights for policy 1, policy_version 727456 (0.0009) [2023-12-26 20:39:24,899][105692] Updated weights for policy 0, policy_version 726860 (0.0009) [2023-12-26 20:39:24,957][105692] Updated weights for policy 0, policy_version 726870 (0.0009) [2023-12-26 20:39:25,017][105692] Updated weights for policy 0, policy_version 726880 (0.0009) [2023-12-26 20:39:25,480][105620] Updated weights for policy 1, policy_version 727466 (0.0009) [2023-12-26 20:39:25,527][105620] Updated weights for policy 1, policy_version 727476 (0.0008) [2023-12-26 20:39:25,581][105620] Updated weights for policy 1, policy_version 727486 (0.0005) [2023-12-26 20:39:25,601][105692] Updated weights for policy 0, policy_version 726890 (0.0008) [2023-12-26 20:39:25,647][105620] Updated weights for policy 1, policy_version 727496 (0.0007) [2023-12-26 20:39:25,650][105692] Updated weights for policy 0, policy_version 726900 (0.0007) [2023-12-26 20:39:25,703][105692] Updated weights for policy 0, policy_version 726910 (0.0009) [2023-12-26 20:39:25,755][105692] Updated weights for policy 0, policy_version 726920 (0.0009) [2023-12-26 20:39:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 372383744. Throughput: 0: 9944.7, 1: 9853.8. Samples: 372391936. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 20:39:26,062][104569] Avg episode reward: [(0, '9100.395'), (1, '9172.523')] [2023-12-26 20:39:26,320][105620] Updated weights for policy 1, policy_version 727506 (0.0009) [2023-12-26 20:39:26,374][105620] Updated weights for policy 1, policy_version 727516 (0.0009) [2023-12-26 20:39:26,425][105620] Updated weights for policy 1, policy_version 727526 (0.0008) [2023-12-26 20:39:26,538][105692] Updated weights for policy 0, policy_version 726930 (0.0009) [2023-12-26 20:39:26,588][105692] Updated weights for policy 0, policy_version 726940 (0.0009) [2023-12-26 20:39:26,645][105692] Updated weights for policy 0, policy_version 726950 (0.0009) [2023-12-26 20:39:27,179][105620] Updated weights for policy 1, policy_version 727536 (0.0009) [2023-12-26 20:39:27,226][105620] Updated weights for policy 1, policy_version 727546 (0.0008) [2023-12-26 20:39:27,283][105620] Updated weights for policy 1, policy_version 727556 (0.0009) [2023-12-26 20:39:27,396][105692] Updated weights for policy 0, policy_version 726960 (0.0006) [2023-12-26 20:39:27,461][105692] Updated weights for policy 0, policy_version 726970 (0.0005) [2023-12-26 20:39:27,522][105692] Updated weights for policy 0, policy_version 726980 (0.0005) [2023-12-26 20:39:28,017][105692] Updated weights for policy 0, policy_version 726990 (0.0007) [2023-12-26 20:39:28,081][105692] Updated weights for policy 0, policy_version 727000 (0.0008) [2023-12-26 20:39:28,110][105620] Updated weights for policy 1, policy_version 727566 (0.0009) [2023-12-26 20:39:28,132][105692] Updated weights for policy 0, policy_version 727010 (0.0007) [2023-12-26 20:39:28,155][105620] Updated weights for policy 1, policy_version 727576 (0.0006) [2023-12-26 20:39:28,203][105620] Updated weights for policy 1, policy_version 727586 (0.0008) [2023-12-26 20:39:28,796][105692] Updated weights for policy 0, policy_version 727020 (0.0007) [2023-12-26 20:39:28,853][105692] Updated weights for policy 0, policy_version 727030 (0.0009) [2023-12-26 20:39:28,910][105692] Updated weights for policy 0, policy_version 727040 (0.0005) [2023-12-26 20:39:29,023][105620] Updated weights for policy 1, policy_version 727596 (0.0009) [2023-12-26 20:39:29,083][105620] Updated weights for policy 1, policy_version 727606 (0.0010) [2023-12-26 20:39:29,140][105620] Updated weights for policy 1, policy_version 727616 (0.0009) [2023-12-26 20:39:29,489][105692] Updated weights for policy 0, policy_version 727050 (0.0005) [2023-12-26 20:39:29,543][105692] Updated weights for policy 0, policy_version 727060 (0.0005) [2023-12-26 20:39:29,602][105692] Updated weights for policy 0, policy_version 727070 (0.0006) [2023-12-26 20:39:29,656][105692] Updated weights for policy 0, policy_version 727080 (0.0006) [2023-12-26 20:39:29,964][105620] Updated weights for policy 1, policy_version 727626 (0.0010) [2023-12-26 20:39:30,022][105620] Updated weights for policy 1, policy_version 727636 (0.0006) [2023-12-26 20:39:30,073][105620] Updated weights for policy 1, policy_version 727646 (0.0007) [2023-12-26 20:39:30,130][105620] Updated weights for policy 1, policy_version 727656 (0.0009) [2023-12-26 20:39:30,244][105692] Updated weights for policy 0, policy_version 727090 (0.0010) [2023-12-26 20:39:30,304][105692] Updated weights for policy 0, policy_version 727100 (0.0009) [2023-12-26 20:39:30,359][105692] Updated weights for policy 0, policy_version 727110 (0.0009) [2023-12-26 20:39:30,919][105620] Updated weights for policy 1, policy_version 727666 (0.0009) [2023-12-26 20:39:30,977][105620] Updated weights for policy 1, policy_version 727676 (0.0009) [2023-12-26 20:39:30,982][105692] Updated weights for policy 0, policy_version 727120 (0.0006) [2023-12-26 20:39:31,036][105692] Updated weights for policy 0, policy_version 727130 (0.0006) [2023-12-26 20:39:31,038][105620] Updated weights for policy 1, policy_version 727686 (0.0008) [2023-12-26 20:39:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 372482048. Throughput: 0: 10006.5, 1: 9788.5. Samples: 372449640. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 20:39:31,062][104569] Avg episode reward: [(0, '9038.202'), (1, '9263.313')] [2023-12-26 20:39:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000727688_186310656.pth... [2023-12-26 20:39:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000726536_186015744.pth [2023-12-26 20:39:31,098][105692] Updated weights for policy 0, policy_version 727140 (0.0009) [2023-12-26 20:39:31,118][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000727144_186179584.pth... [2023-12-26 20:39:31,123][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000725960_185876480.pth [2023-12-26 20:39:31,795][105620] Updated weights for policy 1, policy_version 727696 (0.0008) [2023-12-26 20:39:31,813][105692] Updated weights for policy 0, policy_version 727150 (0.0007) [2023-12-26 20:39:31,857][105620] Updated weights for policy 1, policy_version 727706 (0.0008) [2023-12-26 20:39:31,868][105692] Updated weights for policy 0, policy_version 727160 (0.0005) [2023-12-26 20:39:31,921][105620] Updated weights for policy 1, policy_version 727716 (0.0008) [2023-12-26 20:39:31,932][105692] Updated weights for policy 0, policy_version 727170 (0.0006) [2023-12-26 20:39:32,495][105692] Updated weights for policy 0, policy_version 727180 (0.0007) [2023-12-26 20:39:32,547][105692] Updated weights for policy 0, policy_version 727190 (0.0009) [2023-12-26 20:39:32,599][105692] Updated weights for policy 0, policy_version 727200 (0.0009) [2023-12-26 20:39:32,707][105620] Updated weights for policy 1, policy_version 727726 (0.0008) [2023-12-26 20:39:32,753][105620] Updated weights for policy 1, policy_version 727736 (0.0008) [2023-12-26 20:39:32,801][105620] Updated weights for policy 1, policy_version 727746 (0.0009) [2023-12-26 20:39:33,222][105692] Updated weights for policy 0, policy_version 727210 (0.0008) [2023-12-26 20:39:33,269][105692] Updated weights for policy 0, policy_version 727220 (0.0005) [2023-12-26 20:39:33,327][105692] Updated weights for policy 0, policy_version 727230 (0.0005) [2023-12-26 20:39:33,384][105692] Updated weights for policy 0, policy_version 727240 (0.0005) [2023-12-26 20:39:33,701][105620] Updated weights for policy 1, policy_version 727756 (0.0008) [2023-12-26 20:39:33,752][105620] Updated weights for policy 1, policy_version 727766 (0.0009) [2023-12-26 20:39:33,800][105620] Updated weights for policy 1, policy_version 727776 (0.0009) [2023-12-26 20:39:33,949][105692] Updated weights for policy 0, policy_version 727250 (0.0009) [2023-12-26 20:39:34,007][105692] Updated weights for policy 0, policy_version 727260 (0.0009) [2023-12-26 20:39:34,069][105692] Updated weights for policy 0, policy_version 727270 (0.0009) [2023-12-26 20:39:34,600][105620] Updated weights for policy 1, policy_version 727786 (0.0009) [2023-12-26 20:39:34,651][105620] Updated weights for policy 1, policy_version 727797 (0.0010) [2023-12-26 20:39:34,702][105620] Updated weights for policy 1, policy_version 727807 (0.0009) [2023-12-26 20:39:34,781][105692] Updated weights for policy 0, policy_version 727280 (0.0008) [2023-12-26 20:39:34,835][105692] Updated weights for policy 0, policy_version 727290 (0.0009) [2023-12-26 20:39:34,887][105692] Updated weights for policy 0, policy_version 727300 (0.0009) [2023-12-26 20:39:35,488][105620] Updated weights for policy 1, policy_version 727817 (0.0009) [2023-12-26 20:39:35,556][105620] Updated weights for policy 1, policy_version 727827 (0.0009) [2023-12-26 20:39:35,606][105620] Updated weights for policy 1, policy_version 727837 (0.0009) [2023-12-26 20:39:35,661][105692] Updated weights for policy 0, policy_version 727310 (0.0008) [2023-12-26 20:39:35,664][105620] Updated weights for policy 1, policy_version 727847 (0.0006) [2023-12-26 20:39:35,711][105692] Updated weights for policy 0, policy_version 727320 (0.0009) [2023-12-26 20:39:35,762][105692] Updated weights for policy 0, policy_version 727330 (0.0009) [2023-12-26 20:39:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 372580352. Throughput: 0: 10097.3, 1: 9657.0. Samples: 372568296. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 20:39:36,062][104569] Avg episode reward: [(0, '9050.208'), (1, '9354.293')] [2023-12-26 20:39:36,394][105620] Updated weights for policy 1, policy_version 727857 (0.0010) [2023-12-26 20:39:36,448][105620] Updated weights for policy 1, policy_version 727867 (0.0009) [2023-12-26 20:39:36,498][105620] Updated weights for policy 1, policy_version 727877 (0.0008) [2023-12-26 20:39:36,512][105692] Updated weights for policy 0, policy_version 727340 (0.0009) [2023-12-26 20:39:36,575][105692] Updated weights for policy 0, policy_version 727350 (0.0009) [2023-12-26 20:39:36,630][105692] Updated weights for policy 0, policy_version 727360 (0.0008) [2023-12-26 20:39:37,305][105620] Updated weights for policy 1, policy_version 727887 (0.0008) [2023-12-26 20:39:37,355][105620] Updated weights for policy 1, policy_version 727897 (0.0008) [2023-12-26 20:39:37,391][105692] Updated weights for policy 0, policy_version 727370 (0.0008) [2023-12-26 20:39:37,416][105620] Updated weights for policy 1, policy_version 727907 (0.0009) [2023-12-26 20:39:37,441][105692] Updated weights for policy 0, policy_version 727380 (0.0006) [2023-12-26 20:39:37,495][105692] Updated weights for policy 0, policy_version 727390 (0.0007) [2023-12-26 20:39:37,550][105692] Updated weights for policy 0, policy_version 727400 (0.0005) [2023-12-26 20:39:38,205][105692] Updated weights for policy 0, policy_version 727410 (0.0009) [2023-12-26 20:39:38,232][105620] Updated weights for policy 1, policy_version 727917 (0.0009) [2023-12-26 20:39:38,268][105692] Updated weights for policy 0, policy_version 727420 (0.0007) [2023-12-26 20:39:38,278][105620] Updated weights for policy 1, policy_version 727927 (0.0007) [2023-12-26 20:39:38,334][105692] Updated weights for policy 0, policy_version 727430 (0.0007) [2023-12-26 20:39:38,341][105620] Updated weights for policy 1, policy_version 727937 (0.0006) [2023-12-26 20:39:39,027][105620] Updated weights for policy 1, policy_version 727947 (0.0008) [2023-12-26 20:39:39,078][105620] Updated weights for policy 1, policy_version 727957 (0.0007) [2023-12-26 20:39:39,124][105620] Updated weights for policy 1, policy_version 727967 (0.0009) [2023-12-26 20:39:39,129][105692] Updated weights for policy 0, policy_version 727440 (0.0010) [2023-12-26 20:39:39,176][105692] Updated weights for policy 0, policy_version 727450 (0.0007) [2023-12-26 20:39:39,223][105692] Updated weights for policy 0, policy_version 727460 (0.0008) [2023-12-26 20:39:39,917][105620] Updated weights for policy 1, policy_version 727977 (0.0007) [2023-12-26 20:39:39,954][105692] Updated weights for policy 0, policy_version 727470 (0.0007) [2023-12-26 20:39:39,987][105620] Updated weights for policy 1, policy_version 727987 (0.0008) [2023-12-26 20:39:40,020][105692] Updated weights for policy 0, policy_version 727480 (0.0007) [2023-12-26 20:39:40,045][105620] Updated weights for policy 1, policy_version 727997 (0.0007) [2023-12-26 20:39:40,082][105692] Updated weights for policy 0, policy_version 727490 (0.0006) [2023-12-26 20:39:40,108][105620] Updated weights for policy 1, policy_version 728007 (0.0007) [2023-12-26 20:39:40,789][105692] Updated weights for policy 0, policy_version 727500 (0.0008) [2023-12-26 20:39:40,847][105692] Updated weights for policy 0, policy_version 727510 (0.0009) [2023-12-26 20:39:40,907][105620] Updated weights for policy 1, policy_version 728017 (0.0008) [2023-12-26 20:39:40,909][105692] Updated weights for policy 0, policy_version 727520 (0.0006) [2023-12-26 20:39:40,968][105620] Updated weights for policy 1, policy_version 728027 (0.0007) [2023-12-26 20:39:41,019][105620] Updated weights for policy 1, policy_version 728037 (0.0009) [2023-12-26 20:39:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 372678656. Throughput: 0: 9999.2, 1: 9585.8. Samples: 372681064. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 20:39:41,063][104569] Avg episode reward: [(0, '9183.112'), (1, '9354.966')] [2023-12-26 20:39:41,672][105692] Updated weights for policy 0, policy_version 727530 (0.0008) [2023-12-26 20:39:41,732][105692] Updated weights for policy 0, policy_version 727540 (0.0007) [2023-12-26 20:39:41,798][105692] Updated weights for policy 0, policy_version 727550 (0.0009) [2023-12-26 20:39:41,829][105620] Updated weights for policy 1, policy_version 728047 (0.0008) [2023-12-26 20:39:41,864][105692] Updated weights for policy 0, policy_version 727560 (0.0006) [2023-12-26 20:39:41,888][105620] Updated weights for policy 1, policy_version 728057 (0.0007) [2023-12-26 20:39:41,940][105620] Updated weights for policy 1, policy_version 728067 (0.0009) [2023-12-26 20:39:42,668][105620] Updated weights for policy 1, policy_version 728077 (0.0008) [2023-12-26 20:39:42,679][105692] Updated weights for policy 0, policy_version 727570 (0.0009) [2023-12-26 20:39:42,727][105620] Updated weights for policy 1, policy_version 728087 (0.0008) [2023-12-26 20:39:42,739][105692] Updated weights for policy 0, policy_version 727580 (0.0008) [2023-12-26 20:39:42,789][105620] Updated weights for policy 1, policy_version 728097 (0.0008) [2023-12-26 20:39:42,792][105692] Updated weights for policy 0, policy_version 727590 (0.0008) [2023-12-26 20:39:43,422][105692] Updated weights for policy 0, policy_version 727600 (0.0010) [2023-12-26 20:39:43,444][105620] Updated weights for policy 1, policy_version 728107 (0.0008) [2023-12-26 20:39:43,477][105692] Updated weights for policy 0, policy_version 727610 (0.0010) [2023-12-26 20:39:43,502][105620] Updated weights for policy 1, policy_version 728117 (0.0005) [2023-12-26 20:39:43,522][105692] Updated weights for policy 0, policy_version 727620 (0.0010) [2023-12-26 20:39:43,555][105620] Updated weights for policy 1, policy_version 728127 (0.0005) [2023-12-26 20:39:44,110][105692] Updated weights for policy 0, policy_version 727630 (0.0009) [2023-12-26 20:39:44,117][105620] Updated weights for policy 1, policy_version 728137 (0.0006) [2023-12-26 20:39:44,168][105692] Updated weights for policy 0, policy_version 727640 (0.0008) [2023-12-26 20:39:44,174][105620] Updated weights for policy 1, policy_version 728147 (0.0009) [2023-12-26 20:39:44,226][105692] Updated weights for policy 0, policy_version 727650 (0.0008) [2023-12-26 20:39:44,238][105620] Updated weights for policy 1, policy_version 728157 (0.0011) [2023-12-26 20:39:44,289][105620] Updated weights for policy 1, policy_version 728167 (0.0010) [2023-12-26 20:39:44,981][105692] Updated weights for policy 0, policy_version 727660 (0.0009) [2023-12-26 20:39:45,037][105692] Updated weights for policy 0, policy_version 727670 (0.0011) [2023-12-26 20:39:45,040][105620] Updated weights for policy 1, policy_version 728177 (0.0006) [2023-12-26 20:39:45,093][105620] Updated weights for policy 1, policy_version 728187 (0.0007) [2023-12-26 20:39:45,101][105692] Updated weights for policy 0, policy_version 727680 (0.0011) [2023-12-26 20:39:45,144][105620] Updated weights for policy 1, policy_version 728197 (0.0007) [2023-12-26 20:39:45,856][105692] Updated weights for policy 0, policy_version 727690 (0.0010) [2023-12-26 20:39:45,915][105692] Updated weights for policy 0, policy_version 727700 (0.0010) [2023-12-26 20:39:45,953][105620] Updated weights for policy 1, policy_version 728207 (0.0006) [2023-12-26 20:39:45,968][105692] Updated weights for policy 0, policy_version 727710 (0.0010) [2023-12-26 20:39:46,002][105620] Updated weights for policy 1, policy_version 728217 (0.0005) [2023-12-26 20:39:46,019][105692] Updated weights for policy 0, policy_version 727720 (0.0010) [2023-12-26 20:39:46,060][105620] Updated weights for policy 1, policy_version 728227 (0.0005) [2023-12-26 20:39:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 372768768. Throughput: 0: 9930.8, 1: 9554.7. Samples: 372738752. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 20:39:46,063][104569] Avg episode reward: [(0, '9024.421'), (1, '9171.900')] [2023-12-26 20:39:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000727720_186327040.pth... [2023-12-26 20:39:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000726536_186023936.pth [2023-12-26 20:39:46,088][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000728232_186449920.pth... [2023-12-26 20:39:46,091][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000727112_186163200.pth [2023-12-26 20:39:46,721][105620] Updated weights for policy 1, policy_version 728237 (0.0007) [2023-12-26 20:39:46,744][105692] Updated weights for policy 0, policy_version 727730 (0.0010) [2023-12-26 20:39:46,770][105620] Updated weights for policy 1, policy_version 728247 (0.0005) [2023-12-26 20:39:46,800][105692] Updated weights for policy 0, policy_version 727740 (0.0010) [2023-12-26 20:39:46,837][105620] Updated weights for policy 1, policy_version 728257 (0.0006) [2023-12-26 20:39:46,865][105692] Updated weights for policy 0, policy_version 727750 (0.0011) [2023-12-26 20:39:47,479][105620] Updated weights for policy 1, policy_version 728267 (0.0006) [2023-12-26 20:39:47,538][105620] Updated weights for policy 1, policy_version 728277 (0.0007) [2023-12-26 20:39:47,598][105620] Updated weights for policy 1, policy_version 728287 (0.0006) [2023-12-26 20:39:47,600][105692] Updated weights for policy 0, policy_version 727760 (0.0011) [2023-12-26 20:39:47,652][105692] Updated weights for policy 0, policy_version 727770 (0.0010) [2023-12-26 20:39:47,707][105692] Updated weights for policy 0, policy_version 727780 (0.0010) [2023-12-26 20:39:48,377][105620] Updated weights for policy 1, policy_version 728297 (0.0006) [2023-12-26 20:39:48,427][105620] Updated weights for policy 1, policy_version 728307 (0.0008) [2023-12-26 20:39:48,434][105692] Updated weights for policy 0, policy_version 727790 (0.0010) [2023-12-26 20:39:48,484][105620] Updated weights for policy 1, policy_version 728317 (0.0006) [2023-12-26 20:39:48,493][105692] Updated weights for policy 0, policy_version 727800 (0.0011) [2023-12-26 20:39:48,547][105620] Updated weights for policy 1, policy_version 728327 (0.0005) [2023-12-26 20:39:48,553][105692] Updated weights for policy 0, policy_version 727810 (0.0011) [2023-12-26 20:39:49,178][105692] Updated weights for policy 0, policy_version 727820 (0.0008) [2023-12-26 20:39:49,232][105692] Updated weights for policy 0, policy_version 727830 (0.0008) [2023-12-26 20:39:49,298][105692] Updated weights for policy 0, policy_version 727840 (0.0011) [2023-12-26 20:39:49,301][105620] Updated weights for policy 1, policy_version 728337 (0.0006) [2023-12-26 20:39:49,371][105620] Updated weights for policy 1, policy_version 728347 (0.0008) [2023-12-26 20:39:49,424][105620] Updated weights for policy 1, policy_version 728357 (0.0008) [2023-12-26 20:39:50,035][105692] Updated weights for policy 0, policy_version 727850 (0.0007) [2023-12-26 20:39:50,095][105692] Updated weights for policy 0, policy_version 727860 (0.0008) [2023-12-26 20:39:50,159][105692] Updated weights for policy 0, policy_version 727870 (0.0006) [2023-12-26 20:39:50,187][105620] Updated weights for policy 1, policy_version 728367 (0.0008) [2023-12-26 20:39:50,222][105692] Updated weights for policy 0, policy_version 727880 (0.0008) [2023-12-26 20:39:50,242][105620] Updated weights for policy 1, policy_version 728377 (0.0008) [2023-12-26 20:39:50,297][105620] Updated weights for policy 1, policy_version 728387 (0.0009) [2023-12-26 20:39:50,828][105692] Updated weights for policy 0, policy_version 727890 (0.0008) [2023-12-26 20:39:50,875][105692] Updated weights for policy 0, policy_version 727900 (0.0009) [2023-12-26 20:39:50,929][105692] Updated weights for policy 0, policy_version 727911 (0.0010) [2023-12-26 20:39:51,061][105620] Updated weights for policy 1, policy_version 728397 (0.0008) [2023-12-26 20:39:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 372867072. Throughput: 0: 9913.2, 1: 9504.0. Samples: 372856140. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 20:39:51,063][104569] Avg episode reward: [(0, '8932.949'), (1, '9171.852')] [2023-12-26 20:39:51,127][105620] Updated weights for policy 1, policy_version 728407 (0.0009) [2023-12-26 20:39:51,192][105620] Updated weights for policy 1, policy_version 728417 (0.0009) [2023-12-26 20:39:51,768][105692] Updated weights for policy 0, policy_version 727921 (0.0009) [2023-12-26 20:39:51,820][105692] Updated weights for policy 0, policy_version 727931 (0.0007) [2023-12-26 20:39:51,876][105692] Updated weights for policy 0, policy_version 727941 (0.0005) [2023-12-26 20:39:51,995][105620] Updated weights for policy 1, policy_version 728427 (0.0009) [2023-12-26 20:39:52,059][105620] Updated weights for policy 1, policy_version 728437 (0.0008) [2023-12-26 20:39:52,111][105620] Updated weights for policy 1, policy_version 728447 (0.0009) [2023-12-26 20:39:52,532][105692] Updated weights for policy 0, policy_version 727951 (0.0007) [2023-12-26 20:39:52,594][105692] Updated weights for policy 0, policy_version 727961 (0.0008) [2023-12-26 20:39:52,653][105692] Updated weights for policy 0, policy_version 727971 (0.0009) [2023-12-26 20:39:52,824][105620] Updated weights for policy 1, policy_version 728457 (0.0008) [2023-12-26 20:39:52,884][105620] Updated weights for policy 1, policy_version 728467 (0.0010) [2023-12-26 20:39:52,940][105620] Updated weights for policy 1, policy_version 728477 (0.0010) [2023-12-26 20:39:52,995][105620] Updated weights for policy 1, policy_version 728487 (0.0010) [2023-12-26 20:39:53,297][105692] Updated weights for policy 0, policy_version 727981 (0.0007) [2023-12-26 20:39:53,363][105692] Updated weights for policy 0, policy_version 727991 (0.0007) [2023-12-26 20:39:53,422][105692] Updated weights for policy 0, policy_version 728001 (0.0006) [2023-12-26 20:39:53,710][105620] Updated weights for policy 1, policy_version 728497 (0.0006) [2023-12-26 20:39:53,769][105620] Updated weights for policy 1, policy_version 728507 (0.0007) [2023-12-26 20:39:53,832][105620] Updated weights for policy 1, policy_version 728517 (0.0007) [2023-12-26 20:39:54,167][105692] Updated weights for policy 0, policy_version 728011 (0.0008) [2023-12-26 20:39:54,227][105692] Updated weights for policy 0, policy_version 728021 (0.0005) [2023-12-26 20:39:54,291][105692] Updated weights for policy 0, policy_version 728031 (0.0005) [2023-12-26 20:39:54,527][105620] Updated weights for policy 1, policy_version 728527 (0.0009) [2023-12-26 20:39:54,580][105620] Updated weights for policy 1, policy_version 728537 (0.0010) [2023-12-26 20:39:54,632][105620] Updated weights for policy 1, policy_version 728547 (0.0010) [2023-12-26 20:39:54,797][105692] Updated weights for policy 0, policy_version 728041 (0.0005) [2023-12-26 20:39:54,869][105692] Updated weights for policy 0, policy_version 728051 (0.0005) [2023-12-26 20:39:54,928][105692] Updated weights for policy 0, policy_version 728061 (0.0005) [2023-12-26 20:39:54,984][105692] Updated weights for policy 0, policy_version 728071 (0.0006) [2023-12-26 20:39:55,390][105620] Updated weights for policy 1, policy_version 728557 (0.0009) [2023-12-26 20:39:55,454][105620] Updated weights for policy 1, policy_version 728567 (0.0008) [2023-12-26 20:39:55,517][105620] Updated weights for policy 1, policy_version 728577 (0.0008) [2023-12-26 20:39:55,572][105692] Updated weights for policy 0, policy_version 728081 (0.0007) [2023-12-26 20:39:55,620][105692] Updated weights for policy 0, policy_version 728091 (0.0009) [2023-12-26 20:39:55,679][105692] Updated weights for policy 0, policy_version 728101 (0.0009) [2023-12-26 20:39:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 372965376. Throughput: 0: 10065.6, 1: 9395.2. Samples: 372974784. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 20:39:56,062][104569] Avg episode reward: [(0, '8969.992'), (1, '9173.060')] [2023-12-26 20:39:56,279][105620] Updated weights for policy 1, policy_version 728587 (0.0007) [2023-12-26 20:39:56,332][105620] Updated weights for policy 1, policy_version 728597 (0.0006) [2023-12-26 20:39:56,334][105692] Updated weights for policy 0, policy_version 728111 (0.0009) [2023-12-26 20:39:56,385][105620] Updated weights for policy 1, policy_version 728607 (0.0006) [2023-12-26 20:39:56,401][105692] Updated weights for policy 0, policy_version 728121 (0.0008) [2023-12-26 20:39:56,461][105692] Updated weights for policy 0, policy_version 728131 (0.0009) [2023-12-26 20:39:57,131][105620] Updated weights for policy 1, policy_version 728617 (0.0006) [2023-12-26 20:39:57,139][105692] Updated weights for policy 0, policy_version 728141 (0.0008) [2023-12-26 20:39:57,190][105620] Updated weights for policy 1, policy_version 728627 (0.0007) [2023-12-26 20:39:57,192][105692] Updated weights for policy 0, policy_version 728151 (0.0006) [2023-12-26 20:39:57,234][105620] Updated weights for policy 1, policy_version 728637 (0.0006) [2023-12-26 20:39:57,244][105692] Updated weights for policy 0, policy_version 728161 (0.0008) [2023-12-26 20:39:57,279][105620] Updated weights for policy 1, policy_version 728647 (0.0006) [2023-12-26 20:39:57,857][105692] Updated weights for policy 0, policy_version 728171 (0.0008) [2023-12-26 20:39:57,915][105692] Updated weights for policy 0, policy_version 728181 (0.0009) [2023-12-26 20:39:57,976][105692] Updated weights for policy 0, policy_version 728191 (0.0009) [2023-12-26 20:39:58,093][105620] Updated weights for policy 1, policy_version 728657 (0.0009) [2023-12-26 20:39:58,151][105620] Updated weights for policy 1, policy_version 728667 (0.0009) [2023-12-26 20:39:58,215][105620] Updated weights for policy 1, policy_version 728677 (0.0007) [2023-12-26 20:39:58,735][105692] Updated weights for policy 0, policy_version 728201 (0.0009) [2023-12-26 20:39:58,798][105692] Updated weights for policy 0, policy_version 728211 (0.0010) [2023-12-26 20:39:58,863][105692] Updated weights for policy 0, policy_version 728221 (0.0011) [2023-12-26 20:39:58,925][105692] Updated weights for policy 0, policy_version 728231 (0.0011) [2023-12-26 20:39:58,972][105620] Updated weights for policy 1, policy_version 728687 (0.0007) [2023-12-26 20:39:59,019][105620] Updated weights for policy 1, policy_version 728697 (0.0008) [2023-12-26 20:39:59,072][105620] Updated weights for policy 1, policy_version 728707 (0.0008) [2023-12-26 20:39:59,643][105692] Updated weights for policy 0, policy_version 728241 (0.0009) [2023-12-26 20:39:59,705][105692] Updated weights for policy 0, policy_version 728251 (0.0009) [2023-12-26 20:39:59,761][105692] Updated weights for policy 0, policy_version 728261 (0.0008) [2023-12-26 20:39:59,763][105620] Updated weights for policy 1, policy_version 728717 (0.0007) [2023-12-26 20:39:59,811][105620] Updated weights for policy 1, policy_version 728727 (0.0008) [2023-12-26 20:39:59,868][105620] Updated weights for policy 1, policy_version 728737 (0.0008) [2023-12-26 20:40:00,381][105692] Updated weights for policy 0, policy_version 728271 (0.0008) [2023-12-26 20:40:00,440][105692] Updated weights for policy 0, policy_version 728281 (0.0005) [2023-12-26 20:40:00,504][105692] Updated weights for policy 0, policy_version 728291 (0.0007) [2023-12-26 20:40:00,660][105620] Updated weights for policy 1, policy_version 728747 (0.0006) [2023-12-26 20:40:00,711][105620] Updated weights for policy 1, policy_version 728757 (0.0005) [2023-12-26 20:40:00,767][105620] Updated weights for policy 1, policy_version 728767 (0.0005) [2023-12-26 20:40:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 373063680. Throughput: 0: 10123.9, 1: 9372.6. Samples: 373033172. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 20:40:01,062][104569] Avg episode reward: [(0, '9081.356'), (1, '9173.907')] [2023-12-26 20:40:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000728296_186474496.pth... [2023-12-26 20:40:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000728776_186589184.pth... [2023-12-26 20:40:01,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000727688_186310656.pth [2023-12-26 20:40:01,084][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000727144_186179584.pth [2023-12-26 20:40:01,268][105692] Updated weights for policy 0, policy_version 728301 (0.0009) [2023-12-26 20:40:01,322][105692] Updated weights for policy 0, policy_version 728311 (0.0009) [2023-12-26 20:40:01,385][105692] Updated weights for policy 0, policy_version 728321 (0.0008) [2023-12-26 20:40:01,480][105620] Updated weights for policy 1, policy_version 728777 (0.0008) [2023-12-26 20:40:01,527][105620] Updated weights for policy 1, policy_version 728787 (0.0009) [2023-12-26 20:40:01,573][105620] Updated weights for policy 1, policy_version 728797 (0.0008) [2023-12-26 20:40:01,633][105620] Updated weights for policy 1, policy_version 728807 (0.0010) [2023-12-26 20:40:02,069][105692] Updated weights for policy 0, policy_version 728331 (0.0008) [2023-12-26 20:40:02,138][105692] Updated weights for policy 0, policy_version 728341 (0.0005) [2023-12-26 20:40:02,195][105692] Updated weights for policy 0, policy_version 728351 (0.0005) [2023-12-26 20:40:02,452][105620] Updated weights for policy 1, policy_version 728817 (0.0010) [2023-12-26 20:40:02,511][105620] Updated weights for policy 1, policy_version 728827 (0.0011) [2023-12-26 20:40:02,570][105620] Updated weights for policy 1, policy_version 728837 (0.0010) [2023-12-26 20:40:02,822][105692] Updated weights for policy 0, policy_version 728361 (0.0006) [2023-12-26 20:40:02,876][105692] Updated weights for policy 0, policy_version 728371 (0.0008) [2023-12-26 20:40:02,933][105692] Updated weights for policy 0, policy_version 728381 (0.0008) [2023-12-26 20:40:02,998][105692] Updated weights for policy 0, policy_version 728391 (0.0005) [2023-12-26 20:40:03,311][105620] Updated weights for policy 1, policy_version 728847 (0.0010) [2023-12-26 20:40:03,359][105620] Updated weights for policy 1, policy_version 728857 (0.0010) [2023-12-26 20:40:03,413][105620] Updated weights for policy 1, policy_version 728867 (0.0010) [2023-12-26 20:40:03,559][105692] Updated weights for policy 0, policy_version 728401 (0.0005) [2023-12-26 20:40:03,620][105692] Updated weights for policy 0, policy_version 728411 (0.0006) [2023-12-26 20:40:03,680][105692] Updated weights for policy 0, policy_version 728421 (0.0010) [2023-12-26 20:40:04,185][105620] Updated weights for policy 1, policy_version 728877 (0.0010) [2023-12-26 20:40:04,248][105620] Updated weights for policy 1, policy_version 728887 (0.0011) [2023-12-26 20:40:04,307][105620] Updated weights for policy 1, policy_version 728897 (0.0011) [2023-12-26 20:40:04,360][105692] Updated weights for policy 0, policy_version 728431 (0.0009) [2023-12-26 20:40:04,420][105692] Updated weights for policy 0, policy_version 728441 (0.0008) [2023-12-26 20:40:04,480][105692] Updated weights for policy 0, policy_version 728451 (0.0009) [2023-12-26 20:40:05,056][105620] Updated weights for policy 1, policy_version 728908 (0.0011) [2023-12-26 20:40:05,110][105620] Updated weights for policy 1, policy_version 728918 (0.0009) [2023-12-26 20:40:05,158][105620] Updated weights for policy 1, policy_version 728928 (0.0009) [2023-12-26 20:40:05,187][105692] Updated weights for policy 0, policy_version 728461 (0.0008) [2023-12-26 20:40:05,241][105692] Updated weights for policy 0, policy_version 728471 (0.0009) [2023-12-26 20:40:05,301][105692] Updated weights for policy 0, policy_version 728481 (0.0009) [2023-12-26 20:40:05,833][105620] Updated weights for policy 1, policy_version 728938 (0.0006) [2023-12-26 20:40:05,884][105620] Updated weights for policy 1, policy_version 728948 (0.0005) [2023-12-26 20:40:05,932][105620] Updated weights for policy 1, policy_version 728958 (0.0007) [2023-12-26 20:40:05,981][105620] Updated weights for policy 1, policy_version 728968 (0.0005) [2023-12-26 20:40:06,004][105692] Updated weights for policy 0, policy_version 728491 (0.0008) [2023-12-26 20:40:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 373161984. Throughput: 0: 10067.7, 1: 9309.8. Samples: 373150028. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 20:40:06,063][104569] Avg episode reward: [(0, '8992.176'), (1, '9264.751')] [2023-12-26 20:40:06,063][105692] Updated weights for policy 0, policy_version 728501 (0.0010) [2023-12-26 20:40:06,130][105692] Updated weights for policy 0, policy_version 728511 (0.0010) [2023-12-26 20:40:06,692][105620] Updated weights for policy 1, policy_version 728978 (0.0006) [2023-12-26 20:40:06,741][105620] Updated weights for policy 1, policy_version 728988 (0.0005) [2023-12-26 20:40:06,791][105620] Updated weights for policy 1, policy_version 728998 (0.0008) [2023-12-26 20:40:06,897][105692] Updated weights for policy 0, policy_version 728521 (0.0009) [2023-12-26 20:40:06,951][105692] Updated weights for policy 0, policy_version 728531 (0.0009) [2023-12-26 20:40:07,009][105692] Updated weights for policy 0, policy_version 728541 (0.0009) [2023-12-26 20:40:07,074][105692] Updated weights for policy 0, policy_version 728551 (0.0009) [2023-12-26 20:40:07,512][105620] Updated weights for policy 1, policy_version 729008 (0.0009) [2023-12-26 20:40:07,574][105620] Updated weights for policy 1, policy_version 729018 (0.0009) [2023-12-26 20:40:07,633][105620] Updated weights for policy 1, policy_version 729028 (0.0008) [2023-12-26 20:40:07,822][105692] Updated weights for policy 0, policy_version 728561 (0.0009) [2023-12-26 20:40:07,877][105692] Updated weights for policy 0, policy_version 728571 (0.0009) [2023-12-26 20:40:07,937][105692] Updated weights for policy 0, policy_version 728581 (0.0008) [2023-12-26 20:40:08,332][105620] Updated weights for policy 1, policy_version 729038 (0.0009) [2023-12-26 20:40:08,398][105620] Updated weights for policy 1, policy_version 729048 (0.0008) [2023-12-26 20:40:08,464][105620] Updated weights for policy 1, policy_version 729058 (0.0008) [2023-12-26 20:40:08,745][105692] Updated weights for policy 0, policy_version 728591 (0.0009) [2023-12-26 20:40:08,797][105692] Updated weights for policy 0, policy_version 728601 (0.0009) [2023-12-26 20:40:08,848][105692] Updated weights for policy 0, policy_version 728611 (0.0009) [2023-12-26 20:40:09,179][105620] Updated weights for policy 1, policy_version 729068 (0.0009) [2023-12-26 20:40:09,243][105620] Updated weights for policy 1, policy_version 729078 (0.0010) [2023-12-26 20:40:09,308][105620] Updated weights for policy 1, policy_version 729088 (0.0010) [2023-12-26 20:40:09,683][105692] Updated weights for policy 0, policy_version 728621 (0.0009) [2023-12-26 20:40:09,742][105692] Updated weights for policy 0, policy_version 728631 (0.0009) [2023-12-26 20:40:09,794][105692] Updated weights for policy 0, policy_version 728641 (0.0009) [2023-12-26 20:40:10,069][105620] Updated weights for policy 1, policy_version 729098 (0.0008) [2023-12-26 20:40:10,142][105620] Updated weights for policy 1, policy_version 729108 (0.0010) [2023-12-26 20:40:10,215][105620] Updated weights for policy 1, policy_version 729118 (0.0010) [2023-12-26 20:40:10,290][105620] Updated weights for policy 1, policy_version 729128 (0.0010) [2023-12-26 20:40:10,511][105692] Updated weights for policy 0, policy_version 728651 (0.0008) [2023-12-26 20:40:10,562][105692] Updated weights for policy 0, policy_version 728661 (0.0005) [2023-12-26 20:40:10,620][105692] Updated weights for policy 0, policy_version 728671 (0.0008) [2023-12-26 20:40:11,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 373252096. Throughput: 0: 10036.5, 1: 9333.9. Samples: 373263604. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 20:40:11,063][104569] Avg episode reward: [(0, '9173.420'), (1, '9355.080')] [2023-12-26 20:40:11,082][105620] Updated weights for policy 1, policy_version 729138 (0.0006) [2023-12-26 20:40:11,163][105620] Updated weights for policy 1, policy_version 729148 (0.0007) [2023-12-26 20:40:11,229][105620] Updated weights for policy 1, policy_version 729158 (0.0007) [2023-12-26 20:40:11,348][105692] Updated weights for policy 0, policy_version 728681 (0.0008) [2023-12-26 20:40:11,418][105692] Updated weights for policy 0, policy_version 728691 (0.0008) [2023-12-26 20:40:11,486][105692] Updated weights for policy 0, policy_version 728701 (0.0008) [2023-12-26 20:40:11,549][105692] Updated weights for policy 0, policy_version 728711 (0.0009) [2023-12-26 20:40:11,975][105620] Updated weights for policy 1, policy_version 729168 (0.0010) [2023-12-26 20:40:12,038][105620] Updated weights for policy 1, policy_version 729178 (0.0011) [2023-12-26 20:40:12,101][105620] Updated weights for policy 1, policy_version 729188 (0.0011) [2023-12-26 20:40:12,228][105692] Updated weights for policy 0, policy_version 728721 (0.0008) [2023-12-26 20:40:12,288][105692] Updated weights for policy 0, policy_version 728731 (0.0007) [2023-12-26 20:40:12,349][105692] Updated weights for policy 0, policy_version 728741 (0.0008) [2023-12-26 20:40:12,833][105620] Updated weights for policy 1, policy_version 729198 (0.0010) [2023-12-26 20:40:12,894][105620] Updated weights for policy 1, policy_version 729208 (0.0011) [2023-12-26 20:40:12,950][105620] Updated weights for policy 1, policy_version 729218 (0.0010) [2023-12-26 20:40:13,042][105692] Updated weights for policy 0, policy_version 728751 (0.0008) [2023-12-26 20:40:13,094][105692] Updated weights for policy 0, policy_version 728761 (0.0008) [2023-12-26 20:40:13,154][105692] Updated weights for policy 0, policy_version 728771 (0.0010) [2023-12-26 20:40:13,664][105620] Updated weights for policy 1, policy_version 729228 (0.0011) [2023-12-26 20:40:13,717][105620] Updated weights for policy 1, policy_version 729238 (0.0010) [2023-12-26 20:40:13,769][105620] Updated weights for policy 1, policy_version 729248 (0.0011) [2023-12-26 20:40:13,902][105692] Updated weights for policy 0, policy_version 728781 (0.0009) [2023-12-26 20:40:13,958][105692] Updated weights for policy 0, policy_version 728791 (0.0008) [2023-12-26 20:40:14,017][105692] Updated weights for policy 0, policy_version 728801 (0.0008) [2023-12-26 20:40:14,538][105620] Updated weights for policy 1, policy_version 729258 (0.0010) [2023-12-26 20:40:14,589][105620] Updated weights for policy 1, policy_version 729268 (0.0010) [2023-12-26 20:40:14,598][105692] Updated weights for policy 0, policy_version 728811 (0.0009) [2023-12-26 20:40:14,638][105620] Updated weights for policy 1, policy_version 729278 (0.0010) [2023-12-26 20:40:14,655][105692] Updated weights for policy 0, policy_version 728821 (0.0008) [2023-12-26 20:40:14,697][105620] Updated weights for policy 1, policy_version 729288 (0.0010) [2023-12-26 20:40:14,718][105692] Updated weights for policy 0, policy_version 728831 (0.0011) [2023-12-26 20:40:15,437][105620] Updated weights for policy 1, policy_version 729298 (0.0011) [2023-12-26 20:40:15,494][105692] Updated weights for policy 0, policy_version 728841 (0.0010) [2023-12-26 20:40:15,496][105620] Updated weights for policy 1, policy_version 729308 (0.0010) [2023-12-26 20:40:15,552][105620] Updated weights for policy 1, policy_version 729318 (0.0010) [2023-12-26 20:40:15,554][105692] Updated weights for policy 0, policy_version 728851 (0.0011) [2023-12-26 20:40:15,613][105692] Updated weights for policy 0, policy_version 728861 (0.0010) [2023-12-26 20:40:15,675][105692] Updated weights for policy 0, policy_version 728871 (0.0010) [2023-12-26 20:40:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 373350400. Throughput: 0: 10000.0, 1: 9346.7. Samples: 373320244. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 20:40:16,063][104569] Avg episode reward: [(0, '9351.761'), (1, '9106.548')] [2023-12-26 20:40:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000728872_186621952.pth... [2023-12-26 20:40:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000729320_186728448.pth... [2023-12-26 20:40:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000727720_186327040.pth [2023-12-26 20:40:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000728232_186449920.pth [2023-12-26 20:40:16,303][105620] Updated weights for policy 1, policy_version 729328 (0.0010) [2023-12-26 20:40:16,347][105620] Updated weights for policy 1, policy_version 729338 (0.0010) [2023-12-26 20:40:16,394][105620] Updated weights for policy 1, policy_version 729348 (0.0010) [2023-12-26 20:40:16,442][105692] Updated weights for policy 0, policy_version 728881 (0.0007) [2023-12-26 20:40:16,508][105692] Updated weights for policy 0, policy_version 728891 (0.0008) [2023-12-26 20:40:16,573][105692] Updated weights for policy 0, policy_version 728901 (0.0009) [2023-12-26 20:40:17,155][105620] Updated weights for policy 1, policy_version 729358 (0.0010) [2023-12-26 20:40:17,222][105620] Updated weights for policy 1, policy_version 729368 (0.0010) [2023-12-26 20:40:17,290][105620] Updated weights for policy 1, policy_version 729378 (0.0010) [2023-12-26 20:40:17,316][105692] Updated weights for policy 0, policy_version 728911 (0.0008) [2023-12-26 20:40:17,380][105692] Updated weights for policy 0, policy_version 728921 (0.0007) [2023-12-26 20:40:17,432][105692] Updated weights for policy 0, policy_version 728931 (0.0007) [2023-12-26 20:40:18,018][105620] Updated weights for policy 1, policy_version 729388 (0.0010) [2023-12-26 20:40:18,084][105620] Updated weights for policy 1, policy_version 729398 (0.0010) [2023-12-26 20:40:18,112][105692] Updated weights for policy 0, policy_version 728941 (0.0007) [2023-12-26 20:40:18,135][105620] Updated weights for policy 1, policy_version 729408 (0.0010) [2023-12-26 20:40:18,176][105692] Updated weights for policy 0, policy_version 728951 (0.0005) [2023-12-26 20:40:18,239][105692] Updated weights for policy 0, policy_version 728961 (0.0006) [2023-12-26 20:40:18,812][105692] Updated weights for policy 0, policy_version 728971 (0.0007) [2023-12-26 20:40:18,872][105692] Updated weights for policy 0, policy_version 728981 (0.0008) [2023-12-26 20:40:18,920][105620] Updated weights for policy 1, policy_version 729418 (0.0010) [2023-12-26 20:40:18,926][105692] Updated weights for policy 0, policy_version 728991 (0.0007) [2023-12-26 20:40:18,971][105620] Updated weights for policy 1, policy_version 729428 (0.0010) [2023-12-26 20:40:19,029][105620] Updated weights for policy 1, policy_version 729438 (0.0010) [2023-12-26 20:40:19,083][105620] Updated weights for policy 1, policy_version 729448 (0.0010) [2023-12-26 20:40:19,720][105692] Updated weights for policy 0, policy_version 729001 (0.0006) [2023-12-26 20:40:19,777][105692] Updated weights for policy 0, policy_version 729011 (0.0008) [2023-12-26 20:40:19,837][105692] Updated weights for policy 0, policy_version 729021 (0.0006) [2023-12-26 20:40:19,846][105620] Updated weights for policy 1, policy_version 729458 (0.0011) [2023-12-26 20:40:19,906][105692] Updated weights for policy 0, policy_version 729031 (0.0007) [2023-12-26 20:40:19,911][105620] Updated weights for policy 1, policy_version 729468 (0.0010) [2023-12-26 20:40:19,976][105620] Updated weights for policy 1, policy_version 729478 (0.0009) [2023-12-26 20:40:20,591][105692] Updated weights for policy 0, policy_version 729041 (0.0008) [2023-12-26 20:40:20,655][105692] Updated weights for policy 0, policy_version 729051 (0.0011) [2023-12-26 20:40:20,726][105692] Updated weights for policy 0, policy_version 729061 (0.0009) [2023-12-26 20:40:20,804][105620] Updated weights for policy 1, policy_version 729488 (0.0007) [2023-12-26 20:40:20,869][105620] Updated weights for policy 1, policy_version 729498 (0.0008) [2023-12-26 20:40:20,929][105620] Updated weights for policy 1, policy_version 729508 (0.0011) [2023-12-26 20:40:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 373448704. Throughput: 0: 9855.1, 1: 9421.2. Samples: 373435732. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 20:40:21,063][104569] Avg episode reward: [(0, '9351.858'), (1, '8789.086')] [2023-12-26 20:40:21,324][105692] Updated weights for policy 0, policy_version 729071 (0.0008) [2023-12-26 20:40:21,392][105692] Updated weights for policy 0, policy_version 729081 (0.0009) [2023-12-26 20:40:21,447][105692] Updated weights for policy 0, policy_version 729091 (0.0010) [2023-12-26 20:40:21,695][105620] Updated weights for policy 1, policy_version 729518 (0.0009) [2023-12-26 20:40:21,766][105620] Updated weights for policy 1, policy_version 729528 (0.0008) [2023-12-26 20:40:21,838][105620] Updated weights for policy 1, policy_version 729538 (0.0008) [2023-12-26 20:40:22,262][105692] Updated weights for policy 0, policy_version 729101 (0.0010) [2023-12-26 20:40:22,328][105692] Updated weights for policy 0, policy_version 729111 (0.0011) [2023-12-26 20:40:22,391][105692] Updated weights for policy 0, policy_version 729121 (0.0008) [2023-12-26 20:40:22,586][105620] Updated weights for policy 1, policy_version 729548 (0.0008) [2023-12-26 20:40:22,650][105620] Updated weights for policy 1, policy_version 729558 (0.0008) [2023-12-26 20:40:22,715][105620] Updated weights for policy 1, policy_version 729568 (0.0008) [2023-12-26 20:40:23,059][105692] Updated weights for policy 0, policy_version 729131 (0.0009) [2023-12-26 20:40:23,119][105692] Updated weights for policy 0, policy_version 729141 (0.0009) [2023-12-26 20:40:23,186][105692] Updated weights for policy 0, policy_version 729151 (0.0010) [2023-12-26 20:40:23,443][105620] Updated weights for policy 1, policy_version 729578 (0.0008) [2023-12-26 20:40:23,501][105620] Updated weights for policy 1, policy_version 729588 (0.0009) [2023-12-26 20:40:23,566][105620] Updated weights for policy 1, policy_version 729598 (0.0008) [2023-12-26 20:40:23,623][105620] Updated weights for policy 1, policy_version 729608 (0.0008) [2023-12-26 20:40:23,867][105692] Updated weights for policy 0, policy_version 729161 (0.0007) [2023-12-26 20:40:23,937][105692] Updated weights for policy 0, policy_version 729171 (0.0011) [2023-12-26 20:40:24,007][105692] Updated weights for policy 0, policy_version 729181 (0.0011) [2023-12-26 20:40:24,066][105692] Updated weights for policy 0, policy_version 729191 (0.0011) [2023-12-26 20:40:24,234][105620] Updated weights for policy 1, policy_version 729618 (0.0006) [2023-12-26 20:40:24,288][105620] Updated weights for policy 1, policy_version 729628 (0.0005) [2023-12-26 20:40:24,344][105620] Updated weights for policy 1, policy_version 729638 (0.0005) [2023-12-26 20:40:24,848][105692] Updated weights for policy 0, policy_version 729201 (0.0006) [2023-12-26 20:40:24,875][105620] Updated weights for policy 1, policy_version 729648 (0.0008) [2023-12-26 20:40:24,909][105692] Updated weights for policy 0, policy_version 729211 (0.0006) [2023-12-26 20:40:24,947][105620] Updated weights for policy 1, policy_version 729658 (0.0008) [2023-12-26 20:40:24,955][105692] Updated weights for policy 0, policy_version 729221 (0.0006) [2023-12-26 20:40:25,011][105620] Updated weights for policy 1, policy_version 729668 (0.0007) [2023-12-26 20:40:25,569][105692] Updated weights for policy 0, policy_version 729231 (0.0007) [2023-12-26 20:40:25,635][105692] Updated weights for policy 0, policy_version 729241 (0.0006) [2023-12-26 20:40:25,684][105692] Updated weights for policy 0, policy_version 729251 (0.0006) [2023-12-26 20:40:25,720][105620] Updated weights for policy 1, policy_version 729678 (0.0007) [2023-12-26 20:40:25,785][105620] Updated weights for policy 1, policy_version 729688 (0.0007) [2023-12-26 20:40:25,851][105620] Updated weights for policy 1, policy_version 729698 (0.0010) [2023-12-26 20:40:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 373547008. Throughput: 0: 9891.7, 1: 9490.8. Samples: 373553280. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 20:40:26,063][104569] Avg episode reward: [(0, '9351.629'), (1, '8357.354')] [2023-12-26 20:40:26,307][105692] Updated weights for policy 0, policy_version 729261 (0.0007) [2023-12-26 20:40:26,349][105692] Updated weights for policy 0, policy_version 729271 (0.0005) [2023-12-26 20:40:26,396][105692] Updated weights for policy 0, policy_version 729281 (0.0005) [2023-12-26 20:40:26,629][105620] Updated weights for policy 1, policy_version 729708 (0.0009) [2023-12-26 20:40:26,686][105620] Updated weights for policy 1, policy_version 729718 (0.0008) [2023-12-26 20:40:26,742][105620] Updated weights for policy 1, policy_version 729728 (0.0009) [2023-12-26 20:40:26,943][105692] Updated weights for policy 0, policy_version 729291 (0.0006) [2023-12-26 20:40:27,005][105692] Updated weights for policy 0, policy_version 729301 (0.0011) [2023-12-26 20:40:27,060][105692] Updated weights for policy 0, policy_version 729311 (0.0010) [2023-12-26 20:40:27,526][105620] Updated weights for policy 1, policy_version 729738 (0.0009) [2023-12-26 20:40:27,577][105620] Updated weights for policy 1, policy_version 729748 (0.0008) [2023-12-26 20:40:27,625][105620] Updated weights for policy 1, policy_version 729758 (0.0008) [2023-12-26 20:40:27,672][105620] Updated weights for policy 1, policy_version 729768 (0.0007) [2023-12-26 20:40:27,790][105692] Updated weights for policy 0, policy_version 729321 (0.0010) [2023-12-26 20:40:27,858][105692] Updated weights for policy 0, policy_version 729331 (0.0010) [2023-12-26 20:40:27,915][105692] Updated weights for policy 0, policy_version 729341 (0.0010) [2023-12-26 20:40:27,963][105692] Updated weights for policy 0, policy_version 729351 (0.0010) [2023-12-26 20:40:28,421][105620] Updated weights for policy 1, policy_version 729778 (0.0008) [2023-12-26 20:40:28,473][105620] Updated weights for policy 1, policy_version 729788 (0.0008) [2023-12-26 20:40:28,530][105620] Updated weights for policy 1, policy_version 729798 (0.0008) [2023-12-26 20:40:28,705][105692] Updated weights for policy 0, policy_version 729361 (0.0010) [2023-12-26 20:40:28,757][105692] Updated weights for policy 0, policy_version 729371 (0.0010) [2023-12-26 20:40:28,812][105692] Updated weights for policy 0, policy_version 729381 (0.0010) [2023-12-26 20:40:29,313][105620] Updated weights for policy 1, policy_version 729808 (0.0008) [2023-12-26 20:40:29,376][105620] Updated weights for policy 1, policy_version 729818 (0.0008) [2023-12-26 20:40:29,429][105620] Updated weights for policy 1, policy_version 729828 (0.0009) [2023-12-26 20:40:29,570][105692] Updated weights for policy 0, policy_version 729391 (0.0010) [2023-12-26 20:40:29,624][105692] Updated weights for policy 0, policy_version 729401 (0.0010) [2023-12-26 20:40:29,679][105692] Updated weights for policy 0, policy_version 729411 (0.0009) [2023-12-26 20:40:30,058][105620] Updated weights for policy 1, policy_version 729838 (0.0008) [2023-12-26 20:40:30,112][105620] Updated weights for policy 1, policy_version 729848 (0.0007) [2023-12-26 20:40:30,176][105620] Updated weights for policy 1, policy_version 729858 (0.0006) [2023-12-26 20:40:30,467][105692] Updated weights for policy 0, policy_version 729421 (0.0009) [2023-12-26 20:40:30,523][105692] Updated weights for policy 0, policy_version 729431 (0.0008) [2023-12-26 20:40:30,581][105692] Updated weights for policy 0, policy_version 729441 (0.0007) [2023-12-26 20:40:30,854][105620] Updated weights for policy 1, policy_version 729868 (0.0007) [2023-12-26 20:40:30,904][105620] Updated weights for policy 1, policy_version 729878 (0.0009) [2023-12-26 20:40:30,956][105620] Updated weights for policy 1, policy_version 729888 (0.0009) [2023-12-26 20:40:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 373645312. Throughput: 0: 9960.3, 1: 9451.1. Samples: 373612260. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 20:40:31,062][104569] Avg episode reward: [(0, '9350.968'), (1, '8397.403')] [2023-12-26 20:40:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000729448_186769408.pth... [2023-12-26 20:40:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000729896_186875904.pth... [2023-12-26 20:40:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000728296_186474496.pth [2023-12-26 20:40:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000728776_186589184.pth [2023-12-26 20:40:31,263][105692] Updated weights for policy 0, policy_version 729451 (0.0008) [2023-12-26 20:40:31,309][105692] Updated weights for policy 0, policy_version 729461 (0.0008) [2023-12-26 20:40:31,368][105692] Updated weights for policy 0, policy_version 729471 (0.0009) [2023-12-26 20:40:31,763][105620] Updated weights for policy 1, policy_version 729898 (0.0009) [2023-12-26 20:40:31,824][105620] Updated weights for policy 1, policy_version 729908 (0.0008) [2023-12-26 20:40:31,891][105620] Updated weights for policy 1, policy_version 729918 (0.0010) [2023-12-26 20:40:31,958][105620] Updated weights for policy 1, policy_version 729928 (0.0009) [2023-12-26 20:40:32,106][105692] Updated weights for policy 0, policy_version 729481 (0.0008) [2023-12-26 20:40:32,175][105692] Updated weights for policy 0, policy_version 729491 (0.0005) [2023-12-26 20:40:32,236][105692] Updated weights for policy 0, policy_version 729501 (0.0006) [2023-12-26 20:40:32,287][105692] Updated weights for policy 0, policy_version 729511 (0.0008) [2023-12-26 20:40:32,781][105620] Updated weights for policy 1, policy_version 729938 (0.0005) [2023-12-26 20:40:32,830][105620] Updated weights for policy 1, policy_version 729948 (0.0005) [2023-12-26 20:40:32,842][105692] Updated weights for policy 0, policy_version 729521 (0.0006) [2023-12-26 20:40:32,883][105620] Updated weights for policy 1, policy_version 729958 (0.0005) [2023-12-26 20:40:32,903][105692] Updated weights for policy 0, policy_version 729531 (0.0007) [2023-12-26 20:40:32,958][105692] Updated weights for policy 0, policy_version 729541 (0.0006) [2023-12-26 20:40:33,469][105620] Updated weights for policy 1, policy_version 729968 (0.0006) [2023-12-26 20:40:33,488][105692] Updated weights for policy 0, policy_version 729551 (0.0005) [2023-12-26 20:40:33,531][105620] Updated weights for policy 1, policy_version 729978 (0.0006) [2023-12-26 20:40:33,550][105692] Updated weights for policy 0, policy_version 729561 (0.0005) [2023-12-26 20:40:33,588][105620] Updated weights for policy 1, policy_version 729988 (0.0006) [2023-12-26 20:40:33,615][105692] Updated weights for policy 0, policy_version 729571 (0.0006) [2023-12-26 20:40:34,285][105692] Updated weights for policy 0, policy_version 729581 (0.0008) [2023-12-26 20:40:34,299][105620] Updated weights for policy 1, policy_version 729998 (0.0007) [2023-12-26 20:40:34,338][105692] Updated weights for policy 0, policy_version 729591 (0.0006) [2023-12-26 20:40:34,352][105620] Updated weights for policy 1, policy_version 730008 (0.0007) [2023-12-26 20:40:34,387][105692] Updated weights for policy 0, policy_version 729601 (0.0006) [2023-12-26 20:40:34,412][105620] Updated weights for policy 1, policy_version 730018 (0.0009) [2023-12-26 20:40:35,123][105692] Updated weights for policy 0, policy_version 729611 (0.0007) [2023-12-26 20:40:35,187][105692] Updated weights for policy 0, policy_version 729621 (0.0008) [2023-12-26 20:40:35,194][105620] Updated weights for policy 1, policy_version 730028 (0.0009) [2023-12-26 20:40:35,240][105692] Updated weights for policy 0, policy_version 729631 (0.0007) [2023-12-26 20:40:35,243][105620] Updated weights for policy 1, policy_version 730038 (0.0006) [2023-12-26 20:40:35,300][105620] Updated weights for policy 1, policy_version 730048 (0.0008) [2023-12-26 20:40:35,984][105692] Updated weights for policy 0, policy_version 729641 (0.0006) [2023-12-26 20:40:36,034][105692] Updated weights for policy 0, policy_version 729651 (0.0009) [2023-12-26 20:40:36,061][105620] Updated weights for policy 1, policy_version 730058 (0.0009) [2023-12-26 20:40:36,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.1, 300 sec: 19605.2). Total num frames: 373735424. Throughput: 0: 9998.1, 1: 9455.3. Samples: 373731544. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 20:40:36,063][104569] Avg episode reward: [(0, '9350.424'), (1, '9169.782')] [2023-12-26 20:40:36,083][105692] Updated weights for policy 0, policy_version 729661 (0.0008) [2023-12-26 20:40:36,108][105620] Updated weights for policy 1, policy_version 730068 (0.0007) [2023-12-26 20:40:36,150][105692] Updated weights for policy 0, policy_version 729671 (0.0006) [2023-12-26 20:40:36,172][105620] Updated weights for policy 1, policy_version 730078 (0.0007) [2023-12-26 20:40:36,234][105620] Updated weights for policy 1, policy_version 730088 (0.0009) [2023-12-26 20:40:36,918][105692] Updated weights for policy 0, policy_version 729681 (0.0009) [2023-12-26 20:40:36,985][105692] Updated weights for policy 0, policy_version 729691 (0.0008) [2023-12-26 20:40:37,003][105620] Updated weights for policy 1, policy_version 730098 (0.0007) [2023-12-26 20:40:37,047][105692] Updated weights for policy 0, policy_version 729701 (0.0007) [2023-12-26 20:40:37,058][105620] Updated weights for policy 1, policy_version 730108 (0.0007) [2023-12-26 20:40:37,117][105620] Updated weights for policy 1, policy_version 730118 (0.0008) [2023-12-26 20:40:37,123][105586] KL-divergence is very high: 109.4683 [2023-12-26 20:40:37,809][105620] Updated weights for policy 1, policy_version 730128 (0.0008) [2023-12-26 20:40:37,832][105692] Updated weights for policy 0, policy_version 729711 (0.0006) [2023-12-26 20:40:37,858][105620] Updated weights for policy 1, policy_version 730138 (0.0011) [2023-12-26 20:40:37,880][105692] Updated weights for policy 0, policy_version 729721 (0.0005) [2023-12-26 20:40:37,907][105620] Updated weights for policy 1, policy_version 730148 (0.0010) [2023-12-26 20:40:37,929][105692] Updated weights for policy 0, policy_version 729731 (0.0007) [2023-12-26 20:40:38,606][105620] Updated weights for policy 1, policy_version 730158 (0.0010) [2023-12-26 20:40:38,668][105620] Updated weights for policy 1, policy_version 730168 (0.0009) [2023-12-26 20:40:38,680][105692] Updated weights for policy 0, policy_version 729741 (0.0008) [2023-12-26 20:40:38,725][105620] Updated weights for policy 1, policy_version 730178 (0.0010) [2023-12-26 20:40:38,740][105692] Updated weights for policy 0, policy_version 729751 (0.0006) [2023-12-26 20:40:38,799][105692] Updated weights for policy 0, policy_version 729761 (0.0007) [2023-12-26 20:40:39,438][105620] Updated weights for policy 1, policy_version 730188 (0.0009) [2023-12-26 20:40:39,500][105620] Updated weights for policy 1, policy_version 730198 (0.0005) [2023-12-26 20:40:39,555][105620] Updated weights for policy 1, policy_version 730208 (0.0009) [2023-12-26 20:40:39,594][105692] Updated weights for policy 0, policy_version 729771 (0.0008) [2023-12-26 20:40:39,655][105692] Updated weights for policy 0, policy_version 729781 (0.0009) [2023-12-26 20:40:39,722][105692] Updated weights for policy 0, policy_version 729791 (0.0010) [2023-12-26 20:40:40,252][105620] Updated weights for policy 1, policy_version 730218 (0.0009) [2023-12-26 20:40:40,318][105620] Updated weights for policy 1, policy_version 730228 (0.0009) [2023-12-26 20:40:40,369][105620] Updated weights for policy 1, policy_version 730238 (0.0009) [2023-12-26 20:40:40,420][105620] Updated weights for policy 1, policy_version 730248 (0.0009) [2023-12-26 20:40:40,470][105692] Updated weights for policy 0, policy_version 729801 (0.0009) [2023-12-26 20:40:40,530][105692] Updated weights for policy 0, policy_version 729811 (0.0010) [2023-12-26 20:40:40,589][105692] Updated weights for policy 0, policy_version 729821 (0.0008) [2023-12-26 20:40:40,638][105692] Updated weights for policy 0, policy_version 729831 (0.0008) [2023-12-26 20:40:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19633.0). Total num frames: 373833728. Throughput: 0: 9834.7, 1: 9484.9. Samples: 373844164. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-12-26 20:40:41,062][104569] Avg episode reward: [(0, '9258.952'), (1, '8766.988')] [2023-12-26 20:40:41,220][105620] Updated weights for policy 1, policy_version 730258 (0.0010) [2023-12-26 20:40:41,283][105620] Updated weights for policy 1, policy_version 730268 (0.0011) [2023-12-26 20:40:41,352][105620] Updated weights for policy 1, policy_version 730278 (0.0010) [2023-12-26 20:40:41,363][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000009 [2023-12-26 20:40:41,435][105692] Updated weights for policy 0, policy_version 729841 (0.0009) [2023-12-26 20:40:41,495][105692] Updated weights for policy 0, policy_version 729851 (0.0011) [2023-12-26 20:40:41,551][105692] Updated weights for policy 0, policy_version 729861 (0.0011) [2023-12-26 20:40:42,049][105620] Updated weights for policy 1, policy_version 730288 (0.0011) [2023-12-26 20:40:42,115][105620] Updated weights for policy 1, policy_version 730298 (0.0009) [2023-12-26 20:40:42,180][105620] Updated weights for policy 1, policy_version 730308 (0.0011) [2023-12-26 20:40:42,326][105692] Updated weights for policy 0, policy_version 729871 (0.0011) [2023-12-26 20:40:42,388][105692] Updated weights for policy 0, policy_version 729881 (0.0011) [2023-12-26 20:40:42,443][105692] Updated weights for policy 0, policy_version 729891 (0.0010) [2023-12-26 20:40:42,916][105620] Updated weights for policy 1, policy_version 730318 (0.0011) [2023-12-26 20:40:42,976][105620] Updated weights for policy 1, policy_version 730328 (0.0010) [2023-12-26 20:40:43,038][105620] Updated weights for policy 1, policy_version 730338 (0.0010) [2023-12-26 20:40:43,214][105692] Updated weights for policy 0, policy_version 729901 (0.0011) [2023-12-26 20:40:43,279][105692] Updated weights for policy 0, policy_version 729911 (0.0011) [2023-12-26 20:40:43,345][105692] Updated weights for policy 0, policy_version 729921 (0.0010) [2023-12-26 20:40:43,770][105620] Updated weights for policy 1, policy_version 730348 (0.0009) [2023-12-26 20:40:43,825][105620] Updated weights for policy 1, policy_version 730358 (0.0011) [2023-12-26 20:40:43,878][105620] Updated weights for policy 1, policy_version 730368 (0.0010) [2023-12-26 20:40:44,089][105692] Updated weights for policy 0, policy_version 729931 (0.0010) [2023-12-26 20:40:44,141][105692] Updated weights for policy 0, policy_version 729941 (0.0011) [2023-12-26 20:40:44,197][105692] Updated weights for policy 0, policy_version 729951 (0.0010) [2023-12-26 20:40:44,601][105620] Updated weights for policy 1, policy_version 730378 (0.0010) [2023-12-26 20:40:44,657][105620] Updated weights for policy 1, policy_version 730388 (0.0010) [2023-12-26 20:40:44,709][105620] Updated weights for policy 1, policy_version 730398 (0.0010) [2023-12-26 20:40:44,765][105620] Updated weights for policy 1, policy_version 730408 (0.0010) [2023-12-26 20:40:44,804][105692] Updated weights for policy 0, policy_version 729961 (0.0010) [2023-12-26 20:40:44,855][105692] Updated weights for policy 0, policy_version 729971 (0.0006) [2023-12-26 20:40:44,923][105692] Updated weights for policy 0, policy_version 729981 (0.0007) [2023-12-26 20:40:44,988][105692] Updated weights for policy 0, policy_version 729991 (0.0009) [2023-12-26 20:40:45,463][105620] Updated weights for policy 1, policy_version 730418 (0.0011) [2023-12-26 20:40:45,522][105620] Updated weights for policy 1, policy_version 730428 (0.0011) [2023-12-26 20:40:45,571][105620] Updated weights for policy 1, policy_version 730438 (0.0010) [2023-12-26 20:40:45,760][105692] Updated weights for policy 0, policy_version 730001 (0.0010) [2023-12-26 20:40:45,815][105692] Updated weights for policy 0, policy_version 730011 (0.0010) [2023-12-26 20:40:45,873][105692] Updated weights for policy 0, policy_version 730021 (0.0010) [2023-12-26 20:40:46,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 373932032. Throughput: 0: 9755.1, 1: 9493.7. Samples: 373899372. Policy #0 lag: (min: 28.0, avg: 32.8, max: 60.0) [2023-12-26 20:40:46,063][104569] Avg episode reward: [(0, '9075.923'), (1, '8927.020')] [2023-12-26 20:40:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000730440_187015168.pth... [2023-12-26 20:40:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000730024_186916864.pth... [2023-12-26 20:40:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000729320_186728448.pth [2023-12-26 20:40:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000728872_186621952.pth [2023-12-26 20:40:46,278][105620] Updated weights for policy 1, policy_version 730448 (0.0011) [2023-12-26 20:40:46,337][105620] Updated weights for policy 1, policy_version 730458 (0.0011) [2023-12-26 20:40:46,395][105620] Updated weights for policy 1, policy_version 730468 (0.0010) [2023-12-26 20:40:46,512][105692] Updated weights for policy 0, policy_version 730031 (0.0010) [2023-12-26 20:40:46,574][105692] Updated weights for policy 0, policy_version 730041 (0.0010) [2023-12-26 20:40:46,622][105692] Updated weights for policy 0, policy_version 730051 (0.0010) [2023-12-26 20:40:46,970][105620] Updated weights for policy 1, policy_version 730478 (0.0010) [2023-12-26 20:40:47,021][105620] Updated weights for policy 1, policy_version 730488 (0.0009) [2023-12-26 20:40:47,075][105620] Updated weights for policy 1, policy_version 730498 (0.0010) [2023-12-26 20:40:47,303][105692] Updated weights for policy 0, policy_version 730061 (0.0008) [2023-12-26 20:40:47,362][105692] Updated weights for policy 0, policy_version 730071 (0.0007) [2023-12-26 20:40:47,420][105692] Updated weights for policy 0, policy_version 730081 (0.0010) [2023-12-26 20:40:47,731][105620] Updated weights for policy 1, policy_version 730508 (0.0008) [2023-12-26 20:40:47,787][105620] Updated weights for policy 1, policy_version 730518 (0.0005) [2023-12-26 20:40:47,847][105620] Updated weights for policy 1, policy_version 730528 (0.0007) [2023-12-26 20:40:48,135][105692] Updated weights for policy 0, policy_version 730091 (0.0009) [2023-12-26 20:40:48,197][105692] Updated weights for policy 0, policy_version 730101 (0.0008) [2023-12-26 20:40:48,254][105692] Updated weights for policy 0, policy_version 730111 (0.0011) [2023-12-26 20:40:48,501][105620] Updated weights for policy 1, policy_version 730538 (0.0010) [2023-12-26 20:40:48,561][105620] Updated weights for policy 1, policy_version 730548 (0.0008) [2023-12-26 20:40:48,618][105620] Updated weights for policy 1, policy_version 730558 (0.0008) [2023-12-26 20:40:48,670][105620] Updated weights for policy 1, policy_version 730568 (0.0008) [2023-12-26 20:40:49,000][105692] Updated weights for policy 0, policy_version 730121 (0.0011) [2023-12-26 20:40:49,048][105692] Updated weights for policy 0, policy_version 730131 (0.0009) [2023-12-26 20:40:49,110][105692] Updated weights for policy 0, policy_version 730141 (0.0006) [2023-12-26 20:40:49,164][105692] Updated weights for policy 0, policy_version 730151 (0.0009) [2023-12-26 20:40:49,371][105620] Updated weights for policy 1, policy_version 730578 (0.0009) [2023-12-26 20:40:49,434][105620] Updated weights for policy 1, policy_version 730588 (0.0009) [2023-12-26 20:40:49,498][105620] Updated weights for policy 1, policy_version 730598 (0.0008) [2023-12-26 20:40:49,898][105692] Updated weights for policy 0, policy_version 730161 (0.0010) [2023-12-26 20:40:49,959][105692] Updated weights for policy 0, policy_version 730171 (0.0010) [2023-12-26 20:40:50,015][105692] Updated weights for policy 0, policy_version 730181 (0.0009) [2023-12-26 20:40:50,201][105620] Updated weights for policy 1, policy_version 730608 (0.0009) [2023-12-26 20:40:50,263][105620] Updated weights for policy 1, policy_version 730618 (0.0009) [2023-12-26 20:40:50,318][105620] Updated weights for policy 1, policy_version 730628 (0.0009) [2023-12-26 20:40:50,796][105692] Updated weights for policy 0, policy_version 730191 (0.0008) [2023-12-26 20:40:50,861][105692] Updated weights for policy 0, policy_version 730201 (0.0009) [2023-12-26 20:40:50,920][105692] Updated weights for policy 0, policy_version 730211 (0.0009) [2023-12-26 20:40:51,047][105620] Updated weights for policy 1, policy_version 730638 (0.0010) [2023-12-26 20:40:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 374030336. Throughput: 0: 9728.2, 1: 9625.6. Samples: 374020948. Policy #0 lag: (min: 28.0, avg: 32.8, max: 60.0) [2023-12-26 20:40:51,062][104569] Avg episode reward: [(0, '9077.784'), (1, '9260.697')] [2023-12-26 20:40:51,103][105620] Updated weights for policy 1, policy_version 730648 (0.0010) [2023-12-26 20:40:51,166][105620] Updated weights for policy 1, policy_version 730658 (0.0006) [2023-12-26 20:40:51,714][105692] Updated weights for policy 0, policy_version 730221 (0.0009) [2023-12-26 20:40:51,786][105692] Updated weights for policy 0, policy_version 730231 (0.0008) [2023-12-26 20:40:51,838][105692] Updated weights for policy 0, policy_version 730241 (0.0008) [2023-12-26 20:40:51,886][105620] Updated weights for policy 1, policy_version 730668 (0.0008) [2023-12-26 20:40:51,945][105620] Updated weights for policy 1, policy_version 730678 (0.0010) [2023-12-26 20:40:51,998][105620] Updated weights for policy 1, policy_version 730688 (0.0011) [2023-12-26 20:40:52,589][105692] Updated weights for policy 0, policy_version 730251 (0.0009) [2023-12-26 20:40:52,647][105692] Updated weights for policy 0, policy_version 730261 (0.0010) [2023-12-26 20:40:52,667][105620] Updated weights for policy 1, policy_version 730698 (0.0009) [2023-12-26 20:40:52,715][105692] Updated weights for policy 0, policy_version 730271 (0.0007) [2023-12-26 20:40:52,724][105620] Updated weights for policy 1, policy_version 730708 (0.0005) [2023-12-26 20:40:52,783][105620] Updated weights for policy 1, policy_version 730718 (0.0006) [2023-12-26 20:40:52,847][105620] Updated weights for policy 1, policy_version 730728 (0.0006) [2023-12-26 20:40:53,359][105692] Updated weights for policy 0, policy_version 730281 (0.0010) [2023-12-26 20:40:53,419][105692] Updated weights for policy 0, policy_version 730291 (0.0008) [2023-12-26 20:40:53,481][105692] Updated weights for policy 0, policy_version 730301 (0.0010) [2023-12-26 20:40:53,529][105620] Updated weights for policy 1, policy_version 730738 (0.0011) [2023-12-26 20:40:53,541][105692] Updated weights for policy 0, policy_version 730311 (0.0011) [2023-12-26 20:40:53,587][105620] Updated weights for policy 1, policy_version 730748 (0.0010) [2023-12-26 20:40:53,640][105620] Updated weights for policy 1, policy_version 730758 (0.0010) [2023-12-26 20:40:54,266][105692] Updated weights for policy 0, policy_version 730321 (0.0006) [2023-12-26 20:40:54,335][105692] Updated weights for policy 0, policy_version 730331 (0.0005) [2023-12-26 20:40:54,400][105692] Updated weights for policy 0, policy_version 730341 (0.0005) [2023-12-26 20:40:54,413][105620] Updated weights for policy 1, policy_version 730768 (0.0010) [2023-12-26 20:40:54,476][105620] Updated weights for policy 1, policy_version 730778 (0.0006) [2023-12-26 20:40:54,545][105620] Updated weights for policy 1, policy_version 730788 (0.0006) [2023-12-26 20:40:55,061][105692] Updated weights for policy 0, policy_version 730351 (0.0009) [2023-12-26 20:40:55,078][105620] Updated weights for policy 1, policy_version 730798 (0.0009) [2023-12-26 20:40:55,119][105692] Updated weights for policy 0, policy_version 730361 (0.0010) [2023-12-26 20:40:55,133][105620] Updated weights for policy 1, policy_version 730808 (0.0011) [2023-12-26 20:40:55,175][105692] Updated weights for policy 0, policy_version 730371 (0.0008) [2023-12-26 20:40:55,192][105620] Updated weights for policy 1, policy_version 730818 (0.0010) [2023-12-26 20:40:55,895][105692] Updated weights for policy 0, policy_version 730381 (0.0010) [2023-12-26 20:40:55,924][105620] Updated weights for policy 1, policy_version 730828 (0.0010) [2023-12-26 20:40:55,946][105692] Updated weights for policy 0, policy_version 730391 (0.0010) [2023-12-26 20:40:55,975][105620] Updated weights for policy 1, policy_version 730838 (0.0010) [2023-12-26 20:40:55,994][105692] Updated weights for policy 0, policy_version 730401 (0.0010) [2023-12-26 20:40:56,023][105620] Updated weights for policy 1, policy_version 730848 (0.0010) [2023-12-26 20:40:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 374128640. Throughput: 0: 9744.7, 1: 9675.9. Samples: 374137532. Policy #0 lag: (min: 28.0, avg: 32.8, max: 60.0) [2023-12-26 20:40:56,063][104569] Avg episode reward: [(0, '9260.735'), (1, '9260.566')] [2023-12-26 20:40:56,650][105692] Updated weights for policy 0, policy_version 730411 (0.0009) [2023-12-26 20:40:56,703][105692] Updated weights for policy 0, policy_version 730421 (0.0007) [2023-12-26 20:40:56,753][105692] Updated weights for policy 0, policy_version 730431 (0.0010) [2023-12-26 20:40:56,783][105620] Updated weights for policy 1, policy_version 730858 (0.0010) [2023-12-26 20:40:56,837][105620] Updated weights for policy 1, policy_version 730868 (0.0010) [2023-12-26 20:40:56,887][105620] Updated weights for policy 1, policy_version 730878 (0.0010) [2023-12-26 20:40:56,944][105620] Updated weights for policy 1, policy_version 730888 (0.0010) [2023-12-26 20:40:57,507][105692] Updated weights for policy 0, policy_version 730441 (0.0010) [2023-12-26 20:40:57,557][105692] Updated weights for policy 0, policy_version 730451 (0.0009) [2023-12-26 20:40:57,603][105692] Updated weights for policy 0, policy_version 730461 (0.0009) [2023-12-26 20:40:57,648][105692] Updated weights for policy 0, policy_version 730471 (0.0007) [2023-12-26 20:40:57,694][105620] Updated weights for policy 1, policy_version 730898 (0.0009) [2023-12-26 20:40:57,755][105620] Updated weights for policy 1, policy_version 730908 (0.0009) [2023-12-26 20:40:57,809][105620] Updated weights for policy 1, policy_version 730918 (0.0010) [2023-12-26 20:40:58,257][105692] Updated weights for policy 0, policy_version 730481 (0.0010) [2023-12-26 20:40:58,317][105692] Updated weights for policy 0, policy_version 730491 (0.0010) [2023-12-26 20:40:58,390][105692] Updated weights for policy 0, policy_version 730501 (0.0010) [2023-12-26 20:40:58,719][105620] Updated weights for policy 1, policy_version 730928 (0.0009) [2023-12-26 20:40:58,781][105620] Updated weights for policy 1, policy_version 730938 (0.0009) [2023-12-26 20:40:58,843][105620] Updated weights for policy 1, policy_version 730948 (0.0009) [2023-12-26 20:40:59,247][105692] Updated weights for policy 0, policy_version 730511 (0.0008) [2023-12-26 20:40:59,302][105692] Updated weights for policy 0, policy_version 730521 (0.0006) [2023-12-26 20:40:59,369][105692] Updated weights for policy 0, policy_version 730531 (0.0007) [2023-12-26 20:40:59,566][105620] Updated weights for policy 1, policy_version 730958 (0.0007) [2023-12-26 20:40:59,629][105620] Updated weights for policy 1, policy_version 730968 (0.0008) [2023-12-26 20:40:59,679][105620] Updated weights for policy 1, policy_version 730978 (0.0009) [2023-12-26 20:41:00,019][105692] Updated weights for policy 0, policy_version 730541 (0.0007) [2023-12-26 20:41:00,074][105692] Updated weights for policy 0, policy_version 730551 (0.0008) [2023-12-26 20:41:00,122][105692] Updated weights for policy 0, policy_version 730561 (0.0007) [2023-12-26 20:41:00,435][105620] Updated weights for policy 1, policy_version 730988 (0.0007) [2023-12-26 20:41:00,487][105620] Updated weights for policy 1, policy_version 730998 (0.0005) [2023-12-26 20:41:00,539][105620] Updated weights for policy 1, policy_version 731008 (0.0005) [2023-12-26 20:41:00,786][105692] Updated weights for policy 0, policy_version 730571 (0.0009) [2023-12-26 20:41:00,843][105692] Updated weights for policy 0, policy_version 730581 (0.0005) [2023-12-26 20:41:00,902][105692] Updated weights for policy 0, policy_version 730591 (0.0006) [2023-12-26 20:41:01,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 374226944. Throughput: 0: 9783.7, 1: 9639.5. Samples: 374194292. Policy #0 lag: (min: 28.0, avg: 32.8, max: 60.0) [2023-12-26 20:41:01,062][104569] Avg episode reward: [(0, '9349.671'), (1, '9169.710')] [2023-12-26 20:41:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000730600_187064320.pth... [2023-12-26 20:41:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000731016_187162624.pth... [2023-12-26 20:41:01,069][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000729896_186875904.pth [2023-12-26 20:41:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000729448_186769408.pth [2023-12-26 20:41:01,316][105620] Updated weights for policy 1, policy_version 731018 (0.0008) [2023-12-26 20:41:01,381][105620] Updated weights for policy 1, policy_version 731028 (0.0009) [2023-12-26 20:41:01,446][105620] Updated weights for policy 1, policy_version 731038 (0.0009) [2023-12-26 20:41:01,493][105620] Updated weights for policy 1, policy_version 731048 (0.0008) [2023-12-26 20:41:01,514][105692] Updated weights for policy 0, policy_version 730601 (0.0009) [2023-12-26 20:41:01,567][105692] Updated weights for policy 0, policy_version 730611 (0.0009) [2023-12-26 20:41:01,632][105692] Updated weights for policy 0, policy_version 730621 (0.0009) [2023-12-26 20:41:01,691][105692] Updated weights for policy 0, policy_version 730631 (0.0010) [2023-12-26 20:41:02,201][105620] Updated weights for policy 1, policy_version 731058 (0.0006) [2023-12-26 20:41:02,249][105620] Updated weights for policy 1, policy_version 731068 (0.0008) [2023-12-26 20:41:02,305][105620] Updated weights for policy 1, policy_version 731078 (0.0008) [2023-12-26 20:41:02,417][105692] Updated weights for policy 0, policy_version 730641 (0.0010) [2023-12-26 20:41:02,469][105692] Updated weights for policy 0, policy_version 730651 (0.0010) [2023-12-26 20:41:02,520][105692] Updated weights for policy 0, policy_version 730661 (0.0010) [2023-12-26 20:41:03,056][105620] Updated weights for policy 1, policy_version 731088 (0.0009) [2023-12-26 20:41:03,117][105620] Updated weights for policy 1, policy_version 731098 (0.0009) [2023-12-26 20:41:03,181][105620] Updated weights for policy 1, policy_version 731108 (0.0009) [2023-12-26 20:41:03,213][105692] Updated weights for policy 0, policy_version 730671 (0.0007) [2023-12-26 20:41:03,264][105692] Updated weights for policy 0, policy_version 730681 (0.0005) [2023-12-26 20:41:03,318][105692] Updated weights for policy 0, policy_version 730691 (0.0005) [2023-12-26 20:41:03,970][105692] Updated weights for policy 0, policy_version 730701 (0.0006) [2023-12-26 20:41:03,987][105620] Updated weights for policy 1, policy_version 731118 (0.0007) [2023-12-26 20:41:04,025][105692] Updated weights for policy 0, policy_version 730711 (0.0008) [2023-12-26 20:41:04,045][105620] Updated weights for policy 1, policy_version 731128 (0.0007) [2023-12-26 20:41:04,089][105692] Updated weights for policy 0, policy_version 730721 (0.0009) [2023-12-26 20:41:04,108][105620] Updated weights for policy 1, policy_version 731138 (0.0007) [2023-12-26 20:41:04,819][105692] Updated weights for policy 0, policy_version 730731 (0.0010) [2023-12-26 20:41:04,869][105692] Updated weights for policy 0, policy_version 730741 (0.0008) [2023-12-26 20:41:04,884][105620] Updated weights for policy 1, policy_version 731148 (0.0008) [2023-12-26 20:41:04,929][105692] Updated weights for policy 0, policy_version 730751 (0.0007) [2023-12-26 20:41:04,949][105620] Updated weights for policy 1, policy_version 731158 (0.0007) [2023-12-26 20:41:04,999][105620] Updated weights for policy 1, policy_version 731168 (0.0009) [2023-12-26 20:41:05,603][105692] Updated weights for policy 0, policy_version 730761 (0.0006) [2023-12-26 20:41:05,661][105692] Updated weights for policy 0, policy_version 730771 (0.0009) [2023-12-26 20:41:05,707][105692] Updated weights for policy 0, policy_version 730781 (0.0009) [2023-12-26 20:41:05,761][105692] Updated weights for policy 0, policy_version 730791 (0.0009) [2023-12-26 20:41:05,775][105620] Updated weights for policy 1, policy_version 731178 (0.0009) [2023-12-26 20:41:05,833][105620] Updated weights for policy 1, policy_version 731188 (0.0009) [2023-12-26 20:41:05,891][105620] Updated weights for policy 1, policy_version 731198 (0.0009) [2023-12-26 20:41:05,944][105620] Updated weights for policy 1, policy_version 731208 (0.0009) [2023-12-26 20:41:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 374325248. Throughput: 0: 9803.9, 1: 9646.5. Samples: 374311000. Policy #0 lag: (min: 28.0, avg: 32.8, max: 60.0) [2023-12-26 20:41:06,063][104569] Avg episode reward: [(0, '9349.472'), (1, '9260.792')] [2023-12-26 20:41:06,536][105692] Updated weights for policy 0, policy_version 730801 (0.0009) [2023-12-26 20:41:06,599][105692] Updated weights for policy 0, policy_version 730811 (0.0009) [2023-12-26 20:41:06,654][105692] Updated weights for policy 0, policy_version 730821 (0.0009) [2023-12-26 20:41:06,721][105620] Updated weights for policy 1, policy_version 731218 (0.0009) [2023-12-26 20:41:06,771][105620] Updated weights for policy 1, policy_version 731228 (0.0009) [2023-12-26 20:41:06,826][105620] Updated weights for policy 1, policy_version 731238 (0.0009) [2023-12-26 20:41:07,424][105692] Updated weights for policy 0, policy_version 730831 (0.0009) [2023-12-26 20:41:07,489][105692] Updated weights for policy 0, policy_version 730841 (0.0009) [2023-12-26 20:41:07,546][105692] Updated weights for policy 0, policy_version 730851 (0.0008) [2023-12-26 20:41:07,561][105620] Updated weights for policy 1, policy_version 731248 (0.0007) [2023-12-26 20:41:07,625][105620] Updated weights for policy 1, policy_version 731258 (0.0005) [2023-12-26 20:41:07,676][105620] Updated weights for policy 1, policy_version 731268 (0.0005) [2023-12-26 20:41:08,309][105692] Updated weights for policy 0, policy_version 730861 (0.0008) [2023-12-26 20:41:08,373][105620] Updated weights for policy 1, policy_version 731278 (0.0006) [2023-12-26 20:41:08,376][105692] Updated weights for policy 0, policy_version 730871 (0.0007) [2023-12-26 20:41:08,437][105620] Updated weights for policy 1, policy_version 731288 (0.0006) [2023-12-26 20:41:08,439][105692] Updated weights for policy 0, policy_version 730881 (0.0008) [2023-12-26 20:41:08,492][105620] Updated weights for policy 1, policy_version 731298 (0.0008) [2023-12-26 20:41:09,209][105692] Updated weights for policy 0, policy_version 730891 (0.0007) [2023-12-26 20:41:09,226][105620] Updated weights for policy 1, policy_version 731308 (0.0008) [2023-12-26 20:41:09,275][105692] Updated weights for policy 0, policy_version 730901 (0.0008) [2023-12-26 20:41:09,285][105620] Updated weights for policy 1, policy_version 731318 (0.0007) [2023-12-26 20:41:09,335][105692] Updated weights for policy 0, policy_version 730911 (0.0007) [2023-12-26 20:41:09,354][105620] Updated weights for policy 1, policy_version 731328 (0.0008) [2023-12-26 20:41:10,104][105620] Updated weights for policy 1, policy_version 731338 (0.0009) [2023-12-26 20:41:10,134][105692] Updated weights for policy 0, policy_version 730921 (0.0009) [2023-12-26 20:41:10,164][105620] Updated weights for policy 1, policy_version 731348 (0.0011) [2023-12-26 20:41:10,197][105692] Updated weights for policy 0, policy_version 730931 (0.0010) [2023-12-26 20:41:10,228][105620] Updated weights for policy 1, policy_version 731358 (0.0011) [2023-12-26 20:41:10,259][105692] Updated weights for policy 0, policy_version 730941 (0.0010) [2023-12-26 20:41:10,287][105620] Updated weights for policy 1, policy_version 731368 (0.0011) [2023-12-26 20:41:10,320][105692] Updated weights for policy 0, policy_version 730951 (0.0008) [2023-12-26 20:41:10,890][105692] Updated weights for policy 0, policy_version 730961 (0.0007) [2023-12-26 20:41:10,950][105692] Updated weights for policy 0, policy_version 730971 (0.0011) [2023-12-26 20:41:10,968][105620] Updated weights for policy 1, policy_version 731378 (0.0006) [2023-12-26 20:41:11,004][105692] Updated weights for policy 0, policy_version 730981 (0.0010) [2023-12-26 20:41:11,034][105620] Updated weights for policy 1, policy_version 731388 (0.0008) [2023-12-26 20:41:11,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 374415360. Throughput: 0: 9729.5, 1: 9621.9. Samples: 374424088. Policy #0 lag: (min: 28.0, avg: 32.8, max: 60.0) [2023-12-26 20:41:11,062][104569] Avg episode reward: [(0, '9349.429'), (1, '9262.227')] [2023-12-26 20:41:11,101][105620] Updated weights for policy 1, policy_version 731398 (0.0010) [2023-12-26 20:41:11,747][105692] Updated weights for policy 0, policy_version 730991 (0.0009) [2023-12-26 20:41:11,803][105692] Updated weights for policy 0, policy_version 731001 (0.0009) [2023-12-26 20:41:11,862][105692] Updated weights for policy 0, policy_version 731011 (0.0008) [2023-12-26 20:41:11,879][105620] Updated weights for policy 1, policy_version 731408 (0.0008) [2023-12-26 20:41:11,928][105620] Updated weights for policy 1, policy_version 731418 (0.0008) [2023-12-26 20:41:11,984][105620] Updated weights for policy 1, policy_version 731428 (0.0009) [2023-12-26 20:41:12,598][105692] Updated weights for policy 0, policy_version 731021 (0.0007) [2023-12-26 20:41:12,645][105692] Updated weights for policy 0, policy_version 731031 (0.0009) [2023-12-26 20:41:12,694][105692] Updated weights for policy 0, policy_version 731041 (0.0009) [2023-12-26 20:41:12,786][105620] Updated weights for policy 1, policy_version 731438 (0.0009) [2023-12-26 20:41:12,841][105620] Updated weights for policy 1, policy_version 731448 (0.0009) [2023-12-26 20:41:12,895][105620] Updated weights for policy 1, policy_version 731458 (0.0009) [2023-12-26 20:41:13,497][105692] Updated weights for policy 0, policy_version 731051 (0.0006) [2023-12-26 20:41:13,545][105692] Updated weights for policy 0, policy_version 731061 (0.0005) [2023-12-26 20:41:13,612][105692] Updated weights for policy 0, policy_version 731071 (0.0008) [2023-12-26 20:41:13,653][105620] Updated weights for policy 1, policy_version 731468 (0.0007) [2023-12-26 20:41:13,706][105620] Updated weights for policy 1, policy_version 731478 (0.0009) [2023-12-26 20:41:13,765][105620] Updated weights for policy 1, policy_version 731488 (0.0010) [2023-12-26 20:41:14,294][105692] Updated weights for policy 0, policy_version 731081 (0.0008) [2023-12-26 20:41:14,346][105620] Updated weights for policy 1, policy_version 731498 (0.0006) [2023-12-26 20:41:14,350][105692] Updated weights for policy 0, policy_version 731091 (0.0009) [2023-12-26 20:41:14,397][105620] Updated weights for policy 1, policy_version 731508 (0.0007) [2023-12-26 20:41:14,406][105692] Updated weights for policy 0, policy_version 731101 (0.0009) [2023-12-26 20:41:14,451][105620] Updated weights for policy 1, policy_version 731518 (0.0009) [2023-12-26 20:41:14,467][105692] Updated weights for policy 0, policy_version 731111 (0.0011) [2023-12-26 20:41:14,507][105620] Updated weights for policy 1, policy_version 731528 (0.0007) [2023-12-26 20:41:15,207][105692] Updated weights for policy 0, policy_version 731121 (0.0007) [2023-12-26 20:41:15,261][105692] Updated weights for policy 0, policy_version 731131 (0.0010) [2023-12-26 20:41:15,319][105692] Updated weights for policy 0, policy_version 731141 (0.0007) [2023-12-26 20:41:15,343][105620] Updated weights for policy 1, policy_version 731538 (0.0008) [2023-12-26 20:41:15,398][105620] Updated weights for policy 1, policy_version 731548 (0.0009) [2023-12-26 20:41:15,454][105620] Updated weights for policy 1, policy_version 731558 (0.0009) [2023-12-26 20:41:15,973][105692] Updated weights for policy 0, policy_version 731151 (0.0005) [2023-12-26 20:41:16,026][105692] Updated weights for policy 0, policy_version 731161 (0.0005) [2023-12-26 20:41:16,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 374505472. Throughput: 0: 9682.0, 1: 9616.9. Samples: 374480712. Policy #0 lag: (min: 28.0, avg: 32.8, max: 60.0) [2023-12-26 20:41:16,062][104569] Avg episode reward: [(0, '9349.283'), (1, '9264.062')] [2023-12-26 20:41:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000731560_187301888.pth... [2023-12-26 20:41:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000730440_187015168.pth [2023-12-26 20:41:16,080][105692] Updated weights for policy 0, policy_version 731171 (0.0005) [2023-12-26 20:41:16,104][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000731176_187211776.pth... [2023-12-26 20:41:16,107][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000730024_186916864.pth [2023-12-26 20:41:16,176][105620] Updated weights for policy 1, policy_version 731568 (0.0010) [2023-12-26 20:41:16,234][105620] Updated weights for policy 1, policy_version 731578 (0.0009) [2023-12-26 20:41:16,295][105620] Updated weights for policy 1, policy_version 731588 (0.0008) [2023-12-26 20:41:16,639][105692] Updated weights for policy 0, policy_version 731181 (0.0008) [2023-12-26 20:41:16,688][105692] Updated weights for policy 0, policy_version 731191 (0.0010) [2023-12-26 20:41:16,733][105692] Updated weights for policy 0, policy_version 731201 (0.0010) [2023-12-26 20:41:17,098][105620] Updated weights for policy 1, policy_version 731598 (0.0008) [2023-12-26 20:41:17,144][105620] Updated weights for policy 1, policy_version 731608 (0.0005) [2023-12-26 20:41:17,196][105620] Updated weights for policy 1, policy_version 731618 (0.0008) [2023-12-26 20:41:17,412][105692] Updated weights for policy 0, policy_version 731211 (0.0009) [2023-12-26 20:41:17,464][105692] Updated weights for policy 0, policy_version 731221 (0.0006) [2023-12-26 20:41:17,510][105692] Updated weights for policy 0, policy_version 731231 (0.0005) [2023-12-26 20:41:17,754][105620] Updated weights for policy 1, policy_version 731628 (0.0005) [2023-12-26 20:41:17,799][105620] Updated weights for policy 1, policy_version 731638 (0.0005) [2023-12-26 20:41:17,856][105620] Updated weights for policy 1, policy_version 731648 (0.0008) [2023-12-26 20:41:18,182][105692] Updated weights for policy 0, policy_version 731241 (0.0006) [2023-12-26 20:41:18,236][105692] Updated weights for policy 0, policy_version 731251 (0.0009) [2023-12-26 20:41:18,302][105692] Updated weights for policy 0, policy_version 731261 (0.0007) [2023-12-26 20:41:18,371][105692] Updated weights for policy 0, policy_version 731271 (0.0006) [2023-12-26 20:41:18,640][105620] Updated weights for policy 1, policy_version 731658 (0.0009) [2023-12-26 20:41:18,693][105620] Updated weights for policy 1, policy_version 731668 (0.0009) [2023-12-26 20:41:18,763][105620] Updated weights for policy 1, policy_version 731678 (0.0009) [2023-12-26 20:41:18,832][105620] Updated weights for policy 1, policy_version 731688 (0.0009) [2023-12-26 20:41:18,970][105692] Updated weights for policy 0, policy_version 731281 (0.0007) [2023-12-26 20:41:19,037][105692] Updated weights for policy 0, policy_version 731291 (0.0008) [2023-12-26 20:41:19,105][105692] Updated weights for policy 0, policy_version 731301 (0.0008) [2023-12-26 20:41:19,636][105620] Updated weights for policy 1, policy_version 731698 (0.0007) [2023-12-26 20:41:19,691][105620] Updated weights for policy 1, policy_version 731708 (0.0009) [2023-12-26 20:41:19,741][105620] Updated weights for policy 1, policy_version 731718 (0.0007) [2023-12-26 20:41:19,742][105692] Updated weights for policy 0, policy_version 731311 (0.0008) [2023-12-26 20:41:19,810][105692] Updated weights for policy 0, policy_version 731321 (0.0008) [2023-12-26 20:41:19,871][105692] Updated weights for policy 0, policy_version 731331 (0.0009) [2023-12-26 20:41:20,509][105620] Updated weights for policy 1, policy_version 731728 (0.0006) [2023-12-26 20:41:20,554][105692] Updated weights for policy 0, policy_version 731341 (0.0007) [2023-12-26 20:41:20,571][105620] Updated weights for policy 1, policy_version 731738 (0.0006) [2023-12-26 20:41:20,622][105692] Updated weights for policy 0, policy_version 731351 (0.0008) [2023-12-26 20:41:20,639][105620] Updated weights for policy 1, policy_version 731748 (0.0006) [2023-12-26 20:41:20,686][105692] Updated weights for policy 0, policy_version 731361 (0.0008) [2023-12-26 20:41:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 374611968. Throughput: 0: 9709.6, 1: 9605.6. Samples: 374600724. Policy #0 lag: (min: 28.0, avg: 32.8, max: 60.0) [2023-12-26 20:41:21,063][104569] Avg episode reward: [(0, '9099.888'), (1, '9264.852')] [2023-12-26 20:41:21,324][105620] Updated weights for policy 1, policy_version 731758 (0.0006) [2023-12-26 20:41:21,394][105620] Updated weights for policy 1, policy_version 731768 (0.0008) [2023-12-26 20:41:21,455][105620] Updated weights for policy 1, policy_version 731778 (0.0011) [2023-12-26 20:41:21,503][105692] Updated weights for policy 0, policy_version 731371 (0.0008) [2023-12-26 20:41:21,567][105692] Updated weights for policy 0, policy_version 731381 (0.0008) [2023-12-26 20:41:21,636][105692] Updated weights for policy 0, policy_version 731391 (0.0009) [2023-12-26 20:41:22,147][105620] Updated weights for policy 1, policy_version 731788 (0.0010) [2023-12-26 20:41:22,211][105620] Updated weights for policy 1, policy_version 731798 (0.0008) [2023-12-26 20:41:22,278][105620] Updated weights for policy 1, policy_version 731808 (0.0009) [2023-12-26 20:41:22,420][105692] Updated weights for policy 0, policy_version 731401 (0.0007) [2023-12-26 20:41:22,477][105692] Updated weights for policy 0, policy_version 731411 (0.0008) [2023-12-26 20:41:22,537][105692] Updated weights for policy 0, policy_version 731421 (0.0009) [2023-12-26 20:41:22,596][105692] Updated weights for policy 0, policy_version 731431 (0.0009) [2023-12-26 20:41:22,930][105620] Updated weights for policy 1, policy_version 731818 (0.0008) [2023-12-26 20:41:22,991][105620] Updated weights for policy 1, policy_version 731828 (0.0006) [2023-12-26 20:41:23,051][105620] Updated weights for policy 1, policy_version 731838 (0.0007) [2023-12-26 20:41:23,115][105620] Updated weights for policy 1, policy_version 731848 (0.0008) [2023-12-26 20:41:23,422][105692] Updated weights for policy 0, policy_version 731441 (0.0009) [2023-12-26 20:41:23,475][105692] Updated weights for policy 0, policy_version 731451 (0.0009) [2023-12-26 20:41:23,523][105692] Updated weights for policy 0, policy_version 731461 (0.0009) [2023-12-26 20:41:23,730][105620] Updated weights for policy 1, policy_version 731858 (0.0005) [2023-12-26 20:41:23,787][105620] Updated weights for policy 1, policy_version 731868 (0.0005) [2023-12-26 20:41:23,843][105620] Updated weights for policy 1, policy_version 731878 (0.0005) [2023-12-26 20:41:24,401][105620] Updated weights for policy 1, policy_version 731888 (0.0008) [2023-12-26 20:41:24,413][105692] Updated weights for policy 0, policy_version 731471 (0.0007) [2023-12-26 20:41:24,464][105620] Updated weights for policy 1, policy_version 731898 (0.0008) [2023-12-26 20:41:24,475][105692] Updated weights for policy 0, policy_version 731481 (0.0006) [2023-12-26 20:41:24,528][105620] Updated weights for policy 1, policy_version 731908 (0.0009) [2023-12-26 20:41:24,536][105692] Updated weights for policy 0, policy_version 731491 (0.0006) [2023-12-26 20:41:25,183][105692] Updated weights for policy 0, policy_version 731501 (0.0008) [2023-12-26 20:41:25,239][105692] Updated weights for policy 0, policy_version 731511 (0.0009) [2023-12-26 20:41:25,269][105620] Updated weights for policy 1, policy_version 731918 (0.0008) [2023-12-26 20:41:25,293][105692] Updated weights for policy 0, policy_version 731521 (0.0006) [2023-12-26 20:41:25,333][105620] Updated weights for policy 1, policy_version 731928 (0.0008) [2023-12-26 20:41:25,399][105620] Updated weights for policy 1, policy_version 731938 (0.0010) [2023-12-26 20:41:25,865][105692] Updated weights for policy 0, policy_version 731531 (0.0006) [2023-12-26 20:41:25,915][105692] Updated weights for policy 0, policy_version 731541 (0.0007) [2023-12-26 20:41:25,965][105692] Updated weights for policy 0, policy_version 731551 (0.0008) [2023-12-26 20:41:26,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 374710272. Throughput: 0: 9716.9, 1: 9652.0. Samples: 374715764. Policy #0 lag: (min: 28.0, avg: 32.8, max: 60.0) [2023-12-26 20:41:26,062][104569] Avg episode reward: [(0, '9215.946'), (1, '9355.693')] [2023-12-26 20:41:26,270][105620] Updated weights for policy 1, policy_version 731948 (0.0009) [2023-12-26 20:41:26,333][105620] Updated weights for policy 1, policy_version 731958 (0.0010) [2023-12-26 20:41:26,385][105620] Updated weights for policy 1, policy_version 731968 (0.0009) [2023-12-26 20:41:26,545][105692] Updated weights for policy 0, policy_version 731561 (0.0006) [2023-12-26 20:41:26,596][105692] Updated weights for policy 0, policy_version 731571 (0.0010) [2023-12-26 20:41:26,647][105692] Updated weights for policy 0, policy_version 731581 (0.0010) [2023-12-26 20:41:26,705][105692] Updated weights for policy 0, policy_version 731591 (0.0010) [2023-12-26 20:41:27,165][105620] Updated weights for policy 1, policy_version 731978 (0.0007) [2023-12-26 20:41:27,215][105620] Updated weights for policy 1, policy_version 731988 (0.0007) [2023-12-26 20:41:27,259][105620] Updated weights for policy 1, policy_version 731998 (0.0008) [2023-12-26 20:41:27,305][105620] Updated weights for policy 1, policy_version 732008 (0.0008) [2023-12-26 20:41:27,351][105692] Updated weights for policy 0, policy_version 731601 (0.0010) [2023-12-26 20:41:27,395][105692] Updated weights for policy 0, policy_version 731611 (0.0010) [2023-12-26 20:41:27,445][105692] Updated weights for policy 0, policy_version 731621 (0.0010) [2023-12-26 20:41:27,939][105620] Updated weights for policy 1, policy_version 732018 (0.0008) [2023-12-26 20:41:27,989][105620] Updated weights for policy 1, policy_version 732028 (0.0008) [2023-12-26 20:41:28,040][105620] Updated weights for policy 1, policy_version 732038 (0.0007) [2023-12-26 20:41:28,064][105692] Updated weights for policy 0, policy_version 731631 (0.0010) [2023-12-26 20:41:28,107][105692] Updated weights for policy 0, policy_version 731641 (0.0005) [2023-12-26 20:41:28,155][105692] Updated weights for policy 0, policy_version 731651 (0.0005) [2023-12-26 20:41:28,741][105692] Updated weights for policy 0, policy_version 731661 (0.0008) [2023-12-26 20:41:28,804][105692] Updated weights for policy 0, policy_version 731671 (0.0011) [2023-12-26 20:41:28,859][105692] Updated weights for policy 0, policy_version 731681 (0.0011) [2023-12-26 20:41:28,882][105620] Updated weights for policy 1, policy_version 732048 (0.0006) [2023-12-26 20:41:28,932][105620] Updated weights for policy 1, policy_version 732058 (0.0008) [2023-12-26 20:41:28,988][105620] Updated weights for policy 1, policy_version 732069 (0.0009) [2023-12-26 20:41:29,563][105692] Updated weights for policy 0, policy_version 731691 (0.0010) [2023-12-26 20:41:29,619][105692] Updated weights for policy 0, policy_version 731701 (0.0010) [2023-12-26 20:41:29,638][105620] Updated weights for policy 1, policy_version 732079 (0.0005) [2023-12-26 20:41:29,673][105692] Updated weights for policy 0, policy_version 731711 (0.0011) [2023-12-26 20:41:29,690][105620] Updated weights for policy 1, policy_version 732089 (0.0005) [2023-12-26 20:41:29,738][105620] Updated weights for policy 1, policy_version 732099 (0.0005) [2023-12-26 20:41:30,358][105692] Updated weights for policy 0, policy_version 731721 (0.0010) [2023-12-26 20:41:30,380][105620] Updated weights for policy 1, policy_version 732109 (0.0005) [2023-12-26 20:41:30,427][105692] Updated weights for policy 0, policy_version 731731 (0.0006) [2023-12-26 20:41:30,440][105620] Updated weights for policy 1, policy_version 732119 (0.0006) [2023-12-26 20:41:30,489][105692] Updated weights for policy 0, policy_version 731741 (0.0009) [2023-12-26 20:41:30,502][105620] Updated weights for policy 1, policy_version 732129 (0.0007) [2023-12-26 20:41:30,552][105692] Updated weights for policy 0, policy_version 731751 (0.0011) [2023-12-26 20:41:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 374808576. Throughput: 0: 9889.5, 1: 9654.0. Samples: 374778828. Policy #0 lag: (min: 28.0, avg: 32.8, max: 60.0) [2023-12-26 20:41:31,063][104569] Avg episode reward: [(0, '9348.815'), (1, '9354.628')] [2023-12-26 20:41:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000732136_187449344.pth... [2023-12-26 20:41:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000731752_187359232.pth... [2023-12-26 20:41:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000731016_187162624.pth [2023-12-26 20:41:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000730600_187064320.pth [2023-12-26 20:41:31,147][105692] Updated weights for policy 0, policy_version 731761 (0.0009) [2023-12-26 20:41:31,157][105620] Updated weights for policy 1, policy_version 732139 (0.0006) [2023-12-26 20:41:31,206][105692] Updated weights for policy 0, policy_version 731771 (0.0010) [2023-12-26 20:41:31,216][105620] Updated weights for policy 1, policy_version 732149 (0.0008) [2023-12-26 20:41:31,258][105692] Updated weights for policy 0, policy_version 731781 (0.0010) [2023-12-26 20:41:31,273][105620] Updated weights for policy 1, policy_version 732159 (0.0011) [2023-12-26 20:41:31,955][105620] Updated weights for policy 1, policy_version 732169 (0.0006) [2023-12-26 20:41:32,013][105620] Updated weights for policy 1, policy_version 732179 (0.0006) [2023-12-26 20:41:32,025][105692] Updated weights for policy 0, policy_version 731791 (0.0011) [2023-12-26 20:41:32,067][105620] Updated weights for policy 1, policy_version 732189 (0.0009) [2023-12-26 20:41:32,073][105692] Updated weights for policy 0, policy_version 731801 (0.0010) [2023-12-26 20:41:32,120][105620] Updated weights for policy 1, policy_version 732199 (0.0010) [2023-12-26 20:41:32,130][105692] Updated weights for policy 0, policy_version 731811 (0.0008) [2023-12-26 20:41:32,787][105692] Updated weights for policy 0, policy_version 731821 (0.0010) [2023-12-26 20:41:32,831][105620] Updated weights for policy 1, policy_version 732209 (0.0006) [2023-12-26 20:41:32,848][105692] Updated weights for policy 0, policy_version 731831 (0.0008) [2023-12-26 20:41:32,888][105620] Updated weights for policy 1, policy_version 732219 (0.0010) [2023-12-26 20:41:32,900][105692] Updated weights for policy 0, policy_version 731841 (0.0005) [2023-12-26 20:41:32,946][105620] Updated weights for policy 1, policy_version 732229 (0.0010) [2023-12-26 20:41:33,616][105692] Updated weights for policy 0, policy_version 731851 (0.0006) [2023-12-26 20:41:33,665][105620] Updated weights for policy 1, policy_version 732239 (0.0010) [2023-12-26 20:41:33,674][105692] Updated weights for policy 0, policy_version 731861 (0.0006) [2023-12-26 20:41:33,719][105620] Updated weights for policy 1, policy_version 732249 (0.0010) [2023-12-26 20:41:33,733][105692] Updated weights for policy 0, policy_version 731871 (0.0007) [2023-12-26 20:41:33,774][105620] Updated weights for policy 1, policy_version 732259 (0.0010) [2023-12-26 20:41:34,444][105620] Updated weights for policy 1, policy_version 732269 (0.0008) [2023-12-26 20:41:34,499][105692] Updated weights for policy 0, policy_version 731881 (0.0006) [2023-12-26 20:41:34,514][105620] Updated weights for policy 1, policy_version 732279 (0.0006) [2023-12-26 20:41:34,560][105692] Updated weights for policy 0, policy_version 731891 (0.0009) [2023-12-26 20:41:34,582][105620] Updated weights for policy 1, policy_version 732289 (0.0009) [2023-12-26 20:41:34,620][105692] Updated weights for policy 0, policy_version 731901 (0.0008) [2023-12-26 20:41:34,682][105692] Updated weights for policy 0, policy_version 731911 (0.0009) [2023-12-26 20:41:35,289][105620] Updated weights for policy 1, policy_version 732299 (0.0010) [2023-12-26 20:41:35,354][105620] Updated weights for policy 1, policy_version 732309 (0.0010) [2023-12-26 20:41:35,381][105692] Updated weights for policy 0, policy_version 731921 (0.0010) [2023-12-26 20:41:35,412][105620] Updated weights for policy 1, policy_version 732319 (0.0010) [2023-12-26 20:41:35,436][105692] Updated weights for policy 0, policy_version 731931 (0.0010) [2023-12-26 20:41:35,490][105692] Updated weights for policy 0, policy_version 731941 (0.0010) [2023-12-26 20:41:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 374906880. Throughput: 0: 9898.8, 1: 9641.1. Samples: 374900248. Policy #0 lag: (min: 28.0, avg: 32.8, max: 60.0) [2023-12-26 20:41:36,063][104569] Avg episode reward: [(0, '9349.316'), (1, '9353.993')] [2023-12-26 20:41:36,126][105692] Updated weights for policy 0, policy_version 731951 (0.0011) [2023-12-26 20:41:36,127][105620] Updated weights for policy 1, policy_version 732329 (0.0010) [2023-12-26 20:41:36,190][105692] Updated weights for policy 0, policy_version 731961 (0.0011) [2023-12-26 20:41:36,191][105620] Updated weights for policy 1, policy_version 732339 (0.0011) [2023-12-26 20:41:36,250][105692] Updated weights for policy 0, policy_version 731971 (0.0011) [2023-12-26 20:41:36,251][105620] Updated weights for policy 1, policy_version 732349 (0.0011) [2023-12-26 20:41:36,303][105620] Updated weights for policy 1, policy_version 732359 (0.0011) [2023-12-26 20:41:36,999][105692] Updated weights for policy 0, policy_version 731981 (0.0011) [2023-12-26 20:41:37,040][105620] Updated weights for policy 1, policy_version 732369 (0.0006) [2023-12-26 20:41:37,061][105692] Updated weights for policy 0, policy_version 731991 (0.0011) [2023-12-26 20:41:37,105][105620] Updated weights for policy 1, policy_version 732379 (0.0010) [2023-12-26 20:41:37,131][105692] Updated weights for policy 0, policy_version 732001 (0.0011) [2023-12-26 20:41:37,169][105620] Updated weights for policy 1, policy_version 732389 (0.0010) [2023-12-26 20:41:37,730][105620] Updated weights for policy 1, policy_version 732399 (0.0006) [2023-12-26 20:41:37,780][105620] Updated weights for policy 1, policy_version 732409 (0.0005) [2023-12-26 20:41:37,830][105620] Updated weights for policy 1, policy_version 732419 (0.0005) [2023-12-26 20:41:37,874][105692] Updated weights for policy 0, policy_version 732011 (0.0011) [2023-12-26 20:41:37,934][105692] Updated weights for policy 0, policy_version 732021 (0.0007) [2023-12-26 20:41:37,984][105692] Updated weights for policy 0, policy_version 732031 (0.0008) [2023-12-26 20:41:38,426][105620] Updated weights for policy 1, policy_version 732429 (0.0007) [2023-12-26 20:41:38,488][105620] Updated weights for policy 1, policy_version 732439 (0.0010) [2023-12-26 20:41:38,547][105620] Updated weights for policy 1, policy_version 732449 (0.0010) [2023-12-26 20:41:38,715][105692] Updated weights for policy 0, policy_version 732041 (0.0009) [2023-12-26 20:41:38,776][105692] Updated weights for policy 0, policy_version 732051 (0.0005) [2023-12-26 20:41:38,832][105692] Updated weights for policy 0, policy_version 732061 (0.0005) [2023-12-26 20:41:38,890][105692] Updated weights for policy 0, policy_version 732071 (0.0005) [2023-12-26 20:41:39,256][105620] Updated weights for policy 1, policy_version 732459 (0.0010) [2023-12-26 20:41:39,318][105620] Updated weights for policy 1, policy_version 732469 (0.0008) [2023-12-26 20:41:39,390][105620] Updated weights for policy 1, policy_version 732479 (0.0008) [2023-12-26 20:41:39,540][105692] Updated weights for policy 0, policy_version 732081 (0.0008) [2023-12-26 20:41:39,602][105692] Updated weights for policy 0, policy_version 732091 (0.0006) [2023-12-26 20:41:39,658][105692] Updated weights for policy 0, policy_version 732101 (0.0008) [2023-12-26 20:41:40,172][105620] Updated weights for policy 1, policy_version 732489 (0.0008) [2023-12-26 20:41:40,240][105620] Updated weights for policy 1, policy_version 732499 (0.0006) [2023-12-26 20:41:40,309][105620] Updated weights for policy 1, policy_version 732509 (0.0006) [2023-12-26 20:41:40,374][105620] Updated weights for policy 1, policy_version 732519 (0.0008) [2023-12-26 20:41:40,409][105692] Updated weights for policy 0, policy_version 732111 (0.0006) [2023-12-26 20:41:40,462][105692] Updated weights for policy 0, policy_version 732121 (0.0008) [2023-12-26 20:41:40,517][105692] Updated weights for policy 0, policy_version 732131 (0.0008) [2023-12-26 20:41:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 375005184. Throughput: 0: 9926.6, 1: 9639.1. Samples: 375017988. Policy #0 lag: (min: 28.0, avg: 32.8, max: 60.0) [2023-12-26 20:41:41,062][104569] Avg episode reward: [(0, '9349.836'), (1, '9354.818')] [2023-12-26 20:41:41,123][105620] Updated weights for policy 1, policy_version 732529 (0.0010) [2023-12-26 20:41:41,188][105620] Updated weights for policy 1, policy_version 732539 (0.0010) [2023-12-26 20:41:41,254][105620] Updated weights for policy 1, policy_version 732549 (0.0010) [2023-12-26 20:41:41,314][105692] Updated weights for policy 0, policy_version 732141 (0.0009) [2023-12-26 20:41:41,377][105692] Updated weights for policy 0, policy_version 732151 (0.0008) [2023-12-26 20:41:41,445][105692] Updated weights for policy 0, policy_version 732161 (0.0007) [2023-12-26 20:41:41,990][105620] Updated weights for policy 1, policy_version 732559 (0.0010) [2023-12-26 20:41:42,057][105620] Updated weights for policy 1, policy_version 732569 (0.0011) [2023-12-26 20:41:42,128][105620] Updated weights for policy 1, policy_version 732579 (0.0009) [2023-12-26 20:41:42,191][105692] Updated weights for policy 0, policy_version 732171 (0.0008) [2023-12-26 20:41:42,251][105692] Updated weights for policy 0, policy_version 732181 (0.0011) [2023-12-26 20:41:42,322][105692] Updated weights for policy 0, policy_version 732191 (0.0009) [2023-12-26 20:41:42,816][105620] Updated weights for policy 1, policy_version 732589 (0.0006) [2023-12-26 20:41:42,884][105620] Updated weights for policy 1, policy_version 732599 (0.0008) [2023-12-26 20:41:42,924][105692] Updated weights for policy 0, policy_version 732201 (0.0009) [2023-12-26 20:41:42,946][105620] Updated weights for policy 1, policy_version 732609 (0.0009) [2023-12-26 20:41:42,972][105692] Updated weights for policy 0, policy_version 732211 (0.0010) [2023-12-26 20:41:43,022][105692] Updated weights for policy 0, policy_version 732221 (0.0010) [2023-12-26 20:41:43,083][105692] Updated weights for policy 0, policy_version 732231 (0.0010) [2023-12-26 20:41:43,671][105620] Updated weights for policy 1, policy_version 732619 (0.0008) [2023-12-26 20:41:43,719][105620] Updated weights for policy 1, policy_version 732629 (0.0010) [2023-12-26 20:41:43,769][105692] Updated weights for policy 0, policy_version 732241 (0.0006) [2023-12-26 20:41:43,777][105620] Updated weights for policy 1, policy_version 732639 (0.0010) [2023-12-26 20:41:43,826][105692] Updated weights for policy 0, policy_version 732251 (0.0008) [2023-12-26 20:41:43,874][105692] Updated weights for policy 0, policy_version 732261 (0.0010) [2023-12-26 20:41:44,526][105620] Updated weights for policy 1, policy_version 732649 (0.0010) [2023-12-26 20:41:44,584][105620] Updated weights for policy 1, policy_version 732659 (0.0011) [2023-12-26 20:41:44,593][105692] Updated weights for policy 0, policy_version 732271 (0.0010) [2023-12-26 20:41:44,648][105620] Updated weights for policy 1, policy_version 732669 (0.0009) [2023-12-26 20:41:44,652][105692] Updated weights for policy 0, policy_version 732281 (0.0010) [2023-12-26 20:41:44,707][105692] Updated weights for policy 0, policy_version 732291 (0.0010) [2023-12-26 20:41:44,710][105620] Updated weights for policy 1, policy_version 732679 (0.0005) [2023-12-26 20:41:45,343][105620] Updated weights for policy 1, policy_version 732689 (0.0005) [2023-12-26 20:41:45,399][105692] Updated weights for policy 0, policy_version 732301 (0.0008) [2023-12-26 20:41:45,417][105620] Updated weights for policy 1, policy_version 732699 (0.0008) [2023-12-26 20:41:45,460][105692] Updated weights for policy 0, policy_version 732311 (0.0006) [2023-12-26 20:41:45,484][105620] Updated weights for policy 1, policy_version 732709 (0.0010) [2023-12-26 20:41:45,527][105692] Updated weights for policy 0, policy_version 732321 (0.0006) [2023-12-26 20:41:46,045][105692] Updated weights for policy 0, policy_version 732331 (0.0006) [2023-12-26 20:41:46,055][105620] Updated weights for policy 1, policy_version 732719 (0.0007) [2023-12-26 20:41:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 375103488. Throughput: 0: 9909.5, 1: 9674.8. Samples: 375075584. Policy #0 lag: (min: 28.0, avg: 32.8, max: 60.0) [2023-12-26 20:41:46,062][104569] Avg episode reward: [(0, '9128.695'), (1, '9173.492')] [2023-12-26 20:41:46,112][105692] Updated weights for policy 0, policy_version 732341 (0.0007) [2023-12-26 20:41:46,127][105620] Updated weights for policy 1, policy_version 732729 (0.0005) [2023-12-26 20:41:46,171][105692] Updated weights for policy 0, policy_version 732351 (0.0006) [2023-12-26 20:41:46,199][105620] Updated weights for policy 1, policy_version 732739 (0.0005) [2023-12-26 20:41:46,216][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000732360_187514880.pth... [2023-12-26 20:41:46,220][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000731176_187211776.pth [2023-12-26 20:41:46,220][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_000732360_187514880.pth [2023-12-26 20:41:46,229][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000732744_187604992.pth... [2023-12-26 20:41:46,232][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000731560_187301888.pth [2023-12-26 20:41:46,232][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_000732744_187604992.pth [2023-12-26 20:41:46,690][105620] Updated weights for policy 1, policy_version 732749 (0.0005) [2023-12-26 20:41:46,737][105620] Updated weights for policy 1, policy_version 732759 (0.0005) [2023-12-26 20:41:46,783][105620] Updated weights for policy 1, policy_version 732769 (0.0005) [2023-12-26 20:41:46,850][105692] Updated weights for policy 0, policy_version 732361 (0.0006) [2023-12-26 20:41:46,898][105692] Updated weights for policy 0, policy_version 732371 (0.0011) [2023-12-26 20:41:46,950][105692] Updated weights for policy 0, policy_version 732381 (0.0010) [2023-12-26 20:41:47,019][105692] Updated weights for policy 0, policy_version 732391 (0.0010) [2023-12-26 20:41:47,425][105620] Updated weights for policy 1, policy_version 732779 (0.0007) [2023-12-26 20:41:47,476][105620] Updated weights for policy 1, policy_version 732789 (0.0010) [2023-12-26 20:41:47,530][105620] Updated weights for policy 1, policy_version 732799 (0.0010) [2023-12-26 20:41:47,756][105692] Updated weights for policy 0, policy_version 732401 (0.0006) [2023-12-26 20:41:47,809][105692] Updated weights for policy 0, policy_version 732411 (0.0007) [2023-12-26 20:41:47,861][105692] Updated weights for policy 0, policy_version 732421 (0.0008) [2023-12-26 20:41:48,186][105620] Updated weights for policy 1, policy_version 732809 (0.0008) [2023-12-26 20:41:48,248][105620] Updated weights for policy 1, policy_version 732819 (0.0005) [2023-12-26 20:41:48,316][105620] Updated weights for policy 1, policy_version 732829 (0.0006) [2023-12-26 20:41:48,382][105620] Updated weights for policy 1, policy_version 732839 (0.0008) [2023-12-26 20:41:48,600][105692] Updated weights for policy 0, policy_version 732431 (0.0010) [2023-12-26 20:41:48,659][105692] Updated weights for policy 0, policy_version 732441 (0.0010) [2023-12-26 20:41:48,721][105692] Updated weights for policy 0, policy_version 732451 (0.0010) [2023-12-26 20:41:48,987][105620] Updated weights for policy 1, policy_version 732849 (0.0010) [2023-12-26 20:41:49,043][105620] Updated weights for policy 1, policy_version 732859 (0.0010) [2023-12-26 20:41:49,094][105620] Updated weights for policy 1, policy_version 732869 (0.0010) [2023-12-26 20:41:49,460][105692] Updated weights for policy 0, policy_version 732461 (0.0010) [2023-12-26 20:41:49,518][105692] Updated weights for policy 0, policy_version 732471 (0.0010) [2023-12-26 20:41:49,582][105692] Updated weights for policy 0, policy_version 732481 (0.0010) [2023-12-26 20:41:49,851][105620] Updated weights for policy 1, policy_version 732879 (0.0010) [2023-12-26 20:41:49,921][105620] Updated weights for policy 1, policy_version 732889 (0.0008) [2023-12-26 20:41:49,989][105620] Updated weights for policy 1, policy_version 732899 (0.0009) [2023-12-26 20:41:50,342][105692] Updated weights for policy 0, policy_version 732491 (0.0011) [2023-12-26 20:41:50,397][105692] Updated weights for policy 0, policy_version 732501 (0.0011) [2023-12-26 20:41:50,462][105692] Updated weights for policy 0, policy_version 732511 (0.0011) [2023-12-26 20:41:50,665][105620] Updated weights for policy 1, policy_version 732909 (0.0011) [2023-12-26 20:41:50,725][105620] Updated weights for policy 1, policy_version 732919 (0.0011) [2023-12-26 20:41:50,791][105620] Updated weights for policy 1, policy_version 732929 (0.0011) [2023-12-26 20:41:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 375209984. Throughput: 0: 9910.0, 1: 9849.9. Samples: 375200192. Policy #0 lag: (min: 28.0, avg: 32.8, max: 60.0) [2023-12-26 20:41:51,062][104569] Avg episode reward: [(0, '9128.419'), (1, '8832.955')] [2023-12-26 20:41:51,171][105692] Updated weights for policy 0, policy_version 732521 (0.0010) [2023-12-26 20:41:51,232][105692] Updated weights for policy 0, policy_version 732531 (0.0009) [2023-12-26 20:41:51,296][105692] Updated weights for policy 0, policy_version 732541 (0.0010) [2023-12-26 20:41:51,356][105692] Updated weights for policy 0, policy_version 732551 (0.0011) [2023-12-26 20:41:51,523][105620] Updated weights for policy 1, policy_version 732939 (0.0010) [2023-12-26 20:41:51,584][105620] Updated weights for policy 1, policy_version 732949 (0.0009) [2023-12-26 20:41:51,644][105620] Updated weights for policy 1, policy_version 732959 (0.0007) [2023-12-26 20:41:52,109][105692] Updated weights for policy 0, policy_version 732561 (0.0008) [2023-12-26 20:41:52,168][105692] Updated weights for policy 0, policy_version 732571 (0.0006) [2023-12-26 20:41:52,227][105692] Updated weights for policy 0, policy_version 732581 (0.0007) [2023-12-26 20:41:52,469][105620] Updated weights for policy 1, policy_version 732969 (0.0009) [2023-12-26 20:41:52,524][105620] Updated weights for policy 1, policy_version 732979 (0.0009) [2023-12-26 20:41:52,613][105620] Updated weights for policy 1, policy_version 732989 (0.0009) [2023-12-26 20:41:52,663][105620] Updated weights for policy 1, policy_version 732999 (0.0009) [2023-12-26 20:41:52,879][105692] Updated weights for policy 0, policy_version 732591 (0.0005) [2023-12-26 20:41:52,939][105692] Updated weights for policy 0, policy_version 732601 (0.0008) [2023-12-26 20:41:52,992][105692] Updated weights for policy 0, policy_version 732612 (0.0007) [2023-12-26 20:41:53,348][105620] Updated weights for policy 1, policy_version 733009 (0.0010) [2023-12-26 20:41:53,397][105620] Updated weights for policy 1, policy_version 733019 (0.0010) [2023-12-26 20:41:53,446][105620] Updated weights for policy 1, policy_version 733029 (0.0010) [2023-12-26 20:41:53,618][105692] Updated weights for policy 0, policy_version 732622 (0.0006) [2023-12-26 20:41:53,670][105692] Updated weights for policy 0, policy_version 732632 (0.0007) [2023-12-26 20:41:53,718][105692] Updated weights for policy 0, policy_version 732642 (0.0010) [2023-12-26 20:41:54,030][105620] Updated weights for policy 1, policy_version 733039 (0.0007) [2023-12-26 20:41:54,083][105620] Updated weights for policy 1, policy_version 733049 (0.0010) [2023-12-26 20:41:54,147][105620] Updated weights for policy 1, policy_version 733059 (0.0010) [2023-12-26 20:41:54,311][105692] Updated weights for policy 0, policy_version 732652 (0.0010) [2023-12-26 20:41:54,373][105692] Updated weights for policy 0, policy_version 732662 (0.0010) [2023-12-26 20:41:54,433][105692] Updated weights for policy 0, policy_version 732672 (0.0011) [2023-12-26 20:41:54,878][105620] Updated weights for policy 1, policy_version 733069 (0.0008) [2023-12-26 20:41:54,931][105620] Updated weights for policy 1, policy_version 733080 (0.0010) [2023-12-26 20:41:54,994][105620] Updated weights for policy 1, policy_version 733090 (0.0009) [2023-12-26 20:41:55,052][105692] Updated weights for policy 0, policy_version 732682 (0.0009) [2023-12-26 20:41:55,118][105692] Updated weights for policy 0, policy_version 732692 (0.0006) [2023-12-26 20:41:55,177][105692] Updated weights for policy 0, policy_version 732702 (0.0006) [2023-12-26 20:41:55,238][105692] Updated weights for policy 0, policy_version 732712 (0.0005) [2023-12-26 20:41:55,755][105692] Updated weights for policy 0, policy_version 732722 (0.0005) [2023-12-26 20:41:55,810][105692] Updated weights for policy 0, policy_version 732732 (0.0008) [2023-12-26 20:41:55,869][105692] Updated weights for policy 0, policy_version 732742 (0.0010) [2023-12-26 20:41:55,906][105620] Updated weights for policy 1, policy_version 733100 (0.0010) [2023-12-26 20:41:55,958][105620] Updated weights for policy 1, policy_version 733110 (0.0008) [2023-12-26 20:41:56,013][105620] Updated weights for policy 1, policy_version 733120 (0.0008) [2023-12-26 20:41:56,062][104569] Fps is (10 sec: 21299.5, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 375316480. Throughput: 0: 10056.5, 1: 9848.3. Samples: 375319800. Policy #0 lag: (min: 28.0, avg: 32.8, max: 60.0) [2023-12-26 20:41:56,062][104569] Avg episode reward: [(0, '8991.939'), (1, '8932.981')] [2023-12-26 20:41:56,495][105692] Updated weights for policy 0, policy_version 732752 (0.0007) [2023-12-26 20:41:56,544][105692] Updated weights for policy 0, policy_version 732762 (0.0005) [2023-12-26 20:41:56,598][105692] Updated weights for policy 0, policy_version 732772 (0.0006) [2023-12-26 20:41:56,647][105620] Updated weights for policy 1, policy_version 733130 (0.0007) [2023-12-26 20:41:56,712][105620] Updated weights for policy 1, policy_version 733140 (0.0008) [2023-12-26 20:41:56,775][105620] Updated weights for policy 1, policy_version 733150 (0.0006) [2023-12-26 20:41:56,835][105620] Updated weights for policy 1, policy_version 733160 (0.0005) [2023-12-26 20:41:57,263][105692] Updated weights for policy 0, policy_version 732782 (0.0007) [2023-12-26 20:41:57,327][105692] Updated weights for policy 0, policy_version 732792 (0.0006) [2023-12-26 20:41:57,375][105692] Updated weights for policy 0, policy_version 732802 (0.0010) [2023-12-26 20:41:57,521][105620] Updated weights for policy 1, policy_version 733170 (0.0005) [2023-12-26 20:41:57,576][105620] Updated weights for policy 1, policy_version 733180 (0.0005) [2023-12-26 20:41:57,635][105620] Updated weights for policy 1, policy_version 733190 (0.0005) [2023-12-26 20:41:58,025][105692] Updated weights for policy 0, policy_version 732812 (0.0008) [2023-12-26 20:41:58,078][105692] Updated weights for policy 0, policy_version 732822 (0.0005) [2023-12-26 20:41:58,140][105620] Updated weights for policy 1, policy_version 733201 (0.0006) [2023-12-26 20:41:58,141][105692] Updated weights for policy 0, policy_version 732832 (0.0010) [2023-12-26 20:41:58,209][105620] Updated weights for policy 1, policy_version 733211 (0.0007) [2023-12-26 20:41:58,282][105620] Updated weights for policy 1, policy_version 733221 (0.0011) [2023-12-26 20:41:58,835][105692] Updated weights for policy 0, policy_version 732842 (0.0009) [2023-12-26 20:41:58,909][105692] Updated weights for policy 0, policy_version 732852 (0.0008) [2023-12-26 20:41:58,969][105692] Updated weights for policy 0, policy_version 732862 (0.0011) [2023-12-26 20:41:59,027][105620] Updated weights for policy 1, policy_version 733231 (0.0009) [2023-12-26 20:41:59,036][105692] Updated weights for policy 0, policy_version 732872 (0.0011) [2023-12-26 20:41:59,087][105620] Updated weights for policy 1, policy_version 733241 (0.0008) [2023-12-26 20:41:59,159][105620] Updated weights for policy 1, policy_version 733251 (0.0007) [2023-12-26 20:41:59,757][105692] Updated weights for policy 0, policy_version 732882 (0.0005) [2023-12-26 20:41:59,815][105692] Updated weights for policy 0, policy_version 732892 (0.0010) [2023-12-26 20:41:59,874][105692] Updated weights for policy 0, policy_version 732902 (0.0010) [2023-12-26 20:42:00,031][105620] Updated weights for policy 1, policy_version 733261 (0.0008) [2023-12-26 20:42:00,086][105620] Updated weights for policy 1, policy_version 733271 (0.0008) [2023-12-26 20:42:00,137][105620] Updated weights for policy 1, policy_version 733281 (0.0008) [2023-12-26 20:42:00,522][105692] Updated weights for policy 0, policy_version 732912 (0.0006) [2023-12-26 20:42:00,579][105585] KL-divergence is very high: 119.3984 [2023-12-26 20:42:00,583][105692] Updated weights for policy 0, policy_version 732922 (0.0005) [2023-12-26 20:42:00,628][105585] KL-divergence is very high: 153.3641 [2023-12-26 20:42:00,646][105692] Updated weights for policy 0, policy_version 732932 (0.0010) [2023-12-26 20:42:00,872][105620] Updated weights for policy 1, policy_version 733291 (0.0007) [2023-12-26 20:42:00,931][105620] Updated weights for policy 1, policy_version 733301 (0.0005) [2023-12-26 20:42:00,977][105620] Updated weights for policy 1, policy_version 733311 (0.0009) [2023-12-26 20:42:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 375414784. Throughput: 0: 10107.9, 1: 9937.4. Samples: 375382752. Policy #0 lag: (min: 28.0, avg: 32.8, max: 60.0) [2023-12-26 20:42:01,063][104569] Avg episode reward: [(0, '8913.389'), (1, '9182.450')] [2023-12-26 20:42:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000732936_187662336.pth... [2023-12-26 20:42:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000733320_187752448.pth... [2023-12-26 20:42:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000731752_187359232.pth [2023-12-26 20:42:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000732136_187449344.pth [2023-12-26 20:42:01,277][105692] Updated weights for policy 0, policy_version 732942 (0.0009) [2023-12-26 20:42:01,334][105692] Updated weights for policy 0, policy_version 732952 (0.0008) [2023-12-26 20:42:01,397][105692] Updated weights for policy 0, policy_version 732962 (0.0008) [2023-12-26 20:42:01,679][105620] Updated weights for policy 1, policy_version 733321 (0.0010) [2023-12-26 20:42:01,737][105620] Updated weights for policy 1, policy_version 733331 (0.0009) [2023-12-26 20:42:01,787][105620] Updated weights for policy 1, policy_version 733341 (0.0008) [2023-12-26 20:42:01,841][105620] Updated weights for policy 1, policy_version 733351 (0.0009) [2023-12-26 20:42:02,120][105692] Updated weights for policy 0, policy_version 732972 (0.0008) [2023-12-26 20:42:02,172][105692] Updated weights for policy 0, policy_version 732982 (0.0009) [2023-12-26 20:42:02,219][105692] Updated weights for policy 0, policy_version 732992 (0.0005) [2023-12-26 20:42:02,591][105620] Updated weights for policy 1, policy_version 733361 (0.0008) [2023-12-26 20:42:02,654][105620] Updated weights for policy 1, policy_version 733371 (0.0009) [2023-12-26 20:42:02,708][105620] Updated weights for policy 1, policy_version 733381 (0.0007) [2023-12-26 20:42:02,927][105692] Updated weights for policy 0, policy_version 733002 (0.0006) [2023-12-26 20:42:02,982][105692] Updated weights for policy 0, policy_version 733012 (0.0005) [2023-12-26 20:42:03,046][105692] Updated weights for policy 0, policy_version 733022 (0.0006) [2023-12-26 20:42:03,101][105692] Updated weights for policy 0, policy_version 733032 (0.0008) [2023-12-26 20:42:03,412][105620] Updated weights for policy 1, policy_version 733391 (0.0006) [2023-12-26 20:42:03,455][105620] Updated weights for policy 1, policy_version 733401 (0.0005) [2023-12-26 20:42:03,508][105620] Updated weights for policy 1, policy_version 733411 (0.0006) [2023-12-26 20:42:03,690][105692] Updated weights for policy 0, policy_version 733042 (0.0005) [2023-12-26 20:42:03,734][105692] Updated weights for policy 0, policy_version 733052 (0.0005) [2023-12-26 20:42:03,780][105692] Updated weights for policy 0, policy_version 733062 (0.0005) [2023-12-26 20:42:04,140][105620] Updated weights for policy 1, policy_version 733421 (0.0008) [2023-12-26 20:42:04,203][105620] Updated weights for policy 1, policy_version 733431 (0.0009) [2023-12-26 20:42:04,266][105620] Updated weights for policy 1, policy_version 733441 (0.0008) [2023-12-26 20:42:04,446][105692] Updated weights for policy 0, policy_version 733072 (0.0006) [2023-12-26 20:42:04,509][105692] Updated weights for policy 0, policy_version 733082 (0.0006) [2023-12-26 20:42:04,569][105692] Updated weights for policy 0, policy_version 733092 (0.0007) [2023-12-26 20:42:04,861][105620] Updated weights for policy 1, policy_version 733451 (0.0009) [2023-12-26 20:42:04,916][105620] Updated weights for policy 1, policy_version 733461 (0.0005) [2023-12-26 20:42:04,967][105620] Updated weights for policy 1, policy_version 733471 (0.0005) [2023-12-26 20:42:05,300][105692] Updated weights for policy 0, policy_version 733102 (0.0009) [2023-12-26 20:42:05,353][105692] Updated weights for policy 0, policy_version 733112 (0.0005) [2023-12-26 20:42:05,414][105692] Updated weights for policy 0, policy_version 733122 (0.0005) [2023-12-26 20:42:05,690][105620] Updated weights for policy 1, policy_version 733481 (0.0008) [2023-12-26 20:42:05,745][105620] Updated weights for policy 1, policy_version 733491 (0.0006) [2023-12-26 20:42:05,807][105620] Updated weights for policy 1, policy_version 733501 (0.0005) [2023-12-26 20:42:05,872][105620] Updated weights for policy 1, policy_version 733511 (0.0005) [2023-12-26 20:42:06,038][105692] Updated weights for policy 0, policy_version 733132 (0.0005) [2023-12-26 20:42:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 375513088. Throughput: 0: 10097.8, 1: 9977.3. Samples: 375504100. Policy #0 lag: (min: 28.0, avg: 32.8, max: 60.0) [2023-12-26 20:42:06,063][104569] Avg episode reward: [(0, '4444.454'), (1, '9082.390')] [2023-12-26 20:42:06,111][105692] Updated weights for policy 0, policy_version 733142 (0.0006) [2023-12-26 20:42:06,175][105692] Updated weights for policy 0, policy_version 733152 (0.0007) [2023-12-26 20:42:06,479][105620] Updated weights for policy 1, policy_version 733521 (0.0006) [2023-12-26 20:42:06,540][105620] Updated weights for policy 1, policy_version 733531 (0.0006) [2023-12-26 20:42:06,602][105620] Updated weights for policy 1, policy_version 733541 (0.0007) [2023-12-26 20:42:06,851][105692] Updated weights for policy 0, policy_version 733162 (0.0011) [2023-12-26 20:42:06,905][105692] Updated weights for policy 0, policy_version 733172 (0.0006) [2023-12-26 20:42:06,954][105692] Updated weights for policy 0, policy_version 733182 (0.0005) [2023-12-26 20:42:07,018][105692] Updated weights for policy 0, policy_version 733192 (0.0005) [2023-12-26 20:42:07,157][105620] Updated weights for policy 1, policy_version 733551 (0.0009) [2023-12-26 20:42:07,224][105620] Updated weights for policy 1, policy_version 733561 (0.0008) [2023-12-26 20:42:07,286][105620] Updated weights for policy 1, policy_version 733571 (0.0008) [2023-12-26 20:42:07,590][105692] Updated weights for policy 0, policy_version 733202 (0.0010) [2023-12-26 20:42:07,650][105692] Updated weights for policy 0, policy_version 733212 (0.0006) [2023-12-26 20:42:07,701][105692] Updated weights for policy 0, policy_version 733222 (0.0005) [2023-12-26 20:42:08,036][105620] Updated weights for policy 1, policy_version 733581 (0.0008) [2023-12-26 20:42:08,092][105620] Updated weights for policy 1, policy_version 733591 (0.0008) [2023-12-26 20:42:08,151][105620] Updated weights for policy 1, policy_version 733601 (0.0010) [2023-12-26 20:42:08,237][105692] Updated weights for policy 0, policy_version 733232 (0.0005) [2023-12-26 20:42:08,305][105692] Updated weights for policy 0, policy_version 733242 (0.0006) [2023-12-26 20:42:08,379][105692] Updated weights for policy 0, policy_version 733252 (0.0008) [2023-12-26 20:42:08,926][105620] Updated weights for policy 1, policy_version 733611 (0.0009) [2023-12-26 20:42:08,983][105620] Updated weights for policy 1, policy_version 733621 (0.0010) [2023-12-26 20:42:09,034][105692] Updated weights for policy 0, policy_version 733262 (0.0007) [2023-12-26 20:42:09,038][105620] Updated weights for policy 1, policy_version 733631 (0.0009) [2023-12-26 20:42:09,100][105692] Updated weights for policy 0, policy_version 733272 (0.0005) [2023-12-26 20:42:09,146][105692] Updated weights for policy 0, policy_version 733282 (0.0005) [2023-12-26 20:42:09,776][105620] Updated weights for policy 1, policy_version 733641 (0.0009) [2023-12-26 20:42:09,842][105620] Updated weights for policy 1, policy_version 733651 (0.0009) [2023-12-26 20:42:09,891][105692] Updated weights for policy 0, policy_version 733292 (0.0005) [2023-12-26 20:42:09,909][105620] Updated weights for policy 1, policy_version 733661 (0.0009) [2023-12-26 20:42:09,959][105692] Updated weights for policy 0, policy_version 733302 (0.0008) [2023-12-26 20:42:09,969][105620] Updated weights for policy 1, policy_version 733671 (0.0007) [2023-12-26 20:42:10,021][105692] Updated weights for policy 0, policy_version 733312 (0.0008) [2023-12-26 20:42:10,716][105620] Updated weights for policy 1, policy_version 733681 (0.0009) [2023-12-26 20:42:10,764][105692] Updated weights for policy 0, policy_version 733322 (0.0008) [2023-12-26 20:42:10,770][105620] Updated weights for policy 1, policy_version 733691 (0.0008) [2023-12-26 20:42:10,818][105692] Updated weights for policy 0, policy_version 733332 (0.0006) [2023-12-26 20:42:10,820][105620] Updated weights for policy 1, policy_version 733701 (0.0008) [2023-12-26 20:42:10,876][105692] Updated weights for policy 0, policy_version 733342 (0.0006) [2023-12-26 20:42:10,931][105692] Updated weights for policy 0, policy_version 733352 (0.0006) [2023-12-26 20:42:11,062][104569] Fps is (10 sec: 20480.2, 60 sec: 20070.4, 300 sec: 19688.6). Total num frames: 375619584. Throughput: 0: 10243.3, 1: 9942.3. Samples: 375624116. Policy #0 lag: (min: 28.0, avg: 32.8, max: 60.0) [2023-12-26 20:42:11,062][104569] Avg episode reward: [(0, '5160.884'), (1, '9173.461')] [2023-12-26 20:42:11,652][105620] Updated weights for policy 1, policy_version 733711 (0.0007) [2023-12-26 20:42:11,678][105692] Updated weights for policy 0, policy_version 733362 (0.0006) [2023-12-26 20:42:11,722][105620] Updated weights for policy 1, policy_version 733721 (0.0008) [2023-12-26 20:42:11,745][105692] Updated weights for policy 0, policy_version 733372 (0.0010) [2023-12-26 20:42:11,777][105620] Updated weights for policy 1, policy_version 733731 (0.0007) [2023-12-26 20:42:11,804][105692] Updated weights for policy 0, policy_version 733382 (0.0006) [2023-12-26 20:42:12,460][105620] Updated weights for policy 1, policy_version 733741 (0.0009) [2023-12-26 20:42:12,520][105620] Updated weights for policy 1, policy_version 733751 (0.0010) [2023-12-26 20:42:12,572][105620] Updated weights for policy 1, policy_version 733761 (0.0010) [2023-12-26 20:42:12,644][105692] Updated weights for policy 0, policy_version 733392 (0.0007) [2023-12-26 20:42:12,704][105692] Updated weights for policy 0, policy_version 733402 (0.0008) [2023-12-26 20:42:12,759][105692] Updated weights for policy 0, policy_version 733414 (0.0011) [2023-12-26 20:42:13,179][105620] Updated weights for policy 1, policy_version 733771 (0.0009) [2023-12-26 20:42:13,250][105620] Updated weights for policy 1, policy_version 733781 (0.0005) [2023-12-26 20:42:13,311][105620] Updated weights for policy 1, policy_version 733791 (0.0005) [2023-12-26 20:42:13,443][105692] Updated weights for policy 0, policy_version 733424 (0.0010) [2023-12-26 20:42:13,497][105692] Updated weights for policy 0, policy_version 733434 (0.0006) [2023-12-26 20:42:13,546][105692] Updated weights for policy 0, policy_version 733444 (0.0005) [2023-12-26 20:42:13,896][105620] Updated weights for policy 1, policy_version 733801 (0.0005) [2023-12-26 20:42:13,950][105620] Updated weights for policy 1, policy_version 733811 (0.0005) [2023-12-26 20:42:13,996][105620] Updated weights for policy 1, policy_version 733821 (0.0005) [2023-12-26 20:42:14,055][105620] Updated weights for policy 1, policy_version 733831 (0.0008) [2023-12-26 20:42:14,283][105692] Updated weights for policy 0, policy_version 733454 (0.0008) [2023-12-26 20:42:14,351][105692] Updated weights for policy 0, policy_version 733464 (0.0010) [2023-12-26 20:42:14,417][105692] Updated weights for policy 0, policy_version 733474 (0.0010) [2023-12-26 20:42:14,622][105620] Updated weights for policy 1, policy_version 733841 (0.0010) [2023-12-26 20:42:14,671][105620] Updated weights for policy 1, policy_version 733851 (0.0010) [2023-12-26 20:42:14,715][105620] Updated weights for policy 1, policy_version 733861 (0.0010) [2023-12-26 20:42:15,150][105692] Updated weights for policy 0, policy_version 733484 (0.0008) [2023-12-26 20:42:15,222][105692] Updated weights for policy 0, policy_version 733494 (0.0010) [2023-12-26 20:42:15,287][105692] Updated weights for policy 0, policy_version 733504 (0.0010) [2023-12-26 20:42:15,424][105620] Updated weights for policy 1, policy_version 733871 (0.0011) [2023-12-26 20:42:15,485][105620] Updated weights for policy 1, policy_version 733881 (0.0011) [2023-12-26 20:42:15,551][105620] Updated weights for policy 1, policy_version 733891 (0.0007) [2023-12-26 20:42:15,969][105692] Updated weights for policy 0, policy_version 733514 (0.0007) [2023-12-26 20:42:16,020][105692] Updated weights for policy 0, policy_version 733524 (0.0010) [2023-12-26 20:42:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 20070.4, 300 sec: 19633.0). Total num frames: 375709696. Throughput: 0: 10088.1, 1: 10016.3. Samples: 375683528. Policy #0 lag: (min: 28.0, avg: 32.8, max: 60.0) [2023-12-26 20:42:16,063][104569] Avg episode reward: [(0, '7247.756'), (1, '9355.532')] [2023-12-26 20:42:16,065][105692] Updated weights for policy 0, policy_version 733534 (0.0008) [2023-12-26 20:42:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000733896_187899904.pth... [2023-12-26 20:42:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000732744_187604992.pth [2023-12-26 20:42:16,120][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000733544_187817984.pth... [2023-12-26 20:42:16,122][105692] Updated weights for policy 0, policy_version 733544 (0.0005) [2023-12-26 20:42:16,123][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000732360_187514880.pth [2023-12-26 20:42:16,163][105620] Updated weights for policy 1, policy_version 733901 (0.0008) [2023-12-26 20:42:16,219][105620] Updated weights for policy 1, policy_version 733911 (0.0009) [2023-12-26 20:42:16,277][105620] Updated weights for policy 1, policy_version 733921 (0.0005) [2023-12-26 20:42:16,719][105692] Updated weights for policy 0, policy_version 733554 (0.0005) [2023-12-26 20:42:16,789][105692] Updated weights for policy 0, policy_version 733564 (0.0005) [2023-12-26 20:42:16,853][105692] Updated weights for policy 0, policy_version 733574 (0.0006) [2023-12-26 20:42:16,854][105620] Updated weights for policy 1, policy_version 733931 (0.0007) [2023-12-26 20:42:16,916][105620] Updated weights for policy 1, policy_version 733941 (0.0009) [2023-12-26 20:42:16,976][105620] Updated weights for policy 1, policy_version 733951 (0.0009) [2023-12-26 20:42:17,503][105692] Updated weights for policy 0, policy_version 733584 (0.0010) [2023-12-26 20:42:17,573][105692] Updated weights for policy 0, policy_version 733594 (0.0011) [2023-12-26 20:42:17,642][105692] Updated weights for policy 0, policy_version 733604 (0.0011) [2023-12-26 20:42:17,679][105620] Updated weights for policy 1, policy_version 733961 (0.0008) [2023-12-26 20:42:17,735][105620] Updated weights for policy 1, policy_version 733971 (0.0005) [2023-12-26 20:42:17,786][105620] Updated weights for policy 1, policy_version 733981 (0.0005) [2023-12-26 20:42:17,858][105620] Updated weights for policy 1, policy_version 733991 (0.0005) [2023-12-26 20:42:18,306][105692] Updated weights for policy 0, policy_version 733614 (0.0011) [2023-12-26 20:42:18,373][105692] Updated weights for policy 0, policy_version 733624 (0.0012) [2023-12-26 20:42:18,382][105620] Updated weights for policy 1, policy_version 734001 (0.0007) [2023-12-26 20:42:18,435][105692] Updated weights for policy 0, policy_version 733634 (0.0007) [2023-12-26 20:42:18,444][105620] Updated weights for policy 1, policy_version 734011 (0.0006) [2023-12-26 20:42:18,511][105620] Updated weights for policy 1, policy_version 734021 (0.0006) [2023-12-26 20:42:19,076][105620] Updated weights for policy 1, policy_version 734031 (0.0008) [2023-12-26 20:42:19,133][105620] Updated weights for policy 1, policy_version 734041 (0.0009) [2023-12-26 20:42:19,181][105620] Updated weights for policy 1, policy_version 734051 (0.0007) [2023-12-26 20:42:19,247][105692] Updated weights for policy 0, policy_version 733644 (0.0008) [2023-12-26 20:42:19,305][105692] Updated weights for policy 0, policy_version 733654 (0.0006) [2023-12-26 20:42:19,378][105692] Updated weights for policy 0, policy_version 733664 (0.0009) [2023-12-26 20:42:19,916][105620] Updated weights for policy 1, policy_version 734061 (0.0009) [2023-12-26 20:42:19,981][105620] Updated weights for policy 1, policy_version 734071 (0.0006) [2023-12-26 20:42:20,042][105692] Updated weights for policy 0, policy_version 733674 (0.0008) [2023-12-26 20:42:20,044][105620] Updated weights for policy 1, policy_version 734081 (0.0007) [2023-12-26 20:42:20,113][105692] Updated weights for policy 0, policy_version 733684 (0.0009) [2023-12-26 20:42:20,171][105692] Updated weights for policy 0, policy_version 733694 (0.0008) [2023-12-26 20:42:20,236][105692] Updated weights for policy 0, policy_version 733704 (0.0009) [2023-12-26 20:42:20,789][105620] Updated weights for policy 1, policy_version 734091 (0.0009) [2023-12-26 20:42:20,848][105620] Updated weights for policy 1, policy_version 734101 (0.0008) [2023-12-26 20:42:20,892][105692] Updated weights for policy 0, policy_version 733714 (0.0006) [2023-12-26 20:42:20,909][105620] Updated weights for policy 1, policy_version 734111 (0.0010) [2023-12-26 20:42:20,950][105692] Updated weights for policy 0, policy_version 733724 (0.0005) [2023-12-26 20:42:21,013][105692] Updated weights for policy 0, policy_version 733734 (0.0006) [2023-12-26 20:42:21,062][104569] Fps is (10 sec: 20479.9, 60 sec: 20206.9, 300 sec: 19716.4). Total num frames: 375824384. Throughput: 0: 10065.3, 1: 10094.1. Samples: 375807424. Policy #0 lag: (min: 28.0, avg: 32.8, max: 60.0) [2023-12-26 20:42:21,062][104569] Avg episode reward: [(0, '7747.162'), (1, '9266.188')] [2023-12-26 20:42:21,685][105692] Updated weights for policy 0, policy_version 733744 (0.0007) [2023-12-26 20:42:21,752][105620] Updated weights for policy 1, policy_version 734121 (0.0009) [2023-12-26 20:42:21,753][105692] Updated weights for policy 0, policy_version 733754 (0.0006) [2023-12-26 20:42:21,805][105620] Updated weights for policy 1, policy_version 734131 (0.0006) [2023-12-26 20:42:21,818][105692] Updated weights for policy 0, policy_version 733764 (0.0007) [2023-12-26 20:42:21,857][105620] Updated weights for policy 1, policy_version 734141 (0.0008) [2023-12-26 20:42:21,920][105620] Updated weights for policy 1, policy_version 734151 (0.0009) [2023-12-26 20:42:22,518][105692] Updated weights for policy 0, policy_version 733774 (0.0008) [2023-12-26 20:42:22,579][105692] Updated weights for policy 0, policy_version 733784 (0.0009) [2023-12-26 20:42:22,635][105692] Updated weights for policy 0, policy_version 733794 (0.0009) [2023-12-26 20:42:22,665][105620] Updated weights for policy 1, policy_version 734161 (0.0006) [2023-12-26 20:42:22,731][105620] Updated weights for policy 1, policy_version 734171 (0.0009) [2023-12-26 20:42:22,779][105620] Updated weights for policy 1, policy_version 734181 (0.0009) [2023-12-26 20:42:23,211][105692] Updated weights for policy 0, policy_version 733804 (0.0009) [2023-12-26 20:42:23,271][105692] Updated weights for policy 0, policy_version 733814 (0.0008) [2023-12-26 20:42:23,326][105692] Updated weights for policy 0, policy_version 733824 (0.0011) [2023-12-26 20:42:23,609][105620] Updated weights for policy 1, policy_version 734191 (0.0008) [2023-12-26 20:42:23,666][105620] Updated weights for policy 1, policy_version 734201 (0.0009) [2023-12-26 20:42:23,725][105620] Updated weights for policy 1, policy_version 734211 (0.0008) [2023-12-26 20:42:24,050][105692] Updated weights for policy 0, policy_version 733834 (0.0011) [2023-12-26 20:42:24,104][105692] Updated weights for policy 0, policy_version 733844 (0.0010) [2023-12-26 20:42:24,166][105692] Updated weights for policy 0, policy_version 733854 (0.0010) [2023-12-26 20:42:24,224][105692] Updated weights for policy 0, policy_version 733864 (0.0010) [2023-12-26 20:42:24,416][105620] Updated weights for policy 1, policy_version 734221 (0.0007) [2023-12-26 20:42:24,480][105620] Updated weights for policy 1, policy_version 734231 (0.0005) [2023-12-26 20:42:24,539][105620] Updated weights for policy 1, policy_version 734241 (0.0008) [2023-12-26 20:42:24,908][105692] Updated weights for policy 0, policy_version 733874 (0.0008) [2023-12-26 20:42:24,974][105692] Updated weights for policy 0, policy_version 733884 (0.0009) [2023-12-26 20:42:25,040][105692] Updated weights for policy 0, policy_version 733894 (0.0011) [2023-12-26 20:42:25,189][105620] Updated weights for policy 1, policy_version 734251 (0.0008) [2023-12-26 20:42:25,248][105620] Updated weights for policy 1, policy_version 734261 (0.0005) [2023-12-26 20:42:25,318][105620] Updated weights for policy 1, policy_version 734271 (0.0005) [2023-12-26 20:42:25,718][105692] Updated weights for policy 0, policy_version 733904 (0.0006) [2023-12-26 20:42:25,775][105692] Updated weights for policy 0, policy_version 733914 (0.0005) [2023-12-26 20:42:25,826][105620] Updated weights for policy 1, policy_version 734281 (0.0005) [2023-12-26 20:42:25,837][105692] Updated weights for policy 0, policy_version 733924 (0.0005) [2023-12-26 20:42:25,880][105620] Updated weights for policy 1, policy_version 734291 (0.0005) [2023-12-26 20:42:25,953][105620] Updated weights for policy 1, policy_version 734301 (0.0010) [2023-12-26 20:42:26,007][105620] Updated weights for policy 1, policy_version 734311 (0.0006) [2023-12-26 20:42:26,062][104569] Fps is (10 sec: 21298.9, 60 sec: 20206.8, 300 sec: 19660.8). Total num frames: 375922688. Throughput: 0: 10123.5, 1: 10068.7. Samples: 375926644. Policy #0 lag: (min: 28.0, avg: 32.8, max: 60.0) [2023-12-26 20:42:26,063][104569] Avg episode reward: [(0, '8810.832'), (1, '9175.655')] [2023-12-26 20:42:26,492][105692] Updated weights for policy 0, policy_version 733934 (0.0008) [2023-12-26 20:42:26,551][105620] Updated weights for policy 1, policy_version 734321 (0.0009) [2023-12-26 20:42:26,555][105692] Updated weights for policy 0, policy_version 733944 (0.0010) [2023-12-26 20:42:26,603][105620] Updated weights for policy 1, policy_version 734331 (0.0010) [2023-12-26 20:42:26,617][105692] Updated weights for policy 0, policy_version 733954 (0.0011) [2023-12-26 20:42:26,664][105620] Updated weights for policy 1, policy_version 734341 (0.0010) [2023-12-26 20:42:27,350][105692] Updated weights for policy 0, policy_version 733964 (0.0010) [2023-12-26 20:42:27,396][105620] Updated weights for policy 1, policy_version 734351 (0.0010) [2023-12-26 20:42:27,397][105692] Updated weights for policy 0, policy_version 733974 (0.0010) [2023-12-26 20:42:27,451][105692] Updated weights for policy 0, policy_version 733984 (0.0010) [2023-12-26 20:42:27,453][105620] Updated weights for policy 1, policy_version 734361 (0.0010) [2023-12-26 20:42:27,504][105620] Updated weights for policy 1, policy_version 734371 (0.0010) [2023-12-26 20:42:28,183][105692] Updated weights for policy 0, policy_version 733994 (0.0010) [2023-12-26 20:42:28,243][105692] Updated weights for policy 0, policy_version 734004 (0.0010) [2023-12-26 20:42:28,252][105620] Updated weights for policy 1, policy_version 734381 (0.0010) [2023-12-26 20:42:28,290][105692] Updated weights for policy 0, policy_version 734014 (0.0010) [2023-12-26 20:42:28,313][105620] Updated weights for policy 1, policy_version 734391 (0.0010) [2023-12-26 20:42:28,343][105692] Updated weights for policy 0, policy_version 734024 (0.0009) [2023-12-26 20:42:28,372][105620] Updated weights for policy 1, policy_version 734401 (0.0009) [2023-12-26 20:42:29,008][105620] Updated weights for policy 1, policy_version 734411 (0.0007) [2023-12-26 20:42:29,032][105692] Updated weights for policy 0, policy_version 734034 (0.0010) [2023-12-26 20:42:29,055][105620] Updated weights for policy 1, policy_version 734421 (0.0008) [2023-12-26 20:42:29,079][105692] Updated weights for policy 0, policy_version 734044 (0.0010) [2023-12-26 20:42:29,107][105620] Updated weights for policy 1, policy_version 734431 (0.0006) [2023-12-26 20:42:29,134][105692] Updated weights for policy 0, policy_version 734054 (0.0010) [2023-12-26 20:42:29,778][105620] Updated weights for policy 1, policy_version 734441 (0.0008) [2023-12-26 20:42:29,841][105620] Updated weights for policy 1, policy_version 734451 (0.0011) [2023-12-26 20:42:29,859][105692] Updated weights for policy 0, policy_version 734064 (0.0010) [2023-12-26 20:42:29,905][105620] Updated weights for policy 1, policy_version 734461 (0.0006) [2023-12-26 20:42:29,915][105692] Updated weights for policy 0, policy_version 734074 (0.0010) [2023-12-26 20:42:29,971][105620] Updated weights for policy 1, policy_version 734471 (0.0006) [2023-12-26 20:42:29,978][105692] Updated weights for policy 0, policy_version 734084 (0.0011) [2023-12-26 20:42:30,634][105692] Updated weights for policy 0, policy_version 734094 (0.0011) [2023-12-26 20:42:30,681][105692] Updated weights for policy 0, policy_version 734104 (0.0010) [2023-12-26 20:42:30,736][105692] Updated weights for policy 0, policy_version 734114 (0.0010) [2023-12-26 20:42:30,753][105620] Updated weights for policy 1, policy_version 734481 (0.0007) [2023-12-26 20:42:30,818][105620] Updated weights for policy 1, policy_version 734491 (0.0007) [2023-12-26 20:42:30,872][105620] Updated weights for policy 1, policy_version 734501 (0.0008) [2023-12-26 20:42:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 20207.0, 300 sec: 19688.6). Total num frames: 376020992. Throughput: 0: 10132.3, 1: 10135.9. Samples: 375987652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:42:31,062][104569] Avg episode reward: [(0, '9079.740'), (1, '9265.169')] [2023-12-26 20:42:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000734120_187965440.pth... [2023-12-26 20:42:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000734504_188055552.pth... [2023-12-26 20:42:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000732936_187662336.pth [2023-12-26 20:42:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000733320_187752448.pth [2023-12-26 20:42:31,461][105692] Updated weights for policy 0, policy_version 734124 (0.0010) [2023-12-26 20:42:31,508][105692] Updated weights for policy 0, policy_version 734134 (0.0005) [2023-12-26 20:42:31,564][105692] Updated weights for policy 0, policy_version 734144 (0.0005) [2023-12-26 20:42:31,652][105620] Updated weights for policy 1, policy_version 734511 (0.0008) [2023-12-26 20:42:31,701][105620] Updated weights for policy 1, policy_version 734521 (0.0008) [2023-12-26 20:42:31,761][105620] Updated weights for policy 1, policy_version 734531 (0.0009) [2023-12-26 20:42:32,244][105692] Updated weights for policy 0, policy_version 734154 (0.0007) [2023-12-26 20:42:32,315][105692] Updated weights for policy 0, policy_version 734164 (0.0008) [2023-12-26 20:42:32,381][105692] Updated weights for policy 0, policy_version 734174 (0.0008) [2023-12-26 20:42:32,428][105620] Updated weights for policy 1, policy_version 734541 (0.0008) [2023-12-26 20:42:32,443][105692] Updated weights for policy 0, policy_version 734184 (0.0008) [2023-12-26 20:42:32,486][105620] Updated weights for policy 1, policy_version 734551 (0.0009) [2023-12-26 20:42:32,552][105620] Updated weights for policy 1, policy_version 734561 (0.0009) [2023-12-26 20:42:33,024][105692] Updated weights for policy 0, policy_version 734194 (0.0006) [2023-12-26 20:42:33,085][105692] Updated weights for policy 0, policy_version 734204 (0.0009) [2023-12-26 20:42:33,136][105692] Updated weights for policy 0, policy_version 734214 (0.0009) [2023-12-26 20:42:33,364][105620] Updated weights for policy 1, policy_version 734571 (0.0009) [2023-12-26 20:42:33,414][105620] Updated weights for policy 1, policy_version 734581 (0.0009) [2023-12-26 20:42:33,461][105620] Updated weights for policy 1, policy_version 734591 (0.0009) [2023-12-26 20:42:33,844][105692] Updated weights for policy 0, policy_version 734224 (0.0009) [2023-12-26 20:42:33,895][105692] Updated weights for policy 0, policy_version 734234 (0.0009) [2023-12-26 20:42:33,942][105692] Updated weights for policy 0, policy_version 734244 (0.0009) [2023-12-26 20:42:34,186][105620] Updated weights for policy 1, policy_version 734601 (0.0008) [2023-12-26 20:42:34,232][105620] Updated weights for policy 1, policy_version 734611 (0.0009) [2023-12-26 20:42:34,293][105620] Updated weights for policy 1, policy_version 734621 (0.0010) [2023-12-26 20:42:34,362][105620] Updated weights for policy 1, policy_version 734631 (0.0010) [2023-12-26 20:42:34,680][105692] Updated weights for policy 0, policy_version 734254 (0.0009) [2023-12-26 20:42:34,728][105692] Updated weights for policy 0, policy_version 734264 (0.0009) [2023-12-26 20:42:34,775][105692] Updated weights for policy 0, policy_version 734274 (0.0008) [2023-12-26 20:42:35,156][105620] Updated weights for policy 1, policy_version 734641 (0.0009) [2023-12-26 20:42:35,217][105620] Updated weights for policy 1, policy_version 734651 (0.0008) [2023-12-26 20:42:35,271][105620] Updated weights for policy 1, policy_version 734661 (0.0009) [2023-12-26 20:42:35,531][105692] Updated weights for policy 0, policy_version 734284 (0.0009) [2023-12-26 20:42:35,579][105692] Updated weights for policy 0, policy_version 734294 (0.0008) [2023-12-26 20:42:35,626][105692] Updated weights for policy 0, policy_version 734304 (0.0009) [2023-12-26 20:42:36,023][105620] Updated weights for policy 1, policy_version 734671 (0.0008) [2023-12-26 20:42:36,062][104569] Fps is (10 sec: 18841.9, 60 sec: 20070.4, 300 sec: 19660.8). Total num frames: 376111104. Throughput: 0: 10138.8, 1: 9958.8. Samples: 376104588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:42:36,063][104569] Avg episode reward: [(0, '8872.571'), (1, '9265.121')] [2023-12-26 20:42:36,084][105620] Updated weights for policy 1, policy_version 734681 (0.0007) [2023-12-26 20:42:36,146][105620] Updated weights for policy 1, policy_version 734691 (0.0008) [2023-12-26 20:42:36,391][105692] Updated weights for policy 0, policy_version 734314 (0.0009) [2023-12-26 20:42:36,454][105692] Updated weights for policy 0, policy_version 734324 (0.0009) [2023-12-26 20:42:36,514][105692] Updated weights for policy 0, policy_version 734334 (0.0009) [2023-12-26 20:42:36,575][105692] Updated weights for policy 0, policy_version 734344 (0.0009) [2023-12-26 20:42:36,949][105620] Updated weights for policy 1, policy_version 734701 (0.0010) [2023-12-26 20:42:37,021][105620] Updated weights for policy 1, policy_version 734711 (0.0009) [2023-12-26 20:42:37,074][105620] Updated weights for policy 1, policy_version 734721 (0.0008) [2023-12-26 20:42:37,216][105692] Updated weights for policy 0, policy_version 734354 (0.0010) [2023-12-26 20:42:37,271][105692] Updated weights for policy 0, policy_version 734364 (0.0010) [2023-12-26 20:42:37,329][105692] Updated weights for policy 0, policy_version 734374 (0.0010) [2023-12-26 20:42:37,855][105620] Updated weights for policy 1, policy_version 734731 (0.0008) [2023-12-26 20:42:37,899][105620] Updated weights for policy 1, policy_version 734741 (0.0008) [2023-12-26 20:42:37,959][105620] Updated weights for policy 1, policy_version 734751 (0.0008) [2023-12-26 20:42:38,078][105692] Updated weights for policy 0, policy_version 734384 (0.0011) [2023-12-26 20:42:38,126][105692] Updated weights for policy 0, policy_version 734394 (0.0010) [2023-12-26 20:42:38,180][105692] Updated weights for policy 0, policy_version 734404 (0.0010) [2023-12-26 20:42:38,743][105620] Updated weights for policy 1, policy_version 734761 (0.0008) [2023-12-26 20:42:38,799][105620] Updated weights for policy 1, policy_version 734771 (0.0008) [2023-12-26 20:42:38,862][105620] Updated weights for policy 1, policy_version 734781 (0.0008) [2023-12-26 20:42:38,923][105620] Updated weights for policy 1, policy_version 734791 (0.0006) [2023-12-26 20:42:38,941][105692] Updated weights for policy 0, policy_version 734414 (0.0010) [2023-12-26 20:42:39,003][105692] Updated weights for policy 0, policy_version 734424 (0.0011) [2023-12-26 20:42:39,068][105692] Updated weights for policy 0, policy_version 734434 (0.0010) [2023-12-26 20:42:39,665][105620] Updated weights for policy 1, policy_version 734801 (0.0008) [2023-12-26 20:42:39,718][105620] Updated weights for policy 1, policy_version 734811 (0.0009) [2023-12-26 20:42:39,778][105620] Updated weights for policy 1, policy_version 734821 (0.0009) [2023-12-26 20:42:39,849][105692] Updated weights for policy 0, policy_version 734444 (0.0009) [2023-12-26 20:42:39,916][105692] Updated weights for policy 0, policy_version 734454 (0.0008) [2023-12-26 20:42:39,978][105692] Updated weights for policy 0, policy_version 734464 (0.0008) [2023-12-26 20:42:40,525][105620] Updated weights for policy 1, policy_version 734831 (0.0009) [2023-12-26 20:42:40,596][105620] Updated weights for policy 1, policy_version 734841 (0.0010) [2023-12-26 20:42:40,657][105620] Updated weights for policy 1, policy_version 734852 (0.0008) [2023-12-26 20:42:40,672][105692] Updated weights for policy 0, policy_version 734474 (0.0008) [2023-12-26 20:42:40,728][105692] Updated weights for policy 0, policy_version 734484 (0.0009) [2023-12-26 20:42:40,775][105692] Updated weights for policy 0, policy_version 734494 (0.0008) [2023-12-26 20:42:40,823][105692] Updated weights for policy 0, policy_version 734504 (0.0009) [2023-12-26 20:42:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 20070.4, 300 sec: 19688.6). Total num frames: 376209408. Throughput: 0: 10012.6, 1: 9906.7. Samples: 376216172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:42:41,062][104569] Avg episode reward: [(0, '9217.277'), (1, '9265.156')] [2023-12-26 20:42:41,430][105620] Updated weights for policy 1, policy_version 734862 (0.0008) [2023-12-26 20:42:41,496][105620] Updated weights for policy 1, policy_version 734872 (0.0009) [2023-12-26 20:42:41,562][105620] Updated weights for policy 1, policy_version 734882 (0.0008) [2023-12-26 20:42:41,619][105692] Updated weights for policy 0, policy_version 734514 (0.0008) [2023-12-26 20:42:41,685][105692] Updated weights for policy 0, policy_version 734524 (0.0007) [2023-12-26 20:42:41,760][105692] Updated weights for policy 0, policy_version 734534 (0.0007) [2023-12-26 20:42:42,308][105620] Updated weights for policy 1, policy_version 734892 (0.0008) [2023-12-26 20:42:42,374][105620] Updated weights for policy 1, policy_version 734902 (0.0008) [2023-12-26 20:42:42,433][105620] Updated weights for policy 1, policy_version 734912 (0.0008) [2023-12-26 20:42:42,484][105692] Updated weights for policy 0, policy_version 734544 (0.0010) [2023-12-26 20:42:42,553][105692] Updated weights for policy 0, policy_version 734554 (0.0010) [2023-12-26 20:42:42,608][105692] Updated weights for policy 0, policy_version 734564 (0.0010) [2023-12-26 20:42:43,202][105620] Updated weights for policy 1, policy_version 734922 (0.0006) [2023-12-26 20:42:43,254][105620] Updated weights for policy 1, policy_version 734932 (0.0008) [2023-12-26 20:42:43,310][105620] Updated weights for policy 1, policy_version 734942 (0.0008) [2023-12-26 20:42:43,335][105692] Updated weights for policy 0, policy_version 734574 (0.0010) [2023-12-26 20:42:43,353][105620] Updated weights for policy 1, policy_version 734952 (0.0008) [2023-12-26 20:42:43,397][105692] Updated weights for policy 0, policy_version 734584 (0.0010) [2023-12-26 20:42:43,460][105692] Updated weights for policy 0, policy_version 734594 (0.0010) [2023-12-26 20:42:44,085][105620] Updated weights for policy 1, policy_version 734962 (0.0010) [2023-12-26 20:42:44,143][105620] Updated weights for policy 1, policy_version 734972 (0.0009) [2023-12-26 20:42:44,191][105620] Updated weights for policy 1, policy_version 734982 (0.0007) [2023-12-26 20:42:44,193][105692] Updated weights for policy 0, policy_version 734604 (0.0008) [2023-12-26 20:42:44,259][105692] Updated weights for policy 0, policy_version 734614 (0.0008) [2023-12-26 20:42:44,306][105692] Updated weights for policy 0, policy_version 734624 (0.0006) [2023-12-26 20:42:44,955][105620] Updated weights for policy 1, policy_version 734992 (0.0008) [2023-12-26 20:42:45,008][105620] Updated weights for policy 1, policy_version 735002 (0.0008) [2023-12-26 20:42:45,024][105692] Updated weights for policy 0, policy_version 734634 (0.0008) [2023-12-26 20:42:45,063][105620] Updated weights for policy 1, policy_version 735012 (0.0008) [2023-12-26 20:42:45,084][105692] Updated weights for policy 0, policy_version 734644 (0.0010) [2023-12-26 20:42:45,146][105692] Updated weights for policy 0, policy_version 734654 (0.0010) [2023-12-26 20:42:45,207][105692] Updated weights for policy 0, policy_version 734664 (0.0011) [2023-12-26 20:42:45,752][105620] Updated weights for policy 1, policy_version 735022 (0.0006) [2023-12-26 20:42:45,806][105620] Updated weights for policy 1, policy_version 735032 (0.0008) [2023-12-26 20:42:45,860][105620] Updated weights for policy 1, policy_version 735042 (0.0007) [2023-12-26 20:42:45,862][105692] Updated weights for policy 0, policy_version 734674 (0.0007) [2023-12-26 20:42:45,915][105692] Updated weights for policy 0, policy_version 734684 (0.0008) [2023-12-26 20:42:45,969][105692] Updated weights for policy 0, policy_version 734694 (0.0010) [2023-12-26 20:42:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 20070.4, 300 sec: 19688.6). Total num frames: 376307712. Throughput: 0: 9944.1, 1: 9825.8. Samples: 376272396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:42:46,063][104569] Avg episode reward: [(0, '9349.475'), (1, '9264.780')] [2023-12-26 20:42:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000735048_188194816.pth... [2023-12-26 20:42:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000734696_188112896.pth... [2023-12-26 20:42:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000733544_187817984.pth [2023-12-26 20:42:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000733896_187899904.pth [2023-12-26 20:42:46,529][105620] Updated weights for policy 1, policy_version 735052 (0.0007) [2023-12-26 20:42:46,587][105620] Updated weights for policy 1, policy_version 735062 (0.0008) [2023-12-26 20:42:46,648][105620] Updated weights for policy 1, policy_version 735072 (0.0007) [2023-12-26 20:42:46,748][105692] Updated weights for policy 0, policy_version 734704 (0.0009) [2023-12-26 20:42:46,793][105692] Updated weights for policy 0, policy_version 734714 (0.0008) [2023-12-26 20:42:46,842][105692] Updated weights for policy 0, policy_version 734724 (0.0008) [2023-12-26 20:42:47,401][105620] Updated weights for policy 1, policy_version 735082 (0.0006) [2023-12-26 20:42:47,453][105620] Updated weights for policy 1, policy_version 735092 (0.0006) [2023-12-26 20:42:47,502][105620] Updated weights for policy 1, policy_version 735102 (0.0009) [2023-12-26 20:42:47,543][105692] Updated weights for policy 0, policy_version 734734 (0.0005) [2023-12-26 20:42:47,562][105620] Updated weights for policy 1, policy_version 735112 (0.0007) [2023-12-26 20:42:47,595][105692] Updated weights for policy 0, policy_version 734745 (0.0009) [2023-12-26 20:42:47,652][105692] Updated weights for policy 0, policy_version 734755 (0.0010) [2023-12-26 20:42:48,202][105620] Updated weights for policy 1, policy_version 735122 (0.0008) [2023-12-26 20:42:48,262][105620] Updated weights for policy 1, policy_version 735132 (0.0009) [2023-12-26 20:42:48,330][105620] Updated weights for policy 1, policy_version 735142 (0.0009) [2023-12-26 20:42:48,472][105692] Updated weights for policy 0, policy_version 734765 (0.0009) [2023-12-26 20:42:48,534][105692] Updated weights for policy 0, policy_version 734775 (0.0009) [2023-12-26 20:42:48,593][105692] Updated weights for policy 0, policy_version 734785 (0.0009) [2023-12-26 20:42:49,061][105620] Updated weights for policy 1, policy_version 735152 (0.0008) [2023-12-26 20:42:49,108][105620] Updated weights for policy 1, policy_version 735162 (0.0009) [2023-12-26 20:42:49,154][105620] Updated weights for policy 1, policy_version 735172 (0.0009) [2023-12-26 20:42:49,340][105692] Updated weights for policy 0, policy_version 734795 (0.0008) [2023-12-26 20:42:49,403][105692] Updated weights for policy 0, policy_version 734805 (0.0008) [2023-12-26 20:42:49,455][105692] Updated weights for policy 0, policy_version 734815 (0.0008) [2023-12-26 20:42:49,839][105620] Updated weights for policy 1, policy_version 735182 (0.0008) [2023-12-26 20:42:49,905][105620] Updated weights for policy 1, policy_version 735192 (0.0007) [2023-12-26 20:42:49,971][105620] Updated weights for policy 1, policy_version 735202 (0.0007) [2023-12-26 20:42:50,318][105692] Updated weights for policy 0, policy_version 734825 (0.0008) [2023-12-26 20:42:50,378][105692] Updated weights for policy 0, policy_version 734835 (0.0010) [2023-12-26 20:42:50,431][105692] Updated weights for policy 0, policy_version 734845 (0.0009) [2023-12-26 20:42:50,484][105692] Updated weights for policy 0, policy_version 734855 (0.0009) [2023-12-26 20:42:50,524][105620] Updated weights for policy 1, policy_version 735212 (0.0005) [2023-12-26 20:42:50,587][105620] Updated weights for policy 1, policy_version 735222 (0.0007) [2023-12-26 20:42:50,650][105620] Updated weights for policy 1, policy_version 735232 (0.0009) [2023-12-26 20:42:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 376397824. Throughput: 0: 9831.5, 1: 9825.5. Samples: 376388664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:42:51,062][104569] Avg episode reward: [(0, '9349.846'), (1, '9264.513')] [2023-12-26 20:42:51,296][105692] Updated weights for policy 0, policy_version 734865 (0.0009) [2023-12-26 20:42:51,362][105692] Updated weights for policy 0, policy_version 734875 (0.0010) [2023-12-26 20:42:51,416][105620] Updated weights for policy 1, policy_version 735242 (0.0009) [2023-12-26 20:42:51,423][105692] Updated weights for policy 0, policy_version 734885 (0.0007) [2023-12-26 20:42:51,464][105620] Updated weights for policy 1, policy_version 735252 (0.0007) [2023-12-26 20:42:51,523][105620] Updated weights for policy 1, policy_version 735262 (0.0009) [2023-12-26 20:42:51,585][105620] Updated weights for policy 1, policy_version 735272 (0.0009) [2023-12-26 20:42:52,094][105692] Updated weights for policy 0, policy_version 734895 (0.0006) [2023-12-26 20:42:52,149][105692] Updated weights for policy 0, policy_version 734905 (0.0006) [2023-12-26 20:42:52,215][105692] Updated weights for policy 0, policy_version 734915 (0.0007) [2023-12-26 20:42:52,396][105620] Updated weights for policy 1, policy_version 735282 (0.0007) [2023-12-26 20:42:52,462][105620] Updated weights for policy 1, policy_version 735292 (0.0009) [2023-12-26 20:42:52,526][105620] Updated weights for policy 1, policy_version 735302 (0.0009) [2023-12-26 20:42:52,928][105692] Updated weights for policy 0, policy_version 734925 (0.0006) [2023-12-26 20:42:52,980][105692] Updated weights for policy 0, policy_version 734935 (0.0009) [2023-12-26 20:42:53,033][105692] Updated weights for policy 0, policy_version 734947 (0.0010) [2023-12-26 20:42:53,184][105620] Updated weights for policy 1, policy_version 735312 (0.0006) [2023-12-26 20:42:53,254][105620] Updated weights for policy 1, policy_version 735322 (0.0007) [2023-12-26 20:42:53,315][105620] Updated weights for policy 1, policy_version 735332 (0.0009) [2023-12-26 20:42:53,885][105620] Updated weights for policy 1, policy_version 735342 (0.0007) [2023-12-26 20:42:53,902][105692] Updated weights for policy 0, policy_version 734958 (0.0008) [2023-12-26 20:42:53,940][105620] Updated weights for policy 1, policy_version 735352 (0.0006) [2023-12-26 20:42:53,958][105692] Updated weights for policy 0, policy_version 734968 (0.0007) [2023-12-26 20:42:53,998][105620] Updated weights for policy 1, policy_version 735362 (0.0008) [2023-12-26 20:42:54,023][105692] Updated weights for policy 0, policy_version 734978 (0.0005) [2023-12-26 20:42:54,624][105620] Updated weights for policy 1, policy_version 735372 (0.0007) [2023-12-26 20:42:54,692][105620] Updated weights for policy 1, policy_version 735382 (0.0007) [2023-12-26 20:42:54,760][105620] Updated weights for policy 1, policy_version 735392 (0.0005) [2023-12-26 20:42:54,799][105692] Updated weights for policy 0, policy_version 734988 (0.0008) [2023-12-26 20:42:54,859][105692] Updated weights for policy 0, policy_version 734998 (0.0008) [2023-12-26 20:42:54,920][105692] Updated weights for policy 0, policy_version 735008 (0.0009) [2023-12-26 20:42:55,443][105620] Updated weights for policy 1, policy_version 735402 (0.0006) [2023-12-26 20:42:55,505][105620] Updated weights for policy 1, policy_version 735412 (0.0007) [2023-12-26 20:42:55,564][105620] Updated weights for policy 1, policy_version 735422 (0.0009) [2023-12-26 20:42:55,626][105620] Updated weights for policy 1, policy_version 735432 (0.0009) [2023-12-26 20:42:55,692][105692] Updated weights for policy 0, policy_version 735018 (0.0009) [2023-12-26 20:42:55,745][105692] Updated weights for policy 0, policy_version 735028 (0.0010) [2023-12-26 20:42:55,802][105692] Updated weights for policy 0, policy_version 735038 (0.0013) [2023-12-26 20:42:56,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 376496128. Throughput: 0: 9656.4, 1: 9906.9. Samples: 376504468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:42:56,062][104569] Avg episode reward: [(0, '8796.280'), (1, '9355.535')] [2023-12-26 20:42:56,205][105620] Updated weights for policy 1, policy_version 735442 (0.0007) [2023-12-26 20:42:56,268][105620] Updated weights for policy 1, policy_version 735452 (0.0009) [2023-12-26 20:42:56,314][105620] Updated weights for policy 1, policy_version 735462 (0.0009) [2023-12-26 20:42:56,601][105692] Updated weights for policy 0, policy_version 735049 (0.0009) [2023-12-26 20:42:56,662][105692] Updated weights for policy 0, policy_version 735059 (0.0009) [2023-12-26 20:42:56,715][105692] Updated weights for policy 0, policy_version 735069 (0.0009) [2023-12-26 20:42:56,768][105692] Updated weights for policy 0, policy_version 735079 (0.0008) [2023-12-26 20:42:57,021][105620] Updated weights for policy 1, policy_version 735472 (0.0007) [2023-12-26 20:42:57,066][105620] Updated weights for policy 1, policy_version 735482 (0.0005) [2023-12-26 20:42:57,113][105620] Updated weights for policy 1, policy_version 735492 (0.0007) [2023-12-26 20:42:57,465][105692] Updated weights for policy 0, policy_version 735089 (0.0006) [2023-12-26 20:42:57,514][105692] Updated weights for policy 0, policy_version 735099 (0.0005) [2023-12-26 20:42:57,562][105692] Updated weights for policy 0, policy_version 735109 (0.0005) [2023-12-26 20:42:57,879][105620] Updated weights for policy 1, policy_version 735502 (0.0007) [2023-12-26 20:42:57,940][105620] Updated weights for policy 1, policy_version 735513 (0.0010) [2023-12-26 20:42:57,995][105620] Updated weights for policy 1, policy_version 735523 (0.0009) [2023-12-26 20:42:58,092][105692] Updated weights for policy 0, policy_version 735119 (0.0005) [2023-12-26 20:42:58,155][105692] Updated weights for policy 0, policy_version 735129 (0.0007) [2023-12-26 20:42:58,221][105692] Updated weights for policy 0, policy_version 735139 (0.0010) [2023-12-26 20:42:58,821][105620] Updated weights for policy 1, policy_version 735533 (0.0008) [2023-12-26 20:42:58,870][105620] Updated weights for policy 1, policy_version 735543 (0.0008) [2023-12-26 20:42:58,918][105692] Updated weights for policy 0, policy_version 735149 (0.0008) [2023-12-26 20:42:58,924][105620] Updated weights for policy 1, policy_version 735553 (0.0007) [2023-12-26 20:42:58,973][105692] Updated weights for policy 0, policy_version 735159 (0.0011) [2023-12-26 20:42:59,031][105692] Updated weights for policy 0, policy_version 735169 (0.0010) [2023-12-26 20:42:59,698][105620] Updated weights for policy 1, policy_version 735563 (0.0006) [2023-12-26 20:42:59,759][105620] Updated weights for policy 1, policy_version 735573 (0.0008) [2023-12-26 20:42:59,777][105692] Updated weights for policy 0, policy_version 735179 (0.0009) [2023-12-26 20:42:59,822][105620] Updated weights for policy 1, policy_version 735583 (0.0008) [2023-12-26 20:42:59,835][105692] Updated weights for policy 0, policy_version 735189 (0.0007) [2023-12-26 20:42:59,893][105692] Updated weights for policy 0, policy_version 735199 (0.0007) [2023-12-26 20:43:00,520][105620] Updated weights for policy 1, policy_version 735593 (0.0008) [2023-12-26 20:43:00,532][105692] Updated weights for policy 0, policy_version 735209 (0.0008) [2023-12-26 20:43:00,591][105620] Updated weights for policy 1, policy_version 735603 (0.0008) [2023-12-26 20:43:00,602][105692] Updated weights for policy 0, policy_version 735219 (0.0006) [2023-12-26 20:43:00,651][105620] Updated weights for policy 1, policy_version 735613 (0.0008) [2023-12-26 20:43:00,666][105692] Updated weights for policy 0, policy_version 735229 (0.0005) [2023-12-26 20:43:00,699][105620] Updated weights for policy 1, policy_version 735623 (0.0010) [2023-12-26 20:43:00,722][105692] Updated weights for policy 0, policy_version 735239 (0.0006) [2023-12-26 20:43:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 376594432. Throughput: 0: 9720.1, 1: 9840.1. Samples: 376563736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:43:01,063][104569] Avg episode reward: [(0, '8504.699'), (1, '9355.597')] [2023-12-26 20:43:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000735240_188252160.pth... [2023-12-26 20:43:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000735624_188342272.pth... [2023-12-26 20:43:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000734120_187965440.pth [2023-12-26 20:43:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000734504_188055552.pth [2023-12-26 20:43:01,366][105692] Updated weights for policy 0, policy_version 735249 (0.0008) [2023-12-26 20:43:01,419][105692] Updated weights for policy 0, policy_version 735259 (0.0007) [2023-12-26 20:43:01,443][105620] Updated weights for policy 1, policy_version 735633 (0.0008) [2023-12-26 20:43:01,478][105692] Updated weights for policy 0, policy_version 735269 (0.0005) [2023-12-26 20:43:01,516][105620] Updated weights for policy 1, policy_version 735643 (0.0007) [2023-12-26 20:43:01,578][105620] Updated weights for policy 1, policy_version 735653 (0.0008) [2023-12-26 20:43:02,159][105620] Updated weights for policy 1, policy_version 735663 (0.0006) [2023-12-26 20:43:02,225][105620] Updated weights for policy 1, policy_version 735673 (0.0006) [2023-12-26 20:43:02,270][105692] Updated weights for policy 0, policy_version 735279 (0.0006) [2023-12-26 20:43:02,290][105620] Updated weights for policy 1, policy_version 735683 (0.0008) [2023-12-26 20:43:02,328][105692] Updated weights for policy 0, policy_version 735289 (0.0009) [2023-12-26 20:43:02,395][105692] Updated weights for policy 0, policy_version 735299 (0.0009) [2023-12-26 20:43:02,922][105620] Updated weights for policy 1, policy_version 735693 (0.0009) [2023-12-26 20:43:02,986][105620] Updated weights for policy 1, policy_version 735703 (0.0010) [2023-12-26 20:43:03,017][105692] Updated weights for policy 0, policy_version 735309 (0.0007) [2023-12-26 20:43:03,048][105620] Updated weights for policy 1, policy_version 735713 (0.0010) [2023-12-26 20:43:03,070][105692] Updated weights for policy 0, policy_version 735319 (0.0005) [2023-12-26 20:43:03,126][105692] Updated weights for policy 0, policy_version 735329 (0.0005) [2023-12-26 20:43:03,733][105692] Updated weights for policy 0, policy_version 735339 (0.0007) [2023-12-26 20:43:03,767][105620] Updated weights for policy 1, policy_version 735723 (0.0010) [2023-12-26 20:43:03,787][105692] Updated weights for policy 0, policy_version 735349 (0.0010) [2023-12-26 20:43:03,809][105620] Updated weights for policy 1, policy_version 735733 (0.0007) [2023-12-26 20:43:03,847][105692] Updated weights for policy 0, policy_version 735359 (0.0010) [2023-12-26 20:43:03,866][105620] Updated weights for policy 1, policy_version 735743 (0.0007) [2023-12-26 20:43:04,456][105692] Updated weights for policy 0, policy_version 735369 (0.0009) [2023-12-26 20:43:04,512][105692] Updated weights for policy 0, policy_version 735379 (0.0008) [2023-12-26 20:43:04,572][105692] Updated weights for policy 0, policy_version 735389 (0.0006) [2023-12-26 20:43:04,629][105692] Updated weights for policy 0, policy_version 735399 (0.0005) [2023-12-26 20:43:04,629][105620] Updated weights for policy 1, policy_version 735753 (0.0009) [2023-12-26 20:43:04,674][105620] Updated weights for policy 1, policy_version 735763 (0.0010) [2023-12-26 20:43:04,726][105620] Updated weights for policy 1, policy_version 735773 (0.0010) [2023-12-26 20:43:04,788][105620] Updated weights for policy 1, policy_version 735783 (0.0009) [2023-12-26 20:43:05,319][105692] Updated weights for policy 0, policy_version 735409 (0.0006) [2023-12-26 20:43:05,365][105692] Updated weights for policy 0, policy_version 735419 (0.0005) [2023-12-26 20:43:05,424][105692] Updated weights for policy 0, policy_version 735429 (0.0008) [2023-12-26 20:43:05,503][105620] Updated weights for policy 1, policy_version 735793 (0.0006) [2023-12-26 20:43:05,568][105620] Updated weights for policy 1, policy_version 735803 (0.0007) [2023-12-26 20:43:05,623][105620] Updated weights for policy 1, policy_version 735813 (0.0009) [2023-12-26 20:43:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 376692736. Throughput: 0: 9775.7, 1: 9706.9. Samples: 376684144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:43:06,063][104569] Avg episode reward: [(0, '9260.722'), (1, '9355.902')] [2023-12-26 20:43:06,182][105692] Updated weights for policy 0, policy_version 735439 (0.0007) [2023-12-26 20:43:06,232][105692] Updated weights for policy 0, policy_version 735449 (0.0005) [2023-12-26 20:43:06,248][105620] Updated weights for policy 1, policy_version 735823 (0.0008) [2023-12-26 20:43:06,296][105692] Updated weights for policy 0, policy_version 735459 (0.0007) [2023-12-26 20:43:06,305][105620] Updated weights for policy 1, policy_version 735833 (0.0007) [2023-12-26 20:43:06,364][105620] Updated weights for policy 1, policy_version 735843 (0.0007) [2023-12-26 20:43:06,963][105620] Updated weights for policy 1, policy_version 735853 (0.0009) [2023-12-26 20:43:07,025][105620] Updated weights for policy 1, policy_version 735863 (0.0009) [2023-12-26 20:43:07,090][105692] Updated weights for policy 0, policy_version 735469 (0.0007) [2023-12-26 20:43:07,092][105620] Updated weights for policy 1, policy_version 735873 (0.0008) [2023-12-26 20:43:07,159][105692] Updated weights for policy 0, policy_version 735479 (0.0006) [2023-12-26 20:43:07,210][105692] Updated weights for policy 0, policy_version 735489 (0.0005) [2023-12-26 20:43:07,768][105620] Updated weights for policy 1, policy_version 735883 (0.0009) [2023-12-26 20:43:07,819][105620] Updated weights for policy 1, policy_version 735893 (0.0010) [2023-12-26 20:43:07,842][105692] Updated weights for policy 0, policy_version 735499 (0.0005) [2023-12-26 20:43:07,867][105620] Updated weights for policy 1, policy_version 735903 (0.0010) [2023-12-26 20:43:07,891][105692] Updated weights for policy 0, policy_version 735509 (0.0005) [2023-12-26 20:43:07,944][105692] Updated weights for policy 0, policy_version 735519 (0.0006) [2023-12-26 20:43:08,557][105620] Updated weights for policy 1, policy_version 735913 (0.0010) [2023-12-26 20:43:08,564][105692] Updated weights for policy 0, policy_version 735529 (0.0006) [2023-12-26 20:43:08,614][105620] Updated weights for policy 1, policy_version 735923 (0.0005) [2023-12-26 20:43:08,620][105692] Updated weights for policy 0, policy_version 735539 (0.0010) [2023-12-26 20:43:08,666][105692] Updated weights for policy 0, policy_version 735549 (0.0010) [2023-12-26 20:43:08,672][105620] Updated weights for policy 1, policy_version 735933 (0.0006) [2023-12-26 20:43:08,718][105692] Updated weights for policy 0, policy_version 735559 (0.0010) [2023-12-26 20:43:08,729][105620] Updated weights for policy 1, policy_version 735943 (0.0007) [2023-12-26 20:43:09,319][105620] Updated weights for policy 1, policy_version 735953 (0.0007) [2023-12-26 20:43:09,387][105620] Updated weights for policy 1, policy_version 735963 (0.0008) [2023-12-26 20:43:09,458][105620] Updated weights for policy 1, policy_version 735973 (0.0009) [2023-12-26 20:43:09,511][105692] Updated weights for policy 0, policy_version 735569 (0.0006) [2023-12-26 20:43:09,571][105692] Updated weights for policy 0, policy_version 735579 (0.0007) [2023-12-26 20:43:09,622][105692] Updated weights for policy 0, policy_version 735589 (0.0008) [2023-12-26 20:43:10,240][105620] Updated weights for policy 1, policy_version 735983 (0.0009) [2023-12-26 20:43:10,291][105620] Updated weights for policy 1, policy_version 735993 (0.0009) [2023-12-26 20:43:10,306][105692] Updated weights for policy 0, policy_version 735599 (0.0006) [2023-12-26 20:43:10,357][105620] Updated weights for policy 1, policy_version 736003 (0.0008) [2023-12-26 20:43:10,363][105692] Updated weights for policy 0, policy_version 735609 (0.0007) [2023-12-26 20:43:10,417][105692] Updated weights for policy 0, policy_version 735619 (0.0008) [2023-12-26 20:43:11,046][105692] Updated weights for policy 0, policy_version 735629 (0.0009) [2023-12-26 20:43:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 376791040. Throughput: 0: 9743.4, 1: 9745.1. Samples: 376803624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:43:11,063][104569] Avg episode reward: [(0, '9350.225'), (1, '9356.021')] [2023-12-26 20:43:11,104][105692] Updated weights for policy 0, policy_version 735639 (0.0007) [2023-12-26 20:43:11,174][105692] Updated weights for policy 0, policy_version 735649 (0.0009) [2023-12-26 20:43:11,205][105620] Updated weights for policy 1, policy_version 736013 (0.0007) [2023-12-26 20:43:11,265][105620] Updated weights for policy 1, policy_version 736023 (0.0008) [2023-12-26 20:43:11,321][105620] Updated weights for policy 1, policy_version 736033 (0.0009) [2023-12-26 20:43:11,977][105692] Updated weights for policy 0, policy_version 735659 (0.0008) [2023-12-26 20:43:12,040][105692] Updated weights for policy 0, policy_version 735669 (0.0007) [2023-12-26 20:43:12,099][105692] Updated weights for policy 0, policy_version 735679 (0.0008) [2023-12-26 20:43:12,108][105620] Updated weights for policy 1, policy_version 736043 (0.0009) [2023-12-26 20:43:12,158][105620] Updated weights for policy 1, policy_version 736053 (0.0011) [2023-12-26 20:43:12,207][105620] Updated weights for policy 1, policy_version 736063 (0.0010) [2023-12-26 20:43:12,737][105692] Updated weights for policy 0, policy_version 735689 (0.0008) [2023-12-26 20:43:12,798][105692] Updated weights for policy 0, policy_version 735699 (0.0009) [2023-12-26 20:43:12,849][105692] Updated weights for policy 0, policy_version 735709 (0.0010) [2023-12-26 20:43:12,916][105692] Updated weights for policy 0, policy_version 735719 (0.0010) [2023-12-26 20:43:13,054][105620] Updated weights for policy 1, policy_version 736073 (0.0010) [2023-12-26 20:43:13,113][105620] Updated weights for policy 1, policy_version 736083 (0.0009) [2023-12-26 20:43:13,174][105620] Updated weights for policy 1, policy_version 736093 (0.0009) [2023-12-26 20:43:13,232][105620] Updated weights for policy 1, policy_version 736103 (0.0010) [2023-12-26 20:43:13,537][105692] Updated weights for policy 0, policy_version 735729 (0.0009) [2023-12-26 20:43:13,588][105692] Updated weights for policy 0, policy_version 735739 (0.0008) [2023-12-26 20:43:13,641][105692] Updated weights for policy 0, policy_version 735749 (0.0010) [2023-12-26 20:43:13,987][105620] Updated weights for policy 1, policy_version 736113 (0.0006) [2023-12-26 20:43:14,035][105620] Updated weights for policy 1, policy_version 736123 (0.0006) [2023-12-26 20:43:14,090][105620] Updated weights for policy 1, policy_version 736133 (0.0009) [2023-12-26 20:43:14,463][105692] Updated weights for policy 0, policy_version 735759 (0.0010) [2023-12-26 20:43:14,527][105692] Updated weights for policy 0, policy_version 735769 (0.0009) [2023-12-26 20:43:14,587][105692] Updated weights for policy 0, policy_version 735779 (0.0009) [2023-12-26 20:43:14,745][105620] Updated weights for policy 1, policy_version 736143 (0.0008) [2023-12-26 20:43:14,811][105620] Updated weights for policy 1, policy_version 736153 (0.0010) [2023-12-26 20:43:14,864][105620] Updated weights for policy 1, policy_version 736163 (0.0009) [2023-12-26 20:43:15,241][105692] Updated weights for policy 0, policy_version 735789 (0.0009) [2023-12-26 20:43:15,304][105692] Updated weights for policy 0, policy_version 735799 (0.0008) [2023-12-26 20:43:15,361][105692] Updated weights for policy 0, policy_version 735809 (0.0006) [2023-12-26 20:43:15,733][105620] Updated weights for policy 1, policy_version 736173 (0.0010) [2023-12-26 20:43:15,796][105620] Updated weights for policy 1, policy_version 736183 (0.0010) [2023-12-26 20:43:15,845][105620] Updated weights for policy 1, policy_version 736193 (0.0009) [2023-12-26 20:43:15,993][105692] Updated weights for policy 0, policy_version 735819 (0.0007) [2023-12-26 20:43:16,054][105692] Updated weights for policy 0, policy_version 735829 (0.0010) [2023-12-26 20:43:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 376889344. Throughput: 0: 9756.9, 1: 9652.8. Samples: 376861088. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:43:16,063][104569] Avg episode reward: [(0, '9349.827'), (1, '9264.843')] [2023-12-26 20:43:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000736200_188489728.pth... [2023-12-26 20:43:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000735048_188194816.pth [2023-12-26 20:43:16,115][105692] Updated weights for policy 0, policy_version 735839 (0.0010) [2023-12-26 20:43:16,173][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000735848_188407808.pth... [2023-12-26 20:43:16,178][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000734696_188112896.pth [2023-12-26 20:43:16,656][105620] Updated weights for policy 1, policy_version 736203 (0.0008) [2023-12-26 20:43:16,712][105620] Updated weights for policy 1, policy_version 736213 (0.0010) [2023-12-26 20:43:16,724][105692] Updated weights for policy 0, policy_version 735849 (0.0011) [2023-12-26 20:43:16,770][105620] Updated weights for policy 1, policy_version 736223 (0.0007) [2023-12-26 20:43:16,777][105692] Updated weights for policy 0, policy_version 735859 (0.0010) [2023-12-26 20:43:16,833][105692] Updated weights for policy 0, policy_version 735869 (0.0005) [2023-12-26 20:43:16,883][105692] Updated weights for policy 0, policy_version 735879 (0.0005) [2023-12-26 20:43:17,512][105692] Updated weights for policy 0, policy_version 735889 (0.0010) [2023-12-26 20:43:17,566][105692] Updated weights for policy 0, policy_version 735899 (0.0010) [2023-12-26 20:43:17,600][105620] Updated weights for policy 1, policy_version 736233 (0.0008) [2023-12-26 20:43:17,615][105692] Updated weights for policy 0, policy_version 735909 (0.0010) [2023-12-26 20:43:17,653][105620] Updated weights for policy 1, policy_version 736243 (0.0008) [2023-12-26 20:43:17,716][105620] Updated weights for policy 1, policy_version 736253 (0.0008) [2023-12-26 20:43:17,764][105620] Updated weights for policy 1, policy_version 736263 (0.0008) [2023-12-26 20:43:18,278][105692] Updated weights for policy 0, policy_version 735919 (0.0007) [2023-12-26 20:43:18,346][105692] Updated weights for policy 0, policy_version 735929 (0.0008) [2023-12-26 20:43:18,410][105692] Updated weights for policy 0, policy_version 735939 (0.0008) [2023-12-26 20:43:18,491][105620] Updated weights for policy 1, policy_version 736273 (0.0008) [2023-12-26 20:43:18,551][105620] Updated weights for policy 1, policy_version 736283 (0.0007) [2023-12-26 20:43:18,614][105620] Updated weights for policy 1, policy_version 736293 (0.0007) [2023-12-26 20:43:19,099][105692] Updated weights for policy 0, policy_version 735949 (0.0009) [2023-12-26 20:43:19,153][105692] Updated weights for policy 0, policy_version 735959 (0.0008) [2023-12-26 20:43:19,201][105692] Updated weights for policy 0, policy_version 735969 (0.0009) [2023-12-26 20:43:19,352][105620] Updated weights for policy 1, policy_version 736303 (0.0008) [2023-12-26 20:43:19,405][105620] Updated weights for policy 1, policy_version 736313 (0.0008) [2023-12-26 20:43:19,463][105620] Updated weights for policy 1, policy_version 736323 (0.0009) [2023-12-26 20:43:19,994][105692] Updated weights for policy 0, policy_version 735979 (0.0009) [2023-12-26 20:43:20,061][105692] Updated weights for policy 0, policy_version 735989 (0.0009) [2023-12-26 20:43:20,122][105692] Updated weights for policy 0, policy_version 735999 (0.0009) [2023-12-26 20:43:20,260][105620] Updated weights for policy 1, policy_version 736333 (0.0009) [2023-12-26 20:43:20,315][105620] Updated weights for policy 1, policy_version 736343 (0.0008) [2023-12-26 20:43:20,384][105620] Updated weights for policy 1, policy_version 736353 (0.0008) [2023-12-26 20:43:20,911][105692] Updated weights for policy 0, policy_version 736009 (0.0010) [2023-12-26 20:43:20,962][105692] Updated weights for policy 0, policy_version 736019 (0.0009) [2023-12-26 20:43:21,022][105692] Updated weights for policy 0, policy_version 736029 (0.0010) [2023-12-26 20:43:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 376979456. Throughput: 0: 9757.0, 1: 9615.0. Samples: 376976328. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:43:21,062][104569] Avg episode reward: [(0, '9258.632'), (1, '9264.763')] [2023-12-26 20:43:21,089][105692] Updated weights for policy 0, policy_version 736039 (0.0009) [2023-12-26 20:43:21,126][105620] Updated weights for policy 1, policy_version 736363 (0.0009) [2023-12-26 20:43:21,189][105620] Updated weights for policy 1, policy_version 736373 (0.0009) [2023-12-26 20:43:21,250][105620] Updated weights for policy 1, policy_version 736383 (0.0009) [2023-12-26 20:43:21,932][105692] Updated weights for policy 0, policy_version 736049 (0.0009) [2023-12-26 20:43:21,943][105620] Updated weights for policy 1, policy_version 736393 (0.0009) [2023-12-26 20:43:21,993][105692] Updated weights for policy 0, policy_version 736059 (0.0006) [2023-12-26 20:43:22,006][105620] Updated weights for policy 1, policy_version 736403 (0.0008) [2023-12-26 20:43:22,057][105692] Updated weights for policy 0, policy_version 736069 (0.0009) [2023-12-26 20:43:22,069][105620] Updated weights for policy 1, policy_version 736413 (0.0005) [2023-12-26 20:43:22,127][105620] Updated weights for policy 1, policy_version 736423 (0.0008) [2023-12-26 20:43:22,824][105620] Updated weights for policy 1, policy_version 736433 (0.0009) [2023-12-26 20:43:22,843][105692] Updated weights for policy 0, policy_version 736079 (0.0007) [2023-12-26 20:43:22,880][105620] Updated weights for policy 1, policy_version 736443 (0.0008) [2023-12-26 20:43:22,891][105692] Updated weights for policy 0, policy_version 736089 (0.0008) [2023-12-26 20:43:22,940][105692] Updated weights for policy 0, policy_version 736099 (0.0007) [2023-12-26 20:43:22,942][105620] Updated weights for policy 1, policy_version 736453 (0.0008) [2023-12-26 20:43:23,571][105620] Updated weights for policy 1, policy_version 736463 (0.0009) [2023-12-26 20:43:23,616][105620] Updated weights for policy 1, policy_version 736473 (0.0006) [2023-12-26 20:43:23,660][105620] Updated weights for policy 1, policy_version 736483 (0.0009) [2023-12-26 20:43:23,677][105692] Updated weights for policy 0, policy_version 736109 (0.0009) [2023-12-26 20:43:23,725][105692] Updated weights for policy 0, policy_version 736119 (0.0010) [2023-12-26 20:43:23,777][105692] Updated weights for policy 0, policy_version 736129 (0.0010) [2023-12-26 20:43:24,297][105620] Updated weights for policy 1, policy_version 736493 (0.0008) [2023-12-26 20:43:24,363][105620] Updated weights for policy 1, policy_version 736503 (0.0007) [2023-12-26 20:43:24,424][105620] Updated weights for policy 1, policy_version 736513 (0.0010) [2023-12-26 20:43:24,577][105692] Updated weights for policy 0, policy_version 736139 (0.0010) [2023-12-26 20:43:24,625][105692] Updated weights for policy 0, policy_version 736149 (0.0008) [2023-12-26 20:43:24,679][105692] Updated weights for policy 0, policy_version 736159 (0.0010) [2023-12-26 20:43:24,981][105620] Updated weights for policy 1, policy_version 736523 (0.0007) [2023-12-26 20:43:25,029][105620] Updated weights for policy 1, policy_version 736533 (0.0009) [2023-12-26 20:43:25,077][105620] Updated weights for policy 1, policy_version 736543 (0.0008) [2023-12-26 20:43:25,402][105692] Updated weights for policy 0, policy_version 736169 (0.0010) [2023-12-26 20:43:25,459][105692] Updated weights for policy 0, policy_version 736179 (0.0006) [2023-12-26 20:43:25,511][105692] Updated weights for policy 0, policy_version 736189 (0.0009) [2023-12-26 20:43:25,566][105692] Updated weights for policy 0, policy_version 736199 (0.0010) [2023-12-26 20:43:25,710][105620] Updated weights for policy 1, policy_version 736553 (0.0007) [2023-12-26 20:43:25,776][105620] Updated weights for policy 1, policy_version 736563 (0.0007) [2023-12-26 20:43:25,842][105620] Updated weights for policy 1, policy_version 736573 (0.0006) [2023-12-26 20:43:25,908][105620] Updated weights for policy 1, policy_version 736583 (0.0005) [2023-12-26 20:43:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19660.8). Total num frames: 377085952. Throughput: 0: 9698.5, 1: 9778.6. Samples: 377092640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:43:26,062][104569] Avg episode reward: [(0, '9170.336'), (1, '9355.801')] [2023-12-26 20:43:26,363][105692] Updated weights for policy 0, policy_version 736209 (0.0006) [2023-12-26 20:43:26,413][105692] Updated weights for policy 0, policy_version 736219 (0.0005) [2023-12-26 20:43:26,463][105692] Updated weights for policy 0, policy_version 736229 (0.0005) [2023-12-26 20:43:26,618][105620] Updated weights for policy 1, policy_version 736593 (0.0009) [2023-12-26 20:43:26,671][105620] Updated weights for policy 1, policy_version 736604 (0.0010) [2023-12-26 20:43:26,719][105620] Updated weights for policy 1, policy_version 736615 (0.0008) [2023-12-26 20:43:27,044][105692] Updated weights for policy 0, policy_version 736239 (0.0009) [2023-12-26 20:43:27,092][105692] Updated weights for policy 0, policy_version 736249 (0.0010) [2023-12-26 20:43:27,139][105692] Updated weights for policy 0, policy_version 736259 (0.0010) [2023-12-26 20:43:27,522][105620] Updated weights for policy 1, policy_version 736625 (0.0009) [2023-12-26 20:43:27,571][105620] Updated weights for policy 1, policy_version 736635 (0.0008) [2023-12-26 20:43:27,623][105620] Updated weights for policy 1, policy_version 736645 (0.0008) [2023-12-26 20:43:27,858][105692] Updated weights for policy 0, policy_version 736269 (0.0009) [2023-12-26 20:43:27,910][105692] Updated weights for policy 0, policy_version 736279 (0.0007) [2023-12-26 20:43:27,965][105692] Updated weights for policy 0, policy_version 736289 (0.0005) [2023-12-26 20:43:28,285][105620] Updated weights for policy 1, policy_version 736655 (0.0007) [2023-12-26 20:43:28,353][105620] Updated weights for policy 1, policy_version 736665 (0.0008) [2023-12-26 20:43:28,415][105620] Updated weights for policy 1, policy_version 736675 (0.0006) [2023-12-26 20:43:28,582][105692] Updated weights for policy 0, policy_version 736299 (0.0006) [2023-12-26 20:43:28,640][105692] Updated weights for policy 0, policy_version 736309 (0.0008) [2023-12-26 20:43:28,685][105692] Updated weights for policy 0, policy_version 736319 (0.0010) [2023-12-26 20:43:29,126][105620] Updated weights for policy 1, policy_version 736685 (0.0010) [2023-12-26 20:43:29,174][105620] Updated weights for policy 1, policy_version 736695 (0.0010) [2023-12-26 20:43:29,225][105620] Updated weights for policy 1, policy_version 736705 (0.0010) [2023-12-26 20:43:29,340][105692] Updated weights for policy 0, policy_version 736329 (0.0010) [2023-12-26 20:43:29,391][105692] Updated weights for policy 0, policy_version 736339 (0.0008) [2023-12-26 20:43:29,438][105692] Updated weights for policy 0, policy_version 736349 (0.0008) [2023-12-26 20:43:29,481][105692] Updated weights for policy 0, policy_version 736359 (0.0008) [2023-12-26 20:43:29,966][105620] Updated weights for policy 1, policy_version 736715 (0.0008) [2023-12-26 20:43:30,029][105620] Updated weights for policy 1, policy_version 736725 (0.0007) [2023-12-26 20:43:30,092][105620] Updated weights for policy 1, policy_version 736735 (0.0009) [2023-12-26 20:43:30,297][105692] Updated weights for policy 0, policy_version 736369 (0.0009) [2023-12-26 20:43:30,363][105692] Updated weights for policy 0, policy_version 736379 (0.0010) [2023-12-26 20:43:30,420][105692] Updated weights for policy 0, policy_version 736389 (0.0010) [2023-12-26 20:43:30,805][105620] Updated weights for policy 1, policy_version 736745 (0.0009) [2023-12-26 20:43:30,863][105620] Updated weights for policy 1, policy_version 736757 (0.0010) [2023-12-26 20:43:30,921][105620] Updated weights for policy 1, policy_version 736767 (0.0009) [2023-12-26 20:43:31,011][105692] Updated weights for policy 0, policy_version 736399 (0.0009) [2023-12-26 20:43:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 377184256. Throughput: 0: 9763.6, 1: 9802.7. Samples: 377152876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:43:31,063][104569] Avg episode reward: [(0, '8088.922'), (1, '9264.652')] [2023-12-26 20:43:31,069][105692] Updated weights for policy 0, policy_version 736409 (0.0007) [2023-12-26 20:43:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000736776_188637184.pth... [2023-12-26 20:43:31,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000735624_188342272.pth [2023-12-26 20:43:31,128][105692] Updated weights for policy 0, policy_version 736419 (0.0009) [2023-12-26 20:43:31,163][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000736424_188555264.pth... [2023-12-26 20:43:31,167][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000735240_188252160.pth [2023-12-26 20:43:31,666][105620] Updated weights for policy 1, policy_version 736777 (0.0006) [2023-12-26 20:43:31,717][105620] Updated weights for policy 1, policy_version 736787 (0.0010) [2023-12-26 20:43:31,772][105620] Updated weights for policy 1, policy_version 736797 (0.0006) [2023-12-26 20:43:31,822][105620] Updated weights for policy 1, policy_version 736807 (0.0007) [2023-12-26 20:43:31,945][105692] Updated weights for policy 0, policy_version 736429 (0.0009) [2023-12-26 20:43:32,000][105692] Updated weights for policy 0, policy_version 736439 (0.0010) [2023-12-26 20:43:32,069][105692] Updated weights for policy 0, policy_version 736449 (0.0011) [2023-12-26 20:43:32,457][105620] Updated weights for policy 1, policy_version 736817 (0.0006) [2023-12-26 20:43:32,514][105620] Updated weights for policy 1, policy_version 736827 (0.0005) [2023-12-26 20:43:32,571][105620] Updated weights for policy 1, policy_version 736837 (0.0006) [2023-12-26 20:43:32,819][105692] Updated weights for policy 0, policy_version 736459 (0.0011) [2023-12-26 20:43:32,874][105692] Updated weights for policy 0, policy_version 736469 (0.0010) [2023-12-26 20:43:32,929][105692] Updated weights for policy 0, policy_version 736479 (0.0010) [2023-12-26 20:43:33,145][105620] Updated weights for policy 1, policy_version 736847 (0.0007) [2023-12-26 20:43:33,197][105620] Updated weights for policy 1, policy_version 736857 (0.0006) [2023-12-26 20:43:33,256][105620] Updated weights for policy 1, policy_version 736867 (0.0005) [2023-12-26 20:43:33,667][105692] Updated weights for policy 0, policy_version 736489 (0.0010) [2023-12-26 20:43:33,734][105692] Updated weights for policy 0, policy_version 736499 (0.0011) [2023-12-26 20:43:33,778][105692] Updated weights for policy 0, policy_version 736509 (0.0005) [2023-12-26 20:43:33,851][105692] Updated weights for policy 0, policy_version 736519 (0.0006) [2023-12-26 20:43:33,859][105620] Updated weights for policy 1, policy_version 736877 (0.0007) [2023-12-26 20:43:33,911][105620] Updated weights for policy 1, policy_version 736888 (0.0010) [2023-12-26 20:43:33,964][105620] Updated weights for policy 1, policy_version 736898 (0.0009) [2023-12-26 20:43:34,425][105692] Updated weights for policy 0, policy_version 736529 (0.0005) [2023-12-26 20:43:34,482][105692] Updated weights for policy 0, policy_version 736539 (0.0005) [2023-12-26 20:43:34,529][105692] Updated weights for policy 0, policy_version 736549 (0.0005) [2023-12-26 20:43:34,789][105620] Updated weights for policy 1, policy_version 736909 (0.0009) [2023-12-26 20:43:34,856][105620] Updated weights for policy 1, policy_version 736919 (0.0009) [2023-12-26 20:43:34,927][105620] Updated weights for policy 1, policy_version 736929 (0.0009) [2023-12-26 20:43:35,093][105692] Updated weights for policy 0, policy_version 736559 (0.0008) [2023-12-26 20:43:35,145][105692] Updated weights for policy 0, policy_version 736569 (0.0010) [2023-12-26 20:43:35,207][105692] Updated weights for policy 0, policy_version 736579 (0.0011) [2023-12-26 20:43:35,597][105620] Updated weights for policy 1, policy_version 736939 (0.0010) [2023-12-26 20:43:35,648][105620] Updated weights for policy 1, policy_version 736949 (0.0007) [2023-12-26 20:43:35,700][105620] Updated weights for policy 1, policy_version 736959 (0.0008) [2023-12-26 20:43:35,959][105692] Updated weights for policy 0, policy_version 736589 (0.0010) [2023-12-26 20:43:36,007][105692] Updated weights for policy 0, policy_version 736599 (0.0010) [2023-12-26 20:43:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 377282560. Throughput: 0: 9815.1, 1: 9827.1. Samples: 377272564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:43:36,063][104569] Avg episode reward: [(0, '8110.136'), (1, '9174.257')] [2023-12-26 20:43:36,075][105692] Updated weights for policy 0, policy_version 736609 (0.0010) [2023-12-26 20:43:36,448][105620] Updated weights for policy 1, policy_version 736969 (0.0008) [2023-12-26 20:43:36,513][105620] Updated weights for policy 1, policy_version 736979 (0.0009) [2023-12-26 20:43:36,565][105620] Updated weights for policy 1, policy_version 736989 (0.0010) [2023-12-26 20:43:36,613][105620] Updated weights for policy 1, policy_version 736999 (0.0008) [2023-12-26 20:43:36,729][105692] Updated weights for policy 0, policy_version 736619 (0.0009) [2023-12-26 20:43:36,788][105692] Updated weights for policy 0, policy_version 736629 (0.0010) [2023-12-26 20:43:36,840][105692] Updated weights for policy 0, policy_version 736639 (0.0010) [2023-12-26 20:43:37,423][105692] Updated weights for policy 0, policy_version 736649 (0.0010) [2023-12-26 20:43:37,478][105692] Updated weights for policy 0, policy_version 736659 (0.0006) [2023-12-26 20:43:37,491][105620] Updated weights for policy 1, policy_version 737009 (0.0007) [2023-12-26 20:43:37,536][105692] Updated weights for policy 0, policy_version 736669 (0.0007) [2023-12-26 20:43:37,552][105620] Updated weights for policy 1, policy_version 737019 (0.0009) [2023-12-26 20:43:37,597][105692] Updated weights for policy 0, policy_version 736679 (0.0007) [2023-12-26 20:43:37,610][105620] Updated weights for policy 1, policy_version 737029 (0.0006) [2023-12-26 20:43:38,333][105692] Updated weights for policy 0, policy_version 736689 (0.0009) [2023-12-26 20:43:38,359][105620] Updated weights for policy 1, policy_version 737039 (0.0008) [2023-12-26 20:43:38,394][105692] Updated weights for policy 0, policy_version 736699 (0.0007) [2023-12-26 20:43:38,420][105620] Updated weights for policy 1, policy_version 737049 (0.0008) [2023-12-26 20:43:38,447][105692] Updated weights for policy 0, policy_version 736709 (0.0007) [2023-12-26 20:43:38,474][105620] Updated weights for policy 1, policy_version 737059 (0.0008) [2023-12-26 20:43:39,197][105620] Updated weights for policy 1, policy_version 737069 (0.0009) [2023-12-26 20:43:39,258][105692] Updated weights for policy 0, policy_version 736719 (0.0007) [2023-12-26 20:43:39,259][105620] Updated weights for policy 1, policy_version 737079 (0.0008) [2023-12-26 20:43:39,320][105620] Updated weights for policy 1, policy_version 737089 (0.0006) [2023-12-26 20:43:39,324][105692] Updated weights for policy 0, policy_version 736729 (0.0008) [2023-12-26 20:43:39,388][105692] Updated weights for policy 0, policy_version 736739 (0.0010) [2023-12-26 20:43:40,064][105620] Updated weights for policy 1, policy_version 737099 (0.0010) [2023-12-26 20:43:40,124][105620] Updated weights for policy 1, policy_version 737109 (0.0009) [2023-12-26 20:43:40,134][105692] Updated weights for policy 0, policy_version 736749 (0.0007) [2023-12-26 20:43:40,181][105620] Updated weights for policy 1, policy_version 737119 (0.0008) [2023-12-26 20:43:40,199][105692] Updated weights for policy 0, policy_version 736759 (0.0007) [2023-12-26 20:43:40,258][105692] Updated weights for policy 0, policy_version 736769 (0.0007) [2023-12-26 20:43:40,929][105620] Updated weights for policy 1, policy_version 737129 (0.0007) [2023-12-26 20:43:40,997][105620] Updated weights for policy 1, policy_version 737139 (0.0008) [2023-12-26 20:43:41,053][105692] Updated weights for policy 0, policy_version 736779 (0.0009) [2023-12-26 20:43:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 377372672. Throughput: 0: 9934.0, 1: 9699.2. Samples: 377387964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:43:41,063][104569] Avg episode reward: [(0, '8371.395'), (1, '9174.160')] [2023-12-26 20:43:41,067][105620] Updated weights for policy 1, policy_version 737149 (0.0007) [2023-12-26 20:43:41,116][105692] Updated weights for policy 0, policy_version 736789 (0.0009) [2023-12-26 20:43:41,119][105620] Updated weights for policy 1, policy_version 737159 (0.0006) [2023-12-26 20:43:41,189][105692] Updated weights for policy 0, policy_version 736799 (0.0008) [2023-12-26 20:43:41,903][105620] Updated weights for policy 1, policy_version 737169 (0.0008) [2023-12-26 20:43:41,963][105620] Updated weights for policy 1, policy_version 737179 (0.0009) [2023-12-26 20:43:41,990][105692] Updated weights for policy 0, policy_version 736809 (0.0009) [2023-12-26 20:43:42,021][105620] Updated weights for policy 1, policy_version 737189 (0.0006) [2023-12-26 20:43:42,039][105692] Updated weights for policy 0, policy_version 736819 (0.0011) [2023-12-26 20:43:42,092][105692] Updated weights for policy 0, policy_version 736829 (0.0010) [2023-12-26 20:43:42,138][105692] Updated weights for policy 0, policy_version 736839 (0.0010) [2023-12-26 20:43:42,719][105620] Updated weights for policy 1, policy_version 737199 (0.0007) [2023-12-26 20:43:42,786][105620] Updated weights for policy 1, policy_version 737209 (0.0009) [2023-12-26 20:43:42,844][105620] Updated weights for policy 1, policy_version 737219 (0.0008) [2023-12-26 20:43:42,941][105692] Updated weights for policy 0, policy_version 736849 (0.0006) [2023-12-26 20:43:43,001][105692] Updated weights for policy 0, policy_version 736859 (0.0005) [2023-12-26 20:43:43,056][105692] Updated weights for policy 0, policy_version 736869 (0.0005) [2023-12-26 20:43:43,578][105692] Updated weights for policy 0, policy_version 736879 (0.0005) [2023-12-26 20:43:43,628][105692] Updated weights for policy 0, policy_version 736889 (0.0009) [2023-12-26 20:43:43,657][105620] Updated weights for policy 1, policy_version 737229 (0.0007) [2023-12-26 20:43:43,690][105692] Updated weights for policy 0, policy_version 736899 (0.0008) [2023-12-26 20:43:43,712][105620] Updated weights for policy 1, policy_version 737239 (0.0006) [2023-12-26 20:43:43,762][105620] Updated weights for policy 1, policy_version 737249 (0.0007) [2023-12-26 20:43:44,284][105692] Updated weights for policy 0, policy_version 736909 (0.0008) [2023-12-26 20:43:44,348][105692] Updated weights for policy 0, policy_version 736919 (0.0007) [2023-12-26 20:43:44,420][105692] Updated weights for policy 0, policy_version 736929 (0.0008) [2023-12-26 20:43:44,539][105620] Updated weights for policy 1, policy_version 737259 (0.0009) [2023-12-26 20:43:44,605][105620] Updated weights for policy 1, policy_version 737269 (0.0008) [2023-12-26 20:43:44,666][105620] Updated weights for policy 1, policy_version 737279 (0.0008) [2023-12-26 20:43:45,013][105692] Updated weights for policy 0, policy_version 736939 (0.0008) [2023-12-26 20:43:45,076][105692] Updated weights for policy 0, policy_version 736949 (0.0008) [2023-12-26 20:43:45,136][105692] Updated weights for policy 0, policy_version 736959 (0.0008) [2023-12-26 20:43:45,375][105620] Updated weights for policy 1, policy_version 737289 (0.0008) [2023-12-26 20:43:45,430][105620] Updated weights for policy 1, policy_version 737299 (0.0005) [2023-12-26 20:43:45,487][105620] Updated weights for policy 1, policy_version 737309 (0.0007) [2023-12-26 20:43:45,539][105620] Updated weights for policy 1, policy_version 737319 (0.0006) [2023-12-26 20:43:45,815][105692] Updated weights for policy 0, policy_version 736969 (0.0008) [2023-12-26 20:43:45,864][105692] Updated weights for policy 0, policy_version 736979 (0.0010) [2023-12-26 20:43:45,912][105692] Updated weights for policy 0, policy_version 736989 (0.0010) [2023-12-26 20:43:45,960][105692] Updated weights for policy 0, policy_version 736999 (0.0010) [2023-12-26 20:43:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 377479168. Throughput: 0: 9892.4, 1: 9668.2. Samples: 377443964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:43:46,063][104569] Avg episode reward: [(0, '8809.718'), (1, '9355.578')] [2023-12-26 20:43:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000737000_188702720.pth... [2023-12-26 20:43:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000737320_188776448.pth... [2023-12-26 20:43:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000736200_188489728.pth [2023-12-26 20:43:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000735848_188407808.pth [2023-12-26 20:43:46,160][105620] Updated weights for policy 1, policy_version 737329 (0.0007) [2023-12-26 20:43:46,220][105620] Updated weights for policy 1, policy_version 737339 (0.0005) [2023-12-26 20:43:46,280][105620] Updated weights for policy 1, policy_version 737349 (0.0008) [2023-12-26 20:43:46,716][105692] Updated weights for policy 0, policy_version 737009 (0.0010) [2023-12-26 20:43:46,764][105692] Updated weights for policy 0, policy_version 737019 (0.0010) [2023-12-26 20:43:46,815][105692] Updated weights for policy 0, policy_version 737029 (0.0010) [2023-12-26 20:43:46,993][105620] Updated weights for policy 1, policy_version 737359 (0.0008) [2023-12-26 20:43:47,036][105620] Updated weights for policy 1, policy_version 737369 (0.0007) [2023-12-26 20:43:47,080][105620] Updated weights for policy 1, policy_version 737379 (0.0008) [2023-12-26 20:43:47,576][105692] Updated weights for policy 0, policy_version 737039 (0.0010) [2023-12-26 20:43:47,638][105692] Updated weights for policy 0, policy_version 737049 (0.0010) [2023-12-26 20:43:47,702][105692] Updated weights for policy 0, policy_version 737059 (0.0011) [2023-12-26 20:43:47,890][105620] Updated weights for policy 1, policy_version 737389 (0.0008) [2023-12-26 20:43:47,949][105620] Updated weights for policy 1, policy_version 737399 (0.0008) [2023-12-26 20:43:48,017][105620] Updated weights for policy 1, policy_version 737409 (0.0008) [2023-12-26 20:43:48,372][105692] Updated weights for policy 0, policy_version 737069 (0.0008) [2023-12-26 20:43:48,426][105692] Updated weights for policy 0, policy_version 737079 (0.0008) [2023-12-26 20:43:48,481][105692] Updated weights for policy 0, policy_version 737089 (0.0009) [2023-12-26 20:43:48,742][105620] Updated weights for policy 1, policy_version 737419 (0.0008) [2023-12-26 20:43:48,810][105620] Updated weights for policy 1, policy_version 737429 (0.0006) [2023-12-26 20:43:48,886][105620] Updated weights for policy 1, policy_version 737439 (0.0006) [2023-12-26 20:43:49,210][105692] Updated weights for policy 0, policy_version 737099 (0.0011) [2023-12-26 20:43:49,278][105692] Updated weights for policy 0, policy_version 737109 (0.0011) [2023-12-26 20:43:49,338][105692] Updated weights for policy 0, policy_version 737119 (0.0011) [2023-12-26 20:43:49,563][105620] Updated weights for policy 1, policy_version 737449 (0.0007) [2023-12-26 20:43:49,626][105620] Updated weights for policy 1, policy_version 737459 (0.0005) [2023-12-26 20:43:49,674][105620] Updated weights for policy 1, policy_version 737469 (0.0005) [2023-12-26 20:43:49,723][105620] Updated weights for policy 1, policy_version 737479 (0.0006) [2023-12-26 20:43:49,970][105692] Updated weights for policy 0, policy_version 737129 (0.0008) [2023-12-26 20:43:50,031][105692] Updated weights for policy 0, policy_version 737139 (0.0009) [2023-12-26 20:43:50,084][105692] Updated weights for policy 0, policy_version 737149 (0.0009) [2023-12-26 20:43:50,139][105692] Updated weights for policy 0, policy_version 737159 (0.0009) [2023-12-26 20:43:50,440][105620] Updated weights for policy 1, policy_version 737489 (0.0008) [2023-12-26 20:43:50,499][105620] Updated weights for policy 1, policy_version 737499 (0.0008) [2023-12-26 20:43:50,555][105620] Updated weights for policy 1, policy_version 737509 (0.0009) [2023-12-26 20:43:50,862][105692] Updated weights for policy 0, policy_version 737169 (0.0010) [2023-12-26 20:43:50,926][105692] Updated weights for policy 0, policy_version 737179 (0.0008) [2023-12-26 20:43:50,988][105692] Updated weights for policy 0, policy_version 737189 (0.0011) [2023-12-26 20:43:51,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 377577472. Throughput: 0: 9881.3, 1: 9665.7. Samples: 377563760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:43:51,062][104569] Avg episode reward: [(0, '9260.031'), (1, '9264.385')] [2023-12-26 20:43:51,307][105620] Updated weights for policy 1, policy_version 737519 (0.0008) [2023-12-26 20:43:51,368][105620] Updated weights for policy 1, policy_version 737529 (0.0008) [2023-12-26 20:43:51,441][105620] Updated weights for policy 1, policy_version 737539 (0.0009) [2023-12-26 20:43:51,700][105692] Updated weights for policy 0, policy_version 737199 (0.0008) [2023-12-26 20:43:51,761][105692] Updated weights for policy 0, policy_version 737209 (0.0008) [2023-12-26 20:43:51,813][105692] Updated weights for policy 0, policy_version 737219 (0.0008) [2023-12-26 20:43:52,292][105620] Updated weights for policy 1, policy_version 737549 (0.0009) [2023-12-26 20:43:52,357][105620] Updated weights for policy 1, policy_version 737559 (0.0010) [2023-12-26 20:43:52,421][105620] Updated weights for policy 1, policy_version 737569 (0.0007) [2023-12-26 20:43:52,483][105692] Updated weights for policy 0, policy_version 737229 (0.0009) [2023-12-26 20:43:52,548][105692] Updated weights for policy 0, policy_version 737239 (0.0009) [2023-12-26 20:43:52,616][105692] Updated weights for policy 0, policy_version 737249 (0.0009) [2023-12-26 20:43:53,128][105620] Updated weights for policy 1, policy_version 737579 (0.0009) [2023-12-26 20:43:53,195][105620] Updated weights for policy 1, policy_version 737589 (0.0007) [2023-12-26 20:43:53,255][105620] Updated weights for policy 1, policy_version 737599 (0.0009) [2023-12-26 20:43:53,333][105692] Updated weights for policy 0, policy_version 737259 (0.0009) [2023-12-26 20:43:53,382][105692] Updated weights for policy 0, policy_version 737269 (0.0009) [2023-12-26 20:43:53,429][105692] Updated weights for policy 0, policy_version 737279 (0.0008) [2023-12-26 20:43:53,927][105620] Updated weights for policy 1, policy_version 737609 (0.0009) [2023-12-26 20:43:53,984][105620] Updated weights for policy 1, policy_version 737619 (0.0005) [2023-12-26 20:43:54,036][105620] Updated weights for policy 1, policy_version 737629 (0.0005) [2023-12-26 20:43:54,084][105620] Updated weights for policy 1, policy_version 737639 (0.0005) [2023-12-26 20:43:54,200][105692] Updated weights for policy 0, policy_version 737289 (0.0009) [2023-12-26 20:43:54,262][105692] Updated weights for policy 0, policy_version 737299 (0.0009) [2023-12-26 20:43:54,313][105692] Updated weights for policy 0, policy_version 737309 (0.0009) [2023-12-26 20:43:54,365][105692] Updated weights for policy 0, policy_version 737319 (0.0009) [2023-12-26 20:43:54,653][105620] Updated weights for policy 1, policy_version 737649 (0.0009) [2023-12-26 20:43:54,707][105620] Updated weights for policy 1, policy_version 737659 (0.0010) [2023-12-26 20:43:54,763][105620] Updated weights for policy 1, policy_version 737670 (0.0009) [2023-12-26 20:43:55,054][105692] Updated weights for policy 0, policy_version 737329 (0.0006) [2023-12-26 20:43:55,118][105692] Updated weights for policy 0, policy_version 737339 (0.0006) [2023-12-26 20:43:55,175][105692] Updated weights for policy 0, policy_version 737349 (0.0006) [2023-12-26 20:43:55,603][105620] Updated weights for policy 1, policy_version 737680 (0.0007) [2023-12-26 20:43:55,654][105620] Updated weights for policy 1, policy_version 737690 (0.0007) [2023-12-26 20:43:55,712][105620] Updated weights for policy 1, policy_version 737700 (0.0008) [2023-12-26 20:43:55,748][105692] Updated weights for policy 0, policy_version 737359 (0.0007) [2023-12-26 20:43:55,809][105692] Updated weights for policy 0, policy_version 737369 (0.0008) [2023-12-26 20:43:55,879][105692] Updated weights for policy 0, policy_version 737379 (0.0005) [2023-12-26 20:43:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 377675776. Throughput: 0: 9914.6, 1: 9605.2. Samples: 377682012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:43:56,063][104569] Avg episode reward: [(0, '9259.814'), (1, '9355.667')] [2023-12-26 20:43:56,298][105620] Updated weights for policy 1, policy_version 737710 (0.0007) [2023-12-26 20:43:56,361][105620] Updated weights for policy 1, policy_version 737720 (0.0005) [2023-12-26 20:43:56,421][105620] Updated weights for policy 1, policy_version 737730 (0.0006) [2023-12-26 20:43:56,436][105692] Updated weights for policy 0, policy_version 737389 (0.0006) [2023-12-26 20:43:56,487][105692] Updated weights for policy 0, policy_version 737399 (0.0006) [2023-12-26 20:43:56,531][105692] Updated weights for policy 0, policy_version 737409 (0.0005) [2023-12-26 20:43:57,075][105692] Updated weights for policy 0, policy_version 737419 (0.0006) [2023-12-26 20:43:57,116][105620] Updated weights for policy 1, policy_version 737740 (0.0005) [2023-12-26 20:43:57,137][105692] Updated weights for policy 0, policy_version 737429 (0.0006) [2023-12-26 20:43:57,179][105620] Updated weights for policy 1, policy_version 737750 (0.0006) [2023-12-26 20:43:57,206][105692] Updated weights for policy 0, policy_version 737439 (0.0005) [2023-12-26 20:43:57,231][105620] Updated weights for policy 1, policy_version 737760 (0.0007) [2023-12-26 20:43:57,826][105692] Updated weights for policy 0, policy_version 737449 (0.0005) [2023-12-26 20:43:57,880][105620] Updated weights for policy 1, policy_version 737770 (0.0006) [2023-12-26 20:43:57,881][105692] Updated weights for policy 0, policy_version 737459 (0.0005) [2023-12-26 20:43:57,926][105620] Updated weights for policy 1, policy_version 737780 (0.0008) [2023-12-26 20:43:57,928][105692] Updated weights for policy 0, policy_version 737469 (0.0005) [2023-12-26 20:43:57,979][105620] Updated weights for policy 1, policy_version 737790 (0.0009) [2023-12-26 20:43:57,988][105692] Updated weights for policy 0, policy_version 737479 (0.0007) [2023-12-26 20:43:58,040][105620] Updated weights for policy 1, policy_version 737800 (0.0008) [2023-12-26 20:43:58,776][105692] Updated weights for policy 0, policy_version 737489 (0.0007) [2023-12-26 20:43:58,830][105620] Updated weights for policy 1, policy_version 737810 (0.0008) [2023-12-26 20:43:58,845][105692] Updated weights for policy 0, policy_version 737499 (0.0009) [2023-12-26 20:43:58,899][105620] Updated weights for policy 1, policy_version 737820 (0.0009) [2023-12-26 20:43:58,918][105692] Updated weights for policy 0, policy_version 737509 (0.0008) [2023-12-26 20:43:58,961][105620] Updated weights for policy 1, policy_version 737830 (0.0007) [2023-12-26 20:43:59,723][105620] Updated weights for policy 1, policy_version 737840 (0.0007) [2023-12-26 20:43:59,736][105692] Updated weights for policy 0, policy_version 737519 (0.0008) [2023-12-26 20:43:59,786][105620] Updated weights for policy 1, policy_version 737850 (0.0009) [2023-12-26 20:43:59,788][105692] Updated weights for policy 0, policy_version 737529 (0.0007) [2023-12-26 20:43:59,848][105620] Updated weights for policy 1, policy_version 737860 (0.0008) [2023-12-26 20:43:59,851][105692] Updated weights for policy 0, policy_version 737539 (0.0006) [2023-12-26 20:44:00,575][105620] Updated weights for policy 1, policy_version 737870 (0.0010) [2023-12-26 20:44:00,588][105692] Updated weights for policy 0, policy_version 737549 (0.0006) [2023-12-26 20:44:00,651][105692] Updated weights for policy 0, policy_version 737559 (0.0008) [2023-12-26 20:44:00,654][105620] Updated weights for policy 1, policy_version 737880 (0.0008) [2023-12-26 20:44:00,699][105692] Updated weights for policy 0, policy_version 737569 (0.0009) [2023-12-26 20:44:00,710][105620] Updated weights for policy 1, policy_version 737890 (0.0009) [2023-12-26 20:44:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 377774080. Throughput: 0: 9963.0, 1: 9671.9. Samples: 377744660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:44:01,062][104569] Avg episode reward: [(0, '9257.893'), (1, '9174.921')] [2023-12-26 20:44:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000737576_188850176.pth... [2023-12-26 20:44:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000737896_188923904.pth... [2023-12-26 20:44:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000736424_188555264.pth [2023-12-26 20:44:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000736776_188637184.pth [2023-12-26 20:44:01,355][105692] Updated weights for policy 0, policy_version 737579 (0.0009) [2023-12-26 20:44:01,366][105620] Updated weights for policy 1, policy_version 737900 (0.0009) [2023-12-26 20:44:01,419][105692] Updated weights for policy 0, policy_version 737589 (0.0007) [2023-12-26 20:44:01,430][105620] Updated weights for policy 1, policy_version 737910 (0.0007) [2023-12-26 20:44:01,484][105692] Updated weights for policy 0, policy_version 737599 (0.0005) [2023-12-26 20:44:01,490][105620] Updated weights for policy 1, policy_version 737920 (0.0006) [2023-12-26 20:44:02,212][105692] Updated weights for policy 0, policy_version 737609 (0.0007) [2023-12-26 20:44:02,227][105620] Updated weights for policy 1, policy_version 737930 (0.0005) [2023-12-26 20:44:02,266][105692] Updated weights for policy 0, policy_version 737619 (0.0007) [2023-12-26 20:44:02,293][105620] Updated weights for policy 1, policy_version 737940 (0.0007) [2023-12-26 20:44:02,322][105692] Updated weights for policy 0, policy_version 737629 (0.0006) [2023-12-26 20:44:02,359][105620] Updated weights for policy 1, policy_version 737950 (0.0006) [2023-12-26 20:44:02,384][105692] Updated weights for policy 0, policy_version 737639 (0.0012) [2023-12-26 20:44:02,424][105620] Updated weights for policy 1, policy_version 737960 (0.0007) [2023-12-26 20:44:03,018][105692] Updated weights for policy 0, policy_version 737649 (0.0005) [2023-12-26 20:44:03,072][105692] Updated weights for policy 0, policy_version 737659 (0.0008) [2023-12-26 20:44:03,081][105620] Updated weights for policy 1, policy_version 737970 (0.0006) [2023-12-26 20:44:03,127][105620] Updated weights for policy 1, policy_version 737980 (0.0008) [2023-12-26 20:44:03,129][105692] Updated weights for policy 0, policy_version 737669 (0.0010) [2023-12-26 20:44:03,173][105620] Updated weights for policy 1, policy_version 737990 (0.0007) [2023-12-26 20:44:03,677][105692] Updated weights for policy 0, policy_version 737679 (0.0007) [2023-12-26 20:44:03,741][105692] Updated weights for policy 0, policy_version 737689 (0.0006) [2023-12-26 20:44:03,792][105692] Updated weights for policy 0, policy_version 737699 (0.0005) [2023-12-26 20:44:03,797][105620] Updated weights for policy 1, policy_version 738000 (0.0008) [2023-12-26 20:44:03,861][105620] Updated weights for policy 1, policy_version 738010 (0.0007) [2023-12-26 20:44:03,921][105620] Updated weights for policy 1, policy_version 738020 (0.0008) [2023-12-26 20:44:04,444][105692] Updated weights for policy 0, policy_version 737709 (0.0008) [2023-12-26 20:44:04,502][105692] Updated weights for policy 0, policy_version 737719 (0.0010) [2023-12-26 20:44:04,554][105692] Updated weights for policy 0, policy_version 737729 (0.0010) [2023-12-26 20:44:04,708][105620] Updated weights for policy 1, policy_version 738030 (0.0008) [2023-12-26 20:44:04,770][105620] Updated weights for policy 1, policy_version 738040 (0.0008) [2023-12-26 20:44:04,833][105620] Updated weights for policy 1, policy_version 738050 (0.0008) [2023-12-26 20:44:05,304][105692] Updated weights for policy 0, policy_version 737739 (0.0010) [2023-12-26 20:44:05,359][105692] Updated weights for policy 0, policy_version 737749 (0.0010) [2023-12-26 20:44:05,411][105692] Updated weights for policy 0, policy_version 737759 (0.0010) [2023-12-26 20:44:05,419][105620] Updated weights for policy 1, policy_version 738060 (0.0007) [2023-12-26 20:44:05,475][105620] Updated weights for policy 1, policy_version 738070 (0.0005) [2023-12-26 20:44:05,530][105620] Updated weights for policy 1, policy_version 738080 (0.0007) [2023-12-26 20:44:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.7, 300 sec: 19605.2). Total num frames: 377872384. Throughput: 0: 9950.1, 1: 9757.1. Samples: 377863156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:44:06,063][104569] Avg episode reward: [(0, '9078.641'), (1, '9174.949')] [2023-12-26 20:44:06,162][105692] Updated weights for policy 0, policy_version 737769 (0.0010) [2023-12-26 20:44:06,191][105620] Updated weights for policy 1, policy_version 738090 (0.0007) [2023-12-26 20:44:06,229][105692] Updated weights for policy 0, policy_version 737779 (0.0012) [2023-12-26 20:44:06,255][105620] Updated weights for policy 1, policy_version 738100 (0.0007) [2023-12-26 20:44:06,286][105692] Updated weights for policy 0, policy_version 737789 (0.0011) [2023-12-26 20:44:06,305][105620] Updated weights for policy 1, policy_version 738110 (0.0005) [2023-12-26 20:44:06,347][105692] Updated weights for policy 0, policy_version 737799 (0.0011) [2023-12-26 20:44:06,353][105620] Updated weights for policy 1, policy_version 738120 (0.0006) [2023-12-26 20:44:07,049][105620] Updated weights for policy 1, policy_version 738130 (0.0009) [2023-12-26 20:44:07,093][105692] Updated weights for policy 0, policy_version 737809 (0.0011) [2023-12-26 20:44:07,106][105620] Updated weights for policy 1, policy_version 738140 (0.0007) [2023-12-26 20:44:07,152][105692] Updated weights for policy 0, policy_version 737819 (0.0011) [2023-12-26 20:44:07,162][105620] Updated weights for policy 1, policy_version 738150 (0.0005) [2023-12-26 20:44:07,215][105692] Updated weights for policy 0, policy_version 737829 (0.0011) [2023-12-26 20:44:07,855][105692] Updated weights for policy 0, policy_version 737839 (0.0010) [2023-12-26 20:44:07,916][105692] Updated weights for policy 0, policy_version 737849 (0.0010) [2023-12-26 20:44:07,961][105620] Updated weights for policy 1, policy_version 738160 (0.0006) [2023-12-26 20:44:07,975][105692] Updated weights for policy 0, policy_version 737859 (0.0010) [2023-12-26 20:44:08,010][105620] Updated weights for policy 1, policy_version 738170 (0.0009) [2023-12-26 20:44:08,061][105620] Updated weights for policy 1, policy_version 738180 (0.0008) [2023-12-26 20:44:08,719][105692] Updated weights for policy 0, policy_version 737869 (0.0011) [2023-12-26 20:44:08,774][105692] Updated weights for policy 0, policy_version 737879 (0.0010) [2023-12-26 20:44:08,810][105620] Updated weights for policy 1, policy_version 738190 (0.0008) [2023-12-26 20:44:08,834][105692] Updated weights for policy 0, policy_version 737889 (0.0009) [2023-12-26 20:44:08,860][105620] Updated weights for policy 1, policy_version 738200 (0.0008) [2023-12-26 20:44:08,914][105620] Updated weights for policy 1, policy_version 738210 (0.0007) [2023-12-26 20:44:09,596][105692] Updated weights for policy 0, policy_version 737899 (0.0008) [2023-12-26 20:44:09,659][105692] Updated weights for policy 0, policy_version 737909 (0.0009) [2023-12-26 20:44:09,721][105692] Updated weights for policy 0, policy_version 737919 (0.0009) [2023-12-26 20:44:09,728][105620] Updated weights for policy 1, policy_version 738220 (0.0007) [2023-12-26 20:44:09,778][105620] Updated weights for policy 1, policy_version 738230 (0.0008) [2023-12-26 20:44:09,845][105620] Updated weights for policy 1, policy_version 738240 (0.0008) [2023-12-26 20:44:10,465][105692] Updated weights for policy 0, policy_version 737929 (0.0009) [2023-12-26 20:44:10,527][105692] Updated weights for policy 0, policy_version 737939 (0.0009) [2023-12-26 20:44:10,587][105692] Updated weights for policy 0, policy_version 737949 (0.0009) [2023-12-26 20:44:10,620][105620] Updated weights for policy 1, policy_version 738250 (0.0009) [2023-12-26 20:44:10,649][105692] Updated weights for policy 0, policy_version 737959 (0.0006) [2023-12-26 20:44:10,669][105620] Updated weights for policy 1, policy_version 738260 (0.0008) [2023-12-26 20:44:10,719][105620] Updated weights for policy 1, policy_version 738270 (0.0006) [2023-12-26 20:44:10,771][105620] Updated weights for policy 1, policy_version 738280 (0.0006) [2023-12-26 20:44:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 377970688. Throughput: 0: 10005.0, 1: 9674.1. Samples: 377978200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:44:11,062][104569] Avg episode reward: [(0, '9258.863'), (1, '9178.176')] [2023-12-26 20:44:11,452][105620] Updated weights for policy 1, policy_version 738290 (0.0008) [2023-12-26 20:44:11,511][105692] Updated weights for policy 0, policy_version 737969 (0.0006) [2023-12-26 20:44:11,517][105620] Updated weights for policy 1, policy_version 738300 (0.0008) [2023-12-26 20:44:11,568][105692] Updated weights for policy 0, policy_version 737979 (0.0007) [2023-12-26 20:44:11,577][105620] Updated weights for policy 1, policy_version 738310 (0.0007) [2023-12-26 20:44:11,624][105692] Updated weights for policy 0, policy_version 737989 (0.0008) [2023-12-26 20:44:12,270][105620] Updated weights for policy 1, policy_version 738320 (0.0009) [2023-12-26 20:44:12,327][105620] Updated weights for policy 1, policy_version 738330 (0.0008) [2023-12-26 20:44:12,393][105620] Updated weights for policy 1, policy_version 738340 (0.0009) [2023-12-26 20:44:12,434][105692] Updated weights for policy 0, policy_version 737999 (0.0007) [2023-12-26 20:44:12,493][105692] Updated weights for policy 0, policy_version 738009 (0.0009) [2023-12-26 20:44:12,555][105692] Updated weights for policy 0, policy_version 738019 (0.0008) [2023-12-26 20:44:13,194][105620] Updated weights for policy 1, policy_version 738350 (0.0009) [2023-12-26 20:44:13,216][105692] Updated weights for policy 0, policy_version 738029 (0.0008) [2023-12-26 20:44:13,248][105620] Updated weights for policy 1, policy_version 738360 (0.0006) [2023-12-26 20:44:13,273][105692] Updated weights for policy 0, policy_version 738039 (0.0008) [2023-12-26 20:44:13,300][105620] Updated weights for policy 1, policy_version 738370 (0.0006) [2023-12-26 20:44:13,331][105692] Updated weights for policy 0, policy_version 738049 (0.0009) [2023-12-26 20:44:13,916][105692] Updated weights for policy 0, policy_version 738059 (0.0006) [2023-12-26 20:44:13,966][105692] Updated weights for policy 0, policy_version 738069 (0.0005) [2023-12-26 20:44:14,015][105692] Updated weights for policy 0, policy_version 738079 (0.0005) [2023-12-26 20:44:14,142][105620] Updated weights for policy 1, policy_version 738380 (0.0006) [2023-12-26 20:44:14,192][105620] Updated weights for policy 1, policy_version 738390 (0.0009) [2023-12-26 20:44:14,254][105620] Updated weights for policy 1, policy_version 738400 (0.0009) [2023-12-26 20:44:14,667][105692] Updated weights for policy 0, policy_version 738089 (0.0006) [2023-12-26 20:44:14,718][105692] Updated weights for policy 0, policy_version 738099 (0.0009) [2023-12-26 20:44:14,778][105692] Updated weights for policy 0, policy_version 738109 (0.0008) [2023-12-26 20:44:14,840][105692] Updated weights for policy 0, policy_version 738119 (0.0008) [2023-12-26 20:44:15,006][105620] Updated weights for policy 1, policy_version 738410 (0.0009) [2023-12-26 20:44:15,057][105620] Updated weights for policy 1, policy_version 738420 (0.0009) [2023-12-26 20:44:15,105][105620] Updated weights for policy 1, policy_version 738430 (0.0009) [2023-12-26 20:44:15,163][105620] Updated weights for policy 1, policy_version 738440 (0.0008) [2023-12-26 20:44:15,601][105692] Updated weights for policy 0, policy_version 738129 (0.0009) [2023-12-26 20:44:15,656][105692] Updated weights for policy 0, policy_version 738139 (0.0009) [2023-12-26 20:44:15,705][105692] Updated weights for policy 0, policy_version 738149 (0.0008) [2023-12-26 20:44:15,934][105620] Updated weights for policy 1, policy_version 738450 (0.0008) [2023-12-26 20:44:15,981][105620] Updated weights for policy 1, policy_version 738460 (0.0009) [2023-12-26 20:44:16,027][105620] Updated weights for policy 1, policy_version 738470 (0.0008) [2023-12-26 20:44:16,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 378068992. Throughput: 0: 9927.8, 1: 9648.8. Samples: 378033824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:44:16,062][104569] Avg episode reward: [(0, '9348.376'), (1, '9266.685')] [2023-12-26 20:44:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000738472_189071360.pth... [2023-12-26 20:44:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000738152_188997632.pth... [2023-12-26 20:44:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000737000_188702720.pth [2023-12-26 20:44:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000737320_188776448.pth [2023-12-26 20:44:16,482][105692] Updated weights for policy 0, policy_version 738159 (0.0006) [2023-12-26 20:44:16,537][105692] Updated weights for policy 0, policy_version 738169 (0.0007) [2023-12-26 20:44:16,597][105692] Updated weights for policy 0, policy_version 738180 (0.0010) [2023-12-26 20:44:16,706][105620] Updated weights for policy 1, policy_version 738480 (0.0008) [2023-12-26 20:44:16,770][105620] Updated weights for policy 1, policy_version 738490 (0.0009) [2023-12-26 20:44:16,837][105620] Updated weights for policy 1, policy_version 738500 (0.0010) [2023-12-26 20:44:17,221][105692] Updated weights for policy 0, policy_version 738190 (0.0010) [2023-12-26 20:44:17,276][105692] Updated weights for policy 0, policy_version 738200 (0.0009) [2023-12-26 20:44:17,323][105692] Updated weights for policy 0, policy_version 738210 (0.0008) [2023-12-26 20:44:17,631][105620] Updated weights for policy 1, policy_version 738510 (0.0010) [2023-12-26 20:44:17,697][105620] Updated weights for policy 1, policy_version 738520 (0.0010) [2023-12-26 20:44:17,750][105620] Updated weights for policy 1, policy_version 738530 (0.0010) [2023-12-26 20:44:17,965][105692] Updated weights for policy 0, policy_version 738220 (0.0008) [2023-12-26 20:44:18,015][105692] Updated weights for policy 0, policy_version 738230 (0.0009) [2023-12-26 20:44:18,066][105692] Updated weights for policy 0, policy_version 738240 (0.0006) [2023-12-26 20:44:18,583][105620] Updated weights for policy 1, policy_version 738540 (0.0009) [2023-12-26 20:44:18,655][105620] Updated weights for policy 1, policy_version 738550 (0.0010) [2023-12-26 20:44:18,718][105620] Updated weights for policy 1, policy_version 738560 (0.0008) [2023-12-26 20:44:18,742][105692] Updated weights for policy 0, policy_version 738250 (0.0006) [2023-12-26 20:44:18,795][105692] Updated weights for policy 0, policy_version 738260 (0.0011) [2023-12-26 20:44:18,844][105692] Updated weights for policy 0, policy_version 738270 (0.0010) [2023-12-26 20:44:18,900][105692] Updated weights for policy 0, policy_version 738280 (0.0010) [2023-12-26 20:44:19,428][105620] Updated weights for policy 1, policy_version 738570 (0.0006) [2023-12-26 20:44:19,487][105620] Updated weights for policy 1, policy_version 738580 (0.0008) [2023-12-26 20:44:19,555][105620] Updated weights for policy 1, policy_version 738590 (0.0008) [2023-12-26 20:44:19,623][105620] Updated weights for policy 1, policy_version 738600 (0.0008) [2023-12-26 20:44:19,653][105692] Updated weights for policy 0, policy_version 738290 (0.0008) [2023-12-26 20:44:19,717][105692] Updated weights for policy 0, policy_version 738300 (0.0009) [2023-12-26 20:44:19,784][105692] Updated weights for policy 0, policy_version 738310 (0.0009) [2023-12-26 20:44:20,309][105620] Updated weights for policy 1, policy_version 738610 (0.0008) [2023-12-26 20:44:20,364][105620] Updated weights for policy 1, policy_version 738620 (0.0009) [2023-12-26 20:44:20,417][105620] Updated weights for policy 1, policy_version 738630 (0.0008) [2023-12-26 20:44:20,602][105692] Updated weights for policy 0, policy_version 738320 (0.0008) [2023-12-26 20:44:20,662][105692] Updated weights for policy 0, policy_version 738330 (0.0008) [2023-12-26 20:44:20,725][105692] Updated weights for policy 0, policy_version 738340 (0.0008) [2023-12-26 20:44:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 378159104. Throughput: 0: 9969.6, 1: 9541.0. Samples: 378150540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:44:21,062][104569] Avg episode reward: [(0, '9348.179'), (1, '9355.860')] [2023-12-26 20:44:21,115][105620] Updated weights for policy 1, policy_version 738640 (0.0007) [2023-12-26 20:44:21,175][105620] Updated weights for policy 1, policy_version 738650 (0.0008) [2023-12-26 20:44:21,241][105620] Updated weights for policy 1, policy_version 738660 (0.0008) [2023-12-26 20:44:21,549][105692] Updated weights for policy 0, policy_version 738350 (0.0010) [2023-12-26 20:44:21,601][105692] Updated weights for policy 0, policy_version 738360 (0.0010) [2023-12-26 20:44:21,666][105692] Updated weights for policy 0, policy_version 738370 (0.0010) [2023-12-26 20:44:21,952][105620] Updated weights for policy 1, policy_version 738670 (0.0009) [2023-12-26 20:44:22,014][105620] Updated weights for policy 1, policy_version 738680 (0.0008) [2023-12-26 20:44:22,068][105620] Updated weights for policy 1, policy_version 738690 (0.0008) [2023-12-26 20:44:22,446][105692] Updated weights for policy 0, policy_version 738380 (0.0010) [2023-12-26 20:44:22,501][105692] Updated weights for policy 0, policy_version 738390 (0.0010) [2023-12-26 20:44:22,557][105692] Updated weights for policy 0, policy_version 738400 (0.0010) [2023-12-26 20:44:22,869][105620] Updated weights for policy 1, policy_version 738700 (0.0008) [2023-12-26 20:44:22,924][105620] Updated weights for policy 1, policy_version 738710 (0.0009) [2023-12-26 20:44:22,974][105620] Updated weights for policy 1, policy_version 738720 (0.0008) [2023-12-26 20:44:23,261][105692] Updated weights for policy 0, policy_version 738410 (0.0010) [2023-12-26 20:44:23,323][105692] Updated weights for policy 0, policy_version 738420 (0.0010) [2023-12-26 20:44:23,372][105692] Updated weights for policy 0, policy_version 738430 (0.0008) [2023-12-26 20:44:23,421][105692] Updated weights for policy 0, policy_version 738440 (0.0008) [2023-12-26 20:44:23,749][105620] Updated weights for policy 1, policy_version 738730 (0.0007) [2023-12-26 20:44:23,810][105620] Updated weights for policy 1, policy_version 738740 (0.0007) [2023-12-26 20:44:23,873][105620] Updated weights for policy 1, policy_version 738750 (0.0006) [2023-12-26 20:44:23,926][105620] Updated weights for policy 1, policy_version 738760 (0.0005) [2023-12-26 20:44:24,084][105692] Updated weights for policy 0, policy_version 738450 (0.0005) [2023-12-26 20:44:24,135][105692] Updated weights for policy 0, policy_version 738460 (0.0005) [2023-12-26 20:44:24,197][105692] Updated weights for policy 0, policy_version 738470 (0.0006) [2023-12-26 20:44:24,517][105620] Updated weights for policy 1, policy_version 738770 (0.0006) [2023-12-26 20:44:24,578][105620] Updated weights for policy 1, policy_version 738780 (0.0005) [2023-12-26 20:44:24,638][105620] Updated weights for policy 1, policy_version 738790 (0.0005) [2023-12-26 20:44:24,958][105692] Updated weights for policy 0, policy_version 738480 (0.0008) [2023-12-26 20:44:25,015][105692] Updated weights for policy 0, policy_version 738490 (0.0009) [2023-12-26 20:44:25,069][105692] Updated weights for policy 0, policy_version 738500 (0.0010) [2023-12-26 20:44:25,159][105620] Updated weights for policy 1, policy_version 738800 (0.0005) [2023-12-26 20:44:25,210][105620] Updated weights for policy 1, policy_version 738810 (0.0009) [2023-12-26 20:44:25,264][105620] Updated weights for policy 1, policy_version 738820 (0.0009) [2023-12-26 20:44:25,857][105692] Updated weights for policy 0, policy_version 738511 (0.0009) [2023-12-26 20:44:25,923][105692] Updated weights for policy 0, policy_version 738521 (0.0007) [2023-12-26 20:44:25,938][105620] Updated weights for policy 1, policy_version 738830 (0.0008) [2023-12-26 20:44:25,984][105692] Updated weights for policy 0, policy_version 738531 (0.0009) [2023-12-26 20:44:25,998][105620] Updated weights for policy 1, policy_version 738840 (0.0007) [2023-12-26 20:44:26,052][105620] Updated weights for policy 1, policy_version 738850 (0.0007) [2023-12-26 20:44:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 378257408. Throughput: 0: 9889.3, 1: 9650.7. Samples: 378267264. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:44:26,063][104569] Avg episode reward: [(0, '9259.760'), (1, '9355.900')] [2023-12-26 20:44:26,645][105692] Updated weights for policy 0, policy_version 738541 (0.0010) [2023-12-26 20:44:26,693][105692] Updated weights for policy 0, policy_version 738551 (0.0010) [2023-12-26 20:44:26,740][105692] Updated weights for policy 0, policy_version 738561 (0.0010) [2023-12-26 20:44:26,836][105620] Updated weights for policy 1, policy_version 738860 (0.0007) [2023-12-26 20:44:26,890][105620] Updated weights for policy 1, policy_version 738870 (0.0007) [2023-12-26 20:44:26,937][105620] Updated weights for policy 1, policy_version 738880 (0.0007) [2023-12-26 20:44:27,391][105692] Updated weights for policy 0, policy_version 738571 (0.0009) [2023-12-26 20:44:27,436][105692] Updated weights for policy 0, policy_version 738581 (0.0005) [2023-12-26 20:44:27,481][105692] Updated weights for policy 0, policy_version 738591 (0.0007) [2023-12-26 20:44:27,578][105620] Updated weights for policy 1, policy_version 738890 (0.0008) [2023-12-26 20:44:27,627][105620] Updated weights for policy 1, policy_version 738900 (0.0005) [2023-12-26 20:44:27,660][105586] KL-divergence is very high: 118.0841 [2023-12-26 20:44:27,674][105620] Updated weights for policy 1, policy_version 738910 (0.0005) [2023-12-26 20:44:27,709][105586] KL-divergence is very high: 106.7171 [2023-12-26 20:44:27,757][105620] Updated weights for policy 1, policy_version 738920 (0.0007) [2023-12-26 20:44:28,213][105692] Updated weights for policy 0, policy_version 738601 (0.0010) [2023-12-26 20:44:28,271][105692] Updated weights for policy 0, policy_version 738611 (0.0010) [2023-12-26 20:44:28,333][105692] Updated weights for policy 0, policy_version 738621 (0.0010) [2023-12-26 20:44:28,371][105620] Updated weights for policy 1, policy_version 738930 (0.0006) [2023-12-26 20:44:28,393][105692] Updated weights for policy 0, policy_version 738631 (0.0010) [2023-12-26 20:44:28,428][105620] Updated weights for policy 1, policy_version 738940 (0.0007) [2023-12-26 20:44:28,483][105620] Updated weights for policy 1, policy_version 738950 (0.0009) [2023-12-26 20:44:29,117][105692] Updated weights for policy 0, policy_version 738641 (0.0009) [2023-12-26 20:44:29,171][105692] Updated weights for policy 0, policy_version 738651 (0.0009) [2023-12-26 20:44:29,231][105692] Updated weights for policy 0, policy_version 738661 (0.0008) [2023-12-26 20:44:29,242][105620] Updated weights for policy 1, policy_version 738960 (0.0008) [2023-12-26 20:44:29,306][105620] Updated weights for policy 1, policy_version 738970 (0.0008) [2023-12-26 20:44:29,377][105620] Updated weights for policy 1, policy_version 738980 (0.0008) [2023-12-26 20:44:29,982][105692] Updated weights for policy 0, policy_version 738671 (0.0006) [2023-12-26 20:44:30,034][105692] Updated weights for policy 0, policy_version 738681 (0.0006) [2023-12-26 20:44:30,086][105692] Updated weights for policy 0, policy_version 738691 (0.0007) [2023-12-26 20:44:30,101][105620] Updated weights for policy 1, policy_version 738990 (0.0007) [2023-12-26 20:44:30,164][105620] Updated weights for policy 1, policy_version 739000 (0.0007) [2023-12-26 20:44:30,227][105620] Updated weights for policy 1, policy_version 739010 (0.0008) [2023-12-26 20:44:30,741][105692] Updated weights for policy 0, policy_version 738701 (0.0009) [2023-12-26 20:44:30,807][105692] Updated weights for policy 0, policy_version 738711 (0.0010) [2023-12-26 20:44:30,846][105620] Updated weights for policy 1, policy_version 739020 (0.0008) [2023-12-26 20:44:30,865][105692] Updated weights for policy 0, policy_version 738721 (0.0008) [2023-12-26 20:44:30,905][105620] Updated weights for policy 1, policy_version 739030 (0.0009) [2023-12-26 20:44:30,962][105620] Updated weights for policy 1, policy_version 739040 (0.0010) [2023-12-26 20:44:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 378363904. Throughput: 0: 9925.9, 1: 9712.9. Samples: 378327708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:44:31,063][104569] Avg episode reward: [(0, '9076.196'), (1, '9173.543')] [2023-12-26 20:44:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000738728_189145088.pth... [2023-12-26 20:44:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000739048_189218816.pth... [2023-12-26 20:44:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000737576_188850176.pth [2023-12-26 20:44:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000737896_188923904.pth [2023-12-26 20:44:31,617][105692] Updated weights for policy 0, policy_version 738731 (0.0007) [2023-12-26 20:44:31,671][105692] Updated weights for policy 0, policy_version 738741 (0.0008) [2023-12-26 20:44:31,701][105620] Updated weights for policy 1, policy_version 739050 (0.0010) [2023-12-26 20:44:31,729][105692] Updated weights for policy 0, policy_version 738751 (0.0009) [2023-12-26 20:44:31,761][105620] Updated weights for policy 1, policy_version 739060 (0.0010) [2023-12-26 20:44:31,809][105620] Updated weights for policy 1, policy_version 739070 (0.0010) [2023-12-26 20:44:31,857][105620] Updated weights for policy 1, policy_version 739080 (0.0010) [2023-12-26 20:44:32,504][105692] Updated weights for policy 0, policy_version 738761 (0.0008) [2023-12-26 20:44:32,552][105692] Updated weights for policy 0, policy_version 738771 (0.0008) [2023-12-26 20:44:32,588][105620] Updated weights for policy 1, policy_version 739090 (0.0011) [2023-12-26 20:44:32,610][105692] Updated weights for policy 0, policy_version 738781 (0.0006) [2023-12-26 20:44:32,651][105620] Updated weights for policy 1, policy_version 739100 (0.0011) [2023-12-26 20:44:32,674][105692] Updated weights for policy 0, policy_version 738791 (0.0006) [2023-12-26 20:44:32,707][105620] Updated weights for policy 1, policy_version 739110 (0.0010) [2023-12-26 20:44:33,428][105692] Updated weights for policy 0, policy_version 738801 (0.0005) [2023-12-26 20:44:33,434][105620] Updated weights for policy 1, policy_version 739120 (0.0010) [2023-12-26 20:44:33,480][105692] Updated weights for policy 0, policy_version 738811 (0.0005) [2023-12-26 20:44:33,488][105620] Updated weights for policy 1, policy_version 739130 (0.0010) [2023-12-26 20:44:33,532][105692] Updated weights for policy 0, policy_version 738821 (0.0005) [2023-12-26 20:44:33,536][105620] Updated weights for policy 1, policy_version 739140 (0.0010) [2023-12-26 20:44:34,140][105692] Updated weights for policy 0, policy_version 738831 (0.0007) [2023-12-26 20:44:34,198][105692] Updated weights for policy 0, policy_version 738841 (0.0009) [2023-12-26 20:44:34,258][105692] Updated weights for policy 0, policy_version 738851 (0.0008) [2023-12-26 20:44:34,305][105620] Updated weights for policy 1, policy_version 739150 (0.0010) [2023-12-26 20:44:34,370][105620] Updated weights for policy 1, policy_version 739160 (0.0010) [2023-12-26 20:44:34,426][105620] Updated weights for policy 1, policy_version 739170 (0.0011) [2023-12-26 20:44:35,013][105692] Updated weights for policy 0, policy_version 738861 (0.0008) [2023-12-26 20:44:35,072][105692] Updated weights for policy 0, policy_version 738871 (0.0008) [2023-12-26 20:44:35,124][105692] Updated weights for policy 0, policy_version 738881 (0.0008) [2023-12-26 20:44:35,164][105620] Updated weights for policy 1, policy_version 739180 (0.0010) [2023-12-26 20:44:35,222][105620] Updated weights for policy 1, policy_version 739190 (0.0010) [2023-12-26 20:44:35,277][105620] Updated weights for policy 1, policy_version 739200 (0.0010) [2023-12-26 20:44:35,864][105692] Updated weights for policy 0, policy_version 738891 (0.0008) [2023-12-26 20:44:35,913][105692] Updated weights for policy 0, policy_version 738901 (0.0008) [2023-12-26 20:44:35,975][105692] Updated weights for policy 0, policy_version 738911 (0.0007) [2023-12-26 20:44:36,018][105620] Updated weights for policy 1, policy_version 739210 (0.0010) [2023-12-26 20:44:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 378454016. Throughput: 0: 9865.9, 1: 9689.9. Samples: 378443776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:44:36,062][104569] Avg episode reward: [(0, '9347.525'), (1, '9264.729')] [2023-12-26 20:44:36,083][105620] Updated weights for policy 1, policy_version 739220 (0.0010) [2023-12-26 20:44:36,154][105620] Updated weights for policy 1, policy_version 739230 (0.0011) [2023-12-26 20:44:36,202][105620] Updated weights for policy 1, policy_version 739240 (0.0007) [2023-12-26 20:44:36,665][105692] Updated weights for policy 0, policy_version 738921 (0.0007) [2023-12-26 20:44:36,722][105692] Updated weights for policy 0, policy_version 738931 (0.0008) [2023-12-26 20:44:36,785][105692] Updated weights for policy 0, policy_version 738941 (0.0008) [2023-12-26 20:44:36,841][105692] Updated weights for policy 0, policy_version 738951 (0.0008) [2023-12-26 20:44:36,951][105620] Updated weights for policy 1, policy_version 739250 (0.0010) [2023-12-26 20:44:37,016][105620] Updated weights for policy 1, policy_version 739260 (0.0010) [2023-12-26 20:44:37,082][105620] Updated weights for policy 1, policy_version 739270 (0.0010) [2023-12-26 20:44:37,643][105692] Updated weights for policy 0, policy_version 738961 (0.0009) [2023-12-26 20:44:37,694][105692] Updated weights for policy 0, policy_version 738971 (0.0009) [2023-12-26 20:44:37,744][105692] Updated weights for policy 0, policy_version 738981 (0.0009) [2023-12-26 20:44:37,762][105620] Updated weights for policy 1, policy_version 739280 (0.0007) [2023-12-26 20:44:37,825][105620] Updated weights for policy 1, policy_version 739290 (0.0005) [2023-12-26 20:44:37,883][105620] Updated weights for policy 1, policy_version 739300 (0.0006) [2023-12-26 20:44:38,530][105620] Updated weights for policy 1, policy_version 739310 (0.0006) [2023-12-26 20:44:38,548][105692] Updated weights for policy 0, policy_version 738991 (0.0009) [2023-12-26 20:44:38,582][105620] Updated weights for policy 1, policy_version 739320 (0.0007) [2023-12-26 20:44:38,596][105692] Updated weights for policy 0, policy_version 739001 (0.0006) [2023-12-26 20:44:38,628][105620] Updated weights for policy 1, policy_version 739330 (0.0007) [2023-12-26 20:44:38,652][105692] Updated weights for policy 0, policy_version 739011 (0.0007) [2023-12-26 20:44:39,412][105620] Updated weights for policy 1, policy_version 739340 (0.0009) [2023-12-26 20:44:39,421][105692] Updated weights for policy 0, policy_version 739021 (0.0008) [2023-12-26 20:44:39,471][105620] Updated weights for policy 1, policy_version 739350 (0.0007) [2023-12-26 20:44:39,487][105692] Updated weights for policy 0, policy_version 739031 (0.0006) [2023-12-26 20:44:39,524][105620] Updated weights for policy 1, policy_version 739360 (0.0006) [2023-12-26 20:44:39,543][105692] Updated weights for policy 0, policy_version 739041 (0.0006) [2023-12-26 20:44:40,175][105692] Updated weights for policy 0, policy_version 739051 (0.0006) [2023-12-26 20:44:40,225][105692] Updated weights for policy 0, policy_version 739061 (0.0009) [2023-12-26 20:44:40,275][105620] Updated weights for policy 1, policy_version 739370 (0.0009) [2023-12-26 20:44:40,279][105692] Updated weights for policy 0, policy_version 739071 (0.0007) [2023-12-26 20:44:40,339][105620] Updated weights for policy 1, policy_version 739380 (0.0009) [2023-12-26 20:44:40,407][105620] Updated weights for policy 1, policy_version 739390 (0.0009) [2023-12-26 20:44:40,913][105692] Updated weights for policy 0, policy_version 739081 (0.0008) [2023-12-26 20:44:40,980][105692] Updated weights for policy 0, policy_version 739091 (0.0008) [2023-12-26 20:44:41,039][105692] Updated weights for policy 0, policy_version 739101 (0.0008) [2023-12-26 20:44:41,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 378544128. Throughput: 0: 9781.4, 1: 9696.6. Samples: 378558524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:44:41,062][104569] Avg episode reward: [(0, '9347.775'), (1, '9264.700')] [2023-12-26 20:44:41,110][105692] Updated weights for policy 0, policy_version 739111 (0.0008) [2023-12-26 20:44:41,143][105620] Updated weights for policy 1, policy_version 739401 (0.0009) [2023-12-26 20:44:41,204][105620] Updated weights for policy 1, policy_version 739411 (0.0008) [2023-12-26 20:44:41,268][105620] Updated weights for policy 1, policy_version 739421 (0.0008) [2023-12-26 20:44:41,331][105620] Updated weights for policy 1, policy_version 739431 (0.0008) [2023-12-26 20:44:41,887][105692] Updated weights for policy 0, policy_version 739121 (0.0010) [2023-12-26 20:44:41,949][105692] Updated weights for policy 0, policy_version 739131 (0.0009) [2023-12-26 20:44:42,007][105692] Updated weights for policy 0, policy_version 739141 (0.0008) [2023-12-26 20:44:42,055][105620] Updated weights for policy 1, policy_version 739441 (0.0009) [2023-12-26 20:44:42,117][105620] Updated weights for policy 1, policy_version 739451 (0.0009) [2023-12-26 20:44:42,178][105620] Updated weights for policy 1, policy_version 739461 (0.0008) [2023-12-26 20:44:42,797][105692] Updated weights for policy 0, policy_version 739151 (0.0008) [2023-12-26 20:44:42,855][105692] Updated weights for policy 0, policy_version 739161 (0.0008) [2023-12-26 20:44:42,918][105692] Updated weights for policy 0, policy_version 739171 (0.0009) [2023-12-26 20:44:42,924][105620] Updated weights for policy 1, policy_version 739471 (0.0006) [2023-12-26 20:44:42,978][105620] Updated weights for policy 1, policy_version 739481 (0.0007) [2023-12-26 20:44:43,028][105620] Updated weights for policy 1, policy_version 739491 (0.0008) [2023-12-26 20:44:43,670][105620] Updated weights for policy 1, policy_version 739501 (0.0009) [2023-12-26 20:44:43,720][105620] Updated weights for policy 1, policy_version 739511 (0.0009) [2023-12-26 20:44:43,722][105692] Updated weights for policy 0, policy_version 739181 (0.0006) [2023-12-26 20:44:43,774][105620] Updated weights for policy 1, policy_version 739521 (0.0008) [2023-12-26 20:44:43,778][105692] Updated weights for policy 0, policy_version 739191 (0.0008) [2023-12-26 20:44:43,836][105692] Updated weights for policy 0, policy_version 739201 (0.0009) [2023-12-26 20:44:44,467][105620] Updated weights for policy 1, policy_version 739531 (0.0007) [2023-12-26 20:44:44,538][105620] Updated weights for policy 1, policy_version 739541 (0.0008) [2023-12-26 20:44:44,595][105692] Updated weights for policy 0, policy_version 739211 (0.0008) [2023-12-26 20:44:44,599][105620] Updated weights for policy 1, policy_version 739551 (0.0006) [2023-12-26 20:44:44,655][105692] Updated weights for policy 0, policy_version 739221 (0.0005) [2023-12-26 20:44:44,716][105692] Updated weights for policy 0, policy_version 739231 (0.0005) [2023-12-26 20:44:45,244][105620] Updated weights for policy 1, policy_version 739561 (0.0006) [2023-12-26 20:44:45,311][105620] Updated weights for policy 1, policy_version 739571 (0.0011) [2023-12-26 20:44:45,351][105692] Updated weights for policy 0, policy_version 739241 (0.0007) [2023-12-26 20:44:45,367][105620] Updated weights for policy 1, policy_version 739581 (0.0011) [2023-12-26 20:44:45,418][105692] Updated weights for policy 0, policy_version 739251 (0.0007) [2023-12-26 20:44:45,428][105620] Updated weights for policy 1, policy_version 739591 (0.0011) [2023-12-26 20:44:45,474][105692] Updated weights for policy 0, policy_version 739261 (0.0006) [2023-12-26 20:44:45,525][105692] Updated weights for policy 0, policy_version 739271 (0.0009) [2023-12-26 20:44:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 378642432. Throughput: 0: 9668.0, 1: 9666.8. Samples: 378614732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:44:46,063][104569] Avg episode reward: [(0, '9256.836'), (1, '9264.660')] [2023-12-26 20:44:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000739592_189358080.pth... [2023-12-26 20:44:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000738472_189071360.pth [2023-12-26 20:44:46,080][105692] Updated weights for policy 0, policy_version 739281 (0.0010) [2023-12-26 20:44:46,144][105692] Updated weights for policy 0, policy_version 739291 (0.0010) [2023-12-26 20:44:46,148][105620] Updated weights for policy 1, policy_version 739601 (0.0011) [2023-12-26 20:44:46,203][105692] Updated weights for policy 0, policy_version 739301 (0.0011) [2023-12-26 20:44:46,210][105620] Updated weights for policy 1, policy_version 739611 (0.0010) [2023-12-26 20:44:46,220][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000739304_189292544.pth... [2023-12-26 20:44:46,225][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000738152_188997632.pth [2023-12-26 20:44:46,269][105620] Updated weights for policy 1, policy_version 739621 (0.0010) [2023-12-26 20:44:46,934][105692] Updated weights for policy 0, policy_version 739311 (0.0008) [2023-12-26 20:44:46,980][105692] Updated weights for policy 0, policy_version 739321 (0.0010) [2023-12-26 20:44:46,983][105620] Updated weights for policy 1, policy_version 739631 (0.0010) [2023-12-26 20:44:47,034][105692] Updated weights for policy 0, policy_version 739331 (0.0007) [2023-12-26 20:44:47,040][105620] Updated weights for policy 1, policy_version 739641 (0.0008) [2023-12-26 20:44:47,102][105620] Updated weights for policy 1, policy_version 739651 (0.0009) [2023-12-26 20:44:47,614][105692] Updated weights for policy 0, policy_version 739341 (0.0005) [2023-12-26 20:44:47,676][105692] Updated weights for policy 0, policy_version 739351 (0.0008) [2023-12-26 20:44:47,723][105620] Updated weights for policy 1, policy_version 739661 (0.0006) [2023-12-26 20:44:47,725][105692] Updated weights for policy 0, policy_version 739361 (0.0009) [2023-12-26 20:44:47,773][105620] Updated weights for policy 1, policy_version 739671 (0.0006) [2023-12-26 20:44:47,818][105620] Updated weights for policy 1, policy_version 739681 (0.0006) [2023-12-26 20:44:48,406][105620] Updated weights for policy 1, policy_version 739691 (0.0005) [2023-12-26 20:44:48,466][105620] Updated weights for policy 1, policy_version 739701 (0.0005) [2023-12-26 20:44:48,542][105620] Updated weights for policy 1, policy_version 739711 (0.0006) [2023-12-26 20:44:48,589][105692] Updated weights for policy 0, policy_version 739371 (0.0006) [2023-12-26 20:44:48,650][105692] Updated weights for policy 0, policy_version 739381 (0.0009) [2023-12-26 20:44:48,709][105692] Updated weights for policy 0, policy_version 739391 (0.0008) [2023-12-26 20:44:49,113][105620] Updated weights for policy 1, policy_version 739721 (0.0009) [2023-12-26 20:44:49,161][105620] Updated weights for policy 1, policy_version 739731 (0.0009) [2023-12-26 20:44:49,221][105620] Updated weights for policy 1, policy_version 739741 (0.0009) [2023-12-26 20:44:49,284][105620] Updated weights for policy 1, policy_version 739751 (0.0009) [2023-12-26 20:44:49,450][105692] Updated weights for policy 0, policy_version 739401 (0.0006) [2023-12-26 20:44:49,521][105692] Updated weights for policy 0, policy_version 739411 (0.0010) [2023-12-26 20:44:49,584][105692] Updated weights for policy 0, policy_version 739421 (0.0010) [2023-12-26 20:44:49,641][105692] Updated weights for policy 0, policy_version 739431 (0.0009) [2023-12-26 20:44:49,998][105620] Updated weights for policy 1, policy_version 739761 (0.0010) [2023-12-26 20:44:50,047][105620] Updated weights for policy 1, policy_version 739771 (0.0007) [2023-12-26 20:44:50,098][105620] Updated weights for policy 1, policy_version 739781 (0.0005) [2023-12-26 20:44:50,412][105692] Updated weights for policy 0, policy_version 739441 (0.0006) [2023-12-26 20:44:50,474][105692] Updated weights for policy 0, policy_version 739451 (0.0006) [2023-12-26 20:44:50,539][105692] Updated weights for policy 0, policy_version 739461 (0.0006) [2023-12-26 20:44:50,899][105620] Updated weights for policy 1, policy_version 739791 (0.0006) [2023-12-26 20:44:50,958][105620] Updated weights for policy 1, policy_version 739801 (0.0007) [2023-12-26 20:44:51,022][105620] Updated weights for policy 1, policy_version 739811 (0.0007) [2023-12-26 20:44:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 378748928. Throughput: 0: 9651.1, 1: 9771.2. Samples: 378737156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:44:51,062][104569] Avg episode reward: [(0, '9256.656'), (1, '9355.715')] [2023-12-26 20:44:51,239][105692] Updated weights for policy 0, policy_version 739471 (0.0008) [2023-12-26 20:44:51,302][105692] Updated weights for policy 0, policy_version 739481 (0.0009) [2023-12-26 20:44:51,368][105692] Updated weights for policy 0, policy_version 739491 (0.0009) [2023-12-26 20:44:51,742][105620] Updated weights for policy 1, policy_version 739821 (0.0007) [2023-12-26 20:44:51,797][105620] Updated weights for policy 1, policy_version 739831 (0.0008) [2023-12-26 20:44:51,845][105620] Updated weights for policy 1, policy_version 739841 (0.0009) [2023-12-26 20:44:52,138][105692] Updated weights for policy 0, policy_version 739501 (0.0009) [2023-12-26 20:44:52,198][105692] Updated weights for policy 0, policy_version 739511 (0.0006) [2023-12-26 20:44:52,250][105692] Updated weights for policy 0, policy_version 739521 (0.0006) [2023-12-26 20:44:52,539][105620] Updated weights for policy 1, policy_version 739851 (0.0008) [2023-12-26 20:44:52,608][105620] Updated weights for policy 1, policy_version 739861 (0.0009) [2023-12-26 20:44:52,678][105620] Updated weights for policy 1, policy_version 739871 (0.0010) [2023-12-26 20:44:52,934][105692] Updated weights for policy 0, policy_version 739531 (0.0009) [2023-12-26 20:44:52,991][105692] Updated weights for policy 0, policy_version 739541 (0.0009) [2023-12-26 20:44:53,051][105692] Updated weights for policy 0, policy_version 739551 (0.0009) [2023-12-26 20:44:53,472][105620] Updated weights for policy 1, policy_version 739881 (0.0008) [2023-12-26 20:44:53,546][105620] Updated weights for policy 1, policy_version 739891 (0.0009) [2023-12-26 20:44:53,614][105620] Updated weights for policy 1, policy_version 739901 (0.0008) [2023-12-26 20:44:53,685][105620] Updated weights for policy 1, policy_version 739911 (0.0009) [2023-12-26 20:44:53,692][105692] Updated weights for policy 0, policy_version 739561 (0.0009) [2023-12-26 20:44:53,746][105692] Updated weights for policy 0, policy_version 739571 (0.0007) [2023-12-26 20:44:53,803][105692] Updated weights for policy 0, policy_version 739581 (0.0007) [2023-12-26 20:44:53,862][105692] Updated weights for policy 0, policy_version 739591 (0.0006) [2023-12-26 20:44:54,328][105620] Updated weights for policy 1, policy_version 739921 (0.0008) [2023-12-26 20:44:54,376][105620] Updated weights for policy 1, policy_version 739931 (0.0009) [2023-12-26 20:44:54,429][105620] Updated weights for policy 1, policy_version 739942 (0.0009) [2023-12-26 20:44:54,576][105692] Updated weights for policy 0, policy_version 739601 (0.0008) [2023-12-26 20:44:54,627][105692] Updated weights for policy 0, policy_version 739611 (0.0008) [2023-12-26 20:44:54,675][105692] Updated weights for policy 0, policy_version 739621 (0.0009) [2023-12-26 20:44:55,139][105620] Updated weights for policy 1, policy_version 739952 (0.0006) [2023-12-26 20:44:55,189][105620] Updated weights for policy 1, policy_version 739962 (0.0005) [2023-12-26 20:44:55,237][105620] Updated weights for policy 1, policy_version 739972 (0.0006) [2023-12-26 20:44:55,300][105692] Updated weights for policy 0, policy_version 739631 (0.0006) [2023-12-26 20:44:55,352][105692] Updated weights for policy 0, policy_version 739641 (0.0005) [2023-12-26 20:44:55,403][105692] Updated weights for policy 0, policy_version 739651 (0.0005) [2023-12-26 20:44:55,823][105620] Updated weights for policy 1, policy_version 739982 (0.0008) [2023-12-26 20:44:55,883][105620] Updated weights for policy 1, policy_version 739992 (0.0010) [2023-12-26 20:44:55,933][105692] Updated weights for policy 0, policy_version 739661 (0.0005) [2023-12-26 20:44:55,946][105620] Updated weights for policy 1, policy_version 740002 (0.0010) [2023-12-26 20:44:55,993][105692] Updated weights for policy 0, policy_version 739671 (0.0007) [2023-12-26 20:44:56,053][105692] Updated weights for policy 0, policy_version 739681 (0.0007) [2023-12-26 20:44:56,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 378847232. Throughput: 0: 9725.2, 1: 9784.4. Samples: 378856136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:44:56,062][104569] Avg episode reward: [(0, '9097.923'), (1, '9265.629')] [2023-12-26 20:44:56,630][105620] Updated weights for policy 1, policy_version 740012 (0.0010) [2023-12-26 20:44:56,686][105692] Updated weights for policy 0, policy_version 739691 (0.0007) [2023-12-26 20:44:56,692][105620] Updated weights for policy 1, policy_version 740022 (0.0010) [2023-12-26 20:44:56,732][105692] Updated weights for policy 0, policy_version 739701 (0.0008) [2023-12-26 20:44:56,746][105620] Updated weights for policy 1, policy_version 740032 (0.0010) [2023-12-26 20:44:56,785][105692] Updated weights for policy 0, policy_version 739711 (0.0005) [2023-12-26 20:44:57,457][105692] Updated weights for policy 0, policy_version 739721 (0.0008) [2023-12-26 20:44:57,477][105620] Updated weights for policy 1, policy_version 740042 (0.0011) [2023-12-26 20:44:57,515][105692] Updated weights for policy 0, policy_version 739731 (0.0008) [2023-12-26 20:44:57,524][105620] Updated weights for policy 1, policy_version 740052 (0.0010) [2023-12-26 20:44:57,568][105620] Updated weights for policy 1, policy_version 740062 (0.0010) [2023-12-26 20:44:57,573][105692] Updated weights for policy 0, policy_version 739741 (0.0007) [2023-12-26 20:44:57,616][105620] Updated weights for policy 1, policy_version 740072 (0.0010) [2023-12-26 20:44:57,626][105692] Updated weights for policy 0, policy_version 739751 (0.0006) [2023-12-26 20:44:58,293][105692] Updated weights for policy 0, policy_version 739761 (0.0008) [2023-12-26 20:44:58,352][105692] Updated weights for policy 0, policy_version 739771 (0.0008) [2023-12-26 20:44:58,400][105620] Updated weights for policy 1, policy_version 740082 (0.0008) [2023-12-26 20:44:58,414][105692] Updated weights for policy 0, policy_version 739781 (0.0006) [2023-12-26 20:44:58,462][105620] Updated weights for policy 1, policy_version 740092 (0.0008) [2023-12-26 20:44:58,523][105620] Updated weights for policy 1, policy_version 740102 (0.0009) [2023-12-26 20:44:59,174][105692] Updated weights for policy 0, policy_version 739791 (0.0009) [2023-12-26 20:44:59,230][105692] Updated weights for policy 0, policy_version 739801 (0.0009) [2023-12-26 20:44:59,299][105692] Updated weights for policy 0, policy_version 739811 (0.0008) [2023-12-26 20:44:59,317][105620] Updated weights for policy 1, policy_version 740112 (0.0008) [2023-12-26 20:44:59,383][105620] Updated weights for policy 1, policy_version 740122 (0.0009) [2023-12-26 20:44:59,439][105620] Updated weights for policy 1, policy_version 740132 (0.0010) [2023-12-26 20:45:00,020][105692] Updated weights for policy 0, policy_version 739821 (0.0007) [2023-12-26 20:45:00,087][105692] Updated weights for policy 0, policy_version 739831 (0.0006) [2023-12-26 20:45:00,139][105620] Updated weights for policy 1, policy_version 740142 (0.0007) [2023-12-26 20:45:00,155][105692] Updated weights for policy 0, policy_version 739841 (0.0006) [2023-12-26 20:45:00,201][105620] Updated weights for policy 1, policy_version 740152 (0.0006) [2023-12-26 20:45:00,251][105620] Updated weights for policy 1, policy_version 740162 (0.0006) [2023-12-26 20:45:00,796][105692] Updated weights for policy 0, policy_version 739851 (0.0007) [2023-12-26 20:45:00,849][105692] Updated weights for policy 0, policy_version 739862 (0.0009) [2023-12-26 20:45:00,876][105620] Updated weights for policy 1, policy_version 740172 (0.0006) [2023-12-26 20:45:00,896][105692] Updated weights for policy 0, policy_version 739872 (0.0008) [2023-12-26 20:45:00,923][105620] Updated weights for policy 1, policy_version 740182 (0.0008) [2023-12-26 20:45:00,970][105620] Updated weights for policy 1, policy_version 740193 (0.0009) [2023-12-26 20:45:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 378953728. Throughput: 0: 9820.1, 1: 9796.2. Samples: 378916556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:45:01,062][104569] Avg episode reward: [(0, '8754.657'), (1, '9265.674')] [2023-12-26 20:45:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000740200_189513728.pth... [2023-12-26 20:45:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000739880_189440000.pth... [2023-12-26 20:45:01,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000739048_189218816.pth [2023-12-26 20:45:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000738728_189145088.pth [2023-12-26 20:45:01,590][105692] Updated weights for policy 0, policy_version 739882 (0.0008) [2023-12-26 20:45:01,662][105692] Updated weights for policy 0, policy_version 739892 (0.0007) [2023-12-26 20:45:01,723][105692] Updated weights for policy 0, policy_version 739902 (0.0008) [2023-12-26 20:45:01,757][105620] Updated weights for policy 1, policy_version 740203 (0.0008) [2023-12-26 20:45:01,786][105692] Updated weights for policy 0, policy_version 739912 (0.0008) [2023-12-26 20:45:01,822][105620] Updated weights for policy 1, policy_version 740213 (0.0006) [2023-12-26 20:45:01,872][105620] Updated weights for policy 1, policy_version 740223 (0.0009) [2023-12-26 20:45:02,534][105620] Updated weights for policy 1, policy_version 740233 (0.0008) [2023-12-26 20:45:02,543][105692] Updated weights for policy 0, policy_version 739922 (0.0009) [2023-12-26 20:45:02,586][105620] Updated weights for policy 1, policy_version 740243 (0.0006) [2023-12-26 20:45:02,588][105692] Updated weights for policy 0, policy_version 739932 (0.0010) [2023-12-26 20:45:02,637][105692] Updated weights for policy 0, policy_version 739942 (0.0010) [2023-12-26 20:45:02,642][105620] Updated weights for policy 1, policy_version 740253 (0.0005) [2023-12-26 20:45:02,696][105620] Updated weights for policy 1, policy_version 740263 (0.0005) [2023-12-26 20:45:03,359][105620] Updated weights for policy 1, policy_version 740273 (0.0008) [2023-12-26 20:45:03,391][105692] Updated weights for policy 0, policy_version 739952 (0.0010) [2023-12-26 20:45:03,416][105620] Updated weights for policy 1, policy_version 740283 (0.0008) [2023-12-26 20:45:03,439][105692] Updated weights for policy 0, policy_version 739962 (0.0010) [2023-12-26 20:45:03,459][105620] Updated weights for policy 1, policy_version 740293 (0.0005) [2023-12-26 20:45:03,486][105692] Updated weights for policy 0, policy_version 739972 (0.0010) [2023-12-26 20:45:04,202][105620] Updated weights for policy 1, policy_version 740303 (0.0007) [2023-12-26 20:45:04,265][105620] Updated weights for policy 1, policy_version 740313 (0.0007) [2023-12-26 20:45:04,266][105692] Updated weights for policy 0, policy_version 739982 (0.0011) [2023-12-26 20:45:04,328][105692] Updated weights for policy 0, policy_version 739992 (0.0008) [2023-12-26 20:45:04,328][105620] Updated weights for policy 1, policy_version 740323 (0.0008) [2023-12-26 20:45:04,384][105692] Updated weights for policy 0, policy_version 740002 (0.0011) [2023-12-26 20:45:05,107][105692] Updated weights for policy 0, policy_version 740012 (0.0008) [2023-12-26 20:45:05,118][105620] Updated weights for policy 1, policy_version 740333 (0.0009) [2023-12-26 20:45:05,166][105620] Updated weights for policy 1, policy_version 740343 (0.0008) [2023-12-26 20:45:05,168][105692] Updated weights for policy 0, policy_version 740022 (0.0005) [2023-12-26 20:45:05,220][105620] Updated weights for policy 1, policy_version 740353 (0.0009) [2023-12-26 20:45:05,227][105692] Updated weights for policy 0, policy_version 740032 (0.0009) [2023-12-26 20:45:05,905][105692] Updated weights for policy 0, policy_version 740042 (0.0010) [2023-12-26 20:45:05,956][105692] Updated weights for policy 0, policy_version 740052 (0.0010) [2023-12-26 20:45:06,000][105692] Updated weights for policy 0, policy_version 740062 (0.0010) [2023-12-26 20:45:06,023][105620] Updated weights for policy 1, policy_version 740363 (0.0006) [2023-12-26 20:45:06,045][105692] Updated weights for policy 0, policy_version 740072 (0.0010) [2023-12-26 20:45:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 379043840. Throughput: 0: 9725.8, 1: 9870.4. Samples: 379032368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:45:06,062][104569] Avg episode reward: [(0, '8752.930'), (1, '9173.385')] [2023-12-26 20:45:06,077][105620] Updated weights for policy 1, policy_version 740373 (0.0007) [2023-12-26 20:45:06,136][105620] Updated weights for policy 1, policy_version 740383 (0.0008) [2023-12-26 20:45:06,858][105692] Updated weights for policy 0, policy_version 740082 (0.0010) [2023-12-26 20:45:06,913][105620] Updated weights for policy 1, policy_version 740393 (0.0008) [2023-12-26 20:45:06,924][105692] Updated weights for policy 0, policy_version 740092 (0.0010) [2023-12-26 20:45:06,965][105620] Updated weights for policy 1, policy_version 740403 (0.0010) [2023-12-26 20:45:06,983][105692] Updated weights for policy 0, policy_version 740102 (0.0010) [2023-12-26 20:45:07,013][105620] Updated weights for policy 1, policy_version 740413 (0.0010) [2023-12-26 20:45:07,068][105620] Updated weights for policy 1, policy_version 740423 (0.0010) [2023-12-26 20:45:07,742][105692] Updated weights for policy 0, policy_version 740112 (0.0011) [2023-12-26 20:45:07,796][105692] Updated weights for policy 0, policy_version 740122 (0.0010) [2023-12-26 20:45:07,800][105620] Updated weights for policy 1, policy_version 740433 (0.0010) [2023-12-26 20:45:07,854][105692] Updated weights for policy 0, policy_version 740132 (0.0010) [2023-12-26 20:45:07,858][105620] Updated weights for policy 1, policy_version 740443 (0.0010) [2023-12-26 20:45:07,916][105620] Updated weights for policy 1, policy_version 740453 (0.0010) [2023-12-26 20:45:08,547][105692] Updated weights for policy 0, policy_version 740142 (0.0011) [2023-12-26 20:45:08,598][105620] Updated weights for policy 1, policy_version 740463 (0.0011) [2023-12-26 20:45:08,606][105692] Updated weights for policy 0, policy_version 740152 (0.0011) [2023-12-26 20:45:08,651][105620] Updated weights for policy 1, policy_version 740473 (0.0010) [2023-12-26 20:45:08,670][105692] Updated weights for policy 0, policy_version 740162 (0.0011) [2023-12-26 20:45:08,703][105620] Updated weights for policy 1, policy_version 740483 (0.0010) [2023-12-26 20:45:09,348][105692] Updated weights for policy 0, policy_version 740172 (0.0010) [2023-12-26 20:45:09,415][105692] Updated weights for policy 0, policy_version 740182 (0.0011) [2023-12-26 20:45:09,465][105620] Updated weights for policy 1, policy_version 740493 (0.0008) [2023-12-26 20:45:09,472][105692] Updated weights for policy 0, policy_version 740192 (0.0010) [2023-12-26 20:45:09,512][105620] Updated weights for policy 1, policy_version 740503 (0.0007) [2023-12-26 20:45:09,572][105620] Updated weights for policy 1, policy_version 740513 (0.0009) [2023-12-26 20:45:10,219][105692] Updated weights for policy 0, policy_version 740202 (0.0008) [2023-12-26 20:45:10,282][105692] Updated weights for policy 0, policy_version 740212 (0.0009) [2023-12-26 20:45:10,350][105620] Updated weights for policy 1, policy_version 740523 (0.0008) [2023-12-26 20:45:10,366][105692] Updated weights for policy 0, policy_version 740222 (0.0007) [2023-12-26 20:45:10,405][105620] Updated weights for policy 1, policy_version 740533 (0.0006) [2023-12-26 20:45:10,428][105692] Updated weights for policy 0, policy_version 740232 (0.0009) [2023-12-26 20:45:10,464][105620] Updated weights for policy 1, policy_version 740543 (0.0010) [2023-12-26 20:45:11,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 379133952. Throughput: 0: 9782.5, 1: 9757.2. Samples: 379146548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:45:11,062][104569] Avg episode reward: [(0, '9258.070'), (1, '9081.881')] [2023-12-26 20:45:11,140][105692] Updated weights for policy 0, policy_version 740242 (0.0009) [2023-12-26 20:45:11,208][105692] Updated weights for policy 0, policy_version 740252 (0.0009) [2023-12-26 20:45:11,228][105620] Updated weights for policy 1, policy_version 740553 (0.0008) [2023-12-26 20:45:11,268][105692] Updated weights for policy 0, policy_version 740262 (0.0009) [2023-12-26 20:45:11,295][105620] Updated weights for policy 1, policy_version 740563 (0.0009) [2023-12-26 20:45:11,358][105620] Updated weights for policy 1, policy_version 740573 (0.0009) [2023-12-26 20:45:11,430][105620] Updated weights for policy 1, policy_version 740583 (0.0008) [2023-12-26 20:45:11,987][105692] Updated weights for policy 0, policy_version 740272 (0.0008) [2023-12-26 20:45:12,057][105692] Updated weights for policy 0, policy_version 740282 (0.0008) [2023-12-26 20:45:12,124][105692] Updated weights for policy 0, policy_version 740292 (0.0008) [2023-12-26 20:45:12,182][105620] Updated weights for policy 1, policy_version 740593 (0.0009) [2023-12-26 20:45:12,245][105620] Updated weights for policy 1, policy_version 740603 (0.0009) [2023-12-26 20:45:12,309][105620] Updated weights for policy 1, policy_version 740613 (0.0007) [2023-12-26 20:45:12,806][105692] Updated weights for policy 0, policy_version 740302 (0.0009) [2023-12-26 20:45:12,865][105692] Updated weights for policy 0, policy_version 740312 (0.0010) [2023-12-26 20:45:12,923][105692] Updated weights for policy 0, policy_version 740322 (0.0010) [2023-12-26 20:45:13,051][105620] Updated weights for policy 1, policy_version 740623 (0.0008) [2023-12-26 20:45:13,113][105620] Updated weights for policy 1, policy_version 740633 (0.0008) [2023-12-26 20:45:13,161][105620] Updated weights for policy 1, policy_version 740643 (0.0008) [2023-12-26 20:45:13,577][105692] Updated weights for policy 0, policy_version 740332 (0.0010) [2023-12-26 20:45:13,635][105692] Updated weights for policy 0, policy_version 740342 (0.0010) [2023-12-26 20:45:13,700][105692] Updated weights for policy 0, policy_version 740352 (0.0009) [2023-12-26 20:45:14,023][105620] Updated weights for policy 1, policy_version 740654 (0.0009) [2023-12-26 20:45:14,077][105620] Updated weights for policy 1, policy_version 740665 (0.0010) [2023-12-26 20:45:14,134][105620] Updated weights for policy 1, policy_version 740676 (0.0008) [2023-12-26 20:45:14,248][105692] Updated weights for policy 0, policy_version 740362 (0.0005) [2023-12-26 20:45:14,306][105692] Updated weights for policy 0, policy_version 740372 (0.0010) [2023-12-26 20:45:14,360][105692] Updated weights for policy 0, policy_version 740382 (0.0010) [2023-12-26 20:45:14,403][105692] Updated weights for policy 0, policy_version 740392 (0.0010) [2023-12-26 20:45:14,887][105620] Updated weights for policy 1, policy_version 740686 (0.0008) [2023-12-26 20:45:14,939][105620] Updated weights for policy 1, policy_version 740696 (0.0009) [2023-12-26 20:45:14,998][105620] Updated weights for policy 1, policy_version 740706 (0.0008) [2023-12-26 20:45:15,147][105692] Updated weights for policy 0, policy_version 740402 (0.0009) [2023-12-26 20:45:15,210][105692] Updated weights for policy 0, policy_version 740412 (0.0009) [2023-12-26 20:45:15,272][105692] Updated weights for policy 0, policy_version 740422 (0.0009) [2023-12-26 20:45:15,762][105620] Updated weights for policy 1, policy_version 740716 (0.0009) [2023-12-26 20:45:15,809][105620] Updated weights for policy 1, policy_version 740726 (0.0008) [2023-12-26 20:45:15,855][105620] Updated weights for policy 1, policy_version 740736 (0.0006) [2023-12-26 20:45:15,998][105692] Updated weights for policy 0, policy_version 740432 (0.0008) [2023-12-26 20:45:16,045][105692] Updated weights for policy 0, policy_version 740442 (0.0008) [2023-12-26 20:45:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 379232256. Throughput: 0: 9754.7, 1: 9696.6. Samples: 379203016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:45:16,062][104569] Avg episode reward: [(0, '9166.813'), (1, '9264.097')] [2023-12-26 20:45:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000740744_189652992.pth... [2023-12-26 20:45:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000739592_189358080.pth [2023-12-26 20:45:16,098][105692] Updated weights for policy 0, policy_version 740452 (0.0009) [2023-12-26 20:45:16,121][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000740456_189587456.pth... [2023-12-26 20:45:16,127][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000739304_189292544.pth [2023-12-26 20:45:16,531][105620] Updated weights for policy 1, policy_version 740746 (0.0008) [2023-12-26 20:45:16,586][105620] Updated weights for policy 1, policy_version 740756 (0.0005) [2023-12-26 20:45:16,640][105620] Updated weights for policy 1, policy_version 740766 (0.0006) [2023-12-26 20:45:16,693][105620] Updated weights for policy 1, policy_version 740776 (0.0009) [2023-12-26 20:45:16,924][105692] Updated weights for policy 0, policy_version 740462 (0.0009) [2023-12-26 20:45:16,974][105692] Updated weights for policy 0, policy_version 740472 (0.0008) [2023-12-26 20:45:17,020][105692] Updated weights for policy 0, policy_version 740482 (0.0009) [2023-12-26 20:45:17,400][105620] Updated weights for policy 1, policy_version 740786 (0.0009) [2023-12-26 20:45:17,458][105620] Updated weights for policy 1, policy_version 740796 (0.0009) [2023-12-26 20:45:17,516][105620] Updated weights for policy 1, policy_version 740806 (0.0009) [2023-12-26 20:45:17,785][105692] Updated weights for policy 0, policy_version 740492 (0.0008) [2023-12-26 20:45:17,836][105692] Updated weights for policy 0, policy_version 740502 (0.0010) [2023-12-26 20:45:17,882][105692] Updated weights for policy 0, policy_version 740512 (0.0009) [2023-12-26 20:45:18,274][105620] Updated weights for policy 1, policy_version 740816 (0.0009) [2023-12-26 20:45:18,336][105620] Updated weights for policy 1, policy_version 740826 (0.0008) [2023-12-26 20:45:18,389][105620] Updated weights for policy 1, policy_version 740836 (0.0008) [2023-12-26 20:45:18,675][105692] Updated weights for policy 0, policy_version 740522 (0.0008) [2023-12-26 20:45:18,732][105692] Updated weights for policy 0, policy_version 740532 (0.0008) [2023-12-26 20:45:18,795][105692] Updated weights for policy 0, policy_version 740542 (0.0008) [2023-12-26 20:45:18,854][105692] Updated weights for policy 0, policy_version 740552 (0.0008) [2023-12-26 20:45:19,132][105620] Updated weights for policy 1, policy_version 740846 (0.0009) [2023-12-26 20:45:19,183][105620] Updated weights for policy 1, policy_version 740856 (0.0010) [2023-12-26 20:45:19,242][105620] Updated weights for policy 1, policy_version 740866 (0.0010) [2023-12-26 20:45:19,654][105692] Updated weights for policy 0, policy_version 740562 (0.0009) [2023-12-26 20:45:19,703][105692] Updated weights for policy 0, policy_version 740572 (0.0008) [2023-12-26 20:45:19,761][105692] Updated weights for policy 0, policy_version 740582 (0.0008) [2023-12-26 20:45:20,039][105620] Updated weights for policy 1, policy_version 740876 (0.0011) [2023-12-26 20:45:20,100][105620] Updated weights for policy 1, policy_version 740886 (0.0010) [2023-12-26 20:45:20,163][105620] Updated weights for policy 1, policy_version 740896 (0.0009) [2023-12-26 20:45:20,576][105692] Updated weights for policy 0, policy_version 740592 (0.0007) [2023-12-26 20:45:20,640][105692] Updated weights for policy 0, policy_version 740602 (0.0008) [2023-12-26 20:45:20,699][105692] Updated weights for policy 0, policy_version 740612 (0.0006) [2023-12-26 20:45:20,900][105620] Updated weights for policy 1, policy_version 740906 (0.0007) [2023-12-26 20:45:20,962][105620] Updated weights for policy 1, policy_version 740916 (0.0008) [2023-12-26 20:45:21,027][105620] Updated weights for policy 1, policy_version 740926 (0.0009) [2023-12-26 20:45:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 379322368. Throughput: 0: 9729.7, 1: 9675.9. Samples: 379317028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:45:21,062][104569] Avg episode reward: [(0, '8987.146'), (1, '9264.153')] [2023-12-26 20:45:21,096][105620] Updated weights for policy 1, policy_version 740936 (0.0009) [2023-12-26 20:45:21,411][105692] Updated weights for policy 0, policy_version 740622 (0.0008) [2023-12-26 20:45:21,481][105692] Updated weights for policy 0, policy_version 740632 (0.0008) [2023-12-26 20:45:21,546][105692] Updated weights for policy 0, policy_version 740642 (0.0009) [2023-12-26 20:45:21,859][105620] Updated weights for policy 1, policy_version 740946 (0.0009) [2023-12-26 20:45:21,915][105620] Updated weights for policy 1, policy_version 740956 (0.0009) [2023-12-26 20:45:21,963][105620] Updated weights for policy 1, policy_version 740966 (0.0009) [2023-12-26 20:45:22,288][105692] Updated weights for policy 0, policy_version 740652 (0.0009) [2023-12-26 20:45:22,350][105692] Updated weights for policy 0, policy_version 740662 (0.0009) [2023-12-26 20:45:22,413][105692] Updated weights for policy 0, policy_version 740672 (0.0009) [2023-12-26 20:45:22,740][105620] Updated weights for policy 1, policy_version 740976 (0.0009) [2023-12-26 20:45:22,787][105620] Updated weights for policy 1, policy_version 740986 (0.0009) [2023-12-26 20:45:22,838][105620] Updated weights for policy 1, policy_version 740996 (0.0009) [2023-12-26 20:45:23,152][105692] Updated weights for policy 0, policy_version 740682 (0.0009) [2023-12-26 20:45:23,210][105692] Updated weights for policy 0, policy_version 740692 (0.0009) [2023-12-26 20:45:23,272][105692] Updated weights for policy 0, policy_version 740702 (0.0009) [2023-12-26 20:45:23,334][105692] Updated weights for policy 0, policy_version 740712 (0.0009) [2023-12-26 20:45:23,622][105620] Updated weights for policy 1, policy_version 741006 (0.0007) [2023-12-26 20:45:23,672][105620] Updated weights for policy 1, policy_version 741016 (0.0005) [2023-12-26 20:45:23,726][105620] Updated weights for policy 1, policy_version 741026 (0.0006) [2023-12-26 20:45:24,181][105692] Updated weights for policy 0, policy_version 740722 (0.0008) [2023-12-26 20:45:24,235][105692] Updated weights for policy 0, policy_version 740732 (0.0009) [2023-12-26 20:45:24,267][105620] Updated weights for policy 1, policy_version 741036 (0.0006) [2023-12-26 20:45:24,286][105692] Updated weights for policy 0, policy_version 740742 (0.0006) [2023-12-26 20:45:24,325][105620] Updated weights for policy 1, policy_version 741046 (0.0009) [2023-12-26 20:45:24,394][105620] Updated weights for policy 1, policy_version 741056 (0.0008) [2023-12-26 20:45:25,070][105692] Updated weights for policy 0, policy_version 740752 (0.0008) [2023-12-26 20:45:25,071][105620] Updated weights for policy 1, policy_version 741066 (0.0007) [2023-12-26 20:45:25,113][105692] Updated weights for policy 0, policy_version 740762 (0.0007) [2023-12-26 20:45:25,127][105620] Updated weights for policy 1, policy_version 741076 (0.0007) [2023-12-26 20:45:25,173][105692] Updated weights for policy 0, policy_version 740772 (0.0009) [2023-12-26 20:45:25,181][105620] Updated weights for policy 1, policy_version 741086 (0.0005) [2023-12-26 20:45:25,248][105620] Updated weights for policy 1, policy_version 741096 (0.0005) [2023-12-26 20:45:25,816][105620] Updated weights for policy 1, policy_version 741106 (0.0009) [2023-12-26 20:45:25,877][105620] Updated weights for policy 1, policy_version 741116 (0.0009) [2023-12-26 20:45:25,929][105620] Updated weights for policy 1, policy_version 741126 (0.0008) [2023-12-26 20:45:26,007][105692] Updated weights for policy 0, policy_version 740782 (0.0009) [2023-12-26 20:45:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 379420672. Throughput: 0: 9643.9, 1: 9713.9. Samples: 379429624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:45:26,063][104569] Avg episode reward: [(0, '8901.323'), (1, '9264.159')] [2023-12-26 20:45:26,068][105692] Updated weights for policy 0, policy_version 740792 (0.0009) [2023-12-26 20:45:26,125][105692] Updated weights for policy 0, policy_version 740802 (0.0009) [2023-12-26 20:45:26,692][105620] Updated weights for policy 1, policy_version 741136 (0.0009) [2023-12-26 20:45:26,749][105620] Updated weights for policy 1, policy_version 741146 (0.0009) [2023-12-26 20:45:26,803][105620] Updated weights for policy 1, policy_version 741156 (0.0009) [2023-12-26 20:45:26,848][105692] Updated weights for policy 0, policy_version 740812 (0.0009) [2023-12-26 20:45:26,908][105692] Updated weights for policy 0, policy_version 740822 (0.0009) [2023-12-26 20:45:26,968][105692] Updated weights for policy 0, policy_version 740832 (0.0009) [2023-12-26 20:45:27,444][105620] Updated weights for policy 1, policy_version 741166 (0.0009) [2023-12-26 20:45:27,494][105620] Updated weights for policy 1, policy_version 741176 (0.0008) [2023-12-26 20:45:27,554][105620] Updated weights for policy 1, policy_version 741186 (0.0009) [2023-12-26 20:45:27,753][105692] Updated weights for policy 0, policy_version 740842 (0.0009) [2023-12-26 20:45:27,808][105692] Updated weights for policy 0, policy_version 740852 (0.0008) [2023-12-26 20:45:27,854][105692] Updated weights for policy 0, policy_version 740862 (0.0009) [2023-12-26 20:45:27,900][105692] Updated weights for policy 0, policy_version 740872 (0.0009) [2023-12-26 20:45:28,306][105620] Updated weights for policy 1, policy_version 741196 (0.0008) [2023-12-26 20:45:28,374][105620] Updated weights for policy 1, policy_version 741206 (0.0009) [2023-12-26 20:45:28,439][105620] Updated weights for policy 1, policy_version 741216 (0.0009) [2023-12-26 20:45:28,654][105692] Updated weights for policy 0, policy_version 740882 (0.0009) [2023-12-26 20:45:28,702][105692] Updated weights for policy 0, policy_version 740892 (0.0009) [2023-12-26 20:45:28,748][105692] Updated weights for policy 0, policy_version 740902 (0.0009) [2023-12-26 20:45:29,179][105620] Updated weights for policy 1, policy_version 741226 (0.0009) [2023-12-26 20:45:29,242][105620] Updated weights for policy 1, policy_version 741236 (0.0008) [2023-12-26 20:45:29,303][105620] Updated weights for policy 1, policy_version 741246 (0.0009) [2023-12-26 20:45:29,368][105620] Updated weights for policy 1, policy_version 741256 (0.0009) [2023-12-26 20:45:29,521][105692] Updated weights for policy 0, policy_version 740912 (0.0009) [2023-12-26 20:45:29,578][105692] Updated weights for policy 0, policy_version 740922 (0.0008) [2023-12-26 20:45:29,630][105692] Updated weights for policy 0, policy_version 740932 (0.0008) [2023-12-26 20:45:30,130][105620] Updated weights for policy 1, policy_version 741266 (0.0008) [2023-12-26 20:45:30,186][105620] Updated weights for policy 1, policy_version 741276 (0.0005) [2023-12-26 20:45:30,245][105620] Updated weights for policy 1, policy_version 741286 (0.0005) [2023-12-26 20:45:30,374][105692] Updated weights for policy 0, policy_version 740942 (0.0007) [2023-12-26 20:45:30,432][105692] Updated weights for policy 0, policy_version 740952 (0.0007) [2023-12-26 20:45:30,489][105692] Updated weights for policy 0, policy_version 740962 (0.0009) [2023-12-26 20:45:30,870][105620] Updated weights for policy 1, policy_version 741296 (0.0005) [2023-12-26 20:45:30,920][105620] Updated weights for policy 1, policy_version 741306 (0.0005) [2023-12-26 20:45:30,983][105620] Updated weights for policy 1, policy_version 741316 (0.0005) [2023-12-26 20:45:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 379518976. Throughput: 0: 9656.3, 1: 9727.2. Samples: 379486984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:45:31,062][104569] Avg episode reward: [(0, '8900.458'), (1, '9123.080')] [2023-12-26 20:45:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000741320_189800448.pth... [2023-12-26 20:45:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000740968_189718528.pth... [2023-12-26 20:45:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000740200_189513728.pth [2023-12-26 20:45:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000739880_189440000.pth [2023-12-26 20:45:31,152][105692] Updated weights for policy 0, policy_version 740972 (0.0008) [2023-12-26 20:45:31,212][105692] Updated weights for policy 0, policy_version 740982 (0.0009) [2023-12-26 20:45:31,273][105692] Updated weights for policy 0, policy_version 740992 (0.0009) [2023-12-26 20:45:31,662][105620] Updated weights for policy 1, policy_version 741326 (0.0008) [2023-12-26 20:45:31,721][105620] Updated weights for policy 1, policy_version 741336 (0.0009) [2023-12-26 20:45:31,768][105620] Updated weights for policy 1, policy_version 741346 (0.0008) [2023-12-26 20:45:32,014][105692] Updated weights for policy 0, policy_version 741002 (0.0009) [2023-12-26 20:45:32,071][105692] Updated weights for policy 0, policy_version 741012 (0.0008) [2023-12-26 20:45:32,132][105692] Updated weights for policy 0, policy_version 741022 (0.0008) [2023-12-26 20:45:32,182][105692] Updated weights for policy 0, policy_version 741032 (0.0008) [2023-12-26 20:45:32,527][105620] Updated weights for policy 1, policy_version 741356 (0.0008) [2023-12-26 20:45:32,584][105620] Updated weights for policy 1, policy_version 741366 (0.0008) [2023-12-26 20:45:32,639][105620] Updated weights for policy 1, policy_version 741376 (0.0009) [2023-12-26 20:45:32,930][105692] Updated weights for policy 0, policy_version 741042 (0.0008) [2023-12-26 20:45:32,985][105692] Updated weights for policy 0, policy_version 741052 (0.0008) [2023-12-26 20:45:33,045][105692] Updated weights for policy 0, policy_version 741062 (0.0005) [2023-12-26 20:45:33,446][105620] Updated weights for policy 1, policy_version 741386 (0.0009) [2023-12-26 20:45:33,497][105620] Updated weights for policy 1, policy_version 741396 (0.0009) [2023-12-26 20:45:33,553][105620] Updated weights for policy 1, policy_version 741406 (0.0010) [2023-12-26 20:45:33,600][105692] Updated weights for policy 0, policy_version 741072 (0.0006) [2023-12-26 20:45:33,609][105620] Updated weights for policy 1, policy_version 741416 (0.0008) [2023-12-26 20:45:33,651][105692] Updated weights for policy 0, policy_version 741082 (0.0009) [2023-12-26 20:45:33,696][105692] Updated weights for policy 0, policy_version 741092 (0.0008) [2023-12-26 20:45:34,248][105692] Updated weights for policy 0, policy_version 741102 (0.0007) [2023-12-26 20:45:34,304][105692] Updated weights for policy 0, policy_version 741112 (0.0008) [2023-12-26 20:45:34,366][105692] Updated weights for policy 0, policy_version 741122 (0.0009) [2023-12-26 20:45:34,469][105620] Updated weights for policy 1, policy_version 741426 (0.0009) [2023-12-26 20:45:34,526][105620] Updated weights for policy 1, policy_version 741436 (0.0010) [2023-12-26 20:45:34,582][105620] Updated weights for policy 1, policy_version 741446 (0.0010) [2023-12-26 20:45:35,068][105692] Updated weights for policy 0, policy_version 741132 (0.0009) [2023-12-26 20:45:35,129][105692] Updated weights for policy 0, policy_version 741142 (0.0009) [2023-12-26 20:45:35,187][105692] Updated weights for policy 0, policy_version 741152 (0.0009) [2023-12-26 20:45:35,389][105620] Updated weights for policy 1, policy_version 741456 (0.0009) [2023-12-26 20:45:35,442][105620] Updated weights for policy 1, policy_version 741466 (0.0009) [2023-12-26 20:45:35,508][105620] Updated weights for policy 1, policy_version 741476 (0.0005) [2023-12-26 20:45:35,776][105692] Updated weights for policy 0, policy_version 741162 (0.0006) [2023-12-26 20:45:35,836][105692] Updated weights for policy 0, policy_version 741172 (0.0009) [2023-12-26 20:45:35,890][105692] Updated weights for policy 0, policy_version 741182 (0.0007) [2023-12-26 20:45:35,943][105692] Updated weights for policy 0, policy_version 741192 (0.0005) [2023-12-26 20:45:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 379617280. Throughput: 0: 9693.0, 1: 9561.7. Samples: 379603616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:45:36,063][104569] Avg episode reward: [(0, '9167.200'), (1, '8674.765')] [2023-12-26 20:45:36,193][105620] Updated weights for policy 1, policy_version 741486 (0.0008) [2023-12-26 20:45:36,247][105620] Updated weights for policy 1, policy_version 741496 (0.0010) [2023-12-26 20:45:36,310][105620] Updated weights for policy 1, policy_version 741506 (0.0009) [2023-12-26 20:45:36,583][105692] Updated weights for policy 0, policy_version 741202 (0.0007) [2023-12-26 20:45:36,645][105692] Updated weights for policy 0, policy_version 741212 (0.0007) [2023-12-26 20:45:36,708][105692] Updated weights for policy 0, policy_version 741222 (0.0008) [2023-12-26 20:45:37,215][105620] Updated weights for policy 1, policy_version 741516 (0.0009) [2023-12-26 20:45:37,278][105620] Updated weights for policy 1, policy_version 741526 (0.0008) [2023-12-26 20:45:37,292][105692] Updated weights for policy 0, policy_version 741232 (0.0006) [2023-12-26 20:45:37,341][105620] Updated weights for policy 1, policy_version 741536 (0.0008) [2023-12-26 20:45:37,344][105692] Updated weights for policy 0, policy_version 741242 (0.0005) [2023-12-26 20:45:37,399][105692] Updated weights for policy 0, policy_version 741252 (0.0006) [2023-12-26 20:45:38,054][105692] Updated weights for policy 0, policy_version 741262 (0.0008) [2023-12-26 20:45:38,112][105692] Updated weights for policy 0, policy_version 741272 (0.0009) [2023-12-26 20:45:38,145][105620] Updated weights for policy 1, policy_version 741546 (0.0009) [2023-12-26 20:45:38,174][105692] Updated weights for policy 0, policy_version 741282 (0.0009) [2023-12-26 20:45:38,201][105620] Updated weights for policy 1, policy_version 741556 (0.0008) [2023-12-26 20:45:38,250][105620] Updated weights for policy 1, policy_version 741566 (0.0009) [2023-12-26 20:45:38,948][105620] Updated weights for policy 1, policy_version 741577 (0.0010) [2023-12-26 20:45:38,973][105692] Updated weights for policy 0, policy_version 741292 (0.0007) [2023-12-26 20:45:39,010][105620] Updated weights for policy 1, policy_version 741587 (0.0010) [2023-12-26 20:45:39,030][105692] Updated weights for policy 0, policy_version 741302 (0.0010) [2023-12-26 20:45:39,069][105620] Updated weights for policy 1, policy_version 741597 (0.0007) [2023-12-26 20:45:39,080][105692] Updated weights for policy 0, policy_version 741312 (0.0006) [2023-12-26 20:45:39,129][105620] Updated weights for policy 1, policy_version 741607 (0.0009) [2023-12-26 20:45:39,865][105692] Updated weights for policy 0, policy_version 741322 (0.0006) [2023-12-26 20:45:39,874][105620] Updated weights for policy 1, policy_version 741617 (0.0009) [2023-12-26 20:45:39,931][105692] Updated weights for policy 0, policy_version 741332 (0.0008) [2023-12-26 20:45:39,939][105620] Updated weights for policy 1, policy_version 741627 (0.0008) [2023-12-26 20:45:39,992][105692] Updated weights for policy 0, policy_version 741342 (0.0007) [2023-12-26 20:45:39,998][105620] Updated weights for policy 1, policy_version 741637 (0.0008) [2023-12-26 20:45:40,056][105692] Updated weights for policy 0, policy_version 741352 (0.0008) [2023-12-26 20:45:40,654][105620] Updated weights for policy 1, policy_version 741647 (0.0006) [2023-12-26 20:45:40,701][105620] Updated weights for policy 1, policy_version 741657 (0.0005) [2023-12-26 20:45:40,749][105620] Updated weights for policy 1, policy_version 741667 (0.0006) [2023-12-26 20:45:40,890][105692] Updated weights for policy 0, policy_version 741362 (0.0010) [2023-12-26 20:45:40,948][105692] Updated weights for policy 0, policy_version 741372 (0.0009) [2023-12-26 20:45:41,009][105692] Updated weights for policy 0, policy_version 741382 (0.0009) [2023-12-26 20:45:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 379715584. Throughput: 0: 9692.4, 1: 9517.3. Samples: 379720576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:45:41,063][104569] Avg episode reward: [(0, '9080.329'), (1, '8633.579')] [2023-12-26 20:45:41,415][105620] Updated weights for policy 1, policy_version 741677 (0.0007) [2023-12-26 20:45:41,478][105620] Updated weights for policy 1, policy_version 741687 (0.0009) [2023-12-26 20:45:41,542][105620] Updated weights for policy 1, policy_version 741697 (0.0009) [2023-12-26 20:45:41,875][105692] Updated weights for policy 0, policy_version 741392 (0.0007) [2023-12-26 20:45:41,939][105692] Updated weights for policy 0, policy_version 741402 (0.0010) [2023-12-26 20:45:41,996][105692] Updated weights for policy 0, policy_version 741412 (0.0006) [2023-12-26 20:45:42,376][105620] Updated weights for policy 1, policy_version 741707 (0.0009) [2023-12-26 20:45:42,438][105620] Updated weights for policy 1, policy_version 741717 (0.0009) [2023-12-26 20:45:42,497][105620] Updated weights for policy 1, policy_version 741727 (0.0009) [2023-12-26 20:45:42,683][105692] Updated weights for policy 0, policy_version 741422 (0.0009) [2023-12-26 20:45:42,733][105692] Updated weights for policy 0, policy_version 741432 (0.0007) [2023-12-26 20:45:42,792][105692] Updated weights for policy 0, policy_version 741442 (0.0009) [2023-12-26 20:45:43,309][105620] Updated weights for policy 1, policy_version 741737 (0.0008) [2023-12-26 20:45:43,374][105620] Updated weights for policy 1, policy_version 741747 (0.0009) [2023-12-26 20:45:43,430][105620] Updated weights for policy 1, policy_version 741757 (0.0008) [2023-12-26 20:45:43,471][105692] Updated weights for policy 0, policy_version 741452 (0.0009) [2023-12-26 20:45:43,485][105620] Updated weights for policy 1, policy_version 741767 (0.0007) [2023-12-26 20:45:43,520][105692] Updated weights for policy 0, policy_version 741462 (0.0010) [2023-12-26 20:45:43,579][105692] Updated weights for policy 0, policy_version 741472 (0.0009) [2023-12-26 20:45:44,215][105692] Updated weights for policy 0, policy_version 741482 (0.0010) [2023-12-26 20:45:44,278][105692] Updated weights for policy 0, policy_version 741492 (0.0008) [2023-12-26 20:45:44,342][105692] Updated weights for policy 0, policy_version 741502 (0.0008) [2023-12-26 20:45:44,345][105620] Updated weights for policy 1, policy_version 741777 (0.0009) [2023-12-26 20:45:44,402][105620] Updated weights for policy 1, policy_version 741787 (0.0008) [2023-12-26 20:45:44,407][105692] Updated weights for policy 0, policy_version 741512 (0.0009) [2023-12-26 20:45:44,458][105620] Updated weights for policy 1, policy_version 741797 (0.0010) [2023-12-26 20:45:45,124][105692] Updated weights for policy 0, policy_version 741522 (0.0010) [2023-12-26 20:45:45,189][105692] Updated weights for policy 0, policy_version 741532 (0.0008) [2023-12-26 20:45:45,199][105620] Updated weights for policy 1, policy_version 741807 (0.0010) [2023-12-26 20:45:45,251][105692] Updated weights for policy 0, policy_version 741542 (0.0007) [2023-12-26 20:45:45,265][105620] Updated weights for policy 1, policy_version 741817 (0.0009) [2023-12-26 20:45:45,324][105620] Updated weights for policy 1, policy_version 741827 (0.0009) [2023-12-26 20:45:46,023][105692] Updated weights for policy 0, policy_version 741552 (0.0009) [2023-12-26 20:45:46,062][104569] Fps is (10 sec: 18022.1, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 379797504. Throughput: 0: 9603.5, 1: 9465.8. Samples: 379774680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:45:46,063][104569] Avg episode reward: [(0, '9170.691'), (1, '8723.481')] [2023-12-26 20:45:46,078][105692] Updated weights for policy 0, policy_version 741562 (0.0007) [2023-12-26 20:45:46,080][105620] Updated weights for policy 1, policy_version 741837 (0.0008) [2023-12-26 20:45:46,130][105620] Updated weights for policy 1, policy_version 741847 (0.0007) [2023-12-26 20:45:46,141][105692] Updated weights for policy 0, policy_version 741572 (0.0008) [2023-12-26 20:45:46,162][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000741576_189874176.pth... [2023-12-26 20:45:46,166][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000740456_189587456.pth [2023-12-26 20:45:46,179][105620] Updated weights for policy 1, policy_version 741857 (0.0007) [2023-12-26 20:45:46,216][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000741864_189939712.pth... [2023-12-26 20:45:46,219][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000740744_189652992.pth [2023-12-26 20:45:46,865][105692] Updated weights for policy 0, policy_version 741582 (0.0008) [2023-12-26 20:45:46,930][105692] Updated weights for policy 0, policy_version 741592 (0.0009) [2023-12-26 20:45:46,949][105620] Updated weights for policy 1, policy_version 741867 (0.0009) [2023-12-26 20:45:46,987][105692] Updated weights for policy 0, policy_version 741602 (0.0008) [2023-12-26 20:45:47,006][105620] Updated weights for policy 1, policy_version 741877 (0.0006) [2023-12-26 20:45:47,065][105620] Updated weights for policy 1, policy_version 741887 (0.0008) [2023-12-26 20:45:47,671][105692] Updated weights for policy 0, policy_version 741612 (0.0006) [2023-12-26 20:45:47,730][105692] Updated weights for policy 0, policy_version 741622 (0.0011) [2023-12-26 20:45:47,783][105692] Updated weights for policy 0, policy_version 741632 (0.0006) [2023-12-26 20:45:47,815][105620] Updated weights for policy 1, policy_version 741897 (0.0008) [2023-12-26 20:45:47,865][105620] Updated weights for policy 1, policy_version 741907 (0.0009) [2023-12-26 20:45:47,919][105620] Updated weights for policy 1, policy_version 741917 (0.0009) [2023-12-26 20:45:47,970][105620] Updated weights for policy 1, policy_version 741927 (0.0008) [2023-12-26 20:45:48,449][105692] Updated weights for policy 0, policy_version 741642 (0.0006) [2023-12-26 20:45:48,521][105692] Updated weights for policy 0, policy_version 741652 (0.0008) [2023-12-26 20:45:48,586][105692] Updated weights for policy 0, policy_version 741662 (0.0010) [2023-12-26 20:45:48,653][105692] Updated weights for policy 0, policy_version 741672 (0.0011) [2023-12-26 20:45:48,804][105620] Updated weights for policy 1, policy_version 741937 (0.0008) [2023-12-26 20:45:48,863][105620] Updated weights for policy 1, policy_version 741947 (0.0008) [2023-12-26 20:45:48,917][105620] Updated weights for policy 1, policy_version 741957 (0.0008) [2023-12-26 20:45:49,369][105692] Updated weights for policy 0, policy_version 741682 (0.0009) [2023-12-26 20:45:49,434][105692] Updated weights for policy 0, policy_version 741692 (0.0009) [2023-12-26 20:45:49,489][105692] Updated weights for policy 0, policy_version 741702 (0.0008) [2023-12-26 20:45:49,721][105620] Updated weights for policy 1, policy_version 741967 (0.0008) [2023-12-26 20:45:49,787][105620] Updated weights for policy 1, policy_version 741977 (0.0007) [2023-12-26 20:45:49,854][105620] Updated weights for policy 1, policy_version 741987 (0.0008) [2023-12-26 20:45:50,283][105692] Updated weights for policy 0, policy_version 741712 (0.0008) [2023-12-26 20:45:50,336][105692] Updated weights for policy 0, policy_version 741722 (0.0008) [2023-12-26 20:45:50,397][105692] Updated weights for policy 0, policy_version 741732 (0.0008) [2023-12-26 20:45:50,634][105620] Updated weights for policy 1, policy_version 741997 (0.0011) [2023-12-26 20:45:50,708][105620] Updated weights for policy 1, policy_version 742007 (0.0011) [2023-12-26 20:45:50,775][105620] Updated weights for policy 1, policy_version 742017 (0.0011) [2023-12-26 20:45:51,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19114.7, 300 sec: 19549.7). Total num frames: 379895808. Throughput: 0: 9632.0, 1: 9358.9. Samples: 379886960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 20:45:51,063][104569] Avg episode reward: [(0, '9347.911'), (1, '8729.186')] [2023-12-26 20:45:51,140][105692] Updated weights for policy 0, policy_version 741742 (0.0008) [2023-12-26 20:45:51,201][105692] Updated weights for policy 0, policy_version 741752 (0.0008) [2023-12-26 20:45:51,271][105692] Updated weights for policy 0, policy_version 741762 (0.0009) [2023-12-26 20:45:51,543][105620] Updated weights for policy 1, policy_version 742027 (0.0010) [2023-12-26 20:45:51,588][105620] Updated weights for policy 1, policy_version 742037 (0.0008) [2023-12-26 20:45:51,651][105620] Updated weights for policy 1, policy_version 742047 (0.0008) [2023-12-26 20:45:52,052][105692] Updated weights for policy 0, policy_version 741772 (0.0010) [2023-12-26 20:45:52,105][105692] Updated weights for policy 0, policy_version 741782 (0.0010) [2023-12-26 20:45:52,152][105692] Updated weights for policy 0, policy_version 741792 (0.0009) [2023-12-26 20:45:52,332][105620] Updated weights for policy 1, policy_version 742057 (0.0007) [2023-12-26 20:45:52,397][105620] Updated weights for policy 1, policy_version 742067 (0.0008) [2023-12-26 20:45:52,451][105620] Updated weights for policy 1, policy_version 742077 (0.0005) [2023-12-26 20:45:52,507][105620] Updated weights for policy 1, policy_version 742087 (0.0005) [2023-12-26 20:45:52,847][105692] Updated weights for policy 0, policy_version 741802 (0.0009) [2023-12-26 20:45:52,911][105692] Updated weights for policy 0, policy_version 741812 (0.0009) [2023-12-26 20:45:52,974][105692] Updated weights for policy 0, policy_version 741822 (0.0009) [2023-12-26 20:45:53,041][105692] Updated weights for policy 0, policy_version 741832 (0.0008) [2023-12-26 20:45:53,064][105620] Updated weights for policy 1, policy_version 742097 (0.0010) [2023-12-26 20:45:53,115][105620] Updated weights for policy 1, policy_version 742107 (0.0008) [2023-12-26 20:45:53,161][105620] Updated weights for policy 1, policy_version 742117 (0.0008) [2023-12-26 20:45:53,679][105692] Updated weights for policy 0, policy_version 741842 (0.0005) [2023-12-26 20:45:53,725][105692] Updated weights for policy 0, policy_version 741852 (0.0005) [2023-12-26 20:45:53,772][105692] Updated weights for policy 0, policy_version 741862 (0.0005) [2023-12-26 20:45:53,936][105620] Updated weights for policy 1, policy_version 742127 (0.0007) [2023-12-26 20:45:53,997][105620] Updated weights for policy 1, policy_version 742137 (0.0006) [2023-12-26 20:45:54,056][105620] Updated weights for policy 1, policy_version 742147 (0.0005) [2023-12-26 20:45:54,310][105692] Updated weights for policy 0, policy_version 741872 (0.0006) [2023-12-26 20:45:54,366][105692] Updated weights for policy 0, policy_version 741882 (0.0006) [2023-12-26 20:45:54,412][105692] Updated weights for policy 0, policy_version 741892 (0.0005) [2023-12-26 20:45:54,633][105620] Updated weights for policy 1, policy_version 742157 (0.0006) [2023-12-26 20:45:54,683][105620] Updated weights for policy 1, policy_version 742167 (0.0006) [2023-12-26 20:45:54,742][105620] Updated weights for policy 1, policy_version 742177 (0.0005) [2023-12-26 20:45:55,172][105692] Updated weights for policy 0, policy_version 741902 (0.0007) [2023-12-26 20:45:55,242][105692] Updated weights for policy 0, policy_version 741912 (0.0005) [2023-12-26 20:45:55,310][105692] Updated weights for policy 0, policy_version 741922 (0.0006) [2023-12-26 20:45:55,317][105620] Updated weights for policy 1, policy_version 742187 (0.0005) [2023-12-26 20:45:55,367][105620] Updated weights for policy 1, policy_version 742197 (0.0008) [2023-12-26 20:45:55,422][105620] Updated weights for policy 1, policy_version 742207 (0.0009) [2023-12-26 20:45:55,857][105692] Updated weights for policy 0, policy_version 741932 (0.0008) [2023-12-26 20:45:55,910][105692] Updated weights for policy 0, policy_version 741942 (0.0005) [2023-12-26 20:45:55,967][105692] Updated weights for policy 0, policy_version 741952 (0.0008) [2023-12-26 20:45:56,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 380002304. Throughput: 0: 9669.8, 1: 9451.4. Samples: 380007000. Policy #0 lag: (min: 9.0, avg: 28.2, max: 41.0) [2023-12-26 20:45:56,063][104569] Avg episode reward: [(0, '9168.014'), (1, '8487.882')] [2023-12-26 20:45:56,278][105620] Updated weights for policy 1, policy_version 742217 (0.0009) [2023-12-26 20:45:56,328][105620] Updated weights for policy 1, policy_version 742227 (0.0009) [2023-12-26 20:45:56,375][105620] Updated weights for policy 1, policy_version 742237 (0.0009) [2023-12-26 20:45:56,432][105620] Updated weights for policy 1, policy_version 742247 (0.0008) [2023-12-26 20:45:56,676][105692] Updated weights for policy 0, policy_version 741962 (0.0008) [2023-12-26 20:45:56,730][105692] Updated weights for policy 0, policy_version 741972 (0.0005) [2023-12-26 20:45:56,783][105692] Updated weights for policy 0, policy_version 741982 (0.0007) [2023-12-26 20:45:56,833][105692] Updated weights for policy 0, policy_version 741992 (0.0008) [2023-12-26 20:45:57,227][105620] Updated weights for policy 1, policy_version 742257 (0.0008) [2023-12-26 20:45:57,271][105620] Updated weights for policy 1, policy_version 742267 (0.0008) [2023-12-26 20:45:57,320][105620] Updated weights for policy 1, policy_version 742277 (0.0008) [2023-12-26 20:45:57,513][105692] Updated weights for policy 0, policy_version 742002 (0.0010) [2023-12-26 20:45:57,564][105692] Updated weights for policy 0, policy_version 742012 (0.0010) [2023-12-26 20:45:57,611][105692] Updated weights for policy 0, policy_version 742022 (0.0010) [2023-12-26 20:45:58,037][105620] Updated weights for policy 1, policy_version 742287 (0.0007) [2023-12-26 20:45:58,092][105620] Updated weights for policy 1, policy_version 742297 (0.0008) [2023-12-26 20:45:58,151][105620] Updated weights for policy 1, policy_version 742307 (0.0008) [2023-12-26 20:45:58,390][105692] Updated weights for policy 0, policy_version 742032 (0.0010) [2023-12-26 20:45:58,451][105692] Updated weights for policy 0, policy_version 742042 (0.0008) [2023-12-26 20:45:58,521][105692] Updated weights for policy 0, policy_version 742052 (0.0008) [2023-12-26 20:45:58,955][105620] Updated weights for policy 1, policy_version 742317 (0.0008) [2023-12-26 20:45:59,020][105620] Updated weights for policy 1, policy_version 742327 (0.0010) [2023-12-26 20:45:59,074][105620] Updated weights for policy 1, policy_version 742337 (0.0008) [2023-12-26 20:45:59,258][105692] Updated weights for policy 0, policy_version 742062 (0.0008) [2023-12-26 20:45:59,318][105692] Updated weights for policy 0, policy_version 742072 (0.0005) [2023-12-26 20:45:59,381][105692] Updated weights for policy 0, policy_version 742082 (0.0006) [2023-12-26 20:45:59,711][105620] Updated weights for policy 1, policy_version 742347 (0.0006) [2023-12-26 20:45:59,764][105620] Updated weights for policy 1, policy_version 742357 (0.0010) [2023-12-26 20:45:59,823][105620] Updated weights for policy 1, policy_version 742367 (0.0010) [2023-12-26 20:46:00,167][105692] Updated weights for policy 0, policy_version 742092 (0.0007) [2023-12-26 20:46:00,233][105692] Updated weights for policy 0, policy_version 742102 (0.0009) [2023-12-26 20:46:00,302][105692] Updated weights for policy 0, policy_version 742112 (0.0008) [2023-12-26 20:46:00,525][105620] Updated weights for policy 1, policy_version 742377 (0.0010) [2023-12-26 20:46:00,577][105620] Updated weights for policy 1, policy_version 742387 (0.0007) [2023-12-26 20:46:00,631][105620] Updated weights for policy 1, policy_version 742397 (0.0005) [2023-12-26 20:46:00,680][105620] Updated weights for policy 1, policy_version 742407 (0.0005) [2023-12-26 20:46:00,966][105692] Updated weights for policy 0, policy_version 742122 (0.0008) [2023-12-26 20:46:01,032][105692] Updated weights for policy 0, policy_version 742132 (0.0006) [2023-12-26 20:46:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 18978.1, 300 sec: 19549.7). Total num frames: 380092416. Throughput: 0: 9684.9, 1: 9473.5. Samples: 380065144. Policy #0 lag: (min: 9.0, avg: 28.2, max: 41.0) [2023-12-26 20:46:01,062][104569] Avg episode reward: [(0, '8899.015'), (1, '7763.440')] [2023-12-26 20:46:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000742408_190078976.pth... [2023-12-26 20:46:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000741320_189800448.pth [2023-12-26 20:46:01,084][105692] Updated weights for policy 0, policy_version 742142 (0.0009) [2023-12-26 20:46:01,145][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000742152_190021632.pth... [2023-12-26 20:46:01,147][105692] Updated weights for policy 0, policy_version 742152 (0.0009) [2023-12-26 20:46:01,150][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000740968_189718528.pth [2023-12-26 20:46:01,255][105620] Updated weights for policy 1, policy_version 742417 (0.0007) [2023-12-26 20:46:01,318][105620] Updated weights for policy 1, policy_version 742427 (0.0008) [2023-12-26 20:46:01,384][105620] Updated weights for policy 1, policy_version 742437 (0.0009) [2023-12-26 20:46:01,919][105692] Updated weights for policy 0, policy_version 742162 (0.0009) [2023-12-26 20:46:01,987][105692] Updated weights for policy 0, policy_version 742172 (0.0008) [2023-12-26 20:46:02,048][105692] Updated weights for policy 0, policy_version 742182 (0.0008) [2023-12-26 20:46:02,104][105620] Updated weights for policy 1, policy_version 742447 (0.0007) [2023-12-26 20:46:02,150][105620] Updated weights for policy 1, policy_version 742457 (0.0006) [2023-12-26 20:46:02,204][105620] Updated weights for policy 1, policy_version 742467 (0.0008) [2023-12-26 20:46:02,823][105692] Updated weights for policy 0, policy_version 742192 (0.0010) [2023-12-26 20:46:02,883][105692] Updated weights for policy 0, policy_version 742202 (0.0008) [2023-12-26 20:46:02,946][105692] Updated weights for policy 0, policy_version 742212 (0.0010) [2023-12-26 20:46:02,952][105620] Updated weights for policy 1, policy_version 742477 (0.0007) [2023-12-26 20:46:03,002][105620] Updated weights for policy 1, policy_version 742487 (0.0007) [2023-12-26 20:46:03,052][105620] Updated weights for policy 1, policy_version 742497 (0.0009) [2023-12-26 20:46:03,509][105692] Updated weights for policy 0, policy_version 742222 (0.0007) [2023-12-26 20:46:03,557][105692] Updated weights for policy 0, policy_version 742232 (0.0005) [2023-12-26 20:46:03,607][105692] Updated weights for policy 0, policy_version 742242 (0.0005) [2023-12-26 20:46:03,843][105620] Updated weights for policy 1, policy_version 742507 (0.0010) [2023-12-26 20:46:03,911][105620] Updated weights for policy 1, policy_version 742517 (0.0011) [2023-12-26 20:46:03,978][105620] Updated weights for policy 1, policy_version 742527 (0.0006) [2023-12-26 20:46:04,303][105692] Updated weights for policy 0, policy_version 742252 (0.0007) [2023-12-26 20:46:04,365][105692] Updated weights for policy 0, policy_version 742262 (0.0009) [2023-12-26 20:46:04,427][105692] Updated weights for policy 0, policy_version 742272 (0.0009) [2023-12-26 20:46:04,689][105620] Updated weights for policy 1, policy_version 742537 (0.0008) [2023-12-26 20:46:04,743][105620] Updated weights for policy 1, policy_version 742547 (0.0008) [2023-12-26 20:46:04,798][105620] Updated weights for policy 1, policy_version 742557 (0.0007) [2023-12-26 20:46:04,847][105620] Updated weights for policy 1, policy_version 742567 (0.0008) [2023-12-26 20:46:05,161][105692] Updated weights for policy 0, policy_version 742282 (0.0009) [2023-12-26 20:46:05,209][105692] Updated weights for policy 0, policy_version 742292 (0.0009) [2023-12-26 20:46:05,270][105692] Updated weights for policy 0, policy_version 742302 (0.0009) [2023-12-26 20:46:05,337][105692] Updated weights for policy 0, policy_version 742312 (0.0010) [2023-12-26 20:46:05,590][105620] Updated weights for policy 1, policy_version 742578 (0.0008) [2023-12-26 20:46:05,642][105620] Updated weights for policy 1, policy_version 742588 (0.0005) [2023-12-26 20:46:05,703][105620] Updated weights for policy 1, policy_version 742598 (0.0005) [2023-12-26 20:46:06,027][105692] Updated weights for policy 0, policy_version 742322 (0.0008) [2023-12-26 20:46:06,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19114.6, 300 sec: 19577.5). Total num frames: 380190720. Throughput: 0: 9709.8, 1: 9518.9. Samples: 380182324. Policy #0 lag: (min: 9.0, avg: 28.2, max: 41.0) [2023-12-26 20:46:06,063][104569] Avg episode reward: [(0, '8988.765'), (1, '8099.007')] [2023-12-26 20:46:06,078][105692] Updated weights for policy 0, policy_version 742332 (0.0005) [2023-12-26 20:46:06,140][105692] Updated weights for policy 0, policy_version 742342 (0.0008) [2023-12-26 20:46:06,470][105620] Updated weights for policy 1, policy_version 742608 (0.0005) [2023-12-26 20:46:06,534][105620] Updated weights for policy 1, policy_version 742618 (0.0006) [2023-12-26 20:46:06,594][105620] Updated weights for policy 1, policy_version 742628 (0.0005) [2023-12-26 20:46:06,739][105692] Updated weights for policy 0, policy_version 742352 (0.0009) [2023-12-26 20:46:06,809][105692] Updated weights for policy 0, policy_version 742362 (0.0011) [2023-12-26 20:46:06,873][105692] Updated weights for policy 0, policy_version 742372 (0.0011) [2023-12-26 20:46:07,263][105620] Updated weights for policy 1, policy_version 742638 (0.0008) [2023-12-26 20:46:07,315][105620] Updated weights for policy 1, policy_version 742648 (0.0005) [2023-12-26 20:46:07,370][105620] Updated weights for policy 1, policy_version 742658 (0.0005) [2023-12-26 20:46:07,526][105692] Updated weights for policy 0, policy_version 742382 (0.0007) [2023-12-26 20:46:07,571][105692] Updated weights for policy 0, policy_version 742392 (0.0005) [2023-12-26 20:46:07,617][105692] Updated weights for policy 0, policy_version 742402 (0.0005) [2023-12-26 20:46:08,080][105620] Updated weights for policy 1, policy_version 742668 (0.0005) [2023-12-26 20:46:08,145][105620] Updated weights for policy 1, policy_version 742678 (0.0006) [2023-12-26 20:46:08,157][105692] Updated weights for policy 0, policy_version 742412 (0.0005) [2023-12-26 20:46:08,193][105620] Updated weights for policy 1, policy_version 742688 (0.0009) [2023-12-26 20:46:08,213][105692] Updated weights for policy 0, policy_version 742422 (0.0005) [2023-12-26 20:46:08,274][105692] Updated weights for policy 0, policy_version 742432 (0.0009) [2023-12-26 20:46:08,844][105620] Updated weights for policy 1, policy_version 742698 (0.0008) [2023-12-26 20:46:08,904][105620] Updated weights for policy 1, policy_version 742708 (0.0009) [2023-12-26 20:46:08,962][105620] Updated weights for policy 1, policy_version 742718 (0.0009) [2023-12-26 20:46:08,964][105692] Updated weights for policy 0, policy_version 742442 (0.0009) [2023-12-26 20:46:09,015][105620] Updated weights for policy 1, policy_version 742728 (0.0007) [2023-12-26 20:46:09,017][105692] Updated weights for policy 0, policy_version 742452 (0.0005) [2023-12-26 20:46:09,071][105692] Updated weights for policy 0, policy_version 742462 (0.0009) [2023-12-26 20:46:09,128][105692] Updated weights for policy 0, policy_version 742472 (0.0008) [2023-12-26 20:46:09,818][105620] Updated weights for policy 1, policy_version 742738 (0.0007) [2023-12-26 20:46:09,839][105692] Updated weights for policy 0, policy_version 742482 (0.0009) [2023-12-26 20:46:09,876][105620] Updated weights for policy 1, policy_version 742748 (0.0008) [2023-12-26 20:46:09,896][105692] Updated weights for policy 0, policy_version 742492 (0.0009) [2023-12-26 20:46:09,943][105620] Updated weights for policy 1, policy_version 742758 (0.0007) [2023-12-26 20:46:09,958][105692] Updated weights for policy 0, policy_version 742502 (0.0009) [2023-12-26 20:46:10,547][105620] Updated weights for policy 1, policy_version 742768 (0.0009) [2023-12-26 20:46:10,606][105620] Updated weights for policy 1, policy_version 742778 (0.0010) [2023-12-26 20:46:10,664][105620] Updated weights for policy 1, policy_version 742788 (0.0009) [2023-12-26 20:46:10,746][105692] Updated weights for policy 0, policy_version 742512 (0.0009) [2023-12-26 20:46:10,808][105692] Updated weights for policy 0, policy_version 742522 (0.0007) [2023-12-26 20:46:10,866][105692] Updated weights for policy 0, policy_version 742532 (0.0010) [2023-12-26 20:46:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 380297216. Throughput: 0: 9899.2, 1: 9513.9. Samples: 380303216. Policy #0 lag: (min: 9.0, avg: 28.2, max: 41.0) [2023-12-26 20:46:11,063][104569] Avg episode reward: [(0, '9174.945'), (1, '8810.058')] [2023-12-26 20:46:11,440][105620] Updated weights for policy 1, policy_version 742798 (0.0010) [2023-12-26 20:46:11,495][105620] Updated weights for policy 1, policy_version 742808 (0.0009) [2023-12-26 20:46:11,555][105620] Updated weights for policy 1, policy_version 742818 (0.0009) [2023-12-26 20:46:11,662][105692] Updated weights for policy 0, policy_version 742542 (0.0010) [2023-12-26 20:46:11,735][105692] Updated weights for policy 0, policy_version 742552 (0.0009) [2023-12-26 20:46:11,800][105692] Updated weights for policy 0, policy_version 742562 (0.0009) [2023-12-26 20:46:12,255][105620] Updated weights for policy 1, policy_version 742828 (0.0007) [2023-12-26 20:46:12,325][105620] Updated weights for policy 1, policy_version 742838 (0.0007) [2023-12-26 20:46:12,393][105620] Updated weights for policy 1, policy_version 742848 (0.0009) [2023-12-26 20:46:12,585][105692] Updated weights for policy 0, policy_version 742572 (0.0009) [2023-12-26 20:46:12,652][105692] Updated weights for policy 0, policy_version 742582 (0.0008) [2023-12-26 20:46:12,708][105692] Updated weights for policy 0, policy_version 742592 (0.0009) [2023-12-26 20:46:13,080][105620] Updated weights for policy 1, policy_version 742858 (0.0008) [2023-12-26 20:46:13,137][105620] Updated weights for policy 1, policy_version 742868 (0.0005) [2023-12-26 20:46:13,199][105620] Updated weights for policy 1, policy_version 742878 (0.0005) [2023-12-26 20:46:13,260][105620] Updated weights for policy 1, policy_version 742888 (0.0005) [2023-12-26 20:46:13,397][105692] Updated weights for policy 0, policy_version 742602 (0.0009) [2023-12-26 20:46:13,466][105692] Updated weights for policy 0, policy_version 742612 (0.0006) [2023-12-26 20:46:13,515][105692] Updated weights for policy 0, policy_version 742622 (0.0005) [2023-12-26 20:46:13,565][105692] Updated weights for policy 0, policy_version 742632 (0.0007) [2023-12-26 20:46:13,876][105620] Updated weights for policy 1, policy_version 742898 (0.0007) [2023-12-26 20:46:13,930][105620] Updated weights for policy 1, policy_version 742909 (0.0010) [2023-12-26 20:46:13,982][105620] Updated weights for policy 1, policy_version 742919 (0.0010) [2023-12-26 20:46:14,134][105692] Updated weights for policy 0, policy_version 742642 (0.0008) [2023-12-26 20:46:14,189][105692] Updated weights for policy 0, policy_version 742652 (0.0005) [2023-12-26 20:46:14,245][105692] Updated weights for policy 0, policy_version 742662 (0.0006) [2023-12-26 20:46:14,727][105620] Updated weights for policy 1, policy_version 742929 (0.0005) [2023-12-26 20:46:14,787][105620] Updated weights for policy 1, policy_version 742939 (0.0007) [2023-12-26 20:46:14,848][105620] Updated weights for policy 1, policy_version 742949 (0.0008) [2023-12-26 20:46:14,848][105692] Updated weights for policy 0, policy_version 742672 (0.0008) [2023-12-26 20:46:14,917][105692] Updated weights for policy 0, policy_version 742682 (0.0008) [2023-12-26 20:46:14,990][105692] Updated weights for policy 0, policy_version 742692 (0.0008) [2023-12-26 20:46:15,506][105620] Updated weights for policy 1, policy_version 742959 (0.0009) [2023-12-26 20:46:15,572][105620] Updated weights for policy 1, policy_version 742969 (0.0007) [2023-12-26 20:46:15,636][105620] Updated weights for policy 1, policy_version 742979 (0.0005) [2023-12-26 20:46:15,689][105692] Updated weights for policy 0, policy_version 742702 (0.0010) [2023-12-26 20:46:15,750][105692] Updated weights for policy 0, policy_version 742712 (0.0010) [2023-12-26 20:46:15,808][105692] Updated weights for policy 0, policy_version 742722 (0.0010) [2023-12-26 20:46:16,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 380395520. Throughput: 0: 9897.7, 1: 9538.7. Samples: 380361620. Policy #0 lag: (min: 9.0, avg: 28.2, max: 41.0) [2023-12-26 20:46:16,062][104569] Avg episode reward: [(0, '9079.522'), (1, '8993.517')] [2023-12-26 20:46:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000742984_190226432.pth... [2023-12-26 20:46:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000742728_190169088.pth... [2023-12-26 20:46:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000741864_189939712.pth [2023-12-26 20:46:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000741576_189874176.pth [2023-12-26 20:46:16,274][105620] Updated weights for policy 1, policy_version 742989 (0.0008) [2023-12-26 20:46:16,333][105620] Updated weights for policy 1, policy_version 742999 (0.0006) [2023-12-26 20:46:16,401][105620] Updated weights for policy 1, policy_version 743009 (0.0007) [2023-12-26 20:46:16,545][105692] Updated weights for policy 0, policy_version 742732 (0.0010) [2023-12-26 20:46:16,601][105692] Updated weights for policy 0, policy_version 742742 (0.0010) [2023-12-26 20:46:16,662][105692] Updated weights for policy 0, policy_version 742752 (0.0010) [2023-12-26 20:46:17,025][105620] Updated weights for policy 1, policy_version 743019 (0.0006) [2023-12-26 20:46:17,077][105620] Updated weights for policy 1, policy_version 743029 (0.0008) [2023-12-26 20:46:17,126][105620] Updated weights for policy 1, policy_version 743039 (0.0008) [2023-12-26 20:46:17,406][105692] Updated weights for policy 0, policy_version 742762 (0.0010) [2023-12-26 20:46:17,461][105692] Updated weights for policy 0, policy_version 742772 (0.0010) [2023-12-26 20:46:17,516][105692] Updated weights for policy 0, policy_version 742782 (0.0010) [2023-12-26 20:46:17,575][105692] Updated weights for policy 0, policy_version 742792 (0.0009) [2023-12-26 20:46:17,956][105620] Updated weights for policy 1, policy_version 743049 (0.0008) [2023-12-26 20:46:18,009][105620] Updated weights for policy 1, policy_version 743059 (0.0009) [2023-12-26 20:46:18,064][105620] Updated weights for policy 1, policy_version 743069 (0.0009) [2023-12-26 20:46:18,128][105620] Updated weights for policy 1, policy_version 743079 (0.0009) [2023-12-26 20:46:18,215][105692] Updated weights for policy 0, policy_version 742802 (0.0009) [2023-12-26 20:46:18,273][105692] Updated weights for policy 0, policy_version 742812 (0.0007) [2023-12-26 20:46:18,319][105692] Updated weights for policy 0, policy_version 742822 (0.0007) [2023-12-26 20:46:18,851][105620] Updated weights for policy 1, policy_version 743089 (0.0006) [2023-12-26 20:46:18,902][105620] Updated weights for policy 1, policy_version 743099 (0.0005) [2023-12-26 20:46:18,966][105692] Updated weights for policy 0, policy_version 742832 (0.0006) [2023-12-26 20:46:18,966][105620] Updated weights for policy 1, policy_version 743109 (0.0006) [2023-12-26 20:46:19,022][105692] Updated weights for policy 0, policy_version 742842 (0.0006) [2023-12-26 20:46:19,086][105692] Updated weights for policy 0, policy_version 742852 (0.0005) [2023-12-26 20:46:19,618][105620] Updated weights for policy 1, policy_version 743119 (0.0009) [2023-12-26 20:46:19,688][105620] Updated weights for policy 1, policy_version 743129 (0.0011) [2023-12-26 20:46:19,705][105692] Updated weights for policy 0, policy_version 742862 (0.0005) [2023-12-26 20:46:19,750][105620] Updated weights for policy 1, policy_version 743139 (0.0011) [2023-12-26 20:46:19,768][105692] Updated weights for policy 0, policy_version 742872 (0.0007) [2023-12-26 20:46:19,831][105692] Updated weights for policy 0, policy_version 742882 (0.0010) [2023-12-26 20:46:20,422][105692] Updated weights for policy 0, policy_version 742892 (0.0009) [2023-12-26 20:46:20,493][105692] Updated weights for policy 0, policy_version 742902 (0.0008) [2023-12-26 20:46:20,501][105620] Updated weights for policy 1, policy_version 743149 (0.0008) [2023-12-26 20:46:20,559][105692] Updated weights for policy 0, policy_version 742912 (0.0007) [2023-12-26 20:46:20,564][105620] Updated weights for policy 1, policy_version 743159 (0.0007) [2023-12-26 20:46:20,631][105620] Updated weights for policy 1, policy_version 743169 (0.0010) [2023-12-26 20:46:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 380493824. Throughput: 0: 9927.4, 1: 9621.2. Samples: 380483300. Policy #0 lag: (min: 9.0, avg: 28.2, max: 41.0) [2023-12-26 20:46:21,062][104569] Avg episode reward: [(0, '9257.735'), (1, '9267.759')] [2023-12-26 20:46:21,303][105692] Updated weights for policy 0, policy_version 742922 (0.0012) [2023-12-26 20:46:21,363][105692] Updated weights for policy 0, policy_version 742932 (0.0008) [2023-12-26 20:46:21,373][105620] Updated weights for policy 1, policy_version 743179 (0.0010) [2023-12-26 20:46:21,425][105692] Updated weights for policy 0, policy_version 742942 (0.0007) [2023-12-26 20:46:21,438][105620] Updated weights for policy 1, policy_version 743189 (0.0010) [2023-12-26 20:46:21,486][105692] Updated weights for policy 0, policy_version 742952 (0.0006) [2023-12-26 20:46:21,501][105620] Updated weights for policy 1, policy_version 743199 (0.0011) [2023-12-26 20:46:22,172][105692] Updated weights for policy 0, policy_version 742962 (0.0006) [2023-12-26 20:46:22,190][105620] Updated weights for policy 1, policy_version 743209 (0.0010) [2023-12-26 20:46:22,235][105692] Updated weights for policy 0, policy_version 742972 (0.0006) [2023-12-26 20:46:22,251][105620] Updated weights for policy 1, policy_version 743219 (0.0010) [2023-12-26 20:46:22,300][105692] Updated weights for policy 0, policy_version 742982 (0.0008) [2023-12-26 20:46:22,323][105620] Updated weights for policy 1, policy_version 743230 (0.0010) [2023-12-26 20:46:22,934][105692] Updated weights for policy 0, policy_version 742992 (0.0006) [2023-12-26 20:46:22,979][105692] Updated weights for policy 0, policy_version 743002 (0.0005) [2023-12-26 20:46:23,006][105620] Updated weights for policy 1, policy_version 743241 (0.0009) [2023-12-26 20:46:23,034][105692] Updated weights for policy 0, policy_version 743012 (0.0005) [2023-12-26 20:46:23,062][105620] Updated weights for policy 1, policy_version 743251 (0.0011) [2023-12-26 20:46:23,124][105620] Updated weights for policy 1, policy_version 743261 (0.0011) [2023-12-26 20:46:23,183][105620] Updated weights for policy 1, policy_version 743271 (0.0010) [2023-12-26 20:46:23,580][105692] Updated weights for policy 0, policy_version 743022 (0.0006) [2023-12-26 20:46:23,631][105692] Updated weights for policy 0, policy_version 743032 (0.0005) [2023-12-26 20:46:23,692][105692] Updated weights for policy 0, policy_version 743042 (0.0005) [2023-12-26 20:46:23,932][105620] Updated weights for policy 1, policy_version 743281 (0.0009) [2023-12-26 20:46:23,990][105620] Updated weights for policy 1, policy_version 743291 (0.0010) [2023-12-26 20:46:24,046][105620] Updated weights for policy 1, policy_version 743301 (0.0009) [2023-12-26 20:46:24,258][105692] Updated weights for policy 0, policy_version 743052 (0.0006) [2023-12-26 20:46:24,308][105692] Updated weights for policy 0, policy_version 743062 (0.0005) [2023-12-26 20:46:24,363][105692] Updated weights for policy 0, policy_version 743072 (0.0005) [2023-12-26 20:46:24,846][105620] Updated weights for policy 1, policy_version 743311 (0.0007) [2023-12-26 20:46:24,911][105620] Updated weights for policy 1, policy_version 743321 (0.0005) [2023-12-26 20:46:24,918][105692] Updated weights for policy 0, policy_version 743082 (0.0006) [2023-12-26 20:46:24,973][105620] Updated weights for policy 1, policy_version 743331 (0.0006) [2023-12-26 20:46:24,973][105692] Updated weights for policy 0, policy_version 743092 (0.0010) [2023-12-26 20:46:25,029][105692] Updated weights for policy 0, policy_version 743102 (0.0011) [2023-12-26 20:46:25,090][105692] Updated weights for policy 0, policy_version 743112 (0.0010) [2023-12-26 20:46:25,578][105620] Updated weights for policy 1, policy_version 743341 (0.0011) [2023-12-26 20:46:25,623][105620] Updated weights for policy 1, policy_version 743351 (0.0010) [2023-12-26 20:46:25,671][105620] Updated weights for policy 1, policy_version 743361 (0.0010) [2023-12-26 20:46:25,814][105692] Updated weights for policy 0, policy_version 743122 (0.0007) [2023-12-26 20:46:25,877][105692] Updated weights for policy 0, policy_version 743132 (0.0005) [2023-12-26 20:46:25,939][105692] Updated weights for policy 0, policy_version 743142 (0.0005) [2023-12-26 20:46:26,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 380600320. Throughput: 0: 10023.1, 1: 9636.0. Samples: 380605236. Policy #0 lag: (min: 9.0, avg: 28.2, max: 41.0) [2023-12-26 20:46:26,063][104569] Avg episode reward: [(0, '9255.857'), (1, '9083.376')] [2023-12-26 20:46:26,321][105620] Updated weights for policy 1, policy_version 743371 (0.0009) [2023-12-26 20:46:26,387][105620] Updated weights for policy 1, policy_version 743381 (0.0005) [2023-12-26 20:46:26,452][105620] Updated weights for policy 1, policy_version 743391 (0.0006) [2023-12-26 20:46:26,589][105692] Updated weights for policy 0, policy_version 743152 (0.0006) [2023-12-26 20:46:26,639][105692] Updated weights for policy 0, policy_version 743162 (0.0005) [2023-12-26 20:46:26,694][105692] Updated weights for policy 0, policy_version 743172 (0.0005) [2023-12-26 20:46:27,120][105620] Updated weights for policy 1, policy_version 743401 (0.0006) [2023-12-26 20:46:27,189][105620] Updated weights for policy 1, policy_version 743411 (0.0005) [2023-12-26 20:46:27,251][105620] Updated weights for policy 1, policy_version 743421 (0.0006) [2023-12-26 20:46:27,316][105620] Updated weights for policy 1, policy_version 743431 (0.0007) [2023-12-26 20:46:27,349][105692] Updated weights for policy 0, policy_version 743182 (0.0008) [2023-12-26 20:46:27,408][105692] Updated weights for policy 0, policy_version 743192 (0.0009) [2023-12-26 20:46:27,474][105692] Updated weights for policy 0, policy_version 743202 (0.0008) [2023-12-26 20:46:27,958][105620] Updated weights for policy 1, policy_version 743441 (0.0006) [2023-12-26 20:46:28,011][105620] Updated weights for policy 1, policy_version 743451 (0.0010) [2023-12-26 20:46:28,052][105692] Updated weights for policy 0, policy_version 743212 (0.0005) [2023-12-26 20:46:28,066][105620] Updated weights for policy 1, policy_version 743461 (0.0006) [2023-12-26 20:46:28,112][105692] Updated weights for policy 0, policy_version 743222 (0.0010) [2023-12-26 20:46:28,163][105692] Updated weights for policy 0, policy_version 743232 (0.0010) [2023-12-26 20:46:28,658][105620] Updated weights for policy 1, policy_version 743471 (0.0007) [2023-12-26 20:46:28,705][105620] Updated weights for policy 1, policy_version 743481 (0.0007) [2023-12-26 20:46:28,758][105620] Updated weights for policy 1, policy_version 743491 (0.0006) [2023-12-26 20:46:28,889][105692] Updated weights for policy 0, policy_version 743242 (0.0010) [2023-12-26 20:46:28,944][105692] Updated weights for policy 0, policy_version 743252 (0.0010) [2023-12-26 20:46:28,991][105692] Updated weights for policy 0, policy_version 743262 (0.0010) [2023-12-26 20:46:29,035][105692] Updated weights for policy 0, policy_version 743272 (0.0010) [2023-12-26 20:46:29,490][105620] Updated weights for policy 1, policy_version 743501 (0.0010) [2023-12-26 20:46:29,555][105620] Updated weights for policy 1, policy_version 743511 (0.0010) [2023-12-26 20:46:29,624][105620] Updated weights for policy 1, policy_version 743521 (0.0008) [2023-12-26 20:46:29,798][105692] Updated weights for policy 0, policy_version 743282 (0.0009) [2023-12-26 20:46:29,857][105692] Updated weights for policy 0, policy_version 743292 (0.0008) [2023-12-26 20:46:29,915][105692] Updated weights for policy 0, policy_version 743302 (0.0009) [2023-12-26 20:46:30,228][105620] Updated weights for policy 1, policy_version 743531 (0.0007) [2023-12-26 20:46:30,290][105620] Updated weights for policy 1, policy_version 743541 (0.0010) [2023-12-26 20:46:30,357][105620] Updated weights for policy 1, policy_version 743551 (0.0009) [2023-12-26 20:46:30,541][105692] Updated weights for policy 0, policy_version 743312 (0.0010) [2023-12-26 20:46:30,598][105692] Updated weights for policy 0, policy_version 743322 (0.0010) [2023-12-26 20:46:30,654][105692] Updated weights for policy 0, policy_version 743332 (0.0011) [2023-12-26 20:46:30,972][105620] Updated weights for policy 1, policy_version 743561 (0.0008) [2023-12-26 20:46:31,026][105620] Updated weights for policy 1, policy_version 743571 (0.0005) [2023-12-26 20:46:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 380698624. Throughput: 0: 10103.1, 1: 9781.2. Samples: 380669468. Policy #0 lag: (min: 9.0, avg: 28.2, max: 41.0) [2023-12-26 20:46:31,062][104569] Avg episode reward: [(0, '9164.291'), (1, '8901.231')] [2023-12-26 20:46:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000743336_190324736.pth... [2023-12-26 20:46:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000742152_190021632.pth [2023-12-26 20:46:31,085][105620] Updated weights for policy 1, policy_version 743581 (0.0010) [2023-12-26 20:46:31,147][105620] Updated weights for policy 1, policy_version 743591 (0.0006) [2023-12-26 20:46:31,152][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000743592_190382080.pth... [2023-12-26 20:46:31,156][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000742408_190078976.pth [2023-12-26 20:46:31,352][105692] Updated weights for policy 0, policy_version 743342 (0.0009) [2023-12-26 20:46:31,418][105692] Updated weights for policy 0, policy_version 743352 (0.0007) [2023-12-26 20:46:31,476][105692] Updated weights for policy 0, policy_version 743362 (0.0009) [2023-12-26 20:46:31,837][105620] Updated weights for policy 1, policy_version 743601 (0.0009) [2023-12-26 20:46:31,898][105620] Updated weights for policy 1, policy_version 743611 (0.0009) [2023-12-26 20:46:31,959][105620] Updated weights for policy 1, policy_version 743621 (0.0009) [2023-12-26 20:46:32,279][105692] Updated weights for policy 0, policy_version 743372 (0.0010) [2023-12-26 20:46:32,337][105692] Updated weights for policy 0, policy_version 743382 (0.0009) [2023-12-26 20:46:32,404][105692] Updated weights for policy 0, policy_version 743392 (0.0008) [2023-12-26 20:46:32,633][105620] Updated weights for policy 1, policy_version 743631 (0.0005) [2023-12-26 20:46:32,690][105620] Updated weights for policy 1, policy_version 743641 (0.0005) [2023-12-26 20:46:32,747][105620] Updated weights for policy 1, policy_version 743651 (0.0009) [2023-12-26 20:46:33,131][105692] Updated weights for policy 0, policy_version 743402 (0.0007) [2023-12-26 20:46:33,186][105692] Updated weights for policy 0, policy_version 743412 (0.0010) [2023-12-26 20:46:33,247][105692] Updated weights for policy 0, policy_version 743422 (0.0010) [2023-12-26 20:46:33,304][105692] Updated weights for policy 0, policy_version 743432 (0.0008) [2023-12-26 20:46:33,435][105620] Updated weights for policy 1, policy_version 743661 (0.0010) [2023-12-26 20:46:33,483][105620] Updated weights for policy 1, policy_version 743671 (0.0010) [2023-12-26 20:46:33,530][105620] Updated weights for policy 1, policy_version 743681 (0.0010) [2023-12-26 20:46:33,883][105692] Updated weights for policy 0, policy_version 743442 (0.0006) [2023-12-26 20:46:33,929][105692] Updated weights for policy 0, policy_version 743452 (0.0006) [2023-12-26 20:46:33,975][105692] Updated weights for policy 0, policy_version 743462 (0.0005) [2023-12-26 20:46:34,229][105620] Updated weights for policy 1, policy_version 743691 (0.0009) [2023-12-26 20:46:34,293][105620] Updated weights for policy 1, policy_version 743701 (0.0006) [2023-12-26 20:46:34,353][105620] Updated weights for policy 1, policy_version 743711 (0.0009) [2023-12-26 20:46:34,677][105692] Updated weights for policy 0, policy_version 743472 (0.0008) [2023-12-26 20:46:34,742][105692] Updated weights for policy 0, policy_version 743482 (0.0008) [2023-12-26 20:46:34,808][105692] Updated weights for policy 0, policy_version 743492 (0.0009) [2023-12-26 20:46:34,997][105620] Updated weights for policy 1, policy_version 743721 (0.0009) [2023-12-26 20:46:35,060][105620] Updated weights for policy 1, policy_version 743731 (0.0011) [2023-12-26 20:46:35,119][105620] Updated weights for policy 1, policy_version 743741 (0.0010) [2023-12-26 20:46:35,176][105620] Updated weights for policy 1, policy_version 743751 (0.0010) [2023-12-26 20:46:35,527][105692] Updated weights for policy 0, policy_version 743502 (0.0009) [2023-12-26 20:46:35,581][105692] Updated weights for policy 0, policy_version 743512 (0.0010) [2023-12-26 20:46:35,635][105692] Updated weights for policy 0, policy_version 743522 (0.0010) [2023-12-26 20:46:35,899][105620] Updated weights for policy 1, policy_version 743761 (0.0010) [2023-12-26 20:46:35,957][105620] Updated weights for policy 1, policy_version 743771 (0.0010) [2023-12-26 20:46:36,011][105620] Updated weights for policy 1, policy_version 743781 (0.0010) [2023-12-26 20:46:36,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 380805120. Throughput: 0: 10146.7, 1: 9954.0. Samples: 380791488. Policy #0 lag: (min: 9.0, avg: 28.2, max: 41.0) [2023-12-26 20:46:36,062][104569] Avg episode reward: [(0, '9255.551'), (1, '8992.009')] [2023-12-26 20:46:36,266][105692] Updated weights for policy 0, policy_version 743532 (0.0009) [2023-12-26 20:46:36,321][105692] Updated weights for policy 0, policy_version 743542 (0.0010) [2023-12-26 20:46:36,370][105692] Updated weights for policy 0, policy_version 743552 (0.0010) [2023-12-26 20:46:36,705][105620] Updated weights for policy 1, policy_version 743791 (0.0007) [2023-12-26 20:46:36,775][105620] Updated weights for policy 1, policy_version 743801 (0.0007) [2023-12-26 20:46:36,843][105620] Updated weights for policy 1, policy_version 743811 (0.0005) [2023-12-26 20:46:37,053][105692] Updated weights for policy 0, policy_version 743562 (0.0010) [2023-12-26 20:46:37,106][105692] Updated weights for policy 0, policy_version 743572 (0.0010) [2023-12-26 20:46:37,160][105692] Updated weights for policy 0, policy_version 743582 (0.0010) [2023-12-26 20:46:37,215][105692] Updated weights for policy 0, policy_version 743592 (0.0007) [2023-12-26 20:46:37,475][105620] Updated weights for policy 1, policy_version 743821 (0.0009) [2023-12-26 20:46:37,542][105620] Updated weights for policy 1, policy_version 743831 (0.0011) [2023-12-26 20:46:37,601][105620] Updated weights for policy 1, policy_version 743841 (0.0011) [2023-12-26 20:46:37,805][105692] Updated weights for policy 0, policy_version 743602 (0.0005) [2023-12-26 20:46:37,863][105692] Updated weights for policy 0, policy_version 743612 (0.0006) [2023-12-26 20:46:37,908][105692] Updated weights for policy 0, policy_version 743622 (0.0005) [2023-12-26 20:46:38,360][105620] Updated weights for policy 1, policy_version 743851 (0.0010) [2023-12-26 20:46:38,424][105620] Updated weights for policy 1, policy_version 743861 (0.0007) [2023-12-26 20:46:38,436][105692] Updated weights for policy 0, policy_version 743632 (0.0006) [2023-12-26 20:46:38,485][105692] Updated weights for policy 0, policy_version 743642 (0.0005) [2023-12-26 20:46:38,494][105620] Updated weights for policy 1, policy_version 743871 (0.0011) [2023-12-26 20:46:38,544][105692] Updated weights for policy 0, policy_version 743652 (0.0007) [2023-12-26 20:46:39,107][105692] Updated weights for policy 0, policy_version 743662 (0.0006) [2023-12-26 20:46:39,152][105620] Updated weights for policy 1, policy_version 743881 (0.0008) [2023-12-26 20:46:39,172][105692] Updated weights for policy 0, policy_version 743672 (0.0008) [2023-12-26 20:46:39,210][105620] Updated weights for policy 1, policy_version 743891 (0.0006) [2023-12-26 20:46:39,237][105692] Updated weights for policy 0, policy_version 743682 (0.0010) [2023-12-26 20:46:39,271][105620] Updated weights for policy 1, policy_version 743901 (0.0009) [2023-12-26 20:46:39,324][105620] Updated weights for policy 1, policy_version 743911 (0.0010) [2023-12-26 20:46:39,917][105692] Updated weights for policy 0, policy_version 743692 (0.0009) [2023-12-26 20:46:39,991][105692] Updated weights for policy 0, policy_version 743702 (0.0011) [2023-12-26 20:46:40,036][105620] Updated weights for policy 1, policy_version 743921 (0.0006) [2023-12-26 20:46:40,052][105692] Updated weights for policy 0, policy_version 743712 (0.0011) [2023-12-26 20:46:40,092][105620] Updated weights for policy 1, policy_version 743931 (0.0006) [2023-12-26 20:46:40,149][105620] Updated weights for policy 1, policy_version 743941 (0.0006) [2023-12-26 20:46:40,794][105692] Updated weights for policy 0, policy_version 743722 (0.0009) [2023-12-26 20:46:40,837][105620] Updated weights for policy 1, policy_version 743951 (0.0009) [2023-12-26 20:46:40,853][105692] Updated weights for policy 0, policy_version 743732 (0.0006) [2023-12-26 20:46:40,884][105620] Updated weights for policy 1, policy_version 743961 (0.0009) [2023-12-26 20:46:40,912][105692] Updated weights for policy 0, policy_version 743742 (0.0005) [2023-12-26 20:46:40,943][105620] Updated weights for policy 1, policy_version 743971 (0.0009) [2023-12-26 20:46:40,978][105692] Updated weights for policy 0, policy_version 743752 (0.0005) [2023-12-26 20:46:41,062][104569] Fps is (10 sec: 21299.3, 60 sec: 19933.9, 300 sec: 19688.6). Total num frames: 380911616. Throughput: 0: 10231.4, 1: 9930.5. Samples: 380914284. Policy #0 lag: (min: 9.0, avg: 28.2, max: 41.0) [2023-12-26 20:46:41,062][104569] Avg episode reward: [(0, '9167.925'), (1, '8901.932')] [2023-12-26 20:46:41,663][105692] Updated weights for policy 0, policy_version 743762 (0.0010) [2023-12-26 20:46:41,730][105692] Updated weights for policy 0, policy_version 743772 (0.0010) [2023-12-26 20:46:41,765][105620] Updated weights for policy 1, policy_version 743981 (0.0008) [2023-12-26 20:46:41,796][105692] Updated weights for policy 0, policy_version 743782 (0.0011) [2023-12-26 20:46:41,825][105620] Updated weights for policy 1, policy_version 743991 (0.0006) [2023-12-26 20:46:41,888][105620] Updated weights for policy 1, policy_version 744001 (0.0007) [2023-12-26 20:46:42,569][105692] Updated weights for policy 0, policy_version 743792 (0.0011) [2023-12-26 20:46:42,613][105620] Updated weights for policy 1, policy_version 744011 (0.0005) [2023-12-26 20:46:42,635][105692] Updated weights for policy 0, policy_version 743802 (0.0011) [2023-12-26 20:46:42,670][105620] Updated weights for policy 1, policy_version 744021 (0.0009) [2023-12-26 20:46:42,694][105692] Updated weights for policy 0, policy_version 743812 (0.0011) [2023-12-26 20:46:42,725][105620] Updated weights for policy 1, policy_version 744031 (0.0010) [2023-12-26 20:46:43,377][105620] Updated weights for policy 1, policy_version 744041 (0.0010) [2023-12-26 20:46:43,419][105692] Updated weights for policy 0, policy_version 743822 (0.0011) [2023-12-26 20:46:43,439][105620] Updated weights for policy 1, policy_version 744051 (0.0010) [2023-12-26 20:46:43,473][105692] Updated weights for policy 0, policy_version 743832 (0.0010) [2023-12-26 20:46:43,494][105620] Updated weights for policy 1, policy_version 744061 (0.0010) [2023-12-26 20:46:43,533][105692] Updated weights for policy 0, policy_version 743842 (0.0007) [2023-12-26 20:46:43,555][105620] Updated weights for policy 1, policy_version 744071 (0.0008) [2023-12-26 20:46:44,172][105692] Updated weights for policy 0, policy_version 743852 (0.0007) [2023-12-26 20:46:44,230][105692] Updated weights for policy 0, policy_version 743862 (0.0010) [2023-12-26 20:46:44,278][105620] Updated weights for policy 1, policy_version 744081 (0.0010) [2023-12-26 20:46:44,284][105692] Updated weights for policy 0, policy_version 743872 (0.0010) [2023-12-26 20:46:44,330][105620] Updated weights for policy 1, policy_version 744091 (0.0010) [2023-12-26 20:46:44,384][105620] Updated weights for policy 1, policy_version 744101 (0.0010) [2023-12-26 20:46:44,983][105620] Updated weights for policy 1, policy_version 744111 (0.0010) [2023-12-26 20:46:45,033][105692] Updated weights for policy 0, policy_version 743882 (0.0010) [2023-12-26 20:46:45,050][105620] Updated weights for policy 1, policy_version 744121 (0.0011) [2023-12-26 20:46:45,088][105692] Updated weights for policy 0, policy_version 743892 (0.0010) [2023-12-26 20:46:45,110][105620] Updated weights for policy 1, policy_version 744131 (0.0010) [2023-12-26 20:46:45,148][105692] Updated weights for policy 0, policy_version 743902 (0.0011) [2023-12-26 20:46:45,218][105692] Updated weights for policy 0, policy_version 743912 (0.0011) [2023-12-26 20:46:45,814][105620] Updated weights for policy 1, policy_version 744141 (0.0007) [2023-12-26 20:46:45,866][105620] Updated weights for policy 1, policy_version 744151 (0.0008) [2023-12-26 20:46:45,924][105620] Updated weights for policy 1, policy_version 744161 (0.0008) [2023-12-26 20:46:45,966][105692] Updated weights for policy 0, policy_version 743922 (0.0010) [2023-12-26 20:46:46,014][105692] Updated weights for policy 0, policy_version 743932 (0.0010) [2023-12-26 20:46:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 20070.5, 300 sec: 19633.0). Total num frames: 381001728. Throughput: 0: 10186.5, 1: 9952.2. Samples: 380971384. Policy #0 lag: (min: 9.0, avg: 28.2, max: 41.0) [2023-12-26 20:46:46,062][104569] Avg episode reward: [(0, '9258.034'), (1, '9083.608')] [2023-12-26 20:46:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000744168_190529536.pth... [2023-12-26 20:46:46,071][105692] Updated weights for policy 0, policy_version 743942 (0.0010) [2023-12-26 20:46:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000742984_190226432.pth [2023-12-26 20:46:46,081][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000743944_190480384.pth... [2023-12-26 20:46:46,084][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000742728_190169088.pth [2023-12-26 20:46:46,637][105620] Updated weights for policy 1, policy_version 744171 (0.0007) [2023-12-26 20:46:46,698][105620] Updated weights for policy 1, policy_version 744181 (0.0010) [2023-12-26 20:46:46,758][105620] Updated weights for policy 1, policy_version 744191 (0.0011) [2023-12-26 20:46:46,822][105692] Updated weights for policy 0, policy_version 743952 (0.0010) [2023-12-26 20:46:46,891][105692] Updated weights for policy 0, policy_version 743962 (0.0010) [2023-12-26 20:46:46,952][105692] Updated weights for policy 0, policy_version 743972 (0.0011) [2023-12-26 20:46:47,470][105620] Updated weights for policy 1, policy_version 744201 (0.0010) [2023-12-26 20:46:47,530][105620] Updated weights for policy 1, policy_version 744211 (0.0011) [2023-12-26 20:46:47,580][105620] Updated weights for policy 1, policy_version 744221 (0.0010) [2023-12-26 20:46:47,632][105620] Updated weights for policy 1, policy_version 744231 (0.0010) [2023-12-26 20:46:47,684][105692] Updated weights for policy 0, policy_version 743982 (0.0010) [2023-12-26 20:46:47,745][105692] Updated weights for policy 0, policy_version 743992 (0.0011) [2023-12-26 20:46:47,804][105692] Updated weights for policy 0, policy_version 744002 (0.0010) [2023-12-26 20:46:48,346][105620] Updated weights for policy 1, policy_version 744241 (0.0008) [2023-12-26 20:46:48,400][105620] Updated weights for policy 1, policy_version 744251 (0.0007) [2023-12-26 20:46:48,463][105620] Updated weights for policy 1, policy_version 744261 (0.0009) [2023-12-26 20:46:48,497][105692] Updated weights for policy 0, policy_version 744012 (0.0009) [2023-12-26 20:46:48,561][105692] Updated weights for policy 0, policy_version 744022 (0.0008) [2023-12-26 20:46:48,634][105692] Updated weights for policy 0, policy_version 744032 (0.0008) [2023-12-26 20:46:49,080][105620] Updated weights for policy 1, policy_version 744271 (0.0007) [2023-12-26 20:46:49,129][105620] Updated weights for policy 1, policy_version 744281 (0.0010) [2023-12-26 20:46:49,176][105620] Updated weights for policy 1, policy_version 744291 (0.0006) [2023-12-26 20:46:49,283][105692] Updated weights for policy 0, policy_version 744042 (0.0008) [2023-12-26 20:46:49,346][105692] Updated weights for policy 0, policy_version 744052 (0.0011) [2023-12-26 20:46:49,413][105692] Updated weights for policy 0, policy_version 744062 (0.0011) [2023-12-26 20:46:49,464][105692] Updated weights for policy 0, policy_version 744072 (0.0009) [2023-12-26 20:46:49,953][105620] Updated weights for policy 1, policy_version 744301 (0.0009) [2023-12-26 20:46:50,009][105620] Updated weights for policy 1, policy_version 744311 (0.0008) [2023-12-26 20:46:50,068][105620] Updated weights for policy 1, policy_version 744321 (0.0008) [2023-12-26 20:46:50,156][105692] Updated weights for policy 0, policy_version 744082 (0.0008) [2023-12-26 20:46:50,216][105692] Updated weights for policy 0, policy_version 744092 (0.0008) [2023-12-26 20:46:50,275][105692] Updated weights for policy 0, policy_version 744102 (0.0010) [2023-12-26 20:46:50,815][105620] Updated weights for policy 1, policy_version 744331 (0.0008) [2023-12-26 20:46:50,887][105620] Updated weights for policy 1, policy_version 744341 (0.0005) [2023-12-26 20:46:50,952][105620] Updated weights for policy 1, policy_version 744351 (0.0010) [2023-12-26 20:46:50,992][105692] Updated weights for policy 0, policy_version 744112 (0.0007) [2023-12-26 20:46:51,054][105692] Updated weights for policy 0, policy_version 744122 (0.0010) [2023-12-26 20:46:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 20070.4, 300 sec: 19605.3). Total num frames: 381100032. Throughput: 0: 10193.5, 1: 9975.3. Samples: 381089916. Policy #0 lag: (min: 9.0, avg: 28.2, max: 41.0) [2023-12-26 20:46:51,062][104569] Avg episode reward: [(0, '9172.162'), (1, '9269.445')] [2023-12-26 20:46:51,125][105692] Updated weights for policy 0, policy_version 744132 (0.0010) [2023-12-26 20:46:51,694][105620] Updated weights for policy 1, policy_version 744361 (0.0009) [2023-12-26 20:46:51,756][105620] Updated weights for policy 1, policy_version 744371 (0.0008) [2023-12-26 20:46:51,822][105620] Updated weights for policy 1, policy_version 744381 (0.0009) [2023-12-26 20:46:51,888][105620] Updated weights for policy 1, policy_version 744391 (0.0009) [2023-12-26 20:46:51,953][105692] Updated weights for policy 0, policy_version 744142 (0.0009) [2023-12-26 20:46:52,012][105692] Updated weights for policy 0, policy_version 744152 (0.0010) [2023-12-26 20:46:52,072][105692] Updated weights for policy 0, policy_version 744162 (0.0011) [2023-12-26 20:46:52,619][105620] Updated weights for policy 1, policy_version 744401 (0.0009) [2023-12-26 20:46:52,686][105620] Updated weights for policy 1, policy_version 744411 (0.0008) [2023-12-26 20:46:52,743][105620] Updated weights for policy 1, policy_version 744421 (0.0008) [2023-12-26 20:46:52,838][105692] Updated weights for policy 0, policy_version 744172 (0.0010) [2023-12-26 20:46:52,901][105692] Updated weights for policy 0, policy_version 744182 (0.0010) [2023-12-26 20:46:52,963][105692] Updated weights for policy 0, policy_version 744192 (0.0010) [2023-12-26 20:46:53,368][105620] Updated weights for policy 1, policy_version 744431 (0.0006) [2023-12-26 20:46:53,435][105620] Updated weights for policy 1, policy_version 744441 (0.0005) [2023-12-26 20:46:53,486][105620] Updated weights for policy 1, policy_version 744451 (0.0005) [2023-12-26 20:46:53,662][105692] Updated weights for policy 0, policy_version 744202 (0.0010) [2023-12-26 20:46:53,724][105692] Updated weights for policy 0, policy_version 744212 (0.0005) [2023-12-26 20:46:53,784][105692] Updated weights for policy 0, policy_version 744222 (0.0007) [2023-12-26 20:46:53,848][105692] Updated weights for policy 0, policy_version 744232 (0.0006) [2023-12-26 20:46:54,037][105620] Updated weights for policy 1, policy_version 744461 (0.0007) [2023-12-26 20:46:54,082][105620] Updated weights for policy 1, policy_version 744471 (0.0008) [2023-12-26 20:46:54,143][105620] Updated weights for policy 1, policy_version 744481 (0.0008) [2023-12-26 20:46:54,411][105692] Updated weights for policy 0, policy_version 744242 (0.0006) [2023-12-26 20:46:54,477][105692] Updated weights for policy 0, policy_version 744252 (0.0005) [2023-12-26 20:46:54,530][105692] Updated weights for policy 0, policy_version 744262 (0.0009) [2023-12-26 20:46:54,972][105620] Updated weights for policy 1, policy_version 744491 (0.0008) [2023-12-26 20:46:55,042][105620] Updated weights for policy 1, policy_version 744501 (0.0009) [2023-12-26 20:46:55,097][105620] Updated weights for policy 1, policy_version 744511 (0.0009) [2023-12-26 20:46:55,131][105692] Updated weights for policy 0, policy_version 744272 (0.0006) [2023-12-26 20:46:55,185][105692] Updated weights for policy 0, policy_version 744282 (0.0007) [2023-12-26 20:46:55,231][105692] Updated weights for policy 0, policy_version 744292 (0.0008) [2023-12-26 20:46:55,730][105620] Updated weights for policy 1, policy_version 744521 (0.0009) [2023-12-26 20:46:55,783][105620] Updated weights for policy 1, policy_version 744531 (0.0005) [2023-12-26 20:46:55,847][105620] Updated weights for policy 1, policy_version 744541 (0.0005) [2023-12-26 20:46:55,900][105620] Updated weights for policy 1, policy_version 744551 (0.0006) [2023-12-26 20:46:55,966][105692] Updated weights for policy 0, policy_version 744302 (0.0007) [2023-12-26 20:46:56,022][105692] Updated weights for policy 0, policy_version 744312 (0.0006) [2023-12-26 20:46:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 381198336. Throughput: 0: 10139.3, 1: 9988.5. Samples: 381208964. Policy #0 lag: (min: 9.0, avg: 28.2, max: 41.0) [2023-12-26 20:46:56,062][104569] Avg episode reward: [(0, '9092.816'), (1, '9269.179')] [2023-12-26 20:46:56,069][105692] Updated weights for policy 0, policy_version 744322 (0.0005) [2023-12-26 20:46:56,607][105620] Updated weights for policy 1, policy_version 744561 (0.0009) [2023-12-26 20:46:56,621][105692] Updated weights for policy 0, policy_version 744332 (0.0005) [2023-12-26 20:46:56,659][105620] Updated weights for policy 1, policy_version 744571 (0.0008) [2023-12-26 20:46:56,671][105692] Updated weights for policy 0, policy_version 744342 (0.0005) [2023-12-26 20:46:56,708][105620] Updated weights for policy 1, policy_version 744581 (0.0009) [2023-12-26 20:46:56,727][105692] Updated weights for policy 0, policy_version 744352 (0.0005) [2023-12-26 20:46:57,363][105692] Updated weights for policy 0, policy_version 744362 (0.0006) [2023-12-26 20:46:57,423][105692] Updated weights for policy 0, policy_version 744372 (0.0009) [2023-12-26 20:46:57,481][105692] Updated weights for policy 0, policy_version 744382 (0.0009) [2023-12-26 20:46:57,537][105620] Updated weights for policy 1, policy_version 744591 (0.0008) [2023-12-26 20:46:57,546][105692] Updated weights for policy 0, policy_version 744392 (0.0010) [2023-12-26 20:46:57,590][105620] Updated weights for policy 1, policy_version 744601 (0.0009) [2023-12-26 20:46:57,648][105620] Updated weights for policy 1, policy_version 744611 (0.0006) [2023-12-26 20:46:58,232][105692] Updated weights for policy 0, policy_version 744402 (0.0008) [2023-12-26 20:46:58,294][105692] Updated weights for policy 0, policy_version 744412 (0.0008) [2023-12-26 20:46:58,365][105692] Updated weights for policy 0, policy_version 744422 (0.0010) [2023-12-26 20:46:58,462][105620] Updated weights for policy 1, policy_version 744621 (0.0008) [2023-12-26 20:46:58,523][105620] Updated weights for policy 1, policy_version 744631 (0.0007) [2023-12-26 20:46:58,592][105620] Updated weights for policy 1, policy_version 744641 (0.0007) [2023-12-26 20:46:59,235][105692] Updated weights for policy 0, policy_version 744432 (0.0008) [2023-12-26 20:46:59,294][105692] Updated weights for policy 0, policy_version 744442 (0.0007) [2023-12-26 20:46:59,327][105620] Updated weights for policy 1, policy_version 744651 (0.0010) [2023-12-26 20:46:59,366][105692] Updated weights for policy 0, policy_version 744452 (0.0008) [2023-12-26 20:46:59,391][105620] Updated weights for policy 1, policy_version 744661 (0.0010) [2023-12-26 20:46:59,451][105620] Updated weights for policy 1, policy_version 744671 (0.0011) [2023-12-26 20:47:00,082][105620] Updated weights for policy 1, policy_version 744681 (0.0007) [2023-12-26 20:47:00,145][105620] Updated weights for policy 1, policy_version 744691 (0.0007) [2023-12-26 20:47:00,167][105692] Updated weights for policy 0, policy_version 744462 (0.0007) [2023-12-26 20:47:00,204][105620] Updated weights for policy 1, policy_version 744701 (0.0007) [2023-12-26 20:47:00,228][105692] Updated weights for policy 0, policy_version 744472 (0.0008) [2023-12-26 20:47:00,270][105620] Updated weights for policy 1, policy_version 744711 (0.0005) [2023-12-26 20:47:00,284][105692] Updated weights for policy 0, policy_version 744482 (0.0009) [2023-12-26 20:47:00,870][105620] Updated weights for policy 1, policy_version 744721 (0.0005) [2023-12-26 20:47:00,922][105620] Updated weights for policy 1, policy_version 744731 (0.0005) [2023-12-26 20:47:00,975][105620] Updated weights for policy 1, policy_version 744741 (0.0005) [2023-12-26 20:47:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 20070.4, 300 sec: 19605.3). Total num frames: 381296640. Throughput: 0: 10217.2, 1: 9908.4. Samples: 381267276. Policy #0 lag: (min: 9.0, avg: 28.2, max: 41.0) [2023-12-26 20:47:01,062][104569] Avg episode reward: [(0, '9176.817'), (1, '9264.017')] [2023-12-26 20:47:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000744744_190676992.pth... [2023-12-26 20:47:01,074][105692] Updated weights for policy 0, policy_version 744492 (0.0010) [2023-12-26 20:47:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000743592_190382080.pth [2023-12-26 20:47:01,139][105692] Updated weights for policy 0, policy_version 744502 (0.0009) [2023-12-26 20:47:01,194][105692] Updated weights for policy 0, policy_version 744512 (0.0009) [2023-12-26 20:47:01,242][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000744520_190627840.pth... [2023-12-26 20:47:01,248][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000743336_190324736.pth [2023-12-26 20:47:01,656][105620] Updated weights for policy 1, policy_version 744751 (0.0008) [2023-12-26 20:47:01,725][105620] Updated weights for policy 1, policy_version 744761 (0.0007) [2023-12-26 20:47:01,791][105620] Updated weights for policy 1, policy_version 744771 (0.0008) [2023-12-26 20:47:02,000][105692] Updated weights for policy 0, policy_version 744522 (0.0009) [2023-12-26 20:47:02,047][105692] Updated weights for policy 0, policy_version 744532 (0.0009) [2023-12-26 20:47:02,109][105692] Updated weights for policy 0, policy_version 744542 (0.0009) [2023-12-26 20:47:02,160][105692] Updated weights for policy 0, policy_version 744552 (0.0009) [2023-12-26 20:47:02,549][105620] Updated weights for policy 1, policy_version 744781 (0.0009) [2023-12-26 20:47:02,601][105620] Updated weights for policy 1, policy_version 744791 (0.0008) [2023-12-26 20:47:02,656][105620] Updated weights for policy 1, policy_version 744801 (0.0008) [2023-12-26 20:47:02,867][105692] Updated weights for policy 0, policy_version 744562 (0.0010) [2023-12-26 20:47:02,929][105692] Updated weights for policy 0, policy_version 744572 (0.0007) [2023-12-26 20:47:02,991][105692] Updated weights for policy 0, policy_version 744582 (0.0006) [2023-12-26 20:47:03,281][105620] Updated weights for policy 1, policy_version 744811 (0.0008) [2023-12-26 20:47:03,338][105620] Updated weights for policy 1, policy_version 744821 (0.0005) [2023-12-26 20:47:03,387][105620] Updated weights for policy 1, policy_version 744831 (0.0005) [2023-12-26 20:47:03,712][105692] Updated weights for policy 0, policy_version 744592 (0.0010) [2023-12-26 20:47:03,760][105692] Updated weights for policy 0, policy_version 744602 (0.0010) [2023-12-26 20:47:03,804][105692] Updated weights for policy 0, policy_version 744612 (0.0010) [2023-12-26 20:47:04,037][105620] Updated weights for policy 1, policy_version 744841 (0.0006) [2023-12-26 20:47:04,096][105620] Updated weights for policy 1, policy_version 744851 (0.0009) [2023-12-26 20:47:04,160][105620] Updated weights for policy 1, policy_version 744861 (0.0009) [2023-12-26 20:47:04,250][105620] Updated weights for policy 1, policy_version 744871 (0.0009) [2023-12-26 20:47:04,568][105692] Updated weights for policy 0, policy_version 744622 (0.0009) [2023-12-26 20:47:04,623][105692] Updated weights for policy 0, policy_version 744632 (0.0009) [2023-12-26 20:47:04,686][105692] Updated weights for policy 0, policy_version 744642 (0.0009) [2023-12-26 20:47:04,925][105620] Updated weights for policy 1, policy_version 744881 (0.0005) [2023-12-26 20:47:04,968][105620] Updated weights for policy 1, policy_version 744891 (0.0005) [2023-12-26 20:47:05,023][105620] Updated weights for policy 1, policy_version 744901 (0.0005) [2023-12-26 20:47:05,530][105692] Updated weights for policy 0, policy_version 744652 (0.0009) [2023-12-26 20:47:05,577][105692] Updated weights for policy 0, policy_version 744662 (0.0009) [2023-12-26 20:47:05,618][105620] Updated weights for policy 1, policy_version 744911 (0.0006) [2023-12-26 20:47:05,628][105692] Updated weights for policy 0, policy_version 744672 (0.0007) [2023-12-26 20:47:05,667][105620] Updated weights for policy 1, policy_version 744921 (0.0006) [2023-12-26 20:47:05,715][105620] Updated weights for policy 1, policy_version 744931 (0.0009) [2023-12-26 20:47:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 20070.5, 300 sec: 19577.5). Total num frames: 381394944. Throughput: 0: 10042.5, 1: 9954.7. Samples: 381383172. Policy #0 lag: (min: 9.0, avg: 28.2, max: 41.0) [2023-12-26 20:47:06,062][104569] Avg episode reward: [(0, '8900.425'), (1, '9148.224')] [2023-12-26 20:47:06,405][105620] Updated weights for policy 1, policy_version 744941 (0.0005) [2023-12-26 20:47:06,475][105620] Updated weights for policy 1, policy_version 744951 (0.0005) [2023-12-26 20:47:06,495][105692] Updated weights for policy 0, policy_version 744682 (0.0007) [2023-12-26 20:47:06,540][105620] Updated weights for policy 1, policy_version 744961 (0.0006) [2023-12-26 20:47:06,559][105692] Updated weights for policy 0, policy_version 744692 (0.0009) [2023-12-26 20:47:06,626][105692] Updated weights for policy 0, policy_version 744702 (0.0010) [2023-12-26 20:47:06,684][105692] Updated weights for policy 0, policy_version 744712 (0.0009) [2023-12-26 20:47:07,131][105620] Updated weights for policy 1, policy_version 744971 (0.0007) [2023-12-26 20:47:07,195][105620] Updated weights for policy 1, policy_version 744981 (0.0006) [2023-12-26 20:47:07,246][105620] Updated weights for policy 1, policy_version 744991 (0.0005) [2023-12-26 20:47:07,470][105692] Updated weights for policy 0, policy_version 744722 (0.0008) [2023-12-26 20:47:07,530][105692] Updated weights for policy 0, policy_version 744732 (0.0009) [2023-12-26 20:47:07,587][105692] Updated weights for policy 0, policy_version 744742 (0.0009) [2023-12-26 20:47:07,888][105620] Updated weights for policy 1, policy_version 745001 (0.0006) [2023-12-26 20:47:07,943][105620] Updated weights for policy 1, policy_version 745011 (0.0009) [2023-12-26 20:47:07,990][105620] Updated weights for policy 1, policy_version 745021 (0.0009) [2023-12-26 20:47:08,037][105620] Updated weights for policy 1, policy_version 745031 (0.0009) [2023-12-26 20:47:08,321][105692] Updated weights for policy 0, policy_version 744752 (0.0006) [2023-12-26 20:47:08,384][105692] Updated weights for policy 0, policy_version 744762 (0.0008) [2023-12-26 20:47:08,456][105692] Updated weights for policy 0, policy_version 744772 (0.0008) [2023-12-26 20:47:08,814][105620] Updated weights for policy 1, policy_version 745041 (0.0010) [2023-12-26 20:47:08,876][105620] Updated weights for policy 1, policy_version 745051 (0.0009) [2023-12-26 20:47:08,942][105620] Updated weights for policy 1, policy_version 745061 (0.0010) [2023-12-26 20:47:09,131][105692] Updated weights for policy 0, policy_version 744782 (0.0008) [2023-12-26 20:47:09,186][105692] Updated weights for policy 0, policy_version 744792 (0.0009) [2023-12-26 20:47:09,252][105692] Updated weights for policy 0, policy_version 744802 (0.0009) [2023-12-26 20:47:09,663][105620] Updated weights for policy 1, policy_version 745071 (0.0008) [2023-12-26 20:47:09,724][105620] Updated weights for policy 1, policy_version 745081 (0.0008) [2023-12-26 20:47:09,791][105620] Updated weights for policy 1, policy_version 745091 (0.0010) [2023-12-26 20:47:09,950][105692] Updated weights for policy 0, policy_version 744812 (0.0009) [2023-12-26 20:47:10,015][105692] Updated weights for policy 0, policy_version 744822 (0.0009) [2023-12-26 20:47:10,082][105692] Updated weights for policy 0, policy_version 744832 (0.0010) [2023-12-26 20:47:10,537][105620] Updated weights for policy 1, policy_version 745101 (0.0008) [2023-12-26 20:47:10,603][105620] Updated weights for policy 1, policy_version 745111 (0.0008) [2023-12-26 20:47:10,664][105620] Updated weights for policy 1, policy_version 745121 (0.0005) [2023-12-26 20:47:10,768][105692] Updated weights for policy 0, policy_version 744842 (0.0009) [2023-12-26 20:47:10,826][105692] Updated weights for policy 0, policy_version 744852 (0.0006) [2023-12-26 20:47:10,882][105692] Updated weights for policy 0, policy_version 744862 (0.0008) [2023-12-26 20:47:10,943][105692] Updated weights for policy 0, policy_version 744872 (0.0009) [2023-12-26 20:47:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 381493248. Throughput: 0: 9835.1, 1: 10039.4. Samples: 381499588. Policy #0 lag: (min: 9.0, avg: 28.2, max: 41.0) [2023-12-26 20:47:11,063][104569] Avg episode reward: [(0, '8638.166'), (1, '9063.060')] [2023-12-26 20:47:11,321][105620] Updated weights for policy 1, policy_version 745131 (0.0006) [2023-12-26 20:47:11,389][105620] Updated weights for policy 1, policy_version 745141 (0.0008) [2023-12-26 20:47:11,453][105620] Updated weights for policy 1, policy_version 745151 (0.0008) [2023-12-26 20:47:11,644][105692] Updated weights for policy 0, policy_version 744882 (0.0007) [2023-12-26 20:47:11,707][105692] Updated weights for policy 0, policy_version 744892 (0.0009) [2023-12-26 20:47:11,768][105692] Updated weights for policy 0, policy_version 744902 (0.0009) [2023-12-26 20:47:12,201][105620] Updated weights for policy 1, policy_version 745161 (0.0008) [2023-12-26 20:47:12,267][105620] Updated weights for policy 1, policy_version 745171 (0.0008) [2023-12-26 20:47:12,331][105620] Updated weights for policy 1, policy_version 745181 (0.0007) [2023-12-26 20:47:12,393][105620] Updated weights for policy 1, policy_version 745191 (0.0007) [2023-12-26 20:47:12,507][105692] Updated weights for policy 0, policy_version 744912 (0.0009) [2023-12-26 20:47:12,567][105692] Updated weights for policy 0, policy_version 744922 (0.0009) [2023-12-26 20:47:12,629][105692] Updated weights for policy 0, policy_version 744932 (0.0009) [2023-12-26 20:47:13,086][105620] Updated weights for policy 1, policy_version 745201 (0.0010) [2023-12-26 20:47:13,134][105620] Updated weights for policy 1, policy_version 745211 (0.0010) [2023-12-26 20:47:13,200][105620] Updated weights for policy 1, policy_version 745221 (0.0009) [2023-12-26 20:47:13,392][105692] Updated weights for policy 0, policy_version 744942 (0.0007) [2023-12-26 20:47:13,452][105692] Updated weights for policy 0, policy_version 744952 (0.0005) [2023-12-26 20:47:13,514][105692] Updated weights for policy 0, policy_version 744962 (0.0005) [2023-12-26 20:47:13,948][105620] Updated weights for policy 1, policy_version 745231 (0.0010) [2023-12-26 20:47:14,003][105620] Updated weights for policy 1, policy_version 745241 (0.0010) [2023-12-26 20:47:14,041][105692] Updated weights for policy 0, policy_version 744972 (0.0006) [2023-12-26 20:47:14,063][105620] Updated weights for policy 1, policy_version 745251 (0.0005) [2023-12-26 20:47:14,109][105692] Updated weights for policy 0, policy_version 744982 (0.0006) [2023-12-26 20:47:14,172][105692] Updated weights for policy 0, policy_version 744992 (0.0008) [2023-12-26 20:47:14,683][105620] Updated weights for policy 1, policy_version 745261 (0.0005) [2023-12-26 20:47:14,737][105620] Updated weights for policy 1, policy_version 745271 (0.0005) [2023-12-26 20:47:14,804][105620] Updated weights for policy 1, policy_version 745281 (0.0010) [2023-12-26 20:47:14,884][105692] Updated weights for policy 0, policy_version 745002 (0.0006) [2023-12-26 20:47:14,938][105692] Updated weights for policy 0, policy_version 745012 (0.0006) [2023-12-26 20:47:14,998][105692] Updated weights for policy 0, policy_version 745022 (0.0006) [2023-12-26 20:47:15,068][105692] Updated weights for policy 0, policy_version 745032 (0.0007) [2023-12-26 20:47:15,616][105620] Updated weights for policy 1, policy_version 745291 (0.0008) [2023-12-26 20:47:15,655][105692] Updated weights for policy 0, policy_version 745042 (0.0007) [2023-12-26 20:47:15,665][105620] Updated weights for policy 1, policy_version 745301 (0.0008) [2023-12-26 20:47:15,700][105692] Updated weights for policy 0, policy_version 745052 (0.0006) [2023-12-26 20:47:15,713][105620] Updated weights for policy 1, policy_version 745311 (0.0007) [2023-12-26 20:47:15,751][105692] Updated weights for policy 0, policy_version 745062 (0.0009) [2023-12-26 20:47:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19933.8, 300 sec: 19549.7). Total num frames: 381591552. Throughput: 0: 9765.4, 1: 9951.3. Samples: 381556724. Policy #0 lag: (min: 9.0, avg: 28.2, max: 41.0) [2023-12-26 20:47:16,063][104569] Avg episode reward: [(0, '8993.671'), (1, '9179.642')] [2023-12-26 20:47:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000745320_190824448.pth... [2023-12-26 20:47:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000745064_190767104.pth... [2023-12-26 20:47:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000744168_190529536.pth [2023-12-26 20:47:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000743944_190480384.pth [2023-12-26 20:47:16,471][105620] Updated weights for policy 1, policy_version 745321 (0.0006) [2023-12-26 20:47:16,474][105692] Updated weights for policy 0, policy_version 745072 (0.0010) [2023-12-26 20:47:16,516][105620] Updated weights for policy 1, policy_version 745331 (0.0010) [2023-12-26 20:47:16,525][105692] Updated weights for policy 0, policy_version 745082 (0.0010) [2023-12-26 20:47:16,564][105620] Updated weights for policy 1, policy_version 745341 (0.0010) [2023-12-26 20:47:16,574][105692] Updated weights for policy 0, policy_version 745092 (0.0010) [2023-12-26 20:47:16,612][105620] Updated weights for policy 1, policy_version 745351 (0.0010) [2023-12-26 20:47:17,221][105620] Updated weights for policy 1, policy_version 745361 (0.0007) [2023-12-26 20:47:17,235][105692] Updated weights for policy 0, policy_version 745102 (0.0007) [2023-12-26 20:47:17,282][105620] Updated weights for policy 1, policy_version 745371 (0.0007) [2023-12-26 20:47:17,290][105692] Updated weights for policy 0, policy_version 745112 (0.0005) [2023-12-26 20:47:17,341][105620] Updated weights for policy 1, policy_version 745381 (0.0007) [2023-12-26 20:47:17,342][105692] Updated weights for policy 0, policy_version 745122 (0.0005) [2023-12-26 20:47:17,882][105620] Updated weights for policy 1, policy_version 745391 (0.0008) [2023-12-26 20:47:17,943][105620] Updated weights for policy 1, policy_version 745401 (0.0008) [2023-12-26 20:47:17,953][105692] Updated weights for policy 0, policy_version 745132 (0.0008) [2023-12-26 20:47:18,005][105692] Updated weights for policy 0, policy_version 745142 (0.0010) [2023-12-26 20:47:18,010][105620] Updated weights for policy 1, policy_version 745411 (0.0005) [2023-12-26 20:47:18,058][105692] Updated weights for policy 0, policy_version 745152 (0.0005) [2023-12-26 20:47:18,591][105620] Updated weights for policy 1, policy_version 745421 (0.0008) [2023-12-26 20:47:18,639][105620] Updated weights for policy 1, policy_version 745431 (0.0010) [2023-12-26 20:47:18,695][105620] Updated weights for policy 1, policy_version 745441 (0.0008) [2023-12-26 20:47:18,714][105692] Updated weights for policy 0, policy_version 745162 (0.0006) [2023-12-26 20:47:18,764][105692] Updated weights for policy 0, policy_version 745172 (0.0010) [2023-12-26 20:47:18,830][105692] Updated weights for policy 0, policy_version 745182 (0.0011) [2023-12-26 20:47:18,889][105692] Updated weights for policy 0, policy_version 745192 (0.0010) [2023-12-26 20:47:19,436][105620] Updated weights for policy 1, policy_version 745451 (0.0008) [2023-12-26 20:47:19,500][105620] Updated weights for policy 1, policy_version 745461 (0.0010) [2023-12-26 20:47:19,566][105620] Updated weights for policy 1, policy_version 745471 (0.0007) [2023-12-26 20:47:19,595][105692] Updated weights for policy 0, policy_version 745202 (0.0007) [2023-12-26 20:47:19,656][105692] Updated weights for policy 0, policy_version 745212 (0.0007) [2023-12-26 20:47:19,708][105692] Updated weights for policy 0, policy_version 745222 (0.0005) [2023-12-26 20:47:20,270][105620] Updated weights for policy 1, policy_version 745481 (0.0007) [2023-12-26 20:47:20,333][105620] Updated weights for policy 1, policy_version 745491 (0.0007) [2023-12-26 20:47:20,354][105692] Updated weights for policy 0, policy_version 745232 (0.0008) [2023-12-26 20:47:20,394][105620] Updated weights for policy 1, policy_version 745501 (0.0010) [2023-12-26 20:47:20,409][105692] Updated weights for policy 0, policy_version 745242 (0.0007) [2023-12-26 20:47:20,452][105620] Updated weights for policy 1, policy_version 745511 (0.0007) [2023-12-26 20:47:20,454][105692] Updated weights for policy 0, policy_version 745252 (0.0009) [2023-12-26 20:47:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19933.8, 300 sec: 19549.7). Total num frames: 381689856. Throughput: 0: 9839.7, 1: 9944.9. Samples: 381681796. Policy #0 lag: (min: 9.0, avg: 28.2, max: 41.0) [2023-12-26 20:47:21,063][104569] Avg episode reward: [(0, '9258.924'), (1, '9264.871')] [2023-12-26 20:47:21,150][105692] Updated weights for policy 0, policy_version 745262 (0.0009) [2023-12-26 20:47:21,217][105692] Updated weights for policy 0, policy_version 745272 (0.0010) [2023-12-26 20:47:21,222][105620] Updated weights for policy 1, policy_version 745521 (0.0008) [2023-12-26 20:47:21,285][105692] Updated weights for policy 0, policy_version 745282 (0.0011) [2023-12-26 20:47:21,285][105620] Updated weights for policy 1, policy_version 745531 (0.0008) [2023-12-26 20:47:21,348][105620] Updated weights for policy 1, policy_version 745541 (0.0008) [2023-12-26 20:47:22,039][105692] Updated weights for policy 0, policy_version 745292 (0.0010) [2023-12-26 20:47:22,075][105620] Updated weights for policy 1, policy_version 745551 (0.0010) [2023-12-26 20:47:22,100][105692] Updated weights for policy 0, policy_version 745302 (0.0011) [2023-12-26 20:47:22,136][105620] Updated weights for policy 1, policy_version 745561 (0.0011) [2023-12-26 20:47:22,161][105692] Updated weights for policy 0, policy_version 745312 (0.0011) [2023-12-26 20:47:22,205][105620] Updated weights for policy 1, policy_version 745571 (0.0011) [2023-12-26 20:47:22,830][105692] Updated weights for policy 0, policy_version 745322 (0.0009) [2023-12-26 20:47:22,896][105620] Updated weights for policy 1, policy_version 745581 (0.0008) [2023-12-26 20:47:22,897][105692] Updated weights for policy 0, policy_version 745332 (0.0007) [2023-12-26 20:47:22,955][105692] Updated weights for policy 0, policy_version 745342 (0.0007) [2023-12-26 20:47:22,960][105620] Updated weights for policy 1, policy_version 745591 (0.0006) [2023-12-26 20:47:23,010][105692] Updated weights for policy 0, policy_version 745352 (0.0007) [2023-12-26 20:47:23,026][105620] Updated weights for policy 1, policy_version 745601 (0.0006) [2023-12-26 20:47:23,620][105620] Updated weights for policy 1, policy_version 745611 (0.0008) [2023-12-26 20:47:23,659][105692] Updated weights for policy 0, policy_version 745362 (0.0005) [2023-12-26 20:47:23,669][105620] Updated weights for policy 1, policy_version 745621 (0.0010) [2023-12-26 20:47:23,727][105692] Updated weights for policy 0, policy_version 745372 (0.0005) [2023-12-26 20:47:23,731][105620] Updated weights for policy 1, policy_version 745631 (0.0010) [2023-12-26 20:47:23,785][105692] Updated weights for policy 0, policy_version 745382 (0.0005) [2023-12-26 20:47:24,368][105620] Updated weights for policy 1, policy_version 745641 (0.0011) [2023-12-26 20:47:24,423][105692] Updated weights for policy 0, policy_version 745392 (0.0006) [2023-12-26 20:47:24,428][105620] Updated weights for policy 1, policy_version 745651 (0.0010) [2023-12-26 20:47:24,483][105692] Updated weights for policy 0, policy_version 745402 (0.0008) [2023-12-26 20:47:24,487][105620] Updated weights for policy 1, policy_version 745661 (0.0010) [2023-12-26 20:47:24,547][105692] Updated weights for policy 0, policy_version 745412 (0.0010) [2023-12-26 20:47:24,553][105620] Updated weights for policy 1, policy_version 745671 (0.0008) [2023-12-26 20:47:25,132][105620] Updated weights for policy 1, policy_version 745681 (0.0006) [2023-12-26 20:47:25,185][105620] Updated weights for policy 1, policy_version 745691 (0.0008) [2023-12-26 20:47:25,242][105620] Updated weights for policy 1, policy_version 745701 (0.0008) [2023-12-26 20:47:25,301][105692] Updated weights for policy 0, policy_version 745422 (0.0009) [2023-12-26 20:47:25,363][105692] Updated weights for policy 0, policy_version 745432 (0.0010) [2023-12-26 20:47:25,427][105692] Updated weights for policy 0, policy_version 745442 (0.0010) [2023-12-26 20:47:25,846][105620] Updated weights for policy 1, policy_version 745711 (0.0008) [2023-12-26 20:47:25,902][105620] Updated weights for policy 1, policy_version 745721 (0.0005) [2023-12-26 20:47:25,959][105620] Updated weights for policy 1, policy_version 745731 (0.0008) [2023-12-26 20:47:26,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 381796352. Throughput: 0: 9736.6, 1: 10013.2. Samples: 381803024. Policy #0 lag: (min: 9.0, avg: 28.2, max: 41.0) [2023-12-26 20:47:26,062][104569] Avg episode reward: [(0, '9082.149'), (1, '9065.780')] [2023-12-26 20:47:26,283][105692] Updated weights for policy 0, policy_version 745452 (0.0010) [2023-12-26 20:47:26,337][105692] Updated weights for policy 0, policy_version 745462 (0.0011) [2023-12-26 20:47:26,390][105692] Updated weights for policy 0, policy_version 745472 (0.0008) [2023-12-26 20:47:26,542][105620] Updated weights for policy 1, policy_version 745741 (0.0008) [2023-12-26 20:47:26,608][105620] Updated weights for policy 1, policy_version 745751 (0.0008) [2023-12-26 20:47:26,676][105620] Updated weights for policy 1, policy_version 745761 (0.0008) [2023-12-26 20:47:27,031][105692] Updated weights for policy 0, policy_version 745482 (0.0008) [2023-12-26 20:47:27,088][105692] Updated weights for policy 0, policy_version 745492 (0.0005) [2023-12-26 20:47:27,132][105692] Updated weights for policy 0, policy_version 745502 (0.0005) [2023-12-26 20:47:27,184][105692] Updated weights for policy 0, policy_version 745512 (0.0005) [2023-12-26 20:47:27,349][105620] Updated weights for policy 1, policy_version 745771 (0.0009) [2023-12-26 20:47:27,419][105620] Updated weights for policy 1, policy_version 745781 (0.0011) [2023-12-26 20:47:27,475][105620] Updated weights for policy 1, policy_version 745791 (0.0010) [2023-12-26 20:47:27,860][105692] Updated weights for policy 0, policy_version 745522 (0.0007) [2023-12-26 20:47:27,925][105692] Updated weights for policy 0, policy_version 745532 (0.0008) [2023-12-26 20:47:27,985][105692] Updated weights for policy 0, policy_version 745542 (0.0008) [2023-12-26 20:47:28,209][105620] Updated weights for policy 1, policy_version 745801 (0.0011) [2023-12-26 20:47:28,264][105620] Updated weights for policy 1, policy_version 745811 (0.0010) [2023-12-26 20:47:28,319][105620] Updated weights for policy 1, policy_version 745821 (0.0009) [2023-12-26 20:47:28,380][105620] Updated weights for policy 1, policy_version 745831 (0.0009) [2023-12-26 20:47:28,711][105692] Updated weights for policy 0, policy_version 745552 (0.0009) [2023-12-26 20:47:28,767][105692] Updated weights for policy 0, policy_version 745562 (0.0010) [2023-12-26 20:47:28,815][105692] Updated weights for policy 0, policy_version 745572 (0.0009) [2023-12-26 20:47:29,101][105620] Updated weights for policy 1, policy_version 745841 (0.0007) [2023-12-26 20:47:29,158][105620] Updated weights for policy 1, policy_version 745851 (0.0008) [2023-12-26 20:47:29,218][105620] Updated weights for policy 1, policy_version 745861 (0.0008) [2023-12-26 20:47:29,556][105692] Updated weights for policy 0, policy_version 745582 (0.0006) [2023-12-26 20:47:29,616][105692] Updated weights for policy 0, policy_version 745592 (0.0006) [2023-12-26 20:47:29,678][105692] Updated weights for policy 0, policy_version 745602 (0.0008) [2023-12-26 20:47:30,031][105620] Updated weights for policy 1, policy_version 745871 (0.0009) [2023-12-26 20:47:30,085][105620] Updated weights for policy 1, policy_version 745881 (0.0009) [2023-12-26 20:47:30,142][105620] Updated weights for policy 1, policy_version 745892 (0.0010) [2023-12-26 20:47:30,307][105692] Updated weights for policy 0, policy_version 745612 (0.0007) [2023-12-26 20:47:30,365][105692] Updated weights for policy 0, policy_version 745622 (0.0009) [2023-12-26 20:47:30,423][105692] Updated weights for policy 0, policy_version 745632 (0.0009) [2023-12-26 20:47:30,851][105620] Updated weights for policy 1, policy_version 745902 (0.0009) [2023-12-26 20:47:30,901][105620] Updated weights for policy 1, policy_version 745912 (0.0009) [2023-12-26 20:47:30,958][105620] Updated weights for policy 1, policy_version 745922 (0.0008) [2023-12-26 20:47:31,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19933.8, 300 sec: 19605.3). Total num frames: 381894656. Throughput: 0: 9774.4, 1: 10041.5. Samples: 381863104. Policy #0 lag: (min: 9.0, avg: 28.2, max: 41.0) [2023-12-26 20:47:31,063][104569] Avg episode reward: [(0, '9079.230'), (1, '7572.459')] [2023-12-26 20:47:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000745640_190914560.pth... [2023-12-26 20:47:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000745928_190980096.pth... [2023-12-26 20:47:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000744520_190627840.pth [2023-12-26 20:47:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000744744_190676992.pth [2023-12-26 20:47:31,200][105692] Updated weights for policy 0, policy_version 745642 (0.0009) [2023-12-26 20:47:31,252][105692] Updated weights for policy 0, policy_version 745652 (0.0009) [2023-12-26 20:47:31,313][105692] Updated weights for policy 0, policy_version 745662 (0.0009) [2023-12-26 20:47:31,377][105692] Updated weights for policy 0, policy_version 745672 (0.0009) [2023-12-26 20:47:31,715][105620] Updated weights for policy 1, policy_version 745932 (0.0008) [2023-12-26 20:47:31,780][105620] Updated weights for policy 1, policy_version 745942 (0.0009) [2023-12-26 20:47:31,837][105620] Updated weights for policy 1, policy_version 745952 (0.0010) [2023-12-26 20:47:32,085][105692] Updated weights for policy 0, policy_version 745682 (0.0007) [2023-12-26 20:47:32,146][105692] Updated weights for policy 0, policy_version 745692 (0.0009) [2023-12-26 20:47:32,212][105692] Updated weights for policy 0, policy_version 745702 (0.0009) [2023-12-26 20:47:32,591][105620] Updated weights for policy 1, policy_version 745962 (0.0009) [2023-12-26 20:47:32,648][105620] Updated weights for policy 1, policy_version 745972 (0.0009) [2023-12-26 20:47:32,706][105620] Updated weights for policy 1, policy_version 745982 (0.0009) [2023-12-26 20:47:32,763][105620] Updated weights for policy 1, policy_version 745992 (0.0009) [2023-12-26 20:47:32,943][105692] Updated weights for policy 0, policy_version 745712 (0.0008) [2023-12-26 20:47:33,000][105692] Updated weights for policy 0, policy_version 745722 (0.0010) [2023-12-26 20:47:33,067][105692] Updated weights for policy 0, policy_version 745732 (0.0010) [2023-12-26 20:47:33,398][105620] Updated weights for policy 1, policy_version 746002 (0.0008) [2023-12-26 20:47:33,455][105620] Updated weights for policy 1, policy_version 746012 (0.0009) [2023-12-26 20:47:33,512][105620] Updated weights for policy 1, policy_version 746022 (0.0009) [2023-12-26 20:47:33,851][105692] Updated weights for policy 0, policy_version 745742 (0.0009) [2023-12-26 20:47:33,908][105692] Updated weights for policy 0, policy_version 745752 (0.0009) [2023-12-26 20:47:33,966][105692] Updated weights for policy 0, policy_version 745762 (0.0009) [2023-12-26 20:47:34,217][105620] Updated weights for policy 1, policy_version 746032 (0.0007) [2023-12-26 20:47:34,279][105620] Updated weights for policy 1, policy_version 746042 (0.0006) [2023-12-26 20:47:34,342][105620] Updated weights for policy 1, policy_version 746052 (0.0006) [2023-12-26 20:47:34,864][105692] Updated weights for policy 0, policy_version 745772 (0.0008) [2023-12-26 20:47:34,895][105620] Updated weights for policy 1, policy_version 746062 (0.0007) [2023-12-26 20:47:34,928][105692] Updated weights for policy 0, policy_version 745782 (0.0009) [2023-12-26 20:47:34,959][105620] Updated weights for policy 1, policy_version 746072 (0.0008) [2023-12-26 20:47:34,989][105692] Updated weights for policy 0, policy_version 745792 (0.0010) [2023-12-26 20:47:35,017][105620] Updated weights for policy 1, policy_version 746082 (0.0008) [2023-12-26 20:47:35,752][105692] Updated weights for policy 0, policy_version 745802 (0.0007) [2023-12-26 20:47:35,754][105620] Updated weights for policy 1, policy_version 746092 (0.0009) [2023-12-26 20:47:35,808][105692] Updated weights for policy 0, policy_version 745812 (0.0005) [2023-12-26 20:47:35,813][105620] Updated weights for policy 1, policy_version 746102 (0.0010) [2023-12-26 20:47:35,860][105692] Updated weights for policy 0, policy_version 745822 (0.0008) [2023-12-26 20:47:35,867][105620] Updated weights for policy 1, policy_version 746112 (0.0010) [2023-12-26 20:47:35,912][105692] Updated weights for policy 0, policy_version 745832 (0.0006) [2023-12-26 20:47:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 381992960. Throughput: 0: 9732.0, 1: 10025.6. Samples: 381979008. Policy #0 lag: (min: 9.0, avg: 28.2, max: 41.0) [2023-12-26 20:47:36,063][104569] Avg episode reward: [(0, '9166.635'), (1, '2600.626')] [2023-12-26 20:47:36,558][105692] Updated weights for policy 0, policy_version 745842 (0.0006) [2023-12-26 20:47:36,604][105620] Updated weights for policy 1, policy_version 746122 (0.0010) [2023-12-26 20:47:36,628][105692] Updated weights for policy 0, policy_version 745852 (0.0007) [2023-12-26 20:47:36,664][105620] Updated weights for policy 1, policy_version 746132 (0.0010) [2023-12-26 20:47:36,687][105692] Updated weights for policy 0, policy_version 745862 (0.0008) [2023-12-26 20:47:36,726][105620] Updated weights for policy 1, policy_version 746142 (0.0010) [2023-12-26 20:47:36,781][105620] Updated weights for policy 1, policy_version 746152 (0.0010) [2023-12-26 20:47:37,337][105692] Updated weights for policy 0, policy_version 745872 (0.0006) [2023-12-26 20:47:37,400][105692] Updated weights for policy 0, policy_version 745882 (0.0005) [2023-12-26 20:47:37,461][105692] Updated weights for policy 0, policy_version 745892 (0.0005) [2023-12-26 20:47:37,471][105620] Updated weights for policy 1, policy_version 746162 (0.0010) [2023-12-26 20:47:37,519][105620] Updated weights for policy 1, policy_version 746172 (0.0010) [2023-12-26 20:47:37,570][105620] Updated weights for policy 1, policy_version 746182 (0.0008) [2023-12-26 20:47:38,132][105620] Updated weights for policy 1, policy_version 746192 (0.0005) [2023-12-26 20:47:38,195][105620] Updated weights for policy 1, policy_version 746202 (0.0005) [2023-12-26 20:47:38,246][105620] Updated weights for policy 1, policy_version 746212 (0.0005) [2023-12-26 20:47:38,272][105692] Updated weights for policy 0, policy_version 745902 (0.0007) [2023-12-26 20:47:38,334][105692] Updated weights for policy 0, policy_version 745912 (0.0009) [2023-12-26 20:47:38,398][105692] Updated weights for policy 0, policy_version 745922 (0.0009) [2023-12-26 20:47:38,793][105620] Updated weights for policy 1, policy_version 746222 (0.0008) [2023-12-26 20:47:38,844][105620] Updated weights for policy 1, policy_version 746232 (0.0010) [2023-12-26 20:47:38,899][105620] Updated weights for policy 1, policy_version 746242 (0.0010) [2023-12-26 20:47:39,243][105692] Updated weights for policy 0, policy_version 745932 (0.0009) [2023-12-26 20:47:39,306][105692] Updated weights for policy 0, policy_version 745942 (0.0008) [2023-12-26 20:47:39,375][105692] Updated weights for policy 0, policy_version 745952 (0.0008) [2023-12-26 20:47:39,543][105620] Updated weights for policy 1, policy_version 746252 (0.0009) [2023-12-26 20:47:39,600][105620] Updated weights for policy 1, policy_version 746262 (0.0010) [2023-12-26 20:47:39,662][105620] Updated weights for policy 1, policy_version 746272 (0.0010) [2023-12-26 20:47:40,086][105692] Updated weights for policy 0, policy_version 745962 (0.0009) [2023-12-26 20:47:40,147][105692] Updated weights for policy 0, policy_version 745972 (0.0009) [2023-12-26 20:47:40,210][105692] Updated weights for policy 0, policy_version 745982 (0.0009) [2023-12-26 20:47:40,276][105692] Updated weights for policy 0, policy_version 745992 (0.0009) [2023-12-26 20:47:40,422][105620] Updated weights for policy 1, policy_version 746282 (0.0010) [2023-12-26 20:47:40,489][105620] Updated weights for policy 1, policy_version 746292 (0.0009) [2023-12-26 20:47:40,550][105620] Updated weights for policy 1, policy_version 746302 (0.0007) [2023-12-26 20:47:40,612][105620] Updated weights for policy 1, policy_version 746312 (0.0009) [2023-12-26 20:47:41,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 382083072. Throughput: 0: 9641.2, 1: 10068.3. Samples: 382095888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-12-26 20:47:41,062][104569] Avg episode reward: [(0, '8988.051'), (1, '6665.485')] [2023-12-26 20:47:41,089][105692] Updated weights for policy 0, policy_version 746002 (0.0010) [2023-12-26 20:47:41,151][105692] Updated weights for policy 0, policy_version 746012 (0.0008) [2023-12-26 20:47:41,205][105692] Updated weights for policy 0, policy_version 746022 (0.0008) [2023-12-26 20:47:41,292][105620] Updated weights for policy 1, policy_version 746322 (0.0008) [2023-12-26 20:47:41,359][105620] Updated weights for policy 1, policy_version 746332 (0.0009) [2023-12-26 20:47:41,417][105620] Updated weights for policy 1, policy_version 746342 (0.0007) [2023-12-26 20:47:42,002][105692] Updated weights for policy 0, policy_version 746032 (0.0009) [2023-12-26 20:47:42,054][105692] Updated weights for policy 0, policy_version 746042 (0.0009) [2023-12-26 20:47:42,115][105692] Updated weights for policy 0, policy_version 746052 (0.0009) [2023-12-26 20:47:42,170][105620] Updated weights for policy 1, policy_version 746352 (0.0008) [2023-12-26 20:47:42,219][105620] Updated weights for policy 1, policy_version 746362 (0.0010) [2023-12-26 20:47:42,285][105620] Updated weights for policy 1, policy_version 746372 (0.0009) [2023-12-26 20:47:42,843][105692] Updated weights for policy 0, policy_version 746062 (0.0009) [2023-12-26 20:47:42,903][105692] Updated weights for policy 0, policy_version 746072 (0.0011) [2023-12-26 20:47:42,956][105692] Updated weights for policy 0, policy_version 746082 (0.0005) [2023-12-26 20:47:43,023][105620] Updated weights for policy 1, policy_version 746382 (0.0008) [2023-12-26 20:47:43,079][105620] Updated weights for policy 1, policy_version 746392 (0.0007) [2023-12-26 20:47:43,132][105620] Updated weights for policy 1, policy_version 746402 (0.0008) [2023-12-26 20:47:43,640][105692] Updated weights for policy 0, policy_version 746092 (0.0007) [2023-12-26 20:47:43,688][105692] Updated weights for policy 0, policy_version 746102 (0.0009) [2023-12-26 20:47:43,742][105692] Updated weights for policy 0, policy_version 746112 (0.0010) [2023-12-26 20:47:43,844][105620] Updated weights for policy 1, policy_version 746412 (0.0010) [2023-12-26 20:47:43,911][105620] Updated weights for policy 1, policy_version 746422 (0.0009) [2023-12-26 20:47:43,971][105620] Updated weights for policy 1, policy_version 746432 (0.0005) [2023-12-26 20:47:44,346][105692] Updated weights for policy 0, policy_version 746122 (0.0006) [2023-12-26 20:47:44,398][105692] Updated weights for policy 0, policy_version 746132 (0.0005) [2023-12-26 20:47:44,456][105692] Updated weights for policy 0, policy_version 746143 (0.0008) [2023-12-26 20:47:44,661][105620] Updated weights for policy 1, policy_version 746442 (0.0006) [2023-12-26 20:47:44,727][105620] Updated weights for policy 1, policy_version 746452 (0.0010) [2023-12-26 20:47:44,788][105620] Updated weights for policy 1, policy_version 746462 (0.0008) [2023-12-26 20:47:44,857][105620] Updated weights for policy 1, policy_version 746472 (0.0006) [2023-12-26 20:47:45,149][105692] Updated weights for policy 0, policy_version 746153 (0.0010) [2023-12-26 20:47:45,235][105692] Updated weights for policy 0, policy_version 746163 (0.0005) [2023-12-26 20:47:45,304][105692] Updated weights for policy 0, policy_version 746173 (0.0006) [2023-12-26 20:47:45,364][105692] Updated weights for policy 0, policy_version 746183 (0.0006) [2023-12-26 20:47:45,584][105620] Updated weights for policy 1, policy_version 746482 (0.0007) [2023-12-26 20:47:45,643][105620] Updated weights for policy 1, policy_version 746492 (0.0009) [2023-12-26 20:47:45,692][105620] Updated weights for policy 1, policy_version 746502 (0.0010) [2023-12-26 20:47:46,015][105692] Updated weights for policy 0, policy_version 746193 (0.0008) [2023-12-26 20:47:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19660.7, 300 sec: 19605.2). Total num frames: 382181376. Throughput: 0: 9568.0, 1: 10104.2. Samples: 382152528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-12-26 20:47:46,063][104569] Avg episode reward: [(0, '9167.387'), (1, '8774.640')] [2023-12-26 20:47:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000746504_191127552.pth... [2023-12-26 20:47:46,074][105692] Updated weights for policy 0, policy_version 746203 (0.0008) [2023-12-26 20:47:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000745320_190824448.pth [2023-12-26 20:47:46,133][105692] Updated weights for policy 0, policy_version 746213 (0.0008) [2023-12-26 20:47:46,148][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000746216_191062016.pth... [2023-12-26 20:47:46,153][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000745064_190767104.pth [2023-12-26 20:47:46,383][105620] Updated weights for policy 1, policy_version 746512 (0.0008) [2023-12-26 20:47:46,444][105620] Updated weights for policy 1, policy_version 746522 (0.0010) [2023-12-26 20:47:46,502][105620] Updated weights for policy 1, policy_version 746532 (0.0009) [2023-12-26 20:47:46,871][105692] Updated weights for policy 0, policy_version 746223 (0.0006) [2023-12-26 20:47:46,937][105692] Updated weights for policy 0, policy_version 746233 (0.0008) [2023-12-26 20:47:46,991][105692] Updated weights for policy 0, policy_version 746243 (0.0010) [2023-12-26 20:47:47,081][105620] Updated weights for policy 1, policy_version 746542 (0.0008) [2023-12-26 20:47:47,132][105620] Updated weights for policy 1, policy_version 746552 (0.0008) [2023-12-26 20:47:47,184][105620] Updated weights for policy 1, policy_version 746562 (0.0009) [2023-12-26 20:47:47,585][105692] Updated weights for policy 0, policy_version 746254 (0.0007) [2023-12-26 20:47:47,640][105692] Updated weights for policy 0, policy_version 746264 (0.0006) [2023-12-26 20:47:47,694][105692] Updated weights for policy 0, policy_version 746274 (0.0006) [2023-12-26 20:47:47,838][105620] Updated weights for policy 1, policy_version 746572 (0.0008) [2023-12-26 20:47:47,907][105620] Updated weights for policy 1, policy_version 746582 (0.0010) [2023-12-26 20:47:47,956][105620] Updated weights for policy 1, policy_version 746592 (0.0010) [2023-12-26 20:47:48,315][105692] Updated weights for policy 0, policy_version 746284 (0.0006) [2023-12-26 20:47:48,385][105692] Updated weights for policy 0, policy_version 746294 (0.0006) [2023-12-26 20:47:48,441][105692] Updated weights for policy 0, policy_version 746304 (0.0006) [2023-12-26 20:47:48,511][105620] Updated weights for policy 1, policy_version 746602 (0.0008) [2023-12-26 20:47:48,569][105620] Updated weights for policy 1, policy_version 746612 (0.0009) [2023-12-26 20:47:48,634][105620] Updated weights for policy 1, policy_version 746622 (0.0008) [2023-12-26 20:47:48,686][105620] Updated weights for policy 1, policy_version 746632 (0.0010) [2023-12-26 20:47:49,065][105692] Updated weights for policy 0, policy_version 746314 (0.0005) [2023-12-26 20:47:49,121][105692] Updated weights for policy 0, policy_version 746324 (0.0005) [2023-12-26 20:47:49,174][105692] Updated weights for policy 0, policy_version 746334 (0.0009) [2023-12-26 20:47:49,241][105692] Updated weights for policy 0, policy_version 746344 (0.0011) [2023-12-26 20:47:49,400][105620] Updated weights for policy 1, policy_version 746642 (0.0009) [2023-12-26 20:47:49,456][105620] Updated weights for policy 1, policy_version 746652 (0.0009) [2023-12-26 20:47:49,515][105620] Updated weights for policy 1, policy_version 746662 (0.0009) [2023-12-26 20:47:49,897][105692] Updated weights for policy 0, policy_version 746354 (0.0011) [2023-12-26 20:47:49,966][105692] Updated weights for policy 0, policy_version 746364 (0.0011) [2023-12-26 20:47:50,023][105692] Updated weights for policy 0, policy_version 746374 (0.0009) [2023-12-26 20:47:50,330][105620] Updated weights for policy 1, policy_version 746672 (0.0009) [2023-12-26 20:47:50,396][105620] Updated weights for policy 1, policy_version 746682 (0.0007) [2023-12-26 20:47:50,459][105620] Updated weights for policy 1, policy_version 746692 (0.0007) [2023-12-26 20:47:50,718][105692] Updated weights for policy 0, policy_version 746384 (0.0009) [2023-12-26 20:47:50,785][105692] Updated weights for policy 0, policy_version 746394 (0.0009) [2023-12-26 20:47:50,834][105692] Updated weights for policy 0, policy_version 746404 (0.0009) [2023-12-26 20:47:51,062][104569] Fps is (10 sec: 20479.3, 60 sec: 19797.2, 300 sec: 19633.0). Total num frames: 382287872. Throughput: 0: 9752.9, 1: 10108.3. Samples: 382276932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-12-26 20:47:51,063][104569] Avg episode reward: [(0, '9347.265'), (1, '8323.934')] [2023-12-26 20:47:51,232][105620] Updated weights for policy 1, policy_version 746702 (0.0009) [2023-12-26 20:47:51,302][105620] Updated weights for policy 1, policy_version 746712 (0.0009) [2023-12-26 20:47:51,366][105620] Updated weights for policy 1, policy_version 746722 (0.0008) [2023-12-26 20:47:51,626][105692] Updated weights for policy 0, policy_version 746414 (0.0009) [2023-12-26 20:47:51,682][105692] Updated weights for policy 0, policy_version 746424 (0.0009) [2023-12-26 20:47:51,743][105692] Updated weights for policy 0, policy_version 746434 (0.0008) [2023-12-26 20:47:52,117][105620] Updated weights for policy 1, policy_version 746732 (0.0009) [2023-12-26 20:47:52,174][105620] Updated weights for policy 1, policy_version 746742 (0.0009) [2023-12-26 20:47:52,236][105620] Updated weights for policy 1, policy_version 746752 (0.0006) [2023-12-26 20:47:52,491][105692] Updated weights for policy 0, policy_version 746444 (0.0007) [2023-12-26 20:47:52,547][105692] Updated weights for policy 0, policy_version 746454 (0.0009) [2023-12-26 20:47:52,607][105692] Updated weights for policy 0, policy_version 746464 (0.0009) [2023-12-26 20:47:52,951][105620] Updated weights for policy 1, policy_version 746762 (0.0008) [2023-12-26 20:47:53,019][105620] Updated weights for policy 1, policy_version 746772 (0.0010) [2023-12-26 20:47:53,081][105620] Updated weights for policy 1, policy_version 746782 (0.0009) [2023-12-26 20:47:53,142][105620] Updated weights for policy 1, policy_version 746792 (0.0010) [2023-12-26 20:47:53,268][105692] Updated weights for policy 0, policy_version 746474 (0.0008) [2023-12-26 20:47:53,320][105692] Updated weights for policy 0, policy_version 746484 (0.0008) [2023-12-26 20:47:53,368][105692] Updated weights for policy 0, policy_version 746494 (0.0009) [2023-12-26 20:47:53,419][105692] Updated weights for policy 0, policy_version 746504 (0.0009) [2023-12-26 20:47:53,896][105620] Updated weights for policy 1, policy_version 746802 (0.0006) [2023-12-26 20:47:53,961][105620] Updated weights for policy 1, policy_version 746812 (0.0005) [2023-12-26 20:47:54,025][105620] Updated weights for policy 1, policy_version 746822 (0.0007) [2023-12-26 20:47:54,051][105692] Updated weights for policy 0, policy_version 746514 (0.0010) [2023-12-26 20:47:54,099][105692] Updated weights for policy 0, policy_version 746524 (0.0010) [2023-12-26 20:47:54,148][105692] Updated weights for policy 0, policy_version 746534 (0.0010) [2023-12-26 20:47:54,623][105620] Updated weights for policy 1, policy_version 746832 (0.0008) [2023-12-26 20:47:54,674][105620] Updated weights for policy 1, policy_version 746842 (0.0005) [2023-12-26 20:47:54,723][105620] Updated weights for policy 1, policy_version 746852 (0.0005) [2023-12-26 20:47:54,918][105692] Updated weights for policy 0, policy_version 746544 (0.0007) [2023-12-26 20:47:54,973][105692] Updated weights for policy 0, policy_version 746554 (0.0006) [2023-12-26 20:47:55,029][105692] Updated weights for policy 0, policy_version 746564 (0.0008) [2023-12-26 20:47:55,513][105620] Updated weights for policy 1, policy_version 746862 (0.0008) [2023-12-26 20:47:55,568][105620] Updated weights for policy 1, policy_version 746872 (0.0009) [2023-12-26 20:47:55,618][105620] Updated weights for policy 1, policy_version 746882 (0.0009) [2023-12-26 20:47:55,641][105692] Updated weights for policy 0, policy_version 746574 (0.0008) [2023-12-26 20:47:55,690][105692] Updated weights for policy 0, policy_version 746584 (0.0007) [2023-12-26 20:47:55,746][105692] Updated weights for policy 0, policy_version 746594 (0.0005) [2023-12-26 20:47:56,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.2, 300 sec: 19633.0). Total num frames: 382386176. Throughput: 0: 9859.1, 1: 9995.0. Samples: 382393028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-12-26 20:47:56,063][104569] Avg episode reward: [(0, '9255.527'), (1, '8436.955')] [2023-12-26 20:47:56,357][105692] Updated weights for policy 0, policy_version 746604 (0.0006) [2023-12-26 20:47:56,422][105692] Updated weights for policy 0, policy_version 746614 (0.0005) [2023-12-26 20:47:56,478][105692] Updated weights for policy 0, policy_version 746624 (0.0005) [2023-12-26 20:47:56,478][105620] Updated weights for policy 1, policy_version 746892 (0.0010) [2023-12-26 20:47:56,528][105620] Updated weights for policy 1, policy_version 746902 (0.0009) [2023-12-26 20:47:56,581][105620] Updated weights for policy 1, policy_version 746913 (0.0008) [2023-12-26 20:47:57,120][105692] Updated weights for policy 0, policy_version 746634 (0.0006) [2023-12-26 20:47:57,184][105692] Updated weights for policy 0, policy_version 746644 (0.0009) [2023-12-26 20:47:57,245][105692] Updated weights for policy 0, policy_version 746654 (0.0009) [2023-12-26 20:47:57,278][105620] Updated weights for policy 1, policy_version 746923 (0.0008) [2023-12-26 20:47:57,293][105692] Updated weights for policy 0, policy_version 746664 (0.0007) [2023-12-26 20:47:57,335][105620] Updated weights for policy 1, policy_version 746933 (0.0009) [2023-12-26 20:47:57,395][105620] Updated weights for policy 1, policy_version 746944 (0.0010) [2023-12-26 20:47:58,082][105692] Updated weights for policy 0, policy_version 746674 (0.0009) [2023-12-26 20:47:58,144][105620] Updated weights for policy 1, policy_version 746954 (0.0009) [2023-12-26 20:47:58,146][105692] Updated weights for policy 0, policy_version 746684 (0.0008) [2023-12-26 20:47:58,208][105692] Updated weights for policy 0, policy_version 746694 (0.0008) [2023-12-26 20:47:58,209][105620] Updated weights for policy 1, policy_version 746964 (0.0011) [2023-12-26 20:47:58,273][105620] Updated weights for policy 1, policy_version 746974 (0.0009) [2023-12-26 20:47:58,338][105620] Updated weights for policy 1, policy_version 746984 (0.0010) [2023-12-26 20:47:59,048][105692] Updated weights for policy 0, policy_version 746704 (0.0008) [2023-12-26 20:47:59,097][105692] Updated weights for policy 0, policy_version 746714 (0.0008) [2023-12-26 20:47:59,151][105692] Updated weights for policy 0, policy_version 746724 (0.0008) [2023-12-26 20:47:59,158][105620] Updated weights for policy 1, policy_version 746994 (0.0011) [2023-12-26 20:47:59,224][105620] Updated weights for policy 1, policy_version 747004 (0.0010) [2023-12-26 20:47:59,290][105620] Updated weights for policy 1, policy_version 747014 (0.0009) [2023-12-26 20:47:59,918][105692] Updated weights for policy 0, policy_version 746734 (0.0006) [2023-12-26 20:47:59,981][105692] Updated weights for policy 0, policy_version 746744 (0.0007) [2023-12-26 20:47:59,996][105620] Updated weights for policy 1, policy_version 747024 (0.0006) [2023-12-26 20:48:00,041][105692] Updated weights for policy 0, policy_version 746754 (0.0008) [2023-12-26 20:48:00,050][105620] Updated weights for policy 1, policy_version 747034 (0.0008) [2023-12-26 20:48:00,097][105620] Updated weights for policy 1, policy_version 747044 (0.0007) [2023-12-26 20:48:00,747][105620] Updated weights for policy 1, policy_version 747054 (0.0007) [2023-12-26 20:48:00,804][105692] Updated weights for policy 0, policy_version 746764 (0.0006) [2023-12-26 20:48:00,815][105620] Updated weights for policy 1, policy_version 747064 (0.0005) [2023-12-26 20:48:00,865][105692] Updated weights for policy 0, policy_version 746774 (0.0008) [2023-12-26 20:48:00,880][105620] Updated weights for policy 1, policy_version 747074 (0.0005) [2023-12-26 20:48:00,911][105692] Updated weights for policy 0, policy_version 746784 (0.0007) [2023-12-26 20:48:01,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 382484480. Throughput: 0: 9880.3, 1: 9972.6. Samples: 382450100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-12-26 20:48:01,062][104569] Avg episode reward: [(0, '9255.281'), (1, '8521.963')] [2023-12-26 20:48:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000746792_191209472.pth... [2023-12-26 20:48:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000747080_191275008.pth... [2023-12-26 20:48:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000745640_190914560.pth [2023-12-26 20:48:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000745928_190980096.pth [2023-12-26 20:48:01,473][105620] Updated weights for policy 1, policy_version 747084 (0.0005) [2023-12-26 20:48:01,542][105620] Updated weights for policy 1, policy_version 747094 (0.0006) [2023-12-26 20:48:01,593][105692] Updated weights for policy 0, policy_version 746794 (0.0007) [2023-12-26 20:48:01,605][105620] Updated weights for policy 1, policy_version 747104 (0.0010) [2023-12-26 20:48:01,657][105692] Updated weights for policy 0, policy_version 746804 (0.0010) [2023-12-26 20:48:01,726][105692] Updated weights for policy 0, policy_version 746814 (0.0009) [2023-12-26 20:48:01,791][105692] Updated weights for policy 0, policy_version 746824 (0.0010) [2023-12-26 20:48:02,183][105620] Updated weights for policy 1, policy_version 747114 (0.0009) [2023-12-26 20:48:02,229][105620] Updated weights for policy 1, policy_version 747124 (0.0005) [2023-12-26 20:48:02,292][105620] Updated weights for policy 1, policy_version 747134 (0.0007) [2023-12-26 20:48:02,362][105620] Updated weights for policy 1, policy_version 747144 (0.0008) [2023-12-26 20:48:02,484][105692] Updated weights for policy 0, policy_version 746834 (0.0010) [2023-12-26 20:48:02,538][105692] Updated weights for policy 0, policy_version 746844 (0.0010) [2023-12-26 20:48:02,589][105692] Updated weights for policy 0, policy_version 746854 (0.0005) [2023-12-26 20:48:03,059][105620] Updated weights for policy 1, policy_version 747154 (0.0010) [2023-12-26 20:48:03,109][105620] Updated weights for policy 1, policy_version 747164 (0.0010) [2023-12-26 20:48:03,157][105620] Updated weights for policy 1, policy_version 747174 (0.0010) [2023-12-26 20:48:03,290][105692] Updated weights for policy 0, policy_version 746864 (0.0007) [2023-12-26 20:48:03,352][105692] Updated weights for policy 0, policy_version 746874 (0.0008) [2023-12-26 20:48:03,400][105692] Updated weights for policy 0, policy_version 746884 (0.0008) [2023-12-26 20:48:03,812][105620] Updated weights for policy 1, policy_version 747184 (0.0010) [2023-12-26 20:48:03,865][105620] Updated weights for policy 1, policy_version 747194 (0.0011) [2023-12-26 20:48:03,914][105620] Updated weights for policy 1, policy_version 747204 (0.0011) [2023-12-26 20:48:04,243][105692] Updated weights for policy 0, policy_version 746894 (0.0008) [2023-12-26 20:48:04,303][105692] Updated weights for policy 0, policy_version 746904 (0.0009) [2023-12-26 20:48:04,363][105692] Updated weights for policy 0, policy_version 746914 (0.0010) [2023-12-26 20:48:04,607][105620] Updated weights for policy 1, policy_version 747214 (0.0007) [2023-12-26 20:48:04,662][105620] Updated weights for policy 1, policy_version 747224 (0.0005) [2023-12-26 20:48:04,723][105620] Updated weights for policy 1, policy_version 747234 (0.0005) [2023-12-26 20:48:05,238][105620] Updated weights for policy 1, policy_version 747244 (0.0007) [2023-12-26 20:48:05,278][105692] Updated weights for policy 0, policy_version 746924 (0.0008) [2023-12-26 20:48:05,301][105620] Updated weights for policy 1, policy_version 747254 (0.0010) [2023-12-26 20:48:05,330][105692] Updated weights for policy 0, policy_version 746934 (0.0005) [2023-12-26 20:48:05,356][105620] Updated weights for policy 1, policy_version 747264 (0.0010) [2023-12-26 20:48:05,387][105692] Updated weights for policy 0, policy_version 746944 (0.0005) [2023-12-26 20:48:05,937][105692] Updated weights for policy 0, policy_version 746954 (0.0005) [2023-12-26 20:48:05,988][105692] Updated weights for policy 0, policy_version 746964 (0.0005) [2023-12-26 20:48:06,047][105692] Updated weights for policy 0, policy_version 746974 (0.0006) [2023-12-26 20:48:06,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 382574592. Throughput: 0: 9695.6, 1: 10011.2. Samples: 382568600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-12-26 20:48:06,062][104569] Avg episode reward: [(0, '9255.697'), (1, '8038.665')] [2023-12-26 20:48:06,089][105620] Updated weights for policy 1, policy_version 747274 (0.0011) [2023-12-26 20:48:06,122][105692] Updated weights for policy 0, policy_version 746984 (0.0007) [2023-12-26 20:48:06,153][105620] Updated weights for policy 1, policy_version 747284 (0.0011) [2023-12-26 20:48:06,208][105620] Updated weights for policy 1, policy_version 747294 (0.0010) [2023-12-26 20:48:06,259][105620] Updated weights for policy 1, policy_version 747304 (0.0010) [2023-12-26 20:48:06,794][105692] Updated weights for policy 0, policy_version 746994 (0.0008) [2023-12-26 20:48:06,854][105692] Updated weights for policy 0, policy_version 747004 (0.0008) [2023-12-26 20:48:06,902][105692] Updated weights for policy 0, policy_version 747014 (0.0008) [2023-12-26 20:48:07,025][105620] Updated weights for policy 1, policy_version 747314 (0.0010) [2023-12-26 20:48:07,091][105620] Updated weights for policy 1, policy_version 747324 (0.0011) [2023-12-26 20:48:07,151][105620] Updated weights for policy 1, policy_version 747334 (0.0011) [2023-12-26 20:48:07,749][105692] Updated weights for policy 0, policy_version 747024 (0.0008) [2023-12-26 20:48:07,756][105620] Updated weights for policy 1, policy_version 747344 (0.0006) [2023-12-26 20:48:07,809][105692] Updated weights for policy 0, policy_version 747034 (0.0009) [2023-12-26 20:48:07,813][105620] Updated weights for policy 1, policy_version 747354 (0.0006) [2023-12-26 20:48:07,858][105692] Updated weights for policy 0, policy_version 747044 (0.0008) [2023-12-26 20:48:07,869][105620] Updated weights for policy 1, policy_version 747364 (0.0005) [2023-12-26 20:48:08,440][105620] Updated weights for policy 1, policy_version 747374 (0.0005) [2023-12-26 20:48:08,504][105620] Updated weights for policy 1, policy_version 747384 (0.0006) [2023-12-26 20:48:08,563][105620] Updated weights for policy 1, policy_version 747394 (0.0006) [2023-12-26 20:48:08,655][105692] Updated weights for policy 0, policy_version 747054 (0.0008) [2023-12-26 20:48:08,703][105692] Updated weights for policy 0, policy_version 747064 (0.0007) [2023-12-26 20:48:08,762][105692] Updated weights for policy 0, policy_version 747074 (0.0008) [2023-12-26 20:48:09,144][105620] Updated weights for policy 1, policy_version 747404 (0.0009) [2023-12-26 20:48:09,190][105620] Updated weights for policy 1, policy_version 747414 (0.0005) [2023-12-26 20:48:09,251][105620] Updated weights for policy 1, policy_version 747424 (0.0007) [2023-12-26 20:48:09,633][105692] Updated weights for policy 0, policy_version 747084 (0.0008) [2023-12-26 20:48:09,697][105692] Updated weights for policy 0, policy_version 747094 (0.0009) [2023-12-26 20:48:09,746][105692] Updated weights for policy 0, policy_version 747104 (0.0007) [2023-12-26 20:48:10,004][105620] Updated weights for policy 1, policy_version 747434 (0.0010) [2023-12-26 20:48:10,062][105620] Updated weights for policy 1, policy_version 747444 (0.0010) [2023-12-26 20:48:10,128][105620] Updated weights for policy 1, policy_version 747454 (0.0010) [2023-12-26 20:48:10,191][105620] Updated weights for policy 1, policy_version 747464 (0.0009) [2023-12-26 20:48:10,464][105692] Updated weights for policy 0, policy_version 747114 (0.0009) [2023-12-26 20:48:10,533][105692] Updated weights for policy 0, policy_version 747124 (0.0010) [2023-12-26 20:48:10,593][105692] Updated weights for policy 0, policy_version 747134 (0.0010) [2023-12-26 20:48:10,659][105692] Updated weights for policy 0, policy_version 747144 (0.0005) [2023-12-26 20:48:10,893][105620] Updated weights for policy 1, policy_version 747474 (0.0008) [2023-12-26 20:48:10,945][105620] Updated weights for policy 1, policy_version 747484 (0.0008) [2023-12-26 20:48:11,009][105620] Updated weights for policy 1, policy_version 747494 (0.0008) [2023-12-26 20:48:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 382681088. Throughput: 0: 9619.9, 1: 10002.2. Samples: 382686020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-12-26 20:48:11,063][104569] Avg episode reward: [(0, '9164.644'), (1, '7889.830')] [2023-12-26 20:48:11,328][105692] Updated weights for policy 0, policy_version 747154 (0.0010) [2023-12-26 20:48:11,392][105692] Updated weights for policy 0, policy_version 747164 (0.0008) [2023-12-26 20:48:11,440][105692] Updated weights for policy 0, policy_version 747174 (0.0008) [2023-12-26 20:48:11,792][105620] Updated weights for policy 1, policy_version 747504 (0.0009) [2023-12-26 20:48:11,858][105620] Updated weights for policy 1, policy_version 747514 (0.0009) [2023-12-26 20:48:11,926][105620] Updated weights for policy 1, policy_version 747524 (0.0008) [2023-12-26 20:48:12,256][105692] Updated weights for policy 0, policy_version 747184 (0.0010) [2023-12-26 20:48:12,312][105692] Updated weights for policy 0, policy_version 747194 (0.0009) [2023-12-26 20:48:12,379][105692] Updated weights for policy 0, policy_version 747204 (0.0009) [2023-12-26 20:48:12,719][105620] Updated weights for policy 1, policy_version 747534 (0.0009) [2023-12-26 20:48:12,778][105620] Updated weights for policy 1, policy_version 747544 (0.0008) [2023-12-26 20:48:12,827][105620] Updated weights for policy 1, policy_version 747554 (0.0008) [2023-12-26 20:48:13,170][105692] Updated weights for policy 0, policy_version 747214 (0.0010) [2023-12-26 20:48:13,238][105692] Updated weights for policy 0, policy_version 747224 (0.0010) [2023-12-26 20:48:13,292][105692] Updated weights for policy 0, policy_version 747234 (0.0010) [2023-12-26 20:48:13,544][105620] Updated weights for policy 1, policy_version 747564 (0.0009) [2023-12-26 20:48:13,596][105620] Updated weights for policy 1, policy_version 747574 (0.0010) [2023-12-26 20:48:13,647][105620] Updated weights for policy 1, policy_version 747584 (0.0010) [2023-12-26 20:48:14,021][105692] Updated weights for policy 0, policy_version 747244 (0.0010) [2023-12-26 20:48:14,079][105692] Updated weights for policy 0, policy_version 747254 (0.0010) [2023-12-26 20:48:14,132][105692] Updated weights for policy 0, policy_version 747264 (0.0010) [2023-12-26 20:48:14,421][105620] Updated weights for policy 1, policy_version 747594 (0.0009) [2023-12-26 20:48:14,471][105620] Updated weights for policy 1, policy_version 747604 (0.0007) [2023-12-26 20:48:14,518][105620] Updated weights for policy 1, policy_version 747614 (0.0007) [2023-12-26 20:48:14,568][105620] Updated weights for policy 1, policy_version 747624 (0.0006) [2023-12-26 20:48:14,822][105692] Updated weights for policy 0, policy_version 747274 (0.0009) [2023-12-26 20:48:14,879][105692] Updated weights for policy 0, policy_version 747284 (0.0010) [2023-12-26 20:48:14,937][105692] Updated weights for policy 0, policy_version 747294 (0.0011) [2023-12-26 20:48:15,001][105692] Updated weights for policy 0, policy_version 747304 (0.0011) [2023-12-26 20:48:15,350][105620] Updated weights for policy 1, policy_version 747634 (0.0008) [2023-12-26 20:48:15,411][105620] Updated weights for policy 1, policy_version 747644 (0.0008) [2023-12-26 20:48:15,475][105620] Updated weights for policy 1, policy_version 747654 (0.0008) [2023-12-26 20:48:15,759][105692] Updated weights for policy 0, policy_version 747314 (0.0011) [2023-12-26 20:48:15,806][105692] Updated weights for policy 0, policy_version 747324 (0.0010) [2023-12-26 20:48:15,855][105692] Updated weights for policy 0, policy_version 747334 (0.0011) [2023-12-26 20:48:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 382771200. Throughput: 0: 9576.9, 1: 9948.0. Samples: 382741724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-12-26 20:48:16,063][104569] Avg episode reward: [(0, '9165.174'), (1, '8256.201')] [2023-12-26 20:48:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000747336_191348736.pth... [2023-12-26 20:48:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000747656_191422464.pth... [2023-12-26 20:48:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000746504_191127552.pth [2023-12-26 20:48:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000746216_191062016.pth [2023-12-26 20:48:16,276][105620] Updated weights for policy 1, policy_version 747664 (0.0008) [2023-12-26 20:48:16,334][105620] Updated weights for policy 1, policy_version 747674 (0.0008) [2023-12-26 20:48:16,393][105620] Updated weights for policy 1, policy_version 747684 (0.0008) [2023-12-26 20:48:16,513][105692] Updated weights for policy 0, policy_version 747344 (0.0010) [2023-12-26 20:48:16,568][105692] Updated weights for policy 0, policy_version 747354 (0.0010) [2023-12-26 20:48:16,617][105692] Updated weights for policy 0, policy_version 747364 (0.0006) [2023-12-26 20:48:17,163][105692] Updated weights for policy 0, policy_version 747374 (0.0005) [2023-12-26 20:48:17,213][105692] Updated weights for policy 0, policy_version 747384 (0.0005) [2023-12-26 20:48:17,265][105692] Updated weights for policy 0, policy_version 747394 (0.0005) [2023-12-26 20:48:17,269][105620] Updated weights for policy 1, policy_version 747694 (0.0008) [2023-12-26 20:48:17,323][105620] Updated weights for policy 1, policy_version 747704 (0.0008) [2023-12-26 20:48:17,376][105620] Updated weights for policy 1, policy_version 747715 (0.0010) [2023-12-26 20:48:17,819][105692] Updated weights for policy 0, policy_version 747404 (0.0010) [2023-12-26 20:48:17,874][105692] Updated weights for policy 0, policy_version 747414 (0.0010) [2023-12-26 20:48:17,930][105692] Updated weights for policy 0, policy_version 747424 (0.0011) [2023-12-26 20:48:18,232][105620] Updated weights for policy 1, policy_version 747725 (0.0010) [2023-12-26 20:48:18,289][105620] Updated weights for policy 1, policy_version 747735 (0.0008) [2023-12-26 20:48:18,344][105620] Updated weights for policy 1, policy_version 747745 (0.0009) [2023-12-26 20:48:18,645][105692] Updated weights for policy 0, policy_version 747434 (0.0010) [2023-12-26 20:48:18,708][105692] Updated weights for policy 0, policy_version 747444 (0.0005) [2023-12-26 20:48:18,771][105692] Updated weights for policy 0, policy_version 747454 (0.0007) [2023-12-26 20:48:18,820][105692] Updated weights for policy 0, policy_version 747464 (0.0010) [2023-12-26 20:48:19,209][105620] Updated weights for policy 1, policy_version 747755 (0.0009) [2023-12-26 20:48:19,268][105620] Updated weights for policy 1, policy_version 747765 (0.0008) [2023-12-26 20:48:19,327][105620] Updated weights for policy 1, policy_version 747775 (0.0010) [2023-12-26 20:48:19,389][105692] Updated weights for policy 0, policy_version 747474 (0.0008) [2023-12-26 20:48:19,450][105692] Updated weights for policy 0, policy_version 747484 (0.0008) [2023-12-26 20:48:19,508][105692] Updated weights for policy 0, policy_version 747494 (0.0008) [2023-12-26 20:48:20,011][105620] Updated weights for policy 1, policy_version 747785 (0.0008) [2023-12-26 20:48:20,074][105620] Updated weights for policy 1, policy_version 747795 (0.0008) [2023-12-26 20:48:20,140][105620] Updated weights for policy 1, policy_version 747805 (0.0008) [2023-12-26 20:48:20,195][105620] Updated weights for policy 1, policy_version 747815 (0.0008) [2023-12-26 20:48:20,268][105692] Updated weights for policy 0, policy_version 747504 (0.0010) [2023-12-26 20:48:20,335][105692] Updated weights for policy 0, policy_version 747514 (0.0010) [2023-12-26 20:48:20,387][105692] Updated weights for policy 0, policy_version 747524 (0.0011) [2023-12-26 20:48:20,988][105620] Updated weights for policy 1, policy_version 747825 (0.0009) [2023-12-26 20:48:21,061][105620] Updated weights for policy 1, policy_version 747835 (0.0008) [2023-12-26 20:48:21,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 382861312. Throughput: 0: 9727.0, 1: 9799.9. Samples: 382857720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-12-26 20:48:21,063][104569] Avg episode reward: [(0, '9347.692'), (1, '9137.337')] [2023-12-26 20:48:21,126][105620] Updated weights for policy 1, policy_version 747845 (0.0009) [2023-12-26 20:48:21,160][105692] Updated weights for policy 0, policy_version 747534 (0.0011) [2023-12-26 20:48:21,221][105692] Updated weights for policy 0, policy_version 747544 (0.0011) [2023-12-26 20:48:21,289][105692] Updated weights for policy 0, policy_version 747554 (0.0011) [2023-12-26 20:48:21,929][105620] Updated weights for policy 1, policy_version 747855 (0.0007) [2023-12-26 20:48:21,941][105692] Updated weights for policy 0, policy_version 747564 (0.0010) [2023-12-26 20:48:21,992][105620] Updated weights for policy 1, policy_version 747865 (0.0006) [2023-12-26 20:48:21,998][105692] Updated weights for policy 0, policy_version 747574 (0.0009) [2023-12-26 20:48:22,052][105620] Updated weights for policy 1, policy_version 747875 (0.0006) [2023-12-26 20:48:22,057][105692] Updated weights for policy 0, policy_version 747584 (0.0009) [2023-12-26 20:48:22,728][105620] Updated weights for policy 1, policy_version 747885 (0.0008) [2023-12-26 20:48:22,779][105692] Updated weights for policy 0, policy_version 747594 (0.0009) [2023-12-26 20:48:22,792][105620] Updated weights for policy 1, policy_version 747895 (0.0010) [2023-12-26 20:48:22,831][105692] Updated weights for policy 0, policy_version 747604 (0.0007) [2023-12-26 20:48:22,853][105620] Updated weights for policy 1, policy_version 747905 (0.0009) [2023-12-26 20:48:22,884][105692] Updated weights for policy 0, policy_version 747614 (0.0006) [2023-12-26 20:48:22,937][105692] Updated weights for policy 0, policy_version 747624 (0.0006) [2023-12-26 20:48:23,620][105692] Updated weights for policy 0, policy_version 747634 (0.0006) [2023-12-26 20:48:23,667][105620] Updated weights for policy 1, policy_version 747915 (0.0010) [2023-12-26 20:48:23,677][105692] Updated weights for policy 0, policy_version 747644 (0.0005) [2023-12-26 20:48:23,714][105620] Updated weights for policy 1, policy_version 747925 (0.0008) [2023-12-26 20:48:23,725][105692] Updated weights for policy 0, policy_version 747654 (0.0006) [2023-12-26 20:48:23,772][105620] Updated weights for policy 1, policy_version 747935 (0.0008) [2023-12-26 20:48:24,431][105692] Updated weights for policy 0, policy_version 747664 (0.0009) [2023-12-26 20:48:24,459][105620] Updated weights for policy 1, policy_version 747945 (0.0010) [2023-12-26 20:48:24,485][105692] Updated weights for policy 0, policy_version 747674 (0.0009) [2023-12-26 20:48:24,521][105620] Updated weights for policy 1, policy_version 747955 (0.0010) [2023-12-26 20:48:24,550][105692] Updated weights for policy 0, policy_version 747684 (0.0010) [2023-12-26 20:48:24,577][105620] Updated weights for policy 1, policy_version 747965 (0.0010) [2023-12-26 20:48:24,637][105620] Updated weights for policy 1, policy_version 747975 (0.0010) [2023-12-26 20:48:25,329][105692] Updated weights for policy 0, policy_version 747694 (0.0008) [2023-12-26 20:48:25,389][105620] Updated weights for policy 1, policy_version 747985 (0.0007) [2023-12-26 20:48:25,393][105692] Updated weights for policy 0, policy_version 747704 (0.0008) [2023-12-26 20:48:25,435][105620] Updated weights for policy 1, policy_version 747995 (0.0007) [2023-12-26 20:48:25,445][105692] Updated weights for policy 0, policy_version 747714 (0.0009) [2023-12-26 20:48:25,485][105620] Updated weights for policy 1, policy_version 748005 (0.0007) [2023-12-26 20:48:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 382959616. Throughput: 0: 9782.4, 1: 9660.7. Samples: 382970828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-12-26 20:48:26,062][104569] Avg episode reward: [(0, '9347.211'), (1, '9135.936')] [2023-12-26 20:48:26,167][105692] Updated weights for policy 0, policy_version 747724 (0.0008) [2023-12-26 20:48:26,207][105620] Updated weights for policy 1, policy_version 748015 (0.0007) [2023-12-26 20:48:26,219][105692] Updated weights for policy 0, policy_version 747734 (0.0008) [2023-12-26 20:48:26,266][105620] Updated weights for policy 1, policy_version 748025 (0.0006) [2023-12-26 20:48:26,272][105692] Updated weights for policy 0, policy_version 747744 (0.0007) [2023-12-26 20:48:26,325][105620] Updated weights for policy 1, policy_version 748035 (0.0010) [2023-12-26 20:48:26,901][105620] Updated weights for policy 1, policy_version 748045 (0.0008) [2023-12-26 20:48:26,921][105692] Updated weights for policy 0, policy_version 747754 (0.0006) [2023-12-26 20:48:26,953][105620] Updated weights for policy 1, policy_version 748055 (0.0005) [2023-12-26 20:48:26,969][105692] Updated weights for policy 0, policy_version 747764 (0.0007) [2023-12-26 20:48:27,001][105620] Updated weights for policy 1, policy_version 748065 (0.0005) [2023-12-26 20:48:27,022][105692] Updated weights for policy 0, policy_version 747774 (0.0009) [2023-12-26 20:48:27,073][105692] Updated weights for policy 0, policy_version 747784 (0.0009) [2023-12-26 20:48:27,516][105620] Updated weights for policy 1, policy_version 748075 (0.0006) [2023-12-26 20:48:27,577][105620] Updated weights for policy 1, policy_version 748085 (0.0009) [2023-12-26 20:48:27,625][105620] Updated weights for policy 1, policy_version 748095 (0.0010) [2023-12-26 20:48:27,707][105692] Updated weights for policy 0, policy_version 747794 (0.0007) [2023-12-26 20:48:27,761][105692] Updated weights for policy 0, policy_version 747804 (0.0008) [2023-12-26 20:48:27,812][105692] Updated weights for policy 0, policy_version 747814 (0.0007) [2023-12-26 20:48:28,224][105620] Updated weights for policy 1, policy_version 748105 (0.0010) [2023-12-26 20:48:28,274][105620] Updated weights for policy 1, policy_version 748115 (0.0007) [2023-12-26 20:48:28,331][105620] Updated weights for policy 1, policy_version 748125 (0.0011) [2023-12-26 20:48:28,391][105620] Updated weights for policy 1, policy_version 748135 (0.0010) [2023-12-26 20:48:28,614][105692] Updated weights for policy 0, policy_version 747824 (0.0009) [2023-12-26 20:48:28,676][105692] Updated weights for policy 0, policy_version 747834 (0.0009) [2023-12-26 20:48:28,732][105692] Updated weights for policy 0, policy_version 747844 (0.0008) [2023-12-26 20:48:29,109][105620] Updated weights for policy 1, policy_version 748145 (0.0010) [2023-12-26 20:48:29,159][105620] Updated weights for policy 1, policy_version 748155 (0.0010) [2023-12-26 20:48:29,224][105620] Updated weights for policy 1, policy_version 748165 (0.0010) [2023-12-26 20:48:29,481][105692] Updated weights for policy 0, policy_version 747854 (0.0007) [2023-12-26 20:48:29,532][105692] Updated weights for policy 0, policy_version 747864 (0.0005) [2023-12-26 20:48:29,594][105692] Updated weights for policy 0, policy_version 747874 (0.0006) [2023-12-26 20:48:29,986][105620] Updated weights for policy 1, policy_version 748175 (0.0009) [2023-12-26 20:48:30,040][105620] Updated weights for policy 1, policy_version 748185 (0.0008) [2023-12-26 20:48:30,089][105620] Updated weights for policy 1, policy_version 748195 (0.0007) [2023-12-26 20:48:30,364][105692] Updated weights for policy 0, policy_version 747884 (0.0008) [2023-12-26 20:48:30,414][105692] Updated weights for policy 0, policy_version 747894 (0.0008) [2023-12-26 20:48:30,484][105692] Updated weights for policy 0, policy_version 747904 (0.0009) [2023-12-26 20:48:30,787][105620] Updated weights for policy 1, policy_version 748205 (0.0008) [2023-12-26 20:48:30,841][105620] Updated weights for policy 1, policy_version 748215 (0.0010) [2023-12-26 20:48:30,885][105620] Updated weights for policy 1, policy_version 748225 (0.0010) [2023-12-26 20:48:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 383066112. Throughput: 0: 9826.6, 1: 9789.7. Samples: 383035260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-12-26 20:48:31,062][104569] Avg episode reward: [(0, '9259.612'), (1, '9081.426')] [2023-12-26 20:48:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000748232_191569920.pth... [2023-12-26 20:48:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000747912_191496192.pth... [2023-12-26 20:48:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000747080_191275008.pth [2023-12-26 20:48:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000746792_191209472.pth [2023-12-26 20:48:31,240][105692] Updated weights for policy 0, policy_version 747914 (0.0008) [2023-12-26 20:48:31,306][105692] Updated weights for policy 0, policy_version 747924 (0.0009) [2023-12-26 20:48:31,369][105692] Updated weights for policy 0, policy_version 747934 (0.0008) [2023-12-26 20:48:31,431][105692] Updated weights for policy 0, policy_version 747944 (0.0007) [2023-12-26 20:48:31,659][105620] Updated weights for policy 1, policy_version 748235 (0.0010) [2023-12-26 20:48:31,721][105620] Updated weights for policy 1, policy_version 748245 (0.0010) [2023-12-26 20:48:31,783][105620] Updated weights for policy 1, policy_version 748255 (0.0010) [2023-12-26 20:48:32,200][105692] Updated weights for policy 0, policy_version 747954 (0.0010) [2023-12-26 20:48:32,259][105692] Updated weights for policy 0, policy_version 747964 (0.0010) [2023-12-26 20:48:32,317][105692] Updated weights for policy 0, policy_version 747974 (0.0007) [2023-12-26 20:48:32,487][105620] Updated weights for policy 1, policy_version 748265 (0.0010) [2023-12-26 20:48:32,554][105620] Updated weights for policy 1, policy_version 748275 (0.0005) [2023-12-26 20:48:32,621][105620] Updated weights for policy 1, policy_version 748285 (0.0006) [2023-12-26 20:48:32,679][105620] Updated weights for policy 1, policy_version 748295 (0.0005) [2023-12-26 20:48:32,949][105692] Updated weights for policy 0, policy_version 747984 (0.0007) [2023-12-26 20:48:33,007][105692] Updated weights for policy 0, policy_version 747994 (0.0008) [2023-12-26 20:48:33,062][105692] Updated weights for policy 0, policy_version 748004 (0.0009) [2023-12-26 20:48:33,324][105620] Updated weights for policy 1, policy_version 748305 (0.0009) [2023-12-26 20:48:33,381][105620] Updated weights for policy 1, policy_version 748315 (0.0008) [2023-12-26 20:48:33,428][105620] Updated weights for policy 1, policy_version 748325 (0.0009) [2023-12-26 20:48:33,788][105692] Updated weights for policy 0, policy_version 748014 (0.0010) [2023-12-26 20:48:33,844][105692] Updated weights for policy 0, policy_version 748024 (0.0010) [2023-12-26 20:48:33,905][105692] Updated weights for policy 0, policy_version 748034 (0.0010) [2023-12-26 20:48:34,099][105620] Updated weights for policy 1, policy_version 748335 (0.0006) [2023-12-26 20:48:34,171][105620] Updated weights for policy 1, policy_version 748346 (0.0007) [2023-12-26 20:48:34,225][105620] Updated weights for policy 1, policy_version 748356 (0.0006) [2023-12-26 20:48:34,629][105692] Updated weights for policy 0, policy_version 748044 (0.0010) [2023-12-26 20:48:34,698][105692] Updated weights for policy 0, policy_version 748054 (0.0011) [2023-12-26 20:48:34,752][105692] Updated weights for policy 0, policy_version 748064 (0.0010) [2023-12-26 20:48:34,866][105620] Updated weights for policy 1, policy_version 748366 (0.0006) [2023-12-26 20:48:34,929][105620] Updated weights for policy 1, policy_version 748376 (0.0009) [2023-12-26 20:48:34,993][105620] Updated weights for policy 1, policy_version 748386 (0.0010) [2023-12-26 20:48:35,373][105692] Updated weights for policy 0, policy_version 748074 (0.0009) [2023-12-26 20:48:35,423][105692] Updated weights for policy 0, policy_version 748084 (0.0006) [2023-12-26 20:48:35,472][105692] Updated weights for policy 0, policy_version 748094 (0.0005) [2023-12-26 20:48:35,537][105692] Updated weights for policy 0, policy_version 748104 (0.0008) [2023-12-26 20:48:35,617][105620] Updated weights for policy 1, policy_version 748396 (0.0009) [2023-12-26 20:48:35,670][105620] Updated weights for policy 1, policy_version 748407 (0.0010) [2023-12-26 20:48:35,722][105620] Updated weights for policy 1, policy_version 748417 (0.0008) [2023-12-26 20:48:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 383164416. Throughput: 0: 9698.5, 1: 9752.3. Samples: 383152208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-12-26 20:48:36,062][104569] Avg episode reward: [(0, '9259.719'), (1, '8991.112')] [2023-12-26 20:48:36,080][105692] Updated weights for policy 0, policy_version 748114 (0.0005) [2023-12-26 20:48:36,144][105692] Updated weights for policy 0, policy_version 748124 (0.0008) [2023-12-26 20:48:36,205][105692] Updated weights for policy 0, policy_version 748134 (0.0010) [2023-12-26 20:48:36,388][105620] Updated weights for policy 1, policy_version 748427 (0.0007) [2023-12-26 20:48:36,451][105620] Updated weights for policy 1, policy_version 748437 (0.0008) [2023-12-26 20:48:36,516][105620] Updated weights for policy 1, policy_version 748447 (0.0006) [2023-12-26 20:48:36,865][105692] Updated weights for policy 0, policy_version 748144 (0.0010) [2023-12-26 20:48:36,917][105692] Updated weights for policy 0, policy_version 748154 (0.0009) [2023-12-26 20:48:36,973][105692] Updated weights for policy 0, policy_version 748164 (0.0009) [2023-12-26 20:48:37,156][105620] Updated weights for policy 1, policy_version 748457 (0.0006) [2023-12-26 20:48:37,205][105620] Updated weights for policy 1, policy_version 748467 (0.0009) [2023-12-26 20:48:37,257][105620] Updated weights for policy 1, policy_version 748477 (0.0009) [2023-12-26 20:48:37,313][105620] Updated weights for policy 1, policy_version 748487 (0.0010) [2023-12-26 20:48:37,612][105692] Updated weights for policy 0, policy_version 748174 (0.0007) [2023-12-26 20:48:37,677][105692] Updated weights for policy 0, policy_version 748184 (0.0008) [2023-12-26 20:48:37,731][105692] Updated weights for policy 0, policy_version 748194 (0.0010) [2023-12-26 20:48:38,119][105620] Updated weights for policy 1, policy_version 748497 (0.0006) [2023-12-26 20:48:38,178][105620] Updated weights for policy 1, policy_version 748507 (0.0007) [2023-12-26 20:48:38,235][105620] Updated weights for policy 1, policy_version 748517 (0.0009) [2023-12-26 20:48:38,499][105692] Updated weights for policy 0, policy_version 748204 (0.0009) [2023-12-26 20:48:38,559][105692] Updated weights for policy 0, policy_version 748214 (0.0007) [2023-12-26 20:48:38,613][105692] Updated weights for policy 0, policy_version 748224 (0.0010) [2023-12-26 20:48:38,867][105620] Updated weights for policy 1, policy_version 748527 (0.0009) [2023-12-26 20:48:38,931][105620] Updated weights for policy 1, policy_version 748537 (0.0009) [2023-12-26 20:48:38,993][105620] Updated weights for policy 1, policy_version 748547 (0.0009) [2023-12-26 20:48:39,253][105692] Updated weights for policy 0, policy_version 748234 (0.0009) [2023-12-26 20:48:39,316][105692] Updated weights for policy 0, policy_version 748244 (0.0009) [2023-12-26 20:48:39,384][105692] Updated weights for policy 0, policy_version 748254 (0.0009) [2023-12-26 20:48:39,452][105692] Updated weights for policy 0, policy_version 748264 (0.0007) [2023-12-26 20:48:39,791][105620] Updated weights for policy 1, policy_version 748557 (0.0009) [2023-12-26 20:48:39,857][105620] Updated weights for policy 1, policy_version 748567 (0.0008) [2023-12-26 20:48:39,921][105620] Updated weights for policy 1, policy_version 748577 (0.0008) [2023-12-26 20:48:40,193][105692] Updated weights for policy 0, policy_version 748274 (0.0006) [2023-12-26 20:48:40,262][105692] Updated weights for policy 0, policy_version 748284 (0.0006) [2023-12-26 20:48:40,330][105692] Updated weights for policy 0, policy_version 748294 (0.0008) [2023-12-26 20:48:40,665][105620] Updated weights for policy 1, policy_version 748587 (0.0008) [2023-12-26 20:48:40,729][105620] Updated weights for policy 1, policy_version 748597 (0.0009) [2023-12-26 20:48:40,796][105620] Updated weights for policy 1, policy_version 748607 (0.0007) [2023-12-26 20:48:40,873][105692] Updated weights for policy 0, policy_version 748304 (0.0006) [2023-12-26 20:48:40,938][105692] Updated weights for policy 0, policy_version 748314 (0.0008) [2023-12-26 20:48:40,997][105692] Updated weights for policy 0, policy_version 748324 (0.0010) [2023-12-26 20:48:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 383270912. Throughput: 0: 9770.4, 1: 9803.4. Samples: 383273844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-12-26 20:48:41,062][104569] Avg episode reward: [(0, '9347.456'), (1, '9082.749')] [2023-12-26 20:48:41,507][105620] Updated weights for policy 1, policy_version 748617 (0.0006) [2023-12-26 20:48:41,556][105620] Updated weights for policy 1, policy_version 748627 (0.0009) [2023-12-26 20:48:41,610][105620] Updated weights for policy 1, policy_version 748637 (0.0009) [2023-12-26 20:48:41,676][105620] Updated weights for policy 1, policy_version 748647 (0.0006) [2023-12-26 20:48:41,773][105692] Updated weights for policy 0, policy_version 748334 (0.0010) [2023-12-26 20:48:41,836][105692] Updated weights for policy 0, policy_version 748344 (0.0009) [2023-12-26 20:48:41,889][105692] Updated weights for policy 0, policy_version 748354 (0.0009) [2023-12-26 20:48:42,474][105620] Updated weights for policy 1, policy_version 748657 (0.0008) [2023-12-26 20:48:42,535][105620] Updated weights for policy 1, policy_version 748667 (0.0008) [2023-12-26 20:48:42,579][105692] Updated weights for policy 0, policy_version 748364 (0.0009) [2023-12-26 20:48:42,592][105620] Updated weights for policy 1, policy_version 748677 (0.0006) [2023-12-26 20:48:42,632][105692] Updated weights for policy 0, policy_version 748374 (0.0009) [2023-12-26 20:48:42,688][105692] Updated weights for policy 0, policy_version 748384 (0.0009) [2023-12-26 20:48:43,271][105620] Updated weights for policy 1, policy_version 748687 (0.0009) [2023-12-26 20:48:43,336][105620] Updated weights for policy 1, policy_version 748697 (0.0009) [2023-12-26 20:48:43,387][105620] Updated weights for policy 1, policy_version 748707 (0.0009) [2023-12-26 20:48:43,441][105692] Updated weights for policy 0, policy_version 748394 (0.0009) [2023-12-26 20:48:43,497][105692] Updated weights for policy 0, policy_version 748404 (0.0008) [2023-12-26 20:48:43,558][105692] Updated weights for policy 0, policy_version 748414 (0.0005) [2023-12-26 20:48:43,604][105692] Updated weights for policy 0, policy_version 748424 (0.0008) [2023-12-26 20:48:44,168][105620] Updated weights for policy 1, policy_version 748717 (0.0008) [2023-12-26 20:48:44,223][105620] Updated weights for policy 1, policy_version 748727 (0.0008) [2023-12-26 20:48:44,229][105692] Updated weights for policy 0, policy_version 748434 (0.0007) [2023-12-26 20:48:44,273][105620] Updated weights for policy 1, policy_version 748737 (0.0006) [2023-12-26 20:48:44,279][105692] Updated weights for policy 0, policy_version 748444 (0.0006) [2023-12-26 20:48:44,336][105692] Updated weights for policy 0, policy_version 748454 (0.0006) [2023-12-26 20:48:45,020][105692] Updated weights for policy 0, policy_version 748464 (0.0008) [2023-12-26 20:48:45,077][105692] Updated weights for policy 0, policy_version 748474 (0.0008) [2023-12-26 20:48:45,086][105620] Updated weights for policy 1, policy_version 748747 (0.0009) [2023-12-26 20:48:45,142][105692] Updated weights for policy 0, policy_version 748484 (0.0007) [2023-12-26 20:48:45,148][105620] Updated weights for policy 1, policy_version 748757 (0.0007) [2023-12-26 20:48:45,213][105620] Updated weights for policy 1, policy_version 748767 (0.0008) [2023-12-26 20:48:45,852][105692] Updated weights for policy 0, policy_version 748494 (0.0008) [2023-12-26 20:48:45,908][105692] Updated weights for policy 0, policy_version 748504 (0.0006) [2023-12-26 20:48:45,960][105692] Updated weights for policy 0, policy_version 748514 (0.0005) [2023-12-26 20:48:46,007][105620] Updated weights for policy 1, policy_version 748777 (0.0009) [2023-12-26 20:48:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19605.2). Total num frames: 383361024. Throughput: 0: 9759.1, 1: 9814.7. Samples: 383330924. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-12-26 20:48:46,062][105620] Updated weights for policy 1, policy_version 748787 (0.0008) [2023-12-26 20:48:46,062][104569] Avg episode reward: [(0, '9256.537'), (1, '9172.291')] [2023-12-26 20:48:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000748520_191651840.pth... [2023-12-26 20:48:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000747336_191348736.pth [2023-12-26 20:48:46,126][105620] Updated weights for policy 1, policy_version 748797 (0.0009) [2023-12-26 20:48:46,180][105620] Updated weights for policy 1, policy_version 748807 (0.0010) [2023-12-26 20:48:46,181][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000748808_191717376.pth... [2023-12-26 20:48:46,184][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000747656_191422464.pth [2023-12-26 20:48:46,531][105692] Updated weights for policy 0, policy_version 748524 (0.0010) [2023-12-26 20:48:46,576][105692] Updated weights for policy 0, policy_version 748534 (0.0006) [2023-12-26 20:48:46,623][105692] Updated weights for policy 0, policy_version 748544 (0.0008) [2023-12-26 20:48:47,038][105620] Updated weights for policy 1, policy_version 748817 (0.0009) [2023-12-26 20:48:47,104][105620] Updated weights for policy 1, policy_version 748827 (0.0008) [2023-12-26 20:48:47,152][105620] Updated weights for policy 1, policy_version 748837 (0.0008) [2023-12-26 20:48:47,324][105692] Updated weights for policy 0, policy_version 748554 (0.0007) [2023-12-26 20:48:47,377][105692] Updated weights for policy 0, policy_version 748564 (0.0006) [2023-12-26 20:48:47,426][105692] Updated weights for policy 0, policy_version 748574 (0.0005) [2023-12-26 20:48:47,473][105692] Updated weights for policy 0, policy_version 748584 (0.0005) [2023-12-26 20:48:47,993][105692] Updated weights for policy 0, policy_version 748594 (0.0005) [2023-12-26 20:48:48,025][105620] Updated weights for policy 1, policy_version 748847 (0.0009) [2023-12-26 20:48:48,043][105692] Updated weights for policy 0, policy_version 748604 (0.0005) [2023-12-26 20:48:48,085][105620] Updated weights for policy 1, policy_version 748857 (0.0009) [2023-12-26 20:48:48,098][105692] Updated weights for policy 0, policy_version 748614 (0.0005) [2023-12-26 20:48:48,141][105620] Updated weights for policy 1, policy_version 748868 (0.0010) [2023-12-26 20:48:48,732][105692] Updated weights for policy 0, policy_version 748624 (0.0010) [2023-12-26 20:48:48,793][105692] Updated weights for policy 0, policy_version 748634 (0.0011) [2023-12-26 20:48:48,849][105692] Updated weights for policy 0, policy_version 748644 (0.0011) [2023-12-26 20:48:48,972][105620] Updated weights for policy 1, policy_version 748878 (0.0008) [2023-12-26 20:48:49,025][105620] Updated weights for policy 1, policy_version 748888 (0.0009) [2023-12-26 20:48:49,074][105620] Updated weights for policy 1, policy_version 748898 (0.0008) [2023-12-26 20:48:49,617][105692] Updated weights for policy 0, policy_version 748654 (0.0009) [2023-12-26 20:48:49,672][105692] Updated weights for policy 0, policy_version 748664 (0.0008) [2023-12-26 20:48:49,728][105692] Updated weights for policy 0, policy_version 748674 (0.0011) [2023-12-26 20:48:49,876][105620] Updated weights for policy 1, policy_version 748908 (0.0009) [2023-12-26 20:48:49,944][105620] Updated weights for policy 1, policy_version 748918 (0.0008) [2023-12-26 20:48:50,001][105620] Updated weights for policy 1, policy_version 748928 (0.0007) [2023-12-26 20:48:50,461][105692] Updated weights for policy 0, policy_version 748684 (0.0008) [2023-12-26 20:48:50,531][105692] Updated weights for policy 0, policy_version 748694 (0.0006) [2023-12-26 20:48:50,593][105692] Updated weights for policy 0, policy_version 748704 (0.0008) [2023-12-26 20:48:50,758][105620] Updated weights for policy 1, policy_version 748938 (0.0008) [2023-12-26 20:48:50,819][105620] Updated weights for policy 1, policy_version 748948 (0.0009) [2023-12-26 20:48:50,871][105620] Updated weights for policy 1, policy_version 748958 (0.0009) [2023-12-26 20:48:50,929][105620] Updated weights for policy 1, policy_version 748968 (0.0010) [2023-12-26 20:48:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.4, 300 sec: 19605.3). Total num frames: 383459328. Throughput: 0: 9954.1, 1: 9569.6. Samples: 383447168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-12-26 20:48:51,062][104569] Avg episode reward: [(0, '9074.789'), (1, '9080.780')] [2023-12-26 20:48:51,246][105692] Updated weights for policy 0, policy_version 748714 (0.0008) [2023-12-26 20:48:51,303][105692] Updated weights for policy 0, policy_version 748724 (0.0009) [2023-12-26 20:48:51,370][105692] Updated weights for policy 0, policy_version 748734 (0.0008) [2023-12-26 20:48:51,429][105692] Updated weights for policy 0, policy_version 748744 (0.0007) [2023-12-26 20:48:51,702][105620] Updated weights for policy 1, policy_version 748978 (0.0010) [2023-12-26 20:48:51,771][105620] Updated weights for policy 1, policy_version 748988 (0.0008) [2023-12-26 20:48:51,834][105620] Updated weights for policy 1, policy_version 748998 (0.0008) [2023-12-26 20:48:52,169][105692] Updated weights for policy 0, policy_version 748754 (0.0009) [2023-12-26 20:48:52,231][105692] Updated weights for policy 0, policy_version 748764 (0.0009) [2023-12-26 20:48:52,283][105692] Updated weights for policy 0, policy_version 748774 (0.0009) [2023-12-26 20:48:52,596][105620] Updated weights for policy 1, policy_version 749008 (0.0008) [2023-12-26 20:48:52,648][105620] Updated weights for policy 1, policy_version 749018 (0.0008) [2023-12-26 20:48:52,702][105620] Updated weights for policy 1, policy_version 749028 (0.0008) [2023-12-26 20:48:53,011][105692] Updated weights for policy 0, policy_version 748784 (0.0006) [2023-12-26 20:48:53,076][105692] Updated weights for policy 0, policy_version 748794 (0.0009) [2023-12-26 20:48:53,134][105692] Updated weights for policy 0, policy_version 748804 (0.0010) [2023-12-26 20:48:53,459][105620] Updated weights for policy 1, policy_version 749038 (0.0010) [2023-12-26 20:48:53,508][105620] Updated weights for policy 1, policy_version 749048 (0.0010) [2023-12-26 20:48:53,562][105620] Updated weights for policy 1, policy_version 749058 (0.0010) [2023-12-26 20:48:53,814][105692] Updated weights for policy 0, policy_version 748814 (0.0011) [2023-12-26 20:48:53,862][105692] Updated weights for policy 0, policy_version 748824 (0.0011) [2023-12-26 20:48:53,914][105692] Updated weights for policy 0, policy_version 748834 (0.0010) [2023-12-26 20:48:54,322][105620] Updated weights for policy 1, policy_version 749068 (0.0009) [2023-12-26 20:48:54,379][105620] Updated weights for policy 1, policy_version 749078 (0.0008) [2023-12-26 20:48:54,435][105620] Updated weights for policy 1, policy_version 749088 (0.0005) [2023-12-26 20:48:54,642][105692] Updated weights for policy 0, policy_version 748844 (0.0008) [2023-12-26 20:48:54,685][105692] Updated weights for policy 0, policy_version 748854 (0.0005) [2023-12-26 20:48:54,731][105692] Updated weights for policy 0, policy_version 748864 (0.0005) [2023-12-26 20:48:55,283][105692] Updated weights for policy 0, policy_version 748874 (0.0006) [2023-12-26 20:48:55,289][105620] Updated weights for policy 1, policy_version 749098 (0.0008) [2023-12-26 20:48:55,336][105692] Updated weights for policy 0, policy_version 748884 (0.0010) [2023-12-26 20:48:55,342][105620] Updated weights for policy 1, policy_version 749108 (0.0006) [2023-12-26 20:48:55,390][105692] Updated weights for policy 0, policy_version 748894 (0.0010) [2023-12-26 20:48:55,397][105620] Updated weights for policy 1, policy_version 749118 (0.0006) [2023-12-26 20:48:55,449][105692] Updated weights for policy 0, policy_version 748904 (0.0011) [2023-12-26 20:48:55,460][105620] Updated weights for policy 1, policy_version 749128 (0.0006) [2023-12-26 20:48:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 383549440. Throughput: 0: 10054.2, 1: 9401.7. Samples: 383561540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-12-26 20:48:56,063][104569] Avg episode reward: [(0, '8984.499'), (1, '8905.852')] [2023-12-26 20:48:56,163][105692] Updated weights for policy 0, policy_version 748914 (0.0010) [2023-12-26 20:48:56,182][105620] Updated weights for policy 1, policy_version 749138 (0.0005) [2023-12-26 20:48:56,212][105692] Updated weights for policy 0, policy_version 748924 (0.0010) [2023-12-26 20:48:56,244][105620] Updated weights for policy 1, policy_version 749148 (0.0005) [2023-12-26 20:48:56,267][105692] Updated weights for policy 0, policy_version 748934 (0.0010) [2023-12-26 20:48:56,305][105620] Updated weights for policy 1, policy_version 749158 (0.0005) [2023-12-26 20:48:56,931][105620] Updated weights for policy 1, policy_version 749168 (0.0007) [2023-12-26 20:48:56,981][105620] Updated weights for policy 1, policy_version 749178 (0.0008) [2023-12-26 20:48:57,020][105692] Updated weights for policy 0, policy_version 748944 (0.0010) [2023-12-26 20:48:57,037][105620] Updated weights for policy 1, policy_version 749188 (0.0008) [2023-12-26 20:48:57,077][105692] Updated weights for policy 0, policy_version 748954 (0.0009) [2023-12-26 20:48:57,132][105692] Updated weights for policy 0, policy_version 748964 (0.0005) [2023-12-26 20:48:57,662][105692] Updated weights for policy 0, policy_version 748974 (0.0005) [2023-12-26 20:48:57,726][105692] Updated weights for policy 0, policy_version 748984 (0.0007) [2023-12-26 20:48:57,791][105692] Updated weights for policy 0, policy_version 748994 (0.0010) [2023-12-26 20:48:57,904][105620] Updated weights for policy 1, policy_version 749198 (0.0007) [2023-12-26 20:48:57,948][105620] Updated weights for policy 1, policy_version 749208 (0.0008) [2023-12-26 20:48:58,009][105620] Updated weights for policy 1, policy_version 749218 (0.0008) [2023-12-26 20:48:58,496][105692] Updated weights for policy 0, policy_version 749004 (0.0010) [2023-12-26 20:48:58,565][105692] Updated weights for policy 0, policy_version 749014 (0.0010) [2023-12-26 20:48:58,632][105692] Updated weights for policy 0, policy_version 749024 (0.0010) [2023-12-26 20:48:58,813][105620] Updated weights for policy 1, policy_version 749228 (0.0008) [2023-12-26 20:48:58,884][105620] Updated weights for policy 1, policy_version 749238 (0.0009) [2023-12-26 20:48:58,955][105620] Updated weights for policy 1, policy_version 749248 (0.0009) [2023-12-26 20:48:59,361][105692] Updated weights for policy 0, policy_version 749034 (0.0008) [2023-12-26 20:48:59,424][105692] Updated weights for policy 0, policy_version 749044 (0.0008) [2023-12-26 20:48:59,475][105692] Updated weights for policy 0, policy_version 749054 (0.0008) [2023-12-26 20:48:59,524][105692] Updated weights for policy 0, policy_version 749064 (0.0008) [2023-12-26 20:48:59,715][105620] Updated weights for policy 1, policy_version 749258 (0.0009) [2023-12-26 20:48:59,774][105620] Updated weights for policy 1, policy_version 749268 (0.0007) [2023-12-26 20:48:59,832][105620] Updated weights for policy 1, policy_version 749279 (0.0010) [2023-12-26 20:49:00,251][105692] Updated weights for policy 0, policy_version 749074 (0.0009) [2023-12-26 20:49:00,278][105585] KL-divergence is very high: 127.4245 [2023-12-26 20:49:00,313][105692] Updated weights for policy 0, policy_version 749084 (0.0009) [2023-12-26 20:49:00,326][105585] KL-divergence is very high: 180.6454 [2023-12-26 20:49:00,379][105692] Updated weights for policy 0, policy_version 749094 (0.0009) [2023-12-26 20:49:00,379][105585] KL-divergence is very high: 175.4146 [2023-12-26 20:49:00,532][105620] Updated weights for policy 1, policy_version 749289 (0.0009) [2023-12-26 20:49:00,588][105620] Updated weights for policy 1, policy_version 749299 (0.0006) [2023-12-26 20:49:00,644][105620] Updated weights for policy 1, policy_version 749309 (0.0007) [2023-12-26 20:49:00,697][105620] Updated weights for policy 1, policy_version 749319 (0.0010) [2023-12-26 20:49:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 383647744. Throughput: 0: 10121.2, 1: 9398.7. Samples: 383620116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-12-26 20:49:01,062][104569] Avg episode reward: [(0, '8902.965'), (1, '8819.967')] [2023-12-26 20:49:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000749096_191799296.pth... [2023-12-26 20:49:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000749320_191848448.pth... [2023-12-26 20:49:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000748232_191569920.pth [2023-12-26 20:49:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000747912_191496192.pth [2023-12-26 20:49:01,226][105692] Updated weights for policy 0, policy_version 749104 (0.0011) [2023-12-26 20:49:01,275][105620] Updated weights for policy 1, policy_version 749329 (0.0010) [2023-12-26 20:49:01,288][105692] Updated weights for policy 0, policy_version 749114 (0.0010) [2023-12-26 20:49:01,335][105620] Updated weights for policy 1, policy_version 749339 (0.0009) [2023-12-26 20:49:01,345][105692] Updated weights for policy 0, policy_version 749124 (0.0009) [2023-12-26 20:49:01,399][105620] Updated weights for policy 1, policy_version 749349 (0.0012) [2023-12-26 20:49:02,127][105620] Updated weights for policy 1, policy_version 749359 (0.0009) [2023-12-26 20:49:02,142][105692] Updated weights for policy 0, policy_version 749134 (0.0007) [2023-12-26 20:49:02,181][105620] Updated weights for policy 1, policy_version 749369 (0.0008) [2023-12-26 20:49:02,190][105692] Updated weights for policy 0, policy_version 749144 (0.0007) [2023-12-26 20:49:02,226][105620] Updated weights for policy 1, policy_version 749379 (0.0006) [2023-12-26 20:49:02,240][105692] Updated weights for policy 0, policy_version 749154 (0.0007) [2023-12-26 20:49:02,941][105620] Updated weights for policy 1, policy_version 749389 (0.0008) [2023-12-26 20:49:02,993][105620] Updated weights for policy 1, policy_version 749399 (0.0009) [2023-12-26 20:49:03,028][105692] Updated weights for policy 0, policy_version 749164 (0.0007) [2023-12-26 20:49:03,043][105620] Updated weights for policy 1, policy_version 749409 (0.0009) [2023-12-26 20:49:03,079][105692] Updated weights for policy 0, policy_version 749174 (0.0005) [2023-12-26 20:49:03,130][105692] Updated weights for policy 0, policy_version 749184 (0.0005) [2023-12-26 20:49:03,637][105692] Updated weights for policy 0, policy_version 749194 (0.0005) [2023-12-26 20:49:03,681][105692] Updated weights for policy 0, policy_version 749204 (0.0005) [2023-12-26 20:49:03,724][105692] Updated weights for policy 0, policy_version 749214 (0.0005) [2023-12-26 20:49:03,744][105620] Updated weights for policy 1, policy_version 749419 (0.0007) [2023-12-26 20:49:03,770][105692] Updated weights for policy 0, policy_version 749224 (0.0005) [2023-12-26 20:49:03,793][105620] Updated weights for policy 1, policy_version 749429 (0.0005) [2023-12-26 20:49:03,846][105620] Updated weights for policy 1, policy_version 749439 (0.0006) [2023-12-26 20:49:04,488][105692] Updated weights for policy 0, policy_version 749234 (0.0010) [2023-12-26 20:49:04,540][105620] Updated weights for policy 1, policy_version 749449 (0.0011) [2023-12-26 20:49:04,543][105692] Updated weights for policy 0, policy_version 749245 (0.0009) [2023-12-26 20:49:04,599][105620] Updated weights for policy 1, policy_version 749459 (0.0007) [2023-12-26 20:49:04,609][105692] Updated weights for policy 0, policy_version 749255 (0.0006) [2023-12-26 20:49:04,663][105620] Updated weights for policy 1, policy_version 749469 (0.0006) [2023-12-26 20:49:04,729][105620] Updated weights for policy 1, policy_version 749479 (0.0006) [2023-12-26 20:49:05,364][105692] Updated weights for policy 0, policy_version 749265 (0.0009) [2023-12-26 20:49:05,420][105692] Updated weights for policy 0, policy_version 749275 (0.0009) [2023-12-26 20:49:05,449][105620] Updated weights for policy 1, policy_version 749489 (0.0009) [2023-12-26 20:49:05,465][105692] Updated weights for policy 0, policy_version 749285 (0.0007) [2023-12-26 20:49:05,508][105620] Updated weights for policy 1, policy_version 749499 (0.0009) [2023-12-26 20:49:05,572][105620] Updated weights for policy 1, policy_version 749509 (0.0009) [2023-12-26 20:49:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 383746048. Throughput: 0: 9986.1, 1: 9559.4. Samples: 383737272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-12-26 20:49:06,063][104569] Avg episode reward: [(0, '9084.199'), (1, '8994.364')] [2023-12-26 20:49:06,242][105692] Updated weights for policy 0, policy_version 749295 (0.0011) [2023-12-26 20:49:06,305][105692] Updated weights for policy 0, policy_version 749305 (0.0011) [2023-12-26 20:49:06,331][105620] Updated weights for policy 1, policy_version 749519 (0.0007) [2023-12-26 20:49:06,361][105692] Updated weights for policy 0, policy_version 749315 (0.0010) [2023-12-26 20:49:06,387][105620] Updated weights for policy 1, policy_version 749529 (0.0005) [2023-12-26 20:49:06,435][105620] Updated weights for policy 1, policy_version 749539 (0.0008) [2023-12-26 20:49:07,097][105692] Updated weights for policy 0, policy_version 749325 (0.0010) [2023-12-26 20:49:07,149][105692] Updated weights for policy 0, policy_version 749335 (0.0010) [2023-12-26 20:49:07,202][105620] Updated weights for policy 1, policy_version 749549 (0.0007) [2023-12-26 20:49:07,208][105692] Updated weights for policy 0, policy_version 749345 (0.0011) [2023-12-26 20:49:07,258][105620] Updated weights for policy 1, policy_version 749559 (0.0006) [2023-12-26 20:49:07,313][105620] Updated weights for policy 1, policy_version 749569 (0.0009) [2023-12-26 20:49:07,967][105692] Updated weights for policy 0, policy_version 749355 (0.0011) [2023-12-26 20:49:08,029][105692] Updated weights for policy 0, policy_version 749365 (0.0010) [2023-12-26 20:49:08,075][105620] Updated weights for policy 1, policy_version 749579 (0.0007) [2023-12-26 20:49:08,080][105692] Updated weights for policy 0, policy_version 749375 (0.0010) [2023-12-26 20:49:08,134][105620] Updated weights for policy 1, policy_version 749589 (0.0006) [2023-12-26 20:49:08,200][105620] Updated weights for policy 1, policy_version 749599 (0.0008) [2023-12-26 20:49:08,873][105692] Updated weights for policy 0, policy_version 749385 (0.0010) [2023-12-26 20:49:08,908][105620] Updated weights for policy 1, policy_version 749609 (0.0008) [2023-12-26 20:49:08,936][105692] Updated weights for policy 0, policy_version 749395 (0.0006) [2023-12-26 20:49:08,991][105620] Updated weights for policy 1, policy_version 749619 (0.0009) [2023-12-26 20:49:08,998][105692] Updated weights for policy 0, policy_version 749405 (0.0005) [2023-12-26 20:49:09,050][105620] Updated weights for policy 1, policy_version 749629 (0.0007) [2023-12-26 20:49:09,060][105692] Updated weights for policy 0, policy_version 749415 (0.0006) [2023-12-26 20:49:09,100][105620] Updated weights for policy 1, policy_version 749639 (0.0008) [2023-12-26 20:49:09,657][105692] Updated weights for policy 0, policy_version 749425 (0.0008) [2023-12-26 20:49:09,716][105692] Updated weights for policy 0, policy_version 749435 (0.0009) [2023-12-26 20:49:09,786][105692] Updated weights for policy 0, policy_version 749445 (0.0006) [2023-12-26 20:49:09,931][105620] Updated weights for policy 1, policy_version 749649 (0.0010) [2023-12-26 20:49:10,002][105620] Updated weights for policy 1, policy_version 749659 (0.0009) [2023-12-26 20:49:10,060][105620] Updated weights for policy 1, policy_version 749669 (0.0007) [2023-12-26 20:49:10,574][105692] Updated weights for policy 0, policy_version 749455 (0.0008) [2023-12-26 20:49:10,624][105692] Updated weights for policy 0, policy_version 749465 (0.0009) [2023-12-26 20:49:10,655][105620] Updated weights for policy 1, policy_version 749679 (0.0008) [2023-12-26 20:49:10,677][105692] Updated weights for policy 0, policy_version 749475 (0.0007) [2023-12-26 20:49:10,711][105620] Updated weights for policy 1, policy_version 749689 (0.0006) [2023-12-26 20:49:10,773][105620] Updated weights for policy 1, policy_version 749699 (0.0009) [2023-12-26 20:49:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 383844352. Throughput: 0: 9972.0, 1: 9587.5. Samples: 383851004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-12-26 20:49:11,062][104569] Avg episode reward: [(0, '9338.079'), (1, '8903.253')] [2023-12-26 20:49:11,512][105692] Updated weights for policy 0, policy_version 749485 (0.0008) [2023-12-26 20:49:11,565][105620] Updated weights for policy 1, policy_version 749709 (0.0006) [2023-12-26 20:49:11,573][105692] Updated weights for policy 0, policy_version 749495 (0.0008) [2023-12-26 20:49:11,627][105620] Updated weights for policy 1, policy_version 749719 (0.0006) [2023-12-26 20:49:11,638][105692] Updated weights for policy 0, policy_version 749505 (0.0009) [2023-12-26 20:49:11,689][105620] Updated weights for policy 1, policy_version 749729 (0.0006) [2023-12-26 20:49:12,409][105692] Updated weights for policy 0, policy_version 749515 (0.0010) [2023-12-26 20:49:12,423][105620] Updated weights for policy 1, policy_version 749739 (0.0008) [2023-12-26 20:49:12,470][105692] Updated weights for policy 0, policy_version 749525 (0.0006) [2023-12-26 20:49:12,484][105620] Updated weights for policy 1, policy_version 749749 (0.0007) [2023-12-26 20:49:12,527][105692] Updated weights for policy 0, policy_version 749535 (0.0007) [2023-12-26 20:49:12,541][105620] Updated weights for policy 1, policy_version 749759 (0.0007) [2023-12-26 20:49:13,250][105692] Updated weights for policy 0, policy_version 749545 (0.0005) [2023-12-26 20:49:13,296][105620] Updated weights for policy 1, policy_version 749769 (0.0007) [2023-12-26 20:49:13,306][105692] Updated weights for policy 0, policy_version 749555 (0.0009) [2023-12-26 20:49:13,358][105620] Updated weights for policy 1, policy_version 749779 (0.0008) [2023-12-26 20:49:13,363][105692] Updated weights for policy 0, policy_version 749565 (0.0006) [2023-12-26 20:49:13,409][105620] Updated weights for policy 1, policy_version 749789 (0.0007) [2023-12-26 20:49:13,419][105692] Updated weights for policy 0, policy_version 749575 (0.0010) [2023-12-26 20:49:13,456][105620] Updated weights for policy 1, policy_version 749799 (0.0007) [2023-12-26 20:49:14,142][105692] Updated weights for policy 0, policy_version 749585 (0.0006) [2023-12-26 20:49:14,200][105692] Updated weights for policy 0, policy_version 749595 (0.0008) [2023-12-26 20:49:14,229][105620] Updated weights for policy 1, policy_version 749809 (0.0007) [2023-12-26 20:49:14,261][105692] Updated weights for policy 0, policy_version 749605 (0.0010) [2023-12-26 20:49:14,283][105620] Updated weights for policy 1, policy_version 749819 (0.0006) [2023-12-26 20:49:14,344][105620] Updated weights for policy 1, policy_version 749829 (0.0008) [2023-12-26 20:49:14,959][105692] Updated weights for policy 0, policy_version 749615 (0.0010) [2023-12-26 20:49:15,022][105692] Updated weights for policy 0, policy_version 749625 (0.0010) [2023-12-26 20:49:15,070][105620] Updated weights for policy 1, policy_version 749839 (0.0009) [2023-12-26 20:49:15,081][105692] Updated weights for policy 0, policy_version 749635 (0.0010) [2023-12-26 20:49:15,120][105620] Updated weights for policy 1, policy_version 749849 (0.0006) [2023-12-26 20:49:15,177][105620] Updated weights for policy 1, policy_version 749859 (0.0008) [2023-12-26 20:49:15,761][105692] Updated weights for policy 0, policy_version 749645 (0.0008) [2023-12-26 20:49:15,811][105620] Updated weights for policy 1, policy_version 749869 (0.0006) [2023-12-26 20:49:15,826][105692] Updated weights for policy 0, policy_version 749655 (0.0006) [2023-12-26 20:49:15,869][105620] Updated weights for policy 1, policy_version 749879 (0.0010) [2023-12-26 20:49:15,882][105692] Updated weights for policy 0, policy_version 749665 (0.0005) [2023-12-26 20:49:15,927][105620] Updated weights for policy 1, policy_version 749889 (0.0005) [2023-12-26 20:49:16,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19524.1, 300 sec: 19605.2). Total num frames: 383942656. Throughput: 0: 9910.7, 1: 9438.9. Samples: 383906004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-12-26 20:49:16,064][104569] Avg episode reward: [(0, '9348.108'), (1, '8810.918')] [2023-12-26 20:49:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000749672_191946752.pth... [2023-12-26 20:49:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000749896_191995904.pth... [2023-12-26 20:49:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000748808_191717376.pth [2023-12-26 20:49:16,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000748520_191651840.pth [2023-12-26 20:49:16,494][105692] Updated weights for policy 0, policy_version 749675 (0.0006) [2023-12-26 20:49:16,554][105692] Updated weights for policy 0, policy_version 749685 (0.0005) [2023-12-26 20:49:16,594][105620] Updated weights for policy 1, policy_version 749899 (0.0007) [2023-12-26 20:49:16,611][105692] Updated weights for policy 0, policy_version 749695 (0.0008) [2023-12-26 20:49:16,646][105620] Updated weights for policy 1, policy_version 749909 (0.0010) [2023-12-26 20:49:16,704][105620] Updated weights for policy 1, policy_version 749919 (0.0010) [2023-12-26 20:49:17,240][105692] Updated weights for policy 0, policy_version 749705 (0.0006) [2023-12-26 20:49:17,298][105692] Updated weights for policy 0, policy_version 749715 (0.0008) [2023-12-26 20:49:17,374][105692] Updated weights for policy 0, policy_version 749725 (0.0011) [2023-12-26 20:49:17,431][105692] Updated weights for policy 0, policy_version 749735 (0.0010) [2023-12-26 20:49:17,464][105620] Updated weights for policy 1, policy_version 749929 (0.0010) [2023-12-26 20:49:17,525][105620] Updated weights for policy 1, policy_version 749939 (0.0010) [2023-12-26 20:49:17,573][105620] Updated weights for policy 1, policy_version 749949 (0.0010) [2023-12-26 20:49:17,618][105620] Updated weights for policy 1, policy_version 749959 (0.0010) [2023-12-26 20:49:18,029][105692] Updated weights for policy 0, policy_version 749745 (0.0011) [2023-12-26 20:49:18,083][105692] Updated weights for policy 0, policy_version 749755 (0.0007) [2023-12-26 20:49:18,133][105692] Updated weights for policy 0, policy_version 749765 (0.0008) [2023-12-26 20:49:18,378][105620] Updated weights for policy 1, policy_version 749969 (0.0008) [2023-12-26 20:49:18,446][105620] Updated weights for policy 1, policy_version 749979 (0.0005) [2023-12-26 20:49:18,499][105620] Updated weights for policy 1, policy_version 749989 (0.0009) [2023-12-26 20:49:18,855][105692] Updated weights for policy 0, policy_version 749775 (0.0009) [2023-12-26 20:49:18,910][105692] Updated weights for policy 0, policy_version 749785 (0.0010) [2023-12-26 20:49:18,968][105692] Updated weights for policy 0, policy_version 749795 (0.0010) [2023-12-26 20:49:19,232][105620] Updated weights for policy 1, policy_version 749999 (0.0011) [2023-12-26 20:49:19,293][105620] Updated weights for policy 1, policy_version 750009 (0.0010) [2023-12-26 20:49:19,360][105620] Updated weights for policy 1, policy_version 750019 (0.0009) [2023-12-26 20:49:19,729][105692] Updated weights for policy 0, policy_version 749805 (0.0008) [2023-12-26 20:49:19,788][105692] Updated weights for policy 0, policy_version 749815 (0.0007) [2023-12-26 20:49:19,855][105692] Updated weights for policy 0, policy_version 749825 (0.0008) [2023-12-26 20:49:20,055][105620] Updated weights for policy 1, policy_version 750029 (0.0008) [2023-12-26 20:49:20,114][105620] Updated weights for policy 1, policy_version 750039 (0.0010) [2023-12-26 20:49:20,177][105620] Updated weights for policy 1, policy_version 750049 (0.0010) [2023-12-26 20:49:20,581][105692] Updated weights for policy 0, policy_version 749835 (0.0007) [2023-12-26 20:49:20,647][105692] Updated weights for policy 0, policy_version 749845 (0.0012) [2023-12-26 20:49:20,715][105692] Updated weights for policy 0, policy_version 749855 (0.0010) [2023-12-26 20:49:20,875][105620] Updated weights for policy 1, policy_version 750059 (0.0010) [2023-12-26 20:49:20,934][105620] Updated weights for policy 1, policy_version 750069 (0.0010) [2023-12-26 20:49:20,997][105620] Updated weights for policy 1, policy_version 750079 (0.0010) [2023-12-26 20:49:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 384040960. Throughput: 0: 10005.0, 1: 9411.6. Samples: 384025956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-12-26 20:49:21,063][104569] Avg episode reward: [(0, '9349.316'), (1, '8469.409')] [2023-12-26 20:49:21,479][105692] Updated weights for policy 0, policy_version 749865 (0.0011) [2023-12-26 20:49:21,543][105692] Updated weights for policy 0, policy_version 749875 (0.0010) [2023-12-26 20:49:21,601][105692] Updated weights for policy 0, policy_version 749885 (0.0009) [2023-12-26 20:49:21,663][105692] Updated weights for policy 0, policy_version 749895 (0.0008) [2023-12-26 20:49:21,740][105620] Updated weights for policy 1, policy_version 750089 (0.0010) [2023-12-26 20:49:21,802][105620] Updated weights for policy 1, policy_version 750099 (0.0008) [2023-12-26 20:49:21,863][105620] Updated weights for policy 1, policy_version 750109 (0.0009) [2023-12-26 20:49:21,927][105620] Updated weights for policy 1, policy_version 750119 (0.0010) [2023-12-26 20:49:22,385][105692] Updated weights for policy 0, policy_version 749905 (0.0011) [2023-12-26 20:49:22,444][105692] Updated weights for policy 0, policy_version 749915 (0.0011) [2023-12-26 20:49:22,503][105692] Updated weights for policy 0, policy_version 749925 (0.0011) [2023-12-26 20:49:22,693][105620] Updated weights for policy 1, policy_version 750129 (0.0010) [2023-12-26 20:49:22,748][105620] Updated weights for policy 1, policy_version 750139 (0.0010) [2023-12-26 20:49:22,806][105620] Updated weights for policy 1, policy_version 750149 (0.0010) [2023-12-26 20:49:23,255][105692] Updated weights for policy 0, policy_version 749935 (0.0010) [2023-12-26 20:49:23,304][105692] Updated weights for policy 0, policy_version 749945 (0.0011) [2023-12-26 20:49:23,353][105692] Updated weights for policy 0, policy_version 749955 (0.0010) [2023-12-26 20:49:23,495][105620] Updated weights for policy 1, policy_version 750159 (0.0007) [2023-12-26 20:49:23,550][105620] Updated weights for policy 1, policy_version 750169 (0.0006) [2023-12-26 20:49:23,602][105620] Updated weights for policy 1, policy_version 750179 (0.0008) [2023-12-26 20:49:24,115][105692] Updated weights for policy 0, policy_version 749965 (0.0009) [2023-12-26 20:49:24,165][105620] Updated weights for policy 1, policy_version 750189 (0.0008) [2023-12-26 20:49:24,176][105692] Updated weights for policy 0, policy_version 749975 (0.0007) [2023-12-26 20:49:24,226][105620] Updated weights for policy 1, policy_version 750199 (0.0006) [2023-12-26 20:49:24,232][105692] Updated weights for policy 0, policy_version 749985 (0.0010) [2023-12-26 20:49:24,289][105620] Updated weights for policy 1, policy_version 750209 (0.0005) [2023-12-26 20:49:24,841][105692] Updated weights for policy 0, policy_version 749995 (0.0009) [2023-12-26 20:49:24,877][105620] Updated weights for policy 1, policy_version 750219 (0.0007) [2023-12-26 20:49:24,896][105692] Updated weights for policy 0, policy_version 750005 (0.0005) [2023-12-26 20:49:24,930][105620] Updated weights for policy 1, policy_version 750229 (0.0008) [2023-12-26 20:49:24,950][105692] Updated weights for policy 0, policy_version 750015 (0.0006) [2023-12-26 20:49:24,984][105620] Updated weights for policy 1, policy_version 750239 (0.0009) [2023-12-26 20:49:25,585][105692] Updated weights for policy 0, policy_version 750025 (0.0006) [2023-12-26 20:49:25,619][105620] Updated weights for policy 1, policy_version 750249 (0.0007) [2023-12-26 20:49:25,657][105692] Updated weights for policy 0, policy_version 750035 (0.0010) [2023-12-26 20:49:25,675][105620] Updated weights for policy 1, policy_version 750259 (0.0010) [2023-12-26 20:49:25,716][105692] Updated weights for policy 0, policy_version 750045 (0.0010) [2023-12-26 20:49:25,725][105620] Updated weights for policy 1, policy_version 750269 (0.0005) [2023-12-26 20:49:25,778][105692] Updated weights for policy 0, policy_version 750055 (0.0009) [2023-12-26 20:49:25,784][105620] Updated weights for policy 1, policy_version 750279 (0.0008) [2023-12-26 20:49:26,062][104569] Fps is (10 sec: 19661.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 384139264. Throughput: 0: 9905.4, 1: 9479.7. Samples: 384146176. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 20:49:26,063][104569] Avg episode reward: [(0, '9260.552'), (1, '8203.206')] [2023-12-26 20:49:26,403][105692] Updated weights for policy 0, policy_version 750065 (0.0006) [2023-12-26 20:49:26,452][105692] Updated weights for policy 0, policy_version 750075 (0.0008) [2023-12-26 20:49:26,509][105692] Updated weights for policy 0, policy_version 750085 (0.0007) [2023-12-26 20:49:26,527][105620] Updated weights for policy 1, policy_version 750289 (0.0009) [2023-12-26 20:49:26,585][105620] Updated weights for policy 1, policy_version 750299 (0.0009) [2023-12-26 20:49:26,640][105620] Updated weights for policy 1, policy_version 750309 (0.0009) [2023-12-26 20:49:27,174][105692] Updated weights for policy 0, policy_version 750095 (0.0007) [2023-12-26 20:49:27,224][105692] Updated weights for policy 0, policy_version 750105 (0.0008) [2023-12-26 20:49:27,251][105620] Updated weights for policy 1, policy_version 750319 (0.0005) [2023-12-26 20:49:27,276][105692] Updated weights for policy 0, policy_version 750115 (0.0008) [2023-12-26 20:49:27,311][105620] Updated weights for policy 1, policy_version 750329 (0.0006) [2023-12-26 20:49:27,369][105620] Updated weights for policy 1, policy_version 750339 (0.0010) [2023-12-26 20:49:27,844][105692] Updated weights for policy 0, policy_version 750125 (0.0009) [2023-12-26 20:49:27,894][105692] Updated weights for policy 0, policy_version 750135 (0.0006) [2023-12-26 20:49:27,946][105692] Updated weights for policy 0, policy_version 750145 (0.0005) [2023-12-26 20:49:28,054][105620] Updated weights for policy 1, policy_version 750349 (0.0009) [2023-12-26 20:49:28,098][105620] Updated weights for policy 1, policy_version 750359 (0.0005) [2023-12-26 20:49:28,143][105620] Updated weights for policy 1, policy_version 750369 (0.0005) [2023-12-26 20:49:28,640][105692] Updated weights for policy 0, policy_version 750155 (0.0007) [2023-12-26 20:49:28,702][105692] Updated weights for policy 0, policy_version 750165 (0.0011) [2023-12-26 20:49:28,734][105620] Updated weights for policy 1, policy_version 750379 (0.0007) [2023-12-26 20:49:28,765][105692] Updated weights for policy 0, policy_version 750175 (0.0010) [2023-12-26 20:49:28,793][105620] Updated weights for policy 1, policy_version 750389 (0.0011) [2023-12-26 20:49:28,852][105620] Updated weights for policy 1, policy_version 750399 (0.0010) [2023-12-26 20:49:29,491][105692] Updated weights for policy 0, policy_version 750185 (0.0011) [2023-12-26 20:49:29,553][105692] Updated weights for policy 0, policy_version 750195 (0.0010) [2023-12-26 20:49:29,566][105620] Updated weights for policy 1, policy_version 750409 (0.0011) [2023-12-26 20:49:29,611][105692] Updated weights for policy 0, policy_version 750205 (0.0011) [2023-12-26 20:49:29,625][105620] Updated weights for policy 1, policy_version 750419 (0.0010) [2023-12-26 20:49:29,671][105692] Updated weights for policy 0, policy_version 750215 (0.0011) [2023-12-26 20:49:29,674][105620] Updated weights for policy 1, policy_version 750429 (0.0010) [2023-12-26 20:49:29,726][105620] Updated weights for policy 1, policy_version 750439 (0.0010) [2023-12-26 20:49:30,410][105620] Updated weights for policy 1, policy_version 750449 (0.0007) [2023-12-26 20:49:30,416][105692] Updated weights for policy 0, policy_version 750225 (0.0011) [2023-12-26 20:49:30,464][105620] Updated weights for policy 1, policy_version 750459 (0.0010) [2023-12-26 20:49:30,471][105692] Updated weights for policy 0, policy_version 750235 (0.0010) [2023-12-26 20:49:30,527][105620] Updated weights for policy 1, policy_version 750469 (0.0011) [2023-12-26 20:49:30,530][105692] Updated weights for policy 0, policy_version 750245 (0.0011) [2023-12-26 20:49:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 384237568. Throughput: 0: 9979.1, 1: 9560.2. Samples: 384210192. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 20:49:31,062][104569] Avg episode reward: [(0, '9261.519'), (1, '8627.020')] [2023-12-26 20:49:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000750248_192094208.pth... [2023-12-26 20:49:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000750472_192143360.pth... [2023-12-26 20:49:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000749320_191848448.pth [2023-12-26 20:49:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000749096_191799296.pth [2023-12-26 20:49:31,216][105620] Updated weights for policy 1, policy_version 750479 (0.0011) [2023-12-26 20:49:31,272][105692] Updated weights for policy 0, policy_version 750255 (0.0010) [2023-12-26 20:49:31,275][105620] Updated weights for policy 1, policy_version 750489 (0.0007) [2023-12-26 20:49:31,321][105620] Updated weights for policy 1, policy_version 750499 (0.0008) [2023-12-26 20:49:31,324][105692] Updated weights for policy 0, policy_version 750265 (0.0010) [2023-12-26 20:49:31,392][105692] Updated weights for policy 0, policy_version 750275 (0.0009) [2023-12-26 20:49:31,970][105620] Updated weights for policy 1, policy_version 750509 (0.0008) [2023-12-26 20:49:32,028][105620] Updated weights for policy 1, policy_version 750519 (0.0009) [2023-12-26 20:49:32,094][105620] Updated weights for policy 1, policy_version 750529 (0.0006) [2023-12-26 20:49:32,180][105692] Updated weights for policy 0, policy_version 750285 (0.0008) [2023-12-26 20:49:32,234][105692] Updated weights for policy 0, policy_version 750295 (0.0010) [2023-12-26 20:49:32,294][105692] Updated weights for policy 0, policy_version 750305 (0.0010) [2023-12-26 20:49:32,707][105620] Updated weights for policy 1, policy_version 750539 (0.0005) [2023-12-26 20:49:32,765][105620] Updated weights for policy 1, policy_version 750549 (0.0006) [2023-12-26 20:49:32,830][105620] Updated weights for policy 1, policy_version 750559 (0.0005) [2023-12-26 20:49:33,086][105692] Updated weights for policy 0, policy_version 750315 (0.0005) [2023-12-26 20:49:33,132][105692] Updated weights for policy 0, policy_version 750325 (0.0006) [2023-12-26 20:49:33,174][105692] Updated weights for policy 0, policy_version 750335 (0.0006) [2023-12-26 20:49:33,530][105620] Updated weights for policy 1, policy_version 750569 (0.0006) [2023-12-26 20:49:33,586][105620] Updated weights for policy 1, policy_version 750579 (0.0009) [2023-12-26 20:49:33,639][105620] Updated weights for policy 1, policy_version 750590 (0.0010) [2023-12-26 20:49:33,693][105620] Updated weights for policy 1, policy_version 750600 (0.0009) [2023-12-26 20:49:33,795][105692] Updated weights for policy 0, policy_version 750345 (0.0006) [2023-12-26 20:49:33,851][105692] Updated weights for policy 0, policy_version 750355 (0.0006) [2023-12-26 20:49:33,907][105692] Updated weights for policy 0, policy_version 750365 (0.0005) [2023-12-26 20:49:33,962][105692] Updated weights for policy 0, policy_version 750375 (0.0005) [2023-12-26 20:49:34,507][105692] Updated weights for policy 0, policy_version 750385 (0.0006) [2023-12-26 20:49:34,566][105692] Updated weights for policy 0, policy_version 750395 (0.0005) [2023-12-26 20:49:34,610][105620] Updated weights for policy 1, policy_version 750610 (0.0008) [2023-12-26 20:49:34,624][105692] Updated weights for policy 0, policy_version 750405 (0.0005) [2023-12-26 20:49:34,674][105620] Updated weights for policy 1, policy_version 750620 (0.0011) [2023-12-26 20:49:34,745][105620] Updated weights for policy 1, policy_version 750630 (0.0007) [2023-12-26 20:49:35,275][105692] Updated weights for policy 0, policy_version 750415 (0.0007) [2023-12-26 20:49:35,324][105692] Updated weights for policy 0, policy_version 750425 (0.0008) [2023-12-26 20:49:35,377][105692] Updated weights for policy 0, policy_version 750435 (0.0008) [2023-12-26 20:49:35,428][105620] Updated weights for policy 1, policy_version 750640 (0.0009) [2023-12-26 20:49:35,489][105620] Updated weights for policy 1, policy_version 750650 (0.0010) [2023-12-26 20:49:35,546][105620] Updated weights for policy 1, policy_version 750660 (0.0010) [2023-12-26 20:49:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 384335872. Throughput: 0: 9869.9, 1: 9710.1. Samples: 384328272. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 20:49:36,063][104569] Avg episode reward: [(0, '9259.780'), (1, '8902.973')] [2023-12-26 20:49:36,163][105692] Updated weights for policy 0, policy_version 750445 (0.0007) [2023-12-26 20:49:36,223][105692] Updated weights for policy 0, policy_version 750455 (0.0008) [2023-12-26 20:49:36,271][105692] Updated weights for policy 0, policy_version 750465 (0.0008) [2023-12-26 20:49:36,282][105620] Updated weights for policy 1, policy_version 750670 (0.0010) [2023-12-26 20:49:36,330][105620] Updated weights for policy 1, policy_version 750680 (0.0010) [2023-12-26 20:49:36,400][105620] Updated weights for policy 1, policy_version 750690 (0.0011) [2023-12-26 20:49:37,029][105692] Updated weights for policy 0, policy_version 750475 (0.0007) [2023-12-26 20:49:37,095][105692] Updated weights for policy 0, policy_version 750485 (0.0011) [2023-12-26 20:49:37,117][105620] Updated weights for policy 1, policy_version 750700 (0.0009) [2023-12-26 20:49:37,155][105692] Updated weights for policy 0, policy_version 750495 (0.0011) [2023-12-26 20:49:37,167][105620] Updated weights for policy 1, policy_version 750710 (0.0006) [2023-12-26 20:49:37,223][105620] Updated weights for policy 1, policy_version 750720 (0.0007) [2023-12-26 20:49:37,896][105692] Updated weights for policy 0, policy_version 750505 (0.0010) [2023-12-26 20:49:37,957][105692] Updated weights for policy 0, policy_version 750515 (0.0006) [2023-12-26 20:49:38,002][105620] Updated weights for policy 1, policy_version 750730 (0.0008) [2023-12-26 20:49:38,020][105692] Updated weights for policy 0, policy_version 750525 (0.0006) [2023-12-26 20:49:38,064][105620] Updated weights for policy 1, policy_version 750740 (0.0009) [2023-12-26 20:49:38,077][105692] Updated weights for policy 0, policy_version 750535 (0.0008) [2023-12-26 20:49:38,124][105620] Updated weights for policy 1, policy_version 750750 (0.0009) [2023-12-26 20:49:38,179][105620] Updated weights for policy 1, policy_version 750760 (0.0011) [2023-12-26 20:49:38,719][105692] Updated weights for policy 0, policy_version 750545 (0.0010) [2023-12-26 20:49:38,782][105692] Updated weights for policy 0, policy_version 750555 (0.0011) [2023-12-26 20:49:38,850][105692] Updated weights for policy 0, policy_version 750565 (0.0011) [2023-12-26 20:49:38,957][105620] Updated weights for policy 1, policy_version 750770 (0.0008) [2023-12-26 20:49:39,015][105620] Updated weights for policy 1, policy_version 750780 (0.0010) [2023-12-26 20:49:39,072][105620] Updated weights for policy 1, policy_version 750790 (0.0009) [2023-12-26 20:49:39,570][105692] Updated weights for policy 0, policy_version 750575 (0.0007) [2023-12-26 20:49:39,629][105692] Updated weights for policy 0, policy_version 750585 (0.0008) [2023-12-26 20:49:39,696][105692] Updated weights for policy 0, policy_version 750595 (0.0011) [2023-12-26 20:49:39,882][105620] Updated weights for policy 1, policy_version 750800 (0.0008) [2023-12-26 20:49:39,945][105620] Updated weights for policy 1, policy_version 750810 (0.0007) [2023-12-26 20:49:40,007][105620] Updated weights for policy 1, policy_version 750820 (0.0007) [2023-12-26 20:49:40,448][105692] Updated weights for policy 0, policy_version 750605 (0.0010) [2023-12-26 20:49:40,495][105692] Updated weights for policy 0, policy_version 750615 (0.0009) [2023-12-26 20:49:40,553][105692] Updated weights for policy 0, policy_version 750625 (0.0009) [2023-12-26 20:49:40,665][105620] Updated weights for policy 1, policy_version 750830 (0.0009) [2023-12-26 20:49:40,731][105620] Updated weights for policy 1, policy_version 750840 (0.0009) [2023-12-26 20:49:40,793][105620] Updated weights for policy 1, policy_version 750850 (0.0009) [2023-12-26 20:49:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 384434176. Throughput: 0: 9805.8, 1: 9746.9. Samples: 384441408. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 20:49:41,062][104569] Avg episode reward: [(0, '9259.493'), (1, '8910.207')] [2023-12-26 20:49:41,208][105692] Updated weights for policy 0, policy_version 750635 (0.0010) [2023-12-26 20:49:41,277][105692] Updated weights for policy 0, policy_version 750645 (0.0009) [2023-12-26 20:49:41,339][105692] Updated weights for policy 0, policy_version 750655 (0.0009) [2023-12-26 20:49:41,659][105620] Updated weights for policy 1, policy_version 750860 (0.0009) [2023-12-26 20:49:41,723][105620] Updated weights for policy 1, policy_version 750870 (0.0008) [2023-12-26 20:49:41,790][105620] Updated weights for policy 1, policy_version 750880 (0.0008) [2023-12-26 20:49:42,138][105692] Updated weights for policy 0, policy_version 750665 (0.0010) [2023-12-26 20:49:42,200][105692] Updated weights for policy 0, policy_version 750675 (0.0009) [2023-12-26 20:49:42,262][105692] Updated weights for policy 0, policy_version 750685 (0.0009) [2023-12-26 20:49:42,323][105692] Updated weights for policy 0, policy_version 750695 (0.0008) [2023-12-26 20:49:42,469][105620] Updated weights for policy 1, policy_version 750890 (0.0006) [2023-12-26 20:49:42,529][105620] Updated weights for policy 1, policy_version 750900 (0.0009) [2023-12-26 20:49:42,583][105620] Updated weights for policy 1, policy_version 750910 (0.0009) [2023-12-26 20:49:42,641][105620] Updated weights for policy 1, policy_version 750920 (0.0009) [2023-12-26 20:49:43,008][105692] Updated weights for policy 0, policy_version 750705 (0.0007) [2023-12-26 20:49:43,055][105692] Updated weights for policy 0, policy_version 750715 (0.0005) [2023-12-26 20:49:43,116][105692] Updated weights for policy 0, policy_version 750725 (0.0006) [2023-12-26 20:49:43,469][105620] Updated weights for policy 1, policy_version 750930 (0.0010) [2023-12-26 20:49:43,524][105620] Updated weights for policy 1, policy_version 750940 (0.0010) [2023-12-26 20:49:43,581][105620] Updated weights for policy 1, policy_version 750950 (0.0008) [2023-12-26 20:49:43,853][105692] Updated weights for policy 0, policy_version 750735 (0.0009) [2023-12-26 20:49:43,915][105692] Updated weights for policy 0, policy_version 750745 (0.0009) [2023-12-26 20:49:43,969][105692] Updated weights for policy 0, policy_version 750755 (0.0007) [2023-12-26 20:49:44,398][105620] Updated weights for policy 1, policy_version 750960 (0.0009) [2023-12-26 20:49:44,459][105620] Updated weights for policy 1, policy_version 750970 (0.0009) [2023-12-26 20:49:44,513][105620] Updated weights for policy 1, policy_version 750980 (0.0010) [2023-12-26 20:49:44,600][105692] Updated weights for policy 0, policy_version 750765 (0.0009) [2023-12-26 20:49:44,648][105692] Updated weights for policy 0, policy_version 750775 (0.0009) [2023-12-26 20:49:44,692][105692] Updated weights for policy 0, policy_version 750785 (0.0007) [2023-12-26 20:49:45,331][105620] Updated weights for policy 1, policy_version 750991 (0.0010) [2023-12-26 20:49:45,378][105692] Updated weights for policy 0, policy_version 750795 (0.0005) [2023-12-26 20:49:45,388][105620] Updated weights for policy 1, policy_version 751001 (0.0009) [2023-12-26 20:49:45,431][105692] Updated weights for policy 0, policy_version 750805 (0.0006) [2023-12-26 20:49:45,449][105620] Updated weights for policy 1, policy_version 751011 (0.0008) [2023-12-26 20:49:45,477][105692] Updated weights for policy 0, policy_version 750815 (0.0006) [2023-12-26 20:49:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 384524288. Throughput: 0: 9772.9, 1: 9733.4. Samples: 384497900. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 20:49:46,063][104569] Avg episode reward: [(0, '9259.396'), (1, '8903.850')] [2023-12-26 20:49:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000751016_192282624.pth... [2023-12-26 20:49:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000750824_192241664.pth... [2023-12-26 20:49:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000749896_191995904.pth [2023-12-26 20:49:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000749672_191946752.pth [2023-12-26 20:49:46,188][105692] Updated weights for policy 0, policy_version 750825 (0.0009) [2023-12-26 20:49:46,241][105620] Updated weights for policy 1, policy_version 751021 (0.0008) [2023-12-26 20:49:46,246][105692] Updated weights for policy 0, policy_version 750835 (0.0009) [2023-12-26 20:49:46,291][105620] Updated weights for policy 1, policy_version 751031 (0.0008) [2023-12-26 20:49:46,304][105692] Updated weights for policy 0, policy_version 750845 (0.0008) [2023-12-26 20:49:46,343][105620] Updated weights for policy 1, policy_version 751041 (0.0007) [2023-12-26 20:49:46,357][105692] Updated weights for policy 0, policy_version 750855 (0.0007) [2023-12-26 20:49:46,986][105620] Updated weights for policy 1, policy_version 751051 (0.0007) [2023-12-26 20:49:47,044][105620] Updated weights for policy 1, policy_version 751061 (0.0005) [2023-12-26 20:49:47,098][105620] Updated weights for policy 1, policy_version 751071 (0.0005) [2023-12-26 20:49:47,188][105692] Updated weights for policy 0, policy_version 750865 (0.0008) [2023-12-26 20:49:47,236][105692] Updated weights for policy 0, policy_version 750875 (0.0008) [2023-12-26 20:49:47,295][105692] Updated weights for policy 0, policy_version 750885 (0.0009) [2023-12-26 20:49:47,672][105620] Updated weights for policy 1, policy_version 751081 (0.0006) [2023-12-26 20:49:47,724][105620] Updated weights for policy 1, policy_version 751091 (0.0008) [2023-12-26 20:49:47,774][105620] Updated weights for policy 1, policy_version 751101 (0.0009) [2023-12-26 20:49:47,824][105620] Updated weights for policy 1, policy_version 751111 (0.0008) [2023-12-26 20:49:48,145][105692] Updated weights for policy 0, policy_version 750895 (0.0009) [2023-12-26 20:49:48,204][105692] Updated weights for policy 0, policy_version 750905 (0.0010) [2023-12-26 20:49:48,271][105692] Updated weights for policy 0, policy_version 750915 (0.0010) [2023-12-26 20:49:48,490][105620] Updated weights for policy 1, policy_version 751121 (0.0008) [2023-12-26 20:49:48,547][105620] Updated weights for policy 1, policy_version 751131 (0.0006) [2023-12-26 20:49:48,608][105620] Updated weights for policy 1, policy_version 751141 (0.0008) [2023-12-26 20:49:49,065][105692] Updated weights for policy 0, policy_version 750925 (0.0010) [2023-12-26 20:49:49,117][105692] Updated weights for policy 0, policy_version 750935 (0.0009) [2023-12-26 20:49:49,171][105692] Updated weights for policy 0, policy_version 750945 (0.0009) [2023-12-26 20:49:49,328][105620] Updated weights for policy 1, policy_version 751151 (0.0010) [2023-12-26 20:49:49,394][105620] Updated weights for policy 1, policy_version 751161 (0.0008) [2023-12-26 20:49:49,453][105620] Updated weights for policy 1, policy_version 751171 (0.0009) [2023-12-26 20:49:49,897][105692] Updated weights for policy 0, policy_version 750955 (0.0008) [2023-12-26 20:49:49,954][105692] Updated weights for policy 0, policy_version 750965 (0.0008) [2023-12-26 20:49:50,015][105692] Updated weights for policy 0, policy_version 750975 (0.0006) [2023-12-26 20:49:50,256][105620] Updated weights for policy 1, policy_version 751181 (0.0009) [2023-12-26 20:49:50,303][105620] Updated weights for policy 1, policy_version 751191 (0.0008) [2023-12-26 20:49:50,362][105620] Updated weights for policy 1, policy_version 751201 (0.0008) [2023-12-26 20:49:50,755][105692] Updated weights for policy 0, policy_version 750985 (0.0008) [2023-12-26 20:49:50,806][105692] Updated weights for policy 0, policy_version 750995 (0.0009) [2023-12-26 20:49:50,858][105692] Updated weights for policy 0, policy_version 751005 (0.0009) [2023-12-26 20:49:50,913][105692] Updated weights for policy 0, policy_version 751015 (0.0009) [2023-12-26 20:49:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 384622592. Throughput: 0: 9752.2, 1: 9697.8. Samples: 384612520. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 20:49:51,063][104569] Avg episode reward: [(0, '9258.616'), (1, '8896.483')] [2023-12-26 20:49:51,090][105620] Updated weights for policy 1, policy_version 751211 (0.0007) [2023-12-26 20:49:51,154][105620] Updated weights for policy 1, policy_version 751221 (0.0009) [2023-12-26 20:49:51,202][105620] Updated weights for policy 1, policy_version 751231 (0.0009) [2023-12-26 20:49:51,638][105692] Updated weights for policy 0, policy_version 751025 (0.0007) [2023-12-26 20:49:51,691][105692] Updated weights for policy 0, policy_version 751035 (0.0008) [2023-12-26 20:49:51,758][105692] Updated weights for policy 0, policy_version 751045 (0.0008) [2023-12-26 20:49:51,998][105620] Updated weights for policy 1, policy_version 751241 (0.0010) [2023-12-26 20:49:52,072][105620] Updated weights for policy 1, policy_version 751251 (0.0008) [2023-12-26 20:49:52,136][105620] Updated weights for policy 1, policy_version 751261 (0.0008) [2023-12-26 20:49:52,195][105620] Updated weights for policy 1, policy_version 751271 (0.0006) [2023-12-26 20:49:52,562][105692] Updated weights for policy 0, policy_version 751055 (0.0009) [2023-12-26 20:49:52,621][105692] Updated weights for policy 0, policy_version 751065 (0.0009) [2023-12-26 20:49:52,678][105692] Updated weights for policy 0, policy_version 751075 (0.0009) [2023-12-26 20:49:52,894][105620] Updated weights for policy 1, policy_version 751281 (0.0008) [2023-12-26 20:49:52,955][105620] Updated weights for policy 1, policy_version 751291 (0.0009) [2023-12-26 20:49:53,003][105620] Updated weights for policy 1, policy_version 751301 (0.0008) [2023-12-26 20:49:53,462][105692] Updated weights for policy 0, policy_version 751085 (0.0009) [2023-12-26 20:49:53,521][105692] Updated weights for policy 0, policy_version 751095 (0.0008) [2023-12-26 20:49:53,582][105692] Updated weights for policy 0, policy_version 751105 (0.0005) [2023-12-26 20:49:53,752][105620] Updated weights for policy 1, policy_version 751311 (0.0008) [2023-12-26 20:49:53,807][105620] Updated weights for policy 1, policy_version 751321 (0.0011) [2023-12-26 20:49:53,860][105620] Updated weights for policy 1, policy_version 751331 (0.0011) [2023-12-26 20:49:54,194][105692] Updated weights for policy 0, policy_version 751115 (0.0007) [2023-12-26 20:49:54,250][105692] Updated weights for policy 0, policy_version 751125 (0.0009) [2023-12-26 20:49:54,308][105692] Updated weights for policy 0, policy_version 751135 (0.0009) [2023-12-26 20:49:54,631][105620] Updated weights for policy 1, policy_version 751341 (0.0010) [2023-12-26 20:49:54,694][105620] Updated weights for policy 1, policy_version 751351 (0.0008) [2023-12-26 20:49:54,758][105620] Updated weights for policy 1, policy_version 751361 (0.0010) [2023-12-26 20:49:55,046][105692] Updated weights for policy 0, policy_version 751145 (0.0008) [2023-12-26 20:49:55,102][105692] Updated weights for policy 0, policy_version 751155 (0.0009) [2023-12-26 20:49:55,164][105692] Updated weights for policy 0, policy_version 751165 (0.0009) [2023-12-26 20:49:55,225][105692] Updated weights for policy 0, policy_version 751175 (0.0009) [2023-12-26 20:49:55,506][105620] Updated weights for policy 1, policy_version 751371 (0.0009) [2023-12-26 20:49:55,559][105620] Updated weights for policy 1, policy_version 751381 (0.0009) [2023-12-26 20:49:55,620][105620] Updated weights for policy 1, policy_version 751391 (0.0006) [2023-12-26 20:49:55,932][105692] Updated weights for policy 0, policy_version 751185 (0.0006) [2023-12-26 20:49:55,997][105692] Updated weights for policy 0, policy_version 751195 (0.0005) [2023-12-26 20:49:56,059][105692] Updated weights for policy 0, policy_version 751205 (0.0005) [2023-12-26 20:49:56,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 384712704. Throughput: 0: 9754.6, 1: 9702.4. Samples: 384726568. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 20:49:56,063][104569] Avg episode reward: [(0, '9349.372'), (1, '8825.036')] [2023-12-26 20:49:56,218][105620] Updated weights for policy 1, policy_version 751401 (0.0005) [2023-12-26 20:49:56,269][105620] Updated weights for policy 1, policy_version 751411 (0.0009) [2023-12-26 20:49:56,318][105620] Updated weights for policy 1, policy_version 751421 (0.0010) [2023-12-26 20:49:56,373][105620] Updated weights for policy 1, policy_version 751431 (0.0010) [2023-12-26 20:49:56,591][105692] Updated weights for policy 0, policy_version 751215 (0.0007) [2023-12-26 20:49:56,650][105692] Updated weights for policy 0, policy_version 751225 (0.0008) [2023-12-26 20:49:56,717][105692] Updated weights for policy 0, policy_version 751235 (0.0008) [2023-12-26 20:49:57,109][105620] Updated weights for policy 1, policy_version 751441 (0.0010) [2023-12-26 20:49:57,160][105620] Updated weights for policy 1, policy_version 751451 (0.0010) [2023-12-26 20:49:57,217][105620] Updated weights for policy 1, policy_version 751461 (0.0010) [2023-12-26 20:49:57,425][105692] Updated weights for policy 0, policy_version 751245 (0.0007) [2023-12-26 20:49:57,485][105692] Updated weights for policy 0, policy_version 751255 (0.0005) [2023-12-26 20:49:57,549][105692] Updated weights for policy 0, policy_version 751265 (0.0005) [2023-12-26 20:49:57,926][105620] Updated weights for policy 1, policy_version 751471 (0.0010) [2023-12-26 20:49:57,973][105620] Updated weights for policy 1, policy_version 751481 (0.0010) [2023-12-26 20:49:58,025][105620] Updated weights for policy 1, policy_version 751491 (0.0010) [2023-12-26 20:49:58,212][105692] Updated weights for policy 0, policy_version 751275 (0.0006) [2023-12-26 20:49:58,274][105692] Updated weights for policy 0, policy_version 751285 (0.0009) [2023-12-26 20:49:58,338][105692] Updated weights for policy 0, policy_version 751295 (0.0008) [2023-12-26 20:49:58,813][105620] Updated weights for policy 1, policy_version 751501 (0.0010) [2023-12-26 20:49:58,869][105620] Updated weights for policy 1, policy_version 751511 (0.0010) [2023-12-26 20:49:58,918][105620] Updated weights for policy 1, policy_version 751521 (0.0010) [2023-12-26 20:49:59,091][105692] Updated weights for policy 0, policy_version 751305 (0.0008) [2023-12-26 20:49:59,152][105692] Updated weights for policy 0, policy_version 751315 (0.0006) [2023-12-26 20:49:59,218][105692] Updated weights for policy 0, policy_version 751325 (0.0008) [2023-12-26 20:49:59,280][105692] Updated weights for policy 0, policy_version 751335 (0.0007) [2023-12-26 20:49:59,654][105620] Updated weights for policy 1, policy_version 751531 (0.0011) [2023-12-26 20:49:59,721][105620] Updated weights for policy 1, policy_version 751541 (0.0005) [2023-12-26 20:49:59,779][105620] Updated weights for policy 1, policy_version 751551 (0.0007) [2023-12-26 20:49:59,861][105692] Updated weights for policy 0, policy_version 751345 (0.0008) [2023-12-26 20:49:59,912][105692] Updated weights for policy 0, policy_version 751355 (0.0008) [2023-12-26 20:49:59,973][105692] Updated weights for policy 0, policy_version 751365 (0.0008) [2023-12-26 20:50:00,361][105620] Updated weights for policy 1, policy_version 751561 (0.0010) [2023-12-26 20:50:00,432][105620] Updated weights for policy 1, policy_version 751571 (0.0006) [2023-12-26 20:50:00,494][105620] Updated weights for policy 1, policy_version 751581 (0.0006) [2023-12-26 20:50:00,541][105620] Updated weights for policy 1, policy_version 751591 (0.0005) [2023-12-26 20:50:00,694][105692] Updated weights for policy 0, policy_version 751375 (0.0006) [2023-12-26 20:50:00,744][105692] Updated weights for policy 0, policy_version 751385 (0.0006) [2023-12-26 20:50:00,796][105692] Updated weights for policy 0, policy_version 751395 (0.0008) [2023-12-26 20:50:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 384819200. Throughput: 0: 9839.8, 1: 9741.1. Samples: 384787136. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 20:50:01,063][104569] Avg episode reward: [(0, '9166.108'), (1, '8732.334')] [2023-12-26 20:50:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000751400_192389120.pth... [2023-12-26 20:50:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000751592_192430080.pth... [2023-12-26 20:50:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000750472_192143360.pth [2023-12-26 20:50:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000750248_192094208.pth [2023-12-26 20:50:01,217][105620] Updated weights for policy 1, policy_version 751601 (0.0007) [2023-12-26 20:50:01,277][105620] Updated weights for policy 1, policy_version 751611 (0.0008) [2023-12-26 20:50:01,338][105620] Updated weights for policy 1, policy_version 751621 (0.0009) [2023-12-26 20:50:01,471][105692] Updated weights for policy 0, policy_version 751405 (0.0009) [2023-12-26 20:50:01,531][105692] Updated weights for policy 0, policy_version 751415 (0.0009) [2023-12-26 20:50:01,590][105692] Updated weights for policy 0, policy_version 751425 (0.0009) [2023-12-26 20:50:02,052][105620] Updated weights for policy 1, policy_version 751631 (0.0010) [2023-12-26 20:50:02,107][105620] Updated weights for policy 1, policy_version 751643 (0.0011) [2023-12-26 20:50:02,166][105620] Updated weights for policy 1, policy_version 751653 (0.0009) [2023-12-26 20:50:02,231][105692] Updated weights for policy 0, policy_version 751435 (0.0009) [2023-12-26 20:50:02,288][105692] Updated weights for policy 0, policy_version 751445 (0.0006) [2023-12-26 20:50:02,345][105692] Updated weights for policy 0, policy_version 751455 (0.0006) [2023-12-26 20:50:02,915][105692] Updated weights for policy 0, policy_version 751465 (0.0006) [2023-12-26 20:50:02,948][105620] Updated weights for policy 1, policy_version 751664 (0.0009) [2023-12-26 20:50:02,965][105692] Updated weights for policy 0, policy_version 751475 (0.0005) [2023-12-26 20:50:03,004][105620] Updated weights for policy 1, policy_version 751674 (0.0008) [2023-12-26 20:50:03,017][105692] Updated weights for policy 0, policy_version 751485 (0.0010) [2023-12-26 20:50:03,051][105620] Updated weights for policy 1, policy_version 751684 (0.0006) [2023-12-26 20:50:03,062][105692] Updated weights for policy 0, policy_version 751495 (0.0010) [2023-12-26 20:50:03,757][105620] Updated weights for policy 1, policy_version 751694 (0.0005) [2023-12-26 20:50:03,766][105692] Updated weights for policy 0, policy_version 751505 (0.0011) [2023-12-26 20:50:03,819][105620] Updated weights for policy 1, policy_version 751704 (0.0007) [2023-12-26 20:50:03,825][105692] Updated weights for policy 0, policy_version 751515 (0.0010) [2023-12-26 20:50:03,883][105620] Updated weights for policy 1, policy_version 751714 (0.0010) [2023-12-26 20:50:03,889][105692] Updated weights for policy 0, policy_version 751525 (0.0010) [2023-12-26 20:50:04,615][105692] Updated weights for policy 0, policy_version 751535 (0.0010) [2023-12-26 20:50:04,636][105620] Updated weights for policy 1, policy_version 751724 (0.0011) [2023-12-26 20:50:04,677][105692] Updated weights for policy 0, policy_version 751545 (0.0011) [2023-12-26 20:50:04,688][105620] Updated weights for policy 1, policy_version 751734 (0.0010) [2023-12-26 20:50:04,737][105692] Updated weights for policy 0, policy_version 751555 (0.0011) [2023-12-26 20:50:04,744][105620] Updated weights for policy 1, policy_version 751744 (0.0011) [2023-12-26 20:50:05,285][105692] Updated weights for policy 0, policy_version 751565 (0.0008) [2023-12-26 20:50:05,336][105692] Updated weights for policy 0, policy_version 751575 (0.0005) [2023-12-26 20:50:05,396][105692] Updated weights for policy 0, policy_version 751585 (0.0005) [2023-12-26 20:50:05,490][105620] Updated weights for policy 1, policy_version 751754 (0.0011) [2023-12-26 20:50:05,545][105620] Updated weights for policy 1, policy_version 751764 (0.0010) [2023-12-26 20:50:05,593][105620] Updated weights for policy 1, policy_version 751774 (0.0010) [2023-12-26 20:50:05,644][105620] Updated weights for policy 1, policy_version 751784 (0.0010) [2023-12-26 20:50:05,934][105692] Updated weights for policy 0, policy_version 751595 (0.0005) [2023-12-26 20:50:06,003][105692] Updated weights for policy 0, policy_version 751605 (0.0006) [2023-12-26 20:50:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 384917504. Throughput: 0: 9863.0, 1: 9739.7. Samples: 384908076. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 20:50:06,062][104569] Avg episode reward: [(0, '9165.278'), (1, '8980.392')] [2023-12-26 20:50:06,073][105692] Updated weights for policy 0, policy_version 751615 (0.0006) [2023-12-26 20:50:06,402][105620] Updated weights for policy 1, policy_version 751794 (0.0010) [2023-12-26 20:50:06,461][105620] Updated weights for policy 1, policy_version 751804 (0.0010) [2023-12-26 20:50:06,525][105620] Updated weights for policy 1, policy_version 751814 (0.0011) [2023-12-26 20:50:06,709][105692] Updated weights for policy 0, policy_version 751625 (0.0008) [2023-12-26 20:50:06,779][105692] Updated weights for policy 0, policy_version 751635 (0.0006) [2023-12-26 20:50:06,836][105692] Updated weights for policy 0, policy_version 751645 (0.0008) [2023-12-26 20:50:06,885][105692] Updated weights for policy 0, policy_version 751655 (0.0008) [2023-12-26 20:50:07,268][105620] Updated weights for policy 1, policy_version 751824 (0.0009) [2023-12-26 20:50:07,315][105620] Updated weights for policy 1, policy_version 751834 (0.0008) [2023-12-26 20:50:07,376][105620] Updated weights for policy 1, policy_version 751844 (0.0010) [2023-12-26 20:50:07,537][105692] Updated weights for policy 0, policy_version 751665 (0.0006) [2023-12-26 20:50:07,588][105692] Updated weights for policy 0, policy_version 751675 (0.0005) [2023-12-26 20:50:07,644][105692] Updated weights for policy 0, policy_version 751685 (0.0005) [2023-12-26 20:50:08,113][105620] Updated weights for policy 1, policy_version 751854 (0.0009) [2023-12-26 20:50:08,163][105620] Updated weights for policy 1, policy_version 751864 (0.0010) [2023-12-26 20:50:08,212][105620] Updated weights for policy 1, policy_version 751874 (0.0010) [2023-12-26 20:50:08,293][105692] Updated weights for policy 0, policy_version 751695 (0.0009) [2023-12-26 20:50:08,345][105692] Updated weights for policy 0, policy_version 751705 (0.0011) [2023-12-26 20:50:08,405][105692] Updated weights for policy 0, policy_version 751715 (0.0010) [2023-12-26 20:50:08,937][105620] Updated weights for policy 1, policy_version 751884 (0.0006) [2023-12-26 20:50:09,003][105620] Updated weights for policy 1, policy_version 751894 (0.0006) [2023-12-26 20:50:09,038][105692] Updated weights for policy 0, policy_version 751725 (0.0007) [2023-12-26 20:50:09,062][105620] Updated weights for policy 1, policy_version 751904 (0.0006) [2023-12-26 20:50:09,096][105692] Updated weights for policy 0, policy_version 751735 (0.0008) [2023-12-26 20:50:09,156][105692] Updated weights for policy 0, policy_version 751745 (0.0008) [2023-12-26 20:50:09,778][105620] Updated weights for policy 1, policy_version 751914 (0.0008) [2023-12-26 20:50:09,836][105620] Updated weights for policy 1, policy_version 751924 (0.0009) [2023-12-26 20:50:09,894][105620] Updated weights for policy 1, policy_version 751934 (0.0009) [2023-12-26 20:50:09,947][105692] Updated weights for policy 0, policy_version 751755 (0.0009) [2023-12-26 20:50:09,957][105620] Updated weights for policy 1, policy_version 751944 (0.0008) [2023-12-26 20:50:10,011][105692] Updated weights for policy 0, policy_version 751765 (0.0008) [2023-12-26 20:50:10,075][105692] Updated weights for policy 0, policy_version 751775 (0.0009) [2023-12-26 20:50:10,702][105620] Updated weights for policy 1, policy_version 751954 (0.0005) [2023-12-26 20:50:10,764][105620] Updated weights for policy 1, policy_version 751964 (0.0006) [2023-12-26 20:50:10,815][105620] Updated weights for policy 1, policy_version 751974 (0.0005) [2023-12-26 20:50:10,875][105692] Updated weights for policy 0, policy_version 751785 (0.0009) [2023-12-26 20:50:10,937][105692] Updated weights for policy 0, policy_version 751795 (0.0009) [2023-12-26 20:50:10,996][105692] Updated weights for policy 0, policy_version 751805 (0.0009) [2023-12-26 20:50:11,061][105692] Updated weights for policy 0, policy_version 751815 (0.0010) [2023-12-26 20:50:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 385015808. Throughput: 0: 9960.8, 1: 9665.6. Samples: 385029364. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 20:50:11,062][104569] Avg episode reward: [(0, '9074.534'), (1, '8892.862')] [2023-12-26 20:50:11,435][105620] Updated weights for policy 1, policy_version 751984 (0.0008) [2023-12-26 20:50:11,505][105620] Updated weights for policy 1, policy_version 751994 (0.0007) [2023-12-26 20:50:11,573][105620] Updated weights for policy 1, policy_version 752004 (0.0006) [2023-12-26 20:50:11,935][105692] Updated weights for policy 0, policy_version 751825 (0.0009) [2023-12-26 20:50:11,993][105692] Updated weights for policy 0, policy_version 751835 (0.0008) [2023-12-26 20:50:12,061][105692] Updated weights for policy 0, policy_version 751845 (0.0005) [2023-12-26 20:50:12,269][105620] Updated weights for policy 1, policy_version 752014 (0.0009) [2023-12-26 20:50:12,334][105620] Updated weights for policy 1, policy_version 752024 (0.0008) [2023-12-26 20:50:12,405][105620] Updated weights for policy 1, policy_version 752034 (0.0007) [2023-12-26 20:50:12,780][105692] Updated weights for policy 0, policy_version 751855 (0.0009) [2023-12-26 20:50:12,828][105692] Updated weights for policy 0, policy_version 751865 (0.0009) [2023-12-26 20:50:12,880][105692] Updated weights for policy 0, policy_version 751875 (0.0009) [2023-12-26 20:50:13,143][105620] Updated weights for policy 1, policy_version 752044 (0.0008) [2023-12-26 20:50:13,200][105620] Updated weights for policy 1, policy_version 752054 (0.0007) [2023-12-26 20:50:13,254][105620] Updated weights for policy 1, policy_version 752064 (0.0005) [2023-12-26 20:50:13,522][105692] Updated weights for policy 0, policy_version 751885 (0.0007) [2023-12-26 20:50:13,569][105692] Updated weights for policy 0, policy_version 751895 (0.0005) [2023-12-26 20:50:13,625][105692] Updated weights for policy 0, policy_version 751905 (0.0005) [2023-12-26 20:50:14,026][105620] Updated weights for policy 1, policy_version 752074 (0.0007) [2023-12-26 20:50:14,088][105620] Updated weights for policy 1, policy_version 752084 (0.0010) [2023-12-26 20:50:14,148][105620] Updated weights for policy 1, policy_version 752094 (0.0010) [2023-12-26 20:50:14,211][105620] Updated weights for policy 1, policy_version 752104 (0.0011) [2023-12-26 20:50:14,270][105692] Updated weights for policy 0, policy_version 751915 (0.0006) [2023-12-26 20:50:14,327][105692] Updated weights for policy 0, policy_version 751925 (0.0008) [2023-12-26 20:50:14,390][105692] Updated weights for policy 0, policy_version 751935 (0.0008) [2023-12-26 20:50:14,961][105620] Updated weights for policy 1, policy_version 752114 (0.0011) [2023-12-26 20:50:15,027][105620] Updated weights for policy 1, policy_version 752124 (0.0010) [2023-12-26 20:50:15,090][105620] Updated weights for policy 1, policy_version 752134 (0.0009) [2023-12-26 20:50:15,155][105692] Updated weights for policy 0, policy_version 751945 (0.0008) [2023-12-26 20:50:15,220][105692] Updated weights for policy 0, policy_version 751955 (0.0008) [2023-12-26 20:50:15,284][105692] Updated weights for policy 0, policy_version 751965 (0.0008) [2023-12-26 20:50:15,343][105692] Updated weights for policy 0, policy_version 751975 (0.0008) [2023-12-26 20:50:15,820][105620] Updated weights for policy 1, policy_version 752144 (0.0010) [2023-12-26 20:50:15,868][105620] Updated weights for policy 1, policy_version 752154 (0.0010) [2023-12-26 20:50:15,919][105620] Updated weights for policy 1, policy_version 752164 (0.0010) [2023-12-26 20:50:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.4, 300 sec: 19633.0). Total num frames: 385114112. Throughput: 0: 9865.2, 1: 9600.4. Samples: 385086140. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 20:50:16,062][104569] Avg episode reward: [(0, '9166.153'), (1, '9081.030')] [2023-12-26 20:50:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000752168_192577536.pth... [2023-12-26 20:50:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000751016_192282624.pth [2023-12-26 20:50:16,095][105692] Updated weights for policy 0, policy_version 751985 (0.0008) [2023-12-26 20:50:16,143][105692] Updated weights for policy 0, policy_version 751995 (0.0008) [2023-12-26 20:50:16,191][105692] Updated weights for policy 0, policy_version 752005 (0.0008) [2023-12-26 20:50:16,202][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000752008_192544768.pth... [2023-12-26 20:50:16,205][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000750824_192241664.pth [2023-12-26 20:50:16,687][105620] Updated weights for policy 1, policy_version 752174 (0.0010) [2023-12-26 20:50:16,752][105620] Updated weights for policy 1, policy_version 752184 (0.0010) [2023-12-26 20:50:16,800][105620] Updated weights for policy 1, policy_version 752194 (0.0010) [2023-12-26 20:50:16,968][105692] Updated weights for policy 0, policy_version 752015 (0.0009) [2023-12-26 20:50:17,012][105692] Updated weights for policy 0, policy_version 752025 (0.0007) [2023-12-26 20:50:17,067][105692] Updated weights for policy 0, policy_version 752035 (0.0008) [2023-12-26 20:50:17,546][105620] Updated weights for policy 1, policy_version 752204 (0.0010) [2023-12-26 20:50:17,601][105620] Updated weights for policy 1, policy_version 752214 (0.0010) [2023-12-26 20:50:17,649][105620] Updated weights for policy 1, policy_version 752224 (0.0010) [2023-12-26 20:50:17,852][105692] Updated weights for policy 0, policy_version 752045 (0.0008) [2023-12-26 20:50:17,907][105692] Updated weights for policy 0, policy_version 752055 (0.0008) [2023-12-26 20:50:17,960][105692] Updated weights for policy 0, policy_version 752065 (0.0008) [2023-12-26 20:50:18,387][105620] Updated weights for policy 1, policy_version 752234 (0.0010) [2023-12-26 20:50:18,435][105620] Updated weights for policy 1, policy_version 752244 (0.0008) [2023-12-26 20:50:18,487][105620] Updated weights for policy 1, policy_version 752254 (0.0009) [2023-12-26 20:50:18,540][105620] Updated weights for policy 1, policy_version 752264 (0.0008) [2023-12-26 20:50:18,739][105692] Updated weights for policy 0, policy_version 752075 (0.0009) [2023-12-26 20:50:18,796][105692] Updated weights for policy 0, policy_version 752085 (0.0009) [2023-12-26 20:50:18,852][105692] Updated weights for policy 0, policy_version 752095 (0.0010) [2023-12-26 20:50:19,253][105620] Updated weights for policy 1, policy_version 752274 (0.0009) [2023-12-26 20:50:19,311][105620] Updated weights for policy 1, policy_version 752284 (0.0008) [2023-12-26 20:50:19,376][105620] Updated weights for policy 1, policy_version 752294 (0.0009) [2023-12-26 20:50:19,655][105692] Updated weights for policy 0, policy_version 752105 (0.0009) [2023-12-26 20:50:19,754][105692] Updated weights for policy 0, policy_version 752115 (0.0005) [2023-12-26 20:50:19,817][105692] Updated weights for policy 0, policy_version 752125 (0.0006) [2023-12-26 20:50:19,883][105692] Updated weights for policy 0, policy_version 752135 (0.0009) [2023-12-26 20:50:20,140][105620] Updated weights for policy 1, policy_version 752304 (0.0006) [2023-12-26 20:50:20,210][105620] Updated weights for policy 1, policy_version 752314 (0.0006) [2023-12-26 20:50:20,276][105620] Updated weights for policy 1, policy_version 752324 (0.0006) [2023-12-26 20:50:20,529][105692] Updated weights for policy 0, policy_version 752145 (0.0009) [2023-12-26 20:50:20,597][105692] Updated weights for policy 0, policy_version 752155 (0.0010) [2023-12-26 20:50:20,663][105692] Updated weights for policy 0, policy_version 752165 (0.0009) [2023-12-26 20:50:20,871][105620] Updated weights for policy 1, policy_version 752334 (0.0006) [2023-12-26 20:50:20,934][105620] Updated weights for policy 1, policy_version 752344 (0.0005) [2023-12-26 20:50:20,999][105620] Updated weights for policy 1, policy_version 752354 (0.0007) [2023-12-26 20:50:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 385212416. Throughput: 0: 9783.1, 1: 9536.9. Samples: 385197672. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 20:50:21,062][104569] Avg episode reward: [(0, '9348.249'), (1, '9266.920')] [2023-12-26 20:50:21,276][105692] Updated weights for policy 0, policy_version 752175 (0.0009) [2023-12-26 20:50:21,338][105692] Updated weights for policy 0, policy_version 752185 (0.0009) [2023-12-26 20:50:21,403][105692] Updated weights for policy 0, policy_version 752195 (0.0009) [2023-12-26 20:50:21,647][105620] Updated weights for policy 1, policy_version 752364 (0.0007) [2023-12-26 20:50:21,719][105620] Updated weights for policy 1, policy_version 752374 (0.0008) [2023-12-26 20:50:21,785][105620] Updated weights for policy 1, policy_version 752384 (0.0008) [2023-12-26 20:50:22,154][105692] Updated weights for policy 0, policy_version 752205 (0.0007) [2023-12-26 20:50:22,226][105692] Updated weights for policy 0, policy_version 752215 (0.0010) [2023-12-26 20:50:22,288][105692] Updated weights for policy 0, policy_version 752225 (0.0008) [2023-12-26 20:50:22,419][105620] Updated weights for policy 1, policy_version 752394 (0.0006) [2023-12-26 20:50:22,485][105620] Updated weights for policy 1, policy_version 752404 (0.0009) [2023-12-26 20:50:22,539][105620] Updated weights for policy 1, policy_version 752414 (0.0009) [2023-12-26 20:50:22,601][105620] Updated weights for policy 1, policy_version 752424 (0.0009) [2023-12-26 20:50:23,034][105692] Updated weights for policy 0, policy_version 752235 (0.0008) [2023-12-26 20:50:23,094][105692] Updated weights for policy 0, policy_version 752245 (0.0009) [2023-12-26 20:50:23,153][105692] Updated weights for policy 0, policy_version 752255 (0.0009) [2023-12-26 20:50:23,364][105620] Updated weights for policy 1, policy_version 752434 (0.0009) [2023-12-26 20:50:23,425][105620] Updated weights for policy 1, policy_version 752444 (0.0009) [2023-12-26 20:50:23,482][105620] Updated weights for policy 1, policy_version 752454 (0.0009) [2023-12-26 20:50:23,913][105692] Updated weights for policy 0, policy_version 752265 (0.0009) [2023-12-26 20:50:23,976][105692] Updated weights for policy 0, policy_version 752275 (0.0009) [2023-12-26 20:50:24,035][105692] Updated weights for policy 0, policy_version 752285 (0.0009) [2023-12-26 20:50:24,096][105692] Updated weights for policy 0, policy_version 752295 (0.0009) [2023-12-26 20:50:24,230][105620] Updated weights for policy 1, policy_version 752464 (0.0009) [2023-12-26 20:50:24,290][105620] Updated weights for policy 1, policy_version 752474 (0.0008) [2023-12-26 20:50:24,352][105620] Updated weights for policy 1, policy_version 752484 (0.0009) [2023-12-26 20:50:24,869][105692] Updated weights for policy 0, policy_version 752305 (0.0009) [2023-12-26 20:50:24,934][105692] Updated weights for policy 0, policy_version 752315 (0.0009) [2023-12-26 20:50:24,996][105692] Updated weights for policy 0, policy_version 752325 (0.0009) [2023-12-26 20:50:25,058][105620] Updated weights for policy 1, policy_version 752494 (0.0009) [2023-12-26 20:50:25,106][105620] Updated weights for policy 1, policy_version 752504 (0.0009) [2023-12-26 20:50:25,163][105620] Updated weights for policy 1, policy_version 752515 (0.0009) [2023-12-26 20:50:25,772][105692] Updated weights for policy 0, policy_version 752335 (0.0006) [2023-12-26 20:50:25,823][105692] Updated weights for policy 0, policy_version 752345 (0.0009) [2023-12-26 20:50:25,858][105620] Updated weights for policy 1, policy_version 752525 (0.0006) [2023-12-26 20:50:25,881][105692] Updated weights for policy 0, policy_version 752355 (0.0006) [2023-12-26 20:50:25,906][105620] Updated weights for policy 1, policy_version 752535 (0.0005) [2023-12-26 20:50:25,957][105620] Updated weights for policy 1, policy_version 752545 (0.0009) [2023-12-26 20:50:26,063][104569] Fps is (10 sec: 19659.4, 60 sec: 19524.1, 300 sec: 19633.0). Total num frames: 385310720. Throughput: 0: 9763.6, 1: 9634.4. Samples: 385314332. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 20:50:26,064][104569] Avg episode reward: [(0, '9348.263'), (1, '9258.531')] [2023-12-26 20:50:26,464][105692] Updated weights for policy 0, policy_version 752365 (0.0005) [2023-12-26 20:50:26,520][105692] Updated weights for policy 0, policy_version 752375 (0.0005) [2023-12-26 20:50:26,527][105620] Updated weights for policy 1, policy_version 752555 (0.0008) [2023-12-26 20:50:26,583][105692] Updated weights for policy 0, policy_version 752385 (0.0005) [2023-12-26 20:50:26,597][105620] Updated weights for policy 1, policy_version 752565 (0.0008) [2023-12-26 20:50:26,668][105620] Updated weights for policy 1, policy_version 752575 (0.0008) [2023-12-26 20:50:27,274][105692] Updated weights for policy 0, policy_version 752395 (0.0006) [2023-12-26 20:50:27,306][105620] Updated weights for policy 1, policy_version 752585 (0.0005) [2023-12-26 20:50:27,330][105692] Updated weights for policy 0, policy_version 752405 (0.0009) [2023-12-26 20:50:27,358][105620] Updated weights for policy 1, policy_version 752595 (0.0007) [2023-12-26 20:50:27,392][105692] Updated weights for policy 0, policy_version 752415 (0.0006) [2023-12-26 20:50:27,406][105620] Updated weights for policy 1, policy_version 752605 (0.0005) [2023-12-26 20:50:27,457][105620] Updated weights for policy 1, policy_version 752615 (0.0005) [2023-12-26 20:50:28,125][105620] Updated weights for policy 1, policy_version 752625 (0.0008) [2023-12-26 20:50:28,130][105692] Updated weights for policy 0, policy_version 752425 (0.0007) [2023-12-26 20:50:28,168][105620] Updated weights for policy 1, policy_version 752635 (0.0005) [2023-12-26 20:50:28,175][105692] Updated weights for policy 0, policy_version 752435 (0.0009) [2023-12-26 20:50:28,221][105620] Updated weights for policy 1, policy_version 752645 (0.0009) [2023-12-26 20:50:28,236][105692] Updated weights for policy 0, policy_version 752445 (0.0008) [2023-12-26 20:50:28,293][105692] Updated weights for policy 0, policy_version 752455 (0.0010) [2023-12-26 20:50:28,913][105692] Updated weights for policy 0, policy_version 752465 (0.0007) [2023-12-26 20:50:28,966][105620] Updated weights for policy 1, policy_version 752655 (0.0010) [2023-12-26 20:50:28,967][105692] Updated weights for policy 0, policy_version 752475 (0.0005) [2023-12-26 20:50:29,018][105620] Updated weights for policy 1, policy_version 752665 (0.0010) [2023-12-26 20:50:29,023][105692] Updated weights for policy 0, policy_version 752485 (0.0006) [2023-12-26 20:50:29,075][105620] Updated weights for policy 1, policy_version 752675 (0.0010) [2023-12-26 20:50:29,773][105692] Updated weights for policy 0, policy_version 752495 (0.0008) [2023-12-26 20:50:29,815][105620] Updated weights for policy 1, policy_version 752685 (0.0009) [2023-12-26 20:50:29,837][105692] Updated weights for policy 0, policy_version 752505 (0.0007) [2023-12-26 20:50:29,875][105620] Updated weights for policy 1, policy_version 752695 (0.0010) [2023-12-26 20:50:29,894][105692] Updated weights for policy 0, policy_version 752515 (0.0006) [2023-12-26 20:50:29,929][105620] Updated weights for policy 1, policy_version 752705 (0.0010) [2023-12-26 20:50:30,599][105692] Updated weights for policy 0, policy_version 752525 (0.0007) [2023-12-26 20:50:30,662][105692] Updated weights for policy 0, policy_version 752535 (0.0008) [2023-12-26 20:50:30,691][105620] Updated weights for policy 1, policy_version 752715 (0.0010) [2023-12-26 20:50:30,720][105692] Updated weights for policy 0, policy_version 752545 (0.0008) [2023-12-26 20:50:30,748][105620] Updated weights for policy 1, policy_version 752725 (0.0010) [2023-12-26 20:50:30,799][105620] Updated weights for policy 1, policy_version 752735 (0.0010) [2023-12-26 20:50:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 385409024. Throughput: 0: 9783.7, 1: 9737.0. Samples: 385376328. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 20:50:31,062][104569] Avg episode reward: [(0, '9080.631'), (1, '8884.707')] [2023-12-26 20:50:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000752552_192684032.pth... [2023-12-26 20:50:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000752744_192724992.pth... [2023-12-26 20:50:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000751592_192430080.pth [2023-12-26 20:50:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000751400_192389120.pth [2023-12-26 20:50:31,380][105692] Updated weights for policy 0, policy_version 752555 (0.0007) [2023-12-26 20:50:31,430][105692] Updated weights for policy 0, policy_version 752565 (0.0006) [2023-12-26 20:50:31,481][105692] Updated weights for policy 0, policy_version 752575 (0.0005) [2023-12-26 20:50:31,542][105620] Updated weights for policy 1, policy_version 752745 (0.0010) [2023-12-26 20:50:31,610][105620] Updated weights for policy 1, policy_version 752755 (0.0010) [2023-12-26 20:50:31,673][105620] Updated weights for policy 1, policy_version 752765 (0.0012) [2023-12-26 20:50:31,742][105620] Updated weights for policy 1, policy_version 752775 (0.0011) [2023-12-26 20:50:32,247][105692] Updated weights for policy 0, policy_version 752585 (0.0006) [2023-12-26 20:50:32,308][105692] Updated weights for policy 0, policy_version 752595 (0.0008) [2023-12-26 20:50:32,322][105620] Updated weights for policy 1, policy_version 752785 (0.0008) [2023-12-26 20:50:32,371][105692] Updated weights for policy 0, policy_version 752605 (0.0007) [2023-12-26 20:50:32,389][105620] Updated weights for policy 1, policy_version 752795 (0.0008) [2023-12-26 20:50:32,430][105692] Updated weights for policy 0, policy_version 752615 (0.0009) [2023-12-26 20:50:32,444][105620] Updated weights for policy 1, policy_version 752805 (0.0007) [2023-12-26 20:50:33,178][105692] Updated weights for policy 0, policy_version 752625 (0.0008) [2023-12-26 20:50:33,195][105620] Updated weights for policy 1, policy_version 752815 (0.0008) [2023-12-26 20:50:33,226][105692] Updated weights for policy 0, policy_version 752635 (0.0007) [2023-12-26 20:50:33,240][105620] Updated weights for policy 1, policy_version 752825 (0.0006) [2023-12-26 20:50:33,279][105692] Updated weights for policy 0, policy_version 752645 (0.0008) [2023-12-26 20:50:33,281][105620] Updated weights for policy 1, policy_version 752835 (0.0007) [2023-12-26 20:50:33,922][105692] Updated weights for policy 0, policy_version 752655 (0.0007) [2023-12-26 20:50:33,969][105692] Updated weights for policy 0, policy_version 752665 (0.0008) [2023-12-26 20:50:34,026][105692] Updated weights for policy 0, policy_version 752675 (0.0009) [2023-12-26 20:50:34,083][105620] Updated weights for policy 1, policy_version 752845 (0.0008) [2023-12-26 20:50:34,140][105620] Updated weights for policy 1, policy_version 752855 (0.0009) [2023-12-26 20:50:34,206][105620] Updated weights for policy 1, policy_version 752865 (0.0009) [2023-12-26 20:50:34,705][105692] Updated weights for policy 0, policy_version 752685 (0.0008) [2023-12-26 20:50:34,756][105692] Updated weights for policy 0, policy_version 752695 (0.0008) [2023-12-26 20:50:34,812][105692] Updated weights for policy 0, policy_version 752705 (0.0009) [2023-12-26 20:50:35,026][105620] Updated weights for policy 1, policy_version 752875 (0.0010) [2023-12-26 20:50:35,090][105620] Updated weights for policy 1, policy_version 752885 (0.0008) [2023-12-26 20:50:35,147][105620] Updated weights for policy 1, policy_version 752895 (0.0007) [2023-12-26 20:50:35,567][105692] Updated weights for policy 0, policy_version 752715 (0.0009) [2023-12-26 20:50:35,631][105692] Updated weights for policy 0, policy_version 752725 (0.0008) [2023-12-26 20:50:35,691][105692] Updated weights for policy 0, policy_version 752735 (0.0006) [2023-12-26 20:50:35,939][105620] Updated weights for policy 1, policy_version 752905 (0.0007) [2023-12-26 20:50:35,990][105620] Updated weights for policy 1, policy_version 752915 (0.0008) [2023-12-26 20:50:36,045][105620] Updated weights for policy 1, policy_version 752925 (0.0009) [2023-12-26 20:50:36,062][104569] Fps is (10 sec: 18842.9, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 385499136. Throughput: 0: 9865.1, 1: 9702.9. Samples: 385493080. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 20:50:36,062][104569] Avg episode reward: [(0, '9081.796'), (1, '8882.555')] [2023-12-26 20:50:36,100][105620] Updated weights for policy 1, policy_version 752935 (0.0009) [2023-12-26 20:50:36,317][105692] Updated weights for policy 0, policy_version 752745 (0.0006) [2023-12-26 20:50:36,374][105692] Updated weights for policy 0, policy_version 752755 (0.0009) [2023-12-26 20:50:36,442][105692] Updated weights for policy 0, policy_version 752765 (0.0009) [2023-12-26 20:50:36,503][105692] Updated weights for policy 0, policy_version 752775 (0.0009) [2023-12-26 20:50:36,909][105620] Updated weights for policy 1, policy_version 752945 (0.0010) [2023-12-26 20:50:36,960][105620] Updated weights for policy 1, policy_version 752955 (0.0008) [2023-12-26 20:50:37,013][105620] Updated weights for policy 1, policy_version 752965 (0.0008) [2023-12-26 20:50:37,162][105692] Updated weights for policy 0, policy_version 752785 (0.0009) [2023-12-26 20:50:37,232][105692] Updated weights for policy 0, policy_version 752795 (0.0007) [2023-12-26 20:50:37,306][105692] Updated weights for policy 0, policy_version 752805 (0.0005) [2023-12-26 20:50:37,835][105620] Updated weights for policy 1, policy_version 752975 (0.0009) [2023-12-26 20:50:37,881][105692] Updated weights for policy 0, policy_version 752815 (0.0005) [2023-12-26 20:50:37,886][105620] Updated weights for policy 1, policy_version 752985 (0.0008) [2023-12-26 20:50:37,940][105692] Updated weights for policy 0, policy_version 752825 (0.0008) [2023-12-26 20:50:37,946][105620] Updated weights for policy 1, policy_version 752995 (0.0008) [2023-12-26 20:50:38,004][105692] Updated weights for policy 0, policy_version 752835 (0.0008) [2023-12-26 20:50:38,678][105620] Updated weights for policy 1, policy_version 753005 (0.0007) [2023-12-26 20:50:38,723][105692] Updated weights for policy 0, policy_version 752845 (0.0009) [2023-12-26 20:50:38,739][105620] Updated weights for policy 1, policy_version 753015 (0.0007) [2023-12-26 20:50:38,786][105692] Updated weights for policy 0, policy_version 752855 (0.0011) [2023-12-26 20:50:38,796][105620] Updated weights for policy 1, policy_version 753025 (0.0006) [2023-12-26 20:50:38,845][105692] Updated weights for policy 0, policy_version 752865 (0.0011) [2023-12-26 20:50:39,461][105692] Updated weights for policy 0, policy_version 752875 (0.0008) [2023-12-26 20:50:39,472][105620] Updated weights for policy 1, policy_version 753035 (0.0006) [2023-12-26 20:50:39,517][105692] Updated weights for policy 0, policy_version 752885 (0.0011) [2023-12-26 20:50:39,535][105620] Updated weights for policy 1, policy_version 753045 (0.0006) [2023-12-26 20:50:39,574][105692] Updated weights for policy 0, policy_version 752895 (0.0011) [2023-12-26 20:50:39,598][105620] Updated weights for policy 1, policy_version 753055 (0.0006) [2023-12-26 20:50:40,310][105620] Updated weights for policy 1, policy_version 753065 (0.0007) [2023-12-26 20:50:40,365][105620] Updated weights for policy 1, policy_version 753075 (0.0009) [2023-12-26 20:50:40,409][105692] Updated weights for policy 0, policy_version 752905 (0.0009) [2023-12-26 20:50:40,422][105620] Updated weights for policy 1, policy_version 753085 (0.0008) [2023-12-26 20:50:40,470][105620] Updated weights for policy 1, policy_version 753095 (0.0009) [2023-12-26 20:50:40,470][105692] Updated weights for policy 0, policy_version 752915 (0.0008) [2023-12-26 20:50:40,527][105692] Updated weights for policy 0, policy_version 752925 (0.0009) [2023-12-26 20:50:40,576][105692] Updated weights for policy 0, policy_version 752935 (0.0009) [2023-12-26 20:50:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 385597440. Throughput: 0: 9932.5, 1: 9661.7. Samples: 385608308. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 20:50:41,062][104569] Avg episode reward: [(0, '9201.548'), (1, '8886.117')] [2023-12-26 20:50:41,248][105692] Updated weights for policy 0, policy_version 752945 (0.0010) [2023-12-26 20:50:41,311][105620] Updated weights for policy 1, policy_version 753105 (0.0007) [2023-12-26 20:50:41,313][105692] Updated weights for policy 0, policy_version 752955 (0.0008) [2023-12-26 20:50:41,380][105620] Updated weights for policy 1, policy_version 753115 (0.0008) [2023-12-26 20:50:41,382][105692] Updated weights for policy 0, policy_version 752965 (0.0008) [2023-12-26 20:50:41,443][105620] Updated weights for policy 1, policy_version 753125 (0.0005) [2023-12-26 20:50:42,151][105620] Updated weights for policy 1, policy_version 753135 (0.0006) [2023-12-26 20:50:42,155][105692] Updated weights for policy 0, policy_version 752975 (0.0010) [2023-12-26 20:50:42,212][105620] Updated weights for policy 1, policy_version 753145 (0.0006) [2023-12-26 20:50:42,215][105692] Updated weights for policy 0, policy_version 752985 (0.0011) [2023-12-26 20:50:42,276][105620] Updated weights for policy 1, policy_version 753155 (0.0007) [2023-12-26 20:50:42,278][105692] Updated weights for policy 0, policy_version 752995 (0.0011) [2023-12-26 20:50:42,842][105620] Updated weights for policy 1, policy_version 753165 (0.0006) [2023-12-26 20:50:42,892][105620] Updated weights for policy 1, policy_version 753175 (0.0007) [2023-12-26 20:50:42,941][105620] Updated weights for policy 1, policy_version 753185 (0.0009) [2023-12-26 20:50:43,027][105692] Updated weights for policy 0, policy_version 753005 (0.0010) [2023-12-26 20:50:43,080][105692] Updated weights for policy 0, policy_version 753017 (0.0011) [2023-12-26 20:50:43,125][105692] Updated weights for policy 0, policy_version 753027 (0.0010) [2023-12-26 20:50:43,611][105620] Updated weights for policy 1, policy_version 753196 (0.0009) [2023-12-26 20:50:43,670][105620] Updated weights for policy 1, policy_version 753206 (0.0008) [2023-12-26 20:50:43,734][105620] Updated weights for policy 1, policy_version 753216 (0.0008) [2023-12-26 20:50:43,837][105692] Updated weights for policy 0, policy_version 753037 (0.0008) [2023-12-26 20:50:43,893][105692] Updated weights for policy 0, policy_version 753047 (0.0008) [2023-12-26 20:50:43,947][105692] Updated weights for policy 0, policy_version 753057 (0.0005) [2023-12-26 20:50:44,377][105620] Updated weights for policy 1, policy_version 753226 (0.0008) [2023-12-26 20:50:44,433][105620] Updated weights for policy 1, policy_version 753236 (0.0008) [2023-12-26 20:50:44,496][105620] Updated weights for policy 1, policy_version 753246 (0.0008) [2023-12-26 20:50:44,556][105620] Updated weights for policy 1, policy_version 753256 (0.0008) [2023-12-26 20:50:44,666][105692] Updated weights for policy 0, policy_version 753067 (0.0008) [2023-12-26 20:50:44,710][105692] Updated weights for policy 0, policy_version 753077 (0.0010) [2023-12-26 20:50:44,755][105692] Updated weights for policy 0, policy_version 753087 (0.0010) [2023-12-26 20:50:45,306][105620] Updated weights for policy 1, policy_version 753266 (0.0008) [2023-12-26 20:50:45,366][105620] Updated weights for policy 1, policy_version 753276 (0.0008) [2023-12-26 20:50:45,422][105620] Updated weights for policy 1, policy_version 753286 (0.0008) [2023-12-26 20:50:45,549][105692] Updated weights for policy 0, policy_version 753097 (0.0011) [2023-12-26 20:50:45,610][105692] Updated weights for policy 0, policy_version 753107 (0.0010) [2023-12-26 20:50:45,671][105692] Updated weights for policy 0, policy_version 753117 (0.0010) [2023-12-26 20:50:45,735][105692] Updated weights for policy 0, policy_version 753127 (0.0010) [2023-12-26 20:50:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 385695744. Throughput: 0: 9879.4, 1: 9695.8. Samples: 385668020. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 20:50:46,063][104569] Avg episode reward: [(0, '9017.443'), (1, '8888.914')] [2023-12-26 20:50:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000753128_192831488.pth... [2023-12-26 20:50:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000753288_192864256.pth... [2023-12-26 20:50:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000752168_192577536.pth [2023-12-26 20:50:46,080][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000752008_192544768.pth [2023-12-26 20:50:46,166][105620] Updated weights for policy 1, policy_version 753296 (0.0008) [2023-12-26 20:50:46,228][105620] Updated weights for policy 1, policy_version 753306 (0.0008) [2023-12-26 20:50:46,287][105620] Updated weights for policy 1, policy_version 753316 (0.0008) [2023-12-26 20:50:46,457][105692] Updated weights for policy 0, policy_version 753137 (0.0011) [2023-12-26 20:50:46,505][105692] Updated weights for policy 0, policy_version 753147 (0.0010) [2023-12-26 20:50:46,565][105692] Updated weights for policy 0, policy_version 753157 (0.0008) [2023-12-26 20:50:47,035][105620] Updated weights for policy 1, policy_version 753326 (0.0008) [2023-12-26 20:50:47,099][105620] Updated weights for policy 1, policy_version 753336 (0.0006) [2023-12-26 20:50:47,160][105620] Updated weights for policy 1, policy_version 753346 (0.0007) [2023-12-26 20:50:47,316][105692] Updated weights for policy 0, policy_version 753167 (0.0009) [2023-12-26 20:50:47,363][105692] Updated weights for policy 0, policy_version 753177 (0.0010) [2023-12-26 20:50:47,407][105692] Updated weights for policy 0, policy_version 753187 (0.0010) [2023-12-26 20:50:47,816][105620] Updated weights for policy 1, policy_version 753356 (0.0006) [2023-12-26 20:50:47,878][105620] Updated weights for policy 1, policy_version 753366 (0.0008) [2023-12-26 20:50:47,931][105620] Updated weights for policy 1, policy_version 753376 (0.0010) [2023-12-26 20:50:48,159][105692] Updated weights for policy 0, policy_version 753197 (0.0008) [2023-12-26 20:50:48,221][105692] Updated weights for policy 0, policy_version 753207 (0.0005) [2023-12-26 20:50:48,282][105692] Updated weights for policy 0, policy_version 753217 (0.0005) [2023-12-26 20:50:48,691][105620] Updated weights for policy 1, policy_version 753386 (0.0009) [2023-12-26 20:50:48,750][105620] Updated weights for policy 1, policy_version 753396 (0.0008) [2023-12-26 20:50:48,807][105620] Updated weights for policy 1, policy_version 753406 (0.0009) [2023-12-26 20:50:48,859][105620] Updated weights for policy 1, policy_version 753416 (0.0009) [2023-12-26 20:50:48,962][105692] Updated weights for policy 0, policy_version 753227 (0.0007) [2023-12-26 20:50:49,020][105692] Updated weights for policy 0, policy_version 753237 (0.0010) [2023-12-26 20:50:49,095][105692] Updated weights for policy 0, policy_version 753247 (0.0010) [2023-12-26 20:50:49,560][105620] Updated weights for policy 1, policy_version 753426 (0.0006) [2023-12-26 20:50:49,624][105620] Updated weights for policy 1, policy_version 753436 (0.0011) [2023-12-26 20:50:49,690][105620] Updated weights for policy 1, policy_version 753446 (0.0009) [2023-12-26 20:50:49,849][105692] Updated weights for policy 0, policy_version 753257 (0.0010) [2023-12-26 20:50:49,915][105692] Updated weights for policy 0, policy_version 753267 (0.0010) [2023-12-26 20:50:49,981][105692] Updated weights for policy 0, policy_version 753277 (0.0008) [2023-12-26 20:50:50,044][105692] Updated weights for policy 0, policy_version 753287 (0.0006) [2023-12-26 20:50:50,331][105620] Updated weights for policy 1, policy_version 753456 (0.0006) [2023-12-26 20:50:50,407][105620] Updated weights for policy 1, policy_version 753466 (0.0008) [2023-12-26 20:50:50,474][105620] Updated weights for policy 1, policy_version 753476 (0.0011) [2023-12-26 20:50:50,686][105692] Updated weights for policy 0, policy_version 753297 (0.0008) [2023-12-26 20:50:50,751][105692] Updated weights for policy 0, policy_version 753307 (0.0009) [2023-12-26 20:50:50,814][105692] Updated weights for policy 0, policy_version 753317 (0.0008) [2023-12-26 20:50:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 385794048. Throughput: 0: 9756.1, 1: 9685.9. Samples: 385782964. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 20:50:51,062][104569] Avg episode reward: [(0, '8982.758'), (1, '8983.622')] [2023-12-26 20:50:51,203][105620] Updated weights for policy 1, policy_version 753486 (0.0010) [2023-12-26 20:50:51,255][105620] Updated weights for policy 1, policy_version 753496 (0.0009) [2023-12-26 20:50:51,320][105620] Updated weights for policy 1, policy_version 753506 (0.0007) [2023-12-26 20:50:51,565][105692] Updated weights for policy 0, policy_version 753327 (0.0010) [2023-12-26 20:50:51,613][105692] Updated weights for policy 0, policy_version 753337 (0.0010) [2023-12-26 20:50:51,682][105692] Updated weights for policy 0, policy_version 753347 (0.0010) [2023-12-26 20:50:52,016][105620] Updated weights for policy 1, policy_version 753516 (0.0007) [2023-12-26 20:50:52,070][105620] Updated weights for policy 1, policy_version 753526 (0.0005) [2023-12-26 20:50:52,124][105620] Updated weights for policy 1, policy_version 753536 (0.0005) [2023-12-26 20:50:52,539][105692] Updated weights for policy 0, policy_version 753357 (0.0010) [2023-12-26 20:50:52,596][105692] Updated weights for policy 0, policy_version 753367 (0.0009) [2023-12-26 20:50:52,648][105692] Updated weights for policy 0, policy_version 753377 (0.0010) [2023-12-26 20:50:52,736][105620] Updated weights for policy 1, policy_version 753546 (0.0007) [2023-12-26 20:50:52,789][105620] Updated weights for policy 1, policy_version 753556 (0.0008) [2023-12-26 20:50:52,840][105620] Updated weights for policy 1, policy_version 753566 (0.0006) [2023-12-26 20:50:52,893][105620] Updated weights for policy 1, policy_version 753576 (0.0007) [2023-12-26 20:50:53,397][105692] Updated weights for policy 0, policy_version 753387 (0.0009) [2023-12-26 20:50:53,465][105692] Updated weights for policy 0, policy_version 753397 (0.0007) [2023-12-26 20:50:53,521][105692] Updated weights for policy 0, policy_version 753407 (0.0009) [2023-12-26 20:50:53,662][105620] Updated weights for policy 1, policy_version 753586 (0.0009) [2023-12-26 20:50:53,709][105620] Updated weights for policy 1, policy_version 753596 (0.0009) [2023-12-26 20:50:53,756][105620] Updated weights for policy 1, policy_version 753606 (0.0009) [2023-12-26 20:50:54,160][105692] Updated weights for policy 0, policy_version 753417 (0.0007) [2023-12-26 20:50:54,215][105692] Updated weights for policy 0, policy_version 753427 (0.0006) [2023-12-26 20:50:54,281][105692] Updated weights for policy 0, policy_version 753437 (0.0006) [2023-12-26 20:50:54,332][105692] Updated weights for policy 0, policy_version 753447 (0.0005) [2023-12-26 20:50:54,569][105620] Updated weights for policy 1, policy_version 753616 (0.0008) [2023-12-26 20:50:54,620][105620] Updated weights for policy 1, policy_version 753627 (0.0009) [2023-12-26 20:50:54,666][105620] Updated weights for policy 1, policy_version 753637 (0.0009) [2023-12-26 20:50:54,987][105692] Updated weights for policy 0, policy_version 753457 (0.0009) [2023-12-26 20:50:55,048][105692] Updated weights for policy 0, policy_version 753467 (0.0006) [2023-12-26 20:50:55,098][105692] Updated weights for policy 0, policy_version 753477 (0.0008) [2023-12-26 20:50:55,336][105620] Updated weights for policy 1, policy_version 753647 (0.0006) [2023-12-26 20:50:55,382][105620] Updated weights for policy 1, policy_version 753657 (0.0005) [2023-12-26 20:50:55,438][105620] Updated weights for policy 1, policy_version 753667 (0.0006) [2023-12-26 20:50:55,876][105692] Updated weights for policy 0, policy_version 753487 (0.0009) [2023-12-26 20:50:55,922][105692] Updated weights for policy 0, policy_version 753497 (0.0009) [2023-12-26 20:50:55,968][105692] Updated weights for policy 0, policy_version 753507 (0.0008) [2023-12-26 20:50:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 385892352. Throughput: 0: 9641.7, 1: 9720.3. Samples: 385900652. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 20:50:56,062][104569] Avg episode reward: [(0, '9074.589'), (1, '9075.351')] [2023-12-26 20:50:56,122][105620] Updated weights for policy 1, policy_version 753677 (0.0008) [2023-12-26 20:50:56,185][105620] Updated weights for policy 1, policy_version 753687 (0.0009) [2023-12-26 20:50:56,248][105620] Updated weights for policy 1, policy_version 753697 (0.0009) [2023-12-26 20:50:56,703][105692] Updated weights for policy 0, policy_version 753518 (0.0011) [2023-12-26 20:50:56,755][105692] Updated weights for policy 0, policy_version 753528 (0.0010) [2023-12-26 20:50:56,800][105692] Updated weights for policy 0, policy_version 753538 (0.0010) [2023-12-26 20:50:56,839][105620] Updated weights for policy 1, policy_version 753707 (0.0007) [2023-12-26 20:50:56,896][105620] Updated weights for policy 1, policy_version 753717 (0.0007) [2023-12-26 20:50:56,947][105620] Updated weights for policy 1, policy_version 753727 (0.0008) [2023-12-26 20:50:57,506][105692] Updated weights for policy 0, policy_version 753548 (0.0008) [2023-12-26 20:50:57,562][105692] Updated weights for policy 0, policy_version 753558 (0.0005) [2023-12-26 20:50:57,621][105692] Updated weights for policy 0, policy_version 753568 (0.0005) [2023-12-26 20:50:57,779][105620] Updated weights for policy 1, policy_version 753737 (0.0008) [2023-12-26 20:50:57,845][105620] Updated weights for policy 1, policy_version 753747 (0.0010) [2023-12-26 20:50:57,910][105620] Updated weights for policy 1, policy_version 753757 (0.0009) [2023-12-26 20:50:57,975][105620] Updated weights for policy 1, policy_version 753767 (0.0008) [2023-12-26 20:50:58,182][105692] Updated weights for policy 0, policy_version 753578 (0.0006) [2023-12-26 20:50:58,242][105692] Updated weights for policy 0, policy_version 753588 (0.0008) [2023-12-26 20:50:58,304][105692] Updated weights for policy 0, policy_version 753598 (0.0008) [2023-12-26 20:50:58,367][105692] Updated weights for policy 0, policy_version 753608 (0.0007) [2023-12-26 20:50:58,771][105620] Updated weights for policy 1, policy_version 753777 (0.0008) [2023-12-26 20:50:58,834][105620] Updated weights for policy 1, policy_version 753787 (0.0008) [2023-12-26 20:50:58,907][105620] Updated weights for policy 1, policy_version 753797 (0.0008) [2023-12-26 20:50:59,187][105692] Updated weights for policy 0, policy_version 753618 (0.0010) [2023-12-26 20:50:59,257][105692] Updated weights for policy 0, policy_version 753628 (0.0009) [2023-12-26 20:50:59,324][105692] Updated weights for policy 0, policy_version 753638 (0.0009) [2023-12-26 20:50:59,753][105620] Updated weights for policy 1, policy_version 753807 (0.0009) [2023-12-26 20:50:59,813][105620] Updated weights for policy 1, policy_version 753817 (0.0009) [2023-12-26 20:50:59,869][105620] Updated weights for policy 1, policy_version 753827 (0.0008) [2023-12-26 20:50:59,961][105692] Updated weights for policy 0, policy_version 753648 (0.0008) [2023-12-26 20:51:00,017][105692] Updated weights for policy 0, policy_version 753658 (0.0010) [2023-12-26 20:51:00,070][105692] Updated weights for policy 0, policy_version 753668 (0.0011) [2023-12-26 20:51:00,490][105620] Updated weights for policy 1, policy_version 753837 (0.0009) [2023-12-26 20:51:00,541][105620] Updated weights for policy 1, policy_version 753847 (0.0008) [2023-12-26 20:51:00,594][105620] Updated weights for policy 1, policy_version 753857 (0.0009) [2023-12-26 20:51:00,739][105692] Updated weights for policy 0, policy_version 753678 (0.0008) [2023-12-26 20:51:00,798][105692] Updated weights for policy 0, policy_version 753688 (0.0009) [2023-12-26 20:51:00,849][105692] Updated weights for policy 0, policy_version 753698 (0.0010) [2023-12-26 20:51:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 385990656. Throughput: 0: 9709.7, 1: 9691.2. Samples: 385959180. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-26 20:51:01,062][104569] Avg episode reward: [(0, '9347.302'), (1, '9252.806')] [2023-12-26 20:51:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000753704_192978944.pth... [2023-12-26 20:51:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000753864_193011712.pth... [2023-12-26 20:51:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000752552_192684032.pth [2023-12-26 20:51:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000752744_192724992.pth [2023-12-26 20:51:01,344][105620] Updated weights for policy 1, policy_version 753867 (0.0010) [2023-12-26 20:51:01,405][105620] Updated weights for policy 1, policy_version 753877 (0.0008) [2023-12-26 20:51:01,463][105620] Updated weights for policy 1, policy_version 753887 (0.0006) [2023-12-26 20:51:01,578][105692] Updated weights for policy 0, policy_version 753708 (0.0010) [2023-12-26 20:51:01,641][105692] Updated weights for policy 0, policy_version 753718 (0.0011) [2023-12-26 20:51:01,700][105692] Updated weights for policy 0, policy_version 753728 (0.0010) [2023-12-26 20:51:02,066][105620] Updated weights for policy 1, policy_version 753897 (0.0006) [2023-12-26 20:51:02,127][105620] Updated weights for policy 1, policy_version 753907 (0.0009) [2023-12-26 20:51:02,193][105620] Updated weights for policy 1, policy_version 753917 (0.0009) [2023-12-26 20:51:02,251][105620] Updated weights for policy 1, policy_version 753927 (0.0006) [2023-12-26 20:51:02,413][105692] Updated weights for policy 0, policy_version 753738 (0.0010) [2023-12-26 20:51:02,467][105692] Updated weights for policy 0, policy_version 753748 (0.0010) [2023-12-26 20:51:02,522][105692] Updated weights for policy 0, policy_version 753758 (0.0010) [2023-12-26 20:51:02,570][105692] Updated weights for policy 0, policy_version 753768 (0.0010) [2023-12-26 20:51:02,828][105620] Updated weights for policy 1, policy_version 753937 (0.0010) [2023-12-26 20:51:02,881][105620] Updated weights for policy 1, policy_version 753947 (0.0010) [2023-12-26 20:51:02,940][105620] Updated weights for policy 1, policy_version 753957 (0.0010) [2023-12-26 20:51:03,224][105692] Updated weights for policy 0, policy_version 753778 (0.0006) [2023-12-26 20:51:03,272][105692] Updated weights for policy 0, policy_version 753788 (0.0010) [2023-12-26 20:51:03,320][105692] Updated weights for policy 0, policy_version 753798 (0.0010) [2023-12-26 20:51:03,512][105620] Updated weights for policy 1, policy_version 753967 (0.0006) [2023-12-26 20:51:03,572][105620] Updated weights for policy 1, policy_version 753977 (0.0005) [2023-12-26 20:51:03,622][105620] Updated weights for policy 1, policy_version 753987 (0.0005) [2023-12-26 20:51:04,060][105692] Updated weights for policy 0, policy_version 753808 (0.0011) [2023-12-26 20:51:04,124][105692] Updated weights for policy 0, policy_version 753818 (0.0011) [2023-12-26 20:51:04,188][105692] Updated weights for policy 0, policy_version 753828 (0.0010) [2023-12-26 20:51:04,257][105620] Updated weights for policy 1, policy_version 753997 (0.0007) [2023-12-26 20:51:04,313][105620] Updated weights for policy 1, policy_version 754007 (0.0007) [2023-12-26 20:51:04,372][105620] Updated weights for policy 1, policy_version 754017 (0.0006) [2023-12-26 20:51:04,910][105692] Updated weights for policy 0, policy_version 753838 (0.0010) [2023-12-26 20:51:04,967][105692] Updated weights for policy 0, policy_version 753848 (0.0010) [2023-12-26 20:51:05,014][105620] Updated weights for policy 1, policy_version 754027 (0.0007) [2023-12-26 20:51:05,019][105692] Updated weights for policy 0, policy_version 753858 (0.0010) [2023-12-26 20:51:05,070][105620] Updated weights for policy 1, policy_version 754037 (0.0006) [2023-12-26 20:51:05,130][105620] Updated weights for policy 1, policy_version 754047 (0.0008) [2023-12-26 20:51:05,762][105692] Updated weights for policy 0, policy_version 753868 (0.0010) [2023-12-26 20:51:05,813][105692] Updated weights for policy 0, policy_version 753878 (0.0010) [2023-12-26 20:51:05,862][105692] Updated weights for policy 0, policy_version 753888 (0.0010) [2023-12-26 20:51:05,911][105620] Updated weights for policy 1, policy_version 754057 (0.0008) [2023-12-26 20:51:05,978][105620] Updated weights for policy 1, policy_version 754067 (0.0010) [2023-12-26 20:51:06,047][105620] Updated weights for policy 1, policy_version 754077 (0.0010) [2023-12-26 20:51:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 386088960. Throughput: 0: 9778.1, 1: 9853.4. Samples: 386081092. Policy #0 lag: (min: 2.0, avg: 18.7, max: 34.0) [2023-12-26 20:51:06,063][104569] Avg episode reward: [(0, '9347.479'), (1, '8897.425')] [2023-12-26 20:51:06,114][105620] Updated weights for policy 1, policy_version 754087 (0.0011) [2023-12-26 20:51:06,651][105692] Updated weights for policy 0, policy_version 753898 (0.0010) [2023-12-26 20:51:06,704][105692] Updated weights for policy 0, policy_version 753908 (0.0008) [2023-12-26 20:51:06,756][105692] Updated weights for policy 0, policy_version 753918 (0.0008) [2023-12-26 20:51:06,813][105692] Updated weights for policy 0, policy_version 753928 (0.0007) [2023-12-26 20:51:06,835][105620] Updated weights for policy 1, policy_version 754097 (0.0010) [2023-12-26 20:51:06,901][105620] Updated weights for policy 1, policy_version 754107 (0.0010) [2023-12-26 20:51:06,965][105620] Updated weights for policy 1, policy_version 754117 (0.0010) [2023-12-26 20:51:07,604][105692] Updated weights for policy 0, policy_version 753938 (0.0008) [2023-12-26 20:51:07,666][105692] Updated weights for policy 0, policy_version 753948 (0.0008) [2023-12-26 20:51:07,716][105620] Updated weights for policy 1, policy_version 754127 (0.0010) [2023-12-26 20:51:07,718][105692] Updated weights for policy 0, policy_version 753958 (0.0008) [2023-12-26 20:51:07,777][105620] Updated weights for policy 1, policy_version 754137 (0.0010) [2023-12-26 20:51:07,835][105620] Updated weights for policy 1, policy_version 754147 (0.0010) [2023-12-26 20:51:08,518][105692] Updated weights for policy 0, policy_version 753968 (0.0008) [2023-12-26 20:51:08,561][105620] Updated weights for policy 1, policy_version 754157 (0.0010) [2023-12-26 20:51:08,579][105692] Updated weights for policy 0, policy_version 753978 (0.0006) [2023-12-26 20:51:08,624][105620] Updated weights for policy 1, policy_version 754167 (0.0008) [2023-12-26 20:51:08,634][105692] Updated weights for policy 0, policy_version 753988 (0.0007) [2023-12-26 20:51:08,679][105620] Updated weights for policy 1, policy_version 754177 (0.0008) [2023-12-26 20:51:09,293][105620] Updated weights for policy 1, policy_version 754187 (0.0008) [2023-12-26 20:51:09,366][105620] Updated weights for policy 1, policy_version 754197 (0.0009) [2023-12-26 20:51:09,394][105586] KL-divergence is very high: 126.3124 [2023-12-26 20:51:09,432][105620] Updated weights for policy 1, policy_version 754207 (0.0008) [2023-12-26 20:51:09,439][105692] Updated weights for policy 0, policy_version 753998 (0.0008) [2023-12-26 20:51:09,444][105586] KL-divergence is very high: 244.1004 [2023-12-26 20:51:09,491][105692] Updated weights for policy 0, policy_version 754008 (0.0008) [2023-12-26 20:51:09,554][105692] Updated weights for policy 0, policy_version 754018 (0.0008) [2023-12-26 20:51:10,072][105620] Updated weights for policy 1, policy_version 754217 (0.0008) [2023-12-26 20:51:10,124][105620] Updated weights for policy 1, policy_version 754227 (0.0005) [2023-12-26 20:51:10,180][105620] Updated weights for policy 1, policy_version 754237 (0.0005) [2023-12-26 20:51:10,232][105620] Updated weights for policy 1, policy_version 754247 (0.0007) [2023-12-26 20:51:10,295][105692] Updated weights for policy 0, policy_version 754028 (0.0009) [2023-12-26 20:51:10,362][105692] Updated weights for policy 0, policy_version 754038 (0.0011) [2023-12-26 20:51:10,428][105692] Updated weights for policy 0, policy_version 754048 (0.0008) [2023-12-26 20:51:10,834][105620] Updated weights for policy 1, policy_version 754257 (0.0006) [2023-12-26 20:51:10,890][105620] Updated weights for policy 1, policy_version 754267 (0.0008) [2023-12-26 20:51:10,948][105620] Updated weights for policy 1, policy_version 754277 (0.0007) [2023-12-26 20:51:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 386187264. Throughput: 0: 9751.2, 1: 9831.9. Samples: 386195560. Policy #0 lag: (min: 2.0, avg: 18.7, max: 34.0) [2023-12-26 20:51:11,063][104569] Avg episode reward: [(0, '9347.992'), (1, '8812.339')] [2023-12-26 20:51:11,104][105692] Updated weights for policy 0, policy_version 754058 (0.0006) [2023-12-26 20:51:11,172][105692] Updated weights for policy 0, policy_version 754068 (0.0009) [2023-12-26 20:51:11,239][105692] Updated weights for policy 0, policy_version 754078 (0.0011) [2023-12-26 20:51:11,302][105692] Updated weights for policy 0, policy_version 754088 (0.0010) [2023-12-26 20:51:11,785][105620] Updated weights for policy 1, policy_version 754287 (0.0008) [2023-12-26 20:51:11,845][105620] Updated weights for policy 1, policy_version 754297 (0.0008) [2023-12-26 20:51:11,908][105620] Updated weights for policy 1, policy_version 754307 (0.0008) [2023-12-26 20:51:12,076][105692] Updated weights for policy 0, policy_version 754098 (0.0009) [2023-12-26 20:51:12,129][105692] Updated weights for policy 0, policy_version 754108 (0.0010) [2023-12-26 20:51:12,195][105692] Updated weights for policy 0, policy_version 754118 (0.0011) [2023-12-26 20:51:12,628][105620] Updated weights for policy 1, policy_version 754317 (0.0007) [2023-12-26 20:51:12,685][105620] Updated weights for policy 1, policy_version 754327 (0.0007) [2023-12-26 20:51:12,750][105620] Updated weights for policy 1, policy_version 754337 (0.0007) [2023-12-26 20:51:12,917][105692] Updated weights for policy 0, policy_version 754128 (0.0007) [2023-12-26 20:51:12,973][105692] Updated weights for policy 0, policy_version 754138 (0.0005) [2023-12-26 20:51:13,029][105692] Updated weights for policy 0, policy_version 754148 (0.0005) [2023-12-26 20:51:13,379][105620] Updated weights for policy 1, policy_version 754347 (0.0009) [2023-12-26 20:51:13,433][105620] Updated weights for policy 1, policy_version 754357 (0.0005) [2023-12-26 20:51:13,485][105620] Updated weights for policy 1, policy_version 754367 (0.0005) [2023-12-26 20:51:13,638][105692] Updated weights for policy 0, policy_version 754158 (0.0008) [2023-12-26 20:51:13,698][105692] Updated weights for policy 0, policy_version 754168 (0.0009) [2023-12-26 20:51:13,755][105692] Updated weights for policy 0, policy_version 754178 (0.0009) [2023-12-26 20:51:14,107][105620] Updated weights for policy 1, policy_version 754377 (0.0005) [2023-12-26 20:51:14,159][105620] Updated weights for policy 1, policy_version 754387 (0.0009) [2023-12-26 20:51:14,220][105620] Updated weights for policy 1, policy_version 754397 (0.0009) [2023-12-26 20:51:14,272][105620] Updated weights for policy 1, policy_version 754407 (0.0007) [2023-12-26 20:51:14,535][105692] Updated weights for policy 0, policy_version 754188 (0.0009) [2023-12-26 20:51:14,588][105692] Updated weights for policy 0, policy_version 754198 (0.0010) [2023-12-26 20:51:14,638][105692] Updated weights for policy 0, policy_version 754208 (0.0009) [2023-12-26 20:51:14,884][105620] Updated weights for policy 1, policy_version 754417 (0.0006) [2023-12-26 20:51:14,943][105620] Updated weights for policy 1, policy_version 754427 (0.0008) [2023-12-26 20:51:15,003][105620] Updated weights for policy 1, policy_version 754437 (0.0011) [2023-12-26 20:51:15,378][105692] Updated weights for policy 0, policy_version 754218 (0.0009) [2023-12-26 20:51:15,433][105692] Updated weights for policy 0, policy_version 754228 (0.0009) [2023-12-26 20:51:15,482][105692] Updated weights for policy 0, policy_version 754238 (0.0010) [2023-12-26 20:51:15,539][105692] Updated weights for policy 0, policy_version 754248 (0.0008) [2023-12-26 20:51:15,747][105620] Updated weights for policy 1, policy_version 754447 (0.0011) [2023-12-26 20:51:15,802][105620] Updated weights for policy 1, policy_version 754457 (0.0010) [2023-12-26 20:51:15,853][105620] Updated weights for policy 1, policy_version 754467 (0.0007) [2023-12-26 20:51:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 386285568. Throughput: 0: 9734.3, 1: 9787.8. Samples: 386254824. Policy #0 lag: (min: 2.0, avg: 18.7, max: 34.0) [2023-12-26 20:51:16,062][104569] Avg episode reward: [(0, '9348.162'), (1, '8903.513')] [2023-12-26 20:51:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000754472_193167360.pth... [2023-12-26 20:51:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000753288_192864256.pth [2023-12-26 20:51:16,081][105692] Updated weights for policy 0, policy_version 754258 (0.0010) [2023-12-26 20:51:16,136][105692] Updated weights for policy 0, policy_version 754268 (0.0009) [2023-12-26 20:51:16,190][105692] Updated weights for policy 0, policy_version 754278 (0.0005) [2023-12-26 20:51:16,202][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000754280_193126400.pth... [2023-12-26 20:51:16,207][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000753128_192831488.pth [2023-12-26 20:51:16,617][105620] Updated weights for policy 1, policy_version 754477 (0.0010) [2023-12-26 20:51:16,680][105620] Updated weights for policy 1, policy_version 754487 (0.0008) [2023-12-26 20:51:16,733][105620] Updated weights for policy 1, policy_version 754497 (0.0009) [2023-12-26 20:51:16,854][105692] Updated weights for policy 0, policy_version 754288 (0.0008) [2023-12-26 20:51:16,907][105692] Updated weights for policy 0, policy_version 754298 (0.0009) [2023-12-26 20:51:16,964][105692] Updated weights for policy 0, policy_version 754308 (0.0008) [2023-12-26 20:51:17,328][105620] Updated weights for policy 1, policy_version 754507 (0.0008) [2023-12-26 20:51:17,389][105620] Updated weights for policy 1, policy_version 754518 (0.0010) [2023-12-26 20:51:17,442][105620] Updated weights for policy 1, policy_version 754528 (0.0009) [2023-12-26 20:51:17,556][105692] Updated weights for policy 0, policy_version 754318 (0.0005) [2023-12-26 20:51:17,609][105692] Updated weights for policy 0, policy_version 754328 (0.0005) [2023-12-26 20:51:17,658][105692] Updated weights for policy 0, policy_version 754338 (0.0008) [2023-12-26 20:51:18,102][105620] Updated weights for policy 1, policy_version 754539 (0.0009) [2023-12-26 20:51:18,157][105620] Updated weights for policy 1, policy_version 754549 (0.0006) [2023-12-26 20:51:18,209][105620] Updated weights for policy 1, policy_version 754559 (0.0009) [2023-12-26 20:51:18,419][105692] Updated weights for policy 0, policy_version 754348 (0.0009) [2023-12-26 20:51:18,466][105692] Updated weights for policy 0, policy_version 754358 (0.0008) [2023-12-26 20:51:18,518][105692] Updated weights for policy 0, policy_version 754369 (0.0010) [2023-12-26 20:51:18,865][105620] Updated weights for policy 1, policy_version 754569 (0.0009) [2023-12-26 20:51:18,922][105620] Updated weights for policy 1, policy_version 754579 (0.0008) [2023-12-26 20:51:18,983][105620] Updated weights for policy 1, policy_version 754589 (0.0009) [2023-12-26 20:51:19,038][105620] Updated weights for policy 1, policy_version 754599 (0.0009) [2023-12-26 20:51:19,347][105692] Updated weights for policy 0, policy_version 754380 (0.0009) [2023-12-26 20:51:19,408][105692] Updated weights for policy 0, policy_version 754390 (0.0008) [2023-12-26 20:51:19,465][105692] Updated weights for policy 0, policy_version 754400 (0.0007) [2023-12-26 20:51:19,812][105620] Updated weights for policy 1, policy_version 754609 (0.0008) [2023-12-26 20:51:19,872][105620] Updated weights for policy 1, policy_version 754619 (0.0008) [2023-12-26 20:51:19,924][105620] Updated weights for policy 1, policy_version 754629 (0.0009) [2023-12-26 20:51:20,216][105692] Updated weights for policy 0, policy_version 754410 (0.0007) [2023-12-26 20:51:20,277][105692] Updated weights for policy 0, policy_version 754420 (0.0009) [2023-12-26 20:51:20,328][105692] Updated weights for policy 0, policy_version 754430 (0.0008) [2023-12-26 20:51:20,380][105692] Updated weights for policy 0, policy_version 754440 (0.0009) [2023-12-26 20:51:20,712][105620] Updated weights for policy 1, policy_version 754639 (0.0010) [2023-12-26 20:51:20,775][105620] Updated weights for policy 1, policy_version 754649 (0.0010) [2023-12-26 20:51:20,833][105620] Updated weights for policy 1, policy_version 754659 (0.0009) [2023-12-26 20:51:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 386383872. Throughput: 0: 9726.7, 1: 9858.8. Samples: 386374428. Policy #0 lag: (min: 2.0, avg: 18.7, max: 34.0) [2023-12-26 20:51:21,062][104569] Avg episode reward: [(0, '9254.542'), (1, '9079.137')] [2023-12-26 20:51:21,129][105692] Updated weights for policy 0, policy_version 754450 (0.0009) [2023-12-26 20:51:21,192][105692] Updated weights for policy 0, policy_version 754460 (0.0009) [2023-12-26 20:51:21,252][105692] Updated weights for policy 0, policy_version 754470 (0.0009) [2023-12-26 20:51:21,576][105620] Updated weights for policy 1, policy_version 754669 (0.0009) [2023-12-26 20:51:21,647][105620] Updated weights for policy 1, policy_version 754679 (0.0008) [2023-12-26 20:51:21,707][105620] Updated weights for policy 1, policy_version 754689 (0.0010) [2023-12-26 20:51:22,078][105692] Updated weights for policy 0, policy_version 754480 (0.0009) [2023-12-26 20:51:22,149][105692] Updated weights for policy 0, policy_version 754490 (0.0006) [2023-12-26 20:51:22,218][105692] Updated weights for policy 0, policy_version 754500 (0.0006) [2023-12-26 20:51:22,389][105620] Updated weights for policy 1, policy_version 754699 (0.0010) [2023-12-26 20:51:22,437][105620] Updated weights for policy 1, policy_version 754709 (0.0008) [2023-12-26 20:51:22,490][105620] Updated weights for policy 1, policy_version 754719 (0.0008) [2023-12-26 20:51:22,941][105692] Updated weights for policy 0, policy_version 754510 (0.0009) [2023-12-26 20:51:23,001][105692] Updated weights for policy 0, policy_version 754520 (0.0009) [2023-12-26 20:51:23,057][105692] Updated weights for policy 0, policy_version 754530 (0.0009) [2023-12-26 20:51:23,268][105620] Updated weights for policy 1, policy_version 754729 (0.0008) [2023-12-26 20:51:23,320][105620] Updated weights for policy 1, policy_version 754739 (0.0010) [2023-12-26 20:51:23,381][105620] Updated weights for policy 1, policy_version 754749 (0.0009) [2023-12-26 20:51:23,445][105620] Updated weights for policy 1, policy_version 754759 (0.0009) [2023-12-26 20:51:23,729][105692] Updated weights for policy 0, policy_version 754540 (0.0008) [2023-12-26 20:51:23,785][105692] Updated weights for policy 0, policy_version 754550 (0.0005) [2023-12-26 20:51:23,836][105692] Updated weights for policy 0, policy_version 754560 (0.0005) [2023-12-26 20:51:24,248][105620] Updated weights for policy 1, policy_version 754770 (0.0010) [2023-12-26 20:51:24,302][105620] Updated weights for policy 1, policy_version 754780 (0.0007) [2023-12-26 20:51:24,355][105620] Updated weights for policy 1, policy_version 754790 (0.0007) [2023-12-26 20:51:24,454][105692] Updated weights for policy 0, policy_version 754570 (0.0006) [2023-12-26 20:51:24,511][105692] Updated weights for policy 0, policy_version 754580 (0.0009) [2023-12-26 20:51:24,580][105692] Updated weights for policy 0, policy_version 754590 (0.0006) [2023-12-26 20:51:24,642][105692] Updated weights for policy 0, policy_version 754600 (0.0009) [2023-12-26 20:51:24,960][105620] Updated weights for policy 1, policy_version 754800 (0.0006) [2023-12-26 20:51:25,009][105620] Updated weights for policy 1, policy_version 754810 (0.0005) [2023-12-26 20:51:25,055][105620] Updated weights for policy 1, policy_version 754820 (0.0005) [2023-12-26 20:51:25,308][105692] Updated weights for policy 0, policy_version 754610 (0.0008) [2023-12-26 20:51:25,369][105692] Updated weights for policy 0, policy_version 754620 (0.0005) [2023-12-26 20:51:25,427][105692] Updated weights for policy 0, policy_version 754630 (0.0005) [2023-12-26 20:51:25,592][105620] Updated weights for policy 1, policy_version 754830 (0.0005) [2023-12-26 20:51:25,649][105620] Updated weights for policy 1, policy_version 754840 (0.0007) [2023-12-26 20:51:25,702][105620] Updated weights for policy 1, policy_version 754850 (0.0009) [2023-12-26 20:51:26,044][105692] Updated weights for policy 0, policy_version 754640 (0.0009) [2023-12-26 20:51:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.5, 300 sec: 19605.3). Total num frames: 386482176. Throughput: 0: 9704.0, 1: 9960.0. Samples: 386493188. Policy #0 lag: (min: 2.0, avg: 18.7, max: 34.0) [2023-12-26 20:51:26,062][104569] Avg episode reward: [(0, '9073.600'), (1, '9171.024')] [2023-12-26 20:51:26,101][105692] Updated weights for policy 0, policy_version 754651 (0.0009) [2023-12-26 20:51:26,159][105692] Updated weights for policy 0, policy_version 754661 (0.0009) [2023-12-26 20:51:26,264][105620] Updated weights for policy 1, policy_version 754860 (0.0006) [2023-12-26 20:51:26,315][105620] Updated weights for policy 1, policy_version 754870 (0.0005) [2023-12-26 20:51:26,372][105620] Updated weights for policy 1, policy_version 754880 (0.0005) [2023-12-26 20:51:26,806][105692] Updated weights for policy 0, policy_version 754671 (0.0008) [2023-12-26 20:51:26,861][105692] Updated weights for policy 0, policy_version 754681 (0.0009) [2023-12-26 20:51:26,919][105692] Updated weights for policy 0, policy_version 754691 (0.0009) [2023-12-26 20:51:27,097][105620] Updated weights for policy 1, policy_version 754890 (0.0008) [2023-12-26 20:51:27,155][105620] Updated weights for policy 1, policy_version 754900 (0.0008) [2023-12-26 20:51:27,206][105620] Updated weights for policy 1, policy_version 754910 (0.0008) [2023-12-26 20:51:27,254][105620] Updated weights for policy 1, policy_version 754920 (0.0009) [2023-12-26 20:51:27,723][105692] Updated weights for policy 0, policy_version 754701 (0.0008) [2023-12-26 20:51:27,787][105692] Updated weights for policy 0, policy_version 754711 (0.0009) [2023-12-26 20:51:27,845][105692] Updated weights for policy 0, policy_version 754721 (0.0009) [2023-12-26 20:51:27,951][105620] Updated weights for policy 1, policy_version 754930 (0.0009) [2023-12-26 20:51:28,019][105620] Updated weights for policy 1, policy_version 754940 (0.0009) [2023-12-26 20:51:28,069][105620] Updated weights for policy 1, policy_version 754950 (0.0009) [2023-12-26 20:51:28,616][105692] Updated weights for policy 0, policy_version 754731 (0.0009) [2023-12-26 20:51:28,674][105692] Updated weights for policy 0, policy_version 754741 (0.0009) [2023-12-26 20:51:28,722][105692] Updated weights for policy 0, policy_version 754751 (0.0007) [2023-12-26 20:51:28,776][105620] Updated weights for policy 1, policy_version 754960 (0.0006) [2023-12-26 20:51:28,830][105620] Updated weights for policy 1, policy_version 754970 (0.0006) [2023-12-26 20:51:28,884][105620] Updated weights for policy 1, policy_version 754980 (0.0005) [2023-12-26 20:51:29,451][105620] Updated weights for policy 1, policy_version 754990 (0.0007) [2023-12-26 20:51:29,505][105620] Updated weights for policy 1, policy_version 755000 (0.0006) [2023-12-26 20:51:29,574][105692] Updated weights for policy 0, policy_version 754761 (0.0008) [2023-12-26 20:51:29,575][105620] Updated weights for policy 1, policy_version 755010 (0.0008) [2023-12-26 20:51:29,635][105692] Updated weights for policy 0, policy_version 754771 (0.0007) [2023-12-26 20:51:29,696][105692] Updated weights for policy 0, policy_version 754781 (0.0008) [2023-12-26 20:51:29,757][105692] Updated weights for policy 0, policy_version 754791 (0.0009) [2023-12-26 20:51:30,302][105620] Updated weights for policy 1, policy_version 755020 (0.0008) [2023-12-26 20:51:30,366][105620] Updated weights for policy 1, policy_version 755030 (0.0008) [2023-12-26 20:51:30,423][105620] Updated weights for policy 1, policy_version 755040 (0.0009) [2023-12-26 20:51:30,470][105692] Updated weights for policy 0, policy_version 754801 (0.0005) [2023-12-26 20:51:30,523][105692] Updated weights for policy 0, policy_version 754811 (0.0005) [2023-12-26 20:51:30,582][105692] Updated weights for policy 0, policy_version 754821 (0.0007) [2023-12-26 20:51:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 386580480. Throughput: 0: 9707.7, 1: 9966.2. Samples: 386553340. Policy #0 lag: (min: 2.0, avg: 18.7, max: 34.0) [2023-12-26 20:51:31,062][104569] Avg episode reward: [(0, '9166.703'), (1, '9166.205')] [2023-12-26 20:51:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000755048_193314816.pth... [2023-12-26 20:51:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000754824_193265664.pth... [2023-12-26 20:51:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000753864_193011712.pth [2023-12-26 20:51:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000753704_192978944.pth [2023-12-26 20:51:31,170][105620] Updated weights for policy 1, policy_version 755050 (0.0009) [2023-12-26 20:51:31,236][105620] Updated weights for policy 1, policy_version 755060 (0.0010) [2023-12-26 20:51:31,287][105692] Updated weights for policy 0, policy_version 754831 (0.0007) [2023-12-26 20:51:31,296][105620] Updated weights for policy 1, policy_version 755070 (0.0007) [2023-12-26 20:51:31,351][105692] Updated weights for policy 0, policy_version 754841 (0.0008) [2023-12-26 20:51:31,362][105620] Updated weights for policy 1, policy_version 755080 (0.0011) [2023-12-26 20:51:31,418][105692] Updated weights for policy 0, policy_version 754851 (0.0011) [2023-12-26 20:51:32,033][105620] Updated weights for policy 1, policy_version 755090 (0.0010) [2023-12-26 20:51:32,070][105692] Updated weights for policy 0, policy_version 754861 (0.0011) [2023-12-26 20:51:32,084][105620] Updated weights for policy 1, policy_version 755100 (0.0010) [2023-12-26 20:51:32,126][105692] Updated weights for policy 0, policy_version 754871 (0.0010) [2023-12-26 20:51:32,143][105620] Updated weights for policy 1, policy_version 755110 (0.0010) [2023-12-26 20:51:32,188][105692] Updated weights for policy 0, policy_version 754881 (0.0010) [2023-12-26 20:51:32,891][105620] Updated weights for policy 1, policy_version 755121 (0.0008) [2023-12-26 20:51:32,917][105692] Updated weights for policy 0, policy_version 754891 (0.0010) [2023-12-26 20:51:32,936][105620] Updated weights for policy 1, policy_version 755131 (0.0005) [2023-12-26 20:51:32,970][105692] Updated weights for policy 0, policy_version 754901 (0.0008) [2023-12-26 20:51:32,985][105620] Updated weights for policy 1, policy_version 755141 (0.0005) [2023-12-26 20:51:33,033][105692] Updated weights for policy 0, policy_version 754911 (0.0010) [2023-12-26 20:51:33,602][105620] Updated weights for policy 1, policy_version 755151 (0.0008) [2023-12-26 20:51:33,663][105620] Updated weights for policy 1, policy_version 755161 (0.0009) [2023-12-26 20:51:33,713][105620] Updated weights for policy 1, policy_version 755172 (0.0009) [2023-12-26 20:51:33,770][105692] Updated weights for policy 0, policy_version 754921 (0.0005) [2023-12-26 20:51:33,844][105692] Updated weights for policy 0, policy_version 754931 (0.0005) [2023-12-26 20:51:33,895][105692] Updated weights for policy 0, policy_version 754941 (0.0008) [2023-12-26 20:51:33,953][105692] Updated weights for policy 0, policy_version 754951 (0.0009) [2023-12-26 20:51:34,476][105620] Updated weights for policy 1, policy_version 755183 (0.0009) [2023-12-26 20:51:34,537][105620] Updated weights for policy 1, policy_version 755193 (0.0008) [2023-12-26 20:51:34,588][105620] Updated weights for policy 1, policy_version 755203 (0.0008) [2023-12-26 20:51:34,664][105692] Updated weights for policy 0, policy_version 754961 (0.0008) [2023-12-26 20:51:34,726][105692] Updated weights for policy 0, policy_version 754971 (0.0009) [2023-12-26 20:51:34,795][105692] Updated weights for policy 0, policy_version 754981 (0.0010) [2023-12-26 20:51:35,302][105620] Updated weights for policy 1, policy_version 755213 (0.0008) [2023-12-26 20:51:35,359][105620] Updated weights for policy 1, policy_version 755223 (0.0008) [2023-12-26 20:51:35,425][105620] Updated weights for policy 1, policy_version 755233 (0.0009) [2023-12-26 20:51:35,502][105692] Updated weights for policy 0, policy_version 754991 (0.0009) [2023-12-26 20:51:35,563][105692] Updated weights for policy 0, policy_version 755001 (0.0009) [2023-12-26 20:51:35,620][105692] Updated weights for policy 0, policy_version 755011 (0.0009) [2023-12-26 20:51:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 386678784. Throughput: 0: 9710.7, 1: 10012.2. Samples: 386670496. Policy #0 lag: (min: 2.0, avg: 18.7, max: 34.0) [2023-12-26 20:51:36,062][104569] Avg episode reward: [(0, '9167.525'), (1, '9159.761')] [2023-12-26 20:51:36,138][105620] Updated weights for policy 1, policy_version 755243 (0.0009) [2023-12-26 20:51:36,190][105620] Updated weights for policy 1, policy_version 755253 (0.0008) [2023-12-26 20:51:36,241][105620] Updated weights for policy 1, policy_version 755263 (0.0009) [2023-12-26 20:51:36,380][105692] Updated weights for policy 0, policy_version 755021 (0.0007) [2023-12-26 20:51:36,435][105692] Updated weights for policy 0, policy_version 755031 (0.0008) [2023-12-26 20:51:36,498][105692] Updated weights for policy 0, policy_version 755041 (0.0007) [2023-12-26 20:51:36,985][105620] Updated weights for policy 1, policy_version 755273 (0.0009) [2023-12-26 20:51:37,041][105620] Updated weights for policy 1, policy_version 755283 (0.0010) [2023-12-26 20:51:37,101][105620] Updated weights for policy 1, policy_version 755293 (0.0010) [2023-12-26 20:51:37,161][105620] Updated weights for policy 1, policy_version 755303 (0.0011) [2023-12-26 20:51:37,241][105692] Updated weights for policy 0, policy_version 755051 (0.0007) [2023-12-26 20:51:37,305][105692] Updated weights for policy 0, policy_version 755061 (0.0008) [2023-12-26 20:51:37,373][105692] Updated weights for policy 0, policy_version 755071 (0.0009) [2023-12-26 20:51:37,924][105620] Updated weights for policy 1, policy_version 755313 (0.0011) [2023-12-26 20:51:37,987][105620] Updated weights for policy 1, policy_version 755323 (0.0010) [2023-12-26 20:51:38,047][105620] Updated weights for policy 1, policy_version 755333 (0.0011) [2023-12-26 20:51:38,135][105692] Updated weights for policy 0, policy_version 755081 (0.0009) [2023-12-26 20:51:38,197][105692] Updated weights for policy 0, policy_version 755091 (0.0008) [2023-12-26 20:51:38,262][105692] Updated weights for policy 0, policy_version 755101 (0.0008) [2023-12-26 20:51:38,321][105692] Updated weights for policy 0, policy_version 755111 (0.0008) [2023-12-26 20:51:38,731][105620] Updated weights for policy 1, policy_version 755343 (0.0011) [2023-12-26 20:51:38,790][105620] Updated weights for policy 1, policy_version 755353 (0.0011) [2023-12-26 20:51:38,849][105620] Updated weights for policy 1, policy_version 755363 (0.0010) [2023-12-26 20:51:39,171][105692] Updated weights for policy 0, policy_version 755121 (0.0008) [2023-12-26 20:51:39,234][105692] Updated weights for policy 0, policy_version 755131 (0.0008) [2023-12-26 20:51:39,301][105692] Updated weights for policy 0, policy_version 755141 (0.0008) [2023-12-26 20:51:39,635][105620] Updated weights for policy 1, policy_version 755373 (0.0011) [2023-12-26 20:51:39,689][105620] Updated weights for policy 1, policy_version 755383 (0.0009) [2023-12-26 20:51:39,738][105620] Updated weights for policy 1, policy_version 755393 (0.0010) [2023-12-26 20:51:40,082][105692] Updated weights for policy 0, policy_version 755151 (0.0010) [2023-12-26 20:51:40,143][105692] Updated weights for policy 0, policy_version 755161 (0.0010) [2023-12-26 20:51:40,203][105692] Updated weights for policy 0, policy_version 755171 (0.0010) [2023-12-26 20:51:40,459][105620] Updated weights for policy 1, policy_version 755403 (0.0009) [2023-12-26 20:51:40,490][105586] KL-divergence is very high: 114.2646 [2023-12-26 20:51:40,509][105620] Updated weights for policy 1, policy_version 755413 (0.0005) [2023-12-26 20:51:40,535][105586] KL-divergence is very high: 216.5258 [2023-12-26 20:51:40,563][105620] Updated weights for policy 1, policy_version 755423 (0.0008) [2023-12-26 20:51:40,575][105586] KL-divergence is very high: 248.2971 [2023-12-26 20:51:40,911][105692] Updated weights for policy 0, policy_version 755181 (0.0010) [2023-12-26 20:51:40,966][105692] Updated weights for policy 0, policy_version 755191 (0.0005) [2023-12-26 20:51:41,028][105692] Updated weights for policy 0, policy_version 755201 (0.0006) [2023-12-26 20:51:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 386768896. Throughput: 0: 9650.7, 1: 9960.2. Samples: 386783140. Policy #0 lag: (min: 2.0, avg: 18.7, max: 34.0) [2023-12-26 20:51:41,063][104569] Avg episode reward: [(0, '9166.928'), (1, '8885.922')] [2023-12-26 20:51:41,207][105620] Updated weights for policy 1, policy_version 755433 (0.0010) [2023-12-26 20:51:41,267][105620] Updated weights for policy 1, policy_version 755443 (0.0011) [2023-12-26 20:51:41,334][105620] Updated weights for policy 1, policy_version 755453 (0.0011) [2023-12-26 20:51:41,392][105620] Updated weights for policy 1, policy_version 755463 (0.0010) [2023-12-26 20:51:41,748][105692] Updated weights for policy 0, policy_version 755211 (0.0009) [2023-12-26 20:51:41,806][105692] Updated weights for policy 0, policy_version 755221 (0.0010) [2023-12-26 20:51:41,858][105692] Updated weights for policy 0, policy_version 755231 (0.0011) [2023-12-26 20:51:42,185][105620] Updated weights for policy 1, policy_version 755473 (0.0009) [2023-12-26 20:51:42,245][105620] Updated weights for policy 1, policy_version 755483 (0.0011) [2023-12-26 20:51:42,313][105620] Updated weights for policy 1, policy_version 755493 (0.0010) [2023-12-26 20:51:42,548][105692] Updated weights for policy 0, policy_version 755241 (0.0010) [2023-12-26 20:51:42,615][105692] Updated weights for policy 0, policy_version 755251 (0.0006) [2023-12-26 20:51:42,677][105692] Updated weights for policy 0, policy_version 755261 (0.0006) [2023-12-26 20:51:42,737][105692] Updated weights for policy 0, policy_version 755271 (0.0005) [2023-12-26 20:51:43,030][105620] Updated weights for policy 1, policy_version 755503 (0.0009) [2023-12-26 20:51:43,086][105620] Updated weights for policy 1, policy_version 755513 (0.0008) [2023-12-26 20:51:43,139][105620] Updated weights for policy 1, policy_version 755523 (0.0008) [2023-12-26 20:51:43,318][105692] Updated weights for policy 0, policy_version 755281 (0.0010) [2023-12-26 20:51:43,367][105692] Updated weights for policy 0, policy_version 755291 (0.0010) [2023-12-26 20:51:43,415][105692] Updated weights for policy 0, policy_version 755301 (0.0010) [2023-12-26 20:51:43,857][105620] Updated weights for policy 1, policy_version 755533 (0.0007) [2023-12-26 20:51:43,919][105620] Updated weights for policy 1, policy_version 755543 (0.0005) [2023-12-26 20:51:44,001][105620] Updated weights for policy 1, policy_version 755553 (0.0005) [2023-12-26 20:51:44,050][105692] Updated weights for policy 0, policy_version 755311 (0.0011) [2023-12-26 20:51:44,114][105692] Updated weights for policy 0, policy_version 755321 (0.0010) [2023-12-26 20:51:44,178][105692] Updated weights for policy 0, policy_version 755331 (0.0010) [2023-12-26 20:51:44,534][105620] Updated weights for policy 1, policy_version 755563 (0.0005) [2023-12-26 20:51:44,598][105620] Updated weights for policy 1, policy_version 755573 (0.0007) [2023-12-26 20:51:44,619][105586] KL-divergence is very high: 140.3196 [2023-12-26 20:51:44,666][105620] Updated weights for policy 1, policy_version 755583 (0.0008) [2023-12-26 20:51:44,671][105586] KL-divergence is very high: 163.4376 [2023-12-26 20:51:44,911][105692] Updated weights for policy 0, policy_version 755341 (0.0010) [2023-12-26 20:51:44,977][105692] Updated weights for policy 0, policy_version 755351 (0.0011) [2023-12-26 20:51:45,045][105692] Updated weights for policy 0, policy_version 755361 (0.0011) [2023-12-26 20:51:45,396][105620] Updated weights for policy 1, policy_version 755593 (0.0008) [2023-12-26 20:51:45,448][105620] Updated weights for policy 1, policy_version 755603 (0.0005) [2023-12-26 20:51:45,509][105620] Updated weights for policy 1, policy_version 755613 (0.0005) [2023-12-26 20:51:45,571][105620] Updated weights for policy 1, policy_version 755623 (0.0006) [2023-12-26 20:51:45,795][105692] Updated weights for policy 0, policy_version 755371 (0.0011) [2023-12-26 20:51:45,859][105692] Updated weights for policy 0, policy_version 755381 (0.0007) [2023-12-26 20:51:45,916][105692] Updated weights for policy 0, policy_version 755391 (0.0005) [2023-12-26 20:51:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 386875392. Throughput: 0: 9632.6, 1: 9982.5. Samples: 386841860. Policy #0 lag: (min: 2.0, avg: 18.7, max: 34.0) [2023-12-26 20:51:46,063][104569] Avg episode reward: [(0, '9202.501'), (1, '8892.816')] [2023-12-26 20:51:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000755624_193462272.pth... [2023-12-26 20:51:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000755400_193413120.pth... [2023-12-26 20:51:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000754472_193167360.pth [2023-12-26 20:51:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000754280_193126400.pth [2023-12-26 20:51:46,229][105620] Updated weights for policy 1, policy_version 755633 (0.0007) [2023-12-26 20:51:46,293][105620] Updated weights for policy 1, policy_version 755643 (0.0008) [2023-12-26 20:51:46,344][105620] Updated weights for policy 1, policy_version 755653 (0.0008) [2023-12-26 20:51:46,556][105692] Updated weights for policy 0, policy_version 755401 (0.0006) [2023-12-26 20:51:46,604][105692] Updated weights for policy 0, policy_version 755411 (0.0010) [2023-12-26 20:51:46,660][105692] Updated weights for policy 0, policy_version 755421 (0.0010) [2023-12-26 20:51:46,718][105692] Updated weights for policy 0, policy_version 755431 (0.0011) [2023-12-26 20:51:46,964][105620] Updated weights for policy 1, policy_version 755663 (0.0006) [2023-12-26 20:51:47,026][105620] Updated weights for policy 1, policy_version 755673 (0.0005) [2023-12-26 20:51:47,087][105620] Updated weights for policy 1, policy_version 755683 (0.0009) [2023-12-26 20:51:47,343][105692] Updated weights for policy 0, policy_version 755441 (0.0009) [2023-12-26 20:51:47,400][105692] Updated weights for policy 0, policy_version 755451 (0.0009) [2023-12-26 20:51:47,462][105692] Updated weights for policy 0, policy_version 755461 (0.0009) [2023-12-26 20:51:47,761][105620] Updated weights for policy 1, policy_version 755693 (0.0009) [2023-12-26 20:51:47,810][105620] Updated weights for policy 1, policy_version 755703 (0.0007) [2023-12-26 20:51:47,861][105620] Updated weights for policy 1, policy_version 755713 (0.0005) [2023-12-26 20:51:48,188][105692] Updated weights for policy 0, policy_version 755471 (0.0006) [2023-12-26 20:51:48,241][105692] Updated weights for policy 0, policy_version 755481 (0.0009) [2023-12-26 20:51:48,291][105692] Updated weights for policy 0, policy_version 755492 (0.0009) [2023-12-26 20:51:48,507][105620] Updated weights for policy 1, policy_version 755723 (0.0006) [2023-12-26 20:51:48,574][105620] Updated weights for policy 1, policy_version 755733 (0.0008) [2023-12-26 20:51:48,635][105620] Updated weights for policy 1, policy_version 755743 (0.0006) [2023-12-26 20:51:49,079][105692] Updated weights for policy 0, policy_version 755502 (0.0009) [2023-12-26 20:51:49,129][105692] Updated weights for policy 0, policy_version 755512 (0.0008) [2023-12-26 20:51:49,183][105692] Updated weights for policy 0, policy_version 755523 (0.0010) [2023-12-26 20:51:49,230][105620] Updated weights for policy 1, policy_version 755753 (0.0006) [2023-12-26 20:51:49,290][105620] Updated weights for policy 1, policy_version 755763 (0.0006) [2023-12-26 20:51:49,356][105620] Updated weights for policy 1, policy_version 755773 (0.0010) [2023-12-26 20:51:49,420][105620] Updated weights for policy 1, policy_version 755783 (0.0008) [2023-12-26 20:51:49,953][105692] Updated weights for policy 0, policy_version 755534 (0.0010) [2023-12-26 20:51:50,006][105692] Updated weights for policy 0, policy_version 755544 (0.0009) [2023-12-26 20:51:50,069][105692] Updated weights for policy 0, policy_version 755554 (0.0009) [2023-12-26 20:51:50,176][105620] Updated weights for policy 1, policy_version 755793 (0.0007) [2023-12-26 20:51:50,236][105620] Updated weights for policy 1, policy_version 755803 (0.0007) [2023-12-26 20:51:50,301][105620] Updated weights for policy 1, policy_version 755813 (0.0007) [2023-12-26 20:51:50,906][105692] Updated weights for policy 0, policy_version 755564 (0.0009) [2023-12-26 20:51:50,912][105620] Updated weights for policy 1, policy_version 755823 (0.0008) [2023-12-26 20:51:50,961][105692] Updated weights for policy 0, policy_version 755574 (0.0009) [2023-12-26 20:51:50,975][105620] Updated weights for policy 1, policy_version 755833 (0.0006) [2023-12-26 20:51:51,008][105692] Updated weights for policy 0, policy_version 755584 (0.0009) [2023-12-26 20:51:51,040][105620] Updated weights for policy 1, policy_version 755843 (0.0007) [2023-12-26 20:51:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 386973696. Throughput: 0: 9641.3, 1: 9963.4. Samples: 386963304. Policy #0 lag: (min: 2.0, avg: 18.7, max: 34.0) [2023-12-26 20:51:51,063][104569] Avg episode reward: [(0, '9099.065'), (1, '8982.491')] [2023-12-26 20:51:51,786][105692] Updated weights for policy 0, policy_version 755594 (0.0009) [2023-12-26 20:51:51,793][105620] Updated weights for policy 1, policy_version 755853 (0.0008) [2023-12-26 20:51:51,851][105692] Updated weights for policy 0, policy_version 755604 (0.0008) [2023-12-26 20:51:51,858][105620] Updated weights for policy 1, policy_version 755863 (0.0008) [2023-12-26 20:51:51,909][105692] Updated weights for policy 0, policy_version 755614 (0.0005) [2023-12-26 20:51:51,923][105620] Updated weights for policy 1, policy_version 755873 (0.0009) [2023-12-26 20:51:51,973][105692] Updated weights for policy 0, policy_version 755624 (0.0005) [2023-12-26 20:51:52,571][105692] Updated weights for policy 0, policy_version 755634 (0.0008) [2023-12-26 20:51:52,633][105692] Updated weights for policy 0, policy_version 755644 (0.0009) [2023-12-26 20:51:52,692][105692] Updated weights for policy 0, policy_version 755654 (0.0009) [2023-12-26 20:51:52,731][105620] Updated weights for policy 1, policy_version 755883 (0.0008) [2023-12-26 20:51:52,792][105620] Updated weights for policy 1, policy_version 755893 (0.0008) [2023-12-26 20:51:52,861][105620] Updated weights for policy 1, policy_version 755903 (0.0009) [2023-12-26 20:51:53,422][105692] Updated weights for policy 0, policy_version 755664 (0.0009) [2023-12-26 20:51:53,481][105692] Updated weights for policy 0, policy_version 755674 (0.0009) [2023-12-26 20:51:53,533][105692] Updated weights for policy 0, policy_version 755684 (0.0009) [2023-12-26 20:51:53,582][105620] Updated weights for policy 1, policy_version 755913 (0.0009) [2023-12-26 20:51:53,628][105620] Updated weights for policy 1, policy_version 755923 (0.0008) [2023-12-26 20:51:53,682][105620] Updated weights for policy 1, policy_version 755933 (0.0008) [2023-12-26 20:51:53,733][105620] Updated weights for policy 1, policy_version 755943 (0.0009) [2023-12-26 20:51:54,232][105692] Updated weights for policy 0, policy_version 755694 (0.0009) [2023-12-26 20:51:54,283][105692] Updated weights for policy 0, policy_version 755704 (0.0008) [2023-12-26 20:51:54,337][105692] Updated weights for policy 0, policy_version 755714 (0.0009) [2023-12-26 20:51:54,545][105620] Updated weights for policy 1, policy_version 755953 (0.0008) [2023-12-26 20:51:54,595][105620] Updated weights for policy 1, policy_version 755963 (0.0009) [2023-12-26 20:51:54,642][105620] Updated weights for policy 1, policy_version 755973 (0.0008) [2023-12-26 20:51:55,045][105692] Updated weights for policy 0, policy_version 755724 (0.0008) [2023-12-26 20:51:55,109][105692] Updated weights for policy 0, policy_version 755734 (0.0006) [2023-12-26 20:51:55,165][105692] Updated weights for policy 0, policy_version 755744 (0.0005) [2023-12-26 20:51:55,508][105620] Updated weights for policy 1, policy_version 755983 (0.0008) [2023-12-26 20:51:55,561][105620] Updated weights for policy 1, policy_version 755993 (0.0009) [2023-12-26 20:51:55,623][105620] Updated weights for policy 1, policy_version 756003 (0.0007) [2023-12-26 20:51:55,722][105692] Updated weights for policy 0, policy_version 755754 (0.0005) [2023-12-26 20:51:55,780][105692] Updated weights for policy 0, policy_version 755764 (0.0005) [2023-12-26 20:51:55,833][105692] Updated weights for policy 0, policy_version 755774 (0.0005) [2023-12-26 20:51:55,892][105692] Updated weights for policy 0, policy_version 755784 (0.0005) [2023-12-26 20:51:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 387072000. Throughput: 0: 9727.2, 1: 9868.5. Samples: 387077364. Policy #0 lag: (min: 2.0, avg: 18.7, max: 34.0) [2023-12-26 20:51:56,062][104569] Avg episode reward: [(0, '9244.316'), (1, '8984.007')] [2023-12-26 20:51:56,425][105620] Updated weights for policy 1, policy_version 756013 (0.0009) [2023-12-26 20:51:56,454][105692] Updated weights for policy 0, policy_version 755794 (0.0008) [2023-12-26 20:51:56,475][105620] Updated weights for policy 1, policy_version 756023 (0.0008) [2023-12-26 20:51:56,520][105692] Updated weights for policy 0, policy_version 755804 (0.0005) [2023-12-26 20:51:56,527][105620] Updated weights for policy 1, policy_version 756033 (0.0008) [2023-12-26 20:51:56,567][105692] Updated weights for policy 0, policy_version 755814 (0.0005) [2023-12-26 20:51:57,185][105692] Updated weights for policy 0, policy_version 755824 (0.0006) [2023-12-26 20:51:57,232][105692] Updated weights for policy 0, policy_version 755834 (0.0008) [2023-12-26 20:51:57,279][105692] Updated weights for policy 0, policy_version 755844 (0.0009) [2023-12-26 20:51:57,326][105620] Updated weights for policy 1, policy_version 756043 (0.0009) [2023-12-26 20:51:57,373][105620] Updated weights for policy 1, policy_version 756053 (0.0009) [2023-12-26 20:51:57,424][105620] Updated weights for policy 1, policy_version 756063 (0.0008) [2023-12-26 20:51:57,941][105692] Updated weights for policy 0, policy_version 755854 (0.0005) [2023-12-26 20:51:57,984][105692] Updated weights for policy 0, policy_version 755864 (0.0005) [2023-12-26 20:51:58,048][105692] Updated weights for policy 0, policy_version 755874 (0.0006) [2023-12-26 20:51:58,235][105620] Updated weights for policy 1, policy_version 756073 (0.0008) [2023-12-26 20:51:58,287][105620] Updated weights for policy 1, policy_version 756083 (0.0008) [2023-12-26 20:51:58,344][105620] Updated weights for policy 1, policy_version 756093 (0.0008) [2023-12-26 20:51:58,413][105620] Updated weights for policy 1, policy_version 756103 (0.0009) [2023-12-26 20:51:58,771][105692] Updated weights for policy 0, policy_version 755884 (0.0007) [2023-12-26 20:51:58,841][105692] Updated weights for policy 0, policy_version 755894 (0.0009) [2023-12-26 20:51:58,909][105692] Updated weights for policy 0, policy_version 755904 (0.0008) [2023-12-26 20:51:59,332][105620] Updated weights for policy 1, policy_version 756113 (0.0008) [2023-12-26 20:51:59,407][105620] Updated weights for policy 1, policy_version 756123 (0.0008) [2023-12-26 20:51:59,465][105620] Updated weights for policy 1, policy_version 756133 (0.0008) [2023-12-26 20:51:59,742][105692] Updated weights for policy 0, policy_version 755914 (0.0011) [2023-12-26 20:51:59,794][105692] Updated weights for policy 0, policy_version 755924 (0.0010) [2023-12-26 20:51:59,853][105692] Updated weights for policy 0, policy_version 755934 (0.0011) [2023-12-26 20:51:59,901][105692] Updated weights for policy 0, policy_version 755944 (0.0009) [2023-12-26 20:52:00,229][105620] Updated weights for policy 1, policy_version 756143 (0.0008) [2023-12-26 20:52:00,273][105620] Updated weights for policy 1, policy_version 756153 (0.0007) [2023-12-26 20:52:00,322][105620] Updated weights for policy 1, policy_version 756163 (0.0008) [2023-12-26 20:52:00,664][105692] Updated weights for policy 0, policy_version 755954 (0.0010) [2023-12-26 20:52:00,721][105692] Updated weights for policy 0, policy_version 755964 (0.0010) [2023-12-26 20:52:00,781][105692] Updated weights for policy 0, policy_version 755974 (0.0010) [2023-12-26 20:52:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 387162112. Throughput: 0: 9799.7, 1: 9785.8. Samples: 387136168. Policy #0 lag: (min: 2.0, avg: 18.7, max: 34.0) [2023-12-26 20:52:01,063][104569] Avg episode reward: [(0, '9348.211'), (1, '8797.564')] [2023-12-26 20:52:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000755976_193560576.pth... [2023-12-26 20:52:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000754824_193265664.pth [2023-12-26 20:52:01,094][105620] Updated weights for policy 1, policy_version 756173 (0.0008) [2023-12-26 20:52:01,158][105620] Updated weights for policy 1, policy_version 756183 (0.0008) [2023-12-26 20:52:01,209][105620] Updated weights for policy 1, policy_version 756193 (0.0008) [2023-12-26 20:52:01,243][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000756200_193609728.pth... [2023-12-26 20:52:01,248][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000755048_193314816.pth [2023-12-26 20:52:01,528][105692] Updated weights for policy 0, policy_version 755984 (0.0008) [2023-12-26 20:52:01,584][105692] Updated weights for policy 0, policy_version 755994 (0.0008) [2023-12-26 20:52:01,646][105692] Updated weights for policy 0, policy_version 756004 (0.0008) [2023-12-26 20:52:02,000][105620] Updated weights for policy 1, policy_version 756203 (0.0009) [2023-12-26 20:52:02,060][105620] Updated weights for policy 1, policy_version 756213 (0.0009) [2023-12-26 20:52:02,114][105620] Updated weights for policy 1, policy_version 756223 (0.0009) [2023-12-26 20:52:02,279][105692] Updated weights for policy 0, policy_version 756014 (0.0009) [2023-12-26 20:52:02,340][105692] Updated weights for policy 0, policy_version 756024 (0.0010) [2023-12-26 20:52:02,395][105692] Updated weights for policy 0, policy_version 756034 (0.0010) [2023-12-26 20:52:02,937][105620] Updated weights for policy 1, policy_version 756233 (0.0009) [2023-12-26 20:52:02,998][105620] Updated weights for policy 1, policy_version 756244 (0.0010) [2023-12-26 20:52:03,033][105692] Updated weights for policy 0, policy_version 756044 (0.0008) [2023-12-26 20:52:03,052][105620] Updated weights for policy 1, policy_version 756254 (0.0008) [2023-12-26 20:52:03,090][105692] Updated weights for policy 0, policy_version 756054 (0.0006) [2023-12-26 20:52:03,097][105620] Updated weights for policy 1, policy_version 756264 (0.0007) [2023-12-26 20:52:03,150][105692] Updated weights for policy 0, policy_version 756064 (0.0009) [2023-12-26 20:52:03,721][105692] Updated weights for policy 0, policy_version 756074 (0.0008) [2023-12-26 20:52:03,776][105692] Updated weights for policy 0, policy_version 756084 (0.0005) [2023-12-26 20:52:03,831][105692] Updated weights for policy 0, policy_version 756094 (0.0005) [2023-12-26 20:52:03,894][105692] Updated weights for policy 0, policy_version 756104 (0.0007) [2023-12-26 20:52:03,963][105620] Updated weights for policy 1, policy_version 756274 (0.0009) [2023-12-26 20:52:04,022][105620] Updated weights for policy 1, policy_version 756284 (0.0010) [2023-12-26 20:52:04,081][105620] Updated weights for policy 1, policy_version 756294 (0.0008) [2023-12-26 20:52:04,526][105692] Updated weights for policy 0, policy_version 756114 (0.0005) [2023-12-26 20:52:04,573][105692] Updated weights for policy 0, policy_version 756124 (0.0005) [2023-12-26 20:52:04,620][105692] Updated weights for policy 0, policy_version 756134 (0.0005) [2023-12-26 20:52:04,901][105620] Updated weights for policy 1, policy_version 756304 (0.0008) [2023-12-26 20:52:04,956][105620] Updated weights for policy 1, policy_version 756316 (0.0010) [2023-12-26 20:52:05,009][105620] Updated weights for policy 1, policy_version 756327 (0.0010) [2023-12-26 20:52:05,229][105692] Updated weights for policy 0, policy_version 756144 (0.0010) [2023-12-26 20:52:05,285][105692] Updated weights for policy 0, policy_version 756154 (0.0010) [2023-12-26 20:52:05,350][105692] Updated weights for policy 0, policy_version 756164 (0.0011) [2023-12-26 20:52:05,786][105620] Updated weights for policy 1, policy_version 756337 (0.0008) [2023-12-26 20:52:05,833][105620] Updated weights for policy 1, policy_version 756347 (0.0008) [2023-12-26 20:52:05,877][105620] Updated weights for policy 1, policy_version 756357 (0.0008) [2023-12-26 20:52:06,046][105692] Updated weights for policy 0, policy_version 756174 (0.0007) [2023-12-26 20:52:06,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 387260416. Throughput: 0: 9815.9, 1: 9611.5. Samples: 387248668. Policy #0 lag: (min: 2.0, avg: 18.7, max: 34.0) [2023-12-26 20:52:06,063][104569] Avg episode reward: [(0, '8908.181'), (1, '9063.780')] [2023-12-26 20:52:06,100][105692] Updated weights for policy 0, policy_version 756184 (0.0006) [2023-12-26 20:52:06,171][105692] Updated weights for policy 0, policy_version 756194 (0.0011) [2023-12-26 20:52:06,689][105620] Updated weights for policy 1, policy_version 756367 (0.0008) [2023-12-26 20:52:06,753][105620] Updated weights for policy 1, policy_version 756377 (0.0008) [2023-12-26 20:52:06,808][105620] Updated weights for policy 1, policy_version 756387 (0.0008) [2023-12-26 20:52:06,892][105692] Updated weights for policy 0, policy_version 756204 (0.0011) [2023-12-26 20:52:06,948][105692] Updated weights for policy 0, policy_version 756214 (0.0011) [2023-12-26 20:52:07,018][105692] Updated weights for policy 0, policy_version 756224 (0.0011) [2023-12-26 20:52:07,583][105620] Updated weights for policy 1, policy_version 756397 (0.0008) [2023-12-26 20:52:07,636][105620] Updated weights for policy 1, policy_version 756407 (0.0009) [2023-12-26 20:52:07,674][105692] Updated weights for policy 0, policy_version 756234 (0.0010) [2023-12-26 20:52:07,686][105620] Updated weights for policy 1, policy_version 756417 (0.0009) [2023-12-26 20:52:07,740][105692] Updated weights for policy 0, policy_version 756244 (0.0005) [2023-12-26 20:52:07,803][105692] Updated weights for policy 0, policy_version 756254 (0.0009) [2023-12-26 20:52:07,866][105692] Updated weights for policy 0, policy_version 756264 (0.0009) [2023-12-26 20:52:08,464][105620] Updated weights for policy 1, policy_version 756427 (0.0008) [2023-12-26 20:52:08,530][105620] Updated weights for policy 1, policy_version 756437 (0.0007) [2023-12-26 20:52:08,594][105620] Updated weights for policy 1, policy_version 756447 (0.0006) [2023-12-26 20:52:08,600][105692] Updated weights for policy 0, policy_version 756274 (0.0008) [2023-12-26 20:52:08,662][105692] Updated weights for policy 0, policy_version 756284 (0.0007) [2023-12-26 20:52:08,725][105692] Updated weights for policy 0, policy_version 756294 (0.0009) [2023-12-26 20:52:09,368][105620] Updated weights for policy 1, policy_version 756457 (0.0008) [2023-12-26 20:52:09,402][105692] Updated weights for policy 0, policy_version 756304 (0.0008) [2023-12-26 20:52:09,439][105620] Updated weights for policy 1, policy_version 756467 (0.0008) [2023-12-26 20:52:09,469][105692] Updated weights for policy 0, policy_version 756314 (0.0006) [2023-12-26 20:52:09,504][105620] Updated weights for policy 1, policy_version 756477 (0.0008) [2023-12-26 20:52:09,531][105692] Updated weights for policy 0, policy_version 756324 (0.0006) [2023-12-26 20:52:09,562][105620] Updated weights for policy 1, policy_version 756487 (0.0007) [2023-12-26 20:52:10,309][105692] Updated weights for policy 0, policy_version 756334 (0.0008) [2023-12-26 20:52:10,355][105692] Updated weights for policy 0, policy_version 756344 (0.0007) [2023-12-26 20:52:10,360][105620] Updated weights for policy 1, policy_version 756497 (0.0008) [2023-12-26 20:52:10,412][105692] Updated weights for policy 0, policy_version 756354 (0.0008) [2023-12-26 20:52:10,420][105620] Updated weights for policy 1, policy_version 756507 (0.0007) [2023-12-26 20:52:10,483][105620] Updated weights for policy 1, policy_version 756517 (0.0008) [2023-12-26 20:52:11,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 387350528. Throughput: 0: 9795.0, 1: 9498.5. Samples: 387361396. Policy #0 lag: (min: 2.0, avg: 18.7, max: 34.0) [2023-12-26 20:52:11,062][104569] Avg episode reward: [(0, '8907.815'), (1, '9247.386')] [2023-12-26 20:52:11,182][105692] Updated weights for policy 0, policy_version 756364 (0.0008) [2023-12-26 20:52:11,218][105620] Updated weights for policy 1, policy_version 756527 (0.0006) [2023-12-26 20:52:11,243][105692] Updated weights for policy 0, policy_version 756374 (0.0009) [2023-12-26 20:52:11,284][105620] Updated weights for policy 1, policy_version 756537 (0.0007) [2023-12-26 20:52:11,294][105692] Updated weights for policy 0, policy_version 756384 (0.0008) [2023-12-26 20:52:11,358][105620] Updated weights for policy 1, policy_version 756547 (0.0008) [2023-12-26 20:52:11,990][105692] Updated weights for policy 0, policy_version 756394 (0.0007) [2023-12-26 20:52:12,021][105620] Updated weights for policy 1, policy_version 756557 (0.0009) [2023-12-26 20:52:12,039][105692] Updated weights for policy 0, policy_version 756404 (0.0011) [2023-12-26 20:52:12,077][105620] Updated weights for policy 1, policy_version 756567 (0.0011) [2023-12-26 20:52:12,098][105692] Updated weights for policy 0, policy_version 756414 (0.0011) [2023-12-26 20:52:12,132][105620] Updated weights for policy 1, policy_version 756577 (0.0010) [2023-12-26 20:52:12,153][105692] Updated weights for policy 0, policy_version 756424 (0.0008) [2023-12-26 20:52:12,810][105620] Updated weights for policy 1, policy_version 756587 (0.0010) [2023-12-26 20:52:12,869][105620] Updated weights for policy 1, policy_version 756597 (0.0006) [2023-12-26 20:52:12,898][105692] Updated weights for policy 0, policy_version 756434 (0.0011) [2023-12-26 20:52:12,936][105620] Updated weights for policy 1, policy_version 756607 (0.0008) [2023-12-26 20:52:12,954][105692] Updated weights for policy 0, policy_version 756444 (0.0011) [2023-12-26 20:52:13,014][105692] Updated weights for policy 0, policy_version 756454 (0.0011) [2023-12-26 20:52:13,562][105620] Updated weights for policy 1, policy_version 756617 (0.0008) [2023-12-26 20:52:13,616][105692] Updated weights for policy 0, policy_version 756464 (0.0007) [2023-12-26 20:52:13,627][105620] Updated weights for policy 1, policy_version 756627 (0.0005) [2023-12-26 20:52:13,681][105692] Updated weights for policy 0, policy_version 756474 (0.0006) [2023-12-26 20:52:13,685][105620] Updated weights for policy 1, policy_version 756637 (0.0006) [2023-12-26 20:52:13,750][105692] Updated weights for policy 0, policy_version 756484 (0.0006) [2023-12-26 20:52:13,754][105620] Updated weights for policy 1, policy_version 756647 (0.0006) [2023-12-26 20:52:14,263][105620] Updated weights for policy 1, policy_version 756657 (0.0007) [2023-12-26 20:52:14,327][105620] Updated weights for policy 1, policy_version 756667 (0.0007) [2023-12-26 20:52:14,377][105692] Updated weights for policy 0, policy_version 756494 (0.0006) [2023-12-26 20:52:14,386][105620] Updated weights for policy 1, policy_version 756677 (0.0005) [2023-12-26 20:52:14,441][105692] Updated weights for policy 0, policy_version 756504 (0.0006) [2023-12-26 20:52:14,509][105692] Updated weights for policy 0, policy_version 756514 (0.0006) [2023-12-26 20:52:15,071][105620] Updated weights for policy 1, policy_version 756687 (0.0007) [2023-12-26 20:52:15,120][105620] Updated weights for policy 1, policy_version 756697 (0.0009) [2023-12-26 20:52:15,167][105620] Updated weights for policy 1, policy_version 756707 (0.0007) [2023-12-26 20:52:15,193][105692] Updated weights for policy 0, policy_version 756524 (0.0008) [2023-12-26 20:52:15,256][105692] Updated weights for policy 0, policy_version 756534 (0.0009) [2023-12-26 20:52:15,316][105692] Updated weights for policy 0, policy_version 756544 (0.0009) [2023-12-26 20:52:15,947][105620] Updated weights for policy 1, policy_version 756717 (0.0006) [2023-12-26 20:52:15,969][105692] Updated weights for policy 0, policy_version 756554 (0.0009) [2023-12-26 20:52:16,000][105620] Updated weights for policy 1, policy_version 756727 (0.0005) [2023-12-26 20:52:16,021][105692] Updated weights for policy 0, policy_version 756564 (0.0010) [2023-12-26 20:52:16,053][105620] Updated weights for policy 1, policy_version 756737 (0.0006) [2023-12-26 20:52:16,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 387448832. Throughput: 0: 9818.9, 1: 9505.5. Samples: 387422940. Policy #0 lag: (min: 2.0, avg: 18.7, max: 34.0) [2023-12-26 20:52:16,062][104569] Avg episode reward: [(0, '9088.383'), (1, '8985.523')] [2023-12-26 20:52:16,079][105692] Updated weights for policy 0, policy_version 756574 (0.0010) [2023-12-26 20:52:16,091][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000756744_193748992.pth... [2023-12-26 20:52:16,095][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000755624_193462272.pth [2023-12-26 20:52:16,129][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000756584_193716224.pth... [2023-12-26 20:52:16,131][105692] Updated weights for policy 0, policy_version 756584 (0.0010) [2023-12-26 20:52:16,132][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000755400_193413120.pth [2023-12-26 20:52:16,652][105620] Updated weights for policy 1, policy_version 756747 (0.0007) [2023-12-26 20:52:16,696][105620] Updated weights for policy 1, policy_version 756757 (0.0010) [2023-12-26 20:52:16,745][105620] Updated weights for policy 1, policy_version 756767 (0.0005) [2023-12-26 20:52:16,826][105692] Updated weights for policy 0, policy_version 756594 (0.0005) [2023-12-26 20:52:16,878][105692] Updated weights for policy 0, policy_version 756604 (0.0010) [2023-12-26 20:52:16,928][105692] Updated weights for policy 0, policy_version 756614 (0.0009) [2023-12-26 20:52:17,348][105620] Updated weights for policy 1, policy_version 756777 (0.0005) [2023-12-26 20:52:17,405][105620] Updated weights for policy 1, policy_version 756787 (0.0005) [2023-12-26 20:52:17,454][105620] Updated weights for policy 1, policy_version 756797 (0.0005) [2023-12-26 20:52:17,505][105620] Updated weights for policy 1, policy_version 756807 (0.0005) [2023-12-26 20:52:17,593][105692] Updated weights for policy 0, policy_version 756624 (0.0008) [2023-12-26 20:52:17,654][105692] Updated weights for policy 0, policy_version 756634 (0.0009) [2023-12-26 20:52:17,712][105692] Updated weights for policy 0, policy_version 756645 (0.0011) [2023-12-26 20:52:18,053][105620] Updated weights for policy 1, policy_version 756817 (0.0005) [2023-12-26 20:52:18,109][105620] Updated weights for policy 1, policy_version 756827 (0.0007) [2023-12-26 20:52:18,157][105620] Updated weights for policy 1, policy_version 756837 (0.0009) [2023-12-26 20:52:18,565][105692] Updated weights for policy 0, policy_version 756655 (0.0008) [2023-12-26 20:52:18,625][105692] Updated weights for policy 0, policy_version 756665 (0.0008) [2023-12-26 20:52:18,687][105692] Updated weights for policy 0, policy_version 756675 (0.0008) [2023-12-26 20:52:18,873][105620] Updated weights for policy 1, policy_version 756847 (0.0009) [2023-12-26 20:52:18,933][105620] Updated weights for policy 1, policy_version 756857 (0.0007) [2023-12-26 20:52:18,991][105620] Updated weights for policy 1, policy_version 756867 (0.0008) [2023-12-26 20:52:19,512][105692] Updated weights for policy 0, policy_version 756685 (0.0008) [2023-12-26 20:52:19,566][105692] Updated weights for policy 0, policy_version 756695 (0.0009) [2023-12-26 20:52:19,600][105620] Updated weights for policy 1, policy_version 756877 (0.0009) [2023-12-26 20:52:19,631][105692] Updated weights for policy 0, policy_version 756705 (0.0006) [2023-12-26 20:52:19,659][105620] Updated weights for policy 1, policy_version 756887 (0.0007) [2023-12-26 20:52:19,728][105620] Updated weights for policy 1, policy_version 756897 (0.0007) [2023-12-26 20:52:20,330][105620] Updated weights for policy 1, policy_version 756907 (0.0006) [2023-12-26 20:52:20,387][105620] Updated weights for policy 1, policy_version 756917 (0.0006) [2023-12-26 20:52:20,393][105692] Updated weights for policy 0, policy_version 756715 (0.0007) [2023-12-26 20:52:20,444][105620] Updated weights for policy 1, policy_version 756927 (0.0006) [2023-12-26 20:52:20,450][105692] Updated weights for policy 0, policy_version 756725 (0.0009) [2023-12-26 20:52:20,513][105692] Updated weights for policy 0, policy_version 756735 (0.0009) [2023-12-26 20:52:21,041][105620] Updated weights for policy 1, policy_version 756937 (0.0006) [2023-12-26 20:52:21,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 387555328. Throughput: 0: 9841.2, 1: 9608.6. Samples: 387545736. Policy #0 lag: (min: 2.0, avg: 18.7, max: 34.0) [2023-12-26 20:52:21,063][104569] Avg episode reward: [(0, '9168.166'), (1, '8894.211')] [2023-12-26 20:52:21,108][105620] Updated weights for policy 1, policy_version 756947 (0.0009) [2023-12-26 20:52:21,176][105620] Updated weights for policy 1, policy_version 756957 (0.0010) [2023-12-26 20:52:21,243][105620] Updated weights for policy 1, policy_version 756967 (0.0011) [2023-12-26 20:52:21,265][105692] Updated weights for policy 0, policy_version 756745 (0.0014) [2023-12-26 20:52:21,318][105692] Updated weights for policy 0, policy_version 756755 (0.0008) [2023-12-26 20:52:21,383][105692] Updated weights for policy 0, policy_version 756765 (0.0008) [2023-12-26 20:52:21,440][105692] Updated weights for policy 0, policy_version 756775 (0.0008) [2023-12-26 20:52:22,026][105620] Updated weights for policy 1, policy_version 756977 (0.0006) [2023-12-26 20:52:22,086][105620] Updated weights for policy 1, policy_version 756987 (0.0006) [2023-12-26 20:52:22,145][105620] Updated weights for policy 1, policy_version 756997 (0.0008) [2023-12-26 20:52:22,248][105692] Updated weights for policy 0, policy_version 756785 (0.0010) [2023-12-26 20:52:22,306][105692] Updated weights for policy 0, policy_version 756795 (0.0010) [2023-12-26 20:52:22,367][105692] Updated weights for policy 0, policy_version 756805 (0.0008) [2023-12-26 20:52:22,701][105620] Updated weights for policy 1, policy_version 757007 (0.0009) [2023-12-26 20:52:22,768][105620] Updated weights for policy 1, policy_version 757017 (0.0010) [2023-12-26 20:52:22,829][105620] Updated weights for policy 1, policy_version 757027 (0.0010) [2023-12-26 20:52:23,126][105692] Updated weights for policy 0, policy_version 756815 (0.0009) [2023-12-26 20:52:23,175][105692] Updated weights for policy 0, policy_version 756825 (0.0009) [2023-12-26 20:52:23,237][105692] Updated weights for policy 0, policy_version 756835 (0.0009) [2023-12-26 20:52:23,549][105620] Updated weights for policy 1, policy_version 757037 (0.0010) [2023-12-26 20:52:23,616][105620] Updated weights for policy 1, policy_version 757047 (0.0010) [2023-12-26 20:52:23,683][105620] Updated weights for policy 1, policy_version 757057 (0.0009) [2023-12-26 20:52:23,911][105692] Updated weights for policy 0, policy_version 756845 (0.0009) [2023-12-26 20:52:23,973][105692] Updated weights for policy 0, policy_version 756855 (0.0009) [2023-12-26 20:52:24,036][105692] Updated weights for policy 0, policy_version 756865 (0.0005) [2023-12-26 20:52:24,538][105620] Updated weights for policy 1, policy_version 757067 (0.0010) [2023-12-26 20:52:24,586][105692] Updated weights for policy 0, policy_version 756875 (0.0006) [2023-12-26 20:52:24,592][105620] Updated weights for policy 1, policy_version 757077 (0.0006) [2023-12-26 20:52:24,639][105692] Updated weights for policy 0, policy_version 756885 (0.0008) [2023-12-26 20:52:24,655][105620] Updated weights for policy 1, policy_version 757087 (0.0006) [2023-12-26 20:52:24,689][105692] Updated weights for policy 0, policy_version 756895 (0.0007) [2023-12-26 20:52:25,202][105620] Updated weights for policy 1, policy_version 757097 (0.0006) [2023-12-26 20:52:25,269][105620] Updated weights for policy 1, policy_version 757107 (0.0007) [2023-12-26 20:52:25,336][105620] Updated weights for policy 1, policy_version 757117 (0.0005) [2023-12-26 20:52:25,400][105620] Updated weights for policy 1, policy_version 757127 (0.0005) [2023-12-26 20:52:25,449][105692] Updated weights for policy 0, policy_version 756905 (0.0006) [2023-12-26 20:52:25,500][105692] Updated weights for policy 0, policy_version 756915 (0.0009) [2023-12-26 20:52:25,559][105692] Updated weights for policy 0, policy_version 756925 (0.0009) [2023-12-26 20:52:25,615][105692] Updated weights for policy 0, policy_version 756935 (0.0007) [2023-12-26 20:52:25,982][105620] Updated weights for policy 1, policy_version 757137 (0.0005) [2023-12-26 20:52:26,048][105620] Updated weights for policy 1, policy_version 757147 (0.0005) [2023-12-26 20:52:26,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 387653632. Throughput: 0: 9891.2, 1: 9709.4. Samples: 387665168. Policy #0 lag: (min: 2.0, avg: 18.7, max: 34.0) [2023-12-26 20:52:26,063][104569] Avg episode reward: [(0, '9167.161'), (1, '8882.949')] [2023-12-26 20:52:26,114][105620] Updated weights for policy 1, policy_version 757157 (0.0005) [2023-12-26 20:52:26,222][105692] Updated weights for policy 0, policy_version 756945 (0.0005) [2023-12-26 20:52:26,283][105692] Updated weights for policy 0, policy_version 756955 (0.0009) [2023-12-26 20:52:26,341][105692] Updated weights for policy 0, policy_version 756965 (0.0009) [2023-12-26 20:52:26,623][105620] Updated weights for policy 1, policy_version 757167 (0.0005) [2023-12-26 20:52:26,683][105620] Updated weights for policy 1, policy_version 757177 (0.0005) [2023-12-26 20:52:26,751][105620] Updated weights for policy 1, policy_version 757187 (0.0005) [2023-12-26 20:52:27,013][105692] Updated weights for policy 0, policy_version 756975 (0.0010) [2023-12-26 20:52:27,059][105692] Updated weights for policy 0, policy_version 756985 (0.0009) [2023-12-26 20:52:27,104][105692] Updated weights for policy 0, policy_version 756995 (0.0008) [2023-12-26 20:52:27,347][105620] Updated weights for policy 1, policy_version 757197 (0.0007) [2023-12-26 20:52:27,396][105620] Updated weights for policy 1, policy_version 757207 (0.0005) [2023-12-26 20:52:27,455][105620] Updated weights for policy 1, policy_version 757217 (0.0005) [2023-12-26 20:52:27,820][105692] Updated weights for policy 0, policy_version 757005 (0.0007) [2023-12-26 20:52:27,882][105692] Updated weights for policy 0, policy_version 757015 (0.0006) [2023-12-26 20:52:27,927][105692] Updated weights for policy 0, policy_version 757025 (0.0010) [2023-12-26 20:52:28,203][105620] Updated weights for policy 1, policy_version 757227 (0.0007) [2023-12-26 20:52:28,257][105620] Updated weights for policy 1, policy_version 757237 (0.0007) [2023-12-26 20:52:28,308][105620] Updated weights for policy 1, policy_version 757247 (0.0008) [2023-12-26 20:52:28,581][105692] Updated weights for policy 0, policy_version 757035 (0.0010) [2023-12-26 20:52:28,629][105692] Updated weights for policy 0, policy_version 757045 (0.0010) [2023-12-26 20:52:28,686][105692] Updated weights for policy 0, policy_version 757055 (0.0010) [2023-12-26 20:52:29,046][105620] Updated weights for policy 1, policy_version 757257 (0.0006) [2023-12-26 20:52:29,106][105620] Updated weights for policy 1, policy_version 757267 (0.0008) [2023-12-26 20:52:29,170][105620] Updated weights for policy 1, policy_version 757277 (0.0008) [2023-12-26 20:52:29,236][105620] Updated weights for policy 1, policy_version 757287 (0.0007) [2023-12-26 20:52:29,287][105692] Updated weights for policy 0, policy_version 757065 (0.0009) [2023-12-26 20:52:29,344][105692] Updated weights for policy 0, policy_version 757075 (0.0008) [2023-12-26 20:52:29,399][105692] Updated weights for policy 0, policy_version 757085 (0.0008) [2023-12-26 20:52:29,447][105692] Updated weights for policy 0, policy_version 757095 (0.0008) [2023-12-26 20:52:29,868][105620] Updated weights for policy 1, policy_version 757297 (0.0008) [2023-12-26 20:52:29,937][105620] Updated weights for policy 1, policy_version 757307 (0.0008) [2023-12-26 20:52:30,001][105620] Updated weights for policy 1, policy_version 757317 (0.0008) [2023-12-26 20:52:30,305][105692] Updated weights for policy 0, policy_version 757105 (0.0009) [2023-12-26 20:52:30,360][105692] Updated weights for policy 0, policy_version 757115 (0.0008) [2023-12-26 20:52:30,416][105692] Updated weights for policy 0, policy_version 757125 (0.0008) [2023-12-26 20:52:30,717][105620] Updated weights for policy 1, policy_version 757327 (0.0010) [2023-12-26 20:52:30,776][105620] Updated weights for policy 1, policy_version 757337 (0.0010) [2023-12-26 20:52:30,843][105620] Updated weights for policy 1, policy_version 757347 (0.0010) [2023-12-26 20:52:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 387760128. Throughput: 0: 9929.0, 1: 9766.8. Samples: 387728168. Policy #0 lag: (min: 2.0, avg: 18.7, max: 34.0) [2023-12-26 20:52:31,062][104569] Avg episode reward: [(0, '9077.434'), (1, '8882.420')] [2023-12-26 20:52:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000757352_193904640.pth... [2023-12-26 20:52:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000756200_193609728.pth [2023-12-26 20:52:31,104][105692] Updated weights for policy 0, policy_version 757135 (0.0008) [2023-12-26 20:52:31,176][105692] Updated weights for policy 0, policy_version 757145 (0.0010) [2023-12-26 20:52:31,230][105692] Updated weights for policy 0, policy_version 757155 (0.0010) [2023-12-26 20:52:31,254][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000757160_193863680.pth... [2023-12-26 20:52:31,258][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000755976_193560576.pth [2023-12-26 20:52:31,549][105620] Updated weights for policy 1, policy_version 757357 (0.0009) [2023-12-26 20:52:31,606][105620] Updated weights for policy 1, policy_version 757367 (0.0008) [2023-12-26 20:52:31,668][105620] Updated weights for policy 1, policy_version 757377 (0.0009) [2023-12-26 20:52:32,030][105692] Updated weights for policy 0, policy_version 757165 (0.0007) [2023-12-26 20:52:32,091][105692] Updated weights for policy 0, policy_version 757175 (0.0007) [2023-12-26 20:52:32,149][105692] Updated weights for policy 0, policy_version 757185 (0.0010) [2023-12-26 20:52:32,296][105620] Updated weights for policy 1, policy_version 757387 (0.0008) [2023-12-26 20:52:32,354][105620] Updated weights for policy 1, policy_version 757397 (0.0009) [2023-12-26 20:52:32,418][105620] Updated weights for policy 1, policy_version 757407 (0.0006) [2023-12-26 20:52:32,779][105692] Updated weights for policy 0, policy_version 757195 (0.0009) [2023-12-26 20:52:32,836][105692] Updated weights for policy 0, policy_version 757205 (0.0005) [2023-12-26 20:52:32,894][105692] Updated weights for policy 0, policy_version 757215 (0.0008) [2023-12-26 20:52:32,999][105620] Updated weights for policy 1, policy_version 757417 (0.0005) [2023-12-26 20:52:33,045][105620] Updated weights for policy 1, policy_version 757427 (0.0005) [2023-12-26 20:52:33,095][105620] Updated weights for policy 1, policy_version 757437 (0.0005) [2023-12-26 20:52:33,152][105620] Updated weights for policy 1, policy_version 757447 (0.0005) [2023-12-26 20:52:33,605][105692] Updated weights for policy 0, policy_version 757225 (0.0011) [2023-12-26 20:52:33,665][105692] Updated weights for policy 0, policy_version 757235 (0.0010) [2023-12-26 20:52:33,700][105620] Updated weights for policy 1, policy_version 757457 (0.0010) [2023-12-26 20:52:33,713][105692] Updated weights for policy 0, policy_version 757245 (0.0010) [2023-12-26 20:52:33,758][105620] Updated weights for policy 1, policy_version 757467 (0.0007) [2023-12-26 20:52:33,777][105692] Updated weights for policy 0, policy_version 757255 (0.0010) [2023-12-26 20:52:33,808][105620] Updated weights for policy 1, policy_version 757477 (0.0006) [2023-12-26 20:52:34,425][105620] Updated weights for policy 1, policy_version 757487 (0.0009) [2023-12-26 20:52:34,487][105620] Updated weights for policy 1, policy_version 757497 (0.0010) [2023-12-26 20:52:34,538][105692] Updated weights for policy 0, policy_version 757265 (0.0007) [2023-12-26 20:52:34,551][105620] Updated weights for policy 1, policy_version 757507 (0.0010) [2023-12-26 20:52:34,600][105692] Updated weights for policy 0, policy_version 757275 (0.0009) [2023-12-26 20:52:34,666][105692] Updated weights for policy 0, policy_version 757285 (0.0007) [2023-12-26 20:52:35,187][105620] Updated weights for policy 1, policy_version 757517 (0.0008) [2023-12-26 20:52:35,234][105620] Updated weights for policy 1, policy_version 757527 (0.0005) [2023-12-26 20:52:35,279][105620] Updated weights for policy 1, policy_version 757537 (0.0005) [2023-12-26 20:52:35,387][105692] Updated weights for policy 0, policy_version 757295 (0.0010) [2023-12-26 20:52:35,452][105692] Updated weights for policy 0, policy_version 757305 (0.0010) [2023-12-26 20:52:35,520][105692] Updated weights for policy 0, policy_version 757315 (0.0010) [2023-12-26 20:52:35,842][105620] Updated weights for policy 1, policy_version 757547 (0.0005) [2023-12-26 20:52:35,913][105620] Updated weights for policy 1, policy_version 757557 (0.0005) [2023-12-26 20:52:35,982][105620] Updated weights for policy 1, policy_version 757567 (0.0005) [2023-12-26 20:52:36,062][104569] Fps is (10 sec: 21299.4, 60 sec: 19797.3, 300 sec: 19605.2). Total num frames: 387866624. Throughput: 0: 9906.7, 1: 9804.4. Samples: 387850308. Policy #0 lag: (min: 2.0, avg: 18.7, max: 34.0) [2023-12-26 20:52:36,063][104569] Avg episode reward: [(0, '9078.646'), (1, '8975.688')] [2023-12-26 20:52:36,214][105692] Updated weights for policy 0, policy_version 757325 (0.0009) [2023-12-26 20:52:36,269][105692] Updated weights for policy 0, policy_version 757335 (0.0008) [2023-12-26 20:52:36,327][105692] Updated weights for policy 0, policy_version 757345 (0.0008) [2023-12-26 20:52:36,563][105620] Updated weights for policy 1, policy_version 757577 (0.0005) [2023-12-26 20:52:36,625][105620] Updated weights for policy 1, policy_version 757587 (0.0005) [2023-12-26 20:52:36,697][105620] Updated weights for policy 1, policy_version 757597 (0.0005) [2023-12-26 20:52:36,757][105620] Updated weights for policy 1, policy_version 757607 (0.0005) [2023-12-26 20:52:37,085][105692] Updated weights for policy 0, policy_version 757355 (0.0009) [2023-12-26 20:52:37,144][105692] Updated weights for policy 0, policy_version 757365 (0.0010) [2023-12-26 20:52:37,203][105692] Updated weights for policy 0, policy_version 757375 (0.0010) [2023-12-26 20:52:37,272][105620] Updated weights for policy 1, policy_version 757617 (0.0005) [2023-12-26 20:52:37,319][105620] Updated weights for policy 1, policy_version 757627 (0.0005) [2023-12-26 20:52:37,367][105620] Updated weights for policy 1, policy_version 757637 (0.0006) [2023-12-26 20:52:37,978][105692] Updated weights for policy 0, policy_version 757385 (0.0011) [2023-12-26 20:52:37,985][105620] Updated weights for policy 1, policy_version 757647 (0.0007) [2023-12-26 20:52:38,035][105692] Updated weights for policy 0, policy_version 757395 (0.0007) [2023-12-26 20:52:38,045][105620] Updated weights for policy 1, policy_version 757657 (0.0005) [2023-12-26 20:52:38,090][105692] Updated weights for policy 0, policy_version 757405 (0.0006) [2023-12-26 20:52:38,110][105620] Updated weights for policy 1, policy_version 757667 (0.0008) [2023-12-26 20:52:38,151][105692] Updated weights for policy 0, policy_version 757415 (0.0006) [2023-12-26 20:52:38,651][105620] Updated weights for policy 1, policy_version 757677 (0.0006) [2023-12-26 20:52:38,708][105620] Updated weights for policy 1, policy_version 757687 (0.0005) [2023-12-26 20:52:38,763][105620] Updated weights for policy 1, policy_version 757697 (0.0005) [2023-12-26 20:52:38,805][105692] Updated weights for policy 0, policy_version 757425 (0.0007) [2023-12-26 20:52:38,869][105692] Updated weights for policy 0, policy_version 757435 (0.0008) [2023-12-26 20:52:38,932][105692] Updated weights for policy 0, policy_version 757445 (0.0008) [2023-12-26 20:52:39,449][105620] Updated weights for policy 1, policy_version 757707 (0.0009) [2023-12-26 20:52:39,518][105620] Updated weights for policy 1, policy_version 757717 (0.0009) [2023-12-26 20:52:39,574][105620] Updated weights for policy 1, policy_version 757727 (0.0011) [2023-12-26 20:52:39,694][105692] Updated weights for policy 0, policy_version 757455 (0.0008) [2023-12-26 20:52:39,747][105692] Updated weights for policy 0, policy_version 757465 (0.0008) [2023-12-26 20:52:39,813][105692] Updated weights for policy 0, policy_version 757475 (0.0008) [2023-12-26 20:52:40,342][105620] Updated weights for policy 1, policy_version 757737 (0.0011) [2023-12-26 20:52:40,401][105620] Updated weights for policy 1, policy_version 757747 (0.0010) [2023-12-26 20:52:40,459][105620] Updated weights for policy 1, policy_version 757757 (0.0007) [2023-12-26 20:52:40,513][105620] Updated weights for policy 1, policy_version 757767 (0.0009) [2023-12-26 20:52:40,594][105692] Updated weights for policy 0, policy_version 757485 (0.0009) [2023-12-26 20:52:40,658][105692] Updated weights for policy 0, policy_version 757495 (0.0010) [2023-12-26 20:52:40,723][105692] Updated weights for policy 0, policy_version 757505 (0.0010) [2023-12-26 20:52:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 387964928. Throughput: 0: 9851.4, 1: 10054.5. Samples: 387973128. Policy #0 lag: (min: 2.0, avg: 18.7, max: 34.0) [2023-12-26 20:52:41,062][104569] Avg episode reward: [(0, '9078.840'), (1, '8742.417')] [2023-12-26 20:52:41,217][105620] Updated weights for policy 1, policy_version 757777 (0.0008) [2023-12-26 20:52:41,271][105620] Updated weights for policy 1, policy_version 757787 (0.0008) [2023-12-26 20:52:41,329][105620] Updated weights for policy 1, policy_version 757798 (0.0008) [2023-12-26 20:52:41,451][105692] Updated weights for policy 0, policy_version 757515 (0.0009) [2023-12-26 20:52:41,500][105692] Updated weights for policy 0, policy_version 757525 (0.0005) [2023-12-26 20:52:41,550][105692] Updated weights for policy 0, policy_version 757535 (0.0005) [2023-12-26 20:52:42,052][105620] Updated weights for policy 1, policy_version 757808 (0.0007) [2023-12-26 20:52:42,107][105620] Updated weights for policy 1, policy_version 757818 (0.0009) [2023-12-26 20:52:42,171][105620] Updated weights for policy 1, policy_version 757828 (0.0007) [2023-12-26 20:52:42,237][105692] Updated weights for policy 0, policy_version 757545 (0.0006) [2023-12-26 20:52:42,303][105692] Updated weights for policy 0, policy_version 757555 (0.0009) [2023-12-26 20:52:42,368][105692] Updated weights for policy 0, policy_version 757565 (0.0009) [2023-12-26 20:52:42,430][105692] Updated weights for policy 0, policy_version 757575 (0.0009) [2023-12-26 20:52:42,922][105620] Updated weights for policy 1, policy_version 757838 (0.0006) [2023-12-26 20:52:42,980][105620] Updated weights for policy 1, policy_version 757848 (0.0005) [2023-12-26 20:52:43,029][105620] Updated weights for policy 1, policy_version 757858 (0.0006) [2023-12-26 20:52:43,059][105692] Updated weights for policy 0, policy_version 757585 (0.0010) [2023-12-26 20:52:43,114][105692] Updated weights for policy 0, policy_version 757595 (0.0010) [2023-12-26 20:52:43,165][105692] Updated weights for policy 0, policy_version 757605 (0.0010) [2023-12-26 20:52:43,608][105620] Updated weights for policy 1, policy_version 757868 (0.0005) [2023-12-26 20:52:43,670][105620] Updated weights for policy 1, policy_version 757878 (0.0005) [2023-12-26 20:52:43,720][105620] Updated weights for policy 1, policy_version 757888 (0.0005) [2023-12-26 20:52:43,869][105692] Updated weights for policy 0, policy_version 757615 (0.0007) [2023-12-26 20:52:43,915][105692] Updated weights for policy 0, policy_version 757625 (0.0008) [2023-12-26 20:52:43,984][105692] Updated weights for policy 0, policy_version 757635 (0.0005) [2023-12-26 20:52:44,258][105620] Updated weights for policy 1, policy_version 757898 (0.0005) [2023-12-26 20:52:44,312][105620] Updated weights for policy 1, policy_version 757908 (0.0005) [2023-12-26 20:52:44,363][105620] Updated weights for policy 1, policy_version 757918 (0.0005) [2023-12-26 20:52:44,415][105620] Updated weights for policy 1, policy_version 757928 (0.0005) [2023-12-26 20:52:44,631][105692] Updated weights for policy 0, policy_version 757645 (0.0007) [2023-12-26 20:52:44,683][105692] Updated weights for policy 0, policy_version 757655 (0.0006) [2023-12-26 20:52:44,727][105692] Updated weights for policy 0, policy_version 757665 (0.0010) [2023-12-26 20:52:45,146][105620] Updated weights for policy 1, policy_version 757938 (0.0008) [2023-12-26 20:52:45,206][105620] Updated weights for policy 1, policy_version 757948 (0.0008) [2023-12-26 20:52:45,270][105620] Updated weights for policy 1, policy_version 757958 (0.0008) [2023-12-26 20:52:45,491][105692] Updated weights for policy 0, policy_version 757675 (0.0010) [2023-12-26 20:52:45,560][105692] Updated weights for policy 0, policy_version 757685 (0.0010) [2023-12-26 20:52:45,624][105692] Updated weights for policy 0, policy_version 757695 (0.0009) [2023-12-26 20:52:46,002][105620] Updated weights for policy 1, policy_version 757968 (0.0008) [2023-12-26 20:52:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 388063232. Throughput: 0: 9791.5, 1: 10167.1. Samples: 388034308. Policy #0 lag: (min: 2.0, avg: 18.7, max: 34.0) [2023-12-26 20:52:46,062][105620] Updated weights for policy 1, policy_version 757978 (0.0007) [2023-12-26 20:52:46,063][104569] Avg episode reward: [(0, '8339.484'), (1, '8925.610')] [2023-12-26 20:52:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000757704_194002944.pth... [2023-12-26 20:52:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000756584_193716224.pth [2023-12-26 20:52:46,111][105620] Updated weights for policy 1, policy_version 757988 (0.0008) [2023-12-26 20:52:46,128][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000757992_194068480.pth... [2023-12-26 20:52:46,131][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000756744_193748992.pth [2023-12-26 20:52:46,291][105692] Updated weights for policy 0, policy_version 757705 (0.0010) [2023-12-26 20:52:46,345][105692] Updated weights for policy 0, policy_version 757715 (0.0010) [2023-12-26 20:52:46,403][105692] Updated weights for policy 0, policy_version 757725 (0.0010) [2023-12-26 20:52:46,463][105692] Updated weights for policy 0, policy_version 757735 (0.0010) [2023-12-26 20:52:46,864][105620] Updated weights for policy 1, policy_version 757998 (0.0008) [2023-12-26 20:52:46,912][105620] Updated weights for policy 1, policy_version 758008 (0.0008) [2023-12-26 20:52:46,961][105620] Updated weights for policy 1, policy_version 758018 (0.0008) [2023-12-26 20:52:47,205][105692] Updated weights for policy 0, policy_version 757745 (0.0010) [2023-12-26 20:52:47,272][105692] Updated weights for policy 0, policy_version 757755 (0.0010) [2023-12-26 20:52:47,337][105692] Updated weights for policy 0, policy_version 757765 (0.0006) [2023-12-26 20:52:47,784][105620] Updated weights for policy 1, policy_version 758029 (0.0010) [2023-12-26 20:52:47,841][105620] Updated weights for policy 1, policy_version 758039 (0.0010) [2023-12-26 20:52:47,894][105620] Updated weights for policy 1, policy_version 758050 (0.0010) [2023-12-26 20:52:47,946][105692] Updated weights for policy 0, policy_version 757775 (0.0005) [2023-12-26 20:52:48,007][105692] Updated weights for policy 0, policy_version 757785 (0.0008) [2023-12-26 20:52:48,069][105692] Updated weights for policy 0, policy_version 757795 (0.0009) [2023-12-26 20:52:48,710][105620] Updated weights for policy 1, policy_version 758060 (0.0008) [2023-12-26 20:52:48,773][105692] Updated weights for policy 0, policy_version 757805 (0.0008) [2023-12-26 20:52:48,775][105620] Updated weights for policy 1, policy_version 758070 (0.0007) [2023-12-26 20:52:48,834][105620] Updated weights for policy 1, policy_version 758080 (0.0007) [2023-12-26 20:52:48,835][105692] Updated weights for policy 0, policy_version 757815 (0.0010) [2023-12-26 20:52:48,894][105692] Updated weights for policy 0, policy_version 757825 (0.0011) [2023-12-26 20:52:49,610][105692] Updated weights for policy 0, policy_version 757835 (0.0009) [2023-12-26 20:52:49,630][105620] Updated weights for policy 1, policy_version 758090 (0.0007) [2023-12-26 20:52:49,671][105692] Updated weights for policy 0, policy_version 757845 (0.0006) [2023-12-26 20:52:49,689][105620] Updated weights for policy 1, policy_version 758100 (0.0009) [2023-12-26 20:52:49,728][105692] Updated weights for policy 0, policy_version 757855 (0.0005) [2023-12-26 20:52:49,738][105620] Updated weights for policy 1, policy_version 758110 (0.0009) [2023-12-26 20:52:49,800][105620] Updated weights for policy 1, policy_version 758120 (0.0008) [2023-12-26 20:52:50,356][105692] Updated weights for policy 0, policy_version 757865 (0.0007) [2023-12-26 20:52:50,420][105692] Updated weights for policy 0, policy_version 757875 (0.0008) [2023-12-26 20:52:50,486][105692] Updated weights for policy 0, policy_version 757885 (0.0009) [2023-12-26 20:52:50,548][105692] Updated weights for policy 0, policy_version 757895 (0.0009) [2023-12-26 20:52:50,630][105620] Updated weights for policy 1, policy_version 758130 (0.0008) [2023-12-26 20:52:50,690][105620] Updated weights for policy 1, policy_version 758140 (0.0008) [2023-12-26 20:52:50,753][105620] Updated weights for policy 1, policy_version 758150 (0.0008) [2023-12-26 20:52:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 388161536. Throughput: 0: 9792.7, 1: 10242.3. Samples: 388150244. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-26 20:52:51,063][104569] Avg episode reward: [(0, '8601.675'), (1, '9165.046')] [2023-12-26 20:52:51,243][105692] Updated weights for policy 0, policy_version 757905 (0.0009) [2023-12-26 20:52:51,302][105692] Updated weights for policy 0, policy_version 757915 (0.0008) [2023-12-26 20:52:51,360][105692] Updated weights for policy 0, policy_version 757925 (0.0008) [2023-12-26 20:52:51,515][105620] Updated weights for policy 1, policy_version 758160 (0.0006) [2023-12-26 20:52:51,572][105620] Updated weights for policy 1, policy_version 758170 (0.0005) [2023-12-26 20:52:51,636][105620] Updated weights for policy 1, policy_version 758180 (0.0008) [2023-12-26 20:52:52,159][105692] Updated weights for policy 0, policy_version 757935 (0.0009) [2023-12-26 20:52:52,217][105692] Updated weights for policy 0, policy_version 757945 (0.0008) [2023-12-26 20:52:52,279][105692] Updated weights for policy 0, policy_version 757955 (0.0007) [2023-12-26 20:52:52,350][105620] Updated weights for policy 1, policy_version 758190 (0.0009) [2023-12-26 20:52:52,414][105620] Updated weights for policy 1, policy_version 758200 (0.0008) [2023-12-26 20:52:52,470][105620] Updated weights for policy 1, policy_version 758210 (0.0010) [2023-12-26 20:52:53,008][105692] Updated weights for policy 0, policy_version 757965 (0.0009) [2023-12-26 20:52:53,072][105692] Updated weights for policy 0, policy_version 757975 (0.0009) [2023-12-26 20:52:53,129][105692] Updated weights for policy 0, policy_version 757985 (0.0009) [2023-12-26 20:52:53,260][105620] Updated weights for policy 1, policy_version 758220 (0.0007) [2023-12-26 20:52:53,312][105620] Updated weights for policy 1, policy_version 758230 (0.0009) [2023-12-26 20:52:53,359][105620] Updated weights for policy 1, policy_version 758240 (0.0008) [2023-12-26 20:52:53,793][105692] Updated weights for policy 0, policy_version 757995 (0.0009) [2023-12-26 20:52:53,837][105692] Updated weights for policy 0, policy_version 758005 (0.0010) [2023-12-26 20:52:53,882][105692] Updated weights for policy 0, policy_version 758015 (0.0010) [2023-12-26 20:52:54,149][105620] Updated weights for policy 1, policy_version 758250 (0.0008) [2023-12-26 20:52:54,208][105620] Updated weights for policy 1, policy_version 758260 (0.0008) [2023-12-26 20:52:54,269][105620] Updated weights for policy 1, policy_version 758270 (0.0008) [2023-12-26 20:52:54,323][105620] Updated weights for policy 1, policy_version 758280 (0.0008) [2023-12-26 20:52:54,651][105692] Updated weights for policy 0, policy_version 758025 (0.0010) [2023-12-26 20:52:54,698][105692] Updated weights for policy 0, policy_version 758035 (0.0010) [2023-12-26 20:52:54,753][105692] Updated weights for policy 0, policy_version 758045 (0.0010) [2023-12-26 20:52:54,841][105692] Updated weights for policy 0, policy_version 758055 (0.0010) [2023-12-26 20:52:55,077][105620] Updated weights for policy 1, policy_version 758290 (0.0010) [2023-12-26 20:52:55,130][105620] Updated weights for policy 1, policy_version 758300 (0.0009) [2023-12-26 20:52:55,182][105620] Updated weights for policy 1, policy_version 758310 (0.0008) [2023-12-26 20:52:55,506][105692] Updated weights for policy 0, policy_version 758065 (0.0006) [2023-12-26 20:52:55,564][105692] Updated weights for policy 0, policy_version 758075 (0.0005) [2023-12-26 20:52:55,625][105692] Updated weights for policy 0, policy_version 758085 (0.0010) [2023-12-26 20:52:55,982][105620] Updated weights for policy 1, policy_version 758320 (0.0008) [2023-12-26 20:52:56,034][105620] Updated weights for policy 1, policy_version 758330 (0.0009) [2023-12-26 20:52:56,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 388251648. Throughput: 0: 9805.1, 1: 10251.3. Samples: 388263932. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-26 20:52:56,062][104569] Avg episode reward: [(0, '9136.756'), (1, '9166.347')] [2023-12-26 20:52:56,087][105620] Updated weights for policy 1, policy_version 758340 (0.0009) [2023-12-26 20:52:56,234][105692] Updated weights for policy 0, policy_version 758095 (0.0007) [2023-12-26 20:52:56,300][105692] Updated weights for policy 0, policy_version 758105 (0.0011) [2023-12-26 20:52:56,368][105692] Updated weights for policy 0, policy_version 758115 (0.0010) [2023-12-26 20:52:56,880][105620] Updated weights for policy 1, policy_version 758350 (0.0009) [2023-12-26 20:52:56,927][105620] Updated weights for policy 1, policy_version 758360 (0.0008) [2023-12-26 20:52:56,977][105620] Updated weights for policy 1, policy_version 758370 (0.0009) [2023-12-26 20:52:57,038][105692] Updated weights for policy 0, policy_version 758125 (0.0008) [2023-12-26 20:52:57,101][105692] Updated weights for policy 0, policy_version 758135 (0.0005) [2023-12-26 20:52:57,161][105692] Updated weights for policy 0, policy_version 758145 (0.0005) [2023-12-26 20:52:57,668][105692] Updated weights for policy 0, policy_version 758155 (0.0005) [2023-12-26 20:52:57,735][105692] Updated weights for policy 0, policy_version 758165 (0.0006) [2023-12-26 20:52:57,796][105692] Updated weights for policy 0, policy_version 758175 (0.0010) [2023-12-26 20:52:57,867][105620] Updated weights for policy 1, policy_version 758381 (0.0009) [2023-12-26 20:52:57,921][105620] Updated weights for policy 1, policy_version 758391 (0.0007) [2023-12-26 20:52:57,962][105586] KL-divergence is very high: 129.6099 [2023-12-26 20:52:57,972][105620] Updated weights for policy 1, policy_version 758401 (0.0008) [2023-12-26 20:52:57,979][105586] KL-divergence is very high: 131.3003 [2023-12-26 20:52:58,001][105586] KL-divergence is very high: 105.5730 [2023-12-26 20:52:58,006][105586] KL-divergence is very high: 175.0798 [2023-12-26 20:52:58,502][105692] Updated weights for policy 0, policy_version 758185 (0.0010) [2023-12-26 20:52:58,575][105692] Updated weights for policy 0, policy_version 758196 (0.0013) [2023-12-26 20:52:58,640][105692] Updated weights for policy 0, policy_version 758206 (0.0007) [2023-12-26 20:52:58,701][105692] Updated weights for policy 0, policy_version 758216 (0.0007) [2023-12-26 20:52:58,808][105586] KL-divergence is very high: 109.2238 [2023-12-26 20:52:58,835][105620] Updated weights for policy 1, policy_version 758411 (0.0008) [2023-12-26 20:52:58,914][105620] Updated weights for policy 1, policy_version 758421 (0.0009) [2023-12-26 20:52:58,980][105620] Updated weights for policy 1, policy_version 758431 (0.0009) [2023-12-26 20:52:59,457][105692] Updated weights for policy 0, policy_version 758226 (0.0007) [2023-12-26 20:52:59,515][105692] Updated weights for policy 0, policy_version 758236 (0.0010) [2023-12-26 20:52:59,565][105692] Updated weights for policy 0, policy_version 758246 (0.0010) [2023-12-26 20:52:59,672][105620] Updated weights for policy 1, policy_version 758441 (0.0009) [2023-12-26 20:52:59,748][105620] Updated weights for policy 1, policy_version 758451 (0.0005) [2023-12-26 20:52:59,811][105620] Updated weights for policy 1, policy_version 758461 (0.0008) [2023-12-26 20:52:59,872][105620] Updated weights for policy 1, policy_version 758471 (0.0008) [2023-12-26 20:53:00,327][105692] Updated weights for policy 0, policy_version 758256 (0.0010) [2023-12-26 20:53:00,383][105692] Updated weights for policy 0, policy_version 758266 (0.0010) [2023-12-26 20:53:00,443][105692] Updated weights for policy 0, policy_version 758276 (0.0009) [2023-12-26 20:53:00,560][105620] Updated weights for policy 1, policy_version 758481 (0.0007) [2023-12-26 20:53:00,607][105620] Updated weights for policy 1, policy_version 758491 (0.0008) [2023-12-26 20:53:00,660][105620] Updated weights for policy 1, policy_version 758501 (0.0009) [2023-12-26 20:53:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 388349952. Throughput: 0: 9842.6, 1: 10126.6. Samples: 388321552. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-26 20:53:01,062][104569] Avg episode reward: [(0, '9174.126'), (1, '7911.757')] [2023-12-26 20:53:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000758504_194199552.pth... [2023-12-26 20:53:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000758280_194150400.pth... [2023-12-26 20:53:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000757352_193904640.pth [2023-12-26 20:53:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000757160_193863680.pth [2023-12-26 20:53:01,216][105692] Updated weights for policy 0, policy_version 758286 (0.0010) [2023-12-26 20:53:01,277][105692] Updated weights for policy 0, policy_version 758296 (0.0009) [2023-12-26 20:53:01,334][105692] Updated weights for policy 0, policy_version 758306 (0.0009) [2023-12-26 20:53:01,405][105620] Updated weights for policy 1, policy_version 758511 (0.0008) [2023-12-26 20:53:01,465][105620] Updated weights for policy 1, policy_version 758521 (0.0007) [2023-12-26 20:53:01,524][105620] Updated weights for policy 1, policy_version 758531 (0.0009) [2023-12-26 20:53:02,120][105692] Updated weights for policy 0, policy_version 758316 (0.0009) [2023-12-26 20:53:02,181][105692] Updated weights for policy 0, policy_version 758326 (0.0009) [2023-12-26 20:53:02,240][105692] Updated weights for policy 0, policy_version 758336 (0.0010) [2023-12-26 20:53:02,270][105620] Updated weights for policy 1, policy_version 758541 (0.0009) [2023-12-26 20:53:02,325][105620] Updated weights for policy 1, policy_version 758551 (0.0008) [2023-12-26 20:53:02,388][105620] Updated weights for policy 1, policy_version 758561 (0.0007) [2023-12-26 20:53:02,954][105620] Updated weights for policy 1, policy_version 758571 (0.0006) [2023-12-26 20:53:03,001][105620] Updated weights for policy 1, policy_version 758581 (0.0009) [2023-12-26 20:53:03,050][105692] Updated weights for policy 0, policy_version 758346 (0.0009) [2023-12-26 20:53:03,056][105620] Updated weights for policy 1, policy_version 758591 (0.0008) [2023-12-26 20:53:03,101][105692] Updated weights for policy 0, policy_version 758356 (0.0006) [2023-12-26 20:53:03,160][105692] Updated weights for policy 0, policy_version 758366 (0.0008) [2023-12-26 20:53:03,210][105692] Updated weights for policy 0, policy_version 758376 (0.0008) [2023-12-26 20:53:03,738][105620] Updated weights for policy 1, policy_version 758601 (0.0008) [2023-12-26 20:53:03,788][105620] Updated weights for policy 1, policy_version 758611 (0.0009) [2023-12-26 20:53:03,847][105620] Updated weights for policy 1, policy_version 758621 (0.0008) [2023-12-26 20:53:03,907][105620] Updated weights for policy 1, policy_version 758631 (0.0007) [2023-12-26 20:53:03,999][105692] Updated weights for policy 0, policy_version 758386 (0.0006) [2023-12-26 20:53:04,054][105692] Updated weights for policy 0, policy_version 758396 (0.0005) [2023-12-26 20:53:04,116][105692] Updated weights for policy 0, policy_version 758406 (0.0006) [2023-12-26 20:53:04,561][105620] Updated weights for policy 1, policy_version 758641 (0.0009) [2023-12-26 20:53:04,620][105620] Updated weights for policy 1, policy_version 758651 (0.0009) [2023-12-26 20:53:04,681][105620] Updated weights for policy 1, policy_version 758661 (0.0009) [2023-12-26 20:53:04,859][105692] Updated weights for policy 0, policy_version 758416 (0.0008) [2023-12-26 20:53:04,915][105692] Updated weights for policy 0, policy_version 758426 (0.0008) [2023-12-26 20:53:04,966][105692] Updated weights for policy 0, policy_version 758436 (0.0009) [2023-12-26 20:53:05,451][105620] Updated weights for policy 1, policy_version 758671 (0.0009) [2023-12-26 20:53:05,502][105620] Updated weights for policy 1, policy_version 758681 (0.0009) [2023-12-26 20:53:05,549][105620] Updated weights for policy 1, policy_version 758691 (0.0009) [2023-12-26 20:53:05,651][105692] Updated weights for policy 0, policy_version 758446 (0.0007) [2023-12-26 20:53:05,709][105692] Updated weights for policy 0, policy_version 758456 (0.0008) [2023-12-26 20:53:05,760][105692] Updated weights for policy 0, policy_version 758466 (0.0009) [2023-12-26 20:53:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 388448256. Throughput: 0: 9774.0, 1: 10030.6. Samples: 388436940. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-26 20:53:06,062][104569] Avg episode reward: [(0, '9261.676'), (1, '8091.634')] [2023-12-26 20:53:06,398][105620] Updated weights for policy 1, policy_version 758701 (0.0009) [2023-12-26 20:53:06,405][105692] Updated weights for policy 0, policy_version 758476 (0.0009) [2023-12-26 20:53:06,452][105620] Updated weights for policy 1, policy_version 758711 (0.0005) [2023-12-26 20:53:06,457][105692] Updated weights for policy 0, policy_version 758486 (0.0010) [2023-12-26 20:53:06,501][105620] Updated weights for policy 1, policy_version 758721 (0.0005) [2023-12-26 20:53:06,506][105692] Updated weights for policy 0, policy_version 758496 (0.0010) [2023-12-26 20:53:07,252][105692] Updated weights for policy 0, policy_version 758506 (0.0009) [2023-12-26 20:53:07,291][105620] Updated weights for policy 1, policy_version 758731 (0.0007) [2023-12-26 20:53:07,312][105692] Updated weights for policy 0, policy_version 758516 (0.0005) [2023-12-26 20:53:07,339][105620] Updated weights for policy 1, policy_version 758741 (0.0009) [2023-12-26 20:53:07,355][105692] Updated weights for policy 0, policy_version 758526 (0.0005) [2023-12-26 20:53:07,392][105620] Updated weights for policy 1, policy_version 758751 (0.0009) [2023-12-26 20:53:07,406][105692] Updated weights for policy 0, policy_version 758536 (0.0005) [2023-12-26 20:53:07,929][105692] Updated weights for policy 0, policy_version 758546 (0.0007) [2023-12-26 20:53:07,983][105692] Updated weights for policy 0, policy_version 758556 (0.0010) [2023-12-26 20:53:08,036][105692] Updated weights for policy 0, policy_version 758566 (0.0010) [2023-12-26 20:53:08,275][105620] Updated weights for policy 1, policy_version 758761 (0.0010) [2023-12-26 20:53:08,338][105620] Updated weights for policy 1, policy_version 758772 (0.0009) [2023-12-26 20:53:08,393][105620] Updated weights for policy 1, policy_version 758782 (0.0009) [2023-12-26 20:53:08,450][105620] Updated weights for policy 1, policy_version 758792 (0.0009) [2023-12-26 20:53:08,690][105692] Updated weights for policy 0, policy_version 758576 (0.0006) [2023-12-26 20:53:08,751][105692] Updated weights for policy 0, policy_version 758586 (0.0008) [2023-12-26 20:53:08,810][105692] Updated weights for policy 0, policy_version 758596 (0.0009) [2023-12-26 20:53:09,325][105620] Updated weights for policy 1, policy_version 758802 (0.0009) [2023-12-26 20:53:09,391][105620] Updated weights for policy 1, policy_version 758812 (0.0008) [2023-12-26 20:53:09,412][105692] Updated weights for policy 0, policy_version 758606 (0.0009) [2023-12-26 20:53:09,454][105620] Updated weights for policy 1, policy_version 758822 (0.0008) [2023-12-26 20:53:09,476][105692] Updated weights for policy 0, policy_version 758616 (0.0008) [2023-12-26 20:53:09,532][105692] Updated weights for policy 0, policy_version 758626 (0.0009) [2023-12-26 20:53:10,222][105692] Updated weights for policy 0, policy_version 758636 (0.0009) [2023-12-26 20:53:10,278][105620] Updated weights for policy 1, policy_version 758832 (0.0008) [2023-12-26 20:53:10,284][105692] Updated weights for policy 0, policy_version 758646 (0.0006) [2023-12-26 20:53:10,339][105620] Updated weights for policy 1, policy_version 758842 (0.0007) [2023-12-26 20:53:10,346][105692] Updated weights for policy 0, policy_version 758656 (0.0007) [2023-12-26 20:53:10,391][105620] Updated weights for policy 1, policy_version 758852 (0.0006) [2023-12-26 20:53:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 388538368. Throughput: 0: 9891.5, 1: 9810.2. Samples: 388551740. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-26 20:53:11,063][104569] Avg episode reward: [(0, '8910.881'), (1, '9250.106')] [2023-12-26 20:53:11,073][105692] Updated weights for policy 0, policy_version 758666 (0.0008) [2023-12-26 20:53:11,151][105692] Updated weights for policy 0, policy_version 758676 (0.0009) [2023-12-26 20:53:11,209][105620] Updated weights for policy 1, policy_version 758862 (0.0008) [2023-12-26 20:53:11,215][105692] Updated weights for policy 0, policy_version 758686 (0.0008) [2023-12-26 20:53:11,278][105620] Updated weights for policy 1, policy_version 758872 (0.0008) [2023-12-26 20:53:11,283][105692] Updated weights for policy 0, policy_version 758696 (0.0007) [2023-12-26 20:53:11,349][105620] Updated weights for policy 1, policy_version 758882 (0.0009) [2023-12-26 20:53:12,006][105692] Updated weights for policy 0, policy_version 758706 (0.0009) [2023-12-26 20:53:12,055][105620] Updated weights for policy 1, policy_version 758892 (0.0009) [2023-12-26 20:53:12,062][105692] Updated weights for policy 0, policy_version 758716 (0.0008) [2023-12-26 20:53:12,111][105620] Updated weights for policy 1, policy_version 758902 (0.0006) [2023-12-26 20:53:12,121][105692] Updated weights for policy 0, policy_version 758726 (0.0007) [2023-12-26 20:53:12,170][105620] Updated weights for policy 1, policy_version 758912 (0.0008) [2023-12-26 20:53:12,788][105620] Updated weights for policy 1, policy_version 758922 (0.0008) [2023-12-26 20:53:12,851][105620] Updated weights for policy 1, policy_version 758932 (0.0008) [2023-12-26 20:53:12,912][105620] Updated weights for policy 1, policy_version 758942 (0.0009) [2023-12-26 20:53:12,947][105692] Updated weights for policy 0, policy_version 758736 (0.0007) [2023-12-26 20:53:12,973][105620] Updated weights for policy 1, policy_version 758952 (0.0009) [2023-12-26 20:53:13,010][105692] Updated weights for policy 0, policy_version 758746 (0.0007) [2023-12-26 20:53:13,071][105692] Updated weights for policy 0, policy_version 758756 (0.0006) [2023-12-26 20:53:13,542][105620] Updated weights for policy 1, policy_version 758962 (0.0006) [2023-12-26 20:53:13,600][105620] Updated weights for policy 1, policy_version 758972 (0.0008) [2023-12-26 20:53:13,647][105620] Updated weights for policy 1, policy_version 758982 (0.0009) [2023-12-26 20:53:13,838][105692] Updated weights for policy 0, policy_version 758766 (0.0007) [2023-12-26 20:53:13,901][105692] Updated weights for policy 0, policy_version 758776 (0.0009) [2023-12-26 20:53:13,956][105692] Updated weights for policy 0, policy_version 758786 (0.0008) [2023-12-26 20:53:14,408][105620] Updated weights for policy 1, policy_version 758992 (0.0010) [2023-12-26 20:53:14,462][105620] Updated weights for policy 1, policy_version 759002 (0.0007) [2023-12-26 20:53:14,523][105620] Updated weights for policy 1, policy_version 759012 (0.0010) [2023-12-26 20:53:14,731][105692] Updated weights for policy 0, policy_version 758796 (0.0008) [2023-12-26 20:53:14,787][105692] Updated weights for policy 0, policy_version 758806 (0.0008) [2023-12-26 20:53:14,855][105692] Updated weights for policy 0, policy_version 758816 (0.0008) [2023-12-26 20:53:15,291][105620] Updated weights for policy 1, policy_version 759022 (0.0010) [2023-12-26 20:53:15,358][105620] Updated weights for policy 1, policy_version 759032 (0.0011) [2023-12-26 20:53:15,415][105620] Updated weights for policy 1, policy_version 759042 (0.0011) [2023-12-26 20:53:15,624][105692] Updated weights for policy 0, policy_version 758826 (0.0008) [2023-12-26 20:53:15,687][105692] Updated weights for policy 0, policy_version 758836 (0.0008) [2023-12-26 20:53:15,737][105692] Updated weights for policy 0, policy_version 758846 (0.0008) [2023-12-26 20:53:15,784][105692] Updated weights for policy 0, policy_version 758856 (0.0006) [2023-12-26 20:53:16,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 388636672. Throughput: 0: 9784.3, 1: 9797.9. Samples: 388609372. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-26 20:53:16,062][104569] Avg episode reward: [(0, '8911.166'), (1, '8977.339')] [2023-12-26 20:53:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000758856_194297856.pth... [2023-12-26 20:53:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000759048_194338816.pth... [2023-12-26 20:53:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000757704_194002944.pth [2023-12-26 20:53:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000757992_194068480.pth [2023-12-26 20:53:16,140][105620] Updated weights for policy 1, policy_version 759052 (0.0011) [2023-12-26 20:53:16,195][105620] Updated weights for policy 1, policy_version 759062 (0.0010) [2023-12-26 20:53:16,260][105620] Updated weights for policy 1, policy_version 759072 (0.0011) [2023-12-26 20:53:16,556][105692] Updated weights for policy 0, policy_version 758866 (0.0008) [2023-12-26 20:53:16,605][105692] Updated weights for policy 0, policy_version 758877 (0.0008) [2023-12-26 20:53:16,656][105692] Updated weights for policy 0, policy_version 758887 (0.0008) [2023-12-26 20:53:16,939][105620] Updated weights for policy 1, policy_version 759082 (0.0009) [2023-12-26 20:53:16,999][105620] Updated weights for policy 1, policy_version 759092 (0.0005) [2023-12-26 20:53:17,059][105620] Updated weights for policy 1, policy_version 759102 (0.0005) [2023-12-26 20:53:17,128][105620] Updated weights for policy 1, policy_version 759112 (0.0005) [2023-12-26 20:53:17,381][105692] Updated weights for policy 0, policy_version 758897 (0.0006) [2023-12-26 20:53:17,429][105692] Updated weights for policy 0, policy_version 758907 (0.0006) [2023-12-26 20:53:17,476][105692] Updated weights for policy 0, policy_version 758917 (0.0009) [2023-12-26 20:53:17,696][105620] Updated weights for policy 1, policy_version 759122 (0.0009) [2023-12-26 20:53:17,758][105620] Updated weights for policy 1, policy_version 759132 (0.0010) [2023-12-26 20:53:17,812][105620] Updated weights for policy 1, policy_version 759142 (0.0008) [2023-12-26 20:53:18,085][105692] Updated weights for policy 0, policy_version 758927 (0.0010) [2023-12-26 20:53:18,134][105692] Updated weights for policy 0, policy_version 758937 (0.0010) [2023-12-26 20:53:18,194][105692] Updated weights for policy 0, policy_version 758947 (0.0011) [2023-12-26 20:53:18,616][105620] Updated weights for policy 1, policy_version 759152 (0.0006) [2023-12-26 20:53:18,680][105620] Updated weights for policy 1, policy_version 759162 (0.0005) [2023-12-26 20:53:18,744][105620] Updated weights for policy 1, policy_version 759172 (0.0008) [2023-12-26 20:53:18,863][105692] Updated weights for policy 0, policy_version 758957 (0.0008) [2023-12-26 20:53:18,927][105692] Updated weights for policy 0, policy_version 758967 (0.0005) [2023-12-26 20:53:18,986][105692] Updated weights for policy 0, policy_version 758977 (0.0005) [2023-12-26 20:53:19,514][105620] Updated weights for policy 1, policy_version 759182 (0.0009) [2023-12-26 20:53:19,582][105620] Updated weights for policy 1, policy_version 759192 (0.0008) [2023-12-26 20:53:19,628][105692] Updated weights for policy 0, policy_version 758987 (0.0006) [2023-12-26 20:53:19,643][105620] Updated weights for policy 1, policy_version 759202 (0.0007) [2023-12-26 20:53:19,686][105692] Updated weights for policy 0, policy_version 758997 (0.0008) [2023-12-26 20:53:19,744][105692] Updated weights for policy 0, policy_version 759007 (0.0009) [2023-12-26 20:53:20,359][105620] Updated weights for policy 1, policy_version 759212 (0.0008) [2023-12-26 20:53:20,412][105620] Updated weights for policy 1, policy_version 759222 (0.0008) [2023-12-26 20:53:20,471][105620] Updated weights for policy 1, policy_version 759232 (0.0006) [2023-12-26 20:53:20,563][105692] Updated weights for policy 0, policy_version 759017 (0.0009) [2023-12-26 20:53:20,629][105692] Updated weights for policy 0, policy_version 759027 (0.0009) [2023-12-26 20:53:20,696][105692] Updated weights for policy 0, policy_version 759037 (0.0009) [2023-12-26 20:53:20,755][105692] Updated weights for policy 0, policy_version 759047 (0.0009) [2023-12-26 20:53:21,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 388734976. Throughput: 0: 9808.4, 1: 9648.3. Samples: 388725856. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-26 20:53:21,062][104569] Avg episode reward: [(0, '9306.400'), (1, '8890.581')] [2023-12-26 20:53:21,205][105620] Updated weights for policy 1, policy_version 759242 (0.0006) [2023-12-26 20:53:21,273][105620] Updated weights for policy 1, policy_version 759252 (0.0009) [2023-12-26 20:53:21,340][105620] Updated weights for policy 1, policy_version 759262 (0.0009) [2023-12-26 20:53:21,411][105620] Updated weights for policy 1, policy_version 759272 (0.0007) [2023-12-26 20:53:21,579][105692] Updated weights for policy 0, policy_version 759057 (0.0009) [2023-12-26 20:53:21,642][105692] Updated weights for policy 0, policy_version 759067 (0.0009) [2023-12-26 20:53:21,704][105692] Updated weights for policy 0, policy_version 759077 (0.0010) [2023-12-26 20:53:22,105][105620] Updated weights for policy 1, policy_version 759282 (0.0009) [2023-12-26 20:53:22,165][105620] Updated weights for policy 1, policy_version 759292 (0.0010) [2023-12-26 20:53:22,228][105620] Updated weights for policy 1, policy_version 759302 (0.0010) [2023-12-26 20:53:22,410][105692] Updated weights for policy 0, policy_version 759087 (0.0008) [2023-12-26 20:53:22,458][105692] Updated weights for policy 0, policy_version 759097 (0.0009) [2023-12-26 20:53:22,510][105692] Updated weights for policy 0, policy_version 759107 (0.0009) [2023-12-26 20:53:22,997][105620] Updated weights for policy 1, policy_version 759312 (0.0009) [2023-12-26 20:53:23,053][105620] Updated weights for policy 1, policy_version 759322 (0.0009) [2023-12-26 20:53:23,113][105620] Updated weights for policy 1, policy_version 759332 (0.0009) [2023-12-26 20:53:23,323][105692] Updated weights for policy 0, policy_version 759117 (0.0008) [2023-12-26 20:53:23,377][105692] Updated weights for policy 0, policy_version 759127 (0.0009) [2023-12-26 20:53:23,434][105692] Updated weights for policy 0, policy_version 759137 (0.0009) [2023-12-26 20:53:23,864][105620] Updated weights for policy 1, policy_version 759342 (0.0009) [2023-12-26 20:53:23,928][105620] Updated weights for policy 1, policy_version 759352 (0.0009) [2023-12-26 20:53:23,984][105620] Updated weights for policy 1, policy_version 759362 (0.0009) [2023-12-26 20:53:24,190][105692] Updated weights for policy 0, policy_version 759147 (0.0009) [2023-12-26 20:53:24,248][105692] Updated weights for policy 0, policy_version 759157 (0.0007) [2023-12-26 20:53:24,319][105692] Updated weights for policy 0, policy_version 759167 (0.0008) [2023-12-26 20:53:24,775][105620] Updated weights for policy 1, policy_version 759372 (0.0009) [2023-12-26 20:53:24,826][105620] Updated weights for policy 1, policy_version 759382 (0.0008) [2023-12-26 20:53:24,876][105620] Updated weights for policy 1, policy_version 759392 (0.0009) [2023-12-26 20:53:24,919][105692] Updated weights for policy 0, policy_version 759177 (0.0007) [2023-12-26 20:53:24,983][105692] Updated weights for policy 0, policy_version 759187 (0.0006) [2023-12-26 20:53:25,039][105692] Updated weights for policy 0, policy_version 759197 (0.0009) [2023-12-26 20:53:25,095][105692] Updated weights for policy 0, policy_version 759207 (0.0009) [2023-12-26 20:53:25,718][105692] Updated weights for policy 0, policy_version 759217 (0.0009) [2023-12-26 20:53:25,725][105620] Updated weights for policy 1, policy_version 759402 (0.0009) [2023-12-26 20:53:25,773][105692] Updated weights for policy 0, policy_version 759227 (0.0009) [2023-12-26 20:53:25,775][105620] Updated weights for policy 1, policy_version 759412 (0.0008) [2023-12-26 20:53:25,820][105620] Updated weights for policy 1, policy_version 759422 (0.0005) [2023-12-26 20:53:25,825][105692] Updated weights for policy 0, policy_version 759237 (0.0007) [2023-12-26 20:53:25,880][105620] Updated weights for policy 1, policy_version 759432 (0.0007) [2023-12-26 20:53:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 388833280. Throughput: 0: 9819.2, 1: 9407.3. Samples: 388838320. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-26 20:53:26,063][104569] Avg episode reward: [(0, '9175.037'), (1, '9170.611')] [2023-12-26 20:53:26,385][105692] Updated weights for policy 0, policy_version 759247 (0.0005) [2023-12-26 20:53:26,436][105692] Updated weights for policy 0, policy_version 759257 (0.0005) [2023-12-26 20:53:26,498][105692] Updated weights for policy 0, policy_version 759267 (0.0006) [2023-12-26 20:53:26,609][105620] Updated weights for policy 1, policy_version 759442 (0.0005) [2023-12-26 20:53:26,672][105620] Updated weights for policy 1, policy_version 759452 (0.0005) [2023-12-26 20:53:26,722][105620] Updated weights for policy 1, policy_version 759462 (0.0005) [2023-12-26 20:53:27,129][105692] Updated weights for policy 0, policy_version 759277 (0.0005) [2023-12-26 20:53:27,175][105692] Updated weights for policy 0, policy_version 759287 (0.0005) [2023-12-26 20:53:27,231][105692] Updated weights for policy 0, policy_version 759298 (0.0009) [2023-12-26 20:53:27,269][105620] Updated weights for policy 1, policy_version 759472 (0.0005) [2023-12-26 20:53:27,319][105620] Updated weights for policy 1, policy_version 759482 (0.0008) [2023-12-26 20:53:27,367][105620] Updated weights for policy 1, policy_version 759492 (0.0009) [2023-12-26 20:53:27,875][105692] Updated weights for policy 0, policy_version 759308 (0.0007) [2023-12-26 20:53:27,929][105692] Updated weights for policy 0, policy_version 759318 (0.0006) [2023-12-26 20:53:27,990][105692] Updated weights for policy 0, policy_version 759328 (0.0005) [2023-12-26 20:53:28,036][105620] Updated weights for policy 1, policy_version 759503 (0.0008) [2023-12-26 20:53:28,098][105620] Updated weights for policy 1, policy_version 759513 (0.0009) [2023-12-26 20:53:28,162][105620] Updated weights for policy 1, policy_version 759523 (0.0010) [2023-12-26 20:53:28,583][105692] Updated weights for policy 0, policy_version 759338 (0.0008) [2023-12-26 20:53:28,637][105692] Updated weights for policy 0, policy_version 759348 (0.0010) [2023-12-26 20:53:28,687][105692] Updated weights for policy 0, policy_version 759359 (0.0009) [2023-12-26 20:53:28,784][105620] Updated weights for policy 1, policy_version 759533 (0.0009) [2023-12-26 20:53:28,841][105620] Updated weights for policy 1, policy_version 759543 (0.0007) [2023-12-26 20:53:28,898][105620] Updated weights for policy 1, policy_version 759553 (0.0006) [2023-12-26 20:53:29,474][105692] Updated weights for policy 0, policy_version 759369 (0.0007) [2023-12-26 20:53:29,536][105692] Updated weights for policy 0, policy_version 759379 (0.0009) [2023-12-26 20:53:29,591][105692] Updated weights for policy 0, policy_version 759389 (0.0009) [2023-12-26 20:53:29,620][105620] Updated weights for policy 1, policy_version 759563 (0.0007) [2023-12-26 20:53:29,653][105692] Updated weights for policy 0, policy_version 759399 (0.0009) [2023-12-26 20:53:29,682][105620] Updated weights for policy 1, policy_version 759573 (0.0007) [2023-12-26 20:53:29,737][105620] Updated weights for policy 1, policy_version 759583 (0.0008) [2023-12-26 20:53:30,338][105692] Updated weights for policy 0, policy_version 759409 (0.0009) [2023-12-26 20:53:30,398][105692] Updated weights for policy 0, policy_version 759419 (0.0008) [2023-12-26 20:53:30,448][105692] Updated weights for policy 0, policy_version 759429 (0.0009) [2023-12-26 20:53:30,523][105620] Updated weights for policy 1, policy_version 759593 (0.0008) [2023-12-26 20:53:30,582][105620] Updated weights for policy 1, policy_version 759603 (0.0006) [2023-12-26 20:53:30,631][105620] Updated weights for policy 1, policy_version 759613 (0.0005) [2023-12-26 20:53:30,683][105620] Updated weights for policy 1, policy_version 759623 (0.0008) [2023-12-26 20:53:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 388931584. Throughput: 0: 9895.5, 1: 9434.3. Samples: 388904144. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-26 20:53:31,062][104569] Avg episode reward: [(0, '9082.729'), (1, '9171.232')] [2023-12-26 20:53:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000759624_194486272.pth... [2023-12-26 20:53:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000758504_194199552.pth [2023-12-26 20:53:31,121][105692] Updated weights for policy 0, policy_version 759439 (0.0008) [2023-12-26 20:53:31,185][105692] Updated weights for policy 0, policy_version 759449 (0.0009) [2023-12-26 20:53:31,243][105692] Updated weights for policy 0, policy_version 759459 (0.0009) [2023-12-26 20:53:31,272][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000759464_194453504.pth... [2023-12-26 20:53:31,276][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000758280_194150400.pth [2023-12-26 20:53:31,428][105620] Updated weights for policy 1, policy_version 759633 (0.0010) [2023-12-26 20:53:31,487][105620] Updated weights for policy 1, policy_version 759643 (0.0010) [2023-12-26 20:53:31,558][105620] Updated weights for policy 1, policy_version 759653 (0.0010) [2023-12-26 20:53:31,990][105692] Updated weights for policy 0, policy_version 759469 (0.0009) [2023-12-26 20:53:32,042][105692] Updated weights for policy 0, policy_version 759479 (0.0009) [2023-12-26 20:53:32,099][105692] Updated weights for policy 0, policy_version 759489 (0.0008) [2023-12-26 20:53:32,271][105620] Updated weights for policy 1, policy_version 759663 (0.0007) [2023-12-26 20:53:32,337][105620] Updated weights for policy 1, policy_version 759673 (0.0006) [2023-12-26 20:53:32,404][105620] Updated weights for policy 1, policy_version 759683 (0.0007) [2023-12-26 20:53:32,847][105692] Updated weights for policy 0, policy_version 759499 (0.0009) [2023-12-26 20:53:32,910][105692] Updated weights for policy 0, policy_version 759509 (0.0008) [2023-12-26 20:53:32,958][105692] Updated weights for policy 0, policy_version 759519 (0.0008) [2023-12-26 20:53:32,999][105620] Updated weights for policy 1, policy_version 759693 (0.0005) [2023-12-26 20:53:33,051][105620] Updated weights for policy 1, policy_version 759703 (0.0005) [2023-12-26 20:53:33,109][105620] Updated weights for policy 1, policy_version 759713 (0.0005) [2023-12-26 20:53:33,749][105620] Updated weights for policy 1, policy_version 759723 (0.0007) [2023-12-26 20:53:33,761][105692] Updated weights for policy 0, policy_version 759529 (0.0009) [2023-12-26 20:53:33,803][105692] Updated weights for policy 0, policy_version 759539 (0.0007) [2023-12-26 20:53:33,805][105620] Updated weights for policy 1, policy_version 759733 (0.0010) [2023-12-26 20:53:33,851][105692] Updated weights for policy 0, policy_version 759549 (0.0006) [2023-12-26 20:53:33,859][105620] Updated weights for policy 1, policy_version 759743 (0.0010) [2023-12-26 20:53:33,901][105692] Updated weights for policy 0, policy_version 759559 (0.0006) [2023-12-26 20:53:34,615][105620] Updated weights for policy 1, policy_version 759753 (0.0010) [2023-12-26 20:53:34,621][105692] Updated weights for policy 0, policy_version 759569 (0.0008) [2023-12-26 20:53:34,673][105620] Updated weights for policy 1, policy_version 759763 (0.0007) [2023-12-26 20:53:34,685][105692] Updated weights for policy 0, policy_version 759579 (0.0008) [2023-12-26 20:53:34,732][105620] Updated weights for policy 1, policy_version 759773 (0.0006) [2023-12-26 20:53:34,746][105692] Updated weights for policy 0, policy_version 759589 (0.0009) [2023-12-26 20:53:34,784][105620] Updated weights for policy 1, policy_version 759783 (0.0007) [2023-12-26 20:53:35,497][105692] Updated weights for policy 0, policy_version 759599 (0.0009) [2023-12-26 20:53:35,553][105620] Updated weights for policy 1, policy_version 759793 (0.0008) [2023-12-26 20:53:35,559][105692] Updated weights for policy 0, policy_version 759609 (0.0007) [2023-12-26 20:53:35,610][105620] Updated weights for policy 1, policy_version 759803 (0.0007) [2023-12-26 20:53:35,620][105692] Updated weights for policy 0, policy_version 759619 (0.0007) [2023-12-26 20:53:35,664][105620] Updated weights for policy 1, policy_version 759813 (0.0007) [2023-12-26 20:53:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19521.9). Total num frames: 389029888. Throughput: 0: 9827.6, 1: 9492.2. Samples: 389019632. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-26 20:53:36,063][104569] Avg episode reward: [(0, '9080.504'), (1, '9078.237')] [2023-12-26 20:53:36,311][105692] Updated weights for policy 0, policy_version 759629 (0.0008) [2023-12-26 20:53:36,376][105692] Updated weights for policy 0, policy_version 759639 (0.0009) [2023-12-26 20:53:36,424][105620] Updated weights for policy 1, policy_version 759823 (0.0007) [2023-12-26 20:53:36,437][105692] Updated weights for policy 0, policy_version 759649 (0.0008) [2023-12-26 20:53:36,489][105620] Updated weights for policy 1, policy_version 759833 (0.0006) [2023-12-26 20:53:36,551][105620] Updated weights for policy 1, policy_version 759843 (0.0008) [2023-12-26 20:53:37,166][105692] Updated weights for policy 0, policy_version 759659 (0.0008) [2023-12-26 20:53:37,223][105692] Updated weights for policy 0, policy_version 759669 (0.0010) [2023-12-26 20:53:37,276][105620] Updated weights for policy 1, policy_version 759853 (0.0007) [2023-12-26 20:53:37,289][105692] Updated weights for policy 0, policy_version 759679 (0.0007) [2023-12-26 20:53:37,339][105620] Updated weights for policy 1, policy_version 759863 (0.0008) [2023-12-26 20:53:37,395][105620] Updated weights for policy 1, policy_version 759873 (0.0006) [2023-12-26 20:53:37,872][105692] Updated weights for policy 0, policy_version 759689 (0.0006) [2023-12-26 20:53:37,933][105692] Updated weights for policy 0, policy_version 759699 (0.0005) [2023-12-26 20:53:37,988][105692] Updated weights for policy 0, policy_version 759709 (0.0005) [2023-12-26 20:53:38,029][105620] Updated weights for policy 1, policy_version 759883 (0.0007) [2023-12-26 20:53:38,043][105692] Updated weights for policy 0, policy_version 759719 (0.0006) [2023-12-26 20:53:38,078][105620] Updated weights for policy 1, policy_version 759893 (0.0005) [2023-12-26 20:53:38,132][105620] Updated weights for policy 1, policy_version 759903 (0.0007) [2023-12-26 20:53:38,626][105692] Updated weights for policy 0, policy_version 759729 (0.0009) [2023-12-26 20:53:38,688][105692] Updated weights for policy 0, policy_version 759739 (0.0009) [2023-12-26 20:53:38,751][105692] Updated weights for policy 0, policy_version 759749 (0.0008) [2023-12-26 20:53:38,821][105620] Updated weights for policy 1, policy_version 759913 (0.0007) [2023-12-26 20:53:38,887][105620] Updated weights for policy 1, policy_version 759923 (0.0008) [2023-12-26 20:53:38,939][105620] Updated weights for policy 1, policy_version 759933 (0.0009) [2023-12-26 20:53:38,994][105620] Updated weights for policy 1, policy_version 759943 (0.0010) [2023-12-26 20:53:39,395][105692] Updated weights for policy 0, policy_version 759759 (0.0009) [2023-12-26 20:53:39,462][105692] Updated weights for policy 0, policy_version 759769 (0.0006) [2023-12-26 20:53:39,526][105692] Updated weights for policy 0, policy_version 759779 (0.0008) [2023-12-26 20:53:39,783][105620] Updated weights for policy 1, policy_version 759953 (0.0009) [2023-12-26 20:53:39,847][105620] Updated weights for policy 1, policy_version 759963 (0.0009) [2023-12-26 20:53:39,908][105620] Updated weights for policy 1, policy_version 759973 (0.0009) [2023-12-26 20:53:40,275][105692] Updated weights for policy 0, policy_version 759789 (0.0010) [2023-12-26 20:53:40,334][105692] Updated weights for policy 0, policy_version 759799 (0.0009) [2023-12-26 20:53:40,402][105692] Updated weights for policy 0, policy_version 759809 (0.0009) [2023-12-26 20:53:40,635][105620] Updated weights for policy 1, policy_version 759983 (0.0010) [2023-12-26 20:53:40,696][105620] Updated weights for policy 1, policy_version 759993 (0.0009) [2023-12-26 20:53:40,758][105620] Updated weights for policy 1, policy_version 760003 (0.0010) [2023-12-26 20:53:41,005][105692] Updated weights for policy 0, policy_version 759819 (0.0008) [2023-12-26 20:53:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 389128192. Throughput: 0: 9854.2, 1: 9540.4. Samples: 389136692. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-26 20:53:41,062][104569] Avg episode reward: [(0, '8985.699'), (1, '9076.498')] [2023-12-26 20:53:41,074][105692] Updated weights for policy 0, policy_version 759829 (0.0008) [2023-12-26 20:53:41,141][105692] Updated weights for policy 0, policy_version 759839 (0.0008) [2023-12-26 20:53:41,605][105620] Updated weights for policy 1, policy_version 760013 (0.0009) [2023-12-26 20:53:41,676][105620] Updated weights for policy 1, policy_version 760023 (0.0008) [2023-12-26 20:53:41,748][105620] Updated weights for policy 1, policy_version 760033 (0.0009) [2023-12-26 20:53:41,925][105692] Updated weights for policy 0, policy_version 759849 (0.0009) [2023-12-26 20:53:41,984][105692] Updated weights for policy 0, policy_version 759859 (0.0006) [2023-12-26 20:53:42,045][105692] Updated weights for policy 0, policy_version 759869 (0.0007) [2023-12-26 20:53:42,108][105692] Updated weights for policy 0, policy_version 759879 (0.0009) [2023-12-26 20:53:42,494][105620] Updated weights for policy 1, policy_version 760043 (0.0009) [2023-12-26 20:53:42,551][105620] Updated weights for policy 1, policy_version 760053 (0.0008) [2023-12-26 20:53:42,607][105620] Updated weights for policy 1, policy_version 760063 (0.0008) [2023-12-26 20:53:42,904][105692] Updated weights for policy 0, policy_version 759889 (0.0006) [2023-12-26 20:53:42,956][105692] Updated weights for policy 0, policy_version 759899 (0.0005) [2023-12-26 20:53:43,012][105692] Updated weights for policy 0, policy_version 759909 (0.0008) [2023-12-26 20:53:43,343][105620] Updated weights for policy 1, policy_version 760073 (0.0008) [2023-12-26 20:53:43,398][105620] Updated weights for policy 1, policy_version 760083 (0.0009) [2023-12-26 20:53:43,448][105620] Updated weights for policy 1, policy_version 760093 (0.0009) [2023-12-26 20:53:43,502][105620] Updated weights for policy 1, policy_version 760103 (0.0009) [2023-12-26 20:53:43,660][105692] Updated weights for policy 0, policy_version 759919 (0.0009) [2023-12-26 20:53:43,707][105692] Updated weights for policy 0, policy_version 759929 (0.0009) [2023-12-26 20:53:43,754][105692] Updated weights for policy 0, policy_version 759939 (0.0008) [2023-12-26 20:53:44,280][105620] Updated weights for policy 1, policy_version 760113 (0.0006) [2023-12-26 20:53:44,349][105620] Updated weights for policy 1, policy_version 760123 (0.0006) [2023-12-26 20:53:44,408][105620] Updated weights for policy 1, policy_version 760133 (0.0008) [2023-12-26 20:53:44,522][105692] Updated weights for policy 0, policy_version 759949 (0.0009) [2023-12-26 20:53:44,568][105692] Updated weights for policy 0, policy_version 759959 (0.0008) [2023-12-26 20:53:44,618][105692] Updated weights for policy 0, policy_version 759969 (0.0009) [2023-12-26 20:53:45,090][105620] Updated weights for policy 1, policy_version 760143 (0.0008) [2023-12-26 20:53:45,149][105620] Updated weights for policy 1, policy_version 760153 (0.0009) [2023-12-26 20:53:45,212][105620] Updated weights for policy 1, policy_version 760163 (0.0009) [2023-12-26 20:53:45,381][105692] Updated weights for policy 0, policy_version 759979 (0.0009) [2023-12-26 20:53:45,447][105692] Updated weights for policy 0, policy_version 759989 (0.0009) [2023-12-26 20:53:45,508][105692] Updated weights for policy 0, policy_version 759999 (0.0009) [2023-12-26 20:53:45,981][105620] Updated weights for policy 1, policy_version 760173 (0.0010) [2023-12-26 20:53:46,034][105620] Updated weights for policy 1, policy_version 760183 (0.0010) [2023-12-26 20:53:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.3, 300 sec: 19522.0). Total num frames: 389218304. Throughput: 0: 9811.6, 1: 9580.3. Samples: 389194188. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-26 20:53:46,062][104569] Avg episode reward: [(0, '9168.192'), (1, '8803.794')] [2023-12-26 20:53:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000760008_194592768.pth... [2023-12-26 20:53:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000758856_194297856.pth [2023-12-26 20:53:46,086][105620] Updated weights for policy 1, policy_version 760193 (0.0011) [2023-12-26 20:53:46,118][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000760200_194633728.pth... [2023-12-26 20:53:46,121][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000759048_194338816.pth [2023-12-26 20:53:46,268][105692] Updated weights for policy 0, policy_version 760009 (0.0008) [2023-12-26 20:53:46,323][105692] Updated weights for policy 0, policy_version 760019 (0.0005) [2023-12-26 20:53:46,373][105692] Updated weights for policy 0, policy_version 760029 (0.0005) [2023-12-26 20:53:46,435][105692] Updated weights for policy 0, policy_version 760039 (0.0007) [2023-12-26 20:53:46,788][105620] Updated weights for policy 1, policy_version 760203 (0.0009) [2023-12-26 20:53:46,839][105620] Updated weights for policy 1, policy_version 760213 (0.0005) [2023-12-26 20:53:46,894][105620] Updated weights for policy 1, policy_version 760223 (0.0008) [2023-12-26 20:53:47,179][105692] Updated weights for policy 0, policy_version 760049 (0.0008) [2023-12-26 20:53:47,240][105692] Updated weights for policy 0, policy_version 760059 (0.0008) [2023-12-26 20:53:47,293][105692] Updated weights for policy 0, policy_version 760069 (0.0010) [2023-12-26 20:53:47,503][105620] Updated weights for policy 1, policy_version 760233 (0.0010) [2023-12-26 20:53:47,579][105620] Updated weights for policy 1, policy_version 760243 (0.0009) [2023-12-26 20:53:47,646][105620] Updated weights for policy 1, policy_version 760253 (0.0008) [2023-12-26 20:53:47,706][105620] Updated weights for policy 1, policy_version 760263 (0.0007) [2023-12-26 20:53:48,040][105692] Updated weights for policy 0, policy_version 760079 (0.0009) [2023-12-26 20:53:48,091][105692] Updated weights for policy 0, policy_version 760089 (0.0006) [2023-12-26 20:53:48,140][105692] Updated weights for policy 0, policy_version 760099 (0.0005) [2023-12-26 20:53:48,309][105620] Updated weights for policy 1, policy_version 760273 (0.0008) [2023-12-26 20:53:48,384][105620] Updated weights for policy 1, policy_version 760283 (0.0007) [2023-12-26 20:53:48,457][105620] Updated weights for policy 1, policy_version 760293 (0.0005) [2023-12-26 20:53:48,827][105692] Updated weights for policy 0, policy_version 760109 (0.0008) [2023-12-26 20:53:48,885][105692] Updated weights for policy 0, policy_version 760119 (0.0005) [2023-12-26 20:53:48,943][105692] Updated weights for policy 0, policy_version 760129 (0.0007) [2023-12-26 20:53:49,104][105620] Updated weights for policy 1, policy_version 760303 (0.0009) [2023-12-26 20:53:49,151][105620] Updated weights for policy 1, policy_version 760313 (0.0008) [2023-12-26 20:53:49,198][105620] Updated weights for policy 1, policy_version 760323 (0.0009) [2023-12-26 20:53:49,599][105692] Updated weights for policy 0, policy_version 760139 (0.0006) [2023-12-26 20:53:49,658][105692] Updated weights for policy 0, policy_version 760149 (0.0005) [2023-12-26 20:53:49,715][105692] Updated weights for policy 0, policy_version 760159 (0.0008) [2023-12-26 20:53:50,052][105620] Updated weights for policy 1, policy_version 760333 (0.0009) [2023-12-26 20:53:50,111][105620] Updated weights for policy 1, policy_version 760343 (0.0009) [2023-12-26 20:53:50,173][105620] Updated weights for policy 1, policy_version 760353 (0.0010) [2023-12-26 20:53:50,380][105692] Updated weights for policy 0, policy_version 760169 (0.0009) [2023-12-26 20:53:50,440][105692] Updated weights for policy 0, policy_version 760179 (0.0006) [2023-12-26 20:53:50,505][105692] Updated weights for policy 0, policy_version 760189 (0.0007) [2023-12-26 20:53:50,563][105692] Updated weights for policy 0, policy_version 760199 (0.0010) [2023-12-26 20:53:51,013][105620] Updated weights for policy 1, policy_version 760363 (0.0009) [2023-12-26 20:53:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 389316608. Throughput: 0: 9872.7, 1: 9556.9. Samples: 389311272. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-26 20:53:51,062][104569] Avg episode reward: [(0, '9076.374'), (1, '9081.072')] [2023-12-26 20:53:51,079][105620] Updated weights for policy 1, policy_version 760373 (0.0009) [2023-12-26 20:53:51,135][105620] Updated weights for policy 1, policy_version 760383 (0.0008) [2023-12-26 20:53:51,293][105692] Updated weights for policy 0, policy_version 760209 (0.0011) [2023-12-26 20:53:51,358][105692] Updated weights for policy 0, policy_version 760219 (0.0010) [2023-12-26 20:53:51,422][105692] Updated weights for policy 0, policy_version 760229 (0.0011) [2023-12-26 20:53:51,924][105620] Updated weights for policy 1, policy_version 760393 (0.0008) [2023-12-26 20:53:51,984][105620] Updated weights for policy 1, policy_version 760403 (0.0009) [2023-12-26 20:53:52,039][105620] Updated weights for policy 1, policy_version 760413 (0.0009) [2023-12-26 20:53:52,087][105620] Updated weights for policy 1, policy_version 760423 (0.0009) [2023-12-26 20:53:52,147][105692] Updated weights for policy 0, policy_version 760239 (0.0009) [2023-12-26 20:53:52,198][105692] Updated weights for policy 0, policy_version 760249 (0.0009) [2023-12-26 20:53:52,245][105692] Updated weights for policy 0, policy_version 760259 (0.0008) [2023-12-26 20:53:52,887][105620] Updated weights for policy 1, policy_version 760433 (0.0009) [2023-12-26 20:53:52,947][105620] Updated weights for policy 1, policy_version 760443 (0.0009) [2023-12-26 20:53:52,992][105692] Updated weights for policy 0, policy_version 760269 (0.0007) [2023-12-26 20:53:53,012][105620] Updated weights for policy 1, policy_version 760453 (0.0009) [2023-12-26 20:53:53,042][105692] Updated weights for policy 0, policy_version 760279 (0.0008) [2023-12-26 20:53:53,100][105692] Updated weights for policy 0, policy_version 760289 (0.0010) [2023-12-26 20:53:53,667][105692] Updated weights for policy 0, policy_version 760299 (0.0009) [2023-12-26 20:53:53,715][105692] Updated weights for policy 0, policy_version 760309 (0.0008) [2023-12-26 20:53:53,765][105692] Updated weights for policy 0, policy_version 760319 (0.0010) [2023-12-26 20:53:53,831][105620] Updated weights for policy 1, policy_version 760463 (0.0008) [2023-12-26 20:53:53,885][105620] Updated weights for policy 1, policy_version 760474 (0.0010) [2023-12-26 20:53:53,936][105620] Updated weights for policy 1, policy_version 760484 (0.0009) [2023-12-26 20:53:54,333][105692] Updated weights for policy 0, policy_version 760329 (0.0006) [2023-12-26 20:53:54,380][105692] Updated weights for policy 0, policy_version 760339 (0.0010) [2023-12-26 20:53:54,434][105692] Updated weights for policy 0, policy_version 760349 (0.0008) [2023-12-26 20:53:54,480][105692] Updated weights for policy 0, policy_version 760359 (0.0005) [2023-12-26 20:53:54,718][105620] Updated weights for policy 1, policy_version 760495 (0.0009) [2023-12-26 20:53:54,772][105620] Updated weights for policy 1, policy_version 760505 (0.0007) [2023-12-26 20:53:54,838][105620] Updated weights for policy 1, policy_version 760515 (0.0008) [2023-12-26 20:53:55,160][105692] Updated weights for policy 0, policy_version 760369 (0.0010) [2023-12-26 20:53:55,211][105692] Updated weights for policy 0, policy_version 760379 (0.0010) [2023-12-26 20:53:55,267][105692] Updated weights for policy 0, policy_version 760389 (0.0011) [2023-12-26 20:53:55,621][105620] Updated weights for policy 1, policy_version 760525 (0.0008) [2023-12-26 20:53:55,680][105620] Updated weights for policy 1, policy_version 760535 (0.0008) [2023-12-26 20:53:55,734][105620] Updated weights for policy 1, policy_version 760545 (0.0008) [2023-12-26 20:53:55,962][105692] Updated weights for policy 0, policy_version 760399 (0.0007) [2023-12-26 20:53:56,021][105692] Updated weights for policy 0, policy_version 760409 (0.0010) [2023-12-26 20:53:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 389414912. Throughput: 0: 9852.7, 1: 9566.6. Samples: 389425608. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-26 20:53:56,062][104569] Avg episode reward: [(0, '8985.495'), (1, '8994.690')] [2023-12-26 20:53:56,072][105692] Updated weights for policy 0, policy_version 760419 (0.0010) [2023-12-26 20:53:56,426][105620] Updated weights for policy 1, policy_version 760555 (0.0007) [2023-12-26 20:53:56,491][105620] Updated weights for policy 1, policy_version 760565 (0.0005) [2023-12-26 20:53:56,556][105620] Updated weights for policy 1, policy_version 760575 (0.0005) [2023-12-26 20:53:56,793][105692] Updated weights for policy 0, policy_version 760429 (0.0010) [2023-12-26 20:53:56,843][105692] Updated weights for policy 0, policy_version 760439 (0.0010) [2023-12-26 20:53:56,891][105692] Updated weights for policy 0, policy_version 760449 (0.0010) [2023-12-26 20:53:57,051][105620] Updated weights for policy 1, policy_version 760585 (0.0005) [2023-12-26 20:53:57,109][105620] Updated weights for policy 1, policy_version 760595 (0.0005) [2023-12-26 20:53:57,160][105620] Updated weights for policy 1, policy_version 760605 (0.0005) [2023-12-26 20:53:57,213][105620] Updated weights for policy 1, policy_version 760615 (0.0007) [2023-12-26 20:53:57,650][105692] Updated weights for policy 0, policy_version 760459 (0.0010) [2023-12-26 20:53:57,714][105692] Updated weights for policy 0, policy_version 760469 (0.0010) [2023-12-26 20:53:57,768][105692] Updated weights for policy 0, policy_version 760479 (0.0010) [2023-12-26 20:53:57,915][105620] Updated weights for policy 1, policy_version 760625 (0.0008) [2023-12-26 20:53:57,969][105620] Updated weights for policy 1, policy_version 760635 (0.0007) [2023-12-26 20:53:58,025][105620] Updated weights for policy 1, policy_version 760645 (0.0008) [2023-12-26 20:53:58,504][105692] Updated weights for policy 0, policy_version 760489 (0.0010) [2023-12-26 20:53:58,571][105692] Updated weights for policy 0, policy_version 760499 (0.0006) [2023-12-26 20:53:58,639][105692] Updated weights for policy 0, policy_version 760509 (0.0008) [2023-12-26 20:53:58,699][105692] Updated weights for policy 0, policy_version 760519 (0.0007) [2023-12-26 20:53:58,810][105620] Updated weights for policy 1, policy_version 760655 (0.0008) [2023-12-26 20:53:58,873][105620] Updated weights for policy 1, policy_version 760665 (0.0008) [2023-12-26 20:53:58,933][105620] Updated weights for policy 1, policy_version 760675 (0.0007) [2023-12-26 20:53:59,470][105692] Updated weights for policy 0, policy_version 760529 (0.0009) [2023-12-26 20:53:59,528][105692] Updated weights for policy 0, policy_version 760539 (0.0009) [2023-12-26 20:53:59,587][105692] Updated weights for policy 0, policy_version 760549 (0.0009) [2023-12-26 20:53:59,669][105620] Updated weights for policy 1, policy_version 760685 (0.0009) [2023-12-26 20:53:59,714][105620] Updated weights for policy 1, policy_version 760695 (0.0008) [2023-12-26 20:53:59,764][105620] Updated weights for policy 1, policy_version 760705 (0.0008) [2023-12-26 20:54:00,379][105692] Updated weights for policy 0, policy_version 760559 (0.0009) [2023-12-26 20:54:00,437][105692] Updated weights for policy 0, policy_version 760569 (0.0008) [2023-12-26 20:54:00,498][105692] Updated weights for policy 0, policy_version 760579 (0.0009) [2023-12-26 20:54:00,508][105620] Updated weights for policy 1, policy_version 760715 (0.0008) [2023-12-26 20:54:00,557][105620] Updated weights for policy 1, policy_version 760725 (0.0007) [2023-12-26 20:54:00,603][105620] Updated weights for policy 1, policy_version 760735 (0.0009) [2023-12-26 20:54:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 389513216. Throughput: 0: 9892.4, 1: 9584.8. Samples: 389485844. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-26 20:54:01,062][104569] Avg episode reward: [(0, '9260.140'), (1, '8722.702')] [2023-12-26 20:54:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000760584_194740224.pth... [2023-12-26 20:54:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000760744_194772992.pth... [2023-12-26 20:54:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000759464_194453504.pth [2023-12-26 20:54:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000759624_194486272.pth [2023-12-26 20:54:01,271][105620] Updated weights for policy 1, policy_version 760745 (0.0009) [2023-12-26 20:54:01,314][105692] Updated weights for policy 0, policy_version 760589 (0.0007) [2023-12-26 20:54:01,325][105620] Updated weights for policy 1, policy_version 760755 (0.0006) [2023-12-26 20:54:01,375][105692] Updated weights for policy 0, policy_version 760599 (0.0008) [2023-12-26 20:54:01,391][105620] Updated weights for policy 1, policy_version 760765 (0.0007) [2023-12-26 20:54:01,439][105692] Updated weights for policy 0, policy_version 760609 (0.0007) [2023-12-26 20:54:01,453][105620] Updated weights for policy 1, policy_version 760775 (0.0008) [2023-12-26 20:54:02,191][105620] Updated weights for policy 1, policy_version 760785 (0.0009) [2023-12-26 20:54:02,232][105692] Updated weights for policy 0, policy_version 760619 (0.0006) [2023-12-26 20:54:02,247][105620] Updated weights for policy 1, policy_version 760795 (0.0009) [2023-12-26 20:54:02,292][105692] Updated weights for policy 0, policy_version 760629 (0.0007) [2023-12-26 20:54:02,306][105620] Updated weights for policy 1, policy_version 760805 (0.0007) [2023-12-26 20:54:02,347][105692] Updated weights for policy 0, policy_version 760639 (0.0007) [2023-12-26 20:54:03,022][105692] Updated weights for policy 0, policy_version 760649 (0.0009) [2023-12-26 20:54:03,069][105692] Updated weights for policy 0, policy_version 760659 (0.0009) [2023-12-26 20:54:03,112][105620] Updated weights for policy 1, policy_version 760815 (0.0009) [2023-12-26 20:54:03,114][105692] Updated weights for policy 0, policy_version 760669 (0.0007) [2023-12-26 20:54:03,167][105620] Updated weights for policy 1, policy_version 760825 (0.0009) [2023-12-26 20:54:03,168][105692] Updated weights for policy 0, policy_version 760679 (0.0006) [2023-12-26 20:54:03,230][105620] Updated weights for policy 1, policy_version 760835 (0.0008) [2023-12-26 20:54:03,897][105620] Updated weights for policy 1, policy_version 760845 (0.0008) [2023-12-26 20:54:03,917][105692] Updated weights for policy 0, policy_version 760689 (0.0009) [2023-12-26 20:54:03,956][105620] Updated weights for policy 1, policy_version 760855 (0.0009) [2023-12-26 20:54:03,970][105692] Updated weights for policy 0, policy_version 760699 (0.0006) [2023-12-26 20:54:04,013][105620] Updated weights for policy 1, policy_version 760865 (0.0007) [2023-12-26 20:54:04,023][105692] Updated weights for policy 0, policy_version 760709 (0.0006) [2023-12-26 20:54:04,775][105620] Updated weights for policy 1, policy_version 760875 (0.0007) [2023-12-26 20:54:04,784][105692] Updated weights for policy 0, policy_version 760719 (0.0008) [2023-12-26 20:54:04,824][105620] Updated weights for policy 1, policy_version 760885 (0.0008) [2023-12-26 20:54:04,846][105692] Updated weights for policy 0, policy_version 760729 (0.0007) [2023-12-26 20:54:04,876][105620] Updated weights for policy 1, policy_version 760895 (0.0006) [2023-12-26 20:54:04,905][105692] Updated weights for policy 0, policy_version 760739 (0.0006) [2023-12-26 20:54:05,607][105620] Updated weights for policy 1, policy_version 760905 (0.0006) [2023-12-26 20:54:05,667][105620] Updated weights for policy 1, policy_version 760915 (0.0009) [2023-12-26 20:54:05,672][105692] Updated weights for policy 0, policy_version 760749 (0.0006) [2023-12-26 20:54:05,716][105620] Updated weights for policy 1, policy_version 760925 (0.0009) [2023-12-26 20:54:05,724][105692] Updated weights for policy 0, policy_version 760759 (0.0005) [2023-12-26 20:54:05,765][105620] Updated weights for policy 1, policy_version 760935 (0.0008) [2023-12-26 20:54:05,779][105692] Updated weights for policy 0, policy_version 760769 (0.0005) [2023-12-26 20:54:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 389611520. Throughput: 0: 9807.5, 1: 9574.9. Samples: 389598068. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-26 20:54:06,062][104569] Avg episode reward: [(0, '9169.128'), (1, '8893.343')] [2023-12-26 20:54:06,456][105692] Updated weights for policy 0, policy_version 760779 (0.0006) [2023-12-26 20:54:06,519][105692] Updated weights for policy 0, policy_version 760789 (0.0009) [2023-12-26 20:54:06,572][105620] Updated weights for policy 1, policy_version 760945 (0.0009) [2023-12-26 20:54:06,576][105692] Updated weights for policy 0, policy_version 760799 (0.0007) [2023-12-26 20:54:06,627][105620] Updated weights for policy 1, policy_version 760955 (0.0008) [2023-12-26 20:54:06,684][105620] Updated weights for policy 1, policy_version 760965 (0.0008) [2023-12-26 20:54:07,248][105692] Updated weights for policy 0, policy_version 760809 (0.0008) [2023-12-26 20:54:07,316][105692] Updated weights for policy 0, policy_version 760819 (0.0005) [2023-12-26 20:54:07,370][105692] Updated weights for policy 0, policy_version 760829 (0.0005) [2023-12-26 20:54:07,420][105692] Updated weights for policy 0, policy_version 760839 (0.0005) [2023-12-26 20:54:07,476][105620] Updated weights for policy 1, policy_version 760975 (0.0008) [2023-12-26 20:54:07,536][105620] Updated weights for policy 1, policy_version 760985 (0.0007) [2023-12-26 20:54:07,585][105620] Updated weights for policy 1, policy_version 760995 (0.0005) [2023-12-26 20:54:07,955][105692] Updated weights for policy 0, policy_version 760849 (0.0007) [2023-12-26 20:54:08,001][105692] Updated weights for policy 0, policy_version 760859 (0.0009) [2023-12-26 20:54:08,048][105692] Updated weights for policy 0, policy_version 760869 (0.0008) [2023-12-26 20:54:08,234][105620] Updated weights for policy 1, policy_version 761005 (0.0007) [2023-12-26 20:54:08,282][105620] Updated weights for policy 1, policy_version 761015 (0.0009) [2023-12-26 20:54:08,336][105620] Updated weights for policy 1, policy_version 761025 (0.0009) [2023-12-26 20:54:08,808][105692] Updated weights for policy 0, policy_version 760879 (0.0009) [2023-12-26 20:54:08,861][105692] Updated weights for policy 0, policy_version 760889 (0.0009) [2023-12-26 20:54:08,919][105692] Updated weights for policy 0, policy_version 760899 (0.0009) [2023-12-26 20:54:09,085][105620] Updated weights for policy 1, policy_version 761035 (0.0007) [2023-12-26 20:54:09,131][105620] Updated weights for policy 1, policy_version 761045 (0.0005) [2023-12-26 20:54:09,175][105620] Updated weights for policy 1, policy_version 761055 (0.0005) [2023-12-26 20:54:09,688][105692] Updated weights for policy 0, policy_version 760909 (0.0008) [2023-12-26 20:54:09,745][105692] Updated weights for policy 0, policy_version 760919 (0.0010) [2023-12-26 20:54:09,792][105692] Updated weights for policy 0, policy_version 760929 (0.0008) [2023-12-26 20:54:09,906][105620] Updated weights for policy 1, policy_version 761065 (0.0006) [2023-12-26 20:54:09,971][105620] Updated weights for policy 1, policy_version 761075 (0.0010) [2023-12-26 20:54:10,031][105620] Updated weights for policy 1, policy_version 761085 (0.0009) [2023-12-26 20:54:10,090][105620] Updated weights for policy 1, policy_version 761095 (0.0009) [2023-12-26 20:54:10,551][105692] Updated weights for policy 0, policy_version 760939 (0.0009) [2023-12-26 20:54:10,609][105692] Updated weights for policy 0, policy_version 760949 (0.0009) [2023-12-26 20:54:10,667][105692] Updated weights for policy 0, policy_version 760959 (0.0008) [2023-12-26 20:54:10,888][105620] Updated weights for policy 1, policy_version 761105 (0.0009) [2023-12-26 20:54:10,944][105620] Updated weights for policy 1, policy_version 761115 (0.0009) [2023-12-26 20:54:10,996][105620] Updated weights for policy 1, policy_version 761125 (0.0009) [2023-12-26 20:54:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19549.8). Total num frames: 389709824. Throughput: 0: 9861.1, 1: 9599.9. Samples: 389714064. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-26 20:54:11,062][104569] Avg episode reward: [(0, '9080.197'), (1, '9071.677')] [2023-12-26 20:54:11,470][105692] Updated weights for policy 0, policy_version 760969 (0.0009) [2023-12-26 20:54:11,528][105692] Updated weights for policy 0, policy_version 760979 (0.0009) [2023-12-26 20:54:11,583][105692] Updated weights for policy 0, policy_version 760989 (0.0009) [2023-12-26 20:54:11,646][105692] Updated weights for policy 0, policy_version 760999 (0.0009) [2023-12-26 20:54:11,825][105620] Updated weights for policy 1, policy_version 761135 (0.0009) [2023-12-26 20:54:11,887][105620] Updated weights for policy 1, policy_version 761145 (0.0009) [2023-12-26 20:54:11,944][105620] Updated weights for policy 1, policy_version 761155 (0.0009) [2023-12-26 20:54:12,375][105692] Updated weights for policy 0, policy_version 761009 (0.0007) [2023-12-26 20:54:12,438][105692] Updated weights for policy 0, policy_version 761019 (0.0009) [2023-12-26 20:54:12,500][105692] Updated weights for policy 0, policy_version 761029 (0.0008) [2023-12-26 20:54:12,771][105620] Updated weights for policy 1, policy_version 761165 (0.0009) [2023-12-26 20:54:12,837][105620] Updated weights for policy 1, policy_version 761175 (0.0008) [2023-12-26 20:54:12,899][105620] Updated weights for policy 1, policy_version 761185 (0.0009) [2023-12-26 20:54:13,195][105692] Updated weights for policy 0, policy_version 761039 (0.0009) [2023-12-26 20:54:13,243][105692] Updated weights for policy 0, policy_version 761049 (0.0009) [2023-12-26 20:54:13,302][105692] Updated weights for policy 0, policy_version 761059 (0.0008) [2023-12-26 20:54:13,610][105620] Updated weights for policy 1, policy_version 761195 (0.0009) [2023-12-26 20:54:13,669][105620] Updated weights for policy 1, policy_version 761205 (0.0010) [2023-12-26 20:54:13,723][105620] Updated weights for policy 1, policy_version 761215 (0.0009) [2023-12-26 20:54:13,999][105692] Updated weights for policy 0, policy_version 761069 (0.0009) [2023-12-26 20:54:14,057][105692] Updated weights for policy 0, policy_version 761079 (0.0007) [2023-12-26 20:54:14,113][105692] Updated weights for policy 0, policy_version 761089 (0.0011) [2023-12-26 20:54:14,345][105620] Updated weights for policy 1, policy_version 761225 (0.0010) [2023-12-26 20:54:14,410][105620] Updated weights for policy 1, policy_version 761235 (0.0005) [2023-12-26 20:54:14,462][105620] Updated weights for policy 1, policy_version 761245 (0.0005) [2023-12-26 20:54:14,516][105620] Updated weights for policy 1, policy_version 761255 (0.0005) [2023-12-26 20:54:14,837][105692] Updated weights for policy 0, policy_version 761099 (0.0008) [2023-12-26 20:54:14,899][105692] Updated weights for policy 0, policy_version 761109 (0.0006) [2023-12-26 20:54:14,959][105692] Updated weights for policy 0, policy_version 761119 (0.0007) [2023-12-26 20:54:15,244][105620] Updated weights for policy 1, policy_version 761265 (0.0009) [2023-12-26 20:54:15,300][105620] Updated weights for policy 1, policy_version 761275 (0.0008) [2023-12-26 20:54:15,352][105620] Updated weights for policy 1, policy_version 761285 (0.0010) [2023-12-26 20:54:15,550][105692] Updated weights for policy 0, policy_version 761129 (0.0006) [2023-12-26 20:54:15,614][105692] Updated weights for policy 0, policy_version 761139 (0.0009) [2023-12-26 20:54:15,667][105692] Updated weights for policy 0, policy_version 761149 (0.0007) [2023-12-26 20:54:15,719][105692] Updated weights for policy 0, policy_version 761159 (0.0005) [2023-12-26 20:54:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.8, 300 sec: 19521.9). Total num frames: 389799936. Throughput: 0: 9729.0, 1: 9473.4. Samples: 389768252. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-26 20:54:16,063][104569] Avg episode reward: [(0, '8903.785'), (1, '9008.954')] [2023-12-26 20:54:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000761160_194887680.pth... [2023-12-26 20:54:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000761288_194912256.pth... [2023-12-26 20:54:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000760008_194592768.pth [2023-12-26 20:54:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000760200_194633728.pth [2023-12-26 20:54:16,234][105620] Updated weights for policy 1, policy_version 761295 (0.0009) [2023-12-26 20:54:16,296][105620] Updated weights for policy 1, policy_version 761305 (0.0009) [2023-12-26 20:54:16,320][105692] Updated weights for policy 0, policy_version 761169 (0.0005) [2023-12-26 20:54:16,355][105620] Updated weights for policy 1, policy_version 761315 (0.0008) [2023-12-26 20:54:16,383][105692] Updated weights for policy 0, policy_version 761179 (0.0005) [2023-12-26 20:54:16,447][105692] Updated weights for policy 0, policy_version 761189 (0.0005) [2023-12-26 20:54:17,096][105692] Updated weights for policy 0, policy_version 761199 (0.0008) [2023-12-26 20:54:17,102][105620] Updated weights for policy 1, policy_version 761325 (0.0007) [2023-12-26 20:54:17,147][105620] Updated weights for policy 1, policy_version 761335 (0.0005) [2023-12-26 20:54:17,153][105692] Updated weights for policy 0, policy_version 761209 (0.0007) [2023-12-26 20:54:17,194][105620] Updated weights for policy 1, policy_version 761345 (0.0006) [2023-12-26 20:54:17,211][105692] Updated weights for policy 0, policy_version 761219 (0.0006) [2023-12-26 20:54:17,812][105692] Updated weights for policy 0, policy_version 761229 (0.0006) [2023-12-26 20:54:17,867][105692] Updated weights for policy 0, policy_version 761239 (0.0005) [2023-12-26 20:54:17,926][105692] Updated weights for policy 0, policy_version 761249 (0.0006) [2023-12-26 20:54:18,062][105620] Updated weights for policy 1, policy_version 761355 (0.0009) [2023-12-26 20:54:18,131][105620] Updated weights for policy 1, policy_version 761365 (0.0010) [2023-12-26 20:54:18,186][105620] Updated weights for policy 1, policy_version 761375 (0.0006) [2023-12-26 20:54:18,511][105692] Updated weights for policy 0, policy_version 761259 (0.0007) [2023-12-26 20:54:18,573][105692] Updated weights for policy 0, policy_version 761269 (0.0009) [2023-12-26 20:54:18,625][105692] Updated weights for policy 0, policy_version 761279 (0.0009) [2023-12-26 20:54:18,900][105620] Updated weights for policy 1, policy_version 761385 (0.0009) [2023-12-26 20:54:18,961][105620] Updated weights for policy 1, policy_version 761395 (0.0009) [2023-12-26 20:54:19,014][105620] Updated weights for policy 1, policy_version 761405 (0.0008) [2023-12-26 20:54:19,071][105620] Updated weights for policy 1, policy_version 761415 (0.0009) [2023-12-26 20:54:19,405][105692] Updated weights for policy 0, policy_version 761289 (0.0009) [2023-12-26 20:54:19,458][105692] Updated weights for policy 0, policy_version 761299 (0.0010) [2023-12-26 20:54:19,523][105692] Updated weights for policy 0, policy_version 761309 (0.0009) [2023-12-26 20:54:19,576][105692] Updated weights for policy 0, policy_version 761319 (0.0009) [2023-12-26 20:54:19,864][105620] Updated weights for policy 1, policy_version 761425 (0.0010) [2023-12-26 20:54:19,924][105620] Updated weights for policy 1, policy_version 761435 (0.0009) [2023-12-26 20:54:19,987][105620] Updated weights for policy 1, policy_version 761445 (0.0009) [2023-12-26 20:54:20,366][105692] Updated weights for policy 0, policy_version 761329 (0.0010) [2023-12-26 20:54:20,422][105692] Updated weights for policy 0, policy_version 761339 (0.0010) [2023-12-26 20:54:20,475][105692] Updated weights for policy 0, policy_version 761349 (0.0010) [2023-12-26 20:54:20,623][105620] Updated weights for policy 1, policy_version 761455 (0.0008) [2023-12-26 20:54:20,674][105620] Updated weights for policy 1, policy_version 761465 (0.0008) [2023-12-26 20:54:20,744][105620] Updated weights for policy 1, policy_version 761475 (0.0006) [2023-12-26 20:54:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 389898240. Throughput: 0: 9860.6, 1: 9415.0. Samples: 389887036. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-26 20:54:21,062][104569] Avg episode reward: [(0, '8993.423'), (1, '8735.554')] [2023-12-26 20:54:21,284][105692] Updated weights for policy 0, policy_version 761359 (0.0007) [2023-12-26 20:54:21,351][105692] Updated weights for policy 0, policy_version 761369 (0.0008) [2023-12-26 20:54:21,361][105620] Updated weights for policy 1, policy_version 761485 (0.0008) [2023-12-26 20:54:21,417][105692] Updated weights for policy 0, policy_version 761379 (0.0007) [2023-12-26 20:54:21,451][105620] Updated weights for policy 1, policy_version 761495 (0.0009) [2023-12-26 20:54:21,512][105620] Updated weights for policy 1, policy_version 761505 (0.0011) [2023-12-26 20:54:22,190][105692] Updated weights for policy 0, policy_version 761389 (0.0007) [2023-12-26 20:54:22,224][105620] Updated weights for policy 1, policy_version 761515 (0.0009) [2023-12-26 20:54:22,262][105692] Updated weights for policy 0, policy_version 761399 (0.0006) [2023-12-26 20:54:22,283][105620] Updated weights for policy 1, policy_version 761525 (0.0007) [2023-12-26 20:54:22,319][105692] Updated weights for policy 0, policy_version 761409 (0.0006) [2023-12-26 20:54:22,343][105620] Updated weights for policy 1, policy_version 761535 (0.0008) [2023-12-26 20:54:22,931][105692] Updated weights for policy 0, policy_version 761419 (0.0008) [2023-12-26 20:54:22,989][105692] Updated weights for policy 0, policy_version 761429 (0.0009) [2023-12-26 20:54:23,045][105692] Updated weights for policy 0, policy_version 761439 (0.0009) [2023-12-26 20:54:23,097][105620] Updated weights for policy 1, policy_version 761545 (0.0008) [2023-12-26 20:54:23,153][105620] Updated weights for policy 1, policy_version 761555 (0.0009) [2023-12-26 20:54:23,204][105620] Updated weights for policy 1, policy_version 761565 (0.0010) [2023-12-26 20:54:23,259][105620] Updated weights for policy 1, policy_version 761575 (0.0010) [2023-12-26 20:54:23,833][105692] Updated weights for policy 0, policy_version 761449 (0.0009) [2023-12-26 20:54:23,891][105692] Updated weights for policy 0, policy_version 761459 (0.0008) [2023-12-26 20:54:23,947][105692] Updated weights for policy 0, policy_version 761469 (0.0008) [2023-12-26 20:54:23,999][105620] Updated weights for policy 1, policy_version 761585 (0.0010) [2023-12-26 20:54:24,002][105692] Updated weights for policy 0, policy_version 761479 (0.0008) [2023-12-26 20:54:24,057][105620] Updated weights for policy 1, policy_version 761595 (0.0010) [2023-12-26 20:54:24,118][105620] Updated weights for policy 1, policy_version 761605 (0.0010) [2023-12-26 20:54:24,756][105692] Updated weights for policy 0, policy_version 761489 (0.0010) [2023-12-26 20:54:24,820][105692] Updated weights for policy 0, policy_version 761499 (0.0010) [2023-12-26 20:54:24,835][105620] Updated weights for policy 1, policy_version 761615 (0.0007) [2023-12-26 20:54:24,873][105692] Updated weights for policy 0, policy_version 761509 (0.0009) [2023-12-26 20:54:24,895][105620] Updated weights for policy 1, policy_version 761625 (0.0007) [2023-12-26 20:54:24,955][105620] Updated weights for policy 1, policy_version 761635 (0.0008) [2023-12-26 20:54:25,464][105620] Updated weights for policy 1, policy_version 761645 (0.0005) [2023-12-26 20:54:25,524][105620] Updated weights for policy 1, policy_version 761655 (0.0009) [2023-12-26 20:54:25,538][105692] Updated weights for policy 0, policy_version 761519 (0.0009) [2023-12-26 20:54:25,570][105620] Updated weights for policy 1, policy_version 761665 (0.0005) [2023-12-26 20:54:25,584][105692] Updated weights for policy 0, policy_version 761529 (0.0010) [2023-12-26 20:54:25,632][105692] Updated weights for policy 0, policy_version 761539 (0.0010) [2023-12-26 20:54:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 389996544. Throughput: 0: 9791.1, 1: 9541.9. Samples: 390006680. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-26 20:54:26,063][104569] Avg episode reward: [(0, '8991.891'), (1, '8892.068')] [2023-12-26 20:54:26,179][105620] Updated weights for policy 1, policy_version 761675 (0.0007) [2023-12-26 20:54:26,234][105620] Updated weights for policy 1, policy_version 761685 (0.0010) [2023-12-26 20:54:26,286][105620] Updated weights for policy 1, policy_version 761695 (0.0011) [2023-12-26 20:54:26,407][105692] Updated weights for policy 0, policy_version 761549 (0.0010) [2023-12-26 20:54:26,466][105692] Updated weights for policy 0, policy_version 761559 (0.0011) [2023-12-26 20:54:26,531][105692] Updated weights for policy 0, policy_version 761569 (0.0010) [2023-12-26 20:54:27,021][105620] Updated weights for policy 1, policy_version 761705 (0.0010) [2023-12-26 20:54:27,084][105620] Updated weights for policy 1, policy_version 761715 (0.0010) [2023-12-26 20:54:27,139][105620] Updated weights for policy 1, policy_version 761725 (0.0010) [2023-12-26 20:54:27,165][105692] Updated weights for policy 0, policy_version 761579 (0.0009) [2023-12-26 20:54:27,202][105620] Updated weights for policy 1, policy_version 761735 (0.0009) [2023-12-26 20:54:27,219][105692] Updated weights for policy 0, policy_version 761589 (0.0005) [2023-12-26 20:54:27,268][105692] Updated weights for policy 0, policy_version 761599 (0.0005) [2023-12-26 20:54:27,841][105620] Updated weights for policy 1, policy_version 761745 (0.0008) [2023-12-26 20:54:27,875][105692] Updated weights for policy 0, policy_version 761609 (0.0007) [2023-12-26 20:54:27,900][105620] Updated weights for policy 1, policy_version 761755 (0.0006) [2023-12-26 20:54:27,922][105692] Updated weights for policy 0, policy_version 761619 (0.0010) [2023-12-26 20:54:27,948][105620] Updated weights for policy 1, policy_version 761765 (0.0005) [2023-12-26 20:54:27,967][105692] Updated weights for policy 0, policy_version 761629 (0.0010) [2023-12-26 20:54:28,031][105692] Updated weights for policy 0, policy_version 761639 (0.0010) [2023-12-26 20:54:28,712][105620] Updated weights for policy 1, policy_version 761775 (0.0007) [2023-12-26 20:54:28,760][105620] Updated weights for policy 1, policy_version 761785 (0.0008) [2023-12-26 20:54:28,803][105692] Updated weights for policy 0, policy_version 761649 (0.0011) [2023-12-26 20:54:28,816][105620] Updated weights for policy 1, policy_version 761795 (0.0006) [2023-12-26 20:54:28,851][105692] Updated weights for policy 0, policy_version 761659 (0.0010) [2023-12-26 20:54:28,906][105692] Updated weights for policy 0, policy_version 761669 (0.0011) [2023-12-26 20:54:29,579][105692] Updated weights for policy 0, policy_version 761679 (0.0010) [2023-12-26 20:54:29,634][105692] Updated weights for policy 0, policy_version 761689 (0.0010) [2023-12-26 20:54:29,663][105620] Updated weights for policy 1, policy_version 761805 (0.0006) [2023-12-26 20:54:29,685][105692] Updated weights for policy 0, policy_version 761699 (0.0010) [2023-12-26 20:54:29,718][105620] Updated weights for policy 1, policy_version 761815 (0.0005) [2023-12-26 20:54:29,779][105620] Updated weights for policy 1, policy_version 761825 (0.0008) [2023-12-26 20:54:30,358][105692] Updated weights for policy 0, policy_version 761709 (0.0009) [2023-12-26 20:54:30,412][105692] Updated weights for policy 0, policy_version 761719 (0.0009) [2023-12-26 20:54:30,464][105692] Updated weights for policy 0, policy_version 761729 (0.0009) [2023-12-26 20:54:30,557][105620] Updated weights for policy 1, policy_version 761835 (0.0009) [2023-12-26 20:54:30,613][105620] Updated weights for policy 1, policy_version 761845 (0.0008) [2023-12-26 20:54:30,671][105620] Updated weights for policy 1, policy_version 761856 (0.0010) [2023-12-26 20:54:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 390094848. Throughput: 0: 9803.2, 1: 9572.1. Samples: 390066076. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-26 20:54:31,063][104569] Avg episode reward: [(0, '9079.456'), (1, '9166.153')] [2023-12-26 20:54:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000761736_195035136.pth... [2023-12-26 20:54:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000761864_195059712.pth... [2023-12-26 20:54:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000760744_194772992.pth [2023-12-26 20:54:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000760584_194740224.pth [2023-12-26 20:54:31,166][105692] Updated weights for policy 0, policy_version 761739 (0.0010) [2023-12-26 20:54:31,230][105692] Updated weights for policy 0, policy_version 761749 (0.0010) [2023-12-26 20:54:31,289][105692] Updated weights for policy 0, policy_version 761759 (0.0011) [2023-12-26 20:54:31,491][105620] Updated weights for policy 1, policy_version 761866 (0.0009) [2023-12-26 20:54:31,543][105620] Updated weights for policy 1, policy_version 761876 (0.0008) [2023-12-26 20:54:31,597][105620] Updated weights for policy 1, policy_version 761886 (0.0007) [2023-12-26 20:54:31,660][105620] Updated weights for policy 1, policy_version 761896 (0.0007) [2023-12-26 20:54:32,038][105692] Updated weights for policy 0, policy_version 761769 (0.0010) [2023-12-26 20:54:32,086][105692] Updated weights for policy 0, policy_version 761779 (0.0010) [2023-12-26 20:54:32,139][105692] Updated weights for policy 0, policy_version 761789 (0.0011) [2023-12-26 20:54:32,188][105692] Updated weights for policy 0, policy_version 761799 (0.0010) [2023-12-26 20:54:32,423][105620] Updated weights for policy 1, policy_version 761906 (0.0008) [2023-12-26 20:54:32,484][105620] Updated weights for policy 1, policy_version 761916 (0.0008) [2023-12-26 20:54:32,544][105620] Updated weights for policy 1, policy_version 761926 (0.0008) [2023-12-26 20:54:33,002][105692] Updated weights for policy 0, policy_version 761809 (0.0010) [2023-12-26 20:54:33,066][105692] Updated weights for policy 0, policy_version 761819 (0.0010) [2023-12-26 20:54:33,117][105692] Updated weights for policy 0, policy_version 761829 (0.0009) [2023-12-26 20:54:33,287][105620] Updated weights for policy 1, policy_version 761936 (0.0008) [2023-12-26 20:54:33,338][105620] Updated weights for policy 1, policy_version 761946 (0.0007) [2023-12-26 20:54:33,393][105620] Updated weights for policy 1, policy_version 761956 (0.0008) [2023-12-26 20:54:33,857][105692] Updated weights for policy 0, policy_version 761839 (0.0010) [2023-12-26 20:54:33,909][105692] Updated weights for policy 0, policy_version 761849 (0.0010) [2023-12-26 20:54:33,958][105692] Updated weights for policy 0, policy_version 761859 (0.0010) [2023-12-26 20:54:34,140][105620] Updated weights for policy 1, policy_version 761966 (0.0008) [2023-12-26 20:54:34,200][105620] Updated weights for policy 1, policy_version 761976 (0.0009) [2023-12-26 20:54:34,264][105620] Updated weights for policy 1, policy_version 761986 (0.0008) [2023-12-26 20:54:34,733][105692] Updated weights for policy 0, policy_version 761869 (0.0011) [2023-12-26 20:54:34,792][105692] Updated weights for policy 0, policy_version 761879 (0.0010) [2023-12-26 20:54:34,844][105692] Updated weights for policy 0, policy_version 761889 (0.0005) [2023-12-26 20:54:35,017][105620] Updated weights for policy 1, policy_version 761996 (0.0009) [2023-12-26 20:54:35,079][105620] Updated weights for policy 1, policy_version 762006 (0.0010) [2023-12-26 20:54:35,132][105620] Updated weights for policy 1, policy_version 762016 (0.0010) [2023-12-26 20:54:35,393][105692] Updated weights for policy 0, policy_version 761899 (0.0006) [2023-12-26 20:54:35,456][105692] Updated weights for policy 0, policy_version 761909 (0.0005) [2023-12-26 20:54:35,512][105692] Updated weights for policy 0, policy_version 761919 (0.0005) [2023-12-26 20:54:35,998][105620] Updated weights for policy 1, policy_version 762026 (0.0009) [2023-12-26 20:54:36,046][105620] Updated weights for policy 1, policy_version 762036 (0.0007) [2023-12-26 20:54:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 390184960. Throughput: 0: 9811.3, 1: 9466.9. Samples: 390178792. Policy #0 lag: (min: 27.0, avg: 41.5, max: 59.0) [2023-12-26 20:54:36,063][104569] Avg episode reward: [(0, '8987.719'), (1, '9165.223')] [2023-12-26 20:54:36,110][105692] Updated weights for policy 0, policy_version 761929 (0.0006) [2023-12-26 20:54:36,113][105620] Updated weights for policy 1, policy_version 762046 (0.0006) [2023-12-26 20:54:36,168][105692] Updated weights for policy 0, policy_version 761939 (0.0008) [2023-12-26 20:54:36,180][105620] Updated weights for policy 1, policy_version 762056 (0.0008) [2023-12-26 20:54:36,213][105692] Updated weights for policy 0, policy_version 761949 (0.0006) [2023-12-26 20:54:36,260][105692] Updated weights for policy 0, policy_version 761959 (0.0007) [2023-12-26 20:54:36,946][105692] Updated weights for policy 0, policy_version 761969 (0.0008) [2023-12-26 20:54:36,988][105620] Updated weights for policy 1, policy_version 762066 (0.0009) [2023-12-26 20:54:37,007][105692] Updated weights for policy 0, policy_version 761979 (0.0007) [2023-12-26 20:54:37,052][105620] Updated weights for policy 1, policy_version 762076 (0.0007) [2023-12-26 20:54:37,059][105692] Updated weights for policy 0, policy_version 761989 (0.0005) [2023-12-26 20:54:37,111][105620] Updated weights for policy 1, policy_version 762086 (0.0008) [2023-12-26 20:54:37,739][105692] Updated weights for policy 0, policy_version 761999 (0.0008) [2023-12-26 20:54:37,797][105692] Updated weights for policy 0, policy_version 762009 (0.0008) [2023-12-26 20:54:37,862][105692] Updated weights for policy 0, policy_version 762019 (0.0010) [2023-12-26 20:54:37,897][105620] Updated weights for policy 1, policy_version 762096 (0.0006) [2023-12-26 20:54:37,957][105620] Updated weights for policy 1, policy_version 762106 (0.0005) [2023-12-26 20:54:38,020][105620] Updated weights for policy 1, policy_version 762116 (0.0006) [2023-12-26 20:54:38,555][105620] Updated weights for policy 1, policy_version 762126 (0.0005) [2023-12-26 20:54:38,615][105620] Updated weights for policy 1, policy_version 762136 (0.0008) [2023-12-26 20:54:38,621][105692] Updated weights for policy 0, policy_version 762029 (0.0007) [2023-12-26 20:54:38,664][105620] Updated weights for policy 1, policy_version 762146 (0.0010) [2023-12-26 20:54:38,674][105692] Updated weights for policy 0, policy_version 762039 (0.0006) [2023-12-26 20:54:38,735][105692] Updated weights for policy 0, policy_version 762049 (0.0008) [2023-12-26 20:54:39,359][105692] Updated weights for policy 0, policy_version 762059 (0.0008) [2023-12-26 20:54:39,395][105620] Updated weights for policy 1, policy_version 762156 (0.0010) [2023-12-26 20:54:39,427][105692] Updated weights for policy 0, policy_version 762069 (0.0007) [2023-12-26 20:54:39,460][105620] Updated weights for policy 1, policy_version 762166 (0.0009) [2023-12-26 20:54:39,487][105692] Updated weights for policy 0, policy_version 762079 (0.0007) [2023-12-26 20:54:39,519][105620] Updated weights for policy 1, policy_version 762176 (0.0010) [2023-12-26 20:54:40,226][105692] Updated weights for policy 0, policy_version 762089 (0.0007) [2023-12-26 20:54:40,279][105692] Updated weights for policy 0, policy_version 762099 (0.0008) [2023-12-26 20:54:40,283][105620] Updated weights for policy 1, policy_version 762186 (0.0010) [2023-12-26 20:54:40,332][105692] Updated weights for policy 0, policy_version 762109 (0.0009) [2023-12-26 20:54:40,339][105620] Updated weights for policy 1, policy_version 762196 (0.0009) [2023-12-26 20:54:40,388][105692] Updated weights for policy 0, policy_version 762119 (0.0007) [2023-12-26 20:54:40,396][105620] Updated weights for policy 1, policy_version 762206 (0.0007) [2023-12-26 20:54:40,455][105620] Updated weights for policy 1, policy_version 762216 (0.0005) [2023-12-26 20:54:41,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19251.1, 300 sec: 19521.9). Total num frames: 390283264. Throughput: 0: 9802.1, 1: 9557.8. Samples: 390296812. Policy #0 lag: (min: 27.0, avg: 41.5, max: 59.0) [2023-12-26 20:54:41,063][104569] Avg episode reward: [(0, '8986.987'), (1, '9078.385')] [2023-12-26 20:54:41,084][105620] Updated weights for policy 1, policy_version 762226 (0.0009) [2023-12-26 20:54:41,155][105620] Updated weights for policy 1, policy_version 762236 (0.0008) [2023-12-26 20:54:41,216][105620] Updated weights for policy 1, policy_version 762246 (0.0007) [2023-12-26 20:54:41,235][105692] Updated weights for policy 0, policy_version 762129 (0.0008) [2023-12-26 20:54:41,302][105692] Updated weights for policy 0, policy_version 762139 (0.0009) [2023-12-26 20:54:41,369][105692] Updated weights for policy 0, policy_version 762149 (0.0009) [2023-12-26 20:54:42,000][105620] Updated weights for policy 1, policy_version 762256 (0.0009) [2023-12-26 20:54:42,054][105620] Updated weights for policy 1, policy_version 762266 (0.0008) [2023-12-26 20:54:42,114][105620] Updated weights for policy 1, policy_version 762276 (0.0008) [2023-12-26 20:54:42,138][105692] Updated weights for policy 0, policy_version 762159 (0.0008) [2023-12-26 20:54:42,188][105692] Updated weights for policy 0, policy_version 762169 (0.0009) [2023-12-26 20:54:42,239][105692] Updated weights for policy 0, policy_version 762179 (0.0009) [2023-12-26 20:54:42,844][105620] Updated weights for policy 1, policy_version 762286 (0.0009) [2023-12-26 20:54:42,898][105620] Updated weights for policy 1, policy_version 762296 (0.0009) [2023-12-26 20:54:42,961][105620] Updated weights for policy 1, policy_version 762306 (0.0009) [2023-12-26 20:54:43,012][105692] Updated weights for policy 0, policy_version 762189 (0.0010) [2023-12-26 20:54:43,073][105692] Updated weights for policy 0, policy_version 762199 (0.0008) [2023-12-26 20:54:43,136][105692] Updated weights for policy 0, policy_version 762209 (0.0006) [2023-12-26 20:54:43,699][105692] Updated weights for policy 0, policy_version 762219 (0.0006) [2023-12-26 20:54:43,759][105692] Updated weights for policy 0, policy_version 762229 (0.0008) [2023-12-26 20:54:43,761][105620] Updated weights for policy 1, policy_version 762316 (0.0008) [2023-12-26 20:54:43,814][105620] Updated weights for policy 1, policy_version 762326 (0.0007) [2023-12-26 20:54:43,816][105692] Updated weights for policy 0, policy_version 762239 (0.0007) [2023-12-26 20:54:43,862][105620] Updated weights for policy 1, policy_version 762336 (0.0008) [2023-12-26 20:54:44,542][105692] Updated weights for policy 0, policy_version 762249 (0.0006) [2023-12-26 20:54:44,597][105692] Updated weights for policy 0, policy_version 762259 (0.0010) [2023-12-26 20:54:44,607][105620] Updated weights for policy 1, policy_version 762346 (0.0008) [2023-12-26 20:54:44,652][105692] Updated weights for policy 0, policy_version 762269 (0.0010) [2023-12-26 20:54:44,669][105620] Updated weights for policy 1, policy_version 762356 (0.0005) [2023-12-26 20:54:44,708][105692] Updated weights for policy 0, policy_version 762279 (0.0010) [2023-12-26 20:54:44,732][105620] Updated weights for policy 1, policy_version 762366 (0.0005) [2023-12-26 20:54:44,794][105620] Updated weights for policy 1, policy_version 762376 (0.0007) [2023-12-26 20:54:45,421][105620] Updated weights for policy 1, policy_version 762386 (0.0008) [2023-12-26 20:54:45,464][105692] Updated weights for policy 0, policy_version 762289 (0.0009) [2023-12-26 20:54:45,474][105620] Updated weights for policy 1, policy_version 762396 (0.0008) [2023-12-26 20:54:45,525][105692] Updated weights for policy 0, policy_version 762299 (0.0006) [2023-12-26 20:54:45,527][105620] Updated weights for policy 1, policy_version 762406 (0.0008) [2023-12-26 20:54:45,571][105692] Updated weights for policy 0, policy_version 762309 (0.0005) [2023-12-26 20:54:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 390381568. Throughput: 0: 9794.4, 1: 9465.5. Samples: 390352540. Policy #0 lag: (min: 27.0, avg: 41.5, max: 59.0) [2023-12-26 20:54:46,063][104569] Avg episode reward: [(0, '9259.497'), (1, '8991.962')] [2023-12-26 20:54:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000762312_195182592.pth... [2023-12-26 20:54:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000762408_195198976.pth... [2023-12-26 20:54:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000761160_194887680.pth [2023-12-26 20:54:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000761288_194912256.pth [2023-12-26 20:54:46,142][105620] Updated weights for policy 1, policy_version 762416 (0.0006) [2023-12-26 20:54:46,174][105692] Updated weights for policy 0, policy_version 762319 (0.0006) [2023-12-26 20:54:46,204][105620] Updated weights for policy 1, policy_version 762426 (0.0008) [2023-12-26 20:54:46,235][105692] Updated weights for policy 0, policy_version 762329 (0.0006) [2023-12-26 20:54:46,261][105620] Updated weights for policy 1, policy_version 762436 (0.0008) [2023-12-26 20:54:46,298][105692] Updated weights for policy 0, policy_version 762339 (0.0007) [2023-12-26 20:54:46,930][105692] Updated weights for policy 0, policy_version 762349 (0.0007) [2023-12-26 20:54:46,987][105692] Updated weights for policy 0, policy_version 762359 (0.0009) [2023-12-26 20:54:47,025][105620] Updated weights for policy 1, policy_version 762446 (0.0009) [2023-12-26 20:54:47,044][105692] Updated weights for policy 0, policy_version 762369 (0.0008) [2023-12-26 20:54:47,088][105620] Updated weights for policy 1, policy_version 762457 (0.0007) [2023-12-26 20:54:47,142][105620] Updated weights for policy 1, policy_version 762467 (0.0005) [2023-12-26 20:54:47,624][105692] Updated weights for policy 0, policy_version 762379 (0.0006) [2023-12-26 20:54:47,673][105692] Updated weights for policy 0, policy_version 762389 (0.0008) [2023-12-26 20:54:47,723][105620] Updated weights for policy 1, policy_version 762477 (0.0006) [2023-12-26 20:54:47,724][105692] Updated weights for policy 0, policy_version 762399 (0.0008) [2023-12-26 20:54:47,771][105620] Updated weights for policy 1, policy_version 762487 (0.0005) [2023-12-26 20:54:47,819][105620] Updated weights for policy 1, policy_version 762497 (0.0009) [2023-12-26 20:54:48,450][105692] Updated weights for policy 0, policy_version 762409 (0.0008) [2023-12-26 20:54:48,506][105692] Updated weights for policy 0, policy_version 762419 (0.0008) [2023-12-26 20:54:48,538][105620] Updated weights for policy 1, policy_version 762507 (0.0010) [2023-12-26 20:54:48,552][105692] Updated weights for policy 0, policy_version 762429 (0.0006) [2023-12-26 20:54:48,586][105620] Updated weights for policy 1, policy_version 762517 (0.0010) [2023-12-26 20:54:48,612][105692] Updated weights for policy 0, policy_version 762439 (0.0005) [2023-12-26 20:54:48,641][105620] Updated weights for policy 1, policy_version 762527 (0.0005) [2023-12-26 20:54:49,265][105692] Updated weights for policy 0, policy_version 762449 (0.0007) [2023-12-26 20:54:49,331][105692] Updated weights for policy 0, policy_version 762459 (0.0006) [2023-12-26 20:54:49,357][105620] Updated weights for policy 1, policy_version 762537 (0.0005) [2023-12-26 20:54:49,399][105692] Updated weights for policy 0, policy_version 762469 (0.0008) [2023-12-26 20:54:49,422][105620] Updated weights for policy 1, policy_version 762547 (0.0008) [2023-12-26 20:54:49,477][105620] Updated weights for policy 1, policy_version 762557 (0.0009) [2023-12-26 20:54:49,529][105620] Updated weights for policy 1, policy_version 762567 (0.0010) [2023-12-26 20:54:50,051][105692] Updated weights for policy 0, policy_version 762479 (0.0008) [2023-12-26 20:54:50,104][105692] Updated weights for policy 0, policy_version 762489 (0.0008) [2023-12-26 20:54:50,159][105692] Updated weights for policy 0, policy_version 762499 (0.0008) [2023-12-26 20:54:50,332][105620] Updated weights for policy 1, policy_version 762577 (0.0007) [2023-12-26 20:54:50,403][105620] Updated weights for policy 1, policy_version 762587 (0.0005) [2023-12-26 20:54:50,459][105620] Updated weights for policy 1, policy_version 762597 (0.0007) [2023-12-26 20:54:51,017][105620] Updated weights for policy 1, policy_version 762607 (0.0006) [2023-12-26 20:54:51,034][105692] Updated weights for policy 0, policy_version 762509 (0.0009) [2023-12-26 20:54:51,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 390479872. Throughput: 0: 9954.7, 1: 9550.1. Samples: 390475780. Policy #0 lag: (min: 27.0, avg: 41.5, max: 59.0) [2023-12-26 20:54:51,063][104569] Avg episode reward: [(0, '9348.715'), (1, '9169.300')] [2023-12-26 20:54:51,086][105620] Updated weights for policy 1, policy_version 762617 (0.0007) [2023-12-26 20:54:51,100][105692] Updated weights for policy 0, policy_version 762519 (0.0014) [2023-12-26 20:54:51,143][105620] Updated weights for policy 1, policy_version 762627 (0.0006) [2023-12-26 20:54:51,168][105692] Updated weights for policy 0, policy_version 762529 (0.0008) [2023-12-26 20:54:51,831][105620] Updated weights for policy 1, policy_version 762637 (0.0009) [2023-12-26 20:54:51,861][105692] Updated weights for policy 0, policy_version 762539 (0.0005) [2023-12-26 20:54:51,884][105620] Updated weights for policy 1, policy_version 762647 (0.0009) [2023-12-26 20:54:51,919][105692] Updated weights for policy 0, policy_version 762549 (0.0008) [2023-12-26 20:54:51,942][105620] Updated weights for policy 1, policy_version 762657 (0.0005) [2023-12-26 20:54:51,979][105692] Updated weights for policy 0, policy_version 762559 (0.0008) [2023-12-26 20:54:52,631][105620] Updated weights for policy 1, policy_version 762667 (0.0007) [2023-12-26 20:54:52,685][105620] Updated weights for policy 1, policy_version 762677 (0.0010) [2023-12-26 20:54:52,740][105692] Updated weights for policy 0, policy_version 762569 (0.0008) [2023-12-26 20:54:52,742][105620] Updated weights for policy 1, policy_version 762687 (0.0010) [2023-12-26 20:54:52,793][105692] Updated weights for policy 0, policy_version 762579 (0.0005) [2023-12-26 20:54:52,841][105692] Updated weights for policy 0, policy_version 762589 (0.0008) [2023-12-26 20:54:52,900][105692] Updated weights for policy 0, policy_version 762599 (0.0008) [2023-12-26 20:54:53,489][105620] Updated weights for policy 1, policy_version 762697 (0.0011) [2023-12-26 20:54:53,549][105620] Updated weights for policy 1, policy_version 762707 (0.0011) [2023-12-26 20:54:53,600][105620] Updated weights for policy 1, policy_version 762717 (0.0008) [2023-12-26 20:54:53,659][105620] Updated weights for policy 1, policy_version 762727 (0.0006) [2023-12-26 20:54:53,678][105692] Updated weights for policy 0, policy_version 762609 (0.0008) [2023-12-26 20:54:53,734][105692] Updated weights for policy 0, policy_version 762619 (0.0008) [2023-12-26 20:54:53,787][105692] Updated weights for policy 0, policy_version 762629 (0.0009) [2023-12-26 20:54:54,292][105620] Updated weights for policy 1, policy_version 762737 (0.0006) [2023-12-26 20:54:54,360][105620] Updated weights for policy 1, policy_version 762747 (0.0007) [2023-12-26 20:54:54,411][105692] Updated weights for policy 0, policy_version 762639 (0.0007) [2023-12-26 20:54:54,423][105620] Updated weights for policy 1, policy_version 762757 (0.0008) [2023-12-26 20:54:54,466][105692] Updated weights for policy 0, policy_version 762649 (0.0009) [2023-12-26 20:54:54,522][105692] Updated weights for policy 0, policy_version 762659 (0.0010) [2023-12-26 20:54:55,029][105620] Updated weights for policy 1, policy_version 762767 (0.0007) [2023-12-26 20:54:55,083][105620] Updated weights for policy 1, policy_version 762777 (0.0007) [2023-12-26 20:54:55,141][105620] Updated weights for policy 1, policy_version 762787 (0.0009) [2023-12-26 20:54:55,292][105692] Updated weights for policy 0, policy_version 762669 (0.0010) [2023-12-26 20:54:55,338][105692] Updated weights for policy 0, policy_version 762679 (0.0009) [2023-12-26 20:54:55,389][105692] Updated weights for policy 0, policy_version 762689 (0.0009) [2023-12-26 20:54:55,771][105620] Updated weights for policy 1, policy_version 762797 (0.0007) [2023-12-26 20:54:55,818][105620] Updated weights for policy 1, policy_version 762807 (0.0005) [2023-12-26 20:54:55,868][105620] Updated weights for policy 1, policy_version 762817 (0.0006) [2023-12-26 20:54:56,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 390586368. Throughput: 0: 9871.0, 1: 9677.0. Samples: 390593724. Policy #0 lag: (min: 27.0, avg: 41.5, max: 59.0) [2023-12-26 20:54:56,063][104569] Avg episode reward: [(0, '9257.839'), (1, '9075.318')] [2023-12-26 20:54:56,119][105692] Updated weights for policy 0, policy_version 762699 (0.0008) [2023-12-26 20:54:56,173][105692] Updated weights for policy 0, policy_version 762709 (0.0006) [2023-12-26 20:54:56,226][105692] Updated weights for policy 0, policy_version 762719 (0.0005) [2023-12-26 20:54:56,545][105620] Updated weights for policy 1, policy_version 762827 (0.0008) [2023-12-26 20:54:56,601][105620] Updated weights for policy 1, policy_version 762837 (0.0005) [2023-12-26 20:54:56,657][105620] Updated weights for policy 1, policy_version 762847 (0.0005) [2023-12-26 20:54:56,876][105692] Updated weights for policy 0, policy_version 762729 (0.0007) [2023-12-26 20:54:56,922][105692] Updated weights for policy 0, policy_version 762739 (0.0007) [2023-12-26 20:54:56,975][105692] Updated weights for policy 0, policy_version 762749 (0.0005) [2023-12-26 20:54:57,035][105692] Updated weights for policy 0, policy_version 762759 (0.0010) [2023-12-26 20:54:57,289][105620] Updated weights for policy 1, policy_version 762857 (0.0005) [2023-12-26 20:54:57,341][105620] Updated weights for policy 1, policy_version 762867 (0.0008) [2023-12-26 20:54:57,396][105620] Updated weights for policy 1, policy_version 762877 (0.0005) [2023-12-26 20:54:57,447][105620] Updated weights for policy 1, policy_version 762887 (0.0005) [2023-12-26 20:54:57,717][105692] Updated weights for policy 0, policy_version 762769 (0.0007) [2023-12-26 20:54:57,775][105692] Updated weights for policy 0, policy_version 762779 (0.0010) [2023-12-26 20:54:57,837][105692] Updated weights for policy 0, policy_version 762789 (0.0007) [2023-12-26 20:54:57,984][105620] Updated weights for policy 1, policy_version 762897 (0.0007) [2023-12-26 20:54:58,048][105620] Updated weights for policy 1, policy_version 762907 (0.0008) [2023-12-26 20:54:58,121][105620] Updated weights for policy 1, policy_version 762917 (0.0008) [2023-12-26 20:54:58,444][105692] Updated weights for policy 0, policy_version 762799 (0.0008) [2023-12-26 20:54:58,507][105692] Updated weights for policy 0, policy_version 762809 (0.0011) [2023-12-26 20:54:58,571][105692] Updated weights for policy 0, policy_version 762819 (0.0010) [2023-12-26 20:54:58,853][105620] Updated weights for policy 1, policy_version 762927 (0.0008) [2023-12-26 20:54:58,924][105620] Updated weights for policy 1, policy_version 762937 (0.0013) [2023-12-26 20:54:58,999][105620] Updated weights for policy 1, policy_version 762947 (0.0008) [2023-12-26 20:54:59,329][105692] Updated weights for policy 0, policy_version 762829 (0.0009) [2023-12-26 20:54:59,412][105692] Updated weights for policy 0, policy_version 762842 (0.0008) [2023-12-26 20:54:59,469][105692] Updated weights for policy 0, policy_version 762852 (0.0011) [2023-12-26 20:54:59,796][105620] Updated weights for policy 1, policy_version 762957 (0.0009) [2023-12-26 20:54:59,856][105620] Updated weights for policy 1, policy_version 762967 (0.0010) [2023-12-26 20:54:59,915][105620] Updated weights for policy 1, policy_version 762977 (0.0010) [2023-12-26 20:55:00,148][105692] Updated weights for policy 0, policy_version 762862 (0.0010) [2023-12-26 20:55:00,213][105692] Updated weights for policy 0, policy_version 762872 (0.0010) [2023-12-26 20:55:00,280][105692] Updated weights for policy 0, policy_version 762882 (0.0010) [2023-12-26 20:55:00,607][105620] Updated weights for policy 1, policy_version 762987 (0.0009) [2023-12-26 20:55:00,665][105620] Updated weights for policy 1, policy_version 762997 (0.0005) [2023-12-26 20:55:00,718][105620] Updated weights for policy 1, policy_version 763007 (0.0006) [2023-12-26 20:55:00,914][105692] Updated weights for policy 0, policy_version 762892 (0.0007) [2023-12-26 20:55:00,968][105692] Updated weights for policy 0, policy_version 762902 (0.0005) [2023-12-26 20:55:01,018][105692] Updated weights for policy 0, policy_version 762912 (0.0005) [2023-12-26 20:55:01,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 390684672. Throughput: 0: 9969.4, 1: 9784.1. Samples: 390657160. Policy #0 lag: (min: 27.0, avg: 41.5, max: 59.0) [2023-12-26 20:55:01,063][104569] Avg episode reward: [(0, '9348.401'), (1, '8814.412')] [2023-12-26 20:55:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000763016_195354624.pth... [2023-12-26 20:55:01,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000762920_195338240.pth... [2023-12-26 20:55:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000761864_195059712.pth [2023-12-26 20:55:01,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000761736_195035136.pth [2023-12-26 20:55:01,344][105620] Updated weights for policy 1, policy_version 763017 (0.0008) [2023-12-26 20:55:01,404][105620] Updated weights for policy 1, policy_version 763027 (0.0011) [2023-12-26 20:55:01,469][105620] Updated weights for policy 1, policy_version 763037 (0.0009) [2023-12-26 20:55:01,531][105620] Updated weights for policy 1, policy_version 763047 (0.0006) [2023-12-26 20:55:01,733][105692] Updated weights for policy 0, policy_version 762922 (0.0007) [2023-12-26 20:55:01,795][105692] Updated weights for policy 0, policy_version 762932 (0.0008) [2023-12-26 20:55:01,853][105692] Updated weights for policy 0, policy_version 762942 (0.0008) [2023-12-26 20:55:01,908][105692] Updated weights for policy 0, policy_version 762952 (0.0008) [2023-12-26 20:55:02,285][105620] Updated weights for policy 1, policy_version 763057 (0.0009) [2023-12-26 20:55:02,342][105620] Updated weights for policy 1, policy_version 763067 (0.0009) [2023-12-26 20:55:02,411][105620] Updated weights for policy 1, policy_version 763077 (0.0010) [2023-12-26 20:55:02,596][105692] Updated weights for policy 0, policy_version 762962 (0.0010) [2023-12-26 20:55:02,658][105692] Updated weights for policy 0, policy_version 762972 (0.0008) [2023-12-26 20:55:02,708][105692] Updated weights for policy 0, policy_version 762982 (0.0010) [2023-12-26 20:55:03,164][105620] Updated weights for policy 1, policy_version 763087 (0.0010) [2023-12-26 20:55:03,232][105620] Updated weights for policy 1, policy_version 763097 (0.0010) [2023-12-26 20:55:03,259][105692] Updated weights for policy 0, policy_version 762992 (0.0006) [2023-12-26 20:55:03,293][105620] Updated weights for policy 1, policy_version 763107 (0.0010) [2023-12-26 20:55:03,316][105692] Updated weights for policy 0, policy_version 763002 (0.0005) [2023-12-26 20:55:03,378][105692] Updated weights for policy 0, policy_version 763012 (0.0007) [2023-12-26 20:55:03,955][105620] Updated weights for policy 1, policy_version 763117 (0.0009) [2023-12-26 20:55:04,010][105692] Updated weights for policy 0, policy_version 763022 (0.0007) [2023-12-26 20:55:04,022][105620] Updated weights for policy 1, policy_version 763127 (0.0007) [2023-12-26 20:55:04,073][105692] Updated weights for policy 0, policy_version 763032 (0.0008) [2023-12-26 20:55:04,088][105620] Updated weights for policy 1, policy_version 763137 (0.0007) [2023-12-26 20:55:04,136][105692] Updated weights for policy 0, policy_version 763042 (0.0010) [2023-12-26 20:55:04,757][105620] Updated weights for policy 1, policy_version 763147 (0.0007) [2023-12-26 20:55:04,806][105620] Updated weights for policy 1, policy_version 763157 (0.0006) [2023-12-26 20:55:04,853][105692] Updated weights for policy 0, policy_version 763052 (0.0007) [2023-12-26 20:55:04,872][105620] Updated weights for policy 1, policy_version 763167 (0.0008) [2023-12-26 20:55:04,909][105692] Updated weights for policy 0, policy_version 763062 (0.0005) [2023-12-26 20:55:04,972][105692] Updated weights for policy 0, policy_version 763072 (0.0006) [2023-12-26 20:55:05,554][105692] Updated weights for policy 0, policy_version 763082 (0.0006) [2023-12-26 20:55:05,620][105692] Updated weights for policy 0, policy_version 763092 (0.0005) [2023-12-26 20:55:05,685][105692] Updated weights for policy 0, policy_version 763102 (0.0009) [2023-12-26 20:55:05,694][105620] Updated weights for policy 1, policy_version 763177 (0.0009) [2023-12-26 20:55:05,744][105692] Updated weights for policy 0, policy_version 763112 (0.0010) [2023-12-26 20:55:05,746][105620] Updated weights for policy 1, policy_version 763187 (0.0006) [2023-12-26 20:55:05,795][105620] Updated weights for policy 1, policy_version 763197 (0.0006) [2023-12-26 20:55:05,856][105620] Updated weights for policy 1, policy_version 763207 (0.0005) [2023-12-26 20:55:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 390791168. Throughput: 0: 9923.7, 1: 9858.1. Samples: 390777220. Policy #0 lag: (min: 27.0, avg: 41.5, max: 59.0) [2023-12-26 20:55:06,062][104569] Avg episode reward: [(0, '9256.381'), (1, '9176.553')] [2023-12-26 20:55:06,324][105692] Updated weights for policy 0, policy_version 763122 (0.0011) [2023-12-26 20:55:06,384][105692] Updated weights for policy 0, policy_version 763132 (0.0010) [2023-12-26 20:55:06,441][105692] Updated weights for policy 0, policy_version 763142 (0.0010) [2023-12-26 20:55:06,552][105620] Updated weights for policy 1, policy_version 763217 (0.0008) [2023-12-26 20:55:06,605][105620] Updated weights for policy 1, policy_version 763227 (0.0009) [2023-12-26 20:55:06,662][105620] Updated weights for policy 1, policy_version 763237 (0.0008) [2023-12-26 20:55:07,194][105692] Updated weights for policy 0, policy_version 763152 (0.0008) [2023-12-26 20:55:07,262][105692] Updated weights for policy 0, policy_version 763162 (0.0009) [2023-12-26 20:55:07,316][105692] Updated weights for policy 0, policy_version 763172 (0.0006) [2023-12-26 20:55:07,456][105620] Updated weights for policy 1, policy_version 763248 (0.0009) [2023-12-26 20:55:07,509][105620] Updated weights for policy 1, policy_version 763258 (0.0009) [2023-12-26 20:55:07,572][105620] Updated weights for policy 1, policy_version 763268 (0.0009) [2023-12-26 20:55:07,965][105692] Updated weights for policy 0, policy_version 763182 (0.0006) [2023-12-26 20:55:08,026][105692] Updated weights for policy 0, policy_version 763192 (0.0006) [2023-12-26 20:55:08,072][105692] Updated weights for policy 0, policy_version 763202 (0.0005) [2023-12-26 20:55:08,378][105620] Updated weights for policy 1, policy_version 763278 (0.0008) [2023-12-26 20:55:08,434][105620] Updated weights for policy 1, policy_version 763288 (0.0008) [2023-12-26 20:55:08,491][105620] Updated weights for policy 1, policy_version 763298 (0.0008) [2023-12-26 20:55:08,723][105692] Updated weights for policy 0, policy_version 763212 (0.0007) [2023-12-26 20:55:08,785][105692] Updated weights for policy 0, policy_version 763222 (0.0009) [2023-12-26 20:55:08,849][105692] Updated weights for policy 0, policy_version 763232 (0.0009) [2023-12-26 20:55:09,270][105620] Updated weights for policy 1, policy_version 763308 (0.0007) [2023-12-26 20:55:09,335][105620] Updated weights for policy 1, policy_version 763318 (0.0006) [2023-12-26 20:55:09,403][105620] Updated weights for policy 1, policy_version 763328 (0.0008) [2023-12-26 20:55:09,488][105692] Updated weights for policy 0, policy_version 763242 (0.0008) [2023-12-26 20:55:09,544][105692] Updated weights for policy 0, policy_version 763252 (0.0008) [2023-12-26 20:55:09,597][105692] Updated weights for policy 0, policy_version 763262 (0.0008) [2023-12-26 20:55:09,649][105692] Updated weights for policy 0, policy_version 763272 (0.0008) [2023-12-26 20:55:10,137][105620] Updated weights for policy 1, policy_version 763338 (0.0009) [2023-12-26 20:55:10,196][105620] Updated weights for policy 1, policy_version 763348 (0.0009) [2023-12-26 20:55:10,258][105620] Updated weights for policy 1, policy_version 763358 (0.0009) [2023-12-26 20:55:10,319][105620] Updated weights for policy 1, policy_version 763368 (0.0009) [2023-12-26 20:55:10,449][105692] Updated weights for policy 0, policy_version 763282 (0.0009) [2023-12-26 20:55:10,496][105692] Updated weights for policy 0, policy_version 763292 (0.0009) [2023-12-26 20:55:10,547][105692] Updated weights for policy 0, policy_version 763302 (0.0009) [2023-12-26 20:55:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 390881280. Throughput: 0: 10022.8, 1: 9668.6. Samples: 390892792. Policy #0 lag: (min: 27.0, avg: 41.5, max: 59.0) [2023-12-26 20:55:11,062][104569] Avg episode reward: [(0, '9256.003'), (1, '9083.575')] [2023-12-26 20:55:11,087][105620] Updated weights for policy 1, policy_version 763378 (0.0010) [2023-12-26 20:55:11,156][105620] Updated weights for policy 1, policy_version 763388 (0.0009) [2023-12-26 20:55:11,217][105620] Updated weights for policy 1, policy_version 763398 (0.0009) [2023-12-26 20:55:11,278][105692] Updated weights for policy 0, policy_version 763312 (0.0009) [2023-12-26 20:55:11,344][105692] Updated weights for policy 0, policy_version 763322 (0.0009) [2023-12-26 20:55:11,411][105692] Updated weights for policy 0, policy_version 763332 (0.0010) [2023-12-26 20:55:11,955][105620] Updated weights for policy 1, policy_version 763408 (0.0009) [2023-12-26 20:55:12,008][105620] Updated weights for policy 1, policy_version 763418 (0.0009) [2023-12-26 20:55:12,066][105620] Updated weights for policy 1, policy_version 763428 (0.0009) [2023-12-26 20:55:12,171][105692] Updated weights for policy 0, policy_version 763342 (0.0008) [2023-12-26 20:55:12,220][105692] Updated weights for policy 0, policy_version 763352 (0.0009) [2023-12-26 20:55:12,290][105692] Updated weights for policy 0, policy_version 763362 (0.0008) [2023-12-26 20:55:12,819][105620] Updated weights for policy 1, policy_version 763438 (0.0007) [2023-12-26 20:55:12,867][105620] Updated weights for policy 1, policy_version 763448 (0.0005) [2023-12-26 20:55:12,923][105620] Updated weights for policy 1, policy_version 763458 (0.0006) [2023-12-26 20:55:12,934][105692] Updated weights for policy 0, policy_version 763372 (0.0009) [2023-12-26 20:55:12,985][105692] Updated weights for policy 0, policy_version 763382 (0.0008) [2023-12-26 20:55:13,042][105692] Updated weights for policy 0, policy_version 763392 (0.0010) [2023-12-26 20:55:13,536][105620] Updated weights for policy 1, policy_version 763468 (0.0007) [2023-12-26 20:55:13,591][105620] Updated weights for policy 1, policy_version 763478 (0.0008) [2023-12-26 20:55:13,642][105620] Updated weights for policy 1, policy_version 763488 (0.0008) [2023-12-26 20:55:13,711][105692] Updated weights for policy 0, policy_version 763402 (0.0005) [2023-12-26 20:55:13,769][105692] Updated weights for policy 0, policy_version 763412 (0.0005) [2023-12-26 20:55:13,837][105692] Updated weights for policy 0, policy_version 763422 (0.0005) [2023-12-26 20:55:13,897][105692] Updated weights for policy 0, policy_version 763432 (0.0005) [2023-12-26 20:55:14,409][105620] Updated weights for policy 1, policy_version 763498 (0.0008) [2023-12-26 20:55:14,424][105692] Updated weights for policy 0, policy_version 763442 (0.0006) [2023-12-26 20:55:14,466][105620] Updated weights for policy 1, policy_version 763508 (0.0006) [2023-12-26 20:55:14,481][105692] Updated weights for policy 0, policy_version 763452 (0.0009) [2023-12-26 20:55:14,526][105620] Updated weights for policy 1, policy_version 763518 (0.0005) [2023-12-26 20:55:14,540][105692] Updated weights for policy 0, policy_version 763462 (0.0010) [2023-12-26 20:55:14,578][105620] Updated weights for policy 1, policy_version 763528 (0.0007) [2023-12-26 20:55:15,257][105692] Updated weights for policy 0, policy_version 763472 (0.0011) [2023-12-26 20:55:15,257][105620] Updated weights for policy 1, policy_version 763538 (0.0010) [2023-12-26 20:55:15,320][105692] Updated weights for policy 0, policy_version 763482 (0.0011) [2023-12-26 20:55:15,323][105620] Updated weights for policy 1, policy_version 763548 (0.0009) [2023-12-26 20:55:15,372][105692] Updated weights for policy 0, policy_version 763492 (0.0010) [2023-12-26 20:55:15,380][105620] Updated weights for policy 1, policy_version 763558 (0.0011) [2023-12-26 20:55:16,052][105692] Updated weights for policy 0, policy_version 763502 (0.0008) [2023-12-26 20:55:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 390979584. Throughput: 0: 10014.0, 1: 9684.0. Samples: 390952484. Policy #0 lag: (min: 27.0, avg: 41.5, max: 59.0) [2023-12-26 20:55:16,063][104569] Avg episode reward: [(0, '9258.437'), (1, '8993.625')] [2023-12-26 20:55:16,089][105620] Updated weights for policy 1, policy_version 763568 (0.0011) [2023-12-26 20:55:16,111][105692] Updated weights for policy 0, policy_version 763512 (0.0006) [2023-12-26 20:55:16,145][105620] Updated weights for policy 1, policy_version 763578 (0.0011) [2023-12-26 20:55:16,167][105692] Updated weights for policy 0, policy_version 763522 (0.0005) [2023-12-26 20:55:16,195][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000763528_195493888.pth... [2023-12-26 20:55:16,198][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000762312_195182592.pth [2023-12-26 20:55:16,200][105620] Updated weights for policy 1, policy_version 763588 (0.0009) [2023-12-26 20:55:16,221][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000763592_195502080.pth... [2023-12-26 20:55:16,226][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000762408_195198976.pth [2023-12-26 20:55:16,828][105692] Updated weights for policy 0, policy_version 763532 (0.0007) [2023-12-26 20:55:16,892][105692] Updated weights for policy 0, policy_version 763542 (0.0006) [2023-12-26 20:55:16,945][105692] Updated weights for policy 0, policy_version 763552 (0.0007) [2023-12-26 20:55:16,980][105620] Updated weights for policy 1, policy_version 763598 (0.0005) [2023-12-26 20:55:17,046][105620] Updated weights for policy 1, policy_version 763608 (0.0005) [2023-12-26 20:55:17,108][105620] Updated weights for policy 1, policy_version 763618 (0.0005) [2023-12-26 20:55:17,580][105692] Updated weights for policy 0, policy_version 763562 (0.0007) [2023-12-26 20:55:17,638][105692] Updated weights for policy 0, policy_version 763572 (0.0005) [2023-12-26 20:55:17,696][105692] Updated weights for policy 0, policy_version 763582 (0.0006) [2023-12-26 20:55:17,756][105692] Updated weights for policy 0, policy_version 763592 (0.0006) [2023-12-26 20:55:17,826][105620] Updated weights for policy 1, policy_version 763629 (0.0008) [2023-12-26 20:55:17,884][105620] Updated weights for policy 1, policy_version 763639 (0.0010) [2023-12-26 20:55:17,933][105620] Updated weights for policy 1, policy_version 763650 (0.0008) [2023-12-26 20:55:18,374][105692] Updated weights for policy 0, policy_version 763602 (0.0008) [2023-12-26 20:55:18,433][105692] Updated weights for policy 0, policy_version 763612 (0.0008) [2023-12-26 20:55:18,488][105692] Updated weights for policy 0, policy_version 763622 (0.0008) [2023-12-26 20:55:18,752][105620] Updated weights for policy 1, policy_version 763660 (0.0009) [2023-12-26 20:55:18,812][105620] Updated weights for policy 1, policy_version 763670 (0.0009) [2023-12-26 20:55:18,870][105620] Updated weights for policy 1, policy_version 763680 (0.0008) [2023-12-26 20:55:19,090][105692] Updated weights for policy 0, policy_version 763632 (0.0007) [2023-12-26 20:55:19,135][105692] Updated weights for policy 0, policy_version 763642 (0.0010) [2023-12-26 20:55:19,186][105692] Updated weights for policy 0, policy_version 763652 (0.0010) [2023-12-26 20:55:19,586][105620] Updated weights for policy 1, policy_version 763690 (0.0008) [2023-12-26 20:55:19,641][105620] Updated weights for policy 1, policy_version 763700 (0.0008) [2023-12-26 20:55:19,699][105620] Updated weights for policy 1, policy_version 763710 (0.0008) [2023-12-26 20:55:19,758][105620] Updated weights for policy 1, policy_version 763720 (0.0008) [2023-12-26 20:55:19,982][105692] Updated weights for policy 0, policy_version 763662 (0.0011) [2023-12-26 20:55:20,036][105692] Updated weights for policy 0, policy_version 763672 (0.0011) [2023-12-26 20:55:20,089][105692] Updated weights for policy 0, policy_version 763682 (0.0011) [2023-12-26 20:55:20,439][105620] Updated weights for policy 1, policy_version 763730 (0.0007) [2023-12-26 20:55:20,501][105620] Updated weights for policy 1, policy_version 763740 (0.0005) [2023-12-26 20:55:20,560][105620] Updated weights for policy 1, policy_version 763750 (0.0006) [2023-12-26 20:55:20,863][105692] Updated weights for policy 0, policy_version 763692 (0.0010) [2023-12-26 20:55:20,926][105692] Updated weights for policy 0, policy_version 763702 (0.0009) [2023-12-26 20:55:20,993][105692] Updated weights for policy 0, policy_version 763712 (0.0009) [2023-12-26 20:55:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 391086080. Throughput: 0: 10135.1, 1: 9741.3. Samples: 391073228. Policy #0 lag: (min: 27.0, avg: 41.5, max: 59.0) [2023-12-26 20:55:21,063][104569] Avg episode reward: [(0, '9258.365'), (1, '9083.547')] [2023-12-26 20:55:21,267][105620] Updated weights for policy 1, policy_version 763760 (0.0007) [2023-12-26 20:55:21,322][105620] Updated weights for policy 1, policy_version 763770 (0.0008) [2023-12-26 20:55:21,388][105620] Updated weights for policy 1, policy_version 763780 (0.0008) [2023-12-26 20:55:21,758][105692] Updated weights for policy 0, policy_version 763722 (0.0008) [2023-12-26 20:55:21,806][105692] Updated weights for policy 0, policy_version 763732 (0.0005) [2023-12-26 20:55:21,863][105692] Updated weights for policy 0, policy_version 763742 (0.0005) [2023-12-26 20:55:21,923][105692] Updated weights for policy 0, policy_version 763752 (0.0005) [2023-12-26 20:55:22,146][105620] Updated weights for policy 1, policy_version 763790 (0.0008) [2023-12-26 20:55:22,206][105620] Updated weights for policy 1, policy_version 763800 (0.0008) [2023-12-26 20:55:22,260][105620] Updated weights for policy 1, policy_version 763810 (0.0008) [2023-12-26 20:55:22,633][105692] Updated weights for policy 0, policy_version 763762 (0.0008) [2023-12-26 20:55:22,695][105692] Updated weights for policy 0, policy_version 763772 (0.0010) [2023-12-26 20:55:22,755][105692] Updated weights for policy 0, policy_version 763782 (0.0009) [2023-12-26 20:55:23,011][105620] Updated weights for policy 1, policy_version 763820 (0.0008) [2023-12-26 20:55:23,061][105620] Updated weights for policy 1, policy_version 763830 (0.0008) [2023-12-26 20:55:23,106][105620] Updated weights for policy 1, policy_version 763840 (0.0008) [2023-12-26 20:55:23,471][105692] Updated weights for policy 0, policy_version 763792 (0.0011) [2023-12-26 20:55:23,520][105692] Updated weights for policy 0, policy_version 763802 (0.0010) [2023-12-26 20:55:23,568][105692] Updated weights for policy 0, policy_version 763812 (0.0010) [2023-12-26 20:55:23,839][105620] Updated weights for policy 1, policy_version 763850 (0.0007) [2023-12-26 20:55:23,891][105620] Updated weights for policy 1, policy_version 763860 (0.0005) [2023-12-26 20:55:23,950][105620] Updated weights for policy 1, policy_version 763870 (0.0005) [2023-12-26 20:55:24,006][105620] Updated weights for policy 1, policy_version 763880 (0.0005) [2023-12-26 20:55:24,269][105692] Updated weights for policy 0, policy_version 763822 (0.0009) [2023-12-26 20:55:24,330][105692] Updated weights for policy 0, policy_version 763832 (0.0008) [2023-12-26 20:55:24,379][105692] Updated weights for policy 0, policy_version 763842 (0.0005) [2023-12-26 20:55:24,586][105620] Updated weights for policy 1, policy_version 763890 (0.0006) [2023-12-26 20:55:24,644][105620] Updated weights for policy 1, policy_version 763900 (0.0005) [2023-12-26 20:55:24,702][105620] Updated weights for policy 1, policy_version 763910 (0.0005) [2023-12-26 20:55:25,154][105692] Updated weights for policy 0, policy_version 763852 (0.0007) [2023-12-26 20:55:25,214][105620] Updated weights for policy 1, policy_version 763920 (0.0010) [2023-12-26 20:55:25,217][105692] Updated weights for policy 0, policy_version 763862 (0.0006) [2023-12-26 20:55:25,268][105620] Updated weights for policy 1, policy_version 763930 (0.0011) [2023-12-26 20:55:25,273][105692] Updated weights for policy 0, policy_version 763872 (0.0006) [2023-12-26 20:55:25,324][105620] Updated weights for policy 1, policy_version 763940 (0.0011) [2023-12-26 20:55:25,827][105692] Updated weights for policy 0, policy_version 763882 (0.0006) [2023-12-26 20:55:25,879][105692] Updated weights for policy 0, policy_version 763892 (0.0006) [2023-12-26 20:55:25,924][105692] Updated weights for policy 0, policy_version 763902 (0.0005) [2023-12-26 20:55:25,975][105692] Updated weights for policy 0, policy_version 763912 (0.0005) [2023-12-26 20:55:26,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19797.2, 300 sec: 19577.5). Total num frames: 391184384. Throughput: 0: 10051.6, 1: 9825.5. Samples: 391191280. Policy #0 lag: (min: 27.0, avg: 41.5, max: 59.0) [2023-12-26 20:55:26,063][104569] Avg episode reward: [(0, '8294.977'), (1, '9073.740')] [2023-12-26 20:55:26,098][105620] Updated weights for policy 1, policy_version 763950 (0.0010) [2023-12-26 20:55:26,155][105620] Updated weights for policy 1, policy_version 763960 (0.0010) [2023-12-26 20:55:26,201][105620] Updated weights for policy 1, policy_version 763970 (0.0005) [2023-12-26 20:55:26,544][105692] Updated weights for policy 0, policy_version 763922 (0.0005) [2023-12-26 20:55:26,590][105692] Updated weights for policy 0, policy_version 763932 (0.0005) [2023-12-26 20:55:26,640][105692] Updated weights for policy 0, policy_version 763942 (0.0005) [2023-12-26 20:55:26,742][105620] Updated weights for policy 1, policy_version 763980 (0.0005) [2023-12-26 20:55:26,788][105620] Updated weights for policy 1, policy_version 763990 (0.0005) [2023-12-26 20:55:26,835][105620] Updated weights for policy 1, policy_version 764000 (0.0005) [2023-12-26 20:55:27,210][105692] Updated weights for policy 0, policy_version 763952 (0.0005) [2023-12-26 20:55:27,259][105692] Updated weights for policy 0, policy_version 763962 (0.0007) [2023-12-26 20:55:27,310][105692] Updated weights for policy 0, policy_version 763972 (0.0007) [2023-12-26 20:55:27,380][105620] Updated weights for policy 1, policy_version 764010 (0.0009) [2023-12-26 20:55:27,428][105620] Updated weights for policy 1, policy_version 764020 (0.0005) [2023-12-26 20:55:27,479][105620] Updated weights for policy 1, policy_version 764030 (0.0005) [2023-12-26 20:55:27,539][105620] Updated weights for policy 1, policy_version 764040 (0.0005) [2023-12-26 20:55:28,023][105692] Updated weights for policy 0, policy_version 763982 (0.0007) [2023-12-26 20:55:28,072][105620] Updated weights for policy 1, policy_version 764050 (0.0009) [2023-12-26 20:55:28,086][105692] Updated weights for policy 0, policy_version 763992 (0.0005) [2023-12-26 20:55:28,119][105620] Updated weights for policy 1, policy_version 764060 (0.0008) [2023-12-26 20:55:28,167][105692] Updated weights for policy 0, policy_version 764002 (0.0009) [2023-12-26 20:55:28,169][105620] Updated weights for policy 1, policy_version 764070 (0.0007) [2023-12-26 20:55:28,744][105692] Updated weights for policy 0, policy_version 764012 (0.0008) [2023-12-26 20:55:28,805][105692] Updated weights for policy 0, policy_version 764022 (0.0005) [2023-12-26 20:55:28,858][105692] Updated weights for policy 0, policy_version 764032 (0.0010) [2023-12-26 20:55:28,899][105620] Updated weights for policy 1, policy_version 764080 (0.0009) [2023-12-26 20:55:28,961][105620] Updated weights for policy 1, policy_version 764090 (0.0009) [2023-12-26 20:55:29,020][105620] Updated weights for policy 1, policy_version 764100 (0.0011) [2023-12-26 20:55:29,571][105692] Updated weights for policy 0, policy_version 764042 (0.0011) [2023-12-26 20:55:29,631][105692] Updated weights for policy 0, policy_version 764052 (0.0006) [2023-12-26 20:55:29,634][105620] Updated weights for policy 1, policy_version 764110 (0.0009) [2023-12-26 20:55:29,686][105620] Updated weights for policy 1, policy_version 764120 (0.0010) [2023-12-26 20:55:29,690][105692] Updated weights for policy 0, policy_version 764062 (0.0008) [2023-12-26 20:55:29,738][105620] Updated weights for policy 1, policy_version 764130 (0.0010) [2023-12-26 20:55:29,749][105692] Updated weights for policy 0, policy_version 764072 (0.0011) [2023-12-26 20:55:30,381][105620] Updated weights for policy 1, policy_version 764140 (0.0010) [2023-12-26 20:55:30,442][105620] Updated weights for policy 1, policy_version 764150 (0.0009) [2023-12-26 20:55:30,494][105620] Updated weights for policy 1, policy_version 764160 (0.0007) [2023-12-26 20:55:30,511][105692] Updated weights for policy 0, policy_version 764082 (0.0008) [2023-12-26 20:55:30,560][105692] Updated weights for policy 0, policy_version 764092 (0.0009) [2023-12-26 20:55:30,622][105692] Updated weights for policy 0, policy_version 764102 (0.0009) [2023-12-26 20:55:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 391290880. Throughput: 0: 10169.9, 1: 9990.9. Samples: 391259772. Policy #0 lag: (min: 27.0, avg: 41.5, max: 59.0) [2023-12-26 20:55:31,062][104569] Avg episode reward: [(0, '7804.241'), (1, '9162.959')] [2023-12-26 20:55:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000764104_195641344.pth... [2023-12-26 20:55:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000764168_195649536.pth... [2023-12-26 20:55:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000763016_195354624.pth [2023-12-26 20:55:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000762920_195338240.pth [2023-12-26 20:55:31,108][105620] Updated weights for policy 1, policy_version 764170 (0.0008) [2023-12-26 20:55:31,166][105620] Updated weights for policy 1, policy_version 764180 (0.0008) [2023-12-26 20:55:31,217][105620] Updated weights for policy 1, policy_version 764190 (0.0009) [2023-12-26 20:55:31,279][105620] Updated weights for policy 1, policy_version 764200 (0.0009) [2023-12-26 20:55:31,473][105692] Updated weights for policy 0, policy_version 764112 (0.0009) [2023-12-26 20:55:31,532][105692] Updated weights for policy 0, policy_version 764122 (0.0009) [2023-12-26 20:55:31,586][105692] Updated weights for policy 0, policy_version 764132 (0.0007) [2023-12-26 20:55:31,987][105620] Updated weights for policy 1, policy_version 764210 (0.0009) [2023-12-26 20:55:32,049][105620] Updated weights for policy 1, policy_version 764220 (0.0009) [2023-12-26 20:55:32,115][105620] Updated weights for policy 1, policy_version 764230 (0.0009) [2023-12-26 20:55:32,349][105692] Updated weights for policy 0, policy_version 764142 (0.0008) [2023-12-26 20:55:32,408][105692] Updated weights for policy 0, policy_version 764152 (0.0007) [2023-12-26 20:55:32,463][105692] Updated weights for policy 0, policy_version 764162 (0.0005) [2023-12-26 20:55:32,811][105620] Updated weights for policy 1, policy_version 764240 (0.0009) [2023-12-26 20:55:32,870][105620] Updated weights for policy 1, policy_version 764250 (0.0007) [2023-12-26 20:55:32,930][105620] Updated weights for policy 1, policy_version 764260 (0.0005) [2023-12-26 20:55:33,120][105692] Updated weights for policy 0, policy_version 764172 (0.0005) [2023-12-26 20:55:33,166][105692] Updated weights for policy 0, policy_version 764182 (0.0005) [2023-12-26 20:55:33,223][105692] Updated weights for policy 0, policy_version 764192 (0.0005) [2023-12-26 20:55:33,739][105620] Updated weights for policy 1, policy_version 764270 (0.0007) [2023-12-26 20:55:33,762][105692] Updated weights for policy 0, policy_version 764202 (0.0006) [2023-12-26 20:55:33,798][105620] Updated weights for policy 1, policy_version 764280 (0.0006) [2023-12-26 20:55:33,816][105692] Updated weights for policy 0, policy_version 764212 (0.0010) [2023-12-26 20:55:33,850][105620] Updated weights for policy 1, policy_version 764290 (0.0006) [2023-12-26 20:55:33,860][105692] Updated weights for policy 0, policy_version 764222 (0.0010) [2023-12-26 20:55:33,911][105692] Updated weights for policy 0, policy_version 764232 (0.0010) [2023-12-26 20:55:34,561][105620] Updated weights for policy 1, policy_version 764300 (0.0007) [2023-12-26 20:55:34,624][105620] Updated weights for policy 1, policy_version 764310 (0.0008) [2023-12-26 20:55:34,681][105620] Updated weights for policy 1, policy_version 764320 (0.0008) [2023-12-26 20:55:34,695][105692] Updated weights for policy 0, policy_version 764242 (0.0011) [2023-12-26 20:55:34,750][105692] Updated weights for policy 0, policy_version 764252 (0.0008) [2023-12-26 20:55:34,802][105692] Updated weights for policy 0, policy_version 764262 (0.0011) [2023-12-26 20:55:35,387][105692] Updated weights for policy 0, policy_version 764272 (0.0011) [2023-12-26 20:55:35,446][105692] Updated weights for policy 0, policy_version 764282 (0.0010) [2023-12-26 20:55:35,504][105692] Updated weights for policy 0, policy_version 764292 (0.0010) [2023-12-26 20:55:35,504][105620] Updated weights for policy 1, policy_version 764330 (0.0008) [2023-12-26 20:55:35,564][105620] Updated weights for policy 1, policy_version 764340 (0.0008) [2023-12-26 20:55:35,627][105620] Updated weights for policy 1, policy_version 764350 (0.0008) [2023-12-26 20:55:35,689][105620] Updated weights for policy 1, policy_version 764360 (0.0008) [2023-12-26 20:55:36,062][104569] Fps is (10 sec: 20480.7, 60 sec: 20070.4, 300 sec: 19633.0). Total num frames: 391389184. Throughput: 0: 10090.2, 1: 9969.2. Samples: 391378456. Policy #0 lag: (min: 27.0, avg: 41.5, max: 59.0) [2023-12-26 20:55:36,063][104569] Avg episode reward: [(0, '8468.775'), (1, '8988.997')] [2023-12-26 20:55:36,155][105692] Updated weights for policy 0, policy_version 764302 (0.0009) [2023-12-26 20:55:36,220][105692] Updated weights for policy 0, policy_version 764312 (0.0005) [2023-12-26 20:55:36,283][105692] Updated weights for policy 0, policy_version 764322 (0.0007) [2023-12-26 20:55:36,566][105620] Updated weights for policy 1, policy_version 764370 (0.0009) [2023-12-26 20:55:36,626][105620] Updated weights for policy 1, policy_version 764380 (0.0008) [2023-12-26 20:55:36,692][105620] Updated weights for policy 1, policy_version 764390 (0.0009) [2023-12-26 20:55:36,864][105692] Updated weights for policy 0, policy_version 764332 (0.0006) [2023-12-26 20:55:36,917][105692] Updated weights for policy 0, policy_version 764342 (0.0009) [2023-12-26 20:55:36,971][105692] Updated weights for policy 0, policy_version 764352 (0.0006) [2023-12-26 20:55:37,551][105692] Updated weights for policy 0, policy_version 764362 (0.0006) [2023-12-26 20:55:37,559][105620] Updated weights for policy 1, policy_version 764400 (0.0008) [2023-12-26 20:55:37,612][105692] Updated weights for policy 0, policy_version 764372 (0.0008) [2023-12-26 20:55:37,615][105620] Updated weights for policy 1, policy_version 764410 (0.0007) [2023-12-26 20:55:37,670][105620] Updated weights for policy 1, policy_version 764420 (0.0005) [2023-12-26 20:55:37,675][105692] Updated weights for policy 0, policy_version 764382 (0.0010) [2023-12-26 20:55:37,731][105692] Updated weights for policy 0, policy_version 764392 (0.0009) [2023-12-26 20:55:38,369][105692] Updated weights for policy 0, policy_version 764402 (0.0009) [2023-12-26 20:55:38,428][105692] Updated weights for policy 0, policy_version 764412 (0.0009) [2023-12-26 20:55:38,462][105620] Updated weights for policy 1, policy_version 764430 (0.0007) [2023-12-26 20:55:38,485][105692] Updated weights for policy 0, policy_version 764422 (0.0007) [2023-12-26 20:55:38,526][105620] Updated weights for policy 1, policy_version 764440 (0.0009) [2023-12-26 20:55:38,592][105620] Updated weights for policy 1, policy_version 764450 (0.0009) [2023-12-26 20:55:39,231][105692] Updated weights for policy 0, policy_version 764432 (0.0009) [2023-12-26 20:55:39,293][105692] Updated weights for policy 0, policy_version 764442 (0.0009) [2023-12-26 20:55:39,356][105620] Updated weights for policy 1, policy_version 764460 (0.0008) [2023-12-26 20:55:39,362][105692] Updated weights for policy 0, policy_version 764452 (0.0009) [2023-12-26 20:55:39,414][105620] Updated weights for policy 1, policy_version 764470 (0.0008) [2023-12-26 20:55:39,470][105620] Updated weights for policy 1, policy_version 764480 (0.0009) [2023-12-26 20:55:40,149][105692] Updated weights for policy 0, policy_version 764462 (0.0007) [2023-12-26 20:55:40,212][105692] Updated weights for policy 0, policy_version 764472 (0.0007) [2023-12-26 20:55:40,268][105692] Updated weights for policy 0, policy_version 764482 (0.0005) [2023-12-26 20:55:40,271][105620] Updated weights for policy 1, policy_version 764490 (0.0009) [2023-12-26 20:55:40,336][105620] Updated weights for policy 1, policy_version 764500 (0.0011) [2023-12-26 20:55:40,405][105620] Updated weights for policy 1, policy_version 764510 (0.0010) [2023-12-26 20:55:40,464][105620] Updated weights for policy 1, policy_version 764520 (0.0008) [2023-12-26 20:55:40,915][105692] Updated weights for policy 0, policy_version 764492 (0.0006) [2023-12-26 20:55:40,972][105692] Updated weights for policy 0, policy_version 764502 (0.0005) [2023-12-26 20:55:41,011][105620] Updated weights for policy 1, policy_version 764530 (0.0010) [2023-12-26 20:55:41,032][105692] Updated weights for policy 0, policy_version 764512 (0.0008) [2023-12-26 20:55:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19934.0, 300 sec: 19605.3). Total num frames: 391479296. Throughput: 0: 10255.3, 1: 9787.1. Samples: 391495632. Policy #0 lag: (min: 27.0, avg: 41.5, max: 59.0) [2023-12-26 20:55:41,063][104569] Avg episode reward: [(0, '9259.123'), (1, '8723.058')] [2023-12-26 20:55:41,086][105620] Updated weights for policy 1, policy_version 764540 (0.0013) [2023-12-26 20:55:41,156][105620] Updated weights for policy 1, policy_version 764550 (0.0012) [2023-12-26 20:55:41,851][105620] Updated weights for policy 1, policy_version 764560 (0.0011) [2023-12-26 20:55:41,865][105692] Updated weights for policy 0, policy_version 764522 (0.0007) [2023-12-26 20:55:41,912][105620] Updated weights for policy 1, policy_version 764570 (0.0011) [2023-12-26 20:55:41,931][105692] Updated weights for policy 0, policy_version 764532 (0.0006) [2023-12-26 20:55:41,969][105620] Updated weights for policy 1, policy_version 764580 (0.0011) [2023-12-26 20:55:41,995][105692] Updated weights for policy 0, policy_version 764542 (0.0006) [2023-12-26 20:55:42,049][105692] Updated weights for policy 0, policy_version 764552 (0.0008) [2023-12-26 20:55:42,619][105620] Updated weights for policy 1, policy_version 764590 (0.0008) [2023-12-26 20:55:42,673][105620] Updated weights for policy 1, policy_version 764600 (0.0009) [2023-12-26 20:55:42,739][105620] Updated weights for policy 1, policy_version 764610 (0.0010) [2023-12-26 20:55:42,753][105692] Updated weights for policy 0, policy_version 764562 (0.0006) [2023-12-26 20:55:42,814][105692] Updated weights for policy 0, policy_version 764572 (0.0009) [2023-12-26 20:55:42,878][105692] Updated weights for policy 0, policy_version 764582 (0.0008) [2023-12-26 20:55:43,423][105692] Updated weights for policy 0, policy_version 764592 (0.0008) [2023-12-26 20:55:43,463][105620] Updated weights for policy 1, policy_version 764620 (0.0009) [2023-12-26 20:55:43,488][105692] Updated weights for policy 0, policy_version 764602 (0.0009) [2023-12-26 20:55:43,523][105620] Updated weights for policy 1, policy_version 764630 (0.0006) [2023-12-26 20:55:43,546][105692] Updated weights for policy 0, policy_version 764612 (0.0008) [2023-12-26 20:55:43,582][105620] Updated weights for policy 1, policy_version 764640 (0.0008) [2023-12-26 20:55:44,182][105620] Updated weights for policy 1, policy_version 764650 (0.0010) [2023-12-26 20:55:44,231][105620] Updated weights for policy 1, policy_version 764660 (0.0010) [2023-12-26 20:55:44,275][105620] Updated weights for policy 1, policy_version 764670 (0.0010) [2023-12-26 20:55:44,330][105620] Updated weights for policy 1, policy_version 764680 (0.0010) [2023-12-26 20:55:44,371][105692] Updated weights for policy 0, policy_version 764622 (0.0009) [2023-12-26 20:55:44,422][105692] Updated weights for policy 0, policy_version 764632 (0.0009) [2023-12-26 20:55:44,472][105692] Updated weights for policy 0, policy_version 764642 (0.0009) [2023-12-26 20:55:45,026][105620] Updated weights for policy 1, policy_version 764690 (0.0006) [2023-12-26 20:55:45,095][105620] Updated weights for policy 1, policy_version 764700 (0.0006) [2023-12-26 20:55:45,171][105620] Updated weights for policy 1, policy_version 764710 (0.0006) [2023-12-26 20:55:45,242][105692] Updated weights for policy 0, policy_version 764652 (0.0007) [2023-12-26 20:55:45,300][105692] Updated weights for policy 0, policy_version 764662 (0.0005) [2023-12-26 20:55:45,351][105692] Updated weights for policy 0, policy_version 764672 (0.0005) [2023-12-26 20:55:45,909][105620] Updated weights for policy 1, policy_version 764720 (0.0008) [2023-12-26 20:55:45,933][105692] Updated weights for policy 0, policy_version 764682 (0.0006) [2023-12-26 20:55:45,970][105620] Updated weights for policy 1, policy_version 764730 (0.0007) [2023-12-26 20:55:45,985][105692] Updated weights for policy 0, policy_version 764692 (0.0008) [2023-12-26 20:55:46,026][105620] Updated weights for policy 1, policy_version 764740 (0.0005) [2023-12-26 20:55:46,036][105692] Updated weights for policy 0, policy_version 764702 (0.0008) [2023-12-26 20:55:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 20070.5, 300 sec: 19633.0). Total num frames: 391585792. Throughput: 0: 10202.8, 1: 9753.0. Samples: 391555168. Policy #0 lag: (min: 27.0, avg: 41.5, max: 59.0) [2023-12-26 20:55:46,062][104569] Avg episode reward: [(0, '9350.173'), (1, '8541.308')] [2023-12-26 20:55:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000764744_195796992.pth... [2023-12-26 20:55:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000763592_195502080.pth [2023-12-26 20:55:46,084][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000764712_195796992.pth... [2023-12-26 20:55:46,088][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000763528_195493888.pth [2023-12-26 20:55:46,088][105692] Updated weights for policy 0, policy_version 764712 (0.0008) [2023-12-26 20:55:46,770][105620] Updated weights for policy 1, policy_version 764750 (0.0008) [2023-12-26 20:55:46,826][105620] Updated weights for policy 1, policy_version 764760 (0.0008) [2023-12-26 20:55:46,887][105620] Updated weights for policy 1, policy_version 764770 (0.0006) [2023-12-26 20:55:46,904][105692] Updated weights for policy 0, policy_version 764722 (0.0010) [2023-12-26 20:55:46,967][105692] Updated weights for policy 0, policy_version 764732 (0.0009) [2023-12-26 20:55:47,026][105692] Updated weights for policy 0, policy_version 764742 (0.0010) [2023-12-26 20:55:47,676][105692] Updated weights for policy 0, policy_version 764752 (0.0009) [2023-12-26 20:55:47,699][105620] Updated weights for policy 1, policy_version 764780 (0.0007) [2023-12-26 20:55:47,731][105692] Updated weights for policy 0, policy_version 764762 (0.0006) [2023-12-26 20:55:47,753][105620] Updated weights for policy 1, policy_version 764790 (0.0007) [2023-12-26 20:55:47,784][105692] Updated weights for policy 0, policy_version 764772 (0.0008) [2023-12-26 20:55:47,802][105620] Updated weights for policy 1, policy_version 764800 (0.0007) [2023-12-26 20:55:48,481][105692] Updated weights for policy 0, policy_version 764782 (0.0007) [2023-12-26 20:55:48,543][105692] Updated weights for policy 0, policy_version 764792 (0.0009) [2023-12-26 20:55:48,608][105692] Updated weights for policy 0, policy_version 764802 (0.0009) [2023-12-26 20:55:48,610][105620] Updated weights for policy 1, policy_version 764810 (0.0007) [2023-12-26 20:55:48,676][105620] Updated weights for policy 1, policy_version 764820 (0.0009) [2023-12-26 20:55:48,736][105620] Updated weights for policy 1, policy_version 764830 (0.0009) [2023-12-26 20:55:48,763][105586] KL-divergence is very high: 149.1149 [2023-12-26 20:55:48,801][105620] Updated weights for policy 1, policy_version 764840 (0.0009) [2023-12-26 20:55:49,287][105692] Updated weights for policy 0, policy_version 764812 (0.0007) [2023-12-26 20:55:49,338][105692] Updated weights for policy 0, policy_version 764822 (0.0009) [2023-12-26 20:55:49,400][105692] Updated weights for policy 0, policy_version 764832 (0.0009) [2023-12-26 20:55:49,621][105620] Updated weights for policy 1, policy_version 764850 (0.0010) [2023-12-26 20:55:49,686][105620] Updated weights for policy 1, policy_version 764860 (0.0009) [2023-12-26 20:55:49,749][105620] Updated weights for policy 1, policy_version 764870 (0.0009) [2023-12-26 20:55:50,114][105692] Updated weights for policy 0, policy_version 764842 (0.0008) [2023-12-26 20:55:50,177][105692] Updated weights for policy 0, policy_version 764852 (0.0009) [2023-12-26 20:55:50,239][105692] Updated weights for policy 0, policy_version 764862 (0.0009) [2023-12-26 20:55:50,297][105692] Updated weights for policy 0, policy_version 764872 (0.0009) [2023-12-26 20:55:50,542][105620] Updated weights for policy 1, policy_version 764880 (0.0009) [2023-12-26 20:55:50,604][105620] Updated weights for policy 1, policy_version 764890 (0.0008) [2023-12-26 20:55:50,670][105620] Updated weights for policy 1, policy_version 764900 (0.0009) [2023-12-26 20:55:51,031][105692] Updated weights for policy 0, policy_version 764882 (0.0008) [2023-12-26 20:55:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 391675904. Throughput: 0: 10148.3, 1: 9677.7. Samples: 391669388. Policy #0 lag: (min: 27.0, avg: 41.5, max: 59.0) [2023-12-26 20:55:51,063][104569] Avg episode reward: [(0, '9258.801'), (1, '8715.834')] [2023-12-26 20:55:51,090][105692] Updated weights for policy 0, policy_version 764892 (0.0009) [2023-12-26 20:55:51,155][105692] Updated weights for policy 0, policy_version 764902 (0.0009) [2023-12-26 20:55:51,455][105620] Updated weights for policy 1, policy_version 764910 (0.0009) [2023-12-26 20:55:51,517][105620] Updated weights for policy 1, policy_version 764920 (0.0009) [2023-12-26 20:55:51,571][105620] Updated weights for policy 1, policy_version 764930 (0.0008) [2023-12-26 20:55:51,931][105692] Updated weights for policy 0, policy_version 764912 (0.0009) [2023-12-26 20:55:51,999][105692] Updated weights for policy 0, policy_version 764922 (0.0010) [2023-12-26 20:55:52,056][105692] Updated weights for policy 0, policy_version 764932 (0.0009) [2023-12-26 20:55:52,383][105620] Updated weights for policy 1, policy_version 764940 (0.0009) [2023-12-26 20:55:52,444][105620] Updated weights for policy 1, policy_version 764950 (0.0009) [2023-12-26 20:55:52,506][105620] Updated weights for policy 1, policy_version 764960 (0.0009) [2023-12-26 20:55:52,819][105692] Updated weights for policy 0, policy_version 764942 (0.0009) [2023-12-26 20:55:52,874][105692] Updated weights for policy 0, policy_version 764952 (0.0009) [2023-12-26 20:55:52,931][105692] Updated weights for policy 0, policy_version 764962 (0.0008) [2023-12-26 20:55:53,257][105620] Updated weights for policy 1, policy_version 764970 (0.0009) [2023-12-26 20:55:53,318][105620] Updated weights for policy 1, policy_version 764980 (0.0009) [2023-12-26 20:55:53,375][105620] Updated weights for policy 1, policy_version 764990 (0.0009) [2023-12-26 20:55:53,432][105620] Updated weights for policy 1, policy_version 765000 (0.0009) [2023-12-26 20:55:53,640][105692] Updated weights for policy 0, policy_version 764972 (0.0008) [2023-12-26 20:55:53,699][105692] Updated weights for policy 0, policy_version 764982 (0.0009) [2023-12-26 20:55:53,755][105692] Updated weights for policy 0, policy_version 764992 (0.0009) [2023-12-26 20:55:54,204][105620] Updated weights for policy 1, policy_version 765010 (0.0006) [2023-12-26 20:55:54,265][105620] Updated weights for policy 1, policy_version 765020 (0.0005) [2023-12-26 20:55:54,324][105620] Updated weights for policy 1, policy_version 765030 (0.0005) [2023-12-26 20:55:54,556][105692] Updated weights for policy 0, policy_version 765003 (0.0010) [2023-12-26 20:55:54,615][105692] Updated weights for policy 0, policy_version 765013 (0.0008) [2023-12-26 20:55:54,675][105692] Updated weights for policy 0, policy_version 765023 (0.0008) [2023-12-26 20:55:54,987][105620] Updated weights for policy 1, policy_version 765040 (0.0009) [2023-12-26 20:55:55,053][105620] Updated weights for policy 1, policy_version 765050 (0.0009) [2023-12-26 20:55:55,111][105620] Updated weights for policy 1, policy_version 765060 (0.0010) [2023-12-26 20:55:55,281][105692] Updated weights for policy 0, policy_version 765033 (0.0010) [2023-12-26 20:55:55,338][105692] Updated weights for policy 0, policy_version 765043 (0.0010) [2023-12-26 20:55:55,389][105692] Updated weights for policy 0, policy_version 765053 (0.0010) [2023-12-26 20:55:55,437][105692] Updated weights for policy 0, policy_version 765063 (0.0009) [2023-12-26 20:55:55,879][105620] Updated weights for policy 1, policy_version 765070 (0.0009) [2023-12-26 20:55:55,942][105620] Updated weights for policy 1, policy_version 765082 (0.0009) [2023-12-26 20:55:56,000][105620] Updated weights for policy 1, policy_version 765092 (0.0005) [2023-12-26 20:55:56,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19797.3, 300 sec: 19605.2). Total num frames: 391774208. Throughput: 0: 10054.7, 1: 9704.0. Samples: 391781936. Policy #0 lag: (min: 27.0, avg: 41.5, max: 59.0) [2023-12-26 20:55:56,063][104569] Avg episode reward: [(0, '9222.155'), (1, '8988.572')] [2023-12-26 20:55:56,244][105692] Updated weights for policy 0, policy_version 765073 (0.0009) [2023-12-26 20:55:56,292][105692] Updated weights for policy 0, policy_version 765083 (0.0010) [2023-12-26 20:55:56,341][105692] Updated weights for policy 0, policy_version 765093 (0.0010) [2023-12-26 20:55:56,609][105620] Updated weights for policy 1, policy_version 765102 (0.0007) [2023-12-26 20:55:56,661][105620] Updated weights for policy 1, policy_version 765113 (0.0010) [2023-12-26 20:55:56,714][105620] Updated weights for policy 1, policy_version 765123 (0.0010) [2023-12-26 20:55:56,966][105692] Updated weights for policy 0, policy_version 765103 (0.0009) [2023-12-26 20:55:57,028][105692] Updated weights for policy 0, policy_version 765113 (0.0008) [2023-12-26 20:55:57,090][105692] Updated weights for policy 0, policy_version 765123 (0.0009) [2023-12-26 20:55:57,492][105620] Updated weights for policy 1, policy_version 765133 (0.0007) [2023-12-26 20:55:57,537][105620] Updated weights for policy 1, policy_version 765143 (0.0005) [2023-12-26 20:55:57,589][105620] Updated weights for policy 1, policy_version 765153 (0.0005) [2023-12-26 20:55:57,874][105692] Updated weights for policy 0, policy_version 765133 (0.0010) [2023-12-26 20:55:57,926][105692] Updated weights for policy 0, policy_version 765143 (0.0009) [2023-12-26 20:55:57,980][105692] Updated weights for policy 0, policy_version 765153 (0.0010) [2023-12-26 20:55:58,155][105620] Updated weights for policy 1, policy_version 765163 (0.0008) [2023-12-26 20:55:58,219][105620] Updated weights for policy 1, policy_version 765173 (0.0008) [2023-12-26 20:55:58,271][105620] Updated weights for policy 1, policy_version 765183 (0.0010) [2023-12-26 20:55:58,768][105692] Updated weights for policy 0, policy_version 765163 (0.0010) [2023-12-26 20:55:58,830][105692] Updated weights for policy 0, policy_version 765173 (0.0007) [2023-12-26 20:55:58,893][105692] Updated weights for policy 0, policy_version 765183 (0.0008) [2023-12-26 20:55:59,041][105620] Updated weights for policy 1, policy_version 765193 (0.0010) [2023-12-26 20:55:59,104][105620] Updated weights for policy 1, policy_version 765203 (0.0006) [2023-12-26 20:55:59,171][105620] Updated weights for policy 1, policy_version 765213 (0.0008) [2023-12-26 20:55:59,243][105620] Updated weights for policy 1, policy_version 765223 (0.0010) [2023-12-26 20:55:59,526][105692] Updated weights for policy 0, policy_version 765193 (0.0008) [2023-12-26 20:55:59,578][105692] Updated weights for policy 0, policy_version 765203 (0.0008) [2023-12-26 20:55:59,633][105692] Updated weights for policy 0, policy_version 765213 (0.0009) [2023-12-26 20:55:59,688][105692] Updated weights for policy 0, policy_version 765223 (0.0010) [2023-12-26 20:55:59,940][105620] Updated weights for policy 1, policy_version 765233 (0.0007) [2023-12-26 20:55:59,994][105620] Updated weights for policy 1, policy_version 765243 (0.0008) [2023-12-26 20:56:00,049][105620] Updated weights for policy 1, policy_version 765253 (0.0008) [2023-12-26 20:56:00,400][105692] Updated weights for policy 0, policy_version 765233 (0.0007) [2023-12-26 20:56:00,451][105692] Updated weights for policy 0, policy_version 765243 (0.0008) [2023-12-26 20:56:00,503][105692] Updated weights for policy 0, policy_version 765253 (0.0005) [2023-12-26 20:56:00,723][105620] Updated weights for policy 1, policy_version 765263 (0.0006) [2023-12-26 20:56:00,771][105620] Updated weights for policy 1, policy_version 765273 (0.0005) [2023-12-26 20:56:00,823][105620] Updated weights for policy 1, policy_version 765283 (0.0005) [2023-12-26 20:56:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 391872512. Throughput: 0: 10017.5, 1: 9731.4. Samples: 391841184. Policy #0 lag: (min: 27.0, avg: 41.5, max: 59.0) [2023-12-26 20:56:01,062][104569] Avg episode reward: [(0, '9130.310'), (1, '8717.519')] [2023-12-26 20:56:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000765256_195936256.pth... [2023-12-26 20:56:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000765288_195936256.pth... [2023-12-26 20:56:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000764104_195641344.pth [2023-12-26 20:56:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000764168_195649536.pth [2023-12-26 20:56:01,183][105692] Updated weights for policy 0, policy_version 765263 (0.0008) [2023-12-26 20:56:01,243][105692] Updated weights for policy 0, policy_version 765273 (0.0009) [2023-12-26 20:56:01,302][105692] Updated weights for policy 0, policy_version 765283 (0.0009) [2023-12-26 20:56:01,420][105620] Updated weights for policy 1, policy_version 765293 (0.0007) [2023-12-26 20:56:01,484][105620] Updated weights for policy 1, policy_version 765303 (0.0009) [2023-12-26 20:56:01,545][105620] Updated weights for policy 1, policy_version 765313 (0.0009) [2023-12-26 20:56:02,136][105692] Updated weights for policy 0, policy_version 765293 (0.0009) [2023-12-26 20:56:02,196][105692] Updated weights for policy 0, policy_version 765303 (0.0010) [2023-12-26 20:56:02,236][105620] Updated weights for policy 1, policy_version 765323 (0.0008) [2023-12-26 20:56:02,256][105692] Updated weights for policy 0, policy_version 765313 (0.0008) [2023-12-26 20:56:02,301][105620] Updated weights for policy 1, policy_version 765333 (0.0009) [2023-12-26 20:56:02,365][105620] Updated weights for policy 1, policy_version 765343 (0.0011) [2023-12-26 20:56:02,935][105692] Updated weights for policy 0, policy_version 765323 (0.0007) [2023-12-26 20:56:02,991][105692] Updated weights for policy 0, policy_version 765333 (0.0011) [2023-12-26 20:56:02,998][105620] Updated weights for policy 1, policy_version 765353 (0.0011) [2023-12-26 20:56:03,048][105692] Updated weights for policy 0, policy_version 765343 (0.0008) [2023-12-26 20:56:03,055][105620] Updated weights for policy 1, policy_version 765363 (0.0011) [2023-12-26 20:56:03,104][105620] Updated weights for policy 1, policy_version 765373 (0.0010) [2023-12-26 20:56:03,157][105620] Updated weights for policy 1, policy_version 765383 (0.0010) [2023-12-26 20:56:03,598][105692] Updated weights for policy 0, policy_version 765353 (0.0006) [2023-12-26 20:56:03,653][105692] Updated weights for policy 0, policy_version 765363 (0.0011) [2023-12-26 20:56:03,701][105692] Updated weights for policy 0, policy_version 765373 (0.0010) [2023-12-26 20:56:03,732][105620] Updated weights for policy 1, policy_version 765393 (0.0005) [2023-12-26 20:56:03,756][105692] Updated weights for policy 0, policy_version 765383 (0.0008) [2023-12-26 20:56:03,782][105620] Updated weights for policy 1, policy_version 765403 (0.0007) [2023-12-26 20:56:03,834][105620] Updated weights for policy 1, policy_version 765413 (0.0010) [2023-12-26 20:56:04,455][105692] Updated weights for policy 0, policy_version 765393 (0.0006) [2023-12-26 20:56:04,515][105692] Updated weights for policy 0, policy_version 765403 (0.0007) [2023-12-26 20:56:04,572][105692] Updated weights for policy 0, policy_version 765413 (0.0006) [2023-12-26 20:56:04,607][105620] Updated weights for policy 1, policy_version 765423 (0.0010) [2023-12-26 20:56:04,669][105620] Updated weights for policy 1, policy_version 765433 (0.0011) [2023-12-26 20:56:04,727][105620] Updated weights for policy 1, policy_version 765443 (0.0010) [2023-12-26 20:56:05,233][105692] Updated weights for policy 0, policy_version 765423 (0.0009) [2023-12-26 20:56:05,281][105692] Updated weights for policy 0, policy_version 765433 (0.0010) [2023-12-26 20:56:05,329][105692] Updated weights for policy 0, policy_version 765443 (0.0010) [2023-12-26 20:56:05,419][105620] Updated weights for policy 1, policy_version 765453 (0.0010) [2023-12-26 20:56:05,477][105620] Updated weights for policy 1, policy_version 765463 (0.0010) [2023-12-26 20:56:05,543][105620] Updated weights for policy 1, policy_version 765473 (0.0009) [2023-12-26 20:56:05,989][105692] Updated weights for policy 0, policy_version 765453 (0.0010) [2023-12-26 20:56:06,046][105692] Updated weights for policy 0, policy_version 765463 (0.0010) [2023-12-26 20:56:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.7, 300 sec: 19605.3). Total num frames: 391970816. Throughput: 0: 9958.8, 1: 9826.3. Samples: 391963560. Policy #0 lag: (min: 27.0, avg: 41.5, max: 59.0) [2023-12-26 20:56:06,063][104569] Avg episode reward: [(0, '9166.910'), (1, '8623.698')] [2023-12-26 20:56:06,111][105692] Updated weights for policy 0, policy_version 765473 (0.0010) [2023-12-26 20:56:06,167][105620] Updated weights for policy 1, policy_version 765483 (0.0005) [2023-12-26 20:56:06,218][105620] Updated weights for policy 1, policy_version 765493 (0.0005) [2023-12-26 20:56:06,279][105620] Updated weights for policy 1, policy_version 765503 (0.0006) [2023-12-26 20:56:06,843][105692] Updated weights for policy 0, policy_version 765483 (0.0007) [2023-12-26 20:56:06,901][105620] Updated weights for policy 1, policy_version 765513 (0.0006) [2023-12-26 20:56:06,907][105692] Updated weights for policy 0, policy_version 765493 (0.0005) [2023-12-26 20:56:06,964][105620] Updated weights for policy 1, policy_version 765523 (0.0010) [2023-12-26 20:56:06,972][105692] Updated weights for policy 0, policy_version 765503 (0.0005) [2023-12-26 20:56:07,026][105620] Updated weights for policy 1, policy_version 765533 (0.0011) [2023-12-26 20:56:07,088][105620] Updated weights for policy 1, policy_version 765543 (0.0010) [2023-12-26 20:56:07,531][105692] Updated weights for policy 0, policy_version 765513 (0.0005) [2023-12-26 20:56:07,600][105692] Updated weights for policy 0, policy_version 765523 (0.0008) [2023-12-26 20:56:07,666][105692] Updated weights for policy 0, policy_version 765533 (0.0008) [2023-12-26 20:56:07,724][105692] Updated weights for policy 0, policy_version 765543 (0.0008) [2023-12-26 20:56:07,800][105620] Updated weights for policy 1, policy_version 765553 (0.0007) [2023-12-26 20:56:07,850][105620] Updated weights for policy 1, policy_version 765563 (0.0006) [2023-12-26 20:56:07,905][105620] Updated weights for policy 1, policy_version 765573 (0.0005) [2023-12-26 20:56:08,461][105620] Updated weights for policy 1, policy_version 765583 (0.0005) [2023-12-26 20:56:08,514][105620] Updated weights for policy 1, policy_version 765593 (0.0006) [2023-12-26 20:56:08,575][105620] Updated weights for policy 1, policy_version 765603 (0.0010) [2023-12-26 20:56:08,577][105692] Updated weights for policy 0, policy_version 765553 (0.0007) [2023-12-26 20:56:08,644][105692] Updated weights for policy 0, policy_version 765563 (0.0009) [2023-12-26 20:56:08,712][105692] Updated weights for policy 0, policy_version 765573 (0.0009) [2023-12-26 20:56:09,165][105620] Updated weights for policy 1, policy_version 765613 (0.0005) [2023-12-26 20:56:09,228][105620] Updated weights for policy 1, policy_version 765623 (0.0007) [2023-12-26 20:56:09,291][105620] Updated weights for policy 1, policy_version 765633 (0.0008) [2023-12-26 20:56:09,537][105692] Updated weights for policy 0, policy_version 765583 (0.0006) [2023-12-26 20:56:09,593][105692] Updated weights for policy 0, policy_version 765593 (0.0006) [2023-12-26 20:56:09,645][105692] Updated weights for policy 0, policy_version 765603 (0.0008) [2023-12-26 20:56:10,136][105620] Updated weights for policy 1, policy_version 765643 (0.0007) [2023-12-26 20:56:10,194][105620] Updated weights for policy 1, policy_version 765653 (0.0006) [2023-12-26 20:56:10,251][105620] Updated weights for policy 1, policy_version 765663 (0.0005) [2023-12-26 20:56:10,386][105692] Updated weights for policy 0, policy_version 765613 (0.0007) [2023-12-26 20:56:10,438][105692] Updated weights for policy 0, policy_version 765623 (0.0005) [2023-12-26 20:56:10,488][105692] Updated weights for policy 0, policy_version 765633 (0.0008) [2023-12-26 20:56:10,949][105620] Updated weights for policy 1, policy_version 765673 (0.0008) [2023-12-26 20:56:10,998][105620] Updated weights for policy 1, policy_version 765683 (0.0010) [2023-12-26 20:56:11,055][105620] Updated weights for policy 1, policy_version 765693 (0.0009) [2023-12-26 20:56:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 392069120. Throughput: 0: 9962.6, 1: 9876.2. Samples: 392084024. Policy #0 lag: (min: 27.0, avg: 41.5, max: 59.0) [2023-12-26 20:56:11,063][104569] Avg episode reward: [(0, '9257.543'), (1, '8533.446')] [2023-12-26 20:56:11,127][105620] Updated weights for policy 1, policy_version 765703 (0.0008) [2023-12-26 20:56:11,237][105692] Updated weights for policy 0, policy_version 765643 (0.0009) [2023-12-26 20:56:11,298][105692] Updated weights for policy 0, policy_version 765653 (0.0009) [2023-12-26 20:56:11,384][105692] Updated weights for policy 0, policy_version 765663 (0.0009) [2023-12-26 20:56:11,923][105620] Updated weights for policy 1, policy_version 765713 (0.0008) [2023-12-26 20:56:11,984][105620] Updated weights for policy 1, policy_version 765723 (0.0008) [2023-12-26 20:56:12,045][105620] Updated weights for policy 1, policy_version 765733 (0.0008) [2023-12-26 20:56:12,133][105692] Updated weights for policy 0, policy_version 765673 (0.0009) [2023-12-26 20:56:12,200][105692] Updated weights for policy 0, policy_version 765683 (0.0011) [2023-12-26 20:56:12,266][105692] Updated weights for policy 0, policy_version 765693 (0.0011) [2023-12-26 20:56:12,323][105692] Updated weights for policy 0, policy_version 765703 (0.0010) [2023-12-26 20:56:12,790][105620] Updated weights for policy 1, policy_version 765743 (0.0009) [2023-12-26 20:56:12,854][105620] Updated weights for policy 1, policy_version 765753 (0.0010) [2023-12-26 20:56:12,914][105620] Updated weights for policy 1, policy_version 765763 (0.0010) [2023-12-26 20:56:13,061][105692] Updated weights for policy 0, policy_version 765713 (0.0008) [2023-12-26 20:56:13,124][105692] Updated weights for policy 0, policy_version 765723 (0.0009) [2023-12-26 20:56:13,178][105692] Updated weights for policy 0, policy_version 765733 (0.0008) [2023-12-26 20:56:13,683][105620] Updated weights for policy 1, policy_version 765773 (0.0009) [2023-12-26 20:56:13,737][105620] Updated weights for policy 1, policy_version 765783 (0.0008) [2023-12-26 20:56:13,795][105620] Updated weights for policy 1, policy_version 765793 (0.0010) [2023-12-26 20:56:13,890][105692] Updated weights for policy 0, policy_version 765743 (0.0006) [2023-12-26 20:56:13,947][105692] Updated weights for policy 0, policy_version 765753 (0.0006) [2023-12-26 20:56:14,002][105692] Updated weights for policy 0, policy_version 765763 (0.0005) [2023-12-26 20:56:14,400][105620] Updated weights for policy 1, policy_version 765803 (0.0007) [2023-12-26 20:56:14,453][105620] Updated weights for policy 1, policy_version 765813 (0.0006) [2023-12-26 20:56:14,497][105620] Updated weights for policy 1, policy_version 765823 (0.0009) [2023-12-26 20:56:14,572][105692] Updated weights for policy 0, policy_version 765773 (0.0007) [2023-12-26 20:56:14,622][105692] Updated weights for policy 0, policy_version 765783 (0.0008) [2023-12-26 20:56:14,673][105692] Updated weights for policy 0, policy_version 765793 (0.0008) [2023-12-26 20:56:15,138][105620] Updated weights for policy 1, policy_version 765833 (0.0010) [2023-12-26 20:56:15,196][105620] Updated weights for policy 1, policy_version 765843 (0.0008) [2023-12-26 20:56:15,253][105620] Updated weights for policy 1, policy_version 765853 (0.0009) [2023-12-26 20:56:15,314][105620] Updated weights for policy 1, policy_version 765863 (0.0007) [2023-12-26 20:56:15,342][105692] Updated weights for policy 0, policy_version 765803 (0.0007) [2023-12-26 20:56:15,400][105692] Updated weights for policy 0, policy_version 765813 (0.0008) [2023-12-26 20:56:15,458][105692] Updated weights for policy 0, policy_version 765823 (0.0006) [2023-12-26 20:56:15,993][105620] Updated weights for policy 1, policy_version 765873 (0.0009) [2023-12-26 20:56:16,047][105620] Updated weights for policy 1, policy_version 765883 (0.0009) [2023-12-26 20:56:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 392167424. Throughput: 0: 9825.9, 1: 9709.0. Samples: 392138844. Policy #0 lag: (min: 27.0, avg: 41.5, max: 59.0) [2023-12-26 20:56:16,062][104569] Avg episode reward: [(0, '9348.410'), (1, '8809.990')] [2023-12-26 20:56:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000765832_196083712.pth... [2023-12-26 20:56:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000764712_195796992.pth [2023-12-26 20:56:16,115][105620] Updated weights for policy 1, policy_version 765893 (0.0009) [2023-12-26 20:56:16,128][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000765896_196091904.pth... [2023-12-26 20:56:16,133][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000764744_195796992.pth [2023-12-26 20:56:16,156][105692] Updated weights for policy 0, policy_version 765833 (0.0007) [2023-12-26 20:56:16,224][105692] Updated weights for policy 0, policy_version 765843 (0.0007) [2023-12-26 20:56:16,282][105692] Updated weights for policy 0, policy_version 765853 (0.0009) [2023-12-26 20:56:16,337][105692] Updated weights for policy 0, policy_version 765863 (0.0009) [2023-12-26 20:56:16,853][105620] Updated weights for policy 1, policy_version 765903 (0.0008) [2023-12-26 20:56:16,928][105620] Updated weights for policy 1, policy_version 765913 (0.0009) [2023-12-26 20:56:16,989][105692] Updated weights for policy 0, policy_version 765873 (0.0007) [2023-12-26 20:56:16,990][105620] Updated weights for policy 1, policy_version 765923 (0.0007) [2023-12-26 20:56:17,047][105692] Updated weights for policy 0, policy_version 765883 (0.0008) [2023-12-26 20:56:17,100][105692] Updated weights for policy 0, policy_version 765894 (0.0009) [2023-12-26 20:56:17,540][105620] Updated weights for policy 1, policy_version 765933 (0.0007) [2023-12-26 20:56:17,595][105620] Updated weights for policy 1, policy_version 765943 (0.0007) [2023-12-26 20:56:17,644][105620] Updated weights for policy 1, policy_version 765953 (0.0010) [2023-12-26 20:56:17,880][105692] Updated weights for policy 0, policy_version 765904 (0.0010) [2023-12-26 20:56:17,932][105692] Updated weights for policy 0, policy_version 765914 (0.0009) [2023-12-26 20:56:17,986][105692] Updated weights for policy 0, policy_version 765924 (0.0009) [2023-12-26 20:56:18,199][105620] Updated weights for policy 1, policy_version 765963 (0.0009) [2023-12-26 20:56:18,260][105620] Updated weights for policy 1, policy_version 765973 (0.0008) [2023-12-26 20:56:18,324][105620] Updated weights for policy 1, policy_version 765983 (0.0006) [2023-12-26 20:56:18,791][105692] Updated weights for policy 0, policy_version 765934 (0.0009) [2023-12-26 20:56:18,855][105692] Updated weights for policy 0, policy_version 765944 (0.0008) [2023-12-26 20:56:18,919][105692] Updated weights for policy 0, policy_version 765954 (0.0005) [2023-12-26 20:56:19,021][105620] Updated weights for policy 1, policy_version 765993 (0.0007) [2023-12-26 20:56:19,077][105620] Updated weights for policy 1, policy_version 766003 (0.0005) [2023-12-26 20:56:19,136][105620] Updated weights for policy 1, policy_version 766013 (0.0005) [2023-12-26 20:56:19,192][105620] Updated weights for policy 1, policy_version 766023 (0.0009) [2023-12-26 20:56:19,601][105692] Updated weights for policy 0, policy_version 765964 (0.0007) [2023-12-26 20:56:19,665][105692] Updated weights for policy 0, policy_version 765974 (0.0006) [2023-12-26 20:56:19,720][105692] Updated weights for policy 0, policy_version 765984 (0.0005) [2023-12-26 20:56:20,002][105620] Updated weights for policy 1, policy_version 766033 (0.0008) [2023-12-26 20:56:20,061][105620] Updated weights for policy 1, policy_version 766043 (0.0008) [2023-12-26 20:56:20,118][105620] Updated weights for policy 1, policy_version 766053 (0.0008) [2023-12-26 20:56:20,361][105692] Updated weights for policy 0, policy_version 765994 (0.0006) [2023-12-26 20:56:20,419][105692] Updated weights for policy 0, policy_version 766004 (0.0007) [2023-12-26 20:56:20,464][105692] Updated weights for policy 0, policy_version 766014 (0.0010) [2023-12-26 20:56:20,513][105692] Updated weights for policy 0, policy_version 766024 (0.0010) [2023-12-26 20:56:20,906][105620] Updated weights for policy 1, policy_version 766063 (0.0009) [2023-12-26 20:56:20,966][105620] Updated weights for policy 1, policy_version 766073 (0.0011) [2023-12-26 20:56:21,037][105620] Updated weights for policy 1, policy_version 766083 (0.0011) [2023-12-26 20:56:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 392265728. Throughput: 0: 9852.9, 1: 9778.0. Samples: 392261844. Policy #0 lag: (min: 14.0, avg: 34.2, max: 46.0) [2023-12-26 20:56:21,062][104569] Avg episode reward: [(0, '9348.789'), (1, '9168.831')] [2023-12-26 20:56:21,322][105692] Updated weights for policy 0, policy_version 766034 (0.0011) [2023-12-26 20:56:21,386][105692] Updated weights for policy 0, policy_version 766044 (0.0011) [2023-12-26 20:56:21,443][105692] Updated weights for policy 0, policy_version 766054 (0.0011) [2023-12-26 20:56:21,784][105620] Updated weights for policy 1, policy_version 766093 (0.0009) [2023-12-26 20:56:21,839][105620] Updated weights for policy 1, policy_version 766103 (0.0008) [2023-12-26 20:56:21,898][105620] Updated weights for policy 1, policy_version 766113 (0.0008) [2023-12-26 20:56:22,202][105692] Updated weights for policy 0, policy_version 766064 (0.0011) [2023-12-26 20:56:22,255][105692] Updated weights for policy 0, policy_version 766074 (0.0010) [2023-12-26 20:56:22,311][105692] Updated weights for policy 0, policy_version 766084 (0.0009) [2023-12-26 20:56:22,624][105620] Updated weights for policy 1, policy_version 766123 (0.0006) [2023-12-26 20:56:22,700][105620] Updated weights for policy 1, policy_version 766133 (0.0006) [2023-12-26 20:56:22,766][105620] Updated weights for policy 1, policy_version 766143 (0.0007) [2023-12-26 20:56:23,081][105692] Updated weights for policy 0, policy_version 766094 (0.0010) [2023-12-26 20:56:23,130][105692] Updated weights for policy 0, policy_version 766104 (0.0010) [2023-12-26 20:56:23,183][105692] Updated weights for policy 0, policy_version 766114 (0.0008) [2023-12-26 20:56:23,492][105620] Updated weights for policy 1, policy_version 766153 (0.0009) [2023-12-26 20:56:23,543][105620] Updated weights for policy 1, policy_version 766163 (0.0008) [2023-12-26 20:56:23,594][105620] Updated weights for policy 1, policy_version 766173 (0.0009) [2023-12-26 20:56:23,645][105620] Updated weights for policy 1, policy_version 766183 (0.0009) [2023-12-26 20:56:23,866][105692] Updated weights for policy 0, policy_version 766124 (0.0008) [2023-12-26 20:56:23,920][105692] Updated weights for policy 0, policy_version 766134 (0.0005) [2023-12-26 20:56:23,969][105692] Updated weights for policy 0, policy_version 766144 (0.0005) [2023-12-26 20:56:24,404][105620] Updated weights for policy 1, policy_version 766193 (0.0009) [2023-12-26 20:56:24,450][105620] Updated weights for policy 1, policy_version 766203 (0.0009) [2023-12-26 20:56:24,497][105620] Updated weights for policy 1, policy_version 766213 (0.0009) [2023-12-26 20:56:24,533][105692] Updated weights for policy 0, policy_version 766154 (0.0005) [2023-12-26 20:56:24,591][105692] Updated weights for policy 0, policy_version 766164 (0.0006) [2023-12-26 20:56:24,645][105692] Updated weights for policy 0, policy_version 766174 (0.0009) [2023-12-26 20:56:24,695][105692] Updated weights for policy 0, policy_version 766184 (0.0008) [2023-12-26 20:56:25,207][105620] Updated weights for policy 1, policy_version 766223 (0.0009) [2023-12-26 20:56:25,260][105692] Updated weights for policy 0, policy_version 766194 (0.0005) [2023-12-26 20:56:25,262][105620] Updated weights for policy 1, policy_version 766233 (0.0008) [2023-12-26 20:56:25,315][105692] Updated weights for policy 0, policy_version 766204 (0.0005) [2023-12-26 20:56:25,320][105620] Updated weights for policy 1, policy_version 766243 (0.0009) [2023-12-26 20:56:25,375][105692] Updated weights for policy 0, policy_version 766214 (0.0006) [2023-12-26 20:56:25,932][105620] Updated weights for policy 1, policy_version 766253 (0.0007) [2023-12-26 20:56:25,998][105620] Updated weights for policy 1, policy_version 766263 (0.0007) [2023-12-26 20:56:26,049][105692] Updated weights for policy 0, policy_version 766224 (0.0007) [2023-12-26 20:56:26,051][105620] Updated weights for policy 1, policy_version 766273 (0.0007) [2023-12-26 20:56:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 392364032. Throughput: 0: 9827.0, 1: 9859.7. Samples: 392381528. Policy #0 lag: (min: 14.0, avg: 34.2, max: 46.0) [2023-12-26 20:56:26,062][104569] Avg episode reward: [(0, '9259.925'), (1, '9261.218')] [2023-12-26 20:56:26,109][105692] Updated weights for policy 0, policy_version 766234 (0.0007) [2023-12-26 20:56:26,164][105692] Updated weights for policy 0, policy_version 766244 (0.0009) [2023-12-26 20:56:26,690][105620] Updated weights for policy 1, policy_version 766283 (0.0006) [2023-12-26 20:56:26,759][105620] Updated weights for policy 1, policy_version 766293 (0.0005) [2023-12-26 20:56:26,811][105620] Updated weights for policy 1, policy_version 766303 (0.0005) [2023-12-26 20:56:26,971][105692] Updated weights for policy 0, policy_version 766254 (0.0008) [2023-12-26 20:56:27,019][105692] Updated weights for policy 0, policy_version 766264 (0.0008) [2023-12-26 20:56:27,063][105692] Updated weights for policy 0, policy_version 766274 (0.0008) [2023-12-26 20:56:27,411][105620] Updated weights for policy 1, policy_version 766313 (0.0005) [2023-12-26 20:56:27,459][105620] Updated weights for policy 1, policy_version 766323 (0.0005) [2023-12-26 20:56:27,513][105620] Updated weights for policy 1, policy_version 766333 (0.0005) [2023-12-26 20:56:27,575][105620] Updated weights for policy 1, policy_version 766343 (0.0005) [2023-12-26 20:56:27,924][105692] Updated weights for policy 0, policy_version 766284 (0.0009) [2023-12-26 20:56:27,981][105692] Updated weights for policy 0, policy_version 766295 (0.0010) [2023-12-26 20:56:28,029][105692] Updated weights for policy 0, policy_version 766305 (0.0007) [2023-12-26 20:56:28,139][105620] Updated weights for policy 1, policy_version 766353 (0.0010) [2023-12-26 20:56:28,187][105620] Updated weights for policy 1, policy_version 766363 (0.0010) [2023-12-26 20:56:28,236][105620] Updated weights for policy 1, policy_version 766373 (0.0005) [2023-12-26 20:56:28,818][105620] Updated weights for policy 1, policy_version 766383 (0.0007) [2023-12-26 20:56:28,848][105692] Updated weights for policy 0, policy_version 766315 (0.0007) [2023-12-26 20:56:28,880][105620] Updated weights for policy 1, policy_version 766393 (0.0008) [2023-12-26 20:56:28,906][105692] Updated weights for policy 0, policy_version 766325 (0.0006) [2023-12-26 20:56:28,945][105620] Updated weights for policy 1, policy_version 766403 (0.0007) [2023-12-26 20:56:28,963][105692] Updated weights for policy 0, policy_version 766335 (0.0010) [2023-12-26 20:56:29,635][105620] Updated weights for policy 1, policy_version 766413 (0.0007) [2023-12-26 20:56:29,680][105692] Updated weights for policy 0, policy_version 766345 (0.0008) [2023-12-26 20:56:29,690][105620] Updated weights for policy 1, policy_version 766423 (0.0009) [2023-12-26 20:56:29,734][105692] Updated weights for policy 0, policy_version 766355 (0.0005) [2023-12-26 20:56:29,757][105620] Updated weights for policy 1, policy_version 766433 (0.0009) [2023-12-26 20:56:29,787][105692] Updated weights for policy 0, policy_version 766365 (0.0007) [2023-12-26 20:56:29,843][105692] Updated weights for policy 0, policy_version 766375 (0.0011) [2023-12-26 20:56:30,432][105620] Updated weights for policy 1, policy_version 766443 (0.0008) [2023-12-26 20:56:30,443][105692] Updated weights for policy 0, policy_version 766385 (0.0006) [2023-12-26 20:56:30,489][105620] Updated weights for policy 1, policy_version 766453 (0.0006) [2023-12-26 20:56:30,501][105692] Updated weights for policy 0, policy_version 766395 (0.0006) [2023-12-26 20:56:30,542][105620] Updated weights for policy 1, policy_version 766463 (0.0006) [2023-12-26 20:56:30,548][105692] Updated weights for policy 0, policy_version 766405 (0.0010) [2023-12-26 20:56:31,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 392470528. Throughput: 0: 9759.9, 1: 9941.8. Samples: 392441748. Policy #0 lag: (min: 14.0, avg: 34.2, max: 46.0) [2023-12-26 20:56:31,063][104569] Avg episode reward: [(0, '9170.067'), (1, '9261.506')] [2023-12-26 20:56:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000766408_196231168.pth... [2023-12-26 20:56:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000766472_196239360.pth... [2023-12-26 20:56:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000765256_195936256.pth [2023-12-26 20:56:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000765288_195936256.pth [2023-12-26 20:56:31,208][105692] Updated weights for policy 0, policy_version 766415 (0.0007) [2023-12-26 20:56:31,217][105620] Updated weights for policy 1, policy_version 766473 (0.0007) [2023-12-26 20:56:31,268][105692] Updated weights for policy 0, policy_version 766425 (0.0009) [2023-12-26 20:56:31,282][105620] Updated weights for policy 1, policy_version 766483 (0.0007) [2023-12-26 20:56:31,331][105692] Updated weights for policy 0, policy_version 766435 (0.0010) [2023-12-26 20:56:31,333][105620] Updated weights for policy 1, policy_version 766493 (0.0007) [2023-12-26 20:56:31,397][105620] Updated weights for policy 1, policy_version 766503 (0.0008) [2023-12-26 20:56:32,085][105692] Updated weights for policy 0, policy_version 766445 (0.0009) [2023-12-26 20:56:32,125][105620] Updated weights for policy 1, policy_version 766513 (0.0006) [2023-12-26 20:56:32,144][105692] Updated weights for policy 0, policy_version 766455 (0.0010) [2023-12-26 20:56:32,184][105620] Updated weights for policy 1, policy_version 766523 (0.0005) [2023-12-26 20:56:32,205][105692] Updated weights for policy 0, policy_version 766465 (0.0010) [2023-12-26 20:56:32,247][105620] Updated weights for policy 1, policy_version 766533 (0.0006) [2023-12-26 20:56:32,791][105620] Updated weights for policy 1, policy_version 766543 (0.0006) [2023-12-26 20:56:32,851][105692] Updated weights for policy 0, policy_version 766475 (0.0009) [2023-12-26 20:56:32,855][105620] Updated weights for policy 1, policy_version 766553 (0.0006) [2023-12-26 20:56:32,911][105620] Updated weights for policy 1, policy_version 766563 (0.0006) [2023-12-26 20:56:32,912][105692] Updated weights for policy 0, policy_version 766485 (0.0006) [2023-12-26 20:56:32,974][105692] Updated weights for policy 0, policy_version 766495 (0.0010) [2023-12-26 20:56:33,597][105620] Updated weights for policy 1, policy_version 766573 (0.0007) [2023-12-26 20:56:33,641][105620] Updated weights for policy 1, policy_version 766583 (0.0008) [2023-12-26 20:56:33,695][105620] Updated weights for policy 1, policy_version 766593 (0.0008) [2023-12-26 20:56:33,704][105692] Updated weights for policy 0, policy_version 766505 (0.0010) [2023-12-26 20:56:33,761][105692] Updated weights for policy 0, policy_version 766515 (0.0010) [2023-12-26 20:56:33,808][105692] Updated weights for policy 0, policy_version 766525 (0.0010) [2023-12-26 20:56:33,862][105692] Updated weights for policy 0, policy_version 766535 (0.0009) [2023-12-26 20:56:34,505][105620] Updated weights for policy 1, policy_version 766603 (0.0009) [2023-12-26 20:56:34,536][105692] Updated weights for policy 0, policy_version 766545 (0.0006) [2023-12-26 20:56:34,566][105620] Updated weights for policy 1, policy_version 766613 (0.0008) [2023-12-26 20:56:34,597][105692] Updated weights for policy 0, policy_version 766555 (0.0008) [2023-12-26 20:56:34,636][105620] Updated weights for policy 1, policy_version 766623 (0.0008) [2023-12-26 20:56:34,660][105692] Updated weights for policy 0, policy_version 766565 (0.0006) [2023-12-26 20:56:35,300][105692] Updated weights for policy 0, policy_version 766575 (0.0005) [2023-12-26 20:56:35,307][105620] Updated weights for policy 1, policy_version 766633 (0.0009) [2023-12-26 20:56:35,356][105620] Updated weights for policy 1, policy_version 766643 (0.0009) [2023-12-26 20:56:35,358][105692] Updated weights for policy 0, policy_version 766585 (0.0005) [2023-12-26 20:56:35,406][105620] Updated weights for policy 1, policy_version 766653 (0.0008) [2023-12-26 20:56:35,420][105692] Updated weights for policy 0, policy_version 766595 (0.0006) [2023-12-26 20:56:35,467][105620] Updated weights for policy 1, policy_version 766663 (0.0008) [2023-12-26 20:56:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 392568832. Throughput: 0: 9791.5, 1: 10032.3. Samples: 392561456. Policy #0 lag: (min: 14.0, avg: 34.2, max: 46.0) [2023-12-26 20:56:36,062][104569] Avg episode reward: [(0, '9172.815'), (1, '9262.433')] [2023-12-26 20:56:36,072][105692] Updated weights for policy 0, policy_version 766605 (0.0007) [2023-12-26 20:56:36,135][105692] Updated weights for policy 0, policy_version 766615 (0.0009) [2023-12-26 20:56:36,195][105692] Updated weights for policy 0, policy_version 766625 (0.0009) [2023-12-26 20:56:36,275][105620] Updated weights for policy 1, policy_version 766673 (0.0009) [2023-12-26 20:56:36,333][105620] Updated weights for policy 1, policy_version 766683 (0.0009) [2023-12-26 20:56:36,392][105620] Updated weights for policy 1, policy_version 766693 (0.0009) [2023-12-26 20:56:36,826][105692] Updated weights for policy 0, policy_version 766635 (0.0009) [2023-12-26 20:56:36,874][105692] Updated weights for policy 0, policy_version 766645 (0.0009) [2023-12-26 20:56:36,929][105692] Updated weights for policy 0, policy_version 766655 (0.0009) [2023-12-26 20:56:37,201][105620] Updated weights for policy 1, policy_version 766703 (0.0010) [2023-12-26 20:56:37,261][105620] Updated weights for policy 1, policy_version 766713 (0.0009) [2023-12-26 20:56:37,316][105620] Updated weights for policy 1, policy_version 766723 (0.0009) [2023-12-26 20:56:37,661][105692] Updated weights for policy 0, policy_version 766665 (0.0008) [2023-12-26 20:56:37,727][105692] Updated weights for policy 0, policy_version 766675 (0.0006) [2023-12-26 20:56:37,786][105692] Updated weights for policy 0, policy_version 766685 (0.0009) [2023-12-26 20:56:37,841][105692] Updated weights for policy 0, policy_version 766695 (0.0009) [2023-12-26 20:56:38,100][105620] Updated weights for policy 1, policy_version 766733 (0.0009) [2023-12-26 20:56:38,165][105620] Updated weights for policy 1, policy_version 766743 (0.0009) [2023-12-26 20:56:38,227][105620] Updated weights for policy 1, policy_version 766753 (0.0009) [2023-12-26 20:56:38,527][105692] Updated weights for policy 0, policy_version 766705 (0.0006) [2023-12-26 20:56:38,591][105692] Updated weights for policy 0, policy_version 766715 (0.0006) [2023-12-26 20:56:38,650][105692] Updated weights for policy 0, policy_version 766725 (0.0006) [2023-12-26 20:56:39,041][105620] Updated weights for policy 1, policy_version 766763 (0.0009) [2023-12-26 20:56:39,099][105620] Updated weights for policy 1, policy_version 766773 (0.0009) [2023-12-26 20:56:39,124][105586] KL-divergence is very high: 105.7407 [2023-12-26 20:56:39,158][105620] Updated weights for policy 1, policy_version 766783 (0.0009) [2023-12-26 20:56:39,317][105692] Updated weights for policy 0, policy_version 766735 (0.0010) [2023-12-26 20:56:39,381][105692] Updated weights for policy 0, policy_version 766745 (0.0010) [2023-12-26 20:56:39,442][105692] Updated weights for policy 0, policy_version 766755 (0.0008) [2023-12-26 20:56:39,999][105620] Updated weights for policy 1, policy_version 766793 (0.0010) [2023-12-26 20:56:40,059][105620] Updated weights for policy 1, policy_version 766803 (0.0010) [2023-12-26 20:56:40,079][105692] Updated weights for policy 0, policy_version 766765 (0.0006) [2023-12-26 20:56:40,123][105620] Updated weights for policy 1, policy_version 766813 (0.0009) [2023-12-26 20:56:40,129][105692] Updated weights for policy 0, policy_version 766775 (0.0011) [2023-12-26 20:56:40,185][105692] Updated weights for policy 0, policy_version 766785 (0.0007) [2023-12-26 20:56:40,188][105620] Updated weights for policy 1, policy_version 766823 (0.0008) [2023-12-26 20:56:40,922][105692] Updated weights for policy 0, policy_version 766795 (0.0009) [2023-12-26 20:56:40,961][105620] Updated weights for policy 1, policy_version 766833 (0.0010) [2023-12-26 20:56:40,979][105692] Updated weights for policy 0, policy_version 766805 (0.0006) [2023-12-26 20:56:41,022][105620] Updated weights for policy 1, policy_version 766843 (0.0008) [2023-12-26 20:56:41,049][105692] Updated weights for policy 0, policy_version 766815 (0.0006) [2023-12-26 20:56:41,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 392658944. Throughput: 0: 9876.8, 1: 9995.2. Samples: 392676168. Policy #0 lag: (min: 14.0, avg: 34.2, max: 46.0) [2023-12-26 20:56:41,062][104569] Avg episode reward: [(0, '9263.939'), (1, '9171.192')] [2023-12-26 20:56:41,086][105620] Updated weights for policy 1, policy_version 766853 (0.0008) [2023-12-26 20:56:41,789][105692] Updated weights for policy 0, policy_version 766825 (0.0008) [2023-12-26 20:56:41,848][105692] Updated weights for policy 0, policy_version 766835 (0.0009) [2023-12-26 20:56:41,905][105620] Updated weights for policy 1, policy_version 766863 (0.0008) [2023-12-26 20:56:41,912][105692] Updated weights for policy 0, policy_version 766845 (0.0006) [2023-12-26 20:56:41,967][105620] Updated weights for policy 1, policy_version 766873 (0.0008) [2023-12-26 20:56:41,981][105692] Updated weights for policy 0, policy_version 766855 (0.0008) [2023-12-26 20:56:42,033][105620] Updated weights for policy 1, policy_version 766883 (0.0009) [2023-12-26 20:56:42,570][105692] Updated weights for policy 0, policy_version 766865 (0.0009) [2023-12-26 20:56:42,633][105692] Updated weights for policy 0, policy_version 766875 (0.0009) [2023-12-26 20:56:42,695][105692] Updated weights for policy 0, policy_version 766885 (0.0009) [2023-12-26 20:56:42,846][105620] Updated weights for policy 1, policy_version 766893 (0.0009) [2023-12-26 20:56:42,897][105620] Updated weights for policy 1, policy_version 766903 (0.0009) [2023-12-26 20:56:42,959][105620] Updated weights for policy 1, policy_version 766913 (0.0009) [2023-12-26 20:56:43,408][105692] Updated weights for policy 0, policy_version 766895 (0.0009) [2023-12-26 20:56:43,468][105692] Updated weights for policy 0, policy_version 766905 (0.0007) [2023-12-26 20:56:43,520][105692] Updated weights for policy 0, policy_version 766915 (0.0005) [2023-12-26 20:56:43,789][105620] Updated weights for policy 1, policy_version 766923 (0.0009) [2023-12-26 20:56:43,842][105620] Updated weights for policy 1, policy_version 766934 (0.0009) [2023-12-26 20:56:43,900][105620] Updated weights for policy 1, policy_version 766946 (0.0010) [2023-12-26 20:56:44,072][105692] Updated weights for policy 0, policy_version 766925 (0.0006) [2023-12-26 20:56:44,132][105692] Updated weights for policy 0, policy_version 766935 (0.0009) [2023-12-26 20:56:44,184][105692] Updated weights for policy 0, policy_version 766945 (0.0010) [2023-12-26 20:56:44,711][105620] Updated weights for policy 1, policy_version 766957 (0.0011) [2023-12-26 20:56:44,765][105620] Updated weights for policy 1, policy_version 766967 (0.0009) [2023-12-26 20:56:44,827][105620] Updated weights for policy 1, policy_version 766977 (0.0009) [2023-12-26 20:56:44,829][105692] Updated weights for policy 0, policy_version 766955 (0.0009) [2023-12-26 20:56:44,895][105692] Updated weights for policy 0, policy_version 766965 (0.0008) [2023-12-26 20:56:44,951][105692] Updated weights for policy 0, policy_version 766975 (0.0009) [2023-12-26 20:56:45,609][105620] Updated weights for policy 1, policy_version 766987 (0.0007) [2023-12-26 20:56:45,639][105692] Updated weights for policy 0, policy_version 766985 (0.0009) [2023-12-26 20:56:45,667][105620] Updated weights for policy 1, policy_version 766997 (0.0009) [2023-12-26 20:56:45,701][105692] Updated weights for policy 0, policy_version 766995 (0.0005) [2023-12-26 20:56:45,726][105620] Updated weights for policy 1, policy_version 767007 (0.0008) [2023-12-26 20:56:45,757][105692] Updated weights for policy 0, policy_version 767005 (0.0005) [2023-12-26 20:56:45,824][105692] Updated weights for policy 0, policy_version 767015 (0.0006) [2023-12-26 20:56:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 392765440. Throughput: 0: 9909.6, 1: 9876.3. Samples: 392731552. Policy #0 lag: (min: 14.0, avg: 34.2, max: 46.0) [2023-12-26 20:56:46,063][104569] Avg episode reward: [(0, '9259.492'), (1, '8995.028')] [2023-12-26 20:56:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000767016_196378624.pth... [2023-12-26 20:56:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000767016_196386816.pth... [2023-12-26 20:56:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000765896_196091904.pth [2023-12-26 20:56:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000765832_196083712.pth [2023-12-26 20:56:46,428][105692] Updated weights for policy 0, policy_version 767025 (0.0008) [2023-12-26 20:56:46,477][105692] Updated weights for policy 0, policy_version 767035 (0.0010) [2023-12-26 20:56:46,516][105620] Updated weights for policy 1, policy_version 767017 (0.0009) [2023-12-26 20:56:46,531][105692] Updated weights for policy 0, policy_version 767045 (0.0006) [2023-12-26 20:56:46,570][105620] Updated weights for policy 1, policy_version 767027 (0.0009) [2023-12-26 20:56:46,628][105620] Updated weights for policy 1, policy_version 767037 (0.0010) [2023-12-26 20:56:46,680][105620] Updated weights for policy 1, policy_version 767047 (0.0008) [2023-12-26 20:56:47,137][105692] Updated weights for policy 0, policy_version 767055 (0.0006) [2023-12-26 20:56:47,206][105692] Updated weights for policy 0, policy_version 767065 (0.0006) [2023-12-26 20:56:47,261][105692] Updated weights for policy 0, policy_version 767075 (0.0007) [2023-12-26 20:56:47,565][105620] Updated weights for policy 1, policy_version 767057 (0.0009) [2023-12-26 20:56:47,618][105620] Updated weights for policy 1, policy_version 767067 (0.0010) [2023-12-26 20:56:47,672][105620] Updated weights for policy 1, policy_version 767077 (0.0010) [2023-12-26 20:56:47,793][105692] Updated weights for policy 0, policy_version 767085 (0.0008) [2023-12-26 20:56:47,852][105692] Updated weights for policy 0, policy_version 767095 (0.0006) [2023-12-26 20:56:47,903][105692] Updated weights for policy 0, policy_version 767105 (0.0009) [2023-12-26 20:56:48,545][105692] Updated weights for policy 0, policy_version 767115 (0.0011) [2023-12-26 20:56:48,551][105620] Updated weights for policy 1, policy_version 767087 (0.0008) [2023-12-26 20:56:48,611][105620] Updated weights for policy 1, policy_version 767097 (0.0006) [2023-12-26 20:56:48,611][105692] Updated weights for policy 0, policy_version 767125 (0.0011) [2023-12-26 20:56:48,666][105620] Updated weights for policy 1, policy_version 767107 (0.0007) [2023-12-26 20:56:48,672][105692] Updated weights for policy 0, policy_version 767135 (0.0011) [2023-12-26 20:56:49,424][105692] Updated weights for policy 0, policy_version 767145 (0.0011) [2023-12-26 20:56:49,434][105620] Updated weights for policy 1, policy_version 767117 (0.0006) [2023-12-26 20:56:49,477][105620] Updated weights for policy 1, policy_version 767127 (0.0007) [2023-12-26 20:56:49,484][105692] Updated weights for policy 0, policy_version 767155 (0.0010) [2023-12-26 20:56:49,530][105620] Updated weights for policy 1, policy_version 767137 (0.0005) [2023-12-26 20:56:49,532][105692] Updated weights for policy 0, policy_version 767165 (0.0010) [2023-12-26 20:56:49,583][105692] Updated weights for policy 0, policy_version 767176 (0.0010) [2023-12-26 20:56:50,331][105692] Updated weights for policy 0, policy_version 767186 (0.0010) [2023-12-26 20:56:50,346][105620] Updated weights for policy 1, policy_version 767147 (0.0006) [2023-12-26 20:56:50,366][105586] KL-divergence is very high: 122.6711 [2023-12-26 20:56:50,379][105586] KL-divergence is very high: 145.4846 [2023-12-26 20:56:50,387][105692] Updated weights for policy 0, policy_version 767196 (0.0007) [2023-12-26 20:56:50,410][105620] Updated weights for policy 1, policy_version 767157 (0.0008) [2023-12-26 20:56:50,415][105586] KL-divergence is very high: 139.9395 [2023-12-26 20:56:50,426][105586] KL-divergence is very high: 140.3803 [2023-12-26 20:56:50,449][105692] Updated weights for policy 0, policy_version 767206 (0.0008) [2023-12-26 20:56:50,470][105620] Updated weights for policy 1, policy_version 767167 (0.0009) [2023-12-26 20:56:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 392855552. Throughput: 0: 9998.7, 1: 9679.5. Samples: 392849076. Policy #0 lag: (min: 14.0, avg: 34.2, max: 46.0) [2023-12-26 20:56:51,062][104569] Avg episode reward: [(0, '9080.895'), (1, '8904.964')] [2023-12-26 20:56:51,070][105692] Updated weights for policy 0, policy_version 767216 (0.0008) [2023-12-26 20:56:51,132][105692] Updated weights for policy 0, policy_version 767226 (0.0007) [2023-12-26 20:56:51,199][105692] Updated weights for policy 0, policy_version 767236 (0.0008) [2023-12-26 20:56:51,299][105620] Updated weights for policy 1, policy_version 767177 (0.0008) [2023-12-26 20:56:51,370][105620] Updated weights for policy 1, policy_version 767187 (0.0007) [2023-12-26 20:56:51,436][105620] Updated weights for policy 1, policy_version 767197 (0.0009) [2023-12-26 20:56:51,499][105620] Updated weights for policy 1, policy_version 767207 (0.0009) [2023-12-26 20:56:51,857][105692] Updated weights for policy 0, policy_version 767246 (0.0005) [2023-12-26 20:56:51,913][105692] Updated weights for policy 0, policy_version 767256 (0.0005) [2023-12-26 20:56:51,972][105692] Updated weights for policy 0, policy_version 767266 (0.0006) [2023-12-26 20:56:52,140][105620] Updated weights for policy 1, policy_version 767217 (0.0006) [2023-12-26 20:56:52,194][105620] Updated weights for policy 1, policy_version 767227 (0.0006) [2023-12-26 20:56:52,242][105620] Updated weights for policy 1, policy_version 767237 (0.0005) [2023-12-26 20:56:52,720][105692] Updated weights for policy 0, policy_version 767276 (0.0007) [2023-12-26 20:56:52,786][105692] Updated weights for policy 0, policy_version 767286 (0.0007) [2023-12-26 20:56:52,856][105692] Updated weights for policy 0, policy_version 767296 (0.0009) [2023-12-26 20:56:52,903][105620] Updated weights for policy 1, policy_version 767247 (0.0006) [2023-12-26 20:56:52,959][105586] KL-divergence is very high: 138.3439 [2023-12-26 20:56:52,966][105620] Updated weights for policy 1, policy_version 767257 (0.0008) [2023-12-26 20:56:53,005][105586] KL-divergence is very high: 119.8886 [2023-12-26 20:56:53,024][105620] Updated weights for policy 1, policy_version 767267 (0.0008) [2023-12-26 20:56:53,618][105620] Updated weights for policy 1, policy_version 767277 (0.0007) [2023-12-26 20:56:53,628][105692] Updated weights for policy 0, policy_version 767306 (0.0009) [2023-12-26 20:56:53,678][105620] Updated weights for policy 1, policy_version 767287 (0.0006) [2023-12-26 20:56:53,682][105692] Updated weights for policy 0, policy_version 767316 (0.0006) [2023-12-26 20:56:53,745][105620] Updated weights for policy 1, policy_version 767297 (0.0009) [2023-12-26 20:56:53,747][105692] Updated weights for policy 0, policy_version 767326 (0.0005) [2023-12-26 20:56:53,796][105692] Updated weights for policy 0, policy_version 767336 (0.0006) [2023-12-26 20:56:54,367][105620] Updated weights for policy 1, policy_version 767307 (0.0007) [2023-12-26 20:56:54,430][105620] Updated weights for policy 1, policy_version 767317 (0.0005) [2023-12-26 20:56:54,490][105620] Updated weights for policy 1, policy_version 767327 (0.0007) [2023-12-26 20:56:54,513][105692] Updated weights for policy 0, policy_version 767346 (0.0006) [2023-12-26 20:56:54,576][105692] Updated weights for policy 0, policy_version 767356 (0.0005) [2023-12-26 20:56:54,639][105692] Updated weights for policy 0, policy_version 767366 (0.0005) [2023-12-26 20:56:55,179][105620] Updated weights for policy 1, policy_version 767337 (0.0010) [2023-12-26 20:56:55,236][105620] Updated weights for policy 1, policy_version 767347 (0.0010) [2023-12-26 20:56:55,238][105692] Updated weights for policy 0, policy_version 767376 (0.0005) [2023-12-26 20:56:55,288][105692] Updated weights for policy 0, policy_version 767386 (0.0005) [2023-12-26 20:56:55,301][105620] Updated weights for policy 1, policy_version 767357 (0.0010) [2023-12-26 20:56:55,335][105692] Updated weights for policy 0, policy_version 767396 (0.0005) [2023-12-26 20:56:55,360][105620] Updated weights for policy 1, policy_version 767367 (0.0010) [2023-12-26 20:56:55,927][105692] Updated weights for policy 0, policy_version 767406 (0.0007) [2023-12-26 20:56:55,963][105620] Updated weights for policy 1, policy_version 767377 (0.0009) [2023-12-26 20:56:55,981][105692] Updated weights for policy 0, policy_version 767416 (0.0007) [2023-12-26 20:56:56,022][105620] Updated weights for policy 1, policy_version 767387 (0.0010) [2023-12-26 20:56:56,036][105692] Updated weights for policy 0, policy_version 767426 (0.0005) [2023-12-26 20:56:56,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.9, 300 sec: 19633.0). Total num frames: 392953856. Throughput: 0: 10045.0, 1: 9644.2. Samples: 392970036. Policy #0 lag: (min: 14.0, avg: 34.2, max: 46.0) [2023-12-26 20:56:56,062][104569] Avg episode reward: [(0, '9263.146'), (1, '8718.292')] [2023-12-26 20:56:56,087][105620] Updated weights for policy 1, policy_version 767397 (0.0010) [2023-12-26 20:56:56,689][105692] Updated weights for policy 0, policy_version 767436 (0.0007) [2023-12-26 20:56:56,699][105620] Updated weights for policy 1, policy_version 767407 (0.0008) [2023-12-26 20:56:56,739][105692] Updated weights for policy 0, policy_version 767446 (0.0005) [2023-12-26 20:56:56,743][105620] Updated weights for policy 1, policy_version 767417 (0.0010) [2023-12-26 20:56:56,788][105620] Updated weights for policy 1, policy_version 767427 (0.0010) [2023-12-26 20:56:56,792][105692] Updated weights for policy 0, policy_version 767456 (0.0005) [2023-12-26 20:56:57,400][105620] Updated weights for policy 1, policy_version 767437 (0.0008) [2023-12-26 20:56:57,408][105692] Updated weights for policy 0, policy_version 767466 (0.0007) [2023-12-26 20:56:57,448][105620] Updated weights for policy 1, policy_version 767447 (0.0010) [2023-12-26 20:56:57,463][105692] Updated weights for policy 0, policy_version 767476 (0.0010) [2023-12-26 20:56:57,494][105620] Updated weights for policy 1, policy_version 767457 (0.0005) [2023-12-26 20:56:57,515][105692] Updated weights for policy 0, policy_version 767486 (0.0007) [2023-12-26 20:56:57,572][105692] Updated weights for policy 0, policy_version 767496 (0.0009) [2023-12-26 20:56:58,041][105620] Updated weights for policy 1, policy_version 767467 (0.0005) [2023-12-26 20:56:58,093][105620] Updated weights for policy 1, policy_version 767477 (0.0006) [2023-12-26 20:56:58,153][105620] Updated weights for policy 1, policy_version 767487 (0.0006) [2023-12-26 20:56:58,306][105692] Updated weights for policy 0, policy_version 767506 (0.0010) [2023-12-26 20:56:58,388][105692] Updated weights for policy 0, policy_version 767516 (0.0007) [2023-12-26 20:56:58,453][105692] Updated weights for policy 0, policy_version 767526 (0.0006) [2023-12-26 20:56:58,969][105620] Updated weights for policy 1, policy_version 767497 (0.0006) [2023-12-26 20:56:59,032][105620] Updated weights for policy 1, policy_version 767507 (0.0008) [2023-12-26 20:56:59,100][105620] Updated weights for policy 1, policy_version 767517 (0.0009) [2023-12-26 20:56:59,138][105692] Updated weights for policy 0, policy_version 767536 (0.0010) [2023-12-26 20:56:59,167][105620] Updated weights for policy 1, policy_version 767527 (0.0008) [2023-12-26 20:56:59,196][105692] Updated weights for policy 0, policy_version 767546 (0.0010) [2023-12-26 20:56:59,261][105692] Updated weights for policy 0, policy_version 767556 (0.0009) [2023-12-26 20:56:59,918][105620] Updated weights for policy 1, policy_version 767537 (0.0010) [2023-12-26 20:56:59,982][105620] Updated weights for policy 1, policy_version 767547 (0.0011) [2023-12-26 20:57:00,030][105620] Updated weights for policy 1, policy_version 767557 (0.0010) [2023-12-26 20:57:00,031][105692] Updated weights for policy 0, policy_version 767566 (0.0009) [2023-12-26 20:57:00,078][105692] Updated weights for policy 0, policy_version 767576 (0.0007) [2023-12-26 20:57:00,127][105692] Updated weights for policy 0, policy_version 767586 (0.0008) [2023-12-26 20:57:00,761][105620] Updated weights for policy 1, policy_version 767567 (0.0007) [2023-12-26 20:57:00,806][105620] Updated weights for policy 1, policy_version 767577 (0.0005) [2023-12-26 20:57:00,854][105620] Updated weights for policy 1, policy_version 767587 (0.0006) [2023-12-26 20:57:00,933][105692] Updated weights for policy 0, policy_version 767596 (0.0009) [2023-12-26 20:57:00,991][105692] Updated weights for policy 0, policy_version 767606 (0.0009) [2023-12-26 20:57:01,049][105692] Updated weights for policy 0, policy_version 767617 (0.0008) [2023-12-26 20:57:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 393060352. Throughput: 0: 10134.2, 1: 9760.3. Samples: 393034092. Policy #0 lag: (min: 14.0, avg: 34.2, max: 46.0) [2023-12-26 20:57:01,062][104569] Avg episode reward: [(0, '9351.479'), (1, '8720.001')] [2023-12-26 20:57:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000767592_196526080.pth... [2023-12-26 20:57:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000766472_196239360.pth [2023-12-26 20:57:01,095][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000767624_196542464.pth... [2023-12-26 20:57:01,099][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000766408_196231168.pth [2023-12-26 20:57:01,471][105620] Updated weights for policy 1, policy_version 767597 (0.0005) [2023-12-26 20:57:01,517][105620] Updated weights for policy 1, policy_version 767607 (0.0005) [2023-12-26 20:57:01,566][105620] Updated weights for policy 1, policy_version 767617 (0.0005) [2023-12-26 20:57:01,908][105692] Updated weights for policy 0, policy_version 767627 (0.0008) [2023-12-26 20:57:01,967][105692] Updated weights for policy 0, policy_version 767637 (0.0007) [2023-12-26 20:57:02,038][105692] Updated weights for policy 0, policy_version 767647 (0.0009) [2023-12-26 20:57:02,196][105620] Updated weights for policy 1, policy_version 767627 (0.0006) [2023-12-26 20:57:02,258][105620] Updated weights for policy 1, policy_version 767637 (0.0008) [2023-12-26 20:57:02,325][105620] Updated weights for policy 1, policy_version 767647 (0.0007) [2023-12-26 20:57:02,722][105692] Updated weights for policy 0, policy_version 767657 (0.0008) [2023-12-26 20:57:02,774][105692] Updated weights for policy 0, policy_version 767667 (0.0010) [2023-12-26 20:57:02,825][105692] Updated weights for policy 0, policy_version 767677 (0.0010) [2023-12-26 20:57:02,873][105692] Updated weights for policy 0, policy_version 767687 (0.0010) [2023-12-26 20:57:02,899][105620] Updated weights for policy 1, policy_version 767657 (0.0006) [2023-12-26 20:57:02,956][105620] Updated weights for policy 1, policy_version 767667 (0.0005) [2023-12-26 20:57:03,019][105620] Updated weights for policy 1, policy_version 767677 (0.0005) [2023-12-26 20:57:03,085][105620] Updated weights for policy 1, policy_version 767687 (0.0005) [2023-12-26 20:57:03,643][105692] Updated weights for policy 0, policy_version 767697 (0.0010) [2023-12-26 20:57:03,694][105692] Updated weights for policy 0, policy_version 767707 (0.0010) [2023-12-26 20:57:03,734][105620] Updated weights for policy 1, policy_version 767697 (0.0005) [2023-12-26 20:57:03,753][105692] Updated weights for policy 0, policy_version 767717 (0.0010) [2023-12-26 20:57:03,794][105620] Updated weights for policy 1, policy_version 767707 (0.0007) [2023-12-26 20:57:03,858][105620] Updated weights for policy 1, policy_version 767717 (0.0007) [2023-12-26 20:57:04,526][105620] Updated weights for policy 1, policy_version 767727 (0.0010) [2023-12-26 20:57:04,553][105692] Updated weights for policy 0, policy_version 767727 (0.0010) [2023-12-26 20:57:04,585][105620] Updated weights for policy 1, policy_version 767737 (0.0010) [2023-12-26 20:57:04,601][105692] Updated weights for policy 0, policy_version 767737 (0.0010) [2023-12-26 20:57:04,645][105620] Updated weights for policy 1, policy_version 767747 (0.0010) [2023-12-26 20:57:04,656][105692] Updated weights for policy 0, policy_version 767747 (0.0010) [2023-12-26 20:57:05,353][105620] Updated weights for policy 1, policy_version 767757 (0.0010) [2023-12-26 20:57:05,415][105692] Updated weights for policy 0, policy_version 767757 (0.0010) [2023-12-26 20:57:05,416][105620] Updated weights for policy 1, policy_version 767767 (0.0010) [2023-12-26 20:57:05,468][105692] Updated weights for policy 0, policy_version 767767 (0.0010) [2023-12-26 20:57:05,473][105620] Updated weights for policy 1, policy_version 767777 (0.0011) [2023-12-26 20:57:05,521][105692] Updated weights for policy 0, policy_version 767777 (0.0010) [2023-12-26 20:57:06,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.4, 300 sec: 19688.6). Total num frames: 393158656. Throughput: 0: 10023.8, 1: 9723.4. Samples: 393150468. Policy #0 lag: (min: 14.0, avg: 34.2, max: 46.0) [2023-12-26 20:57:06,063][104569] Avg episode reward: [(0, '9263.224'), (1, '8994.924')] [2023-12-26 20:57:06,082][105620] Updated weights for policy 1, policy_version 767787 (0.0009) [2023-12-26 20:57:06,144][105620] Updated weights for policy 1, policy_version 767797 (0.0006) [2023-12-26 20:57:06,176][105692] Updated weights for policy 0, policy_version 767787 (0.0010) [2023-12-26 20:57:06,208][105620] Updated weights for policy 1, policy_version 767807 (0.0009) [2023-12-26 20:57:06,237][105692] Updated weights for policy 0, policy_version 767797 (0.0006) [2023-12-26 20:57:06,293][105692] Updated weights for policy 0, policy_version 767807 (0.0007) [2023-12-26 20:57:06,829][105620] Updated weights for policy 1, policy_version 767817 (0.0010) [2023-12-26 20:57:06,885][105620] Updated weights for policy 1, policy_version 767827 (0.0010) [2023-12-26 20:57:06,951][105620] Updated weights for policy 1, policy_version 767837 (0.0010) [2023-12-26 20:57:07,024][105620] Updated weights for policy 1, policy_version 767847 (0.0009) [2023-12-26 20:57:07,053][105692] Updated weights for policy 0, policy_version 767817 (0.0010) [2023-12-26 20:57:07,112][105692] Updated weights for policy 0, policy_version 767827 (0.0008) [2023-12-26 20:57:07,173][105692] Updated weights for policy 0, policy_version 767837 (0.0007) [2023-12-26 20:57:07,231][105692] Updated weights for policy 0, policy_version 767847 (0.0005) [2023-12-26 20:57:07,682][105620] Updated weights for policy 1, policy_version 767857 (0.0008) [2023-12-26 20:57:07,739][105620] Updated weights for policy 1, policy_version 767867 (0.0011) [2023-12-26 20:57:07,787][105620] Updated weights for policy 1, policy_version 767877 (0.0010) [2023-12-26 20:57:07,850][105692] Updated weights for policy 0, policy_version 767857 (0.0010) [2023-12-26 20:57:07,908][105692] Updated weights for policy 0, policy_version 767867 (0.0010) [2023-12-26 20:57:07,956][105692] Updated weights for policy 0, policy_version 767877 (0.0010) [2023-12-26 20:57:08,556][105620] Updated weights for policy 1, policy_version 767887 (0.0010) [2023-12-26 20:57:08,622][105620] Updated weights for policy 1, policy_version 767897 (0.0010) [2023-12-26 20:57:08,679][105692] Updated weights for policy 0, policy_version 767887 (0.0008) [2023-12-26 20:57:08,692][105620] Updated weights for policy 1, policy_version 767907 (0.0011) [2023-12-26 20:57:08,735][105692] Updated weights for policy 0, policy_version 767897 (0.0009) [2023-12-26 20:57:08,786][105692] Updated weights for policy 0, policy_version 767907 (0.0008) [2023-12-26 20:57:09,412][105620] Updated weights for policy 1, policy_version 767917 (0.0010) [2023-12-26 20:57:09,464][105692] Updated weights for policy 0, policy_version 767917 (0.0008) [2023-12-26 20:57:09,481][105620] Updated weights for policy 1, policy_version 767927 (0.0010) [2023-12-26 20:57:09,525][105692] Updated weights for policy 0, policy_version 767927 (0.0010) [2023-12-26 20:57:09,547][105620] Updated weights for policy 1, policy_version 767937 (0.0008) [2023-12-26 20:57:09,585][105692] Updated weights for policy 0, policy_version 767937 (0.0010) [2023-12-26 20:57:10,299][105620] Updated weights for policy 1, policy_version 767947 (0.0009) [2023-12-26 20:57:10,358][105620] Updated weights for policy 1, policy_version 767957 (0.0011) [2023-12-26 20:57:10,360][105692] Updated weights for policy 0, policy_version 767947 (0.0010) [2023-12-26 20:57:10,410][105620] Updated weights for policy 1, policy_version 767967 (0.0009) [2023-12-26 20:57:10,422][105692] Updated weights for policy 0, policy_version 767957 (0.0010) [2023-12-26 20:57:10,485][105692] Updated weights for policy 0, policy_version 767967 (0.0007) [2023-12-26 20:57:11,012][105620] Updated weights for policy 1, policy_version 767977 (0.0010) [2023-12-26 20:57:11,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 393256960. Throughput: 0: 9953.6, 1: 9797.5. Samples: 393270332. Policy #0 lag: (min: 14.0, avg: 34.2, max: 46.0) [2023-12-26 20:57:11,063][104569] Avg episode reward: [(0, '8996.727'), (1, '8993.788')] [2023-12-26 20:57:11,078][105620] Updated weights for policy 1, policy_version 767987 (0.0006) [2023-12-26 20:57:11,142][105620] Updated weights for policy 1, policy_version 767997 (0.0007) [2023-12-26 20:57:11,145][105692] Updated weights for policy 0, policy_version 767977 (0.0005) [2023-12-26 20:57:11,209][105620] Updated weights for policy 1, policy_version 768007 (0.0007) [2023-12-26 20:57:11,215][105692] Updated weights for policy 0, policy_version 767987 (0.0006) [2023-12-26 20:57:11,280][105692] Updated weights for policy 0, policy_version 767997 (0.0010) [2023-12-26 20:57:11,326][105692] Updated weights for policy 0, policy_version 768007 (0.0010) [2023-12-26 20:57:11,850][105620] Updated weights for policy 1, policy_version 768017 (0.0008) [2023-12-26 20:57:11,909][105620] Updated weights for policy 1, policy_version 768027 (0.0008) [2023-12-26 20:57:11,976][105620] Updated weights for policy 1, policy_version 768037 (0.0008) [2023-12-26 20:57:12,046][105692] Updated weights for policy 0, policy_version 768017 (0.0011) [2023-12-26 20:57:12,091][105692] Updated weights for policy 0, policy_version 768027 (0.0010) [2023-12-26 20:57:12,157][105692] Updated weights for policy 0, policy_version 768037 (0.0011) [2023-12-26 20:57:12,753][105620] Updated weights for policy 1, policy_version 768047 (0.0009) [2023-12-26 20:57:12,823][105620] Updated weights for policy 1, policy_version 768057 (0.0010) [2023-12-26 20:57:12,863][105692] Updated weights for policy 0, policy_version 768047 (0.0007) [2023-12-26 20:57:12,876][105620] Updated weights for policy 1, policy_version 768067 (0.0009) [2023-12-26 20:57:12,927][105692] Updated weights for policy 0, policy_version 768057 (0.0005) [2023-12-26 20:57:12,983][105692] Updated weights for policy 0, policy_version 768067 (0.0005) [2023-12-26 20:57:13,551][105692] Updated weights for policy 0, policy_version 768077 (0.0008) [2023-12-26 20:57:13,609][105692] Updated weights for policy 0, policy_version 768087 (0.0010) [2023-12-26 20:57:13,670][105692] Updated weights for policy 0, policy_version 768097 (0.0010) [2023-12-26 20:57:13,691][105620] Updated weights for policy 1, policy_version 768077 (0.0007) [2023-12-26 20:57:13,746][105620] Updated weights for policy 1, policy_version 768087 (0.0008) [2023-12-26 20:57:13,802][105620] Updated weights for policy 1, policy_version 768097 (0.0008) [2023-12-26 20:57:14,374][105692] Updated weights for policy 0, policy_version 768107 (0.0009) [2023-12-26 20:57:14,422][105692] Updated weights for policy 0, policy_version 768117 (0.0007) [2023-12-26 20:57:14,445][105620] Updated weights for policy 1, policy_version 768107 (0.0007) [2023-12-26 20:57:14,471][105692] Updated weights for policy 0, policy_version 768127 (0.0008) [2023-12-26 20:57:14,518][105620] Updated weights for policy 1, policy_version 768117 (0.0008) [2023-12-26 20:57:14,585][105620] Updated weights for policy 1, policy_version 768127 (0.0007) [2023-12-26 20:57:15,141][105692] Updated weights for policy 0, policy_version 768137 (0.0006) [2023-12-26 20:57:15,201][105692] Updated weights for policy 0, policy_version 768147 (0.0006) [2023-12-26 20:57:15,241][105620] Updated weights for policy 1, policy_version 768137 (0.0006) [2023-12-26 20:57:15,263][105692] Updated weights for policy 0, policy_version 768157 (0.0010) [2023-12-26 20:57:15,298][105620] Updated weights for policy 1, policy_version 768147 (0.0010) [2023-12-26 20:57:15,325][105692] Updated weights for policy 0, policy_version 768167 (0.0009) [2023-12-26 20:57:15,356][105620] Updated weights for policy 1, policy_version 768157 (0.0011) [2023-12-26 20:57:15,406][105620] Updated weights for policy 1, policy_version 768167 (0.0011) [2023-12-26 20:57:16,001][105620] Updated weights for policy 1, policy_version 768177 (0.0006) [2023-12-26 20:57:16,031][105692] Updated weights for policy 0, policy_version 768177 (0.0006) [2023-12-26 20:57:16,053][105620] Updated weights for policy 1, policy_version 768187 (0.0005) [2023-12-26 20:57:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 393355264. Throughput: 0: 10056.0, 1: 9674.3. Samples: 393329612. Policy #0 lag: (min: 14.0, avg: 34.2, max: 46.0) [2023-12-26 20:57:16,062][104569] Avg episode reward: [(0, '9003.519'), (1, '8807.831')] [2023-12-26 20:57:16,088][105692] Updated weights for policy 0, policy_version 768187 (0.0009) [2023-12-26 20:57:16,106][105620] Updated weights for policy 1, policy_version 768197 (0.0010) [2023-12-26 20:57:16,122][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000768200_196681728.pth... [2023-12-26 20:57:16,125][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000767016_196378624.pth [2023-12-26 20:57:16,137][105692] Updated weights for policy 0, policy_version 768197 (0.0009) [2023-12-26 20:57:16,155][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000768200_196689920.pth... [2023-12-26 20:57:16,159][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000767016_196386816.pth [2023-12-26 20:57:16,698][105692] Updated weights for policy 0, policy_version 768207 (0.0005) [2023-12-26 20:57:16,756][105692] Updated weights for policy 0, policy_version 768217 (0.0005) [2023-12-26 20:57:16,788][105620] Updated weights for policy 1, policy_version 768207 (0.0007) [2023-12-26 20:57:16,820][105692] Updated weights for policy 0, policy_version 768227 (0.0010) [2023-12-26 20:57:16,840][105586] KL-divergence is very high: 383.6775 [2023-12-26 20:57:16,847][105620] Updated weights for policy 1, policy_version 768217 (0.0006) [2023-12-26 20:57:16,881][105586] KL-divergence is very high: 756.2779 [2023-12-26 20:57:16,894][105620] Updated weights for policy 1, policy_version 768227 (0.0008) [2023-12-26 20:57:17,465][105692] Updated weights for policy 0, policy_version 768237 (0.0011) [2023-12-26 20:57:17,523][105692] Updated weights for policy 0, policy_version 768247 (0.0010) [2023-12-26 20:57:17,550][105620] Updated weights for policy 1, policy_version 768237 (0.0005) [2023-12-26 20:57:17,586][105692] Updated weights for policy 0, policy_version 768257 (0.0011) [2023-12-26 20:57:17,605][105620] Updated weights for policy 1, policy_version 768247 (0.0006) [2023-12-26 20:57:17,669][105620] Updated weights for policy 1, policy_version 768257 (0.0008) [2023-12-26 20:57:18,209][105692] Updated weights for policy 0, policy_version 768267 (0.0010) [2023-12-26 20:57:18,266][105692] Updated weights for policy 0, policy_version 768277 (0.0010) [2023-12-26 20:57:18,322][105692] Updated weights for policy 0, policy_version 768287 (0.0010) [2023-12-26 20:57:18,455][105620] Updated weights for policy 1, policy_version 768267 (0.0007) [2023-12-26 20:57:18,503][105620] Updated weights for policy 1, policy_version 768277 (0.0007) [2023-12-26 20:57:18,548][105620] Updated weights for policy 1, policy_version 768287 (0.0008) [2023-12-26 20:57:18,934][105692] Updated weights for policy 0, policy_version 768297 (0.0009) [2023-12-26 20:57:19,000][105692] Updated weights for policy 0, policy_version 768307 (0.0010) [2023-12-26 20:57:19,055][105692] Updated weights for policy 0, policy_version 768317 (0.0010) [2023-12-26 20:57:19,110][105692] Updated weights for policy 0, policy_version 768327 (0.0010) [2023-12-26 20:57:19,342][105620] Updated weights for policy 1, policy_version 768297 (0.0009) [2023-12-26 20:57:19,409][105620] Updated weights for policy 1, policy_version 768307 (0.0007) [2023-12-26 20:57:19,470][105620] Updated weights for policy 1, policy_version 768317 (0.0007) [2023-12-26 20:57:19,538][105620] Updated weights for policy 1, policy_version 768327 (0.0008) [2023-12-26 20:57:19,862][105692] Updated weights for policy 0, policy_version 768337 (0.0008) [2023-12-26 20:57:19,933][105692] Updated weights for policy 0, policy_version 768347 (0.0007) [2023-12-26 20:57:19,995][105692] Updated weights for policy 0, policy_version 768357 (0.0006) [2023-12-26 20:57:20,284][105620] Updated weights for policy 1, policy_version 768337 (0.0008) [2023-12-26 20:57:20,350][105620] Updated weights for policy 1, policy_version 768347 (0.0008) [2023-12-26 20:57:20,407][105620] Updated weights for policy 1, policy_version 768357 (0.0009) [2023-12-26 20:57:20,760][105692] Updated weights for policy 0, policy_version 768367 (0.0008) [2023-12-26 20:57:20,825][105692] Updated weights for policy 0, policy_version 768377 (0.0011) [2023-12-26 20:57:20,891][105692] Updated weights for policy 0, policy_version 768387 (0.0010) [2023-12-26 20:57:21,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19933.9, 300 sec: 19688.6). Total num frames: 393461760. Throughput: 0: 10113.8, 1: 9689.7. Samples: 393452612. Policy #0 lag: (min: 14.0, avg: 34.2, max: 46.0) [2023-12-26 20:57:21,062][104569] Avg episode reward: [(0, '9171.608'), (1, '8984.173')] [2023-12-26 20:57:21,188][105620] Updated weights for policy 1, policy_version 768367 (0.0008) [2023-12-26 20:57:21,256][105620] Updated weights for policy 1, policy_version 768377 (0.0008) [2023-12-26 20:57:21,323][105620] Updated weights for policy 1, policy_version 768387 (0.0008) [2023-12-26 20:57:21,630][105692] Updated weights for policy 0, policy_version 768397 (0.0010) [2023-12-26 20:57:21,686][105692] Updated weights for policy 0, policy_version 768407 (0.0011) [2023-12-26 20:57:21,753][105692] Updated weights for policy 0, policy_version 768417 (0.0010) [2023-12-26 20:57:22,058][105620] Updated weights for policy 1, policy_version 768397 (0.0009) [2023-12-26 20:57:22,126][105620] Updated weights for policy 1, policy_version 768407 (0.0008) [2023-12-26 20:57:22,188][105620] Updated weights for policy 1, policy_version 768417 (0.0008) [2023-12-26 20:57:22,545][105692] Updated weights for policy 0, policy_version 768427 (0.0009) [2023-12-26 20:57:22,608][105692] Updated weights for policy 0, policy_version 768438 (0.0009) [2023-12-26 20:57:22,666][105692] Updated weights for policy 0, policy_version 768448 (0.0009) [2023-12-26 20:57:22,907][105620] Updated weights for policy 1, policy_version 768427 (0.0008) [2023-12-26 20:57:22,955][105620] Updated weights for policy 1, policy_version 768437 (0.0008) [2023-12-26 20:57:23,010][105620] Updated weights for policy 1, policy_version 768447 (0.0009) [2023-12-26 20:57:23,394][105692] Updated weights for policy 0, policy_version 768458 (0.0008) [2023-12-26 20:57:23,460][105692] Updated weights for policy 0, policy_version 768468 (0.0007) [2023-12-26 20:57:23,513][105692] Updated weights for policy 0, policy_version 768478 (0.0010) [2023-12-26 20:57:23,698][105620] Updated weights for policy 1, policy_version 768457 (0.0009) [2023-12-26 20:57:23,754][105620] Updated weights for policy 1, policy_version 768467 (0.0005) [2023-12-26 20:57:23,808][105620] Updated weights for policy 1, policy_version 768477 (0.0009) [2023-12-26 20:57:23,860][105620] Updated weights for policy 1, policy_version 768487 (0.0009) [2023-12-26 20:57:24,141][105692] Updated weights for policy 0, policy_version 768489 (0.0008) [2023-12-26 20:57:24,195][105692] Updated weights for policy 0, policy_version 768499 (0.0010) [2023-12-26 20:57:24,263][105692] Updated weights for policy 0, policy_version 768509 (0.0008) [2023-12-26 20:57:24,331][105692] Updated weights for policy 0, policy_version 768519 (0.0005) [2023-12-26 20:57:24,506][105620] Updated weights for policy 1, policy_version 768497 (0.0008) [2023-12-26 20:57:24,568][105620] Updated weights for policy 1, policy_version 768507 (0.0009) [2023-12-26 20:57:24,629][105620] Updated weights for policy 1, policy_version 768517 (0.0009) [2023-12-26 20:57:25,004][105692] Updated weights for policy 0, policy_version 768529 (0.0006) [2023-12-26 20:57:25,073][105692] Updated weights for policy 0, policy_version 768539 (0.0010) [2023-12-26 20:57:25,142][105692] Updated weights for policy 0, policy_version 768549 (0.0010) [2023-12-26 20:57:25,268][105620] Updated weights for policy 1, policy_version 768527 (0.0007) [2023-12-26 20:57:25,317][105620] Updated weights for policy 1, policy_version 768537 (0.0005) [2023-12-26 20:57:25,375][105620] Updated weights for policy 1, policy_version 768547 (0.0006) [2023-12-26 20:57:25,702][105692] Updated weights for policy 0, policy_version 768559 (0.0009) [2023-12-26 20:57:25,768][105692] Updated weights for policy 0, policy_version 768569 (0.0007) [2023-12-26 20:57:25,840][105692] Updated weights for policy 0, policy_version 768579 (0.0008) [2023-12-26 20:57:25,921][105620] Updated weights for policy 1, policy_version 768557 (0.0006) [2023-12-26 20:57:25,971][105620] Updated weights for policy 1, policy_version 768567 (0.0005) [2023-12-26 20:57:26,018][105620] Updated weights for policy 1, policy_version 768577 (0.0007) [2023-12-26 20:57:26,062][104569] Fps is (10 sec: 21299.2, 60 sec: 20070.4, 300 sec: 19688.6). Total num frames: 393568256. Throughput: 0: 10061.0, 1: 9836.8. Samples: 393571568. Policy #0 lag: (min: 14.0, avg: 34.2, max: 46.0) [2023-12-26 20:57:26,062][104569] Avg episode reward: [(0, '9171.979'), (1, '9256.487')] [2023-12-26 20:57:26,384][105692] Updated weights for policy 0, policy_version 768589 (0.0007) [2023-12-26 20:57:26,444][105692] Updated weights for policy 0, policy_version 768599 (0.0006) [2023-12-26 20:57:26,488][105692] Updated weights for policy 0, policy_version 768609 (0.0006) [2023-12-26 20:57:26,782][105620] Updated weights for policy 1, policy_version 768587 (0.0009) [2023-12-26 20:57:26,837][105620] Updated weights for policy 1, policy_version 768597 (0.0007) [2023-12-26 20:57:26,898][105620] Updated weights for policy 1, policy_version 768607 (0.0005) [2023-12-26 20:57:27,138][105692] Updated weights for policy 0, policy_version 768619 (0.0005) [2023-12-26 20:57:27,191][105692] Updated weights for policy 0, policy_version 768629 (0.0005) [2023-12-26 20:57:27,252][105692] Updated weights for policy 0, policy_version 768639 (0.0009) [2023-12-26 20:57:27,643][105620] Updated weights for policy 1, policy_version 768617 (0.0006) [2023-12-26 20:57:27,690][105620] Updated weights for policy 1, policy_version 768627 (0.0008) [2023-12-26 20:57:27,741][105620] Updated weights for policy 1, policy_version 768637 (0.0008) [2023-12-26 20:57:27,785][105620] Updated weights for policy 1, policy_version 768647 (0.0008) [2023-12-26 20:57:27,937][105692] Updated weights for policy 0, policy_version 768649 (0.0010) [2023-12-26 20:57:27,991][105692] Updated weights for policy 0, policy_version 768659 (0.0010) [2023-12-26 20:57:28,042][105692] Updated weights for policy 0, policy_version 768669 (0.0010) [2023-12-26 20:57:28,092][105692] Updated weights for policy 0, policy_version 768679 (0.0007) [2023-12-26 20:57:28,589][105620] Updated weights for policy 1, policy_version 768657 (0.0007) [2023-12-26 20:57:28,647][105620] Updated weights for policy 1, policy_version 768667 (0.0008) [2023-12-26 20:57:28,710][105620] Updated weights for policy 1, policy_version 768677 (0.0008) [2023-12-26 20:57:28,801][105692] Updated weights for policy 0, policy_version 768689 (0.0010) [2023-12-26 20:57:28,859][105692] Updated weights for policy 0, policy_version 768699 (0.0010) [2023-12-26 20:57:28,914][105692] Updated weights for policy 0, policy_version 768709 (0.0011) [2023-12-26 20:57:29,526][105620] Updated weights for policy 1, policy_version 768687 (0.0009) [2023-12-26 20:57:29,584][105692] Updated weights for policy 0, policy_version 768719 (0.0007) [2023-12-26 20:57:29,591][105620] Updated weights for policy 1, policy_version 768697 (0.0009) [2023-12-26 20:57:29,626][105586] KL-divergence is very high: 105.4243 [2023-12-26 20:57:29,632][105692] Updated weights for policy 0, policy_version 768729 (0.0005) [2023-12-26 20:57:29,647][105620] Updated weights for policy 1, policy_version 768707 (0.0008) [2023-12-26 20:57:29,680][105692] Updated weights for policy 0, policy_version 768739 (0.0005) [2023-12-26 20:57:30,365][105692] Updated weights for policy 0, policy_version 768749 (0.0007) [2023-12-26 20:57:30,396][105620] Updated weights for policy 1, policy_version 768717 (0.0009) [2023-12-26 20:57:30,411][105692] Updated weights for policy 0, policy_version 768759 (0.0007) [2023-12-26 20:57:30,449][105620] Updated weights for policy 1, policy_version 768727 (0.0007) [2023-12-26 20:57:30,459][105692] Updated weights for policy 0, policy_version 768769 (0.0006) [2023-12-26 20:57:30,505][105620] Updated weights for policy 1, policy_version 768737 (0.0009) [2023-12-26 20:57:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 393658368. Throughput: 0: 10112.7, 1: 9882.5. Samples: 393631336. Policy #0 lag: (min: 14.0, avg: 34.2, max: 46.0) [2023-12-26 20:57:31,062][104569] Avg episode reward: [(0, '9355.888'), (1, '9168.660')] [2023-12-26 20:57:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000768776_196837376.pth... [2023-12-26 20:57:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000768744_196820992.pth... [2023-12-26 20:57:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000767592_196526080.pth [2023-12-26 20:57:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000767624_196542464.pth [2023-12-26 20:57:31,194][105692] Updated weights for policy 0, policy_version 768779 (0.0007) [2023-12-26 20:57:31,250][105692] Updated weights for policy 0, policy_version 768789 (0.0009) [2023-12-26 20:57:31,286][105620] Updated weights for policy 1, policy_version 768747 (0.0007) [2023-12-26 20:57:31,305][105692] Updated weights for policy 0, policy_version 768799 (0.0010) [2023-12-26 20:57:31,344][105620] Updated weights for policy 1, policy_version 768757 (0.0009) [2023-12-26 20:57:31,410][105620] Updated weights for policy 1, policy_version 768767 (0.0008) [2023-12-26 20:57:32,047][105692] Updated weights for policy 0, policy_version 768809 (0.0012) [2023-12-26 20:57:32,110][105692] Updated weights for policy 0, policy_version 768819 (0.0010) [2023-12-26 20:57:32,120][105620] Updated weights for policy 1, policy_version 768777 (0.0009) [2023-12-26 20:57:32,167][105692] Updated weights for policy 0, policy_version 768829 (0.0011) [2023-12-26 20:57:32,177][105620] Updated weights for policy 1, policy_version 768787 (0.0006) [2023-12-26 20:57:32,219][105692] Updated weights for policy 0, policy_version 768839 (0.0010) [2023-12-26 20:57:32,238][105620] Updated weights for policy 1, policy_version 768797 (0.0005) [2023-12-26 20:57:32,302][105620] Updated weights for policy 1, policy_version 768807 (0.0008) [2023-12-26 20:57:32,832][105692] Updated weights for policy 0, policy_version 768849 (0.0006) [2023-12-26 20:57:32,891][105692] Updated weights for policy 0, policy_version 768859 (0.0006) [2023-12-26 20:57:32,947][105692] Updated weights for policy 0, policy_version 768869 (0.0009) [2023-12-26 20:57:33,042][105620] Updated weights for policy 1, policy_version 768817 (0.0010) [2023-12-26 20:57:33,103][105620] Updated weights for policy 1, policy_version 768827 (0.0005) [2023-12-26 20:57:33,170][105620] Updated weights for policy 1, policy_version 768837 (0.0005) [2023-12-26 20:57:33,620][105692] Updated weights for policy 0, policy_version 768879 (0.0009) [2023-12-26 20:57:33,676][105692] Updated weights for policy 0, policy_version 768889 (0.0013) [2023-12-26 20:57:33,738][105692] Updated weights for policy 0, policy_version 768900 (0.0008) [2023-12-26 20:57:33,783][105620] Updated weights for policy 1, policy_version 768847 (0.0009) [2023-12-26 20:57:33,847][105620] Updated weights for policy 1, policy_version 768857 (0.0007) [2023-12-26 20:57:33,902][105620] Updated weights for policy 1, policy_version 768867 (0.0005) [2023-12-26 20:57:34,486][105692] Updated weights for policy 0, policy_version 768910 (0.0007) [2023-12-26 20:57:34,547][105692] Updated weights for policy 0, policy_version 768920 (0.0008) [2023-12-26 20:57:34,582][105620] Updated weights for policy 1, policy_version 768877 (0.0008) [2023-12-26 20:57:34,608][105692] Updated weights for policy 0, policy_version 768930 (0.0009) [2023-12-26 20:57:34,644][105620] Updated weights for policy 1, policy_version 768887 (0.0008) [2023-12-26 20:57:34,699][105620] Updated weights for policy 1, policy_version 768897 (0.0009) [2023-12-26 20:57:35,373][105620] Updated weights for policy 1, policy_version 768907 (0.0009) [2023-12-26 20:57:35,434][105692] Updated weights for policy 0, policy_version 768940 (0.0007) [2023-12-26 20:57:35,439][105620] Updated weights for policy 1, policy_version 768917 (0.0010) [2023-12-26 20:57:35,484][105620] Updated weights for policy 1, policy_version 768927 (0.0009) [2023-12-26 20:57:35,486][105692] Updated weights for policy 0, policy_version 768950 (0.0006) [2023-12-26 20:57:35,548][105692] Updated weights for policy 0, policy_version 768960 (0.0006) [2023-12-26 20:57:36,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 393756672. Throughput: 0: 10002.5, 1: 9985.4. Samples: 393748532. Policy #0 lag: (min: 14.0, avg: 34.2, max: 46.0) [2023-12-26 20:57:36,063][104569] Avg episode reward: [(0, '9085.358'), (1, '8900.508')] [2023-12-26 20:57:36,151][105692] Updated weights for policy 0, policy_version 768970 (0.0006) [2023-12-26 20:57:36,200][105692] Updated weights for policy 0, policy_version 768980 (0.0008) [2023-12-26 20:57:36,217][105620] Updated weights for policy 1, policy_version 768937 (0.0009) [2023-12-26 20:57:36,256][105692] Updated weights for policy 0, policy_version 768990 (0.0006) [2023-12-26 20:57:36,278][105620] Updated weights for policy 1, policy_version 768947 (0.0011) [2023-12-26 20:57:36,312][105692] Updated weights for policy 0, policy_version 769000 (0.0007) [2023-12-26 20:57:36,334][105620] Updated weights for policy 1, policy_version 768957 (0.0011) [2023-12-26 20:57:36,399][105620] Updated weights for policy 1, policy_version 768967 (0.0011) [2023-12-26 20:57:36,998][105692] Updated weights for policy 0, policy_version 769010 (0.0006) [2023-12-26 20:57:37,047][105692] Updated weights for policy 0, policy_version 769020 (0.0005) [2023-12-26 20:57:37,093][105692] Updated weights for policy 0, policy_version 769030 (0.0005) [2023-12-26 20:57:37,173][105620] Updated weights for policy 1, policy_version 768977 (0.0010) [2023-12-26 20:57:37,227][105620] Updated weights for policy 1, policy_version 768987 (0.0010) [2023-12-26 20:57:37,282][105620] Updated weights for policy 1, policy_version 768997 (0.0009) [2023-12-26 20:57:37,709][105692] Updated weights for policy 0, policy_version 769040 (0.0005) [2023-12-26 20:57:37,779][105692] Updated weights for policy 0, policy_version 769050 (0.0005) [2023-12-26 20:57:37,838][105692] Updated weights for policy 0, policy_version 769061 (0.0009) [2023-12-26 20:57:38,033][105620] Updated weights for policy 1, policy_version 769007 (0.0006) [2023-12-26 20:57:38,097][105620] Updated weights for policy 1, policy_version 769017 (0.0009) [2023-12-26 20:57:38,149][105620] Updated weights for policy 1, policy_version 769027 (0.0009) [2023-12-26 20:57:38,452][105692] Updated weights for policy 0, policy_version 769071 (0.0010) [2023-12-26 20:57:38,514][105692] Updated weights for policy 0, policy_version 769081 (0.0011) [2023-12-26 20:57:38,580][105692] Updated weights for policy 0, policy_version 769091 (0.0011) [2023-12-26 20:57:38,909][105620] Updated weights for policy 1, policy_version 769037 (0.0010) [2023-12-26 20:57:38,971][105620] Updated weights for policy 1, policy_version 769047 (0.0010) [2023-12-26 20:57:39,023][105620] Updated weights for policy 1, policy_version 769057 (0.0010) [2023-12-26 20:57:39,283][105692] Updated weights for policy 0, policy_version 769101 (0.0010) [2023-12-26 20:57:39,342][105692] Updated weights for policy 0, policy_version 769111 (0.0011) [2023-12-26 20:57:39,407][105692] Updated weights for policy 0, policy_version 769121 (0.0011) [2023-12-26 20:57:39,796][105620] Updated weights for policy 1, policy_version 769067 (0.0010) [2023-12-26 20:57:39,861][105620] Updated weights for policy 1, policy_version 769077 (0.0011) [2023-12-26 20:57:39,930][105620] Updated weights for policy 1, policy_version 769087 (0.0011) [2023-12-26 20:57:40,182][105692] Updated weights for policy 0, policy_version 769131 (0.0011) [2023-12-26 20:57:40,248][105692] Updated weights for policy 0, policy_version 769141 (0.0011) [2023-12-26 20:57:40,311][105692] Updated weights for policy 0, policy_version 769151 (0.0011) [2023-12-26 20:57:40,628][105620] Updated weights for policy 1, policy_version 769097 (0.0010) [2023-12-26 20:57:40,680][105620] Updated weights for policy 1, policy_version 769107 (0.0007) [2023-12-26 20:57:40,737][105620] Updated weights for policy 1, policy_version 769117 (0.0007) [2023-12-26 20:57:40,795][105620] Updated weights for policy 1, policy_version 769127 (0.0008) [2023-12-26 20:57:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19933.8, 300 sec: 19633.0). Total num frames: 393854976. Throughput: 0: 10007.0, 1: 9892.3. Samples: 393865508. Policy #0 lag: (min: 14.0, avg: 34.2, max: 46.0) [2023-12-26 20:57:41,063][104569] Avg episode reward: [(0, '8908.107'), (1, '8636.241')] [2023-12-26 20:57:41,117][105692] Updated weights for policy 0, policy_version 769161 (0.0010) [2023-12-26 20:57:41,183][105692] Updated weights for policy 0, policy_version 769171 (0.0008) [2023-12-26 20:57:41,246][105692] Updated weights for policy 0, policy_version 769181 (0.0008) [2023-12-26 20:57:41,308][105692] Updated weights for policy 0, policy_version 769191 (0.0008) [2023-12-26 20:57:41,524][105620] Updated weights for policy 1, policy_version 769137 (0.0009) [2023-12-26 20:57:41,585][105620] Updated weights for policy 1, policy_version 769147 (0.0009) [2023-12-26 20:57:41,653][105620] Updated weights for policy 1, policy_version 769157 (0.0008) [2023-12-26 20:57:42,079][105692] Updated weights for policy 0, policy_version 769201 (0.0009) [2023-12-26 20:57:42,130][105692] Updated weights for policy 0, policy_version 769211 (0.0009) [2023-12-26 20:57:42,189][105692] Updated weights for policy 0, policy_version 769221 (0.0009) [2023-12-26 20:57:42,400][105620] Updated weights for policy 1, policy_version 769167 (0.0007) [2023-12-26 20:57:42,457][105620] Updated weights for policy 1, policy_version 769177 (0.0005) [2023-12-26 20:57:42,511][105620] Updated weights for policy 1, policy_version 769187 (0.0006) [2023-12-26 20:57:43,037][105692] Updated weights for policy 0, policy_version 769231 (0.0007) [2023-12-26 20:57:43,099][105692] Updated weights for policy 0, policy_version 769241 (0.0009) [2023-12-26 20:57:43,140][105620] Updated weights for policy 1, policy_version 769197 (0.0007) [2023-12-26 20:57:43,151][105692] Updated weights for policy 0, policy_version 769251 (0.0007) [2023-12-26 20:57:43,191][105620] Updated weights for policy 1, policy_version 769207 (0.0007) [2023-12-26 20:57:43,239][105620] Updated weights for policy 1, policy_version 769217 (0.0009) [2023-12-26 20:57:43,936][105692] Updated weights for policy 0, policy_version 769261 (0.0007) [2023-12-26 20:57:43,954][105620] Updated weights for policy 1, policy_version 769227 (0.0009) [2023-12-26 20:57:43,993][105692] Updated weights for policy 0, policy_version 769271 (0.0006) [2023-12-26 20:57:44,003][105620] Updated weights for policy 1, policy_version 769237 (0.0008) [2023-12-26 20:57:44,046][105692] Updated weights for policy 0, policy_version 769281 (0.0007) [2023-12-26 20:57:44,049][105620] Updated weights for policy 1, policy_version 769247 (0.0006) [2023-12-26 20:57:44,738][105620] Updated weights for policy 1, policy_version 769257 (0.0007) [2023-12-26 20:57:44,805][105620] Updated weights for policy 1, policy_version 769267 (0.0009) [2023-12-26 20:57:44,859][105620] Updated weights for policy 1, policy_version 769277 (0.0007) [2023-12-26 20:57:44,861][105692] Updated weights for policy 0, policy_version 769291 (0.0007) [2023-12-26 20:57:44,916][105620] Updated weights for policy 1, policy_version 769287 (0.0008) [2023-12-26 20:57:44,919][105692] Updated weights for policy 0, policy_version 769301 (0.0006) [2023-12-26 20:57:44,975][105692] Updated weights for policy 0, policy_version 769311 (0.0008) [2023-12-26 20:57:45,686][105620] Updated weights for policy 1, policy_version 769297 (0.0011) [2023-12-26 20:57:45,744][105620] Updated weights for policy 1, policy_version 769307 (0.0010) [2023-12-26 20:57:45,782][105692] Updated weights for policy 0, policy_version 769321 (0.0008) [2023-12-26 20:57:45,806][105620] Updated weights for policy 1, policy_version 769317 (0.0008) [2023-12-26 20:57:45,838][105692] Updated weights for policy 0, policy_version 769331 (0.0008) [2023-12-26 20:57:45,886][105692] Updated weights for policy 0, policy_version 769341 (0.0008) [2023-12-26 20:57:45,944][105692] Updated weights for policy 0, policy_version 769351 (0.0009) [2023-12-26 20:57:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 393953280. Throughput: 0: 9884.4, 1: 9831.8. Samples: 393921320. Policy #0 lag: (min: 14.0, avg: 34.2, max: 46.0) [2023-12-26 20:57:46,062][104569] Avg episode reward: [(0, '8813.527'), (1, '8548.519')] [2023-12-26 20:57:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000769320_196968448.pth... [2023-12-26 20:57:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000769352_196984832.pth... [2023-12-26 20:57:46,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000768200_196681728.pth [2023-12-26 20:57:46,080][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000768200_196689920.pth [2023-12-26 20:57:46,509][105620] Updated weights for policy 1, policy_version 769327 (0.0007) [2023-12-26 20:57:46,571][105620] Updated weights for policy 1, policy_version 769337 (0.0010) [2023-12-26 20:57:46,620][105692] Updated weights for policy 0, policy_version 769361 (0.0010) [2023-12-26 20:57:46,626][105620] Updated weights for policy 1, policy_version 769347 (0.0010) [2023-12-26 20:57:46,679][105692] Updated weights for policy 0, policy_version 769371 (0.0008) [2023-12-26 20:57:46,742][105692] Updated weights for policy 0, policy_version 769381 (0.0007) [2023-12-26 20:57:47,183][105620] Updated weights for policy 1, policy_version 769357 (0.0008) [2023-12-26 20:57:47,243][105620] Updated weights for policy 1, policy_version 769367 (0.0005) [2023-12-26 20:57:47,302][105692] Updated weights for policy 0, policy_version 769391 (0.0006) [2023-12-26 20:57:47,304][105620] Updated weights for policy 1, policy_version 769377 (0.0005) [2023-12-26 20:57:47,353][105692] Updated weights for policy 0, policy_version 769401 (0.0005) [2023-12-26 20:57:47,419][105692] Updated weights for policy 0, policy_version 769411 (0.0008) [2023-12-26 20:57:47,947][105620] Updated weights for policy 1, policy_version 769387 (0.0009) [2023-12-26 20:57:47,993][105692] Updated weights for policy 0, policy_version 769421 (0.0007) [2023-12-26 20:57:47,994][105620] Updated weights for policy 1, policy_version 769397 (0.0011) [2023-12-26 20:57:48,058][105620] Updated weights for policy 1, policy_version 769407 (0.0011) [2023-12-26 20:57:48,058][105692] Updated weights for policy 0, policy_version 769431 (0.0010) [2023-12-26 20:57:48,119][105692] Updated weights for policy 0, policy_version 769441 (0.0011) [2023-12-26 20:57:48,791][105620] Updated weights for policy 1, policy_version 769417 (0.0009) [2023-12-26 20:57:48,821][105692] Updated weights for policy 0, policy_version 769451 (0.0008) [2023-12-26 20:57:48,850][105620] Updated weights for policy 1, policy_version 769427 (0.0011) [2023-12-26 20:57:48,881][105692] Updated weights for policy 0, policy_version 769461 (0.0005) [2023-12-26 20:57:48,902][105620] Updated weights for policy 1, policy_version 769437 (0.0010) [2023-12-26 20:57:48,942][105692] Updated weights for policy 0, policy_version 769471 (0.0006) [2023-12-26 20:57:48,952][105620] Updated weights for policy 1, policy_version 769447 (0.0010) [2023-12-26 20:57:49,603][105620] Updated weights for policy 1, policy_version 769457 (0.0010) [2023-12-26 20:57:49,661][105620] Updated weights for policy 1, policy_version 769467 (0.0005) [2023-12-26 20:57:49,725][105620] Updated weights for policy 1, policy_version 769477 (0.0010) [2023-12-26 20:57:49,754][105692] Updated weights for policy 0, policy_version 769481 (0.0007) [2023-12-26 20:57:49,807][105692] Updated weights for policy 0, policy_version 769491 (0.0008) [2023-12-26 20:57:49,869][105692] Updated weights for policy 0, policy_version 769501 (0.0007) [2023-12-26 20:57:49,934][105692] Updated weights for policy 0, policy_version 769511 (0.0007) [2023-12-26 20:57:50,479][105620] Updated weights for policy 1, policy_version 769487 (0.0010) [2023-12-26 20:57:50,541][105620] Updated weights for policy 1, policy_version 769497 (0.0010) [2023-12-26 20:57:50,605][105620] Updated weights for policy 1, policy_version 769507 (0.0011) [2023-12-26 20:57:50,724][105692] Updated weights for policy 0, policy_version 769521 (0.0008) [2023-12-26 20:57:50,785][105692] Updated weights for policy 0, policy_version 769531 (0.0009) [2023-12-26 20:57:50,830][105692] Updated weights for policy 0, policy_version 769541 (0.0008) [2023-12-26 20:57:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.8, 300 sec: 19660.8). Total num frames: 394051584. Throughput: 0: 9957.1, 1: 9820.3. Samples: 394040448. Policy #0 lag: (min: 14.0, avg: 34.2, max: 46.0) [2023-12-26 20:57:51,063][104569] Avg episode reward: [(0, '8623.660'), (1, '8818.779')] [2023-12-26 20:57:51,350][105620] Updated weights for policy 1, policy_version 769517 (0.0009) [2023-12-26 20:57:51,424][105620] Updated weights for policy 1, policy_version 769527 (0.0009) [2023-12-26 20:57:51,485][105620] Updated weights for policy 1, policy_version 769537 (0.0007) [2023-12-26 20:57:51,667][105692] Updated weights for policy 0, policy_version 769551 (0.0007) [2023-12-26 20:57:51,724][105692] Updated weights for policy 0, policy_version 769561 (0.0009) [2023-12-26 20:57:51,777][105692] Updated weights for policy 0, policy_version 769572 (0.0008) [2023-12-26 20:57:52,072][105620] Updated weights for policy 1, policy_version 769547 (0.0006) [2023-12-26 20:57:52,135][105620] Updated weights for policy 1, policy_version 769557 (0.0006) [2023-12-26 20:57:52,211][105620] Updated weights for policy 1, policy_version 769567 (0.0006) [2023-12-26 20:57:52,508][105692] Updated weights for policy 0, policy_version 769582 (0.0007) [2023-12-26 20:57:52,572][105692] Updated weights for policy 0, policy_version 769592 (0.0006) [2023-12-26 20:57:52,634][105692] Updated weights for policy 0, policy_version 769602 (0.0011) [2023-12-26 20:57:52,869][105620] Updated weights for policy 1, policy_version 769577 (0.0012) [2023-12-26 20:57:52,932][105620] Updated weights for policy 1, policy_version 769587 (0.0011) [2023-12-26 20:57:52,996][105620] Updated weights for policy 1, policy_version 769597 (0.0011) [2023-12-26 20:57:53,059][105620] Updated weights for policy 1, policy_version 769607 (0.0011) [2023-12-26 20:57:53,333][105692] Updated weights for policy 0, policy_version 769612 (0.0010) [2023-12-26 20:57:53,393][105692] Updated weights for policy 0, policy_version 769622 (0.0008) [2023-12-26 20:57:53,450][105692] Updated weights for policy 0, policy_version 769632 (0.0008) [2023-12-26 20:57:53,778][105620] Updated weights for policy 1, policy_version 769617 (0.0010) [2023-12-26 20:57:53,826][105620] Updated weights for policy 1, policy_version 769627 (0.0010) [2023-12-26 20:57:53,875][105620] Updated weights for policy 1, policy_version 769637 (0.0010) [2023-12-26 20:57:54,224][105692] Updated weights for policy 0, policy_version 769642 (0.0008) [2023-12-26 20:57:54,276][105692] Updated weights for policy 0, policy_version 769652 (0.0008) [2023-12-26 20:57:54,332][105692] Updated weights for policy 0, policy_version 769662 (0.0008) [2023-12-26 20:57:54,389][105692] Updated weights for policy 0, policy_version 769672 (0.0008) [2023-12-26 20:57:54,644][105620] Updated weights for policy 1, policy_version 769647 (0.0010) [2023-12-26 20:57:54,699][105620] Updated weights for policy 1, policy_version 769657 (0.0010) [2023-12-26 20:57:54,751][105620] Updated weights for policy 1, policy_version 769667 (0.0010) [2023-12-26 20:57:55,063][105692] Updated weights for policy 0, policy_version 769682 (0.0008) [2023-12-26 20:57:55,119][105692] Updated weights for policy 0, policy_version 769692 (0.0008) [2023-12-26 20:57:55,176][105692] Updated weights for policy 0, policy_version 769702 (0.0008) [2023-12-26 20:57:55,487][105620] Updated weights for policy 1, policy_version 769677 (0.0010) [2023-12-26 20:57:55,545][105620] Updated weights for policy 1, policy_version 769687 (0.0011) [2023-12-26 20:57:55,600][105620] Updated weights for policy 1, policy_version 769697 (0.0010) [2023-12-26 20:57:55,975][105692] Updated weights for policy 0, policy_version 769712 (0.0008) [2023-12-26 20:57:56,039][105692] Updated weights for policy 0, policy_version 769722 (0.0008) [2023-12-26 20:57:56,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 394141696. Throughput: 0: 9894.1, 1: 9765.9. Samples: 394155032. Policy #0 lag: (min: 14.0, avg: 34.2, max: 46.0) [2023-12-26 20:57:56,063][104569] Avg episode reward: [(0, '9025.805'), (1, '8816.541')] [2023-12-26 20:57:56,103][105692] Updated weights for policy 0, policy_version 769732 (0.0009) [2023-12-26 20:57:56,338][105620] Updated weights for policy 1, policy_version 769707 (0.0010) [2023-12-26 20:57:56,400][105620] Updated weights for policy 1, policy_version 769717 (0.0010) [2023-12-26 20:57:56,464][105620] Updated weights for policy 1, policy_version 769727 (0.0010) [2023-12-26 20:57:56,855][105692] Updated weights for policy 0, policy_version 769742 (0.0007) [2023-12-26 20:57:56,914][105692] Updated weights for policy 0, policy_version 769752 (0.0008) [2023-12-26 20:57:56,972][105692] Updated weights for policy 0, policy_version 769762 (0.0010) [2023-12-26 20:57:57,201][105620] Updated weights for policy 1, policy_version 769737 (0.0010) [2023-12-26 20:57:57,262][105620] Updated weights for policy 1, policy_version 769747 (0.0010) [2023-12-26 20:57:57,318][105620] Updated weights for policy 1, policy_version 769757 (0.0007) [2023-12-26 20:57:57,375][105620] Updated weights for policy 1, policy_version 769767 (0.0008) [2023-12-26 20:57:57,654][105692] Updated weights for policy 0, policy_version 769772 (0.0008) [2023-12-26 20:57:57,711][105692] Updated weights for policy 0, policy_version 769782 (0.0005) [2023-12-26 20:57:57,760][105692] Updated weights for policy 0, policy_version 769792 (0.0006) [2023-12-26 20:57:58,058][105620] Updated weights for policy 1, policy_version 769777 (0.0009) [2023-12-26 20:57:58,112][105620] Updated weights for policy 1, policy_version 769787 (0.0008) [2023-12-26 20:57:58,178][105620] Updated weights for policy 1, policy_version 769797 (0.0008) [2023-12-26 20:57:58,298][105692] Updated weights for policy 0, policy_version 769802 (0.0010) [2023-12-26 20:57:58,373][105692] Updated weights for policy 0, policy_version 769812 (0.0008) [2023-12-26 20:57:58,440][105692] Updated weights for policy 0, policy_version 769822 (0.0009) [2023-12-26 20:57:58,507][105692] Updated weights for policy 0, policy_version 769832 (0.0009) [2023-12-26 20:57:59,085][105620] Updated weights for policy 1, policy_version 769807 (0.0009) [2023-12-26 20:57:59,148][105620] Updated weights for policy 1, policy_version 769817 (0.0008) [2023-12-26 20:57:59,209][105620] Updated weights for policy 1, policy_version 769827 (0.0008) [2023-12-26 20:57:59,288][105692] Updated weights for policy 0, policy_version 769842 (0.0009) [2023-12-26 20:57:59,357][105692] Updated weights for policy 0, policy_version 769852 (0.0008) [2023-12-26 20:57:59,424][105692] Updated weights for policy 0, policy_version 769862 (0.0007) [2023-12-26 20:57:59,997][105620] Updated weights for policy 1, policy_version 769837 (0.0008) [2023-12-26 20:58:00,055][105620] Updated weights for policy 1, policy_version 769847 (0.0009) [2023-12-26 20:58:00,082][105692] Updated weights for policy 0, policy_version 769872 (0.0006) [2023-12-26 20:58:00,108][105620] Updated weights for policy 1, policy_version 769857 (0.0007) [2023-12-26 20:58:00,134][105692] Updated weights for policy 0, policy_version 769882 (0.0007) [2023-12-26 20:58:00,190][105692] Updated weights for policy 0, policy_version 769892 (0.0006) [2023-12-26 20:58:00,855][105620] Updated weights for policy 1, policy_version 769867 (0.0006) [2023-12-26 20:58:00,914][105620] Updated weights for policy 1, policy_version 769877 (0.0006) [2023-12-26 20:58:00,954][105692] Updated weights for policy 0, policy_version 769902 (0.0008) [2023-12-26 20:58:00,980][105620] Updated weights for policy 1, policy_version 769887 (0.0009) [2023-12-26 20:58:01,013][105692] Updated weights for policy 0, policy_version 769912 (0.0008) [2023-12-26 20:58:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 394240000. Throughput: 0: 9865.3, 1: 9749.7. Samples: 394212288. Policy #0 lag: (min: 14.0, avg: 34.2, max: 46.0) [2023-12-26 20:58:01,062][104569] Avg episode reward: [(0, '9269.303'), (1, '8632.973')] [2023-12-26 20:58:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000769896_197115904.pth... [2023-12-26 20:58:01,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000768744_196820992.pth [2023-12-26 20:58:01,071][105692] Updated weights for policy 0, policy_version 769922 (0.0009) [2023-12-26 20:58:01,101][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000769928_197132288.pth... [2023-12-26 20:58:01,107][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000768776_196837376.pth [2023-12-26 20:58:01,664][105620] Updated weights for policy 1, policy_version 769897 (0.0006) [2023-12-26 20:58:01,717][105692] Updated weights for policy 0, policy_version 769932 (0.0006) [2023-12-26 20:58:01,723][105620] Updated weights for policy 1, policy_version 769907 (0.0009) [2023-12-26 20:58:01,780][105692] Updated weights for policy 0, policy_version 769942 (0.0008) [2023-12-26 20:58:01,787][105620] Updated weights for policy 1, policy_version 769917 (0.0008) [2023-12-26 20:58:01,834][105692] Updated weights for policy 0, policy_version 769952 (0.0007) [2023-12-26 20:58:01,836][105620] Updated weights for policy 1, policy_version 769927 (0.0006) [2023-12-26 20:58:02,578][105620] Updated weights for policy 1, policy_version 769937 (0.0008) [2023-12-26 20:58:02,580][105692] Updated weights for policy 0, policy_version 769962 (0.0007) [2023-12-26 20:58:02,634][105620] Updated weights for policy 1, policy_version 769947 (0.0006) [2023-12-26 20:58:02,640][105692] Updated weights for policy 0, policy_version 769972 (0.0008) [2023-12-26 20:58:02,691][105620] Updated weights for policy 1, policy_version 769957 (0.0007) [2023-12-26 20:58:02,700][105692] Updated weights for policy 0, policy_version 769982 (0.0006) [2023-12-26 20:58:02,767][105692] Updated weights for policy 0, policy_version 769992 (0.0008) [2023-12-26 20:58:03,386][105620] Updated weights for policy 1, policy_version 769967 (0.0006) [2023-12-26 20:58:03,449][105620] Updated weights for policy 1, policy_version 769977 (0.0007) [2023-12-26 20:58:03,461][105692] Updated weights for policy 0, policy_version 770002 (0.0006) [2023-12-26 20:58:03,508][105620] Updated weights for policy 1, policy_version 769987 (0.0007) [2023-12-26 20:58:03,525][105692] Updated weights for policy 0, policy_version 770012 (0.0010) [2023-12-26 20:58:03,577][105692] Updated weights for policy 0, policy_version 770022 (0.0007) [2023-12-26 20:58:04,136][105620] Updated weights for policy 1, policy_version 769997 (0.0006) [2023-12-26 20:58:04,199][105620] Updated weights for policy 1, policy_version 770007 (0.0009) [2023-12-26 20:58:04,257][105620] Updated weights for policy 1, policy_version 770017 (0.0009) [2023-12-26 20:58:04,305][105692] Updated weights for policy 0, policy_version 770032 (0.0008) [2023-12-26 20:58:04,362][105692] Updated weights for policy 0, policy_version 770042 (0.0008) [2023-12-26 20:58:04,433][105692] Updated weights for policy 0, policy_version 770052 (0.0006) [2023-12-26 20:58:04,851][105620] Updated weights for policy 1, policy_version 770027 (0.0006) [2023-12-26 20:58:04,918][105620] Updated weights for policy 1, policy_version 770037 (0.0010) [2023-12-26 20:58:04,980][105692] Updated weights for policy 0, policy_version 770062 (0.0006) [2023-12-26 20:58:04,981][105620] Updated weights for policy 1, policy_version 770047 (0.0011) [2023-12-26 20:58:05,038][105692] Updated weights for policy 0, policy_version 770072 (0.0006) [2023-12-26 20:58:05,100][105692] Updated weights for policy 0, policy_version 770082 (0.0006) [2023-12-26 20:58:05,684][105692] Updated weights for policy 0, policy_version 770092 (0.0008) [2023-12-26 20:58:05,711][105620] Updated weights for policy 1, policy_version 770057 (0.0011) [2023-12-26 20:58:05,735][105692] Updated weights for policy 0, policy_version 770102 (0.0005) [2023-12-26 20:58:05,759][105620] Updated weights for policy 1, policy_version 770067 (0.0010) [2023-12-26 20:58:05,784][105692] Updated weights for policy 0, policy_version 770112 (0.0005) [2023-12-26 20:58:05,808][105620] Updated weights for policy 1, policy_version 770077 (0.0010) [2023-12-26 20:58:05,853][105620] Updated weights for policy 1, policy_version 770087 (0.0010) [2023-12-26 20:58:06,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 394346496. Throughput: 0: 9762.1, 1: 9729.2. Samples: 394329724. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-12-26 20:58:06,063][104569] Avg episode reward: [(0, '9269.128'), (1, '8904.775')] [2023-12-26 20:58:06,433][105692] Updated weights for policy 0, policy_version 770122 (0.0006) [2023-12-26 20:58:06,498][105692] Updated weights for policy 0, policy_version 770132 (0.0010) [2023-12-26 20:58:06,563][105692] Updated weights for policy 0, policy_version 770142 (0.0011) [2023-12-26 20:58:06,626][105692] Updated weights for policy 0, policy_version 770152 (0.0011) [2023-12-26 20:58:06,673][105620] Updated weights for policy 1, policy_version 770097 (0.0008) [2023-12-26 20:58:06,742][105620] Updated weights for policy 1, policy_version 770107 (0.0008) [2023-12-26 20:58:06,809][105620] Updated weights for policy 1, policy_version 770117 (0.0008) [2023-12-26 20:58:07,385][105692] Updated weights for policy 0, policy_version 770162 (0.0011) [2023-12-26 20:58:07,447][105692] Updated weights for policy 0, policy_version 770172 (0.0010) [2023-12-26 20:58:07,510][105692] Updated weights for policy 0, policy_version 770182 (0.0011) [2023-12-26 20:58:07,558][105620] Updated weights for policy 1, policy_version 770127 (0.0008) [2023-12-26 20:58:07,617][105620] Updated weights for policy 1, policy_version 770137 (0.0008) [2023-12-26 20:58:07,675][105620] Updated weights for policy 1, policy_version 770147 (0.0007) [2023-12-26 20:58:08,247][105692] Updated weights for policy 0, policy_version 770192 (0.0010) [2023-12-26 20:58:08,304][105692] Updated weights for policy 0, policy_version 770202 (0.0010) [2023-12-26 20:58:08,369][105692] Updated weights for policy 0, policy_version 770212 (0.0011) [2023-12-26 20:58:08,456][105620] Updated weights for policy 1, policy_version 770157 (0.0008) [2023-12-26 20:58:08,522][105620] Updated weights for policy 1, policy_version 770167 (0.0008) [2023-12-26 20:58:08,589][105620] Updated weights for policy 1, policy_version 770177 (0.0008) [2023-12-26 20:58:09,116][105692] Updated weights for policy 0, policy_version 770222 (0.0011) [2023-12-26 20:58:09,173][105692] Updated weights for policy 0, policy_version 770232 (0.0011) [2023-12-26 20:58:09,236][105692] Updated weights for policy 0, policy_version 770242 (0.0011) [2023-12-26 20:58:09,351][105620] Updated weights for policy 1, policy_version 770187 (0.0008) [2023-12-26 20:58:09,421][105620] Updated weights for policy 1, policy_version 770197 (0.0008) [2023-12-26 20:58:09,481][105620] Updated weights for policy 1, policy_version 770207 (0.0008) [2023-12-26 20:58:09,893][105692] Updated weights for policy 0, policy_version 770252 (0.0010) [2023-12-26 20:58:09,953][105692] Updated weights for policy 0, policy_version 770262 (0.0011) [2023-12-26 20:58:10,008][105692] Updated weights for policy 0, policy_version 770272 (0.0010) [2023-12-26 20:58:10,174][105620] Updated weights for policy 1, policy_version 770217 (0.0008) [2023-12-26 20:58:10,243][105620] Updated weights for policy 1, policy_version 770227 (0.0009) [2023-12-26 20:58:10,304][105620] Updated weights for policy 1, policy_version 770237 (0.0010) [2023-12-26 20:58:10,358][105620] Updated weights for policy 1, policy_version 770247 (0.0009) [2023-12-26 20:58:10,715][105692] Updated weights for policy 0, policy_version 770282 (0.0006) [2023-12-26 20:58:10,770][105692] Updated weights for policy 0, policy_version 770292 (0.0009) [2023-12-26 20:58:10,821][105692] Updated weights for policy 0, policy_version 770302 (0.0009) [2023-12-26 20:58:10,869][105692] Updated weights for policy 0, policy_version 770312 (0.0009) [2023-12-26 20:58:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.9, 300 sec: 19660.8). Total num frames: 394436608. Throughput: 0: 9814.2, 1: 9615.0. Samples: 394445884. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-12-26 20:58:11,063][104569] Avg episode reward: [(0, '9356.637'), (1, '8727.390')] [2023-12-26 20:58:11,100][105620] Updated weights for policy 1, policy_version 770257 (0.0007) [2023-12-26 20:58:11,165][105620] Updated weights for policy 1, policy_version 770267 (0.0007) [2023-12-26 20:58:11,233][105620] Updated weights for policy 1, policy_version 770277 (0.0009) [2023-12-26 20:58:11,666][105692] Updated weights for policy 0, policy_version 770322 (0.0008) [2023-12-26 20:58:11,740][105692] Updated weights for policy 0, policy_version 770332 (0.0009) [2023-12-26 20:58:11,799][105692] Updated weights for policy 0, policy_version 770342 (0.0010) [2023-12-26 20:58:11,869][105620] Updated weights for policy 1, policy_version 770287 (0.0006) [2023-12-26 20:58:11,927][105620] Updated weights for policy 1, policy_version 770297 (0.0005) [2023-12-26 20:58:11,988][105620] Updated weights for policy 1, policy_version 770307 (0.0006) [2023-12-26 20:58:12,587][105692] Updated weights for policy 0, policy_version 770352 (0.0006) [2023-12-26 20:58:12,650][105692] Updated weights for policy 0, policy_version 770362 (0.0008) [2023-12-26 20:58:12,695][105692] Updated weights for policy 0, policy_version 770372 (0.0008) [2023-12-26 20:58:12,703][105620] Updated weights for policy 1, policy_version 770317 (0.0007) [2023-12-26 20:58:12,765][105620] Updated weights for policy 1, policy_version 770327 (0.0008) [2023-12-26 20:58:12,826][105620] Updated weights for policy 1, policy_version 770337 (0.0009) [2023-12-26 20:58:13,430][105692] Updated weights for policy 0, policy_version 770382 (0.0010) [2023-12-26 20:58:13,477][105692] Updated weights for policy 0, policy_version 770392 (0.0009) [2023-12-26 20:58:13,522][105692] Updated weights for policy 0, policy_version 770402 (0.0008) [2023-12-26 20:58:13,578][105620] Updated weights for policy 1, policy_version 770347 (0.0009) [2023-12-26 20:58:13,632][105620] Updated weights for policy 1, policy_version 770357 (0.0008) [2023-12-26 20:58:13,681][105620] Updated weights for policy 1, policy_version 770367 (0.0009) [2023-12-26 20:58:14,330][105620] Updated weights for policy 1, policy_version 770377 (0.0009) [2023-12-26 20:58:14,353][105692] Updated weights for policy 0, policy_version 770412 (0.0007) [2023-12-26 20:58:14,395][105620] Updated weights for policy 1, policy_version 770387 (0.0011) [2023-12-26 20:58:14,415][105692] Updated weights for policy 0, policy_version 770422 (0.0006) [2023-12-26 20:58:14,456][105620] Updated weights for policy 1, policy_version 770397 (0.0008) [2023-12-26 20:58:14,470][105692] Updated weights for policy 0, policy_version 770432 (0.0008) [2023-12-26 20:58:14,519][105620] Updated weights for policy 1, policy_version 770407 (0.0007) [2023-12-26 20:58:15,199][105692] Updated weights for policy 0, policy_version 770442 (0.0008) [2023-12-26 20:58:15,261][105692] Updated weights for policy 0, policy_version 770452 (0.0008) [2023-12-26 20:58:15,276][105620] Updated weights for policy 1, policy_version 770417 (0.0011) [2023-12-26 20:58:15,323][105692] Updated weights for policy 0, policy_version 770462 (0.0006) [2023-12-26 20:58:15,336][105620] Updated weights for policy 1, policy_version 770427 (0.0011) [2023-12-26 20:58:15,388][105692] Updated weights for policy 0, policy_version 770472 (0.0008) [2023-12-26 20:58:15,394][105620] Updated weights for policy 1, policy_version 770437 (0.0007) [2023-12-26 20:58:16,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 394526720. Throughput: 0: 9693.8, 1: 9647.4. Samples: 394501692. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-12-26 20:58:16,063][104569] Avg episode reward: [(0, '8911.738'), (1, '8729.133')] [2023-12-26 20:58:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000770472_197271552.pth... [2023-12-26 20:58:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000769352_196984832.pth [2023-12-26 20:58:16,102][105620] Updated weights for policy 1, policy_version 770447 (0.0010) [2023-12-26 20:58:16,148][105692] Updated weights for policy 0, policy_version 770482 (0.0007) [2023-12-26 20:58:16,162][105620] Updated weights for policy 1, policy_version 770457 (0.0011) [2023-12-26 20:58:16,207][105692] Updated weights for policy 0, policy_version 770492 (0.0011) [2023-12-26 20:58:16,214][105620] Updated weights for policy 1, policy_version 770467 (0.0011) [2023-12-26 20:58:16,240][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000770472_197263360.pth... [2023-12-26 20:58:16,243][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000769320_196968448.pth [2023-12-26 20:58:16,258][105692] Updated weights for policy 0, policy_version 770502 (0.0010) [2023-12-26 20:58:16,857][105692] Updated weights for policy 0, policy_version 770512 (0.0006) [2023-12-26 20:58:16,921][105692] Updated weights for policy 0, policy_version 770522 (0.0005) [2023-12-26 20:58:16,957][105620] Updated weights for policy 1, policy_version 770477 (0.0011) [2023-12-26 20:58:16,980][105692] Updated weights for policy 0, policy_version 770532 (0.0005) [2023-12-26 20:58:17,010][105620] Updated weights for policy 1, policy_version 770487 (0.0011) [2023-12-26 20:58:17,062][105620] Updated weights for policy 1, policy_version 770497 (0.0011) [2023-12-26 20:58:17,597][105692] Updated weights for policy 0, policy_version 770542 (0.0005) [2023-12-26 20:58:17,644][105692] Updated weights for policy 0, policy_version 770552 (0.0005) [2023-12-26 20:58:17,688][105692] Updated weights for policy 0, policy_version 770562 (0.0005) [2023-12-26 20:58:17,825][105620] Updated weights for policy 1, policy_version 770507 (0.0010) [2023-12-26 20:58:17,887][105620] Updated weights for policy 1, policy_version 770517 (0.0011) [2023-12-26 20:58:17,943][105620] Updated weights for policy 1, policy_version 770527 (0.0011) [2023-12-26 20:58:18,337][105692] Updated weights for policy 0, policy_version 770572 (0.0006) [2023-12-26 20:58:18,394][105692] Updated weights for policy 0, policy_version 770582 (0.0008) [2023-12-26 20:58:18,453][105692] Updated weights for policy 0, policy_version 770592 (0.0009) [2023-12-26 20:58:18,633][105620] Updated weights for policy 1, policy_version 770537 (0.0010) [2023-12-26 20:58:18,695][105620] Updated weights for policy 1, policy_version 770547 (0.0007) [2023-12-26 20:58:18,764][105620] Updated weights for policy 1, policy_version 770557 (0.0010) [2023-12-26 20:58:18,840][105620] Updated weights for policy 1, policy_version 770567 (0.0011) [2023-12-26 20:58:19,129][105692] Updated weights for policy 0, policy_version 770602 (0.0010) [2023-12-26 20:58:19,185][105692] Updated weights for policy 0, policy_version 770612 (0.0008) [2023-12-26 20:58:19,243][105692] Updated weights for policy 0, policy_version 770622 (0.0008) [2023-12-26 20:58:19,306][105692] Updated weights for policy 0, policy_version 770632 (0.0009) [2023-12-26 20:58:19,520][105620] Updated weights for policy 1, policy_version 770577 (0.0009) [2023-12-26 20:58:19,575][105620] Updated weights for policy 1, policy_version 770587 (0.0006) [2023-12-26 20:58:19,638][105620] Updated weights for policy 1, policy_version 770597 (0.0006) [2023-12-26 20:58:20,158][105692] Updated weights for policy 0, policy_version 770642 (0.0008) [2023-12-26 20:58:20,209][105692] Updated weights for policy 0, policy_version 770652 (0.0008) [2023-12-26 20:58:20,266][105692] Updated weights for policy 0, policy_version 770662 (0.0008) [2023-12-26 20:58:20,304][105620] Updated weights for policy 1, policy_version 770607 (0.0007) [2023-12-26 20:58:20,353][105620] Updated weights for policy 1, policy_version 770617 (0.0008) [2023-12-26 20:58:20,401][105620] Updated weights for policy 1, policy_version 770627 (0.0008) [2023-12-26 20:58:21,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 394625024. Throughput: 0: 9693.1, 1: 9689.0. Samples: 394620724. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-12-26 20:58:21,063][104569] Avg episode reward: [(0, '8907.845'), (1, '8900.239')] [2023-12-26 20:58:21,113][105692] Updated weights for policy 0, policy_version 770672 (0.0008) [2023-12-26 20:58:21,158][105620] Updated weights for policy 1, policy_version 770637 (0.0008) [2023-12-26 20:58:21,177][105692] Updated weights for policy 0, policy_version 770682 (0.0007) [2023-12-26 20:58:21,216][105620] Updated weights for policy 1, policy_version 770647 (0.0011) [2023-12-26 20:58:21,237][105692] Updated weights for policy 0, policy_version 770692 (0.0007) [2023-12-26 20:58:21,282][105620] Updated weights for policy 1, policy_version 770657 (0.0010) [2023-12-26 20:58:21,959][105692] Updated weights for policy 0, policy_version 770702 (0.0009) [2023-12-26 20:58:22,024][105692] Updated weights for policy 0, policy_version 770712 (0.0009) [2023-12-26 20:58:22,051][105620] Updated weights for policy 1, policy_version 770667 (0.0008) [2023-12-26 20:58:22,091][105692] Updated weights for policy 0, policy_version 770722 (0.0007) [2023-12-26 20:58:22,106][105620] Updated weights for policy 1, policy_version 770677 (0.0006) [2023-12-26 20:58:22,170][105620] Updated weights for policy 1, policy_version 770687 (0.0011) [2023-12-26 20:58:22,864][105620] Updated weights for policy 1, policy_version 770697 (0.0007) [2023-12-26 20:58:22,887][105692] Updated weights for policy 0, policy_version 770732 (0.0008) [2023-12-26 20:58:22,922][105620] Updated weights for policy 1, policy_version 770707 (0.0011) [2023-12-26 20:58:22,940][105692] Updated weights for policy 0, policy_version 770742 (0.0005) [2023-12-26 20:58:22,981][105620] Updated weights for policy 1, policy_version 770717 (0.0011) [2023-12-26 20:58:22,989][105692] Updated weights for policy 0, policy_version 770752 (0.0007) [2023-12-26 20:58:23,031][105620] Updated weights for policy 1, policy_version 770727 (0.0011) [2023-12-26 20:58:23,717][105692] Updated weights for policy 0, policy_version 770762 (0.0007) [2023-12-26 20:58:23,776][105692] Updated weights for policy 0, policy_version 770772 (0.0007) [2023-12-26 20:58:23,776][105620] Updated weights for policy 1, policy_version 770737 (0.0008) [2023-12-26 20:58:23,781][105586] KL-divergence is very high: 211.5614 [2023-12-26 20:58:23,829][105586] KL-divergence is very high: 365.2858 [2023-12-26 20:58:23,829][105692] Updated weights for policy 0, policy_version 770782 (0.0009) [2023-12-26 20:58:23,835][105620] Updated weights for policy 1, policy_version 770747 (0.0008) [2023-12-26 20:58:23,878][105586] KL-divergence is very high: 391.8428 [2023-12-26 20:58:23,881][105692] Updated weights for policy 0, policy_version 770792 (0.0006) [2023-12-26 20:58:23,899][105620] Updated weights for policy 1, policy_version 770757 (0.0009) [2023-12-26 20:58:24,490][105620] Updated weights for policy 1, policy_version 770767 (0.0006) [2023-12-26 20:58:24,544][105620] Updated weights for policy 1, policy_version 770777 (0.0006) [2023-12-26 20:58:24,597][105620] Updated weights for policy 1, policy_version 770787 (0.0006) [2023-12-26 20:58:24,606][105692] Updated weights for policy 0, policy_version 770802 (0.0008) [2023-12-26 20:58:24,665][105692] Updated weights for policy 0, policy_version 770812 (0.0007) [2023-12-26 20:58:24,721][105692] Updated weights for policy 0, policy_version 770822 (0.0009) [2023-12-26 20:58:25,227][105620] Updated weights for policy 1, policy_version 770797 (0.0008) [2023-12-26 20:58:25,273][105620] Updated weights for policy 1, policy_version 770807 (0.0005) [2023-12-26 20:58:25,316][105620] Updated weights for policy 1, policy_version 770817 (0.0005) [2023-12-26 20:58:25,493][105692] Updated weights for policy 0, policy_version 770832 (0.0010) [2023-12-26 20:58:25,547][105692] Updated weights for policy 0, policy_version 770842 (0.0010) [2023-12-26 20:58:25,608][105692] Updated weights for policy 0, policy_version 770852 (0.0010) [2023-12-26 20:58:25,992][105620] Updated weights for policy 1, policy_version 770827 (0.0007) [2023-12-26 20:58:26,043][105620] Updated weights for policy 1, policy_version 770837 (0.0010) [2023-12-26 20:58:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19633.0). Total num frames: 394723328. Throughput: 0: 9587.2, 1: 9751.6. Samples: 394735752. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-12-26 20:58:26,063][104569] Avg episode reward: [(0, '8988.031'), (1, '8894.402')] [2023-12-26 20:58:26,097][105620] Updated weights for policy 1, policy_version 770847 (0.0010) [2023-12-26 20:58:26,279][105692] Updated weights for policy 0, policy_version 770862 (0.0010) [2023-12-26 20:58:26,333][105692] Updated weights for policy 0, policy_version 770872 (0.0010) [2023-12-26 20:58:26,392][105692] Updated weights for policy 0, policy_version 770882 (0.0010) [2023-12-26 20:58:26,775][105620] Updated weights for policy 1, policy_version 770857 (0.0010) [2023-12-26 20:58:26,826][105620] Updated weights for policy 1, policy_version 770867 (0.0005) [2023-12-26 20:58:26,869][105620] Updated weights for policy 1, policy_version 770877 (0.0005) [2023-12-26 20:58:26,929][105620] Updated weights for policy 1, policy_version 770887 (0.0005) [2023-12-26 20:58:27,144][105692] Updated weights for policy 0, policy_version 770892 (0.0010) [2023-12-26 20:58:27,196][105692] Updated weights for policy 0, policy_version 770902 (0.0010) [2023-12-26 20:58:27,247][105692] Updated weights for policy 0, policy_version 770912 (0.0010) [2023-12-26 20:58:27,490][105620] Updated weights for policy 1, policy_version 770897 (0.0005) [2023-12-26 20:58:27,538][105620] Updated weights for policy 1, policy_version 770907 (0.0005) [2023-12-26 20:58:27,585][105620] Updated weights for policy 1, policy_version 770917 (0.0009) [2023-12-26 20:58:27,997][105692] Updated weights for policy 0, policy_version 770922 (0.0010) [2023-12-26 20:58:28,048][105692] Updated weights for policy 0, policy_version 770932 (0.0010) [2023-12-26 20:58:28,106][105692] Updated weights for policy 0, policy_version 770942 (0.0010) [2023-12-26 20:58:28,114][105620] Updated weights for policy 1, policy_version 770927 (0.0009) [2023-12-26 20:58:28,160][105692] Updated weights for policy 0, policy_version 770952 (0.0010) [2023-12-26 20:58:28,172][105620] Updated weights for policy 1, policy_version 770937 (0.0010) [2023-12-26 20:58:28,236][105620] Updated weights for policy 1, policy_version 770947 (0.0010) [2023-12-26 20:58:28,851][105692] Updated weights for policy 0, policy_version 770962 (0.0010) [2023-12-26 20:58:28,906][105692] Updated weights for policy 0, policy_version 770972 (0.0010) [2023-12-26 20:58:28,957][105692] Updated weights for policy 0, policy_version 770982 (0.0010) [2023-12-26 20:58:28,975][105620] Updated weights for policy 1, policy_version 770957 (0.0010) [2023-12-26 20:58:29,031][105620] Updated weights for policy 1, policy_version 770967 (0.0010) [2023-12-26 20:58:29,081][105620] Updated weights for policy 1, policy_version 770977 (0.0010) [2023-12-26 20:58:29,668][105692] Updated weights for policy 0, policy_version 770992 (0.0006) [2023-12-26 20:58:29,729][105692] Updated weights for policy 0, policy_version 771002 (0.0006) [2023-12-26 20:58:29,779][105620] Updated weights for policy 1, policy_version 770987 (0.0009) [2023-12-26 20:58:29,794][105692] Updated weights for policy 0, policy_version 771012 (0.0006) [2023-12-26 20:58:29,840][105620] Updated weights for policy 1, policy_version 770997 (0.0008) [2023-12-26 20:58:29,895][105620] Updated weights for policy 1, policy_version 771007 (0.0010) [2023-12-26 20:58:30,478][105692] Updated weights for policy 0, policy_version 771022 (0.0006) [2023-12-26 20:58:30,529][105692] Updated weights for policy 0, policy_version 771032 (0.0005) [2023-12-26 20:58:30,558][105620] Updated weights for policy 1, policy_version 771017 (0.0009) [2023-12-26 20:58:30,575][105692] Updated weights for policy 0, policy_version 771042 (0.0005) [2023-12-26 20:58:30,615][105620] Updated weights for policy 1, policy_version 771027 (0.0005) [2023-12-26 20:58:30,676][105620] Updated weights for policy 1, policy_version 771037 (0.0005) [2023-12-26 20:58:30,728][105620] Updated weights for policy 1, policy_version 771047 (0.0005) [2023-12-26 20:58:31,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 394829824. Throughput: 0: 9662.6, 1: 9831.2. Samples: 394798540. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-12-26 20:58:31,062][104569] Avg episode reward: [(0, '9081.281'), (1, '9078.466')] [2023-12-26 20:58:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000771048_197419008.pth... [2023-12-26 20:58:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000771048_197410816.pth... [2023-12-26 20:58:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000769928_197132288.pth [2023-12-26 20:58:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000769896_197115904.pth [2023-12-26 20:58:31,160][105692] Updated weights for policy 0, policy_version 771052 (0.0008) [2023-12-26 20:58:31,226][105692] Updated weights for policy 0, policy_version 771062 (0.0009) [2023-12-26 20:58:31,276][105620] Updated weights for policy 1, policy_version 771057 (0.0008) [2023-12-26 20:58:31,290][105692] Updated weights for policy 0, policy_version 771072 (0.0008) [2023-12-26 20:58:31,332][105620] Updated weights for policy 1, policy_version 771067 (0.0007) [2023-12-26 20:58:31,394][105620] Updated weights for policy 1, policy_version 771077 (0.0009) [2023-12-26 20:58:32,042][105692] Updated weights for policy 0, policy_version 771082 (0.0007) [2023-12-26 20:58:32,108][105692] Updated weights for policy 0, policy_version 771092 (0.0007) [2023-12-26 20:58:32,114][105620] Updated weights for policy 1, policy_version 771087 (0.0008) [2023-12-26 20:58:32,167][105692] Updated weights for policy 0, policy_version 771102 (0.0006) [2023-12-26 20:58:32,170][105620] Updated weights for policy 1, policy_version 771097 (0.0008) [2023-12-26 20:58:32,223][105620] Updated weights for policy 1, policy_version 771107 (0.0008) [2023-12-26 20:58:32,226][105692] Updated weights for policy 0, policy_version 771112 (0.0009) [2023-12-26 20:58:32,881][105692] Updated weights for policy 0, policy_version 771122 (0.0005) [2023-12-26 20:58:32,939][105692] Updated weights for policy 0, policy_version 771132 (0.0006) [2023-12-26 20:58:32,963][105620] Updated weights for policy 1, policy_version 771117 (0.0007) [2023-12-26 20:58:32,987][105692] Updated weights for policy 0, policy_version 771142 (0.0007) [2023-12-26 20:58:33,023][105620] Updated weights for policy 1, policy_version 771127 (0.0007) [2023-12-26 20:58:33,072][105620] Updated weights for policy 1, policy_version 771137 (0.0005) [2023-12-26 20:58:33,700][105692] Updated weights for policy 0, policy_version 771152 (0.0009) [2023-12-26 20:58:33,752][105692] Updated weights for policy 0, policy_version 771162 (0.0008) [2023-12-26 20:58:33,758][105620] Updated weights for policy 1, policy_version 771147 (0.0006) [2023-12-26 20:58:33,805][105692] Updated weights for policy 0, policy_version 771172 (0.0007) [2023-12-26 20:58:33,811][105620] Updated weights for policy 1, policy_version 771157 (0.0007) [2023-12-26 20:58:33,863][105620] Updated weights for policy 1, policy_version 771167 (0.0010) [2023-12-26 20:58:34,537][105620] Updated weights for policy 1, policy_version 771177 (0.0009) [2023-12-26 20:58:34,588][105692] Updated weights for policy 0, policy_version 771182 (0.0006) [2023-12-26 20:58:34,597][105620] Updated weights for policy 1, policy_version 771187 (0.0010) [2023-12-26 20:58:34,650][105692] Updated weights for policy 0, policy_version 771192 (0.0008) [2023-12-26 20:58:34,660][105620] Updated weights for policy 1, policy_version 771197 (0.0010) [2023-12-26 20:58:34,710][105692] Updated weights for policy 0, policy_version 771202 (0.0005) [2023-12-26 20:58:34,716][105620] Updated weights for policy 1, policy_version 771207 (0.0010) [2023-12-26 20:58:35,422][105692] Updated weights for policy 0, policy_version 771212 (0.0007) [2023-12-26 20:58:35,458][105620] Updated weights for policy 1, policy_version 771217 (0.0010) [2023-12-26 20:58:35,473][105692] Updated weights for policy 0, policy_version 771222 (0.0008) [2023-12-26 20:58:35,518][105620] Updated weights for policy 1, policy_version 771227 (0.0010) [2023-12-26 20:58:35,527][105692] Updated weights for policy 0, policy_version 771232 (0.0007) [2023-12-26 20:58:35,578][105620] Updated weights for policy 1, policy_version 771237 (0.0010) [2023-12-26 20:58:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.2, 300 sec: 19660.8). Total num frames: 394928128. Throughput: 0: 9685.2, 1: 9832.1. Samples: 394918728. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-12-26 20:58:36,063][104569] Avg episode reward: [(0, '8991.425'), (1, '9262.235')] [2023-12-26 20:58:36,256][105620] Updated weights for policy 1, policy_version 771247 (0.0011) [2023-12-26 20:58:36,307][105620] Updated weights for policy 1, policy_version 771257 (0.0010) [2023-12-26 20:58:36,333][105692] Updated weights for policy 0, policy_version 771242 (0.0007) [2023-12-26 20:58:36,359][105620] Updated weights for policy 1, policy_version 771267 (0.0010) [2023-12-26 20:58:36,396][105692] Updated weights for policy 0, policy_version 771252 (0.0011) [2023-12-26 20:58:36,453][105692] Updated weights for policy 0, policy_version 771262 (0.0010) [2023-12-26 20:58:36,513][105692] Updated weights for policy 0, policy_version 771272 (0.0011) [2023-12-26 20:58:37,144][105620] Updated weights for policy 1, policy_version 771277 (0.0010) [2023-12-26 20:58:37,202][105692] Updated weights for policy 0, policy_version 771282 (0.0006) [2023-12-26 20:58:37,203][105620] Updated weights for policy 1, policy_version 771287 (0.0010) [2023-12-26 20:58:37,258][105692] Updated weights for policy 0, policy_version 771292 (0.0006) [2023-12-26 20:58:37,267][105620] Updated weights for policy 1, policy_version 771297 (0.0010) [2023-12-26 20:58:37,314][105692] Updated weights for policy 0, policy_version 771302 (0.0005) [2023-12-26 20:58:37,995][105620] Updated weights for policy 1, policy_version 771307 (0.0010) [2023-12-26 20:58:38,030][105692] Updated weights for policy 0, policy_version 771312 (0.0005) [2023-12-26 20:58:38,051][105620] Updated weights for policy 1, policy_version 771317 (0.0008) [2023-12-26 20:58:38,096][105692] Updated weights for policy 0, policy_version 771322 (0.0007) [2023-12-26 20:58:38,104][105620] Updated weights for policy 1, policy_version 771327 (0.0008) [2023-12-26 20:58:38,151][105692] Updated weights for policy 0, policy_version 771332 (0.0005) [2023-12-26 20:58:38,826][105692] Updated weights for policy 0, policy_version 771342 (0.0009) [2023-12-26 20:58:38,866][105620] Updated weights for policy 1, policy_version 771337 (0.0009) [2023-12-26 20:58:38,876][105692] Updated weights for policy 0, policy_version 771352 (0.0008) [2023-12-26 20:58:38,921][105620] Updated weights for policy 1, policy_version 771347 (0.0007) [2023-12-26 20:58:38,937][105692] Updated weights for policy 0, policy_version 771362 (0.0009) [2023-12-26 20:58:38,986][105620] Updated weights for policy 1, policy_version 771357 (0.0006) [2023-12-26 20:58:39,034][105620] Updated weights for policy 1, policy_version 771367 (0.0009) [2023-12-26 20:58:39,601][105692] Updated weights for policy 0, policy_version 771372 (0.0007) [2023-12-26 20:58:39,664][105692] Updated weights for policy 0, policy_version 771382 (0.0005) [2023-12-26 20:58:39,733][105692] Updated weights for policy 0, policy_version 771392 (0.0007) [2023-12-26 20:58:39,790][105620] Updated weights for policy 1, policy_version 771377 (0.0006) [2023-12-26 20:58:39,854][105620] Updated weights for policy 1, policy_version 771387 (0.0008) [2023-12-26 20:58:39,921][105620] Updated weights for policy 1, policy_version 771397 (0.0009) [2023-12-26 20:58:40,375][105692] Updated weights for policy 0, policy_version 771402 (0.0009) [2023-12-26 20:58:40,443][105692] Updated weights for policy 0, policy_version 771412 (0.0009) [2023-12-26 20:58:40,505][105692] Updated weights for policy 0, policy_version 771422 (0.0009) [2023-12-26 20:58:40,554][105692] Updated weights for policy 0, policy_version 771432 (0.0009) [2023-12-26 20:58:40,622][105620] Updated weights for policy 1, policy_version 771407 (0.0008) [2023-12-26 20:58:40,673][105620] Updated weights for policy 1, policy_version 771417 (0.0009) [2023-12-26 20:58:40,726][105620] Updated weights for policy 1, policy_version 771427 (0.0006) [2023-12-26 20:58:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 395026432. Throughput: 0: 9738.0, 1: 9791.7. Samples: 395033864. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-12-26 20:58:41,062][104569] Avg episode reward: [(0, '8061.767'), (1, '9169.362')] [2023-12-26 20:58:41,278][105692] Updated weights for policy 0, policy_version 771442 (0.0006) [2023-12-26 20:58:41,332][105692] Updated weights for policy 0, policy_version 771452 (0.0006) [2023-12-26 20:58:41,405][105692] Updated weights for policy 0, policy_version 771462 (0.0007) [2023-12-26 20:58:41,496][105620] Updated weights for policy 1, policy_version 771437 (0.0008) [2023-12-26 20:58:41,558][105620] Updated weights for policy 1, policy_version 771447 (0.0010) [2023-12-26 20:58:41,621][105620] Updated weights for policy 1, policy_version 771457 (0.0010) [2023-12-26 20:58:42,061][105692] Updated weights for policy 0, policy_version 771472 (0.0008) [2023-12-26 20:58:42,126][105692] Updated weights for policy 0, policy_version 771482 (0.0007) [2023-12-26 20:58:42,186][105692] Updated weights for policy 0, policy_version 771492 (0.0006) [2023-12-26 20:58:42,412][105620] Updated weights for policy 1, policy_version 771467 (0.0008) [2023-12-26 20:58:42,466][105620] Updated weights for policy 1, policy_version 771477 (0.0009) [2023-12-26 20:58:42,524][105620] Updated weights for policy 1, policy_version 771488 (0.0010) [2023-12-26 20:58:42,890][105692] Updated weights for policy 0, policy_version 771502 (0.0010) [2023-12-26 20:58:42,941][105692] Updated weights for policy 0, policy_version 771512 (0.0010) [2023-12-26 20:58:43,001][105692] Updated weights for policy 0, policy_version 771522 (0.0009) [2023-12-26 20:58:43,309][105620] Updated weights for policy 1, policy_version 771498 (0.0007) [2023-12-26 20:58:43,365][105620] Updated weights for policy 1, policy_version 771508 (0.0005) [2023-12-26 20:58:43,422][105620] Updated weights for policy 1, policy_version 771518 (0.0009) [2023-12-26 20:58:43,478][105620] Updated weights for policy 1, policy_version 771528 (0.0009) [2023-12-26 20:58:43,629][105692] Updated weights for policy 0, policy_version 771532 (0.0008) [2023-12-26 20:58:43,676][105692] Updated weights for policy 0, policy_version 771542 (0.0005) [2023-12-26 20:58:43,725][105692] Updated weights for policy 0, policy_version 771552 (0.0005) [2023-12-26 20:58:44,056][105620] Updated weights for policy 1, policy_version 771538 (0.0007) [2023-12-26 20:58:44,116][105620] Updated weights for policy 1, policy_version 771548 (0.0008) [2023-12-26 20:58:44,166][105620] Updated weights for policy 1, policy_version 771558 (0.0010) [2023-12-26 20:58:44,407][105692] Updated weights for policy 0, policy_version 771562 (0.0008) [2023-12-26 20:58:44,458][105692] Updated weights for policy 0, policy_version 771572 (0.0010) [2023-12-26 20:58:44,505][105692] Updated weights for policy 0, policy_version 771582 (0.0008) [2023-12-26 20:58:44,551][105692] Updated weights for policy 0, policy_version 771592 (0.0005) [2023-12-26 20:58:44,920][105620] Updated weights for policy 1, policy_version 771568 (0.0007) [2023-12-26 20:58:44,983][105620] Updated weights for policy 1, policy_version 771578 (0.0008) [2023-12-26 20:58:45,039][105620] Updated weights for policy 1, policy_version 771588 (0.0011) [2023-12-26 20:58:45,192][105692] Updated weights for policy 0, policy_version 771602 (0.0010) [2023-12-26 20:58:45,251][105692] Updated weights for policy 0, policy_version 771612 (0.0010) [2023-12-26 20:58:45,320][105692] Updated weights for policy 0, policy_version 771622 (0.0011) [2023-12-26 20:58:45,782][105620] Updated weights for policy 1, policy_version 771598 (0.0011) [2023-12-26 20:58:45,830][105620] Updated weights for policy 1, policy_version 771608 (0.0010) [2023-12-26 20:58:45,881][105620] Updated weights for policy 1, policy_version 771618 (0.0010) [2023-12-26 20:58:46,041][105692] Updated weights for policy 0, policy_version 771632 (0.0010) [2023-12-26 20:58:46,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 395124736. Throughput: 0: 9754.5, 1: 9825.5. Samples: 395093388. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-12-26 20:58:46,062][104569] Avg episode reward: [(0, '7286.705'), (1, '9175.994')] [2023-12-26 20:58:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000771624_197558272.pth... [2023-12-26 20:58:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000770472_197263360.pth [2023-12-26 20:58:46,093][105692] Updated weights for policy 0, policy_version 771642 (0.0010) [2023-12-26 20:58:46,146][105692] Updated weights for policy 0, policy_version 771652 (0.0008) [2023-12-26 20:58:46,170][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000771656_197574656.pth... [2023-12-26 20:58:46,174][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000770472_197271552.pth [2023-12-26 20:58:46,587][105620] Updated weights for policy 1, policy_version 771628 (0.0008) [2023-12-26 20:58:46,641][105620] Updated weights for policy 1, policy_version 771638 (0.0005) [2023-12-26 20:58:46,697][105620] Updated weights for policy 1, policy_version 771648 (0.0008) [2023-12-26 20:58:46,818][105692] Updated weights for policy 0, policy_version 771662 (0.0007) [2023-12-26 20:58:46,873][105692] Updated weights for policy 0, policy_version 771672 (0.0005) [2023-12-26 20:58:46,919][105692] Updated weights for policy 0, policy_version 771682 (0.0005) [2023-12-26 20:58:47,280][105620] Updated weights for policy 1, policy_version 771658 (0.0008) [2023-12-26 20:58:47,345][105620] Updated weights for policy 1, policy_version 771668 (0.0011) [2023-12-26 20:58:47,407][105620] Updated weights for policy 1, policy_version 771678 (0.0010) [2023-12-26 20:58:47,468][105620] Updated weights for policy 1, policy_version 771688 (0.0010) [2023-12-26 20:58:47,587][105692] Updated weights for policy 0, policy_version 771692 (0.0008) [2023-12-26 20:58:47,646][105692] Updated weights for policy 0, policy_version 771702 (0.0005) [2023-12-26 20:58:47,718][105692] Updated weights for policy 0, policy_version 771712 (0.0005) [2023-12-26 20:58:48,174][105620] Updated weights for policy 1, policy_version 771698 (0.0011) [2023-12-26 20:58:48,239][105620] Updated weights for policy 1, policy_version 771708 (0.0011) [2023-12-26 20:58:48,288][105692] Updated weights for policy 0, policy_version 771722 (0.0005) [2023-12-26 20:58:48,305][105620] Updated weights for policy 1, policy_version 771718 (0.0011) [2023-12-26 20:58:48,350][105692] Updated weights for policy 0, policy_version 771732 (0.0007) [2023-12-26 20:58:48,412][105692] Updated weights for policy 0, policy_version 771742 (0.0009) [2023-12-26 20:58:48,468][105692] Updated weights for policy 0, policy_version 771752 (0.0010) [2023-12-26 20:58:49,037][105620] Updated weights for policy 1, policy_version 771728 (0.0011) [2023-12-26 20:58:49,096][105620] Updated weights for policy 1, policy_version 771738 (0.0010) [2023-12-26 20:58:49,144][105620] Updated weights for policy 1, policy_version 771748 (0.0010) [2023-12-26 20:58:49,223][105692] Updated weights for policy 0, policy_version 771762 (0.0010) [2023-12-26 20:58:49,290][105692] Updated weights for policy 0, policy_version 771772 (0.0009) [2023-12-26 20:58:49,350][105692] Updated weights for policy 0, policy_version 771782 (0.0008) [2023-12-26 20:58:49,968][105620] Updated weights for policy 1, policy_version 771758 (0.0010) [2023-12-26 20:58:50,016][105620] Updated weights for policy 1, policy_version 771768 (0.0009) [2023-12-26 20:58:50,072][105620] Updated weights for policy 1, policy_version 771778 (0.0008) [2023-12-26 20:58:50,083][105692] Updated weights for policy 0, policy_version 771792 (0.0007) [2023-12-26 20:58:50,131][105692] Updated weights for policy 0, policy_version 771802 (0.0008) [2023-12-26 20:58:50,190][105692] Updated weights for policy 0, policy_version 771812 (0.0009) [2023-12-26 20:58:50,842][105620] Updated weights for policy 1, policy_version 771788 (0.0009) [2023-12-26 20:58:50,902][105620] Updated weights for policy 1, policy_version 771798 (0.0008) [2023-12-26 20:58:50,946][105586] KL-divergence is very high: 134.5373 [2023-12-26 20:58:50,964][105620] Updated weights for policy 1, policy_version 771808 (0.0008) [2023-12-26 20:58:50,976][105692] Updated weights for policy 0, policy_version 771822 (0.0010) [2023-12-26 20:58:50,999][105586] KL-divergence is very high: 127.6953 [2023-12-26 20:58:51,036][105692] Updated weights for policy 0, policy_version 771832 (0.0008) [2023-12-26 20:58:51,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 395223040. Throughput: 0: 9824.1, 1: 9811.9. Samples: 395213344. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-12-26 20:58:51,062][104569] Avg episode reward: [(0, '8223.097'), (1, '8996.607')] [2023-12-26 20:58:51,096][105692] Updated weights for policy 0, policy_version 771842 (0.0008) [2023-12-26 20:58:51,690][105620] Updated weights for policy 1, policy_version 771818 (0.0010) [2023-12-26 20:58:51,759][105620] Updated weights for policy 1, policy_version 771828 (0.0010) [2023-12-26 20:58:51,818][105620] Updated weights for policy 1, policy_version 771838 (0.0010) [2023-12-26 20:58:51,855][105692] Updated weights for policy 0, policy_version 771852 (0.0008) [2023-12-26 20:58:51,876][105620] Updated weights for policy 1, policy_version 771848 (0.0009) [2023-12-26 20:58:51,909][105692] Updated weights for policy 0, policy_version 771862 (0.0008) [2023-12-26 20:58:51,972][105692] Updated weights for policy 0, policy_version 771872 (0.0008) [2023-12-26 20:58:52,536][105620] Updated weights for policy 1, policy_version 771858 (0.0010) [2023-12-26 20:58:52,601][105620] Updated weights for policy 1, policy_version 771868 (0.0010) [2023-12-26 20:58:52,670][105620] Updated weights for policy 1, policy_version 771878 (0.0010) [2023-12-26 20:58:52,707][105692] Updated weights for policy 0, policy_version 771882 (0.0007) [2023-12-26 20:58:52,768][105692] Updated weights for policy 0, policy_version 771892 (0.0006) [2023-12-26 20:58:52,837][105692] Updated weights for policy 0, policy_version 771902 (0.0006) [2023-12-26 20:58:52,896][105692] Updated weights for policy 0, policy_version 771912 (0.0006) [2023-12-26 20:58:53,399][105620] Updated weights for policy 1, policy_version 771888 (0.0010) [2023-12-26 20:58:53,452][105692] Updated weights for policy 0, policy_version 771922 (0.0006) [2023-12-26 20:58:53,453][105620] Updated weights for policy 1, policy_version 771898 (0.0010) [2023-12-26 20:58:53,508][105620] Updated weights for policy 1, policy_version 771908 (0.0010) [2023-12-26 20:58:53,517][105692] Updated weights for policy 0, policy_version 771932 (0.0006) [2023-12-26 20:58:53,577][105692] Updated weights for policy 0, policy_version 771942 (0.0008) [2023-12-26 20:58:54,089][105620] Updated weights for policy 1, policy_version 771918 (0.0007) [2023-12-26 20:58:54,140][105620] Updated weights for policy 1, policy_version 771928 (0.0007) [2023-12-26 20:58:54,182][105620] Updated weights for policy 1, policy_version 771938 (0.0005) [2023-12-26 20:58:54,399][105692] Updated weights for policy 0, policy_version 771952 (0.0010) [2023-12-26 20:58:54,453][105692] Updated weights for policy 0, policy_version 771962 (0.0010) [2023-12-26 20:58:54,506][105692] Updated weights for policy 0, policy_version 771972 (0.0010) [2023-12-26 20:58:54,786][105620] Updated weights for policy 1, policy_version 771948 (0.0005) [2023-12-26 20:58:54,857][105620] Updated weights for policy 1, policy_version 771958 (0.0006) [2023-12-26 20:58:54,923][105620] Updated weights for policy 1, policy_version 771968 (0.0005) [2023-12-26 20:58:55,279][105692] Updated weights for policy 0, policy_version 771982 (0.0009) [2023-12-26 20:58:55,337][105692] Updated weights for policy 0, policy_version 771993 (0.0010) [2023-12-26 20:58:55,389][105692] Updated weights for policy 0, policy_version 772004 (0.0009) [2023-12-26 20:58:55,500][105620] Updated weights for policy 1, policy_version 771978 (0.0008) [2023-12-26 20:58:55,561][105620] Updated weights for policy 1, policy_version 771988 (0.0009) [2023-12-26 20:58:55,616][105620] Updated weights for policy 1, policy_version 771998 (0.0009) [2023-12-26 20:58:55,675][105620] Updated weights for policy 1, policy_version 772008 (0.0009) [2023-12-26 20:58:56,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 395321344. Throughput: 0: 9723.8, 1: 9951.3. Samples: 395331268. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-12-26 20:58:56,063][104569] Avg episode reward: [(0, '6760.088'), (1, '8625.905')] [2023-12-26 20:58:56,215][105692] Updated weights for policy 0, policy_version 772014 (0.0010) [2023-12-26 20:58:56,265][105692] Updated weights for policy 0, policy_version 772024 (0.0008) [2023-12-26 20:58:56,297][105620] Updated weights for policy 1, policy_version 772018 (0.0007) [2023-12-26 20:58:56,330][105692] Updated weights for policy 0, policy_version 772034 (0.0008) [2023-12-26 20:58:56,351][105620] Updated weights for policy 1, policy_version 772028 (0.0008) [2023-12-26 20:58:56,409][105620] Updated weights for policy 1, policy_version 772038 (0.0006) [2023-12-26 20:58:56,945][105620] Updated weights for policy 1, policy_version 772048 (0.0006) [2023-12-26 20:58:56,994][105620] Updated weights for policy 1, policy_version 772058 (0.0005) [2023-12-26 20:58:57,061][105620] Updated weights for policy 1, policy_version 772068 (0.0008) [2023-12-26 20:58:57,208][105692] Updated weights for policy 0, policy_version 772044 (0.0008) [2023-12-26 20:58:57,259][105692] Updated weights for policy 0, policy_version 772055 (0.0009) [2023-12-26 20:58:57,320][105692] Updated weights for policy 0, policy_version 772066 (0.0010) [2023-12-26 20:58:57,613][105620] Updated weights for policy 1, policy_version 772078 (0.0005) [2023-12-26 20:58:57,675][105620] Updated weights for policy 1, policy_version 772088 (0.0005) [2023-12-26 20:58:57,728][105620] Updated weights for policy 1, policy_version 772098 (0.0006) [2023-12-26 20:58:58,134][105692] Updated weights for policy 0, policy_version 772076 (0.0009) [2023-12-26 20:58:58,201][105692] Updated weights for policy 0, policy_version 772086 (0.0008) [2023-12-26 20:58:58,260][105692] Updated weights for policy 0, policy_version 772096 (0.0008) [2023-12-26 20:58:58,432][105620] Updated weights for policy 1, policy_version 772108 (0.0009) [2023-12-26 20:58:58,495][105620] Updated weights for policy 1, policy_version 772118 (0.0008) [2023-12-26 20:58:58,562][105620] Updated weights for policy 1, policy_version 772128 (0.0008) [2023-12-26 20:58:59,046][105692] Updated weights for policy 0, policy_version 772106 (0.0007) [2023-12-26 20:58:59,111][105692] Updated weights for policy 0, policy_version 772116 (0.0009) [2023-12-26 20:58:59,177][105692] Updated weights for policy 0, policy_version 772126 (0.0009) [2023-12-26 20:58:59,251][105692] Updated weights for policy 0, policy_version 772136 (0.0009) [2023-12-26 20:58:59,306][105620] Updated weights for policy 1, policy_version 772138 (0.0008) [2023-12-26 20:58:59,372][105620] Updated weights for policy 1, policy_version 772148 (0.0008) [2023-12-26 20:58:59,429][105620] Updated weights for policy 1, policy_version 772158 (0.0005) [2023-12-26 20:58:59,491][105620] Updated weights for policy 1, policy_version 772168 (0.0005) [2023-12-26 20:59:00,063][105692] Updated weights for policy 0, policy_version 772146 (0.0006) [2023-12-26 20:59:00,112][105692] Updated weights for policy 0, policy_version 772156 (0.0006) [2023-12-26 20:59:00,115][105620] Updated weights for policy 1, policy_version 772178 (0.0009) [2023-12-26 20:59:00,161][105692] Updated weights for policy 0, policy_version 772166 (0.0005) [2023-12-26 20:59:00,176][105620] Updated weights for policy 1, policy_version 772188 (0.0010) [2023-12-26 20:59:00,234][105620] Updated weights for policy 1, policy_version 772198 (0.0010) [2023-12-26 20:59:00,781][105692] Updated weights for policy 0, policy_version 772176 (0.0007) [2023-12-26 20:59:00,827][105692] Updated weights for policy 0, policy_version 772186 (0.0005) [2023-12-26 20:59:00,875][105692] Updated weights for policy 0, policy_version 772196 (0.0005) [2023-12-26 20:59:00,887][105620] Updated weights for policy 1, policy_version 772208 (0.0006) [2023-12-26 20:59:00,935][105620] Updated weights for policy 1, policy_version 772218 (0.0005) [2023-12-26 20:59:01,000][105620] Updated weights for policy 1, policy_version 772228 (0.0005) [2023-12-26 20:59:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19716.3). Total num frames: 395427840. Throughput: 0: 9696.3, 1: 10031.5. Samples: 395389448. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-12-26 20:59:01,063][104569] Avg episode reward: [(0, '3555.212'), (1, '8988.718')] [2023-12-26 20:59:01,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000772200_197713920.pth... [2023-12-26 20:59:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000772232_197713920.pth... [2023-12-26 20:59:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000771048_197419008.pth [2023-12-26 20:59:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000771048_197410816.pth [2023-12-26 20:59:01,526][105692] Updated weights for policy 0, policy_version 772206 (0.0006) [2023-12-26 20:59:01,593][105692] Updated weights for policy 0, policy_version 772216 (0.0006) [2023-12-26 20:59:01,612][105620] Updated weights for policy 1, policy_version 772238 (0.0009) [2023-12-26 20:59:01,653][105692] Updated weights for policy 0, policy_version 772226 (0.0009) [2023-12-26 20:59:01,668][105620] Updated weights for policy 1, policy_version 772248 (0.0011) [2023-12-26 20:59:01,731][105620] Updated weights for policy 1, policy_version 772258 (0.0009) [2023-12-26 20:59:02,267][105692] Updated weights for policy 0, policy_version 772236 (0.0009) [2023-12-26 20:59:02,333][105692] Updated weights for policy 0, policy_version 772246 (0.0011) [2023-12-26 20:59:02,402][105692] Updated weights for policy 0, policy_version 772256 (0.0010) [2023-12-26 20:59:02,468][105620] Updated weights for policy 1, policy_version 772268 (0.0011) [2023-12-26 20:59:02,527][105620] Updated weights for policy 1, policy_version 772278 (0.0010) [2023-12-26 20:59:02,586][105620] Updated weights for policy 1, policy_version 772288 (0.0010) [2023-12-26 20:59:03,124][105692] Updated weights for policy 0, policy_version 772266 (0.0009) [2023-12-26 20:59:03,177][105692] Updated weights for policy 0, policy_version 772276 (0.0005) [2023-12-26 20:59:03,211][105620] Updated weights for policy 1, policy_version 772298 (0.0011) [2023-12-26 20:59:03,221][105692] Updated weights for policy 0, policy_version 772286 (0.0005) [2023-12-26 20:59:03,255][105620] Updated weights for policy 1, policy_version 772308 (0.0010) [2023-12-26 20:59:03,266][105692] Updated weights for policy 0, policy_version 772296 (0.0005) [2023-12-26 20:59:03,316][105620] Updated weights for policy 1, policy_version 772318 (0.0011) [2023-12-26 20:59:03,385][105620] Updated weights for policy 1, policy_version 772328 (0.0011) [2023-12-26 20:59:03,926][105692] Updated weights for policy 0, policy_version 772306 (0.0011) [2023-12-26 20:59:03,982][105692] Updated weights for policy 0, policy_version 772316 (0.0011) [2023-12-26 20:59:04,042][105692] Updated weights for policy 0, policy_version 772326 (0.0011) [2023-12-26 20:59:04,118][105620] Updated weights for policy 1, policy_version 772338 (0.0011) [2023-12-26 20:59:04,177][105620] Updated weights for policy 1, policy_version 772348 (0.0011) [2023-12-26 20:59:04,226][105620] Updated weights for policy 1, policy_version 772358 (0.0011) [2023-12-26 20:59:04,814][105692] Updated weights for policy 0, policy_version 772336 (0.0011) [2023-12-26 20:59:04,872][105692] Updated weights for policy 0, policy_version 772346 (0.0010) [2023-12-26 20:59:04,913][105620] Updated weights for policy 1, policy_version 772368 (0.0010) [2023-12-26 20:59:04,926][105692] Updated weights for policy 0, policy_version 772356 (0.0011) [2023-12-26 20:59:04,972][105620] Updated weights for policy 1, policy_version 772378 (0.0010) [2023-12-26 20:59:05,038][105620] Updated weights for policy 1, policy_version 772388 (0.0006) [2023-12-26 20:59:05,546][105692] Updated weights for policy 0, policy_version 772366 (0.0008) [2023-12-26 20:59:05,566][105620] Updated weights for policy 1, policy_version 772398 (0.0005) [2023-12-26 20:59:05,608][105692] Updated weights for policy 0, policy_version 772376 (0.0005) [2023-12-26 20:59:05,612][105620] Updated weights for policy 1, policy_version 772408 (0.0005) [2023-12-26 20:59:05,673][105620] Updated weights for policy 1, policy_version 772418 (0.0005) [2023-12-26 20:59:05,677][105692] Updated weights for policy 0, policy_version 772386 (0.0005) [2023-12-26 20:59:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19716.3). Total num frames: 395526144. Throughput: 0: 9692.0, 1: 10091.2. Samples: 395510968. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-12-26 20:59:06,063][104569] Avg episode reward: [(0, '6971.940'), (1, '9169.546')] [2023-12-26 20:59:06,234][105620] Updated weights for policy 1, policy_version 772428 (0.0007) [2023-12-26 20:59:06,286][105620] Updated weights for policy 1, policy_version 772438 (0.0010) [2023-12-26 20:59:06,339][105620] Updated weights for policy 1, policy_version 772448 (0.0010) [2023-12-26 20:59:06,370][105692] Updated weights for policy 0, policy_version 772396 (0.0009) [2023-12-26 20:59:06,422][105692] Updated weights for policy 0, policy_version 772406 (0.0010) [2023-12-26 20:59:06,467][105692] Updated weights for policy 0, policy_version 772416 (0.0010) [2023-12-26 20:59:07,080][105620] Updated weights for policy 1, policy_version 772458 (0.0010) [2023-12-26 20:59:07,144][105620] Updated weights for policy 1, policy_version 772468 (0.0006) [2023-12-26 20:59:07,210][105620] Updated weights for policy 1, policy_version 772478 (0.0009) [2023-12-26 20:59:07,243][105692] Updated weights for policy 0, policy_version 772426 (0.0011) [2023-12-26 20:59:07,273][105620] Updated weights for policy 1, policy_version 772488 (0.0006) [2023-12-26 20:59:07,303][105692] Updated weights for policy 0, policy_version 772436 (0.0011) [2023-12-26 20:59:07,364][105692] Updated weights for policy 0, policy_version 772446 (0.0011) [2023-12-26 20:59:07,415][105692] Updated weights for policy 0, policy_version 772456 (0.0010) [2023-12-26 20:59:07,874][105620] Updated weights for policy 1, policy_version 772498 (0.0010) [2023-12-26 20:59:07,932][105620] Updated weights for policy 1, policy_version 772508 (0.0010) [2023-12-26 20:59:07,989][105620] Updated weights for policy 1, policy_version 772518 (0.0010) [2023-12-26 20:59:08,173][105692] Updated weights for policy 0, policy_version 772466 (0.0008) [2023-12-26 20:59:08,227][105692] Updated weights for policy 0, policy_version 772476 (0.0010) [2023-12-26 20:59:08,275][105692] Updated weights for policy 0, policy_version 772486 (0.0010) [2023-12-26 20:59:08,620][105620] Updated weights for policy 1, policy_version 772528 (0.0007) [2023-12-26 20:59:08,678][105620] Updated weights for policy 1, policy_version 772538 (0.0009) [2023-12-26 20:59:08,736][105620] Updated weights for policy 1, policy_version 772548 (0.0008) [2023-12-26 20:59:09,072][105692] Updated weights for policy 0, policy_version 772496 (0.0009) [2023-12-26 20:59:09,126][105692] Updated weights for policy 0, policy_version 772506 (0.0009) [2023-12-26 20:59:09,179][105692] Updated weights for policy 0, policy_version 772516 (0.0006) [2023-12-26 20:59:09,507][105620] Updated weights for policy 1, policy_version 772558 (0.0009) [2023-12-26 20:59:09,567][105620] Updated weights for policy 1, policy_version 772568 (0.0010) [2023-12-26 20:59:09,624][105620] Updated weights for policy 1, policy_version 772578 (0.0009) [2023-12-26 20:59:09,947][105692] Updated weights for policy 0, policy_version 772526 (0.0008) [2023-12-26 20:59:10,009][105692] Updated weights for policy 0, policy_version 772536 (0.0007) [2023-12-26 20:59:10,069][105692] Updated weights for policy 0, policy_version 772546 (0.0009) [2023-12-26 20:59:10,455][105620] Updated weights for policy 1, policy_version 772588 (0.0009) [2023-12-26 20:59:10,506][105620] Updated weights for policy 1, policy_version 772598 (0.0009) [2023-12-26 20:59:10,559][105620] Updated weights for policy 1, policy_version 772608 (0.0007) [2023-12-26 20:59:10,750][105692] Updated weights for policy 0, policy_version 772556 (0.0009) [2023-12-26 20:59:10,813][105692] Updated weights for policy 0, policy_version 772566 (0.0009) [2023-12-26 20:59:10,865][105692] Updated weights for policy 0, policy_version 772576 (0.0008) [2023-12-26 20:59:11,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19797.3, 300 sec: 19744.1). Total num frames: 395624448. Throughput: 0: 9762.9, 1: 10127.0. Samples: 395630796. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-12-26 20:59:11,062][104569] Avg episode reward: [(0, '9086.117'), (1, '9263.286')] [2023-12-26 20:59:11,255][105620] Updated weights for policy 1, policy_version 772618 (0.0010) [2023-12-26 20:59:11,313][105620] Updated weights for policy 1, policy_version 772628 (0.0009) [2023-12-26 20:59:11,376][105620] Updated weights for policy 1, policy_version 772638 (0.0006) [2023-12-26 20:59:11,435][105620] Updated weights for policy 1, policy_version 772648 (0.0006) [2023-12-26 20:59:11,718][105692] Updated weights for policy 0, policy_version 772586 (0.0008) [2023-12-26 20:59:11,792][105692] Updated weights for policy 0, policy_version 772596 (0.0009) [2023-12-26 20:59:11,859][105692] Updated weights for policy 0, policy_version 772606 (0.0009) [2023-12-26 20:59:11,922][105692] Updated weights for policy 0, policy_version 772616 (0.0008) [2023-12-26 20:59:12,134][105620] Updated weights for policy 1, policy_version 772658 (0.0009) [2023-12-26 20:59:12,186][105620] Updated weights for policy 1, policy_version 772668 (0.0008) [2023-12-26 20:59:12,234][105620] Updated weights for policy 1, policy_version 772678 (0.0009) [2023-12-26 20:59:12,671][105692] Updated weights for policy 0, policy_version 772626 (0.0009) [2023-12-26 20:59:12,726][105692] Updated weights for policy 0, policy_version 772636 (0.0009) [2023-12-26 20:59:12,785][105692] Updated weights for policy 0, policy_version 772646 (0.0009) [2023-12-26 20:59:13,063][105620] Updated weights for policy 1, policy_version 772688 (0.0010) [2023-12-26 20:59:13,118][105620] Updated weights for policy 1, policy_version 772698 (0.0009) [2023-12-26 20:59:13,170][105620] Updated weights for policy 1, policy_version 772708 (0.0009) [2023-12-26 20:59:13,430][105692] Updated weights for policy 0, policy_version 772656 (0.0006) [2023-12-26 20:59:13,480][105692] Updated weights for policy 0, policy_version 772666 (0.0005) [2023-12-26 20:59:13,530][105692] Updated weights for policy 0, policy_version 772676 (0.0006) [2023-12-26 20:59:14,041][105620] Updated weights for policy 1, policy_version 772718 (0.0010) [2023-12-26 20:59:14,092][105620] Updated weights for policy 1, policy_version 772728 (0.0009) [2023-12-26 20:59:14,155][105620] Updated weights for policy 1, policy_version 772738 (0.0005) [2023-12-26 20:59:14,175][105692] Updated weights for policy 0, policy_version 772686 (0.0007) [2023-12-26 20:59:14,222][105692] Updated weights for policy 0, policy_version 772696 (0.0006) [2023-12-26 20:59:14,275][105692] Updated weights for policy 0, policy_version 772706 (0.0005) [2023-12-26 20:59:14,836][105620] Updated weights for policy 1, policy_version 772748 (0.0005) [2023-12-26 20:59:14,851][105586] KL-divergence is very high: 302.6866 [2023-12-26 20:59:14,870][105692] Updated weights for policy 0, policy_version 772716 (0.0007) [2023-12-26 20:59:14,889][105620] Updated weights for policy 1, policy_version 772758 (0.0007) [2023-12-26 20:59:14,891][105586] KL-divergence is very high: 467.8040 [2023-12-26 20:59:14,933][105692] Updated weights for policy 0, policy_version 772726 (0.0008) [2023-12-26 20:59:14,937][105586] KL-divergence is very high: 497.0211 [2023-12-26 20:59:14,949][105620] Updated weights for policy 1, policy_version 772768 (0.0006) [2023-12-26 20:59:14,984][105586] KL-divergence is very high: 478.5540 [2023-12-26 20:59:14,989][105692] Updated weights for policy 0, policy_version 772736 (0.0008) [2023-12-26 20:59:15,661][105620] Updated weights for policy 1, policy_version 772778 (0.0007) [2023-12-26 20:59:15,721][105620] Updated weights for policy 1, policy_version 772788 (0.0008) [2023-12-26 20:59:15,727][105692] Updated weights for policy 0, policy_version 772746 (0.0008) [2023-12-26 20:59:15,770][105620] Updated weights for policy 1, policy_version 772798 (0.0007) [2023-12-26 20:59:15,777][105692] Updated weights for policy 0, policy_version 772756 (0.0009) [2023-12-26 20:59:15,819][105620] Updated weights for policy 1, policy_version 772808 (0.0008) [2023-12-26 20:59:15,829][105692] Updated weights for policy 0, policy_version 772766 (0.0007) [2023-12-26 20:59:15,890][105692] Updated weights for policy 0, policy_version 772776 (0.0009) [2023-12-26 20:59:16,062][104569] Fps is (10 sec: 19660.0, 60 sec: 19933.7, 300 sec: 19744.1). Total num frames: 395722752. Throughput: 0: 9726.0, 1: 9997.1. Samples: 395686092. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-12-26 20:59:16,063][104569] Avg episode reward: [(0, '8998.487'), (1, '8998.517')] [2023-12-26 20:59:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000772776_197861376.pth... [2023-12-26 20:59:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000772808_197861376.pth... [2023-12-26 20:59:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000771656_197574656.pth [2023-12-26 20:59:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000771624_197558272.pth [2023-12-26 20:59:16,532][105620] Updated weights for policy 1, policy_version 772818 (0.0008) [2023-12-26 20:59:16,584][105620] Updated weights for policy 1, policy_version 772828 (0.0009) [2023-12-26 20:59:16,636][105620] Updated weights for policy 1, policy_version 772838 (0.0007) [2023-12-26 20:59:16,687][105692] Updated weights for policy 0, policy_version 772786 (0.0008) [2023-12-26 20:59:16,744][105692] Updated weights for policy 0, policy_version 772796 (0.0005) [2023-12-26 20:59:16,792][105692] Updated weights for policy 0, policy_version 772806 (0.0005) [2023-12-26 20:59:17,339][105620] Updated weights for policy 1, policy_version 772848 (0.0009) [2023-12-26 20:59:17,397][105620] Updated weights for policy 1, policy_version 772858 (0.0009) [2023-12-26 20:59:17,456][105620] Updated weights for policy 1, policy_version 772868 (0.0009) [2023-12-26 20:59:17,501][105692] Updated weights for policy 0, policy_version 772816 (0.0008) [2023-12-26 20:59:17,548][105692] Updated weights for policy 0, policy_version 772826 (0.0009) [2023-12-26 20:59:17,599][105692] Updated weights for policy 0, policy_version 772836 (0.0009) [2023-12-26 20:59:18,177][105620] Updated weights for policy 1, policy_version 772878 (0.0006) [2023-12-26 20:59:18,241][105620] Updated weights for policy 1, policy_version 772888 (0.0005) [2023-12-26 20:59:18,311][105620] Updated weights for policy 1, policy_version 772898 (0.0005) [2023-12-26 20:59:18,436][105692] Updated weights for policy 0, policy_version 772846 (0.0007) [2023-12-26 20:59:18,504][105692] Updated weights for policy 0, policy_version 772856 (0.0007) [2023-12-26 20:59:18,573][105692] Updated weights for policy 0, policy_version 772866 (0.0006) [2023-12-26 20:59:18,973][105620] Updated weights for policy 1, policy_version 772908 (0.0008) [2023-12-26 20:59:19,032][105620] Updated weights for policy 1, policy_version 772918 (0.0008) [2023-12-26 20:59:19,087][105620] Updated weights for policy 1, policy_version 772928 (0.0008) [2023-12-26 20:59:19,260][105692] Updated weights for policy 0, policy_version 772876 (0.0011) [2023-12-26 20:59:19,310][105692] Updated weights for policy 0, policy_version 772886 (0.0011) [2023-12-26 20:59:19,376][105692] Updated weights for policy 0, policy_version 772896 (0.0010) [2023-12-26 20:59:19,857][105620] Updated weights for policy 1, policy_version 772938 (0.0008) [2023-12-26 20:59:19,923][105620] Updated weights for policy 1, policy_version 772948 (0.0009) [2023-12-26 20:59:19,992][105620] Updated weights for policy 1, policy_version 772958 (0.0009) [2023-12-26 20:59:20,052][105692] Updated weights for policy 0, policy_version 772906 (0.0010) [2023-12-26 20:59:20,053][105620] Updated weights for policy 1, policy_version 772968 (0.0009) [2023-12-26 20:59:20,112][105692] Updated weights for policy 0, policy_version 772916 (0.0009) [2023-12-26 20:59:20,169][105692] Updated weights for policy 0, policy_version 772926 (0.0009) [2023-12-26 20:59:20,217][105692] Updated weights for policy 0, policy_version 772936 (0.0009) [2023-12-26 20:59:20,795][105620] Updated weights for policy 1, policy_version 772978 (0.0010) [2023-12-26 20:59:20,849][105620] Updated weights for policy 1, policy_version 772988 (0.0008) [2023-12-26 20:59:20,910][105620] Updated weights for policy 1, policy_version 772998 (0.0008) [2023-12-26 20:59:21,001][105692] Updated weights for policy 0, policy_version 772946 (0.0008) [2023-12-26 20:59:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19797.4, 300 sec: 19716.3). Total num frames: 395812864. Throughput: 0: 9702.1, 1: 9951.0. Samples: 395803112. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-12-26 20:59:21,063][104569] Avg episode reward: [(0, '9175.339'), (1, '8466.530')] [2023-12-26 20:59:21,063][105692] Updated weights for policy 0, policy_version 772956 (0.0009) [2023-12-26 20:59:21,121][105692] Updated weights for policy 0, policy_version 772966 (0.0009) [2023-12-26 20:59:21,764][105620] Updated weights for policy 1, policy_version 773008 (0.0009) [2023-12-26 20:59:21,826][105620] Updated weights for policy 1, policy_version 773018 (0.0009) [2023-12-26 20:59:21,889][105620] Updated weights for policy 1, policy_version 773028 (0.0009) [2023-12-26 20:59:21,896][105692] Updated weights for policy 0, policy_version 772976 (0.0007) [2023-12-26 20:59:21,954][105692] Updated weights for policy 0, policy_version 772986 (0.0010) [2023-12-26 20:59:22,013][105692] Updated weights for policy 0, policy_version 772996 (0.0009) [2023-12-26 20:59:22,685][105692] Updated weights for policy 0, policy_version 773006 (0.0007) [2023-12-26 20:59:22,731][105620] Updated weights for policy 1, policy_version 773038 (0.0009) [2023-12-26 20:59:22,739][105692] Updated weights for policy 0, policy_version 773016 (0.0005) [2023-12-26 20:59:22,790][105692] Updated weights for policy 0, policy_version 773026 (0.0009) [2023-12-26 20:59:22,791][105620] Updated weights for policy 1, policy_version 773048 (0.0007) [2023-12-26 20:59:22,849][105620] Updated weights for policy 1, policy_version 773058 (0.0006) [2023-12-26 20:59:23,463][105692] Updated weights for policy 0, policy_version 773036 (0.0011) [2023-12-26 20:59:23,517][105692] Updated weights for policy 0, policy_version 773046 (0.0010) [2023-12-26 20:59:23,561][105585] KL-divergence is very high: 124.5306 [2023-12-26 20:59:23,579][105692] Updated weights for policy 0, policy_version 773056 (0.0010) [2023-12-26 20:59:23,606][105585] KL-divergence is very high: 210.1074 [2023-12-26 20:59:23,650][105620] Updated weights for policy 1, policy_version 773068 (0.0008) [2023-12-26 20:59:23,716][105620] Updated weights for policy 1, policy_version 773078 (0.0008) [2023-12-26 20:59:23,781][105620] Updated weights for policy 1, policy_version 773088 (0.0008) [2023-12-26 20:59:24,211][105692] Updated weights for policy 0, policy_version 773066 (0.0010) [2023-12-26 20:59:24,262][105692] Updated weights for policy 0, policy_version 773076 (0.0009) [2023-12-26 20:59:24,320][105692] Updated weights for policy 0, policy_version 773086 (0.0008) [2023-12-26 20:59:24,384][105692] Updated weights for policy 0, policy_version 773096 (0.0008) [2023-12-26 20:59:24,612][105620] Updated weights for policy 1, policy_version 773098 (0.0009) [2023-12-26 20:59:24,666][105620] Updated weights for policy 1, policy_version 773108 (0.0010) [2023-12-26 20:59:24,720][105620] Updated weights for policy 1, policy_version 773118 (0.0009) [2023-12-26 20:59:24,770][105620] Updated weights for policy 1, policy_version 773128 (0.0009) [2023-12-26 20:59:25,025][105692] Updated weights for policy 0, policy_version 773106 (0.0008) [2023-12-26 20:59:25,085][105692] Updated weights for policy 0, policy_version 773116 (0.0009) [2023-12-26 20:59:25,132][105692] Updated weights for policy 0, policy_version 773126 (0.0009) [2023-12-26 20:59:25,528][105620] Updated weights for policy 1, policy_version 773138 (0.0009) [2023-12-26 20:59:25,578][105620] Updated weights for policy 1, policy_version 773148 (0.0008) [2023-12-26 20:59:25,627][105620] Updated weights for policy 1, policy_version 773158 (0.0008) [2023-12-26 20:59:25,872][105692] Updated weights for policy 0, policy_version 773136 (0.0006) [2023-12-26 20:59:25,919][105692] Updated weights for policy 0, policy_version 773146 (0.0006) [2023-12-26 20:59:25,964][105692] Updated weights for policy 0, policy_version 773156 (0.0008) [2023-12-26 20:59:26,062][104569] Fps is (10 sec: 18842.5, 60 sec: 19797.4, 300 sec: 19716.4). Total num frames: 395911168. Throughput: 0: 9744.9, 1: 9855.2. Samples: 395915868. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-12-26 20:59:26,063][104569] Avg episode reward: [(0, '9175.071'), (1, '8552.836')] [2023-12-26 20:59:26,498][105620] Updated weights for policy 1, policy_version 773168 (0.0009) [2023-12-26 20:59:26,546][105692] Updated weights for policy 0, policy_version 773166 (0.0005) [2023-12-26 20:59:26,553][105620] Updated weights for policy 1, policy_version 773178 (0.0010) [2023-12-26 20:59:26,605][105620] Updated weights for policy 1, policy_version 773188 (0.0009) [2023-12-26 20:59:26,611][105692] Updated weights for policy 0, policy_version 773176 (0.0005) [2023-12-26 20:59:26,678][105692] Updated weights for policy 0, policy_version 773186 (0.0005) [2023-12-26 20:59:27,258][105692] Updated weights for policy 0, policy_version 773196 (0.0007) [2023-12-26 20:59:27,261][105620] Updated weights for policy 1, policy_version 773198 (0.0007) [2023-12-26 20:59:27,312][105620] Updated weights for policy 1, policy_version 773208 (0.0009) [2023-12-26 20:59:27,316][105692] Updated weights for policy 0, policy_version 773206 (0.0010) [2023-12-26 20:59:27,365][105620] Updated weights for policy 1, policy_version 773218 (0.0006) [2023-12-26 20:59:27,371][105692] Updated weights for policy 0, policy_version 773216 (0.0010) [2023-12-26 20:59:27,977][105692] Updated weights for policy 0, policy_version 773226 (0.0009) [2023-12-26 20:59:28,040][105692] Updated weights for policy 0, policy_version 773236 (0.0008) [2023-12-26 20:59:28,088][105692] Updated weights for policy 0, policy_version 773246 (0.0008) [2023-12-26 20:59:28,139][105692] Updated weights for policy 0, policy_version 773256 (0.0005) [2023-12-26 20:59:28,189][105620] Updated weights for policy 1, policy_version 773228 (0.0006) [2023-12-26 20:59:28,244][105620] Updated weights for policy 1, policy_version 773238 (0.0009) [2023-12-26 20:59:28,294][105620] Updated weights for policy 1, policy_version 773248 (0.0009) [2023-12-26 20:59:28,855][105692] Updated weights for policy 0, policy_version 773266 (0.0009) [2023-12-26 20:59:28,909][105692] Updated weights for policy 0, policy_version 773276 (0.0009) [2023-12-26 20:59:28,959][105692] Updated weights for policy 0, policy_version 773286 (0.0009) [2023-12-26 20:59:29,071][105620] Updated weights for policy 1, policy_version 773258 (0.0008) [2023-12-26 20:59:29,118][105620] Updated weights for policy 1, policy_version 773268 (0.0009) [2023-12-26 20:59:29,166][105620] Updated weights for policy 1, policy_version 773278 (0.0009) [2023-12-26 20:59:29,216][105620] Updated weights for policy 1, policy_version 773288 (0.0009) [2023-12-26 20:59:29,721][105692] Updated weights for policy 0, policy_version 773296 (0.0010) [2023-12-26 20:59:29,779][105692] Updated weights for policy 0, policy_version 773306 (0.0010) [2023-12-26 20:59:29,834][105692] Updated weights for policy 0, policy_version 773316 (0.0010) [2023-12-26 20:59:30,015][105620] Updated weights for policy 1, policy_version 773298 (0.0008) [2023-12-26 20:59:30,066][105620] Updated weights for policy 1, policy_version 773308 (0.0008) [2023-12-26 20:59:30,119][105620] Updated weights for policy 1, policy_version 773318 (0.0008) [2023-12-26 20:59:30,557][105692] Updated weights for policy 0, policy_version 773326 (0.0007) [2023-12-26 20:59:30,609][105692] Updated weights for policy 0, policy_version 773336 (0.0006) [2023-12-26 20:59:30,669][105692] Updated weights for policy 0, policy_version 773346 (0.0005) [2023-12-26 20:59:30,973][105620] Updated weights for policy 1, policy_version 773328 (0.0009) [2023-12-26 20:59:31,028][105620] Updated weights for policy 1, policy_version 773338 (0.0009) [2023-12-26 20:59:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19716.3). Total num frames: 396001280. Throughput: 0: 9805.2, 1: 9832.0. Samples: 395977064. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-12-26 20:59:31,062][104569] Avg episode reward: [(0, '9179.203'), (1, '8722.729')] [2023-12-26 20:59:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000773352_198008832.pth... [2023-12-26 20:59:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000772200_197713920.pth [2023-12-26 20:59:31,092][105620] Updated weights for policy 1, policy_version 773348 (0.0007) [2023-12-26 20:59:31,116][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000773352_198000640.pth... [2023-12-26 20:59:31,122][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000772232_197713920.pth [2023-12-26 20:59:31,240][105692] Updated weights for policy 0, policy_version 773356 (0.0007) [2023-12-26 20:59:31,297][105692] Updated weights for policy 0, policy_version 773366 (0.0008) [2023-12-26 20:59:31,358][105692] Updated weights for policy 0, policy_version 773376 (0.0008) [2023-12-26 20:59:31,887][105620] Updated weights for policy 1, policy_version 773358 (0.0009) [2023-12-26 20:59:31,970][105620] Updated weights for policy 1, policy_version 773368 (0.0009) [2023-12-26 20:59:32,024][105620] Updated weights for policy 1, policy_version 773378 (0.0009) [2023-12-26 20:59:32,074][105692] Updated weights for policy 0, policy_version 773386 (0.0009) [2023-12-26 20:59:32,125][105692] Updated weights for policy 0, policy_version 773396 (0.0009) [2023-12-26 20:59:32,185][105692] Updated weights for policy 0, policy_version 773406 (0.0008) [2023-12-26 20:59:32,247][105692] Updated weights for policy 0, policy_version 773416 (0.0006) [2023-12-26 20:59:32,774][105620] Updated weights for policy 1, policy_version 773388 (0.0009) [2023-12-26 20:59:32,825][105620] Updated weights for policy 1, policy_version 773398 (0.0008) [2023-12-26 20:59:32,873][105620] Updated weights for policy 1, policy_version 773408 (0.0009) [2023-12-26 20:59:32,894][105692] Updated weights for policy 0, policy_version 773426 (0.0005) [2023-12-26 20:59:32,939][105692] Updated weights for policy 0, policy_version 773436 (0.0008) [2023-12-26 20:59:32,997][105692] Updated weights for policy 0, policy_version 773446 (0.0008) [2023-12-26 20:59:33,511][105620] Updated weights for policy 1, policy_version 773418 (0.0007) [2023-12-26 20:59:33,570][105620] Updated weights for policy 1, policy_version 773428 (0.0007) [2023-12-26 20:59:33,627][105620] Updated weights for policy 1, policy_version 773438 (0.0007) [2023-12-26 20:59:33,682][105620] Updated weights for policy 1, policy_version 773448 (0.0009) [2023-12-26 20:59:33,684][105692] Updated weights for policy 0, policy_version 773456 (0.0007) [2023-12-26 20:59:33,746][105692] Updated weights for policy 0, policy_version 773466 (0.0005) [2023-12-26 20:59:33,807][105692] Updated weights for policy 0, policy_version 773476 (0.0005) [2023-12-26 20:59:34,361][105692] Updated weights for policy 0, policy_version 773486 (0.0007) [2023-12-26 20:59:34,423][105692] Updated weights for policy 0, policy_version 773496 (0.0009) [2023-12-26 20:59:34,478][105692] Updated weights for policy 0, policy_version 773506 (0.0009) [2023-12-26 20:59:34,521][105620] Updated weights for policy 1, policy_version 773458 (0.0008) [2023-12-26 20:59:34,572][105620] Updated weights for policy 1, policy_version 773468 (0.0009) [2023-12-26 20:59:34,633][105620] Updated weights for policy 1, policy_version 773478 (0.0009) [2023-12-26 20:59:35,179][105692] Updated weights for policy 0, policy_version 773516 (0.0006) [2023-12-26 20:59:35,230][105692] Updated weights for policy 0, policy_version 773526 (0.0005) [2023-12-26 20:59:35,283][105692] Updated weights for policy 0, policy_version 773536 (0.0005) [2023-12-26 20:59:35,460][105620] Updated weights for policy 1, policy_version 773488 (0.0006) [2023-12-26 20:59:35,512][105620] Updated weights for policy 1, policy_version 773498 (0.0005) [2023-12-26 20:59:35,569][105620] Updated weights for policy 1, policy_version 773508 (0.0005) [2023-12-26 20:59:35,868][105692] Updated weights for policy 0, policy_version 773546 (0.0006) [2023-12-26 20:59:35,932][105692] Updated weights for policy 0, policy_version 773556 (0.0010) [2023-12-26 20:59:35,986][105692] Updated weights for policy 0, policy_version 773566 (0.0010) [2023-12-26 20:59:36,044][105692] Updated weights for policy 0, policy_version 773576 (0.0010) [2023-12-26 20:59:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19744.1). Total num frames: 396107776. Throughput: 0: 9802.8, 1: 9732.3. Samples: 396092424. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-12-26 20:59:36,063][104569] Avg episode reward: [(0, '9179.334'), (1, '8801.187')] [2023-12-26 20:59:36,263][105620] Updated weights for policy 1, policy_version 773518 (0.0007) [2023-12-26 20:59:36,328][105620] Updated weights for policy 1, policy_version 773528 (0.0009) [2023-12-26 20:59:36,389][105620] Updated weights for policy 1, policy_version 773538 (0.0009) [2023-12-26 20:59:36,772][105692] Updated weights for policy 0, policy_version 773586 (0.0010) [2023-12-26 20:59:36,820][105692] Updated weights for policy 0, policy_version 773596 (0.0010) [2023-12-26 20:59:36,869][105692] Updated weights for policy 0, policy_version 773606 (0.0008) [2023-12-26 20:59:37,055][105620] Updated weights for policy 1, policy_version 773548 (0.0007) [2023-12-26 20:59:37,102][105620] Updated weights for policy 1, policy_version 773558 (0.0005) [2023-12-26 20:59:37,149][105620] Updated weights for policy 1, policy_version 773568 (0.0006) [2023-12-26 20:59:37,571][105692] Updated weights for policy 0, policy_version 773616 (0.0009) [2023-12-26 20:59:37,627][105692] Updated weights for policy 0, policy_version 773626 (0.0010) [2023-12-26 20:59:37,692][105692] Updated weights for policy 0, policy_version 773636 (0.0010) [2023-12-26 20:59:37,880][105620] Updated weights for policy 1, policy_version 773578 (0.0008) [2023-12-26 20:59:37,941][105620] Updated weights for policy 1, policy_version 773591 (0.0010) [2023-12-26 20:59:37,999][105620] Updated weights for policy 1, policy_version 773601 (0.0010) [2023-12-26 20:59:38,288][105692] Updated weights for policy 0, policy_version 773646 (0.0010) [2023-12-26 20:59:38,353][105692] Updated weights for policy 0, policy_version 773656 (0.0009) [2023-12-26 20:59:38,413][105692] Updated weights for policy 0, policy_version 773666 (0.0006) [2023-12-26 20:59:38,853][105620] Updated weights for policy 1, policy_version 773611 (0.0007) [2023-12-26 20:59:38,919][105620] Updated weights for policy 1, policy_version 773621 (0.0008) [2023-12-26 20:59:38,979][105620] Updated weights for policy 1, policy_version 773631 (0.0008) [2023-12-26 20:59:39,081][105692] Updated weights for policy 0, policy_version 773676 (0.0005) [2023-12-26 20:59:39,135][105692] Updated weights for policy 0, policy_version 773686 (0.0005) [2023-12-26 20:59:39,184][105692] Updated weights for policy 0, policy_version 773696 (0.0010) [2023-12-26 20:59:39,648][105620] Updated weights for policy 1, policy_version 773641 (0.0009) [2023-12-26 20:59:39,708][105620] Updated weights for policy 1, policy_version 773651 (0.0008) [2023-12-26 20:59:39,769][105620] Updated weights for policy 1, policy_version 773661 (0.0009) [2023-12-26 20:59:39,842][105620] Updated weights for policy 1, policy_version 773671 (0.0008) [2023-12-26 20:59:39,930][105692] Updated weights for policy 0, policy_version 773706 (0.0010) [2023-12-26 20:59:39,987][105692] Updated weights for policy 0, policy_version 773716 (0.0009) [2023-12-26 20:59:40,040][105692] Updated weights for policy 0, policy_version 773726 (0.0011) [2023-12-26 20:59:40,094][105692] Updated weights for policy 0, policy_version 773736 (0.0011) [2023-12-26 20:59:40,593][105620] Updated weights for policy 1, policy_version 773681 (0.0008) [2023-12-26 20:59:40,641][105620] Updated weights for policy 1, policy_version 773691 (0.0008) [2023-12-26 20:59:40,689][105620] Updated weights for policy 1, policy_version 773701 (0.0008) [2023-12-26 20:59:40,816][105692] Updated weights for policy 0, policy_version 773746 (0.0009) [2023-12-26 20:59:40,868][105692] Updated weights for policy 0, policy_version 773756 (0.0009) [2023-12-26 20:59:40,926][105692] Updated weights for policy 0, policy_version 773766 (0.0009) [2023-12-26 20:59:41,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19744.1). Total num frames: 396206080. Throughput: 0: 9928.7, 1: 9630.1. Samples: 396211408. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-12-26 20:59:41,062][104569] Avg episode reward: [(0, '9267.684'), (1, '9075.388')] [2023-12-26 20:59:41,519][105620] Updated weights for policy 1, policy_version 773711 (0.0009) [2023-12-26 20:59:41,585][105620] Updated weights for policy 1, policy_version 773721 (0.0010) [2023-12-26 20:59:41,646][105620] Updated weights for policy 1, policy_version 773731 (0.0009) [2023-12-26 20:59:41,671][105692] Updated weights for policy 0, policy_version 773776 (0.0009) [2023-12-26 20:59:41,724][105692] Updated weights for policy 0, policy_version 773786 (0.0009) [2023-12-26 20:59:41,793][105692] Updated weights for policy 0, policy_version 773796 (0.0009) [2023-12-26 20:59:42,402][105620] Updated weights for policy 1, policy_version 773741 (0.0007) [2023-12-26 20:59:42,464][105620] Updated weights for policy 1, policy_version 773751 (0.0006) [2023-12-26 20:59:42,526][105620] Updated weights for policy 1, policy_version 773761 (0.0009) [2023-12-26 20:59:42,570][105692] Updated weights for policy 0, policy_version 773806 (0.0008) [2023-12-26 20:59:42,621][105692] Updated weights for policy 0, policy_version 773816 (0.0009) [2023-12-26 20:59:42,676][105692] Updated weights for policy 0, policy_version 773826 (0.0009) [2023-12-26 20:59:43,221][105620] Updated weights for policy 1, policy_version 773771 (0.0008) [2023-12-26 20:59:43,278][105620] Updated weights for policy 1, policy_version 773781 (0.0009) [2023-12-26 20:59:43,330][105620] Updated weights for policy 1, policy_version 773791 (0.0009) [2023-12-26 20:59:43,365][105692] Updated weights for policy 0, policy_version 773836 (0.0008) [2023-12-26 20:59:43,422][105692] Updated weights for policy 0, policy_version 773846 (0.0009) [2023-12-26 20:59:43,484][105692] Updated weights for policy 0, policy_version 773856 (0.0010) [2023-12-26 20:59:44,075][105620] Updated weights for policy 1, policy_version 773801 (0.0008) [2023-12-26 20:59:44,132][105692] Updated weights for policy 0, policy_version 773866 (0.0006) [2023-12-26 20:59:44,135][105620] Updated weights for policy 1, policy_version 773811 (0.0008) [2023-12-26 20:59:44,189][105692] Updated weights for policy 0, policy_version 773876 (0.0008) [2023-12-26 20:59:44,191][105620] Updated weights for policy 1, policy_version 773821 (0.0008) [2023-12-26 20:59:44,241][105620] Updated weights for policy 1, policy_version 773831 (0.0009) [2023-12-26 20:59:44,247][105692] Updated weights for policy 0, policy_version 773886 (0.0009) [2023-12-26 20:59:44,298][105692] Updated weights for policy 0, policy_version 773896 (0.0010) [2023-12-26 20:59:45,015][105692] Updated weights for policy 0, policy_version 773906 (0.0011) [2023-12-26 20:59:45,018][105620] Updated weights for policy 1, policy_version 773841 (0.0006) [2023-12-26 20:59:45,079][105692] Updated weights for policy 0, policy_version 773916 (0.0011) [2023-12-26 20:59:45,080][105620] Updated weights for policy 1, policy_version 773851 (0.0007) [2023-12-26 20:59:45,147][105620] Updated weights for policy 1, policy_version 773861 (0.0006) [2023-12-26 20:59:45,147][105692] Updated weights for policy 0, policy_version 773926 (0.0011) [2023-12-26 20:59:45,725][105620] Updated weights for policy 1, policy_version 773871 (0.0007) [2023-12-26 20:59:45,773][105620] Updated weights for policy 1, policy_version 773881 (0.0007) [2023-12-26 20:59:45,816][105620] Updated weights for policy 1, policy_version 773891 (0.0007) [2023-12-26 20:59:45,853][105692] Updated weights for policy 0, policy_version 773936 (0.0008) [2023-12-26 20:59:45,904][105692] Updated weights for policy 0, policy_version 773946 (0.0005) [2023-12-26 20:59:45,958][105692] Updated weights for policy 0, policy_version 773956 (0.0006) [2023-12-26 20:59:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.7, 300 sec: 19744.1). Total num frames: 396304384. Throughput: 0: 9998.7, 1: 9527.1. Samples: 396268112. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-12-26 20:59:46,063][104569] Avg episode reward: [(0, '9262.887'), (1, '9167.666')] [2023-12-26 20:59:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000773896_198139904.pth... [2023-12-26 20:59:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000773960_198164480.pth... [2023-12-26 20:59:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000772808_197861376.pth [2023-12-26 20:59:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000772776_197861376.pth [2023-12-26 20:59:46,591][105692] Updated weights for policy 0, policy_version 773966 (0.0007) [2023-12-26 20:59:46,638][105692] Updated weights for policy 0, policy_version 773976 (0.0005) [2023-12-26 20:59:46,645][105620] Updated weights for policy 1, policy_version 773901 (0.0007) [2023-12-26 20:59:46,694][105692] Updated weights for policy 0, policy_version 773986 (0.0006) [2023-12-26 20:59:46,705][105620] Updated weights for policy 1, policy_version 773911 (0.0009) [2023-12-26 20:59:46,761][105620] Updated weights for policy 1, policy_version 773921 (0.0008) [2023-12-26 20:59:47,314][105692] Updated weights for policy 0, policy_version 773996 (0.0008) [2023-12-26 20:59:47,372][105692] Updated weights for policy 0, policy_version 774006 (0.0008) [2023-12-26 20:59:47,423][105692] Updated weights for policy 0, policy_version 774016 (0.0009) [2023-12-26 20:59:47,428][105620] Updated weights for policy 1, policy_version 773931 (0.0008) [2023-12-26 20:59:47,486][105620] Updated weights for policy 1, policy_version 773941 (0.0009) [2023-12-26 20:59:47,544][105620] Updated weights for policy 1, policy_version 773951 (0.0008) [2023-12-26 20:59:48,126][105692] Updated weights for policy 0, policy_version 774026 (0.0006) [2023-12-26 20:59:48,193][105692] Updated weights for policy 0, policy_version 774036 (0.0009) [2023-12-26 20:59:48,244][105692] Updated weights for policy 0, policy_version 774046 (0.0009) [2023-12-26 20:59:48,282][105620] Updated weights for policy 1, policy_version 773961 (0.0009) [2023-12-26 20:59:48,306][105692] Updated weights for policy 0, policy_version 774056 (0.0009) [2023-12-26 20:59:48,346][105620] Updated weights for policy 1, policy_version 773971 (0.0007) [2023-12-26 20:59:48,400][105620] Updated weights for policy 1, policy_version 773981 (0.0009) [2023-12-26 20:59:48,450][105620] Updated weights for policy 1, policy_version 773991 (0.0009) [2023-12-26 20:59:48,972][105692] Updated weights for policy 0, policy_version 774066 (0.0005) [2023-12-26 20:59:49,026][105692] Updated weights for policy 0, policy_version 774076 (0.0007) [2023-12-26 20:59:49,090][105692] Updated weights for policy 0, policy_version 774086 (0.0007) [2023-12-26 20:59:49,264][105620] Updated weights for policy 1, policy_version 774001 (0.0008) [2023-12-26 20:59:49,324][105620] Updated weights for policy 1, policy_version 774011 (0.0006) [2023-12-26 20:59:49,397][105620] Updated weights for policy 1, policy_version 774021 (0.0010) [2023-12-26 20:59:49,821][105692] Updated weights for policy 0, policy_version 774096 (0.0009) [2023-12-26 20:59:49,849][105585] KL-divergence is very high: 246.4530 [2023-12-26 20:59:49,883][105692] Updated weights for policy 0, policy_version 774106 (0.0009) [2023-12-26 20:59:49,895][105585] KL-divergence is very high: 441.5052 [2023-12-26 20:59:49,948][105585] KL-divergence is very high: 520.6002 [2023-12-26 20:59:49,949][105692] Updated weights for policy 0, policy_version 774116 (0.0009) [2023-12-26 20:59:50,130][105620] Updated weights for policy 1, policy_version 774031 (0.0009) [2023-12-26 20:59:50,197][105620] Updated weights for policy 1, policy_version 774041 (0.0009) [2023-12-26 20:59:50,251][105620] Updated weights for policy 1, policy_version 774051 (0.0009) [2023-12-26 20:59:50,671][105692] Updated weights for policy 0, policy_version 774126 (0.0008) [2023-12-26 20:59:50,733][105692] Updated weights for policy 0, policy_version 774136 (0.0008) [2023-12-26 20:59:50,785][105692] Updated weights for policy 0, policy_version 774146 (0.0008) [2023-12-26 20:59:51,024][105620] Updated weights for policy 1, policy_version 774061 (0.0009) [2023-12-26 20:59:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 396394496. Throughput: 0: 10030.7, 1: 9420.9. Samples: 396386288. Policy #0 lag: (min: 1.0, avg: 16.5, max: 33.0) [2023-12-26 20:59:51,063][104569] Avg episode reward: [(0, '9263.421'), (1, '9166.630')] [2023-12-26 20:59:51,095][105620] Updated weights for policy 1, policy_version 774071 (0.0011) [2023-12-26 20:59:51,158][105620] Updated weights for policy 1, policy_version 774081 (0.0009) [2023-12-26 20:59:51,581][105692] Updated weights for policy 0, policy_version 774156 (0.0008) [2023-12-26 20:59:51,635][105692] Updated weights for policy 0, policy_version 774166 (0.0008) [2023-12-26 20:59:51,691][105692] Updated weights for policy 0, policy_version 774176 (0.0010) [2023-12-26 20:59:51,882][105620] Updated weights for policy 1, policy_version 774091 (0.0007) [2023-12-26 20:59:51,932][105620] Updated weights for policy 1, policy_version 774101 (0.0008) [2023-12-26 20:59:51,996][105620] Updated weights for policy 1, policy_version 774111 (0.0008) [2023-12-26 20:59:52,485][105692] Updated weights for policy 0, policy_version 774186 (0.0009) [2023-12-26 20:59:52,544][105692] Updated weights for policy 0, policy_version 774196 (0.0009) [2023-12-26 20:59:52,607][105692] Updated weights for policy 0, policy_version 774206 (0.0009) [2023-12-26 20:59:52,673][105692] Updated weights for policy 0, policy_version 774216 (0.0009) [2023-12-26 20:59:52,719][105620] Updated weights for policy 1, policy_version 774121 (0.0009) [2023-12-26 20:59:52,777][105620] Updated weights for policy 1, policy_version 774131 (0.0008) [2023-12-26 20:59:52,837][105620] Updated weights for policy 1, policy_version 774141 (0.0005) [2023-12-26 20:59:52,902][105620] Updated weights for policy 1, policy_version 774151 (0.0006) [2023-12-26 20:59:53,512][105692] Updated weights for policy 0, policy_version 774226 (0.0005) [2023-12-26 20:59:53,513][105620] Updated weights for policy 1, policy_version 774161 (0.0006) [2023-12-26 20:59:53,571][105692] Updated weights for policy 0, policy_version 774236 (0.0005) [2023-12-26 20:59:53,573][105620] Updated weights for policy 1, policy_version 774171 (0.0007) [2023-12-26 20:59:53,615][105692] Updated weights for policy 0, policy_version 774246 (0.0005) [2023-12-26 20:59:53,618][105620] Updated weights for policy 1, policy_version 774181 (0.0010) [2023-12-26 20:59:54,157][105692] Updated weights for policy 0, policy_version 774256 (0.0005) [2023-12-26 20:59:54,225][105692] Updated weights for policy 0, policy_version 774266 (0.0006) [2023-12-26 20:59:54,261][105620] Updated weights for policy 1, policy_version 774191 (0.0007) [2023-12-26 20:59:54,284][105692] Updated weights for policy 0, policy_version 774276 (0.0009) [2023-12-26 20:59:54,311][105620] Updated weights for policy 1, policy_version 774201 (0.0006) [2023-12-26 20:59:54,358][105620] Updated weights for policy 1, policy_version 774211 (0.0005) [2023-12-26 20:59:54,967][105692] Updated weights for policy 0, policy_version 774286 (0.0009) [2023-12-26 20:59:55,016][105692] Updated weights for policy 0, policy_version 774296 (0.0008) [2023-12-26 20:59:55,042][105620] Updated weights for policy 1, policy_version 774221 (0.0006) [2023-12-26 20:59:55,077][105692] Updated weights for policy 0, policy_version 774306 (0.0009) [2023-12-26 20:59:55,101][105620] Updated weights for policy 1, policy_version 774231 (0.0006) [2023-12-26 20:59:55,169][105620] Updated weights for policy 1, policy_version 774241 (0.0006) [2023-12-26 20:59:55,671][105692] Updated weights for policy 0, policy_version 774316 (0.0006) [2023-12-26 20:59:55,724][105692] Updated weights for policy 0, policy_version 774326 (0.0005) [2023-12-26 20:59:55,781][105692] Updated weights for policy 0, policy_version 774336 (0.0006) [2023-12-26 20:59:55,970][105620] Updated weights for policy 1, policy_version 774251 (0.0009) [2023-12-26 20:59:56,014][105620] Updated weights for policy 1, policy_version 774261 (0.0008) [2023-12-26 20:59:56,061][105620] Updated weights for policy 1, policy_version 774271 (0.0007) [2023-12-26 20:59:56,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 396492800. Throughput: 0: 10021.4, 1: 9363.6. Samples: 396503124. Policy #0 lag: (min: 1.0, avg: 16.5, max: 33.0) [2023-12-26 20:59:56,063][104569] Avg episode reward: [(0, '9085.374'), (1, '9259.785')] [2023-12-26 20:59:56,369][105692] Updated weights for policy 0, policy_version 774346 (0.0005) [2023-12-26 20:59:56,430][105692] Updated weights for policy 0, policy_version 774356 (0.0007) [2023-12-26 20:59:56,478][105692] Updated weights for policy 0, policy_version 774366 (0.0010) [2023-12-26 20:59:56,530][105692] Updated weights for policy 0, policy_version 774376 (0.0010) [2023-12-26 20:59:56,748][105620] Updated weights for policy 1, policy_version 774281 (0.0008) [2023-12-26 20:59:56,799][105620] Updated weights for policy 1, policy_version 774291 (0.0008) [2023-12-26 20:59:56,843][105620] Updated weights for policy 1, policy_version 774301 (0.0008) [2023-12-26 20:59:56,895][105620] Updated weights for policy 1, policy_version 774311 (0.0008) [2023-12-26 20:59:57,153][105692] Updated weights for policy 0, policy_version 774386 (0.0010) [2023-12-26 20:59:57,207][105692] Updated weights for policy 0, policy_version 774396 (0.0010) [2023-12-26 20:59:57,252][105692] Updated weights for policy 0, policy_version 774406 (0.0010) [2023-12-26 20:59:57,598][105620] Updated weights for policy 1, policy_version 774321 (0.0011) [2023-12-26 20:59:57,657][105620] Updated weights for policy 1, policy_version 774331 (0.0009) [2023-12-26 20:59:57,714][105620] Updated weights for policy 1, policy_version 774341 (0.0007) [2023-12-26 20:59:57,923][105692] Updated weights for policy 0, policy_version 774416 (0.0007) [2023-12-26 20:59:57,988][105692] Updated weights for policy 0, policy_version 774426 (0.0010) [2023-12-26 20:59:58,050][105692] Updated weights for policy 0, policy_version 774436 (0.0010) [2023-12-26 20:59:58,324][105620] Updated weights for policy 1, policy_version 774351 (0.0009) [2023-12-26 20:59:58,394][105620] Updated weights for policy 1, policy_version 774361 (0.0009) [2023-12-26 20:59:58,452][105620] Updated weights for policy 1, policy_version 774371 (0.0011) [2023-12-26 20:59:58,816][105692] Updated weights for policy 0, policy_version 774446 (0.0009) [2023-12-26 20:59:58,895][105692] Updated weights for policy 0, policy_version 774456 (0.0008) [2023-12-26 20:59:58,953][105692] Updated weights for policy 0, policy_version 774466 (0.0007) [2023-12-26 20:59:59,223][105620] Updated weights for policy 1, policy_version 774381 (0.0010) [2023-12-26 20:59:59,283][105620] Updated weights for policy 1, policy_version 774391 (0.0008) [2023-12-26 20:59:59,331][105620] Updated weights for policy 1, policy_version 774401 (0.0008) [2023-12-26 20:59:59,721][105692] Updated weights for policy 0, policy_version 774476 (0.0009) [2023-12-26 20:59:59,768][105692] Updated weights for policy 0, policy_version 774486 (0.0008) [2023-12-26 20:59:59,820][105692] Updated weights for policy 0, policy_version 774496 (0.0009) [2023-12-26 21:00:00,141][105620] Updated weights for policy 1, policy_version 774411 (0.0010) [2023-12-26 21:00:00,202][105620] Updated weights for policy 1, policy_version 774421 (0.0009) [2023-12-26 21:00:00,252][105620] Updated weights for policy 1, policy_version 774431 (0.0009) [2023-12-26 21:00:00,619][105692] Updated weights for policy 0, policy_version 774506 (0.0009) [2023-12-26 21:00:00,673][105692] Updated weights for policy 0, policy_version 774517 (0.0010) [2023-12-26 21:00:00,724][105692] Updated weights for policy 0, policy_version 774528 (0.0009) [2023-12-26 21:00:00,919][105620] Updated weights for policy 1, policy_version 774441 (0.0008) [2023-12-26 21:00:00,970][105620] Updated weights for policy 1, policy_version 774451 (0.0005) [2023-12-26 21:00:01,018][105620] Updated weights for policy 1, policy_version 774461 (0.0005) [2023-12-26 21:00:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19660.8). Total num frames: 396591104. Throughput: 0: 10111.7, 1: 9422.5. Samples: 396565124. Policy #0 lag: (min: 1.0, avg: 16.5, max: 33.0) [2023-12-26 21:00:01,062][104569] Avg episode reward: [(0, '9084.858'), (1, '9169.543')] [2023-12-26 21:00:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000774536_198311936.pth... [2023-12-26 21:00:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000773352_198008832.pth [2023-12-26 21:00:01,078][105620] Updated weights for policy 1, policy_version 774471 (0.0007) [2023-12-26 21:00:01,084][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000774472_198287360.pth... [2023-12-26 21:00:01,090][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000773352_198000640.pth [2023-12-26 21:00:01,569][105692] Updated weights for policy 0, policy_version 774538 (0.0008) [2023-12-26 21:00:01,628][105692] Updated weights for policy 0, policy_version 774548 (0.0010) [2023-12-26 21:00:01,689][105692] Updated weights for policy 0, policy_version 774558 (0.0010) [2023-12-26 21:00:01,691][105620] Updated weights for policy 1, policy_version 774481 (0.0006) [2023-12-26 21:00:01,754][105692] Updated weights for policy 0, policy_version 774568 (0.0009) [2023-12-26 21:00:01,759][105620] Updated weights for policy 1, policy_version 774491 (0.0008) [2023-12-26 21:00:01,826][105620] Updated weights for policy 1, policy_version 774501 (0.0007) [2023-12-26 21:00:02,425][105620] Updated weights for policy 1, policy_version 774511 (0.0007) [2023-12-26 21:00:02,478][105692] Updated weights for policy 0, policy_version 774578 (0.0010) [2023-12-26 21:00:02,479][105620] Updated weights for policy 1, policy_version 774521 (0.0006) [2023-12-26 21:00:02,525][105620] Updated weights for policy 1, policy_version 774531 (0.0008) [2023-12-26 21:00:02,540][105692] Updated weights for policy 0, policy_version 774588 (0.0010) [2023-12-26 21:00:02,604][105692] Updated weights for policy 0, policy_version 774598 (0.0010) [2023-12-26 21:00:03,126][105620] Updated weights for policy 1, policy_version 774541 (0.0008) [2023-12-26 21:00:03,195][105620] Updated weights for policy 1, policy_version 774551 (0.0007) [2023-12-26 21:00:03,219][105692] Updated weights for policy 0, policy_version 774608 (0.0006) [2023-12-26 21:00:03,250][105620] Updated weights for policy 1, policy_version 774561 (0.0008) [2023-12-26 21:00:03,280][105692] Updated weights for policy 0, policy_version 774618 (0.0005) [2023-12-26 21:00:03,348][105692] Updated weights for policy 0, policy_version 774628 (0.0005) [2023-12-26 21:00:03,858][105692] Updated weights for policy 0, policy_version 774638 (0.0007) [2023-12-26 21:00:03,916][105620] Updated weights for policy 1, policy_version 774571 (0.0007) [2023-12-26 21:00:03,928][105692] Updated weights for policy 0, policy_version 774648 (0.0009) [2023-12-26 21:00:03,969][105620] Updated weights for policy 1, policy_version 774581 (0.0005) [2023-12-26 21:00:03,988][105692] Updated weights for policy 0, policy_version 774658 (0.0009) [2023-12-26 21:00:04,027][105620] Updated weights for policy 1, policy_version 774591 (0.0005) [2023-12-26 21:00:04,697][105692] Updated weights for policy 0, policy_version 774668 (0.0008) [2023-12-26 21:00:04,745][105620] Updated weights for policy 1, policy_version 774601 (0.0009) [2023-12-26 21:00:04,746][105692] Updated weights for policy 0, policy_version 774678 (0.0008) [2023-12-26 21:00:04,793][105620] Updated weights for policy 1, policy_version 774611 (0.0006) [2023-12-26 21:00:04,799][105692] Updated weights for policy 0, policy_version 774688 (0.0007) [2023-12-26 21:00:04,848][105620] Updated weights for policy 1, policy_version 774621 (0.0006) [2023-12-26 21:00:04,899][105620] Updated weights for policy 1, policy_version 774631 (0.0008) [2023-12-26 21:00:05,416][105692] Updated weights for policy 0, policy_version 774698 (0.0006) [2023-12-26 21:00:05,471][105692] Updated weights for policy 0, policy_version 774708 (0.0009) [2023-12-26 21:00:05,529][105692] Updated weights for policy 0, policy_version 774718 (0.0006) [2023-12-26 21:00:05,535][105620] Updated weights for policy 1, policy_version 774641 (0.0007) [2023-12-26 21:00:05,577][105692] Updated weights for policy 0, policy_version 774728 (0.0007) [2023-12-26 21:00:05,583][105620] Updated weights for policy 1, policy_version 774651 (0.0006) [2023-12-26 21:00:05,638][105620] Updated weights for policy 1, policy_version 774661 (0.0009) [2023-12-26 21:00:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19716.3). Total num frames: 396697600. Throughput: 0: 10094.0, 1: 9485.1. Samples: 396684168. Policy #0 lag: (min: 1.0, avg: 16.5, max: 33.0) [2023-12-26 21:00:06,062][104569] Avg episode reward: [(0, '8811.688'), (1, '9169.694')] [2023-12-26 21:00:06,286][105620] Updated weights for policy 1, policy_version 774671 (0.0009) [2023-12-26 21:00:06,335][105620] Updated weights for policy 1, policy_version 774681 (0.0009) [2023-12-26 21:00:06,351][105692] Updated weights for policy 0, policy_version 774738 (0.0009) [2023-12-26 21:00:06,384][105620] Updated weights for policy 1, policy_version 774691 (0.0009) [2023-12-26 21:00:06,408][105692] Updated weights for policy 0, policy_version 774748 (0.0009) [2023-12-26 21:00:06,466][105692] Updated weights for policy 0, policy_version 774758 (0.0009) [2023-12-26 21:00:07,122][105620] Updated weights for policy 1, policy_version 774701 (0.0010) [2023-12-26 21:00:07,152][105692] Updated weights for policy 0, policy_version 774768 (0.0006) [2023-12-26 21:00:07,174][105620] Updated weights for policy 1, policy_version 774711 (0.0011) [2023-12-26 21:00:07,205][105692] Updated weights for policy 0, policy_version 774778 (0.0005) [2023-12-26 21:00:07,220][105620] Updated weights for policy 1, policy_version 774721 (0.0010) [2023-12-26 21:00:07,262][105692] Updated weights for policy 0, policy_version 774788 (0.0007) [2023-12-26 21:00:07,959][105620] Updated weights for policy 1, policy_version 774731 (0.0007) [2023-12-26 21:00:07,962][105692] Updated weights for policy 0, policy_version 774798 (0.0006) [2023-12-26 21:00:08,021][105692] Updated weights for policy 0, policy_version 774808 (0.0005) [2023-12-26 21:00:08,025][105620] Updated weights for policy 1, policy_version 774741 (0.0008) [2023-12-26 21:00:08,072][105692] Updated weights for policy 0, policy_version 774818 (0.0006) [2023-12-26 21:00:08,085][105620] Updated weights for policy 1, policy_version 774751 (0.0010) [2023-12-26 21:00:08,626][105620] Updated weights for policy 1, policy_version 774761 (0.0007) [2023-12-26 21:00:08,685][105620] Updated weights for policy 1, policy_version 774771 (0.0011) [2023-12-26 21:00:08,741][105620] Updated weights for policy 1, policy_version 774781 (0.0011) [2023-12-26 21:00:08,801][105620] Updated weights for policy 1, policy_version 774791 (0.0011) [2023-12-26 21:00:08,824][105692] Updated weights for policy 0, policy_version 774828 (0.0007) [2023-12-26 21:00:08,877][105692] Updated weights for policy 0, policy_version 774838 (0.0010) [2023-12-26 21:00:08,930][105692] Updated weights for policy 0, policy_version 774848 (0.0011) [2023-12-26 21:00:09,551][105620] Updated weights for policy 1, policy_version 774801 (0.0007) [2023-12-26 21:00:09,613][105620] Updated weights for policy 1, policy_version 774811 (0.0006) [2023-12-26 21:00:09,677][105620] Updated weights for policy 1, policy_version 774821 (0.0007) [2023-12-26 21:00:09,732][105692] Updated weights for policy 0, policy_version 774858 (0.0010) [2023-12-26 21:00:09,785][105692] Updated weights for policy 0, policy_version 774868 (0.0011) [2023-12-26 21:00:09,847][105692] Updated weights for policy 0, policy_version 774878 (0.0011) [2023-12-26 21:00:09,911][105692] Updated weights for policy 0, policy_version 774888 (0.0009) [2023-12-26 21:00:10,307][105620] Updated weights for policy 1, policy_version 774831 (0.0006) [2023-12-26 21:00:10,376][105620] Updated weights for policy 1, policy_version 774841 (0.0009) [2023-12-26 21:00:10,448][105620] Updated weights for policy 1, policy_version 774851 (0.0010) [2023-12-26 21:00:10,608][105692] Updated weights for policy 0, policy_version 774898 (0.0006) [2023-12-26 21:00:10,671][105692] Updated weights for policy 0, policy_version 774908 (0.0005) [2023-12-26 21:00:10,736][105692] Updated weights for policy 0, policy_version 774918 (0.0009) [2023-12-26 21:00:11,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.2, 300 sec: 19716.3). Total num frames: 396795904. Throughput: 0: 10057.0, 1: 9682.4. Samples: 396804144. Policy #0 lag: (min: 1.0, avg: 16.5, max: 33.0) [2023-12-26 21:00:11,063][104569] Avg episode reward: [(0, '8639.235'), (1, '9258.210')] [2023-12-26 21:00:11,231][105620] Updated weights for policy 1, policy_version 774861 (0.0009) [2023-12-26 21:00:11,291][105620] Updated weights for policy 1, policy_version 774871 (0.0009) [2023-12-26 21:00:11,346][105620] Updated weights for policy 1, policy_version 774881 (0.0009) [2023-12-26 21:00:11,438][105692] Updated weights for policy 0, policy_version 774928 (0.0009) [2023-12-26 21:00:11,497][105692] Updated weights for policy 0, policy_version 774938 (0.0008) [2023-12-26 21:00:11,556][105692] Updated weights for policy 0, policy_version 774948 (0.0008) [2023-12-26 21:00:12,126][105620] Updated weights for policy 1, policy_version 774891 (0.0008) [2023-12-26 21:00:12,183][105620] Updated weights for policy 1, policy_version 774901 (0.0005) [2023-12-26 21:00:12,248][105620] Updated weights for policy 1, policy_version 774911 (0.0008) [2023-12-26 21:00:12,347][105692] Updated weights for policy 0, policy_version 774958 (0.0008) [2023-12-26 21:00:12,404][105692] Updated weights for policy 0, policy_version 774968 (0.0009) [2023-12-26 21:00:12,477][105692] Updated weights for policy 0, policy_version 774978 (0.0010) [2023-12-26 21:00:12,860][105620] Updated weights for policy 1, policy_version 774921 (0.0009) [2023-12-26 21:00:12,906][105620] Updated weights for policy 1, policy_version 774931 (0.0009) [2023-12-26 21:00:12,961][105620] Updated weights for policy 1, policy_version 774941 (0.0009) [2023-12-26 21:00:13,011][105620] Updated weights for policy 1, policy_version 774951 (0.0008) [2023-12-26 21:00:13,291][105692] Updated weights for policy 0, policy_version 774988 (0.0008) [2023-12-26 21:00:13,351][105692] Updated weights for policy 0, policy_version 774998 (0.0009) [2023-12-26 21:00:13,409][105692] Updated weights for policy 0, policy_version 775008 (0.0009) [2023-12-26 21:00:13,683][105620] Updated weights for policy 1, policy_version 774961 (0.0009) [2023-12-26 21:00:13,736][105620] Updated weights for policy 1, policy_version 774971 (0.0008) [2023-12-26 21:00:13,790][105620] Updated weights for policy 1, policy_version 774981 (0.0007) [2023-12-26 21:00:14,122][105692] Updated weights for policy 0, policy_version 775018 (0.0008) [2023-12-26 21:00:14,182][105692] Updated weights for policy 0, policy_version 775028 (0.0005) [2023-12-26 21:00:14,235][105692] Updated weights for policy 0, policy_version 775038 (0.0007) [2023-12-26 21:00:14,282][105692] Updated weights for policy 0, policy_version 775048 (0.0008) [2023-12-26 21:00:14,408][105620] Updated weights for policy 1, policy_version 774991 (0.0008) [2023-12-26 21:00:14,470][105620] Updated weights for policy 1, policy_version 775001 (0.0009) [2023-12-26 21:00:14,530][105620] Updated weights for policy 1, policy_version 775011 (0.0009) [2023-12-26 21:00:14,948][105692] Updated weights for policy 0, policy_version 775058 (0.0010) [2023-12-26 21:00:15,011][105692] Updated weights for policy 0, policy_version 775068 (0.0010) [2023-12-26 21:00:15,073][105692] Updated weights for policy 0, policy_version 775078 (0.0009) [2023-12-26 21:00:15,241][105620] Updated weights for policy 1, policy_version 775021 (0.0009) [2023-12-26 21:00:15,300][105620] Updated weights for policy 1, policy_version 775031 (0.0009) [2023-12-26 21:00:15,359][105620] Updated weights for policy 1, policy_version 775041 (0.0009) [2023-12-26 21:00:15,884][105692] Updated weights for policy 0, policy_version 775088 (0.0009) [2023-12-26 21:00:15,947][105692] Updated weights for policy 0, policy_version 775098 (0.0009) [2023-12-26 21:00:16,004][105692] Updated weights for policy 0, policy_version 775108 (0.0008) [2023-12-26 21:00:16,040][105620] Updated weights for policy 1, policy_version 775051 (0.0010) [2023-12-26 21:00:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.4, 300 sec: 19688.6). Total num frames: 396894208. Throughput: 0: 9917.1, 1: 9732.0. Samples: 396861272. Policy #0 lag: (min: 1.0, avg: 16.5, max: 33.0) [2023-12-26 21:00:16,063][104569] Avg episode reward: [(0, '8907.935'), (1, '9169.838')] [2023-12-26 21:00:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000775112_198459392.pth... [2023-12-26 21:00:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000773960_198164480.pth [2023-12-26 21:00:16,101][105620] Updated weights for policy 1, policy_version 775061 (0.0010) [2023-12-26 21:00:16,159][105620] Updated weights for policy 1, policy_version 775071 (0.0010) [2023-12-26 21:00:16,209][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000775080_198443008.pth... [2023-12-26 21:00:16,213][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000773896_198139904.pth [2023-12-26 21:00:16,760][105692] Updated weights for policy 0, policy_version 775118 (0.0006) [2023-12-26 21:00:16,809][105692] Updated weights for policy 0, policy_version 775128 (0.0005) [2023-12-26 21:00:16,840][105620] Updated weights for policy 1, policy_version 775081 (0.0010) [2023-12-26 21:00:16,864][105692] Updated weights for policy 0, policy_version 775138 (0.0005) [2023-12-26 21:00:16,899][105620] Updated weights for policy 1, policy_version 775091 (0.0008) [2023-12-26 21:00:16,954][105620] Updated weights for policy 1, policy_version 775102 (0.0009) [2023-12-26 21:00:16,999][105620] Updated weights for policy 1, policy_version 775112 (0.0007) [2023-12-26 21:00:17,601][105692] Updated weights for policy 0, policy_version 775148 (0.0010) [2023-12-26 21:00:17,653][105692] Updated weights for policy 0, policy_version 775158 (0.0008) [2023-12-26 21:00:17,688][105620] Updated weights for policy 1, policy_version 775122 (0.0008) [2023-12-26 21:00:17,710][105692] Updated weights for policy 0, policy_version 775168 (0.0008) [2023-12-26 21:00:17,740][105620] Updated weights for policy 1, policy_version 775132 (0.0007) [2023-12-26 21:00:17,796][105620] Updated weights for policy 1, policy_version 775142 (0.0008) [2023-12-26 21:00:18,388][105620] Updated weights for policy 1, policy_version 775152 (0.0010) [2023-12-26 21:00:18,447][105620] Updated weights for policy 1, policy_version 775162 (0.0008) [2023-12-26 21:00:18,449][105692] Updated weights for policy 0, policy_version 775178 (0.0006) [2023-12-26 21:00:18,507][105692] Updated weights for policy 0, policy_version 775188 (0.0007) [2023-12-26 21:00:18,509][105620] Updated weights for policy 1, policy_version 775172 (0.0007) [2023-12-26 21:00:18,564][105692] Updated weights for policy 0, policy_version 775198 (0.0006) [2023-12-26 21:00:18,620][105692] Updated weights for policy 0, policy_version 775208 (0.0009) [2023-12-26 21:00:19,304][105692] Updated weights for policy 0, policy_version 775218 (0.0005) [2023-12-26 21:00:19,371][105692] Updated weights for policy 0, policy_version 775228 (0.0007) [2023-12-26 21:00:19,383][105620] Updated weights for policy 1, policy_version 775182 (0.0008) [2023-12-26 21:00:19,435][105692] Updated weights for policy 0, policy_version 775238 (0.0006) [2023-12-26 21:00:19,447][105620] Updated weights for policy 1, policy_version 775192 (0.0007) [2023-12-26 21:00:19,506][105620] Updated weights for policy 1, policy_version 775202 (0.0009) [2023-12-26 21:00:20,189][105692] Updated weights for policy 0, policy_version 775248 (0.0009) [2023-12-26 21:00:20,194][105620] Updated weights for policy 1, policy_version 775212 (0.0008) [2023-12-26 21:00:20,247][105692] Updated weights for policy 0, policy_version 775258 (0.0010) [2023-12-26 21:00:20,258][105620] Updated weights for policy 1, policy_version 775222 (0.0007) [2023-12-26 21:00:20,297][105692] Updated weights for policy 0, policy_version 775268 (0.0006) [2023-12-26 21:00:20,323][105620] Updated weights for policy 1, policy_version 775232 (0.0007) [2023-12-26 21:00:20,959][105620] Updated weights for policy 1, policy_version 775242 (0.0009) [2023-12-26 21:00:21,015][105620] Updated weights for policy 1, policy_version 775252 (0.0006) [2023-12-26 21:00:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 396984320. Throughput: 0: 9847.0, 1: 9851.2. Samples: 396978840. Policy #0 lag: (min: 1.0, avg: 16.5, max: 33.0) [2023-12-26 21:00:21,063][104569] Avg episode reward: [(0, '8215.394'), (1, '9170.878')] [2023-12-26 21:00:21,084][105620] Updated weights for policy 1, policy_version 775262 (0.0010) [2023-12-26 21:00:21,148][105620] Updated weights for policy 1, policy_version 775272 (0.0009) [2023-12-26 21:00:21,165][105692] Updated weights for policy 0, policy_version 775278 (0.0006) [2023-12-26 21:00:21,225][105692] Updated weights for policy 0, policy_version 775288 (0.0008) [2023-12-26 21:00:21,281][105692] Updated weights for policy 0, policy_version 775298 (0.0007) [2023-12-26 21:00:21,961][105620] Updated weights for policy 1, policy_version 775282 (0.0010) [2023-12-26 21:00:22,019][105692] Updated weights for policy 0, policy_version 775308 (0.0009) [2023-12-26 21:00:22,021][105620] Updated weights for policy 1, policy_version 775292 (0.0007) [2023-12-26 21:00:22,077][105692] Updated weights for policy 0, policy_version 775318 (0.0007) [2023-12-26 21:00:22,079][105620] Updated weights for policy 1, policy_version 775302 (0.0006) [2023-12-26 21:00:22,131][105692] Updated weights for policy 0, policy_version 775328 (0.0008) [2023-12-26 21:00:22,855][105620] Updated weights for policy 1, policy_version 775312 (0.0009) [2023-12-26 21:00:22,880][105692] Updated weights for policy 0, policy_version 775338 (0.0008) [2023-12-26 21:00:22,918][105620] Updated weights for policy 1, policy_version 775322 (0.0010) [2023-12-26 21:00:22,938][105692] Updated weights for policy 0, policy_version 775348 (0.0005) [2023-12-26 21:00:22,979][105620] Updated weights for policy 1, policy_version 775332 (0.0009) [2023-12-26 21:00:22,994][105692] Updated weights for policy 0, policy_version 775358 (0.0006) [2023-12-26 21:00:23,046][105692] Updated weights for policy 0, policy_version 775368 (0.0006) [2023-12-26 21:00:23,687][105692] Updated weights for policy 0, policy_version 775379 (0.0010) [2023-12-26 21:00:23,737][105692] Updated weights for policy 0, policy_version 775389 (0.0008) [2023-12-26 21:00:23,738][105620] Updated weights for policy 1, policy_version 775342 (0.0007) [2023-12-26 21:00:23,793][105620] Updated weights for policy 1, policy_version 775352 (0.0006) [2023-12-26 21:00:23,803][105692] Updated weights for policy 0, policy_version 775399 (0.0007) [2023-12-26 21:00:23,843][105620] Updated weights for policy 1, policy_version 775362 (0.0007) [2023-12-26 21:00:24,568][105692] Updated weights for policy 0, policy_version 775409 (0.0008) [2023-12-26 21:00:24,570][105620] Updated weights for policy 1, policy_version 775372 (0.0009) [2023-12-26 21:00:24,625][105620] Updated weights for policy 1, policy_version 775382 (0.0010) [2023-12-26 21:00:24,627][105692] Updated weights for policy 0, policy_version 775419 (0.0005) [2023-12-26 21:00:24,670][105620] Updated weights for policy 1, policy_version 775392 (0.0010) [2023-12-26 21:00:24,688][105692] Updated weights for policy 0, policy_version 775429 (0.0007) [2023-12-26 21:00:25,311][105692] Updated weights for policy 0, policy_version 775439 (0.0009) [2023-12-26 21:00:25,359][105692] Updated weights for policy 0, policy_version 775449 (0.0005) [2023-12-26 21:00:25,403][105692] Updated weights for policy 0, policy_version 775459 (0.0005) [2023-12-26 21:00:25,415][105620] Updated weights for policy 1, policy_version 775402 (0.0007) [2023-12-26 21:00:25,465][105620] Updated weights for policy 1, policy_version 775412 (0.0009) [2023-12-26 21:00:25,522][105620] Updated weights for policy 1, policy_version 775422 (0.0009) [2023-12-26 21:00:25,986][105692] Updated weights for policy 0, policy_version 775469 (0.0007) [2023-12-26 21:00:26,044][105692] Updated weights for policy 0, policy_version 775479 (0.0009) [2023-12-26 21:00:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 397082624. Throughput: 0: 9745.4, 1: 9869.2. Samples: 397094064. Policy #0 lag: (min: 1.0, avg: 16.5, max: 33.0) [2023-12-26 21:00:26,062][104569] Avg episode reward: [(0, '4445.977'), (1, '9350.615')] [2023-12-26 21:00:26,098][105692] Updated weights for policy 0, policy_version 775489 (0.0010) [2023-12-26 21:00:26,161][105620] Updated weights for policy 1, policy_version 775433 (0.0009) [2023-12-26 21:00:26,217][105620] Updated weights for policy 1, policy_version 775443 (0.0005) [2023-12-26 21:00:26,269][105620] Updated weights for policy 1, policy_version 775453 (0.0006) [2023-12-26 21:00:26,321][105620] Updated weights for policy 1, policy_version 775463 (0.0008) [2023-12-26 21:00:26,851][105620] Updated weights for policy 1, policy_version 775473 (0.0008) [2023-12-26 21:00:26,895][105620] Updated weights for policy 1, policy_version 775483 (0.0006) [2023-12-26 21:00:26,943][105620] Updated weights for policy 1, policy_version 775493 (0.0005) [2023-12-26 21:00:26,961][105692] Updated weights for policy 0, policy_version 775499 (0.0009) [2023-12-26 21:00:27,017][105692] Updated weights for policy 0, policy_version 775509 (0.0010) [2023-12-26 21:00:27,075][105692] Updated weights for policy 0, policy_version 775519 (0.0008) [2023-12-26 21:00:27,606][105620] Updated weights for policy 1, policy_version 775503 (0.0005) [2023-12-26 21:00:27,651][105620] Updated weights for policy 1, policy_version 775513 (0.0005) [2023-12-26 21:00:27,709][105620] Updated weights for policy 1, policy_version 775523 (0.0005) [2023-12-26 21:00:27,732][105692] Updated weights for policy 0, policy_version 775529 (0.0009) [2023-12-26 21:00:27,785][105692] Updated weights for policy 0, policy_version 775540 (0.0010) [2023-12-26 21:00:27,840][105692] Updated weights for policy 0, policy_version 775552 (0.0010) [2023-12-26 21:00:28,279][105620] Updated weights for policy 1, policy_version 775533 (0.0005) [2023-12-26 21:00:28,338][105620] Updated weights for policy 1, policy_version 775543 (0.0006) [2023-12-26 21:00:28,402][105620] Updated weights for policy 1, policy_version 775553 (0.0009) [2023-12-26 21:00:28,531][105692] Updated weights for policy 0, policy_version 775562 (0.0007) [2023-12-26 21:00:28,592][105692] Updated weights for policy 0, policy_version 775572 (0.0009) [2023-12-26 21:00:28,646][105692] Updated weights for policy 0, policy_version 775583 (0.0009) [2023-12-26 21:00:29,034][105620] Updated weights for policy 1, policy_version 775563 (0.0010) [2023-12-26 21:00:29,094][105620] Updated weights for policy 1, policy_version 775573 (0.0009) [2023-12-26 21:00:29,150][105620] Updated weights for policy 1, policy_version 775583 (0.0010) [2023-12-26 21:00:29,266][105692] Updated weights for policy 0, policy_version 775593 (0.0006) [2023-12-26 21:00:29,336][105692] Updated weights for policy 0, policy_version 775603 (0.0008) [2023-12-26 21:00:29,397][105692] Updated weights for policy 0, policy_version 775613 (0.0008) [2023-12-26 21:00:29,452][105692] Updated weights for policy 0, policy_version 775623 (0.0009) [2023-12-26 21:00:30,014][105620] Updated weights for policy 1, policy_version 775593 (0.0010) [2023-12-26 21:00:30,071][105620] Updated weights for policy 1, policy_version 775603 (0.0009) [2023-12-26 21:00:30,076][105692] Updated weights for policy 0, policy_version 775633 (0.0006) [2023-12-26 21:00:30,123][105692] Updated weights for policy 0, policy_version 775643 (0.0005) [2023-12-26 21:00:30,130][105620] Updated weights for policy 1, policy_version 775613 (0.0009) [2023-12-26 21:00:30,173][105692] Updated weights for policy 0, policy_version 775653 (0.0007) [2023-12-26 21:00:30,179][105620] Updated weights for policy 1, policy_version 775623 (0.0008) [2023-12-26 21:00:30,859][105692] Updated weights for policy 0, policy_version 775663 (0.0006) [2023-12-26 21:00:30,913][105692] Updated weights for policy 0, policy_version 775673 (0.0005) [2023-12-26 21:00:30,948][105620] Updated weights for policy 1, policy_version 775633 (0.0008) [2023-12-26 21:00:30,967][105692] Updated weights for policy 0, policy_version 775683 (0.0005) [2023-12-26 21:00:31,000][105620] Updated weights for policy 1, policy_version 775643 (0.0008) [2023-12-26 21:00:31,056][105620] Updated weights for policy 1, policy_version 775653 (0.0008) [2023-12-26 21:00:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 397189120. Throughput: 0: 9767.7, 1: 10014.9. Samples: 397158324. Policy #0 lag: (min: 1.0, avg: 16.5, max: 33.0) [2023-12-26 21:00:31,062][104569] Avg episode reward: [(0, '6573.368'), (1, '9261.968')] [2023-12-26 21:00:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000775688_198606848.pth... [2023-12-26 21:00:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000774536_198311936.pth [2023-12-26 21:00:31,075][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000775656_198590464.pth... [2023-12-26 21:00:31,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000774472_198287360.pth [2023-12-26 21:00:31,685][105692] Updated weights for policy 0, policy_version 775693 (0.0008) [2023-12-26 21:00:31,741][105692] Updated weights for policy 0, policy_version 775703 (0.0010) [2023-12-26 21:00:31,809][105620] Updated weights for policy 1, policy_version 775663 (0.0007) [2023-12-26 21:00:31,810][105692] Updated weights for policy 0, policy_version 775713 (0.0011) [2023-12-26 21:00:31,862][105620] Updated weights for policy 1, policy_version 775673 (0.0007) [2023-12-26 21:00:31,912][105620] Updated weights for policy 1, policy_version 775683 (0.0006) [2023-12-26 21:00:32,523][105692] Updated weights for policy 0, policy_version 775723 (0.0009) [2023-12-26 21:00:32,562][105620] Updated weights for policy 1, policy_version 775693 (0.0006) [2023-12-26 21:00:32,580][105692] Updated weights for policy 0, policy_version 775733 (0.0005) [2023-12-26 21:00:32,622][105620] Updated weights for policy 1, policy_version 775703 (0.0007) [2023-12-26 21:00:32,634][105692] Updated weights for policy 0, policy_version 775743 (0.0006) [2023-12-26 21:00:32,672][105620] Updated weights for policy 1, policy_version 775713 (0.0006) [2023-12-26 21:00:33,344][105692] Updated weights for policy 0, policy_version 775753 (0.0008) [2023-12-26 21:00:33,397][105692] Updated weights for policy 0, policy_version 775763 (0.0009) [2023-12-26 21:00:33,415][105620] Updated weights for policy 1, policy_version 775723 (0.0008) [2023-12-26 21:00:33,445][105692] Updated weights for policy 0, policy_version 775773 (0.0007) [2023-12-26 21:00:33,476][105620] Updated weights for policy 1, policy_version 775733 (0.0010) [2023-12-26 21:00:33,494][105692] Updated weights for policy 0, policy_version 775783 (0.0009) [2023-12-26 21:00:33,534][105620] Updated weights for policy 1, policy_version 775743 (0.0009) [2023-12-26 21:00:34,212][105620] Updated weights for policy 1, policy_version 775753 (0.0010) [2023-12-26 21:00:34,267][105692] Updated weights for policy 0, policy_version 775793 (0.0008) [2023-12-26 21:00:34,273][105620] Updated weights for policy 1, policy_version 775763 (0.0006) [2023-12-26 21:00:34,330][105692] Updated weights for policy 0, policy_version 775803 (0.0008) [2023-12-26 21:00:34,334][105620] Updated weights for policy 1, policy_version 775773 (0.0008) [2023-12-26 21:00:34,387][105692] Updated weights for policy 0, policy_version 775813 (0.0009) [2023-12-26 21:00:34,388][105620] Updated weights for policy 1, policy_version 775783 (0.0005) [2023-12-26 21:00:35,066][105692] Updated weights for policy 0, policy_version 775823 (0.0010) [2023-12-26 21:00:35,118][105692] Updated weights for policy 0, policy_version 775833 (0.0010) [2023-12-26 21:00:35,122][105620] Updated weights for policy 1, policy_version 775793 (0.0008) [2023-12-26 21:00:35,172][105692] Updated weights for policy 0, policy_version 775843 (0.0010) [2023-12-26 21:00:35,181][105620] Updated weights for policy 1, policy_version 775803 (0.0010) [2023-12-26 21:00:35,240][105620] Updated weights for policy 1, policy_version 775813 (0.0010) [2023-12-26 21:00:35,808][105692] Updated weights for policy 0, policy_version 775853 (0.0010) [2023-12-26 21:00:35,859][105692] Updated weights for policy 0, policy_version 775863 (0.0010) [2023-12-26 21:00:35,869][105620] Updated weights for policy 1, policy_version 775823 (0.0010) [2023-12-26 21:00:35,908][105692] Updated weights for policy 0, policy_version 775873 (0.0010) [2023-12-26 21:00:35,922][105620] Updated weights for policy 1, policy_version 775833 (0.0005) [2023-12-26 21:00:35,976][105620] Updated weights for policy 1, policy_version 775843 (0.0007) [2023-12-26 21:00:36,062][104569] Fps is (10 sec: 21298.3, 60 sec: 19797.3, 300 sec: 19716.3). Total num frames: 397295616. Throughput: 0: 9740.1, 1: 10016.5. Samples: 397275344. Policy #0 lag: (min: 1.0, avg: 16.5, max: 33.0) [2023-12-26 21:00:36,063][104569] Avg episode reward: [(0, '991.618'), (1, '9167.842')] [2023-12-26 21:00:36,648][105692] Updated weights for policy 0, policy_version 775883 (0.0009) [2023-12-26 21:00:36,716][105692] Updated weights for policy 0, policy_version 775893 (0.0008) [2023-12-26 21:00:36,758][105620] Updated weights for policy 1, policy_version 775853 (0.0009) [2023-12-26 21:00:36,807][105692] Updated weights for policy 0, policy_version 775903 (0.0008) [2023-12-26 21:00:36,829][105620] Updated weights for policy 1, policy_version 775863 (0.0011) [2023-12-26 21:00:36,892][105620] Updated weights for policy 1, policy_version 775873 (0.0011) [2023-12-26 21:00:37,479][105692] Updated weights for policy 0, policy_version 775913 (0.0008) [2023-12-26 21:00:37,534][105692] Updated weights for policy 0, policy_version 775923 (0.0011) [2023-12-26 21:00:37,596][105692] Updated weights for policy 0, policy_version 775933 (0.0010) [2023-12-26 21:00:37,640][105620] Updated weights for policy 1, policy_version 775883 (0.0011) [2023-12-26 21:00:37,665][105692] Updated weights for policy 0, policy_version 775943 (0.0010) [2023-12-26 21:00:37,706][105620] Updated weights for policy 1, policy_version 775893 (0.0011) [2023-12-26 21:00:37,769][105620] Updated weights for policy 1, policy_version 775903 (0.0011) [2023-12-26 21:00:38,468][105620] Updated weights for policy 1, policy_version 775913 (0.0010) [2023-12-26 21:00:38,468][105692] Updated weights for policy 0, policy_version 775953 (0.0009) [2023-12-26 21:00:38,528][105692] Updated weights for policy 0, policy_version 775963 (0.0008) [2023-12-26 21:00:38,529][105620] Updated weights for policy 1, policy_version 775923 (0.0006) [2023-12-26 21:00:38,584][105692] Updated weights for policy 0, policy_version 775973 (0.0007) [2023-12-26 21:00:38,585][105620] Updated weights for policy 1, policy_version 775933 (0.0006) [2023-12-26 21:00:38,642][105620] Updated weights for policy 1, policy_version 775943 (0.0009) [2023-12-26 21:00:39,318][105620] Updated weights for policy 1, policy_version 775953 (0.0010) [2023-12-26 21:00:39,372][105692] Updated weights for policy 0, policy_version 775983 (0.0008) [2023-12-26 21:00:39,387][105620] Updated weights for policy 1, policy_version 775963 (0.0009) [2023-12-26 21:00:39,441][105692] Updated weights for policy 0, policy_version 775993 (0.0008) [2023-12-26 21:00:39,445][105620] Updated weights for policy 1, policy_version 775973 (0.0010) [2023-12-26 21:00:39,504][105692] Updated weights for policy 0, policy_version 776003 (0.0006) [2023-12-26 21:00:40,213][105692] Updated weights for policy 0, policy_version 776013 (0.0007) [2023-12-26 21:00:40,238][105620] Updated weights for policy 1, policy_version 775983 (0.0011) [2023-12-26 21:00:40,273][105692] Updated weights for policy 0, policy_version 776023 (0.0005) [2023-12-26 21:00:40,301][105620] Updated weights for policy 1, policy_version 775993 (0.0011) [2023-12-26 21:00:40,331][105692] Updated weights for policy 0, policy_version 776033 (0.0006) [2023-12-26 21:00:40,365][105620] Updated weights for policy 1, policy_version 776003 (0.0011) [2023-12-26 21:00:41,025][105692] Updated weights for policy 0, policy_version 776043 (0.0007) [2023-12-26 21:00:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 397377536. Throughput: 0: 9741.7, 1: 9979.7. Samples: 397390588. Policy #0 lag: (min: 1.0, avg: 16.5, max: 33.0) [2023-12-26 21:00:41,062][104569] Avg episode reward: [(0, '1216.898'), (1, '9071.009')] [2023-12-26 21:00:41,087][105692] Updated weights for policy 0, policy_version 776053 (0.0008) [2023-12-26 21:00:41,100][105620] Updated weights for policy 1, policy_version 776013 (0.0010) [2023-12-26 21:00:41,152][105692] Updated weights for policy 0, policy_version 776063 (0.0008) [2023-12-26 21:00:41,165][105620] Updated weights for policy 1, policy_version 776023 (0.0010) [2023-12-26 21:00:41,234][105620] Updated weights for policy 1, policy_version 776033 (0.0010) [2023-12-26 21:00:41,936][105692] Updated weights for policy 0, policy_version 776073 (0.0007) [2023-12-26 21:00:42,004][105692] Updated weights for policy 0, policy_version 776083 (0.0006) [2023-12-26 21:00:42,005][105620] Updated weights for policy 1, policy_version 776043 (0.0009) [2023-12-26 21:00:42,063][105692] Updated weights for policy 0, policy_version 776093 (0.0006) [2023-12-26 21:00:42,065][105620] Updated weights for policy 1, policy_version 776053 (0.0008) [2023-12-26 21:00:42,079][105586] KL-divergence is very high: 131.7165 [2023-12-26 21:00:42,119][105692] Updated weights for policy 0, policy_version 776103 (0.0008) [2023-12-26 21:00:42,123][105620] Updated weights for policy 1, policy_version 776063 (0.0008) [2023-12-26 21:00:42,125][105586] KL-divergence is very high: 213.0562 [2023-12-26 21:00:42,178][105586] KL-divergence is very high: 170.9552 [2023-12-26 21:00:42,805][105692] Updated weights for policy 0, policy_version 776113 (0.0007) [2023-12-26 21:00:42,859][105692] Updated weights for policy 0, policy_version 776123 (0.0008) [2023-12-26 21:00:42,919][105692] Updated weights for policy 0, policy_version 776133 (0.0009) [2023-12-26 21:00:42,971][105620] Updated weights for policy 1, policy_version 776073 (0.0009) [2023-12-26 21:00:43,039][105620] Updated weights for policy 1, policy_version 776083 (0.0009) [2023-12-26 21:00:43,100][105620] Updated weights for policy 1, policy_version 776093 (0.0009) [2023-12-26 21:00:43,173][105620] Updated weights for policy 1, policy_version 776103 (0.0009) [2023-12-26 21:00:43,646][105692] Updated weights for policy 0, policy_version 776143 (0.0009) [2023-12-26 21:00:43,702][105692] Updated weights for policy 0, policy_version 776153 (0.0009) [2023-12-26 21:00:43,762][105692] Updated weights for policy 0, policy_version 776163 (0.0009) [2023-12-26 21:00:43,909][105620] Updated weights for policy 1, policy_version 776113 (0.0006) [2023-12-26 21:00:43,969][105620] Updated weights for policy 1, policy_version 776123 (0.0008) [2023-12-26 21:00:44,026][105620] Updated weights for policy 1, policy_version 776133 (0.0008) [2023-12-26 21:00:44,466][105692] Updated weights for policy 0, policy_version 776173 (0.0009) [2023-12-26 21:00:44,513][105692] Updated weights for policy 0, policy_version 776183 (0.0009) [2023-12-26 21:00:44,562][105692] Updated weights for policy 0, policy_version 776193 (0.0009) [2023-12-26 21:00:44,694][105620] Updated weights for policy 1, policy_version 776143 (0.0007) [2023-12-26 21:00:44,751][105620] Updated weights for policy 1, policy_version 776153 (0.0008) [2023-12-26 21:00:44,815][105620] Updated weights for policy 1, policy_version 776163 (0.0007) [2023-12-26 21:00:45,385][105692] Updated weights for policy 0, policy_version 776203 (0.0010) [2023-12-26 21:00:45,449][105692] Updated weights for policy 0, policy_version 776213 (0.0010) [2023-12-26 21:00:45,512][105692] Updated weights for policy 0, policy_version 776223 (0.0008) [2023-12-26 21:00:45,540][105620] Updated weights for policy 1, policy_version 776173 (0.0005) [2023-12-26 21:00:45,599][105620] Updated weights for policy 1, policy_version 776183 (0.0007) [2023-12-26 21:00:45,659][105620] Updated weights for policy 1, policy_version 776193 (0.0009) [2023-12-26 21:00:46,062][104569] Fps is (10 sec: 18023.0, 60 sec: 19524.4, 300 sec: 19660.8). Total num frames: 397475840. Throughput: 0: 9679.9, 1: 9899.3. Samples: 397446188. Policy #0 lag: (min: 1.0, avg: 16.5, max: 33.0) [2023-12-26 21:00:46,062][104569] Avg episode reward: [(0, '6462.228'), (1, '8981.023')] [2023-12-26 21:00:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000776232_198746112.pth... [2023-12-26 21:00:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000776200_198729728.pth... [2023-12-26 21:00:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000775112_198459392.pth [2023-12-26 21:00:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000775080_198443008.pth [2023-12-26 21:00:46,305][105692] Updated weights for policy 0, policy_version 776233 (0.0010) [2023-12-26 21:00:46,317][105620] Updated weights for policy 1, policy_version 776203 (0.0008) [2023-12-26 21:00:46,361][105692] Updated weights for policy 0, policy_version 776243 (0.0011) [2023-12-26 21:00:46,378][105620] Updated weights for policy 1, policy_version 776213 (0.0006) [2023-12-26 21:00:46,418][105692] Updated weights for policy 0, policy_version 776253 (0.0010) [2023-12-26 21:00:46,438][105620] Updated weights for policy 1, policy_version 776223 (0.0005) [2023-12-26 21:00:46,477][105692] Updated weights for policy 0, policy_version 776263 (0.0010) [2023-12-26 21:00:47,052][105620] Updated weights for policy 1, policy_version 776233 (0.0006) [2023-12-26 21:00:47,118][105620] Updated weights for policy 1, policy_version 776243 (0.0008) [2023-12-26 21:00:47,178][105692] Updated weights for policy 0, policy_version 776273 (0.0011) [2023-12-26 21:00:47,182][105620] Updated weights for policy 1, policy_version 776253 (0.0010) [2023-12-26 21:00:47,235][105692] Updated weights for policy 0, policy_version 776283 (0.0011) [2023-12-26 21:00:47,244][105620] Updated weights for policy 1, policy_version 776263 (0.0007) [2023-12-26 21:00:47,301][105692] Updated weights for policy 0, policy_version 776293 (0.0011) [2023-12-26 21:00:47,962][105620] Updated weights for policy 1, policy_version 776273 (0.0010) [2023-12-26 21:00:48,017][105620] Updated weights for policy 1, policy_version 776283 (0.0010) [2023-12-26 21:00:48,017][105692] Updated weights for policy 0, policy_version 776303 (0.0011) [2023-12-26 21:00:48,074][105692] Updated weights for policy 0, policy_version 776313 (0.0011) [2023-12-26 21:00:48,083][105620] Updated weights for policy 1, policy_version 776293 (0.0010) [2023-12-26 21:00:48,137][105692] Updated weights for policy 0, policy_version 776323 (0.0011) [2023-12-26 21:00:48,807][105620] Updated weights for policy 1, policy_version 776303 (0.0010) [2023-12-26 21:00:48,846][105692] Updated weights for policy 0, policy_version 776333 (0.0011) [2023-12-26 21:00:48,871][105620] Updated weights for policy 1, policy_version 776313 (0.0009) [2023-12-26 21:00:48,905][105692] Updated weights for policy 0, policy_version 776343 (0.0011) [2023-12-26 21:00:48,932][105620] Updated weights for policy 1, policy_version 776323 (0.0009) [2023-12-26 21:00:48,963][105692] Updated weights for policy 0, policy_version 776353 (0.0011) [2023-12-26 21:00:49,783][105620] Updated weights for policy 1, policy_version 776333 (0.0009) [2023-12-26 21:00:49,809][105692] Updated weights for policy 0, policy_version 776363 (0.0008) [2023-12-26 21:00:49,843][105620] Updated weights for policy 1, policy_version 776343 (0.0010) [2023-12-26 21:00:49,875][105692] Updated weights for policy 0, policy_version 776373 (0.0008) [2023-12-26 21:00:49,906][105620] Updated weights for policy 1, policy_version 776353 (0.0009) [2023-12-26 21:00:49,946][105692] Updated weights for policy 0, policy_version 776383 (0.0008) [2023-12-26 21:00:50,626][105620] Updated weights for policy 1, policy_version 776363 (0.0009) [2023-12-26 21:00:50,681][105620] Updated weights for policy 1, policy_version 776373 (0.0010) [2023-12-26 21:00:50,742][105692] Updated weights for policy 0, policy_version 776393 (0.0008) [2023-12-26 21:00:50,749][105620] Updated weights for policy 1, policy_version 776383 (0.0010) [2023-12-26 21:00:50,803][105692] Updated weights for policy 0, policy_version 776403 (0.0011) [2023-12-26 21:00:50,863][105692] Updated weights for policy 0, policy_version 776413 (0.0011) [2023-12-26 21:00:50,929][105692] Updated weights for policy 0, policy_version 776423 (0.0009) [2023-12-26 21:00:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 397574144. Throughput: 0: 9637.7, 1: 9820.5. Samples: 397559788. Policy #0 lag: (min: 1.0, avg: 16.5, max: 33.0) [2023-12-26 21:00:51,062][104569] Avg episode reward: [(0, '7029.117'), (1, '9073.037')] [2023-12-26 21:00:51,509][105620] Updated weights for policy 1, policy_version 776393 (0.0010) [2023-12-26 21:00:51,572][105620] Updated weights for policy 1, policy_version 776403 (0.0008) [2023-12-26 21:00:51,637][105620] Updated weights for policy 1, policy_version 776413 (0.0008) [2023-12-26 21:00:51,704][105620] Updated weights for policy 1, policy_version 776423 (0.0008) [2023-12-26 21:00:51,737][105692] Updated weights for policy 0, policy_version 776434 (0.0010) [2023-12-26 21:00:51,803][105692] Updated weights for policy 0, policy_version 776444 (0.0010) [2023-12-26 21:00:51,863][105692] Updated weights for policy 0, policy_version 776454 (0.0011) [2023-12-26 21:00:52,524][105620] Updated weights for policy 1, policy_version 776433 (0.0008) [2023-12-26 21:00:52,583][105620] Updated weights for policy 1, policy_version 776443 (0.0008) [2023-12-26 21:00:52,633][105692] Updated weights for policy 0, policy_version 776464 (0.0011) [2023-12-26 21:00:52,647][105620] Updated weights for policy 1, policy_version 776453 (0.0006) [2023-12-26 21:00:52,686][105692] Updated weights for policy 0, policy_version 776474 (0.0011) [2023-12-26 21:00:52,736][105692] Updated weights for policy 0, policy_version 776484 (0.0011) [2023-12-26 21:00:53,424][105620] Updated weights for policy 1, policy_version 776463 (0.0005) [2023-12-26 21:00:53,490][105620] Updated weights for policy 1, policy_version 776473 (0.0007) [2023-12-26 21:00:53,535][105692] Updated weights for policy 0, policy_version 776494 (0.0010) [2023-12-26 21:00:53,549][105620] Updated weights for policy 1, policy_version 776483 (0.0005) [2023-12-26 21:00:53,591][105692] Updated weights for policy 0, policy_version 776504 (0.0010) [2023-12-26 21:00:53,656][105692] Updated weights for policy 0, policy_version 776514 (0.0010) [2023-12-26 21:00:54,203][105620] Updated weights for policy 1, policy_version 776493 (0.0007) [2023-12-26 21:00:54,277][105620] Updated weights for policy 1, policy_version 776503 (0.0008) [2023-12-26 21:00:54,335][105620] Updated weights for policy 1, policy_version 776513 (0.0008) [2023-12-26 21:00:54,394][105692] Updated weights for policy 0, policy_version 776524 (0.0010) [2023-12-26 21:00:54,452][105692] Updated weights for policy 0, policy_version 776534 (0.0010) [2023-12-26 21:00:54,508][105692] Updated weights for policy 0, policy_version 776544 (0.0011) [2023-12-26 21:00:55,094][105620] Updated weights for policy 1, policy_version 776523 (0.0008) [2023-12-26 21:00:55,144][105620] Updated weights for policy 1, policy_version 776533 (0.0006) [2023-12-26 21:00:55,201][105620] Updated weights for policy 1, policy_version 776543 (0.0005) [2023-12-26 21:00:55,206][105692] Updated weights for policy 0, policy_version 776554 (0.0010) [2023-12-26 21:00:55,269][105692] Updated weights for policy 0, policy_version 776564 (0.0010) [2023-12-26 21:00:55,323][105692] Updated weights for policy 0, policy_version 776574 (0.0010) [2023-12-26 21:00:55,373][105692] Updated weights for policy 0, policy_version 776584 (0.0010) [2023-12-26 21:00:55,840][105620] Updated weights for policy 1, policy_version 776553 (0.0006) [2023-12-26 21:00:55,893][105620] Updated weights for policy 1, policy_version 776563 (0.0010) [2023-12-26 21:00:55,953][105620] Updated weights for policy 1, policy_version 776575 (0.0011) [2023-12-26 21:00:56,008][105692] Updated weights for policy 0, policy_version 776594 (0.0007) [2023-12-26 21:00:56,059][105692] Updated weights for policy 0, policy_version 776604 (0.0005) [2023-12-26 21:00:56,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 397664256. Throughput: 0: 9567.9, 1: 9713.9. Samples: 397671824. Policy #0 lag: (min: 1.0, avg: 16.5, max: 33.0) [2023-12-26 21:00:56,063][104569] Avg episode reward: [(0, '8994.140'), (1, '9255.238')] [2023-12-26 21:00:56,125][105692] Updated weights for policy 0, policy_version 776614 (0.0007) [2023-12-26 21:00:56,724][105692] Updated weights for policy 0, policy_version 776624 (0.0006) [2023-12-26 21:00:56,782][105692] Updated weights for policy 0, policy_version 776634 (0.0005) [2023-12-26 21:00:56,823][105620] Updated weights for policy 1, policy_version 776585 (0.0008) [2023-12-26 21:00:56,833][105692] Updated weights for policy 0, policy_version 776644 (0.0005) [2023-12-26 21:00:56,875][105620] Updated weights for policy 1, policy_version 776595 (0.0008) [2023-12-26 21:00:56,932][105620] Updated weights for policy 1, policy_version 776605 (0.0010) [2023-12-26 21:00:56,984][105620] Updated weights for policy 1, policy_version 776616 (0.0010) [2023-12-26 21:00:57,372][105692] Updated weights for policy 0, policy_version 776654 (0.0008) [2023-12-26 21:00:57,427][105692] Updated weights for policy 0, policy_version 776665 (0.0010) [2023-12-26 21:00:57,480][105692] Updated weights for policy 0, policy_version 776677 (0.0009) [2023-12-26 21:00:57,698][105620] Updated weights for policy 1, policy_version 776626 (0.0005) [2023-12-26 21:00:57,748][105620] Updated weights for policy 1, policy_version 776636 (0.0006) [2023-12-26 21:00:57,796][105620] Updated weights for policy 1, policy_version 776646 (0.0009) [2023-12-26 21:00:58,258][105692] Updated weights for policy 0, policy_version 776687 (0.0007) [2023-12-26 21:00:58,355][105692] Updated weights for policy 0, policy_version 776697 (0.0008) [2023-12-26 21:00:58,421][105692] Updated weights for policy 0, policy_version 776707 (0.0008) [2023-12-26 21:00:58,484][105620] Updated weights for policy 1, policy_version 776656 (0.0009) [2023-12-26 21:00:58,550][105620] Updated weights for policy 1, policy_version 776666 (0.0007) [2023-12-26 21:00:58,622][105620] Updated weights for policy 1, policy_version 776676 (0.0007) [2023-12-26 21:00:59,244][105692] Updated weights for policy 0, policy_version 776717 (0.0008) [2023-12-26 21:00:59,317][105692] Updated weights for policy 0, policy_version 776727 (0.0008) [2023-12-26 21:00:59,388][105692] Updated weights for policy 0, policy_version 776737 (0.0009) [2023-12-26 21:00:59,413][105620] Updated weights for policy 1, policy_version 776686 (0.0007) [2023-12-26 21:00:59,471][105620] Updated weights for policy 1, policy_version 776696 (0.0005) [2023-12-26 21:00:59,520][105620] Updated weights for policy 1, policy_version 776706 (0.0005) [2023-12-26 21:01:00,175][105692] Updated weights for policy 0, policy_version 776747 (0.0008) [2023-12-26 21:01:00,219][105620] Updated weights for policy 1, policy_version 776716 (0.0007) [2023-12-26 21:01:00,234][105692] Updated weights for policy 0, policy_version 776757 (0.0007) [2023-12-26 21:01:00,276][105620] Updated weights for policy 1, policy_version 776726 (0.0007) [2023-12-26 21:01:00,290][105692] Updated weights for policy 0, policy_version 776767 (0.0006) [2023-12-26 21:01:00,329][105620] Updated weights for policy 1, policy_version 776736 (0.0008) [2023-12-26 21:01:00,993][105692] Updated weights for policy 0, policy_version 776777 (0.0008) [2023-12-26 21:01:01,051][105692] Updated weights for policy 0, policy_version 776787 (0.0008) [2023-12-26 21:01:01,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 397754368. Throughput: 0: 9675.9, 1: 9665.5. Samples: 397731636. Policy #0 lag: (min: 1.0, avg: 16.5, max: 33.0) [2023-12-26 21:01:01,062][104569] Avg episode reward: [(0, '8906.894'), (1, '9350.579')] [2023-12-26 21:01:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000776744_198868992.pth... [2023-12-26 21:01:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000775656_198590464.pth [2023-12-26 21:01:01,104][105620] Updated weights for policy 1, policy_version 776746 (0.0008) [2023-12-26 21:01:01,114][105692] Updated weights for policy 0, policy_version 776797 (0.0008) [2023-12-26 21:01:01,174][105620] Updated weights for policy 1, policy_version 776756 (0.0009) [2023-12-26 21:01:01,177][105692] Updated weights for policy 0, policy_version 776807 (0.0008) [2023-12-26 21:01:01,183][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000776808_198893568.pth... [2023-12-26 21:01:01,188][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000775688_198606848.pth [2023-12-26 21:01:01,230][105620] Updated weights for policy 1, policy_version 776766 (0.0008) [2023-12-26 21:01:01,287][105620] Updated weights for policy 1, policy_version 776776 (0.0009) [2023-12-26 21:01:01,902][105692] Updated weights for policy 0, policy_version 776817 (0.0006) [2023-12-26 21:01:01,956][105692] Updated weights for policy 0, policy_version 776827 (0.0005) [2023-12-26 21:01:02,012][105692] Updated weights for policy 0, policy_version 776837 (0.0005) [2023-12-26 21:01:02,053][105620] Updated weights for policy 1, policy_version 776786 (0.0009) [2023-12-26 21:01:02,105][105620] Updated weights for policy 1, policy_version 776796 (0.0008) [2023-12-26 21:01:02,161][105620] Updated weights for policy 1, policy_version 776806 (0.0009) [2023-12-26 21:01:02,711][105692] Updated weights for policy 0, policy_version 776847 (0.0008) [2023-12-26 21:01:02,767][105692] Updated weights for policy 0, policy_version 776857 (0.0007) [2023-12-26 21:01:02,824][105692] Updated weights for policy 0, policy_version 776867 (0.0010) [2023-12-26 21:01:02,879][105620] Updated weights for policy 1, policy_version 776816 (0.0007) [2023-12-26 21:01:02,935][105620] Updated weights for policy 1, policy_version 776826 (0.0005) [2023-12-26 21:01:02,994][105620] Updated weights for policy 1, policy_version 776836 (0.0005) [2023-12-26 21:01:03,596][105620] Updated weights for policy 1, policy_version 776846 (0.0009) [2023-12-26 21:01:03,604][105692] Updated weights for policy 0, policy_version 776877 (0.0007) [2023-12-26 21:01:03,652][105620] Updated weights for policy 1, policy_version 776856 (0.0010) [2023-12-26 21:01:03,659][105692] Updated weights for policy 0, policy_version 776887 (0.0005) [2023-12-26 21:01:03,712][105620] Updated weights for policy 1, policy_version 776866 (0.0010) [2023-12-26 21:01:03,716][105692] Updated weights for policy 0, policy_version 776897 (0.0008) [2023-12-26 21:01:04,421][105692] Updated weights for policy 0, policy_version 776907 (0.0010) [2023-12-26 21:01:04,427][105620] Updated weights for policy 1, policy_version 776876 (0.0009) [2023-12-26 21:01:04,477][105692] Updated weights for policy 0, policy_version 776917 (0.0010) [2023-12-26 21:01:04,491][105620] Updated weights for policy 1, policy_version 776886 (0.0008) [2023-12-26 21:01:04,537][105692] Updated weights for policy 0, policy_version 776927 (0.0009) [2023-12-26 21:01:04,553][105620] Updated weights for policy 1, policy_version 776896 (0.0006) [2023-12-26 21:01:04,554][105586] KL-divergence is very high: 100.9652 [2023-12-26 21:01:05,148][105620] Updated weights for policy 1, policy_version 776906 (0.0007) [2023-12-26 21:01:05,211][105692] Updated weights for policy 0, policy_version 776937 (0.0011) [2023-12-26 21:01:05,212][105620] Updated weights for policy 1, policy_version 776916 (0.0005) [2023-12-26 21:01:05,256][105692] Updated weights for policy 0, policy_version 776947 (0.0010) [2023-12-26 21:01:05,261][105620] Updated weights for policy 1, policy_version 776926 (0.0005) [2023-12-26 21:01:05,315][105692] Updated weights for policy 0, policy_version 776957 (0.0010) [2023-12-26 21:01:05,315][105620] Updated weights for policy 1, policy_version 776936 (0.0005) [2023-12-26 21:01:05,373][105692] Updated weights for policy 0, policy_version 776967 (0.0010) [2023-12-26 21:01:05,875][105620] Updated weights for policy 1, policy_version 776946 (0.0009) [2023-12-26 21:01:05,930][105620] Updated weights for policy 1, policy_version 776956 (0.0010) [2023-12-26 21:01:05,975][105620] Updated weights for policy 1, policy_version 776966 (0.0010) [2023-12-26 21:01:06,044][105692] Updated weights for policy 0, policy_version 776977 (0.0006) [2023-12-26 21:01:06,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 397860864. Throughput: 0: 9631.5, 1: 9670.3. Samples: 397847420. Policy #0 lag: (min: 1.0, avg: 16.5, max: 33.0) [2023-12-26 21:01:06,062][104569] Avg episode reward: [(0, '9087.990'), (1, '9173.673')] [2023-12-26 21:01:06,103][105692] Updated weights for policy 0, policy_version 776987 (0.0009) [2023-12-26 21:01:06,166][105692] Updated weights for policy 0, policy_version 776997 (0.0010) [2023-12-26 21:01:06,727][105620] Updated weights for policy 1, policy_version 776976 (0.0011) [2023-12-26 21:01:06,793][105620] Updated weights for policy 1, policy_version 776986 (0.0009) [2023-12-26 21:01:06,849][105620] Updated weights for policy 1, policy_version 776996 (0.0011) [2023-12-26 21:01:06,889][105692] Updated weights for policy 0, policy_version 777007 (0.0011) [2023-12-26 21:01:06,949][105692] Updated weights for policy 0, policy_version 777017 (0.0010) [2023-12-26 21:01:07,008][105692] Updated weights for policy 0, policy_version 777027 (0.0010) [2023-12-26 21:01:07,552][105620] Updated weights for policy 1, policy_version 777006 (0.0007) [2023-12-26 21:01:07,613][105620] Updated weights for policy 1, policy_version 777016 (0.0005) [2023-12-26 21:01:07,670][105620] Updated weights for policy 1, policy_version 777026 (0.0005) [2023-12-26 21:01:07,762][105692] Updated weights for policy 0, policy_version 777037 (0.0009) [2023-12-26 21:01:07,822][105692] Updated weights for policy 0, policy_version 777047 (0.0007) [2023-12-26 21:01:07,871][105692] Updated weights for policy 0, policy_version 777057 (0.0007) [2023-12-26 21:01:08,338][105620] Updated weights for policy 1, policy_version 777036 (0.0008) [2023-12-26 21:01:08,395][105620] Updated weights for policy 1, policy_version 777046 (0.0008) [2023-12-26 21:01:08,452][105620] Updated weights for policy 1, policy_version 777056 (0.0009) [2023-12-26 21:01:08,568][105692] Updated weights for policy 0, policy_version 777067 (0.0007) [2023-12-26 21:01:08,616][105692] Updated weights for policy 0, policy_version 777077 (0.0008) [2023-12-26 21:01:08,672][105692] Updated weights for policy 0, policy_version 777087 (0.0009) [2023-12-26 21:01:09,243][105620] Updated weights for policy 1, policy_version 777066 (0.0008) [2023-12-26 21:01:09,316][105620] Updated weights for policy 1, policy_version 777076 (0.0006) [2023-12-26 21:01:09,384][105620] Updated weights for policy 1, policy_version 777086 (0.0008) [2023-12-26 21:01:09,447][105620] Updated weights for policy 1, policy_version 777096 (0.0008) [2023-12-26 21:01:09,493][105692] Updated weights for policy 0, policy_version 777097 (0.0010) [2023-12-26 21:01:09,549][105692] Updated weights for policy 0, policy_version 777107 (0.0008) [2023-12-26 21:01:09,611][105692] Updated weights for policy 0, policy_version 777117 (0.0009) [2023-12-26 21:01:09,670][105692] Updated weights for policy 0, policy_version 777127 (0.0009) [2023-12-26 21:01:10,151][105620] Updated weights for policy 1, policy_version 777106 (0.0006) [2023-12-26 21:01:10,216][105620] Updated weights for policy 1, policy_version 777116 (0.0008) [2023-12-26 21:01:10,277][105620] Updated weights for policy 1, policy_version 777126 (0.0008) [2023-12-26 21:01:10,458][105692] Updated weights for policy 0, policy_version 777137 (0.0006) [2023-12-26 21:01:10,527][105692] Updated weights for policy 0, policy_version 777147 (0.0006) [2023-12-26 21:01:10,594][105692] Updated weights for policy 0, policy_version 777157 (0.0011) [2023-12-26 21:01:10,947][105620] Updated weights for policy 1, policy_version 777136 (0.0009) [2023-12-26 21:01:11,012][105620] Updated weights for policy 1, policy_version 777146 (0.0010) [2023-12-26 21:01:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 397950976. Throughput: 0: 9634.6, 1: 9716.7. Samples: 397964876. Policy #0 lag: (min: 1.0, avg: 16.5, max: 33.0) [2023-12-26 21:01:11,063][104569] Avg episode reward: [(0, '9268.700'), (1, '8989.312')] [2023-12-26 21:01:11,076][105620] Updated weights for policy 1, policy_version 777156 (0.0008) [2023-12-26 21:01:11,180][105692] Updated weights for policy 0, policy_version 777167 (0.0009) [2023-12-26 21:01:11,242][105692] Updated weights for policy 0, policy_version 777177 (0.0009) [2023-12-26 21:01:11,303][105692] Updated weights for policy 0, policy_version 777187 (0.0010) [2023-12-26 21:01:11,881][105620] Updated weights for policy 1, policy_version 777166 (0.0007) [2023-12-26 21:01:11,937][105620] Updated weights for policy 1, policy_version 777176 (0.0008) [2023-12-26 21:01:11,984][105620] Updated weights for policy 1, policy_version 777186 (0.0008) [2023-12-26 21:01:12,080][105692] Updated weights for policy 0, policy_version 777197 (0.0010) [2023-12-26 21:01:12,131][105692] Updated weights for policy 0, policy_version 777207 (0.0010) [2023-12-26 21:01:12,180][105692] Updated weights for policy 0, policy_version 777217 (0.0010) [2023-12-26 21:01:12,766][105620] Updated weights for policy 1, policy_version 777196 (0.0008) [2023-12-26 21:01:12,823][105620] Updated weights for policy 1, policy_version 777206 (0.0009) [2023-12-26 21:01:12,859][105586] KL-divergence is very high: 104.4740 [2023-12-26 21:01:12,883][105620] Updated weights for policy 1, policy_version 777216 (0.0008) [2023-12-26 21:01:12,909][105586] KL-divergence is very high: 102.4023 [2023-12-26 21:01:12,970][105692] Updated weights for policy 0, policy_version 777227 (0.0010) [2023-12-26 21:01:13,039][105692] Updated weights for policy 0, policy_version 777237 (0.0010) [2023-12-26 21:01:13,103][105692] Updated weights for policy 0, policy_version 777247 (0.0009) [2023-12-26 21:01:13,596][105620] Updated weights for policy 1, policy_version 777226 (0.0008) [2023-12-26 21:01:13,647][105620] Updated weights for policy 1, policy_version 777236 (0.0009) [2023-12-26 21:01:13,705][105620] Updated weights for policy 1, policy_version 777246 (0.0009) [2023-12-26 21:01:13,768][105620] Updated weights for policy 1, policy_version 777256 (0.0009) [2023-12-26 21:01:13,809][105692] Updated weights for policy 0, policy_version 777257 (0.0009) [2023-12-26 21:01:13,871][105692] Updated weights for policy 0, policy_version 777267 (0.0010) [2023-12-26 21:01:13,922][105692] Updated weights for policy 0, policy_version 777277 (0.0009) [2023-12-26 21:01:13,973][105692] Updated weights for policy 0, policy_version 777287 (0.0009) [2023-12-26 21:01:14,452][105620] Updated weights for policy 1, policy_version 777266 (0.0009) [2023-12-26 21:01:14,506][105620] Updated weights for policy 1, policy_version 777276 (0.0009) [2023-12-26 21:01:14,552][105620] Updated weights for policy 1, policy_version 777286 (0.0008) [2023-12-26 21:01:14,796][105692] Updated weights for policy 0, policy_version 777297 (0.0008) [2023-12-26 21:01:14,863][105692] Updated weights for policy 0, policy_version 777307 (0.0007) [2023-12-26 21:01:14,923][105692] Updated weights for policy 0, policy_version 777317 (0.0009) [2023-12-26 21:01:15,341][105620] Updated weights for policy 1, policy_version 777296 (0.0007) [2023-12-26 21:01:15,402][105586] KL-divergence is very high: 118.2368 [2023-12-26 21:01:15,402][105620] Updated weights for policy 1, policy_version 777306 (0.0009) [2023-12-26 21:01:15,460][105586] KL-divergence is very high: 215.8593 [2023-12-26 21:01:15,475][105620] Updated weights for policy 1, policy_version 777316 (0.0007) [2023-12-26 21:01:15,598][105692] Updated weights for policy 0, policy_version 777327 (0.0006) [2023-12-26 21:01:15,664][105692] Updated weights for policy 0, policy_version 777337 (0.0009) [2023-12-26 21:01:15,726][105692] Updated weights for policy 0, policy_version 777347 (0.0008) [2023-12-26 21:01:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 398049280. Throughput: 0: 9607.5, 1: 9565.7. Samples: 398021116. Policy #0 lag: (min: 1.0, avg: 16.5, max: 33.0) [2023-12-26 21:01:16,062][104569] Avg episode reward: [(0, '9266.764'), (1, '8808.479')] [2023-12-26 21:01:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000777352_199032832.pth... [2023-12-26 21:01:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000777320_199016448.pth... [2023-12-26 21:01:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000776232_198746112.pth [2023-12-26 21:01:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000776200_198729728.pth [2023-12-26 21:01:16,123][105620] Updated weights for policy 1, policy_version 777326 (0.0008) [2023-12-26 21:01:16,168][105620] Updated weights for policy 1, policy_version 777336 (0.0008) [2023-12-26 21:01:16,216][105620] Updated weights for policy 1, policy_version 777346 (0.0008) [2023-12-26 21:01:16,429][105692] Updated weights for policy 0, policy_version 777357 (0.0007) [2023-12-26 21:01:16,483][105692] Updated weights for policy 0, policy_version 777367 (0.0010) [2023-12-26 21:01:16,528][105692] Updated weights for policy 0, policy_version 777377 (0.0010) [2023-12-26 21:01:16,971][105620] Updated weights for policy 1, policy_version 777356 (0.0008) [2023-12-26 21:01:17,017][105620] Updated weights for policy 1, policy_version 777366 (0.0009) [2023-12-26 21:01:17,075][105620] Updated weights for policy 1, policy_version 777376 (0.0009) [2023-12-26 21:01:17,225][105692] Updated weights for policy 0, policy_version 777387 (0.0009) [2023-12-26 21:01:17,288][105692] Updated weights for policy 0, policy_version 777397 (0.0005) [2023-12-26 21:01:17,343][105692] Updated weights for policy 0, policy_version 777407 (0.0005) [2023-12-26 21:01:17,874][105620] Updated weights for policy 1, policy_version 777386 (0.0008) [2023-12-26 21:01:17,930][105620] Updated weights for policy 1, policy_version 777396 (0.0008) [2023-12-26 21:01:17,994][105620] Updated weights for policy 1, policy_version 777406 (0.0008) [2023-12-26 21:01:17,999][105692] Updated weights for policy 0, policy_version 777417 (0.0006) [2023-12-26 21:01:18,048][105692] Updated weights for policy 0, policy_version 777427 (0.0010) [2023-12-26 21:01:18,049][105620] Updated weights for policy 1, policy_version 777416 (0.0006) [2023-12-26 21:01:18,110][105692] Updated weights for policy 0, policy_version 777437 (0.0006) [2023-12-26 21:01:18,167][105692] Updated weights for policy 0, policy_version 777447 (0.0005) [2023-12-26 21:01:18,767][105692] Updated weights for policy 0, policy_version 777457 (0.0006) [2023-12-26 21:01:18,822][105692] Updated weights for policy 0, policy_version 777467 (0.0011) [2023-12-26 21:01:18,828][105620] Updated weights for policy 1, policy_version 777426 (0.0005) [2023-12-26 21:01:18,881][105692] Updated weights for policy 0, policy_version 777477 (0.0009) [2023-12-26 21:01:18,887][105620] Updated weights for policy 1, policy_version 777436 (0.0007) [2023-12-26 21:01:18,945][105620] Updated weights for policy 1, policy_version 777446 (0.0007) [2023-12-26 21:01:19,656][105692] Updated weights for policy 0, policy_version 777487 (0.0010) [2023-12-26 21:01:19,719][105692] Updated weights for policy 0, policy_version 777497 (0.0010) [2023-12-26 21:01:19,726][105620] Updated weights for policy 1, policy_version 777456 (0.0008) [2023-12-26 21:01:19,782][105692] Updated weights for policy 0, policy_version 777507 (0.0006) [2023-12-26 21:01:19,788][105620] Updated weights for policy 1, policy_version 777466 (0.0009) [2023-12-26 21:01:19,856][105620] Updated weights for policy 1, policy_version 777476 (0.0008) [2023-12-26 21:01:20,529][105692] Updated weights for policy 0, policy_version 777517 (0.0009) [2023-12-26 21:01:20,577][105692] Updated weights for policy 0, policy_version 777527 (0.0010) [2023-12-26 21:01:20,625][105620] Updated weights for policy 1, policy_version 777486 (0.0008) [2023-12-26 21:01:20,640][105692] Updated weights for policy 0, policy_version 777537 (0.0011) [2023-12-26 21:01:20,690][105620] Updated weights for policy 1, policy_version 777496 (0.0010) [2023-12-26 21:01:20,742][105620] Updated weights for policy 1, policy_version 777506 (0.0008) [2023-12-26 21:01:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 398147584. Throughput: 0: 9594.6, 1: 9533.3. Samples: 398136096. Policy #0 lag: (min: 1.0, avg: 16.5, max: 33.0) [2023-12-26 21:01:21,062][104569] Avg episode reward: [(0, '9266.690'), (1, '8992.684')] [2023-12-26 21:01:21,441][105692] Updated weights for policy 0, policy_version 777547 (0.0010) [2023-12-26 21:01:21,461][105620] Updated weights for policy 1, policy_version 777516 (0.0007) [2023-12-26 21:01:21,505][105692] Updated weights for policy 0, policy_version 777557 (0.0009) [2023-12-26 21:01:21,520][105620] Updated weights for policy 1, policy_version 777526 (0.0006) [2023-12-26 21:01:21,569][105692] Updated weights for policy 0, policy_version 777567 (0.0008) [2023-12-26 21:01:21,578][105620] Updated weights for policy 1, policy_version 777536 (0.0006) [2023-12-26 21:01:22,248][105620] Updated weights for policy 1, policy_version 777546 (0.0007) [2023-12-26 21:01:22,267][105692] Updated weights for policy 0, policy_version 777577 (0.0008) [2023-12-26 21:01:22,311][105620] Updated weights for policy 1, policy_version 777556 (0.0009) [2023-12-26 21:01:22,326][105692] Updated weights for policy 0, policy_version 777587 (0.0006) [2023-12-26 21:01:22,372][105620] Updated weights for policy 1, policy_version 777566 (0.0008) [2023-12-26 21:01:22,388][105692] Updated weights for policy 0, policy_version 777597 (0.0008) [2023-12-26 21:01:22,434][105620] Updated weights for policy 1, policy_version 777576 (0.0009) [2023-12-26 21:01:22,445][105692] Updated weights for policy 0, policy_version 777607 (0.0006) [2023-12-26 21:01:23,118][105620] Updated weights for policy 1, policy_version 777586 (0.0009) [2023-12-26 21:01:23,176][105620] Updated weights for policy 1, policy_version 777596 (0.0007) [2023-12-26 21:01:23,198][105692] Updated weights for policy 0, policy_version 777617 (0.0011) [2023-12-26 21:01:23,245][105692] Updated weights for policy 0, policy_version 777627 (0.0007) [2023-12-26 21:01:23,248][105620] Updated weights for policy 1, policy_version 777606 (0.0006) [2023-12-26 21:01:23,291][105692] Updated weights for policy 0, policy_version 777637 (0.0005) [2023-12-26 21:01:23,928][105692] Updated weights for policy 0, policy_version 777647 (0.0005) [2023-12-26 21:01:23,983][105692] Updated weights for policy 0, policy_version 777657 (0.0007) [2023-12-26 21:01:24,046][105692] Updated weights for policy 0, policy_version 777667 (0.0008) [2023-12-26 21:01:24,061][105620] Updated weights for policy 1, policy_version 777616 (0.0006) [2023-12-26 21:01:24,114][105620] Updated weights for policy 1, policy_version 777626 (0.0006) [2023-12-26 21:01:24,161][105620] Updated weights for policy 1, policy_version 777636 (0.0009) [2023-12-26 21:01:24,682][105692] Updated weights for policy 0, policy_version 777677 (0.0008) [2023-12-26 21:01:24,742][105692] Updated weights for policy 0, policy_version 777687 (0.0007) [2023-12-26 21:01:24,801][105692] Updated weights for policy 0, policy_version 777697 (0.0007) [2023-12-26 21:01:24,935][105620] Updated weights for policy 1, policy_version 777646 (0.0007) [2023-12-26 21:01:24,990][105620] Updated weights for policy 1, policy_version 777656 (0.0005) [2023-12-26 21:01:25,050][105620] Updated weights for policy 1, policy_version 777666 (0.0006) [2023-12-26 21:01:25,442][105692] Updated weights for policy 0, policy_version 777707 (0.0009) [2023-12-26 21:01:25,501][105692] Updated weights for policy 0, policy_version 777718 (0.0010) [2023-12-26 21:01:25,554][105692] Updated weights for policy 0, policy_version 777728 (0.0010) [2023-12-26 21:01:25,561][105620] Updated weights for policy 1, policy_version 777676 (0.0006) [2023-12-26 21:01:25,616][105620] Updated weights for policy 1, policy_version 777686 (0.0009) [2023-12-26 21:01:25,664][105620] Updated weights for policy 1, policy_version 777696 (0.0010) [2023-12-26 21:01:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 398245888. Throughput: 0: 9617.4, 1: 9586.2. Samples: 398254744. Policy #0 lag: (min: 1.0, avg: 16.5, max: 33.0) [2023-12-26 21:01:26,062][104569] Avg episode reward: [(0, '9264.990'), (1, '9169.756')] [2023-12-26 21:01:26,221][105692] Updated weights for policy 0, policy_version 777738 (0.0008) [2023-12-26 21:01:26,276][105692] Updated weights for policy 0, policy_version 777748 (0.0007) [2023-12-26 21:01:26,320][105620] Updated weights for policy 1, policy_version 777706 (0.0010) [2023-12-26 21:01:26,329][105692] Updated weights for policy 0, policy_version 777758 (0.0005) [2023-12-26 21:01:26,375][105620] Updated weights for policy 1, policy_version 777716 (0.0010) [2023-12-26 21:01:26,381][105692] Updated weights for policy 0, policy_version 777768 (0.0005) [2023-12-26 21:01:26,441][105620] Updated weights for policy 1, policy_version 777726 (0.0005) [2023-12-26 21:01:26,496][105620] Updated weights for policy 1, policy_version 777736 (0.0005) [2023-12-26 21:01:27,096][105692] Updated weights for policy 0, policy_version 777778 (0.0007) [2023-12-26 21:01:27,125][105620] Updated weights for policy 1, policy_version 777746 (0.0010) [2023-12-26 21:01:27,143][105692] Updated weights for policy 0, policy_version 777788 (0.0005) [2023-12-26 21:01:27,180][105620] Updated weights for policy 1, policy_version 777756 (0.0009) [2023-12-26 21:01:27,194][105692] Updated weights for policy 0, policy_version 777798 (0.0005) [2023-12-26 21:01:27,230][105620] Updated weights for policy 1, policy_version 777766 (0.0009) [2023-12-26 21:01:27,856][105692] Updated weights for policy 0, policy_version 777808 (0.0005) [2023-12-26 21:01:27,858][105620] Updated weights for policy 1, policy_version 777776 (0.0007) [2023-12-26 21:01:27,913][105620] Updated weights for policy 1, policy_version 777786 (0.0005) [2023-12-26 21:01:27,917][105692] Updated weights for policy 0, policy_version 777818 (0.0006) [2023-12-26 21:01:27,972][105620] Updated weights for policy 1, policy_version 777796 (0.0009) [2023-12-26 21:01:27,976][105692] Updated weights for policy 0, policy_version 777828 (0.0010) [2023-12-26 21:01:28,510][105620] Updated weights for policy 1, policy_version 777806 (0.0006) [2023-12-26 21:01:28,562][105620] Updated weights for policy 1, policy_version 777816 (0.0005) [2023-12-26 21:01:28,610][105620] Updated weights for policy 1, policy_version 777826 (0.0009) [2023-12-26 21:01:28,661][105692] Updated weights for policy 0, policy_version 777838 (0.0010) [2023-12-26 21:01:28,726][105692] Updated weights for policy 0, policy_version 777848 (0.0011) [2023-12-26 21:01:28,788][105692] Updated weights for policy 0, policy_version 777858 (0.0010) [2023-12-26 21:01:29,231][105620] Updated weights for policy 1, policy_version 777836 (0.0010) [2023-12-26 21:01:29,297][105620] Updated weights for policy 1, policy_version 777846 (0.0008) [2023-12-26 21:01:29,366][105620] Updated weights for policy 1, policy_version 777856 (0.0011) [2023-12-26 21:01:29,537][105692] Updated weights for policy 0, policy_version 777868 (0.0009) [2023-12-26 21:01:29,586][105692] Updated weights for policy 0, policy_version 777878 (0.0008) [2023-12-26 21:01:29,633][105692] Updated weights for policy 0, policy_version 777888 (0.0008) [2023-12-26 21:01:30,103][105620] Updated weights for policy 1, policy_version 777866 (0.0011) [2023-12-26 21:01:30,166][105620] Updated weights for policy 1, policy_version 777876 (0.0011) [2023-12-26 21:01:30,224][105620] Updated weights for policy 1, policy_version 777886 (0.0010) [2023-12-26 21:01:30,272][105620] Updated weights for policy 1, policy_version 777896 (0.0010) [2023-12-26 21:01:30,425][105692] Updated weights for policy 0, policy_version 777898 (0.0008) [2023-12-26 21:01:30,479][105692] Updated weights for policy 0, policy_version 777908 (0.0008) [2023-12-26 21:01:30,535][105692] Updated weights for policy 0, policy_version 777918 (0.0008) [2023-12-26 21:01:30,579][105692] Updated weights for policy 0, policy_version 777928 (0.0008) [2023-12-26 21:01:31,038][105620] Updated weights for policy 1, policy_version 777906 (0.0008) [2023-12-26 21:01:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.1, 300 sec: 19577.5). Total num frames: 398344192. Throughput: 0: 9656.6, 1: 9735.8. Samples: 398318848. Policy #0 lag: (min: 1.0, avg: 16.5, max: 33.0) [2023-12-26 21:01:31,063][104569] Avg episode reward: [(0, '9264.939'), (1, '9076.019')] [2023-12-26 21:01:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000777928_199180288.pth... [2023-12-26 21:01:31,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000776808_198893568.pth [2023-12-26 21:01:31,107][105620] Updated weights for policy 1, policy_version 777916 (0.0006) [2023-12-26 21:01:31,179][105620] Updated weights for policy 1, policy_version 777926 (0.0010) [2023-12-26 21:01:31,186][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000777928_199172096.pth... [2023-12-26 21:01:31,189][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000776744_198868992.pth [2023-12-26 21:01:31,302][105692] Updated weights for policy 0, policy_version 777938 (0.0007) [2023-12-26 21:01:31,372][105692] Updated weights for policy 0, policy_version 777948 (0.0007) [2023-12-26 21:01:31,428][105692] Updated weights for policy 0, policy_version 777958 (0.0006) [2023-12-26 21:01:31,907][105620] Updated weights for policy 1, policy_version 777936 (0.0007) [2023-12-26 21:01:31,958][105620] Updated weights for policy 1, policy_version 777946 (0.0005) [2023-12-26 21:01:32,009][105620] Updated weights for policy 1, policy_version 777956 (0.0005) [2023-12-26 21:01:32,165][105692] Updated weights for policy 0, policy_version 777968 (0.0009) [2023-12-26 21:01:32,223][105692] Updated weights for policy 0, policy_version 777978 (0.0009) [2023-12-26 21:01:32,256][105585] KL-divergence is very high: 160.3293 [2023-12-26 21:01:32,281][105692] Updated weights for policy 0, policy_version 777988 (0.0007) [2023-12-26 21:01:32,725][105620] Updated weights for policy 1, policy_version 777966 (0.0009) [2023-12-26 21:01:32,774][105620] Updated weights for policy 1, policy_version 777976 (0.0008) [2023-12-26 21:01:32,839][105620] Updated weights for policy 1, policy_version 777986 (0.0007) [2023-12-26 21:01:33,028][105692] Updated weights for policy 0, policy_version 777998 (0.0008) [2023-12-26 21:01:33,089][105692] Updated weights for policy 0, policy_version 778008 (0.0009) [2023-12-26 21:01:33,143][105692] Updated weights for policy 0, policy_version 778018 (0.0009) [2023-12-26 21:01:33,499][105620] Updated weights for policy 1, policy_version 777996 (0.0009) [2023-12-26 21:01:33,560][105620] Updated weights for policy 1, policy_version 778006 (0.0010) [2023-12-26 21:01:33,614][105620] Updated weights for policy 1, policy_version 778016 (0.0010) [2023-12-26 21:01:33,827][105692] Updated weights for policy 0, policy_version 778028 (0.0009) [2023-12-26 21:01:33,881][105692] Updated weights for policy 0, policy_version 778038 (0.0007) [2023-12-26 21:01:33,947][105692] Updated weights for policy 0, policy_version 778048 (0.0007) [2023-12-26 21:01:34,365][105620] Updated weights for policy 1, policy_version 778027 (0.0009) [2023-12-26 21:01:34,427][105620] Updated weights for policy 1, policy_version 778037 (0.0006) [2023-12-26 21:01:34,444][105586] KL-divergence is very high: 103.1466 [2023-12-26 21:01:34,449][105586] KL-divergence is very high: 117.3052 [2023-12-26 21:01:34,467][105586] KL-divergence is very high: 161.7465 [2023-12-26 21:01:34,488][105620] Updated weights for policy 1, policy_version 778047 (0.0008) [2023-12-26 21:01:34,575][105692] Updated weights for policy 0, policy_version 778059 (0.0010) [2023-12-26 21:01:34,631][105692] Updated weights for policy 0, policy_version 778069 (0.0006) [2023-12-26 21:01:34,693][105692] Updated weights for policy 0, policy_version 778079 (0.0006) [2023-12-26 21:01:35,226][105620] Updated weights for policy 1, policy_version 778057 (0.0009) [2023-12-26 21:01:35,275][105620] Updated weights for policy 1, policy_version 778067 (0.0010) [2023-12-26 21:01:35,323][105620] Updated weights for policy 1, policy_version 778077 (0.0010) [2023-12-26 21:01:35,375][105620] Updated weights for policy 1, policy_version 778087 (0.0010) [2023-12-26 21:01:35,450][105692] Updated weights for policy 0, policy_version 778089 (0.0011) [2023-12-26 21:01:35,507][105692] Updated weights for policy 0, policy_version 778099 (0.0010) [2023-12-26 21:01:35,566][105692] Updated weights for policy 0, policy_version 778109 (0.0010) [2023-12-26 21:01:35,617][105692] Updated weights for policy 0, policy_version 778119 (0.0011) [2023-12-26 21:01:35,987][105620] Updated weights for policy 1, policy_version 778097 (0.0006) [2023-12-26 21:01:36,035][105620] Updated weights for policy 1, policy_version 778107 (0.0005) [2023-12-26 21:01:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19114.8, 300 sec: 19605.3). Total num frames: 398442496. Throughput: 0: 9708.2, 1: 9741.3. Samples: 398435016. Policy #0 lag: (min: 1.0, avg: 16.5, max: 33.0) [2023-12-26 21:01:36,062][104569] Avg episode reward: [(0, '8993.475'), (1, '7568.290')] [2023-12-26 21:01:36,087][105620] Updated weights for policy 1, policy_version 778117 (0.0005) [2023-12-26 21:01:36,369][105692] Updated weights for policy 0, policy_version 778129 (0.0011) [2023-12-26 21:01:36,439][105692] Updated weights for policy 0, policy_version 778139 (0.0011) [2023-12-26 21:01:36,499][105692] Updated weights for policy 0, policy_version 778149 (0.0011) [2023-12-26 21:01:36,714][105620] Updated weights for policy 1, policy_version 778127 (0.0007) [2023-12-26 21:01:36,778][105620] Updated weights for policy 1, policy_version 778137 (0.0006) [2023-12-26 21:01:36,847][105620] Updated weights for policy 1, policy_version 778147 (0.0007) [2023-12-26 21:01:37,233][105692] Updated weights for policy 0, policy_version 778159 (0.0011) [2023-12-26 21:01:37,288][105692] Updated weights for policy 0, policy_version 778169 (0.0010) [2023-12-26 21:01:37,336][105692] Updated weights for policy 0, policy_version 778179 (0.0010) [2023-12-26 21:01:37,393][105620] Updated weights for policy 1, policy_version 778157 (0.0007) [2023-12-26 21:01:37,448][105620] Updated weights for policy 1, policy_version 778167 (0.0010) [2023-12-26 21:01:37,496][105620] Updated weights for policy 1, policy_version 778177 (0.0010) [2023-12-26 21:01:38,104][105692] Updated weights for policy 0, policy_version 778189 (0.0010) [2023-12-26 21:01:38,113][105620] Updated weights for policy 1, policy_version 778187 (0.0009) [2023-12-26 21:01:38,149][105692] Updated weights for policy 0, policy_version 778199 (0.0010) [2023-12-26 21:01:38,176][105620] Updated weights for policy 1, policy_version 778197 (0.0008) [2023-12-26 21:01:38,197][105692] Updated weights for policy 0, policy_version 778209 (0.0010) [2023-12-26 21:01:38,234][105620] Updated weights for policy 1, policy_version 778207 (0.0011) [2023-12-26 21:01:38,949][105620] Updated weights for policy 1, policy_version 778217 (0.0010) [2023-12-26 21:01:38,950][105692] Updated weights for policy 0, policy_version 778219 (0.0010) [2023-12-26 21:01:39,009][105692] Updated weights for policy 0, policy_version 778229 (0.0010) [2023-12-26 21:01:39,012][105620] Updated weights for policy 1, policy_version 778227 (0.0011) [2023-12-26 21:01:39,066][105620] Updated weights for policy 1, policy_version 778237 (0.0010) [2023-12-26 21:01:39,071][105692] Updated weights for policy 0, policy_version 778239 (0.0010) [2023-12-26 21:01:39,115][105620] Updated weights for policy 1, policy_version 778247 (0.0010) [2023-12-26 21:01:39,769][105692] Updated weights for policy 0, policy_version 778249 (0.0010) [2023-12-26 21:01:39,839][105692] Updated weights for policy 0, policy_version 778259 (0.0007) [2023-12-26 21:01:39,903][105620] Updated weights for policy 1, policy_version 778257 (0.0011) [2023-12-26 21:01:39,905][105692] Updated weights for policy 0, policy_version 778269 (0.0008) [2023-12-26 21:01:39,969][105620] Updated weights for policy 1, policy_version 778267 (0.0011) [2023-12-26 21:01:39,973][105692] Updated weights for policy 0, policy_version 778279 (0.0010) [2023-12-26 21:01:40,029][105620] Updated weights for policy 1, policy_version 778277 (0.0011) [2023-12-26 21:01:40,668][105692] Updated weights for policy 0, policy_version 778289 (0.0007) [2023-12-26 21:01:40,730][105692] Updated weights for policy 0, policy_version 778299 (0.0007) [2023-12-26 21:01:40,778][105620] Updated weights for policy 1, policy_version 778287 (0.0010) [2023-12-26 21:01:40,797][105692] Updated weights for policy 0, policy_version 778309 (0.0010) [2023-12-26 21:01:40,835][105620] Updated weights for policy 1, policy_version 778297 (0.0011) [2023-12-26 21:01:40,892][105620] Updated weights for policy 1, policy_version 778307 (0.0011) [2023-12-26 21:01:41,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 398548992. Throughput: 0: 9743.4, 1: 9846.4. Samples: 398553364. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-12-26 21:01:41,062][104569] Avg episode reward: [(0, '9084.796'), (1, '7594.931')] [2023-12-26 21:01:41,521][105692] Updated weights for policy 0, policy_version 778319 (0.0010) [2023-12-26 21:01:41,577][105692] Updated weights for policy 0, policy_version 778329 (0.0009) [2023-12-26 21:01:41,619][105620] Updated weights for policy 1, policy_version 778317 (0.0009) [2023-12-26 21:01:41,637][105692] Updated weights for policy 0, policy_version 778339 (0.0008) [2023-12-26 21:01:41,687][105620] Updated weights for policy 1, policy_version 778327 (0.0007) [2023-12-26 21:01:41,753][105620] Updated weights for policy 1, policy_version 778337 (0.0007) [2023-12-26 21:01:42,359][105692] Updated weights for policy 0, policy_version 778349 (0.0007) [2023-12-26 21:01:42,420][105692] Updated weights for policy 0, policy_version 778359 (0.0008) [2023-12-26 21:01:42,449][105620] Updated weights for policy 1, policy_version 778347 (0.0010) [2023-12-26 21:01:42,478][105692] Updated weights for policy 0, policy_version 778369 (0.0007) [2023-12-26 21:01:42,505][105620] Updated weights for policy 1, policy_version 778357 (0.0010) [2023-12-26 21:01:42,570][105620] Updated weights for policy 1, policy_version 778367 (0.0006) [2023-12-26 21:01:43,148][105620] Updated weights for policy 1, policy_version 778377 (0.0007) [2023-12-26 21:01:43,210][105620] Updated weights for policy 1, policy_version 778387 (0.0008) [2023-12-26 21:01:43,271][105620] Updated weights for policy 1, policy_version 778397 (0.0008) [2023-12-26 21:01:43,277][105692] Updated weights for policy 0, policy_version 778379 (0.0006) [2023-12-26 21:01:43,320][105620] Updated weights for policy 1, policy_version 778407 (0.0008) [2023-12-26 21:01:43,338][105692] Updated weights for policy 0, policy_version 778389 (0.0007) [2023-12-26 21:01:43,403][105692] Updated weights for policy 0, policy_version 778399 (0.0008) [2023-12-26 21:01:43,960][105620] Updated weights for policy 1, policy_version 778417 (0.0005) [2023-12-26 21:01:44,012][105620] Updated weights for policy 1, policy_version 778427 (0.0005) [2023-12-26 21:01:44,074][105620] Updated weights for policy 1, policy_version 778437 (0.0005) [2023-12-26 21:01:44,117][105692] Updated weights for policy 0, policy_version 778409 (0.0009) [2023-12-26 21:01:44,175][105692] Updated weights for policy 0, policy_version 778419 (0.0010) [2023-12-26 21:01:44,237][105692] Updated weights for policy 0, policy_version 778429 (0.0007) [2023-12-26 21:01:44,303][105692] Updated weights for policy 0, policy_version 778439 (0.0005) [2023-12-26 21:01:44,753][105620] Updated weights for policy 1, policy_version 778447 (0.0009) [2023-12-26 21:01:44,814][105620] Updated weights for policy 1, policy_version 778457 (0.0010) [2023-12-26 21:01:44,867][105620] Updated weights for policy 1, policy_version 778467 (0.0012) [2023-12-26 21:01:44,999][105692] Updated weights for policy 0, policy_version 778449 (0.0008) [2023-12-26 21:01:45,065][105692] Updated weights for policy 0, policy_version 778459 (0.0008) [2023-12-26 21:01:45,133][105692] Updated weights for policy 0, policy_version 778469 (0.0008) [2023-12-26 21:01:45,615][105620] Updated weights for policy 1, policy_version 778477 (0.0011) [2023-12-26 21:01:45,679][105620] Updated weights for policy 1, policy_version 778487 (0.0011) [2023-12-26 21:01:45,737][105620] Updated weights for policy 1, policy_version 778497 (0.0011) [2023-12-26 21:01:45,801][105692] Updated weights for policy 0, policy_version 778479 (0.0007) [2023-12-26 21:01:45,858][105692] Updated weights for policy 0, policy_version 778489 (0.0008) [2023-12-26 21:01:45,911][105692] Updated weights for policy 0, policy_version 778499 (0.0007) [2023-12-26 21:01:46,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 398647296. Throughput: 0: 9657.7, 1: 9925.8. Samples: 398612892. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-12-26 21:01:46,062][104569] Avg episode reward: [(0, '9265.249'), (1, '8687.680')] [2023-12-26 21:01:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000778504_199327744.pth... [2023-12-26 21:01:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000778504_199319552.pth... [2023-12-26 21:01:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000777320_199016448.pth [2023-12-26 21:01:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000777352_199032832.pth [2023-12-26 21:01:46,074][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_000778504_199319552.pth [2023-12-26 21:01:46,075][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_000778504_199327744.pth [2023-12-26 21:01:46,391][105620] Updated weights for policy 1, policy_version 778507 (0.0009) [2023-12-26 21:01:46,442][105620] Updated weights for policy 1, policy_version 778517 (0.0006) [2023-12-26 21:01:46,490][105620] Updated weights for policy 1, policy_version 778527 (0.0010) [2023-12-26 21:01:46,616][105692] Updated weights for policy 0, policy_version 778509 (0.0009) [2023-12-26 21:01:46,671][105692] Updated weights for policy 0, policy_version 778519 (0.0010) [2023-12-26 21:01:46,729][105692] Updated weights for policy 0, policy_version 778529 (0.0010) [2023-12-26 21:01:47,193][105620] Updated weights for policy 1, policy_version 778537 (0.0010) [2023-12-26 21:01:47,248][105620] Updated weights for policy 1, policy_version 778547 (0.0008) [2023-12-26 21:01:47,298][105620] Updated weights for policy 1, policy_version 778557 (0.0007) [2023-12-26 21:01:47,352][105620] Updated weights for policy 1, policy_version 778567 (0.0008) [2023-12-26 21:01:47,364][105692] Updated weights for policy 0, policy_version 778539 (0.0010) [2023-12-26 21:01:47,419][105692] Updated weights for policy 0, policy_version 778549 (0.0010) [2023-12-26 21:01:47,470][105692] Updated weights for policy 0, policy_version 778559 (0.0010) [2023-12-26 21:01:48,103][105620] Updated weights for policy 1, policy_version 778577 (0.0009) [2023-12-26 21:01:48,126][105692] Updated weights for policy 0, policy_version 778569 (0.0010) [2023-12-26 21:01:48,162][105620] Updated weights for policy 1, policy_version 778587 (0.0010) [2023-12-26 21:01:48,174][105692] Updated weights for policy 0, policy_version 778579 (0.0005) [2023-12-26 21:01:48,214][105620] Updated weights for policy 1, policy_version 778597 (0.0010) [2023-12-26 21:01:48,219][105692] Updated weights for policy 0, policy_version 778589 (0.0005) [2023-12-26 21:01:48,261][105692] Updated weights for policy 0, policy_version 778599 (0.0005) [2023-12-26 21:01:48,872][105692] Updated weights for policy 0, policy_version 778609 (0.0006) [2023-12-26 21:01:48,939][105620] Updated weights for policy 1, policy_version 778607 (0.0007) [2023-12-26 21:01:48,939][105692] Updated weights for policy 0, policy_version 778619 (0.0006) [2023-12-26 21:01:48,990][105620] Updated weights for policy 1, policy_version 778617 (0.0008) [2023-12-26 21:01:49,009][105692] Updated weights for policy 0, policy_version 778629 (0.0006) [2023-12-26 21:01:49,052][105620] Updated weights for policy 1, policy_version 778627 (0.0010) [2023-12-26 21:01:49,692][105692] Updated weights for policy 0, policy_version 778639 (0.0006) [2023-12-26 21:01:49,758][105692] Updated weights for policy 0, policy_version 778649 (0.0010) [2023-12-26 21:01:49,813][105620] Updated weights for policy 1, policy_version 778637 (0.0010) [2023-12-26 21:01:49,818][105692] Updated weights for policy 0, policy_version 778659 (0.0010) [2023-12-26 21:01:49,879][105620] Updated weights for policy 1, policy_version 778647 (0.0010) [2023-12-26 21:01:49,947][105620] Updated weights for policy 1, policy_version 778657 (0.0006) [2023-12-26 21:01:50,460][105692] Updated weights for policy 0, policy_version 778669 (0.0007) [2023-12-26 21:01:50,516][105692] Updated weights for policy 0, policy_version 778679 (0.0006) [2023-12-26 21:01:50,574][105692] Updated weights for policy 0, policy_version 778689 (0.0005) [2023-12-26 21:01:50,664][105620] Updated weights for policy 1, policy_version 778667 (0.0009) [2023-12-26 21:01:50,725][105620] Updated weights for policy 1, policy_version 778677 (0.0006) [2023-12-26 21:01:50,797][105620] Updated weights for policy 1, policy_version 778687 (0.0006) [2023-12-26 21:01:51,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 398745600. Throughput: 0: 9783.3, 1: 9896.2. Samples: 398733004. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-12-26 21:01:51,063][104569] Avg episode reward: [(0, '9264.598'), (1, '8986.262')] [2023-12-26 21:01:51,271][105692] Updated weights for policy 0, policy_version 778699 (0.0007) [2023-12-26 21:01:51,341][105692] Updated weights for policy 0, policy_version 778709 (0.0006) [2023-12-26 21:01:51,409][105692] Updated weights for policy 0, policy_version 778719 (0.0009) [2023-12-26 21:01:51,444][105620] Updated weights for policy 1, policy_version 778697 (0.0006) [2023-12-26 21:01:51,496][105620] Updated weights for policy 1, policy_version 778707 (0.0009) [2023-12-26 21:01:51,552][105620] Updated weights for policy 1, policy_version 778717 (0.0008) [2023-12-26 21:01:51,607][105620] Updated weights for policy 1, policy_version 778727 (0.0007) [2023-12-26 21:01:52,147][105692] Updated weights for policy 0, policy_version 778729 (0.0009) [2023-12-26 21:01:52,210][105692] Updated weights for policy 0, policy_version 778739 (0.0011) [2023-12-26 21:01:52,275][105692] Updated weights for policy 0, policy_version 778749 (0.0011) [2023-12-26 21:01:52,333][105692] Updated weights for policy 0, policy_version 778759 (0.0010) [2023-12-26 21:01:52,355][105620] Updated weights for policy 1, policy_version 778737 (0.0006) [2023-12-26 21:01:52,416][105620] Updated weights for policy 1, policy_version 778747 (0.0008) [2023-12-26 21:01:52,480][105620] Updated weights for policy 1, policy_version 778757 (0.0009) [2023-12-26 21:01:52,944][105692] Updated weights for policy 0, policy_version 778769 (0.0008) [2023-12-26 21:01:52,990][105692] Updated weights for policy 0, policy_version 778779 (0.0008) [2023-12-26 21:01:53,042][105692] Updated weights for policy 0, policy_version 778789 (0.0008) [2023-12-26 21:01:53,251][105620] Updated weights for policy 1, policy_version 778767 (0.0009) [2023-12-26 21:01:53,305][105620] Updated weights for policy 1, policy_version 778778 (0.0009) [2023-12-26 21:01:53,355][105620] Updated weights for policy 1, policy_version 778788 (0.0007) [2023-12-26 21:01:53,731][105692] Updated weights for policy 0, policy_version 778799 (0.0006) [2023-12-26 21:01:53,791][105692] Updated weights for policy 0, policy_version 778809 (0.0005) [2023-12-26 21:01:53,843][105692] Updated weights for policy 0, policy_version 778819 (0.0006) [2023-12-26 21:01:54,122][105620] Updated weights for policy 1, policy_version 778798 (0.0005) [2023-12-26 21:01:54,187][105620] Updated weights for policy 1, policy_version 778808 (0.0005) [2023-12-26 21:01:54,246][105620] Updated weights for policy 1, policy_version 778818 (0.0007) [2023-12-26 21:01:54,573][105692] Updated weights for policy 0, policy_version 778829 (0.0010) [2023-12-26 21:01:54,636][105692] Updated weights for policy 0, policy_version 778839 (0.0010) [2023-12-26 21:01:54,699][105692] Updated weights for policy 0, policy_version 778849 (0.0009) [2023-12-26 21:01:54,882][105620] Updated weights for policy 1, policy_version 778828 (0.0009) [2023-12-26 21:01:54,942][105620] Updated weights for policy 1, policy_version 778838 (0.0009) [2023-12-26 21:01:55,007][105620] Updated weights for policy 1, policy_version 778848 (0.0009) [2023-12-26 21:01:55,405][105692] Updated weights for policy 0, policy_version 778859 (0.0009) [2023-12-26 21:01:55,460][105692] Updated weights for policy 0, policy_version 778869 (0.0009) [2023-12-26 21:01:55,507][105692] Updated weights for policy 0, policy_version 778879 (0.0009) [2023-12-26 21:01:55,769][105620] Updated weights for policy 1, policy_version 778858 (0.0009) [2023-12-26 21:01:55,816][105620] Updated weights for policy 1, policy_version 778868 (0.0009) [2023-12-26 21:01:55,862][105620] Updated weights for policy 1, policy_version 778878 (0.0008) [2023-12-26 21:01:55,912][105620] Updated weights for policy 1, policy_version 778888 (0.0008) [2023-12-26 21:01:56,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19605.2). Total num frames: 398843904. Throughput: 0: 9829.5, 1: 9840.1. Samples: 398850008. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-12-26 21:01:56,063][104569] Avg episode reward: [(0, '9264.543'), (1, '9166.878')] [2023-12-26 21:01:56,293][105692] Updated weights for policy 0, policy_version 778889 (0.0009) [2023-12-26 21:01:56,347][105692] Updated weights for policy 0, policy_version 778899 (0.0008) [2023-12-26 21:01:56,404][105692] Updated weights for policy 0, policy_version 778909 (0.0010) [2023-12-26 21:01:56,463][105692] Updated weights for policy 0, policy_version 778919 (0.0011) [2023-12-26 21:01:56,650][105620] Updated weights for policy 1, policy_version 778898 (0.0005) [2023-12-26 21:01:56,712][105620] Updated weights for policy 1, policy_version 778908 (0.0006) [2023-12-26 21:01:56,773][105620] Updated weights for policy 1, policy_version 778918 (0.0006) [2023-12-26 21:01:57,234][105692] Updated weights for policy 0, policy_version 778929 (0.0010) [2023-12-26 21:01:57,284][105620] Updated weights for policy 1, policy_version 778928 (0.0006) [2023-12-26 21:01:57,293][105692] Updated weights for policy 0, policy_version 778939 (0.0010) [2023-12-26 21:01:57,335][105620] Updated weights for policy 1, policy_version 778938 (0.0006) [2023-12-26 21:01:57,348][105692] Updated weights for policy 0, policy_version 778949 (0.0010) [2023-12-26 21:01:57,402][105620] Updated weights for policy 1, policy_version 778948 (0.0005) [2023-12-26 21:01:58,005][105692] Updated weights for policy 0, policy_version 778959 (0.0009) [2023-12-26 21:01:58,052][105692] Updated weights for policy 0, policy_version 778969 (0.0007) [2023-12-26 21:01:58,072][105620] Updated weights for policy 1, policy_version 778958 (0.0008) [2023-12-26 21:01:58,074][105585] KL-divergence is very high: 140.6429 [2023-12-26 21:01:58,106][105692] Updated weights for policy 0, policy_version 778979 (0.0005) [2023-12-26 21:01:58,115][105585] KL-divergence is very high: 196.0605 [2023-12-26 21:01:58,127][105620] Updated weights for policy 1, policy_version 778968 (0.0010) [2023-12-26 21:01:58,194][105620] Updated weights for policy 1, policy_version 778978 (0.0008) [2023-12-26 21:01:58,932][105620] Updated weights for policy 1, policy_version 778989 (0.0007) [2023-12-26 21:01:58,977][105692] Updated weights for policy 0, policy_version 778989 (0.0009) [2023-12-26 21:01:58,995][105620] Updated weights for policy 1, policy_version 778999 (0.0009) [2023-12-26 21:01:59,038][105692] Updated weights for policy 0, policy_version 778999 (0.0008) [2023-12-26 21:01:59,052][105620] Updated weights for policy 1, policy_version 779009 (0.0008) [2023-12-26 21:01:59,101][105692] Updated weights for policy 0, policy_version 779009 (0.0006) [2023-12-26 21:01:59,819][105620] Updated weights for policy 1, policy_version 779019 (0.0008) [2023-12-26 21:01:59,827][105692] Updated weights for policy 0, policy_version 779019 (0.0009) [2023-12-26 21:01:59,885][105620] Updated weights for policy 1, policy_version 779029 (0.0007) [2023-12-26 21:01:59,894][105692] Updated weights for policy 0, policy_version 779029 (0.0007) [2023-12-26 21:01:59,947][105620] Updated weights for policy 1, policy_version 779039 (0.0008) [2023-12-26 21:01:59,952][105692] Updated weights for policy 0, policy_version 779039 (0.0007) [2023-12-26 21:02:00,609][105692] Updated weights for policy 0, policy_version 779049 (0.0008) [2023-12-26 21:02:00,671][105692] Updated weights for policy 0, policy_version 779059 (0.0009) [2023-12-26 21:02:00,732][105620] Updated weights for policy 1, policy_version 779049 (0.0007) [2023-12-26 21:02:00,733][105692] Updated weights for policy 0, policy_version 779069 (0.0009) [2023-12-26 21:02:00,780][105620] Updated weights for policy 1, policy_version 779059 (0.0006) [2023-12-26 21:02:00,790][105692] Updated weights for policy 0, policy_version 779079 (0.0008) [2023-12-26 21:02:00,828][105620] Updated weights for policy 1, policy_version 779069 (0.0007) [2023-12-26 21:02:00,874][105620] Updated weights for policy 1, policy_version 779079 (0.0009) [2023-12-26 21:02:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 398942208. Throughput: 0: 9827.1, 1: 9934.1. Samples: 398910372. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-12-26 21:02:01,062][104569] Avg episode reward: [(0, '9173.075'), (1, '9351.305')] [2023-12-26 21:02:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000779080_199475200.pth... [2023-12-26 21:02:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000779080_199467008.pth... [2023-12-26 21:02:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000777928_199180288.pth [2023-12-26 21:02:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000777928_199172096.pth [2023-12-26 21:02:01,560][105692] Updated weights for policy 0, policy_version 779089 (0.0010) [2023-12-26 21:02:01,599][105620] Updated weights for policy 1, policy_version 779089 (0.0006) [2023-12-26 21:02:01,628][105692] Updated weights for policy 0, policy_version 779099 (0.0008) [2023-12-26 21:02:01,672][105620] Updated weights for policy 1, policy_version 779099 (0.0008) [2023-12-26 21:02:01,688][105692] Updated weights for policy 0, policy_version 779109 (0.0008) [2023-12-26 21:02:01,737][105620] Updated weights for policy 1, policy_version 779109 (0.0008) [2023-12-26 21:02:02,302][105620] Updated weights for policy 1, policy_version 779119 (0.0006) [2023-12-26 21:02:02,363][105620] Updated weights for policy 1, policy_version 779129 (0.0008) [2023-12-26 21:02:02,369][105692] Updated weights for policy 0, policy_version 779119 (0.0008) [2023-12-26 21:02:02,416][105620] Updated weights for policy 1, policy_version 779139 (0.0008) [2023-12-26 21:02:02,431][105692] Updated weights for policy 0, policy_version 779129 (0.0006) [2023-12-26 21:02:02,489][105692] Updated weights for policy 0, policy_version 779139 (0.0007) [2023-12-26 21:02:03,071][105620] Updated weights for policy 1, policy_version 779149 (0.0009) [2023-12-26 21:02:03,119][105620] Updated weights for policy 1, policy_version 779159 (0.0009) [2023-12-26 21:02:03,175][105620] Updated weights for policy 1, policy_version 779169 (0.0010) [2023-12-26 21:02:03,232][105692] Updated weights for policy 0, policy_version 779149 (0.0007) [2023-12-26 21:02:03,280][105692] Updated weights for policy 0, policy_version 779159 (0.0008) [2023-12-26 21:02:03,328][105692] Updated weights for policy 0, policy_version 779169 (0.0008) [2023-12-26 21:02:03,936][105620] Updated weights for policy 1, policy_version 779179 (0.0011) [2023-12-26 21:02:03,992][105620] Updated weights for policy 1, policy_version 779189 (0.0011) [2023-12-26 21:02:04,051][105620] Updated weights for policy 1, policy_version 779199 (0.0011) [2023-12-26 21:02:04,103][105692] Updated weights for policy 0, policy_version 779179 (0.0008) [2023-12-26 21:02:04,163][105692] Updated weights for policy 0, policy_version 779189 (0.0006) [2023-12-26 21:02:04,228][105692] Updated weights for policy 0, policy_version 779199 (0.0006) [2023-12-26 21:02:04,820][105620] Updated weights for policy 1, policy_version 779209 (0.0011) [2023-12-26 21:02:04,832][105692] Updated weights for policy 0, policy_version 779209 (0.0006) [2023-12-26 21:02:04,882][105620] Updated weights for policy 1, policy_version 779219 (0.0009) [2023-12-26 21:02:04,901][105692] Updated weights for policy 0, policy_version 779219 (0.0006) [2023-12-26 21:02:04,903][105586] KL-divergence is very high: 215.2455 [2023-12-26 21:02:04,944][105620] Updated weights for policy 1, policy_version 779229 (0.0008) [2023-12-26 21:02:04,947][105586] KL-divergence is very high: 412.7172 [2023-12-26 21:02:04,961][105692] Updated weights for policy 0, policy_version 779229 (0.0005) [2023-12-26 21:02:04,988][105586] KL-divergence is very high: 454.0179 [2023-12-26 21:02:04,995][105620] Updated weights for policy 1, policy_version 779239 (0.0008) [2023-12-26 21:02:05,017][105692] Updated weights for policy 0, policy_version 779239 (0.0006) [2023-12-26 21:02:05,605][105692] Updated weights for policy 0, policy_version 779249 (0.0009) [2023-12-26 21:02:05,656][105692] Updated weights for policy 0, policy_version 779259 (0.0010) [2023-12-26 21:02:05,711][105692] Updated weights for policy 0, policy_version 779269 (0.0007) [2023-12-26 21:02:05,751][105586] KL-divergence is very high: 265.7743 [2023-12-26 21:02:05,777][105620] Updated weights for policy 1, policy_version 779249 (0.0009) [2023-12-26 21:02:05,808][105586] KL-divergence is very high: 283.8859 [2023-12-26 21:02:05,840][105620] Updated weights for policy 1, policy_version 779259 (0.0009) [2023-12-26 21:02:05,848][105586] KL-divergence is very high: 268.4433 [2023-12-26 21:02:05,891][105620] Updated weights for policy 1, policy_version 779269 (0.0006) [2023-12-26 21:02:05,892][105586] KL-divergence is very high: 259.4564 [2023-12-26 21:02:06,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 399040512. Throughput: 0: 9784.3, 1: 9988.8. Samples: 399025884. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-12-26 21:02:06,063][104569] Avg episode reward: [(0, '9172.271'), (1, '8853.890')] [2023-12-26 21:02:06,526][105692] Updated weights for policy 0, policy_version 779279 (0.0009) [2023-12-26 21:02:06,565][105620] Updated weights for policy 1, policy_version 779279 (0.0006) [2023-12-26 21:02:06,591][105692] Updated weights for policy 0, policy_version 779289 (0.0009) [2023-12-26 21:02:06,617][105586] KL-divergence is very high: 118.1642 [2023-12-26 21:02:06,622][105620] Updated weights for policy 1, policy_version 779289 (0.0007) [2023-12-26 21:02:06,637][105586] KL-divergence is very high: 133.4535 [2023-12-26 21:02:06,652][105692] Updated weights for policy 0, policy_version 779299 (0.0009) [2023-12-26 21:02:06,668][105586] KL-divergence is very high: 144.1045 [2023-12-26 21:02:06,686][105620] Updated weights for policy 1, policy_version 779299 (0.0006) [2023-12-26 21:02:06,688][105586] KL-divergence is very high: 137.1057 [2023-12-26 21:02:07,236][105620] Updated weights for policy 1, policy_version 779309 (0.0006) [2023-12-26 21:02:07,295][105620] Updated weights for policy 1, policy_version 779319 (0.0006) [2023-12-26 21:02:07,353][105620] Updated weights for policy 1, policy_version 779329 (0.0006) [2023-12-26 21:02:07,375][105692] Updated weights for policy 0, policy_version 779309 (0.0009) [2023-12-26 21:02:07,431][105692] Updated weights for policy 0, policy_version 779319 (0.0009) [2023-12-26 21:02:07,487][105692] Updated weights for policy 0, policy_version 779329 (0.0009) [2023-12-26 21:02:08,040][105620] Updated weights for policy 1, policy_version 779339 (0.0009) [2023-12-26 21:02:08,103][105620] Updated weights for policy 1, policy_version 779349 (0.0007) [2023-12-26 21:02:08,174][105620] Updated weights for policy 1, policy_version 779359 (0.0008) [2023-12-26 21:02:08,196][105692] Updated weights for policy 0, policy_version 779339 (0.0007) [2023-12-26 21:02:08,247][105692] Updated weights for policy 0, policy_version 779349 (0.0006) [2023-12-26 21:02:08,299][105692] Updated weights for policy 0, policy_version 779359 (0.0006) [2023-12-26 21:02:08,874][105620] Updated weights for policy 1, policy_version 779369 (0.0011) [2023-12-26 21:02:08,929][105620] Updated weights for policy 1, policy_version 779379 (0.0011) [2023-12-26 21:02:08,980][105692] Updated weights for policy 0, policy_version 779369 (0.0007) [2023-12-26 21:02:08,984][105620] Updated weights for policy 1, policy_version 779389 (0.0010) [2023-12-26 21:02:09,047][105692] Updated weights for policy 0, policy_version 779379 (0.0005) [2023-12-26 21:02:09,051][105620] Updated weights for policy 1, policy_version 779399 (0.0008) [2023-12-26 21:02:09,103][105692] Updated weights for policy 0, policy_version 779389 (0.0005) [2023-12-26 21:02:09,168][105692] Updated weights for policy 0, policy_version 779399 (0.0005) [2023-12-26 21:02:09,764][105620] Updated weights for policy 1, policy_version 779409 (0.0010) [2023-12-26 21:02:09,826][105620] Updated weights for policy 1, policy_version 779419 (0.0010) [2023-12-26 21:02:09,848][105692] Updated weights for policy 0, policy_version 779409 (0.0007) [2023-12-26 21:02:09,894][105620] Updated weights for policy 1, policy_version 779429 (0.0009) [2023-12-26 21:02:09,905][105692] Updated weights for policy 0, policy_version 779419 (0.0009) [2023-12-26 21:02:09,967][105692] Updated weights for policy 0, policy_version 779429 (0.0009) [2023-12-26 21:02:10,580][105620] Updated weights for policy 1, policy_version 779439 (0.0008) [2023-12-26 21:02:10,627][105620] Updated weights for policy 1, policy_version 779449 (0.0009) [2023-12-26 21:02:10,678][105620] Updated weights for policy 1, policy_version 779459 (0.0009) [2023-12-26 21:02:10,762][105692] Updated weights for policy 0, policy_version 779439 (0.0009) [2023-12-26 21:02:10,813][105692] Updated weights for policy 0, policy_version 779449 (0.0009) [2023-12-26 21:02:10,868][105692] Updated weights for policy 0, policy_version 779459 (0.0009) [2023-12-26 21:02:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 399138816. Throughput: 0: 9775.8, 1: 9988.6. Samples: 399144144. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-12-26 21:02:11,062][104569] Avg episode reward: [(0, '9353.310'), (1, '8762.658')] [2023-12-26 21:02:11,501][105620] Updated weights for policy 1, policy_version 779469 (0.0009) [2023-12-26 21:02:11,563][105620] Updated weights for policy 1, policy_version 779479 (0.0009) [2023-12-26 21:02:11,630][105620] Updated weights for policy 1, policy_version 779489 (0.0010) [2023-12-26 21:02:11,664][105692] Updated weights for policy 0, policy_version 779469 (0.0009) [2023-12-26 21:02:11,731][105692] Updated weights for policy 0, policy_version 779479 (0.0009) [2023-12-26 21:02:11,790][105692] Updated weights for policy 0, policy_version 779489 (0.0009) [2023-12-26 21:02:12,321][105620] Updated weights for policy 1, policy_version 779499 (0.0006) [2023-12-26 21:02:12,392][105620] Updated weights for policy 1, policy_version 779509 (0.0009) [2023-12-26 21:02:12,452][105620] Updated weights for policy 1, policy_version 779519 (0.0008) [2023-12-26 21:02:12,601][105692] Updated weights for policy 0, policy_version 779499 (0.0009) [2023-12-26 21:02:12,663][105692] Updated weights for policy 0, policy_version 779509 (0.0008) [2023-12-26 21:02:12,717][105692] Updated weights for policy 0, policy_version 779519 (0.0009) [2023-12-26 21:02:13,213][105620] Updated weights for policy 1, policy_version 779529 (0.0009) [2023-12-26 21:02:13,280][105620] Updated weights for policy 1, policy_version 779539 (0.0009) [2023-12-26 21:02:13,343][105620] Updated weights for policy 1, policy_version 779549 (0.0008) [2023-12-26 21:02:13,399][105620] Updated weights for policy 1, policy_version 779559 (0.0009) [2023-12-26 21:02:13,430][105692] Updated weights for policy 0, policy_version 779529 (0.0009) [2023-12-26 21:02:13,485][105692] Updated weights for policy 0, policy_version 779539 (0.0008) [2023-12-26 21:02:13,538][105692] Updated weights for policy 0, policy_version 779549 (0.0008) [2023-12-26 21:02:13,587][105692] Updated weights for policy 0, policy_version 779559 (0.0007) [2023-12-26 21:02:14,143][105620] Updated weights for policy 1, policy_version 779569 (0.0010) [2023-12-26 21:02:14,207][105620] Updated weights for policy 1, policy_version 779579 (0.0010) [2023-12-26 21:02:14,263][105620] Updated weights for policy 1, policy_version 779589 (0.0008) [2023-12-26 21:02:14,366][105692] Updated weights for policy 0, policy_version 779569 (0.0008) [2023-12-26 21:02:14,415][105692] Updated weights for policy 0, policy_version 779579 (0.0008) [2023-12-26 21:02:14,459][105692] Updated weights for policy 0, policy_version 779589 (0.0007) [2023-12-26 21:02:14,994][105620] Updated weights for policy 1, policy_version 779599 (0.0011) [2023-12-26 21:02:15,057][105620] Updated weights for policy 1, policy_version 779609 (0.0011) [2023-12-26 21:02:15,092][105692] Updated weights for policy 0, policy_version 779599 (0.0008) [2023-12-26 21:02:15,117][105620] Updated weights for policy 1, policy_version 779619 (0.0010) [2023-12-26 21:02:15,156][105692] Updated weights for policy 0, policy_version 779609 (0.0010) [2023-12-26 21:02:15,208][105692] Updated weights for policy 0, policy_version 779619 (0.0010) [2023-12-26 21:02:15,863][105620] Updated weights for policy 1, policy_version 779629 (0.0010) [2023-12-26 21:02:15,921][105620] Updated weights for policy 1, policy_version 779639 (0.0010) [2023-12-26 21:02:15,962][105692] Updated weights for policy 0, policy_version 779629 (0.0010) [2023-12-26 21:02:15,977][105620] Updated weights for policy 1, policy_version 779649 (0.0010) [2023-12-26 21:02:16,014][105692] Updated weights for policy 0, policy_version 779639 (0.0010) [2023-12-26 21:02:16,058][105692] Updated weights for policy 0, policy_version 779649 (0.0010) [2023-12-26 21:02:16,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 399228928. Throughput: 0: 9695.5, 1: 9874.7. Samples: 399199512. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-12-26 21:02:16,063][104569] Avg episode reward: [(0, '9261.956'), (1, '9078.646')] [2023-12-26 21:02:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000779656_199614464.pth... [2023-12-26 21:02:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000778504_199319552.pth [2023-12-26 21:02:16,089][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000779656_199622656.pth... [2023-12-26 21:02:16,092][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000778504_199327744.pth [2023-12-26 21:02:16,645][105620] Updated weights for policy 1, policy_version 779659 (0.0009) [2023-12-26 21:02:16,698][105620] Updated weights for policy 1, policy_version 779669 (0.0010) [2023-12-26 21:02:16,750][105620] Updated weights for policy 1, policy_version 779679 (0.0010) [2023-12-26 21:02:16,811][105692] Updated weights for policy 0, policy_version 779659 (0.0010) [2023-12-26 21:02:16,860][105692] Updated weights for policy 0, policy_version 779669 (0.0008) [2023-12-26 21:02:16,911][105692] Updated weights for policy 0, policy_version 779679 (0.0009) [2023-12-26 21:02:17,362][105620] Updated weights for policy 1, policy_version 779689 (0.0011) [2023-12-26 21:02:17,410][105620] Updated weights for policy 1, policy_version 779699 (0.0006) [2023-12-26 21:02:17,464][105620] Updated weights for policy 1, policy_version 779709 (0.0005) [2023-12-26 21:02:17,525][105620] Updated weights for policy 1, policy_version 779719 (0.0005) [2023-12-26 21:02:17,551][105692] Updated weights for policy 0, policy_version 779689 (0.0008) [2023-12-26 21:02:17,606][105692] Updated weights for policy 0, policy_version 779699 (0.0005) [2023-12-26 21:02:17,665][105692] Updated weights for policy 0, policy_version 779709 (0.0005) [2023-12-26 21:02:17,722][105692] Updated weights for policy 0, policy_version 779719 (0.0005) [2023-12-26 21:02:18,071][105620] Updated weights for policy 1, policy_version 779729 (0.0010) [2023-12-26 21:02:18,120][105620] Updated weights for policy 1, policy_version 779739 (0.0010) [2023-12-26 21:02:18,176][105620] Updated weights for policy 1, policy_version 779749 (0.0010) [2023-12-26 21:02:18,270][105692] Updated weights for policy 0, policy_version 779729 (0.0009) [2023-12-26 21:02:18,334][105692] Updated weights for policy 0, policy_version 779739 (0.0011) [2023-12-26 21:02:18,400][105692] Updated weights for policy 0, policy_version 779749 (0.0010) [2023-12-26 21:02:18,903][105620] Updated weights for policy 1, policy_version 779759 (0.0009) [2023-12-26 21:02:18,952][105620] Updated weights for policy 1, policy_version 779769 (0.0010) [2023-12-26 21:02:19,007][105620] Updated weights for policy 1, policy_version 779779 (0.0010) [2023-12-26 21:02:19,156][105692] Updated weights for policy 0, policy_version 779759 (0.0007) [2023-12-26 21:02:19,203][105692] Updated weights for policy 0, policy_version 779769 (0.0007) [2023-12-26 21:02:19,266][105692] Updated weights for policy 0, policy_version 779779 (0.0009) [2023-12-26 21:02:19,795][105620] Updated weights for policy 1, policy_version 779789 (0.0010) [2023-12-26 21:02:19,864][105620] Updated weights for policy 1, policy_version 779799 (0.0009) [2023-12-26 21:02:19,915][105620] Updated weights for policy 1, policy_version 779809 (0.0010) [2023-12-26 21:02:19,920][105692] Updated weights for policy 0, policy_version 779789 (0.0007) [2023-12-26 21:02:19,989][105692] Updated weights for policy 0, policy_version 779799 (0.0009) [2023-12-26 21:02:20,053][105692] Updated weights for policy 0, policy_version 779809 (0.0009) [2023-12-26 21:02:20,624][105620] Updated weights for policy 1, policy_version 779820 (0.0010) [2023-12-26 21:02:20,682][105620] Updated weights for policy 1, policy_version 779830 (0.0011) [2023-12-26 21:02:20,742][105620] Updated weights for policy 1, policy_version 779840 (0.0010) [2023-12-26 21:02:20,777][105692] Updated weights for policy 0, policy_version 779819 (0.0008) [2023-12-26 21:02:20,842][105692] Updated weights for policy 0, policy_version 779829 (0.0008) [2023-12-26 21:02:20,902][105692] Updated weights for policy 0, policy_version 779839 (0.0008) [2023-12-26 21:02:21,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 399335424. Throughput: 0: 9762.0, 1: 9918.4. Samples: 399320636. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-12-26 21:02:21,063][104569] Avg episode reward: [(0, '9261.039'), (1, '8987.871')] [2023-12-26 21:02:21,515][105620] Updated weights for policy 1, policy_version 779850 (0.0010) [2023-12-26 21:02:21,584][105620] Updated weights for policy 1, policy_version 779860 (0.0008) [2023-12-26 21:02:21,620][105692] Updated weights for policy 0, policy_version 779849 (0.0009) [2023-12-26 21:02:21,650][105620] Updated weights for policy 1, policy_version 779870 (0.0011) [2023-12-26 21:02:21,686][105692] Updated weights for policy 0, policy_version 779859 (0.0008) [2023-12-26 21:02:21,714][105620] Updated weights for policy 1, policy_version 779880 (0.0010) [2023-12-26 21:02:21,754][105692] Updated weights for policy 0, policy_version 779869 (0.0009) [2023-12-26 21:02:21,816][105692] Updated weights for policy 0, policy_version 779879 (0.0009) [2023-12-26 21:02:22,288][105620] Updated weights for policy 1, policy_version 779890 (0.0010) [2023-12-26 21:02:22,351][105620] Updated weights for policy 1, policy_version 779900 (0.0010) [2023-12-26 21:02:22,418][105620] Updated weights for policy 1, policy_version 779910 (0.0009) [2023-12-26 21:02:22,652][105692] Updated weights for policy 0, policy_version 779889 (0.0008) [2023-12-26 21:02:22,709][105692] Updated weights for policy 0, policy_version 779899 (0.0009) [2023-12-26 21:02:22,761][105692] Updated weights for policy 0, policy_version 779909 (0.0008) [2023-12-26 21:02:23,127][105620] Updated weights for policy 1, policy_version 779920 (0.0009) [2023-12-26 21:02:23,187][105620] Updated weights for policy 1, policy_version 779930 (0.0005) [2023-12-26 21:02:23,261][105620] Updated weights for policy 1, policy_version 779940 (0.0005) [2023-12-26 21:02:23,568][105692] Updated weights for policy 0, policy_version 779919 (0.0009) [2023-12-26 21:02:23,624][105692] Updated weights for policy 0, policy_version 779930 (0.0009) [2023-12-26 21:02:23,682][105692] Updated weights for policy 0, policy_version 779940 (0.0009) [2023-12-26 21:02:23,770][105620] Updated weights for policy 1, policy_version 779950 (0.0005) [2023-12-26 21:02:23,827][105620] Updated weights for policy 1, policy_version 779960 (0.0005) [2023-12-26 21:02:23,893][105620] Updated weights for policy 1, policy_version 779970 (0.0005) [2023-12-26 21:02:24,397][105692] Updated weights for policy 0, policy_version 779950 (0.0007) [2023-12-26 21:02:24,452][105692] Updated weights for policy 0, policy_version 779960 (0.0008) [2023-12-26 21:02:24,503][105620] Updated weights for policy 1, policy_version 779980 (0.0007) [2023-12-26 21:02:24,506][105692] Updated weights for policy 0, policy_version 779970 (0.0008) [2023-12-26 21:02:24,547][105620] Updated weights for policy 1, policy_version 779990 (0.0010) [2023-12-26 21:02:24,592][105620] Updated weights for policy 1, policy_version 780000 (0.0010) [2023-12-26 21:02:25,230][105620] Updated weights for policy 1, policy_version 780010 (0.0010) [2023-12-26 21:02:25,263][105692] Updated weights for policy 0, policy_version 779980 (0.0006) [2023-12-26 21:02:25,285][105620] Updated weights for policy 1, policy_version 780020 (0.0010) [2023-12-26 21:02:25,315][105692] Updated weights for policy 0, policy_version 779990 (0.0005) [2023-12-26 21:02:25,333][105620] Updated weights for policy 1, policy_version 780030 (0.0010) [2023-12-26 21:02:25,368][105692] Updated weights for policy 0, policy_version 780000 (0.0005) [2023-12-26 21:02:25,381][105620] Updated weights for policy 1, policy_version 780040 (0.0010) [2023-12-26 21:02:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 399425536. Throughput: 0: 9731.6, 1: 9942.2. Samples: 399438688. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-12-26 21:02:26,062][104569] Avg episode reward: [(0, '9260.015'), (1, '9169.324')] [2023-12-26 21:02:26,067][105692] Updated weights for policy 0, policy_version 780010 (0.0007) [2023-12-26 21:02:26,113][105692] Updated weights for policy 0, policy_version 780020 (0.0009) [2023-12-26 21:02:26,152][105620] Updated weights for policy 1, policy_version 780050 (0.0007) [2023-12-26 21:02:26,168][105692] Updated weights for policy 0, policy_version 780030 (0.0007) [2023-12-26 21:02:26,208][105620] Updated weights for policy 1, policy_version 780060 (0.0005) [2023-12-26 21:02:26,220][105692] Updated weights for policy 0, policy_version 780040 (0.0008) [2023-12-26 21:02:26,262][105620] Updated weights for policy 1, policy_version 780070 (0.0007) [2023-12-26 21:02:26,924][105692] Updated weights for policy 0, policy_version 780050 (0.0009) [2023-12-26 21:02:26,970][105692] Updated weights for policy 0, policy_version 780060 (0.0008) [2023-12-26 21:02:26,994][105620] Updated weights for policy 1, policy_version 780080 (0.0007) [2023-12-26 21:02:27,016][105692] Updated weights for policy 0, policy_version 780070 (0.0008) [2023-12-26 21:02:27,042][105620] Updated weights for policy 1, policy_version 780090 (0.0008) [2023-12-26 21:02:27,088][105620] Updated weights for policy 1, policy_version 780100 (0.0009) [2023-12-26 21:02:27,792][105620] Updated weights for policy 1, policy_version 780110 (0.0007) [2023-12-26 21:02:27,830][105692] Updated weights for policy 0, policy_version 780080 (0.0008) [2023-12-26 21:02:27,853][105620] Updated weights for policy 1, policy_version 780120 (0.0009) [2023-12-26 21:02:27,879][105692] Updated weights for policy 0, policy_version 780090 (0.0008) [2023-12-26 21:02:27,908][105620] Updated weights for policy 1, policy_version 780130 (0.0009) [2023-12-26 21:02:27,928][105692] Updated weights for policy 0, policy_version 780100 (0.0008) [2023-12-26 21:02:28,523][105692] Updated weights for policy 0, policy_version 780110 (0.0009) [2023-12-26 21:02:28,580][105692] Updated weights for policy 0, policy_version 780120 (0.0008) [2023-12-26 21:02:28,638][105692] Updated weights for policy 0, policy_version 780130 (0.0005) [2023-12-26 21:02:28,642][105620] Updated weights for policy 1, policy_version 780140 (0.0009) [2023-12-26 21:02:28,696][105620] Updated weights for policy 1, policy_version 780150 (0.0009) [2023-12-26 21:02:28,754][105620] Updated weights for policy 1, policy_version 780160 (0.0009) [2023-12-26 21:02:29,320][105692] Updated weights for policy 0, policy_version 780140 (0.0006) [2023-12-26 21:02:29,379][105692] Updated weights for policy 0, policy_version 780150 (0.0009) [2023-12-26 21:02:29,438][105692] Updated weights for policy 0, policy_version 780160 (0.0010) [2023-12-26 21:02:29,549][105620] Updated weights for policy 1, policy_version 780170 (0.0009) [2023-12-26 21:02:29,606][105620] Updated weights for policy 1, policy_version 780180 (0.0009) [2023-12-26 21:02:29,671][105620] Updated weights for policy 1, policy_version 780190 (0.0009) [2023-12-26 21:02:29,724][105620] Updated weights for policy 1, policy_version 780200 (0.0008) [2023-12-26 21:02:30,145][105692] Updated weights for policy 0, policy_version 780170 (0.0008) [2023-12-26 21:02:30,194][105692] Updated weights for policy 0, policy_version 780180 (0.0005) [2023-12-26 21:02:30,241][105692] Updated weights for policy 0, policy_version 780190 (0.0006) [2023-12-26 21:02:30,290][105692] Updated weights for policy 0, policy_version 780200 (0.0005) [2023-12-26 21:02:30,454][105620] Updated weights for policy 1, policy_version 780210 (0.0007) [2023-12-26 21:02:30,517][105620] Updated weights for policy 1, policy_version 780220 (0.0008) [2023-12-26 21:02:30,573][105620] Updated weights for policy 1, policy_version 780230 (0.0009) [2023-12-26 21:02:30,985][105692] Updated weights for policy 0, policy_version 780210 (0.0009) [2023-12-26 21:02:31,049][105692] Updated weights for policy 0, policy_version 780220 (0.0008) [2023-12-26 21:02:31,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 399523840. Throughput: 0: 9781.1, 1: 9874.9. Samples: 399497408. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-12-26 21:02:31,063][104569] Avg episode reward: [(0, '9260.015'), (1, '9168.319')] [2023-12-26 21:02:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000780232_199761920.pth... [2023-12-26 21:02:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000779080_199467008.pth [2023-12-26 21:02:31,108][105692] Updated weights for policy 0, policy_version 780230 (0.0009) [2023-12-26 21:02:31,118][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000780232_199770112.pth... [2023-12-26 21:02:31,123][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000779080_199475200.pth [2023-12-26 21:02:31,307][105620] Updated weights for policy 1, policy_version 780240 (0.0008) [2023-12-26 21:02:31,369][105620] Updated weights for policy 1, policy_version 780250 (0.0006) [2023-12-26 21:02:31,435][105620] Updated weights for policy 1, policy_version 780260 (0.0009) [2023-12-26 21:02:31,912][105692] Updated weights for policy 0, policy_version 780240 (0.0009) [2023-12-26 21:02:31,971][105692] Updated weights for policy 0, policy_version 780250 (0.0009) [2023-12-26 21:02:32,032][105692] Updated weights for policy 0, policy_version 780260 (0.0010) [2023-12-26 21:02:32,126][105620] Updated weights for policy 1, policy_version 780270 (0.0009) [2023-12-26 21:02:32,180][105620] Updated weights for policy 1, policy_version 780280 (0.0007) [2023-12-26 21:02:32,237][105620] Updated weights for policy 1, policy_version 780290 (0.0009) [2023-12-26 21:02:32,793][105692] Updated weights for policy 0, policy_version 780270 (0.0009) [2023-12-26 21:02:32,844][105692] Updated weights for policy 0, policy_version 780280 (0.0009) [2023-12-26 21:02:32,896][105692] Updated weights for policy 0, policy_version 780291 (0.0009) [2023-12-26 21:02:32,941][105620] Updated weights for policy 1, policy_version 780300 (0.0008) [2023-12-26 21:02:33,003][105620] Updated weights for policy 1, policy_version 780310 (0.0009) [2023-12-26 21:02:33,050][105620] Updated weights for policy 1, policy_version 780320 (0.0008) [2023-12-26 21:02:33,670][105620] Updated weights for policy 1, policy_version 780330 (0.0006) [2023-12-26 21:02:33,727][105692] Updated weights for policy 0, policy_version 780301 (0.0008) [2023-12-26 21:02:33,731][105620] Updated weights for policy 1, policy_version 780340 (0.0005) [2023-12-26 21:02:33,779][105620] Updated weights for policy 1, policy_version 780350 (0.0005) [2023-12-26 21:02:33,782][105692] Updated weights for policy 0, policy_version 780311 (0.0008) [2023-12-26 21:02:33,831][105620] Updated weights for policy 1, policy_version 780360 (0.0005) [2023-12-26 21:02:33,831][105692] Updated weights for policy 0, policy_version 780321 (0.0009) [2023-12-26 21:02:34,445][105620] Updated weights for policy 1, policy_version 780370 (0.0009) [2023-12-26 21:02:34,499][105620] Updated weights for policy 1, policy_version 780380 (0.0007) [2023-12-26 21:02:34,563][105620] Updated weights for policy 1, policy_version 780390 (0.0006) [2023-12-26 21:02:34,628][105692] Updated weights for policy 0, policy_version 780331 (0.0009) [2023-12-26 21:02:34,694][105692] Updated weights for policy 0, policy_version 780341 (0.0008) [2023-12-26 21:02:34,758][105692] Updated weights for policy 0, policy_version 780351 (0.0005) [2023-12-26 21:02:35,245][105620] Updated weights for policy 1, policy_version 780400 (0.0007) [2023-12-26 21:02:35,309][105620] Updated weights for policy 1, policy_version 780410 (0.0008) [2023-12-26 21:02:35,379][105620] Updated weights for policy 1, policy_version 780420 (0.0007) [2023-12-26 21:02:35,540][105692] Updated weights for policy 0, policy_version 780361 (0.0009) [2023-12-26 21:02:35,607][105692] Updated weights for policy 0, policy_version 780371 (0.0010) [2023-12-26 21:02:35,663][105692] Updated weights for policy 0, policy_version 780381 (0.0009) [2023-12-26 21:02:35,711][105692] Updated weights for policy 0, policy_version 780391 (0.0009) [2023-12-26 21:02:36,018][105620] Updated weights for policy 1, policy_version 780430 (0.0008) [2023-12-26 21:02:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 399622144. Throughput: 0: 9665.9, 1: 9919.4. Samples: 399614340. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-12-26 21:02:36,062][104569] Avg episode reward: [(0, '9260.059'), (1, '9080.817')] [2023-12-26 21:02:36,072][105620] Updated weights for policy 1, policy_version 780440 (0.0009) [2023-12-26 21:02:36,143][105620] Updated weights for policy 1, policy_version 780450 (0.0009) [2023-12-26 21:02:36,415][105692] Updated weights for policy 0, policy_version 780401 (0.0006) [2023-12-26 21:02:36,467][105692] Updated weights for policy 0, policy_version 780411 (0.0006) [2023-12-26 21:02:36,522][105692] Updated weights for policy 0, policy_version 780421 (0.0006) [2023-12-26 21:02:36,934][105620] Updated weights for policy 1, policy_version 780460 (0.0008) [2023-12-26 21:02:36,994][105620] Updated weights for policy 1, policy_version 780470 (0.0006) [2023-12-26 21:02:37,063][105620] Updated weights for policy 1, policy_version 780480 (0.0006) [2023-12-26 21:02:37,298][105692] Updated weights for policy 0, policy_version 780431 (0.0008) [2023-12-26 21:02:37,362][105692] Updated weights for policy 0, policy_version 780441 (0.0009) [2023-12-26 21:02:37,391][105585] KL-divergence is very high: 106.2759 [2023-12-26 21:02:37,412][105692] Updated weights for policy 0, policy_version 780451 (0.0009) [2023-12-26 21:02:37,437][105585] KL-divergence is very high: 101.7854 [2023-12-26 21:02:37,635][105620] Updated weights for policy 1, policy_version 780490 (0.0006) [2023-12-26 21:02:37,695][105620] Updated weights for policy 1, policy_version 780500 (0.0008) [2023-12-26 21:02:37,762][105620] Updated weights for policy 1, policy_version 780510 (0.0008) [2023-12-26 21:02:37,823][105620] Updated weights for policy 1, policy_version 780520 (0.0007) [2023-12-26 21:02:38,191][105692] Updated weights for policy 0, policy_version 780461 (0.0010) [2023-12-26 21:02:38,236][105692] Updated weights for policy 0, policy_version 780471 (0.0010) [2023-12-26 21:02:38,287][105692] Updated weights for policy 0, policy_version 780481 (0.0010) [2023-12-26 21:02:38,527][105620] Updated weights for policy 1, policy_version 780530 (0.0008) [2023-12-26 21:02:38,598][105620] Updated weights for policy 1, policy_version 780540 (0.0006) [2023-12-26 21:02:38,653][105620] Updated weights for policy 1, policy_version 780550 (0.0006) [2023-12-26 21:02:39,048][105692] Updated weights for policy 0, policy_version 780491 (0.0010) [2023-12-26 21:02:39,099][105692] Updated weights for policy 0, policy_version 780501 (0.0010) [2023-12-26 21:02:39,154][105692] Updated weights for policy 0, policy_version 780511 (0.0010) [2023-12-26 21:02:39,381][105620] Updated weights for policy 1, policy_version 780560 (0.0008) [2023-12-26 21:02:39,445][105620] Updated weights for policy 1, policy_version 780570 (0.0008) [2023-12-26 21:02:39,500][105620] Updated weights for policy 1, policy_version 780580 (0.0008) [2023-12-26 21:02:39,955][105692] Updated weights for policy 0, policy_version 780521 (0.0010) [2023-12-26 21:02:40,021][105692] Updated weights for policy 0, policy_version 780531 (0.0005) [2023-12-26 21:02:40,085][105692] Updated weights for policy 0, policy_version 780541 (0.0008) [2023-12-26 21:02:40,147][105692] Updated weights for policy 0, policy_version 780551 (0.0008) [2023-12-26 21:02:40,282][105620] Updated weights for policy 1, policy_version 780590 (0.0008) [2023-12-26 21:02:40,343][105620] Updated weights for policy 1, policy_version 780600 (0.0010) [2023-12-26 21:02:40,400][105620] Updated weights for policy 1, policy_version 780610 (0.0005) [2023-12-26 21:02:40,829][105692] Updated weights for policy 0, policy_version 780561 (0.0009) [2023-12-26 21:02:40,877][105692] Updated weights for policy 0, policy_version 780571 (0.0010) [2023-12-26 21:02:40,925][105692] Updated weights for policy 0, policy_version 780581 (0.0010) [2023-12-26 21:02:41,062][105620] Updated weights for policy 1, policy_version 780620 (0.0008) [2023-12-26 21:02:41,063][104569] Fps is (10 sec: 19659.1, 60 sec: 19524.0, 300 sec: 19549.7). Total num frames: 399720448. Throughput: 0: 9568.3, 1: 9954.4. Samples: 399728544. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-12-26 21:02:41,063][104569] Avg episode reward: [(0, '9260.643'), (1, '9174.945')] [2023-12-26 21:02:41,116][105620] Updated weights for policy 1, policy_version 780630 (0.0008) [2023-12-26 21:02:41,171][105620] Updated weights for policy 1, policy_version 780640 (0.0008) [2023-12-26 21:02:41,677][105692] Updated weights for policy 0, policy_version 780591 (0.0011) [2023-12-26 21:02:41,745][105692] Updated weights for policy 0, policy_version 780601 (0.0010) [2023-12-26 21:02:41,812][105692] Updated weights for policy 0, policy_version 780611 (0.0011) [2023-12-26 21:02:41,888][105620] Updated weights for policy 1, policy_version 780650 (0.0009) [2023-12-26 21:02:41,951][105620] Updated weights for policy 1, policy_version 780660 (0.0010) [2023-12-26 21:02:42,011][105620] Updated weights for policy 1, policy_version 780670 (0.0011) [2023-12-26 21:02:42,081][105620] Updated weights for policy 1, policy_version 780680 (0.0009) [2023-12-26 21:02:42,504][105692] Updated weights for policy 0, policy_version 780621 (0.0011) [2023-12-26 21:02:42,567][105692] Updated weights for policy 0, policy_version 780631 (0.0009) [2023-12-26 21:02:42,627][105692] Updated weights for policy 0, policy_version 780641 (0.0011) [2023-12-26 21:02:42,827][105620] Updated weights for policy 1, policy_version 780690 (0.0008) [2023-12-26 21:02:42,882][105620] Updated weights for policy 1, policy_version 780700 (0.0007) [2023-12-26 21:02:42,937][105620] Updated weights for policy 1, policy_version 780710 (0.0008) [2023-12-26 21:02:43,268][105692] Updated weights for policy 0, policy_version 780651 (0.0009) [2023-12-26 21:02:43,332][105692] Updated weights for policy 0, policy_version 780661 (0.0007) [2023-12-26 21:02:43,388][105692] Updated weights for policy 0, policy_version 780671 (0.0008) [2023-12-26 21:02:43,723][105620] Updated weights for policy 1, policy_version 780720 (0.0010) [2023-12-26 21:02:43,781][105620] Updated weights for policy 1, policy_version 780730 (0.0008) [2023-12-26 21:02:43,836][105620] Updated weights for policy 1, policy_version 780740 (0.0009) [2023-12-26 21:02:44,019][105692] Updated weights for policy 0, policy_version 780681 (0.0006) [2023-12-26 21:02:44,071][105692] Updated weights for policy 0, policy_version 780691 (0.0009) [2023-12-26 21:02:44,118][105692] Updated weights for policy 0, policy_version 780701 (0.0009) [2023-12-26 21:02:44,171][105692] Updated weights for policy 0, policy_version 780711 (0.0009) [2023-12-26 21:02:44,544][105620] Updated weights for policy 1, policy_version 780750 (0.0009) [2023-12-26 21:02:44,589][105620] Updated weights for policy 1, policy_version 780760 (0.0008) [2023-12-26 21:02:44,636][105620] Updated weights for policy 1, policy_version 780770 (0.0008) [2023-12-26 21:02:44,963][105692] Updated weights for policy 0, policy_version 780721 (0.0010) [2023-12-26 21:02:45,026][105692] Updated weights for policy 0, policy_version 780731 (0.0009) [2023-12-26 21:02:45,088][105692] Updated weights for policy 0, policy_version 780741 (0.0009) [2023-12-26 21:02:45,377][105620] Updated weights for policy 1, policy_version 780780 (0.0009) [2023-12-26 21:02:45,442][105620] Updated weights for policy 1, policy_version 780790 (0.0008) [2023-12-26 21:02:45,501][105620] Updated weights for policy 1, policy_version 780800 (0.0008) [2023-12-26 21:02:45,895][105692] Updated weights for policy 0, policy_version 780751 (0.0009) [2023-12-26 21:02:45,955][105692] Updated weights for policy 0, policy_version 780761 (0.0009) [2023-12-26 21:02:46,027][105692] Updated weights for policy 0, policy_version 780771 (0.0009) [2023-12-26 21:02:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 399818752. Throughput: 0: 9605.6, 1: 9860.4. Samples: 399786340. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-12-26 21:02:46,062][104569] Avg episode reward: [(0, '9261.621'), (1, '9084.302')] [2023-12-26 21:02:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000780776_199909376.pth... [2023-12-26 21:02:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000780808_199909376.pth... [2023-12-26 21:02:46,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000779656_199622656.pth [2023-12-26 21:02:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000779656_199614464.pth [2023-12-26 21:02:46,178][105620] Updated weights for policy 1, policy_version 780810 (0.0009) [2023-12-26 21:02:46,247][105620] Updated weights for policy 1, policy_version 780820 (0.0008) [2023-12-26 21:02:46,311][105620] Updated weights for policy 1, policy_version 780830 (0.0009) [2023-12-26 21:02:46,369][105620] Updated weights for policy 1, policy_version 780840 (0.0006) [2023-12-26 21:02:46,794][105692] Updated weights for policy 0, policy_version 780781 (0.0009) [2023-12-26 21:02:46,847][105692] Updated weights for policy 0, policy_version 780791 (0.0009) [2023-12-26 21:02:46,900][105692] Updated weights for policy 0, policy_version 780801 (0.0009) [2023-12-26 21:02:47,013][105620] Updated weights for policy 1, policy_version 780850 (0.0007) [2023-12-26 21:02:47,070][105620] Updated weights for policy 1, policy_version 780860 (0.0009) [2023-12-26 21:02:47,118][105620] Updated weights for policy 1, policy_version 780870 (0.0009) [2023-12-26 21:02:47,707][105692] Updated weights for policy 0, policy_version 780811 (0.0009) [2023-12-26 21:02:47,757][105692] Updated weights for policy 0, policy_version 780821 (0.0009) [2023-12-26 21:02:47,785][105620] Updated weights for policy 1, policy_version 780880 (0.0006) [2023-12-26 21:02:47,810][105692] Updated weights for policy 0, policy_version 780831 (0.0007) [2023-12-26 21:02:47,841][105620] Updated weights for policy 1, policy_version 780890 (0.0007) [2023-12-26 21:02:47,886][105620] Updated weights for policy 1, policy_version 780900 (0.0008) [2023-12-26 21:02:48,590][105620] Updated weights for policy 1, policy_version 780910 (0.0006) [2023-12-26 21:02:48,624][105692] Updated weights for policy 0, policy_version 780841 (0.0007) [2023-12-26 21:02:48,654][105620] Updated weights for policy 1, policy_version 780920 (0.0008) [2023-12-26 21:02:48,691][105692] Updated weights for policy 0, policy_version 780851 (0.0007) [2023-12-26 21:02:48,717][105620] Updated weights for policy 1, policy_version 780930 (0.0008) [2023-12-26 21:02:48,744][105692] Updated weights for policy 0, policy_version 780861 (0.0006) [2023-12-26 21:02:48,807][105692] Updated weights for policy 0, policy_version 780871 (0.0008) [2023-12-26 21:02:49,421][105620] Updated weights for policy 1, policy_version 780940 (0.0008) [2023-12-26 21:02:49,469][105620] Updated weights for policy 1, policy_version 780950 (0.0008) [2023-12-26 21:02:49,517][105620] Updated weights for policy 1, policy_version 780960 (0.0008) [2023-12-26 21:02:49,569][105692] Updated weights for policy 0, policy_version 780881 (0.0006) [2023-12-26 21:02:49,622][105692] Updated weights for policy 0, policy_version 780891 (0.0009) [2023-12-26 21:02:49,680][105692] Updated weights for policy 0, policy_version 780901 (0.0005) [2023-12-26 21:02:50,342][105620] Updated weights for policy 1, policy_version 780970 (0.0008) [2023-12-26 21:02:50,380][105692] Updated weights for policy 0, policy_version 780911 (0.0007) [2023-12-26 21:02:50,395][105620] Updated weights for policy 1, policy_version 780980 (0.0007) [2023-12-26 21:02:50,441][105692] Updated weights for policy 0, policy_version 780921 (0.0008) [2023-12-26 21:02:50,451][105620] Updated weights for policy 1, policy_version 780990 (0.0006) [2023-12-26 21:02:50,500][105692] Updated weights for policy 0, policy_version 780931 (0.0007) [2023-12-26 21:02:50,502][105620] Updated weights for policy 1, policy_version 781000 (0.0008) [2023-12-26 21:02:51,062][104569] Fps is (10 sec: 18843.0, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 399908864. Throughput: 0: 9534.7, 1: 9893.5. Samples: 399900156. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-12-26 21:02:51,062][104569] Avg episode reward: [(0, '9170.269'), (1, '9085.517')] [2023-12-26 21:02:51,198][105692] Updated weights for policy 0, policy_version 780941 (0.0009) [2023-12-26 21:02:51,259][105692] Updated weights for policy 0, policy_version 780951 (0.0008) [2023-12-26 21:02:51,307][105692] Updated weights for policy 0, policy_version 780961 (0.0007) [2023-12-26 21:02:51,314][105620] Updated weights for policy 1, policy_version 781010 (0.0007) [2023-12-26 21:02:51,380][105620] Updated weights for policy 1, policy_version 781020 (0.0007) [2023-12-26 21:02:51,446][105620] Updated weights for policy 1, policy_version 781030 (0.0008) [2023-12-26 21:02:52,118][105620] Updated weights for policy 1, policy_version 781040 (0.0006) [2023-12-26 21:02:52,182][105620] Updated weights for policy 1, policy_version 781050 (0.0009) [2023-12-26 21:02:52,209][105692] Updated weights for policy 0, policy_version 780971 (0.0009) [2023-12-26 21:02:52,239][105620] Updated weights for policy 1, policy_version 781060 (0.0006) [2023-12-26 21:02:52,265][105692] Updated weights for policy 0, policy_version 780981 (0.0009) [2023-12-26 21:02:52,316][105692] Updated weights for policy 0, policy_version 780991 (0.0008) [2023-12-26 21:02:52,983][105620] Updated weights for policy 1, policy_version 781070 (0.0006) [2023-12-26 21:02:53,038][105620] Updated weights for policy 1, policy_version 781080 (0.0006) [2023-12-26 21:02:53,045][105692] Updated weights for policy 0, policy_version 781001 (0.0009) [2023-12-26 21:02:53,090][105620] Updated weights for policy 1, policy_version 781090 (0.0010) [2023-12-26 21:02:53,093][105692] Updated weights for policy 0, policy_version 781011 (0.0005) [2023-12-26 21:02:53,100][105585] KL-divergence is very high: 129.9377 [2023-12-26 21:02:53,139][105585] KL-divergence is very high: 153.8047 [2023-12-26 21:02:53,146][105692] Updated weights for policy 0, policy_version 781021 (0.0009) [2023-12-26 21:02:53,199][105692] Updated weights for policy 0, policy_version 781031 (0.0006) [2023-12-26 21:02:53,803][105620] Updated weights for policy 1, policy_version 781100 (0.0008) [2023-12-26 21:02:53,851][105620] Updated weights for policy 1, policy_version 781110 (0.0010) [2023-12-26 21:02:53,856][105692] Updated weights for policy 0, policy_version 781041 (0.0005) [2023-12-26 21:02:53,900][105620] Updated weights for policy 1, policy_version 781120 (0.0010) [2023-12-26 21:02:53,905][105692] Updated weights for policy 0, policy_version 781051 (0.0005) [2023-12-26 21:02:53,955][105692] Updated weights for policy 0, policy_version 781061 (0.0006) [2023-12-26 21:02:54,563][105692] Updated weights for policy 0, policy_version 781071 (0.0006) [2023-12-26 21:02:54,631][105620] Updated weights for policy 1, policy_version 781130 (0.0009) [2023-12-26 21:02:54,637][105692] Updated weights for policy 0, policy_version 781081 (0.0005) [2023-12-26 21:02:54,678][105620] Updated weights for policy 1, policy_version 781140 (0.0007) [2023-12-26 21:02:54,683][105692] Updated weights for policy 0, policy_version 781091 (0.0005) [2023-12-26 21:02:54,737][105620] Updated weights for policy 1, policy_version 781150 (0.0010) [2023-12-26 21:02:54,792][105620] Updated weights for policy 1, policy_version 781160 (0.0010) [2023-12-26 21:02:55,265][105692] Updated weights for policy 0, policy_version 781101 (0.0008) [2023-12-26 21:02:55,313][105692] Updated weights for policy 0, policy_version 781111 (0.0010) [2023-12-26 21:02:55,370][105692] Updated weights for policy 0, policy_version 781121 (0.0010) [2023-12-26 21:02:55,505][105620] Updated weights for policy 1, policy_version 781170 (0.0010) [2023-12-26 21:02:55,563][105620] Updated weights for policy 1, policy_version 781180 (0.0010) [2023-12-26 21:02:55,625][105620] Updated weights for policy 1, policy_version 781190 (0.0010) [2023-12-26 21:02:55,944][105692] Updated weights for policy 0, policy_version 781131 (0.0009) [2023-12-26 21:02:55,997][105692] Updated weights for policy 0, policy_version 781141 (0.0005) [2023-12-26 21:02:56,052][105692] Updated weights for policy 0, policy_version 781151 (0.0005) [2023-12-26 21:02:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 400007168. Throughput: 0: 9576.3, 1: 9847.2. Samples: 400018200. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-12-26 21:02:56,062][104569] Avg episode reward: [(0, '8987.664'), (1, '9172.121')] [2023-12-26 21:02:56,323][105620] Updated weights for policy 1, policy_version 781200 (0.0008) [2023-12-26 21:02:56,372][105620] Updated weights for policy 1, policy_version 781210 (0.0010) [2023-12-26 21:02:56,420][105620] Updated weights for policy 1, policy_version 781220 (0.0007) [2023-12-26 21:02:56,564][105692] Updated weights for policy 0, policy_version 781161 (0.0005) [2023-12-26 21:02:56,623][105692] Updated weights for policy 0, policy_version 781171 (0.0006) [2023-12-26 21:02:56,691][105692] Updated weights for policy 0, policy_version 781181 (0.0010) [2023-12-26 21:02:56,735][105692] Updated weights for policy 0, policy_version 781191 (0.0010) [2023-12-26 21:02:57,063][105620] Updated weights for policy 1, policy_version 781230 (0.0008) [2023-12-26 21:02:57,110][105620] Updated weights for policy 1, policy_version 781240 (0.0010) [2023-12-26 21:02:57,162][105620] Updated weights for policy 1, policy_version 781250 (0.0010) [2023-12-26 21:02:57,401][105692] Updated weights for policy 0, policy_version 781201 (0.0006) [2023-12-26 21:02:57,469][105692] Updated weights for policy 0, policy_version 781211 (0.0006) [2023-12-26 21:02:57,525][105692] Updated weights for policy 0, policy_version 781221 (0.0010) [2023-12-26 21:02:57,782][105620] Updated weights for policy 1, policy_version 781260 (0.0008) [2023-12-26 21:02:57,832][105620] Updated weights for policy 1, policy_version 781270 (0.0005) [2023-12-26 21:02:57,878][105620] Updated weights for policy 1, policy_version 781280 (0.0005) [2023-12-26 21:02:58,224][105692] Updated weights for policy 0, policy_version 781231 (0.0009) [2023-12-26 21:02:58,281][105692] Updated weights for policy 0, policy_version 781241 (0.0008) [2023-12-26 21:02:58,340][105692] Updated weights for policy 0, policy_version 781251 (0.0008) [2023-12-26 21:02:58,533][105620] Updated weights for policy 1, policy_version 781290 (0.0006) [2023-12-26 21:02:58,596][105620] Updated weights for policy 1, policy_version 781300 (0.0011) [2023-12-26 21:02:58,660][105620] Updated weights for policy 1, policy_version 781310 (0.0011) [2023-12-26 21:02:58,726][105620] Updated weights for policy 1, policy_version 781320 (0.0010) [2023-12-26 21:02:59,136][105692] Updated weights for policy 0, policy_version 781261 (0.0009) [2023-12-26 21:02:59,191][105692] Updated weights for policy 0, policy_version 781271 (0.0008) [2023-12-26 21:02:59,259][105692] Updated weights for policy 0, policy_version 781281 (0.0008) [2023-12-26 21:02:59,494][105620] Updated weights for policy 1, policy_version 781330 (0.0011) [2023-12-26 21:02:59,546][105620] Updated weights for policy 1, policy_version 781340 (0.0010) [2023-12-26 21:02:59,605][105620] Updated weights for policy 1, policy_version 781350 (0.0011) [2023-12-26 21:03:00,025][105692] Updated weights for policy 0, policy_version 781291 (0.0008) [2023-12-26 21:03:00,081][105692] Updated weights for policy 0, policy_version 781301 (0.0009) [2023-12-26 21:03:00,135][105692] Updated weights for policy 0, policy_version 781311 (0.0010) [2023-12-26 21:03:00,272][105620] Updated weights for policy 1, policy_version 781360 (0.0008) [2023-12-26 21:03:00,336][105620] Updated weights for policy 1, policy_version 781370 (0.0008) [2023-12-26 21:03:00,398][105620] Updated weights for policy 1, policy_version 781380 (0.0008) [2023-12-26 21:03:00,849][105692] Updated weights for policy 0, policy_version 781321 (0.0010) [2023-12-26 21:03:00,896][105692] Updated weights for policy 0, policy_version 781331 (0.0010) [2023-12-26 21:03:00,941][105692] Updated weights for policy 0, policy_version 781341 (0.0010) [2023-12-26 21:03:00,988][105620] Updated weights for policy 1, policy_version 781390 (0.0006) [2023-12-26 21:03:00,996][105692] Updated weights for policy 0, policy_version 781351 (0.0010) [2023-12-26 21:03:01,054][105620] Updated weights for policy 1, policy_version 781400 (0.0007) [2023-12-26 21:03:01,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 400113664. Throughput: 0: 9690.5, 1: 9937.6. Samples: 400082776. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-12-26 21:03:01,063][104569] Avg episode reward: [(0, '9169.452'), (1, '8988.428')] [2023-12-26 21:03:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000781352_200056832.pth... [2023-12-26 21:03:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000780232_199770112.pth [2023-12-26 21:03:01,113][105620] Updated weights for policy 1, policy_version 781410 (0.0006) [2023-12-26 21:03:01,151][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000781416_200065024.pth... [2023-12-26 21:03:01,155][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000780232_199761920.pth [2023-12-26 21:03:01,670][105692] Updated weights for policy 0, policy_version 781361 (0.0009) [2023-12-26 21:03:01,717][105692] Updated weights for policy 0, policy_version 781371 (0.0008) [2023-12-26 21:03:01,777][105692] Updated weights for policy 0, policy_version 781381 (0.0009) [2023-12-26 21:03:01,807][105620] Updated weights for policy 1, policy_version 781420 (0.0007) [2023-12-26 21:03:01,871][105620] Updated weights for policy 1, policy_version 781430 (0.0009) [2023-12-26 21:03:01,924][105620] Updated weights for policy 1, policy_version 781440 (0.0010) [2023-12-26 21:03:02,590][105692] Updated weights for policy 0, policy_version 781391 (0.0008) [2023-12-26 21:03:02,591][105620] Updated weights for policy 1, policy_version 781450 (0.0010) [2023-12-26 21:03:02,648][105692] Updated weights for policy 0, policy_version 781401 (0.0007) [2023-12-26 21:03:02,657][105620] Updated weights for policy 1, policy_version 781460 (0.0010) [2023-12-26 21:03:02,706][105692] Updated weights for policy 0, policy_version 781411 (0.0007) [2023-12-26 21:03:02,712][105620] Updated weights for policy 1, policy_version 781470 (0.0010) [2023-12-26 21:03:02,766][105620] Updated weights for policy 1, policy_version 781480 (0.0008) [2023-12-26 21:03:03,450][105692] Updated weights for policy 0, policy_version 781421 (0.0007) [2023-12-26 21:03:03,501][105620] Updated weights for policy 1, policy_version 781490 (0.0005) [2023-12-26 21:03:03,502][105692] Updated weights for policy 0, policy_version 781431 (0.0009) [2023-12-26 21:03:03,552][105692] Updated weights for policy 0, policy_version 781441 (0.0008) [2023-12-26 21:03:03,556][105620] Updated weights for policy 1, policy_version 781500 (0.0007) [2023-12-26 21:03:03,612][105620] Updated weights for policy 1, policy_version 781510 (0.0008) [2023-12-26 21:03:04,268][105692] Updated weights for policy 0, policy_version 781451 (0.0008) [2023-12-26 21:03:04,329][105692] Updated weights for policy 0, policy_version 781461 (0.0007) [2023-12-26 21:03:04,346][105620] Updated weights for policy 1, policy_version 781520 (0.0010) [2023-12-26 21:03:04,389][105692] Updated weights for policy 0, policy_version 781471 (0.0007) [2023-12-26 21:03:04,407][105620] Updated weights for policy 1, policy_version 781530 (0.0007) [2023-12-26 21:03:04,466][105620] Updated weights for policy 1, policy_version 781540 (0.0008) [2023-12-26 21:03:05,159][105692] Updated weights for policy 0, policy_version 781481 (0.0006) [2023-12-26 21:03:05,184][105620] Updated weights for policy 1, policy_version 781550 (0.0009) [2023-12-26 21:03:05,207][105692] Updated weights for policy 0, policy_version 781491 (0.0006) [2023-12-26 21:03:05,233][105620] Updated weights for policy 1, policy_version 781560 (0.0008) [2023-12-26 21:03:05,260][105692] Updated weights for policy 0, policy_version 781501 (0.0006) [2023-12-26 21:03:05,286][105620] Updated weights for policy 1, policy_version 781570 (0.0009) [2023-12-26 21:03:05,317][105692] Updated weights for policy 0, policy_version 781511 (0.0008) [2023-12-26 21:03:05,991][105620] Updated weights for policy 1, policy_version 781580 (0.0007) [2023-12-26 21:03:06,035][105620] Updated weights for policy 1, policy_version 781590 (0.0005) [2023-12-26 21:03:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 400203776. Throughput: 0: 9596.3, 1: 9921.5. Samples: 400198932. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-12-26 21:03:06,062][104569] Avg episode reward: [(0, '9351.863'), (1, '9080.489')] [2023-12-26 21:03:06,082][105620] Updated weights for policy 1, policy_version 781600 (0.0006) [2023-12-26 21:03:06,084][105692] Updated weights for policy 0, policy_version 781521 (0.0010) [2023-12-26 21:03:06,142][105692] Updated weights for policy 0, policy_version 781531 (0.0009) [2023-12-26 21:03:06,207][105692] Updated weights for policy 0, policy_version 781541 (0.0010) [2023-12-26 21:03:06,846][105620] Updated weights for policy 1, policy_version 781610 (0.0008) [2023-12-26 21:03:06,906][105620] Updated weights for policy 1, policy_version 781620 (0.0008) [2023-12-26 21:03:06,960][105692] Updated weights for policy 0, policy_version 781551 (0.0007) [2023-12-26 21:03:06,965][105620] Updated weights for policy 1, policy_version 781630 (0.0006) [2023-12-26 21:03:07,015][105692] Updated weights for policy 0, policy_version 781561 (0.0005) [2023-12-26 21:03:07,026][105620] Updated weights for policy 1, policy_version 781640 (0.0005) [2023-12-26 21:03:07,073][105692] Updated weights for policy 0, policy_version 781571 (0.0006) [2023-12-26 21:03:07,759][105620] Updated weights for policy 1, policy_version 781650 (0.0005) [2023-12-26 21:03:07,791][105692] Updated weights for policy 0, policy_version 781581 (0.0006) [2023-12-26 21:03:07,812][105620] Updated weights for policy 1, policy_version 781660 (0.0005) [2023-12-26 21:03:07,849][105692] Updated weights for policy 0, policy_version 781591 (0.0008) [2023-12-26 21:03:07,865][105620] Updated weights for policy 1, policy_version 781670 (0.0005) [2023-12-26 21:03:07,907][105692] Updated weights for policy 0, policy_version 781601 (0.0009) [2023-12-26 21:03:08,410][105620] Updated weights for policy 1, policy_version 781680 (0.0006) [2023-12-26 21:03:08,465][105620] Updated weights for policy 1, policy_version 781690 (0.0005) [2023-12-26 21:03:08,527][105620] Updated weights for policy 1, policy_version 781700 (0.0005) [2023-12-26 21:03:08,770][105692] Updated weights for policy 0, policy_version 781611 (0.0009) [2023-12-26 21:03:08,830][105692] Updated weights for policy 0, policy_version 781621 (0.0006) [2023-12-26 21:03:08,890][105692] Updated weights for policy 0, policy_version 781631 (0.0005) [2023-12-26 21:03:09,162][105620] Updated weights for policy 1, policy_version 781710 (0.0007) [2023-12-26 21:03:09,223][105620] Updated weights for policy 1, policy_version 781720 (0.0006) [2023-12-26 21:03:09,283][105620] Updated weights for policy 1, policy_version 781730 (0.0007) [2023-12-26 21:03:09,684][105692] Updated weights for policy 0, policy_version 781641 (0.0008) [2023-12-26 21:03:09,749][105692] Updated weights for policy 0, policy_version 781651 (0.0010) [2023-12-26 21:03:09,814][105692] Updated weights for policy 0, policy_version 781661 (0.0009) [2023-12-26 21:03:09,882][105692] Updated weights for policy 0, policy_version 781671 (0.0009) [2023-12-26 21:03:09,938][105620] Updated weights for policy 1, policy_version 781740 (0.0008) [2023-12-26 21:03:10,002][105620] Updated weights for policy 1, policy_version 781750 (0.0007) [2023-12-26 21:03:10,057][105620] Updated weights for policy 1, policy_version 781760 (0.0007) [2023-12-26 21:03:10,694][105692] Updated weights for policy 0, policy_version 781681 (0.0011) [2023-12-26 21:03:10,710][105620] Updated weights for policy 1, policy_version 781770 (0.0007) [2023-12-26 21:03:10,760][105692] Updated weights for policy 0, policy_version 781691 (0.0009) [2023-12-26 21:03:10,772][105620] Updated weights for policy 1, policy_version 781780 (0.0006) [2023-12-26 21:03:10,817][105692] Updated weights for policy 0, policy_version 781701 (0.0005) [2023-12-26 21:03:10,828][105620] Updated weights for policy 1, policy_version 781790 (0.0007) [2023-12-26 21:03:10,881][105620] Updated weights for policy 1, policy_version 781800 (0.0006) [2023-12-26 21:03:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 400310272. Throughput: 0: 9555.3, 1: 9912.4. Samples: 400314736. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-12-26 21:03:11,063][104569] Avg episode reward: [(0, '9351.715'), (1, '9354.993')] [2023-12-26 21:03:11,468][105692] Updated weights for policy 0, policy_version 781711 (0.0007) [2023-12-26 21:03:11,528][105692] Updated weights for policy 0, policy_version 781721 (0.0010) [2023-12-26 21:03:11,600][105692] Updated weights for policy 0, policy_version 781731 (0.0010) [2023-12-26 21:03:11,632][105620] Updated weights for policy 1, policy_version 781810 (0.0007) [2023-12-26 21:03:11,701][105620] Updated weights for policy 1, policy_version 781820 (0.0008) [2023-12-26 21:03:11,770][105620] Updated weights for policy 1, policy_version 781830 (0.0008) [2023-12-26 21:03:12,427][105692] Updated weights for policy 0, policy_version 781741 (0.0008) [2023-12-26 21:03:12,428][105620] Updated weights for policy 1, policy_version 781840 (0.0006) [2023-12-26 21:03:12,477][105620] Updated weights for policy 1, policy_version 781850 (0.0005) [2023-12-26 21:03:12,484][105692] Updated weights for policy 0, policy_version 781751 (0.0009) [2023-12-26 21:03:12,529][105620] Updated weights for policy 1, policy_version 781860 (0.0005) [2023-12-26 21:03:12,541][105692] Updated weights for policy 0, policy_version 781761 (0.0009) [2023-12-26 21:03:13,114][105620] Updated weights for policy 1, policy_version 781870 (0.0006) [2023-12-26 21:03:13,163][105620] Updated weights for policy 1, policy_version 781880 (0.0006) [2023-12-26 21:03:13,212][105620] Updated weights for policy 1, policy_version 781890 (0.0005) [2023-12-26 21:03:13,280][105692] Updated weights for policy 0, policy_version 781771 (0.0008) [2023-12-26 21:03:13,331][105692] Updated weights for policy 0, policy_version 781781 (0.0009) [2023-12-26 21:03:13,383][105692] Updated weights for policy 0, policy_version 781791 (0.0009) [2023-12-26 21:03:13,913][105620] Updated weights for policy 1, policy_version 781900 (0.0009) [2023-12-26 21:03:13,971][105620] Updated weights for policy 1, policy_version 781910 (0.0009) [2023-12-26 21:03:14,017][105620] Updated weights for policy 1, policy_version 781920 (0.0009) [2023-12-26 21:03:14,180][105692] Updated weights for policy 0, policy_version 781801 (0.0009) [2023-12-26 21:03:14,241][105692] Updated weights for policy 0, policy_version 781811 (0.0009) [2023-12-26 21:03:14,302][105692] Updated weights for policy 0, policy_version 781821 (0.0009) [2023-12-26 21:03:14,364][105692] Updated weights for policy 0, policy_version 781831 (0.0007) [2023-12-26 21:03:14,823][105620] Updated weights for policy 1, policy_version 781930 (0.0009) [2023-12-26 21:03:14,882][105620] Updated weights for policy 1, policy_version 781940 (0.0009) [2023-12-26 21:03:14,938][105620] Updated weights for policy 1, policy_version 781950 (0.0009) [2023-12-26 21:03:15,000][105620] Updated weights for policy 1, policy_version 781960 (0.0009) [2023-12-26 21:03:15,002][105692] Updated weights for policy 0, policy_version 781841 (0.0005) [2023-12-26 21:03:15,089][105692] Updated weights for policy 0, policy_version 781851 (0.0005) [2023-12-26 21:03:15,151][105692] Updated weights for policy 0, policy_version 781861 (0.0008) [2023-12-26 21:03:15,810][105620] Updated weights for policy 1, policy_version 781970 (0.0006) [2023-12-26 21:03:15,841][105692] Updated weights for policy 0, policy_version 781871 (0.0009) [2023-12-26 21:03:15,859][105620] Updated weights for policy 1, policy_version 781980 (0.0005) [2023-12-26 21:03:15,897][105692] Updated weights for policy 0, policy_version 781881 (0.0009) [2023-12-26 21:03:15,905][105620] Updated weights for policy 1, policy_version 781990 (0.0005) [2023-12-26 21:03:15,965][105692] Updated weights for policy 0, policy_version 781891 (0.0005) [2023-12-26 21:03:16,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 400408576. Throughput: 0: 9496.2, 1: 9973.1. Samples: 400373524. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-12-26 21:03:16,062][104569] Avg episode reward: [(0, '9351.668'), (1, '9080.437')] [2023-12-26 21:03:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000781896_200196096.pth... [2023-12-26 21:03:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000781992_200212480.pth... [2023-12-26 21:03:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000780776_199909376.pth [2023-12-26 21:03:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000780808_199909376.pth [2023-12-26 21:03:16,517][105620] Updated weights for policy 1, policy_version 782000 (0.0009) [2023-12-26 21:03:16,569][105692] Updated weights for policy 0, policy_version 781901 (0.0007) [2023-12-26 21:03:16,578][105620] Updated weights for policy 1, policy_version 782010 (0.0010) [2023-12-26 21:03:16,631][105692] Updated weights for policy 0, policy_version 781911 (0.0010) [2023-12-26 21:03:16,633][105620] Updated weights for policy 1, policy_version 782020 (0.0010) [2023-12-26 21:03:16,683][105692] Updated weights for policy 0, policy_version 781921 (0.0010) [2023-12-26 21:03:17,327][105692] Updated weights for policy 0, policy_version 781931 (0.0009) [2023-12-26 21:03:17,386][105692] Updated weights for policy 0, policy_version 781941 (0.0007) [2023-12-26 21:03:17,440][105620] Updated weights for policy 1, policy_version 782030 (0.0010) [2023-12-26 21:03:17,451][105692] Updated weights for policy 0, policy_version 781951 (0.0008) [2023-12-26 21:03:17,498][105620] Updated weights for policy 1, policy_version 782040 (0.0007) [2023-12-26 21:03:17,561][105620] Updated weights for policy 1, policy_version 782050 (0.0006) [2023-12-26 21:03:18,065][105692] Updated weights for policy 0, policy_version 781961 (0.0005) [2023-12-26 21:03:18,124][105692] Updated weights for policy 0, policy_version 781971 (0.0007) [2023-12-26 21:03:18,176][105692] Updated weights for policy 0, policy_version 781981 (0.0010) [2023-12-26 21:03:18,234][105692] Updated weights for policy 0, policy_version 781991 (0.0010) [2023-12-26 21:03:18,290][105620] Updated weights for policy 1, policy_version 782060 (0.0008) [2023-12-26 21:03:18,355][105620] Updated weights for policy 1, policy_version 782070 (0.0008) [2023-12-26 21:03:18,403][105620] Updated weights for policy 1, policy_version 782080 (0.0008) [2023-12-26 21:03:18,974][105692] Updated weights for policy 0, policy_version 782001 (0.0010) [2023-12-26 21:03:19,022][105692] Updated weights for policy 0, policy_version 782011 (0.0010) [2023-12-26 21:03:19,073][105692] Updated weights for policy 0, policy_version 782021 (0.0010) [2023-12-26 21:03:19,162][105620] Updated weights for policy 1, policy_version 782090 (0.0009) [2023-12-26 21:03:19,209][105620] Updated weights for policy 1, policy_version 782100 (0.0010) [2023-12-26 21:03:19,278][105620] Updated weights for policy 1, policy_version 782110 (0.0008) [2023-12-26 21:03:19,334][105620] Updated weights for policy 1, policy_version 782120 (0.0006) [2023-12-26 21:03:19,857][105692] Updated weights for policy 0, policy_version 782031 (0.0010) [2023-12-26 21:03:19,921][105692] Updated weights for policy 0, policy_version 782041 (0.0010) [2023-12-26 21:03:19,970][105620] Updated weights for policy 1, policy_version 782130 (0.0011) [2023-12-26 21:03:19,989][105692] Updated weights for policy 0, policy_version 782051 (0.0007) [2023-12-26 21:03:20,038][105620] Updated weights for policy 1, policy_version 782140 (0.0011) [2023-12-26 21:03:20,103][105620] Updated weights for policy 1, policy_version 782150 (0.0011) [2023-12-26 21:03:20,614][105692] Updated weights for policy 0, policy_version 782061 (0.0009) [2023-12-26 21:03:20,679][105692] Updated weights for policy 0, policy_version 782071 (0.0007) [2023-12-26 21:03:20,749][105692] Updated weights for policy 0, policy_version 782081 (0.0008) [2023-12-26 21:03:20,863][105620] Updated weights for policy 1, policy_version 782160 (0.0009) [2023-12-26 21:03:20,933][105620] Updated weights for policy 1, policy_version 782170 (0.0006) [2023-12-26 21:03:20,997][105620] Updated weights for policy 1, policy_version 782180 (0.0006) [2023-12-26 21:03:21,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 400506880. Throughput: 0: 9593.1, 1: 9896.7. Samples: 400491388. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-12-26 21:03:21,063][104569] Avg episode reward: [(0, '9351.334'), (1, '8899.612')] [2023-12-26 21:03:21,478][105692] Updated weights for policy 0, policy_version 782091 (0.0009) [2023-12-26 21:03:21,537][105692] Updated weights for policy 0, policy_version 782101 (0.0008) [2023-12-26 21:03:21,591][105692] Updated weights for policy 0, policy_version 782111 (0.0006) [2023-12-26 21:03:21,698][105620] Updated weights for policy 1, policy_version 782190 (0.0009) [2023-12-26 21:03:21,765][105620] Updated weights for policy 1, policy_version 782200 (0.0009) [2023-12-26 21:03:21,819][105620] Updated weights for policy 1, policy_version 782210 (0.0009) [2023-12-26 21:03:22,364][105692] Updated weights for policy 0, policy_version 782121 (0.0008) [2023-12-26 21:03:22,430][105692] Updated weights for policy 0, policy_version 782131 (0.0011) [2023-12-26 21:03:22,483][105692] Updated weights for policy 0, policy_version 782141 (0.0011) [2023-12-26 21:03:22,533][105692] Updated weights for policy 0, policy_version 782151 (0.0009) [2023-12-26 21:03:22,570][105620] Updated weights for policy 1, policy_version 782220 (0.0008) [2023-12-26 21:03:22,622][105620] Updated weights for policy 1, policy_version 782230 (0.0008) [2023-12-26 21:03:22,674][105620] Updated weights for policy 1, policy_version 782240 (0.0008) [2023-12-26 21:03:23,302][105692] Updated weights for policy 0, policy_version 782161 (0.0010) [2023-12-26 21:03:23,356][105692] Updated weights for policy 0, policy_version 782171 (0.0010) [2023-12-26 21:03:23,400][105692] Updated weights for policy 0, policy_version 782181 (0.0010) [2023-12-26 21:03:23,445][105620] Updated weights for policy 1, policy_version 782250 (0.0008) [2023-12-26 21:03:23,507][105620] Updated weights for policy 1, policy_version 782260 (0.0008) [2023-12-26 21:03:23,554][105620] Updated weights for policy 1, policy_version 782270 (0.0008) [2023-12-26 21:03:23,598][105620] Updated weights for policy 1, policy_version 782280 (0.0008) [2023-12-26 21:03:24,160][105692] Updated weights for policy 0, policy_version 782191 (0.0010) [2023-12-26 21:03:24,223][105692] Updated weights for policy 0, policy_version 782201 (0.0010) [2023-12-26 21:03:24,275][105692] Updated weights for policy 0, policy_version 782211 (0.0010) [2023-12-26 21:03:24,294][105620] Updated weights for policy 1, policy_version 782290 (0.0010) [2023-12-26 21:03:24,339][105620] Updated weights for policy 1, policy_version 782300 (0.0010) [2023-12-26 21:03:24,390][105620] Updated weights for policy 1, policy_version 782310 (0.0010) [2023-12-26 21:03:25,019][105692] Updated weights for policy 0, policy_version 782221 (0.0010) [2023-12-26 21:03:25,070][105692] Updated weights for policy 0, policy_version 782231 (0.0010) [2023-12-26 21:03:25,077][105585] KL-divergence is very high: 130.7886 [2023-12-26 21:03:25,089][105620] Updated weights for policy 1, policy_version 782320 (0.0007) [2023-12-26 21:03:25,103][105585] KL-divergence is very high: 141.6242 [2023-12-26 21:03:25,114][105585] KL-divergence is very high: 165.2596 [2023-12-26 21:03:25,126][105585] KL-divergence is very high: 107.9673 [2023-12-26 21:03:25,131][105692] Updated weights for policy 0, policy_version 782241 (0.0010) [2023-12-26 21:03:25,145][105620] Updated weights for policy 1, policy_version 782330 (0.0006) [2023-12-26 21:03:25,149][105585] KL-divergence is very high: 209.9505 [2023-12-26 21:03:25,161][105585] KL-divergence is very high: 192.9103 [2023-12-26 21:03:25,188][105620] Updated weights for policy 1, policy_version 782340 (0.0008) [2023-12-26 21:03:25,850][105692] Updated weights for policy 0, policy_version 782251 (0.0010) [2023-12-26 21:03:25,914][105692] Updated weights for policy 0, policy_version 782261 (0.0011) [2023-12-26 21:03:25,944][105620] Updated weights for policy 1, policy_version 782350 (0.0007) [2023-12-26 21:03:25,973][105692] Updated weights for policy 0, policy_version 782271 (0.0010) [2023-12-26 21:03:26,002][105620] Updated weights for policy 1, policy_version 782360 (0.0010) [2023-12-26 21:03:26,056][105620] Updated weights for policy 1, policy_version 782370 (0.0008) [2023-12-26 21:03:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 400596992. Throughput: 0: 9641.2, 1: 9863.9. Samples: 400606256. Policy #0 lag: (min: 12.0, avg: 38.6, max: 40.0) [2023-12-26 21:03:26,062][104569] Avg episode reward: [(0, '8998.105'), (1, '9174.494')] [2023-12-26 21:03:26,673][105692] Updated weights for policy 0, policy_version 782281 (0.0011) [2023-12-26 21:03:26,705][105620] Updated weights for policy 1, policy_version 782380 (0.0010) [2023-12-26 21:03:26,731][105692] Updated weights for policy 0, policy_version 782291 (0.0010) [2023-12-26 21:03:26,750][105620] Updated weights for policy 1, policy_version 782390 (0.0005) [2023-12-26 21:03:26,788][105692] Updated weights for policy 0, policy_version 782301 (0.0010) [2023-12-26 21:03:26,801][105620] Updated weights for policy 1, policy_version 782400 (0.0009) [2023-12-26 21:03:26,846][105692] Updated weights for policy 0, policy_version 782311 (0.0010) [2023-12-26 21:03:27,438][105620] Updated weights for policy 1, policy_version 782410 (0.0007) [2023-12-26 21:03:27,492][105620] Updated weights for policy 1, policy_version 782420 (0.0006) [2023-12-26 21:03:27,544][105620] Updated weights for policy 1, policy_version 782430 (0.0009) [2023-12-26 21:03:27,566][105692] Updated weights for policy 0, policy_version 782321 (0.0010) [2023-12-26 21:03:27,591][105620] Updated weights for policy 1, policy_version 782440 (0.0006) [2023-12-26 21:03:27,620][105692] Updated weights for policy 0, policy_version 782331 (0.0010) [2023-12-26 21:03:27,676][105692] Updated weights for policy 0, policy_version 782341 (0.0009) [2023-12-26 21:03:28,220][105620] Updated weights for policy 1, policy_version 782450 (0.0005) [2023-12-26 21:03:28,273][105620] Updated weights for policy 1, policy_version 782460 (0.0005) [2023-12-26 21:03:28,336][105620] Updated weights for policy 1, policy_version 782470 (0.0005) [2023-12-26 21:03:28,371][105692] Updated weights for policy 0, policy_version 782351 (0.0010) [2023-12-26 21:03:28,425][105692] Updated weights for policy 0, policy_version 782361 (0.0009) [2023-12-26 21:03:28,488][105692] Updated weights for policy 0, policy_version 782371 (0.0009) [2023-12-26 21:03:29,013][105620] Updated weights for policy 1, policy_version 782480 (0.0009) [2023-12-26 21:03:29,058][105620] Updated weights for policy 1, policy_version 782490 (0.0010) [2023-12-26 21:03:29,099][105692] Updated weights for policy 0, policy_version 782381 (0.0006) [2023-12-26 21:03:29,103][105620] Updated weights for policy 1, policy_version 782500 (0.0010) [2023-12-26 21:03:29,162][105692] Updated weights for policy 0, policy_version 782391 (0.0008) [2023-12-26 21:03:29,224][105692] Updated weights for policy 0, policy_version 782401 (0.0008) [2023-12-26 21:03:29,908][105620] Updated weights for policy 1, policy_version 782510 (0.0009) [2023-12-26 21:03:29,908][105692] Updated weights for policy 0, policy_version 782411 (0.0006) [2023-12-26 21:03:29,973][105620] Updated weights for policy 1, policy_version 782520 (0.0008) [2023-12-26 21:03:29,974][105692] Updated weights for policy 0, policy_version 782421 (0.0008) [2023-12-26 21:03:30,028][105620] Updated weights for policy 1, policy_version 782530 (0.0008) [2023-12-26 21:03:30,034][105692] Updated weights for policy 0, policy_version 782431 (0.0009) [2023-12-26 21:03:30,700][105620] Updated weights for policy 1, policy_version 782540 (0.0008) [2023-12-26 21:03:30,747][105586] KL-divergence is very high: 127.4284 [2023-12-26 21:03:30,751][105692] Updated weights for policy 0, policy_version 782441 (0.0005) [2023-12-26 21:03:30,753][105620] Updated weights for policy 1, policy_version 782550 (0.0008) [2023-12-26 21:03:30,788][105586] KL-divergence is very high: 189.2344 [2023-12-26 21:03:30,801][105620] Updated weights for policy 1, policy_version 782560 (0.0005) [2023-12-26 21:03:30,803][105692] Updated weights for policy 0, policy_version 782451 (0.0007) [2023-12-26 21:03:30,827][105586] KL-divergence is very high: 176.5116 [2023-12-26 21:03:30,852][105692] Updated weights for policy 0, policy_version 782461 (0.0006) [2023-12-26 21:03:30,887][105585] KL-divergence is very high: 141.4200 [2023-12-26 21:03:30,902][105692] Updated weights for policy 0, policy_version 782471 (0.0009) [2023-12-26 21:03:31,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 400703488. Throughput: 0: 9624.5, 1: 9966.8. Samples: 400667952. Policy #0 lag: (min: 12.0, avg: 38.6, max: 40.0) [2023-12-26 21:03:31,063][104569] Avg episode reward: [(0, '8643.314'), (1, '9263.145')] [2023-12-26 21:03:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000782472_200343552.pth... [2023-12-26 21:03:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000782568_200359936.pth... [2023-12-26 21:03:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000781352_200056832.pth [2023-12-26 21:03:31,097][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000781416_200065024.pth [2023-12-26 21:03:31,524][105620] Updated weights for policy 1, policy_version 782570 (0.0008) [2023-12-26 21:03:31,583][105620] Updated weights for policy 1, policy_version 782580 (0.0009) [2023-12-26 21:03:31,647][105620] Updated weights for policy 1, policy_version 782590 (0.0008) [2023-12-26 21:03:31,678][105692] Updated weights for policy 0, policy_version 782481 (0.0007) [2023-12-26 21:03:31,700][105620] Updated weights for policy 1, policy_version 782600 (0.0006) [2023-12-26 21:03:31,739][105692] Updated weights for policy 0, policy_version 782491 (0.0009) [2023-12-26 21:03:31,794][105692] Updated weights for policy 0, policy_version 782501 (0.0009) [2023-12-26 21:03:32,367][105620] Updated weights for policy 1, policy_version 782610 (0.0010) [2023-12-26 21:03:32,423][105620] Updated weights for policy 1, policy_version 782620 (0.0010) [2023-12-26 21:03:32,476][105620] Updated weights for policy 1, policy_version 782630 (0.0010) [2023-12-26 21:03:32,611][105692] Updated weights for policy 0, policy_version 782511 (0.0008) [2023-12-26 21:03:32,667][105692] Updated weights for policy 0, policy_version 782521 (0.0008) [2023-12-26 21:03:32,723][105692] Updated weights for policy 0, policy_version 782531 (0.0009) [2023-12-26 21:03:33,253][105620] Updated weights for policy 1, policy_version 782640 (0.0010) [2023-12-26 21:03:33,304][105620] Updated weights for policy 1, policy_version 782650 (0.0010) [2023-12-26 21:03:33,352][105620] Updated weights for policy 1, policy_version 782660 (0.0010) [2023-12-26 21:03:33,365][105692] Updated weights for policy 0, policy_version 782541 (0.0008) [2023-12-26 21:03:33,415][105692] Updated weights for policy 0, policy_version 782551 (0.0005) [2023-12-26 21:03:33,468][105692] Updated weights for policy 0, policy_version 782561 (0.0005) [2023-12-26 21:03:33,970][105620] Updated weights for policy 1, policy_version 782670 (0.0010) [2023-12-26 21:03:33,986][105692] Updated weights for policy 0, policy_version 782571 (0.0006) [2023-12-26 21:03:34,028][105620] Updated weights for policy 1, policy_version 782680 (0.0010) [2023-12-26 21:03:34,042][105692] Updated weights for policy 0, policy_version 782581 (0.0005) [2023-12-26 21:03:34,088][105692] Updated weights for policy 0, policy_version 782591 (0.0005) [2023-12-26 21:03:34,089][105620] Updated weights for policy 1, policy_version 782690 (0.0010) [2023-12-26 21:03:34,778][105692] Updated weights for policy 0, policy_version 782601 (0.0006) [2023-12-26 21:03:34,827][105620] Updated weights for policy 1, policy_version 782700 (0.0010) [2023-12-26 21:03:34,827][105692] Updated weights for policy 0, policy_version 782611 (0.0011) [2023-12-26 21:03:34,875][105620] Updated weights for policy 1, policy_version 782710 (0.0010) [2023-12-26 21:03:34,875][105692] Updated weights for policy 0, policy_version 782621 (0.0010) [2023-12-26 21:03:34,927][105692] Updated weights for policy 0, policy_version 782631 (0.0010) [2023-12-26 21:03:34,927][105620] Updated weights for policy 1, policy_version 782720 (0.0010) [2023-12-26 21:03:35,607][105620] Updated weights for policy 1, policy_version 782730 (0.0009) [2023-12-26 21:03:35,670][105620] Updated weights for policy 1, policy_version 782740 (0.0008) [2023-12-26 21:03:35,689][105692] Updated weights for policy 0, policy_version 782641 (0.0010) [2023-12-26 21:03:35,732][105620] Updated weights for policy 1, policy_version 782750 (0.0010) [2023-12-26 21:03:35,748][105692] Updated weights for policy 0, policy_version 782651 (0.0010) [2023-12-26 21:03:35,785][105620] Updated weights for policy 1, policy_version 782760 (0.0010) [2023-12-26 21:03:35,804][105692] Updated weights for policy 0, policy_version 782661 (0.0010) [2023-12-26 21:03:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 400801792. Throughput: 0: 9780.4, 1: 9943.1. Samples: 400787712. Policy #0 lag: (min: 12.0, avg: 38.6, max: 40.0) [2023-12-26 21:03:36,062][104569] Avg episode reward: [(0, '8553.735'), (1, '9173.825')] [2023-12-26 21:03:36,422][105620] Updated weights for policy 1, policy_version 782770 (0.0008) [2023-12-26 21:03:36,483][105620] Updated weights for policy 1, policy_version 782780 (0.0009) [2023-12-26 21:03:36,543][105620] Updated weights for policy 1, policy_version 782790 (0.0008) [2023-12-26 21:03:36,575][105692] Updated weights for policy 0, policy_version 782671 (0.0010) [2023-12-26 21:03:36,632][105692] Updated weights for policy 0, policy_version 782681 (0.0011) [2023-12-26 21:03:36,686][105692] Updated weights for policy 0, policy_version 782691 (0.0011) [2023-12-26 21:03:37,298][105692] Updated weights for policy 0, policy_version 782701 (0.0010) [2023-12-26 21:03:37,306][105620] Updated weights for policy 1, policy_version 782800 (0.0009) [2023-12-26 21:03:37,353][105692] Updated weights for policy 0, policy_version 782711 (0.0010) [2023-12-26 21:03:37,365][105620] Updated weights for policy 1, policy_version 782810 (0.0008) [2023-12-26 21:03:37,421][105692] Updated weights for policy 0, policy_version 782721 (0.0008) [2023-12-26 21:03:37,423][105620] Updated weights for policy 1, policy_version 782820 (0.0008) [2023-12-26 21:03:38,020][105692] Updated weights for policy 0, policy_version 782731 (0.0006) [2023-12-26 21:03:38,086][105692] Updated weights for policy 0, policy_version 782741 (0.0006) [2023-12-26 21:03:38,103][105585] KL-divergence is very high: 122.0567 [2023-12-26 21:03:38,109][105585] KL-divergence is very high: 116.6611 [2023-12-26 21:03:38,148][105692] Updated weights for policy 0, policy_version 782751 (0.0006) [2023-12-26 21:03:38,153][105585] KL-divergence is very high: 110.0208 [2023-12-26 21:03:38,160][105585] KL-divergence is very high: 111.4074 [2023-12-26 21:03:38,277][105620] Updated weights for policy 1, policy_version 782830 (0.0009) [2023-12-26 21:03:38,330][105620] Updated weights for policy 1, policy_version 782841 (0.0010) [2023-12-26 21:03:38,391][105620] Updated weights for policy 1, policy_version 782851 (0.0010) [2023-12-26 21:03:38,694][105692] Updated weights for policy 0, policy_version 782761 (0.0005) [2023-12-26 21:03:38,759][105692] Updated weights for policy 0, policy_version 782771 (0.0005) [2023-12-26 21:03:38,824][105692] Updated weights for policy 0, policy_version 782781 (0.0009) [2023-12-26 21:03:38,882][105692] Updated weights for policy 0, policy_version 782791 (0.0011) [2023-12-26 21:03:39,285][105620] Updated weights for policy 1, policy_version 782861 (0.0008) [2023-12-26 21:03:39,348][105620] Updated weights for policy 1, policy_version 782871 (0.0008) [2023-12-26 21:03:39,407][105620] Updated weights for policy 1, policy_version 782881 (0.0009) [2023-12-26 21:03:39,520][105692] Updated weights for policy 0, policy_version 782801 (0.0009) [2023-12-26 21:03:39,570][105692] Updated weights for policy 0, policy_version 782811 (0.0010) [2023-12-26 21:03:39,624][105692] Updated weights for policy 0, policy_version 782821 (0.0009) [2023-12-26 21:03:40,227][105620] Updated weights for policy 1, policy_version 782891 (0.0009) [2023-12-26 21:03:40,292][105620] Updated weights for policy 1, policy_version 782901 (0.0011) [2023-12-26 21:03:40,350][105620] Updated weights for policy 1, policy_version 782911 (0.0009) [2023-12-26 21:03:40,431][105692] Updated weights for policy 0, policy_version 782831 (0.0009) [2023-12-26 21:03:40,480][105692] Updated weights for policy 0, policy_version 782841 (0.0008) [2023-12-26 21:03:40,526][105692] Updated weights for policy 0, policy_version 782851 (0.0008) [2023-12-26 21:03:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.6, 300 sec: 19549.7). Total num frames: 400891904. Throughput: 0: 9779.2, 1: 9894.0. Samples: 400903496. Policy #0 lag: (min: 12.0, avg: 38.6, max: 40.0) [2023-12-26 21:03:41,062][104569] Avg episode reward: [(0, '8645.506'), (1, '9173.566')] [2023-12-26 21:03:41,088][105620] Updated weights for policy 1, policy_version 782921 (0.0010) [2023-12-26 21:03:41,155][105620] Updated weights for policy 1, policy_version 782931 (0.0010) [2023-12-26 21:03:41,208][105620] Updated weights for policy 1, policy_version 782941 (0.0010) [2023-12-26 21:03:41,265][105620] Updated weights for policy 1, policy_version 782951 (0.0011) [2023-12-26 21:03:41,363][105692] Updated weights for policy 0, policy_version 782861 (0.0008) [2023-12-26 21:03:41,427][105692] Updated weights for policy 0, policy_version 782871 (0.0008) [2023-12-26 21:03:41,494][105692] Updated weights for policy 0, policy_version 782881 (0.0006) [2023-12-26 21:03:42,068][105620] Updated weights for policy 1, policy_version 782961 (0.0010) [2023-12-26 21:03:42,132][105620] Updated weights for policy 1, policy_version 782971 (0.0010) [2023-12-26 21:03:42,169][105692] Updated weights for policy 0, policy_version 782891 (0.0007) [2023-12-26 21:03:42,188][105620] Updated weights for policy 1, policy_version 782981 (0.0010) [2023-12-26 21:03:42,223][105692] Updated weights for policy 0, policy_version 782901 (0.0007) [2023-12-26 21:03:42,288][105692] Updated weights for policy 0, policy_version 782911 (0.0009) [2023-12-26 21:03:42,940][105620] Updated weights for policy 1, policy_version 782991 (0.0010) [2023-12-26 21:03:42,998][105620] Updated weights for policy 1, policy_version 783001 (0.0010) [2023-12-26 21:03:43,056][105620] Updated weights for policy 1, policy_version 783011 (0.0010) [2023-12-26 21:03:43,059][105692] Updated weights for policy 0, policy_version 782921 (0.0008) [2023-12-26 21:03:43,112][105692] Updated weights for policy 0, policy_version 782931 (0.0008) [2023-12-26 21:03:43,160][105692] Updated weights for policy 0, policy_version 782941 (0.0008) [2023-12-26 21:03:43,213][105692] Updated weights for policy 0, policy_version 782951 (0.0009) [2023-12-26 21:03:43,697][105620] Updated weights for policy 1, policy_version 783021 (0.0010) [2023-12-26 21:03:43,755][105620] Updated weights for policy 1, policy_version 783031 (0.0010) [2023-12-26 21:03:43,769][105586] KL-divergence is very high: 121.3291 [2023-12-26 21:03:43,775][105586] KL-divergence is very high: 107.6806 [2023-12-26 21:03:43,788][105586] KL-divergence is very high: 121.9347 [2023-12-26 21:03:43,794][105586] KL-divergence is very high: 112.6320 [2023-12-26 21:03:43,800][105586] KL-divergence is very high: 109.3836 [2023-12-26 21:03:43,815][105620] Updated weights for policy 1, policy_version 783041 (0.0011) [2023-12-26 21:03:43,816][105586] KL-divergence is very high: 115.8540 [2023-12-26 21:03:43,821][105692] Updated weights for policy 0, policy_version 782961 (0.0011) [2023-12-26 21:03:43,822][105586] KL-divergence is very high: 102.4108 [2023-12-26 21:03:43,884][105692] Updated weights for policy 0, policy_version 782971 (0.0010) [2023-12-26 21:03:43,944][105692] Updated weights for policy 0, policy_version 782981 (0.0008) [2023-12-26 21:03:44,542][105620] Updated weights for policy 1, policy_version 783051 (0.0011) [2023-12-26 21:03:44,600][105620] Updated weights for policy 1, policy_version 783061 (0.0010) [2023-12-26 21:03:44,633][105692] Updated weights for policy 0, policy_version 782991 (0.0007) [2023-12-26 21:03:44,658][105620] Updated weights for policy 1, policy_version 783071 (0.0010) [2023-12-26 21:03:44,692][105692] Updated weights for policy 0, policy_version 783001 (0.0007) [2023-12-26 21:03:44,749][105692] Updated weights for policy 0, policy_version 783011 (0.0007) [2023-12-26 21:03:45,345][105620] Updated weights for policy 1, policy_version 783081 (0.0010) [2023-12-26 21:03:45,415][105620] Updated weights for policy 1, policy_version 783091 (0.0007) [2023-12-26 21:03:45,474][105620] Updated weights for policy 1, policy_version 783101 (0.0008) [2023-12-26 21:03:45,475][105692] Updated weights for policy 0, policy_version 783021 (0.0008) [2023-12-26 21:03:45,524][105692] Updated weights for policy 0, policy_version 783031 (0.0007) [2023-12-26 21:03:45,530][105620] Updated weights for policy 1, policy_version 783111 (0.0006) [2023-12-26 21:03:45,570][105692] Updated weights for policy 0, policy_version 783041 (0.0008) [2023-12-26 21:03:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 400990208. Throughput: 0: 9707.0, 1: 9813.3. Samples: 400961188. Policy #0 lag: (min: 12.0, avg: 38.6, max: 40.0) [2023-12-26 21:03:46,062][104569] Avg episode reward: [(0, '9089.613'), (1, '8221.041')] [2023-12-26 21:03:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000783048_200491008.pth... [2023-12-26 21:03:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000783112_200499200.pth... [2023-12-26 21:03:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000781896_200196096.pth [2023-12-26 21:03:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000781992_200212480.pth [2023-12-26 21:03:46,145][105692] Updated weights for policy 0, policy_version 783051 (0.0009) [2023-12-26 21:03:46,206][105692] Updated weights for policy 0, policy_version 783061 (0.0010) [2023-12-26 21:03:46,261][105692] Updated weights for policy 0, policy_version 783071 (0.0008) [2023-12-26 21:03:46,263][105620] Updated weights for policy 1, policy_version 783121 (0.0007) [2023-12-26 21:03:46,318][105620] Updated weights for policy 1, policy_version 783131 (0.0007) [2023-12-26 21:03:46,370][105620] Updated weights for policy 1, policy_version 783141 (0.0005) [2023-12-26 21:03:46,935][105620] Updated weights for policy 1, policy_version 783151 (0.0006) [2023-12-26 21:03:46,991][105620] Updated weights for policy 1, policy_version 783161 (0.0007) [2023-12-26 21:03:47,053][105620] Updated weights for policy 1, policy_version 783171 (0.0009) [2023-12-26 21:03:47,091][105692] Updated weights for policy 0, policy_version 783081 (0.0006) [2023-12-26 21:03:47,144][105692] Updated weights for policy 0, policy_version 783091 (0.0008) [2023-12-26 21:03:47,199][105692] Updated weights for policy 0, policy_version 783101 (0.0009) [2023-12-26 21:03:47,246][105692] Updated weights for policy 0, policy_version 783111 (0.0009) [2023-12-26 21:03:47,758][105620] Updated weights for policy 1, policy_version 783181 (0.0007) [2023-12-26 21:03:47,804][105620] Updated weights for policy 1, policy_version 783191 (0.0005) [2023-12-26 21:03:47,852][105620] Updated weights for policy 1, policy_version 783201 (0.0005) [2023-12-26 21:03:48,079][105692] Updated weights for policy 0, policy_version 783121 (0.0010) [2023-12-26 21:03:48,136][105692] Updated weights for policy 0, policy_version 783132 (0.0010) [2023-12-26 21:03:48,199][105692] Updated weights for policy 0, policy_version 783142 (0.0010) [2023-12-26 21:03:48,438][105620] Updated weights for policy 1, policy_version 783211 (0.0005) [2023-12-26 21:03:48,495][105620] Updated weights for policy 1, policy_version 783221 (0.0006) [2023-12-26 21:03:48,548][105620] Updated weights for policy 1, policy_version 783231 (0.0006) [2023-12-26 21:03:49,069][105692] Updated weights for policy 0, policy_version 783152 (0.0009) [2023-12-26 21:03:49,120][105692] Updated weights for policy 0, policy_version 783163 (0.0010) [2023-12-26 21:03:49,130][105620] Updated weights for policy 1, policy_version 783241 (0.0008) [2023-12-26 21:03:49,169][105692] Updated weights for policy 0, policy_version 783173 (0.0007) [2023-12-26 21:03:49,193][105620] Updated weights for policy 1, policy_version 783251 (0.0009) [2023-12-26 21:03:49,256][105620] Updated weights for policy 1, policy_version 783261 (0.0009) [2023-12-26 21:03:49,305][105620] Updated weights for policy 1, policy_version 783271 (0.0008) [2023-12-26 21:03:49,920][105692] Updated weights for policy 0, policy_version 783183 (0.0010) [2023-12-26 21:03:49,981][105692] Updated weights for policy 0, policy_version 783193 (0.0010) [2023-12-26 21:03:50,030][105620] Updated weights for policy 1, policy_version 783281 (0.0010) [2023-12-26 21:03:50,037][105692] Updated weights for policy 0, policy_version 783203 (0.0010) [2023-12-26 21:03:50,092][105620] Updated weights for policy 1, policy_version 783291 (0.0007) [2023-12-26 21:03:50,148][105620] Updated weights for policy 1, policy_version 783301 (0.0008) [2023-12-26 21:03:50,728][105692] Updated weights for policy 0, policy_version 783213 (0.0008) [2023-12-26 21:03:50,798][105692] Updated weights for policy 0, policy_version 783223 (0.0005) [2023-12-26 21:03:50,859][105692] Updated weights for policy 0, policy_version 783233 (0.0005) [2023-12-26 21:03:50,956][105620] Updated weights for policy 1, policy_version 783311 (0.0008) [2023-12-26 21:03:51,015][105620] Updated weights for policy 1, policy_version 783321 (0.0008) [2023-12-26 21:03:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 401088512. Throughput: 0: 9694.8, 1: 9878.4. Samples: 401079724. Policy #0 lag: (min: 12.0, avg: 38.6, max: 40.0) [2023-12-26 21:03:51,064][104569] Avg episode reward: [(0, '8820.863'), (1, '8395.034')] [2023-12-26 21:03:51,075][105620] Updated weights for policy 1, policy_version 783331 (0.0009) [2023-12-26 21:03:51,506][105692] Updated weights for policy 0, policy_version 783243 (0.0007) [2023-12-26 21:03:51,568][105692] Updated weights for policy 0, policy_version 783253 (0.0009) [2023-12-26 21:03:51,621][105692] Updated weights for policy 0, policy_version 783263 (0.0009) [2023-12-26 21:03:51,892][105620] Updated weights for policy 1, policy_version 783341 (0.0007) [2023-12-26 21:03:51,963][105620] Updated weights for policy 1, policy_version 783351 (0.0005) [2023-12-26 21:03:52,027][105620] Updated weights for policy 1, policy_version 783361 (0.0007) [2023-12-26 21:03:52,430][105692] Updated weights for policy 0, policy_version 783273 (0.0010) [2023-12-26 21:03:52,486][105692] Updated weights for policy 0, policy_version 783283 (0.0010) [2023-12-26 21:03:52,548][105692] Updated weights for policy 0, policy_version 783293 (0.0008) [2023-12-26 21:03:52,612][105692] Updated weights for policy 0, policy_version 783303 (0.0009) [2023-12-26 21:03:52,718][105620] Updated weights for policy 1, policy_version 783371 (0.0009) [2023-12-26 21:03:52,781][105620] Updated weights for policy 1, policy_version 783381 (0.0008) [2023-12-26 21:03:52,844][105620] Updated weights for policy 1, policy_version 783391 (0.0008) [2023-12-26 21:03:53,243][105692] Updated weights for policy 0, policy_version 783313 (0.0009) [2023-12-26 21:03:53,305][105692] Updated weights for policy 0, policy_version 783323 (0.0010) [2023-12-26 21:03:53,356][105692] Updated weights for policy 0, policy_version 783333 (0.0010) [2023-12-26 21:03:53,609][105620] Updated weights for policy 1, policy_version 783401 (0.0009) [2023-12-26 21:03:53,671][105620] Updated weights for policy 1, policy_version 783411 (0.0007) [2023-12-26 21:03:53,728][105620] Updated weights for policy 1, policy_version 783421 (0.0008) [2023-12-26 21:03:53,783][105620] Updated weights for policy 1, policy_version 783431 (0.0008) [2023-12-26 21:03:54,082][105692] Updated weights for policy 0, policy_version 783343 (0.0010) [2023-12-26 21:03:54,143][105692] Updated weights for policy 0, policy_version 783353 (0.0010) [2023-12-26 21:03:54,198][105692] Updated weights for policy 0, policy_version 783363 (0.0010) [2023-12-26 21:03:54,531][105620] Updated weights for policy 1, policy_version 783441 (0.0005) [2023-12-26 21:03:54,584][105620] Updated weights for policy 1, policy_version 783451 (0.0005) [2023-12-26 21:03:54,640][105620] Updated weights for policy 1, policy_version 783461 (0.0005) [2023-12-26 21:03:54,976][105692] Updated weights for policy 0, policy_version 783373 (0.0009) [2023-12-26 21:03:55,039][105692] Updated weights for policy 0, policy_version 783383 (0.0010) [2023-12-26 21:03:55,099][105692] Updated weights for policy 0, policy_version 783393 (0.0009) [2023-12-26 21:03:55,190][105620] Updated weights for policy 1, policy_version 783471 (0.0008) [2023-12-26 21:03:55,247][105620] Updated weights for policy 1, policy_version 783482 (0.0010) [2023-12-26 21:03:55,297][105620] Updated weights for policy 1, policy_version 783492 (0.0008) [2023-12-26 21:03:55,846][105692] Updated weights for policy 0, policy_version 783403 (0.0008) [2023-12-26 21:03:55,902][105692] Updated weights for policy 0, policy_version 783413 (0.0007) [2023-12-26 21:03:55,960][105692] Updated weights for policy 0, policy_version 783423 (0.0009) [2023-12-26 21:03:56,042][105620] Updated weights for policy 1, policy_version 783502 (0.0009) [2023-12-26 21:03:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 401186816. Throughput: 0: 9801.9, 1: 9773.6. Samples: 401195632. Policy #0 lag: (min: 12.0, avg: 38.6, max: 40.0) [2023-12-26 21:03:56,063][104569] Avg episode reward: [(0, '8732.003'), (1, '9005.487')] [2023-12-26 21:03:56,094][105620] Updated weights for policy 1, policy_version 783512 (0.0009) [2023-12-26 21:03:56,148][105620] Updated weights for policy 1, policy_version 783523 (0.0010) [2023-12-26 21:03:56,617][105692] Updated weights for policy 0, policy_version 783433 (0.0009) [2023-12-26 21:03:56,674][105692] Updated weights for policy 0, policy_version 783443 (0.0008) [2023-12-26 21:03:56,734][105692] Updated weights for policy 0, policy_version 783453 (0.0010) [2023-12-26 21:03:56,800][105692] Updated weights for policy 0, policy_version 783463 (0.0007) [2023-12-26 21:03:56,949][105620] Updated weights for policy 1, policy_version 783534 (0.0010) [2023-12-26 21:03:57,015][105620] Updated weights for policy 1, policy_version 783544 (0.0009) [2023-12-26 21:03:57,068][105620] Updated weights for policy 1, policy_version 783554 (0.0010) [2023-12-26 21:03:57,377][105692] Updated weights for policy 0, policy_version 783473 (0.0006) [2023-12-26 21:03:57,431][105692] Updated weights for policy 0, policy_version 783483 (0.0005) [2023-12-26 21:03:57,483][105692] Updated weights for policy 0, policy_version 783493 (0.0005) [2023-12-26 21:03:57,781][105620] Updated weights for policy 1, policy_version 783564 (0.0009) [2023-12-26 21:03:57,836][105620] Updated weights for policy 1, policy_version 783574 (0.0010) [2023-12-26 21:03:57,894][105620] Updated weights for policy 1, policy_version 783584 (0.0010) [2023-12-26 21:03:57,997][105692] Updated weights for policy 0, policy_version 783503 (0.0005) [2023-12-26 21:03:58,055][105692] Updated weights for policy 0, policy_version 783513 (0.0005) [2023-12-26 21:03:58,110][105692] Updated weights for policy 0, policy_version 783523 (0.0009) [2023-12-26 21:03:58,674][105620] Updated weights for policy 1, policy_version 783594 (0.0010) [2023-12-26 21:03:58,736][105620] Updated weights for policy 1, policy_version 783604 (0.0009) [2023-12-26 21:03:58,799][105620] Updated weights for policy 1, policy_version 783614 (0.0008) [2023-12-26 21:03:58,858][105620] Updated weights for policy 1, policy_version 783624 (0.0009) [2023-12-26 21:03:58,861][105692] Updated weights for policy 0, policy_version 783533 (0.0010) [2023-12-26 21:03:58,927][105692] Updated weights for policy 0, policy_version 783543 (0.0010) [2023-12-26 21:03:58,978][105692] Updated weights for policy 0, policy_version 783553 (0.0010) [2023-12-26 21:03:59,600][105620] Updated weights for policy 1, policy_version 783634 (0.0006) [2023-12-26 21:03:59,648][105620] Updated weights for policy 1, policy_version 783644 (0.0010) [2023-12-26 21:03:59,697][105620] Updated weights for policy 1, policy_version 783654 (0.0010) [2023-12-26 21:03:59,744][105692] Updated weights for policy 0, policy_version 783563 (0.0010) [2023-12-26 21:03:59,801][105692] Updated weights for policy 0, policy_version 783573 (0.0010) [2023-12-26 21:03:59,858][105692] Updated weights for policy 0, policy_version 783583 (0.0011) [2023-12-26 21:04:00,494][105620] Updated weights for policy 1, policy_version 783664 (0.0009) [2023-12-26 21:04:00,549][105620] Updated weights for policy 1, policy_version 783674 (0.0010) [2023-12-26 21:04:00,603][105620] Updated weights for policy 1, policy_version 783684 (0.0010) [2023-12-26 21:04:00,636][105692] Updated weights for policy 0, policy_version 783593 (0.0008) [2023-12-26 21:04:00,689][105692] Updated weights for policy 0, policy_version 783603 (0.0007) [2023-12-26 21:04:00,757][105692] Updated weights for policy 0, policy_version 783613 (0.0007) [2023-12-26 21:04:00,808][105692] Updated weights for policy 0, policy_version 783623 (0.0006) [2023-12-26 21:04:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 401285120. Throughput: 0: 9912.6, 1: 9688.1. Samples: 401255556. Policy #0 lag: (min: 12.0, avg: 38.6, max: 40.0) [2023-12-26 21:04:01,062][104569] Avg episode reward: [(0, '8641.123'), (1, '9118.488')] [2023-12-26 21:04:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000783624_200638464.pth... [2023-12-26 21:04:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000783688_200646656.pth... [2023-12-26 21:04:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000782472_200343552.pth [2023-12-26 21:04:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000782568_200359936.pth [2023-12-26 21:04:01,282][105620] Updated weights for policy 1, policy_version 783694 (0.0010) [2023-12-26 21:04:01,344][105620] Updated weights for policy 1, policy_version 783704 (0.0007) [2023-12-26 21:04:01,407][105620] Updated weights for policy 1, policy_version 783714 (0.0008) [2023-12-26 21:04:01,490][105692] Updated weights for policy 0, policy_version 783633 (0.0009) [2023-12-26 21:04:01,540][105692] Updated weights for policy 0, policy_version 783643 (0.0008) [2023-12-26 21:04:01,589][105692] Updated weights for policy 0, policy_version 783653 (0.0009) [2023-12-26 21:04:02,135][105620] Updated weights for policy 1, policy_version 783724 (0.0008) [2023-12-26 21:04:02,185][105620] Updated weights for policy 1, policy_version 783734 (0.0005) [2023-12-26 21:04:02,239][105620] Updated weights for policy 1, policy_version 783744 (0.0005) [2023-12-26 21:04:02,391][105692] Updated weights for policy 0, policy_version 783663 (0.0008) [2023-12-26 21:04:02,458][105692] Updated weights for policy 0, policy_version 783673 (0.0008) [2023-12-26 21:04:02,524][105692] Updated weights for policy 0, policy_version 783683 (0.0008) [2023-12-26 21:04:02,855][105620] Updated weights for policy 1, policy_version 783754 (0.0008) [2023-12-26 21:04:02,909][105620] Updated weights for policy 1, policy_version 783764 (0.0005) [2023-12-26 21:04:02,966][105620] Updated weights for policy 1, policy_version 783774 (0.0005) [2023-12-26 21:04:03,020][105620] Updated weights for policy 1, policy_version 783784 (0.0005) [2023-12-26 21:04:03,242][105692] Updated weights for policy 0, policy_version 783693 (0.0008) [2023-12-26 21:04:03,293][105692] Updated weights for policy 0, policy_version 783703 (0.0008) [2023-12-26 21:04:03,341][105692] Updated weights for policy 0, policy_version 783713 (0.0008) [2023-12-26 21:04:03,710][105620] Updated weights for policy 1, policy_version 783794 (0.0010) [2023-12-26 21:04:03,771][105620] Updated weights for policy 1, policy_version 783804 (0.0007) [2023-12-26 21:04:03,829][105620] Updated weights for policy 1, policy_version 783814 (0.0008) [2023-12-26 21:04:04,062][105692] Updated weights for policy 0, policy_version 783723 (0.0008) [2023-12-26 21:04:04,121][105692] Updated weights for policy 0, policy_version 783733 (0.0010) [2023-12-26 21:04:04,173][105692] Updated weights for policy 0, policy_version 783743 (0.0009) [2023-12-26 21:04:04,561][105620] Updated weights for policy 1, policy_version 783824 (0.0007) [2023-12-26 21:04:04,624][105620] Updated weights for policy 1, policy_version 783834 (0.0005) [2023-12-26 21:04:04,685][105620] Updated weights for policy 1, policy_version 783844 (0.0005) [2023-12-26 21:04:04,835][105692] Updated weights for policy 0, policy_version 783753 (0.0008) [2023-12-26 21:04:04,904][105692] Updated weights for policy 0, policy_version 783763 (0.0005) [2023-12-26 21:04:04,975][105692] Updated weights for policy 0, policy_version 783773 (0.0006) [2023-12-26 21:04:05,041][105692] Updated weights for policy 0, policy_version 783783 (0.0007) [2023-12-26 21:04:05,225][105620] Updated weights for policy 1, policy_version 783854 (0.0006) [2023-12-26 21:04:05,302][105620] Updated weights for policy 1, policy_version 783864 (0.0005) [2023-12-26 21:04:05,368][105620] Updated weights for policy 1, policy_version 783874 (0.0005) [2023-12-26 21:04:05,648][105692] Updated weights for policy 0, policy_version 783793 (0.0008) [2023-12-26 21:04:05,716][105692] Updated weights for policy 0, policy_version 783803 (0.0008) [2023-12-26 21:04:05,778][105692] Updated weights for policy 0, policy_version 783813 (0.0008) [2023-12-26 21:04:05,969][105620] Updated weights for policy 1, policy_version 783884 (0.0006) [2023-12-26 21:04:06,031][105620] Updated weights for policy 1, policy_version 783894 (0.0005) [2023-12-26 21:04:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 401383424. Throughput: 0: 9822.9, 1: 9757.7. Samples: 401372508. Policy #0 lag: (min: 12.0, avg: 38.6, max: 40.0) [2023-12-26 21:04:06,062][104569] Avg episode reward: [(0, '8821.069'), (1, '7926.520')] [2023-12-26 21:04:06,106][105620] Updated weights for policy 1, policy_version 783904 (0.0006) [2023-12-26 21:04:06,503][105692] Updated weights for policy 0, policy_version 783823 (0.0008) [2023-12-26 21:04:06,567][105692] Updated weights for policy 0, policy_version 783833 (0.0008) [2023-12-26 21:04:06,632][105692] Updated weights for policy 0, policy_version 783843 (0.0008) [2023-12-26 21:04:06,785][105620] Updated weights for policy 1, policy_version 783914 (0.0008) [2023-12-26 21:04:06,838][105620] Updated weights for policy 1, policy_version 783924 (0.0011) [2023-12-26 21:04:06,894][105620] Updated weights for policy 1, policy_version 783934 (0.0011) [2023-12-26 21:04:06,942][105620] Updated weights for policy 1, policy_version 783944 (0.0010) [2023-12-26 21:04:07,411][105692] Updated weights for policy 0, policy_version 783853 (0.0009) [2023-12-26 21:04:07,468][105692] Updated weights for policy 0, policy_version 783863 (0.0010) [2023-12-26 21:04:07,528][105692] Updated weights for policy 0, policy_version 783873 (0.0009) [2023-12-26 21:04:07,642][105620] Updated weights for policy 1, policy_version 783954 (0.0011) [2023-12-26 21:04:07,694][105620] Updated weights for policy 1, policy_version 783964 (0.0010) [2023-12-26 21:04:07,750][105620] Updated weights for policy 1, policy_version 783974 (0.0011) [2023-12-26 21:04:08,306][105692] Updated weights for policy 0, policy_version 783883 (0.0009) [2023-12-26 21:04:08,372][105692] Updated weights for policy 0, policy_version 783893 (0.0008) [2023-12-26 21:04:08,433][105692] Updated weights for policy 0, policy_version 783903 (0.0008) [2023-12-26 21:04:08,530][105620] Updated weights for policy 1, policy_version 783984 (0.0011) [2023-12-26 21:04:08,592][105620] Updated weights for policy 1, policy_version 783994 (0.0010) [2023-12-26 21:04:08,658][105620] Updated weights for policy 1, policy_version 784004 (0.0011) [2023-12-26 21:04:09,213][105692] Updated weights for policy 0, policy_version 783913 (0.0008) [2023-12-26 21:04:09,279][105692] Updated weights for policy 0, policy_version 783923 (0.0009) [2023-12-26 21:04:09,337][105692] Updated weights for policy 0, policy_version 783933 (0.0008) [2023-12-26 21:04:09,409][105692] Updated weights for policy 0, policy_version 783943 (0.0008) [2023-12-26 21:04:09,410][105620] Updated weights for policy 1, policy_version 784014 (0.0011) [2023-12-26 21:04:09,478][105620] Updated weights for policy 1, policy_version 784024 (0.0011) [2023-12-26 21:04:09,537][105620] Updated weights for policy 1, policy_version 784034 (0.0011) [2023-12-26 21:04:10,202][105620] Updated weights for policy 1, policy_version 784044 (0.0008) [2023-12-26 21:04:10,268][105620] Updated weights for policy 1, policy_version 784054 (0.0007) [2023-12-26 21:04:10,271][105692] Updated weights for policy 0, policy_version 783953 (0.0008) [2023-12-26 21:04:10,291][105586] KL-divergence is very high: 151.5455 [2023-12-26 21:04:10,324][105620] Updated weights for policy 1, policy_version 784064 (0.0008) [2023-12-26 21:04:10,326][105692] Updated weights for policy 0, policy_version 783963 (0.0005) [2023-12-26 21:04:10,339][105586] KL-divergence is very high: 110.8167 [2023-12-26 21:04:10,385][105692] Updated weights for policy 0, policy_version 783973 (0.0007) [2023-12-26 21:04:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 401473536. Throughput: 0: 9772.4, 1: 9801.8. Samples: 401487096. Policy #0 lag: (min: 12.0, avg: 38.6, max: 40.0) [2023-12-26 21:04:11,062][104569] Avg episode reward: [(0, '8818.519'), (1, '6949.607')] [2023-12-26 21:04:11,078][105620] Updated weights for policy 1, policy_version 784074 (0.0009) [2023-12-26 21:04:11,098][105692] Updated weights for policy 0, policy_version 783983 (0.0009) [2023-12-26 21:04:11,143][105620] Updated weights for policy 1, policy_version 784084 (0.0007) [2023-12-26 21:04:11,169][105692] Updated weights for policy 0, policy_version 783993 (0.0008) [2023-12-26 21:04:11,204][105620] Updated weights for policy 1, policy_version 784094 (0.0006) [2023-12-26 21:04:11,237][105692] Updated weights for policy 0, policy_version 784003 (0.0010) [2023-12-26 21:04:11,266][105620] Updated weights for policy 1, policy_version 784104 (0.0008) [2023-12-26 21:04:11,966][105692] Updated weights for policy 0, policy_version 784013 (0.0008) [2023-12-26 21:04:12,032][105692] Updated weights for policy 0, policy_version 784023 (0.0008) [2023-12-26 21:04:12,064][105620] Updated weights for policy 1, policy_version 784114 (0.0010) [2023-12-26 21:04:12,088][105692] Updated weights for policy 0, policy_version 784033 (0.0009) [2023-12-26 21:04:12,121][105620] Updated weights for policy 1, policy_version 784124 (0.0010) [2023-12-26 21:04:12,176][105620] Updated weights for policy 1, policy_version 784134 (0.0010) [2023-12-26 21:04:12,865][105692] Updated weights for policy 0, policy_version 784043 (0.0009) [2023-12-26 21:04:12,931][105692] Updated weights for policy 0, policy_version 784053 (0.0006) [2023-12-26 21:04:12,973][105620] Updated weights for policy 1, policy_version 784144 (0.0010) [2023-12-26 21:04:12,991][105692] Updated weights for policy 0, policy_version 784063 (0.0006) [2023-12-26 21:04:13,022][105620] Updated weights for policy 1, policy_version 784154 (0.0010) [2023-12-26 21:04:13,086][105620] Updated weights for policy 1, policy_version 784164 (0.0010) [2023-12-26 21:04:13,611][105692] Updated weights for policy 0, policy_version 784073 (0.0008) [2023-12-26 21:04:13,661][105692] Updated weights for policy 0, policy_version 784083 (0.0008) [2023-12-26 21:04:13,716][105692] Updated weights for policy 0, policy_version 784093 (0.0005) [2023-12-26 21:04:13,774][105692] Updated weights for policy 0, policy_version 784103 (0.0005) [2023-12-26 21:04:13,799][105620] Updated weights for policy 1, policy_version 784174 (0.0010) [2023-12-26 21:04:13,863][105620] Updated weights for policy 1, policy_version 784184 (0.0010) [2023-12-26 21:04:13,922][105620] Updated weights for policy 1, policy_version 784194 (0.0010) [2023-12-26 21:04:14,380][105692] Updated weights for policy 0, policy_version 784113 (0.0010) [2023-12-26 21:04:14,432][105692] Updated weights for policy 0, policy_version 784123 (0.0009) [2023-12-26 21:04:14,497][105692] Updated weights for policy 0, policy_version 784133 (0.0006) [2023-12-26 21:04:14,623][105620] Updated weights for policy 1, policy_version 784204 (0.0008) [2023-12-26 21:04:14,678][105620] Updated weights for policy 1, policy_version 784214 (0.0007) [2023-12-26 21:04:14,742][105620] Updated weights for policy 1, policy_version 784224 (0.0009) [2023-12-26 21:04:15,238][105692] Updated weights for policy 0, policy_version 784143 (0.0010) [2023-12-26 21:04:15,299][105692] Updated weights for policy 0, policy_version 784153 (0.0009) [2023-12-26 21:04:15,356][105692] Updated weights for policy 0, policy_version 784163 (0.0011) [2023-12-26 21:04:15,360][105620] Updated weights for policy 1, policy_version 784234 (0.0008) [2023-12-26 21:04:15,420][105620] Updated weights for policy 1, policy_version 784244 (0.0010) [2023-12-26 21:04:15,469][105620] Updated weights for policy 1, policy_version 784254 (0.0008) [2023-12-26 21:04:15,522][105620] Updated weights for policy 1, policy_version 784264 (0.0007) [2023-12-26 21:04:16,015][105692] Updated weights for policy 0, policy_version 784173 (0.0011) [2023-12-26 21:04:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 401571840. Throughput: 0: 9767.9, 1: 9699.5. Samples: 401543984. Policy #0 lag: (min: 12.0, avg: 38.6, max: 40.0) [2023-12-26 21:04:16,062][104569] Avg episode reward: [(0, '8902.180'), (1, '7978.458')] [2023-12-26 21:04:16,067][105692] Updated weights for policy 0, policy_version 784183 (0.0008) [2023-12-26 21:04:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000784264_200794112.pth... [2023-12-26 21:04:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000783112_200499200.pth [2023-12-26 21:04:16,131][105692] Updated weights for policy 0, policy_version 784193 (0.0005) [2023-12-26 21:04:16,175][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000784200_200785920.pth... [2023-12-26 21:04:16,179][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000783048_200491008.pth [2023-12-26 21:04:16,212][105620] Updated weights for policy 1, policy_version 784274 (0.0008) [2023-12-26 21:04:16,266][105620] Updated weights for policy 1, policy_version 784284 (0.0009) [2023-12-26 21:04:16,323][105620] Updated weights for policy 1, policy_version 784294 (0.0009) [2023-12-26 21:04:16,700][105692] Updated weights for policy 0, policy_version 784203 (0.0005) [2023-12-26 21:04:16,766][105692] Updated weights for policy 0, policy_version 784213 (0.0005) [2023-12-26 21:04:16,829][105692] Updated weights for policy 0, policy_version 784223 (0.0005) [2023-12-26 21:04:17,009][105620] Updated weights for policy 1, policy_version 784304 (0.0006) [2023-12-26 21:04:17,071][105620] Updated weights for policy 1, policy_version 784314 (0.0006) [2023-12-26 21:04:17,134][105620] Updated weights for policy 1, policy_version 784324 (0.0008) [2023-12-26 21:04:17,314][105692] Updated weights for policy 0, policy_version 784233 (0.0005) [2023-12-26 21:04:17,377][105692] Updated weights for policy 0, policy_version 784243 (0.0007) [2023-12-26 21:04:17,435][105692] Updated weights for policy 0, policy_version 784253 (0.0010) [2023-12-26 21:04:17,493][105692] Updated weights for policy 0, policy_version 784263 (0.0011) [2023-12-26 21:04:17,865][105620] Updated weights for policy 1, policy_version 784334 (0.0009) [2023-12-26 21:04:17,912][105620] Updated weights for policy 1, policy_version 784344 (0.0007) [2023-12-26 21:04:17,960][105620] Updated weights for policy 1, policy_version 784354 (0.0005) [2023-12-26 21:04:18,183][105692] Updated weights for policy 0, policy_version 784273 (0.0006) [2023-12-26 21:04:18,233][105692] Updated weights for policy 0, policy_version 784283 (0.0005) [2023-12-26 21:04:18,285][105692] Updated weights for policy 0, policy_version 784293 (0.0007) [2023-12-26 21:04:18,729][105620] Updated weights for policy 1, policy_version 784364 (0.0006) [2023-12-26 21:04:18,790][105620] Updated weights for policy 1, policy_version 784374 (0.0008) [2023-12-26 21:04:18,841][105620] Updated weights for policy 1, policy_version 784384 (0.0008) [2023-12-26 21:04:18,992][105692] Updated weights for policy 0, policy_version 784303 (0.0010) [2023-12-26 21:04:19,037][105692] Updated weights for policy 0, policy_version 784313 (0.0010) [2023-12-26 21:04:19,089][105692] Updated weights for policy 0, policy_version 784323 (0.0010) [2023-12-26 21:04:19,636][105620] Updated weights for policy 1, policy_version 784394 (0.0009) [2023-12-26 21:04:19,692][105620] Updated weights for policy 1, policy_version 784404 (0.0010) [2023-12-26 21:04:19,734][105692] Updated weights for policy 0, policy_version 784333 (0.0011) [2023-12-26 21:04:19,760][105620] Updated weights for policy 1, policy_version 784414 (0.0006) [2023-12-26 21:04:19,794][105692] Updated weights for policy 0, policy_version 784343 (0.0011) [2023-12-26 21:04:19,822][105620] Updated weights for policy 1, policy_version 784424 (0.0006) [2023-12-26 21:04:19,859][105692] Updated weights for policy 0, policy_version 784353 (0.0009) [2023-12-26 21:04:20,461][105692] Updated weights for policy 0, policy_version 784363 (0.0006) [2023-12-26 21:04:20,529][105692] Updated weights for policy 0, policy_version 784373 (0.0005) [2023-12-26 21:04:20,596][105692] Updated weights for policy 0, policy_version 784383 (0.0006) [2023-12-26 21:04:20,599][105620] Updated weights for policy 1, policy_version 784434 (0.0008) [2023-12-26 21:04:20,661][105620] Updated weights for policy 1, policy_version 784444 (0.0007) [2023-12-26 21:04:20,724][105620] Updated weights for policy 1, policy_version 784454 (0.0007) [2023-12-26 21:04:21,062][104569] Fps is (10 sec: 20479.4, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 401678336. Throughput: 0: 9827.7, 1: 9695.3. Samples: 401666252. Policy #0 lag: (min: 12.0, avg: 38.6, max: 40.0) [2023-12-26 21:04:21,063][104569] Avg episode reward: [(0, '9082.670'), (1, '9090.029')] [2023-12-26 21:04:21,320][105692] Updated weights for policy 0, policy_version 784393 (0.0008) [2023-12-26 21:04:21,390][105692] Updated weights for policy 0, policy_version 784403 (0.0008) [2023-12-26 21:04:21,452][105692] Updated weights for policy 0, policy_version 784413 (0.0007) [2023-12-26 21:04:21,478][105620] Updated weights for policy 1, policy_version 784464 (0.0009) [2023-12-26 21:04:21,513][105692] Updated weights for policy 0, policy_version 784423 (0.0006) [2023-12-26 21:04:21,540][105620] Updated weights for policy 1, policy_version 784474 (0.0007) [2023-12-26 21:04:21,602][105620] Updated weights for policy 1, policy_version 784484 (0.0008) [2023-12-26 21:04:22,290][105692] Updated weights for policy 0, policy_version 784433 (0.0008) [2023-12-26 21:04:22,356][105692] Updated weights for policy 0, policy_version 784443 (0.0009) [2023-12-26 21:04:22,381][105620] Updated weights for policy 1, policy_version 784494 (0.0008) [2023-12-26 21:04:22,425][105692] Updated weights for policy 0, policy_version 784453 (0.0007) [2023-12-26 21:04:22,435][105620] Updated weights for policy 1, policy_version 784504 (0.0008) [2023-12-26 21:04:22,486][105620] Updated weights for policy 1, policy_version 784514 (0.0008) [2023-12-26 21:04:23,135][105692] Updated weights for policy 0, policy_version 784463 (0.0009) [2023-12-26 21:04:23,199][105692] Updated weights for policy 0, policy_version 784473 (0.0005) [2023-12-26 21:04:23,242][105620] Updated weights for policy 1, policy_version 784524 (0.0009) [2023-12-26 21:04:23,266][105692] Updated weights for policy 0, policy_version 784483 (0.0005) [2023-12-26 21:04:23,308][105620] Updated weights for policy 1, policy_version 784534 (0.0007) [2023-12-26 21:04:23,368][105620] Updated weights for policy 1, policy_version 784544 (0.0009) [2023-12-26 21:04:23,783][105692] Updated weights for policy 0, policy_version 784493 (0.0007) [2023-12-26 21:04:23,833][105692] Updated weights for policy 0, policy_version 784503 (0.0009) [2023-12-26 21:04:23,880][105692] Updated weights for policy 0, policy_version 784513 (0.0009) [2023-12-26 21:04:24,163][105620] Updated weights for policy 1, policy_version 784554 (0.0010) [2023-12-26 21:04:24,214][105620] Updated weights for policy 1, policy_version 784564 (0.0009) [2023-12-26 21:04:24,267][105620] Updated weights for policy 1, policy_version 784575 (0.0010) [2023-12-26 21:04:24,560][105692] Updated weights for policy 0, policy_version 784523 (0.0009) [2023-12-26 21:04:24,618][105692] Updated weights for policy 0, policy_version 784533 (0.0008) [2023-12-26 21:04:24,671][105692] Updated weights for policy 0, policy_version 784543 (0.0008) [2023-12-26 21:04:25,082][105620] Updated weights for policy 1, policy_version 784585 (0.0010) [2023-12-26 21:04:25,151][105620] Updated weights for policy 1, policy_version 784595 (0.0007) [2023-12-26 21:04:25,203][105620] Updated weights for policy 1, policy_version 784605 (0.0005) [2023-12-26 21:04:25,258][105620] Updated weights for policy 1, policy_version 784615 (0.0005) [2023-12-26 21:04:25,263][105692] Updated weights for policy 0, policy_version 784553 (0.0009) [2023-12-26 21:04:25,319][105692] Updated weights for policy 0, policy_version 784563 (0.0005) [2023-12-26 21:04:25,383][105692] Updated weights for policy 0, policy_version 784573 (0.0005) [2023-12-26 21:04:25,442][105692] Updated weights for policy 0, policy_version 784583 (0.0005) [2023-12-26 21:04:25,902][105620] Updated weights for policy 1, policy_version 784625 (0.0006) [2023-12-26 21:04:25,959][105620] Updated weights for policy 1, policy_version 784635 (0.0005) [2023-12-26 21:04:26,024][105620] Updated weights for policy 1, policy_version 784645 (0.0005) [2023-12-26 21:04:26,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 401776640. Throughput: 0: 9887.5, 1: 9701.9. Samples: 401785016. Policy #0 lag: (min: 12.0, avg: 38.6, max: 40.0) [2023-12-26 21:04:26,063][104569] Avg episode reward: [(0, '9264.858'), (1, '9259.968')] [2023-12-26 21:04:26,067][105692] Updated weights for policy 0, policy_version 784593 (0.0009) [2023-12-26 21:04:26,122][105692] Updated weights for policy 0, policy_version 784603 (0.0009) [2023-12-26 21:04:26,180][105692] Updated weights for policy 0, policy_version 784614 (0.0009) [2023-12-26 21:04:26,534][105620] Updated weights for policy 1, policy_version 784655 (0.0006) [2023-12-26 21:04:26,579][105620] Updated weights for policy 1, policy_version 784665 (0.0005) [2023-12-26 21:04:26,631][105620] Updated weights for policy 1, policy_version 784675 (0.0005) [2023-12-26 21:04:26,965][105692] Updated weights for policy 0, policy_version 784624 (0.0010) [2023-12-26 21:04:27,019][105692] Updated weights for policy 0, policy_version 784636 (0.0010) [2023-12-26 21:04:27,074][105692] Updated weights for policy 0, policy_version 784647 (0.0010) [2023-12-26 21:04:27,147][105620] Updated weights for policy 1, policy_version 784685 (0.0005) [2023-12-26 21:04:27,198][105620] Updated weights for policy 1, policy_version 784695 (0.0005) [2023-12-26 21:04:27,260][105620] Updated weights for policy 1, policy_version 784705 (0.0005) [2023-12-26 21:04:27,866][105692] Updated weights for policy 0, policy_version 784657 (0.0008) [2023-12-26 21:04:27,869][105620] Updated weights for policy 1, policy_version 784715 (0.0005) [2023-12-26 21:04:27,914][105692] Updated weights for policy 0, policy_version 784667 (0.0008) [2023-12-26 21:04:27,931][105620] Updated weights for policy 1, policy_version 784725 (0.0006) [2023-12-26 21:04:27,972][105692] Updated weights for policy 0, policy_version 784677 (0.0009) [2023-12-26 21:04:27,993][105620] Updated weights for policy 1, policy_version 784735 (0.0008) [2023-12-26 21:04:28,629][105692] Updated weights for policy 0, policy_version 784687 (0.0007) [2023-12-26 21:04:28,681][105620] Updated weights for policy 1, policy_version 784745 (0.0010) [2023-12-26 21:04:28,691][105692] Updated weights for policy 0, policy_version 784697 (0.0005) [2023-12-26 21:04:28,745][105692] Updated weights for policy 0, policy_version 784707 (0.0005) [2023-12-26 21:04:28,746][105620] Updated weights for policy 1, policy_version 784755 (0.0009) [2023-12-26 21:04:28,801][105620] Updated weights for policy 1, policy_version 784765 (0.0005) [2023-12-26 21:04:28,859][105620] Updated weights for policy 1, policy_version 784775 (0.0008) [2023-12-26 21:04:29,295][105692] Updated weights for policy 0, policy_version 784717 (0.0005) [2023-12-26 21:04:29,366][105692] Updated weights for policy 0, policy_version 784727 (0.0009) [2023-12-26 21:04:29,427][105692] Updated weights for policy 0, policy_version 784737 (0.0011) [2023-12-26 21:04:29,641][105620] Updated weights for policy 1, policy_version 784785 (0.0009) [2023-12-26 21:04:29,698][105620] Updated weights for policy 1, policy_version 784795 (0.0010) [2023-12-26 21:04:29,753][105620] Updated weights for policy 1, policy_version 784805 (0.0010) [2023-12-26 21:04:30,099][105692] Updated weights for policy 0, policy_version 784747 (0.0010) [2023-12-26 21:04:30,160][105692] Updated weights for policy 0, policy_version 784757 (0.0010) [2023-12-26 21:04:30,218][105692] Updated weights for policy 0, policy_version 784767 (0.0010) [2023-12-26 21:04:30,479][105620] Updated weights for policy 1, policy_version 784815 (0.0007) [2023-12-26 21:04:30,540][105620] Updated weights for policy 1, policy_version 784825 (0.0005) [2023-12-26 21:04:30,594][105620] Updated weights for policy 1, policy_version 784835 (0.0005) [2023-12-26 21:04:30,923][105692] Updated weights for policy 0, policy_version 784777 (0.0010) [2023-12-26 21:04:30,976][105692] Updated weights for policy 0, policy_version 784787 (0.0009) [2023-12-26 21:04:31,023][105692] Updated weights for policy 0, policy_version 784797 (0.0010) [2023-12-26 21:04:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 401874944. Throughput: 0: 9876.0, 1: 9831.1. Samples: 401848008. Policy #0 lag: (min: 12.0, avg: 38.6, max: 40.0) [2023-12-26 21:04:31,062][104569] Avg episode reward: [(0, '9267.147'), (1, '9260.430')] [2023-12-26 21:04:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000784840_200941568.pth... [2023-12-26 21:04:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000783688_200646656.pth [2023-12-26 21:04:31,091][105692] Updated weights for policy 0, policy_version 784807 (0.0011) [2023-12-26 21:04:31,096][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000784808_200941568.pth... [2023-12-26 21:04:31,101][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000783624_200638464.pth [2023-12-26 21:04:31,150][105620] Updated weights for policy 1, policy_version 784845 (0.0006) [2023-12-26 21:04:31,212][105620] Updated weights for policy 1, policy_version 784855 (0.0008) [2023-12-26 21:04:31,271][105620] Updated weights for policy 1, policy_version 784865 (0.0008) [2023-12-26 21:04:31,863][105692] Updated weights for policy 0, policy_version 784817 (0.0008) [2023-12-26 21:04:31,931][105692] Updated weights for policy 0, policy_version 784827 (0.0009) [2023-12-26 21:04:31,995][105620] Updated weights for policy 1, policy_version 784875 (0.0008) [2023-12-26 21:04:32,002][105692] Updated weights for policy 0, policy_version 784837 (0.0010) [2023-12-26 21:04:32,054][105620] Updated weights for policy 1, policy_version 784885 (0.0005) [2023-12-26 21:04:32,110][105620] Updated weights for policy 1, policy_version 784895 (0.0006) [2023-12-26 21:04:32,770][105692] Updated weights for policy 0, policy_version 784847 (0.0009) [2023-12-26 21:04:32,801][105620] Updated weights for policy 1, policy_version 784905 (0.0009) [2023-12-26 21:04:32,826][105692] Updated weights for policy 0, policy_version 784857 (0.0008) [2023-12-26 21:04:32,852][105620] Updated weights for policy 1, policy_version 784915 (0.0007) [2023-12-26 21:04:32,878][105692] Updated weights for policy 0, policy_version 784867 (0.0006) [2023-12-26 21:04:32,918][105620] Updated weights for policy 1, policy_version 784925 (0.0006) [2023-12-26 21:04:32,972][105620] Updated weights for policy 1, policy_version 784935 (0.0009) [2023-12-26 21:04:33,600][105620] Updated weights for policy 1, policy_version 784945 (0.0007) [2023-12-26 21:04:33,656][105620] Updated weights for policy 1, policy_version 784955 (0.0005) [2023-12-26 21:04:33,694][105692] Updated weights for policy 0, policy_version 784877 (0.0007) [2023-12-26 21:04:33,715][105620] Updated weights for policy 1, policy_version 784965 (0.0008) [2023-12-26 21:04:33,749][105692] Updated weights for policy 0, policy_version 784887 (0.0007) [2023-12-26 21:04:33,809][105692] Updated weights for policy 0, policy_version 784897 (0.0009) [2023-12-26 21:04:34,411][105620] Updated weights for policy 1, policy_version 784975 (0.0008) [2023-12-26 21:04:34,468][105620] Updated weights for policy 1, policy_version 784986 (0.0010) [2023-12-26 21:04:34,525][105620] Updated weights for policy 1, policy_version 784996 (0.0009) [2023-12-26 21:04:34,536][105692] Updated weights for policy 0, policy_version 784907 (0.0008) [2023-12-26 21:04:34,588][105692] Updated weights for policy 0, policy_version 784917 (0.0009) [2023-12-26 21:04:34,649][105692] Updated weights for policy 0, policy_version 784927 (0.0008) [2023-12-26 21:04:35,323][105620] Updated weights for policy 1, policy_version 785006 (0.0008) [2023-12-26 21:04:35,374][105620] Updated weights for policy 1, policy_version 785016 (0.0010) [2023-12-26 21:04:35,415][105692] Updated weights for policy 0, policy_version 784937 (0.0008) [2023-12-26 21:04:35,430][105620] Updated weights for policy 1, policy_version 785026 (0.0010) [2023-12-26 21:04:35,475][105692] Updated weights for policy 0, policy_version 784947 (0.0006) [2023-12-26 21:04:35,529][105692] Updated weights for policy 0, policy_version 784957 (0.0010) [2023-12-26 21:04:35,587][105692] Updated weights for policy 0, policy_version 784967 (0.0010) [2023-12-26 21:04:36,027][105620] Updated weights for policy 1, policy_version 785036 (0.0010) [2023-12-26 21:04:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 401973248. Throughput: 0: 9914.1, 1: 9770.9. Samples: 401965544. Policy #0 lag: (min: 12.0, avg: 38.6, max: 40.0) [2023-12-26 21:04:36,062][104569] Avg episode reward: [(0, '9089.148'), (1, '9262.636')] [2023-12-26 21:04:36,081][105620] Updated weights for policy 1, policy_version 785046 (0.0010) [2023-12-26 21:04:36,149][105620] Updated weights for policy 1, policy_version 785056 (0.0007) [2023-12-26 21:04:36,283][105692] Updated weights for policy 0, policy_version 784977 (0.0011) [2023-12-26 21:04:36,353][105692] Updated weights for policy 0, policy_version 784987 (0.0011) [2023-12-26 21:04:36,419][105692] Updated weights for policy 0, policy_version 784997 (0.0011) [2023-12-26 21:04:36,745][105620] Updated weights for policy 1, policy_version 785066 (0.0006) [2023-12-26 21:04:36,804][105620] Updated weights for policy 1, policy_version 785076 (0.0005) [2023-12-26 21:04:36,868][105620] Updated weights for policy 1, policy_version 785086 (0.0009) [2023-12-26 21:04:36,933][105620] Updated weights for policy 1, policy_version 785096 (0.0010) [2023-12-26 21:04:37,137][105692] Updated weights for policy 0, policy_version 785007 (0.0007) [2023-12-26 21:04:37,195][105692] Updated weights for policy 0, policy_version 785017 (0.0005) [2023-12-26 21:04:37,256][105692] Updated weights for policy 0, policy_version 785027 (0.0005) [2023-12-26 21:04:37,537][105620] Updated weights for policy 1, policy_version 785106 (0.0010) [2023-12-26 21:04:37,596][105620] Updated weights for policy 1, policy_version 785116 (0.0010) [2023-12-26 21:04:37,652][105620] Updated weights for policy 1, policy_version 785126 (0.0010) [2023-12-26 21:04:37,911][105692] Updated weights for policy 0, policy_version 785037 (0.0007) [2023-12-26 21:04:37,976][105692] Updated weights for policy 0, policy_version 785047 (0.0005) [2023-12-26 21:04:38,038][105692] Updated weights for policy 0, policy_version 785057 (0.0008) [2023-12-26 21:04:38,385][105620] Updated weights for policy 1, policy_version 785136 (0.0010) [2023-12-26 21:04:38,434][105620] Updated weights for policy 1, policy_version 785146 (0.0011) [2023-12-26 21:04:38,483][105620] Updated weights for policy 1, policy_version 785156 (0.0010) [2023-12-26 21:04:38,607][105692] Updated weights for policy 0, policy_version 785067 (0.0006) [2023-12-26 21:04:38,679][105692] Updated weights for policy 0, policy_version 785077 (0.0005) [2023-12-26 21:04:38,742][105692] Updated weights for policy 0, policy_version 785087 (0.0010) [2023-12-26 21:04:39,260][105620] Updated weights for policy 1, policy_version 785166 (0.0008) [2023-12-26 21:04:39,329][105620] Updated weights for policy 1, policy_version 785176 (0.0007) [2023-12-26 21:04:39,396][105620] Updated weights for policy 1, policy_version 785186 (0.0008) [2023-12-26 21:04:39,494][105692] Updated weights for policy 0, policy_version 785097 (0.0010) [2023-12-26 21:04:39,557][105692] Updated weights for policy 0, policy_version 785107 (0.0011) [2023-12-26 21:04:39,628][105692] Updated weights for policy 0, policy_version 785117 (0.0010) [2023-12-26 21:04:39,690][105692] Updated weights for policy 0, policy_version 785127 (0.0011) [2023-12-26 21:04:40,103][105620] Updated weights for policy 1, policy_version 785196 (0.0007) [2023-12-26 21:04:40,164][105620] Updated weights for policy 1, policy_version 785206 (0.0008) [2023-12-26 21:04:40,223][105620] Updated weights for policy 1, policy_version 785216 (0.0008) [2023-12-26 21:04:40,402][105692] Updated weights for policy 0, policy_version 785137 (0.0011) [2023-12-26 21:04:40,469][105692] Updated weights for policy 0, policy_version 785147 (0.0011) [2023-12-26 21:04:40,528][105692] Updated weights for policy 0, policy_version 785157 (0.0011) [2023-12-26 21:04:41,020][105620] Updated weights for policy 1, policy_version 785226 (0.0008) [2023-12-26 21:04:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 402071552. Throughput: 0: 9921.6, 1: 9833.2. Samples: 402084596. Policy #0 lag: (min: 12.0, avg: 38.6, max: 40.0) [2023-12-26 21:04:41,063][104569] Avg episode reward: [(0, '8913.336'), (1, '9082.334')] [2023-12-26 21:04:41,078][105620] Updated weights for policy 1, policy_version 785236 (0.0010) [2023-12-26 21:04:41,134][105620] Updated weights for policy 1, policy_version 785246 (0.0011) [2023-12-26 21:04:41,148][105692] Updated weights for policy 0, policy_version 785167 (0.0011) [2023-12-26 21:04:41,194][105620] Updated weights for policy 1, policy_version 785256 (0.0011) [2023-12-26 21:04:41,208][105692] Updated weights for policy 0, policy_version 785177 (0.0008) [2023-12-26 21:04:41,274][105692] Updated weights for policy 0, policy_version 785187 (0.0007) [2023-12-26 21:04:41,988][105620] Updated weights for policy 1, policy_version 785266 (0.0009) [2023-12-26 21:04:42,046][105692] Updated weights for policy 0, policy_version 785197 (0.0006) [2023-12-26 21:04:42,049][105620] Updated weights for policy 1, policy_version 785276 (0.0007) [2023-12-26 21:04:42,105][105692] Updated weights for policy 0, policy_version 785207 (0.0006) [2023-12-26 21:04:42,110][105620] Updated weights for policy 1, policy_version 785286 (0.0007) [2023-12-26 21:04:42,166][105692] Updated weights for policy 0, policy_version 785217 (0.0009) [2023-12-26 21:04:42,757][105620] Updated weights for policy 1, policy_version 785296 (0.0010) [2023-12-26 21:04:42,815][105620] Updated weights for policy 1, policy_version 785306 (0.0010) [2023-12-26 21:04:42,875][105620] Updated weights for policy 1, policy_version 785316 (0.0010) [2023-12-26 21:04:43,007][105692] Updated weights for policy 0, policy_version 785227 (0.0009) [2023-12-26 21:04:43,067][105692] Updated weights for policy 0, policy_version 785237 (0.0008) [2023-12-26 21:04:43,123][105692] Updated weights for policy 0, policy_version 785247 (0.0008) [2023-12-26 21:04:43,608][105620] Updated weights for policy 1, policy_version 785326 (0.0010) [2023-12-26 21:04:43,680][105620] Updated weights for policy 1, policy_version 785336 (0.0010) [2023-12-26 21:04:43,741][105620] Updated weights for policy 1, policy_version 785346 (0.0010) [2023-12-26 21:04:43,750][105692] Updated weights for policy 0, policy_version 785257 (0.0008) [2023-12-26 21:04:43,807][105692] Updated weights for policy 0, policy_version 785267 (0.0005) [2023-12-26 21:04:43,855][105692] Updated weights for policy 0, policy_version 785277 (0.0006) [2023-12-26 21:04:43,897][105692] Updated weights for policy 0, policy_version 785287 (0.0006) [2023-12-26 21:04:44,337][105620] Updated weights for policy 1, policy_version 785356 (0.0010) [2023-12-26 21:04:44,393][105620] Updated weights for policy 1, policy_version 785366 (0.0009) [2023-12-26 21:04:44,440][105620] Updated weights for policy 1, policy_version 785376 (0.0008) [2023-12-26 21:04:44,486][105692] Updated weights for policy 0, policy_version 785297 (0.0005) [2023-12-26 21:04:44,546][105692] Updated weights for policy 0, policy_version 785307 (0.0005) [2023-12-26 21:04:44,614][105692] Updated weights for policy 0, policy_version 785317 (0.0007) [2023-12-26 21:04:45,268][105620] Updated weights for policy 1, policy_version 785386 (0.0009) [2023-12-26 21:04:45,326][105692] Updated weights for policy 0, policy_version 785327 (0.0007) [2023-12-26 21:04:45,328][105620] Updated weights for policy 1, policy_version 785396 (0.0009) [2023-12-26 21:04:45,388][105692] Updated weights for policy 0, policy_version 785337 (0.0007) [2023-12-26 21:04:45,390][105620] Updated weights for policy 1, policy_version 785406 (0.0007) [2023-12-26 21:04:45,452][105620] Updated weights for policy 1, policy_version 785416 (0.0007) [2023-12-26 21:04:45,453][105692] Updated weights for policy 0, policy_version 785347 (0.0009) [2023-12-26 21:04:46,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 402169856. Throughput: 0: 9829.8, 1: 9864.3. Samples: 402141796. Policy #0 lag: (min: 12.0, avg: 38.6, max: 40.0) [2023-12-26 21:04:46,063][104569] Avg episode reward: [(0, '8999.115'), (1, '9174.264')] [2023-12-26 21:04:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000785352_201080832.pth... [2023-12-26 21:04:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000785416_201089024.pth... [2023-12-26 21:04:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000784200_200785920.pth [2023-12-26 21:04:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000784264_200794112.pth [2023-12-26 21:04:46,180][105620] Updated weights for policy 1, policy_version 785426 (0.0008) [2023-12-26 21:04:46,232][105692] Updated weights for policy 0, policy_version 785357 (0.0008) [2023-12-26 21:04:46,238][105620] Updated weights for policy 1, policy_version 785436 (0.0008) [2023-12-26 21:04:46,277][105692] Updated weights for policy 0, policy_version 785367 (0.0006) [2023-12-26 21:04:46,280][105620] Updated weights for policy 1, policy_version 785446 (0.0006) [2023-12-26 21:04:46,322][105692] Updated weights for policy 0, policy_version 785377 (0.0007) [2023-12-26 21:04:46,941][105692] Updated weights for policy 0, policy_version 785387 (0.0006) [2023-12-26 21:04:46,999][105692] Updated weights for policy 0, policy_version 785397 (0.0010) [2023-12-26 21:04:47,054][105692] Updated weights for policy 0, policy_version 785407 (0.0007) [2023-12-26 21:04:47,107][105620] Updated weights for policy 1, policy_version 785456 (0.0008) [2023-12-26 21:04:47,159][105620] Updated weights for policy 1, policy_version 785466 (0.0010) [2023-12-26 21:04:47,217][105620] Updated weights for policy 1, policy_version 785476 (0.0009) [2023-12-26 21:04:47,617][105692] Updated weights for policy 0, policy_version 785417 (0.0005) [2023-12-26 21:04:47,665][105692] Updated weights for policy 0, policy_version 785427 (0.0007) [2023-12-26 21:04:47,724][105692] Updated weights for policy 0, policy_version 785437 (0.0007) [2023-12-26 21:04:47,784][105692] Updated weights for policy 0, policy_version 785447 (0.0005) [2023-12-26 21:04:48,133][105620] Updated weights for policy 1, policy_version 785487 (0.0009) [2023-12-26 21:04:48,194][105620] Updated weights for policy 1, policy_version 785497 (0.0009) [2023-12-26 21:04:48,259][105620] Updated weights for policy 1, policy_version 785507 (0.0009) [2023-12-26 21:04:48,327][105692] Updated weights for policy 0, policy_version 785457 (0.0006) [2023-12-26 21:04:48,393][105692] Updated weights for policy 0, policy_version 785467 (0.0007) [2023-12-26 21:04:48,458][105692] Updated weights for policy 0, policy_version 785477 (0.0007) [2023-12-26 21:04:49,056][105692] Updated weights for policy 0, policy_version 785487 (0.0006) [2023-12-26 21:04:49,075][105620] Updated weights for policy 1, policy_version 785517 (0.0008) [2023-12-26 21:04:49,122][105692] Updated weights for policy 0, policy_version 785497 (0.0008) [2023-12-26 21:04:49,132][105620] Updated weights for policy 1, policy_version 785527 (0.0008) [2023-12-26 21:04:49,178][105692] Updated weights for policy 0, policy_version 785507 (0.0007) [2023-12-26 21:04:49,191][105620] Updated weights for policy 1, policy_version 785537 (0.0007) [2023-12-26 21:04:49,838][105692] Updated weights for policy 0, policy_version 785517 (0.0008) [2023-12-26 21:04:49,898][105692] Updated weights for policy 0, policy_version 785527 (0.0009) [2023-12-26 21:04:49,919][105585] KL-divergence is very high: 309.8629 [2023-12-26 21:04:49,968][105692] Updated weights for policy 0, policy_version 785537 (0.0007) [2023-12-26 21:04:49,974][105585] KL-divergence is very high: 476.5199 [2023-12-26 21:04:49,980][105620] Updated weights for policy 1, policy_version 785547 (0.0008) [2023-12-26 21:04:50,047][105620] Updated weights for policy 1, policy_version 785557 (0.0008) [2023-12-26 21:04:50,113][105620] Updated weights for policy 1, policy_version 785567 (0.0007) [2023-12-26 21:04:50,670][105692] Updated weights for policy 0, policy_version 785547 (0.0008) [2023-12-26 21:04:50,727][105692] Updated weights for policy 0, policy_version 785557 (0.0009) [2023-12-26 21:04:50,787][105692] Updated weights for policy 0, policy_version 785567 (0.0006) [2023-12-26 21:04:50,808][105620] Updated weights for policy 1, policy_version 785577 (0.0006) [2023-12-26 21:04:50,861][105620] Updated weights for policy 1, policy_version 785587 (0.0010) [2023-12-26 21:04:50,922][105620] Updated weights for policy 1, policy_version 785597 (0.0005) [2023-12-26 21:04:50,986][105620] Updated weights for policy 1, policy_version 785607 (0.0007) [2023-12-26 21:04:51,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 402276352. Throughput: 0: 10024.4, 1: 9721.1. Samples: 402261056. Policy #0 lag: (min: 12.0, avg: 38.6, max: 40.0) [2023-12-26 21:04:51,062][104569] Avg episode reward: [(0, '9174.066'), (1, '9264.610')] [2023-12-26 21:04:51,522][105692] Updated weights for policy 0, policy_version 785577 (0.0007) [2023-12-26 21:04:51,586][105692] Updated weights for policy 0, policy_version 785587 (0.0011) [2023-12-26 21:04:51,657][105692] Updated weights for policy 0, policy_version 785597 (0.0010) [2023-12-26 21:04:51,687][105620] Updated weights for policy 1, policy_version 785617 (0.0006) [2023-12-26 21:04:51,719][105692] Updated weights for policy 0, policy_version 785607 (0.0011) [2023-12-26 21:04:51,758][105620] Updated weights for policy 1, policy_version 785627 (0.0008) [2023-12-26 21:04:51,814][105620] Updated weights for policy 1, policy_version 785637 (0.0008) [2023-12-26 21:04:52,391][105692] Updated weights for policy 0, policy_version 785617 (0.0008) [2023-12-26 21:04:52,452][105692] Updated weights for policy 0, policy_version 785627 (0.0008) [2023-12-26 21:04:52,516][105692] Updated weights for policy 0, policy_version 785637 (0.0009) [2023-12-26 21:04:52,538][105620] Updated weights for policy 1, policy_version 785647 (0.0010) [2023-12-26 21:04:52,590][105620] Updated weights for policy 1, policy_version 785657 (0.0011) [2023-12-26 21:04:52,642][105620] Updated weights for policy 1, policy_version 785667 (0.0010) [2023-12-26 21:04:53,252][105692] Updated weights for policy 0, policy_version 785647 (0.0008) [2023-12-26 21:04:53,255][105620] Updated weights for policy 1, policy_version 785677 (0.0008) [2023-12-26 21:04:53,300][105692] Updated weights for policy 0, policy_version 785657 (0.0009) [2023-12-26 21:04:53,306][105620] Updated weights for policy 1, policy_version 785687 (0.0005) [2023-12-26 21:04:53,355][105692] Updated weights for policy 0, policy_version 785667 (0.0007) [2023-12-26 21:04:53,365][105620] Updated weights for policy 1, policy_version 785697 (0.0008) [2023-12-26 21:04:53,932][105692] Updated weights for policy 0, policy_version 785677 (0.0006) [2023-12-26 21:04:53,987][105692] Updated weights for policy 0, policy_version 785687 (0.0008) [2023-12-26 21:04:54,042][105692] Updated weights for policy 0, policy_version 785697 (0.0010) [2023-12-26 21:04:54,074][105620] Updated weights for policy 1, policy_version 785707 (0.0009) [2023-12-26 21:04:54,126][105620] Updated weights for policy 1, policy_version 785717 (0.0005) [2023-12-26 21:04:54,180][105620] Updated weights for policy 1, policy_version 785727 (0.0006) [2023-12-26 21:04:54,683][105692] Updated weights for policy 0, policy_version 785707 (0.0010) [2023-12-26 21:04:54,738][105692] Updated weights for policy 0, policy_version 785717 (0.0010) [2023-12-26 21:04:54,787][105692] Updated weights for policy 0, policy_version 785727 (0.0010) [2023-12-26 21:04:54,885][105620] Updated weights for policy 1, policy_version 785737 (0.0006) [2023-12-26 21:04:54,956][105620] Updated weights for policy 1, policy_version 785747 (0.0007) [2023-12-26 21:04:55,016][105620] Updated weights for policy 1, policy_version 785757 (0.0006) [2023-12-26 21:04:55,073][105620] Updated weights for policy 1, policy_version 785767 (0.0005) [2023-12-26 21:04:55,392][105692] Updated weights for policy 0, policy_version 785737 (0.0010) [2023-12-26 21:04:55,447][105692] Updated weights for policy 0, policy_version 785747 (0.0010) [2023-12-26 21:04:55,503][105692] Updated weights for policy 0, policy_version 785757 (0.0010) [2023-12-26 21:04:55,559][105692] Updated weights for policy 0, policy_version 785767 (0.0010) [2023-12-26 21:04:55,769][105620] Updated weights for policy 1, policy_version 785777 (0.0008) [2023-12-26 21:04:55,825][105620] Updated weights for policy 1, policy_version 785787 (0.0007) [2023-12-26 21:04:55,878][105620] Updated weights for policy 1, policy_version 785797 (0.0008) [2023-12-26 21:04:56,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 402374656. Throughput: 0: 10162.3, 1: 9722.9. Samples: 402381932. Policy #0 lag: (min: 12.0, avg: 38.6, max: 40.0) [2023-12-26 21:04:56,063][104569] Avg episode reward: [(0, '9081.355'), (1, '9082.207')] [2023-12-26 21:04:56,225][105692] Updated weights for policy 0, policy_version 785777 (0.0010) [2023-12-26 21:04:56,281][105692] Updated weights for policy 0, policy_version 785787 (0.0007) [2023-12-26 21:04:56,334][105692] Updated weights for policy 0, policy_version 785797 (0.0005) [2023-12-26 21:04:56,663][105620] Updated weights for policy 1, policy_version 785807 (0.0008) [2023-12-26 21:04:56,725][105620] Updated weights for policy 1, policy_version 785817 (0.0010) [2023-12-26 21:04:56,784][105620] Updated weights for policy 1, policy_version 785828 (0.0011) [2023-12-26 21:04:56,877][105692] Updated weights for policy 0, policy_version 785807 (0.0005) [2023-12-26 21:04:56,926][105692] Updated weights for policy 0, policy_version 785817 (0.0010) [2023-12-26 21:04:56,971][105692] Updated weights for policy 0, policy_version 785827 (0.0009) [2023-12-26 21:04:57,448][105620] Updated weights for policy 1, policy_version 785839 (0.0007) [2023-12-26 21:04:57,502][105620] Updated weights for policy 1, policy_version 785849 (0.0008) [2023-12-26 21:04:57,559][105620] Updated weights for policy 1, policy_version 785859 (0.0008) [2023-12-26 21:04:57,663][105692] Updated weights for policy 0, policy_version 785837 (0.0008) [2023-12-26 21:04:57,721][105692] Updated weights for policy 0, policy_version 785847 (0.0010) [2023-12-26 21:04:57,778][105692] Updated weights for policy 0, policy_version 785857 (0.0007) [2023-12-26 21:04:58,299][105620] Updated weights for policy 1, policy_version 785869 (0.0008) [2023-12-26 21:04:58,366][105620] Updated weights for policy 1, policy_version 785879 (0.0009) [2023-12-26 21:04:58,411][105692] Updated weights for policy 0, policy_version 785867 (0.0007) [2023-12-26 21:04:58,431][105620] Updated weights for policy 1, policy_version 785889 (0.0008) [2023-12-26 21:04:58,470][105692] Updated weights for policy 0, policy_version 785877 (0.0008) [2023-12-26 21:04:58,535][105692] Updated weights for policy 0, policy_version 785887 (0.0007) [2023-12-26 21:04:59,207][105620] Updated weights for policy 1, policy_version 785899 (0.0009) [2023-12-26 21:04:59,276][105620] Updated weights for policy 1, policy_version 785909 (0.0007) [2023-12-26 21:04:59,343][105620] Updated weights for policy 1, policy_version 785919 (0.0008) [2023-12-26 21:04:59,395][105692] Updated weights for policy 0, policy_version 785897 (0.0008) [2023-12-26 21:04:59,461][105692] Updated weights for policy 0, policy_version 785907 (0.0006) [2023-12-26 21:04:59,522][105692] Updated weights for policy 0, policy_version 785917 (0.0005) [2023-12-26 21:04:59,587][105692] Updated weights for policy 0, policy_version 785927 (0.0007) [2023-12-26 21:04:59,977][105620] Updated weights for policy 1, policy_version 785929 (0.0008) [2023-12-26 21:05:00,044][105620] Updated weights for policy 1, policy_version 785939 (0.0007) [2023-12-26 21:05:00,108][105620] Updated weights for policy 1, policy_version 785949 (0.0005) [2023-12-26 21:05:00,165][105620] Updated weights for policy 1, policy_version 785959 (0.0008) [2023-12-26 21:05:00,228][105692] Updated weights for policy 0, policy_version 785937 (0.0008) [2023-12-26 21:05:00,281][105692] Updated weights for policy 0, policy_version 785947 (0.0009) [2023-12-26 21:05:00,347][105692] Updated weights for policy 0, policy_version 785957 (0.0010) [2023-12-26 21:05:00,708][105620] Updated weights for policy 1, policy_version 785969 (0.0010) [2023-12-26 21:05:00,756][105620] Updated weights for policy 1, policy_version 785979 (0.0005) [2023-12-26 21:05:00,810][105620] Updated weights for policy 1, policy_version 785989 (0.0005) [2023-12-26 21:05:01,030][105692] Updated weights for policy 0, policy_version 785967 (0.0008) [2023-12-26 21:05:01,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 402472960. Throughput: 0: 10240.8, 1: 9743.1. Samples: 402443264. Policy #0 lag: (min: 12.0, avg: 38.6, max: 40.0) [2023-12-26 21:05:01,063][104569] Avg episode reward: [(0, '9171.209'), (1, '9086.161')] [2023-12-26 21:05:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000785992_201236480.pth... [2023-12-26 21:05:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000784840_200941568.pth [2023-12-26 21:05:01,090][105692] Updated weights for policy 0, policy_version 785977 (0.0009) [2023-12-26 21:05:01,165][105692] Updated weights for policy 0, policy_version 785987 (0.0010) [2023-12-26 21:05:01,192][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000785992_201244672.pth... [2023-12-26 21:05:01,197][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000784808_200941568.pth [2023-12-26 21:05:01,478][105620] Updated weights for policy 1, policy_version 785999 (0.0009) [2023-12-26 21:05:01,508][105586] KL-divergence is very high: 127.1498 [2023-12-26 21:05:01,539][105620] Updated weights for policy 1, policy_version 786009 (0.0010) [2023-12-26 21:05:01,558][105586] KL-divergence is very high: 193.6983 [2023-12-26 21:05:01,585][105620] Updated weights for policy 1, policy_version 786019 (0.0007) [2023-12-26 21:05:01,594][105586] KL-divergence is very high: 148.6553 [2023-12-26 21:05:01,894][105692] Updated weights for policy 0, policy_version 785997 (0.0010) [2023-12-26 21:05:01,946][105692] Updated weights for policy 0, policy_version 786007 (0.0008) [2023-12-26 21:05:01,998][105692] Updated weights for policy 0, policy_version 786017 (0.0009) [2023-12-26 21:05:02,349][105620] Updated weights for policy 1, policy_version 786029 (0.0008) [2023-12-26 21:05:02,418][105620] Updated weights for policy 1, policy_version 786039 (0.0008) [2023-12-26 21:05:02,475][105620] Updated weights for policy 1, policy_version 786049 (0.0009) [2023-12-26 21:05:02,722][105692] Updated weights for policy 0, policy_version 786027 (0.0008) [2023-12-26 21:05:02,783][105692] Updated weights for policy 0, policy_version 786037 (0.0008) [2023-12-26 21:05:02,840][105692] Updated weights for policy 0, policy_version 786047 (0.0007) [2023-12-26 21:05:03,320][105620] Updated weights for policy 1, policy_version 786059 (0.0009) [2023-12-26 21:05:03,372][105620] Updated weights for policy 1, policy_version 786069 (0.0009) [2023-12-26 21:05:03,396][105692] Updated weights for policy 0, policy_version 786057 (0.0005) [2023-12-26 21:05:03,422][105620] Updated weights for policy 1, policy_version 786080 (0.0010) [2023-12-26 21:05:03,449][105692] Updated weights for policy 0, policy_version 786067 (0.0005) [2023-12-26 21:05:03,499][105692] Updated weights for policy 0, policy_version 786077 (0.0005) [2023-12-26 21:05:03,562][105692] Updated weights for policy 0, policy_version 786087 (0.0005) [2023-12-26 21:05:04,111][105692] Updated weights for policy 0, policy_version 786097 (0.0006) [2023-12-26 21:05:04,166][105692] Updated weights for policy 0, policy_version 786107 (0.0010) [2023-12-26 21:05:04,225][105692] Updated weights for policy 0, policy_version 786117 (0.0011) [2023-12-26 21:05:04,256][105620] Updated weights for policy 1, policy_version 786090 (0.0008) [2023-12-26 21:05:04,317][105620] Updated weights for policy 1, policy_version 786100 (0.0009) [2023-12-26 21:05:04,378][105620] Updated weights for policy 1, policy_version 786110 (0.0009) [2023-12-26 21:05:04,429][105620] Updated weights for policy 1, policy_version 786120 (0.0010) [2023-12-26 21:05:04,952][105692] Updated weights for policy 0, policy_version 786127 (0.0011) [2023-12-26 21:05:05,004][105692] Updated weights for policy 0, policy_version 786137 (0.0010) [2023-12-26 21:05:05,059][105692] Updated weights for policy 0, policy_version 786147 (0.0010) [2023-12-26 21:05:05,075][105620] Updated weights for policy 1, policy_version 786130 (0.0005) [2023-12-26 21:05:05,131][105620] Updated weights for policy 1, policy_version 786140 (0.0005) [2023-12-26 21:05:05,195][105620] Updated weights for policy 1, policy_version 786150 (0.0005) [2023-12-26 21:05:05,701][105692] Updated weights for policy 0, policy_version 786157 (0.0010) [2023-12-26 21:05:05,759][105692] Updated weights for policy 0, policy_version 786167 (0.0010) [2023-12-26 21:05:05,784][105620] Updated weights for policy 1, policy_version 786160 (0.0007) [2023-12-26 21:05:05,810][105692] Updated weights for policy 0, policy_version 786177 (0.0010) [2023-12-26 21:05:05,845][105620] Updated weights for policy 1, policy_version 786170 (0.0008) [2023-12-26 21:05:05,916][105620] Updated weights for policy 1, policy_version 786180 (0.0008) [2023-12-26 21:05:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19933.8, 300 sec: 19605.3). Total num frames: 402579456. Throughput: 0: 10176.6, 1: 9738.3. Samples: 402562416. Policy #0 lag: (min: 12.0, avg: 38.6, max: 40.0) [2023-12-26 21:05:06,063][104569] Avg episode reward: [(0, '9082.756'), (1, '9176.754')] [2023-12-26 21:05:06,568][105692] Updated weights for policy 0, policy_version 786187 (0.0010) [2023-12-26 21:05:06,619][105620] Updated weights for policy 1, policy_version 786190 (0.0007) [2023-12-26 21:05:06,633][105692] Updated weights for policy 0, policy_version 786197 (0.0011) [2023-12-26 21:05:06,684][105620] Updated weights for policy 1, policy_version 786200 (0.0006) [2023-12-26 21:05:06,690][105692] Updated weights for policy 0, policy_version 786207 (0.0011) [2023-12-26 21:05:06,744][105620] Updated weights for policy 1, policy_version 786210 (0.0007) [2023-12-26 21:05:07,385][105692] Updated weights for policy 0, policy_version 786217 (0.0010) [2023-12-26 21:05:07,423][105585] KL-divergence is very high: 124.6774 [2023-12-26 21:05:07,440][105692] Updated weights for policy 0, policy_version 786227 (0.0008) [2023-12-26 21:05:07,471][105585] KL-divergence is very high: 205.2532 [2023-12-26 21:05:07,486][105620] Updated weights for policy 1, policy_version 786220 (0.0009) [2023-12-26 21:05:07,501][105692] Updated weights for policy 0, policy_version 786237 (0.0010) [2023-12-26 21:05:07,517][105585] KL-divergence is very high: 217.3040 [2023-12-26 21:05:07,540][105620] Updated weights for policy 1, policy_version 786230 (0.0010) [2023-12-26 21:05:07,557][105692] Updated weights for policy 0, policy_version 786247 (0.0010) [2023-12-26 21:05:07,590][105620] Updated weights for policy 1, policy_version 786240 (0.0010) [2023-12-26 21:05:08,132][105692] Updated weights for policy 0, policy_version 786257 (0.0006) [2023-12-26 21:05:08,197][105692] Updated weights for policy 0, policy_version 786267 (0.0006) [2023-12-26 21:05:08,242][105620] Updated weights for policy 1, policy_version 786250 (0.0010) [2023-12-26 21:05:08,249][105692] Updated weights for policy 0, policy_version 786277 (0.0005) [2023-12-26 21:05:08,291][105620] Updated weights for policy 1, policy_version 786260 (0.0010) [2023-12-26 21:05:08,350][105620] Updated weights for policy 1, policy_version 786270 (0.0010) [2023-12-26 21:05:08,410][105620] Updated weights for policy 1, policy_version 786280 (0.0011) [2023-12-26 21:05:08,914][105692] Updated weights for policy 0, policy_version 786287 (0.0009) [2023-12-26 21:05:08,973][105692] Updated weights for policy 0, policy_version 786297 (0.0011) [2023-12-26 21:05:09,030][105692] Updated weights for policy 0, policy_version 786307 (0.0011) [2023-12-26 21:05:09,131][105620] Updated weights for policy 1, policy_version 786290 (0.0008) [2023-12-26 21:05:09,196][105620] Updated weights for policy 1, policy_version 786300 (0.0008) [2023-12-26 21:05:09,261][105620] Updated weights for policy 1, policy_version 786310 (0.0009) [2023-12-26 21:05:09,840][105692] Updated weights for policy 0, policy_version 786317 (0.0009) [2023-12-26 21:05:09,915][105692] Updated weights for policy 0, policy_version 786327 (0.0006) [2023-12-26 21:05:09,977][105692] Updated weights for policy 0, policy_version 786337 (0.0006) [2023-12-26 21:05:10,059][105620] Updated weights for policy 1, policy_version 786320 (0.0009) [2023-12-26 21:05:10,127][105620] Updated weights for policy 1, policy_version 786330 (0.0009) [2023-12-26 21:05:10,190][105620] Updated weights for policy 1, policy_version 786340 (0.0009) [2023-12-26 21:05:10,678][105692] Updated weights for policy 0, policy_version 786347 (0.0007) [2023-12-26 21:05:10,742][105692] Updated weights for policy 0, policy_version 786357 (0.0008) [2023-12-26 21:05:10,802][105692] Updated weights for policy 0, policy_version 786367 (0.0008) [2023-12-26 21:05:10,907][105620] Updated weights for policy 1, policy_version 786350 (0.0010) [2023-12-26 21:05:10,960][105620] Updated weights for policy 1, policy_version 786360 (0.0010) [2023-12-26 21:05:11,016][105620] Updated weights for policy 1, policy_version 786370 (0.0010) [2023-12-26 21:05:11,062][104569] Fps is (10 sec: 20480.3, 60 sec: 20070.4, 300 sec: 19605.3). Total num frames: 402677760. Throughput: 0: 10098.9, 1: 9819.5. Samples: 402681344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:05:11,063][104569] Avg episode reward: [(0, '8991.727'), (1, '9172.409')] [2023-12-26 21:05:11,551][105692] Updated weights for policy 0, policy_version 786377 (0.0007) [2023-12-26 21:05:11,610][105692] Updated weights for policy 0, policy_version 786387 (0.0006) [2023-12-26 21:05:11,674][105692] Updated weights for policy 0, policy_version 786397 (0.0006) [2023-12-26 21:05:11,740][105692] Updated weights for policy 0, policy_version 786407 (0.0008) [2023-12-26 21:05:11,841][105620] Updated weights for policy 1, policy_version 786380 (0.0012) [2023-12-26 21:05:11,899][105620] Updated weights for policy 1, policy_version 786390 (0.0011) [2023-12-26 21:05:11,951][105620] Updated weights for policy 1, policy_version 786400 (0.0010) [2023-12-26 21:05:12,411][105692] Updated weights for policy 0, policy_version 786417 (0.0008) [2023-12-26 21:05:12,473][105692] Updated weights for policy 0, policy_version 786427 (0.0009) [2023-12-26 21:05:12,533][105692] Updated weights for policy 0, policy_version 786438 (0.0009) [2023-12-26 21:05:12,652][105620] Updated weights for policy 1, policy_version 786410 (0.0006) [2023-12-26 21:05:12,715][105620] Updated weights for policy 1, policy_version 786420 (0.0007) [2023-12-26 21:05:12,771][105620] Updated weights for policy 1, policy_version 786430 (0.0008) [2023-12-26 21:05:12,816][105620] Updated weights for policy 1, policy_version 786440 (0.0008) [2023-12-26 21:05:13,326][105692] Updated weights for policy 0, policy_version 786448 (0.0009) [2023-12-26 21:05:13,381][105692] Updated weights for policy 0, policy_version 786458 (0.0009) [2023-12-26 21:05:13,439][105692] Updated weights for policy 0, policy_version 786468 (0.0010) [2023-12-26 21:05:13,537][105620] Updated weights for policy 1, policy_version 786450 (0.0007) [2023-12-26 21:05:13,600][105620] Updated weights for policy 1, policy_version 786460 (0.0008) [2023-12-26 21:05:13,659][105620] Updated weights for policy 1, policy_version 786470 (0.0007) [2023-12-26 21:05:14,244][105692] Updated weights for policy 0, policy_version 786478 (0.0009) [2023-12-26 21:05:14,308][105692] Updated weights for policy 0, policy_version 786488 (0.0010) [2023-12-26 21:05:14,373][105620] Updated weights for policy 1, policy_version 786480 (0.0006) [2023-12-26 21:05:14,374][105692] Updated weights for policy 0, policy_version 786498 (0.0010) [2023-12-26 21:05:14,436][105620] Updated weights for policy 1, policy_version 786490 (0.0005) [2023-12-26 21:05:14,502][105620] Updated weights for policy 1, policy_version 786500 (0.0005) [2023-12-26 21:05:15,099][105692] Updated weights for policy 0, policy_version 786508 (0.0009) [2023-12-26 21:05:15,139][105620] Updated weights for policy 1, policy_version 786510 (0.0008) [2023-12-26 21:05:15,158][105692] Updated weights for policy 0, policy_version 786518 (0.0009) [2023-12-26 21:05:15,192][105620] Updated weights for policy 1, policy_version 786520 (0.0010) [2023-12-26 21:05:15,215][105692] Updated weights for policy 0, policy_version 786528 (0.0006) [2023-12-26 21:05:15,252][105620] Updated weights for policy 1, policy_version 786530 (0.0011) [2023-12-26 21:05:15,920][105620] Updated weights for policy 1, policy_version 786540 (0.0010) [2023-12-26 21:05:15,961][105692] Updated weights for policy 0, policy_version 786538 (0.0006) [2023-12-26 21:05:15,982][105620] Updated weights for policy 1, policy_version 786550 (0.0011) [2023-12-26 21:05:16,021][105692] Updated weights for policy 0, policy_version 786548 (0.0006) [2023-12-26 21:05:16,037][105620] Updated weights for policy 1, policy_version 786560 (0.0010) [2023-12-26 21:05:16,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 402759680. Throughput: 0: 10065.0, 1: 9687.8. Samples: 402736880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:05:16,062][104569] Avg episode reward: [(0, '8462.143'), (1, '9172.561')] [2023-12-26 21:05:16,083][105692] Updated weights for policy 0, policy_version 786558 (0.0006) [2023-12-26 21:05:16,083][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000786568_201383936.pth... [2023-12-26 21:05:16,088][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000785416_201089024.pth [2023-12-26 21:05:16,146][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000786568_201392128.pth... [2023-12-26 21:05:16,147][105692] Updated weights for policy 0, policy_version 786568 (0.0005) [2023-12-26 21:05:16,151][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000785352_201080832.pth [2023-12-26 21:05:16,656][105620] Updated weights for policy 1, policy_version 786570 (0.0009) [2023-12-26 21:05:16,720][105620] Updated weights for policy 1, policy_version 786580 (0.0007) [2023-12-26 21:05:16,785][105620] Updated weights for policy 1, policy_version 786590 (0.0010) [2023-12-26 21:05:16,811][105692] Updated weights for policy 0, policy_version 786578 (0.0007) [2023-12-26 21:05:16,850][105620] Updated weights for policy 1, policy_version 786600 (0.0010) [2023-12-26 21:05:16,867][105692] Updated weights for policy 0, policy_version 786588 (0.0006) [2023-12-26 21:05:16,926][105692] Updated weights for policy 0, policy_version 786598 (0.0006) [2023-12-26 21:05:17,561][105620] Updated weights for policy 1, policy_version 786610 (0.0010) [2023-12-26 21:05:17,565][105692] Updated weights for policy 0, policy_version 786608 (0.0006) [2023-12-26 21:05:17,616][105620] Updated weights for policy 1, policy_version 786620 (0.0010) [2023-12-26 21:05:17,630][105692] Updated weights for policy 0, policy_version 786618 (0.0006) [2023-12-26 21:05:17,681][105620] Updated weights for policy 1, policy_version 786630 (0.0010) [2023-12-26 21:05:17,694][105692] Updated weights for policy 0, policy_version 786628 (0.0005) [2023-12-26 21:05:18,181][105692] Updated weights for policy 0, policy_version 786638 (0.0008) [2023-12-26 21:05:18,229][105692] Updated weights for policy 0, policy_version 786648 (0.0010) [2023-12-26 21:05:18,276][105692] Updated weights for policy 0, policy_version 786658 (0.0009) [2023-12-26 21:05:18,434][105620] Updated weights for policy 1, policy_version 786640 (0.0010) [2023-12-26 21:05:18,493][105620] Updated weights for policy 1, policy_version 786650 (0.0010) [2023-12-26 21:05:18,559][105620] Updated weights for policy 1, policy_version 786660 (0.0010) [2023-12-26 21:05:18,958][105692] Updated weights for policy 0, policy_version 786668 (0.0008) [2023-12-26 21:05:19,016][105692] Updated weights for policy 0, policy_version 786678 (0.0008) [2023-12-26 21:05:19,076][105692] Updated weights for policy 0, policy_version 786688 (0.0008) [2023-12-26 21:05:19,300][105620] Updated weights for policy 1, policy_version 786670 (0.0010) [2023-12-26 21:05:19,361][105620] Updated weights for policy 1, policy_version 786680 (0.0011) [2023-12-26 21:05:19,420][105620] Updated weights for policy 1, policy_version 786690 (0.0011) [2023-12-26 21:05:19,741][105692] Updated weights for policy 0, policy_version 786698 (0.0008) [2023-12-26 21:05:19,800][105692] Updated weights for policy 0, policy_version 786708 (0.0006) [2023-12-26 21:05:19,869][105692] Updated weights for policy 0, policy_version 786718 (0.0008) [2023-12-26 21:05:19,928][105692] Updated weights for policy 0, policy_version 786728 (0.0008) [2023-12-26 21:05:20,203][105620] Updated weights for policy 1, policy_version 786700 (0.0010) [2023-12-26 21:05:20,273][105620] Updated weights for policy 1, policy_version 786710 (0.0011) [2023-12-26 21:05:20,333][105620] Updated weights for policy 1, policy_version 786720 (0.0010) [2023-12-26 21:05:20,667][105692] Updated weights for policy 0, policy_version 786738 (0.0011) [2023-12-26 21:05:20,725][105692] Updated weights for policy 0, policy_version 786748 (0.0011) [2023-12-26 21:05:20,778][105692] Updated weights for policy 0, policy_version 786758 (0.0011) [2023-12-26 21:05:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 402866176. Throughput: 0: 10142.4, 1: 9674.2. Samples: 402857292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:05:21,062][104569] Avg episode reward: [(0, '9008.306'), (1, '9175.687')] [2023-12-26 21:05:21,081][105620] Updated weights for policy 1, policy_version 786730 (0.0009) [2023-12-26 21:05:21,137][105620] Updated weights for policy 1, policy_version 786740 (0.0007) [2023-12-26 21:05:21,195][105620] Updated weights for policy 1, policy_version 786750 (0.0006) [2023-12-26 21:05:21,257][105620] Updated weights for policy 1, policy_version 786760 (0.0006) [2023-12-26 21:05:21,651][105692] Updated weights for policy 0, policy_version 786768 (0.0009) [2023-12-26 21:05:21,719][105692] Updated weights for policy 0, policy_version 786778 (0.0007) [2023-12-26 21:05:21,780][105692] Updated weights for policy 0, policy_version 786788 (0.0010) [2023-12-26 21:05:21,961][105620] Updated weights for policy 1, policy_version 786770 (0.0011) [2023-12-26 21:05:22,024][105620] Updated weights for policy 1, policy_version 786780 (0.0010) [2023-12-26 21:05:22,083][105620] Updated weights for policy 1, policy_version 786790 (0.0010) [2023-12-26 21:05:22,500][105692] Updated weights for policy 0, policy_version 786798 (0.0008) [2023-12-26 21:05:22,563][105692] Updated weights for policy 0, policy_version 786808 (0.0006) [2023-12-26 21:05:22,631][105692] Updated weights for policy 0, policy_version 786818 (0.0006) [2023-12-26 21:05:22,861][105620] Updated weights for policy 1, policy_version 786800 (0.0010) [2023-12-26 21:05:22,924][105620] Updated weights for policy 1, policy_version 786810 (0.0010) [2023-12-26 21:05:22,982][105620] Updated weights for policy 1, policy_version 786820 (0.0010) [2023-12-26 21:05:23,295][105692] Updated weights for policy 0, policy_version 786828 (0.0009) [2023-12-26 21:05:23,352][105692] Updated weights for policy 0, policy_version 786838 (0.0010) [2023-12-26 21:05:23,412][105692] Updated weights for policy 0, policy_version 786848 (0.0011) [2023-12-26 21:05:23,672][105620] Updated weights for policy 1, policy_version 786830 (0.0007) [2023-12-26 21:05:23,721][105620] Updated weights for policy 1, policy_version 786840 (0.0005) [2023-12-26 21:05:23,772][105620] Updated weights for policy 1, policy_version 786850 (0.0005) [2023-12-26 21:05:23,988][105692] Updated weights for policy 0, policy_version 786858 (0.0010) [2023-12-26 21:05:24,037][105692] Updated weights for policy 0, policy_version 786868 (0.0009) [2023-12-26 21:05:24,088][105692] Updated weights for policy 0, policy_version 786879 (0.0009) [2023-12-26 21:05:24,380][105620] Updated weights for policy 1, policy_version 786860 (0.0007) [2023-12-26 21:05:24,434][105620] Updated weights for policy 1, policy_version 786870 (0.0009) [2023-12-26 21:05:24,487][105620] Updated weights for policy 1, policy_version 786880 (0.0009) [2023-12-26 21:05:24,784][105692] Updated weights for policy 0, policy_version 786889 (0.0009) [2023-12-26 21:05:24,842][105692] Updated weights for policy 0, policy_version 786899 (0.0006) [2023-12-26 21:05:24,900][105692] Updated weights for policy 0, policy_version 786909 (0.0007) [2023-12-26 21:05:24,955][105692] Updated weights for policy 0, policy_version 786919 (0.0006) [2023-12-26 21:05:25,245][105620] Updated weights for policy 1, policy_version 786890 (0.0010) [2023-12-26 21:05:25,310][105620] Updated weights for policy 1, policy_version 786900 (0.0010) [2023-12-26 21:05:25,372][105620] Updated weights for policy 1, policy_version 786910 (0.0009) [2023-12-26 21:05:25,430][105620] Updated weights for policy 1, policy_version 786920 (0.0010) [2023-12-26 21:05:25,560][105692] Updated weights for policy 0, policy_version 786929 (0.0005) [2023-12-26 21:05:25,616][105692] Updated weights for policy 0, policy_version 786939 (0.0005) [2023-12-26 21:05:25,684][105692] Updated weights for policy 0, policy_version 786949 (0.0008) [2023-12-26 21:05:26,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 402964480. Throughput: 0: 10178.8, 1: 9628.1. Samples: 402975904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:05:26,062][104569] Avg episode reward: [(0, '9265.062'), (1, '8746.798')] [2023-12-26 21:05:26,116][105620] Updated weights for policy 1, policy_version 786930 (0.0010) [2023-12-26 21:05:26,182][105620] Updated weights for policy 1, policy_version 786940 (0.0010) [2023-12-26 21:05:26,237][105620] Updated weights for policy 1, policy_version 786950 (0.0009) [2023-12-26 21:05:26,442][105692] Updated weights for policy 0, policy_version 786959 (0.0008) [2023-12-26 21:05:26,497][105692] Updated weights for policy 0, policy_version 786969 (0.0008) [2023-12-26 21:05:26,549][105692] Updated weights for policy 0, policy_version 786979 (0.0008) [2023-12-26 21:05:26,917][105620] Updated weights for policy 1, policy_version 786960 (0.0006) [2023-12-26 21:05:26,971][105620] Updated weights for policy 1, policy_version 786970 (0.0010) [2023-12-26 21:05:27,025][105620] Updated weights for policy 1, policy_version 786980 (0.0010) [2023-12-26 21:05:27,313][105692] Updated weights for policy 0, policy_version 786989 (0.0008) [2023-12-26 21:05:27,368][105692] Updated weights for policy 0, policy_version 786999 (0.0008) [2023-12-26 21:05:27,421][105692] Updated weights for policy 0, policy_version 787009 (0.0008) [2023-12-26 21:05:27,733][105620] Updated weights for policy 1, policy_version 786990 (0.0010) [2023-12-26 21:05:27,778][105620] Updated weights for policy 1, policy_version 787000 (0.0010) [2023-12-26 21:05:27,824][105620] Updated weights for policy 1, policy_version 787010 (0.0010) [2023-12-26 21:05:28,095][105692] Updated weights for policy 0, policy_version 787019 (0.0008) [2023-12-26 21:05:28,142][105692] Updated weights for policy 0, policy_version 787029 (0.0010) [2023-12-26 21:05:28,197][105692] Updated weights for policy 0, policy_version 787039 (0.0009) [2023-12-26 21:05:28,500][105620] Updated weights for policy 1, policy_version 787020 (0.0010) [2023-12-26 21:05:28,569][105620] Updated weights for policy 1, policy_version 787030 (0.0011) [2023-12-26 21:05:28,634][105620] Updated weights for policy 1, policy_version 787040 (0.0010) [2023-12-26 21:05:28,825][105692] Updated weights for policy 0, policy_version 787049 (0.0009) [2023-12-26 21:05:28,876][105692] Updated weights for policy 0, policy_version 787059 (0.0010) [2023-12-26 21:05:28,934][105692] Updated weights for policy 0, policy_version 787069 (0.0010) [2023-12-26 21:05:28,981][105692] Updated weights for policy 0, policy_version 787079 (0.0010) [2023-12-26 21:05:29,328][105620] Updated weights for policy 1, policy_version 787050 (0.0010) [2023-12-26 21:05:29,395][105620] Updated weights for policy 1, policy_version 787060 (0.0010) [2023-12-26 21:05:29,457][105620] Updated weights for policy 1, policy_version 787070 (0.0010) [2023-12-26 21:05:29,523][105620] Updated weights for policy 1, policy_version 787080 (0.0011) [2023-12-26 21:05:29,686][105692] Updated weights for policy 0, policy_version 787089 (0.0006) [2023-12-26 21:05:29,740][105692] Updated weights for policy 0, policy_version 787099 (0.0008) [2023-12-26 21:05:29,801][105692] Updated weights for policy 0, policy_version 787109 (0.0005) [2023-12-26 21:05:30,283][105620] Updated weights for policy 1, policy_version 787090 (0.0010) [2023-12-26 21:05:30,344][105620] Updated weights for policy 1, policy_version 787100 (0.0010) [2023-12-26 21:05:30,395][105620] Updated weights for policy 1, policy_version 787110 (0.0010) [2023-12-26 21:05:30,420][105692] Updated weights for policy 0, policy_version 787119 (0.0008) [2023-12-26 21:05:30,471][105692] Updated weights for policy 0, policy_version 787129 (0.0005) [2023-12-26 21:05:30,528][105692] Updated weights for policy 0, policy_version 787139 (0.0008) [2023-12-26 21:05:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 403062784. Throughput: 0: 10181.4, 1: 9668.2. Samples: 403035024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:05:31,063][104569] Avg episode reward: [(0, '9265.235'), (1, '8743.159')] [2023-12-26 21:05:31,099][105692] Updated weights for policy 0, policy_version 787149 (0.0007) [2023-12-26 21:05:31,115][105620] Updated weights for policy 1, policy_version 787120 (0.0011) [2023-12-26 21:05:31,160][105692] Updated weights for policy 0, policy_version 787159 (0.0008) [2023-12-26 21:05:31,175][105620] Updated weights for policy 1, policy_version 787130 (0.0010) [2023-12-26 21:05:31,218][105692] Updated weights for policy 0, policy_version 787169 (0.0008) [2023-12-26 21:05:31,230][105620] Updated weights for policy 1, policy_version 787140 (0.0010) [2023-12-26 21:05:31,251][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000787144_201531392.pth... [2023-12-26 21:05:31,252][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000787176_201547776.pth... [2023-12-26 21:05:31,255][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000785992_201236480.pth [2023-12-26 21:05:31,257][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000785992_201244672.pth [2023-12-26 21:05:31,979][105692] Updated weights for policy 0, policy_version 787179 (0.0007) [2023-12-26 21:05:31,993][105620] Updated weights for policy 1, policy_version 787150 (0.0011) [2023-12-26 21:05:32,028][105692] Updated weights for policy 0, policy_version 787189 (0.0006) [2023-12-26 21:05:32,045][105620] Updated weights for policy 1, policy_version 787160 (0.0010) [2023-12-26 21:05:32,078][105692] Updated weights for policy 0, policy_version 787199 (0.0009) [2023-12-26 21:05:32,100][105620] Updated weights for policy 1, policy_version 787170 (0.0010) [2023-12-26 21:05:32,846][105692] Updated weights for policy 0, policy_version 787209 (0.0006) [2023-12-26 21:05:32,872][105620] Updated weights for policy 1, policy_version 787180 (0.0010) [2023-12-26 21:05:32,908][105692] Updated weights for policy 0, policy_version 787219 (0.0007) [2023-12-26 21:05:32,920][105620] Updated weights for policy 1, policy_version 787191 (0.0006) [2023-12-26 21:05:32,959][105692] Updated weights for policy 0, policy_version 787229 (0.0007) [2023-12-26 21:05:32,968][105620] Updated weights for policy 1, policy_version 787201 (0.0008) [2023-12-26 21:05:33,012][105692] Updated weights for policy 0, policy_version 787239 (0.0008) [2023-12-26 21:05:33,546][105620] Updated weights for policy 1, policy_version 787211 (0.0008) [2023-12-26 21:05:33,592][105620] Updated weights for policy 1, policy_version 787221 (0.0005) [2023-12-26 21:05:33,644][105620] Updated weights for policy 1, policy_version 787231 (0.0005) [2023-12-26 21:05:33,737][105692] Updated weights for policy 0, policy_version 787249 (0.0008) [2023-12-26 21:05:33,788][105692] Updated weights for policy 0, policy_version 787259 (0.0007) [2023-12-26 21:05:33,842][105692] Updated weights for policy 0, policy_version 787269 (0.0006) [2023-12-26 21:05:34,224][105620] Updated weights for policy 1, policy_version 787241 (0.0005) [2023-12-26 21:05:34,282][105620] Updated weights for policy 1, policy_version 787251 (0.0006) [2023-12-26 21:05:34,344][105620] Updated weights for policy 1, policy_version 787261 (0.0007) [2023-12-26 21:05:34,414][105620] Updated weights for policy 1, policy_version 787271 (0.0008) [2023-12-26 21:05:34,659][105692] Updated weights for policy 0, policy_version 787279 (0.0008) [2023-12-26 21:05:34,720][105692] Updated weights for policy 0, policy_version 787289 (0.0009) [2023-12-26 21:05:34,780][105692] Updated weights for policy 0, policy_version 787299 (0.0008) [2023-12-26 21:05:35,018][105620] Updated weights for policy 1, policy_version 787281 (0.0008) [2023-12-26 21:05:35,069][105620] Updated weights for policy 1, policy_version 787291 (0.0008) [2023-12-26 21:05:35,122][105620] Updated weights for policy 1, policy_version 787301 (0.0009) [2023-12-26 21:05:35,596][105692] Updated weights for policy 0, policy_version 787309 (0.0009) [2023-12-26 21:05:35,643][105692] Updated weights for policy 0, policy_version 787319 (0.0008) [2023-12-26 21:05:35,690][105692] Updated weights for policy 0, policy_version 787329 (0.0008) [2023-12-26 21:05:35,748][105620] Updated weights for policy 1, policy_version 787311 (0.0008) [2023-12-26 21:05:35,798][105620] Updated weights for policy 1, policy_version 787321 (0.0009) [2023-12-26 21:05:35,855][105620] Updated weights for policy 1, policy_version 787331 (0.0008) [2023-12-26 21:05:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 403169280. Throughput: 0: 10062.6, 1: 9829.7. Samples: 403156208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:05:36,062][104569] Avg episode reward: [(0, '9176.074'), (1, '9084.312')] [2023-12-26 21:05:36,490][105692] Updated weights for policy 0, policy_version 787339 (0.0007) [2023-12-26 21:05:36,556][105692] Updated weights for policy 0, policy_version 787349 (0.0009) [2023-12-26 21:05:36,601][105620] Updated weights for policy 1, policy_version 787341 (0.0008) [2023-12-26 21:05:36,613][105692] Updated weights for policy 0, policy_version 787359 (0.0008) [2023-12-26 21:05:36,665][105620] Updated weights for policy 1, policy_version 787351 (0.0007) [2023-12-26 21:05:36,730][105620] Updated weights for policy 1, policy_version 787361 (0.0009) [2023-12-26 21:05:37,368][105692] Updated weights for policy 0, policy_version 787369 (0.0008) [2023-12-26 21:05:37,423][105692] Updated weights for policy 0, policy_version 787379 (0.0010) [2023-12-26 21:05:37,480][105692] Updated weights for policy 0, policy_version 787389 (0.0010) [2023-12-26 21:05:37,502][105620] Updated weights for policy 1, policy_version 787371 (0.0008) [2023-12-26 21:05:37,546][105692] Updated weights for policy 0, policy_version 787399 (0.0010) [2023-12-26 21:05:37,558][105620] Updated weights for policy 1, policy_version 787381 (0.0008) [2023-12-26 21:05:37,625][105620] Updated weights for policy 1, policy_version 787391 (0.0008) [2023-12-26 21:05:38,151][105692] Updated weights for policy 0, policy_version 787409 (0.0009) [2023-12-26 21:05:38,213][105692] Updated weights for policy 0, policy_version 787419 (0.0009) [2023-12-26 21:05:38,266][105620] Updated weights for policy 1, policy_version 787401 (0.0008) [2023-12-26 21:05:38,266][105692] Updated weights for policy 0, policy_version 787429 (0.0009) [2023-12-26 21:05:38,319][105620] Updated weights for policy 1, policy_version 787411 (0.0009) [2023-12-26 21:05:38,390][105620] Updated weights for policy 1, policy_version 787421 (0.0010) [2023-12-26 21:05:38,452][105620] Updated weights for policy 1, policy_version 787431 (0.0008) [2023-12-26 21:05:38,917][105692] Updated weights for policy 0, policy_version 787439 (0.0006) [2023-12-26 21:05:38,970][105692] Updated weights for policy 0, policy_version 787449 (0.0006) [2023-12-26 21:05:39,017][105692] Updated weights for policy 0, policy_version 787459 (0.0006) [2023-12-26 21:05:39,295][105620] Updated weights for policy 1, policy_version 787441 (0.0009) [2023-12-26 21:05:39,357][105620] Updated weights for policy 1, policy_version 787451 (0.0009) [2023-12-26 21:05:39,424][105620] Updated weights for policy 1, policy_version 787461 (0.0009) [2023-12-26 21:05:39,712][105692] Updated weights for policy 0, policy_version 787469 (0.0009) [2023-12-26 21:05:39,765][105692] Updated weights for policy 0, policy_version 787479 (0.0009) [2023-12-26 21:05:39,824][105692] Updated weights for policy 0, policy_version 787489 (0.0009) [2023-12-26 21:05:40,220][105620] Updated weights for policy 1, policy_version 787471 (0.0010) [2023-12-26 21:05:40,286][105620] Updated weights for policy 1, policy_version 787481 (0.0009) [2023-12-26 21:05:40,342][105620] Updated weights for policy 1, policy_version 787491 (0.0009) [2023-12-26 21:05:40,547][105692] Updated weights for policy 0, policy_version 787499 (0.0009) [2023-12-26 21:05:40,609][105692] Updated weights for policy 0, policy_version 787509 (0.0009) [2023-12-26 21:05:40,657][105692] Updated weights for policy 0, policy_version 787519 (0.0009) [2023-12-26 21:05:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 403259392. Throughput: 0: 9989.9, 1: 9758.4. Samples: 403270604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:05:41,063][104569] Avg episode reward: [(0, '8731.273'), (1, '9175.681')] [2023-12-26 21:05:41,117][105620] Updated weights for policy 1, policy_version 787501 (0.0010) [2023-12-26 21:05:41,194][105620] Updated weights for policy 1, policy_version 787511 (0.0009) [2023-12-26 21:05:41,246][105586] KL-divergence is very high: 107.7767 [2023-12-26 21:05:41,261][105620] Updated weights for policy 1, policy_version 787521 (0.0009) [2023-12-26 21:05:41,299][105586] KL-divergence is very high: 123.9675 [2023-12-26 21:05:41,460][105692] Updated weights for policy 0, policy_version 787529 (0.0009) [2023-12-26 21:05:41,525][105692] Updated weights for policy 0, policy_version 787539 (0.0009) [2023-12-26 21:05:41,594][105692] Updated weights for policy 0, policy_version 787549 (0.0009) [2023-12-26 21:05:41,667][105692] Updated weights for policy 0, policy_version 787559 (0.0009) [2023-12-26 21:05:42,064][105620] Updated weights for policy 1, policy_version 787531 (0.0009) [2023-12-26 21:05:42,131][105620] Updated weights for policy 1, policy_version 787541 (0.0010) [2023-12-26 21:05:42,197][105620] Updated weights for policy 1, policy_version 787551 (0.0010) [2023-12-26 21:05:42,331][105692] Updated weights for policy 0, policy_version 787569 (0.0009) [2023-12-26 21:05:42,394][105692] Updated weights for policy 0, policy_version 787579 (0.0009) [2023-12-26 21:05:42,451][105692] Updated weights for policy 0, policy_version 787589 (0.0008) [2023-12-26 21:05:42,990][105620] Updated weights for policy 1, policy_version 787561 (0.0009) [2023-12-26 21:05:43,045][105620] Updated weights for policy 1, policy_version 787571 (0.0008) [2023-12-26 21:05:43,094][105620] Updated weights for policy 1, policy_version 787581 (0.0008) [2023-12-26 21:05:43,152][105620] Updated weights for policy 1, policy_version 787591 (0.0008) [2023-12-26 21:05:43,209][105692] Updated weights for policy 0, policy_version 787599 (0.0009) [2023-12-26 21:05:43,256][105692] Updated weights for policy 0, policy_version 787609 (0.0009) [2023-12-26 21:05:43,309][105692] Updated weights for policy 0, policy_version 787620 (0.0010) [2023-12-26 21:05:43,855][105620] Updated weights for policy 1, policy_version 787601 (0.0010) [2023-12-26 21:05:43,903][105620] Updated weights for policy 1, policy_version 787611 (0.0010) [2023-12-26 21:05:43,953][105620] Updated weights for policy 1, policy_version 787621 (0.0010) [2023-12-26 21:05:43,986][105692] Updated weights for policy 0, policy_version 787630 (0.0009) [2023-12-26 21:05:44,052][105692] Updated weights for policy 0, policy_version 787640 (0.0008) [2023-12-26 21:05:44,112][105692] Updated weights for policy 0, policy_version 787650 (0.0006) [2023-12-26 21:05:44,712][105692] Updated weights for policy 0, policy_version 787660 (0.0006) [2023-12-26 21:05:44,723][105620] Updated weights for policy 1, policy_version 787631 (0.0010) [2023-12-26 21:05:44,770][105692] Updated weights for policy 0, policy_version 787670 (0.0006) [2023-12-26 21:05:44,792][105620] Updated weights for policy 1, policy_version 787641 (0.0007) [2023-12-26 21:05:44,830][105692] Updated weights for policy 0, policy_version 787680 (0.0009) [2023-12-26 21:05:44,854][105620] Updated weights for policy 1, policy_version 787651 (0.0009) [2023-12-26 21:05:45,578][105620] Updated weights for policy 1, policy_version 787661 (0.0011) [2023-12-26 21:05:45,585][105692] Updated weights for policy 0, policy_version 787690 (0.0010) [2023-12-26 21:05:45,627][105620] Updated weights for policy 1, policy_version 787671 (0.0010) [2023-12-26 21:05:45,634][105692] Updated weights for policy 0, policy_version 787700 (0.0005) [2023-12-26 21:05:45,676][105620] Updated weights for policy 1, policy_version 787681 (0.0010) [2023-12-26 21:05:45,686][105692] Updated weights for policy 0, policy_version 787710 (0.0006) [2023-12-26 21:05:45,740][105692] Updated weights for policy 0, policy_version 787720 (0.0007) [2023-12-26 21:05:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 403357696. Throughput: 0: 9883.4, 1: 9725.0. Samples: 403325640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:05:46,063][104569] Avg episode reward: [(0, '8467.492'), (1, '9264.637')] [2023-12-26 21:05:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000787720_201687040.pth... [2023-12-26 21:05:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000787688_201670656.pth... [2023-12-26 21:05:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000786568_201383936.pth [2023-12-26 21:05:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000786568_201392128.pth [2023-12-26 21:05:46,407][105620] Updated weights for policy 1, policy_version 787691 (0.0011) [2023-12-26 21:05:46,421][105692] Updated weights for policy 0, policy_version 787730 (0.0007) [2023-12-26 21:05:46,471][105620] Updated weights for policy 1, policy_version 787701 (0.0011) [2023-12-26 21:05:46,475][105692] Updated weights for policy 0, policy_version 787740 (0.0008) [2023-12-26 21:05:46,533][105620] Updated weights for policy 1, policy_version 787711 (0.0010) [2023-12-26 21:05:46,540][105692] Updated weights for policy 0, policy_version 787750 (0.0009) [2023-12-26 21:05:47,193][105620] Updated weights for policy 1, policy_version 787721 (0.0010) [2023-12-26 21:05:47,242][105692] Updated weights for policy 0, policy_version 787760 (0.0007) [2023-12-26 21:05:47,248][105620] Updated weights for policy 1, policy_version 787731 (0.0010) [2023-12-26 21:05:47,293][105620] Updated weights for policy 1, policy_version 787741 (0.0010) [2023-12-26 21:05:47,294][105692] Updated weights for policy 0, policy_version 787770 (0.0006) [2023-12-26 21:05:47,339][105620] Updated weights for policy 1, policy_version 787751 (0.0010) [2023-12-26 21:05:47,348][105692] Updated weights for policy 0, policy_version 787780 (0.0008) [2023-12-26 21:05:48,089][105620] Updated weights for policy 1, policy_version 787761 (0.0011) [2023-12-26 21:05:48,100][105692] Updated weights for policy 0, policy_version 787790 (0.0007) [2023-12-26 21:05:48,142][105620] Updated weights for policy 1, policy_version 787771 (0.0010) [2023-12-26 21:05:48,154][105692] Updated weights for policy 0, policy_version 787800 (0.0006) [2023-12-26 21:05:48,201][105620] Updated weights for policy 1, policy_version 787781 (0.0010) [2023-12-26 21:05:48,205][105692] Updated weights for policy 0, policy_version 787810 (0.0010) [2023-12-26 21:05:48,870][105692] Updated weights for policy 0, policy_version 787820 (0.0010) [2023-12-26 21:05:48,934][105692] Updated weights for policy 0, policy_version 787830 (0.0011) [2023-12-26 21:05:48,962][105620] Updated weights for policy 1, policy_version 787791 (0.0011) [2023-12-26 21:05:48,994][105692] Updated weights for policy 0, policy_version 787840 (0.0011) [2023-12-26 21:05:49,015][105620] Updated weights for policy 1, policy_version 787801 (0.0011) [2023-12-26 21:05:49,076][105620] Updated weights for policy 1, policy_version 787811 (0.0011) [2023-12-26 21:05:49,621][105692] Updated weights for policy 0, policy_version 787850 (0.0007) [2023-12-26 21:05:49,691][105692] Updated weights for policy 0, policy_version 787860 (0.0006) [2023-12-26 21:05:49,720][105620] Updated weights for policy 1, policy_version 787821 (0.0009) [2023-12-26 21:05:49,752][105692] Updated weights for policy 0, policy_version 787870 (0.0007) [2023-12-26 21:05:49,779][105620] Updated weights for policy 1, policy_version 787831 (0.0006) [2023-12-26 21:05:49,816][105692] Updated weights for policy 0, policy_version 787880 (0.0008) [2023-12-26 21:05:49,849][105620] Updated weights for policy 1, policy_version 787841 (0.0007) [2023-12-26 21:05:50,464][105692] Updated weights for policy 0, policy_version 787890 (0.0007) [2023-12-26 21:05:50,527][105692] Updated weights for policy 0, policy_version 787900 (0.0008) [2023-12-26 21:05:50,531][105620] Updated weights for policy 1, policy_version 787851 (0.0009) [2023-12-26 21:05:50,590][105692] Updated weights for policy 0, policy_version 787910 (0.0007) [2023-12-26 21:05:50,597][105620] Updated weights for policy 1, policy_version 787861 (0.0008) [2023-12-26 21:05:50,664][105620] Updated weights for policy 1, policy_version 787871 (0.0006) [2023-12-26 21:05:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 403456000. Throughput: 0: 9882.9, 1: 9723.3. Samples: 403444696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:05:51,062][104569] Avg episode reward: [(0, '8554.415'), (1, '9012.277')] [2023-12-26 21:05:51,346][105692] Updated weights for policy 0, policy_version 787920 (0.0007) [2023-12-26 21:05:51,389][105620] Updated weights for policy 1, policy_version 787881 (0.0008) [2023-12-26 21:05:51,406][105692] Updated weights for policy 0, policy_version 787930 (0.0008) [2023-12-26 21:05:51,451][105620] Updated weights for policy 1, policy_version 787891 (0.0009) [2023-12-26 21:05:51,465][105692] Updated weights for policy 0, policy_version 787940 (0.0006) [2023-12-26 21:05:51,513][105620] Updated weights for policy 1, policy_version 787901 (0.0010) [2023-12-26 21:05:51,582][105620] Updated weights for policy 1, policy_version 787911 (0.0011) [2023-12-26 21:05:52,072][105692] Updated weights for policy 0, policy_version 787950 (0.0005) [2023-12-26 21:05:52,137][105692] Updated weights for policy 0, policy_version 787960 (0.0008) [2023-12-26 21:05:52,193][105692] Updated weights for policy 0, policy_version 787970 (0.0008) [2023-12-26 21:05:52,289][105620] Updated weights for policy 1, policy_version 787921 (0.0011) [2023-12-26 21:05:52,354][105620] Updated weights for policy 1, policy_version 787931 (0.0013) [2023-12-26 21:05:52,414][105620] Updated weights for policy 1, policy_version 787941 (0.0006) [2023-12-26 21:05:52,880][105692] Updated weights for policy 0, policy_version 787980 (0.0009) [2023-12-26 21:05:52,930][105692] Updated weights for policy 0, policy_version 787990 (0.0009) [2023-12-26 21:05:52,989][105692] Updated weights for policy 0, policy_version 788000 (0.0005) [2023-12-26 21:05:53,129][105620] Updated weights for policy 1, policy_version 787951 (0.0007) [2023-12-26 21:05:53,185][105620] Updated weights for policy 1, policy_version 787961 (0.0008) [2023-12-26 21:05:53,233][105620] Updated weights for policy 1, policy_version 787971 (0.0008) [2023-12-26 21:05:53,730][105692] Updated weights for policy 0, policy_version 788010 (0.0008) [2023-12-26 21:05:53,788][105692] Updated weights for policy 0, policy_version 788020 (0.0006) [2023-12-26 21:05:53,847][105692] Updated weights for policy 0, policy_version 788030 (0.0005) [2023-12-26 21:05:53,898][105692] Updated weights for policy 0, policy_version 788040 (0.0005) [2023-12-26 21:05:53,923][105620] Updated weights for policy 1, policy_version 787981 (0.0009) [2023-12-26 21:05:53,989][105620] Updated weights for policy 1, policy_version 787991 (0.0010) [2023-12-26 21:05:54,048][105620] Updated weights for policy 1, policy_version 788001 (0.0008) [2023-12-26 21:05:54,526][105692] Updated weights for policy 0, policy_version 788050 (0.0009) [2023-12-26 21:05:54,591][105692] Updated weights for policy 0, policy_version 788060 (0.0008) [2023-12-26 21:05:54,657][105692] Updated weights for policy 0, policy_version 788070 (0.0005) [2023-12-26 21:05:54,823][105620] Updated weights for policy 1, policy_version 788011 (0.0008) [2023-12-26 21:05:54,877][105620] Updated weights for policy 1, policy_version 788021 (0.0008) [2023-12-26 21:05:54,930][105620] Updated weights for policy 1, policy_version 788031 (0.0007) [2023-12-26 21:05:55,283][105692] Updated weights for policy 0, policy_version 788080 (0.0008) [2023-12-26 21:05:55,342][105692] Updated weights for policy 0, policy_version 788090 (0.0007) [2023-12-26 21:05:55,397][105692] Updated weights for policy 0, policy_version 788100 (0.0008) [2023-12-26 21:05:55,663][105620] Updated weights for policy 1, policy_version 788041 (0.0011) [2023-12-26 21:05:55,721][105620] Updated weights for policy 1, policy_version 788051 (0.0010) [2023-12-26 21:05:55,779][105620] Updated weights for policy 1, policy_version 788061 (0.0010) [2023-12-26 21:05:55,837][105620] Updated weights for policy 1, policy_version 788071 (0.0010) [2023-12-26 21:05:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 403554304. Throughput: 0: 9899.1, 1: 9693.7. Samples: 403563020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:05:56,062][104569] Avg episode reward: [(0, '9087.782'), (1, '9097.716')] [2023-12-26 21:05:56,186][105692] Updated weights for policy 0, policy_version 788110 (0.0009) [2023-12-26 21:05:56,240][105692] Updated weights for policy 0, policy_version 788120 (0.0008) [2023-12-26 21:05:56,303][105692] Updated weights for policy 0, policy_version 788130 (0.0008) [2023-12-26 21:05:56,537][105620] Updated weights for policy 1, policy_version 788081 (0.0010) [2023-12-26 21:05:56,594][105620] Updated weights for policy 1, policy_version 788091 (0.0010) [2023-12-26 21:05:56,653][105620] Updated weights for policy 1, policy_version 788101 (0.0010) [2023-12-26 21:05:57,062][105692] Updated weights for policy 0, policy_version 788140 (0.0009) [2023-12-26 21:05:57,115][105692] Updated weights for policy 0, policy_version 788150 (0.0010) [2023-12-26 21:05:57,166][105692] Updated weights for policy 0, policy_version 788160 (0.0010) [2023-12-26 21:05:57,291][105620] Updated weights for policy 1, policy_version 788111 (0.0007) [2023-12-26 21:05:57,343][105620] Updated weights for policy 1, policy_version 788121 (0.0008) [2023-12-26 21:05:57,401][105620] Updated weights for policy 1, policy_version 788131 (0.0006) [2023-12-26 21:05:57,822][105692] Updated weights for policy 0, policy_version 788170 (0.0009) [2023-12-26 21:05:57,875][105692] Updated weights for policy 0, policy_version 788180 (0.0005) [2023-12-26 21:05:57,930][105692] Updated weights for policy 0, policy_version 788190 (0.0008) [2023-12-26 21:05:57,985][105692] Updated weights for policy 0, policy_version 788200 (0.0010) [2023-12-26 21:05:58,038][105620] Updated weights for policy 1, policy_version 788141 (0.0007) [2023-12-26 21:05:58,094][105620] Updated weights for policy 1, policy_version 788151 (0.0009) [2023-12-26 21:05:58,164][105620] Updated weights for policy 1, policy_version 788161 (0.0008) [2023-12-26 21:05:58,691][105692] Updated weights for policy 0, policy_version 788210 (0.0007) [2023-12-26 21:05:58,758][105692] Updated weights for policy 0, policy_version 788220 (0.0007) [2023-12-26 21:05:58,825][105692] Updated weights for policy 0, policy_version 788230 (0.0007) [2023-12-26 21:05:58,968][105620] Updated weights for policy 1, policy_version 788171 (0.0006) [2023-12-26 21:05:59,031][105620] Updated weights for policy 1, policy_version 788181 (0.0008) [2023-12-26 21:05:59,082][105620] Updated weights for policy 1, policy_version 788191 (0.0008) [2023-12-26 21:05:59,623][105692] Updated weights for policy 0, policy_version 788240 (0.0008) [2023-12-26 21:05:59,684][105692] Updated weights for policy 0, policy_version 788250 (0.0009) [2023-12-26 21:05:59,740][105692] Updated weights for policy 0, policy_version 788260 (0.0009) [2023-12-26 21:05:59,820][105620] Updated weights for policy 1, policy_version 788201 (0.0008) [2023-12-26 21:05:59,881][105620] Updated weights for policy 1, policy_version 788211 (0.0009) [2023-12-26 21:05:59,945][105620] Updated weights for policy 1, policy_version 788221 (0.0008) [2023-12-26 21:06:00,002][105620] Updated weights for policy 1, policy_version 788231 (0.0008) [2023-12-26 21:06:00,529][105692] Updated weights for policy 0, policy_version 788270 (0.0008) [2023-12-26 21:06:00,577][105692] Updated weights for policy 0, policy_version 788280 (0.0009) [2023-12-26 21:06:00,621][105692] Updated weights for policy 0, policy_version 788290 (0.0008) [2023-12-26 21:06:00,738][105620] Updated weights for policy 1, policy_version 788241 (0.0010) [2023-12-26 21:06:00,791][105620] Updated weights for policy 1, policy_version 788251 (0.0010) [2023-12-26 21:06:00,849][105620] Updated weights for policy 1, policy_version 788261 (0.0009) [2023-12-26 21:06:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 403652608. Throughput: 0: 9948.5, 1: 9729.9. Samples: 403622412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:06:01,063][104569] Avg episode reward: [(0, '9350.385'), (1, '9178.698')] [2023-12-26 21:06:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000788296_201834496.pth... [2023-12-26 21:06:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000788264_201818112.pth... [2023-12-26 21:06:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000787176_201547776.pth [2023-12-26 21:06:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000787144_201531392.pth [2023-12-26 21:06:01,366][105692] Updated weights for policy 0, policy_version 788300 (0.0008) [2023-12-26 21:06:01,432][105692] Updated weights for policy 0, policy_version 788310 (0.0008) [2023-12-26 21:06:01,473][105620] Updated weights for policy 1, policy_version 788271 (0.0006) [2023-12-26 21:06:01,498][105692] Updated weights for policy 0, policy_version 788320 (0.0006) [2023-12-26 21:06:01,534][105620] Updated weights for policy 1, policy_version 788281 (0.0009) [2023-12-26 21:06:01,592][105620] Updated weights for policy 1, policy_version 788291 (0.0010) [2023-12-26 21:06:02,089][105692] Updated weights for policy 0, policy_version 788330 (0.0006) [2023-12-26 21:06:02,135][105692] Updated weights for policy 0, policy_version 788340 (0.0005) [2023-12-26 21:06:02,190][105692] Updated weights for policy 0, policy_version 788350 (0.0006) [2023-12-26 21:06:02,253][105692] Updated weights for policy 0, policy_version 788360 (0.0008) [2023-12-26 21:06:02,303][105620] Updated weights for policy 1, policy_version 788301 (0.0010) [2023-12-26 21:06:02,370][105620] Updated weights for policy 1, policy_version 788311 (0.0011) [2023-12-26 21:06:02,432][105620] Updated weights for policy 1, policy_version 788321 (0.0010) [2023-12-26 21:06:02,959][105692] Updated weights for policy 0, policy_version 788370 (0.0009) [2023-12-26 21:06:03,015][105692] Updated weights for policy 0, policy_version 788380 (0.0008) [2023-12-26 21:06:03,074][105692] Updated weights for policy 0, policy_version 788390 (0.0008) [2023-12-26 21:06:03,161][105620] Updated weights for policy 1, policy_version 788331 (0.0010) [2023-12-26 21:06:03,205][105620] Updated weights for policy 1, policy_version 788341 (0.0010) [2023-12-26 21:06:03,256][105620] Updated weights for policy 1, policy_version 788351 (0.0010) [2023-12-26 21:06:03,655][105692] Updated weights for policy 0, policy_version 788400 (0.0006) [2023-12-26 21:06:03,708][105692] Updated weights for policy 0, policy_version 788410 (0.0005) [2023-12-26 21:06:03,760][105692] Updated weights for policy 0, policy_version 788420 (0.0005) [2023-12-26 21:06:04,054][105620] Updated weights for policy 1, policy_version 788361 (0.0010) [2023-12-26 21:06:04,113][105620] Updated weights for policy 1, policy_version 788371 (0.0011) [2023-12-26 21:06:04,173][105620] Updated weights for policy 1, policy_version 788381 (0.0011) [2023-12-26 21:06:04,239][105620] Updated weights for policy 1, policy_version 788391 (0.0011) [2023-12-26 21:06:04,416][105692] Updated weights for policy 0, policy_version 788430 (0.0005) [2023-12-26 21:06:04,475][105692] Updated weights for policy 0, policy_version 788440 (0.0006) [2023-12-26 21:06:04,535][105692] Updated weights for policy 0, policy_version 788450 (0.0005) [2023-12-26 21:06:04,918][105620] Updated weights for policy 1, policy_version 788401 (0.0006) [2023-12-26 21:06:04,967][105620] Updated weights for policy 1, policy_version 788411 (0.0005) [2023-12-26 21:06:05,030][105620] Updated weights for policy 1, policy_version 788421 (0.0005) [2023-12-26 21:06:05,084][105692] Updated weights for policy 0, policy_version 788460 (0.0007) [2023-12-26 21:06:05,142][105692] Updated weights for policy 0, policy_version 788470 (0.0009) [2023-12-26 21:06:05,188][105692] Updated weights for policy 0, policy_version 788480 (0.0008) [2023-12-26 21:06:05,603][105620] Updated weights for policy 1, policy_version 788431 (0.0007) [2023-12-26 21:06:05,669][105620] Updated weights for policy 1, policy_version 788441 (0.0008) [2023-12-26 21:06:05,733][105620] Updated weights for policy 1, policy_version 788451 (0.0008) [2023-12-26 21:06:05,945][105692] Updated weights for policy 0, policy_version 788490 (0.0008) [2023-12-26 21:06:05,996][105692] Updated weights for policy 0, policy_version 788500 (0.0009) [2023-12-26 21:06:06,058][105692] Updated weights for policy 0, policy_version 788510 (0.0010) [2023-12-26 21:06:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 403750912. Throughput: 0: 9920.1, 1: 9720.7. Samples: 403741128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:06:06,063][104569] Avg episode reward: [(0, '8826.217'), (1, '9124.450')] [2023-12-26 21:06:06,126][105692] Updated weights for policy 0, policy_version 788520 (0.0009) [2023-12-26 21:06:06,332][105620] Updated weights for policy 1, policy_version 788461 (0.0007) [2023-12-26 21:06:06,393][105620] Updated weights for policy 1, policy_version 788471 (0.0008) [2023-12-26 21:06:06,453][105620] Updated weights for policy 1, policy_version 788481 (0.0006) [2023-12-26 21:06:06,887][105692] Updated weights for policy 0, policy_version 788530 (0.0009) [2023-12-26 21:06:06,952][105692] Updated weights for policy 0, policy_version 788540 (0.0010) [2023-12-26 21:06:07,016][105692] Updated weights for policy 0, policy_version 788550 (0.0007) [2023-12-26 21:06:07,128][105620] Updated weights for policy 1, policy_version 788491 (0.0006) [2023-12-26 21:06:07,202][105620] Updated weights for policy 1, policy_version 788501 (0.0008) [2023-12-26 21:06:07,266][105620] Updated weights for policy 1, policy_version 788511 (0.0008) [2023-12-26 21:06:07,652][105692] Updated weights for policy 0, policy_version 788560 (0.0009) [2023-12-26 21:06:07,700][105692] Updated weights for policy 0, policy_version 788570 (0.0010) [2023-12-26 21:06:07,749][105692] Updated weights for policy 0, policy_version 788580 (0.0010) [2023-12-26 21:06:08,035][105620] Updated weights for policy 1, policy_version 788521 (0.0009) [2023-12-26 21:06:08,095][105620] Updated weights for policy 1, policy_version 788531 (0.0008) [2023-12-26 21:06:08,158][105620] Updated weights for policy 1, policy_version 788541 (0.0008) [2023-12-26 21:06:08,217][105620] Updated weights for policy 1, policy_version 788551 (0.0008) [2023-12-26 21:06:08,495][105692] Updated weights for policy 0, policy_version 788590 (0.0009) [2023-12-26 21:06:08,541][105692] Updated weights for policy 0, policy_version 788600 (0.0008) [2023-12-26 21:06:08,598][105692] Updated weights for policy 0, policy_version 788610 (0.0010) [2023-12-26 21:06:08,988][105620] Updated weights for policy 1, policy_version 788561 (0.0008) [2023-12-26 21:06:09,040][105620] Updated weights for policy 1, policy_version 788571 (0.0008) [2023-12-26 21:06:09,091][105620] Updated weights for policy 1, policy_version 788581 (0.0006) [2023-12-26 21:06:09,408][105692] Updated weights for policy 0, policy_version 788620 (0.0009) [2023-12-26 21:06:09,476][105692] Updated weights for policy 0, policy_version 788630 (0.0006) [2023-12-26 21:06:09,540][105692] Updated weights for policy 0, policy_version 788640 (0.0008) [2023-12-26 21:06:09,843][105620] Updated weights for policy 1, policy_version 788591 (0.0008) [2023-12-26 21:06:09,908][105620] Updated weights for policy 1, policy_version 788601 (0.0009) [2023-12-26 21:06:09,975][105620] Updated weights for policy 1, policy_version 788611 (0.0009) [2023-12-26 21:06:10,266][105692] Updated weights for policy 0, policy_version 788650 (0.0008) [2023-12-26 21:06:10,314][105692] Updated weights for policy 0, policy_version 788660 (0.0011) [2023-12-26 21:06:10,367][105692] Updated weights for policy 0, policy_version 788670 (0.0010) [2023-12-26 21:06:10,419][105692] Updated weights for policy 0, policy_version 788680 (0.0010) [2023-12-26 21:06:10,721][105620] Updated weights for policy 1, policy_version 788621 (0.0009) [2023-12-26 21:06:10,768][105620] Updated weights for policy 1, policy_version 788631 (0.0009) [2023-12-26 21:06:10,817][105620] Updated weights for policy 1, policy_version 788642 (0.0008) [2023-12-26 21:06:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19660.8). Total num frames: 403849216. Throughput: 0: 9863.7, 1: 9730.0. Samples: 403857624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:06:11,063][104569] Avg episode reward: [(0, '8826.040'), (1, '8573.374')] [2023-12-26 21:06:11,154][105692] Updated weights for policy 0, policy_version 788690 (0.0010) [2023-12-26 21:06:11,206][105692] Updated weights for policy 0, policy_version 788700 (0.0009) [2023-12-26 21:06:11,267][105692] Updated weights for policy 0, policy_version 788710 (0.0011) [2023-12-26 21:06:11,596][105620] Updated weights for policy 1, policy_version 788652 (0.0008) [2023-12-26 21:06:11,670][105620] Updated weights for policy 1, policy_version 788662 (0.0008) [2023-12-26 21:06:11,739][105620] Updated weights for policy 1, policy_version 788672 (0.0009) [2023-12-26 21:06:12,038][105692] Updated weights for policy 0, policy_version 788720 (0.0011) [2023-12-26 21:06:12,091][105692] Updated weights for policy 0, policy_version 788730 (0.0011) [2023-12-26 21:06:12,151][105692] Updated weights for policy 0, policy_version 788740 (0.0011) [2023-12-26 21:06:12,462][105620] Updated weights for policy 1, policy_version 788682 (0.0010) [2023-12-26 21:06:12,525][105620] Updated weights for policy 1, policy_version 788692 (0.0010) [2023-12-26 21:06:12,573][105620] Updated weights for policy 1, policy_version 788702 (0.0010) [2023-12-26 21:06:12,622][105620] Updated weights for policy 1, policy_version 788712 (0.0010) [2023-12-26 21:06:12,867][105692] Updated weights for policy 0, policy_version 788750 (0.0007) [2023-12-26 21:06:12,921][105692] Updated weights for policy 0, policy_version 788760 (0.0005) [2023-12-26 21:06:12,979][105692] Updated weights for policy 0, policy_version 788770 (0.0008) [2023-12-26 21:06:13,295][105620] Updated weights for policy 1, policy_version 788722 (0.0005) [2023-12-26 21:06:13,359][105620] Updated weights for policy 1, policy_version 788732 (0.0005) [2023-12-26 21:06:13,419][105620] Updated weights for policy 1, policy_version 788742 (0.0006) [2023-12-26 21:06:13,540][105692] Updated weights for policy 0, policy_version 788780 (0.0008) [2023-12-26 21:06:13,610][105692] Updated weights for policy 0, policy_version 788790 (0.0010) [2023-12-26 21:06:13,678][105692] Updated weights for policy 0, policy_version 788800 (0.0005) [2023-12-26 21:06:13,964][105620] Updated weights for policy 1, policy_version 788752 (0.0006) [2023-12-26 21:06:14,018][105620] Updated weights for policy 1, policy_version 788762 (0.0005) [2023-12-26 21:06:14,073][105620] Updated weights for policy 1, policy_version 788772 (0.0005) [2023-12-26 21:06:14,289][105692] Updated weights for policy 0, policy_version 788810 (0.0005) [2023-12-26 21:06:14,359][105692] Updated weights for policy 0, policy_version 788820 (0.0005) [2023-12-26 21:06:14,424][105692] Updated weights for policy 0, policy_version 788830 (0.0009) [2023-12-26 21:06:14,479][105692] Updated weights for policy 0, policy_version 788840 (0.0009) [2023-12-26 21:06:14,696][105620] Updated weights for policy 1, policy_version 788782 (0.0008) [2023-12-26 21:06:14,774][105620] Updated weights for policy 1, policy_version 788792 (0.0007) [2023-12-26 21:06:14,833][105620] Updated weights for policy 1, policy_version 788802 (0.0006) [2023-12-26 21:06:15,132][105692] Updated weights for policy 0, policy_version 788850 (0.0008) [2023-12-26 21:06:15,200][105692] Updated weights for policy 0, policy_version 788860 (0.0008) [2023-12-26 21:06:15,267][105692] Updated weights for policy 0, policy_version 788870 (0.0008) [2023-12-26 21:06:15,471][105620] Updated weights for policy 1, policy_version 788812 (0.0008) [2023-12-26 21:06:15,534][105620] Updated weights for policy 1, policy_version 788822 (0.0009) [2023-12-26 21:06:15,606][105620] Updated weights for policy 1, policy_version 788832 (0.0010) [2023-12-26 21:06:15,940][105692] Updated weights for policy 0, policy_version 788880 (0.0008) [2023-12-26 21:06:15,997][105692] Updated weights for policy 0, policy_version 788890 (0.0010) [2023-12-26 21:06:16,049][105692] Updated weights for policy 0, policy_version 788900 (0.0010) [2023-12-26 21:06:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 403947520. Throughput: 0: 9905.5, 1: 9735.7. Samples: 403918880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:06:16,063][104569] Avg episode reward: [(0, '9005.160'), (1, '8452.225')] [2023-12-26 21:06:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000788840_201965568.pth... [2023-12-26 21:06:16,074][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000788904_201990144.pth... [2023-12-26 21:06:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000787688_201670656.pth [2023-12-26 21:06:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000787720_201687040.pth [2023-12-26 21:06:16,364][105620] Updated weights for policy 1, policy_version 788842 (0.0009) [2023-12-26 21:06:16,420][105620] Updated weights for policy 1, policy_version 788852 (0.0005) [2023-12-26 21:06:16,469][105620] Updated weights for policy 1, policy_version 788862 (0.0005) [2023-12-26 21:06:16,517][105620] Updated weights for policy 1, policy_version 788872 (0.0005) [2023-12-26 21:06:16,752][105692] Updated weights for policy 0, policy_version 788910 (0.0010) [2023-12-26 21:06:16,801][105692] Updated weights for policy 0, policy_version 788920 (0.0010) [2023-12-26 21:06:16,854][105692] Updated weights for policy 0, policy_version 788930 (0.0010) [2023-12-26 21:06:17,241][105620] Updated weights for policy 1, policy_version 788882 (0.0009) [2023-12-26 21:06:17,301][105620] Updated weights for policy 1, policy_version 788892 (0.0007) [2023-12-26 21:06:17,367][105620] Updated weights for policy 1, policy_version 788902 (0.0009) [2023-12-26 21:06:17,539][105692] Updated weights for policy 0, policy_version 788940 (0.0007) [2023-12-26 21:06:17,598][105692] Updated weights for policy 0, policy_version 788950 (0.0008) [2023-12-26 21:06:17,652][105692] Updated weights for policy 0, policy_version 788960 (0.0009) [2023-12-26 21:06:17,673][105585] KL-divergence is very high: 118.1426 [2023-12-26 21:06:18,130][105620] Updated weights for policy 1, policy_version 788912 (0.0009) [2023-12-26 21:06:18,188][105620] Updated weights for policy 1, policy_version 788922 (0.0010) [2023-12-26 21:06:18,242][105620] Updated weights for policy 1, policy_version 788932 (0.0010) [2023-12-26 21:06:18,320][105692] Updated weights for policy 0, policy_version 788970 (0.0008) [2023-12-26 21:06:18,385][105692] Updated weights for policy 0, policy_version 788980 (0.0007) [2023-12-26 21:06:18,448][105692] Updated weights for policy 0, policy_version 788990 (0.0006) [2023-12-26 21:06:18,516][105692] Updated weights for policy 0, policy_version 789000 (0.0006) [2023-12-26 21:06:18,972][105620] Updated weights for policy 1, policy_version 788942 (0.0010) [2023-12-26 21:06:19,027][105620] Updated weights for policy 1, policy_version 788952 (0.0010) [2023-12-26 21:06:19,081][105620] Updated weights for policy 1, policy_version 788962 (0.0010) [2023-12-26 21:06:19,171][105692] Updated weights for policy 0, policy_version 789010 (0.0005) [2023-12-26 21:06:19,237][105692] Updated weights for policy 0, policy_version 789020 (0.0007) [2023-12-26 21:06:19,293][105692] Updated weights for policy 0, policy_version 789030 (0.0009) [2023-12-26 21:06:19,892][105620] Updated weights for policy 1, policy_version 788972 (0.0009) [2023-12-26 21:06:19,957][105620] Updated weights for policy 1, policy_version 788982 (0.0009) [2023-12-26 21:06:20,020][105620] Updated weights for policy 1, policy_version 788992 (0.0008) [2023-12-26 21:06:20,034][105692] Updated weights for policy 0, policy_version 789040 (0.0009) [2023-12-26 21:06:20,092][105692] Updated weights for policy 0, policy_version 789050 (0.0008) [2023-12-26 21:06:20,140][105692] Updated weights for policy 0, policy_version 789060 (0.0008) [2023-12-26 21:06:20,733][105620] Updated weights for policy 1, policy_version 789002 (0.0007) [2023-12-26 21:06:20,789][105620] Updated weights for policy 1, policy_version 789012 (0.0009) [2023-12-26 21:06:20,841][105620] Updated weights for policy 1, policy_version 789022 (0.0009) [2023-12-26 21:06:20,900][105620] Updated weights for policy 1, policy_version 789032 (0.0008) [2023-12-26 21:06:20,955][105692] Updated weights for policy 0, policy_version 789070 (0.0009) [2023-12-26 21:06:21,017][105692] Updated weights for policy 0, policy_version 789080 (0.0009) [2023-12-26 21:06:21,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 404045824. Throughput: 0: 9926.2, 1: 9650.8. Samples: 404037176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:06:21,062][104569] Avg episode reward: [(0, '9260.508'), (1, '8808.111')] [2023-12-26 21:06:21,079][105692] Updated weights for policy 0, policy_version 789090 (0.0009) [2023-12-26 21:06:21,719][105620] Updated weights for policy 1, policy_version 789042 (0.0007) [2023-12-26 21:06:21,761][105692] Updated weights for policy 0, policy_version 789100 (0.0007) [2023-12-26 21:06:21,786][105620] Updated weights for policy 1, policy_version 789052 (0.0009) [2023-12-26 21:06:21,822][105692] Updated weights for policy 0, policy_version 789110 (0.0008) [2023-12-26 21:06:21,848][105620] Updated weights for policy 1, policy_version 789062 (0.0006) [2023-12-26 21:06:21,884][105692] Updated weights for policy 0, policy_version 789120 (0.0007) [2023-12-26 21:06:22,550][105692] Updated weights for policy 0, policy_version 789130 (0.0009) [2023-12-26 21:06:22,606][105692] Updated weights for policy 0, policy_version 789140 (0.0009) [2023-12-26 21:06:22,654][105620] Updated weights for policy 1, policy_version 789072 (0.0008) [2023-12-26 21:06:22,660][105692] Updated weights for policy 0, policy_version 789150 (0.0008) [2023-12-26 21:06:22,714][105620] Updated weights for policy 1, policy_version 789082 (0.0006) [2023-12-26 21:06:22,720][105692] Updated weights for policy 0, policy_version 789160 (0.0007) [2023-12-26 21:06:22,774][105620] Updated weights for policy 1, policy_version 789092 (0.0008) [2023-12-26 21:06:23,436][105620] Updated weights for policy 1, policy_version 789102 (0.0009) [2023-12-26 21:06:23,490][105620] Updated weights for policy 1, policy_version 789112 (0.0007) [2023-12-26 21:06:23,500][105692] Updated weights for policy 0, policy_version 789170 (0.0009) [2023-12-26 21:06:23,535][105620] Updated weights for policy 1, policy_version 789122 (0.0007) [2023-12-26 21:06:23,549][105692] Updated weights for policy 0, policy_version 789180 (0.0008) [2023-12-26 21:06:23,611][105692] Updated weights for policy 0, policy_version 789190 (0.0008) [2023-12-26 21:06:24,154][105620] Updated weights for policy 1, policy_version 789132 (0.0006) [2023-12-26 21:06:24,215][105620] Updated weights for policy 1, policy_version 789142 (0.0008) [2023-12-26 21:06:24,273][105620] Updated weights for policy 1, policy_version 789152 (0.0009) [2023-12-26 21:06:24,278][105692] Updated weights for policy 0, policy_version 789200 (0.0006) [2023-12-26 21:06:24,335][105692] Updated weights for policy 0, policy_version 789210 (0.0005) [2023-12-26 21:06:24,380][105692] Updated weights for policy 0, policy_version 789220 (0.0008) [2023-12-26 21:06:25,043][105692] Updated weights for policy 0, policy_version 789230 (0.0007) [2023-12-26 21:06:25,063][105620] Updated weights for policy 1, policy_version 789162 (0.0009) [2023-12-26 21:06:25,106][105692] Updated weights for policy 0, policy_version 789240 (0.0008) [2023-12-26 21:06:25,120][105620] Updated weights for policy 1, policy_version 789172 (0.0006) [2023-12-26 21:06:25,158][105692] Updated weights for policy 0, policy_version 789250 (0.0010) [2023-12-26 21:06:25,180][105620] Updated weights for policy 1, policy_version 789182 (0.0006) [2023-12-26 21:06:25,236][105620] Updated weights for policy 1, policy_version 789192 (0.0010) [2023-12-26 21:06:25,805][105692] Updated weights for policy 0, policy_version 789260 (0.0009) [2023-12-26 21:06:25,852][105692] Updated weights for policy 0, policy_version 789270 (0.0006) [2023-12-26 21:06:25,899][105692] Updated weights for policy 0, policy_version 789280 (0.0006) [2023-12-26 21:06:26,032][105620] Updated weights for policy 1, policy_version 789202 (0.0010) [2023-12-26 21:06:26,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 404144128. Throughput: 0: 9939.9, 1: 9659.5. Samples: 404152576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:06:26,062][104569] Avg episode reward: [(0, '9079.534'), (1, '9168.836')] [2023-12-26 21:06:26,089][105620] Updated weights for policy 1, policy_version 789212 (0.0007) [2023-12-26 21:06:26,148][105620] Updated weights for policy 1, policy_version 789222 (0.0007) [2023-12-26 21:06:26,495][105692] Updated weights for policy 0, policy_version 789290 (0.0005) [2023-12-26 21:06:26,548][105692] Updated weights for policy 0, policy_version 789300 (0.0005) [2023-12-26 21:06:26,599][105692] Updated weights for policy 0, policy_version 789310 (0.0005) [2023-12-26 21:06:26,654][105692] Updated weights for policy 0, policy_version 789320 (0.0005) [2023-12-26 21:06:27,033][105620] Updated weights for policy 1, policy_version 789232 (0.0009) [2023-12-26 21:06:27,086][105620] Updated weights for policy 1, policy_version 789242 (0.0009) [2023-12-26 21:06:27,142][105620] Updated weights for policy 1, policy_version 789252 (0.0009) [2023-12-26 21:06:27,185][105692] Updated weights for policy 0, policy_version 789330 (0.0007) [2023-12-26 21:06:27,236][105692] Updated weights for policy 0, policy_version 789340 (0.0006) [2023-12-26 21:06:27,283][105692] Updated weights for policy 0, policy_version 789350 (0.0009) [2023-12-26 21:06:27,902][105692] Updated weights for policy 0, policy_version 789360 (0.0008) [2023-12-26 21:06:27,950][105692] Updated weights for policy 0, policy_version 789370 (0.0006) [2023-12-26 21:06:27,969][105620] Updated weights for policy 1, policy_version 789262 (0.0008) [2023-12-26 21:06:28,004][105692] Updated weights for policy 0, policy_version 789380 (0.0006) [2023-12-26 21:06:28,024][105620] Updated weights for policy 1, policy_version 789272 (0.0008) [2023-12-26 21:06:28,076][105620] Updated weights for policy 1, policy_version 789282 (0.0006) [2023-12-26 21:06:28,602][105692] Updated weights for policy 0, policy_version 789390 (0.0008) [2023-12-26 21:06:28,658][105692] Updated weights for policy 0, policy_version 789400 (0.0009) [2023-12-26 21:06:28,675][105620] Updated weights for policy 1, policy_version 789292 (0.0007) [2023-12-26 21:06:28,715][105692] Updated weights for policy 0, policy_version 789410 (0.0007) [2023-12-26 21:06:28,727][105620] Updated weights for policy 1, policy_version 789302 (0.0005) [2023-12-26 21:06:28,779][105620] Updated weights for policy 1, policy_version 789312 (0.0005) [2023-12-26 21:06:29,347][105620] Updated weights for policy 1, policy_version 789322 (0.0006) [2023-12-26 21:06:29,382][105692] Updated weights for policy 0, policy_version 789420 (0.0007) [2023-12-26 21:06:29,412][105620] Updated weights for policy 1, policy_version 789332 (0.0008) [2023-12-26 21:06:29,436][105692] Updated weights for policy 0, policy_version 789430 (0.0008) [2023-12-26 21:06:29,474][105620] Updated weights for policy 1, policy_version 789342 (0.0011) [2023-12-26 21:06:29,498][105692] Updated weights for policy 0, policy_version 789440 (0.0008) [2023-12-26 21:06:29,533][105620] Updated weights for policy 1, policy_version 789352 (0.0011) [2023-12-26 21:06:30,238][105692] Updated weights for policy 0, policy_version 789450 (0.0011) [2023-12-26 21:06:30,262][105620] Updated weights for policy 1, policy_version 789362 (0.0011) [2023-12-26 21:06:30,300][105692] Updated weights for policy 0, policy_version 789460 (0.0010) [2023-12-26 21:06:30,324][105620] Updated weights for policy 1, policy_version 789372 (0.0011) [2023-12-26 21:06:30,356][105692] Updated weights for policy 0, policy_version 789470 (0.0009) [2023-12-26 21:06:30,384][105620] Updated weights for policy 1, policy_version 789382 (0.0011) [2023-12-26 21:06:30,416][105692] Updated weights for policy 0, policy_version 789480 (0.0006) [2023-12-26 21:06:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 404242432. Throughput: 0: 10104.5, 1: 9681.6. Samples: 404216016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:06:31,063][104569] Avg episode reward: [(0, '9078.711'), (1, '9351.521')] [2023-12-26 21:06:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000789384_202104832.pth... [2023-12-26 21:06:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000788264_201818112.pth [2023-12-26 21:06:31,093][105692] Updated weights for policy 0, policy_version 789490 (0.0011) [2023-12-26 21:06:31,162][105620] Updated weights for policy 1, policy_version 789392 (0.0010) [2023-12-26 21:06:31,168][105692] Updated weights for policy 0, policy_version 789500 (0.0009) [2023-12-26 21:06:31,215][105620] Updated weights for policy 1, policy_version 789402 (0.0011) [2023-12-26 21:06:31,222][105692] Updated weights for policy 0, policy_version 789510 (0.0006) [2023-12-26 21:06:31,229][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000789512_202145792.pth... [2023-12-26 21:06:31,240][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000788296_201834496.pth [2023-12-26 21:06:31,273][105620] Updated weights for policy 1, policy_version 789412 (0.0011) [2023-12-26 21:06:31,973][105692] Updated weights for policy 0, policy_version 789520 (0.0008) [2023-12-26 21:06:32,028][105692] Updated weights for policy 0, policy_version 789530 (0.0007) [2023-12-26 21:06:32,029][105620] Updated weights for policy 1, policy_version 789422 (0.0009) [2023-12-26 21:06:32,079][105692] Updated weights for policy 0, policy_version 789540 (0.0006) [2023-12-26 21:06:32,089][105620] Updated weights for policy 1, policy_version 789432 (0.0008) [2023-12-26 21:06:32,151][105620] Updated weights for policy 1, policy_version 789442 (0.0009) [2023-12-26 21:06:32,801][105692] Updated weights for policy 0, policy_version 789550 (0.0007) [2023-12-26 21:06:32,866][105692] Updated weights for policy 0, policy_version 789560 (0.0008) [2023-12-26 21:06:32,919][105620] Updated weights for policy 1, policy_version 789452 (0.0009) [2023-12-26 21:06:32,922][105692] Updated weights for policy 0, policy_version 789570 (0.0007) [2023-12-26 21:06:32,970][105620] Updated weights for policy 1, policy_version 789462 (0.0006) [2023-12-26 21:06:33,019][105620] Updated weights for policy 1, policy_version 789472 (0.0007) [2023-12-26 21:06:33,584][105620] Updated weights for policy 1, policy_version 789482 (0.0005) [2023-12-26 21:06:33,642][105620] Updated weights for policy 1, policy_version 789492 (0.0005) [2023-12-26 21:06:33,705][105620] Updated weights for policy 1, policy_version 789502 (0.0005) [2023-12-26 21:06:33,758][105692] Updated weights for policy 0, policy_version 789580 (0.0007) [2023-12-26 21:06:33,761][105620] Updated weights for policy 1, policy_version 789512 (0.0008) [2023-12-26 21:06:33,811][105692] Updated weights for policy 0, policy_version 789590 (0.0009) [2023-12-26 21:06:33,864][105692] Updated weights for policy 0, policy_version 789600 (0.0009) [2023-12-26 21:06:34,408][105620] Updated weights for policy 1, policy_version 789522 (0.0010) [2023-12-26 21:06:34,470][105620] Updated weights for policy 1, policy_version 789532 (0.0010) [2023-12-26 21:06:34,536][105620] Updated weights for policy 1, policy_version 789542 (0.0011) [2023-12-26 21:06:34,559][105692] Updated weights for policy 0, policy_version 789610 (0.0008) [2023-12-26 21:06:34,619][105692] Updated weights for policy 0, policy_version 789620 (0.0008) [2023-12-26 21:06:34,679][105692] Updated weights for policy 0, policy_version 789630 (0.0008) [2023-12-26 21:06:34,743][105692] Updated weights for policy 0, policy_version 789640 (0.0008) [2023-12-26 21:06:35,262][105620] Updated weights for policy 1, policy_version 789552 (0.0010) [2023-12-26 21:06:35,319][105620] Updated weights for policy 1, policy_version 789562 (0.0010) [2023-12-26 21:06:35,380][105620] Updated weights for policy 1, policy_version 789572 (0.0010) [2023-12-26 21:06:35,490][105692] Updated weights for policy 0, policy_version 789650 (0.0008) [2023-12-26 21:06:35,538][105692] Updated weights for policy 0, policy_version 789660 (0.0008) [2023-12-26 21:06:35,582][105692] Updated weights for policy 0, policy_version 789670 (0.0007) [2023-12-26 21:06:36,024][105620] Updated weights for policy 1, policy_version 789582 (0.0007) [2023-12-26 21:06:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 404340736. Throughput: 0: 10018.9, 1: 9711.4. Samples: 404332560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:06:36,063][104569] Avg episode reward: [(0, '9078.431'), (1, '9350.462')] [2023-12-26 21:06:36,070][105620] Updated weights for policy 1, policy_version 789592 (0.0005) [2023-12-26 21:06:36,128][105620] Updated weights for policy 1, policy_version 789602 (0.0010) [2023-12-26 21:06:36,298][105692] Updated weights for policy 0, policy_version 789680 (0.0010) [2023-12-26 21:06:36,358][105692] Updated weights for policy 0, policy_version 789690 (0.0011) [2023-12-26 21:06:36,425][105692] Updated weights for policy 0, policy_version 789700 (0.0010) [2023-12-26 21:06:36,764][105620] Updated weights for policy 1, policy_version 789612 (0.0010) [2023-12-26 21:06:36,818][105620] Updated weights for policy 1, policy_version 789622 (0.0011) [2023-12-26 21:06:36,866][105620] Updated weights for policy 1, policy_version 789632 (0.0010) [2023-12-26 21:06:37,123][105692] Updated weights for policy 0, policy_version 789710 (0.0009) [2023-12-26 21:06:37,183][105692] Updated weights for policy 0, policy_version 789720 (0.0008) [2023-12-26 21:06:37,239][105692] Updated weights for policy 0, policy_version 789730 (0.0008) [2023-12-26 21:06:37,618][105620] Updated weights for policy 1, policy_version 789642 (0.0010) [2023-12-26 21:06:37,667][105620] Updated weights for policy 1, policy_version 789652 (0.0010) [2023-12-26 21:06:37,715][105620] Updated weights for policy 1, policy_version 789662 (0.0010) [2023-12-26 21:06:37,775][105620] Updated weights for policy 1, policy_version 789672 (0.0011) [2023-12-26 21:06:37,896][105692] Updated weights for policy 0, policy_version 789740 (0.0008) [2023-12-26 21:06:37,944][105692] Updated weights for policy 0, policy_version 789750 (0.0008) [2023-12-26 21:06:37,994][105692] Updated weights for policy 0, policy_version 789760 (0.0008) [2023-12-26 21:06:38,549][105620] Updated weights for policy 1, policy_version 789682 (0.0010) [2023-12-26 21:06:38,607][105620] Updated weights for policy 1, policy_version 789692 (0.0010) [2023-12-26 21:06:38,665][105620] Updated weights for policy 1, policy_version 789702 (0.0010) [2023-12-26 21:06:38,772][105692] Updated weights for policy 0, policy_version 789770 (0.0009) [2023-12-26 21:06:38,836][105692] Updated weights for policy 0, policy_version 789780 (0.0007) [2023-12-26 21:06:38,898][105692] Updated weights for policy 0, policy_version 789790 (0.0008) [2023-12-26 21:06:38,958][105692] Updated weights for policy 0, policy_version 789800 (0.0008) [2023-12-26 21:06:39,420][105620] Updated weights for policy 1, policy_version 789712 (0.0011) [2023-12-26 21:06:39,490][105620] Updated weights for policy 1, policy_version 789722 (0.0011) [2023-12-26 21:06:39,553][105620] Updated weights for policy 1, policy_version 789732 (0.0010) [2023-12-26 21:06:39,757][105692] Updated weights for policy 0, policy_version 789810 (0.0009) [2023-12-26 21:06:39,813][105692] Updated weights for policy 0, policy_version 789820 (0.0010) [2023-12-26 21:06:39,885][105692] Updated weights for policy 0, policy_version 789830 (0.0007) [2023-12-26 21:06:40,250][105620] Updated weights for policy 1, policy_version 789742 (0.0007) [2023-12-26 21:06:40,315][105620] Updated weights for policy 1, policy_version 789752 (0.0009) [2023-12-26 21:06:40,381][105620] Updated weights for policy 1, policy_version 789762 (0.0009) [2023-12-26 21:06:40,656][105692] Updated weights for policy 0, policy_version 789840 (0.0009) [2023-12-26 21:06:40,709][105692] Updated weights for policy 0, policy_version 789850 (0.0009) [2023-12-26 21:06:40,761][105692] Updated weights for policy 0, policy_version 789860 (0.0009) [2023-12-26 21:06:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 404439040. Throughput: 0: 9935.7, 1: 9735.0. Samples: 404448204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:06:41,063][104569] Avg episode reward: [(0, '9170.828'), (1, '9169.622')] [2023-12-26 21:06:41,072][105620] Updated weights for policy 1, policy_version 789772 (0.0009) [2023-12-26 21:06:41,136][105620] Updated weights for policy 1, policy_version 789782 (0.0008) [2023-12-26 21:06:41,203][105620] Updated weights for policy 1, policy_version 789792 (0.0009) [2023-12-26 21:06:41,584][105692] Updated weights for policy 0, policy_version 789870 (0.0009) [2023-12-26 21:06:41,646][105692] Updated weights for policy 0, policy_version 789880 (0.0010) [2023-12-26 21:06:41,711][105692] Updated weights for policy 0, policy_version 789890 (0.0009) [2023-12-26 21:06:42,008][105620] Updated weights for policy 1, policy_version 789802 (0.0009) [2023-12-26 21:06:42,067][105620] Updated weights for policy 1, policy_version 789812 (0.0009) [2023-12-26 21:06:42,115][105586] KL-divergence is very high: 103.0968 [2023-12-26 21:06:42,128][105620] Updated weights for policy 1, policy_version 789822 (0.0008) [2023-12-26 21:06:42,173][105586] KL-divergence is very high: 103.1616 [2023-12-26 21:06:42,197][105620] Updated weights for policy 1, policy_version 789832 (0.0006) [2023-12-26 21:06:42,563][105692] Updated weights for policy 0, policy_version 789900 (0.0009) [2023-12-26 21:06:42,616][105692] Updated weights for policy 0, policy_version 789910 (0.0009) [2023-12-26 21:06:42,675][105692] Updated weights for policy 0, policy_version 789920 (0.0009) [2023-12-26 21:06:42,874][105620] Updated weights for policy 1, policy_version 789842 (0.0009) [2023-12-26 21:06:42,938][105620] Updated weights for policy 1, policy_version 789852 (0.0009) [2023-12-26 21:06:42,997][105620] Updated weights for policy 1, policy_version 789862 (0.0008) [2023-12-26 21:06:43,465][105692] Updated weights for policy 0, policy_version 789930 (0.0010) [2023-12-26 21:06:43,512][105692] Updated weights for policy 0, policy_version 789940 (0.0007) [2023-12-26 21:06:43,558][105692] Updated weights for policy 0, policy_version 789950 (0.0005) [2023-12-26 21:06:43,605][105692] Updated weights for policy 0, policy_version 789960 (0.0005) [2023-12-26 21:06:43,784][105620] Updated weights for policy 1, policy_version 789872 (0.0009) [2023-12-26 21:06:43,836][105620] Updated weights for policy 1, policy_version 789882 (0.0010) [2023-12-26 21:06:43,892][105620] Updated weights for policy 1, policy_version 789893 (0.0009) [2023-12-26 21:06:44,206][105692] Updated weights for policy 0, policy_version 789970 (0.0010) [2023-12-26 21:06:44,260][105692] Updated weights for policy 0, policy_version 789980 (0.0009) [2023-12-26 21:06:44,309][105692] Updated weights for policy 0, policy_version 789990 (0.0010) [2023-12-26 21:06:44,760][105620] Updated weights for policy 1, policy_version 789903 (0.0008) [2023-12-26 21:06:44,829][105620] Updated weights for policy 1, policy_version 789913 (0.0007) [2023-12-26 21:06:44,896][105620] Updated weights for policy 1, policy_version 789923 (0.0005) [2023-12-26 21:06:45,064][105692] Updated weights for policy 0, policy_version 790000 (0.0011) [2023-12-26 21:06:45,130][105692] Updated weights for policy 0, policy_version 790010 (0.0011) [2023-12-26 21:06:45,199][105692] Updated weights for policy 0, policy_version 790020 (0.0011) [2023-12-26 21:06:45,550][105620] Updated weights for policy 1, policy_version 789933 (0.0008) [2023-12-26 21:06:45,613][105620] Updated weights for policy 1, policy_version 789943 (0.0008) [2023-12-26 21:06:45,677][105620] Updated weights for policy 1, policy_version 789953 (0.0009) [2023-12-26 21:06:45,926][105692] Updated weights for policy 0, policy_version 790030 (0.0011) [2023-12-26 21:06:45,984][105692] Updated weights for policy 0, policy_version 790040 (0.0010) [2023-12-26 21:06:46,032][105692] Updated weights for policy 0, policy_version 790050 (0.0010) [2023-12-26 21:06:46,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 404537344. Throughput: 0: 9854.9, 1: 9671.8. Samples: 404501112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:06:46,063][104569] Avg episode reward: [(0, '9170.825'), (1, '8987.733')] [2023-12-26 21:06:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000790056_202285056.pth... [2023-12-26 21:06:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000789960_202252288.pth... [2023-12-26 21:06:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000788904_201990144.pth [2023-12-26 21:06:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000788840_201965568.pth [2023-12-26 21:06:46,379][105620] Updated weights for policy 1, policy_version 789963 (0.0008) [2023-12-26 21:06:46,429][105620] Updated weights for policy 1, policy_version 789973 (0.0005) [2023-12-26 21:06:46,480][105620] Updated weights for policy 1, policy_version 789983 (0.0005) [2023-12-26 21:06:46,770][105692] Updated weights for policy 0, policy_version 790060 (0.0011) [2023-12-26 21:06:46,828][105692] Updated weights for policy 0, policy_version 790070 (0.0011) [2023-12-26 21:06:46,880][105692] Updated weights for policy 0, policy_version 790080 (0.0010) [2023-12-26 21:06:47,145][105620] Updated weights for policy 1, policy_version 789993 (0.0005) [2023-12-26 21:06:47,208][105620] Updated weights for policy 1, policy_version 790003 (0.0009) [2023-12-26 21:06:47,272][105620] Updated weights for policy 1, policy_version 790013 (0.0008) [2023-12-26 21:06:47,317][105620] Updated weights for policy 1, policy_version 790023 (0.0008) [2023-12-26 21:06:47,592][105692] Updated weights for policy 0, policy_version 790090 (0.0010) [2023-12-26 21:06:47,644][105692] Updated weights for policy 0, policy_version 790100 (0.0009) [2023-12-26 21:06:47,692][105692] Updated weights for policy 0, policy_version 790110 (0.0008) [2023-12-26 21:06:47,752][105692] Updated weights for policy 0, policy_version 790120 (0.0009) [2023-12-26 21:06:48,103][105620] Updated weights for policy 1, policy_version 790033 (0.0008) [2023-12-26 21:06:48,161][105620] Updated weights for policy 1, policy_version 790043 (0.0009) [2023-12-26 21:06:48,220][105620] Updated weights for policy 1, policy_version 790053 (0.0009) [2023-12-26 21:06:48,598][105692] Updated weights for policy 0, policy_version 790130 (0.0009) [2023-12-26 21:06:48,660][105692] Updated weights for policy 0, policy_version 790140 (0.0009) [2023-12-26 21:06:48,720][105692] Updated weights for policy 0, policy_version 790150 (0.0009) [2023-12-26 21:06:48,967][105620] Updated weights for policy 1, policy_version 790063 (0.0007) [2023-12-26 21:06:49,032][105620] Updated weights for policy 1, policy_version 790073 (0.0007) [2023-12-26 21:06:49,101][105620] Updated weights for policy 1, policy_version 790083 (0.0006) [2023-12-26 21:06:49,561][105692] Updated weights for policy 0, policy_version 790160 (0.0008) [2023-12-26 21:06:49,622][105692] Updated weights for policy 0, policy_version 790170 (0.0008) [2023-12-26 21:06:49,683][105692] Updated weights for policy 0, policy_version 790180 (0.0009) [2023-12-26 21:06:49,855][105620] Updated weights for policy 1, policy_version 790093 (0.0007) [2023-12-26 21:06:49,924][105620] Updated weights for policy 1, policy_version 790103 (0.0008) [2023-12-26 21:06:49,982][105620] Updated weights for policy 1, policy_version 790113 (0.0006) [2023-12-26 21:06:50,401][105692] Updated weights for policy 0, policy_version 790190 (0.0008) [2023-12-26 21:06:50,455][105692] Updated weights for policy 0, policy_version 790200 (0.0010) [2023-12-26 21:06:50,508][105692] Updated weights for policy 0, policy_version 790210 (0.0009) [2023-12-26 21:06:50,618][105620] Updated weights for policy 1, policy_version 790123 (0.0007) [2023-12-26 21:06:50,693][105620] Updated weights for policy 1, policy_version 790133 (0.0010) [2023-12-26 21:06:50,763][105620] Updated weights for policy 1, policy_version 790143 (0.0010) [2023-12-26 21:06:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 404627456. Throughput: 0: 9795.3, 1: 9636.4. Samples: 404615556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:06:51,063][104569] Avg episode reward: [(0, '6960.996'), (1, '9078.604')] [2023-12-26 21:06:51,188][105692] Updated weights for policy 0, policy_version 790220 (0.0009) [2023-12-26 21:06:51,243][105692] Updated weights for policy 0, policy_version 790230 (0.0008) [2023-12-26 21:06:51,303][105692] Updated weights for policy 0, policy_version 790240 (0.0006) [2023-12-26 21:06:51,573][105620] Updated weights for policy 1, policy_version 790153 (0.0009) [2023-12-26 21:06:51,637][105620] Updated weights for policy 1, policy_version 790163 (0.0008) [2023-12-26 21:06:51,697][105620] Updated weights for policy 1, policy_version 790173 (0.0008) [2023-12-26 21:06:51,767][105620] Updated weights for policy 1, policy_version 790183 (0.0008) [2023-12-26 21:06:51,985][105692] Updated weights for policy 0, policy_version 790250 (0.0007) [2023-12-26 21:06:52,044][105692] Updated weights for policy 0, policy_version 790260 (0.0008) [2023-12-26 21:06:52,096][105692] Updated weights for policy 0, policy_version 790270 (0.0008) [2023-12-26 21:06:52,155][105692] Updated weights for policy 0, policy_version 790280 (0.0008) [2023-12-26 21:06:52,540][105620] Updated weights for policy 1, policy_version 790193 (0.0008) [2023-12-26 21:06:52,593][105620] Updated weights for policy 1, policy_version 790203 (0.0009) [2023-12-26 21:06:52,647][105620] Updated weights for policy 1, policy_version 790213 (0.0009) [2023-12-26 21:06:52,818][105692] Updated weights for policy 0, policy_version 790290 (0.0009) [2023-12-26 21:06:52,881][105692] Updated weights for policy 0, policy_version 790300 (0.0010) [2023-12-26 21:06:52,937][105692] Updated weights for policy 0, policy_version 790310 (0.0009) [2023-12-26 21:06:53,419][105620] Updated weights for policy 1, policy_version 790223 (0.0010) [2023-12-26 21:06:53,470][105620] Updated weights for policy 1, policy_version 790233 (0.0010) [2023-12-26 21:06:53,518][105620] Updated weights for policy 1, policy_version 790243 (0.0010) [2023-12-26 21:06:53,664][105692] Updated weights for policy 0, policy_version 790320 (0.0010) [2023-12-26 21:06:53,722][105692] Updated weights for policy 0, policy_version 790330 (0.0010) [2023-12-26 21:06:53,779][105692] Updated weights for policy 0, policy_version 790340 (0.0009) [2023-12-26 21:06:54,269][105620] Updated weights for policy 1, policy_version 790253 (0.0010) [2023-12-26 21:06:54,323][105620] Updated weights for policy 1, policy_version 790263 (0.0010) [2023-12-26 21:06:54,388][105620] Updated weights for policy 1, policy_version 790273 (0.0010) [2023-12-26 21:06:54,415][105692] Updated weights for policy 0, policy_version 790350 (0.0008) [2023-12-26 21:06:54,466][105692] Updated weights for policy 0, policy_version 790360 (0.0010) [2023-12-26 21:06:54,514][105692] Updated weights for policy 0, policy_version 790370 (0.0006) [2023-12-26 21:06:55,071][105620] Updated weights for policy 1, policy_version 790283 (0.0010) [2023-12-26 21:06:55,125][105620] Updated weights for policy 1, policy_version 790293 (0.0006) [2023-12-26 21:06:55,179][105620] Updated weights for policy 1, policy_version 790303 (0.0005) [2023-12-26 21:06:55,230][105692] Updated weights for policy 0, policy_version 790380 (0.0005) [2023-12-26 21:06:55,288][105692] Updated weights for policy 0, policy_version 790390 (0.0005) [2023-12-26 21:06:55,334][105692] Updated weights for policy 0, policy_version 790400 (0.0005) [2023-12-26 21:06:55,766][105620] Updated weights for policy 1, policy_version 790313 (0.0008) [2023-12-26 21:06:55,823][105620] Updated weights for policy 1, policy_version 790323 (0.0010) [2023-12-26 21:06:55,860][105692] Updated weights for policy 0, policy_version 790410 (0.0005) [2023-12-26 21:06:55,872][105620] Updated weights for policy 1, policy_version 790333 (0.0009) [2023-12-26 21:06:55,904][105692] Updated weights for policy 0, policy_version 790420 (0.0005) [2023-12-26 21:06:55,921][105620] Updated weights for policy 1, policy_version 790343 (0.0005) [2023-12-26 21:06:55,951][105692] Updated weights for policy 0, policy_version 790430 (0.0005) [2023-12-26 21:06:55,998][105692] Updated weights for policy 0, policy_version 790440 (0.0005) [2023-12-26 21:06:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.7, 300 sec: 19633.0). Total num frames: 404733952. Throughput: 0: 9863.3, 1: 9626.2. Samples: 404734652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:06:56,063][104569] Avg episode reward: [(0, '6986.096'), (1, '9262.225')] [2023-12-26 21:06:56,642][105692] Updated weights for policy 0, policy_version 790450 (0.0005) [2023-12-26 21:06:56,697][105692] Updated weights for policy 0, policy_version 790460 (0.0005) [2023-12-26 21:06:56,697][105620] Updated weights for policy 1, policy_version 790353 (0.0010) [2023-12-26 21:06:56,742][105620] Updated weights for policy 1, policy_version 790363 (0.0010) [2023-12-26 21:06:56,759][105692] Updated weights for policy 0, policy_version 790470 (0.0005) [2023-12-26 21:06:56,792][105620] Updated weights for policy 1, policy_version 790373 (0.0008) [2023-12-26 21:06:57,383][105692] Updated weights for policy 0, policy_version 790480 (0.0007) [2023-12-26 21:06:57,451][105692] Updated weights for policy 0, policy_version 790490 (0.0008) [2023-12-26 21:06:57,463][105620] Updated weights for policy 1, policy_version 790383 (0.0005) [2023-12-26 21:06:57,509][105620] Updated weights for policy 1, policy_version 790393 (0.0006) [2023-12-26 21:06:57,511][105692] Updated weights for policy 0, policy_version 790500 (0.0007) [2023-12-26 21:06:57,566][105620] Updated weights for policy 1, policy_version 790403 (0.0005) [2023-12-26 21:06:58,065][105692] Updated weights for policy 0, policy_version 790510 (0.0009) [2023-12-26 21:06:58,119][105692] Updated weights for policy 0, policy_version 790520 (0.0006) [2023-12-26 21:06:58,141][105620] Updated weights for policy 1, policy_version 790413 (0.0006) [2023-12-26 21:06:58,184][105692] Updated weights for policy 0, policy_version 790530 (0.0007) [2023-12-26 21:06:58,207][105620] Updated weights for policy 1, policy_version 790423 (0.0008) [2023-12-26 21:06:58,270][105620] Updated weights for policy 1, policy_version 790433 (0.0008) [2023-12-26 21:06:59,035][105692] Updated weights for policy 0, policy_version 790540 (0.0007) [2023-12-26 21:06:59,063][105620] Updated weights for policy 1, policy_version 790443 (0.0008) [2023-12-26 21:06:59,093][105692] Updated weights for policy 0, policy_version 790550 (0.0007) [2023-12-26 21:06:59,123][105620] Updated weights for policy 1, policy_version 790453 (0.0008) [2023-12-26 21:06:59,154][105692] Updated weights for policy 0, policy_version 790560 (0.0007) [2023-12-26 21:06:59,186][105620] Updated weights for policy 1, policy_version 790463 (0.0008) [2023-12-26 21:06:59,846][105620] Updated weights for policy 1, policy_version 790473 (0.0007) [2023-12-26 21:06:59,916][105620] Updated weights for policy 1, policy_version 790483 (0.0007) [2023-12-26 21:06:59,983][105620] Updated weights for policy 1, policy_version 790493 (0.0007) [2023-12-26 21:06:59,990][105692] Updated weights for policy 0, policy_version 790570 (0.0009) [2023-12-26 21:07:00,046][105692] Updated weights for policy 0, policy_version 790580 (0.0008) [2023-12-26 21:07:00,049][105620] Updated weights for policy 1, policy_version 790503 (0.0006) [2023-12-26 21:07:00,101][105692] Updated weights for policy 0, policy_version 790590 (0.0008) [2023-12-26 21:07:00,146][105692] Updated weights for policy 0, policy_version 790600 (0.0009) [2023-12-26 21:07:00,761][105620] Updated weights for policy 1, policy_version 790513 (0.0009) [2023-12-26 21:07:00,811][105620] Updated weights for policy 1, policy_version 790523 (0.0008) [2023-12-26 21:07:00,826][105692] Updated weights for policy 0, policy_version 790610 (0.0006) [2023-12-26 21:07:00,857][105620] Updated weights for policy 1, policy_version 790533 (0.0007) [2023-12-26 21:07:00,882][105692] Updated weights for policy 0, policy_version 790620 (0.0007) [2023-12-26 21:07:00,930][105692] Updated weights for policy 0, policy_version 790630 (0.0010) [2023-12-26 21:07:01,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 404832256. Throughput: 0: 9922.5, 1: 9621.1. Samples: 404798340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:07:01,062][104569] Avg episode reward: [(0, '8411.037'), (1, '9140.706')] [2023-12-26 21:07:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000790632_202432512.pth... [2023-12-26 21:07:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000790536_202399744.pth... [2023-12-26 21:07:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000789384_202104832.pth [2023-12-26 21:07:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000789512_202145792.pth [2023-12-26 21:07:01,529][105692] Updated weights for policy 0, policy_version 790640 (0.0007) [2023-12-26 21:07:01,588][105692] Updated weights for policy 0, policy_version 790650 (0.0007) [2023-12-26 21:07:01,649][105692] Updated weights for policy 0, policy_version 790660 (0.0007) [2023-12-26 21:07:01,746][105620] Updated weights for policy 1, policy_version 790543 (0.0008) [2023-12-26 21:07:01,804][105620] Updated weights for policy 1, policy_version 790553 (0.0009) [2023-12-26 21:07:01,867][105620] Updated weights for policy 1, policy_version 790563 (0.0008) [2023-12-26 21:07:02,233][105692] Updated weights for policy 0, policy_version 790670 (0.0006) [2023-12-26 21:07:02,298][105692] Updated weights for policy 0, policy_version 790680 (0.0007) [2023-12-26 21:07:02,353][105692] Updated weights for policy 0, policy_version 790690 (0.0007) [2023-12-26 21:07:02,629][105620] Updated weights for policy 1, policy_version 790573 (0.0006) [2023-12-26 21:07:02,687][105620] Updated weights for policy 1, policy_version 790583 (0.0009) [2023-12-26 21:07:02,733][105620] Updated weights for policy 1, policy_version 790593 (0.0009) [2023-12-26 21:07:03,060][105692] Updated weights for policy 0, policy_version 790700 (0.0010) [2023-12-26 21:07:03,106][105692] Updated weights for policy 0, policy_version 790710 (0.0008) [2023-12-26 21:07:03,160][105692] Updated weights for policy 0, policy_version 790720 (0.0009) [2023-12-26 21:07:03,518][105620] Updated weights for policy 1, policy_version 790603 (0.0009) [2023-12-26 21:07:03,566][105620] Updated weights for policy 1, policy_version 790613 (0.0009) [2023-12-26 21:07:03,618][105620] Updated weights for policy 1, policy_version 790623 (0.0009) [2023-12-26 21:07:03,844][105692] Updated weights for policy 0, policy_version 790730 (0.0008) [2023-12-26 21:07:03,902][105692] Updated weights for policy 0, policy_version 790740 (0.0007) [2023-12-26 21:07:03,964][105692] Updated weights for policy 0, policy_version 790750 (0.0006) [2023-12-26 21:07:04,025][105692] Updated weights for policy 0, policy_version 790760 (0.0007) [2023-12-26 21:07:04,425][105620] Updated weights for policy 1, policy_version 790633 (0.0009) [2023-12-26 21:07:04,477][105620] Updated weights for policy 1, policy_version 790643 (0.0006) [2023-12-26 21:07:04,543][105620] Updated weights for policy 1, policy_version 790653 (0.0009) [2023-12-26 21:07:04,589][105620] Updated weights for policy 1, policy_version 790663 (0.0009) [2023-12-26 21:07:04,680][105692] Updated weights for policy 0, policy_version 790770 (0.0009) [2023-12-26 21:07:04,730][105692] Updated weights for policy 0, policy_version 790780 (0.0008) [2023-12-26 21:07:04,776][105692] Updated weights for policy 0, policy_version 790790 (0.0009) [2023-12-26 21:07:05,328][105620] Updated weights for policy 1, policy_version 790673 (0.0009) [2023-12-26 21:07:05,376][105620] Updated weights for policy 1, policy_version 790683 (0.0007) [2023-12-26 21:07:05,430][105620] Updated weights for policy 1, policy_version 790693 (0.0005) [2023-12-26 21:07:05,551][105692] Updated weights for policy 0, policy_version 790800 (0.0010) [2023-12-26 21:07:05,617][105692] Updated weights for policy 0, policy_version 790810 (0.0010) [2023-12-26 21:07:05,672][105692] Updated weights for policy 0, policy_version 790820 (0.0009) [2023-12-26 21:07:06,054][105620] Updated weights for policy 1, policy_version 790703 (0.0005) [2023-12-26 21:07:06,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 404922368. Throughput: 0: 9910.0, 1: 9578.9. Samples: 404914176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:07:06,062][104569] Avg episode reward: [(0, '8911.902'), (1, '9048.058')] [2023-12-26 21:07:06,124][105620] Updated weights for policy 1, policy_version 790713 (0.0008) [2023-12-26 21:07:06,178][105620] Updated weights for policy 1, policy_version 790723 (0.0008) [2023-12-26 21:07:06,331][105692] Updated weights for policy 0, policy_version 790830 (0.0008) [2023-12-26 21:07:06,395][105692] Updated weights for policy 0, policy_version 790840 (0.0008) [2023-12-26 21:07:06,461][105692] Updated weights for policy 0, policy_version 790850 (0.0005) [2023-12-26 21:07:06,945][105620] Updated weights for policy 1, policy_version 790733 (0.0011) [2023-12-26 21:07:06,997][105620] Updated weights for policy 1, policy_version 790743 (0.0009) [2023-12-26 21:07:07,049][105620] Updated weights for policy 1, policy_version 790753 (0.0007) [2023-12-26 21:07:07,128][105692] Updated weights for policy 0, policy_version 790860 (0.0009) [2023-12-26 21:07:07,193][105692] Updated weights for policy 0, policy_version 790870 (0.0009) [2023-12-26 21:07:07,254][105692] Updated weights for policy 0, policy_version 790880 (0.0010) [2023-12-26 21:07:07,804][105620] Updated weights for policy 1, policy_version 790763 (0.0006) [2023-12-26 21:07:07,872][105620] Updated weights for policy 1, policy_version 790773 (0.0005) [2023-12-26 21:07:07,941][105620] Updated weights for policy 1, policy_version 790783 (0.0005) [2023-12-26 21:07:07,956][105692] Updated weights for policy 0, policy_version 790890 (0.0009) [2023-12-26 21:07:08,008][105692] Updated weights for policy 0, policy_version 790900 (0.0006) [2023-12-26 21:07:08,062][105692] Updated weights for policy 0, policy_version 790910 (0.0005) [2023-12-26 21:07:08,119][105692] Updated weights for policy 0, policy_version 790920 (0.0009) [2023-12-26 21:07:08,643][105620] Updated weights for policy 1, policy_version 790793 (0.0007) [2023-12-26 21:07:08,698][105620] Updated weights for policy 1, policy_version 790803 (0.0008) [2023-12-26 21:07:08,746][105620] Updated weights for policy 1, policy_version 790813 (0.0007) [2023-12-26 21:07:08,796][105620] Updated weights for policy 1, policy_version 790823 (0.0007) [2023-12-26 21:07:08,809][105692] Updated weights for policy 0, policy_version 790930 (0.0011) [2023-12-26 21:07:08,872][105692] Updated weights for policy 0, policy_version 790940 (0.0011) [2023-12-26 21:07:08,935][105692] Updated weights for policy 0, policy_version 790950 (0.0011) [2023-12-26 21:07:09,581][105620] Updated weights for policy 1, policy_version 790833 (0.0008) [2023-12-26 21:07:09,633][105620] Updated weights for policy 1, policy_version 790843 (0.0009) [2023-12-26 21:07:09,677][105692] Updated weights for policy 0, policy_version 790960 (0.0007) [2023-12-26 21:07:09,687][105620] Updated weights for policy 1, policy_version 790853 (0.0007) [2023-12-26 21:07:09,737][105692] Updated weights for policy 0, policy_version 790970 (0.0008) [2023-12-26 21:07:09,807][105692] Updated weights for policy 0, policy_version 790980 (0.0009) [2023-12-26 21:07:10,461][105620] Updated weights for policy 1, policy_version 790863 (0.0008) [2023-12-26 21:07:10,492][105692] Updated weights for policy 0, policy_version 790990 (0.0008) [2023-12-26 21:07:10,531][105620] Updated weights for policy 1, policy_version 790873 (0.0009) [2023-12-26 21:07:10,557][105692] Updated weights for policy 0, policy_version 791000 (0.0007) [2023-12-26 21:07:10,582][105620] Updated weights for policy 1, policy_version 790883 (0.0010) [2023-12-26 21:07:10,617][105692] Updated weights for policy 0, policy_version 791010 (0.0008) [2023-12-26 21:07:11,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 405020672. Throughput: 0: 9911.3, 1: 9585.0. Samples: 405029908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:07:11,063][104569] Avg episode reward: [(0, '9002.237'), (1, '9167.865')] [2023-12-26 21:07:11,279][105620] Updated weights for policy 1, policy_version 790893 (0.0007) [2023-12-26 21:07:11,346][105620] Updated weights for policy 1, policy_version 790903 (0.0009) [2023-12-26 21:07:11,415][105620] Updated weights for policy 1, policy_version 790913 (0.0007) [2023-12-26 21:07:11,416][105692] Updated weights for policy 0, policy_version 791020 (0.0007) [2023-12-26 21:07:11,467][105692] Updated weights for policy 0, policy_version 791030 (0.0007) [2023-12-26 21:07:11,515][105692] Updated weights for policy 0, policy_version 791040 (0.0009) [2023-12-26 21:07:12,147][105692] Updated weights for policy 0, policy_version 791050 (0.0008) [2023-12-26 21:07:12,186][105620] Updated weights for policy 1, policy_version 790923 (0.0007) [2023-12-26 21:07:12,205][105692] Updated weights for policy 0, policy_version 791060 (0.0006) [2023-12-26 21:07:12,247][105620] Updated weights for policy 1, policy_version 790933 (0.0007) [2023-12-26 21:07:12,265][105692] Updated weights for policy 0, policy_version 791070 (0.0007) [2023-12-26 21:07:12,306][105620] Updated weights for policy 1, policy_version 790943 (0.0008) [2023-12-26 21:07:12,326][105692] Updated weights for policy 0, policy_version 791080 (0.0008) [2023-12-26 21:07:12,943][105692] Updated weights for policy 0, policy_version 791090 (0.0009) [2023-12-26 21:07:13,003][105692] Updated weights for policy 0, policy_version 791100 (0.0005) [2023-12-26 21:07:13,062][105692] Updated weights for policy 0, policy_version 791110 (0.0005) [2023-12-26 21:07:13,194][105620] Updated weights for policy 1, policy_version 790953 (0.0008) [2023-12-26 21:07:13,256][105620] Updated weights for policy 1, policy_version 790963 (0.0009) [2023-12-26 21:07:13,328][105620] Updated weights for policy 1, policy_version 790973 (0.0010) [2023-12-26 21:07:13,390][105620] Updated weights for policy 1, policy_version 790983 (0.0010) [2023-12-26 21:07:13,590][105692] Updated weights for policy 0, policy_version 791120 (0.0005) [2023-12-26 21:07:13,639][105692] Updated weights for policy 0, policy_version 791130 (0.0005) [2023-12-26 21:07:13,703][105692] Updated weights for policy 0, policy_version 791140 (0.0006) [2023-12-26 21:07:14,244][105692] Updated weights for policy 0, policy_version 791150 (0.0006) [2023-12-26 21:07:14,294][105620] Updated weights for policy 1, policy_version 790993 (0.0009) [2023-12-26 21:07:14,309][105692] Updated weights for policy 0, policy_version 791160 (0.0007) [2023-12-26 21:07:14,358][105620] Updated weights for policy 1, policy_version 791003 (0.0008) [2023-12-26 21:07:14,366][105692] Updated weights for policy 0, policy_version 791170 (0.0006) [2023-12-26 21:07:14,413][105620] Updated weights for policy 1, policy_version 791013 (0.0006) [2023-12-26 21:07:14,951][105692] Updated weights for policy 0, policy_version 791180 (0.0009) [2023-12-26 21:07:15,009][105692] Updated weights for policy 0, policy_version 791190 (0.0009) [2023-12-26 21:07:15,078][105692] Updated weights for policy 0, policy_version 791201 (0.0010) [2023-12-26 21:07:15,256][105620] Updated weights for policy 1, policy_version 791023 (0.0010) [2023-12-26 21:07:15,328][105620] Updated weights for policy 1, policy_version 791033 (0.0009) [2023-12-26 21:07:15,398][105620] Updated weights for policy 1, policy_version 791043 (0.0010) [2023-12-26 21:07:15,693][105692] Updated weights for policy 0, policy_version 791211 (0.0009) [2023-12-26 21:07:15,755][105692] Updated weights for policy 0, policy_version 791221 (0.0006) [2023-12-26 21:07:15,816][105692] Updated weights for policy 0, policy_version 791231 (0.0006) [2023-12-26 21:07:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.4, 300 sec: 19605.3). Total num frames: 405118976. Throughput: 0: 9839.3, 1: 9526.4. Samples: 405087472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:07:16,062][104569] Avg episode reward: [(0, '9174.382'), (1, '9260.387')] [2023-12-26 21:07:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000791240_202588160.pth... [2023-12-26 21:07:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000791048_202530816.pth... [2023-12-26 21:07:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000790056_202285056.pth [2023-12-26 21:07:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000789960_202252288.pth [2023-12-26 21:07:16,289][105620] Updated weights for policy 1, policy_version 791053 (0.0010) [2023-12-26 21:07:16,342][105620] Updated weights for policy 1, policy_version 791063 (0.0009) [2023-12-26 21:07:16,385][105692] Updated weights for policy 0, policy_version 791241 (0.0010) [2023-12-26 21:07:16,395][105620] Updated weights for policy 1, policy_version 791073 (0.0009) [2023-12-26 21:07:16,441][105692] Updated weights for policy 0, policy_version 791251 (0.0007) [2023-12-26 21:07:16,502][105692] Updated weights for policy 0, policy_version 791261 (0.0008) [2023-12-26 21:07:16,554][105692] Updated weights for policy 0, policy_version 791271 (0.0008) [2023-12-26 21:07:17,178][105620] Updated weights for policy 1, policy_version 791083 (0.0009) [2023-12-26 21:07:17,236][105620] Updated weights for policy 1, policy_version 791093 (0.0009) [2023-12-26 21:07:17,293][105620] Updated weights for policy 1, policy_version 791103 (0.0008) [2023-12-26 21:07:17,299][105692] Updated weights for policy 0, policy_version 791281 (0.0006) [2023-12-26 21:07:17,353][105692] Updated weights for policy 0, policy_version 791291 (0.0005) [2023-12-26 21:07:17,403][105692] Updated weights for policy 0, policy_version 791301 (0.0009) [2023-12-26 21:07:18,055][105620] Updated weights for policy 1, policy_version 791113 (0.0008) [2023-12-26 21:07:18,122][105620] Updated weights for policy 1, policy_version 791123 (0.0008) [2023-12-26 21:07:18,149][105692] Updated weights for policy 0, policy_version 791311 (0.0008) [2023-12-26 21:07:18,183][105620] Updated weights for policy 1, policy_version 791133 (0.0008) [2023-12-26 21:07:18,209][105692] Updated weights for policy 0, policy_version 791321 (0.0006) [2023-12-26 21:07:18,240][105620] Updated weights for policy 1, policy_version 791143 (0.0006) [2023-12-26 21:07:18,265][105692] Updated weights for policy 0, policy_version 791331 (0.0007) [2023-12-26 21:07:19,028][105692] Updated weights for policy 0, policy_version 791341 (0.0007) [2023-12-26 21:07:19,033][105620] Updated weights for policy 1, policy_version 791153 (0.0009) [2023-12-26 21:07:19,081][105620] Updated weights for policy 1, policy_version 791163 (0.0009) [2023-12-26 21:07:19,089][105692] Updated weights for policy 0, policy_version 791351 (0.0005) [2023-12-26 21:07:19,130][105620] Updated weights for policy 1, policy_version 791173 (0.0007) [2023-12-26 21:07:19,140][105692] Updated weights for policy 0, policy_version 791361 (0.0008) [2023-12-26 21:07:19,809][105692] Updated weights for policy 0, policy_version 791371 (0.0009) [2023-12-26 21:07:19,878][105692] Updated weights for policy 0, policy_version 791381 (0.0010) [2023-12-26 21:07:19,948][105692] Updated weights for policy 0, policy_version 791391 (0.0011) [2023-12-26 21:07:19,967][105620] Updated weights for policy 1, policy_version 791183 (0.0005) [2023-12-26 21:07:20,030][105620] Updated weights for policy 1, policy_version 791193 (0.0009) [2023-12-26 21:07:20,098][105620] Updated weights for policy 1, policy_version 791203 (0.0008) [2023-12-26 21:07:20,677][105692] Updated weights for policy 0, policy_version 791401 (0.0011) [2023-12-26 21:07:20,739][105692] Updated weights for policy 0, policy_version 791411 (0.0010) [2023-12-26 21:07:20,798][105692] Updated weights for policy 0, policy_version 791421 (0.0010) [2023-12-26 21:07:20,865][105692] Updated weights for policy 0, policy_version 791431 (0.0011) [2023-12-26 21:07:20,867][105620] Updated weights for policy 1, policy_version 791213 (0.0007) [2023-12-26 21:07:20,928][105620] Updated weights for policy 1, policy_version 791223 (0.0008) [2023-12-26 21:07:20,988][105620] Updated weights for policy 1, policy_version 791233 (0.0008) [2023-12-26 21:07:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 405217280. Throughput: 0: 9965.6, 1: 9364.0. Samples: 405202392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:07:21,062][104569] Avg episode reward: [(0, '9262.426'), (1, '9262.015')] [2023-12-26 21:07:21,613][105692] Updated weights for policy 0, policy_version 791441 (0.0008) [2023-12-26 21:07:21,675][105692] Updated weights for policy 0, policy_version 791451 (0.0008) [2023-12-26 21:07:21,721][105620] Updated weights for policy 1, policy_version 791243 (0.0008) [2023-12-26 21:07:21,742][105692] Updated weights for policy 0, policy_version 791461 (0.0008) [2023-12-26 21:07:21,780][105620] Updated weights for policy 1, policy_version 791253 (0.0007) [2023-12-26 21:07:21,836][105620] Updated weights for policy 1, policy_version 791263 (0.0006) [2023-12-26 21:07:22,460][105620] Updated weights for policy 1, policy_version 791273 (0.0007) [2023-12-26 21:07:22,520][105620] Updated weights for policy 1, policy_version 791283 (0.0009) [2023-12-26 21:07:22,570][105692] Updated weights for policy 0, policy_version 791471 (0.0008) [2023-12-26 21:07:22,587][105620] Updated weights for policy 1, policy_version 791293 (0.0008) [2023-12-26 21:07:22,629][105692] Updated weights for policy 0, policy_version 791481 (0.0008) [2023-12-26 21:07:22,652][105620] Updated weights for policy 1, policy_version 791303 (0.0007) [2023-12-26 21:07:22,694][105692] Updated weights for policy 0, policy_version 791491 (0.0006) [2023-12-26 21:07:23,299][105692] Updated weights for policy 0, policy_version 791501 (0.0009) [2023-12-26 21:07:23,342][105620] Updated weights for policy 1, policy_version 791313 (0.0008) [2023-12-26 21:07:23,348][105692] Updated weights for policy 0, policy_version 791511 (0.0007) [2023-12-26 21:07:23,395][105620] Updated weights for policy 1, policy_version 791323 (0.0007) [2023-12-26 21:07:23,409][105692] Updated weights for policy 0, policy_version 791521 (0.0008) [2023-12-26 21:07:23,446][105620] Updated weights for policy 1, policy_version 791333 (0.0008) [2023-12-26 21:07:24,115][105692] Updated weights for policy 0, policy_version 791531 (0.0008) [2023-12-26 21:07:24,147][105620] Updated weights for policy 1, policy_version 791343 (0.0008) [2023-12-26 21:07:24,165][105692] Updated weights for policy 0, policy_version 791541 (0.0007) [2023-12-26 21:07:24,201][105620] Updated weights for policy 1, policy_version 791353 (0.0007) [2023-12-26 21:07:24,217][105692] Updated weights for policy 0, policy_version 791551 (0.0009) [2023-12-26 21:07:24,257][105620] Updated weights for policy 1, policy_version 791363 (0.0008) [2023-12-26 21:07:24,935][105620] Updated weights for policy 1, policy_version 791373 (0.0007) [2023-12-26 21:07:24,959][105692] Updated weights for policy 0, policy_version 791561 (0.0010) [2023-12-26 21:07:24,993][105620] Updated weights for policy 1, policy_version 791383 (0.0007) [2023-12-26 21:07:25,011][105692] Updated weights for policy 0, policy_version 791571 (0.0010) [2023-12-26 21:07:25,052][105620] Updated weights for policy 1, policy_version 791393 (0.0005) [2023-12-26 21:07:25,066][105692] Updated weights for policy 0, policy_version 791581 (0.0010) [2023-12-26 21:07:25,117][105692] Updated weights for policy 0, policy_version 791591 (0.0010) [2023-12-26 21:07:25,826][105620] Updated weights for policy 1, policy_version 791403 (0.0007) [2023-12-26 21:07:25,833][105692] Updated weights for policy 0, policy_version 791601 (0.0008) [2023-12-26 21:07:25,882][105620] Updated weights for policy 1, policy_version 791413 (0.0010) [2023-12-26 21:07:25,883][105692] Updated weights for policy 0, policy_version 791611 (0.0009) [2023-12-26 21:07:25,942][105620] Updated weights for policy 1, policy_version 791423 (0.0011) [2023-12-26 21:07:25,944][105692] Updated weights for policy 0, policy_version 791621 (0.0007) [2023-12-26 21:07:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 405315584. Throughput: 0: 9997.3, 1: 9342.9. Samples: 405318516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:07:26,063][104569] Avg episode reward: [(0, '8903.914'), (1, '8585.586')] [2023-12-26 21:07:26,576][105692] Updated weights for policy 0, policy_version 791631 (0.0006) [2023-12-26 21:07:26,627][105692] Updated weights for policy 0, policy_version 791641 (0.0005) [2023-12-26 21:07:26,641][105620] Updated weights for policy 1, policy_version 791433 (0.0010) [2023-12-26 21:07:26,679][105692] Updated weights for policy 0, policy_version 791651 (0.0010) [2023-12-26 21:07:26,696][105620] Updated weights for policy 1, policy_version 791443 (0.0005) [2023-12-26 21:07:26,750][105620] Updated weights for policy 1, policy_version 791453 (0.0007) [2023-12-26 21:07:26,804][105620] Updated weights for policy 1, policy_version 791463 (0.0008) [2023-12-26 21:07:27,352][105692] Updated weights for policy 0, policy_version 791661 (0.0008) [2023-12-26 21:07:27,407][105692] Updated weights for policy 0, policy_version 791671 (0.0005) [2023-12-26 21:07:27,469][105692] Updated weights for policy 0, policy_version 791681 (0.0007) [2023-12-26 21:07:27,563][105620] Updated weights for policy 1, policy_version 791473 (0.0005) [2023-12-26 21:07:27,614][105620] Updated weights for policy 1, policy_version 791483 (0.0005) [2023-12-26 21:07:27,619][105586] KL-divergence is very high: 104.5002 [2023-12-26 21:07:27,664][105620] Updated weights for policy 1, policy_version 791493 (0.0005) [2023-12-26 21:07:28,058][105692] Updated weights for policy 0, policy_version 791691 (0.0009) [2023-12-26 21:07:28,113][105692] Updated weights for policy 0, policy_version 791701 (0.0005) [2023-12-26 21:07:28,163][105692] Updated weights for policy 0, policy_version 791711 (0.0005) [2023-12-26 21:07:28,402][105620] Updated weights for policy 1, policy_version 791503 (0.0007) [2023-12-26 21:07:28,462][105620] Updated weights for policy 1, policy_version 791513 (0.0006) [2023-12-26 21:07:28,529][105620] Updated weights for policy 1, policy_version 791523 (0.0006) [2023-12-26 21:07:28,766][105692] Updated weights for policy 0, policy_version 791721 (0.0006) [2023-12-26 21:07:28,824][105692] Updated weights for policy 0, policy_version 791731 (0.0010) [2023-12-26 21:07:28,882][105692] Updated weights for policy 0, policy_version 791741 (0.0010) [2023-12-26 21:07:28,937][105692] Updated weights for policy 0, policy_version 791751 (0.0010) [2023-12-26 21:07:29,273][105620] Updated weights for policy 1, policy_version 791533 (0.0010) [2023-12-26 21:07:29,339][105620] Updated weights for policy 1, policy_version 791543 (0.0012) [2023-12-26 21:07:29,396][105620] Updated weights for policy 1, policy_version 791553 (0.0010) [2023-12-26 21:07:29,689][105692] Updated weights for policy 0, policy_version 791761 (0.0010) [2023-12-26 21:07:29,747][105692] Updated weights for policy 0, policy_version 791771 (0.0010) [2023-12-26 21:07:29,799][105692] Updated weights for policy 0, policy_version 791781 (0.0010) [2023-12-26 21:07:30,134][105620] Updated weights for policy 1, policy_version 791563 (0.0009) [2023-12-26 21:07:30,183][105620] Updated weights for policy 1, policy_version 791573 (0.0005) [2023-12-26 21:07:30,235][105620] Updated weights for policy 1, policy_version 791583 (0.0006) [2023-12-26 21:07:30,642][105692] Updated weights for policy 0, policy_version 791791 (0.0009) [2023-12-26 21:07:30,704][105692] Updated weights for policy 0, policy_version 791801 (0.0008) [2023-12-26 21:07:30,761][105692] Updated weights for policy 0, policy_version 791811 (0.0006) [2023-12-26 21:07:30,831][105620] Updated weights for policy 1, policy_version 791593 (0.0006) [2023-12-26 21:07:30,883][105620] Updated weights for policy 1, policy_version 791603 (0.0010) [2023-12-26 21:07:30,935][105620] Updated weights for policy 1, policy_version 791613 (0.0010) [2023-12-26 21:07:30,989][105620] Updated weights for policy 1, policy_version 791623 (0.0010) [2023-12-26 21:07:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 405413888. Throughput: 0: 10155.0, 1: 9381.5. Samples: 405380252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:07:31,062][104569] Avg episode reward: [(0, '8904.869'), (1, '5519.976')] [2023-12-26 21:07:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000791816_202735616.pth... [2023-12-26 21:07:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000791624_202678272.pth... [2023-12-26 21:07:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000790632_202432512.pth [2023-12-26 21:07:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000790536_202399744.pth [2023-12-26 21:07:31,431][105692] Updated weights for policy 0, policy_version 791821 (0.0005) [2023-12-26 21:07:31,495][105692] Updated weights for policy 0, policy_version 791831 (0.0005) [2023-12-26 21:07:31,549][105692] Updated weights for policy 0, policy_version 791841 (0.0005) [2023-12-26 21:07:31,784][105620] Updated weights for policy 1, policy_version 791633 (0.0008) [2023-12-26 21:07:31,845][105620] Updated weights for policy 1, policy_version 791643 (0.0008) [2023-12-26 21:07:31,907][105620] Updated weights for policy 1, policy_version 791653 (0.0008) [2023-12-26 21:07:32,237][105692] Updated weights for policy 0, policy_version 791851 (0.0006) [2023-12-26 21:07:32,292][105692] Updated weights for policy 0, policy_version 791861 (0.0008) [2023-12-26 21:07:32,356][105692] Updated weights for policy 0, policy_version 791871 (0.0008) [2023-12-26 21:07:32,688][105620] Updated weights for policy 1, policy_version 791663 (0.0010) [2023-12-26 21:07:32,751][105620] Updated weights for policy 1, policy_version 791673 (0.0011) [2023-12-26 21:07:32,809][105620] Updated weights for policy 1, policy_version 791683 (0.0010) [2023-12-26 21:07:32,819][105586] KL-divergence is very high: 116.7045 [2023-12-26 21:07:32,829][105586] KL-divergence is very high: 103.0216 [2023-12-26 21:07:33,152][105692] Updated weights for policy 0, policy_version 791881 (0.0008) [2023-12-26 21:07:33,216][105692] Updated weights for policy 0, policy_version 791891 (0.0009) [2023-12-26 21:07:33,274][105692] Updated weights for policy 0, policy_version 791901 (0.0009) [2023-12-26 21:07:33,322][105692] Updated weights for policy 0, policy_version 791911 (0.0008) [2023-12-26 21:07:33,429][105586] KL-divergence is very high: 112.0275 [2023-12-26 21:07:33,437][105586] KL-divergence is very high: 158.8680 [2023-12-26 21:07:33,445][105586] KL-divergence is very high: 106.0381 [2023-12-26 21:07:33,464][105586] KL-divergence is very high: 135.9954 [2023-12-26 21:07:33,464][105620] Updated weights for policy 1, policy_version 791693 (0.0010) [2023-12-26 21:07:33,470][105586] KL-divergence is very high: 134.1430 [2023-12-26 21:07:33,483][105586] KL-divergence is very high: 119.7193 [2023-12-26 21:07:33,488][105586] KL-divergence is very high: 162.0858 [2023-12-26 21:07:33,493][105586] KL-divergence is very high: 101.4529 [2023-12-26 21:07:33,508][105586] KL-divergence is very high: 145.8766 [2023-12-26 21:07:33,513][105586] KL-divergence is very high: 138.8769 [2023-12-26 21:07:33,521][105620] Updated weights for policy 1, policy_version 791703 (0.0010) [2023-12-26 21:07:33,524][105586] KL-divergence is very high: 117.5928 [2023-12-26 21:07:33,530][105586] KL-divergence is very high: 147.8361 [2023-12-26 21:07:33,552][105586] KL-divergence is very high: 138.0044 [2023-12-26 21:07:33,559][105586] KL-divergence is very high: 165.6774 [2023-12-26 21:07:33,572][105586] KL-divergence is very high: 175.4618 [2023-12-26 21:07:33,577][105620] Updated weights for policy 1, policy_version 791713 (0.0009) [2023-12-26 21:07:33,577][105586] KL-divergence is very high: 217.7162 [2023-12-26 21:07:33,583][105586] KL-divergence is very high: 137.4490 [2023-12-26 21:07:33,598][105586] KL-divergence is very high: 161.7414 [2023-12-26 21:07:33,602][105586] KL-divergence is very high: 161.0860 [2023-12-26 21:07:34,019][105692] Updated weights for policy 0, policy_version 791921 (0.0007) [2023-12-26 21:07:34,063][105692] Updated weights for policy 0, policy_version 791931 (0.0008) [2023-12-26 21:07:34,134][105692] Updated weights for policy 0, policy_version 791941 (0.0010) [2023-12-26 21:07:34,269][105620] Updated weights for policy 1, policy_version 791723 (0.0008) [2023-12-26 21:07:34,341][105620] Updated weights for policy 1, policy_version 791733 (0.0009) [2023-12-26 21:07:34,413][105620] Updated weights for policy 1, policy_version 791743 (0.0011) [2023-12-26 21:07:34,906][105692] Updated weights for policy 0, policy_version 791951 (0.0008) [2023-12-26 21:07:34,959][105692] Updated weights for policy 0, policy_version 791961 (0.0009) [2023-12-26 21:07:35,005][105692] Updated weights for policy 0, policy_version 791971 (0.0008) [2023-12-26 21:07:35,045][105620] Updated weights for policy 1, policy_version 791753 (0.0011) [2023-12-26 21:07:35,103][105620] Updated weights for policy 1, policy_version 791763 (0.0009) [2023-12-26 21:07:35,161][105620] Updated weights for policy 1, policy_version 791773 (0.0009) [2023-12-26 21:07:35,208][105620] Updated weights for policy 1, policy_version 791783 (0.0009) [2023-12-26 21:07:35,807][105692] Updated weights for policy 0, policy_version 791981 (0.0009) [2023-12-26 21:07:35,855][105692] Updated weights for policy 0, policy_version 791991 (0.0009) [2023-12-26 21:07:35,882][105620] Updated weights for policy 1, policy_version 791793 (0.0006) [2023-12-26 21:07:35,904][105692] Updated weights for policy 0, policy_version 792001 (0.0009) [2023-12-26 21:07:35,936][105620] Updated weights for policy 1, policy_version 791803 (0.0005) [2023-12-26 21:07:35,997][105620] Updated weights for policy 1, policy_version 791813 (0.0005) [2023-12-26 21:07:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19633.1). Total num frames: 405512192. Throughput: 0: 10137.7, 1: 9434.1. Samples: 405496288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:07:36,063][104569] Avg episode reward: [(0, '9262.653'), (1, '3142.952')] [2023-12-26 21:07:36,601][105620] Updated weights for policy 1, policy_version 791823 (0.0008) [2023-12-26 21:07:36,662][105620] Updated weights for policy 1, policy_version 791833 (0.0008) [2023-12-26 21:07:36,665][105692] Updated weights for policy 0, policy_version 792011 (0.0007) [2023-12-26 21:07:36,724][105620] Updated weights for policy 1, policy_version 791843 (0.0006) [2023-12-26 21:07:36,728][105692] Updated weights for policy 0, policy_version 792021 (0.0007) [2023-12-26 21:07:36,800][105692] Updated weights for policy 0, policy_version 792031 (0.0009) [2023-12-26 21:07:37,314][105620] Updated weights for policy 1, policy_version 791853 (0.0005) [2023-12-26 21:07:37,375][105620] Updated weights for policy 1, policy_version 791863 (0.0008) [2023-12-26 21:07:37,436][105620] Updated weights for policy 1, policy_version 791873 (0.0009) [2023-12-26 21:07:37,645][105692] Updated weights for policy 0, policy_version 792041 (0.0009) [2023-12-26 21:07:37,709][105692] Updated weights for policy 0, policy_version 792051 (0.0009) [2023-12-26 21:07:37,773][105692] Updated weights for policy 0, policy_version 792061 (0.0008) [2023-12-26 21:07:37,839][105692] Updated weights for policy 0, policy_version 792071 (0.0010) [2023-12-26 21:07:38,063][105620] Updated weights for policy 1, policy_version 791883 (0.0007) [2023-12-26 21:07:38,116][105620] Updated weights for policy 1, policy_version 791893 (0.0008) [2023-12-26 21:07:38,168][105620] Updated weights for policy 1, policy_version 791903 (0.0009) [2023-12-26 21:07:38,648][105692] Updated weights for policy 0, policy_version 792081 (0.0009) [2023-12-26 21:07:38,695][105692] Updated weights for policy 0, policy_version 792091 (0.0009) [2023-12-26 21:07:38,761][105692] Updated weights for policy 0, policy_version 792101 (0.0009) [2023-12-26 21:07:38,825][105620] Updated weights for policy 1, policy_version 791913 (0.0009) [2023-12-26 21:07:38,887][105620] Updated weights for policy 1, policy_version 791923 (0.0009) [2023-12-26 21:07:38,942][105620] Updated weights for policy 1, policy_version 791933 (0.0009) [2023-12-26 21:07:38,993][105620] Updated weights for policy 1, policy_version 791943 (0.0009) [2023-12-26 21:07:39,508][105692] Updated weights for policy 0, policy_version 792111 (0.0006) [2023-12-26 21:07:39,569][105692] Updated weights for policy 0, policy_version 792121 (0.0007) [2023-12-26 21:07:39,635][105692] Updated weights for policy 0, policy_version 792131 (0.0010) [2023-12-26 21:07:39,799][105620] Updated weights for policy 1, policy_version 791953 (0.0008) [2023-12-26 21:07:39,867][105620] Updated weights for policy 1, policy_version 791963 (0.0008) [2023-12-26 21:07:39,932][105620] Updated weights for policy 1, policy_version 791973 (0.0008) [2023-12-26 21:07:40,343][105692] Updated weights for policy 0, policy_version 792141 (0.0009) [2023-12-26 21:07:40,410][105692] Updated weights for policy 0, policy_version 792151 (0.0008) [2023-12-26 21:07:40,475][105692] Updated weights for policy 0, policy_version 792161 (0.0009) [2023-12-26 21:07:40,744][105620] Updated weights for policy 1, policy_version 791983 (0.0008) [2023-12-26 21:07:40,809][105620] Updated weights for policy 1, policy_version 791993 (0.0009) [2023-12-26 21:07:40,869][105620] Updated weights for policy 1, policy_version 792003 (0.0008) [2023-12-26 21:07:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 405602304. Throughput: 0: 9984.0, 1: 9484.5. Samples: 405610728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:07:41,062][104569] Avg episode reward: [(0, '9173.271'), (1, '6946.972')] [2023-12-26 21:07:41,134][105692] Updated weights for policy 0, policy_version 792171 (0.0010) [2023-12-26 21:07:41,186][105692] Updated weights for policy 0, policy_version 792181 (0.0008) [2023-12-26 21:07:41,246][105692] Updated weights for policy 0, policy_version 792191 (0.0009) [2023-12-26 21:07:41,730][105620] Updated weights for policy 1, policy_version 792013 (0.0009) [2023-12-26 21:07:41,790][105586] KL-divergence is very high: 119.4468 [2023-12-26 21:07:41,791][105620] Updated weights for policy 1, policy_version 792023 (0.0008) [2023-12-26 21:07:41,807][105586] KL-divergence is very high: 122.3927 [2023-12-26 21:07:41,838][105586] KL-divergence is very high: 108.5136 [2023-12-26 21:07:41,850][105620] Updated weights for policy 1, policy_version 792033 (0.0009) [2023-12-26 21:07:41,983][105692] Updated weights for policy 0, policy_version 792201 (0.0009) [2023-12-26 21:07:42,042][105692] Updated weights for policy 0, policy_version 792211 (0.0009) [2023-12-26 21:07:42,100][105692] Updated weights for policy 0, policy_version 792221 (0.0009) [2023-12-26 21:07:42,159][105692] Updated weights for policy 0, policy_version 792231 (0.0009) [2023-12-26 21:07:42,674][105620] Updated weights for policy 1, policy_version 792043 (0.0010) [2023-12-26 21:07:42,736][105620] Updated weights for policy 1, policy_version 792053 (0.0010) [2023-12-26 21:07:42,799][105620] Updated weights for policy 1, policy_version 792063 (0.0009) [2023-12-26 21:07:42,827][105692] Updated weights for policy 0, policy_version 792241 (0.0008) [2023-12-26 21:07:42,884][105692] Updated weights for policy 0, policy_version 792251 (0.0008) [2023-12-26 21:07:42,940][105692] Updated weights for policy 0, policy_version 792261 (0.0009) [2023-12-26 21:07:43,480][105620] Updated weights for policy 1, policy_version 792073 (0.0007) [2023-12-26 21:07:43,529][105620] Updated weights for policy 1, policy_version 792083 (0.0005) [2023-12-26 21:07:43,577][105620] Updated weights for policy 1, policy_version 792093 (0.0005) [2023-12-26 21:07:43,630][105620] Updated weights for policy 1, policy_version 792103 (0.0005) [2023-12-26 21:07:43,773][105692] Updated weights for policy 0, policy_version 792271 (0.0010) [2023-12-26 21:07:43,828][105692] Updated weights for policy 0, policy_version 792281 (0.0010) [2023-12-26 21:07:43,879][105692] Updated weights for policy 0, policy_version 792293 (0.0007) [2023-12-26 21:07:44,166][105620] Updated weights for policy 1, policy_version 792113 (0.0007) [2023-12-26 21:07:44,216][105620] Updated weights for policy 1, policy_version 792123 (0.0008) [2023-12-26 21:07:44,265][105620] Updated weights for policy 1, policy_version 792133 (0.0009) [2023-12-26 21:07:44,579][105692] Updated weights for policy 0, policy_version 792303 (0.0005) [2023-12-26 21:07:44,636][105692] Updated weights for policy 0, policy_version 792313 (0.0005) [2023-12-26 21:07:44,703][105692] Updated weights for policy 0, policy_version 792323 (0.0006) [2023-12-26 21:07:45,020][105620] Updated weights for policy 1, policy_version 792143 (0.0007) [2023-12-26 21:07:45,091][105620] Updated weights for policy 1, policy_version 792153 (0.0008) [2023-12-26 21:07:45,147][105620] Updated weights for policy 1, policy_version 792163 (0.0008) [2023-12-26 21:07:45,350][105692] Updated weights for policy 0, policy_version 792333 (0.0007) [2023-12-26 21:07:45,413][105692] Updated weights for policy 0, policy_version 792343 (0.0009) [2023-12-26 21:07:45,474][105692] Updated weights for policy 0, policy_version 792353 (0.0008) [2023-12-26 21:07:45,897][105620] Updated weights for policy 1, policy_version 792173 (0.0009) [2023-12-26 21:07:45,958][105620] Updated weights for policy 1, policy_version 792183 (0.0010) [2023-12-26 21:07:46,021][105620] Updated weights for policy 1, policy_version 792193 (0.0010) [2023-12-26 21:07:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 405700608. Throughput: 0: 9890.5, 1: 9445.1. Samples: 405668444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:07:46,063][104569] Avg episode reward: [(0, '9081.981'), (1, '7028.566')] [2023-12-26 21:07:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000792360_202874880.pth... [2023-12-26 21:07:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000792200_202825728.pth... [2023-12-26 21:07:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000791240_202588160.pth [2023-12-26 21:07:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000791048_202530816.pth [2023-12-26 21:07:46,147][105692] Updated weights for policy 0, policy_version 792363 (0.0007) [2023-12-26 21:07:46,217][105692] Updated weights for policy 0, policy_version 792373 (0.0006) [2023-12-26 21:07:46,278][105692] Updated weights for policy 0, policy_version 792383 (0.0011) [2023-12-26 21:07:46,722][105620] Updated weights for policy 1, policy_version 792203 (0.0010) [2023-12-26 21:07:46,778][105620] Updated weights for policy 1, policy_version 792213 (0.0010) [2023-12-26 21:07:46,831][105620] Updated weights for policy 1, policy_version 792223 (0.0010) [2023-12-26 21:07:46,981][105692] Updated weights for policy 0, policy_version 792393 (0.0011) [2023-12-26 21:07:47,030][105692] Updated weights for policy 0, policy_version 792403 (0.0010) [2023-12-26 21:07:47,076][105692] Updated weights for policy 0, policy_version 792413 (0.0010) [2023-12-26 21:07:47,134][105692] Updated weights for policy 0, policy_version 792423 (0.0010) [2023-12-26 21:07:47,548][105620] Updated weights for policy 1, policy_version 792233 (0.0010) [2023-12-26 21:07:47,606][105620] Updated weights for policy 1, policy_version 792243 (0.0008) [2023-12-26 21:07:47,667][105620] Updated weights for policy 1, policy_version 792253 (0.0007) [2023-12-26 21:07:47,729][105620] Updated weights for policy 1, policy_version 792263 (0.0008) [2023-12-26 21:07:47,847][105692] Updated weights for policy 0, policy_version 792433 (0.0010) [2023-12-26 21:07:47,898][105692] Updated weights for policy 0, policy_version 792443 (0.0010) [2023-12-26 21:07:47,947][105692] Updated weights for policy 0, policy_version 792453 (0.0010) [2023-12-26 21:07:48,360][105620] Updated weights for policy 1, policy_version 792273 (0.0007) [2023-12-26 21:07:48,418][105620] Updated weights for policy 1, policy_version 792283 (0.0008) [2023-12-26 21:07:48,476][105620] Updated weights for policy 1, policy_version 792293 (0.0008) [2023-12-26 21:07:48,594][105692] Updated weights for policy 0, policy_version 792463 (0.0006) [2023-12-26 21:07:48,654][105692] Updated weights for policy 0, policy_version 792473 (0.0006) [2023-12-26 21:07:48,706][105692] Updated weights for policy 0, policy_version 792483 (0.0006) [2023-12-26 21:07:49,217][105620] Updated weights for policy 1, policy_version 792303 (0.0007) [2023-12-26 21:07:49,286][105620] Updated weights for policy 1, policy_version 792313 (0.0008) [2023-12-26 21:07:49,356][105620] Updated weights for policy 1, policy_version 792323 (0.0008) [2023-12-26 21:07:49,412][105692] Updated weights for policy 0, policy_version 792493 (0.0011) [2023-12-26 21:07:49,465][105692] Updated weights for policy 0, policy_version 792503 (0.0010) [2023-12-26 21:07:49,524][105692] Updated weights for policy 0, policy_version 792513 (0.0011) [2023-12-26 21:07:50,125][105620] Updated weights for policy 1, policy_version 792333 (0.0008) [2023-12-26 21:07:50,191][105620] Updated weights for policy 1, policy_version 792343 (0.0010) [2023-12-26 21:07:50,253][105620] Updated weights for policy 1, policy_version 792353 (0.0009) [2023-12-26 21:07:50,264][105692] Updated weights for policy 0, policy_version 792523 (0.0010) [2023-12-26 21:07:50,317][105692] Updated weights for policy 0, policy_version 792533 (0.0011) [2023-12-26 21:07:50,381][105692] Updated weights for policy 0, policy_version 792543 (0.0011) [2023-12-26 21:07:50,955][105620] Updated weights for policy 1, policy_version 792363 (0.0007) [2023-12-26 21:07:51,021][105620] Updated weights for policy 1, policy_version 792373 (0.0008) [2023-12-26 21:07:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 405790720. Throughput: 0: 9882.8, 1: 9514.3. Samples: 405787044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:07:51,062][104569] Avg episode reward: [(0, '8994.438'), (1, '7406.393')] [2023-12-26 21:07:51,084][105620] Updated weights for policy 1, policy_version 792383 (0.0008) [2023-12-26 21:07:51,144][105692] Updated weights for policy 0, policy_version 792553 (0.0010) [2023-12-26 21:07:51,210][105692] Updated weights for policy 0, policy_version 792563 (0.0008) [2023-12-26 21:07:51,271][105692] Updated weights for policy 0, policy_version 792573 (0.0008) [2023-12-26 21:07:51,330][105692] Updated weights for policy 0, policy_version 792583 (0.0008) [2023-12-26 21:07:51,841][105620] Updated weights for policy 1, policy_version 792393 (0.0008) [2023-12-26 21:07:51,899][105620] Updated weights for policy 1, policy_version 792403 (0.0005) [2023-12-26 21:07:51,956][105620] Updated weights for policy 1, policy_version 792413 (0.0007) [2023-12-26 21:07:52,011][105620] Updated weights for policy 1, policy_version 792423 (0.0008) [2023-12-26 21:07:52,041][105692] Updated weights for policy 0, policy_version 792593 (0.0007) [2023-12-26 21:07:52,095][105692] Updated weights for policy 0, policy_version 792603 (0.0006) [2023-12-26 21:07:52,145][105692] Updated weights for policy 0, policy_version 792613 (0.0005) [2023-12-26 21:07:52,724][105692] Updated weights for policy 0, policy_version 792623 (0.0006) [2023-12-26 21:07:52,788][105692] Updated weights for policy 0, policy_version 792633 (0.0007) [2023-12-26 21:07:52,794][105620] Updated weights for policy 1, policy_version 792433 (0.0010) [2023-12-26 21:07:52,851][105620] Updated weights for policy 1, policy_version 792443 (0.0009) [2023-12-26 21:07:52,852][105692] Updated weights for policy 0, policy_version 792643 (0.0006) [2023-12-26 21:07:52,905][105620] Updated weights for policy 1, policy_version 792453 (0.0008) [2023-12-26 21:07:53,503][105692] Updated weights for policy 0, policy_version 792653 (0.0009) [2023-12-26 21:07:53,565][105692] Updated weights for policy 0, policy_version 792663 (0.0009) [2023-12-26 21:07:53,631][105692] Updated weights for policy 0, policy_version 792673 (0.0009) [2023-12-26 21:07:53,692][105620] Updated weights for policy 1, policy_version 792463 (0.0009) [2023-12-26 21:07:53,753][105620] Updated weights for policy 1, policy_version 792473 (0.0009) [2023-12-26 21:07:53,804][105620] Updated weights for policy 1, policy_version 792483 (0.0009) [2023-12-26 21:07:54,393][105692] Updated weights for policy 0, policy_version 792683 (0.0009) [2023-12-26 21:07:54,458][105692] Updated weights for policy 0, policy_version 792693 (0.0009) [2023-12-26 21:07:54,523][105692] Updated weights for policy 0, policy_version 792703 (0.0009) [2023-12-26 21:07:54,562][105620] Updated weights for policy 1, policy_version 792493 (0.0009) [2023-12-26 21:07:54,626][105620] Updated weights for policy 1, policy_version 792503 (0.0007) [2023-12-26 21:07:54,689][105620] Updated weights for policy 1, policy_version 792513 (0.0009) [2023-12-26 21:07:55,272][105692] Updated weights for policy 0, policy_version 792713 (0.0007) [2023-12-26 21:07:55,330][105692] Updated weights for policy 0, policy_version 792723 (0.0009) [2023-12-26 21:07:55,393][105692] Updated weights for policy 0, policy_version 792733 (0.0008) [2023-12-26 21:07:55,402][105620] Updated weights for policy 1, policy_version 792523 (0.0009) [2023-12-26 21:07:55,442][105692] Updated weights for policy 0, policy_version 792743 (0.0008) [2023-12-26 21:07:55,458][105620] Updated weights for policy 1, policy_version 792533 (0.0010) [2023-12-26 21:07:55,512][105620] Updated weights for policy 1, policy_version 792543 (0.0009) [2023-12-26 21:07:56,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.3, 300 sec: 19577.5). Total num frames: 405889024. Throughput: 0: 9872.9, 1: 9481.3. Samples: 405900844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:07:56,062][104569] Avg episode reward: [(0, '9170.746'), (1, '9167.209')] [2023-12-26 21:07:56,196][105692] Updated weights for policy 0, policy_version 792753 (0.0009) [2023-12-26 21:07:56,253][105692] Updated weights for policy 0, policy_version 792763 (0.0009) [2023-12-26 21:07:56,272][105620] Updated weights for policy 1, policy_version 792553 (0.0009) [2023-12-26 21:07:56,302][105692] Updated weights for policy 0, policy_version 792773 (0.0006) [2023-12-26 21:07:56,328][105620] Updated weights for policy 1, policy_version 792563 (0.0009) [2023-12-26 21:07:56,386][105620] Updated weights for policy 1, policy_version 792573 (0.0010) [2023-12-26 21:07:56,444][105620] Updated weights for policy 1, policy_version 792583 (0.0010) [2023-12-26 21:07:56,856][105692] Updated weights for policy 0, policy_version 792783 (0.0006) [2023-12-26 21:07:56,913][105692] Updated weights for policy 0, policy_version 792793 (0.0005) [2023-12-26 21:07:56,966][105692] Updated weights for policy 0, policy_version 792803 (0.0005) [2023-12-26 21:07:57,337][105620] Updated weights for policy 1, policy_version 792593 (0.0009) [2023-12-26 21:07:57,384][105620] Updated weights for policy 1, policy_version 792603 (0.0009) [2023-12-26 21:07:57,441][105620] Updated weights for policy 1, policy_version 792613 (0.0009) [2023-12-26 21:07:57,566][105692] Updated weights for policy 0, policy_version 792813 (0.0007) [2023-12-26 21:07:57,614][105692] Updated weights for policy 0, policy_version 792823 (0.0008) [2023-12-26 21:07:57,664][105692] Updated weights for policy 0, policy_version 792833 (0.0009) [2023-12-26 21:07:58,192][105620] Updated weights for policy 1, policy_version 792623 (0.0009) [2023-12-26 21:07:58,242][105620] Updated weights for policy 1, policy_version 792633 (0.0008) [2023-12-26 21:07:58,300][105620] Updated weights for policy 1, policy_version 792643 (0.0009) [2023-12-26 21:07:58,455][105692] Updated weights for policy 0, policy_version 792843 (0.0008) [2023-12-26 21:07:58,523][105692] Updated weights for policy 0, policy_version 792853 (0.0006) [2023-12-26 21:07:58,593][105692] Updated weights for policy 0, policy_version 792863 (0.0008) [2023-12-26 21:07:59,097][105620] Updated weights for policy 1, policy_version 792653 (0.0008) [2023-12-26 21:07:59,165][105620] Updated weights for policy 1, policy_version 792663 (0.0008) [2023-12-26 21:07:59,233][105620] Updated weights for policy 1, policy_version 792673 (0.0007) [2023-12-26 21:07:59,287][105692] Updated weights for policy 0, policy_version 792873 (0.0012) [2023-12-26 21:07:59,350][105692] Updated weights for policy 0, policy_version 792883 (0.0008) [2023-12-26 21:07:59,419][105692] Updated weights for policy 0, policy_version 792893 (0.0007) [2023-12-26 21:07:59,482][105692] Updated weights for policy 0, policy_version 792903 (0.0010) [2023-12-26 21:07:59,992][105620] Updated weights for policy 1, policy_version 792683 (0.0008) [2023-12-26 21:08:00,057][105620] Updated weights for policy 1, policy_version 792693 (0.0006) [2023-12-26 21:08:00,119][105620] Updated weights for policy 1, policy_version 792703 (0.0006) [2023-12-26 21:08:00,162][105692] Updated weights for policy 0, policy_version 792913 (0.0011) [2023-12-26 21:08:00,221][105692] Updated weights for policy 0, policy_version 792923 (0.0011) [2023-12-26 21:08:00,276][105692] Updated weights for policy 0, policy_version 792933 (0.0010) [2023-12-26 21:08:00,829][105692] Updated weights for policy 0, policy_version 792943 (0.0007) [2023-12-26 21:08:00,899][105692] Updated weights for policy 0, policy_version 792953 (0.0005) [2023-12-26 21:08:00,917][105620] Updated weights for policy 1, policy_version 792713 (0.0006) [2023-12-26 21:08:00,960][105692] Updated weights for policy 0, policy_version 792963 (0.0005) [2023-12-26 21:08:00,975][105620] Updated weights for policy 1, policy_version 792723 (0.0007) [2023-12-26 21:08:01,038][105620] Updated weights for policy 1, policy_version 792733 (0.0008) [2023-12-26 21:08:01,052][105586] KL-divergence is very high: 117.2349 [2023-12-26 21:08:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 405987328. Throughput: 0: 9858.7, 1: 9493.6. Samples: 405958324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:08:01,062][104569] Avg episode reward: [(0, '9192.407'), (1, '8978.991')] [2023-12-26 21:08:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000792968_203030528.pth... [2023-12-26 21:08:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000791816_202735616.pth [2023-12-26 21:08:01,100][105620] Updated weights for policy 1, policy_version 792743 (0.0009) [2023-12-26 21:08:01,103][105586] KL-divergence is very high: 214.5820 [2023-12-26 21:08:01,108][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000792744_202964992.pth... [2023-12-26 21:08:01,113][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000791624_202678272.pth [2023-12-26 21:08:01,514][105692] Updated weights for policy 0, policy_version 792973 (0.0008) [2023-12-26 21:08:01,573][105692] Updated weights for policy 0, policy_version 792983 (0.0010) [2023-12-26 21:08:01,635][105692] Updated weights for policy 0, policy_version 792993 (0.0009) [2023-12-26 21:08:01,883][105586] KL-divergence is very high: 180.3408 [2023-12-26 21:08:01,888][105586] KL-divergence is very high: 180.0991 [2023-12-26 21:08:01,897][105586] KL-divergence is very high: 174.1123 [2023-12-26 21:08:01,925][105586] KL-divergence is very high: 122.4181 [2023-12-26 21:08:01,927][105620] Updated weights for policy 1, policy_version 792753 (0.0008) [2023-12-26 21:08:01,930][105586] KL-divergence is very high: 210.3673 [2023-12-26 21:08:01,944][105586] KL-divergence is very high: 380.1070 [2023-12-26 21:08:01,948][105586] KL-divergence is very high: 305.8680 [2023-12-26 21:08:01,962][105586] KL-divergence is very high: 356.3082 [2023-12-26 21:08:01,967][105586] KL-divergence is very high: 529.4674 [2023-12-26 21:08:01,971][105620] Updated weights for policy 1, policy_version 792763 (0.0008) [2023-12-26 21:08:01,977][105586] KL-divergence is very high: 105.8881 [2023-12-26 21:08:01,982][105586] KL-divergence is very high: 526.5900 [2023-12-26 21:08:01,987][105586] KL-divergence is very high: 378.5712 [2023-12-26 21:08:02,001][105586] KL-divergence is very high: 320.9770 [2023-12-26 21:08:02,006][105586] KL-divergence is very high: 448.5063 [2023-12-26 21:08:02,020][105620] Updated weights for policy 1, policy_version 792773 (0.0008) [2023-12-26 21:08:02,020][105586] KL-divergence is very high: 402.6500 [2023-12-26 21:08:02,025][105586] KL-divergence is very high: 260.7194 [2023-12-26 21:08:02,342][105692] Updated weights for policy 0, policy_version 793003 (0.0010) [2023-12-26 21:08:02,404][105692] Updated weights for policy 0, policy_version 793013 (0.0011) [2023-12-26 21:08:02,467][105692] Updated weights for policy 0, policy_version 793023 (0.0010) [2023-12-26 21:08:02,811][105586] KL-divergence is very high: 277.1158 [2023-12-26 21:08:02,851][105620] Updated weights for policy 1, policy_version 792783 (0.0008) [2023-12-26 21:08:02,859][105586] KL-divergence is very high: 227.7261 [2023-12-26 21:08:02,905][105586] KL-divergence is very high: 268.7368 [2023-12-26 21:08:02,910][105620] Updated weights for policy 1, policy_version 792793 (0.0008) [2023-12-26 21:08:02,954][105586] KL-divergence is very high: 442.2152 [2023-12-26 21:08:02,973][105620] Updated weights for policy 1, policy_version 792803 (0.0008) [2023-12-26 21:08:03,147][105692] Updated weights for policy 0, policy_version 793033 (0.0010) [2023-12-26 21:08:03,195][105692] Updated weights for policy 0, policy_version 793043 (0.0005) [2023-12-26 21:08:03,247][105692] Updated weights for policy 0, policy_version 793053 (0.0005) [2023-12-26 21:08:03,295][105692] Updated weights for policy 0, policy_version 793063 (0.0005) [2023-12-26 21:08:03,825][105692] Updated weights for policy 0, policy_version 793073 (0.0007) [2023-12-26 21:08:03,858][105620] Updated weights for policy 1, policy_version 792813 (0.0009) [2023-12-26 21:08:03,888][105692] Updated weights for policy 0, policy_version 793083 (0.0010) [2023-12-26 21:08:03,919][105620] Updated weights for policy 1, policy_version 792823 (0.0006) [2023-12-26 21:08:03,948][105692] Updated weights for policy 0, policy_version 793093 (0.0011) [2023-12-26 21:08:03,972][105620] Updated weights for policy 1, policy_version 792833 (0.0009) [2023-12-26 21:08:04,661][105692] Updated weights for policy 0, policy_version 793103 (0.0010) [2023-12-26 21:08:04,721][105692] Updated weights for policy 0, policy_version 793113 (0.0010) [2023-12-26 21:08:04,730][105620] Updated weights for policy 1, policy_version 792843 (0.0007) [2023-12-26 21:08:04,779][105692] Updated weights for policy 0, policy_version 793123 (0.0010) [2023-12-26 21:08:04,789][105620] Updated weights for policy 1, policy_version 792853 (0.0005) [2023-12-26 21:08:04,846][105620] Updated weights for policy 1, policy_version 792863 (0.0007) [2023-12-26 21:08:05,409][105692] Updated weights for policy 0, policy_version 793133 (0.0008) [2023-12-26 21:08:05,459][105692] Updated weights for policy 0, policy_version 793143 (0.0007) [2023-12-26 21:08:05,507][105692] Updated weights for policy 0, policy_version 793153 (0.0010) [2023-12-26 21:08:05,634][105620] Updated weights for policy 1, policy_version 792873 (0.0008) [2023-12-26 21:08:05,692][105620] Updated weights for policy 1, policy_version 792883 (0.0008) [2023-12-26 21:08:05,754][105620] Updated weights for policy 1, policy_version 792893 (0.0006) [2023-12-26 21:08:05,813][105620] Updated weights for policy 1, policy_version 792903 (0.0008) [2023-12-26 21:08:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 406085632. Throughput: 0: 9858.6, 1: 9495.1. Samples: 406073308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:08:06,062][104569] Avg episode reward: [(0, '9100.907'), (1, '8702.114')] [2023-12-26 21:08:06,268][105692] Updated weights for policy 0, policy_version 793163 (0.0010) [2023-12-26 21:08:06,332][105692] Updated weights for policy 0, policy_version 793173 (0.0011) [2023-12-26 21:08:06,398][105692] Updated weights for policy 0, policy_version 793183 (0.0010) [2023-12-26 21:08:06,563][105620] Updated weights for policy 1, policy_version 792913 (0.0010) [2023-12-26 21:08:06,615][105620] Updated weights for policy 1, policy_version 792923 (0.0011) [2023-12-26 21:08:06,675][105620] Updated weights for policy 1, policy_version 792933 (0.0011) [2023-12-26 21:08:07,037][105692] Updated weights for policy 0, policy_version 793193 (0.0010) [2023-12-26 21:08:07,083][105692] Updated weights for policy 0, policy_version 793203 (0.0005) [2023-12-26 21:08:07,137][105692] Updated weights for policy 0, policy_version 793213 (0.0005) [2023-12-26 21:08:07,187][105692] Updated weights for policy 0, policy_version 793223 (0.0005) [2023-12-26 21:08:07,438][105620] Updated weights for policy 1, policy_version 792943 (0.0010) [2023-12-26 21:08:07,489][105620] Updated weights for policy 1, policy_version 792953 (0.0010) [2023-12-26 21:08:07,537][105620] Updated weights for policy 1, policy_version 792963 (0.0010) [2023-12-26 21:08:07,819][105692] Updated weights for policy 0, policy_version 793233 (0.0006) [2023-12-26 21:08:07,903][105692] Updated weights for policy 0, policy_version 793243 (0.0006) [2023-12-26 21:08:07,969][105692] Updated weights for policy 0, policy_version 793253 (0.0008) [2023-12-26 21:08:08,197][105620] Updated weights for policy 1, policy_version 792973 (0.0007) [2023-12-26 21:08:08,256][105620] Updated weights for policy 1, policy_version 792983 (0.0006) [2023-12-26 21:08:08,321][105620] Updated weights for policy 1, policy_version 792993 (0.0006) [2023-12-26 21:08:08,531][105692] Updated weights for policy 0, policy_version 793263 (0.0009) [2023-12-26 21:08:08,585][105692] Updated weights for policy 0, policy_version 793273 (0.0010) [2023-12-26 21:08:08,636][105692] Updated weights for policy 0, policy_version 793283 (0.0009) [2023-12-26 21:08:08,998][105620] Updated weights for policy 1, policy_version 793003 (0.0009) [2023-12-26 21:08:09,056][105620] Updated weights for policy 1, policy_version 793013 (0.0010) [2023-12-26 21:08:09,101][105620] Updated weights for policy 1, policy_version 793023 (0.0010) [2023-12-26 21:08:09,374][105692] Updated weights for policy 0, policy_version 793293 (0.0009) [2023-12-26 21:08:09,441][105692] Updated weights for policy 0, policy_version 793303 (0.0008) [2023-12-26 21:08:09,505][105692] Updated weights for policy 0, policy_version 793313 (0.0008) [2023-12-26 21:08:09,822][105620] Updated weights for policy 1, policy_version 793033 (0.0009) [2023-12-26 21:08:09,889][105620] Updated weights for policy 1, policy_version 793043 (0.0007) [2023-12-26 21:08:09,955][105620] Updated weights for policy 1, policy_version 793053 (0.0009) [2023-12-26 21:08:10,014][105620] Updated weights for policy 1, policy_version 793063 (0.0009) [2023-12-26 21:08:10,156][105692] Updated weights for policy 0, policy_version 793323 (0.0008) [2023-12-26 21:08:10,208][105692] Updated weights for policy 0, policy_version 793333 (0.0008) [2023-12-26 21:08:10,257][105692] Updated weights for policy 0, policy_version 793343 (0.0009) [2023-12-26 21:08:10,725][105620] Updated weights for policy 1, policy_version 793073 (0.0006) [2023-12-26 21:08:10,788][105620] Updated weights for policy 1, policy_version 793083 (0.0007) [2023-12-26 21:08:10,854][105620] Updated weights for policy 1, policy_version 793093 (0.0010) [2023-12-26 21:08:10,953][105692] Updated weights for policy 0, policy_version 793353 (0.0009) [2023-12-26 21:08:11,005][105692] Updated weights for policy 0, policy_version 793363 (0.0008) [2023-12-26 21:08:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 406183936. Throughput: 0: 9962.2, 1: 9490.2. Samples: 406193872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:08:11,063][104569] Avg episode reward: [(0, '9080.276'), (1, '8750.646')] [2023-12-26 21:08:11,071][105692] Updated weights for policy 0, policy_version 793373 (0.0010) [2023-12-26 21:08:11,131][105692] Updated weights for policy 0, policy_version 793383 (0.0008) [2023-12-26 21:08:11,595][105620] Updated weights for policy 1, policy_version 793103 (0.0010) [2023-12-26 21:08:11,658][105620] Updated weights for policy 1, policy_version 793113 (0.0009) [2023-12-26 21:08:11,725][105620] Updated weights for policy 1, policy_version 793123 (0.0007) [2023-12-26 21:08:11,919][105692] Updated weights for policy 0, policy_version 793393 (0.0010) [2023-12-26 21:08:11,978][105692] Updated weights for policy 0, policy_version 793403 (0.0011) [2023-12-26 21:08:12,038][105692] Updated weights for policy 0, policy_version 793413 (0.0011) [2023-12-26 21:08:12,450][105620] Updated weights for policy 1, policy_version 793133 (0.0008) [2023-12-26 21:08:12,512][105620] Updated weights for policy 1, policy_version 793143 (0.0010) [2023-12-26 21:08:12,567][105620] Updated weights for policy 1, policy_version 793153 (0.0010) [2023-12-26 21:08:12,801][105692] Updated weights for policy 0, policy_version 793423 (0.0009) [2023-12-26 21:08:12,861][105692] Updated weights for policy 0, policy_version 793433 (0.0008) [2023-12-26 21:08:12,926][105692] Updated weights for policy 0, policy_version 793443 (0.0010) [2023-12-26 21:08:13,249][105620] Updated weights for policy 1, policy_version 793163 (0.0009) [2023-12-26 21:08:13,316][105620] Updated weights for policy 1, policy_version 793173 (0.0009) [2023-12-26 21:08:13,386][105620] Updated weights for policy 1, policy_version 793183 (0.0010) [2023-12-26 21:08:13,548][105692] Updated weights for policy 0, policy_version 793453 (0.0010) [2023-12-26 21:08:13,600][105692] Updated weights for policy 0, policy_version 793463 (0.0008) [2023-12-26 21:08:13,647][105692] Updated weights for policy 0, policy_version 793473 (0.0009) [2023-12-26 21:08:14,098][105620] Updated weights for policy 1, policy_version 793193 (0.0009) [2023-12-26 21:08:14,154][105620] Updated weights for policy 1, policy_version 793203 (0.0010) [2023-12-26 21:08:14,215][105620] Updated weights for policy 1, policy_version 793215 (0.0009) [2023-12-26 21:08:14,245][105692] Updated weights for policy 0, policy_version 793483 (0.0008) [2023-12-26 21:08:14,303][105692] Updated weights for policy 0, policy_version 793493 (0.0009) [2023-12-26 21:08:14,359][105692] Updated weights for policy 0, policy_version 793503 (0.0010) [2023-12-26 21:08:14,898][105620] Updated weights for policy 1, policy_version 793225 (0.0006) [2023-12-26 21:08:14,960][105620] Updated weights for policy 1, policy_version 793235 (0.0009) [2023-12-26 21:08:15,023][105620] Updated weights for policy 1, policy_version 793245 (0.0007) [2023-12-26 21:08:15,083][105620] Updated weights for policy 1, policy_version 793255 (0.0009) [2023-12-26 21:08:15,170][105692] Updated weights for policy 0, policy_version 793513 (0.0009) [2023-12-26 21:08:15,233][105692] Updated weights for policy 0, policy_version 793523 (0.0009) [2023-12-26 21:08:15,296][105692] Updated weights for policy 0, policy_version 793533 (0.0009) [2023-12-26 21:08:15,360][105692] Updated weights for policy 0, policy_version 793543 (0.0006) [2023-12-26 21:08:15,762][105620] Updated weights for policy 1, policy_version 793265 (0.0005) [2023-12-26 21:08:15,822][105620] Updated weights for policy 1, policy_version 793275 (0.0007) [2023-12-26 21:08:15,890][105620] Updated weights for policy 1, policy_version 793285 (0.0009) [2023-12-26 21:08:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 406282240. Throughput: 0: 9867.0, 1: 9476.7. Samples: 406250720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:08:16,062][104569] Avg episode reward: [(0, '9261.991'), (1, '9262.053')] [2023-12-26 21:08:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000793288_203104256.pth... [2023-12-26 21:08:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000792200_202825728.pth [2023-12-26 21:08:16,083][105692] Updated weights for policy 0, policy_version 793553 (0.0008) [2023-12-26 21:08:16,144][105692] Updated weights for policy 0, policy_version 793563 (0.0008) [2023-12-26 21:08:16,202][105692] Updated weights for policy 0, policy_version 793573 (0.0009) [2023-12-26 21:08:16,215][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000793576_203186176.pth... [2023-12-26 21:08:16,219][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000792360_202874880.pth [2023-12-26 21:08:16,698][105620] Updated weights for policy 1, policy_version 793295 (0.0009) [2023-12-26 21:08:16,747][105620] Updated weights for policy 1, policy_version 793305 (0.0008) [2023-12-26 21:08:16,784][105692] Updated weights for policy 0, policy_version 793583 (0.0009) [2023-12-26 21:08:16,808][105620] Updated weights for policy 1, policy_version 793315 (0.0008) [2023-12-26 21:08:16,834][105692] Updated weights for policy 0, policy_version 793593 (0.0007) [2023-12-26 21:08:16,882][105692] Updated weights for policy 0, policy_version 793603 (0.0009) [2023-12-26 21:08:17,550][105620] Updated weights for policy 1, policy_version 793325 (0.0009) [2023-12-26 21:08:17,610][105620] Updated weights for policy 1, policy_version 793335 (0.0009) [2023-12-26 21:08:17,610][105692] Updated weights for policy 0, policy_version 793613 (0.0007) [2023-12-26 21:08:17,670][105620] Updated weights for policy 1, policy_version 793345 (0.0009) [2023-12-26 21:08:17,671][105692] Updated weights for policy 0, policy_version 793623 (0.0005) [2023-12-26 21:08:17,720][105692] Updated weights for policy 0, policy_version 793633 (0.0007) [2023-12-26 21:08:18,397][105692] Updated weights for policy 0, policy_version 793643 (0.0006) [2023-12-26 21:08:18,454][105692] Updated weights for policy 0, policy_version 793653 (0.0007) [2023-12-26 21:08:18,476][105620] Updated weights for policy 1, policy_version 793355 (0.0007) [2023-12-26 21:08:18,510][105692] Updated weights for policy 0, policy_version 793663 (0.0009) [2023-12-26 21:08:18,530][105620] Updated weights for policy 1, policy_version 793365 (0.0007) [2023-12-26 21:08:18,588][105620] Updated weights for policy 1, policy_version 793375 (0.0008) [2023-12-26 21:08:19,197][105692] Updated weights for policy 0, policy_version 793673 (0.0006) [2023-12-26 21:08:19,268][105692] Updated weights for policy 0, policy_version 793683 (0.0008) [2023-12-26 21:08:19,330][105692] Updated weights for policy 0, policy_version 793693 (0.0008) [2023-12-26 21:08:19,374][105620] Updated weights for policy 1, policy_version 793385 (0.0007) [2023-12-26 21:08:19,396][105692] Updated weights for policy 0, policy_version 793703 (0.0008) [2023-12-26 21:08:19,430][105620] Updated weights for policy 1, policy_version 793395 (0.0008) [2023-12-26 21:08:19,492][105620] Updated weights for policy 1, policy_version 793405 (0.0009) [2023-12-26 21:08:19,556][105620] Updated weights for policy 1, policy_version 793415 (0.0009) [2023-12-26 21:08:20,120][105692] Updated weights for policy 0, policy_version 793713 (0.0009) [2023-12-26 21:08:20,183][105692] Updated weights for policy 0, policy_version 793723 (0.0009) [2023-12-26 21:08:20,276][105692] Updated weights for policy 0, policy_version 793733 (0.0010) [2023-12-26 21:08:20,341][105620] Updated weights for policy 1, policy_version 793425 (0.0008) [2023-12-26 21:08:20,408][105620] Updated weights for policy 1, policy_version 793435 (0.0010) [2023-12-26 21:08:20,470][105620] Updated weights for policy 1, policy_version 793445 (0.0009) [2023-12-26 21:08:21,023][105692] Updated weights for policy 0, policy_version 793743 (0.0009) [2023-12-26 21:08:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 406372352. Throughput: 0: 9947.7, 1: 9399.3. Samples: 406366900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:08:21,062][104569] Avg episode reward: [(0, '9350.768'), (1, '9261.579')] [2023-12-26 21:08:21,091][105692] Updated weights for policy 0, policy_version 793753 (0.0009) [2023-12-26 21:08:21,161][105692] Updated weights for policy 0, policy_version 793763 (0.0008) [2023-12-26 21:08:21,210][105620] Updated weights for policy 1, policy_version 793455 (0.0008) [2023-12-26 21:08:21,267][105620] Updated weights for policy 1, policy_version 793465 (0.0008) [2023-12-26 21:08:21,325][105620] Updated weights for policy 1, policy_version 793475 (0.0009) [2023-12-26 21:08:21,936][105692] Updated weights for policy 0, policy_version 793773 (0.0007) [2023-12-26 21:08:21,999][105692] Updated weights for policy 0, policy_version 793783 (0.0009) [2023-12-26 21:08:22,059][105692] Updated weights for policy 0, policy_version 793793 (0.0009) [2023-12-26 21:08:22,118][105620] Updated weights for policy 1, policy_version 793485 (0.0008) [2023-12-26 21:08:22,180][105620] Updated weights for policy 1, policy_version 793495 (0.0008) [2023-12-26 21:08:22,241][105620] Updated weights for policy 1, policy_version 793505 (0.0009) [2023-12-26 21:08:22,882][105692] Updated weights for policy 0, policy_version 793803 (0.0009) [2023-12-26 21:08:22,910][105620] Updated weights for policy 1, policy_version 793515 (0.0009) [2023-12-26 21:08:22,941][105692] Updated weights for policy 0, policy_version 793813 (0.0007) [2023-12-26 21:08:22,963][105620] Updated weights for policy 1, policy_version 793525 (0.0007) [2023-12-26 21:08:23,007][105692] Updated weights for policy 0, policy_version 793823 (0.0007) [2023-12-26 21:08:23,034][105620] Updated weights for policy 1, policy_version 793535 (0.0007) [2023-12-26 21:08:23,736][105620] Updated weights for policy 1, policy_version 793545 (0.0008) [2023-12-26 21:08:23,792][105620] Updated weights for policy 1, policy_version 793555 (0.0005) [2023-12-26 21:08:23,800][105692] Updated weights for policy 0, policy_version 793833 (0.0007) [2023-12-26 21:08:23,851][105620] Updated weights for policy 1, policy_version 793565 (0.0005) [2023-12-26 21:08:23,860][105692] Updated weights for policy 0, policy_version 793843 (0.0009) [2023-12-26 21:08:23,906][105620] Updated weights for policy 1, policy_version 793575 (0.0007) [2023-12-26 21:08:23,909][105692] Updated weights for policy 0, policy_version 793853 (0.0006) [2023-12-26 21:08:23,970][105692] Updated weights for policy 0, policy_version 793863 (0.0009) [2023-12-26 21:08:24,593][105620] Updated weights for policy 1, policy_version 793585 (0.0005) [2023-12-26 21:08:24,649][105620] Updated weights for policy 1, policy_version 793595 (0.0005) [2023-12-26 21:08:24,701][105620] Updated weights for policy 1, policy_version 793605 (0.0005) [2023-12-26 21:08:24,765][105692] Updated weights for policy 0, policy_version 793873 (0.0009) [2023-12-26 21:08:24,818][105692] Updated weights for policy 0, policy_version 793883 (0.0010) [2023-12-26 21:08:24,872][105692] Updated weights for policy 0, policy_version 793893 (0.0010) [2023-12-26 21:08:25,312][105620] Updated weights for policy 1, policy_version 793615 (0.0007) [2023-12-26 21:08:25,381][105620] Updated weights for policy 1, policy_version 793625 (0.0006) [2023-12-26 21:08:25,436][105620] Updated weights for policy 1, policy_version 793635 (0.0006) [2023-12-26 21:08:25,742][105692] Updated weights for policy 0, policy_version 793903 (0.0007) [2023-12-26 21:08:25,799][105692] Updated weights for policy 0, policy_version 793913 (0.0005) [2023-12-26 21:08:25,856][105692] Updated weights for policy 0, policy_version 793923 (0.0006) [2023-12-26 21:08:26,050][105620] Updated weights for policy 1, policy_version 793645 (0.0005) [2023-12-26 21:08:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 406470656. Throughput: 0: 9909.5, 1: 9394.9. Samples: 406479428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:08:26,062][104569] Avg episode reward: [(0, '9259.390'), (1, '9262.240')] [2023-12-26 21:08:26,100][105620] Updated weights for policy 1, policy_version 793655 (0.0005) [2023-12-26 21:08:26,155][105620] Updated weights for policy 1, policy_version 793665 (0.0006) [2023-12-26 21:08:26,594][105692] Updated weights for policy 0, policy_version 793933 (0.0009) [2023-12-26 21:08:26,648][105692] Updated weights for policy 0, policy_version 793943 (0.0008) [2023-12-26 21:08:26,701][105692] Updated weights for policy 0, policy_version 793953 (0.0009) [2023-12-26 21:08:26,861][105620] Updated weights for policy 1, policy_version 793675 (0.0009) [2023-12-26 21:08:26,914][105620] Updated weights for policy 1, policy_version 793685 (0.0010) [2023-12-26 21:08:26,971][105620] Updated weights for policy 1, policy_version 793695 (0.0010) [2023-12-26 21:08:27,493][105692] Updated weights for policy 0, policy_version 793963 (0.0008) [2023-12-26 21:08:27,538][105692] Updated weights for policy 0, policy_version 793973 (0.0008) [2023-12-26 21:08:27,593][105692] Updated weights for policy 0, policy_version 793983 (0.0008) [2023-12-26 21:08:27,685][105620] Updated weights for policy 1, policy_version 793705 (0.0010) [2023-12-26 21:08:27,733][105620] Updated weights for policy 1, policy_version 793715 (0.0010) [2023-12-26 21:08:27,787][105620] Updated weights for policy 1, policy_version 793725 (0.0010) [2023-12-26 21:08:27,844][105620] Updated weights for policy 1, policy_version 793735 (0.0010) [2023-12-26 21:08:28,416][105692] Updated weights for policy 0, policy_version 793993 (0.0008) [2023-12-26 21:08:28,481][105692] Updated weights for policy 0, policy_version 794003 (0.0007) [2023-12-26 21:08:28,514][105620] Updated weights for policy 1, policy_version 793745 (0.0008) [2023-12-26 21:08:28,532][105692] Updated weights for policy 0, policy_version 794013 (0.0007) [2023-12-26 21:08:28,571][105620] Updated weights for policy 1, policy_version 793755 (0.0009) [2023-12-26 21:08:28,582][105692] Updated weights for policy 0, policy_version 794023 (0.0007) [2023-12-26 21:08:28,637][105620] Updated weights for policy 1, policy_version 793765 (0.0011) [2023-12-26 21:08:29,254][105620] Updated weights for policy 1, policy_version 793775 (0.0009) [2023-12-26 21:08:29,310][105620] Updated weights for policy 1, policy_version 793785 (0.0006) [2023-12-26 21:08:29,379][105620] Updated weights for policy 1, policy_version 793795 (0.0011) [2023-12-26 21:08:29,422][105692] Updated weights for policy 0, policy_version 794033 (0.0007) [2023-12-26 21:08:29,475][105692] Updated weights for policy 0, policy_version 794043 (0.0008) [2023-12-26 21:08:29,528][105692] Updated weights for policy 0, policy_version 794053 (0.0008) [2023-12-26 21:08:30,087][105620] Updated weights for policy 1, policy_version 793805 (0.0011) [2023-12-26 21:08:30,145][105620] Updated weights for policy 1, policy_version 793815 (0.0010) [2023-12-26 21:08:30,205][105620] Updated weights for policy 1, policy_version 793825 (0.0010) [2023-12-26 21:08:30,312][105692] Updated weights for policy 0, policy_version 794063 (0.0007) [2023-12-26 21:08:30,370][105692] Updated weights for policy 0, policy_version 794073 (0.0007) [2023-12-26 21:08:30,430][105692] Updated weights for policy 0, policy_version 794083 (0.0007) [2023-12-26 21:08:30,866][105620] Updated weights for policy 1, policy_version 793835 (0.0009) [2023-12-26 21:08:30,911][105620] Updated weights for policy 1, policy_version 793845 (0.0005) [2023-12-26 21:08:30,956][105620] Updated weights for policy 1, policy_version 793855 (0.0005) [2023-12-26 21:08:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 406568960. Throughput: 0: 9866.1, 1: 9423.7. Samples: 406536480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:08:31,063][104569] Avg episode reward: [(0, '8728.699'), (1, '9079.326')] [2023-12-26 21:08:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000793864_203251712.pth... [2023-12-26 21:08:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000794088_203317248.pth... [2023-12-26 21:08:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000792744_202964992.pth [2023-12-26 21:08:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000792968_203030528.pth [2023-12-26 21:08:31,246][105692] Updated weights for policy 0, policy_version 794093 (0.0008) [2023-12-26 21:08:31,305][105692] Updated weights for policy 0, policy_version 794103 (0.0008) [2023-12-26 21:08:31,369][105692] Updated weights for policy 0, policy_version 794113 (0.0009) [2023-12-26 21:08:31,677][105620] Updated weights for policy 1, policy_version 793865 (0.0006) [2023-12-26 21:08:31,739][105620] Updated weights for policy 1, policy_version 793875 (0.0009) [2023-12-26 21:08:31,791][105620] Updated weights for policy 1, policy_version 793885 (0.0009) [2023-12-26 21:08:31,844][105620] Updated weights for policy 1, policy_version 793895 (0.0009) [2023-12-26 21:08:32,222][105692] Updated weights for policy 0, policy_version 794123 (0.0008) [2023-12-26 21:08:32,285][105692] Updated weights for policy 0, policy_version 794133 (0.0008) [2023-12-26 21:08:32,351][105692] Updated weights for policy 0, policy_version 794143 (0.0008) [2023-12-26 21:08:32,682][105620] Updated weights for policy 1, policy_version 793905 (0.0009) [2023-12-26 21:08:32,728][105620] Updated weights for policy 1, policy_version 793915 (0.0008) [2023-12-26 21:08:32,789][105620] Updated weights for policy 1, policy_version 793925 (0.0009) [2023-12-26 21:08:33,060][105692] Updated weights for policy 0, policy_version 794153 (0.0009) [2023-12-26 21:08:33,105][105692] Updated weights for policy 0, policy_version 794163 (0.0008) [2023-12-26 21:08:33,153][105692] Updated weights for policy 0, policy_version 794173 (0.0009) [2023-12-26 21:08:33,209][105692] Updated weights for policy 0, policy_version 794183 (0.0009) [2023-12-26 21:08:33,488][105620] Updated weights for policy 1, policy_version 793935 (0.0009) [2023-12-26 21:08:33,539][105620] Updated weights for policy 1, policy_version 793945 (0.0008) [2023-12-26 21:08:33,604][105620] Updated weights for policy 1, policy_version 793955 (0.0008) [2023-12-26 21:08:33,957][105692] Updated weights for policy 0, policy_version 794193 (0.0008) [2023-12-26 21:08:34,008][105692] Updated weights for policy 0, policy_version 794203 (0.0009) [2023-12-26 21:08:34,064][105692] Updated weights for policy 0, policy_version 794213 (0.0010) [2023-12-26 21:08:34,351][105620] Updated weights for policy 1, policy_version 793965 (0.0009) [2023-12-26 21:08:34,409][105620] Updated weights for policy 1, policy_version 793975 (0.0009) [2023-12-26 21:08:34,463][105620] Updated weights for policy 1, policy_version 793985 (0.0009) [2023-12-26 21:08:34,868][105692] Updated weights for policy 0, policy_version 794224 (0.0010) [2023-12-26 21:08:34,934][105692] Updated weights for policy 0, policy_version 794234 (0.0009) [2023-12-26 21:08:35,005][105692] Updated weights for policy 0, policy_version 794244 (0.0010) [2023-12-26 21:08:35,151][105620] Updated weights for policy 1, policy_version 793995 (0.0009) [2023-12-26 21:08:35,213][105620] Updated weights for policy 1, policy_version 794005 (0.0010) [2023-12-26 21:08:35,262][105620] Updated weights for policy 1, policy_version 794015 (0.0010) [2023-12-26 21:08:35,638][105692] Updated weights for policy 0, policy_version 794254 (0.0008) [2023-12-26 21:08:35,692][105692] Updated weights for policy 0, policy_version 794264 (0.0010) [2023-12-26 21:08:35,752][105692] Updated weights for policy 0, policy_version 794274 (0.0009) [2023-12-26 21:08:35,973][105620] Updated weights for policy 1, policy_version 794025 (0.0010) [2023-12-26 21:08:36,036][105620] Updated weights for policy 1, policy_version 794035 (0.0010) [2023-12-26 21:08:36,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19549.7). Total num frames: 406659072. Throughput: 0: 9725.6, 1: 9423.9. Samples: 406648772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:08:36,062][104569] Avg episode reward: [(0, '8831.782'), (1, '9169.883')] [2023-12-26 21:08:36,107][105620] Updated weights for policy 1, policy_version 794045 (0.0010) [2023-12-26 21:08:36,166][105620] Updated weights for policy 1, policy_version 794055 (0.0010) [2023-12-26 21:08:36,408][105692] Updated weights for policy 0, policy_version 794284 (0.0009) [2023-12-26 21:08:36,461][105692] Updated weights for policy 0, policy_version 794294 (0.0011) [2023-12-26 21:08:36,513][105692] Updated weights for policy 0, policy_version 794304 (0.0010) [2023-12-26 21:08:36,910][105620] Updated weights for policy 1, policy_version 794065 (0.0011) [2023-12-26 21:08:36,967][105620] Updated weights for policy 1, policy_version 794075 (0.0010) [2023-12-26 21:08:37,022][105620] Updated weights for policy 1, policy_version 794085 (0.0010) [2023-12-26 21:08:37,148][105692] Updated weights for policy 0, policy_version 794314 (0.0009) [2023-12-26 21:08:37,212][105692] Updated weights for policy 0, policy_version 794324 (0.0007) [2023-12-26 21:08:37,262][105692] Updated weights for policy 0, policy_version 794334 (0.0007) [2023-12-26 21:08:37,322][105692] Updated weights for policy 0, policy_version 794344 (0.0006) [2023-12-26 21:08:37,777][105620] Updated weights for policy 1, policy_version 794095 (0.0010) [2023-12-26 21:08:37,833][105620] Updated weights for policy 1, policy_version 794105 (0.0010) [2023-12-26 21:08:37,880][105692] Updated weights for policy 0, policy_version 794354 (0.0008) [2023-12-26 21:08:37,896][105620] Updated weights for policy 1, policy_version 794115 (0.0010) [2023-12-26 21:08:37,930][105692] Updated weights for policy 0, policy_version 794364 (0.0007) [2023-12-26 21:08:37,981][105692] Updated weights for policy 0, policy_version 794374 (0.0005) [2023-12-26 21:08:38,646][105620] Updated weights for policy 1, policy_version 794125 (0.0011) [2023-12-26 21:08:38,652][105692] Updated weights for policy 0, policy_version 794384 (0.0010) [2023-12-26 21:08:38,695][105620] Updated weights for policy 1, policy_version 794135 (0.0011) [2023-12-26 21:08:38,716][105692] Updated weights for policy 0, policy_version 794394 (0.0006) [2023-12-26 21:08:38,749][105620] Updated weights for policy 1, policy_version 794145 (0.0009) [2023-12-26 21:08:38,776][105692] Updated weights for policy 0, policy_version 794404 (0.0006) [2023-12-26 21:08:39,365][105620] Updated weights for policy 1, policy_version 794155 (0.0007) [2023-12-26 21:08:39,385][105692] Updated weights for policy 0, policy_version 794414 (0.0009) [2023-12-26 21:08:39,432][105620] Updated weights for policy 1, policy_version 794165 (0.0009) [2023-12-26 21:08:39,451][105692] Updated weights for policy 0, policy_version 794424 (0.0009) [2023-12-26 21:08:39,491][105620] Updated weights for policy 1, policy_version 794175 (0.0006) [2023-12-26 21:08:39,512][105692] Updated weights for policy 0, policy_version 794434 (0.0008) [2023-12-26 21:08:40,187][105620] Updated weights for policy 1, policy_version 794185 (0.0007) [2023-12-26 21:08:40,253][105620] Updated weights for policy 1, policy_version 794195 (0.0009) [2023-12-26 21:08:40,315][105620] Updated weights for policy 1, policy_version 794205 (0.0008) [2023-12-26 21:08:40,318][105692] Updated weights for policy 0, policy_version 794444 (0.0007) [2023-12-26 21:08:40,373][105620] Updated weights for policy 1, policy_version 794215 (0.0007) [2023-12-26 21:08:40,379][105692] Updated weights for policy 0, policy_version 794454 (0.0006) [2023-12-26 21:08:40,436][105692] Updated weights for policy 0, policy_version 794464 (0.0008) [2023-12-26 21:08:41,054][105692] Updated weights for policy 0, policy_version 794474 (0.0009) [2023-12-26 21:08:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 406757376. Throughput: 0: 9803.5, 1: 9493.2. Samples: 406769196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:08:41,062][104569] Avg episode reward: [(0, '9172.265'), (1, '8786.376')] [2023-12-26 21:08:41,106][105692] Updated weights for policy 0, policy_version 794484 (0.0009) [2023-12-26 21:08:41,161][105620] Updated weights for policy 1, policy_version 794225 (0.0008) [2023-12-26 21:08:41,170][105692] Updated weights for policy 0, policy_version 794494 (0.0009) [2023-12-26 21:08:41,227][105620] Updated weights for policy 1, policy_version 794235 (0.0008) [2023-12-26 21:08:41,229][105692] Updated weights for policy 0, policy_version 794504 (0.0008) [2023-12-26 21:08:41,287][105620] Updated weights for policy 1, policy_version 794245 (0.0007) [2023-12-26 21:08:42,018][105692] Updated weights for policy 0, policy_version 794514 (0.0008) [2023-12-26 21:08:42,044][105620] Updated weights for policy 1, policy_version 794255 (0.0006) [2023-12-26 21:08:42,082][105692] Updated weights for policy 0, policy_version 794524 (0.0006) [2023-12-26 21:08:42,100][105620] Updated weights for policy 1, policy_version 794265 (0.0008) [2023-12-26 21:08:42,142][105692] Updated weights for policy 0, policy_version 794534 (0.0007) [2023-12-26 21:08:42,165][105620] Updated weights for policy 1, policy_version 794275 (0.0006) [2023-12-26 21:08:42,789][105620] Updated weights for policy 1, policy_version 794285 (0.0008) [2023-12-26 21:08:42,847][105620] Updated weights for policy 1, policy_version 794295 (0.0009) [2023-12-26 21:08:42,906][105620] Updated weights for policy 1, policy_version 794305 (0.0009) [2023-12-26 21:08:42,907][105692] Updated weights for policy 0, policy_version 794544 (0.0008) [2023-12-26 21:08:42,969][105692] Updated weights for policy 0, policy_version 794554 (0.0008) [2023-12-26 21:08:43,038][105692] Updated weights for policy 0, policy_version 794564 (0.0009) [2023-12-26 21:08:43,599][105620] Updated weights for policy 1, policy_version 794315 (0.0007) [2023-12-26 21:08:43,658][105620] Updated weights for policy 1, policy_version 794326 (0.0010) [2023-12-26 21:08:43,717][105620] Updated weights for policy 1, policy_version 794337 (0.0010) [2023-12-26 21:08:43,740][105692] Updated weights for policy 0, policy_version 794574 (0.0008) [2023-12-26 21:08:43,796][105692] Updated weights for policy 0, policy_version 794584 (0.0008) [2023-12-26 21:08:43,860][105692] Updated weights for policy 0, policy_version 794594 (0.0010) [2023-12-26 21:08:44,446][105620] Updated weights for policy 1, policy_version 794347 (0.0009) [2023-12-26 21:08:44,493][105620] Updated weights for policy 1, policy_version 794357 (0.0008) [2023-12-26 21:08:44,550][105620] Updated weights for policy 1, policy_version 794367 (0.0010) [2023-12-26 21:08:44,580][105692] Updated weights for policy 0, policy_version 794604 (0.0009) [2023-12-26 21:08:44,641][105692] Updated weights for policy 0, policy_version 794614 (0.0008) [2023-12-26 21:08:44,706][105692] Updated weights for policy 0, policy_version 794624 (0.0009) [2023-12-26 21:08:45,329][105620] Updated weights for policy 1, policy_version 794377 (0.0010) [2023-12-26 21:08:45,389][105620] Updated weights for policy 1, policy_version 794387 (0.0009) [2023-12-26 21:08:45,443][105692] Updated weights for policy 0, policy_version 794634 (0.0009) [2023-12-26 21:08:45,449][105620] Updated weights for policy 1, policy_version 794397 (0.0008) [2023-12-26 21:08:45,499][105692] Updated weights for policy 0, policy_version 794644 (0.0007) [2023-12-26 21:08:45,501][105620] Updated weights for policy 1, policy_version 794407 (0.0006) [2023-12-26 21:08:45,562][105692] Updated weights for policy 0, policy_version 794654 (0.0008) [2023-12-26 21:08:45,620][105692] Updated weights for policy 0, policy_version 794664 (0.0009) [2023-12-26 21:08:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 406855680. Throughput: 0: 9747.8, 1: 9567.4. Samples: 406827508. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 21:08:46,062][104569] Avg episode reward: [(0, '8999.959'), (1, '8878.263')] [2023-12-26 21:08:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000794664_203464704.pth... [2023-12-26 21:08:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000794408_203390976.pth... [2023-12-26 21:08:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000793288_203104256.pth [2023-12-26 21:08:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000793576_203186176.pth [2023-12-26 21:08:46,279][105620] Updated weights for policy 1, policy_version 794417 (0.0008) [2023-12-26 21:08:46,316][105692] Updated weights for policy 0, policy_version 794674 (0.0008) [2023-12-26 21:08:46,331][105620] Updated weights for policy 1, policy_version 794427 (0.0006) [2023-12-26 21:08:46,380][105692] Updated weights for policy 0, policy_version 794684 (0.0009) [2023-12-26 21:08:46,390][105620] Updated weights for policy 1, policy_version 794437 (0.0006) [2023-12-26 21:08:46,438][105692] Updated weights for policy 0, policy_version 794694 (0.0008) [2023-12-26 21:08:47,067][105620] Updated weights for policy 1, policy_version 794447 (0.0005) [2023-12-26 21:08:47,145][105620] Updated weights for policy 1, policy_version 794457 (0.0005) [2023-12-26 21:08:47,206][105620] Updated weights for policy 1, policy_version 794467 (0.0005) [2023-12-26 21:08:47,233][105692] Updated weights for policy 0, policy_version 794704 (0.0009) [2023-12-26 21:08:47,285][105692] Updated weights for policy 0, policy_version 794714 (0.0010) [2023-12-26 21:08:47,340][105692] Updated weights for policy 0, policy_version 794726 (0.0010) [2023-12-26 21:08:47,689][105620] Updated weights for policy 1, policy_version 794477 (0.0005) [2023-12-26 21:08:47,743][105620] Updated weights for policy 1, policy_version 794487 (0.0005) [2023-12-26 21:08:47,796][105620] Updated weights for policy 1, policy_version 794497 (0.0005) [2023-12-26 21:08:48,243][105692] Updated weights for policy 0, policy_version 794736 (0.0010) [2023-12-26 21:08:48,313][105692] Updated weights for policy 0, policy_version 794746 (0.0010) [2023-12-26 21:08:48,340][105620] Updated weights for policy 1, policy_version 794507 (0.0007) [2023-12-26 21:08:48,381][105692] Updated weights for policy 0, policy_version 794756 (0.0009) [2023-12-26 21:08:48,414][105620] Updated weights for policy 1, policy_version 794517 (0.0010) [2023-12-26 21:08:48,482][105620] Updated weights for policy 1, policy_version 794527 (0.0009) [2023-12-26 21:08:49,175][105692] Updated weights for policy 0, policy_version 794766 (0.0008) [2023-12-26 21:08:49,181][105620] Updated weights for policy 1, policy_version 794537 (0.0010) [2023-12-26 21:08:49,243][105692] Updated weights for policy 0, policy_version 794776 (0.0007) [2023-12-26 21:08:49,248][105620] Updated weights for policy 1, policy_version 794547 (0.0007) [2023-12-26 21:08:49,312][105692] Updated weights for policy 0, policy_version 794786 (0.0006) [2023-12-26 21:08:49,322][105620] Updated weights for policy 1, policy_version 794557 (0.0007) [2023-12-26 21:08:49,383][105620] Updated weights for policy 1, policy_version 794567 (0.0010) [2023-12-26 21:08:50,025][105692] Updated weights for policy 0, policy_version 794796 (0.0007) [2023-12-26 21:08:50,089][105692] Updated weights for policy 0, policy_version 794806 (0.0008) [2023-12-26 21:08:50,130][105620] Updated weights for policy 1, policy_version 794577 (0.0009) [2023-12-26 21:08:50,153][105692] Updated weights for policy 0, policy_version 794816 (0.0006) [2023-12-26 21:08:50,206][105620] Updated weights for policy 1, policy_version 794587 (0.0008) [2023-12-26 21:08:50,264][105620] Updated weights for policy 1, policy_version 794597 (0.0008) [2023-12-26 21:08:50,831][105692] Updated weights for policy 0, policy_version 794826 (0.0009) [2023-12-26 21:08:50,890][105692] Updated weights for policy 0, policy_version 794836 (0.0009) [2023-12-26 21:08:50,949][105692] Updated weights for policy 0, policy_version 794846 (0.0009) [2023-12-26 21:08:51,012][105692] Updated weights for policy 0, policy_version 794856 (0.0009) [2023-12-26 21:08:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 406953984. Throughput: 0: 9564.3, 1: 9763.5. Samples: 406943060. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 21:08:51,062][104569] Avg episode reward: [(0, '7697.542'), (1, '9354.963')] [2023-12-26 21:08:51,073][105620] Updated weights for policy 1, policy_version 794607 (0.0008) [2023-12-26 21:08:51,139][105620] Updated weights for policy 1, policy_version 794617 (0.0009) [2023-12-26 21:08:51,195][105620] Updated weights for policy 1, policy_version 794627 (0.0006) [2023-12-26 21:08:51,742][105692] Updated weights for policy 0, policy_version 794866 (0.0007) [2023-12-26 21:08:51,802][105692] Updated weights for policy 0, policy_version 794876 (0.0008) [2023-12-26 21:08:51,860][105692] Updated weights for policy 0, policy_version 794886 (0.0008) [2023-12-26 21:08:51,949][105620] Updated weights for policy 1, policy_version 794637 (0.0008) [2023-12-26 21:08:52,001][105620] Updated weights for policy 1, policy_version 794647 (0.0010) [2023-12-26 21:08:52,056][105620] Updated weights for policy 1, policy_version 794657 (0.0010) [2023-12-26 21:08:52,516][105692] Updated weights for policy 0, policy_version 794896 (0.0007) [2023-12-26 21:08:52,579][105692] Updated weights for policy 0, policy_version 794906 (0.0008) [2023-12-26 21:08:52,637][105692] Updated weights for policy 0, policy_version 794916 (0.0008) [2023-12-26 21:08:52,725][105620] Updated weights for policy 1, policy_version 794667 (0.0010) [2023-12-26 21:08:52,781][105620] Updated weights for policy 1, policy_version 794677 (0.0010) [2023-12-26 21:08:52,846][105620] Updated weights for policy 1, policy_version 794687 (0.0010) [2023-12-26 21:08:53,280][105692] Updated weights for policy 0, policy_version 794926 (0.0008) [2023-12-26 21:08:53,331][105692] Updated weights for policy 0, policy_version 794936 (0.0005) [2023-12-26 21:08:53,396][105692] Updated weights for policy 0, policy_version 794946 (0.0005) [2023-12-26 21:08:53,477][105620] Updated weights for policy 1, policy_version 794697 (0.0010) [2023-12-26 21:08:53,539][105620] Updated weights for policy 1, policy_version 794707 (0.0008) [2023-12-26 21:08:53,609][105620] Updated weights for policy 1, policy_version 794717 (0.0007) [2023-12-26 21:08:53,670][105620] Updated weights for policy 1, policy_version 794727 (0.0009) [2023-12-26 21:08:54,033][105692] Updated weights for policy 0, policy_version 794956 (0.0007) [2023-12-26 21:08:54,094][105692] Updated weights for policy 0, policy_version 794966 (0.0007) [2023-12-26 21:08:54,146][105692] Updated weights for policy 0, policy_version 794976 (0.0005) [2023-12-26 21:08:54,248][105620] Updated weights for policy 1, policy_version 794737 (0.0005) [2023-12-26 21:08:54,307][105620] Updated weights for policy 1, policy_version 794747 (0.0010) [2023-12-26 21:08:54,372][105620] Updated weights for policy 1, policy_version 794757 (0.0010) [2023-12-26 21:08:54,805][105692] Updated weights for policy 0, policy_version 794986 (0.0008) [2023-12-26 21:08:54,867][105692] Updated weights for policy 0, policy_version 794996 (0.0008) [2023-12-26 21:08:54,919][105692] Updated weights for policy 0, policy_version 795006 (0.0008) [2023-12-26 21:08:54,975][105692] Updated weights for policy 0, policy_version 795016 (0.0008) [2023-12-26 21:08:55,040][105620] Updated weights for policy 1, policy_version 794767 (0.0010) [2023-12-26 21:08:55,102][105620] Updated weights for policy 1, policy_version 794777 (0.0010) [2023-12-26 21:08:55,159][105620] Updated weights for policy 1, policy_version 794787 (0.0008) [2023-12-26 21:08:55,749][105620] Updated weights for policy 1, policy_version 794797 (0.0005) [2023-12-26 21:08:55,762][105692] Updated weights for policy 0, policy_version 795026 (0.0009) [2023-12-26 21:08:55,795][105620] Updated weights for policy 1, policy_version 794807 (0.0005) [2023-12-26 21:08:55,810][105692] Updated weights for policy 0, policy_version 795036 (0.0010) [2023-12-26 21:08:55,846][105620] Updated weights for policy 1, policy_version 794817 (0.0005) [2023-12-26 21:08:55,861][105692] Updated weights for policy 0, policy_version 795046 (0.0008) [2023-12-26 21:08:56,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 407060480. Throughput: 0: 9521.0, 1: 9831.6. Samples: 407064740. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 21:08:56,063][104569] Avg episode reward: [(0, '6761.591'), (1, '9263.150')] [2023-12-26 21:08:56,491][105620] Updated weights for policy 1, policy_version 794827 (0.0007) [2023-12-26 21:08:56,543][105620] Updated weights for policy 1, policy_version 794837 (0.0008) [2023-12-26 21:08:56,604][105620] Updated weights for policy 1, policy_version 794847 (0.0009) [2023-12-26 21:08:56,664][105692] Updated weights for policy 0, policy_version 795056 (0.0008) [2023-12-26 21:08:56,710][105692] Updated weights for policy 0, policy_version 795066 (0.0009) [2023-12-26 21:08:56,756][105692] Updated weights for policy 0, policy_version 795076 (0.0008) [2023-12-26 21:08:57,340][105620] Updated weights for policy 1, policy_version 794857 (0.0007) [2023-12-26 21:08:57,402][105620] Updated weights for policy 1, policy_version 794867 (0.0009) [2023-12-26 21:08:57,458][105620] Updated weights for policy 1, policy_version 794877 (0.0009) [2023-12-26 21:08:57,516][105620] Updated weights for policy 1, policy_version 794887 (0.0008) [2023-12-26 21:08:57,529][105692] Updated weights for policy 0, policy_version 795086 (0.0008) [2023-12-26 21:08:57,583][105692] Updated weights for policy 0, policy_version 795096 (0.0009) [2023-12-26 21:08:57,629][105692] Updated weights for policy 0, policy_version 795106 (0.0008) [2023-12-26 21:08:58,257][105620] Updated weights for policy 1, policy_version 794897 (0.0006) [2023-12-26 21:08:58,321][105620] Updated weights for policy 1, policy_version 794907 (0.0007) [2023-12-26 21:08:58,385][105620] Updated weights for policy 1, policy_version 794917 (0.0008) [2023-12-26 21:08:58,423][105692] Updated weights for policy 0, policy_version 795116 (0.0008) [2023-12-26 21:08:58,481][105692] Updated weights for policy 0, policy_version 795126 (0.0007) [2023-12-26 21:08:58,547][105692] Updated weights for policy 0, policy_version 795136 (0.0007) [2023-12-26 21:08:59,189][105620] Updated weights for policy 1, policy_version 794927 (0.0007) [2023-12-26 21:08:59,261][105620] Updated weights for policy 1, policy_version 794937 (0.0009) [2023-12-26 21:08:59,319][105692] Updated weights for policy 0, policy_version 795146 (0.0007) [2023-12-26 21:08:59,328][105620] Updated weights for policy 1, policy_version 794947 (0.0008) [2023-12-26 21:08:59,388][105692] Updated weights for policy 0, policy_version 795156 (0.0008) [2023-12-26 21:08:59,454][105692] Updated weights for policy 0, policy_version 795166 (0.0008) [2023-12-26 21:08:59,515][105692] Updated weights for policy 0, policy_version 795176 (0.0008) [2023-12-26 21:09:00,072][105620] Updated weights for policy 1, policy_version 794957 (0.0009) [2023-12-26 21:09:00,127][105620] Updated weights for policy 1, policy_version 794967 (0.0008) [2023-12-26 21:09:00,180][105620] Updated weights for policy 1, policy_version 794977 (0.0009) [2023-12-26 21:09:00,238][105692] Updated weights for policy 0, policy_version 795186 (0.0009) [2023-12-26 21:09:00,297][105692] Updated weights for policy 0, policy_version 795196 (0.0009) [2023-12-26 21:09:00,353][105692] Updated weights for policy 0, policy_version 795206 (0.0009) [2023-12-26 21:09:00,896][105620] Updated weights for policy 1, policy_version 794987 (0.0006) [2023-12-26 21:09:00,951][105620] Updated weights for policy 1, policy_version 794997 (0.0005) [2023-12-26 21:09:01,003][105620] Updated weights for policy 1, policy_version 795007 (0.0006) [2023-12-26 21:09:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 407150592. Throughput: 0: 9488.6, 1: 9850.1. Samples: 407120964. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 21:09:01,063][104569] Avg episode reward: [(0, '692.969'), (1, '9262.480')] [2023-12-26 21:09:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000795208_203603968.pth... [2023-12-26 21:09:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000795016_203546624.pth... [2023-12-26 21:09:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000793864_203251712.pth [2023-12-26 21:09:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000794088_203317248.pth [2023-12-26 21:09:01,159][105692] Updated weights for policy 0, policy_version 795216 (0.0009) [2023-12-26 21:09:01,228][105692] Updated weights for policy 0, policy_version 795226 (0.0008) [2023-12-26 21:09:01,293][105692] Updated weights for policy 0, policy_version 795236 (0.0008) [2023-12-26 21:09:01,697][105620] Updated weights for policy 1, policy_version 795017 (0.0007) [2023-12-26 21:09:01,766][105620] Updated weights for policy 1, policy_version 795027 (0.0007) [2023-12-26 21:09:01,828][105620] Updated weights for policy 1, policy_version 795037 (0.0008) [2023-12-26 21:09:01,894][105620] Updated weights for policy 1, policy_version 795047 (0.0009) [2023-12-26 21:09:02,004][105692] Updated weights for policy 0, policy_version 795246 (0.0007) [2023-12-26 21:09:02,061][105692] Updated weights for policy 0, policy_version 795256 (0.0008) [2023-12-26 21:09:02,119][105692] Updated weights for policy 0, policy_version 795266 (0.0009) [2023-12-26 21:09:02,562][105620] Updated weights for policy 1, policy_version 795057 (0.0010) [2023-12-26 21:09:02,624][105620] Updated weights for policy 1, policy_version 795067 (0.0007) [2023-12-26 21:09:02,686][105620] Updated weights for policy 1, policy_version 795077 (0.0006) [2023-12-26 21:09:02,959][105692] Updated weights for policy 0, policy_version 795276 (0.0009) [2023-12-26 21:09:03,018][105692] Updated weights for policy 0, policy_version 795287 (0.0009) [2023-12-26 21:09:03,066][105692] Updated weights for policy 0, policy_version 795297 (0.0008) [2023-12-26 21:09:03,295][105620] Updated weights for policy 1, policy_version 795087 (0.0010) [2023-12-26 21:09:03,346][105620] Updated weights for policy 1, policy_version 795097 (0.0010) [2023-12-26 21:09:03,394][105620] Updated weights for policy 1, policy_version 795107 (0.0010) [2023-12-26 21:09:03,834][105692] Updated weights for policy 0, policy_version 795307 (0.0008) [2023-12-26 21:09:03,897][105692] Updated weights for policy 0, policy_version 795317 (0.0008) [2023-12-26 21:09:03,957][105692] Updated weights for policy 0, policy_version 795327 (0.0007) [2023-12-26 21:09:04,139][105620] Updated weights for policy 1, policy_version 795117 (0.0011) [2023-12-26 21:09:04,208][105620] Updated weights for policy 1, policy_version 795127 (0.0011) [2023-12-26 21:09:04,273][105620] Updated weights for policy 1, policy_version 795137 (0.0008) [2023-12-26 21:09:04,719][105692] Updated weights for policy 0, policy_version 795337 (0.0007) [2023-12-26 21:09:04,773][105692] Updated weights for policy 0, policy_version 795347 (0.0010) [2023-12-26 21:09:04,825][105692] Updated weights for policy 0, policy_version 795357 (0.0009) [2023-12-26 21:09:04,874][105692] Updated weights for policy 0, policy_version 795367 (0.0008) [2023-12-26 21:09:04,935][105620] Updated weights for policy 1, policy_version 795147 (0.0011) [2023-12-26 21:09:04,998][105620] Updated weights for policy 1, policy_version 795157 (0.0008) [2023-12-26 21:09:05,068][105620] Updated weights for policy 1, policy_version 795167 (0.0006) [2023-12-26 21:09:05,612][105692] Updated weights for policy 0, policy_version 795377 (0.0006) [2023-12-26 21:09:05,623][105620] Updated weights for policy 1, policy_version 795177 (0.0006) [2023-12-26 21:09:05,670][105692] Updated weights for policy 0, policy_version 795387 (0.0005) [2023-12-26 21:09:05,677][105620] Updated weights for policy 1, policy_version 795187 (0.0005) [2023-12-26 21:09:05,732][105692] Updated weights for policy 0, policy_version 795397 (0.0006) [2023-12-26 21:09:05,739][105620] Updated weights for policy 1, policy_version 795197 (0.0005) [2023-12-26 21:09:05,786][105620] Updated weights for policy 1, policy_version 795207 (0.0005) [2023-12-26 21:09:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 407248896. Throughput: 0: 9357.2, 1: 9925.9. Samples: 407234644. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 21:09:06,063][104569] Avg episode reward: [(0, '656.093'), (1, '9170.762')] [2023-12-26 21:09:06,365][105692] Updated weights for policy 0, policy_version 795407 (0.0008) [2023-12-26 21:09:06,407][105620] Updated weights for policy 1, policy_version 795217 (0.0006) [2023-12-26 21:09:06,433][105692] Updated weights for policy 0, policy_version 795417 (0.0008) [2023-12-26 21:09:06,463][105620] Updated weights for policy 1, policy_version 795227 (0.0006) [2023-12-26 21:09:06,499][105692] Updated weights for policy 0, policy_version 795427 (0.0008) [2023-12-26 21:09:06,519][105620] Updated weights for policy 1, policy_version 795237 (0.0006) [2023-12-26 21:09:07,113][105692] Updated weights for policy 0, policy_version 795437 (0.0008) [2023-12-26 21:09:07,155][105620] Updated weights for policy 1, policy_version 795247 (0.0010) [2023-12-26 21:09:07,168][105692] Updated weights for policy 0, policy_version 795447 (0.0005) [2023-12-26 21:09:07,220][105620] Updated weights for policy 1, policy_version 795257 (0.0006) [2023-12-26 21:09:07,230][105692] Updated weights for policy 0, policy_version 795457 (0.0005) [2023-12-26 21:09:07,275][105620] Updated weights for policy 1, policy_version 795267 (0.0006) [2023-12-26 21:09:07,778][105692] Updated weights for policy 0, policy_version 795467 (0.0005) [2023-12-26 21:09:07,833][105692] Updated weights for policy 0, policy_version 795477 (0.0005) [2023-12-26 21:09:07,884][105692] Updated weights for policy 0, policy_version 795487 (0.0005) [2023-12-26 21:09:07,998][105620] Updated weights for policy 1, policy_version 795277 (0.0007) [2023-12-26 21:09:08,050][105620] Updated weights for policy 1, policy_version 795287 (0.0008) [2023-12-26 21:09:08,111][105620] Updated weights for policy 1, policy_version 795297 (0.0009) [2023-12-26 21:09:08,570][105692] Updated weights for policy 0, policy_version 795497 (0.0006) [2023-12-26 21:09:08,622][105692] Updated weights for policy 0, policy_version 795507 (0.0010) [2023-12-26 21:09:08,678][105692] Updated weights for policy 0, policy_version 795517 (0.0010) [2023-12-26 21:09:08,735][105692] Updated weights for policy 0, policy_version 795527 (0.0011) [2023-12-26 21:09:08,867][105620] Updated weights for policy 1, policy_version 795307 (0.0008) [2023-12-26 21:09:08,932][105620] Updated weights for policy 1, policy_version 795317 (0.0010) [2023-12-26 21:09:08,996][105620] Updated weights for policy 1, policy_version 795327 (0.0011) [2023-12-26 21:09:09,517][105692] Updated weights for policy 0, policy_version 795537 (0.0011) [2023-12-26 21:09:09,585][105692] Updated weights for policy 0, policy_version 795547 (0.0011) [2023-12-26 21:09:09,647][105692] Updated weights for policy 0, policy_version 795557 (0.0011) [2023-12-26 21:09:09,740][105620] Updated weights for policy 1, policy_version 795337 (0.0010) [2023-12-26 21:09:09,809][105620] Updated weights for policy 1, policy_version 795347 (0.0009) [2023-12-26 21:09:09,879][105620] Updated weights for policy 1, policy_version 795357 (0.0009) [2023-12-26 21:09:09,947][105620] Updated weights for policy 1, policy_version 795367 (0.0009) [2023-12-26 21:09:10,407][105692] Updated weights for policy 0, policy_version 795567 (0.0009) [2023-12-26 21:09:10,461][105692] Updated weights for policy 0, policy_version 795577 (0.0009) [2023-12-26 21:09:10,514][105692] Updated weights for policy 0, policy_version 795587 (0.0007) [2023-12-26 21:09:10,677][105620] Updated weights for policy 1, policy_version 795377 (0.0009) [2023-12-26 21:09:10,734][105620] Updated weights for policy 1, policy_version 795387 (0.0009) [2023-12-26 21:09:10,781][105620] Updated weights for policy 1, policy_version 795397 (0.0009) [2023-12-26 21:09:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 407347200. Throughput: 0: 9531.5, 1: 9919.1. Samples: 407354704. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 21:09:11,063][104569] Avg episode reward: [(0, '693.770'), (1, '9169.829')] [2023-12-26 21:09:11,299][105692] Updated weights for policy 0, policy_version 795597 (0.0009) [2023-12-26 21:09:11,349][105692] Updated weights for policy 0, policy_version 795607 (0.0009) [2023-12-26 21:09:11,414][105692] Updated weights for policy 0, policy_version 795617 (0.0010) [2023-12-26 21:09:11,588][105620] Updated weights for policy 1, policy_version 795407 (0.0007) [2023-12-26 21:09:11,651][105620] Updated weights for policy 1, policy_version 795417 (0.0008) [2023-12-26 21:09:11,707][105620] Updated weights for policy 1, policy_version 795427 (0.0007) [2023-12-26 21:09:12,228][105692] Updated weights for policy 0, policy_version 795627 (0.0011) [2023-12-26 21:09:12,292][105692] Updated weights for policy 0, policy_version 795637 (0.0011) [2023-12-26 21:09:12,361][105692] Updated weights for policy 0, policy_version 795647 (0.0011) [2023-12-26 21:09:12,477][105620] Updated weights for policy 1, policy_version 795437 (0.0009) [2023-12-26 21:09:12,532][105620] Updated weights for policy 1, policy_version 795447 (0.0009) [2023-12-26 21:09:12,595][105620] Updated weights for policy 1, policy_version 795457 (0.0008) [2023-12-26 21:09:13,084][105692] Updated weights for policy 0, policy_version 795657 (0.0011) [2023-12-26 21:09:13,135][105692] Updated weights for policy 0, policy_version 795667 (0.0010) [2023-12-26 21:09:13,191][105692] Updated weights for policy 0, policy_version 795677 (0.0010) [2023-12-26 21:09:13,252][105692] Updated weights for policy 0, policy_version 795687 (0.0010) [2023-12-26 21:09:13,293][105620] Updated weights for policy 1, policy_version 795467 (0.0007) [2023-12-26 21:09:13,348][105620] Updated weights for policy 1, policy_version 795477 (0.0009) [2023-12-26 21:09:13,401][105620] Updated weights for policy 1, policy_version 795487 (0.0010) [2023-12-26 21:09:13,843][105692] Updated weights for policy 0, policy_version 795697 (0.0006) [2023-12-26 21:09:13,900][105692] Updated weights for policy 0, policy_version 795707 (0.0005) [2023-12-26 21:09:13,957][105692] Updated weights for policy 0, policy_version 795717 (0.0009) [2023-12-26 21:09:14,244][105620] Updated weights for policy 1, policy_version 795497 (0.0009) [2023-12-26 21:09:14,307][105620] Updated weights for policy 1, policy_version 795507 (0.0006) [2023-12-26 21:09:14,366][105620] Updated weights for policy 1, policy_version 795517 (0.0005) [2023-12-26 21:09:14,424][105620] Updated weights for policy 1, policy_version 795527 (0.0007) [2023-12-26 21:09:14,506][105692] Updated weights for policy 0, policy_version 795727 (0.0005) [2023-12-26 21:09:14,552][105692] Updated weights for policy 0, policy_version 795737 (0.0005) [2023-12-26 21:09:14,605][105692] Updated weights for policy 0, policy_version 795747 (0.0006) [2023-12-26 21:09:15,025][105620] Updated weights for policy 1, policy_version 795537 (0.0009) [2023-12-26 21:09:15,087][105620] Updated weights for policy 1, policy_version 795547 (0.0008) [2023-12-26 21:09:15,148][105620] Updated weights for policy 1, policy_version 795557 (0.0009) [2023-12-26 21:09:15,294][105692] Updated weights for policy 0, policy_version 795757 (0.0008) [2023-12-26 21:09:15,357][105692] Updated weights for policy 0, policy_version 795767 (0.0010) [2023-12-26 21:09:15,413][105692] Updated weights for policy 0, policy_version 795777 (0.0010) [2023-12-26 21:09:15,918][105620] Updated weights for policy 1, policy_version 795567 (0.0008) [2023-12-26 21:09:15,977][105620] Updated weights for policy 1, policy_version 795577 (0.0008) [2023-12-26 21:09:16,040][105620] Updated weights for policy 1, policy_version 795587 (0.0008) [2023-12-26 21:09:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 407437312. Throughput: 0: 9560.0, 1: 9870.2. Samples: 407410840. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 21:09:16,062][104569] Avg episode reward: [(0, '716.238'), (1, '9170.600')] [2023-12-26 21:09:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000795592_203694080.pth... [2023-12-26 21:09:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000795784_203751424.pth... [2023-12-26 21:09:16,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000794664_203464704.pth [2023-12-26 21:09:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000794408_203390976.pth [2023-12-26 21:09:16,168][105692] Updated weights for policy 0, policy_version 795787 (0.0010) [2023-12-26 21:09:16,223][105692] Updated weights for policy 0, policy_version 795797 (0.0010) [2023-12-26 21:09:16,278][105692] Updated weights for policy 0, policy_version 795807 (0.0010) [2023-12-26 21:09:16,844][105620] Updated weights for policy 1, policy_version 795597 (0.0009) [2023-12-26 21:09:16,904][105620] Updated weights for policy 1, policy_version 795607 (0.0009) [2023-12-26 21:09:16,921][105692] Updated weights for policy 0, policy_version 795817 (0.0010) [2023-12-26 21:09:16,954][105620] Updated weights for policy 1, policy_version 795617 (0.0008) [2023-12-26 21:09:16,984][105692] Updated weights for policy 0, policy_version 795827 (0.0005) [2023-12-26 21:09:17,049][105692] Updated weights for policy 0, policy_version 795837 (0.0005) [2023-12-26 21:09:17,117][105692] Updated weights for policy 0, policy_version 795847 (0.0005) [2023-12-26 21:09:17,649][105692] Updated weights for policy 0, policy_version 795857 (0.0008) [2023-12-26 21:09:17,699][105692] Updated weights for policy 0, policy_version 795867 (0.0008) [2023-12-26 21:09:17,752][105692] Updated weights for policy 0, policy_version 795877 (0.0005) [2023-12-26 21:09:17,820][105620] Updated weights for policy 1, policy_version 795627 (0.0009) [2023-12-26 21:09:17,889][105620] Updated weights for policy 1, policy_version 795637 (0.0010) [2023-12-26 21:09:17,953][105620] Updated weights for policy 1, policy_version 795647 (0.0009) [2023-12-26 21:09:18,412][105692] Updated weights for policy 0, policy_version 795887 (0.0008) [2023-12-26 21:09:18,479][105692] Updated weights for policy 0, policy_version 795897 (0.0009) [2023-12-26 21:09:18,532][105692] Updated weights for policy 0, policy_version 795907 (0.0009) [2023-12-26 21:09:18,715][105620] Updated weights for policy 1, policy_version 795657 (0.0009) [2023-12-26 21:09:18,770][105620] Updated weights for policy 1, policy_version 795667 (0.0009) [2023-12-26 21:09:18,825][105620] Updated weights for policy 1, policy_version 795677 (0.0009) [2023-12-26 21:09:18,891][105620] Updated weights for policy 1, policy_version 795687 (0.0009) [2023-12-26 21:09:19,277][105692] Updated weights for policy 0, policy_version 795917 (0.0009) [2023-12-26 21:09:19,340][105692] Updated weights for policy 0, policy_version 795927 (0.0008) [2023-12-26 21:09:19,399][105692] Updated weights for policy 0, policy_version 795937 (0.0008) [2023-12-26 21:09:19,705][105620] Updated weights for policy 1, policy_version 795697 (0.0006) [2023-12-26 21:09:19,768][105620] Updated weights for policy 1, policy_version 795707 (0.0005) [2023-12-26 21:09:19,826][105620] Updated weights for policy 1, policy_version 795717 (0.0009) [2023-12-26 21:09:20,210][105692] Updated weights for policy 0, policy_version 795947 (0.0009) [2023-12-26 21:09:20,274][105692] Updated weights for policy 0, policy_version 795957 (0.0009) [2023-12-26 21:09:20,333][105692] Updated weights for policy 0, policy_version 795967 (0.0009) [2023-12-26 21:09:20,567][105620] Updated weights for policy 1, policy_version 795727 (0.0009) [2023-12-26 21:09:20,642][105620] Updated weights for policy 1, policy_version 795737 (0.0009) [2023-12-26 21:09:20,705][105620] Updated weights for policy 1, policy_version 795747 (0.0009) [2023-12-26 21:09:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 407535616. Throughput: 0: 9766.7, 1: 9777.4. Samples: 407528256. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 21:09:21,062][104569] Avg episode reward: [(0, '671.259'), (1, '9262.209')] [2023-12-26 21:09:21,093][105692] Updated weights for policy 0, policy_version 795977 (0.0009) [2023-12-26 21:09:21,164][105692] Updated weights for policy 0, policy_version 795987 (0.0007) [2023-12-26 21:09:21,236][105692] Updated weights for policy 0, policy_version 795997 (0.0008) [2023-12-26 21:09:21,301][105692] Updated weights for policy 0, policy_version 796007 (0.0007) [2023-12-26 21:09:21,550][105620] Updated weights for policy 1, policy_version 795757 (0.0009) [2023-12-26 21:09:21,611][105620] Updated weights for policy 1, policy_version 795767 (0.0008) [2023-12-26 21:09:21,683][105620] Updated weights for policy 1, policy_version 795777 (0.0009) [2023-12-26 21:09:22,028][105692] Updated weights for policy 0, policy_version 796017 (0.0006) [2023-12-26 21:09:22,089][105692] Updated weights for policy 0, policy_version 796027 (0.0006) [2023-12-26 21:09:22,148][105692] Updated weights for policy 0, policy_version 796037 (0.0006) [2023-12-26 21:09:22,390][105620] Updated weights for policy 1, policy_version 795787 (0.0007) [2023-12-26 21:09:22,441][105620] Updated weights for policy 1, policy_version 795797 (0.0006) [2023-12-26 21:09:22,501][105620] Updated weights for policy 1, policy_version 795807 (0.0010) [2023-12-26 21:09:22,859][105692] Updated weights for policy 0, policy_version 796047 (0.0006) [2023-12-26 21:09:22,918][105692] Updated weights for policy 0, policy_version 796057 (0.0006) [2023-12-26 21:09:22,972][105692] Updated weights for policy 0, policy_version 796067 (0.0010) [2023-12-26 21:09:23,112][105620] Updated weights for policy 1, policy_version 795817 (0.0010) [2023-12-26 21:09:23,168][105620] Updated weights for policy 1, policy_version 795827 (0.0010) [2023-12-26 21:09:23,213][105620] Updated weights for policy 1, policy_version 795837 (0.0010) [2023-12-26 21:09:23,276][105620] Updated weights for policy 1, policy_version 795847 (0.0010) [2023-12-26 21:09:23,643][105692] Updated weights for policy 0, policy_version 796077 (0.0009) [2023-12-26 21:09:23,703][105692] Updated weights for policy 0, policy_version 796087 (0.0008) [2023-12-26 21:09:23,756][105692] Updated weights for policy 0, policy_version 796097 (0.0008) [2023-12-26 21:09:24,001][105620] Updated weights for policy 1, policy_version 795857 (0.0006) [2023-12-26 21:09:24,067][105620] Updated weights for policy 1, policy_version 795867 (0.0005) [2023-12-26 21:09:24,134][105620] Updated weights for policy 1, policy_version 795877 (0.0007) [2023-12-26 21:09:24,631][105692] Updated weights for policy 0, policy_version 796107 (0.0008) [2023-12-26 21:09:24,679][105692] Updated weights for policy 0, policy_version 796117 (0.0007) [2023-12-26 21:09:24,685][105620] Updated weights for policy 1, policy_version 795887 (0.0008) [2023-12-26 21:09:24,726][105692] Updated weights for policy 0, policy_version 796127 (0.0007) [2023-12-26 21:09:24,747][105620] Updated weights for policy 1, policy_version 795897 (0.0008) [2023-12-26 21:09:24,804][105620] Updated weights for policy 1, policy_version 795907 (0.0008) [2023-12-26 21:09:25,511][105692] Updated weights for policy 0, policy_version 796137 (0.0006) [2023-12-26 21:09:25,560][105692] Updated weights for policy 0, policy_version 796147 (0.0009) [2023-12-26 21:09:25,562][105620] Updated weights for policy 1, policy_version 795917 (0.0008) [2023-12-26 21:09:25,608][105692] Updated weights for policy 0, policy_version 796157 (0.0008) [2023-12-26 21:09:25,621][105620] Updated weights for policy 1, policy_version 795927 (0.0006) [2023-12-26 21:09:25,660][105692] Updated weights for policy 0, policy_version 796167 (0.0007) [2023-12-26 21:09:25,678][105620] Updated weights for policy 1, policy_version 795937 (0.0010) [2023-12-26 21:09:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 407633920. Throughput: 0: 9623.3, 1: 9780.7. Samples: 407642376. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 21:09:26,063][104569] Avg episode reward: [(0, '694.599'), (1, '9262.065')] [2023-12-26 21:09:26,382][105620] Updated weights for policy 1, policy_version 795947 (0.0009) [2023-12-26 21:09:26,430][105620] Updated weights for policy 1, policy_version 795957 (0.0009) [2023-12-26 21:09:26,449][105692] Updated weights for policy 0, policy_version 796177 (0.0009) [2023-12-26 21:09:26,485][105620] Updated weights for policy 1, policy_version 795967 (0.0006) [2023-12-26 21:09:26,504][105692] Updated weights for policy 0, policy_version 796187 (0.0007) [2023-12-26 21:09:26,569][105692] Updated weights for policy 0, policy_version 796197 (0.0008) [2023-12-26 21:09:27,169][105620] Updated weights for policy 1, policy_version 795977 (0.0007) [2023-12-26 21:09:27,222][105620] Updated weights for policy 1, policy_version 795987 (0.0006) [2023-12-26 21:09:27,265][105620] Updated weights for policy 1, policy_version 795997 (0.0005) [2023-12-26 21:09:27,312][105620] Updated weights for policy 1, policy_version 796007 (0.0007) [2023-12-26 21:09:27,374][105692] Updated weights for policy 0, policy_version 796207 (0.0008) [2023-12-26 21:09:27,435][105692] Updated weights for policy 0, policy_version 796217 (0.0008) [2023-12-26 21:09:27,502][105692] Updated weights for policy 0, policy_version 796227 (0.0009) [2023-12-26 21:09:27,919][105620] Updated weights for policy 1, policy_version 796017 (0.0009) [2023-12-26 21:09:27,965][105620] Updated weights for policy 1, policy_version 796027 (0.0008) [2023-12-26 21:09:28,019][105620] Updated weights for policy 1, policy_version 796037 (0.0009) [2023-12-26 21:09:28,294][105692] Updated weights for policy 0, policy_version 796237 (0.0009) [2023-12-26 21:09:28,361][105692] Updated weights for policy 0, policy_version 796247 (0.0008) [2023-12-26 21:09:28,420][105692] Updated weights for policy 0, policy_version 796257 (0.0008) [2023-12-26 21:09:28,711][105620] Updated weights for policy 1, policy_version 796047 (0.0009) [2023-12-26 21:09:28,765][105620] Updated weights for policy 1, policy_version 796057 (0.0010) [2023-12-26 21:09:28,815][105620] Updated weights for policy 1, policy_version 796067 (0.0009) [2023-12-26 21:09:29,059][105692] Updated weights for policy 0, policy_version 796267 (0.0009) [2023-12-26 21:09:29,117][105692] Updated weights for policy 0, policy_version 796277 (0.0010) [2023-12-26 21:09:29,170][105692] Updated weights for policy 0, policy_version 796287 (0.0009) [2023-12-26 21:09:29,562][105620] Updated weights for policy 1, policy_version 796077 (0.0010) [2023-12-26 21:09:29,624][105620] Updated weights for policy 1, policy_version 796087 (0.0008) [2023-12-26 21:09:29,676][105620] Updated weights for policy 1, policy_version 796097 (0.0008) [2023-12-26 21:09:29,935][105692] Updated weights for policy 0, policy_version 796297 (0.0007) [2023-12-26 21:09:29,994][105692] Updated weights for policy 0, policy_version 796307 (0.0009) [2023-12-26 21:09:30,056][105692] Updated weights for policy 0, policy_version 796317 (0.0009) [2023-12-26 21:09:30,114][105692] Updated weights for policy 0, policy_version 796327 (0.0009) [2023-12-26 21:09:30,443][105620] Updated weights for policy 1, policy_version 796107 (0.0009) [2023-12-26 21:09:30,506][105620] Updated weights for policy 1, policy_version 796117 (0.0009) [2023-12-26 21:09:30,563][105620] Updated weights for policy 1, policy_version 796127 (0.0009) [2023-12-26 21:09:30,864][105692] Updated weights for policy 0, policy_version 796337 (0.0008) [2023-12-26 21:09:30,925][105692] Updated weights for policy 0, policy_version 796347 (0.0009) [2023-12-26 21:09:30,983][105692] Updated weights for policy 0, policy_version 796357 (0.0008) [2023-12-26 21:09:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 407732224. Throughput: 0: 9570.0, 1: 9807.3. Samples: 407699484. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 21:09:31,062][104569] Avg episode reward: [(0, '838.182'), (1, '9260.420')] [2023-12-26 21:09:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000796136_203833344.pth... [2023-12-26 21:09:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000796360_203898880.pth... [2023-12-26 21:09:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000795016_203546624.pth [2023-12-26 21:09:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000795208_203603968.pth [2023-12-26 21:09:31,325][105620] Updated weights for policy 1, policy_version 796137 (0.0009) [2023-12-26 21:09:31,395][105620] Updated weights for policy 1, policy_version 796147 (0.0009) [2023-12-26 21:09:31,454][105620] Updated weights for policy 1, policy_version 796157 (0.0009) [2023-12-26 21:09:31,513][105620] Updated weights for policy 1, policy_version 796167 (0.0009) [2023-12-26 21:09:31,746][105692] Updated weights for policy 0, policy_version 796367 (0.0008) [2023-12-26 21:09:31,795][105692] Updated weights for policy 0, policy_version 796377 (0.0008) [2023-12-26 21:09:31,852][105692] Updated weights for policy 0, policy_version 796387 (0.0010) [2023-12-26 21:09:32,208][105620] Updated weights for policy 1, policy_version 796177 (0.0006) [2023-12-26 21:09:32,261][105620] Updated weights for policy 1, policy_version 796187 (0.0006) [2023-12-26 21:09:32,316][105620] Updated weights for policy 1, policy_version 796197 (0.0010) [2023-12-26 21:09:32,662][105692] Updated weights for policy 0, policy_version 796397 (0.0009) [2023-12-26 21:09:32,716][105692] Updated weights for policy 0, policy_version 796408 (0.0011) [2023-12-26 21:09:32,781][105692] Updated weights for policy 0, policy_version 796418 (0.0007) [2023-12-26 21:09:32,952][105620] Updated weights for policy 1, policy_version 796207 (0.0009) [2023-12-26 21:09:33,002][105620] Updated weights for policy 1, policy_version 796217 (0.0008) [2023-12-26 21:09:33,052][105620] Updated weights for policy 1, policy_version 796227 (0.0008) [2023-12-26 21:09:33,606][105692] Updated weights for policy 0, policy_version 796428 (0.0007) [2023-12-26 21:09:33,621][105620] Updated weights for policy 1, policy_version 796237 (0.0005) [2023-12-26 21:09:33,663][105692] Updated weights for policy 0, policy_version 796438 (0.0009) [2023-12-26 21:09:33,684][105620] Updated weights for policy 1, policy_version 796247 (0.0005) [2023-12-26 21:09:33,720][105692] Updated weights for policy 0, policy_version 796448 (0.0009) [2023-12-26 21:09:33,745][105620] Updated weights for policy 1, policy_version 796257 (0.0005) [2023-12-26 21:09:34,342][105620] Updated weights for policy 1, policy_version 796267 (0.0006) [2023-12-26 21:09:34,400][105620] Updated weights for policy 1, policy_version 796277 (0.0009) [2023-12-26 21:09:34,460][105620] Updated weights for policy 1, policy_version 796287 (0.0009) [2023-12-26 21:09:34,512][105692] Updated weights for policy 0, policy_version 796458 (0.0009) [2023-12-26 21:09:34,567][105692] Updated weights for policy 0, policy_version 796468 (0.0009) [2023-12-26 21:09:34,622][105692] Updated weights for policy 0, policy_version 796478 (0.0009) [2023-12-26 21:09:34,678][105692] Updated weights for policy 0, policy_version 796488 (0.0009) [2023-12-26 21:09:35,231][105620] Updated weights for policy 1, policy_version 796297 (0.0009) [2023-12-26 21:09:35,296][105620] Updated weights for policy 1, policy_version 796307 (0.0009) [2023-12-26 21:09:35,354][105620] Updated weights for policy 1, policy_version 796317 (0.0009) [2023-12-26 21:09:35,412][105620] Updated weights for policy 1, policy_version 796327 (0.0008) [2023-12-26 21:09:35,447][105692] Updated weights for policy 0, policy_version 796498 (0.0005) [2023-12-26 21:09:35,500][105692] Updated weights for policy 0, policy_version 796508 (0.0008) [2023-12-26 21:09:35,559][105692] Updated weights for policy 0, policy_version 796518 (0.0009) [2023-12-26 21:09:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 407822336. Throughput: 0: 9584.2, 1: 9799.1. Samples: 407815308. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 21:09:36,063][104569] Avg episode reward: [(0, '839.432'), (1, '9352.264')] [2023-12-26 21:09:36,184][105620] Updated weights for policy 1, policy_version 796337 (0.0009) [2023-12-26 21:09:36,252][105620] Updated weights for policy 1, policy_version 796347 (0.0010) [2023-12-26 21:09:36,290][105692] Updated weights for policy 0, policy_version 796528 (0.0006) [2023-12-26 21:09:36,312][105620] Updated weights for policy 1, policy_version 796357 (0.0007) [2023-12-26 21:09:36,355][105692] Updated weights for policy 0, policy_version 796538 (0.0007) [2023-12-26 21:09:36,414][105692] Updated weights for policy 0, policy_version 796548 (0.0009) [2023-12-26 21:09:37,050][105620] Updated weights for policy 1, policy_version 796367 (0.0010) [2023-12-26 21:09:37,102][105620] Updated weights for policy 1, policy_version 796377 (0.0010) [2023-12-26 21:09:37,120][105692] Updated weights for policy 0, policy_version 796558 (0.0008) [2023-12-26 21:09:37,157][105620] Updated weights for policy 1, policy_version 796387 (0.0010) [2023-12-26 21:09:37,172][105692] Updated weights for policy 0, policy_version 796568 (0.0006) [2023-12-26 21:09:37,224][105692] Updated weights for policy 0, policy_version 796578 (0.0007) [2023-12-26 21:09:37,828][105620] Updated weights for policy 1, policy_version 796397 (0.0008) [2023-12-26 21:09:37,892][105620] Updated weights for policy 1, policy_version 796407 (0.0005) [2023-12-26 21:09:37,952][105620] Updated weights for policy 1, policy_version 796417 (0.0006) [2023-12-26 21:09:38,091][105692] Updated weights for policy 0, policy_version 796588 (0.0009) [2023-12-26 21:09:38,161][105692] Updated weights for policy 0, policy_version 796598 (0.0009) [2023-12-26 21:09:38,219][105692] Updated weights for policy 0, policy_version 796608 (0.0010) [2023-12-26 21:09:38,491][105620] Updated weights for policy 1, policy_version 796427 (0.0007) [2023-12-26 21:09:38,555][105620] Updated weights for policy 1, policy_version 796437 (0.0008) [2023-12-26 21:09:38,622][105620] Updated weights for policy 1, policy_version 796447 (0.0009) [2023-12-26 21:09:39,098][105692] Updated weights for policy 0, policy_version 796618 (0.0010) [2023-12-26 21:09:39,169][105692] Updated weights for policy 0, policy_version 796628 (0.0010) [2023-12-26 21:09:39,245][105620] Updated weights for policy 1, policy_version 796457 (0.0010) [2023-12-26 21:09:39,247][105692] Updated weights for policy 0, policy_version 796638 (0.0009) [2023-12-26 21:09:39,293][105692] Updated weights for policy 0, policy_version 796648 (0.0007) [2023-12-26 21:09:39,297][105620] Updated weights for policy 1, policy_version 796467 (0.0008) [2023-12-26 21:09:39,366][105620] Updated weights for policy 1, policy_version 796477 (0.0008) [2023-12-26 21:09:39,432][105620] Updated weights for policy 1, policy_version 796487 (0.0008) [2023-12-26 21:09:40,097][105692] Updated weights for policy 0, policy_version 796658 (0.0006) [2023-12-26 21:09:40,135][105620] Updated weights for policy 1, policy_version 796497 (0.0008) [2023-12-26 21:09:40,161][105692] Updated weights for policy 0, policy_version 796668 (0.0006) [2023-12-26 21:09:40,196][105620] Updated weights for policy 1, policy_version 796507 (0.0007) [2023-12-26 21:09:40,214][105692] Updated weights for policy 0, policy_version 796678 (0.0009) [2023-12-26 21:09:40,262][105620] Updated weights for policy 1, policy_version 796517 (0.0006) [2023-12-26 21:09:40,903][105620] Updated weights for policy 1, policy_version 796527 (0.0008) [2023-12-26 21:09:40,969][105620] Updated weights for policy 1, policy_version 796537 (0.0006) [2023-12-26 21:09:41,000][105692] Updated weights for policy 0, policy_version 796688 (0.0010) [2023-12-26 21:09:41,030][105620] Updated weights for policy 1, policy_version 796547 (0.0011) [2023-12-26 21:09:41,062][105692] Updated weights for policy 0, policy_version 796698 (0.0010) [2023-12-26 21:09:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 407920640. Throughput: 0: 9419.4, 1: 9798.7. Samples: 407929552. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 21:09:41,062][104569] Avg episode reward: [(0, '920.185'), (1, '8946.459')] [2023-12-26 21:09:41,122][105692] Updated weights for policy 0, policy_version 796708 (0.0011) [2023-12-26 21:09:41,711][105620] Updated weights for policy 1, policy_version 796557 (0.0010) [2023-12-26 21:09:41,770][105620] Updated weights for policy 1, policy_version 796567 (0.0012) [2023-12-26 21:09:41,829][105620] Updated weights for policy 1, policy_version 796577 (0.0006) [2023-12-26 21:09:41,926][105692] Updated weights for policy 0, policy_version 796718 (0.0009) [2023-12-26 21:09:41,981][105692] Updated weights for policy 0, policy_version 796728 (0.0008) [2023-12-26 21:09:42,043][105692] Updated weights for policy 0, policy_version 796738 (0.0009) [2023-12-26 21:09:42,576][105620] Updated weights for policy 1, policy_version 796587 (0.0008) [2023-12-26 21:09:42,632][105620] Updated weights for policy 1, policy_version 796597 (0.0006) [2023-12-26 21:09:42,691][105620] Updated weights for policy 1, policy_version 796607 (0.0006) [2023-12-26 21:09:42,719][105692] Updated weights for policy 0, policy_version 796748 (0.0008) [2023-12-26 21:09:42,776][105692] Updated weights for policy 0, policy_version 796758 (0.0005) [2023-12-26 21:09:42,832][105692] Updated weights for policy 0, policy_version 796768 (0.0005) [2023-12-26 21:09:43,347][105620] Updated weights for policy 1, policy_version 796617 (0.0008) [2023-12-26 21:09:43,403][105620] Updated weights for policy 1, policy_version 796627 (0.0010) [2023-12-26 21:09:43,460][105620] Updated weights for policy 1, policy_version 796637 (0.0009) [2023-12-26 21:09:43,518][105620] Updated weights for policy 1, policy_version 796647 (0.0010) [2023-12-26 21:09:43,538][105692] Updated weights for policy 0, policy_version 796778 (0.0008) [2023-12-26 21:09:43,598][105692] Updated weights for policy 0, policy_version 796788 (0.0008) [2023-12-26 21:09:43,647][105692] Updated weights for policy 0, policy_version 796798 (0.0008) [2023-12-26 21:09:43,699][105692] Updated weights for policy 0, policy_version 796808 (0.0008) [2023-12-26 21:09:44,218][105620] Updated weights for policy 1, policy_version 796657 (0.0006) [2023-12-26 21:09:44,274][105620] Updated weights for policy 1, policy_version 796667 (0.0005) [2023-12-26 21:09:44,338][105620] Updated weights for policy 1, policy_version 796677 (0.0006) [2023-12-26 21:09:44,365][105692] Updated weights for policy 0, policy_version 796818 (0.0005) [2023-12-26 21:09:44,426][105692] Updated weights for policy 0, policy_version 796828 (0.0008) [2023-12-26 21:09:44,485][105692] Updated weights for policy 0, policy_version 796838 (0.0010) [2023-12-26 21:09:44,848][105620] Updated weights for policy 1, policy_version 796687 (0.0006) [2023-12-26 21:09:44,906][105620] Updated weights for policy 1, policy_version 796697 (0.0009) [2023-12-26 21:09:44,955][105620] Updated weights for policy 1, policy_version 796707 (0.0010) [2023-12-26 21:09:45,085][105692] Updated weights for policy 0, policy_version 796848 (0.0010) [2023-12-26 21:09:45,152][105692] Updated weights for policy 0, policy_version 796858 (0.0011) [2023-12-26 21:09:45,214][105692] Updated weights for policy 0, policy_version 796868 (0.0011) [2023-12-26 21:09:45,609][105620] Updated weights for policy 1, policy_version 796717 (0.0008) [2023-12-26 21:09:45,672][105620] Updated weights for policy 1, policy_version 796727 (0.0009) [2023-12-26 21:09:45,726][105620] Updated weights for policy 1, policy_version 796737 (0.0010) [2023-12-26 21:09:45,828][105692] Updated weights for policy 0, policy_version 796878 (0.0007) [2023-12-26 21:09:45,885][105692] Updated weights for policy 0, policy_version 796888 (0.0005) [2023-12-26 21:09:45,947][105692] Updated weights for policy 0, policy_version 796898 (0.0007) [2023-12-26 21:09:46,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 408027136. Throughput: 0: 9437.1, 1: 9812.3. Samples: 407987184. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 21:09:46,062][104569] Avg episode reward: [(0, '1010.191'), (1, '8846.401')] [2023-12-26 21:09:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000796744_203988992.pth... [2023-12-26 21:09:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000796904_204038144.pth... [2023-12-26 21:09:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000795592_203694080.pth [2023-12-26 21:09:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000795784_203751424.pth [2023-12-26 21:09:46,445][105620] Updated weights for policy 1, policy_version 796747 (0.0010) [2023-12-26 21:09:46,512][105620] Updated weights for policy 1, policy_version 796757 (0.0010) [2023-12-26 21:09:46,574][105620] Updated weights for policy 1, policy_version 796767 (0.0010) [2023-12-26 21:09:46,653][105692] Updated weights for policy 0, policy_version 796908 (0.0010) [2023-12-26 21:09:46,717][105692] Updated weights for policy 0, policy_version 796918 (0.0009) [2023-12-26 21:09:46,774][105692] Updated weights for policy 0, policy_version 796928 (0.0007) [2023-12-26 21:09:47,301][105620] Updated weights for policy 1, policy_version 796777 (0.0010) [2023-12-26 21:09:47,361][105620] Updated weights for policy 1, policy_version 796787 (0.0005) [2023-12-26 21:09:47,403][105692] Updated weights for policy 0, policy_version 796938 (0.0006) [2023-12-26 21:09:47,419][105620] Updated weights for policy 1, policy_version 796797 (0.0006) [2023-12-26 21:09:47,453][105692] Updated weights for policy 0, policy_version 796948 (0.0010) [2023-12-26 21:09:47,475][105620] Updated weights for policy 1, policy_version 796807 (0.0005) [2023-12-26 21:09:47,504][105692] Updated weights for policy 0, policy_version 796958 (0.0010) [2023-12-26 21:09:47,559][105692] Updated weights for policy 0, policy_version 796968 (0.0010) [2023-12-26 21:09:47,993][105620] Updated weights for policy 1, policy_version 796817 (0.0006) [2023-12-26 21:09:48,062][105620] Updated weights for policy 1, policy_version 796827 (0.0005) [2023-12-26 21:09:48,126][105620] Updated weights for policy 1, policy_version 796837 (0.0007) [2023-12-26 21:09:48,314][105692] Updated weights for policy 0, policy_version 796978 (0.0010) [2023-12-26 21:09:48,377][105692] Updated weights for policy 0, policy_version 796988 (0.0007) [2023-12-26 21:09:48,447][105692] Updated weights for policy 0, policy_version 796998 (0.0007) [2023-12-26 21:09:48,765][105620] Updated weights for policy 1, policy_version 796847 (0.0011) [2023-12-26 21:09:48,818][105620] Updated weights for policy 1, policy_version 796857 (0.0011) [2023-12-26 21:09:48,870][105620] Updated weights for policy 1, policy_version 796867 (0.0011) [2023-12-26 21:09:49,040][105692] Updated weights for policy 0, policy_version 797008 (0.0005) [2023-12-26 21:09:49,095][105692] Updated weights for policy 0, policy_version 797018 (0.0007) [2023-12-26 21:09:49,146][105692] Updated weights for policy 0, policy_version 797028 (0.0010) [2023-12-26 21:09:49,638][105620] Updated weights for policy 1, policy_version 796877 (0.0011) [2023-12-26 21:09:49,687][105620] Updated weights for policy 1, policy_version 796887 (0.0010) [2023-12-26 21:09:49,739][105620] Updated weights for policy 1, policy_version 796897 (0.0011) [2023-12-26 21:09:49,876][105692] Updated weights for policy 0, policy_version 797038 (0.0009) [2023-12-26 21:09:49,941][105692] Updated weights for policy 0, policy_version 797048 (0.0007) [2023-12-26 21:09:50,006][105692] Updated weights for policy 0, policy_version 797058 (0.0006) [2023-12-26 21:09:50,483][105620] Updated weights for policy 1, policy_version 796907 (0.0010) [2023-12-26 21:09:50,551][105620] Updated weights for policy 1, policy_version 796917 (0.0008) [2023-12-26 21:09:50,619][105620] Updated weights for policy 1, policy_version 796927 (0.0008) [2023-12-26 21:09:50,787][105692] Updated weights for policy 0, policy_version 797068 (0.0009) [2023-12-26 21:09:50,849][105692] Updated weights for policy 0, policy_version 797078 (0.0008) [2023-12-26 21:09:50,908][105692] Updated weights for policy 0, policy_version 797088 (0.0009) [2023-12-26 21:09:51,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 408125440. Throughput: 0: 9632.1, 1: 9886.6. Samples: 408112984. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 21:09:51,063][104569] Avg episode reward: [(0, '1123.754'), (1, '8849.021')] [2023-12-26 21:09:51,391][105620] Updated weights for policy 1, policy_version 796937 (0.0008) [2023-12-26 21:09:51,461][105620] Updated weights for policy 1, policy_version 796947 (0.0009) [2023-12-26 21:09:51,532][105620] Updated weights for policy 1, policy_version 796957 (0.0006) [2023-12-26 21:09:51,605][105620] Updated weights for policy 1, policy_version 796967 (0.0006) [2023-12-26 21:09:51,668][105692] Updated weights for policy 0, policy_version 797098 (0.0009) [2023-12-26 21:09:51,748][105692] Updated weights for policy 0, policy_version 797108 (0.0008) [2023-12-26 21:09:51,817][105692] Updated weights for policy 0, policy_version 797118 (0.0008) [2023-12-26 21:09:51,888][105692] Updated weights for policy 0, policy_version 797128 (0.0007) [2023-12-26 21:09:52,234][105620] Updated weights for policy 1, policy_version 796977 (0.0009) [2023-12-26 21:09:52,301][105620] Updated weights for policy 1, policy_version 796987 (0.0009) [2023-12-26 21:09:52,383][105620] Updated weights for policy 1, policy_version 796997 (0.0007) [2023-12-26 21:09:52,647][105692] Updated weights for policy 0, policy_version 797138 (0.0008) [2023-12-26 21:09:52,702][105692] Updated weights for policy 0, policy_version 797148 (0.0009) [2023-12-26 21:09:52,756][105692] Updated weights for policy 0, policy_version 797158 (0.0009) [2023-12-26 21:09:53,059][105620] Updated weights for policy 1, policy_version 797007 (0.0009) [2023-12-26 21:09:53,118][105620] Updated weights for policy 1, policy_version 797017 (0.0009) [2023-12-26 21:09:53,179][105620] Updated weights for policy 1, policy_version 797027 (0.0009) [2023-12-26 21:09:53,542][105692] Updated weights for policy 0, policy_version 797168 (0.0009) [2023-12-26 21:09:53,590][105692] Updated weights for policy 0, policy_version 797178 (0.0009) [2023-12-26 21:09:53,644][105692] Updated weights for policy 0, policy_version 797188 (0.0009) [2023-12-26 21:09:53,930][105620] Updated weights for policy 1, policy_version 797037 (0.0009) [2023-12-26 21:09:53,992][105620] Updated weights for policy 1, policy_version 797047 (0.0009) [2023-12-26 21:09:54,054][105620] Updated weights for policy 1, policy_version 797057 (0.0008) [2023-12-26 21:09:54,436][105692] Updated weights for policy 0, policy_version 797198 (0.0008) [2023-12-26 21:09:54,499][105692] Updated weights for policy 0, policy_version 797208 (0.0009) [2023-12-26 21:09:54,562][105692] Updated weights for policy 0, policy_version 797218 (0.0009) [2023-12-26 21:09:54,822][105620] Updated weights for policy 1, policy_version 797067 (0.0009) [2023-12-26 21:09:54,870][105620] Updated weights for policy 1, policy_version 797077 (0.0009) [2023-12-26 21:09:54,929][105620] Updated weights for policy 1, policy_version 797087 (0.0009) [2023-12-26 21:09:55,316][105692] Updated weights for policy 0, policy_version 797228 (0.0009) [2023-12-26 21:09:55,376][105692] Updated weights for policy 0, policy_version 797238 (0.0008) [2023-12-26 21:09:55,427][105692] Updated weights for policy 0, policy_version 797248 (0.0009) [2023-12-26 21:09:55,693][105620] Updated weights for policy 1, policy_version 797097 (0.0009) [2023-12-26 21:09:55,749][105620] Updated weights for policy 1, policy_version 797107 (0.0008) [2023-12-26 21:09:55,800][105620] Updated weights for policy 1, policy_version 797117 (0.0008) [2023-12-26 21:09:55,861][105620] Updated weights for policy 1, policy_version 797127 (0.0008) [2023-12-26 21:09:56,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 408215552. Throughput: 0: 9488.7, 1: 9817.7. Samples: 408223492. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 21:09:56,062][104569] Avg episode reward: [(0, '880.922'), (1, '9349.510')] [2023-12-26 21:09:56,210][105692] Updated weights for policy 0, policy_version 797258 (0.0009) [2023-12-26 21:09:56,264][105692] Updated weights for policy 0, policy_version 797268 (0.0009) [2023-12-26 21:09:56,314][105692] Updated weights for policy 0, policy_version 797278 (0.0009) [2023-12-26 21:09:56,361][105692] Updated weights for policy 0, policy_version 797288 (0.0009) [2023-12-26 21:09:56,572][105620] Updated weights for policy 1, policy_version 797137 (0.0009) [2023-12-26 21:09:56,629][105620] Updated weights for policy 1, policy_version 797147 (0.0009) [2023-12-26 21:09:56,688][105620] Updated weights for policy 1, policy_version 797157 (0.0008) [2023-12-26 21:09:57,171][105692] Updated weights for policy 0, policy_version 797298 (0.0008) [2023-12-26 21:09:57,230][105692] Updated weights for policy 0, policy_version 797308 (0.0008) [2023-12-26 21:09:57,288][105692] Updated weights for policy 0, policy_version 797318 (0.0008) [2023-12-26 21:09:57,364][105620] Updated weights for policy 1, policy_version 797167 (0.0009) [2023-12-26 21:09:57,417][105620] Updated weights for policy 1, policy_version 797177 (0.0008) [2023-12-26 21:09:57,470][105620] Updated weights for policy 1, policy_version 797187 (0.0008) [2023-12-26 21:09:58,042][105692] Updated weights for policy 0, policy_version 797328 (0.0009) [2023-12-26 21:09:58,095][105692] Updated weights for policy 0, policy_version 797339 (0.0010) [2023-12-26 21:09:58,153][105692] Updated weights for policy 0, policy_version 797350 (0.0010) [2023-12-26 21:09:58,170][105620] Updated weights for policy 1, policy_version 797197 (0.0009) [2023-12-26 21:09:58,225][105620] Updated weights for policy 1, policy_version 797207 (0.0009) [2023-12-26 21:09:58,284][105620] Updated weights for policy 1, policy_version 797217 (0.0009) [2023-12-26 21:09:58,973][105692] Updated weights for policy 0, policy_version 797360 (0.0008) [2023-12-26 21:09:59,025][105692] Updated weights for policy 0, policy_version 797370 (0.0008) [2023-12-26 21:09:59,073][105692] Updated weights for policy 0, policy_version 797380 (0.0007) [2023-12-26 21:09:59,086][105620] Updated weights for policy 1, policy_version 797227 (0.0009) [2023-12-26 21:09:59,157][105620] Updated weights for policy 1, policy_version 797237 (0.0007) [2023-12-26 21:09:59,214][105620] Updated weights for policy 1, policy_version 797247 (0.0007) [2023-12-26 21:09:59,869][105620] Updated weights for policy 1, policy_version 797257 (0.0008) [2023-12-26 21:09:59,903][105692] Updated weights for policy 0, policy_version 797390 (0.0009) [2023-12-26 21:09:59,921][105620] Updated weights for policy 1, policy_version 797267 (0.0006) [2023-12-26 21:09:59,968][105692] Updated weights for policy 0, policy_version 797400 (0.0008) [2023-12-26 21:09:59,988][105620] Updated weights for policy 1, policy_version 797277 (0.0008) [2023-12-26 21:10:00,026][105692] Updated weights for policy 0, policy_version 797410 (0.0006) [2023-12-26 21:10:00,050][105620] Updated weights for policy 1, policy_version 797287 (0.0008) [2023-12-26 21:10:00,594][105692] Updated weights for policy 0, policy_version 797420 (0.0006) [2023-12-26 21:10:00,651][105692] Updated weights for policy 0, policy_version 797430 (0.0008) [2023-12-26 21:10:00,707][105692] Updated weights for policy 0, policy_version 797442 (0.0007) [2023-12-26 21:10:00,880][105620] Updated weights for policy 1, policy_version 797297 (0.0010) [2023-12-26 21:10:00,942][105620] Updated weights for policy 1, policy_version 797308 (0.0010) [2023-12-26 21:10:00,993][105620] Updated weights for policy 1, policy_version 797319 (0.0010) [2023-12-26 21:10:01,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19438.7). Total num frames: 408313856. Throughput: 0: 9464.5, 1: 9842.0. Samples: 408279628. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 21:10:01,062][104569] Avg episode reward: [(0, '520.855'), (1, '9349.041')] [2023-12-26 21:10:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000797448_204177408.pth... [2023-12-26 21:10:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000797320_204136448.pth... [2023-12-26 21:10:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000796136_203833344.pth [2023-12-26 21:10:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000796360_203898880.pth [2023-12-26 21:10:01,376][105692] Updated weights for policy 0, policy_version 797452 (0.0008) [2023-12-26 21:10:01,442][105692] Updated weights for policy 0, policy_version 797462 (0.0006) [2023-12-26 21:10:01,495][105692] Updated weights for policy 0, policy_version 797472 (0.0005) [2023-12-26 21:10:01,851][105620] Updated weights for policy 1, policy_version 797329 (0.0008) [2023-12-26 21:10:01,907][105620] Updated weights for policy 1, policy_version 797339 (0.0008) [2023-12-26 21:10:01,961][105620] Updated weights for policy 1, policy_version 797349 (0.0008) [2023-12-26 21:10:02,163][105692] Updated weights for policy 0, policy_version 797482 (0.0006) [2023-12-26 21:10:02,224][105692] Updated weights for policy 0, policy_version 797492 (0.0010) [2023-12-26 21:10:02,288][105692] Updated weights for policy 0, policy_version 797502 (0.0009) [2023-12-26 21:10:02,342][105692] Updated weights for policy 0, policy_version 797512 (0.0005) [2023-12-26 21:10:02,749][105620] Updated weights for policy 1, policy_version 797359 (0.0006) [2023-12-26 21:10:02,808][105620] Updated weights for policy 1, policy_version 797369 (0.0006) [2023-12-26 21:10:02,862][105620] Updated weights for policy 1, policy_version 797379 (0.0006) [2023-12-26 21:10:03,043][105692] Updated weights for policy 0, policy_version 797522 (0.0005) [2023-12-26 21:10:03,107][105692] Updated weights for policy 0, policy_version 797532 (0.0006) [2023-12-26 21:10:03,173][105692] Updated weights for policy 0, policy_version 797542 (0.0006) [2023-12-26 21:10:03,476][105620] Updated weights for policy 1, policy_version 797389 (0.0007) [2023-12-26 21:10:03,530][105620] Updated weights for policy 1, policy_version 797399 (0.0009) [2023-12-26 21:10:03,585][105620] Updated weights for policy 1, policy_version 797409 (0.0009) [2023-12-26 21:10:03,751][105692] Updated weights for policy 0, policy_version 797552 (0.0005) [2023-12-26 21:10:03,797][105692] Updated weights for policy 0, policy_version 797562 (0.0005) [2023-12-26 21:10:03,850][105692] Updated weights for policy 0, policy_version 797572 (0.0006) [2023-12-26 21:10:04,461][105620] Updated weights for policy 1, policy_version 797419 (0.0009) [2023-12-26 21:10:04,467][105692] Updated weights for policy 0, policy_version 797582 (0.0007) [2023-12-26 21:10:04,521][105692] Updated weights for policy 0, policy_version 797592 (0.0006) [2023-12-26 21:10:04,529][105620] Updated weights for policy 1, policy_version 797429 (0.0008) [2023-12-26 21:10:04,585][105620] Updated weights for policy 1, policy_version 797439 (0.0009) [2023-12-26 21:10:04,588][105692] Updated weights for policy 0, policy_version 797602 (0.0006) [2023-12-26 21:10:05,208][105692] Updated weights for policy 0, policy_version 797612 (0.0008) [2023-12-26 21:10:05,259][105692] Updated weights for policy 0, policy_version 797622 (0.0005) [2023-12-26 21:10:05,324][105692] Updated weights for policy 0, policy_version 797632 (0.0006) [2023-12-26 21:10:05,415][105620] Updated weights for policy 1, policy_version 797449 (0.0007) [2023-12-26 21:10:05,466][105620] Updated weights for policy 1, policy_version 797459 (0.0009) [2023-12-26 21:10:05,511][105620] Updated weights for policy 1, policy_version 797469 (0.0008) [2023-12-26 21:10:05,559][105620] Updated weights for policy 1, policy_version 797479 (0.0009) [2023-12-26 21:10:06,042][105692] Updated weights for policy 0, policy_version 797642 (0.0009) [2023-12-26 21:10:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 408403968. Throughput: 0: 9446.0, 1: 9858.0. Samples: 408396936. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 21:10:06,062][104569] Avg episode reward: [(0, '620.200'), (1, '9349.932')] [2023-12-26 21:10:06,120][105692] Updated weights for policy 0, policy_version 797652 (0.0009) [2023-12-26 21:10:06,182][105692] Updated weights for policy 0, policy_version 797663 (0.0010) [2023-12-26 21:10:06,308][105620] Updated weights for policy 1, policy_version 797489 (0.0009) [2023-12-26 21:10:06,360][105620] Updated weights for policy 1, policy_version 797499 (0.0009) [2023-12-26 21:10:06,412][105620] Updated weights for policy 1, policy_version 797509 (0.0009) [2023-12-26 21:10:06,846][105692] Updated weights for policy 0, policy_version 797673 (0.0008) [2023-12-26 21:10:06,906][105692] Updated weights for policy 0, policy_version 797683 (0.0006) [2023-12-26 21:10:06,961][105692] Updated weights for policy 0, policy_version 797693 (0.0007) [2023-12-26 21:10:07,023][105692] Updated weights for policy 0, policy_version 797703 (0.0005) [2023-12-26 21:10:07,222][105620] Updated weights for policy 1, policy_version 797519 (0.0007) [2023-12-26 21:10:07,275][105620] Updated weights for policy 1, policy_version 797529 (0.0005) [2023-12-26 21:10:07,329][105620] Updated weights for policy 1, policy_version 797539 (0.0008) [2023-12-26 21:10:07,639][105692] Updated weights for policy 0, policy_version 797713 (0.0010) [2023-12-26 21:10:07,697][105692] Updated weights for policy 0, policy_version 797723 (0.0010) [2023-12-26 21:10:07,758][105692] Updated weights for policy 0, policy_version 797733 (0.0009) [2023-12-26 21:10:07,934][105620] Updated weights for policy 1, policy_version 797549 (0.0008) [2023-12-26 21:10:07,991][105620] Updated weights for policy 1, policy_version 797559 (0.0010) [2023-12-26 21:10:08,051][105620] Updated weights for policy 1, policy_version 797569 (0.0010) [2023-12-26 21:10:08,557][105692] Updated weights for policy 0, policy_version 797743 (0.0010) [2023-12-26 21:10:08,618][105692] Updated weights for policy 0, policy_version 797753 (0.0010) [2023-12-26 21:10:08,676][105692] Updated weights for policy 0, policy_version 797763 (0.0010) [2023-12-26 21:10:08,699][105620] Updated weights for policy 1, policy_version 797579 (0.0009) [2023-12-26 21:10:08,757][105620] Updated weights for policy 1, policy_version 797589 (0.0005) [2023-12-26 21:10:08,819][105620] Updated weights for policy 1, policy_version 797599 (0.0005) [2023-12-26 21:10:09,424][105620] Updated weights for policy 1, policy_version 797609 (0.0007) [2023-12-26 21:10:09,477][105620] Updated weights for policy 1, policy_version 797619 (0.0011) [2023-12-26 21:10:09,526][105620] Updated weights for policy 1, policy_version 797629 (0.0011) [2023-12-26 21:10:09,545][105692] Updated weights for policy 0, policy_version 797773 (0.0008) [2023-12-26 21:10:09,589][105620] Updated weights for policy 1, policy_version 797639 (0.0011) [2023-12-26 21:10:09,608][105692] Updated weights for policy 0, policy_version 797783 (0.0007) [2023-12-26 21:10:09,669][105692] Updated weights for policy 0, policy_version 797793 (0.0008) [2023-12-26 21:10:10,359][105620] Updated weights for policy 1, policy_version 797649 (0.0008) [2023-12-26 21:10:10,418][105620] Updated weights for policy 1, policy_version 797659 (0.0009) [2023-12-26 21:10:10,457][105692] Updated weights for policy 0, policy_version 797803 (0.0008) [2023-12-26 21:10:10,475][105620] Updated weights for policy 1, policy_version 797669 (0.0006) [2023-12-26 21:10:10,511][105692] Updated weights for policy 0, policy_version 797813 (0.0009) [2023-12-26 21:10:10,566][105692] Updated weights for policy 0, policy_version 797824 (0.0010) [2023-12-26 21:10:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 408502272. Throughput: 0: 9471.1, 1: 9881.6. Samples: 408513248. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 21:10:11,062][104569] Avg episode reward: [(0, '792.686'), (1, '9076.607')] [2023-12-26 21:10:11,165][105620] Updated weights for policy 1, policy_version 797679 (0.0009) [2023-12-26 21:10:11,227][105620] Updated weights for policy 1, policy_version 797689 (0.0010) [2023-12-26 21:10:11,259][105692] Updated weights for policy 0, policy_version 797834 (0.0009) [2023-12-26 21:10:11,295][105620] Updated weights for policy 1, policy_version 797699 (0.0011) [2023-12-26 21:10:11,322][105692] Updated weights for policy 0, policy_version 797844 (0.0007) [2023-12-26 21:10:11,391][105692] Updated weights for policy 0, policy_version 797854 (0.0009) [2023-12-26 21:10:11,450][105692] Updated weights for policy 0, policy_version 797864 (0.0010) [2023-12-26 21:10:11,978][105620] Updated weights for policy 1, policy_version 797709 (0.0008) [2023-12-26 21:10:12,047][105620] Updated weights for policy 1, policy_version 797719 (0.0006) [2023-12-26 21:10:12,120][105620] Updated weights for policy 1, policy_version 797729 (0.0006) [2023-12-26 21:10:12,219][105692] Updated weights for policy 0, policy_version 797874 (0.0009) [2023-12-26 21:10:12,278][105692] Updated weights for policy 0, policy_version 797884 (0.0009) [2023-12-26 21:10:12,342][105692] Updated weights for policy 0, policy_version 797894 (0.0010) [2023-12-26 21:10:12,674][105620] Updated weights for policy 1, policy_version 797739 (0.0008) [2023-12-26 21:10:12,731][105620] Updated weights for policy 1, policy_version 797749 (0.0005) [2023-12-26 21:10:12,795][105620] Updated weights for policy 1, policy_version 797759 (0.0009) [2023-12-26 21:10:13,176][105692] Updated weights for policy 0, policy_version 797904 (0.0009) [2023-12-26 21:10:13,237][105692] Updated weights for policy 0, policy_version 797914 (0.0010) [2023-12-26 21:10:13,290][105692] Updated weights for policy 0, policy_version 797924 (0.0010) [2023-12-26 21:10:13,445][105620] Updated weights for policy 1, policy_version 797769 (0.0008) [2023-12-26 21:10:13,491][105620] Updated weights for policy 1, policy_version 797779 (0.0005) [2023-12-26 21:10:13,543][105620] Updated weights for policy 1, policy_version 797789 (0.0007) [2023-12-26 21:10:13,594][105620] Updated weights for policy 1, policy_version 797800 (0.0009) [2023-12-26 21:10:13,998][105692] Updated weights for policy 0, policy_version 797934 (0.0009) [2023-12-26 21:10:14,061][105692] Updated weights for policy 0, policy_version 797944 (0.0009) [2023-12-26 21:10:14,112][105692] Updated weights for policy 0, policy_version 797954 (0.0008) [2023-12-26 21:10:14,365][105620] Updated weights for policy 1, policy_version 797810 (0.0009) [2023-12-26 21:10:14,420][105620] Updated weights for policy 1, policy_version 797820 (0.0009) [2023-12-26 21:10:14,474][105620] Updated weights for policy 1, policy_version 797830 (0.0009) [2023-12-26 21:10:14,771][105692] Updated weights for policy 0, policy_version 797964 (0.0008) [2023-12-26 21:10:14,833][105692] Updated weights for policy 0, policy_version 797974 (0.0009) [2023-12-26 21:10:14,893][105692] Updated weights for policy 0, policy_version 797984 (0.0009) [2023-12-26 21:10:15,261][105620] Updated weights for policy 1, policy_version 797840 (0.0009) [2023-12-26 21:10:15,312][105620] Updated weights for policy 1, policy_version 797850 (0.0009) [2023-12-26 21:10:15,364][105620] Updated weights for policy 1, policy_version 797860 (0.0009) [2023-12-26 21:10:15,655][105692] Updated weights for policy 0, policy_version 797994 (0.0009) [2023-12-26 21:10:15,720][105692] Updated weights for policy 0, policy_version 798004 (0.0009) [2023-12-26 21:10:15,782][105692] Updated weights for policy 0, policy_version 798014 (0.0009) [2023-12-26 21:10:15,839][105692] Updated weights for policy 0, policy_version 798024 (0.0009) [2023-12-26 21:10:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 408600576. Throughput: 0: 9500.6, 1: 9890.5. Samples: 408572084. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 21:10:16,062][104569] Avg episode reward: [(0, '917.023'), (1, '9077.111')] [2023-12-26 21:10:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000798024_204324864.pth... [2023-12-26 21:10:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000797864_204275712.pth... [2023-12-26 21:10:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000796904_204038144.pth [2023-12-26 21:10:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000796744_203988992.pth [2023-12-26 21:10:16,131][105620] Updated weights for policy 1, policy_version 797870 (0.0009) [2023-12-26 21:10:16,190][105620] Updated weights for policy 1, policy_version 797880 (0.0009) [2023-12-26 21:10:16,248][105620] Updated weights for policy 1, policy_version 797890 (0.0009) [2023-12-26 21:10:16,565][105692] Updated weights for policy 0, policy_version 798034 (0.0008) [2023-12-26 21:10:16,627][105692] Updated weights for policy 0, policy_version 798044 (0.0007) [2023-12-26 21:10:16,682][105692] Updated weights for policy 0, policy_version 798054 (0.0010) [2023-12-26 21:10:16,987][105620] Updated weights for policy 1, policy_version 797900 (0.0008) [2023-12-26 21:10:17,046][105620] Updated weights for policy 1, policy_version 797910 (0.0006) [2023-12-26 21:10:17,097][105620] Updated weights for policy 1, policy_version 797920 (0.0005) [2023-12-26 21:10:17,398][105692] Updated weights for policy 0, policy_version 798064 (0.0011) [2023-12-26 21:10:17,457][105692] Updated weights for policy 0, policy_version 798074 (0.0008) [2023-12-26 21:10:17,516][105692] Updated weights for policy 0, policy_version 798084 (0.0005) [2023-12-26 21:10:17,725][105620] Updated weights for policy 1, policy_version 797930 (0.0009) [2023-12-26 21:10:17,780][105620] Updated weights for policy 1, policy_version 797940 (0.0010) [2023-12-26 21:10:17,838][105620] Updated weights for policy 1, policy_version 797950 (0.0011) [2023-12-26 21:10:17,890][105620] Updated weights for policy 1, policy_version 797960 (0.0011) [2023-12-26 21:10:18,047][105692] Updated weights for policy 0, policy_version 798094 (0.0005) [2023-12-26 21:10:18,116][105692] Updated weights for policy 0, policy_version 798104 (0.0006) [2023-12-26 21:10:18,181][105692] Updated weights for policy 0, policy_version 798114 (0.0006) [2023-12-26 21:10:18,554][105620] Updated weights for policy 1, policy_version 797970 (0.0005) [2023-12-26 21:10:18,616][105620] Updated weights for policy 1, policy_version 797980 (0.0006) [2023-12-26 21:10:18,675][105620] Updated weights for policy 1, policy_version 797990 (0.0005) [2023-12-26 21:10:18,888][105692] Updated weights for policy 0, policy_version 798124 (0.0010) [2023-12-26 21:10:18,951][105692] Updated weights for policy 0, policy_version 798134 (0.0011) [2023-12-26 21:10:19,007][105692] Updated weights for policy 0, policy_version 798144 (0.0010) [2023-12-26 21:10:19,275][105620] Updated weights for policy 1, policy_version 798000 (0.0010) [2023-12-26 21:10:19,341][105620] Updated weights for policy 1, policy_version 798010 (0.0012) [2023-12-26 21:10:19,410][105620] Updated weights for policy 1, policy_version 798020 (0.0010) [2023-12-26 21:10:19,692][105692] Updated weights for policy 0, policy_version 798154 (0.0009) [2023-12-26 21:10:19,748][105692] Updated weights for policy 0, policy_version 798164 (0.0007) [2023-12-26 21:10:19,804][105692] Updated weights for policy 0, policy_version 798174 (0.0005) [2023-12-26 21:10:19,866][105692] Updated weights for policy 0, policy_version 798184 (0.0011) [2023-12-26 21:10:20,196][105620] Updated weights for policy 1, policy_version 798030 (0.0011) [2023-12-26 21:10:20,256][105620] Updated weights for policy 1, policy_version 798040 (0.0011) [2023-12-26 21:10:20,302][105620] Updated weights for policy 1, policy_version 798050 (0.0010) [2023-12-26 21:10:20,542][105692] Updated weights for policy 0, policy_version 798194 (0.0008) [2023-12-26 21:10:20,603][105692] Updated weights for policy 0, policy_version 798204 (0.0012) [2023-12-26 21:10:20,665][105692] Updated weights for policy 0, policy_version 798214 (0.0008) [2023-12-26 21:10:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 408698880. Throughput: 0: 9612.1, 1: 9850.0. Samples: 408691104. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 21:10:21,063][104569] Avg episode reward: [(0, '615.444'), (1, '9258.777')] [2023-12-26 21:10:21,075][105620] Updated weights for policy 1, policy_version 798060 (0.0009) [2023-12-26 21:10:21,142][105620] Updated weights for policy 1, policy_version 798070 (0.0008) [2023-12-26 21:10:21,201][105620] Updated weights for policy 1, policy_version 798080 (0.0010) [2023-12-26 21:10:21,485][105692] Updated weights for policy 0, policy_version 798224 (0.0008) [2023-12-26 21:10:21,545][105692] Updated weights for policy 0, policy_version 798234 (0.0009) [2023-12-26 21:10:21,604][105692] Updated weights for policy 0, policy_version 798244 (0.0010) [2023-12-26 21:10:21,831][105620] Updated weights for policy 1, policy_version 798090 (0.0010) [2023-12-26 21:10:21,894][105620] Updated weights for policy 1, policy_version 798100 (0.0011) [2023-12-26 21:10:21,954][105620] Updated weights for policy 1, policy_version 798110 (0.0011) [2023-12-26 21:10:22,018][105620] Updated weights for policy 1, policy_version 798120 (0.0011) [2023-12-26 21:10:22,395][105692] Updated weights for policy 0, policy_version 798254 (0.0009) [2023-12-26 21:10:22,461][105692] Updated weights for policy 0, policy_version 798264 (0.0008) [2023-12-26 21:10:22,530][105692] Updated weights for policy 0, policy_version 798274 (0.0007) [2023-12-26 21:10:22,781][105620] Updated weights for policy 1, policy_version 798130 (0.0006) [2023-12-26 21:10:22,847][105620] Updated weights for policy 1, policy_version 798140 (0.0007) [2023-12-26 21:10:22,920][105620] Updated weights for policy 1, policy_version 798150 (0.0007) [2023-12-26 21:10:23,222][105692] Updated weights for policy 0, policy_version 798284 (0.0006) [2023-12-26 21:10:23,279][105692] Updated weights for policy 0, policy_version 798294 (0.0008) [2023-12-26 21:10:23,335][105692] Updated weights for policy 0, policy_version 798304 (0.0009) [2023-12-26 21:10:23,594][105620] Updated weights for policy 1, policy_version 798160 (0.0008) [2023-12-26 21:10:23,668][105620] Updated weights for policy 1, policy_version 798170 (0.0007) [2023-12-26 21:10:23,725][105620] Updated weights for policy 1, policy_version 798180 (0.0009) [2023-12-26 21:10:23,958][105692] Updated weights for policy 0, policy_version 798314 (0.0008) [2023-12-26 21:10:24,021][105692] Updated weights for policy 0, policy_version 798324 (0.0008) [2023-12-26 21:10:24,086][105692] Updated weights for policy 0, policy_version 798334 (0.0010) [2023-12-26 21:10:24,145][105692] Updated weights for policy 0, policy_version 798344 (0.0009) [2023-12-26 21:10:24,468][105620] Updated weights for policy 1, policy_version 798190 (0.0008) [2023-12-26 21:10:24,528][105620] Updated weights for policy 1, policy_version 798200 (0.0008) [2023-12-26 21:10:24,583][105620] Updated weights for policy 1, policy_version 798210 (0.0008) [2023-12-26 21:10:24,865][105692] Updated weights for policy 0, policy_version 798354 (0.0010) [2023-12-26 21:10:24,913][105692] Updated weights for policy 0, policy_version 798364 (0.0008) [2023-12-26 21:10:24,968][105692] Updated weights for policy 0, policy_version 798374 (0.0009) [2023-12-26 21:10:25,355][105620] Updated weights for policy 1, policy_version 798220 (0.0006) [2023-12-26 21:10:25,406][105620] Updated weights for policy 1, policy_version 798230 (0.0005) [2023-12-26 21:10:25,452][105620] Updated weights for policy 1, policy_version 798240 (0.0005) [2023-12-26 21:10:25,750][105692] Updated weights for policy 0, policy_version 798384 (0.0010) [2023-12-26 21:10:25,810][105692] Updated weights for policy 0, policy_version 798394 (0.0009) [2023-12-26 21:10:25,864][105692] Updated weights for policy 0, policy_version 798404 (0.0010) [2023-12-26 21:10:25,992][105620] Updated weights for policy 1, policy_version 798250 (0.0006) [2023-12-26 21:10:26,053][105620] Updated weights for policy 1, policy_version 798260 (0.0009) [2023-12-26 21:10:26,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.8, 300 sec: 19438.7). Total num frames: 408797184. Throughput: 0: 9703.8, 1: 9801.4. Samples: 408807284. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-26 21:10:26,062][104569] Avg episode reward: [(0, '481.301'), (1, '9168.065')] [2023-12-26 21:10:26,115][105620] Updated weights for policy 1, policy_version 798270 (0.0009) [2023-12-26 21:10:26,174][105620] Updated weights for policy 1, policy_version 798280 (0.0009) [2023-12-26 21:10:26,601][105692] Updated weights for policy 0, policy_version 798414 (0.0007) [2023-12-26 21:10:26,653][105692] Updated weights for policy 0, policy_version 798424 (0.0005) [2023-12-26 21:10:26,708][105692] Updated weights for policy 0, policy_version 798434 (0.0008) [2023-12-26 21:10:26,918][105620] Updated weights for policy 1, policy_version 798290 (0.0009) [2023-12-26 21:10:26,968][105620] Updated weights for policy 1, policy_version 798300 (0.0009) [2023-12-26 21:10:27,028][105620] Updated weights for policy 1, policy_version 798310 (0.0009) [2023-12-26 21:10:27,460][105692] Updated weights for policy 0, policy_version 798444 (0.0009) [2023-12-26 21:10:27,512][105692] Updated weights for policy 0, policy_version 798454 (0.0009) [2023-12-26 21:10:27,570][105692] Updated weights for policy 0, policy_version 798465 (0.0010) [2023-12-26 21:10:27,718][105620] Updated weights for policy 1, policy_version 798320 (0.0006) [2023-12-26 21:10:27,775][105620] Updated weights for policy 1, policy_version 798330 (0.0005) [2023-12-26 21:10:27,833][105620] Updated weights for policy 1, policy_version 798340 (0.0005) [2023-12-26 21:10:28,388][105620] Updated weights for policy 1, policy_version 798350 (0.0009) [2023-12-26 21:10:28,438][105692] Updated weights for policy 0, policy_version 798475 (0.0009) [2023-12-26 21:10:28,442][105620] Updated weights for policy 1, policy_version 798360 (0.0007) [2023-12-26 21:10:28,492][105620] Updated weights for policy 1, policy_version 798370 (0.0009) [2023-12-26 21:10:28,494][105692] Updated weights for policy 0, policy_version 798485 (0.0006) [2023-12-26 21:10:28,552][105692] Updated weights for policy 0, policy_version 798495 (0.0007) [2023-12-26 21:10:29,120][105620] Updated weights for policy 1, policy_version 798380 (0.0008) [2023-12-26 21:10:29,178][105620] Updated weights for policy 1, policy_version 798390 (0.0006) [2023-12-26 21:10:29,246][105620] Updated weights for policy 1, policy_version 798400 (0.0006) [2023-12-26 21:10:29,379][105692] Updated weights for policy 0, policy_version 798505 (0.0008) [2023-12-26 21:10:29,447][105692] Updated weights for policy 0, policy_version 798515 (0.0009) [2023-12-26 21:10:29,517][105692] Updated weights for policy 0, policy_version 798525 (0.0010) [2023-12-26 21:10:29,587][105692] Updated weights for policy 0, policy_version 798535 (0.0010) [2023-12-26 21:10:29,785][105620] Updated weights for policy 1, policy_version 798410 (0.0005) [2023-12-26 21:10:29,836][105620] Updated weights for policy 1, policy_version 798420 (0.0006) [2023-12-26 21:10:29,890][105620] Updated weights for policy 1, policy_version 798430 (0.0007) [2023-12-26 21:10:29,955][105620] Updated weights for policy 1, policy_version 798440 (0.0008) [2023-12-26 21:10:30,343][105692] Updated weights for policy 0, policy_version 798545 (0.0010) [2023-12-26 21:10:30,389][105692] Updated weights for policy 0, policy_version 798555 (0.0008) [2023-12-26 21:10:30,442][105692] Updated weights for policy 0, policy_version 798565 (0.0008) [2023-12-26 21:10:30,659][105620] Updated weights for policy 1, policy_version 798450 (0.0009) [2023-12-26 21:10:30,705][105620] Updated weights for policy 1, policy_version 798460 (0.0009) [2023-12-26 21:10:30,752][105620] Updated weights for policy 1, policy_version 798470 (0.0009) [2023-12-26 21:10:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 408895488. Throughput: 0: 9676.0, 1: 9836.7. Samples: 408865256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:10:31,063][104569] Avg episode reward: [(0, '455.812'), (1, '9259.973')] [2023-12-26 21:10:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000798568_204464128.pth... [2023-12-26 21:10:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000798472_204431360.pth... [2023-12-26 21:10:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000797320_204136448.pth [2023-12-26 21:10:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000797448_204177408.pth [2023-12-26 21:10:31,151][105692] Updated weights for policy 0, policy_version 798575 (0.0008) [2023-12-26 21:10:31,206][105692] Updated weights for policy 0, policy_version 798585 (0.0008) [2023-12-26 21:10:31,263][105692] Updated weights for policy 0, policy_version 798595 (0.0007) [2023-12-26 21:10:31,582][105620] Updated weights for policy 1, policy_version 798480 (0.0010) [2023-12-26 21:10:31,647][105620] Updated weights for policy 1, policy_version 798490 (0.0009) [2023-12-26 21:10:31,710][105620] Updated weights for policy 1, policy_version 798500 (0.0008) [2023-12-26 21:10:31,913][105692] Updated weights for policy 0, policy_version 798605 (0.0006) [2023-12-26 21:10:31,973][105692] Updated weights for policy 0, policy_version 798615 (0.0006) [2023-12-26 21:10:32,023][105692] Updated weights for policy 0, policy_version 798625 (0.0005) [2023-12-26 21:10:32,558][105620] Updated weights for policy 1, policy_version 798510 (0.0008) [2023-12-26 21:10:32,607][105620] Updated weights for policy 1, policy_version 798520 (0.0008) [2023-12-26 21:10:32,649][105692] Updated weights for policy 0, policy_version 798635 (0.0007) [2023-12-26 21:10:32,664][105620] Updated weights for policy 1, policy_version 798530 (0.0006) [2023-12-26 21:10:32,715][105692] Updated weights for policy 0, policy_version 798645 (0.0010) [2023-12-26 21:10:32,764][105692] Updated weights for policy 0, policy_version 798655 (0.0010) [2023-12-26 21:10:33,345][105620] Updated weights for policy 1, policy_version 798540 (0.0006) [2023-12-26 21:10:33,401][105620] Updated weights for policy 1, policy_version 798550 (0.0005) [2023-12-26 21:10:33,466][105620] Updated weights for policy 1, policy_version 798560 (0.0006) [2023-12-26 21:10:33,519][105692] Updated weights for policy 0, policy_version 798665 (0.0010) [2023-12-26 21:10:33,563][105692] Updated weights for policy 0, policy_version 798675 (0.0010) [2023-12-26 21:10:33,608][105692] Updated weights for policy 0, policy_version 798685 (0.0010) [2023-12-26 21:10:33,652][105692] Updated weights for policy 0, policy_version 798695 (0.0010) [2023-12-26 21:10:33,971][105620] Updated weights for policy 1, policy_version 798570 (0.0006) [2023-12-26 21:10:34,032][105620] Updated weights for policy 1, policy_version 798580 (0.0009) [2023-12-26 21:10:34,079][105620] Updated weights for policy 1, policy_version 798590 (0.0005) [2023-12-26 21:10:34,134][105620] Updated weights for policy 1, policy_version 798600 (0.0006) [2023-12-26 21:10:34,388][105692] Updated weights for policy 0, policy_version 798705 (0.0010) [2023-12-26 21:10:34,455][105692] Updated weights for policy 0, policy_version 798715 (0.0010) [2023-12-26 21:10:34,515][105692] Updated weights for policy 0, policy_version 798725 (0.0010) [2023-12-26 21:10:34,827][105620] Updated weights for policy 1, policy_version 798610 (0.0005) [2023-12-26 21:10:34,896][105620] Updated weights for policy 1, policy_version 798620 (0.0005) [2023-12-26 21:10:34,953][105620] Updated weights for policy 1, policy_version 798630 (0.0005) [2023-12-26 21:10:35,248][105692] Updated weights for policy 0, policy_version 798735 (0.0010) [2023-12-26 21:10:35,296][105692] Updated weights for policy 0, policy_version 798745 (0.0010) [2023-12-26 21:10:35,341][105692] Updated weights for policy 0, policy_version 798755 (0.0010) [2023-12-26 21:10:35,579][105620] Updated weights for policy 1, policy_version 798640 (0.0007) [2023-12-26 21:10:35,635][105620] Updated weights for policy 1, policy_version 798650 (0.0007) [2023-12-26 21:10:35,694][105620] Updated weights for policy 1, policy_version 798660 (0.0007) [2023-12-26 21:10:36,052][105692] Updated weights for policy 0, policy_version 798765 (0.0008) [2023-12-26 21:10:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 408993792. Throughput: 0: 9566.5, 1: 9810.8. Samples: 408984964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:10:36,063][104569] Avg episode reward: [(0, '437.362'), (1, '9350.723')] [2023-12-26 21:10:36,117][105692] Updated weights for policy 0, policy_version 798775 (0.0007) [2023-12-26 21:10:36,179][105692] Updated weights for policy 0, policy_version 798785 (0.0009) [2023-12-26 21:10:36,310][105620] Updated weights for policy 1, policy_version 798670 (0.0008) [2023-12-26 21:10:36,377][105620] Updated weights for policy 1, policy_version 798680 (0.0007) [2023-12-26 21:10:36,443][105620] Updated weights for policy 1, policy_version 798690 (0.0006) [2023-12-26 21:10:36,970][105692] Updated weights for policy 0, policy_version 798795 (0.0010) [2023-12-26 21:10:37,025][105620] Updated weights for policy 1, policy_version 798700 (0.0009) [2023-12-26 21:10:37,032][105692] Updated weights for policy 0, policy_version 798805 (0.0009) [2023-12-26 21:10:37,079][105620] Updated weights for policy 1, policy_version 798710 (0.0007) [2023-12-26 21:10:37,091][105692] Updated weights for policy 0, policy_version 798815 (0.0009) [2023-12-26 21:10:37,140][105620] Updated weights for policy 1, policy_version 798720 (0.0010) [2023-12-26 21:10:37,818][105692] Updated weights for policy 0, policy_version 798825 (0.0008) [2023-12-26 21:10:37,872][105692] Updated weights for policy 0, policy_version 798835 (0.0005) [2023-12-26 21:10:37,927][105692] Updated weights for policy 0, policy_version 798845 (0.0005) [2023-12-26 21:10:37,931][105620] Updated weights for policy 1, policy_version 798730 (0.0009) [2023-12-26 21:10:37,980][105620] Updated weights for policy 1, policy_version 798740 (0.0008) [2023-12-26 21:10:37,987][105692] Updated weights for policy 0, policy_version 798855 (0.0009) [2023-12-26 21:10:38,038][105620] Updated weights for policy 1, policy_version 798750 (0.0010) [2023-12-26 21:10:38,111][105620] Updated weights for policy 1, policy_version 798760 (0.0008) [2023-12-26 21:10:38,620][105692] Updated weights for policy 0, policy_version 798865 (0.0009) [2023-12-26 21:10:38,676][105692] Updated weights for policy 0, policy_version 798875 (0.0009) [2023-12-26 21:10:38,731][105692] Updated weights for policy 0, policy_version 798885 (0.0009) [2023-12-26 21:10:38,894][105620] Updated weights for policy 1, policy_version 798770 (0.0009) [2023-12-26 21:10:38,955][105620] Updated weights for policy 1, policy_version 798780 (0.0008) [2023-12-26 21:10:39,010][105620] Updated weights for policy 1, policy_version 798790 (0.0009) [2023-12-26 21:10:39,534][105692] Updated weights for policy 0, policy_version 798895 (0.0008) [2023-12-26 21:10:39,593][105692] Updated weights for policy 0, policy_version 798905 (0.0008) [2023-12-26 21:10:39,652][105692] Updated weights for policy 0, policy_version 798915 (0.0009) [2023-12-26 21:10:39,790][105620] Updated weights for policy 1, policy_version 798800 (0.0009) [2023-12-26 21:10:39,853][105620] Updated weights for policy 1, policy_version 798810 (0.0009) [2023-12-26 21:10:39,915][105620] Updated weights for policy 1, policy_version 798820 (0.0009) [2023-12-26 21:10:40,424][105692] Updated weights for policy 0, policy_version 798925 (0.0008) [2023-12-26 21:10:40,488][105692] Updated weights for policy 0, policy_version 798935 (0.0008) [2023-12-26 21:10:40,544][105692] Updated weights for policy 0, policy_version 798945 (0.0009) [2023-12-26 21:10:40,681][105620] Updated weights for policy 1, policy_version 798830 (0.0009) [2023-12-26 21:10:40,737][105620] Updated weights for policy 1, policy_version 798840 (0.0009) [2023-12-26 21:10:40,792][105620] Updated weights for policy 1, policy_version 798850 (0.0009) [2023-12-26 21:10:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 409092096. Throughput: 0: 9624.8, 1: 9833.2. Samples: 409099100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:10:41,063][104569] Avg episode reward: [(0, '472.263'), (1, '9259.785')] [2023-12-26 21:10:41,325][105692] Updated weights for policy 0, policy_version 798955 (0.0009) [2023-12-26 21:10:41,395][105692] Updated weights for policy 0, policy_version 798965 (0.0009) [2023-12-26 21:10:41,460][105692] Updated weights for policy 0, policy_version 798975 (0.0009) [2023-12-26 21:10:41,587][105620] Updated weights for policy 1, policy_version 798860 (0.0009) [2023-12-26 21:10:41,648][105620] Updated weights for policy 1, policy_version 798870 (0.0010) [2023-12-26 21:10:41,712][105620] Updated weights for policy 1, policy_version 798880 (0.0009) [2023-12-26 21:10:42,234][105692] Updated weights for policy 0, policy_version 798985 (0.0009) [2023-12-26 21:10:42,301][105692] Updated weights for policy 0, policy_version 798995 (0.0011) [2023-12-26 21:10:42,366][105692] Updated weights for policy 0, policy_version 799005 (0.0011) [2023-12-26 21:10:42,429][105692] Updated weights for policy 0, policy_version 799015 (0.0011) [2023-12-26 21:10:42,506][105620] Updated weights for policy 1, policy_version 798890 (0.0009) [2023-12-26 21:10:42,566][105620] Updated weights for policy 1, policy_version 798900 (0.0008) [2023-12-26 21:10:42,617][105620] Updated weights for policy 1, policy_version 798910 (0.0009) [2023-12-26 21:10:42,662][105620] Updated weights for policy 1, policy_version 798920 (0.0009) [2023-12-26 21:10:43,178][105692] Updated weights for policy 0, policy_version 799025 (0.0010) [2023-12-26 21:10:43,243][105692] Updated weights for policy 0, policy_version 799035 (0.0010) [2023-12-26 21:10:43,304][105692] Updated weights for policy 0, policy_version 799045 (0.0010) [2023-12-26 21:10:43,448][105620] Updated weights for policy 1, policy_version 798930 (0.0008) [2023-12-26 21:10:43,493][105620] Updated weights for policy 1, policy_version 798940 (0.0008) [2023-12-26 21:10:43,545][105620] Updated weights for policy 1, policy_version 798950 (0.0008) [2023-12-26 21:10:43,941][105692] Updated weights for policy 0, policy_version 799055 (0.0007) [2023-12-26 21:10:44,001][105692] Updated weights for policy 0, policy_version 799065 (0.0006) [2023-12-26 21:10:44,049][105692] Updated weights for policy 0, policy_version 799075 (0.0010) [2023-12-26 21:10:44,310][105620] Updated weights for policy 1, policy_version 798960 (0.0010) [2023-12-26 21:10:44,377][105620] Updated weights for policy 1, policy_version 798970 (0.0008) [2023-12-26 21:10:44,435][105620] Updated weights for policy 1, policy_version 798980 (0.0006) [2023-12-26 21:10:44,710][105692] Updated weights for policy 0, policy_version 799085 (0.0009) [2023-12-26 21:10:44,768][105692] Updated weights for policy 0, policy_version 799095 (0.0011) [2023-12-26 21:10:44,832][105692] Updated weights for policy 0, policy_version 799106 (0.0007) [2023-12-26 21:10:44,994][105620] Updated weights for policy 1, policy_version 798990 (0.0008) [2023-12-26 21:10:45,047][105620] Updated weights for policy 1, policy_version 799000 (0.0010) [2023-12-26 21:10:45,109][105620] Updated weights for policy 1, policy_version 799010 (0.0008) [2023-12-26 21:10:45,586][105692] Updated weights for policy 0, policy_version 799116 (0.0009) [2023-12-26 21:10:45,639][105692] Updated weights for policy 0, policy_version 799126 (0.0007) [2023-12-26 21:10:45,684][105692] Updated weights for policy 0, policy_version 799136 (0.0008) [2023-12-26 21:10:45,782][105620] Updated weights for policy 1, policy_version 799020 (0.0007) [2023-12-26 21:10:45,849][105620] Updated weights for policy 1, policy_version 799030 (0.0010) [2023-12-26 21:10:45,903][105620] Updated weights for policy 1, policy_version 799040 (0.0010) [2023-12-26 21:10:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 409190400. Throughput: 0: 9631.5, 1: 9777.0. Samples: 409153016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:10:46,063][104569] Avg episode reward: [(0, '579.750'), (1, '6167.083')] [2023-12-26 21:10:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000799144_204611584.pth... [2023-12-26 21:10:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000799048_204578816.pth... [2023-12-26 21:10:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000798024_204324864.pth [2023-12-26 21:10:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000797864_204275712.pth [2023-12-26 21:10:46,411][105692] Updated weights for policy 0, policy_version 799146 (0.0008) [2023-12-26 21:10:46,469][105692] Updated weights for policy 0, policy_version 799156 (0.0008) [2023-12-26 21:10:46,524][105692] Updated weights for policy 0, policy_version 799166 (0.0010) [2023-12-26 21:10:46,573][105692] Updated weights for policy 0, policy_version 799176 (0.0010) [2023-12-26 21:10:46,630][105620] Updated weights for policy 1, policy_version 799050 (0.0010) [2023-12-26 21:10:46,684][105620] Updated weights for policy 1, policy_version 799060 (0.0010) [2023-12-26 21:10:46,734][105620] Updated weights for policy 1, policy_version 799070 (0.0006) [2023-12-26 21:10:46,783][105620] Updated weights for policy 1, policy_version 799080 (0.0010) [2023-12-26 21:10:47,223][105692] Updated weights for policy 0, policy_version 799186 (0.0010) [2023-12-26 21:10:47,286][105692] Updated weights for policy 0, policy_version 799196 (0.0009) [2023-12-26 21:10:47,341][105692] Updated weights for policy 0, policy_version 799206 (0.0010) [2023-12-26 21:10:47,541][105620] Updated weights for policy 1, policy_version 799090 (0.0010) [2023-12-26 21:10:47,605][105620] Updated weights for policy 1, policy_version 799100 (0.0010) [2023-12-26 21:10:47,670][105620] Updated weights for policy 1, policy_version 799110 (0.0010) [2023-12-26 21:10:48,016][105692] Updated weights for policy 0, policy_version 799216 (0.0008) [2023-12-26 21:10:48,067][105692] Updated weights for policy 0, policy_version 799226 (0.0010) [2023-12-26 21:10:48,115][105692] Updated weights for policy 0, policy_version 799236 (0.0010) [2023-12-26 21:10:48,338][105620] Updated weights for policy 1, policy_version 799120 (0.0010) [2023-12-26 21:10:48,409][105620] Updated weights for policy 1, policy_version 799130 (0.0011) [2023-12-26 21:10:48,475][105620] Updated weights for policy 1, policy_version 799140 (0.0011) [2023-12-26 21:10:48,861][105692] Updated weights for policy 0, policy_version 799246 (0.0007) [2023-12-26 21:10:48,916][105692] Updated weights for policy 0, policy_version 799256 (0.0005) [2023-12-26 21:10:48,971][105692] Updated weights for policy 0, policy_version 799266 (0.0007) [2023-12-26 21:10:49,244][105620] Updated weights for policy 1, policy_version 799150 (0.0009) [2023-12-26 21:10:49,308][105620] Updated weights for policy 1, policy_version 799160 (0.0008) [2023-12-26 21:10:49,377][105620] Updated weights for policy 1, policy_version 799170 (0.0008) [2023-12-26 21:10:49,691][105692] Updated weights for policy 0, policy_version 799276 (0.0009) [2023-12-26 21:10:49,739][105692] Updated weights for policy 0, policy_version 799286 (0.0010) [2023-12-26 21:10:49,790][105692] Updated weights for policy 0, policy_version 799296 (0.0008) [2023-12-26 21:10:50,057][105620] Updated weights for policy 1, policy_version 799180 (0.0005) [2023-12-26 21:10:50,114][105620] Updated weights for policy 1, policy_version 799190 (0.0006) [2023-12-26 21:10:50,171][105620] Updated weights for policy 1, policy_version 799200 (0.0006) [2023-12-26 21:10:50,520][105692] Updated weights for policy 0, policy_version 799306 (0.0008) [2023-12-26 21:10:50,574][105692] Updated weights for policy 0, policy_version 799316 (0.0010) [2023-12-26 21:10:50,637][105692] Updated weights for policy 0, policy_version 799326 (0.0009) [2023-12-26 21:10:50,695][105692] Updated weights for policy 0, policy_version 799336 (0.0009) [2023-12-26 21:10:50,905][105620] Updated weights for policy 1, policy_version 799210 (0.0008) [2023-12-26 21:10:50,964][105620] Updated weights for policy 1, policy_version 799220 (0.0008) [2023-12-26 21:10:51,024][105620] Updated weights for policy 1, policy_version 799230 (0.0008) [2023-12-26 21:10:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 409280512. Throughput: 0: 9590.9, 1: 9899.1. Samples: 409273988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:10:51,062][104569] Avg episode reward: [(0, '677.141'), (1, '4716.448')] [2023-12-26 21:10:51,087][105620] Updated weights for policy 1, policy_version 799240 (0.0008) [2023-12-26 21:10:51,508][105692] Updated weights for policy 0, policy_version 799346 (0.0010) [2023-12-26 21:10:51,564][105692] Updated weights for policy 0, policy_version 799356 (0.0010) [2023-12-26 21:10:51,621][105692] Updated weights for policy 0, policy_version 799366 (0.0011) [2023-12-26 21:10:51,873][105620] Updated weights for policy 1, policy_version 799250 (0.0006) [2023-12-26 21:10:51,926][105620] Updated weights for policy 1, policy_version 799260 (0.0007) [2023-12-26 21:10:51,980][105620] Updated weights for policy 1, policy_version 799270 (0.0005) [2023-12-26 21:10:52,338][105692] Updated weights for policy 0, policy_version 799376 (0.0011) [2023-12-26 21:10:52,403][105692] Updated weights for policy 0, policy_version 799386 (0.0011) [2023-12-26 21:10:52,462][105692] Updated weights for policy 0, policy_version 799396 (0.0010) [2023-12-26 21:10:52,689][105620] Updated weights for policy 1, policy_version 799280 (0.0009) [2023-12-26 21:10:52,742][105620] Updated weights for policy 1, policy_version 799290 (0.0009) [2023-12-26 21:10:52,805][105620] Updated weights for policy 1, policy_version 799300 (0.0009) [2023-12-26 21:10:53,126][105692] Updated weights for policy 0, policy_version 799406 (0.0008) [2023-12-26 21:10:53,196][105692] Updated weights for policy 0, policy_version 799416 (0.0007) [2023-12-26 21:10:53,250][105692] Updated weights for policy 0, policy_version 799426 (0.0010) [2023-12-26 21:10:53,570][105620] Updated weights for policy 1, policy_version 799310 (0.0007) [2023-12-26 21:10:53,631][105620] Updated weights for policy 1, policy_version 799320 (0.0005) [2023-12-26 21:10:53,678][105620] Updated weights for policy 1, policy_version 799330 (0.0005) [2023-12-26 21:10:53,879][105692] Updated weights for policy 0, policy_version 799436 (0.0009) [2023-12-26 21:10:53,938][105692] Updated weights for policy 0, policy_version 799446 (0.0010) [2023-12-26 21:10:53,997][105692] Updated weights for policy 0, policy_version 799456 (0.0010) [2023-12-26 21:10:54,438][105620] Updated weights for policy 1, policy_version 799340 (0.0007) [2023-12-26 21:10:54,501][105620] Updated weights for policy 1, policy_version 799350 (0.0008) [2023-12-26 21:10:54,561][105620] Updated weights for policy 1, policy_version 799360 (0.0008) [2023-12-26 21:10:54,753][105692] Updated weights for policy 0, policy_version 799466 (0.0010) [2023-12-26 21:10:54,802][105692] Updated weights for policy 0, policy_version 799476 (0.0011) [2023-12-26 21:10:54,861][105692] Updated weights for policy 0, policy_version 799486 (0.0011) [2023-12-26 21:10:54,913][105692] Updated weights for policy 0, policy_version 799496 (0.0011) [2023-12-26 21:10:55,324][105620] Updated weights for policy 1, policy_version 799370 (0.0008) [2023-12-26 21:10:55,377][105620] Updated weights for policy 1, policy_version 799380 (0.0008) [2023-12-26 21:10:55,430][105620] Updated weights for policy 1, policy_version 799390 (0.0008) [2023-12-26 21:10:55,492][105620] Updated weights for policy 1, policy_version 799400 (0.0008) [2023-12-26 21:10:55,679][105692] Updated weights for policy 0, policy_version 799506 (0.0005) [2023-12-26 21:10:55,736][105692] Updated weights for policy 0, policy_version 799516 (0.0005) [2023-12-26 21:10:55,795][105692] Updated weights for policy 0, policy_version 799526 (0.0005) [2023-12-26 21:10:56,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 409378816. Throughput: 0: 9615.1, 1: 9817.1. Samples: 409387696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:10:56,063][104569] Avg episode reward: [(0, '1106.047'), (1, '7290.487')] [2023-12-26 21:10:56,238][105620] Updated weights for policy 1, policy_version 799410 (0.0010) [2023-12-26 21:10:56,287][105620] Updated weights for policy 1, policy_version 799420 (0.0010) [2023-12-26 21:10:56,341][105620] Updated weights for policy 1, policy_version 799430 (0.0009) [2023-12-26 21:10:56,474][105692] Updated weights for policy 0, policy_version 799536 (0.0010) [2023-12-26 21:10:56,518][105692] Updated weights for policy 0, policy_version 799546 (0.0010) [2023-12-26 21:10:56,562][105692] Updated weights for policy 0, policy_version 799556 (0.0010) [2023-12-26 21:10:57,093][105620] Updated weights for policy 1, policy_version 799440 (0.0010) [2023-12-26 21:10:57,141][105620] Updated weights for policy 1, policy_version 799450 (0.0010) [2023-12-26 21:10:57,185][105620] Updated weights for policy 1, policy_version 799460 (0.0010) [2023-12-26 21:10:57,284][105692] Updated weights for policy 0, policy_version 799566 (0.0010) [2023-12-26 21:10:57,357][105692] Updated weights for policy 0, policy_version 799576 (0.0009) [2023-12-26 21:10:57,414][105692] Updated weights for policy 0, policy_version 799586 (0.0007) [2023-12-26 21:10:57,938][105620] Updated weights for policy 1, policy_version 799470 (0.0010) [2023-12-26 21:10:57,984][105620] Updated weights for policy 1, policy_version 799480 (0.0008) [2023-12-26 21:10:58,033][105620] Updated weights for policy 1, policy_version 799490 (0.0008) [2023-12-26 21:10:58,118][105692] Updated weights for policy 0, policy_version 799596 (0.0007) [2023-12-26 21:10:58,179][105692] Updated weights for policy 0, policy_version 799606 (0.0009) [2023-12-26 21:10:58,232][105692] Updated weights for policy 0, policy_version 799616 (0.0009) [2023-12-26 21:10:58,866][105620] Updated weights for policy 1, policy_version 799500 (0.0008) [2023-12-26 21:10:58,923][105620] Updated weights for policy 1, policy_version 799510 (0.0008) [2023-12-26 21:10:58,984][105620] Updated weights for policy 1, policy_version 799520 (0.0006) [2023-12-26 21:10:59,041][105692] Updated weights for policy 0, policy_version 799626 (0.0009) [2023-12-26 21:10:59,088][105692] Updated weights for policy 0, policy_version 799636 (0.0007) [2023-12-26 21:10:59,138][105692] Updated weights for policy 0, policy_version 799646 (0.0005) [2023-12-26 21:10:59,194][105692] Updated weights for policy 0, policy_version 799656 (0.0005) [2023-12-26 21:10:59,728][105620] Updated weights for policy 1, policy_version 799530 (0.0007) [2023-12-26 21:10:59,782][105620] Updated weights for policy 1, policy_version 799540 (0.0010) [2023-12-26 21:10:59,854][105620] Updated weights for policy 1, policy_version 799550 (0.0008) [2023-12-26 21:10:59,892][105692] Updated weights for policy 0, policy_version 799666 (0.0009) [2023-12-26 21:10:59,909][105620] Updated weights for policy 1, policy_version 799560 (0.0008) [2023-12-26 21:10:59,943][105692] Updated weights for policy 0, policy_version 799676 (0.0007) [2023-12-26 21:11:00,032][105692] Updated weights for policy 0, policy_version 799686 (0.0009) [2023-12-26 21:11:00,475][105620] Updated weights for policy 1, policy_version 799570 (0.0011) [2023-12-26 21:11:00,530][105620] Updated weights for policy 1, policy_version 799580 (0.0006) [2023-12-26 21:11:00,596][105620] Updated weights for policy 1, policy_version 799590 (0.0009) [2023-12-26 21:11:00,857][105692] Updated weights for policy 0, policy_version 799696 (0.0008) [2023-12-26 21:11:00,907][105692] Updated weights for policy 0, policy_version 799706 (0.0008) [2023-12-26 21:11:00,949][105692] Updated weights for policy 0, policy_version 799716 (0.0007) [2023-12-26 21:11:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 409477120. Throughput: 0: 9642.6, 1: 9746.0. Samples: 409444568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:11:01,063][104569] Avg episode reward: [(0, '1191.524'), (1, '9259.613')] [2023-12-26 21:11:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000799720_204759040.pth... [2023-12-26 21:11:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000799592_204718080.pth... [2023-12-26 21:11:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000798568_204464128.pth [2023-12-26 21:11:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000798472_204431360.pth [2023-12-26 21:11:01,336][105620] Updated weights for policy 1, policy_version 799600 (0.0007) [2023-12-26 21:11:01,405][105620] Updated weights for policy 1, policy_version 799610 (0.0008) [2023-12-26 21:11:01,461][105620] Updated weights for policy 1, policy_version 799620 (0.0006) [2023-12-26 21:11:01,740][105692] Updated weights for policy 0, policy_version 799726 (0.0009) [2023-12-26 21:11:01,798][105692] Updated weights for policy 0, policy_version 799736 (0.0010) [2023-12-26 21:11:01,858][105692] Updated weights for policy 0, policy_version 799746 (0.0011) [2023-12-26 21:11:02,118][105620] Updated weights for policy 1, policy_version 799630 (0.0006) [2023-12-26 21:11:02,181][105620] Updated weights for policy 1, policy_version 799640 (0.0008) [2023-12-26 21:11:02,232][105620] Updated weights for policy 1, policy_version 799650 (0.0009) [2023-12-26 21:11:02,549][105692] Updated weights for policy 0, policy_version 799756 (0.0010) [2023-12-26 21:11:02,607][105692] Updated weights for policy 0, policy_version 799766 (0.0010) [2023-12-26 21:11:02,669][105692] Updated weights for policy 0, policy_version 799776 (0.0010) [2023-12-26 21:11:02,997][105620] Updated weights for policy 1, policy_version 799660 (0.0009) [2023-12-26 21:11:03,045][105620] Updated weights for policy 1, policy_version 799670 (0.0008) [2023-12-26 21:11:03,093][105620] Updated weights for policy 1, policy_version 799680 (0.0007) [2023-12-26 21:11:03,349][105692] Updated weights for policy 0, policy_version 799786 (0.0008) [2023-12-26 21:11:03,409][105692] Updated weights for policy 0, policy_version 799796 (0.0010) [2023-12-26 21:11:03,467][105692] Updated weights for policy 0, policy_version 799806 (0.0010) [2023-12-26 21:11:03,522][105692] Updated weights for policy 0, policy_version 799816 (0.0010) [2023-12-26 21:11:03,782][105620] Updated weights for policy 1, policy_version 799690 (0.0008) [2023-12-26 21:11:03,834][105620] Updated weights for policy 1, policy_version 799701 (0.0008) [2023-12-26 21:11:03,901][105620] Updated weights for policy 1, policy_version 799711 (0.0009) [2023-12-26 21:11:04,251][105692] Updated weights for policy 0, policy_version 799826 (0.0009) [2023-12-26 21:11:04,302][105692] Updated weights for policy 0, policy_version 799836 (0.0009) [2023-12-26 21:11:04,353][105692] Updated weights for policy 0, policy_version 799846 (0.0008) [2023-12-26 21:11:04,629][105620] Updated weights for policy 1, policy_version 799721 (0.0009) [2023-12-26 21:11:04,692][105620] Updated weights for policy 1, policy_version 799731 (0.0006) [2023-12-26 21:11:04,759][105620] Updated weights for policy 1, policy_version 799741 (0.0006) [2023-12-26 21:11:04,824][105620] Updated weights for policy 1, policy_version 799751 (0.0006) [2023-12-26 21:11:05,247][105692] Updated weights for policy 0, policy_version 799856 (0.0009) [2023-12-26 21:11:05,307][105692] Updated weights for policy 0, policy_version 799866 (0.0008) [2023-12-26 21:11:05,339][105620] Updated weights for policy 1, policy_version 799761 (0.0006) [2023-12-26 21:11:05,357][105692] Updated weights for policy 0, policy_version 799876 (0.0008) [2023-12-26 21:11:05,385][105620] Updated weights for policy 1, policy_version 799771 (0.0008) [2023-12-26 21:11:05,434][105620] Updated weights for policy 1, policy_version 799781 (0.0010) [2023-12-26 21:11:06,049][105620] Updated weights for policy 1, policy_version 799791 (0.0010) [2023-12-26 21:11:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 409567232. Throughput: 0: 9555.3, 1: 9772.5. Samples: 409560852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:11:06,062][104569] Avg episode reward: [(0, '3515.943'), (1, '9351.293')] [2023-12-26 21:11:06,097][105620] Updated weights for policy 1, policy_version 799801 (0.0010) [2023-12-26 21:11:06,166][105620] Updated weights for policy 1, policy_version 799811 (0.0009) [2023-12-26 21:11:06,192][105692] Updated weights for policy 0, policy_version 799886 (0.0009) [2023-12-26 21:11:06,245][105692] Updated weights for policy 0, policy_version 799896 (0.0009) [2023-12-26 21:11:06,299][105692] Updated weights for policy 0, policy_version 799906 (0.0010) [2023-12-26 21:11:06,901][105620] Updated weights for policy 1, policy_version 799821 (0.0007) [2023-12-26 21:11:06,958][105620] Updated weights for policy 1, policy_version 799831 (0.0006) [2023-12-26 21:11:07,012][105620] Updated weights for policy 1, policy_version 799841 (0.0005) [2023-12-26 21:11:07,086][105692] Updated weights for policy 0, policy_version 799916 (0.0008) [2023-12-26 21:11:07,142][105692] Updated weights for policy 0, policy_version 799926 (0.0010) [2023-12-26 21:11:07,196][105692] Updated weights for policy 0, policy_version 799936 (0.0011) [2023-12-26 21:11:07,625][105620] Updated weights for policy 1, policy_version 799851 (0.0007) [2023-12-26 21:11:07,685][105620] Updated weights for policy 1, policy_version 799861 (0.0010) [2023-12-26 21:11:07,743][105620] Updated weights for policy 1, policy_version 799871 (0.0008) [2023-12-26 21:11:07,915][105692] Updated weights for policy 0, policy_version 799946 (0.0010) [2023-12-26 21:11:07,963][105692] Updated weights for policy 0, policy_version 799956 (0.0008) [2023-12-26 21:11:08,007][105692] Updated weights for policy 0, policy_version 799966 (0.0008) [2023-12-26 21:11:08,055][105692] Updated weights for policy 0, policy_version 799976 (0.0007) [2023-12-26 21:11:08,396][105620] Updated weights for policy 1, policy_version 799881 (0.0006) [2023-12-26 21:11:08,449][105620] Updated weights for policy 1, policy_version 799891 (0.0011) [2023-12-26 21:11:08,505][105620] Updated weights for policy 1, policy_version 799901 (0.0010) [2023-12-26 21:11:08,560][105620] Updated weights for policy 1, policy_version 799911 (0.0010) [2023-12-26 21:11:08,866][105692] Updated weights for policy 0, policy_version 799986 (0.0008) [2023-12-26 21:11:08,915][105692] Updated weights for policy 0, policy_version 799996 (0.0008) [2023-12-26 21:11:08,972][105692] Updated weights for policy 0, policy_version 800006 (0.0008) [2023-12-26 21:11:09,368][105620] Updated weights for policy 1, policy_version 799921 (0.0009) [2023-12-26 21:11:09,432][105620] Updated weights for policy 1, policy_version 799931 (0.0009) [2023-12-26 21:11:09,495][105620] Updated weights for policy 1, policy_version 799941 (0.0009) [2023-12-26 21:11:09,715][105692] Updated weights for policy 0, policy_version 800016 (0.0010) [2023-12-26 21:11:09,767][105692] Updated weights for policy 0, policy_version 800026 (0.0010) [2023-12-26 21:11:09,837][105692] Updated weights for policy 0, policy_version 800036 (0.0011) [2023-12-26 21:11:10,193][105620] Updated weights for policy 1, policy_version 799951 (0.0010) [2023-12-26 21:11:10,253][105620] Updated weights for policy 1, policy_version 799961 (0.0010) [2023-12-26 21:11:10,312][105620] Updated weights for policy 1, policy_version 799971 (0.0010) [2023-12-26 21:11:10,577][105692] Updated weights for policy 0, policy_version 800046 (0.0010) [2023-12-26 21:11:10,632][105692] Updated weights for policy 0, policy_version 800056 (0.0010) [2023-12-26 21:11:10,691][105692] Updated weights for policy 0, policy_version 800066 (0.0010) [2023-12-26 21:11:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 409665536. Throughput: 0: 9489.8, 1: 9830.2. Samples: 409676684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:11:11,062][104569] Avg episode reward: [(0, '3281.938'), (1, '9351.604')] [2023-12-26 21:11:11,065][105620] Updated weights for policy 1, policy_version 799981 (0.0010) [2023-12-26 21:11:11,134][105620] Updated weights for policy 1, policy_version 799991 (0.0008) [2023-12-26 21:11:11,191][105620] Updated weights for policy 1, policy_version 800001 (0.0010) [2023-12-26 21:11:11,464][105692] Updated weights for policy 0, policy_version 800076 (0.0010) [2023-12-26 21:11:11,519][105692] Updated weights for policy 0, policy_version 800086 (0.0009) [2023-12-26 21:11:11,568][105692] Updated weights for policy 0, policy_version 800096 (0.0008) [2023-12-26 21:11:12,011][105620] Updated weights for policy 1, policy_version 800011 (0.0009) [2023-12-26 21:11:12,079][105620] Updated weights for policy 1, policy_version 800021 (0.0005) [2023-12-26 21:11:12,135][105620] Updated weights for policy 1, policy_version 800031 (0.0006) [2023-12-26 21:11:12,421][105692] Updated weights for policy 0, policy_version 800106 (0.0008) [2023-12-26 21:11:12,472][105692] Updated weights for policy 0, policy_version 800116 (0.0006) [2023-12-26 21:11:12,531][105692] Updated weights for policy 0, policy_version 800126 (0.0006) [2023-12-26 21:11:12,600][105692] Updated weights for policy 0, policy_version 800136 (0.0006) [2023-12-26 21:11:12,761][105620] Updated weights for policy 1, policy_version 800041 (0.0009) [2023-12-26 21:11:12,826][105620] Updated weights for policy 1, policy_version 800051 (0.0006) [2023-12-26 21:11:12,881][105620] Updated weights for policy 1, policy_version 800061 (0.0005) [2023-12-26 21:11:12,931][105620] Updated weights for policy 1, policy_version 800071 (0.0005) [2023-12-26 21:11:13,188][105692] Updated weights for policy 0, policy_version 800146 (0.0006) [2023-12-26 21:11:13,245][105692] Updated weights for policy 0, policy_version 800156 (0.0009) [2023-12-26 21:11:13,296][105692] Updated weights for policy 0, policy_version 800166 (0.0009) [2023-12-26 21:11:13,476][105620] Updated weights for policy 1, policy_version 800081 (0.0010) [2023-12-26 21:11:13,524][105620] Updated weights for policy 1, policy_version 800091 (0.0010) [2023-12-26 21:11:13,569][105620] Updated weights for policy 1, policy_version 800101 (0.0010) [2023-12-26 21:11:13,995][105692] Updated weights for policy 0, policy_version 800176 (0.0010) [2023-12-26 21:11:14,050][105692] Updated weights for policy 0, policy_version 800186 (0.0010) [2023-12-26 21:11:14,102][105692] Updated weights for policy 0, policy_version 800196 (0.0010) [2023-12-26 21:11:14,316][105620] Updated weights for policy 1, policy_version 800111 (0.0010) [2023-12-26 21:11:14,374][105620] Updated weights for policy 1, policy_version 800121 (0.0010) [2023-12-26 21:11:14,422][105620] Updated weights for policy 1, policy_version 800131 (0.0010) [2023-12-26 21:11:14,858][105692] Updated weights for policy 0, policy_version 800206 (0.0011) [2023-12-26 21:11:14,907][105692] Updated weights for policy 0, policy_version 800216 (0.0010) [2023-12-26 21:11:14,963][105692] Updated weights for policy 0, policy_version 800226 (0.0010) [2023-12-26 21:11:15,193][105620] Updated weights for policy 1, policy_version 800141 (0.0009) [2023-12-26 21:11:15,254][105620] Updated weights for policy 1, policy_version 800151 (0.0008) [2023-12-26 21:11:15,310][105620] Updated weights for policy 1, policy_version 800161 (0.0008) [2023-12-26 21:11:15,730][105692] Updated weights for policy 0, policy_version 800236 (0.0010) [2023-12-26 21:11:15,781][105692] Updated weights for policy 0, policy_version 800246 (0.0010) [2023-12-26 21:11:15,829][105692] Updated weights for policy 0, policy_version 800256 (0.0010) [2023-12-26 21:11:16,053][105620] Updated weights for policy 1, policy_version 800171 (0.0009) [2023-12-26 21:11:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 409763840. Throughput: 0: 9534.1, 1: 9818.6. Samples: 409736128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:11:16,062][104569] Avg episode reward: [(0, '4731.458'), (1, '9352.108')] [2023-12-26 21:11:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000800264_204898304.pth... [2023-12-26 21:11:16,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000799144_204611584.pth [2023-12-26 21:11:16,108][105620] Updated weights for policy 1, policy_version 800181 (0.0010) [2023-12-26 21:11:16,163][105620] Updated weights for policy 1, policy_version 800191 (0.0010) [2023-12-26 21:11:16,207][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000800200_204873728.pth... [2023-12-26 21:11:16,210][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000799048_204578816.pth [2023-12-26 21:11:16,418][105692] Updated weights for policy 0, policy_version 800266 (0.0008) [2023-12-26 21:11:16,480][105692] Updated weights for policy 0, policy_version 800276 (0.0011) [2023-12-26 21:11:16,543][105692] Updated weights for policy 0, policy_version 800286 (0.0011) [2023-12-26 21:11:16,606][105692] Updated weights for policy 0, policy_version 800296 (0.0010) [2023-12-26 21:11:16,910][105620] Updated weights for policy 1, policy_version 800201 (0.0010) [2023-12-26 21:11:16,971][105620] Updated weights for policy 1, policy_version 800211 (0.0010) [2023-12-26 21:11:17,015][105620] Updated weights for policy 1, policy_version 800221 (0.0010) [2023-12-26 21:11:17,064][105620] Updated weights for policy 1, policy_version 800231 (0.0010) [2023-12-26 21:11:17,323][105692] Updated weights for policy 0, policy_version 800306 (0.0010) [2023-12-26 21:11:17,383][105692] Updated weights for policy 0, policy_version 800316 (0.0008) [2023-12-26 21:11:17,442][105692] Updated weights for policy 0, policy_version 800326 (0.0005) [2023-12-26 21:11:17,817][105620] Updated weights for policy 1, policy_version 800241 (0.0010) [2023-12-26 21:11:17,872][105620] Updated weights for policy 1, policy_version 800251 (0.0010) [2023-12-26 21:11:17,923][105620] Updated weights for policy 1, policy_version 800261 (0.0010) [2023-12-26 21:11:18,127][105692] Updated weights for policy 0, policy_version 800336 (0.0007) [2023-12-26 21:11:18,201][105692] Updated weights for policy 0, policy_version 800346 (0.0007) [2023-12-26 21:11:18,251][105692] Updated weights for policy 0, policy_version 800356 (0.0006) [2023-12-26 21:11:18,684][105620] Updated weights for policy 1, policy_version 800271 (0.0010) [2023-12-26 21:11:18,736][105620] Updated weights for policy 1, policy_version 800281 (0.0010) [2023-12-26 21:11:18,791][105620] Updated weights for policy 1, policy_version 800291 (0.0011) [2023-12-26 21:11:18,833][105692] Updated weights for policy 0, policy_version 800366 (0.0005) [2023-12-26 21:11:18,882][105692] Updated weights for policy 0, policy_version 800376 (0.0005) [2023-12-26 21:11:18,930][105692] Updated weights for policy 0, policy_version 800386 (0.0006) [2023-12-26 21:11:19,512][105620] Updated weights for policy 1, policy_version 800301 (0.0010) [2023-12-26 21:11:19,569][105620] Updated weights for policy 1, policy_version 800311 (0.0011) [2023-12-26 21:11:19,601][105692] Updated weights for policy 0, policy_version 800396 (0.0008) [2023-12-26 21:11:19,632][105620] Updated weights for policy 1, policy_version 800321 (0.0011) [2023-12-26 21:11:19,658][105692] Updated weights for policy 0, policy_version 800406 (0.0011) [2023-12-26 21:11:19,714][105692] Updated weights for policy 0, policy_version 800416 (0.0011) [2023-12-26 21:11:20,339][105620] Updated weights for policy 1, policy_version 800331 (0.0011) [2023-12-26 21:11:20,404][105620] Updated weights for policy 1, policy_version 800341 (0.0011) [2023-12-26 21:11:20,453][105620] Updated weights for policy 1, policy_version 800351 (0.0011) [2023-12-26 21:11:20,506][105692] Updated weights for policy 0, policy_version 800426 (0.0011) [2023-12-26 21:11:20,557][105692] Updated weights for policy 0, policy_version 800436 (0.0010) [2023-12-26 21:11:20,629][105692] Updated weights for policy 0, policy_version 800446 (0.0010) [2023-12-26 21:11:20,692][105692] Updated weights for policy 0, policy_version 800456 (0.0011) [2023-12-26 21:11:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 409862144. Throughput: 0: 9597.6, 1: 9702.9. Samples: 409853488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:11:21,063][104569] Avg episode reward: [(0, '4201.709'), (1, '9260.688')] [2023-12-26 21:11:21,181][105620] Updated weights for policy 1, policy_version 800361 (0.0010) [2023-12-26 21:11:21,245][105620] Updated weights for policy 1, policy_version 800371 (0.0010) [2023-12-26 21:11:21,320][105620] Updated weights for policy 1, policy_version 800381 (0.0009) [2023-12-26 21:11:21,391][105620] Updated weights for policy 1, policy_version 800391 (0.0009) [2023-12-26 21:11:21,502][105692] Updated weights for policy 0, policy_version 800466 (0.0008) [2023-12-26 21:11:21,567][105692] Updated weights for policy 0, policy_version 800476 (0.0008) [2023-12-26 21:11:21,626][105692] Updated weights for policy 0, policy_version 800486 (0.0009) [2023-12-26 21:11:22,103][105620] Updated weights for policy 1, policy_version 800401 (0.0011) [2023-12-26 21:11:22,168][105620] Updated weights for policy 1, policy_version 800411 (0.0011) [2023-12-26 21:11:22,223][105620] Updated weights for policy 1, policy_version 800421 (0.0010) [2023-12-26 21:11:22,314][105692] Updated weights for policy 0, policy_version 800496 (0.0008) [2023-12-26 21:11:22,384][105692] Updated weights for policy 0, policy_version 800506 (0.0009) [2023-12-26 21:11:22,444][105692] Updated weights for policy 0, policy_version 800516 (0.0008) [2023-12-26 21:11:22,956][105620] Updated weights for policy 1, policy_version 800431 (0.0009) [2023-12-26 21:11:23,012][105620] Updated weights for policy 1, policy_version 800441 (0.0008) [2023-12-26 21:11:23,072][105620] Updated weights for policy 1, policy_version 800451 (0.0008) [2023-12-26 21:11:23,194][105692] Updated weights for policy 0, policy_version 800526 (0.0007) [2023-12-26 21:11:23,256][105692] Updated weights for policy 0, policy_version 800536 (0.0006) [2023-12-26 21:11:23,307][105692] Updated weights for policy 0, policy_version 800546 (0.0010) [2023-12-26 21:11:23,833][105620] Updated weights for policy 1, policy_version 800461 (0.0009) [2023-12-26 21:11:23,898][105620] Updated weights for policy 1, policy_version 800471 (0.0010) [2023-12-26 21:11:23,901][105692] Updated weights for policy 0, policy_version 800556 (0.0008) [2023-12-26 21:11:23,949][105692] Updated weights for policy 0, policy_version 800566 (0.0009) [2023-12-26 21:11:23,955][105620] Updated weights for policy 1, policy_version 800481 (0.0007) [2023-12-26 21:11:23,993][105586] KL-divergence is very high: 204.7350 [2023-12-26 21:11:24,012][105692] Updated weights for policy 0, policy_version 800576 (0.0011) [2023-12-26 21:11:24,698][105692] Updated weights for policy 0, policy_version 800586 (0.0009) [2023-12-26 21:11:24,753][105692] Updated weights for policy 0, policy_version 800596 (0.0010) [2023-12-26 21:11:24,767][105620] Updated weights for policy 1, policy_version 800491 (0.0005) [2023-12-26 21:11:24,816][105692] Updated weights for policy 0, policy_version 800606 (0.0009) [2023-12-26 21:11:24,823][105620] Updated weights for policy 1, policy_version 800501 (0.0007) [2023-12-26 21:11:24,879][105692] Updated weights for policy 0, policy_version 800616 (0.0006) [2023-12-26 21:11:24,883][105620] Updated weights for policy 1, policy_version 800511 (0.0008) [2023-12-26 21:11:25,495][105692] Updated weights for policy 0, policy_version 800626 (0.0009) [2023-12-26 21:11:25,546][105692] Updated weights for policy 0, policy_version 800636 (0.0010) [2023-12-26 21:11:25,594][105692] Updated weights for policy 0, policy_version 800646 (0.0010) [2023-12-26 21:11:25,708][105620] Updated weights for policy 1, policy_version 800521 (0.0009) [2023-12-26 21:11:25,757][105620] Updated weights for policy 1, policy_version 800531 (0.0008) [2023-12-26 21:11:25,802][105620] Updated weights for policy 1, policy_version 800541 (0.0008) [2023-12-26 21:11:25,851][105620] Updated weights for policy 1, policy_version 800551 (0.0008) [2023-12-26 21:11:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 409960448. Throughput: 0: 9647.7, 1: 9637.9. Samples: 409966956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:11:26,063][104569] Avg episode reward: [(0, '4013.480'), (1, '9078.291')] [2023-12-26 21:11:26,274][105692] Updated weights for policy 0, policy_version 800656 (0.0010) [2023-12-26 21:11:26,333][105692] Updated weights for policy 0, policy_version 800666 (0.0010) [2023-12-26 21:11:26,398][105692] Updated weights for policy 0, policy_version 800676 (0.0010) [2023-12-26 21:11:26,552][105620] Updated weights for policy 1, policy_version 800561 (0.0006) [2023-12-26 21:11:26,607][105620] Updated weights for policy 1, policy_version 800571 (0.0005) [2023-12-26 21:11:26,662][105620] Updated weights for policy 1, policy_version 800581 (0.0005) [2023-12-26 21:11:27,123][105692] Updated weights for policy 0, policy_version 800686 (0.0010) [2023-12-26 21:11:27,187][105692] Updated weights for policy 0, policy_version 800696 (0.0009) [2023-12-26 21:11:27,254][105692] Updated weights for policy 0, policy_version 800706 (0.0009) [2023-12-26 21:11:27,357][105620] Updated weights for policy 1, policy_version 800591 (0.0009) [2023-12-26 21:11:27,411][105620] Updated weights for policy 1, policy_version 800601 (0.0009) [2023-12-26 21:11:27,474][105620] Updated weights for policy 1, policy_version 800611 (0.0005) [2023-12-26 21:11:27,936][105692] Updated weights for policy 0, policy_version 800716 (0.0007) [2023-12-26 21:11:27,982][105692] Updated weights for policy 0, policy_version 800726 (0.0005) [2023-12-26 21:11:28,028][105692] Updated weights for policy 0, policy_version 800736 (0.0005) [2023-12-26 21:11:28,106][105620] Updated weights for policy 1, policy_version 800621 (0.0007) [2023-12-26 21:11:28,165][105620] Updated weights for policy 1, policy_version 800631 (0.0009) [2023-12-26 21:11:28,216][105620] Updated weights for policy 1, policy_version 800641 (0.0010) [2023-12-26 21:11:28,640][105692] Updated weights for policy 0, policy_version 800746 (0.0006) [2023-12-26 21:11:28,688][105692] Updated weights for policy 0, policy_version 800756 (0.0008) [2023-12-26 21:11:28,732][105692] Updated weights for policy 0, policy_version 800766 (0.0007) [2023-12-26 21:11:28,785][105692] Updated weights for policy 0, policy_version 800776 (0.0008) [2023-12-26 21:11:28,993][105620] Updated weights for policy 1, policy_version 800651 (0.0010) [2023-12-26 21:11:29,040][105620] Updated weights for policy 1, policy_version 800661 (0.0008) [2023-12-26 21:11:29,087][105620] Updated weights for policy 1, policy_version 800671 (0.0009) [2023-12-26 21:11:29,545][105692] Updated weights for policy 0, policy_version 800786 (0.0005) [2023-12-26 21:11:29,608][105692] Updated weights for policy 0, policy_version 800796 (0.0007) [2023-12-26 21:11:29,663][105692] Updated weights for policy 0, policy_version 800806 (0.0009) [2023-12-26 21:11:29,906][105620] Updated weights for policy 1, policy_version 800681 (0.0008) [2023-12-26 21:11:29,966][105620] Updated weights for policy 1, policy_version 800691 (0.0009) [2023-12-26 21:11:30,019][105620] Updated weights for policy 1, policy_version 800701 (0.0008) [2023-12-26 21:11:30,079][105620] Updated weights for policy 1, policy_version 800711 (0.0010) [2023-12-26 21:11:30,312][105692] Updated weights for policy 0, policy_version 800816 (0.0007) [2023-12-26 21:11:30,367][105692] Updated weights for policy 0, policy_version 800826 (0.0009) [2023-12-26 21:11:30,414][105692] Updated weights for policy 0, policy_version 800836 (0.0009) [2023-12-26 21:11:30,820][105620] Updated weights for policy 1, policy_version 800721 (0.0008) [2023-12-26 21:11:30,879][105620] Updated weights for policy 1, policy_version 800731 (0.0010) [2023-12-26 21:11:30,930][105620] Updated weights for policy 1, policy_version 800741 (0.0010) [2023-12-26 21:11:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 410058752. Throughput: 0: 9728.8, 1: 9731.0. Samples: 410028708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:11:31,062][104569] Avg episode reward: [(0, '5379.864'), (1, '8605.544')] [2023-12-26 21:11:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000800840_205045760.pth... [2023-12-26 21:11:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000800744_205012992.pth... [2023-12-26 21:11:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000799720_204759040.pth [2023-12-26 21:11:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000799592_204718080.pth [2023-12-26 21:11:31,197][105692] Updated weights for policy 0, policy_version 800847 (0.0009) [2023-12-26 21:11:31,249][105692] Updated weights for policy 0, policy_version 800857 (0.0009) [2023-12-26 21:11:31,310][105692] Updated weights for policy 0, policy_version 800867 (0.0008) [2023-12-26 21:11:31,681][105620] Updated weights for policy 1, policy_version 800751 (0.0009) [2023-12-26 21:11:31,738][105620] Updated weights for policy 1, policy_version 800761 (0.0008) [2023-12-26 21:11:31,796][105620] Updated weights for policy 1, policy_version 800771 (0.0008) [2023-12-26 21:11:32,075][105692] Updated weights for policy 0, policy_version 800877 (0.0009) [2023-12-26 21:11:32,130][105692] Updated weights for policy 0, policy_version 800887 (0.0008) [2023-12-26 21:11:32,177][105692] Updated weights for policy 0, policy_version 800897 (0.0006) [2023-12-26 21:11:32,446][105620] Updated weights for policy 1, policy_version 800781 (0.0006) [2023-12-26 21:11:32,509][105620] Updated weights for policy 1, policy_version 800791 (0.0006) [2023-12-26 21:11:32,564][105620] Updated weights for policy 1, policy_version 800801 (0.0005) [2023-12-26 21:11:32,894][105692] Updated weights for policy 0, policy_version 800907 (0.0005) [2023-12-26 21:11:32,965][105692] Updated weights for policy 0, policy_version 800917 (0.0005) [2023-12-26 21:11:33,028][105692] Updated weights for policy 0, policy_version 800927 (0.0005) [2023-12-26 21:11:33,143][105620] Updated weights for policy 1, policy_version 800811 (0.0007) [2023-12-26 21:11:33,202][105620] Updated weights for policy 1, policy_version 800821 (0.0008) [2023-12-26 21:11:33,246][105620] Updated weights for policy 1, policy_version 800831 (0.0010) [2023-12-26 21:11:33,567][105692] Updated weights for policy 0, policy_version 800937 (0.0005) [2023-12-26 21:11:33,622][105692] Updated weights for policy 0, policy_version 800947 (0.0005) [2023-12-26 21:11:33,677][105692] Updated weights for policy 0, policy_version 800957 (0.0005) [2023-12-26 21:11:33,726][105692] Updated weights for policy 0, policy_version 800967 (0.0005) [2023-12-26 21:11:33,982][105620] Updated weights for policy 1, policy_version 800841 (0.0010) [2023-12-26 21:11:34,037][105620] Updated weights for policy 1, policy_version 800851 (0.0008) [2023-12-26 21:11:34,093][105620] Updated weights for policy 1, policy_version 800861 (0.0007) [2023-12-26 21:11:34,158][105620] Updated weights for policy 1, policy_version 800871 (0.0008) [2023-12-26 21:11:34,378][105692] Updated weights for policy 0, policy_version 800977 (0.0010) [2023-12-26 21:11:34,434][105692] Updated weights for policy 0, policy_version 800987 (0.0011) [2023-12-26 21:11:34,501][105692] Updated weights for policy 0, policy_version 800997 (0.0011) [2023-12-26 21:11:34,811][105620] Updated weights for policy 1, policy_version 800881 (0.0008) [2023-12-26 21:11:34,874][105620] Updated weights for policy 1, policy_version 800891 (0.0008) [2023-12-26 21:11:34,932][105620] Updated weights for policy 1, policy_version 800901 (0.0008) [2023-12-26 21:11:35,254][105692] Updated weights for policy 0, policy_version 801007 (0.0008) [2023-12-26 21:11:35,309][105692] Updated weights for policy 0, policy_version 801017 (0.0006) [2023-12-26 21:11:35,368][105692] Updated weights for policy 0, policy_version 801027 (0.0007) [2023-12-26 21:11:35,693][105620] Updated weights for policy 1, policy_version 800911 (0.0008) [2023-12-26 21:11:35,739][105620] Updated weights for policy 1, policy_version 800921 (0.0006) [2023-12-26 21:11:35,811][105620] Updated weights for policy 1, policy_version 800931 (0.0005) [2023-12-26 21:11:36,015][105692] Updated weights for policy 0, policy_version 801037 (0.0008) [2023-12-26 21:11:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 410157056. Throughput: 0: 9718.8, 1: 9703.7. Samples: 410148004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:11:36,063][104569] Avg episode reward: [(0, '5481.863'), (1, '8703.503')] [2023-12-26 21:11:36,079][105692] Updated weights for policy 0, policy_version 801047 (0.0005) [2023-12-26 21:11:36,143][105692] Updated weights for policy 0, policy_version 801057 (0.0007) [2023-12-26 21:11:36,366][105620] Updated weights for policy 1, policy_version 800941 (0.0006) [2023-12-26 21:11:36,418][105620] Updated weights for policy 1, policy_version 800951 (0.0005) [2023-12-26 21:11:36,476][105620] Updated weights for policy 1, policy_version 800961 (0.0006) [2023-12-26 21:11:36,710][105692] Updated weights for policy 0, policy_version 801067 (0.0009) [2023-12-26 21:11:36,769][105692] Updated weights for policy 0, policy_version 801077 (0.0010) [2023-12-26 21:11:36,834][105692] Updated weights for policy 0, policy_version 801087 (0.0010) [2023-12-26 21:11:37,174][105620] Updated weights for policy 1, policy_version 800971 (0.0009) [2023-12-26 21:11:37,218][105620] Updated weights for policy 1, policy_version 800981 (0.0008) [2023-12-26 21:11:37,266][105620] Updated weights for policy 1, policy_version 800991 (0.0008) [2023-12-26 21:11:37,601][105692] Updated weights for policy 0, policy_version 801097 (0.0011) [2023-12-26 21:11:37,669][105692] Updated weights for policy 0, policy_version 801107 (0.0011) [2023-12-26 21:11:37,727][105692] Updated weights for policy 0, policy_version 801117 (0.0011) [2023-12-26 21:11:37,789][105692] Updated weights for policy 0, policy_version 801127 (0.0011) [2023-12-26 21:11:38,077][105620] Updated weights for policy 1, policy_version 801001 (0.0008) [2023-12-26 21:11:38,135][105620] Updated weights for policy 1, policy_version 801011 (0.0008) [2023-12-26 21:11:38,193][105620] Updated weights for policy 1, policy_version 801021 (0.0007) [2023-12-26 21:11:38,240][105620] Updated weights for policy 1, policy_version 801031 (0.0008) [2023-12-26 21:11:38,547][105692] Updated weights for policy 0, policy_version 801137 (0.0011) [2023-12-26 21:11:38,606][105692] Updated weights for policy 0, policy_version 801147 (0.0011) [2023-12-26 21:11:38,661][105692] Updated weights for policy 0, policy_version 801157 (0.0011) [2023-12-26 21:11:38,997][105620] Updated weights for policy 1, policy_version 801041 (0.0008) [2023-12-26 21:11:39,040][105586] KL-divergence is very high: 111.6494 [2023-12-26 21:11:39,045][105620] Updated weights for policy 1, policy_version 801051 (0.0008) [2023-12-26 21:11:39,078][105586] KL-divergence is very high: 127.3101 [2023-12-26 21:11:39,098][105620] Updated weights for policy 1, policy_version 801061 (0.0008) [2023-12-26 21:11:39,414][105692] Updated weights for policy 0, policy_version 801167 (0.0011) [2023-12-26 21:11:39,474][105692] Updated weights for policy 0, policy_version 801177 (0.0010) [2023-12-26 21:11:39,541][105692] Updated weights for policy 0, policy_version 801187 (0.0011) [2023-12-26 21:11:39,899][105620] Updated weights for policy 1, policy_version 801071 (0.0008) [2023-12-26 21:11:39,962][105620] Updated weights for policy 1, policy_version 801081 (0.0008) [2023-12-26 21:11:40,026][105620] Updated weights for policy 1, policy_version 801091 (0.0007) [2023-12-26 21:11:40,303][105692] Updated weights for policy 0, policy_version 801197 (0.0011) [2023-12-26 21:11:40,367][105692] Updated weights for policy 0, policy_version 801207 (0.0011) [2023-12-26 21:11:40,434][105692] Updated weights for policy 0, policy_version 801217 (0.0011) [2023-12-26 21:11:40,652][105620] Updated weights for policy 1, policy_version 801101 (0.0008) [2023-12-26 21:11:40,710][105620] Updated weights for policy 1, policy_version 801111 (0.0010) [2023-12-26 21:11:40,769][105620] Updated weights for policy 1, policy_version 801121 (0.0007) [2023-12-26 21:11:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 410255360. Throughput: 0: 9717.3, 1: 9751.8. Samples: 410263808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:11:41,062][104569] Avg episode reward: [(0, '5885.311'), (1, '8993.890')] [2023-12-26 21:11:41,125][105692] Updated weights for policy 0, policy_version 801227 (0.0011) [2023-12-26 21:11:41,194][105692] Updated weights for policy 0, policy_version 801237 (0.0011) [2023-12-26 21:11:41,257][105692] Updated weights for policy 0, policy_version 801247 (0.0010) [2023-12-26 21:11:41,488][105620] Updated weights for policy 1, policy_version 801131 (0.0007) [2023-12-26 21:11:41,550][105620] Updated weights for policy 1, policy_version 801141 (0.0007) [2023-12-26 21:11:41,619][105620] Updated weights for policy 1, policy_version 801151 (0.0007) [2023-12-26 21:11:42,052][105692] Updated weights for policy 0, policy_version 801257 (0.0010) [2023-12-26 21:11:42,110][105692] Updated weights for policy 0, policy_version 801267 (0.0009) [2023-12-26 21:11:42,171][105692] Updated weights for policy 0, policy_version 801277 (0.0009) [2023-12-26 21:11:42,224][105692] Updated weights for policy 0, policy_version 801287 (0.0009) [2023-12-26 21:11:42,405][105620] Updated weights for policy 1, policy_version 801161 (0.0006) [2023-12-26 21:11:42,461][105620] Updated weights for policy 1, policy_version 801171 (0.0006) [2023-12-26 21:11:42,519][105620] Updated weights for policy 1, policy_version 801181 (0.0006) [2023-12-26 21:11:42,566][105620] Updated weights for policy 1, policy_version 801191 (0.0009) [2023-12-26 21:11:43,037][105692] Updated weights for policy 0, policy_version 801297 (0.0006) [2023-12-26 21:11:43,092][105692] Updated weights for policy 0, policy_version 801307 (0.0005) [2023-12-26 21:11:43,150][105692] Updated weights for policy 0, policy_version 801317 (0.0007) [2023-12-26 21:11:43,337][105620] Updated weights for policy 1, policy_version 801201 (0.0008) [2023-12-26 21:11:43,397][105620] Updated weights for policy 1, policy_version 801211 (0.0008) [2023-12-26 21:11:43,458][105620] Updated weights for policy 1, policy_version 801221 (0.0009) [2023-12-26 21:11:43,804][105692] Updated weights for policy 0, policy_version 801327 (0.0007) [2023-12-26 21:11:43,858][105692] Updated weights for policy 0, policy_version 801337 (0.0009) [2023-12-26 21:11:43,909][105692] Updated weights for policy 0, policy_version 801347 (0.0009) [2023-12-26 21:11:44,217][105620] Updated weights for policy 1, policy_version 801231 (0.0009) [2023-12-26 21:11:44,266][105620] Updated weights for policy 1, policy_version 801241 (0.0009) [2023-12-26 21:11:44,319][105620] Updated weights for policy 1, policy_version 801251 (0.0011) [2023-12-26 21:11:44,659][105692] Updated weights for policy 0, policy_version 801357 (0.0008) [2023-12-26 21:11:44,706][105692] Updated weights for policy 0, policy_version 801367 (0.0009) [2023-12-26 21:11:44,752][105692] Updated weights for policy 0, policy_version 801377 (0.0008) [2023-12-26 21:11:44,996][105620] Updated weights for policy 1, policy_version 801261 (0.0006) [2023-12-26 21:11:45,058][105620] Updated weights for policy 1, policy_version 801271 (0.0006) [2023-12-26 21:11:45,121][105620] Updated weights for policy 1, policy_version 801281 (0.0005) [2023-12-26 21:11:45,613][105692] Updated weights for policy 0, policy_version 801387 (0.0010) [2023-12-26 21:11:45,672][105692] Updated weights for policy 0, policy_version 801397 (0.0009) [2023-12-26 21:11:45,711][105620] Updated weights for policy 1, policy_version 801291 (0.0007) [2023-12-26 21:11:45,721][105692] Updated weights for policy 0, policy_version 801407 (0.0005) [2023-12-26 21:11:45,771][105620] Updated weights for policy 1, policy_version 801301 (0.0009) [2023-12-26 21:11:45,820][105620] Updated weights for policy 1, policy_version 801311 (0.0009) [2023-12-26 21:11:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 410353664. Throughput: 0: 9698.1, 1: 9752.4. Samples: 410319836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:11:46,062][104569] Avg episode reward: [(0, '7118.998'), (1, '9171.004')] [2023-12-26 21:11:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000801320_205160448.pth... [2023-12-26 21:11:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000801416_205193216.pth... [2023-12-26 21:11:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000800200_204873728.pth [2023-12-26 21:11:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000800264_204898304.pth [2023-12-26 21:11:46,481][105692] Updated weights for policy 0, policy_version 801417 (0.0006) [2023-12-26 21:11:46,517][105620] Updated weights for policy 1, policy_version 801321 (0.0009) [2023-12-26 21:11:46,544][105692] Updated weights for policy 0, policy_version 801427 (0.0008) [2023-12-26 21:11:46,572][105620] Updated weights for policy 1, policy_version 801331 (0.0008) [2023-12-26 21:11:46,594][105692] Updated weights for policy 0, policy_version 801437 (0.0008) [2023-12-26 21:11:46,628][105620] Updated weights for policy 1, policy_version 801341 (0.0008) [2023-12-26 21:11:46,650][105692] Updated weights for policy 0, policy_version 801447 (0.0007) [2023-12-26 21:11:46,678][105620] Updated weights for policy 1, policy_version 801351 (0.0008) [2023-12-26 21:11:47,408][105692] Updated weights for policy 0, policy_version 801457 (0.0007) [2023-12-26 21:11:47,435][105620] Updated weights for policy 1, policy_version 801361 (0.0009) [2023-12-26 21:11:47,461][105692] Updated weights for policy 0, policy_version 801467 (0.0008) [2023-12-26 21:11:47,493][105620] Updated weights for policy 1, policy_version 801371 (0.0007) [2023-12-26 21:11:47,519][105692] Updated weights for policy 0, policy_version 801477 (0.0008) [2023-12-26 21:11:47,549][105620] Updated weights for policy 1, policy_version 801381 (0.0008) [2023-12-26 21:11:48,266][105692] Updated weights for policy 0, policy_version 801487 (0.0008) [2023-12-26 21:11:48,306][105620] Updated weights for policy 1, policy_version 801391 (0.0008) [2023-12-26 21:11:48,317][105692] Updated weights for policy 0, policy_version 801497 (0.0007) [2023-12-26 21:11:48,374][105620] Updated weights for policy 1, policy_version 801401 (0.0008) [2023-12-26 21:11:48,383][105692] Updated weights for policy 0, policy_version 801507 (0.0009) [2023-12-26 21:11:48,435][105620] Updated weights for policy 1, policy_version 801411 (0.0008) [2023-12-26 21:11:49,072][105692] Updated weights for policy 0, policy_version 801517 (0.0005) [2023-12-26 21:11:49,133][105692] Updated weights for policy 0, policy_version 801527 (0.0008) [2023-12-26 21:11:49,152][105620] Updated weights for policy 1, policy_version 801421 (0.0007) [2023-12-26 21:11:49,184][105692] Updated weights for policy 0, policy_version 801537 (0.0008) [2023-12-26 21:11:49,194][105620] Updated weights for policy 1, policy_version 801431 (0.0008) [2023-12-26 21:11:49,256][105620] Updated weights for policy 1, policy_version 801441 (0.0006) [2023-12-26 21:11:49,865][105692] Updated weights for policy 0, policy_version 801547 (0.0007) [2023-12-26 21:11:49,932][105692] Updated weights for policy 0, policy_version 801557 (0.0008) [2023-12-26 21:11:49,984][105692] Updated weights for policy 0, policy_version 801567 (0.0008) [2023-12-26 21:11:50,005][105620] Updated weights for policy 1, policy_version 801451 (0.0006) [2023-12-26 21:11:50,067][105620] Updated weights for policy 1, policy_version 801461 (0.0008) [2023-12-26 21:11:50,132][105620] Updated weights for policy 1, policy_version 801471 (0.0006) [2023-12-26 21:11:50,724][105692] Updated weights for policy 0, policy_version 801577 (0.0007) [2023-12-26 21:11:50,780][105692] Updated weights for policy 0, policy_version 801587 (0.0008) [2023-12-26 21:11:50,835][105692] Updated weights for policy 0, policy_version 801597 (0.0008) [2023-12-26 21:11:50,842][105620] Updated weights for policy 1, policy_version 801481 (0.0009) [2023-12-26 21:11:50,894][105620] Updated weights for policy 1, policy_version 801491 (0.0008) [2023-12-26 21:11:50,895][105692] Updated weights for policy 0, policy_version 801607 (0.0007) [2023-12-26 21:11:50,948][105620] Updated weights for policy 1, policy_version 801501 (0.0008) [2023-12-26 21:11:51,007][105620] Updated weights for policy 1, policy_version 801511 (0.0008) [2023-12-26 21:11:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 410451968. Throughput: 0: 9689.9, 1: 9736.8. Samples: 410435056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:11:51,063][104569] Avg episode reward: [(0, '5191.930'), (1, '9261.997')] [2023-12-26 21:11:51,651][105692] Updated weights for policy 0, policy_version 801617 (0.0009) [2023-12-26 21:11:51,716][105692] Updated weights for policy 0, policy_version 801627 (0.0008) [2023-12-26 21:11:51,756][105620] Updated weights for policy 1, policy_version 801521 (0.0007) [2023-12-26 21:11:51,774][105692] Updated weights for policy 0, policy_version 801637 (0.0008) [2023-12-26 21:11:51,813][105620] Updated weights for policy 1, policy_version 801531 (0.0009) [2023-12-26 21:11:51,867][105620] Updated weights for policy 1, policy_version 801541 (0.0010) [2023-12-26 21:11:52,457][105692] Updated weights for policy 0, policy_version 801647 (0.0007) [2023-12-26 21:11:52,524][105692] Updated weights for policy 0, policy_version 801657 (0.0006) [2023-12-26 21:11:52,593][105692] Updated weights for policy 0, policy_version 801667 (0.0006) [2023-12-26 21:11:52,637][105620] Updated weights for policy 1, policy_version 801552 (0.0010) [2023-12-26 21:11:52,706][105620] Updated weights for policy 1, policy_version 801562 (0.0009) [2023-12-26 21:11:52,777][105620] Updated weights for policy 1, policy_version 801573 (0.0010) [2023-12-26 21:11:53,200][105692] Updated weights for policy 0, policy_version 801677 (0.0006) [2023-12-26 21:11:53,245][105692] Updated weights for policy 0, policy_version 801687 (0.0005) [2023-12-26 21:11:53,298][105692] Updated weights for policy 0, policy_version 801697 (0.0008) [2023-12-26 21:11:53,434][105620] Updated weights for policy 1, policy_version 801583 (0.0007) [2023-12-26 21:11:53,499][105620] Updated weights for policy 1, policy_version 801593 (0.0006) [2023-12-26 21:11:53,567][105620] Updated weights for policy 1, policy_version 801603 (0.0007) [2023-12-26 21:11:54,044][105692] Updated weights for policy 0, policy_version 801707 (0.0009) [2023-12-26 21:11:54,089][105692] Updated weights for policy 0, policy_version 801717 (0.0008) [2023-12-26 21:11:54,143][105692] Updated weights for policy 0, policy_version 801727 (0.0006) [2023-12-26 21:11:54,261][105620] Updated weights for policy 1, policy_version 801613 (0.0011) [2023-12-26 21:11:54,307][105620] Updated weights for policy 1, policy_version 801623 (0.0011) [2023-12-26 21:11:54,360][105620] Updated weights for policy 1, policy_version 801633 (0.0010) [2023-12-26 21:11:54,948][105692] Updated weights for policy 0, policy_version 801737 (0.0008) [2023-12-26 21:11:55,013][105692] Updated weights for policy 0, policy_version 801747 (0.0008) [2023-12-26 21:11:55,076][105692] Updated weights for policy 0, policy_version 801757 (0.0008) [2023-12-26 21:11:55,135][105692] Updated weights for policy 0, policy_version 801767 (0.0008) [2023-12-26 21:11:55,144][105620] Updated weights for policy 1, policy_version 801643 (0.0010) [2023-12-26 21:11:55,202][105620] Updated weights for policy 1, policy_version 801653 (0.0008) [2023-12-26 21:11:55,260][105620] Updated weights for policy 1, policy_version 801663 (0.0010) [2023-12-26 21:11:55,910][105692] Updated weights for policy 0, policy_version 801777 (0.0008) [2023-12-26 21:11:55,969][105692] Updated weights for policy 0, policy_version 801787 (0.0008) [2023-12-26 21:11:55,971][105620] Updated weights for policy 1, policy_version 801673 (0.0008) [2023-12-26 21:11:56,029][105692] Updated weights for policy 0, policy_version 801797 (0.0005) [2023-12-26 21:11:56,030][105620] Updated weights for policy 1, policy_version 801683 (0.0011) [2023-12-26 21:11:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 410542080. Throughput: 0: 9767.6, 1: 9651.4. Samples: 410550540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:11:56,063][104569] Avg episode reward: [(0, '5770.607'), (1, '9352.629')] [2023-12-26 21:11:56,087][105620] Updated weights for policy 1, policy_version 801693 (0.0009) [2023-12-26 21:11:56,145][105620] Updated weights for policy 1, policy_version 801703 (0.0006) [2023-12-26 21:11:56,799][105692] Updated weights for policy 0, policy_version 801807 (0.0008) [2023-12-26 21:11:56,830][105620] Updated weights for policy 1, policy_version 801713 (0.0006) [2023-12-26 21:11:56,864][105692] Updated weights for policy 0, policy_version 801817 (0.0008) [2023-12-26 21:11:56,878][105620] Updated weights for policy 1, policy_version 801723 (0.0005) [2023-12-26 21:11:56,927][105692] Updated weights for policy 0, policy_version 801827 (0.0009) [2023-12-26 21:11:56,927][105620] Updated weights for policy 1, policy_version 801733 (0.0007) [2023-12-26 21:11:57,583][105692] Updated weights for policy 0, policy_version 801837 (0.0009) [2023-12-26 21:11:57,630][105692] Updated weights for policy 0, policy_version 801847 (0.0005) [2023-12-26 21:11:57,673][105620] Updated weights for policy 1, policy_version 801743 (0.0009) [2023-12-26 21:11:57,675][105692] Updated weights for policy 0, policy_version 801857 (0.0005) [2023-12-26 21:11:57,730][105620] Updated weights for policy 1, policy_version 801753 (0.0009) [2023-12-26 21:11:57,784][105620] Updated weights for policy 1, policy_version 801765 (0.0010) [2023-12-26 21:11:58,242][105692] Updated weights for policy 0, policy_version 801867 (0.0006) [2023-12-26 21:11:58,304][105692] Updated weights for policy 0, policy_version 801877 (0.0009) [2023-12-26 21:11:58,369][105692] Updated weights for policy 0, policy_version 801887 (0.0008) [2023-12-26 21:11:58,668][105620] Updated weights for policy 1, policy_version 801775 (0.0008) [2023-12-26 21:11:58,725][105620] Updated weights for policy 1, policy_version 801785 (0.0008) [2023-12-26 21:11:58,793][105620] Updated weights for policy 1, policy_version 801795 (0.0008) [2023-12-26 21:11:59,165][105692] Updated weights for policy 0, policy_version 801897 (0.0008) [2023-12-26 21:11:59,225][105692] Updated weights for policy 0, policy_version 801907 (0.0007) [2023-12-26 21:11:59,288][105692] Updated weights for policy 0, policy_version 801917 (0.0010) [2023-12-26 21:11:59,351][105692] Updated weights for policy 0, policy_version 801927 (0.0011) [2023-12-26 21:11:59,532][105620] Updated weights for policy 1, policy_version 801805 (0.0010) [2023-12-26 21:11:59,591][105620] Updated weights for policy 1, policy_version 801815 (0.0011) [2023-12-26 21:11:59,644][105620] Updated weights for policy 1, policy_version 801825 (0.0010) [2023-12-26 21:12:00,045][105692] Updated weights for policy 0, policy_version 801937 (0.0011) [2023-12-26 21:12:00,093][105692] Updated weights for policy 0, policy_version 801947 (0.0010) [2023-12-26 21:12:00,147][105692] Updated weights for policy 0, policy_version 801957 (0.0008) [2023-12-26 21:12:00,365][105620] Updated weights for policy 1, policy_version 801835 (0.0009) [2023-12-26 21:12:00,435][105620] Updated weights for policy 1, policy_version 801845 (0.0007) [2023-12-26 21:12:00,490][105620] Updated weights for policy 1, policy_version 801855 (0.0009) [2023-12-26 21:12:00,780][105692] Updated weights for policy 0, policy_version 801967 (0.0008) [2023-12-26 21:12:00,829][105692] Updated weights for policy 0, policy_version 801977 (0.0010) [2023-12-26 21:12:00,886][105692] Updated weights for policy 0, policy_version 801987 (0.0010) [2023-12-26 21:12:01,038][105620] Updated weights for policy 1, policy_version 801865 (0.0010) [2023-12-26 21:12:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 410640384. Throughput: 0: 9790.1, 1: 9582.4. Samples: 410607892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:12:01,063][104569] Avg episode reward: [(0, '6731.782'), (1, '9260.474')] [2023-12-26 21:12:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000801992_205340672.pth... [2023-12-26 21:12:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000800840_205045760.pth [2023-12-26 21:12:01,102][105620] Updated weights for policy 1, policy_version 801875 (0.0008) [2023-12-26 21:12:01,170][105620] Updated weights for policy 1, policy_version 801885 (0.0014) [2023-12-26 21:12:01,219][105620] Updated weights for policy 1, policy_version 801895 (0.0008) [2023-12-26 21:12:01,222][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000801896_205307904.pth... [2023-12-26 21:12:01,224][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000800744_205012992.pth [2023-12-26 21:12:01,633][105692] Updated weights for policy 0, policy_version 801997 (0.0009) [2023-12-26 21:12:01,691][105692] Updated weights for policy 0, policy_version 802007 (0.0008) [2023-12-26 21:12:01,756][105692] Updated weights for policy 0, policy_version 802017 (0.0009) [2023-12-26 21:12:01,921][105620] Updated weights for policy 1, policy_version 801905 (0.0010) [2023-12-26 21:12:01,979][105620] Updated weights for policy 1, policy_version 801915 (0.0010) [2023-12-26 21:12:02,042][105620] Updated weights for policy 1, policy_version 801925 (0.0010) [2023-12-26 21:12:02,518][105692] Updated weights for policy 0, policy_version 802027 (0.0009) [2023-12-26 21:12:02,577][105692] Updated weights for policy 0, policy_version 802037 (0.0009) [2023-12-26 21:12:02,633][105692] Updated weights for policy 0, policy_version 802047 (0.0008) [2023-12-26 21:12:02,751][105620] Updated weights for policy 1, policy_version 801935 (0.0008) [2023-12-26 21:12:02,799][105620] Updated weights for policy 1, policy_version 801945 (0.0008) [2023-12-26 21:12:02,847][105620] Updated weights for policy 1, policy_version 801955 (0.0007) [2023-12-26 21:12:03,286][105692] Updated weights for policy 0, policy_version 802057 (0.0009) [2023-12-26 21:12:03,337][105692] Updated weights for policy 0, policy_version 802067 (0.0005) [2023-12-26 21:12:03,383][105692] Updated weights for policy 0, policy_version 802077 (0.0005) [2023-12-26 21:12:03,431][105692] Updated weights for policy 0, policy_version 802087 (0.0007) [2023-12-26 21:12:03,701][105620] Updated weights for policy 1, policy_version 801965 (0.0008) [2023-12-26 21:12:03,760][105620] Updated weights for policy 1, policy_version 801975 (0.0009) [2023-12-26 21:12:03,823][105620] Updated weights for policy 1, policy_version 801985 (0.0009) [2023-12-26 21:12:04,104][105692] Updated weights for policy 0, policy_version 802097 (0.0009) [2023-12-26 21:12:04,167][105692] Updated weights for policy 0, policy_version 802107 (0.0008) [2023-12-26 21:12:04,231][105692] Updated weights for policy 0, policy_version 802117 (0.0009) [2023-12-26 21:12:04,611][105620] Updated weights for policy 1, policy_version 801995 (0.0009) [2023-12-26 21:12:04,665][105620] Updated weights for policy 1, policy_version 802005 (0.0010) [2023-12-26 21:12:04,722][105620] Updated weights for policy 1, policy_version 802015 (0.0009) [2023-12-26 21:12:04,905][105692] Updated weights for policy 0, policy_version 802127 (0.0009) [2023-12-26 21:12:04,965][105692] Updated weights for policy 0, policy_version 802137 (0.0009) [2023-12-26 21:12:05,020][105692] Updated weights for policy 0, policy_version 802147 (0.0009) [2023-12-26 21:12:05,401][105620] Updated weights for policy 1, policy_version 802025 (0.0009) [2023-12-26 21:12:05,464][105620] Updated weights for policy 1, policy_version 802035 (0.0006) [2023-12-26 21:12:05,526][105620] Updated weights for policy 1, policy_version 802045 (0.0008) [2023-12-26 21:12:05,573][105620] Updated weights for policy 1, policy_version 802055 (0.0008) [2023-12-26 21:12:05,663][105692] Updated weights for policy 0, policy_version 802157 (0.0008) [2023-12-26 21:12:05,709][105692] Updated weights for policy 0, policy_version 802167 (0.0005) [2023-12-26 21:12:05,755][105692] Updated weights for policy 0, policy_version 802177 (0.0005) [2023-12-26 21:12:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19383.1). Total num frames: 410738688. Throughput: 0: 9757.6, 1: 9614.6. Samples: 410725236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:12:06,063][104569] Avg episode reward: [(0, '7885.282'), (1, '9077.206')] [2023-12-26 21:12:06,359][105620] Updated weights for policy 1, policy_version 802065 (0.0009) [2023-12-26 21:12:06,413][105620] Updated weights for policy 1, policy_version 802075 (0.0009) [2023-12-26 21:12:06,419][105692] Updated weights for policy 0, policy_version 802187 (0.0006) [2023-12-26 21:12:06,470][105620] Updated weights for policy 1, policy_version 802085 (0.0006) [2023-12-26 21:12:06,476][105692] Updated weights for policy 0, policy_version 802197 (0.0008) [2023-12-26 21:12:06,530][105692] Updated weights for policy 0, policy_version 802207 (0.0008) [2023-12-26 21:12:07,266][105620] Updated weights for policy 1, policy_version 802095 (0.0009) [2023-12-26 21:12:07,272][105692] Updated weights for policy 0, policy_version 802217 (0.0010) [2023-12-26 21:12:07,326][105692] Updated weights for policy 0, policy_version 802227 (0.0006) [2023-12-26 21:12:07,330][105620] Updated weights for policy 1, policy_version 802105 (0.0007) [2023-12-26 21:12:07,382][105692] Updated weights for policy 0, policy_version 802237 (0.0007) [2023-12-26 21:12:07,384][105620] Updated weights for policy 1, policy_version 802115 (0.0007) [2023-12-26 21:12:07,429][105692] Updated weights for policy 0, policy_version 802247 (0.0008) [2023-12-26 21:12:08,125][105620] Updated weights for policy 1, policy_version 802125 (0.0009) [2023-12-26 21:12:08,181][105620] Updated weights for policy 1, policy_version 802135 (0.0008) [2023-12-26 21:12:08,193][105692] Updated weights for policy 0, policy_version 802257 (0.0007) [2023-12-26 21:12:08,239][105620] Updated weights for policy 1, policy_version 802145 (0.0008) [2023-12-26 21:12:08,253][105692] Updated weights for policy 0, policy_version 802267 (0.0007) [2023-12-26 21:12:08,316][105692] Updated weights for policy 0, policy_version 802277 (0.0006) [2023-12-26 21:12:09,037][105620] Updated weights for policy 1, policy_version 802155 (0.0008) [2023-12-26 21:12:09,040][105692] Updated weights for policy 0, policy_version 802287 (0.0007) [2023-12-26 21:12:09,088][105692] Updated weights for policy 0, policy_version 802297 (0.0008) [2023-12-26 21:12:09,099][105620] Updated weights for policy 1, policy_version 802165 (0.0007) [2023-12-26 21:12:09,141][105692] Updated weights for policy 0, policy_version 802307 (0.0009) [2023-12-26 21:12:09,161][105620] Updated weights for policy 1, policy_version 802175 (0.0007) [2023-12-26 21:12:09,879][105620] Updated weights for policy 1, policy_version 802185 (0.0009) [2023-12-26 21:12:09,945][105620] Updated weights for policy 1, policy_version 802195 (0.0008) [2023-12-26 21:12:09,959][105692] Updated weights for policy 0, policy_version 802317 (0.0009) [2023-12-26 21:12:10,009][105620] Updated weights for policy 1, policy_version 802205 (0.0005) [2023-12-26 21:12:10,021][105692] Updated weights for policy 0, policy_version 802327 (0.0008) [2023-12-26 21:12:10,074][105620] Updated weights for policy 1, policy_version 802215 (0.0006) [2023-12-26 21:12:10,082][105692] Updated weights for policy 0, policy_version 802337 (0.0006) [2023-12-26 21:12:10,742][105692] Updated weights for policy 0, policy_version 802347 (0.0007) [2023-12-26 21:12:10,792][105692] Updated weights for policy 0, policy_version 802357 (0.0007) [2023-12-26 21:12:10,804][105620] Updated weights for policy 1, policy_version 802225 (0.0007) [2023-12-26 21:12:10,838][105692] Updated weights for policy 0, policy_version 802367 (0.0007) [2023-12-26 21:12:10,862][105620] Updated weights for policy 1, policy_version 802235 (0.0008) [2023-12-26 21:12:10,913][105620] Updated weights for policy 1, policy_version 802245 (0.0005) [2023-12-26 21:12:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 410836992. Throughput: 0: 9764.6, 1: 9655.0. Samples: 410840832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:12:11,062][104569] Avg episode reward: [(0, '8252.921'), (1, '9077.664')] [2023-12-26 21:12:11,643][105620] Updated weights for policy 1, policy_version 802255 (0.0008) [2023-12-26 21:12:11,700][105620] Updated weights for policy 1, policy_version 802265 (0.0009) [2023-12-26 21:12:11,730][105692] Updated weights for policy 0, policy_version 802377 (0.0009) [2023-12-26 21:12:11,769][105620] Updated weights for policy 1, policy_version 802275 (0.0008) [2023-12-26 21:12:11,800][105692] Updated weights for policy 0, policy_version 802387 (0.0007) [2023-12-26 21:12:11,861][105692] Updated weights for policy 0, policy_version 802397 (0.0009) [2023-12-26 21:12:11,919][105692] Updated weights for policy 0, policy_version 802407 (0.0007) [2023-12-26 21:12:12,526][105692] Updated weights for policy 0, policy_version 802417 (0.0008) [2023-12-26 21:12:12,571][105692] Updated weights for policy 0, policy_version 802427 (0.0008) [2023-12-26 21:12:12,614][105620] Updated weights for policy 1, policy_version 802285 (0.0008) [2023-12-26 21:12:12,633][105692] Updated weights for policy 0, policy_version 802437 (0.0007) [2023-12-26 21:12:12,670][105620] Updated weights for policy 1, policy_version 802295 (0.0007) [2023-12-26 21:12:12,725][105620] Updated weights for policy 1, policy_version 802305 (0.0009) [2023-12-26 21:12:13,344][105692] Updated weights for policy 0, policy_version 802447 (0.0007) [2023-12-26 21:12:13,410][105692] Updated weights for policy 0, policy_version 802457 (0.0008) [2023-12-26 21:12:13,471][105692] Updated weights for policy 0, policy_version 802467 (0.0010) [2023-12-26 21:12:13,494][105620] Updated weights for policy 1, policy_version 802315 (0.0008) [2023-12-26 21:12:13,550][105620] Updated weights for policy 1, policy_version 802325 (0.0006) [2023-12-26 21:12:13,602][105620] Updated weights for policy 1, policy_version 802335 (0.0011) [2023-12-26 21:12:14,170][105620] Updated weights for policy 1, policy_version 802345 (0.0011) [2023-12-26 21:12:14,231][105620] Updated weights for policy 1, policy_version 802355 (0.0010) [2023-12-26 21:12:14,286][105620] Updated weights for policy 1, policy_version 802365 (0.0010) [2023-12-26 21:12:14,303][105692] Updated weights for policy 0, policy_version 802477 (0.0007) [2023-12-26 21:12:14,341][105620] Updated weights for policy 1, policy_version 802375 (0.0010) [2023-12-26 21:12:14,362][105692] Updated weights for policy 0, policy_version 802487 (0.0006) [2023-12-26 21:12:14,419][105692] Updated weights for policy 0, policy_version 802497 (0.0006) [2023-12-26 21:12:15,073][105620] Updated weights for policy 1, policy_version 802385 (0.0011) [2023-12-26 21:12:15,127][105692] Updated weights for policy 0, policy_version 802507 (0.0005) [2023-12-26 21:12:15,129][105620] Updated weights for policy 1, policy_version 802395 (0.0010) [2023-12-26 21:12:15,180][105692] Updated weights for policy 0, policy_version 802517 (0.0005) [2023-12-26 21:12:15,189][105620] Updated weights for policy 1, policy_version 802405 (0.0011) [2023-12-26 21:12:15,239][105692] Updated weights for policy 0, policy_version 802527 (0.0007) [2023-12-26 21:12:15,873][105620] Updated weights for policy 1, policy_version 802415 (0.0009) [2023-12-26 21:12:15,919][105620] Updated weights for policy 1, policy_version 802425 (0.0009) [2023-12-26 21:12:15,970][105620] Updated weights for policy 1, policy_version 802435 (0.0009) [2023-12-26 21:12:16,015][105692] Updated weights for policy 0, policy_version 802537 (0.0008) [2023-12-26 21:12:16,060][105692] Updated weights for policy 0, policy_version 802547 (0.0008) [2023-12-26 21:12:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 410927104. Throughput: 0: 9703.3, 1: 9604.0. Samples: 410897536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:12:16,063][104569] Avg episode reward: [(0, '8417.080'), (1, '9173.497')] [2023-12-26 21:12:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000802440_205447168.pth... [2023-12-26 21:12:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000801320_205160448.pth [2023-12-26 21:12:16,113][105692] Updated weights for policy 0, policy_version 802557 (0.0009) [2023-12-26 21:12:16,167][105692] Updated weights for policy 0, policy_version 802567 (0.0007) [2023-12-26 21:12:16,170][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000802568_205488128.pth... [2023-12-26 21:12:16,173][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000801416_205193216.pth [2023-12-26 21:12:16,614][105620] Updated weights for policy 1, policy_version 802445 (0.0009) [2023-12-26 21:12:16,664][105620] Updated weights for policy 1, policy_version 802455 (0.0008) [2023-12-26 21:12:16,710][105620] Updated weights for policy 1, policy_version 802465 (0.0008) [2023-12-26 21:12:16,952][105692] Updated weights for policy 0, policy_version 802577 (0.0008) [2023-12-26 21:12:17,006][105692] Updated weights for policy 0, policy_version 802587 (0.0008) [2023-12-26 21:12:17,059][105692] Updated weights for policy 0, policy_version 802597 (0.0008) [2023-12-26 21:12:17,332][105620] Updated weights for policy 1, policy_version 802475 (0.0007) [2023-12-26 21:12:17,386][105620] Updated weights for policy 1, policy_version 802485 (0.0008) [2023-12-26 21:12:17,437][105620] Updated weights for policy 1, policy_version 802495 (0.0009) [2023-12-26 21:12:17,888][105692] Updated weights for policy 0, policy_version 802607 (0.0009) [2023-12-26 21:12:17,950][105692] Updated weights for policy 0, policy_version 802617 (0.0009) [2023-12-26 21:12:18,011][105692] Updated weights for policy 0, policy_version 802627 (0.0009) [2023-12-26 21:12:18,094][105620] Updated weights for policy 1, policy_version 802505 (0.0009) [2023-12-26 21:12:18,145][105620] Updated weights for policy 1, policy_version 802515 (0.0009) [2023-12-26 21:12:18,206][105620] Updated weights for policy 1, policy_version 802525 (0.0009) [2023-12-26 21:12:18,268][105620] Updated weights for policy 1, policy_version 802535 (0.0009) [2023-12-26 21:12:18,723][105692] Updated weights for policy 0, policy_version 802637 (0.0008) [2023-12-26 21:12:18,786][105692] Updated weights for policy 0, policy_version 802647 (0.0008) [2023-12-26 21:12:18,852][105692] Updated weights for policy 0, policy_version 802657 (0.0008) [2023-12-26 21:12:19,034][105620] Updated weights for policy 1, policy_version 802545 (0.0009) [2023-12-26 21:12:19,086][105620] Updated weights for policy 1, policy_version 802555 (0.0009) [2023-12-26 21:12:19,141][105620] Updated weights for policy 1, policy_version 802565 (0.0009) [2023-12-26 21:12:19,581][105692] Updated weights for policy 0, policy_version 802667 (0.0009) [2023-12-26 21:12:19,642][105692] Updated weights for policy 0, policy_version 802677 (0.0009) [2023-12-26 21:12:19,693][105692] Updated weights for policy 0, policy_version 802687 (0.0008) [2023-12-26 21:12:19,975][105620] Updated weights for policy 1, policy_version 802575 (0.0008) [2023-12-26 21:12:20,033][105620] Updated weights for policy 1, policy_version 802585 (0.0010) [2023-12-26 21:12:20,099][105620] Updated weights for policy 1, policy_version 802595 (0.0010) [2023-12-26 21:12:20,389][105692] Updated weights for policy 0, policy_version 802697 (0.0006) [2023-12-26 21:12:20,446][105692] Updated weights for policy 0, policy_version 802707 (0.0009) [2023-12-26 21:12:20,514][105692] Updated weights for policy 0, policy_version 802717 (0.0006) [2023-12-26 21:12:20,582][105692] Updated weights for policy 0, policy_version 802727 (0.0006) [2023-12-26 21:12:20,899][105620] Updated weights for policy 1, policy_version 802605 (0.0010) [2023-12-26 21:12:20,950][105620] Updated weights for policy 1, policy_version 802615 (0.0010) [2023-12-26 21:12:20,997][105620] Updated weights for policy 1, policy_version 802625 (0.0008) [2023-12-26 21:12:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19355.3). Total num frames: 411025408. Throughput: 0: 9600.0, 1: 9622.5. Samples: 411013016. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 21:12:21,062][104569] Avg episode reward: [(0, '9081.827'), (1, '9176.458')] [2023-12-26 21:12:21,293][105692] Updated weights for policy 0, policy_version 802737 (0.0008) [2023-12-26 21:12:21,364][105692] Updated weights for policy 0, policy_version 802747 (0.0009) [2023-12-26 21:12:21,433][105692] Updated weights for policy 0, policy_version 802757 (0.0008) [2023-12-26 21:12:21,839][105620] Updated weights for policy 1, policy_version 802635 (0.0009) [2023-12-26 21:12:21,898][105620] Updated weights for policy 1, policy_version 802645 (0.0010) [2023-12-26 21:12:21,964][105620] Updated weights for policy 1, policy_version 802655 (0.0010) [2023-12-26 21:12:22,211][105692] Updated weights for policy 0, policy_version 802767 (0.0008) [2023-12-26 21:12:22,264][105692] Updated weights for policy 0, policy_version 802777 (0.0008) [2023-12-26 21:12:22,324][105692] Updated weights for policy 0, policy_version 802787 (0.0007) [2023-12-26 21:12:22,736][105620] Updated weights for policy 1, policy_version 802665 (0.0011) [2023-12-26 21:12:22,798][105620] Updated weights for policy 1, policy_version 802675 (0.0010) [2023-12-26 21:12:22,861][105620] Updated weights for policy 1, policy_version 802685 (0.0010) [2023-12-26 21:12:22,923][105620] Updated weights for policy 1, policy_version 802695 (0.0010) [2023-12-26 21:12:23,109][105692] Updated weights for policy 0, policy_version 802797 (0.0008) [2023-12-26 21:12:23,158][105692] Updated weights for policy 0, policy_version 802807 (0.0008) [2023-12-26 21:12:23,214][105692] Updated weights for policy 0, policy_version 802817 (0.0008) [2023-12-26 21:12:23,660][105620] Updated weights for policy 1, policy_version 802705 (0.0010) [2023-12-26 21:12:23,712][105620] Updated weights for policy 1, policy_version 802715 (0.0010) [2023-12-26 21:12:23,759][105620] Updated weights for policy 1, policy_version 802725 (0.0010) [2023-12-26 21:12:23,994][105692] Updated weights for policy 0, policy_version 802827 (0.0009) [2023-12-26 21:12:24,038][105692] Updated weights for policy 0, policy_version 802837 (0.0010) [2023-12-26 21:12:24,086][105692] Updated weights for policy 0, policy_version 802847 (0.0010) [2023-12-26 21:12:24,426][105620] Updated weights for policy 1, policy_version 802735 (0.0007) [2023-12-26 21:12:24,492][105620] Updated weights for policy 1, policy_version 802745 (0.0005) [2023-12-26 21:12:24,566][105620] Updated weights for policy 1, policy_version 802755 (0.0006) [2023-12-26 21:12:24,852][105692] Updated weights for policy 0, policy_version 802857 (0.0010) [2023-12-26 21:12:24,917][105692] Updated weights for policy 0, policy_version 802867 (0.0011) [2023-12-26 21:12:24,984][105692] Updated weights for policy 0, policy_version 802877 (0.0011) [2023-12-26 21:12:25,040][105692] Updated weights for policy 0, policy_version 802887 (0.0010) [2023-12-26 21:12:25,257][105620] Updated weights for policy 1, policy_version 802765 (0.0011) [2023-12-26 21:12:25,323][105620] Updated weights for policy 1, policy_version 802775 (0.0010) [2023-12-26 21:12:25,378][105620] Updated weights for policy 1, policy_version 802785 (0.0010) [2023-12-26 21:12:25,751][105692] Updated weights for policy 0, policy_version 802897 (0.0009) [2023-12-26 21:12:25,806][105692] Updated weights for policy 0, policy_version 802907 (0.0011) [2023-12-26 21:12:25,858][105692] Updated weights for policy 0, policy_version 802917 (0.0008) [2023-12-26 21:12:26,060][105620] Updated weights for policy 1, policy_version 802795 (0.0008) [2023-12-26 21:12:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19327.6). Total num frames: 411115520. Throughput: 0: 9560.1, 1: 9578.0. Samples: 411125024. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 21:12:26,062][104569] Avg episode reward: [(0, '9169.670'), (1, '9081.312')] [2023-12-26 21:12:26,125][105620] Updated weights for policy 1, policy_version 802805 (0.0010) [2023-12-26 21:12:26,190][105620] Updated weights for policy 1, policy_version 802815 (0.0010) [2023-12-26 21:12:26,463][105692] Updated weights for policy 0, policy_version 802927 (0.0005) [2023-12-26 21:12:26,511][105692] Updated weights for policy 0, policy_version 802937 (0.0009) [2023-12-26 21:12:26,563][105692] Updated weights for policy 0, policy_version 802947 (0.0009) [2023-12-26 21:12:26,821][105620] Updated weights for policy 1, policy_version 802825 (0.0010) [2023-12-26 21:12:26,881][105620] Updated weights for policy 1, policy_version 802835 (0.0009) [2023-12-26 21:12:26,943][105620] Updated weights for policy 1, policy_version 802846 (0.0010) [2023-12-26 21:12:27,005][105620] Updated weights for policy 1, policy_version 802856 (0.0010) [2023-12-26 21:12:27,182][105692] Updated weights for policy 0, policy_version 802957 (0.0005) [2023-12-26 21:12:27,231][105692] Updated weights for policy 0, policy_version 802967 (0.0005) [2023-12-26 21:12:27,283][105692] Updated weights for policy 0, policy_version 802977 (0.0007) [2023-12-26 21:12:27,724][105620] Updated weights for policy 1, policy_version 802866 (0.0009) [2023-12-26 21:12:27,777][105620] Updated weights for policy 1, policy_version 802876 (0.0009) [2023-12-26 21:12:27,829][105620] Updated weights for policy 1, policy_version 802886 (0.0008) [2023-12-26 21:12:27,903][105692] Updated weights for policy 0, policy_version 802987 (0.0008) [2023-12-26 21:12:27,954][105692] Updated weights for policy 0, policy_version 802997 (0.0005) [2023-12-26 21:12:28,010][105692] Updated weights for policy 0, policy_version 803007 (0.0005) [2023-12-26 21:12:28,578][105692] Updated weights for policy 0, policy_version 803017 (0.0005) [2023-12-26 21:12:28,591][105620] Updated weights for policy 1, policy_version 802896 (0.0007) [2023-12-26 21:12:28,644][105692] Updated weights for policy 0, policy_version 803027 (0.0006) [2023-12-26 21:12:28,656][105620] Updated weights for policy 1, policy_version 802906 (0.0007) [2023-12-26 21:12:28,709][105692] Updated weights for policy 0, policy_version 803037 (0.0006) [2023-12-26 21:12:28,717][105620] Updated weights for policy 1, policy_version 802916 (0.0007) [2023-12-26 21:12:28,770][105692] Updated weights for policy 0, policy_version 803047 (0.0005) [2023-12-26 21:12:29,319][105692] Updated weights for policy 0, policy_version 803057 (0.0010) [2023-12-26 21:12:29,376][105692] Updated weights for policy 0, policy_version 803067 (0.0008) [2023-12-26 21:12:29,426][105692] Updated weights for policy 0, policy_version 803077 (0.0008) [2023-12-26 21:12:29,481][105620] Updated weights for policy 1, policy_version 802926 (0.0009) [2023-12-26 21:12:29,536][105620] Updated weights for policy 1, policy_version 802936 (0.0007) [2023-12-26 21:12:29,588][105620] Updated weights for policy 1, policy_version 802946 (0.0005) [2023-12-26 21:12:30,116][105692] Updated weights for policy 0, policy_version 803087 (0.0010) [2023-12-26 21:12:30,164][105692] Updated weights for policy 0, policy_version 803097 (0.0010) [2023-12-26 21:12:30,213][105692] Updated weights for policy 0, policy_version 803107 (0.0010) [2023-12-26 21:12:30,379][105620] Updated weights for policy 1, policy_version 802956 (0.0007) [2023-12-26 21:12:30,437][105620] Updated weights for policy 1, policy_version 802966 (0.0009) [2023-12-26 21:12:30,490][105620] Updated weights for policy 1, policy_version 802976 (0.0009) [2023-12-26 21:12:30,909][105692] Updated weights for policy 0, policy_version 803117 (0.0009) [2023-12-26 21:12:30,953][105692] Updated weights for policy 0, policy_version 803127 (0.0010) [2023-12-26 21:12:31,003][105692] Updated weights for policy 0, policy_version 803137 (0.0010) [2023-12-26 21:12:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 411222016. Throughput: 0: 9690.3, 1: 9604.7. Samples: 411188112. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 21:12:31,062][104569] Avg episode reward: [(0, '9255.419'), (1, '9169.753')] [2023-12-26 21:12:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000802984_205586432.pth... [2023-12-26 21:12:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000803144_205635584.pth... [2023-12-26 21:12:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000801896_205307904.pth [2023-12-26 21:12:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000801992_205340672.pth [2023-12-26 21:12:31,256][105620] Updated weights for policy 1, policy_version 802986 (0.0006) [2023-12-26 21:12:31,308][105620] Updated weights for policy 1, policy_version 802996 (0.0008) [2023-12-26 21:12:31,369][105620] Updated weights for policy 1, policy_version 803006 (0.0008) [2023-12-26 21:12:31,428][105620] Updated weights for policy 1, policy_version 803016 (0.0008) [2023-12-26 21:12:31,793][105692] Updated weights for policy 0, policy_version 803147 (0.0010) [2023-12-26 21:12:31,848][105692] Updated weights for policy 0, policy_version 803157 (0.0009) [2023-12-26 21:12:31,896][105692] Updated weights for policy 0, policy_version 803167 (0.0009) [2023-12-26 21:12:32,187][105620] Updated weights for policy 1, policy_version 803026 (0.0010) [2023-12-26 21:12:32,240][105620] Updated weights for policy 1, policy_version 803036 (0.0009) [2023-12-26 21:12:32,301][105620] Updated weights for policy 1, policy_version 803046 (0.0010) [2023-12-26 21:12:32,588][105692] Updated weights for policy 0, policy_version 803177 (0.0009) [2023-12-26 21:12:32,641][105692] Updated weights for policy 0, policy_version 803187 (0.0010) [2023-12-26 21:12:32,699][105692] Updated weights for policy 0, policy_version 803197 (0.0009) [2023-12-26 21:12:32,756][105692] Updated weights for policy 0, policy_version 803207 (0.0009) [2023-12-26 21:12:32,999][105620] Updated weights for policy 1, policy_version 803056 (0.0009) [2023-12-26 21:12:33,063][105620] Updated weights for policy 1, policy_version 803066 (0.0009) [2023-12-26 21:12:33,113][105620] Updated weights for policy 1, policy_version 803076 (0.0008) [2023-12-26 21:12:33,517][105692] Updated weights for policy 0, policy_version 803217 (0.0009) [2023-12-26 21:12:33,563][105692] Updated weights for policy 0, policy_version 803227 (0.0009) [2023-12-26 21:12:33,609][105692] Updated weights for policy 0, policy_version 803237 (0.0008) [2023-12-26 21:12:33,806][105620] Updated weights for policy 1, policy_version 803086 (0.0007) [2023-12-26 21:12:33,853][105620] Updated weights for policy 1, policy_version 803096 (0.0006) [2023-12-26 21:12:33,900][105620] Updated weights for policy 1, policy_version 803106 (0.0009) [2023-12-26 21:12:34,430][105692] Updated weights for policy 0, policy_version 803247 (0.0009) [2023-12-26 21:12:34,490][105692] Updated weights for policy 0, policy_version 803257 (0.0009) [2023-12-26 21:12:34,552][105692] Updated weights for policy 0, policy_version 803267 (0.0009) [2023-12-26 21:12:34,678][105620] Updated weights for policy 1, policy_version 803116 (0.0009) [2023-12-26 21:12:34,749][105620] Updated weights for policy 1, policy_version 803126 (0.0009) [2023-12-26 21:12:34,811][105620] Updated weights for policy 1, policy_version 803136 (0.0009) [2023-12-26 21:12:35,315][105692] Updated weights for policy 0, policy_version 803277 (0.0007) [2023-12-26 21:12:35,369][105692] Updated weights for policy 0, policy_version 803287 (0.0005) [2023-12-26 21:12:35,426][105692] Updated weights for policy 0, policy_version 803297 (0.0005) [2023-12-26 21:12:35,520][105620] Updated weights for policy 1, policy_version 803146 (0.0007) [2023-12-26 21:12:35,571][105620] Updated weights for policy 1, policy_version 803156 (0.0005) [2023-12-26 21:12:35,622][105620] Updated weights for policy 1, policy_version 803166 (0.0005) [2023-12-26 21:12:35,674][105620] Updated weights for policy 1, policy_version 803176 (0.0006) [2023-12-26 21:12:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19355.3). Total num frames: 411312128. Throughput: 0: 9755.5, 1: 9542.7. Samples: 411303472. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 21:12:36,062][104569] Avg episode reward: [(0, '9170.800'), (1, '8990.863')] [2023-12-26 21:12:36,103][105692] Updated weights for policy 0, policy_version 803307 (0.0008) [2023-12-26 21:12:36,173][105692] Updated weights for policy 0, policy_version 803317 (0.0010) [2023-12-26 21:12:36,230][105692] Updated weights for policy 0, policy_version 803327 (0.0011) [2023-12-26 21:12:36,407][105620] Updated weights for policy 1, policy_version 803186 (0.0008) [2023-12-26 21:12:36,475][105620] Updated weights for policy 1, policy_version 803196 (0.0008) [2023-12-26 21:12:36,546][105620] Updated weights for policy 1, policy_version 803206 (0.0008) [2023-12-26 21:12:36,972][105692] Updated weights for policy 0, policy_version 803337 (0.0010) [2023-12-26 21:12:37,025][105692] Updated weights for policy 0, policy_version 803347 (0.0009) [2023-12-26 21:12:37,082][105692] Updated weights for policy 0, policy_version 803357 (0.0010) [2023-12-26 21:12:37,142][105692] Updated weights for policy 0, policy_version 803368 (0.0010) [2023-12-26 21:12:37,239][105620] Updated weights for policy 1, policy_version 803216 (0.0008) [2023-12-26 21:12:37,286][105620] Updated weights for policy 1, policy_version 803226 (0.0008) [2023-12-26 21:12:37,345][105620] Updated weights for policy 1, policy_version 803236 (0.0009) [2023-12-26 21:12:37,845][105692] Updated weights for policy 0, policy_version 803378 (0.0005) [2023-12-26 21:12:37,902][105692] Updated weights for policy 0, policy_version 803388 (0.0005) [2023-12-26 21:12:37,960][105692] Updated weights for policy 0, policy_version 803398 (0.0005) [2023-12-26 21:12:38,110][105620] Updated weights for policy 1, policy_version 803246 (0.0008) [2023-12-26 21:12:38,162][105620] Updated weights for policy 1, policy_version 803256 (0.0009) [2023-12-26 21:12:38,221][105620] Updated weights for policy 1, policy_version 803266 (0.0009) [2023-12-26 21:12:38,681][105692] Updated weights for policy 0, policy_version 803408 (0.0009) [2023-12-26 21:12:38,746][105692] Updated weights for policy 0, policy_version 803418 (0.0009) [2023-12-26 21:12:38,808][105692] Updated weights for policy 0, policy_version 803428 (0.0009) [2023-12-26 21:12:38,881][105620] Updated weights for policy 1, policy_version 803276 (0.0008) [2023-12-26 21:12:38,942][105620] Updated weights for policy 1, policy_version 803286 (0.0010) [2023-12-26 21:12:39,004][105620] Updated weights for policy 1, policy_version 803296 (0.0010) [2023-12-26 21:12:39,626][105692] Updated weights for policy 0, policy_version 803438 (0.0009) [2023-12-26 21:12:39,690][105692] Updated weights for policy 0, policy_version 803448 (0.0008) [2023-12-26 21:12:39,755][105692] Updated weights for policy 0, policy_version 803458 (0.0008) [2023-12-26 21:12:39,770][105620] Updated weights for policy 1, policy_version 803306 (0.0010) [2023-12-26 21:12:39,826][105620] Updated weights for policy 1, policy_version 803316 (0.0010) [2023-12-26 21:12:39,893][105620] Updated weights for policy 1, policy_version 803326 (0.0009) [2023-12-26 21:12:39,958][105620] Updated weights for policy 1, policy_version 803336 (0.0011) [2023-12-26 21:12:40,458][105692] Updated weights for policy 0, policy_version 803468 (0.0007) [2023-12-26 21:12:40,511][105692] Updated weights for policy 0, policy_version 803478 (0.0008) [2023-12-26 21:12:40,572][105692] Updated weights for policy 0, policy_version 803488 (0.0008) [2023-12-26 21:12:40,688][105620] Updated weights for policy 1, policy_version 803346 (0.0005) [2023-12-26 21:12:40,747][105620] Updated weights for policy 1, policy_version 803356 (0.0006) [2023-12-26 21:12:40,803][105620] Updated weights for policy 1, policy_version 803366 (0.0009) [2023-12-26 21:12:41,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19355.3). Total num frames: 411410432. Throughput: 0: 9738.2, 1: 9560.2. Samples: 411418968. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 21:12:41,063][104569] Avg episode reward: [(0, '9167.440'), (1, '8899.324')] [2023-12-26 21:12:41,360][105692] Updated weights for policy 0, policy_version 803498 (0.0008) [2023-12-26 21:12:41,435][105692] Updated weights for policy 0, policy_version 803508 (0.0009) [2023-12-26 21:12:41,449][105620] Updated weights for policy 1, policy_version 803376 (0.0008) [2023-12-26 21:12:41,500][105692] Updated weights for policy 0, policy_version 803518 (0.0007) [2023-12-26 21:12:41,510][105620] Updated weights for policy 1, policy_version 803386 (0.0007) [2023-12-26 21:12:41,558][105692] Updated weights for policy 0, policy_version 803528 (0.0007) [2023-12-26 21:12:41,569][105620] Updated weights for policy 1, policy_version 803396 (0.0007) [2023-12-26 21:12:42,254][105692] Updated weights for policy 0, policy_version 803538 (0.0009) [2023-12-26 21:12:42,306][105692] Updated weights for policy 0, policy_version 803548 (0.0008) [2023-12-26 21:12:42,355][105620] Updated weights for policy 1, policy_version 803406 (0.0009) [2023-12-26 21:12:42,376][105692] Updated weights for policy 0, policy_version 803558 (0.0008) [2023-12-26 21:12:42,416][105620] Updated weights for policy 1, policy_version 803416 (0.0008) [2023-12-26 21:12:42,473][105620] Updated weights for policy 1, policy_version 803426 (0.0009) [2023-12-26 21:12:43,153][105620] Updated weights for policy 1, policy_version 803436 (0.0008) [2023-12-26 21:12:43,164][105692] Updated weights for policy 0, policy_version 803568 (0.0009) [2023-12-26 21:12:43,208][105620] Updated weights for policy 1, policy_version 803446 (0.0006) [2023-12-26 21:12:43,220][105692] Updated weights for policy 0, policy_version 803578 (0.0008) [2023-12-26 21:12:43,266][105620] Updated weights for policy 1, policy_version 803456 (0.0007) [2023-12-26 21:12:43,274][105692] Updated weights for policy 0, policy_version 803588 (0.0005) [2023-12-26 21:12:43,876][105692] Updated weights for policy 0, policy_version 803598 (0.0005) [2023-12-26 21:12:43,930][105692] Updated weights for policy 0, policy_version 803608 (0.0006) [2023-12-26 21:12:43,981][105692] Updated weights for policy 0, policy_version 803618 (0.0008) [2023-12-26 21:12:44,021][105620] Updated weights for policy 1, policy_version 803466 (0.0007) [2023-12-26 21:12:44,067][105620] Updated weights for policy 1, policy_version 803476 (0.0009) [2023-12-26 21:12:44,114][105620] Updated weights for policy 1, policy_version 803486 (0.0008) [2023-12-26 21:12:44,160][105620] Updated weights for policy 1, policy_version 803496 (0.0008) [2023-12-26 21:12:44,661][105692] Updated weights for policy 0, policy_version 803628 (0.0008) [2023-12-26 21:12:44,725][105692] Updated weights for policy 0, policy_version 803638 (0.0005) [2023-12-26 21:12:44,793][105692] Updated weights for policy 0, policy_version 803648 (0.0009) [2023-12-26 21:12:44,879][105620] Updated weights for policy 1, policy_version 803506 (0.0011) [2023-12-26 21:12:44,947][105620] Updated weights for policy 1, policy_version 803516 (0.0011) [2023-12-26 21:12:45,011][105620] Updated weights for policy 1, policy_version 803526 (0.0011) [2023-12-26 21:12:45,465][105692] Updated weights for policy 0, policy_version 803658 (0.0010) [2023-12-26 21:12:45,533][105692] Updated weights for policy 0, policy_version 803668 (0.0010) [2023-12-26 21:12:45,598][105692] Updated weights for policy 0, policy_version 803678 (0.0010) [2023-12-26 21:12:45,634][105620] Updated weights for policy 1, policy_version 803536 (0.0010) [2023-12-26 21:12:45,660][105692] Updated weights for policy 0, policy_version 803688 (0.0010) [2023-12-26 21:12:45,694][105620] Updated weights for policy 1, policy_version 803546 (0.0011) [2023-12-26 21:12:45,729][105586] KL-divergence is very high: 119.2069 [2023-12-26 21:12:45,756][105620] Updated weights for policy 1, policy_version 803556 (0.0010) [2023-12-26 21:12:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 411508736. Throughput: 0: 9703.5, 1: 9582.4. Samples: 411475760. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 21:12:46,063][104569] Avg episode reward: [(0, '9073.301'), (1, '9045.843')] [2023-12-26 21:12:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000803560_205733888.pth... [2023-12-26 21:12:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000803688_205774848.pth... [2023-12-26 21:12:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000802440_205447168.pth [2023-12-26 21:12:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000802568_205488128.pth [2023-12-26 21:12:46,212][105692] Updated weights for policy 0, policy_version 803698 (0.0006) [2023-12-26 21:12:46,274][105692] Updated weights for policy 0, policy_version 803708 (0.0005) [2023-12-26 21:12:46,294][105620] Updated weights for policy 1, policy_version 803566 (0.0006) [2023-12-26 21:12:46,322][105692] Updated weights for policy 0, policy_version 803718 (0.0005) [2023-12-26 21:12:46,354][105620] Updated weights for policy 1, policy_version 803576 (0.0006) [2023-12-26 21:12:46,412][105620] Updated weights for policy 1, policy_version 803586 (0.0011) [2023-12-26 21:12:46,913][105692] Updated weights for policy 0, policy_version 803728 (0.0006) [2023-12-26 21:12:46,967][105692] Updated weights for policy 0, policy_version 803738 (0.0007) [2023-12-26 21:12:46,975][105620] Updated weights for policy 1, policy_version 803596 (0.0009) [2023-12-26 21:12:47,022][105692] Updated weights for policy 0, policy_version 803748 (0.0010) [2023-12-26 21:12:47,023][105620] Updated weights for policy 1, policy_version 803606 (0.0010) [2023-12-26 21:12:47,075][105620] Updated weights for policy 1, policy_version 803616 (0.0010) [2023-12-26 21:12:47,600][105692] Updated weights for policy 0, policy_version 803758 (0.0007) [2023-12-26 21:12:47,655][105692] Updated weights for policy 0, policy_version 803768 (0.0005) [2023-12-26 21:12:47,712][105692] Updated weights for policy 0, policy_version 803778 (0.0006) [2023-12-26 21:12:47,813][105620] Updated weights for policy 1, policy_version 803626 (0.0008) [2023-12-26 21:12:47,874][105620] Updated weights for policy 1, policy_version 803636 (0.0010) [2023-12-26 21:12:47,922][105620] Updated weights for policy 1, policy_version 803646 (0.0010) [2023-12-26 21:12:47,966][105620] Updated weights for policy 1, policy_version 803656 (0.0010) [2023-12-26 21:12:48,347][105692] Updated weights for policy 0, policy_version 803788 (0.0007) [2023-12-26 21:12:48,410][105692] Updated weights for policy 0, policy_version 803798 (0.0011) [2023-12-26 21:12:48,475][105692] Updated weights for policy 0, policy_version 803808 (0.0010) [2023-12-26 21:12:48,680][105620] Updated weights for policy 1, policy_version 803666 (0.0005) [2023-12-26 21:12:48,731][105620] Updated weights for policy 1, policy_version 803676 (0.0011) [2023-12-26 21:12:48,784][105620] Updated weights for policy 1, policy_version 803686 (0.0010) [2023-12-26 21:12:49,101][105692] Updated weights for policy 0, policy_version 803818 (0.0006) [2023-12-26 21:12:49,155][105692] Updated weights for policy 0, policy_version 803828 (0.0006) [2023-12-26 21:12:49,215][105692] Updated weights for policy 0, policy_version 803838 (0.0006) [2023-12-26 21:12:49,286][105692] Updated weights for policy 0, policy_version 803848 (0.0007) [2023-12-26 21:12:49,448][105620] Updated weights for policy 1, policy_version 803696 (0.0006) [2023-12-26 21:12:49,500][105620] Updated weights for policy 1, policy_version 803706 (0.0005) [2023-12-26 21:12:49,560][105620] Updated weights for policy 1, policy_version 803716 (0.0008) [2023-12-26 21:12:50,073][105692] Updated weights for policy 0, policy_version 803858 (0.0010) [2023-12-26 21:12:50,131][105692] Updated weights for policy 0, policy_version 803868 (0.0010) [2023-12-26 21:12:50,152][105620] Updated weights for policy 1, policy_version 803726 (0.0007) [2023-12-26 21:12:50,194][105692] Updated weights for policy 0, policy_version 803878 (0.0007) [2023-12-26 21:12:50,211][105620] Updated weights for policy 1, policy_version 803736 (0.0010) [2023-12-26 21:12:50,272][105620] Updated weights for policy 1, policy_version 803746 (0.0010) [2023-12-26 21:12:50,887][105620] Updated weights for policy 1, policy_version 803756 (0.0010) [2023-12-26 21:12:50,910][105692] Updated weights for policy 0, policy_version 803888 (0.0010) [2023-12-26 21:12:50,945][105620] Updated weights for policy 1, policy_version 803766 (0.0010) [2023-12-26 21:12:50,969][105692] Updated weights for policy 0, policy_version 803898 (0.0011) [2023-12-26 21:12:51,015][105620] Updated weights for policy 1, policy_version 803776 (0.0009) [2023-12-26 21:12:51,030][105692] Updated weights for policy 0, policy_version 803908 (0.0011) [2023-12-26 21:12:51,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 411615232. Throughput: 0: 9828.3, 1: 9722.6. Samples: 411605024. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 21:12:51,062][104569] Avg episode reward: [(0, '9073.369'), (1, '8932.333')] [2023-12-26 21:12:51,708][105620] Updated weights for policy 1, policy_version 803786 (0.0008) [2023-12-26 21:12:51,730][105692] Updated weights for policy 0, policy_version 803918 (0.0007) [2023-12-26 21:12:51,778][105620] Updated weights for policy 1, policy_version 803796 (0.0008) [2023-12-26 21:12:51,800][105692] Updated weights for policy 0, policy_version 803928 (0.0008) [2023-12-26 21:12:51,839][105620] Updated weights for policy 1, policy_version 803806 (0.0008) [2023-12-26 21:12:51,864][105692] Updated weights for policy 0, policy_version 803938 (0.0008) [2023-12-26 21:12:51,900][105620] Updated weights for policy 1, policy_version 803816 (0.0008) [2023-12-26 21:12:52,540][105692] Updated weights for policy 0, policy_version 803948 (0.0009) [2023-12-26 21:12:52,590][105692] Updated weights for policy 0, policy_version 803958 (0.0009) [2023-12-26 21:12:52,644][105692] Updated weights for policy 0, policy_version 803968 (0.0008) [2023-12-26 21:12:52,707][105620] Updated weights for policy 1, policy_version 803826 (0.0008) [2023-12-26 21:12:52,767][105620] Updated weights for policy 1, policy_version 803836 (0.0009) [2023-12-26 21:12:52,817][105620] Updated weights for policy 1, policy_version 803846 (0.0009) [2023-12-26 21:12:53,394][105692] Updated weights for policy 0, policy_version 803978 (0.0007) [2023-12-26 21:12:53,460][105692] Updated weights for policy 0, policy_version 803988 (0.0010) [2023-12-26 21:12:53,514][105692] Updated weights for policy 0, policy_version 803998 (0.0007) [2023-12-26 21:12:53,515][105620] Updated weights for policy 1, policy_version 803856 (0.0008) [2023-12-26 21:12:53,571][105620] Updated weights for policy 1, policy_version 803866 (0.0007) [2023-12-26 21:12:53,573][105692] Updated weights for policy 0, policy_version 804008 (0.0006) [2023-12-26 21:12:53,635][105620] Updated weights for policy 1, policy_version 803876 (0.0009) [2023-12-26 21:12:54,238][105620] Updated weights for policy 1, policy_version 803886 (0.0009) [2023-12-26 21:12:54,298][105620] Updated weights for policy 1, policy_version 803896 (0.0005) [2023-12-26 21:12:54,300][105692] Updated weights for policy 0, policy_version 804018 (0.0008) [2023-12-26 21:12:54,342][105620] Updated weights for policy 1, policy_version 803906 (0.0006) [2023-12-26 21:12:54,357][105692] Updated weights for policy 0, policy_version 804028 (0.0009) [2023-12-26 21:12:54,409][105692] Updated weights for policy 0, policy_version 804038 (0.0008) [2023-12-26 21:12:55,057][105620] Updated weights for policy 1, policy_version 803916 (0.0007) [2023-12-26 21:12:55,112][105620] Updated weights for policy 1, policy_version 803926 (0.0009) [2023-12-26 21:12:55,168][105620] Updated weights for policy 1, policy_version 803936 (0.0008) [2023-12-26 21:12:55,171][105692] Updated weights for policy 0, policy_version 804048 (0.0006) [2023-12-26 21:12:55,235][105692] Updated weights for policy 0, policy_version 804058 (0.0009) [2023-12-26 21:12:55,296][105692] Updated weights for policy 0, policy_version 804068 (0.0009) [2023-12-26 21:12:55,943][105620] Updated weights for policy 1, policy_version 803946 (0.0008) [2023-12-26 21:12:55,993][105620] Updated weights for policy 1, policy_version 803956 (0.0008) [2023-12-26 21:12:56,000][105586] KL-divergence is very high: 126.1796 [2023-12-26 21:12:56,033][105692] Updated weights for policy 0, policy_version 804078 (0.0008) [2023-12-26 21:12:56,046][105586] KL-divergence is very high: 203.7948 [2023-12-26 21:12:56,051][105620] Updated weights for policy 1, policy_version 803966 (0.0007) [2023-12-26 21:12:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 411705344. Throughput: 0: 9766.5, 1: 9787.7. Samples: 411720772. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 21:12:56,062][104569] Avg episode reward: [(0, '8805.888'), (1, '8700.699')] [2023-12-26 21:12:56,082][105692] Updated weights for policy 0, policy_version 804088 (0.0006) [2023-12-26 21:12:56,089][105586] KL-divergence is very high: 195.0762 [2023-12-26 21:12:56,106][105620] Updated weights for policy 1, policy_version 803976 (0.0008) [2023-12-26 21:12:56,129][105692] Updated weights for policy 0, policy_version 804098 (0.0009) [2023-12-26 21:12:56,821][105692] Updated weights for policy 0, policy_version 804108 (0.0007) [2023-12-26 21:12:56,864][105620] Updated weights for policy 1, policy_version 803986 (0.0010) [2023-12-26 21:12:56,875][105692] Updated weights for policy 0, policy_version 804118 (0.0006) [2023-12-26 21:12:56,909][105620] Updated weights for policy 1, policy_version 803996 (0.0010) [2023-12-26 21:12:56,920][105692] Updated weights for policy 0, policy_version 804128 (0.0006) [2023-12-26 21:12:56,964][105620] Updated weights for policy 1, policy_version 804006 (0.0009) [2023-12-26 21:12:57,649][105692] Updated weights for policy 0, policy_version 804138 (0.0005) [2023-12-26 21:12:57,672][105620] Updated weights for policy 1, policy_version 804016 (0.0007) [2023-12-26 21:12:57,702][105692] Updated weights for policy 0, policy_version 804148 (0.0007) [2023-12-26 21:12:57,735][105620] Updated weights for policy 1, policy_version 804026 (0.0006) [2023-12-26 21:12:57,751][105692] Updated weights for policy 0, policy_version 804158 (0.0008) [2023-12-26 21:12:57,788][105620] Updated weights for policy 1, policy_version 804036 (0.0006) [2023-12-26 21:12:57,800][105692] Updated weights for policy 0, policy_version 804168 (0.0005) [2023-12-26 21:12:58,442][105692] Updated weights for policy 0, policy_version 804178 (0.0009) [2023-12-26 21:12:58,493][105620] Updated weights for policy 1, policy_version 804046 (0.0006) [2023-12-26 21:12:58,500][105692] Updated weights for policy 0, policy_version 804188 (0.0007) [2023-12-26 21:12:58,551][105620] Updated weights for policy 1, policy_version 804056 (0.0008) [2023-12-26 21:12:58,566][105692] Updated weights for policy 0, policy_version 804198 (0.0007) [2023-12-26 21:12:58,618][105620] Updated weights for policy 1, policy_version 804066 (0.0007) [2023-12-26 21:12:59,405][105620] Updated weights for policy 1, policy_version 804076 (0.0008) [2023-12-26 21:12:59,426][105692] Updated weights for policy 0, policy_version 804208 (0.0008) [2023-12-26 21:12:59,472][105620] Updated weights for policy 1, policy_version 804086 (0.0009) [2023-12-26 21:12:59,490][105692] Updated weights for policy 0, policy_version 804218 (0.0006) [2023-12-26 21:12:59,533][105620] Updated weights for policy 1, policy_version 804097 (0.0009) [2023-12-26 21:12:59,543][105692] Updated weights for policy 0, policy_version 804228 (0.0006) [2023-12-26 21:13:00,212][105620] Updated weights for policy 1, policy_version 804107 (0.0009) [2023-12-26 21:13:00,268][105692] Updated weights for policy 0, policy_version 804238 (0.0007) [2023-12-26 21:13:00,270][105620] Updated weights for policy 1, policy_version 804117 (0.0007) [2023-12-26 21:13:00,317][105692] Updated weights for policy 0, policy_version 804248 (0.0006) [2023-12-26 21:13:00,327][105620] Updated weights for policy 1, policy_version 804127 (0.0007) [2023-12-26 21:13:00,366][105692] Updated weights for policy 0, policy_version 804258 (0.0006) [2023-12-26 21:13:00,986][105620] Updated weights for policy 1, policy_version 804137 (0.0007) [2023-12-26 21:13:01,049][105620] Updated weights for policy 1, policy_version 804147 (0.0006) [2023-12-26 21:13:01,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 411803648. Throughput: 0: 9813.2, 1: 9805.9. Samples: 411780396. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 21:13:01,063][104569] Avg episode reward: [(0, '8804.838'), (1, '8909.500')] [2023-12-26 21:13:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000804264_205922304.pth... [2023-12-26 21:13:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000803144_205635584.pth [2023-12-26 21:13:01,114][105620] Updated weights for policy 1, policy_version 804157 (0.0007) [2023-12-26 21:13:01,178][105620] Updated weights for policy 1, policy_version 804167 (0.0007) [2023-12-26 21:13:01,181][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000804168_205889536.pth... [2023-12-26 21:13:01,184][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000802984_205586432.pth [2023-12-26 21:13:01,199][105692] Updated weights for policy 0, policy_version 804268 (0.0008) [2023-12-26 21:13:01,252][105692] Updated weights for policy 0, policy_version 804278 (0.0008) [2023-12-26 21:13:01,302][105692] Updated weights for policy 0, policy_version 804288 (0.0005) [2023-12-26 21:13:01,836][105620] Updated weights for policy 1, policy_version 804177 (0.0007) [2023-12-26 21:13:01,895][105620] Updated weights for policy 1, policy_version 804187 (0.0010) [2023-12-26 21:13:01,950][105620] Updated weights for policy 1, policy_version 804198 (0.0010) [2023-12-26 21:13:02,065][105692] Updated weights for policy 0, policy_version 804298 (0.0006) [2023-12-26 21:13:02,115][105692] Updated weights for policy 0, policy_version 804308 (0.0009) [2023-12-26 21:13:02,168][105692] Updated weights for policy 0, policy_version 804318 (0.0008) [2023-12-26 21:13:02,224][105692] Updated weights for policy 0, policy_version 804328 (0.0008) [2023-12-26 21:13:02,660][105620] Updated weights for policy 1, policy_version 804208 (0.0008) [2023-12-26 21:13:02,706][105620] Updated weights for policy 1, policy_version 804218 (0.0006) [2023-12-26 21:13:02,755][105620] Updated weights for policy 1, policy_version 804228 (0.0005) [2023-12-26 21:13:03,087][105692] Updated weights for policy 0, policy_version 804338 (0.0009) [2023-12-26 21:13:03,139][105692] Updated weights for policy 0, policy_version 804348 (0.0010) [2023-12-26 21:13:03,195][105692] Updated weights for policy 0, policy_version 804358 (0.0009) [2023-12-26 21:13:03,299][105620] Updated weights for policy 1, policy_version 804238 (0.0007) [2023-12-26 21:13:03,355][105620] Updated weights for policy 1, policy_version 804248 (0.0008) [2023-12-26 21:13:03,411][105620] Updated weights for policy 1, policy_version 804258 (0.0009) [2023-12-26 21:13:03,960][105692] Updated weights for policy 0, policy_version 804368 (0.0009) [2023-12-26 21:13:04,001][105620] Updated weights for policy 1, policy_version 804268 (0.0008) [2023-12-26 21:13:04,019][105692] Updated weights for policy 0, policy_version 804378 (0.0008) [2023-12-26 21:13:04,068][105620] Updated weights for policy 1, policy_version 804278 (0.0005) [2023-12-26 21:13:04,079][105692] Updated weights for policy 0, policy_version 804388 (0.0009) [2023-12-26 21:13:04,138][105620] Updated weights for policy 1, policy_version 804288 (0.0006) [2023-12-26 21:13:04,700][105692] Updated weights for policy 0, policy_version 804398 (0.0007) [2023-12-26 21:13:04,763][105692] Updated weights for policy 0, policy_version 804408 (0.0005) [2023-12-26 21:13:04,825][105692] Updated weights for policy 0, policy_version 804418 (0.0006) [2023-12-26 21:13:04,929][105620] Updated weights for policy 1, policy_version 804298 (0.0009) [2023-12-26 21:13:04,990][105620] Updated weights for policy 1, policy_version 804308 (0.0009) [2023-12-26 21:13:05,048][105620] Updated weights for policy 1, policy_version 804318 (0.0010) [2023-12-26 21:13:05,115][105620] Updated weights for policy 1, policy_version 804328 (0.0009) [2023-12-26 21:13:05,363][105692] Updated weights for policy 0, policy_version 804428 (0.0006) [2023-12-26 21:13:05,423][105692] Updated weights for policy 0, policy_version 804438 (0.0008) [2023-12-26 21:13:05,476][105692] Updated weights for policy 0, policy_version 804448 (0.0005) [2023-12-26 21:13:05,925][105620] Updated weights for policy 1, policy_version 804338 (0.0008) [2023-12-26 21:13:05,979][105620] Updated weights for policy 1, policy_version 804348 (0.0009) [2023-12-26 21:13:06,032][105620] Updated weights for policy 1, policy_version 804358 (0.0008) [2023-12-26 21:13:06,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 411910144. Throughput: 0: 9800.8, 1: 9833.4. Samples: 411896560. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 21:13:06,063][104569] Avg episode reward: [(0, '9342.669'), (1, '9264.671')] [2023-12-26 21:13:06,125][105692] Updated weights for policy 0, policy_version 804458 (0.0005) [2023-12-26 21:13:06,183][105692] Updated weights for policy 0, policy_version 804468 (0.0008) [2023-12-26 21:13:06,243][105692] Updated weights for policy 0, policy_version 804478 (0.0005) [2023-12-26 21:13:06,294][105692] Updated weights for policy 0, policy_version 804488 (0.0008) [2023-12-26 21:13:06,798][105620] Updated weights for policy 1, policy_version 804368 (0.0007) [2023-12-26 21:13:06,856][105620] Updated weights for policy 1, policy_version 804378 (0.0007) [2023-12-26 21:13:06,916][105620] Updated weights for policy 1, policy_version 804388 (0.0008) [2023-12-26 21:13:07,051][105692] Updated weights for policy 0, policy_version 804498 (0.0010) [2023-12-26 21:13:07,113][105692] Updated weights for policy 0, policy_version 804508 (0.0010) [2023-12-26 21:13:07,168][105692] Updated weights for policy 0, policy_version 804518 (0.0010) [2023-12-26 21:13:07,643][105620] Updated weights for policy 1, policy_version 804398 (0.0008) [2023-12-26 21:13:07,698][105620] Updated weights for policy 1, policy_version 804410 (0.0010) [2023-12-26 21:13:07,755][105620] Updated weights for policy 1, policy_version 804420 (0.0009) [2023-12-26 21:13:07,848][105692] Updated weights for policy 0, policy_version 804528 (0.0006) [2023-12-26 21:13:07,896][105692] Updated weights for policy 0, policy_version 804538 (0.0005) [2023-12-26 21:13:07,949][105692] Updated weights for policy 0, policy_version 804548 (0.0005) [2023-12-26 21:13:08,604][105692] Updated weights for policy 0, policy_version 804558 (0.0006) [2023-12-26 21:13:08,621][105620] Updated weights for policy 1, policy_version 804430 (0.0009) [2023-12-26 21:13:08,665][105692] Updated weights for policy 0, policy_version 804568 (0.0006) [2023-12-26 21:13:08,684][105620] Updated weights for policy 1, policy_version 804440 (0.0008) [2023-12-26 21:13:08,726][105692] Updated weights for policy 0, policy_version 804578 (0.0007) [2023-12-26 21:13:08,750][105620] Updated weights for policy 1, policy_version 804450 (0.0008) [2023-12-26 21:13:09,479][105692] Updated weights for policy 0, policy_version 804588 (0.0006) [2023-12-26 21:13:09,533][105692] Updated weights for policy 0, policy_version 804598 (0.0007) [2023-12-26 21:13:09,545][105620] Updated weights for policy 1, policy_version 804460 (0.0008) [2023-12-26 21:13:09,590][105692] Updated weights for policy 0, policy_version 804608 (0.0007) [2023-12-26 21:13:09,612][105620] Updated weights for policy 1, policy_version 804470 (0.0007) [2023-12-26 21:13:09,676][105620] Updated weights for policy 1, policy_version 804480 (0.0006) [2023-12-26 21:13:09,685][105586] KL-divergence is very high: 104.1450 [2023-12-26 21:13:10,319][105620] Updated weights for policy 1, policy_version 804490 (0.0007) [2023-12-26 21:13:10,383][105620] Updated weights for policy 1, policy_version 804500 (0.0008) [2023-12-26 21:13:10,417][105692] Updated weights for policy 0, policy_version 804618 (0.0008) [2023-12-26 21:13:10,440][105620] Updated weights for policy 1, policy_version 804510 (0.0007) [2023-12-26 21:13:10,481][105692] Updated weights for policy 0, policy_version 804628 (0.0008) [2023-12-26 21:13:10,492][105620] Updated weights for policy 1, policy_version 804520 (0.0006) [2023-12-26 21:13:10,552][105692] Updated weights for policy 0, policy_version 804638 (0.0008) [2023-12-26 21:13:10,612][105692] Updated weights for policy 0, policy_version 804648 (0.0008) [2023-12-26 21:13:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 412000256. Throughput: 0: 9895.5, 1: 9798.8. Samples: 412011268. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 21:13:11,063][104569] Avg episode reward: [(0, '9343.309'), (1, '9117.805')] [2023-12-26 21:13:11,232][105620] Updated weights for policy 1, policy_version 804530 (0.0009) [2023-12-26 21:13:11,292][105620] Updated weights for policy 1, policy_version 804540 (0.0008) [2023-12-26 21:13:11,305][105692] Updated weights for policy 0, policy_version 804658 (0.0007) [2023-12-26 21:13:11,356][105620] Updated weights for policy 1, policy_version 804550 (0.0008) [2023-12-26 21:13:11,369][105692] Updated weights for policy 0, policy_version 804668 (0.0011) [2023-12-26 21:13:11,436][105692] Updated weights for policy 0, policy_version 804678 (0.0011) [2023-12-26 21:13:12,121][105692] Updated weights for policy 0, policy_version 804688 (0.0010) [2023-12-26 21:13:12,180][105692] Updated weights for policy 0, policy_version 804698 (0.0010) [2023-12-26 21:13:12,219][105620] Updated weights for policy 1, policy_version 804560 (0.0006) [2023-12-26 21:13:12,233][105692] Updated weights for policy 0, policy_version 804708 (0.0010) [2023-12-26 21:13:12,279][105620] Updated weights for policy 1, policy_version 804570 (0.0007) [2023-12-26 21:13:12,349][105620] Updated weights for policy 1, policy_version 804580 (0.0009) [2023-12-26 21:13:12,863][105692] Updated weights for policy 0, policy_version 804718 (0.0009) [2023-12-26 21:13:12,927][105692] Updated weights for policy 0, policy_version 804728 (0.0009) [2023-12-26 21:13:12,991][105692] Updated weights for policy 0, policy_version 804738 (0.0009) [2023-12-26 21:13:13,179][105620] Updated weights for policy 1, policy_version 804590 (0.0010) [2023-12-26 21:13:13,241][105620] Updated weights for policy 1, policy_version 804600 (0.0009) [2023-12-26 21:13:13,308][105620] Updated weights for policy 1, policy_version 804610 (0.0008) [2023-12-26 21:13:13,697][105692] Updated weights for policy 0, policy_version 804748 (0.0009) [2023-12-26 21:13:13,756][105692] Updated weights for policy 0, policy_version 804758 (0.0009) [2023-12-26 21:13:13,812][105692] Updated weights for policy 0, policy_version 804768 (0.0009) [2023-12-26 21:13:14,048][105620] Updated weights for policy 1, policy_version 804620 (0.0008) [2023-12-26 21:13:14,107][105620] Updated weights for policy 1, policy_version 804630 (0.0009) [2023-12-26 21:13:14,159][105620] Updated weights for policy 1, policy_version 804640 (0.0010) [2023-12-26 21:13:14,456][105692] Updated weights for policy 0, policy_version 804778 (0.0008) [2023-12-26 21:13:14,516][105692] Updated weights for policy 0, policy_version 804788 (0.0006) [2023-12-26 21:13:14,570][105692] Updated weights for policy 0, policy_version 804798 (0.0005) [2023-12-26 21:13:14,626][105692] Updated weights for policy 0, policy_version 804808 (0.0005) [2023-12-26 21:13:14,975][105620] Updated weights for policy 1, policy_version 804650 (0.0009) [2023-12-26 21:13:15,046][105620] Updated weights for policy 1, policy_version 804660 (0.0007) [2023-12-26 21:13:15,116][105620] Updated weights for policy 1, policy_version 804670 (0.0006) [2023-12-26 21:13:15,176][105620] Updated weights for policy 1, policy_version 804680 (0.0006) [2023-12-26 21:13:15,188][105692] Updated weights for policy 0, policy_version 804818 (0.0006) [2023-12-26 21:13:15,256][105692] Updated weights for policy 0, policy_version 804828 (0.0006) [2023-12-26 21:13:15,315][105692] Updated weights for policy 0, policy_version 804838 (0.0006) [2023-12-26 21:13:15,757][105620] Updated weights for policy 1, policy_version 804690 (0.0010) [2023-12-26 21:13:15,805][105620] Updated weights for policy 1, policy_version 804700 (0.0010) [2023-12-26 21:13:15,853][105620] Updated weights for policy 1, policy_version 804710 (0.0010) [2023-12-26 21:13:15,901][105692] Updated weights for policy 0, policy_version 804848 (0.0007) [2023-12-26 21:13:15,951][105692] Updated weights for policy 0, policy_version 804858 (0.0007) [2023-12-26 21:13:16,015][105692] Updated weights for policy 0, policy_version 804868 (0.0006) [2023-12-26 21:13:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 412106752. Throughput: 0: 9810.9, 1: 9745.4. Samples: 412068148. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 21:13:16,063][104569] Avg episode reward: [(0, '9069.835'), (1, '8939.718')] [2023-12-26 21:13:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000804872_206077952.pth... [2023-12-26 21:13:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000804712_206028800.pth... [2023-12-26 21:13:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000803688_205774848.pth [2023-12-26 21:13:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000803560_205733888.pth [2023-12-26 21:13:16,598][105620] Updated weights for policy 1, policy_version 804720 (0.0009) [2023-12-26 21:13:16,648][105620] Updated weights for policy 1, policy_version 804730 (0.0008) [2023-12-26 21:13:16,696][105620] Updated weights for policy 1, policy_version 804740 (0.0009) [2023-12-26 21:13:16,723][105692] Updated weights for policy 0, policy_version 804878 (0.0008) [2023-12-26 21:13:16,780][105692] Updated weights for policy 0, policy_version 804888 (0.0007) [2023-12-26 21:13:16,834][105692] Updated weights for policy 0, policy_version 804898 (0.0005) [2023-12-26 21:13:17,319][105620] Updated weights for policy 1, policy_version 804750 (0.0006) [2023-12-26 21:13:17,377][105620] Updated weights for policy 1, policy_version 804760 (0.0006) [2023-12-26 21:13:17,423][105620] Updated weights for policy 1, policy_version 804770 (0.0009) [2023-12-26 21:13:17,489][105692] Updated weights for policy 0, policy_version 804908 (0.0007) [2023-12-26 21:13:17,537][105692] Updated weights for policy 0, policy_version 804918 (0.0010) [2023-12-26 21:13:17,591][105692] Updated weights for policy 0, policy_version 804928 (0.0009) [2023-12-26 21:13:18,069][105620] Updated weights for policy 1, policy_version 804780 (0.0007) [2023-12-26 21:13:18,123][105620] Updated weights for policy 1, policy_version 804790 (0.0009) [2023-12-26 21:13:18,175][105620] Updated weights for policy 1, policy_version 804800 (0.0010) [2023-12-26 21:13:18,209][105692] Updated weights for policy 0, policy_version 804938 (0.0009) [2023-12-26 21:13:18,264][105692] Updated weights for policy 0, policy_version 804948 (0.0009) [2023-12-26 21:13:18,320][105692] Updated weights for policy 0, policy_version 804958 (0.0008) [2023-12-26 21:13:18,385][105692] Updated weights for policy 0, policy_version 804968 (0.0007) [2023-12-26 21:13:18,940][105620] Updated weights for policy 1, policy_version 804810 (0.0011) [2023-12-26 21:13:19,005][105620] Updated weights for policy 1, policy_version 804820 (0.0010) [2023-12-26 21:13:19,070][105620] Updated weights for policy 1, policy_version 804830 (0.0010) [2023-12-26 21:13:19,122][105620] Updated weights for policy 1, policy_version 804840 (0.0010) [2023-12-26 21:13:19,155][105692] Updated weights for policy 0, policy_version 804978 (0.0009) [2023-12-26 21:13:19,207][105692] Updated weights for policy 0, policy_version 804988 (0.0010) [2023-12-26 21:13:19,269][105692] Updated weights for policy 0, policy_version 804998 (0.0007) [2023-12-26 21:13:19,837][105620] Updated weights for policy 1, policy_version 804851 (0.0009) [2023-12-26 21:13:19,904][105620] Updated weights for policy 1, policy_version 804861 (0.0011) [2023-12-26 21:13:19,972][105620] Updated weights for policy 1, policy_version 804871 (0.0010) [2023-12-26 21:13:20,059][105692] Updated weights for policy 0, policy_version 805008 (0.0010) [2023-12-26 21:13:20,125][105692] Updated weights for policy 0, policy_version 805018 (0.0011) [2023-12-26 21:13:20,195][105692] Updated weights for policy 0, policy_version 805028 (0.0011) [2023-12-26 21:13:20,659][105620] Updated weights for policy 1, policy_version 804881 (0.0011) [2023-12-26 21:13:20,706][105620] Updated weights for policy 1, policy_version 804891 (0.0011) [2023-12-26 21:13:20,767][105620] Updated weights for policy 1, policy_version 804901 (0.0011) [2023-12-26 21:13:20,887][105692] Updated weights for policy 0, policy_version 805038 (0.0009) [2023-12-26 21:13:20,950][105692] Updated weights for policy 0, policy_version 805048 (0.0011) [2023-12-26 21:13:21,003][105692] Updated weights for policy 0, policy_version 805058 (0.0011) [2023-12-26 21:13:21,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 412205056. Throughput: 0: 9886.6, 1: 9814.1. Samples: 412190004. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 21:13:21,062][104569] Avg episode reward: [(0, '9069.532'), (1, '8557.405')] [2023-12-26 21:13:21,457][105620] Updated weights for policy 1, policy_version 804911 (0.0010) [2023-12-26 21:13:21,526][105620] Updated weights for policy 1, policy_version 804921 (0.0008) [2023-12-26 21:13:21,589][105620] Updated weights for policy 1, policy_version 804931 (0.0007) [2023-12-26 21:13:21,777][105692] Updated weights for policy 0, policy_version 805068 (0.0008) [2023-12-26 21:13:21,840][105692] Updated weights for policy 0, policy_version 805078 (0.0009) [2023-12-26 21:13:21,903][105692] Updated weights for policy 0, policy_version 805088 (0.0009) [2023-12-26 21:13:22,318][105620] Updated weights for policy 1, policy_version 804941 (0.0008) [2023-12-26 21:13:22,387][105620] Updated weights for policy 1, policy_version 804951 (0.0007) [2023-12-26 21:13:22,446][105620] Updated weights for policy 1, policy_version 804961 (0.0005) [2023-12-26 21:13:22,742][105692] Updated weights for policy 0, policy_version 805098 (0.0009) [2023-12-26 21:13:22,808][105692] Updated weights for policy 0, policy_version 805108 (0.0009) [2023-12-26 21:13:22,872][105692] Updated weights for policy 0, policy_version 805118 (0.0009) [2023-12-26 21:13:22,934][105692] Updated weights for policy 0, policy_version 805128 (0.0010) [2023-12-26 21:13:23,069][105620] Updated weights for policy 1, policy_version 804971 (0.0006) [2023-12-26 21:13:23,136][105620] Updated weights for policy 1, policy_version 804981 (0.0009) [2023-12-26 21:13:23,201][105620] Updated weights for policy 1, policy_version 804991 (0.0009) [2023-12-26 21:13:23,652][105692] Updated weights for policy 0, policy_version 805138 (0.0010) [2023-12-26 21:13:23,705][105692] Updated weights for policy 0, policy_version 805148 (0.0010) [2023-12-26 21:13:23,754][105692] Updated weights for policy 0, policy_version 805158 (0.0010) [2023-12-26 21:13:23,853][105620] Updated weights for policy 1, policy_version 805001 (0.0009) [2023-12-26 21:13:23,903][105620] Updated weights for policy 1, policy_version 805011 (0.0007) [2023-12-26 21:13:23,970][105620] Updated weights for policy 1, policy_version 805021 (0.0005) [2023-12-26 21:13:24,025][105620] Updated weights for policy 1, policy_version 805031 (0.0007) [2023-12-26 21:13:24,546][105692] Updated weights for policy 0, policy_version 805168 (0.0009) [2023-12-26 21:13:24,608][105692] Updated weights for policy 0, policy_version 805178 (0.0010) [2023-12-26 21:13:24,670][105692] Updated weights for policy 0, policy_version 805188 (0.0009) [2023-12-26 21:13:24,688][105620] Updated weights for policy 1, policy_version 805041 (0.0005) [2023-12-26 21:13:24,740][105620] Updated weights for policy 1, policy_version 805051 (0.0006) [2023-12-26 21:13:24,804][105620] Updated weights for policy 1, policy_version 805061 (0.0009) [2023-12-26 21:13:25,465][105692] Updated weights for policy 0, policy_version 805198 (0.0009) [2023-12-26 21:13:25,520][105692] Updated weights for policy 0, policy_version 805208 (0.0006) [2023-12-26 21:13:25,522][105620] Updated weights for policy 1, policy_version 805071 (0.0010) [2023-12-26 21:13:25,571][105692] Updated weights for policy 0, policy_version 805218 (0.0007) [2023-12-26 21:13:25,581][105620] Updated weights for policy 1, policy_version 805081 (0.0010) [2023-12-26 21:13:25,640][105620] Updated weights for policy 1, policy_version 805091 (0.0010) [2023-12-26 21:13:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 412295168. Throughput: 0: 9831.7, 1: 9854.8. Samples: 412304864. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 21:13:26,063][104569] Avg episode reward: [(0, '9252.543'), (1, '8789.119')] [2023-12-26 21:13:26,356][105692] Updated weights for policy 0, policy_version 805228 (0.0007) [2023-12-26 21:13:26,393][105620] Updated weights for policy 1, policy_version 805101 (0.0010) [2023-12-26 21:13:26,411][105692] Updated weights for policy 0, policy_version 805238 (0.0007) [2023-12-26 21:13:26,444][105620] Updated weights for policy 1, policy_version 805111 (0.0010) [2023-12-26 21:13:26,469][105692] Updated weights for policy 0, policy_version 805248 (0.0006) [2023-12-26 21:13:26,502][105620] Updated weights for policy 1, policy_version 805121 (0.0010) [2023-12-26 21:13:27,148][105620] Updated weights for policy 1, policy_version 805131 (0.0009) [2023-12-26 21:13:27,154][105692] Updated weights for policy 0, policy_version 805258 (0.0007) [2023-12-26 21:13:27,207][105620] Updated weights for policy 1, policy_version 805141 (0.0006) [2023-12-26 21:13:27,210][105692] Updated weights for policy 0, policy_version 805268 (0.0010) [2023-12-26 21:13:27,261][105620] Updated weights for policy 1, policy_version 805151 (0.0005) [2023-12-26 21:13:27,264][105692] Updated weights for policy 0, policy_version 805278 (0.0010) [2023-12-26 21:13:27,327][105692] Updated weights for policy 0, policy_version 805288 (0.0007) [2023-12-26 21:13:27,891][105620] Updated weights for policy 1, policy_version 805161 (0.0006) [2023-12-26 21:13:27,909][105692] Updated weights for policy 0, policy_version 805298 (0.0009) [2023-12-26 21:13:27,949][105620] Updated weights for policy 1, policy_version 805171 (0.0005) [2023-12-26 21:13:27,967][105692] Updated weights for policy 0, policy_version 805308 (0.0010) [2023-12-26 21:13:28,007][105620] Updated weights for policy 1, policy_version 805181 (0.0007) [2023-12-26 21:13:28,016][105692] Updated weights for policy 0, policy_version 805318 (0.0008) [2023-12-26 21:13:28,072][105620] Updated weights for policy 1, policy_version 805191 (0.0010) [2023-12-26 21:13:28,634][105620] Updated weights for policy 1, policy_version 805201 (0.0011) [2023-12-26 21:13:28,694][105620] Updated weights for policy 1, policy_version 805211 (0.0011) [2023-12-26 21:13:28,748][105620] Updated weights for policy 1, policy_version 805221 (0.0011) [2023-12-26 21:13:28,758][105692] Updated weights for policy 0, policy_version 805328 (0.0007) [2023-12-26 21:13:28,817][105692] Updated weights for policy 0, policy_version 805338 (0.0008) [2023-12-26 21:13:28,881][105692] Updated weights for policy 0, policy_version 805348 (0.0009) [2023-12-26 21:13:29,491][105620] Updated weights for policy 1, policy_version 805231 (0.0008) [2023-12-26 21:13:29,555][105620] Updated weights for policy 1, policy_version 805241 (0.0008) [2023-12-26 21:13:29,585][105692] Updated weights for policy 0, policy_version 805358 (0.0007) [2023-12-26 21:13:29,611][105620] Updated weights for policy 1, policy_version 805251 (0.0006) [2023-12-26 21:13:29,637][105692] Updated weights for policy 0, policy_version 805368 (0.0005) [2023-12-26 21:13:29,700][105692] Updated weights for policy 0, policy_version 805378 (0.0008) [2023-12-26 21:13:30,349][105620] Updated weights for policy 1, policy_version 805261 (0.0008) [2023-12-26 21:13:30,355][105692] Updated weights for policy 0, policy_version 805388 (0.0008) [2023-12-26 21:13:30,396][105620] Updated weights for policy 1, policy_version 805271 (0.0008) [2023-12-26 21:13:30,413][105692] Updated weights for policy 0, policy_version 805398 (0.0005) [2023-12-26 21:13:30,447][105620] Updated weights for policy 1, policy_version 805281 (0.0008) [2023-12-26 21:13:30,477][105692] Updated weights for policy 0, policy_version 805408 (0.0005) [2023-12-26 21:13:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 412393472. Throughput: 0: 9877.8, 1: 9927.0. Samples: 412366976. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 21:13:31,063][104569] Avg episode reward: [(0, '9252.530'), (1, '9022.185')] [2023-12-26 21:13:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000805416_206217216.pth... [2023-12-26 21:13:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000805288_206176256.pth... [2023-12-26 21:13:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000804168_205889536.pth [2023-12-26 21:13:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000804264_205922304.pth [2023-12-26 21:13:31,102][105692] Updated weights for policy 0, policy_version 805418 (0.0007) [2023-12-26 21:13:31,171][105692] Updated weights for policy 0, policy_version 805428 (0.0008) [2023-12-26 21:13:31,230][105692] Updated weights for policy 0, policy_version 805438 (0.0007) [2023-12-26 21:13:31,260][105620] Updated weights for policy 1, policy_version 805291 (0.0007) [2023-12-26 21:13:31,290][105692] Updated weights for policy 0, policy_version 805448 (0.0009) [2023-12-26 21:13:31,315][105620] Updated weights for policy 1, policy_version 805301 (0.0007) [2023-12-26 21:13:31,370][105620] Updated weights for policy 1, policy_version 805311 (0.0008) [2023-12-26 21:13:31,997][105692] Updated weights for policy 0, policy_version 805458 (0.0005) [2023-12-26 21:13:32,049][105692] Updated weights for policy 0, policy_version 805468 (0.0007) [2023-12-26 21:13:32,100][105692] Updated weights for policy 0, policy_version 805478 (0.0009) [2023-12-26 21:13:32,158][105620] Updated weights for policy 1, policy_version 805321 (0.0008) [2023-12-26 21:13:32,208][105620] Updated weights for policy 1, policy_version 805331 (0.0005) [2023-12-26 21:13:32,270][105620] Updated weights for policy 1, policy_version 805341 (0.0006) [2023-12-26 21:13:32,336][105620] Updated weights for policy 1, policy_version 805351 (0.0007) [2023-12-26 21:13:32,904][105692] Updated weights for policy 0, policy_version 805488 (0.0008) [2023-12-26 21:13:32,952][105620] Updated weights for policy 1, policy_version 805361 (0.0010) [2023-12-26 21:13:32,954][105692] Updated weights for policy 0, policy_version 805498 (0.0008) [2023-12-26 21:13:33,000][105620] Updated weights for policy 1, policy_version 805371 (0.0010) [2023-12-26 21:13:33,008][105692] Updated weights for policy 0, policy_version 805508 (0.0005) [2023-12-26 21:13:33,048][105620] Updated weights for policy 1, policy_version 805381 (0.0010) [2023-12-26 21:13:33,691][105692] Updated weights for policy 0, policy_version 805518 (0.0008) [2023-12-26 21:13:33,736][105620] Updated weights for policy 1, policy_version 805391 (0.0007) [2023-12-26 21:13:33,746][105692] Updated weights for policy 0, policy_version 805528 (0.0008) [2023-12-26 21:13:33,784][105620] Updated weights for policy 1, policy_version 805401 (0.0005) [2023-12-26 21:13:33,794][105692] Updated weights for policy 0, policy_version 805538 (0.0009) [2023-12-26 21:13:33,838][105620] Updated weights for policy 1, policy_version 805411 (0.0005) [2023-12-26 21:13:34,493][105620] Updated weights for policy 1, policy_version 805421 (0.0008) [2023-12-26 21:13:34,556][105620] Updated weights for policy 1, policy_version 805431 (0.0011) [2023-12-26 21:13:34,616][105620] Updated weights for policy 1, policy_version 805441 (0.0011) [2023-12-26 21:13:34,623][105692] Updated weights for policy 0, policy_version 805548 (0.0009) [2023-12-26 21:13:34,677][105692] Updated weights for policy 0, policy_version 805558 (0.0009) [2023-12-26 21:13:34,737][105692] Updated weights for policy 0, policy_version 805568 (0.0008) [2023-12-26 21:13:35,361][105620] Updated weights for policy 1, policy_version 805451 (0.0009) [2023-12-26 21:13:35,418][105620] Updated weights for policy 1, policy_version 805461 (0.0010) [2023-12-26 21:13:35,474][105620] Updated weights for policy 1, policy_version 805471 (0.0011) [2023-12-26 21:13:35,489][105692] Updated weights for policy 0, policy_version 805578 (0.0008) [2023-12-26 21:13:35,545][105692] Updated weights for policy 0, policy_version 805588 (0.0007) [2023-12-26 21:13:35,598][105692] Updated weights for policy 0, policy_version 805598 (0.0008) [2023-12-26 21:13:35,652][105692] Updated weights for policy 0, policy_version 805608 (0.0008) [2023-12-26 21:13:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 412491776. Throughput: 0: 9726.1, 1: 9797.0. Samples: 412483564. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 21:13:36,063][104569] Avg episode reward: [(0, '9252.607'), (1, '9204.494')] [2023-12-26 21:13:36,218][105620] Updated weights for policy 1, policy_version 805481 (0.0010) [2023-12-26 21:13:36,292][105620] Updated weights for policy 1, policy_version 805491 (0.0011) [2023-12-26 21:13:36,355][105620] Updated weights for policy 1, policy_version 805501 (0.0011) [2023-12-26 21:13:36,411][105620] Updated weights for policy 1, policy_version 805511 (0.0011) [2023-12-26 21:13:36,434][105692] Updated weights for policy 0, policy_version 805618 (0.0006) [2023-12-26 21:13:36,519][105692] Updated weights for policy 0, policy_version 805628 (0.0008) [2023-12-26 21:13:36,582][105692] Updated weights for policy 0, policy_version 805638 (0.0008) [2023-12-26 21:13:37,158][105620] Updated weights for policy 1, policy_version 805521 (0.0010) [2023-12-26 21:13:37,214][105620] Updated weights for policy 1, policy_version 805531 (0.0011) [2023-12-26 21:13:37,262][105620] Updated weights for policy 1, policy_version 805541 (0.0010) [2023-12-26 21:13:37,326][105692] Updated weights for policy 0, policy_version 805648 (0.0008) [2023-12-26 21:13:37,378][105692] Updated weights for policy 0, policy_version 805658 (0.0008) [2023-12-26 21:13:37,439][105692] Updated weights for policy 0, policy_version 805668 (0.0008) [2023-12-26 21:13:38,023][105620] Updated weights for policy 1, policy_version 805551 (0.0010) [2023-12-26 21:13:38,045][105586] KL-divergence is very high: 136.0347 [2023-12-26 21:13:38,081][105620] Updated weights for policy 1, policy_version 805561 (0.0010) [2023-12-26 21:13:38,085][105586] KL-divergence is very high: 232.6635 [2023-12-26 21:13:38,127][105586] KL-divergence is very high: 237.0231 [2023-12-26 21:13:38,134][105620] Updated weights for policy 1, policy_version 805571 (0.0010) [2023-12-26 21:13:38,204][105692] Updated weights for policy 0, policy_version 805678 (0.0008) [2023-12-26 21:13:38,256][105692] Updated weights for policy 0, policy_version 805688 (0.0008) [2023-12-26 21:13:38,305][105692] Updated weights for policy 0, policy_version 805698 (0.0008) [2023-12-26 21:13:38,892][105620] Updated weights for policy 1, policy_version 805581 (0.0010) [2023-12-26 21:13:38,956][105620] Updated weights for policy 1, policy_version 805591 (0.0011) [2023-12-26 21:13:39,017][105620] Updated weights for policy 1, policy_version 805601 (0.0011) [2023-12-26 21:13:39,087][105692] Updated weights for policy 0, policy_version 805708 (0.0009) [2023-12-26 21:13:39,136][105692] Updated weights for policy 0, policy_version 805718 (0.0008) [2023-12-26 21:13:39,195][105692] Updated weights for policy 0, policy_version 805728 (0.0008) [2023-12-26 21:13:39,789][105620] Updated weights for policy 1, policy_version 805611 (0.0010) [2023-12-26 21:13:39,857][105620] Updated weights for policy 1, policy_version 805621 (0.0010) [2023-12-26 21:13:39,922][105620] Updated weights for policy 1, policy_version 805631 (0.0011) [2023-12-26 21:13:40,056][105692] Updated weights for policy 0, policy_version 805738 (0.0009) [2023-12-26 21:13:40,124][105692] Updated weights for policy 0, policy_version 805748 (0.0009) [2023-12-26 21:13:40,192][105692] Updated weights for policy 0, policy_version 805758 (0.0009) [2023-12-26 21:13:40,260][105692] Updated weights for policy 0, policy_version 805768 (0.0008) [2023-12-26 21:13:40,715][105620] Updated weights for policy 1, policy_version 805641 (0.0011) [2023-12-26 21:13:40,775][105620] Updated weights for policy 1, policy_version 805651 (0.0010) [2023-12-26 21:13:40,830][105620] Updated weights for policy 1, policy_version 805661 (0.0011) [2023-12-26 21:13:40,889][105620] Updated weights for policy 1, policy_version 805671 (0.0011) [2023-12-26 21:13:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 412581888. Throughput: 0: 9670.5, 1: 9695.7. Samples: 412592252. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 21:13:41,062][105692] Updated weights for policy 0, policy_version 805778 (0.0008) [2023-12-26 21:13:41,063][104569] Avg episode reward: [(0, '9070.947'), (1, '9171.178')] [2023-12-26 21:13:41,130][105692] Updated weights for policy 0, policy_version 805788 (0.0009) [2023-12-26 21:13:41,190][105692] Updated weights for policy 0, policy_version 805798 (0.0008) [2023-12-26 21:13:41,686][105620] Updated weights for policy 1, policy_version 805681 (0.0011) [2023-12-26 21:13:41,753][105620] Updated weights for policy 1, policy_version 805691 (0.0011) [2023-12-26 21:13:41,817][105620] Updated weights for policy 1, policy_version 805701 (0.0011) [2023-12-26 21:13:41,978][105692] Updated weights for policy 0, policy_version 805808 (0.0008) [2023-12-26 21:13:42,031][105692] Updated weights for policy 0, policy_version 805818 (0.0008) [2023-12-26 21:13:42,092][105692] Updated weights for policy 0, policy_version 805828 (0.0008) [2023-12-26 21:13:42,574][105620] Updated weights for policy 1, policy_version 805711 (0.0010) [2023-12-26 21:13:42,626][105620] Updated weights for policy 1, policy_version 805721 (0.0011) [2023-12-26 21:13:42,678][105620] Updated weights for policy 1, policy_version 805731 (0.0011) [2023-12-26 21:13:42,890][105692] Updated weights for policy 0, policy_version 805838 (0.0008) [2023-12-26 21:13:42,943][105692] Updated weights for policy 0, policy_version 805848 (0.0008) [2023-12-26 21:13:42,996][105692] Updated weights for policy 0, policy_version 805858 (0.0008) [2023-12-26 21:13:43,452][105620] Updated weights for policy 1, policy_version 805741 (0.0010) [2023-12-26 21:13:43,503][105620] Updated weights for policy 1, policy_version 805751 (0.0010) [2023-12-26 21:13:43,547][105620] Updated weights for policy 1, policy_version 805761 (0.0009) [2023-12-26 21:13:43,764][105692] Updated weights for policy 0, policy_version 805868 (0.0008) [2023-12-26 21:13:43,812][105692] Updated weights for policy 0, policy_version 805878 (0.0008) [2023-12-26 21:13:43,867][105692] Updated weights for policy 0, policy_version 805888 (0.0008) [2023-12-26 21:13:44,298][105620] Updated weights for policy 1, policy_version 805771 (0.0009) [2023-12-26 21:13:44,347][105620] Updated weights for policy 1, policy_version 805781 (0.0010) [2023-12-26 21:13:44,406][105620] Updated weights for policy 1, policy_version 805791 (0.0010) [2023-12-26 21:13:44,688][105692] Updated weights for policy 0, policy_version 805898 (0.0008) [2023-12-26 21:13:44,741][105692] Updated weights for policy 0, policy_version 805908 (0.0008) [2023-12-26 21:13:44,799][105692] Updated weights for policy 0, policy_version 805918 (0.0009) [2023-12-26 21:13:44,857][105692] Updated weights for policy 0, policy_version 805928 (0.0007) [2023-12-26 21:13:45,081][105620] Updated weights for policy 1, policy_version 805801 (0.0010) [2023-12-26 21:13:45,139][105620] Updated weights for policy 1, policy_version 805811 (0.0005) [2023-12-26 21:13:45,196][105620] Updated weights for policy 1, policy_version 805821 (0.0005) [2023-12-26 21:13:45,251][105620] Updated weights for policy 1, policy_version 805831 (0.0006) [2023-12-26 21:13:45,688][105692] Updated weights for policy 0, policy_version 805938 (0.0010) [2023-12-26 21:13:45,745][105692] Updated weights for policy 0, policy_version 805949 (0.0010) [2023-12-26 21:13:45,778][105620] Updated weights for policy 1, policy_version 805841 (0.0006) [2023-12-26 21:13:45,796][105692] Updated weights for policy 0, policy_version 805959 (0.0008) [2023-12-26 21:13:45,834][105620] Updated weights for policy 1, policy_version 805851 (0.0007) [2023-12-26 21:13:45,881][105620] Updated weights for policy 1, policy_version 805861 (0.0005) [2023-12-26 21:13:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 412680192. Throughput: 0: 9585.4, 1: 9663.0. Samples: 412646576. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 21:13:46,063][104569] Avg episode reward: [(0, '9162.306'), (1, '8845.572')] [2023-12-26 21:13:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000805960_206356480.pth... [2023-12-26 21:13:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000805864_206323712.pth... [2023-12-26 21:13:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000804872_206077952.pth [2023-12-26 21:13:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000804712_206028800.pth [2023-12-26 21:13:46,428][105620] Updated weights for policy 1, policy_version 805871 (0.0005) [2023-12-26 21:13:46,472][105620] Updated weights for policy 1, policy_version 805881 (0.0005) [2023-12-26 21:13:46,522][105620] Updated weights for policy 1, policy_version 805891 (0.0005) [2023-12-26 21:13:46,683][105692] Updated weights for policy 0, policy_version 805969 (0.0006) [2023-12-26 21:13:46,740][105692] Updated weights for policy 0, policy_version 805979 (0.0009) [2023-12-26 21:13:46,790][105692] Updated weights for policy 0, policy_version 805989 (0.0006) [2023-12-26 21:13:47,081][105620] Updated weights for policy 1, policy_version 805901 (0.0005) [2023-12-26 21:13:47,151][105620] Updated weights for policy 1, policy_version 805911 (0.0006) [2023-12-26 21:13:47,199][105620] Updated weights for policy 1, policy_version 805921 (0.0006) [2023-12-26 21:13:47,611][105692] Updated weights for policy 0, policy_version 805999 (0.0007) [2023-12-26 21:13:47,667][105692] Updated weights for policy 0, policy_version 806009 (0.0008) [2023-12-26 21:13:47,715][105692] Updated weights for policy 0, policy_version 806019 (0.0008) [2023-12-26 21:13:47,753][105620] Updated weights for policy 1, policy_version 805931 (0.0006) [2023-12-26 21:13:47,811][105620] Updated weights for policy 1, policy_version 805941 (0.0010) [2023-12-26 21:13:47,876][105620] Updated weights for policy 1, policy_version 805951 (0.0010) [2023-12-26 21:13:48,379][105692] Updated weights for policy 0, policy_version 806029 (0.0009) [2023-12-26 21:13:48,446][105692] Updated weights for policy 0, policy_version 806039 (0.0006) [2023-12-26 21:13:48,501][105692] Updated weights for policy 0, policy_version 806049 (0.0008) [2023-12-26 21:13:48,561][105620] Updated weights for policy 1, policy_version 805961 (0.0009) [2023-12-26 21:13:48,634][105620] Updated weights for policy 1, policy_version 805971 (0.0011) [2023-12-26 21:13:48,687][105620] Updated weights for policy 1, policy_version 805981 (0.0010) [2023-12-26 21:13:48,745][105620] Updated weights for policy 1, policy_version 805991 (0.0011) [2023-12-26 21:13:49,171][105692] Updated weights for policy 0, policy_version 806059 (0.0007) [2023-12-26 21:13:49,227][105692] Updated weights for policy 0, policy_version 806069 (0.0007) [2023-12-26 21:13:49,295][105692] Updated weights for policy 0, policy_version 806079 (0.0009) [2023-12-26 21:13:49,495][105620] Updated weights for policy 1, policy_version 806001 (0.0011) [2023-12-26 21:13:49,557][105620] Updated weights for policy 1, policy_version 806011 (0.0010) [2023-12-26 21:13:49,617][105620] Updated weights for policy 1, policy_version 806021 (0.0010) [2023-12-26 21:13:49,982][105692] Updated weights for policy 0, policy_version 806089 (0.0009) [2023-12-26 21:13:50,044][105692] Updated weights for policy 0, policy_version 806099 (0.0008) [2023-12-26 21:13:50,104][105692] Updated weights for policy 0, policy_version 806109 (0.0009) [2023-12-26 21:13:50,160][105692] Updated weights for policy 0, policy_version 806119 (0.0009) [2023-12-26 21:13:50,398][105620] Updated weights for policy 1, policy_version 806031 (0.0009) [2023-12-26 21:13:50,450][105620] Updated weights for policy 1, policy_version 806041 (0.0008) [2023-12-26 21:13:50,501][105620] Updated weights for policy 1, policy_version 806051 (0.0007) [2023-12-26 21:13:50,890][105692] Updated weights for policy 0, policy_version 806129 (0.0011) [2023-12-26 21:13:50,957][105692] Updated weights for policy 0, policy_version 806139 (0.0011) [2023-12-26 21:13:51,014][105692] Updated weights for policy 0, policy_version 806149 (0.0011) [2023-12-26 21:13:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 412778496. Throughput: 0: 9583.7, 1: 9741.7. Samples: 412766200. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 21:13:51,062][104569] Avg episode reward: [(0, '9344.682'), (1, '8755.044')] [2023-12-26 21:13:51,317][105620] Updated weights for policy 1, policy_version 806061 (0.0010) [2023-12-26 21:13:51,380][105620] Updated weights for policy 1, policy_version 806071 (0.0010) [2023-12-26 21:13:51,443][105620] Updated weights for policy 1, policy_version 806081 (0.0009) [2023-12-26 21:13:51,779][105692] Updated weights for policy 0, policy_version 806159 (0.0008) [2023-12-26 21:13:51,836][105692] Updated weights for policy 0, policy_version 806169 (0.0010) [2023-12-26 21:13:51,892][105692] Updated weights for policy 0, policy_version 806179 (0.0010) [2023-12-26 21:13:52,128][105620] Updated weights for policy 1, policy_version 806091 (0.0008) [2023-12-26 21:13:52,187][105620] Updated weights for policy 1, policy_version 806101 (0.0008) [2023-12-26 21:13:52,253][105620] Updated weights for policy 1, policy_version 806111 (0.0007) [2023-12-26 21:13:52,728][105692] Updated weights for policy 0, policy_version 806189 (0.0010) [2023-12-26 21:13:52,780][105692] Updated weights for policy 0, policy_version 806199 (0.0009) [2023-12-26 21:13:52,841][105692] Updated weights for policy 0, policy_version 806209 (0.0009) [2023-12-26 21:13:52,953][105620] Updated weights for policy 1, policy_version 806121 (0.0007) [2023-12-26 21:13:53,006][105620] Updated weights for policy 1, policy_version 806131 (0.0009) [2023-12-26 21:13:53,058][105620] Updated weights for policy 1, policy_version 806141 (0.0009) [2023-12-26 21:13:53,111][105620] Updated weights for policy 1, policy_version 806151 (0.0010) [2023-12-26 21:13:53,597][105692] Updated weights for policy 0, policy_version 806219 (0.0009) [2023-12-26 21:13:53,661][105692] Updated weights for policy 0, policy_version 806229 (0.0009) [2023-12-26 21:13:53,726][105692] Updated weights for policy 0, policy_version 806239 (0.0008) [2023-12-26 21:13:53,853][105620] Updated weights for policy 1, policy_version 806161 (0.0006) [2023-12-26 21:13:53,907][105620] Updated weights for policy 1, policy_version 806171 (0.0008) [2023-12-26 21:13:53,968][105620] Updated weights for policy 1, policy_version 806181 (0.0008) [2023-12-26 21:13:54,314][105692] Updated weights for policy 0, policy_version 806249 (0.0006) [2023-12-26 21:13:54,369][105692] Updated weights for policy 0, policy_version 806259 (0.0010) [2023-12-26 21:13:54,431][105692] Updated weights for policy 0, policy_version 806269 (0.0006) [2023-12-26 21:13:54,492][105692] Updated weights for policy 0, policy_version 806279 (0.0008) [2023-12-26 21:13:54,649][105620] Updated weights for policy 1, policy_version 806191 (0.0008) [2023-12-26 21:13:54,708][105620] Updated weights for policy 1, policy_version 806201 (0.0010) [2023-12-26 21:13:54,765][105620] Updated weights for policy 1, policy_version 806211 (0.0007) [2023-12-26 21:13:55,163][105692] Updated weights for policy 0, policy_version 806289 (0.0009) [2023-12-26 21:13:55,230][105692] Updated weights for policy 0, policy_version 806299 (0.0006) [2023-12-26 21:13:55,290][105692] Updated weights for policy 0, policy_version 806309 (0.0006) [2023-12-26 21:13:55,478][105620] Updated weights for policy 1, policy_version 806221 (0.0008) [2023-12-26 21:13:55,530][105620] Updated weights for policy 1, policy_version 806231 (0.0009) [2023-12-26 21:13:55,587][105620] Updated weights for policy 1, policy_version 806242 (0.0010) [2023-12-26 21:13:55,840][105692] Updated weights for policy 0, policy_version 806319 (0.0006) [2023-12-26 21:13:55,898][105692] Updated weights for policy 0, policy_version 806329 (0.0006) [2023-12-26 21:13:55,960][105692] Updated weights for policy 0, policy_version 806339 (0.0009) [2023-12-26 21:13:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 412876800. Throughput: 0: 9558.4, 1: 9801.1. Samples: 412882448. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 21:13:56,063][104569] Avg episode reward: [(0, '9253.000'), (1, '8839.170')] [2023-12-26 21:13:56,292][105620] Updated weights for policy 1, policy_version 806252 (0.0008) [2023-12-26 21:13:56,354][105620] Updated weights for policy 1, policy_version 806262 (0.0010) [2023-12-26 21:13:56,414][105620] Updated weights for policy 1, policy_version 806272 (0.0009) [2023-12-26 21:13:56,693][105692] Updated weights for policy 0, policy_version 806349 (0.0009) [2023-12-26 21:13:56,746][105692] Updated weights for policy 0, policy_version 806359 (0.0009) [2023-12-26 21:13:56,797][105692] Updated weights for policy 0, policy_version 806370 (0.0010) [2023-12-26 21:13:57,020][105620] Updated weights for policy 1, policy_version 806282 (0.0009) [2023-12-26 21:13:57,073][105620] Updated weights for policy 1, policy_version 806292 (0.0009) [2023-12-26 21:13:57,122][105620] Updated weights for policy 1, policy_version 806302 (0.0010) [2023-12-26 21:13:57,181][105620] Updated weights for policy 1, policy_version 806312 (0.0010) [2023-12-26 21:13:57,567][105692] Updated weights for policy 0, policy_version 806380 (0.0007) [2023-12-26 21:13:57,631][105692] Updated weights for policy 0, policy_version 806390 (0.0005) [2023-12-26 21:13:57,700][105692] Updated weights for policy 0, policy_version 806400 (0.0006) [2023-12-26 21:13:57,890][105620] Updated weights for policy 1, policy_version 806322 (0.0010) [2023-12-26 21:13:57,955][105620] Updated weights for policy 1, policy_version 806332 (0.0010) [2023-12-26 21:13:58,029][105620] Updated weights for policy 1, policy_version 806342 (0.0008) [2023-12-26 21:13:58,360][105692] Updated weights for policy 0, policy_version 806410 (0.0008) [2023-12-26 21:13:58,420][105692] Updated weights for policy 0, policy_version 806420 (0.0009) [2023-12-26 21:13:58,480][105692] Updated weights for policy 0, policy_version 806430 (0.0008) [2023-12-26 21:13:58,544][105692] Updated weights for policy 0, policy_version 806440 (0.0008) [2023-12-26 21:13:58,853][105620] Updated weights for policy 1, policy_version 806352 (0.0008) [2023-12-26 21:13:58,916][105620] Updated weights for policy 1, policy_version 806362 (0.0006) [2023-12-26 21:13:58,978][105620] Updated weights for policy 1, policy_version 806372 (0.0008) [2023-12-26 21:13:59,361][105692] Updated weights for policy 0, policy_version 806450 (0.0008) [2023-12-26 21:13:59,420][105692] Updated weights for policy 0, policy_version 806460 (0.0009) [2023-12-26 21:13:59,473][105692] Updated weights for policy 0, policy_version 806470 (0.0008) [2023-12-26 21:13:59,666][105620] Updated weights for policy 1, policy_version 806382 (0.0008) [2023-12-26 21:13:59,726][105620] Updated weights for policy 1, policy_version 806392 (0.0005) [2023-12-26 21:13:59,777][105620] Updated weights for policy 1, policy_version 806402 (0.0006) [2023-12-26 21:14:00,191][105692] Updated weights for policy 0, policy_version 806480 (0.0008) [2023-12-26 21:14:00,260][105692] Updated weights for policy 0, policy_version 806490 (0.0005) [2023-12-26 21:14:00,325][105692] Updated weights for policy 0, policy_version 806500 (0.0008) [2023-12-26 21:14:00,499][105620] Updated weights for policy 1, policy_version 806412 (0.0008) [2023-12-26 21:14:00,565][105620] Updated weights for policy 1, policy_version 806422 (0.0009) [2023-12-26 21:14:00,628][105620] Updated weights for policy 1, policy_version 806432 (0.0009) [2023-12-26 21:14:00,969][105692] Updated weights for policy 0, policy_version 806510 (0.0008) [2023-12-26 21:14:01,033][105692] Updated weights for policy 0, policy_version 806520 (0.0006) [2023-12-26 21:14:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 412966912. Throughput: 0: 9525.4, 1: 9867.0. Samples: 412940804. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 21:14:01,062][104569] Avg episode reward: [(0, '9252.019'), (1, '9084.965')] [2023-12-26 21:14:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000806440_206471168.pth... [2023-12-26 21:14:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000805288_206176256.pth [2023-12-26 21:14:01,097][105692] Updated weights for policy 0, policy_version 806530 (0.0009) [2023-12-26 21:14:01,130][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000806536_206503936.pth... [2023-12-26 21:14:01,134][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000805416_206217216.pth [2023-12-26 21:14:01,380][105620] Updated weights for policy 1, policy_version 806442 (0.0009) [2023-12-26 21:14:01,442][105620] Updated weights for policy 1, policy_version 806452 (0.0008) [2023-12-26 21:14:01,497][105586] KL-divergence is very high: 132.9289 [2023-12-26 21:14:01,502][105620] Updated weights for policy 1, policy_version 806462 (0.0008) [2023-12-26 21:14:01,539][105586] KL-divergence is very high: 117.0007 [2023-12-26 21:14:01,558][105620] Updated weights for policy 1, policy_version 806472 (0.0008) [2023-12-26 21:14:01,843][105692] Updated weights for policy 0, policy_version 806540 (0.0010) [2023-12-26 21:14:01,897][105692] Updated weights for policy 0, policy_version 806550 (0.0008) [2023-12-26 21:14:01,968][105692] Updated weights for policy 0, policy_version 806560 (0.0006) [2023-12-26 21:14:02,387][105620] Updated weights for policy 1, policy_version 806482 (0.0008) [2023-12-26 21:14:02,443][105620] Updated weights for policy 1, policy_version 806492 (0.0009) [2023-12-26 21:14:02,510][105620] Updated weights for policy 1, policy_version 806502 (0.0009) [2023-12-26 21:14:02,571][105692] Updated weights for policy 0, policy_version 806570 (0.0007) [2023-12-26 21:14:02,633][105692] Updated weights for policy 0, policy_version 806580 (0.0009) [2023-12-26 21:14:02,691][105692] Updated weights for policy 0, policy_version 806590 (0.0009) [2023-12-26 21:14:02,752][105692] Updated weights for policy 0, policy_version 806600 (0.0008) [2023-12-26 21:14:03,252][105620] Updated weights for policy 1, policy_version 806512 (0.0009) [2023-12-26 21:14:03,305][105620] Updated weights for policy 1, policy_version 806523 (0.0009) [2023-12-26 21:14:03,375][105620] Updated weights for policy 1, policy_version 806533 (0.0009) [2023-12-26 21:14:03,444][105692] Updated weights for policy 0, policy_version 806610 (0.0005) [2023-12-26 21:14:03,489][105692] Updated weights for policy 0, policy_version 806620 (0.0005) [2023-12-26 21:14:03,539][105692] Updated weights for policy 0, policy_version 806630 (0.0005) [2023-12-26 21:14:04,050][105620] Updated weights for policy 1, policy_version 806543 (0.0007) [2023-12-26 21:14:04,102][105620] Updated weights for policy 1, policy_version 806553 (0.0005) [2023-12-26 21:14:04,160][105620] Updated weights for policy 1, policy_version 806563 (0.0005) [2023-12-26 21:14:04,263][105692] Updated weights for policy 0, policy_version 806640 (0.0008) [2023-12-26 21:14:04,321][105692] Updated weights for policy 0, policy_version 806650 (0.0010) [2023-12-26 21:14:04,372][105692] Updated weights for policy 0, policy_version 806660 (0.0009) [2023-12-26 21:14:04,863][105620] Updated weights for policy 1, policy_version 806573 (0.0008) [2023-12-26 21:14:04,908][105620] Updated weights for policy 1, policy_version 806583 (0.0010) [2023-12-26 21:14:04,956][105620] Updated weights for policy 1, policy_version 806593 (0.0010) [2023-12-26 21:14:05,132][105692] Updated weights for policy 0, policy_version 806670 (0.0008) [2023-12-26 21:14:05,177][105692] Updated weights for policy 0, policy_version 806680 (0.0008) [2023-12-26 21:14:05,225][105692] Updated weights for policy 0, policy_version 806690 (0.0007) [2023-12-26 21:14:05,610][105620] Updated weights for policy 1, policy_version 806603 (0.0009) [2023-12-26 21:14:05,665][105620] Updated weights for policy 1, policy_version 806613 (0.0005) [2023-12-26 21:14:05,717][105620] Updated weights for policy 1, policy_version 806623 (0.0005) [2023-12-26 21:14:05,995][105692] Updated weights for policy 0, policy_version 806700 (0.0009) [2023-12-26 21:14:06,050][105692] Updated weights for policy 0, policy_version 806710 (0.0009) [2023-12-26 21:14:06,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 413065216. Throughput: 0: 9439.4, 1: 9812.6. Samples: 413056344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:14:06,062][104569] Avg episode reward: [(0, '9343.075'), (1, '9078.496')] [2023-12-26 21:14:06,116][105692] Updated weights for policy 0, policy_version 806720 (0.0008) [2023-12-26 21:14:06,403][105620] Updated weights for policy 1, policy_version 806633 (0.0007) [2023-12-26 21:14:06,473][105620] Updated weights for policy 1, policy_version 806643 (0.0007) [2023-12-26 21:14:06,524][105620] Updated weights for policy 1, policy_version 806653 (0.0005) [2023-12-26 21:14:06,581][105620] Updated weights for policy 1, policy_version 806663 (0.0006) [2023-12-26 21:14:06,724][105692] Updated weights for policy 0, policy_version 806730 (0.0007) [2023-12-26 21:14:06,786][105692] Updated weights for policy 0, policy_version 806740 (0.0005) [2023-12-26 21:14:06,849][105692] Updated weights for policy 0, policy_version 806750 (0.0006) [2023-12-26 21:14:06,912][105692] Updated weights for policy 0, policy_version 806760 (0.0010) [2023-12-26 21:14:07,208][105620] Updated weights for policy 1, policy_version 806673 (0.0006) [2023-12-26 21:14:07,268][105620] Updated weights for policy 1, policy_version 806683 (0.0006) [2023-12-26 21:14:07,330][105620] Updated weights for policy 1, policy_version 806693 (0.0008) [2023-12-26 21:14:07,610][105692] Updated weights for policy 0, policy_version 806770 (0.0005) [2023-12-26 21:14:07,673][105692] Updated weights for policy 0, policy_version 806780 (0.0005) [2023-12-26 21:14:07,738][105692] Updated weights for policy 0, policy_version 806790 (0.0006) [2023-12-26 21:14:07,957][105620] Updated weights for policy 1, policy_version 806703 (0.0007) [2023-12-26 21:14:08,015][105620] Updated weights for policy 1, policy_version 806713 (0.0005) [2023-12-26 21:14:08,073][105620] Updated weights for policy 1, policy_version 806723 (0.0005) [2023-12-26 21:14:08,472][105692] Updated weights for policy 0, policy_version 806800 (0.0009) [2023-12-26 21:14:08,541][105692] Updated weights for policy 0, policy_version 806810 (0.0010) [2023-12-26 21:14:08,601][105692] Updated weights for policy 0, policy_version 806820 (0.0009) [2023-12-26 21:14:08,606][105620] Updated weights for policy 1, policy_version 806733 (0.0005) [2023-12-26 21:14:08,666][105620] Updated weights for policy 1, policy_version 806743 (0.0006) [2023-12-26 21:14:08,727][105620] Updated weights for policy 1, policy_version 806753 (0.0008) [2023-12-26 21:14:09,302][105620] Updated weights for policy 1, policy_version 806763 (0.0010) [2023-12-26 21:14:09,377][105620] Updated weights for policy 1, policy_version 806774 (0.0009) [2023-12-26 21:14:09,435][105692] Updated weights for policy 0, policy_version 806830 (0.0008) [2023-12-26 21:14:09,444][105620] Updated weights for policy 1, policy_version 806784 (0.0008) [2023-12-26 21:14:09,503][105692] Updated weights for policy 0, policy_version 806840 (0.0007) [2023-12-26 21:14:09,566][105692] Updated weights for policy 0, policy_version 806850 (0.0005) [2023-12-26 21:14:10,162][105620] Updated weights for policy 1, policy_version 806794 (0.0007) [2023-12-26 21:14:10,210][105620] Updated weights for policy 1, policy_version 806804 (0.0009) [2023-12-26 21:14:10,269][105620] Updated weights for policy 1, policy_version 806814 (0.0009) [2023-12-26 21:14:10,312][105692] Updated weights for policy 0, policy_version 806860 (0.0007) [2023-12-26 21:14:10,326][105620] Updated weights for policy 1, policy_version 806824 (0.0010) [2023-12-26 21:14:10,377][105692] Updated weights for policy 0, policy_version 806870 (0.0008) [2023-12-26 21:14:10,437][105692] Updated weights for policy 0, policy_version 806880 (0.0009) [2023-12-26 21:14:11,039][105620] Updated weights for policy 1, policy_version 806834 (0.0009) [2023-12-26 21:14:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 413163520. Throughput: 0: 9471.6, 1: 9904.5. Samples: 413176788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:14:11,063][104569] Avg episode reward: [(0, '9343.805'), (1, '8806.347')] [2023-12-26 21:14:11,108][105620] Updated weights for policy 1, policy_version 806844 (0.0010) [2023-12-26 21:14:11,172][105620] Updated weights for policy 1, policy_version 806854 (0.0009) [2023-12-26 21:14:11,256][105692] Updated weights for policy 0, policy_version 806890 (0.0009) [2023-12-26 21:14:11,319][105692] Updated weights for policy 0, policy_version 806900 (0.0009) [2023-12-26 21:14:11,385][105692] Updated weights for policy 0, policy_version 806910 (0.0009) [2023-12-26 21:14:11,444][105692] Updated weights for policy 0, policy_version 806920 (0.0009) [2023-12-26 21:14:11,955][105620] Updated weights for policy 1, policy_version 806864 (0.0008) [2023-12-26 21:14:12,008][105620] Updated weights for policy 1, policy_version 806874 (0.0006) [2023-12-26 21:14:12,070][105620] Updated weights for policy 1, policy_version 806884 (0.0006) [2023-12-26 21:14:12,119][105692] Updated weights for policy 0, policy_version 806930 (0.0009) [2023-12-26 21:14:12,176][105692] Updated weights for policy 0, policy_version 806940 (0.0009) [2023-12-26 21:14:12,235][105692] Updated weights for policy 0, policy_version 806950 (0.0010) [2023-12-26 21:14:12,745][105620] Updated weights for policy 1, policy_version 806894 (0.0008) [2023-12-26 21:14:12,799][105620] Updated weights for policy 1, policy_version 806904 (0.0009) [2023-12-26 21:14:12,847][105620] Updated weights for policy 1, policy_version 806914 (0.0009) [2023-12-26 21:14:13,036][105692] Updated weights for policy 0, policy_version 806960 (0.0007) [2023-12-26 21:14:13,091][105692] Updated weights for policy 0, policy_version 806970 (0.0006) [2023-12-26 21:14:13,146][105692] Updated weights for policy 0, policy_version 806980 (0.0006) [2023-12-26 21:14:13,519][105620] Updated weights for policy 1, policy_version 806924 (0.0008) [2023-12-26 21:14:13,563][105620] Updated weights for policy 1, policy_version 806934 (0.0005) [2023-12-26 21:14:13,611][105620] Updated weights for policy 1, policy_version 806944 (0.0009) [2023-12-26 21:14:13,697][105692] Updated weights for policy 0, policy_version 806990 (0.0009) [2023-12-26 21:14:13,739][105692] Updated weights for policy 0, policy_version 807000 (0.0009) [2023-12-26 21:14:13,787][105692] Updated weights for policy 0, policy_version 807010 (0.0008) [2023-12-26 21:14:14,362][105620] Updated weights for policy 1, policy_version 806954 (0.0010) [2023-12-26 21:14:14,406][105620] Updated weights for policy 1, policy_version 806964 (0.0011) [2023-12-26 21:14:14,451][105620] Updated weights for policy 1, policy_version 806974 (0.0010) [2023-12-26 21:14:14,500][105620] Updated weights for policy 1, policy_version 806984 (0.0010) [2023-12-26 21:14:14,561][105692] Updated weights for policy 0, policy_version 807020 (0.0008) [2023-12-26 21:14:14,616][105692] Updated weights for policy 0, policy_version 807030 (0.0008) [2023-12-26 21:14:14,663][105692] Updated weights for policy 0, policy_version 807040 (0.0005) [2023-12-26 21:14:15,253][105620] Updated weights for policy 1, policy_version 806994 (0.0011) [2023-12-26 21:14:15,324][105620] Updated weights for policy 1, policy_version 807004 (0.0011) [2023-12-26 21:14:15,339][105692] Updated weights for policy 0, policy_version 807050 (0.0006) [2023-12-26 21:14:15,386][105620] Updated weights for policy 1, policy_version 807014 (0.0007) [2023-12-26 21:14:15,402][105692] Updated weights for policy 0, policy_version 807060 (0.0011) [2023-12-26 21:14:15,459][105692] Updated weights for policy 0, policy_version 807070 (0.0011) [2023-12-26 21:14:15,525][105692] Updated weights for policy 0, policy_version 807080 (0.0008) [2023-12-26 21:14:16,059][105620] Updated weights for policy 1, policy_version 807024 (0.0007) [2023-12-26 21:14:16,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19251.1, 300 sec: 19410.9). Total num frames: 413261824. Throughput: 0: 9453.5, 1: 9843.9. Samples: 413235360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:14:16,063][104569] Avg episode reward: [(0, '9251.803'), (1, '5849.830')] [2023-12-26 21:14:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000807080_206643200.pth... [2023-12-26 21:14:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000805960_206356480.pth [2023-12-26 21:14:16,107][105620] Updated weights for policy 1, policy_version 807034 (0.0008) [2023-12-26 21:14:16,162][105692] Updated weights for policy 0, policy_version 807090 (0.0007) [2023-12-26 21:14:16,165][105620] Updated weights for policy 1, policy_version 807044 (0.0006) [2023-12-26 21:14:16,189][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000807048_206626816.pth... [2023-12-26 21:14:16,192][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000805864_206323712.pth [2023-12-26 21:14:16,217][105692] Updated weights for policy 0, policy_version 807100 (0.0009) [2023-12-26 21:14:16,277][105692] Updated weights for policy 0, policy_version 807110 (0.0008) [2023-12-26 21:14:16,827][105620] Updated weights for policy 1, policy_version 807054 (0.0008) [2023-12-26 21:14:16,884][105620] Updated weights for policy 1, policy_version 807064 (0.0010) [2023-12-26 21:14:16,930][105692] Updated weights for policy 0, policy_version 807120 (0.0011) [2023-12-26 21:14:16,941][105620] Updated weights for policy 1, policy_version 807074 (0.0009) [2023-12-26 21:14:16,979][105692] Updated weights for policy 0, policy_version 807130 (0.0010) [2023-12-26 21:14:17,034][105692] Updated weights for policy 0, policy_version 807140 (0.0011) [2023-12-26 21:14:17,679][105620] Updated weights for policy 1, policy_version 807084 (0.0010) [2023-12-26 21:14:17,722][105692] Updated weights for policy 0, policy_version 807150 (0.0009) [2023-12-26 21:14:17,743][105620] Updated weights for policy 1, policy_version 807094 (0.0010) [2023-12-26 21:14:17,789][105692] Updated weights for policy 0, policy_version 807160 (0.0008) [2023-12-26 21:14:17,807][105620] Updated weights for policy 1, policy_version 807104 (0.0010) [2023-12-26 21:14:17,829][105585] KL-divergence is very high: 100.8416 [2023-12-26 21:14:17,851][105692] Updated weights for policy 0, policy_version 807170 (0.0007) [2023-12-26 21:14:18,519][105620] Updated weights for policy 1, policy_version 807114 (0.0010) [2023-12-26 21:14:18,522][105692] Updated weights for policy 0, policy_version 807180 (0.0008) [2023-12-26 21:14:18,570][105620] Updated weights for policy 1, policy_version 807124 (0.0010) [2023-12-26 21:14:18,582][105692] Updated weights for policy 0, policy_version 807190 (0.0011) [2023-12-26 21:14:18,630][105620] Updated weights for policy 1, policy_version 807134 (0.0011) [2023-12-26 21:14:18,647][105692] Updated weights for policy 0, policy_version 807200 (0.0008) [2023-12-26 21:14:18,683][105620] Updated weights for policy 1, policy_version 807144 (0.0011) [2023-12-26 21:14:19,233][105692] Updated weights for policy 0, policy_version 807210 (0.0006) [2023-12-26 21:14:19,293][105692] Updated weights for policy 0, policy_version 807220 (0.0007) [2023-12-26 21:14:19,354][105692] Updated weights for policy 0, policy_version 807230 (0.0008) [2023-12-26 21:14:19,415][105692] Updated weights for policy 0, policy_version 807240 (0.0009) [2023-12-26 21:14:19,481][105620] Updated weights for policy 1, policy_version 807154 (0.0009) [2023-12-26 21:14:19,549][105620] Updated weights for policy 1, policy_version 807164 (0.0009) [2023-12-26 21:14:19,604][105620] Updated weights for policy 1, policy_version 807174 (0.0007) [2023-12-26 21:14:20,165][105692] Updated weights for policy 0, policy_version 807250 (0.0010) [2023-12-26 21:14:20,221][105692] Updated weights for policy 0, policy_version 807260 (0.0010) [2023-12-26 21:14:20,274][105692] Updated weights for policy 0, policy_version 807270 (0.0011) [2023-12-26 21:14:20,391][105620] Updated weights for policy 1, policy_version 807184 (0.0009) [2023-12-26 21:14:20,445][105620] Updated weights for policy 1, policy_version 807194 (0.0009) [2023-12-26 21:14:20,499][105620] Updated weights for policy 1, policy_version 807204 (0.0008) [2023-12-26 21:14:21,044][105692] Updated weights for policy 0, policy_version 807280 (0.0011) [2023-12-26 21:14:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 413360128. Throughput: 0: 9524.0, 1: 9828.8. Samples: 413354440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:14:21,062][104569] Avg episode reward: [(0, '938.742'), (1, '5821.084')] [2023-12-26 21:14:21,105][105692] Updated weights for policy 0, policy_version 807290 (0.0011) [2023-12-26 21:14:21,172][105692] Updated weights for policy 0, policy_version 807300 (0.0010) [2023-12-26 21:14:21,296][105620] Updated weights for policy 1, policy_version 807214 (0.0007) [2023-12-26 21:14:21,364][105620] Updated weights for policy 1, policy_version 807224 (0.0007) [2023-12-26 21:14:21,428][105620] Updated weights for policy 1, policy_version 807234 (0.0006) [2023-12-26 21:14:21,958][105692] Updated weights for policy 0, policy_version 807310 (0.0009) [2023-12-26 21:14:22,006][105692] Updated weights for policy 0, policy_version 807320 (0.0010) [2023-12-26 21:14:22,064][105692] Updated weights for policy 0, policy_version 807330 (0.0009) [2023-12-26 21:14:22,202][105620] Updated weights for policy 1, policy_version 807244 (0.0007) [2023-12-26 21:14:22,259][105620] Updated weights for policy 1, policy_version 807255 (0.0011) [2023-12-26 21:14:22,324][105620] Updated weights for policy 1, policy_version 807265 (0.0008) [2023-12-26 21:14:22,744][105692] Updated weights for policy 0, policy_version 807340 (0.0010) [2023-12-26 21:14:22,796][105692] Updated weights for policy 0, policy_version 807350 (0.0010) [2023-12-26 21:14:22,852][105692] Updated weights for policy 0, policy_version 807360 (0.0010) [2023-12-26 21:14:23,149][105620] Updated weights for policy 1, policy_version 807275 (0.0010) [2023-12-26 21:14:23,203][105620] Updated weights for policy 1, policy_version 807285 (0.0009) [2023-12-26 21:14:23,267][105620] Updated weights for policy 1, policy_version 807295 (0.0009) [2023-12-26 21:14:23,558][105692] Updated weights for policy 0, policy_version 807370 (0.0010) [2023-12-26 21:14:23,618][105692] Updated weights for policy 0, policy_version 807380 (0.0010) [2023-12-26 21:14:23,673][105692] Updated weights for policy 0, policy_version 807390 (0.0010) [2023-12-26 21:14:23,734][105692] Updated weights for policy 0, policy_version 807400 (0.0010) [2023-12-26 21:14:24,002][105620] Updated weights for policy 1, policy_version 807305 (0.0007) [2023-12-26 21:14:24,067][105620] Updated weights for policy 1, policy_version 807315 (0.0005) [2023-12-26 21:14:24,124][105620] Updated weights for policy 1, policy_version 807325 (0.0009) [2023-12-26 21:14:24,179][105620] Updated weights for policy 1, policy_version 807335 (0.0010) [2023-12-26 21:14:24,464][105692] Updated weights for policy 0, policy_version 807410 (0.0010) [2023-12-26 21:14:24,516][105692] Updated weights for policy 0, policy_version 807420 (0.0010) [2023-12-26 21:14:24,571][105692] Updated weights for policy 0, policy_version 807430 (0.0010) [2023-12-26 21:14:24,858][105620] Updated weights for policy 1, policy_version 807345 (0.0006) [2023-12-26 21:14:24,917][105620] Updated weights for policy 1, policy_version 807355 (0.0007) [2023-12-26 21:14:24,974][105620] Updated weights for policy 1, policy_version 807365 (0.0006) [2023-12-26 21:14:25,330][105692] Updated weights for policy 0, policy_version 807440 (0.0010) [2023-12-26 21:14:25,381][105692] Updated weights for policy 0, policy_version 807450 (0.0010) [2023-12-26 21:14:25,442][105692] Updated weights for policy 0, policy_version 807460 (0.0010) [2023-12-26 21:14:25,671][105620] Updated weights for policy 1, policy_version 807375 (0.0008) [2023-12-26 21:14:25,726][105620] Updated weights for policy 1, policy_version 807385 (0.0008) [2023-12-26 21:14:25,780][105620] Updated weights for policy 1, policy_version 807395 (0.0008) [2023-12-26 21:14:26,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 413458432. Throughput: 0: 9608.7, 1: 9853.0. Samples: 413468028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:14:26,062][104569] Avg episode reward: [(0, '883.323'), (1, '7208.484')] [2023-12-26 21:14:26,206][105692] Updated weights for policy 0, policy_version 807470 (0.0011) [2023-12-26 21:14:26,265][105692] Updated weights for policy 0, policy_version 807480 (0.0011) [2023-12-26 21:14:26,331][105692] Updated weights for policy 0, policy_version 807490 (0.0008) [2023-12-26 21:14:26,371][105620] Updated weights for policy 1, policy_version 807405 (0.0007) [2023-12-26 21:14:26,431][105620] Updated weights for policy 1, policy_version 807415 (0.0008) [2023-12-26 21:14:26,495][105620] Updated weights for policy 1, policy_version 807425 (0.0008) [2023-12-26 21:14:26,957][105692] Updated weights for policy 0, policy_version 807500 (0.0005) [2023-12-26 21:14:27,019][105692] Updated weights for policy 0, policy_version 807510 (0.0006) [2023-12-26 21:14:27,084][105692] Updated weights for policy 0, policy_version 807520 (0.0005) [2023-12-26 21:14:27,315][105620] Updated weights for policy 1, policy_version 807435 (0.0008) [2023-12-26 21:14:27,368][105620] Updated weights for policy 1, policy_version 807445 (0.0009) [2023-12-26 21:14:27,416][105620] Updated weights for policy 1, policy_version 807455 (0.0008) [2023-12-26 21:14:27,727][105692] Updated weights for policy 0, policy_version 807530 (0.0005) [2023-12-26 21:14:27,773][105692] Updated weights for policy 0, policy_version 807540 (0.0005) [2023-12-26 21:14:27,834][105692] Updated weights for policy 0, policy_version 807550 (0.0005) [2023-12-26 21:14:27,888][105692] Updated weights for policy 0, policy_version 807560 (0.0005) [2023-12-26 21:14:28,080][105620] Updated weights for policy 1, policy_version 807465 (0.0008) [2023-12-26 21:14:28,128][105620] Updated weights for policy 1, policy_version 807475 (0.0010) [2023-12-26 21:14:28,186][105620] Updated weights for policy 1, policy_version 807485 (0.0010) [2023-12-26 21:14:28,244][105620] Updated weights for policy 1, policy_version 807495 (0.0010) [2023-12-26 21:14:28,528][105692] Updated weights for policy 0, policy_version 807570 (0.0010) [2023-12-26 21:14:28,577][105692] Updated weights for policy 0, policy_version 807580 (0.0008) [2023-12-26 21:14:28,630][105692] Updated weights for policy 0, policy_version 807590 (0.0009) [2023-12-26 21:14:28,876][105620] Updated weights for policy 1, policy_version 807505 (0.0009) [2023-12-26 21:14:28,925][105620] Updated weights for policy 1, policy_version 807515 (0.0005) [2023-12-26 21:14:28,974][105620] Updated weights for policy 1, policy_version 807525 (0.0005) [2023-12-26 21:14:29,399][105692] Updated weights for policy 0, policy_version 807600 (0.0011) [2023-12-26 21:14:29,455][105692] Updated weights for policy 0, policy_version 807610 (0.0011) [2023-12-26 21:14:29,518][105692] Updated weights for policy 0, policy_version 807620 (0.0011) [2023-12-26 21:14:29,561][105620] Updated weights for policy 1, policy_version 807535 (0.0009) [2023-12-26 21:14:29,616][105620] Updated weights for policy 1, policy_version 807545 (0.0010) [2023-12-26 21:14:29,671][105620] Updated weights for policy 1, policy_version 807555 (0.0010) [2023-12-26 21:14:30,174][105692] Updated weights for policy 0, policy_version 807630 (0.0008) [2023-12-26 21:14:30,238][105692] Updated weights for policy 0, policy_version 807640 (0.0008) [2023-12-26 21:14:30,305][105692] Updated weights for policy 0, policy_version 807650 (0.0008) [2023-12-26 21:14:30,398][105620] Updated weights for policy 1, policy_version 807565 (0.0010) [2023-12-26 21:14:30,457][105620] Updated weights for policy 1, policy_version 807575 (0.0011) [2023-12-26 21:14:30,516][105620] Updated weights for policy 1, policy_version 807585 (0.0010) [2023-12-26 21:14:30,906][105692] Updated weights for policy 0, policy_version 807660 (0.0008) [2023-12-26 21:14:30,960][105692] Updated weights for policy 0, policy_version 807670 (0.0010) [2023-12-26 21:14:31,019][105692] Updated weights for policy 0, policy_version 807680 (0.0006) [2023-12-26 21:14:31,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 413556736. Throughput: 0: 9701.6, 1: 9927.4. Samples: 413529880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:14:31,063][104569] Avg episode reward: [(0, '741.977'), (1, '9021.825')] [2023-12-26 21:14:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000807592_206766080.pth... [2023-12-26 21:14:31,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000807688_206798848.pth... [2023-12-26 21:14:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000806440_206471168.pth [2023-12-26 21:14:31,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000806536_206503936.pth [2023-12-26 21:14:31,227][105620] Updated weights for policy 1, policy_version 807595 (0.0011) [2023-12-26 21:14:31,287][105620] Updated weights for policy 1, policy_version 807605 (0.0011) [2023-12-26 21:14:31,353][105620] Updated weights for policy 1, policy_version 807615 (0.0011) [2023-12-26 21:14:31,788][105692] Updated weights for policy 0, policy_version 807690 (0.0009) [2023-12-26 21:14:31,843][105692] Updated weights for policy 0, policy_version 807700 (0.0008) [2023-12-26 21:14:31,906][105692] Updated weights for policy 0, policy_version 807710 (0.0008) [2023-12-26 21:14:31,969][105692] Updated weights for policy 0, policy_version 807720 (0.0008) [2023-12-26 21:14:32,060][105620] Updated weights for policy 1, policy_version 807625 (0.0010) [2023-12-26 21:14:32,125][105620] Updated weights for policy 1, policy_version 807635 (0.0011) [2023-12-26 21:14:32,181][105620] Updated weights for policy 1, policy_version 807645 (0.0010) [2023-12-26 21:14:32,239][105620] Updated weights for policy 1, policy_version 807655 (0.0010) [2023-12-26 21:14:32,632][105692] Updated weights for policy 0, policy_version 807730 (0.0008) [2023-12-26 21:14:32,697][105692] Updated weights for policy 0, policy_version 807740 (0.0008) [2023-12-26 21:14:32,762][105692] Updated weights for policy 0, policy_version 807750 (0.0008) [2023-12-26 21:14:33,025][105620] Updated weights for policy 1, policy_version 807665 (0.0009) [2023-12-26 21:14:33,081][105620] Updated weights for policy 1, policy_version 807675 (0.0009) [2023-12-26 21:14:33,137][105620] Updated weights for policy 1, policy_version 807685 (0.0009) [2023-12-26 21:14:33,440][105692] Updated weights for policy 0, policy_version 807760 (0.0009) [2023-12-26 21:14:33,491][105692] Updated weights for policy 0, policy_version 807770 (0.0009) [2023-12-26 21:14:33,544][105692] Updated weights for policy 0, policy_version 807780 (0.0008) [2023-12-26 21:14:33,894][105620] Updated weights for policy 1, policy_version 807695 (0.0008) [2023-12-26 21:14:33,959][105620] Updated weights for policy 1, policy_version 807705 (0.0009) [2023-12-26 21:14:34,025][105620] Updated weights for policy 1, policy_version 807715 (0.0009) [2023-12-26 21:14:34,235][105692] Updated weights for policy 0, policy_version 807790 (0.0010) [2023-12-26 21:14:34,284][105692] Updated weights for policy 0, policy_version 807800 (0.0009) [2023-12-26 21:14:34,339][105692] Updated weights for policy 0, policy_version 807810 (0.0009) [2023-12-26 21:14:34,761][105620] Updated weights for policy 1, policy_version 807725 (0.0007) [2023-12-26 21:14:34,819][105620] Updated weights for policy 1, policy_version 807735 (0.0010) [2023-12-26 21:14:34,877][105620] Updated weights for policy 1, policy_version 807745 (0.0009) [2023-12-26 21:14:35,069][105692] Updated weights for policy 0, policy_version 807820 (0.0008) [2023-12-26 21:14:35,132][105692] Updated weights for policy 0, policy_version 807830 (0.0009) [2023-12-26 21:14:35,198][105692] Updated weights for policy 0, policy_version 807840 (0.0008) [2023-12-26 21:14:35,647][105620] Updated weights for policy 1, policy_version 807755 (0.0008) [2023-12-26 21:14:35,702][105620] Updated weights for policy 1, policy_version 807765 (0.0005) [2023-12-26 21:14:35,755][105620] Updated weights for policy 1, policy_version 807775 (0.0008) [2023-12-26 21:14:35,984][105692] Updated weights for policy 0, policy_version 807850 (0.0008) [2023-12-26 21:14:36,045][105692] Updated weights for policy 0, policy_version 807860 (0.0010) [2023-12-26 21:14:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 413655040. Throughput: 0: 9796.7, 1: 9760.9. Samples: 413646296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:14:36,063][104569] Avg episode reward: [(0, '533.357'), (1, '9174.777')] [2023-12-26 21:14:36,116][105692] Updated weights for policy 0, policy_version 807870 (0.0009) [2023-12-26 21:14:36,184][105692] Updated weights for policy 0, policy_version 807880 (0.0010) [2023-12-26 21:14:36,335][105620] Updated weights for policy 1, policy_version 807785 (0.0010) [2023-12-26 21:14:36,401][105620] Updated weights for policy 1, policy_version 807795 (0.0006) [2023-12-26 21:14:36,462][105620] Updated weights for policy 1, policy_version 807805 (0.0006) [2023-12-26 21:14:36,529][105620] Updated weights for policy 1, policy_version 807815 (0.0007) [2023-12-26 21:14:37,049][105692] Updated weights for policy 0, policy_version 807890 (0.0008) [2023-12-26 21:14:37,111][105692] Updated weights for policy 0, policy_version 807900 (0.0009) [2023-12-26 21:14:37,131][105620] Updated weights for policy 1, policy_version 807825 (0.0010) [2023-12-26 21:14:37,174][105692] Updated weights for policy 0, policy_version 807910 (0.0010) [2023-12-26 21:14:37,192][105620] Updated weights for policy 1, policy_version 807835 (0.0009) [2023-12-26 21:14:37,265][105620] Updated weights for policy 1, policy_version 807845 (0.0007) [2023-12-26 21:14:37,913][105620] Updated weights for policy 1, policy_version 807855 (0.0008) [2023-12-26 21:14:37,960][105620] Updated weights for policy 1, policy_version 807865 (0.0006) [2023-12-26 21:14:37,988][105692] Updated weights for policy 0, policy_version 807920 (0.0008) [2023-12-26 21:14:38,007][105620] Updated weights for policy 1, policy_version 807875 (0.0007) [2023-12-26 21:14:38,052][105692] Updated weights for policy 0, policy_version 807930 (0.0006) [2023-12-26 21:14:38,124][105692] Updated weights for policy 0, policy_version 807940 (0.0005) [2023-12-26 21:14:38,765][105692] Updated weights for policy 0, policy_version 807950 (0.0008) [2023-12-26 21:14:38,795][105620] Updated weights for policy 1, policy_version 807885 (0.0008) [2023-12-26 21:14:38,825][105692] Updated weights for policy 0, policy_version 807960 (0.0011) [2023-12-26 21:14:38,840][105620] Updated weights for policy 1, policy_version 807895 (0.0006) [2023-12-26 21:14:38,885][105692] Updated weights for policy 0, policy_version 807970 (0.0011) [2023-12-26 21:14:38,888][105620] Updated weights for policy 1, policy_version 807905 (0.0006) [2023-12-26 21:14:39,546][105620] Updated weights for policy 1, policy_version 807915 (0.0008) [2023-12-26 21:14:39,561][105692] Updated weights for policy 0, policy_version 807980 (0.0010) [2023-12-26 21:14:39,611][105620] Updated weights for policy 1, policy_version 807925 (0.0008) [2023-12-26 21:14:39,623][105692] Updated weights for policy 0, policy_version 807990 (0.0007) [2023-12-26 21:14:39,669][105620] Updated weights for policy 1, policy_version 807935 (0.0007) [2023-12-26 21:14:39,679][105692] Updated weights for policy 0, policy_version 808000 (0.0011) [2023-12-26 21:14:40,393][105692] Updated weights for policy 0, policy_version 808010 (0.0010) [2023-12-26 21:14:40,453][105692] Updated weights for policy 0, policy_version 808020 (0.0007) [2023-12-26 21:14:40,484][105620] Updated weights for policy 1, policy_version 807945 (0.0006) [2023-12-26 21:14:40,506][105692] Updated weights for policy 0, policy_version 808030 (0.0009) [2023-12-26 21:14:40,541][105620] Updated weights for policy 1, policy_version 807955 (0.0006) [2023-12-26 21:14:40,565][105692] Updated weights for policy 0, policy_version 808040 (0.0007) [2023-12-26 21:14:40,603][105620] Updated weights for policy 1, policy_version 807965 (0.0007) [2023-12-26 21:14:40,661][105620] Updated weights for policy 1, policy_version 807975 (0.0009) [2023-12-26 21:14:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 413753344. Throughput: 0: 9732.3, 1: 9820.9. Samples: 413762340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:14:41,062][104569] Avg episode reward: [(0, '659.719'), (1, '8448.072')] [2023-12-26 21:14:41,255][105692] Updated weights for policy 0, policy_version 808050 (0.0011) [2023-12-26 21:14:41,316][105692] Updated weights for policy 0, policy_version 808060 (0.0009) [2023-12-26 21:14:41,387][105692] Updated weights for policy 0, policy_version 808070 (0.0009) [2023-12-26 21:14:41,497][105620] Updated weights for policy 1, policy_version 807985 (0.0008) [2023-12-26 21:14:41,552][105620] Updated weights for policy 1, policy_version 807995 (0.0009) [2023-12-26 21:14:41,614][105620] Updated weights for policy 1, policy_version 808005 (0.0008) [2023-12-26 21:14:42,096][105692] Updated weights for policy 0, policy_version 808080 (0.0010) [2023-12-26 21:14:42,160][105692] Updated weights for policy 0, policy_version 808090 (0.0010) [2023-12-26 21:14:42,220][105692] Updated weights for policy 0, policy_version 808100 (0.0011) [2023-12-26 21:14:42,367][105620] Updated weights for policy 1, policy_version 808015 (0.0007) [2023-12-26 21:14:42,441][105620] Updated weights for policy 1, policy_version 808025 (0.0006) [2023-12-26 21:14:42,498][105586] KL-divergence is very high: 304.5110 [2023-12-26 21:14:42,512][105620] Updated weights for policy 1, policy_version 808035 (0.0005) [2023-12-26 21:14:42,880][105692] Updated weights for policy 0, policy_version 808110 (0.0010) [2023-12-26 21:14:42,936][105692] Updated weights for policy 0, policy_version 808120 (0.0010) [2023-12-26 21:14:42,985][105692] Updated weights for policy 0, policy_version 808130 (0.0010) [2023-12-26 21:14:43,194][105620] Updated weights for policy 1, policy_version 808045 (0.0006) [2023-12-26 21:14:43,263][105620] Updated weights for policy 1, policy_version 808055 (0.0006) [2023-12-26 21:14:43,323][105620] Updated weights for policy 1, policy_version 808065 (0.0006) [2023-12-26 21:14:43,739][105692] Updated weights for policy 0, policy_version 808140 (0.0010) [2023-12-26 21:14:43,794][105692] Updated weights for policy 0, policy_version 808150 (0.0010) [2023-12-26 21:14:43,851][105692] Updated weights for policy 0, policy_version 808160 (0.0010) [2023-12-26 21:14:43,902][105620] Updated weights for policy 1, policy_version 808075 (0.0007) [2023-12-26 21:14:43,951][105620] Updated weights for policy 1, policy_version 808085 (0.0008) [2023-12-26 21:14:44,009][105620] Updated weights for policy 1, policy_version 808095 (0.0010) [2023-12-26 21:14:44,542][105692] Updated weights for policy 0, policy_version 808170 (0.0010) [2023-12-26 21:14:44,599][105692] Updated weights for policy 0, policy_version 808180 (0.0010) [2023-12-26 21:14:44,657][105692] Updated weights for policy 0, policy_version 808190 (0.0010) [2023-12-26 21:14:44,711][105692] Updated weights for policy 0, policy_version 808200 (0.0010) [2023-12-26 21:14:44,806][105620] Updated weights for policy 1, policy_version 808105 (0.0009) [2023-12-26 21:14:44,872][105620] Updated weights for policy 1, policy_version 808115 (0.0009) [2023-12-26 21:14:44,923][105620] Updated weights for policy 1, policy_version 808125 (0.0008) [2023-12-26 21:14:44,987][105620] Updated weights for policy 1, policy_version 808135 (0.0009) [2023-12-26 21:14:45,407][105692] Updated weights for policy 0, policy_version 808210 (0.0005) [2023-12-26 21:14:45,464][105692] Updated weights for policy 0, policy_version 808220 (0.0005) [2023-12-26 21:14:45,519][105692] Updated weights for policy 0, policy_version 808230 (0.0005) [2023-12-26 21:14:45,820][105620] Updated weights for policy 1, policy_version 808145 (0.0009) [2023-12-26 21:14:45,873][105620] Updated weights for policy 1, policy_version 808155 (0.0009) [2023-12-26 21:14:45,919][105620] Updated weights for policy 1, policy_version 808165 (0.0008) [2023-12-26 21:14:46,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 413851648. Throughput: 0: 9749.3, 1: 9815.0. Samples: 413821196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:14:46,062][104569] Avg episode reward: [(0, '1065.143'), (1, '6108.113')] [2023-12-26 21:14:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000808232_206938112.pth... [2023-12-26 21:14:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000808168_206913536.pth... [2023-12-26 21:14:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000807080_206643200.pth [2023-12-26 21:14:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000807048_206626816.pth [2023-12-26 21:14:46,142][105692] Updated weights for policy 0, policy_version 808240 (0.0008) [2023-12-26 21:14:46,204][105692] Updated weights for policy 0, policy_version 808250 (0.0009) [2023-12-26 21:14:46,266][105692] Updated weights for policy 0, policy_version 808260 (0.0008) [2023-12-26 21:14:46,684][105620] Updated weights for policy 1, policy_version 808175 (0.0009) [2023-12-26 21:14:46,732][105620] Updated weights for policy 1, policy_version 808185 (0.0009) [2023-12-26 21:14:46,789][105620] Updated weights for policy 1, policy_version 808195 (0.0009) [2023-12-26 21:14:46,820][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000008 [2023-12-26 21:14:47,027][105692] Updated weights for policy 0, policy_version 808270 (0.0009) [2023-12-26 21:14:47,085][105692] Updated weights for policy 0, policy_version 808280 (0.0009) [2023-12-26 21:14:47,136][105692] Updated weights for policy 0, policy_version 808290 (0.0009) [2023-12-26 21:14:47,556][105620] Updated weights for policy 1, policy_version 808205 (0.0007) [2023-12-26 21:14:47,605][105620] Updated weights for policy 1, policy_version 808215 (0.0005) [2023-12-26 21:14:47,657][105620] Updated weights for policy 1, policy_version 808225 (0.0005) [2023-12-26 21:14:47,872][105692] Updated weights for policy 0, policy_version 808300 (0.0007) [2023-12-26 21:14:47,926][105692] Updated weights for policy 0, policy_version 808310 (0.0005) [2023-12-26 21:14:47,986][105692] Updated weights for policy 0, policy_version 808320 (0.0005) [2023-12-26 21:14:48,408][105620] Updated weights for policy 1, policy_version 808235 (0.0008) [2023-12-26 21:14:48,466][105620] Updated weights for policy 1, policy_version 808245 (0.0009) [2023-12-26 21:14:48,524][105620] Updated weights for policy 1, policy_version 808255 (0.0010) [2023-12-26 21:14:48,556][105692] Updated weights for policy 0, policy_version 808330 (0.0005) [2023-12-26 21:14:48,613][105692] Updated weights for policy 0, policy_version 808340 (0.0005) [2023-12-26 21:14:48,678][105692] Updated weights for policy 0, policy_version 808350 (0.0008) [2023-12-26 21:14:48,738][105692] Updated weights for policy 0, policy_version 808360 (0.0009) [2023-12-26 21:14:49,175][105620] Updated weights for policy 1, policy_version 808265 (0.0007) [2023-12-26 21:14:49,242][105620] Updated weights for policy 1, policy_version 808275 (0.0008) [2023-12-26 21:14:49,312][105620] Updated weights for policy 1, policy_version 808285 (0.0006) [2023-12-26 21:14:49,378][105620] Updated weights for policy 1, policy_version 808295 (0.0009) [2023-12-26 21:14:49,556][105692] Updated weights for policy 0, policy_version 808371 (0.0009) [2023-12-26 21:14:49,603][105692] Updated weights for policy 0, policy_version 808381 (0.0008) [2023-12-26 21:14:49,672][105692] Updated weights for policy 0, policy_version 808391 (0.0007) [2023-12-26 21:14:50,099][105620] Updated weights for policy 1, policy_version 808305 (0.0010) [2023-12-26 21:14:50,149][105620] Updated weights for policy 1, policy_version 808315 (0.0009) [2023-12-26 21:14:50,204][105620] Updated weights for policy 1, policy_version 808325 (0.0009) [2023-12-26 21:14:50,331][105692] Updated weights for policy 0, policy_version 808401 (0.0006) [2023-12-26 21:14:50,392][105692] Updated weights for policy 0, policy_version 808411 (0.0006) [2023-12-26 21:14:50,452][105692] Updated weights for policy 0, policy_version 808421 (0.0005) [2023-12-26 21:14:51,052][105620] Updated weights for policy 1, policy_version 808335 (0.0009) [2023-12-26 21:14:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 413941760. Throughput: 0: 9767.7, 1: 9791.1. Samples: 413936488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:14:51,063][104569] Avg episode reward: [(0, '3186.322'), (1, '6953.967')] [2023-12-26 21:14:51,125][105620] Updated weights for policy 1, policy_version 808345 (0.0008) [2023-12-26 21:14:51,139][105692] Updated weights for policy 0, policy_version 808431 (0.0008) [2023-12-26 21:14:51,191][105620] Updated weights for policy 1, policy_version 808355 (0.0006) [2023-12-26 21:14:51,200][105692] Updated weights for policy 0, policy_version 808441 (0.0007) [2023-12-26 21:14:51,260][105692] Updated weights for policy 0, policy_version 808451 (0.0007) [2023-12-26 21:14:51,875][105620] Updated weights for policy 1, policy_version 808365 (0.0007) [2023-12-26 21:14:51,937][105620] Updated weights for policy 1, policy_version 808375 (0.0010) [2023-12-26 21:14:52,000][105620] Updated weights for policy 1, policy_version 808385 (0.0009) [2023-12-26 21:14:52,018][105692] Updated weights for policy 0, policy_version 808461 (0.0008) [2023-12-26 21:14:52,073][105692] Updated weights for policy 0, policy_version 808471 (0.0006) [2023-12-26 21:14:52,133][105692] Updated weights for policy 0, policy_version 808481 (0.0005) [2023-12-26 21:14:52,686][105620] Updated weights for policy 1, policy_version 808395 (0.0007) [2023-12-26 21:14:52,721][105692] Updated weights for policy 0, policy_version 808491 (0.0007) [2023-12-26 21:14:52,751][105620] Updated weights for policy 1, policy_version 808405 (0.0006) [2023-12-26 21:14:52,781][105692] Updated weights for policy 0, policy_version 808501 (0.0011) [2023-12-26 21:14:52,809][105620] Updated weights for policy 1, policy_version 808415 (0.0005) [2023-12-26 21:14:52,843][105692] Updated weights for policy 0, policy_version 808511 (0.0011) [2023-12-26 21:14:53,351][105620] Updated weights for policy 1, policy_version 808425 (0.0006) [2023-12-26 21:14:53,411][105620] Updated weights for policy 1, policy_version 808435 (0.0009) [2023-12-26 21:14:53,444][105692] Updated weights for policy 0, policy_version 808521 (0.0010) [2023-12-26 21:14:53,462][105620] Updated weights for policy 1, policy_version 808445 (0.0009) [2023-12-26 21:14:53,495][105692] Updated weights for policy 0, policy_version 808531 (0.0005) [2023-12-26 21:14:53,510][105620] Updated weights for policy 1, policy_version 808455 (0.0009) [2023-12-26 21:14:53,543][105692] Updated weights for policy 0, policy_version 808541 (0.0005) [2023-12-26 21:14:53,592][105692] Updated weights for policy 0, policy_version 808551 (0.0005) [2023-12-26 21:14:54,127][105692] Updated weights for policy 0, policy_version 808561 (0.0005) [2023-12-26 21:14:54,180][105692] Updated weights for policy 0, policy_version 808571 (0.0005) [2023-12-26 21:14:54,238][105692] Updated weights for policy 0, policy_version 808581 (0.0005) [2023-12-26 21:14:54,354][105620] Updated weights for policy 1, policy_version 808465 (0.0009) [2023-12-26 21:14:54,406][105620] Updated weights for policy 1, policy_version 808475 (0.0010) [2023-12-26 21:14:54,464][105620] Updated weights for policy 1, policy_version 808485 (0.0010) [2023-12-26 21:14:54,752][105692] Updated weights for policy 0, policy_version 808591 (0.0009) [2023-12-26 21:14:54,806][105692] Updated weights for policy 0, policy_version 808601 (0.0010) [2023-12-26 21:14:54,857][105692] Updated weights for policy 0, policy_version 808611 (0.0010) [2023-12-26 21:14:55,295][105620] Updated weights for policy 1, policy_version 808495 (0.0008) [2023-12-26 21:14:55,343][105620] Updated weights for policy 1, policy_version 808505 (0.0008) [2023-12-26 21:14:55,390][105620] Updated weights for policy 1, policy_version 808515 (0.0008) [2023-12-26 21:14:55,573][105692] Updated weights for policy 0, policy_version 808621 (0.0007) [2023-12-26 21:14:55,621][105692] Updated weights for policy 0, policy_version 808631 (0.0005) [2023-12-26 21:14:55,672][105692] Updated weights for policy 0, policy_version 808641 (0.0005) [2023-12-26 21:14:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 414048256. Throughput: 0: 9969.1, 1: 9616.3. Samples: 414058128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:14:56,062][104569] Avg episode reward: [(0, '5560.183'), (1, '8902.182')] [2023-12-26 21:14:56,191][105620] Updated weights for policy 1, policy_version 808525 (0.0009) [2023-12-26 21:14:56,256][105620] Updated weights for policy 1, policy_version 808535 (0.0008) [2023-12-26 21:14:56,307][105692] Updated weights for policy 0, policy_version 808651 (0.0006) [2023-12-26 21:14:56,314][105620] Updated weights for policy 1, policy_version 808545 (0.0007) [2023-12-26 21:14:56,357][105692] Updated weights for policy 0, policy_version 808661 (0.0007) [2023-12-26 21:14:56,406][105692] Updated weights for policy 0, policy_version 808671 (0.0009) [2023-12-26 21:14:57,084][105692] Updated weights for policy 0, policy_version 808681 (0.0008) [2023-12-26 21:14:57,108][105620] Updated weights for policy 1, policy_version 808555 (0.0007) [2023-12-26 21:14:57,135][105692] Updated weights for policy 0, policy_version 808691 (0.0005) [2023-12-26 21:14:57,160][105620] Updated weights for policy 1, policy_version 808565 (0.0009) [2023-12-26 21:14:57,193][105692] Updated weights for policy 0, policy_version 808701 (0.0006) [2023-12-26 21:14:57,217][105620] Updated weights for policy 1, policy_version 808575 (0.0009) [2023-12-26 21:14:57,249][105692] Updated weights for policy 0, policy_version 808711 (0.0010) [2023-12-26 21:14:57,894][105692] Updated weights for policy 0, policy_version 808721 (0.0008) [2023-12-26 21:14:57,927][105620] Updated weights for policy 1, policy_version 808585 (0.0008) [2023-12-26 21:14:57,945][105692] Updated weights for policy 0, policy_version 808731 (0.0008) [2023-12-26 21:14:57,987][105620] Updated weights for policy 1, policy_version 808595 (0.0008) [2023-12-26 21:14:57,999][105692] Updated weights for policy 0, policy_version 808741 (0.0005) [2023-12-26 21:14:58,045][105620] Updated weights for policy 1, policy_version 808605 (0.0009) [2023-12-26 21:14:58,097][105620] Updated weights for policy 1, policy_version 808615 (0.0009) [2023-12-26 21:14:58,752][105692] Updated weights for policy 0, policy_version 808751 (0.0007) [2023-12-26 21:14:58,815][105692] Updated weights for policy 0, policy_version 808761 (0.0008) [2023-12-26 21:14:58,877][105692] Updated weights for policy 0, policy_version 808771 (0.0008) [2023-12-26 21:14:58,917][105620] Updated weights for policy 1, policy_version 808625 (0.0008) [2023-12-26 21:14:58,979][105620] Updated weights for policy 1, policy_version 808635 (0.0008) [2023-12-26 21:14:59,038][105620] Updated weights for policy 1, policy_version 808645 (0.0009) [2023-12-26 21:14:59,610][105692] Updated weights for policy 0, policy_version 808781 (0.0007) [2023-12-26 21:14:59,663][105692] Updated weights for policy 0, policy_version 808791 (0.0005) [2023-12-26 21:14:59,721][105692] Updated weights for policy 0, policy_version 808801 (0.0009) [2023-12-26 21:14:59,892][105620] Updated weights for policy 1, policy_version 808655 (0.0009) [2023-12-26 21:14:59,949][105620] Updated weights for policy 1, policy_version 808665 (0.0009) [2023-12-26 21:15:00,008][105620] Updated weights for policy 1, policy_version 808675 (0.0010) [2023-12-26 21:15:00,398][105692] Updated weights for policy 0, policy_version 808811 (0.0008) [2023-12-26 21:15:00,445][105692] Updated weights for policy 0, policy_version 808821 (0.0005) [2023-12-26 21:15:00,496][105692] Updated weights for policy 0, policy_version 808831 (0.0005) [2023-12-26 21:15:00,726][105620] Updated weights for policy 1, policy_version 808685 (0.0007) [2023-12-26 21:15:00,774][105620] Updated weights for policy 1, policy_version 808695 (0.0006) [2023-12-26 21:15:00,819][105620] Updated weights for policy 1, policy_version 808705 (0.0005) [2023-12-26 21:15:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 414146560. Throughput: 0: 10015.5, 1: 9566.4. Samples: 414116544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:15:01,063][104569] Avg episode reward: [(0, '5635.874'), (1, '9074.172')] [2023-12-26 21:15:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000808840_207093760.pth... [2023-12-26 21:15:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000808712_207052800.pth... [2023-12-26 21:15:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000807592_206766080.pth [2023-12-26 21:15:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000807688_206798848.pth [2023-12-26 21:15:01,286][105692] Updated weights for policy 0, policy_version 808841 (0.0007) [2023-12-26 21:15:01,340][105692] Updated weights for policy 0, policy_version 808851 (0.0009) [2023-12-26 21:15:01,402][105692] Updated weights for policy 0, policy_version 808861 (0.0009) [2023-12-26 21:15:01,430][105620] Updated weights for policy 1, policy_version 808715 (0.0006) [2023-12-26 21:15:01,448][105692] Updated weights for policy 0, policy_version 808871 (0.0006) [2023-12-26 21:15:01,490][105620] Updated weights for policy 1, policy_version 808725 (0.0008) [2023-12-26 21:15:01,556][105620] Updated weights for policy 1, policy_version 808735 (0.0008) [2023-12-26 21:15:02,206][105620] Updated weights for policy 1, policy_version 808745 (0.0006) [2023-12-26 21:15:02,255][105692] Updated weights for policy 0, policy_version 808881 (0.0010) [2023-12-26 21:15:02,274][105620] Updated weights for policy 1, policy_version 808755 (0.0007) [2023-12-26 21:15:02,318][105692] Updated weights for policy 0, policy_version 808891 (0.0011) [2023-12-26 21:15:02,336][105620] Updated weights for policy 1, policy_version 808765 (0.0008) [2023-12-26 21:15:02,384][105692] Updated weights for policy 0, policy_version 808901 (0.0008) [2023-12-26 21:15:02,406][105620] Updated weights for policy 1, policy_version 808775 (0.0011) [2023-12-26 21:15:02,999][105692] Updated weights for policy 0, policy_version 808911 (0.0006) [2023-12-26 21:15:03,057][105692] Updated weights for policy 0, policy_version 808921 (0.0007) [2023-12-26 21:15:03,112][105692] Updated weights for policy 0, policy_version 808931 (0.0006) [2023-12-26 21:15:03,114][105620] Updated weights for policy 1, policy_version 808785 (0.0008) [2023-12-26 21:15:03,170][105620] Updated weights for policy 1, policy_version 808795 (0.0009) [2023-12-26 21:15:03,233][105620] Updated weights for policy 1, policy_version 808805 (0.0008) [2023-12-26 21:15:03,840][105692] Updated weights for policy 0, policy_version 808941 (0.0006) [2023-12-26 21:15:03,906][105692] Updated weights for policy 0, policy_version 808951 (0.0007) [2023-12-26 21:15:03,912][105620] Updated weights for policy 1, policy_version 808815 (0.0007) [2023-12-26 21:15:03,966][105692] Updated weights for policy 0, policy_version 808961 (0.0006) [2023-12-26 21:15:03,976][105620] Updated weights for policy 1, policy_version 808825 (0.0007) [2023-12-26 21:15:04,028][105620] Updated weights for policy 1, policy_version 808835 (0.0008) [2023-12-26 21:15:04,715][105692] Updated weights for policy 0, policy_version 808971 (0.0009) [2023-12-26 21:15:04,734][105620] Updated weights for policy 1, policy_version 808845 (0.0008) [2023-12-26 21:15:04,772][105692] Updated weights for policy 0, policy_version 808981 (0.0008) [2023-12-26 21:15:04,795][105620] Updated weights for policy 1, policy_version 808855 (0.0006) [2023-12-26 21:15:04,829][105692] Updated weights for policy 0, policy_version 808991 (0.0007) [2023-12-26 21:15:04,858][105620] Updated weights for policy 1, policy_version 808865 (0.0007) [2023-12-26 21:15:05,445][105620] Updated weights for policy 1, policy_version 808875 (0.0007) [2023-12-26 21:15:05,505][105620] Updated weights for policy 1, policy_version 808885 (0.0005) [2023-12-26 21:15:05,554][105620] Updated weights for policy 1, policy_version 808895 (0.0005) [2023-12-26 21:15:05,622][105692] Updated weights for policy 0, policy_version 809001 (0.0007) [2023-12-26 21:15:05,669][105692] Updated weights for policy 0, policy_version 809011 (0.0006) [2023-12-26 21:15:05,714][105692] Updated weights for policy 0, policy_version 809021 (0.0005) [2023-12-26 21:15:05,762][105692] Updated weights for policy 0, policy_version 809031 (0.0005) [2023-12-26 21:15:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 414244864. Throughput: 0: 9929.8, 1: 9584.2. Samples: 414232576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:15:06,063][104569] Avg episode reward: [(0, '5920.904'), (1, '9353.608')] [2023-12-26 21:15:06,296][105620] Updated weights for policy 1, policy_version 808905 (0.0007) [2023-12-26 21:15:06,355][105620] Updated weights for policy 1, policy_version 808915 (0.0009) [2023-12-26 21:15:06,416][105620] Updated weights for policy 1, policy_version 808925 (0.0009) [2023-12-26 21:15:06,475][105620] Updated weights for policy 1, policy_version 808935 (0.0007) [2023-12-26 21:15:06,477][105692] Updated weights for policy 0, policy_version 809041 (0.0006) [2023-12-26 21:15:06,538][105692] Updated weights for policy 0, policy_version 809051 (0.0009) [2023-12-26 21:15:06,601][105692] Updated weights for policy 0, policy_version 809061 (0.0009) [2023-12-26 21:15:07,232][105620] Updated weights for policy 1, policy_version 808945 (0.0007) [2023-12-26 21:15:07,284][105620] Updated weights for policy 1, policy_version 808955 (0.0006) [2023-12-26 21:15:07,341][105620] Updated weights for policy 1, policy_version 808965 (0.0008) [2023-12-26 21:15:07,366][105692] Updated weights for policy 0, policy_version 809071 (0.0009) [2023-12-26 21:15:07,430][105692] Updated weights for policy 0, policy_version 809081 (0.0008) [2023-12-26 21:15:07,497][105692] Updated weights for policy 0, policy_version 809091 (0.0005) [2023-12-26 21:15:08,003][105620] Updated weights for policy 1, policy_version 808975 (0.0010) [2023-12-26 21:15:08,073][105620] Updated weights for policy 1, policy_version 808985 (0.0011) [2023-12-26 21:15:08,140][105620] Updated weights for policy 1, policy_version 808995 (0.0007) [2023-12-26 21:15:08,186][105692] Updated weights for policy 0, policy_version 809101 (0.0006) [2023-12-26 21:15:08,235][105692] Updated weights for policy 0, policy_version 809111 (0.0008) [2023-12-26 21:15:08,297][105692] Updated weights for policy 0, policy_version 809121 (0.0008) [2023-12-26 21:15:08,738][105620] Updated weights for policy 1, policy_version 809005 (0.0007) [2023-12-26 21:15:08,806][105620] Updated weights for policy 1, policy_version 809015 (0.0011) [2023-12-26 21:15:08,871][105620] Updated weights for policy 1, policy_version 809025 (0.0011) [2023-12-26 21:15:09,082][105692] Updated weights for policy 0, policy_version 809131 (0.0009) [2023-12-26 21:15:09,132][105692] Updated weights for policy 0, policy_version 809141 (0.0008) [2023-12-26 21:15:09,183][105692] Updated weights for policy 0, policy_version 809151 (0.0008) [2023-12-26 21:15:09,559][105620] Updated weights for policy 1, policy_version 809035 (0.0009) [2023-12-26 21:15:09,612][105620] Updated weights for policy 1, policy_version 809045 (0.0006) [2023-12-26 21:15:09,663][105620] Updated weights for policy 1, policy_version 809055 (0.0006) [2023-12-26 21:15:10,059][105692] Updated weights for policy 0, policy_version 809161 (0.0008) [2023-12-26 21:15:10,120][105692] Updated weights for policy 0, policy_version 809171 (0.0010) [2023-12-26 21:15:10,182][105692] Updated weights for policy 0, policy_version 809181 (0.0006) [2023-12-26 21:15:10,246][105692] Updated weights for policy 0, policy_version 809191 (0.0006) [2023-12-26 21:15:10,317][105620] Updated weights for policy 1, policy_version 809065 (0.0006) [2023-12-26 21:15:10,383][105620] Updated weights for policy 1, policy_version 809075 (0.0008) [2023-12-26 21:15:10,450][105620] Updated weights for policy 1, policy_version 809085 (0.0007) [2023-12-26 21:15:10,521][105620] Updated weights for policy 1, policy_version 809095 (0.0006) [2023-12-26 21:15:11,010][105692] Updated weights for policy 0, policy_version 809201 (0.0009) [2023-12-26 21:15:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 414334976. Throughput: 0: 9872.6, 1: 9718.4. Samples: 414349624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:15:11,063][104569] Avg episode reward: [(0, '6523.742'), (1, '8209.623')] [2023-12-26 21:15:11,072][105692] Updated weights for policy 0, policy_version 809211 (0.0009) [2023-12-26 21:15:11,137][105692] Updated weights for policy 0, policy_version 809221 (0.0009) [2023-12-26 21:15:11,210][105620] Updated weights for policy 1, policy_version 809105 (0.0009) [2023-12-26 21:15:11,279][105620] Updated weights for policy 1, policy_version 809115 (0.0009) [2023-12-26 21:15:11,344][105620] Updated weights for policy 1, policy_version 809125 (0.0009) [2023-12-26 21:15:11,918][105692] Updated weights for policy 0, policy_version 809231 (0.0008) [2023-12-26 21:15:11,981][105692] Updated weights for policy 0, policy_version 809241 (0.0007) [2023-12-26 21:15:12,045][105692] Updated weights for policy 0, policy_version 809251 (0.0006) [2023-12-26 21:15:12,063][105620] Updated weights for policy 1, policy_version 809135 (0.0009) [2023-12-26 21:15:12,124][105620] Updated weights for policy 1, policy_version 809145 (0.0010) [2023-12-26 21:15:12,178][105620] Updated weights for policy 1, policy_version 809155 (0.0007) [2023-12-26 21:15:12,820][105692] Updated weights for policy 0, policy_version 809261 (0.0006) [2023-12-26 21:15:12,829][105620] Updated weights for policy 1, policy_version 809165 (0.0007) [2023-12-26 21:15:12,871][105692] Updated weights for policy 0, policy_version 809271 (0.0007) [2023-12-26 21:15:12,891][105620] Updated weights for policy 1, policy_version 809175 (0.0008) [2023-12-26 21:15:12,918][105692] Updated weights for policy 0, policy_version 809281 (0.0006) [2023-12-26 21:15:12,958][105620] Updated weights for policy 1, policy_version 809185 (0.0007) [2023-12-26 21:15:13,640][105620] Updated weights for policy 1, policy_version 809195 (0.0008) [2023-12-26 21:15:13,692][105620] Updated weights for policy 1, policy_version 809205 (0.0008) [2023-12-26 21:15:13,707][105692] Updated weights for policy 0, policy_version 809291 (0.0008) [2023-12-26 21:15:13,752][105620] Updated weights for policy 1, policy_version 809215 (0.0006) [2023-12-26 21:15:13,758][105692] Updated weights for policy 0, policy_version 809301 (0.0010) [2023-12-26 21:15:13,813][105692] Updated weights for policy 0, policy_version 809311 (0.0010) [2023-12-26 21:15:14,430][105620] Updated weights for policy 1, policy_version 809225 (0.0005) [2023-12-26 21:15:14,484][105620] Updated weights for policy 1, policy_version 809235 (0.0005) [2023-12-26 21:15:14,529][105620] Updated weights for policy 1, policy_version 809245 (0.0005) [2023-12-26 21:15:14,569][105692] Updated weights for policy 0, policy_version 809321 (0.0010) [2023-12-26 21:15:14,581][105620] Updated weights for policy 1, policy_version 809255 (0.0010) [2023-12-26 21:15:14,621][105692] Updated weights for policy 0, policy_version 809331 (0.0008) [2023-12-26 21:15:14,663][105692] Updated weights for policy 0, policy_version 809341 (0.0006) [2023-12-26 21:15:14,711][105692] Updated weights for policy 0, policy_version 809351 (0.0005) [2023-12-26 21:15:15,262][105620] Updated weights for policy 1, policy_version 809265 (0.0011) [2023-12-26 21:15:15,318][105620] Updated weights for policy 1, policy_version 809275 (0.0011) [2023-12-26 21:15:15,372][105620] Updated weights for policy 1, policy_version 809285 (0.0011) [2023-12-26 21:15:15,400][105692] Updated weights for policy 0, policy_version 809361 (0.0010) [2023-12-26 21:15:15,460][105692] Updated weights for policy 0, policy_version 809371 (0.0011) [2023-12-26 21:15:15,516][105692] Updated weights for policy 0, policy_version 809381 (0.0010) [2023-12-26 21:15:16,027][105620] Updated weights for policy 1, policy_version 809295 (0.0007) [2023-12-26 21:15:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 414433280. Throughput: 0: 9780.5, 1: 9684.2. Samples: 414405788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:15:16,063][104569] Avg episode reward: [(0, '7052.152'), (1, '7878.476')] [2023-12-26 21:15:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000809384_207233024.pth... [2023-12-26 21:15:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000808232_206938112.pth [2023-12-26 21:15:16,077][105620] Updated weights for policy 1, policy_version 809305 (0.0007) [2023-12-26 21:15:16,125][105620] Updated weights for policy 1, policy_version 809315 (0.0010) [2023-12-26 21:15:16,148][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000809320_207208448.pth... [2023-12-26 21:15:16,151][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000808168_206913536.pth [2023-12-26 21:15:16,249][105692] Updated weights for policy 0, policy_version 809391 (0.0009) [2023-12-26 21:15:16,297][105692] Updated weights for policy 0, policy_version 809401 (0.0008) [2023-12-26 21:15:16,342][105692] Updated weights for policy 0, policy_version 809411 (0.0008) [2023-12-26 21:15:16,866][105620] Updated weights for policy 1, policy_version 809325 (0.0010) [2023-12-26 21:15:16,925][105620] Updated weights for policy 1, policy_version 809335 (0.0010) [2023-12-26 21:15:16,969][105620] Updated weights for policy 1, policy_version 809345 (0.0010) [2023-12-26 21:15:17,062][105692] Updated weights for policy 0, policy_version 809421 (0.0008) [2023-12-26 21:15:17,120][105692] Updated weights for policy 0, policy_version 809431 (0.0006) [2023-12-26 21:15:17,177][105692] Updated weights for policy 0, policy_version 809441 (0.0005) [2023-12-26 21:15:17,700][105620] Updated weights for policy 1, policy_version 809355 (0.0010) [2023-12-26 21:15:17,752][105620] Updated weights for policy 1, policy_version 809365 (0.0010) [2023-12-26 21:15:17,811][105620] Updated weights for policy 1, policy_version 809375 (0.0011) [2023-12-26 21:15:17,830][105692] Updated weights for policy 0, policy_version 809451 (0.0006) [2023-12-26 21:15:17,889][105692] Updated weights for policy 0, policy_version 809461 (0.0010) [2023-12-26 21:15:17,954][105692] Updated weights for policy 0, policy_version 809471 (0.0010) [2023-12-26 21:15:18,479][105620] Updated weights for policy 1, policy_version 809385 (0.0006) [2023-12-26 21:15:18,527][105620] Updated weights for policy 1, policy_version 809395 (0.0010) [2023-12-26 21:15:18,582][105620] Updated weights for policy 1, policy_version 809405 (0.0010) [2023-12-26 21:15:18,630][105692] Updated weights for policy 0, policy_version 809481 (0.0010) [2023-12-26 21:15:18,648][105620] Updated weights for policy 1, policy_version 809415 (0.0010) [2023-12-26 21:15:18,695][105692] Updated weights for policy 0, policy_version 809491 (0.0006) [2023-12-26 21:15:18,760][105692] Updated weights for policy 0, policy_version 809501 (0.0006) [2023-12-26 21:15:18,820][105692] Updated weights for policy 0, policy_version 809511 (0.0008) [2023-12-26 21:15:19,357][105620] Updated weights for policy 1, policy_version 809425 (0.0009) [2023-12-26 21:15:19,425][105620] Updated weights for policy 1, policy_version 809435 (0.0008) [2023-12-26 21:15:19,488][105620] Updated weights for policy 1, policy_version 809445 (0.0008) [2023-12-26 21:15:19,548][105692] Updated weights for policy 0, policy_version 809521 (0.0008) [2023-12-26 21:15:19,612][105692] Updated weights for policy 0, policy_version 809531 (0.0009) [2023-12-26 21:15:19,677][105692] Updated weights for policy 0, policy_version 809541 (0.0007) [2023-12-26 21:15:20,157][105620] Updated weights for policy 1, policy_version 809455 (0.0009) [2023-12-26 21:15:20,213][105620] Updated weights for policy 1, policy_version 809465 (0.0008) [2023-12-26 21:15:20,270][105620] Updated weights for policy 1, policy_version 809475 (0.0009) [2023-12-26 21:15:20,383][105692] Updated weights for policy 0, policy_version 809551 (0.0008) [2023-12-26 21:15:20,453][105692] Updated weights for policy 0, policy_version 809561 (0.0008) [2023-12-26 21:15:20,512][105692] Updated weights for policy 0, policy_version 809571 (0.0009) [2023-12-26 21:15:20,971][105620] Updated weights for policy 1, policy_version 809485 (0.0008) [2023-12-26 21:15:21,038][105620] Updated weights for policy 1, policy_version 809495 (0.0008) [2023-12-26 21:15:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 414531584. Throughput: 0: 9784.6, 1: 9750.7. Samples: 414525384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:15:21,063][104569] Avg episode reward: [(0, '7541.840'), (1, '8783.756')] [2023-12-26 21:15:21,108][105620] Updated weights for policy 1, policy_version 809505 (0.0007) [2023-12-26 21:15:21,211][105692] Updated weights for policy 0, policy_version 809581 (0.0008) [2023-12-26 21:15:21,273][105692] Updated weights for policy 0, policy_version 809591 (0.0007) [2023-12-26 21:15:21,326][105692] Updated weights for policy 0, policy_version 809601 (0.0008) [2023-12-26 21:15:21,868][105620] Updated weights for policy 1, policy_version 809515 (0.0010) [2023-12-26 21:15:21,932][105620] Updated weights for policy 1, policy_version 809525 (0.0010) [2023-12-26 21:15:21,995][105620] Updated weights for policy 1, policy_version 809535 (0.0010) [2023-12-26 21:15:22,076][105692] Updated weights for policy 0, policy_version 809611 (0.0008) [2023-12-26 21:15:22,140][105692] Updated weights for policy 0, policy_version 809621 (0.0008) [2023-12-26 21:15:22,193][105692] Updated weights for policy 0, policy_version 809631 (0.0008) [2023-12-26 21:15:22,754][105620] Updated weights for policy 1, policy_version 809545 (0.0011) [2023-12-26 21:15:22,803][105620] Updated weights for policy 1, policy_version 809555 (0.0011) [2023-12-26 21:15:22,860][105620] Updated weights for policy 1, policy_version 809565 (0.0011) [2023-12-26 21:15:22,922][105620] Updated weights for policy 1, policy_version 809575 (0.0010) [2023-12-26 21:15:22,977][105692] Updated weights for policy 0, policy_version 809641 (0.0008) [2023-12-26 21:15:23,030][105692] Updated weights for policy 0, policy_version 809651 (0.0008) [2023-12-26 21:15:23,083][105692] Updated weights for policy 0, policy_version 809661 (0.0008) [2023-12-26 21:15:23,154][105692] Updated weights for policy 0, policy_version 809671 (0.0008) [2023-12-26 21:15:23,621][105620] Updated weights for policy 1, policy_version 809585 (0.0010) [2023-12-26 21:15:23,669][105620] Updated weights for policy 1, policy_version 809595 (0.0010) [2023-12-26 21:15:23,717][105620] Updated weights for policy 1, policy_version 809605 (0.0010) [2023-12-26 21:15:23,929][105692] Updated weights for policy 0, policy_version 809681 (0.0008) [2023-12-26 21:15:23,974][105692] Updated weights for policy 0, policy_version 809691 (0.0008) [2023-12-26 21:15:24,023][105692] Updated weights for policy 0, policy_version 809701 (0.0005) [2023-12-26 21:15:24,497][105620] Updated weights for policy 1, policy_version 809615 (0.0009) [2023-12-26 21:15:24,549][105620] Updated weights for policy 1, policy_version 809625 (0.0010) [2023-12-26 21:15:24,597][105620] Updated weights for policy 1, policy_version 809635 (0.0010) [2023-12-26 21:15:24,751][105692] Updated weights for policy 0, policy_version 809711 (0.0009) [2023-12-26 21:15:24,816][105692] Updated weights for policy 0, policy_version 809721 (0.0010) [2023-12-26 21:15:24,882][105692] Updated weights for policy 0, policy_version 809731 (0.0011) [2023-12-26 21:15:25,303][105620] Updated weights for policy 1, policy_version 809645 (0.0010) [2023-12-26 21:15:25,368][105620] Updated weights for policy 1, policy_version 809655 (0.0010) [2023-12-26 21:15:25,430][105620] Updated weights for policy 1, policy_version 809665 (0.0011) [2023-12-26 21:15:25,608][105692] Updated weights for policy 0, policy_version 809741 (0.0010) [2023-12-26 21:15:25,659][105692] Updated weights for policy 0, policy_version 809751 (0.0010) [2023-12-26 21:15:25,713][105692] Updated weights for policy 0, policy_version 809761 (0.0008) [2023-12-26 21:15:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 414629888. Throughput: 0: 9800.6, 1: 9688.9. Samples: 414639372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:15:26,063][104569] Avg episode reward: [(0, '7804.670'), (1, '9278.952')] [2023-12-26 21:15:26,163][105620] Updated weights for policy 1, policy_version 809675 (0.0010) [2023-12-26 21:15:26,212][105620] Updated weights for policy 1, policy_version 809685 (0.0010) [2023-12-26 21:15:26,260][105620] Updated weights for policy 1, policy_version 809695 (0.0010) [2023-12-26 21:15:26,347][105692] Updated weights for policy 0, policy_version 809771 (0.0007) [2023-12-26 21:15:26,405][105692] Updated weights for policy 0, policy_version 809781 (0.0007) [2023-12-26 21:15:26,469][105692] Updated weights for policy 0, policy_version 809791 (0.0009) [2023-12-26 21:15:27,028][105620] Updated weights for policy 1, policy_version 809705 (0.0010) [2023-12-26 21:15:27,082][105620] Updated weights for policy 1, policy_version 809715 (0.0010) [2023-12-26 21:15:27,142][105620] Updated weights for policy 1, policy_version 809725 (0.0010) [2023-12-26 21:15:27,190][105692] Updated weights for policy 0, policy_version 809801 (0.0011) [2023-12-26 21:15:27,206][105620] Updated weights for policy 1, policy_version 809735 (0.0010) [2023-12-26 21:15:27,240][105692] Updated weights for policy 0, policy_version 809811 (0.0010) [2023-12-26 21:15:27,288][105692] Updated weights for policy 0, policy_version 809821 (0.0010) [2023-12-26 21:15:27,341][105692] Updated weights for policy 0, policy_version 809831 (0.0008) [2023-12-26 21:15:27,906][105692] Updated weights for policy 0, policy_version 809841 (0.0008) [2023-12-26 21:15:27,948][105620] Updated weights for policy 1, policy_version 809745 (0.0010) [2023-12-26 21:15:27,962][105692] Updated weights for policy 0, policy_version 809851 (0.0007) [2023-12-26 21:15:28,009][105620] Updated weights for policy 1, policy_version 809755 (0.0011) [2023-12-26 21:15:28,022][105692] Updated weights for policy 0, policy_version 809861 (0.0011) [2023-12-26 21:15:28,069][105620] Updated weights for policy 1, policy_version 809765 (0.0010) [2023-12-26 21:15:28,763][105692] Updated weights for policy 0, policy_version 809871 (0.0010) [2023-12-26 21:15:28,815][105620] Updated weights for policy 1, policy_version 809775 (0.0010) [2023-12-26 21:15:28,818][105692] Updated weights for policy 0, policy_version 809881 (0.0010) [2023-12-26 21:15:28,874][105620] Updated weights for policy 1, policy_version 809785 (0.0010) [2023-12-26 21:15:28,879][105692] Updated weights for policy 0, policy_version 809891 (0.0010) [2023-12-26 21:15:28,933][105620] Updated weights for policy 1, policy_version 809795 (0.0010) [2023-12-26 21:15:29,585][105692] Updated weights for policy 0, policy_version 809901 (0.0008) [2023-12-26 21:15:29,634][105692] Updated weights for policy 0, policy_version 809911 (0.0005) [2023-12-26 21:15:29,695][105692] Updated weights for policy 0, policy_version 809921 (0.0005) [2023-12-26 21:15:29,740][105620] Updated weights for policy 1, policy_version 809805 (0.0010) [2023-12-26 21:15:29,798][105620] Updated weights for policy 1, policy_version 809815 (0.0010) [2023-12-26 21:15:29,860][105620] Updated weights for policy 1, policy_version 809825 (0.0011) [2023-12-26 21:15:30,270][105692] Updated weights for policy 0, policy_version 809931 (0.0006) [2023-12-26 21:15:30,325][105692] Updated weights for policy 0, policy_version 809941 (0.0011) [2023-12-26 21:15:30,381][105692] Updated weights for policy 0, policy_version 809951 (0.0006) [2023-12-26 21:15:30,600][105620] Updated weights for policy 1, policy_version 809835 (0.0010) [2023-12-26 21:15:30,645][105620] Updated weights for policy 1, policy_version 809845 (0.0010) [2023-12-26 21:15:30,696][105620] Updated weights for policy 1, policy_version 809855 (0.0010) [2023-12-26 21:15:30,909][105692] Updated weights for policy 0, policy_version 809961 (0.0005) [2023-12-26 21:15:30,972][105692] Updated weights for policy 0, policy_version 809971 (0.0006) [2023-12-26 21:15:31,029][105692] Updated weights for policy 0, policy_version 809981 (0.0006) [2023-12-26 21:15:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 414728192. Throughput: 0: 9840.5, 1: 9661.2. Samples: 414698776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:15:31,062][104569] Avg episode reward: [(0, '8729.212'), (1, '9353.502')] [2023-12-26 21:15:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000809864_207347712.pth... [2023-12-26 21:15:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000808712_207052800.pth [2023-12-26 21:15:31,094][105692] Updated weights for policy 0, policy_version 809991 (0.0007) [2023-12-26 21:15:31,099][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000809992_207388672.pth... [2023-12-26 21:15:31,105][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000808840_207093760.pth [2023-12-26 21:15:31,395][105620] Updated weights for policy 1, policy_version 809865 (0.0010) [2023-12-26 21:15:31,455][105620] Updated weights for policy 1, policy_version 809875 (0.0011) [2023-12-26 21:15:31,506][105620] Updated weights for policy 1, policy_version 809885 (0.0010) [2023-12-26 21:15:31,561][105620] Updated weights for policy 1, policy_version 809895 (0.0011) [2023-12-26 21:15:31,767][105692] Updated weights for policy 0, policy_version 810001 (0.0010) [2023-12-26 21:15:31,821][105692] Updated weights for policy 0, policy_version 810011 (0.0010) [2023-12-26 21:15:31,878][105692] Updated weights for policy 0, policy_version 810021 (0.0010) [2023-12-26 21:15:32,336][105620] Updated weights for policy 1, policy_version 809905 (0.0011) [2023-12-26 21:15:32,403][105620] Updated weights for policy 1, policy_version 809915 (0.0011) [2023-12-26 21:15:32,461][105620] Updated weights for policy 1, policy_version 809925 (0.0010) [2023-12-26 21:15:32,653][105692] Updated weights for policy 0, policy_version 810031 (0.0009) [2023-12-26 21:15:32,709][105692] Updated weights for policy 0, policy_version 810041 (0.0005) [2023-12-26 21:15:32,763][105692] Updated weights for policy 0, policy_version 810051 (0.0006) [2023-12-26 21:15:33,189][105620] Updated weights for policy 1, policy_version 809935 (0.0010) [2023-12-26 21:15:33,240][105620] Updated weights for policy 1, policy_version 809945 (0.0010) [2023-12-26 21:15:33,288][105620] Updated weights for policy 1, policy_version 809955 (0.0010) [2023-12-26 21:15:33,453][105692] Updated weights for policy 0, policy_version 810061 (0.0008) [2023-12-26 21:15:33,509][105692] Updated weights for policy 0, policy_version 810071 (0.0007) [2023-12-26 21:15:33,564][105692] Updated weights for policy 0, policy_version 810081 (0.0006) [2023-12-26 21:15:33,886][105620] Updated weights for policy 1, policy_version 809965 (0.0008) [2023-12-26 21:15:33,937][105620] Updated weights for policy 1, policy_version 809975 (0.0005) [2023-12-26 21:15:33,996][105620] Updated weights for policy 1, policy_version 809985 (0.0010) [2023-12-26 21:15:34,111][105692] Updated weights for policy 0, policy_version 810091 (0.0008) [2023-12-26 21:15:34,170][105692] Updated weights for policy 0, policy_version 810101 (0.0011) [2023-12-26 21:15:34,231][105692] Updated weights for policy 0, policy_version 810111 (0.0010) [2023-12-26 21:15:34,701][105620] Updated weights for policy 1, policy_version 809995 (0.0010) [2023-12-26 21:15:34,761][105620] Updated weights for policy 1, policy_version 810005 (0.0011) [2023-12-26 21:15:34,824][105620] Updated weights for policy 1, policy_version 810015 (0.0010) [2023-12-26 21:15:34,974][105692] Updated weights for policy 0, policy_version 810121 (0.0010) [2023-12-26 21:15:35,045][105692] Updated weights for policy 0, policy_version 810131 (0.0007) [2023-12-26 21:15:35,108][105692] Updated weights for policy 0, policy_version 810141 (0.0011) [2023-12-26 21:15:35,172][105692] Updated weights for policy 0, policy_version 810151 (0.0011) [2023-12-26 21:15:35,393][105620] Updated weights for policy 1, policy_version 810025 (0.0010) [2023-12-26 21:15:35,450][105620] Updated weights for policy 1, policy_version 810035 (0.0009) [2023-12-26 21:15:35,509][105620] Updated weights for policy 1, policy_version 810045 (0.0011) [2023-12-26 21:15:35,562][105620] Updated weights for policy 1, policy_version 810055 (0.0011) [2023-12-26 21:15:35,912][105692] Updated weights for policy 0, policy_version 810161 (0.0008) [2023-12-26 21:15:35,978][105692] Updated weights for policy 0, policy_version 810171 (0.0011) [2023-12-26 21:15:36,033][105692] Updated weights for policy 0, policy_version 810181 (0.0011) [2023-12-26 21:15:36,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19660.9, 300 sec: 19466.4). Total num frames: 414834688. Throughput: 0: 9927.0, 1: 9718.7. Samples: 414820544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:15:36,062][104569] Avg episode reward: [(0, '8375.327'), (1, '9171.110')] [2023-12-26 21:15:36,256][105620] Updated weights for policy 1, policy_version 810065 (0.0011) [2023-12-26 21:15:36,316][105620] Updated weights for policy 1, policy_version 810075 (0.0011) [2023-12-26 21:15:36,382][105620] Updated weights for policy 1, policy_version 810085 (0.0011) [2023-12-26 21:15:36,731][105692] Updated weights for policy 0, policy_version 810191 (0.0007) [2023-12-26 21:15:36,795][105692] Updated weights for policy 0, policy_version 810201 (0.0006) [2023-12-26 21:15:36,858][105692] Updated weights for policy 0, policy_version 810211 (0.0007) [2023-12-26 21:15:37,134][105620] Updated weights for policy 1, policy_version 810095 (0.0011) [2023-12-26 21:15:37,186][105620] Updated weights for policy 1, policy_version 810105 (0.0010) [2023-12-26 21:15:37,245][105620] Updated weights for policy 1, policy_version 810115 (0.0010) [2023-12-26 21:15:37,499][105692] Updated weights for policy 0, policy_version 810221 (0.0008) [2023-12-26 21:15:37,562][105692] Updated weights for policy 0, policy_version 810231 (0.0010) [2023-12-26 21:15:37,621][105692] Updated weights for policy 0, policy_version 810241 (0.0011) [2023-12-26 21:15:37,896][105620] Updated weights for policy 1, policy_version 810125 (0.0008) [2023-12-26 21:15:37,958][105620] Updated weights for policy 1, policy_version 810135 (0.0005) [2023-12-26 21:15:38,020][105620] Updated weights for policy 1, policy_version 810145 (0.0008) [2023-12-26 21:15:38,247][105692] Updated weights for policy 0, policy_version 810251 (0.0009) [2023-12-26 21:15:38,319][105692] Updated weights for policy 0, policy_version 810261 (0.0006) [2023-12-26 21:15:38,387][105692] Updated weights for policy 0, policy_version 810271 (0.0008) [2023-12-26 21:15:38,686][105620] Updated weights for policy 1, policy_version 810155 (0.0010) [2023-12-26 21:15:38,745][105620] Updated weights for policy 1, policy_version 810165 (0.0009) [2023-12-26 21:15:38,811][105620] Updated weights for policy 1, policy_version 810175 (0.0009) [2023-12-26 21:15:39,061][105692] Updated weights for policy 0, policy_version 810281 (0.0008) [2023-12-26 21:15:39,122][105692] Updated weights for policy 0, policy_version 810291 (0.0009) [2023-12-26 21:15:39,177][105692] Updated weights for policy 0, policy_version 810301 (0.0008) [2023-12-26 21:15:39,231][105692] Updated weights for policy 0, policy_version 810311 (0.0009) [2023-12-26 21:15:39,614][105620] Updated weights for policy 1, policy_version 810185 (0.0008) [2023-12-26 21:15:39,676][105620] Updated weights for policy 1, policy_version 810195 (0.0008) [2023-12-26 21:15:39,729][105620] Updated weights for policy 1, policy_version 810205 (0.0009) [2023-12-26 21:15:39,777][105620] Updated weights for policy 1, policy_version 810215 (0.0009) [2023-12-26 21:15:40,029][105692] Updated weights for policy 0, policy_version 810321 (0.0008) [2023-12-26 21:15:40,099][105692] Updated weights for policy 0, policy_version 810331 (0.0005) [2023-12-26 21:15:40,166][105692] Updated weights for policy 0, policy_version 810341 (0.0006) [2023-12-26 21:15:40,473][105620] Updated weights for policy 1, policy_version 810225 (0.0006) [2023-12-26 21:15:40,533][105620] Updated weights for policy 1, policy_version 810235 (0.0008) [2023-12-26 21:15:40,589][105620] Updated weights for policy 1, policy_version 810245 (0.0008) [2023-12-26 21:15:40,870][105692] Updated weights for policy 0, policy_version 810351 (0.0009) [2023-12-26 21:15:40,925][105692] Updated weights for policy 0, policy_version 810361 (0.0010) [2023-12-26 21:15:40,983][105692] Updated weights for policy 0, policy_version 810371 (0.0010) [2023-12-26 21:15:41,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 414932992. Throughput: 0: 9774.8, 1: 9808.9. Samples: 414939396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:15:41,063][104569] Avg episode reward: [(0, '7733.013'), (1, '8988.461')] [2023-12-26 21:15:41,326][105620] Updated weights for policy 1, policy_version 810255 (0.0009) [2023-12-26 21:15:41,403][105620] Updated weights for policy 1, policy_version 810265 (0.0007) [2023-12-26 21:15:41,471][105620] Updated weights for policy 1, policy_version 810275 (0.0008) [2023-12-26 21:15:41,770][105692] Updated weights for policy 0, policy_version 810381 (0.0010) [2023-12-26 21:15:41,831][105692] Updated weights for policy 0, policy_version 810391 (0.0007) [2023-12-26 21:15:41,890][105692] Updated weights for policy 0, policy_version 810401 (0.0010) [2023-12-26 21:15:42,273][105620] Updated weights for policy 1, policy_version 810285 (0.0009) [2023-12-26 21:15:42,336][105620] Updated weights for policy 1, policy_version 810295 (0.0008) [2023-12-26 21:15:42,400][105620] Updated weights for policy 1, policy_version 810305 (0.0008) [2023-12-26 21:15:42,599][105692] Updated weights for policy 0, policy_version 810411 (0.0009) [2023-12-26 21:15:42,670][105692] Updated weights for policy 0, policy_version 810421 (0.0006) [2023-12-26 21:15:42,730][105692] Updated weights for policy 0, policy_version 810431 (0.0005) [2023-12-26 21:15:43,145][105620] Updated weights for policy 1, policy_version 810315 (0.0008) [2023-12-26 21:15:43,209][105620] Updated weights for policy 1, policy_version 810325 (0.0011) [2023-12-26 21:15:43,275][105620] Updated weights for policy 1, policy_version 810335 (0.0007) [2023-12-26 21:15:43,317][105692] Updated weights for policy 0, policy_version 810441 (0.0006) [2023-12-26 21:15:43,376][105692] Updated weights for policy 0, policy_version 810451 (0.0010) [2023-12-26 21:15:43,433][105692] Updated weights for policy 0, policy_version 810461 (0.0011) [2023-12-26 21:15:43,486][105692] Updated weights for policy 0, policy_version 810471 (0.0011) [2023-12-26 21:15:43,955][105620] Updated weights for policy 1, policy_version 810345 (0.0010) [2023-12-26 21:15:44,014][105620] Updated weights for policy 1, policy_version 810355 (0.0007) [2023-12-26 21:15:44,084][105620] Updated weights for policy 1, policy_version 810365 (0.0006) [2023-12-26 21:15:44,149][105620] Updated weights for policy 1, policy_version 810375 (0.0006) [2023-12-26 21:15:44,167][105692] Updated weights for policy 0, policy_version 810481 (0.0009) [2023-12-26 21:15:44,226][105692] Updated weights for policy 0, policy_version 810491 (0.0010) [2023-12-26 21:15:44,281][105692] Updated weights for policy 0, policy_version 810501 (0.0010) [2023-12-26 21:15:44,702][105620] Updated weights for policy 1, policy_version 810385 (0.0006) [2023-12-26 21:15:44,765][105620] Updated weights for policy 1, policy_version 810395 (0.0006) [2023-12-26 21:15:44,832][105620] Updated weights for policy 1, policy_version 810405 (0.0008) [2023-12-26 21:15:45,092][105692] Updated weights for policy 0, policy_version 810511 (0.0009) [2023-12-26 21:15:45,156][105692] Updated weights for policy 0, policy_version 810521 (0.0009) [2023-12-26 21:15:45,219][105692] Updated weights for policy 0, policy_version 810531 (0.0009) [2023-12-26 21:15:45,575][105620] Updated weights for policy 1, policy_version 810415 (0.0009) [2023-12-26 21:15:45,628][105620] Updated weights for policy 1, policy_version 810425 (0.0009) [2023-12-26 21:15:45,695][105620] Updated weights for policy 1, policy_version 810435 (0.0009) [2023-12-26 21:15:45,994][105692] Updated weights for policy 0, policy_version 810541 (0.0009) [2023-12-26 21:15:46,047][105692] Updated weights for policy 0, policy_version 810551 (0.0010) [2023-12-26 21:15:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 415023104. Throughput: 0: 9718.9, 1: 9824.8. Samples: 414996012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:15:46,063][104569] Avg episode reward: [(0, '7733.856'), (1, '8989.069')] [2023-12-26 21:15:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000810440_207495168.pth... [2023-12-26 21:15:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000809320_207208448.pth [2023-12-26 21:15:46,116][105692] Updated weights for policy 0, policy_version 810561 (0.0010) [2023-12-26 21:15:46,159][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000810568_207536128.pth... [2023-12-26 21:15:46,164][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000809384_207233024.pth [2023-12-26 21:15:46,418][105620] Updated weights for policy 1, policy_version 810445 (0.0009) [2023-12-26 21:15:46,468][105620] Updated weights for policy 1, policy_version 810455 (0.0009) [2023-12-26 21:15:46,523][105620] Updated weights for policy 1, policy_version 810465 (0.0009) [2023-12-26 21:15:46,889][105692] Updated weights for policy 0, policy_version 810571 (0.0009) [2023-12-26 21:15:46,952][105692] Updated weights for policy 0, policy_version 810581 (0.0008) [2023-12-26 21:15:47,010][105692] Updated weights for policy 0, policy_version 810591 (0.0008) [2023-12-26 21:15:47,323][105620] Updated weights for policy 1, policy_version 810475 (0.0007) [2023-12-26 21:15:47,382][105620] Updated weights for policy 1, policy_version 810485 (0.0009) [2023-12-26 21:15:47,450][105620] Updated weights for policy 1, policy_version 810495 (0.0009) [2023-12-26 21:15:47,667][105692] Updated weights for policy 0, policy_version 810601 (0.0006) [2023-12-26 21:15:47,725][105692] Updated weights for policy 0, policy_version 810611 (0.0009) [2023-12-26 21:15:47,775][105692] Updated weights for policy 0, policy_version 810621 (0.0009) [2023-12-26 21:15:47,829][105692] Updated weights for policy 0, policy_version 810631 (0.0008) [2023-12-26 21:15:48,151][105620] Updated weights for policy 1, policy_version 810505 (0.0009) [2023-12-26 21:15:48,204][105620] Updated weights for policy 1, policy_version 810515 (0.0008) [2023-12-26 21:15:48,273][105620] Updated weights for policy 1, policy_version 810525 (0.0007) [2023-12-26 21:15:48,339][105620] Updated weights for policy 1, policy_version 810535 (0.0008) [2023-12-26 21:15:48,665][105692] Updated weights for policy 0, policy_version 810641 (0.0009) [2023-12-26 21:15:48,728][105692] Updated weights for policy 0, policy_version 810651 (0.0008) [2023-12-26 21:15:48,787][105692] Updated weights for policy 0, policy_version 810661 (0.0008) [2023-12-26 21:15:49,037][105620] Updated weights for policy 1, policy_version 810545 (0.0007) [2023-12-26 21:15:49,095][105620] Updated weights for policy 1, policy_version 810555 (0.0007) [2023-12-26 21:15:49,166][105620] Updated weights for policy 1, policy_version 810565 (0.0011) [2023-12-26 21:15:49,508][105692] Updated weights for policy 0, policy_version 810671 (0.0008) [2023-12-26 21:15:49,578][105692] Updated weights for policy 0, policy_version 810681 (0.0008) [2023-12-26 21:15:49,643][105692] Updated weights for policy 0, policy_version 810691 (0.0009) [2023-12-26 21:15:49,914][105620] Updated weights for policy 1, policy_version 810575 (0.0010) [2023-12-26 21:15:49,978][105620] Updated weights for policy 1, policy_version 810585 (0.0008) [2023-12-26 21:15:50,042][105620] Updated weights for policy 1, policy_version 810595 (0.0008) [2023-12-26 21:15:50,258][105692] Updated weights for policy 0, policy_version 810701 (0.0008) [2023-12-26 21:15:50,319][105692] Updated weights for policy 0, policy_version 810711 (0.0008) [2023-12-26 21:15:50,373][105692] Updated weights for policy 0, policy_version 810721 (0.0010) [2023-12-26 21:15:50,769][105620] Updated weights for policy 1, policy_version 810605 (0.0007) [2023-12-26 21:15:50,838][105620] Updated weights for policy 1, policy_version 810615 (0.0008) [2023-12-26 21:15:50,901][105620] Updated weights for policy 1, policy_version 810625 (0.0009) [2023-12-26 21:15:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 415121408. Throughput: 0: 9682.1, 1: 9816.3. Samples: 415110004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:15:51,063][104569] Avg episode reward: [(0, '8189.658'), (1, '9080.914')] [2023-12-26 21:15:51,160][105692] Updated weights for policy 0, policy_version 810731 (0.0009) [2023-12-26 21:15:51,228][105692] Updated weights for policy 0, policy_version 810741 (0.0009) [2023-12-26 21:15:51,297][105692] Updated weights for policy 0, policy_version 810751 (0.0008) [2023-12-26 21:15:51,669][105620] Updated weights for policy 1, policy_version 810635 (0.0009) [2023-12-26 21:15:51,739][105620] Updated weights for policy 1, policy_version 810645 (0.0009) [2023-12-26 21:15:51,800][105620] Updated weights for policy 1, policy_version 810655 (0.0008) [2023-12-26 21:15:52,014][105692] Updated weights for policy 0, policy_version 810761 (0.0008) [2023-12-26 21:15:52,077][105692] Updated weights for policy 0, policy_version 810771 (0.0006) [2023-12-26 21:15:52,137][105692] Updated weights for policy 0, policy_version 810781 (0.0008) [2023-12-26 21:15:52,196][105692] Updated weights for policy 0, policy_version 810791 (0.0010) [2023-12-26 21:15:52,612][105620] Updated weights for policy 1, policy_version 810665 (0.0006) [2023-12-26 21:15:52,659][105620] Updated weights for policy 1, policy_version 810675 (0.0007) [2023-12-26 21:15:52,716][105620] Updated weights for policy 1, policy_version 810685 (0.0008) [2023-12-26 21:15:52,779][105620] Updated weights for policy 1, policy_version 810695 (0.0009) [2023-12-26 21:15:52,849][105692] Updated weights for policy 0, policy_version 810801 (0.0009) [2023-12-26 21:15:52,906][105692] Updated weights for policy 0, policy_version 810811 (0.0008) [2023-12-26 21:15:52,960][105692] Updated weights for policy 0, policy_version 810821 (0.0009) [2023-12-26 21:15:53,434][105620] Updated weights for policy 1, policy_version 810705 (0.0010) [2023-12-26 21:15:53,492][105620] Updated weights for policy 1, policy_version 810715 (0.0010) [2023-12-26 21:15:53,540][105620] Updated weights for policy 1, policy_version 810725 (0.0010) [2023-12-26 21:15:53,787][105692] Updated weights for policy 0, policy_version 810831 (0.0009) [2023-12-26 21:15:53,840][105692] Updated weights for policy 0, policy_version 810841 (0.0010) [2023-12-26 21:15:53,892][105692] Updated weights for policy 0, policy_version 810852 (0.0010) [2023-12-26 21:15:54,140][105620] Updated weights for policy 1, policy_version 810735 (0.0007) [2023-12-26 21:15:54,202][105620] Updated weights for policy 1, policy_version 810745 (0.0005) [2023-12-26 21:15:54,254][105620] Updated weights for policy 1, policy_version 810755 (0.0005) [2023-12-26 21:15:54,702][105692] Updated weights for policy 0, policy_version 810862 (0.0010) [2023-12-26 21:15:54,754][105692] Updated weights for policy 0, policy_version 810872 (0.0010) [2023-12-26 21:15:54,808][105692] Updated weights for policy 0, policy_version 810882 (0.0009) [2023-12-26 21:15:54,814][105620] Updated weights for policy 1, policy_version 810765 (0.0007) [2023-12-26 21:15:54,873][105620] Updated weights for policy 1, policy_version 810775 (0.0010) [2023-12-26 21:15:54,930][105620] Updated weights for policy 1, policy_version 810785 (0.0010) [2023-12-26 21:15:55,574][105692] Updated weights for policy 0, policy_version 810892 (0.0006) [2023-12-26 21:15:55,628][105692] Updated weights for policy 0, policy_version 810902 (0.0008) [2023-12-26 21:15:55,681][105692] Updated weights for policy 0, policy_version 810912 (0.0009) [2023-12-26 21:15:55,684][105620] Updated weights for policy 1, policy_version 810795 (0.0009) [2023-12-26 21:15:55,734][105620] Updated weights for policy 1, policy_version 810805 (0.0005) [2023-12-26 21:15:55,782][105620] Updated weights for policy 1, policy_version 810815 (0.0005) [2023-12-26 21:15:56,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.1, 300 sec: 19466.4). Total num frames: 415219712. Throughput: 0: 9713.3, 1: 9775.6. Samples: 415226632. Policy #0 lag: (min: 2.0, avg: 17.7, max: 34.0) [2023-12-26 21:15:56,063][104569] Avg episode reward: [(0, '8457.420'), (1, '9172.296')] [2023-12-26 21:15:56,411][105620] Updated weights for policy 1, policy_version 810825 (0.0006) [2023-12-26 21:15:56,438][105692] Updated weights for policy 0, policy_version 810922 (0.0008) [2023-12-26 21:15:56,460][105620] Updated weights for policy 1, policy_version 810835 (0.0010) [2023-12-26 21:15:56,494][105692] Updated weights for policy 0, policy_version 810932 (0.0006) [2023-12-26 21:15:56,513][105620] Updated weights for policy 1, policy_version 810845 (0.0010) [2023-12-26 21:15:56,544][105692] Updated weights for policy 0, policy_version 810942 (0.0005) [2023-12-26 21:15:56,563][105620] Updated weights for policy 1, policy_version 810855 (0.0011) [2023-12-26 21:15:56,597][105692] Updated weights for policy 0, policy_version 810952 (0.0007) [2023-12-26 21:15:57,229][105620] Updated weights for policy 1, policy_version 810865 (0.0010) [2023-12-26 21:15:57,283][105620] Updated weights for policy 1, policy_version 810875 (0.0005) [2023-12-26 21:15:57,339][105620] Updated weights for policy 1, policy_version 810885 (0.0010) [2023-12-26 21:15:57,356][105692] Updated weights for policy 0, policy_version 810962 (0.0006) [2023-12-26 21:15:57,414][105692] Updated weights for policy 0, policy_version 810972 (0.0008) [2023-12-26 21:15:57,469][105692] Updated weights for policy 0, policy_version 810982 (0.0008) [2023-12-26 21:15:57,977][105620] Updated weights for policy 1, policy_version 810895 (0.0007) [2023-12-26 21:15:58,022][105620] Updated weights for policy 1, policy_version 810905 (0.0005) [2023-12-26 21:15:58,076][105620] Updated weights for policy 1, policy_version 810915 (0.0005) [2023-12-26 21:15:58,294][105692] Updated weights for policy 0, policy_version 810992 (0.0008) [2023-12-26 21:15:58,357][105692] Updated weights for policy 0, policy_version 811002 (0.0008) [2023-12-26 21:15:58,424][105692] Updated weights for policy 0, policy_version 811012 (0.0009) [2023-12-26 21:15:58,893][105620] Updated weights for policy 1, policy_version 810925 (0.0007) [2023-12-26 21:15:58,959][105620] Updated weights for policy 1, policy_version 810935 (0.0009) [2023-12-26 21:15:59,019][105620] Updated weights for policy 1, policy_version 810945 (0.0008) [2023-12-26 21:15:59,282][105692] Updated weights for policy 0, policy_version 811022 (0.0008) [2023-12-26 21:15:59,346][105692] Updated weights for policy 0, policy_version 811032 (0.0008) [2023-12-26 21:15:59,417][105692] Updated weights for policy 0, policy_version 811042 (0.0008) [2023-12-26 21:15:59,741][105620] Updated weights for policy 1, policy_version 810955 (0.0007) [2023-12-26 21:15:59,798][105620] Updated weights for policy 1, policy_version 810965 (0.0011) [2023-12-26 21:15:59,868][105620] Updated weights for policy 1, policy_version 810975 (0.0011) [2023-12-26 21:16:00,257][105692] Updated weights for policy 0, policy_version 811052 (0.0009) [2023-12-26 21:16:00,308][105692] Updated weights for policy 0, policy_version 811062 (0.0007) [2023-12-26 21:16:00,366][105692] Updated weights for policy 0, policy_version 811072 (0.0008) [2023-12-26 21:16:00,592][105620] Updated weights for policy 1, policy_version 810985 (0.0010) [2023-12-26 21:16:00,653][105620] Updated weights for policy 1, policy_version 810995 (0.0009) [2023-12-26 21:16:00,704][105620] Updated weights for policy 1, policy_version 811005 (0.0010) [2023-12-26 21:16:00,755][105620] Updated weights for policy 1, policy_version 811015 (0.0010) [2023-12-26 21:16:01,058][105692] Updated weights for policy 0, policy_version 811082 (0.0008) [2023-12-26 21:16:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 415309824. Throughput: 0: 9715.2, 1: 9783.6. Samples: 415283236. Policy #0 lag: (min: 2.0, avg: 17.7, max: 34.0) [2023-12-26 21:16:01,063][104569] Avg episode reward: [(0, '8452.132'), (1, '9262.876')] [2023-12-26 21:16:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000811016_207642624.pth... [2023-12-26 21:16:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000809864_207347712.pth [2023-12-26 21:16:01,114][105692] Updated weights for policy 0, policy_version 811092 (0.0008) [2023-12-26 21:16:01,180][105692] Updated weights for policy 0, policy_version 811102 (0.0009) [2023-12-26 21:16:01,236][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000811112_207675392.pth... [2023-12-26 21:16:01,239][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000809992_207388672.pth [2023-12-26 21:16:01,238][105692] Updated weights for policy 0, policy_version 811112 (0.0008) [2023-12-26 21:16:01,504][105620] Updated weights for policy 1, policy_version 811025 (0.0009) [2023-12-26 21:16:01,553][105620] Updated weights for policy 1, policy_version 811035 (0.0009) [2023-12-26 21:16:01,607][105620] Updated weights for policy 1, policy_version 811045 (0.0009) [2023-12-26 21:16:01,938][105692] Updated weights for policy 0, policy_version 811122 (0.0009) [2023-12-26 21:16:02,000][105692] Updated weights for policy 0, policy_version 811132 (0.0009) [2023-12-26 21:16:02,057][105692] Updated weights for policy 0, policy_version 811142 (0.0009) [2023-12-26 21:16:02,353][105620] Updated weights for policy 1, policy_version 811055 (0.0008) [2023-12-26 21:16:02,401][105620] Updated weights for policy 1, policy_version 811065 (0.0006) [2023-12-26 21:16:02,454][105620] Updated weights for policy 1, policy_version 811075 (0.0006) [2023-12-26 21:16:02,853][105692] Updated weights for policy 0, policy_version 811152 (0.0009) [2023-12-26 21:16:02,914][105692] Updated weights for policy 0, policy_version 811162 (0.0009) [2023-12-26 21:16:02,967][105692] Updated weights for policy 0, policy_version 811172 (0.0009) [2023-12-26 21:16:03,165][105620] Updated weights for policy 1, policy_version 811085 (0.0009) [2023-12-26 21:16:03,215][105620] Updated weights for policy 1, policy_version 811095 (0.0009) [2023-12-26 21:16:03,266][105620] Updated weights for policy 1, policy_version 811105 (0.0009) [2023-12-26 21:16:03,708][105692] Updated weights for policy 0, policy_version 811182 (0.0008) [2023-12-26 21:16:03,762][105692] Updated weights for policy 0, policy_version 811192 (0.0008) [2023-12-26 21:16:03,823][105692] Updated weights for policy 0, policy_version 811202 (0.0009) [2023-12-26 21:16:04,072][105620] Updated weights for policy 1, policy_version 811115 (0.0009) [2023-12-26 21:16:04,136][105620] Updated weights for policy 1, policy_version 811125 (0.0008) [2023-12-26 21:16:04,200][105620] Updated weights for policy 1, policy_version 811135 (0.0009) [2023-12-26 21:16:04,565][105692] Updated weights for policy 0, policy_version 811212 (0.0009) [2023-12-26 21:16:04,631][105692] Updated weights for policy 0, policy_version 811222 (0.0010) [2023-12-26 21:16:04,695][105692] Updated weights for policy 0, policy_version 811232 (0.0007) [2023-12-26 21:16:05,012][105620] Updated weights for policy 1, policy_version 811145 (0.0009) [2023-12-26 21:16:05,067][105620] Updated weights for policy 1, policy_version 811155 (0.0009) [2023-12-26 21:16:05,123][105620] Updated weights for policy 1, policy_version 811165 (0.0010) [2023-12-26 21:16:05,176][105620] Updated weights for policy 1, policy_version 811175 (0.0010) [2023-12-26 21:16:05,278][105692] Updated weights for policy 0, policy_version 811242 (0.0006) [2023-12-26 21:16:05,338][105692] Updated weights for policy 0, policy_version 811252 (0.0006) [2023-12-26 21:16:05,401][105692] Updated weights for policy 0, policy_version 811262 (0.0009) [2023-12-26 21:16:05,467][105692] Updated weights for policy 0, policy_version 811272 (0.0009) [2023-12-26 21:16:06,024][105692] Updated weights for policy 0, policy_version 811282 (0.0005) [2023-12-26 21:16:06,046][105620] Updated weights for policy 1, policy_version 811185 (0.0009) [2023-12-26 21:16:06,062][104569] Fps is (10 sec: 18022.8, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 415399936. Throughput: 0: 9632.9, 1: 9693.2. Samples: 415395060. Policy #0 lag: (min: 2.0, avg: 17.7, max: 34.0) [2023-12-26 21:16:06,063][104569] Avg episode reward: [(0, '8288.778'), (1, '9172.102')] [2023-12-26 21:16:06,082][105692] Updated weights for policy 0, policy_version 811292 (0.0005) [2023-12-26 21:16:06,109][105620] Updated weights for policy 1, policy_version 811195 (0.0009) [2023-12-26 21:16:06,151][105692] Updated weights for policy 0, policy_version 811302 (0.0008) [2023-12-26 21:16:06,170][105620] Updated weights for policy 1, policy_version 811205 (0.0007) [2023-12-26 21:16:06,809][105692] Updated weights for policy 0, policy_version 811312 (0.0006) [2023-12-26 21:16:06,877][105692] Updated weights for policy 0, policy_version 811322 (0.0006) [2023-12-26 21:16:06,944][105692] Updated weights for policy 0, policy_version 811332 (0.0007) [2023-12-26 21:16:07,009][105620] Updated weights for policy 1, policy_version 811215 (0.0009) [2023-12-26 21:16:07,078][105620] Updated weights for policy 1, policy_version 811225 (0.0009) [2023-12-26 21:16:07,137][105620] Updated weights for policy 1, policy_version 811235 (0.0009) [2023-12-26 21:16:07,616][105692] Updated weights for policy 0, policy_version 811342 (0.0008) [2023-12-26 21:16:07,663][105692] Updated weights for policy 0, policy_version 811352 (0.0009) [2023-12-26 21:16:07,714][105692] Updated weights for policy 0, policy_version 811362 (0.0009) [2023-12-26 21:16:07,906][105620] Updated weights for policy 1, policy_version 811245 (0.0009) [2023-12-26 21:16:07,961][105620] Updated weights for policy 1, policy_version 811255 (0.0009) [2023-12-26 21:16:08,008][105620] Updated weights for policy 1, policy_version 811265 (0.0008) [2023-12-26 21:16:08,475][105692] Updated weights for policy 0, policy_version 811372 (0.0007) [2023-12-26 21:16:08,527][105692] Updated weights for policy 0, policy_version 811382 (0.0006) [2023-12-26 21:16:08,582][105692] Updated weights for policy 0, policy_version 811392 (0.0009) [2023-12-26 21:16:08,829][105620] Updated weights for policy 1, policy_version 811275 (0.0009) [2023-12-26 21:16:08,888][105620] Updated weights for policy 1, policy_version 811285 (0.0008) [2023-12-26 21:16:08,949][105620] Updated weights for policy 1, policy_version 811295 (0.0007) [2023-12-26 21:16:09,268][105692] Updated weights for policy 0, policy_version 811402 (0.0009) [2023-12-26 21:16:09,338][105692] Updated weights for policy 0, policy_version 811412 (0.0011) [2023-12-26 21:16:09,404][105692] Updated weights for policy 0, policy_version 811422 (0.0011) [2023-12-26 21:16:09,468][105692] Updated weights for policy 0, policy_version 811432 (0.0011) [2023-12-26 21:16:09,734][105620] Updated weights for policy 1, policy_version 811305 (0.0010) [2023-12-26 21:16:09,804][105620] Updated weights for policy 1, policy_version 811315 (0.0008) [2023-12-26 21:16:09,874][105620] Updated weights for policy 1, policy_version 811325 (0.0008) [2023-12-26 21:16:09,931][105620] Updated weights for policy 1, policy_version 811335 (0.0008) [2023-12-26 21:16:10,234][105692] Updated weights for policy 0, policy_version 811442 (0.0009) [2023-12-26 21:16:10,290][105692] Updated weights for policy 0, policy_version 811452 (0.0009) [2023-12-26 21:16:10,346][105692] Updated weights for policy 0, policy_version 811462 (0.0009) [2023-12-26 21:16:10,666][105620] Updated weights for policy 1, policy_version 811345 (0.0008) [2023-12-26 21:16:10,713][105620] Updated weights for policy 1, policy_version 811355 (0.0009) [2023-12-26 21:16:10,774][105620] Updated weights for policy 1, policy_version 811366 (0.0010) [2023-12-26 21:16:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 415498240. Throughput: 0: 9715.2, 1: 9579.3. Samples: 415507624. Policy #0 lag: (min: 2.0, avg: 17.7, max: 34.0) [2023-12-26 21:16:11,062][104569] Avg episode reward: [(0, '8296.368'), (1, '9171.812')] [2023-12-26 21:16:11,078][105692] Updated weights for policy 0, policy_version 811472 (0.0008) [2023-12-26 21:16:11,144][105692] Updated weights for policy 0, policy_version 811482 (0.0008) [2023-12-26 21:16:11,207][105692] Updated weights for policy 0, policy_version 811492 (0.0007) [2023-12-26 21:16:11,649][105620] Updated weights for policy 1, policy_version 811376 (0.0010) [2023-12-26 21:16:11,715][105620] Updated weights for policy 1, policy_version 811386 (0.0008) [2023-12-26 21:16:11,779][105620] Updated weights for policy 1, policy_version 811396 (0.0008) [2023-12-26 21:16:11,916][105692] Updated weights for policy 0, policy_version 811502 (0.0007) [2023-12-26 21:16:11,977][105692] Updated weights for policy 0, policy_version 811512 (0.0008) [2023-12-26 21:16:12,046][105692] Updated weights for policy 0, policy_version 811522 (0.0008) [2023-12-26 21:16:12,510][105620] Updated weights for policy 1, policy_version 811406 (0.0009) [2023-12-26 21:16:12,568][105620] Updated weights for policy 1, policy_version 811416 (0.0007) [2023-12-26 21:16:12,628][105620] Updated weights for policy 1, policy_version 811426 (0.0007) [2023-12-26 21:16:12,787][105692] Updated weights for policy 0, policy_version 811532 (0.0008) [2023-12-26 21:16:12,844][105692] Updated weights for policy 0, policy_version 811542 (0.0008) [2023-12-26 21:16:12,893][105692] Updated weights for policy 0, policy_version 811552 (0.0005) [2023-12-26 21:16:13,389][105620] Updated weights for policy 1, policy_version 811436 (0.0007) [2023-12-26 21:16:13,465][105620] Updated weights for policy 1, policy_version 811446 (0.0005) [2023-12-26 21:16:13,514][105692] Updated weights for policy 0, policy_version 811562 (0.0005) [2023-12-26 21:16:13,531][105620] Updated weights for policy 1, policy_version 811456 (0.0005) [2023-12-26 21:16:13,604][105692] Updated weights for policy 0, policy_version 811572 (0.0006) [2023-12-26 21:16:13,667][105692] Updated weights for policy 0, policy_version 811582 (0.0005) [2023-12-26 21:16:13,727][105692] Updated weights for policy 0, policy_version 811592 (0.0005) [2023-12-26 21:16:14,018][105620] Updated weights for policy 1, policy_version 811466 (0.0005) [2023-12-26 21:16:14,070][105620] Updated weights for policy 1, policy_version 811476 (0.0005) [2023-12-26 21:16:14,132][105620] Updated weights for policy 1, policy_version 811486 (0.0005) [2023-12-26 21:16:14,196][105620] Updated weights for policy 1, policy_version 811496 (0.0005) [2023-12-26 21:16:14,330][105692] Updated weights for policy 0, policy_version 811602 (0.0006) [2023-12-26 21:16:14,378][105692] Updated weights for policy 0, policy_version 811612 (0.0010) [2023-12-26 21:16:14,427][105692] Updated weights for policy 0, policy_version 811622 (0.0010) [2023-12-26 21:16:14,711][105620] Updated weights for policy 1, policy_version 811506 (0.0006) [2023-12-26 21:16:14,777][105620] Updated weights for policy 1, policy_version 811516 (0.0009) [2023-12-26 21:16:14,832][105620] Updated weights for policy 1, policy_version 811526 (0.0010) [2023-12-26 21:16:15,178][105692] Updated weights for policy 0, policy_version 811632 (0.0009) [2023-12-26 21:16:15,249][105692] Updated weights for policy 0, policy_version 811642 (0.0005) [2023-12-26 21:16:15,311][105692] Updated weights for policy 0, policy_version 811652 (0.0005) [2023-12-26 21:16:15,573][105620] Updated weights for policy 1, policy_version 811536 (0.0006) [2023-12-26 21:16:15,632][105620] Updated weights for policy 1, policy_version 811546 (0.0005) [2023-12-26 21:16:15,687][105620] Updated weights for policy 1, policy_version 811556 (0.0008) [2023-12-26 21:16:15,937][105692] Updated weights for policy 0, policy_version 811662 (0.0008) [2023-12-26 21:16:15,985][105692] Updated weights for policy 0, policy_version 811672 (0.0010) [2023-12-26 21:16:16,034][105692] Updated weights for policy 0, policy_version 811682 (0.0010) [2023-12-26 21:16:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 415596544. Throughput: 0: 9693.0, 1: 9610.6. Samples: 415567444. Policy #0 lag: (min: 2.0, avg: 17.7, max: 34.0) [2023-12-26 21:16:16,062][104569] Avg episode reward: [(0, '8813.278'), (1, '9262.456')] [2023-12-26 21:16:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000811688_207822848.pth... [2023-12-26 21:16:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000811560_207781888.pth... [2023-12-26 21:16:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000810568_207536128.pth [2023-12-26 21:16:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000810440_207495168.pth [2023-12-26 21:16:16,358][105620] Updated weights for policy 1, policy_version 811566 (0.0010) [2023-12-26 21:16:16,423][105620] Updated weights for policy 1, policy_version 811576 (0.0010) [2023-12-26 21:16:16,481][105620] Updated weights for policy 1, policy_version 811586 (0.0010) [2023-12-26 21:16:16,628][105692] Updated weights for policy 0, policy_version 811692 (0.0010) [2023-12-26 21:16:16,689][105692] Updated weights for policy 0, policy_version 811702 (0.0008) [2023-12-26 21:16:16,734][105692] Updated weights for policy 0, policy_version 811712 (0.0009) [2023-12-26 21:16:17,217][105620] Updated weights for policy 1, policy_version 811596 (0.0010) [2023-12-26 21:16:17,268][105620] Updated weights for policy 1, policy_version 811606 (0.0010) [2023-12-26 21:16:17,329][105692] Updated weights for policy 0, policy_version 811722 (0.0005) [2023-12-26 21:16:17,330][105620] Updated weights for policy 1, policy_version 811616 (0.0010) [2023-12-26 21:16:17,377][105692] Updated weights for policy 0, policy_version 811732 (0.0005) [2023-12-26 21:16:17,431][105692] Updated weights for policy 0, policy_version 811742 (0.0005) [2023-12-26 21:16:17,489][105692] Updated weights for policy 0, policy_version 811752 (0.0005) [2023-12-26 21:16:18,073][105620] Updated weights for policy 1, policy_version 811626 (0.0010) [2023-12-26 21:16:18,091][105692] Updated weights for policy 0, policy_version 811762 (0.0007) [2023-12-26 21:16:18,128][105620] Updated weights for policy 1, policy_version 811636 (0.0010) [2023-12-26 21:16:18,151][105692] Updated weights for policy 0, policy_version 811772 (0.0006) [2023-12-26 21:16:18,187][105620] Updated weights for policy 1, policy_version 811646 (0.0007) [2023-12-26 21:16:18,204][105692] Updated weights for policy 0, policy_version 811782 (0.0007) [2023-12-26 21:16:18,253][105620] Updated weights for policy 1, policy_version 811656 (0.0005) [2023-12-26 21:16:18,813][105620] Updated weights for policy 1, policy_version 811666 (0.0009) [2023-12-26 21:16:18,865][105620] Updated weights for policy 1, policy_version 811676 (0.0010) [2023-12-26 21:16:18,887][105692] Updated weights for policy 0, policy_version 811792 (0.0006) [2023-12-26 21:16:18,918][105620] Updated weights for policy 1, policy_version 811686 (0.0008) [2023-12-26 21:16:18,952][105692] Updated weights for policy 0, policy_version 811802 (0.0008) [2023-12-26 21:16:19,007][105692] Updated weights for policy 0, policy_version 811812 (0.0010) [2023-12-26 21:16:19,543][105620] Updated weights for policy 1, policy_version 811696 (0.0010) [2023-12-26 21:16:19,607][105620] Updated weights for policy 1, policy_version 811706 (0.0007) [2023-12-26 21:16:19,670][105620] Updated weights for policy 1, policy_version 811716 (0.0005) [2023-12-26 21:16:19,806][105692] Updated weights for policy 0, policy_version 811822 (0.0009) [2023-12-26 21:16:19,871][105692] Updated weights for policy 0, policy_version 811832 (0.0008) [2023-12-26 21:16:19,938][105692] Updated weights for policy 0, policy_version 811842 (0.0009) [2023-12-26 21:16:20,394][105620] Updated weights for policy 1, policy_version 811726 (0.0008) [2023-12-26 21:16:20,452][105620] Updated weights for policy 1, policy_version 811736 (0.0010) [2023-12-26 21:16:20,514][105620] Updated weights for policy 1, policy_version 811746 (0.0010) [2023-12-26 21:16:20,725][105692] Updated weights for policy 0, policy_version 811852 (0.0008) [2023-12-26 21:16:20,776][105692] Updated weights for policy 0, policy_version 811862 (0.0008) [2023-12-26 21:16:20,826][105692] Updated weights for policy 0, policy_version 811872 (0.0008) [2023-12-26 21:16:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 415703040. Throughput: 0: 9686.9, 1: 9697.2. Samples: 415692832. Policy #0 lag: (min: 2.0, avg: 17.7, max: 34.0) [2023-12-26 21:16:21,062][104569] Avg episode reward: [(0, '8987.115'), (1, '9170.900')] [2023-12-26 21:16:21,246][105620] Updated weights for policy 1, policy_version 811756 (0.0009) [2023-12-26 21:16:21,301][105620] Updated weights for policy 1, policy_version 811766 (0.0011) [2023-12-26 21:16:21,364][105620] Updated weights for policy 1, policy_version 811776 (0.0010) [2023-12-26 21:16:21,604][105692] Updated weights for policy 0, policy_version 811882 (0.0007) [2023-12-26 21:16:21,675][105692] Updated weights for policy 0, policy_version 811892 (0.0007) [2023-12-26 21:16:21,737][105692] Updated weights for policy 0, policy_version 811902 (0.0010) [2023-12-26 21:16:21,805][105692] Updated weights for policy 0, policy_version 811912 (0.0010) [2023-12-26 21:16:22,069][105620] Updated weights for policy 1, policy_version 811786 (0.0006) [2023-12-26 21:16:22,132][105620] Updated weights for policy 1, policy_version 811796 (0.0011) [2023-12-26 21:16:22,192][105620] Updated weights for policy 1, policy_version 811806 (0.0011) [2023-12-26 21:16:22,258][105620] Updated weights for policy 1, policy_version 811816 (0.0011) [2023-12-26 21:16:22,549][105692] Updated weights for policy 0, policy_version 811922 (0.0010) [2023-12-26 21:16:22,614][105692] Updated weights for policy 0, policy_version 811932 (0.0010) [2023-12-26 21:16:22,677][105692] Updated weights for policy 0, policy_version 811943 (0.0009) [2023-12-26 21:16:22,955][105620] Updated weights for policy 1, policy_version 811826 (0.0009) [2023-12-26 21:16:23,011][105620] Updated weights for policy 1, policy_version 811836 (0.0008) [2023-12-26 21:16:23,059][105620] Updated weights for policy 1, policy_version 811846 (0.0008) [2023-12-26 21:16:23,380][105692] Updated weights for policy 0, policy_version 811953 (0.0010) [2023-12-26 21:16:23,434][105692] Updated weights for policy 0, policy_version 811964 (0.0010) [2023-12-26 21:16:23,487][105692] Updated weights for policy 0, policy_version 811975 (0.0010) [2023-12-26 21:16:23,684][105620] Updated weights for policy 1, policy_version 811856 (0.0006) [2023-12-26 21:16:23,729][105620] Updated weights for policy 1, policy_version 811866 (0.0005) [2023-12-26 21:16:23,782][105620] Updated weights for policy 1, policy_version 811876 (0.0006) [2023-12-26 21:16:24,305][105620] Updated weights for policy 1, policy_version 811886 (0.0007) [2023-12-26 21:16:24,365][105620] Updated weights for policy 1, policy_version 811896 (0.0005) [2023-12-26 21:16:24,413][105692] Updated weights for policy 0, policy_version 811985 (0.0006) [2023-12-26 21:16:24,428][105620] Updated weights for policy 1, policy_version 811906 (0.0009) [2023-12-26 21:16:24,473][105692] Updated weights for policy 0, policy_version 811995 (0.0006) [2023-12-26 21:16:24,528][105692] Updated weights for policy 0, policy_version 812005 (0.0010) [2023-12-26 21:16:24,957][105620] Updated weights for policy 1, policy_version 811916 (0.0008) [2023-12-26 21:16:25,027][105620] Updated weights for policy 1, policy_version 811926 (0.0011) [2023-12-26 21:16:25,085][105620] Updated weights for policy 1, policy_version 811936 (0.0008) [2023-12-26 21:16:25,241][105692] Updated weights for policy 0, policy_version 812015 (0.0008) [2023-12-26 21:16:25,297][105692] Updated weights for policy 0, policy_version 812025 (0.0006) [2023-12-26 21:16:25,355][105692] Updated weights for policy 0, policy_version 812035 (0.0006) [2023-12-26 21:16:25,714][105620] Updated weights for policy 1, policy_version 811946 (0.0006) [2023-12-26 21:16:25,771][105620] Updated weights for policy 1, policy_version 811956 (0.0010) [2023-12-26 21:16:25,832][105620] Updated weights for policy 1, policy_version 811966 (0.0010) [2023-12-26 21:16:25,886][105620] Updated weights for policy 1, policy_version 811976 (0.0010) [2023-12-26 21:16:25,902][105692] Updated weights for policy 0, policy_version 812045 (0.0006) [2023-12-26 21:16:25,949][105692] Updated weights for policy 0, policy_version 812055 (0.0005) [2023-12-26 21:16:26,003][105692] Updated weights for policy 0, policy_version 812065 (0.0005) [2023-12-26 21:16:26,062][104569] Fps is (10 sec: 21299.6, 60 sec: 19660.9, 300 sec: 19494.2). Total num frames: 415809536. Throughput: 0: 9599.1, 1: 9764.5. Samples: 415810756. Policy #0 lag: (min: 2.0, avg: 17.7, max: 34.0) [2023-12-26 21:16:26,062][104569] Avg episode reward: [(0, '8993.369'), (1, '9169.955')] [2023-12-26 21:16:26,531][105692] Updated weights for policy 0, policy_version 812075 (0.0005) [2023-12-26 21:16:26,590][105692] Updated weights for policy 0, policy_version 812085 (0.0005) [2023-12-26 21:16:26,610][105620] Updated weights for policy 1, policy_version 811986 (0.0010) [2023-12-26 21:16:26,645][105692] Updated weights for policy 0, policy_version 812095 (0.0005) [2023-12-26 21:16:26,666][105620] Updated weights for policy 1, policy_version 811996 (0.0010) [2023-12-26 21:16:26,728][105620] Updated weights for policy 1, policy_version 812006 (0.0010) [2023-12-26 21:16:27,230][105692] Updated weights for policy 0, policy_version 812105 (0.0006) [2023-12-26 21:16:27,294][105692] Updated weights for policy 0, policy_version 812115 (0.0005) [2023-12-26 21:16:27,353][105692] Updated weights for policy 0, policy_version 812125 (0.0008) [2023-12-26 21:16:27,418][105692] Updated weights for policy 0, policy_version 812135 (0.0005) [2023-12-26 21:16:27,458][105620] Updated weights for policy 1, policy_version 812016 (0.0007) [2023-12-26 21:16:27,527][105620] Updated weights for policy 1, policy_version 812026 (0.0006) [2023-12-26 21:16:27,587][105620] Updated weights for policy 1, policy_version 812036 (0.0005) [2023-12-26 21:16:27,930][105692] Updated weights for policy 0, policy_version 812145 (0.0005) [2023-12-26 21:16:27,991][105692] Updated weights for policy 0, policy_version 812155 (0.0006) [2023-12-26 21:16:28,044][105692] Updated weights for policy 0, policy_version 812165 (0.0010) [2023-12-26 21:16:28,210][105620] Updated weights for policy 1, policy_version 812046 (0.0008) [2023-12-26 21:16:28,266][105620] Updated weights for policy 1, policy_version 812056 (0.0010) [2023-12-26 21:16:28,322][105620] Updated weights for policy 1, policy_version 812066 (0.0011) [2023-12-26 21:16:28,709][105692] Updated weights for policy 0, policy_version 812175 (0.0008) [2023-12-26 21:16:28,766][105692] Updated weights for policy 0, policy_version 812185 (0.0009) [2023-12-26 21:16:28,817][105692] Updated weights for policy 0, policy_version 812195 (0.0008) [2023-12-26 21:16:28,956][105620] Updated weights for policy 1, policy_version 812076 (0.0010) [2023-12-26 21:16:29,009][105620] Updated weights for policy 1, policy_version 812086 (0.0010) [2023-12-26 21:16:29,072][105620] Updated weights for policy 1, policy_version 812096 (0.0007) [2023-12-26 21:16:29,606][105692] Updated weights for policy 0, policy_version 812205 (0.0008) [2023-12-26 21:16:29,663][105692] Updated weights for policy 0, policy_version 812215 (0.0005) [2023-12-26 21:16:29,715][105692] Updated weights for policy 0, policy_version 812225 (0.0005) [2023-12-26 21:16:29,774][105620] Updated weights for policy 1, policy_version 812106 (0.0010) [2023-12-26 21:16:29,836][105620] Updated weights for policy 1, policy_version 812116 (0.0011) [2023-12-26 21:16:29,900][105620] Updated weights for policy 1, policy_version 812126 (0.0011) [2023-12-26 21:16:29,964][105620] Updated weights for policy 1, policy_version 812136 (0.0011) [2023-12-26 21:16:30,436][105692] Updated weights for policy 0, policy_version 812235 (0.0006) [2023-12-26 21:16:30,493][105692] Updated weights for policy 0, policy_version 812245 (0.0005) [2023-12-26 21:16:30,539][105692] Updated weights for policy 0, policy_version 812255 (0.0005) [2023-12-26 21:16:30,694][105620] Updated weights for policy 1, policy_version 812146 (0.0010) [2023-12-26 21:16:30,746][105620] Updated weights for policy 1, policy_version 812156 (0.0010) [2023-12-26 21:16:30,796][105620] Updated weights for policy 1, policy_version 812166 (0.0010) [2023-12-26 21:16:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 415907840. Throughput: 0: 9755.2, 1: 9832.1. Samples: 415877440. Policy #0 lag: (min: 2.0, avg: 17.7, max: 34.0) [2023-12-26 21:16:31,062][104569] Avg episode reward: [(0, '8587.180'), (1, '9261.168')] [2023-12-26 21:16:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000812264_207970304.pth... [2023-12-26 21:16:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000812168_207937536.pth... [2023-12-26 21:16:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000811112_207675392.pth [2023-12-26 21:16:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000811016_207642624.pth [2023-12-26 21:16:31,221][105692] Updated weights for policy 0, policy_version 812265 (0.0007) [2023-12-26 21:16:31,278][105692] Updated weights for policy 0, policy_version 812275 (0.0008) [2023-12-26 21:16:31,327][105692] Updated weights for policy 0, policy_version 812285 (0.0008) [2023-12-26 21:16:31,385][105692] Updated weights for policy 0, policy_version 812295 (0.0008) [2023-12-26 21:16:31,562][105620] Updated weights for policy 1, policy_version 812176 (0.0010) [2023-12-26 21:16:31,621][105620] Updated weights for policy 1, policy_version 812186 (0.0010) [2023-12-26 21:16:31,683][105620] Updated weights for policy 1, policy_version 812196 (0.0008) [2023-12-26 21:16:32,178][105692] Updated weights for policy 0, policy_version 812305 (0.0010) [2023-12-26 21:16:32,229][105692] Updated weights for policy 0, policy_version 812315 (0.0010) [2023-12-26 21:16:32,286][105692] Updated weights for policy 0, policy_version 812325 (0.0010) [2023-12-26 21:16:32,319][105620] Updated weights for policy 1, policy_version 812206 (0.0006) [2023-12-26 21:16:32,385][105620] Updated weights for policy 1, policy_version 812216 (0.0006) [2023-12-26 21:16:32,442][105620] Updated weights for policy 1, policy_version 812226 (0.0005) [2023-12-26 21:16:32,974][105692] Updated weights for policy 0, policy_version 812335 (0.0007) [2023-12-26 21:16:33,042][105692] Updated weights for policy 0, policy_version 812345 (0.0005) [2023-12-26 21:16:33,079][105620] Updated weights for policy 1, policy_version 812236 (0.0006) [2023-12-26 21:16:33,107][105692] Updated weights for policy 0, policy_version 812355 (0.0006) [2023-12-26 21:16:33,133][105620] Updated weights for policy 1, policy_version 812246 (0.0008) [2023-12-26 21:16:33,187][105620] Updated weights for policy 1, policy_version 812256 (0.0005) [2023-12-26 21:16:33,701][105692] Updated weights for policy 0, policy_version 812365 (0.0007) [2023-12-26 21:16:33,758][105692] Updated weights for policy 0, policy_version 812375 (0.0008) [2023-12-26 21:16:33,823][105692] Updated weights for policy 0, policy_version 812385 (0.0010) [2023-12-26 21:16:33,846][105620] Updated weights for policy 1, policy_version 812266 (0.0006) [2023-12-26 21:16:33,913][105620] Updated weights for policy 1, policy_version 812276 (0.0007) [2023-12-26 21:16:33,979][105620] Updated weights for policy 1, policy_version 812286 (0.0008) [2023-12-26 21:16:34,049][105620] Updated weights for policy 1, policy_version 812296 (0.0008) [2023-12-26 21:16:34,496][105692] Updated weights for policy 0, policy_version 812395 (0.0010) [2023-12-26 21:16:34,553][105692] Updated weights for policy 0, policy_version 812405 (0.0010) [2023-12-26 21:16:34,612][105692] Updated weights for policy 0, policy_version 812415 (0.0008) [2023-12-26 21:16:34,748][105620] Updated weights for policy 1, policy_version 812306 (0.0010) [2023-12-26 21:16:34,796][105620] Updated weights for policy 1, policy_version 812316 (0.0010) [2023-12-26 21:16:34,847][105620] Updated weights for policy 1, policy_version 812326 (0.0010) [2023-12-26 21:16:35,186][105692] Updated weights for policy 0, policy_version 812425 (0.0005) [2023-12-26 21:16:35,235][105692] Updated weights for policy 0, policy_version 812435 (0.0005) [2023-12-26 21:16:35,285][105692] Updated weights for policy 0, policy_version 812445 (0.0005) [2023-12-26 21:16:35,331][105692] Updated weights for policy 0, policy_version 812455 (0.0007) [2023-12-26 21:16:35,570][105620] Updated weights for policy 1, policy_version 812336 (0.0009) [2023-12-26 21:16:35,621][105620] Updated weights for policy 1, policy_version 812346 (0.0009) [2023-12-26 21:16:35,688][105620] Updated weights for policy 1, policy_version 812356 (0.0008) [2023-12-26 21:16:36,000][105692] Updated weights for policy 0, policy_version 812465 (0.0007) [2023-12-26 21:16:36,058][105692] Updated weights for policy 0, policy_version 812475 (0.0010) [2023-12-26 21:16:36,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 416006144. Throughput: 0: 9828.7, 1: 9865.2. Samples: 415996232. Policy #0 lag: (min: 2.0, avg: 17.7, max: 34.0) [2023-12-26 21:16:36,063][104569] Avg episode reward: [(0, '8671.398'), (1, '9170.107')] [2023-12-26 21:16:36,117][105692] Updated weights for policy 0, policy_version 812485 (0.0007) [2023-12-26 21:16:36,436][105620] Updated weights for policy 1, policy_version 812366 (0.0009) [2023-12-26 21:16:36,495][105620] Updated weights for policy 1, policy_version 812376 (0.0010) [2023-12-26 21:16:36,555][105620] Updated weights for policy 1, policy_version 812386 (0.0011) [2023-12-26 21:16:36,759][105692] Updated weights for policy 0, policy_version 812495 (0.0009) [2023-12-26 21:16:36,811][105692] Updated weights for policy 0, policy_version 812505 (0.0008) [2023-12-26 21:16:36,859][105692] Updated weights for policy 0, policy_version 812515 (0.0005) [2023-12-26 21:16:37,293][105620] Updated weights for policy 1, policy_version 812396 (0.0010) [2023-12-26 21:16:37,350][105620] Updated weights for policy 1, policy_version 812406 (0.0010) [2023-12-26 21:16:37,412][105620] Updated weights for policy 1, policy_version 812416 (0.0010) [2023-12-26 21:16:37,507][105692] Updated weights for policy 0, policy_version 812525 (0.0008) [2023-12-26 21:16:37,573][105692] Updated weights for policy 0, policy_version 812535 (0.0008) [2023-12-26 21:16:37,634][105692] Updated weights for policy 0, policy_version 812545 (0.0010) [2023-12-26 21:16:38,154][105620] Updated weights for policy 1, policy_version 812426 (0.0011) [2023-12-26 21:16:38,202][105620] Updated weights for policy 1, policy_version 812436 (0.0010) [2023-12-26 21:16:38,270][105620] Updated weights for policy 1, policy_version 812446 (0.0010) [2023-12-26 21:16:38,340][105620] Updated weights for policy 1, policy_version 812456 (0.0010) [2023-12-26 21:16:38,368][105692] Updated weights for policy 0, policy_version 812555 (0.0011) [2023-12-26 21:16:38,424][105692] Updated weights for policy 0, policy_version 812565 (0.0010) [2023-12-26 21:16:38,471][105692] Updated weights for policy 0, policy_version 812575 (0.0010) [2023-12-26 21:16:39,089][105620] Updated weights for policy 1, policy_version 812466 (0.0010) [2023-12-26 21:16:39,159][105620] Updated weights for policy 1, policy_version 812476 (0.0006) [2023-12-26 21:16:39,181][105692] Updated weights for policy 0, policy_version 812585 (0.0010) [2023-12-26 21:16:39,224][105620] Updated weights for policy 1, policy_version 812486 (0.0007) [2023-12-26 21:16:39,239][105692] Updated weights for policy 0, policy_version 812595 (0.0007) [2023-12-26 21:16:39,291][105692] Updated weights for policy 0, policy_version 812605 (0.0007) [2023-12-26 21:16:39,344][105692] Updated weights for policy 0, policy_version 812615 (0.0007) [2023-12-26 21:16:39,929][105620] Updated weights for policy 1, policy_version 812496 (0.0008) [2023-12-26 21:16:39,992][105620] Updated weights for policy 1, policy_version 812506 (0.0008) [2023-12-26 21:16:40,053][105620] Updated weights for policy 1, policy_version 812516 (0.0008) [2023-12-26 21:16:40,059][105692] Updated weights for policy 0, policy_version 812625 (0.0009) [2023-12-26 21:16:40,111][105692] Updated weights for policy 0, policy_version 812635 (0.0009) [2023-12-26 21:16:40,162][105692] Updated weights for policy 0, policy_version 812645 (0.0009) [2023-12-26 21:16:40,738][105620] Updated weights for policy 1, policy_version 812526 (0.0008) [2023-12-26 21:16:40,790][105620] Updated weights for policy 1, policy_version 812536 (0.0008) [2023-12-26 21:16:40,851][105620] Updated weights for policy 1, policy_version 812546 (0.0007) [2023-12-26 21:16:40,983][105692] Updated weights for policy 0, policy_version 812655 (0.0006) [2023-12-26 21:16:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 416104448. Throughput: 0: 9930.0, 1: 9802.9. Samples: 416114608. Policy #0 lag: (min: 2.0, avg: 17.7, max: 34.0) [2023-12-26 21:16:41,062][104569] Avg episode reward: [(0, '9254.475'), (1, '9261.513')] [2023-12-26 21:16:41,067][105692] Updated weights for policy 0, policy_version 812665 (0.0007) [2023-12-26 21:16:41,135][105692] Updated weights for policy 0, policy_version 812675 (0.0007) [2023-12-26 21:16:41,602][105620] Updated weights for policy 1, policy_version 812556 (0.0006) [2023-12-26 21:16:41,673][105620] Updated weights for policy 1, policy_version 812566 (0.0008) [2023-12-26 21:16:41,743][105620] Updated weights for policy 1, policy_version 812576 (0.0010) [2023-12-26 21:16:41,900][105692] Updated weights for policy 0, policy_version 812685 (0.0010) [2023-12-26 21:16:41,958][105692] Updated weights for policy 0, policy_version 812695 (0.0009) [2023-12-26 21:16:42,016][105692] Updated weights for policy 0, policy_version 812705 (0.0007) [2023-12-26 21:16:42,603][105620] Updated weights for policy 1, policy_version 812586 (0.0010) [2023-12-26 21:16:42,639][105692] Updated weights for policy 0, policy_version 812715 (0.0007) [2023-12-26 21:16:42,670][105620] Updated weights for policy 1, policy_version 812596 (0.0008) [2023-12-26 21:16:42,702][105692] Updated weights for policy 0, policy_version 812725 (0.0008) [2023-12-26 21:16:42,734][105620] Updated weights for policy 1, policy_version 812606 (0.0008) [2023-12-26 21:16:42,765][105692] Updated weights for policy 0, policy_version 812735 (0.0009) [2023-12-26 21:16:42,795][105620] Updated weights for policy 1, policy_version 812616 (0.0011) [2023-12-26 21:16:43,479][105692] Updated weights for policy 0, policy_version 812745 (0.0008) [2023-12-26 21:16:43,516][105620] Updated weights for policy 1, policy_version 812626 (0.0010) [2023-12-26 21:16:43,534][105692] Updated weights for policy 0, policy_version 812755 (0.0011) [2023-12-26 21:16:43,571][105620] Updated weights for policy 1, policy_version 812636 (0.0010) [2023-12-26 21:16:43,587][105692] Updated weights for policy 0, policy_version 812765 (0.0009) [2023-12-26 21:16:43,625][105620] Updated weights for policy 1, policy_version 812646 (0.0008) [2023-12-26 21:16:43,635][105692] Updated weights for policy 0, policy_version 812775 (0.0007) [2023-12-26 21:16:44,276][105692] Updated weights for policy 0, policy_version 812785 (0.0005) [2023-12-26 21:16:44,327][105692] Updated weights for policy 0, policy_version 812795 (0.0005) [2023-12-26 21:16:44,378][105692] Updated weights for policy 0, policy_version 812805 (0.0005) [2023-12-26 21:16:44,399][105620] Updated weights for policy 1, policy_version 812656 (0.0011) [2023-12-26 21:16:44,466][105620] Updated weights for policy 1, policy_version 812666 (0.0011) [2023-12-26 21:16:44,529][105620] Updated weights for policy 1, policy_version 812676 (0.0011) [2023-12-26 21:16:45,110][105692] Updated weights for policy 0, policy_version 812815 (0.0006) [2023-12-26 21:16:45,171][105692] Updated weights for policy 0, policy_version 812825 (0.0008) [2023-12-26 21:16:45,185][105620] Updated weights for policy 1, policy_version 812686 (0.0009) [2023-12-26 21:16:45,235][105692] Updated weights for policy 0, policy_version 812835 (0.0008) [2023-12-26 21:16:45,249][105620] Updated weights for policy 1, policy_version 812696 (0.0008) [2023-12-26 21:16:45,303][105620] Updated weights for policy 1, policy_version 812706 (0.0009) [2023-12-26 21:16:45,931][105692] Updated weights for policy 0, policy_version 812845 (0.0008) [2023-12-26 21:16:45,987][105692] Updated weights for policy 0, policy_version 812855 (0.0010) [2023-12-26 21:16:46,039][105692] Updated weights for policy 0, policy_version 812865 (0.0006) [2023-12-26 21:16:46,061][105620] Updated weights for policy 1, policy_version 812716 (0.0007) [2023-12-26 21:16:46,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 416194560. Throughput: 0: 9986.6, 1: 9737.7. Samples: 416170828. Policy #0 lag: (min: 2.0, avg: 17.7, max: 34.0) [2023-12-26 21:16:46,062][104569] Avg episode reward: [(0, '6261.750'), (1, '9352.966')] [2023-12-26 21:16:46,077][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000812872_208125952.pth... [2023-12-26 21:16:46,081][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000811688_207822848.pth [2023-12-26 21:16:46,123][105620] Updated weights for policy 1, policy_version 812726 (0.0006) [2023-12-26 21:16:46,176][105620] Updated weights for policy 1, policy_version 812736 (0.0009) [2023-12-26 21:16:46,217][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000812744_208084992.pth... [2023-12-26 21:16:46,223][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000811560_207781888.pth [2023-12-26 21:16:46,642][105692] Updated weights for policy 0, policy_version 812875 (0.0009) [2023-12-26 21:16:46,690][105692] Updated weights for policy 0, policy_version 812885 (0.0005) [2023-12-26 21:16:46,744][105692] Updated weights for policy 0, policy_version 812895 (0.0005) [2023-12-26 21:16:46,926][105620] Updated weights for policy 1, policy_version 812746 (0.0008) [2023-12-26 21:16:46,983][105620] Updated weights for policy 1, policy_version 812756 (0.0010) [2023-12-26 21:16:47,039][105620] Updated weights for policy 1, policy_version 812766 (0.0010) [2023-12-26 21:16:47,098][105620] Updated weights for policy 1, policy_version 812776 (0.0011) [2023-12-26 21:16:47,357][105692] Updated weights for policy 0, policy_version 812905 (0.0006) [2023-12-26 21:16:47,414][105692] Updated weights for policy 0, policy_version 812915 (0.0010) [2023-12-26 21:16:47,469][105692] Updated weights for policy 0, policy_version 812925 (0.0010) [2023-12-26 21:16:47,529][105692] Updated weights for policy 0, policy_version 812935 (0.0010) [2023-12-26 21:16:47,863][105620] Updated weights for policy 1, policy_version 812786 (0.0010) [2023-12-26 21:16:47,925][105620] Updated weights for policy 1, policy_version 812796 (0.0010) [2023-12-26 21:16:47,993][105620] Updated weights for policy 1, policy_version 812806 (0.0009) [2023-12-26 21:16:48,198][105692] Updated weights for policy 0, policy_version 812945 (0.0007) [2023-12-26 21:16:48,261][105692] Updated weights for policy 0, policy_version 812955 (0.0007) [2023-12-26 21:16:48,319][105692] Updated weights for policy 0, policy_version 812965 (0.0010) [2023-12-26 21:16:48,769][105620] Updated weights for policy 1, policy_version 812816 (0.0010) [2023-12-26 21:16:48,828][105620] Updated weights for policy 1, policy_version 812826 (0.0010) [2023-12-26 21:16:48,886][105620] Updated weights for policy 1, policy_version 812836 (0.0010) [2023-12-26 21:16:49,000][105692] Updated weights for policy 0, policy_version 812975 (0.0010) [2023-12-26 21:16:49,052][105692] Updated weights for policy 0, policy_version 812985 (0.0010) [2023-12-26 21:16:49,102][105692] Updated weights for policy 0, policy_version 812995 (0.0010) [2023-12-26 21:16:49,654][105620] Updated weights for policy 1, policy_version 812846 (0.0010) [2023-12-26 21:16:49,712][105620] Updated weights for policy 1, policy_version 812856 (0.0010) [2023-12-26 21:16:49,774][105620] Updated weights for policy 1, policy_version 812866 (0.0010) [2023-12-26 21:16:49,885][105692] Updated weights for policy 0, policy_version 813005 (0.0008) [2023-12-26 21:16:49,947][105692] Updated weights for policy 0, policy_version 813015 (0.0007) [2023-12-26 21:16:49,999][105692] Updated weights for policy 0, policy_version 813025 (0.0009) [2023-12-26 21:16:50,438][105620] Updated weights for policy 1, policy_version 812876 (0.0011) [2023-12-26 21:16:50,488][105620] Updated weights for policy 1, policy_version 812886 (0.0008) [2023-12-26 21:16:50,541][105620] Updated weights for policy 1, policy_version 812896 (0.0010) [2023-12-26 21:16:50,632][105692] Updated weights for policy 0, policy_version 813035 (0.0008) [2023-12-26 21:16:50,700][105692] Updated weights for policy 0, policy_version 813045 (0.0009) [2023-12-26 21:16:50,767][105692] Updated weights for policy 0, policy_version 813055 (0.0005) [2023-12-26 21:16:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 416301056. Throughput: 0: 10124.8, 1: 9726.8. Samples: 416288380. Policy #0 lag: (min: 2.0, avg: 17.7, max: 34.0) [2023-12-26 21:16:51,062][104569] Avg episode reward: [(0, '6528.240'), (1, '9261.799')] [2023-12-26 21:16:51,303][105620] Updated weights for policy 1, policy_version 812906 (0.0008) [2023-12-26 21:16:51,361][105620] Updated weights for policy 1, policy_version 812916 (0.0011) [2023-12-26 21:16:51,429][105620] Updated weights for policy 1, policy_version 812926 (0.0009) [2023-12-26 21:16:51,444][105692] Updated weights for policy 0, policy_version 813065 (0.0006) [2023-12-26 21:16:51,488][105620] Updated weights for policy 1, policy_version 812936 (0.0011) [2023-12-26 21:16:51,494][105692] Updated weights for policy 0, policy_version 813075 (0.0008) [2023-12-26 21:16:51,557][105692] Updated weights for policy 0, policy_version 813085 (0.0005) [2023-12-26 21:16:51,623][105692] Updated weights for policy 0, policy_version 813095 (0.0006) [2023-12-26 21:16:52,254][105620] Updated weights for policy 1, policy_version 812946 (0.0011) [2023-12-26 21:16:52,310][105620] Updated weights for policy 1, policy_version 812956 (0.0011) [2023-12-26 21:16:52,363][105692] Updated weights for policy 0, policy_version 813105 (0.0007) [2023-12-26 21:16:52,371][105620] Updated weights for policy 1, policy_version 812966 (0.0011) [2023-12-26 21:16:52,424][105692] Updated weights for policy 0, policy_version 813115 (0.0009) [2023-12-26 21:16:52,472][105692] Updated weights for policy 0, policy_version 813125 (0.0008) [2023-12-26 21:16:53,124][105620] Updated weights for policy 1, policy_version 812976 (0.0011) [2023-12-26 21:16:53,151][105692] Updated weights for policy 0, policy_version 813135 (0.0006) [2023-12-26 21:16:53,172][105620] Updated weights for policy 1, policy_version 812986 (0.0010) [2023-12-26 21:16:53,208][105692] Updated weights for policy 0, policy_version 813145 (0.0005) [2023-12-26 21:16:53,235][105620] Updated weights for policy 1, policy_version 812996 (0.0010) [2023-12-26 21:16:53,265][105692] Updated weights for policy 0, policy_version 813155 (0.0010) [2023-12-26 21:16:53,947][105692] Updated weights for policy 0, policy_version 813165 (0.0010) [2023-12-26 21:16:53,958][105620] Updated weights for policy 1, policy_version 813006 (0.0007) [2023-12-26 21:16:53,999][105692] Updated weights for policy 0, policy_version 813175 (0.0008) [2023-12-26 21:16:54,012][105620] Updated weights for policy 1, policy_version 813016 (0.0007) [2023-12-26 21:16:54,055][105692] Updated weights for policy 0, policy_version 813185 (0.0007) [2023-12-26 21:16:54,069][105620] Updated weights for policy 1, policy_version 813026 (0.0011) [2023-12-26 21:16:54,788][105692] Updated weights for policy 0, policy_version 813195 (0.0010) [2023-12-26 21:16:54,809][105620] Updated weights for policy 1, policy_version 813036 (0.0011) [2023-12-26 21:16:54,846][105692] Updated weights for policy 0, policy_version 813205 (0.0010) [2023-12-26 21:16:54,868][105620] Updated weights for policy 1, policy_version 813046 (0.0011) [2023-12-26 21:16:54,901][105692] Updated weights for policy 0, policy_version 813215 (0.0010) [2023-12-26 21:16:54,924][105620] Updated weights for policy 1, policy_version 813056 (0.0010) [2023-12-26 21:16:55,613][105692] Updated weights for policy 0, policy_version 813225 (0.0010) [2023-12-26 21:16:55,629][105620] Updated weights for policy 1, policy_version 813066 (0.0010) [2023-12-26 21:16:55,661][105692] Updated weights for policy 0, policy_version 813235 (0.0010) [2023-12-26 21:16:55,689][105620] Updated weights for policy 1, policy_version 813076 (0.0005) [2023-12-26 21:16:55,709][105692] Updated weights for policy 0, policy_version 813245 (0.0010) [2023-12-26 21:16:55,736][105620] Updated weights for policy 1, policy_version 813086 (0.0005) [2023-12-26 21:16:55,765][105692] Updated weights for policy 0, policy_version 813255 (0.0009) [2023-12-26 21:16:55,789][105620] Updated weights for policy 1, policy_version 813096 (0.0006) [2023-12-26 21:16:56,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.9, 300 sec: 19521.9). Total num frames: 416399360. Throughput: 0: 10110.5, 1: 9862.2. Samples: 416406396. Policy #0 lag: (min: 2.0, avg: 17.7, max: 34.0) [2023-12-26 21:16:56,063][104569] Avg episode reward: [(0, '8275.928'), (1, '9261.932')] [2023-12-26 21:16:56,321][105620] Updated weights for policy 1, policy_version 813106 (0.0011) [2023-12-26 21:16:56,368][105620] Updated weights for policy 1, policy_version 813116 (0.0010) [2023-12-26 21:16:56,406][105692] Updated weights for policy 0, policy_version 813265 (0.0006) [2023-12-26 21:16:56,421][105620] Updated weights for policy 1, policy_version 813126 (0.0010) [2023-12-26 21:16:56,454][105692] Updated weights for policy 0, policy_version 813275 (0.0007) [2023-12-26 21:16:56,502][105692] Updated weights for policy 0, policy_version 813285 (0.0010) [2023-12-26 21:16:57,076][105620] Updated weights for policy 1, policy_version 813136 (0.0006) [2023-12-26 21:16:57,125][105620] Updated weights for policy 1, policy_version 813146 (0.0005) [2023-12-26 21:16:57,179][105692] Updated weights for policy 0, policy_version 813295 (0.0010) [2023-12-26 21:16:57,185][105620] Updated weights for policy 1, policy_version 813156 (0.0006) [2023-12-26 21:16:57,228][105692] Updated weights for policy 0, policy_version 813305 (0.0010) [2023-12-26 21:16:57,284][105692] Updated weights for policy 0, policy_version 813315 (0.0010) [2023-12-26 21:16:57,761][105620] Updated weights for policy 1, policy_version 813166 (0.0007) [2023-12-26 21:16:57,820][105620] Updated weights for policy 1, policy_version 813176 (0.0008) [2023-12-26 21:16:57,872][105620] Updated weights for policy 1, policy_version 813186 (0.0008) [2023-12-26 21:16:58,028][105692] Updated weights for policy 0, policy_version 813325 (0.0010) [2023-12-26 21:16:58,082][105692] Updated weights for policy 0, policy_version 813335 (0.0010) [2023-12-26 21:16:58,145][105692] Updated weights for policy 0, policy_version 813345 (0.0010) [2023-12-26 21:16:58,631][105620] Updated weights for policy 1, policy_version 813196 (0.0008) [2023-12-26 21:16:58,693][105620] Updated weights for policy 1, policy_version 813206 (0.0008) [2023-12-26 21:16:58,758][105620] Updated weights for policy 1, policy_version 813216 (0.0008) [2023-12-26 21:16:58,951][105692] Updated weights for policy 0, policy_version 813355 (0.0010) [2023-12-26 21:16:59,017][105692] Updated weights for policy 0, policy_version 813365 (0.0010) [2023-12-26 21:16:59,073][105692] Updated weights for policy 0, policy_version 813375 (0.0008) [2023-12-26 21:16:59,516][105620] Updated weights for policy 1, policy_version 813226 (0.0008) [2023-12-26 21:16:59,582][105620] Updated weights for policy 1, policy_version 813236 (0.0008) [2023-12-26 21:16:59,642][105620] Updated weights for policy 1, policy_version 813246 (0.0008) [2023-12-26 21:16:59,709][105620] Updated weights for policy 1, policy_version 813256 (0.0008) [2023-12-26 21:16:59,789][105692] Updated weights for policy 0, policy_version 813385 (0.0010) [2023-12-26 21:16:59,850][105692] Updated weights for policy 0, policy_version 813395 (0.0011) [2023-12-26 21:16:59,916][105692] Updated weights for policy 0, policy_version 813405 (0.0011) [2023-12-26 21:16:59,985][105692] Updated weights for policy 0, policy_version 813415 (0.0008) [2023-12-26 21:17:00,471][105620] Updated weights for policy 1, policy_version 813266 (0.0010) [2023-12-26 21:17:00,522][105620] Updated weights for policy 1, policy_version 813276 (0.0010) [2023-12-26 21:17:00,576][105620] Updated weights for policy 1, policy_version 813286 (0.0010) [2023-12-26 21:17:00,745][105692] Updated weights for policy 0, policy_version 813425 (0.0008) [2023-12-26 21:17:00,792][105692] Updated weights for policy 0, policy_version 813435 (0.0005) [2023-12-26 21:17:00,842][105692] Updated weights for policy 0, policy_version 813445 (0.0005) [2023-12-26 21:17:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19521.9). Total num frames: 416497664. Throughput: 0: 10103.6, 1: 9901.6. Samples: 416467680. Policy #0 lag: (min: 2.0, avg: 17.7, max: 34.0) [2023-12-26 21:17:01,063][104569] Avg episode reward: [(0, '8635.052'), (1, '9293.265')] [2023-12-26 21:17:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000813448_208273408.pth... [2023-12-26 21:17:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000813288_208224256.pth... [2023-12-26 21:17:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000812168_207937536.pth [2023-12-26 21:17:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000812264_207970304.pth [2023-12-26 21:17:01,316][105620] Updated weights for policy 1, policy_version 813296 (0.0006) [2023-12-26 21:17:01,386][105620] Updated weights for policy 1, policy_version 813306 (0.0008) [2023-12-26 21:17:01,449][105620] Updated weights for policy 1, policy_version 813316 (0.0010) [2023-12-26 21:17:01,528][105692] Updated weights for policy 0, policy_version 813455 (0.0007) [2023-12-26 21:17:01,591][105692] Updated weights for policy 0, policy_version 813465 (0.0005) [2023-12-26 21:17:01,656][105692] Updated weights for policy 0, policy_version 813475 (0.0007) [2023-12-26 21:17:02,058][105620] Updated weights for policy 1, policy_version 813326 (0.0010) [2023-12-26 21:17:02,123][105620] Updated weights for policy 1, policy_version 813336 (0.0010) [2023-12-26 21:17:02,181][105620] Updated weights for policy 1, policy_version 813346 (0.0010) [2023-12-26 21:17:02,280][105692] Updated weights for policy 0, policy_version 813485 (0.0006) [2023-12-26 21:17:02,342][105692] Updated weights for policy 0, policy_version 813495 (0.0008) [2023-12-26 21:17:02,397][105692] Updated weights for policy 0, policy_version 813505 (0.0009) [2023-12-26 21:17:02,854][105620] Updated weights for policy 1, policy_version 813356 (0.0008) [2023-12-26 21:17:02,898][105620] Updated weights for policy 1, policy_version 813366 (0.0005) [2023-12-26 21:17:02,952][105620] Updated weights for policy 1, policy_version 813376 (0.0005) [2023-12-26 21:17:03,123][105692] Updated weights for policy 0, policy_version 813515 (0.0007) [2023-12-26 21:17:03,181][105692] Updated weights for policy 0, policy_version 813525 (0.0005) [2023-12-26 21:17:03,246][105692] Updated weights for policy 0, policy_version 813535 (0.0008) [2023-12-26 21:17:03,494][105620] Updated weights for policy 1, policy_version 813386 (0.0005) [2023-12-26 21:17:03,548][105620] Updated weights for policy 1, policy_version 813396 (0.0005) [2023-12-26 21:17:03,594][105620] Updated weights for policy 1, policy_version 813406 (0.0005) [2023-12-26 21:17:03,645][105620] Updated weights for policy 1, policy_version 813416 (0.0005) [2023-12-26 21:17:03,770][105692] Updated weights for policy 0, policy_version 813545 (0.0005) [2023-12-26 21:17:03,855][105692] Updated weights for policy 0, policy_version 813555 (0.0006) [2023-12-26 21:17:03,915][105692] Updated weights for policy 0, policy_version 813565 (0.0008) [2023-12-26 21:17:03,973][105692] Updated weights for policy 0, policy_version 813575 (0.0008) [2023-12-26 21:17:04,281][105620] Updated weights for policy 1, policy_version 813426 (0.0011) [2023-12-26 21:17:04,332][105620] Updated weights for policy 1, policy_version 813436 (0.0008) [2023-12-26 21:17:04,384][105620] Updated weights for policy 1, policy_version 813446 (0.0010) [2023-12-26 21:17:04,595][105692] Updated weights for policy 0, policy_version 813585 (0.0009) [2023-12-26 21:17:04,648][105692] Updated weights for policy 0, policy_version 813595 (0.0005) [2023-12-26 21:17:04,698][105692] Updated weights for policy 0, policy_version 813605 (0.0005) [2023-12-26 21:17:05,084][105620] Updated weights for policy 1, policy_version 813456 (0.0006) [2023-12-26 21:17:05,136][105620] Updated weights for policy 1, policy_version 813466 (0.0007) [2023-12-26 21:17:05,191][105620] Updated weights for policy 1, policy_version 813476 (0.0011) [2023-12-26 21:17:05,298][105692] Updated weights for policy 0, policy_version 813615 (0.0006) [2023-12-26 21:17:05,349][105692] Updated weights for policy 0, policy_version 813625 (0.0007) [2023-12-26 21:17:05,410][105692] Updated weights for policy 0, policy_version 813635 (0.0005) [2023-12-26 21:17:05,928][105620] Updated weights for policy 1, policy_version 813486 (0.0010) [2023-12-26 21:17:05,933][105692] Updated weights for policy 0, policy_version 813645 (0.0005) [2023-12-26 21:17:05,986][105620] Updated weights for policy 1, policy_version 813496 (0.0010) [2023-12-26 21:17:05,997][105692] Updated weights for policy 0, policy_version 813655 (0.0005) [2023-12-26 21:17:06,035][105620] Updated weights for policy 1, policy_version 813506 (0.0010) [2023-12-26 21:17:06,058][105692] Updated weights for policy 0, policy_version 813665 (0.0009) [2023-12-26 21:17:06,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19933.9, 300 sec: 19522.0). Total num frames: 416595968. Throughput: 0: 10050.7, 1: 9872.7. Samples: 416589384. Policy #0 lag: (min: 2.0, avg: 17.7, max: 34.0) [2023-12-26 21:17:06,062][104569] Avg episode reward: [(0, '9079.020'), (1, '9293.313')] [2023-12-26 21:17:06,687][105692] Updated weights for policy 0, policy_version 813675 (0.0006) [2023-12-26 21:17:06,746][105692] Updated weights for policy 0, policy_version 813685 (0.0006) [2023-12-26 21:17:06,807][105692] Updated weights for policy 0, policy_version 813695 (0.0010) [2023-12-26 21:17:06,814][105620] Updated weights for policy 1, policy_version 813516 (0.0011) [2023-12-26 21:17:06,873][105620] Updated weights for policy 1, policy_version 813526 (0.0010) [2023-12-26 21:17:06,925][105620] Updated weights for policy 1, policy_version 813536 (0.0010) [2023-12-26 21:17:07,385][105692] Updated weights for policy 0, policy_version 813705 (0.0011) [2023-12-26 21:17:07,443][105692] Updated weights for policy 0, policy_version 813715 (0.0010) [2023-12-26 21:17:07,505][105692] Updated weights for policy 0, policy_version 813725 (0.0011) [2023-12-26 21:17:07,560][105692] Updated weights for policy 0, policy_version 813735 (0.0010) [2023-12-26 21:17:07,561][105620] Updated weights for policy 1, policy_version 813546 (0.0009) [2023-12-26 21:17:07,615][105620] Updated weights for policy 1, policy_version 813556 (0.0007) [2023-12-26 21:17:07,678][105620] Updated weights for policy 1, policy_version 813566 (0.0006) [2023-12-26 21:17:07,734][105620] Updated weights for policy 1, policy_version 813576 (0.0008) [2023-12-26 21:17:08,299][105620] Updated weights for policy 1, policy_version 813586 (0.0005) [2023-12-26 21:17:08,304][105692] Updated weights for policy 0, policy_version 813745 (0.0010) [2023-12-26 21:17:08,360][105620] Updated weights for policy 1, policy_version 813596 (0.0008) [2023-12-26 21:17:08,361][105692] Updated weights for policy 0, policy_version 813755 (0.0009) [2023-12-26 21:17:08,418][105692] Updated weights for policy 0, policy_version 813765 (0.0006) [2023-12-26 21:17:08,422][105620] Updated weights for policy 1, policy_version 813606 (0.0009) [2023-12-26 21:17:08,953][105620] Updated weights for policy 1, policy_version 813616 (0.0006) [2023-12-26 21:17:09,011][105620] Updated weights for policy 1, policy_version 813626 (0.0005) [2023-12-26 21:17:09,068][105620] Updated weights for policy 1, policy_version 813636 (0.0006) [2023-12-26 21:17:09,128][105692] Updated weights for policy 0, policy_version 813775 (0.0007) [2023-12-26 21:17:09,190][105692] Updated weights for policy 0, policy_version 813785 (0.0010) [2023-12-26 21:17:09,250][105692] Updated weights for policy 0, policy_version 813795 (0.0009) [2023-12-26 21:17:09,749][105620] Updated weights for policy 1, policy_version 813646 (0.0008) [2023-12-26 21:17:09,806][105620] Updated weights for policy 1, policy_version 813656 (0.0011) [2023-12-26 21:17:09,871][105620] Updated weights for policy 1, policy_version 813666 (0.0011) [2023-12-26 21:17:10,043][105692] Updated weights for policy 0, policy_version 813805 (0.0008) [2023-12-26 21:17:10,097][105692] Updated weights for policy 0, policy_version 813815 (0.0009) [2023-12-26 21:17:10,148][105692] Updated weights for policy 0, policy_version 813825 (0.0009) [2023-12-26 21:17:10,542][105620] Updated weights for policy 1, policy_version 813676 (0.0009) [2023-12-26 21:17:10,605][105620] Updated weights for policy 1, policy_version 813686 (0.0008) [2023-12-26 21:17:10,672][105620] Updated weights for policy 1, policy_version 813696 (0.0008) [2023-12-26 21:17:10,852][105692] Updated weights for policy 0, policy_version 813835 (0.0009) [2023-12-26 21:17:10,900][105692] Updated weights for policy 0, policy_version 813845 (0.0008) [2023-12-26 21:17:10,953][105692] Updated weights for policy 0, policy_version 813855 (0.0005) [2023-12-26 21:17:11,062][104569] Fps is (10 sec: 21299.2, 60 sec: 20206.9, 300 sec: 19605.3). Total num frames: 416710656. Throughput: 0: 10226.8, 1: 9859.7. Samples: 416714652. Policy #0 lag: (min: 2.0, avg: 17.7, max: 34.0) [2023-12-26 21:17:11,063][104569] Avg episode reward: [(0, '9170.051'), (1, '9116.933')] [2023-12-26 21:17:11,403][105620] Updated weights for policy 1, policy_version 813706 (0.0007) [2023-12-26 21:17:11,461][105620] Updated weights for policy 1, policy_version 813716 (0.0009) [2023-12-26 21:17:11,516][105620] Updated weights for policy 1, policy_version 813726 (0.0009) [2023-12-26 21:17:11,573][105620] Updated weights for policy 1, policy_version 813736 (0.0009) [2023-12-26 21:17:11,783][105692] Updated weights for policy 0, policy_version 813865 (0.0008) [2023-12-26 21:17:11,845][105692] Updated weights for policy 0, policy_version 813875 (0.0009) [2023-12-26 21:17:11,904][105692] Updated weights for policy 0, policy_version 813885 (0.0009) [2023-12-26 21:17:11,969][105692] Updated weights for policy 0, policy_version 813895 (0.0008) [2023-12-26 21:17:12,361][105620] Updated weights for policy 1, policy_version 813746 (0.0008) [2023-12-26 21:17:12,423][105620] Updated weights for policy 1, policy_version 813756 (0.0009) [2023-12-26 21:17:12,489][105620] Updated weights for policy 1, policy_version 813766 (0.0010) [2023-12-26 21:17:12,670][105692] Updated weights for policy 0, policy_version 813905 (0.0006) [2023-12-26 21:17:12,723][105692] Updated weights for policy 0, policy_version 813915 (0.0007) [2023-12-26 21:17:12,775][105692] Updated weights for policy 0, policy_version 813925 (0.0011) [2023-12-26 21:17:13,236][105620] Updated weights for policy 1, policy_version 813776 (0.0007) [2023-12-26 21:17:13,295][105620] Updated weights for policy 1, policy_version 813786 (0.0008) [2023-12-26 21:17:13,353][105620] Updated weights for policy 1, policy_version 813796 (0.0008) [2023-12-26 21:17:13,519][105692] Updated weights for policy 0, policy_version 813935 (0.0011) [2023-12-26 21:17:13,577][105692] Updated weights for policy 0, policy_version 813945 (0.0010) [2023-12-26 21:17:13,625][105692] Updated weights for policy 0, policy_version 813955 (0.0010) [2023-12-26 21:17:14,003][105620] Updated weights for policy 1, policy_version 813806 (0.0007) [2023-12-26 21:17:14,066][105620] Updated weights for policy 1, policy_version 813816 (0.0007) [2023-12-26 21:17:14,120][105620] Updated weights for policy 1, policy_version 813826 (0.0005) [2023-12-26 21:17:14,369][105692] Updated weights for policy 0, policy_version 813965 (0.0010) [2023-12-26 21:17:14,434][105692] Updated weights for policy 0, policy_version 813975 (0.0006) [2023-12-26 21:17:14,491][105692] Updated weights for policy 0, policy_version 813985 (0.0005) [2023-12-26 21:17:14,664][105620] Updated weights for policy 1, policy_version 813836 (0.0006) [2023-12-26 21:17:14,731][105620] Updated weights for policy 1, policy_version 813846 (0.0005) [2023-12-26 21:17:14,787][105620] Updated weights for policy 1, policy_version 813856 (0.0007) [2023-12-26 21:17:15,159][105692] Updated weights for policy 0, policy_version 813995 (0.0007) [2023-12-26 21:17:15,219][105692] Updated weights for policy 0, policy_version 814005 (0.0011) [2023-12-26 21:17:15,274][105692] Updated weights for policy 0, policy_version 814015 (0.0011) [2023-12-26 21:17:15,492][105620] Updated weights for policy 1, policy_version 813866 (0.0007) [2023-12-26 21:17:15,558][105620] Updated weights for policy 1, policy_version 813876 (0.0008) [2023-12-26 21:17:15,620][105620] Updated weights for policy 1, policy_version 813886 (0.0008) [2023-12-26 21:17:15,677][105620] Updated weights for policy 1, policy_version 813896 (0.0008) [2023-12-26 21:17:16,033][105692] Updated weights for policy 0, policy_version 814025 (0.0010) [2023-12-26 21:17:16,062][104569] Fps is (10 sec: 20479.9, 60 sec: 20070.4, 300 sec: 19577.5). Total num frames: 416800768. Throughput: 0: 10063.0, 1: 9811.4. Samples: 416771788. Policy #0 lag: (min: 2.0, avg: 17.7, max: 34.0) [2023-12-26 21:17:16,062][104569] Avg episode reward: [(0, '8133.125'), (1, '9024.623')] [2023-12-26 21:17:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000813896_208379904.pth... [2023-12-26 21:17:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000812744_208084992.pth [2023-12-26 21:17:16,087][105692] Updated weights for policy 0, policy_version 814035 (0.0005) [2023-12-26 21:17:16,150][105692] Updated weights for policy 0, policy_version 814045 (0.0006) [2023-12-26 21:17:16,212][105692] Updated weights for policy 0, policy_version 814055 (0.0009) [2023-12-26 21:17:16,217][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000814056_208429056.pth... [2023-12-26 21:17:16,221][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000812872_208125952.pth [2023-12-26 21:17:16,409][105620] Updated weights for policy 1, policy_version 813906 (0.0005) [2023-12-26 21:17:16,455][105620] Updated weights for policy 1, policy_version 813916 (0.0005) [2023-12-26 21:17:16,499][105620] Updated weights for policy 1, policy_version 813926 (0.0005) [2023-12-26 21:17:16,937][105692] Updated weights for policy 0, policy_version 814065 (0.0008) [2023-12-26 21:17:17,000][105692] Updated weights for policy 0, policy_version 814075 (0.0008) [2023-12-26 21:17:17,064][105692] Updated weights for policy 0, policy_version 814085 (0.0008) [2023-12-26 21:17:17,225][105620] Updated weights for policy 1, policy_version 813936 (0.0009) [2023-12-26 21:17:17,292][105620] Updated weights for policy 1, policy_version 813946 (0.0006) [2023-12-26 21:17:17,352][105620] Updated weights for policy 1, policy_version 813956 (0.0008) [2023-12-26 21:17:17,782][105692] Updated weights for policy 0, policy_version 814095 (0.0009) [2023-12-26 21:17:17,840][105692] Updated weights for policy 0, policy_version 814105 (0.0009) [2023-12-26 21:17:17,894][105692] Updated weights for policy 0, policy_version 814115 (0.0009) [2023-12-26 21:17:18,076][105620] Updated weights for policy 1, policy_version 813966 (0.0009) [2023-12-26 21:17:18,130][105620] Updated weights for policy 1, policy_version 813976 (0.0009) [2023-12-26 21:17:18,176][105620] Updated weights for policy 1, policy_version 813986 (0.0008) [2023-12-26 21:17:18,677][105692] Updated weights for policy 0, policy_version 814125 (0.0008) [2023-12-26 21:17:18,726][105692] Updated weights for policy 0, policy_version 814135 (0.0008) [2023-12-26 21:17:18,785][105692] Updated weights for policy 0, policy_version 814145 (0.0008) [2023-12-26 21:17:18,911][105620] Updated weights for policy 1, policy_version 813996 (0.0009) [2023-12-26 21:17:18,965][105620] Updated weights for policy 1, policy_version 814006 (0.0009) [2023-12-26 21:17:19,024][105620] Updated weights for policy 1, policy_version 814016 (0.0010) [2023-12-26 21:17:19,621][105692] Updated weights for policy 0, policy_version 814155 (0.0008) [2023-12-26 21:17:19,677][105692] Updated weights for policy 0, policy_version 814165 (0.0010) [2023-12-26 21:17:19,722][105620] Updated weights for policy 1, policy_version 814026 (0.0010) [2023-12-26 21:17:19,737][105692] Updated weights for policy 0, policy_version 814175 (0.0008) [2023-12-26 21:17:19,776][105620] Updated weights for policy 1, policy_version 814036 (0.0007) [2023-12-26 21:17:19,842][105620] Updated weights for policy 1, policy_version 814046 (0.0008) [2023-12-26 21:17:19,907][105620] Updated weights for policy 1, policy_version 814056 (0.0008) [2023-12-26 21:17:20,517][105692] Updated weights for policy 0, policy_version 814185 (0.0007) [2023-12-26 21:17:20,575][105692] Updated weights for policy 0, policy_version 814195 (0.0009) [2023-12-26 21:17:20,636][105692] Updated weights for policy 0, policy_version 814205 (0.0009) [2023-12-26 21:17:20,688][105620] Updated weights for policy 1, policy_version 814066 (0.0009) [2023-12-26 21:17:20,697][105692] Updated weights for policy 0, policy_version 814215 (0.0007) [2023-12-26 21:17:20,750][105620] Updated weights for policy 1, policy_version 814076 (0.0008) [2023-12-26 21:17:20,812][105620] Updated weights for policy 1, policy_version 814086 (0.0009) [2023-12-26 21:17:21,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 416899072. Throughput: 0: 9993.0, 1: 9803.5. Samples: 416887072. Policy #0 lag: (min: 2.0, avg: 17.7, max: 34.0) [2023-12-26 21:17:21,062][104569] Avg episode reward: [(0, '7381.195'), (1, '9151.311')] [2023-12-26 21:17:21,492][105692] Updated weights for policy 0, policy_version 814225 (0.0006) [2023-12-26 21:17:21,545][105692] Updated weights for policy 0, policy_version 814235 (0.0007) [2023-12-26 21:17:21,606][105692] Updated weights for policy 0, policy_version 814245 (0.0008) [2023-12-26 21:17:21,648][105620] Updated weights for policy 1, policy_version 814096 (0.0009) [2023-12-26 21:17:21,717][105620] Updated weights for policy 1, policy_version 814106 (0.0008) [2023-12-26 21:17:21,784][105620] Updated weights for policy 1, policy_version 814116 (0.0009) [2023-12-26 21:17:22,435][105692] Updated weights for policy 0, policy_version 814255 (0.0009) [2023-12-26 21:17:22,484][105692] Updated weights for policy 0, policy_version 814265 (0.0008) [2023-12-26 21:17:22,498][105620] Updated weights for policy 1, policy_version 814126 (0.0008) [2023-12-26 21:17:22,544][105620] Updated weights for policy 1, policy_version 814136 (0.0006) [2023-12-26 21:17:22,545][105692] Updated weights for policy 0, policy_version 814275 (0.0009) [2023-12-26 21:17:22,592][105620] Updated weights for policy 1, policy_version 814146 (0.0009) [2023-12-26 21:17:23,317][105692] Updated weights for policy 0, policy_version 814285 (0.0009) [2023-12-26 21:17:23,364][105620] Updated weights for policy 1, policy_version 814156 (0.0009) [2023-12-26 21:17:23,371][105692] Updated weights for policy 0, policy_version 814295 (0.0008) [2023-12-26 21:17:23,410][105620] Updated weights for policy 1, policy_version 814166 (0.0006) [2023-12-26 21:17:23,430][105692] Updated weights for policy 0, policy_version 814305 (0.0008) [2023-12-26 21:17:23,455][105620] Updated weights for policy 1, policy_version 814176 (0.0008) [2023-12-26 21:17:24,153][105692] Updated weights for policy 0, policy_version 814315 (0.0008) [2023-12-26 21:17:24,211][105692] Updated weights for policy 0, policy_version 814325 (0.0009) [2023-12-26 21:17:24,242][105620] Updated weights for policy 1, policy_version 814186 (0.0009) [2023-12-26 21:17:24,262][105692] Updated weights for policy 0, policy_version 814335 (0.0009) [2023-12-26 21:17:24,295][105620] Updated weights for policy 1, policy_version 814196 (0.0008) [2023-12-26 21:17:24,356][105620] Updated weights for policy 1, policy_version 814206 (0.0008) [2023-12-26 21:17:24,416][105620] Updated weights for policy 1, policy_version 814216 (0.0009) [2023-12-26 21:17:24,875][105692] Updated weights for policy 0, policy_version 814345 (0.0007) [2023-12-26 21:17:24,933][105692] Updated weights for policy 0, policy_version 814355 (0.0005) [2023-12-26 21:17:24,986][105692] Updated weights for policy 0, policy_version 814365 (0.0005) [2023-12-26 21:17:25,037][105692] Updated weights for policy 0, policy_version 814375 (0.0005) [2023-12-26 21:17:25,292][105620] Updated weights for policy 1, policy_version 814226 (0.0010) [2023-12-26 21:17:25,347][105620] Updated weights for policy 1, policy_version 814236 (0.0010) [2023-12-26 21:17:25,408][105620] Updated weights for policy 1, policy_version 814246 (0.0009) [2023-12-26 21:17:25,572][105692] Updated weights for policy 0, policy_version 814385 (0.0006) [2023-12-26 21:17:25,639][105692] Updated weights for policy 0, policy_version 814395 (0.0005) [2023-12-26 21:17:25,693][105692] Updated weights for policy 0, policy_version 814405 (0.0005) [2023-12-26 21:17:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 416989184. Throughput: 0: 9932.9, 1: 9730.5. Samples: 416999460. Policy #0 lag: (min: 2.0, avg: 17.7, max: 34.0) [2023-12-26 21:17:26,063][104569] Avg episode reward: [(0, '7318.634'), (1, '9242.416')] [2023-12-26 21:17:26,191][105620] Updated weights for policy 1, policy_version 814256 (0.0009) [2023-12-26 21:17:26,257][105620] Updated weights for policy 1, policy_version 814266 (0.0008) [2023-12-26 21:17:26,262][105692] Updated weights for policy 0, policy_version 814415 (0.0007) [2023-12-26 21:17:26,316][105692] Updated weights for policy 0, policy_version 814425 (0.0007) [2023-12-26 21:17:26,318][105620] Updated weights for policy 1, policy_version 814276 (0.0007) [2023-12-26 21:17:26,376][105692] Updated weights for policy 0, policy_version 814435 (0.0006) [2023-12-26 21:17:26,990][105692] Updated weights for policy 0, policy_version 814445 (0.0007) [2023-12-26 21:17:27,038][105692] Updated weights for policy 0, policy_version 814455 (0.0005) [2023-12-26 21:17:27,089][105692] Updated weights for policy 0, policy_version 814465 (0.0009) [2023-12-26 21:17:27,140][105620] Updated weights for policy 1, policy_version 814286 (0.0008) [2023-12-26 21:17:27,195][105620] Updated weights for policy 1, policy_version 814296 (0.0009) [2023-12-26 21:17:27,243][105620] Updated weights for policy 1, policy_version 814306 (0.0009) [2023-12-26 21:17:27,730][105692] Updated weights for policy 0, policy_version 814475 (0.0010) [2023-12-26 21:17:27,794][105692] Updated weights for policy 0, policy_version 814485 (0.0007) [2023-12-26 21:17:27,860][105692] Updated weights for policy 0, policy_version 814495 (0.0006) [2023-12-26 21:17:28,050][105620] Updated weights for policy 1, policy_version 814316 (0.0007) [2023-12-26 21:17:28,107][105620] Updated weights for policy 1, policy_version 814326 (0.0009) [2023-12-26 21:17:28,168][105620] Updated weights for policy 1, policy_version 814336 (0.0009) [2023-12-26 21:17:28,538][105692] Updated weights for policy 0, policy_version 814505 (0.0008) [2023-12-26 21:17:28,586][105692] Updated weights for policy 0, policy_version 814515 (0.0008) [2023-12-26 21:17:28,633][105692] Updated weights for policy 0, policy_version 814525 (0.0009) [2023-12-26 21:17:28,691][105692] Updated weights for policy 0, policy_version 814535 (0.0009) [2023-12-26 21:17:28,940][105620] Updated weights for policy 1, policy_version 814346 (0.0008) [2023-12-26 21:17:28,993][105620] Updated weights for policy 1, policy_version 814357 (0.0010) [2023-12-26 21:17:29,045][105620] Updated weights for policy 1, policy_version 814367 (0.0009) [2023-12-26 21:17:29,384][105692] Updated weights for policy 0, policy_version 814545 (0.0007) [2023-12-26 21:17:29,439][105692] Updated weights for policy 0, policy_version 814555 (0.0006) [2023-12-26 21:17:29,494][105692] Updated weights for policy 0, policy_version 814565 (0.0005) [2023-12-26 21:17:29,906][105620] Updated weights for policy 1, policy_version 814377 (0.0009) [2023-12-26 21:17:29,969][105620] Updated weights for policy 1, policy_version 814387 (0.0008) [2023-12-26 21:17:30,028][105620] Updated weights for policy 1, policy_version 814397 (0.0008) [2023-12-26 21:17:30,091][105620] Updated weights for policy 1, policy_version 814407 (0.0008) [2023-12-26 21:17:30,134][105692] Updated weights for policy 0, policy_version 814575 (0.0009) [2023-12-26 21:17:30,194][105692] Updated weights for policy 0, policy_version 814585 (0.0011) [2023-12-26 21:17:30,257][105692] Updated weights for policy 0, policy_version 814595 (0.0010) [2023-12-26 21:17:30,856][105620] Updated weights for policy 1, policy_version 814417 (0.0010) [2023-12-26 21:17:30,908][105620] Updated weights for policy 1, policy_version 814428 (0.0010) [2023-12-26 21:17:30,935][105692] Updated weights for policy 0, policy_version 814605 (0.0010) [2023-12-26 21:17:30,949][105620] Updated weights for policy 1, policy_version 814438 (0.0006) [2023-12-26 21:17:30,996][105692] Updated weights for policy 0, policy_version 814615 (0.0009) [2023-12-26 21:17:31,057][105692] Updated weights for policy 0, policy_version 814625 (0.0010) [2023-12-26 21:17:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 417087488. Throughput: 0: 10000.0, 1: 9724.9. Samples: 417058448. Policy #0 lag: (min: 2.0, avg: 17.7, max: 34.0) [2023-12-26 21:17:31,062][104569] Avg episode reward: [(0, '6513.268'), (1, '9351.082')] [2023-12-26 21:17:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000814440_208519168.pth... [2023-12-26 21:17:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000813288_208224256.pth [2023-12-26 21:17:31,095][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000814632_208576512.pth... [2023-12-26 21:17:31,099][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000813448_208273408.pth [2023-12-26 21:17:31,798][105620] Updated weights for policy 1, policy_version 814448 (0.0009) [2023-12-26 21:17:31,838][105692] Updated weights for policy 0, policy_version 814635 (0.0008) [2023-12-26 21:17:31,852][105620] Updated weights for policy 1, policy_version 814458 (0.0009) [2023-12-26 21:17:31,899][105620] Updated weights for policy 1, policy_version 814468 (0.0007) [2023-12-26 21:17:31,904][105692] Updated weights for policy 0, policy_version 814645 (0.0007) [2023-12-26 21:17:31,957][105692] Updated weights for policy 0, policy_version 814655 (0.0008) [2023-12-26 21:17:32,669][105620] Updated weights for policy 1, policy_version 814478 (0.0010) [2023-12-26 21:17:32,714][105692] Updated weights for policy 0, policy_version 814665 (0.0008) [2023-12-26 21:17:32,725][105620] Updated weights for policy 1, policy_version 814488 (0.0010) [2023-12-26 21:17:32,767][105692] Updated weights for policy 0, policy_version 814675 (0.0005) [2023-12-26 21:17:32,777][105620] Updated weights for policy 1, policy_version 814498 (0.0010) [2023-12-26 21:17:32,823][105692] Updated weights for policy 0, policy_version 814685 (0.0007) [2023-12-26 21:17:32,874][105692] Updated weights for policy 0, policy_version 814695 (0.0010) [2023-12-26 21:17:33,403][105620] Updated weights for policy 1, policy_version 814508 (0.0008) [2023-12-26 21:17:33,445][105620] Updated weights for policy 1, policy_version 814518 (0.0005) [2023-12-26 21:17:33,503][105620] Updated weights for policy 1, policy_version 814528 (0.0005) [2023-12-26 21:17:33,589][105692] Updated weights for policy 0, policy_version 814705 (0.0006) [2023-12-26 21:17:33,641][105692] Updated weights for policy 0, policy_version 814715 (0.0005) [2023-12-26 21:17:33,704][105692] Updated weights for policy 0, policy_version 814725 (0.0005) [2023-12-26 21:17:34,101][105620] Updated weights for policy 1, policy_version 814538 (0.0006) [2023-12-26 21:17:34,156][105620] Updated weights for policy 1, policy_version 814548 (0.0009) [2023-12-26 21:17:34,221][105620] Updated weights for policy 1, policy_version 814558 (0.0010) [2023-12-26 21:17:34,244][105692] Updated weights for policy 0, policy_version 814735 (0.0005) [2023-12-26 21:17:34,281][105620] Updated weights for policy 1, policy_version 814568 (0.0011) [2023-12-26 21:17:34,305][105692] Updated weights for policy 0, policy_version 814745 (0.0007) [2023-12-26 21:17:34,366][105692] Updated weights for policy 0, policy_version 814755 (0.0007) [2023-12-26 21:17:35,017][105620] Updated weights for policy 1, policy_version 814578 (0.0009) [2023-12-26 21:17:35,048][105692] Updated weights for policy 0, policy_version 814765 (0.0006) [2023-12-26 21:17:35,071][105620] Updated weights for policy 1, policy_version 814588 (0.0007) [2023-12-26 21:17:35,102][105692] Updated weights for policy 0, policy_version 814775 (0.0006) [2023-12-26 21:17:35,127][105620] Updated weights for policy 1, policy_version 814598 (0.0008) [2023-12-26 21:17:35,159][105692] Updated weights for policy 0, policy_version 814785 (0.0007) [2023-12-26 21:17:35,846][105620] Updated weights for policy 1, policy_version 814608 (0.0006) [2023-12-26 21:17:35,852][105692] Updated weights for policy 0, policy_version 814795 (0.0008) [2023-12-26 21:17:35,904][105620] Updated weights for policy 1, policy_version 814618 (0.0006) [2023-12-26 21:17:35,910][105692] Updated weights for policy 0, policy_version 814805 (0.0005) [2023-12-26 21:17:35,965][105692] Updated weights for policy 0, policy_version 814815 (0.0005) [2023-12-26 21:17:35,971][105620] Updated weights for policy 1, policy_version 814628 (0.0005) [2023-12-26 21:17:36,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 417193984. Throughput: 0: 9971.0, 1: 9750.2. Samples: 417175836. Policy #0 lag: (min: 2.0, avg: 17.7, max: 34.0) [2023-12-26 21:17:36,062][104569] Avg episode reward: [(0, '6745.442'), (1, '9350.965')] [2023-12-26 21:17:36,590][105620] Updated weights for policy 1, policy_version 814638 (0.0009) [2023-12-26 21:17:36,621][105692] Updated weights for policy 0, policy_version 814825 (0.0006) [2023-12-26 21:17:36,655][105620] Updated weights for policy 1, policy_version 814648 (0.0011) [2023-12-26 21:17:36,689][105692] Updated weights for policy 0, policy_version 814835 (0.0011) [2023-12-26 21:17:36,717][105620] Updated weights for policy 1, policy_version 814658 (0.0007) [2023-12-26 21:17:36,757][105692] Updated weights for policy 0, policy_version 814845 (0.0011) [2023-12-26 21:17:36,819][105692] Updated weights for policy 0, policy_version 814855 (0.0011) [2023-12-26 21:17:37,321][105620] Updated weights for policy 1, policy_version 814668 (0.0008) [2023-12-26 21:17:37,374][105620] Updated weights for policy 1, policy_version 814678 (0.0006) [2023-12-26 21:17:37,423][105620] Updated weights for policy 1, policy_version 814688 (0.0010) [2023-12-26 21:17:37,539][105692] Updated weights for policy 0, policy_version 814865 (0.0006) [2023-12-26 21:17:37,592][105692] Updated weights for policy 0, policy_version 814875 (0.0006) [2023-12-26 21:17:37,642][105692] Updated weights for policy 0, policy_version 814885 (0.0008) [2023-12-26 21:17:38,205][105620] Updated weights for policy 1, policy_version 814698 (0.0010) [2023-12-26 21:17:38,258][105620] Updated weights for policy 1, policy_version 814708 (0.0007) [2023-12-26 21:17:38,306][105692] Updated weights for policy 0, policy_version 814895 (0.0007) [2023-12-26 21:17:38,310][105620] Updated weights for policy 1, policy_version 814718 (0.0009) [2023-12-26 21:17:38,368][105692] Updated weights for policy 0, policy_version 814905 (0.0007) [2023-12-26 21:17:38,375][105620] Updated weights for policy 1, policy_version 814728 (0.0008) [2023-12-26 21:17:38,423][105692] Updated weights for policy 0, policy_version 814915 (0.0005) [2023-12-26 21:17:38,990][105692] Updated weights for policy 0, policy_version 814925 (0.0008) [2023-12-26 21:17:39,053][105692] Updated weights for policy 0, policy_version 814935 (0.0009) [2023-12-26 21:17:39,118][105692] Updated weights for policy 0, policy_version 814945 (0.0009) [2023-12-26 21:17:39,204][105620] Updated weights for policy 1, policy_version 814738 (0.0010) [2023-12-26 21:17:39,266][105620] Updated weights for policy 1, policy_version 814748 (0.0009) [2023-12-26 21:17:39,319][105620] Updated weights for policy 1, policy_version 814758 (0.0008) [2023-12-26 21:17:39,887][105692] Updated weights for policy 0, policy_version 814955 (0.0008) [2023-12-26 21:17:39,959][105692] Updated weights for policy 0, policy_version 814965 (0.0009) [2023-12-26 21:17:40,019][105692] Updated weights for policy 0, policy_version 814975 (0.0010) [2023-12-26 21:17:40,067][105620] Updated weights for policy 1, policy_version 814768 (0.0007) [2023-12-26 21:17:40,125][105620] Updated weights for policy 1, policy_version 814778 (0.0009) [2023-12-26 21:17:40,176][105620] Updated weights for policy 1, policy_version 814788 (0.0009) [2023-12-26 21:17:40,654][105692] Updated weights for policy 0, policy_version 814985 (0.0007) [2023-12-26 21:17:40,714][105692] Updated weights for policy 0, policy_version 814995 (0.0005) [2023-12-26 21:17:40,779][105692] Updated weights for policy 0, policy_version 815005 (0.0009) [2023-12-26 21:17:40,842][105692] Updated weights for policy 0, policy_version 815015 (0.0007) [2023-12-26 21:17:41,026][105620] Updated weights for policy 1, policy_version 814798 (0.0009) [2023-12-26 21:17:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 417284096. Throughput: 0: 9996.2, 1: 9718.7. Samples: 417293564. Policy #0 lag: (min: 4.0, avg: 8.8, max: 36.0) [2023-12-26 21:17:41,063][104569] Avg episode reward: [(0, '7317.114'), (1, '9077.062')] [2023-12-26 21:17:41,093][105620] Updated weights for policy 1, policy_version 814808 (0.0008) [2023-12-26 21:17:41,159][105620] Updated weights for policy 1, policy_version 814818 (0.0008) [2023-12-26 21:17:41,583][105692] Updated weights for policy 0, policy_version 815025 (0.0008) [2023-12-26 21:17:41,647][105692] Updated weights for policy 0, policy_version 815035 (0.0008) [2023-12-26 21:17:41,707][105692] Updated weights for policy 0, policy_version 815045 (0.0007) [2023-12-26 21:17:41,945][105620] Updated weights for policy 1, policy_version 814828 (0.0007) [2023-12-26 21:17:42,007][105620] Updated weights for policy 1, policy_version 814838 (0.0006) [2023-12-26 21:17:42,070][105620] Updated weights for policy 1, policy_version 814848 (0.0011) [2023-12-26 21:17:42,463][105692] Updated weights for policy 0, policy_version 815055 (0.0008) [2023-12-26 21:17:42,521][105692] Updated weights for policy 0, policy_version 815065 (0.0008) [2023-12-26 21:17:42,588][105692] Updated weights for policy 0, policy_version 815075 (0.0008) [2023-12-26 21:17:42,724][105620] Updated weights for policy 1, policy_version 814858 (0.0011) [2023-12-26 21:17:42,777][105620] Updated weights for policy 1, policy_version 814868 (0.0008) [2023-12-26 21:17:42,836][105620] Updated weights for policy 1, policy_version 814878 (0.0009) [2023-12-26 21:17:42,894][105620] Updated weights for policy 1, policy_version 814888 (0.0009) [2023-12-26 21:17:43,231][105692] Updated weights for policy 0, policy_version 815085 (0.0007) [2023-12-26 21:17:43,294][105692] Updated weights for policy 0, policy_version 815096 (0.0009) [2023-12-26 21:17:43,344][105692] Updated weights for policy 0, policy_version 815106 (0.0009) [2023-12-26 21:17:43,518][105620] Updated weights for policy 1, policy_version 814898 (0.0008) [2023-12-26 21:17:43,567][105620] Updated weights for policy 1, policy_version 814908 (0.0008) [2023-12-26 21:17:43,626][105620] Updated weights for policy 1, policy_version 814918 (0.0005) [2023-12-26 21:17:43,992][105692] Updated weights for policy 0, policy_version 815116 (0.0009) [2023-12-26 21:17:44,045][105692] Updated weights for policy 0, policy_version 815126 (0.0008) [2023-12-26 21:17:44,102][105692] Updated weights for policy 0, policy_version 815136 (0.0009) [2023-12-26 21:17:44,257][105620] Updated weights for policy 1, policy_version 814928 (0.0005) [2023-12-26 21:17:44,325][105620] Updated weights for policy 1, policy_version 814938 (0.0007) [2023-12-26 21:17:44,376][105620] Updated weights for policy 1, policy_version 814948 (0.0008) [2023-12-26 21:17:44,768][105692] Updated weights for policy 0, policy_version 815146 (0.0007) [2023-12-26 21:17:44,835][105692] Updated weights for policy 0, policy_version 815156 (0.0006) [2023-12-26 21:17:44,905][105692] Updated weights for policy 0, policy_version 815166 (0.0006) [2023-12-26 21:17:44,966][105692] Updated weights for policy 0, policy_version 815176 (0.0006) [2023-12-26 21:17:45,050][105620] Updated weights for policy 1, policy_version 814958 (0.0009) [2023-12-26 21:17:45,109][105620] Updated weights for policy 1, policy_version 814968 (0.0009) [2023-12-26 21:17:45,165][105620] Updated weights for policy 1, policy_version 814978 (0.0008) [2023-12-26 21:17:45,618][105692] Updated weights for policy 0, policy_version 815186 (0.0009) [2023-12-26 21:17:45,670][105692] Updated weights for policy 0, policy_version 815196 (0.0009) [2023-12-26 21:17:45,728][105692] Updated weights for policy 0, policy_version 815206 (0.0010) [2023-12-26 21:17:45,884][105620] Updated weights for policy 1, policy_version 814988 (0.0008) [2023-12-26 21:17:45,941][105620] Updated weights for policy 1, policy_version 814998 (0.0008) [2023-12-26 21:17:45,993][105620] Updated weights for policy 1, policy_version 815008 (0.0008) [2023-12-26 21:17:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19933.8, 300 sec: 19577.5). Total num frames: 417390592. Throughput: 0: 9967.5, 1: 9701.8. Samples: 417352800. Policy #0 lag: (min: 4.0, avg: 8.8, max: 36.0) [2023-12-26 21:17:46,063][104569] Avg episode reward: [(0, '7566.053'), (1, '8985.730')] [2023-12-26 21:17:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000815208_208723968.pth... [2023-12-26 21:17:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000815016_208666624.pth... [2023-12-26 21:17:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000813896_208379904.pth [2023-12-26 21:17:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000814056_208429056.pth [2023-12-26 21:17:46,539][105692] Updated weights for policy 0, policy_version 815216 (0.0009) [2023-12-26 21:17:46,588][105692] Updated weights for policy 0, policy_version 815226 (0.0009) [2023-12-26 21:17:46,636][105692] Updated weights for policy 0, policy_version 815236 (0.0009) [2023-12-26 21:17:46,754][105620] Updated weights for policy 1, policy_version 815018 (0.0009) [2023-12-26 21:17:46,801][105620] Updated weights for policy 1, policy_version 815028 (0.0009) [2023-12-26 21:17:46,849][105620] Updated weights for policy 1, policy_version 815038 (0.0009) [2023-12-26 21:17:46,897][105620] Updated weights for policy 1, policy_version 815048 (0.0009) [2023-12-26 21:17:47,450][105692] Updated weights for policy 0, policy_version 815246 (0.0009) [2023-12-26 21:17:47,503][105692] Updated weights for policy 0, policy_version 815256 (0.0009) [2023-12-26 21:17:47,554][105692] Updated weights for policy 0, policy_version 815266 (0.0009) [2023-12-26 21:17:47,593][105620] Updated weights for policy 1, policy_version 815058 (0.0005) [2023-12-26 21:17:47,644][105620] Updated weights for policy 1, policy_version 815068 (0.0005) [2023-12-26 21:17:47,690][105620] Updated weights for policy 1, policy_version 815078 (0.0005) [2023-12-26 21:17:48,341][105620] Updated weights for policy 1, policy_version 815088 (0.0005) [2023-12-26 21:17:48,395][105692] Updated weights for policy 0, policy_version 815276 (0.0009) [2023-12-26 21:17:48,402][105620] Updated weights for policy 1, policy_version 815098 (0.0008) [2023-12-26 21:17:48,449][105692] Updated weights for policy 0, policy_version 815286 (0.0007) [2023-12-26 21:17:48,455][105620] Updated weights for policy 1, policy_version 815108 (0.0007) [2023-12-26 21:17:48,504][105692] Updated weights for policy 0, policy_version 815296 (0.0007) [2023-12-26 21:17:49,100][105620] Updated weights for policy 1, policy_version 815118 (0.0006) [2023-12-26 21:17:49,160][105620] Updated weights for policy 1, policy_version 815128 (0.0005) [2023-12-26 21:17:49,227][105620] Updated weights for policy 1, policy_version 815138 (0.0006) [2023-12-26 21:17:49,376][105692] Updated weights for policy 0, policy_version 815306 (0.0010) [2023-12-26 21:17:49,435][105692] Updated weights for policy 0, policy_version 815316 (0.0007) [2023-12-26 21:17:49,499][105692] Updated weights for policy 0, policy_version 815326 (0.0006) [2023-12-26 21:17:49,559][105692] Updated weights for policy 0, policy_version 815336 (0.0009) [2023-12-26 21:17:49,941][105620] Updated weights for policy 1, policy_version 815148 (0.0008) [2023-12-26 21:17:50,008][105620] Updated weights for policy 1, policy_version 815158 (0.0008) [2023-12-26 21:17:50,067][105620] Updated weights for policy 1, policy_version 815168 (0.0007) [2023-12-26 21:17:50,278][105692] Updated weights for policy 0, policy_version 815346 (0.0010) [2023-12-26 21:17:50,337][105692] Updated weights for policy 0, policy_version 815356 (0.0011) [2023-12-26 21:17:50,396][105692] Updated weights for policy 0, policy_version 815366 (0.0011) [2023-12-26 21:17:50,753][105620] Updated weights for policy 1, policy_version 815178 (0.0007) [2023-12-26 21:17:50,813][105620] Updated weights for policy 1, policy_version 815188 (0.0006) [2023-12-26 21:17:50,872][105620] Updated weights for policy 1, policy_version 815198 (0.0006) [2023-12-26 21:17:50,934][105620] Updated weights for policy 1, policy_version 815208 (0.0006) [2023-12-26 21:17:51,027][105692] Updated weights for policy 0, policy_version 815376 (0.0008) [2023-12-26 21:17:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 417480704. Throughput: 0: 9869.2, 1: 9691.6. Samples: 417469616. Policy #0 lag: (min: 4.0, avg: 8.8, max: 36.0) [2023-12-26 21:17:51,062][104569] Avg episode reward: [(0, '6222.882'), (1, '9075.289')] [2023-12-26 21:17:51,094][105692] Updated weights for policy 0, policy_version 815386 (0.0009) [2023-12-26 21:17:51,148][105692] Updated weights for policy 0, policy_version 815396 (0.0011) [2023-12-26 21:17:51,598][105620] Updated weights for policy 1, policy_version 815218 (0.0010) [2023-12-26 21:17:51,664][105620] Updated weights for policy 1, policy_version 815228 (0.0010) [2023-12-26 21:17:51,683][105586] KL-divergence is very high: 101.3751 [2023-12-26 21:17:51,723][105620] Updated weights for policy 1, policy_version 815238 (0.0010) [2023-12-26 21:17:51,849][105692] Updated weights for policy 0, policy_version 815406 (0.0007) [2023-12-26 21:17:51,909][105692] Updated weights for policy 0, policy_version 815416 (0.0010) [2023-12-26 21:17:51,964][105692] Updated weights for policy 0, policy_version 815426 (0.0011) [2023-12-26 21:17:52,340][105620] Updated weights for policy 1, policy_version 815248 (0.0009) [2023-12-26 21:17:52,398][105620] Updated weights for policy 1, policy_version 815258 (0.0010) [2023-12-26 21:17:52,453][105620] Updated weights for policy 1, policy_version 815268 (0.0006) [2023-12-26 21:17:52,551][105692] Updated weights for policy 0, policy_version 815436 (0.0009) [2023-12-26 21:17:52,611][105692] Updated weights for policy 0, policy_version 815446 (0.0008) [2023-12-26 21:17:52,677][105692] Updated weights for policy 0, policy_version 815456 (0.0010) [2023-12-26 21:17:53,047][105620] Updated weights for policy 1, policy_version 815278 (0.0007) [2023-12-26 21:17:53,115][105620] Updated weights for policy 1, policy_version 815288 (0.0006) [2023-12-26 21:17:53,183][105620] Updated weights for policy 1, policy_version 815298 (0.0005) [2023-12-26 21:17:53,322][105692] Updated weights for policy 0, policy_version 815466 (0.0010) [2023-12-26 21:17:53,383][105692] Updated weights for policy 0, policy_version 815476 (0.0006) [2023-12-26 21:17:53,439][105692] Updated weights for policy 0, policy_version 815486 (0.0006) [2023-12-26 21:17:53,498][105692] Updated weights for policy 0, policy_version 815496 (0.0006) [2023-12-26 21:17:53,686][105620] Updated weights for policy 1, policy_version 815308 (0.0005) [2023-12-26 21:17:53,740][105620] Updated weights for policy 1, policy_version 815318 (0.0005) [2023-12-26 21:17:53,803][105620] Updated weights for policy 1, policy_version 815328 (0.0005) [2023-12-26 21:17:54,169][105692] Updated weights for policy 0, policy_version 815506 (0.0005) [2023-12-26 21:17:54,236][105692] Updated weights for policy 0, policy_version 815516 (0.0006) [2023-12-26 21:17:54,301][105692] Updated weights for policy 0, policy_version 815526 (0.0006) [2023-12-26 21:17:54,436][105620] Updated weights for policy 1, policy_version 815338 (0.0006) [2023-12-26 21:17:54,494][105620] Updated weights for policy 1, policy_version 815348 (0.0010) [2023-12-26 21:17:54,550][105620] Updated weights for policy 1, policy_version 815358 (0.0010) [2023-12-26 21:17:54,609][105620] Updated weights for policy 1, policy_version 815368 (0.0007) [2023-12-26 21:17:54,929][105692] Updated weights for policy 0, policy_version 815536 (0.0010) [2023-12-26 21:17:54,985][105692] Updated weights for policy 0, policy_version 815546 (0.0011) [2023-12-26 21:17:55,038][105692] Updated weights for policy 0, policy_version 815556 (0.0011) [2023-12-26 21:17:55,339][105620] Updated weights for policy 1, policy_version 815378 (0.0008) [2023-12-26 21:17:55,383][105620] Updated weights for policy 1, policy_version 815388 (0.0008) [2023-12-26 21:17:55,434][105620] Updated weights for policy 1, policy_version 815398 (0.0008) [2023-12-26 21:17:55,789][105692] Updated weights for policy 0, policy_version 815566 (0.0010) [2023-12-26 21:17:55,840][105692] Updated weights for policy 0, policy_version 815576 (0.0010) [2023-12-26 21:17:55,888][105692] Updated weights for policy 0, policy_version 815586 (0.0010) [2023-12-26 21:17:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 417587200. Throughput: 0: 9855.6, 1: 9709.1. Samples: 417595060. Policy #0 lag: (min: 4.0, avg: 8.8, max: 36.0) [2023-12-26 21:17:56,063][104569] Avg episode reward: [(0, '7302.425'), (1, '9166.315')] [2023-12-26 21:17:56,212][105620] Updated weights for policy 1, policy_version 815408 (0.0010) [2023-12-26 21:17:56,270][105620] Updated weights for policy 1, policy_version 815418 (0.0010) [2023-12-26 21:17:56,334][105620] Updated weights for policy 1, policy_version 815428 (0.0009) [2023-12-26 21:17:56,636][105692] Updated weights for policy 0, policy_version 815596 (0.0010) [2023-12-26 21:17:56,694][105692] Updated weights for policy 0, policy_version 815606 (0.0010) [2023-12-26 21:17:56,748][105692] Updated weights for policy 0, policy_version 815616 (0.0009) [2023-12-26 21:17:56,880][105620] Updated weights for policy 1, policy_version 815438 (0.0008) [2023-12-26 21:17:56,942][105620] Updated weights for policy 1, policy_version 815448 (0.0010) [2023-12-26 21:17:57,009][105620] Updated weights for policy 1, policy_version 815458 (0.0010) [2023-12-26 21:17:57,384][105692] Updated weights for policy 0, policy_version 815626 (0.0006) [2023-12-26 21:17:57,437][105692] Updated weights for policy 0, policy_version 815636 (0.0005) [2023-12-26 21:17:57,496][105692] Updated weights for policy 0, policy_version 815646 (0.0005) [2023-12-26 21:17:57,545][105692] Updated weights for policy 0, policy_version 815656 (0.0007) [2023-12-26 21:17:57,685][105620] Updated weights for policy 1, policy_version 815468 (0.0008) [2023-12-26 21:17:57,744][105620] Updated weights for policy 1, policy_version 815478 (0.0010) [2023-12-26 21:17:57,796][105620] Updated weights for policy 1, policy_version 815488 (0.0010) [2023-12-26 21:17:58,222][105692] Updated weights for policy 0, policy_version 815666 (0.0011) [2023-12-26 21:17:58,285][105692] Updated weights for policy 0, policy_version 815676 (0.0010) [2023-12-26 21:17:58,350][105692] Updated weights for policy 0, policy_version 815686 (0.0010) [2023-12-26 21:17:58,511][105620] Updated weights for policy 1, policy_version 815498 (0.0010) [2023-12-26 21:17:58,571][105620] Updated weights for policy 1, policy_version 815508 (0.0007) [2023-12-26 21:17:58,634][105620] Updated weights for policy 1, policy_version 815518 (0.0011) [2023-12-26 21:17:58,695][105620] Updated weights for policy 1, policy_version 815528 (0.0008) [2023-12-26 21:17:59,089][105692] Updated weights for policy 0, policy_version 815696 (0.0007) [2023-12-26 21:17:59,144][105692] Updated weights for policy 0, policy_version 815706 (0.0007) [2023-12-26 21:17:59,207][105692] Updated weights for policy 0, policy_version 815716 (0.0010) [2023-12-26 21:17:59,448][105620] Updated weights for policy 1, policy_version 815538 (0.0008) [2023-12-26 21:17:59,503][105620] Updated weights for policy 1, policy_version 815548 (0.0008) [2023-12-26 21:17:59,562][105620] Updated weights for policy 1, policy_version 815558 (0.0008) [2023-12-26 21:17:59,871][105692] Updated weights for policy 0, policy_version 815726 (0.0010) [2023-12-26 21:17:59,926][105692] Updated weights for policy 0, policy_version 815736 (0.0010) [2023-12-26 21:17:59,985][105692] Updated weights for policy 0, policy_version 815746 (0.0011) [2023-12-26 21:18:00,226][105620] Updated weights for policy 1, policy_version 815568 (0.0006) [2023-12-26 21:18:00,278][105620] Updated weights for policy 1, policy_version 815578 (0.0005) [2023-12-26 21:18:00,329][105620] Updated weights for policy 1, policy_version 815588 (0.0006) [2023-12-26 21:18:00,707][105692] Updated weights for policy 0, policy_version 815756 (0.0008) [2023-12-26 21:18:00,781][105692] Updated weights for policy 0, policy_version 815766 (0.0006) [2023-12-26 21:18:00,835][105692] Updated weights for policy 0, policy_version 815776 (0.0009) [2023-12-26 21:18:01,037][105620] Updated weights for policy 1, policy_version 815598 (0.0007) [2023-12-26 21:18:01,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 417685504. Throughput: 0: 9876.6, 1: 9744.1. Samples: 417654724. Policy #0 lag: (min: 4.0, avg: 8.8, max: 36.0) [2023-12-26 21:18:01,063][104569] Avg episode reward: [(0, '8645.028'), (1, '9167.332')] [2023-12-26 21:18:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000815784_208871424.pth... [2023-12-26 21:18:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000814632_208576512.pth [2023-12-26 21:18:01,104][105620] Updated weights for policy 1, policy_version 815608 (0.0009) [2023-12-26 21:18:01,167][105620] Updated weights for policy 1, policy_version 815618 (0.0009) [2023-12-26 21:18:01,202][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000815624_208822272.pth... [2023-12-26 21:18:01,206][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000814440_208519168.pth [2023-12-26 21:18:01,565][105692] Updated weights for policy 0, policy_version 815786 (0.0010) [2023-12-26 21:18:01,620][105692] Updated weights for policy 0, policy_version 815797 (0.0010) [2023-12-26 21:18:01,686][105692] Updated weights for policy 0, policy_version 815807 (0.0008) [2023-12-26 21:18:01,923][105620] Updated weights for policy 1, policy_version 815628 (0.0007) [2023-12-26 21:18:01,981][105620] Updated weights for policy 1, policy_version 815638 (0.0008) [2023-12-26 21:18:02,034][105620] Updated weights for policy 1, policy_version 815648 (0.0010) [2023-12-26 21:18:02,409][105692] Updated weights for policy 0, policy_version 815817 (0.0008) [2023-12-26 21:18:02,473][105692] Updated weights for policy 0, policy_version 815827 (0.0007) [2023-12-26 21:18:02,529][105692] Updated weights for policy 0, policy_version 815837 (0.0006) [2023-12-26 21:18:02,598][105692] Updated weights for policy 0, policy_version 815847 (0.0011) [2023-12-26 21:18:02,694][105620] Updated weights for policy 1, policy_version 815658 (0.0009) [2023-12-26 21:18:02,747][105620] Updated weights for policy 1, policy_version 815668 (0.0008) [2023-12-26 21:18:02,807][105620] Updated weights for policy 1, policy_version 815678 (0.0008) [2023-12-26 21:18:02,864][105620] Updated weights for policy 1, policy_version 815688 (0.0005) [2023-12-26 21:18:03,205][105692] Updated weights for policy 0, policy_version 815857 (0.0008) [2023-12-26 21:18:03,259][105692] Updated weights for policy 0, policy_version 815867 (0.0008) [2023-12-26 21:18:03,317][105692] Updated weights for policy 0, policy_version 815877 (0.0010) [2023-12-26 21:18:03,515][105620] Updated weights for policy 1, policy_version 815698 (0.0007) [2023-12-26 21:18:03,570][105620] Updated weights for policy 1, policy_version 815708 (0.0008) [2023-12-26 21:18:03,630][105620] Updated weights for policy 1, policy_version 815718 (0.0008) [2023-12-26 21:18:04,028][105692] Updated weights for policy 0, policy_version 815887 (0.0010) [2023-12-26 21:18:04,090][105692] Updated weights for policy 0, policy_version 815897 (0.0010) [2023-12-26 21:18:04,145][105692] Updated weights for policy 0, policy_version 815907 (0.0010) [2023-12-26 21:18:04,417][105620] Updated weights for policy 1, policy_version 815728 (0.0008) [2023-12-26 21:18:04,475][105620] Updated weights for policy 1, policy_version 815738 (0.0007) [2023-12-26 21:18:04,533][105620] Updated weights for policy 1, policy_version 815748 (0.0005) [2023-12-26 21:18:04,764][105692] Updated weights for policy 0, policy_version 815917 (0.0008) [2023-12-26 21:18:04,812][105692] Updated weights for policy 0, policy_version 815927 (0.0009) [2023-12-26 21:18:04,860][105692] Updated weights for policy 0, policy_version 815937 (0.0010) [2023-12-26 21:18:05,354][105620] Updated weights for policy 1, policy_version 815758 (0.0008) [2023-12-26 21:18:05,406][105620] Updated weights for policy 1, policy_version 815768 (0.0008) [2023-12-26 21:18:05,461][105620] Updated weights for policy 1, policy_version 815778 (0.0008) [2023-12-26 21:18:05,504][105692] Updated weights for policy 0, policy_version 815947 (0.0009) [2023-12-26 21:18:05,562][105692] Updated weights for policy 0, policy_version 815957 (0.0010) [2023-12-26 21:18:05,617][105692] Updated weights for policy 0, policy_version 815967 (0.0009) [2023-12-26 21:18:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 417783808. Throughput: 0: 9958.5, 1: 9731.4. Samples: 417773116. Policy #0 lag: (min: 4.0, avg: 8.8, max: 36.0) [2023-12-26 21:18:06,062][104569] Avg episode reward: [(0, '7856.036'), (1, '9167.461')] [2023-12-26 21:18:06,165][105620] Updated weights for policy 1, policy_version 815788 (0.0009) [2023-12-26 21:18:06,172][105692] Updated weights for policy 0, policy_version 815977 (0.0005) [2023-12-26 21:18:06,218][105620] Updated weights for policy 1, policy_version 815798 (0.0008) [2023-12-26 21:18:06,236][105692] Updated weights for policy 0, policy_version 815987 (0.0006) [2023-12-26 21:18:06,282][105620] Updated weights for policy 1, policy_version 815808 (0.0008) [2023-12-26 21:18:06,305][105692] Updated weights for policy 0, policy_version 815997 (0.0005) [2023-12-26 21:18:06,374][105692] Updated weights for policy 0, policy_version 816007 (0.0006) [2023-12-26 21:18:07,037][105692] Updated weights for policy 0, policy_version 816017 (0.0008) [2023-12-26 21:18:07,075][105620] Updated weights for policy 1, policy_version 815818 (0.0008) [2023-12-26 21:18:07,090][105692] Updated weights for policy 0, policy_version 816027 (0.0008) [2023-12-26 21:18:07,129][105620] Updated weights for policy 1, policy_version 815828 (0.0006) [2023-12-26 21:18:07,143][105692] Updated weights for policy 0, policy_version 816037 (0.0006) [2023-12-26 21:18:07,176][105620] Updated weights for policy 1, policy_version 815838 (0.0007) [2023-12-26 21:18:07,236][105620] Updated weights for policy 1, policy_version 815848 (0.0009) [2023-12-26 21:18:07,889][105620] Updated weights for policy 1, policy_version 815858 (0.0009) [2023-12-26 21:18:07,958][105620] Updated weights for policy 1, policy_version 815868 (0.0010) [2023-12-26 21:18:07,983][105692] Updated weights for policy 0, policy_version 816047 (0.0005) [2023-12-26 21:18:08,018][105620] Updated weights for policy 1, policy_version 815878 (0.0008) [2023-12-26 21:18:08,039][105692] Updated weights for policy 0, policy_version 816057 (0.0007) [2023-12-26 21:18:08,101][105692] Updated weights for policy 0, policy_version 816067 (0.0010) [2023-12-26 21:18:08,762][105620] Updated weights for policy 1, policy_version 815888 (0.0009) [2023-12-26 21:18:08,816][105620] Updated weights for policy 1, policy_version 815898 (0.0009) [2023-12-26 21:18:08,864][105620] Updated weights for policy 1, policy_version 815908 (0.0008) [2023-12-26 21:18:08,872][105692] Updated weights for policy 0, policy_version 816077 (0.0009) [2023-12-26 21:18:08,932][105692] Updated weights for policy 0, policy_version 816087 (0.0008) [2023-12-26 21:18:08,991][105692] Updated weights for policy 0, policy_version 816097 (0.0009) [2023-12-26 21:18:09,667][105620] Updated weights for policy 1, policy_version 815918 (0.0008) [2023-12-26 21:18:09,726][105620] Updated weights for policy 1, policy_version 815928 (0.0009) [2023-12-26 21:18:09,762][105692] Updated weights for policy 0, policy_version 816107 (0.0009) [2023-12-26 21:18:09,789][105620] Updated weights for policy 1, policy_version 815938 (0.0008) [2023-12-26 21:18:09,822][105692] Updated weights for policy 0, policy_version 816117 (0.0008) [2023-12-26 21:18:09,882][105692] Updated weights for policy 0, policy_version 816127 (0.0008) [2023-12-26 21:18:10,441][105620] Updated weights for policy 1, policy_version 815948 (0.0007) [2023-12-26 21:18:10,502][105620] Updated weights for policy 1, policy_version 815958 (0.0009) [2023-12-26 21:18:10,566][105620] Updated weights for policy 1, policy_version 815968 (0.0006) [2023-12-26 21:18:10,614][105692] Updated weights for policy 0, policy_version 816137 (0.0009) [2023-12-26 21:18:10,668][105692] Updated weights for policy 0, policy_version 816147 (0.0009) [2023-12-26 21:18:10,724][105692] Updated weights for policy 0, policy_version 816157 (0.0009) [2023-12-26 21:18:10,786][105692] Updated weights for policy 0, policy_version 816167 (0.0009) [2023-12-26 21:18:11,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 417882112. Throughput: 0: 9987.0, 1: 9811.6. Samples: 417890392. Policy #0 lag: (min: 4.0, avg: 8.8, max: 36.0) [2023-12-26 21:18:11,062][104569] Avg episode reward: [(0, '7317.114'), (1, '8894.153')] [2023-12-26 21:18:11,266][105620] Updated weights for policy 1, policy_version 815978 (0.0008) [2023-12-26 21:18:11,325][105620] Updated weights for policy 1, policy_version 815988 (0.0006) [2023-12-26 21:18:11,395][105620] Updated weights for policy 1, policy_version 815998 (0.0008) [2023-12-26 21:18:11,459][105620] Updated weights for policy 1, policy_version 816008 (0.0009) [2023-12-26 21:18:11,554][105692] Updated weights for policy 0, policy_version 816177 (0.0006) [2023-12-26 21:18:11,620][105692] Updated weights for policy 0, policy_version 816187 (0.0007) [2023-12-26 21:18:11,695][105692] Updated weights for policy 0, policy_version 816197 (0.0009) [2023-12-26 21:18:12,219][105620] Updated weights for policy 1, policy_version 816018 (0.0006) [2023-12-26 21:18:12,280][105620] Updated weights for policy 1, policy_version 816028 (0.0007) [2023-12-26 21:18:12,348][105620] Updated weights for policy 1, policy_version 816038 (0.0006) [2023-12-26 21:18:12,471][105692] Updated weights for policy 0, policy_version 816207 (0.0009) [2023-12-26 21:18:12,523][105692] Updated weights for policy 0, policy_version 816217 (0.0008) [2023-12-26 21:18:12,573][105692] Updated weights for policy 0, policy_version 816227 (0.0008) [2023-12-26 21:18:13,000][105620] Updated weights for policy 1, policy_version 816048 (0.0009) [2023-12-26 21:18:13,065][105620] Updated weights for policy 1, policy_version 816058 (0.0010) [2023-12-26 21:18:13,125][105620] Updated weights for policy 1, policy_version 816068 (0.0008) [2023-12-26 21:18:13,406][105692] Updated weights for policy 0, policy_version 816237 (0.0010) [2023-12-26 21:18:13,460][105692] Updated weights for policy 0, policy_version 816247 (0.0010) [2023-12-26 21:18:13,517][105692] Updated weights for policy 0, policy_version 816257 (0.0010) [2023-12-26 21:18:13,826][105620] Updated weights for policy 1, policy_version 816078 (0.0008) [2023-12-26 21:18:13,888][105620] Updated weights for policy 1, policy_version 816088 (0.0009) [2023-12-26 21:18:13,947][105620] Updated weights for policy 1, policy_version 816098 (0.0009) [2023-12-26 21:18:14,322][105692] Updated weights for policy 0, policy_version 816267 (0.0009) [2023-12-26 21:18:14,395][105692] Updated weights for policy 0, policy_version 816277 (0.0007) [2023-12-26 21:18:14,457][105692] Updated weights for policy 0, policy_version 816287 (0.0007) [2023-12-26 21:18:14,612][105620] Updated weights for policy 1, policy_version 816108 (0.0009) [2023-12-26 21:18:14,668][105620] Updated weights for policy 1, policy_version 816118 (0.0009) [2023-12-26 21:18:14,730][105620] Updated weights for policy 1, policy_version 816128 (0.0010) [2023-12-26 21:18:15,069][105692] Updated weights for policy 0, policy_version 816297 (0.0006) [2023-12-26 21:18:15,132][105692] Updated weights for policy 0, policy_version 816307 (0.0011) [2023-12-26 21:18:15,195][105692] Updated weights for policy 0, policy_version 816317 (0.0011) [2023-12-26 21:18:15,251][105692] Updated weights for policy 0, policy_version 816327 (0.0011) [2023-12-26 21:18:15,356][105620] Updated weights for policy 1, policy_version 816138 (0.0008) [2023-12-26 21:18:15,420][105620] Updated weights for policy 1, policy_version 816148 (0.0011) [2023-12-26 21:18:15,486][105620] Updated weights for policy 1, policy_version 816158 (0.0011) [2023-12-26 21:18:15,552][105620] Updated weights for policy 1, policy_version 816168 (0.0011) [2023-12-26 21:18:16,014][105692] Updated weights for policy 0, policy_version 816337 (0.0010) [2023-12-26 21:18:16,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 417972224. Throughput: 0: 9847.2, 1: 9874.8. Samples: 417945940. Policy #0 lag: (min: 4.0, avg: 8.8, max: 36.0) [2023-12-26 21:18:16,063][104569] Avg episode reward: [(0, '8273.438'), (1, '8803.681')] [2023-12-26 21:18:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000816168_208961536.pth... [2023-12-26 21:18:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000815016_208666624.pth [2023-12-26 21:18:16,079][105692] Updated weights for policy 0, policy_version 816347 (0.0009) [2023-12-26 21:18:16,141][105692] Updated weights for policy 0, policy_version 816357 (0.0006) [2023-12-26 21:18:16,156][105585] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000006 [2023-12-26 21:18:16,157][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000816360_209018880.pth... [2023-12-26 21:18:16,160][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000815208_208723968.pth [2023-12-26 21:18:16,184][105620] Updated weights for policy 1, policy_version 816178 (0.0010) [2023-12-26 21:18:16,246][105620] Updated weights for policy 1, policy_version 816188 (0.0008) [2023-12-26 21:18:16,318][105620] Updated weights for policy 1, policy_version 816198 (0.0007) [2023-12-26 21:18:16,774][105692] Updated weights for policy 0, policy_version 816367 (0.0005) [2023-12-26 21:18:16,827][105692] Updated weights for policy 0, policy_version 816377 (0.0005) [2023-12-26 21:18:16,879][105692] Updated weights for policy 0, policy_version 816387 (0.0007) [2023-12-26 21:18:17,011][105620] Updated weights for policy 1, policy_version 816208 (0.0009) [2023-12-26 21:18:17,059][105620] Updated weights for policy 1, policy_version 816218 (0.0010) [2023-12-26 21:18:17,103][105620] Updated weights for policy 1, policy_version 816228 (0.0010) [2023-12-26 21:18:17,649][105692] Updated weights for policy 0, policy_version 816397 (0.0009) [2023-12-26 21:18:17,695][105692] Updated weights for policy 0, policy_version 816407 (0.0008) [2023-12-26 21:18:17,725][105620] Updated weights for policy 1, policy_version 816238 (0.0008) [2023-12-26 21:18:17,740][105692] Updated weights for policy 0, policy_version 816417 (0.0006) [2023-12-26 21:18:17,784][105620] Updated weights for policy 1, policy_version 816248 (0.0010) [2023-12-26 21:18:17,845][105620] Updated weights for policy 1, policy_version 816258 (0.0009) [2023-12-26 21:18:18,462][105692] Updated weights for policy 0, policy_version 816427 (0.0008) [2023-12-26 21:18:18,520][105692] Updated weights for policy 0, policy_version 816437 (0.0008) [2023-12-26 21:18:18,575][105692] Updated weights for policy 0, policy_version 816447 (0.0009) [2023-12-26 21:18:18,633][105620] Updated weights for policy 1, policy_version 816268 (0.0010) [2023-12-26 21:18:18,696][105620] Updated weights for policy 1, policy_version 816278 (0.0010) [2023-12-26 21:18:18,758][105620] Updated weights for policy 1, policy_version 816288 (0.0011) [2023-12-26 21:18:19,291][105692] Updated weights for policy 0, policy_version 816457 (0.0008) [2023-12-26 21:18:19,362][105692] Updated weights for policy 0, policy_version 816467 (0.0006) [2023-12-26 21:18:19,429][105692] Updated weights for policy 0, policy_version 816477 (0.0009) [2023-12-26 21:18:19,490][105692] Updated weights for policy 0, policy_version 816487 (0.0006) [2023-12-26 21:18:19,491][105620] Updated weights for policy 1, policy_version 816298 (0.0010) [2023-12-26 21:18:19,552][105620] Updated weights for policy 1, policy_version 816308 (0.0006) [2023-12-26 21:18:19,616][105620] Updated weights for policy 1, policy_version 816318 (0.0006) [2023-12-26 21:18:19,678][105620] Updated weights for policy 1, policy_version 816328 (0.0009) [2023-12-26 21:18:20,071][105692] Updated weights for policy 0, policy_version 816497 (0.0006) [2023-12-26 21:18:20,121][105692] Updated weights for policy 0, policy_version 816507 (0.0006) [2023-12-26 21:18:20,182][105692] Updated weights for policy 0, policy_version 816517 (0.0006) [2023-12-26 21:18:20,422][105620] Updated weights for policy 1, policy_version 816338 (0.0011) [2023-12-26 21:18:20,487][105620] Updated weights for policy 1, policy_version 816348 (0.0011) [2023-12-26 21:18:20,542][105620] Updated weights for policy 1, policy_version 816358 (0.0010) [2023-12-26 21:18:20,868][105692] Updated weights for policy 0, policy_version 816527 (0.0007) [2023-12-26 21:18:20,930][105692] Updated weights for policy 0, policy_version 816537 (0.0009) [2023-12-26 21:18:20,991][105692] Updated weights for policy 0, policy_version 816547 (0.0009) [2023-12-26 21:18:21,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 418078720. Throughput: 0: 9820.6, 1: 9948.8. Samples: 418065460. Policy #0 lag: (min: 4.0, avg: 8.8, max: 36.0) [2023-12-26 21:18:21,063][104569] Avg episode reward: [(0, '8539.568'), (1, '9168.510')] [2023-12-26 21:18:21,274][105620] Updated weights for policy 1, policy_version 816368 (0.0010) [2023-12-26 21:18:21,339][105620] Updated weights for policy 1, policy_version 816378 (0.0011) [2023-12-26 21:18:21,405][105620] Updated weights for policy 1, policy_version 816388 (0.0008) [2023-12-26 21:18:21,854][105692] Updated weights for policy 0, policy_version 816557 (0.0009) [2023-12-26 21:18:21,914][105692] Updated weights for policy 0, policy_version 816567 (0.0009) [2023-12-26 21:18:21,976][105692] Updated weights for policy 0, policy_version 816577 (0.0009) [2023-12-26 21:18:22,066][105620] Updated weights for policy 1, policy_version 816398 (0.0009) [2023-12-26 21:18:22,123][105620] Updated weights for policy 1, policy_version 816408 (0.0009) [2023-12-26 21:18:22,181][105620] Updated weights for policy 1, policy_version 816418 (0.0010) [2023-12-26 21:18:22,699][105692] Updated weights for policy 0, policy_version 816587 (0.0009) [2023-12-26 21:18:22,748][105692] Updated weights for policy 0, policy_version 816597 (0.0009) [2023-12-26 21:18:22,800][105692] Updated weights for policy 0, policy_version 816607 (0.0009) [2023-12-26 21:18:22,933][105620] Updated weights for policy 1, policy_version 816428 (0.0009) [2023-12-26 21:18:23,002][105620] Updated weights for policy 1, policy_version 816438 (0.0009) [2023-12-26 21:18:23,059][105620] Updated weights for policy 1, policy_version 816448 (0.0009) [2023-12-26 21:18:23,589][105692] Updated weights for policy 0, policy_version 816617 (0.0009) [2023-12-26 21:18:23,640][105692] Updated weights for policy 0, policy_version 816627 (0.0009) [2023-12-26 21:18:23,693][105692] Updated weights for policy 0, policy_version 816637 (0.0009) [2023-12-26 21:18:23,747][105692] Updated weights for policy 0, policy_version 816647 (0.0008) [2023-12-26 21:18:23,796][105620] Updated weights for policy 1, policy_version 816458 (0.0009) [2023-12-26 21:18:23,847][105620] Updated weights for policy 1, policy_version 816468 (0.0009) [2023-12-26 21:18:23,909][105620] Updated weights for policy 1, policy_version 816478 (0.0009) [2023-12-26 21:18:23,967][105620] Updated weights for policy 1, policy_version 816488 (0.0009) [2023-12-26 21:18:24,465][105692] Updated weights for policy 0, policy_version 816657 (0.0006) [2023-12-26 21:18:24,517][105692] Updated weights for policy 0, policy_version 816667 (0.0006) [2023-12-26 21:18:24,564][105692] Updated weights for policy 0, policy_version 816677 (0.0008) [2023-12-26 21:18:24,762][105620] Updated weights for policy 1, policy_version 816498 (0.0010) [2023-12-26 21:18:24,830][105620] Updated weights for policy 1, policy_version 816508 (0.0010) [2023-12-26 21:18:24,891][105620] Updated weights for policy 1, policy_version 816518 (0.0010) [2023-12-26 21:18:25,316][105692] Updated weights for policy 0, policy_version 816687 (0.0008) [2023-12-26 21:18:25,376][105692] Updated weights for policy 0, policy_version 816697 (0.0008) [2023-12-26 21:18:25,432][105692] Updated weights for policy 0, policy_version 816707 (0.0008) [2023-12-26 21:18:25,600][105620] Updated weights for policy 1, policy_version 816528 (0.0010) [2023-12-26 21:18:25,658][105620] Updated weights for policy 1, policy_version 816538 (0.0010) [2023-12-26 21:18:25,706][105620] Updated weights for policy 1, policy_version 816548 (0.0010) [2023-12-26 21:18:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 418168832. Throughput: 0: 9739.2, 1: 9936.3. Samples: 418178960. Policy #0 lag: (min: 4.0, avg: 8.8, max: 36.0) [2023-12-26 21:18:26,063][104569] Avg episode reward: [(0, '8542.237'), (1, '9166.975')] [2023-12-26 21:18:26,084][105692] Updated weights for policy 0, policy_version 816717 (0.0007) [2023-12-26 21:18:26,147][105692] Updated weights for policy 0, policy_version 816727 (0.0006) [2023-12-26 21:18:26,213][105692] Updated weights for policy 0, policy_version 816737 (0.0005) [2023-12-26 21:18:26,379][105620] Updated weights for policy 1, policy_version 816558 (0.0010) [2023-12-26 21:18:26,430][105620] Updated weights for policy 1, policy_version 816568 (0.0010) [2023-12-26 21:18:26,484][105620] Updated weights for policy 1, policy_version 816578 (0.0010) [2023-12-26 21:18:26,787][105692] Updated weights for policy 0, policy_version 816747 (0.0005) [2023-12-26 21:18:26,836][105692] Updated weights for policy 0, policy_version 816757 (0.0005) [2023-12-26 21:18:26,893][105692] Updated weights for policy 0, policy_version 816767 (0.0005) [2023-12-26 21:18:27,074][105620] Updated weights for policy 1, policy_version 816588 (0.0007) [2023-12-26 21:18:27,127][105620] Updated weights for policy 1, policy_version 816598 (0.0005) [2023-12-26 21:18:27,173][105620] Updated weights for policy 1, policy_version 816608 (0.0005) [2023-12-26 21:18:27,554][105692] Updated weights for policy 0, policy_version 816777 (0.0006) [2023-12-26 21:18:27,604][105692] Updated weights for policy 0, policy_version 816787 (0.0008) [2023-12-26 21:18:27,661][105692] Updated weights for policy 0, policy_version 816797 (0.0009) [2023-12-26 21:18:27,722][105692] Updated weights for policy 0, policy_version 816807 (0.0005) [2023-12-26 21:18:27,803][105620] Updated weights for policy 1, policy_version 816618 (0.0005) [2023-12-26 21:18:27,857][105620] Updated weights for policy 1, policy_version 816628 (0.0005) [2023-12-26 21:18:27,912][105620] Updated weights for policy 1, policy_version 816638 (0.0010) [2023-12-26 21:18:27,957][105620] Updated weights for policy 1, policy_version 816648 (0.0008) [2023-12-26 21:18:28,423][105692] Updated weights for policy 0, policy_version 816817 (0.0008) [2023-12-26 21:18:28,484][105692] Updated weights for policy 0, policy_version 816827 (0.0009) [2023-12-26 21:18:28,542][105692] Updated weights for policy 0, policy_version 816837 (0.0009) [2023-12-26 21:18:28,679][105620] Updated weights for policy 1, policy_version 816658 (0.0009) [2023-12-26 21:18:28,730][105620] Updated weights for policy 1, policy_version 816668 (0.0009) [2023-12-26 21:18:28,778][105620] Updated weights for policy 1, policy_version 816678 (0.0009) [2023-12-26 21:18:29,194][105692] Updated weights for policy 0, policy_version 816847 (0.0007) [2023-12-26 21:18:29,254][105692] Updated weights for policy 0, policy_version 816857 (0.0009) [2023-12-26 21:18:29,305][105692] Updated weights for policy 0, policy_version 816867 (0.0008) [2023-12-26 21:18:29,630][105620] Updated weights for policy 1, policy_version 816688 (0.0007) [2023-12-26 21:18:29,687][105620] Updated weights for policy 1, policy_version 816698 (0.0006) [2023-12-26 21:18:29,752][105620] Updated weights for policy 1, policy_version 816708 (0.0005) [2023-12-26 21:18:30,089][105692] Updated weights for policy 0, policy_version 816878 (0.0009) [2023-12-26 21:18:30,146][105692] Updated weights for policy 0, policy_version 816888 (0.0010) [2023-12-26 21:18:30,199][105692] Updated weights for policy 0, policy_version 816898 (0.0010) [2023-12-26 21:18:30,364][105620] Updated weights for policy 1, policy_version 816718 (0.0008) [2023-12-26 21:18:30,424][105620] Updated weights for policy 1, policy_version 816728 (0.0007) [2023-12-26 21:18:30,506][105620] Updated weights for policy 1, policy_version 816738 (0.0007) [2023-12-26 21:18:30,984][105692] Updated weights for policy 0, policy_version 816908 (0.0009) [2023-12-26 21:18:31,040][105692] Updated weights for policy 0, policy_version 816918 (0.0009) [2023-12-26 21:18:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 418267136. Throughput: 0: 9807.7, 1: 9968.2. Samples: 418242712. Policy #0 lag: (min: 4.0, avg: 8.8, max: 36.0) [2023-12-26 21:18:31,063][104569] Avg episode reward: [(0, '8816.291'), (1, '9259.207')] [2023-12-26 21:18:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000816744_209108992.pth... [2023-12-26 21:18:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000815624_208822272.pth [2023-12-26 21:18:31,098][105692] Updated weights for policy 0, policy_version 816928 (0.0007) [2023-12-26 21:18:31,117][105620] Updated weights for policy 1, policy_version 816748 (0.0008) [2023-12-26 21:18:31,149][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000816936_209166336.pth... [2023-12-26 21:18:31,153][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000815784_208871424.pth [2023-12-26 21:18:31,188][105620] Updated weights for policy 1, policy_version 816758 (0.0009) [2023-12-26 21:18:31,246][105620] Updated weights for policy 1, policy_version 816768 (0.0008) [2023-12-26 21:18:31,891][105692] Updated weights for policy 0, policy_version 816938 (0.0009) [2023-12-26 21:18:31,951][105692] Updated weights for policy 0, policy_version 816948 (0.0009) [2023-12-26 21:18:31,981][105620] Updated weights for policy 1, policy_version 816778 (0.0008) [2023-12-26 21:18:32,009][105692] Updated weights for policy 0, policy_version 816958 (0.0007) [2023-12-26 21:18:32,043][105620] Updated weights for policy 1, policy_version 816788 (0.0006) [2023-12-26 21:18:32,069][105692] Updated weights for policy 0, policy_version 816968 (0.0007) [2023-12-26 21:18:32,104][105620] Updated weights for policy 1, policy_version 816798 (0.0006) [2023-12-26 21:18:32,167][105620] Updated weights for policy 1, policy_version 816808 (0.0008) [2023-12-26 21:18:32,804][105692] Updated weights for policy 0, policy_version 816978 (0.0009) [2023-12-26 21:18:32,865][105692] Updated weights for policy 0, policy_version 816988 (0.0009) [2023-12-26 21:18:32,919][105620] Updated weights for policy 1, policy_version 816818 (0.0007) [2023-12-26 21:18:32,921][105692] Updated weights for policy 0, policy_version 816998 (0.0007) [2023-12-26 21:18:32,967][105620] Updated weights for policy 1, policy_version 816828 (0.0007) [2023-12-26 21:18:33,023][105620] Updated weights for policy 1, policy_version 816838 (0.0008) [2023-12-26 21:18:33,678][105692] Updated weights for policy 0, policy_version 817008 (0.0010) [2023-12-26 21:18:33,731][105692] Updated weights for policy 0, policy_version 817018 (0.0009) [2023-12-26 21:18:33,736][105620] Updated weights for policy 1, policy_version 816848 (0.0006) [2023-12-26 21:18:33,790][105620] Updated weights for policy 1, policy_version 816858 (0.0005) [2023-12-26 21:18:33,793][105692] Updated weights for policy 0, policy_version 817028 (0.0009) [2023-12-26 21:18:33,841][105620] Updated weights for policy 1, policy_version 816868 (0.0008) [2023-12-26 21:18:34,522][105620] Updated weights for policy 1, policy_version 816878 (0.0010) [2023-12-26 21:18:34,583][105620] Updated weights for policy 1, policy_version 816888 (0.0011) [2023-12-26 21:18:34,589][105692] Updated weights for policy 0, policy_version 817038 (0.0007) [2023-12-26 21:18:34,643][105620] Updated weights for policy 1, policy_version 816898 (0.0011) [2023-12-26 21:18:34,646][105692] Updated weights for policy 0, policy_version 817048 (0.0006) [2023-12-26 21:18:34,705][105692] Updated weights for policy 0, policy_version 817058 (0.0007) [2023-12-26 21:18:35,306][105620] Updated weights for policy 1, policy_version 816908 (0.0008) [2023-12-26 21:18:35,372][105620] Updated weights for policy 1, policy_version 816918 (0.0005) [2023-12-26 21:18:35,432][105620] Updated weights for policy 1, policy_version 816928 (0.0006) [2023-12-26 21:18:35,540][105692] Updated weights for policy 0, policy_version 817068 (0.0009) [2023-12-26 21:18:35,598][105692] Updated weights for policy 0, policy_version 817078 (0.0009) [2023-12-26 21:18:35,645][105692] Updated weights for policy 0, policy_version 817088 (0.0009) [2023-12-26 21:18:36,002][105620] Updated weights for policy 1, policy_version 816938 (0.0007) [2023-12-26 21:18:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 418365440. Throughput: 0: 9801.0, 1: 9936.2. Samples: 418357788. Policy #0 lag: (min: 4.0, avg: 8.8, max: 36.0) [2023-12-26 21:18:36,062][104569] Avg episode reward: [(0, '8907.821'), (1, '9259.761')] [2023-12-26 21:18:36,067][105620] Updated weights for policy 1, policy_version 816948 (0.0005) [2023-12-26 21:18:36,138][105620] Updated weights for policy 1, policy_version 816958 (0.0007) [2023-12-26 21:18:36,184][105620] Updated weights for policy 1, policy_version 816968 (0.0006) [2023-12-26 21:18:36,393][105692] Updated weights for policy 0, policy_version 817098 (0.0009) [2023-12-26 21:18:36,459][105692] Updated weights for policy 0, policy_version 817108 (0.0009) [2023-12-26 21:18:36,532][105692] Updated weights for policy 0, policy_version 817118 (0.0010) [2023-12-26 21:18:36,592][105692] Updated weights for policy 0, policy_version 817128 (0.0009) [2023-12-26 21:18:36,761][105620] Updated weights for policy 1, policy_version 816978 (0.0006) [2023-12-26 21:18:36,823][105620] Updated weights for policy 1, policy_version 816988 (0.0006) [2023-12-26 21:18:36,879][105620] Updated weights for policy 1, policy_version 816998 (0.0005) [2023-12-26 21:18:37,345][105692] Updated weights for policy 0, policy_version 817138 (0.0010) [2023-12-26 21:18:37,393][105692] Updated weights for policy 0, policy_version 817148 (0.0009) [2023-12-26 21:18:37,394][105620] Updated weights for policy 1, policy_version 817008 (0.0005) [2023-12-26 21:18:37,442][105692] Updated weights for policy 0, policy_version 817158 (0.0008) [2023-12-26 21:18:37,446][105620] Updated weights for policy 1, policy_version 817018 (0.0005) [2023-12-26 21:18:37,502][105620] Updated weights for policy 1, policy_version 817028 (0.0005) [2023-12-26 21:18:38,083][105620] Updated weights for policy 1, policy_version 817038 (0.0007) [2023-12-26 21:18:38,138][105620] Updated weights for policy 1, policy_version 817048 (0.0009) [2023-12-26 21:18:38,185][105620] Updated weights for policy 1, policy_version 817058 (0.0008) [2023-12-26 21:18:38,297][105692] Updated weights for policy 0, policy_version 817168 (0.0007) [2023-12-26 21:18:38,359][105692] Updated weights for policy 0, policy_version 817178 (0.0009) [2023-12-26 21:18:38,419][105692] Updated weights for policy 0, policy_version 817188 (0.0009) [2023-12-26 21:18:38,897][105620] Updated weights for policy 1, policy_version 817068 (0.0008) [2023-12-26 21:18:38,963][105620] Updated weights for policy 1, policy_version 817078 (0.0006) [2023-12-26 21:18:39,022][105620] Updated weights for policy 1, policy_version 817088 (0.0006) [2023-12-26 21:18:39,238][105692] Updated weights for policy 0, policy_version 817198 (0.0008) [2023-12-26 21:18:39,302][105692] Updated weights for policy 0, policy_version 817208 (0.0008) [2023-12-26 21:18:39,364][105692] Updated weights for policy 0, policy_version 817218 (0.0008) [2023-12-26 21:18:39,714][105620] Updated weights for policy 1, policy_version 817098 (0.0006) [2023-12-26 21:18:39,779][105620] Updated weights for policy 1, policy_version 817108 (0.0008) [2023-12-26 21:18:39,845][105620] Updated weights for policy 1, policy_version 817118 (0.0009) [2023-12-26 21:18:39,859][105586] KL-divergence is very high: 193.3600 [2023-12-26 21:18:39,908][105620] Updated weights for policy 1, policy_version 817128 (0.0008) [2023-12-26 21:18:40,171][105692] Updated weights for policy 0, policy_version 817228 (0.0008) [2023-12-26 21:18:40,230][105692] Updated weights for policy 0, policy_version 817238 (0.0010) [2023-12-26 21:18:40,297][105692] Updated weights for policy 0, policy_version 817248 (0.0009) [2023-12-26 21:18:40,624][105620] Updated weights for policy 1, policy_version 817138 (0.0009) [2023-12-26 21:18:40,675][105620] Updated weights for policy 1, policy_version 817148 (0.0008) [2023-12-26 21:18:40,729][105620] Updated weights for policy 1, policy_version 817158 (0.0009) [2023-12-26 21:18:40,992][105692] Updated weights for policy 0, policy_version 817258 (0.0010) [2023-12-26 21:18:41,057][105692] Updated weights for policy 0, policy_version 817268 (0.0009) [2023-12-26 21:18:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 418463744. Throughput: 0: 9594.0, 1: 9954.0. Samples: 418474720. Policy #0 lag: (min: 4.0, avg: 8.8, max: 36.0) [2023-12-26 21:18:41,063][104569] Avg episode reward: [(0, '8731.916'), (1, '8892.865')] [2023-12-26 21:18:41,115][105692] Updated weights for policy 0, policy_version 817278 (0.0009) [2023-12-26 21:18:41,176][105692] Updated weights for policy 0, policy_version 817288 (0.0009) [2023-12-26 21:18:41,527][105620] Updated weights for policy 1, policy_version 817168 (0.0009) [2023-12-26 21:18:41,582][105620] Updated weights for policy 1, policy_version 817178 (0.0010) [2023-12-26 21:18:41,648][105620] Updated weights for policy 1, policy_version 817188 (0.0009) [2023-12-26 21:18:41,929][105692] Updated weights for policy 0, policy_version 817298 (0.0009) [2023-12-26 21:18:41,997][105692] Updated weights for policy 0, policy_version 817308 (0.0009) [2023-12-26 21:18:42,060][105692] Updated weights for policy 0, policy_version 817318 (0.0009) [2023-12-26 21:18:42,423][105620] Updated weights for policy 1, policy_version 817198 (0.0009) [2023-12-26 21:18:42,478][105620] Updated weights for policy 1, policy_version 817208 (0.0009) [2023-12-26 21:18:42,533][105620] Updated weights for policy 1, policy_version 817218 (0.0009) [2023-12-26 21:18:42,800][105692] Updated weights for policy 0, policy_version 817328 (0.0009) [2023-12-26 21:18:42,855][105692] Updated weights for policy 0, policy_version 817338 (0.0009) [2023-12-26 21:18:42,910][105692] Updated weights for policy 0, policy_version 817348 (0.0009) [2023-12-26 21:18:43,278][105620] Updated weights for policy 1, policy_version 817228 (0.0007) [2023-12-26 21:18:43,332][105620] Updated weights for policy 1, policy_version 817238 (0.0005) [2023-12-26 21:18:43,397][105620] Updated weights for policy 1, policy_version 817248 (0.0005) [2023-12-26 21:18:43,718][105692] Updated weights for policy 0, policy_version 817358 (0.0009) [2023-12-26 21:18:43,769][105692] Updated weights for policy 0, policy_version 817368 (0.0009) [2023-12-26 21:18:43,832][105692] Updated weights for policy 0, policy_version 817378 (0.0009) [2023-12-26 21:18:44,086][105620] Updated weights for policy 1, policy_version 817258 (0.0008) [2023-12-26 21:18:44,148][105620] Updated weights for policy 1, policy_version 817268 (0.0009) [2023-12-26 21:18:44,206][105620] Updated weights for policy 1, policy_version 817278 (0.0009) [2023-12-26 21:18:44,263][105620] Updated weights for policy 1, policy_version 817288 (0.0008) [2023-12-26 21:18:44,588][105692] Updated weights for policy 0, policy_version 817388 (0.0009) [2023-12-26 21:18:44,643][105692] Updated weights for policy 0, policy_version 817398 (0.0009) [2023-12-26 21:18:44,701][105692] Updated weights for policy 0, policy_version 817409 (0.0010) [2023-12-26 21:18:44,973][105620] Updated weights for policy 1, policy_version 817298 (0.0009) [2023-12-26 21:18:45,032][105620] Updated weights for policy 1, policy_version 817308 (0.0009) [2023-12-26 21:18:45,094][105620] Updated weights for policy 1, policy_version 817318 (0.0009) [2023-12-26 21:18:45,501][105692] Updated weights for policy 0, policy_version 817419 (0.0009) [2023-12-26 21:18:45,561][105692] Updated weights for policy 0, policy_version 817429 (0.0009) [2023-12-26 21:18:45,619][105692] Updated weights for policy 0, policy_version 817439 (0.0009) [2023-12-26 21:18:45,842][105620] Updated weights for policy 1, policy_version 817328 (0.0008) [2023-12-26 21:18:45,878][105586] KL-divergence is very high: 146.6158 [2023-12-26 21:18:45,897][105620] Updated weights for policy 1, policy_version 817338 (0.0009) [2023-12-26 21:18:45,926][105586] KL-divergence is very high: 220.4688 [2023-12-26 21:18:45,961][105620] Updated weights for policy 1, policy_version 817348 (0.0009) [2023-12-26 21:18:45,980][105586] KL-divergence is very high: 200.5638 [2023-12-26 21:18:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19605.2). Total num frames: 418562048. Throughput: 0: 9558.7, 1: 9905.9. Samples: 418530632. Policy #0 lag: (min: 4.0, avg: 8.8, max: 36.0) [2023-12-26 21:18:46,063][104569] Avg episode reward: [(0, '8465.047'), (1, '8892.793')] [2023-12-26 21:18:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000817352_209264640.pth... [2023-12-26 21:18:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000817448_209297408.pth... [2023-12-26 21:18:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000816360_209018880.pth [2023-12-26 21:18:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000816168_208961536.pth [2023-12-26 21:18:46,402][105692] Updated weights for policy 0, policy_version 817449 (0.0009) [2023-12-26 21:18:46,460][105692] Updated weights for policy 0, policy_version 817459 (0.0009) [2023-12-26 21:18:46,522][105692] Updated weights for policy 0, policy_version 817469 (0.0010) [2023-12-26 21:18:46,586][105692] Updated weights for policy 0, policy_version 817479 (0.0009) [2023-12-26 21:18:46,596][105620] Updated weights for policy 1, policy_version 817358 (0.0010) [2023-12-26 21:18:46,647][105620] Updated weights for policy 1, policy_version 817368 (0.0010) [2023-12-26 21:18:46,706][105620] Updated weights for policy 1, policy_version 817378 (0.0010) [2023-12-26 21:18:47,351][105692] Updated weights for policy 0, policy_version 817489 (0.0008) [2023-12-26 21:18:47,410][105692] Updated weights for policy 0, policy_version 817499 (0.0008) [2023-12-26 21:18:47,455][105620] Updated weights for policy 1, policy_version 817388 (0.0010) [2023-12-26 21:18:47,469][105692] Updated weights for policy 0, policy_version 817509 (0.0006) [2023-12-26 21:18:47,510][105620] Updated weights for policy 1, policy_version 817398 (0.0010) [2023-12-26 21:18:47,568][105620] Updated weights for policy 1, policy_version 817408 (0.0009) [2023-12-26 21:18:48,183][105620] Updated weights for policy 1, policy_version 817418 (0.0008) [2023-12-26 21:18:48,254][105620] Updated weights for policy 1, policy_version 817428 (0.0010) [2023-12-26 21:18:48,299][105692] Updated weights for policy 0, policy_version 817519 (0.0008) [2023-12-26 21:18:48,309][105620] Updated weights for policy 1, policy_version 817438 (0.0010) [2023-12-26 21:18:48,362][105692] Updated weights for policy 0, policy_version 817529 (0.0007) [2023-12-26 21:18:48,369][105620] Updated weights for policy 1, policy_version 817448 (0.0009) [2023-12-26 21:18:48,422][105692] Updated weights for policy 0, policy_version 817540 (0.0010) [2023-12-26 21:18:49,080][105692] Updated weights for policy 0, policy_version 817550 (0.0008) [2023-12-26 21:18:49,109][105620] Updated weights for policy 1, policy_version 817458 (0.0005) [2023-12-26 21:18:49,142][105692] Updated weights for policy 0, policy_version 817560 (0.0008) [2023-12-26 21:18:49,163][105620] Updated weights for policy 1, policy_version 817468 (0.0006) [2023-12-26 21:18:49,203][105692] Updated weights for policy 0, policy_version 817570 (0.0008) [2023-12-26 21:18:49,229][105620] Updated weights for policy 1, policy_version 817478 (0.0006) [2023-12-26 21:18:49,855][105692] Updated weights for policy 0, policy_version 817580 (0.0008) [2023-12-26 21:18:49,914][105692] Updated weights for policy 0, policy_version 817590 (0.0008) [2023-12-26 21:18:49,926][105620] Updated weights for policy 1, policy_version 817488 (0.0009) [2023-12-26 21:18:49,984][105692] Updated weights for policy 0, policy_version 817600 (0.0008) [2023-12-26 21:18:49,991][105620] Updated weights for policy 1, policy_version 817498 (0.0007) [2023-12-26 21:18:50,059][105620] Updated weights for policy 1, policy_version 817508 (0.0009) [2023-12-26 21:18:50,593][105692] Updated weights for policy 0, policy_version 817610 (0.0007) [2023-12-26 21:18:50,658][105692] Updated weights for policy 0, policy_version 817620 (0.0008) [2023-12-26 21:18:50,715][105692] Updated weights for policy 0, policy_version 817630 (0.0009) [2023-12-26 21:18:50,775][105692] Updated weights for policy 0, policy_version 817640 (0.0009) [2023-12-26 21:18:50,857][105620] Updated weights for policy 1, policy_version 817518 (0.0009) [2023-12-26 21:18:50,923][105620] Updated weights for policy 1, policy_version 817528 (0.0010) [2023-12-26 21:18:50,982][105620] Updated weights for policy 1, policy_version 817538 (0.0011) [2023-12-26 21:18:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 418660352. Throughput: 0: 9462.1, 1: 9918.5. Samples: 418645244. Policy #0 lag: (min: 4.0, avg: 8.8, max: 36.0) [2023-12-26 21:18:51,062][104569] Avg episode reward: [(0, '8728.085'), (1, '9074.712')] [2023-12-26 21:18:51,524][105692] Updated weights for policy 0, policy_version 817650 (0.0009) [2023-12-26 21:18:51,583][105692] Updated weights for policy 0, policy_version 817660 (0.0011) [2023-12-26 21:18:51,645][105692] Updated weights for policy 0, policy_version 817670 (0.0009) [2023-12-26 21:18:51,738][105620] Updated weights for policy 1, policy_version 817548 (0.0011) [2023-12-26 21:18:51,804][105620] Updated weights for policy 1, policy_version 817558 (0.0008) [2023-12-26 21:18:51,865][105620] Updated weights for policy 1, policy_version 817568 (0.0008) [2023-12-26 21:18:52,353][105692] Updated weights for policy 0, policy_version 817680 (0.0006) [2023-12-26 21:18:52,408][105692] Updated weights for policy 0, policy_version 817690 (0.0010) [2023-12-26 21:18:52,457][105692] Updated weights for policy 0, policy_version 817700 (0.0011) [2023-12-26 21:18:52,686][105620] Updated weights for policy 1, policy_version 817578 (0.0009) [2023-12-26 21:18:52,738][105620] Updated weights for policy 1, policy_version 817588 (0.0009) [2023-12-26 21:18:52,802][105620] Updated weights for policy 1, policy_version 817598 (0.0008) [2023-12-26 21:18:52,854][105620] Updated weights for policy 1, policy_version 817608 (0.0006) [2023-12-26 21:18:53,174][105692] Updated weights for policy 0, policy_version 817710 (0.0011) [2023-12-26 21:18:53,238][105692] Updated weights for policy 0, policy_version 817720 (0.0010) [2023-12-26 21:18:53,309][105692] Updated weights for policy 0, policy_version 817730 (0.0010) [2023-12-26 21:18:53,437][105620] Updated weights for policy 1, policy_version 817618 (0.0005) [2023-12-26 21:18:53,491][105620] Updated weights for policy 1, policy_version 817628 (0.0006) [2023-12-26 21:18:53,545][105620] Updated weights for policy 1, policy_version 817638 (0.0005) [2023-12-26 21:18:53,905][105692] Updated weights for policy 0, policy_version 817740 (0.0009) [2023-12-26 21:18:53,961][105692] Updated weights for policy 0, policy_version 817750 (0.0005) [2023-12-26 21:18:54,019][105692] Updated weights for policy 0, policy_version 817760 (0.0005) [2023-12-26 21:18:54,214][105620] Updated weights for policy 1, policy_version 817648 (0.0007) [2023-12-26 21:18:54,274][105620] Updated weights for policy 1, policy_version 817658 (0.0008) [2023-12-26 21:18:54,337][105620] Updated weights for policy 1, policy_version 817668 (0.0008) [2023-12-26 21:18:54,724][105692] Updated weights for policy 0, policy_version 817770 (0.0005) [2023-12-26 21:18:54,777][105692] Updated weights for policy 0, policy_version 817780 (0.0005) [2023-12-26 21:18:54,827][105692] Updated weights for policy 0, policy_version 817790 (0.0006) [2023-12-26 21:18:54,882][105692] Updated weights for policy 0, policy_version 817800 (0.0005) [2023-12-26 21:18:55,010][105620] Updated weights for policy 1, policy_version 817678 (0.0007) [2023-12-26 21:18:55,071][105620] Updated weights for policy 1, policy_version 817688 (0.0009) [2023-12-26 21:18:55,130][105620] Updated weights for policy 1, policy_version 817698 (0.0011) [2023-12-26 21:18:55,506][105692] Updated weights for policy 0, policy_version 817810 (0.0008) [2023-12-26 21:18:55,574][105692] Updated weights for policy 0, policy_version 817820 (0.0008) [2023-12-26 21:18:55,636][105692] Updated weights for policy 0, policy_version 817830 (0.0008) [2023-12-26 21:18:55,813][105620] Updated weights for policy 1, policy_version 817708 (0.0009) [2023-12-26 21:18:55,871][105620] Updated weights for policy 1, policy_version 817718 (0.0005) [2023-12-26 21:18:55,937][105620] Updated weights for policy 1, policy_version 817728 (0.0006) [2023-12-26 21:18:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 418758656. Throughput: 0: 9497.5, 1: 9942.8. Samples: 418765208. Policy #0 lag: (min: 4.0, avg: 8.8, max: 36.0) [2023-12-26 21:18:56,063][104569] Avg episode reward: [(0, '8811.522'), (1, '9073.068')] [2023-12-26 21:18:56,385][105692] Updated weights for policy 0, policy_version 817840 (0.0009) [2023-12-26 21:18:56,436][105692] Updated weights for policy 0, policy_version 817850 (0.0008) [2023-12-26 21:18:56,496][105692] Updated weights for policy 0, policy_version 817860 (0.0006) [2023-12-26 21:18:56,607][105620] Updated weights for policy 1, policy_version 817738 (0.0006) [2023-12-26 21:18:56,655][105620] Updated weights for policy 1, policy_version 817748 (0.0009) [2023-12-26 21:18:56,701][105620] Updated weights for policy 1, policy_version 817758 (0.0008) [2023-12-26 21:18:56,747][105620] Updated weights for policy 1, policy_version 817768 (0.0008) [2023-12-26 21:18:57,072][105692] Updated weights for policy 0, policy_version 817870 (0.0008) [2023-12-26 21:18:57,125][105692] Updated weights for policy 0, policy_version 817880 (0.0010) [2023-12-26 21:18:57,177][105692] Updated weights for policy 0, policy_version 817891 (0.0009) [2023-12-26 21:18:57,530][105620] Updated weights for policy 1, policy_version 817778 (0.0005) [2023-12-26 21:18:57,580][105620] Updated weights for policy 1, policy_version 817788 (0.0005) [2023-12-26 21:18:57,639][105620] Updated weights for policy 1, policy_version 817798 (0.0006) [2023-12-26 21:18:57,936][105692] Updated weights for policy 0, policy_version 817901 (0.0009) [2023-12-26 21:18:57,990][105692] Updated weights for policy 0, policy_version 817911 (0.0010) [2023-12-26 21:18:58,047][105692] Updated weights for policy 0, policy_version 817921 (0.0010) [2023-12-26 21:18:58,351][105620] Updated weights for policy 1, policy_version 817808 (0.0008) [2023-12-26 21:18:58,420][105620] Updated weights for policy 1, policy_version 817818 (0.0008) [2023-12-26 21:18:58,479][105620] Updated weights for policy 1, policy_version 817828 (0.0010) [2023-12-26 21:18:58,864][105692] Updated weights for policy 0, policy_version 817931 (0.0010) [2023-12-26 21:18:58,939][105692] Updated weights for policy 0, policy_version 817941 (0.0009) [2023-12-26 21:18:58,998][105692] Updated weights for policy 0, policy_version 817951 (0.0006) [2023-12-26 21:18:59,169][105620] Updated weights for policy 1, policy_version 817838 (0.0008) [2023-12-26 21:18:59,243][105620] Updated weights for policy 1, policy_version 817848 (0.0008) [2023-12-26 21:18:59,314][105620] Updated weights for policy 1, policy_version 817858 (0.0010) [2023-12-26 21:18:59,637][105692] Updated weights for policy 0, policy_version 817961 (0.0006) [2023-12-26 21:18:59,701][105692] Updated weights for policy 0, policy_version 817971 (0.0009) [2023-12-26 21:18:59,756][105692] Updated weights for policy 0, policy_version 817981 (0.0009) [2023-12-26 21:18:59,803][105692] Updated weights for policy 0, policy_version 817991 (0.0009) [2023-12-26 21:19:00,088][105620] Updated weights for policy 1, policy_version 817868 (0.0010) [2023-12-26 21:19:00,141][105620] Updated weights for policy 1, policy_version 817878 (0.0010) [2023-12-26 21:19:00,203][105620] Updated weights for policy 1, policy_version 817888 (0.0009) [2023-12-26 21:19:00,449][105692] Updated weights for policy 0, policy_version 818001 (0.0006) [2023-12-26 21:19:00,516][105692] Updated weights for policy 0, policy_version 818011 (0.0005) [2023-12-26 21:19:00,578][105692] Updated weights for policy 0, policy_version 818021 (0.0008) [2023-12-26 21:19:00,981][105620] Updated weights for policy 1, policy_version 817898 (0.0011) [2023-12-26 21:19:01,051][105620] Updated weights for policy 1, policy_version 817908 (0.0011) [2023-12-26 21:19:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 418848768. Throughput: 0: 9571.5, 1: 9915.9. Samples: 418822868. Policy #0 lag: (min: 4.0, avg: 8.8, max: 36.0) [2023-12-26 21:19:01,062][104569] Avg episode reward: [(0, '8723.035'), (1, '8982.211')] [2023-12-26 21:19:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000818024_209444864.pth... [2023-12-26 21:19:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000816936_209166336.pth [2023-12-26 21:19:01,113][105620] Updated weights for policy 1, policy_version 817918 (0.0006) [2023-12-26 21:19:01,179][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000817928_209412096.pth... [2023-12-26 21:19:01,180][105620] Updated weights for policy 1, policy_version 817928 (0.0007) [2023-12-26 21:19:01,182][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000816744_209108992.pth [2023-12-26 21:19:01,215][105692] Updated weights for policy 0, policy_version 818031 (0.0009) [2023-12-26 21:19:01,276][105692] Updated weights for policy 0, policy_version 818041 (0.0009) [2023-12-26 21:19:01,334][105692] Updated weights for policy 0, policy_version 818051 (0.0008) [2023-12-26 21:19:01,828][105620] Updated weights for policy 1, policy_version 817938 (0.0010) [2023-12-26 21:19:01,889][105620] Updated weights for policy 1, policy_version 817948 (0.0009) [2023-12-26 21:19:01,946][105620] Updated weights for policy 1, policy_version 817958 (0.0010) [2023-12-26 21:19:02,076][105692] Updated weights for policy 0, policy_version 818061 (0.0007) [2023-12-26 21:19:02,143][105692] Updated weights for policy 0, policy_version 818071 (0.0008) [2023-12-26 21:19:02,202][105692] Updated weights for policy 0, policy_version 818081 (0.0008) [2023-12-26 21:19:02,736][105620] Updated weights for policy 1, policy_version 817968 (0.0008) [2023-12-26 21:19:02,792][105620] Updated weights for policy 1, policy_version 817978 (0.0007) [2023-12-26 21:19:02,855][105620] Updated weights for policy 1, policy_version 817988 (0.0010) [2023-12-26 21:19:02,937][105692] Updated weights for policy 0, policy_version 818091 (0.0006) [2023-12-26 21:19:02,997][105692] Updated weights for policy 0, policy_version 818101 (0.0008) [2023-12-26 21:19:03,047][105692] Updated weights for policy 0, policy_version 818111 (0.0009) [2023-12-26 21:19:03,498][105620] Updated weights for policy 1, policy_version 817998 (0.0007) [2023-12-26 21:19:03,549][105620] Updated weights for policy 1, policy_version 818009 (0.0010) [2023-12-26 21:19:03,588][105692] Updated weights for policy 0, policy_version 818121 (0.0006) [2023-12-26 21:19:03,600][105620] Updated weights for policy 1, policy_version 818019 (0.0008) [2023-12-26 21:19:03,643][105692] Updated weights for policy 0, policy_version 818131 (0.0005) [2023-12-26 21:19:03,701][105692] Updated weights for policy 0, policy_version 818141 (0.0005) [2023-12-26 21:19:03,757][105692] Updated weights for policy 0, policy_version 818151 (0.0005) [2023-12-26 21:19:04,380][105620] Updated weights for policy 1, policy_version 818029 (0.0009) [2023-12-26 21:19:04,444][105620] Updated weights for policy 1, policy_version 818039 (0.0009) [2023-12-26 21:19:04,475][105692] Updated weights for policy 0, policy_version 818161 (0.0005) [2023-12-26 21:19:04,509][105620] Updated weights for policy 1, policy_version 818049 (0.0009) [2023-12-26 21:19:04,535][105692] Updated weights for policy 0, policy_version 818171 (0.0006) [2023-12-26 21:19:04,591][105692] Updated weights for policy 0, policy_version 818181 (0.0005) [2023-12-26 21:19:05,232][105692] Updated weights for policy 0, policy_version 818191 (0.0005) [2023-12-26 21:19:05,284][105620] Updated weights for policy 1, policy_version 818059 (0.0009) [2023-12-26 21:19:05,302][105692] Updated weights for policy 0, policy_version 818201 (0.0005) [2023-12-26 21:19:05,345][105620] Updated weights for policy 1, policy_version 818069 (0.0009) [2023-12-26 21:19:05,356][105692] Updated weights for policy 0, policy_version 818211 (0.0005) [2023-12-26 21:19:05,398][105620] Updated weights for policy 1, policy_version 818079 (0.0009) [2023-12-26 21:19:06,045][105692] Updated weights for policy 0, policy_version 818221 (0.0007) [2023-12-26 21:19:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 418947072. Throughput: 0: 9613.7, 1: 9838.8. Samples: 418940820. Policy #0 lag: (min: 4.0, avg: 8.8, max: 36.0) [2023-12-26 21:19:06,063][104569] Avg episode reward: [(0, '9083.566'), (1, '8983.675')] [2023-12-26 21:19:06,069][105620] Updated weights for policy 1, policy_version 818089 (0.0007) [2023-12-26 21:19:06,105][105692] Updated weights for policy 0, policy_version 818231 (0.0007) [2023-12-26 21:19:06,125][105620] Updated weights for policy 1, policy_version 818099 (0.0007) [2023-12-26 21:19:06,172][105692] Updated weights for policy 0, policy_version 818241 (0.0009) [2023-12-26 21:19:06,185][105620] Updated weights for policy 1, policy_version 818109 (0.0007) [2023-12-26 21:19:06,240][105620] Updated weights for policy 1, policy_version 818119 (0.0005) [2023-12-26 21:19:06,858][105620] Updated weights for policy 1, policy_version 818129 (0.0008) [2023-12-26 21:19:06,917][105620] Updated weights for policy 1, policy_version 818139 (0.0009) [2023-12-26 21:19:06,973][105620] Updated weights for policy 1, policy_version 818149 (0.0006) [2023-12-26 21:19:07,001][105692] Updated weights for policy 0, policy_version 818251 (0.0008) [2023-12-26 21:19:07,060][105692] Updated weights for policy 0, policy_version 818261 (0.0009) [2023-12-26 21:19:07,108][105692] Updated weights for policy 0, policy_version 818271 (0.0008) [2023-12-26 21:19:07,612][105620] Updated weights for policy 1, policy_version 818159 (0.0010) [2023-12-26 21:19:07,660][105620] Updated weights for policy 1, policy_version 818169 (0.0010) [2023-12-26 21:19:07,717][105620] Updated weights for policy 1, policy_version 818179 (0.0010) [2023-12-26 21:19:07,830][105692] Updated weights for policy 0, policy_version 818281 (0.0009) [2023-12-26 21:19:07,886][105692] Updated weights for policy 0, policy_version 818291 (0.0005) [2023-12-26 21:19:07,936][105692] Updated weights for policy 0, policy_version 818301 (0.0005) [2023-12-26 21:19:07,989][105692] Updated weights for policy 0, policy_version 818311 (0.0005) [2023-12-26 21:19:08,361][105620] Updated weights for policy 1, policy_version 818189 (0.0009) [2023-12-26 21:19:08,425][105620] Updated weights for policy 1, policy_version 818199 (0.0008) [2023-12-26 21:19:08,479][105620] Updated weights for policy 1, policy_version 818209 (0.0007) [2023-12-26 21:19:08,646][105692] Updated weights for policy 0, policy_version 818321 (0.0010) [2023-12-26 21:19:08,708][105692] Updated weights for policy 0, policy_version 818331 (0.0010) [2023-12-26 21:19:08,770][105692] Updated weights for policy 0, policy_version 818341 (0.0009) [2023-12-26 21:19:09,245][105620] Updated weights for policy 1, policy_version 818219 (0.0009) [2023-12-26 21:19:09,303][105620] Updated weights for policy 1, policy_version 818229 (0.0010) [2023-12-26 21:19:09,361][105620] Updated weights for policy 1, policy_version 818239 (0.0009) [2023-12-26 21:19:09,490][105692] Updated weights for policy 0, policy_version 818351 (0.0009) [2023-12-26 21:19:09,550][105692] Updated weights for policy 0, policy_version 818361 (0.0009) [2023-12-26 21:19:09,610][105692] Updated weights for policy 0, policy_version 818371 (0.0009) [2023-12-26 21:19:10,114][105620] Updated weights for policy 1, policy_version 818249 (0.0009) [2023-12-26 21:19:10,180][105620] Updated weights for policy 1, policy_version 818259 (0.0009) [2023-12-26 21:19:10,245][105620] Updated weights for policy 1, policy_version 818269 (0.0008) [2023-12-26 21:19:10,299][105620] Updated weights for policy 1, policy_version 818279 (0.0009) [2023-12-26 21:19:10,356][105692] Updated weights for policy 0, policy_version 818381 (0.0009) [2023-12-26 21:19:10,430][105692] Updated weights for policy 0, policy_version 818391 (0.0009) [2023-12-26 21:19:10,485][105692] Updated weights for policy 0, policy_version 818401 (0.0009) [2023-12-26 21:19:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 419045376. Throughput: 0: 9615.1, 1: 9909.8. Samples: 419057580. Policy #0 lag: (min: 4.0, avg: 8.8, max: 36.0) [2023-12-26 21:19:11,062][104569] Avg episode reward: [(0, '9173.169'), (1, '9166.729')] [2023-12-26 21:19:11,073][105620] Updated weights for policy 1, policy_version 818289 (0.0009) [2023-12-26 21:19:11,140][105620] Updated weights for policy 1, policy_version 818299 (0.0009) [2023-12-26 21:19:11,201][105620] Updated weights for policy 1, policy_version 818309 (0.0008) [2023-12-26 21:19:11,260][105692] Updated weights for policy 0, policy_version 818411 (0.0009) [2023-12-26 21:19:11,324][105692] Updated weights for policy 0, policy_version 818421 (0.0010) [2023-12-26 21:19:11,394][105692] Updated weights for policy 0, policy_version 818431 (0.0008) [2023-12-26 21:19:12,044][105620] Updated weights for policy 1, policy_version 818319 (0.0009) [2023-12-26 21:19:12,107][105620] Updated weights for policy 1, policy_version 818329 (0.0009) [2023-12-26 21:19:12,112][105692] Updated weights for policy 0, policy_version 818441 (0.0006) [2023-12-26 21:19:12,153][105620] Updated weights for policy 1, policy_version 818339 (0.0008) [2023-12-26 21:19:12,164][105692] Updated weights for policy 0, policy_version 818451 (0.0007) [2023-12-26 21:19:12,214][105692] Updated weights for policy 0, policy_version 818461 (0.0008) [2023-12-26 21:19:12,271][105692] Updated weights for policy 0, policy_version 818471 (0.0008) [2023-12-26 21:19:12,910][105620] Updated weights for policy 1, policy_version 818349 (0.0007) [2023-12-26 21:19:12,913][105692] Updated weights for policy 0, policy_version 818481 (0.0008) [2023-12-26 21:19:12,974][105692] Updated weights for policy 0, policy_version 818491 (0.0006) [2023-12-26 21:19:12,981][105620] Updated weights for policy 1, policy_version 818359 (0.0009) [2023-12-26 21:19:13,034][105692] Updated weights for policy 0, policy_version 818501 (0.0006) [2023-12-26 21:19:13,035][105620] Updated weights for policy 1, policy_version 818369 (0.0008) [2023-12-26 21:19:13,674][105620] Updated weights for policy 1, policy_version 818379 (0.0007) [2023-12-26 21:19:13,721][105620] Updated weights for policy 1, policy_version 818389 (0.0008) [2023-12-26 21:19:13,781][105620] Updated weights for policy 1, policy_version 818399 (0.0009) [2023-12-26 21:19:13,816][105692] Updated weights for policy 0, policy_version 818511 (0.0007) [2023-12-26 21:19:13,865][105692] Updated weights for policy 0, policy_version 818521 (0.0007) [2023-12-26 21:19:13,920][105692] Updated weights for policy 0, policy_version 818531 (0.0005) [2023-12-26 21:19:14,564][105620] Updated weights for policy 1, policy_version 818409 (0.0009) [2023-12-26 21:19:14,603][105692] Updated weights for policy 0, policy_version 818541 (0.0006) [2023-12-26 21:19:14,623][105620] Updated weights for policy 1, policy_version 818419 (0.0008) [2023-12-26 21:19:14,657][105692] Updated weights for policy 0, policy_version 818551 (0.0006) [2023-12-26 21:19:14,675][105620] Updated weights for policy 1, policy_version 818429 (0.0009) [2023-12-26 21:19:14,718][105692] Updated weights for policy 0, policy_version 818561 (0.0006) [2023-12-26 21:19:14,738][105620] Updated weights for policy 1, policy_version 818439 (0.0008) [2023-12-26 21:19:15,449][105692] Updated weights for policy 0, policy_version 818571 (0.0008) [2023-12-26 21:19:15,507][105692] Updated weights for policy 0, policy_version 818581 (0.0006) [2023-12-26 21:19:15,520][105620] Updated weights for policy 1, policy_version 818449 (0.0009) [2023-12-26 21:19:15,558][105692] Updated weights for policy 0, policy_version 818591 (0.0005) [2023-12-26 21:19:15,585][105620] Updated weights for policy 1, policy_version 818459 (0.0009) [2023-12-26 21:19:15,641][105620] Updated weights for policy 1, policy_version 818469 (0.0010) [2023-12-26 21:19:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.3, 300 sec: 19605.2). Total num frames: 419143680. Throughput: 0: 9546.8, 1: 9817.5. Samples: 419114108. Policy #0 lag: (min: 4.0, avg: 8.8, max: 36.0) [2023-12-26 21:19:16,063][104569] Avg episode reward: [(0, '9262.287'), (1, '8983.249')] [2023-12-26 21:19:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000818600_209592320.pth... [2023-12-26 21:19:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000818472_209551360.pth... [2023-12-26 21:19:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000817352_209264640.pth [2023-12-26 21:19:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000817448_209297408.pth [2023-12-26 21:19:16,231][105692] Updated weights for policy 0, policy_version 818601 (0.0006) [2023-12-26 21:19:16,292][105692] Updated weights for policy 0, policy_version 818611 (0.0009) [2023-12-26 21:19:16,343][105692] Updated weights for policy 0, policy_version 818621 (0.0010) [2023-12-26 21:19:16,363][105620] Updated weights for policy 1, policy_version 818480 (0.0007) [2023-12-26 21:19:16,399][105692] Updated weights for policy 0, policy_version 818631 (0.0008) [2023-12-26 21:19:16,414][105620] Updated weights for policy 1, policy_version 818490 (0.0008) [2023-12-26 21:19:16,471][105620] Updated weights for policy 1, policy_version 818500 (0.0009) [2023-12-26 21:19:17,171][105620] Updated weights for policy 1, policy_version 818510 (0.0009) [2023-12-26 21:19:17,197][105692] Updated weights for policy 0, policy_version 818641 (0.0006) [2023-12-26 21:19:17,227][105620] Updated weights for policy 1, policy_version 818520 (0.0011) [2023-12-26 21:19:17,253][105692] Updated weights for policy 0, policy_version 818651 (0.0006) [2023-12-26 21:19:17,285][105620] Updated weights for policy 1, policy_version 818530 (0.0011) [2023-12-26 21:19:17,304][105692] Updated weights for policy 0, policy_version 818661 (0.0006) [2023-12-26 21:19:17,959][105620] Updated weights for policy 1, policy_version 818540 (0.0009) [2023-12-26 21:19:18,026][105620] Updated weights for policy 1, policy_version 818550 (0.0006) [2023-12-26 21:19:18,055][105692] Updated weights for policy 0, policy_version 818671 (0.0009) [2023-12-26 21:19:18,087][105620] Updated weights for policy 1, policy_version 818560 (0.0006) [2023-12-26 21:19:18,113][105692] Updated weights for policy 0, policy_version 818681 (0.0008) [2023-12-26 21:19:18,168][105692] Updated weights for policy 0, policy_version 818691 (0.0005) [2023-12-26 21:19:18,760][105620] Updated weights for policy 1, policy_version 818570 (0.0007) [2023-12-26 21:19:18,765][105692] Updated weights for policy 0, policy_version 818701 (0.0008) [2023-12-26 21:19:18,817][105620] Updated weights for policy 1, policy_version 818580 (0.0011) [2023-12-26 21:19:18,821][105692] Updated weights for policy 0, policy_version 818711 (0.0010) [2023-12-26 21:19:18,873][105620] Updated weights for policy 1, policy_version 818590 (0.0010) [2023-12-26 21:19:18,879][105692] Updated weights for policy 0, policy_version 818721 (0.0010) [2023-12-26 21:19:18,926][105620] Updated weights for policy 1, policy_version 818600 (0.0011) [2023-12-26 21:19:19,575][105692] Updated weights for policy 0, policy_version 818731 (0.0010) [2023-12-26 21:19:19,626][105692] Updated weights for policy 0, policy_version 818741 (0.0008) [2023-12-26 21:19:19,680][105692] Updated weights for policy 0, policy_version 818751 (0.0008) [2023-12-26 21:19:19,694][105620] Updated weights for policy 1, policy_version 818610 (0.0010) [2023-12-26 21:19:19,757][105620] Updated weights for policy 1, policy_version 818620 (0.0011) [2023-12-26 21:19:19,821][105620] Updated weights for policy 1, policy_version 818630 (0.0010) [2023-12-26 21:19:20,430][105692] Updated weights for policy 0, policy_version 818761 (0.0006) [2023-12-26 21:19:20,499][105692] Updated weights for policy 0, policy_version 818771 (0.0009) [2023-12-26 21:19:20,560][105692] Updated weights for policy 0, policy_version 818781 (0.0008) [2023-12-26 21:19:20,566][105620] Updated weights for policy 1, policy_version 818640 (0.0009) [2023-12-26 21:19:20,623][105692] Updated weights for policy 0, policy_version 818791 (0.0007) [2023-12-26 21:19:20,633][105620] Updated weights for policy 1, policy_version 818650 (0.0007) [2023-12-26 21:19:20,691][105620] Updated weights for policy 1, policy_version 818660 (0.0009) [2023-12-26 21:19:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 419241984. Throughput: 0: 9634.0, 1: 9786.4. Samples: 419231704. Policy #0 lag: (min: 4.0, avg: 8.8, max: 36.0) [2023-12-26 21:19:21,062][104569] Avg episode reward: [(0, '9260.161'), (1, '8721.925')] [2023-12-26 21:19:21,402][105692] Updated weights for policy 0, policy_version 818801 (0.0008) [2023-12-26 21:19:21,427][105620] Updated weights for policy 1, policy_version 818670 (0.0008) [2023-12-26 21:19:21,465][105692] Updated weights for policy 0, policy_version 818811 (0.0008) [2023-12-26 21:19:21,486][105620] Updated weights for policy 1, policy_version 818680 (0.0009) [2023-12-26 21:19:21,522][105692] Updated weights for policy 0, policy_version 818821 (0.0008) [2023-12-26 21:19:21,541][105620] Updated weights for policy 1, policy_version 818690 (0.0007) [2023-12-26 21:19:22,192][105692] Updated weights for policy 0, policy_version 818831 (0.0007) [2023-12-26 21:19:22,260][105692] Updated weights for policy 0, policy_version 818841 (0.0007) [2023-12-26 21:19:22,332][105692] Updated weights for policy 0, policy_version 818851 (0.0007) [2023-12-26 21:19:22,345][105620] Updated weights for policy 1, policy_version 818700 (0.0009) [2023-12-26 21:19:22,417][105620] Updated weights for policy 1, policy_version 818710 (0.0008) [2023-12-26 21:19:22,487][105620] Updated weights for policy 1, policy_version 818720 (0.0008) [2023-12-26 21:19:22,904][105692] Updated weights for policy 0, policy_version 818861 (0.0007) [2023-12-26 21:19:22,958][105692] Updated weights for policy 0, policy_version 818871 (0.0005) [2023-12-26 21:19:23,023][105692] Updated weights for policy 0, policy_version 818881 (0.0005) [2023-12-26 21:19:23,233][105620] Updated weights for policy 1, policy_version 818730 (0.0008) [2023-12-26 21:19:23,292][105620] Updated weights for policy 1, policy_version 818740 (0.0005) [2023-12-26 21:19:23,350][105620] Updated weights for policy 1, policy_version 818750 (0.0006) [2023-12-26 21:19:23,420][105620] Updated weights for policy 1, policy_version 818760 (0.0010) [2023-12-26 21:19:23,552][105692] Updated weights for policy 0, policy_version 818891 (0.0007) [2023-12-26 21:19:23,606][105692] Updated weights for policy 0, policy_version 818901 (0.0005) [2023-12-26 21:19:23,677][105692] Updated weights for policy 0, policy_version 818911 (0.0005) [2023-12-26 21:19:23,993][105620] Updated weights for policy 1, policy_version 818770 (0.0005) [2023-12-26 21:19:24,055][105620] Updated weights for policy 1, policy_version 818780 (0.0005) [2023-12-26 21:19:24,118][105620] Updated weights for policy 1, policy_version 818790 (0.0005) [2023-12-26 21:19:24,197][105692] Updated weights for policy 0, policy_version 818921 (0.0005) [2023-12-26 21:19:24,259][105692] Updated weights for policy 0, policy_version 818931 (0.0008) [2023-12-26 21:19:24,309][105692] Updated weights for policy 0, policy_version 818941 (0.0008) [2023-12-26 21:19:24,362][105692] Updated weights for policy 0, policy_version 818951 (0.0009) [2023-12-26 21:19:24,792][105620] Updated weights for policy 1, policy_version 818800 (0.0005) [2023-12-26 21:19:24,838][105620] Updated weights for policy 1, policy_version 818810 (0.0005) [2023-12-26 21:19:24,890][105620] Updated weights for policy 1, policy_version 818820 (0.0010) [2023-12-26 21:19:24,996][105692] Updated weights for policy 0, policy_version 818961 (0.0006) [2023-12-26 21:19:25,056][105692] Updated weights for policy 0, policy_version 818971 (0.0005) [2023-12-26 21:19:25,111][105692] Updated weights for policy 0, policy_version 818981 (0.0005) [2023-12-26 21:19:25,671][105620] Updated weights for policy 1, policy_version 818830 (0.0007) [2023-12-26 21:19:25,727][105620] Updated weights for policy 1, policy_version 818840 (0.0005) [2023-12-26 21:19:25,737][105692] Updated weights for policy 0, policy_version 818991 (0.0008) [2023-12-26 21:19:25,779][105620] Updated weights for policy 1, policy_version 818850 (0.0005) [2023-12-26 21:19:25,785][105692] Updated weights for policy 0, policy_version 819001 (0.0007) [2023-12-26 21:19:25,832][105692] Updated weights for policy 0, policy_version 819011 (0.0007) [2023-12-26 21:19:26,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 419348480. Throughput: 0: 9895.0, 1: 9651.0. Samples: 419354292. Policy #0 lag: (min: 4.0, avg: 8.8, max: 36.0) [2023-12-26 21:19:26,062][104569] Avg episode reward: [(0, '9081.255'), (1, '8990.173')] [2023-12-26 21:19:26,474][105620] Updated weights for policy 1, policy_version 818860 (0.0007) [2023-12-26 21:19:26,532][105620] Updated weights for policy 1, policy_version 818870 (0.0010) [2023-12-26 21:19:26,586][105620] Updated weights for policy 1, policy_version 818880 (0.0010) [2023-12-26 21:19:26,640][105692] Updated weights for policy 0, policy_version 819021 (0.0008) [2023-12-26 21:19:26,701][105692] Updated weights for policy 0, policy_version 819031 (0.0008) [2023-12-26 21:19:26,763][105692] Updated weights for policy 0, policy_version 819041 (0.0008) [2023-12-26 21:19:27,247][105620] Updated weights for policy 1, policy_version 818890 (0.0010) [2023-12-26 21:19:27,299][105620] Updated weights for policy 1, policy_version 818900 (0.0010) [2023-12-26 21:19:27,342][105692] Updated weights for policy 0, policy_version 819051 (0.0007) [2023-12-26 21:19:27,358][105620] Updated weights for policy 1, policy_version 818910 (0.0011) [2023-12-26 21:19:27,390][105692] Updated weights for policy 0, policy_version 819061 (0.0005) [2023-12-26 21:19:27,413][105620] Updated weights for policy 1, policy_version 818920 (0.0010) [2023-12-26 21:19:27,438][105692] Updated weights for policy 0, policy_version 819071 (0.0005) [2023-12-26 21:19:28,014][105692] Updated weights for policy 0, policy_version 819081 (0.0005) [2023-12-26 21:19:28,056][105692] Updated weights for policy 0, policy_version 819091 (0.0007) [2023-12-26 21:19:28,103][105692] Updated weights for policy 0, policy_version 819101 (0.0007) [2023-12-26 21:19:28,148][105692] Updated weights for policy 0, policy_version 819111 (0.0005) [2023-12-26 21:19:28,220][105620] Updated weights for policy 1, policy_version 818930 (0.0010) [2023-12-26 21:19:28,271][105620] Updated weights for policy 1, policy_version 818940 (0.0010) [2023-12-26 21:19:28,332][105620] Updated weights for policy 1, policy_version 818950 (0.0010) [2023-12-26 21:19:28,903][105692] Updated weights for policy 0, policy_version 819121 (0.0010) [2023-12-26 21:19:28,964][105692] Updated weights for policy 0, policy_version 819131 (0.0010) [2023-12-26 21:19:28,984][105620] Updated weights for policy 1, policy_version 818960 (0.0010) [2023-12-26 21:19:29,018][105692] Updated weights for policy 0, policy_version 819141 (0.0010) [2023-12-26 21:19:29,035][105620] Updated weights for policy 1, policy_version 818970 (0.0010) [2023-12-26 21:19:29,093][105620] Updated weights for policy 1, policy_version 818980 (0.0010) [2023-12-26 21:19:29,728][105692] Updated weights for policy 0, policy_version 819151 (0.0010) [2023-12-26 21:19:29,772][105692] Updated weights for policy 0, policy_version 819161 (0.0010) [2023-12-26 21:19:29,825][105692] Updated weights for policy 0, policy_version 819171 (0.0010) [2023-12-26 21:19:29,831][105620] Updated weights for policy 1, policy_version 818990 (0.0010) [2023-12-26 21:19:29,892][105620] Updated weights for policy 1, policy_version 819000 (0.0008) [2023-12-26 21:19:29,954][105620] Updated weights for policy 1, policy_version 819010 (0.0008) [2023-12-26 21:19:30,602][105692] Updated weights for policy 0, policy_version 819181 (0.0010) [2023-12-26 21:19:30,664][105692] Updated weights for policy 0, policy_version 819191 (0.0010) [2023-12-26 21:19:30,666][105620] Updated weights for policy 1, policy_version 819020 (0.0009) [2023-12-26 21:19:30,716][105692] Updated weights for policy 0, policy_version 819201 (0.0010) [2023-12-26 21:19:30,721][105620] Updated weights for policy 1, policy_version 819030 (0.0010) [2023-12-26 21:19:30,779][105620] Updated weights for policy 1, policy_version 819040 (0.0010) [2023-12-26 21:19:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 419446784. Throughput: 0: 9969.9, 1: 9689.8. Samples: 419415316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:19:31,062][104569] Avg episode reward: [(0, '9083.527'), (1, '9080.646')] [2023-12-26 21:19:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000819208_209747968.pth... [2023-12-26 21:19:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000819048_209698816.pth... [2023-12-26 21:19:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000818024_209444864.pth [2023-12-26 21:19:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000817928_209412096.pth [2023-12-26 21:19:31,465][105692] Updated weights for policy 0, policy_version 819211 (0.0010) [2023-12-26 21:19:31,503][105620] Updated weights for policy 1, policy_version 819050 (0.0010) [2023-12-26 21:19:31,509][105692] Updated weights for policy 0, policy_version 819221 (0.0010) [2023-12-26 21:19:31,552][105620] Updated weights for policy 1, policy_version 819060 (0.0010) [2023-12-26 21:19:31,569][105692] Updated weights for policy 0, policy_version 819231 (0.0010) [2023-12-26 21:19:31,570][105586] KL-divergence is very high: 154.4767 [2023-12-26 21:19:31,604][105620] Updated weights for policy 1, policy_version 819070 (0.0010) [2023-12-26 21:19:31,619][105586] KL-divergence is very high: 162.1508 [2023-12-26 21:19:31,669][105620] Updated weights for policy 1, policy_version 819080 (0.0011) [2023-12-26 21:19:32,319][105620] Updated weights for policy 1, policy_version 819090 (0.0007) [2023-12-26 21:19:32,356][105692] Updated weights for policy 0, policy_version 819241 (0.0010) [2023-12-26 21:19:32,382][105620] Updated weights for policy 1, policy_version 819100 (0.0011) [2023-12-26 21:19:32,417][105692] Updated weights for policy 0, policy_version 819251 (0.0010) [2023-12-26 21:19:32,440][105620] Updated weights for policy 1, policy_version 819110 (0.0009) [2023-12-26 21:19:32,472][105692] Updated weights for policy 0, policy_version 819261 (0.0010) [2023-12-26 21:19:32,523][105692] Updated weights for policy 0, policy_version 819271 (0.0009) [2023-12-26 21:19:33,127][105620] Updated weights for policy 1, policy_version 819120 (0.0010) [2023-12-26 21:19:33,187][105620] Updated weights for policy 1, policy_version 819130 (0.0011) [2023-12-26 21:19:33,198][105692] Updated weights for policy 0, policy_version 819281 (0.0008) [2023-12-26 21:19:33,236][105620] Updated weights for policy 1, policy_version 819140 (0.0010) [2023-12-26 21:19:33,256][105692] Updated weights for policy 0, policy_version 819291 (0.0010) [2023-12-26 21:19:33,314][105692] Updated weights for policy 0, policy_version 819301 (0.0010) [2023-12-26 21:19:33,937][105692] Updated weights for policy 0, policy_version 819311 (0.0007) [2023-12-26 21:19:33,983][105692] Updated weights for policy 0, policy_version 819321 (0.0005) [2023-12-26 21:19:33,994][105620] Updated weights for policy 1, policy_version 819150 (0.0011) [2023-12-26 21:19:34,031][105692] Updated weights for policy 0, policy_version 819331 (0.0005) [2023-12-26 21:19:34,049][105620] Updated weights for policy 1, policy_version 819160 (0.0010) [2023-12-26 21:19:34,104][105620] Updated weights for policy 1, policy_version 819170 (0.0010) [2023-12-26 21:19:34,632][105692] Updated weights for policy 0, policy_version 819341 (0.0007) [2023-12-26 21:19:34,685][105692] Updated weights for policy 0, policy_version 819351 (0.0009) [2023-12-26 21:19:34,741][105692] Updated weights for policy 0, policy_version 819361 (0.0011) [2023-12-26 21:19:34,877][105620] Updated weights for policy 1, policy_version 819180 (0.0011) [2023-12-26 21:19:34,931][105620] Updated weights for policy 1, policy_version 819190 (0.0009) [2023-12-26 21:19:34,997][105620] Updated weights for policy 1, policy_version 819200 (0.0008) [2023-12-26 21:19:35,511][105692] Updated weights for policy 0, policy_version 819371 (0.0010) [2023-12-26 21:19:35,536][105620] Updated weights for policy 1, policy_version 819210 (0.0007) [2023-12-26 21:19:35,576][105692] Updated weights for policy 0, policy_version 819381 (0.0010) [2023-12-26 21:19:35,594][105620] Updated weights for policy 1, policy_version 819220 (0.0009) [2023-12-26 21:19:35,620][105692] Updated weights for policy 0, policy_version 819391 (0.0010) [2023-12-26 21:19:35,653][105620] Updated weights for policy 1, policy_version 819230 (0.0010) [2023-12-26 21:19:35,712][105620] Updated weights for policy 1, policy_version 819240 (0.0010) [2023-12-26 21:19:36,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.7, 300 sec: 19633.0). Total num frames: 419545088. Throughput: 0: 10058.6, 1: 9663.9. Samples: 419532764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:19:36,063][104569] Avg episode reward: [(0, '9262.222'), (1, '9080.736')] [2023-12-26 21:19:36,353][105692] Updated weights for policy 0, policy_version 819401 (0.0010) [2023-12-26 21:19:36,417][105620] Updated weights for policy 1, policy_version 819250 (0.0008) [2023-12-26 21:19:36,423][105692] Updated weights for policy 0, policy_version 819411 (0.0007) [2023-12-26 21:19:36,477][105620] Updated weights for policy 1, policy_version 819260 (0.0007) [2023-12-26 21:19:36,493][105692] Updated weights for policy 0, policy_version 819421 (0.0006) [2023-12-26 21:19:36,540][105620] Updated weights for policy 1, policy_version 819270 (0.0009) [2023-12-26 21:19:36,562][105692] Updated weights for policy 0, policy_version 819431 (0.0006) [2023-12-26 21:19:37,227][105620] Updated weights for policy 1, policy_version 819280 (0.0011) [2023-12-26 21:19:37,249][105692] Updated weights for policy 0, policy_version 819441 (0.0010) [2023-12-26 21:19:37,283][105620] Updated weights for policy 1, policy_version 819290 (0.0011) [2023-12-26 21:19:37,308][105692] Updated weights for policy 0, policy_version 819451 (0.0010) [2023-12-26 21:19:37,344][105620] Updated weights for policy 1, policy_version 819300 (0.0008) [2023-12-26 21:19:37,374][105692] Updated weights for policy 0, policy_version 819461 (0.0010) [2023-12-26 21:19:37,993][105620] Updated weights for policy 1, policy_version 819310 (0.0008) [2023-12-26 21:19:38,038][105620] Updated weights for policy 1, policy_version 819320 (0.0010) [2023-12-26 21:19:38,086][105620] Updated weights for policy 1, policy_version 819330 (0.0010) [2023-12-26 21:19:38,116][105692] Updated weights for policy 0, policy_version 819471 (0.0010) [2023-12-26 21:19:38,179][105692] Updated weights for policy 0, policy_version 819481 (0.0010) [2023-12-26 21:19:38,235][105692] Updated weights for policy 0, policy_version 819491 (0.0011) [2023-12-26 21:19:38,875][105692] Updated weights for policy 0, policy_version 819501 (0.0008) [2023-12-26 21:19:38,891][105620] Updated weights for policy 1, policy_version 819340 (0.0007) [2023-12-26 21:19:38,925][105692] Updated weights for policy 0, policy_version 819511 (0.0005) [2023-12-26 21:19:38,943][105620] Updated weights for policy 1, policy_version 819350 (0.0008) [2023-12-26 21:19:38,974][105692] Updated weights for policy 0, policy_version 819521 (0.0005) [2023-12-26 21:19:38,991][105620] Updated weights for policy 1, policy_version 819360 (0.0009) [2023-12-26 21:19:39,605][105692] Updated weights for policy 0, policy_version 819531 (0.0005) [2023-12-26 21:19:39,670][105692] Updated weights for policy 0, policy_version 819541 (0.0007) [2023-12-26 21:19:39,730][105692] Updated weights for policy 0, policy_version 819551 (0.0011) [2023-12-26 21:19:39,779][105620] Updated weights for policy 1, policy_version 819370 (0.0008) [2023-12-26 21:19:39,847][105620] Updated weights for policy 1, policy_version 819380 (0.0007) [2023-12-26 21:19:39,917][105620] Updated weights for policy 1, policy_version 819390 (0.0010) [2023-12-26 21:19:39,985][105620] Updated weights for policy 1, policy_version 819400 (0.0009) [2023-12-26 21:19:40,404][105692] Updated weights for policy 0, policy_version 819561 (0.0011) [2023-12-26 21:19:40,470][105692] Updated weights for policy 0, policy_version 819571 (0.0006) [2023-12-26 21:19:40,521][105692] Updated weights for policy 0, policy_version 819581 (0.0009) [2023-12-26 21:19:40,575][105692] Updated weights for policy 0, policy_version 819591 (0.0010) [2023-12-26 21:19:40,722][105620] Updated weights for policy 1, policy_version 819410 (0.0005) [2023-12-26 21:19:40,794][105620] Updated weights for policy 1, policy_version 819420 (0.0006) [2023-12-26 21:19:40,845][105620] Updated weights for policy 1, policy_version 819430 (0.0005) [2023-12-26 21:19:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 419643392. Throughput: 0: 10023.5, 1: 9679.9. Samples: 419651864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:19:41,063][104569] Avg episode reward: [(0, '9351.771'), (1, '8906.800')] [2023-12-26 21:19:41,194][105692] Updated weights for policy 0, policy_version 819601 (0.0008) [2023-12-26 21:19:41,258][105692] Updated weights for policy 0, policy_version 819611 (0.0009) [2023-12-26 21:19:41,318][105692] Updated weights for policy 0, policy_version 819621 (0.0008) [2023-12-26 21:19:41,533][105620] Updated weights for policy 1, policy_version 819440 (0.0006) [2023-12-26 21:19:41,592][105620] Updated weights for policy 1, policy_version 819450 (0.0007) [2023-12-26 21:19:41,662][105620] Updated weights for policy 1, policy_version 819460 (0.0008) [2023-12-26 21:19:42,102][105692] Updated weights for policy 0, policy_version 819631 (0.0009) [2023-12-26 21:19:42,156][105692] Updated weights for policy 0, policy_version 819641 (0.0010) [2023-12-26 21:19:42,219][105692] Updated weights for policy 0, policy_version 819651 (0.0009) [2023-12-26 21:19:42,380][105620] Updated weights for policy 1, policy_version 819470 (0.0009) [2023-12-26 21:19:42,434][105620] Updated weights for policy 1, policy_version 819480 (0.0010) [2023-12-26 21:19:42,490][105620] Updated weights for policy 1, policy_version 819490 (0.0009) [2023-12-26 21:19:42,857][105692] Updated weights for policy 0, policy_version 819661 (0.0008) [2023-12-26 21:19:42,917][105692] Updated weights for policy 0, policy_version 819671 (0.0005) [2023-12-26 21:19:42,975][105692] Updated weights for policy 0, policy_version 819681 (0.0009) [2023-12-26 21:19:43,358][105620] Updated weights for policy 1, policy_version 819500 (0.0009) [2023-12-26 21:19:43,415][105620] Updated weights for policy 1, policy_version 819510 (0.0009) [2023-12-26 21:19:43,473][105620] Updated weights for policy 1, policy_version 819520 (0.0009) [2023-12-26 21:19:43,654][105692] Updated weights for policy 0, policy_version 819691 (0.0006) [2023-12-26 21:19:43,722][105692] Updated weights for policy 0, policy_version 819701 (0.0010) [2023-12-26 21:19:43,786][105692] Updated weights for policy 0, policy_version 819711 (0.0009) [2023-12-26 21:19:44,181][105620] Updated weights for policy 1, policy_version 819530 (0.0007) [2023-12-26 21:19:44,245][105620] Updated weights for policy 1, policy_version 819540 (0.0008) [2023-12-26 21:19:44,310][105620] Updated weights for policy 1, policy_version 819550 (0.0009) [2023-12-26 21:19:44,367][105620] Updated weights for policy 1, policy_version 819560 (0.0008) [2023-12-26 21:19:44,583][105692] Updated weights for policy 0, policy_version 819721 (0.0008) [2023-12-26 21:19:44,645][105692] Updated weights for policy 0, policy_version 819731 (0.0005) [2023-12-26 21:19:44,703][105692] Updated weights for policy 0, policy_version 819741 (0.0005) [2023-12-26 21:19:44,766][105692] Updated weights for policy 0, policy_version 819751 (0.0008) [2023-12-26 21:19:45,018][105620] Updated weights for policy 1, policy_version 819570 (0.0011) [2023-12-26 21:19:45,074][105620] Updated weights for policy 1, policy_version 819580 (0.0010) [2023-12-26 21:19:45,133][105620] Updated weights for policy 1, policy_version 819590 (0.0010) [2023-12-26 21:19:45,469][105692] Updated weights for policy 0, policy_version 819761 (0.0008) [2023-12-26 21:19:45,532][105692] Updated weights for policy 0, policy_version 819771 (0.0010) [2023-12-26 21:19:45,595][105692] Updated weights for policy 0, policy_version 819781 (0.0009) [2023-12-26 21:19:45,855][105620] Updated weights for policy 1, policy_version 819600 (0.0010) [2023-12-26 21:19:45,917][105620] Updated weights for policy 1, policy_version 819610 (0.0010) [2023-12-26 21:19:45,985][105620] Updated weights for policy 1, policy_version 819620 (0.0010) [2023-12-26 21:19:46,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.9, 300 sec: 19660.8). Total num frames: 419741696. Throughput: 0: 10047.9, 1: 9653.8. Samples: 419709444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:19:46,062][104569] Avg episode reward: [(0, '9267.090'), (1, '8723.137')] [2023-12-26 21:19:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000819784_209895424.pth... [2023-12-26 21:19:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000819624_209846272.pth... [2023-12-26 21:19:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000818472_209551360.pth [2023-12-26 21:19:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000818600_209592320.pth [2023-12-26 21:19:46,311][105692] Updated weights for policy 0, policy_version 819791 (0.0010) [2023-12-26 21:19:46,370][105692] Updated weights for policy 0, policy_version 819801 (0.0010) [2023-12-26 21:19:46,432][105692] Updated weights for policy 0, policy_version 819811 (0.0010) [2023-12-26 21:19:46,681][105620] Updated weights for policy 1, policy_version 819630 (0.0010) [2023-12-26 21:19:46,736][105620] Updated weights for policy 1, policy_version 819640 (0.0010) [2023-12-26 21:19:46,783][105620] Updated weights for policy 1, policy_version 819650 (0.0010) [2023-12-26 21:19:47,173][105692] Updated weights for policy 0, policy_version 819821 (0.0009) [2023-12-26 21:19:47,221][105692] Updated weights for policy 0, policy_version 819831 (0.0010) [2023-12-26 21:19:47,273][105692] Updated weights for policy 0, policy_version 819841 (0.0010) [2023-12-26 21:19:47,526][105620] Updated weights for policy 1, policy_version 819660 (0.0010) [2023-12-26 21:19:47,586][105620] Updated weights for policy 1, policy_version 819670 (0.0010) [2023-12-26 21:19:47,640][105620] Updated weights for policy 1, policy_version 819680 (0.0010) [2023-12-26 21:19:48,047][105692] Updated weights for policy 0, policy_version 819851 (0.0010) [2023-12-26 21:19:48,100][105692] Updated weights for policy 0, policy_version 819861 (0.0008) [2023-12-26 21:19:48,151][105692] Updated weights for policy 0, policy_version 819871 (0.0008) [2023-12-26 21:19:48,357][105620] Updated weights for policy 1, policy_version 819690 (0.0010) [2023-12-26 21:19:48,425][105620] Updated weights for policy 1, policy_version 819700 (0.0007) [2023-12-26 21:19:48,490][105620] Updated weights for policy 1, policy_version 819710 (0.0005) [2023-12-26 21:19:48,559][105620] Updated weights for policy 1, policy_version 819720 (0.0005) [2023-12-26 21:19:48,846][105692] Updated weights for policy 0, policy_version 819881 (0.0007) [2023-12-26 21:19:48,905][105692] Updated weights for policy 0, policy_version 819891 (0.0009) [2023-12-26 21:19:48,955][105692] Updated weights for policy 0, policy_version 819901 (0.0008) [2023-12-26 21:19:49,012][105692] Updated weights for policy 0, policy_version 819911 (0.0005) [2023-12-26 21:19:49,169][105620] Updated weights for policy 1, policy_version 819730 (0.0008) [2023-12-26 21:19:49,225][105620] Updated weights for policy 1, policy_version 819740 (0.0009) [2023-12-26 21:19:49,280][105620] Updated weights for policy 1, policy_version 819750 (0.0009) [2023-12-26 21:19:49,750][105692] Updated weights for policy 0, policy_version 819921 (0.0009) [2023-12-26 21:19:49,804][105692] Updated weights for policy 0, policy_version 819931 (0.0008) [2023-12-26 21:19:49,866][105692] Updated weights for policy 0, policy_version 819941 (0.0009) [2023-12-26 21:19:50,085][105620] Updated weights for policy 1, policy_version 819760 (0.0009) [2023-12-26 21:19:50,148][105620] Updated weights for policy 1, policy_version 819770 (0.0009) [2023-12-26 21:19:50,206][105620] Updated weights for policy 1, policy_version 819780 (0.0009) [2023-12-26 21:19:50,563][105692] Updated weights for policy 0, policy_version 819951 (0.0006) [2023-12-26 21:19:50,621][105692] Updated weights for policy 0, policy_version 819961 (0.0008) [2023-12-26 21:19:50,679][105692] Updated weights for policy 0, policy_version 819971 (0.0009) [2023-12-26 21:19:50,903][105620] Updated weights for policy 1, policy_version 819790 (0.0007) [2023-12-26 21:19:50,967][105620] Updated weights for policy 1, policy_version 819800 (0.0006) [2023-12-26 21:19:51,033][105620] Updated weights for policy 1, policy_version 819810 (0.0008) [2023-12-26 21:19:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 419831808. Throughput: 0: 9943.5, 1: 9698.3. Samples: 419824700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:19:51,062][104569] Avg episode reward: [(0, '9175.606'), (1, '8810.306')] [2023-12-26 21:19:51,427][105692] Updated weights for policy 0, policy_version 819981 (0.0009) [2023-12-26 21:19:51,487][105692] Updated weights for policy 0, policy_version 819991 (0.0009) [2023-12-26 21:19:51,551][105692] Updated weights for policy 0, policy_version 820001 (0.0008) [2023-12-26 21:19:51,763][105620] Updated weights for policy 1, policy_version 819820 (0.0009) [2023-12-26 21:19:51,823][105620] Updated weights for policy 1, policy_version 819830 (0.0011) [2023-12-26 21:19:51,886][105620] Updated weights for policy 1, policy_version 819840 (0.0010) [2023-12-26 21:19:52,280][105692] Updated weights for policy 0, policy_version 820011 (0.0008) [2023-12-26 21:19:52,342][105692] Updated weights for policy 0, policy_version 820021 (0.0008) [2023-12-26 21:19:52,403][105692] Updated weights for policy 0, policy_version 820031 (0.0009) [2023-12-26 21:19:52,652][105620] Updated weights for policy 1, policy_version 819850 (0.0011) [2023-12-26 21:19:52,718][105620] Updated weights for policy 1, policy_version 819860 (0.0011) [2023-12-26 21:19:52,781][105620] Updated weights for policy 1, policy_version 819870 (0.0010) [2023-12-26 21:19:52,840][105620] Updated weights for policy 1, policy_version 819880 (0.0010) [2023-12-26 21:19:53,240][105692] Updated weights for policy 0, policy_version 820041 (0.0008) [2023-12-26 21:19:53,297][105692] Updated weights for policy 0, policy_version 820051 (0.0009) [2023-12-26 21:19:53,352][105692] Updated weights for policy 0, policy_version 820061 (0.0005) [2023-12-26 21:19:53,417][105692] Updated weights for policy 0, policy_version 820071 (0.0005) [2023-12-26 21:19:53,456][105620] Updated weights for policy 1, policy_version 819890 (0.0006) [2023-12-26 21:19:53,510][105620] Updated weights for policy 1, policy_version 819900 (0.0009) [2023-12-26 21:19:53,560][105620] Updated weights for policy 1, policy_version 819910 (0.0008) [2023-12-26 21:19:54,127][105692] Updated weights for policy 0, policy_version 820081 (0.0009) [2023-12-26 21:19:54,191][105692] Updated weights for policy 0, policy_version 820091 (0.0006) [2023-12-26 21:19:54,251][105692] Updated weights for policy 0, policy_version 820101 (0.0006) [2023-12-26 21:19:54,273][105620] Updated weights for policy 1, policy_version 819920 (0.0006) [2023-12-26 21:19:54,324][105620] Updated weights for policy 1, policy_version 819930 (0.0005) [2023-12-26 21:19:54,396][105620] Updated weights for policy 1, policy_version 819940 (0.0006) [2023-12-26 21:19:54,820][105692] Updated weights for policy 0, policy_version 820111 (0.0008) [2023-12-26 21:19:54,873][105692] Updated weights for policy 0, policy_version 820121 (0.0008) [2023-12-26 21:19:54,941][105692] Updated weights for policy 0, policy_version 820131 (0.0009) [2023-12-26 21:19:55,101][105620] Updated weights for policy 1, policy_version 819950 (0.0008) [2023-12-26 21:19:55,158][105620] Updated weights for policy 1, policy_version 819960 (0.0010) [2023-12-26 21:19:55,210][105620] Updated weights for policy 1, policy_version 819971 (0.0009) [2023-12-26 21:19:55,593][105692] Updated weights for policy 0, policy_version 820141 (0.0009) [2023-12-26 21:19:55,643][105692] Updated weights for policy 0, policy_version 820151 (0.0005) [2023-12-26 21:19:55,692][105692] Updated weights for policy 0, policy_version 820161 (0.0009) [2023-12-26 21:19:56,013][105620] Updated weights for policy 1, policy_version 819982 (0.0010) [2023-12-26 21:19:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 419930112. Throughput: 0: 9977.8, 1: 9662.3. Samples: 419941384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:19:56,062][104569] Avg episode reward: [(0, '9167.690'), (1, '8904.973')] [2023-12-26 21:19:56,075][105620] Updated weights for policy 1, policy_version 819992 (0.0010) [2023-12-26 21:19:56,136][105620] Updated weights for policy 1, policy_version 820002 (0.0009) [2023-12-26 21:19:56,362][105692] Updated weights for policy 0, policy_version 820171 (0.0009) [2023-12-26 21:19:56,423][105692] Updated weights for policy 0, policy_version 820181 (0.0009) [2023-12-26 21:19:56,487][105692] Updated weights for policy 0, policy_version 820191 (0.0009) [2023-12-26 21:19:56,910][105620] Updated weights for policy 1, policy_version 820012 (0.0009) [2023-12-26 21:19:56,956][105620] Updated weights for policy 1, policy_version 820022 (0.0008) [2023-12-26 21:19:57,002][105620] Updated weights for policy 1, policy_version 820032 (0.0008) [2023-12-26 21:19:57,208][105692] Updated weights for policy 0, policy_version 820201 (0.0009) [2023-12-26 21:19:57,262][105692] Updated weights for policy 0, policy_version 820211 (0.0009) [2023-12-26 21:19:57,315][105692] Updated weights for policy 0, policy_version 820221 (0.0009) [2023-12-26 21:19:57,374][105692] Updated weights for policy 0, policy_version 820231 (0.0008) [2023-12-26 21:19:57,765][105620] Updated weights for policy 1, policy_version 820042 (0.0009) [2023-12-26 21:19:57,811][105620] Updated weights for policy 1, policy_version 820052 (0.0008) [2023-12-26 21:19:57,861][105620] Updated weights for policy 1, policy_version 820063 (0.0009) [2023-12-26 21:19:58,062][105692] Updated weights for policy 0, policy_version 820241 (0.0009) [2023-12-26 21:19:58,115][105692] Updated weights for policy 0, policy_version 820251 (0.0008) [2023-12-26 21:19:58,173][105692] Updated weights for policy 0, policy_version 820261 (0.0008) [2023-12-26 21:19:58,668][105620] Updated weights for policy 1, policy_version 820075 (0.0010) [2023-12-26 21:19:58,738][105620] Updated weights for policy 1, policy_version 820085 (0.0009) [2023-12-26 21:19:58,815][105620] Updated weights for policy 1, policy_version 820095 (0.0009) [2023-12-26 21:19:59,039][105692] Updated weights for policy 0, policy_version 820271 (0.0010) [2023-12-26 21:19:59,103][105692] Updated weights for policy 0, policy_version 820281 (0.0007) [2023-12-26 21:19:59,172][105692] Updated weights for policy 0, policy_version 820291 (0.0007) [2023-12-26 21:19:59,577][105620] Updated weights for policy 1, policy_version 820105 (0.0008) [2023-12-26 21:19:59,636][105620] Updated weights for policy 1, policy_version 820115 (0.0007) [2023-12-26 21:19:59,691][105620] Updated weights for policy 1, policy_version 820125 (0.0006) [2023-12-26 21:19:59,738][105620] Updated weights for policy 1, policy_version 820135 (0.0008) [2023-12-26 21:19:59,982][105692] Updated weights for policy 0, policy_version 820301 (0.0008) [2023-12-26 21:20:00,034][105692] Updated weights for policy 0, policy_version 820311 (0.0009) [2023-12-26 21:20:00,085][105692] Updated weights for policy 0, policy_version 820321 (0.0009) [2023-12-26 21:20:00,423][105620] Updated weights for policy 1, policy_version 820145 (0.0010) [2023-12-26 21:20:00,468][105620] Updated weights for policy 1, policy_version 820155 (0.0010) [2023-12-26 21:20:00,523][105620] Updated weights for policy 1, policy_version 820165 (0.0010) [2023-12-26 21:20:00,882][105692] Updated weights for policy 0, policy_version 820331 (0.0010) [2023-12-26 21:20:00,935][105692] Updated weights for policy 0, policy_version 820342 (0.0010) [2023-12-26 21:20:00,987][105692] Updated weights for policy 0, policy_version 820352 (0.0009) [2023-12-26 21:20:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 420028416. Throughput: 0: 9988.7, 1: 9636.8. Samples: 419997248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:20:01,062][104569] Avg episode reward: [(0, '9258.791'), (1, '8758.520')] [2023-12-26 21:20:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000820360_210042880.pth... [2023-12-26 21:20:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000820168_209985536.pth... [2023-12-26 21:20:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000819048_209698816.pth [2023-12-26 21:20:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000819208_209747968.pth [2023-12-26 21:20:01,203][105620] Updated weights for policy 1, policy_version 820175 (0.0010) [2023-12-26 21:20:01,259][105620] Updated weights for policy 1, policy_version 820185 (0.0010) [2023-12-26 21:20:01,325][105620] Updated weights for policy 1, policy_version 820195 (0.0005) [2023-12-26 21:20:01,760][105692] Updated weights for policy 0, policy_version 820362 (0.0010) [2023-12-26 21:20:01,812][105692] Updated weights for policy 0, policy_version 820372 (0.0008) [2023-12-26 21:20:01,861][105692] Updated weights for policy 0, policy_version 820382 (0.0008) [2023-12-26 21:20:01,912][105692] Updated weights for policy 0, policy_version 820392 (0.0009) [2023-12-26 21:20:02,057][105620] Updated weights for policy 1, policy_version 820205 (0.0007) [2023-12-26 21:20:02,119][105620] Updated weights for policy 1, policy_version 820215 (0.0006) [2023-12-26 21:20:02,182][105620] Updated weights for policy 1, policy_version 820225 (0.0008) [2023-12-26 21:20:02,689][105692] Updated weights for policy 0, policy_version 820402 (0.0010) [2023-12-26 21:20:02,741][105692] Updated weights for policy 0, policy_version 820412 (0.0010) [2023-12-26 21:20:02,809][105692] Updated weights for policy 0, policy_version 820422 (0.0010) [2023-12-26 21:20:02,831][105620] Updated weights for policy 1, policy_version 820235 (0.0005) [2023-12-26 21:20:02,897][105620] Updated weights for policy 1, policy_version 820245 (0.0009) [2023-12-26 21:20:02,953][105620] Updated weights for policy 1, policy_version 820255 (0.0008) [2023-12-26 21:20:03,525][105692] Updated weights for policy 0, policy_version 820432 (0.0007) [2023-12-26 21:20:03,581][105692] Updated weights for policy 0, policy_version 820442 (0.0008) [2023-12-26 21:20:03,613][105620] Updated weights for policy 1, policy_version 820265 (0.0006) [2023-12-26 21:20:03,632][105692] Updated weights for policy 0, policy_version 820452 (0.0005) [2023-12-26 21:20:03,675][105620] Updated weights for policy 1, policy_version 820275 (0.0010) [2023-12-26 21:20:03,747][105620] Updated weights for policy 1, policy_version 820285 (0.0010) [2023-12-26 21:20:03,801][105620] Updated weights for policy 1, policy_version 820295 (0.0010) [2023-12-26 21:20:04,308][105692] Updated weights for policy 0, policy_version 820462 (0.0006) [2023-12-26 21:20:04,368][105692] Updated weights for policy 0, policy_version 820472 (0.0007) [2023-12-26 21:20:04,429][105692] Updated weights for policy 0, policy_version 820482 (0.0010) [2023-12-26 21:20:04,483][105620] Updated weights for policy 1, policy_version 820305 (0.0009) [2023-12-26 21:20:04,537][105620] Updated weights for policy 1, policy_version 820315 (0.0008) [2023-12-26 21:20:04,593][105620] Updated weights for policy 1, policy_version 820325 (0.0008) [2023-12-26 21:20:05,145][105692] Updated weights for policy 0, policy_version 820492 (0.0007) [2023-12-26 21:20:05,208][105692] Updated weights for policy 0, policy_version 820502 (0.0009) [2023-12-26 21:20:05,267][105692] Updated weights for policy 0, policy_version 820512 (0.0009) [2023-12-26 21:20:05,363][105620] Updated weights for policy 1, policy_version 820335 (0.0008) [2023-12-26 21:20:05,412][105620] Updated weights for policy 1, policy_version 820345 (0.0009) [2023-12-26 21:20:05,473][105620] Updated weights for policy 1, policy_version 820355 (0.0010) [2023-12-26 21:20:05,938][105692] Updated weights for policy 0, policy_version 820522 (0.0008) [2023-12-26 21:20:05,994][105692] Updated weights for policy 0, policy_version 820532 (0.0005) [2023-12-26 21:20:06,040][105692] Updated weights for policy 0, policy_version 820542 (0.0005) [2023-12-26 21:20:06,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19524.2, 300 sec: 19605.2). Total num frames: 420118528. Throughput: 0: 9887.4, 1: 9687.1. Samples: 420112564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:20:06,063][104569] Avg episode reward: [(0, '9171.640'), (1, '8938.007')] [2023-12-26 21:20:06,083][105692] Updated weights for policy 0, policy_version 820552 (0.0005) [2023-12-26 21:20:06,279][105620] Updated weights for policy 1, policy_version 820365 (0.0009) [2023-12-26 21:20:06,352][105620] Updated weights for policy 1, policy_version 820375 (0.0007) [2023-12-26 21:20:06,415][105620] Updated weights for policy 1, policy_version 820385 (0.0008) [2023-12-26 21:20:06,649][105692] Updated weights for policy 0, policy_version 820562 (0.0005) [2023-12-26 21:20:06,709][105692] Updated weights for policy 0, policy_version 820572 (0.0007) [2023-12-26 21:20:06,774][105692] Updated weights for policy 0, policy_version 820582 (0.0008) [2023-12-26 21:20:07,188][105620] Updated weights for policy 1, policy_version 820395 (0.0008) [2023-12-26 21:20:07,252][105620] Updated weights for policy 1, policy_version 820405 (0.0009) [2023-12-26 21:20:07,311][105620] Updated weights for policy 1, policy_version 820415 (0.0009) [2023-12-26 21:20:07,346][105692] Updated weights for policy 0, policy_version 820592 (0.0005) [2023-12-26 21:20:07,395][105692] Updated weights for policy 0, policy_version 820602 (0.0005) [2023-12-26 21:20:07,442][105692] Updated weights for policy 0, policy_version 820612 (0.0005) [2023-12-26 21:20:08,094][105620] Updated weights for policy 1, policy_version 820425 (0.0009) [2023-12-26 21:20:08,155][105620] Updated weights for policy 1, policy_version 820435 (0.0008) [2023-12-26 21:20:08,161][105692] Updated weights for policy 0, policy_version 820622 (0.0007) [2023-12-26 21:20:08,211][105692] Updated weights for policy 0, policy_version 820632 (0.0007) [2023-12-26 21:20:08,217][105620] Updated weights for policy 1, policy_version 820445 (0.0008) [2023-12-26 21:20:08,267][105692] Updated weights for policy 0, policy_version 820642 (0.0007) [2023-12-26 21:20:08,278][105620] Updated weights for policy 1, policy_version 820455 (0.0008) [2023-12-26 21:20:08,981][105620] Updated weights for policy 1, policy_version 820465 (0.0009) [2023-12-26 21:20:09,042][105620] Updated weights for policy 1, policy_version 820475 (0.0009) [2023-12-26 21:20:09,047][105692] Updated weights for policy 0, policy_version 820652 (0.0009) [2023-12-26 21:20:09,101][105620] Updated weights for policy 1, policy_version 820485 (0.0007) [2023-12-26 21:20:09,103][105692] Updated weights for policy 0, policy_version 820662 (0.0006) [2023-12-26 21:20:09,159][105692] Updated weights for policy 0, policy_version 820672 (0.0008) [2023-12-26 21:20:09,922][105620] Updated weights for policy 1, policy_version 820495 (0.0008) [2023-12-26 21:20:09,936][105692] Updated weights for policy 0, policy_version 820682 (0.0009) [2023-12-26 21:20:09,990][105620] Updated weights for policy 1, policy_version 820505 (0.0008) [2023-12-26 21:20:10,012][105692] Updated weights for policy 0, policy_version 820692 (0.0006) [2023-12-26 21:20:10,057][105620] Updated weights for policy 1, policy_version 820515 (0.0008) [2023-12-26 21:20:10,081][105692] Updated weights for policy 0, policy_version 820702 (0.0007) [2023-12-26 21:20:10,139][105692] Updated weights for policy 0, policy_version 820712 (0.0008) [2023-12-26 21:20:10,793][105692] Updated weights for policy 0, policy_version 820722 (0.0008) [2023-12-26 21:20:10,802][105620] Updated weights for policy 1, policy_version 820525 (0.0007) [2023-12-26 21:20:10,843][105692] Updated weights for policy 0, policy_version 820732 (0.0009) [2023-12-26 21:20:10,860][105620] Updated weights for policy 1, policy_version 820535 (0.0005) [2023-12-26 21:20:10,907][105692] Updated weights for policy 0, policy_version 820742 (0.0010) [2023-12-26 21:20:10,915][105620] Updated weights for policy 1, policy_version 820545 (0.0005) [2023-12-26 21:20:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.7, 300 sec: 19633.0). Total num frames: 420225024. Throughput: 0: 9820.5, 1: 9612.1. Samples: 420228760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:20:11,063][104569] Avg episode reward: [(0, '9080.076'), (1, '9168.312')] [2023-12-26 21:20:11,556][105620] Updated weights for policy 1, policy_version 820555 (0.0007) [2023-12-26 21:20:11,621][105620] Updated weights for policy 1, policy_version 820565 (0.0009) [2023-12-26 21:20:11,685][105620] Updated weights for policy 1, policy_version 820575 (0.0008) [2023-12-26 21:20:11,756][105692] Updated weights for policy 0, policy_version 820752 (0.0008) [2023-12-26 21:20:11,813][105692] Updated weights for policy 0, policy_version 820762 (0.0009) [2023-12-26 21:20:11,868][105692] Updated weights for policy 0, policy_version 820772 (0.0008) [2023-12-26 21:20:12,478][105620] Updated weights for policy 1, policy_version 820585 (0.0009) [2023-12-26 21:20:12,547][105620] Updated weights for policy 1, policy_version 820595 (0.0007) [2023-12-26 21:20:12,606][105620] Updated weights for policy 1, policy_version 820605 (0.0009) [2023-12-26 21:20:12,636][105692] Updated weights for policy 0, policy_version 820782 (0.0007) [2023-12-26 21:20:12,676][105620] Updated weights for policy 1, policy_version 820615 (0.0007) [2023-12-26 21:20:12,702][105692] Updated weights for policy 0, policy_version 820792 (0.0008) [2023-12-26 21:20:12,760][105692] Updated weights for policy 0, policy_version 820802 (0.0009) [2023-12-26 21:20:13,267][105620] Updated weights for policy 1, policy_version 820625 (0.0006) [2023-12-26 21:20:13,333][105620] Updated weights for policy 1, policy_version 820635 (0.0008) [2023-12-26 21:20:13,390][105620] Updated weights for policy 1, policy_version 820645 (0.0006) [2023-12-26 21:20:13,451][105692] Updated weights for policy 0, policy_version 820812 (0.0009) [2023-12-26 21:20:13,517][105692] Updated weights for policy 0, policy_version 820822 (0.0007) [2023-12-26 21:20:13,575][105692] Updated weights for policy 0, policy_version 820832 (0.0010) [2023-12-26 21:20:13,924][105620] Updated weights for policy 1, policy_version 820655 (0.0006) [2023-12-26 21:20:13,976][105620] Updated weights for policy 1, policy_version 820665 (0.0007) [2023-12-26 21:20:14,039][105620] Updated weights for policy 1, policy_version 820675 (0.0008) [2023-12-26 21:20:14,247][105692] Updated weights for policy 0, policy_version 820843 (0.0010) [2023-12-26 21:20:14,306][105692] Updated weights for policy 0, policy_version 820853 (0.0009) [2023-12-26 21:20:14,362][105692] Updated weights for policy 0, policy_version 820863 (0.0007) [2023-12-26 21:20:14,752][105620] Updated weights for policy 1, policy_version 820685 (0.0009) [2023-12-26 21:20:14,810][105620] Updated weights for policy 1, policy_version 820695 (0.0009) [2023-12-26 21:20:14,875][105620] Updated weights for policy 1, policy_version 820705 (0.0009) [2023-12-26 21:20:15,137][105692] Updated weights for policy 0, policy_version 820873 (0.0007) [2023-12-26 21:20:15,192][105692] Updated weights for policy 0, policy_version 820883 (0.0010) [2023-12-26 21:20:15,256][105692] Updated weights for policy 0, policy_version 820893 (0.0009) [2023-12-26 21:20:15,325][105692] Updated weights for policy 0, policy_version 820903 (0.0010) [2023-12-26 21:20:15,647][105620] Updated weights for policy 1, policy_version 820715 (0.0010) [2023-12-26 21:20:15,704][105620] Updated weights for policy 1, policy_version 820725 (0.0009) [2023-12-26 21:20:15,753][105620] Updated weights for policy 1, policy_version 820736 (0.0009) [2023-12-26 21:20:15,964][105692] Updated weights for policy 0, policy_version 820913 (0.0008) [2023-12-26 21:20:16,025][105692] Updated weights for policy 0, policy_version 820923 (0.0008) [2023-12-26 21:20:16,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.4, 300 sec: 19605.3). Total num frames: 420315136. Throughput: 0: 9739.1, 1: 9652.5. Samples: 420287940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:20:16,062][104569] Avg episode reward: [(0, '8632.464'), (1, '9169.728')] [2023-12-26 21:20:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000820744_210132992.pth... [2023-12-26 21:20:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000819624_209846272.pth [2023-12-26 21:20:16,079][105692] Updated weights for policy 0, policy_version 820933 (0.0009) [2023-12-26 21:20:16,092][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000820936_210190336.pth... [2023-12-26 21:20:16,095][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000819784_209895424.pth [2023-12-26 21:20:16,467][105620] Updated weights for policy 1, policy_version 820746 (0.0009) [2023-12-26 21:20:16,515][105620] Updated weights for policy 1, policy_version 820756 (0.0010) [2023-12-26 21:20:16,569][105620] Updated weights for policy 1, policy_version 820766 (0.0010) [2023-12-26 21:20:16,624][105620] Updated weights for policy 1, policy_version 820776 (0.0009) [2023-12-26 21:20:16,826][105692] Updated weights for policy 0, policy_version 820943 (0.0009) [2023-12-26 21:20:16,877][105692] Updated weights for policy 0, policy_version 820953 (0.0008) [2023-12-26 21:20:16,933][105692] Updated weights for policy 0, policy_version 820963 (0.0010) [2023-12-26 21:20:17,305][105620] Updated weights for policy 1, policy_version 820786 (0.0007) [2023-12-26 21:20:17,368][105620] Updated weights for policy 1, policy_version 820796 (0.0008) [2023-12-26 21:20:17,436][105620] Updated weights for policy 1, policy_version 820806 (0.0009) [2023-12-26 21:20:17,775][105692] Updated weights for policy 0, policy_version 820974 (0.0010) [2023-12-26 21:20:17,834][105692] Updated weights for policy 0, policy_version 820984 (0.0009) [2023-12-26 21:20:17,887][105692] Updated weights for policy 0, policy_version 820994 (0.0009) [2023-12-26 21:20:18,056][105620] Updated weights for policy 1, policy_version 820816 (0.0009) [2023-12-26 21:20:18,104][105620] Updated weights for policy 1, policy_version 820826 (0.0007) [2023-12-26 21:20:18,152][105620] Updated weights for policy 1, policy_version 820836 (0.0009) [2023-12-26 21:20:18,719][105692] Updated weights for policy 0, policy_version 821004 (0.0009) [2023-12-26 21:20:18,773][105692] Updated weights for policy 0, policy_version 821014 (0.0008) [2023-12-26 21:20:18,826][105692] Updated weights for policy 0, policy_version 821024 (0.0008) [2023-12-26 21:20:18,945][105620] Updated weights for policy 1, policy_version 820846 (0.0010) [2023-12-26 21:20:19,007][105620] Updated weights for policy 1, policy_version 820856 (0.0010) [2023-12-26 21:20:19,065][105620] Updated weights for policy 1, policy_version 820866 (0.0010) [2023-12-26 21:20:19,639][105692] Updated weights for policy 0, policy_version 821034 (0.0007) [2023-12-26 21:20:19,702][105692] Updated weights for policy 0, policy_version 821044 (0.0009) [2023-12-26 21:20:19,757][105692] Updated weights for policy 0, policy_version 821054 (0.0008) [2023-12-26 21:20:19,769][105620] Updated weights for policy 1, policy_version 820876 (0.0008) [2023-12-26 21:20:19,816][105692] Updated weights for policy 0, policy_version 821064 (0.0007) [2023-12-26 21:20:19,840][105620] Updated weights for policy 1, policy_version 820886 (0.0009) [2023-12-26 21:20:19,900][105620] Updated weights for policy 1, policy_version 820896 (0.0010) [2023-12-26 21:20:20,640][105692] Updated weights for policy 0, policy_version 821074 (0.0009) [2023-12-26 21:20:20,699][105692] Updated weights for policy 0, policy_version 821084 (0.0009) [2023-12-26 21:20:20,761][105692] Updated weights for policy 0, policy_version 821094 (0.0009) [2023-12-26 21:20:20,776][105620] Updated weights for policy 1, policy_version 820906 (0.0009) [2023-12-26 21:20:20,833][105620] Updated weights for policy 1, policy_version 820916 (0.0009) [2023-12-26 21:20:20,894][105620] Updated weights for policy 1, policy_version 820926 (0.0007) [2023-12-26 21:20:20,953][105620] Updated weights for policy 1, policy_version 820936 (0.0005) [2023-12-26 21:20:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 420413440. Throughput: 0: 9648.9, 1: 9657.7. Samples: 420401560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:20:21,062][104569] Avg episode reward: [(0, '8277.082'), (1, '9171.062')] [2023-12-26 21:20:21,564][105692] Updated weights for policy 0, policy_version 821104 (0.0009) [2023-12-26 21:20:21,625][105692] Updated weights for policy 0, policy_version 821114 (0.0009) [2023-12-26 21:20:21,689][105692] Updated weights for policy 0, policy_version 821124 (0.0008) [2023-12-26 21:20:21,737][105620] Updated weights for policy 1, policy_version 820946 (0.0010) [2023-12-26 21:20:21,808][105620] Updated weights for policy 1, policy_version 820956 (0.0006) [2023-12-26 21:20:21,873][105620] Updated weights for policy 1, policy_version 820966 (0.0006) [2023-12-26 21:20:22,385][105692] Updated weights for policy 0, policy_version 821134 (0.0007) [2023-12-26 21:20:22,454][105692] Updated weights for policy 0, policy_version 821144 (0.0006) [2023-12-26 21:20:22,496][105620] Updated weights for policy 1, policy_version 820976 (0.0008) [2023-12-26 21:20:22,515][105692] Updated weights for policy 0, policy_version 821154 (0.0007) [2023-12-26 21:20:22,552][105620] Updated weights for policy 1, policy_version 820986 (0.0010) [2023-12-26 21:20:22,621][105620] Updated weights for policy 1, policy_version 820996 (0.0009) [2023-12-26 21:20:23,181][105692] Updated weights for policy 0, policy_version 821164 (0.0007) [2023-12-26 21:20:23,249][105620] Updated weights for policy 1, policy_version 821006 (0.0006) [2023-12-26 21:20:23,251][105692] Updated weights for policy 0, policy_version 821174 (0.0007) [2023-12-26 21:20:23,314][105620] Updated weights for policy 1, policy_version 821016 (0.0008) [2023-12-26 21:20:23,320][105692] Updated weights for policy 0, policy_version 821184 (0.0008) [2023-12-26 21:20:23,366][105620] Updated weights for policy 1, policy_version 821026 (0.0008) [2023-12-26 21:20:23,934][105692] Updated weights for policy 0, policy_version 821194 (0.0006) [2023-12-26 21:20:23,996][105692] Updated weights for policy 0, policy_version 821204 (0.0010) [2023-12-26 21:20:24,046][105620] Updated weights for policy 1, policy_version 821036 (0.0008) [2023-12-26 21:20:24,061][105692] Updated weights for policy 0, policy_version 821214 (0.0009) [2023-12-26 21:20:24,110][105620] Updated weights for policy 1, policy_version 821046 (0.0007) [2023-12-26 21:20:24,116][105692] Updated weights for policy 0, policy_version 821224 (0.0007) [2023-12-26 21:20:24,159][105620] Updated weights for policy 1, policy_version 821056 (0.0008) [2023-12-26 21:20:24,762][105692] Updated weights for policy 0, policy_version 821234 (0.0008) [2023-12-26 21:20:24,812][105620] Updated weights for policy 1, policy_version 821066 (0.0006) [2023-12-26 21:20:24,822][105692] Updated weights for policy 0, policy_version 821244 (0.0008) [2023-12-26 21:20:24,860][105620] Updated weights for policy 1, policy_version 821076 (0.0010) [2023-12-26 21:20:24,874][105692] Updated weights for policy 0, policy_version 821254 (0.0006) [2023-12-26 21:20:24,912][105620] Updated weights for policy 1, policy_version 821086 (0.0010) [2023-12-26 21:20:24,974][105620] Updated weights for policy 1, policy_version 821096 (0.0010) [2023-12-26 21:20:25,579][105620] Updated weights for policy 1, policy_version 821106 (0.0005) [2023-12-26 21:20:25,589][105692] Updated weights for policy 0, policy_version 821264 (0.0009) [2023-12-26 21:20:25,635][105620] Updated weights for policy 1, policy_version 821116 (0.0010) [2023-12-26 21:20:25,647][105692] Updated weights for policy 0, policy_version 821274 (0.0008) [2023-12-26 21:20:25,695][105692] Updated weights for policy 0, policy_version 821284 (0.0005) [2023-12-26 21:20:25,699][105620] Updated weights for policy 1, policy_version 821126 (0.0005) [2023-12-26 21:20:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 420511744. Throughput: 0: 9619.0, 1: 9686.0. Samples: 420520588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:20:26,062][104569] Avg episode reward: [(0, '8634.063'), (1, '9261.127')] [2023-12-26 21:20:26,347][105620] Updated weights for policy 1, policy_version 821136 (0.0008) [2023-12-26 21:20:26,402][105692] Updated weights for policy 0, policy_version 821294 (0.0009) [2023-12-26 21:20:26,404][105620] Updated weights for policy 1, policy_version 821146 (0.0007) [2023-12-26 21:20:26,452][105620] Updated weights for policy 1, policy_version 821156 (0.0007) [2023-12-26 21:20:26,457][105692] Updated weights for policy 0, policy_version 821304 (0.0010) [2023-12-26 21:20:26,513][105692] Updated weights for policy 0, policy_version 821314 (0.0010) [2023-12-26 21:20:27,083][105692] Updated weights for policy 0, policy_version 821324 (0.0010) [2023-12-26 21:20:27,140][105692] Updated weights for policy 0, policy_version 821334 (0.0010) [2023-12-26 21:20:27,201][105692] Updated weights for policy 0, policy_version 821344 (0.0010) [2023-12-26 21:20:27,289][105620] Updated weights for policy 1, policy_version 821166 (0.0007) [2023-12-26 21:20:27,351][105620] Updated weights for policy 1, policy_version 821176 (0.0007) [2023-12-26 21:20:27,416][105620] Updated weights for policy 1, policy_version 821186 (0.0007) [2023-12-26 21:20:27,921][105692] Updated weights for policy 0, policy_version 821354 (0.0010) [2023-12-26 21:20:27,969][105692] Updated weights for policy 0, policy_version 821364 (0.0010) [2023-12-26 21:20:28,019][105692] Updated weights for policy 0, policy_version 821374 (0.0010) [2023-12-26 21:20:28,054][105620] Updated weights for policy 1, policy_version 821196 (0.0007) [2023-12-26 21:20:28,071][105692] Updated weights for policy 0, policy_version 821384 (0.0010) [2023-12-26 21:20:28,109][105620] Updated weights for policy 1, policy_version 821206 (0.0005) [2023-12-26 21:20:28,169][105620] Updated weights for policy 1, policy_version 821216 (0.0005) [2023-12-26 21:20:28,717][105692] Updated weights for policy 0, policy_version 821394 (0.0006) [2023-12-26 21:20:28,769][105585] KL-divergence is very high: 135.6998 [2023-12-26 21:20:28,773][105692] Updated weights for policy 0, policy_version 821404 (0.0011) [2023-12-26 21:20:28,807][105585] KL-divergence is very high: 219.7108 [2023-12-26 21:20:28,821][105692] Updated weights for policy 0, policy_version 821414 (0.0010) [2023-12-26 21:20:28,860][105620] Updated weights for policy 1, policy_version 821226 (0.0006) [2023-12-26 21:20:28,914][105620] Updated weights for policy 1, policy_version 821237 (0.0010) [2023-12-26 21:20:28,963][105620] Updated weights for policy 1, policy_version 821247 (0.0008) [2023-12-26 21:20:29,527][105692] Updated weights for policy 0, policy_version 821424 (0.0010) [2023-12-26 21:20:29,575][105692] Updated weights for policy 0, policy_version 821434 (0.0010) [2023-12-26 21:20:29,624][105692] Updated weights for policy 0, policy_version 821444 (0.0005) [2023-12-26 21:20:29,785][105620] Updated weights for policy 1, policy_version 821257 (0.0008) [2023-12-26 21:20:29,850][105620] Updated weights for policy 1, policy_version 821267 (0.0009) [2023-12-26 21:20:29,913][105620] Updated weights for policy 1, policy_version 821277 (0.0008) [2023-12-26 21:20:29,977][105620] Updated weights for policy 1, policy_version 821287 (0.0008) [2023-12-26 21:20:30,235][105692] Updated weights for policy 0, policy_version 821454 (0.0008) [2023-12-26 21:20:30,290][105692] Updated weights for policy 0, policy_version 821464 (0.0011) [2023-12-26 21:20:30,338][105692] Updated weights for policy 0, policy_version 821474 (0.0011) [2023-12-26 21:20:30,815][105620] Updated weights for policy 1, policy_version 821297 (0.0009) [2023-12-26 21:20:30,876][105620] Updated weights for policy 1, policy_version 821307 (0.0010) [2023-12-26 21:20:30,912][105692] Updated weights for policy 0, policy_version 821484 (0.0005) [2023-12-26 21:20:30,929][105620] Updated weights for policy 1, policy_version 821317 (0.0009) [2023-12-26 21:20:30,961][105692] Updated weights for policy 0, policy_version 821494 (0.0005) [2023-12-26 21:20:31,016][105692] Updated weights for policy 0, policy_version 821504 (0.0006) [2023-12-26 21:20:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 420610048. Throughput: 0: 9639.9, 1: 9724.8. Samples: 420580856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:20:31,062][104569] Avg episode reward: [(0, '8633.707'), (1, '9351.939')] [2023-12-26 21:20:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000821320_210280448.pth... [2023-12-26 21:20:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000821512_210337792.pth... [2023-12-26 21:20:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000820168_209985536.pth [2023-12-26 21:20:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000820360_210042880.pth [2023-12-26 21:20:31,699][105692] Updated weights for policy 0, policy_version 821514 (0.0011) [2023-12-26 21:20:31,755][105692] Updated weights for policy 0, policy_version 821524 (0.0007) [2023-12-26 21:20:31,763][105620] Updated weights for policy 1, policy_version 821327 (0.0009) [2023-12-26 21:20:31,812][105692] Updated weights for policy 0, policy_version 821534 (0.0006) [2023-12-26 21:20:31,830][105620] Updated weights for policy 1, policy_version 821337 (0.0008) [2023-12-26 21:20:31,868][105692] Updated weights for policy 0, policy_version 821544 (0.0007) [2023-12-26 21:20:31,892][105620] Updated weights for policy 1, policy_version 821347 (0.0007) [2023-12-26 21:20:32,542][105692] Updated weights for policy 0, policy_version 821554 (0.0009) [2023-12-26 21:20:32,607][105692] Updated weights for policy 0, policy_version 821564 (0.0009) [2023-12-26 21:20:32,656][105692] Updated weights for policy 0, policy_version 821574 (0.0008) [2023-12-26 21:20:32,673][105620] Updated weights for policy 1, policy_version 821357 (0.0010) [2023-12-26 21:20:32,733][105620] Updated weights for policy 1, policy_version 821367 (0.0008) [2023-12-26 21:20:32,784][105620] Updated weights for policy 1, policy_version 821377 (0.0009) [2023-12-26 21:20:33,417][105692] Updated weights for policy 0, policy_version 821584 (0.0009) [2023-12-26 21:20:33,473][105692] Updated weights for policy 0, policy_version 821594 (0.0010) [2023-12-26 21:20:33,517][105620] Updated weights for policy 1, policy_version 821387 (0.0009) [2023-12-26 21:20:33,528][105692] Updated weights for policy 0, policy_version 821604 (0.0008) [2023-12-26 21:20:33,565][105620] Updated weights for policy 1, policy_version 821397 (0.0010) [2023-12-26 21:20:33,613][105620] Updated weights for policy 1, policy_version 821407 (0.0010) [2023-12-26 21:20:34,308][105692] Updated weights for policy 0, policy_version 821614 (0.0008) [2023-12-26 21:20:34,347][105585] KL-divergence is very high: 100.1550 [2023-12-26 21:20:34,360][105585] KL-divergence is very high: 105.8847 [2023-12-26 21:20:34,372][105692] Updated weights for policy 0, policy_version 821624 (0.0008) [2023-12-26 21:20:34,380][105620] Updated weights for policy 1, policy_version 821417 (0.0010) [2023-12-26 21:20:34,432][105692] Updated weights for policy 0, policy_version 821634 (0.0008) [2023-12-26 21:20:34,448][105620] Updated weights for policy 1, policy_version 821427 (0.0009) [2023-12-26 21:20:34,511][105620] Updated weights for policy 1, policy_version 821437 (0.0010) [2023-12-26 21:20:34,564][105620] Updated weights for policy 1, policy_version 821447 (0.0011) [2023-12-26 21:20:35,186][105620] Updated weights for policy 1, policy_version 821457 (0.0011) [2023-12-26 21:20:35,241][105620] Updated weights for policy 1, policy_version 821467 (0.0010) [2023-12-26 21:20:35,247][105692] Updated weights for policy 0, policy_version 821644 (0.0008) [2023-12-26 21:20:35,292][105620] Updated weights for policy 1, policy_version 821477 (0.0010) [2023-12-26 21:20:35,305][105692] Updated weights for policy 0, policy_version 821654 (0.0006) [2023-12-26 21:20:35,340][105585] KL-divergence is very high: 104.5950 [2023-12-26 21:20:35,363][105692] Updated weights for policy 0, policy_version 821664 (0.0008) [2023-12-26 21:20:35,941][105620] Updated weights for policy 1, policy_version 821487 (0.0007) [2023-12-26 21:20:36,007][105620] Updated weights for policy 1, policy_version 821497 (0.0007) [2023-12-26 21:20:36,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.3, 300 sec: 19549.7). Total num frames: 420700160. Throughput: 0: 9738.3, 1: 9633.8. Samples: 420696444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:20:36,063][104569] Avg episode reward: [(0, '7394.915'), (1, '9169.155')] [2023-12-26 21:20:36,069][105620] Updated weights for policy 1, policy_version 821507 (0.0010) [2023-12-26 21:20:36,108][105692] Updated weights for policy 0, policy_version 821674 (0.0007) [2023-12-26 21:20:36,177][105692] Updated weights for policy 0, policy_version 821684 (0.0006) [2023-12-26 21:20:36,233][105692] Updated weights for policy 0, policy_version 821694 (0.0006) [2023-12-26 21:20:36,293][105692] Updated weights for policy 0, policy_version 821704 (0.0005) [2023-12-26 21:20:36,753][105620] Updated weights for policy 1, policy_version 821517 (0.0008) [2023-12-26 21:20:36,807][105620] Updated weights for policy 1, policy_version 821527 (0.0006) [2023-12-26 21:20:36,860][105620] Updated weights for policy 1, policy_version 821537 (0.0010) [2023-12-26 21:20:36,954][105692] Updated weights for policy 0, policy_version 821714 (0.0010) [2023-12-26 21:20:37,016][105692] Updated weights for policy 0, policy_version 821724 (0.0009) [2023-12-26 21:20:37,068][105692] Updated weights for policy 0, policy_version 821734 (0.0009) [2023-12-26 21:20:37,533][105620] Updated weights for policy 1, policy_version 821547 (0.0007) [2023-12-26 21:20:37,591][105620] Updated weights for policy 1, policy_version 821557 (0.0011) [2023-12-26 21:20:37,651][105620] Updated weights for policy 1, policy_version 821567 (0.0011) [2023-12-26 21:20:37,865][105692] Updated weights for policy 0, policy_version 821744 (0.0008) [2023-12-26 21:20:37,908][105692] Updated weights for policy 0, policy_version 821754 (0.0008) [2023-12-26 21:20:37,956][105692] Updated weights for policy 0, policy_version 821764 (0.0007) [2023-12-26 21:20:38,388][105620] Updated weights for policy 1, policy_version 821577 (0.0011) [2023-12-26 21:20:38,448][105620] Updated weights for policy 1, policy_version 821587 (0.0008) [2023-12-26 21:20:38,506][105620] Updated weights for policy 1, policy_version 821597 (0.0010) [2023-12-26 21:20:38,565][105620] Updated weights for policy 1, policy_version 821607 (0.0011) [2023-12-26 21:20:38,718][105692] Updated weights for policy 0, policy_version 821774 (0.0008) [2023-12-26 21:20:38,787][105692] Updated weights for policy 0, policy_version 821784 (0.0008) [2023-12-26 21:20:38,852][105692] Updated weights for policy 0, policy_version 821794 (0.0008) [2023-12-26 21:20:39,319][105620] Updated weights for policy 1, policy_version 821617 (0.0010) [2023-12-26 21:20:39,389][105620] Updated weights for policy 1, policy_version 821627 (0.0011) [2023-12-26 21:20:39,459][105620] Updated weights for policy 1, policy_version 821637 (0.0011) [2023-12-26 21:20:39,524][105692] Updated weights for policy 0, policy_version 821804 (0.0008) [2023-12-26 21:20:39,583][105692] Updated weights for policy 0, policy_version 821814 (0.0007) [2023-12-26 21:20:39,649][105692] Updated weights for policy 0, policy_version 821824 (0.0008) [2023-12-26 21:20:40,230][105620] Updated weights for policy 1, policy_version 821647 (0.0009) [2023-12-26 21:20:40,298][105620] Updated weights for policy 1, policy_version 821657 (0.0005) [2023-12-26 21:20:40,364][105620] Updated weights for policy 1, policy_version 821667 (0.0006) [2023-12-26 21:20:40,399][105692] Updated weights for policy 0, policy_version 821834 (0.0009) [2023-12-26 21:20:40,452][105692] Updated weights for policy 0, policy_version 821844 (0.0010) [2023-12-26 21:20:40,506][105692] Updated weights for policy 0, policy_version 821855 (0.0010) [2023-12-26 21:20:40,890][105620] Updated weights for policy 1, policy_version 821677 (0.0006) [2023-12-26 21:20:40,944][105620] Updated weights for policy 1, policy_version 821687 (0.0007) [2023-12-26 21:20:40,994][105620] Updated weights for policy 1, policy_version 821697 (0.0005) [2023-12-26 21:20:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 420806656. Throughput: 0: 9677.0, 1: 9686.5. Samples: 420812744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:20:41,063][104569] Avg episode reward: [(0, '7477.996'), (1, '9078.440')] [2023-12-26 21:20:41,400][105692] Updated weights for policy 0, policy_version 821865 (0.0009) [2023-12-26 21:20:41,463][105692] Updated weights for policy 0, policy_version 821875 (0.0006) [2023-12-26 21:20:41,522][105692] Updated weights for policy 0, policy_version 821885 (0.0008) [2023-12-26 21:20:41,582][105692] Updated weights for policy 0, policy_version 821895 (0.0009) [2023-12-26 21:20:41,742][105620] Updated weights for policy 1, policy_version 821707 (0.0007) [2023-12-26 21:20:41,806][105620] Updated weights for policy 1, policy_version 821717 (0.0008) [2023-12-26 21:20:41,867][105620] Updated weights for policy 1, policy_version 821727 (0.0008) [2023-12-26 21:20:42,332][105692] Updated weights for policy 0, policy_version 821905 (0.0009) [2023-12-26 21:20:42,394][105692] Updated weights for policy 0, policy_version 821915 (0.0008) [2023-12-26 21:20:42,456][105692] Updated weights for policy 0, policy_version 821925 (0.0005) [2023-12-26 21:20:42,684][105620] Updated weights for policy 1, policy_version 821737 (0.0009) [2023-12-26 21:20:42,748][105620] Updated weights for policy 1, policy_version 821747 (0.0009) [2023-12-26 21:20:42,813][105620] Updated weights for policy 1, policy_version 821757 (0.0008) [2023-12-26 21:20:42,877][105620] Updated weights for policy 1, policy_version 821767 (0.0006) [2023-12-26 21:20:43,103][105692] Updated weights for policy 0, policy_version 821935 (0.0005) [2023-12-26 21:20:43,160][105692] Updated weights for policy 0, policy_version 821945 (0.0006) [2023-12-26 21:20:43,216][105692] Updated weights for policy 0, policy_version 821955 (0.0005) [2023-12-26 21:20:43,451][105620] Updated weights for policy 1, policy_version 821777 (0.0005) [2023-12-26 21:20:43,510][105620] Updated weights for policy 1, policy_version 821787 (0.0008) [2023-12-26 21:20:43,565][105620] Updated weights for policy 1, policy_version 821797 (0.0008) [2023-12-26 21:20:43,913][105692] Updated weights for policy 0, policy_version 821965 (0.0006) [2023-12-26 21:20:43,970][105692] Updated weights for policy 0, policy_version 821975 (0.0005) [2023-12-26 21:20:44,040][105692] Updated weights for policy 0, policy_version 821985 (0.0009) [2023-12-26 21:20:44,121][105620] Updated weights for policy 1, policy_version 821807 (0.0010) [2023-12-26 21:20:44,180][105620] Updated weights for policy 1, policy_version 821817 (0.0010) [2023-12-26 21:20:44,238][105620] Updated weights for policy 1, policy_version 821827 (0.0010) [2023-12-26 21:20:44,760][105692] Updated weights for policy 0, policy_version 821995 (0.0009) [2023-12-26 21:20:44,827][105692] Updated weights for policy 0, policy_version 822005 (0.0007) [2023-12-26 21:20:44,892][105692] Updated weights for policy 0, policy_version 822015 (0.0006) [2023-12-26 21:20:45,000][105620] Updated weights for policy 1, policy_version 821837 (0.0011) [2023-12-26 21:20:45,077][105620] Updated weights for policy 1, policy_version 821847 (0.0011) [2023-12-26 21:20:45,147][105620] Updated weights for policy 1, policy_version 821857 (0.0010) [2023-12-26 21:20:45,509][105692] Updated weights for policy 0, policy_version 822025 (0.0008) [2023-12-26 21:20:45,567][105692] Updated weights for policy 0, policy_version 822035 (0.0008) [2023-12-26 21:20:45,619][105692] Updated weights for policy 0, policy_version 822045 (0.0008) [2023-12-26 21:20:45,668][105692] Updated weights for policy 0, policy_version 822055 (0.0008) [2023-12-26 21:20:45,854][105620] Updated weights for policy 1, policy_version 821867 (0.0010) [2023-12-26 21:20:45,912][105620] Updated weights for policy 1, policy_version 821877 (0.0008) [2023-12-26 21:20:45,970][105620] Updated weights for policy 1, policy_version 821887 (0.0009) [2023-12-26 21:20:46,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 420904960. Throughput: 0: 9672.3, 1: 9765.1. Samples: 420871932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:20:46,063][104569] Avg episode reward: [(0, '2857.481'), (1, '9260.796')] [2023-12-26 21:20:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000822056_210477056.pth... [2023-12-26 21:20:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000821896_210427904.pth... [2023-12-26 21:20:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000820936_210190336.pth [2023-12-26 21:20:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000820744_210132992.pth [2023-12-26 21:20:46,414][105692] Updated weights for policy 0, policy_version 822065 (0.0009) [2023-12-26 21:20:46,478][105692] Updated weights for policy 0, policy_version 822075 (0.0009) [2023-12-26 21:20:46,523][105692] Updated weights for policy 0, policy_version 822085 (0.0008) [2023-12-26 21:20:46,725][105620] Updated weights for policy 1, policy_version 821897 (0.0009) [2023-12-26 21:20:46,778][105620] Updated weights for policy 1, policy_version 821907 (0.0009) [2023-12-26 21:20:46,831][105620] Updated weights for policy 1, policy_version 821917 (0.0008) [2023-12-26 21:20:46,887][105620] Updated weights for policy 1, policy_version 821927 (0.0009) [2023-12-26 21:20:47,209][105692] Updated weights for policy 0, policy_version 822095 (0.0005) [2023-12-26 21:20:47,275][105692] Updated weights for policy 0, policy_version 822105 (0.0005) [2023-12-26 21:20:47,342][105692] Updated weights for policy 0, policy_version 822115 (0.0007) [2023-12-26 21:20:47,653][105620] Updated weights for policy 1, policy_version 821937 (0.0008) [2023-12-26 21:20:47,708][105620] Updated weights for policy 1, policy_version 821947 (0.0005) [2023-12-26 21:20:47,771][105620] Updated weights for policy 1, policy_version 821957 (0.0006) [2023-12-26 21:20:47,862][105692] Updated weights for policy 0, policy_version 822125 (0.0006) [2023-12-26 21:20:47,926][105692] Updated weights for policy 0, policy_version 822135 (0.0005) [2023-12-26 21:20:47,991][105692] Updated weights for policy 0, policy_version 822145 (0.0005) [2023-12-26 21:20:48,531][105620] Updated weights for policy 1, policy_version 821967 (0.0006) [2023-12-26 21:20:48,586][105620] Updated weights for policy 1, policy_version 821977 (0.0005) [2023-12-26 21:20:48,590][105692] Updated weights for policy 0, policy_version 822155 (0.0006) [2023-12-26 21:20:48,643][105620] Updated weights for policy 1, policy_version 821987 (0.0005) [2023-12-26 21:20:48,657][105692] Updated weights for policy 0, policy_version 822165 (0.0006) [2023-12-26 21:20:48,729][105692] Updated weights for policy 0, policy_version 822175 (0.0006) [2023-12-26 21:20:49,320][105692] Updated weights for policy 0, policy_version 822185 (0.0008) [2023-12-26 21:20:49,385][105692] Updated weights for policy 0, policy_version 822195 (0.0009) [2023-12-26 21:20:49,432][105692] Updated weights for policy 0, policy_version 822205 (0.0010) [2023-12-26 21:20:49,453][105620] Updated weights for policy 1, policy_version 821997 (0.0007) [2023-12-26 21:20:49,483][105692] Updated weights for policy 0, policy_version 822215 (0.0008) [2023-12-26 21:20:49,510][105620] Updated weights for policy 1, policy_version 822007 (0.0007) [2023-12-26 21:20:49,576][105620] Updated weights for policy 1, policy_version 822017 (0.0008) [2023-12-26 21:20:50,247][105692] Updated weights for policy 0, policy_version 822225 (0.0009) [2023-12-26 21:20:50,303][105692] Updated weights for policy 0, policy_version 822235 (0.0009) [2023-12-26 21:20:50,337][105620] Updated weights for policy 1, policy_version 822027 (0.0008) [2023-12-26 21:20:50,357][105692] Updated weights for policy 0, policy_version 822245 (0.0007) [2023-12-26 21:20:50,399][105620] Updated weights for policy 1, policy_version 822037 (0.0009) [2023-12-26 21:20:50,461][105620] Updated weights for policy 1, policy_version 822047 (0.0009) [2023-12-26 21:20:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 420995072. Throughput: 0: 9832.3, 1: 9662.0. Samples: 420989804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:20:51,062][104569] Avg episode reward: [(0, '6448.061'), (1, '9076.154')] [2023-12-26 21:20:51,134][105692] Updated weights for policy 0, policy_version 822255 (0.0008) [2023-12-26 21:20:51,192][105620] Updated weights for policy 1, policy_version 822057 (0.0009) [2023-12-26 21:20:51,205][105692] Updated weights for policy 0, policy_version 822265 (0.0006) [2023-12-26 21:20:51,254][105620] Updated weights for policy 1, policy_version 822067 (0.0008) [2023-12-26 21:20:51,270][105692] Updated weights for policy 0, policy_version 822275 (0.0007) [2023-12-26 21:20:51,314][105620] Updated weights for policy 1, policy_version 822077 (0.0007) [2023-12-26 21:20:51,377][105620] Updated weights for policy 1, policy_version 822087 (0.0009) [2023-12-26 21:20:52,040][105620] Updated weights for policy 1, policy_version 822097 (0.0009) [2023-12-26 21:20:52,047][105692] Updated weights for policy 0, policy_version 822285 (0.0009) [2023-12-26 21:20:52,102][105620] Updated weights for policy 1, policy_version 822107 (0.0007) [2023-12-26 21:20:52,104][105692] Updated weights for policy 0, policy_version 822295 (0.0007) [2023-12-26 21:20:52,155][105620] Updated weights for policy 1, policy_version 822117 (0.0007) [2023-12-26 21:20:52,160][105692] Updated weights for policy 0, policy_version 822305 (0.0007) [2023-12-26 21:20:52,879][105620] Updated weights for policy 1, policy_version 822127 (0.0008) [2023-12-26 21:20:52,939][105620] Updated weights for policy 1, policy_version 822137 (0.0008) [2023-12-26 21:20:52,957][105692] Updated weights for policy 0, policy_version 822315 (0.0008) [2023-12-26 21:20:52,996][105620] Updated weights for policy 1, policy_version 822148 (0.0008) [2023-12-26 21:20:53,017][105692] Updated weights for policy 0, policy_version 822325 (0.0006) [2023-12-26 21:20:53,084][105692] Updated weights for policy 0, policy_version 822335 (0.0011) [2023-12-26 21:20:53,624][105620] Updated weights for policy 1, policy_version 822158 (0.0007) [2023-12-26 21:20:53,671][105620] Updated weights for policy 1, policy_version 822168 (0.0010) [2023-12-26 21:20:53,719][105620] Updated weights for policy 1, policy_version 822178 (0.0010) [2023-12-26 21:20:53,834][105692] Updated weights for policy 0, policy_version 822345 (0.0010) [2023-12-26 21:20:53,881][105692] Updated weights for policy 0, policy_version 822355 (0.0007) [2023-12-26 21:20:53,932][105692] Updated weights for policy 0, policy_version 822365 (0.0005) [2023-12-26 21:20:53,993][105692] Updated weights for policy 0, policy_version 822375 (0.0011) [2023-12-26 21:20:54,370][105620] Updated weights for policy 1, policy_version 822188 (0.0008) [2023-12-26 21:20:54,434][105620] Updated weights for policy 1, policy_version 822198 (0.0007) [2023-12-26 21:20:54,491][105620] Updated weights for policy 1, policy_version 822208 (0.0008) [2023-12-26 21:20:54,721][105692] Updated weights for policy 0, policy_version 822385 (0.0010) [2023-12-26 21:20:54,776][105692] Updated weights for policy 0, policy_version 822395 (0.0010) [2023-12-26 21:20:54,825][105692] Updated weights for policy 0, policy_version 822405 (0.0008) [2023-12-26 21:20:55,195][105620] Updated weights for policy 1, policy_version 822218 (0.0010) [2023-12-26 21:20:55,266][105620] Updated weights for policy 1, policy_version 822228 (0.0010) [2023-12-26 21:20:55,334][105620] Updated weights for policy 1, policy_version 822238 (0.0010) [2023-12-26 21:20:55,400][105620] Updated weights for policy 1, policy_version 822248 (0.0011) [2023-12-26 21:20:55,478][105692] Updated weights for policy 0, policy_version 822415 (0.0008) [2023-12-26 21:20:55,531][105692] Updated weights for policy 0, policy_version 822425 (0.0005) [2023-12-26 21:20:55,577][105692] Updated weights for policy 0, policy_version 822435 (0.0008) [2023-12-26 21:20:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 421093376. Throughput: 0: 9724.6, 1: 9771.0. Samples: 421106060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:20:56,062][104569] Avg episode reward: [(0, '8636.400'), (1, '9075.845')] [2023-12-26 21:20:56,091][105620] Updated weights for policy 1, policy_version 822258 (0.0005) [2023-12-26 21:20:56,144][105620] Updated weights for policy 1, policy_version 822268 (0.0005) [2023-12-26 21:20:56,202][105620] Updated weights for policy 1, policy_version 822278 (0.0006) [2023-12-26 21:20:56,322][105692] Updated weights for policy 0, policy_version 822445 (0.0007) [2023-12-26 21:20:56,377][105692] Updated weights for policy 0, policy_version 822455 (0.0008) [2023-12-26 21:20:56,447][105692] Updated weights for policy 0, policy_version 822465 (0.0005) [2023-12-26 21:20:56,790][105620] Updated weights for policy 1, policy_version 822288 (0.0007) [2023-12-26 21:20:56,838][105620] Updated weights for policy 1, policy_version 822298 (0.0010) [2023-12-26 21:20:56,886][105620] Updated weights for policy 1, policy_version 822308 (0.0010) [2023-12-26 21:20:57,246][105692] Updated weights for policy 0, policy_version 822475 (0.0009) [2023-12-26 21:20:57,298][105692] Updated weights for policy 0, policy_version 822485 (0.0009) [2023-12-26 21:20:57,350][105692] Updated weights for policy 0, policy_version 822495 (0.0010) [2023-12-26 21:20:57,466][105620] Updated weights for policy 1, policy_version 822318 (0.0010) [2023-12-26 21:20:57,515][105620] Updated weights for policy 1, policy_version 822328 (0.0005) [2023-12-26 21:20:57,561][105620] Updated weights for policy 1, policy_version 822338 (0.0005) [2023-12-26 21:20:58,002][105692] Updated weights for policy 0, policy_version 822505 (0.0009) [2023-12-26 21:20:58,053][105692] Updated weights for policy 0, policy_version 822515 (0.0010) [2023-12-26 21:20:58,104][105620] Updated weights for policy 1, policy_version 822348 (0.0005) [2023-12-26 21:20:58,105][105692] Updated weights for policy 0, policy_version 822525 (0.0010) [2023-12-26 21:20:58,163][105692] Updated weights for policy 0, policy_version 822535 (0.0011) [2023-12-26 21:20:58,170][105620] Updated weights for policy 1, policy_version 822358 (0.0007) [2023-12-26 21:20:58,228][105620] Updated weights for policy 1, policy_version 822368 (0.0007) [2023-12-26 21:20:58,960][105620] Updated weights for policy 1, policy_version 822378 (0.0008) [2023-12-26 21:20:59,005][105692] Updated weights for policy 0, policy_version 822545 (0.0008) [2023-12-26 21:20:59,027][105620] Updated weights for policy 1, policy_version 822388 (0.0008) [2023-12-26 21:20:59,064][105692] Updated weights for policy 0, policy_version 822555 (0.0008) [2023-12-26 21:20:59,088][105620] Updated weights for policy 1, policy_version 822398 (0.0007) [2023-12-26 21:20:59,128][105692] Updated weights for policy 0, policy_version 822565 (0.0009) [2023-12-26 21:20:59,148][105620] Updated weights for policy 1, policy_version 822408 (0.0009) [2023-12-26 21:20:59,864][105620] Updated weights for policy 1, policy_version 822418 (0.0008) [2023-12-26 21:20:59,904][105692] Updated weights for policy 0, policy_version 822575 (0.0008) [2023-12-26 21:20:59,921][105620] Updated weights for policy 1, policy_version 822428 (0.0006) [2023-12-26 21:20:59,968][105692] Updated weights for policy 0, policy_version 822585 (0.0007) [2023-12-26 21:20:59,987][105620] Updated weights for policy 1, policy_version 822438 (0.0006) [2023-12-26 21:21:00,028][105692] Updated weights for policy 0, policy_version 822595 (0.0009) [2023-12-26 21:21:00,559][105620] Updated weights for policy 1, policy_version 822448 (0.0006) [2023-12-26 21:21:00,628][105620] Updated weights for policy 1, policy_version 822458 (0.0008) [2023-12-26 21:21:00,686][105620] Updated weights for policy 1, policy_version 822468 (0.0010) [2023-12-26 21:21:00,728][105692] Updated weights for policy 0, policy_version 822605 (0.0007) [2023-12-26 21:21:00,774][105692] Updated weights for policy 0, policy_version 822615 (0.0010) [2023-12-26 21:21:00,836][105692] Updated weights for policy 0, policy_version 822625 (0.0011) [2023-12-26 21:21:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 421199872. Throughput: 0: 9738.4, 1: 9817.6. Samples: 421167960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:21:01,062][104569] Avg episode reward: [(0, '8725.410'), (1, '9322.156')] [2023-12-26 21:21:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000822472_210575360.pth... [2023-12-26 21:21:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000822632_210624512.pth... [2023-12-26 21:21:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000821320_210280448.pth [2023-12-26 21:21:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000821512_210337792.pth [2023-12-26 21:21:01,292][105620] Updated weights for policy 1, policy_version 822478 (0.0011) [2023-12-26 21:21:01,357][105620] Updated weights for policy 1, policy_version 822488 (0.0011) [2023-12-26 21:21:01,376][105586] KL-divergence is very high: 124.1501 [2023-12-26 21:21:01,421][105620] Updated weights for policy 1, policy_version 822498 (0.0007) [2023-12-26 21:21:01,430][105586] KL-divergence is very high: 117.2496 [2023-12-26 21:21:01,535][105692] Updated weights for policy 0, policy_version 822635 (0.0010) [2023-12-26 21:21:01,584][105692] Updated weights for policy 0, policy_version 822645 (0.0010) [2023-12-26 21:21:01,634][105692] Updated weights for policy 0, policy_version 822655 (0.0009) [2023-12-26 21:21:02,162][105620] Updated weights for policy 1, policy_version 822508 (0.0011) [2023-12-26 21:21:02,221][105620] Updated weights for policy 1, policy_version 822518 (0.0010) [2023-12-26 21:21:02,282][105620] Updated weights for policy 1, policy_version 822528 (0.0011) [2023-12-26 21:21:02,309][105692] Updated weights for policy 0, policy_version 822665 (0.0006) [2023-12-26 21:21:02,374][105692] Updated weights for policy 0, policy_version 822675 (0.0007) [2023-12-26 21:21:02,434][105692] Updated weights for policy 0, policy_version 822685 (0.0006) [2023-12-26 21:21:02,497][105692] Updated weights for policy 0, policy_version 822695 (0.0010) [2023-12-26 21:21:02,948][105620] Updated weights for policy 1, policy_version 822538 (0.0010) [2023-12-26 21:21:03,005][105620] Updated weights for policy 1, policy_version 822548 (0.0006) [2023-12-26 21:21:03,070][105620] Updated weights for policy 1, policy_version 822558 (0.0006) [2023-12-26 21:21:03,128][105620] Updated weights for policy 1, policy_version 822568 (0.0010) [2023-12-26 21:21:03,191][105692] Updated weights for policy 0, policy_version 822705 (0.0005) [2023-12-26 21:21:03,236][105692] Updated weights for policy 0, policy_version 822715 (0.0005) [2023-12-26 21:21:03,282][105692] Updated weights for policy 0, policy_version 822725 (0.0006) [2023-12-26 21:21:03,779][105620] Updated weights for policy 1, policy_version 822578 (0.0010) [2023-12-26 21:21:03,826][105692] Updated weights for policy 0, policy_version 822735 (0.0005) [2023-12-26 21:21:03,841][105620] Updated weights for policy 1, policy_version 822588 (0.0008) [2023-12-26 21:21:03,892][105692] Updated weights for policy 0, policy_version 822745 (0.0011) [2023-12-26 21:21:03,902][105620] Updated weights for policy 1, policy_version 822598 (0.0006) [2023-12-26 21:21:03,943][105692] Updated weights for policy 0, policy_version 822755 (0.0007) [2023-12-26 21:21:04,635][105692] Updated weights for policy 0, policy_version 822765 (0.0007) [2023-12-26 21:21:04,668][105620] Updated weights for policy 1, policy_version 822608 (0.0006) [2023-12-26 21:21:04,687][105692] Updated weights for policy 0, policy_version 822775 (0.0008) [2023-12-26 21:21:04,731][105620] Updated weights for policy 1, policy_version 822618 (0.0006) [2023-12-26 21:21:04,740][105692] Updated weights for policy 0, policy_version 822785 (0.0007) [2023-12-26 21:21:04,800][105620] Updated weights for policy 1, policy_version 822628 (0.0006) [2023-12-26 21:21:05,362][105692] Updated weights for policy 0, policy_version 822795 (0.0007) [2023-12-26 21:21:05,417][105692] Updated weights for policy 0, policy_version 822805 (0.0005) [2023-12-26 21:21:05,466][105692] Updated weights for policy 0, policy_version 822815 (0.0006) [2023-12-26 21:21:05,480][105620] Updated weights for policy 1, policy_version 822638 (0.0007) [2023-12-26 21:21:05,534][105620] Updated weights for policy 1, policy_version 822648 (0.0007) [2023-12-26 21:21:05,578][105620] Updated weights for policy 1, policy_version 822658 (0.0010) [2023-12-26 21:21:06,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 421298176. Throughput: 0: 9842.0, 1: 9852.3. Samples: 421287808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:21:06,063][104569] Avg episode reward: [(0, '8539.747'), (1, '9155.535')] [2023-12-26 21:21:06,095][105692] Updated weights for policy 0, policy_version 822825 (0.0010) [2023-12-26 21:21:06,164][105692] Updated weights for policy 0, policy_version 822835 (0.0009) [2023-12-26 21:21:06,221][105692] Updated weights for policy 0, policy_version 822845 (0.0011) [2023-12-26 21:21:06,230][105620] Updated weights for policy 1, policy_version 822668 (0.0008) [2023-12-26 21:21:06,289][105692] Updated weights for policy 0, policy_version 822855 (0.0011) [2023-12-26 21:21:06,318][105620] Updated weights for policy 1, policy_version 822678 (0.0010) [2023-12-26 21:21:06,388][105620] Updated weights for policy 1, policy_version 822688 (0.0010) [2023-12-26 21:21:07,030][105692] Updated weights for policy 0, policy_version 822865 (0.0011) [2023-12-26 21:21:07,043][105620] Updated weights for policy 1, policy_version 822698 (0.0009) [2023-12-26 21:21:07,082][105692] Updated weights for policy 0, policy_version 822875 (0.0010) [2023-12-26 21:21:07,102][105620] Updated weights for policy 1, policy_version 822708 (0.0007) [2023-12-26 21:21:07,138][105692] Updated weights for policy 0, policy_version 822885 (0.0010) [2023-12-26 21:21:07,152][105620] Updated weights for policy 1, policy_version 822718 (0.0007) [2023-12-26 21:21:07,214][105620] Updated weights for policy 1, policy_version 822728 (0.0008) [2023-12-26 21:21:07,754][105692] Updated weights for policy 0, policy_version 822895 (0.0008) [2023-12-26 21:21:07,813][105620] Updated weights for policy 1, policy_version 822738 (0.0008) [2023-12-26 21:21:07,818][105692] Updated weights for policy 0, policy_version 822905 (0.0005) [2023-12-26 21:21:07,865][105620] Updated weights for policy 1, policy_version 822748 (0.0010) [2023-12-26 21:21:07,878][105692] Updated weights for policy 0, policy_version 822915 (0.0009) [2023-12-26 21:21:07,922][105620] Updated weights for policy 1, policy_version 822758 (0.0010) [2023-12-26 21:21:08,565][105692] Updated weights for policy 0, policy_version 822925 (0.0010) [2023-12-26 21:21:08,627][105692] Updated weights for policy 0, policy_version 822935 (0.0010) [2023-12-26 21:21:08,653][105620] Updated weights for policy 1, policy_version 822768 (0.0008) [2023-12-26 21:21:08,682][105692] Updated weights for policy 0, policy_version 822945 (0.0010) [2023-12-26 21:21:08,712][105620] Updated weights for policy 1, policy_version 822778 (0.0006) [2023-12-26 21:21:08,771][105620] Updated weights for policy 1, policy_version 822788 (0.0008) [2023-12-26 21:21:09,450][105692] Updated weights for policy 0, policy_version 822955 (0.0009) [2023-12-26 21:21:09,507][105692] Updated weights for policy 0, policy_version 822965 (0.0006) [2023-12-26 21:21:09,522][105620] Updated weights for policy 1, policy_version 822798 (0.0009) [2023-12-26 21:21:09,566][105692] Updated weights for policy 0, policy_version 822975 (0.0011) [2023-12-26 21:21:09,571][105620] Updated weights for policy 1, policy_version 822808 (0.0011) [2023-12-26 21:21:09,624][105620] Updated weights for policy 1, policy_version 822818 (0.0010) [2023-12-26 21:21:10,323][105692] Updated weights for policy 0, policy_version 822985 (0.0011) [2023-12-26 21:21:10,355][105620] Updated weights for policy 1, policy_version 822828 (0.0010) [2023-12-26 21:21:10,389][105692] Updated weights for policy 0, policy_version 822995 (0.0010) [2023-12-26 21:21:10,408][105620] Updated weights for policy 1, policy_version 822838 (0.0006) [2023-12-26 21:21:10,456][105692] Updated weights for policy 0, policy_version 823005 (0.0008) [2023-12-26 21:21:10,472][105620] Updated weights for policy 1, policy_version 822848 (0.0009) [2023-12-26 21:21:10,520][105692] Updated weights for policy 0, policy_version 823015 (0.0006) [2023-12-26 21:21:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 421396480. Throughput: 0: 9877.9, 1: 9836.4. Samples: 421407736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:21:11,062][104569] Avg episode reward: [(0, '8093.375'), (1, '9151.017')] [2023-12-26 21:21:11,096][105692] Updated weights for policy 0, policy_version 823025 (0.0008) [2023-12-26 21:21:11,165][105692] Updated weights for policy 0, policy_version 823035 (0.0008) [2023-12-26 21:21:11,221][105692] Updated weights for policy 0, policy_version 823045 (0.0008) [2023-12-26 21:21:11,307][105620] Updated weights for policy 1, policy_version 822858 (0.0009) [2023-12-26 21:21:11,380][105620] Updated weights for policy 1, policy_version 822868 (0.0011) [2023-12-26 21:21:11,445][105620] Updated weights for policy 1, policy_version 822878 (0.0008) [2023-12-26 21:21:11,505][105620] Updated weights for policy 1, policy_version 822888 (0.0008) [2023-12-26 21:21:12,023][105692] Updated weights for policy 0, policy_version 823055 (0.0010) [2023-12-26 21:21:12,089][105692] Updated weights for policy 0, policy_version 823065 (0.0008) [2023-12-26 21:21:12,149][105692] Updated weights for policy 0, policy_version 823075 (0.0011) [2023-12-26 21:21:12,241][105620] Updated weights for policy 1, policy_version 822898 (0.0009) [2023-12-26 21:21:12,307][105620] Updated weights for policy 1, policy_version 822908 (0.0009) [2023-12-26 21:21:12,368][105620] Updated weights for policy 1, policy_version 822918 (0.0008) [2023-12-26 21:21:12,781][105692] Updated weights for policy 0, policy_version 823085 (0.0007) [2023-12-26 21:21:12,830][105692] Updated weights for policy 0, policy_version 823095 (0.0005) [2023-12-26 21:21:12,887][105692] Updated weights for policy 0, policy_version 823105 (0.0007) [2023-12-26 21:21:13,210][105620] Updated weights for policy 1, policy_version 822928 (0.0006) [2023-12-26 21:21:13,267][105620] Updated weights for policy 1, policy_version 822938 (0.0006) [2023-12-26 21:21:13,330][105620] Updated weights for policy 1, policy_version 822948 (0.0005) [2023-12-26 21:21:13,488][105692] Updated weights for policy 0, policy_version 823115 (0.0009) [2023-12-26 21:21:13,545][105692] Updated weights for policy 0, policy_version 823125 (0.0005) [2023-12-26 21:21:13,601][105692] Updated weights for policy 0, policy_version 823135 (0.0005) [2023-12-26 21:21:13,986][105620] Updated weights for policy 1, policy_version 822958 (0.0007) [2023-12-26 21:21:14,038][105620] Updated weights for policy 1, policy_version 822968 (0.0010) [2023-12-26 21:21:14,097][105620] Updated weights for policy 1, policy_version 822978 (0.0010) [2023-12-26 21:21:14,176][105692] Updated weights for policy 0, policy_version 823145 (0.0005) [2023-12-26 21:21:14,222][105692] Updated weights for policy 0, policy_version 823155 (0.0005) [2023-12-26 21:21:14,269][105692] Updated weights for policy 0, policy_version 823165 (0.0005) [2023-12-26 21:21:14,315][105692] Updated weights for policy 0, policy_version 823175 (0.0005) [2023-12-26 21:21:14,784][105620] Updated weights for policy 1, policy_version 822988 (0.0010) [2023-12-26 21:21:14,848][105620] Updated weights for policy 1, policy_version 822998 (0.0009) [2023-12-26 21:21:14,911][105620] Updated weights for policy 1, policy_version 823008 (0.0010) [2023-12-26 21:21:14,978][105692] Updated weights for policy 0, policy_version 823185 (0.0010) [2023-12-26 21:21:15,041][105692] Updated weights for policy 0, policy_version 823195 (0.0011) [2023-12-26 21:21:15,101][105692] Updated weights for policy 0, policy_version 823205 (0.0010) [2023-12-26 21:21:15,544][105620] Updated weights for policy 1, policy_version 823018 (0.0010) [2023-12-26 21:21:15,592][105620] Updated weights for policy 1, policy_version 823028 (0.0010) [2023-12-26 21:21:15,641][105620] Updated weights for policy 1, policy_version 823038 (0.0010) [2023-12-26 21:21:15,686][105620] Updated weights for policy 1, policy_version 823048 (0.0007) [2023-12-26 21:21:15,833][105692] Updated weights for policy 0, policy_version 823215 (0.0007) [2023-12-26 21:21:15,897][105692] Updated weights for policy 0, policy_version 823225 (0.0005) [2023-12-26 21:21:15,963][105692] Updated weights for policy 0, policy_version 823235 (0.0007) [2023-12-26 21:21:16,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 421502976. Throughput: 0: 9877.7, 1: 9802.1. Samples: 421466448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:21:16,062][104569] Avg episode reward: [(0, '8186.424'), (1, '9258.883')] [2023-12-26 21:21:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000823240_210780160.pth... [2023-12-26 21:21:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000823048_210722816.pth... [2023-12-26 21:21:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000821896_210427904.pth [2023-12-26 21:21:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000822056_210477056.pth [2023-12-26 21:21:16,292][105620] Updated weights for policy 1, policy_version 823058 (0.0005) [2023-12-26 21:21:16,355][105620] Updated weights for policy 1, policy_version 823068 (0.0005) [2023-12-26 21:21:16,420][105620] Updated weights for policy 1, policy_version 823078 (0.0007) [2023-12-26 21:21:16,739][105692] Updated weights for policy 0, policy_version 823245 (0.0008) [2023-12-26 21:21:16,783][105692] Updated weights for policy 0, policy_version 823255 (0.0008) [2023-12-26 21:21:16,838][105692] Updated weights for policy 0, policy_version 823265 (0.0008) [2023-12-26 21:21:17,077][105620] Updated weights for policy 1, policy_version 823088 (0.0010) [2023-12-26 21:21:17,137][105620] Updated weights for policy 1, policy_version 823098 (0.0010) [2023-12-26 21:21:17,197][105620] Updated weights for policy 1, policy_version 823108 (0.0009) [2023-12-26 21:21:17,618][105692] Updated weights for policy 0, policy_version 823275 (0.0009) [2023-12-26 21:21:17,663][105692] Updated weights for policy 0, policy_version 823285 (0.0007) [2023-12-26 21:21:17,719][105692] Updated weights for policy 0, policy_version 823295 (0.0005) [2023-12-26 21:21:17,928][105620] Updated weights for policy 1, policy_version 823118 (0.0010) [2023-12-26 21:21:17,975][105620] Updated weights for policy 1, policy_version 823128 (0.0009) [2023-12-26 21:21:18,026][105620] Updated weights for policy 1, policy_version 823138 (0.0009) [2023-12-26 21:21:18,338][105692] Updated weights for policy 0, policy_version 823305 (0.0005) [2023-12-26 21:21:18,398][105692] Updated weights for policy 0, policy_version 823315 (0.0008) [2023-12-26 21:21:18,461][105692] Updated weights for policy 0, policy_version 823325 (0.0009) [2023-12-26 21:21:18,517][105692] Updated weights for policy 0, policy_version 823335 (0.0009) [2023-12-26 21:21:18,726][105620] Updated weights for policy 1, policy_version 823148 (0.0009) [2023-12-26 21:21:18,792][105620] Updated weights for policy 1, policy_version 823158 (0.0009) [2023-12-26 21:21:18,858][105620] Updated weights for policy 1, policy_version 823168 (0.0009) [2023-12-26 21:21:19,271][105692] Updated weights for policy 0, policy_version 823345 (0.0009) [2023-12-26 21:21:19,335][105692] Updated weights for policy 0, policy_version 823355 (0.0009) [2023-12-26 21:21:19,399][105692] Updated weights for policy 0, policy_version 823365 (0.0008) [2023-12-26 21:21:19,624][105620] Updated weights for policy 1, policy_version 823178 (0.0008) [2023-12-26 21:21:19,687][105620] Updated weights for policy 1, policy_version 823188 (0.0008) [2023-12-26 21:21:19,754][105620] Updated weights for policy 1, policy_version 823198 (0.0009) [2023-12-26 21:21:19,820][105620] Updated weights for policy 1, policy_version 823208 (0.0009) [2023-12-26 21:21:20,190][105692] Updated weights for policy 0, policy_version 823375 (0.0007) [2023-12-26 21:21:20,257][105692] Updated weights for policy 0, policy_version 823385 (0.0008) [2023-12-26 21:21:20,326][105692] Updated weights for policy 0, policy_version 823395 (0.0006) [2023-12-26 21:21:20,582][105620] Updated weights for policy 1, policy_version 823218 (0.0008) [2023-12-26 21:21:20,648][105620] Updated weights for policy 1, policy_version 823228 (0.0009) [2023-12-26 21:21:20,718][105620] Updated weights for policy 1, policy_version 823238 (0.0009) [2023-12-26 21:21:20,985][105692] Updated weights for policy 0, policy_version 823405 (0.0010) [2023-12-26 21:21:21,044][105692] Updated weights for policy 0, policy_version 823415 (0.0008) [2023-12-26 21:21:21,062][104569] Fps is (10 sec: 19659.7, 60 sec: 19660.6, 300 sec: 19605.2). Total num frames: 421593088. Throughput: 0: 9836.1, 1: 9920.3. Samples: 421585496. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 21:21:21,063][104569] Avg episode reward: [(0, '8452.900'), (1, '9350.633')] [2023-12-26 21:21:21,104][105692] Updated weights for policy 0, policy_version 823425 (0.0006) [2023-12-26 21:21:21,542][105620] Updated weights for policy 1, policy_version 823248 (0.0009) [2023-12-26 21:21:21,604][105620] Updated weights for policy 1, policy_version 823258 (0.0010) [2023-12-26 21:21:21,670][105620] Updated weights for policy 1, policy_version 823268 (0.0009) [2023-12-26 21:21:21,868][105692] Updated weights for policy 0, policy_version 823435 (0.0008) [2023-12-26 21:21:21,931][105692] Updated weights for policy 0, policy_version 823445 (0.0011) [2023-12-26 21:21:21,994][105692] Updated weights for policy 0, policy_version 823455 (0.0011) [2023-12-26 21:21:22,427][105620] Updated weights for policy 1, policy_version 823278 (0.0008) [2023-12-26 21:21:22,483][105620] Updated weights for policy 1, policy_version 823288 (0.0008) [2023-12-26 21:21:22,540][105620] Updated weights for policy 1, policy_version 823298 (0.0008) [2023-12-26 21:21:22,750][105692] Updated weights for policy 0, policy_version 823465 (0.0011) [2023-12-26 21:21:22,803][105692] Updated weights for policy 0, policy_version 823475 (0.0011) [2023-12-26 21:21:22,856][105692] Updated weights for policy 0, policy_version 823485 (0.0010) [2023-12-26 21:21:22,905][105692] Updated weights for policy 0, policy_version 823495 (0.0010) [2023-12-26 21:21:23,176][105620] Updated weights for policy 1, policy_version 823308 (0.0007) [2023-12-26 21:21:23,225][105620] Updated weights for policy 1, policy_version 823318 (0.0006) [2023-12-26 21:21:23,291][105620] Updated weights for policy 1, policy_version 823328 (0.0009) [2023-12-26 21:21:23,649][105692] Updated weights for policy 0, policy_version 823505 (0.0006) [2023-12-26 21:21:23,712][105692] Updated weights for policy 0, policy_version 823515 (0.0005) [2023-12-26 21:21:23,770][105692] Updated weights for policy 0, policy_version 823525 (0.0008) [2023-12-26 21:21:23,855][105620] Updated weights for policy 1, policy_version 823338 (0.0007) [2023-12-26 21:21:23,902][105620] Updated weights for policy 1, policy_version 823348 (0.0008) [2023-12-26 21:21:23,949][105620] Updated weights for policy 1, policy_version 823358 (0.0008) [2023-12-26 21:21:24,003][105620] Updated weights for policy 1, policy_version 823368 (0.0009) [2023-12-26 21:21:24,424][105692] Updated weights for policy 0, policy_version 823535 (0.0007) [2023-12-26 21:21:24,478][105692] Updated weights for policy 0, policy_version 823545 (0.0005) [2023-12-26 21:21:24,542][105692] Updated weights for policy 0, policy_version 823555 (0.0005) [2023-12-26 21:21:24,794][105620] Updated weights for policy 1, policy_version 823378 (0.0007) [2023-12-26 21:21:24,850][105620] Updated weights for policy 1, policy_version 823388 (0.0008) [2023-12-26 21:21:24,915][105620] Updated weights for policy 1, policy_version 823398 (0.0008) [2023-12-26 21:21:25,150][105692] Updated weights for policy 0, policy_version 823565 (0.0005) [2023-12-26 21:21:25,201][105692] Updated weights for policy 0, policy_version 823575 (0.0010) [2023-12-26 21:21:25,249][105692] Updated weights for policy 0, policy_version 823585 (0.0010) [2023-12-26 21:21:25,710][105620] Updated weights for policy 1, policy_version 823408 (0.0009) [2023-12-26 21:21:25,764][105620] Updated weights for policy 1, policy_version 823419 (0.0010) [2023-12-26 21:21:25,822][105620] Updated weights for policy 1, policy_version 823430 (0.0010) [2023-12-26 21:21:25,854][105692] Updated weights for policy 0, policy_version 823595 (0.0010) [2023-12-26 21:21:25,906][105692] Updated weights for policy 0, policy_version 823605 (0.0010) [2023-12-26 21:21:25,953][105692] Updated weights for policy 0, policy_version 823615 (0.0010) [2023-12-26 21:21:26,062][104569] Fps is (10 sec: 19660.0, 60 sec: 19797.2, 300 sec: 19633.0). Total num frames: 421699584. Throughput: 0: 9902.5, 1: 9840.7. Samples: 421701196. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 21:21:26,063][104569] Avg episode reward: [(0, '8809.365'), (1, '9167.861')] [2023-12-26 21:21:26,510][105620] Updated weights for policy 1, policy_version 823440 (0.0007) [2023-12-26 21:21:26,562][105620] Updated weights for policy 1, policy_version 823450 (0.0008) [2023-12-26 21:21:26,622][105620] Updated weights for policy 1, policy_version 823460 (0.0008) [2023-12-26 21:21:26,656][105692] Updated weights for policy 0, policy_version 823625 (0.0010) [2023-12-26 21:21:26,715][105692] Updated weights for policy 0, policy_version 823635 (0.0006) [2023-12-26 21:21:26,777][105692] Updated weights for policy 0, policy_version 823645 (0.0009) [2023-12-26 21:21:26,829][105692] Updated weights for policy 0, policy_version 823655 (0.0010) [2023-12-26 21:21:27,400][105620] Updated weights for policy 1, policy_version 823470 (0.0006) [2023-12-26 21:21:27,451][105620] Updated weights for policy 1, policy_version 823480 (0.0009) [2023-12-26 21:21:27,496][105692] Updated weights for policy 0, policy_version 823665 (0.0006) [2023-12-26 21:21:27,498][105620] Updated weights for policy 1, policy_version 823490 (0.0008) [2023-12-26 21:21:27,543][105692] Updated weights for policy 0, policy_version 823675 (0.0005) [2023-12-26 21:21:27,596][105692] Updated weights for policy 0, policy_version 823685 (0.0005) [2023-12-26 21:21:28,188][105692] Updated weights for policy 0, policy_version 823695 (0.0008) [2023-12-26 21:21:28,238][105692] Updated weights for policy 0, policy_version 823705 (0.0009) [2023-12-26 21:21:28,291][105692] Updated weights for policy 0, policy_version 823715 (0.0008) [2023-12-26 21:21:28,308][105620] Updated weights for policy 1, policy_version 823500 (0.0008) [2023-12-26 21:21:28,371][105620] Updated weights for policy 1, policy_version 823510 (0.0009) [2023-12-26 21:21:28,430][105620] Updated weights for policy 1, policy_version 823520 (0.0009) [2023-12-26 21:21:29,077][105692] Updated weights for policy 0, policy_version 823725 (0.0007) [2023-12-26 21:21:29,148][105692] Updated weights for policy 0, policy_version 823735 (0.0006) [2023-12-26 21:21:29,187][105620] Updated weights for policy 1, policy_version 823530 (0.0009) [2023-12-26 21:21:29,196][105692] Updated weights for policy 0, policy_version 823745 (0.0009) [2023-12-26 21:21:29,250][105620] Updated weights for policy 1, policy_version 823540 (0.0008) [2023-12-26 21:21:29,311][105620] Updated weights for policy 1, policy_version 823550 (0.0010) [2023-12-26 21:21:29,381][105620] Updated weights for policy 1, policy_version 823560 (0.0007) [2023-12-26 21:21:29,842][105692] Updated weights for policy 0, policy_version 823755 (0.0008) [2023-12-26 21:21:29,905][105692] Updated weights for policy 0, policy_version 823765 (0.0009) [2023-12-26 21:21:29,972][105692] Updated weights for policy 0, policy_version 823775 (0.0009) [2023-12-26 21:21:30,148][105620] Updated weights for policy 1, policy_version 823570 (0.0010) [2023-12-26 21:21:30,209][105620] Updated weights for policy 1, policy_version 823580 (0.0010) [2023-12-26 21:21:30,261][105620] Updated weights for policy 1, policy_version 823590 (0.0010) [2023-12-26 21:21:30,778][105692] Updated weights for policy 0, policy_version 823785 (0.0008) [2023-12-26 21:21:30,842][105692] Updated weights for policy 0, policy_version 823795 (0.0009) [2023-12-26 21:21:30,848][105620] Updated weights for policy 1, policy_version 823600 (0.0006) [2023-12-26 21:21:30,899][105620] Updated weights for policy 1, policy_version 823610 (0.0006) [2023-12-26 21:21:30,899][105692] Updated weights for policy 0, policy_version 823805 (0.0008) [2023-12-26 21:21:30,949][105692] Updated weights for policy 0, policy_version 823815 (0.0006) [2023-12-26 21:21:30,950][105620] Updated weights for policy 1, policy_version 823620 (0.0007) [2023-12-26 21:21:31,062][104569] Fps is (10 sec: 20480.9, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 421797888. Throughput: 0: 9969.1, 1: 9810.2. Samples: 421762004. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 21:21:31,062][104569] Avg episode reward: [(0, '8625.075'), (1, '8719.971')] [2023-12-26 21:21:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000823816_210927616.pth... [2023-12-26 21:21:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000823624_210870272.pth... [2023-12-26 21:21:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000822632_210624512.pth [2023-12-26 21:21:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000822472_210575360.pth [2023-12-26 21:21:31,599][105620] Updated weights for policy 1, policy_version 823630 (0.0006) [2023-12-26 21:21:31,664][105620] Updated weights for policy 1, policy_version 823640 (0.0010) [2023-12-26 21:21:31,727][105620] Updated weights for policy 1, policy_version 823650 (0.0008) [2023-12-26 21:21:31,758][105692] Updated weights for policy 0, policy_version 823825 (0.0008) [2023-12-26 21:21:31,809][105692] Updated weights for policy 0, policy_version 823835 (0.0005) [2023-12-26 21:21:31,866][105692] Updated weights for policy 0, policy_version 823845 (0.0005) [2023-12-26 21:21:32,467][105620] Updated weights for policy 1, policy_version 823660 (0.0007) [2023-12-26 21:21:32,527][105620] Updated weights for policy 1, policy_version 823670 (0.0008) [2023-12-26 21:21:32,538][105692] Updated weights for policy 0, policy_version 823855 (0.0007) [2023-12-26 21:21:32,584][105620] Updated weights for policy 1, policy_version 823680 (0.0007) [2023-12-26 21:21:32,594][105692] Updated weights for policy 0, policy_version 823865 (0.0007) [2023-12-26 21:21:32,656][105692] Updated weights for policy 0, policy_version 823875 (0.0007) [2023-12-26 21:21:33,330][105692] Updated weights for policy 0, policy_version 823885 (0.0009) [2023-12-26 21:21:33,376][105620] Updated weights for policy 1, policy_version 823690 (0.0007) [2023-12-26 21:21:33,377][105692] Updated weights for policy 0, policy_version 823895 (0.0010) [2023-12-26 21:21:33,428][105692] Updated weights for policy 0, policy_version 823905 (0.0010) [2023-12-26 21:21:33,431][105620] Updated weights for policy 1, policy_version 823700 (0.0006) [2023-12-26 21:21:33,489][105620] Updated weights for policy 1, policy_version 823710 (0.0006) [2023-12-26 21:21:33,545][105620] Updated weights for policy 1, policy_version 823720 (0.0005) [2023-12-26 21:21:34,079][105692] Updated weights for policy 0, policy_version 823915 (0.0010) [2023-12-26 21:21:34,138][105692] Updated weights for policy 0, policy_version 823925 (0.0011) [2023-12-26 21:21:34,175][105620] Updated weights for policy 1, policy_version 823730 (0.0008) [2023-12-26 21:21:34,206][105692] Updated weights for policy 0, policy_version 823935 (0.0011) [2023-12-26 21:21:34,234][105620] Updated weights for policy 1, policy_version 823740 (0.0010) [2023-12-26 21:21:34,299][105620] Updated weights for policy 1, policy_version 823750 (0.0008) [2023-12-26 21:21:34,962][105692] Updated weights for policy 0, policy_version 823945 (0.0011) [2023-12-26 21:21:35,021][105692] Updated weights for policy 0, policy_version 823955 (0.0011) [2023-12-26 21:21:35,058][105620] Updated weights for policy 1, policy_version 823760 (0.0011) [2023-12-26 21:21:35,070][105692] Updated weights for policy 0, policy_version 823965 (0.0011) [2023-12-26 21:21:35,118][105620] Updated weights for policy 1, policy_version 823770 (0.0011) [2023-12-26 21:21:35,126][105692] Updated weights for policy 0, policy_version 823975 (0.0011) [2023-12-26 21:21:35,174][105620] Updated weights for policy 1, policy_version 823780 (0.0010) [2023-12-26 21:21:35,888][105692] Updated weights for policy 0, policy_version 823985 (0.0011) [2023-12-26 21:21:35,918][105620] Updated weights for policy 1, policy_version 823790 (0.0008) [2023-12-26 21:21:35,946][105692] Updated weights for policy 0, policy_version 823995 (0.0011) [2023-12-26 21:21:35,980][105620] Updated weights for policy 1, policy_version 823800 (0.0006) [2023-12-26 21:21:36,011][105692] Updated weights for policy 0, policy_version 824005 (0.0010) [2023-12-26 21:21:36,041][105620] Updated weights for policy 1, policy_version 823810 (0.0007) [2023-12-26 21:21:36,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 421888000. Throughput: 0: 9877.8, 1: 9883.4. Samples: 421879056. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 21:21:36,063][104569] Avg episode reward: [(0, '8812.040'), (1, '8713.528')] [2023-12-26 21:21:36,753][105692] Updated weights for policy 0, policy_version 824015 (0.0011) [2023-12-26 21:21:36,754][105620] Updated weights for policy 1, policy_version 823820 (0.0007) [2023-12-26 21:21:36,808][105692] Updated weights for policy 0, policy_version 824025 (0.0011) [2023-12-26 21:21:36,815][105620] Updated weights for policy 1, policy_version 823830 (0.0006) [2023-12-26 21:21:36,867][105692] Updated weights for policy 0, policy_version 824035 (0.0010) [2023-12-26 21:21:36,873][105620] Updated weights for policy 1, policy_version 823840 (0.0006) [2023-12-26 21:21:37,561][105620] Updated weights for policy 1, policy_version 823850 (0.0007) [2023-12-26 21:21:37,565][105692] Updated weights for policy 0, policy_version 824045 (0.0008) [2023-12-26 21:21:37,621][105620] Updated weights for policy 1, policy_version 823860 (0.0009) [2023-12-26 21:21:37,634][105692] Updated weights for policy 0, policy_version 824055 (0.0005) [2023-12-26 21:21:37,681][105692] Updated weights for policy 0, policy_version 824065 (0.0005) [2023-12-26 21:21:37,683][105620] Updated weights for policy 1, policy_version 823870 (0.0005) [2023-12-26 21:21:37,741][105620] Updated weights for policy 1, policy_version 823880 (0.0006) [2023-12-26 21:21:38,199][105692] Updated weights for policy 0, policy_version 824075 (0.0005) [2023-12-26 21:21:38,253][105692] Updated weights for policy 0, policy_version 824085 (0.0005) [2023-12-26 21:21:38,301][105692] Updated weights for policy 0, policy_version 824095 (0.0005) [2023-12-26 21:21:38,432][105620] Updated weights for policy 1, policy_version 823890 (0.0009) [2023-12-26 21:21:38,491][105620] Updated weights for policy 1, policy_version 823900 (0.0008) [2023-12-26 21:21:38,550][105620] Updated weights for policy 1, policy_version 823910 (0.0008) [2023-12-26 21:21:39,008][105692] Updated weights for policy 0, policy_version 824105 (0.0009) [2023-12-26 21:21:39,053][105692] Updated weights for policy 0, policy_version 824115 (0.0010) [2023-12-26 21:21:39,108][105692] Updated weights for policy 0, policy_version 824125 (0.0010) [2023-12-26 21:21:39,160][105692] Updated weights for policy 0, policy_version 824135 (0.0010) [2023-12-26 21:21:39,251][105620] Updated weights for policy 1, policy_version 823920 (0.0009) [2023-12-26 21:21:39,320][105620] Updated weights for policy 1, policy_version 823930 (0.0010) [2023-12-26 21:21:39,386][105620] Updated weights for policy 1, policy_version 823940 (0.0008) [2023-12-26 21:21:39,887][105692] Updated weights for policy 0, policy_version 824145 (0.0010) [2023-12-26 21:21:39,948][105692] Updated weights for policy 0, policy_version 824155 (0.0009) [2023-12-26 21:21:40,014][105692] Updated weights for policy 0, policy_version 824165 (0.0011) [2023-12-26 21:21:40,215][105620] Updated weights for policy 1, policy_version 823950 (0.0008) [2023-12-26 21:21:40,272][105620] Updated weights for policy 1, policy_version 823960 (0.0008) [2023-12-26 21:21:40,333][105620] Updated weights for policy 1, policy_version 823970 (0.0008) [2023-12-26 21:21:40,714][105692] Updated weights for policy 0, policy_version 824175 (0.0011) [2023-12-26 21:21:40,763][105692] Updated weights for policy 0, policy_version 824185 (0.0010) [2023-12-26 21:21:40,819][105692] Updated weights for policy 0, policy_version 824195 (0.0011) [2023-12-26 21:21:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 421986304. Throughput: 0: 9958.0, 1: 9801.9. Samples: 421995260. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 21:21:41,063][104569] Avg episode reward: [(0, '8991.662'), (1, '8881.649')] [2023-12-26 21:21:41,137][105620] Updated weights for policy 1, policy_version 823980 (0.0009) [2023-12-26 21:21:41,199][105620] Updated weights for policy 1, policy_version 823990 (0.0007) [2023-12-26 21:21:41,265][105620] Updated weights for policy 1, policy_version 824000 (0.0007) [2023-12-26 21:21:41,620][105692] Updated weights for policy 0, policy_version 824205 (0.0010) [2023-12-26 21:21:41,693][105692] Updated weights for policy 0, policy_version 824215 (0.0009) [2023-12-26 21:21:41,758][105692] Updated weights for policy 0, policy_version 824225 (0.0010) [2023-12-26 21:21:42,069][105620] Updated weights for policy 1, policy_version 824010 (0.0010) [2023-12-26 21:21:42,130][105620] Updated weights for policy 1, policy_version 824020 (0.0008) [2023-12-26 21:21:42,186][105620] Updated weights for policy 1, policy_version 824030 (0.0006) [2023-12-26 21:21:42,246][105620] Updated weights for policy 1, policy_version 824040 (0.0007) [2023-12-26 21:21:42,615][105692] Updated weights for policy 0, policy_version 824235 (0.0009) [2023-12-26 21:21:42,674][105692] Updated weights for policy 0, policy_version 824245 (0.0008) [2023-12-26 21:21:42,733][105692] Updated weights for policy 0, policy_version 824255 (0.0008) [2023-12-26 21:21:43,050][105620] Updated weights for policy 1, policy_version 824050 (0.0007) [2023-12-26 21:21:43,108][105620] Updated weights for policy 1, policy_version 824060 (0.0009) [2023-12-26 21:21:43,156][105620] Updated weights for policy 1, policy_version 824070 (0.0010) [2023-12-26 21:21:43,328][105692] Updated weights for policy 0, policy_version 824265 (0.0008) [2023-12-26 21:21:43,385][105692] Updated weights for policy 0, policy_version 824275 (0.0007) [2023-12-26 21:21:43,437][105692] Updated weights for policy 0, policy_version 824285 (0.0010) [2023-12-26 21:21:43,495][105692] Updated weights for policy 0, policy_version 824295 (0.0008) [2023-12-26 21:21:43,981][105620] Updated weights for policy 1, policy_version 824081 (0.0010) [2023-12-26 21:21:44,030][105692] Updated weights for policy 0, policy_version 824305 (0.0005) [2023-12-26 21:21:44,033][105620] Updated weights for policy 1, policy_version 824091 (0.0009) [2023-12-26 21:21:44,079][105692] Updated weights for policy 0, policy_version 824315 (0.0006) [2023-12-26 21:21:44,080][105620] Updated weights for policy 1, policy_version 824101 (0.0009) [2023-12-26 21:21:44,129][105692] Updated weights for policy 0, policy_version 824325 (0.0005) [2023-12-26 21:21:44,769][105692] Updated weights for policy 0, policy_version 824335 (0.0007) [2023-12-26 21:21:44,841][105692] Updated weights for policy 0, policy_version 824345 (0.0011) [2023-12-26 21:21:44,904][105692] Updated weights for policy 0, policy_version 824355 (0.0010) [2023-12-26 21:21:44,931][105620] Updated weights for policy 1, policy_version 824111 (0.0006) [2023-12-26 21:21:44,987][105620] Updated weights for policy 1, policy_version 824121 (0.0008) [2023-12-26 21:21:45,046][105620] Updated weights for policy 1, policy_version 824131 (0.0008) [2023-12-26 21:21:45,587][105692] Updated weights for policy 0, policy_version 824365 (0.0010) [2023-12-26 21:21:45,642][105692] Updated weights for policy 0, policy_version 824375 (0.0010) [2023-12-26 21:21:45,691][105692] Updated weights for policy 0, policy_version 824385 (0.0010) [2023-12-26 21:21:45,850][105620] Updated weights for policy 1, policy_version 824141 (0.0008) [2023-12-26 21:21:45,909][105620] Updated weights for policy 1, policy_version 824151 (0.0009) [2023-12-26 21:21:45,964][105620] Updated weights for policy 1, policy_version 824161 (0.0009) [2023-12-26 21:21:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.7, 300 sec: 19605.2). Total num frames: 422084608. Throughput: 0: 9964.7, 1: 9645.5. Samples: 422050428. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 21:21:46,063][104569] Avg episode reward: [(0, '9076.077'), (1, '8669.932')] [2023-12-26 21:21:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000824392_211075072.pth... [2023-12-26 21:21:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000824168_211009536.pth... [2023-12-26 21:21:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000823240_210780160.pth [2023-12-26 21:21:46,075][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_000824392_211075072.pth [2023-12-26 21:21:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000823048_210722816.pth [2023-12-26 21:21:46,079][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_000824168_211009536.pth [2023-12-26 21:21:46,404][105692] Updated weights for policy 0, policy_version 824395 (0.0009) [2023-12-26 21:21:46,465][105692] Updated weights for policy 0, policy_version 824405 (0.0010) [2023-12-26 21:21:46,521][105692] Updated weights for policy 0, policy_version 824415 (0.0010) [2023-12-26 21:21:46,722][105620] Updated weights for policy 1, policy_version 824171 (0.0007) [2023-12-26 21:21:46,743][105586] KL-divergence is very high: 194.3528 [2023-12-26 21:21:46,775][105620] Updated weights for policy 1, policy_version 824183 (0.0010) [2023-12-26 21:21:46,780][105586] KL-divergence is very high: 317.3041 [2023-12-26 21:21:46,818][105586] KL-divergence is very high: 296.9228 [2023-12-26 21:21:46,832][105620] Updated weights for policy 1, policy_version 824194 (0.0010) [2023-12-26 21:21:47,119][105692] Updated weights for policy 0, policy_version 824425 (0.0010) [2023-12-26 21:21:47,172][105692] Updated weights for policy 0, policy_version 824435 (0.0010) [2023-12-26 21:21:47,220][105692] Updated weights for policy 0, policy_version 824445 (0.0010) [2023-12-26 21:21:47,273][105692] Updated weights for policy 0, policy_version 824455 (0.0010) [2023-12-26 21:21:47,681][105620] Updated weights for policy 1, policy_version 824204 (0.0010) [2023-12-26 21:21:47,736][105620] Updated weights for policy 1, policy_version 824214 (0.0008) [2023-12-26 21:21:47,802][105620] Updated weights for policy 1, policy_version 824224 (0.0010) [2023-12-26 21:21:47,932][105692] Updated weights for policy 0, policy_version 824465 (0.0009) [2023-12-26 21:21:47,980][105692] Updated weights for policy 0, policy_version 824475 (0.0009) [2023-12-26 21:21:48,031][105692] Updated weights for policy 0, policy_version 824485 (0.0009) [2023-12-26 21:21:48,570][105620] Updated weights for policy 1, policy_version 824234 (0.0009) [2023-12-26 21:21:48,627][105620] Updated weights for policy 1, policy_version 824244 (0.0009) [2023-12-26 21:21:48,689][105620] Updated weights for policy 1, policy_version 824254 (0.0009) [2023-12-26 21:21:48,743][105620] Updated weights for policy 1, policy_version 824264 (0.0009) [2023-12-26 21:21:48,823][105692] Updated weights for policy 0, policy_version 824495 (0.0008) [2023-12-26 21:21:48,881][105692] Updated weights for policy 0, policy_version 824505 (0.0008) [2023-12-26 21:21:48,943][105692] Updated weights for policy 0, policy_version 824515 (0.0009) [2023-12-26 21:21:49,529][105620] Updated weights for policy 1, policy_version 824274 (0.0010) [2023-12-26 21:21:49,578][105620] Updated weights for policy 1, policy_version 824284 (0.0009) [2023-12-26 21:21:49,632][105620] Updated weights for policy 1, policy_version 824294 (0.0009) [2023-12-26 21:21:49,696][105692] Updated weights for policy 0, policy_version 824525 (0.0009) [2023-12-26 21:21:49,756][105692] Updated weights for policy 0, policy_version 824535 (0.0009) [2023-12-26 21:21:49,812][105692] Updated weights for policy 0, policy_version 824545 (0.0010) [2023-12-26 21:21:50,394][105620] Updated weights for policy 1, policy_version 824304 (0.0009) [2023-12-26 21:21:50,461][105620] Updated weights for policy 1, policy_version 824314 (0.0010) [2023-12-26 21:21:50,521][105620] Updated weights for policy 1, policy_version 824324 (0.0008) [2023-12-26 21:21:50,626][105692] Updated weights for policy 0, policy_version 824555 (0.0008) [2023-12-26 21:21:50,680][105692] Updated weights for policy 0, policy_version 824565 (0.0008) [2023-12-26 21:21:50,741][105692] Updated weights for policy 0, policy_version 824575 (0.0008) [2023-12-26 21:21:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 422174720. Throughput: 0: 10008.0, 1: 9505.6. Samples: 422165920. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 21:21:51,063][104569] Avg episode reward: [(0, '8896.021'), (1, '8774.428')] [2023-12-26 21:21:51,212][105620] Updated weights for policy 1, policy_version 824334 (0.0008) [2023-12-26 21:21:51,276][105620] Updated weights for policy 1, policy_version 824344 (0.0008) [2023-12-26 21:21:51,330][105620] Updated weights for policy 1, policy_version 824354 (0.0010) [2023-12-26 21:21:51,554][105692] Updated weights for policy 0, policy_version 824585 (0.0009) [2023-12-26 21:21:51,623][105692] Updated weights for policy 0, policy_version 824595 (0.0008) [2023-12-26 21:21:51,684][105692] Updated weights for policy 0, policy_version 824605 (0.0007) [2023-12-26 21:21:51,755][105692] Updated weights for policy 0, policy_version 824615 (0.0008) [2023-12-26 21:21:52,087][105620] Updated weights for policy 1, policy_version 824364 (0.0010) [2023-12-26 21:21:52,149][105620] Updated weights for policy 1, policy_version 824374 (0.0010) [2023-12-26 21:21:52,205][105620] Updated weights for policy 1, policy_version 824384 (0.0007) [2023-12-26 21:21:52,521][105692] Updated weights for policy 0, policy_version 824625 (0.0008) [2023-12-26 21:21:52,583][105692] Updated weights for policy 0, policy_version 824635 (0.0009) [2023-12-26 21:21:52,643][105692] Updated weights for policy 0, policy_version 824645 (0.0008) [2023-12-26 21:21:52,939][105620] Updated weights for policy 1, policy_version 824394 (0.0006) [2023-12-26 21:21:52,988][105620] Updated weights for policy 1, policy_version 824404 (0.0010) [2023-12-26 21:21:53,040][105620] Updated weights for policy 1, policy_version 824414 (0.0010) [2023-12-26 21:21:53,089][105620] Updated weights for policy 1, policy_version 824424 (0.0010) [2023-12-26 21:21:53,285][105692] Updated weights for policy 0, policy_version 824655 (0.0006) [2023-12-26 21:21:53,353][105692] Updated weights for policy 0, policy_version 824665 (0.0006) [2023-12-26 21:21:53,412][105692] Updated weights for policy 0, policy_version 824675 (0.0010) [2023-12-26 21:21:53,875][105620] Updated weights for policy 1, policy_version 824434 (0.0010) [2023-12-26 21:21:53,940][105620] Updated weights for policy 1, policy_version 824444 (0.0010) [2023-12-26 21:21:54,001][105620] Updated weights for policy 1, policy_version 824454 (0.0010) [2023-12-26 21:21:54,052][105692] Updated weights for policy 0, policy_version 824685 (0.0010) [2023-12-26 21:21:54,108][105692] Updated weights for policy 0, policy_version 824695 (0.0010) [2023-12-26 21:21:54,159][105692] Updated weights for policy 0, policy_version 824705 (0.0010) [2023-12-26 21:21:54,715][105620] Updated weights for policy 1, policy_version 824464 (0.0010) [2023-12-26 21:21:54,759][105620] Updated weights for policy 1, policy_version 824474 (0.0010) [2023-12-26 21:21:54,814][105620] Updated weights for policy 1, policy_version 824484 (0.0010) [2023-12-26 21:21:54,834][105586] KL-divergence is very high: 107.9021 [2023-12-26 21:21:54,848][105692] Updated weights for policy 0, policy_version 824715 (0.0009) [2023-12-26 21:21:54,906][105692] Updated weights for policy 0, policy_version 824725 (0.0006) [2023-12-26 21:21:54,962][105692] Updated weights for policy 0, policy_version 824735 (0.0007) [2023-12-26 21:21:55,549][105620] Updated weights for policy 1, policy_version 824494 (0.0009) [2023-12-26 21:21:55,559][105692] Updated weights for policy 0, policy_version 824745 (0.0006) [2023-12-26 21:21:55,603][105620] Updated weights for policy 1, policy_version 824504 (0.0005) [2023-12-26 21:21:55,617][105692] Updated weights for policy 0, policy_version 824755 (0.0010) [2023-12-26 21:21:55,659][105620] Updated weights for policy 1, policy_version 824514 (0.0005) [2023-12-26 21:21:55,675][105692] Updated weights for policy 0, policy_version 824765 (0.0010) [2023-12-26 21:21:55,731][105692] Updated weights for policy 0, policy_version 824775 (0.0010) [2023-12-26 21:21:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 422273024. Throughput: 0: 9962.7, 1: 9439.0. Samples: 422280816. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 21:21:56,063][104569] Avg episode reward: [(0, '8807.925'), (1, '9059.709')] [2023-12-26 21:21:56,364][105620] Updated weights for policy 1, policy_version 824524 (0.0007) [2023-12-26 21:21:56,413][105620] Updated weights for policy 1, policy_version 824534 (0.0008) [2023-12-26 21:21:56,471][105620] Updated weights for policy 1, policy_version 824544 (0.0010) [2023-12-26 21:21:56,474][105692] Updated weights for policy 0, policy_version 824785 (0.0011) [2023-12-26 21:21:56,530][105692] Updated weights for policy 0, policy_version 824795 (0.0011) [2023-12-26 21:21:56,583][105692] Updated weights for policy 0, policy_version 824805 (0.0009) [2023-12-26 21:21:57,161][105620] Updated weights for policy 1, policy_version 824554 (0.0007) [2023-12-26 21:21:57,210][105620] Updated weights for policy 1, policy_version 824564 (0.0005) [2023-12-26 21:21:57,257][105620] Updated weights for policy 1, policy_version 824574 (0.0005) [2023-12-26 21:21:57,311][105620] Updated weights for policy 1, policy_version 824584 (0.0006) [2023-12-26 21:21:57,340][105692] Updated weights for policy 0, policy_version 824815 (0.0010) [2023-12-26 21:21:57,408][105692] Updated weights for policy 0, policy_version 824825 (0.0011) [2023-12-26 21:21:57,476][105692] Updated weights for policy 0, policy_version 824835 (0.0010) [2023-12-26 21:21:57,882][105620] Updated weights for policy 1, policy_version 824594 (0.0006) [2023-12-26 21:21:57,941][105620] Updated weights for policy 1, policy_version 824604 (0.0005) [2023-12-26 21:21:58,009][105620] Updated weights for policy 1, policy_version 824614 (0.0007) [2023-12-26 21:21:58,186][105692] Updated weights for policy 0, policy_version 824845 (0.0009) [2023-12-26 21:21:58,248][105692] Updated weights for policy 0, policy_version 824855 (0.0008) [2023-12-26 21:21:58,307][105692] Updated weights for policy 0, policy_version 824865 (0.0008) [2023-12-26 21:21:58,713][105620] Updated weights for policy 1, policy_version 824624 (0.0007) [2023-12-26 21:21:58,770][105620] Updated weights for policy 1, policy_version 824634 (0.0006) [2023-12-26 21:21:58,844][105620] Updated weights for policy 1, policy_version 824644 (0.0008) [2023-12-26 21:21:59,124][105692] Updated weights for policy 0, policy_version 824875 (0.0009) [2023-12-26 21:21:59,183][105692] Updated weights for policy 0, policy_version 824885 (0.0010) [2023-12-26 21:21:59,256][105692] Updated weights for policy 0, policy_version 824895 (0.0011) [2023-12-26 21:21:59,577][105620] Updated weights for policy 1, policy_version 824654 (0.0006) [2023-12-26 21:21:59,632][105620] Updated weights for policy 1, policy_version 824664 (0.0008) [2023-12-26 21:21:59,684][105620] Updated weights for policy 1, policy_version 824674 (0.0009) [2023-12-26 21:22:00,019][105692] Updated weights for policy 0, policy_version 824905 (0.0010) [2023-12-26 21:22:00,082][105692] Updated weights for policy 0, policy_version 824915 (0.0007) [2023-12-26 21:22:00,133][105692] Updated weights for policy 0, policy_version 824925 (0.0009) [2023-12-26 21:22:00,186][105692] Updated weights for policy 0, policy_version 824935 (0.0009) [2023-12-26 21:22:00,386][105620] Updated weights for policy 1, policy_version 824684 (0.0009) [2023-12-26 21:22:00,445][105620] Updated weights for policy 1, policy_version 824694 (0.0008) [2023-12-26 21:22:00,509][105620] Updated weights for policy 1, policy_version 824704 (0.0008) [2023-12-26 21:22:00,982][105692] Updated weights for policy 0, policy_version 824945 (0.0007) [2023-12-26 21:22:01,033][105692] Updated weights for policy 0, policy_version 824955 (0.0006) [2023-12-26 21:22:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 422363136. Throughput: 0: 9887.9, 1: 9520.8. Samples: 422339840. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 21:22:01,063][104569] Avg episode reward: [(0, '7781.524'), (1, '9060.188')] [2023-12-26 21:22:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000824712_211148800.pth... [2023-12-26 21:22:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000823624_210870272.pth [2023-12-26 21:22:01,096][105692] Updated weights for policy 0, policy_version 824965 (0.0007) [2023-12-26 21:22:01,105][105620] Updated weights for policy 1, policy_version 824714 (0.0009) [2023-12-26 21:22:01,114][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000824968_211222528.pth... [2023-12-26 21:22:01,119][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000823816_210927616.pth [2023-12-26 21:22:01,171][105620] Updated weights for policy 1, policy_version 824724 (0.0009) [2023-12-26 21:22:01,231][105620] Updated weights for policy 1, policy_version 824734 (0.0007) [2023-12-26 21:22:01,289][105620] Updated weights for policy 1, policy_version 824744 (0.0006) [2023-12-26 21:22:01,902][105692] Updated weights for policy 0, policy_version 824975 (0.0007) [2023-12-26 21:22:01,913][105620] Updated weights for policy 1, policy_version 824754 (0.0008) [2023-12-26 21:22:01,963][105692] Updated weights for policy 0, policy_version 824985 (0.0006) [2023-12-26 21:22:01,977][105620] Updated weights for policy 1, policy_version 824764 (0.0007) [2023-12-26 21:22:02,022][105692] Updated weights for policy 0, policy_version 824995 (0.0009) [2023-12-26 21:22:02,037][105620] Updated weights for policy 1, policy_version 824774 (0.0005) [2023-12-26 21:22:02,692][105620] Updated weights for policy 1, policy_version 824784 (0.0008) [2023-12-26 21:22:02,745][105620] Updated weights for policy 1, policy_version 824794 (0.0009) [2023-12-26 21:22:02,791][105692] Updated weights for policy 0, policy_version 825005 (0.0007) [2023-12-26 21:22:02,801][105620] Updated weights for policy 1, policy_version 824804 (0.0008) [2023-12-26 21:22:02,847][105692] Updated weights for policy 0, policy_version 825015 (0.0005) [2023-12-26 21:22:02,910][105692] Updated weights for policy 0, policy_version 825025 (0.0005) [2023-12-26 21:22:03,416][105692] Updated weights for policy 0, policy_version 825035 (0.0005) [2023-12-26 21:22:03,468][105692] Updated weights for policy 0, policy_version 825045 (0.0006) [2023-12-26 21:22:03,514][105692] Updated weights for policy 0, policy_version 825055 (0.0009) [2023-12-26 21:22:03,538][105620] Updated weights for policy 1, policy_version 824814 (0.0008) [2023-12-26 21:22:03,587][105620] Updated weights for policy 1, policy_version 824824 (0.0009) [2023-12-26 21:22:03,648][105620] Updated weights for policy 1, policy_version 824834 (0.0009) [2023-12-26 21:22:04,144][105692] Updated weights for policy 0, policy_version 825065 (0.0009) [2023-12-26 21:22:04,208][105692] Updated weights for policy 0, policy_version 825075 (0.0009) [2023-12-26 21:22:04,269][105692] Updated weights for policy 0, policy_version 825085 (0.0009) [2023-12-26 21:22:04,327][105692] Updated weights for policy 0, policy_version 825095 (0.0009) [2023-12-26 21:22:04,460][105620] Updated weights for policy 1, policy_version 824844 (0.0009) [2023-12-26 21:22:04,527][105620] Updated weights for policy 1, policy_version 824854 (0.0008) [2023-12-26 21:22:04,592][105620] Updated weights for policy 1, policy_version 824864 (0.0007) [2023-12-26 21:22:04,984][105692] Updated weights for policy 0, policy_version 825105 (0.0006) [2023-12-26 21:22:05,037][105692] Updated weights for policy 0, policy_version 825115 (0.0005) [2023-12-26 21:22:05,093][105692] Updated weights for policy 0, policy_version 825125 (0.0005) [2023-12-26 21:22:05,228][105620] Updated weights for policy 1, policy_version 824874 (0.0006) [2023-12-26 21:22:05,292][105620] Updated weights for policy 1, policy_version 824884 (0.0010) [2023-12-26 21:22:05,350][105620] Updated weights for policy 1, policy_version 824894 (0.0010) [2023-12-26 21:22:05,399][105620] Updated weights for policy 1, policy_version 824904 (0.0010) [2023-12-26 21:22:05,735][105692] Updated weights for policy 0, policy_version 825135 (0.0009) [2023-12-26 21:22:05,793][105692] Updated weights for policy 0, policy_version 825145 (0.0010) [2023-12-26 21:22:05,851][105692] Updated weights for policy 0, policy_version 825155 (0.0009) [2023-12-26 21:22:05,956][105620] Updated weights for policy 1, policy_version 824914 (0.0009) [2023-12-26 21:22:06,008][105620] Updated weights for policy 1, policy_version 824924 (0.0010) [2023-12-26 21:22:06,060][105620] Updated weights for policy 1, policy_version 824934 (0.0010) [2023-12-26 21:22:06,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 422469632. Throughput: 0: 9858.3, 1: 9526.6. Samples: 422457808. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 21:22:06,063][104569] Avg episode reward: [(0, '7480.757'), (1, '9257.483')] [2023-12-26 21:22:06,645][105692] Updated weights for policy 0, policy_version 825165 (0.0008) [2023-12-26 21:22:06,710][105692] Updated weights for policy 0, policy_version 825175 (0.0008) [2023-12-26 21:22:06,774][105692] Updated weights for policy 0, policy_version 825185 (0.0008) [2023-12-26 21:22:06,818][105620] Updated weights for policy 1, policy_version 824944 (0.0010) [2023-12-26 21:22:06,885][105620] Updated weights for policy 1, policy_version 824954 (0.0011) [2023-12-26 21:22:06,937][105620] Updated weights for policy 1, policy_version 824964 (0.0010) [2023-12-26 21:22:07,564][105692] Updated weights for policy 0, policy_version 825195 (0.0007) [2023-12-26 21:22:07,618][105620] Updated weights for policy 1, policy_version 824974 (0.0009) [2023-12-26 21:22:07,627][105692] Updated weights for policy 0, policy_version 825205 (0.0006) [2023-12-26 21:22:07,674][105620] Updated weights for policy 1, policy_version 824984 (0.0007) [2023-12-26 21:22:07,680][105692] Updated weights for policy 0, policy_version 825215 (0.0007) [2023-12-26 21:22:07,726][105620] Updated weights for policy 1, policy_version 824994 (0.0006) [2023-12-26 21:22:08,311][105620] Updated weights for policy 1, policy_version 825004 (0.0006) [2023-12-26 21:22:08,370][105620] Updated weights for policy 1, policy_version 825014 (0.0010) [2023-12-26 21:22:08,432][105620] Updated weights for policy 1, policy_version 825024 (0.0009) [2023-12-26 21:22:08,511][105692] Updated weights for policy 0, policy_version 825225 (0.0008) [2023-12-26 21:22:08,572][105692] Updated weights for policy 0, policy_version 825235 (0.0009) [2023-12-26 21:22:08,631][105692] Updated weights for policy 0, policy_version 825245 (0.0010) [2023-12-26 21:22:08,690][105692] Updated weights for policy 0, policy_version 825256 (0.0011) [2023-12-26 21:22:09,094][105620] Updated weights for policy 1, policy_version 825034 (0.0008) [2023-12-26 21:22:09,151][105620] Updated weights for policy 1, policy_version 825044 (0.0005) [2023-12-26 21:22:09,209][105620] Updated weights for policy 1, policy_version 825054 (0.0006) [2023-12-26 21:22:09,272][105620] Updated weights for policy 1, policy_version 825064 (0.0009) [2023-12-26 21:22:09,513][105692] Updated weights for policy 0, policy_version 825266 (0.0009) [2023-12-26 21:22:09,575][105692] Updated weights for policy 0, policy_version 825276 (0.0009) [2023-12-26 21:22:09,636][105692] Updated weights for policy 0, policy_version 825286 (0.0008) [2023-12-26 21:22:09,942][105620] Updated weights for policy 1, policy_version 825074 (0.0007) [2023-12-26 21:22:09,997][105620] Updated weights for policy 1, policy_version 825084 (0.0006) [2023-12-26 21:22:10,051][105620] Updated weights for policy 1, policy_version 825094 (0.0006) [2023-12-26 21:22:10,431][105692] Updated weights for policy 0, policy_version 825296 (0.0010) [2023-12-26 21:22:10,487][105692] Updated weights for policy 0, policy_version 825306 (0.0010) [2023-12-26 21:22:10,545][105692] Updated weights for policy 0, policy_version 825316 (0.0010) [2023-12-26 21:22:10,769][105620] Updated weights for policy 1, policy_version 825104 (0.0008) [2023-12-26 21:22:10,814][105620] Updated weights for policy 1, policy_version 825114 (0.0008) [2023-12-26 21:22:10,872][105620] Updated weights for policy 1, policy_version 825124 (0.0008) [2023-12-26 21:22:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 422567936. Throughput: 0: 9765.0, 1: 9644.0. Samples: 422574592. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 21:22:11,062][104569] Avg episode reward: [(0, '7268.973'), (1, '9166.242')] [2023-12-26 21:22:11,315][105692] Updated weights for policy 0, policy_version 825326 (0.0011) [2023-12-26 21:22:11,382][105692] Updated weights for policy 0, policy_version 825336 (0.0009) [2023-12-26 21:22:11,450][105692] Updated weights for policy 0, policy_version 825346 (0.0010) [2023-12-26 21:22:11,632][105620] Updated weights for policy 1, policy_version 825134 (0.0007) [2023-12-26 21:22:11,693][105620] Updated weights for policy 1, policy_version 825144 (0.0008) [2023-12-26 21:22:11,759][105620] Updated weights for policy 1, policy_version 825154 (0.0009) [2023-12-26 21:22:12,220][105692] Updated weights for policy 0, policy_version 825356 (0.0011) [2023-12-26 21:22:12,283][105692] Updated weights for policy 0, policy_version 825366 (0.0010) [2023-12-26 21:22:12,352][105692] Updated weights for policy 0, policy_version 825376 (0.0009) [2023-12-26 21:22:12,540][105620] Updated weights for policy 1, policy_version 825164 (0.0009) [2023-12-26 21:22:12,596][105620] Updated weights for policy 1, policy_version 825174 (0.0008) [2023-12-26 21:22:12,653][105620] Updated weights for policy 1, policy_version 825184 (0.0008) [2023-12-26 21:22:13,114][105692] Updated weights for policy 0, policy_version 825386 (0.0010) [2023-12-26 21:22:13,170][105692] Updated weights for policy 0, policy_version 825396 (0.0005) [2023-12-26 21:22:13,226][105692] Updated weights for policy 0, policy_version 825406 (0.0010) [2023-12-26 21:22:13,287][105692] Updated weights for policy 0, policy_version 825416 (0.0009) [2023-12-26 21:22:13,421][105620] Updated weights for policy 1, policy_version 825194 (0.0006) [2023-12-26 21:22:13,470][105620] Updated weights for policy 1, policy_version 825204 (0.0005) [2023-12-26 21:22:13,523][105620] Updated weights for policy 1, policy_version 825214 (0.0005) [2023-12-26 21:22:13,567][105620] Updated weights for policy 1, policy_version 825224 (0.0005) [2023-12-26 21:22:13,891][105692] Updated weights for policy 0, policy_version 825426 (0.0006) [2023-12-26 21:22:13,954][105692] Updated weights for policy 0, policy_version 825436 (0.0007) [2023-12-26 21:22:14,009][105692] Updated weights for policy 0, policy_version 825446 (0.0008) [2023-12-26 21:22:14,275][105620] Updated weights for policy 1, policy_version 825234 (0.0009) [2023-12-26 21:22:14,327][105620] Updated weights for policy 1, policy_version 825244 (0.0010) [2023-12-26 21:22:14,375][105620] Updated weights for policy 1, policy_version 825254 (0.0010) [2023-12-26 21:22:14,595][105692] Updated weights for policy 0, policy_version 825456 (0.0006) [2023-12-26 21:22:14,649][105692] Updated weights for policy 0, policy_version 825466 (0.0009) [2023-12-26 21:22:14,693][105692] Updated weights for policy 0, policy_version 825476 (0.0010) [2023-12-26 21:22:15,067][105620] Updated weights for policy 1, policy_version 825264 (0.0009) [2023-12-26 21:22:15,129][105620] Updated weights for policy 1, policy_version 825274 (0.0009) [2023-12-26 21:22:15,189][105620] Updated weights for policy 1, policy_version 825284 (0.0009) [2023-12-26 21:22:15,369][105692] Updated weights for policy 0, policy_version 825486 (0.0010) [2023-12-26 21:22:15,428][105692] Updated weights for policy 0, policy_version 825496 (0.0011) [2023-12-26 21:22:15,484][105692] Updated weights for policy 0, policy_version 825506 (0.0010) [2023-12-26 21:22:16,042][105620] Updated weights for policy 1, policy_version 825294 (0.0009) [2023-12-26 21:22:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 422658048. Throughput: 0: 9676.1, 1: 9633.7. Samples: 422630944. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 21:22:16,062][104569] Avg episode reward: [(0, '7477.030'), (1, '9257.153')] [2023-12-26 21:22:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000825512_211361792.pth... [2023-12-26 21:22:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000824392_211075072.pth [2023-12-26 21:22:16,095][105620] Updated weights for policy 1, policy_version 825304 (0.0008) [2023-12-26 21:22:16,110][105692] Updated weights for policy 0, policy_version 825516 (0.0008) [2023-12-26 21:22:16,141][105620] Updated weights for policy 1, policy_version 825314 (0.0006) [2023-12-26 21:22:16,158][105692] Updated weights for policy 0, policy_version 825526 (0.0010) [2023-12-26 21:22:16,170][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000825320_211304448.pth... [2023-12-26 21:22:16,173][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000824168_211009536.pth [2023-12-26 21:22:16,207][105692] Updated weights for policy 0, policy_version 825536 (0.0010) [2023-12-26 21:22:16,776][105620] Updated weights for policy 1, policy_version 825324 (0.0008) [2023-12-26 21:22:16,837][105620] Updated weights for policy 1, policy_version 825334 (0.0011) [2023-12-26 21:22:16,896][105620] Updated weights for policy 1, policy_version 825344 (0.0011) [2023-12-26 21:22:16,968][105692] Updated weights for policy 0, policy_version 825546 (0.0010) [2023-12-26 21:22:17,024][105692] Updated weights for policy 0, policy_version 825556 (0.0010) [2023-12-26 21:22:17,082][105692] Updated weights for policy 0, policy_version 825566 (0.0011) [2023-12-26 21:22:17,131][105692] Updated weights for policy 0, policy_version 825576 (0.0010) [2023-12-26 21:22:17,566][105620] Updated weights for policy 1, policy_version 825354 (0.0010) [2023-12-26 21:22:17,628][105620] Updated weights for policy 1, policy_version 825364 (0.0011) [2023-12-26 21:22:17,697][105620] Updated weights for policy 1, policy_version 825374 (0.0010) [2023-12-26 21:22:17,763][105620] Updated weights for policy 1, policy_version 825384 (0.0008) [2023-12-26 21:22:17,810][105692] Updated weights for policy 0, policy_version 825586 (0.0005) [2023-12-26 21:22:17,863][105692] Updated weights for policy 0, policy_version 825596 (0.0005) [2023-12-26 21:22:17,915][105692] Updated weights for policy 0, policy_version 825606 (0.0007) [2023-12-26 21:22:18,334][105620] Updated weights for policy 1, policy_version 825394 (0.0006) [2023-12-26 21:22:18,389][105620] Updated weights for policy 1, policy_version 825404 (0.0007) [2023-12-26 21:22:18,447][105620] Updated weights for policy 1, policy_version 825414 (0.0006) [2023-12-26 21:22:18,518][105692] Updated weights for policy 0, policy_version 825616 (0.0010) [2023-12-26 21:22:18,581][105692] Updated weights for policy 0, policy_version 825626 (0.0009) [2023-12-26 21:22:18,634][105692] Updated weights for policy 0, policy_version 825636 (0.0010) [2023-12-26 21:22:19,033][105620] Updated weights for policy 1, policy_version 825424 (0.0009) [2023-12-26 21:22:19,078][105620] Updated weights for policy 1, policy_version 825434 (0.0010) [2023-12-26 21:22:19,132][105620] Updated weights for policy 1, policy_version 825444 (0.0010) [2023-12-26 21:22:19,371][105692] Updated weights for policy 0, policy_version 825646 (0.0009) [2023-12-26 21:22:19,429][105692] Updated weights for policy 0, policy_version 825656 (0.0007) [2023-12-26 21:22:19,494][105692] Updated weights for policy 0, policy_version 825666 (0.0008) [2023-12-26 21:22:19,929][105620] Updated weights for policy 1, policy_version 825454 (0.0009) [2023-12-26 21:22:19,990][105620] Updated weights for policy 1, policy_version 825464 (0.0009) [2023-12-26 21:22:20,057][105620] Updated weights for policy 1, policy_version 825474 (0.0006) [2023-12-26 21:22:20,248][105692] Updated weights for policy 0, policy_version 825676 (0.0009) [2023-12-26 21:22:20,313][105692] Updated weights for policy 0, policy_version 825686 (0.0007) [2023-12-26 21:22:20,374][105692] Updated weights for policy 0, policy_version 825696 (0.0010) [2023-12-26 21:22:20,759][105620] Updated weights for policy 1, policy_version 825484 (0.0007) [2023-12-26 21:22:20,832][105620] Updated weights for policy 1, policy_version 825494 (0.0008) [2023-12-26 21:22:20,897][105620] Updated weights for policy 1, policy_version 825504 (0.0008) [2023-12-26 21:22:21,064][104569] Fps is (10 sec: 19656.2, 60 sec: 19523.7, 300 sec: 19577.3). Total num frames: 422764544. Throughput: 0: 9777.2, 1: 9666.5. Samples: 422754068. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 21:22:21,065][104569] Avg episode reward: [(0, '7377.396'), (1, '9213.646')] [2023-12-26 21:22:21,071][105692] Updated weights for policy 0, policy_version 825706 (0.0011) [2023-12-26 21:22:21,145][105692] Updated weights for policy 0, policy_version 825716 (0.0011) [2023-12-26 21:22:21,208][105692] Updated weights for policy 0, policy_version 825726 (0.0011) [2023-12-26 21:22:21,274][105692] Updated weights for policy 0, policy_version 825736 (0.0011) [2023-12-26 21:22:21,655][105620] Updated weights for policy 1, policy_version 825514 (0.0008) [2023-12-26 21:22:21,724][105620] Updated weights for policy 1, policy_version 825524 (0.0008) [2023-12-26 21:22:21,790][105620] Updated weights for policy 1, policy_version 825534 (0.0009) [2023-12-26 21:22:21,850][105620] Updated weights for policy 1, policy_version 825544 (0.0008) [2023-12-26 21:22:22,066][105692] Updated weights for policy 0, policy_version 825746 (0.0011) [2023-12-26 21:22:22,124][105692] Updated weights for policy 0, policy_version 825756 (0.0011) [2023-12-26 21:22:22,188][105692] Updated weights for policy 0, policy_version 825766 (0.0011) [2023-12-26 21:22:22,565][105620] Updated weights for policy 1, policy_version 825554 (0.0006) [2023-12-26 21:22:22,626][105620] Updated weights for policy 1, policy_version 825564 (0.0005) [2023-12-26 21:22:22,699][105620] Updated weights for policy 1, policy_version 825574 (0.0005) [2023-12-26 21:22:22,886][105692] Updated weights for policy 0, policy_version 825776 (0.0009) [2023-12-26 21:22:22,948][105692] Updated weights for policy 0, policy_version 825786 (0.0008) [2023-12-26 21:22:23,011][105692] Updated weights for policy 0, policy_version 825796 (0.0009) [2023-12-26 21:22:23,419][105620] Updated weights for policy 1, policy_version 825584 (0.0009) [2023-12-26 21:22:23,482][105620] Updated weights for policy 1, policy_version 825594 (0.0008) [2023-12-26 21:22:23,544][105620] Updated weights for policy 1, policy_version 825604 (0.0008) [2023-12-26 21:22:23,708][105692] Updated weights for policy 0, policy_version 825806 (0.0010) [2023-12-26 21:22:23,763][105692] Updated weights for policy 0, policy_version 825816 (0.0009) [2023-12-26 21:22:23,819][105692] Updated weights for policy 0, policy_version 825826 (0.0009) [2023-12-26 21:22:24,234][105620] Updated weights for policy 1, policy_version 825614 (0.0006) [2023-12-26 21:22:24,285][105620] Updated weights for policy 1, policy_version 825624 (0.0006) [2023-12-26 21:22:24,338][105620] Updated weights for policy 1, policy_version 825634 (0.0008) [2023-12-26 21:22:24,526][105692] Updated weights for policy 0, policy_version 825836 (0.0009) [2023-12-26 21:22:24,588][105692] Updated weights for policy 0, policy_version 825846 (0.0009) [2023-12-26 21:22:24,647][105692] Updated weights for policy 0, policy_version 825856 (0.0009) [2023-12-26 21:22:25,037][105620] Updated weights for policy 1, policy_version 825644 (0.0008) [2023-12-26 21:22:25,095][105620] Updated weights for policy 1, policy_version 825654 (0.0005) [2023-12-26 21:22:25,153][105620] Updated weights for policy 1, policy_version 825664 (0.0005) [2023-12-26 21:22:25,456][105692] Updated weights for policy 0, policy_version 825866 (0.0008) [2023-12-26 21:22:25,510][105692] Updated weights for policy 0, policy_version 825876 (0.0005) [2023-12-26 21:22:25,556][105692] Updated weights for policy 0, policy_version 825886 (0.0005) [2023-12-26 21:22:25,603][105692] Updated weights for policy 0, policy_version 825896 (0.0006) [2023-12-26 21:22:25,708][105620] Updated weights for policy 1, policy_version 825674 (0.0005) [2023-12-26 21:22:25,758][105620] Updated weights for policy 1, policy_version 825684 (0.0005) [2023-12-26 21:22:25,811][105620] Updated weights for policy 1, policy_version 825694 (0.0005) [2023-12-26 21:22:25,864][105620] Updated weights for policy 1, policy_version 825704 (0.0005) [2023-12-26 21:22:26,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 422862848. Throughput: 0: 9700.6, 1: 9754.3. Samples: 422870732. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 21:22:26,062][104569] Avg episode reward: [(0, '7009.826'), (1, '9212.883')] [2023-12-26 21:22:26,373][105692] Updated weights for policy 0, policy_version 825906 (0.0008) [2023-12-26 21:22:26,416][105620] Updated weights for policy 1, policy_version 825714 (0.0008) [2023-12-26 21:22:26,434][105692] Updated weights for policy 0, policy_version 825916 (0.0007) [2023-12-26 21:22:26,465][105620] Updated weights for policy 1, policy_version 825724 (0.0007) [2023-12-26 21:22:26,484][105692] Updated weights for policy 0, policy_version 825926 (0.0007) [2023-12-26 21:22:26,512][105620] Updated weights for policy 1, policy_version 825734 (0.0007) [2023-12-26 21:22:27,177][105692] Updated weights for policy 0, policy_version 825936 (0.0008) [2023-12-26 21:22:27,230][105692] Updated weights for policy 0, policy_version 825946 (0.0009) [2023-12-26 21:22:27,261][105620] Updated weights for policy 1, policy_version 825744 (0.0008) [2023-12-26 21:22:27,276][105692] Updated weights for policy 0, policy_version 825956 (0.0007) [2023-12-26 21:22:27,315][105620] Updated weights for policy 1, policy_version 825754 (0.0007) [2023-12-26 21:22:27,377][105620] Updated weights for policy 1, policy_version 825764 (0.0006) [2023-12-26 21:22:28,018][105620] Updated weights for policy 1, policy_version 825774 (0.0008) [2023-12-26 21:22:28,039][105692] Updated weights for policy 0, policy_version 825966 (0.0009) [2023-12-26 21:22:28,068][105620] Updated weights for policy 1, policy_version 825784 (0.0007) [2023-12-26 21:22:28,102][105692] Updated weights for policy 0, policy_version 825976 (0.0009) [2023-12-26 21:22:28,127][105620] Updated weights for policy 1, policy_version 825794 (0.0006) [2023-12-26 21:22:28,164][105692] Updated weights for policy 0, policy_version 825986 (0.0009) [2023-12-26 21:22:28,721][105620] Updated weights for policy 1, policy_version 825804 (0.0007) [2023-12-26 21:22:28,769][105620] Updated weights for policy 1, policy_version 825814 (0.0007) [2023-12-26 21:22:28,817][105620] Updated weights for policy 1, policy_version 825824 (0.0009) [2023-12-26 21:22:28,947][105692] Updated weights for policy 0, policy_version 825996 (0.0009) [2023-12-26 21:22:29,006][105692] Updated weights for policy 0, policy_version 826006 (0.0009) [2023-12-26 21:22:29,056][105692] Updated weights for policy 0, policy_version 826016 (0.0009) [2023-12-26 21:22:29,554][105620] Updated weights for policy 1, policy_version 825834 (0.0009) [2023-12-26 21:22:29,620][105620] Updated weights for policy 1, policy_version 825844 (0.0010) [2023-12-26 21:22:29,678][105620] Updated weights for policy 1, policy_version 825854 (0.0010) [2023-12-26 21:22:29,740][105620] Updated weights for policy 1, policy_version 825864 (0.0010) [2023-12-26 21:22:29,790][105692] Updated weights for policy 0, policy_version 826026 (0.0008) [2023-12-26 21:22:29,854][105692] Updated weights for policy 0, policy_version 826036 (0.0007) [2023-12-26 21:22:29,913][105692] Updated weights for policy 0, policy_version 826046 (0.0008) [2023-12-26 21:22:29,965][105692] Updated weights for policy 0, policy_version 826056 (0.0007) [2023-12-26 21:22:30,464][105620] Updated weights for policy 1, policy_version 825874 (0.0010) [2023-12-26 21:22:30,513][105620] Updated weights for policy 1, policy_version 825884 (0.0009) [2023-12-26 21:22:30,578][105620] Updated weights for policy 1, policy_version 825894 (0.0005) [2023-12-26 21:22:30,684][105692] Updated weights for policy 0, policy_version 826066 (0.0010) [2023-12-26 21:22:30,734][105692] Updated weights for policy 0, policy_version 826076 (0.0010) [2023-12-26 21:22:30,788][105692] Updated weights for policy 0, policy_version 826086 (0.0010) [2023-12-26 21:22:31,062][104569] Fps is (10 sec: 19665.3, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 422961152. Throughput: 0: 9679.8, 1: 9870.6. Samples: 422930192. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 21:22:31,062][104569] Avg episode reward: [(0, '6231.669'), (1, '9076.995')] [2023-12-26 21:22:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000826088_211509248.pth... [2023-12-26 21:22:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000825896_211451904.pth... [2023-12-26 21:22:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000824968_211222528.pth [2023-12-26 21:22:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000824712_211148800.pth [2023-12-26 21:22:31,273][105620] Updated weights for policy 1, policy_version 825904 (0.0010) [2023-12-26 21:22:31,321][105620] Updated weights for policy 1, policy_version 825914 (0.0009) [2023-12-26 21:22:31,382][105620] Updated weights for policy 1, policy_version 825924 (0.0010) [2023-12-26 21:22:31,495][105692] Updated weights for policy 0, policy_version 826096 (0.0009) [2023-12-26 21:22:31,552][105692] Updated weights for policy 0, policy_version 826106 (0.0009) [2023-12-26 21:22:31,607][105692] Updated weights for policy 0, policy_version 826116 (0.0009) [2023-12-26 21:22:32,193][105620] Updated weights for policy 1, policy_version 825934 (0.0009) [2023-12-26 21:22:32,240][105620] Updated weights for policy 1, policy_version 825944 (0.0008) [2023-12-26 21:22:32,300][105620] Updated weights for policy 1, policy_version 825954 (0.0008) [2023-12-26 21:22:32,386][105692] Updated weights for policy 0, policy_version 826126 (0.0011) [2023-12-26 21:22:32,445][105692] Updated weights for policy 0, policy_version 826136 (0.0010) [2023-12-26 21:22:32,503][105692] Updated weights for policy 0, policy_version 826146 (0.0010) [2023-12-26 21:22:33,043][105620] Updated weights for policy 1, policy_version 825964 (0.0007) [2023-12-26 21:22:33,108][105620] Updated weights for policy 1, policy_version 825974 (0.0005) [2023-12-26 21:22:33,156][105692] Updated weights for policy 0, policy_version 826156 (0.0008) [2023-12-26 21:22:33,167][105620] Updated weights for policy 1, policy_version 825984 (0.0005) [2023-12-26 21:22:33,201][105692] Updated weights for policy 0, policy_version 826166 (0.0005) [2023-12-26 21:22:33,249][105692] Updated weights for policy 0, policy_version 826176 (0.0005) [2023-12-26 21:22:33,765][105620] Updated weights for policy 1, policy_version 825994 (0.0005) [2023-12-26 21:22:33,819][105620] Updated weights for policy 1, policy_version 826004 (0.0005) [2023-12-26 21:22:33,871][105620] Updated weights for policy 1, policy_version 826014 (0.0006) [2023-12-26 21:22:33,906][105692] Updated weights for policy 0, policy_version 826186 (0.0006) [2023-12-26 21:22:33,934][105620] Updated weights for policy 1, policy_version 826024 (0.0007) [2023-12-26 21:22:33,965][105692] Updated weights for policy 0, policy_version 826196 (0.0006) [2023-12-26 21:22:34,031][105692] Updated weights for policy 0, policy_version 826206 (0.0010) [2023-12-26 21:22:34,086][105692] Updated weights for policy 0, policy_version 826216 (0.0010) [2023-12-26 21:22:34,521][105620] Updated weights for policy 1, policy_version 826034 (0.0007) [2023-12-26 21:22:34,592][105620] Updated weights for policy 1, policy_version 826044 (0.0007) [2023-12-26 21:22:34,662][105620] Updated weights for policy 1, policy_version 826054 (0.0008) [2023-12-26 21:22:34,820][105692] Updated weights for policy 0, policy_version 826226 (0.0010) [2023-12-26 21:22:34,881][105692] Updated weights for policy 0, policy_version 826236 (0.0010) [2023-12-26 21:22:34,952][105692] Updated weights for policy 0, policy_version 826246 (0.0009) [2023-12-26 21:22:35,267][105620] Updated weights for policy 1, policy_version 826064 (0.0006) [2023-12-26 21:22:35,313][105620] Updated weights for policy 1, policy_version 826074 (0.0005) [2023-12-26 21:22:35,364][105620] Updated weights for policy 1, policy_version 826084 (0.0005) [2023-12-26 21:22:35,523][105692] Updated weights for policy 0, policy_version 826256 (0.0006) [2023-12-26 21:22:35,578][105692] Updated weights for policy 0, policy_version 826266 (0.0010) [2023-12-26 21:22:35,622][105692] Updated weights for policy 0, policy_version 826276 (0.0010) [2023-12-26 21:22:36,042][105620] Updated weights for policy 1, policy_version 826094 (0.0008) [2023-12-26 21:22:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 423059456. Throughput: 0: 9613.1, 1: 10027.2. Samples: 423049732. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 21:22:36,062][104569] Avg episode reward: [(0, '7328.098'), (1, '9190.178')] [2023-12-26 21:22:36,099][105620] Updated weights for policy 1, policy_version 826104 (0.0010) [2023-12-26 21:22:36,167][105620] Updated weights for policy 1, policy_version 826114 (0.0009) [2023-12-26 21:22:36,236][105692] Updated weights for policy 0, policy_version 826286 (0.0009) [2023-12-26 21:22:36,298][105692] Updated weights for policy 0, policy_version 826296 (0.0008) [2023-12-26 21:22:36,360][105692] Updated weights for policy 0, policy_version 826306 (0.0008) [2023-12-26 21:22:36,997][105620] Updated weights for policy 1, policy_version 826124 (0.0008) [2023-12-26 21:22:36,999][105692] Updated weights for policy 0, policy_version 826316 (0.0007) [2023-12-26 21:22:37,045][105620] Updated weights for policy 1, policy_version 826134 (0.0005) [2023-12-26 21:22:37,051][105692] Updated weights for policy 0, policy_version 826326 (0.0008) [2023-12-26 21:22:37,092][105620] Updated weights for policy 1, policy_version 826144 (0.0008) [2023-12-26 21:22:37,095][105692] Updated weights for policy 0, policy_version 826336 (0.0009) [2023-12-26 21:22:37,682][105620] Updated weights for policy 1, policy_version 826154 (0.0009) [2023-12-26 21:22:37,735][105620] Updated weights for policy 1, policy_version 826164 (0.0008) [2023-12-26 21:22:37,790][105620] Updated weights for policy 1, policy_version 826174 (0.0010) [2023-12-26 21:22:37,835][105692] Updated weights for policy 0, policy_version 826346 (0.0007) [2023-12-26 21:22:37,849][105620] Updated weights for policy 1, policy_version 826184 (0.0011) [2023-12-26 21:22:37,886][105692] Updated weights for policy 0, policy_version 826356 (0.0007) [2023-12-26 21:22:37,935][105692] Updated weights for policy 0, policy_version 826366 (0.0008) [2023-12-26 21:22:37,983][105692] Updated weights for policy 0, policy_version 826376 (0.0008) [2023-12-26 21:22:38,492][105620] Updated weights for policy 1, policy_version 826194 (0.0010) [2023-12-26 21:22:38,552][105620] Updated weights for policy 1, policy_version 826204 (0.0010) [2023-12-26 21:22:38,614][105620] Updated weights for policy 1, policy_version 826214 (0.0011) [2023-12-26 21:22:38,762][105692] Updated weights for policy 0, policy_version 826386 (0.0008) [2023-12-26 21:22:38,827][105692] Updated weights for policy 0, policy_version 826396 (0.0009) [2023-12-26 21:22:38,889][105692] Updated weights for policy 0, policy_version 826406 (0.0009) [2023-12-26 21:22:39,263][105620] Updated weights for policy 1, policy_version 826224 (0.0009) [2023-12-26 21:22:39,330][105620] Updated weights for policy 1, policy_version 826234 (0.0009) [2023-12-26 21:22:39,395][105620] Updated weights for policy 1, policy_version 826244 (0.0007) [2023-12-26 21:22:39,725][105692] Updated weights for policy 0, policy_version 826416 (0.0009) [2023-12-26 21:22:39,785][105692] Updated weights for policy 0, policy_version 826426 (0.0009) [2023-12-26 21:22:39,844][105692] Updated weights for policy 0, policy_version 826436 (0.0009) [2023-12-26 21:22:40,097][105620] Updated weights for policy 1, policy_version 826254 (0.0010) [2023-12-26 21:22:40,150][105620] Updated weights for policy 1, policy_version 826264 (0.0011) [2023-12-26 21:22:40,209][105620] Updated weights for policy 1, policy_version 826274 (0.0011) [2023-12-26 21:22:40,601][105692] Updated weights for policy 0, policy_version 826446 (0.0007) [2023-12-26 21:22:40,654][105692] Updated weights for policy 0, policy_version 826456 (0.0008) [2023-12-26 21:22:40,706][105692] Updated weights for policy 0, policy_version 826466 (0.0008) [2023-12-26 21:22:40,928][105620] Updated weights for policy 1, policy_version 826284 (0.0008) [2023-12-26 21:22:40,990][105620] Updated weights for policy 1, policy_version 826294 (0.0005) [2023-12-26 21:22:41,047][105620] Updated weights for policy 1, policy_version 826304 (0.0009) [2023-12-26 21:22:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 423157760. Throughput: 0: 9648.4, 1: 10118.6. Samples: 423170328. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 21:22:41,063][104569] Avg episode reward: [(0, '7748.193'), (1, '9186.519')] [2023-12-26 21:22:41,515][105692] Updated weights for policy 0, policy_version 826476 (0.0008) [2023-12-26 21:22:41,571][105692] Updated weights for policy 0, policy_version 826486 (0.0008) [2023-12-26 21:22:41,641][105692] Updated weights for policy 0, policy_version 826496 (0.0008) [2023-12-26 21:22:41,771][105620] Updated weights for policy 1, policy_version 826314 (0.0008) [2023-12-26 21:22:41,823][105620] Updated weights for policy 1, policy_version 826324 (0.0011) [2023-12-26 21:22:41,887][105620] Updated weights for policy 1, policy_version 826334 (0.0011) [2023-12-26 21:22:41,939][105620] Updated weights for policy 1, policy_version 826344 (0.0011) [2023-12-26 21:22:42,411][105692] Updated weights for policy 0, policy_version 826506 (0.0009) [2023-12-26 21:22:42,460][105692] Updated weights for policy 0, policy_version 826516 (0.0008) [2023-12-26 21:22:42,519][105692] Updated weights for policy 0, policy_version 826526 (0.0008) [2023-12-26 21:22:42,584][105692] Updated weights for policy 0, policy_version 826536 (0.0008) [2023-12-26 21:22:42,719][105620] Updated weights for policy 1, policy_version 826354 (0.0011) [2023-12-26 21:22:42,779][105620] Updated weights for policy 1, policy_version 826364 (0.0011) [2023-12-26 21:22:42,842][105620] Updated weights for policy 1, policy_version 826374 (0.0011) [2023-12-26 21:22:43,365][105692] Updated weights for policy 0, policy_version 826546 (0.0008) [2023-12-26 21:22:43,409][105692] Updated weights for policy 0, policy_version 826556 (0.0008) [2023-12-26 21:22:43,458][105692] Updated weights for policy 0, policy_version 826566 (0.0008) [2023-12-26 21:22:43,609][105620] Updated weights for policy 1, policy_version 826384 (0.0011) [2023-12-26 21:22:43,664][105620] Updated weights for policy 1, policy_version 826394 (0.0010) [2023-12-26 21:22:43,716][105620] Updated weights for policy 1, policy_version 826404 (0.0010) [2023-12-26 21:22:44,292][105692] Updated weights for policy 0, policy_version 826576 (0.0007) [2023-12-26 21:22:44,356][105620] Updated weights for policy 1, policy_version 826414 (0.0008) [2023-12-26 21:22:44,362][105692] Updated weights for policy 0, policy_version 826586 (0.0005) [2023-12-26 21:22:44,415][105620] Updated weights for policy 1, policy_version 826424 (0.0011) [2023-12-26 21:22:44,417][105692] Updated weights for policy 0, policy_version 826596 (0.0007) [2023-12-26 21:22:44,467][105620] Updated weights for policy 1, policy_version 826434 (0.0010) [2023-12-26 21:22:45,069][105692] Updated weights for policy 0, policy_version 826606 (0.0007) [2023-12-26 21:22:45,126][105692] Updated weights for policy 0, policy_version 826616 (0.0008) [2023-12-26 21:22:45,185][105692] Updated weights for policy 0, policy_version 826626 (0.0008) [2023-12-26 21:22:45,218][105620] Updated weights for policy 1, policy_version 826444 (0.0011) [2023-12-26 21:22:45,285][105620] Updated weights for policy 1, policy_version 826454 (0.0009) [2023-12-26 21:22:45,354][105620] Updated weights for policy 1, policy_version 826464 (0.0006) [2023-12-26 21:22:45,934][105692] Updated weights for policy 0, policy_version 826636 (0.0009) [2023-12-26 21:22:45,980][105620] Updated weights for policy 1, policy_version 826474 (0.0007) [2023-12-26 21:22:45,982][105692] Updated weights for policy 0, policy_version 826646 (0.0010) [2023-12-26 21:22:46,030][105692] Updated weights for policy 0, policy_version 826656 (0.0010) [2023-12-26 21:22:46,032][105620] Updated weights for policy 1, policy_version 826484 (0.0006) [2023-12-26 21:22:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 423247872. Throughput: 0: 9615.8, 1: 10053.9. Samples: 423224976. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 21:22:46,062][104569] Avg episode reward: [(0, '7973.551'), (1, '9256.841')] [2023-12-26 21:22:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000826664_211656704.pth... [2023-12-26 21:22:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000825512_211361792.pth [2023-12-26 21:22:46,081][105620] Updated weights for policy 1, policy_version 826494 (0.0006) [2023-12-26 21:22:46,125][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000826504_211607552.pth... [2023-12-26 21:22:46,126][105620] Updated weights for policy 1, policy_version 826504 (0.0008) [2023-12-26 21:22:46,129][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000825320_211304448.pth [2023-12-26 21:22:46,711][105692] Updated weights for policy 0, policy_version 826666 (0.0010) [2023-12-26 21:22:46,763][105692] Updated weights for policy 0, policy_version 826676 (0.0010) [2023-12-26 21:22:46,784][105620] Updated weights for policy 1, policy_version 826514 (0.0005) [2023-12-26 21:22:46,814][105692] Updated weights for policy 0, policy_version 826686 (0.0010) [2023-12-26 21:22:46,832][105620] Updated weights for policy 1, policy_version 826524 (0.0005) [2023-12-26 21:22:46,862][105692] Updated weights for policy 0, policy_version 826696 (0.0010) [2023-12-26 21:22:46,881][105620] Updated weights for policy 1, policy_version 826534 (0.0006) [2023-12-26 21:22:47,580][105692] Updated weights for policy 0, policy_version 826706 (0.0006) [2023-12-26 21:22:47,590][105620] Updated weights for policy 1, policy_version 826544 (0.0008) [2023-12-26 21:22:47,636][105692] Updated weights for policy 0, policy_version 826716 (0.0010) [2023-12-26 21:22:47,639][105620] Updated weights for policy 1, policy_version 826554 (0.0005) [2023-12-26 21:22:47,687][105620] Updated weights for policy 1, policy_version 826564 (0.0009) [2023-12-26 21:22:47,701][105692] Updated weights for policy 0, policy_version 826726 (0.0007) [2023-12-26 21:22:48,365][105692] Updated weights for policy 0, policy_version 826736 (0.0010) [2023-12-26 21:22:48,402][105620] Updated weights for policy 1, policy_version 826574 (0.0006) [2023-12-26 21:22:48,428][105692] Updated weights for policy 0, policy_version 826746 (0.0008) [2023-12-26 21:22:48,467][105620] Updated weights for policy 1, policy_version 826584 (0.0006) [2023-12-26 21:22:48,486][105692] Updated weights for policy 0, policy_version 826756 (0.0008) [2023-12-26 21:22:48,530][105620] Updated weights for policy 1, policy_version 826594 (0.0007) [2023-12-26 21:22:49,176][105692] Updated weights for policy 0, policy_version 826766 (0.0008) [2023-12-26 21:22:49,243][105692] Updated weights for policy 0, policy_version 826776 (0.0007) [2023-12-26 21:22:49,302][105692] Updated weights for policy 0, policy_version 826786 (0.0010) [2023-12-26 21:22:49,329][105620] Updated weights for policy 1, policy_version 826604 (0.0008) [2023-12-26 21:22:49,395][105620] Updated weights for policy 1, policy_version 826614 (0.0008) [2023-12-26 21:22:49,454][105620] Updated weights for policy 1, policy_version 826624 (0.0008) [2023-12-26 21:22:50,009][105692] Updated weights for policy 0, policy_version 826796 (0.0008) [2023-12-26 21:22:50,077][105692] Updated weights for policy 0, policy_version 826806 (0.0008) [2023-12-26 21:22:50,136][105692] Updated weights for policy 0, policy_version 826816 (0.0009) [2023-12-26 21:22:50,239][105620] Updated weights for policy 1, policy_version 826634 (0.0008) [2023-12-26 21:22:50,303][105620] Updated weights for policy 1, policy_version 826644 (0.0009) [2023-12-26 21:22:50,356][105620] Updated weights for policy 1, policy_version 826654 (0.0010) [2023-12-26 21:22:50,406][105620] Updated weights for policy 1, policy_version 826664 (0.0009) [2023-12-26 21:22:50,858][105692] Updated weights for policy 0, policy_version 826826 (0.0009) [2023-12-26 21:22:50,919][105692] Updated weights for policy 0, policy_version 826836 (0.0011) [2023-12-26 21:22:50,986][105692] Updated weights for policy 0, policy_version 826846 (0.0011) [2023-12-26 21:22:51,049][105692] Updated weights for policy 0, policy_version 826856 (0.0011) [2023-12-26 21:22:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 423354368. Throughput: 0: 9648.4, 1: 10011.0. Samples: 423342480. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 21:22:51,062][104569] Avg episode reward: [(0, '6811.763'), (1, '9257.742')] [2023-12-26 21:22:51,177][105620] Updated weights for policy 1, policy_version 826674 (0.0007) [2023-12-26 21:22:51,236][105620] Updated weights for policy 1, policy_version 826684 (0.0006) [2023-12-26 21:22:51,301][105620] Updated weights for policy 1, policy_version 826694 (0.0008) [2023-12-26 21:22:51,790][105692] Updated weights for policy 0, policy_version 826866 (0.0010) [2023-12-26 21:22:51,862][105692] Updated weights for policy 0, policy_version 826876 (0.0007) [2023-12-26 21:22:51,922][105692] Updated weights for policy 0, policy_version 826886 (0.0006) [2023-12-26 21:22:52,018][105620] Updated weights for policy 1, policy_version 826704 (0.0006) [2023-12-26 21:22:52,072][105620] Updated weights for policy 1, policy_version 826714 (0.0007) [2023-12-26 21:22:52,121][105620] Updated weights for policy 1, policy_version 826724 (0.0006) [2023-12-26 21:22:52,618][105692] Updated weights for policy 0, policy_version 826896 (0.0010) [2023-12-26 21:22:52,686][105692] Updated weights for policy 0, policy_version 826906 (0.0010) [2023-12-26 21:22:52,737][105620] Updated weights for policy 1, policy_version 826734 (0.0006) [2023-12-26 21:22:52,742][105692] Updated weights for policy 0, policy_version 826916 (0.0011) [2023-12-26 21:22:52,787][105620] Updated weights for policy 1, policy_version 826744 (0.0007) [2023-12-26 21:22:52,836][105620] Updated weights for policy 1, policy_version 826754 (0.0008) [2023-12-26 21:22:53,443][105692] Updated weights for policy 0, policy_version 826926 (0.0007) [2023-12-26 21:22:53,505][105692] Updated weights for policy 0, policy_version 826936 (0.0005) [2023-12-26 21:22:53,571][105692] Updated weights for policy 0, policy_version 826946 (0.0006) [2023-12-26 21:22:53,658][105620] Updated weights for policy 1, policy_version 826764 (0.0008) [2023-12-26 21:22:53,713][105620] Updated weights for policy 1, policy_version 826774 (0.0009) [2023-12-26 21:22:53,779][105620] Updated weights for policy 1, policy_version 826784 (0.0009) [2023-12-26 21:22:54,174][105692] Updated weights for policy 0, policy_version 826956 (0.0007) [2023-12-26 21:22:54,242][105692] Updated weights for policy 0, policy_version 826966 (0.0006) [2023-12-26 21:22:54,301][105692] Updated weights for policy 0, policy_version 826976 (0.0009) [2023-12-26 21:22:54,621][105620] Updated weights for policy 1, policy_version 826794 (0.0008) [2023-12-26 21:22:54,682][105620] Updated weights for policy 1, policy_version 826804 (0.0009) [2023-12-26 21:22:54,740][105620] Updated weights for policy 1, policy_version 826814 (0.0009) [2023-12-26 21:22:54,791][105620] Updated weights for policy 1, policy_version 826824 (0.0009) [2023-12-26 21:22:54,908][105692] Updated weights for policy 0, policy_version 826986 (0.0008) [2023-12-26 21:22:54,979][105692] Updated weights for policy 0, policy_version 826996 (0.0007) [2023-12-26 21:22:55,033][105692] Updated weights for policy 0, policy_version 827006 (0.0010) [2023-12-26 21:22:55,089][105692] Updated weights for policy 0, policy_version 827016 (0.0009) [2023-12-26 21:22:55,533][105620] Updated weights for policy 1, policy_version 826834 (0.0009) [2023-12-26 21:22:55,587][105620] Updated weights for policy 1, policy_version 826844 (0.0008) [2023-12-26 21:22:55,637][105620] Updated weights for policy 1, policy_version 826854 (0.0009) [2023-12-26 21:22:55,804][105692] Updated weights for policy 0, policy_version 827026 (0.0009) [2023-12-26 21:22:55,851][105692] Updated weights for policy 0, policy_version 827036 (0.0008) [2023-12-26 21:22:55,898][105692] Updated weights for policy 0, policy_version 827046 (0.0009) [2023-12-26 21:22:56,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 423452672. Throughput: 0: 9763.6, 1: 9885.6. Samples: 423458804. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 21:22:56,062][104569] Avg episode reward: [(0, '7408.389'), (1, '8890.898')] [2023-12-26 21:22:56,455][105620] Updated weights for policy 1, policy_version 826864 (0.0009) [2023-12-26 21:22:56,504][105620] Updated weights for policy 1, policy_version 826874 (0.0008) [2023-12-26 21:22:56,560][105620] Updated weights for policy 1, policy_version 826884 (0.0008) [2023-12-26 21:22:56,563][105692] Updated weights for policy 0, policy_version 827056 (0.0007) [2023-12-26 21:22:56,607][105692] Updated weights for policy 0, policy_version 827066 (0.0008) [2023-12-26 21:22:56,654][105692] Updated weights for policy 0, policy_version 827076 (0.0009) [2023-12-26 21:22:57,330][105620] Updated weights for policy 1, policy_version 826894 (0.0008) [2023-12-26 21:22:57,381][105692] Updated weights for policy 0, policy_version 827086 (0.0007) [2023-12-26 21:22:57,394][105620] Updated weights for policy 1, policy_version 826904 (0.0008) [2023-12-26 21:22:57,442][105692] Updated weights for policy 0, policy_version 827096 (0.0006) [2023-12-26 21:22:57,458][105620] Updated weights for policy 1, policy_version 826914 (0.0009) [2023-12-26 21:22:57,507][105692] Updated weights for policy 0, policy_version 827106 (0.0005) [2023-12-26 21:22:58,042][105692] Updated weights for policy 0, policy_version 827116 (0.0005) [2023-12-26 21:22:58,094][105692] Updated weights for policy 0, policy_version 827126 (0.0006) [2023-12-26 21:22:58,156][105692] Updated weights for policy 0, policy_version 827136 (0.0009) [2023-12-26 21:22:58,289][105620] Updated weights for policy 1, policy_version 826924 (0.0010) [2023-12-26 21:22:58,363][105620] Updated weights for policy 1, policy_version 826934 (0.0009) [2023-12-26 21:22:58,430][105620] Updated weights for policy 1, policy_version 826944 (0.0008) [2023-12-26 21:22:59,007][105692] Updated weights for policy 0, policy_version 827146 (0.0010) [2023-12-26 21:22:59,070][105692] Updated weights for policy 0, policy_version 827156 (0.0010) [2023-12-26 21:22:59,134][105692] Updated weights for policy 0, policy_version 827166 (0.0009) [2023-12-26 21:22:59,198][105692] Updated weights for policy 0, policy_version 827176 (0.0008) [2023-12-26 21:22:59,228][105620] Updated weights for policy 1, policy_version 826954 (0.0009) [2023-12-26 21:22:59,300][105620] Updated weights for policy 1, policy_version 826964 (0.0008) [2023-12-26 21:22:59,365][105620] Updated weights for policy 1, policy_version 826974 (0.0007) [2023-12-26 21:22:59,427][105620] Updated weights for policy 1, policy_version 826984 (0.0008) [2023-12-26 21:22:59,906][105692] Updated weights for policy 0, policy_version 827186 (0.0011) [2023-12-26 21:22:59,974][105692] Updated weights for policy 0, policy_version 827196 (0.0011) [2023-12-26 21:23:00,034][105692] Updated weights for policy 0, policy_version 827206 (0.0011) [2023-12-26 21:23:00,201][105620] Updated weights for policy 1, policy_version 826994 (0.0007) [2023-12-26 21:23:00,253][105620] Updated weights for policy 1, policy_version 827004 (0.0008) [2023-12-26 21:23:00,301][105620] Updated weights for policy 1, policy_version 827014 (0.0008) [2023-12-26 21:23:00,701][105692] Updated weights for policy 0, policy_version 827216 (0.0007) [2023-12-26 21:23:00,755][105692] Updated weights for policy 0, policy_version 827226 (0.0005) [2023-12-26 21:23:00,812][105692] Updated weights for policy 0, policy_version 827236 (0.0006) [2023-12-26 21:23:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 423542784. Throughput: 0: 9837.9, 1: 9828.9. Samples: 423515948. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-26 21:23:01,062][104569] Avg episode reward: [(0, '7560.534'), (1, '8946.544')] [2023-12-26 21:23:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000827016_211738624.pth... [2023-12-26 21:23:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000827240_211804160.pth... [2023-12-26 21:23:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000825896_211451904.pth [2023-12-26 21:23:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000826088_211509248.pth [2023-12-26 21:23:01,139][105620] Updated weights for policy 1, policy_version 827024 (0.0010) [2023-12-26 21:23:01,187][105620] Updated weights for policy 1, policy_version 827034 (0.0010) [2023-12-26 21:23:01,242][105620] Updated weights for policy 1, policy_version 827044 (0.0010) [2023-12-26 21:23:01,431][105692] Updated weights for policy 0, policy_version 827246 (0.0005) [2023-12-26 21:23:01,481][105692] Updated weights for policy 0, policy_version 827256 (0.0005) [2023-12-26 21:23:01,534][105692] Updated weights for policy 0, policy_version 827266 (0.0007) [2023-12-26 21:23:02,016][105620] Updated weights for policy 1, policy_version 827054 (0.0010) [2023-12-26 21:23:02,067][105620] Updated weights for policy 1, policy_version 827064 (0.0010) [2023-12-26 21:23:02,115][105620] Updated weights for policy 1, policy_version 827074 (0.0010) [2023-12-26 21:23:02,230][105692] Updated weights for policy 0, policy_version 827276 (0.0008) [2023-12-26 21:23:02,283][105692] Updated weights for policy 0, policy_version 827286 (0.0008) [2023-12-26 21:23:02,333][105692] Updated weights for policy 0, policy_version 827296 (0.0008) [2023-12-26 21:23:02,882][105620] Updated weights for policy 1, policy_version 827084 (0.0010) [2023-12-26 21:23:02,921][105692] Updated weights for policy 0, policy_version 827306 (0.0008) [2023-12-26 21:23:02,933][105620] Updated weights for policy 1, policy_version 827094 (0.0010) [2023-12-26 21:23:02,967][105692] Updated weights for policy 0, policy_version 827316 (0.0005) [2023-12-26 21:23:02,997][105620] Updated weights for policy 1, policy_version 827104 (0.0009) [2023-12-26 21:23:03,015][105692] Updated weights for policy 0, policy_version 827326 (0.0007) [2023-12-26 21:23:03,074][105692] Updated weights for policy 0, policy_version 827336 (0.0009) [2023-12-26 21:23:03,676][105620] Updated weights for policy 1, policy_version 827114 (0.0006) [2023-12-26 21:23:03,722][105620] Updated weights for policy 1, policy_version 827124 (0.0008) [2023-12-26 21:23:03,768][105620] Updated weights for policy 1, policy_version 827134 (0.0008) [2023-12-26 21:23:03,825][105620] Updated weights for policy 1, policy_version 827144 (0.0007) [2023-12-26 21:23:03,831][105692] Updated weights for policy 0, policy_version 827346 (0.0009) [2023-12-26 21:23:03,892][105692] Updated weights for policy 0, policy_version 827356 (0.0009) [2023-12-26 21:23:03,955][105692] Updated weights for policy 0, policy_version 827366 (0.0009) [2023-12-26 21:23:04,616][105620] Updated weights for policy 1, policy_version 827154 (0.0011) [2023-12-26 21:23:04,678][105620] Updated weights for policy 1, policy_version 827164 (0.0010) [2023-12-26 21:23:04,720][105692] Updated weights for policy 0, policy_version 827376 (0.0007) [2023-12-26 21:23:04,734][105620] Updated weights for policy 1, policy_version 827174 (0.0010) [2023-12-26 21:23:04,766][105692] Updated weights for policy 0, policy_version 827386 (0.0006) [2023-12-26 21:23:04,814][105692] Updated weights for policy 0, policy_version 827396 (0.0008) [2023-12-26 21:23:05,473][105620] Updated weights for policy 1, policy_version 827184 (0.0010) [2023-12-26 21:23:05,527][105620] Updated weights for policy 1, policy_version 827194 (0.0010) [2023-12-26 21:23:05,584][105692] Updated weights for policy 0, policy_version 827406 (0.0006) [2023-12-26 21:23:05,589][105620] Updated weights for policy 1, policy_version 827204 (0.0011) [2023-12-26 21:23:05,637][105692] Updated weights for policy 0, policy_version 827416 (0.0007) [2023-12-26 21:23:05,684][105692] Updated weights for policy 0, policy_version 827426 (0.0007) [2023-12-26 21:23:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 423641088. Throughput: 0: 9773.3, 1: 9716.4. Samples: 423631060. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 21:23:06,063][104569] Avg episode reward: [(0, '7919.183'), (1, '9222.689')] [2023-12-26 21:23:06,345][105620] Updated weights for policy 1, policy_version 827214 (0.0010) [2023-12-26 21:23:06,402][105620] Updated weights for policy 1, policy_version 827224 (0.0011) [2023-12-26 21:23:06,459][105620] Updated weights for policy 1, policy_version 827234 (0.0011) [2023-12-26 21:23:06,472][105692] Updated weights for policy 0, policy_version 827436 (0.0009) [2023-12-26 21:23:06,531][105692] Updated weights for policy 0, policy_version 827446 (0.0011) [2023-12-26 21:23:06,584][105692] Updated weights for policy 0, policy_version 827456 (0.0011) [2023-12-26 21:23:07,212][105620] Updated weights for policy 1, policy_version 827244 (0.0008) [2023-12-26 21:23:07,268][105620] Updated weights for policy 1, policy_version 827254 (0.0009) [2023-12-26 21:23:07,328][105620] Updated weights for policy 1, policy_version 827264 (0.0011) [2023-12-26 21:23:07,361][105692] Updated weights for policy 0, policy_version 827466 (0.0010) [2023-12-26 21:23:07,418][105692] Updated weights for policy 0, policy_version 827476 (0.0008) [2023-12-26 21:23:07,469][105692] Updated weights for policy 0, policy_version 827486 (0.0010) [2023-12-26 21:23:07,518][105692] Updated weights for policy 0, policy_version 827496 (0.0008) [2023-12-26 21:23:08,050][105620] Updated weights for policy 1, policy_version 827274 (0.0010) [2023-12-26 21:23:08,095][105620] Updated weights for policy 1, policy_version 827284 (0.0006) [2023-12-26 21:23:08,143][105620] Updated weights for policy 1, policy_version 827294 (0.0009) [2023-12-26 21:23:08,192][105620] Updated weights for policy 1, policy_version 827304 (0.0008) [2023-12-26 21:23:08,252][105692] Updated weights for policy 0, policy_version 827506 (0.0008) [2023-12-26 21:23:08,310][105692] Updated weights for policy 0, policy_version 827516 (0.0013) [2023-12-26 21:23:08,368][105692] Updated weights for policy 0, policy_version 827527 (0.0010) [2023-12-26 21:23:08,924][105620] Updated weights for policy 1, policy_version 827314 (0.0005) [2023-12-26 21:23:08,976][105620] Updated weights for policy 1, policy_version 827324 (0.0005) [2023-12-26 21:23:09,024][105620] Updated weights for policy 1, policy_version 827334 (0.0005) [2023-12-26 21:23:09,084][105692] Updated weights for policy 0, policy_version 827537 (0.0009) [2023-12-26 21:23:09,134][105692] Updated weights for policy 0, policy_version 827547 (0.0009) [2023-12-26 21:23:09,199][105692] Updated weights for policy 0, policy_version 827557 (0.0009) [2023-12-26 21:23:09,756][105620] Updated weights for policy 1, policy_version 827344 (0.0007) [2023-12-26 21:23:09,821][105620] Updated weights for policy 1, policy_version 827354 (0.0008) [2023-12-26 21:23:09,888][105620] Updated weights for policy 1, policy_version 827364 (0.0009) [2023-12-26 21:23:10,000][105692] Updated weights for policy 0, policy_version 827567 (0.0010) [2023-12-26 21:23:10,052][105692] Updated weights for policy 0, policy_version 827577 (0.0010) [2023-12-26 21:23:10,102][105692] Updated weights for policy 0, policy_version 827587 (0.0010) [2023-12-26 21:23:10,658][105620] Updated weights for policy 1, policy_version 827374 (0.0008) [2023-12-26 21:23:10,715][105620] Updated weights for policy 1, policy_version 827384 (0.0008) [2023-12-26 21:23:10,776][105620] Updated weights for policy 1, policy_version 827394 (0.0008) [2023-12-26 21:23:10,874][105692] Updated weights for policy 0, policy_version 827597 (0.0011) [2023-12-26 21:23:10,922][105692] Updated weights for policy 0, policy_version 827607 (0.0010) [2023-12-26 21:23:10,971][105692] Updated weights for policy 0, policy_version 827617 (0.0010) [2023-12-26 21:23:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 423739392. Throughput: 0: 9750.8, 1: 9634.9. Samples: 423743088. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 21:23:11,062][104569] Avg episode reward: [(0, '8010.753'), (1, '9257.033')] [2023-12-26 21:23:11,590][105620] Updated weights for policy 1, policy_version 827404 (0.0008) [2023-12-26 21:23:11,659][105620] Updated weights for policy 1, policy_version 827414 (0.0008) [2023-12-26 21:23:11,729][105620] Updated weights for policy 1, policy_version 827424 (0.0009) [2023-12-26 21:23:11,762][105692] Updated weights for policy 0, policy_version 827627 (0.0010) [2023-12-26 21:23:11,818][105692] Updated weights for policy 0, policy_version 827637 (0.0011) [2023-12-26 21:23:11,878][105692] Updated weights for policy 0, policy_version 827647 (0.0011) [2023-12-26 21:23:12,524][105620] Updated weights for policy 1, policy_version 827434 (0.0008) [2023-12-26 21:23:12,573][105620] Updated weights for policy 1, policy_version 827444 (0.0010) [2023-12-26 21:23:12,598][105692] Updated weights for policy 0, policy_version 827657 (0.0010) [2023-12-26 21:23:12,625][105620] Updated weights for policy 1, policy_version 827454 (0.0010) [2023-12-26 21:23:12,655][105692] Updated weights for policy 0, policy_version 827667 (0.0006) [2023-12-26 21:23:12,677][105620] Updated weights for policy 1, policy_version 827464 (0.0010) [2023-12-26 21:23:12,712][105692] Updated weights for policy 0, policy_version 827677 (0.0008) [2023-12-26 21:23:12,777][105692] Updated weights for policy 0, policy_version 827687 (0.0009) [2023-12-26 21:23:13,289][105620] Updated weights for policy 1, policy_version 827474 (0.0009) [2023-12-26 21:23:13,338][105620] Updated weights for policy 1, policy_version 827484 (0.0008) [2023-12-26 21:23:13,401][105620] Updated weights for policy 1, policy_version 827494 (0.0009) [2023-12-26 21:23:13,539][105692] Updated weights for policy 0, policy_version 827697 (0.0006) [2023-12-26 21:23:13,591][105692] Updated weights for policy 0, policy_version 827707 (0.0006) [2023-12-26 21:23:13,646][105692] Updated weights for policy 0, policy_version 827717 (0.0006) [2023-12-26 21:23:14,202][105692] Updated weights for policy 0, policy_version 827727 (0.0007) [2023-12-26 21:23:14,233][105620] Updated weights for policy 1, policy_version 827504 (0.0007) [2023-12-26 21:23:14,263][105692] Updated weights for policy 0, policy_version 827737 (0.0007) [2023-12-26 21:23:14,299][105620] Updated weights for policy 1, policy_version 827514 (0.0008) [2023-12-26 21:23:14,317][105692] Updated weights for policy 0, policy_version 827747 (0.0005) [2023-12-26 21:23:14,360][105620] Updated weights for policy 1, policy_version 827524 (0.0009) [2023-12-26 21:23:14,967][105692] Updated weights for policy 0, policy_version 827757 (0.0007) [2023-12-26 21:23:15,030][105692] Updated weights for policy 0, policy_version 827767 (0.0010) [2023-12-26 21:23:15,085][105692] Updated weights for policy 0, policy_version 827777 (0.0009) [2023-12-26 21:23:15,160][105620] Updated weights for policy 1, policy_version 827534 (0.0007) [2023-12-26 21:23:15,213][105620] Updated weights for policy 1, policy_version 827544 (0.0008) [2023-12-26 21:23:15,272][105620] Updated weights for policy 1, policy_version 827554 (0.0009) [2023-12-26 21:23:15,905][105692] Updated weights for policy 0, policy_version 827787 (0.0009) [2023-12-26 21:23:15,960][105692] Updated weights for policy 0, policy_version 827797 (0.0008) [2023-12-26 21:23:16,009][105692] Updated weights for policy 0, policy_version 827807 (0.0008) [2023-12-26 21:23:16,025][105620] Updated weights for policy 1, policy_version 827564 (0.0009) [2023-12-26 21:23:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 423829504. Throughput: 0: 9755.0, 1: 9560.2. Samples: 423799376. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 21:23:16,063][104569] Avg episode reward: [(0, '8814.300'), (1, '9165.566')] [2023-12-26 21:23:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000827816_211951616.pth... [2023-12-26 21:23:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000826664_211656704.pth [2023-12-26 21:23:16,078][105620] Updated weights for policy 1, policy_version 827574 (0.0010) [2023-12-26 21:23:16,136][105620] Updated weights for policy 1, policy_version 827584 (0.0007) [2023-12-26 21:23:16,186][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000827592_211886080.pth... [2023-12-26 21:23:16,190][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000826504_211607552.pth [2023-12-26 21:23:16,729][105620] Updated weights for policy 1, policy_version 827594 (0.0009) [2023-12-26 21:23:16,793][105620] Updated weights for policy 1, policy_version 827604 (0.0010) [2023-12-26 21:23:16,822][105692] Updated weights for policy 0, policy_version 827817 (0.0006) [2023-12-26 21:23:16,858][105620] Updated weights for policy 1, policy_version 827614 (0.0010) [2023-12-26 21:23:16,882][105692] Updated weights for policy 0, policy_version 827827 (0.0005) [2023-12-26 21:23:16,916][105620] Updated weights for policy 1, policy_version 827624 (0.0010) [2023-12-26 21:23:16,937][105692] Updated weights for policy 0, policy_version 827837 (0.0005) [2023-12-26 21:23:16,997][105692] Updated weights for policy 0, policy_version 827847 (0.0009) [2023-12-26 21:23:17,616][105692] Updated weights for policy 0, policy_version 827857 (0.0010) [2023-12-26 21:23:17,635][105620] Updated weights for policy 1, policy_version 827634 (0.0008) [2023-12-26 21:23:17,679][105692] Updated weights for policy 0, policy_version 827867 (0.0010) [2023-12-26 21:23:17,695][105620] Updated weights for policy 1, policy_version 827644 (0.0010) [2023-12-26 21:23:17,737][105692] Updated weights for policy 0, policy_version 827877 (0.0010) [2023-12-26 21:23:17,757][105620] Updated weights for policy 1, policy_version 827654 (0.0011) [2023-12-26 21:23:18,428][105692] Updated weights for policy 0, policy_version 827887 (0.0011) [2023-12-26 21:23:18,455][105620] Updated weights for policy 1, policy_version 827664 (0.0010) [2023-12-26 21:23:18,487][105692] Updated weights for policy 0, policy_version 827897 (0.0011) [2023-12-26 21:23:18,514][105620] Updated weights for policy 1, policy_version 827674 (0.0010) [2023-12-26 21:23:18,542][105692] Updated weights for policy 0, policy_version 827907 (0.0010) [2023-12-26 21:23:18,573][105620] Updated weights for policy 1, policy_version 827684 (0.0011) [2023-12-26 21:23:19,180][105620] Updated weights for policy 1, policy_version 827694 (0.0006) [2023-12-26 21:23:19,245][105620] Updated weights for policy 1, policy_version 827704 (0.0006) [2023-12-26 21:23:19,274][105692] Updated weights for policy 0, policy_version 827917 (0.0011) [2023-12-26 21:23:19,303][105620] Updated weights for policy 1, policy_version 827714 (0.0007) [2023-12-26 21:23:19,330][105692] Updated weights for policy 0, policy_version 827927 (0.0007) [2023-12-26 21:23:19,394][105692] Updated weights for policy 0, policy_version 827937 (0.0011) [2023-12-26 21:23:19,993][105620] Updated weights for policy 1, policy_version 827724 (0.0008) [2023-12-26 21:23:20,059][105620] Updated weights for policy 1, policy_version 827734 (0.0010) [2023-12-26 21:23:20,107][105692] Updated weights for policy 0, policy_version 827947 (0.0007) [2023-12-26 21:23:20,128][105620] Updated weights for policy 1, policy_version 827744 (0.0008) [2023-12-26 21:23:20,172][105692] Updated weights for policy 0, policy_version 827957 (0.0007) [2023-12-26 21:23:20,237][105692] Updated weights for policy 0, policy_version 827967 (0.0009) [2023-12-26 21:23:20,834][105692] Updated weights for policy 0, policy_version 827977 (0.0009) [2023-12-26 21:23:20,898][105692] Updated weights for policy 0, policy_version 827987 (0.0009) [2023-12-26 21:23:20,952][105692] Updated weights for policy 0, policy_version 827997 (0.0009) [2023-12-26 21:23:20,975][105620] Updated weights for policy 1, policy_version 827754 (0.0010) [2023-12-26 21:23:21,002][105692] Updated weights for policy 0, policy_version 828007 (0.0007) [2023-12-26 21:23:21,036][105620] Updated weights for policy 1, policy_version 827764 (0.0008) [2023-12-26 21:23:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19388.5, 300 sec: 19522.0). Total num frames: 423927808. Throughput: 0: 9771.5, 1: 9539.6. Samples: 423918732. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 21:23:21,062][104569] Avg episode reward: [(0, '8286.715'), (1, '9074.725')] [2023-12-26 21:23:21,097][105620] Updated weights for policy 1, policy_version 827774 (0.0008) [2023-12-26 21:23:21,162][105620] Updated weights for policy 1, policy_version 827784 (0.0009) [2023-12-26 21:23:21,829][105692] Updated weights for policy 0, policy_version 828017 (0.0008) [2023-12-26 21:23:21,889][105692] Updated weights for policy 0, policy_version 828027 (0.0008) [2023-12-26 21:23:21,937][105692] Updated weights for policy 0, policy_version 828037 (0.0009) [2023-12-26 21:23:21,954][105620] Updated weights for policy 1, policy_version 827794 (0.0007) [2023-12-26 21:23:22,013][105620] Updated weights for policy 1, policy_version 827804 (0.0009) [2023-12-26 21:23:22,065][105620] Updated weights for policy 1, policy_version 827814 (0.0008) [2023-12-26 21:23:22,729][105692] Updated weights for policy 0, policy_version 828047 (0.0008) [2023-12-26 21:23:22,776][105692] Updated weights for policy 0, policy_version 828057 (0.0008) [2023-12-26 21:23:22,832][105692] Updated weights for policy 0, policy_version 828067 (0.0008) [2023-12-26 21:23:22,857][105620] Updated weights for policy 1, policy_version 827824 (0.0007) [2023-12-26 21:23:22,915][105620] Updated weights for policy 1, policy_version 827834 (0.0008) [2023-12-26 21:23:22,968][105620] Updated weights for policy 1, policy_version 827845 (0.0010) [2023-12-26 21:23:23,556][105692] Updated weights for policy 0, policy_version 828077 (0.0009) [2023-12-26 21:23:23,605][105692] Updated weights for policy 0, policy_version 828087 (0.0010) [2023-12-26 21:23:23,649][105620] Updated weights for policy 1, policy_version 827855 (0.0006) [2023-12-26 21:23:23,659][105692] Updated weights for policy 0, policy_version 828097 (0.0010) [2023-12-26 21:23:23,710][105620] Updated weights for policy 1, policy_version 827865 (0.0005) [2023-12-26 21:23:23,771][105620] Updated weights for policy 1, policy_version 827875 (0.0008) [2023-12-26 21:23:24,306][105692] Updated weights for policy 0, policy_version 828107 (0.0010) [2023-12-26 21:23:24,367][105692] Updated weights for policy 0, policy_version 828117 (0.0010) [2023-12-26 21:23:24,377][105620] Updated weights for policy 1, policy_version 827885 (0.0010) [2023-12-26 21:23:24,422][105692] Updated weights for policy 0, policy_version 828127 (0.0010) [2023-12-26 21:23:24,437][105620] Updated weights for policy 1, policy_version 827895 (0.0010) [2023-12-26 21:23:24,495][105620] Updated weights for policy 1, policy_version 827905 (0.0010) [2023-12-26 21:23:25,152][105692] Updated weights for policy 0, policy_version 828137 (0.0011) [2023-12-26 21:23:25,191][105620] Updated weights for policy 1, policy_version 827915 (0.0010) [2023-12-26 21:23:25,216][105692] Updated weights for policy 0, policy_version 828147 (0.0010) [2023-12-26 21:23:25,242][105620] Updated weights for policy 1, policy_version 827925 (0.0010) [2023-12-26 21:23:25,285][105692] Updated weights for policy 0, policy_version 828157 (0.0010) [2023-12-26 21:23:25,304][105620] Updated weights for policy 1, policy_version 827935 (0.0010) [2023-12-26 21:23:25,344][105692] Updated weights for policy 0, policy_version 828167 (0.0010) [2023-12-26 21:23:26,057][105620] Updated weights for policy 1, policy_version 827945 (0.0011) [2023-12-26 21:23:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 424017920. Throughput: 0: 9745.8, 1: 9435.3. Samples: 424033476. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 21:23:26,063][104569] Avg episode reward: [(0, '8193.241'), (1, '9074.322')] [2023-12-26 21:23:26,070][105692] Updated weights for policy 0, policy_version 828177 (0.0010) [2023-12-26 21:23:26,112][105620] Updated weights for policy 1, policy_version 827955 (0.0010) [2023-12-26 21:23:26,128][105692] Updated weights for policy 0, policy_version 828187 (0.0010) [2023-12-26 21:23:26,170][105620] Updated weights for policy 1, policy_version 827965 (0.0010) [2023-12-26 21:23:26,183][105692] Updated weights for policy 0, policy_version 828197 (0.0010) [2023-12-26 21:23:26,222][105620] Updated weights for policy 1, policy_version 827975 (0.0010) [2023-12-26 21:23:26,820][105620] Updated weights for policy 1, policy_version 827985 (0.0006) [2023-12-26 21:23:26,841][105692] Updated weights for policy 0, policy_version 828207 (0.0007) [2023-12-26 21:23:26,865][105620] Updated weights for policy 1, policy_version 827995 (0.0005) [2023-12-26 21:23:26,893][105692] Updated weights for policy 0, policy_version 828217 (0.0010) [2023-12-26 21:23:26,914][105620] Updated weights for policy 1, policy_version 828005 (0.0008) [2023-12-26 21:23:26,951][105692] Updated weights for policy 0, policy_version 828227 (0.0010) [2023-12-26 21:23:27,524][105620] Updated weights for policy 1, policy_version 828015 (0.0007) [2023-12-26 21:23:27,580][105620] Updated weights for policy 1, policy_version 828025 (0.0007) [2023-12-26 21:23:27,591][105692] Updated weights for policy 0, policy_version 828237 (0.0009) [2023-12-26 21:23:27,641][105620] Updated weights for policy 1, policy_version 828035 (0.0010) [2023-12-26 21:23:27,658][105692] Updated weights for policy 0, policy_version 828247 (0.0006) [2023-12-26 21:23:27,711][105692] Updated weights for policy 0, policy_version 828257 (0.0010) [2023-12-26 21:23:28,267][105620] Updated weights for policy 1, policy_version 828045 (0.0009) [2023-12-26 21:23:28,311][105620] Updated weights for policy 1, policy_version 828055 (0.0008) [2023-12-26 21:23:28,371][105620] Updated weights for policy 1, policy_version 828065 (0.0008) [2023-12-26 21:23:28,419][105692] Updated weights for policy 0, policy_version 828267 (0.0009) [2023-12-26 21:23:28,486][105692] Updated weights for policy 0, policy_version 828277 (0.0006) [2023-12-26 21:23:28,542][105692] Updated weights for policy 0, policy_version 828287 (0.0008) [2023-12-26 21:23:29,163][105692] Updated weights for policy 0, policy_version 828297 (0.0009) [2023-12-26 21:23:29,178][105620] Updated weights for policy 1, policy_version 828075 (0.0009) [2023-12-26 21:23:29,229][105692] Updated weights for policy 0, policy_version 828307 (0.0007) [2023-12-26 21:23:29,246][105620] Updated weights for policy 1, policy_version 828085 (0.0011) [2023-12-26 21:23:29,293][105692] Updated weights for policy 0, policy_version 828317 (0.0009) [2023-12-26 21:23:29,298][105620] Updated weights for policy 1, policy_version 828095 (0.0011) [2023-12-26 21:23:29,356][105692] Updated weights for policy 0, policy_version 828327 (0.0009) [2023-12-26 21:23:30,014][105692] Updated weights for policy 0, policy_version 828337 (0.0010) [2023-12-26 21:23:30,043][105620] Updated weights for policy 1, policy_version 828105 (0.0011) [2023-12-26 21:23:30,076][105692] Updated weights for policy 0, policy_version 828347 (0.0010) [2023-12-26 21:23:30,098][105620] Updated weights for policy 1, policy_version 828115 (0.0010) [2023-12-26 21:23:30,134][105692] Updated weights for policy 0, policy_version 828357 (0.0010) [2023-12-26 21:23:30,150][105620] Updated weights for policy 1, policy_version 828125 (0.0010) [2023-12-26 21:23:30,195][105620] Updated weights for policy 1, policy_version 828135 (0.0010) [2023-12-26 21:23:30,875][105692] Updated weights for policy 0, policy_version 828367 (0.0010) [2023-12-26 21:23:30,905][105620] Updated weights for policy 1, policy_version 828145 (0.0006) [2023-12-26 21:23:30,930][105692] Updated weights for policy 0, policy_version 828377 (0.0010) [2023-12-26 21:23:30,943][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000005 [2023-12-26 21:23:30,985][105692] Updated weights for policy 0, policy_version 828387 (0.0011) [2023-12-26 21:23:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 424132608. Throughput: 0: 9833.1, 1: 9514.2. Samples: 424095604. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 21:23:31,063][104569] Avg episode reward: [(0, '8429.311'), (1, '9164.982')] [2023-12-26 21:23:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000828392_212099072.pth... [2023-12-26 21:23:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000828152_212033536.pth... [2023-12-26 21:23:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000827240_211804160.pth [2023-12-26 21:23:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000827016_211738624.pth [2023-12-26 21:23:31,727][105620] Updated weights for policy 1, policy_version 828155 (0.0007) [2023-12-26 21:23:31,749][105692] Updated weights for policy 0, policy_version 828397 (0.0010) [2023-12-26 21:23:31,795][105620] Updated weights for policy 1, policy_version 828165 (0.0008) [2023-12-26 21:23:31,813][105692] Updated weights for policy 0, policy_version 828407 (0.0010) [2023-12-26 21:23:31,843][105620] Updated weights for policy 1, policy_version 828175 (0.0008) [2023-12-26 21:23:31,871][105692] Updated weights for policy 0, policy_version 828417 (0.0010) [2023-12-26 21:23:32,575][105692] Updated weights for policy 0, policy_version 828427 (0.0009) [2023-12-26 21:23:32,626][105620] Updated weights for policy 1, policy_version 828185 (0.0008) [2023-12-26 21:23:32,638][105692] Updated weights for policy 0, policy_version 828437 (0.0005) [2023-12-26 21:23:32,675][105620] Updated weights for policy 1, policy_version 828196 (0.0010) [2023-12-26 21:23:32,699][105692] Updated weights for policy 0, policy_version 828447 (0.0007) [2023-12-26 21:23:32,739][105620] Updated weights for policy 1, policy_version 828206 (0.0008) [2023-12-26 21:23:32,798][105620] Updated weights for policy 1, policy_version 828216 (0.0010) [2023-12-26 21:23:33,296][105692] Updated weights for policy 0, policy_version 828457 (0.0006) [2023-12-26 21:23:33,354][105692] Updated weights for policy 0, policy_version 828467 (0.0010) [2023-12-26 21:23:33,418][105692] Updated weights for policy 0, policy_version 828477 (0.0010) [2023-12-26 21:23:33,482][105692] Updated weights for policy 0, policy_version 828487 (0.0010) [2023-12-26 21:23:33,511][105620] Updated weights for policy 1, policy_version 828226 (0.0008) [2023-12-26 21:23:33,558][105620] Updated weights for policy 1, policy_version 828236 (0.0006) [2023-12-26 21:23:33,607][105620] Updated weights for policy 1, policy_version 828246 (0.0006) [2023-12-26 21:23:34,213][105620] Updated weights for policy 1, policy_version 828256 (0.0010) [2023-12-26 21:23:34,217][105692] Updated weights for policy 0, policy_version 828497 (0.0011) [2023-12-26 21:23:34,269][105692] Updated weights for policy 0, policy_version 828507 (0.0010) [2023-12-26 21:23:34,273][105620] Updated weights for policy 1, policy_version 828266 (0.0010) [2023-12-26 21:23:34,321][105692] Updated weights for policy 0, policy_version 828517 (0.0010) [2023-12-26 21:23:34,325][105620] Updated weights for policy 1, policy_version 828276 (0.0010) [2023-12-26 21:23:35,008][105692] Updated weights for policy 0, policy_version 828527 (0.0007) [2023-12-26 21:23:35,072][105692] Updated weights for policy 0, policy_version 828537 (0.0005) [2023-12-26 21:23:35,099][105620] Updated weights for policy 1, policy_version 828286 (0.0010) [2023-12-26 21:23:35,129][105692] Updated weights for policy 0, policy_version 828547 (0.0006) [2023-12-26 21:23:35,155][105620] Updated weights for policy 1, policy_version 828296 (0.0010) [2023-12-26 21:23:35,222][105620] Updated weights for policy 1, policy_version 828306 (0.0011) [2023-12-26 21:23:35,690][105692] Updated weights for policy 0, policy_version 828557 (0.0006) [2023-12-26 21:23:35,746][105692] Updated weights for policy 0, policy_version 828567 (0.0005) [2023-12-26 21:23:35,807][105692] Updated weights for policy 0, policy_version 828577 (0.0006) [2023-12-26 21:23:35,962][105620] Updated weights for policy 1, policy_version 828316 (0.0011) [2023-12-26 21:23:36,017][105620] Updated weights for policy 1, policy_version 828326 (0.0010) [2023-12-26 21:23:36,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19387.6, 300 sec: 19521.9). Total num frames: 424222720. Throughput: 0: 9838.7, 1: 9510.7. Samples: 424213208. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 21:23:36,063][104569] Avg episode reward: [(0, '8497.629'), (1, '9347.777')] [2023-12-26 21:23:36,079][105620] Updated weights for policy 1, policy_version 828336 (0.0010) [2023-12-26 21:23:36,463][105692] Updated weights for policy 0, policy_version 828587 (0.0007) [2023-12-26 21:23:36,520][105692] Updated weights for policy 0, policy_version 828597 (0.0008) [2023-12-26 21:23:36,584][105692] Updated weights for policy 0, policy_version 828607 (0.0009) [2023-12-26 21:23:36,764][105620] Updated weights for policy 1, policy_version 828346 (0.0009) [2023-12-26 21:23:36,831][105620] Updated weights for policy 1, policy_version 828356 (0.0008) [2023-12-26 21:23:36,893][105620] Updated weights for policy 1, policy_version 828366 (0.0006) [2023-12-26 21:23:36,940][105620] Updated weights for policy 1, policy_version 828376 (0.0005) [2023-12-26 21:23:37,324][105692] Updated weights for policy 0, policy_version 828617 (0.0009) [2023-12-26 21:23:37,391][105692] Updated weights for policy 0, policy_version 828627 (0.0006) [2023-12-26 21:23:37,460][105692] Updated weights for policy 0, policy_version 828637 (0.0007) [2023-12-26 21:23:37,516][105620] Updated weights for policy 1, policy_version 828386 (0.0008) [2023-12-26 21:23:37,524][105692] Updated weights for policy 0, policy_version 828647 (0.0007) [2023-12-26 21:23:37,580][105620] Updated weights for policy 1, policy_version 828396 (0.0009) [2023-12-26 21:23:37,647][105620] Updated weights for policy 1, policy_version 828406 (0.0006) [2023-12-26 21:23:38,242][105692] Updated weights for policy 0, policy_version 828657 (0.0008) [2023-12-26 21:23:38,278][105620] Updated weights for policy 1, policy_version 828416 (0.0010) [2023-12-26 21:23:38,289][105692] Updated weights for policy 0, policy_version 828667 (0.0007) [2023-12-26 21:23:38,347][105620] Updated weights for policy 1, policy_version 828426 (0.0010) [2023-12-26 21:23:38,348][105692] Updated weights for policy 0, policy_version 828677 (0.0008) [2023-12-26 21:23:38,403][105620] Updated weights for policy 1, policy_version 828436 (0.0010) [2023-12-26 21:23:39,067][105692] Updated weights for policy 0, policy_version 828687 (0.0006) [2023-12-26 21:23:39,129][105692] Updated weights for policy 0, policy_version 828697 (0.0005) [2023-12-26 21:23:39,157][105620] Updated weights for policy 1, policy_version 828446 (0.0010) [2023-12-26 21:23:39,194][105692] Updated weights for policy 0, policy_version 828707 (0.0008) [2023-12-26 21:23:39,213][105620] Updated weights for policy 1, policy_version 828456 (0.0007) [2023-12-26 21:23:39,276][105620] Updated weights for policy 1, policy_version 828466 (0.0009) [2023-12-26 21:23:39,853][105692] Updated weights for policy 0, policy_version 828717 (0.0007) [2023-12-26 21:23:39,922][105692] Updated weights for policy 0, policy_version 828727 (0.0009) [2023-12-26 21:23:39,986][105692] Updated weights for policy 0, policy_version 828737 (0.0009) [2023-12-26 21:23:40,037][105620] Updated weights for policy 1, policy_version 828476 (0.0009) [2023-12-26 21:23:40,095][105620] Updated weights for policy 1, policy_version 828486 (0.0009) [2023-12-26 21:23:40,159][105620] Updated weights for policy 1, policy_version 828496 (0.0008) [2023-12-26 21:23:40,782][105692] Updated weights for policy 0, policy_version 828747 (0.0009) [2023-12-26 21:23:40,842][105692] Updated weights for policy 0, policy_version 828757 (0.0010) [2023-12-26 21:23:40,848][105620] Updated weights for policy 1, policy_version 828506 (0.0005) [2023-12-26 21:23:40,896][105620] Updated weights for policy 1, policy_version 828516 (0.0005) [2023-12-26 21:23:40,906][105692] Updated weights for policy 0, policy_version 828767 (0.0009) [2023-12-26 21:23:40,951][105620] Updated weights for policy 1, policy_version 828526 (0.0006) [2023-12-26 21:23:40,999][105620] Updated weights for policy 1, policy_version 828536 (0.0006) [2023-12-26 21:23:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 424329216. Throughput: 0: 9846.5, 1: 9582.6. Samples: 424333112. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 21:23:41,062][104569] Avg episode reward: [(0, '8354.323'), (1, '9256.410')] [2023-12-26 21:23:41,632][105620] Updated weights for policy 1, policy_version 828546 (0.0011) [2023-12-26 21:23:41,685][105620] Updated weights for policy 1, policy_version 828556 (0.0009) [2023-12-26 21:23:41,751][105620] Updated weights for policy 1, policy_version 828566 (0.0008) [2023-12-26 21:23:41,804][105692] Updated weights for policy 0, policy_version 828777 (0.0008) [2023-12-26 21:23:41,870][105692] Updated weights for policy 0, policy_version 828787 (0.0009) [2023-12-26 21:23:41,928][105692] Updated weights for policy 0, policy_version 828797 (0.0008) [2023-12-26 21:23:41,976][105692] Updated weights for policy 0, policy_version 828807 (0.0008) [2023-12-26 21:23:42,497][105620] Updated weights for policy 1, policy_version 828576 (0.0010) [2023-12-26 21:23:42,546][105620] Updated weights for policy 1, policy_version 828586 (0.0009) [2023-12-26 21:23:42,595][105620] Updated weights for policy 1, policy_version 828596 (0.0008) [2023-12-26 21:23:42,747][105692] Updated weights for policy 0, policy_version 828817 (0.0005) [2023-12-26 21:23:42,805][105692] Updated weights for policy 0, policy_version 828827 (0.0007) [2023-12-26 21:23:42,870][105692] Updated weights for policy 0, policy_version 828837 (0.0008) [2023-12-26 21:23:43,387][105620] Updated weights for policy 1, policy_version 828606 (0.0008) [2023-12-26 21:23:43,441][105620] Updated weights for policy 1, policy_version 828616 (0.0009) [2023-12-26 21:23:43,488][105620] Updated weights for policy 1, policy_version 828626 (0.0008) [2023-12-26 21:23:43,505][105692] Updated weights for policy 0, policy_version 828847 (0.0006) [2023-12-26 21:23:43,559][105692] Updated weights for policy 0, policy_version 828858 (0.0009) [2023-12-26 21:23:43,617][105692] Updated weights for policy 0, policy_version 828868 (0.0009) [2023-12-26 21:23:44,284][105620] Updated weights for policy 1, policy_version 828636 (0.0007) [2023-12-26 21:23:44,287][105692] Updated weights for policy 0, policy_version 828878 (0.0008) [2023-12-26 21:23:44,333][105620] Updated weights for policy 1, policy_version 828646 (0.0006) [2023-12-26 21:23:44,343][105692] Updated weights for policy 0, policy_version 828888 (0.0006) [2023-12-26 21:23:44,378][105620] Updated weights for policy 1, policy_version 828656 (0.0005) [2023-12-26 21:23:44,399][105692] Updated weights for policy 0, policy_version 828898 (0.0007) [2023-12-26 21:23:45,113][105620] Updated weights for policy 1, policy_version 828666 (0.0007) [2023-12-26 21:23:45,175][105620] Updated weights for policy 1, policy_version 828676 (0.0009) [2023-12-26 21:23:45,191][105692] Updated weights for policy 0, policy_version 828908 (0.0007) [2023-12-26 21:23:45,237][105620] Updated weights for policy 1, policy_version 828686 (0.0008) [2023-12-26 21:23:45,256][105692] Updated weights for policy 0, policy_version 828918 (0.0007) [2023-12-26 21:23:45,303][105620] Updated weights for policy 1, policy_version 828696 (0.0007) [2023-12-26 21:23:45,323][105692] Updated weights for policy 0, policy_version 828928 (0.0007) [2023-12-26 21:23:45,990][105620] Updated weights for policy 1, policy_version 828706 (0.0008) [2023-12-26 21:23:46,058][105620] Updated weights for policy 1, policy_version 828716 (0.0008) [2023-12-26 21:23:46,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 424411136. Throughput: 0: 9751.9, 1: 9643.8. Samples: 424388756. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 21:23:46,062][104569] Avg episode reward: [(0, '8619.960'), (1, '9046.685')] [2023-12-26 21:23:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000828936_212238336.pth... [2023-12-26 21:23:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000827816_211951616.pth [2023-12-26 21:23:46,094][105692] Updated weights for policy 0, policy_version 828938 (0.0009) [2023-12-26 21:23:46,123][105620] Updated weights for policy 1, policy_version 828726 (0.0008) [2023-12-26 21:23:46,134][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000828728_212180992.pth... [2023-12-26 21:23:46,139][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000827592_211886080.pth [2023-12-26 21:23:46,153][105692] Updated weights for policy 0, policy_version 828948 (0.0007) [2023-12-26 21:23:46,213][105692] Updated weights for policy 0, policy_version 828958 (0.0012) [2023-12-26 21:23:46,699][105620] Updated weights for policy 1, policy_version 828736 (0.0008) [2023-12-26 21:23:46,767][105620] Updated weights for policy 1, policy_version 828746 (0.0007) [2023-12-26 21:23:46,831][105620] Updated weights for policy 1, policy_version 828756 (0.0007) [2023-12-26 21:23:47,044][105692] Updated weights for policy 0, policy_version 828969 (0.0010) [2023-12-26 21:23:47,108][105692] Updated weights for policy 0, policy_version 828979 (0.0010) [2023-12-26 21:23:47,174][105692] Updated weights for policy 0, policy_version 828989 (0.0008) [2023-12-26 21:23:47,230][105692] Updated weights for policy 0, policy_version 828999 (0.0010) [2023-12-26 21:23:47,464][105620] Updated weights for policy 1, policy_version 828766 (0.0009) [2023-12-26 21:23:47,527][105620] Updated weights for policy 1, policy_version 828776 (0.0008) [2023-12-26 21:23:47,591][105620] Updated weights for policy 1, policy_version 828786 (0.0006) [2023-12-26 21:23:47,852][105692] Updated weights for policy 0, policy_version 829009 (0.0009) [2023-12-26 21:23:47,911][105692] Updated weights for policy 0, policy_version 829019 (0.0006) [2023-12-26 21:23:47,978][105692] Updated weights for policy 0, policy_version 829029 (0.0006) [2023-12-26 21:23:48,199][105620] Updated weights for policy 1, policy_version 828796 (0.0005) [2023-12-26 21:23:48,249][105620] Updated weights for policy 1, policy_version 828806 (0.0005) [2023-12-26 21:23:48,299][105620] Updated weights for policy 1, policy_version 828816 (0.0005) [2023-12-26 21:23:48,566][105692] Updated weights for policy 0, policy_version 829039 (0.0006) [2023-12-26 21:23:48,631][105692] Updated weights for policy 0, policy_version 829049 (0.0008) [2023-12-26 21:23:48,703][105692] Updated weights for policy 0, policy_version 829059 (0.0008) [2023-12-26 21:23:49,028][105620] Updated weights for policy 1, policy_version 828826 (0.0007) [2023-12-26 21:23:49,089][105620] Updated weights for policy 1, policy_version 828836 (0.0010) [2023-12-26 21:23:49,153][105620] Updated weights for policy 1, policy_version 828846 (0.0009) [2023-12-26 21:23:49,218][105620] Updated weights for policy 1, policy_version 828856 (0.0009) [2023-12-26 21:23:49,284][105692] Updated weights for policy 0, policy_version 829069 (0.0007) [2023-12-26 21:23:49,351][105692] Updated weights for policy 0, policy_version 829079 (0.0008) [2023-12-26 21:23:49,408][105692] Updated weights for policy 0, policy_version 829089 (0.0006) [2023-12-26 21:23:49,983][105620] Updated weights for policy 1, policy_version 828866 (0.0009) [2023-12-26 21:23:50,037][105620] Updated weights for policy 1, policy_version 828876 (0.0008) [2023-12-26 21:23:50,097][105620] Updated weights for policy 1, policy_version 828886 (0.0010) [2023-12-26 21:23:50,133][105692] Updated weights for policy 0, policy_version 829099 (0.0008) [2023-12-26 21:23:50,188][105692] Updated weights for policy 0, policy_version 829109 (0.0009) [2023-12-26 21:23:50,246][105692] Updated weights for policy 0, policy_version 829119 (0.0009) [2023-12-26 21:23:50,851][105620] Updated weights for policy 1, policy_version 828896 (0.0008) [2023-12-26 21:23:50,917][105620] Updated weights for policy 1, policy_version 828906 (0.0007) [2023-12-26 21:23:50,973][105620] Updated weights for policy 1, policy_version 828916 (0.0009) [2023-12-26 21:23:51,029][105692] Updated weights for policy 0, policy_version 829129 (0.0009) [2023-12-26 21:23:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 424517632. Throughput: 0: 9740.5, 1: 9763.5. Samples: 424508740. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 21:23:51,063][104569] Avg episode reward: [(0, '8721.714'), (1, '9138.796')] [2023-12-26 21:23:51,098][105692] Updated weights for policy 0, policy_version 829139 (0.0009) [2023-12-26 21:23:51,162][105692] Updated weights for policy 0, policy_version 829149 (0.0009) [2023-12-26 21:23:51,213][105692] Updated weights for policy 0, policy_version 829159 (0.0008) [2023-12-26 21:23:51,668][105620] Updated weights for policy 1, policy_version 828926 (0.0010) [2023-12-26 21:23:51,736][105620] Updated weights for policy 1, policy_version 828936 (0.0009) [2023-12-26 21:23:51,805][105620] Updated weights for policy 1, policy_version 828946 (0.0011) [2023-12-26 21:23:52,027][105692] Updated weights for policy 0, policy_version 829169 (0.0008) [2023-12-26 21:23:52,071][105692] Updated weights for policy 0, policy_version 829179 (0.0008) [2023-12-26 21:23:52,124][105692] Updated weights for policy 0, policy_version 829189 (0.0008) [2023-12-26 21:23:52,543][105620] Updated weights for policy 1, policy_version 828956 (0.0011) [2023-12-26 21:23:52,606][105620] Updated weights for policy 1, policy_version 828966 (0.0010) [2023-12-26 21:23:52,666][105620] Updated weights for policy 1, policy_version 828976 (0.0010) [2023-12-26 21:23:52,893][105692] Updated weights for policy 0, policy_version 829199 (0.0006) [2023-12-26 21:23:52,947][105692] Updated weights for policy 0, policy_version 829209 (0.0005) [2023-12-26 21:23:52,995][105692] Updated weights for policy 0, policy_version 829219 (0.0005) [2023-12-26 21:23:53,409][105620] Updated weights for policy 1, policy_version 828986 (0.0010) [2023-12-26 21:23:53,474][105620] Updated weights for policy 1, policy_version 828996 (0.0005) [2023-12-26 21:23:53,530][105620] Updated weights for policy 1, policy_version 829006 (0.0005) [2023-12-26 21:23:53,590][105620] Updated weights for policy 1, policy_version 829016 (0.0005) [2023-12-26 21:23:53,692][105692] Updated weights for policy 0, policy_version 829229 (0.0008) [2023-12-26 21:23:53,744][105692] Updated weights for policy 0, policy_version 829239 (0.0006) [2023-12-26 21:23:53,799][105692] Updated weights for policy 0, policy_version 829249 (0.0006) [2023-12-26 21:23:54,128][105620] Updated weights for policy 1, policy_version 829026 (0.0008) [2023-12-26 21:23:54,193][105620] Updated weights for policy 1, policy_version 829036 (0.0010) [2023-12-26 21:23:54,257][105620] Updated weights for policy 1, policy_version 829046 (0.0010) [2023-12-26 21:23:54,407][105692] Updated weights for policy 0, policy_version 829259 (0.0007) [2023-12-26 21:23:54,472][105692] Updated weights for policy 0, policy_version 829269 (0.0005) [2023-12-26 21:23:54,532][105692] Updated weights for policy 0, policy_version 829279 (0.0005) [2023-12-26 21:23:54,953][105620] Updated weights for policy 1, policy_version 829056 (0.0010) [2023-12-26 21:23:55,012][105620] Updated weights for policy 1, policy_version 829066 (0.0010) [2023-12-26 21:23:55,074][105620] Updated weights for policy 1, policy_version 829076 (0.0010) [2023-12-26 21:23:55,185][105692] Updated weights for policy 0, policy_version 829289 (0.0007) [2023-12-26 21:23:55,233][105692] Updated weights for policy 0, policy_version 829299 (0.0010) [2023-12-26 21:23:55,280][105692] Updated weights for policy 0, policy_version 829309 (0.0010) [2023-12-26 21:23:55,330][105692] Updated weights for policy 0, policy_version 829319 (0.0010) [2023-12-26 21:23:55,801][105620] Updated weights for policy 1, policy_version 829086 (0.0010) [2023-12-26 21:23:55,846][105620] Updated weights for policy 1, policy_version 829096 (0.0010) [2023-12-26 21:23:55,904][105620] Updated weights for policy 1, policy_version 829106 (0.0010) [2023-12-26 21:23:56,001][105692] Updated weights for policy 0, policy_version 829329 (0.0008) [2023-12-26 21:23:56,057][105692] Updated weights for policy 0, policy_version 829339 (0.0008) [2023-12-26 21:23:56,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 424615936. Throughput: 0: 9788.2, 1: 9806.6. Samples: 424624860. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 21:23:56,063][104569] Avg episode reward: [(0, '8813.847'), (1, '9166.858')] [2023-12-26 21:23:56,116][105692] Updated weights for policy 0, policy_version 829349 (0.0009) [2023-12-26 21:23:56,671][105620] Updated weights for policy 1, policy_version 829116 (0.0010) [2023-12-26 21:23:56,733][105620] Updated weights for policy 1, policy_version 829126 (0.0010) [2023-12-26 21:23:56,788][105620] Updated weights for policy 1, policy_version 829136 (0.0010) [2023-12-26 21:23:56,844][105692] Updated weights for policy 0, policy_version 829359 (0.0010) [2023-12-26 21:23:56,897][105692] Updated weights for policy 0, policy_version 829369 (0.0010) [2023-12-26 21:23:56,953][105692] Updated weights for policy 0, policy_version 829379 (0.0010) [2023-12-26 21:23:57,529][105620] Updated weights for policy 1, policy_version 829146 (0.0010) [2023-12-26 21:23:57,589][105620] Updated weights for policy 1, policy_version 829156 (0.0008) [2023-12-26 21:23:57,639][105692] Updated weights for policy 0, policy_version 829389 (0.0008) [2023-12-26 21:23:57,650][105620] Updated weights for policy 1, policy_version 829166 (0.0008) [2023-12-26 21:23:57,698][105692] Updated weights for policy 0, policy_version 829399 (0.0005) [2023-12-26 21:23:57,709][105620] Updated weights for policy 1, policy_version 829176 (0.0007) [2023-12-26 21:23:57,760][105692] Updated weights for policy 0, policy_version 829409 (0.0005) [2023-12-26 21:23:58,351][105692] Updated weights for policy 0, policy_version 829419 (0.0006) [2023-12-26 21:23:58,417][105692] Updated weights for policy 0, policy_version 829429 (0.0010) [2023-12-26 21:23:58,425][105620] Updated weights for policy 1, policy_version 829186 (0.0007) [2023-12-26 21:23:58,480][105692] Updated weights for policy 0, policy_version 829439 (0.0011) [2023-12-26 21:23:58,490][105620] Updated weights for policy 1, policy_version 829196 (0.0008) [2023-12-26 21:23:58,547][105620] Updated weights for policy 1, policy_version 829206 (0.0008) [2023-12-26 21:23:59,263][105692] Updated weights for policy 0, policy_version 829449 (0.0010) [2023-12-26 21:23:59,323][105692] Updated weights for policy 0, policy_version 829459 (0.0010) [2023-12-26 21:23:59,353][105620] Updated weights for policy 1, policy_version 829216 (0.0007) [2023-12-26 21:23:59,388][105692] Updated weights for policy 0, policy_version 829469 (0.0009) [2023-12-26 21:23:59,411][105620] Updated weights for policy 1, policy_version 829226 (0.0006) [2023-12-26 21:23:59,443][105692] Updated weights for policy 0, policy_version 829479 (0.0010) [2023-12-26 21:23:59,462][105620] Updated weights for policy 1, policy_version 829236 (0.0007) [2023-12-26 21:24:00,094][105692] Updated weights for policy 0, policy_version 829489 (0.0008) [2023-12-26 21:24:00,161][105692] Updated weights for policy 0, policy_version 829499 (0.0007) [2023-12-26 21:24:00,214][105692] Updated weights for policy 0, policy_version 829509 (0.0006) [2023-12-26 21:24:00,336][105620] Updated weights for policy 1, policy_version 829246 (0.0008) [2023-12-26 21:24:00,400][105620] Updated weights for policy 1, policy_version 829256 (0.0009) [2023-12-26 21:24:00,470][105620] Updated weights for policy 1, policy_version 829266 (0.0009) [2023-12-26 21:24:00,899][105692] Updated weights for policy 0, policy_version 829519 (0.0011) [2023-12-26 21:24:00,944][105692] Updated weights for policy 0, policy_version 829529 (0.0010) [2023-12-26 21:24:01,014][105692] Updated weights for policy 0, policy_version 829539 (0.0011) [2023-12-26 21:24:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 424714240. Throughput: 0: 9861.3, 1: 9799.3. Samples: 424684104. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 21:24:01,062][104569] Avg episode reward: [(0, '8991.387'), (1, '9167.206')] [2023-12-26 21:24:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000829544_212393984.pth... [2023-12-26 21:24:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000828392_212099072.pth [2023-12-26 21:24:01,094][105620] Updated weights for policy 1, policy_version 829276 (0.0007) [2023-12-26 21:24:01,148][105620] Updated weights for policy 1, policy_version 829286 (0.0007) [2023-12-26 21:24:01,205][105620] Updated weights for policy 1, policy_version 829296 (0.0005) [2023-12-26 21:24:01,255][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000829304_212328448.pth... [2023-12-26 21:24:01,259][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000828152_212033536.pth [2023-12-26 21:24:01,734][105692] Updated weights for policy 0, policy_version 829549 (0.0009) [2023-12-26 21:24:01,795][105692] Updated weights for policy 0, policy_version 829559 (0.0010) [2023-12-26 21:24:01,850][105692] Updated weights for policy 0, policy_version 829569 (0.0010) [2023-12-26 21:24:01,912][105620] Updated weights for policy 1, policy_version 829306 (0.0006) [2023-12-26 21:24:01,971][105620] Updated weights for policy 1, policy_version 829317 (0.0006) [2023-12-26 21:24:02,030][105620] Updated weights for policy 1, policy_version 829327 (0.0010) [2023-12-26 21:24:02,476][105692] Updated weights for policy 0, policy_version 829579 (0.0011) [2023-12-26 21:24:02,524][105692] Updated weights for policy 0, policy_version 829589 (0.0010) [2023-12-26 21:24:02,575][105692] Updated weights for policy 0, policy_version 829599 (0.0010) [2023-12-26 21:24:02,760][105620] Updated weights for policy 1, policy_version 829337 (0.0009) [2023-12-26 21:24:02,820][105620] Updated weights for policy 1, policy_version 829347 (0.0007) [2023-12-26 21:24:02,875][105620] Updated weights for policy 1, policy_version 829357 (0.0007) [2023-12-26 21:24:02,925][105620] Updated weights for policy 1, policy_version 829367 (0.0007) [2023-12-26 21:24:03,329][105692] Updated weights for policy 0, policy_version 829609 (0.0010) [2023-12-26 21:24:03,392][105692] Updated weights for policy 0, policy_version 829619 (0.0010) [2023-12-26 21:24:03,440][105692] Updated weights for policy 0, policy_version 829629 (0.0010) [2023-12-26 21:24:03,490][105692] Updated weights for policy 0, policy_version 829639 (0.0010) [2023-12-26 21:24:03,610][105620] Updated weights for policy 1, policy_version 829377 (0.0007) [2023-12-26 21:24:03,665][105620] Updated weights for policy 1, policy_version 829387 (0.0008) [2023-12-26 21:24:03,709][105620] Updated weights for policy 1, policy_version 829397 (0.0008) [2023-12-26 21:24:04,260][105692] Updated weights for policy 0, policy_version 829649 (0.0009) [2023-12-26 21:24:04,315][105692] Updated weights for policy 0, policy_version 829659 (0.0009) [2023-12-26 21:24:04,376][105692] Updated weights for policy 0, policy_version 829669 (0.0009) [2023-12-26 21:24:04,409][105620] Updated weights for policy 1, policy_version 829407 (0.0008) [2023-12-26 21:24:04,456][105620] Updated weights for policy 1, policy_version 829417 (0.0009) [2023-12-26 21:24:04,503][105620] Updated weights for policy 1, policy_version 829427 (0.0008) [2023-12-26 21:24:05,060][105692] Updated weights for policy 0, policy_version 829679 (0.0006) [2023-12-26 21:24:05,122][105692] Updated weights for policy 0, policy_version 829689 (0.0005) [2023-12-26 21:24:05,164][105620] Updated weights for policy 1, policy_version 829437 (0.0007) [2023-12-26 21:24:05,180][105692] Updated weights for policy 0, policy_version 829699 (0.0007) [2023-12-26 21:24:05,230][105620] Updated weights for policy 1, policy_version 829447 (0.0005) [2023-12-26 21:24:05,292][105620] Updated weights for policy 1, policy_version 829457 (0.0005) [2023-12-26 21:24:05,848][105692] Updated weights for policy 0, policy_version 829709 (0.0009) [2023-12-26 21:24:05,903][105692] Updated weights for policy 0, policy_version 829719 (0.0010) [2023-12-26 21:24:05,962][105692] Updated weights for policy 0, policy_version 829729 (0.0011) [2023-12-26 21:24:05,966][105620] Updated weights for policy 1, policy_version 829467 (0.0009) [2023-12-26 21:24:06,027][105620] Updated weights for policy 1, policy_version 829477 (0.0010) [2023-12-26 21:24:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 424812544. Throughput: 0: 9831.6, 1: 9767.6. Samples: 424800700. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 21:24:06,063][104569] Avg episode reward: [(0, '8813.269'), (1, '9080.889')] [2023-12-26 21:24:06,082][105620] Updated weights for policy 1, policy_version 829487 (0.0008) [2023-12-26 21:24:06,727][105692] Updated weights for policy 0, policy_version 829739 (0.0009) [2023-12-26 21:24:06,770][105620] Updated weights for policy 1, policy_version 829497 (0.0008) [2023-12-26 21:24:06,781][105692] Updated weights for policy 0, policy_version 829749 (0.0005) [2023-12-26 21:24:06,826][105620] Updated weights for policy 1, policy_version 829507 (0.0008) [2023-12-26 21:24:06,835][105692] Updated weights for policy 0, policy_version 829759 (0.0006) [2023-12-26 21:24:06,887][105620] Updated weights for policy 1, policy_version 829517 (0.0007) [2023-12-26 21:24:06,945][105620] Updated weights for policy 1, policy_version 829527 (0.0006) [2023-12-26 21:24:07,431][105692] Updated weights for policy 0, policy_version 829769 (0.0006) [2023-12-26 21:24:07,494][105692] Updated weights for policy 0, policy_version 829779 (0.0010) [2023-12-26 21:24:07,558][105692] Updated weights for policy 0, policy_version 829789 (0.0010) [2023-12-26 21:24:07,593][105620] Updated weights for policy 1, policy_version 829537 (0.0007) [2023-12-26 21:24:07,617][105692] Updated weights for policy 0, policy_version 829799 (0.0011) [2023-12-26 21:24:07,641][105620] Updated weights for policy 1, policy_version 829547 (0.0008) [2023-12-26 21:24:07,690][105620] Updated weights for policy 1, policy_version 829557 (0.0008) [2023-12-26 21:24:08,314][105692] Updated weights for policy 0, policy_version 829809 (0.0006) [2023-12-26 21:24:08,380][105692] Updated weights for policy 0, policy_version 829819 (0.0011) [2023-12-26 21:24:08,445][105692] Updated weights for policy 0, policy_version 829829 (0.0010) [2023-12-26 21:24:08,481][105620] Updated weights for policy 1, policy_version 829567 (0.0008) [2023-12-26 21:24:08,537][105620] Updated weights for policy 1, policy_version 829577 (0.0008) [2023-12-26 21:24:08,589][105620] Updated weights for policy 1, policy_version 829587 (0.0008) [2023-12-26 21:24:09,140][105692] Updated weights for policy 0, policy_version 829839 (0.0011) [2023-12-26 21:24:09,199][105692] Updated weights for policy 0, policy_version 829849 (0.0011) [2023-12-26 21:24:09,250][105620] Updated weights for policy 1, policy_version 829597 (0.0009) [2023-12-26 21:24:09,258][105692] Updated weights for policy 0, policy_version 829859 (0.0008) [2023-12-26 21:24:09,320][105620] Updated weights for policy 1, policy_version 829607 (0.0008) [2023-12-26 21:24:09,386][105620] Updated weights for policy 1, policy_version 829617 (0.0008) [2023-12-26 21:24:10,050][105692] Updated weights for policy 0, policy_version 829869 (0.0009) [2023-12-26 21:24:10,120][105692] Updated weights for policy 0, policy_version 829879 (0.0011) [2023-12-26 21:24:10,185][105620] Updated weights for policy 1, policy_version 829627 (0.0007) [2023-12-26 21:24:10,186][105692] Updated weights for policy 0, policy_version 829889 (0.0011) [2023-12-26 21:24:10,246][105620] Updated weights for policy 1, policy_version 829637 (0.0007) [2023-12-26 21:24:10,308][105620] Updated weights for policy 1, policy_version 829647 (0.0008) [2023-12-26 21:24:10,921][105692] Updated weights for policy 0, policy_version 829899 (0.0011) [2023-12-26 21:24:10,983][105692] Updated weights for policy 0, policy_version 829909 (0.0010) [2023-12-26 21:24:11,035][105620] Updated weights for policy 1, policy_version 829657 (0.0008) [2023-12-26 21:24:11,053][105692] Updated weights for policy 0, policy_version 829919 (0.0010) [2023-12-26 21:24:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 424902656. Throughput: 0: 9852.0, 1: 9824.2. Samples: 424918904. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 21:24:11,063][104569] Avg episode reward: [(0, '8717.189'), (1, '9029.847')] [2023-12-26 21:24:11,107][105620] Updated weights for policy 1, policy_version 829667 (0.0006) [2023-12-26 21:24:11,177][105620] Updated weights for policy 1, policy_version 829677 (0.0008) [2023-12-26 21:24:11,251][105620] Updated weights for policy 1, policy_version 829687 (0.0008) [2023-12-26 21:24:11,778][105692] Updated weights for policy 0, policy_version 829929 (0.0007) [2023-12-26 21:24:11,847][105692] Updated weights for policy 0, policy_version 829939 (0.0006) [2023-12-26 21:24:11,908][105692] Updated weights for policy 0, policy_version 829949 (0.0007) [2023-12-26 21:24:11,963][105620] Updated weights for policy 1, policy_version 829697 (0.0007) [2023-12-26 21:24:11,970][105692] Updated weights for policy 0, policy_version 829959 (0.0008) [2023-12-26 21:24:12,025][105620] Updated weights for policy 1, policy_version 829707 (0.0009) [2023-12-26 21:24:12,089][105620] Updated weights for policy 1, policy_version 829717 (0.0009) [2023-12-26 21:24:12,627][105692] Updated weights for policy 0, policy_version 829969 (0.0009) [2023-12-26 21:24:12,691][105692] Updated weights for policy 0, policy_version 829979 (0.0009) [2023-12-26 21:24:12,747][105692] Updated weights for policy 0, policy_version 829989 (0.0007) [2023-12-26 21:24:12,854][105620] Updated weights for policy 1, policy_version 829727 (0.0009) [2023-12-26 21:24:12,904][105620] Updated weights for policy 1, policy_version 829737 (0.0008) [2023-12-26 21:24:12,951][105620] Updated weights for policy 1, policy_version 829747 (0.0008) [2023-12-26 21:24:13,572][105620] Updated weights for policy 1, policy_version 829757 (0.0006) [2023-12-26 21:24:13,574][105692] Updated weights for policy 0, policy_version 829999 (0.0008) [2023-12-26 21:24:13,624][105620] Updated weights for policy 1, policy_version 829767 (0.0006) [2023-12-26 21:24:13,637][105692] Updated weights for policy 0, policy_version 830009 (0.0009) [2023-12-26 21:24:13,681][105620] Updated weights for policy 1, policy_version 829777 (0.0005) [2023-12-26 21:24:13,697][105692] Updated weights for policy 0, policy_version 830019 (0.0008) [2023-12-26 21:24:14,283][105620] Updated weights for policy 1, policy_version 829787 (0.0006) [2023-12-26 21:24:14,341][105620] Updated weights for policy 1, policy_version 829797 (0.0009) [2023-12-26 21:24:14,403][105620] Updated weights for policy 1, policy_version 829807 (0.0008) [2023-12-26 21:24:14,479][105692] Updated weights for policy 0, policy_version 830029 (0.0007) [2023-12-26 21:24:14,536][105692] Updated weights for policy 0, policy_version 830039 (0.0007) [2023-12-26 21:24:14,593][105692] Updated weights for policy 0, policy_version 830049 (0.0007) [2023-12-26 21:24:15,144][105620] Updated weights for policy 1, policy_version 829817 (0.0009) [2023-12-26 21:24:15,202][105620] Updated weights for policy 1, policy_version 829827 (0.0009) [2023-12-26 21:24:15,267][105620] Updated weights for policy 1, policy_version 829837 (0.0009) [2023-12-26 21:24:15,300][105692] Updated weights for policy 0, policy_version 830059 (0.0008) [2023-12-26 21:24:15,331][105620] Updated weights for policy 1, policy_version 829847 (0.0008) [2023-12-26 21:24:15,363][105692] Updated weights for policy 0, policy_version 830069 (0.0008) [2023-12-26 21:24:15,429][105692] Updated weights for policy 0, policy_version 830079 (0.0010) [2023-12-26 21:24:16,025][105620] Updated weights for policy 1, policy_version 829857 (0.0008) [2023-12-26 21:24:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 425000960. Throughput: 0: 9803.2, 1: 9787.5. Samples: 424977188. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 21:24:16,063][104569] Avg episode reward: [(0, '8640.818'), (1, '9116.624')] [2023-12-26 21:24:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000830088_212533248.pth... [2023-12-26 21:24:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000828936_212238336.pth [2023-12-26 21:24:16,079][105620] Updated weights for policy 1, policy_version 829867 (0.0009) [2023-12-26 21:24:16,144][105620] Updated weights for policy 1, policy_version 829877 (0.0009) [2023-12-26 21:24:16,161][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000829880_212475904.pth... [2023-12-26 21:24:16,165][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000828728_212180992.pth [2023-12-26 21:24:16,199][105692] Updated weights for policy 0, policy_version 830089 (0.0010) [2023-12-26 21:24:16,260][105692] Updated weights for policy 0, policy_version 830099 (0.0009) [2023-12-26 21:24:16,316][105692] Updated weights for policy 0, policy_version 830109 (0.0009) [2023-12-26 21:24:16,371][105692] Updated weights for policy 0, policy_version 830119 (0.0009) [2023-12-26 21:24:16,890][105620] Updated weights for policy 1, policy_version 829887 (0.0006) [2023-12-26 21:24:16,947][105620] Updated weights for policy 1, policy_version 829897 (0.0007) [2023-12-26 21:24:16,998][105620] Updated weights for policy 1, policy_version 829907 (0.0009) [2023-12-26 21:24:17,164][105692] Updated weights for policy 0, policy_version 830129 (0.0009) [2023-12-26 21:24:17,217][105692] Updated weights for policy 0, policy_version 830139 (0.0009) [2023-12-26 21:24:17,278][105692] Updated weights for policy 0, policy_version 830149 (0.0009) [2023-12-26 21:24:17,710][105620] Updated weights for policy 1, policy_version 829917 (0.0009) [2023-12-26 21:24:17,764][105620] Updated weights for policy 1, policy_version 829927 (0.0009) [2023-12-26 21:24:17,818][105620] Updated weights for policy 1, policy_version 829937 (0.0010) [2023-12-26 21:24:17,943][105692] Updated weights for policy 0, policy_version 830159 (0.0009) [2023-12-26 21:24:18,005][105692] Updated weights for policy 0, policy_version 830169 (0.0009) [2023-12-26 21:24:18,065][105692] Updated weights for policy 0, policy_version 830179 (0.0008) [2023-12-26 21:24:18,640][105620] Updated weights for policy 1, policy_version 829947 (0.0009) [2023-12-26 21:24:18,695][105620] Updated weights for policy 1, policy_version 829957 (0.0009) [2023-12-26 21:24:18,750][105620] Updated weights for policy 1, policy_version 829967 (0.0008) [2023-12-26 21:24:18,785][105692] Updated weights for policy 0, policy_version 830189 (0.0007) [2023-12-26 21:24:18,842][105692] Updated weights for policy 0, policy_version 830199 (0.0008) [2023-12-26 21:24:18,900][105692] Updated weights for policy 0, policy_version 830209 (0.0009) [2023-12-26 21:24:19,535][105620] Updated weights for policy 1, policy_version 829977 (0.0007) [2023-12-26 21:24:19,593][105620] Updated weights for policy 1, policy_version 829987 (0.0007) [2023-12-26 21:24:19,647][105620] Updated weights for policy 1, policy_version 829997 (0.0008) [2023-12-26 21:24:19,665][105692] Updated weights for policy 0, policy_version 830219 (0.0009) [2023-12-26 21:24:19,706][105620] Updated weights for policy 1, policy_version 830007 (0.0007) [2023-12-26 21:24:19,713][105692] Updated weights for policy 0, policy_version 830229 (0.0008) [2023-12-26 21:24:19,761][105692] Updated weights for policy 0, policy_version 830239 (0.0008) [2023-12-26 21:24:20,491][105692] Updated weights for policy 0, policy_version 830249 (0.0008) [2023-12-26 21:24:20,512][105620] Updated weights for policy 1, policy_version 830017 (0.0011) [2023-12-26 21:24:20,545][105692] Updated weights for policy 0, policy_version 830259 (0.0011) [2023-12-26 21:24:20,566][105620] Updated weights for policy 1, policy_version 830027 (0.0011) [2023-12-26 21:24:20,611][105692] Updated weights for policy 0, policy_version 830269 (0.0011) [2023-12-26 21:24:20,628][105620] Updated weights for policy 1, policy_version 830037 (0.0010) [2023-12-26 21:24:20,666][105692] Updated weights for policy 0, policy_version 830279 (0.0009) [2023-12-26 21:24:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 425099264. Throughput: 0: 9733.6, 1: 9743.7. Samples: 425089680. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 21:24:21,062][104569] Avg episode reward: [(0, '8917.207'), (1, '9075.934')] [2023-12-26 21:24:21,359][105692] Updated weights for policy 0, policy_version 830289 (0.0009) [2023-12-26 21:24:21,396][105620] Updated weights for policy 1, policy_version 830047 (0.0009) [2023-12-26 21:24:21,428][105692] Updated weights for policy 0, policy_version 830299 (0.0008) [2023-12-26 21:24:21,452][105620] Updated weights for policy 1, policy_version 830057 (0.0008) [2023-12-26 21:24:21,477][105692] Updated weights for policy 0, policy_version 830309 (0.0011) [2023-12-26 21:24:21,511][105620] Updated weights for policy 1, policy_version 830067 (0.0008) [2023-12-26 21:24:22,266][105692] Updated weights for policy 0, policy_version 830319 (0.0008) [2023-12-26 21:24:22,271][105620] Updated weights for policy 1, policy_version 830077 (0.0008) [2023-12-26 21:24:22,329][105692] Updated weights for policy 0, policy_version 830329 (0.0009) [2023-12-26 21:24:22,329][105620] Updated weights for policy 1, policy_version 830087 (0.0006) [2023-12-26 21:24:22,387][105620] Updated weights for policy 1, policy_version 830097 (0.0009) [2023-12-26 21:24:22,389][105692] Updated weights for policy 0, policy_version 830339 (0.0009) [2023-12-26 21:24:23,117][105620] Updated weights for policy 1, policy_version 830107 (0.0008) [2023-12-26 21:24:23,136][105692] Updated weights for policy 0, policy_version 830349 (0.0008) [2023-12-26 21:24:23,180][105620] Updated weights for policy 1, policy_version 830117 (0.0008) [2023-12-26 21:24:23,195][105692] Updated weights for policy 0, policy_version 830359 (0.0006) [2023-12-26 21:24:23,241][105620] Updated weights for policy 1, policy_version 830127 (0.0008) [2023-12-26 21:24:23,258][105692] Updated weights for policy 0, policy_version 830369 (0.0006) [2023-12-26 21:24:23,949][105692] Updated weights for policy 0, policy_version 830379 (0.0007) [2023-12-26 21:24:24,005][105692] Updated weights for policy 0, policy_version 830389 (0.0010) [2023-12-26 21:24:24,016][105620] Updated weights for policy 1, policy_version 830137 (0.0007) [2023-12-26 21:24:24,062][105692] Updated weights for policy 0, policy_version 830399 (0.0009) [2023-12-26 21:24:24,064][105620] Updated weights for policy 1, policy_version 830147 (0.0005) [2023-12-26 21:24:24,117][105620] Updated weights for policy 1, policy_version 830157 (0.0007) [2023-12-26 21:24:24,170][105620] Updated weights for policy 1, policy_version 830167 (0.0008) [2023-12-26 21:24:24,816][105620] Updated weights for policy 1, policy_version 830177 (0.0008) [2023-12-26 21:24:24,856][105692] Updated weights for policy 0, policy_version 830409 (0.0007) [2023-12-26 21:24:24,872][105620] Updated weights for policy 1, policy_version 830188 (0.0008) [2023-12-26 21:24:24,902][105692] Updated weights for policy 0, policy_version 830419 (0.0008) [2023-12-26 21:24:24,917][105620] Updated weights for policy 1, policy_version 830198 (0.0006) [2023-12-26 21:24:24,955][105692] Updated weights for policy 0, policy_version 830429 (0.0007) [2023-12-26 21:24:25,009][105692] Updated weights for policy 0, policy_version 830439 (0.0005) [2023-12-26 21:24:25,553][105692] Updated weights for policy 0, policy_version 830449 (0.0005) [2023-12-26 21:24:25,614][105692] Updated weights for policy 0, policy_version 830459 (0.0005) [2023-12-26 21:24:25,656][105692] Updated weights for policy 0, policy_version 830469 (0.0005) [2023-12-26 21:24:25,779][105620] Updated weights for policy 1, policy_version 830208 (0.0009) [2023-12-26 21:24:25,829][105620] Updated weights for policy 1, policy_version 830218 (0.0009) [2023-12-26 21:24:25,878][105620] Updated weights for policy 1, policy_version 830228 (0.0008) [2023-12-26 21:24:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.9, 300 sec: 19494.2). Total num frames: 425197568. Throughput: 0: 9694.6, 1: 9652.3. Samples: 425203720. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 21:24:26,062][104569] Avg episode reward: [(0, '8817.033'), (1, '9167.806')] [2023-12-26 21:24:26,315][105692] Updated weights for policy 0, policy_version 830479 (0.0006) [2023-12-26 21:24:26,372][105692] Updated weights for policy 0, policy_version 830489 (0.0008) [2023-12-26 21:24:26,429][105692] Updated weights for policy 0, policy_version 830499 (0.0005) [2023-12-26 21:24:26,671][105620] Updated weights for policy 1, policy_version 830238 (0.0009) [2023-12-26 21:24:26,735][105620] Updated weights for policy 1, policy_version 830248 (0.0009) [2023-12-26 21:24:26,795][105620] Updated weights for policy 1, policy_version 830258 (0.0009) [2023-12-26 21:24:27,136][105692] Updated weights for policy 0, policy_version 830509 (0.0009) [2023-12-26 21:24:27,190][105692] Updated weights for policy 0, policy_version 830519 (0.0009) [2023-12-26 21:24:27,237][105692] Updated weights for policy 0, policy_version 830529 (0.0009) [2023-12-26 21:24:27,532][105620] Updated weights for policy 1, policy_version 830268 (0.0008) [2023-12-26 21:24:27,579][105620] Updated weights for policy 1, policy_version 830278 (0.0009) [2023-12-26 21:24:27,637][105620] Updated weights for policy 1, policy_version 830288 (0.0009) [2023-12-26 21:24:27,966][105692] Updated weights for policy 0, policy_version 830539 (0.0009) [2023-12-26 21:24:28,026][105692] Updated weights for policy 0, policy_version 830549 (0.0009) [2023-12-26 21:24:28,076][105692] Updated weights for policy 0, policy_version 830559 (0.0009) [2023-12-26 21:24:28,401][105620] Updated weights for policy 1, policy_version 830298 (0.0009) [2023-12-26 21:24:28,467][105620] Updated weights for policy 1, policy_version 830308 (0.0005) [2023-12-26 21:24:28,519][105620] Updated weights for policy 1, policy_version 830318 (0.0006) [2023-12-26 21:24:28,570][105620] Updated weights for policy 1, policy_version 830328 (0.0010) [2023-12-26 21:24:28,796][105692] Updated weights for policy 0, policy_version 830569 (0.0008) [2023-12-26 21:24:28,860][105692] Updated weights for policy 0, policy_version 830579 (0.0008) [2023-12-26 21:24:28,916][105692] Updated weights for policy 0, policy_version 830589 (0.0008) [2023-12-26 21:24:28,968][105692] Updated weights for policy 0, policy_version 830599 (0.0009) [2023-12-26 21:24:29,332][105620] Updated weights for policy 1, policy_version 830338 (0.0010) [2023-12-26 21:24:29,393][105620] Updated weights for policy 1, policy_version 830348 (0.0009) [2023-12-26 21:24:29,440][105620] Updated weights for policy 1, policy_version 830358 (0.0009) [2023-12-26 21:24:29,622][105692] Updated weights for policy 0, policy_version 830609 (0.0008) [2023-12-26 21:24:29,682][105692] Updated weights for policy 0, policy_version 830619 (0.0009) [2023-12-26 21:24:29,746][105692] Updated weights for policy 0, policy_version 830629 (0.0009) [2023-12-26 21:24:30,223][105620] Updated weights for policy 1, policy_version 830368 (0.0009) [2023-12-26 21:24:30,272][105620] Updated weights for policy 1, policy_version 830378 (0.0008) [2023-12-26 21:24:30,321][105620] Updated weights for policy 1, policy_version 830388 (0.0008) [2023-12-26 21:24:30,495][105692] Updated weights for policy 0, policy_version 830639 (0.0008) [2023-12-26 21:24:30,543][105692] Updated weights for policy 0, policy_version 830649 (0.0009) [2023-12-26 21:24:30,602][105692] Updated weights for policy 0, policy_version 830659 (0.0009) [2023-12-26 21:24:31,062][104569] Fps is (10 sec: 18840.4, 60 sec: 19251.0, 300 sec: 19466.4). Total num frames: 425287680. Throughput: 0: 9742.9, 1: 9643.3. Samples: 425261144. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 21:24:31,063][104569] Avg episode reward: [(0, '8904.107'), (1, '9259.208')] [2023-12-26 21:24:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000830664_212680704.pth... [2023-12-26 21:24:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000829544_212393984.pth [2023-12-26 21:24:31,081][105620] Updated weights for policy 1, policy_version 830398 (0.0009) [2023-12-26 21:24:31,147][105620] Updated weights for policy 1, policy_version 830408 (0.0010) [2023-12-26 21:24:31,209][105620] Updated weights for policy 1, policy_version 830418 (0.0010) [2023-12-26 21:24:31,245][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000830424_212615168.pth... [2023-12-26 21:24:31,249][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000829304_212328448.pth [2023-12-26 21:24:31,260][105692] Updated weights for policy 0, policy_version 830669 (0.0009) [2023-12-26 21:24:31,320][105692] Updated weights for policy 0, policy_version 830679 (0.0008) [2023-12-26 21:24:31,386][105692] Updated weights for policy 0, policy_version 830689 (0.0009) [2023-12-26 21:24:31,939][105620] Updated weights for policy 1, policy_version 830428 (0.0010) [2023-12-26 21:24:32,005][105620] Updated weights for policy 1, policy_version 830438 (0.0010) [2023-12-26 21:24:32,063][105620] Updated weights for policy 1, policy_version 830448 (0.0010) [2023-12-26 21:24:32,111][105692] Updated weights for policy 0, policy_version 830699 (0.0008) [2023-12-26 21:24:32,166][105692] Updated weights for policy 0, policy_version 830709 (0.0008) [2023-12-26 21:24:32,221][105692] Updated weights for policy 0, policy_version 830719 (0.0008) [2023-12-26 21:24:32,761][105620] Updated weights for policy 1, policy_version 830458 (0.0009) [2023-12-26 21:24:32,829][105620] Updated weights for policy 1, policy_version 830468 (0.0005) [2023-12-26 21:24:32,849][105692] Updated weights for policy 0, policy_version 830729 (0.0008) [2023-12-26 21:24:32,890][105620] Updated weights for policy 1, policy_version 830478 (0.0005) [2023-12-26 21:24:32,910][105692] Updated weights for policy 0, policy_version 830739 (0.0008) [2023-12-26 21:24:32,949][105620] Updated weights for policy 1, policy_version 830488 (0.0007) [2023-12-26 21:24:32,959][105692] Updated weights for policy 0, policy_version 830749 (0.0006) [2023-12-26 21:24:33,012][105692] Updated weights for policy 0, policy_version 830759 (0.0005) [2023-12-26 21:24:33,525][105620] Updated weights for policy 1, policy_version 830498 (0.0010) [2023-12-26 21:24:33,575][105620] Updated weights for policy 1, policy_version 830508 (0.0010) [2023-12-26 21:24:33,626][105620] Updated weights for policy 1, policy_version 830518 (0.0010) [2023-12-26 21:24:33,654][105692] Updated weights for policy 0, policy_version 830769 (0.0010) [2023-12-26 21:24:33,709][105692] Updated weights for policy 0, policy_version 830779 (0.0006) [2023-12-26 21:24:33,764][105692] Updated weights for policy 0, policy_version 830789 (0.0006) [2023-12-26 21:24:34,304][105620] Updated weights for policy 1, policy_version 830528 (0.0006) [2023-12-26 21:24:34,329][105692] Updated weights for policy 0, policy_version 830799 (0.0007) [2023-12-26 21:24:34,361][105620] Updated weights for policy 1, policy_version 830538 (0.0006) [2023-12-26 21:24:34,390][105692] Updated weights for policy 0, policy_version 830809 (0.0005) [2023-12-26 21:24:34,427][105620] Updated weights for policy 1, policy_version 830548 (0.0008) [2023-12-26 21:24:34,451][105692] Updated weights for policy 0, policy_version 830819 (0.0007) [2023-12-26 21:24:35,026][105620] Updated weights for policy 1, policy_version 830558 (0.0006) [2023-12-26 21:24:35,082][105620] Updated weights for policy 1, policy_version 830568 (0.0005) [2023-12-26 21:24:35,138][105620] Updated weights for policy 1, policy_version 830578 (0.0005) [2023-12-26 21:24:35,173][105692] Updated weights for policy 0, policy_version 830829 (0.0009) [2023-12-26 21:24:35,226][105692] Updated weights for policy 0, policy_version 830839 (0.0009) [2023-12-26 21:24:35,278][105692] Updated weights for policy 0, policy_version 830849 (0.0010) [2023-12-26 21:24:35,671][105620] Updated weights for policy 1, policy_version 830588 (0.0007) [2023-12-26 21:24:35,723][105620] Updated weights for policy 1, policy_version 830598 (0.0008) [2023-12-26 21:24:35,774][105620] Updated weights for policy 1, policy_version 830608 (0.0009) [2023-12-26 21:24:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.4, 300 sec: 19494.2). Total num frames: 425394176. Throughput: 0: 9816.6, 1: 9626.0. Samples: 425383656. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 21:24:36,063][104569] Avg episode reward: [(0, '9082.879'), (1, '9168.790')] [2023-12-26 21:24:36,089][105692] Updated weights for policy 0, policy_version 830860 (0.0010) [2023-12-26 21:24:36,156][105692] Updated weights for policy 0, policy_version 830870 (0.0011) [2023-12-26 21:24:36,215][105692] Updated weights for policy 0, policy_version 830880 (0.0011) [2023-12-26 21:24:36,562][105620] Updated weights for policy 1, policy_version 830618 (0.0008) [2023-12-26 21:24:36,622][105620] Updated weights for policy 1, policy_version 830628 (0.0008) [2023-12-26 21:24:36,699][105620] Updated weights for policy 1, policy_version 830638 (0.0007) [2023-12-26 21:24:36,764][105620] Updated weights for policy 1, policy_version 830648 (0.0008) [2023-12-26 21:24:36,954][105692] Updated weights for policy 0, policy_version 830890 (0.0011) [2023-12-26 21:24:36,998][105692] Updated weights for policy 0, policy_version 830900 (0.0010) [2023-12-26 21:24:37,048][105692] Updated weights for policy 0, policy_version 830910 (0.0010) [2023-12-26 21:24:37,094][105692] Updated weights for policy 0, policy_version 830920 (0.0008) [2023-12-26 21:24:37,442][105620] Updated weights for policy 1, policy_version 830658 (0.0005) [2023-12-26 21:24:37,512][105620] Updated weights for policy 1, policy_version 830668 (0.0005) [2023-12-26 21:24:37,576][105620] Updated weights for policy 1, policy_version 830678 (0.0006) [2023-12-26 21:24:37,838][105692] Updated weights for policy 0, policy_version 830930 (0.0007) [2023-12-26 21:24:37,889][105692] Updated weights for policy 0, policy_version 830940 (0.0009) [2023-12-26 21:24:37,944][105692] Updated weights for policy 0, policy_version 830950 (0.0009) [2023-12-26 21:24:38,196][105620] Updated weights for policy 1, policy_version 830688 (0.0007) [2023-12-26 21:24:38,243][105620] Updated weights for policy 1, policy_version 830698 (0.0008) [2023-12-26 21:24:38,289][105620] Updated weights for policy 1, policy_version 830708 (0.0008) [2023-12-26 21:24:38,657][105692] Updated weights for policy 0, policy_version 830960 (0.0010) [2023-12-26 21:24:38,717][105692] Updated weights for policy 0, policy_version 830970 (0.0011) [2023-12-26 21:24:38,776][105692] Updated weights for policy 0, policy_version 830980 (0.0010) [2023-12-26 21:24:39,069][105620] Updated weights for policy 1, policy_version 830718 (0.0009) [2023-12-26 21:24:39,121][105620] Updated weights for policy 1, policy_version 830728 (0.0008) [2023-12-26 21:24:39,180][105620] Updated weights for policy 1, policy_version 830738 (0.0006) [2023-12-26 21:24:39,550][105692] Updated weights for policy 0, policy_version 830990 (0.0011) [2023-12-26 21:24:39,615][105692] Updated weights for policy 0, policy_version 831000 (0.0011) [2023-12-26 21:24:39,681][105692] Updated weights for policy 0, policy_version 831010 (0.0011) [2023-12-26 21:24:39,975][105620] Updated weights for policy 1, policy_version 830748 (0.0009) [2023-12-26 21:24:40,041][105620] Updated weights for policy 1, policy_version 830758 (0.0006) [2023-12-26 21:24:40,105][105620] Updated weights for policy 1, policy_version 830768 (0.0006) [2023-12-26 21:24:40,379][105692] Updated weights for policy 0, policy_version 831020 (0.0011) [2023-12-26 21:24:40,439][105692] Updated weights for policy 0, policy_version 831030 (0.0011) [2023-12-26 21:24:40,498][105692] Updated weights for policy 0, policy_version 831040 (0.0011) [2023-12-26 21:24:40,869][105620] Updated weights for policy 1, policy_version 830778 (0.0009) [2023-12-26 21:24:40,930][105620] Updated weights for policy 1, policy_version 830788 (0.0009) [2023-12-26 21:24:40,983][105620] Updated weights for policy 1, policy_version 830798 (0.0009) [2023-12-26 21:24:41,040][105620] Updated weights for policy 1, policy_version 830808 (0.0009) [2023-12-26 21:24:41,062][104569] Fps is (10 sec: 20481.5, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 425492480. Throughput: 0: 9778.8, 1: 9632.9. Samples: 425498384. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 21:24:41,062][104569] Avg episode reward: [(0, '8631.207'), (1, '9168.906')] [2023-12-26 21:24:41,139][105692] Updated weights for policy 0, policy_version 831050 (0.0010) [2023-12-26 21:24:41,201][105692] Updated weights for policy 0, policy_version 831060 (0.0009) [2023-12-26 21:24:41,261][105692] Updated weights for policy 0, policy_version 831070 (0.0009) [2023-12-26 21:24:41,325][105692] Updated weights for policy 0, policy_version 831080 (0.0009) [2023-12-26 21:24:41,872][105620] Updated weights for policy 1, policy_version 830818 (0.0008) [2023-12-26 21:24:41,924][105620] Updated weights for policy 1, policy_version 830828 (0.0009) [2023-12-26 21:24:41,982][105620] Updated weights for policy 1, policy_version 830838 (0.0010) [2023-12-26 21:24:42,105][105692] Updated weights for policy 0, policy_version 831090 (0.0007) [2023-12-26 21:24:42,169][105692] Updated weights for policy 0, policy_version 831100 (0.0011) [2023-12-26 21:24:42,232][105692] Updated weights for policy 0, policy_version 831110 (0.0010) [2023-12-26 21:24:42,725][105620] Updated weights for policy 1, policy_version 830848 (0.0008) [2023-12-26 21:24:42,785][105620] Updated weights for policy 1, policy_version 830858 (0.0008) [2023-12-26 21:24:42,830][105620] Updated weights for policy 1, policy_version 830868 (0.0008) [2023-12-26 21:24:42,975][105692] Updated weights for policy 0, policy_version 831120 (0.0010) [2023-12-26 21:24:43,033][105692] Updated weights for policy 0, policy_version 831130 (0.0010) [2023-12-26 21:24:43,081][105692] Updated weights for policy 0, policy_version 831140 (0.0010) [2023-12-26 21:24:43,549][105620] Updated weights for policy 1, policy_version 830878 (0.0007) [2023-12-26 21:24:43,603][105620] Updated weights for policy 1, policy_version 830888 (0.0006) [2023-12-26 21:24:43,658][105620] Updated weights for policy 1, policy_version 830898 (0.0006) [2023-12-26 21:24:43,806][105692] Updated weights for policy 0, policy_version 831150 (0.0011) [2023-12-26 21:24:43,868][105692] Updated weights for policy 0, policy_version 831160 (0.0010) [2023-12-26 21:24:43,929][105692] Updated weights for policy 0, policy_version 831170 (0.0010) [2023-12-26 21:24:44,221][105620] Updated weights for policy 1, policy_version 830908 (0.0005) [2023-12-26 21:24:44,288][105620] Updated weights for policy 1, policy_version 830918 (0.0006) [2023-12-26 21:24:44,345][105620] Updated weights for policy 1, policy_version 830928 (0.0008) [2023-12-26 21:24:44,597][105692] Updated weights for policy 0, policy_version 831180 (0.0010) [2023-12-26 21:24:44,660][105692] Updated weights for policy 0, policy_version 831190 (0.0010) [2023-12-26 21:24:44,712][105692] Updated weights for policy 0, policy_version 831200 (0.0010) [2023-12-26 21:24:44,968][105620] Updated weights for policy 1, policy_version 830938 (0.0008) [2023-12-26 21:24:45,036][105620] Updated weights for policy 1, policy_version 830948 (0.0006) [2023-12-26 21:24:45,101][105620] Updated weights for policy 1, policy_version 830958 (0.0005) [2023-12-26 21:24:45,172][105620] Updated weights for policy 1, policy_version 830968 (0.0007) [2023-12-26 21:24:45,473][105692] Updated weights for policy 0, policy_version 831210 (0.0011) [2023-12-26 21:24:45,527][105692] Updated weights for policy 0, policy_version 831220 (0.0011) [2023-12-26 21:24:45,583][105692] Updated weights for policy 0, policy_version 831230 (0.0011) [2023-12-26 21:24:45,647][105692] Updated weights for policy 0, policy_version 831240 (0.0011) [2023-12-26 21:24:45,797][105620] Updated weights for policy 1, policy_version 830978 (0.0010) [2023-12-26 21:24:45,862][105620] Updated weights for policy 1, policy_version 830988 (0.0010) [2023-12-26 21:24:45,927][105620] Updated weights for policy 1, policy_version 830998 (0.0010) [2023-12-26 21:24:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 425590784. Throughput: 0: 9743.5, 1: 9657.5. Samples: 425557156. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 21:24:46,063][104569] Avg episode reward: [(0, '8900.041'), (1, '9077.134')] [2023-12-26 21:24:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000831240_212828160.pth... [2023-12-26 21:24:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000831000_212762624.pth... [2023-12-26 21:24:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000830088_212533248.pth [2023-12-26 21:24:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000829880_212475904.pth [2023-12-26 21:24:46,403][105692] Updated weights for policy 0, policy_version 831250 (0.0011) [2023-12-26 21:24:46,459][105692] Updated weights for policy 0, policy_version 831260 (0.0010) [2023-12-26 21:24:46,518][105692] Updated weights for policy 0, policy_version 831270 (0.0011) [2023-12-26 21:24:46,593][105620] Updated weights for policy 1, policy_version 831008 (0.0006) [2023-12-26 21:24:46,661][105620] Updated weights for policy 1, policy_version 831018 (0.0006) [2023-12-26 21:24:46,724][105620] Updated weights for policy 1, policy_version 831028 (0.0008) [2023-12-26 21:24:47,269][105692] Updated weights for policy 0, policy_version 831280 (0.0010) [2023-12-26 21:24:47,334][105692] Updated weights for policy 0, policy_version 831290 (0.0010) [2023-12-26 21:24:47,374][105620] Updated weights for policy 1, policy_version 831038 (0.0008) [2023-12-26 21:24:47,392][105692] Updated weights for policy 0, policy_version 831300 (0.0010) [2023-12-26 21:24:47,431][105620] Updated weights for policy 1, policy_version 831048 (0.0006) [2023-12-26 21:24:47,486][105620] Updated weights for policy 1, policy_version 831058 (0.0008) [2023-12-26 21:24:48,115][105692] Updated weights for policy 0, policy_version 831310 (0.0010) [2023-12-26 21:24:48,170][105692] Updated weights for policy 0, policy_version 831320 (0.0010) [2023-12-26 21:24:48,180][105620] Updated weights for policy 1, policy_version 831068 (0.0008) [2023-12-26 21:24:48,218][105692] Updated weights for policy 0, policy_version 831330 (0.0010) [2023-12-26 21:24:48,224][105620] Updated weights for policy 1, policy_version 831078 (0.0005) [2023-12-26 21:24:48,274][105620] Updated weights for policy 1, policy_version 831088 (0.0007) [2023-12-26 21:24:48,975][105692] Updated weights for policy 0, policy_version 831340 (0.0011) [2023-12-26 21:24:49,027][105692] Updated weights for policy 0, policy_version 831350 (0.0010) [2023-12-26 21:24:49,031][105620] Updated weights for policy 1, policy_version 831098 (0.0007) [2023-12-26 21:24:49,076][105692] Updated weights for policy 0, policy_version 831360 (0.0010) [2023-12-26 21:24:49,097][105620] Updated weights for policy 1, policy_version 831108 (0.0009) [2023-12-26 21:24:49,164][105620] Updated weights for policy 1, policy_version 831118 (0.0008) [2023-12-26 21:24:49,234][105620] Updated weights for policy 1, policy_version 831128 (0.0007) [2023-12-26 21:24:49,877][105692] Updated weights for policy 0, policy_version 831370 (0.0010) [2023-12-26 21:24:49,921][105620] Updated weights for policy 1, policy_version 831138 (0.0007) [2023-12-26 21:24:49,944][105692] Updated weights for policy 0, policy_version 831380 (0.0011) [2023-12-26 21:24:49,983][105620] Updated weights for policy 1, policy_version 831148 (0.0007) [2023-12-26 21:24:50,014][105692] Updated weights for policy 0, policy_version 831390 (0.0011) [2023-12-26 21:24:50,038][105620] Updated weights for policy 1, policy_version 831158 (0.0007) [2023-12-26 21:24:50,073][105692] Updated weights for policy 0, policy_version 831400 (0.0011) [2023-12-26 21:24:50,655][105692] Updated weights for policy 0, policy_version 831410 (0.0005) [2023-12-26 21:24:50,726][105692] Updated weights for policy 0, policy_version 831420 (0.0009) [2023-12-26 21:24:50,778][105692] Updated weights for policy 0, policy_version 831430 (0.0011) [2023-12-26 21:24:50,834][105620] Updated weights for policy 1, policy_version 831168 (0.0010) [2023-12-26 21:24:50,893][105620] Updated weights for policy 1, policy_version 831178 (0.0010) [2023-12-26 21:24:50,950][105620] Updated weights for policy 1, policy_version 831188 (0.0009) [2023-12-26 21:24:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 425689088. Throughput: 0: 9698.2, 1: 9719.7. Samples: 425674500. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-26 21:24:51,062][104569] Avg episode reward: [(0, '9083.101'), (1, '9077.284')] [2023-12-26 21:24:51,402][105692] Updated weights for policy 0, policy_version 831440 (0.0009) [2023-12-26 21:24:51,462][105692] Updated weights for policy 0, policy_version 831450 (0.0008) [2023-12-26 21:24:51,523][105692] Updated weights for policy 0, policy_version 831460 (0.0008) [2023-12-26 21:24:51,745][105620] Updated weights for policy 1, policy_version 831198 (0.0010) [2023-12-26 21:24:51,799][105620] Updated weights for policy 1, policy_version 831208 (0.0009) [2023-12-26 21:24:51,856][105620] Updated weights for policy 1, policy_version 831218 (0.0009) [2023-12-26 21:24:52,216][105692] Updated weights for policy 0, policy_version 831470 (0.0008) [2023-12-26 21:24:52,273][105692] Updated weights for policy 0, policy_version 831480 (0.0007) [2023-12-26 21:24:52,332][105692] Updated weights for policy 0, policy_version 831490 (0.0009) [2023-12-26 21:24:52,702][105620] Updated weights for policy 1, policy_version 831228 (0.0009) [2023-12-26 21:24:52,777][105620] Updated weights for policy 1, policy_version 831238 (0.0010) [2023-12-26 21:24:52,844][105620] Updated weights for policy 1, policy_version 831248 (0.0009) [2023-12-26 21:24:52,925][105692] Updated weights for policy 0, policy_version 831500 (0.0007) [2023-12-26 21:24:52,992][105692] Updated weights for policy 0, policy_version 831510 (0.0005) [2023-12-26 21:24:53,061][105692] Updated weights for policy 0, policy_version 831520 (0.0009) [2023-12-26 21:24:53,492][105620] Updated weights for policy 1, policy_version 831258 (0.0009) [2023-12-26 21:24:53,549][105620] Updated weights for policy 1, policy_version 831268 (0.0010) [2023-12-26 21:24:53,607][105620] Updated weights for policy 1, policy_version 831278 (0.0009) [2023-12-26 21:24:53,616][105692] Updated weights for policy 0, policy_version 831530 (0.0009) [2023-12-26 21:24:53,663][105692] Updated weights for policy 0, policy_version 831540 (0.0008) [2023-12-26 21:24:53,678][105620] Updated weights for policy 1, policy_version 831288 (0.0008) [2023-12-26 21:24:53,712][105692] Updated weights for policy 0, policy_version 831550 (0.0008) [2023-12-26 21:24:54,357][105692] Updated weights for policy 0, policy_version 831561 (0.0007) [2023-12-26 21:24:54,395][105620] Updated weights for policy 1, policy_version 831298 (0.0009) [2023-12-26 21:24:54,412][105692] Updated weights for policy 0, policy_version 831571 (0.0010) [2023-12-26 21:24:54,447][105620] Updated weights for policy 1, policy_version 831308 (0.0010) [2023-12-26 21:24:54,467][105692] Updated weights for policy 0, policy_version 831581 (0.0010) [2023-12-26 21:24:54,508][105620] Updated weights for policy 1, policy_version 831318 (0.0008) [2023-12-26 21:24:54,519][105692] Updated weights for policy 0, policy_version 831591 (0.0010) [2023-12-26 21:24:55,134][105620] Updated weights for policy 1, policy_version 831328 (0.0006) [2023-12-26 21:24:55,190][105620] Updated weights for policy 1, policy_version 831338 (0.0011) [2023-12-26 21:24:55,237][105620] Updated weights for policy 1, policy_version 831348 (0.0007) [2023-12-26 21:24:55,283][105692] Updated weights for policy 0, policy_version 831601 (0.0011) [2023-12-26 21:24:55,340][105692] Updated weights for policy 0, policy_version 831611 (0.0010) [2023-12-26 21:24:55,394][105692] Updated weights for policy 0, policy_version 831621 (0.0010) [2023-12-26 21:24:55,861][105620] Updated weights for policy 1, policy_version 831358 (0.0006) [2023-12-26 21:24:55,911][105620] Updated weights for policy 1, policy_version 831368 (0.0005) [2023-12-26 21:24:55,959][105620] Updated weights for policy 1, policy_version 831378 (0.0005) [2023-12-26 21:24:56,063][104569] Fps is (10 sec: 19659.5, 60 sec: 19524.0, 300 sec: 19521.9). Total num frames: 425787392. Throughput: 0: 9789.5, 1: 9690.6. Samples: 425795524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:24:56,063][104569] Avg episode reward: [(0, '8992.896'), (1, '9260.604')] [2023-12-26 21:24:56,136][105692] Updated weights for policy 0, policy_version 831631 (0.0009) [2023-12-26 21:24:56,194][105692] Updated weights for policy 0, policy_version 831641 (0.0010) [2023-12-26 21:24:56,252][105692] Updated weights for policy 0, policy_version 831651 (0.0010) [2023-12-26 21:24:56,635][105620] Updated weights for policy 1, policy_version 831388 (0.0007) [2023-12-26 21:24:56,689][105620] Updated weights for policy 1, policy_version 831398 (0.0008) [2023-12-26 21:24:56,754][105620] Updated weights for policy 1, policy_version 831408 (0.0007) [2023-12-26 21:24:56,987][105692] Updated weights for policy 0, policy_version 831661 (0.0010) [2023-12-26 21:24:57,034][105692] Updated weights for policy 0, policy_version 831671 (0.0010) [2023-12-26 21:24:57,085][105692] Updated weights for policy 0, policy_version 831681 (0.0010) [2023-12-26 21:24:57,470][105620] Updated weights for policy 1, policy_version 831418 (0.0008) [2023-12-26 21:24:57,521][105620] Updated weights for policy 1, policy_version 831428 (0.0010) [2023-12-26 21:24:57,582][105620] Updated weights for policy 1, policy_version 831438 (0.0010) [2023-12-26 21:24:57,640][105620] Updated weights for policy 1, policy_version 831448 (0.0010) [2023-12-26 21:24:57,773][105692] Updated weights for policy 0, policy_version 831691 (0.0009) [2023-12-26 21:24:57,827][105692] Updated weights for policy 0, policy_version 831701 (0.0010) [2023-12-26 21:24:57,887][105692] Updated weights for policy 0, policy_version 831711 (0.0010) [2023-12-26 21:24:58,254][105620] Updated weights for policy 1, policy_version 831458 (0.0011) [2023-12-26 21:24:58,327][105620] Updated weights for policy 1, policy_version 831469 (0.0010) [2023-12-26 21:24:58,391][105620] Updated weights for policy 1, policy_version 831479 (0.0007) [2023-12-26 21:24:58,644][105692] Updated weights for policy 0, policy_version 831721 (0.0010) [2023-12-26 21:24:58,711][105692] Updated weights for policy 0, policy_version 831731 (0.0010) [2023-12-26 21:24:58,780][105692] Updated weights for policy 0, policy_version 831741 (0.0009) [2023-12-26 21:24:58,848][105692] Updated weights for policy 0, policy_version 831751 (0.0008) [2023-12-26 21:24:59,151][105620] Updated weights for policy 1, policy_version 831489 (0.0007) [2023-12-26 21:24:59,217][105620] Updated weights for policy 1, policy_version 831499 (0.0007) [2023-12-26 21:24:59,283][105620] Updated weights for policy 1, policy_version 831509 (0.0008) [2023-12-26 21:24:59,690][105692] Updated weights for policy 0, policy_version 831761 (0.0009) [2023-12-26 21:24:59,755][105692] Updated weights for policy 0, policy_version 831771 (0.0009) [2023-12-26 21:24:59,809][105692] Updated weights for policy 0, policy_version 831781 (0.0009) [2023-12-26 21:25:00,041][105620] Updated weights for policy 1, policy_version 831519 (0.0009) [2023-12-26 21:25:00,105][105620] Updated weights for policy 1, policy_version 831529 (0.0009) [2023-12-26 21:25:00,167][105620] Updated weights for policy 1, policy_version 831539 (0.0009) [2023-12-26 21:25:00,549][105692] Updated weights for policy 0, policy_version 831791 (0.0007) [2023-12-26 21:25:00,611][105692] Updated weights for policy 0, policy_version 831801 (0.0009) [2023-12-26 21:25:00,670][105692] Updated weights for policy 0, policy_version 831811 (0.0009) [2023-12-26 21:25:00,930][105620] Updated weights for policy 1, policy_version 831549 (0.0009) [2023-12-26 21:25:00,982][105620] Updated weights for policy 1, policy_version 831560 (0.0010) [2023-12-26 21:25:01,044][105620] Updated weights for policy 1, policy_version 831570 (0.0009) [2023-12-26 21:25:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 425877504. Throughput: 0: 9808.0, 1: 9691.2. Samples: 425854652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:25:01,062][104569] Avg episode reward: [(0, '9167.753'), (1, '9077.804')] [2023-12-26 21:25:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000831816_212975616.pth... [2023-12-26 21:25:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000830664_212680704.pth [2023-12-26 21:25:01,074][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000831576_212910080.pth... [2023-12-26 21:25:01,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000830424_212615168.pth [2023-12-26 21:25:01,281][105692] Updated weights for policy 0, policy_version 831821 (0.0008) [2023-12-26 21:25:01,335][105692] Updated weights for policy 0, policy_version 831831 (0.0006) [2023-12-26 21:25:01,405][105692] Updated weights for policy 0, policy_version 831841 (0.0009) [2023-12-26 21:25:01,833][105620] Updated weights for policy 1, policy_version 831580 (0.0007) [2023-12-26 21:25:01,886][105620] Updated weights for policy 1, policy_version 831590 (0.0005) [2023-12-26 21:25:01,946][105620] Updated weights for policy 1, policy_version 831600 (0.0006) [2023-12-26 21:25:02,200][105692] Updated weights for policy 0, policy_version 831851 (0.0010) [2023-12-26 21:25:02,250][105692] Updated weights for policy 0, policy_version 831861 (0.0010) [2023-12-26 21:25:02,297][105692] Updated weights for policy 0, policy_version 831871 (0.0010) [2023-12-26 21:25:02,559][105620] Updated weights for policy 1, policy_version 831610 (0.0006) [2023-12-26 21:25:02,609][105620] Updated weights for policy 1, policy_version 831620 (0.0007) [2023-12-26 21:25:02,668][105620] Updated weights for policy 1, policy_version 831630 (0.0008) [2023-12-26 21:25:02,719][105620] Updated weights for policy 1, policy_version 831640 (0.0008) [2023-12-26 21:25:03,088][105692] Updated weights for policy 0, policy_version 831881 (0.0011) [2023-12-26 21:25:03,138][105692] Updated weights for policy 0, policy_version 831891 (0.0008) [2023-12-26 21:25:03,192][105692] Updated weights for policy 0, policy_version 831901 (0.0008) [2023-12-26 21:25:03,243][105692] Updated weights for policy 0, policy_version 831911 (0.0009) [2023-12-26 21:25:03,461][105620] Updated weights for policy 1, policy_version 831650 (0.0008) [2023-12-26 21:25:03,507][105620] Updated weights for policy 1, policy_version 831660 (0.0009) [2023-12-26 21:25:03,563][105620] Updated weights for policy 1, policy_version 831670 (0.0009) [2023-12-26 21:25:04,003][105692] Updated weights for policy 0, policy_version 831921 (0.0010) [2023-12-26 21:25:04,053][105692] Updated weights for policy 0, policy_version 831931 (0.0009) [2023-12-26 21:25:04,115][105692] Updated weights for policy 0, policy_version 831941 (0.0010) [2023-12-26 21:25:04,259][105620] Updated weights for policy 1, policy_version 831680 (0.0006) [2023-12-26 21:25:04,322][105620] Updated weights for policy 1, policy_version 831690 (0.0009) [2023-12-26 21:25:04,380][105620] Updated weights for policy 1, policy_version 831700 (0.0009) [2023-12-26 21:25:04,918][105692] Updated weights for policy 0, policy_version 831951 (0.0009) [2023-12-26 21:25:04,984][105692] Updated weights for policy 0, policy_version 831961 (0.0009) [2023-12-26 21:25:05,042][105692] Updated weights for policy 0, policy_version 831971 (0.0009) [2023-12-26 21:25:05,093][105620] Updated weights for policy 1, policy_version 831710 (0.0009) [2023-12-26 21:25:05,142][105620] Updated weights for policy 1, policy_version 831720 (0.0008) [2023-12-26 21:25:05,200][105620] Updated weights for policy 1, policy_version 831730 (0.0008) [2023-12-26 21:25:05,662][105692] Updated weights for policy 0, policy_version 831981 (0.0008) [2023-12-26 21:25:05,726][105692] Updated weights for policy 0, policy_version 831991 (0.0008) [2023-12-26 21:25:05,776][105692] Updated weights for policy 0, policy_version 832001 (0.0005) [2023-12-26 21:25:06,008][105620] Updated weights for policy 1, policy_version 831740 (0.0009) [2023-12-26 21:25:06,062][104569] Fps is (10 sec: 18843.2, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 425975808. Throughput: 0: 9773.8, 1: 9730.7. Samples: 425967380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:25:06,063][104569] Avg episode reward: [(0, '9167.445'), (1, '9169.125')] [2023-12-26 21:25:06,074][105620] Updated weights for policy 1, policy_version 831750 (0.0009) [2023-12-26 21:25:06,138][105620] Updated weights for policy 1, policy_version 831760 (0.0008) [2023-12-26 21:25:06,380][105692] Updated weights for policy 0, policy_version 832011 (0.0006) [2023-12-26 21:25:06,447][105692] Updated weights for policy 0, policy_version 832021 (0.0009) [2023-12-26 21:25:06,511][105692] Updated weights for policy 0, policy_version 832031 (0.0009) [2023-12-26 21:25:06,859][105620] Updated weights for policy 1, policy_version 831770 (0.0009) [2023-12-26 21:25:06,915][105620] Updated weights for policy 1, policy_version 831780 (0.0005) [2023-12-26 21:25:06,976][105620] Updated weights for policy 1, policy_version 831790 (0.0005) [2023-12-26 21:25:07,039][105620] Updated weights for policy 1, policy_version 831800 (0.0006) [2023-12-26 21:25:07,160][105692] Updated weights for policy 0, policy_version 832041 (0.0006) [2023-12-26 21:25:07,230][105692] Updated weights for policy 0, policy_version 832051 (0.0007) [2023-12-26 21:25:07,283][105692] Updated weights for policy 0, policy_version 832061 (0.0008) [2023-12-26 21:25:07,337][105692] Updated weights for policy 0, policy_version 832071 (0.0005) [2023-12-26 21:25:07,671][105620] Updated weights for policy 1, policy_version 831810 (0.0009) [2023-12-26 21:25:07,717][105620] Updated weights for policy 1, policy_version 831820 (0.0008) [2023-12-26 21:25:07,767][105620] Updated weights for policy 1, policy_version 831830 (0.0009) [2023-12-26 21:25:08,047][105692] Updated weights for policy 0, policy_version 832081 (0.0008) [2023-12-26 21:25:08,102][105692] Updated weights for policy 0, policy_version 832091 (0.0009) [2023-12-26 21:25:08,157][105692] Updated weights for policy 0, policy_version 832101 (0.0009) [2023-12-26 21:25:08,467][105620] Updated weights for policy 1, policy_version 831840 (0.0009) [2023-12-26 21:25:08,517][105620] Updated weights for policy 1, policy_version 831850 (0.0008) [2023-12-26 21:25:08,569][105620] Updated weights for policy 1, policy_version 831860 (0.0009) [2023-12-26 21:25:08,936][105692] Updated weights for policy 0, policy_version 832111 (0.0006) [2023-12-26 21:25:09,009][105692] Updated weights for policy 0, policy_version 832121 (0.0005) [2023-12-26 21:25:09,077][105692] Updated weights for policy 0, policy_version 832131 (0.0007) [2023-12-26 21:25:09,182][105620] Updated weights for policy 1, policy_version 831870 (0.0009) [2023-12-26 21:25:09,251][105620] Updated weights for policy 1, policy_version 831880 (0.0009) [2023-12-26 21:25:09,317][105620] Updated weights for policy 1, policy_version 831890 (0.0006) [2023-12-26 21:25:09,746][105692] Updated weights for policy 0, policy_version 832141 (0.0009) [2023-12-26 21:25:09,804][105692] Updated weights for policy 0, policy_version 832151 (0.0009) [2023-12-26 21:25:09,870][105692] Updated weights for policy 0, policy_version 832161 (0.0009) [2023-12-26 21:25:10,082][105620] Updated weights for policy 1, policy_version 831900 (0.0008) [2023-12-26 21:25:10,128][105586] KL-divergence is very high: 231.7170 [2023-12-26 21:25:10,132][105620] Updated weights for policy 1, policy_version 831910 (0.0008) [2023-12-26 21:25:10,178][105586] KL-divergence is very high: 352.5286 [2023-12-26 21:25:10,199][105620] Updated weights for policy 1, policy_version 831920 (0.0006) [2023-12-26 21:25:10,231][105586] KL-divergence is very high: 329.5552 [2023-12-26 21:25:10,659][105692] Updated weights for policy 0, policy_version 832171 (0.0009) [2023-12-26 21:25:10,715][105692] Updated weights for policy 0, policy_version 832181 (0.0010) [2023-12-26 21:25:10,769][105692] Updated weights for policy 0, policy_version 832191 (0.0011) [2023-12-26 21:25:10,853][105620] Updated weights for policy 1, policy_version 831930 (0.0006) [2023-12-26 21:25:10,925][105620] Updated weights for policy 1, policy_version 831940 (0.0005) [2023-12-26 21:25:10,985][105620] Updated weights for policy 1, policy_version 831950 (0.0006) [2023-12-26 21:25:11,058][105620] Updated weights for policy 1, policy_version 831960 (0.0006) [2023-12-26 21:25:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 426082304. Throughput: 0: 9788.5, 1: 9821.7. Samples: 426086180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:25:11,063][104569] Avg episode reward: [(0, '9258.258'), (1, '9259.862')] [2023-12-26 21:25:11,579][105692] Updated weights for policy 0, policy_version 832201 (0.0009) [2023-12-26 21:25:11,646][105692] Updated weights for policy 0, policy_version 832211 (0.0008) [2023-12-26 21:25:11,711][105692] Updated weights for policy 0, policy_version 832221 (0.0006) [2023-12-26 21:25:11,770][105620] Updated weights for policy 1, policy_version 831970 (0.0009) [2023-12-26 21:25:11,776][105692] Updated weights for policy 0, policy_version 832231 (0.0008) [2023-12-26 21:25:11,837][105620] Updated weights for policy 1, policy_version 831980 (0.0006) [2023-12-26 21:25:11,903][105620] Updated weights for policy 1, policy_version 831990 (0.0005) [2023-12-26 21:25:12,543][105620] Updated weights for policy 1, policy_version 832000 (0.0007) [2023-12-26 21:25:12,553][105692] Updated weights for policy 0, policy_version 832241 (0.0005) [2023-12-26 21:25:12,602][105620] Updated weights for policy 1, policy_version 832010 (0.0009) [2023-12-26 21:25:12,621][105692] Updated weights for policy 0, policy_version 832251 (0.0006) [2023-12-26 21:25:12,663][105620] Updated weights for policy 1, policy_version 832020 (0.0007) [2023-12-26 21:25:12,681][105692] Updated weights for policy 0, policy_version 832261 (0.0007) [2023-12-26 21:25:13,277][105620] Updated weights for policy 1, policy_version 832030 (0.0006) [2023-12-26 21:25:13,334][105620] Updated weights for policy 1, policy_version 832040 (0.0009) [2023-12-26 21:25:13,399][105620] Updated weights for policy 1, policy_version 832050 (0.0009) [2023-12-26 21:25:13,454][105692] Updated weights for policy 0, policy_version 832271 (0.0007) [2023-12-26 21:25:13,519][105692] Updated weights for policy 0, policy_version 832281 (0.0006) [2023-12-26 21:25:13,571][105692] Updated weights for policy 0, policy_version 832291 (0.0009) [2023-12-26 21:25:14,108][105620] Updated weights for policy 1, policy_version 832060 (0.0009) [2023-12-26 21:25:14,173][105620] Updated weights for policy 1, policy_version 832070 (0.0009) [2023-12-26 21:25:14,235][105620] Updated weights for policy 1, policy_version 832080 (0.0010) [2023-12-26 21:25:14,288][105692] Updated weights for policy 0, policy_version 832301 (0.0007) [2023-12-26 21:25:14,351][105692] Updated weights for policy 0, policy_version 832311 (0.0005) [2023-12-26 21:25:14,420][105692] Updated weights for policy 0, policy_version 832321 (0.0005) [2023-12-26 21:25:14,974][105620] Updated weights for policy 1, policy_version 832090 (0.0010) [2023-12-26 21:25:15,038][105620] Updated weights for policy 1, policy_version 832100 (0.0011) [2023-12-26 21:25:15,067][105692] Updated weights for policy 0, policy_version 832331 (0.0005) [2023-12-26 21:25:15,101][105620] Updated weights for policy 1, policy_version 832110 (0.0011) [2023-12-26 21:25:15,127][105692] Updated weights for policy 0, policy_version 832341 (0.0010) [2023-12-26 21:25:15,164][105620] Updated weights for policy 1, policy_version 832120 (0.0010) [2023-12-26 21:25:15,190][105692] Updated weights for policy 0, policy_version 832351 (0.0010) [2023-12-26 21:25:15,854][105692] Updated weights for policy 0, policy_version 832361 (0.0010) [2023-12-26 21:25:15,876][105620] Updated weights for policy 1, policy_version 832130 (0.0005) [2023-12-26 21:25:15,899][105692] Updated weights for policy 0, policy_version 832371 (0.0010) [2023-12-26 21:25:15,942][105620] Updated weights for policy 1, policy_version 832140 (0.0009) [2023-12-26 21:25:15,950][105692] Updated weights for policy 0, policy_version 832381 (0.0010) [2023-12-26 21:25:15,991][105620] Updated weights for policy 1, policy_version 832150 (0.0011) [2023-12-26 21:25:16,003][105692] Updated weights for policy 0, policy_version 832391 (0.0011) [2023-12-26 21:25:16,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 426180608. Throughput: 0: 9733.0, 1: 9869.7. Samples: 426143256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:25:16,062][104569] Avg episode reward: [(0, '8986.235'), (1, '9168.733')] [2023-12-26 21:25:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000832392_213123072.pth... [2023-12-26 21:25:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000832152_213057536.pth... [2023-12-26 21:25:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000831000_212762624.pth [2023-12-26 21:25:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000831240_212828160.pth [2023-12-26 21:25:16,669][105620] Updated weights for policy 1, policy_version 832160 (0.0009) [2023-12-26 21:25:16,728][105620] Updated weights for policy 1, policy_version 832170 (0.0008) [2023-12-26 21:25:16,729][105692] Updated weights for policy 0, policy_version 832401 (0.0007) [2023-12-26 21:25:16,781][105692] Updated weights for policy 0, policy_version 832411 (0.0007) [2023-12-26 21:25:16,791][105620] Updated weights for policy 1, policy_version 832180 (0.0007) [2023-12-26 21:25:16,846][105692] Updated weights for policy 0, policy_version 832421 (0.0008) [2023-12-26 21:25:17,386][105620] Updated weights for policy 1, policy_version 832190 (0.0007) [2023-12-26 21:25:17,432][105620] Updated weights for policy 1, policy_version 832200 (0.0005) [2023-12-26 21:25:17,481][105620] Updated weights for policy 1, policy_version 832210 (0.0008) [2023-12-26 21:25:17,510][105692] Updated weights for policy 0, policy_version 832431 (0.0006) [2023-12-26 21:25:17,558][105692] Updated weights for policy 0, policy_version 832441 (0.0010) [2023-12-26 21:25:17,623][105692] Updated weights for policy 0, policy_version 832451 (0.0010) [2023-12-26 21:25:18,184][105620] Updated weights for policy 1, policy_version 832220 (0.0009) [2023-12-26 21:25:18,233][105620] Updated weights for policy 1, policy_version 832230 (0.0010) [2023-12-26 21:25:18,272][105692] Updated weights for policy 0, policy_version 832461 (0.0008) [2023-12-26 21:25:18,289][105620] Updated weights for policy 1, policy_version 832240 (0.0010) [2023-12-26 21:25:18,330][105692] Updated weights for policy 0, policy_version 832471 (0.0006) [2023-12-26 21:25:18,391][105692] Updated weights for policy 0, policy_version 832481 (0.0008) [2023-12-26 21:25:19,007][105620] Updated weights for policy 1, policy_version 832250 (0.0010) [2023-12-26 21:25:19,067][105620] Updated weights for policy 1, policy_version 832260 (0.0005) [2023-12-26 21:25:19,128][105692] Updated weights for policy 0, policy_version 832491 (0.0010) [2023-12-26 21:25:19,129][105620] Updated weights for policy 1, policy_version 832270 (0.0007) [2023-12-26 21:25:19,187][105692] Updated weights for policy 0, policy_version 832501 (0.0010) [2023-12-26 21:25:19,191][105620] Updated weights for policy 1, policy_version 832280 (0.0010) [2023-12-26 21:25:19,250][105692] Updated weights for policy 0, policy_version 832511 (0.0011) [2023-12-26 21:25:19,904][105692] Updated weights for policy 0, policy_version 832521 (0.0010) [2023-12-26 21:25:19,912][105620] Updated weights for policy 1, policy_version 832290 (0.0009) [2023-12-26 21:25:19,974][105692] Updated weights for policy 0, policy_version 832531 (0.0009) [2023-12-26 21:25:19,978][105620] Updated weights for policy 1, policy_version 832300 (0.0009) [2023-12-26 21:25:20,032][105692] Updated weights for policy 0, policy_version 832541 (0.0009) [2023-12-26 21:25:20,032][105620] Updated weights for policy 1, policy_version 832310 (0.0009) [2023-12-26 21:25:20,088][105692] Updated weights for policy 0, policy_version 832551 (0.0008) [2023-12-26 21:25:20,801][105692] Updated weights for policy 0, policy_version 832561 (0.0011) [2023-12-26 21:25:20,854][105692] Updated weights for policy 0, policy_version 832571 (0.0011) [2023-12-26 21:25:20,877][105620] Updated weights for policy 1, policy_version 832320 (0.0008) [2023-12-26 21:25:20,909][105692] Updated weights for policy 0, policy_version 832581 (0.0007) [2023-12-26 21:25:20,939][105620] Updated weights for policy 1, policy_version 832330 (0.0008) [2023-12-26 21:25:20,999][105620] Updated weights for policy 1, policy_version 832340 (0.0008) [2023-12-26 21:25:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 426278912. Throughput: 0: 9679.6, 1: 9859.4. Samples: 426262912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:25:21,062][104569] Avg episode reward: [(0, '8805.506'), (1, '9260.778')] [2023-12-26 21:25:21,672][105692] Updated weights for policy 0, policy_version 832591 (0.0008) [2023-12-26 21:25:21,737][105692] Updated weights for policy 0, policy_version 832601 (0.0008) [2023-12-26 21:25:21,740][105620] Updated weights for policy 1, policy_version 832350 (0.0009) [2023-12-26 21:25:21,794][105692] Updated weights for policy 0, policy_version 832611 (0.0010) [2023-12-26 21:25:21,801][105620] Updated weights for policy 1, policy_version 832360 (0.0009) [2023-12-26 21:25:21,861][105620] Updated weights for policy 1, policy_version 832370 (0.0007) [2023-12-26 21:25:22,587][105692] Updated weights for policy 0, policy_version 832621 (0.0009) [2023-12-26 21:25:22,607][105620] Updated weights for policy 1, policy_version 832380 (0.0008) [2023-12-26 21:25:22,651][105692] Updated weights for policy 0, policy_version 832631 (0.0008) [2023-12-26 21:25:22,679][105620] Updated weights for policy 1, policy_version 832390 (0.0007) [2023-12-26 21:25:22,706][105692] Updated weights for policy 0, policy_version 832641 (0.0007) [2023-12-26 21:25:22,737][105620] Updated weights for policy 1, policy_version 832400 (0.0006) [2023-12-26 21:25:23,450][105620] Updated weights for policy 1, policy_version 832410 (0.0009) [2023-12-26 21:25:23,473][105692] Updated weights for policy 0, policy_version 832651 (0.0008) [2023-12-26 21:25:23,503][105620] Updated weights for policy 1, policy_version 832420 (0.0009) [2023-12-26 21:25:23,522][105692] Updated weights for policy 0, policy_version 832661 (0.0006) [2023-12-26 21:25:23,548][105620] Updated weights for policy 1, policy_version 832430 (0.0010) [2023-12-26 21:25:23,571][105692] Updated weights for policy 0, policy_version 832671 (0.0006) [2023-12-26 21:25:23,596][105620] Updated weights for policy 1, policy_version 832440 (0.0010) [2023-12-26 21:25:24,231][105620] Updated weights for policy 1, policy_version 832450 (0.0010) [2023-12-26 21:25:24,293][105620] Updated weights for policy 1, policy_version 832460 (0.0010) [2023-12-26 21:25:24,352][105620] Updated weights for policy 1, policy_version 832470 (0.0005) [2023-12-26 21:25:24,392][105692] Updated weights for policy 0, policy_version 832681 (0.0006) [2023-12-26 21:25:24,452][105692] Updated weights for policy 0, policy_version 832691 (0.0006) [2023-12-26 21:25:24,504][105692] Updated weights for policy 0, policy_version 832701 (0.0010) [2023-12-26 21:25:24,561][105692] Updated weights for policy 0, policy_version 832711 (0.0006) [2023-12-26 21:25:24,989][105620] Updated weights for policy 1, policy_version 832480 (0.0009) [2023-12-26 21:25:25,045][105620] Updated weights for policy 1, policy_version 832490 (0.0010) [2023-12-26 21:25:25,095][105620] Updated weights for policy 1, policy_version 832500 (0.0009) [2023-12-26 21:25:25,129][105692] Updated weights for policy 0, policy_version 832721 (0.0007) [2023-12-26 21:25:25,185][105692] Updated weights for policy 0, policy_version 832731 (0.0009) [2023-12-26 21:25:25,239][105692] Updated weights for policy 0, policy_version 832741 (0.0009) [2023-12-26 21:25:25,734][105620] Updated weights for policy 1, policy_version 832510 (0.0008) [2023-12-26 21:25:25,791][105620] Updated weights for policy 1, policy_version 832520 (0.0010) [2023-12-26 21:25:25,840][105620] Updated weights for policy 1, policy_version 832530 (0.0010) [2023-12-26 21:25:26,047][105692] Updated weights for policy 0, policy_version 832751 (0.0010) [2023-12-26 21:25:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 426369024. Throughput: 0: 9700.3, 1: 9859.2. Samples: 426378560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:25:26,062][104569] Avg episode reward: [(0, '8897.327'), (1, '9260.402')] [2023-12-26 21:25:26,103][105692] Updated weights for policy 0, policy_version 832761 (0.0010) [2023-12-26 21:25:26,161][105692] Updated weights for policy 0, policy_version 832771 (0.0010) [2023-12-26 21:25:26,476][105620] Updated weights for policy 1, policy_version 832540 (0.0010) [2023-12-26 21:25:26,523][105620] Updated weights for policy 1, policy_version 832550 (0.0010) [2023-12-26 21:25:26,588][105620] Updated weights for policy 1, policy_version 832560 (0.0005) [2023-12-26 21:25:26,878][105692] Updated weights for policy 0, policy_version 832781 (0.0008) [2023-12-26 21:25:26,925][105692] Updated weights for policy 0, policy_version 832791 (0.0010) [2023-12-26 21:25:26,972][105692] Updated weights for policy 0, policy_version 832801 (0.0005) [2023-12-26 21:25:27,137][105620] Updated weights for policy 1, policy_version 832570 (0.0005) [2023-12-26 21:25:27,201][105620] Updated weights for policy 1, policy_version 832580 (0.0005) [2023-12-26 21:25:27,270][105620] Updated weights for policy 1, policy_version 832590 (0.0005) [2023-12-26 21:25:27,326][105620] Updated weights for policy 1, policy_version 832600 (0.0006) [2023-12-26 21:25:27,533][105692] Updated weights for policy 0, policy_version 832811 (0.0005) [2023-12-26 21:25:27,590][105692] Updated weights for policy 0, policy_version 832821 (0.0005) [2023-12-26 21:25:27,647][105692] Updated weights for policy 0, policy_version 832831 (0.0008) [2023-12-26 21:25:27,944][105620] Updated weights for policy 1, policy_version 832610 (0.0010) [2023-12-26 21:25:28,002][105620] Updated weights for policy 1, policy_version 832620 (0.0009) [2023-12-26 21:25:28,058][105620] Updated weights for policy 1, policy_version 832630 (0.0005) [2023-12-26 21:25:28,334][105692] Updated weights for policy 0, policy_version 832841 (0.0010) [2023-12-26 21:25:28,393][105692] Updated weights for policy 0, policy_version 832851 (0.0010) [2023-12-26 21:25:28,438][105692] Updated weights for policy 0, policy_version 832861 (0.0010) [2023-12-26 21:25:28,490][105692] Updated weights for policy 0, policy_version 832871 (0.0010) [2023-12-26 21:25:28,654][105620] Updated weights for policy 1, policy_version 832640 (0.0009) [2023-12-26 21:25:28,715][105620] Updated weights for policy 1, policy_version 832650 (0.0006) [2023-12-26 21:25:28,779][105620] Updated weights for policy 1, policy_version 832660 (0.0010) [2023-12-26 21:25:29,270][105692] Updated weights for policy 0, policy_version 832881 (0.0009) [2023-12-26 21:25:29,338][105692] Updated weights for policy 0, policy_version 832891 (0.0006) [2023-12-26 21:25:29,401][105692] Updated weights for policy 0, policy_version 832901 (0.0011) [2023-12-26 21:25:29,438][105620] Updated weights for policy 1, policy_version 832670 (0.0007) [2023-12-26 21:25:29,490][105620] Updated weights for policy 1, policy_version 832680 (0.0005) [2023-12-26 21:25:29,552][105620] Updated weights for policy 1, policy_version 832690 (0.0007) [2023-12-26 21:25:30,112][105692] Updated weights for policy 0, policy_version 832911 (0.0010) [2023-12-26 21:25:30,168][105692] Updated weights for policy 0, policy_version 832921 (0.0010) [2023-12-26 21:25:30,203][105620] Updated weights for policy 1, policy_version 832700 (0.0008) [2023-12-26 21:25:30,220][105692] Updated weights for policy 0, policy_version 832931 (0.0010) [2023-12-26 21:25:30,258][105620] Updated weights for policy 1, policy_version 832710 (0.0006) [2023-12-26 21:25:30,322][105620] Updated weights for policy 1, policy_version 832720 (0.0005) [2023-12-26 21:25:30,945][105692] Updated weights for policy 0, policy_version 832941 (0.0008) [2023-12-26 21:25:30,994][105620] Updated weights for policy 1, policy_version 832730 (0.0007) [2023-12-26 21:25:30,996][105692] Updated weights for policy 0, policy_version 832951 (0.0005) [2023-12-26 21:25:31,055][105692] Updated weights for policy 0, policy_version 832961 (0.0007) [2023-12-26 21:25:31,057][105620] Updated weights for policy 1, policy_version 832740 (0.0009) [2023-12-26 21:25:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19661.0, 300 sec: 19549.7). Total num frames: 426467328. Throughput: 0: 9724.6, 1: 9961.6. Samples: 426443028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:25:31,062][104569] Avg episode reward: [(0, '9079.607'), (1, '9260.465')] [2023-12-26 21:25:31,096][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000832968_213270528.pth... [2023-12-26 21:25:31,100][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000831816_212975616.pth [2023-12-26 21:25:31,113][105620] Updated weights for policy 1, policy_version 832750 (0.0008) [2023-12-26 21:25:31,177][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000832760_213213184.pth... [2023-12-26 21:25:31,178][105620] Updated weights for policy 1, policy_version 832760 (0.0007) [2023-12-26 21:25:31,181][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000831576_212910080.pth [2023-12-26 21:25:31,786][105692] Updated weights for policy 0, policy_version 832971 (0.0007) [2023-12-26 21:25:31,851][105692] Updated weights for policy 0, policy_version 832981 (0.0005) [2023-12-26 21:25:31,897][105620] Updated weights for policy 1, policy_version 832770 (0.0008) [2023-12-26 21:25:31,909][105692] Updated weights for policy 0, policy_version 832991 (0.0005) [2023-12-26 21:25:31,957][105620] Updated weights for policy 1, policy_version 832780 (0.0010) [2023-12-26 21:25:32,014][105620] Updated weights for policy 1, policy_version 832790 (0.0010) [2023-12-26 21:25:32,472][105692] Updated weights for policy 0, policy_version 833001 (0.0006) [2023-12-26 21:25:32,524][105692] Updated weights for policy 0, policy_version 833011 (0.0010) [2023-12-26 21:25:32,589][105692] Updated weights for policy 0, policy_version 833021 (0.0011) [2023-12-26 21:25:32,654][105692] Updated weights for policy 0, policy_version 833031 (0.0011) [2023-12-26 21:25:32,787][105620] Updated weights for policy 1, policy_version 832801 (0.0010) [2023-12-26 21:25:32,848][105620] Updated weights for policy 1, policy_version 832811 (0.0010) [2023-12-26 21:25:32,902][105620] Updated weights for policy 1, policy_version 832821 (0.0010) [2023-12-26 21:25:33,215][105692] Updated weights for policy 0, policy_version 833041 (0.0006) [2023-12-26 21:25:33,273][105692] Updated weights for policy 0, policy_version 833051 (0.0006) [2023-12-26 21:25:33,320][105692] Updated weights for policy 0, policy_version 833061 (0.0009) [2023-12-26 21:25:33,694][105620] Updated weights for policy 1, policy_version 832831 (0.0009) [2023-12-26 21:25:33,751][105620] Updated weights for policy 1, policy_version 832841 (0.0009) [2023-12-26 21:25:33,809][105620] Updated weights for policy 1, policy_version 832851 (0.0009) [2023-12-26 21:25:34,014][105692] Updated weights for policy 0, policy_version 833071 (0.0009) [2023-12-26 21:25:34,069][105692] Updated weights for policy 0, policy_version 833081 (0.0009) [2023-12-26 21:25:34,130][105692] Updated weights for policy 0, policy_version 833091 (0.0009) [2023-12-26 21:25:34,573][105620] Updated weights for policy 1, policy_version 832861 (0.0008) [2023-12-26 21:25:34,637][105620] Updated weights for policy 1, policy_version 832871 (0.0006) [2023-12-26 21:25:34,692][105620] Updated weights for policy 1, policy_version 832881 (0.0006) [2023-12-26 21:25:34,959][105692] Updated weights for policy 0, policy_version 833101 (0.0009) [2023-12-26 21:25:35,016][105692] Updated weights for policy 0, policy_version 833111 (0.0009) [2023-12-26 21:25:35,077][105692] Updated weights for policy 0, policy_version 833121 (0.0009) [2023-12-26 21:25:35,377][105620] Updated weights for policy 1, policy_version 832891 (0.0007) [2023-12-26 21:25:35,435][105620] Updated weights for policy 1, policy_version 832901 (0.0008) [2023-12-26 21:25:35,487][105620] Updated weights for policy 1, policy_version 832911 (0.0010) [2023-12-26 21:25:35,746][105692] Updated weights for policy 0, policy_version 833131 (0.0008) [2023-12-26 21:25:35,801][105692] Updated weights for policy 0, policy_version 833141 (0.0005) [2023-12-26 21:25:35,851][105692] Updated weights for policy 0, policy_version 833151 (0.0009) [2023-12-26 21:25:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 426573824. Throughput: 0: 9812.2, 1: 9904.5. Samples: 426561752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:25:36,062][104569] Avg episode reward: [(0, '8988.367'), (1, '9077.569')] [2023-12-26 21:25:36,215][105620] Updated weights for policy 1, policy_version 832922 (0.0009) [2023-12-26 21:25:36,285][105620] Updated weights for policy 1, policy_version 832932 (0.0009) [2023-12-26 21:25:36,350][105620] Updated weights for policy 1, policy_version 832942 (0.0008) [2023-12-26 21:25:36,419][105620] Updated weights for policy 1, policy_version 832952 (0.0009) [2023-12-26 21:25:36,516][105692] Updated weights for policy 0, policy_version 833161 (0.0008) [2023-12-26 21:25:36,584][105692] Updated weights for policy 0, policy_version 833171 (0.0008) [2023-12-26 21:25:36,643][105692] Updated weights for policy 0, policy_version 833182 (0.0008) [2023-12-26 21:25:36,698][105692] Updated weights for policy 0, policy_version 833192 (0.0007) [2023-12-26 21:25:37,198][105620] Updated weights for policy 1, policy_version 832962 (0.0009) [2023-12-26 21:25:37,264][105620] Updated weights for policy 1, policy_version 832972 (0.0007) [2023-12-26 21:25:37,291][105586] KL-divergence is very high: 173.3179 [2023-12-26 21:25:37,327][105620] Updated weights for policy 1, policy_version 832982 (0.0006) [2023-12-26 21:25:37,386][105692] Updated weights for policy 0, policy_version 833202 (0.0010) [2023-12-26 21:25:37,445][105692] Updated weights for policy 0, policy_version 833213 (0.0010) [2023-12-26 21:25:37,497][105692] Updated weights for policy 0, policy_version 833223 (0.0009) [2023-12-26 21:25:37,981][105620] Updated weights for policy 1, policy_version 832992 (0.0009) [2023-12-26 21:25:38,040][105620] Updated weights for policy 1, policy_version 833002 (0.0006) [2023-12-26 21:25:38,098][105620] Updated weights for policy 1, policy_version 833012 (0.0005) [2023-12-26 21:25:38,250][105692] Updated weights for policy 0, policy_version 833233 (0.0009) [2023-12-26 21:25:38,308][105692] Updated weights for policy 0, policy_version 833244 (0.0010) [2023-12-26 21:25:38,314][105585] KL-divergence is very high: 110.6133 [2023-12-26 21:25:38,369][105692] Updated weights for policy 0, policy_version 833254 (0.0008) [2023-12-26 21:25:38,703][105620] Updated weights for policy 1, policy_version 833022 (0.0008) [2023-12-26 21:25:38,763][105620] Updated weights for policy 1, policy_version 833032 (0.0009) [2023-12-26 21:25:38,818][105620] Updated weights for policy 1, policy_version 833042 (0.0009) [2023-12-26 21:25:39,134][105692] Updated weights for policy 0, policy_version 833264 (0.0006) [2023-12-26 21:25:39,193][105692] Updated weights for policy 0, policy_version 833274 (0.0009) [2023-12-26 21:25:39,265][105692] Updated weights for policy 0, policy_version 833284 (0.0009) [2023-12-26 21:25:39,587][105620] Updated weights for policy 1, policy_version 833052 (0.0010) [2023-12-26 21:25:39,638][105620] Updated weights for policy 1, policy_version 833062 (0.0008) [2023-12-26 21:25:39,702][105620] Updated weights for policy 1, policy_version 833072 (0.0009) [2023-12-26 21:25:39,993][105692] Updated weights for policy 0, policy_version 833294 (0.0009) [2023-12-26 21:25:40,051][105692] Updated weights for policy 0, policy_version 833304 (0.0009) [2023-12-26 21:25:40,113][105692] Updated weights for policy 0, policy_version 833314 (0.0007) [2023-12-26 21:25:40,493][105620] Updated weights for policy 1, policy_version 833082 (0.0009) [2023-12-26 21:25:40,561][105620] Updated weights for policy 1, policy_version 833092 (0.0009) [2023-12-26 21:25:40,619][105620] Updated weights for policy 1, policy_version 833102 (0.0009) [2023-12-26 21:25:40,679][105620] Updated weights for policy 1, policy_version 833112 (0.0005) [2023-12-26 21:25:40,887][105692] Updated weights for policy 0, policy_version 833324 (0.0009) [2023-12-26 21:25:40,937][105692] Updated weights for policy 0, policy_version 833334 (0.0009) [2023-12-26 21:25:40,991][105692] Updated weights for policy 0, policy_version 833344 (0.0009) [2023-12-26 21:25:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 426672128. Throughput: 0: 9687.5, 1: 9901.5. Samples: 426677012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:25:41,062][104569] Avg episode reward: [(0, '8715.077'), (1, '8804.365')] [2023-12-26 21:25:41,324][105620] Updated weights for policy 1, policy_version 833122 (0.0007) [2023-12-26 21:25:41,373][105620] Updated weights for policy 1, policy_version 833132 (0.0007) [2023-12-26 21:25:41,441][105620] Updated weights for policy 1, policy_version 833142 (0.0007) [2023-12-26 21:25:41,786][105692] Updated weights for policy 0, policy_version 833354 (0.0008) [2023-12-26 21:25:41,838][105692] Updated weights for policy 0, policy_version 833364 (0.0005) [2023-12-26 21:25:41,895][105692] Updated weights for policy 0, policy_version 833374 (0.0007) [2023-12-26 21:25:41,949][105692] Updated weights for policy 0, policy_version 833384 (0.0009) [2023-12-26 21:25:42,278][105620] Updated weights for policy 1, policy_version 833152 (0.0009) [2023-12-26 21:25:42,341][105620] Updated weights for policy 1, policy_version 833162 (0.0010) [2023-12-26 21:25:42,405][105620] Updated weights for policy 1, policy_version 833172 (0.0009) [2023-12-26 21:25:42,736][105692] Updated weights for policy 0, policy_version 833394 (0.0009) [2023-12-26 21:25:42,799][105692] Updated weights for policy 0, policy_version 833404 (0.0009) [2023-12-26 21:25:42,857][105692] Updated weights for policy 0, policy_version 833414 (0.0010) [2023-12-26 21:25:43,076][105620] Updated weights for policy 1, policy_version 833182 (0.0007) [2023-12-26 21:25:43,123][105620] Updated weights for policy 1, policy_version 833192 (0.0005) [2023-12-26 21:25:43,172][105620] Updated weights for policy 1, policy_version 833202 (0.0005) [2023-12-26 21:25:43,708][105692] Updated weights for policy 0, policy_version 833424 (0.0010) [2023-12-26 21:25:43,764][105692] Updated weights for policy 0, policy_version 833434 (0.0009) [2023-12-26 21:25:43,795][105620] Updated weights for policy 1, policy_version 833212 (0.0007) [2023-12-26 21:25:43,821][105692] Updated weights for policy 0, policy_version 833444 (0.0008) [2023-12-26 21:25:43,844][105620] Updated weights for policy 1, policy_version 833222 (0.0006) [2023-12-26 21:25:43,894][105620] Updated weights for policy 1, policy_version 833232 (0.0009) [2023-12-26 21:25:44,512][105692] Updated weights for policy 0, policy_version 833454 (0.0009) [2023-12-26 21:25:44,559][105692] Updated weights for policy 0, policy_version 833464 (0.0009) [2023-12-26 21:25:44,607][105692] Updated weights for policy 0, policy_version 833474 (0.0009) [2023-12-26 21:25:44,682][105620] Updated weights for policy 1, policy_version 833242 (0.0009) [2023-12-26 21:25:44,729][105620] Updated weights for policy 1, policy_version 833252 (0.0008) [2023-12-26 21:25:44,785][105620] Updated weights for policy 1, policy_version 833262 (0.0009) [2023-12-26 21:25:44,850][105620] Updated weights for policy 1, policy_version 833272 (0.0010) [2023-12-26 21:25:45,443][105692] Updated weights for policy 0, policy_version 833484 (0.0009) [2023-12-26 21:25:45,501][105692] Updated weights for policy 0, policy_version 833494 (0.0009) [2023-12-26 21:25:45,536][105620] Updated weights for policy 1, policy_version 833282 (0.0010) [2023-12-26 21:25:45,558][105692] Updated weights for policy 0, policy_version 833504 (0.0006) [2023-12-26 21:25:45,595][105620] Updated weights for policy 1, policy_version 833292 (0.0010) [2023-12-26 21:25:45,659][105620] Updated weights for policy 1, policy_version 833302 (0.0009) [2023-12-26 21:25:46,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 426762240. Throughput: 0: 9624.0, 1: 9892.1. Samples: 426732884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:25:46,063][104569] Avg episode reward: [(0, '8893.519'), (1, '8804.110')] [2023-12-26 21:25:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000833304_213352448.pth... [2023-12-26 21:25:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000833512_213409792.pth... [2023-12-26 21:25:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000832152_213057536.pth [2023-12-26 21:25:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000832392_213123072.pth [2023-12-26 21:25:46,286][105620] Updated weights for policy 1, policy_version 833312 (0.0006) [2023-12-26 21:25:46,339][105620] Updated weights for policy 1, policy_version 833322 (0.0006) [2023-12-26 21:25:46,383][105692] Updated weights for policy 0, policy_version 833514 (0.0006) [2023-12-26 21:25:46,395][105620] Updated weights for policy 1, policy_version 833332 (0.0005) [2023-12-26 21:25:46,434][105692] Updated weights for policy 0, policy_version 833525 (0.0009) [2023-12-26 21:25:46,495][105692] Updated weights for policy 0, policy_version 833535 (0.0009) [2023-12-26 21:25:46,996][105620] Updated weights for policy 1, policy_version 833342 (0.0006) [2023-12-26 21:25:47,053][105620] Updated weights for policy 1, policy_version 833352 (0.0009) [2023-12-26 21:25:47,111][105620] Updated weights for policy 1, policy_version 833362 (0.0009) [2023-12-26 21:25:47,342][105692] Updated weights for policy 0, policy_version 833545 (0.0008) [2023-12-26 21:25:47,399][105692] Updated weights for policy 0, policy_version 833555 (0.0008) [2023-12-26 21:25:47,462][105692] Updated weights for policy 0, policy_version 833565 (0.0009) [2023-12-26 21:25:47,522][105692] Updated weights for policy 0, policy_version 833575 (0.0008) [2023-12-26 21:25:47,808][105620] Updated weights for policy 1, policy_version 833372 (0.0009) [2023-12-26 21:25:47,866][105620] Updated weights for policy 1, policy_version 833382 (0.0008) [2023-12-26 21:25:47,917][105620] Updated weights for policy 1, policy_version 833392 (0.0009) [2023-12-26 21:25:48,301][105692] Updated weights for policy 0, policy_version 833585 (0.0009) [2023-12-26 21:25:48,365][105692] Updated weights for policy 0, policy_version 833595 (0.0010) [2023-12-26 21:25:48,417][105692] Updated weights for policy 0, policy_version 833605 (0.0011) [2023-12-26 21:25:48,580][105620] Updated weights for policy 1, policy_version 833402 (0.0008) [2023-12-26 21:25:48,647][105620] Updated weights for policy 1, policy_version 833412 (0.0008) [2023-12-26 21:25:48,718][105620] Updated weights for policy 1, policy_version 833422 (0.0008) [2023-12-26 21:25:48,777][105620] Updated weights for policy 1, policy_version 833432 (0.0008) [2023-12-26 21:25:49,153][105692] Updated weights for policy 0, policy_version 833615 (0.0007) [2023-12-26 21:25:49,214][105692] Updated weights for policy 0, policy_version 833625 (0.0007) [2023-12-26 21:25:49,280][105692] Updated weights for policy 0, policy_version 833635 (0.0007) [2023-12-26 21:25:49,569][105620] Updated weights for policy 1, policy_version 833442 (0.0010) [2023-12-26 21:25:49,631][105620] Updated weights for policy 1, policy_version 833452 (0.0010) [2023-12-26 21:25:49,697][105620] Updated weights for policy 1, policy_version 833462 (0.0010) [2023-12-26 21:25:49,979][105692] Updated weights for policy 0, policy_version 833645 (0.0007) [2023-12-26 21:25:50,045][105692] Updated weights for policy 0, policy_version 833655 (0.0008) [2023-12-26 21:25:50,104][105692] Updated weights for policy 0, policy_version 833665 (0.0008) [2023-12-26 21:25:50,435][105620] Updated weights for policy 1, policy_version 833472 (0.0006) [2023-12-26 21:25:50,492][105620] Updated weights for policy 1, policy_version 833482 (0.0005) [2023-12-26 21:25:50,548][105620] Updated weights for policy 1, policy_version 833492 (0.0009) [2023-12-26 21:25:50,903][105692] Updated weights for policy 0, policy_version 833675 (0.0008) [2023-12-26 21:25:50,967][105692] Updated weights for policy 0, policy_version 833685 (0.0008) [2023-12-26 21:25:51,028][105692] Updated weights for policy 0, policy_version 833695 (0.0008) [2023-12-26 21:25:51,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 426852352. Throughput: 0: 9616.9, 1: 9924.6. Samples: 426846744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:25:51,062][104569] Avg episode reward: [(0, '8814.952'), (1, '8987.055')] [2023-12-26 21:25:51,279][105620] Updated weights for policy 1, policy_version 833502 (0.0010) [2023-12-26 21:25:51,334][105620] Updated weights for policy 1, policy_version 833512 (0.0010) [2023-12-26 21:25:51,404][105620] Updated weights for policy 1, policy_version 833522 (0.0010) [2023-12-26 21:25:51,824][105692] Updated weights for policy 0, policy_version 833705 (0.0009) [2023-12-26 21:25:51,881][105692] Updated weights for policy 0, policy_version 833715 (0.0008) [2023-12-26 21:25:51,940][105692] Updated weights for policy 0, policy_version 833725 (0.0008) [2023-12-26 21:25:52,000][105692] Updated weights for policy 0, policy_version 833735 (0.0008) [2023-12-26 21:25:52,166][105620] Updated weights for policy 1, policy_version 833532 (0.0010) [2023-12-26 21:25:52,231][105620] Updated weights for policy 1, policy_version 833542 (0.0010) [2023-12-26 21:25:52,290][105620] Updated weights for policy 1, policy_version 833552 (0.0011) [2023-12-26 21:25:52,778][105692] Updated weights for policy 0, policy_version 833745 (0.0008) [2023-12-26 21:25:52,837][105692] Updated weights for policy 0, policy_version 833755 (0.0006) [2023-12-26 21:25:52,889][105692] Updated weights for policy 0, policy_version 833765 (0.0005) [2023-12-26 21:25:53,001][105620] Updated weights for policy 1, policy_version 833562 (0.0009) [2023-12-26 21:25:53,067][105620] Updated weights for policy 1, policy_version 833572 (0.0006) [2023-12-26 21:25:53,126][105620] Updated weights for policy 1, policy_version 833582 (0.0006) [2023-12-26 21:25:53,185][105620] Updated weights for policy 1, policy_version 833592 (0.0007) [2023-12-26 21:25:53,638][105692] Updated weights for policy 0, policy_version 833775 (0.0007) [2023-12-26 21:25:53,692][105692] Updated weights for policy 0, policy_version 833785 (0.0005) [2023-12-26 21:25:53,753][105692] Updated weights for policy 0, policy_version 833795 (0.0005) [2023-12-26 21:25:53,794][105620] Updated weights for policy 1, policy_version 833602 (0.0010) [2023-12-26 21:25:53,849][105620] Updated weights for policy 1, policy_version 833612 (0.0010) [2023-12-26 21:25:53,918][105620] Updated weights for policy 1, policy_version 833622 (0.0009) [2023-12-26 21:25:54,379][105692] Updated weights for policy 0, policy_version 833805 (0.0008) [2023-12-26 21:25:54,438][105692] Updated weights for policy 0, policy_version 833815 (0.0010) [2023-12-26 21:25:54,490][105692] Updated weights for policy 0, policy_version 833825 (0.0009) [2023-12-26 21:25:54,512][105620] Updated weights for policy 1, policy_version 833632 (0.0006) [2023-12-26 21:25:54,580][105620] Updated weights for policy 1, policy_version 833642 (0.0006) [2023-12-26 21:25:54,639][105620] Updated weights for policy 1, policy_version 833652 (0.0006) [2023-12-26 21:25:55,271][105692] Updated weights for policy 0, policy_version 833835 (0.0008) [2023-12-26 21:25:55,273][105620] Updated weights for policy 1, policy_version 833662 (0.0007) [2023-12-26 21:25:55,321][105620] Updated weights for policy 1, policy_version 833672 (0.0005) [2023-12-26 21:25:55,323][105692] Updated weights for policy 0, policy_version 833845 (0.0009) [2023-12-26 21:25:55,366][105620] Updated weights for policy 1, policy_version 833682 (0.0007) [2023-12-26 21:25:55,369][105692] Updated weights for policy 0, policy_version 833855 (0.0010) [2023-12-26 21:25:56,011][105692] Updated weights for policy 0, policy_version 833865 (0.0010) [2023-12-26 21:25:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19388.0, 300 sec: 19494.2). Total num frames: 426950656. Throughput: 0: 9533.9, 1: 9959.6. Samples: 426963392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:25:56,063][104569] Avg episode reward: [(0, '8642.280'), (1, '9078.956')] [2023-12-26 21:25:56,070][105692] Updated weights for policy 0, policy_version 833875 (0.0005) [2023-12-26 21:25:56,118][105620] Updated weights for policy 1, policy_version 833692 (0.0009) [2023-12-26 21:25:56,125][105692] Updated weights for policy 0, policy_version 833885 (0.0007) [2023-12-26 21:25:56,174][105620] Updated weights for policy 1, policy_version 833702 (0.0009) [2023-12-26 21:25:56,180][105692] Updated weights for policy 0, policy_version 833895 (0.0005) [2023-12-26 21:25:56,227][105620] Updated weights for policy 1, policy_version 833712 (0.0010) [2023-12-26 21:25:56,841][105620] Updated weights for policy 1, policy_version 833723 (0.0008) [2023-12-26 21:25:56,883][105620] Updated weights for policy 1, policy_version 833733 (0.0005) [2023-12-26 21:25:56,931][105692] Updated weights for policy 0, policy_version 833905 (0.0008) [2023-12-26 21:25:56,933][105620] Updated weights for policy 1, policy_version 833743 (0.0007) [2023-12-26 21:25:56,991][105692] Updated weights for policy 0, policy_version 833915 (0.0005) [2023-12-26 21:25:57,057][105692] Updated weights for policy 0, policy_version 833925 (0.0005) [2023-12-26 21:25:57,466][105620] Updated weights for policy 1, policy_version 833753 (0.0005) [2023-12-26 21:25:57,520][105620] Updated weights for policy 1, policy_version 833763 (0.0008) [2023-12-26 21:25:57,570][105620] Updated weights for policy 1, policy_version 833773 (0.0007) [2023-12-26 21:25:57,629][105620] Updated weights for policy 1, policy_version 833783 (0.0008) [2023-12-26 21:25:57,646][105692] Updated weights for policy 0, policy_version 833935 (0.0009) [2023-12-26 21:25:57,717][105692] Updated weights for policy 0, policy_version 833945 (0.0010) [2023-12-26 21:25:57,784][105692] Updated weights for policy 0, policy_version 833955 (0.0010) [2023-12-26 21:25:58,336][105620] Updated weights for policy 1, policy_version 833793 (0.0008) [2023-12-26 21:25:58,403][105620] Updated weights for policy 1, policy_version 833803 (0.0007) [2023-12-26 21:25:58,472][105620] Updated weights for policy 1, policy_version 833813 (0.0008) [2023-12-26 21:25:58,500][105692] Updated weights for policy 0, policy_version 833965 (0.0009) [2023-12-26 21:25:58,560][105692] Updated weights for policy 0, policy_version 833975 (0.0008) [2023-12-26 21:25:58,626][105692] Updated weights for policy 0, policy_version 833985 (0.0009) [2023-12-26 21:25:59,282][105620] Updated weights for policy 1, policy_version 833823 (0.0008) [2023-12-26 21:25:59,348][105620] Updated weights for policy 1, policy_version 833833 (0.0008) [2023-12-26 21:25:59,410][105620] Updated weights for policy 1, policy_version 833843 (0.0008) [2023-12-26 21:25:59,476][105692] Updated weights for policy 0, policy_version 833995 (0.0009) [2023-12-26 21:25:59,522][105692] Updated weights for policy 0, policy_version 834005 (0.0008) [2023-12-26 21:25:59,572][105692] Updated weights for policy 0, policy_version 834015 (0.0008) [2023-12-26 21:26:00,185][105692] Updated weights for policy 0, policy_version 834025 (0.0009) [2023-12-26 21:26:00,233][105692] Updated weights for policy 0, policy_version 834035 (0.0009) [2023-12-26 21:26:00,240][105620] Updated weights for policy 1, policy_version 833853 (0.0009) [2023-12-26 21:26:00,283][105692] Updated weights for policy 0, policy_version 834045 (0.0007) [2023-12-26 21:26:00,292][105620] Updated weights for policy 1, policy_version 833863 (0.0007) [2023-12-26 21:26:00,333][105692] Updated weights for policy 0, policy_version 834055 (0.0008) [2023-12-26 21:26:00,351][105620] Updated weights for policy 1, policy_version 833873 (0.0005) [2023-12-26 21:26:00,917][105620] Updated weights for policy 1, policy_version 833883 (0.0006) [2023-12-26 21:26:00,964][105620] Updated weights for policy 1, policy_version 833893 (0.0009) [2023-12-26 21:26:01,018][105620] Updated weights for policy 1, policy_version 833903 (0.0008) [2023-12-26 21:26:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 427048960. Throughput: 0: 9621.8, 1: 9977.2. Samples: 427025216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:26:01,063][104569] Avg episode reward: [(0, '8906.506'), (1, '8987.744')] [2023-12-26 21:26:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000834056_213549056.pth... [2023-12-26 21:26:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000832968_213270528.pth [2023-12-26 21:26:01,077][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000833912_213508096.pth... [2023-12-26 21:26:01,082][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000832760_213213184.pth [2023-12-26 21:26:01,197][105692] Updated weights for policy 0, policy_version 834065 (0.0009) [2023-12-26 21:26:01,257][105692] Updated weights for policy 0, policy_version 834075 (0.0010) [2023-12-26 21:26:01,309][105692] Updated weights for policy 0, policy_version 834085 (0.0009) [2023-12-26 21:26:01,656][105620] Updated weights for policy 1, policy_version 833913 (0.0007) [2023-12-26 21:26:01,726][105620] Updated weights for policy 1, policy_version 833923 (0.0008) [2023-12-26 21:26:01,796][105620] Updated weights for policy 1, policy_version 833933 (0.0010) [2023-12-26 21:26:01,863][105620] Updated weights for policy 1, policy_version 833943 (0.0010) [2023-12-26 21:26:02,132][105692] Updated weights for policy 0, policy_version 834095 (0.0010) [2023-12-26 21:26:02,194][105692] Updated weights for policy 0, policy_version 834105 (0.0010) [2023-12-26 21:26:02,257][105692] Updated weights for policy 0, policy_version 834115 (0.0010) [2023-12-26 21:26:02,607][105620] Updated weights for policy 1, policy_version 833953 (0.0008) [2023-12-26 21:26:02,665][105620] Updated weights for policy 1, policy_version 833963 (0.0008) [2023-12-26 21:26:02,723][105620] Updated weights for policy 1, policy_version 833973 (0.0008) [2023-12-26 21:26:03,000][105692] Updated weights for policy 0, policy_version 834125 (0.0011) [2023-12-26 21:26:03,052][105692] Updated weights for policy 0, policy_version 834135 (0.0010) [2023-12-26 21:26:03,097][105692] Updated weights for policy 0, policy_version 834145 (0.0010) [2023-12-26 21:26:03,502][105620] Updated weights for policy 1, policy_version 833983 (0.0009) [2023-12-26 21:26:03,516][105586] KL-divergence is very high: 139.3753 [2023-12-26 21:26:03,549][105620] Updated weights for policy 1, policy_version 833993 (0.0008) [2023-12-26 21:26:03,555][105586] KL-divergence is very high: 223.2531 [2023-12-26 21:26:03,605][105586] KL-divergence is very high: 211.4646 [2023-12-26 21:26:03,614][105620] Updated weights for policy 1, policy_version 834003 (0.0009) [2023-12-26 21:26:03,790][105692] Updated weights for policy 0, policy_version 834155 (0.0010) [2023-12-26 21:26:03,849][105692] Updated weights for policy 0, policy_version 834165 (0.0010) [2023-12-26 21:26:03,907][105692] Updated weights for policy 0, policy_version 834175 (0.0009) [2023-12-26 21:26:04,393][105620] Updated weights for policy 1, policy_version 834013 (0.0010) [2023-12-26 21:26:04,446][105620] Updated weights for policy 1, policy_version 834023 (0.0010) [2023-12-26 21:26:04,496][105620] Updated weights for policy 1, policy_version 834033 (0.0011) [2023-12-26 21:26:04,693][105692] Updated weights for policy 0, policy_version 834185 (0.0009) [2023-12-26 21:26:04,751][105692] Updated weights for policy 0, policy_version 834195 (0.0007) [2023-12-26 21:26:04,811][105692] Updated weights for policy 0, policy_version 834205 (0.0008) [2023-12-26 21:26:04,875][105692] Updated weights for policy 0, policy_version 834215 (0.0008) [2023-12-26 21:26:05,271][105620] Updated weights for policy 1, policy_version 834043 (0.0010) [2023-12-26 21:26:05,319][105620] Updated weights for policy 1, policy_version 834053 (0.0010) [2023-12-26 21:26:05,373][105620] Updated weights for policy 1, policy_version 834063 (0.0010) [2023-12-26 21:26:05,607][105692] Updated weights for policy 0, policy_version 834225 (0.0008) [2023-12-26 21:26:05,661][105692] Updated weights for policy 0, policy_version 834235 (0.0008) [2023-12-26 21:26:05,713][105692] Updated weights for policy 0, policy_version 834245 (0.0008) [2023-12-26 21:26:06,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 427147264. Throughput: 0: 9518.7, 1: 9928.1. Samples: 427138020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:26:06,063][104569] Avg episode reward: [(0, '8942.775'), (1, '9072.744')] [2023-12-26 21:26:06,126][105620] Updated weights for policy 1, policy_version 834073 (0.0010) [2023-12-26 21:26:06,179][105620] Updated weights for policy 1, policy_version 834083 (0.0010) [2023-12-26 21:26:06,240][105620] Updated weights for policy 1, policy_version 834093 (0.0011) [2023-12-26 21:26:06,300][105620] Updated weights for policy 1, policy_version 834103 (0.0010) [2023-12-26 21:26:06,488][105692] Updated weights for policy 0, policy_version 834255 (0.0009) [2023-12-26 21:26:06,544][105692] Updated weights for policy 0, policy_version 834265 (0.0008) [2023-12-26 21:26:06,600][105692] Updated weights for policy 0, policy_version 834275 (0.0008) [2023-12-26 21:26:07,061][105620] Updated weights for policy 1, policy_version 834113 (0.0011) [2023-12-26 21:26:07,120][105620] Updated weights for policy 1, policy_version 834123 (0.0010) [2023-12-26 21:26:07,182][105620] Updated weights for policy 1, policy_version 834133 (0.0010) [2023-12-26 21:26:07,379][105692] Updated weights for policy 0, policy_version 834285 (0.0008) [2023-12-26 21:26:07,435][105692] Updated weights for policy 0, policy_version 834295 (0.0008) [2023-12-26 21:26:07,488][105692] Updated weights for policy 0, policy_version 834305 (0.0008) [2023-12-26 21:26:07,840][105620] Updated weights for policy 1, policy_version 834143 (0.0007) [2023-12-26 21:26:07,889][105620] Updated weights for policy 1, policy_version 834153 (0.0005) [2023-12-26 21:26:07,938][105620] Updated weights for policy 1, policy_version 834163 (0.0005) [2023-12-26 21:26:08,405][105692] Updated weights for policy 0, policy_version 834315 (0.0008) [2023-12-26 21:26:08,462][105692] Updated weights for policy 0, policy_version 834325 (0.0009) [2023-12-26 21:26:08,508][105620] Updated weights for policy 1, policy_version 834173 (0.0007) [2023-12-26 21:26:08,518][105692] Updated weights for policy 0, policy_version 834335 (0.0008) [2023-12-26 21:26:08,567][105620] Updated weights for policy 1, policy_version 834183 (0.0011) [2023-12-26 21:26:08,630][105620] Updated weights for policy 1, policy_version 834193 (0.0011) [2023-12-26 21:26:09,293][105692] Updated weights for policy 0, policy_version 834345 (0.0006) [2023-12-26 21:26:09,355][105692] Updated weights for policy 0, policy_version 834355 (0.0008) [2023-12-26 21:26:09,367][105620] Updated weights for policy 1, policy_version 834203 (0.0011) [2023-12-26 21:26:09,422][105692] Updated weights for policy 0, policy_version 834365 (0.0007) [2023-12-26 21:26:09,432][105620] Updated weights for policy 1, policy_version 834213 (0.0011) [2023-12-26 21:26:09,485][105692] Updated weights for policy 0, policy_version 834375 (0.0007) [2023-12-26 21:26:09,499][105620] Updated weights for policy 1, policy_version 834223 (0.0011) [2023-12-26 21:26:10,170][105620] Updated weights for policy 1, policy_version 834233 (0.0011) [2023-12-26 21:26:10,233][105620] Updated weights for policy 1, policy_version 834243 (0.0011) [2023-12-26 21:26:10,290][105620] Updated weights for policy 1, policy_version 834253 (0.0011) [2023-12-26 21:26:10,296][105692] Updated weights for policy 0, policy_version 834385 (0.0006) [2023-12-26 21:26:10,355][105620] Updated weights for policy 1, policy_version 834263 (0.0011) [2023-12-26 21:26:10,361][105692] Updated weights for policy 0, policy_version 834395 (0.0006) [2023-12-26 21:26:10,416][105692] Updated weights for policy 0, policy_version 834405 (0.0008) [2023-12-26 21:26:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 427237376. Throughput: 0: 9429.7, 1: 9949.2. Samples: 427250616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:26:11,063][104569] Avg episode reward: [(0, '8581.520'), (1, '9164.467')] [2023-12-26 21:26:11,089][105620] Updated weights for policy 1, policy_version 834273 (0.0011) [2023-12-26 21:26:11,158][105620] Updated weights for policy 1, policy_version 834283 (0.0011) [2023-12-26 21:26:11,218][105620] Updated weights for policy 1, policy_version 834293 (0.0011) [2023-12-26 21:26:11,224][105692] Updated weights for policy 0, policy_version 834415 (0.0008) [2023-12-26 21:26:11,285][105692] Updated weights for policy 0, policy_version 834425 (0.0008) [2023-12-26 21:26:11,350][105692] Updated weights for policy 0, policy_version 834435 (0.0008) [2023-12-26 21:26:11,919][105620] Updated weights for policy 1, policy_version 834303 (0.0008) [2023-12-26 21:26:11,984][105620] Updated weights for policy 1, policy_version 834313 (0.0008) [2023-12-26 21:26:12,052][105620] Updated weights for policy 1, policy_version 834323 (0.0007) [2023-12-26 21:26:12,199][105692] Updated weights for policy 0, policy_version 834445 (0.0009) [2023-12-26 21:26:12,249][105692] Updated weights for policy 0, policy_version 834455 (0.0009) [2023-12-26 21:26:12,310][105692] Updated weights for policy 0, policy_version 834465 (0.0008) [2023-12-26 21:26:12,716][105620] Updated weights for policy 1, policy_version 834333 (0.0008) [2023-12-26 21:26:12,774][105620] Updated weights for policy 1, policy_version 834343 (0.0009) [2023-12-26 21:26:12,827][105620] Updated weights for policy 1, policy_version 834353 (0.0008) [2023-12-26 21:26:13,044][105692] Updated weights for policy 0, policy_version 834475 (0.0008) [2023-12-26 21:26:13,099][105692] Updated weights for policy 0, policy_version 834485 (0.0009) [2023-12-26 21:26:13,159][105692] Updated weights for policy 0, policy_version 834495 (0.0010) [2023-12-26 21:26:13,542][105620] Updated weights for policy 1, policy_version 834363 (0.0006) [2023-12-26 21:26:13,609][105620] Updated weights for policy 1, policy_version 834373 (0.0009) [2023-12-26 21:26:13,665][105620] Updated weights for policy 1, policy_version 834383 (0.0008) [2023-12-26 21:26:13,866][105692] Updated weights for policy 0, policy_version 834505 (0.0010) [2023-12-26 21:26:13,918][105692] Updated weights for policy 0, policy_version 834515 (0.0010) [2023-12-26 21:26:13,980][105692] Updated weights for policy 0, policy_version 834525 (0.0010) [2023-12-26 21:26:14,042][105692] Updated weights for policy 0, policy_version 834535 (0.0010) [2023-12-26 21:26:14,374][105620] Updated weights for policy 1, policy_version 834393 (0.0008) [2023-12-26 21:26:14,434][105620] Updated weights for policy 1, policy_version 834403 (0.0009) [2023-12-26 21:26:14,492][105620] Updated weights for policy 1, policy_version 834413 (0.0010) [2023-12-26 21:26:14,550][105620] Updated weights for policy 1, policy_version 834423 (0.0010) [2023-12-26 21:26:14,706][105692] Updated weights for policy 0, policy_version 834545 (0.0009) [2023-12-26 21:26:14,759][105692] Updated weights for policy 0, policy_version 834555 (0.0009) [2023-12-26 21:26:14,821][105692] Updated weights for policy 0, policy_version 834565 (0.0009) [2023-12-26 21:26:15,252][105620] Updated weights for policy 1, policy_version 834433 (0.0009) [2023-12-26 21:26:15,305][105620] Updated weights for policy 1, policy_version 834443 (0.0009) [2023-12-26 21:26:15,361][105620] Updated weights for policy 1, policy_version 834453 (0.0009) [2023-12-26 21:26:15,587][105692] Updated weights for policy 0, policy_version 834575 (0.0010) [2023-12-26 21:26:15,647][105692] Updated weights for policy 0, policy_version 834585 (0.0010) [2023-12-26 21:26:15,701][105692] Updated weights for policy 0, policy_version 834595 (0.0007) [2023-12-26 21:26:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 427335680. Throughput: 0: 9361.0, 1: 9847.3. Samples: 427307408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:26:16,062][104569] Avg episode reward: [(0, '8458.398'), (1, '9261.311')] [2023-12-26 21:26:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000834600_213688320.pth... [2023-12-26 21:26:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000833512_213409792.pth [2023-12-26 21:26:16,113][105620] Updated weights for policy 1, policy_version 834463 (0.0008) [2023-12-26 21:26:16,170][105620] Updated weights for policy 1, policy_version 834473 (0.0009) [2023-12-26 21:26:16,221][105620] Updated weights for policy 1, policy_version 834483 (0.0010) [2023-12-26 21:26:16,242][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000834488_213655552.pth... [2023-12-26 21:26:16,245][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000833304_213352448.pth [2023-12-26 21:26:16,301][105692] Updated weights for policy 0, policy_version 834605 (0.0007) [2023-12-26 21:26:16,350][105692] Updated weights for policy 0, policy_version 834615 (0.0008) [2023-12-26 21:26:16,405][105692] Updated weights for policy 0, policy_version 834625 (0.0008) [2023-12-26 21:26:16,931][105620] Updated weights for policy 1, policy_version 834493 (0.0008) [2023-12-26 21:26:16,992][105620] Updated weights for policy 1, policy_version 834503 (0.0005) [2023-12-26 21:26:17,050][105620] Updated weights for policy 1, policy_version 834513 (0.0006) [2023-12-26 21:26:17,139][105692] Updated weights for policy 0, policy_version 834635 (0.0008) [2023-12-26 21:26:17,204][105692] Updated weights for policy 0, policy_version 834645 (0.0006) [2023-12-26 21:26:17,264][105692] Updated weights for policy 0, policy_version 834655 (0.0010) [2023-12-26 21:26:17,689][105620] Updated weights for policy 1, policy_version 834523 (0.0009) [2023-12-26 21:26:17,740][105620] Updated weights for policy 1, policy_version 834533 (0.0005) [2023-12-26 21:26:17,789][105620] Updated weights for policy 1, policy_version 834543 (0.0006) [2023-12-26 21:26:17,937][105692] Updated weights for policy 0, policy_version 834665 (0.0010) [2023-12-26 21:26:17,990][105692] Updated weights for policy 0, policy_version 834675 (0.0010) [2023-12-26 21:26:18,045][105692] Updated weights for policy 0, policy_version 834685 (0.0009) [2023-12-26 21:26:18,093][105692] Updated weights for policy 0, policy_version 834695 (0.0008) [2023-12-26 21:26:18,431][105620] Updated weights for policy 1, policy_version 834553 (0.0006) [2023-12-26 21:26:18,490][105620] Updated weights for policy 1, policy_version 834563 (0.0010) [2023-12-26 21:26:18,512][105586] KL-divergence is very high: 187.9947 [2023-12-26 21:26:18,528][105586] KL-divergence is very high: 216.5518 [2023-12-26 21:26:18,552][105620] Updated weights for policy 1, policy_version 834573 (0.0010) [2023-12-26 21:26:18,566][105586] KL-divergence is very high: 220.4751 [2023-12-26 21:26:18,580][105586] KL-divergence is very high: 208.2921 [2023-12-26 21:26:18,618][105586] KL-divergence is very high: 133.5712 [2023-12-26 21:26:18,618][105620] Updated weights for policy 1, policy_version 834583 (0.0010) [2023-12-26 21:26:18,874][105692] Updated weights for policy 0, policy_version 834705 (0.0011) [2023-12-26 21:26:18,941][105692] Updated weights for policy 0, policy_version 834715 (0.0011) [2023-12-26 21:26:19,006][105692] Updated weights for policy 0, policy_version 834725 (0.0011) [2023-12-26 21:26:19,232][105620] Updated weights for policy 1, policy_version 834593 (0.0010) [2023-12-26 21:26:19,291][105620] Updated weights for policy 1, policy_version 834603 (0.0009) [2023-12-26 21:26:19,355][105620] Updated weights for policy 1, policy_version 834613 (0.0009) [2023-12-26 21:26:19,786][105692] Updated weights for policy 0, policy_version 834735 (0.0007) [2023-12-26 21:26:19,855][105692] Updated weights for policy 0, policy_version 834745 (0.0008) [2023-12-26 21:26:19,920][105692] Updated weights for policy 0, policy_version 834755 (0.0009) [2023-12-26 21:26:20,137][105620] Updated weights for policy 1, policy_version 834623 (0.0009) [2023-12-26 21:26:20,187][105620] Updated weights for policy 1, policy_version 834633 (0.0008) [2023-12-26 21:26:20,241][105620] Updated weights for policy 1, policy_version 834643 (0.0008) [2023-12-26 21:26:20,720][105692] Updated weights for policy 0, policy_version 834765 (0.0010) [2023-12-26 21:26:20,776][105692] Updated weights for policy 0, policy_version 834775 (0.0011) [2023-12-26 21:26:20,835][105692] Updated weights for policy 0, policy_version 834785 (0.0010) [2023-12-26 21:26:20,971][105620] Updated weights for policy 1, policy_version 834653 (0.0009) [2023-12-26 21:26:21,038][105620] Updated weights for policy 1, policy_version 834663 (0.0009) [2023-12-26 21:26:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19438.7). Total num frames: 427433984. Throughput: 0: 9317.2, 1: 9879.5. Samples: 427425608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:26:21,063][104569] Avg episode reward: [(0, '8097.123'), (1, '8438.120')] [2023-12-26 21:26:21,102][105620] Updated weights for policy 1, policy_version 834673 (0.0008) [2023-12-26 21:26:21,657][105692] Updated weights for policy 0, policy_version 834795 (0.0010) [2023-12-26 21:26:21,721][105692] Updated weights for policy 0, policy_version 834805 (0.0007) [2023-12-26 21:26:21,784][105692] Updated weights for policy 0, policy_version 834815 (0.0006) [2023-12-26 21:26:21,946][105620] Updated weights for policy 1, policy_version 834683 (0.0009) [2023-12-26 21:26:22,010][105620] Updated weights for policy 1, policy_version 834693 (0.0011) [2023-12-26 21:26:22,070][105620] Updated weights for policy 1, policy_version 834703 (0.0011) [2023-12-26 21:26:22,446][105692] Updated weights for policy 0, policy_version 834825 (0.0006) [2023-12-26 21:26:22,500][105692] Updated weights for policy 0, policy_version 834835 (0.0008) [2023-12-26 21:26:22,557][105692] Updated weights for policy 0, policy_version 834845 (0.0008) [2023-12-26 21:26:22,613][105692] Updated weights for policy 0, policy_version 834855 (0.0009) [2023-12-26 21:26:22,820][105620] Updated weights for policy 1, policy_version 834713 (0.0010) [2023-12-26 21:26:22,890][105620] Updated weights for policy 1, policy_version 834723 (0.0006) [2023-12-26 21:26:22,958][105620] Updated weights for policy 1, policy_version 834733 (0.0006) [2023-12-26 21:26:23,028][105620] Updated weights for policy 1, policy_version 834743 (0.0006) [2023-12-26 21:26:23,332][105692] Updated weights for policy 0, policy_version 834865 (0.0010) [2023-12-26 21:26:23,391][105692] Updated weights for policy 0, policy_version 834875 (0.0010) [2023-12-26 21:26:23,445][105692] Updated weights for policy 0, policy_version 834885 (0.0009) [2023-12-26 21:26:23,558][105620] Updated weights for policy 1, policy_version 834753 (0.0005) [2023-12-26 21:26:23,629][105620] Updated weights for policy 1, policy_version 834763 (0.0006) [2023-12-26 21:26:23,683][105620] Updated weights for policy 1, policy_version 834773 (0.0009) [2023-12-26 21:26:24,164][105692] Updated weights for policy 0, policy_version 834895 (0.0007) [2023-12-26 21:26:24,216][105692] Updated weights for policy 0, policy_version 834905 (0.0005) [2023-12-26 21:26:24,275][105692] Updated weights for policy 0, policy_version 834915 (0.0005) [2023-12-26 21:26:24,387][105620] Updated weights for policy 1, policy_version 834783 (0.0007) [2023-12-26 21:26:24,434][105620] Updated weights for policy 1, policy_version 834793 (0.0005) [2023-12-26 21:26:24,483][105620] Updated weights for policy 1, policy_version 834803 (0.0005) [2023-12-26 21:26:24,795][105692] Updated weights for policy 0, policy_version 834925 (0.0005) [2023-12-26 21:26:24,849][105692] Updated weights for policy 0, policy_version 834935 (0.0010) [2023-12-26 21:26:24,902][105692] Updated weights for policy 0, policy_version 834946 (0.0010) [2023-12-26 21:26:25,018][105620] Updated weights for policy 1, policy_version 834813 (0.0007) [2023-12-26 21:26:25,067][105620] Updated weights for policy 1, policy_version 834823 (0.0008) [2023-12-26 21:26:25,116][105620] Updated weights for policy 1, policy_version 834833 (0.0008) [2023-12-26 21:26:25,674][105692] Updated weights for policy 0, policy_version 834956 (0.0010) [2023-12-26 21:26:25,722][105692] Updated weights for policy 0, policy_version 834966 (0.0010) [2023-12-26 21:26:25,777][105692] Updated weights for policy 0, policy_version 834976 (0.0010) [2023-12-26 21:26:25,916][105620] Updated weights for policy 1, policy_version 834843 (0.0009) [2023-12-26 21:26:25,965][105620] Updated weights for policy 1, policy_version 834853 (0.0008) [2023-12-26 21:26:26,010][105620] Updated weights for policy 1, policy_version 834863 (0.0008) [2023-12-26 21:26:26,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 427540480. Throughput: 0: 9340.9, 1: 9926.4. Samples: 427544040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:26:26,063][104569] Avg episode reward: [(0, '8000.323'), (1, '8552.900')] [2023-12-26 21:26:26,530][105692] Updated weights for policy 0, policy_version 834986 (0.0010) [2023-12-26 21:26:26,588][105692] Updated weights for policy 0, policy_version 834996 (0.0010) [2023-12-26 21:26:26,650][105692] Updated weights for policy 0, policy_version 835006 (0.0010) [2023-12-26 21:26:26,705][105692] Updated weights for policy 0, policy_version 835016 (0.0010) [2023-12-26 21:26:26,796][105620] Updated weights for policy 1, policy_version 834873 (0.0008) [2023-12-26 21:26:26,848][105620] Updated weights for policy 1, policy_version 834883 (0.0008) [2023-12-26 21:26:26,891][105620] Updated weights for policy 1, policy_version 834893 (0.0007) [2023-12-26 21:26:26,939][105620] Updated weights for policy 1, policy_version 834903 (0.0008) [2023-12-26 21:26:27,448][105692] Updated weights for policy 0, policy_version 835026 (0.0009) [2023-12-26 21:26:27,491][105692] Updated weights for policy 0, policy_version 835036 (0.0005) [2023-12-26 21:26:27,538][105692] Updated weights for policy 0, policy_version 835046 (0.0006) [2023-12-26 21:26:27,716][105620] Updated weights for policy 1, policy_version 834913 (0.0008) [2023-12-26 21:26:27,765][105620] Updated weights for policy 1, policy_version 834923 (0.0008) [2023-12-26 21:26:27,820][105620] Updated weights for policy 1, policy_version 834933 (0.0009) [2023-12-26 21:26:28,239][105692] Updated weights for policy 0, policy_version 835056 (0.0006) [2023-12-26 21:26:28,286][105692] Updated weights for policy 0, policy_version 835066 (0.0006) [2023-12-26 21:26:28,345][105692] Updated weights for policy 0, policy_version 835076 (0.0006) [2023-12-26 21:26:28,526][105620] Updated weights for policy 1, policy_version 834943 (0.0010) [2023-12-26 21:26:28,581][105620] Updated weights for policy 1, policy_version 834953 (0.0010) [2023-12-26 21:26:28,636][105620] Updated weights for policy 1, policy_version 834963 (0.0009) [2023-12-26 21:26:29,022][105692] Updated weights for policy 0, policy_version 835086 (0.0007) [2023-12-26 21:26:29,078][105692] Updated weights for policy 0, policy_version 835096 (0.0006) [2023-12-26 21:26:29,143][105692] Updated weights for policy 0, policy_version 835106 (0.0005) [2023-12-26 21:26:29,355][105620] Updated weights for policy 1, policy_version 834973 (0.0007) [2023-12-26 21:26:29,412][105620] Updated weights for policy 1, policy_version 834983 (0.0009) [2023-12-26 21:26:29,471][105620] Updated weights for policy 1, policy_version 834994 (0.0010) [2023-12-26 21:26:29,763][105692] Updated weights for policy 0, policy_version 835116 (0.0007) [2023-12-26 21:26:29,813][105692] Updated weights for policy 0, policy_version 835126 (0.0006) [2023-12-26 21:26:29,875][105692] Updated weights for policy 0, policy_version 835136 (0.0006) [2023-12-26 21:26:30,279][105620] Updated weights for policy 1, policy_version 835004 (0.0008) [2023-12-26 21:26:30,341][105620] Updated weights for policy 1, policy_version 835014 (0.0008) [2023-12-26 21:26:30,399][105620] Updated weights for policy 1, policy_version 835024 (0.0009) [2023-12-26 21:26:30,531][105692] Updated weights for policy 0, policy_version 835146 (0.0008) [2023-12-26 21:26:30,578][105692] Updated weights for policy 0, policy_version 835156 (0.0009) [2023-12-26 21:26:30,632][105692] Updated weights for policy 0, policy_version 835166 (0.0009) [2023-12-26 21:26:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 427630592. Throughput: 0: 9396.0, 1: 9902.1. Samples: 427601296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:26:31,063][104569] Avg episode reward: [(0, '7816.236'), (1, '8838.535')] [2023-12-26 21:26:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000835176_213835776.pth... [2023-12-26 21:26:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000835032_213794816.pth... [2023-12-26 21:26:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000833912_213508096.pth [2023-12-26 21:26:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000834056_213549056.pth [2023-12-26 21:26:31,157][105620] Updated weights for policy 1, policy_version 835034 (0.0009) [2023-12-26 21:26:31,205][105620] Updated weights for policy 1, policy_version 835044 (0.0009) [2023-12-26 21:26:31,263][105620] Updated weights for policy 1, policy_version 835054 (0.0009) [2023-12-26 21:26:31,331][105620] Updated weights for policy 1, policy_version 835064 (0.0009) [2023-12-26 21:26:31,357][105692] Updated weights for policy 0, policy_version 835177 (0.0010) [2023-12-26 21:26:31,420][105692] Updated weights for policy 0, policy_version 835187 (0.0010) [2023-12-26 21:26:31,471][105692] Updated weights for policy 0, policy_version 835197 (0.0010) [2023-12-26 21:26:31,526][105692] Updated weights for policy 0, policy_version 835207 (0.0010) [2023-12-26 21:26:32,106][105620] Updated weights for policy 1, policy_version 835074 (0.0007) [2023-12-26 21:26:32,173][105620] Updated weights for policy 1, policy_version 835084 (0.0009) [2023-12-26 21:26:32,209][105692] Updated weights for policy 0, policy_version 835217 (0.0010) [2023-12-26 21:26:32,229][105620] Updated weights for policy 1, policy_version 835094 (0.0007) [2023-12-26 21:26:32,277][105692] Updated weights for policy 0, policy_version 835227 (0.0010) [2023-12-26 21:26:32,340][105692] Updated weights for policy 0, policy_version 835237 (0.0009) [2023-12-26 21:26:32,935][105620] Updated weights for policy 1, policy_version 835104 (0.0006) [2023-12-26 21:26:32,957][105692] Updated weights for policy 0, policy_version 835247 (0.0010) [2023-12-26 21:26:32,985][105620] Updated weights for policy 1, policy_version 835114 (0.0005) [2023-12-26 21:26:33,020][105692] Updated weights for policy 0, policy_version 835257 (0.0010) [2023-12-26 21:26:33,037][105620] Updated weights for policy 1, policy_version 835124 (0.0005) [2023-12-26 21:26:33,078][105692] Updated weights for policy 0, policy_version 835267 (0.0010) [2023-12-26 21:26:33,638][105692] Updated weights for policy 0, policy_version 835277 (0.0008) [2023-12-26 21:26:33,674][105620] Updated weights for policy 1, policy_version 835134 (0.0005) [2023-12-26 21:26:33,684][105692] Updated weights for policy 0, policy_version 835287 (0.0006) [2023-12-26 21:26:33,733][105692] Updated weights for policy 0, policy_version 835297 (0.0007) [2023-12-26 21:26:33,735][105620] Updated weights for policy 1, policy_version 835144 (0.0005) [2023-12-26 21:26:33,800][105620] Updated weights for policy 1, policy_version 835154 (0.0005) [2023-12-26 21:26:34,338][105692] Updated weights for policy 0, policy_version 835307 (0.0008) [2023-12-26 21:26:34,352][105620] Updated weights for policy 1, policy_version 835164 (0.0005) [2023-12-26 21:26:34,396][105692] Updated weights for policy 0, policy_version 835317 (0.0010) [2023-12-26 21:26:34,403][105620] Updated weights for policy 1, policy_version 835174 (0.0006) [2023-12-26 21:26:34,449][105620] Updated weights for policy 1, policy_version 835184 (0.0010) [2023-12-26 21:26:34,462][105692] Updated weights for policy 0, policy_version 835327 (0.0010) [2023-12-26 21:26:35,121][105692] Updated weights for policy 0, policy_version 835337 (0.0010) [2023-12-26 21:26:35,176][105692] Updated weights for policy 0, policy_version 835347 (0.0010) [2023-12-26 21:26:35,199][105620] Updated weights for policy 1, policy_version 835194 (0.0008) [2023-12-26 21:26:35,250][105620] Updated weights for policy 1, policy_version 835204 (0.0010) [2023-12-26 21:26:35,261][105692] Updated weights for policy 0, policy_version 835357 (0.0010) [2023-12-26 21:26:35,308][105620] Updated weights for policy 1, policy_version 835214 (0.0010) [2023-12-26 21:26:35,319][105692] Updated weights for policy 0, policy_version 835367 (0.0010) [2023-12-26 21:26:35,363][105620] Updated weights for policy 1, policy_version 835224 (0.0010) [2023-12-26 21:26:35,903][105692] Updated weights for policy 0, policy_version 835377 (0.0006) [2023-12-26 21:26:35,938][105620] Updated weights for policy 1, policy_version 835234 (0.0005) [2023-12-26 21:26:35,959][105692] Updated weights for policy 0, policy_version 835387 (0.0010) [2023-12-26 21:26:35,994][105620] Updated weights for policy 1, policy_version 835244 (0.0005) [2023-12-26 21:26:36,017][105692] Updated weights for policy 0, policy_version 835397 (0.0010) [2023-12-26 21:26:36,050][105620] Updated weights for policy 1, policy_version 835254 (0.0005) [2023-12-26 21:26:36,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 427745280. Throughput: 0: 9612.6, 1: 9882.2. Samples: 427724012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:26:36,062][104569] Avg episode reward: [(0, '8084.024'), (1, '8972.152')] [2023-12-26 21:26:36,676][105692] Updated weights for policy 0, policy_version 835407 (0.0007) [2023-12-26 21:26:36,696][105620] Updated weights for policy 1, policy_version 835264 (0.0009) [2023-12-26 21:26:36,744][105692] Updated weights for policy 0, policy_version 835417 (0.0006) [2023-12-26 21:26:36,755][105620] Updated weights for policy 1, policy_version 835274 (0.0006) [2023-12-26 21:26:36,812][105620] Updated weights for policy 1, policy_version 835284 (0.0009) [2023-12-26 21:26:36,812][105692] Updated weights for policy 0, policy_version 835427 (0.0005) [2023-12-26 21:26:37,392][105692] Updated weights for policy 0, policy_version 835437 (0.0006) [2023-12-26 21:26:37,446][105692] Updated weights for policy 0, policy_version 835447 (0.0005) [2023-12-26 21:26:37,470][105620] Updated weights for policy 1, policy_version 835294 (0.0008) [2023-12-26 21:26:37,496][105586] KL-divergence is very high: 339.7438 [2023-12-26 21:26:37,512][105692] Updated weights for policy 0, policy_version 835457 (0.0007) [2023-12-26 21:26:37,528][105620] Updated weights for policy 1, policy_version 835304 (0.0007) [2023-12-26 21:26:37,548][105586] KL-divergence is very high: 574.2443 [2023-12-26 21:26:37,593][105620] Updated weights for policy 1, policy_version 835314 (0.0008) [2023-12-26 21:26:37,601][105586] KL-divergence is very high: 582.3039 [2023-12-26 21:26:38,228][105692] Updated weights for policy 0, policy_version 835467 (0.0007) [2023-12-26 21:26:38,282][105692] Updated weights for policy 0, policy_version 835477 (0.0005) [2023-12-26 21:26:38,322][105620] Updated weights for policy 1, policy_version 835324 (0.0010) [2023-12-26 21:26:38,340][105692] Updated weights for policy 0, policy_version 835487 (0.0007) [2023-12-26 21:26:38,383][105620] Updated weights for policy 1, policy_version 835334 (0.0009) [2023-12-26 21:26:38,440][105620] Updated weights for policy 1, policy_version 835344 (0.0011) [2023-12-26 21:26:39,006][105692] Updated weights for policy 0, policy_version 835497 (0.0007) [2023-12-26 21:26:39,056][105692] Updated weights for policy 0, policy_version 835507 (0.0007) [2023-12-26 21:26:39,122][105692] Updated weights for policy 0, policy_version 835517 (0.0008) [2023-12-26 21:26:39,178][105692] Updated weights for policy 0, policy_version 835527 (0.0009) [2023-12-26 21:26:39,196][105620] Updated weights for policy 1, policy_version 835354 (0.0010) [2023-12-26 21:26:39,262][105620] Updated weights for policy 1, policy_version 835364 (0.0008) [2023-12-26 21:26:39,336][105620] Updated weights for policy 1, policy_version 835374 (0.0008) [2023-12-26 21:26:39,873][105692] Updated weights for policy 0, policy_version 835537 (0.0008) [2023-12-26 21:26:39,926][105692] Updated weights for policy 0, policy_version 835548 (0.0008) [2023-12-26 21:26:39,994][105692] Updated weights for policy 0, policy_version 835558 (0.0006) [2023-12-26 21:26:40,085][105620] Updated weights for policy 1, policy_version 835386 (0.0010) [2023-12-26 21:26:40,155][105620] Updated weights for policy 1, policy_version 835396 (0.0011) [2023-12-26 21:26:40,214][105620] Updated weights for policy 1, policy_version 835406 (0.0009) [2023-12-26 21:26:40,280][105620] Updated weights for policy 1, policy_version 835416 (0.0009) [2023-12-26 21:26:40,608][105692] Updated weights for policy 0, policy_version 835568 (0.0010) [2023-12-26 21:26:40,668][105692] Updated weights for policy 0, policy_version 835578 (0.0010) [2023-12-26 21:26:40,728][105692] Updated weights for policy 0, policy_version 835588 (0.0010) [2023-12-26 21:26:40,906][105620] Updated weights for policy 1, policy_version 835426 (0.0011) [2023-12-26 21:26:40,972][105620] Updated weights for policy 1, policy_version 835436 (0.0010) [2023-12-26 21:26:41,047][105620] Updated weights for policy 1, policy_version 835446 (0.0010) [2023-12-26 21:26:41,062][104569] Fps is (10 sec: 21299.4, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 427843584. Throughput: 0: 9779.7, 1: 9869.5. Samples: 427847600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:26:41,063][104569] Avg episode reward: [(0, '8443.544'), (1, '9077.813')] [2023-12-26 21:26:41,543][105692] Updated weights for policy 0, policy_version 835598 (0.0011) [2023-12-26 21:26:41,593][105692] Updated weights for policy 0, policy_version 835608 (0.0011) [2023-12-26 21:26:41,661][105692] Updated weights for policy 0, policy_version 835618 (0.0010) [2023-12-26 21:26:41,788][105620] Updated weights for policy 1, policy_version 835456 (0.0006) [2023-12-26 21:26:41,846][105620] Updated weights for policy 1, policy_version 835466 (0.0008) [2023-12-26 21:26:41,911][105620] Updated weights for policy 1, policy_version 835476 (0.0009) [2023-12-26 21:26:42,449][105692] Updated weights for policy 0, policy_version 835628 (0.0012) [2023-12-26 21:26:42,512][105692] Updated weights for policy 0, policy_version 835638 (0.0008) [2023-12-26 21:26:42,579][105692] Updated weights for policy 0, policy_version 835648 (0.0007) [2023-12-26 21:26:42,630][105620] Updated weights for policy 1, policy_version 835486 (0.0010) [2023-12-26 21:26:42,687][105620] Updated weights for policy 1, policy_version 835496 (0.0008) [2023-12-26 21:26:42,771][105620] Updated weights for policy 1, policy_version 835506 (0.0011) [2023-12-26 21:26:43,308][105692] Updated weights for policy 0, policy_version 835658 (0.0007) [2023-12-26 21:26:43,365][105692] Updated weights for policy 0, policy_version 835668 (0.0008) [2023-12-26 21:26:43,424][105620] Updated weights for policy 1, policy_version 835516 (0.0010) [2023-12-26 21:26:43,426][105692] Updated weights for policy 0, policy_version 835678 (0.0007) [2023-12-26 21:26:43,469][105620] Updated weights for policy 1, policy_version 835526 (0.0010) [2023-12-26 21:26:43,478][105692] Updated weights for policy 0, policy_version 835688 (0.0010) [2023-12-26 21:26:43,517][105620] Updated weights for policy 1, policy_version 835536 (0.0010) [2023-12-26 21:26:44,168][105620] Updated weights for policy 1, policy_version 835546 (0.0011) [2023-12-26 21:26:44,220][105692] Updated weights for policy 0, policy_version 835698 (0.0011) [2023-12-26 21:26:44,236][105620] Updated weights for policy 1, policy_version 835556 (0.0007) [2023-12-26 21:26:44,277][105692] Updated weights for policy 0, policy_version 835708 (0.0006) [2023-12-26 21:26:44,300][105620] Updated weights for policy 1, policy_version 835566 (0.0011) [2023-12-26 21:26:44,328][105692] Updated weights for policy 0, policy_version 835718 (0.0006) [2023-12-26 21:26:44,360][105620] Updated weights for policy 1, policy_version 835576 (0.0011) [2023-12-26 21:26:45,033][105692] Updated weights for policy 0, policy_version 835728 (0.0011) [2023-12-26 21:26:45,096][105692] Updated weights for policy 0, policy_version 835738 (0.0010) [2023-12-26 21:26:45,116][105620] Updated weights for policy 1, policy_version 835586 (0.0008) [2023-12-26 21:26:45,157][105692] Updated weights for policy 0, policy_version 835748 (0.0010) [2023-12-26 21:26:45,179][105620] Updated weights for policy 1, policy_version 835596 (0.0007) [2023-12-26 21:26:45,239][105620] Updated weights for policy 1, policy_version 835606 (0.0008) [2023-12-26 21:26:45,808][105692] Updated weights for policy 0, policy_version 835758 (0.0007) [2023-12-26 21:26:45,874][105692] Updated weights for policy 0, policy_version 835768 (0.0005) [2023-12-26 21:26:45,909][105620] Updated weights for policy 1, policy_version 835616 (0.0010) [2023-12-26 21:26:45,930][105692] Updated weights for policy 0, policy_version 835778 (0.0005) [2023-12-26 21:26:45,967][105620] Updated weights for policy 1, policy_version 835626 (0.0010) [2023-12-26 21:26:46,027][105620] Updated weights for policy 1, policy_version 835636 (0.0011) [2023-12-26 21:26:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 427941888. Throughput: 0: 9710.2, 1: 9827.0. Samples: 427904392. Policy #0 lag: (min: 23.0, avg: 36.7, max: 55.0) [2023-12-26 21:26:46,063][104569] Avg episode reward: [(0, '8035.338'), (1, '9169.175')] [2023-12-26 21:26:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000835640_213950464.pth... [2023-12-26 21:26:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000835784_213991424.pth... [2023-12-26 21:26:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000834488_213655552.pth [2023-12-26 21:26:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000834600_213688320.pth [2023-12-26 21:26:46,607][105692] Updated weights for policy 0, policy_version 835788 (0.0007) [2023-12-26 21:26:46,653][105692] Updated weights for policy 0, policy_version 835798 (0.0009) [2023-12-26 21:26:46,668][105620] Updated weights for policy 1, policy_version 835646 (0.0009) [2023-12-26 21:26:46,703][105692] Updated weights for policy 0, policy_version 835808 (0.0007) [2023-12-26 21:26:46,727][105620] Updated weights for policy 1, policy_version 835656 (0.0008) [2023-12-26 21:26:46,788][105620] Updated weights for policy 1, policy_version 835666 (0.0009) [2023-12-26 21:26:47,439][105692] Updated weights for policy 0, policy_version 835818 (0.0006) [2023-12-26 21:26:47,459][105620] Updated weights for policy 1, policy_version 835676 (0.0008) [2023-12-26 21:26:47,496][105692] Updated weights for policy 0, policy_version 835828 (0.0005) [2023-12-26 21:26:47,520][105620] Updated weights for policy 1, policy_version 835686 (0.0009) [2023-12-26 21:26:47,552][105692] Updated weights for policy 0, policy_version 835838 (0.0007) [2023-12-26 21:26:47,582][105620] Updated weights for policy 1, policy_version 835696 (0.0008) [2023-12-26 21:26:47,613][105692] Updated weights for policy 0, policy_version 835848 (0.0006) [2023-12-26 21:26:48,158][105692] Updated weights for policy 0, policy_version 835858 (0.0006) [2023-12-26 21:26:48,212][105692] Updated weights for policy 0, policy_version 835868 (0.0005) [2023-12-26 21:26:48,272][105692] Updated weights for policy 0, policy_version 835878 (0.0005) [2023-12-26 21:26:48,481][105620] Updated weights for policy 1, policy_version 835706 (0.0009) [2023-12-26 21:26:48,533][105620] Updated weights for policy 1, policy_version 835716 (0.0008) [2023-12-26 21:26:48,584][105620] Updated weights for policy 1, policy_version 835726 (0.0008) [2023-12-26 21:26:48,640][105620] Updated weights for policy 1, policy_version 835736 (0.0008) [2023-12-26 21:26:48,958][105692] Updated weights for policy 0, policy_version 835888 (0.0009) [2023-12-26 21:26:49,005][105692] Updated weights for policy 0, policy_version 835898 (0.0010) [2023-12-26 21:26:49,050][105692] Updated weights for policy 0, policy_version 835908 (0.0010) [2023-12-26 21:26:49,445][105620] Updated weights for policy 1, policy_version 835746 (0.0009) [2023-12-26 21:26:49,502][105620] Updated weights for policy 1, policy_version 835756 (0.0008) [2023-12-26 21:26:49,549][105586] KL-divergence is very high: 117.1147 [2023-12-26 21:26:49,562][105620] Updated weights for policy 1, policy_version 835766 (0.0009) [2023-12-26 21:26:49,758][105692] Updated weights for policy 0, policy_version 835918 (0.0008) [2023-12-26 21:26:49,819][105692] Updated weights for policy 0, policy_version 835928 (0.0010) [2023-12-26 21:26:49,884][105692] Updated weights for policy 0, policy_version 835938 (0.0010) [2023-12-26 21:26:50,317][105620] Updated weights for policy 1, policy_version 835776 (0.0008) [2023-12-26 21:26:50,372][105620] Updated weights for policy 1, policy_version 835786 (0.0008) [2023-12-26 21:26:50,428][105620] Updated weights for policy 1, policy_version 835796 (0.0008) [2023-12-26 21:26:50,663][105692] Updated weights for policy 0, policy_version 835948 (0.0010) [2023-12-26 21:26:50,723][105692] Updated weights for policy 0, policy_version 835958 (0.0009) [2023-12-26 21:26:50,783][105692] Updated weights for policy 0, policy_version 835968 (0.0009) [2023-12-26 21:26:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 428032000. Throughput: 0: 9844.8, 1: 9816.9. Samples: 428022796. Policy #0 lag: (min: 23.0, avg: 36.7, max: 55.0) [2023-12-26 21:26:51,062][104569] Avg episode reward: [(0, '8164.581'), (1, '9171.313')] [2023-12-26 21:26:51,244][105620] Updated weights for policy 1, policy_version 835806 (0.0009) [2023-12-26 21:26:51,306][105620] Updated weights for policy 1, policy_version 835816 (0.0009) [2023-12-26 21:26:51,373][105620] Updated weights for policy 1, policy_version 835826 (0.0009) [2023-12-26 21:26:51,510][105692] Updated weights for policy 0, policy_version 835978 (0.0008) [2023-12-26 21:26:51,568][105692] Updated weights for policy 0, policy_version 835988 (0.0005) [2023-12-26 21:26:51,639][105692] Updated weights for policy 0, policy_version 835998 (0.0007) [2023-12-26 21:26:51,700][105692] Updated weights for policy 0, policy_version 836008 (0.0009) [2023-12-26 21:26:52,169][105620] Updated weights for policy 1, policy_version 835836 (0.0009) [2023-12-26 21:26:52,216][105620] Updated weights for policy 1, policy_version 835846 (0.0009) [2023-12-26 21:26:52,270][105620] Updated weights for policy 1, policy_version 835856 (0.0009) [2023-12-26 21:26:52,416][105692] Updated weights for policy 0, policy_version 836018 (0.0009) [2023-12-26 21:26:52,469][105692] Updated weights for policy 0, policy_version 836028 (0.0008) [2023-12-26 21:26:52,519][105692] Updated weights for policy 0, policy_version 836038 (0.0008) [2023-12-26 21:26:53,011][105620] Updated weights for policy 1, policy_version 835866 (0.0008) [2023-12-26 21:26:53,064][105620] Updated weights for policy 1, policy_version 835876 (0.0010) [2023-12-26 21:26:53,118][105620] Updated weights for policy 1, policy_version 835886 (0.0010) [2023-12-26 21:26:53,215][105692] Updated weights for policy 0, policy_version 836048 (0.0009) [2023-12-26 21:26:53,265][105692] Updated weights for policy 0, policy_version 836058 (0.0009) [2023-12-26 21:26:53,315][105692] Updated weights for policy 0, policy_version 836068 (0.0008) [2023-12-26 21:26:53,766][105620] Updated weights for policy 1, policy_version 835897 (0.0009) [2023-12-26 21:26:53,810][105620] Updated weights for policy 1, policy_version 835907 (0.0005) [2023-12-26 21:26:53,857][105620] Updated weights for policy 1, policy_version 835917 (0.0006) [2023-12-26 21:26:53,909][105620] Updated weights for policy 1, policy_version 835927 (0.0008) [2023-12-26 21:26:54,190][105692] Updated weights for policy 0, policy_version 836078 (0.0009) [2023-12-26 21:26:54,245][105692] Updated weights for policy 0, policy_version 836088 (0.0009) [2023-12-26 21:26:54,307][105692] Updated weights for policy 0, policy_version 836098 (0.0009) [2023-12-26 21:26:54,547][105620] Updated weights for policy 1, policy_version 835937 (0.0008) [2023-12-26 21:26:54,597][105620] Updated weights for policy 1, policy_version 835947 (0.0009) [2023-12-26 21:26:54,647][105620] Updated weights for policy 1, policy_version 835957 (0.0009) [2023-12-26 21:26:55,042][105692] Updated weights for policy 0, policy_version 836108 (0.0009) [2023-12-26 21:26:55,099][105692] Updated weights for policy 0, policy_version 836118 (0.0009) [2023-12-26 21:26:55,158][105692] Updated weights for policy 0, policy_version 836128 (0.0009) [2023-12-26 21:26:55,376][105620] Updated weights for policy 1, policy_version 835967 (0.0009) [2023-12-26 21:26:55,436][105620] Updated weights for policy 1, policy_version 835977 (0.0008) [2023-12-26 21:26:55,500][105620] Updated weights for policy 1, policy_version 835987 (0.0008) [2023-12-26 21:26:55,981][105692] Updated weights for policy 0, policy_version 836138 (0.0010) [2023-12-26 21:26:56,035][105692] Updated weights for policy 0, policy_version 836148 (0.0010) [2023-12-26 21:26:56,062][104569] Fps is (10 sec: 18022.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 428122112. Throughput: 0: 9899.3, 1: 9795.9. Samples: 428136896. Policy #0 lag: (min: 23.0, avg: 36.7, max: 55.0) [2023-12-26 21:26:56,062][104569] Avg episode reward: [(0, '8443.666'), (1, '8903.680')] [2023-12-26 21:26:56,097][105692] Updated weights for policy 0, policy_version 836158 (0.0009) [2023-12-26 21:26:56,117][105620] Updated weights for policy 1, policy_version 835997 (0.0008) [2023-12-26 21:26:56,151][105692] Updated weights for policy 0, policy_version 836168 (0.0007) [2023-12-26 21:26:56,175][105620] Updated weights for policy 1, policy_version 836007 (0.0009) [2023-12-26 21:26:56,229][105620] Updated weights for policy 1, policy_version 836017 (0.0008) [2023-12-26 21:26:56,860][105620] Updated weights for policy 1, policy_version 836027 (0.0008) [2023-12-26 21:26:56,906][105620] Updated weights for policy 1, policy_version 836037 (0.0005) [2023-12-26 21:26:56,962][105620] Updated weights for policy 1, policy_version 836047 (0.0006) [2023-12-26 21:26:56,976][105692] Updated weights for policy 0, policy_version 836178 (0.0009) [2023-12-26 21:26:57,029][105692] Updated weights for policy 0, policy_version 836188 (0.0008) [2023-12-26 21:26:57,086][105692] Updated weights for policy 0, policy_version 836198 (0.0008) [2023-12-26 21:26:57,559][105620] Updated weights for policy 1, policy_version 836057 (0.0007) [2023-12-26 21:26:57,604][105620] Updated weights for policy 1, policy_version 836067 (0.0009) [2023-12-26 21:26:57,651][105620] Updated weights for policy 1, policy_version 836077 (0.0009) [2023-12-26 21:26:57,707][105620] Updated weights for policy 1, policy_version 836087 (0.0008) [2023-12-26 21:26:57,882][105692] Updated weights for policy 0, policy_version 836208 (0.0009) [2023-12-26 21:26:57,941][105692] Updated weights for policy 0, policy_version 836218 (0.0009) [2023-12-26 21:26:58,005][105692] Updated weights for policy 0, policy_version 836228 (0.0009) [2023-12-26 21:26:58,484][105620] Updated weights for policy 1, policy_version 836097 (0.0008) [2023-12-26 21:26:58,541][105620] Updated weights for policy 1, policy_version 836107 (0.0008) [2023-12-26 21:26:58,610][105620] Updated weights for policy 1, policy_version 836117 (0.0009) [2023-12-26 21:26:58,827][105692] Updated weights for policy 0, policy_version 836238 (0.0008) [2023-12-26 21:26:58,892][105692] Updated weights for policy 0, policy_version 836248 (0.0007) [2023-12-26 21:26:58,965][105692] Updated weights for policy 0, policy_version 836258 (0.0008) [2023-12-26 21:26:59,490][105620] Updated weights for policy 1, policy_version 836127 (0.0011) [2023-12-26 21:26:59,545][105620] Updated weights for policy 1, policy_version 836137 (0.0010) [2023-12-26 21:26:59,603][105620] Updated weights for policy 1, policy_version 836147 (0.0009) [2023-12-26 21:26:59,778][105692] Updated weights for policy 0, policy_version 836268 (0.0009) [2023-12-26 21:26:59,841][105692] Updated weights for policy 0, policy_version 836278 (0.0008) [2023-12-26 21:26:59,911][105692] Updated weights for policy 0, policy_version 836288 (0.0010) [2023-12-26 21:27:00,238][105620] Updated weights for policy 1, policy_version 836157 (0.0008) [2023-12-26 21:27:00,282][105620] Updated weights for policy 1, policy_version 836167 (0.0010) [2023-12-26 21:27:00,327][105620] Updated weights for policy 1, policy_version 836177 (0.0010) [2023-12-26 21:27:00,721][105692] Updated weights for policy 0, policy_version 836298 (0.0010) [2023-12-26 21:27:00,779][105692] Updated weights for policy 0, policy_version 836308 (0.0010) [2023-12-26 21:27:00,839][105692] Updated weights for policy 0, policy_version 836319 (0.0011) [2023-12-26 21:27:00,957][105620] Updated weights for policy 1, policy_version 836187 (0.0010) [2023-12-26 21:27:01,014][105620] Updated weights for policy 1, policy_version 836197 (0.0006) [2023-12-26 21:27:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 428220416. Throughput: 0: 9863.3, 1: 9815.8. Samples: 428192968. Policy #0 lag: (min: 23.0, avg: 36.7, max: 55.0) [2023-12-26 21:27:01,063][104569] Avg episode reward: [(0, '7991.503'), (1, '8811.508')] [2023-12-26 21:27:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000836328_214130688.pth... [2023-12-26 21:27:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000835176_213835776.pth [2023-12-26 21:27:01,078][105620] Updated weights for policy 1, policy_version 836207 (0.0008) [2023-12-26 21:27:01,130][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000836216_214097920.pth... [2023-12-26 21:27:01,133][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000835032_213794816.pth [2023-12-26 21:27:01,677][105692] Updated weights for policy 0, policy_version 836329 (0.0009) [2023-12-26 21:27:01,737][105692] Updated weights for policy 0, policy_version 836339 (0.0006) [2023-12-26 21:27:01,781][105620] Updated weights for policy 1, policy_version 836217 (0.0009) [2023-12-26 21:27:01,800][105692] Updated weights for policy 0, policy_version 836349 (0.0008) [2023-12-26 21:27:01,845][105620] Updated weights for policy 1, policy_version 836227 (0.0010) [2023-12-26 21:27:01,862][105692] Updated weights for policy 0, policy_version 836359 (0.0009) [2023-12-26 21:27:01,905][105620] Updated weights for policy 1, policy_version 836237 (0.0009) [2023-12-26 21:27:01,974][105620] Updated weights for policy 1, policy_version 836247 (0.0011) [2023-12-26 21:27:02,595][105692] Updated weights for policy 0, policy_version 836369 (0.0008) [2023-12-26 21:27:02,655][105692] Updated weights for policy 0, policy_version 836379 (0.0008) [2023-12-26 21:27:02,707][105692] Updated weights for policy 0, policy_version 836389 (0.0008) [2023-12-26 21:27:02,719][105620] Updated weights for policy 1, policy_version 836257 (0.0010) [2023-12-26 21:27:02,773][105620] Updated weights for policy 1, policy_version 836267 (0.0010) [2023-12-26 21:27:02,824][105620] Updated weights for policy 1, policy_version 836277 (0.0010) [2023-12-26 21:27:03,460][105620] Updated weights for policy 1, policy_version 836287 (0.0010) [2023-12-26 21:27:03,507][105620] Updated weights for policy 1, policy_version 836297 (0.0010) [2023-12-26 21:27:03,521][105692] Updated weights for policy 0, policy_version 836399 (0.0006) [2023-12-26 21:27:03,562][105620] Updated weights for policy 1, policy_version 836307 (0.0010) [2023-12-26 21:27:03,575][105692] Updated weights for policy 0, policy_version 836409 (0.0007) [2023-12-26 21:27:03,623][105692] Updated weights for policy 0, policy_version 836419 (0.0007) [2023-12-26 21:27:04,339][105620] Updated weights for policy 1, policy_version 836317 (0.0010) [2023-12-26 21:27:04,401][105620] Updated weights for policy 1, policy_version 836327 (0.0009) [2023-12-26 21:27:04,416][105692] Updated weights for policy 0, policy_version 836429 (0.0007) [2023-12-26 21:27:04,455][105620] Updated weights for policy 1, policy_version 836337 (0.0008) [2023-12-26 21:27:04,470][105692] Updated weights for policy 0, policy_version 836439 (0.0006) [2023-12-26 21:27:04,527][105692] Updated weights for policy 0, policy_version 836449 (0.0007) [2023-12-26 21:27:05,122][105620] Updated weights for policy 1, policy_version 836347 (0.0009) [2023-12-26 21:27:05,176][105620] Updated weights for policy 1, policy_version 836357 (0.0009) [2023-12-26 21:27:05,222][105620] Updated weights for policy 1, policy_version 836367 (0.0009) [2023-12-26 21:27:05,330][105692] Updated weights for policy 0, policy_version 836459 (0.0009) [2023-12-26 21:27:05,391][105692] Updated weights for policy 0, policy_version 836469 (0.0009) [2023-12-26 21:27:05,455][105692] Updated weights for policy 0, policy_version 836479 (0.0009) [2023-12-26 21:27:05,984][105620] Updated weights for policy 1, policy_version 836377 (0.0009) [2023-12-26 21:27:06,032][105620] Updated weights for policy 1, policy_version 836387 (0.0006) [2023-12-26 21:27:06,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 428310528. Throughput: 0: 9732.3, 1: 9828.4. Samples: 428305844. Policy #0 lag: (min: 23.0, avg: 36.7, max: 55.0) [2023-12-26 21:27:06,063][104569] Avg episode reward: [(0, '7812.263'), (1, '9006.687')] [2023-12-26 21:27:06,078][105620] Updated weights for policy 1, policy_version 836397 (0.0005) [2023-12-26 21:27:06,141][105620] Updated weights for policy 1, policy_version 836407 (0.0008) [2023-12-26 21:27:06,257][105692] Updated weights for policy 0, policy_version 836489 (0.0009) [2023-12-26 21:27:06,320][105692] Updated weights for policy 0, policy_version 836499 (0.0007) [2023-12-26 21:27:06,384][105692] Updated weights for policy 0, policy_version 836509 (0.0006) [2023-12-26 21:27:06,443][105692] Updated weights for policy 0, policy_version 836519 (0.0006) [2023-12-26 21:27:06,836][105620] Updated weights for policy 1, policy_version 836417 (0.0008) [2023-12-26 21:27:06,900][105620] Updated weights for policy 1, policy_version 836427 (0.0008) [2023-12-26 21:27:06,959][105620] Updated weights for policy 1, policy_version 836437 (0.0006) [2023-12-26 21:27:07,162][105692] Updated weights for policy 0, policy_version 836529 (0.0010) [2023-12-26 21:27:07,228][105692] Updated weights for policy 0, policy_version 836539 (0.0010) [2023-12-26 21:27:07,295][105692] Updated weights for policy 0, policy_version 836549 (0.0010) [2023-12-26 21:27:07,636][105620] Updated weights for policy 1, policy_version 836447 (0.0007) [2023-12-26 21:27:07,695][105620] Updated weights for policy 1, policy_version 836457 (0.0008) [2023-12-26 21:27:07,755][105620] Updated weights for policy 1, policy_version 836467 (0.0007) [2023-12-26 21:27:08,021][105692] Updated weights for policy 0, policy_version 836559 (0.0010) [2023-12-26 21:27:08,082][105692] Updated weights for policy 0, policy_version 836569 (0.0010) [2023-12-26 21:27:08,143][105692] Updated weights for policy 0, policy_version 836579 (0.0010) [2023-12-26 21:27:08,473][105620] Updated weights for policy 1, policy_version 836477 (0.0005) [2023-12-26 21:27:08,540][105620] Updated weights for policy 1, policy_version 836487 (0.0008) [2023-12-26 21:27:08,600][105620] Updated weights for policy 1, policy_version 836497 (0.0006) [2023-12-26 21:27:08,864][105692] Updated weights for policy 0, policy_version 836589 (0.0010) [2023-12-26 21:27:08,919][105692] Updated weights for policy 0, policy_version 836599 (0.0010) [2023-12-26 21:27:08,977][105692] Updated weights for policy 0, policy_version 836609 (0.0010) [2023-12-26 21:27:09,247][105620] Updated weights for policy 1, policy_version 836507 (0.0008) [2023-12-26 21:27:09,311][105620] Updated weights for policy 1, policy_version 836517 (0.0011) [2023-12-26 21:27:09,374][105620] Updated weights for policy 1, policy_version 836527 (0.0010) [2023-12-26 21:27:09,691][105692] Updated weights for policy 0, policy_version 836619 (0.0010) [2023-12-26 21:27:09,753][105692] Updated weights for policy 0, policy_version 836629 (0.0009) [2023-12-26 21:27:09,817][105692] Updated weights for policy 0, policy_version 836639 (0.0009) [2023-12-26 21:27:10,074][105620] Updated weights for policy 1, policy_version 836537 (0.0009) [2023-12-26 21:27:10,132][105620] Updated weights for policy 1, policy_version 836547 (0.0005) [2023-12-26 21:27:10,197][105620] Updated weights for policy 1, policy_version 836557 (0.0006) [2023-12-26 21:27:10,263][105620] Updated weights for policy 1, policy_version 836567 (0.0006) [2023-12-26 21:27:10,677][105692] Updated weights for policy 0, policy_version 836649 (0.0009) [2023-12-26 21:27:10,733][105692] Updated weights for policy 0, policy_version 836659 (0.0009) [2023-12-26 21:27:10,794][105692] Updated weights for policy 0, policy_version 836669 (0.0010) [2023-12-26 21:27:10,848][105692] Updated weights for policy 0, policy_version 836679 (0.0009) [2023-12-26 21:27:10,921][105620] Updated weights for policy 1, policy_version 836577 (0.0009) [2023-12-26 21:27:10,976][105620] Updated weights for policy 1, policy_version 836587 (0.0008) [2023-12-26 21:27:11,054][105620] Updated weights for policy 1, policy_version 836597 (0.0006) [2023-12-26 21:27:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 428408832. Throughput: 0: 9646.9, 1: 9824.4. Samples: 428420244. Policy #0 lag: (min: 23.0, avg: 36.7, max: 55.0) [2023-12-26 21:27:11,062][104569] Avg episode reward: [(0, '7636.625'), (1, '9005.183')] [2023-12-26 21:27:11,647][105692] Updated weights for policy 0, policy_version 836689 (0.0009) [2023-12-26 21:27:11,699][105692] Updated weights for policy 0, policy_version 836699 (0.0009) [2023-12-26 21:27:11,766][105692] Updated weights for policy 0, policy_version 836709 (0.0009) [2023-12-26 21:27:11,781][105620] Updated weights for policy 1, policy_version 836607 (0.0006) [2023-12-26 21:27:11,843][105620] Updated weights for policy 1, policy_version 836617 (0.0006) [2023-12-26 21:27:11,912][105620] Updated weights for policy 1, policy_version 836627 (0.0006) [2023-12-26 21:27:12,583][105692] Updated weights for policy 0, policy_version 836719 (0.0008) [2023-12-26 21:27:12,620][105620] Updated weights for policy 1, policy_version 836637 (0.0007) [2023-12-26 21:27:12,644][105692] Updated weights for policy 0, policy_version 836729 (0.0009) [2023-12-26 21:27:12,683][105620] Updated weights for policy 1, policy_version 836647 (0.0007) [2023-12-26 21:27:12,706][105692] Updated weights for policy 0, policy_version 836739 (0.0007) [2023-12-26 21:27:12,749][105620] Updated weights for policy 1, policy_version 836657 (0.0008) [2023-12-26 21:27:13,416][105620] Updated weights for policy 1, policy_version 836667 (0.0008) [2023-12-26 21:27:13,449][105692] Updated weights for policy 0, policy_version 836749 (0.0005) [2023-12-26 21:27:13,473][105620] Updated weights for policy 1, policy_version 836677 (0.0008) [2023-12-26 21:27:13,495][105692] Updated weights for policy 0, policy_version 836759 (0.0005) [2023-12-26 21:27:13,534][105620] Updated weights for policy 1, policy_version 836687 (0.0009) [2023-12-26 21:27:13,541][105692] Updated weights for policy 0, policy_version 836769 (0.0005) [2023-12-26 21:27:14,180][105620] Updated weights for policy 1, policy_version 836697 (0.0006) [2023-12-26 21:27:14,236][105692] Updated weights for policy 0, policy_version 836779 (0.0006) [2023-12-26 21:27:14,237][105620] Updated weights for policy 1, policy_version 836707 (0.0006) [2023-12-26 21:27:14,290][105620] Updated weights for policy 1, policy_version 836717 (0.0005) [2023-12-26 21:27:14,290][105692] Updated weights for policy 0, policy_version 836789 (0.0005) [2023-12-26 21:27:14,344][105620] Updated weights for policy 1, policy_version 836727 (0.0005) [2023-12-26 21:27:14,347][105692] Updated weights for policy 0, policy_version 836799 (0.0008) [2023-12-26 21:27:15,037][105692] Updated weights for policy 0, policy_version 836809 (0.0009) [2023-12-26 21:27:15,079][105620] Updated weights for policy 1, policy_version 836737 (0.0006) [2023-12-26 21:27:15,093][105692] Updated weights for policy 0, policy_version 836819 (0.0011) [2023-12-26 21:27:15,144][105620] Updated weights for policy 1, policy_version 836747 (0.0007) [2023-12-26 21:27:15,150][105692] Updated weights for policy 0, policy_version 836829 (0.0011) [2023-12-26 21:27:15,208][105620] Updated weights for policy 1, policy_version 836757 (0.0005) [2023-12-26 21:27:15,210][105692] Updated weights for policy 0, policy_version 836839 (0.0011) [2023-12-26 21:27:15,949][105692] Updated weights for policy 0, policy_version 836849 (0.0011) [2023-12-26 21:27:15,951][105620] Updated weights for policy 1, policy_version 836767 (0.0006) [2023-12-26 21:27:15,997][105692] Updated weights for policy 0, policy_version 836859 (0.0010) [2023-12-26 21:27:16,007][105620] Updated weights for policy 1, policy_version 836777 (0.0005) [2023-12-26 21:27:16,045][105692] Updated weights for policy 0, policy_version 836869 (0.0010) [2023-12-26 21:27:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19466.5). Total num frames: 428507136. Throughput: 0: 9602.4, 1: 9839.7. Samples: 428476192. Policy #0 lag: (min: 23.0, avg: 36.7, max: 55.0) [2023-12-26 21:27:16,063][104569] Avg episode reward: [(0, '7491.062'), (1, '8986.771')] [2023-12-26 21:27:16,066][105620] Updated weights for policy 1, policy_version 836787 (0.0006) [2023-12-26 21:27:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000836872_214269952.pth... [2023-12-26 21:27:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000835784_213991424.pth [2023-12-26 21:27:16,094][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000836792_214245376.pth... [2023-12-26 21:27:16,098][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000835640_213950464.pth [2023-12-26 21:27:16,741][105692] Updated weights for policy 0, policy_version 836879 (0.0010) [2023-12-26 21:27:16,810][105692] Updated weights for policy 0, policy_version 836889 (0.0008) [2023-12-26 21:27:16,815][105620] Updated weights for policy 1, policy_version 836797 (0.0009) [2023-12-26 21:27:16,863][105620] Updated weights for policy 1, policy_version 836807 (0.0010) [2023-12-26 21:27:16,869][105692] Updated weights for policy 0, policy_version 836899 (0.0006) [2023-12-26 21:27:16,908][105620] Updated weights for policy 1, policy_version 836817 (0.0010) [2023-12-26 21:27:17,585][105692] Updated weights for policy 0, policy_version 836909 (0.0006) [2023-12-26 21:27:17,647][105692] Updated weights for policy 0, policy_version 836919 (0.0008) [2023-12-26 21:27:17,663][105620] Updated weights for policy 1, policy_version 836827 (0.0008) [2023-12-26 21:27:17,713][105692] Updated weights for policy 0, policy_version 836929 (0.0008) [2023-12-26 21:27:17,725][105620] Updated weights for policy 1, policy_version 836837 (0.0008) [2023-12-26 21:27:17,781][105620] Updated weights for policy 1, policy_version 836847 (0.0009) [2023-12-26 21:27:18,483][105692] Updated weights for policy 0, policy_version 836939 (0.0009) [2023-12-26 21:27:18,528][105692] Updated weights for policy 0, policy_version 836949 (0.0010) [2023-12-26 21:27:18,570][105620] Updated weights for policy 1, policy_version 836857 (0.0008) [2023-12-26 21:27:18,588][105692] Updated weights for policy 0, policy_version 836959 (0.0008) [2023-12-26 21:27:18,632][105620] Updated weights for policy 1, policy_version 836867 (0.0005) [2023-12-26 21:27:18,691][105620] Updated weights for policy 1, policy_version 836877 (0.0005) [2023-12-26 21:27:18,751][105620] Updated weights for policy 1, policy_version 836887 (0.0005) [2023-12-26 21:27:19,300][105692] Updated weights for policy 0, policy_version 836969 (0.0011) [2023-12-26 21:27:19,365][105692] Updated weights for policy 0, policy_version 836979 (0.0009) [2023-12-26 21:27:19,426][105692] Updated weights for policy 0, policy_version 836989 (0.0008) [2023-12-26 21:27:19,445][105620] Updated weights for policy 1, policy_version 836897 (0.0006) [2023-12-26 21:27:19,477][105692] Updated weights for policy 0, policy_version 836999 (0.0007) [2023-12-26 21:27:19,513][105620] Updated weights for policy 1, policy_version 836907 (0.0008) [2023-12-26 21:27:19,578][105620] Updated weights for policy 1, policy_version 836917 (0.0008) [2023-12-26 21:27:20,227][105692] Updated weights for policy 0, policy_version 837009 (0.0010) [2023-12-26 21:27:20,296][105692] Updated weights for policy 0, policy_version 837019 (0.0010) [2023-12-26 21:27:20,331][105620] Updated weights for policy 1, policy_version 836927 (0.0006) [2023-12-26 21:27:20,360][105692] Updated weights for policy 0, policy_version 837029 (0.0008) [2023-12-26 21:27:20,386][105620] Updated weights for policy 1, policy_version 836937 (0.0005) [2023-12-26 21:27:20,439][105620] Updated weights for policy 1, policy_version 836947 (0.0009) [2023-12-26 21:27:21,021][105692] Updated weights for policy 0, policy_version 837039 (0.0007) [2023-12-26 21:27:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 428597248. Throughput: 0: 9467.7, 1: 9799.4. Samples: 428591036. Policy #0 lag: (min: 23.0, avg: 36.7, max: 55.0) [2023-12-26 21:27:21,063][104569] Avg episode reward: [(0, '7907.764'), (1, '9169.916')] [2023-12-26 21:27:21,085][105692] Updated weights for policy 0, policy_version 837049 (0.0009) [2023-12-26 21:27:21,148][105692] Updated weights for policy 0, policy_version 837059 (0.0009) [2023-12-26 21:27:21,216][105620] Updated weights for policy 1, policy_version 836957 (0.0009) [2023-12-26 21:27:21,284][105620] Updated weights for policy 1, policy_version 836967 (0.0009) [2023-12-26 21:27:21,352][105620] Updated weights for policy 1, policy_version 836977 (0.0008) [2023-12-26 21:27:21,989][105692] Updated weights for policy 0, policy_version 837069 (0.0008) [2023-12-26 21:27:22,015][105620] Updated weights for policy 1, policy_version 836987 (0.0008) [2023-12-26 21:27:22,047][105692] Updated weights for policy 0, policy_version 837079 (0.0006) [2023-12-26 21:27:22,071][105620] Updated weights for policy 1, policy_version 836997 (0.0006) [2023-12-26 21:27:22,100][105692] Updated weights for policy 0, policy_version 837089 (0.0007) [2023-12-26 21:27:22,125][105620] Updated weights for policy 1, policy_version 837007 (0.0006) [2023-12-26 21:27:22,780][105620] Updated weights for policy 1, policy_version 837017 (0.0009) [2023-12-26 21:27:22,836][105620] Updated weights for policy 1, policy_version 837027 (0.0009) [2023-12-26 21:27:22,897][105620] Updated weights for policy 1, policy_version 837037 (0.0007) [2023-12-26 21:27:22,900][105692] Updated weights for policy 0, policy_version 837099 (0.0007) [2023-12-26 21:27:22,959][105692] Updated weights for policy 0, policy_version 837109 (0.0007) [2023-12-26 21:27:22,965][105620] Updated weights for policy 1, policy_version 837047 (0.0007) [2023-12-26 21:27:23,020][105692] Updated weights for policy 0, policy_version 837119 (0.0008) [2023-12-26 21:27:23,672][105620] Updated weights for policy 1, policy_version 837057 (0.0009) [2023-12-26 21:27:23,726][105620] Updated weights for policy 1, policy_version 837067 (0.0010) [2023-12-26 21:27:23,772][105620] Updated weights for policy 1, policy_version 837077 (0.0008) [2023-12-26 21:27:23,787][105692] Updated weights for policy 0, policy_version 837129 (0.0008) [2023-12-26 21:27:23,849][105692] Updated weights for policy 0, policy_version 837139 (0.0009) [2023-12-26 21:27:23,910][105692] Updated weights for policy 0, policy_version 837149 (0.0009) [2023-12-26 21:27:23,970][105692] Updated weights for policy 0, policy_version 837159 (0.0008) [2023-12-26 21:27:24,417][105620] Updated weights for policy 1, policy_version 837087 (0.0006) [2023-12-26 21:27:24,481][105620] Updated weights for policy 1, policy_version 837097 (0.0005) [2023-12-26 21:27:24,541][105620] Updated weights for policy 1, policy_version 837107 (0.0009) [2023-12-26 21:27:24,702][105692] Updated weights for policy 0, policy_version 837169 (0.0006) [2023-12-26 21:27:24,756][105692] Updated weights for policy 0, policy_version 837179 (0.0005) [2023-12-26 21:27:24,803][105692] Updated weights for policy 0, policy_version 837189 (0.0005) [2023-12-26 21:27:25,079][105620] Updated weights for policy 1, policy_version 837117 (0.0008) [2023-12-26 21:27:25,150][105620] Updated weights for policy 1, policy_version 837127 (0.0010) [2023-12-26 21:27:25,218][105620] Updated weights for policy 1, policy_version 837137 (0.0010) [2023-12-26 21:27:25,388][105692] Updated weights for policy 0, policy_version 837199 (0.0005) [2023-12-26 21:27:25,454][105692] Updated weights for policy 0, policy_version 837209 (0.0005) [2023-12-26 21:27:25,517][105692] Updated weights for policy 0, policy_version 837219 (0.0005) [2023-12-26 21:27:25,764][105620] Updated weights for policy 1, policy_version 837147 (0.0007) [2023-12-26 21:27:25,835][105620] Updated weights for policy 1, policy_version 837157 (0.0005) [2023-12-26 21:27:25,901][105620] Updated weights for policy 1, policy_version 837167 (0.0005) [2023-12-26 21:27:26,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 428703744. Throughput: 0: 9342.9, 1: 9853.1. Samples: 428711420. Policy #0 lag: (min: 23.0, avg: 36.7, max: 55.0) [2023-12-26 21:27:26,062][104569] Avg episode reward: [(0, '8452.694'), (1, '9352.551')] [2023-12-26 21:27:26,065][105692] Updated weights for policy 0, policy_version 837229 (0.0005) [2023-12-26 21:27:26,131][105692] Updated weights for policy 0, policy_version 837239 (0.0005) [2023-12-26 21:27:26,190][105692] Updated weights for policy 0, policy_version 837249 (0.0005) [2023-12-26 21:27:26,415][105620] Updated weights for policy 1, policy_version 837177 (0.0005) [2023-12-26 21:27:26,479][105620] Updated weights for policy 1, policy_version 837187 (0.0005) [2023-12-26 21:27:26,550][105620] Updated weights for policy 1, policy_version 837197 (0.0005) [2023-12-26 21:27:26,610][105620] Updated weights for policy 1, policy_version 837207 (0.0005) [2023-12-26 21:27:26,816][105692] Updated weights for policy 0, policy_version 837259 (0.0006) [2023-12-26 21:27:26,873][105692] Updated weights for policy 0, policy_version 837269 (0.0008) [2023-12-26 21:27:26,931][105692] Updated weights for policy 0, policy_version 837279 (0.0008) [2023-12-26 21:27:27,185][105620] Updated weights for policy 1, policy_version 837217 (0.0005) [2023-12-26 21:27:27,242][105620] Updated weights for policy 1, policy_version 837227 (0.0005) [2023-12-26 21:27:27,294][105620] Updated weights for policy 1, policy_version 837237 (0.0005) [2023-12-26 21:27:27,610][105692] Updated weights for policy 0, policy_version 837289 (0.0008) [2023-12-26 21:27:27,658][105692] Updated weights for policy 0, policy_version 837299 (0.0005) [2023-12-26 21:27:27,713][105692] Updated weights for policy 0, policy_version 837309 (0.0005) [2023-12-26 21:27:27,760][105692] Updated weights for policy 0, policy_version 837319 (0.0005) [2023-12-26 21:27:27,952][105620] Updated weights for policy 1, policy_version 837247 (0.0010) [2023-12-26 21:27:28,004][105620] Updated weights for policy 1, policy_version 837257 (0.0010) [2023-12-26 21:27:28,052][105620] Updated weights for policy 1, policy_version 837267 (0.0010) [2023-12-26 21:27:28,397][105692] Updated weights for policy 0, policy_version 837329 (0.0009) [2023-12-26 21:27:28,456][105692] Updated weights for policy 0, policy_version 837339 (0.0010) [2023-12-26 21:27:28,509][105692] Updated weights for policy 0, policy_version 837349 (0.0010) [2023-12-26 21:27:28,724][105620] Updated weights for policy 1, policy_version 837277 (0.0010) [2023-12-26 21:27:28,786][105620] Updated weights for policy 1, policy_version 837287 (0.0010) [2023-12-26 21:27:28,850][105620] Updated weights for policy 1, policy_version 837297 (0.0010) [2023-12-26 21:27:29,331][105692] Updated weights for policy 0, policy_version 837359 (0.0010) [2023-12-26 21:27:29,394][105692] Updated weights for policy 0, policy_version 837369 (0.0009) [2023-12-26 21:27:29,452][105692] Updated weights for policy 0, policy_version 837379 (0.0009) [2023-12-26 21:27:29,495][105620] Updated weights for policy 1, policy_version 837307 (0.0009) [2023-12-26 21:27:29,553][105620] Updated weights for policy 1, policy_version 837317 (0.0005) [2023-12-26 21:27:29,610][105620] Updated weights for policy 1, policy_version 837327 (0.0007) [2023-12-26 21:27:30,234][105620] Updated weights for policy 1, policy_version 837337 (0.0009) [2023-12-26 21:27:30,287][105620] Updated weights for policy 1, policy_version 837347 (0.0008) [2023-12-26 21:27:30,293][105692] Updated weights for policy 0, policy_version 837389 (0.0008) [2023-12-26 21:27:30,339][105620] Updated weights for policy 1, policy_version 837357 (0.0007) [2023-12-26 21:27:30,349][105692] Updated weights for policy 0, policy_version 837399 (0.0008) [2023-12-26 21:27:30,405][105620] Updated weights for policy 1, policy_version 837367 (0.0008) [2023-12-26 21:27:30,410][105692] Updated weights for policy 0, policy_version 837409 (0.0007) [2023-12-26 21:27:31,031][105692] Updated weights for policy 0, policy_version 837419 (0.0007) [2023-12-26 21:27:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 428802048. Throughput: 0: 9434.1, 1: 9933.2. Samples: 428775920. Policy #0 lag: (min: 23.0, avg: 36.7, max: 55.0) [2023-12-26 21:27:31,063][104569] Avg episode reward: [(0, '8444.651'), (1, '8986.669')] [2023-12-26 21:27:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000837368_214392832.pth... [2023-12-26 21:27:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000836216_214097920.pth [2023-12-26 21:27:31,088][105692] Updated weights for policy 0, policy_version 837429 (0.0008) [2023-12-26 21:27:31,143][105692] Updated weights for policy 0, policy_version 837439 (0.0008) [2023-12-26 21:27:31,193][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000837448_214417408.pth... [2023-12-26 21:27:31,197][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000836328_214130688.pth [2023-12-26 21:27:31,216][105620] Updated weights for policy 1, policy_version 837377 (0.0008) [2023-12-26 21:27:31,279][105620] Updated weights for policy 1, policy_version 837387 (0.0010) [2023-12-26 21:27:31,345][105620] Updated weights for policy 1, policy_version 837397 (0.0009) [2023-12-26 21:27:31,910][105692] Updated weights for policy 0, policy_version 837449 (0.0006) [2023-12-26 21:27:31,960][105692] Updated weights for policy 0, policy_version 837459 (0.0008) [2023-12-26 21:27:32,014][105692] Updated weights for policy 0, policy_version 837469 (0.0008) [2023-12-26 21:27:32,033][105620] Updated weights for policy 1, policy_version 837407 (0.0008) [2023-12-26 21:27:32,075][105692] Updated weights for policy 0, policy_version 837479 (0.0007) [2023-12-26 21:27:32,086][105620] Updated weights for policy 1, policy_version 837417 (0.0008) [2023-12-26 21:27:32,142][105620] Updated weights for policy 1, policy_version 837427 (0.0009) [2023-12-26 21:27:32,698][105692] Updated weights for policy 0, policy_version 837489 (0.0009) [2023-12-26 21:27:32,760][105692] Updated weights for policy 0, policy_version 837499 (0.0010) [2023-12-26 21:27:32,810][105692] Updated weights for policy 0, policy_version 837509 (0.0008) [2023-12-26 21:27:32,973][105620] Updated weights for policy 1, policy_version 837437 (0.0010) [2023-12-26 21:27:33,041][105620] Updated weights for policy 1, policy_version 837447 (0.0009) [2023-12-26 21:27:33,116][105620] Updated weights for policy 1, policy_version 837457 (0.0009) [2023-12-26 21:27:33,473][105692] Updated weights for policy 0, policy_version 837519 (0.0008) [2023-12-26 21:27:33,522][105692] Updated weights for policy 0, policy_version 837529 (0.0009) [2023-12-26 21:27:33,573][105692] Updated weights for policy 0, policy_version 837539 (0.0009) [2023-12-26 21:27:33,799][105620] Updated weights for policy 1, policy_version 837467 (0.0007) [2023-12-26 21:27:33,845][105620] Updated weights for policy 1, policy_version 837477 (0.0008) [2023-12-26 21:27:33,890][105620] Updated weights for policy 1, policy_version 837487 (0.0008) [2023-12-26 21:27:34,301][105692] Updated weights for policy 0, policy_version 837549 (0.0007) [2023-12-26 21:27:34,358][105692] Updated weights for policy 0, policy_version 837559 (0.0006) [2023-12-26 21:27:34,414][105692] Updated weights for policy 0, policy_version 837569 (0.0008) [2023-12-26 21:27:34,674][105620] Updated weights for policy 1, policy_version 837497 (0.0009) [2023-12-26 21:27:34,740][105620] Updated weights for policy 1, policy_version 837507 (0.0008) [2023-12-26 21:27:34,802][105620] Updated weights for policy 1, policy_version 837517 (0.0009) [2023-12-26 21:27:34,866][105620] Updated weights for policy 1, policy_version 837527 (0.0009) [2023-12-26 21:27:35,061][105692] Updated weights for policy 0, policy_version 837579 (0.0010) [2023-12-26 21:27:35,119][105692] Updated weights for policy 0, policy_version 837589 (0.0009) [2023-12-26 21:27:35,182][105692] Updated weights for policy 0, policy_version 837599 (0.0009) [2023-12-26 21:27:35,624][105620] Updated weights for policy 1, policy_version 837537 (0.0008) [2023-12-26 21:27:35,687][105620] Updated weights for policy 1, policy_version 837547 (0.0007) [2023-12-26 21:27:35,739][105620] Updated weights for policy 1, policy_version 837557 (0.0005) [2023-12-26 21:27:35,958][105692] Updated weights for policy 0, policy_version 837609 (0.0007) [2023-12-26 21:27:36,010][105692] Updated weights for policy 0, policy_version 837619 (0.0009) [2023-12-26 21:27:36,057][105692] Updated weights for policy 0, policy_version 837629 (0.0008) [2023-12-26 21:27:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.1, 300 sec: 19466.4). Total num frames: 428900352. Throughput: 0: 9365.0, 1: 9944.1. Samples: 428891712. Policy #0 lag: (min: 23.0, avg: 36.7, max: 55.0) [2023-12-26 21:27:36,063][104569] Avg episode reward: [(0, '8443.935'), (1, '8987.909')] [2023-12-26 21:27:36,114][105692] Updated weights for policy 0, policy_version 837639 (0.0010) [2023-12-26 21:27:36,362][105620] Updated weights for policy 1, policy_version 837567 (0.0006) [2023-12-26 21:27:36,431][105620] Updated weights for policy 1, policy_version 837577 (0.0006) [2023-12-26 21:27:36,501][105620] Updated weights for policy 1, policy_version 837587 (0.0006) [2023-12-26 21:27:37,032][105692] Updated weights for policy 0, policy_version 837649 (0.0009) [2023-12-26 21:27:37,063][105620] Updated weights for policy 1, policy_version 837597 (0.0006) [2023-12-26 21:27:37,086][105692] Updated weights for policy 0, policy_version 837659 (0.0007) [2023-12-26 21:27:37,131][105620] Updated weights for policy 1, policy_version 837607 (0.0007) [2023-12-26 21:27:37,146][105692] Updated weights for policy 0, policy_version 837669 (0.0005) [2023-12-26 21:27:37,190][105620] Updated weights for policy 1, policy_version 837617 (0.0009) [2023-12-26 21:27:37,736][105692] Updated weights for policy 0, policy_version 837679 (0.0006) [2023-12-26 21:27:37,791][105692] Updated weights for policy 0, policy_version 837689 (0.0009) [2023-12-26 21:27:37,843][105692] Updated weights for policy 0, policy_version 837699 (0.0010) [2023-12-26 21:27:37,915][105620] Updated weights for policy 1, policy_version 837627 (0.0009) [2023-12-26 21:27:37,974][105620] Updated weights for policy 1, policy_version 837637 (0.0008) [2023-12-26 21:27:38,034][105620] Updated weights for policy 1, policy_version 837647 (0.0008) [2023-12-26 21:27:38,580][105692] Updated weights for policy 0, policy_version 837709 (0.0010) [2023-12-26 21:27:38,646][105692] Updated weights for policy 0, policy_version 837719 (0.0010) [2023-12-26 21:27:38,711][105692] Updated weights for policy 0, policy_version 837729 (0.0010) [2023-12-26 21:27:38,784][105620] Updated weights for policy 1, policy_version 837657 (0.0008) [2023-12-26 21:27:38,844][105620] Updated weights for policy 1, policy_version 837667 (0.0008) [2023-12-26 21:27:38,912][105620] Updated weights for policy 1, policy_version 837677 (0.0008) [2023-12-26 21:27:38,979][105620] Updated weights for policy 1, policy_version 837687 (0.0008) [2023-12-26 21:27:39,446][105692] Updated weights for policy 0, policy_version 837739 (0.0010) [2023-12-26 21:27:39,495][105692] Updated weights for policy 0, policy_version 837749 (0.0008) [2023-12-26 21:27:39,543][105692] Updated weights for policy 0, policy_version 837759 (0.0007) [2023-12-26 21:27:39,713][105620] Updated weights for policy 1, policy_version 837697 (0.0009) [2023-12-26 21:27:39,766][105620] Updated weights for policy 1, policy_version 837707 (0.0009) [2023-12-26 21:27:39,832][105620] Updated weights for policy 1, policy_version 837717 (0.0009) [2023-12-26 21:27:40,271][105692] Updated weights for policy 0, policy_version 837769 (0.0007) [2023-12-26 21:27:40,326][105692] Updated weights for policy 0, policy_version 837779 (0.0008) [2023-12-26 21:27:40,378][105692] Updated weights for policy 0, policy_version 837789 (0.0008) [2023-12-26 21:27:40,441][105692] Updated weights for policy 0, policy_version 837799 (0.0008) [2023-12-26 21:27:40,576][105620] Updated weights for policy 1, policy_version 837727 (0.0006) [2023-12-26 21:27:40,629][105620] Updated weights for policy 1, policy_version 837737 (0.0009) [2023-12-26 21:27:40,684][105620] Updated weights for policy 1, policy_version 837747 (0.0010) [2023-12-26 21:27:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 428998656. Throughput: 0: 9406.5, 1: 9951.7. Samples: 429008016. Policy #0 lag: (min: 23.0, avg: 36.7, max: 55.0) [2023-12-26 21:27:41,062][104569] Avg episode reward: [(0, '8443.145'), (1, '9122.072')] [2023-12-26 21:27:41,167][105692] Updated weights for policy 0, policy_version 837809 (0.0009) [2023-12-26 21:27:41,232][105692] Updated weights for policy 0, policy_version 837819 (0.0008) [2023-12-26 21:27:41,300][105692] Updated weights for policy 0, policy_version 837829 (0.0009) [2023-12-26 21:27:41,462][105620] Updated weights for policy 1, policy_version 837757 (0.0009) [2023-12-26 21:27:41,525][105620] Updated weights for policy 1, policy_version 837767 (0.0009) [2023-12-26 21:27:41,582][105620] Updated weights for policy 1, policy_version 837777 (0.0009) [2023-12-26 21:27:42,095][105692] Updated weights for policy 0, policy_version 837839 (0.0010) [2023-12-26 21:27:42,163][105692] Updated weights for policy 0, policy_version 837849 (0.0011) [2023-12-26 21:27:42,232][105692] Updated weights for policy 0, policy_version 837859 (0.0011) [2023-12-26 21:27:42,439][105620] Updated weights for policy 1, policy_version 837787 (0.0008) [2023-12-26 21:27:42,498][105620] Updated weights for policy 1, policy_version 837797 (0.0009) [2023-12-26 21:27:42,564][105620] Updated weights for policy 1, policy_version 837807 (0.0009) [2023-12-26 21:27:42,911][105692] Updated weights for policy 0, policy_version 837869 (0.0008) [2023-12-26 21:27:42,974][105692] Updated weights for policy 0, policy_version 837879 (0.0007) [2023-12-26 21:27:43,033][105692] Updated weights for policy 0, policy_version 837889 (0.0008) [2023-12-26 21:27:43,293][105620] Updated weights for policy 1, policy_version 837817 (0.0009) [2023-12-26 21:27:43,344][105620] Updated weights for policy 1, policy_version 837827 (0.0005) [2023-12-26 21:27:43,396][105620] Updated weights for policy 1, policy_version 837837 (0.0005) [2023-12-26 21:27:43,450][105620] Updated weights for policy 1, policy_version 837847 (0.0007) [2023-12-26 21:27:43,614][105692] Updated weights for policy 0, policy_version 837899 (0.0007) [2023-12-26 21:27:43,668][105692] Updated weights for policy 0, policy_version 837909 (0.0007) [2023-12-26 21:27:43,715][105692] Updated weights for policy 0, policy_version 837919 (0.0008) [2023-12-26 21:27:44,184][105620] Updated weights for policy 1, policy_version 837857 (0.0009) [2023-12-26 21:27:44,244][105620] Updated weights for policy 1, policy_version 837867 (0.0008) [2023-12-26 21:27:44,293][105620] Updated weights for policy 1, policy_version 837877 (0.0009) [2023-12-26 21:27:44,380][105692] Updated weights for policy 0, policy_version 837929 (0.0009) [2023-12-26 21:27:44,438][105692] Updated weights for policy 0, policy_version 837939 (0.0005) [2023-12-26 21:27:44,505][105692] Updated weights for policy 0, policy_version 837949 (0.0006) [2023-12-26 21:27:44,567][105692] Updated weights for policy 0, policy_version 837959 (0.0006) [2023-12-26 21:27:45,076][105620] Updated weights for policy 1, policy_version 837887 (0.0008) [2023-12-26 21:27:45,141][105620] Updated weights for policy 1, policy_version 837897 (0.0009) [2023-12-26 21:27:45,205][105620] Updated weights for policy 1, policy_version 837907 (0.0007) [2023-12-26 21:27:45,216][105586] KL-divergence is very high: 109.0434 [2023-12-26 21:27:45,217][105692] Updated weights for policy 0, policy_version 837969 (0.0008) [2023-12-26 21:27:45,279][105692] Updated weights for policy 0, policy_version 837979 (0.0009) [2023-12-26 21:27:45,335][105692] Updated weights for policy 0, policy_version 837989 (0.0009) [2023-12-26 21:27:45,809][105586] KL-divergence is very high: 102.2169 [2023-12-26 21:27:45,833][105620] Updated weights for policy 1, policy_version 837917 (0.0005) [2023-12-26 21:27:45,894][105620] Updated weights for policy 1, policy_version 837927 (0.0005) [2023-12-26 21:27:45,954][105620] Updated weights for policy 1, policy_version 837937 (0.0005) [2023-12-26 21:27:46,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19251.1, 300 sec: 19466.4). Total num frames: 429096960. Throughput: 0: 9488.6, 1: 9902.9. Samples: 429065588. Policy #0 lag: (min: 23.0, avg: 36.7, max: 55.0) [2023-12-26 21:27:46,063][104569] Avg episode reward: [(0, '8623.610'), (1, '8828.749')] [2023-12-26 21:27:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000837992_214556672.pth... [2023-12-26 21:27:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000837944_214540288.pth... [2023-12-26 21:27:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000836872_214269952.pth [2023-12-26 21:27:46,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000836792_214245376.pth [2023-12-26 21:27:46,155][105692] Updated weights for policy 0, policy_version 837999 (0.0007) [2023-12-26 21:27:46,203][105692] Updated weights for policy 0, policy_version 838009 (0.0005) [2023-12-26 21:27:46,251][105692] Updated weights for policy 0, policy_version 838019 (0.0005) [2023-12-26 21:27:46,608][105620] Updated weights for policy 1, policy_version 837947 (0.0006) [2023-12-26 21:27:46,666][105620] Updated weights for policy 1, policy_version 837957 (0.0005) [2023-12-26 21:27:46,714][105620] Updated weights for policy 1, policy_version 837967 (0.0008) [2023-12-26 21:27:46,855][105692] Updated weights for policy 0, policy_version 838029 (0.0006) [2023-12-26 21:27:46,906][105692] Updated weights for policy 0, policy_version 838039 (0.0008) [2023-12-26 21:27:46,974][105692] Updated weights for policy 0, policy_version 838049 (0.0009) [2023-12-26 21:27:47,474][105620] Updated weights for policy 1, policy_version 837977 (0.0010) [2023-12-26 21:27:47,520][105620] Updated weights for policy 1, policy_version 837987 (0.0008) [2023-12-26 21:27:47,566][105620] Updated weights for policy 1, policy_version 837997 (0.0009) [2023-12-26 21:27:47,624][105620] Updated weights for policy 1, policy_version 838007 (0.0010) [2023-12-26 21:27:47,679][105692] Updated weights for policy 0, policy_version 838059 (0.0009) [2023-12-26 21:27:47,742][105692] Updated weights for policy 0, policy_version 838069 (0.0010) [2023-12-26 21:27:47,786][105692] Updated weights for policy 0, policy_version 838079 (0.0010) [2023-12-26 21:27:48,307][105620] Updated weights for policy 1, policy_version 838017 (0.0007) [2023-12-26 21:27:48,376][105620] Updated weights for policy 1, policy_version 838027 (0.0009) [2023-12-26 21:27:48,383][105692] Updated weights for policy 0, policy_version 838089 (0.0006) [2023-12-26 21:27:48,429][105620] Updated weights for policy 1, policy_version 838037 (0.0009) [2023-12-26 21:27:48,442][105692] Updated weights for policy 0, policy_version 838099 (0.0009) [2023-12-26 21:27:48,494][105692] Updated weights for policy 0, policy_version 838109 (0.0010) [2023-12-26 21:27:48,546][105692] Updated weights for policy 0, policy_version 838119 (0.0010) [2023-12-26 21:27:49,172][105620] Updated weights for policy 1, policy_version 838047 (0.0008) [2023-12-26 21:27:49,242][105620] Updated weights for policy 1, policy_version 838057 (0.0008) [2023-12-26 21:27:49,291][105692] Updated weights for policy 0, policy_version 838129 (0.0008) [2023-12-26 21:27:49,305][105620] Updated weights for policy 1, policy_version 838067 (0.0008) [2023-12-26 21:27:49,355][105692] Updated weights for policy 0, policy_version 838139 (0.0011) [2023-12-26 21:27:49,416][105692] Updated weights for policy 0, policy_version 838149 (0.0009) [2023-12-26 21:27:50,047][105620] Updated weights for policy 1, policy_version 838077 (0.0007) [2023-12-26 21:27:50,115][105620] Updated weights for policy 1, policy_version 838087 (0.0005) [2023-12-26 21:27:50,175][105620] Updated weights for policy 1, policy_version 838097 (0.0006) [2023-12-26 21:27:50,179][105692] Updated weights for policy 0, policy_version 838159 (0.0010) [2023-12-26 21:27:50,237][105692] Updated weights for policy 0, policy_version 838169 (0.0007) [2023-12-26 21:27:50,297][105692] Updated weights for policy 0, policy_version 838179 (0.0011) [2023-12-26 21:27:50,773][105620] Updated weights for policy 1, policy_version 838107 (0.0007) [2023-12-26 21:27:50,838][105620] Updated weights for policy 1, policy_version 838117 (0.0009) [2023-12-26 21:27:50,901][105620] Updated weights for policy 1, policy_version 838127 (0.0008) [2023-12-26 21:27:50,973][105692] Updated weights for policy 0, policy_version 838189 (0.0011) [2023-12-26 21:27:51,032][105692] Updated weights for policy 0, policy_version 838199 (0.0011) [2023-12-26 21:27:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 429195264. Throughput: 0: 9666.3, 1: 9850.1. Samples: 429184084. Policy #0 lag: (min: 23.0, avg: 36.7, max: 55.0) [2023-12-26 21:27:51,063][104569] Avg episode reward: [(0, '8797.412'), (1, '8875.851')] [2023-12-26 21:27:51,095][105692] Updated weights for policy 0, policy_version 838209 (0.0011) [2023-12-26 21:27:51,587][105620] Updated weights for policy 1, policy_version 838137 (0.0007) [2023-12-26 21:27:51,655][105620] Updated weights for policy 1, policy_version 838147 (0.0009) [2023-12-26 21:27:51,704][105620] Updated weights for policy 1, policy_version 838157 (0.0010) [2023-12-26 21:27:51,767][105620] Updated weights for policy 1, policy_version 838167 (0.0009) [2023-12-26 21:27:51,832][105692] Updated weights for policy 0, policy_version 838219 (0.0010) [2023-12-26 21:27:51,894][105692] Updated weights for policy 0, policy_version 838229 (0.0011) [2023-12-26 21:27:51,953][105692] Updated weights for policy 0, policy_version 838239 (0.0011) [2023-12-26 21:27:52,509][105620] Updated weights for policy 1, policy_version 838177 (0.0010) [2023-12-26 21:27:52,568][105620] Updated weights for policy 1, policy_version 838187 (0.0010) [2023-12-26 21:27:52,637][105620] Updated weights for policy 1, policy_version 838197 (0.0010) [2023-12-26 21:27:52,688][105692] Updated weights for policy 0, policy_version 838249 (0.0010) [2023-12-26 21:27:52,739][105692] Updated weights for policy 0, policy_version 838259 (0.0009) [2023-12-26 21:27:52,785][105692] Updated weights for policy 0, policy_version 838269 (0.0009) [2023-12-26 21:27:52,841][105692] Updated weights for policy 0, policy_version 838279 (0.0005) [2023-12-26 21:27:53,363][105620] Updated weights for policy 1, policy_version 838207 (0.0010) [2023-12-26 21:27:53,426][105620] Updated weights for policy 1, policy_version 838217 (0.0011) [2023-12-26 21:27:53,496][105620] Updated weights for policy 1, policy_version 838227 (0.0009) [2023-12-26 21:27:53,540][105692] Updated weights for policy 0, policy_version 838289 (0.0006) [2023-12-26 21:27:53,596][105692] Updated weights for policy 0, policy_version 838299 (0.0005) [2023-12-26 21:27:53,649][105692] Updated weights for policy 0, policy_version 838309 (0.0005) [2023-12-26 21:27:54,170][105620] Updated weights for policy 1, policy_version 838237 (0.0010) [2023-12-26 21:27:54,220][105620] Updated weights for policy 1, policy_version 838247 (0.0009) [2023-12-26 21:27:54,283][105620] Updated weights for policy 1, policy_version 838257 (0.0008) [2023-12-26 21:27:54,312][105692] Updated weights for policy 0, policy_version 838319 (0.0006) [2023-12-26 21:27:54,366][105692] Updated weights for policy 0, policy_version 838329 (0.0010) [2023-12-26 21:27:54,418][105692] Updated weights for policy 0, policy_version 838339 (0.0010) [2023-12-26 21:27:55,031][105620] Updated weights for policy 1, policy_version 838267 (0.0007) [2023-12-26 21:27:55,097][105620] Updated weights for policy 1, policy_version 838277 (0.0008) [2023-12-26 21:27:55,160][105620] Updated weights for policy 1, policy_version 838287 (0.0008) [2023-12-26 21:27:55,185][105692] Updated weights for policy 0, policy_version 838349 (0.0010) [2023-12-26 21:27:55,233][105692] Updated weights for policy 0, policy_version 838359 (0.0010) [2023-12-26 21:27:55,277][105692] Updated weights for policy 0, policy_version 838369 (0.0010) [2023-12-26 21:27:55,871][105620] Updated weights for policy 1, policy_version 838297 (0.0006) [2023-12-26 21:27:55,929][105620] Updated weights for policy 1, policy_version 838307 (0.0010) [2023-12-26 21:27:55,980][105620] Updated weights for policy 1, policy_version 838317 (0.0010) [2023-12-26 21:27:56,031][105620] Updated weights for policy 1, policy_version 838327 (0.0010) [2023-12-26 21:27:56,039][105692] Updated weights for policy 0, policy_version 838379 (0.0010) [2023-12-26 21:27:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 429293568. Throughput: 0: 9747.8, 1: 9834.5. Samples: 429301452. Policy #0 lag: (min: 23.0, avg: 36.7, max: 55.0) [2023-12-26 21:27:56,063][104569] Avg episode reward: [(0, '8710.350'), (1, '9077.366')] [2023-12-26 21:27:56,087][105692] Updated weights for policy 0, policy_version 838389 (0.0010) [2023-12-26 21:27:56,135][105692] Updated weights for policy 0, policy_version 838399 (0.0009) [2023-12-26 21:27:56,664][105620] Updated weights for policy 1, policy_version 838337 (0.0006) [2023-12-26 21:27:56,728][105620] Updated weights for policy 1, policy_version 838347 (0.0010) [2023-12-26 21:27:56,783][105620] Updated weights for policy 1, policy_version 838357 (0.0010) [2023-12-26 21:27:56,866][105692] Updated weights for policy 0, policy_version 838409 (0.0005) [2023-12-26 21:27:56,917][105692] Updated weights for policy 0, policy_version 838419 (0.0008) [2023-12-26 21:27:56,961][105692] Updated weights for policy 0, policy_version 838429 (0.0010) [2023-12-26 21:27:57,009][105692] Updated weights for policy 0, policy_version 838439 (0.0010) [2023-12-26 21:27:57,420][105620] Updated weights for policy 1, policy_version 838367 (0.0010) [2023-12-26 21:27:57,485][105620] Updated weights for policy 1, policy_version 838377 (0.0010) [2023-12-26 21:27:57,546][105620] Updated weights for policy 1, policy_version 838387 (0.0010) [2023-12-26 21:27:57,603][105692] Updated weights for policy 0, policy_version 838449 (0.0006) [2023-12-26 21:27:57,665][105692] Updated weights for policy 0, policy_version 838459 (0.0008) [2023-12-26 21:27:57,732][105692] Updated weights for policy 0, policy_version 838469 (0.0008) [2023-12-26 21:27:58,267][105620] Updated weights for policy 1, policy_version 838397 (0.0010) [2023-12-26 21:27:58,330][105620] Updated weights for policy 1, policy_version 838407 (0.0010) [2023-12-26 21:27:58,400][105620] Updated weights for policy 1, policy_version 838417 (0.0010) [2023-12-26 21:27:58,400][105692] Updated weights for policy 0, policy_version 838479 (0.0007) [2023-12-26 21:27:58,471][105692] Updated weights for policy 0, policy_version 838489 (0.0010) [2023-12-26 21:27:58,533][105692] Updated weights for policy 0, policy_version 838499 (0.0008) [2023-12-26 21:27:59,235][105620] Updated weights for policy 1, policy_version 838427 (0.0009) [2023-12-26 21:27:59,299][105620] Updated weights for policy 1, policy_version 838437 (0.0010) [2023-12-26 21:27:59,315][105692] Updated weights for policy 0, policy_version 838509 (0.0008) [2023-12-26 21:27:59,373][105620] Updated weights for policy 1, policy_version 838447 (0.0009) [2023-12-26 21:27:59,386][105692] Updated weights for policy 0, policy_version 838519 (0.0008) [2023-12-26 21:27:59,437][105692] Updated weights for policy 0, policy_version 838529 (0.0005) [2023-12-26 21:28:00,040][105620] Updated weights for policy 1, policy_version 838457 (0.0008) [2023-12-26 21:28:00,103][105620] Updated weights for policy 1, policy_version 838467 (0.0011) [2023-12-26 21:28:00,160][105620] Updated weights for policy 1, policy_version 838477 (0.0010) [2023-12-26 21:28:00,229][105620] Updated weights for policy 1, policy_version 838487 (0.0007) [2023-12-26 21:28:00,230][105692] Updated weights for policy 0, policy_version 838539 (0.0008) [2023-12-26 21:28:00,286][105692] Updated weights for policy 0, policy_version 838549 (0.0009) [2023-12-26 21:28:00,345][105692] Updated weights for policy 0, policy_version 838559 (0.0008) [2023-12-26 21:28:00,898][105620] Updated weights for policy 1, policy_version 838497 (0.0010) [2023-12-26 21:28:00,949][105620] Updated weights for policy 1, policy_version 838507 (0.0010) [2023-12-26 21:28:01,000][105620] Updated weights for policy 1, policy_version 838517 (0.0010) [2023-12-26 21:28:01,019][105692] Updated weights for policy 0, policy_version 838569 (0.0008) [2023-12-26 21:28:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 429391872. Throughput: 0: 9827.2, 1: 9844.2. Samples: 429361396. Policy #0 lag: (min: 23.0, avg: 36.7, max: 55.0) [2023-12-26 21:28:01,062][104569] Avg episode reward: [(0, '8809.473'), (1, '8906.546')] [2023-12-26 21:28:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000838520_214687744.pth... [2023-12-26 21:28:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000837368_214392832.pth [2023-12-26 21:28:01,082][105692] Updated weights for policy 0, policy_version 838579 (0.0008) [2023-12-26 21:28:01,148][105692] Updated weights for policy 0, policy_version 838589 (0.0008) [2023-12-26 21:28:01,207][105692] Updated weights for policy 0, policy_version 838599 (0.0008) [2023-12-26 21:28:01,209][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000838600_214712320.pth... [2023-12-26 21:28:01,212][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000837448_214417408.pth [2023-12-26 21:28:01,770][105620] Updated weights for policy 1, policy_version 838527 (0.0011) [2023-12-26 21:28:01,818][105620] Updated weights for policy 1, policy_version 838537 (0.0010) [2023-12-26 21:28:01,880][105620] Updated weights for policy 1, policy_version 838547 (0.0010) [2023-12-26 21:28:01,933][105692] Updated weights for policy 0, policy_version 838609 (0.0007) [2023-12-26 21:28:01,992][105692] Updated weights for policy 0, policy_version 838619 (0.0008) [2023-12-26 21:28:02,059][105692] Updated weights for policy 0, policy_version 838629 (0.0008) [2023-12-26 21:28:02,556][105620] Updated weights for policy 1, policy_version 838557 (0.0008) [2023-12-26 21:28:02,608][105620] Updated weights for policy 1, policy_version 838567 (0.0005) [2023-12-26 21:28:02,661][105620] Updated weights for policy 1, policy_version 838577 (0.0008) [2023-12-26 21:28:02,819][105692] Updated weights for policy 0, policy_version 838639 (0.0006) [2023-12-26 21:28:02,880][105692] Updated weights for policy 0, policy_version 838649 (0.0007) [2023-12-26 21:28:02,939][105692] Updated weights for policy 0, policy_version 838659 (0.0007) [2023-12-26 21:28:03,402][105620] Updated weights for policy 1, policy_version 838587 (0.0010) [2023-12-26 21:28:03,459][105620] Updated weights for policy 1, policy_version 838597 (0.0010) [2023-12-26 21:28:03,510][105620] Updated weights for policy 1, policy_version 838607 (0.0010) [2023-12-26 21:28:03,548][105692] Updated weights for policy 0, policy_version 838669 (0.0006) [2023-12-26 21:28:03,610][105692] Updated weights for policy 0, policy_version 838679 (0.0008) [2023-12-26 21:28:03,656][105692] Updated weights for policy 0, policy_version 838689 (0.0008) [2023-12-26 21:28:04,215][105620] Updated weights for policy 1, policy_version 838617 (0.0010) [2023-12-26 21:28:04,275][105620] Updated weights for policy 1, policy_version 838627 (0.0009) [2023-12-26 21:28:04,334][105620] Updated weights for policy 1, policy_version 838637 (0.0009) [2023-12-26 21:28:04,382][105620] Updated weights for policy 1, policy_version 838647 (0.0009) [2023-12-26 21:28:04,424][105692] Updated weights for policy 0, policy_version 838699 (0.0009) [2023-12-26 21:28:04,488][105692] Updated weights for policy 0, policy_version 838709 (0.0009) [2023-12-26 21:28:04,551][105692] Updated weights for policy 0, policy_version 838719 (0.0009) [2023-12-26 21:28:05,155][105620] Updated weights for policy 1, policy_version 838657 (0.0006) [2023-12-26 21:28:05,211][105620] Updated weights for policy 1, policy_version 838667 (0.0005) [2023-12-26 21:28:05,267][105620] Updated weights for policy 1, policy_version 838677 (0.0005) [2023-12-26 21:28:05,337][105692] Updated weights for policy 0, policy_version 838729 (0.0009) [2023-12-26 21:28:05,395][105692] Updated weights for policy 0, policy_version 838739 (0.0009) [2023-12-26 21:28:05,449][105692] Updated weights for policy 0, policy_version 838749 (0.0009) [2023-12-26 21:28:05,511][105692] Updated weights for policy 0, policy_version 838759 (0.0009) [2023-12-26 21:28:05,845][105620] Updated weights for policy 1, policy_version 838687 (0.0008) [2023-12-26 21:28:05,904][105620] Updated weights for policy 1, policy_version 838697 (0.0009) [2023-12-26 21:28:05,967][105620] Updated weights for policy 1, policy_version 838707 (0.0009) [2023-12-26 21:28:06,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 429490176. Throughput: 0: 9811.4, 1: 9878.1. Samples: 429477060. Policy #0 lag: (min: 23.0, avg: 36.7, max: 55.0) [2023-12-26 21:28:06,062][104569] Avg episode reward: [(0, '8893.722'), (1, '8668.528')] [2023-12-26 21:28:06,363][105692] Updated weights for policy 0, policy_version 838769 (0.0010) [2023-12-26 21:28:06,432][105692] Updated weights for policy 0, policy_version 838779 (0.0009) [2023-12-26 21:28:06,508][105692] Updated weights for policy 0, policy_version 838789 (0.0010) [2023-12-26 21:28:06,574][105620] Updated weights for policy 1, policy_version 838717 (0.0007) [2023-12-26 21:28:06,636][105620] Updated weights for policy 1, policy_version 838727 (0.0007) [2023-12-26 21:28:06,699][105620] Updated weights for policy 1, policy_version 838737 (0.0006) [2023-12-26 21:28:07,326][105692] Updated weights for policy 0, policy_version 838799 (0.0009) [2023-12-26 21:28:07,361][105620] Updated weights for policy 1, policy_version 838747 (0.0009) [2023-12-26 21:28:07,383][105692] Updated weights for policy 0, policy_version 838809 (0.0006) [2023-12-26 21:28:07,420][105620] Updated weights for policy 1, policy_version 838757 (0.0011) [2023-12-26 21:28:07,442][105692] Updated weights for policy 0, policy_version 838819 (0.0007) [2023-12-26 21:28:07,476][105620] Updated weights for policy 1, policy_version 838767 (0.0010) [2023-12-26 21:28:08,196][105692] Updated weights for policy 0, policy_version 838829 (0.0008) [2023-12-26 21:28:08,229][105620] Updated weights for policy 1, policy_version 838777 (0.0011) [2023-12-26 21:28:08,246][105692] Updated weights for policy 0, policy_version 838839 (0.0007) [2023-12-26 21:28:08,291][105620] Updated weights for policy 1, policy_version 838787 (0.0010) [2023-12-26 21:28:08,301][105692] Updated weights for policy 0, policy_version 838849 (0.0005) [2023-12-26 21:28:08,353][105620] Updated weights for policy 1, policy_version 838797 (0.0008) [2023-12-26 21:28:08,410][105620] Updated weights for policy 1, policy_version 838807 (0.0006) [2023-12-26 21:28:08,986][105620] Updated weights for policy 1, policy_version 838817 (0.0006) [2023-12-26 21:28:09,044][105620] Updated weights for policy 1, policy_version 838827 (0.0005) [2023-12-26 21:28:09,103][105620] Updated weights for policy 1, policy_version 838837 (0.0006) [2023-12-26 21:28:09,168][105692] Updated weights for policy 0, policy_version 838859 (0.0008) [2023-12-26 21:28:09,224][105692] Updated weights for policy 0, policy_version 838869 (0.0008) [2023-12-26 21:28:09,289][105692] Updated weights for policy 0, policy_version 838879 (0.0009) [2023-12-26 21:28:09,811][105620] Updated weights for policy 1, policy_version 838847 (0.0011) [2023-12-26 21:28:09,876][105620] Updated weights for policy 1, policy_version 838857 (0.0008) [2023-12-26 21:28:09,934][105620] Updated weights for policy 1, policy_version 838867 (0.0007) [2023-12-26 21:28:10,095][105692] Updated weights for policy 0, policy_version 838889 (0.0008) [2023-12-26 21:28:10,163][105692] Updated weights for policy 0, policy_version 838899 (0.0008) [2023-12-26 21:28:10,230][105692] Updated weights for policy 0, policy_version 838909 (0.0008) [2023-12-26 21:28:10,290][105692] Updated weights for policy 0, policy_version 838919 (0.0008) [2023-12-26 21:28:10,646][105620] Updated weights for policy 1, policy_version 838877 (0.0009) [2023-12-26 21:28:10,707][105620] Updated weights for policy 1, policy_version 838887 (0.0011) [2023-12-26 21:28:10,769][105620] Updated weights for policy 1, policy_version 838897 (0.0011) [2023-12-26 21:28:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 429580288. Throughput: 0: 9686.8, 1: 9858.5. Samples: 429590956. Policy #0 lag: (min: 23.0, avg: 36.7, max: 55.0) [2023-12-26 21:28:11,062][104569] Avg episode reward: [(0, '8713.369'), (1, '8917.074')] [2023-12-26 21:28:11,081][105692] Updated weights for policy 0, policy_version 838929 (0.0008) [2023-12-26 21:28:11,144][105692] Updated weights for policy 0, policy_version 838939 (0.0007) [2023-12-26 21:28:11,215][105692] Updated weights for policy 0, policy_version 838949 (0.0008) [2023-12-26 21:28:11,502][105620] Updated weights for policy 1, policy_version 838907 (0.0009) [2023-12-26 21:28:11,558][105620] Updated weights for policy 1, policy_version 838917 (0.0006) [2023-12-26 21:28:11,608][105620] Updated weights for policy 1, policy_version 838927 (0.0007) [2023-12-26 21:28:11,941][105692] Updated weights for policy 0, policy_version 838959 (0.0006) [2023-12-26 21:28:12,003][105692] Updated weights for policy 0, policy_version 838969 (0.0007) [2023-12-26 21:28:12,063][105692] Updated weights for policy 0, policy_version 838979 (0.0006) [2023-12-26 21:28:12,422][105620] Updated weights for policy 1, policy_version 838937 (0.0008) [2023-12-26 21:28:12,478][105620] Updated weights for policy 1, policy_version 838947 (0.0008) [2023-12-26 21:28:12,528][105620] Updated weights for policy 1, policy_version 838957 (0.0009) [2023-12-26 21:28:12,581][105620] Updated weights for policy 1, policy_version 838967 (0.0009) [2023-12-26 21:28:12,657][105692] Updated weights for policy 0, policy_version 838989 (0.0006) [2023-12-26 21:28:12,715][105692] Updated weights for policy 0, policy_version 838999 (0.0009) [2023-12-26 21:28:12,779][105692] Updated weights for policy 0, policy_version 839009 (0.0009) [2023-12-26 21:28:13,424][105620] Updated weights for policy 1, policy_version 838977 (0.0009) [2023-12-26 21:28:13,455][105692] Updated weights for policy 0, policy_version 839019 (0.0008) [2023-12-26 21:28:13,482][105620] Updated weights for policy 1, policy_version 838987 (0.0009) [2023-12-26 21:28:13,515][105692] Updated weights for policy 0, policy_version 839029 (0.0005) [2023-12-26 21:28:13,544][105620] Updated weights for policy 1, policy_version 838997 (0.0008) [2023-12-26 21:28:13,571][105692] Updated weights for policy 0, policy_version 839039 (0.0007) [2023-12-26 21:28:14,249][105692] Updated weights for policy 0, policy_version 839049 (0.0008) [2023-12-26 21:28:14,284][105620] Updated weights for policy 1, policy_version 839007 (0.0008) [2023-12-26 21:28:14,309][105692] Updated weights for policy 0, policy_version 839059 (0.0006) [2023-12-26 21:28:14,339][105620] Updated weights for policy 1, policy_version 839017 (0.0008) [2023-12-26 21:28:14,373][105692] Updated weights for policy 0, policy_version 839069 (0.0005) [2023-12-26 21:28:14,390][105620] Updated weights for policy 1, policy_version 839027 (0.0008) [2023-12-26 21:28:14,439][105692] Updated weights for policy 0, policy_version 839079 (0.0005) [2023-12-26 21:28:15,034][105692] Updated weights for policy 0, policy_version 839089 (0.0007) [2023-12-26 21:28:15,094][105692] Updated weights for policy 0, policy_version 839099 (0.0008) [2023-12-26 21:28:15,149][105692] Updated weights for policy 0, policy_version 839109 (0.0008) [2023-12-26 21:28:15,174][105620] Updated weights for policy 1, policy_version 839037 (0.0010) [2023-12-26 21:28:15,227][105620] Updated weights for policy 1, policy_version 839047 (0.0010) [2023-12-26 21:28:15,283][105620] Updated weights for policy 1, policy_version 839057 (0.0010) [2023-12-26 21:28:15,891][105620] Updated weights for policy 1, policy_version 839067 (0.0008) [2023-12-26 21:28:15,942][105620] Updated weights for policy 1, policy_version 839077 (0.0005) [2023-12-26 21:28:15,987][105692] Updated weights for policy 0, policy_version 839119 (0.0006) [2023-12-26 21:28:15,990][105620] Updated weights for policy 1, policy_version 839087 (0.0007) [2023-12-26 21:28:16,045][105692] Updated weights for policy 0, policy_version 839129 (0.0005) [2023-12-26 21:28:16,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 429678592. Throughput: 0: 9659.1, 1: 9726.0. Samples: 429648252. Policy #0 lag: (min: 23.0, avg: 36.7, max: 55.0) [2023-12-26 21:28:16,063][104569] Avg episode reward: [(0, '8425.407'), (1, '8803.224')] [2023-12-26 21:28:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000839096_214835200.pth... [2023-12-26 21:28:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000837944_214540288.pth [2023-12-26 21:28:16,103][105692] Updated weights for policy 0, policy_version 839139 (0.0005) [2023-12-26 21:28:16,124][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000839144_214851584.pth... [2023-12-26 21:28:16,128][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000837992_214556672.pth [2023-12-26 21:28:16,616][105692] Updated weights for policy 0, policy_version 839149 (0.0007) [2023-12-26 21:28:16,657][105620] Updated weights for policy 1, policy_version 839097 (0.0008) [2023-12-26 21:28:16,660][105692] Updated weights for policy 0, policy_version 839159 (0.0008) [2023-12-26 21:28:16,712][105692] Updated weights for policy 0, policy_version 839169 (0.0005) [2023-12-26 21:28:16,726][105620] Updated weights for policy 1, policy_version 839107 (0.0008) [2023-12-26 21:28:16,781][105620] Updated weights for policy 1, policy_version 839117 (0.0009) [2023-12-26 21:28:16,838][105620] Updated weights for policy 1, policy_version 839127 (0.0010) [2023-12-26 21:28:17,284][105692] Updated weights for policy 0, policy_version 839179 (0.0005) [2023-12-26 21:28:17,348][105692] Updated weights for policy 0, policy_version 839189 (0.0005) [2023-12-26 21:28:17,413][105692] Updated weights for policy 0, policy_version 839199 (0.0005) [2023-12-26 21:28:17,531][105620] Updated weights for policy 1, policy_version 839137 (0.0010) [2023-12-26 21:28:17,601][105620] Updated weights for policy 1, policy_version 839147 (0.0009) [2023-12-26 21:28:17,668][105620] Updated weights for policy 1, policy_version 839157 (0.0005) [2023-12-26 21:28:17,905][105692] Updated weights for policy 0, policy_version 839209 (0.0005) [2023-12-26 21:28:17,959][105692] Updated weights for policy 0, policy_version 839219 (0.0005) [2023-12-26 21:28:18,003][105692] Updated weights for policy 0, policy_version 839229 (0.0005) [2023-12-26 21:28:18,050][105692] Updated weights for policy 0, policy_version 839239 (0.0005) [2023-12-26 21:28:18,333][105620] Updated weights for policy 1, policy_version 839167 (0.0008) [2023-12-26 21:28:18,400][105620] Updated weights for policy 1, policy_version 839177 (0.0009) [2023-12-26 21:28:18,464][105620] Updated weights for policy 1, policy_version 839187 (0.0009) [2023-12-26 21:28:18,779][105692] Updated weights for policy 0, policy_version 839249 (0.0008) [2023-12-26 21:28:18,840][105692] Updated weights for policy 0, policy_version 839259 (0.0008) [2023-12-26 21:28:18,900][105692] Updated weights for policy 0, policy_version 839269 (0.0005) [2023-12-26 21:28:19,166][105620] Updated weights for policy 1, policy_version 839197 (0.0007) [2023-12-26 21:28:19,226][105620] Updated weights for policy 1, policy_version 839207 (0.0006) [2023-12-26 21:28:19,293][105620] Updated weights for policy 1, policy_version 839217 (0.0007) [2023-12-26 21:28:19,489][105692] Updated weights for policy 0, policy_version 839279 (0.0009) [2023-12-26 21:28:19,539][105692] Updated weights for policy 0, policy_version 839289 (0.0006) [2023-12-26 21:28:19,594][105692] Updated weights for policy 0, policy_version 839299 (0.0008) [2023-12-26 21:28:20,027][105620] Updated weights for policy 1, policy_version 839227 (0.0009) [2023-12-26 21:28:20,088][105620] Updated weights for policy 1, policy_version 839237 (0.0011) [2023-12-26 21:28:20,146][105620] Updated weights for policy 1, policy_version 839247 (0.0011) [2023-12-26 21:28:20,316][105692] Updated weights for policy 0, policy_version 839309 (0.0011) [2023-12-26 21:28:20,375][105692] Updated weights for policy 0, policy_version 839319 (0.0010) [2023-12-26 21:28:20,438][105692] Updated weights for policy 0, policy_version 839329 (0.0011) [2023-12-26 21:28:20,847][105620] Updated weights for policy 1, policy_version 839257 (0.0011) [2023-12-26 21:28:20,891][105586] KL-divergence is very high: 132.2834 [2023-12-26 21:28:20,901][105620] Updated weights for policy 1, policy_version 839267 (0.0011) [2023-12-26 21:28:20,932][105586] KL-divergence is very high: 248.9171 [2023-12-26 21:28:20,957][105620] Updated weights for policy 1, policy_version 839277 (0.0011) [2023-12-26 21:28:20,984][105586] KL-divergence is very high: 201.0384 [2023-12-26 21:28:21,021][105620] Updated weights for policy 1, policy_version 839287 (0.0011) [2023-12-26 21:28:21,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 429785088. Throughput: 0: 9805.2, 1: 9787.3. Samples: 429773376. Policy #0 lag: (min: 23.0, avg: 36.7, max: 55.0) [2023-12-26 21:28:21,062][104569] Avg episode reward: [(0, '8635.720'), (1, '8574.921')] [2023-12-26 21:28:21,153][105692] Updated weights for policy 0, policy_version 839339 (0.0009) [2023-12-26 21:28:21,218][105692] Updated weights for policy 0, policy_version 839349 (0.0006) [2023-12-26 21:28:21,286][105692] Updated weights for policy 0, policy_version 839359 (0.0008) [2023-12-26 21:28:21,817][105620] Updated weights for policy 1, policy_version 839297 (0.0011) [2023-12-26 21:28:21,887][105620] Updated weights for policy 1, policy_version 839307 (0.0011) [2023-12-26 21:28:21,951][105620] Updated weights for policy 1, policy_version 839317 (0.0011) [2023-12-26 21:28:22,012][105692] Updated weights for policy 0, policy_version 839369 (0.0006) [2023-12-26 21:28:22,068][105692] Updated weights for policy 0, policy_version 839379 (0.0008) [2023-12-26 21:28:22,117][105692] Updated weights for policy 0, policy_version 839389 (0.0008) [2023-12-26 21:28:22,174][105692] Updated weights for policy 0, policy_version 839399 (0.0009) [2023-12-26 21:28:22,605][105620] Updated weights for policy 1, policy_version 839327 (0.0009) [2023-12-26 21:28:22,674][105620] Updated weights for policy 1, policy_version 839337 (0.0008) [2023-12-26 21:28:22,742][105620] Updated weights for policy 1, policy_version 839347 (0.0009) [2023-12-26 21:28:23,060][105692] Updated weights for policy 0, policy_version 839409 (0.0009) [2023-12-26 21:28:23,128][105692] Updated weights for policy 0, policy_version 839419 (0.0008) [2023-12-26 21:28:23,191][105692] Updated weights for policy 0, policy_version 839429 (0.0010) [2023-12-26 21:28:23,360][105620] Updated weights for policy 1, policy_version 839357 (0.0009) [2023-12-26 21:28:23,426][105620] Updated weights for policy 1, policy_version 839367 (0.0010) [2023-12-26 21:28:23,483][105620] Updated weights for policy 1, policy_version 839377 (0.0010) [2023-12-26 21:28:23,950][105692] Updated weights for policy 0, policy_version 839439 (0.0009) [2023-12-26 21:28:24,009][105692] Updated weights for policy 0, policy_version 839450 (0.0009) [2023-12-26 21:28:24,069][105692] Updated weights for policy 0, policy_version 839460 (0.0005) [2023-12-26 21:28:24,226][105620] Updated weights for policy 1, policy_version 839387 (0.0010) [2023-12-26 21:28:24,282][105620] Updated weights for policy 1, policy_version 839397 (0.0010) [2023-12-26 21:28:24,335][105620] Updated weights for policy 1, policy_version 839407 (0.0011) [2023-12-26 21:28:24,853][105692] Updated weights for policy 0, policy_version 839470 (0.0008) [2023-12-26 21:28:24,916][105692] Updated weights for policy 0, policy_version 839480 (0.0008) [2023-12-26 21:28:24,978][105692] Updated weights for policy 0, policy_version 839490 (0.0008) [2023-12-26 21:28:25,046][105620] Updated weights for policy 1, policy_version 839417 (0.0010) [2023-12-26 21:28:25,095][105620] Updated weights for policy 1, policy_version 839427 (0.0010) [2023-12-26 21:28:25,143][105620] Updated weights for policy 1, policy_version 839437 (0.0010) [2023-12-26 21:28:25,191][105620] Updated weights for policy 1, policy_version 839447 (0.0010) [2023-12-26 21:28:25,730][105692] Updated weights for policy 0, policy_version 839500 (0.0009) [2023-12-26 21:28:25,782][105692] Updated weights for policy 0, policy_version 839510 (0.0009) [2023-12-26 21:28:25,843][105692] Updated weights for policy 0, policy_version 839520 (0.0008) [2023-12-26 21:28:25,861][105620] Updated weights for policy 1, policy_version 839457 (0.0006) [2023-12-26 21:28:25,929][105620] Updated weights for policy 1, policy_version 839467 (0.0008) [2023-12-26 21:28:25,987][105620] Updated weights for policy 1, policy_version 839477 (0.0009) [2023-12-26 21:28:26,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 429883392. Throughput: 0: 9756.0, 1: 9793.0. Samples: 429887720. Policy #0 lag: (min: 23.0, avg: 36.7, max: 55.0) [2023-12-26 21:28:26,062][104569] Avg episode reward: [(0, '8352.968'), (1, '8988.750')] [2023-12-26 21:28:26,481][105692] Updated weights for policy 0, policy_version 839530 (0.0005) [2023-12-26 21:28:26,534][105692] Updated weights for policy 0, policy_version 839540 (0.0005) [2023-12-26 21:28:26,559][105585] KL-divergence is very high: 138.3698 [2023-12-26 21:28:26,572][105585] KL-divergence is very high: 171.7504 [2023-12-26 21:28:26,595][105692] Updated weights for policy 0, policy_version 839550 (0.0005) [2023-12-26 21:28:26,603][105620] Updated weights for policy 1, policy_version 839487 (0.0008) [2023-12-26 21:28:26,607][105585] KL-divergence is very high: 236.2262 [2023-12-26 21:28:26,620][105585] KL-divergence is very high: 259.3770 [2023-12-26 21:28:26,651][105692] Updated weights for policy 0, policy_version 839560 (0.0008) [2023-12-26 21:28:26,666][105620] Updated weights for policy 1, policy_version 839497 (0.0009) [2023-12-26 21:28:26,725][105620] Updated weights for policy 1, policy_version 839507 (0.0008) [2023-12-26 21:28:27,280][105692] Updated weights for policy 0, policy_version 839570 (0.0009) [2023-12-26 21:28:27,336][105692] Updated weights for policy 0, policy_version 839580 (0.0007) [2023-12-26 21:28:27,388][105692] Updated weights for policy 0, policy_version 839590 (0.0007) [2023-12-26 21:28:27,494][105620] Updated weights for policy 1, policy_version 839517 (0.0007) [2023-12-26 21:28:27,548][105620] Updated weights for policy 1, policy_version 839527 (0.0005) [2023-12-26 21:28:27,618][105620] Updated weights for policy 1, policy_version 839537 (0.0005) [2023-12-26 21:28:28,027][105692] Updated weights for policy 0, policy_version 839600 (0.0006) [2023-12-26 21:28:28,075][105692] Updated weights for policy 0, policy_version 839610 (0.0006) [2023-12-26 21:28:28,131][105692] Updated weights for policy 0, policy_version 839620 (0.0010) [2023-12-26 21:28:28,249][105620] Updated weights for policy 1, policy_version 839547 (0.0006) [2023-12-26 21:28:28,300][105620] Updated weights for policy 1, policy_version 839557 (0.0008) [2023-12-26 21:28:28,359][105620] Updated weights for policy 1, policy_version 839567 (0.0008) [2023-12-26 21:28:28,869][105692] Updated weights for policy 0, policy_version 839630 (0.0010) [2023-12-26 21:28:28,931][105692] Updated weights for policy 0, policy_version 839640 (0.0010) [2023-12-26 21:28:28,991][105620] Updated weights for policy 1, policy_version 839577 (0.0008) [2023-12-26 21:28:28,996][105692] Updated weights for policy 0, policy_version 839650 (0.0011) [2023-12-26 21:28:29,050][105620] Updated weights for policy 1, policy_version 839587 (0.0006) [2023-12-26 21:28:29,104][105620] Updated weights for policy 1, policy_version 839597 (0.0008) [2023-12-26 21:28:29,169][105620] Updated weights for policy 1, policy_version 839607 (0.0008) [2023-12-26 21:28:29,721][105692] Updated weights for policy 0, policy_version 839660 (0.0011) [2023-12-26 21:28:29,772][105692] Updated weights for policy 0, policy_version 839670 (0.0010) [2023-12-26 21:28:29,837][105692] Updated weights for policy 0, policy_version 839680 (0.0011) [2023-12-26 21:28:29,855][105620] Updated weights for policy 1, policy_version 839617 (0.0008) [2023-12-26 21:28:29,916][105620] Updated weights for policy 1, policy_version 839627 (0.0008) [2023-12-26 21:28:29,984][105620] Updated weights for policy 1, policy_version 839637 (0.0008) [2023-12-26 21:28:30,605][105692] Updated weights for policy 0, policy_version 839690 (0.0011) [2023-12-26 21:28:30,653][105692] Updated weights for policy 0, policy_version 839700 (0.0010) [2023-12-26 21:28:30,707][105692] Updated weights for policy 0, policy_version 839710 (0.0010) [2023-12-26 21:28:30,722][105620] Updated weights for policy 1, policy_version 839647 (0.0007) [2023-12-26 21:28:30,766][105692] Updated weights for policy 0, policy_version 839720 (0.0011) [2023-12-26 21:28:30,775][105620] Updated weights for policy 1, policy_version 839657 (0.0006) [2023-12-26 21:28:30,823][105620] Updated weights for policy 1, policy_version 839667 (0.0008) [2023-12-26 21:28:31,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 429981696. Throughput: 0: 9798.3, 1: 9858.4. Samples: 429950132. Policy #0 lag: (min: 23.0, avg: 36.7, max: 55.0) [2023-12-26 21:28:31,062][104569] Avg episode reward: [(0, '8175.580'), (1, '8909.703')] [2023-12-26 21:28:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000839720_214999040.pth... [2023-12-26 21:28:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000839672_214982656.pth... [2023-12-26 21:28:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000838600_214712320.pth [2023-12-26 21:28:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000838520_214687744.pth [2023-12-26 21:28:31,553][105692] Updated weights for policy 0, policy_version 839730 (0.0010) [2023-12-26 21:28:31,577][105620] Updated weights for policy 1, policy_version 839677 (0.0007) [2023-12-26 21:28:31,605][105692] Updated weights for policy 0, policy_version 839740 (0.0010) [2023-12-26 21:28:31,637][105620] Updated weights for policy 1, policy_version 839687 (0.0006) [2023-12-26 21:28:31,666][105692] Updated weights for policy 0, policy_version 839750 (0.0011) [2023-12-26 21:28:31,690][105620] Updated weights for policy 1, policy_version 839697 (0.0006) [2023-12-26 21:28:32,339][105620] Updated weights for policy 1, policy_version 839707 (0.0008) [2023-12-26 21:28:32,389][105620] Updated weights for policy 1, policy_version 839717 (0.0007) [2023-12-26 21:28:32,435][105620] Updated weights for policy 1, policy_version 839727 (0.0005) [2023-12-26 21:28:32,446][105692] Updated weights for policy 0, policy_version 839760 (0.0010) [2023-12-26 21:28:32,497][105692] Updated weights for policy 0, policy_version 839770 (0.0010) [2023-12-26 21:28:32,557][105692] Updated weights for policy 0, policy_version 839780 (0.0010) [2023-12-26 21:28:33,054][105620] Updated weights for policy 1, policy_version 839737 (0.0005) [2023-12-26 21:28:33,116][105620] Updated weights for policy 1, policy_version 839747 (0.0008) [2023-12-26 21:28:33,179][105620] Updated weights for policy 1, policy_version 839757 (0.0008) [2023-12-26 21:28:33,242][105620] Updated weights for policy 1, policy_version 839767 (0.0008) [2023-12-26 21:28:33,312][105692] Updated weights for policy 0, policy_version 839790 (0.0010) [2023-12-26 21:28:33,373][105692] Updated weights for policy 0, policy_version 839800 (0.0010) [2023-12-26 21:28:33,425][105585] KL-divergence is very high: 148.3836 [2023-12-26 21:28:33,431][105692] Updated weights for policy 0, policy_version 839810 (0.0010) [2023-12-26 21:28:34,016][105692] Updated weights for policy 0, policy_version 839820 (0.0010) [2023-12-26 21:28:34,039][105620] Updated weights for policy 1, policy_version 839777 (0.0007) [2023-12-26 21:28:34,064][105692] Updated weights for policy 0, policy_version 839830 (0.0010) [2023-12-26 21:28:34,094][105620] Updated weights for policy 1, policy_version 839787 (0.0006) [2023-12-26 21:28:34,116][105692] Updated weights for policy 0, policy_version 839840 (0.0010) [2023-12-26 21:28:34,154][105620] Updated weights for policy 1, policy_version 839797 (0.0007) [2023-12-26 21:28:34,886][105692] Updated weights for policy 0, policy_version 839850 (0.0008) [2023-12-26 21:28:34,923][105620] Updated weights for policy 1, policy_version 839807 (0.0008) [2023-12-26 21:28:34,947][105692] Updated weights for policy 0, policy_version 839860 (0.0010) [2023-12-26 21:28:34,970][105620] Updated weights for policy 1, policy_version 839817 (0.0008) [2023-12-26 21:28:34,999][105692] Updated weights for policy 0, policy_version 839870 (0.0010) [2023-12-26 21:28:35,026][105620] Updated weights for policy 1, policy_version 839827 (0.0006) [2023-12-26 21:28:35,062][105692] Updated weights for policy 0, policy_version 839880 (0.0011) [2023-12-26 21:28:35,792][105620] Updated weights for policy 1, policy_version 839837 (0.0006) [2023-12-26 21:28:35,798][105692] Updated weights for policy 0, policy_version 839890 (0.0011) [2023-12-26 21:28:35,843][105620] Updated weights for policy 1, policy_version 839847 (0.0005) [2023-12-26 21:28:35,857][105692] Updated weights for policy 0, policy_version 839900 (0.0011) [2023-12-26 21:28:35,906][105620] Updated weights for policy 1, policy_version 839857 (0.0005) [2023-12-26 21:28:35,919][105692] Updated weights for policy 0, policy_version 839910 (0.0010) [2023-12-26 21:28:35,944][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000001 [2023-12-26 21:28:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 430080000. Throughput: 0: 9721.2, 1: 9868.1. Samples: 430065604. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-26 21:28:36,063][104569] Avg episode reward: [(0, '7459.939'), (1, '8666.901')] [2023-12-26 21:28:36,467][105620] Updated weights for policy 1, policy_version 839867 (0.0007) [2023-12-26 21:28:36,526][105620] Updated weights for policy 1, policy_version 839877 (0.0011) [2023-12-26 21:28:36,586][105620] Updated weights for policy 1, policy_version 839887 (0.0011) [2023-12-26 21:28:36,663][105692] Updated weights for policy 0, policy_version 839920 (0.0011) [2023-12-26 21:28:36,718][105692] Updated weights for policy 0, policy_version 839930 (0.0011) [2023-12-26 21:28:36,780][105692] Updated weights for policy 0, policy_version 839940 (0.0011) [2023-12-26 21:28:37,246][105620] Updated weights for policy 1, policy_version 839897 (0.0010) [2023-12-26 21:28:37,306][105620] Updated weights for policy 1, policy_version 839907 (0.0006) [2023-12-26 21:28:37,366][105620] Updated weights for policy 1, policy_version 839917 (0.0009) [2023-12-26 21:28:37,426][105620] Updated weights for policy 1, policy_version 839927 (0.0009) [2023-12-26 21:28:37,537][105692] Updated weights for policy 0, policy_version 839950 (0.0007) [2023-12-26 21:28:37,592][105692] Updated weights for policy 0, policy_version 839960 (0.0008) [2023-12-26 21:28:37,642][105692] Updated weights for policy 0, policy_version 839970 (0.0009) [2023-12-26 21:28:38,103][105620] Updated weights for policy 1, policy_version 839937 (0.0010) [2023-12-26 21:28:38,158][105620] Updated weights for policy 1, policy_version 839947 (0.0010) [2023-12-26 21:28:38,212][105620] Updated weights for policy 1, policy_version 839957 (0.0010) [2023-12-26 21:28:38,313][105692] Updated weights for policy 0, policy_version 839980 (0.0008) [2023-12-26 21:28:38,373][105692] Updated weights for policy 0, policy_version 839990 (0.0009) [2023-12-26 21:28:38,426][105692] Updated weights for policy 0, policy_version 840000 (0.0009) [2023-12-26 21:28:38,906][105620] Updated weights for policy 1, policy_version 839967 (0.0006) [2023-12-26 21:28:38,959][105620] Updated weights for policy 1, policy_version 839977 (0.0005) [2023-12-26 21:28:39,019][105620] Updated weights for policy 1, policy_version 839987 (0.0008) [2023-12-26 21:28:39,239][105692] Updated weights for policy 0, policy_version 840010 (0.0010) [2023-12-26 21:28:39,307][105692] Updated weights for policy 0, policy_version 840020 (0.0009) [2023-12-26 21:28:39,374][105692] Updated weights for policy 0, policy_version 840030 (0.0008) [2023-12-26 21:28:39,440][105692] Updated weights for policy 0, policy_version 840040 (0.0008) [2023-12-26 21:28:39,762][105620] Updated weights for policy 1, policy_version 839997 (0.0009) [2023-12-26 21:28:39,810][105620] Updated weights for policy 1, policy_version 840007 (0.0008) [2023-12-26 21:28:39,883][105620] Updated weights for policy 1, policy_version 840017 (0.0008) [2023-12-26 21:28:40,108][105692] Updated weights for policy 0, policy_version 840050 (0.0009) [2023-12-26 21:28:40,167][105692] Updated weights for policy 0, policy_version 840061 (0.0010) [2023-12-26 21:28:40,242][105692] Updated weights for policy 0, policy_version 840071 (0.0009) [2023-12-26 21:28:40,535][105620] Updated weights for policy 1, policy_version 840027 (0.0007) [2023-12-26 21:28:40,593][105620] Updated weights for policy 1, policy_version 840037 (0.0005) [2023-12-26 21:28:40,647][105620] Updated weights for policy 1, policy_version 840047 (0.0005) [2023-12-26 21:28:41,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 430170112. Throughput: 0: 9659.5, 1: 9920.1. Samples: 430182532. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-26 21:28:41,062][104569] Avg episode reward: [(0, '7366.281'), (1, '8990.700')] [2023-12-26 21:28:41,069][105692] Updated weights for policy 0, policy_version 840081 (0.0008) [2023-12-26 21:28:41,138][105692] Updated weights for policy 0, policy_version 840091 (0.0007) [2023-12-26 21:28:41,199][105692] Updated weights for policy 0, policy_version 840101 (0.0010) [2023-12-26 21:28:41,362][105620] Updated weights for policy 1, policy_version 840057 (0.0008) [2023-12-26 21:28:41,439][105620] Updated weights for policy 1, policy_version 840067 (0.0011) [2023-12-26 21:28:41,504][105620] Updated weights for policy 1, policy_version 840077 (0.0009) [2023-12-26 21:28:41,572][105620] Updated weights for policy 1, policy_version 840087 (0.0006) [2023-12-26 21:28:42,002][105692] Updated weights for policy 0, policy_version 840111 (0.0010) [2023-12-26 21:28:42,058][105692] Updated weights for policy 0, policy_version 840121 (0.0009) [2023-12-26 21:28:42,123][105692] Updated weights for policy 0, policy_version 840131 (0.0008) [2023-12-26 21:28:42,244][105620] Updated weights for policy 1, policy_version 840097 (0.0010) [2023-12-26 21:28:42,303][105620] Updated weights for policy 1, policy_version 840107 (0.0010) [2023-12-26 21:28:42,371][105620] Updated weights for policy 1, policy_version 840117 (0.0009) [2023-12-26 21:28:42,812][105692] Updated weights for policy 0, policy_version 840141 (0.0007) [2023-12-26 21:28:42,872][105692] Updated weights for policy 0, policy_version 840151 (0.0005) [2023-12-26 21:28:42,928][105692] Updated weights for policy 0, policy_version 840161 (0.0009) [2023-12-26 21:28:43,045][105620] Updated weights for policy 1, policy_version 840127 (0.0010) [2023-12-26 21:28:43,110][105620] Updated weights for policy 1, policy_version 840137 (0.0009) [2023-12-26 21:28:43,183][105620] Updated weights for policy 1, policy_version 840147 (0.0005) [2023-12-26 21:28:43,513][105692] Updated weights for policy 0, policy_version 840171 (0.0008) [2023-12-26 21:28:43,570][105692] Updated weights for policy 0, policy_version 840181 (0.0007) [2023-12-26 21:28:43,634][105692] Updated weights for policy 0, policy_version 840191 (0.0010) [2023-12-26 21:28:43,895][105620] Updated weights for policy 1, policy_version 840157 (0.0007) [2023-12-26 21:28:43,959][105620] Updated weights for policy 1, policy_version 840167 (0.0010) [2023-12-26 21:28:44,027][105620] Updated weights for policy 1, policy_version 840177 (0.0008) [2023-12-26 21:28:44,268][105692] Updated weights for policy 0, policy_version 840201 (0.0005) [2023-12-26 21:28:44,331][105692] Updated weights for policy 0, policy_version 840211 (0.0009) [2023-12-26 21:28:44,386][105692] Updated weights for policy 0, policy_version 840221 (0.0009) [2023-12-26 21:28:44,439][105692] Updated weights for policy 0, policy_version 840231 (0.0009) [2023-12-26 21:28:44,726][105620] Updated weights for policy 1, policy_version 840187 (0.0007) [2023-12-26 21:28:44,789][105620] Updated weights for policy 1, policy_version 840197 (0.0009) [2023-12-26 21:28:44,846][105620] Updated weights for policy 1, policy_version 840207 (0.0006) [2023-12-26 21:28:45,213][105692] Updated weights for policy 0, policy_version 840241 (0.0010) [2023-12-26 21:28:45,273][105692] Updated weights for policy 0, policy_version 840251 (0.0010) [2023-12-26 21:28:45,326][105692] Updated weights for policy 0, policy_version 840261 (0.0009) [2023-12-26 21:28:45,498][105620] Updated weights for policy 1, policy_version 840217 (0.0006) [2023-12-26 21:28:45,553][105620] Updated weights for policy 1, policy_version 840227 (0.0009) [2023-12-26 21:28:45,605][105620] Updated weights for policy 1, policy_version 840237 (0.0010) [2023-12-26 21:28:45,653][105620] Updated weights for policy 1, policy_version 840247 (0.0009) [2023-12-26 21:28:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.4, 300 sec: 19494.2). Total num frames: 430268416. Throughput: 0: 9644.0, 1: 9906.1. Samples: 430241152. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-26 21:28:46,063][104569] Avg episode reward: [(0, '8087.841'), (1, '9179.034')] [2023-12-26 21:28:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000840248_215130112.pth... [2023-12-26 21:28:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000839096_214835200.pth [2023-12-26 21:28:46,101][105692] Updated weights for policy 0, policy_version 840271 (0.0009) [2023-12-26 21:28:46,158][105692] Updated weights for policy 0, policy_version 840281 (0.0009) [2023-12-26 21:28:46,216][105692] Updated weights for policy 0, policy_version 840291 (0.0009) [2023-12-26 21:28:46,251][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000840296_215146496.pth... [2023-12-26 21:28:46,256][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000839144_214851584.pth [2023-12-26 21:28:46,350][105620] Updated weights for policy 1, policy_version 840257 (0.0008) [2023-12-26 21:28:46,417][105620] Updated weights for policy 1, policy_version 840267 (0.0006) [2023-12-26 21:28:46,474][105620] Updated weights for policy 1, policy_version 840277 (0.0005) [2023-12-26 21:28:46,989][105620] Updated weights for policy 1, policy_version 840287 (0.0005) [2023-12-26 21:28:47,051][105620] Updated weights for policy 1, policy_version 840297 (0.0005) [2023-12-26 21:28:47,117][105692] Updated weights for policy 0, policy_version 840301 (0.0009) [2023-12-26 21:28:47,121][105620] Updated weights for policy 1, policy_version 840307 (0.0005) [2023-12-26 21:28:47,175][105692] Updated weights for policy 0, policy_version 840311 (0.0009) [2023-12-26 21:28:47,234][105692] Updated weights for policy 0, policy_version 840322 (0.0010) [2023-12-26 21:28:47,641][105620] Updated weights for policy 1, policy_version 840317 (0.0007) [2023-12-26 21:28:47,699][105620] Updated weights for policy 1, policy_version 840327 (0.0009) [2023-12-26 21:28:47,764][105620] Updated weights for policy 1, policy_version 840337 (0.0006) [2023-12-26 21:28:48,037][105692] Updated weights for policy 0, policy_version 840332 (0.0010) [2023-12-26 21:28:48,084][105692] Updated weights for policy 0, policy_version 840342 (0.0009) [2023-12-26 21:28:48,135][105692] Updated weights for policy 0, policy_version 840352 (0.0009) [2023-12-26 21:28:48,404][105620] Updated weights for policy 1, policy_version 840347 (0.0008) [2023-12-26 21:28:48,468][105620] Updated weights for policy 1, policy_version 840357 (0.0008) [2023-12-26 21:28:48,530][105620] Updated weights for policy 1, policy_version 840367 (0.0009) [2023-12-26 21:28:48,969][105692] Updated weights for policy 0, policy_version 840362 (0.0009) [2023-12-26 21:28:49,035][105692] Updated weights for policy 0, policy_version 840372 (0.0008) [2023-12-26 21:28:49,090][105692] Updated weights for policy 0, policy_version 840382 (0.0005) [2023-12-26 21:28:49,142][105692] Updated weights for policy 0, policy_version 840392 (0.0005) [2023-12-26 21:28:49,164][105620] Updated weights for policy 1, policy_version 840377 (0.0010) [2023-12-26 21:28:49,217][105620] Updated weights for policy 1, policy_version 840387 (0.0010) [2023-12-26 21:28:49,276][105620] Updated weights for policy 1, policy_version 840397 (0.0007) [2023-12-26 21:28:49,336][105620] Updated weights for policy 1, policy_version 840407 (0.0008) [2023-12-26 21:28:49,747][105692] Updated weights for policy 0, policy_version 840402 (0.0005) [2023-12-26 21:28:49,802][105692] Updated weights for policy 0, policy_version 840412 (0.0011) [2023-12-26 21:28:49,865][105692] Updated weights for policy 0, policy_version 840422 (0.0008) [2023-12-26 21:28:50,200][105620] Updated weights for policy 1, policy_version 840417 (0.0008) [2023-12-26 21:28:50,256][105620] Updated weights for policy 1, policy_version 840427 (0.0008) [2023-12-26 21:28:50,317][105620] Updated weights for policy 1, policy_version 840437 (0.0009) [2023-12-26 21:28:50,527][105692] Updated weights for policy 0, policy_version 840432 (0.0006) [2023-12-26 21:28:50,585][105692] Updated weights for policy 0, policy_version 840442 (0.0009) [2023-12-26 21:28:50,648][105692] Updated weights for policy 0, policy_version 840452 (0.0006) [2023-12-26 21:28:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 430366720. Throughput: 0: 9587.8, 1: 10013.3. Samples: 430359112. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-26 21:28:51,063][104569] Avg episode reward: [(0, '8265.765'), (1, '9086.124')] [2023-12-26 21:28:51,149][105620] Updated weights for policy 1, policy_version 840447 (0.0009) [2023-12-26 21:28:51,204][105620] Updated weights for policy 1, policy_version 840457 (0.0006) [2023-12-26 21:28:51,261][105620] Updated weights for policy 1, policy_version 840467 (0.0006) [2023-12-26 21:28:51,350][105692] Updated weights for policy 0, policy_version 840462 (0.0008) [2023-12-26 21:28:51,416][105692] Updated weights for policy 0, policy_version 840472 (0.0007) [2023-12-26 21:28:51,484][105692] Updated weights for policy 0, policy_version 840482 (0.0006) [2023-12-26 21:28:51,872][105620] Updated weights for policy 1, policy_version 840477 (0.0006) [2023-12-26 21:28:51,927][105620] Updated weights for policy 1, policy_version 840487 (0.0008) [2023-12-26 21:28:51,985][105620] Updated weights for policy 1, policy_version 840497 (0.0008) [2023-12-26 21:28:52,182][105692] Updated weights for policy 0, policy_version 840492 (0.0008) [2023-12-26 21:28:52,247][105692] Updated weights for policy 0, policy_version 840502 (0.0010) [2023-12-26 21:28:52,312][105692] Updated weights for policy 0, policy_version 840512 (0.0009) [2023-12-26 21:28:52,733][105620] Updated weights for policy 1, policy_version 840507 (0.0007) [2023-12-26 21:28:52,793][105620] Updated weights for policy 1, policy_version 840517 (0.0008) [2023-12-26 21:28:52,850][105620] Updated weights for policy 1, policy_version 840527 (0.0008) [2023-12-26 21:28:53,037][105692] Updated weights for policy 0, policy_version 840522 (0.0011) [2023-12-26 21:28:53,094][105692] Updated weights for policy 0, policy_version 840532 (0.0007) [2023-12-26 21:28:53,149][105692] Updated weights for policy 0, policy_version 840542 (0.0009) [2023-12-26 21:28:53,212][105692] Updated weights for policy 0, policy_version 840552 (0.0007) [2023-12-26 21:28:53,611][105620] Updated weights for policy 1, policy_version 840537 (0.0008) [2023-12-26 21:28:53,661][105620] Updated weights for policy 1, policy_version 840547 (0.0008) [2023-12-26 21:28:53,718][105620] Updated weights for policy 1, policy_version 840557 (0.0008) [2023-12-26 21:28:53,776][105620] Updated weights for policy 1, policy_version 840567 (0.0010) [2023-12-26 21:28:53,877][105692] Updated weights for policy 0, policy_version 840562 (0.0008) [2023-12-26 21:28:53,928][105692] Updated weights for policy 0, policy_version 840572 (0.0008) [2023-12-26 21:28:53,971][105585] KL-divergence is very high: 121.5001 [2023-12-26 21:28:53,989][105692] Updated weights for policy 0, policy_version 840582 (0.0008) [2023-12-26 21:28:54,546][105620] Updated weights for policy 1, policy_version 840577 (0.0010) [2023-12-26 21:28:54,595][105620] Updated weights for policy 1, policy_version 840587 (0.0011) [2023-12-26 21:28:54,644][105620] Updated weights for policy 1, policy_version 840597 (0.0010) [2023-12-26 21:28:54,706][105692] Updated weights for policy 0, policy_version 840592 (0.0009) [2023-12-26 21:28:54,760][105692] Updated weights for policy 0, policy_version 840602 (0.0006) [2023-12-26 21:28:54,824][105692] Updated weights for policy 0, policy_version 840612 (0.0005) [2023-12-26 21:28:55,364][105692] Updated weights for policy 0, policy_version 840622 (0.0005) [2023-12-26 21:28:55,416][105692] Updated weights for policy 0, policy_version 840632 (0.0005) [2023-12-26 21:28:55,428][105620] Updated weights for policy 1, policy_version 840607 (0.0010) [2023-12-26 21:28:55,475][105692] Updated weights for policy 0, policy_version 840642 (0.0005) [2023-12-26 21:28:55,489][105620] Updated weights for policy 1, policy_version 840617 (0.0010) [2023-12-26 21:28:55,557][105620] Updated weights for policy 1, policy_version 840627 (0.0010) [2023-12-26 21:28:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 430465024. Throughput: 0: 9807.5, 1: 9877.6. Samples: 430476784. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-26 21:28:56,062][104569] Avg episode reward: [(0, '8268.448'), (1, '9019.800')] [2023-12-26 21:28:56,098][105692] Updated weights for policy 0, policy_version 840652 (0.0007) [2023-12-26 21:28:56,153][105692] Updated weights for policy 0, policy_version 840662 (0.0010) [2023-12-26 21:28:56,201][105692] Updated weights for policy 0, policy_version 840672 (0.0011) [2023-12-26 21:28:56,247][105620] Updated weights for policy 1, policy_version 840637 (0.0008) [2023-12-26 21:28:56,309][105620] Updated weights for policy 1, policy_version 840647 (0.0005) [2023-12-26 21:28:56,367][105620] Updated weights for policy 1, policy_version 840657 (0.0005) [2023-12-26 21:28:56,903][105692] Updated weights for policy 0, policy_version 840682 (0.0010) [2023-12-26 21:28:56,949][105692] Updated weights for policy 0, policy_version 840692 (0.0010) [2023-12-26 21:28:56,993][105692] Updated weights for policy 0, policy_version 840702 (0.0008) [2023-12-26 21:28:56,999][105620] Updated weights for policy 1, policy_version 840667 (0.0007) [2023-12-26 21:28:57,051][105620] Updated weights for policy 1, policy_version 840677 (0.0010) [2023-12-26 21:28:57,053][105692] Updated weights for policy 0, policy_version 840712 (0.0005) [2023-12-26 21:28:57,097][105620] Updated weights for policy 1, policy_version 840687 (0.0010) [2023-12-26 21:28:57,715][105620] Updated weights for policy 1, policy_version 840697 (0.0009) [2023-12-26 21:28:57,770][105620] Updated weights for policy 1, policy_version 840707 (0.0005) [2023-12-26 21:28:57,824][105620] Updated weights for policy 1, policy_version 840717 (0.0005) [2023-12-26 21:28:57,877][105620] Updated weights for policy 1, policy_version 840727 (0.0005) [2023-12-26 21:28:57,916][105692] Updated weights for policy 0, policy_version 840722 (0.0008) [2023-12-26 21:28:57,970][105692] Updated weights for policy 0, policy_version 840734 (0.0010) [2023-12-26 21:28:58,487][105620] Updated weights for policy 1, policy_version 840737 (0.0007) [2023-12-26 21:28:58,565][105620] Updated weights for policy 1, policy_version 840747 (0.0009) [2023-12-26 21:28:58,630][105620] Updated weights for policy 1, policy_version 840757 (0.0007) [2023-12-26 21:28:58,843][105692] Updated weights for policy 0, policy_version 840745 (0.0010) [2023-12-26 21:28:58,920][105692] Updated weights for policy 0, policy_version 840755 (0.0008) [2023-12-26 21:28:58,984][105692] Updated weights for policy 0, policy_version 840765 (0.0009) [2023-12-26 21:28:59,038][105692] Updated weights for policy 0, policy_version 840775 (0.0010) [2023-12-26 21:28:59,314][105620] Updated weights for policy 1, policy_version 840767 (0.0008) [2023-12-26 21:28:59,377][105620] Updated weights for policy 1, policy_version 840777 (0.0008) [2023-12-26 21:28:59,437][105620] Updated weights for policy 1, policy_version 840787 (0.0009) [2023-12-26 21:28:59,767][105692] Updated weights for policy 0, policy_version 840785 (0.0009) [2023-12-26 21:28:59,824][105692] Updated weights for policy 0, policy_version 840795 (0.0008) [2023-12-26 21:28:59,882][105692] Updated weights for policy 0, policy_version 840805 (0.0009) [2023-12-26 21:29:00,180][105620] Updated weights for policy 1, policy_version 840797 (0.0007) [2023-12-26 21:29:00,235][105620] Updated weights for policy 1, policy_version 840807 (0.0006) [2023-12-26 21:29:00,281][105620] Updated weights for policy 1, policy_version 840817 (0.0008) [2023-12-26 21:29:00,625][105692] Updated weights for policy 0, policy_version 840815 (0.0010) [2023-12-26 21:29:00,676][105692] Updated weights for policy 0, policy_version 840825 (0.0009) [2023-12-26 21:29:00,723][105692] Updated weights for policy 0, policy_version 840835 (0.0006) [2023-12-26 21:29:00,938][105620] Updated weights for policy 1, policy_version 840827 (0.0007) [2023-12-26 21:29:00,990][105620] Updated weights for policy 1, policy_version 840837 (0.0009) [2023-12-26 21:29:01,055][105620] Updated weights for policy 1, policy_version 840847 (0.0009) [2023-12-26 21:29:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 430563328. Throughput: 0: 9741.3, 1: 9998.6. Samples: 430536544. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-26 21:29:01,063][104569] Avg episode reward: [(0, '8449.305'), (1, '8917.977')] [2023-12-26 21:29:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000840840_215285760.pth... [2023-12-26 21:29:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000839720_214999040.pth [2023-12-26 21:29:01,108][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000840856_215285760.pth... [2023-12-26 21:29:01,112][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000839672_214982656.pth [2023-12-26 21:29:01,433][105692] Updated weights for policy 0, policy_version 840845 (0.0007) [2023-12-26 21:29:01,497][105692] Updated weights for policy 0, policy_version 840855 (0.0008) [2023-12-26 21:29:01,548][105692] Updated weights for policy 0, policy_version 840865 (0.0008) [2023-12-26 21:29:01,676][105620] Updated weights for policy 1, policy_version 840857 (0.0007) [2023-12-26 21:29:01,733][105620] Updated weights for policy 1, policy_version 840867 (0.0008) [2023-12-26 21:29:01,796][105620] Updated weights for policy 1, policy_version 840877 (0.0006) [2023-12-26 21:29:01,863][105620] Updated weights for policy 1, policy_version 840887 (0.0005) [2023-12-26 21:29:02,305][105692] Updated weights for policy 0, policy_version 840875 (0.0008) [2023-12-26 21:29:02,370][105692] Updated weights for policy 0, policy_version 840885 (0.0011) [2023-12-26 21:29:02,443][105692] Updated weights for policy 0, policy_version 840895 (0.0011) [2023-12-26 21:29:02,518][105620] Updated weights for policy 1, policy_version 840897 (0.0005) [2023-12-26 21:29:02,583][105620] Updated weights for policy 1, policy_version 840907 (0.0007) [2023-12-26 21:29:02,637][105620] Updated weights for policy 1, policy_version 840917 (0.0007) [2023-12-26 21:29:03,044][105692] Updated weights for policy 0, policy_version 840905 (0.0006) [2023-12-26 21:29:03,112][105692] Updated weights for policy 0, policy_version 840915 (0.0006) [2023-12-26 21:29:03,158][105620] Updated weights for policy 1, policy_version 840927 (0.0006) [2023-12-26 21:29:03,164][105692] Updated weights for policy 0, policy_version 840925 (0.0010) [2023-12-26 21:29:03,215][105620] Updated weights for policy 1, policy_version 840937 (0.0006) [2023-12-26 21:29:03,215][105692] Updated weights for policy 0, policy_version 840935 (0.0010) [2023-12-26 21:29:03,277][105620] Updated weights for policy 1, policy_version 840947 (0.0006) [2023-12-26 21:29:03,818][105620] Updated weights for policy 1, policy_version 840957 (0.0007) [2023-12-26 21:29:03,853][105692] Updated weights for policy 0, policy_version 840945 (0.0007) [2023-12-26 21:29:03,878][105620] Updated weights for policy 1, policy_version 840967 (0.0008) [2023-12-26 21:29:03,916][105692] Updated weights for policy 0, policy_version 840955 (0.0006) [2023-12-26 21:29:03,941][105620] Updated weights for policy 1, policy_version 840977 (0.0008) [2023-12-26 21:29:03,978][105692] Updated weights for policy 0, policy_version 840965 (0.0005) [2023-12-26 21:29:04,523][105692] Updated weights for policy 0, policy_version 840975 (0.0005) [2023-12-26 21:29:04,585][105692] Updated weights for policy 0, policy_version 840985 (0.0005) [2023-12-26 21:29:04,647][105692] Updated weights for policy 0, policy_version 840995 (0.0007) [2023-12-26 21:29:04,777][105620] Updated weights for policy 1, policy_version 840987 (0.0009) [2023-12-26 21:29:04,841][105620] Updated weights for policy 1, policy_version 840997 (0.0009) [2023-12-26 21:29:04,898][105620] Updated weights for policy 1, policy_version 841007 (0.0009) [2023-12-26 21:29:05,338][105692] Updated weights for policy 0, policy_version 841005 (0.0009) [2023-12-26 21:29:05,386][105692] Updated weights for policy 0, policy_version 841015 (0.0009) [2023-12-26 21:29:05,440][105692] Updated weights for policy 0, policy_version 841025 (0.0008) [2023-12-26 21:29:05,649][105620] Updated weights for policy 1, policy_version 841017 (0.0010) [2023-12-26 21:29:05,704][105620] Updated weights for policy 1, policy_version 841027 (0.0007) [2023-12-26 21:29:05,759][105620] Updated weights for policy 1, policy_version 841037 (0.0005) [2023-12-26 21:29:05,818][105620] Updated weights for policy 1, policy_version 841047 (0.0005) [2023-12-26 21:29:06,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 430669824. Throughput: 0: 9646.6, 1: 10039.0. Samples: 430659228. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-26 21:29:06,063][104569] Avg episode reward: [(0, '8533.769'), (1, '8685.217')] [2023-12-26 21:29:06,177][105692] Updated weights for policy 0, policy_version 841035 (0.0009) [2023-12-26 21:29:06,240][105692] Updated weights for policy 0, policy_version 841045 (0.0008) [2023-12-26 21:29:06,300][105692] Updated weights for policy 0, policy_version 841055 (0.0008) [2023-12-26 21:29:06,496][105620] Updated weights for policy 1, policy_version 841057 (0.0008) [2023-12-26 21:29:06,548][105620] Updated weights for policy 1, policy_version 841067 (0.0008) [2023-12-26 21:29:06,598][105620] Updated weights for policy 1, policy_version 841077 (0.0008) [2023-12-26 21:29:07,060][105692] Updated weights for policy 0, policy_version 841065 (0.0008) [2023-12-26 21:29:07,119][105692] Updated weights for policy 0, policy_version 841075 (0.0009) [2023-12-26 21:29:07,169][105692] Updated weights for policy 0, policy_version 841085 (0.0008) [2023-12-26 21:29:07,220][105692] Updated weights for policy 0, policy_version 841095 (0.0009) [2023-12-26 21:29:07,374][105620] Updated weights for policy 1, policy_version 841087 (0.0010) [2023-12-26 21:29:07,436][105620] Updated weights for policy 1, policy_version 841097 (0.0009) [2023-12-26 21:29:07,499][105620] Updated weights for policy 1, policy_version 841107 (0.0006) [2023-12-26 21:29:07,944][105692] Updated weights for policy 0, policy_version 841105 (0.0007) [2023-12-26 21:29:07,999][105692] Updated weights for policy 0, policy_version 841115 (0.0010) [2023-12-26 21:29:08,057][105692] Updated weights for policy 0, policy_version 841125 (0.0010) [2023-12-26 21:29:08,247][105620] Updated weights for policy 1, policy_version 841117 (0.0007) [2023-12-26 21:29:08,312][105620] Updated weights for policy 1, policy_version 841127 (0.0008) [2023-12-26 21:29:08,375][105620] Updated weights for policy 1, policy_version 841137 (0.0007) [2023-12-26 21:29:08,820][105692] Updated weights for policy 0, policy_version 841135 (0.0011) [2023-12-26 21:29:08,884][105692] Updated weights for policy 0, policy_version 841145 (0.0011) [2023-12-26 21:29:08,949][105692] Updated weights for policy 0, policy_version 841155 (0.0011) [2023-12-26 21:29:09,131][105620] Updated weights for policy 1, policy_version 841147 (0.0008) [2023-12-26 21:29:09,183][105620] Updated weights for policy 1, policy_version 841157 (0.0008) [2023-12-26 21:29:09,262][105620] Updated weights for policy 1, policy_version 841167 (0.0007) [2023-12-26 21:29:09,697][105692] Updated weights for policy 0, policy_version 841165 (0.0011) [2023-12-26 21:29:09,756][105692] Updated weights for policy 0, policy_version 841175 (0.0011) [2023-12-26 21:29:09,815][105692] Updated weights for policy 0, policy_version 841185 (0.0008) [2023-12-26 21:29:10,010][105620] Updated weights for policy 1, policy_version 841177 (0.0007) [2023-12-26 21:29:10,073][105620] Updated weights for policy 1, policy_version 841187 (0.0008) [2023-12-26 21:29:10,138][105620] Updated weights for policy 1, policy_version 841197 (0.0006) [2023-12-26 21:29:10,199][105620] Updated weights for policy 1, policy_version 841207 (0.0008) [2023-12-26 21:29:10,553][105692] Updated weights for policy 0, policy_version 841195 (0.0009) [2023-12-26 21:29:10,605][105692] Updated weights for policy 0, policy_version 841205 (0.0010) [2023-12-26 21:29:10,657][105692] Updated weights for policy 0, policy_version 841215 (0.0010) [2023-12-26 21:29:10,931][105620] Updated weights for policy 1, policy_version 841217 (0.0008) [2023-12-26 21:29:10,997][105620] Updated weights for policy 1, policy_version 841227 (0.0007) [2023-12-26 21:29:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 430759936. Throughput: 0: 9679.5, 1: 9976.4. Samples: 430772240. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-26 21:29:11,063][104569] Avg episode reward: [(0, '8442.352'), (1, '8835.760')] [2023-12-26 21:29:11,066][105620] Updated weights for policy 1, policy_version 841237 (0.0008) [2023-12-26 21:29:11,446][105692] Updated weights for policy 0, policy_version 841225 (0.0010) [2023-12-26 21:29:11,502][105692] Updated weights for policy 0, policy_version 841235 (0.0008) [2023-12-26 21:29:11,558][105692] Updated weights for policy 0, policy_version 841245 (0.0008) [2023-12-26 21:29:11,615][105692] Updated weights for policy 0, policy_version 841255 (0.0008) [2023-12-26 21:29:11,912][105620] Updated weights for policy 1, policy_version 841248 (0.0010) [2023-12-26 21:29:11,980][105620] Updated weights for policy 1, policy_version 841258 (0.0010) [2023-12-26 21:29:12,040][105620] Updated weights for policy 1, policy_version 841268 (0.0010) [2023-12-26 21:29:12,263][105692] Updated weights for policy 0, policy_version 841265 (0.0006) [2023-12-26 21:29:12,327][105692] Updated weights for policy 0, policy_version 841275 (0.0009) [2023-12-26 21:29:12,393][105692] Updated weights for policy 0, policy_version 841285 (0.0008) [2023-12-26 21:29:12,807][105620] Updated weights for policy 1, policy_version 841278 (0.0007) [2023-12-26 21:29:12,856][105620] Updated weights for policy 1, policy_version 841288 (0.0005) [2023-12-26 21:29:12,919][105620] Updated weights for policy 1, policy_version 841298 (0.0008) [2023-12-26 21:29:13,245][105692] Updated weights for policy 0, policy_version 841295 (0.0008) [2023-12-26 21:29:13,327][105692] Updated weights for policy 0, policy_version 841305 (0.0009) [2023-12-26 21:29:13,383][105692] Updated weights for policy 0, policy_version 841315 (0.0009) [2023-12-26 21:29:13,455][105620] Updated weights for policy 1, policy_version 841308 (0.0008) [2023-12-26 21:29:13,508][105620] Updated weights for policy 1, policy_version 841318 (0.0011) [2023-12-26 21:29:13,556][105620] Updated weights for policy 1, policy_version 841328 (0.0010) [2023-12-26 21:29:14,000][105692] Updated weights for policy 0, policy_version 841325 (0.0007) [2023-12-26 21:29:14,060][105692] Updated weights for policy 0, policy_version 841335 (0.0008) [2023-12-26 21:29:14,124][105692] Updated weights for policy 0, policy_version 841345 (0.0009) [2023-12-26 21:29:14,267][105620] Updated weights for policy 1, policy_version 841338 (0.0009) [2023-12-26 21:29:14,323][105620] Updated weights for policy 1, policy_version 841348 (0.0005) [2023-12-26 21:29:14,385][105620] Updated weights for policy 1, policy_version 841358 (0.0006) [2023-12-26 21:29:14,451][105620] Updated weights for policy 1, policy_version 841368 (0.0006) [2023-12-26 21:29:14,886][105692] Updated weights for policy 0, policy_version 841355 (0.0009) [2023-12-26 21:29:14,952][105692] Updated weights for policy 0, policy_version 841365 (0.0008) [2023-12-26 21:29:15,012][105692] Updated weights for policy 0, policy_version 841375 (0.0009) [2023-12-26 21:29:15,117][105620] Updated weights for policy 1, policy_version 841378 (0.0008) [2023-12-26 21:29:15,178][105620] Updated weights for policy 1, policy_version 841388 (0.0007) [2023-12-26 21:29:15,243][105620] Updated weights for policy 1, policy_version 841398 (0.0009) [2023-12-26 21:29:15,723][105692] Updated weights for policy 0, policy_version 841385 (0.0009) [2023-12-26 21:29:15,782][105692] Updated weights for policy 0, policy_version 841395 (0.0010) [2023-12-26 21:29:15,844][105692] Updated weights for policy 0, policy_version 841405 (0.0009) [2023-12-26 21:29:15,909][105692] Updated weights for policy 0, policy_version 841415 (0.0009) [2023-12-26 21:29:15,995][105620] Updated weights for policy 1, policy_version 841408 (0.0009) [2023-12-26 21:29:16,053][105620] Updated weights for policy 1, policy_version 841418 (0.0009) [2023-12-26 21:29:16,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 430858240. Throughput: 0: 9581.6, 1: 9945.2. Samples: 430828836. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-26 21:29:16,062][104569] Avg episode reward: [(0, '8533.059'), (1, '9094.595')] [2023-12-26 21:29:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000841416_215433216.pth... [2023-12-26 21:29:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000840296_215146496.pth [2023-12-26 21:29:16,115][105620] Updated weights for policy 1, policy_version 841428 (0.0009) [2023-12-26 21:29:16,132][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000841432_215433216.pth... [2023-12-26 21:29:16,136][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000840248_215130112.pth [2023-12-26 21:29:16,621][105692] Updated weights for policy 0, policy_version 841425 (0.0008) [2023-12-26 21:29:16,682][105692] Updated weights for policy 0, policy_version 841435 (0.0008) [2023-12-26 21:29:16,748][105692] Updated weights for policy 0, policy_version 841445 (0.0007) [2023-12-26 21:29:16,797][105620] Updated weights for policy 1, policy_version 841438 (0.0009) [2023-12-26 21:29:16,844][105620] Updated weights for policy 1, policy_version 841448 (0.0008) [2023-12-26 21:29:16,893][105620] Updated weights for policy 1, policy_version 841458 (0.0009) [2023-12-26 21:29:17,337][105692] Updated weights for policy 0, policy_version 841455 (0.0006) [2023-12-26 21:29:17,388][105692] Updated weights for policy 0, policy_version 841465 (0.0005) [2023-12-26 21:29:17,441][105692] Updated weights for policy 0, policy_version 841475 (0.0005) [2023-12-26 21:29:17,737][105620] Updated weights for policy 1, policy_version 841468 (0.0009) [2023-12-26 21:29:17,783][105620] Updated weights for policy 1, policy_version 841478 (0.0008) [2023-12-26 21:29:17,831][105620] Updated weights for policy 1, policy_version 841488 (0.0009) [2023-12-26 21:29:18,008][105692] Updated weights for policy 0, policy_version 841485 (0.0007) [2023-12-26 21:29:18,066][105692] Updated weights for policy 0, policy_version 841495 (0.0009) [2023-12-26 21:29:18,127][105692] Updated weights for policy 0, policy_version 841505 (0.0008) [2023-12-26 21:29:18,582][105620] Updated weights for policy 1, policy_version 841498 (0.0008) [2023-12-26 21:29:18,648][105620] Updated weights for policy 1, policy_version 841508 (0.0007) [2023-12-26 21:29:18,704][105620] Updated weights for policy 1, policy_version 841518 (0.0009) [2023-12-26 21:29:18,755][105620] Updated weights for policy 1, policy_version 841528 (0.0009) [2023-12-26 21:29:18,889][105692] Updated weights for policy 0, policy_version 841515 (0.0009) [2023-12-26 21:29:18,948][105692] Updated weights for policy 0, policy_version 841525 (0.0009) [2023-12-26 21:29:18,995][105692] Updated weights for policy 0, policy_version 841535 (0.0008) [2023-12-26 21:29:19,551][105620] Updated weights for policy 1, policy_version 841538 (0.0008) [2023-12-26 21:29:19,607][105620] Updated weights for policy 1, policy_version 841548 (0.0009) [2023-12-26 21:29:19,668][105620] Updated weights for policy 1, policy_version 841558 (0.0010) [2023-12-26 21:29:19,746][105692] Updated weights for policy 0, policy_version 841545 (0.0008) [2023-12-26 21:29:19,811][105692] Updated weights for policy 0, policy_version 841555 (0.0008) [2023-12-26 21:29:19,877][105692] Updated weights for policy 0, policy_version 841565 (0.0009) [2023-12-26 21:29:19,944][105692] Updated weights for policy 0, policy_version 841575 (0.0009) [2023-12-26 21:29:20,499][105620] Updated weights for policy 1, policy_version 841568 (0.0009) [2023-12-26 21:29:20,552][105620] Updated weights for policy 1, policy_version 841578 (0.0008) [2023-12-26 21:29:20,607][105692] Updated weights for policy 0, policy_version 841585 (0.0007) [2023-12-26 21:29:20,620][105620] Updated weights for policy 1, policy_version 841588 (0.0007) [2023-12-26 21:29:20,676][105692] Updated weights for policy 0, policy_version 841595 (0.0007) [2023-12-26 21:29:20,740][105692] Updated weights for policy 0, policy_version 841605 (0.0007) [2023-12-26 21:29:21,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 430956544. Throughput: 0: 9664.9, 1: 9891.0. Samples: 430945620. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-26 21:29:21,062][104569] Avg episode reward: [(0, '8718.685'), (1, '9187.836')] [2023-12-26 21:29:21,397][105692] Updated weights for policy 0, policy_version 841615 (0.0009) [2023-12-26 21:29:21,445][105620] Updated weights for policy 1, policy_version 841598 (0.0009) [2023-12-26 21:29:21,462][105692] Updated weights for policy 0, policy_version 841625 (0.0008) [2023-12-26 21:29:21,500][105620] Updated weights for policy 1, policy_version 841608 (0.0009) [2023-12-26 21:29:21,521][105692] Updated weights for policy 0, policy_version 841635 (0.0008) [2023-12-26 21:29:21,557][105620] Updated weights for policy 1, policy_version 841618 (0.0007) [2023-12-26 21:29:22,264][105620] Updated weights for policy 1, policy_version 841628 (0.0008) [2023-12-26 21:29:22,282][105692] Updated weights for policy 0, policy_version 841645 (0.0009) [2023-12-26 21:29:22,327][105620] Updated weights for policy 1, policy_version 841638 (0.0007) [2023-12-26 21:29:22,350][105692] Updated weights for policy 0, policy_version 841655 (0.0011) [2023-12-26 21:29:22,392][105620] Updated weights for policy 1, policy_version 841648 (0.0007) [2023-12-26 21:29:22,413][105692] Updated weights for policy 0, policy_version 841665 (0.0011) [2023-12-26 21:29:23,021][105620] Updated weights for policy 1, policy_version 841658 (0.0008) [2023-12-26 21:29:23,085][105620] Updated weights for policy 1, policy_version 841668 (0.0005) [2023-12-26 21:29:23,148][105620] Updated weights for policy 1, policy_version 841678 (0.0008) [2023-12-26 21:29:23,184][105692] Updated weights for policy 0, policy_version 841675 (0.0010) [2023-12-26 21:29:23,201][105620] Updated weights for policy 1, policy_version 841688 (0.0005) [2023-12-26 21:29:23,246][105692] Updated weights for policy 0, policy_version 841685 (0.0011) [2023-12-26 21:29:23,304][105692] Updated weights for policy 0, policy_version 841695 (0.0010) [2023-12-26 21:29:23,761][105620] Updated weights for policy 1, policy_version 841698 (0.0009) [2023-12-26 21:29:23,819][105620] Updated weights for policy 1, policy_version 841708 (0.0007) [2023-12-26 21:29:23,884][105620] Updated weights for policy 1, policy_version 841718 (0.0005) [2023-12-26 21:29:23,997][105692] Updated weights for policy 0, policy_version 841705 (0.0010) [2023-12-26 21:29:24,051][105692] Updated weights for policy 0, policy_version 841715 (0.0005) [2023-12-26 21:29:24,110][105692] Updated weights for policy 0, policy_version 841725 (0.0005) [2023-12-26 21:29:24,174][105692] Updated weights for policy 0, policy_version 841735 (0.0010) [2023-12-26 21:29:24,488][105620] Updated weights for policy 1, policy_version 841728 (0.0009) [2023-12-26 21:29:24,542][105620] Updated weights for policy 1, policy_version 841738 (0.0010) [2023-12-26 21:29:24,601][105620] Updated weights for policy 1, policy_version 841748 (0.0010) [2023-12-26 21:29:24,857][105692] Updated weights for policy 0, policy_version 841745 (0.0006) [2023-12-26 21:29:24,915][105692] Updated weights for policy 0, policy_version 841755 (0.0009) [2023-12-26 21:29:24,981][105692] Updated weights for policy 0, policy_version 841765 (0.0007) [2023-12-26 21:29:25,372][105620] Updated weights for policy 1, policy_version 841758 (0.0010) [2023-12-26 21:29:25,431][105620] Updated weights for policy 1, policy_version 841768 (0.0011) [2023-12-26 21:29:25,484][105620] Updated weights for policy 1, policy_version 841778 (0.0011) [2023-12-26 21:29:25,591][105692] Updated weights for policy 0, policy_version 841775 (0.0008) [2023-12-26 21:29:25,640][105692] Updated weights for policy 0, policy_version 841785 (0.0008) [2023-12-26 21:29:25,697][105692] Updated weights for policy 0, policy_version 841795 (0.0008) [2023-12-26 21:29:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19549.8). Total num frames: 431054848. Throughput: 0: 9745.5, 1: 9843.4. Samples: 431064032. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-26 21:29:26,063][104569] Avg episode reward: [(0, '8900.987'), (1, '9140.129')] [2023-12-26 21:29:26,196][105620] Updated weights for policy 1, policy_version 841788 (0.0009) [2023-12-26 21:29:26,252][105620] Updated weights for policy 1, policy_version 841798 (0.0005) [2023-12-26 21:29:26,309][105620] Updated weights for policy 1, policy_version 841808 (0.0005) [2023-12-26 21:29:26,548][105692] Updated weights for policy 0, policy_version 841805 (0.0009) [2023-12-26 21:29:26,615][105692] Updated weights for policy 0, policy_version 841815 (0.0008) [2023-12-26 21:29:26,681][105692] Updated weights for policy 0, policy_version 841825 (0.0007) [2023-12-26 21:29:26,918][105620] Updated weights for policy 1, policy_version 841818 (0.0006) [2023-12-26 21:29:26,972][105620] Updated weights for policy 1, policy_version 841828 (0.0009) [2023-12-26 21:29:27,021][105620] Updated weights for policy 1, policy_version 841838 (0.0008) [2023-12-26 21:29:27,075][105620] Updated weights for policy 1, policy_version 841848 (0.0009) [2023-12-26 21:29:27,389][105692] Updated weights for policy 0, policy_version 841835 (0.0009) [2023-12-26 21:29:27,445][105692] Updated weights for policy 0, policy_version 841845 (0.0009) [2023-12-26 21:29:27,496][105692] Updated weights for policy 0, policy_version 841855 (0.0009) [2023-12-26 21:29:27,862][105620] Updated weights for policy 1, policy_version 841858 (0.0005) [2023-12-26 21:29:27,907][105620] Updated weights for policy 1, policy_version 841868 (0.0005) [2023-12-26 21:29:27,952][105620] Updated weights for policy 1, policy_version 841878 (0.0005) [2023-12-26 21:29:28,255][105692] Updated weights for policy 0, policy_version 841865 (0.0009) [2023-12-26 21:29:28,311][105692] Updated weights for policy 0, policy_version 841875 (0.0009) [2023-12-26 21:29:28,369][105692] Updated weights for policy 0, policy_version 841885 (0.0010) [2023-12-26 21:29:28,432][105692] Updated weights for policy 0, policy_version 841895 (0.0009) [2023-12-26 21:29:28,503][105620] Updated weights for policy 1, policy_version 841888 (0.0008) [2023-12-26 21:29:28,553][105620] Updated weights for policy 1, policy_version 841898 (0.0008) [2023-12-26 21:29:28,600][105620] Updated weights for policy 1, policy_version 841908 (0.0009) [2023-12-26 21:29:29,214][105692] Updated weights for policy 0, policy_version 841905 (0.0009) [2023-12-26 21:29:29,279][105692] Updated weights for policy 0, policy_version 841915 (0.0007) [2023-12-26 21:29:29,327][105692] Updated weights for policy 0, policy_version 841925 (0.0008) [2023-12-26 21:29:29,350][105620] Updated weights for policy 1, policy_version 841918 (0.0008) [2023-12-26 21:29:29,422][105620] Updated weights for policy 1, policy_version 841928 (0.0009) [2023-12-26 21:29:29,477][105620] Updated weights for policy 1, policy_version 841938 (0.0009) [2023-12-26 21:29:30,059][105692] Updated weights for policy 0, policy_version 841935 (0.0009) [2023-12-26 21:29:30,115][105692] Updated weights for policy 0, policy_version 841945 (0.0006) [2023-12-26 21:29:30,173][105692] Updated weights for policy 0, policy_version 841955 (0.0006) [2023-12-26 21:29:30,203][105620] Updated weights for policy 1, policy_version 841948 (0.0009) [2023-12-26 21:29:30,273][105620] Updated weights for policy 1, policy_version 841958 (0.0009) [2023-12-26 21:29:30,324][105620] Updated weights for policy 1, policy_version 841968 (0.0010) [2023-12-26 21:29:30,912][105692] Updated weights for policy 0, policy_version 841965 (0.0007) [2023-12-26 21:29:30,959][105692] Updated weights for policy 0, policy_version 841975 (0.0009) [2023-12-26 21:29:30,983][105620] Updated weights for policy 1, policy_version 841978 (0.0008) [2023-12-26 21:29:31,017][105692] Updated weights for policy 0, policy_version 841985 (0.0009) [2023-12-26 21:29:31,049][105620] Updated weights for policy 1, policy_version 841988 (0.0008) [2023-12-26 21:29:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 431153152. Throughput: 0: 9689.4, 1: 9903.4. Samples: 431122824. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-26 21:29:31,062][104569] Avg episode reward: [(0, '8902.801'), (1, '9211.272')] [2023-12-26 21:29:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000841992_215580672.pth... [2023-12-26 21:29:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000840840_215285760.pth [2023-12-26 21:29:31,103][105620] Updated weights for policy 1, policy_version 841998 (0.0006) [2023-12-26 21:29:31,162][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000842008_215580672.pth... [2023-12-26 21:29:31,163][105620] Updated weights for policy 1, policy_version 842008 (0.0009) [2023-12-26 21:29:31,167][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000840856_215285760.pth [2023-12-26 21:29:31,832][105692] Updated weights for policy 0, policy_version 841995 (0.0010) [2023-12-26 21:29:31,893][105692] Updated weights for policy 0, policy_version 842005 (0.0010) [2023-12-26 21:29:31,949][105620] Updated weights for policy 1, policy_version 842018 (0.0009) [2023-12-26 21:29:31,950][105692] Updated weights for policy 0, policy_version 842015 (0.0009) [2023-12-26 21:29:32,008][105620] Updated weights for policy 1, policy_version 842028 (0.0008) [2023-12-26 21:29:32,071][105620] Updated weights for policy 1, policy_version 842038 (0.0006) [2023-12-26 21:29:32,714][105620] Updated weights for policy 1, policy_version 842048 (0.0006) [2023-12-26 21:29:32,764][105692] Updated weights for policy 0, policy_version 842025 (0.0007) [2023-12-26 21:29:32,768][105620] Updated weights for policy 1, policy_version 842058 (0.0007) [2023-12-26 21:29:32,815][105692] Updated weights for policy 0, policy_version 842035 (0.0007) [2023-12-26 21:29:32,817][105620] Updated weights for policy 1, policy_version 842068 (0.0008) [2023-12-26 21:29:32,866][105692] Updated weights for policy 0, policy_version 842045 (0.0008) [2023-12-26 21:29:32,914][105692] Updated weights for policy 0, policy_version 842056 (0.0009) [2023-12-26 21:29:33,565][105620] Updated weights for policy 1, policy_version 842078 (0.0007) [2023-12-26 21:29:33,612][105692] Updated weights for policy 0, policy_version 842066 (0.0007) [2023-12-26 21:29:33,618][105620] Updated weights for policy 1, policy_version 842088 (0.0007) [2023-12-26 21:29:33,658][105692] Updated weights for policy 0, policy_version 842076 (0.0008) [2023-12-26 21:29:33,672][105620] Updated weights for policy 1, policy_version 842098 (0.0007) [2023-12-26 21:29:33,715][105692] Updated weights for policy 0, policy_version 842086 (0.0007) [2023-12-26 21:29:34,437][105620] Updated weights for policy 1, policy_version 842108 (0.0008) [2023-12-26 21:29:34,454][105692] Updated weights for policy 0, policy_version 842096 (0.0006) [2023-12-26 21:29:34,505][105620] Updated weights for policy 1, policy_version 842118 (0.0011) [2023-12-26 21:29:34,515][105692] Updated weights for policy 0, policy_version 842106 (0.0005) [2023-12-26 21:29:34,565][105620] Updated weights for policy 1, policy_version 842128 (0.0011) [2023-12-26 21:29:34,572][105692] Updated weights for policy 0, policy_version 842116 (0.0005) [2023-12-26 21:29:35,152][105692] Updated weights for policy 0, policy_version 842126 (0.0007) [2023-12-26 21:29:35,204][105692] Updated weights for policy 0, policy_version 842136 (0.0008) [2023-12-26 21:29:35,260][105692] Updated weights for policy 0, policy_version 842146 (0.0008) [2023-12-26 21:29:35,295][105620] Updated weights for policy 1, policy_version 842138 (0.0011) [2023-12-26 21:29:35,348][105620] Updated weights for policy 1, policy_version 842148 (0.0009) [2023-12-26 21:29:35,400][105620] Updated weights for policy 1, policy_version 842158 (0.0005) [2023-12-26 21:29:35,466][105620] Updated weights for policy 1, policy_version 842168 (0.0007) [2023-12-26 21:29:36,024][105692] Updated weights for policy 0, policy_version 842156 (0.0008) [2023-12-26 21:29:36,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 431243264. Throughput: 0: 9715.2, 1: 9786.5. Samples: 431236688. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-26 21:29:36,063][104569] Avg episode reward: [(0, '8991.224'), (1, '9258.875')] [2023-12-26 21:29:36,068][105692] Updated weights for policy 0, policy_version 842166 (0.0008) [2023-12-26 21:29:36,124][105692] Updated weights for policy 0, policy_version 842176 (0.0009) [2023-12-26 21:29:36,183][105620] Updated weights for policy 1, policy_version 842178 (0.0008) [2023-12-26 21:29:36,247][105620] Updated weights for policy 1, policy_version 842188 (0.0008) [2023-12-26 21:29:36,315][105620] Updated weights for policy 1, policy_version 842198 (0.0008) [2023-12-26 21:29:36,935][105692] Updated weights for policy 0, policy_version 842186 (0.0007) [2023-12-26 21:29:36,993][105692] Updated weights for policy 0, policy_version 842196 (0.0009) [2023-12-26 21:29:37,026][105620] Updated weights for policy 1, policy_version 842208 (0.0009) [2023-12-26 21:29:37,052][105692] Updated weights for policy 0, policy_version 842206 (0.0007) [2023-12-26 21:29:37,079][105620] Updated weights for policy 1, policy_version 842218 (0.0007) [2023-12-26 21:29:37,106][105692] Updated weights for policy 0, policy_version 842216 (0.0007) [2023-12-26 21:29:37,136][105620] Updated weights for policy 1, policy_version 842228 (0.0007) [2023-12-26 21:29:37,873][105692] Updated weights for policy 0, policy_version 842226 (0.0008) [2023-12-26 21:29:37,897][105620] Updated weights for policy 1, policy_version 842238 (0.0008) [2023-12-26 21:29:37,940][105692] Updated weights for policy 0, policy_version 842236 (0.0010) [2023-12-26 21:29:37,968][105620] Updated weights for policy 1, policy_version 842248 (0.0008) [2023-12-26 21:29:38,004][105692] Updated weights for policy 0, policy_version 842246 (0.0007) [2023-12-26 21:29:38,035][105620] Updated weights for policy 1, policy_version 842258 (0.0008) [2023-12-26 21:29:38,699][105620] Updated weights for policy 1, policy_version 842268 (0.0008) [2023-12-26 21:29:38,757][105692] Updated weights for policy 0, policy_version 842256 (0.0005) [2023-12-26 21:29:38,762][105620] Updated weights for policy 1, policy_version 842278 (0.0010) [2023-12-26 21:29:38,814][105692] Updated weights for policy 0, policy_version 842266 (0.0008) [2023-12-26 21:29:38,825][105620] Updated weights for policy 1, policy_version 842288 (0.0010) [2023-12-26 21:29:38,872][105692] Updated weights for policy 0, policy_version 842276 (0.0010) [2023-12-26 21:29:39,595][105692] Updated weights for policy 0, policy_version 842286 (0.0010) [2023-12-26 21:29:39,598][105620] Updated weights for policy 1, policy_version 842298 (0.0008) [2023-12-26 21:29:39,647][105620] Updated weights for policy 1, policy_version 842308 (0.0006) [2023-12-26 21:29:39,650][105692] Updated weights for policy 0, policy_version 842296 (0.0008) [2023-12-26 21:29:39,706][105620] Updated weights for policy 1, policy_version 842318 (0.0007) [2023-12-26 21:29:39,708][105692] Updated weights for policy 0, policy_version 842306 (0.0006) [2023-12-26 21:29:39,766][105620] Updated weights for policy 1, policy_version 842328 (0.0007) [2023-12-26 21:29:40,512][105692] Updated weights for policy 0, policy_version 842316 (0.0007) [2023-12-26 21:29:40,548][105620] Updated weights for policy 1, policy_version 842338 (0.0007) [2023-12-26 21:29:40,568][105692] Updated weights for policy 0, policy_version 842326 (0.0006) [2023-12-26 21:29:40,604][105620] Updated weights for policy 1, policy_version 842348 (0.0007) [2023-12-26 21:29:40,623][105692] Updated weights for policy 0, policy_version 842336 (0.0009) [2023-12-26 21:29:40,663][105620] Updated weights for policy 1, policy_version 842358 (0.0007) [2023-12-26 21:29:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 431341568. Throughput: 0: 9592.4, 1: 9806.5. Samples: 431349732. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-26 21:29:41,062][104569] Avg episode reward: [(0, '8810.300'), (1, '9167.310')] [2023-12-26 21:29:41,356][105620] Updated weights for policy 1, policy_version 842368 (0.0008) [2023-12-26 21:29:41,418][105620] Updated weights for policy 1, policy_version 842378 (0.0009) [2023-12-26 21:29:41,478][105692] Updated weights for policy 0, policy_version 842346 (0.0007) [2023-12-26 21:29:41,480][105620] Updated weights for policy 1, policy_version 842388 (0.0008) [2023-12-26 21:29:41,534][105692] Updated weights for policy 0, policy_version 842356 (0.0008) [2023-12-26 21:29:41,596][105692] Updated weights for policy 0, policy_version 842366 (0.0009) [2023-12-26 21:29:41,661][105692] Updated weights for policy 0, policy_version 842376 (0.0008) [2023-12-26 21:29:42,185][105620] Updated weights for policy 1, policy_version 842398 (0.0008) [2023-12-26 21:29:42,241][105620] Updated weights for policy 1, policy_version 842408 (0.0009) [2023-12-26 21:29:42,307][105620] Updated weights for policy 1, policy_version 842418 (0.0009) [2023-12-26 21:29:42,500][105692] Updated weights for policy 0, policy_version 842386 (0.0009) [2023-12-26 21:29:42,562][105692] Updated weights for policy 0, policy_version 842396 (0.0009) [2023-12-26 21:29:42,618][105692] Updated weights for policy 0, policy_version 842406 (0.0009) [2023-12-26 21:29:42,967][105620] Updated weights for policy 1, policy_version 842428 (0.0008) [2023-12-26 21:29:43,036][105620] Updated weights for policy 1, policy_version 842438 (0.0006) [2023-12-26 21:29:43,100][105620] Updated weights for policy 1, policy_version 842448 (0.0009) [2023-12-26 21:29:43,416][105692] Updated weights for policy 0, policy_version 842416 (0.0008) [2023-12-26 21:29:43,463][105692] Updated weights for policy 0, policy_version 842426 (0.0009) [2023-12-26 21:29:43,509][105692] Updated weights for policy 0, policy_version 842436 (0.0008) [2023-12-26 21:29:43,788][105620] Updated weights for policy 1, policy_version 842458 (0.0009) [2023-12-26 21:29:43,845][105620] Updated weights for policy 1, policy_version 842468 (0.0009) [2023-12-26 21:29:43,896][105620] Updated weights for policy 1, policy_version 842478 (0.0009) [2023-12-26 21:29:43,944][105620] Updated weights for policy 1, policy_version 842488 (0.0008) [2023-12-26 21:29:44,234][105692] Updated weights for policy 0, policy_version 842446 (0.0009) [2023-12-26 21:29:44,286][105692] Updated weights for policy 0, policy_version 842456 (0.0008) [2023-12-26 21:29:44,330][105692] Updated weights for policy 0, policy_version 842466 (0.0008) [2023-12-26 21:29:44,698][105620] Updated weights for policy 1, policy_version 842498 (0.0006) [2023-12-26 21:29:44,750][105620] Updated weights for policy 1, policy_version 842508 (0.0007) [2023-12-26 21:29:44,821][105620] Updated weights for policy 1, policy_version 842518 (0.0009) [2023-12-26 21:29:45,138][105692] Updated weights for policy 0, policy_version 842476 (0.0008) [2023-12-26 21:29:45,204][105692] Updated weights for policy 0, policy_version 842486 (0.0008) [2023-12-26 21:29:45,271][105692] Updated weights for policy 0, policy_version 842496 (0.0008) [2023-12-26 21:29:45,509][105620] Updated weights for policy 1, policy_version 842528 (0.0006) [2023-12-26 21:29:45,588][105620] Updated weights for policy 1, policy_version 842538 (0.0005) [2023-12-26 21:29:45,645][105620] Updated weights for policy 1, policy_version 842548 (0.0005) [2023-12-26 21:29:46,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19387.6, 300 sec: 19466.4). Total num frames: 431431680. Throughput: 0: 9547.8, 1: 9758.0. Samples: 431405316. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-26 21:29:46,064][104569] Avg episode reward: [(0, '8631.600'), (1, '9076.488')] [2023-12-26 21:29:46,065][105692] Updated weights for policy 0, policy_version 842506 (0.0008) [2023-12-26 21:29:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000842552_215719936.pth... [2023-12-26 21:29:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000841432_215433216.pth [2023-12-26 21:29:46,110][105692] Updated weights for policy 0, policy_version 842516 (0.0008) [2023-12-26 21:29:46,164][105692] Updated weights for policy 0, policy_version 842526 (0.0008) [2023-12-26 21:29:46,182][105620] Updated weights for policy 1, policy_version 842558 (0.0006) [2023-12-26 21:29:46,218][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000842536_215719936.pth... [2023-12-26 21:29:46,218][105692] Updated weights for policy 0, policy_version 842536 (0.0008) [2023-12-26 21:29:46,223][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000841416_215433216.pth [2023-12-26 21:29:46,247][105620] Updated weights for policy 1, policy_version 842568 (0.0006) [2023-12-26 21:29:46,311][105620] Updated weights for policy 1, policy_version 842578 (0.0005) [2023-12-26 21:29:46,857][105620] Updated weights for policy 1, policy_version 842588 (0.0005) [2023-12-26 21:29:46,917][105620] Updated weights for policy 1, policy_version 842598 (0.0005) [2023-12-26 21:29:46,971][105692] Updated weights for policy 0, policy_version 842546 (0.0009) [2023-12-26 21:29:46,972][105620] Updated weights for policy 1, policy_version 842608 (0.0005) [2023-12-26 21:29:47,031][105692] Updated weights for policy 0, policy_version 842556 (0.0008) [2023-12-26 21:29:47,089][105692] Updated weights for policy 0, policy_version 842566 (0.0010) [2023-12-26 21:29:47,595][105620] Updated weights for policy 1, policy_version 842618 (0.0006) [2023-12-26 21:29:47,658][105620] Updated weights for policy 1, policy_version 842628 (0.0011) [2023-12-26 21:29:47,717][105620] Updated weights for policy 1, policy_version 842638 (0.0010) [2023-12-26 21:29:47,761][105692] Updated weights for policy 0, policy_version 842576 (0.0010) [2023-12-26 21:29:47,762][105620] Updated weights for policy 1, policy_version 842648 (0.0010) [2023-12-26 21:29:47,818][105692] Updated weights for policy 0, policy_version 842586 (0.0007) [2023-12-26 21:29:47,884][105692] Updated weights for policy 0, policy_version 842596 (0.0005) [2023-12-26 21:29:48,483][105620] Updated weights for policy 1, policy_version 842658 (0.0007) [2023-12-26 21:29:48,514][105692] Updated weights for policy 0, policy_version 842606 (0.0006) [2023-12-26 21:29:48,544][105620] Updated weights for policy 1, policy_version 842668 (0.0008) [2023-12-26 21:29:48,572][105692] Updated weights for policy 0, policy_version 842616 (0.0005) [2023-12-26 21:29:48,600][105620] Updated weights for policy 1, policy_version 842678 (0.0007) [2023-12-26 21:29:48,628][105692] Updated weights for policy 0, policy_version 842626 (0.0010) [2023-12-26 21:29:49,242][105620] Updated weights for policy 1, policy_version 842688 (0.0008) [2023-12-26 21:29:49,280][105692] Updated weights for policy 0, policy_version 842636 (0.0010) [2023-12-26 21:29:49,289][105620] Updated weights for policy 1, policy_version 842698 (0.0009) [2023-12-26 21:29:49,343][105692] Updated weights for policy 0, policy_version 842646 (0.0010) [2023-12-26 21:29:49,353][105620] Updated weights for policy 1, policy_version 842708 (0.0008) [2023-12-26 21:29:49,411][105692] Updated weights for policy 0, policy_version 842656 (0.0010) [2023-12-26 21:29:50,111][105620] Updated weights for policy 1, policy_version 842718 (0.0009) [2023-12-26 21:29:50,161][105692] Updated weights for policy 0, policy_version 842666 (0.0010) [2023-12-26 21:29:50,166][105620] Updated weights for policy 1, policy_version 842728 (0.0008) [2023-12-26 21:29:50,214][105692] Updated weights for policy 0, policy_version 842676 (0.0011) [2023-12-26 21:29:50,224][105620] Updated weights for policy 1, policy_version 842738 (0.0006) [2023-12-26 21:29:50,273][105692] Updated weights for policy 0, policy_version 842686 (0.0010) [2023-12-26 21:29:50,331][105692] Updated weights for policy 0, policy_version 842696 (0.0010) [2023-12-26 21:29:50,927][105620] Updated weights for policy 1, policy_version 842748 (0.0007) [2023-12-26 21:29:50,981][105620] Updated weights for policy 1, policy_version 842758 (0.0008) [2023-12-26 21:29:51,040][105620] Updated weights for policy 1, policy_version 842768 (0.0009) [2023-12-26 21:29:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19466.5). Total num frames: 431529984. Throughput: 0: 9493.6, 1: 9774.8. Samples: 431526304. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-26 21:29:51,062][104569] Avg episode reward: [(0, '8635.211'), (1, '8903.707')] [2023-12-26 21:29:51,121][105692] Updated weights for policy 0, policy_version 842706 (0.0009) [2023-12-26 21:29:51,191][105692] Updated weights for policy 0, policy_version 842716 (0.0009) [2023-12-26 21:29:51,261][105692] Updated weights for policy 0, policy_version 842726 (0.0009) [2023-12-26 21:29:51,813][105620] Updated weights for policy 1, policy_version 842778 (0.0007) [2023-12-26 21:29:51,871][105620] Updated weights for policy 1, policy_version 842788 (0.0009) [2023-12-26 21:29:51,929][105620] Updated weights for policy 1, policy_version 842798 (0.0009) [2023-12-26 21:29:51,987][105620] Updated weights for policy 1, policy_version 842808 (0.0009) [2023-12-26 21:29:52,059][105692] Updated weights for policy 0, policy_version 842736 (0.0010) [2023-12-26 21:29:52,114][105692] Updated weights for policy 0, policy_version 842746 (0.0009) [2023-12-26 21:29:52,182][105692] Updated weights for policy 0, policy_version 842756 (0.0009) [2023-12-26 21:29:52,680][105620] Updated weights for policy 1, policy_version 842818 (0.0009) [2023-12-26 21:29:52,729][105620] Updated weights for policy 1, policy_version 842828 (0.0007) [2023-12-26 21:29:52,780][105620] Updated weights for policy 1, policy_version 842838 (0.0008) [2023-12-26 21:29:52,947][105692] Updated weights for policy 0, policy_version 842766 (0.0010) [2023-12-26 21:29:53,009][105692] Updated weights for policy 0, policy_version 842776 (0.0009) [2023-12-26 21:29:53,072][105692] Updated weights for policy 0, policy_version 842786 (0.0009) [2023-12-26 21:29:53,467][105620] Updated weights for policy 1, policy_version 842848 (0.0008) [2023-12-26 21:29:53,535][105620] Updated weights for policy 1, policy_version 842858 (0.0010) [2023-12-26 21:29:53,601][105620] Updated weights for policy 1, policy_version 842868 (0.0010) [2023-12-26 21:29:53,720][105692] Updated weights for policy 0, policy_version 842796 (0.0008) [2023-12-26 21:29:53,776][105692] Updated weights for policy 0, policy_version 842806 (0.0007) [2023-12-26 21:29:53,825][105692] Updated weights for policy 0, policy_version 842816 (0.0008) [2023-12-26 21:29:54,423][105620] Updated weights for policy 1, policy_version 842878 (0.0009) [2023-12-26 21:29:54,470][105620] Updated weights for policy 1, policy_version 842888 (0.0009) [2023-12-26 21:29:54,499][105692] Updated weights for policy 0, policy_version 842826 (0.0008) [2023-12-26 21:29:54,526][105620] Updated weights for policy 1, policy_version 842898 (0.0007) [2023-12-26 21:29:54,548][105692] Updated weights for policy 0, policy_version 842836 (0.0005) [2023-12-26 21:29:54,599][105692] Updated weights for policy 0, policy_version 842846 (0.0006) [2023-12-26 21:29:54,646][105692] Updated weights for policy 0, policy_version 842856 (0.0005) [2023-12-26 21:29:55,314][105620] Updated weights for policy 1, policy_version 842908 (0.0008) [2023-12-26 21:29:55,320][105692] Updated weights for policy 0, policy_version 842866 (0.0006) [2023-12-26 21:29:55,365][105620] Updated weights for policy 1, policy_version 842918 (0.0006) [2023-12-26 21:29:55,369][105692] Updated weights for policy 0, policy_version 842876 (0.0006) [2023-12-26 21:29:55,414][105620] Updated weights for policy 1, policy_version 842928 (0.0009) [2023-12-26 21:29:55,427][105692] Updated weights for policy 0, policy_version 842886 (0.0006) [2023-12-26 21:29:56,062][104569] Fps is (10 sec: 19661.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 431628288. Throughput: 0: 9523.8, 1: 9774.0. Samples: 431640640. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-26 21:29:56,062][104569] Avg episode reward: [(0, '8904.366'), (1, '9079.511')] [2023-12-26 21:29:56,065][105692] Updated weights for policy 0, policy_version 842896 (0.0006) [2023-12-26 21:29:56,126][105692] Updated weights for policy 0, policy_version 842906 (0.0008) [2023-12-26 21:29:56,186][105692] Updated weights for policy 0, policy_version 842916 (0.0007) [2023-12-26 21:29:56,235][105620] Updated weights for policy 1, policy_version 842938 (0.0009) [2023-12-26 21:29:56,299][105620] Updated weights for policy 1, policy_version 842948 (0.0008) [2023-12-26 21:29:56,356][105620] Updated weights for policy 1, policy_version 842958 (0.0009) [2023-12-26 21:29:56,408][105620] Updated weights for policy 1, policy_version 842968 (0.0009) [2023-12-26 21:29:56,849][105692] Updated weights for policy 0, policy_version 842926 (0.0008) [2023-12-26 21:29:56,899][105692] Updated weights for policy 0, policy_version 842936 (0.0009) [2023-12-26 21:29:56,952][105692] Updated weights for policy 0, policy_version 842946 (0.0009) [2023-12-26 21:29:57,131][105620] Updated weights for policy 1, policy_version 842978 (0.0008) [2023-12-26 21:29:57,179][105620] Updated weights for policy 1, policy_version 842988 (0.0009) [2023-12-26 21:29:57,225][105620] Updated weights for policy 1, policy_version 842998 (0.0009) [2023-12-26 21:29:57,719][105692] Updated weights for policy 0, policy_version 842956 (0.0008) [2023-12-26 21:29:57,766][105692] Updated weights for policy 0, policy_version 842966 (0.0009) [2023-12-26 21:29:57,811][105692] Updated weights for policy 0, policy_version 842976 (0.0008) [2023-12-26 21:29:57,935][105620] Updated weights for policy 1, policy_version 843008 (0.0009) [2023-12-26 21:29:57,982][105620] Updated weights for policy 1, policy_version 843018 (0.0008) [2023-12-26 21:29:58,053][105620] Updated weights for policy 1, policy_version 843028 (0.0009) [2023-12-26 21:29:58,598][105692] Updated weights for policy 0, policy_version 842986 (0.0009) [2023-12-26 21:29:58,661][105692] Updated weights for policy 0, policy_version 842996 (0.0009) [2023-12-26 21:29:58,728][105692] Updated weights for policy 0, policy_version 843006 (0.0009) [2023-12-26 21:29:58,796][105692] Updated weights for policy 0, policy_version 843016 (0.0008) [2023-12-26 21:29:58,884][105620] Updated weights for policy 1, policy_version 843038 (0.0010) [2023-12-26 21:29:58,953][105620] Updated weights for policy 1, policy_version 843048 (0.0010) [2023-12-26 21:29:59,017][105620] Updated weights for policy 1, policy_version 843058 (0.0009) [2023-12-26 21:29:59,556][105692] Updated weights for policy 0, policy_version 843026 (0.0009) [2023-12-26 21:29:59,619][105692] Updated weights for policy 0, policy_version 843036 (0.0009) [2023-12-26 21:29:59,679][105692] Updated weights for policy 0, policy_version 843046 (0.0008) [2023-12-26 21:29:59,799][105620] Updated weights for policy 1, policy_version 843068 (0.0009) [2023-12-26 21:29:59,859][105620] Updated weights for policy 1, policy_version 843078 (0.0009) [2023-12-26 21:29:59,928][105620] Updated weights for policy 1, policy_version 843088 (0.0008) [2023-12-26 21:30:00,477][105692] Updated weights for policy 0, policy_version 843056 (0.0008) [2023-12-26 21:30:00,532][105692] Updated weights for policy 0, policy_version 843066 (0.0007) [2023-12-26 21:30:00,547][105620] Updated weights for policy 1, policy_version 843098 (0.0006) [2023-12-26 21:30:00,581][105692] Updated weights for policy 0, policy_version 843076 (0.0006) [2023-12-26 21:30:00,603][105620] Updated weights for policy 1, policy_version 843108 (0.0008) [2023-12-26 21:30:00,657][105620] Updated weights for policy 1, policy_version 843118 (0.0009) [2023-12-26 21:30:00,706][105620] Updated weights for policy 1, policy_version 843128 (0.0009) [2023-12-26 21:30:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 431726592. Throughput: 0: 9568.6, 1: 9735.7. Samples: 431697532. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-26 21:30:01,062][104569] Avg episode reward: [(0, '5221.407'), (1, '9112.358')] [2023-12-26 21:30:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000843080_215859200.pth... [2023-12-26 21:30:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000843128_215867392.pth... [2023-12-26 21:30:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000842008_215580672.pth [2023-12-26 21:30:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000841992_215580672.pth [2023-12-26 21:30:01,312][105692] Updated weights for policy 0, policy_version 843086 (0.0005) [2023-12-26 21:30:01,369][105692] Updated weights for policy 0, policy_version 843096 (0.0008) [2023-12-26 21:30:01,433][105692] Updated weights for policy 0, policy_version 843106 (0.0007) [2023-12-26 21:30:01,480][105620] Updated weights for policy 1, policy_version 843138 (0.0008) [2023-12-26 21:30:01,544][105620] Updated weights for policy 1, policy_version 843148 (0.0005) [2023-12-26 21:30:01,611][105620] Updated weights for policy 1, policy_version 843158 (0.0007) [2023-12-26 21:30:02,205][105692] Updated weights for policy 0, policy_version 843116 (0.0009) [2023-12-26 21:30:02,271][105692] Updated weights for policy 0, policy_version 843126 (0.0008) [2023-12-26 21:30:02,325][105692] Updated weights for policy 0, policy_version 843136 (0.0005) [2023-12-26 21:30:02,339][105620] Updated weights for policy 1, policy_version 843168 (0.0007) [2023-12-26 21:30:02,410][105620] Updated weights for policy 1, policy_version 843178 (0.0008) [2023-12-26 21:30:02,470][105620] Updated weights for policy 1, policy_version 843188 (0.0007) [2023-12-26 21:30:03,087][105620] Updated weights for policy 1, policy_version 843198 (0.0005) [2023-12-26 21:30:03,105][105692] Updated weights for policy 0, policy_version 843146 (0.0008) [2023-12-26 21:30:03,144][105620] Updated weights for policy 1, policy_version 843208 (0.0006) [2023-12-26 21:30:03,155][105692] Updated weights for policy 0, policy_version 843156 (0.0008) [2023-12-26 21:30:03,200][105620] Updated weights for policy 1, policy_version 843218 (0.0006) [2023-12-26 21:30:03,211][105692] Updated weights for policy 0, policy_version 843166 (0.0009) [2023-12-26 21:30:03,265][105692] Updated weights for policy 0, policy_version 843176 (0.0009) [2023-12-26 21:30:03,705][105620] Updated weights for policy 1, policy_version 843228 (0.0005) [2023-12-26 21:30:03,756][105620] Updated weights for policy 1, policy_version 843238 (0.0005) [2023-12-26 21:30:03,808][105620] Updated weights for policy 1, policy_version 843248 (0.0005) [2023-12-26 21:30:04,186][105692] Updated weights for policy 0, policy_version 843186 (0.0008) [2023-12-26 21:30:04,248][105692] Updated weights for policy 0, policy_version 843196 (0.0008) [2023-12-26 21:30:04,315][105692] Updated weights for policy 0, policy_version 843206 (0.0008) [2023-12-26 21:30:04,437][105620] Updated weights for policy 1, policy_version 843258 (0.0007) [2023-12-26 21:30:04,498][105620] Updated weights for policy 1, policy_version 843268 (0.0010) [2023-12-26 21:30:04,553][105620] Updated weights for policy 1, policy_version 843278 (0.0007) [2023-12-26 21:30:04,607][105620] Updated weights for policy 1, policy_version 843288 (0.0005) [2023-12-26 21:30:05,124][105692] Updated weights for policy 0, policy_version 843216 (0.0010) [2023-12-26 21:30:05,186][105586] KL-divergence is very high: 110.7287 [2023-12-26 21:30:05,186][105692] Updated weights for policy 0, policy_version 843226 (0.0009) [2023-12-26 21:30:05,189][105620] Updated weights for policy 1, policy_version 843298 (0.0007) [2023-12-26 21:30:05,222][105586] KL-divergence is very high: 182.8960 [2023-12-26 21:30:05,239][105620] Updated weights for policy 1, policy_version 843308 (0.0008) [2023-12-26 21:30:05,250][105692] Updated weights for policy 0, policy_version 843236 (0.0006) [2023-12-26 21:30:05,259][105586] KL-divergence is very high: 159.3060 [2023-12-26 21:30:05,292][105620] Updated weights for policy 1, policy_version 843319 (0.0009) [2023-12-26 21:30:05,958][105620] Updated weights for policy 1, policy_version 843329 (0.0006) [2023-12-26 21:30:06,012][105620] Updated weights for policy 1, policy_version 843339 (0.0005) [2023-12-26 21:30:06,017][105692] Updated weights for policy 0, policy_version 843246 (0.0007) [2023-12-26 21:30:06,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19114.7, 300 sec: 19438.6). Total num frames: 431816704. Throughput: 0: 9416.1, 1: 9873.6. Samples: 431813664. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-26 21:30:06,063][104569] Avg episode reward: [(0, '4286.695'), (1, '8408.051')] [2023-12-26 21:30:06,064][105620] Updated weights for policy 1, policy_version 843349 (0.0005) [2023-12-26 21:30:06,066][105692] Updated weights for policy 0, policy_version 843256 (0.0008) [2023-12-26 21:30:06,121][105692] Updated weights for policy 0, policy_version 843266 (0.0010) [2023-12-26 21:30:06,691][105620] Updated weights for policy 1, policy_version 843359 (0.0005) [2023-12-26 21:30:06,740][105620] Updated weights for policy 1, policy_version 843369 (0.0005) [2023-12-26 21:30:06,802][105620] Updated weights for policy 1, policy_version 843379 (0.0007) [2023-12-26 21:30:06,981][105692] Updated weights for policy 0, policy_version 843276 (0.0007) [2023-12-26 21:30:07,048][105692] Updated weights for policy 0, policy_version 843286 (0.0007) [2023-12-26 21:30:07,109][105692] Updated weights for policy 0, policy_version 843296 (0.0011) [2023-12-26 21:30:07,536][105620] Updated weights for policy 1, policy_version 843389 (0.0007) [2023-12-26 21:30:07,585][105620] Updated weights for policy 1, policy_version 843399 (0.0008) [2023-12-26 21:30:07,637][105620] Updated weights for policy 1, policy_version 843409 (0.0008) [2023-12-26 21:30:07,738][105692] Updated weights for policy 0, policy_version 843306 (0.0006) [2023-12-26 21:30:07,790][105692] Updated weights for policy 0, policy_version 843316 (0.0005) [2023-12-26 21:30:07,838][105692] Updated weights for policy 0, policy_version 843326 (0.0005) [2023-12-26 21:30:07,891][105692] Updated weights for policy 0, policy_version 843336 (0.0005) [2023-12-26 21:30:08,462][105620] Updated weights for policy 1, policy_version 843419 (0.0007) [2023-12-26 21:30:08,486][105692] Updated weights for policy 0, policy_version 843346 (0.0008) [2023-12-26 21:30:08,516][105620] Updated weights for policy 1, policy_version 843429 (0.0008) [2023-12-26 21:30:08,545][105692] Updated weights for policy 0, policy_version 843356 (0.0008) [2023-12-26 21:30:08,576][105620] Updated weights for policy 1, policy_version 843439 (0.0007) [2023-12-26 21:30:08,607][105692] Updated weights for policy 0, policy_version 843366 (0.0007) [2023-12-26 21:30:09,219][105692] Updated weights for policy 0, policy_version 843376 (0.0009) [2023-12-26 21:30:09,279][105692] Updated weights for policy 0, policy_version 843386 (0.0009) [2023-12-26 21:30:09,341][105692] Updated weights for policy 0, policy_version 843396 (0.0010) [2023-12-26 21:30:09,407][105620] Updated weights for policy 1, policy_version 843449 (0.0007) [2023-12-26 21:30:09,462][105620] Updated weights for policy 1, policy_version 843459 (0.0008) [2023-12-26 21:30:09,514][105620] Updated weights for policy 1, policy_version 843469 (0.0009) [2023-12-26 21:30:09,569][105620] Updated weights for policy 1, policy_version 843479 (0.0010) [2023-12-26 21:30:10,141][105692] Updated weights for policy 0, policy_version 843406 (0.0009) [2023-12-26 21:30:10,196][105692] Updated weights for policy 0, policy_version 843416 (0.0009) [2023-12-26 21:30:10,253][105692] Updated weights for policy 0, policy_version 843426 (0.0008) [2023-12-26 21:30:10,334][105620] Updated weights for policy 1, policy_version 843489 (0.0008) [2023-12-26 21:30:10,398][105620] Updated weights for policy 1, policy_version 843499 (0.0007) [2023-12-26 21:30:10,456][105620] Updated weights for policy 1, policy_version 843509 (0.0006) [2023-12-26 21:30:10,961][105692] Updated weights for policy 0, policy_version 843436 (0.0009) [2023-12-26 21:30:11,031][105692] Updated weights for policy 0, policy_version 843446 (0.0009) [2023-12-26 21:30:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 431915008. Throughput: 0: 9368.3, 1: 9860.0. Samples: 431929304. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-26 21:30:11,062][104569] Avg episode reward: [(0, '7197.108'), (1, '7432.815')] [2023-12-26 21:30:11,096][105692] Updated weights for policy 0, policy_version 843456 (0.0010) [2023-12-26 21:30:11,155][105620] Updated weights for policy 1, policy_version 843519 (0.0007) [2023-12-26 21:30:11,212][105620] Updated weights for policy 1, policy_version 843529 (0.0006) [2023-12-26 21:30:11,278][105620] Updated weights for policy 1, policy_version 843539 (0.0006) [2023-12-26 21:30:11,895][105692] Updated weights for policy 0, policy_version 843466 (0.0009) [2023-12-26 21:30:11,953][105692] Updated weights for policy 0, policy_version 843476 (0.0007) [2023-12-26 21:30:12,015][105692] Updated weights for policy 0, policy_version 843486 (0.0008) [2023-12-26 21:30:12,050][105620] Updated weights for policy 1, policy_version 843549 (0.0008) [2023-12-26 21:30:12,073][105692] Updated weights for policy 0, policy_version 843496 (0.0006) [2023-12-26 21:30:12,114][105620] Updated weights for policy 1, policy_version 843559 (0.0008) [2023-12-26 21:30:12,161][105620] Updated weights for policy 1, policy_version 843569 (0.0008) [2023-12-26 21:30:12,806][105692] Updated weights for policy 0, policy_version 843506 (0.0008) [2023-12-26 21:30:12,860][105692] Updated weights for policy 0, policy_version 843516 (0.0009) [2023-12-26 21:30:12,906][105692] Updated weights for policy 0, policy_version 843526 (0.0008) [2023-12-26 21:30:12,936][105620] Updated weights for policy 1, policy_version 843579 (0.0009) [2023-12-26 21:30:12,990][105620] Updated weights for policy 1, policy_version 843589 (0.0008) [2023-12-26 21:30:13,055][105620] Updated weights for policy 1, policy_version 843599 (0.0009) [2023-12-26 21:30:13,665][105692] Updated weights for policy 0, policy_version 843536 (0.0009) [2023-12-26 21:30:13,713][105692] Updated weights for policy 0, policy_version 843546 (0.0009) [2023-12-26 21:30:13,759][105692] Updated weights for policy 0, policy_version 843556 (0.0009) [2023-12-26 21:30:13,814][105620] Updated weights for policy 1, policy_version 843609 (0.0008) [2023-12-26 21:30:13,872][105620] Updated weights for policy 1, policy_version 843619 (0.0009) [2023-12-26 21:30:13,920][105620] Updated weights for policy 1, policy_version 843629 (0.0008) [2023-12-26 21:30:13,967][105620] Updated weights for policy 1, policy_version 843639 (0.0009) [2023-12-26 21:30:14,534][105692] Updated weights for policy 0, policy_version 843566 (0.0007) [2023-12-26 21:30:14,592][105692] Updated weights for policy 0, policy_version 843576 (0.0005) [2023-12-26 21:30:14,645][105692] Updated weights for policy 0, policy_version 843586 (0.0005) [2023-12-26 21:30:14,660][105620] Updated weights for policy 1, policy_version 843649 (0.0009) [2023-12-26 21:30:14,715][105620] Updated weights for policy 1, policy_version 843659 (0.0009) [2023-12-26 21:30:14,767][105620] Updated weights for policy 1, policy_version 843669 (0.0009) [2023-12-26 21:30:15,254][105692] Updated weights for policy 0, policy_version 843596 (0.0007) [2023-12-26 21:30:15,321][105692] Updated weights for policy 0, policy_version 843606 (0.0009) [2023-12-26 21:30:15,384][105692] Updated weights for policy 0, policy_version 843616 (0.0009) [2023-12-26 21:30:15,600][105620] Updated weights for policy 1, policy_version 843679 (0.0007) [2023-12-26 21:30:15,652][105620] Updated weights for policy 1, policy_version 843689 (0.0006) [2023-12-26 21:30:15,720][105620] Updated weights for policy 1, policy_version 843699 (0.0008) [2023-12-26 21:30:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 432013312. Throughput: 0: 9395.0, 1: 9764.9. Samples: 431985024. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-26 21:30:16,062][104569] Avg episode reward: [(0, '8414.707'), (1, '8157.936')] [2023-12-26 21:30:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000843624_215998464.pth... [2023-12-26 21:30:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000843704_216014848.pth... [2023-12-26 21:30:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000842536_215719936.pth [2023-12-26 21:30:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000842552_215719936.pth [2023-12-26 21:30:16,121][105692] Updated weights for policy 0, policy_version 843626 (0.0009) [2023-12-26 21:30:16,176][105692] Updated weights for policy 0, policy_version 843636 (0.0009) [2023-12-26 21:30:16,239][105692] Updated weights for policy 0, policy_version 843646 (0.0009) [2023-12-26 21:30:16,302][105692] Updated weights for policy 0, policy_version 843656 (0.0009) [2023-12-26 21:30:16,438][105620] Updated weights for policy 1, policy_version 843709 (0.0009) [2023-12-26 21:30:16,495][105620] Updated weights for policy 1, policy_version 843719 (0.0009) [2023-12-26 21:30:16,558][105620] Updated weights for policy 1, policy_version 843729 (0.0009) [2023-12-26 21:30:17,052][105692] Updated weights for policy 0, policy_version 843666 (0.0009) [2023-12-26 21:30:17,107][105692] Updated weights for policy 0, policy_version 843676 (0.0009) [2023-12-26 21:30:17,169][105692] Updated weights for policy 0, policy_version 843686 (0.0009) [2023-12-26 21:30:17,304][105620] Updated weights for policy 1, policy_version 843739 (0.0009) [2023-12-26 21:30:17,355][105620] Updated weights for policy 1, policy_version 843749 (0.0009) [2023-12-26 21:30:17,404][105620] Updated weights for policy 1, policy_version 843759 (0.0008) [2023-12-26 21:30:17,926][105692] Updated weights for policy 0, policy_version 843696 (0.0008) [2023-12-26 21:30:17,985][105692] Updated weights for policy 0, policy_version 843706 (0.0009) [2023-12-26 21:30:18,052][105692] Updated weights for policy 0, policy_version 843716 (0.0009) [2023-12-26 21:30:18,126][105620] Updated weights for policy 1, policy_version 843769 (0.0009) [2023-12-26 21:30:18,172][105620] Updated weights for policy 1, policy_version 843779 (0.0009) [2023-12-26 21:30:18,233][105620] Updated weights for policy 1, policy_version 843789 (0.0009) [2023-12-26 21:30:18,293][105620] Updated weights for policy 1, policy_version 843799 (0.0009) [2023-12-26 21:30:18,800][105692] Updated weights for policy 0, policy_version 843726 (0.0009) [2023-12-26 21:30:18,864][105692] Updated weights for policy 0, policy_version 843736 (0.0009) [2023-12-26 21:30:18,930][105692] Updated weights for policy 0, policy_version 843746 (0.0009) [2023-12-26 21:30:19,064][105620] Updated weights for policy 1, policy_version 843809 (0.0009) [2023-12-26 21:30:19,128][105620] Updated weights for policy 1, policy_version 843819 (0.0009) [2023-12-26 21:30:19,191][105620] Updated weights for policy 1, policy_version 843829 (0.0007) [2023-12-26 21:30:19,672][105692] Updated weights for policy 0, policy_version 843756 (0.0009) [2023-12-26 21:30:19,720][105692] Updated weights for policy 0, policy_version 843766 (0.0009) [2023-12-26 21:30:19,772][105692] Updated weights for policy 0, policy_version 843776 (0.0009) [2023-12-26 21:30:19,977][105620] Updated weights for policy 1, policy_version 843839 (0.0008) [2023-12-26 21:30:20,043][105620] Updated weights for policy 1, policy_version 843849 (0.0006) [2023-12-26 21:30:20,106][105620] Updated weights for policy 1, policy_version 843859 (0.0006) [2023-12-26 21:30:20,558][105692] Updated weights for policy 0, policy_version 843786 (0.0009) [2023-12-26 21:30:20,618][105692] Updated weights for policy 0, policy_version 843796 (0.0011) [2023-12-26 21:30:20,680][105692] Updated weights for policy 0, policy_version 843806 (0.0012) [2023-12-26 21:30:20,719][105620] Updated weights for policy 1, policy_version 843869 (0.0008) [2023-12-26 21:30:20,742][105692] Updated weights for policy 0, policy_version 843816 (0.0010) [2023-12-26 21:30:20,779][105620] Updated weights for policy 1, policy_version 843879 (0.0008) [2023-12-26 21:30:20,842][105620] Updated weights for policy 1, policy_version 843889 (0.0011) [2023-12-26 21:30:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 432111616. Throughput: 0: 9422.3, 1: 9734.1. Samples: 432098724. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-26 21:30:21,063][104569] Avg episode reward: [(0, '8770.018'), (1, '8588.863')] [2023-12-26 21:30:21,494][105692] Updated weights for policy 0, policy_version 843826 (0.0006) [2023-12-26 21:30:21,532][105620] Updated weights for policy 1, policy_version 843899 (0.0009) [2023-12-26 21:30:21,558][105692] Updated weights for policy 0, policy_version 843836 (0.0010) [2023-12-26 21:30:21,592][105620] Updated weights for policy 1, policy_version 843909 (0.0011) [2023-12-26 21:30:21,615][105692] Updated weights for policy 0, policy_version 843846 (0.0010) [2023-12-26 21:30:21,661][105620] Updated weights for policy 1, policy_version 843919 (0.0011) [2023-12-26 21:30:22,338][105692] Updated weights for policy 0, policy_version 843856 (0.0008) [2023-12-26 21:30:22,389][105692] Updated weights for policy 0, policy_version 843866 (0.0008) [2023-12-26 21:30:22,442][105620] Updated weights for policy 1, policy_version 843929 (0.0010) [2023-12-26 21:30:22,452][105692] Updated weights for policy 0, policy_version 843876 (0.0008) [2023-12-26 21:30:22,505][105620] Updated weights for policy 1, policy_version 843939 (0.0010) [2023-12-26 21:30:22,567][105620] Updated weights for policy 1, policy_version 843949 (0.0010) [2023-12-26 21:30:22,619][105620] Updated weights for policy 1, policy_version 843959 (0.0010) [2023-12-26 21:30:23,131][105692] Updated weights for policy 0, policy_version 843886 (0.0006) [2023-12-26 21:30:23,181][105692] Updated weights for policy 0, policy_version 843896 (0.0006) [2023-12-26 21:30:23,246][105692] Updated weights for policy 0, policy_version 843906 (0.0006) [2023-12-26 21:30:23,363][105620] Updated weights for policy 1, policy_version 843969 (0.0010) [2023-12-26 21:30:23,422][105620] Updated weights for policy 1, policy_version 843979 (0.0007) [2023-12-26 21:30:23,494][105620] Updated weights for policy 1, policy_version 843989 (0.0006) [2023-12-26 21:30:23,950][105692] Updated weights for policy 0, policy_version 843916 (0.0007) [2023-12-26 21:30:24,012][105692] Updated weights for policy 0, policy_version 843926 (0.0009) [2023-12-26 21:30:24,071][105692] Updated weights for policy 0, policy_version 843936 (0.0007) [2023-12-26 21:30:24,083][105620] Updated weights for policy 1, policy_version 843999 (0.0008) [2023-12-26 21:30:24,141][105620] Updated weights for policy 1, policy_version 844009 (0.0008) [2023-12-26 21:30:24,199][105620] Updated weights for policy 1, policy_version 844019 (0.0009) [2023-12-26 21:30:24,772][105692] Updated weights for policy 0, policy_version 843946 (0.0006) [2023-12-26 21:30:24,829][105692] Updated weights for policy 0, policy_version 843956 (0.0010) [2023-12-26 21:30:24,880][105692] Updated weights for policy 0, policy_version 843966 (0.0010) [2023-12-26 21:30:24,918][105620] Updated weights for policy 1, policy_version 844029 (0.0008) [2023-12-26 21:30:24,931][105692] Updated weights for policy 0, policy_version 843976 (0.0010) [2023-12-26 21:30:24,973][105620] Updated weights for policy 1, policy_version 844039 (0.0006) [2023-12-26 21:30:25,032][105620] Updated weights for policy 1, policy_version 844049 (0.0006) [2023-12-26 21:30:25,571][105620] Updated weights for policy 1, policy_version 844059 (0.0006) [2023-12-26 21:30:25,625][105620] Updated weights for policy 1, policy_version 844069 (0.0010) [2023-12-26 21:30:25,659][105692] Updated weights for policy 0, policy_version 843986 (0.0010) [2023-12-26 21:30:25,680][105620] Updated weights for policy 1, policy_version 844079 (0.0010) [2023-12-26 21:30:25,707][105692] Updated weights for policy 0, policy_version 843996 (0.0010) [2023-12-26 21:30:25,761][105692] Updated weights for policy 0, policy_version 844006 (0.0010) [2023-12-26 21:30:26,062][104569] Fps is (10 sec: 19659.9, 60 sec: 19251.0, 300 sec: 19466.4). Total num frames: 432209920. Throughput: 0: 9449.2, 1: 9823.9. Samples: 432217036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:30:26,063][104569] Avg episode reward: [(0, '8912.201'), (1, '8497.776')] [2023-12-26 21:30:26,420][105620] Updated weights for policy 1, policy_version 844089 (0.0010) [2023-12-26 21:30:26,479][105620] Updated weights for policy 1, policy_version 844099 (0.0011) [2023-12-26 21:30:26,497][105692] Updated weights for policy 0, policy_version 844016 (0.0010) [2023-12-26 21:30:26,542][105620] Updated weights for policy 1, policy_version 844109 (0.0010) [2023-12-26 21:30:26,558][105692] Updated weights for policy 0, policy_version 844026 (0.0009) [2023-12-26 21:30:26,600][105620] Updated weights for policy 1, policy_version 844119 (0.0010) [2023-12-26 21:30:26,616][105692] Updated weights for policy 0, policy_version 844036 (0.0010) [2023-12-26 21:30:27,275][105692] Updated weights for policy 0, policy_version 844046 (0.0007) [2023-12-26 21:30:27,338][105692] Updated weights for policy 0, policy_version 844056 (0.0008) [2023-12-26 21:30:27,347][105620] Updated weights for policy 1, policy_version 844129 (0.0010) [2023-12-26 21:30:27,396][105692] Updated weights for policy 0, policy_version 844066 (0.0010) [2023-12-26 21:30:27,403][105620] Updated weights for policy 1, policy_version 844139 (0.0010) [2023-12-26 21:30:27,463][105620] Updated weights for policy 1, policy_version 844149 (0.0010) [2023-12-26 21:30:28,101][105692] Updated weights for policy 0, policy_version 844076 (0.0010) [2023-12-26 21:30:28,129][105620] Updated weights for policy 1, policy_version 844159 (0.0010) [2023-12-26 21:30:28,161][105692] Updated weights for policy 0, policy_version 844086 (0.0009) [2023-12-26 21:30:28,194][105620] Updated weights for policy 1, policy_version 844169 (0.0011) [2023-12-26 21:30:28,225][105692] Updated weights for policy 0, policy_version 844096 (0.0008) [2023-12-26 21:30:28,246][105620] Updated weights for policy 1, policy_version 844179 (0.0011) [2023-12-26 21:30:28,949][105692] Updated weights for policy 0, policy_version 844106 (0.0009) [2023-12-26 21:30:28,983][105620] Updated weights for policy 1, policy_version 844189 (0.0011) [2023-12-26 21:30:29,007][105692] Updated weights for policy 0, policy_version 844116 (0.0010) [2023-12-26 21:30:29,035][105620] Updated weights for policy 1, policy_version 844199 (0.0010) [2023-12-26 21:30:29,069][105692] Updated weights for policy 0, policy_version 844126 (0.0010) [2023-12-26 21:30:29,085][105620] Updated weights for policy 1, policy_version 844209 (0.0010) [2023-12-26 21:30:29,131][105692] Updated weights for policy 0, policy_version 844136 (0.0011) [2023-12-26 21:30:29,729][105620] Updated weights for policy 1, policy_version 844219 (0.0011) [2023-12-26 21:30:29,787][105620] Updated weights for policy 1, policy_version 844229 (0.0010) [2023-12-26 21:30:29,845][105692] Updated weights for policy 0, policy_version 844146 (0.0008) [2023-12-26 21:30:29,851][105620] Updated weights for policy 1, policy_version 844239 (0.0010) [2023-12-26 21:30:29,915][105692] Updated weights for policy 0, policy_version 844156 (0.0008) [2023-12-26 21:30:29,977][105692] Updated weights for policy 0, policy_version 844166 (0.0009) [2023-12-26 21:30:30,512][105620] Updated weights for policy 1, policy_version 844249 (0.0010) [2023-12-26 21:30:30,560][105620] Updated weights for policy 1, policy_version 844259 (0.0010) [2023-12-26 21:30:30,618][105620] Updated weights for policy 1, policy_version 844269 (0.0010) [2023-12-26 21:30:30,672][105620] Updated weights for policy 1, policy_version 844279 (0.0010) [2023-12-26 21:30:30,729][105692] Updated weights for policy 0, policy_version 844176 (0.0010) [2023-12-26 21:30:30,776][105692] Updated weights for policy 0, policy_version 844186 (0.0010) [2023-12-26 21:30:30,833][105692] Updated weights for policy 0, policy_version 844196 (0.0010) [2023-12-26 21:30:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 432308224. Throughput: 0: 9538.5, 1: 9800.8. Samples: 432275576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:30:31,062][104569] Avg episode reward: [(0, '9088.180'), (1, '8990.379')] [2023-12-26 21:30:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000844280_216162304.pth... [2023-12-26 21:30:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000844200_216145920.pth... [2023-12-26 21:30:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000843080_215859200.pth [2023-12-26 21:30:31,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000843128_215867392.pth [2023-12-26 21:30:31,433][105620] Updated weights for policy 1, policy_version 844289 (0.0010) [2023-12-26 21:30:31,498][105620] Updated weights for policy 1, policy_version 844299 (0.0010) [2023-12-26 21:30:31,537][105692] Updated weights for policy 0, policy_version 844206 (0.0007) [2023-12-26 21:30:31,553][105620] Updated weights for policy 1, policy_version 844309 (0.0011) [2023-12-26 21:30:31,598][105692] Updated weights for policy 0, policy_version 844216 (0.0005) [2023-12-26 21:30:31,667][105692] Updated weights for policy 0, policy_version 844226 (0.0008) [2023-12-26 21:30:32,184][105620] Updated weights for policy 1, policy_version 844319 (0.0007) [2023-12-26 21:30:32,243][105620] Updated weights for policy 1, policy_version 844329 (0.0006) [2023-12-26 21:30:32,245][105692] Updated weights for policy 0, policy_version 844236 (0.0008) [2023-12-26 21:30:32,303][105692] Updated weights for policy 0, policy_version 844246 (0.0007) [2023-12-26 21:30:32,310][105620] Updated weights for policy 1, policy_version 844339 (0.0010) [2023-12-26 21:30:32,358][105692] Updated weights for policy 0, policy_version 844256 (0.0006) [2023-12-26 21:30:32,885][105620] Updated weights for policy 1, policy_version 844349 (0.0008) [2023-12-26 21:30:32,931][105620] Updated weights for policy 1, policy_version 844359 (0.0005) [2023-12-26 21:30:32,985][105620] Updated weights for policy 1, policy_version 844369 (0.0005) [2023-12-26 21:30:33,029][105692] Updated weights for policy 0, policy_version 844266 (0.0009) [2023-12-26 21:30:33,081][105692] Updated weights for policy 0, policy_version 844276 (0.0010) [2023-12-26 21:30:33,134][105692] Updated weights for policy 0, policy_version 844286 (0.0009) [2023-12-26 21:30:33,196][105692] Updated weights for policy 0, policy_version 844296 (0.0010) [2023-12-26 21:30:33,504][105620] Updated weights for policy 1, policy_version 844379 (0.0008) [2023-12-26 21:30:33,555][105620] Updated weights for policy 1, policy_version 844389 (0.0005) [2023-12-26 21:30:33,602][105620] Updated weights for policy 1, policy_version 844399 (0.0005) [2023-12-26 21:30:33,937][105692] Updated weights for policy 0, policy_version 844306 (0.0010) [2023-12-26 21:30:33,991][105692] Updated weights for policy 0, policy_version 844316 (0.0010) [2023-12-26 21:30:34,046][105692] Updated weights for policy 0, policy_version 844326 (0.0010) [2023-12-26 21:30:34,145][105620] Updated weights for policy 1, policy_version 844409 (0.0006) [2023-12-26 21:30:34,205][105620] Updated weights for policy 1, policy_version 844419 (0.0009) [2023-12-26 21:30:34,273][105620] Updated weights for policy 1, policy_version 844429 (0.0008) [2023-12-26 21:30:34,339][105620] Updated weights for policy 1, policy_version 844439 (0.0009) [2023-12-26 21:30:34,793][105692] Updated weights for policy 0, policy_version 844336 (0.0010) [2023-12-26 21:30:34,855][105692] Updated weights for policy 0, policy_version 844346 (0.0010) [2023-12-26 21:30:34,906][105692] Updated weights for policy 0, policy_version 844356 (0.0010) [2023-12-26 21:30:35,062][105620] Updated weights for policy 1, policy_version 844449 (0.0005) [2023-12-26 21:30:35,115][105620] Updated weights for policy 1, policy_version 844459 (0.0005) [2023-12-26 21:30:35,169][105620] Updated weights for policy 1, policy_version 844469 (0.0005) [2023-12-26 21:30:35,528][105692] Updated weights for policy 0, policy_version 844366 (0.0007) [2023-12-26 21:30:35,590][105692] Updated weights for policy 0, policy_version 844376 (0.0005) [2023-12-26 21:30:35,662][105692] Updated weights for policy 0, policy_version 844386 (0.0006) [2023-12-26 21:30:35,733][105620] Updated weights for policy 1, policy_version 844479 (0.0005) [2023-12-26 21:30:35,787][105620] Updated weights for policy 1, policy_version 844489 (0.0005) [2023-12-26 21:30:35,835][105620] Updated weights for policy 1, policy_version 844499 (0.0005) [2023-12-26 21:30:36,062][104569] Fps is (10 sec: 20481.3, 60 sec: 19524.4, 300 sec: 19466.4). Total num frames: 432414720. Throughput: 0: 9554.4, 1: 9839.4. Samples: 432399024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:30:36,062][104569] Avg episode reward: [(0, '9086.935'), (1, '9169.686')] [2023-12-26 21:30:36,279][105692] Updated weights for policy 0, policy_version 844396 (0.0011) [2023-12-26 21:30:36,343][105692] Updated weights for policy 0, policy_version 844406 (0.0011) [2023-12-26 21:30:36,400][105692] Updated weights for policy 0, policy_version 844416 (0.0011) [2023-12-26 21:30:36,557][105620] Updated weights for policy 1, policy_version 844509 (0.0007) [2023-12-26 21:30:36,617][105620] Updated weights for policy 1, policy_version 844519 (0.0008) [2023-12-26 21:30:36,681][105620] Updated weights for policy 1, policy_version 844529 (0.0009) [2023-12-26 21:30:37,140][105692] Updated weights for policy 0, policy_version 844426 (0.0010) [2023-12-26 21:30:37,189][105692] Updated weights for policy 0, policy_version 844436 (0.0010) [2023-12-26 21:30:37,234][105692] Updated weights for policy 0, policy_version 844446 (0.0011) [2023-12-26 21:30:37,283][105692] Updated weights for policy 0, policy_version 844456 (0.0010) [2023-12-26 21:30:37,442][105620] Updated weights for policy 1, policy_version 844539 (0.0008) [2023-12-26 21:30:37,504][105620] Updated weights for policy 1, policy_version 844549 (0.0008) [2023-12-26 21:30:37,553][105620] Updated weights for policy 1, policy_version 844559 (0.0007) [2023-12-26 21:30:38,034][105692] Updated weights for policy 0, policy_version 844466 (0.0007) [2023-12-26 21:30:38,082][105692] Updated weights for policy 0, policy_version 844476 (0.0005) [2023-12-26 21:30:38,134][105692] Updated weights for policy 0, policy_version 844486 (0.0006) [2023-12-26 21:30:38,226][105620] Updated weights for policy 1, policy_version 844569 (0.0009) [2023-12-26 21:30:38,289][105620] Updated weights for policy 1, policy_version 844579 (0.0005) [2023-12-26 21:30:38,350][105620] Updated weights for policy 1, policy_version 844589 (0.0009) [2023-12-26 21:30:38,412][105620] Updated weights for policy 1, policy_version 844599 (0.0010) [2023-12-26 21:30:38,862][105692] Updated weights for policy 0, policy_version 844496 (0.0011) [2023-12-26 21:30:38,924][105692] Updated weights for policy 0, policy_version 844506 (0.0008) [2023-12-26 21:30:38,974][105692] Updated weights for policy 0, policy_version 844516 (0.0011) [2023-12-26 21:30:39,026][105620] Updated weights for policy 1, policy_version 844609 (0.0009) [2023-12-26 21:30:39,081][105620] Updated weights for policy 1, policy_version 844619 (0.0010) [2023-12-26 21:30:39,136][105620] Updated weights for policy 1, policy_version 844629 (0.0010) [2023-12-26 21:30:39,687][105692] Updated weights for policy 0, policy_version 844526 (0.0009) [2023-12-26 21:30:39,745][105692] Updated weights for policy 0, policy_version 844536 (0.0008) [2023-12-26 21:30:39,813][105692] Updated weights for policy 0, policy_version 844546 (0.0009) [2023-12-26 21:30:39,869][105620] Updated weights for policy 1, policy_version 844639 (0.0010) [2023-12-26 21:30:39,930][105620] Updated weights for policy 1, policy_version 844649 (0.0008) [2023-12-26 21:30:39,988][105620] Updated weights for policy 1, policy_version 844659 (0.0005) [2023-12-26 21:30:40,508][105692] Updated weights for policy 0, policy_version 844556 (0.0007) [2023-12-26 21:30:40,563][105692] Updated weights for policy 0, policy_version 844566 (0.0005) [2023-12-26 21:30:40,618][105692] Updated weights for policy 0, policy_version 844576 (0.0005) [2023-12-26 21:30:40,708][105620] Updated weights for policy 1, policy_version 844669 (0.0010) [2023-12-26 21:30:40,767][105620] Updated weights for policy 1, policy_version 844679 (0.0010) [2023-12-26 21:30:40,822][105620] Updated weights for policy 1, policy_version 844689 (0.0009) [2023-12-26 21:30:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 432513024. Throughput: 0: 9601.9, 1: 9943.5. Samples: 432520180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:30:41,063][104569] Avg episode reward: [(0, '9261.332'), (1, '8988.431')] [2023-12-26 21:30:41,215][105692] Updated weights for policy 0, policy_version 844586 (0.0006) [2023-12-26 21:30:41,277][105692] Updated weights for policy 0, policy_version 844596 (0.0009) [2023-12-26 21:30:41,332][105692] Updated weights for policy 0, policy_version 844607 (0.0010) [2023-12-26 21:30:41,565][105620] Updated weights for policy 1, policy_version 844699 (0.0010) [2023-12-26 21:30:41,631][105620] Updated weights for policy 1, policy_version 844709 (0.0008) [2023-12-26 21:30:41,698][105620] Updated weights for policy 1, policy_version 844719 (0.0007) [2023-12-26 21:30:42,131][105692] Updated weights for policy 0, policy_version 844617 (0.0008) [2023-12-26 21:30:42,194][105692] Updated weights for policy 0, policy_version 844627 (0.0009) [2023-12-26 21:30:42,258][105692] Updated weights for policy 0, policy_version 844637 (0.0009) [2023-12-26 21:30:42,326][105692] Updated weights for policy 0, policy_version 844647 (0.0009) [2023-12-26 21:30:42,495][105620] Updated weights for policy 1, policy_version 844729 (0.0010) [2023-12-26 21:30:42,559][105620] Updated weights for policy 1, policy_version 844739 (0.0008) [2023-12-26 21:30:42,624][105620] Updated weights for policy 1, policy_version 844749 (0.0008) [2023-12-26 21:30:42,688][105620] Updated weights for policy 1, policy_version 844759 (0.0007) [2023-12-26 21:30:43,134][105692] Updated weights for policy 0, policy_version 844657 (0.0009) [2023-12-26 21:30:43,197][105692] Updated weights for policy 0, policy_version 844667 (0.0010) [2023-12-26 21:30:43,265][105692] Updated weights for policy 0, policy_version 844677 (0.0010) [2023-12-26 21:30:43,338][105620] Updated weights for policy 1, policy_version 844769 (0.0009) [2023-12-26 21:30:43,401][105620] Updated weights for policy 1, policy_version 844779 (0.0009) [2023-12-26 21:30:43,457][105620] Updated weights for policy 1, policy_version 844789 (0.0010) [2023-12-26 21:30:44,057][105692] Updated weights for policy 0, policy_version 844687 (0.0009) [2023-12-26 21:30:44,116][105692] Updated weights for policy 0, policy_version 844697 (0.0010) [2023-12-26 21:30:44,180][105620] Updated weights for policy 1, policy_version 844799 (0.0007) [2023-12-26 21:30:44,180][105692] Updated weights for policy 0, policy_version 844707 (0.0010) [2023-12-26 21:30:44,244][105620] Updated weights for policy 1, policy_version 844809 (0.0008) [2023-12-26 21:30:44,301][105620] Updated weights for policy 1, policy_version 844819 (0.0008) [2023-12-26 21:30:44,984][105692] Updated weights for policy 0, policy_version 844717 (0.0007) [2023-12-26 21:30:44,989][105620] Updated weights for policy 1, policy_version 844829 (0.0009) [2023-12-26 21:30:45,051][105692] Updated weights for policy 0, policy_version 844727 (0.0006) [2023-12-26 21:30:45,054][105620] Updated weights for policy 1, policy_version 844839 (0.0008) [2023-12-26 21:30:45,113][105692] Updated weights for policy 0, policy_version 844737 (0.0007) [2023-12-26 21:30:45,115][105620] Updated weights for policy 1, policy_version 844849 (0.0007) [2023-12-26 21:30:45,795][105692] Updated weights for policy 0, policy_version 844747 (0.0006) [2023-12-26 21:30:45,812][105620] Updated weights for policy 1, policy_version 844859 (0.0008) [2023-12-26 21:30:45,844][105692] Updated weights for policy 0, policy_version 844757 (0.0009) [2023-12-26 21:30:45,875][105620] Updated weights for policy 1, policy_version 844869 (0.0007) [2023-12-26 21:30:45,905][105692] Updated weights for policy 0, policy_version 844767 (0.0008) [2023-12-26 21:30:45,929][105620] Updated weights for policy 1, policy_version 844879 (0.0006) [2023-12-26 21:30:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.9, 300 sec: 19521.9). Total num frames: 432611328. Throughput: 0: 9569.3, 1: 9949.2. Samples: 432575868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:30:46,063][104569] Avg episode reward: [(0, '9167.253'), (1, '9170.378')] [2023-12-26 21:30:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000844776_216293376.pth... [2023-12-26 21:30:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000844888_216317952.pth... [2023-12-26 21:30:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000843704_216014848.pth [2023-12-26 21:30:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000843624_215998464.pth [2023-12-26 21:30:46,676][105620] Updated weights for policy 1, policy_version 844889 (0.0008) [2023-12-26 21:30:46,685][105692] Updated weights for policy 0, policy_version 844777 (0.0006) [2023-12-26 21:30:46,733][105620] Updated weights for policy 1, policy_version 844899 (0.0006) [2023-12-26 21:30:46,739][105692] Updated weights for policy 0, policy_version 844787 (0.0007) [2023-12-26 21:30:46,791][105620] Updated weights for policy 1, policy_version 844909 (0.0006) [2023-12-26 21:30:46,801][105692] Updated weights for policy 0, policy_version 844797 (0.0009) [2023-12-26 21:30:46,836][105620] Updated weights for policy 1, policy_version 844919 (0.0006) [2023-12-26 21:30:46,859][105692] Updated weights for policy 0, policy_version 844807 (0.0007) [2023-12-26 21:30:47,576][105620] Updated weights for policy 1, policy_version 844929 (0.0006) [2023-12-26 21:30:47,638][105620] Updated weights for policy 1, policy_version 844939 (0.0006) [2023-12-26 21:30:47,657][105692] Updated weights for policy 0, policy_version 844817 (0.0007) [2023-12-26 21:30:47,703][105620] Updated weights for policy 1, policy_version 844949 (0.0009) [2023-12-26 21:30:47,714][105692] Updated weights for policy 0, policy_version 844827 (0.0007) [2023-12-26 21:30:47,771][105692] Updated weights for policy 0, policy_version 844837 (0.0010) [2023-12-26 21:30:48,379][105620] Updated weights for policy 1, policy_version 844959 (0.0007) [2023-12-26 21:30:48,443][105620] Updated weights for policy 1, policy_version 844969 (0.0008) [2023-12-26 21:30:48,511][105620] Updated weights for policy 1, policy_version 844979 (0.0009) [2023-12-26 21:30:48,546][105692] Updated weights for policy 0, policy_version 844847 (0.0007) [2023-12-26 21:30:48,600][105692] Updated weights for policy 0, policy_version 844857 (0.0008) [2023-12-26 21:30:48,650][105692] Updated weights for policy 0, policy_version 844867 (0.0008) [2023-12-26 21:30:49,261][105620] Updated weights for policy 1, policy_version 844989 (0.0011) [2023-12-26 21:30:49,320][105620] Updated weights for policy 1, policy_version 844999 (0.0007) [2023-12-26 21:30:49,369][105692] Updated weights for policy 0, policy_version 844877 (0.0009) [2023-12-26 21:30:49,392][105620] Updated weights for policy 1, policy_version 845009 (0.0008) [2023-12-26 21:30:49,437][105692] Updated weights for policy 0, policy_version 844887 (0.0006) [2023-12-26 21:30:49,508][105692] Updated weights for policy 0, policy_version 844897 (0.0009) [2023-12-26 21:30:50,115][105620] Updated weights for policy 1, policy_version 845019 (0.0010) [2023-12-26 21:30:50,167][105620] Updated weights for policy 1, policy_version 845029 (0.0009) [2023-12-26 21:30:50,221][105620] Updated weights for policy 1, policy_version 845039 (0.0008) [2023-12-26 21:30:50,302][105692] Updated weights for policy 0, policy_version 844907 (0.0009) [2023-12-26 21:30:50,362][105692] Updated weights for policy 0, policy_version 844917 (0.0009) [2023-12-26 21:30:50,427][105692] Updated weights for policy 0, policy_version 844927 (0.0009) [2023-12-26 21:30:51,015][105620] Updated weights for policy 1, policy_version 845049 (0.0009) [2023-12-26 21:30:51,062][104569] Fps is (10 sec: 18022.6, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 432693248. Throughput: 0: 9598.1, 1: 9859.5. Samples: 432689252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:30:51,062][104569] Avg episode reward: [(0, '9076.771'), (1, '9170.598')] [2023-12-26 21:30:51,088][105620] Updated weights for policy 1, policy_version 845059 (0.0008) [2023-12-26 21:30:51,160][105620] Updated weights for policy 1, policy_version 845069 (0.0009) [2023-12-26 21:30:51,211][105620] Updated weights for policy 1, policy_version 845079 (0.0010) [2023-12-26 21:30:51,228][105692] Updated weights for policy 0, policy_version 844938 (0.0010) [2023-12-26 21:30:51,294][105692] Updated weights for policy 0, policy_version 844948 (0.0010) [2023-12-26 21:30:51,365][105692] Updated weights for policy 0, policy_version 844958 (0.0008) [2023-12-26 21:30:51,428][105692] Updated weights for policy 0, policy_version 844968 (0.0009) [2023-12-26 21:30:51,998][105620] Updated weights for policy 1, policy_version 845089 (0.0009) [2023-12-26 21:30:52,047][105620] Updated weights for policy 1, policy_version 845099 (0.0009) [2023-12-26 21:30:52,110][105620] Updated weights for policy 1, policy_version 845109 (0.0009) [2023-12-26 21:30:52,200][105692] Updated weights for policy 0, policy_version 844978 (0.0006) [2023-12-26 21:30:52,268][105692] Updated weights for policy 0, policy_version 844988 (0.0006) [2023-12-26 21:30:52,331][105692] Updated weights for policy 0, policy_version 844998 (0.0007) [2023-12-26 21:30:52,965][105620] Updated weights for policy 1, policy_version 845119 (0.0010) [2023-12-26 21:30:53,012][105692] Updated weights for policy 0, policy_version 845008 (0.0009) [2023-12-26 21:30:53,015][105620] Updated weights for policy 1, policy_version 845129 (0.0007) [2023-12-26 21:30:53,065][105620] Updated weights for policy 1, policy_version 845139 (0.0007) [2023-12-26 21:30:53,070][105692] Updated weights for policy 0, policy_version 845018 (0.0009) [2023-12-26 21:30:53,127][105692] Updated weights for policy 0, policy_version 845028 (0.0006) [2023-12-26 21:30:53,806][105620] Updated weights for policy 1, policy_version 845149 (0.0006) [2023-12-26 21:30:53,854][105620] Updated weights for policy 1, policy_version 845159 (0.0005) [2023-12-26 21:30:53,876][105692] Updated weights for policy 0, policy_version 845038 (0.0007) [2023-12-26 21:30:53,901][105620] Updated weights for policy 1, policy_version 845169 (0.0005) [2023-12-26 21:30:53,934][105692] Updated weights for policy 0, policy_version 845048 (0.0008) [2023-12-26 21:30:53,989][105692] Updated weights for policy 0, policy_version 845058 (0.0008) [2023-12-26 21:30:54,598][105620] Updated weights for policy 1, policy_version 845179 (0.0006) [2023-12-26 21:30:54,656][105620] Updated weights for policy 1, policy_version 845189 (0.0005) [2023-12-26 21:30:54,723][105620] Updated weights for policy 1, policy_version 845199 (0.0009) [2023-12-26 21:30:54,807][105692] Updated weights for policy 0, policy_version 845068 (0.0009) [2023-12-26 21:30:54,860][105692] Updated weights for policy 0, policy_version 845078 (0.0008) [2023-12-26 21:30:54,906][105692] Updated weights for policy 0, policy_version 845088 (0.0008) [2023-12-26 21:30:55,326][105620] Updated weights for policy 1, policy_version 845209 (0.0010) [2023-12-26 21:30:55,377][105620] Updated weights for policy 1, policy_version 845219 (0.0010) [2023-12-26 21:30:55,425][105620] Updated weights for policy 1, policy_version 845229 (0.0010) [2023-12-26 21:30:55,476][105620] Updated weights for policy 1, policy_version 845239 (0.0010) [2023-12-26 21:30:55,720][105692] Updated weights for policy 0, policy_version 845098 (0.0008) [2023-12-26 21:30:55,778][105692] Updated weights for policy 0, policy_version 845108 (0.0006) [2023-12-26 21:30:55,837][105692] Updated weights for policy 0, policy_version 845118 (0.0006) [2023-12-26 21:30:55,890][105692] Updated weights for policy 0, policy_version 845128 (0.0007) [2023-12-26 21:30:56,061][105620] Updated weights for policy 1, policy_version 845249 (0.0007) [2023-12-26 21:30:56,062][104569] Fps is (10 sec: 18022.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 432791552. Throughput: 0: 9524.3, 1: 9850.9. Samples: 432801188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:30:56,062][104569] Avg episode reward: [(0, '9166.642'), (1, '9044.930')] [2023-12-26 21:30:56,125][105620] Updated weights for policy 1, policy_version 845259 (0.0010) [2023-12-26 21:30:56,184][105620] Updated weights for policy 1, policy_version 845269 (0.0010) [2023-12-26 21:30:56,454][105692] Updated weights for policy 0, policy_version 845138 (0.0011) [2023-12-26 21:30:56,509][105692] Updated weights for policy 0, policy_version 845148 (0.0010) [2023-12-26 21:30:56,568][105692] Updated weights for policy 0, policy_version 845158 (0.0010) [2023-12-26 21:30:56,902][105620] Updated weights for policy 1, policy_version 845279 (0.0007) [2023-12-26 21:30:56,973][105620] Updated weights for policy 1, policy_version 845289 (0.0005) [2023-12-26 21:30:57,032][105620] Updated weights for policy 1, policy_version 845299 (0.0006) [2023-12-26 21:30:57,137][105692] Updated weights for policy 0, policy_version 845168 (0.0009) [2023-12-26 21:30:57,188][105692] Updated weights for policy 0, policy_version 845178 (0.0010) [2023-12-26 21:30:57,249][105692] Updated weights for policy 0, policy_version 845188 (0.0010) [2023-12-26 21:30:57,531][105620] Updated weights for policy 1, policy_version 845309 (0.0008) [2023-12-26 21:30:57,579][105620] Updated weights for policy 1, policy_version 845319 (0.0010) [2023-12-26 21:30:57,627][105620] Updated weights for policy 1, policy_version 845329 (0.0010) [2023-12-26 21:30:57,984][105692] Updated weights for policy 0, policy_version 845198 (0.0010) [2023-12-26 21:30:58,031][105692] Updated weights for policy 0, policy_version 845208 (0.0010) [2023-12-26 21:30:58,085][105692] Updated weights for policy 0, policy_version 845218 (0.0010) [2023-12-26 21:30:58,395][105620] Updated weights for policy 1, policy_version 845339 (0.0010) [2023-12-26 21:30:58,461][105620] Updated weights for policy 1, policy_version 845349 (0.0009) [2023-12-26 21:30:58,528][105620] Updated weights for policy 1, policy_version 845359 (0.0011) [2023-12-26 21:30:58,938][105692] Updated weights for policy 0, policy_version 845228 (0.0008) [2023-12-26 21:30:59,002][105692] Updated weights for policy 0, policy_version 845238 (0.0008) [2023-12-26 21:30:59,062][105692] Updated weights for policy 0, policy_version 845248 (0.0008) [2023-12-26 21:30:59,412][105620] Updated weights for policy 1, policy_version 845369 (0.0010) [2023-12-26 21:30:59,468][105620] Updated weights for policy 1, policy_version 845379 (0.0008) [2023-12-26 21:30:59,523][105620] Updated weights for policy 1, policy_version 845389 (0.0009) [2023-12-26 21:30:59,585][105620] Updated weights for policy 1, policy_version 845399 (0.0008) [2023-12-26 21:30:59,758][105692] Updated weights for policy 0, policy_version 845258 (0.0008) [2023-12-26 21:30:59,812][105692] Updated weights for policy 0, policy_version 845269 (0.0009) [2023-12-26 21:30:59,878][105692] Updated weights for policy 0, policy_version 845279 (0.0007) [2023-12-26 21:31:00,355][105620] Updated weights for policy 1, policy_version 845409 (0.0010) [2023-12-26 21:31:00,406][105620] Updated weights for policy 1, policy_version 845419 (0.0009) [2023-12-26 21:31:00,467][105620] Updated weights for policy 1, policy_version 845429 (0.0009) [2023-12-26 21:31:00,625][105692] Updated weights for policy 0, policy_version 845289 (0.0009) [2023-12-26 21:31:00,687][105692] Updated weights for policy 0, policy_version 845299 (0.0010) [2023-12-26 21:31:00,757][105692] Updated weights for policy 0, policy_version 845309 (0.0010) [2023-12-26 21:31:00,813][105692] Updated weights for policy 0, policy_version 845319 (0.0010) [2023-12-26 21:31:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 432889856. Throughput: 0: 9602.1, 1: 9899.7. Samples: 432862600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:31:01,062][104569] Avg episode reward: [(0, '9255.576'), (1, '9045.243')] [2023-12-26 21:31:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000845320_216432640.pth... [2023-12-26 21:31:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000844200_216145920.pth [2023-12-26 21:31:01,097][105620] Updated weights for policy 1, policy_version 845439 (0.0009) [2023-12-26 21:31:01,159][105620] Updated weights for policy 1, policy_version 845449 (0.0009) [2023-12-26 21:31:01,219][105620] Updated weights for policy 1, policy_version 845459 (0.0009) [2023-12-26 21:31:01,243][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000845464_216465408.pth... [2023-12-26 21:31:01,248][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000844280_216162304.pth [2023-12-26 21:31:01,603][105692] Updated weights for policy 0, policy_version 845329 (0.0006) [2023-12-26 21:31:01,670][105692] Updated weights for policy 0, policy_version 845339 (0.0009) [2023-12-26 21:31:01,740][105692] Updated weights for policy 0, policy_version 845349 (0.0009) [2023-12-26 21:31:02,022][105620] Updated weights for policy 1, policy_version 845469 (0.0007) [2023-12-26 21:31:02,084][105620] Updated weights for policy 1, policy_version 845479 (0.0006) [2023-12-26 21:31:02,141][105620] Updated weights for policy 1, policy_version 845489 (0.0005) [2023-12-26 21:31:02,428][105692] Updated weights for policy 0, policy_version 845359 (0.0010) [2023-12-26 21:31:02,486][105692] Updated weights for policy 0, policy_version 845369 (0.0010) [2023-12-26 21:31:02,551][105692] Updated weights for policy 0, policy_version 845379 (0.0010) [2023-12-26 21:31:02,773][105620] Updated weights for policy 1, policy_version 845499 (0.0006) [2023-12-26 21:31:02,821][105620] Updated weights for policy 1, policy_version 845509 (0.0005) [2023-12-26 21:31:02,875][105620] Updated weights for policy 1, policy_version 845519 (0.0005) [2023-12-26 21:31:03,291][105692] Updated weights for policy 0, policy_version 845389 (0.0010) [2023-12-26 21:31:03,339][105692] Updated weights for policy 0, policy_version 845399 (0.0010) [2023-12-26 21:31:03,387][105692] Updated weights for policy 0, policy_version 845409 (0.0010) [2023-12-26 21:31:03,431][105620] Updated weights for policy 1, policy_version 845529 (0.0006) [2023-12-26 21:31:03,479][105620] Updated weights for policy 1, policy_version 845539 (0.0008) [2023-12-26 21:31:03,524][105620] Updated weights for policy 1, policy_version 845549 (0.0008) [2023-12-26 21:31:03,571][105620] Updated weights for policy 1, policy_version 845559 (0.0007) [2023-12-26 21:31:04,121][105692] Updated weights for policy 0, policy_version 845419 (0.0009) [2023-12-26 21:31:04,186][105692] Updated weights for policy 0, policy_version 845429 (0.0006) [2023-12-26 21:31:04,235][105692] Updated weights for policy 0, policy_version 845439 (0.0006) [2023-12-26 21:31:04,375][105620] Updated weights for policy 1, policy_version 845569 (0.0011) [2023-12-26 21:31:04,435][105620] Updated weights for policy 1, policy_version 845579 (0.0010) [2023-12-26 21:31:04,502][105620] Updated weights for policy 1, policy_version 845589 (0.0009) [2023-12-26 21:31:04,957][105692] Updated weights for policy 0, policy_version 845449 (0.0006) [2023-12-26 21:31:05,012][105692] Updated weights for policy 0, policy_version 845459 (0.0009) [2023-12-26 21:31:05,068][105692] Updated weights for policy 0, policy_version 845469 (0.0009) [2023-12-26 21:31:05,130][105692] Updated weights for policy 0, policy_version 845479 (0.0010) [2023-12-26 21:31:05,276][105620] Updated weights for policy 1, policy_version 845599 (0.0009) [2023-12-26 21:31:05,339][105620] Updated weights for policy 1, policy_version 845609 (0.0008) [2023-12-26 21:31:05,396][105620] Updated weights for policy 1, policy_version 845619 (0.0008) [2023-12-26 21:31:05,910][105692] Updated weights for policy 0, policy_version 845489 (0.0010) [2023-12-26 21:31:05,960][105692] Updated weights for policy 0, policy_version 845499 (0.0010) [2023-12-26 21:31:06,004][105692] Updated weights for policy 0, policy_version 845509 (0.0010) [2023-12-26 21:31:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 432988160. Throughput: 0: 9573.0, 1: 9953.4. Samples: 432977408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:31:06,062][104569] Avg episode reward: [(0, '9253.392'), (1, '9171.725')] [2023-12-26 21:31:06,157][105620] Updated weights for policy 1, policy_version 845629 (0.0008) [2023-12-26 21:31:06,220][105620] Updated weights for policy 1, policy_version 845639 (0.0008) [2023-12-26 21:31:06,280][105620] Updated weights for policy 1, policy_version 845649 (0.0008) [2023-12-26 21:31:06,729][105692] Updated weights for policy 0, policy_version 845519 (0.0010) [2023-12-26 21:31:06,793][105692] Updated weights for policy 0, policy_version 845529 (0.0009) [2023-12-26 21:31:06,853][105692] Updated weights for policy 0, policy_version 845539 (0.0009) [2023-12-26 21:31:07,025][105620] Updated weights for policy 1, policy_version 845659 (0.0009) [2023-12-26 21:31:07,089][105620] Updated weights for policy 1, policy_version 845669 (0.0008) [2023-12-26 21:31:07,149][105620] Updated weights for policy 1, policy_version 845679 (0.0009) [2023-12-26 21:31:07,544][105692] Updated weights for policy 0, policy_version 845549 (0.0008) [2023-12-26 21:31:07,597][105692] Updated weights for policy 0, policy_version 845559 (0.0008) [2023-12-26 21:31:07,652][105692] Updated weights for policy 0, policy_version 845569 (0.0009) [2023-12-26 21:31:07,928][105620] Updated weights for policy 1, policy_version 845689 (0.0008) [2023-12-26 21:31:07,992][105620] Updated weights for policy 1, policy_version 845699 (0.0009) [2023-12-26 21:31:08,054][105620] Updated weights for policy 1, policy_version 845709 (0.0008) [2023-12-26 21:31:08,117][105620] Updated weights for policy 1, policy_version 845719 (0.0008) [2023-12-26 21:31:08,410][105692] Updated weights for policy 0, policy_version 845579 (0.0008) [2023-12-26 21:31:08,476][105692] Updated weights for policy 0, policy_version 845589 (0.0006) [2023-12-26 21:31:08,526][105692] Updated weights for policy 0, policy_version 845599 (0.0006) [2023-12-26 21:31:08,917][105620] Updated weights for policy 1, policy_version 845729 (0.0006) [2023-12-26 21:31:08,972][105620] Updated weights for policy 1, policy_version 845739 (0.0009) [2023-12-26 21:31:09,022][105620] Updated weights for policy 1, policy_version 845749 (0.0009) [2023-12-26 21:31:09,063][105692] Updated weights for policy 0, policy_version 845609 (0.0005) [2023-12-26 21:31:09,114][105692] Updated weights for policy 0, policy_version 845619 (0.0005) [2023-12-26 21:31:09,164][105692] Updated weights for policy 0, policy_version 845629 (0.0007) [2023-12-26 21:31:09,233][105692] Updated weights for policy 0, policy_version 845639 (0.0009) [2023-12-26 21:31:09,833][105620] Updated weights for policy 1, policy_version 845759 (0.0009) [2023-12-26 21:31:09,894][105620] Updated weights for policy 1, policy_version 845769 (0.0007) [2023-12-26 21:31:09,939][105692] Updated weights for policy 0, policy_version 845649 (0.0010) [2023-12-26 21:31:09,965][105620] Updated weights for policy 1, policy_version 845779 (0.0006) [2023-12-26 21:31:09,995][105692] Updated weights for policy 0, policy_version 845659 (0.0010) [2023-12-26 21:31:10,048][105692] Updated weights for policy 0, policy_version 845669 (0.0009) [2023-12-26 21:31:10,649][105692] Updated weights for policy 0, policy_version 845679 (0.0006) [2023-12-26 21:31:10,707][105692] Updated weights for policy 0, policy_version 845689 (0.0006) [2023-12-26 21:31:10,772][105692] Updated weights for policy 0, policy_version 845699 (0.0006) [2023-12-26 21:31:10,790][105620] Updated weights for policy 1, policy_version 845789 (0.0007) [2023-12-26 21:31:10,855][105620] Updated weights for policy 1, policy_version 845799 (0.0008) [2023-12-26 21:31:10,913][105620] Updated weights for policy 1, policy_version 845809 (0.0009) [2023-12-26 21:31:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 433086464. Throughput: 0: 9644.5, 1: 9785.6. Samples: 433091380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:31:11,062][104569] Avg episode reward: [(0, '8906.273'), (1, '9080.311')] [2023-12-26 21:31:11,418][105692] Updated weights for policy 0, policy_version 845709 (0.0006) [2023-12-26 21:31:11,473][105692] Updated weights for policy 0, policy_version 845719 (0.0006) [2023-12-26 21:31:11,539][105692] Updated weights for policy 0, policy_version 845729 (0.0006) [2023-12-26 21:31:11,654][105620] Updated weights for policy 1, policy_version 845820 (0.0009) [2023-12-26 21:31:11,732][105620] Updated weights for policy 1, policy_version 845830 (0.0008) [2023-12-26 21:31:11,796][105620] Updated weights for policy 1, policy_version 845840 (0.0008) [2023-12-26 21:31:12,359][105692] Updated weights for policy 0, policy_version 845739 (0.0008) [2023-12-26 21:31:12,423][105692] Updated weights for policy 0, policy_version 845749 (0.0009) [2023-12-26 21:31:12,434][105620] Updated weights for policy 1, policy_version 845850 (0.0008) [2023-12-26 21:31:12,478][105692] Updated weights for policy 0, policy_version 845759 (0.0007) [2023-12-26 21:31:12,487][105620] Updated weights for policy 1, policy_version 845860 (0.0006) [2023-12-26 21:31:12,541][105620] Updated weights for policy 1, policy_version 845870 (0.0007) [2023-12-26 21:31:12,606][105620] Updated weights for policy 1, policy_version 845880 (0.0009) [2023-12-26 21:31:13,185][105692] Updated weights for policy 0, policy_version 845769 (0.0006) [2023-12-26 21:31:13,247][105692] Updated weights for policy 0, policy_version 845779 (0.0005) [2023-12-26 21:31:13,257][105620] Updated weights for policy 1, policy_version 845890 (0.0006) [2023-12-26 21:31:13,304][105620] Updated weights for policy 1, policy_version 845900 (0.0006) [2023-12-26 21:31:13,307][105692] Updated weights for policy 0, policy_version 845789 (0.0006) [2023-12-26 21:31:13,356][105692] Updated weights for policy 0, policy_version 845799 (0.0008) [2023-12-26 21:31:13,361][105620] Updated weights for policy 1, policy_version 845910 (0.0005) [2023-12-26 21:31:13,904][105620] Updated weights for policy 1, policy_version 845920 (0.0009) [2023-12-26 21:31:13,964][105620] Updated weights for policy 1, policy_version 845931 (0.0010) [2023-12-26 21:31:14,021][105620] Updated weights for policy 1, policy_version 845941 (0.0005) [2023-12-26 21:31:14,104][105692] Updated weights for policy 0, policy_version 845809 (0.0009) [2023-12-26 21:31:14,168][105692] Updated weights for policy 0, policy_version 845819 (0.0009) [2023-12-26 21:31:14,230][105692] Updated weights for policy 0, policy_version 845829 (0.0010) [2023-12-26 21:31:14,747][105620] Updated weights for policy 1, policy_version 845951 (0.0008) [2023-12-26 21:31:14,822][105620] Updated weights for policy 1, policy_version 845961 (0.0007) [2023-12-26 21:31:14,866][105692] Updated weights for policy 0, policy_version 845839 (0.0008) [2023-12-26 21:31:14,891][105620] Updated weights for policy 1, policy_version 845971 (0.0009) [2023-12-26 21:31:14,920][105692] Updated weights for policy 0, policy_version 845849 (0.0008) [2023-12-26 21:31:14,972][105692] Updated weights for policy 0, policy_version 845859 (0.0005) [2023-12-26 21:31:15,584][105620] Updated weights for policy 1, policy_version 845981 (0.0009) [2023-12-26 21:31:15,633][105620] Updated weights for policy 1, policy_version 845991 (0.0010) [2023-12-26 21:31:15,670][105692] Updated weights for policy 0, policy_version 845869 (0.0007) [2023-12-26 21:31:15,682][105620] Updated weights for policy 1, policy_version 846001 (0.0010) [2023-12-26 21:31:15,719][105692] Updated weights for policy 0, policy_version 845879 (0.0007) [2023-12-26 21:31:15,782][105692] Updated weights for policy 0, policy_version 845889 (0.0008) [2023-12-26 21:31:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 433184768. Throughput: 0: 9622.4, 1: 9852.3. Samples: 433151936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:31:16,062][104569] Avg episode reward: [(0, '8907.646'), (1, '9171.848')] [2023-12-26 21:31:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000845896_216580096.pth... [2023-12-26 21:31:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000846008_216604672.pth... [2023-12-26 21:31:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000844776_216293376.pth [2023-12-26 21:31:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000844888_216317952.pth [2023-12-26 21:31:16,373][105692] Updated weights for policy 0, policy_version 845899 (0.0009) [2023-12-26 21:31:16,433][105692] Updated weights for policy 0, policy_version 845909 (0.0005) [2023-12-26 21:31:16,470][105620] Updated weights for policy 1, policy_version 846011 (0.0011) [2023-12-26 21:31:16,490][105692] Updated weights for policy 0, policy_version 845919 (0.0010) [2023-12-26 21:31:16,527][105620] Updated weights for policy 1, policy_version 846021 (0.0011) [2023-12-26 21:31:16,586][105620] Updated weights for policy 1, policy_version 846031 (0.0011) [2023-12-26 21:31:17,135][105692] Updated weights for policy 0, policy_version 845929 (0.0009) [2023-12-26 21:31:17,193][105692] Updated weights for policy 0, policy_version 845939 (0.0010) [2023-12-26 21:31:17,262][105692] Updated weights for policy 0, policy_version 845949 (0.0005) [2023-12-26 21:31:17,317][105692] Updated weights for policy 0, policy_version 845959 (0.0006) [2023-12-26 21:31:17,368][105620] Updated weights for policy 1, policy_version 846041 (0.0010) [2023-12-26 21:31:17,429][105620] Updated weights for policy 1, policy_version 846051 (0.0008) [2023-12-26 21:31:17,480][105620] Updated weights for policy 1, policy_version 846061 (0.0005) [2023-12-26 21:31:17,531][105620] Updated weights for policy 1, policy_version 846071 (0.0005) [2023-12-26 21:31:17,980][105692] Updated weights for policy 0, policy_version 845969 (0.0010) [2023-12-26 21:31:18,031][105692] Updated weights for policy 0, policy_version 845979 (0.0010) [2023-12-26 21:31:18,085][105692] Updated weights for policy 0, policy_version 845989 (0.0010) [2023-12-26 21:31:18,197][105620] Updated weights for policy 1, policy_version 846081 (0.0005) [2023-12-26 21:31:18,258][105620] Updated weights for policy 1, policy_version 846091 (0.0006) [2023-12-26 21:31:18,310][105620] Updated weights for policy 1, policy_version 846101 (0.0010) [2023-12-26 21:31:18,827][105692] Updated weights for policy 0, policy_version 845999 (0.0011) [2023-12-26 21:31:18,875][105620] Updated weights for policy 1, policy_version 846111 (0.0008) [2023-12-26 21:31:18,884][105692] Updated weights for policy 0, policy_version 846009 (0.0011) [2023-12-26 21:31:18,926][105620] Updated weights for policy 1, policy_version 846121 (0.0006) [2023-12-26 21:31:18,936][105692] Updated weights for policy 0, policy_version 846019 (0.0011) [2023-12-26 21:31:18,978][105620] Updated weights for policy 1, policy_version 846131 (0.0007) [2023-12-26 21:31:19,593][105692] Updated weights for policy 0, policy_version 846029 (0.0009) [2023-12-26 21:31:19,660][105692] Updated weights for policy 0, policy_version 846039 (0.0008) [2023-12-26 21:31:19,719][105692] Updated weights for policy 0, policy_version 846049 (0.0009) [2023-12-26 21:31:19,814][105620] Updated weights for policy 1, policy_version 846141 (0.0009) [2023-12-26 21:31:19,882][105620] Updated weights for policy 1, policy_version 846151 (0.0008) [2023-12-26 21:31:19,936][105620] Updated weights for policy 1, policy_version 846161 (0.0008) [2023-12-26 21:31:20,426][105692] Updated weights for policy 0, policy_version 846059 (0.0010) [2023-12-26 21:31:20,489][105692] Updated weights for policy 0, policy_version 846069 (0.0011) [2023-12-26 21:31:20,544][105692] Updated weights for policy 0, policy_version 846079 (0.0011) [2023-12-26 21:31:20,633][105620] Updated weights for policy 1, policy_version 846171 (0.0007) [2023-12-26 21:31:20,698][105620] Updated weights for policy 1, policy_version 846181 (0.0006) [2023-12-26 21:31:20,764][105620] Updated weights for policy 1, policy_version 846191 (0.0008) [2023-12-26 21:31:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 433283072. Throughput: 0: 9691.4, 1: 9701.3. Samples: 433271700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:31:21,063][104569] Avg episode reward: [(0, '8817.112'), (1, '9263.394')] [2023-12-26 21:31:21,346][105692] Updated weights for policy 0, policy_version 846089 (0.0010) [2023-12-26 21:31:21,413][105692] Updated weights for policy 0, policy_version 846099 (0.0011) [2023-12-26 21:31:21,450][105620] Updated weights for policy 1, policy_version 846201 (0.0007) [2023-12-26 21:31:21,477][105692] Updated weights for policy 0, policy_version 846109 (0.0011) [2023-12-26 21:31:21,506][105620] Updated weights for policy 1, policy_version 846211 (0.0010) [2023-12-26 21:31:21,536][105692] Updated weights for policy 0, policy_version 846119 (0.0010) [2023-12-26 21:31:21,567][105620] Updated weights for policy 1, policy_version 846221 (0.0007) [2023-12-26 21:31:21,631][105620] Updated weights for policy 1, policy_version 846231 (0.0009) [2023-12-26 21:31:22,213][105692] Updated weights for policy 0, policy_version 846129 (0.0009) [2023-12-26 21:31:22,282][105692] Updated weights for policy 0, policy_version 846139 (0.0006) [2023-12-26 21:31:22,344][105692] Updated weights for policy 0, policy_version 846149 (0.0010) [2023-12-26 21:31:22,482][105620] Updated weights for policy 1, policy_version 846241 (0.0010) [2023-12-26 21:31:22,534][105620] Updated weights for policy 1, policy_version 846251 (0.0009) [2023-12-26 21:31:22,586][105620] Updated weights for policy 1, policy_version 846261 (0.0008) [2023-12-26 21:31:23,057][105692] Updated weights for policy 0, policy_version 846159 (0.0011) [2023-12-26 21:31:23,109][105692] Updated weights for policy 0, policy_version 846169 (0.0010) [2023-12-26 21:31:23,172][105692] Updated weights for policy 0, policy_version 846179 (0.0010) [2023-12-26 21:31:23,377][105620] Updated weights for policy 1, policy_version 846271 (0.0008) [2023-12-26 21:31:23,430][105620] Updated weights for policy 1, policy_version 846281 (0.0009) [2023-12-26 21:31:23,496][105620] Updated weights for policy 1, policy_version 846291 (0.0009) [2023-12-26 21:31:23,864][105692] Updated weights for policy 0, policy_version 846189 (0.0009) [2023-12-26 21:31:23,925][105692] Updated weights for policy 0, policy_version 846199 (0.0005) [2023-12-26 21:31:23,981][105692] Updated weights for policy 0, policy_version 846209 (0.0006) [2023-12-26 21:31:24,244][105620] Updated weights for policy 1, policy_version 846301 (0.0009) [2023-12-26 21:31:24,299][105620] Updated weights for policy 1, policy_version 846311 (0.0010) [2023-12-26 21:31:24,364][105620] Updated weights for policy 1, policy_version 846321 (0.0010) [2023-12-26 21:31:24,690][105692] Updated weights for policy 0, policy_version 846219 (0.0010) [2023-12-26 21:31:24,745][105692] Updated weights for policy 0, policy_version 846229 (0.0009) [2023-12-26 21:31:24,797][105692] Updated weights for policy 0, policy_version 846239 (0.0008) [2023-12-26 21:31:25,059][105620] Updated weights for policy 1, policy_version 846331 (0.0009) [2023-12-26 21:31:25,111][105620] Updated weights for policy 1, policy_version 846341 (0.0010) [2023-12-26 21:31:25,160][105620] Updated weights for policy 1, policy_version 846351 (0.0010) [2023-12-26 21:31:25,560][105692] Updated weights for policy 0, policy_version 846249 (0.0008) [2023-12-26 21:31:25,611][105692] Updated weights for policy 0, policy_version 846259 (0.0005) [2023-12-26 21:31:25,657][105692] Updated weights for policy 0, policy_version 846269 (0.0005) [2023-12-26 21:31:25,703][105692] Updated weights for policy 0, policy_version 846279 (0.0005) [2023-12-26 21:31:25,917][105620] Updated weights for policy 1, policy_version 846361 (0.0010) [2023-12-26 21:31:25,975][105620] Updated weights for policy 1, policy_version 846371 (0.0010) [2023-12-26 21:31:26,027][105620] Updated weights for policy 1, policy_version 846381 (0.0006) [2023-12-26 21:31:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.9, 300 sec: 19466.4). Total num frames: 433373184. Throughput: 0: 9637.3, 1: 9598.8. Samples: 433385800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:31:26,062][104569] Avg episode reward: [(0, '9167.144'), (1, '9354.539')] [2023-12-26 21:31:26,077][105620] Updated weights for policy 1, policy_version 846391 (0.0007) [2023-12-26 21:31:26,289][105692] Updated weights for policy 0, policy_version 846289 (0.0005) [2023-12-26 21:31:26,351][105692] Updated weights for policy 0, policy_version 846299 (0.0005) [2023-12-26 21:31:26,411][105692] Updated weights for policy 0, policy_version 846309 (0.0005) [2023-12-26 21:31:26,828][105620] Updated weights for policy 1, policy_version 846401 (0.0010) [2023-12-26 21:31:26,876][105620] Updated weights for policy 1, policy_version 846411 (0.0010) [2023-12-26 21:31:26,923][105620] Updated weights for policy 1, policy_version 846421 (0.0010) [2023-12-26 21:31:26,988][105692] Updated weights for policy 0, policy_version 846319 (0.0007) [2023-12-26 21:31:27,043][105692] Updated weights for policy 0, policy_version 846329 (0.0008) [2023-12-26 21:31:27,100][105692] Updated weights for policy 0, policy_version 846339 (0.0008) [2023-12-26 21:31:27,680][105620] Updated weights for policy 1, policy_version 846431 (0.0010) [2023-12-26 21:31:27,736][105620] Updated weights for policy 1, policy_version 846441 (0.0010) [2023-12-26 21:31:27,795][105620] Updated weights for policy 1, policy_version 846451 (0.0009) [2023-12-26 21:31:27,858][105692] Updated weights for policy 0, policy_version 846349 (0.0007) [2023-12-26 21:31:27,901][105692] Updated weights for policy 0, policy_version 846359 (0.0008) [2023-12-26 21:31:27,955][105692] Updated weights for policy 0, policy_version 846369 (0.0008) [2023-12-26 21:31:28,519][105620] Updated weights for policy 1, policy_version 846461 (0.0008) [2023-12-26 21:31:28,571][105620] Updated weights for policy 1, policy_version 846471 (0.0010) [2023-12-26 21:31:28,590][105692] Updated weights for policy 0, policy_version 846379 (0.0008) [2023-12-26 21:31:28,623][105620] Updated weights for policy 1, policy_version 846481 (0.0010) [2023-12-26 21:31:28,657][105692] Updated weights for policy 0, policy_version 846389 (0.0008) [2023-12-26 21:31:28,721][105692] Updated weights for policy 0, policy_version 846399 (0.0008) [2023-12-26 21:31:29,271][105692] Updated weights for policy 0, policy_version 846409 (0.0008) [2023-12-26 21:31:29,329][105692] Updated weights for policy 0, policy_version 846419 (0.0008) [2023-12-26 21:31:29,386][105692] Updated weights for policy 0, policy_version 846429 (0.0008) [2023-12-26 21:31:29,389][105620] Updated weights for policy 1, policy_version 846491 (0.0010) [2023-12-26 21:31:29,437][105692] Updated weights for policy 0, policy_version 846439 (0.0008) [2023-12-26 21:31:29,441][105620] Updated weights for policy 1, policy_version 846501 (0.0007) [2023-12-26 21:31:29,488][105620] Updated weights for policy 1, policy_version 846511 (0.0008) [2023-12-26 21:31:30,147][105620] Updated weights for policy 1, policy_version 846521 (0.0007) [2023-12-26 21:31:30,185][105692] Updated weights for policy 0, policy_version 846449 (0.0009) [2023-12-26 21:31:30,208][105620] Updated weights for policy 1, policy_version 846531 (0.0005) [2023-12-26 21:31:30,236][105692] Updated weights for policy 0, policy_version 846459 (0.0010) [2023-12-26 21:31:30,268][105620] Updated weights for policy 1, policy_version 846541 (0.0006) [2023-12-26 21:31:30,292][105692] Updated weights for policy 0, policy_version 846469 (0.0010) [2023-12-26 21:31:30,319][105620] Updated weights for policy 1, policy_version 846551 (0.0007) [2023-12-26 21:31:30,923][105620] Updated weights for policy 1, policy_version 846561 (0.0006) [2023-12-26 21:31:30,958][105692] Updated weights for policy 0, policy_version 846479 (0.0010) [2023-12-26 21:31:30,990][105620] Updated weights for policy 1, policy_version 846571 (0.0005) [2023-12-26 21:31:31,013][105692] Updated weights for policy 0, policy_version 846489 (0.0010) [2023-12-26 21:31:31,054][105620] Updated weights for policy 1, policy_version 846581 (0.0007) [2023-12-26 21:31:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 433471488. Throughput: 0: 9742.6, 1: 9607.4. Samples: 433446616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:31:31,063][104569] Avg episode reward: [(0, '9172.757'), (1, '9353.938')] [2023-12-26 21:31:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000846584_216752128.pth... [2023-12-26 21:31:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000845464_216465408.pth [2023-12-26 21:31:31,081][105692] Updated weights for policy 0, policy_version 846499 (0.0010) [2023-12-26 21:31:31,110][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000846504_216735744.pth... [2023-12-26 21:31:31,113][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000845320_216432640.pth [2023-12-26 21:31:31,740][105620] Updated weights for policy 1, policy_version 846591 (0.0009) [2023-12-26 21:31:31,800][105620] Updated weights for policy 1, policy_version 846601 (0.0007) [2023-12-26 21:31:31,854][105620] Updated weights for policy 1, policy_version 846611 (0.0007) [2023-12-26 21:31:31,855][105692] Updated weights for policy 0, policy_version 846509 (0.0011) [2023-12-26 21:31:31,911][105692] Updated weights for policy 0, policy_version 846519 (0.0010) [2023-12-26 21:31:31,962][105692] Updated weights for policy 0, policy_version 846529 (0.0010) [2023-12-26 21:31:32,526][105620] Updated weights for policy 1, policy_version 846621 (0.0007) [2023-12-26 21:31:32,586][105620] Updated weights for policy 1, policy_version 846631 (0.0008) [2023-12-26 21:31:32,645][105620] Updated weights for policy 1, policy_version 846641 (0.0008) [2023-12-26 21:31:32,673][105692] Updated weights for policy 0, policy_version 846539 (0.0009) [2023-12-26 21:31:32,730][105692] Updated weights for policy 0, policy_version 846549 (0.0008) [2023-12-26 21:31:32,794][105692] Updated weights for policy 0, policy_version 846559 (0.0008) [2023-12-26 21:31:33,302][105620] Updated weights for policy 1, policy_version 846651 (0.0009) [2023-12-26 21:31:33,360][105620] Updated weights for policy 1, policy_version 846661 (0.0010) [2023-12-26 21:31:33,416][105620] Updated weights for policy 1, policy_version 846671 (0.0010) [2023-12-26 21:31:33,615][105692] Updated weights for policy 0, policy_version 846569 (0.0008) [2023-12-26 21:31:33,665][105692] Updated weights for policy 0, policy_version 846579 (0.0006) [2023-12-26 21:31:33,712][105692] Updated weights for policy 0, policy_version 846589 (0.0008) [2023-12-26 21:31:33,758][105692] Updated weights for policy 0, policy_version 846599 (0.0009) [2023-12-26 21:31:34,139][105620] Updated weights for policy 1, policy_version 846681 (0.0006) [2023-12-26 21:31:34,207][105620] Updated weights for policy 1, policy_version 846691 (0.0007) [2023-12-26 21:31:34,275][105620] Updated weights for policy 1, policy_version 846701 (0.0008) [2023-12-26 21:31:34,333][105620] Updated weights for policy 1, policy_version 846711 (0.0008) [2023-12-26 21:31:34,402][105692] Updated weights for policy 0, policy_version 846609 (0.0007) [2023-12-26 21:31:34,463][105692] Updated weights for policy 0, policy_version 846619 (0.0007) [2023-12-26 21:31:34,509][105692] Updated weights for policy 0, policy_version 846629 (0.0008) [2023-12-26 21:31:35,104][105620] Updated weights for policy 1, policy_version 846721 (0.0009) [2023-12-26 21:31:35,154][105620] Updated weights for policy 1, policy_version 846731 (0.0008) [2023-12-26 21:31:35,182][105692] Updated weights for policy 0, policy_version 846639 (0.0008) [2023-12-26 21:31:35,207][105620] Updated weights for policy 1, policy_version 846741 (0.0007) [2023-12-26 21:31:35,240][105692] Updated weights for policy 0, policy_version 846649 (0.0007) [2023-12-26 21:31:35,304][105692] Updated weights for policy 0, policy_version 846659 (0.0009) [2023-12-26 21:31:35,930][105692] Updated weights for policy 0, policy_version 846669 (0.0007) [2023-12-26 21:31:35,988][105692] Updated weights for policy 0, policy_version 846679 (0.0005) [2023-12-26 21:31:36,045][105692] Updated weights for policy 0, policy_version 846689 (0.0005) [2023-12-26 21:31:36,046][105620] Updated weights for policy 1, policy_version 846751 (0.0008) [2023-12-26 21:31:36,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19251.1, 300 sec: 19410.9). Total num frames: 433569792. Throughput: 0: 9842.3, 1: 9650.3. Samples: 433566424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:31:36,063][104569] Avg episode reward: [(0, '9048.830'), (1, '9266.114')] [2023-12-26 21:31:36,114][105620] Updated weights for policy 1, policy_version 846761 (0.0009) [2023-12-26 21:31:36,178][105620] Updated weights for policy 1, policy_version 846771 (0.0009) [2023-12-26 21:31:36,691][105692] Updated weights for policy 0, policy_version 846699 (0.0007) [2023-12-26 21:31:36,750][105692] Updated weights for policy 0, policy_version 846709 (0.0010) [2023-12-26 21:31:36,812][105692] Updated weights for policy 0, policy_version 846719 (0.0009) [2023-12-26 21:31:36,890][105620] Updated weights for policy 1, policy_version 846781 (0.0010) [2023-12-26 21:31:36,946][105620] Updated weights for policy 1, policy_version 846791 (0.0011) [2023-12-26 21:31:36,998][105620] Updated weights for policy 1, policy_version 846801 (0.0011) [2023-12-26 21:31:37,511][105692] Updated weights for policy 0, policy_version 846729 (0.0008) [2023-12-26 21:31:37,562][105692] Updated weights for policy 0, policy_version 846739 (0.0005) [2023-12-26 21:31:37,610][105692] Updated weights for policy 0, policy_version 846749 (0.0006) [2023-12-26 21:31:37,658][105620] Updated weights for policy 1, policy_version 846811 (0.0009) [2023-12-26 21:31:37,669][105692] Updated weights for policy 0, policy_version 846759 (0.0006) [2023-12-26 21:31:37,724][105620] Updated weights for policy 1, policy_version 846821 (0.0006) [2023-12-26 21:31:37,791][105620] Updated weights for policy 1, policy_version 846831 (0.0010) [2023-12-26 21:31:38,368][105692] Updated weights for policy 0, policy_version 846769 (0.0010) [2023-12-26 21:31:38,427][105692] Updated weights for policy 0, policy_version 846779 (0.0010) [2023-12-26 21:31:38,485][105692] Updated weights for policy 0, policy_version 846789 (0.0010) [2023-12-26 21:31:38,500][105620] Updated weights for policy 1, policy_version 846841 (0.0010) [2023-12-26 21:31:38,544][105620] Updated weights for policy 1, policy_version 846851 (0.0008) [2023-12-26 21:31:38,593][105620] Updated weights for policy 1, policy_version 846861 (0.0008) [2023-12-26 21:31:38,643][105620] Updated weights for policy 1, policy_version 846871 (0.0008) [2023-12-26 21:31:39,217][105692] Updated weights for policy 0, policy_version 846799 (0.0011) [2023-12-26 21:31:39,282][105692] Updated weights for policy 0, policy_version 846809 (0.0010) [2023-12-26 21:31:39,350][105692] Updated weights for policy 0, policy_version 846819 (0.0011) [2023-12-26 21:31:39,479][105620] Updated weights for policy 1, policy_version 846881 (0.0008) [2023-12-26 21:31:39,546][105620] Updated weights for policy 1, policy_version 846891 (0.0008) [2023-12-26 21:31:39,616][105620] Updated weights for policy 1, policy_version 846901 (0.0007) [2023-12-26 21:31:40,099][105692] Updated weights for policy 0, policy_version 846829 (0.0011) [2023-12-26 21:31:40,163][105692] Updated weights for policy 0, policy_version 846839 (0.0011) [2023-12-26 21:31:40,229][105692] Updated weights for policy 0, policy_version 846849 (0.0011) [2023-12-26 21:31:40,285][105620] Updated weights for policy 1, policy_version 846911 (0.0006) [2023-12-26 21:31:40,346][105620] Updated weights for policy 1, policy_version 846921 (0.0008) [2023-12-26 21:31:40,402][105620] Updated weights for policy 1, policy_version 846931 (0.0008) [2023-12-26 21:31:40,975][105692] Updated weights for policy 0, policy_version 846859 (0.0011) [2023-12-26 21:31:41,034][105692] Updated weights for policy 0, policy_version 846869 (0.0011) [2023-12-26 21:31:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 433668096. Throughput: 0: 9969.0, 1: 9618.5. Samples: 433682624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:31:41,062][104569] Avg episode reward: [(0, '9048.180'), (1, '9264.186')] [2023-12-26 21:31:41,092][105692] Updated weights for policy 0, policy_version 846879 (0.0011) [2023-12-26 21:31:41,190][105620] Updated weights for policy 1, policy_version 846941 (0.0009) [2023-12-26 21:31:41,247][105620] Updated weights for policy 1, policy_version 846951 (0.0010) [2023-12-26 21:31:41,314][105620] Updated weights for policy 1, policy_version 846961 (0.0011) [2023-12-26 21:31:41,839][105692] Updated weights for policy 0, policy_version 846889 (0.0010) [2023-12-26 21:31:41,902][105692] Updated weights for policy 0, policy_version 846899 (0.0011) [2023-12-26 21:31:41,962][105692] Updated weights for policy 0, policy_version 846909 (0.0011) [2023-12-26 21:31:42,027][105692] Updated weights for policy 0, policy_version 846919 (0.0009) [2023-12-26 21:31:42,137][105620] Updated weights for policy 1, policy_version 846971 (0.0010) [2023-12-26 21:31:42,196][105620] Updated weights for policy 1, policy_version 846981 (0.0008) [2023-12-26 21:31:42,262][105620] Updated weights for policy 1, policy_version 846991 (0.0009) [2023-12-26 21:31:42,748][105692] Updated weights for policy 0, policy_version 846929 (0.0006) [2023-12-26 21:31:42,815][105692] Updated weights for policy 0, policy_version 846939 (0.0005) [2023-12-26 21:31:42,879][105692] Updated weights for policy 0, policy_version 846949 (0.0008) [2023-12-26 21:31:42,967][105620] Updated weights for policy 1, policy_version 847001 (0.0008) [2023-12-26 21:31:43,027][105620] Updated weights for policy 1, policy_version 847011 (0.0008) [2023-12-26 21:31:43,074][105620] Updated weights for policy 1, policy_version 847021 (0.0008) [2023-12-26 21:31:43,123][105620] Updated weights for policy 1, policy_version 847031 (0.0006) [2023-12-26 21:31:43,562][105692] Updated weights for policy 0, policy_version 846959 (0.0009) [2023-12-26 21:31:43,620][105692] Updated weights for policy 0, policy_version 846969 (0.0009) [2023-12-26 21:31:43,682][105692] Updated weights for policy 0, policy_version 846979 (0.0009) [2023-12-26 21:31:43,829][105620] Updated weights for policy 1, policy_version 847041 (0.0006) [2023-12-26 21:31:43,875][105620] Updated weights for policy 1, policy_version 847051 (0.0005) [2023-12-26 21:31:43,919][105620] Updated weights for policy 1, policy_version 847061 (0.0005) [2023-12-26 21:31:44,263][105692] Updated weights for policy 0, policy_version 846989 (0.0007) [2023-12-26 21:31:44,326][105692] Updated weights for policy 0, policy_version 846999 (0.0005) [2023-12-26 21:31:44,387][105692] Updated weights for policy 0, policy_version 847009 (0.0005) [2023-12-26 21:31:44,462][105620] Updated weights for policy 1, policy_version 847071 (0.0006) [2023-12-26 21:31:44,523][105620] Updated weights for policy 1, policy_version 847081 (0.0008) [2023-12-26 21:31:44,583][105620] Updated weights for policy 1, policy_version 847091 (0.0007) [2023-12-26 21:31:45,086][105692] Updated weights for policy 0, policy_version 847019 (0.0007) [2023-12-26 21:31:45,144][105692] Updated weights for policy 0, policy_version 847029 (0.0008) [2023-12-26 21:31:45,208][105692] Updated weights for policy 0, policy_version 847039 (0.0008) [2023-12-26 21:31:45,297][105620] Updated weights for policy 1, policy_version 847101 (0.0008) [2023-12-26 21:31:45,361][105620] Updated weights for policy 1, policy_version 847111 (0.0008) [2023-12-26 21:31:45,419][105620] Updated weights for policy 1, policy_version 847121 (0.0009) [2023-12-26 21:31:45,921][105692] Updated weights for policy 0, policy_version 847049 (0.0008) [2023-12-26 21:31:45,979][105692] Updated weights for policy 0, policy_version 847059 (0.0009) [2023-12-26 21:31:46,031][105692] Updated weights for policy 0, policy_version 847069 (0.0009) [2023-12-26 21:31:46,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 433766400. Throughput: 0: 9895.2, 1: 9595.9. Samples: 433739700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:31:46,062][104569] Avg episode reward: [(0, '8980.782'), (1, '9258.682')] [2023-12-26 21:31:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000847128_216891392.pth... [2023-12-26 21:31:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000846008_216604672.pth [2023-12-26 21:31:46,087][105692] Updated weights for policy 0, policy_version 847079 (0.0009) [2023-12-26 21:31:46,090][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000847080_216883200.pth... [2023-12-26 21:31:46,093][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000845896_216580096.pth [2023-12-26 21:31:46,176][105620] Updated weights for policy 1, policy_version 847131 (0.0009) [2023-12-26 21:31:46,230][105620] Updated weights for policy 1, policy_version 847142 (0.0010) [2023-12-26 21:31:46,283][105620] Updated weights for policy 1, policy_version 847152 (0.0009) [2023-12-26 21:31:46,843][105692] Updated weights for policy 0, policy_version 847089 (0.0009) [2023-12-26 21:31:46,911][105692] Updated weights for policy 0, policy_version 847099 (0.0010) [2023-12-26 21:31:46,964][105620] Updated weights for policy 1, policy_version 847163 (0.0008) [2023-12-26 21:31:46,978][105692] Updated weights for policy 0, policy_version 847109 (0.0009) [2023-12-26 21:31:47,025][105620] Updated weights for policy 1, policy_version 847173 (0.0009) [2023-12-26 21:31:47,086][105620] Updated weights for policy 1, policy_version 847183 (0.0009) [2023-12-26 21:31:47,737][105692] Updated weights for policy 0, policy_version 847119 (0.0007) [2023-12-26 21:31:47,798][105692] Updated weights for policy 0, policy_version 847129 (0.0007) [2023-12-26 21:31:47,844][105620] Updated weights for policy 1, policy_version 847193 (0.0009) [2023-12-26 21:31:47,846][105692] Updated weights for policy 0, policy_version 847139 (0.0005) [2023-12-26 21:31:47,898][105620] Updated weights for policy 1, policy_version 847203 (0.0010) [2023-12-26 21:31:47,956][105620] Updated weights for policy 1, policy_version 847213 (0.0010) [2023-12-26 21:31:48,019][105620] Updated weights for policy 1, policy_version 847223 (0.0010) [2023-12-26 21:31:48,434][105692] Updated weights for policy 0, policy_version 847149 (0.0007) [2023-12-26 21:31:48,485][105692] Updated weights for policy 0, policy_version 847159 (0.0009) [2023-12-26 21:31:48,549][105692] Updated weights for policy 0, policy_version 847169 (0.0009) [2023-12-26 21:31:48,744][105620] Updated weights for policy 1, policy_version 847233 (0.0006) [2023-12-26 21:31:48,806][105620] Updated weights for policy 1, policy_version 847243 (0.0005) [2023-12-26 21:31:48,863][105620] Updated weights for policy 1, policy_version 847253 (0.0005) [2023-12-26 21:31:49,383][105620] Updated weights for policy 1, policy_version 847263 (0.0006) [2023-12-26 21:31:49,445][105620] Updated weights for policy 1, policy_version 847273 (0.0007) [2023-12-26 21:31:49,458][105692] Updated weights for policy 0, policy_version 847179 (0.0009) [2023-12-26 21:31:49,498][105620] Updated weights for policy 1, policy_version 847283 (0.0008) [2023-12-26 21:31:49,517][105692] Updated weights for policy 0, policy_version 847189 (0.0009) [2023-12-26 21:31:49,581][105692] Updated weights for policy 0, policy_version 847199 (0.0008) [2023-12-26 21:31:50,206][105620] Updated weights for policy 1, policy_version 847293 (0.0008) [2023-12-26 21:31:50,271][105620] Updated weights for policy 1, policy_version 847303 (0.0009) [2023-12-26 21:31:50,334][105620] Updated weights for policy 1, policy_version 847313 (0.0009) [2023-12-26 21:31:50,346][105692] Updated weights for policy 0, policy_version 847209 (0.0008) [2023-12-26 21:31:50,407][105692] Updated weights for policy 0, policy_version 847219 (0.0008) [2023-12-26 21:31:50,467][105692] Updated weights for policy 0, policy_version 847229 (0.0006) [2023-12-26 21:31:50,520][105692] Updated weights for policy 0, policy_version 847239 (0.0005) [2023-12-26 21:31:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 433864704. Throughput: 0: 9942.3, 1: 9645.9. Samples: 433858876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:31:51,062][104569] Avg episode reward: [(0, '9068.635'), (1, '8511.805')] [2023-12-26 21:31:51,079][105620] Updated weights for policy 1, policy_version 847323 (0.0010) [2023-12-26 21:31:51,146][105620] Updated weights for policy 1, policy_version 847333 (0.0009) [2023-12-26 21:31:51,211][105620] Updated weights for policy 1, policy_version 847343 (0.0009) [2023-12-26 21:31:51,285][105692] Updated weights for policy 0, policy_version 847249 (0.0008) [2023-12-26 21:31:51,341][105692] Updated weights for policy 0, policy_version 847259 (0.0009) [2023-12-26 21:31:51,426][105692] Updated weights for policy 0, policy_version 847269 (0.0009) [2023-12-26 21:31:52,014][105620] Updated weights for policy 1, policy_version 847353 (0.0007) [2023-12-26 21:31:52,070][105692] Updated weights for policy 0, policy_version 847279 (0.0007) [2023-12-26 21:31:52,073][105620] Updated weights for policy 1, policy_version 847363 (0.0009) [2023-12-26 21:31:52,128][105620] Updated weights for policy 1, policy_version 847373 (0.0007) [2023-12-26 21:31:52,135][105692] Updated weights for policy 0, policy_version 847289 (0.0007) [2023-12-26 21:31:52,168][105586] KL-divergence is very high: 124.8995 [2023-12-26 21:31:52,186][105586] KL-divergence is very high: 117.4533 [2023-12-26 21:31:52,192][105620] Updated weights for policy 1, policy_version 847383 (0.0010) [2023-12-26 21:31:52,195][105692] Updated weights for policy 0, policy_version 847299 (0.0008) [2023-12-26 21:31:52,917][105586] KL-divergence is very high: 115.6934 [2023-12-26 21:31:52,929][105586] KL-divergence is very high: 108.8034 [2023-12-26 21:31:52,934][105692] Updated weights for policy 0, policy_version 847309 (0.0008) [2023-12-26 21:31:52,968][105620] Updated weights for policy 1, policy_version 847393 (0.0008) [2023-12-26 21:31:52,984][105692] Updated weights for policy 0, policy_version 847319 (0.0007) [2023-12-26 21:31:53,020][105620] Updated weights for policy 1, policy_version 847403 (0.0006) [2023-12-26 21:31:53,038][105692] Updated weights for policy 0, policy_version 847329 (0.0006) [2023-12-26 21:31:53,087][105620] Updated weights for policy 1, policy_version 847413 (0.0009) [2023-12-26 21:31:53,759][105692] Updated weights for policy 0, policy_version 847339 (0.0006) [2023-12-26 21:31:53,821][105692] Updated weights for policy 0, policy_version 847349 (0.0009) [2023-12-26 21:31:53,827][105620] Updated weights for policy 1, policy_version 847423 (0.0008) [2023-12-26 21:31:53,869][105692] Updated weights for policy 0, policy_version 847359 (0.0007) [2023-12-26 21:31:53,886][105620] Updated weights for policy 1, policy_version 847433 (0.0008) [2023-12-26 21:31:53,946][105620] Updated weights for policy 1, policy_version 847443 (0.0008) [2023-12-26 21:31:54,485][105692] Updated weights for policy 0, policy_version 847369 (0.0008) [2023-12-26 21:31:54,538][105692] Updated weights for policy 0, policy_version 847379 (0.0005) [2023-12-26 21:31:54,582][105692] Updated weights for policy 0, policy_version 847389 (0.0005) [2023-12-26 21:31:54,638][105692] Updated weights for policy 0, policy_version 847399 (0.0006) [2023-12-26 21:31:54,783][105620] Updated weights for policy 1, policy_version 847453 (0.0009) [2023-12-26 21:31:54,843][105620] Updated weights for policy 1, policy_version 847464 (0.0012) [2023-12-26 21:31:54,897][105620] Updated weights for policy 1, policy_version 847474 (0.0010) [2023-12-26 21:31:55,272][105692] Updated weights for policy 0, policy_version 847409 (0.0008) [2023-12-26 21:31:55,320][105692] Updated weights for policy 0, policy_version 847419 (0.0009) [2023-12-26 21:31:55,375][105692] Updated weights for policy 0, policy_version 847430 (0.0009) [2023-12-26 21:31:55,652][105620] Updated weights for policy 1, policy_version 847485 (0.0009) [2023-12-26 21:31:55,699][105620] Updated weights for policy 1, policy_version 847495 (0.0009) [2023-12-26 21:31:55,749][105620] Updated weights for policy 1, policy_version 847506 (0.0010) [2023-12-26 21:31:56,051][105692] Updated weights for policy 0, policy_version 847440 (0.0009) [2023-12-26 21:31:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 433963008. Throughput: 0: 9899.2, 1: 9658.9. Samples: 433971496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:31:56,063][104569] Avg episode reward: [(0, '9337.122'), (1, '6632.953')] [2023-12-26 21:31:56,113][105692] Updated weights for policy 0, policy_version 847450 (0.0009) [2023-12-26 21:31:56,177][105692] Updated weights for policy 0, policy_version 847460 (0.0009) [2023-12-26 21:31:56,431][105620] Updated weights for policy 1, policy_version 847516 (0.0009) [2023-12-26 21:31:56,488][105620] Updated weights for policy 1, policy_version 847526 (0.0009) [2023-12-26 21:31:56,549][105620] Updated weights for policy 1, policy_version 847536 (0.0009) [2023-12-26 21:31:56,928][105692] Updated weights for policy 0, policy_version 847470 (0.0008) [2023-12-26 21:31:56,975][105692] Updated weights for policy 0, policy_version 847480 (0.0009) [2023-12-26 21:31:57,023][105692] Updated weights for policy 0, policy_version 847490 (0.0009) [2023-12-26 21:31:57,279][105620] Updated weights for policy 1, policy_version 847546 (0.0009) [2023-12-26 21:31:57,341][105620] Updated weights for policy 1, policy_version 847556 (0.0006) [2023-12-26 21:31:57,398][105620] Updated weights for policy 1, policy_version 847566 (0.0005) [2023-12-26 21:31:57,463][105620] Updated weights for policy 1, policy_version 847576 (0.0006) [2023-12-26 21:31:57,828][105692] Updated weights for policy 0, policy_version 847500 (0.0008) [2023-12-26 21:31:57,886][105692] Updated weights for policy 0, policy_version 847510 (0.0005) [2023-12-26 21:31:57,944][105692] Updated weights for policy 0, policy_version 847520 (0.0006) [2023-12-26 21:31:58,039][105620] Updated weights for policy 1, policy_version 847586 (0.0010) [2023-12-26 21:31:58,098][105620] Updated weights for policy 1, policy_version 847596 (0.0010) [2023-12-26 21:31:58,167][105620] Updated weights for policy 1, policy_version 847606 (0.0008) [2023-12-26 21:31:58,624][105692] Updated weights for policy 0, policy_version 847530 (0.0007) [2023-12-26 21:31:58,682][105692] Updated weights for policy 0, policy_version 847540 (0.0009) [2023-12-26 21:31:58,750][105692] Updated weights for policy 0, policy_version 847550 (0.0009) [2023-12-26 21:31:58,827][105692] Updated weights for policy 0, policy_version 847560 (0.0009) [2023-12-26 21:31:58,994][105620] Updated weights for policy 1, policy_version 847616 (0.0009) [2023-12-26 21:31:59,057][105620] Updated weights for policy 1, policy_version 847626 (0.0008) [2023-12-26 21:31:59,123][105620] Updated weights for policy 1, policy_version 847636 (0.0009) [2023-12-26 21:31:59,567][105692] Updated weights for policy 0, policy_version 847570 (0.0009) [2023-12-26 21:31:59,628][105692] Updated weights for policy 0, policy_version 847581 (0.0007) [2023-12-26 21:31:59,692][105692] Updated weights for policy 0, policy_version 847591 (0.0005) [2023-12-26 21:31:59,844][105620] Updated weights for policy 1, policy_version 847646 (0.0009) [2023-12-26 21:31:59,900][105620] Updated weights for policy 1, policy_version 847656 (0.0006) [2023-12-26 21:31:59,961][105620] Updated weights for policy 1, policy_version 847666 (0.0007) [2023-12-26 21:32:00,308][105692] Updated weights for policy 0, policy_version 847601 (0.0005) [2023-12-26 21:32:00,364][105692] Updated weights for policy 0, policy_version 847611 (0.0007) [2023-12-26 21:32:00,427][105692] Updated weights for policy 0, policy_version 847621 (0.0009) [2023-12-26 21:32:00,609][105620] Updated weights for policy 1, policy_version 847676 (0.0008) [2023-12-26 21:32:00,663][105620] Updated weights for policy 1, policy_version 847686 (0.0010) [2023-12-26 21:32:00,720][105620] Updated weights for policy 1, policy_version 847696 (0.0010) [2023-12-26 21:32:01,000][105692] Updated weights for policy 0, policy_version 847631 (0.0006) [2023-12-26 21:32:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 434061312. Throughput: 0: 9918.6, 1: 9627.5. Samples: 434031512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:32:01,063][104569] Avg episode reward: [(0, '9246.834'), (1, '7642.175')] [2023-12-26 21:32:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000847704_217038848.pth... [2023-12-26 21:32:01,069][105692] Updated weights for policy 0, policy_version 847641 (0.0007) [2023-12-26 21:32:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000846584_216752128.pth [2023-12-26 21:32:01,132][105692] Updated weights for policy 0, policy_version 847651 (0.0009) [2023-12-26 21:32:01,160][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000847656_217030656.pth... [2023-12-26 21:32:01,165][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000846504_216735744.pth [2023-12-26 21:32:01,352][105620] Updated weights for policy 1, policy_version 847706 (0.0009) [2023-12-26 21:32:01,411][105620] Updated weights for policy 1, policy_version 847716 (0.0010) [2023-12-26 21:32:01,459][105620] Updated weights for policy 1, policy_version 847726 (0.0010) [2023-12-26 21:32:01,507][105620] Updated weights for policy 1, policy_version 847736 (0.0009) [2023-12-26 21:32:01,800][105692] Updated weights for policy 0, policy_version 847661 (0.0008) [2023-12-26 21:32:01,859][105692] Updated weights for policy 0, policy_version 847671 (0.0009) [2023-12-26 21:32:01,914][105692] Updated weights for policy 0, policy_version 847681 (0.0009) [2023-12-26 21:32:02,279][105620] Updated weights for policy 1, policy_version 847746 (0.0011) [2023-12-26 21:32:02,345][105620] Updated weights for policy 1, policy_version 847756 (0.0011) [2023-12-26 21:32:02,413][105620] Updated weights for policy 1, policy_version 847766 (0.0009) [2023-12-26 21:32:02,619][105692] Updated weights for policy 0, policy_version 847691 (0.0008) [2023-12-26 21:32:02,667][105692] Updated weights for policy 0, policy_version 847701 (0.0008) [2023-12-26 21:32:02,718][105692] Updated weights for policy 0, policy_version 847711 (0.0007) [2023-12-26 21:32:03,150][105620] Updated weights for policy 1, policy_version 847776 (0.0006) [2023-12-26 21:32:03,214][105620] Updated weights for policy 1, policy_version 847786 (0.0007) [2023-12-26 21:32:03,279][105620] Updated weights for policy 1, policy_version 847796 (0.0007) [2023-12-26 21:32:03,557][105692] Updated weights for policy 0, policy_version 847721 (0.0008) [2023-12-26 21:32:03,610][105692] Updated weights for policy 0, policy_version 847731 (0.0009) [2023-12-26 21:32:03,657][105692] Updated weights for policy 0, policy_version 847741 (0.0007) [2023-12-26 21:32:03,701][105692] Updated weights for policy 0, policy_version 847751 (0.0008) [2023-12-26 21:32:03,832][105620] Updated weights for policy 1, policy_version 847806 (0.0008) [2023-12-26 21:32:03,894][105620] Updated weights for policy 1, policy_version 847816 (0.0011) [2023-12-26 21:32:03,950][105620] Updated weights for policy 1, policy_version 847826 (0.0011) [2023-12-26 21:32:04,456][105692] Updated weights for policy 0, policy_version 847761 (0.0010) [2023-12-26 21:32:04,516][105692] Updated weights for policy 0, policy_version 847771 (0.0011) [2023-12-26 21:32:04,580][105692] Updated weights for policy 0, policy_version 847781 (0.0010) [2023-12-26 21:32:04,632][105620] Updated weights for policy 1, policy_version 847836 (0.0009) [2023-12-26 21:32:04,681][105620] Updated weights for policy 1, policy_version 847846 (0.0005) [2023-12-26 21:32:04,726][105620] Updated weights for policy 1, policy_version 847856 (0.0005) [2023-12-26 21:32:05,306][105692] Updated weights for policy 0, policy_version 847791 (0.0010) [2023-12-26 21:32:05,361][105692] Updated weights for policy 0, policy_version 847801 (0.0011) [2023-12-26 21:32:05,406][105620] Updated weights for policy 1, policy_version 847866 (0.0008) [2023-12-26 21:32:05,421][105692] Updated weights for policy 0, policy_version 847811 (0.0011) [2023-12-26 21:32:05,465][105620] Updated weights for policy 1, policy_version 847876 (0.0010) [2023-12-26 21:32:05,534][105620] Updated weights for policy 1, policy_version 847886 (0.0010) [2023-12-26 21:32:05,596][105620] Updated weights for policy 1, policy_version 847896 (0.0010) [2023-12-26 21:32:06,058][105692] Updated weights for policy 0, policy_version 847821 (0.0008) [2023-12-26 21:32:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 434159616. Throughput: 0: 9847.9, 1: 9700.6. Samples: 434151384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:32:06,063][104569] Avg episode reward: [(0, '8964.972'), (1, '9172.207')] [2023-12-26 21:32:06,110][105692] Updated weights for policy 0, policy_version 847831 (0.0006) [2023-12-26 21:32:06,173][105692] Updated weights for policy 0, policy_version 847841 (0.0007) [2023-12-26 21:32:06,175][105620] Updated weights for policy 1, policy_version 847906 (0.0007) [2023-12-26 21:32:06,237][105620] Updated weights for policy 1, policy_version 847916 (0.0011) [2023-12-26 21:32:06,301][105620] Updated weights for policy 1, policy_version 847926 (0.0011) [2023-12-26 21:32:06,761][105692] Updated weights for policy 0, policy_version 847851 (0.0006) [2023-12-26 21:32:06,829][105692] Updated weights for policy 0, policy_version 847861 (0.0005) [2023-12-26 21:32:06,901][105692] Updated weights for policy 0, policy_version 847871 (0.0006) [2023-12-26 21:32:06,946][105620] Updated weights for policy 1, policy_version 847936 (0.0009) [2023-12-26 21:32:06,998][105620] Updated weights for policy 1, policy_version 847946 (0.0006) [2023-12-26 21:32:07,051][105620] Updated weights for policy 1, policy_version 847956 (0.0011) [2023-12-26 21:32:07,518][105692] Updated weights for policy 0, policy_version 847881 (0.0007) [2023-12-26 21:32:07,585][105692] Updated weights for policy 0, policy_version 847891 (0.0006) [2023-12-26 21:32:07,644][105692] Updated weights for policy 0, policy_version 847901 (0.0005) [2023-12-26 21:32:07,700][105692] Updated weights for policy 0, policy_version 847911 (0.0005) [2023-12-26 21:32:07,809][105620] Updated weights for policy 1, policy_version 847966 (0.0008) [2023-12-26 21:32:07,867][105620] Updated weights for policy 1, policy_version 847976 (0.0010) [2023-12-26 21:32:07,924][105620] Updated weights for policy 1, policy_version 847986 (0.0010) [2023-12-26 21:32:08,319][105692] Updated weights for policy 0, policy_version 847921 (0.0008) [2023-12-26 21:32:08,383][105692] Updated weights for policy 0, policy_version 847931 (0.0009) [2023-12-26 21:32:08,446][105692] Updated weights for policy 0, policy_version 847941 (0.0009) [2023-12-26 21:32:08,636][105620] Updated weights for policy 1, policy_version 847996 (0.0009) [2023-12-26 21:32:08,704][105620] Updated weights for policy 1, policy_version 848006 (0.0006) [2023-12-26 21:32:08,764][105620] Updated weights for policy 1, policy_version 848016 (0.0006) [2023-12-26 21:32:09,288][105692] Updated weights for policy 0, policy_version 847951 (0.0009) [2023-12-26 21:32:09,328][105620] Updated weights for policy 1, policy_version 848026 (0.0006) [2023-12-26 21:32:09,355][105692] Updated weights for policy 0, policy_version 847961 (0.0009) [2023-12-26 21:32:09,393][105620] Updated weights for policy 1, policy_version 848036 (0.0009) [2023-12-26 21:32:09,416][105692] Updated weights for policy 0, policy_version 847971 (0.0008) [2023-12-26 21:32:09,461][105620] Updated weights for policy 1, policy_version 848046 (0.0006) [2023-12-26 21:32:09,528][105620] Updated weights for policy 1, policy_version 848056 (0.0005) [2023-12-26 21:32:10,139][105692] Updated weights for policy 0, policy_version 847981 (0.0008) [2023-12-26 21:32:10,203][105692] Updated weights for policy 0, policy_version 847991 (0.0010) [2023-12-26 21:32:10,258][105692] Updated weights for policy 0, policy_version 848001 (0.0007) [2023-12-26 21:32:10,261][105620] Updated weights for policy 1, policy_version 848066 (0.0006) [2023-12-26 21:32:10,318][105620] Updated weights for policy 1, policy_version 848076 (0.0008) [2023-12-26 21:32:10,379][105620] Updated weights for policy 1, policy_version 848086 (0.0009) [2023-12-26 21:32:10,924][105692] Updated weights for policy 0, policy_version 848011 (0.0006) [2023-12-26 21:32:10,969][105692] Updated weights for policy 0, policy_version 848021 (0.0006) [2023-12-26 21:32:11,026][105692] Updated weights for policy 0, policy_version 848031 (0.0008) [2023-12-26 21:32:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 434257920. Throughput: 0: 9902.9, 1: 9793.2. Samples: 434272128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:32:11,063][104569] Avg episode reward: [(0, '8738.637'), (1, '9170.462')] [2023-12-26 21:32:11,134][105620] Updated weights for policy 1, policy_version 848096 (0.0009) [2023-12-26 21:32:11,193][105620] Updated weights for policy 1, policy_version 848106 (0.0008) [2023-12-26 21:32:11,256][105620] Updated weights for policy 1, policy_version 848116 (0.0008) [2023-12-26 21:32:11,843][105692] Updated weights for policy 0, policy_version 848041 (0.0009) [2023-12-26 21:32:11,918][105692] Updated weights for policy 0, policy_version 848051 (0.0010) [2023-12-26 21:32:11,978][105692] Updated weights for policy 0, policy_version 848061 (0.0008) [2023-12-26 21:32:11,992][105620] Updated weights for policy 1, policy_version 848126 (0.0008) [2023-12-26 21:32:12,039][105692] Updated weights for policy 0, policy_version 848071 (0.0007) [2023-12-26 21:32:12,054][105620] Updated weights for policy 1, policy_version 848136 (0.0006) [2023-12-26 21:32:12,121][105620] Updated weights for policy 1, policy_version 848146 (0.0007) [2023-12-26 21:32:12,727][105692] Updated weights for policy 0, policy_version 848081 (0.0011) [2023-12-26 21:32:12,793][105692] Updated weights for policy 0, policy_version 848091 (0.0011) [2023-12-26 21:32:12,796][105620] Updated weights for policy 1, policy_version 848156 (0.0006) [2023-12-26 21:32:12,845][105692] Updated weights for policy 0, policy_version 848101 (0.0011) [2023-12-26 21:32:12,853][105620] Updated weights for policy 1, policy_version 848166 (0.0005) [2023-12-26 21:32:12,903][105620] Updated weights for policy 1, policy_version 848176 (0.0008) [2023-12-26 21:32:13,519][105692] Updated weights for policy 0, policy_version 848111 (0.0010) [2023-12-26 21:32:13,551][105620] Updated weights for policy 1, policy_version 848186 (0.0007) [2023-12-26 21:32:13,575][105692] Updated weights for policy 0, policy_version 848121 (0.0010) [2023-12-26 21:32:13,604][105620] Updated weights for policy 1, policy_version 848196 (0.0005) [2023-12-26 21:32:13,637][105692] Updated weights for policy 0, policy_version 848131 (0.0010) [2023-12-26 21:32:13,667][105620] Updated weights for policy 1, policy_version 848206 (0.0006) [2023-12-26 21:32:13,731][105620] Updated weights for policy 1, policy_version 848216 (0.0008) [2023-12-26 21:32:14,380][105692] Updated weights for policy 0, policy_version 848141 (0.0010) [2023-12-26 21:32:14,402][105620] Updated weights for policy 1, policy_version 848226 (0.0007) [2023-12-26 21:32:14,440][105692] Updated weights for policy 0, policy_version 848151 (0.0007) [2023-12-26 21:32:14,464][105620] Updated weights for policy 1, policy_version 848236 (0.0008) [2023-12-26 21:32:14,506][105692] Updated weights for policy 0, policy_version 848161 (0.0005) [2023-12-26 21:32:14,533][105620] Updated weights for policy 1, policy_version 848246 (0.0005) [2023-12-26 21:32:15,107][105692] Updated weights for policy 0, policy_version 848171 (0.0006) [2023-12-26 21:32:15,169][105692] Updated weights for policy 0, policy_version 848181 (0.0008) [2023-12-26 21:32:15,226][105620] Updated weights for policy 1, policy_version 848256 (0.0005) [2023-12-26 21:32:15,227][105692] Updated weights for policy 0, policy_version 848191 (0.0009) [2023-12-26 21:32:15,274][105620] Updated weights for policy 1, policy_version 848266 (0.0005) [2023-12-26 21:32:15,335][105620] Updated weights for policy 1, policy_version 848276 (0.0006) [2023-12-26 21:32:15,970][105620] Updated weights for policy 1, policy_version 848286 (0.0007) [2023-12-26 21:32:16,020][105620] Updated weights for policy 1, policy_version 848296 (0.0005) [2023-12-26 21:32:16,050][105692] Updated weights for policy 0, policy_version 848201 (0.0009) [2023-12-26 21:32:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 434356224. Throughput: 0: 9841.2, 1: 9824.8. Samples: 434331588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:32:16,063][104569] Avg episode reward: [(0, '9062.650'), (1, '9171.019')] [2023-12-26 21:32:16,075][105620] Updated weights for policy 1, policy_version 848306 (0.0006) [2023-12-26 21:32:16,101][105692] Updated weights for policy 0, policy_version 848211 (0.0008) [2023-12-26 21:32:16,108][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000848312_217194496.pth... [2023-12-26 21:32:16,111][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000847128_216891392.pth [2023-12-26 21:32:16,146][105692] Updated weights for policy 0, policy_version 848221 (0.0008) [2023-12-26 21:32:16,204][105692] Updated weights for policy 0, policy_version 848231 (0.0009) [2023-12-26 21:32:16,209][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000848232_217178112.pth... [2023-12-26 21:32:16,213][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000847080_216883200.pth [2023-12-26 21:32:16,762][105620] Updated weights for policy 1, policy_version 848316 (0.0008) [2023-12-26 21:32:16,809][105692] Updated weights for policy 0, policy_version 848241 (0.0008) [2023-12-26 21:32:16,811][105620] Updated weights for policy 1, policy_version 848326 (0.0006) [2023-12-26 21:32:16,864][105620] Updated weights for policy 1, policy_version 848336 (0.0006) [2023-12-26 21:32:16,866][105692] Updated weights for policy 0, policy_version 848251 (0.0007) [2023-12-26 21:32:16,919][105692] Updated weights for policy 0, policy_version 848261 (0.0007) [2023-12-26 21:32:17,623][105620] Updated weights for policy 1, policy_version 848346 (0.0006) [2023-12-26 21:32:17,687][105620] Updated weights for policy 1, policy_version 848356 (0.0007) [2023-12-26 21:32:17,688][105692] Updated weights for policy 0, policy_version 848271 (0.0009) [2023-12-26 21:32:17,746][105620] Updated weights for policy 1, policy_version 848366 (0.0006) [2023-12-26 21:32:17,752][105692] Updated weights for policy 0, policy_version 848281 (0.0011) [2023-12-26 21:32:17,812][105620] Updated weights for policy 1, policy_version 848376 (0.0005) [2023-12-26 21:32:17,814][105692] Updated weights for policy 0, policy_version 848291 (0.0011) [2023-12-26 21:32:18,446][105692] Updated weights for policy 0, policy_version 848301 (0.0008) [2023-12-26 21:32:18,500][105692] Updated weights for policy 0, policy_version 848311 (0.0006) [2023-12-26 21:32:18,561][105692] Updated weights for policy 0, policy_version 848321 (0.0008) [2023-12-26 21:32:18,611][105620] Updated weights for policy 1, policy_version 848386 (0.0006) [2023-12-26 21:32:18,672][105620] Updated weights for policy 1, policy_version 848396 (0.0006) [2023-12-26 21:32:18,723][105620] Updated weights for policy 1, policy_version 848406 (0.0008) [2023-12-26 21:32:19,264][105692] Updated weights for policy 0, policy_version 848331 (0.0011) [2023-12-26 21:32:19,317][105692] Updated weights for policy 0, policy_version 848341 (0.0011) [2023-12-26 21:32:19,388][105692] Updated weights for policy 0, policy_version 848351 (0.0011) [2023-12-26 21:32:19,464][105620] Updated weights for policy 1, policy_version 848416 (0.0006) [2023-12-26 21:32:19,528][105620] Updated weights for policy 1, policy_version 848426 (0.0008) [2023-12-26 21:32:19,578][105620] Updated weights for policy 1, policy_version 848436 (0.0008) [2023-12-26 21:32:20,158][105692] Updated weights for policy 0, policy_version 848361 (0.0010) [2023-12-26 21:32:20,210][105692] Updated weights for policy 0, policy_version 848371 (0.0008) [2023-12-26 21:32:20,279][105692] Updated weights for policy 0, policy_version 848381 (0.0008) [2023-12-26 21:32:20,292][105620] Updated weights for policy 1, policy_version 848446 (0.0008) [2023-12-26 21:32:20,328][105692] Updated weights for policy 0, policy_version 848391 (0.0008) [2023-12-26 21:32:20,345][105620] Updated weights for policy 1, policy_version 848456 (0.0006) [2023-12-26 21:32:20,408][105620] Updated weights for policy 1, policy_version 848466 (0.0008) [2023-12-26 21:32:21,048][105620] Updated weights for policy 1, policy_version 848476 (0.0008) [2023-12-26 21:32:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 434454528. Throughput: 0: 9829.2, 1: 9782.3. Samples: 434448936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:32:21,062][104569] Avg episode reward: [(0, '9052.721'), (1, '9263.429')] [2023-12-26 21:32:21,095][105692] Updated weights for policy 0, policy_version 848401 (0.0007) [2023-12-26 21:32:21,139][105620] Updated weights for policy 1, policy_version 848486 (0.0008) [2023-12-26 21:32:21,164][105692] Updated weights for policy 0, policy_version 848411 (0.0008) [2023-12-26 21:32:21,208][105620] Updated weights for policy 1, policy_version 848496 (0.0007) [2023-12-26 21:32:21,226][105692] Updated weights for policy 0, policy_version 848421 (0.0007) [2023-12-26 21:32:21,935][105692] Updated weights for policy 0, policy_version 848431 (0.0009) [2023-12-26 21:32:21,959][105620] Updated weights for policy 1, policy_version 848506 (0.0008) [2023-12-26 21:32:21,990][105692] Updated weights for policy 0, policy_version 848441 (0.0006) [2023-12-26 21:32:22,011][105620] Updated weights for policy 1, policy_version 848516 (0.0008) [2023-12-26 21:32:22,058][105692] Updated weights for policy 0, policy_version 848451 (0.0006) [2023-12-26 21:32:22,072][105620] Updated weights for policy 1, policy_version 848526 (0.0008) [2023-12-26 21:32:22,126][105620] Updated weights for policy 1, policy_version 848536 (0.0009) [2023-12-26 21:32:22,752][105692] Updated weights for policy 0, policy_version 848461 (0.0008) [2023-12-26 21:32:22,817][105692] Updated weights for policy 0, policy_version 848471 (0.0009) [2023-12-26 21:32:22,876][105692] Updated weights for policy 0, policy_version 848481 (0.0008) [2023-12-26 21:32:22,899][105620] Updated weights for policy 1, policy_version 848546 (0.0009) [2023-12-26 21:32:22,959][105620] Updated weights for policy 1, policy_version 848556 (0.0008) [2023-12-26 21:32:23,016][105620] Updated weights for policy 1, policy_version 848566 (0.0010) [2023-12-26 21:32:23,621][105692] Updated weights for policy 0, policy_version 848491 (0.0007) [2023-12-26 21:32:23,670][105692] Updated weights for policy 0, policy_version 848501 (0.0008) [2023-12-26 21:32:23,717][105692] Updated weights for policy 0, policy_version 848511 (0.0009) [2023-12-26 21:32:23,783][105620] Updated weights for policy 1, policy_version 848576 (0.0008) [2023-12-26 21:32:23,838][105620] Updated weights for policy 1, policy_version 848586 (0.0010) [2023-12-26 21:32:23,906][105620] Updated weights for policy 1, policy_version 848596 (0.0010) [2023-12-26 21:32:24,447][105692] Updated weights for policy 0, policy_version 848521 (0.0008) [2023-12-26 21:32:24,512][105692] Updated weights for policy 0, policy_version 848531 (0.0011) [2023-12-26 21:32:24,575][105692] Updated weights for policy 0, policy_version 848541 (0.0010) [2023-12-26 21:32:24,623][105692] Updated weights for policy 0, policy_version 848551 (0.0010) [2023-12-26 21:32:24,643][105620] Updated weights for policy 1, policy_version 848606 (0.0009) [2023-12-26 21:32:24,695][105620] Updated weights for policy 1, policy_version 848616 (0.0008) [2023-12-26 21:32:24,742][105620] Updated weights for policy 1, policy_version 848626 (0.0008) [2023-12-26 21:32:25,370][105692] Updated weights for policy 0, policy_version 848561 (0.0009) [2023-12-26 21:32:25,416][105692] Updated weights for policy 0, policy_version 848571 (0.0007) [2023-12-26 21:32:25,473][105692] Updated weights for policy 0, policy_version 848581 (0.0006) [2023-12-26 21:32:25,475][105620] Updated weights for policy 1, policy_version 848636 (0.0008) [2023-12-26 21:32:25,523][105620] Updated weights for policy 1, policy_version 848646 (0.0005) [2023-12-26 21:32:25,585][105620] Updated weights for policy 1, policy_version 848656 (0.0006) [2023-12-26 21:32:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 434552832. Throughput: 0: 9773.0, 1: 9806.0. Samples: 434563680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:32:26,063][104569] Avg episode reward: [(0, '8873.903'), (1, '9263.173')] [2023-12-26 21:32:26,104][105692] Updated weights for policy 0, policy_version 848591 (0.0008) [2023-12-26 21:32:26,155][105692] Updated weights for policy 0, policy_version 848601 (0.0008) [2023-12-26 21:32:26,209][105692] Updated weights for policy 0, policy_version 848611 (0.0010) [2023-12-26 21:32:26,267][105620] Updated weights for policy 1, policy_version 848666 (0.0005) [2023-12-26 21:32:26,314][105620] Updated weights for policy 1, policy_version 848676 (0.0005) [2023-12-26 21:32:26,372][105620] Updated weights for policy 1, policy_version 848686 (0.0006) [2023-12-26 21:32:26,433][105620] Updated weights for policy 1, policy_version 848696 (0.0009) [2023-12-26 21:32:26,933][105692] Updated weights for policy 0, policy_version 848621 (0.0007) [2023-12-26 21:32:26,977][105692] Updated weights for policy 0, policy_version 848631 (0.0006) [2023-12-26 21:32:27,028][105692] Updated weights for policy 0, policy_version 848641 (0.0008) [2023-12-26 21:32:27,041][105620] Updated weights for policy 1, policy_version 848706 (0.0005) [2023-12-26 21:32:27,098][105620] Updated weights for policy 1, policy_version 848716 (0.0006) [2023-12-26 21:32:27,155][105620] Updated weights for policy 1, policy_version 848726 (0.0009) [2023-12-26 21:32:27,790][105692] Updated weights for policy 0, policy_version 848651 (0.0008) [2023-12-26 21:32:27,813][105620] Updated weights for policy 1, policy_version 848736 (0.0007) [2023-12-26 21:32:27,847][105692] Updated weights for policy 0, policy_version 848661 (0.0008) [2023-12-26 21:32:27,864][105620] Updated weights for policy 1, policy_version 848746 (0.0005) [2023-12-26 21:32:27,906][105692] Updated weights for policy 0, policy_version 848671 (0.0009) [2023-12-26 21:32:27,910][105620] Updated weights for policy 1, policy_version 848756 (0.0005) [2023-12-26 21:32:28,465][105620] Updated weights for policy 1, policy_version 848766 (0.0007) [2023-12-26 21:32:28,512][105620] Updated weights for policy 1, policy_version 848776 (0.0008) [2023-12-26 21:32:28,564][105620] Updated weights for policy 1, policy_version 848786 (0.0008) [2023-12-26 21:32:28,757][105692] Updated weights for policy 0, policy_version 848681 (0.0010) [2023-12-26 21:32:28,823][105692] Updated weights for policy 0, policy_version 848691 (0.0009) [2023-12-26 21:32:28,886][105692] Updated weights for policy 0, policy_version 848701 (0.0009) [2023-12-26 21:32:28,947][105692] Updated weights for policy 0, policy_version 848711 (0.0009) [2023-12-26 21:32:29,264][105620] Updated weights for policy 1, policy_version 848796 (0.0006) [2023-12-26 21:32:29,322][105620] Updated weights for policy 1, policy_version 848806 (0.0008) [2023-12-26 21:32:29,393][105620] Updated weights for policy 1, policy_version 848816 (0.0007) [2023-12-26 21:32:29,683][105692] Updated weights for policy 0, policy_version 848722 (0.0010) [2023-12-26 21:32:29,737][105692] Updated weights for policy 0, policy_version 848732 (0.0010) [2023-12-26 21:32:29,791][105692] Updated weights for policy 0, policy_version 848744 (0.0010) [2023-12-26 21:32:30,024][105620] Updated weights for policy 1, policy_version 848826 (0.0006) [2023-12-26 21:32:30,086][105620] Updated weights for policy 1, policy_version 848836 (0.0005) [2023-12-26 21:32:30,147][105620] Updated weights for policy 1, policy_version 848846 (0.0005) [2023-12-26 21:32:30,205][105620] Updated weights for policy 1, policy_version 848856 (0.0005) [2023-12-26 21:32:30,609][105692] Updated weights for policy 0, policy_version 848754 (0.0006) [2023-12-26 21:32:30,660][105692] Updated weights for policy 0, policy_version 848764 (0.0005) [2023-12-26 21:32:30,713][105692] Updated weights for policy 0, policy_version 848774 (0.0005) [2023-12-26 21:32:30,842][105620] Updated weights for policy 1, policy_version 848866 (0.0010) [2023-12-26 21:32:30,903][105620] Updated weights for policy 1, policy_version 848876 (0.0009) [2023-12-26 21:32:30,956][105620] Updated weights for policy 1, policy_version 848886 (0.0010) [2023-12-26 21:32:31,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 434659328. Throughput: 0: 9780.6, 1: 9907.6. Samples: 434625672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:32:31,063][104569] Avg episode reward: [(0, '9064.961'), (1, '9077.512')] [2023-12-26 21:32:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000848888_217341952.pth... [2023-12-26 21:32:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000848776_217317376.pth... [2023-12-26 21:32:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000847656_217030656.pth [2023-12-26 21:32:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000847704_217038848.pth [2023-12-26 21:32:31,294][105692] Updated weights for policy 0, policy_version 848784 (0.0008) [2023-12-26 21:32:31,361][105692] Updated weights for policy 0, policy_version 848794 (0.0008) [2023-12-26 21:32:31,413][105692] Updated weights for policy 0, policy_version 848804 (0.0008) [2023-12-26 21:32:31,765][105620] Updated weights for policy 1, policy_version 848896 (0.0010) [2023-12-26 21:32:31,830][105620] Updated weights for policy 1, policy_version 848906 (0.0010) [2023-12-26 21:32:31,888][105620] Updated weights for policy 1, policy_version 848916 (0.0010) [2023-12-26 21:32:32,190][105692] Updated weights for policy 0, policy_version 848814 (0.0008) [2023-12-26 21:32:32,242][105692] Updated weights for policy 0, policy_version 848824 (0.0008) [2023-12-26 21:32:32,298][105692] Updated weights for policy 0, policy_version 848834 (0.0008) [2023-12-26 21:32:32,614][105620] Updated weights for policy 1, policy_version 848926 (0.0010) [2023-12-26 21:32:32,673][105620] Updated weights for policy 1, policy_version 848936 (0.0010) [2023-12-26 21:32:32,728][105620] Updated weights for policy 1, policy_version 848946 (0.0010) [2023-12-26 21:32:33,106][105692] Updated weights for policy 0, policy_version 848844 (0.0008) [2023-12-26 21:32:33,154][105692] Updated weights for policy 0, policy_version 848854 (0.0008) [2023-12-26 21:32:33,206][105692] Updated weights for policy 0, policy_version 848864 (0.0008) [2023-12-26 21:32:33,464][105620] Updated weights for policy 1, policy_version 848956 (0.0010) [2023-12-26 21:32:33,508][105620] Updated weights for policy 1, policy_version 848966 (0.0010) [2023-12-26 21:32:33,552][105620] Updated weights for policy 1, policy_version 848976 (0.0010) [2023-12-26 21:32:33,970][105692] Updated weights for policy 0, policy_version 848874 (0.0008) [2023-12-26 21:32:34,027][105692] Updated weights for policy 0, policy_version 848884 (0.0008) [2023-12-26 21:32:34,085][105692] Updated weights for policy 0, policy_version 848894 (0.0008) [2023-12-26 21:32:34,133][105692] Updated weights for policy 0, policy_version 848904 (0.0008) [2023-12-26 21:32:34,320][105620] Updated weights for policy 1, policy_version 848986 (0.0010) [2023-12-26 21:32:34,386][105620] Updated weights for policy 1, policy_version 848996 (0.0011) [2023-12-26 21:32:34,438][105620] Updated weights for policy 1, policy_version 849006 (0.0011) [2023-12-26 21:32:34,497][105620] Updated weights for policy 1, policy_version 849016 (0.0010) [2023-12-26 21:32:34,872][105692] Updated weights for policy 0, policy_version 848914 (0.0008) [2023-12-26 21:32:34,925][105692] Updated weights for policy 0, policy_version 848924 (0.0006) [2023-12-26 21:32:34,975][105692] Updated weights for policy 0, policy_version 848934 (0.0008) [2023-12-26 21:32:35,252][105620] Updated weights for policy 1, policy_version 849026 (0.0010) [2023-12-26 21:32:35,302][105620] Updated weights for policy 1, policy_version 849036 (0.0010) [2023-12-26 21:32:35,356][105620] Updated weights for policy 1, policy_version 849046 (0.0010) [2023-12-26 21:32:35,684][105692] Updated weights for policy 0, policy_version 848944 (0.0008) [2023-12-26 21:32:35,741][105692] Updated weights for policy 0, policy_version 848954 (0.0008) [2023-12-26 21:32:35,799][105692] Updated weights for policy 0, policy_version 848964 (0.0009) [2023-12-26 21:32:36,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 434749440. Throughput: 0: 9745.9, 1: 9829.9. Samples: 434739792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:32:36,063][104569] Avg episode reward: [(0, '9071.659'), (1, '9077.732')] [2023-12-26 21:32:36,076][105620] Updated weights for policy 1, policy_version 849056 (0.0009) [2023-12-26 21:32:36,135][105620] Updated weights for policy 1, policy_version 849066 (0.0008) [2023-12-26 21:32:36,186][105620] Updated weights for policy 1, policy_version 849076 (0.0008) [2023-12-26 21:32:36,541][105692] Updated weights for policy 0, policy_version 848974 (0.0010) [2023-12-26 21:32:36,593][105692] Updated weights for policy 0, policy_version 848984 (0.0007) [2023-12-26 21:32:36,652][105692] Updated weights for policy 0, policy_version 848994 (0.0009) [2023-12-26 21:32:37,036][105620] Updated weights for policy 1, policy_version 849086 (0.0009) [2023-12-26 21:32:37,100][105620] Updated weights for policy 1, policy_version 849096 (0.0009) [2023-12-26 21:32:37,148][105620] Updated weights for policy 1, policy_version 849106 (0.0009) [2023-12-26 21:32:37,347][105692] Updated weights for policy 0, policy_version 849004 (0.0010) [2023-12-26 21:32:37,401][105692] Updated weights for policy 0, policy_version 849014 (0.0009) [2023-12-26 21:32:37,449][105692] Updated weights for policy 0, policy_version 849024 (0.0009) [2023-12-26 21:32:37,903][105620] Updated weights for policy 1, policy_version 849116 (0.0008) [2023-12-26 21:32:37,952][105620] Updated weights for policy 1, policy_version 849126 (0.0011) [2023-12-26 21:32:38,007][105620] Updated weights for policy 1, policy_version 849137 (0.0011) [2023-12-26 21:32:38,116][105692] Updated weights for policy 0, policy_version 849034 (0.0008) [2023-12-26 21:32:38,172][105692] Updated weights for policy 0, policy_version 849044 (0.0009) [2023-12-26 21:32:38,240][105692] Updated weights for policy 0, policy_version 849054 (0.0009) [2023-12-26 21:32:38,297][105692] Updated weights for policy 0, policy_version 849064 (0.0007) [2023-12-26 21:32:38,671][105620] Updated weights for policy 1, policy_version 849147 (0.0009) [2023-12-26 21:32:38,735][105620] Updated weights for policy 1, policy_version 849157 (0.0011) [2023-12-26 21:32:38,791][105620] Updated weights for policy 1, policy_version 849167 (0.0010) [2023-12-26 21:32:38,988][105692] Updated weights for policy 0, policy_version 849074 (0.0011) [2023-12-26 21:32:39,051][105692] Updated weights for policy 0, policy_version 849084 (0.0008) [2023-12-26 21:32:39,111][105692] Updated weights for policy 0, policy_version 849094 (0.0006) [2023-12-26 21:32:39,373][105620] Updated weights for policy 1, policy_version 849177 (0.0006) [2023-12-26 21:32:39,440][105620] Updated weights for policy 1, policy_version 849187 (0.0010) [2023-12-26 21:32:39,493][105620] Updated weights for policy 1, policy_version 849197 (0.0010) [2023-12-26 21:32:39,553][105620] Updated weights for policy 1, policy_version 849207 (0.0010) [2023-12-26 21:32:39,754][105692] Updated weights for policy 0, policy_version 849104 (0.0009) [2023-12-26 21:32:39,815][105692] Updated weights for policy 0, policy_version 849114 (0.0011) [2023-12-26 21:32:39,881][105692] Updated weights for policy 0, policy_version 849124 (0.0008) [2023-12-26 21:32:40,337][105620] Updated weights for policy 1, policy_version 849217 (0.0009) [2023-12-26 21:32:40,401][105620] Updated weights for policy 1, policy_version 849227 (0.0008) [2023-12-26 21:32:40,455][105620] Updated weights for policy 1, policy_version 849237 (0.0008) [2023-12-26 21:32:40,654][105692] Updated weights for policy 0, policy_version 849134 (0.0011) [2023-12-26 21:32:40,706][105692] Updated weights for policy 0, policy_version 849144 (0.0010) [2023-12-26 21:32:40,765][105692] Updated weights for policy 0, policy_version 849154 (0.0011) [2023-12-26 21:32:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 434847744. Throughput: 0: 9760.6, 1: 9910.2. Samples: 434856680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:32:41,062][104569] Avg episode reward: [(0, '8933.088'), (1, '9170.088')] [2023-12-26 21:32:41,234][105620] Updated weights for policy 1, policy_version 849247 (0.0008) [2023-12-26 21:32:41,303][105620] Updated weights for policy 1, policy_version 849257 (0.0008) [2023-12-26 21:32:41,371][105620] Updated weights for policy 1, policy_version 849267 (0.0008) [2023-12-26 21:32:41,521][105692] Updated weights for policy 0, policy_version 849164 (0.0010) [2023-12-26 21:32:41,580][105692] Updated weights for policy 0, policy_version 849174 (0.0011) [2023-12-26 21:32:41,649][105692] Updated weights for policy 0, policy_version 849184 (0.0009) [2023-12-26 21:32:42,137][105620] Updated weights for policy 1, policy_version 849277 (0.0007) [2023-12-26 21:32:42,207][105620] Updated weights for policy 1, policy_version 849287 (0.0006) [2023-12-26 21:32:42,262][105620] Updated weights for policy 1, policy_version 849297 (0.0006) [2023-12-26 21:32:42,336][105692] Updated weights for policy 0, policy_version 849194 (0.0007) [2023-12-26 21:32:42,397][105692] Updated weights for policy 0, policy_version 849204 (0.0007) [2023-12-26 21:32:42,462][105692] Updated weights for policy 0, policy_version 849214 (0.0006) [2023-12-26 21:32:42,521][105692] Updated weights for policy 0, policy_version 849224 (0.0007) [2023-12-26 21:32:42,966][105620] Updated weights for policy 1, policy_version 849307 (0.0010) [2023-12-26 21:32:43,028][105620] Updated weights for policy 1, policy_version 849317 (0.0010) [2023-12-26 21:32:43,083][105620] Updated weights for policy 1, policy_version 849327 (0.0010) [2023-12-26 21:32:43,191][105692] Updated weights for policy 0, policy_version 849234 (0.0008) [2023-12-26 21:32:43,259][105692] Updated weights for policy 0, policy_version 849244 (0.0005) [2023-12-26 21:32:43,323][105692] Updated weights for policy 0, policy_version 849254 (0.0005) [2023-12-26 21:32:43,747][105620] Updated weights for policy 1, policy_version 849337 (0.0010) [2023-12-26 21:32:43,795][105620] Updated weights for policy 1, policy_version 849347 (0.0008) [2023-12-26 21:32:43,839][105620] Updated weights for policy 1, policy_version 849357 (0.0008) [2023-12-26 21:32:43,888][105620] Updated weights for policy 1, policy_version 849367 (0.0008) [2023-12-26 21:32:43,955][105692] Updated weights for policy 0, policy_version 849264 (0.0009) [2023-12-26 21:32:44,013][105692] Updated weights for policy 0, policy_version 849274 (0.0010) [2023-12-26 21:32:44,069][105692] Updated weights for policy 0, policy_version 849284 (0.0010) [2023-12-26 21:32:44,665][105620] Updated weights for policy 1, policy_version 849377 (0.0008) [2023-12-26 21:32:44,712][105620] Updated weights for policy 1, policy_version 849387 (0.0007) [2023-12-26 21:32:44,763][105620] Updated weights for policy 1, policy_version 849397 (0.0008) [2023-12-26 21:32:44,818][105692] Updated weights for policy 0, policy_version 849294 (0.0010) [2023-12-26 21:32:44,870][105692] Updated weights for policy 0, policy_version 849304 (0.0011) [2023-12-26 21:32:44,931][105692] Updated weights for policy 0, policy_version 849314 (0.0011) [2023-12-26 21:32:45,560][105620] Updated weights for policy 1, policy_version 849407 (0.0009) [2023-12-26 21:32:45,615][105620] Updated weights for policy 1, policy_version 849417 (0.0008) [2023-12-26 21:32:45,673][105620] Updated weights for policy 1, policy_version 849427 (0.0008) [2023-12-26 21:32:45,685][105692] Updated weights for policy 0, policy_version 849324 (0.0011) [2023-12-26 21:32:45,747][105692] Updated weights for policy 0, policy_version 849334 (0.0010) [2023-12-26 21:32:45,805][105692] Updated weights for policy 0, policy_version 849344 (0.0010) [2023-12-26 21:32:46,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 434946048. Throughput: 0: 9769.0, 1: 9864.4. Samples: 434915012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:32:46,062][104569] Avg episode reward: [(0, '8752.421'), (1, '9078.951')] [2023-12-26 21:32:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000849352_217464832.pth... [2023-12-26 21:32:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000849432_217481216.pth... [2023-12-26 21:32:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000848232_217178112.pth [2023-12-26 21:32:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000848312_217194496.pth [2023-12-26 21:32:46,443][105620] Updated weights for policy 1, policy_version 849437 (0.0007) [2023-12-26 21:32:46,494][105620] Updated weights for policy 1, policy_version 849447 (0.0005) [2023-12-26 21:32:46,546][105620] Updated weights for policy 1, policy_version 849457 (0.0008) [2023-12-26 21:32:46,551][105692] Updated weights for policy 0, policy_version 849354 (0.0010) [2023-12-26 21:32:46,609][105692] Updated weights for policy 0, policy_version 849364 (0.0010) [2023-12-26 21:32:46,673][105692] Updated weights for policy 0, policy_version 849374 (0.0010) [2023-12-26 21:32:46,729][105692] Updated weights for policy 0, policy_version 849384 (0.0010) [2023-12-26 21:32:47,133][105620] Updated weights for policy 1, policy_version 849467 (0.0009) [2023-12-26 21:32:47,187][105620] Updated weights for policy 1, policy_version 849477 (0.0006) [2023-12-26 21:32:47,238][105620] Updated weights for policy 1, policy_version 849487 (0.0005) [2023-12-26 21:32:47,465][105692] Updated weights for policy 0, policy_version 849394 (0.0010) [2023-12-26 21:32:47,520][105692] Updated weights for policy 0, policy_version 849404 (0.0010) [2023-12-26 21:32:47,574][105692] Updated weights for policy 0, policy_version 849414 (0.0010) [2023-12-26 21:32:47,897][105620] Updated weights for policy 1, policy_version 849497 (0.0006) [2023-12-26 21:32:47,949][105620] Updated weights for policy 1, policy_version 849507 (0.0010) [2023-12-26 21:32:48,004][105620] Updated weights for policy 1, policy_version 849517 (0.0010) [2023-12-26 21:32:48,063][105620] Updated weights for policy 1, policy_version 849527 (0.0010) [2023-12-26 21:32:48,248][105692] Updated weights for policy 0, policy_version 849424 (0.0006) [2023-12-26 21:32:48,308][105692] Updated weights for policy 0, policy_version 849434 (0.0005) [2023-12-26 21:32:48,381][105692] Updated weights for policy 0, policy_version 849444 (0.0010) [2023-12-26 21:32:48,835][105620] Updated weights for policy 1, policy_version 849537 (0.0008) [2023-12-26 21:32:48,901][105620] Updated weights for policy 1, policy_version 849547 (0.0010) [2023-12-26 21:32:48,966][105620] Updated weights for policy 1, policy_version 849557 (0.0011) [2023-12-26 21:32:49,102][105692] Updated weights for policy 0, policy_version 849454 (0.0011) [2023-12-26 21:32:49,162][105692] Updated weights for policy 0, policy_version 849464 (0.0011) [2023-12-26 21:32:49,230][105692] Updated weights for policy 0, policy_version 849474 (0.0011) [2023-12-26 21:32:49,679][105620] Updated weights for policy 1, policy_version 849567 (0.0011) [2023-12-26 21:32:49,742][105620] Updated weights for policy 1, policy_version 849577 (0.0010) [2023-12-26 21:32:49,797][105620] Updated weights for policy 1, policy_version 849587 (0.0010) [2023-12-26 21:32:49,952][105692] Updated weights for policy 0, policy_version 849484 (0.0011) [2023-12-26 21:32:50,013][105692] Updated weights for policy 0, policy_version 849494 (0.0011) [2023-12-26 21:32:50,075][105692] Updated weights for policy 0, policy_version 849504 (0.0010) [2023-12-26 21:32:50,568][105620] Updated weights for policy 1, policy_version 849597 (0.0010) [2023-12-26 21:32:50,633][105620] Updated weights for policy 1, policy_version 849607 (0.0010) [2023-12-26 21:32:50,699][105620] Updated weights for policy 1, policy_version 849617 (0.0011) [2023-12-26 21:32:50,804][105692] Updated weights for policy 0, policy_version 849514 (0.0011) [2023-12-26 21:32:50,856][105692] Updated weights for policy 0, policy_version 849524 (0.0010) [2023-12-26 21:32:50,915][105692] Updated weights for policy 0, policy_version 849534 (0.0010) [2023-12-26 21:32:50,977][105692] Updated weights for policy 0, policy_version 849544 (0.0008) [2023-12-26 21:32:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 435044352. Throughput: 0: 9725.6, 1: 9804.7. Samples: 435030240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:32:51,062][104569] Avg episode reward: [(0, '8787.677'), (1, '9078.380')] [2023-12-26 21:32:51,461][105620] Updated weights for policy 1, policy_version 849627 (0.0011) [2023-12-26 21:32:51,510][105620] Updated weights for policy 1, policy_version 849637 (0.0010) [2023-12-26 21:32:51,565][105620] Updated weights for policy 1, policy_version 849647 (0.0011) [2023-12-26 21:32:51,605][105692] Updated weights for policy 0, policy_version 849554 (0.0006) [2023-12-26 21:32:51,672][105692] Updated weights for policy 0, policy_version 849564 (0.0009) [2023-12-26 21:32:51,733][105692] Updated weights for policy 0, policy_version 849574 (0.0010) [2023-12-26 21:32:52,334][105620] Updated weights for policy 1, policy_version 849657 (0.0011) [2023-12-26 21:32:52,371][105692] Updated weights for policy 0, policy_version 849584 (0.0007) [2023-12-26 21:32:52,395][105620] Updated weights for policy 1, policy_version 849667 (0.0010) [2023-12-26 21:32:52,429][105692] Updated weights for policy 0, policy_version 849594 (0.0006) [2023-12-26 21:32:52,448][105620] Updated weights for policy 1, policy_version 849677 (0.0010) [2023-12-26 21:32:52,490][105692] Updated weights for policy 0, policy_version 849604 (0.0005) [2023-12-26 21:32:52,508][105620] Updated weights for policy 1, policy_version 849687 (0.0011) [2023-12-26 21:32:53,188][105620] Updated weights for policy 1, policy_version 849697 (0.0007) [2023-12-26 21:32:53,190][105692] Updated weights for policy 0, policy_version 849614 (0.0008) [2023-12-26 21:32:53,242][105620] Updated weights for policy 1, policy_version 849707 (0.0005) [2023-12-26 21:32:53,244][105692] Updated weights for policy 0, policy_version 849624 (0.0009) [2023-12-26 21:32:53,294][105620] Updated weights for policy 1, policy_version 849717 (0.0005) [2023-12-26 21:32:53,307][105692] Updated weights for policy 0, policy_version 849634 (0.0009) [2023-12-26 21:32:53,892][105692] Updated weights for policy 0, policy_version 849644 (0.0007) [2023-12-26 21:32:53,951][105692] Updated weights for policy 0, policy_version 849654 (0.0009) [2023-12-26 21:32:54,012][105692] Updated weights for policy 0, policy_version 849664 (0.0009) [2023-12-26 21:32:54,032][105620] Updated weights for policy 1, policy_version 849727 (0.0008) [2023-12-26 21:32:54,084][105620] Updated weights for policy 1, policy_version 849737 (0.0007) [2023-12-26 21:32:54,145][105620] Updated weights for policy 1, policy_version 849747 (0.0007) [2023-12-26 21:32:54,777][105692] Updated weights for policy 0, policy_version 849674 (0.0008) [2023-12-26 21:32:54,826][105620] Updated weights for policy 1, policy_version 849757 (0.0006) [2023-12-26 21:32:54,828][105692] Updated weights for policy 0, policy_version 849684 (0.0005) [2023-12-26 21:32:54,883][105692] Updated weights for policy 0, policy_version 849694 (0.0006) [2023-12-26 21:32:54,888][105620] Updated weights for policy 1, policy_version 849767 (0.0011) [2023-12-26 21:32:54,935][105692] Updated weights for policy 0, policy_version 849704 (0.0006) [2023-12-26 21:32:54,945][105620] Updated weights for policy 1, policy_version 849777 (0.0011) [2023-12-26 21:32:55,690][105620] Updated weights for policy 1, policy_version 849787 (0.0010) [2023-12-26 21:32:55,697][105692] Updated weights for policy 0, policy_version 849714 (0.0007) [2023-12-26 21:32:55,746][105620] Updated weights for policy 1, policy_version 849797 (0.0011) [2023-12-26 21:32:55,756][105692] Updated weights for policy 0, policy_version 849724 (0.0005) [2023-12-26 21:32:55,794][105620] Updated weights for policy 1, policy_version 849807 (0.0010) [2023-12-26 21:32:55,804][105692] Updated weights for policy 0, policy_version 849734 (0.0005) [2023-12-26 21:32:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 435142656. Throughput: 0: 9728.7, 1: 9726.7. Samples: 435147620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:32:56,062][104569] Avg episode reward: [(0, '8790.290'), (1, '9031.836')] [2023-12-26 21:32:56,492][105692] Updated weights for policy 0, policy_version 849744 (0.0007) [2023-12-26 21:32:56,532][105620] Updated weights for policy 1, policy_version 849817 (0.0010) [2023-12-26 21:32:56,543][105692] Updated weights for policy 0, policy_version 849754 (0.0007) [2023-12-26 21:32:56,584][105620] Updated weights for policy 1, policy_version 849827 (0.0010) [2023-12-26 21:32:56,602][105692] Updated weights for policy 0, policy_version 849764 (0.0006) [2023-12-26 21:32:56,636][105620] Updated weights for policy 1, policy_version 849837 (0.0010) [2023-12-26 21:32:56,688][105620] Updated weights for policy 1, policy_version 849847 (0.0010) [2023-12-26 21:32:57,176][105692] Updated weights for policy 0, policy_version 849774 (0.0005) [2023-12-26 21:32:57,222][105692] Updated weights for policy 0, policy_version 849784 (0.0005) [2023-12-26 21:32:57,270][105692] Updated weights for policy 0, policy_version 849794 (0.0005) [2023-12-26 21:32:57,339][105620] Updated weights for policy 1, policy_version 849857 (0.0010) [2023-12-26 21:32:57,400][105620] Updated weights for policy 1, policy_version 849867 (0.0010) [2023-12-26 21:32:57,464][105620] Updated weights for policy 1, policy_version 849877 (0.0010) [2023-12-26 21:32:57,972][105692] Updated weights for policy 0, policy_version 849804 (0.0006) [2023-12-26 21:32:58,027][105692] Updated weights for policy 0, policy_version 849814 (0.0006) [2023-12-26 21:32:58,088][105692] Updated weights for policy 0, policy_version 849824 (0.0005) [2023-12-26 21:32:58,195][105620] Updated weights for policy 1, policy_version 849887 (0.0010) [2023-12-26 21:32:58,251][105620] Updated weights for policy 1, policy_version 849897 (0.0011) [2023-12-26 21:32:58,314][105620] Updated weights for policy 1, policy_version 849907 (0.0011) [2023-12-26 21:32:58,840][105692] Updated weights for policy 0, policy_version 849834 (0.0008) [2023-12-26 21:32:58,911][105692] Updated weights for policy 0, policy_version 849844 (0.0009) [2023-12-26 21:32:58,968][105692] Updated weights for policy 0, policy_version 849854 (0.0009) [2023-12-26 21:32:59,028][105692] Updated weights for policy 0, policy_version 849864 (0.0008) [2023-12-26 21:32:59,129][105620] Updated weights for policy 1, policy_version 849917 (0.0010) [2023-12-26 21:32:59,188][105620] Updated weights for policy 1, policy_version 849927 (0.0008) [2023-12-26 21:32:59,278][105620] Updated weights for policy 1, policy_version 849937 (0.0009) [2023-12-26 21:32:59,804][105692] Updated weights for policy 0, policy_version 849874 (0.0009) [2023-12-26 21:32:59,857][105692] Updated weights for policy 0, policy_version 849884 (0.0009) [2023-12-26 21:32:59,916][105692] Updated weights for policy 0, policy_version 849894 (0.0009) [2023-12-26 21:33:00,042][105620] Updated weights for policy 1, policy_version 849947 (0.0010) [2023-12-26 21:33:00,103][105620] Updated weights for policy 1, policy_version 849957 (0.0005) [2023-12-26 21:33:00,163][105620] Updated weights for policy 1, policy_version 849967 (0.0005) [2023-12-26 21:33:00,647][105692] Updated weights for policy 0, policy_version 849905 (0.0010) [2023-12-26 21:33:00,700][105692] Updated weights for policy 0, policy_version 849916 (0.0010) [2023-12-26 21:33:00,744][105620] Updated weights for policy 1, policy_version 849977 (0.0006) [2023-12-26 21:33:00,753][105692] Updated weights for policy 0, policy_version 849926 (0.0006) [2023-12-26 21:33:00,795][105620] Updated weights for policy 1, policy_version 849987 (0.0010) [2023-12-26 21:33:00,839][105620] Updated weights for policy 1, policy_version 849997 (0.0010) [2023-12-26 21:33:00,894][105620] Updated weights for policy 1, policy_version 850007 (0.0010) [2023-12-26 21:33:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 435240960. Throughput: 0: 9763.1, 1: 9705.2. Samples: 435207660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:33:01,063][104569] Avg episode reward: [(0, '9253.619'), (1, '8940.076')] [2023-12-26 21:33:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000850008_217628672.pth... [2023-12-26 21:33:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000849928_217612288.pth... [2023-12-26 21:33:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000848888_217341952.pth [2023-12-26 21:33:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000848776_217317376.pth [2023-12-26 21:33:01,523][105692] Updated weights for policy 0, policy_version 849936 (0.0010) [2023-12-26 21:33:01,577][105692] Updated weights for policy 0, policy_version 849946 (0.0009) [2023-12-26 21:33:01,641][105692] Updated weights for policy 0, policy_version 849956 (0.0009) [2023-12-26 21:33:01,667][105620] Updated weights for policy 1, policy_version 850017 (0.0009) [2023-12-26 21:33:01,727][105620] Updated weights for policy 1, policy_version 850027 (0.0011) [2023-12-26 21:33:01,789][105620] Updated weights for policy 1, policy_version 850037 (0.0011) [2023-12-26 21:33:02,338][105692] Updated weights for policy 0, policy_version 849966 (0.0009) [2023-12-26 21:33:02,408][105692] Updated weights for policy 0, policy_version 849976 (0.0005) [2023-12-26 21:33:02,471][105692] Updated weights for policy 0, policy_version 849986 (0.0007) [2023-12-26 21:33:02,473][105620] Updated weights for policy 1, policy_version 850047 (0.0010) [2023-12-26 21:33:02,529][105620] Updated weights for policy 1, policy_version 850057 (0.0010) [2023-12-26 21:33:02,581][105620] Updated weights for policy 1, policy_version 850067 (0.0010) [2023-12-26 21:33:03,084][105692] Updated weights for policy 0, policy_version 849996 (0.0007) [2023-12-26 21:33:03,143][105692] Updated weights for policy 0, policy_version 850006 (0.0010) [2023-12-26 21:33:03,198][105620] Updated weights for policy 1, policy_version 850077 (0.0010) [2023-12-26 21:33:03,201][105692] Updated weights for policy 0, policy_version 850016 (0.0010) [2023-12-26 21:33:03,259][105620] Updated weights for policy 1, policy_version 850087 (0.0010) [2023-12-26 21:33:03,314][105620] Updated weights for policy 1, policy_version 850097 (0.0008) [2023-12-26 21:33:03,768][105692] Updated weights for policy 0, policy_version 850026 (0.0009) [2023-12-26 21:33:03,823][105692] Updated weights for policy 0, policy_version 850036 (0.0005) [2023-12-26 21:33:03,890][105692] Updated weights for policy 0, policy_version 850046 (0.0007) [2023-12-26 21:33:03,944][105692] Updated weights for policy 0, policy_version 850056 (0.0005) [2023-12-26 21:33:03,979][105620] Updated weights for policy 1, policy_version 850107 (0.0006) [2023-12-26 21:33:04,035][105620] Updated weights for policy 1, policy_version 850117 (0.0011) [2023-12-26 21:33:04,098][105620] Updated weights for policy 1, policy_version 850127 (0.0011) [2023-12-26 21:33:04,558][105692] Updated weights for policy 0, policy_version 850066 (0.0008) [2023-12-26 21:33:04,619][105692] Updated weights for policy 0, policy_version 850076 (0.0010) [2023-12-26 21:33:04,678][105692] Updated weights for policy 0, policy_version 850086 (0.0010) [2023-12-26 21:33:04,776][105620] Updated weights for policy 1, policy_version 850137 (0.0010) [2023-12-26 21:33:04,838][105620] Updated weights for policy 1, policy_version 850147 (0.0005) [2023-12-26 21:33:04,894][105620] Updated weights for policy 1, policy_version 850157 (0.0005) [2023-12-26 21:33:04,963][105620] Updated weights for policy 1, policy_version 850167 (0.0005) [2023-12-26 21:33:05,355][105692] Updated weights for policy 0, policy_version 850096 (0.0006) [2023-12-26 21:33:05,413][105692] Updated weights for policy 0, policy_version 850106 (0.0005) [2023-12-26 21:33:05,469][105692] Updated weights for policy 0, policy_version 850116 (0.0010) [2023-12-26 21:33:05,508][105620] Updated weights for policy 1, policy_version 850177 (0.0005) [2023-12-26 21:33:05,552][105620] Updated weights for policy 1, policy_version 850187 (0.0005) [2023-12-26 21:33:05,595][105620] Updated weights for policy 1, policy_version 850197 (0.0005) [2023-12-26 21:33:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 435339264. Throughput: 0: 9783.4, 1: 9770.0. Samples: 435328844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:33:06,063][104569] Avg episode reward: [(0, '8840.938'), (1, '8924.199')] [2023-12-26 21:33:06,085][105692] Updated weights for policy 0, policy_version 850126 (0.0007) [2023-12-26 21:33:06,148][105692] Updated weights for policy 0, policy_version 850136 (0.0010) [2023-12-26 21:33:06,214][105620] Updated weights for policy 1, policy_version 850207 (0.0007) [2023-12-26 21:33:06,229][105692] Updated weights for policy 0, policy_version 850146 (0.0008) [2023-12-26 21:33:06,278][105620] Updated weights for policy 1, policy_version 850217 (0.0009) [2023-12-26 21:33:06,345][105620] Updated weights for policy 1, policy_version 850227 (0.0009) [2023-12-26 21:33:06,934][105692] Updated weights for policy 0, policy_version 850156 (0.0008) [2023-12-26 21:33:06,989][105692] Updated weights for policy 0, policy_version 850166 (0.0009) [2023-12-26 21:33:07,047][105692] Updated weights for policy 0, policy_version 850176 (0.0010) [2023-12-26 21:33:07,071][105620] Updated weights for policy 1, policy_version 850237 (0.0007) [2023-12-26 21:33:07,124][105620] Updated weights for policy 1, policy_version 850247 (0.0007) [2023-12-26 21:33:07,185][105620] Updated weights for policy 1, policy_version 850257 (0.0009) [2023-12-26 21:33:07,835][105692] Updated weights for policy 0, policy_version 850187 (0.0009) [2023-12-26 21:33:07,890][105692] Updated weights for policy 0, policy_version 850197 (0.0006) [2023-12-26 21:33:07,936][105692] Updated weights for policy 0, policy_version 850207 (0.0008) [2023-12-26 21:33:07,939][105620] Updated weights for policy 1, policy_version 850267 (0.0008) [2023-12-26 21:33:07,992][105620] Updated weights for policy 1, policy_version 850277 (0.0007) [2023-12-26 21:33:08,045][105620] Updated weights for policy 1, policy_version 850287 (0.0008) [2023-12-26 21:33:08,609][105692] Updated weights for policy 0, policy_version 850217 (0.0007) [2023-12-26 21:33:08,664][105692] Updated weights for policy 0, policy_version 850227 (0.0009) [2023-12-26 21:33:08,720][105692] Updated weights for policy 0, policy_version 850237 (0.0009) [2023-12-26 21:33:08,772][105692] Updated weights for policy 0, policy_version 850247 (0.0007) [2023-12-26 21:33:08,800][105620] Updated weights for policy 1, policy_version 850297 (0.0009) [2023-12-26 21:33:08,855][105620] Updated weights for policy 1, policy_version 850307 (0.0008) [2023-12-26 21:33:08,908][105620] Updated weights for policy 1, policy_version 850317 (0.0008) [2023-12-26 21:33:08,967][105620] Updated weights for policy 1, policy_version 850327 (0.0008) [2023-12-26 21:33:09,549][105692] Updated weights for policy 0, policy_version 850257 (0.0010) [2023-12-26 21:33:09,608][105692] Updated weights for policy 0, policy_version 850267 (0.0010) [2023-12-26 21:33:09,676][105692] Updated weights for policy 0, policy_version 850277 (0.0011) [2023-12-26 21:33:09,741][105620] Updated weights for policy 1, policy_version 850337 (0.0009) [2023-12-26 21:33:09,798][105620] Updated weights for policy 1, policy_version 850347 (0.0008) [2023-12-26 21:33:09,867][105620] Updated weights for policy 1, policy_version 850357 (0.0009) [2023-12-26 21:33:10,439][105692] Updated weights for policy 0, policy_version 850287 (0.0008) [2023-12-26 21:33:10,497][105692] Updated weights for policy 0, policy_version 850297 (0.0005) [2023-12-26 21:33:10,555][105692] Updated weights for policy 0, policy_version 850307 (0.0006) [2023-12-26 21:33:10,626][105620] Updated weights for policy 1, policy_version 850367 (0.0010) [2023-12-26 21:33:10,684][105620] Updated weights for policy 1, policy_version 850377 (0.0010) [2023-12-26 21:33:10,744][105620] Updated weights for policy 1, policy_version 850387 (0.0008) [2023-12-26 21:33:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 435437568. Throughput: 0: 9809.9, 1: 9782.3. Samples: 435445328. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:33:11,063][104569] Avg episode reward: [(0, '8839.945'), (1, '9016.472')] [2023-12-26 21:33:11,156][105692] Updated weights for policy 0, policy_version 850317 (0.0010) [2023-12-26 21:33:11,218][105692] Updated weights for policy 0, policy_version 850327 (0.0011) [2023-12-26 21:33:11,282][105692] Updated weights for policy 0, policy_version 850337 (0.0011) [2023-12-26 21:33:11,545][105620] Updated weights for policy 1, policy_version 850397 (0.0008) [2023-12-26 21:33:11,600][105620] Updated weights for policy 1, policy_version 850407 (0.0008) [2023-12-26 21:33:11,663][105620] Updated weights for policy 1, policy_version 850417 (0.0007) [2023-12-26 21:33:12,068][105692] Updated weights for policy 0, policy_version 850347 (0.0011) [2023-12-26 21:33:12,121][105692] Updated weights for policy 0, policy_version 850357 (0.0011) [2023-12-26 21:33:12,181][105692] Updated weights for policy 0, policy_version 850367 (0.0011) [2023-12-26 21:33:12,459][105620] Updated weights for policy 1, policy_version 850427 (0.0008) [2023-12-26 21:33:12,519][105620] Updated weights for policy 1, policy_version 850437 (0.0009) [2023-12-26 21:33:12,574][105620] Updated weights for policy 1, policy_version 850447 (0.0008) [2023-12-26 21:33:12,945][105692] Updated weights for policy 0, policy_version 850377 (0.0011) [2023-12-26 21:33:12,990][105692] Updated weights for policy 0, policy_version 850387 (0.0010) [2023-12-26 21:33:13,038][105692] Updated weights for policy 0, policy_version 850397 (0.0010) [2023-12-26 21:33:13,082][105692] Updated weights for policy 0, policy_version 850407 (0.0010) [2023-12-26 21:33:13,338][105620] Updated weights for policy 1, policy_version 850457 (0.0008) [2023-12-26 21:33:13,385][105620] Updated weights for policy 1, policy_version 850467 (0.0008) [2023-12-26 21:33:13,444][105620] Updated weights for policy 1, policy_version 850477 (0.0008) [2023-12-26 21:33:13,488][105620] Updated weights for policy 1, policy_version 850487 (0.0008) [2023-12-26 21:33:13,872][105692] Updated weights for policy 0, policy_version 850417 (0.0010) [2023-12-26 21:33:13,923][105692] Updated weights for policy 0, policy_version 850427 (0.0010) [2023-12-26 21:33:13,971][105692] Updated weights for policy 0, policy_version 850437 (0.0010) [2023-12-26 21:33:14,263][105620] Updated weights for policy 1, policy_version 850497 (0.0008) [2023-12-26 21:33:14,319][105620] Updated weights for policy 1, policy_version 850507 (0.0008) [2023-12-26 21:33:14,370][105620] Updated weights for policy 1, policy_version 850517 (0.0008) [2023-12-26 21:33:14,720][105692] Updated weights for policy 0, policy_version 850447 (0.0010) [2023-12-26 21:33:14,772][105692] Updated weights for policy 0, policy_version 850457 (0.0010) [2023-12-26 21:33:14,834][105692] Updated weights for policy 0, policy_version 850467 (0.0006) [2023-12-26 21:33:15,117][105620] Updated weights for policy 1, policy_version 850527 (0.0007) [2023-12-26 21:33:15,187][105620] Updated weights for policy 1, policy_version 850537 (0.0006) [2023-12-26 21:33:15,252][105620] Updated weights for policy 1, policy_version 850547 (0.0010) [2023-12-26 21:33:15,439][105692] Updated weights for policy 0, policy_version 850477 (0.0008) [2023-12-26 21:33:15,498][105692] Updated weights for policy 0, policy_version 850487 (0.0011) [2023-12-26 21:33:15,557][105692] Updated weights for policy 0, policy_version 850497 (0.0011) [2023-12-26 21:33:15,872][105620] Updated weights for policy 1, policy_version 850557 (0.0011) [2023-12-26 21:33:15,937][105620] Updated weights for policy 1, policy_version 850567 (0.0010) [2023-12-26 21:33:15,999][105620] Updated weights for policy 1, policy_version 850577 (0.0010) [2023-12-26 21:33:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 435535872. Throughput: 0: 9804.4, 1: 9645.0. Samples: 435500892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:33:16,062][104569] Avg episode reward: [(0, '9163.638'), (1, '9016.910')] [2023-12-26 21:33:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000850584_217776128.pth... [2023-12-26 21:33:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000850504_217759744.pth... [2023-12-26 21:33:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000849352_217464832.pth [2023-12-26 21:33:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000849432_217481216.pth [2023-12-26 21:33:16,278][105692] Updated weights for policy 0, policy_version 850507 (0.0011) [2023-12-26 21:33:16,327][105692] Updated weights for policy 0, policy_version 850517 (0.0010) [2023-12-26 21:33:16,378][105692] Updated weights for policy 0, policy_version 850527 (0.0010) [2023-12-26 21:33:16,606][105620] Updated weights for policy 1, policy_version 850587 (0.0009) [2023-12-26 21:33:16,662][105620] Updated weights for policy 1, policy_version 850597 (0.0005) [2023-12-26 21:33:16,714][105620] Updated weights for policy 1, policy_version 850607 (0.0007) [2023-12-26 21:33:17,145][105692] Updated weights for policy 0, policy_version 850537 (0.0010) [2023-12-26 21:33:17,195][105692] Updated weights for policy 0, policy_version 850547 (0.0011) [2023-12-26 21:33:17,244][105692] Updated weights for policy 0, policy_version 850557 (0.0011) [2023-12-26 21:33:17,284][105620] Updated weights for policy 1, policy_version 850617 (0.0010) [2023-12-26 21:33:17,296][105692] Updated weights for policy 0, policy_version 850567 (0.0011) [2023-12-26 21:33:17,345][105620] Updated weights for policy 1, policy_version 850627 (0.0009) [2023-12-26 21:33:17,404][105620] Updated weights for policy 1, policy_version 850637 (0.0010) [2023-12-26 21:33:17,455][105620] Updated weights for policy 1, policy_version 850647 (0.0010) [2023-12-26 21:33:18,030][105620] Updated weights for policy 1, policy_version 850657 (0.0008) [2023-12-26 21:33:18,067][105692] Updated weights for policy 0, policy_version 850577 (0.0010) [2023-12-26 21:33:18,078][105620] Updated weights for policy 1, policy_version 850667 (0.0005) [2023-12-26 21:33:18,122][105692] Updated weights for policy 0, policy_version 850587 (0.0010) [2023-12-26 21:33:18,125][105620] Updated weights for policy 1, policy_version 850677 (0.0005) [2023-12-26 21:33:18,178][105692] Updated weights for policy 0, policy_version 850597 (0.0010) [2023-12-26 21:33:18,814][105620] Updated weights for policy 1, policy_version 850687 (0.0005) [2023-12-26 21:33:18,887][105620] Updated weights for policy 1, policy_version 850697 (0.0006) [2023-12-26 21:33:18,929][105692] Updated weights for policy 0, policy_version 850607 (0.0008) [2023-12-26 21:33:18,952][105620] Updated weights for policy 1, policy_version 850707 (0.0011) [2023-12-26 21:33:18,993][105692] Updated weights for policy 0, policy_version 850617 (0.0010) [2023-12-26 21:33:19,046][105692] Updated weights for policy 0, policy_version 850627 (0.0011) [2023-12-26 21:33:19,759][105620] Updated weights for policy 1, policy_version 850717 (0.0011) [2023-12-26 21:33:19,781][105692] Updated weights for policy 0, policy_version 850637 (0.0011) [2023-12-26 21:33:19,820][105620] Updated weights for policy 1, policy_version 850727 (0.0011) [2023-12-26 21:33:19,851][105692] Updated weights for policy 0, policy_version 850647 (0.0010) [2023-12-26 21:33:19,894][105620] Updated weights for policy 1, policy_version 850737 (0.0008) [2023-12-26 21:33:19,917][105692] Updated weights for policy 0, policy_version 850657 (0.0010) [2023-12-26 21:33:20,631][105692] Updated weights for policy 0, policy_version 850667 (0.0010) [2023-12-26 21:33:20,661][105620] Updated weights for policy 1, policy_version 850747 (0.0009) [2023-12-26 21:33:20,691][105692] Updated weights for policy 0, policy_version 850677 (0.0008) [2023-12-26 21:33:20,714][105620] Updated weights for policy 1, policy_version 850757 (0.0007) [2023-12-26 21:33:20,746][105692] Updated weights for policy 0, policy_version 850687 (0.0006) [2023-12-26 21:33:20,773][105620] Updated weights for policy 1, policy_version 850767 (0.0008) [2023-12-26 21:33:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 435634176. Throughput: 0: 9828.9, 1: 9745.6. Samples: 435620636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:33:21,062][104569] Avg episode reward: [(0, '8984.673'), (1, '9080.050')] [2023-12-26 21:33:21,429][105692] Updated weights for policy 0, policy_version 850697 (0.0008) [2023-12-26 21:33:21,484][105692] Updated weights for policy 0, policy_version 850707 (0.0008) [2023-12-26 21:33:21,535][105692] Updated weights for policy 0, policy_version 850717 (0.0009) [2023-12-26 21:33:21,585][105692] Updated weights for policy 0, policy_version 850727 (0.0008) [2023-12-26 21:33:21,610][105620] Updated weights for policy 1, policy_version 850777 (0.0009) [2023-12-26 21:33:21,672][105620] Updated weights for policy 1, policy_version 850787 (0.0007) [2023-12-26 21:33:21,737][105620] Updated weights for policy 1, policy_version 850797 (0.0007) [2023-12-26 21:33:21,789][105620] Updated weights for policy 1, policy_version 850807 (0.0008) [2023-12-26 21:33:22,382][105692] Updated weights for policy 0, policy_version 850737 (0.0010) [2023-12-26 21:33:22,443][105692] Updated weights for policy 0, policy_version 850747 (0.0011) [2023-12-26 21:33:22,485][105620] Updated weights for policy 1, policy_version 850817 (0.0008) [2023-12-26 21:33:22,500][105692] Updated weights for policy 0, policy_version 850757 (0.0011) [2023-12-26 21:33:22,548][105620] Updated weights for policy 1, policy_version 850827 (0.0009) [2023-12-26 21:33:22,610][105620] Updated weights for policy 1, policy_version 850837 (0.0009) [2023-12-26 21:33:23,233][105692] Updated weights for policy 0, policy_version 850767 (0.0008) [2023-12-26 21:33:23,302][105692] Updated weights for policy 0, policy_version 850777 (0.0009) [2023-12-26 21:33:23,355][105692] Updated weights for policy 0, policy_version 850787 (0.0009) [2023-12-26 21:33:23,367][105620] Updated weights for policy 1, policy_version 850847 (0.0008) [2023-12-26 21:33:23,422][105620] Updated weights for policy 1, policy_version 850857 (0.0008) [2023-12-26 21:33:23,473][105620] Updated weights for policy 1, policy_version 850867 (0.0008) [2023-12-26 21:33:23,939][105692] Updated weights for policy 0, policy_version 850797 (0.0009) [2023-12-26 21:33:24,000][105692] Updated weights for policy 0, policy_version 850807 (0.0010) [2023-12-26 21:33:24,061][105692] Updated weights for policy 0, policy_version 850817 (0.0010) [2023-12-26 21:33:24,243][105620] Updated weights for policy 1, policy_version 850877 (0.0009) [2023-12-26 21:33:24,304][105620] Updated weights for policy 1, policy_version 850887 (0.0010) [2023-12-26 21:33:24,362][105620] Updated weights for policy 1, policy_version 850897 (0.0010) [2023-12-26 21:33:24,805][105692] Updated weights for policy 0, policy_version 850827 (0.0010) [2023-12-26 21:33:24,856][105692] Updated weights for policy 0, policy_version 850837 (0.0010) [2023-12-26 21:33:24,914][105692] Updated weights for policy 0, policy_version 850847 (0.0010) [2023-12-26 21:33:25,092][105620] Updated weights for policy 1, policy_version 850907 (0.0010) [2023-12-26 21:33:25,157][105620] Updated weights for policy 1, policy_version 850917 (0.0010) [2023-12-26 21:33:25,215][105620] Updated weights for policy 1, policy_version 850927 (0.0010) [2023-12-26 21:33:25,651][105692] Updated weights for policy 0, policy_version 850857 (0.0010) [2023-12-26 21:33:25,704][105692] Updated weights for policy 0, policy_version 850867 (0.0011) [2023-12-26 21:33:25,774][105692] Updated weights for policy 0, policy_version 850877 (0.0011) [2023-12-26 21:33:25,825][105692] Updated weights for policy 0, policy_version 850887 (0.0010) [2023-12-26 21:33:25,848][105620] Updated weights for policy 1, policy_version 850937 (0.0010) [2023-12-26 21:33:25,912][105620] Updated weights for policy 1, policy_version 850947 (0.0008) [2023-12-26 21:33:25,970][105620] Updated weights for policy 1, policy_version 850957 (0.0010) [2023-12-26 21:33:26,021][105620] Updated weights for policy 1, policy_version 850967 (0.0010) [2023-12-26 21:33:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 435732480. Throughput: 0: 9807.4, 1: 9723.7. Samples: 435735580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:33:26,062][104569] Avg episode reward: [(0, '8985.668'), (1, '9171.159')] [2023-12-26 21:33:26,578][105692] Updated weights for policy 0, policy_version 850897 (0.0011) [2023-12-26 21:33:26,636][105692] Updated weights for policy 0, policy_version 850907 (0.0010) [2023-12-26 21:33:26,673][105620] Updated weights for policy 1, policy_version 850977 (0.0010) [2023-12-26 21:33:26,694][105692] Updated weights for policy 0, policy_version 850917 (0.0010) [2023-12-26 21:33:26,732][105620] Updated weights for policy 1, policy_version 850987 (0.0009) [2023-12-26 21:33:26,795][105620] Updated weights for policy 1, policy_version 850997 (0.0008) [2023-12-26 21:33:27,271][105692] Updated weights for policy 0, policy_version 850927 (0.0009) [2023-12-26 21:33:27,323][105692] Updated weights for policy 0, policy_version 850937 (0.0008) [2023-12-26 21:33:27,350][105620] Updated weights for policy 1, policy_version 851007 (0.0011) [2023-12-26 21:33:27,371][105692] Updated weights for policy 0, policy_version 850947 (0.0005) [2023-12-26 21:33:27,411][105620] Updated weights for policy 1, policy_version 851017 (0.0010) [2023-12-26 21:33:27,472][105620] Updated weights for policy 1, policy_version 851027 (0.0009) [2023-12-26 21:33:27,908][105692] Updated weights for policy 0, policy_version 850957 (0.0005) [2023-12-26 21:33:27,959][105692] Updated weights for policy 0, policy_version 850967 (0.0005) [2023-12-26 21:33:28,008][105692] Updated weights for policy 0, policy_version 850977 (0.0005) [2023-12-26 21:33:28,037][105620] Updated weights for policy 1, policy_version 851037 (0.0007) [2023-12-26 21:33:28,099][105620] Updated weights for policy 1, policy_version 851047 (0.0009) [2023-12-26 21:33:28,159][105620] Updated weights for policy 1, policy_version 851057 (0.0009) [2023-12-26 21:33:28,731][105692] Updated weights for policy 0, policy_version 850987 (0.0009) [2023-12-26 21:33:28,790][105692] Updated weights for policy 0, policy_version 850997 (0.0010) [2023-12-26 21:33:28,850][105692] Updated weights for policy 0, policy_version 851007 (0.0006) [2023-12-26 21:33:28,868][105620] Updated weights for policy 1, policy_version 851067 (0.0008) [2023-12-26 21:33:28,918][105620] Updated weights for policy 1, policy_version 851077 (0.0008) [2023-12-26 21:33:28,979][105620] Updated weights for policy 1, policy_version 851087 (0.0010) [2023-12-26 21:33:29,401][105692] Updated weights for policy 0, policy_version 851017 (0.0005) [2023-12-26 21:33:29,463][105692] Updated weights for policy 0, policy_version 851027 (0.0007) [2023-12-26 21:33:29,528][105692] Updated weights for policy 0, policy_version 851037 (0.0010) [2023-12-26 21:33:29,593][105692] Updated weights for policy 0, policy_version 851047 (0.0010) [2023-12-26 21:33:29,867][105620] Updated weights for policy 1, policy_version 851097 (0.0010) [2023-12-26 21:33:29,924][105620] Updated weights for policy 1, policy_version 851107 (0.0008) [2023-12-26 21:33:29,983][105620] Updated weights for policy 1, policy_version 851117 (0.0008) [2023-12-26 21:33:30,029][105620] Updated weights for policy 1, policy_version 851127 (0.0006) [2023-12-26 21:33:30,215][105692] Updated weights for policy 0, policy_version 851057 (0.0006) [2023-12-26 21:33:30,279][105692] Updated weights for policy 0, policy_version 851067 (0.0005) [2023-12-26 21:33:30,340][105692] Updated weights for policy 0, policy_version 851077 (0.0006) [2023-12-26 21:33:30,707][105620] Updated weights for policy 1, policy_version 851137 (0.0010) [2023-12-26 21:33:30,760][105620] Updated weights for policy 1, policy_version 851147 (0.0010) [2023-12-26 21:33:30,820][105620] Updated weights for policy 1, policy_version 851157 (0.0009) [2023-12-26 21:33:30,854][105692] Updated weights for policy 0, policy_version 851087 (0.0009) [2023-12-26 21:33:30,912][105692] Updated weights for policy 0, policy_version 851097 (0.0007) [2023-12-26 21:33:30,967][105692] Updated weights for policy 0, policy_version 851107 (0.0006) [2023-12-26 21:33:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 435838976. Throughput: 0: 9861.5, 1: 9808.7. Samples: 435800172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:33:31,062][104569] Avg episode reward: [(0, '9000.671'), (1, '9354.176')] [2023-12-26 21:33:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000851112_217915392.pth... [2023-12-26 21:33:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000851160_217923584.pth... [2023-12-26 21:33:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000849928_217612288.pth [2023-12-26 21:33:31,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000850008_217628672.pth [2023-12-26 21:33:31,536][105620] Updated weights for policy 1, policy_version 851167 (0.0005) [2023-12-26 21:33:31,589][105620] Updated weights for policy 1, policy_version 851177 (0.0005) [2023-12-26 21:33:31,661][105620] Updated weights for policy 1, policy_version 851187 (0.0007) [2023-12-26 21:33:31,708][105692] Updated weights for policy 0, policy_version 851117 (0.0006) [2023-12-26 21:33:31,776][105692] Updated weights for policy 0, policy_version 851127 (0.0007) [2023-12-26 21:33:31,830][105692] Updated weights for policy 0, policy_version 851137 (0.0006) [2023-12-26 21:33:32,224][105620] Updated weights for policy 1, policy_version 851197 (0.0006) [2023-12-26 21:33:32,285][105620] Updated weights for policy 1, policy_version 851207 (0.0008) [2023-12-26 21:33:32,348][105620] Updated weights for policy 1, policy_version 851217 (0.0009) [2023-12-26 21:33:32,540][105692] Updated weights for policy 0, policy_version 851147 (0.0007) [2023-12-26 21:33:32,593][105692] Updated weights for policy 0, policy_version 851157 (0.0010) [2023-12-26 21:33:32,646][105692] Updated weights for policy 0, policy_version 851168 (0.0009) [2023-12-26 21:33:32,964][105620] Updated weights for policy 1, policy_version 851227 (0.0007) [2023-12-26 21:33:33,023][105620] Updated weights for policy 1, policy_version 851237 (0.0009) [2023-12-26 21:33:33,082][105620] Updated weights for policy 1, policy_version 851247 (0.0009) [2023-12-26 21:33:33,455][105692] Updated weights for policy 0, policy_version 851178 (0.0009) [2023-12-26 21:33:33,502][105692] Updated weights for policy 0, policy_version 851188 (0.0007) [2023-12-26 21:33:33,545][105692] Updated weights for policy 0, policy_version 851198 (0.0008) [2023-12-26 21:33:33,595][105692] Updated weights for policy 0, policy_version 851208 (0.0008) [2023-12-26 21:33:33,782][105620] Updated weights for policy 1, policy_version 851257 (0.0009) [2023-12-26 21:33:33,832][105620] Updated weights for policy 1, policy_version 851267 (0.0009) [2023-12-26 21:33:33,887][105620] Updated weights for policy 1, policy_version 851277 (0.0009) [2023-12-26 21:33:33,948][105620] Updated weights for policy 1, policy_version 851287 (0.0008) [2023-12-26 21:33:34,338][105692] Updated weights for policy 0, policy_version 851218 (0.0009) [2023-12-26 21:33:34,394][105692] Updated weights for policy 0, policy_version 851228 (0.0008) [2023-12-26 21:33:34,447][105692] Updated weights for policy 0, policy_version 851238 (0.0008) [2023-12-26 21:33:34,739][105620] Updated weights for policy 1, policy_version 851297 (0.0010) [2023-12-26 21:33:34,802][105620] Updated weights for policy 1, policy_version 851307 (0.0011) [2023-12-26 21:33:34,861][105620] Updated weights for policy 1, policy_version 851317 (0.0011) [2023-12-26 21:33:35,238][105692] Updated weights for policy 0, policy_version 851248 (0.0008) [2023-12-26 21:33:35,293][105692] Updated weights for policy 0, policy_version 851258 (0.0005) [2023-12-26 21:33:35,357][105692] Updated weights for policy 0, policy_version 851268 (0.0007) [2023-12-26 21:33:35,540][105620] Updated weights for policy 1, policy_version 851327 (0.0009) [2023-12-26 21:33:35,602][105586] KL-divergence is very high: 336.3523 [2023-12-26 21:33:35,609][105620] Updated weights for policy 1, policy_version 851337 (0.0011) [2023-12-26 21:33:35,656][105586] KL-divergence is very high: 523.1404 [2023-12-26 21:33:35,674][105620] Updated weights for policy 1, policy_version 851347 (0.0010) [2023-12-26 21:33:36,014][105692] Updated weights for policy 0, policy_version 851278 (0.0008) [2023-12-26 21:33:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.9, 300 sec: 19522.0). Total num frames: 435929088. Throughput: 0: 9949.1, 1: 9822.1. Samples: 435919948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:33:36,062][104569] Avg episode reward: [(0, '8994.392'), (1, '8864.702')] [2023-12-26 21:33:36,073][105692] Updated weights for policy 0, policy_version 851288 (0.0008) [2023-12-26 21:33:36,138][105692] Updated weights for policy 0, policy_version 851298 (0.0008) [2023-12-26 21:33:36,347][105620] Updated weights for policy 1, policy_version 851357 (0.0009) [2023-12-26 21:33:36,415][105620] Updated weights for policy 1, policy_version 851367 (0.0007) [2023-12-26 21:33:36,476][105620] Updated weights for policy 1, policy_version 851377 (0.0009) [2023-12-26 21:33:36,838][105692] Updated weights for policy 0, policy_version 851308 (0.0008) [2023-12-26 21:33:36,897][105692] Updated weights for policy 0, policy_version 851318 (0.0011) [2023-12-26 21:33:36,949][105692] Updated weights for policy 0, policy_version 851328 (0.0010) [2023-12-26 21:33:37,106][105620] Updated weights for policy 1, policy_version 851387 (0.0010) [2023-12-26 21:33:37,162][105620] Updated weights for policy 1, policy_version 851397 (0.0006) [2023-12-26 21:33:37,227][105620] Updated weights for policy 1, policy_version 851407 (0.0010) [2023-12-26 21:33:37,600][105692] Updated weights for policy 0, policy_version 851338 (0.0009) [2023-12-26 21:33:37,653][105692] Updated weights for policy 0, policy_version 851348 (0.0008) [2023-12-26 21:33:37,712][105692] Updated weights for policy 0, policy_version 851358 (0.0006) [2023-12-26 21:33:37,771][105692] Updated weights for policy 0, policy_version 851368 (0.0005) [2023-12-26 21:33:38,062][105620] Updated weights for policy 1, policy_version 851417 (0.0010) [2023-12-26 21:33:38,127][105620] Updated weights for policy 1, policy_version 851427 (0.0010) [2023-12-26 21:33:38,183][105620] Updated weights for policy 1, policy_version 851437 (0.0008) [2023-12-26 21:33:38,239][105620] Updated weights for policy 1, policy_version 851447 (0.0009) [2023-12-26 21:33:38,343][105692] Updated weights for policy 0, policy_version 851378 (0.0009) [2023-12-26 21:33:38,400][105692] Updated weights for policy 0, policy_version 851388 (0.0009) [2023-12-26 21:33:38,457][105692] Updated weights for policy 0, policy_version 851398 (0.0009) [2023-12-26 21:33:38,983][105620] Updated weights for policy 1, policy_version 851457 (0.0009) [2023-12-26 21:33:39,040][105620] Updated weights for policy 1, policy_version 851467 (0.0009) [2023-12-26 21:33:39,102][105620] Updated weights for policy 1, policy_version 851477 (0.0009) [2023-12-26 21:33:39,117][105692] Updated weights for policy 0, policy_version 851408 (0.0006) [2023-12-26 21:33:39,166][105692] Updated weights for policy 0, policy_version 851418 (0.0005) [2023-12-26 21:33:39,226][105692] Updated weights for policy 0, policy_version 851428 (0.0007) [2023-12-26 21:33:39,894][105620] Updated weights for policy 1, policy_version 851487 (0.0007) [2023-12-26 21:33:39,920][105692] Updated weights for policy 0, policy_version 851438 (0.0009) [2023-12-26 21:33:39,960][105620] Updated weights for policy 1, policy_version 851497 (0.0009) [2023-12-26 21:33:39,983][105692] Updated weights for policy 0, policy_version 851448 (0.0008) [2023-12-26 21:33:40,018][105620] Updated weights for policy 1, policy_version 851507 (0.0007) [2023-12-26 21:33:40,044][105692] Updated weights for policy 0, policy_version 851458 (0.0007) [2023-12-26 21:33:40,724][105620] Updated weights for policy 1, policy_version 851517 (0.0005) [2023-12-26 21:33:40,740][105692] Updated weights for policy 0, policy_version 851468 (0.0009) [2023-12-26 21:33:40,791][105620] Updated weights for policy 1, policy_version 851527 (0.0006) [2023-12-26 21:33:40,795][105692] Updated weights for policy 0, policy_version 851478 (0.0008) [2023-12-26 21:33:40,841][105692] Updated weights for policy 0, policy_version 851488 (0.0008) [2023-12-26 21:33:40,851][105620] Updated weights for policy 1, policy_version 851537 (0.0009) [2023-12-26 21:33:41,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.2, 300 sec: 19549.7). Total num frames: 436035584. Throughput: 0: 9974.1, 1: 9837.4. Samples: 436039140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:33:41,063][104569] Avg episode reward: [(0, '8976.521'), (1, '8775.023')] [2023-12-26 21:33:41,543][105692] Updated weights for policy 0, policy_version 851498 (0.0007) [2023-12-26 21:33:41,583][105620] Updated weights for policy 1, policy_version 851547 (0.0010) [2023-12-26 21:33:41,608][105692] Updated weights for policy 0, policy_version 851508 (0.0009) [2023-12-26 21:33:41,640][105620] Updated weights for policy 1, policy_version 851557 (0.0011) [2023-12-26 21:33:41,677][105692] Updated weights for policy 0, policy_version 851518 (0.0008) [2023-12-26 21:33:41,708][105620] Updated weights for policy 1, policy_version 851567 (0.0011) [2023-12-26 21:33:41,746][105692] Updated weights for policy 0, policy_version 851528 (0.0010) [2023-12-26 21:33:42,355][105620] Updated weights for policy 1, policy_version 851577 (0.0008) [2023-12-26 21:33:42,419][105620] Updated weights for policy 1, policy_version 851587 (0.0010) [2023-12-26 21:33:42,471][105620] Updated weights for policy 1, policy_version 851597 (0.0011) [2023-12-26 21:33:42,510][105692] Updated weights for policy 0, policy_version 851538 (0.0011) [2023-12-26 21:33:42,524][105620] Updated weights for policy 1, policy_version 851607 (0.0010) [2023-12-26 21:33:42,566][105692] Updated weights for policy 0, policy_version 851548 (0.0011) [2023-12-26 21:33:42,620][105692] Updated weights for policy 0, policy_version 851558 (0.0010) [2023-12-26 21:33:43,192][105620] Updated weights for policy 1, policy_version 851617 (0.0006) [2023-12-26 21:33:43,246][105692] Updated weights for policy 0, policy_version 851568 (0.0006) [2023-12-26 21:33:43,258][105620] Updated weights for policy 1, policy_version 851627 (0.0005) [2023-12-26 21:33:43,301][105692] Updated weights for policy 0, policy_version 851578 (0.0005) [2023-12-26 21:33:43,327][105620] Updated weights for policy 1, policy_version 851637 (0.0007) [2023-12-26 21:33:43,346][105692] Updated weights for policy 0, policy_version 851588 (0.0006) [2023-12-26 21:33:43,944][105692] Updated weights for policy 0, policy_version 851598 (0.0006) [2023-12-26 21:33:43,951][105620] Updated weights for policy 1, policy_version 851647 (0.0007) [2023-12-26 21:33:43,996][105692] Updated weights for policy 0, policy_version 851608 (0.0005) [2023-12-26 21:33:44,014][105620] Updated weights for policy 1, policy_version 851657 (0.0006) [2023-12-26 21:33:44,053][105692] Updated weights for policy 0, policy_version 851618 (0.0005) [2023-12-26 21:33:44,073][105620] Updated weights for policy 1, policy_version 851667 (0.0009) [2023-12-26 21:33:44,716][105620] Updated weights for policy 1, policy_version 851677 (0.0009) [2023-12-26 21:33:44,743][105692] Updated weights for policy 0, policy_version 851628 (0.0007) [2023-12-26 21:33:44,767][105620] Updated weights for policy 1, policy_version 851687 (0.0006) [2023-12-26 21:33:44,806][105692] Updated weights for policy 0, policy_version 851638 (0.0011) [2023-12-26 21:33:44,828][105620] Updated weights for policy 1, policy_version 851697 (0.0007) [2023-12-26 21:33:44,869][105692] Updated weights for policy 0, policy_version 851648 (0.0006) [2023-12-26 21:33:45,513][105692] Updated weights for policy 0, policy_version 851658 (0.0008) [2023-12-26 21:33:45,522][105620] Updated weights for policy 1, policy_version 851707 (0.0006) [2023-12-26 21:33:45,562][105692] Updated weights for policy 0, policy_version 851668 (0.0011) [2023-12-26 21:33:45,579][105620] Updated weights for policy 1, policy_version 851717 (0.0005) [2023-12-26 21:33:45,605][105692] Updated weights for policy 0, policy_version 851678 (0.0006) [2023-12-26 21:33:45,625][105620] Updated weights for policy 1, policy_version 851727 (0.0005) [2023-12-26 21:33:45,658][105692] Updated weights for policy 0, policy_version 851688 (0.0005) [2023-12-26 21:33:46,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 436133888. Throughput: 0: 9955.6, 1: 9870.1. Samples: 436099812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:33:46,063][104569] Avg episode reward: [(0, '8802.403'), (1, '8932.900')] [2023-12-26 21:33:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000851688_218062848.pth... [2023-12-26 21:33:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000851736_218071040.pth... [2023-12-26 21:33:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000850584_217776128.pth [2023-12-26 21:33:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000850504_217759744.pth [2023-12-26 21:33:46,196][105620] Updated weights for policy 1, policy_version 851737 (0.0006) [2023-12-26 21:33:46,259][105692] Updated weights for policy 0, policy_version 851698 (0.0011) [2023-12-26 21:33:46,264][105620] Updated weights for policy 1, policy_version 851747 (0.0010) [2023-12-26 21:33:46,316][105620] Updated weights for policy 1, policy_version 851757 (0.0010) [2023-12-26 21:33:46,316][105692] Updated weights for policy 0, policy_version 851708 (0.0010) [2023-12-26 21:33:46,362][105692] Updated weights for policy 0, policy_version 851718 (0.0005) [2023-12-26 21:33:46,387][105620] Updated weights for policy 1, policy_version 851767 (0.0010) [2023-12-26 21:33:46,973][105692] Updated weights for policy 0, policy_version 851728 (0.0010) [2023-12-26 21:33:47,011][105620] Updated weights for policy 1, policy_version 851777 (0.0008) [2023-12-26 21:33:47,038][105692] Updated weights for policy 0, policy_version 851738 (0.0006) [2023-12-26 21:33:47,064][105620] Updated weights for policy 1, policy_version 851787 (0.0007) [2023-12-26 21:33:47,086][105692] Updated weights for policy 0, policy_version 851748 (0.0011) [2023-12-26 21:33:47,120][105620] Updated weights for policy 1, policy_version 851797 (0.0006) [2023-12-26 21:33:47,703][105692] Updated weights for policy 0, policy_version 851758 (0.0008) [2023-12-26 21:33:47,724][105620] Updated weights for policy 1, policy_version 851807 (0.0006) [2023-12-26 21:33:47,755][105692] Updated weights for policy 0, policy_version 851768 (0.0005) [2023-12-26 21:33:47,779][105620] Updated weights for policy 1, policy_version 851817 (0.0007) [2023-12-26 21:33:47,809][105692] Updated weights for policy 0, policy_version 851778 (0.0005) [2023-12-26 21:33:47,831][105620] Updated weights for policy 1, policy_version 851827 (0.0009) [2023-12-26 21:33:48,526][105620] Updated weights for policy 1, policy_version 851837 (0.0008) [2023-12-26 21:33:48,548][105692] Updated weights for policy 0, policy_version 851788 (0.0006) [2023-12-26 21:33:48,587][105620] Updated weights for policy 1, policy_version 851847 (0.0007) [2023-12-26 21:33:48,612][105692] Updated weights for policy 0, policy_version 851798 (0.0009) [2023-12-26 21:33:48,639][105620] Updated weights for policy 1, policy_version 851857 (0.0005) [2023-12-26 21:33:48,673][105692] Updated weights for policy 0, policy_version 851808 (0.0008) [2023-12-26 21:33:49,355][105620] Updated weights for policy 1, policy_version 851867 (0.0009) [2023-12-26 21:33:49,423][105620] Updated weights for policy 1, policy_version 851877 (0.0009) [2023-12-26 21:33:49,432][105692] Updated weights for policy 0, policy_version 851818 (0.0008) [2023-12-26 21:33:49,482][105620] Updated weights for policy 1, policy_version 851887 (0.0008) [2023-12-26 21:33:49,484][105692] Updated weights for policy 0, policy_version 851828 (0.0008) [2023-12-26 21:33:49,539][105692] Updated weights for policy 0, policy_version 851838 (0.0008) [2023-12-26 21:33:49,604][105692] Updated weights for policy 0, policy_version 851848 (0.0009) [2023-12-26 21:33:50,254][105620] Updated weights for policy 1, policy_version 851897 (0.0007) [2023-12-26 21:33:50,318][105620] Updated weights for policy 1, policy_version 851907 (0.0011) [2023-12-26 21:33:50,378][105620] Updated weights for policy 1, policy_version 851917 (0.0009) [2023-12-26 21:33:50,388][105692] Updated weights for policy 0, policy_version 851858 (0.0008) [2023-12-26 21:33:50,435][105620] Updated weights for policy 1, policy_version 851927 (0.0007) [2023-12-26 21:33:50,454][105692] Updated weights for policy 0, policy_version 851868 (0.0007) [2023-12-26 21:33:50,519][105692] Updated weights for policy 0, policy_version 851878 (0.0008) [2023-12-26 21:33:51,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 436232192. Throughput: 0: 10014.2, 1: 9895.2. Samples: 436224768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:33:51,062][104569] Avg episode reward: [(0, '8890.068'), (1, '8293.572')] [2023-12-26 21:33:51,229][105620] Updated weights for policy 1, policy_version 851937 (0.0009) [2023-12-26 21:33:51,292][105620] Updated weights for policy 1, policy_version 851947 (0.0008) [2023-12-26 21:33:51,330][105692] Updated weights for policy 0, policy_version 851888 (0.0007) [2023-12-26 21:33:51,363][105620] Updated weights for policy 1, policy_version 851957 (0.0008) [2023-12-26 21:33:51,396][105692] Updated weights for policy 0, policy_version 851898 (0.0007) [2023-12-26 21:33:51,450][105692] Updated weights for policy 0, policy_version 851908 (0.0008) [2023-12-26 21:33:52,124][105692] Updated weights for policy 0, policy_version 851918 (0.0007) [2023-12-26 21:33:52,174][105692] Updated weights for policy 0, policy_version 851928 (0.0006) [2023-12-26 21:33:52,176][105620] Updated weights for policy 1, policy_version 851967 (0.0007) [2023-12-26 21:33:52,221][105620] Updated weights for policy 1, policy_version 851977 (0.0008) [2023-12-26 21:33:52,232][105692] Updated weights for policy 0, policy_version 851938 (0.0008) [2023-12-26 21:33:52,276][105620] Updated weights for policy 1, policy_version 851987 (0.0008) [2023-12-26 21:33:52,928][105692] Updated weights for policy 0, policy_version 851948 (0.0007) [2023-12-26 21:33:52,989][105692] Updated weights for policy 0, policy_version 851958 (0.0006) [2023-12-26 21:33:53,023][105620] Updated weights for policy 1, policy_version 851997 (0.0010) [2023-12-26 21:33:53,050][105692] Updated weights for policy 0, policy_version 851968 (0.0007) [2023-12-26 21:33:53,075][105620] Updated weights for policy 1, policy_version 852007 (0.0011) [2023-12-26 21:33:53,127][105620] Updated weights for policy 1, policy_version 852017 (0.0010) [2023-12-26 21:33:53,660][105692] Updated weights for policy 0, policy_version 851978 (0.0006) [2023-12-26 21:33:53,729][105692] Updated weights for policy 0, policy_version 851988 (0.0006) [2023-12-26 21:33:53,787][105692] Updated weights for policy 0, policy_version 851998 (0.0005) [2023-12-26 21:33:53,835][105692] Updated weights for policy 0, policy_version 852008 (0.0005) [2023-12-26 21:33:53,850][105620] Updated weights for policy 1, policy_version 852027 (0.0009) [2023-12-26 21:33:53,895][105620] Updated weights for policy 1, policy_version 852037 (0.0005) [2023-12-26 21:33:53,941][105620] Updated weights for policy 1, policy_version 852047 (0.0006) [2023-12-26 21:33:54,552][105692] Updated weights for policy 0, policy_version 852018 (0.0009) [2023-12-26 21:33:54,586][105620] Updated weights for policy 1, policy_version 852057 (0.0007) [2023-12-26 21:33:54,599][105692] Updated weights for policy 0, policy_version 852028 (0.0008) [2023-12-26 21:33:54,634][105620] Updated weights for policy 1, policy_version 852067 (0.0007) [2023-12-26 21:33:54,646][105692] Updated weights for policy 0, policy_version 852038 (0.0008) [2023-12-26 21:33:54,683][105620] Updated weights for policy 1, policy_version 852077 (0.0005) [2023-12-26 21:33:54,729][105620] Updated weights for policy 1, policy_version 852087 (0.0005) [2023-12-26 21:33:55,399][105620] Updated weights for policy 1, policy_version 852097 (0.0005) [2023-12-26 21:33:55,438][105692] Updated weights for policy 0, policy_version 852048 (0.0010) [2023-12-26 21:33:55,459][105620] Updated weights for policy 1, policy_version 852107 (0.0006) [2023-12-26 21:33:55,497][105692] Updated weights for policy 0, policy_version 852058 (0.0010) [2023-12-26 21:33:55,513][105620] Updated weights for policy 1, policy_version 852117 (0.0008) [2023-12-26 21:33:55,550][105692] Updated weights for policy 0, policy_version 852068 (0.0009) [2023-12-26 21:33:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 436330496. Throughput: 0: 10014.1, 1: 9900.4. Samples: 436341484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:33:56,063][104569] Avg episode reward: [(0, '8892.337'), (1, '8595.187')] [2023-12-26 21:33:56,084][105692] Updated weights for policy 0, policy_version 852078 (0.0005) [2023-12-26 21:33:56,144][105692] Updated weights for policy 0, policy_version 852088 (0.0005) [2023-12-26 21:33:56,195][105692] Updated weights for policy 0, policy_version 852098 (0.0007) [2023-12-26 21:33:56,207][105620] Updated weights for policy 1, policy_version 852127 (0.0010) [2023-12-26 21:33:56,268][105620] Updated weights for policy 1, policy_version 852137 (0.0010) [2023-12-26 21:33:56,337][105620] Updated weights for policy 1, policy_version 852147 (0.0010) [2023-12-26 21:33:56,833][105692] Updated weights for policy 0, policy_version 852108 (0.0006) [2023-12-26 21:33:56,898][105692] Updated weights for policy 0, policy_version 852118 (0.0007) [2023-12-26 21:33:56,956][105692] Updated weights for policy 0, policy_version 852128 (0.0008) [2023-12-26 21:33:57,075][105620] Updated weights for policy 1, policy_version 852157 (0.0010) [2023-12-26 21:33:57,119][105620] Updated weights for policy 1, policy_version 852167 (0.0010) [2023-12-26 21:33:57,166][105620] Updated weights for policy 1, policy_version 852177 (0.0010) [2023-12-26 21:33:57,580][105692] Updated weights for policy 0, policy_version 852138 (0.0007) [2023-12-26 21:33:57,642][105692] Updated weights for policy 0, policy_version 852148 (0.0005) [2023-12-26 21:33:57,699][105692] Updated weights for policy 0, policy_version 852158 (0.0005) [2023-12-26 21:33:57,757][105692] Updated weights for policy 0, policy_version 852168 (0.0006) [2023-12-26 21:33:57,916][105620] Updated weights for policy 1, policy_version 852187 (0.0010) [2023-12-26 21:33:57,983][105620] Updated weights for policy 1, policy_version 852197 (0.0007) [2023-12-26 21:33:58,042][105620] Updated weights for policy 1, policy_version 852207 (0.0009) [2023-12-26 21:33:58,305][105692] Updated weights for policy 0, policy_version 852178 (0.0008) [2023-12-26 21:33:58,373][105692] Updated weights for policy 0, policy_version 852188 (0.0009) [2023-12-26 21:33:58,435][105692] Updated weights for policy 0, policy_version 852198 (0.0008) [2023-12-26 21:33:58,838][105620] Updated weights for policy 1, policy_version 852217 (0.0008) [2023-12-26 21:33:58,908][105620] Updated weights for policy 1, policy_version 852227 (0.0009) [2023-12-26 21:33:58,968][105620] Updated weights for policy 1, policy_version 852237 (0.0008) [2023-12-26 21:33:59,031][105620] Updated weights for policy 1, policy_version 852247 (0.0008) [2023-12-26 21:33:59,253][105692] Updated weights for policy 0, policy_version 852208 (0.0008) [2023-12-26 21:33:59,320][105692] Updated weights for policy 0, policy_version 852218 (0.0010) [2023-12-26 21:33:59,385][105692] Updated weights for policy 0, policy_version 852228 (0.0011) [2023-12-26 21:33:59,840][105620] Updated weights for policy 1, policy_version 852257 (0.0010) [2023-12-26 21:33:59,901][105620] Updated weights for policy 1, policy_version 852267 (0.0008) [2023-12-26 21:33:59,961][105620] Updated weights for policy 1, policy_version 852277 (0.0006) [2023-12-26 21:34:00,075][105692] Updated weights for policy 0, policy_version 852238 (0.0008) [2023-12-26 21:34:00,146][105692] Updated weights for policy 0, policy_version 852248 (0.0006) [2023-12-26 21:34:00,205][105692] Updated weights for policy 0, policy_version 852258 (0.0008) [2023-12-26 21:34:00,649][105620] Updated weights for policy 1, policy_version 852287 (0.0008) [2023-12-26 21:34:00,695][105620] Updated weights for policy 1, policy_version 852297 (0.0009) [2023-12-26 21:34:00,751][105620] Updated weights for policy 1, policy_version 852307 (0.0009) [2023-12-26 21:34:00,903][105692] Updated weights for policy 0, policy_version 852268 (0.0007) [2023-12-26 21:34:00,949][105692] Updated weights for policy 0, policy_version 852278 (0.0008) [2023-12-26 21:34:00,995][105692] Updated weights for policy 0, policy_version 852288 (0.0008) [2023-12-26 21:34:01,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19933.8, 300 sec: 19549.7). Total num frames: 436436992. Throughput: 0: 10127.9, 1: 9914.0. Samples: 436402780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:34:01,062][104569] Avg episode reward: [(0, '8985.013'), (1, '9079.086')] [2023-12-26 21:34:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000852296_218218496.pth... [2023-12-26 21:34:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000852312_218218496.pth... [2023-12-26 21:34:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000851160_217923584.pth [2023-12-26 21:34:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000851112_217915392.pth [2023-12-26 21:34:01,505][105620] Updated weights for policy 1, policy_version 852317 (0.0009) [2023-12-26 21:34:01,550][105620] Updated weights for policy 1, policy_version 852327 (0.0008) [2023-12-26 21:34:01,598][105620] Updated weights for policy 1, policy_version 852337 (0.0009) [2023-12-26 21:34:01,782][105692] Updated weights for policy 0, policy_version 852298 (0.0009) [2023-12-26 21:34:01,832][105692] Updated weights for policy 0, policy_version 852308 (0.0009) [2023-12-26 21:34:01,894][105692] Updated weights for policy 0, policy_version 852318 (0.0008) [2023-12-26 21:34:01,954][105692] Updated weights for policy 0, policy_version 852328 (0.0011) [2023-12-26 21:34:02,360][105620] Updated weights for policy 1, policy_version 852347 (0.0009) [2023-12-26 21:34:02,414][105620] Updated weights for policy 1, policy_version 852357 (0.0009) [2023-12-26 21:34:02,471][105620] Updated weights for policy 1, policy_version 852367 (0.0009) [2023-12-26 21:34:02,596][105692] Updated weights for policy 0, policy_version 852338 (0.0007) [2023-12-26 21:34:02,660][105692] Updated weights for policy 0, policy_version 852348 (0.0008) [2023-12-26 21:34:02,721][105692] Updated weights for policy 0, policy_version 852358 (0.0010) [2023-12-26 21:34:03,267][105620] Updated weights for policy 1, policy_version 852377 (0.0009) [2023-12-26 21:34:03,268][105692] Updated weights for policy 0, policy_version 852368 (0.0007) [2023-12-26 21:34:03,316][105620] Updated weights for policy 1, policy_version 852387 (0.0005) [2023-12-26 21:34:03,320][105692] Updated weights for policy 0, policy_version 852378 (0.0010) [2023-12-26 21:34:03,366][105620] Updated weights for policy 1, policy_version 852397 (0.0005) [2023-12-26 21:34:03,377][105692] Updated weights for policy 0, policy_version 852388 (0.0010) [2023-12-26 21:34:03,421][105620] Updated weights for policy 1, policy_version 852407 (0.0008) [2023-12-26 21:34:04,040][105692] Updated weights for policy 0, policy_version 852398 (0.0010) [2023-12-26 21:34:04,103][105692] Updated weights for policy 0, policy_version 852408 (0.0010) [2023-12-26 21:34:04,168][105692] Updated weights for policy 0, policy_version 852418 (0.0007) [2023-12-26 21:34:04,193][105620] Updated weights for policy 1, policy_version 852417 (0.0009) [2023-12-26 21:34:04,255][105620] Updated weights for policy 1, policy_version 852427 (0.0009) [2023-12-26 21:34:04,317][105620] Updated weights for policy 1, policy_version 852437 (0.0009) [2023-12-26 21:34:04,900][105692] Updated weights for policy 0, policy_version 852428 (0.0008) [2023-12-26 21:34:04,954][105692] Updated weights for policy 0, policy_version 852438 (0.0006) [2023-12-26 21:34:05,017][105692] Updated weights for policy 0, policy_version 852448 (0.0005) [2023-12-26 21:34:05,096][105620] Updated weights for policy 1, policy_version 852447 (0.0009) [2023-12-26 21:34:05,150][105620] Updated weights for policy 1, policy_version 852457 (0.0010) [2023-12-26 21:34:05,202][105620] Updated weights for policy 1, policy_version 852468 (0.0009) [2023-12-26 21:34:05,527][105692] Updated weights for policy 0, policy_version 852458 (0.0005) [2023-12-26 21:34:05,578][105692] Updated weights for policy 0, policy_version 852468 (0.0005) [2023-12-26 21:34:05,632][105692] Updated weights for policy 0, policy_version 852478 (0.0010) [2023-12-26 21:34:05,696][105692] Updated weights for policy 0, policy_version 852488 (0.0010) [2023-12-26 21:34:06,060][105620] Updated weights for policy 1, policy_version 852478 (0.0009) [2023-12-26 21:34:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 436527104. Throughput: 0: 10168.7, 1: 9755.1. Samples: 436517208. Policy #0 lag: (min: 29.0, avg: 36.8, max: 61.0) [2023-12-26 21:34:06,063][104569] Avg episode reward: [(0, '3604.327'), (1, '9078.081')] [2023-12-26 21:34:06,113][105620] Updated weights for policy 1, policy_version 852488 (0.0008) [2023-12-26 21:34:06,168][105620] Updated weights for policy 1, policy_version 852498 (0.0006) [2023-12-26 21:34:06,405][105692] Updated weights for policy 0, policy_version 852498 (0.0010) [2023-12-26 21:34:06,469][105692] Updated weights for policy 0, policy_version 852508 (0.0011) [2023-12-26 21:34:06,529][105692] Updated weights for policy 0, policy_version 852518 (0.0011) [2023-12-26 21:34:06,859][105620] Updated weights for policy 1, policy_version 852508 (0.0008) [2023-12-26 21:34:06,920][105620] Updated weights for policy 1, policy_version 852518 (0.0007) [2023-12-26 21:34:06,980][105620] Updated weights for policy 1, policy_version 852528 (0.0008) [2023-12-26 21:34:07,234][105692] Updated weights for policy 0, policy_version 852528 (0.0006) [2023-12-26 21:34:07,282][105692] Updated weights for policy 0, policy_version 852538 (0.0005) [2023-12-26 21:34:07,335][105692] Updated weights for policy 0, policy_version 852548 (0.0006) [2023-12-26 21:34:07,731][105620] Updated weights for policy 1, policy_version 852538 (0.0009) [2023-12-26 21:34:07,791][105620] Updated weights for policy 1, policy_version 852548 (0.0005) [2023-12-26 21:34:07,856][105620] Updated weights for policy 1, policy_version 852558 (0.0005) [2023-12-26 21:34:07,916][105620] Updated weights for policy 1, policy_version 852568 (0.0005) [2023-12-26 21:34:08,109][105692] Updated weights for policy 0, policy_version 852558 (0.0008) [2023-12-26 21:34:08,167][105692] Updated weights for policy 0, policy_version 852568 (0.0009) [2023-12-26 21:34:08,227][105692] Updated weights for policy 0, policy_version 852578 (0.0009) [2023-12-26 21:34:08,512][105620] Updated weights for policy 1, policy_version 852578 (0.0010) [2023-12-26 21:34:08,570][105620] Updated weights for policy 1, policy_version 852588 (0.0009) [2023-12-26 21:34:08,628][105620] Updated weights for policy 1, policy_version 852598 (0.0009) [2023-12-26 21:34:09,012][105692] Updated weights for policy 0, policy_version 852588 (0.0009) [2023-12-26 21:34:09,065][105692] Updated weights for policy 0, policy_version 852598 (0.0009) [2023-12-26 21:34:09,112][105692] Updated weights for policy 0, policy_version 852608 (0.0009) [2023-12-26 21:34:09,330][105620] Updated weights for policy 1, policy_version 852608 (0.0009) [2023-12-26 21:34:09,391][105620] Updated weights for policy 1, policy_version 852618 (0.0008) [2023-12-26 21:34:09,454][105620] Updated weights for policy 1, policy_version 852628 (0.0007) [2023-12-26 21:34:09,862][105692] Updated weights for policy 0, policy_version 852618 (0.0008) [2023-12-26 21:34:09,928][105692] Updated weights for policy 0, policy_version 852628 (0.0008) [2023-12-26 21:34:09,989][105692] Updated weights for policy 0, policy_version 852638 (0.0006) [2023-12-26 21:34:10,046][105692] Updated weights for policy 0, policy_version 852648 (0.0008) [2023-12-26 21:34:10,240][105620] Updated weights for policy 1, policy_version 852638 (0.0008) [2023-12-26 21:34:10,296][105620] Updated weights for policy 1, policy_version 852648 (0.0009) [2023-12-26 21:34:10,353][105620] Updated weights for policy 1, policy_version 852658 (0.0009) [2023-12-26 21:34:10,757][105692] Updated weights for policy 0, policy_version 852658 (0.0009) [2023-12-26 21:34:10,817][105692] Updated weights for policy 0, policy_version 852668 (0.0008) [2023-12-26 21:34:10,882][105692] Updated weights for policy 0, policy_version 852678 (0.0009) [2023-12-26 21:34:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 436625408. Throughput: 0: 10189.6, 1: 9761.6. Samples: 436633384. Policy #0 lag: (min: 29.0, avg: 36.8, max: 61.0) [2023-12-26 21:34:11,062][104569] Avg episode reward: [(0, '5548.800'), (1, '8987.347')] [2023-12-26 21:34:11,218][105620] Updated weights for policy 1, policy_version 852668 (0.0009) [2023-12-26 21:34:11,277][105620] Updated weights for policy 1, policy_version 852678 (0.0009) [2023-12-26 21:34:11,335][105620] Updated weights for policy 1, policy_version 852688 (0.0010) [2023-12-26 21:34:11,595][105692] Updated weights for policy 0, policy_version 852688 (0.0009) [2023-12-26 21:34:11,660][105692] Updated weights for policy 0, policy_version 852698 (0.0009) [2023-12-26 21:34:11,731][105692] Updated weights for policy 0, policy_version 852708 (0.0008) [2023-12-26 21:34:12,115][105620] Updated weights for policy 1, policy_version 852698 (0.0008) [2023-12-26 21:34:12,164][105620] Updated weights for policy 1, policy_version 852708 (0.0011) [2023-12-26 21:34:12,214][105620] Updated weights for policy 1, policy_version 852718 (0.0011) [2023-12-26 21:34:12,270][105620] Updated weights for policy 1, policy_version 852728 (0.0008) [2023-12-26 21:34:12,507][105692] Updated weights for policy 0, policy_version 852718 (0.0010) [2023-12-26 21:34:12,562][105692] Updated weights for policy 0, policy_version 852728 (0.0005) [2023-12-26 21:34:12,617][105692] Updated weights for policy 0, policy_version 852738 (0.0005) [2023-12-26 21:34:13,082][105620] Updated weights for policy 1, policy_version 852738 (0.0009) [2023-12-26 21:34:13,136][105620] Updated weights for policy 1, policy_version 852748 (0.0010) [2023-12-26 21:34:13,192][105620] Updated weights for policy 1, policy_version 852758 (0.0006) [2023-12-26 21:34:13,227][105692] Updated weights for policy 0, policy_version 852748 (0.0006) [2023-12-26 21:34:13,286][105692] Updated weights for policy 0, policy_version 852758 (0.0006) [2023-12-26 21:34:13,336][105692] Updated weights for policy 0, policy_version 852768 (0.0008) [2023-12-26 21:34:13,975][105692] Updated weights for policy 0, policy_version 852778 (0.0007) [2023-12-26 21:34:14,006][105620] Updated weights for policy 1, policy_version 852768 (0.0008) [2023-12-26 21:34:14,024][105692] Updated weights for policy 0, policy_version 852788 (0.0006) [2023-12-26 21:34:14,059][105620] Updated weights for policy 1, policy_version 852778 (0.0007) [2023-12-26 21:34:14,077][105692] Updated weights for policy 0, policy_version 852798 (0.0009) [2023-12-26 21:34:14,118][105620] Updated weights for policy 1, policy_version 852788 (0.0005) [2023-12-26 21:34:14,125][105692] Updated weights for policy 0, policy_version 852808 (0.0010) [2023-12-26 21:34:14,736][105692] Updated weights for policy 0, policy_version 852818 (0.0006) [2023-12-26 21:34:14,797][105692] Updated weights for policy 0, policy_version 852828 (0.0010) [2023-12-26 21:34:14,835][105620] Updated weights for policy 1, policy_version 852798 (0.0006) [2023-12-26 21:34:14,850][105692] Updated weights for policy 0, policy_version 852838 (0.0011) [2023-12-26 21:34:14,898][105620] Updated weights for policy 1, policy_version 852808 (0.0008) [2023-12-26 21:34:14,959][105620] Updated weights for policy 1, policy_version 852818 (0.0007) [2023-12-26 21:34:15,586][105692] Updated weights for policy 0, policy_version 852848 (0.0011) [2023-12-26 21:34:15,651][105692] Updated weights for policy 0, policy_version 852858 (0.0010) [2023-12-26 21:34:15,702][105692] Updated weights for policy 0, policy_version 852868 (0.0010) [2023-12-26 21:34:15,708][105620] Updated weights for policy 1, policy_version 852828 (0.0007) [2023-12-26 21:34:15,765][105620] Updated weights for policy 1, policy_version 852838 (0.0009) [2023-12-26 21:34:15,823][105620] Updated weights for policy 1, policy_version 852848 (0.0010) [2023-12-26 21:34:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 436723712. Throughput: 0: 10130.2, 1: 9626.3. Samples: 436689216. Policy #0 lag: (min: 29.0, avg: 36.8, max: 61.0) [2023-12-26 21:34:16,063][104569] Avg episode reward: [(0, '7545.697'), (1, '9078.443')] [2023-12-26 21:34:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000852872_218365952.pth... [2023-12-26 21:34:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000852856_218357760.pth... [2023-12-26 21:34:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000851688_218062848.pth [2023-12-26 21:34:16,092][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000851736_218071040.pth [2023-12-26 21:34:16,242][105692] Updated weights for policy 0, policy_version 852878 (0.0007) [2023-12-26 21:34:16,302][105692] Updated weights for policy 0, policy_version 852888 (0.0005) [2023-12-26 21:34:16,358][105692] Updated weights for policy 0, policy_version 852898 (0.0005) [2023-12-26 21:34:16,587][105620] Updated weights for policy 1, policy_version 852858 (0.0009) [2023-12-26 21:34:16,639][105620] Updated weights for policy 1, policy_version 852868 (0.0005) [2023-12-26 21:34:16,707][105620] Updated weights for policy 1, policy_version 852878 (0.0007) [2023-12-26 21:34:16,762][105620] Updated weights for policy 1, policy_version 852888 (0.0011) [2023-12-26 21:34:16,861][105692] Updated weights for policy 0, policy_version 852908 (0.0005) [2023-12-26 21:34:16,915][105692] Updated weights for policy 0, policy_version 852918 (0.0006) [2023-12-26 21:34:16,963][105692] Updated weights for policy 0, policy_version 852928 (0.0010) [2023-12-26 21:34:17,431][105620] Updated weights for policy 1, policy_version 852898 (0.0005) [2023-12-26 21:34:17,501][105620] Updated weights for policy 1, policy_version 852908 (0.0005) [2023-12-26 21:34:17,564][105620] Updated weights for policy 1, policy_version 852918 (0.0005) [2023-12-26 21:34:17,707][105692] Updated weights for policy 0, policy_version 852938 (0.0010) [2023-12-26 21:34:17,752][105692] Updated weights for policy 0, policy_version 852948 (0.0010) [2023-12-26 21:34:17,809][105692] Updated weights for policy 0, policy_version 852958 (0.0007) [2023-12-26 21:34:17,877][105692] Updated weights for policy 0, policy_version 852968 (0.0007) [2023-12-26 21:34:18,090][105620] Updated weights for policy 1, policy_version 852928 (0.0009) [2023-12-26 21:34:18,148][105620] Updated weights for policy 1, policy_version 852938 (0.0010) [2023-12-26 21:34:18,204][105620] Updated weights for policy 1, policy_version 852948 (0.0008) [2023-12-26 21:34:18,499][105692] Updated weights for policy 0, policy_version 852978 (0.0006) [2023-12-26 21:34:18,559][105692] Updated weights for policy 0, policy_version 852988 (0.0006) [2023-12-26 21:34:18,626][105692] Updated weights for policy 0, policy_version 852998 (0.0005) [2023-12-26 21:34:19,080][105620] Updated weights for policy 1, policy_version 852958 (0.0007) [2023-12-26 21:34:19,149][105620] Updated weights for policy 1, policy_version 852968 (0.0005) [2023-12-26 21:34:19,162][105692] Updated weights for policy 0, policy_version 853008 (0.0008) [2023-12-26 21:34:19,213][105692] Updated weights for policy 0, policy_version 853018 (0.0008) [2023-12-26 21:34:19,219][105620] Updated weights for policy 1, policy_version 852978 (0.0007) [2023-12-26 21:34:19,281][105692] Updated weights for policy 0, policy_version 853028 (0.0008) [2023-12-26 21:34:19,954][105692] Updated weights for policy 0, policy_version 853038 (0.0007) [2023-12-26 21:34:19,960][105620] Updated weights for policy 1, policy_version 852988 (0.0007) [2023-12-26 21:34:20,019][105620] Updated weights for policy 1, policy_version 852998 (0.0006) [2023-12-26 21:34:20,021][105692] Updated weights for policy 0, policy_version 853048 (0.0007) [2023-12-26 21:34:20,083][105692] Updated weights for policy 0, policy_version 853058 (0.0006) [2023-12-26 21:34:20,084][105620] Updated weights for policy 1, policy_version 853008 (0.0008) [2023-12-26 21:34:20,764][105692] Updated weights for policy 0, policy_version 853068 (0.0008) [2023-12-26 21:34:20,827][105692] Updated weights for policy 0, policy_version 853078 (0.0010) [2023-12-26 21:34:20,870][105620] Updated weights for policy 1, policy_version 853018 (0.0009) [2023-12-26 21:34:20,890][105692] Updated weights for policy 0, policy_version 853088 (0.0009) [2023-12-26 21:34:20,925][105620] Updated weights for policy 1, policy_version 853028 (0.0009) [2023-12-26 21:34:20,974][105620] Updated weights for policy 1, policy_version 853038 (0.0009) [2023-12-26 21:34:21,031][105620] Updated weights for policy 1, policy_version 853048 (0.0008) [2023-12-26 21:34:21,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19933.8, 300 sec: 19577.5). Total num frames: 436830208. Throughput: 0: 10247.5, 1: 9602.3. Samples: 436813192. Policy #0 lag: (min: 29.0, avg: 36.8, max: 61.0) [2023-12-26 21:34:21,065][104569] Avg episode reward: [(0, '5800.892'), (1, '9175.022')] [2023-12-26 21:34:21,664][105692] Updated weights for policy 0, policy_version 853098 (0.0008) [2023-12-26 21:34:21,736][105692] Updated weights for policy 0, policy_version 853108 (0.0009) [2023-12-26 21:34:21,801][105692] Updated weights for policy 0, policy_version 853118 (0.0008) [2023-12-26 21:34:21,854][105692] Updated weights for policy 0, policy_version 853128 (0.0008) [2023-12-26 21:34:21,877][105620] Updated weights for policy 1, policy_version 853058 (0.0006) [2023-12-26 21:34:21,948][105620] Updated weights for policy 1, policy_version 853068 (0.0006) [2023-12-26 21:34:22,014][105620] Updated weights for policy 1, policy_version 853078 (0.0007) [2023-12-26 21:34:22,650][105692] Updated weights for policy 0, policy_version 853138 (0.0011) [2023-12-26 21:34:22,716][105692] Updated weights for policy 0, policy_version 853148 (0.0011) [2023-12-26 21:34:22,748][105620] Updated weights for policy 1, policy_version 853088 (0.0006) [2023-12-26 21:34:22,776][105692] Updated weights for policy 0, policy_version 853158 (0.0010) [2023-12-26 21:34:22,801][105620] Updated weights for policy 1, policy_version 853098 (0.0005) [2023-12-26 21:34:22,858][105620] Updated weights for policy 1, policy_version 853108 (0.0005) [2023-12-26 21:34:23,508][105692] Updated weights for policy 0, policy_version 853168 (0.0010) [2023-12-26 21:34:23,565][105620] Updated weights for policy 1, policy_version 853118 (0.0008) [2023-12-26 21:34:23,566][105692] Updated weights for policy 0, policy_version 853178 (0.0008) [2023-12-26 21:34:23,621][105620] Updated weights for policy 1, policy_version 853128 (0.0010) [2023-12-26 21:34:23,655][105692] Updated weights for policy 0, policy_version 853188 (0.0007) [2023-12-26 21:34:23,679][105620] Updated weights for policy 1, policy_version 853138 (0.0010) [2023-12-26 21:34:24,201][105692] Updated weights for policy 0, policy_version 853198 (0.0006) [2023-12-26 21:34:24,251][105692] Updated weights for policy 0, policy_version 853208 (0.0010) [2023-12-26 21:34:24,298][105692] Updated weights for policy 0, policy_version 853218 (0.0010) [2023-12-26 21:34:24,328][105620] Updated weights for policy 1, policy_version 853148 (0.0009) [2023-12-26 21:34:24,386][105620] Updated weights for policy 1, policy_version 853158 (0.0005) [2023-12-26 21:34:24,451][105620] Updated weights for policy 1, policy_version 853168 (0.0008) [2023-12-26 21:34:24,870][105692] Updated weights for policy 0, policy_version 853228 (0.0005) [2023-12-26 21:34:24,924][105692] Updated weights for policy 0, policy_version 853238 (0.0009) [2023-12-26 21:34:24,979][105585] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000008 [2023-12-26 21:34:24,982][105692] Updated weights for policy 0, policy_version 853248 (0.0010) [2023-12-26 21:34:25,054][105620] Updated weights for policy 1, policy_version 853178 (0.0006) [2023-12-26 21:34:25,102][105620] Updated weights for policy 1, policy_version 853188 (0.0010) [2023-12-26 21:34:25,151][105620] Updated weights for policy 1, policy_version 853198 (0.0010) [2023-12-26 21:34:25,199][105620] Updated weights for policy 1, policy_version 853208 (0.0010) [2023-12-26 21:34:25,552][105692] Updated weights for policy 0, policy_version 853258 (0.0005) [2023-12-26 21:34:25,605][105692] Updated weights for policy 0, policy_version 853268 (0.0005) [2023-12-26 21:34:25,664][105692] Updated weights for policy 0, policy_version 853278 (0.0005) [2023-12-26 21:34:25,976][105620] Updated weights for policy 1, policy_version 853218 (0.0006) [2023-12-26 21:34:26,039][105620] Updated weights for policy 1, policy_version 853228 (0.0008) [2023-12-26 21:34:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 436920320. Throughput: 0: 10257.3, 1: 9604.5. Samples: 436932916. Policy #0 lag: (min: 29.0, avg: 36.8, max: 61.0) [2023-12-26 21:34:26,063][104569] Avg episode reward: [(0, '7427.047'), (1, '6934.361')] [2023-12-26 21:34:26,091][105620] Updated weights for policy 1, policy_version 853238 (0.0009) [2023-12-26 21:34:26,310][105692] Updated weights for policy 0, policy_version 853288 (0.0006) [2023-12-26 21:34:26,365][105692] Updated weights for policy 0, policy_version 853298 (0.0008) [2023-12-26 21:34:26,412][105692] Updated weights for policy 0, policy_version 853308 (0.0007) [2023-12-26 21:34:26,822][105620] Updated weights for policy 1, policy_version 853248 (0.0011) [2023-12-26 21:34:26,884][105620] Updated weights for policy 1, policy_version 853258 (0.0010) [2023-12-26 21:34:26,942][105620] Updated weights for policy 1, policy_version 853268 (0.0010) [2023-12-26 21:34:27,076][105692] Updated weights for policy 0, policy_version 853318 (0.0007) [2023-12-26 21:34:27,143][105692] Updated weights for policy 0, policy_version 853328 (0.0009) [2023-12-26 21:34:27,203][105692] Updated weights for policy 0, policy_version 853338 (0.0008) [2023-12-26 21:34:27,667][105620] Updated weights for policy 1, policy_version 853278 (0.0010) [2023-12-26 21:34:27,719][105620] Updated weights for policy 1, policy_version 853288 (0.0009) [2023-12-26 21:34:27,783][105620] Updated weights for policy 1, policy_version 853298 (0.0010) [2023-12-26 21:34:27,948][105692] Updated weights for policy 0, policy_version 853348 (0.0008) [2023-12-26 21:34:28,010][105692] Updated weights for policy 0, policy_version 853358 (0.0008) [2023-12-26 21:34:28,068][105692] Updated weights for policy 0, policy_version 853368 (0.0006) [2023-12-26 21:34:28,383][105620] Updated weights for policy 1, policy_version 853308 (0.0009) [2023-12-26 21:34:28,450][105620] Updated weights for policy 1, policy_version 853318 (0.0005) [2023-12-26 21:34:28,519][105620] Updated weights for policy 1, policy_version 853328 (0.0006) [2023-12-26 21:34:28,722][105692] Updated weights for policy 0, policy_version 853378 (0.0006) [2023-12-26 21:34:28,767][105692] Updated weights for policy 0, policy_version 853388 (0.0010) [2023-12-26 21:34:28,828][105692] Updated weights for policy 0, policy_version 853398 (0.0010) [2023-12-26 21:34:28,889][105692] Updated weights for policy 0, policy_version 853408 (0.0010) [2023-12-26 21:34:29,172][105620] Updated weights for policy 1, policy_version 853338 (0.0009) [2023-12-26 21:34:29,241][105620] Updated weights for policy 1, policy_version 853348 (0.0007) [2023-12-26 21:34:29,306][105620] Updated weights for policy 1, policy_version 853358 (0.0007) [2023-12-26 21:34:29,373][105620] Updated weights for policy 1, policy_version 853368 (0.0010) [2023-12-26 21:34:29,604][105692] Updated weights for policy 0, policy_version 853418 (0.0009) [2023-12-26 21:34:29,656][105692] Updated weights for policy 0, policy_version 853428 (0.0010) [2023-12-26 21:34:29,709][105692] Updated weights for policy 0, policy_version 853438 (0.0009) [2023-12-26 21:34:30,060][105620] Updated weights for policy 1, policy_version 853378 (0.0011) [2023-12-26 21:34:30,112][105620] Updated weights for policy 1, policy_version 853388 (0.0010) [2023-12-26 21:34:30,167][105620] Updated weights for policy 1, policy_version 853398 (0.0010) [2023-12-26 21:34:30,403][105692] Updated weights for policy 0, policy_version 853448 (0.0010) [2023-12-26 21:34:30,467][105692] Updated weights for policy 0, policy_version 853458 (0.0010) [2023-12-26 21:34:30,530][105692] Updated weights for policy 0, policy_version 853468 (0.0010) [2023-12-26 21:34:30,911][105620] Updated weights for policy 1, policy_version 853408 (0.0009) [2023-12-26 21:34:30,969][105620] Updated weights for policy 1, policy_version 853418 (0.0010) [2023-12-26 21:34:31,038][105620] Updated weights for policy 1, policy_version 853428 (0.0009) [2023-12-26 21:34:31,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 437026816. Throughput: 0: 10259.8, 1: 9601.3. Samples: 436993560. Policy #0 lag: (min: 29.0, avg: 36.8, max: 61.0) [2023-12-26 21:34:31,062][104569] Avg episode reward: [(0, '8534.030'), (1, '7490.602')] [2023-12-26 21:34:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000853472_218521600.pth... [2023-12-26 21:34:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000853432_218505216.pth... [2023-12-26 21:34:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000852312_218218496.pth [2023-12-26 21:34:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000852296_218218496.pth [2023-12-26 21:34:31,210][105692] Updated weights for policy 0, policy_version 853478 (0.0011) [2023-12-26 21:34:31,271][105692] Updated weights for policy 0, policy_version 853488 (0.0011) [2023-12-26 21:34:31,320][105692] Updated weights for policy 0, policy_version 853498 (0.0011) [2023-12-26 21:34:31,808][105620] Updated weights for policy 1, policy_version 853438 (0.0008) [2023-12-26 21:34:31,865][105620] Updated weights for policy 1, policy_version 853448 (0.0008) [2023-12-26 21:34:31,918][105620] Updated weights for policy 1, policy_version 853458 (0.0008) [2023-12-26 21:34:32,087][105692] Updated weights for policy 0, policy_version 853508 (0.0010) [2023-12-26 21:34:32,145][105692] Updated weights for policy 0, policy_version 853518 (0.0010) [2023-12-26 21:34:32,203][105692] Updated weights for policy 0, policy_version 853528 (0.0010) [2023-12-26 21:34:32,669][105620] Updated weights for policy 1, policy_version 853468 (0.0007) [2023-12-26 21:34:32,729][105620] Updated weights for policy 1, policy_version 853478 (0.0008) [2023-12-26 21:34:32,795][105620] Updated weights for policy 1, policy_version 853488 (0.0008) [2023-12-26 21:34:32,961][105692] Updated weights for policy 0, policy_version 853538 (0.0009) [2023-12-26 21:34:33,016][105692] Updated weights for policy 0, policy_version 853548 (0.0007) [2023-12-26 21:34:33,066][105692] Updated weights for policy 0, policy_version 853558 (0.0010) [2023-12-26 21:34:33,111][105692] Updated weights for policy 0, policy_version 853568 (0.0010) [2023-12-26 21:34:33,539][105620] Updated weights for policy 1, policy_version 853498 (0.0007) [2023-12-26 21:34:33,586][105620] Updated weights for policy 1, policy_version 853508 (0.0007) [2023-12-26 21:34:33,633][105620] Updated weights for policy 1, policy_version 853518 (0.0008) [2023-12-26 21:34:33,686][105620] Updated weights for policy 1, policy_version 853528 (0.0008) [2023-12-26 21:34:33,842][105692] Updated weights for policy 0, policy_version 853578 (0.0010) [2023-12-26 21:34:33,899][105692] Updated weights for policy 0, policy_version 853588 (0.0010) [2023-12-26 21:34:33,953][105692] Updated weights for policy 0, policy_version 853598 (0.0010) [2023-12-26 21:34:34,468][105620] Updated weights for policy 1, policy_version 853538 (0.0009) [2023-12-26 21:34:34,528][105620] Updated weights for policy 1, policy_version 853548 (0.0008) [2023-12-26 21:34:34,584][105620] Updated weights for policy 1, policy_version 853558 (0.0007) [2023-12-26 21:34:34,669][105692] Updated weights for policy 0, policy_version 853608 (0.0010) [2023-12-26 21:34:34,735][105692] Updated weights for policy 0, policy_version 853618 (0.0011) [2023-12-26 21:34:34,800][105692] Updated weights for policy 0, policy_version 853628 (0.0010) [2023-12-26 21:34:35,329][105620] Updated weights for policy 1, policy_version 853568 (0.0008) [2023-12-26 21:34:35,374][105620] Updated weights for policy 1, policy_version 853578 (0.0008) [2023-12-26 21:34:35,429][105620] Updated weights for policy 1, policy_version 853588 (0.0008) [2023-12-26 21:34:35,530][105692] Updated weights for policy 0, policy_version 853638 (0.0011) [2023-12-26 21:34:35,575][105692] Updated weights for policy 0, policy_version 853648 (0.0010) [2023-12-26 21:34:35,632][105692] Updated weights for policy 0, policy_version 853658 (0.0011) [2023-12-26 21:34:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 437116928. Throughput: 0: 10163.4, 1: 9469.3. Samples: 437108244. Policy #0 lag: (min: 29.0, avg: 36.8, max: 61.0) [2023-12-26 21:34:36,063][104569] Avg episode reward: [(0, '8984.031'), (1, '8553.565')] [2023-12-26 21:34:36,197][105620] Updated weights for policy 1, policy_version 853598 (0.0008) [2023-12-26 21:34:36,250][105620] Updated weights for policy 1, policy_version 853608 (0.0008) [2023-12-26 21:34:36,313][105620] Updated weights for policy 1, policy_version 853618 (0.0008) [2023-12-26 21:34:36,392][105692] Updated weights for policy 0, policy_version 853668 (0.0011) [2023-12-26 21:34:36,457][105692] Updated weights for policy 0, policy_version 853678 (0.0011) [2023-12-26 21:34:36,521][105692] Updated weights for policy 0, policy_version 853688 (0.0011) [2023-12-26 21:34:37,114][105620] Updated weights for policy 1, policy_version 853628 (0.0008) [2023-12-26 21:34:37,164][105620] Updated weights for policy 1, policy_version 853638 (0.0008) [2023-12-26 21:34:37,226][105620] Updated weights for policy 1, policy_version 853648 (0.0006) [2023-12-26 21:34:37,260][105692] Updated weights for policy 0, policy_version 853698 (0.0010) [2023-12-26 21:34:37,312][105692] Updated weights for policy 0, policy_version 853708 (0.0008) [2023-12-26 21:34:37,371][105692] Updated weights for policy 0, policy_version 853718 (0.0009) [2023-12-26 21:34:37,429][105692] Updated weights for policy 0, policy_version 853728 (0.0009) [2023-12-26 21:34:38,024][105692] Updated weights for policy 0, policy_version 853738 (0.0005) [2023-12-26 21:34:38,041][105620] Updated weights for policy 1, policy_version 853658 (0.0007) [2023-12-26 21:34:38,083][105692] Updated weights for policy 0, policy_version 853748 (0.0011) [2023-12-26 21:34:38,089][105620] Updated weights for policy 1, policy_version 853668 (0.0005) [2023-12-26 21:34:38,132][105692] Updated weights for policy 0, policy_version 853758 (0.0010) [2023-12-26 21:34:38,144][105620] Updated weights for policy 1, policy_version 853678 (0.0006) [2023-12-26 21:34:38,210][105620] Updated weights for policy 1, policy_version 853688 (0.0009) [2023-12-26 21:34:38,783][105692] Updated weights for policy 0, policy_version 853768 (0.0006) [2023-12-26 21:34:38,846][105692] Updated weights for policy 0, policy_version 853778 (0.0008) [2023-12-26 21:34:38,908][105692] Updated weights for policy 0, policy_version 853788 (0.0011) [2023-12-26 21:34:39,017][105620] Updated weights for policy 1, policy_version 853698 (0.0010) [2023-12-26 21:34:39,070][105620] Updated weights for policy 1, policy_version 853708 (0.0010) [2023-12-26 21:34:39,124][105620] Updated weights for policy 1, policy_version 853718 (0.0010) [2023-12-26 21:34:39,503][105692] Updated weights for policy 0, policy_version 853798 (0.0007) [2023-12-26 21:34:39,574][105692] Updated weights for policy 0, policy_version 853808 (0.0006) [2023-12-26 21:34:39,634][105692] Updated weights for policy 0, policy_version 853818 (0.0005) [2023-12-26 21:34:39,945][105620] Updated weights for policy 1, policy_version 853728 (0.0009) [2023-12-26 21:34:40,016][105620] Updated weights for policy 1, policy_version 853738 (0.0010) [2023-12-26 21:34:40,077][105620] Updated weights for policy 1, policy_version 853748 (0.0011) [2023-12-26 21:34:40,280][105692] Updated weights for policy 0, policy_version 853828 (0.0008) [2023-12-26 21:34:40,336][105692] Updated weights for policy 0, policy_version 853838 (0.0010) [2023-12-26 21:34:40,394][105692] Updated weights for policy 0, policy_version 853848 (0.0010) [2023-12-26 21:34:40,840][105620] Updated weights for policy 1, policy_version 853758 (0.0010) [2023-12-26 21:34:40,888][105620] Updated weights for policy 1, policy_version 853768 (0.0005) [2023-12-26 21:34:40,945][105620] Updated weights for policy 1, policy_version 853778 (0.0008) [2023-12-26 21:34:41,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 437215232. Throughput: 0: 10212.2, 1: 9363.6. Samples: 437222392. Policy #0 lag: (min: 29.0, avg: 36.8, max: 61.0) [2023-12-26 21:34:41,062][104569] Avg episode reward: [(0, '9076.520'), (1, '8986.835')] [2023-12-26 21:34:41,138][105692] Updated weights for policy 0, policy_version 853858 (0.0010) [2023-12-26 21:34:41,205][105692] Updated weights for policy 0, policy_version 853868 (0.0008) [2023-12-26 21:34:41,272][105692] Updated weights for policy 0, policy_version 853878 (0.0011) [2023-12-26 21:34:41,338][105692] Updated weights for policy 0, policy_version 853888 (0.0011) [2023-12-26 21:34:41,715][105620] Updated weights for policy 1, policy_version 853788 (0.0011) [2023-12-26 21:34:41,784][105620] Updated weights for policy 1, policy_version 853798 (0.0009) [2023-12-26 21:34:41,851][105620] Updated weights for policy 1, policy_version 853808 (0.0009) [2023-12-26 21:34:42,094][105692] Updated weights for policy 0, policy_version 853898 (0.0008) [2023-12-26 21:34:42,151][105692] Updated weights for policy 0, policy_version 853908 (0.0009) [2023-12-26 21:34:42,216][105692] Updated weights for policy 0, policy_version 853918 (0.0008) [2023-12-26 21:34:42,615][105620] Updated weights for policy 1, policy_version 853818 (0.0010) [2023-12-26 21:34:42,660][105620] Updated weights for policy 1, policy_version 853828 (0.0009) [2023-12-26 21:34:42,722][105620] Updated weights for policy 1, policy_version 853838 (0.0010) [2023-12-26 21:34:42,791][105620] Updated weights for policy 1, policy_version 853848 (0.0010) [2023-12-26 21:34:42,984][105692] Updated weights for policy 0, policy_version 853928 (0.0008) [2023-12-26 21:34:43,047][105692] Updated weights for policy 0, policy_version 853938 (0.0008) [2023-12-26 21:34:43,111][105692] Updated weights for policy 0, policy_version 853948 (0.0008) [2023-12-26 21:34:43,374][105620] Updated weights for policy 1, policy_version 853858 (0.0009) [2023-12-26 21:34:43,432][105620] Updated weights for policy 1, policy_version 853868 (0.0010) [2023-12-26 21:34:43,487][105620] Updated weights for policy 1, policy_version 853878 (0.0008) [2023-12-26 21:34:43,840][105692] Updated weights for policy 0, policy_version 853958 (0.0007) [2023-12-26 21:34:43,900][105692] Updated weights for policy 0, policy_version 853968 (0.0005) [2023-12-26 21:34:43,958][105692] Updated weights for policy 0, policy_version 853978 (0.0005) [2023-12-26 21:34:44,032][105620] Updated weights for policy 1, policy_version 853888 (0.0005) [2023-12-26 21:34:44,082][105620] Updated weights for policy 1, policy_version 853898 (0.0005) [2023-12-26 21:34:44,141][105620] Updated weights for policy 1, policy_version 853908 (0.0010) [2023-12-26 21:34:44,542][105692] Updated weights for policy 0, policy_version 853988 (0.0007) [2023-12-26 21:34:44,590][105692] Updated weights for policy 0, policy_version 853998 (0.0009) [2023-12-26 21:34:44,638][105692] Updated weights for policy 0, policy_version 854008 (0.0009) [2023-12-26 21:34:44,858][105620] Updated weights for policy 1, policy_version 853918 (0.0009) [2023-12-26 21:34:44,918][105620] Updated weights for policy 1, policy_version 853928 (0.0008) [2023-12-26 21:34:44,980][105620] Updated weights for policy 1, policy_version 853938 (0.0008) [2023-12-26 21:34:45,424][105692] Updated weights for policy 0, policy_version 854018 (0.0008) [2023-12-26 21:34:45,489][105692] Updated weights for policy 0, policy_version 854028 (0.0005) [2023-12-26 21:34:45,543][105692] Updated weights for policy 0, policy_version 854038 (0.0009) [2023-12-26 21:34:45,590][105692] Updated weights for policy 0, policy_version 854048 (0.0008) [2023-12-26 21:34:45,748][105620] Updated weights for policy 1, policy_version 853948 (0.0007) [2023-12-26 21:34:45,794][105620] Updated weights for policy 1, policy_version 853958 (0.0006) [2023-12-26 21:34:45,840][105620] Updated weights for policy 1, policy_version 853968 (0.0006) [2023-12-26 21:34:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19660.7, 300 sec: 19605.2). Total num frames: 437313536. Throughput: 0: 10074.8, 1: 9431.2. Samples: 437280552. Policy #0 lag: (min: 29.0, avg: 36.8, max: 61.0) [2023-12-26 21:34:46,063][104569] Avg episode reward: [(0, '9165.941'), (1, '9078.928')] [2023-12-26 21:34:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000853976_218644480.pth... [2023-12-26 21:34:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000854048_218669056.pth... [2023-12-26 21:34:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000852872_218365952.pth [2023-12-26 21:34:46,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000852856_218357760.pth [2023-12-26 21:34:46,356][105692] Updated weights for policy 0, policy_version 854058 (0.0006) [2023-12-26 21:34:46,407][105692] Updated weights for policy 0, policy_version 854068 (0.0007) [2023-12-26 21:34:46,461][105620] Updated weights for policy 1, policy_version 853978 (0.0006) [2023-12-26 21:34:46,464][105692] Updated weights for policy 0, policy_version 854078 (0.0008) [2023-12-26 21:34:46,507][105620] Updated weights for policy 1, policy_version 853988 (0.0008) [2023-12-26 21:34:46,561][105620] Updated weights for policy 1, policy_version 853998 (0.0005) [2023-12-26 21:34:46,610][105620] Updated weights for policy 1, policy_version 854008 (0.0005) [2023-12-26 21:34:47,229][105692] Updated weights for policy 0, policy_version 854088 (0.0009) [2023-12-26 21:34:47,286][105692] Updated weights for policy 0, policy_version 854098 (0.0009) [2023-12-26 21:34:47,302][105620] Updated weights for policy 1, policy_version 854018 (0.0006) [2023-12-26 21:34:47,345][105692] Updated weights for policy 0, policy_version 854108 (0.0008) [2023-12-26 21:34:47,350][105620] Updated weights for policy 1, policy_version 854028 (0.0005) [2023-12-26 21:34:47,397][105620] Updated weights for policy 1, policy_version 854038 (0.0005) [2023-12-26 21:34:48,008][105620] Updated weights for policy 1, policy_version 854048 (0.0006) [2023-12-26 21:34:48,046][105692] Updated weights for policy 0, policy_version 854118 (0.0007) [2023-12-26 21:34:48,076][105620] Updated weights for policy 1, policy_version 854058 (0.0006) [2023-12-26 21:34:48,106][105692] Updated weights for policy 0, policy_version 854128 (0.0005) [2023-12-26 21:34:48,124][105620] Updated weights for policy 1, policy_version 854068 (0.0006) [2023-12-26 21:34:48,163][105692] Updated weights for policy 0, policy_version 854138 (0.0006) [2023-12-26 21:34:48,827][105692] Updated weights for policy 0, policy_version 854148 (0.0007) [2023-12-26 21:34:48,838][105620] Updated weights for policy 1, policy_version 854078 (0.0008) [2023-12-26 21:34:48,886][105692] Updated weights for policy 0, policy_version 854158 (0.0009) [2023-12-26 21:34:48,900][105620] Updated weights for policy 1, policy_version 854088 (0.0007) [2023-12-26 21:34:48,944][105692] Updated weights for policy 0, policy_version 854168 (0.0007) [2023-12-26 21:34:48,960][105620] Updated weights for policy 1, policy_version 854098 (0.0009) [2023-12-26 21:34:49,652][105620] Updated weights for policy 1, policy_version 854108 (0.0008) [2023-12-26 21:34:49,656][105692] Updated weights for policy 0, policy_version 854178 (0.0010) [2023-12-26 21:34:49,709][105692] Updated weights for policy 0, policy_version 854188 (0.0008) [2023-12-26 21:34:49,711][105620] Updated weights for policy 1, policy_version 854118 (0.0008) [2023-12-26 21:34:49,757][105692] Updated weights for policy 0, policy_version 854198 (0.0008) [2023-12-26 21:34:49,771][105620] Updated weights for policy 1, policy_version 854128 (0.0007) [2023-12-26 21:34:49,810][105692] Updated weights for policy 0, policy_version 854208 (0.0006) [2023-12-26 21:34:50,503][105620] Updated weights for policy 1, policy_version 854138 (0.0008) [2023-12-26 21:34:50,565][105620] Updated weights for policy 1, policy_version 854148 (0.0008) [2023-12-26 21:34:50,625][105620] Updated weights for policy 1, policy_version 854158 (0.0007) [2023-12-26 21:34:50,636][105692] Updated weights for policy 0, policy_version 854218 (0.0007) [2023-12-26 21:34:50,686][105620] Updated weights for policy 1, policy_version 854168 (0.0009) [2023-12-26 21:34:50,695][105692] Updated weights for policy 0, policy_version 854228 (0.0007) [2023-12-26 21:34:50,745][105692] Updated weights for policy 0, policy_version 854238 (0.0009) [2023-12-26 21:34:51,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.7, 300 sec: 19605.2). Total num frames: 437411840. Throughput: 0: 10069.5, 1: 9551.4. Samples: 437400152. Policy #0 lag: (min: 29.0, avg: 36.8, max: 61.0) [2023-12-26 21:34:51,063][104569] Avg episode reward: [(0, '8984.533'), (1, '9258.337')] [2023-12-26 21:34:51,452][105620] Updated weights for policy 1, policy_version 854178 (0.0009) [2023-12-26 21:34:51,509][105620] Updated weights for policy 1, policy_version 854188 (0.0007) [2023-12-26 21:34:51,569][105620] Updated weights for policy 1, policy_version 854198 (0.0009) [2023-12-26 21:34:51,571][105692] Updated weights for policy 0, policy_version 854248 (0.0009) [2023-12-26 21:34:51,634][105692] Updated weights for policy 0, policy_version 854258 (0.0009) [2023-12-26 21:34:51,703][105692] Updated weights for policy 0, policy_version 854268 (0.0009) [2023-12-26 21:34:52,217][105620] Updated weights for policy 1, policy_version 854208 (0.0008) [2023-12-26 21:34:52,283][105620] Updated weights for policy 1, policy_version 854218 (0.0008) [2023-12-26 21:34:52,349][105620] Updated weights for policy 1, policy_version 854228 (0.0009) [2023-12-26 21:34:52,536][105692] Updated weights for policy 0, policy_version 854278 (0.0009) [2023-12-26 21:34:52,594][105692] Updated weights for policy 0, policy_version 854288 (0.0009) [2023-12-26 21:34:52,652][105692] Updated weights for policy 0, policy_version 854298 (0.0008) [2023-12-26 21:34:53,123][105620] Updated weights for policy 1, policy_version 854238 (0.0009) [2023-12-26 21:34:53,173][105620] Updated weights for policy 1, policy_version 854248 (0.0008) [2023-12-26 21:34:53,234][105620] Updated weights for policy 1, policy_version 854258 (0.0008) [2023-12-26 21:34:53,387][105692] Updated weights for policy 0, policy_version 854308 (0.0010) [2023-12-26 21:34:53,438][105692] Updated weights for policy 0, policy_version 854318 (0.0010) [2023-12-26 21:34:53,492][105692] Updated weights for policy 0, policy_version 854330 (0.0010) [2023-12-26 21:34:53,957][105620] Updated weights for policy 1, policy_version 854268 (0.0009) [2023-12-26 21:34:54,004][105620] Updated weights for policy 1, policy_version 854278 (0.0008) [2023-12-26 21:34:54,059][105620] Updated weights for policy 1, policy_version 854288 (0.0009) [2023-12-26 21:34:54,312][105692] Updated weights for policy 0, policy_version 854340 (0.0009) [2023-12-26 21:34:54,375][105692] Updated weights for policy 0, policy_version 854350 (0.0009) [2023-12-26 21:34:54,434][105692] Updated weights for policy 0, policy_version 854360 (0.0009) [2023-12-26 21:34:54,742][105620] Updated weights for policy 1, policy_version 854298 (0.0007) [2023-12-26 21:34:54,812][105620] Updated weights for policy 1, policy_version 854308 (0.0005) [2023-12-26 21:34:54,868][105620] Updated weights for policy 1, policy_version 854318 (0.0005) [2023-12-26 21:34:54,939][105620] Updated weights for policy 1, policy_version 854328 (0.0005) [2023-12-26 21:34:55,244][105692] Updated weights for policy 0, policy_version 854370 (0.0009) [2023-12-26 21:34:55,313][105692] Updated weights for policy 0, policy_version 854380 (0.0010) [2023-12-26 21:34:55,372][105692] Updated weights for policy 0, policy_version 854390 (0.0010) [2023-12-26 21:34:55,497][105620] Updated weights for policy 1, policy_version 854338 (0.0009) [2023-12-26 21:34:55,544][105620] Updated weights for policy 1, policy_version 854348 (0.0009) [2023-12-26 21:34:55,597][105620] Updated weights for policy 1, policy_version 854358 (0.0009) [2023-12-26 21:34:56,062][104569] Fps is (10 sec: 18842.4, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 437501952. Throughput: 0: 9914.4, 1: 9603.6. Samples: 437511692. Policy #0 lag: (min: 29.0, avg: 36.8, max: 61.0) [2023-12-26 21:34:56,062][104569] Avg episode reward: [(0, '8982.622'), (1, '8983.695')] [2023-12-26 21:34:56,151][105692] Updated weights for policy 0, policy_version 854401 (0.0009) [2023-12-26 21:34:56,202][105692] Updated weights for policy 0, policy_version 854411 (0.0009) [2023-12-26 21:34:56,250][105692] Updated weights for policy 0, policy_version 854421 (0.0009) [2023-12-26 21:34:56,296][105692] Updated weights for policy 0, policy_version 854431 (0.0009) [2023-12-26 21:34:56,373][105620] Updated weights for policy 1, policy_version 854369 (0.0009) [2023-12-26 21:34:56,419][105620] Updated weights for policy 1, policy_version 854379 (0.0009) [2023-12-26 21:34:56,474][105620] Updated weights for policy 1, policy_version 854389 (0.0009) [2023-12-26 21:34:57,070][105692] Updated weights for policy 0, policy_version 854441 (0.0009) [2023-12-26 21:34:57,130][105692] Updated weights for policy 0, policy_version 854451 (0.0009) [2023-12-26 21:34:57,189][105692] Updated weights for policy 0, policy_version 854461 (0.0009) [2023-12-26 21:34:57,239][105620] Updated weights for policy 1, policy_version 854399 (0.0009) [2023-12-26 21:34:57,303][105620] Updated weights for policy 1, policy_version 854409 (0.0009) [2023-12-26 21:34:57,362][105620] Updated weights for policy 1, policy_version 854419 (0.0007) [2023-12-26 21:34:57,988][105692] Updated weights for policy 0, policy_version 854471 (0.0009) [2023-12-26 21:34:58,035][105692] Updated weights for policy 0, policy_version 854481 (0.0009) [2023-12-26 21:34:58,049][105620] Updated weights for policy 1, policy_version 854429 (0.0007) [2023-12-26 21:34:58,083][105692] Updated weights for policy 0, policy_version 854491 (0.0006) [2023-12-26 21:34:58,107][105620] Updated weights for policy 1, policy_version 854439 (0.0008) [2023-12-26 21:34:58,172][105620] Updated weights for policy 1, policy_version 854449 (0.0009) [2023-12-26 21:34:58,907][105692] Updated weights for policy 0, policy_version 854501 (0.0009) [2023-12-26 21:34:58,957][105620] Updated weights for policy 1, policy_version 854459 (0.0008) [2023-12-26 21:34:58,962][105692] Updated weights for policy 0, policy_version 854511 (0.0009) [2023-12-26 21:34:59,005][105620] Updated weights for policy 1, policy_version 854469 (0.0005) [2023-12-26 21:34:59,015][105692] Updated weights for policy 0, policy_version 854521 (0.0010) [2023-12-26 21:34:59,061][105620] Updated weights for policy 1, policy_version 854479 (0.0006) [2023-12-26 21:34:59,760][105692] Updated weights for policy 0, policy_version 854531 (0.0010) [2023-12-26 21:34:59,808][105692] Updated weights for policy 0, policy_version 854541 (0.0010) [2023-12-26 21:34:59,871][105692] Updated weights for policy 0, policy_version 854551 (0.0010) [2023-12-26 21:34:59,876][105620] Updated weights for policy 1, policy_version 854489 (0.0008) [2023-12-26 21:34:59,940][105620] Updated weights for policy 1, policy_version 854499 (0.0008) [2023-12-26 21:34:59,992][105620] Updated weights for policy 1, policy_version 854509 (0.0008) [2023-12-26 21:35:00,048][105620] Updated weights for policy 1, policy_version 854519 (0.0008) [2023-12-26 21:35:00,629][105692] Updated weights for policy 0, policy_version 854562 (0.0010) [2023-12-26 21:35:00,687][105692] Updated weights for policy 0, policy_version 854572 (0.0010) [2023-12-26 21:35:00,742][105692] Updated weights for policy 0, policy_version 854582 (0.0008) [2023-12-26 21:35:00,830][105620] Updated weights for policy 1, policy_version 854529 (0.0007) [2023-12-26 21:35:00,896][105620] Updated weights for policy 1, policy_version 854539 (0.0005) [2023-12-26 21:35:00,962][105620] Updated weights for policy 1, policy_version 854549 (0.0006) [2023-12-26 21:35:01,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 437600256. Throughput: 0: 9870.0, 1: 9647.8. Samples: 437567516. Policy #0 lag: (min: 29.0, avg: 36.8, max: 61.0) [2023-12-26 21:35:01,063][104569] Avg episode reward: [(0, '9340.300'), (1, '9076.671')] [2023-12-26 21:35:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000854592_218808320.pth... [2023-12-26 21:35:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000854552_218791936.pth... [2023-12-26 21:35:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000853472_218521600.pth [2023-12-26 21:35:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000853432_218505216.pth [2023-12-26 21:35:01,514][105692] Updated weights for policy 0, policy_version 854593 (0.0010) [2023-12-26 21:35:01,571][105692] Updated weights for policy 0, policy_version 854603 (0.0009) [2023-12-26 21:35:01,613][105620] Updated weights for policy 1, policy_version 854559 (0.0008) [2023-12-26 21:35:01,631][105692] Updated weights for policy 0, policy_version 854613 (0.0006) [2023-12-26 21:35:01,682][105620] Updated weights for policy 1, policy_version 854569 (0.0008) [2023-12-26 21:35:01,689][105692] Updated weights for policy 0, policy_version 854623 (0.0006) [2023-12-26 21:35:01,750][105620] Updated weights for policy 1, policy_version 854579 (0.0010) [2023-12-26 21:35:02,435][105692] Updated weights for policy 0, policy_version 854633 (0.0009) [2023-12-26 21:35:02,485][105692] Updated weights for policy 0, policy_version 854643 (0.0009) [2023-12-26 21:35:02,507][105620] Updated weights for policy 1, policy_version 854589 (0.0007) [2023-12-26 21:35:02,536][105692] Updated weights for policy 0, policy_version 854653 (0.0006) [2023-12-26 21:35:02,570][105620] Updated weights for policy 1, policy_version 854599 (0.0009) [2023-12-26 21:35:02,623][105620] Updated weights for policy 1, policy_version 854609 (0.0008) [2023-12-26 21:35:03,222][105692] Updated weights for policy 0, policy_version 854663 (0.0006) [2023-12-26 21:35:03,274][105692] Updated weights for policy 0, policy_version 854673 (0.0005) [2023-12-26 21:35:03,324][105692] Updated weights for policy 0, policy_version 854683 (0.0005) [2023-12-26 21:35:03,381][105620] Updated weights for policy 1, policy_version 854619 (0.0009) [2023-12-26 21:35:03,431][105620] Updated weights for policy 1, policy_version 854629 (0.0009) [2023-12-26 21:35:03,489][105620] Updated weights for policy 1, policy_version 854639 (0.0009) [2023-12-26 21:35:03,931][105692] Updated weights for policy 0, policy_version 854693 (0.0007) [2023-12-26 21:35:03,993][105692] Updated weights for policy 0, policy_version 854703 (0.0009) [2023-12-26 21:35:04,055][105692] Updated weights for policy 0, policy_version 854713 (0.0009) [2023-12-26 21:35:04,271][105620] Updated weights for policy 1, policy_version 854649 (0.0009) [2023-12-26 21:35:04,326][105620] Updated weights for policy 1, policy_version 854659 (0.0009) [2023-12-26 21:35:04,382][105620] Updated weights for policy 1, policy_version 854669 (0.0009) [2023-12-26 21:35:04,440][105620] Updated weights for policy 1, policy_version 854679 (0.0009) [2023-12-26 21:35:04,789][105692] Updated weights for policy 0, policy_version 854723 (0.0008) [2023-12-26 21:35:04,840][105692] Updated weights for policy 0, policy_version 854733 (0.0005) [2023-12-26 21:35:04,893][105692] Updated weights for policy 0, policy_version 854743 (0.0005) [2023-12-26 21:35:05,313][105620] Updated weights for policy 1, policy_version 854689 (0.0009) [2023-12-26 21:35:05,357][105620] Updated weights for policy 1, policy_version 854699 (0.0006) [2023-12-26 21:35:05,412][105620] Updated weights for policy 1, policy_version 854709 (0.0005) [2023-12-26 21:35:05,433][105692] Updated weights for policy 0, policy_version 854753 (0.0006) [2023-12-26 21:35:05,491][105692] Updated weights for policy 0, policy_version 854763 (0.0010) [2023-12-26 21:35:05,556][105692] Updated weights for policy 0, policy_version 854773 (0.0010) [2023-12-26 21:35:05,628][105692] Updated weights for policy 0, policy_version 854783 (0.0010) [2023-12-26 21:35:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 437690368. Throughput: 0: 9679.9, 1: 9594.6. Samples: 437680544. Policy #0 lag: (min: 29.0, avg: 36.8, max: 61.0) [2023-12-26 21:35:06,062][104569] Avg episode reward: [(0, '9247.865'), (1, '9352.412')] [2023-12-26 21:35:06,086][105620] Updated weights for policy 1, policy_version 854719 (0.0005) [2023-12-26 21:35:06,152][105620] Updated weights for policy 1, policy_version 854729 (0.0007) [2023-12-26 21:35:06,216][105620] Updated weights for policy 1, policy_version 854739 (0.0007) [2023-12-26 21:35:06,250][105692] Updated weights for policy 0, policy_version 854793 (0.0010) [2023-12-26 21:35:06,312][105692] Updated weights for policy 0, policy_version 854803 (0.0010) [2023-12-26 21:35:06,371][105692] Updated weights for policy 0, policy_version 854813 (0.0010) [2023-12-26 21:35:06,919][105620] Updated weights for policy 1, policy_version 854749 (0.0008) [2023-12-26 21:35:06,991][105620] Updated weights for policy 1, policy_version 854759 (0.0010) [2023-12-26 21:35:07,050][105620] Updated weights for policy 1, policy_version 854769 (0.0009) [2023-12-26 21:35:07,103][105692] Updated weights for policy 0, policy_version 854823 (0.0010) [2023-12-26 21:35:07,177][105692] Updated weights for policy 0, policy_version 854833 (0.0010) [2023-12-26 21:35:07,243][105692] Updated weights for policy 0, policy_version 854843 (0.0009) [2023-12-26 21:35:07,631][105620] Updated weights for policy 1, policy_version 854779 (0.0008) [2023-12-26 21:35:07,686][105620] Updated weights for policy 1, policy_version 854789 (0.0009) [2023-12-26 21:35:07,743][105620] Updated weights for policy 1, policy_version 854799 (0.0010) [2023-12-26 21:35:07,957][105692] Updated weights for policy 0, policy_version 854853 (0.0007) [2023-12-26 21:35:08,019][105692] Updated weights for policy 0, policy_version 854863 (0.0007) [2023-12-26 21:35:08,071][105692] Updated weights for policy 0, policy_version 854873 (0.0009) [2023-12-26 21:35:08,436][105620] Updated weights for policy 1, policy_version 854809 (0.0010) [2023-12-26 21:35:08,485][105620] Updated weights for policy 1, policy_version 854819 (0.0008) [2023-12-26 21:35:08,538][105620] Updated weights for policy 1, policy_version 854829 (0.0005) [2023-12-26 21:35:08,603][105620] Updated weights for policy 1, policy_version 854839 (0.0005) [2023-12-26 21:35:08,722][105692] Updated weights for policy 0, policy_version 854883 (0.0009) [2023-12-26 21:35:08,788][105692] Updated weights for policy 0, policy_version 854893 (0.0011) [2023-12-26 21:35:08,848][105692] Updated weights for policy 0, policy_version 854903 (0.0011) [2023-12-26 21:35:09,180][105620] Updated weights for policy 1, policy_version 854849 (0.0006) [2023-12-26 21:35:09,246][105620] Updated weights for policy 1, policy_version 854859 (0.0008) [2023-12-26 21:35:09,305][105620] Updated weights for policy 1, policy_version 854869 (0.0010) [2023-12-26 21:35:09,524][105692] Updated weights for policy 0, policy_version 854913 (0.0011) [2023-12-26 21:35:09,572][105692] Updated weights for policy 0, policy_version 854923 (0.0010) [2023-12-26 21:35:09,624][105692] Updated weights for policy 0, policy_version 854933 (0.0011) [2023-12-26 21:35:09,684][105692] Updated weights for policy 0, policy_version 854943 (0.0010) [2023-12-26 21:35:10,039][105620] Updated weights for policy 1, policy_version 854879 (0.0008) [2023-12-26 21:35:10,087][105620] Updated weights for policy 1, policy_version 854889 (0.0007) [2023-12-26 21:35:10,139][105620] Updated weights for policy 1, policy_version 854899 (0.0008) [2023-12-26 21:35:10,414][105692] Updated weights for policy 0, policy_version 854953 (0.0011) [2023-12-26 21:35:10,479][105692] Updated weights for policy 0, policy_version 854963 (0.0010) [2023-12-26 21:35:10,543][105692] Updated weights for policy 0, policy_version 854973 (0.0011) [2023-12-26 21:35:10,779][105620] Updated weights for policy 1, policy_version 854909 (0.0007) [2023-12-26 21:35:10,845][105620] Updated weights for policy 1, policy_version 854919 (0.0006) [2023-12-26 21:35:10,912][105620] Updated weights for policy 1, policy_version 854929 (0.0006) [2023-12-26 21:35:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 437796864. Throughput: 0: 9673.5, 1: 9681.1. Samples: 437803872. Policy #0 lag: (min: 29.0, avg: 36.8, max: 61.0) [2023-12-26 21:35:11,062][104569] Avg episode reward: [(0, '9001.201'), (1, '9077.137')] [2023-12-26 21:35:11,278][105692] Updated weights for policy 0, policy_version 854983 (0.0010) [2023-12-26 21:35:11,344][105692] Updated weights for policy 0, policy_version 854993 (0.0010) [2023-12-26 21:35:11,411][105692] Updated weights for policy 0, policy_version 855003 (0.0009) [2023-12-26 21:35:11,629][105620] Updated weights for policy 1, policy_version 854939 (0.0007) [2023-12-26 21:35:11,695][105620] Updated weights for policy 1, policy_version 854949 (0.0007) [2023-12-26 21:35:11,753][105620] Updated weights for policy 1, policy_version 854959 (0.0008) [2023-12-26 21:35:12,129][105692] Updated weights for policy 0, policy_version 855013 (0.0008) [2023-12-26 21:35:12,187][105692] Updated weights for policy 0, policy_version 855023 (0.0009) [2023-12-26 21:35:12,248][105692] Updated weights for policy 0, policy_version 855033 (0.0008) [2023-12-26 21:35:12,558][105620] Updated weights for policy 1, policy_version 854969 (0.0008) [2023-12-26 21:35:12,607][105620] Updated weights for policy 1, policy_version 854979 (0.0010) [2023-12-26 21:35:12,654][105620] Updated weights for policy 1, policy_version 854989 (0.0010) [2023-12-26 21:35:12,714][105620] Updated weights for policy 1, policy_version 854999 (0.0010) [2023-12-26 21:35:12,995][105692] Updated weights for policy 0, policy_version 855043 (0.0008) [2023-12-26 21:35:13,052][105692] Updated weights for policy 0, policy_version 855053 (0.0006) [2023-12-26 21:35:13,111][105692] Updated weights for policy 0, policy_version 855063 (0.0006) [2023-12-26 21:35:13,153][105585] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000002 [2023-12-26 21:35:13,442][105620] Updated weights for policy 1, policy_version 855009 (0.0011) [2023-12-26 21:35:13,501][105620] Updated weights for policy 1, policy_version 855019 (0.0010) [2023-12-26 21:35:13,556][105620] Updated weights for policy 1, policy_version 855029 (0.0011) [2023-12-26 21:35:13,812][105692] Updated weights for policy 0, policy_version 855073 (0.0008) [2023-12-26 21:35:13,860][105692] Updated weights for policy 0, policy_version 855083 (0.0008) [2023-12-26 21:35:13,907][105692] Updated weights for policy 0, policy_version 855093 (0.0008) [2023-12-26 21:35:13,962][105692] Updated weights for policy 0, policy_version 855103 (0.0009) [2023-12-26 21:35:14,229][105620] Updated weights for policy 1, policy_version 855039 (0.0010) [2023-12-26 21:35:14,283][105620] Updated weights for policy 1, policy_version 855049 (0.0010) [2023-12-26 21:35:14,334][105620] Updated weights for policy 1, policy_version 855059 (0.0010) [2023-12-26 21:35:14,689][105692] Updated weights for policy 0, policy_version 855113 (0.0006) [2023-12-26 21:35:14,755][105692] Updated weights for policy 0, policy_version 855123 (0.0006) [2023-12-26 21:35:14,816][105692] Updated weights for policy 0, policy_version 855133 (0.0007) [2023-12-26 21:35:15,093][105620] Updated weights for policy 1, policy_version 855069 (0.0010) [2023-12-26 21:35:15,155][105620] Updated weights for policy 1, policy_version 855079 (0.0010) [2023-12-26 21:35:15,217][105620] Updated weights for policy 1, policy_version 855089 (0.0011) [2023-12-26 21:35:15,505][105692] Updated weights for policy 0, policy_version 855143 (0.0009) [2023-12-26 21:35:15,553][105692] Updated weights for policy 0, policy_version 855153 (0.0010) [2023-12-26 21:35:15,601][105692] Updated weights for policy 0, policy_version 855163 (0.0010) [2023-12-26 21:35:15,941][105620] Updated weights for policy 1, policy_version 855099 (0.0007) [2023-12-26 21:35:16,000][105620] Updated weights for policy 1, policy_version 855109 (0.0010) [2023-12-26 21:35:16,059][105620] Updated weights for policy 1, policy_version 855119 (0.0010) [2023-12-26 21:35:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 437886976. Throughput: 0: 9625.6, 1: 9624.1. Samples: 437859796. Policy #0 lag: (min: 29.0, avg: 36.8, max: 61.0) [2023-12-26 21:35:16,062][104569] Avg episode reward: [(0, '8913.744'), (1, '9077.175')] [2023-12-26 21:35:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000855168_218955776.pth... [2023-12-26 21:35:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000854048_218669056.pth [2023-12-26 21:35:16,109][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000855128_218939392.pth... [2023-12-26 21:35:16,113][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000853976_218644480.pth [2023-12-26 21:35:16,296][105692] Updated weights for policy 0, policy_version 855173 (0.0008) [2023-12-26 21:35:16,347][105692] Updated weights for policy 0, policy_version 855183 (0.0005) [2023-12-26 21:35:16,409][105692] Updated weights for policy 0, policy_version 855193 (0.0005) [2023-12-26 21:35:16,799][105620] Updated weights for policy 1, policy_version 855129 (0.0010) [2023-12-26 21:35:16,867][105620] Updated weights for policy 1, policy_version 855139 (0.0010) [2023-12-26 21:35:16,928][105620] Updated weights for policy 1, policy_version 855149 (0.0010) [2023-12-26 21:35:16,948][105692] Updated weights for policy 0, policy_version 855203 (0.0007) [2023-12-26 21:35:16,977][105620] Updated weights for policy 1, policy_version 855159 (0.0010) [2023-12-26 21:35:17,007][105692] Updated weights for policy 0, policy_version 855213 (0.0005) [2023-12-26 21:35:17,063][105692] Updated weights for policy 0, policy_version 855223 (0.0008) [2023-12-26 21:35:17,657][105692] Updated weights for policy 0, policy_version 855233 (0.0005) [2023-12-26 21:35:17,707][105620] Updated weights for policy 1, policy_version 855169 (0.0010) [2023-12-26 21:35:17,724][105692] Updated weights for policy 0, policy_version 855243 (0.0006) [2023-12-26 21:35:17,758][105620] Updated weights for policy 1, policy_version 855179 (0.0010) [2023-12-26 21:35:17,784][105692] Updated weights for policy 0, policy_version 855253 (0.0006) [2023-12-26 21:35:17,809][105620] Updated weights for policy 1, policy_version 855189 (0.0010) [2023-12-26 21:35:17,836][105692] Updated weights for policy 0, policy_version 855263 (0.0007) [2023-12-26 21:35:18,510][105692] Updated weights for policy 0, policy_version 855273 (0.0008) [2023-12-26 21:35:18,563][105620] Updated weights for policy 1, policy_version 855199 (0.0010) [2023-12-26 21:35:18,564][105692] Updated weights for policy 0, policy_version 855283 (0.0009) [2023-12-26 21:35:18,625][105620] Updated weights for policy 1, policy_version 855209 (0.0011) [2023-12-26 21:35:18,631][105692] Updated weights for policy 0, policy_version 855293 (0.0008) [2023-12-26 21:35:18,681][105620] Updated weights for policy 1, policy_version 855219 (0.0010) [2023-12-26 21:35:19,442][105692] Updated weights for policy 0, policy_version 855303 (0.0006) [2023-12-26 21:35:19,454][105620] Updated weights for policy 1, policy_version 855229 (0.0011) [2023-12-26 21:35:19,502][105692] Updated weights for policy 0, policy_version 855313 (0.0010) [2023-12-26 21:35:19,516][105620] Updated weights for policy 1, policy_version 855239 (0.0010) [2023-12-26 21:35:19,556][105692] Updated weights for policy 0, policy_version 855323 (0.0006) [2023-12-26 21:35:19,578][105620] Updated weights for policy 1, policy_version 855249 (0.0010) [2023-12-26 21:35:20,230][105620] Updated weights for policy 1, policy_version 855259 (0.0009) [2023-12-26 21:35:20,296][105620] Updated weights for policy 1, policy_version 855269 (0.0009) [2023-12-26 21:35:20,359][105620] Updated weights for policy 1, policy_version 855279 (0.0009) [2023-12-26 21:35:20,395][105692] Updated weights for policy 0, policy_version 855333 (0.0007) [2023-12-26 21:35:20,449][105692] Updated weights for policy 0, policy_version 855343 (0.0008) [2023-12-26 21:35:20,505][105692] Updated weights for policy 0, policy_version 855353 (0.0009) [2023-12-26 21:35:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 437985280. Throughput: 0: 9688.4, 1: 9636.8. Samples: 437977876. Policy #0 lag: (min: 29.0, avg: 36.8, max: 61.0) [2023-12-26 21:35:21,062][104569] Avg episode reward: [(0, '8733.668'), (1, '9170.443')] [2023-12-26 21:35:21,085][105620] Updated weights for policy 1, policy_version 855289 (0.0009) [2023-12-26 21:35:21,153][105620] Updated weights for policy 1, policy_version 855299 (0.0008) [2023-12-26 21:35:21,219][105620] Updated weights for policy 1, policy_version 855309 (0.0009) [2023-12-26 21:35:21,279][105620] Updated weights for policy 1, policy_version 855319 (0.0009) [2023-12-26 21:35:21,304][105692] Updated weights for policy 0, policy_version 855363 (0.0009) [2023-12-26 21:35:21,376][105692] Updated weights for policy 0, policy_version 855373 (0.0008) [2023-12-26 21:35:21,438][105692] Updated weights for policy 0, policy_version 855383 (0.0007) [2023-12-26 21:35:22,109][105620] Updated weights for policy 1, policy_version 855329 (0.0009) [2023-12-26 21:35:22,172][105620] Updated weights for policy 1, policy_version 855339 (0.0010) [2023-12-26 21:35:22,196][105692] Updated weights for policy 0, policy_version 855393 (0.0007) [2023-12-26 21:35:22,231][105620] Updated weights for policy 1, policy_version 855349 (0.0008) [2023-12-26 21:35:22,259][105692] Updated weights for policy 0, policy_version 855403 (0.0008) [2023-12-26 21:35:22,323][105692] Updated weights for policy 0, policy_version 855413 (0.0009) [2023-12-26 21:35:22,383][105692] Updated weights for policy 0, policy_version 855423 (0.0009) [2023-12-26 21:35:23,015][105620] Updated weights for policy 1, policy_version 855359 (0.0008) [2023-12-26 21:35:23,070][105620] Updated weights for policy 1, policy_version 855369 (0.0009) [2023-12-26 21:35:23,116][105692] Updated weights for policy 0, policy_version 855433 (0.0007) [2023-12-26 21:35:23,118][105620] Updated weights for policy 1, policy_version 855379 (0.0008) [2023-12-26 21:35:23,168][105692] Updated weights for policy 0, policy_version 855443 (0.0008) [2023-12-26 21:35:23,214][105692] Updated weights for policy 0, policy_version 855453 (0.0008) [2023-12-26 21:35:23,829][105620] Updated weights for policy 1, policy_version 855389 (0.0006) [2023-12-26 21:35:23,887][105620] Updated weights for policy 1, policy_version 855399 (0.0007) [2023-12-26 21:35:23,907][105692] Updated weights for policy 0, policy_version 855463 (0.0009) [2023-12-26 21:35:23,943][105620] Updated weights for policy 1, policy_version 855409 (0.0007) [2023-12-26 21:35:23,959][105692] Updated weights for policy 0, policy_version 855473 (0.0006) [2023-12-26 21:35:24,012][105692] Updated weights for policy 0, policy_version 855483 (0.0009) [2023-12-26 21:35:24,643][105620] Updated weights for policy 1, policy_version 855419 (0.0007) [2023-12-26 21:35:24,681][105692] Updated weights for policy 0, policy_version 855493 (0.0009) [2023-12-26 21:35:24,700][105620] Updated weights for policy 1, policy_version 855429 (0.0005) [2023-12-26 21:35:24,738][105692] Updated weights for policy 0, policy_version 855503 (0.0009) [2023-12-26 21:35:24,744][105620] Updated weights for policy 1, policy_version 855439 (0.0005) [2023-12-26 21:35:24,792][105692] Updated weights for policy 0, policy_version 855513 (0.0007) [2023-12-26 21:35:25,357][105620] Updated weights for policy 1, policy_version 855449 (0.0007) [2023-12-26 21:35:25,412][105620] Updated weights for policy 1, policy_version 855459 (0.0006) [2023-12-26 21:35:25,470][105620] Updated weights for policy 1, policy_version 855469 (0.0006) [2023-12-26 21:35:25,537][105620] Updated weights for policy 1, policy_version 855479 (0.0009) [2023-12-26 21:35:25,590][105692] Updated weights for policy 0, policy_version 855523 (0.0008) [2023-12-26 21:35:25,642][105692] Updated weights for policy 0, policy_version 855533 (0.0006) [2023-12-26 21:35:25,695][105692] Updated weights for policy 0, policy_version 855543 (0.0005) [2023-12-26 21:35:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 438083584. Throughput: 0: 9586.5, 1: 9739.3. Samples: 438092060. Policy #0 lag: (min: 29.0, avg: 36.8, max: 61.0) [2023-12-26 21:35:26,063][104569] Avg episode reward: [(0, '9071.378'), (1, '9057.230')] [2023-12-26 21:35:26,207][105620] Updated weights for policy 1, policy_version 855489 (0.0008) [2023-12-26 21:35:26,255][105620] Updated weights for policy 1, policy_version 855499 (0.0008) [2023-12-26 21:35:26,310][105620] Updated weights for policy 1, policy_version 855509 (0.0008) [2023-12-26 21:35:26,446][105692] Updated weights for policy 0, policy_version 855553 (0.0009) [2023-12-26 21:35:26,501][105692] Updated weights for policy 0, policy_version 855563 (0.0010) [2023-12-26 21:35:26,555][105692] Updated weights for policy 0, policy_version 855573 (0.0010) [2023-12-26 21:35:26,612][105692] Updated weights for policy 0, policy_version 855583 (0.0010) [2023-12-26 21:35:27,059][105620] Updated weights for policy 1, policy_version 855519 (0.0008) [2023-12-26 21:35:27,107][105620] Updated weights for policy 1, policy_version 855529 (0.0007) [2023-12-26 21:35:27,155][105620] Updated weights for policy 1, policy_version 855539 (0.0008) [2023-12-26 21:35:27,355][105692] Updated weights for policy 0, policy_version 855593 (0.0010) [2023-12-26 21:35:27,406][105692] Updated weights for policy 0, policy_version 855603 (0.0010) [2023-12-26 21:35:27,460][105692] Updated weights for policy 0, policy_version 855613 (0.0010) [2023-12-26 21:35:27,933][105620] Updated weights for policy 1, policy_version 855549 (0.0008) [2023-12-26 21:35:27,981][105620] Updated weights for policy 1, policy_version 855559 (0.0008) [2023-12-26 21:35:28,029][105620] Updated weights for policy 1, policy_version 855569 (0.0008) [2023-12-26 21:35:28,217][105692] Updated weights for policy 0, policy_version 855623 (0.0010) [2023-12-26 21:35:28,286][105692] Updated weights for policy 0, policy_version 855633 (0.0010) [2023-12-26 21:35:28,348][105692] Updated weights for policy 0, policy_version 855643 (0.0011) [2023-12-26 21:35:28,816][105620] Updated weights for policy 1, policy_version 855579 (0.0008) [2023-12-26 21:35:28,868][105620] Updated weights for policy 1, policy_version 855589 (0.0008) [2023-12-26 21:35:28,916][105620] Updated weights for policy 1, policy_version 855599 (0.0008) [2023-12-26 21:35:29,078][105692] Updated weights for policy 0, policy_version 855653 (0.0011) [2023-12-26 21:35:29,137][105692] Updated weights for policy 0, policy_version 855663 (0.0011) [2023-12-26 21:35:29,193][105692] Updated weights for policy 0, policy_version 855673 (0.0010) [2023-12-26 21:35:29,615][105620] Updated weights for policy 1, policy_version 855609 (0.0008) [2023-12-26 21:35:29,673][105620] Updated weights for policy 1, policy_version 855619 (0.0007) [2023-12-26 21:35:29,729][105620] Updated weights for policy 1, policy_version 855629 (0.0009) [2023-12-26 21:35:29,782][105620] Updated weights for policy 1, policy_version 855639 (0.0009) [2023-12-26 21:35:29,895][105692] Updated weights for policy 0, policy_version 855683 (0.0008) [2023-12-26 21:35:29,953][105692] Updated weights for policy 0, policy_version 855693 (0.0010) [2023-12-26 21:35:30,000][105692] Updated weights for policy 0, policy_version 855703 (0.0008) [2023-12-26 21:35:30,494][105620] Updated weights for policy 1, policy_version 855649 (0.0010) [2023-12-26 21:35:30,548][105620] Updated weights for policy 1, policy_version 855659 (0.0010) [2023-12-26 21:35:30,598][105620] Updated weights for policy 1, policy_version 855669 (0.0010) [2023-12-26 21:35:30,799][105692] Updated weights for policy 0, policy_version 855713 (0.0008) [2023-12-26 21:35:30,872][105692] Updated weights for policy 0, policy_version 855723 (0.0005) [2023-12-26 21:35:30,935][105692] Updated weights for policy 0, policy_version 855733 (0.0005) [2023-12-26 21:35:30,988][105692] Updated weights for policy 0, policy_version 855743 (0.0005) [2023-12-26 21:35:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.1, 300 sec: 19549.7). Total num frames: 438181888. Throughput: 0: 9599.1, 1: 9687.4. Samples: 438148440. Policy #0 lag: (min: 29.0, avg: 36.8, max: 61.0) [2023-12-26 21:35:31,063][104569] Avg episode reward: [(0, '9071.166'), (1, '9078.298')] [2023-12-26 21:35:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000855744_219103232.pth... [2023-12-26 21:35:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000855672_219078656.pth... [2023-12-26 21:35:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000854552_218791936.pth [2023-12-26 21:35:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000854592_218808320.pth [2023-12-26 21:35:31,258][105620] Updated weights for policy 1, policy_version 855679 (0.0010) [2023-12-26 21:35:31,317][105620] Updated weights for policy 1, policy_version 855689 (0.0010) [2023-12-26 21:35:31,377][105620] Updated weights for policy 1, policy_version 855699 (0.0010) [2023-12-26 21:35:31,688][105692] Updated weights for policy 0, policy_version 855753 (0.0009) [2023-12-26 21:35:31,752][105692] Updated weights for policy 0, policy_version 855763 (0.0010) [2023-12-26 21:35:31,818][105692] Updated weights for policy 0, policy_version 855773 (0.0010) [2023-12-26 21:35:31,955][105620] Updated weights for policy 1, policy_version 855709 (0.0007) [2023-12-26 21:35:32,006][105620] Updated weights for policy 1, policy_version 855719 (0.0007) [2023-12-26 21:35:32,052][105620] Updated weights for policy 1, policy_version 855729 (0.0007) [2023-12-26 21:35:32,651][105620] Updated weights for policy 1, policy_version 855739 (0.0009) [2023-12-26 21:35:32,681][105692] Updated weights for policy 0, policy_version 855783 (0.0008) [2023-12-26 21:35:32,704][105620] Updated weights for policy 1, policy_version 855749 (0.0005) [2023-12-26 21:35:32,733][105692] Updated weights for policy 0, policy_version 855793 (0.0007) [2023-12-26 21:35:32,755][105620] Updated weights for policy 1, policy_version 855759 (0.0010) [2023-12-26 21:35:32,793][105692] Updated weights for policy 0, policy_version 855803 (0.0006) [2023-12-26 21:35:33,439][105620] Updated weights for policy 1, policy_version 855769 (0.0010) [2023-12-26 21:35:33,486][105620] Updated weights for policy 1, policy_version 855779 (0.0008) [2023-12-26 21:35:33,523][105692] Updated weights for policy 0, policy_version 855813 (0.0006) [2023-12-26 21:35:33,536][105620] Updated weights for policy 1, policy_version 855789 (0.0010) [2023-12-26 21:35:33,577][105692] Updated weights for policy 0, policy_version 855823 (0.0006) [2023-12-26 21:35:33,587][105620] Updated weights for policy 1, policy_version 855799 (0.0010) [2023-12-26 21:35:33,622][105692] Updated weights for policy 0, policy_version 855833 (0.0010) [2023-12-26 21:35:34,219][105692] Updated weights for policy 0, policy_version 855843 (0.0011) [2023-12-26 21:35:34,280][105692] Updated weights for policy 0, policy_version 855853 (0.0011) [2023-12-26 21:35:34,286][105620] Updated weights for policy 1, policy_version 855809 (0.0006) [2023-12-26 21:35:34,346][105692] Updated weights for policy 0, policy_version 855863 (0.0011) [2023-12-26 21:35:34,347][105620] Updated weights for policy 1, policy_version 855819 (0.0008) [2023-12-26 21:35:34,404][105620] Updated weights for policy 1, policy_version 855829 (0.0006) [2023-12-26 21:35:35,094][105692] Updated weights for policy 0, policy_version 855873 (0.0011) [2023-12-26 21:35:35,111][105620] Updated weights for policy 1, policy_version 855839 (0.0005) [2023-12-26 21:35:35,152][105692] Updated weights for policy 0, policy_version 855883 (0.0010) [2023-12-26 21:35:35,161][105620] Updated weights for policy 1, policy_version 855849 (0.0008) [2023-12-26 21:35:35,209][105692] Updated weights for policy 0, policy_version 855893 (0.0009) [2023-12-26 21:35:35,217][105620] Updated weights for policy 1, policy_version 855859 (0.0010) [2023-12-26 21:35:35,269][105692] Updated weights for policy 0, policy_version 855903 (0.0009) [2023-12-26 21:35:35,866][105620] Updated weights for policy 1, policy_version 855869 (0.0010) [2023-12-26 21:35:35,911][105620] Updated weights for policy 1, policy_version 855879 (0.0010) [2023-12-26 21:35:35,955][105692] Updated weights for policy 0, policy_version 855913 (0.0010) [2023-12-26 21:35:35,972][105620] Updated weights for policy 1, policy_version 855889 (0.0010) [2023-12-26 21:35:36,019][105692] Updated weights for policy 0, policy_version 855923 (0.0010) [2023-12-26 21:35:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 438280192. Throughput: 0: 9547.1, 1: 9744.5. Samples: 438268272. Policy #0 lag: (min: 29.0, avg: 36.8, max: 61.0) [2023-12-26 21:35:36,063][104569] Avg episode reward: [(0, '8983.910'), (1, '9078.211')] [2023-12-26 21:35:36,074][105692] Updated weights for policy 0, policy_version 855933 (0.0010) [2023-12-26 21:35:36,750][105620] Updated weights for policy 1, policy_version 855899 (0.0010) [2023-12-26 21:35:36,813][105620] Updated weights for policy 1, policy_version 855909 (0.0011) [2023-12-26 21:35:36,819][105692] Updated weights for policy 0, policy_version 855943 (0.0010) [2023-12-26 21:35:36,855][105586] KL-divergence is very high: 119.5999 [2023-12-26 21:35:36,876][105620] Updated weights for policy 1, policy_version 855919 (0.0011) [2023-12-26 21:35:36,878][105692] Updated weights for policy 0, policy_version 855953 (0.0011) [2023-12-26 21:35:36,900][105586] KL-divergence is very high: 125.0549 [2023-12-26 21:35:36,933][105692] Updated weights for policy 0, policy_version 855963 (0.0009) [2023-12-26 21:35:37,633][105620] Updated weights for policy 1, policy_version 855929 (0.0011) [2023-12-26 21:35:37,656][105692] Updated weights for policy 0, policy_version 855973 (0.0008) [2023-12-26 21:35:37,697][105620] Updated weights for policy 1, policy_version 855939 (0.0011) [2023-12-26 21:35:37,720][105692] Updated weights for policy 0, policy_version 855983 (0.0006) [2023-12-26 21:35:37,759][105620] Updated weights for policy 1, policy_version 855949 (0.0011) [2023-12-26 21:35:37,777][105692] Updated weights for policy 0, policy_version 855993 (0.0006) [2023-12-26 21:35:37,819][105620] Updated weights for policy 1, policy_version 855959 (0.0011) [2023-12-26 21:35:38,499][105692] Updated weights for policy 0, policy_version 856003 (0.0006) [2023-12-26 21:35:38,544][105620] Updated weights for policy 1, policy_version 855969 (0.0007) [2023-12-26 21:35:38,556][105692] Updated weights for policy 0, policy_version 856013 (0.0005) [2023-12-26 21:35:38,593][105620] Updated weights for policy 1, policy_version 855979 (0.0010) [2023-12-26 21:35:38,610][105692] Updated weights for policy 0, policy_version 856023 (0.0006) [2023-12-26 21:35:38,650][105620] Updated weights for policy 1, policy_version 855989 (0.0011) [2023-12-26 21:35:39,234][105692] Updated weights for policy 0, policy_version 856033 (0.0006) [2023-12-26 21:35:39,299][105692] Updated weights for policy 0, policy_version 856043 (0.0009) [2023-12-26 21:35:39,363][105692] Updated weights for policy 0, policy_version 856053 (0.0007) [2023-12-26 21:35:39,384][105620] Updated weights for policy 1, policy_version 855999 (0.0010) [2023-12-26 21:35:39,432][105692] Updated weights for policy 0, policy_version 856063 (0.0007) [2023-12-26 21:35:39,446][105620] Updated weights for policy 1, policy_version 856009 (0.0010) [2023-12-26 21:35:39,508][105620] Updated weights for policy 1, policy_version 856019 (0.0008) [2023-12-26 21:35:40,234][105692] Updated weights for policy 0, policy_version 856073 (0.0008) [2023-12-26 21:35:40,268][105620] Updated weights for policy 1, policy_version 856029 (0.0009) [2023-12-26 21:35:40,291][105692] Updated weights for policy 0, policy_version 856083 (0.0007) [2023-12-26 21:35:40,332][105620] Updated weights for policy 1, policy_version 856039 (0.0007) [2023-12-26 21:35:40,356][105692] Updated weights for policy 0, policy_version 856093 (0.0009) [2023-12-26 21:35:40,392][105620] Updated weights for policy 1, policy_version 856049 (0.0007) [2023-12-26 21:35:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 438370304. Throughput: 0: 9655.9, 1: 9701.0. Samples: 438382752. Policy #0 lag: (min: 29.0, avg: 36.8, max: 61.0) [2023-12-26 21:35:41,063][104569] Avg episode reward: [(0, '8993.128'), (1, '9168.870')] [2023-12-26 21:35:41,091][105620] Updated weights for policy 1, policy_version 856059 (0.0008) [2023-12-26 21:35:41,162][105620] Updated weights for policy 1, policy_version 856069 (0.0009) [2023-12-26 21:35:41,164][105692] Updated weights for policy 0, policy_version 856103 (0.0009) [2023-12-26 21:35:41,224][105620] Updated weights for policy 1, policy_version 856079 (0.0006) [2023-12-26 21:35:41,227][105692] Updated weights for policy 0, policy_version 856113 (0.0007) [2023-12-26 21:35:41,289][105692] Updated weights for policy 0, policy_version 856123 (0.0008) [2023-12-26 21:35:42,022][105692] Updated weights for policy 0, policy_version 856133 (0.0009) [2023-12-26 21:35:42,023][105620] Updated weights for policy 1, policy_version 856089 (0.0008) [2023-12-26 21:35:42,081][105620] Updated weights for policy 1, policy_version 856099 (0.0006) [2023-12-26 21:35:42,086][105692] Updated weights for policy 0, policy_version 856143 (0.0011) [2023-12-26 21:35:42,138][105620] Updated weights for policy 1, policy_version 856109 (0.0006) [2023-12-26 21:35:42,140][105692] Updated weights for policy 0, policy_version 856153 (0.0007) [2023-12-26 21:35:42,191][105620] Updated weights for policy 1, policy_version 856119 (0.0007) [2023-12-26 21:35:42,785][105692] Updated weights for policy 0, policy_version 856163 (0.0007) [2023-12-26 21:35:42,834][105692] Updated weights for policy 0, policy_version 856173 (0.0009) [2023-12-26 21:35:42,887][105692] Updated weights for policy 0, policy_version 856183 (0.0008) [2023-12-26 21:35:43,012][105620] Updated weights for policy 1, policy_version 856129 (0.0009) [2023-12-26 21:35:43,067][105620] Updated weights for policy 1, policy_version 856139 (0.0009) [2023-12-26 21:35:43,128][105620] Updated weights for policy 1, policy_version 856149 (0.0009) [2023-12-26 21:35:43,619][105692] Updated weights for policy 0, policy_version 856193 (0.0009) [2023-12-26 21:35:43,679][105692] Updated weights for policy 0, policy_version 856203 (0.0008) [2023-12-26 21:35:43,745][105692] Updated weights for policy 0, policy_version 856213 (0.0007) [2023-12-26 21:35:43,808][105692] Updated weights for policy 0, policy_version 856223 (0.0006) [2023-12-26 21:35:43,873][105620] Updated weights for policy 1, policy_version 856159 (0.0010) [2023-12-26 21:35:43,925][105620] Updated weights for policy 1, policy_version 856169 (0.0010) [2023-12-26 21:35:43,979][105620] Updated weights for policy 1, policy_version 856179 (0.0010) [2023-12-26 21:35:44,438][105692] Updated weights for policy 0, policy_version 856233 (0.0009) [2023-12-26 21:35:44,486][105692] Updated weights for policy 0, policy_version 856243 (0.0010) [2023-12-26 21:35:44,531][105692] Updated weights for policy 0, policy_version 856253 (0.0010) [2023-12-26 21:35:44,581][105620] Updated weights for policy 1, policy_version 856189 (0.0007) [2023-12-26 21:35:44,641][105620] Updated weights for policy 1, policy_version 856199 (0.0009) [2023-12-26 21:35:44,699][105620] Updated weights for policy 1, policy_version 856209 (0.0010) [2023-12-26 21:35:45,269][105692] Updated weights for policy 0, policy_version 856263 (0.0010) [2023-12-26 21:35:45,329][105692] Updated weights for policy 0, policy_version 856273 (0.0010) [2023-12-26 21:35:45,396][105692] Updated weights for policy 0, policy_version 856283 (0.0010) [2023-12-26 21:35:45,429][105620] Updated weights for policy 1, policy_version 856219 (0.0011) [2023-12-26 21:35:45,482][105620] Updated weights for policy 1, policy_version 856229 (0.0011) [2023-12-26 21:35:45,531][105620] Updated weights for policy 1, policy_version 856239 (0.0011) [2023-12-26 21:35:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.3, 300 sec: 19577.5). Total num frames: 438468608. Throughput: 0: 9692.8, 1: 9666.2. Samples: 438438672. Policy #0 lag: (min: 29.0, avg: 36.8, max: 61.0) [2023-12-26 21:35:46,062][104569] Avg episode reward: [(0, '9081.440'), (1, '9260.289')] [2023-12-26 21:35:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000856248_219226112.pth... [2023-12-26 21:35:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000856288_219242496.pth... [2023-12-26 21:35:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000855128_218939392.pth [2023-12-26 21:35:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000855168_218955776.pth [2023-12-26 21:35:46,138][105692] Updated weights for policy 0, policy_version 856293 (0.0010) [2023-12-26 21:35:46,186][105692] Updated weights for policy 0, policy_version 856303 (0.0010) [2023-12-26 21:35:46,244][105692] Updated weights for policy 0, policy_version 856313 (0.0010) [2023-12-26 21:35:46,293][105620] Updated weights for policy 1, policy_version 856249 (0.0010) [2023-12-26 21:35:46,344][105620] Updated weights for policy 1, policy_version 856259 (0.0010) [2023-12-26 21:35:46,392][105620] Updated weights for policy 1, policy_version 856269 (0.0010) [2023-12-26 21:35:46,451][105620] Updated weights for policy 1, policy_version 856279 (0.0010) [2023-12-26 21:35:46,994][105692] Updated weights for policy 0, policy_version 856323 (0.0010) [2023-12-26 21:35:47,041][105692] Updated weights for policy 0, policy_version 856333 (0.0010) [2023-12-26 21:35:47,093][105692] Updated weights for policy 0, policy_version 856343 (0.0010) [2023-12-26 21:35:47,216][105620] Updated weights for policy 1, policy_version 856289 (0.0010) [2023-12-26 21:35:47,286][105620] Updated weights for policy 1, policy_version 856299 (0.0008) [2023-12-26 21:35:47,334][105620] Updated weights for policy 1, policy_version 856309 (0.0009) [2023-12-26 21:35:47,848][105692] Updated weights for policy 0, policy_version 856353 (0.0010) [2023-12-26 21:35:47,907][105692] Updated weights for policy 0, policy_version 856363 (0.0010) [2023-12-26 21:35:47,920][105620] Updated weights for policy 1, policy_version 856319 (0.0007) [2023-12-26 21:35:47,962][105692] Updated weights for policy 0, policy_version 856373 (0.0010) [2023-12-26 21:35:47,976][105620] Updated weights for policy 1, policy_version 856329 (0.0005) [2023-12-26 21:35:48,017][105692] Updated weights for policy 0, policy_version 856383 (0.0010) [2023-12-26 21:35:48,027][105620] Updated weights for policy 1, policy_version 856339 (0.0005) [2023-12-26 21:35:48,718][105620] Updated weights for policy 1, policy_version 856349 (0.0007) [2023-12-26 21:35:48,725][105692] Updated weights for policy 0, policy_version 856393 (0.0006) [2023-12-26 21:35:48,774][105620] Updated weights for policy 1, policy_version 856359 (0.0006) [2023-12-26 21:35:48,779][105692] Updated weights for policy 0, policy_version 856403 (0.0006) [2023-12-26 21:35:48,837][105620] Updated weights for policy 1, policy_version 856369 (0.0007) [2023-12-26 21:35:48,842][105692] Updated weights for policy 0, policy_version 856413 (0.0007) [2023-12-26 21:35:49,497][105620] Updated weights for policy 1, policy_version 856379 (0.0009) [2023-12-26 21:35:49,554][105620] Updated weights for policy 1, policy_version 856389 (0.0007) [2023-12-26 21:35:49,560][105692] Updated weights for policy 0, policy_version 856423 (0.0009) [2023-12-26 21:35:49,611][105692] Updated weights for policy 0, policy_version 856433 (0.0010) [2023-12-26 21:35:49,612][105620] Updated weights for policy 1, policy_version 856399 (0.0006) [2023-12-26 21:35:49,663][105692] Updated weights for policy 0, policy_version 856443 (0.0010) [2023-12-26 21:35:50,375][105692] Updated weights for policy 0, policy_version 856453 (0.0010) [2023-12-26 21:35:50,405][105620] Updated weights for policy 1, policy_version 856409 (0.0006) [2023-12-26 21:35:50,433][105692] Updated weights for policy 0, policy_version 856463 (0.0011) [2023-12-26 21:35:50,459][105620] Updated weights for policy 1, policy_version 856419 (0.0007) [2023-12-26 21:35:50,489][105692] Updated weights for policy 0, policy_version 856473 (0.0011) [2023-12-26 21:35:50,504][105620] Updated weights for policy 1, policy_version 856429 (0.0005) [2023-12-26 21:35:50,557][105620] Updated weights for policy 1, policy_version 856439 (0.0007) [2023-12-26 21:35:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.3, 300 sec: 19577.5). Total num frames: 438566912. Throughput: 0: 9704.8, 1: 9784.4. Samples: 438557560. Policy #0 lag: (min: 29.0, avg: 36.8, max: 61.0) [2023-12-26 21:35:51,063][104569] Avg episode reward: [(0, '9246.560'), (1, '9259.185')] [2023-12-26 21:35:51,213][105692] Updated weights for policy 0, policy_version 856483 (0.0009) [2023-12-26 21:35:51,274][105692] Updated weights for policy 0, policy_version 856493 (0.0009) [2023-12-26 21:35:51,333][105692] Updated weights for policy 0, policy_version 856503 (0.0006) [2023-12-26 21:35:51,354][105620] Updated weights for policy 1, policy_version 856449 (0.0010) [2023-12-26 21:35:51,419][105620] Updated weights for policy 1, policy_version 856459 (0.0010) [2023-12-26 21:35:51,478][105620] Updated weights for policy 1, policy_version 856469 (0.0010) [2023-12-26 21:35:52,062][105692] Updated weights for policy 0, policy_version 856513 (0.0007) [2023-12-26 21:35:52,125][105692] Updated weights for policy 0, policy_version 856523 (0.0006) [2023-12-26 21:35:52,192][105692] Updated weights for policy 0, policy_version 856533 (0.0006) [2023-12-26 21:35:52,230][105620] Updated weights for policy 1, policy_version 856479 (0.0011) [2023-12-26 21:35:52,257][105692] Updated weights for policy 0, policy_version 856543 (0.0006) [2023-12-26 21:35:52,295][105620] Updated weights for policy 1, policy_version 856489 (0.0012) [2023-12-26 21:35:52,367][105620] Updated weights for policy 1, policy_version 856499 (0.0011) [2023-12-26 21:35:52,940][105692] Updated weights for policy 0, policy_version 856553 (0.0009) [2023-12-26 21:35:52,999][105692] Updated weights for policy 0, policy_version 856563 (0.0009) [2023-12-26 21:35:53,042][105692] Updated weights for policy 0, policy_version 856573 (0.0007) [2023-12-26 21:35:53,055][105620] Updated weights for policy 1, policy_version 856509 (0.0011) [2023-12-26 21:35:53,113][105620] Updated weights for policy 1, policy_version 856519 (0.0010) [2023-12-26 21:35:53,179][105620] Updated weights for policy 1, policy_version 856529 (0.0011) [2023-12-26 21:35:53,775][105692] Updated weights for policy 0, policy_version 856583 (0.0006) [2023-12-26 21:35:53,836][105692] Updated weights for policy 0, policy_version 856593 (0.0008) [2023-12-26 21:35:53,887][105620] Updated weights for policy 1, policy_version 856539 (0.0009) [2023-12-26 21:35:53,894][105692] Updated weights for policy 0, policy_version 856603 (0.0008) [2023-12-26 21:35:53,942][105620] Updated weights for policy 1, policy_version 856549 (0.0005) [2023-12-26 21:35:54,003][105620] Updated weights for policy 1, policy_version 856559 (0.0006) [2023-12-26 21:35:54,534][105692] Updated weights for policy 0, policy_version 856613 (0.0007) [2023-12-26 21:35:54,595][105620] Updated weights for policy 1, policy_version 856569 (0.0008) [2023-12-26 21:35:54,601][105692] Updated weights for policy 0, policy_version 856623 (0.0009) [2023-12-26 21:35:54,646][105620] Updated weights for policy 1, policy_version 856579 (0.0009) [2023-12-26 21:35:54,651][105692] Updated weights for policy 0, policy_version 856633 (0.0007) [2023-12-26 21:35:54,702][105620] Updated weights for policy 1, policy_version 856589 (0.0005) [2023-12-26 21:35:54,757][105620] Updated weights for policy 1, policy_version 856599 (0.0006) [2023-12-26 21:35:55,370][105692] Updated weights for policy 0, policy_version 856643 (0.0008) [2023-12-26 21:35:55,431][105692] Updated weights for policy 0, policy_version 856653 (0.0005) [2023-12-26 21:35:55,431][105620] Updated weights for policy 1, policy_version 856609 (0.0007) [2023-12-26 21:35:55,485][105620] Updated weights for policy 1, policy_version 856619 (0.0005) [2023-12-26 21:35:55,494][105692] Updated weights for policy 0, policy_version 856663 (0.0006) [2023-12-26 21:35:55,544][105620] Updated weights for policy 1, policy_version 856629 (0.0005) [2023-12-26 21:35:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 438665216. Throughput: 0: 9654.5, 1: 9736.5. Samples: 438676468. Policy #0 lag: (min: 29.0, avg: 36.8, max: 61.0) [2023-12-26 21:35:56,063][104569] Avg episode reward: [(0, '9248.093'), (1, '9167.990')] [2023-12-26 21:35:56,136][105620] Updated weights for policy 1, policy_version 856639 (0.0008) [2023-12-26 21:35:56,186][105620] Updated weights for policy 1, policy_version 856649 (0.0008) [2023-12-26 21:35:56,204][105692] Updated weights for policy 0, policy_version 856673 (0.0006) [2023-12-26 21:35:56,243][105620] Updated weights for policy 1, policy_version 856659 (0.0007) [2023-12-26 21:35:56,264][105692] Updated weights for policy 0, policy_version 856683 (0.0008) [2023-12-26 21:35:56,323][105692] Updated weights for policy 0, policy_version 856693 (0.0008) [2023-12-26 21:35:56,378][105692] Updated weights for policy 0, policy_version 856703 (0.0009) [2023-12-26 21:35:56,978][105692] Updated weights for policy 0, policy_version 856713 (0.0006) [2023-12-26 21:35:57,024][105692] Updated weights for policy 0, policy_version 856723 (0.0005) [2023-12-26 21:35:57,054][105620] Updated weights for policy 1, policy_version 856669 (0.0008) [2023-12-26 21:35:57,085][105692] Updated weights for policy 0, policy_version 856733 (0.0005) [2023-12-26 21:35:57,105][105620] Updated weights for policy 1, policy_version 856679 (0.0009) [2023-12-26 21:35:57,176][105620] Updated weights for policy 1, policy_version 856689 (0.0009) [2023-12-26 21:35:57,741][105692] Updated weights for policy 0, policy_version 856743 (0.0005) [2023-12-26 21:35:57,799][105692] Updated weights for policy 0, policy_version 856753 (0.0005) [2023-12-26 21:35:57,805][105620] Updated weights for policy 1, policy_version 856699 (0.0009) [2023-12-26 21:35:57,847][105692] Updated weights for policy 0, policy_version 856763 (0.0005) [2023-12-26 21:35:57,866][105620] Updated weights for policy 1, policy_version 856709 (0.0010) [2023-12-26 21:35:57,923][105620] Updated weights for policy 1, policy_version 856719 (0.0010) [2023-12-26 21:35:58,602][105692] Updated weights for policy 0, policy_version 856773 (0.0007) [2023-12-26 21:35:58,654][105692] Updated weights for policy 0, policy_version 856783 (0.0008) [2023-12-26 21:35:58,669][105620] Updated weights for policy 1, policy_version 856729 (0.0010) [2023-12-26 21:35:58,716][105692] Updated weights for policy 0, policy_version 856793 (0.0008) [2023-12-26 21:35:58,731][105620] Updated weights for policy 1, policy_version 856739 (0.0008) [2023-12-26 21:35:58,795][105620] Updated weights for policy 1, policy_version 856749 (0.0007) [2023-12-26 21:35:58,863][105620] Updated weights for policy 1, policy_version 856759 (0.0007) [2023-12-26 21:35:59,451][105692] Updated weights for policy 0, policy_version 856803 (0.0007) [2023-12-26 21:35:59,511][105692] Updated weights for policy 0, policy_version 856813 (0.0007) [2023-12-26 21:35:59,547][105620] Updated weights for policy 1, policy_version 856769 (0.0010) [2023-12-26 21:35:59,565][105692] Updated weights for policy 0, policy_version 856823 (0.0008) [2023-12-26 21:35:59,610][105620] Updated weights for policy 1, policy_version 856779 (0.0010) [2023-12-26 21:35:59,665][105620] Updated weights for policy 1, policy_version 856789 (0.0010) [2023-12-26 21:36:00,249][105692] Updated weights for policy 0, policy_version 856833 (0.0006) [2023-12-26 21:36:00,302][105692] Updated weights for policy 0, policy_version 856843 (0.0007) [2023-12-26 21:36:00,339][105620] Updated weights for policy 1, policy_version 856799 (0.0007) [2023-12-26 21:36:00,361][105692] Updated weights for policy 0, policy_version 856853 (0.0008) [2023-12-26 21:36:00,407][105620] Updated weights for policy 1, policy_version 856809 (0.0009) [2023-12-26 21:36:00,410][105692] Updated weights for policy 0, policy_version 856863 (0.0007) [2023-12-26 21:36:00,459][105620] Updated weights for policy 1, policy_version 856819 (0.0010) [2023-12-26 21:36:00,969][105692] Updated weights for policy 0, policy_version 856873 (0.0006) [2023-12-26 21:36:01,018][105620] Updated weights for policy 1, policy_version 856829 (0.0010) [2023-12-26 21:36:01,038][105692] Updated weights for policy 0, policy_version 856883 (0.0008) [2023-12-26 21:36:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 438763520. Throughput: 0: 9696.4, 1: 9767.8. Samples: 438735684. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-12-26 21:36:01,062][104569] Avg episode reward: [(0, '9170.854'), (1, '8866.577')] [2023-12-26 21:36:01,082][105620] Updated weights for policy 1, policy_version 856839 (0.0010) [2023-12-26 21:36:01,097][105692] Updated weights for policy 0, policy_version 856893 (0.0008) [2023-12-26 21:36:01,116][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000856896_219398144.pth... [2023-12-26 21:36:01,120][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000855744_219103232.pth [2023-12-26 21:36:01,139][105620] Updated weights for policy 1, policy_version 856849 (0.0010) [2023-12-26 21:36:01,181][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000856856_219381760.pth... [2023-12-26 21:36:01,184][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000855672_219078656.pth [2023-12-26 21:36:01,873][105620] Updated weights for policy 1, policy_version 856859 (0.0010) [2023-12-26 21:36:01,884][105692] Updated weights for policy 0, policy_version 856903 (0.0006) [2023-12-26 21:36:01,928][105692] Updated weights for policy 0, policy_version 856913 (0.0006) [2023-12-26 21:36:01,929][105620] Updated weights for policy 1, policy_version 856869 (0.0010) [2023-12-26 21:36:01,953][105586] KL-divergence is very high: 472.9539 [2023-12-26 21:36:01,976][105692] Updated weights for policy 0, policy_version 856923 (0.0008) [2023-12-26 21:36:01,992][105620] Updated weights for policy 1, policy_version 856879 (0.0010) [2023-12-26 21:36:02,003][105586] KL-divergence is very high: 802.1124 [2023-12-26 21:36:02,721][105692] Updated weights for policy 0, policy_version 856933 (0.0008) [2023-12-26 21:36:02,736][105620] Updated weights for policy 1, policy_version 856889 (0.0010) [2023-12-26 21:36:02,777][105692] Updated weights for policy 0, policy_version 856943 (0.0008) [2023-12-26 21:36:02,791][105620] Updated weights for policy 1, policy_version 856899 (0.0010) [2023-12-26 21:36:02,832][105692] Updated weights for policy 0, policy_version 856953 (0.0005) [2023-12-26 21:36:02,853][105620] Updated weights for policy 1, policy_version 856909 (0.0010) [2023-12-26 21:36:02,905][105620] Updated weights for policy 1, policy_version 856919 (0.0010) [2023-12-26 21:36:03,626][105620] Updated weights for policy 1, policy_version 856929 (0.0006) [2023-12-26 21:36:03,628][105692] Updated weights for policy 0, policy_version 856963 (0.0006) [2023-12-26 21:36:03,687][105692] Updated weights for policy 0, policy_version 856973 (0.0009) [2023-12-26 21:36:03,690][105620] Updated weights for policy 1, policy_version 856939 (0.0010) [2023-12-26 21:36:03,743][105692] Updated weights for policy 0, policy_version 856983 (0.0007) [2023-12-26 21:36:03,752][105620] Updated weights for policy 1, policy_version 856949 (0.0007) [2023-12-26 21:36:04,428][105620] Updated weights for policy 1, policy_version 856959 (0.0010) [2023-12-26 21:36:04,495][105620] Updated weights for policy 1, policy_version 856969 (0.0008) [2023-12-26 21:36:04,537][105692] Updated weights for policy 0, policy_version 856993 (0.0007) [2023-12-26 21:36:04,555][105620] Updated weights for policy 1, policy_version 856979 (0.0010) [2023-12-26 21:36:04,591][105692] Updated weights for policy 0, policy_version 857003 (0.0011) [2023-12-26 21:36:04,651][105692] Updated weights for policy 0, policy_version 857013 (0.0011) [2023-12-26 21:36:04,713][105692] Updated weights for policy 0, policy_version 857023 (0.0009) [2023-12-26 21:36:05,210][105620] Updated weights for policy 1, policy_version 856989 (0.0008) [2023-12-26 21:36:05,263][105620] Updated weights for policy 1, policy_version 856999 (0.0005) [2023-12-26 21:36:05,314][105620] Updated weights for policy 1, policy_version 857009 (0.0005) [2023-12-26 21:36:05,460][105692] Updated weights for policy 0, policy_version 857033 (0.0011) [2023-12-26 21:36:05,511][105692] Updated weights for policy 0, policy_version 857043 (0.0010) [2023-12-26 21:36:05,559][105692] Updated weights for policy 0, policy_version 857053 (0.0010) [2023-12-26 21:36:05,922][105620] Updated weights for policy 1, policy_version 857019 (0.0005) [2023-12-26 21:36:05,976][105620] Updated weights for policy 1, policy_version 857029 (0.0005) [2023-12-26 21:36:06,031][105620] Updated weights for policy 1, policy_version 857039 (0.0005) [2023-12-26 21:36:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 438861824. Throughput: 0: 9627.8, 1: 9842.2. Samples: 438854028. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-12-26 21:36:06,063][104569] Avg episode reward: [(0, '9173.111'), (1, '8796.500')] [2023-12-26 21:36:06,190][105692] Updated weights for policy 0, policy_version 857063 (0.0010) [2023-12-26 21:36:06,247][105692] Updated weights for policy 0, policy_version 857073 (0.0011) [2023-12-26 21:36:06,301][105692] Updated weights for policy 0, policy_version 857083 (0.0011) [2023-12-26 21:36:06,685][105620] Updated weights for policy 1, policy_version 857049 (0.0005) [2023-12-26 21:36:06,752][105620] Updated weights for policy 1, policy_version 857059 (0.0008) [2023-12-26 21:36:06,811][105620] Updated weights for policy 1, policy_version 857069 (0.0009) [2023-12-26 21:36:06,863][105620] Updated weights for policy 1, policy_version 857079 (0.0010) [2023-12-26 21:36:07,074][105692] Updated weights for policy 0, policy_version 857093 (0.0011) [2023-12-26 21:36:07,129][105692] Updated weights for policy 0, policy_version 857103 (0.0010) [2023-12-26 21:36:07,201][105692] Updated weights for policy 0, policy_version 857113 (0.0008) [2023-12-26 21:36:07,634][105620] Updated weights for policy 1, policy_version 857089 (0.0008) [2023-12-26 21:36:07,696][105620] Updated weights for policy 1, policy_version 857099 (0.0011) [2023-12-26 21:36:07,750][105620] Updated weights for policy 1, policy_version 857109 (0.0006) [2023-12-26 21:36:07,893][105692] Updated weights for policy 0, policy_version 857123 (0.0008) [2023-12-26 21:36:07,953][105692] Updated weights for policy 0, policy_version 857133 (0.0009) [2023-12-26 21:36:08,007][105692] Updated weights for policy 0, policy_version 857143 (0.0009) [2023-12-26 21:36:08,338][105620] Updated weights for policy 1, policy_version 857119 (0.0006) [2023-12-26 21:36:08,399][105620] Updated weights for policy 1, policy_version 857129 (0.0008) [2023-12-26 21:36:08,452][105620] Updated weights for policy 1, policy_version 857139 (0.0011) [2023-12-26 21:36:08,723][105692] Updated weights for policy 0, policy_version 857153 (0.0006) [2023-12-26 21:36:08,775][105692] Updated weights for policy 0, policy_version 857163 (0.0010) [2023-12-26 21:36:08,830][105692] Updated weights for policy 0, policy_version 857173 (0.0007) [2023-12-26 21:36:08,891][105692] Updated weights for policy 0, policy_version 857183 (0.0007) [2023-12-26 21:36:09,042][105620] Updated weights for policy 1, policy_version 857149 (0.0006) [2023-12-26 21:36:09,097][105620] Updated weights for policy 1, policy_version 857159 (0.0005) [2023-12-26 21:36:09,166][105620] Updated weights for policy 1, policy_version 857169 (0.0006) [2023-12-26 21:36:09,644][105692] Updated weights for policy 0, policy_version 857193 (0.0009) [2023-12-26 21:36:09,707][105692] Updated weights for policy 0, policy_version 857203 (0.0009) [2023-12-26 21:36:09,767][105692] Updated weights for policy 0, policy_version 857213 (0.0008) [2023-12-26 21:36:09,842][105620] Updated weights for policy 1, policy_version 857179 (0.0008) [2023-12-26 21:36:09,903][105620] Updated weights for policy 1, policy_version 857189 (0.0009) [2023-12-26 21:36:09,968][105620] Updated weights for policy 1, policy_version 857199 (0.0007) [2023-12-26 21:36:10,523][105692] Updated weights for policy 0, policy_version 857223 (0.0010) [2023-12-26 21:36:10,579][105692] Updated weights for policy 0, policy_version 857233 (0.0010) [2023-12-26 21:36:10,632][105692] Updated weights for policy 0, policy_version 857243 (0.0010) [2023-12-26 21:36:10,760][105620] Updated weights for policy 1, policy_version 857209 (0.0009) [2023-12-26 21:36:10,812][105620] Updated weights for policy 1, policy_version 857219 (0.0008) [2023-12-26 21:36:10,857][105620] Updated weights for policy 1, policy_version 857229 (0.0008) [2023-12-26 21:36:10,909][105620] Updated weights for policy 1, policy_version 857239 (0.0008) [2023-12-26 21:36:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 438968320. Throughput: 0: 9672.5, 1: 9921.1. Samples: 438973768. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-12-26 21:36:11,062][104569] Avg episode reward: [(0, '8982.070'), (1, '9088.392')] [2023-12-26 21:36:11,418][105692] Updated weights for policy 0, policy_version 857253 (0.0010) [2023-12-26 21:36:11,485][105692] Updated weights for policy 0, policy_version 857263 (0.0009) [2023-12-26 21:36:11,548][105692] Updated weights for policy 0, policy_version 857273 (0.0009) [2023-12-26 21:36:11,739][105620] Updated weights for policy 1, policy_version 857249 (0.0009) [2023-12-26 21:36:11,795][105620] Updated weights for policy 1, policy_version 857259 (0.0008) [2023-12-26 21:36:11,845][105620] Updated weights for policy 1, policy_version 857269 (0.0006) [2023-12-26 21:36:12,287][105692] Updated weights for policy 0, policy_version 857283 (0.0010) [2023-12-26 21:36:12,351][105692] Updated weights for policy 0, policy_version 857293 (0.0009) [2023-12-26 21:36:12,427][105692] Updated weights for policy 0, policy_version 857303 (0.0007) [2023-12-26 21:36:12,531][105620] Updated weights for policy 1, policy_version 857279 (0.0008) [2023-12-26 21:36:12,596][105620] Updated weights for policy 1, policy_version 857289 (0.0009) [2023-12-26 21:36:12,659][105620] Updated weights for policy 1, policy_version 857299 (0.0008) [2023-12-26 21:36:13,084][105692] Updated weights for policy 0, policy_version 857313 (0.0011) [2023-12-26 21:36:13,135][105692] Updated weights for policy 0, policy_version 857323 (0.0010) [2023-12-26 21:36:13,187][105692] Updated weights for policy 0, policy_version 857333 (0.0010) [2023-12-26 21:36:13,235][105692] Updated weights for policy 0, policy_version 857343 (0.0010) [2023-12-26 21:36:13,450][105620] Updated weights for policy 1, policy_version 857309 (0.0009) [2023-12-26 21:36:13,504][105620] Updated weights for policy 1, policy_version 857319 (0.0008) [2023-12-26 21:36:13,562][105620] Updated weights for policy 1, policy_version 857329 (0.0008) [2023-12-26 21:36:13,949][105692] Updated weights for policy 0, policy_version 857353 (0.0010) [2023-12-26 21:36:14,003][105692] Updated weights for policy 0, policy_version 857363 (0.0010) [2023-12-26 21:36:14,048][105692] Updated weights for policy 0, policy_version 857373 (0.0010) [2023-12-26 21:36:14,342][105620] Updated weights for policy 1, policy_version 857339 (0.0007) [2023-12-26 21:36:14,398][105620] Updated weights for policy 1, policy_version 857349 (0.0006) [2023-12-26 21:36:14,449][105620] Updated weights for policy 1, policy_version 857359 (0.0009) [2023-12-26 21:36:14,745][105692] Updated weights for policy 0, policy_version 857383 (0.0008) [2023-12-26 21:36:14,811][105692] Updated weights for policy 0, policy_version 857393 (0.0008) [2023-12-26 21:36:14,876][105692] Updated weights for policy 0, policy_version 857403 (0.0008) [2023-12-26 21:36:15,257][105620] Updated weights for policy 1, policy_version 857369 (0.0009) [2023-12-26 21:36:15,313][105620] Updated weights for policy 1, policy_version 857379 (0.0009) [2023-12-26 21:36:15,376][105620] Updated weights for policy 1, policy_version 857389 (0.0010) [2023-12-26 21:36:15,435][105620] Updated weights for policy 1, policy_version 857399 (0.0009) [2023-12-26 21:36:15,529][105692] Updated weights for policy 0, policy_version 857413 (0.0009) [2023-12-26 21:36:15,581][105692] Updated weights for policy 0, policy_version 857423 (0.0009) [2023-12-26 21:36:15,640][105692] Updated weights for policy 0, policy_version 857433 (0.0010) [2023-12-26 21:36:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 439058432. Throughput: 0: 9684.0, 1: 9910.6. Samples: 439030200. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-12-26 21:36:16,063][104569] Avg episode reward: [(0, '8743.157'), (1, '9340.095')] [2023-12-26 21:36:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000857440_219537408.pth... [2023-12-26 21:36:16,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000856288_219242496.pth [2023-12-26 21:36:16,105][105620] Updated weights for policy 1, policy_version 857409 (0.0011) [2023-12-26 21:36:16,163][105620] Updated weights for policy 1, policy_version 857419 (0.0010) [2023-12-26 21:36:16,232][105620] Updated weights for policy 1, policy_version 857429 (0.0011) [2023-12-26 21:36:16,248][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000857432_219529216.pth... [2023-12-26 21:36:16,253][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000856248_219226112.pth [2023-12-26 21:36:16,317][105692] Updated weights for policy 0, policy_version 857444 (0.0009) [2023-12-26 21:36:16,377][105692] Updated weights for policy 0, policy_version 857454 (0.0008) [2023-12-26 21:36:16,438][105692] Updated weights for policy 0, policy_version 857464 (0.0008) [2023-12-26 21:36:16,971][105620] Updated weights for policy 1, policy_version 857439 (0.0010) [2023-12-26 21:36:17,040][105620] Updated weights for policy 1, policy_version 857449 (0.0011) [2023-12-26 21:36:17,095][105692] Updated weights for policy 0, policy_version 857474 (0.0008) [2023-12-26 21:36:17,099][105620] Updated weights for policy 1, policy_version 857459 (0.0011) [2023-12-26 21:36:17,153][105692] Updated weights for policy 0, policy_version 857484 (0.0010) [2023-12-26 21:36:17,210][105692] Updated weights for policy 0, policy_version 857494 (0.0010) [2023-12-26 21:36:17,254][105692] Updated weights for policy 0, policy_version 857504 (0.0010) [2023-12-26 21:36:17,736][105620] Updated weights for policy 1, policy_version 857469 (0.0010) [2023-12-26 21:36:17,781][105620] Updated weights for policy 1, policy_version 857479 (0.0010) [2023-12-26 21:36:17,839][105620] Updated weights for policy 1, policy_version 857489 (0.0010) [2023-12-26 21:36:17,968][105692] Updated weights for policy 0, policy_version 857514 (0.0005) [2023-12-26 21:36:18,025][105692] Updated weights for policy 0, policy_version 857524 (0.0005) [2023-12-26 21:36:18,076][105692] Updated weights for policy 0, policy_version 857534 (0.0005) [2023-12-26 21:36:18,444][105620] Updated weights for policy 1, policy_version 857499 (0.0010) [2023-12-26 21:36:18,497][105620] Updated weights for policy 1, policy_version 857509 (0.0011) [2023-12-26 21:36:18,549][105620] Updated weights for policy 1, policy_version 857519 (0.0010) [2023-12-26 21:36:18,764][105692] Updated weights for policy 0, policy_version 857544 (0.0009) [2023-12-26 21:36:18,830][105692] Updated weights for policy 0, policy_version 857554 (0.0010) [2023-12-26 21:36:18,885][105692] Updated weights for policy 0, policy_version 857564 (0.0010) [2023-12-26 21:36:19,271][105620] Updated weights for policy 1, policy_version 857529 (0.0011) [2023-12-26 21:36:19,334][105620] Updated weights for policy 1, policy_version 857539 (0.0008) [2023-12-26 21:36:19,402][105620] Updated weights for policy 1, policy_version 857549 (0.0010) [2023-12-26 21:36:19,460][105620] Updated weights for policy 1, policy_version 857559 (0.0010) [2023-12-26 21:36:19,666][105692] Updated weights for policy 0, policy_version 857574 (0.0010) [2023-12-26 21:36:19,726][105692] Updated weights for policy 0, policy_version 857584 (0.0008) [2023-12-26 21:36:19,783][105692] Updated weights for policy 0, policy_version 857594 (0.0008) [2023-12-26 21:36:20,194][105620] Updated weights for policy 1, policy_version 857569 (0.0011) [2023-12-26 21:36:20,257][105620] Updated weights for policy 1, policy_version 857579 (0.0010) [2023-12-26 21:36:20,326][105620] Updated weights for policy 1, policy_version 857589 (0.0010) [2023-12-26 21:36:20,508][105692] Updated weights for policy 0, policy_version 857604 (0.0009) [2023-12-26 21:36:20,554][105692] Updated weights for policy 0, policy_version 857614 (0.0008) [2023-12-26 21:36:20,625][105692] Updated weights for policy 0, policy_version 857624 (0.0008) [2023-12-26 21:36:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 439156736. Throughput: 0: 9758.4, 1: 9823.9. Samples: 439149476. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-12-26 21:36:21,062][105620] Updated weights for policy 1, policy_version 857599 (0.0009) [2023-12-26 21:36:21,062][104569] Avg episode reward: [(0, '8827.965'), (1, '9348.803')] [2023-12-26 21:36:21,124][105620] Updated weights for policy 1, policy_version 857609 (0.0010) [2023-12-26 21:36:21,195][105620] Updated weights for policy 1, policy_version 857619 (0.0007) [2023-12-26 21:36:21,384][105692] Updated weights for policy 0, policy_version 857634 (0.0009) [2023-12-26 21:36:21,449][105692] Updated weights for policy 0, policy_version 857644 (0.0009) [2023-12-26 21:36:21,508][105692] Updated weights for policy 0, policy_version 857654 (0.0010) [2023-12-26 21:36:21,572][105692] Updated weights for policy 0, policy_version 857664 (0.0010) [2023-12-26 21:36:21,906][105620] Updated weights for policy 1, policy_version 857629 (0.0007) [2023-12-26 21:36:21,958][105620] Updated weights for policy 1, policy_version 857639 (0.0008) [2023-12-26 21:36:22,025][105620] Updated weights for policy 1, policy_version 857649 (0.0010) [2023-12-26 21:36:22,330][105692] Updated weights for policy 0, policy_version 857674 (0.0009) [2023-12-26 21:36:22,391][105692] Updated weights for policy 0, policy_version 857684 (0.0007) [2023-12-26 21:36:22,449][105692] Updated weights for policy 0, policy_version 857694 (0.0006) [2023-12-26 21:36:22,765][105620] Updated weights for policy 1, policy_version 857659 (0.0010) [2023-12-26 21:36:22,833][105620] Updated weights for policy 1, policy_version 857669 (0.0011) [2023-12-26 21:36:22,892][105620] Updated weights for policy 1, policy_version 857679 (0.0011) [2023-12-26 21:36:23,137][105692] Updated weights for policy 0, policy_version 857704 (0.0006) [2023-12-26 21:36:23,188][105692] Updated weights for policy 0, policy_version 857714 (0.0009) [2023-12-26 21:36:23,238][105692] Updated weights for policy 0, policy_version 857724 (0.0009) [2023-12-26 21:36:23,630][105620] Updated weights for policy 1, policy_version 857689 (0.0009) [2023-12-26 21:36:23,681][105620] Updated weights for policy 1, policy_version 857699 (0.0007) [2023-12-26 21:36:23,727][105620] Updated weights for policy 1, policy_version 857709 (0.0008) [2023-12-26 21:36:23,774][105620] Updated weights for policy 1, policy_version 857719 (0.0009) [2023-12-26 21:36:23,916][105692] Updated weights for policy 0, policy_version 857734 (0.0009) [2023-12-26 21:36:23,966][105692] Updated weights for policy 0, policy_version 857744 (0.0008) [2023-12-26 21:36:24,022][105692] Updated weights for policy 0, policy_version 857754 (0.0010) [2023-12-26 21:36:24,526][105620] Updated weights for policy 1, policy_version 857729 (0.0008) [2023-12-26 21:36:24,587][105620] Updated weights for policy 1, policy_version 857739 (0.0009) [2023-12-26 21:36:24,648][105620] Updated weights for policy 1, policy_version 857749 (0.0007) [2023-12-26 21:36:24,779][105692] Updated weights for policy 0, policy_version 857765 (0.0010) [2023-12-26 21:36:24,835][105692] Updated weights for policy 0, policy_version 857775 (0.0009) [2023-12-26 21:36:24,886][105692] Updated weights for policy 0, policy_version 857785 (0.0009) [2023-12-26 21:36:25,323][105620] Updated weights for policy 1, policy_version 857759 (0.0006) [2023-12-26 21:36:25,370][105620] Updated weights for policy 1, policy_version 857769 (0.0007) [2023-12-26 21:36:25,427][105620] Updated weights for policy 1, policy_version 857779 (0.0008) [2023-12-26 21:36:25,676][105692] Updated weights for policy 0, policy_version 857795 (0.0009) [2023-12-26 21:36:25,723][105692] Updated weights for policy 0, policy_version 857805 (0.0008) [2023-12-26 21:36:25,777][105692] Updated weights for policy 0, policy_version 857815 (0.0009) [2023-12-26 21:36:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 439255040. Throughput: 0: 9751.9, 1: 9834.1. Samples: 439264120. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-12-26 21:36:26,062][104569] Avg episode reward: [(0, '8975.703'), (1, '9256.961')] [2023-12-26 21:36:26,105][105620] Updated weights for policy 1, policy_version 857789 (0.0009) [2023-12-26 21:36:26,159][105620] Updated weights for policy 1, policy_version 857799 (0.0010) [2023-12-26 21:36:26,213][105620] Updated weights for policy 1, policy_version 857809 (0.0010) [2023-12-26 21:36:26,496][105692] Updated weights for policy 0, policy_version 857825 (0.0008) [2023-12-26 21:36:26,548][105692] Updated weights for policy 0, policy_version 857835 (0.0009) [2023-12-26 21:36:26,602][105692] Updated weights for policy 0, policy_version 857847 (0.0010) [2023-12-26 21:36:26,889][105620] Updated weights for policy 1, policy_version 857819 (0.0010) [2023-12-26 21:36:26,942][105620] Updated weights for policy 1, policy_version 857829 (0.0010) [2023-12-26 21:36:26,995][105620] Updated weights for policy 1, policy_version 857839 (0.0009) [2023-12-26 21:36:27,182][105692] Updated weights for policy 0, policy_version 857857 (0.0005) [2023-12-26 21:36:27,244][105692] Updated weights for policy 0, policy_version 857867 (0.0005) [2023-12-26 21:36:27,306][105692] Updated weights for policy 0, policy_version 857877 (0.0006) [2023-12-26 21:36:27,358][105692] Updated weights for policy 0, policy_version 857887 (0.0009) [2023-12-26 21:36:27,823][105620] Updated weights for policy 1, policy_version 857849 (0.0009) [2023-12-26 21:36:27,884][105620] Updated weights for policy 1, policy_version 857859 (0.0009) [2023-12-26 21:36:27,947][105620] Updated weights for policy 1, policy_version 857869 (0.0008) [2023-12-26 21:36:27,996][105620] Updated weights for policy 1, policy_version 857879 (0.0009) [2023-12-26 21:36:28,048][105692] Updated weights for policy 0, policy_version 857897 (0.0006) [2023-12-26 21:36:28,095][105692] Updated weights for policy 0, policy_version 857907 (0.0008) [2023-12-26 21:36:28,149][105692] Updated weights for policy 0, policy_version 857917 (0.0006) [2023-12-26 21:36:28,742][105692] Updated weights for policy 0, policy_version 857927 (0.0006) [2023-12-26 21:36:28,805][105692] Updated weights for policy 0, policy_version 857937 (0.0009) [2023-12-26 21:36:28,836][105620] Updated weights for policy 1, policy_version 857889 (0.0008) [2023-12-26 21:36:28,857][105692] Updated weights for policy 0, policy_version 857947 (0.0008) [2023-12-26 21:36:28,889][105620] Updated weights for policy 1, policy_version 857899 (0.0006) [2023-12-26 21:36:28,941][105620] Updated weights for policy 1, policy_version 857909 (0.0009) [2023-12-26 21:36:29,598][105692] Updated weights for policy 0, policy_version 857957 (0.0009) [2023-12-26 21:36:29,658][105692] Updated weights for policy 0, policy_version 857967 (0.0008) [2023-12-26 21:36:29,667][105620] Updated weights for policy 1, policy_version 857919 (0.0006) [2023-12-26 21:36:29,706][105692] Updated weights for policy 0, policy_version 857977 (0.0009) [2023-12-26 21:36:29,722][105620] Updated weights for policy 1, policy_version 857929 (0.0006) [2023-12-26 21:36:29,781][105620] Updated weights for policy 1, policy_version 857939 (0.0006) [2023-12-26 21:36:30,392][105620] Updated weights for policy 1, policy_version 857949 (0.0009) [2023-12-26 21:36:30,451][105620] Updated weights for policy 1, policy_version 857959 (0.0009) [2023-12-26 21:36:30,509][105620] Updated weights for policy 1, policy_version 857969 (0.0009) [2023-12-26 21:36:30,524][105692] Updated weights for policy 0, policy_version 857987 (0.0008) [2023-12-26 21:36:30,585][105692] Updated weights for policy 0, policy_version 857997 (0.0010) [2023-12-26 21:36:30,649][105692] Updated weights for policy 0, policy_version 858007 (0.0007) [2023-12-26 21:36:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 439353344. Throughput: 0: 9810.3, 1: 9854.9. Samples: 439323604. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-12-26 21:36:31,062][104569] Avg episode reward: [(0, '9160.077'), (1, '8558.995')] [2023-12-26 21:36:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000858016_219684864.pth... [2023-12-26 21:36:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000857976_219668480.pth... [2023-12-26 21:36:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000856896_219398144.pth [2023-12-26 21:36:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000856856_219381760.pth [2023-12-26 21:36:31,259][105620] Updated weights for policy 1, policy_version 857979 (0.0006) [2023-12-26 21:36:31,283][105692] Updated weights for policy 0, policy_version 858017 (0.0006) [2023-12-26 21:36:31,321][105620] Updated weights for policy 1, policy_version 857989 (0.0006) [2023-12-26 21:36:31,346][105692] Updated weights for policy 0, policy_version 858027 (0.0008) [2023-12-26 21:36:31,389][105620] Updated weights for policy 1, policy_version 857999 (0.0008) [2023-12-26 21:36:31,413][105692] Updated weights for policy 0, policy_version 858037 (0.0008) [2023-12-26 21:36:31,469][105692] Updated weights for policy 0, policy_version 858047 (0.0008) [2023-12-26 21:36:32,126][105620] Updated weights for policy 1, policy_version 858009 (0.0008) [2023-12-26 21:36:32,182][105620] Updated weights for policy 1, policy_version 858019 (0.0007) [2023-12-26 21:36:32,184][105692] Updated weights for policy 0, policy_version 858057 (0.0009) [2023-12-26 21:36:32,238][105620] Updated weights for policy 1, policy_version 858029 (0.0006) [2023-12-26 21:36:32,244][105692] Updated weights for policy 0, policy_version 858067 (0.0007) [2023-12-26 21:36:32,303][105692] Updated weights for policy 0, policy_version 858077 (0.0008) [2023-12-26 21:36:32,304][105620] Updated weights for policy 1, policy_version 858039 (0.0009) [2023-12-26 21:36:33,017][105620] Updated weights for policy 1, policy_version 858049 (0.0008) [2023-12-26 21:36:33,063][105692] Updated weights for policy 0, policy_version 858087 (0.0008) [2023-12-26 21:36:33,076][105620] Updated weights for policy 1, policy_version 858059 (0.0008) [2023-12-26 21:36:33,111][105692] Updated weights for policy 0, policy_version 858097 (0.0006) [2023-12-26 21:36:33,134][105620] Updated weights for policy 1, policy_version 858069 (0.0007) [2023-12-26 21:36:33,168][105692] Updated weights for policy 0, policy_version 858107 (0.0009) [2023-12-26 21:36:33,813][105620] Updated weights for policy 1, policy_version 858079 (0.0005) [2023-12-26 21:36:33,813][105692] Updated weights for policy 0, policy_version 858117 (0.0005) [2023-12-26 21:36:33,873][105692] Updated weights for policy 0, policy_version 858127 (0.0005) [2023-12-26 21:36:33,876][105620] Updated weights for policy 1, policy_version 858089 (0.0006) [2023-12-26 21:36:33,930][105692] Updated weights for policy 0, policy_version 858137 (0.0007) [2023-12-26 21:36:33,939][105620] Updated weights for policy 1, policy_version 858099 (0.0006) [2023-12-26 21:36:34,531][105620] Updated weights for policy 1, policy_version 858109 (0.0005) [2023-12-26 21:36:34,595][105620] Updated weights for policy 1, policy_version 858119 (0.0006) [2023-12-26 21:36:34,596][105692] Updated weights for policy 0, policy_version 858147 (0.0011) [2023-12-26 21:36:34,653][105692] Updated weights for policy 0, policy_version 858157 (0.0011) [2023-12-26 21:36:34,654][105620] Updated weights for policy 1, policy_version 858129 (0.0006) [2023-12-26 21:36:34,716][105692] Updated weights for policy 0, policy_version 858167 (0.0010) [2023-12-26 21:36:35,334][105620] Updated weights for policy 1, policy_version 858139 (0.0006) [2023-12-26 21:36:35,388][105620] Updated weights for policy 1, policy_version 858149 (0.0005) [2023-12-26 21:36:35,399][105692] Updated weights for policy 0, policy_version 858177 (0.0007) [2023-12-26 21:36:35,454][105620] Updated weights for policy 1, policy_version 858159 (0.0006) [2023-12-26 21:36:35,469][105692] Updated weights for policy 0, policy_version 858187 (0.0006) [2023-12-26 21:36:35,530][105692] Updated weights for policy 0, policy_version 858197 (0.0007) [2023-12-26 21:36:35,585][105692] Updated weights for policy 0, policy_version 858207 (0.0009) [2023-12-26 21:36:36,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19524.2, 300 sec: 19605.2). Total num frames: 439451648. Throughput: 0: 9824.6, 1: 9844.1. Samples: 439442660. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-12-26 21:36:36,063][104569] Avg episode reward: [(0, '8986.889'), (1, '8733.361')] [2023-12-26 21:36:36,175][105692] Updated weights for policy 0, policy_version 858217 (0.0007) [2023-12-26 21:36:36,176][105620] Updated weights for policy 1, policy_version 858169 (0.0008) [2023-12-26 21:36:36,242][105692] Updated weights for policy 0, policy_version 858227 (0.0007) [2023-12-26 21:36:36,244][105620] Updated weights for policy 1, policy_version 858179 (0.0008) [2023-12-26 21:36:36,305][105692] Updated weights for policy 0, policy_version 858237 (0.0007) [2023-12-26 21:36:36,312][105620] Updated weights for policy 1, policy_version 858189 (0.0006) [2023-12-26 21:36:36,376][105620] Updated weights for policy 1, policy_version 858199 (0.0009) [2023-12-26 21:36:36,978][105692] Updated weights for policy 0, policy_version 858247 (0.0009) [2023-12-26 21:36:37,029][105692] Updated weights for policy 0, policy_version 858257 (0.0009) [2023-12-26 21:36:37,085][105692] Updated weights for policy 0, policy_version 858267 (0.0009) [2023-12-26 21:36:37,135][105620] Updated weights for policy 1, policy_version 858209 (0.0006) [2023-12-26 21:36:37,188][105620] Updated weights for policy 1, policy_version 858219 (0.0008) [2023-12-26 21:36:37,243][105620] Updated weights for policy 1, policy_version 858229 (0.0009) [2023-12-26 21:36:37,918][105692] Updated weights for policy 0, policy_version 858277 (0.0009) [2023-12-26 21:36:37,975][105692] Updated weights for policy 0, policy_version 858287 (0.0008) [2023-12-26 21:36:37,998][105620] Updated weights for policy 1, policy_version 858239 (0.0008) [2023-12-26 21:36:38,034][105692] Updated weights for policy 0, policy_version 858297 (0.0007) [2023-12-26 21:36:38,060][105620] Updated weights for policy 1, policy_version 858249 (0.0007) [2023-12-26 21:36:38,120][105620] Updated weights for policy 1, policy_version 858259 (0.0007) [2023-12-26 21:36:38,813][105692] Updated weights for policy 0, policy_version 858307 (0.0008) [2023-12-26 21:36:38,849][105620] Updated weights for policy 1, policy_version 858269 (0.0008) [2023-12-26 21:36:38,866][105692] Updated weights for policy 0, policy_version 858317 (0.0008) [2023-12-26 21:36:38,908][105620] Updated weights for policy 1, policy_version 858279 (0.0007) [2023-12-26 21:36:38,925][105692] Updated weights for policy 0, policy_version 858327 (0.0007) [2023-12-26 21:36:38,962][105620] Updated weights for policy 1, policy_version 858289 (0.0007) [2023-12-26 21:36:39,678][105692] Updated weights for policy 0, policy_version 858337 (0.0008) [2023-12-26 21:36:39,721][105620] Updated weights for policy 1, policy_version 858299 (0.0009) [2023-12-26 21:36:39,742][105692] Updated weights for policy 0, policy_version 858347 (0.0011) [2023-12-26 21:36:39,773][105620] Updated weights for policy 1, policy_version 858309 (0.0009) [2023-12-26 21:36:39,798][105692] Updated weights for policy 0, policy_version 858357 (0.0010) [2023-12-26 21:36:39,837][105620] Updated weights for policy 1, policy_version 858319 (0.0006) [2023-12-26 21:36:39,859][105692] Updated weights for policy 0, policy_version 858367 (0.0011) [2023-12-26 21:36:40,596][105692] Updated weights for policy 0, policy_version 858377 (0.0006) [2023-12-26 21:36:40,622][105620] Updated weights for policy 1, policy_version 858329 (0.0007) [2023-12-26 21:36:40,652][105692] Updated weights for policy 0, policy_version 858387 (0.0010) [2023-12-26 21:36:40,675][105620] Updated weights for policy 1, policy_version 858339 (0.0005) [2023-12-26 21:36:40,701][105692] Updated weights for policy 0, policy_version 858397 (0.0011) [2023-12-26 21:36:40,729][105620] Updated weights for policy 1, policy_version 858349 (0.0006) [2023-12-26 21:36:40,795][105620] Updated weights for policy 1, policy_version 858359 (0.0008) [2023-12-26 21:36:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 439549952. Throughput: 0: 9796.7, 1: 9761.4. Samples: 439556580. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-12-26 21:36:41,062][104569] Avg episode reward: [(0, '8987.277'), (1, '8843.693')] [2023-12-26 21:36:41,499][105692] Updated weights for policy 0, policy_version 858407 (0.0011) [2023-12-26 21:36:41,563][105692] Updated weights for policy 0, policy_version 858417 (0.0011) [2023-12-26 21:36:41,594][105620] Updated weights for policy 1, policy_version 858369 (0.0010) [2023-12-26 21:36:41,628][105692] Updated weights for policy 0, policy_version 858427 (0.0010) [2023-12-26 21:36:41,659][105620] Updated weights for policy 1, policy_version 858379 (0.0007) [2023-12-26 21:36:41,730][105620] Updated weights for policy 1, policy_version 858389 (0.0010) [2023-12-26 21:36:42,390][105620] Updated weights for policy 1, policy_version 858399 (0.0008) [2023-12-26 21:36:42,403][105692] Updated weights for policy 0, policy_version 858437 (0.0008) [2023-12-26 21:36:42,443][105620] Updated weights for policy 1, policy_version 858409 (0.0006) [2023-12-26 21:36:42,467][105692] Updated weights for policy 0, policy_version 858447 (0.0011) [2023-12-26 21:36:42,507][105620] Updated weights for policy 1, policy_version 858419 (0.0007) [2023-12-26 21:36:42,533][105692] Updated weights for policy 0, policy_version 858457 (0.0011) [2023-12-26 21:36:43,226][105692] Updated weights for policy 0, policy_version 858467 (0.0008) [2023-12-26 21:36:43,246][105620] Updated weights for policy 1, policy_version 858429 (0.0006) [2023-12-26 21:36:43,285][105692] Updated weights for policy 0, policy_version 858477 (0.0009) [2023-12-26 21:36:43,306][105620] Updated weights for policy 1, policy_version 858439 (0.0006) [2023-12-26 21:36:43,339][105692] Updated weights for policy 0, policy_version 858487 (0.0008) [2023-12-26 21:36:43,369][105620] Updated weights for policy 1, policy_version 858449 (0.0008) [2023-12-26 21:36:43,962][105620] Updated weights for policy 1, policy_version 858459 (0.0007) [2023-12-26 21:36:44,015][105620] Updated weights for policy 1, policy_version 858469 (0.0005) [2023-12-26 21:36:44,068][105620] Updated weights for policy 1, policy_version 858479 (0.0005) [2023-12-26 21:36:44,104][105692] Updated weights for policy 0, policy_version 858497 (0.0007) [2023-12-26 21:36:44,164][105692] Updated weights for policy 0, policy_version 858507 (0.0011) [2023-12-26 21:36:44,219][105692] Updated weights for policy 0, policy_version 858517 (0.0010) [2023-12-26 21:36:44,268][105692] Updated weights for policy 0, policy_version 858527 (0.0010) [2023-12-26 21:36:44,759][105620] Updated weights for policy 1, policy_version 858489 (0.0008) [2023-12-26 21:36:44,820][105620] Updated weights for policy 1, policy_version 858499 (0.0009) [2023-12-26 21:36:44,881][105620] Updated weights for policy 1, policy_version 858509 (0.0008) [2023-12-26 21:36:44,898][105692] Updated weights for policy 0, policy_version 858537 (0.0011) [2023-12-26 21:36:44,938][105620] Updated weights for policy 1, policy_version 858519 (0.0008) [2023-12-26 21:36:44,954][105692] Updated weights for policy 0, policy_version 858547 (0.0011) [2023-12-26 21:36:45,003][105692] Updated weights for policy 0, policy_version 858557 (0.0010) [2023-12-26 21:36:45,653][105620] Updated weights for policy 1, policy_version 858529 (0.0007) [2023-12-26 21:36:45,703][105620] Updated weights for policy 1, policy_version 858539 (0.0007) [2023-12-26 21:36:45,760][105620] Updated weights for policy 1, policy_version 858549 (0.0008) [2023-12-26 21:36:45,773][105692] Updated weights for policy 0, policy_version 858567 (0.0011) [2023-12-26 21:36:45,831][105692] Updated weights for policy 0, policy_version 858577 (0.0010) [2023-12-26 21:36:45,888][105692] Updated weights for policy 0, policy_version 858587 (0.0008) [2023-12-26 21:36:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.7, 300 sec: 19605.2). Total num frames: 439648256. Throughput: 0: 9731.8, 1: 9788.2. Samples: 439614088. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-12-26 21:36:46,063][104569] Avg episode reward: [(0, '8894.483'), (1, '8996.787')] [2023-12-26 21:36:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000858592_219832320.pth... [2023-12-26 21:36:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000858552_219815936.pth... [2023-12-26 21:36:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000857432_219529216.pth [2023-12-26 21:36:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000857440_219537408.pth [2023-12-26 21:36:46,454][105692] Updated weights for policy 0, policy_version 858597 (0.0005) [2023-12-26 21:36:46,500][105692] Updated weights for policy 0, policy_version 858607 (0.0005) [2023-12-26 21:36:46,553][105692] Updated weights for policy 0, policy_version 858617 (0.0006) [2023-12-26 21:36:46,608][105620] Updated weights for policy 1, policy_version 858559 (0.0008) [2023-12-26 21:36:46,667][105620] Updated weights for policy 1, policy_version 858569 (0.0010) [2023-12-26 21:36:46,726][105620] Updated weights for policy 1, policy_version 858579 (0.0009) [2023-12-26 21:36:47,217][105692] Updated weights for policy 0, policy_version 858627 (0.0009) [2023-12-26 21:36:47,278][105692] Updated weights for policy 0, policy_version 858637 (0.0010) [2023-12-26 21:36:47,342][105692] Updated weights for policy 0, policy_version 858647 (0.0010) [2023-12-26 21:36:47,426][105620] Updated weights for policy 1, policy_version 858589 (0.0010) [2023-12-26 21:36:47,490][105620] Updated weights for policy 1, policy_version 858599 (0.0010) [2023-12-26 21:36:47,555][105620] Updated weights for policy 1, policy_version 858609 (0.0010) [2023-12-26 21:36:48,011][105692] Updated weights for policy 0, policy_version 858657 (0.0010) [2023-12-26 21:36:48,070][105692] Updated weights for policy 0, policy_version 858667 (0.0008) [2023-12-26 21:36:48,121][105620] Updated weights for policy 1, policy_version 858619 (0.0009) [2023-12-26 21:36:48,129][105692] Updated weights for policy 0, policy_version 858677 (0.0010) [2023-12-26 21:36:48,174][105620] Updated weights for policy 1, policy_version 858629 (0.0007) [2023-12-26 21:36:48,181][105692] Updated weights for policy 0, policy_version 858687 (0.0010) [2023-12-26 21:36:48,223][105620] Updated weights for policy 1, policy_version 858639 (0.0010) [2023-12-26 21:36:48,837][105692] Updated weights for policy 0, policy_version 858697 (0.0006) [2023-12-26 21:36:48,891][105692] Updated weights for policy 0, policy_version 858707 (0.0005) [2023-12-26 21:36:48,937][105692] Updated weights for policy 0, policy_version 858717 (0.0005) [2023-12-26 21:36:49,015][105620] Updated weights for policy 1, policy_version 858649 (0.0010) [2023-12-26 21:36:49,074][105620] Updated weights for policy 1, policy_version 858659 (0.0010) [2023-12-26 21:36:49,133][105620] Updated weights for policy 1, policy_version 858669 (0.0010) [2023-12-26 21:36:49,192][105620] Updated weights for policy 1, policy_version 858679 (0.0008) [2023-12-26 21:36:49,578][105692] Updated weights for policy 0, policy_version 858727 (0.0008) [2023-12-26 21:36:49,638][105692] Updated weights for policy 0, policy_version 858737 (0.0009) [2023-12-26 21:36:49,695][105692] Updated weights for policy 0, policy_version 858747 (0.0005) [2023-12-26 21:36:49,995][105620] Updated weights for policy 1, policy_version 858689 (0.0006) [2023-12-26 21:36:50,054][105620] Updated weights for policy 1, policy_version 858699 (0.0006) [2023-12-26 21:36:50,113][105620] Updated weights for policy 1, policy_version 858709 (0.0006) [2023-12-26 21:36:50,416][105692] Updated weights for policy 0, policy_version 858757 (0.0009) [2023-12-26 21:36:50,477][105692] Updated weights for policy 0, policy_version 858767 (0.0010) [2023-12-26 21:36:50,531][105692] Updated weights for policy 0, policy_version 858777 (0.0010) [2023-12-26 21:36:50,828][105620] Updated weights for policy 1, policy_version 858719 (0.0008) [2023-12-26 21:36:50,895][105620] Updated weights for policy 1, policy_version 858729 (0.0008) [2023-12-26 21:36:50,956][105620] Updated weights for policy 1, policy_version 858739 (0.0009) [2023-12-26 21:36:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 439746560. Throughput: 0: 9834.3, 1: 9723.1. Samples: 439734108. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-12-26 21:36:51,063][104569] Avg episode reward: [(0, '9070.402'), (1, '8944.943')] [2023-12-26 21:36:51,275][105692] Updated weights for policy 0, policy_version 858787 (0.0010) [2023-12-26 21:36:51,334][105692] Updated weights for policy 0, policy_version 858797 (0.0010) [2023-12-26 21:36:51,402][105692] Updated weights for policy 0, policy_version 858807 (0.0009) [2023-12-26 21:36:51,710][105620] Updated weights for policy 1, policy_version 858749 (0.0008) [2023-12-26 21:36:51,777][105620] Updated weights for policy 1, policy_version 858759 (0.0008) [2023-12-26 21:36:51,843][105620] Updated weights for policy 1, policy_version 858769 (0.0009) [2023-12-26 21:36:52,167][105692] Updated weights for policy 0, policy_version 858817 (0.0008) [2023-12-26 21:36:52,211][105692] Updated weights for policy 0, policy_version 858827 (0.0008) [2023-12-26 21:36:52,256][105692] Updated weights for policy 0, policy_version 858837 (0.0008) [2023-12-26 21:36:52,323][105692] Updated weights for policy 0, policy_version 858847 (0.0009) [2023-12-26 21:36:52,529][105620] Updated weights for policy 1, policy_version 858779 (0.0009) [2023-12-26 21:36:52,591][105620] Updated weights for policy 1, policy_version 858789 (0.0008) [2023-12-26 21:36:52,653][105620] Updated weights for policy 1, policy_version 858799 (0.0009) [2023-12-26 21:36:53,097][105692] Updated weights for policy 0, policy_version 858857 (0.0008) [2023-12-26 21:36:53,154][105692] Updated weights for policy 0, policy_version 858867 (0.0009) [2023-12-26 21:36:53,207][105692] Updated weights for policy 0, policy_version 858877 (0.0009) [2023-12-26 21:36:53,324][105620] Updated weights for policy 1, policy_version 858809 (0.0007) [2023-12-26 21:36:53,384][105620] Updated weights for policy 1, policy_version 858819 (0.0010) [2023-12-26 21:36:53,439][105620] Updated weights for policy 1, policy_version 858829 (0.0009) [2023-12-26 21:36:53,493][105620] Updated weights for policy 1, policy_version 858839 (0.0009) [2023-12-26 21:36:53,998][105692] Updated weights for policy 0, policy_version 858887 (0.0007) [2023-12-26 21:36:54,053][105692] Updated weights for policy 0, policy_version 858897 (0.0009) [2023-12-26 21:36:54,110][105692] Updated weights for policy 0, policy_version 858907 (0.0008) [2023-12-26 21:36:54,145][105620] Updated weights for policy 1, policy_version 858849 (0.0006) [2023-12-26 21:36:54,203][105620] Updated weights for policy 1, policy_version 858859 (0.0009) [2023-12-26 21:36:54,261][105620] Updated weights for policy 1, policy_version 858869 (0.0008) [2023-12-26 21:36:54,804][105692] Updated weights for policy 0, policy_version 858917 (0.0006) [2023-12-26 21:36:54,855][105692] Updated weights for policy 0, policy_version 858927 (0.0005) [2023-12-26 21:36:54,911][105692] Updated weights for policy 0, policy_version 858937 (0.0005) [2023-12-26 21:36:54,939][105620] Updated weights for policy 1, policy_version 858879 (0.0008) [2023-12-26 21:36:55,003][105620] Updated weights for policy 1, policy_version 858889 (0.0010) [2023-12-26 21:36:55,067][105620] Updated weights for policy 1, policy_version 858899 (0.0009) [2023-12-26 21:36:55,473][105692] Updated weights for policy 0, policy_version 858947 (0.0005) [2023-12-26 21:36:55,531][105692] Updated weights for policy 0, policy_version 858957 (0.0005) [2023-12-26 21:36:55,583][105692] Updated weights for policy 0, policy_version 858967 (0.0005) [2023-12-26 21:36:55,686][105620] Updated weights for policy 1, policy_version 858909 (0.0009) [2023-12-26 21:36:55,734][105620] Updated weights for policy 1, policy_version 858919 (0.0008) [2023-12-26 21:36:55,782][105620] Updated weights for policy 1, policy_version 858929 (0.0008) [2023-12-26 21:36:56,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 439844864. Throughput: 0: 9847.1, 1: 9676.6. Samples: 439852336. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-12-26 21:36:56,062][104569] Avg episode reward: [(0, '9251.166'), (1, '9075.057')] [2023-12-26 21:36:56,254][105692] Updated weights for policy 0, policy_version 858977 (0.0007) [2023-12-26 21:36:56,302][105692] Updated weights for policy 0, policy_version 858987 (0.0010) [2023-12-26 21:36:56,351][105692] Updated weights for policy 0, policy_version 858997 (0.0010) [2023-12-26 21:36:56,386][105620] Updated weights for policy 1, policy_version 858939 (0.0008) [2023-12-26 21:36:56,404][105692] Updated weights for policy 0, policy_version 859007 (0.0010) [2023-12-26 21:36:56,441][105620] Updated weights for policy 1, policy_version 858949 (0.0010) [2023-12-26 21:36:56,510][105620] Updated weights for policy 1, policy_version 858959 (0.0006) [2023-12-26 21:36:57,030][105620] Updated weights for policy 1, policy_version 858969 (0.0005) [2023-12-26 21:36:57,074][105692] Updated weights for policy 0, policy_version 859017 (0.0006) [2023-12-26 21:36:57,093][105620] Updated weights for policy 1, policy_version 858979 (0.0005) [2023-12-26 21:36:57,126][105692] Updated weights for policy 0, policy_version 859027 (0.0005) [2023-12-26 21:36:57,151][105620] Updated weights for policy 1, policy_version 858989 (0.0005) [2023-12-26 21:36:57,187][105692] Updated weights for policy 0, policy_version 859037 (0.0006) [2023-12-26 21:36:57,204][105620] Updated weights for policy 1, policy_version 858999 (0.0005) [2023-12-26 21:36:57,762][105620] Updated weights for policy 1, policy_version 859009 (0.0005) [2023-12-26 21:36:57,819][105620] Updated weights for policy 1, policy_version 859019 (0.0006) [2023-12-26 21:36:57,867][105692] Updated weights for policy 0, policy_version 859047 (0.0005) [2023-12-26 21:36:57,874][105620] Updated weights for policy 1, policy_version 859029 (0.0010) [2023-12-26 21:36:57,932][105692] Updated weights for policy 0, policy_version 859057 (0.0005) [2023-12-26 21:36:57,982][105692] Updated weights for policy 0, policy_version 859067 (0.0005) [2023-12-26 21:36:58,555][105620] Updated weights for policy 1, policy_version 859039 (0.0007) [2023-12-26 21:36:58,618][105620] Updated weights for policy 1, policy_version 859049 (0.0009) [2023-12-26 21:36:58,691][105620] Updated weights for policy 1, policy_version 859059 (0.0010) [2023-12-26 21:36:58,701][105692] Updated weights for policy 0, policy_version 859077 (0.0007) [2023-12-26 21:36:58,759][105692] Updated weights for policy 0, policy_version 859087 (0.0008) [2023-12-26 21:36:58,825][105692] Updated weights for policy 0, policy_version 859097 (0.0007) [2023-12-26 21:36:59,478][105620] Updated weights for policy 1, policy_version 859069 (0.0008) [2023-12-26 21:36:59,529][105620] Updated weights for policy 1, policy_version 859079 (0.0010) [2023-12-26 21:36:59,581][105620] Updated weights for policy 1, policy_version 859089 (0.0010) [2023-12-26 21:36:59,618][105692] Updated weights for policy 0, policy_version 859107 (0.0008) [2023-12-26 21:36:59,682][105692] Updated weights for policy 0, policy_version 859117 (0.0009) [2023-12-26 21:36:59,737][105692] Updated weights for policy 0, policy_version 859127 (0.0009) [2023-12-26 21:37:00,290][105620] Updated weights for policy 1, policy_version 859099 (0.0010) [2023-12-26 21:37:00,345][105620] Updated weights for policy 1, policy_version 859109 (0.0010) [2023-12-26 21:37:00,402][105620] Updated weights for policy 1, policy_version 859119 (0.0010) [2023-12-26 21:37:00,490][105692] Updated weights for policy 0, policy_version 859137 (0.0008) [2023-12-26 21:37:00,548][105692] Updated weights for policy 0, policy_version 859147 (0.0005) [2023-12-26 21:37:00,604][105692] Updated weights for policy 0, policy_version 859157 (0.0005) [2023-12-26 21:37:00,663][105692] Updated weights for policy 0, policy_version 859167 (0.0005) [2023-12-26 21:37:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 439943168. Throughput: 0: 9887.5, 1: 9807.2. Samples: 439916456. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-12-26 21:37:01,062][104569] Avg episode reward: [(0, '9342.254'), (1, '9074.020')] [2023-12-26 21:37:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000859168_219979776.pth... [2023-12-26 21:37:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000858016_219684864.pth [2023-12-26 21:37:01,084][105620] Updated weights for policy 1, policy_version 859129 (0.0010) [2023-12-26 21:37:01,147][105620] Updated weights for policy 1, policy_version 859139 (0.0008) [2023-12-26 21:37:01,209][105620] Updated weights for policy 1, policy_version 859149 (0.0008) [2023-12-26 21:37:01,275][105620] Updated weights for policy 1, policy_version 859159 (0.0008) [2023-12-26 21:37:01,279][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000859160_219971584.pth... [2023-12-26 21:37:01,283][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000857976_219668480.pth [2023-12-26 21:37:01,298][105692] Updated weights for policy 0, policy_version 859177 (0.0010) [2023-12-26 21:37:01,364][105692] Updated weights for policy 0, policy_version 859187 (0.0011) [2023-12-26 21:37:01,421][105692] Updated weights for policy 0, policy_version 859197 (0.0011) [2023-12-26 21:37:01,945][105620] Updated weights for policy 1, policy_version 859169 (0.0008) [2023-12-26 21:37:01,993][105620] Updated weights for policy 1, policy_version 859179 (0.0008) [2023-12-26 21:37:02,053][105620] Updated weights for policy 1, policy_version 859189 (0.0008) [2023-12-26 21:37:02,155][105692] Updated weights for policy 0, policy_version 859207 (0.0011) [2023-12-26 21:37:02,213][105692] Updated weights for policy 0, policy_version 859217 (0.0010) [2023-12-26 21:37:02,278][105692] Updated weights for policy 0, policy_version 859227 (0.0011) [2023-12-26 21:37:02,835][105620] Updated weights for policy 1, policy_version 859199 (0.0008) [2023-12-26 21:37:02,898][105620] Updated weights for policy 1, policy_version 859209 (0.0008) [2023-12-26 21:37:02,962][105620] Updated weights for policy 1, policy_version 859219 (0.0009) [2023-12-26 21:37:02,980][105692] Updated weights for policy 0, policy_version 859237 (0.0011) [2023-12-26 21:37:03,033][105692] Updated weights for policy 0, policy_version 859247 (0.0009) [2023-12-26 21:37:03,086][105692] Updated weights for policy 0, policy_version 859257 (0.0006) [2023-12-26 21:37:03,656][105692] Updated weights for policy 0, policy_version 859267 (0.0007) [2023-12-26 21:37:03,701][105692] Updated weights for policy 0, policy_version 859277 (0.0005) [2023-12-26 21:37:03,751][105692] Updated weights for policy 0, policy_version 859287 (0.0006) [2023-12-26 21:37:03,787][105620] Updated weights for policy 1, policy_version 859229 (0.0008) [2023-12-26 21:37:03,841][105620] Updated weights for policy 1, policy_version 859239 (0.0009) [2023-12-26 21:37:03,905][105620] Updated weights for policy 1, policy_version 859249 (0.0009) [2023-12-26 21:37:04,374][105692] Updated weights for policy 0, policy_version 859297 (0.0006) [2023-12-26 21:37:04,427][105692] Updated weights for policy 0, policy_version 859307 (0.0011) [2023-12-26 21:37:04,477][105692] Updated weights for policy 0, policy_version 859317 (0.0010) [2023-12-26 21:37:04,538][105692] Updated weights for policy 0, policy_version 859327 (0.0010) [2023-12-26 21:37:04,650][105620] Updated weights for policy 1, policy_version 859259 (0.0009) [2023-12-26 21:37:04,717][105620] Updated weights for policy 1, policy_version 859269 (0.0009) [2023-12-26 21:37:04,773][105620] Updated weights for policy 1, policy_version 859279 (0.0008) [2023-12-26 21:37:05,246][105692] Updated weights for policy 0, policy_version 859337 (0.0010) [2023-12-26 21:37:05,294][105692] Updated weights for policy 0, policy_version 859347 (0.0010) [2023-12-26 21:37:05,348][105692] Updated weights for policy 0, policy_version 859357 (0.0010) [2023-12-26 21:37:05,539][105620] Updated weights for policy 1, policy_version 859289 (0.0009) [2023-12-26 21:37:05,598][105620] Updated weights for policy 1, policy_version 859299 (0.0006) [2023-12-26 21:37:05,656][105620] Updated weights for policy 1, policy_version 859309 (0.0008) [2023-12-26 21:37:05,716][105620] Updated weights for policy 1, policy_version 859319 (0.0008) [2023-12-26 21:37:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 440041472. Throughput: 0: 9881.5, 1: 9756.3. Samples: 440033176. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-12-26 21:37:06,062][104569] Avg episode reward: [(0, '9342.255'), (1, '9257.291')] [2023-12-26 21:37:06,096][105692] Updated weights for policy 0, policy_version 859367 (0.0011) [2023-12-26 21:37:06,150][105692] Updated weights for policy 0, policy_version 859377 (0.0011) [2023-12-26 21:37:06,199][105692] Updated weights for policy 0, policy_version 859387 (0.0010) [2023-12-26 21:37:06,462][105620] Updated weights for policy 1, policy_version 859329 (0.0006) [2023-12-26 21:37:06,523][105620] Updated weights for policy 1, policy_version 859339 (0.0006) [2023-12-26 21:37:06,584][105620] Updated weights for policy 1, policy_version 859349 (0.0006) [2023-12-26 21:37:06,927][105692] Updated weights for policy 0, policy_version 859397 (0.0010) [2023-12-26 21:37:06,993][105692] Updated weights for policy 0, policy_version 859407 (0.0010) [2023-12-26 21:37:07,052][105692] Updated weights for policy 0, policy_version 859417 (0.0010) [2023-12-26 21:37:07,199][105620] Updated weights for policy 1, policy_version 859359 (0.0006) [2023-12-26 21:37:07,255][105620] Updated weights for policy 1, policy_version 859369 (0.0008) [2023-12-26 21:37:07,308][105620] Updated weights for policy 1, policy_version 859379 (0.0008) [2023-12-26 21:37:07,795][105692] Updated weights for policy 0, policy_version 859427 (0.0009) [2023-12-26 21:37:07,841][105692] Updated weights for policy 0, policy_version 859437 (0.0010) [2023-12-26 21:37:07,893][105692] Updated weights for policy 0, policy_version 859447 (0.0010) [2023-12-26 21:37:07,958][105620] Updated weights for policy 1, policy_version 859389 (0.0008) [2023-12-26 21:37:08,011][105620] Updated weights for policy 1, policy_version 859399 (0.0008) [2023-12-26 21:37:08,067][105620] Updated weights for policy 1, policy_version 859409 (0.0008) [2023-12-26 21:37:08,687][105692] Updated weights for policy 0, policy_version 859457 (0.0010) [2023-12-26 21:37:08,751][105692] Updated weights for policy 0, policy_version 859467 (0.0009) [2023-12-26 21:37:08,792][105620] Updated weights for policy 1, policy_version 859419 (0.0007) [2023-12-26 21:37:08,813][105692] Updated weights for policy 0, policy_version 859477 (0.0011) [2023-12-26 21:37:08,854][105620] Updated weights for policy 1, policy_version 859429 (0.0006) [2023-12-26 21:37:08,870][105692] Updated weights for policy 0, policy_version 859487 (0.0007) [2023-12-26 21:37:08,913][105620] Updated weights for policy 1, policy_version 859439 (0.0007) [2023-12-26 21:37:09,629][105692] Updated weights for policy 0, policy_version 859497 (0.0009) [2023-12-26 21:37:09,640][105620] Updated weights for policy 1, policy_version 859449 (0.0007) [2023-12-26 21:37:09,690][105692] Updated weights for policy 0, policy_version 859507 (0.0007) [2023-12-26 21:37:09,700][105620] Updated weights for policy 1, policy_version 859459 (0.0007) [2023-12-26 21:37:09,747][105692] Updated weights for policy 0, policy_version 859517 (0.0008) [2023-12-26 21:37:09,762][105620] Updated weights for policy 1, policy_version 859469 (0.0007) [2023-12-26 21:37:09,818][105620] Updated weights for policy 1, policy_version 859479 (0.0009) [2023-12-26 21:37:10,474][105620] Updated weights for policy 1, policy_version 859489 (0.0006) [2023-12-26 21:37:10,542][105620] Updated weights for policy 1, policy_version 859499 (0.0007) [2023-12-26 21:37:10,577][105692] Updated weights for policy 0, policy_version 859527 (0.0009) [2023-12-26 21:37:10,602][105620] Updated weights for policy 1, policy_version 859509 (0.0006) [2023-12-26 21:37:10,635][105692] Updated weights for policy 0, policy_version 859537 (0.0010) [2023-12-26 21:37:10,698][105692] Updated weights for policy 0, policy_version 859547 (0.0009) [2023-12-26 21:37:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 440139776. Throughput: 0: 9866.1, 1: 9809.3. Samples: 440149516. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-12-26 21:37:11,062][104569] Avg episode reward: [(0, '9253.396'), (1, '8797.369')] [2023-12-26 21:37:11,303][105620] Updated weights for policy 1, policy_version 859519 (0.0006) [2023-12-26 21:37:11,371][105620] Updated weights for policy 1, policy_version 859529 (0.0009) [2023-12-26 21:37:11,439][105620] Updated weights for policy 1, policy_version 859539 (0.0007) [2023-12-26 21:37:11,461][105692] Updated weights for policy 0, policy_version 859557 (0.0008) [2023-12-26 21:37:11,518][105692] Updated weights for policy 0, policy_version 859567 (0.0006) [2023-12-26 21:37:11,583][105692] Updated weights for policy 0, policy_version 859577 (0.0005) [2023-12-26 21:37:12,208][105620] Updated weights for policy 1, policy_version 859549 (0.0006) [2023-12-26 21:37:12,217][105692] Updated weights for policy 0, policy_version 859587 (0.0007) [2023-12-26 21:37:12,262][105620] Updated weights for policy 1, policy_version 859559 (0.0007) [2023-12-26 21:37:12,279][105692] Updated weights for policy 0, policy_version 859597 (0.0007) [2023-12-26 21:37:12,325][105620] Updated weights for policy 1, policy_version 859569 (0.0007) [2023-12-26 21:37:12,340][105692] Updated weights for policy 0, policy_version 859607 (0.0007) [2023-12-26 21:37:13,001][105692] Updated weights for policy 0, policy_version 859617 (0.0008) [2023-12-26 21:37:13,053][105692] Updated weights for policy 0, policy_version 859627 (0.0005) [2023-12-26 21:37:13,068][105620] Updated weights for policy 1, policy_version 859579 (0.0007) [2023-12-26 21:37:13,115][105692] Updated weights for policy 0, policy_version 859637 (0.0011) [2023-12-26 21:37:13,122][105620] Updated weights for policy 1, policy_version 859589 (0.0008) [2023-12-26 21:37:13,178][105692] Updated weights for policy 0, policy_version 859647 (0.0010) [2023-12-26 21:37:13,178][105620] Updated weights for policy 1, policy_version 859599 (0.0007) [2023-12-26 21:37:13,751][105692] Updated weights for policy 0, policy_version 859657 (0.0010) [2023-12-26 21:37:13,799][105692] Updated weights for policy 0, policy_version 859667 (0.0010) [2023-12-26 21:37:13,847][105692] Updated weights for policy 0, policy_version 859677 (0.0010) [2023-12-26 21:37:13,853][105620] Updated weights for policy 1, policy_version 859609 (0.0010) [2023-12-26 21:37:13,916][105620] Updated weights for policy 1, policy_version 859619 (0.0009) [2023-12-26 21:37:13,970][105620] Updated weights for policy 1, policy_version 859631 (0.0010) [2023-12-26 21:37:14,499][105692] Updated weights for policy 0, policy_version 859687 (0.0010) [2023-12-26 21:37:14,557][105692] Updated weights for policy 0, policy_version 859697 (0.0011) [2023-12-26 21:37:14,612][105692] Updated weights for policy 0, policy_version 859707 (0.0010) [2023-12-26 21:37:14,795][105620] Updated weights for policy 1, policy_version 859642 (0.0010) [2023-12-26 21:37:14,857][105620] Updated weights for policy 1, policy_version 859652 (0.0008) [2023-12-26 21:37:14,914][105620] Updated weights for policy 1, policy_version 859662 (0.0007) [2023-12-26 21:37:14,977][105620] Updated weights for policy 1, policy_version 859672 (0.0005) [2023-12-26 21:37:15,364][105692] Updated weights for policy 0, policy_version 859717 (0.0010) [2023-12-26 21:37:15,427][105692] Updated weights for policy 0, policy_version 859727 (0.0011) [2023-12-26 21:37:15,487][105692] Updated weights for policy 0, policy_version 859737 (0.0011) [2023-12-26 21:37:15,574][105620] Updated weights for policy 1, policy_version 859682 (0.0008) [2023-12-26 21:37:15,630][105620] Updated weights for policy 1, policy_version 859692 (0.0007) [2023-12-26 21:37:15,689][105620] Updated weights for policy 1, policy_version 859702 (0.0005) [2023-12-26 21:37:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19605.2). Total num frames: 440238080. Throughput: 0: 9848.0, 1: 9816.8. Samples: 440208528. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-12-26 21:37:16,063][104569] Avg episode reward: [(0, '9166.706'), (1, '8613.666')] [2023-12-26 21:37:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000859744_220127232.pth... [2023-12-26 21:37:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000859704_220110848.pth... [2023-12-26 21:37:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000858552_219815936.pth [2023-12-26 21:37:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000858592_219832320.pth [2023-12-26 21:37:16,257][105692] Updated weights for policy 0, policy_version 859747 (0.0010) [2023-12-26 21:37:16,312][105620] Updated weights for policy 1, policy_version 859712 (0.0009) [2023-12-26 21:37:16,313][105692] Updated weights for policy 0, policy_version 859757 (0.0006) [2023-12-26 21:37:16,361][105620] Updated weights for policy 1, policy_version 859722 (0.0008) [2023-12-26 21:37:16,367][105692] Updated weights for policy 0, policy_version 859767 (0.0006) [2023-12-26 21:37:16,413][105620] Updated weights for policy 1, policy_version 859732 (0.0010) [2023-12-26 21:37:17,046][105692] Updated weights for policy 0, policy_version 859777 (0.0006) [2023-12-26 21:37:17,093][105620] Updated weights for policy 1, policy_version 859742 (0.0007) [2023-12-26 21:37:17,107][105692] Updated weights for policy 0, policy_version 859787 (0.0005) [2023-12-26 21:37:17,156][105620] Updated weights for policy 1, policy_version 859752 (0.0005) [2023-12-26 21:37:17,163][105692] Updated weights for policy 0, policy_version 859797 (0.0006) [2023-12-26 21:37:17,218][105692] Updated weights for policy 0, policy_version 859807 (0.0006) [2023-12-26 21:37:17,223][105620] Updated weights for policy 1, policy_version 859762 (0.0005) [2023-12-26 21:37:17,724][105692] Updated weights for policy 0, policy_version 859817 (0.0006) [2023-12-26 21:37:17,733][105620] Updated weights for policy 1, policy_version 859772 (0.0006) [2023-12-26 21:37:17,771][105692] Updated weights for policy 0, policy_version 859827 (0.0005) [2023-12-26 21:37:17,793][105620] Updated weights for policy 1, policy_version 859782 (0.0005) [2023-12-26 21:37:17,830][105692] Updated weights for policy 0, policy_version 859837 (0.0006) [2023-12-26 21:37:17,853][105620] Updated weights for policy 1, policy_version 859792 (0.0009) [2023-12-26 21:37:18,563][105620] Updated weights for policy 1, policy_version 859803 (0.0010) [2023-12-26 21:37:18,569][105692] Updated weights for policy 0, policy_version 859847 (0.0008) [2023-12-26 21:37:18,626][105620] Updated weights for policy 1, policy_version 859813 (0.0010) [2023-12-26 21:37:18,629][105692] Updated weights for policy 0, policy_version 859857 (0.0006) [2023-12-26 21:37:18,685][105692] Updated weights for policy 0, policy_version 859867 (0.0007) [2023-12-26 21:37:18,685][105620] Updated weights for policy 1, policy_version 859823 (0.0010) [2023-12-26 21:37:19,439][105620] Updated weights for policy 1, policy_version 859833 (0.0010) [2023-12-26 21:37:19,469][105692] Updated weights for policy 0, policy_version 859877 (0.0008) [2023-12-26 21:37:19,505][105620] Updated weights for policy 1, policy_version 859843 (0.0009) [2023-12-26 21:37:19,533][105692] Updated weights for policy 0, policy_version 859887 (0.0006) [2023-12-26 21:37:19,579][105620] Updated weights for policy 1, policy_version 859853 (0.0006) [2023-12-26 21:37:19,599][105692] Updated weights for policy 0, policy_version 859897 (0.0010) [2023-12-26 21:37:19,641][105620] Updated weights for policy 1, policy_version 859863 (0.0010) [2023-12-26 21:37:20,332][105692] Updated weights for policy 0, policy_version 859907 (0.0009) [2023-12-26 21:37:20,345][105620] Updated weights for policy 1, policy_version 859873 (0.0010) [2023-12-26 21:37:20,388][105692] Updated weights for policy 0, policy_version 859917 (0.0007) [2023-12-26 21:37:20,401][105620] Updated weights for policy 1, policy_version 859883 (0.0010) [2023-12-26 21:37:20,438][105692] Updated weights for policy 0, policy_version 859927 (0.0005) [2023-12-26 21:37:20,460][105620] Updated weights for policy 1, policy_version 859893 (0.0010) [2023-12-26 21:37:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 440336384. Throughput: 0: 9866.9, 1: 9845.3. Samples: 440329704. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-12-26 21:37:21,063][104569] Avg episode reward: [(0, '9077.985'), (1, '9166.003')] [2023-12-26 21:37:21,204][105692] Updated weights for policy 0, policy_version 859937 (0.0006) [2023-12-26 21:37:21,218][105620] Updated weights for policy 1, policy_version 859903 (0.0011) [2023-12-26 21:37:21,265][105692] Updated weights for policy 0, policy_version 859947 (0.0006) [2023-12-26 21:37:21,279][105620] Updated weights for policy 1, policy_version 859913 (0.0011) [2023-12-26 21:37:21,321][105692] Updated weights for policy 0, policy_version 859957 (0.0007) [2023-12-26 21:37:21,339][105620] Updated weights for policy 1, policy_version 859923 (0.0007) [2023-12-26 21:37:21,382][105692] Updated weights for policy 0, policy_version 859967 (0.0008) [2023-12-26 21:37:22,049][105620] Updated weights for policy 1, policy_version 859933 (0.0008) [2023-12-26 21:37:22,109][105620] Updated weights for policy 1, policy_version 859943 (0.0008) [2023-12-26 21:37:22,161][105620] Updated weights for policy 1, policy_version 859953 (0.0006) [2023-12-26 21:37:22,175][105692] Updated weights for policy 0, policy_version 859977 (0.0008) [2023-12-26 21:37:22,226][105692] Updated weights for policy 0, policy_version 859987 (0.0007) [2023-12-26 21:37:22,286][105692] Updated weights for policy 0, policy_version 859997 (0.0009) [2023-12-26 21:37:22,874][105620] Updated weights for policy 1, policy_version 859963 (0.0006) [2023-12-26 21:37:22,939][105620] Updated weights for policy 1, policy_version 859973 (0.0007) [2023-12-26 21:37:22,998][105620] Updated weights for policy 1, policy_version 859983 (0.0009) [2023-12-26 21:37:23,146][105692] Updated weights for policy 0, policy_version 860007 (0.0009) [2023-12-26 21:37:23,217][105692] Updated weights for policy 0, policy_version 860017 (0.0007) [2023-12-26 21:37:23,279][105692] Updated weights for policy 0, policy_version 860027 (0.0009) [2023-12-26 21:37:23,770][105620] Updated weights for policy 1, policy_version 859993 (0.0009) [2023-12-26 21:37:23,835][105620] Updated weights for policy 1, policy_version 860003 (0.0009) [2023-12-26 21:37:23,897][105620] Updated weights for policy 1, policy_version 860013 (0.0009) [2023-12-26 21:37:23,935][105692] Updated weights for policy 0, policy_version 860037 (0.0010) [2023-12-26 21:37:23,954][105620] Updated weights for policy 1, policy_version 860023 (0.0007) [2023-12-26 21:37:23,983][105692] Updated weights for policy 0, policy_version 860047 (0.0007) [2023-12-26 21:37:24,035][105692] Updated weights for policy 0, policy_version 860057 (0.0009) [2023-12-26 21:37:24,674][105620] Updated weights for policy 1, policy_version 860033 (0.0009) [2023-12-26 21:37:24,736][105620] Updated weights for policy 1, policy_version 860043 (0.0009) [2023-12-26 21:37:24,796][105620] Updated weights for policy 1, policy_version 860053 (0.0009) [2023-12-26 21:37:24,809][105692] Updated weights for policy 0, policy_version 860067 (0.0007) [2023-12-26 21:37:24,866][105692] Updated weights for policy 0, policy_version 860077 (0.0009) [2023-12-26 21:37:24,921][105692] Updated weights for policy 0, policy_version 860087 (0.0009) [2023-12-26 21:37:25,507][105620] Updated weights for policy 1, policy_version 860063 (0.0010) [2023-12-26 21:37:25,572][105620] Updated weights for policy 1, policy_version 860073 (0.0009) [2023-12-26 21:37:25,636][105620] Updated weights for policy 1, policy_version 860083 (0.0008) [2023-12-26 21:37:25,643][105692] Updated weights for policy 0, policy_version 860097 (0.0008) [2023-12-26 21:37:25,695][105692] Updated weights for policy 0, policy_version 860107 (0.0009) [2023-12-26 21:37:25,759][105692] Updated weights for policy 0, policy_version 860117 (0.0009) [2023-12-26 21:37:25,816][105692] Updated weights for policy 0, policy_version 860127 (0.0009) [2023-12-26 21:37:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 440434688. Throughput: 0: 9818.3, 1: 9861.8. Samples: 440442188. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-12-26 21:37:26,063][104569] Avg episode reward: [(0, '9169.097'), (1, '9349.652')] [2023-12-26 21:37:26,368][105620] Updated weights for policy 1, policy_version 860093 (0.0007) [2023-12-26 21:37:26,430][105620] Updated weights for policy 1, policy_version 860103 (0.0005) [2023-12-26 21:37:26,484][105620] Updated weights for policy 1, policy_version 860113 (0.0006) [2023-12-26 21:37:26,502][105692] Updated weights for policy 0, policy_version 860137 (0.0006) [2023-12-26 21:37:26,551][105692] Updated weights for policy 0, policy_version 860147 (0.0005) [2023-12-26 21:37:26,595][105692] Updated weights for policy 0, policy_version 860157 (0.0005) [2023-12-26 21:37:27,132][105620] Updated weights for policy 1, policy_version 860123 (0.0005) [2023-12-26 21:37:27,132][105692] Updated weights for policy 0, policy_version 860167 (0.0009) [2023-12-26 21:37:27,180][105692] Updated weights for policy 0, policy_version 860177 (0.0010) [2023-12-26 21:37:27,197][105620] Updated weights for policy 1, policy_version 860133 (0.0005) [2023-12-26 21:37:27,235][105692] Updated weights for policy 0, policy_version 860187 (0.0010) [2023-12-26 21:37:27,260][105620] Updated weights for policy 1, policy_version 860143 (0.0005) [2023-12-26 21:37:27,764][105620] Updated weights for policy 1, policy_version 860153 (0.0006) [2023-12-26 21:37:27,828][105620] Updated weights for policy 1, policy_version 860163 (0.0005) [2023-12-26 21:37:27,891][105620] Updated weights for policy 1, policy_version 860173 (0.0005) [2023-12-26 21:37:27,944][105620] Updated weights for policy 1, policy_version 860183 (0.0005) [2023-12-26 21:37:27,995][105692] Updated weights for policy 0, policy_version 860197 (0.0010) [2023-12-26 21:37:28,059][105692] Updated weights for policy 0, policy_version 860207 (0.0008) [2023-12-26 21:37:28,103][105692] Updated weights for policy 0, policy_version 860217 (0.0010) [2023-12-26 21:37:28,660][105620] Updated weights for policy 1, policy_version 860193 (0.0007) [2023-12-26 21:37:28,716][105692] Updated weights for policy 0, policy_version 860227 (0.0010) [2023-12-26 21:37:28,719][105620] Updated weights for policy 1, policy_version 860203 (0.0005) [2023-12-26 21:37:28,765][105692] Updated weights for policy 0, policy_version 860237 (0.0010) [2023-12-26 21:37:28,781][105620] Updated weights for policy 1, policy_version 860213 (0.0005) [2023-12-26 21:37:28,814][105692] Updated weights for policy 0, policy_version 860247 (0.0010) [2023-12-26 21:37:29,446][105620] Updated weights for policy 1, policy_version 860223 (0.0007) [2023-12-26 21:37:29,491][105620] Updated weights for policy 1, policy_version 860233 (0.0008) [2023-12-26 21:37:29,535][105620] Updated weights for policy 1, policy_version 860243 (0.0008) [2023-12-26 21:37:29,536][105692] Updated weights for policy 0, policy_version 860257 (0.0010) [2023-12-26 21:37:29,593][105692] Updated weights for policy 0, policy_version 860267 (0.0008) [2023-12-26 21:37:29,648][105692] Updated weights for policy 0, policy_version 860277 (0.0010) [2023-12-26 21:37:29,710][105692] Updated weights for policy 0, policy_version 860287 (0.0010) [2023-12-26 21:37:30,369][105620] Updated weights for policy 1, policy_version 860253 (0.0007) [2023-12-26 21:37:30,390][105692] Updated weights for policy 0, policy_version 860297 (0.0007) [2023-12-26 21:37:30,428][105620] Updated weights for policy 1, policy_version 860263 (0.0008) [2023-12-26 21:37:30,447][105692] Updated weights for policy 0, policy_version 860307 (0.0007) [2023-12-26 21:37:30,476][105620] Updated weights for policy 1, policy_version 860273 (0.0005) [2023-12-26 21:37:30,493][105692] Updated weights for policy 0, policy_version 860317 (0.0005) [2023-12-26 21:37:31,049][105692] Updated weights for policy 0, policy_version 860327 (0.0009) [2023-12-26 21:37:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 440532992. Throughput: 0: 9931.5, 1: 9911.9. Samples: 440507036. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-12-26 21:37:31,062][104569] Avg episode reward: [(0, '8897.128'), (1, '9168.865')] [2023-12-26 21:37:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000860280_220258304.pth... [2023-12-26 21:37:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000859160_219971584.pth [2023-12-26 21:37:31,108][105692] Updated weights for policy 0, policy_version 860337 (0.0010) [2023-12-26 21:37:31,166][105692] Updated weights for policy 0, policy_version 860347 (0.0008) [2023-12-26 21:37:31,189][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000860352_220282880.pth... [2023-12-26 21:37:31,191][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000859168_219979776.pth [2023-12-26 21:37:31,332][105620] Updated weights for policy 1, policy_version 860283 (0.0006) [2023-12-26 21:37:31,404][105620] Updated weights for policy 1, policy_version 860293 (0.0009) [2023-12-26 21:37:31,456][105620] Updated weights for policy 1, policy_version 860303 (0.0010) [2023-12-26 21:37:31,779][105692] Updated weights for policy 0, policy_version 860357 (0.0007) [2023-12-26 21:37:31,845][105692] Updated weights for policy 0, policy_version 860367 (0.0008) [2023-12-26 21:37:31,902][105692] Updated weights for policy 0, policy_version 860377 (0.0009) [2023-12-26 21:37:32,294][105620] Updated weights for policy 1, policy_version 860313 (0.0008) [2023-12-26 21:37:32,358][105620] Updated weights for policy 1, policy_version 860323 (0.0008) [2023-12-26 21:37:32,412][105620] Updated weights for policy 1, policy_version 860333 (0.0009) [2023-12-26 21:37:32,466][105620] Updated weights for policy 1, policy_version 860343 (0.0008) [2023-12-26 21:37:32,587][105692] Updated weights for policy 0, policy_version 860387 (0.0007) [2023-12-26 21:37:32,640][105692] Updated weights for policy 0, policy_version 860397 (0.0005) [2023-12-26 21:37:32,699][105692] Updated weights for policy 0, policy_version 860407 (0.0007) [2023-12-26 21:37:33,202][105620] Updated weights for policy 1, policy_version 860353 (0.0010) [2023-12-26 21:37:33,260][105620] Updated weights for policy 1, policy_version 860363 (0.0010) [2023-12-26 21:37:33,304][105620] Updated weights for policy 1, policy_version 860373 (0.0010) [2023-12-26 21:37:33,392][105692] Updated weights for policy 0, policy_version 860417 (0.0010) [2023-12-26 21:37:33,444][105692] Updated weights for policy 0, policy_version 860427 (0.0006) [2023-12-26 21:37:33,500][105692] Updated weights for policy 0, policy_version 860437 (0.0010) [2023-12-26 21:37:33,559][105692] Updated weights for policy 0, policy_version 860447 (0.0010) [2023-12-26 21:37:34,024][105620] Updated weights for policy 1, policy_version 860383 (0.0010) [2023-12-26 21:37:34,079][105620] Updated weights for policy 1, policy_version 860393 (0.0010) [2023-12-26 21:37:34,146][105620] Updated weights for policy 1, policy_version 860403 (0.0011) [2023-12-26 21:37:34,308][105692] Updated weights for policy 0, policy_version 860457 (0.0010) [2023-12-26 21:37:34,375][105692] Updated weights for policy 0, policy_version 860467 (0.0011) [2023-12-26 21:37:34,441][105692] Updated weights for policy 0, policy_version 860477 (0.0010) [2023-12-26 21:37:34,910][105620] Updated weights for policy 1, policy_version 860413 (0.0011) [2023-12-26 21:37:34,969][105620] Updated weights for policy 1, policy_version 860423 (0.0011) [2023-12-26 21:37:35,032][105620] Updated weights for policy 1, policy_version 860433 (0.0010) [2023-12-26 21:37:35,196][105692] Updated weights for policy 0, policy_version 860487 (0.0010) [2023-12-26 21:37:35,260][105692] Updated weights for policy 0, policy_version 860497 (0.0010) [2023-12-26 21:37:35,308][105692] Updated weights for policy 0, policy_version 860507 (0.0010) [2023-12-26 21:37:35,623][105620] Updated weights for policy 1, policy_version 860443 (0.0006) [2023-12-26 21:37:35,683][105620] Updated weights for policy 1, policy_version 860453 (0.0009) [2023-12-26 21:37:35,741][105620] Updated weights for policy 1, policy_version 860463 (0.0010) [2023-12-26 21:37:35,942][105692] Updated weights for policy 0, policy_version 860517 (0.0008) [2023-12-26 21:37:35,991][105692] Updated weights for policy 0, policy_version 860527 (0.0005) [2023-12-26 21:37:36,042][105692] Updated weights for policy 0, policy_version 860537 (0.0005) [2023-12-26 21:37:36,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 440631296. Throughput: 0: 9899.8, 1: 9845.9. Samples: 440622664. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-12-26 21:37:36,062][104569] Avg episode reward: [(0, '8805.898'), (1, '8302.432')] [2023-12-26 21:37:36,448][105620] Updated weights for policy 1, policy_version 860473 (0.0010) [2023-12-26 21:37:36,518][105620] Updated weights for policy 1, policy_version 860483 (0.0006) [2023-12-26 21:37:36,575][105620] Updated weights for policy 1, policy_version 860493 (0.0007) [2023-12-26 21:37:36,635][105692] Updated weights for policy 0, policy_version 860547 (0.0007) [2023-12-26 21:37:36,639][105620] Updated weights for policy 1, policy_version 860503 (0.0006) [2023-12-26 21:37:36,694][105692] Updated weights for policy 0, policy_version 860557 (0.0009) [2023-12-26 21:37:36,761][105692] Updated weights for policy 0, policy_version 860567 (0.0005) [2023-12-26 21:37:37,275][105620] Updated weights for policy 1, policy_version 860513 (0.0008) [2023-12-26 21:37:37,336][105620] Updated weights for policy 1, policy_version 860523 (0.0007) [2023-12-26 21:37:37,391][105620] Updated weights for policy 1, policy_version 860533 (0.0010) [2023-12-26 21:37:37,438][105692] Updated weights for policy 0, policy_version 860577 (0.0006) [2023-12-26 21:37:37,498][105692] Updated weights for policy 0, policy_version 860587 (0.0010) [2023-12-26 21:37:37,560][105692] Updated weights for policy 0, policy_version 860597 (0.0009) [2023-12-26 21:37:37,616][105692] Updated weights for policy 0, policy_version 860607 (0.0011) [2023-12-26 21:37:38,035][105620] Updated weights for policy 1, policy_version 860543 (0.0010) [2023-12-26 21:37:38,100][105620] Updated weights for policy 1, policy_version 860553 (0.0008) [2023-12-26 21:37:38,158][105620] Updated weights for policy 1, policy_version 860563 (0.0009) [2023-12-26 21:37:38,183][105586] KL-divergence is very high: 105.4052 [2023-12-26 21:37:38,289][105692] Updated weights for policy 0, policy_version 860617 (0.0008) [2023-12-26 21:37:38,352][105692] Updated weights for policy 0, policy_version 860627 (0.0009) [2023-12-26 21:37:38,411][105692] Updated weights for policy 0, policy_version 860637 (0.0008) [2023-12-26 21:37:38,955][105620] Updated weights for policy 1, policy_version 860573 (0.0009) [2023-12-26 21:37:39,018][105620] Updated weights for policy 1, policy_version 860583 (0.0011) [2023-12-26 21:37:39,077][105620] Updated weights for policy 1, policy_version 860593 (0.0010) [2023-12-26 21:37:39,088][105692] Updated weights for policy 0, policy_version 860647 (0.0006) [2023-12-26 21:37:39,148][105692] Updated weights for policy 0, policy_version 860657 (0.0008) [2023-12-26 21:37:39,213][105692] Updated weights for policy 0, policy_version 860667 (0.0009) [2023-12-26 21:37:39,785][105620] Updated weights for policy 1, policy_version 860603 (0.0010) [2023-12-26 21:37:39,851][105620] Updated weights for policy 1, policy_version 860613 (0.0008) [2023-12-26 21:37:39,920][105620] Updated weights for policy 1, policy_version 860623 (0.0009) [2023-12-26 21:37:39,978][105692] Updated weights for policy 0, policy_version 860677 (0.0009) [2023-12-26 21:37:40,040][105692] Updated weights for policy 0, policy_version 860687 (0.0008) [2023-12-26 21:37:40,095][105692] Updated weights for policy 0, policy_version 860697 (0.0009) [2023-12-26 21:37:40,603][105620] Updated weights for policy 1, policy_version 860633 (0.0007) [2023-12-26 21:37:40,660][105620] Updated weights for policy 1, policy_version 860643 (0.0005) [2023-12-26 21:37:40,718][105620] Updated weights for policy 1, policy_version 860653 (0.0005) [2023-12-26 21:37:40,773][105620] Updated weights for policy 1, policy_version 860663 (0.0005) [2023-12-26 21:37:40,938][105692] Updated weights for policy 0, policy_version 860707 (0.0008) [2023-12-26 21:37:40,984][105692] Updated weights for policy 0, policy_version 860717 (0.0006) [2023-12-26 21:37:41,042][105692] Updated weights for policy 0, policy_version 860727 (0.0009) [2023-12-26 21:37:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 440729600. Throughput: 0: 9927.8, 1: 9865.8. Samples: 440743048. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-12-26 21:37:41,062][104569] Avg episode reward: [(0, '9165.902'), (1, '6732.514')] [2023-12-26 21:37:41,448][105620] Updated weights for policy 1, policy_version 860673 (0.0008) [2023-12-26 21:37:41,505][105620] Updated weights for policy 1, policy_version 860683 (0.0008) [2023-12-26 21:37:41,561][105620] Updated weights for policy 1, policy_version 860693 (0.0008) [2023-12-26 21:37:41,858][105692] Updated weights for policy 0, policy_version 860737 (0.0011) [2023-12-26 21:37:41,917][105692] Updated weights for policy 0, policy_version 860747 (0.0010) [2023-12-26 21:37:41,979][105692] Updated weights for policy 0, policy_version 860757 (0.0010) [2023-12-26 21:37:42,035][105692] Updated weights for policy 0, policy_version 860767 (0.0010) [2023-12-26 21:37:42,376][105620] Updated weights for policy 1, policy_version 860703 (0.0008) [2023-12-26 21:37:42,437][105620] Updated weights for policy 1, policy_version 860713 (0.0008) [2023-12-26 21:37:42,494][105620] Updated weights for policy 1, policy_version 860723 (0.0008) [2023-12-26 21:37:42,817][105692] Updated weights for policy 0, policy_version 860777 (0.0011) [2023-12-26 21:37:42,881][105692] Updated weights for policy 0, policy_version 860787 (0.0010) [2023-12-26 21:37:42,939][105692] Updated weights for policy 0, policy_version 860797 (0.0010) [2023-12-26 21:37:43,280][105620] Updated weights for policy 1, policy_version 860733 (0.0009) [2023-12-26 21:37:43,343][105620] Updated weights for policy 1, policy_version 860743 (0.0010) [2023-12-26 21:37:43,404][105620] Updated weights for policy 1, policy_version 860753 (0.0009) [2023-12-26 21:37:43,538][105692] Updated weights for policy 0, policy_version 860807 (0.0007) [2023-12-26 21:37:43,588][105692] Updated weights for policy 0, policy_version 860817 (0.0005) [2023-12-26 21:37:43,642][105692] Updated weights for policy 0, policy_version 860827 (0.0005) [2023-12-26 21:37:44,169][105692] Updated weights for policy 0, policy_version 860837 (0.0008) [2023-12-26 21:37:44,225][105692] Updated weights for policy 0, policy_version 860847 (0.0010) [2023-12-26 21:37:44,235][105620] Updated weights for policy 1, policy_version 860763 (0.0010) [2023-12-26 21:37:44,284][105692] Updated weights for policy 0, policy_version 860857 (0.0011) [2023-12-26 21:37:44,292][105620] Updated weights for policy 1, policy_version 860773 (0.0011) [2023-12-26 21:37:44,353][105620] Updated weights for policy 1, policy_version 860783 (0.0007) [2023-12-26 21:37:44,984][105620] Updated weights for policy 1, policy_version 860793 (0.0005) [2023-12-26 21:37:45,012][105692] Updated weights for policy 0, policy_version 860867 (0.0009) [2023-12-26 21:37:45,050][105620] Updated weights for policy 1, policy_version 860803 (0.0006) [2023-12-26 21:37:45,072][105692] Updated weights for policy 0, policy_version 860877 (0.0008) [2023-12-26 21:37:45,109][105620] Updated weights for policy 1, policy_version 860813 (0.0007) [2023-12-26 21:37:45,131][105692] Updated weights for policy 0, policy_version 860887 (0.0008) [2023-12-26 21:37:45,173][105620] Updated weights for policy 1, policy_version 860823 (0.0008) [2023-12-26 21:37:45,795][105620] Updated weights for policy 1, policy_version 860833 (0.0008) [2023-12-26 21:37:45,852][105620] Updated weights for policy 1, policy_version 860843 (0.0009) [2023-12-26 21:37:45,904][105620] Updated weights for policy 1, policy_version 860853 (0.0009) [2023-12-26 21:37:45,905][105692] Updated weights for policy 0, policy_version 860897 (0.0006) [2023-12-26 21:37:45,969][105692] Updated weights for policy 0, policy_version 860907 (0.0005) [2023-12-26 21:37:46,027][105692] Updated weights for policy 0, policy_version 860917 (0.0005) [2023-12-26 21:37:46,062][104569] Fps is (10 sec: 19659.5, 60 sec: 19660.7, 300 sec: 19605.2). Total num frames: 440827904. Throughput: 0: 9884.7, 1: 9690.9. Samples: 440797368. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-12-26 21:37:46,063][104569] Avg episode reward: [(0, '9162.938'), (1, '7776.760')] [2023-12-26 21:37:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000860856_220405760.pth... [2023-12-26 21:37:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000859704_220110848.pth [2023-12-26 21:37:46,094][105692] Updated weights for policy 0, policy_version 860927 (0.0005) [2023-12-26 21:37:46,096][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000860928_220430336.pth... [2023-12-26 21:37:46,101][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000859744_220127232.pth [2023-12-26 21:37:46,659][105620] Updated weights for policy 1, policy_version 860863 (0.0007) [2023-12-26 21:37:46,691][105692] Updated weights for policy 0, policy_version 860937 (0.0008) [2023-12-26 21:37:46,709][105620] Updated weights for policy 1, policy_version 860873 (0.0009) [2023-12-26 21:37:46,747][105692] Updated weights for policy 0, policy_version 860947 (0.0006) [2023-12-26 21:37:46,761][105620] Updated weights for policy 1, policy_version 860883 (0.0007) [2023-12-26 21:37:46,803][105692] Updated weights for policy 0, policy_version 860957 (0.0006) [2023-12-26 21:37:47,462][105692] Updated weights for policy 0, policy_version 860967 (0.0005) [2023-12-26 21:37:47,516][105692] Updated weights for policy 0, policy_version 860977 (0.0005) [2023-12-26 21:37:47,542][105620] Updated weights for policy 1, policy_version 860893 (0.0006) [2023-12-26 21:37:47,571][105692] Updated weights for policy 0, policy_version 860987 (0.0005) [2023-12-26 21:37:47,594][105620] Updated weights for policy 1, policy_version 860903 (0.0009) [2023-12-26 21:37:47,646][105620] Updated weights for policy 1, policy_version 860913 (0.0010) [2023-12-26 21:37:48,174][105692] Updated weights for policy 0, policy_version 860997 (0.0005) [2023-12-26 21:37:48,222][105692] Updated weights for policy 0, policy_version 861007 (0.0005) [2023-12-26 21:37:48,284][105692] Updated weights for policy 0, policy_version 861017 (0.0005) [2023-12-26 21:37:48,402][105620] Updated weights for policy 1, policy_version 860923 (0.0009) [2023-12-26 21:37:48,460][105620] Updated weights for policy 1, policy_version 860933 (0.0009) [2023-12-26 21:37:48,521][105620] Updated weights for policy 1, policy_version 860943 (0.0008) [2023-12-26 21:37:48,987][105692] Updated weights for policy 0, policy_version 861027 (0.0009) [2023-12-26 21:37:49,035][105692] Updated weights for policy 0, policy_version 861037 (0.0010) [2023-12-26 21:37:49,090][105692] Updated weights for policy 0, policy_version 861047 (0.0010) [2023-12-26 21:37:49,281][105620] Updated weights for policy 1, policy_version 860953 (0.0008) [2023-12-26 21:37:49,343][105620] Updated weights for policy 1, policy_version 860963 (0.0008) [2023-12-26 21:37:49,402][105620] Updated weights for policy 1, policy_version 860973 (0.0008) [2023-12-26 21:37:49,454][105620] Updated weights for policy 1, policy_version 860983 (0.0008) [2023-12-26 21:37:49,862][105692] Updated weights for policy 0, policy_version 861057 (0.0010) [2023-12-26 21:37:49,918][105692] Updated weights for policy 0, policy_version 861067 (0.0011) [2023-12-26 21:37:49,969][105692] Updated weights for policy 0, policy_version 861077 (0.0010) [2023-12-26 21:37:50,018][105692] Updated weights for policy 0, policy_version 861087 (0.0010) [2023-12-26 21:37:50,236][105620] Updated weights for policy 1, policy_version 860993 (0.0008) [2023-12-26 21:37:50,292][105620] Updated weights for policy 1, policy_version 861003 (0.0008) [2023-12-26 21:37:50,339][105620] Updated weights for policy 1, policy_version 861013 (0.0008) [2023-12-26 21:37:50,798][105692] Updated weights for policy 0, policy_version 861097 (0.0009) [2023-12-26 21:37:50,847][105692] Updated weights for policy 0, policy_version 861107 (0.0011) [2023-12-26 21:37:50,906][105692] Updated weights for policy 0, policy_version 861117 (0.0010) [2023-12-26 21:37:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 440926208. Throughput: 0: 9924.5, 1: 9713.9. Samples: 440916904. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-12-26 21:37:51,062][104569] Avg episode reward: [(0, '9253.424'), (1, '8809.829')] [2023-12-26 21:37:51,135][105620] Updated weights for policy 1, policy_version 861023 (0.0009) [2023-12-26 21:37:51,193][105620] Updated weights for policy 1, policy_version 861033 (0.0009) [2023-12-26 21:37:51,259][105620] Updated weights for policy 1, policy_version 861043 (0.0009) [2023-12-26 21:37:51,701][105692] Updated weights for policy 0, policy_version 861127 (0.0008) [2023-12-26 21:37:51,771][105692] Updated weights for policy 0, policy_version 861137 (0.0010) [2023-12-26 21:37:51,840][105692] Updated weights for policy 0, policy_version 861147 (0.0009) [2023-12-26 21:37:51,986][105620] Updated weights for policy 1, policy_version 861053 (0.0009) [2023-12-26 21:37:52,033][105620] Updated weights for policy 1, policy_version 861063 (0.0009) [2023-12-26 21:37:52,082][105620] Updated weights for policy 1, policy_version 861073 (0.0006) [2023-12-26 21:37:52,597][105692] Updated weights for policy 0, policy_version 861157 (0.0009) [2023-12-26 21:37:52,646][105692] Updated weights for policy 0, policy_version 861167 (0.0009) [2023-12-26 21:37:52,699][105692] Updated weights for policy 0, policy_version 861177 (0.0008) [2023-12-26 21:37:52,831][105620] Updated weights for policy 1, policy_version 861083 (0.0006) [2023-12-26 21:37:52,886][105620] Updated weights for policy 1, policy_version 861093 (0.0009) [2023-12-26 21:37:52,940][105620] Updated weights for policy 1, policy_version 861103 (0.0008) [2023-12-26 21:37:53,502][105692] Updated weights for policy 0, policy_version 861187 (0.0009) [2023-12-26 21:37:53,565][105692] Updated weights for policy 0, policy_version 861197 (0.0009) [2023-12-26 21:37:53,617][105692] Updated weights for policy 0, policy_version 861207 (0.0009) [2023-12-26 21:37:53,697][105620] Updated weights for policy 1, policy_version 861113 (0.0009) [2023-12-26 21:37:53,757][105620] Updated weights for policy 1, policy_version 861123 (0.0009) [2023-12-26 21:37:53,821][105620] Updated weights for policy 1, policy_version 861133 (0.0009) [2023-12-26 21:37:53,868][105620] Updated weights for policy 1, policy_version 861143 (0.0009) [2023-12-26 21:37:54,418][105692] Updated weights for policy 0, policy_version 861217 (0.0009) [2023-12-26 21:37:54,474][105692] Updated weights for policy 0, policy_version 861227 (0.0005) [2023-12-26 21:37:54,528][105692] Updated weights for policy 0, policy_version 861237 (0.0009) [2023-12-26 21:37:54,578][105692] Updated weights for policy 0, policy_version 861248 (0.0009) [2023-12-26 21:37:54,593][105620] Updated weights for policy 1, policy_version 861153 (0.0006) [2023-12-26 21:37:54,652][105620] Updated weights for policy 1, policy_version 861163 (0.0005) [2023-12-26 21:37:54,715][105620] Updated weights for policy 1, policy_version 861173 (0.0005) [2023-12-26 21:37:55,290][105620] Updated weights for policy 1, policy_version 861183 (0.0009) [2023-12-26 21:37:55,356][105620] Updated weights for policy 1, policy_version 861193 (0.0008) [2023-12-26 21:37:55,417][105620] Updated weights for policy 1, policy_version 861203 (0.0006) [2023-12-26 21:37:55,434][105692] Updated weights for policy 0, policy_version 861258 (0.0008) [2023-12-26 21:37:55,492][105692] Updated weights for policy 0, policy_version 861268 (0.0008) [2023-12-26 21:37:55,548][105692] Updated weights for policy 0, policy_version 861278 (0.0008) [2023-12-26 21:37:56,062][104569] Fps is (10 sec: 18842.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 441016320. Throughput: 0: 9863.6, 1: 9676.0. Samples: 441028800. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-12-26 21:37:56,063][104569] Avg episode reward: [(0, '9073.024'), (1, '8984.563')] [2023-12-26 21:37:56,108][105620] Updated weights for policy 1, policy_version 861213 (0.0007) [2023-12-26 21:37:56,154][105620] Updated weights for policy 1, policy_version 861223 (0.0005) [2023-12-26 21:37:56,207][105620] Updated weights for policy 1, policy_version 861233 (0.0005) [2023-12-26 21:37:56,365][105692] Updated weights for policy 0, policy_version 861288 (0.0009) [2023-12-26 21:37:56,437][105692] Updated weights for policy 0, policy_version 861298 (0.0010) [2023-12-26 21:37:56,505][105692] Updated weights for policy 0, policy_version 861308 (0.0010) [2023-12-26 21:37:56,731][105620] Updated weights for policy 1, policy_version 861243 (0.0007) [2023-12-26 21:37:56,785][105620] Updated weights for policy 1, policy_version 861253 (0.0010) [2023-12-26 21:37:56,839][105620] Updated weights for policy 1, policy_version 861263 (0.0010) [2023-12-26 21:37:57,347][105692] Updated weights for policy 0, policy_version 861318 (0.0009) [2023-12-26 21:37:57,406][105692] Updated weights for policy 0, policy_version 861328 (0.0008) [2023-12-26 21:37:57,461][105692] Updated weights for policy 0, policy_version 861338 (0.0008) [2023-12-26 21:37:57,470][105620] Updated weights for policy 1, policy_version 861273 (0.0009) [2023-12-26 21:37:57,527][105620] Updated weights for policy 1, policy_version 861283 (0.0010) [2023-12-26 21:37:57,594][105620] Updated weights for policy 1, policy_version 861293 (0.0010) [2023-12-26 21:37:57,651][105620] Updated weights for policy 1, policy_version 861303 (0.0010) [2023-12-26 21:37:58,259][105620] Updated weights for policy 1, policy_version 861313 (0.0008) [2023-12-26 21:37:58,286][105692] Updated weights for policy 0, policy_version 861348 (0.0009) [2023-12-26 21:37:58,317][105620] Updated weights for policy 1, policy_version 861323 (0.0008) [2023-12-26 21:37:58,352][105692] Updated weights for policy 0, policy_version 861358 (0.0010) [2023-12-26 21:37:58,380][105620] Updated weights for policy 1, policy_version 861333 (0.0007) [2023-12-26 21:37:58,418][105692] Updated weights for policy 0, policy_version 861368 (0.0008) [2023-12-26 21:37:59,102][105620] Updated weights for policy 1, policy_version 861343 (0.0010) [2023-12-26 21:37:59,163][105620] Updated weights for policy 1, policy_version 861353 (0.0007) [2023-12-26 21:37:59,225][105620] Updated weights for policy 1, policy_version 861363 (0.0008) [2023-12-26 21:37:59,347][105692] Updated weights for policy 0, policy_version 861378 (0.0009) [2023-12-26 21:37:59,403][105692] Updated weights for policy 0, policy_version 861388 (0.0009) [2023-12-26 21:37:59,449][105692] Updated weights for policy 0, policy_version 861398 (0.0008) [2023-12-26 21:37:59,497][105692] Updated weights for policy 0, policy_version 861408 (0.0009) [2023-12-26 21:38:00,020][105620] Updated weights for policy 1, policy_version 861373 (0.0009) [2023-12-26 21:38:00,084][105620] Updated weights for policy 1, policy_version 861383 (0.0009) [2023-12-26 21:38:00,139][105620] Updated weights for policy 1, policy_version 861393 (0.0009) [2023-12-26 21:38:00,264][105692] Updated weights for policy 0, policy_version 861418 (0.0005) [2023-12-26 21:38:00,330][105692] Updated weights for policy 0, policy_version 861428 (0.0005) [2023-12-26 21:38:00,390][105692] Updated weights for policy 0, policy_version 861438 (0.0009) [2023-12-26 21:38:00,828][105620] Updated weights for policy 1, policy_version 861403 (0.0008) [2023-12-26 21:38:00,887][105620] Updated weights for policy 1, policy_version 861413 (0.0005) [2023-12-26 21:38:00,943][105620] Updated weights for policy 1, policy_version 861423 (0.0005) [2023-12-26 21:38:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 441114624. Throughput: 0: 9739.0, 1: 9772.2. Samples: 441086528. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-12-26 21:38:01,062][104569] Avg episode reward: [(0, '8999.066'), (1, '9168.168')] [2023-12-26 21:38:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000861432_220553216.pth... [2023-12-26 21:38:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000860280_220258304.pth [2023-12-26 21:38:01,086][105692] Updated weights for policy 0, policy_version 861448 (0.0007) [2023-12-26 21:38:01,157][105692] Updated weights for policy 0, policy_version 861458 (0.0009) [2023-12-26 21:38:01,211][105692] Updated weights for policy 0, policy_version 861468 (0.0010) [2023-12-26 21:38:01,226][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000861472_220569600.pth... [2023-12-26 21:38:01,231][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000860352_220282880.pth [2023-12-26 21:38:01,521][105620] Updated weights for policy 1, policy_version 861433 (0.0005) [2023-12-26 21:38:01,581][105620] Updated weights for policy 1, policy_version 861443 (0.0007) [2023-12-26 21:38:01,641][105620] Updated weights for policy 1, policy_version 861453 (0.0009) [2023-12-26 21:38:01,704][105620] Updated weights for policy 1, policy_version 861463 (0.0009) [2023-12-26 21:38:02,045][105692] Updated weights for policy 0, policy_version 861479 (0.0009) [2023-12-26 21:38:02,093][105692] Updated weights for policy 0, policy_version 861489 (0.0009) [2023-12-26 21:38:02,147][105692] Updated weights for policy 0, policy_version 861499 (0.0008) [2023-12-26 21:38:02,341][105620] Updated weights for policy 1, policy_version 861473 (0.0008) [2023-12-26 21:38:02,407][105620] Updated weights for policy 1, policy_version 861483 (0.0006) [2023-12-26 21:38:02,463][105620] Updated weights for policy 1, policy_version 861493 (0.0006) [2023-12-26 21:38:02,975][105692] Updated weights for policy 0, policy_version 861509 (0.0010) [2023-12-26 21:38:03,028][105692] Updated weights for policy 0, policy_version 861521 (0.0010) [2023-12-26 21:38:03,087][105692] Updated weights for policy 0, policy_version 861531 (0.0009) [2023-12-26 21:38:03,089][105620] Updated weights for policy 1, policy_version 861503 (0.0005) [2023-12-26 21:38:03,134][105620] Updated weights for policy 1, policy_version 861513 (0.0007) [2023-12-26 21:38:03,180][105620] Updated weights for policy 1, policy_version 861523 (0.0009) [2023-12-26 21:38:03,740][105620] Updated weights for policy 1, policy_version 861533 (0.0009) [2023-12-26 21:38:03,787][105620] Updated weights for policy 1, policy_version 861543 (0.0010) [2023-12-26 21:38:03,831][105620] Updated weights for policy 1, policy_version 861553 (0.0010) [2023-12-26 21:38:03,950][105692] Updated weights for policy 0, policy_version 861541 (0.0008) [2023-12-26 21:38:04,000][105692] Updated weights for policy 0, policy_version 861551 (0.0008) [2023-12-26 21:38:04,049][105692] Updated weights for policy 0, policy_version 861561 (0.0008) [2023-12-26 21:38:04,518][105620] Updated weights for policy 1, policy_version 861563 (0.0009) [2023-12-26 21:38:04,567][105620] Updated weights for policy 1, policy_version 861573 (0.0005) [2023-12-26 21:38:04,625][105620] Updated weights for policy 1, policy_version 861583 (0.0005) [2023-12-26 21:38:04,826][105692] Updated weights for policy 0, policy_version 861571 (0.0008) [2023-12-26 21:38:04,882][105692] Updated weights for policy 0, policy_version 861581 (0.0005) [2023-12-26 21:38:04,943][105692] Updated weights for policy 0, policy_version 861591 (0.0006) [2023-12-26 21:38:05,229][105620] Updated weights for policy 1, policy_version 861593 (0.0005) [2023-12-26 21:38:05,295][105620] Updated weights for policy 1, policy_version 861603 (0.0009) [2023-12-26 21:38:05,344][105620] Updated weights for policy 1, policy_version 861613 (0.0005) [2023-12-26 21:38:05,403][105620] Updated weights for policy 1, policy_version 861623 (0.0005) [2023-12-26 21:38:05,602][105692] Updated weights for policy 0, policy_version 861601 (0.0010) [2023-12-26 21:38:05,657][105692] Updated weights for policy 0, policy_version 861611 (0.0005) [2023-12-26 21:38:05,703][105692] Updated weights for policy 0, policy_version 861621 (0.0005) [2023-12-26 21:38:05,747][105692] Updated weights for policy 0, policy_version 861631 (0.0005) [2023-12-26 21:38:06,002][105620] Updated weights for policy 1, policy_version 861633 (0.0008) [2023-12-26 21:38:06,058][105620] Updated weights for policy 1, policy_version 861643 (0.0008) [2023-12-26 21:38:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 441212928. Throughput: 0: 9577.5, 1: 9825.7. Samples: 441202848. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-12-26 21:38:06,063][104569] Avg episode reward: [(0, '8827.597'), (1, '9169.379')] [2023-12-26 21:38:06,126][105620] Updated weights for policy 1, policy_version 861653 (0.0007) [2023-12-26 21:38:06,524][105692] Updated weights for policy 0, policy_version 861641 (0.0010) [2023-12-26 21:38:06,586][105692] Updated weights for policy 0, policy_version 861651 (0.0009) [2023-12-26 21:38:06,646][105692] Updated weights for policy 0, policy_version 861661 (0.0011) [2023-12-26 21:38:06,725][105620] Updated weights for policy 1, policy_version 861663 (0.0008) [2023-12-26 21:38:06,783][105620] Updated weights for policy 1, policy_version 861673 (0.0008) [2023-12-26 21:38:06,847][105620] Updated weights for policy 1, policy_version 861683 (0.0006) [2023-12-26 21:38:07,373][105692] Updated weights for policy 0, policy_version 861671 (0.0007) [2023-12-26 21:38:07,423][105692] Updated weights for policy 0, policy_version 861681 (0.0006) [2023-12-26 21:38:07,475][105692] Updated weights for policy 0, policy_version 861691 (0.0010) [2023-12-26 21:38:07,515][105620] Updated weights for policy 1, policy_version 861693 (0.0007) [2023-12-26 21:38:07,591][105620] Updated weights for policy 1, policy_version 861703 (0.0006) [2023-12-26 21:38:07,657][105620] Updated weights for policy 1, policy_version 861713 (0.0006) [2023-12-26 21:38:08,163][105620] Updated weights for policy 1, policy_version 861723 (0.0006) [2023-12-26 21:38:08,206][105692] Updated weights for policy 0, policy_version 861701 (0.0011) [2023-12-26 21:38:08,215][105620] Updated weights for policy 1, policy_version 861733 (0.0006) [2023-12-26 21:38:08,269][105692] Updated weights for policy 0, policy_version 861711 (0.0009) [2023-12-26 21:38:08,280][105620] Updated weights for policy 1, policy_version 861743 (0.0008) [2023-12-26 21:38:08,337][105692] Updated weights for policy 0, policy_version 861721 (0.0008) [2023-12-26 21:38:09,015][105692] Updated weights for policy 0, policy_version 861731 (0.0009) [2023-12-26 21:38:09,065][105692] Updated weights for policy 0, policy_version 861741 (0.0005) [2023-12-26 21:38:09,084][105620] Updated weights for policy 1, policy_version 861753 (0.0008) [2023-12-26 21:38:09,111][105692] Updated weights for policy 0, policy_version 861751 (0.0005) [2023-12-26 21:38:09,137][105620] Updated weights for policy 1, policy_version 861763 (0.0009) [2023-12-26 21:38:09,189][105620] Updated weights for policy 1, policy_version 861773 (0.0009) [2023-12-26 21:38:09,249][105620] Updated weights for policy 1, policy_version 861783 (0.0009) [2023-12-26 21:38:09,805][105692] Updated weights for policy 0, policy_version 861761 (0.0006) [2023-12-26 21:38:09,873][105692] Updated weights for policy 0, policy_version 861771 (0.0010) [2023-12-26 21:38:09,939][105692] Updated weights for policy 0, policy_version 861781 (0.0010) [2023-12-26 21:38:09,993][105692] Updated weights for policy 0, policy_version 861791 (0.0011) [2023-12-26 21:38:10,039][105620] Updated weights for policy 1, policy_version 861793 (0.0006) [2023-12-26 21:38:10,105][105620] Updated weights for policy 1, policy_version 861803 (0.0007) [2023-12-26 21:38:10,165][105620] Updated weights for policy 1, policy_version 861813 (0.0009) [2023-12-26 21:38:10,748][105692] Updated weights for policy 0, policy_version 861801 (0.0011) [2023-12-26 21:38:10,800][105692] Updated weights for policy 0, policy_version 861811 (0.0010) [2023-12-26 21:38:10,816][105620] Updated weights for policy 1, policy_version 861823 (0.0006) [2023-12-26 21:38:10,857][105692] Updated weights for policy 0, policy_version 861821 (0.0011) [2023-12-26 21:38:10,881][105620] Updated weights for policy 1, policy_version 861833 (0.0007) [2023-12-26 21:38:10,950][105620] Updated weights for policy 1, policy_version 861843 (0.0005) [2023-12-26 21:38:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 441319424. Throughput: 0: 9642.7, 1: 9947.1. Samples: 441323724. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-12-26 21:38:11,063][104569] Avg episode reward: [(0, '8901.418'), (1, '8988.753')] [2023-12-26 21:38:11,579][105620] Updated weights for policy 1, policy_version 861853 (0.0005) [2023-12-26 21:38:11,646][105620] Updated weights for policy 1, policy_version 861863 (0.0010) [2023-12-26 21:38:11,647][105692] Updated weights for policy 0, policy_version 861831 (0.0009) [2023-12-26 21:38:11,699][105620] Updated weights for policy 1, policy_version 861873 (0.0007) [2023-12-26 21:38:11,709][105692] Updated weights for policy 0, policy_version 861841 (0.0008) [2023-12-26 21:38:11,774][105692] Updated weights for policy 0, policy_version 861851 (0.0009) [2023-12-26 21:38:12,392][105620] Updated weights for policy 1, policy_version 861883 (0.0009) [2023-12-26 21:38:12,447][105620] Updated weights for policy 1, policy_version 861893 (0.0010) [2023-12-26 21:38:12,506][105620] Updated weights for policy 1, policy_version 861903 (0.0010) [2023-12-26 21:38:12,519][105692] Updated weights for policy 0, policy_version 861861 (0.0008) [2023-12-26 21:38:12,572][105692] Updated weights for policy 0, policy_version 861871 (0.0006) [2023-12-26 21:38:12,635][105692] Updated weights for policy 0, policy_version 861881 (0.0008) [2023-12-26 21:38:13,163][105620] Updated weights for policy 1, policy_version 861913 (0.0010) [2023-12-26 21:38:13,216][105620] Updated weights for policy 1, policy_version 861923 (0.0009) [2023-12-26 21:38:13,264][105620] Updated weights for policy 1, policy_version 861933 (0.0009) [2023-12-26 21:38:13,322][105620] Updated weights for policy 1, policy_version 861943 (0.0009) [2023-12-26 21:38:13,437][105692] Updated weights for policy 0, policy_version 861891 (0.0008) [2023-12-26 21:38:13,490][105692] Updated weights for policy 0, policy_version 861901 (0.0008) [2023-12-26 21:38:13,536][105692] Updated weights for policy 0, policy_version 861911 (0.0008) [2023-12-26 21:38:14,073][105620] Updated weights for policy 1, policy_version 861953 (0.0008) [2023-12-26 21:38:14,124][105620] Updated weights for policy 1, policy_version 861963 (0.0009) [2023-12-26 21:38:14,182][105620] Updated weights for policy 1, policy_version 861973 (0.0009) [2023-12-26 21:38:14,274][105692] Updated weights for policy 0, policy_version 861921 (0.0009) [2023-12-26 21:38:14,330][105692] Updated weights for policy 0, policy_version 861931 (0.0008) [2023-12-26 21:38:14,395][105692] Updated weights for policy 0, policy_version 861941 (0.0006) [2023-12-26 21:38:14,444][105692] Updated weights for policy 0, policy_version 861951 (0.0009) [2023-12-26 21:38:14,980][105620] Updated weights for policy 1, policy_version 861983 (0.0007) [2023-12-26 21:38:15,050][105620] Updated weights for policy 1, policy_version 861993 (0.0006) [2023-12-26 21:38:15,113][105620] Updated weights for policy 1, policy_version 862003 (0.0010) [2023-12-26 21:38:15,177][105692] Updated weights for policy 0, policy_version 861961 (0.0008) [2023-12-26 21:38:15,245][105692] Updated weights for policy 0, policy_version 861971 (0.0007) [2023-12-26 21:38:15,307][105692] Updated weights for policy 0, policy_version 861981 (0.0005) [2023-12-26 21:38:15,866][105692] Updated weights for policy 0, policy_version 861991 (0.0007) [2023-12-26 21:38:15,912][105692] Updated weights for policy 0, policy_version 862001 (0.0008) [2023-12-26 21:38:15,967][105620] Updated weights for policy 1, policy_version 862013 (0.0007) [2023-12-26 21:38:15,973][105692] Updated weights for policy 0, policy_version 862011 (0.0010) [2023-12-26 21:38:15,988][105586] KL-divergence is very high: 141.3429 [2023-12-26 21:38:16,027][105620] Updated weights for policy 1, policy_version 862023 (0.0007) [2023-12-26 21:38:16,034][105586] KL-divergence is very high: 213.6662 [2023-12-26 21:38:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 441409536. Throughput: 0: 9521.5, 1: 9900.1. Samples: 441381008. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-12-26 21:38:16,062][104569] Avg episode reward: [(0, '8900.577'), (1, '8808.434')] [2023-12-26 21:38:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000862016_220708864.pth... [2023-12-26 21:38:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000860928_220430336.pth [2023-12-26 21:38:16,082][105586] KL-divergence is very high: 209.5125 [2023-12-26 21:38:16,086][105620] Updated weights for policy 1, policy_version 862033 (0.0008) [2023-12-26 21:38:16,121][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000862040_220708864.pth... [2023-12-26 21:38:16,124][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000860856_220405760.pth [2023-12-26 21:38:16,692][105692] Updated weights for policy 0, policy_version 862021 (0.0010) [2023-12-26 21:38:16,750][105692] Updated weights for policy 0, policy_version 862031 (0.0010) [2023-12-26 21:38:16,802][105692] Updated weights for policy 0, policy_version 862041 (0.0010) [2023-12-26 21:38:16,835][105620] Updated weights for policy 1, policy_version 862043 (0.0007) [2023-12-26 21:38:16,891][105620] Updated weights for policy 1, policy_version 862053 (0.0010) [2023-12-26 21:38:16,943][105620] Updated weights for policy 1, policy_version 862063 (0.0010) [2023-12-26 21:38:17,537][105692] Updated weights for policy 0, policy_version 862051 (0.0009) [2023-12-26 21:38:17,610][105692] Updated weights for policy 0, policy_version 862061 (0.0005) [2023-12-26 21:38:17,677][105692] Updated weights for policy 0, policy_version 862071 (0.0005) [2023-12-26 21:38:17,704][105620] Updated weights for policy 1, policy_version 862073 (0.0010) [2023-12-26 21:38:17,770][105620] Updated weights for policy 1, policy_version 862083 (0.0011) [2023-12-26 21:38:17,829][105620] Updated weights for policy 1, policy_version 862093 (0.0010) [2023-12-26 21:38:17,879][105620] Updated weights for policy 1, policy_version 862103 (0.0008) [2023-12-26 21:38:18,205][105692] Updated weights for policy 0, policy_version 862081 (0.0006) [2023-12-26 21:38:18,266][105692] Updated weights for policy 0, policy_version 862091 (0.0009) [2023-12-26 21:38:18,322][105692] Updated weights for policy 0, policy_version 862101 (0.0008) [2023-12-26 21:38:18,373][105692] Updated weights for policy 0, policy_version 862111 (0.0009) [2023-12-26 21:38:18,593][105620] Updated weights for policy 1, policy_version 862113 (0.0009) [2023-12-26 21:38:18,656][105620] Updated weights for policy 1, policy_version 862123 (0.0010) [2023-12-26 21:38:18,709][105620] Updated weights for policy 1, policy_version 862133 (0.0009) [2023-12-26 21:38:18,993][105692] Updated weights for policy 0, policy_version 862121 (0.0009) [2023-12-26 21:38:19,055][105692] Updated weights for policy 0, policy_version 862131 (0.0009) [2023-12-26 21:38:19,105][105692] Updated weights for policy 0, policy_version 862141 (0.0009) [2023-12-26 21:38:19,553][105620] Updated weights for policy 1, policy_version 862143 (0.0010) [2023-12-26 21:38:19,603][105620] Updated weights for policy 1, policy_version 862153 (0.0007) [2023-12-26 21:38:19,653][105620] Updated weights for policy 1, policy_version 862163 (0.0010) [2023-12-26 21:38:19,825][105692] Updated weights for policy 0, policy_version 862151 (0.0008) [2023-12-26 21:38:19,887][105692] Updated weights for policy 0, policy_version 862161 (0.0008) [2023-12-26 21:38:19,954][105692] Updated weights for policy 0, policy_version 862171 (0.0006) [2023-12-26 21:38:20,369][105620] Updated weights for policy 1, policy_version 862173 (0.0011) [2023-12-26 21:38:20,431][105620] Updated weights for policy 1, policy_version 862183 (0.0011) [2023-12-26 21:38:20,492][105620] Updated weights for policy 1, policy_version 862193 (0.0011) [2023-12-26 21:38:20,620][105692] Updated weights for policy 0, policy_version 862181 (0.0008) [2023-12-26 21:38:20,678][105692] Updated weights for policy 0, policy_version 862191 (0.0009) [2023-12-26 21:38:20,749][105692] Updated weights for policy 0, policy_version 862201 (0.0008) [2023-12-26 21:38:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 441507840. Throughput: 0: 9550.0, 1: 9899.6. Samples: 441497896. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-12-26 21:38:21,062][104569] Avg episode reward: [(0, '8899.605'), (1, '8899.716')] [2023-12-26 21:38:21,253][105620] Updated weights for policy 1, policy_version 862203 (0.0011) [2023-12-26 21:38:21,317][105620] Updated weights for policy 1, policy_version 862213 (0.0011) [2023-12-26 21:38:21,390][105620] Updated weights for policy 1, policy_version 862223 (0.0011) [2023-12-26 21:38:21,551][105692] Updated weights for policy 0, policy_version 862211 (0.0008) [2023-12-26 21:38:21,616][105692] Updated weights for policy 0, policy_version 862221 (0.0008) [2023-12-26 21:38:21,674][105692] Updated weights for policy 0, policy_version 862231 (0.0008) [2023-12-26 21:38:22,090][105620] Updated weights for policy 1, policy_version 862233 (0.0011) [2023-12-26 21:38:22,150][105620] Updated weights for policy 1, policy_version 862243 (0.0011) [2023-12-26 21:38:22,217][105620] Updated weights for policy 1, policy_version 862253 (0.0011) [2023-12-26 21:38:22,280][105620] Updated weights for policy 1, policy_version 862263 (0.0010) [2023-12-26 21:38:22,482][105692] Updated weights for policy 0, policy_version 862241 (0.0009) [2023-12-26 21:38:22,532][105692] Updated weights for policy 0, policy_version 862251 (0.0008) [2023-12-26 21:38:22,584][105692] Updated weights for policy 0, policy_version 862261 (0.0007) [2023-12-26 21:38:22,649][105692] Updated weights for policy 0, policy_version 862271 (0.0008) [2023-12-26 21:38:22,934][105620] Updated weights for policy 1, policy_version 862273 (0.0008) [2023-12-26 21:38:22,994][105620] Updated weights for policy 1, policy_version 862283 (0.0006) [2023-12-26 21:38:23,050][105620] Updated weights for policy 1, policy_version 862293 (0.0009) [2023-12-26 21:38:23,476][105692] Updated weights for policy 0, policy_version 862281 (0.0009) [2023-12-26 21:38:23,545][105692] Updated weights for policy 0, policy_version 862291 (0.0005) [2023-12-26 21:38:23,607][105692] Updated weights for policy 0, policy_version 862301 (0.0008) [2023-12-26 21:38:23,761][105620] Updated weights for policy 1, policy_version 862303 (0.0008) [2023-12-26 21:38:23,821][105620] Updated weights for policy 1, policy_version 862313 (0.0008) [2023-12-26 21:38:23,876][105620] Updated weights for policy 1, policy_version 862323 (0.0009) [2023-12-26 21:38:24,339][105692] Updated weights for policy 0, policy_version 862311 (0.0009) [2023-12-26 21:38:24,392][105692] Updated weights for policy 0, policy_version 862321 (0.0010) [2023-12-26 21:38:24,449][105692] Updated weights for policy 0, policy_version 862331 (0.0010) [2023-12-26 21:38:24,582][105620] Updated weights for policy 1, policy_version 862333 (0.0009) [2023-12-26 21:38:24,643][105620] Updated weights for policy 1, policy_version 862343 (0.0009) [2023-12-26 21:38:24,694][105620] Updated weights for policy 1, policy_version 862353 (0.0009) [2023-12-26 21:38:25,219][105692] Updated weights for policy 0, policy_version 862341 (0.0008) [2023-12-26 21:38:25,282][105692] Updated weights for policy 0, policy_version 862351 (0.0009) [2023-12-26 21:38:25,344][105692] Updated weights for policy 0, policy_version 862361 (0.0009) [2023-12-26 21:38:25,430][105620] Updated weights for policy 1, policy_version 862363 (0.0009) [2023-12-26 21:38:25,489][105620] Updated weights for policy 1, policy_version 862373 (0.0009) [2023-12-26 21:38:25,545][105620] Updated weights for policy 1, policy_version 862383 (0.0009) [2023-12-26 21:38:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 441597952. Throughput: 0: 9425.9, 1: 9838.2. Samples: 441609936. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-12-26 21:38:26,063][104569] Avg episode reward: [(0, '8632.442'), (1, '8897.789')] [2023-12-26 21:38:26,155][105692] Updated weights for policy 0, policy_version 862371 (0.0010) [2023-12-26 21:38:26,205][105620] Updated weights for policy 1, policy_version 862393 (0.0008) [2023-12-26 21:38:26,205][105692] Updated weights for policy 0, policy_version 862381 (0.0008) [2023-12-26 21:38:26,258][105620] Updated weights for policy 1, policy_version 862403 (0.0006) [2023-12-26 21:38:26,258][105692] Updated weights for policy 0, policy_version 862391 (0.0009) [2023-12-26 21:38:26,305][105620] Updated weights for policy 1, policy_version 862413 (0.0005) [2023-12-26 21:38:26,363][105620] Updated weights for policy 1, policy_version 862423 (0.0006) [2023-12-26 21:38:26,938][105620] Updated weights for policy 1, policy_version 862433 (0.0006) [2023-12-26 21:38:26,987][105620] Updated weights for policy 1, policy_version 862443 (0.0007) [2023-12-26 21:38:27,033][105620] Updated weights for policy 1, policy_version 862453 (0.0008) [2023-12-26 21:38:27,079][105692] Updated weights for policy 0, policy_version 862401 (0.0009) [2023-12-26 21:38:27,136][105692] Updated weights for policy 0, policy_version 862411 (0.0009) [2023-12-26 21:38:27,196][105692] Updated weights for policy 0, policy_version 862421 (0.0009) [2023-12-26 21:38:27,255][105692] Updated weights for policy 0, policy_version 862431 (0.0010) [2023-12-26 21:38:27,760][105620] Updated weights for policy 1, policy_version 862463 (0.0007) [2023-12-26 21:38:27,817][105620] Updated weights for policy 1, policy_version 862473 (0.0009) [2023-12-26 21:38:27,863][105620] Updated weights for policy 1, policy_version 862483 (0.0008) [2023-12-26 21:38:27,996][105692] Updated weights for policy 0, policy_version 862441 (0.0009) [2023-12-26 21:38:28,046][105585] KL-divergence is very high: 179.0101 [2023-12-26 21:38:28,050][105692] Updated weights for policy 0, policy_version 862451 (0.0009) [2023-12-26 21:38:28,092][105585] KL-divergence is very high: 304.4918 [2023-12-26 21:38:28,110][105692] Updated weights for policy 0, policy_version 862461 (0.0010) [2023-12-26 21:38:28,587][105620] Updated weights for policy 1, policy_version 862493 (0.0010) [2023-12-26 21:38:28,645][105620] Updated weights for policy 1, policy_version 862503 (0.0009) [2023-12-26 21:38:28,704][105620] Updated weights for policy 1, policy_version 862513 (0.0009) [2023-12-26 21:38:28,891][105692] Updated weights for policy 0, policy_version 862471 (0.0008) [2023-12-26 21:38:28,956][105692] Updated weights for policy 0, policy_version 862481 (0.0008) [2023-12-26 21:38:29,024][105692] Updated weights for policy 0, policy_version 862491 (0.0008) [2023-12-26 21:38:29,385][105620] Updated weights for policy 1, policy_version 862523 (0.0010) [2023-12-26 21:38:29,441][105620] Updated weights for policy 1, policy_version 862533 (0.0008) [2023-12-26 21:38:29,503][105620] Updated weights for policy 1, policy_version 862543 (0.0009) [2023-12-26 21:38:29,786][105692] Updated weights for policy 0, policy_version 862501 (0.0009) [2023-12-26 21:38:29,844][105692] Updated weights for policy 0, policy_version 862511 (0.0008) [2023-12-26 21:38:29,907][105692] Updated weights for policy 0, policy_version 862521 (0.0008) [2023-12-26 21:38:30,284][105620] Updated weights for policy 1, policy_version 862553 (0.0010) [2023-12-26 21:38:30,353][105620] Updated weights for policy 1, policy_version 862563 (0.0008) [2023-12-26 21:38:30,413][105620] Updated weights for policy 1, policy_version 862573 (0.0007) [2023-12-26 21:38:30,469][105620] Updated weights for policy 1, policy_version 862583 (0.0009) [2023-12-26 21:38:30,594][105692] Updated weights for policy 0, policy_version 862531 (0.0008) [2023-12-26 21:38:30,645][105692] Updated weights for policy 0, policy_version 862541 (0.0009) [2023-12-26 21:38:30,695][105692] Updated weights for policy 0, policy_version 862551 (0.0008) [2023-12-26 21:38:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 441696256. Throughput: 0: 9387.7, 1: 9954.9. Samples: 441667772. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-12-26 21:38:31,062][104569] Avg episode reward: [(0, '8721.805'), (1, '8898.419')] [2023-12-26 21:38:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000862560_220848128.pth... [2023-12-26 21:38:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000862584_220848128.pth... [2023-12-26 21:38:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000861472_220569600.pth [2023-12-26 21:38:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000861432_220553216.pth [2023-12-26 21:38:31,144][105620] Updated weights for policy 1, policy_version 862593 (0.0009) [2023-12-26 21:38:31,204][105620] Updated weights for policy 1, policy_version 862603 (0.0008) [2023-12-26 21:38:31,265][105620] Updated weights for policy 1, policy_version 862613 (0.0008) [2023-12-26 21:38:31,515][105692] Updated weights for policy 0, policy_version 862562 (0.0010) [2023-12-26 21:38:31,571][105692] Updated weights for policy 0, policy_version 862572 (0.0009) [2023-12-26 21:38:31,632][105692] Updated weights for policy 0, policy_version 862582 (0.0009) [2023-12-26 21:38:31,694][105692] Updated weights for policy 0, policy_version 862592 (0.0008) [2023-12-26 21:38:31,983][105620] Updated weights for policy 1, policy_version 862623 (0.0009) [2023-12-26 21:38:32,038][105620] Updated weights for policy 1, policy_version 862633 (0.0009) [2023-12-26 21:38:32,093][105620] Updated weights for policy 1, policy_version 862643 (0.0009) [2023-12-26 21:38:32,533][105692] Updated weights for policy 0, policy_version 862602 (0.0008) [2023-12-26 21:38:32,591][105692] Updated weights for policy 0, policy_version 862612 (0.0008) [2023-12-26 21:38:32,653][105692] Updated weights for policy 0, policy_version 862622 (0.0005) [2023-12-26 21:38:32,848][105620] Updated weights for policy 1, policy_version 862653 (0.0009) [2023-12-26 21:38:32,903][105620] Updated weights for policy 1, policy_version 862663 (0.0009) [2023-12-26 21:38:32,952][105620] Updated weights for policy 1, policy_version 862673 (0.0009) [2023-12-26 21:38:33,337][105692] Updated weights for policy 0, policy_version 862632 (0.0008) [2023-12-26 21:38:33,387][105692] Updated weights for policy 0, policy_version 862642 (0.0009) [2023-12-26 21:38:33,434][105692] Updated weights for policy 0, policy_version 862652 (0.0009) [2023-12-26 21:38:33,732][105620] Updated weights for policy 1, policy_version 862683 (0.0009) [2023-12-26 21:38:33,777][105620] Updated weights for policy 1, policy_version 862693 (0.0006) [2023-12-26 21:38:33,831][105620] Updated weights for policy 1, policy_version 862703 (0.0005) [2023-12-26 21:38:34,182][105692] Updated weights for policy 0, policy_version 862662 (0.0009) [2023-12-26 21:38:34,238][105692] Updated weights for policy 0, policy_version 862672 (0.0009) [2023-12-26 21:38:34,292][105692] Updated weights for policy 0, policy_version 862682 (0.0009) [2023-12-26 21:38:34,474][105620] Updated weights for policy 1, policy_version 862713 (0.0006) [2023-12-26 21:38:34,531][105620] Updated weights for policy 1, policy_version 862723 (0.0007) [2023-12-26 21:38:34,586][105620] Updated weights for policy 1, policy_version 862733 (0.0006) [2023-12-26 21:38:34,639][105620] Updated weights for policy 1, policy_version 862743 (0.0007) [2023-12-26 21:38:35,111][105692] Updated weights for policy 0, policy_version 862692 (0.0008) [2023-12-26 21:38:35,158][105692] Updated weights for policy 0, policy_version 862702 (0.0008) [2023-12-26 21:38:35,203][105692] Updated weights for policy 0, policy_version 862712 (0.0008) [2023-12-26 21:38:35,229][105620] Updated weights for policy 1, policy_version 862753 (0.0007) [2023-12-26 21:38:35,272][105620] Updated weights for policy 1, policy_version 862763 (0.0005) [2023-12-26 21:38:35,320][105620] Updated weights for policy 1, policy_version 862773 (0.0006) [2023-12-26 21:38:35,897][105692] Updated weights for policy 0, policy_version 862722 (0.0007) [2023-12-26 21:38:35,945][105692] Updated weights for policy 0, policy_version 862732 (0.0010) [2023-12-26 21:38:35,990][105692] Updated weights for policy 0, policy_version 862742 (0.0010) [2023-12-26 21:38:36,042][105692] Updated weights for policy 0, policy_version 862752 (0.0011) [2023-12-26 21:38:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 441794560. Throughput: 0: 9248.0, 1: 9990.2. Samples: 441782624. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-12-26 21:38:36,063][104569] Avg episode reward: [(0, '8905.388'), (1, '8991.310')] [2023-12-26 21:38:36,117][105620] Updated weights for policy 1, policy_version 862783 (0.0009) [2023-12-26 21:38:36,173][105620] Updated weights for policy 1, policy_version 862793 (0.0006) [2023-12-26 21:38:36,226][105620] Updated weights for policy 1, policy_version 862803 (0.0007) [2023-12-26 21:38:36,809][105692] Updated weights for policy 0, policy_version 862762 (0.0005) [2023-12-26 21:38:36,862][105692] Updated weights for policy 0, policy_version 862772 (0.0005) [2023-12-26 21:38:36,921][105692] Updated weights for policy 0, policy_version 862782 (0.0005) [2023-12-26 21:38:37,055][105620] Updated weights for policy 1, policy_version 862813 (0.0008) [2023-12-26 21:38:37,107][105620] Updated weights for policy 1, policy_version 862823 (0.0008) [2023-12-26 21:38:37,167][105620] Updated weights for policy 1, policy_version 862833 (0.0008) [2023-12-26 21:38:37,588][105692] Updated weights for policy 0, policy_version 862792 (0.0010) [2023-12-26 21:38:37,644][105692] Updated weights for policy 0, policy_version 862802 (0.0010) [2023-12-26 21:38:37,707][105692] Updated weights for policy 0, policy_version 862812 (0.0011) [2023-12-26 21:38:37,986][105620] Updated weights for policy 1, policy_version 862843 (0.0008) [2023-12-26 21:38:38,037][105620] Updated weights for policy 1, policy_version 862853 (0.0006) [2023-12-26 21:38:38,106][105620] Updated weights for policy 1, policy_version 862863 (0.0008) [2023-12-26 21:38:38,366][105692] Updated weights for policy 0, policy_version 862823 (0.0010) [2023-12-26 21:38:38,418][105692] Updated weights for policy 0, policy_version 862833 (0.0011) [2023-12-26 21:38:38,479][105692] Updated weights for policy 0, policy_version 862843 (0.0011) [2023-12-26 21:38:38,883][105620] Updated weights for policy 1, policy_version 862873 (0.0008) [2023-12-26 21:38:38,943][105620] Updated weights for policy 1, policy_version 862883 (0.0008) [2023-12-26 21:38:39,003][105620] Updated weights for policy 1, policy_version 862893 (0.0005) [2023-12-26 21:38:39,067][105620] Updated weights for policy 1, policy_version 862903 (0.0005) [2023-12-26 21:38:39,227][105692] Updated weights for policy 0, policy_version 862853 (0.0011) [2023-12-26 21:38:39,286][105692] Updated weights for policy 0, policy_version 862863 (0.0011) [2023-12-26 21:38:39,354][105692] Updated weights for policy 0, policy_version 862873 (0.0010) [2023-12-26 21:38:39,734][105620] Updated weights for policy 1, policy_version 862913 (0.0010) [2023-12-26 21:38:39,790][105620] Updated weights for policy 1, policy_version 862923 (0.0011) [2023-12-26 21:38:39,855][105620] Updated weights for policy 1, policy_version 862933 (0.0010) [2023-12-26 21:38:40,092][105692] Updated weights for policy 0, policy_version 862883 (0.0010) [2023-12-26 21:38:40,142][105692] Updated weights for policy 0, policy_version 862893 (0.0011) [2023-12-26 21:38:40,195][105692] Updated weights for policy 0, policy_version 862903 (0.0010) [2023-12-26 21:38:40,613][105620] Updated weights for policy 1, policy_version 862943 (0.0010) [2023-12-26 21:38:40,671][105620] Updated weights for policy 1, policy_version 862953 (0.0010) [2023-12-26 21:38:40,737][105620] Updated weights for policy 1, policy_version 862963 (0.0006) [2023-12-26 21:38:40,904][105692] Updated weights for policy 0, policy_version 862913 (0.0010) [2023-12-26 21:38:40,964][105692] Updated weights for policy 0, policy_version 862923 (0.0010) [2023-12-26 21:38:41,023][105692] Updated weights for policy 0, policy_version 862933 (0.0010) [2023-12-26 21:38:41,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 441884672. Throughput: 0: 9345.2, 1: 9927.3. Samples: 441896060. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-12-26 21:38:41,063][104569] Avg episode reward: [(0, '8638.635'), (1, '9081.257')] [2023-12-26 21:38:41,091][105692] Updated weights for policy 0, policy_version 862943 (0.0010) [2023-12-26 21:38:41,464][105620] Updated weights for policy 1, policy_version 862973 (0.0009) [2023-12-26 21:38:41,521][105620] Updated weights for policy 1, policy_version 862983 (0.0011) [2023-12-26 21:38:41,581][105620] Updated weights for policy 1, policy_version 862993 (0.0009) [2023-12-26 21:38:41,770][105692] Updated weights for policy 0, policy_version 862953 (0.0010) [2023-12-26 21:38:41,829][105692] Updated weights for policy 0, policy_version 862963 (0.0010) [2023-12-26 21:38:41,888][105692] Updated weights for policy 0, policy_version 862973 (0.0011) [2023-12-26 21:38:42,345][105620] Updated weights for policy 1, policy_version 863003 (0.0009) [2023-12-26 21:38:42,410][105620] Updated weights for policy 1, policy_version 863013 (0.0007) [2023-12-26 21:38:42,468][105620] Updated weights for policy 1, policy_version 863023 (0.0008) [2023-12-26 21:38:42,610][105692] Updated weights for policy 0, policy_version 862983 (0.0010) [2023-12-26 21:38:42,672][105692] Updated weights for policy 0, policy_version 862993 (0.0009) [2023-12-26 21:38:42,731][105692] Updated weights for policy 0, policy_version 863003 (0.0008) [2023-12-26 21:38:43,200][105620] Updated weights for policy 1, policy_version 863033 (0.0009) [2023-12-26 21:38:43,252][105620] Updated weights for policy 1, policy_version 863043 (0.0010) [2023-12-26 21:38:43,315][105620] Updated weights for policy 1, policy_version 863053 (0.0009) [2023-12-26 21:38:43,370][105620] Updated weights for policy 1, policy_version 863063 (0.0010) [2023-12-26 21:38:43,462][105692] Updated weights for policy 0, policy_version 863013 (0.0008) [2023-12-26 21:38:43,521][105692] Updated weights for policy 0, policy_version 863023 (0.0007) [2023-12-26 21:38:43,577][105692] Updated weights for policy 0, policy_version 863033 (0.0008) [2023-12-26 21:38:44,020][105620] Updated weights for policy 1, policy_version 863073 (0.0007) [2023-12-26 21:38:44,088][105620] Updated weights for policy 1, policy_version 863083 (0.0010) [2023-12-26 21:38:44,154][105620] Updated weights for policy 1, policy_version 863093 (0.0010) [2023-12-26 21:38:44,216][105692] Updated weights for policy 0, policy_version 863043 (0.0008) [2023-12-26 21:38:44,261][105692] Updated weights for policy 0, policy_version 863053 (0.0008) [2023-12-26 21:38:44,313][105692] Updated weights for policy 0, policy_version 863063 (0.0008) [2023-12-26 21:38:44,895][105620] Updated weights for policy 1, policy_version 863103 (0.0008) [2023-12-26 21:38:44,954][105620] Updated weights for policy 1, policy_version 863113 (0.0008) [2023-12-26 21:38:45,014][105620] Updated weights for policy 1, policy_version 863123 (0.0011) [2023-12-26 21:38:45,016][105692] Updated weights for policy 0, policy_version 863073 (0.0008) [2023-12-26 21:38:45,079][105692] Updated weights for policy 0, policy_version 863083 (0.0011) [2023-12-26 21:38:45,141][105692] Updated weights for policy 0, policy_version 863093 (0.0011) [2023-12-26 21:38:45,208][105692] Updated weights for policy 0, policy_version 863103 (0.0010) [2023-12-26 21:38:45,768][105620] Updated weights for policy 1, policy_version 863133 (0.0011) [2023-12-26 21:38:45,813][105620] Updated weights for policy 1, policy_version 863143 (0.0010) [2023-12-26 21:38:45,868][105620] Updated weights for policy 1, policy_version 863153 (0.0010) [2023-12-26 21:38:45,893][105692] Updated weights for policy 0, policy_version 863113 (0.0010) [2023-12-26 21:38:45,951][105692] Updated weights for policy 0, policy_version 863123 (0.0010) [2023-12-26 21:38:46,009][105692] Updated weights for policy 0, policy_version 863133 (0.0010) [2023-12-26 21:38:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.9, 300 sec: 19521.9). Total num frames: 441991168. Throughput: 0: 9436.0, 1: 9859.6. Samples: 441954832. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-12-26 21:38:46,062][104569] Avg episode reward: [(0, '8901.217'), (1, '8337.428')] [2023-12-26 21:38:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000863136_220995584.pth... [2023-12-26 21:38:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000863160_220995584.pth... [2023-12-26 21:38:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000862016_220708864.pth [2023-12-26 21:38:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000862040_220708864.pth [2023-12-26 21:38:46,615][105620] Updated weights for policy 1, policy_version 863163 (0.0010) [2023-12-26 21:38:46,677][105620] Updated weights for policy 1, policy_version 863173 (0.0010) [2023-12-26 21:38:46,728][105620] Updated weights for policy 1, policy_version 863183 (0.0009) [2023-12-26 21:38:46,746][105692] Updated weights for policy 0, policy_version 863143 (0.0008) [2023-12-26 21:38:46,792][105692] Updated weights for policy 0, policy_version 863153 (0.0005) [2023-12-26 21:38:46,840][105692] Updated weights for policy 0, policy_version 863163 (0.0005) [2023-12-26 21:38:47,456][105620] Updated weights for policy 1, policy_version 863193 (0.0007) [2023-12-26 21:38:47,505][105692] Updated weights for policy 0, policy_version 863173 (0.0008) [2023-12-26 21:38:47,507][105620] Updated weights for policy 1, policy_version 863203 (0.0010) [2023-12-26 21:38:47,558][105620] Updated weights for policy 1, policy_version 863213 (0.0010) [2023-12-26 21:38:47,570][105692] Updated weights for policy 0, policy_version 863183 (0.0010) [2023-12-26 21:38:47,617][105620] Updated weights for policy 1, policy_version 863223 (0.0010) [2023-12-26 21:38:47,628][105692] Updated weights for policy 0, policy_version 863193 (0.0010) [2023-12-26 21:38:48,276][105620] Updated weights for policy 1, policy_version 863233 (0.0006) [2023-12-26 21:38:48,327][105620] Updated weights for policy 1, policy_version 863243 (0.0006) [2023-12-26 21:38:48,390][105620] Updated weights for policy 1, policy_version 863253 (0.0009) [2023-12-26 21:38:48,390][105692] Updated weights for policy 0, policy_version 863203 (0.0010) [2023-12-26 21:38:48,447][105692] Updated weights for policy 0, policy_version 863213 (0.0011) [2023-12-26 21:38:48,505][105692] Updated weights for policy 0, policy_version 863223 (0.0010) [2023-12-26 21:38:49,074][105620] Updated weights for policy 1, policy_version 863263 (0.0009) [2023-12-26 21:38:49,128][105620] Updated weights for policy 1, policy_version 863273 (0.0010) [2023-12-26 21:38:49,155][105692] Updated weights for policy 0, policy_version 863233 (0.0010) [2023-12-26 21:38:49,186][105620] Updated weights for policy 1, policy_version 863283 (0.0009) [2023-12-26 21:38:49,209][105692] Updated weights for policy 0, policy_version 863243 (0.0006) [2023-12-26 21:38:49,267][105692] Updated weights for policy 0, policy_version 863253 (0.0010) [2023-12-26 21:38:49,312][105692] Updated weights for policy 0, policy_version 863263 (0.0010) [2023-12-26 21:38:49,920][105620] Updated weights for policy 1, policy_version 863293 (0.0009) [2023-12-26 21:38:49,982][105620] Updated weights for policy 1, policy_version 863303 (0.0008) [2023-12-26 21:38:50,042][105620] Updated weights for policy 1, policy_version 863313 (0.0008) [2023-12-26 21:38:50,052][105692] Updated weights for policy 0, policy_version 863273 (0.0008) [2023-12-26 21:38:50,107][105692] Updated weights for policy 0, policy_version 863283 (0.0008) [2023-12-26 21:38:50,159][105692] Updated weights for policy 0, policy_version 863293 (0.0008) [2023-12-26 21:38:50,756][105692] Updated weights for policy 0, policy_version 863303 (0.0009) [2023-12-26 21:38:50,804][105692] Updated weights for policy 0, policy_version 863313 (0.0009) [2023-12-26 21:38:50,850][105692] Updated weights for policy 0, policy_version 863323 (0.0008) [2023-12-26 21:38:50,886][105620] Updated weights for policy 1, policy_version 863323 (0.0009) [2023-12-26 21:38:50,942][105620] Updated weights for policy 1, policy_version 863333 (0.0009) [2023-12-26 21:38:50,990][105620] Updated weights for policy 1, policy_version 863343 (0.0009) [2023-12-26 21:38:51,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 442089472. Throughput: 0: 9610.7, 1: 9733.1. Samples: 442073316. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-12-26 21:38:51,062][104569] Avg episode reward: [(0, '8824.371'), (1, '8084.172')] [2023-12-26 21:38:51,546][105692] Updated weights for policy 0, policy_version 863333 (0.0006) [2023-12-26 21:38:51,602][105692] Updated weights for policy 0, policy_version 863343 (0.0006) [2023-12-26 21:38:51,670][105692] Updated weights for policy 0, policy_version 863353 (0.0009) [2023-12-26 21:38:51,776][105620] Updated weights for policy 1, policy_version 863353 (0.0010) [2023-12-26 21:38:51,836][105620] Updated weights for policy 1, policy_version 863363 (0.0009) [2023-12-26 21:38:51,904][105620] Updated weights for policy 1, policy_version 863373 (0.0008) [2023-12-26 21:38:51,967][105620] Updated weights for policy 1, policy_version 863383 (0.0009) [2023-12-26 21:38:52,449][105692] Updated weights for policy 0, policy_version 863363 (0.0010) [2023-12-26 21:38:52,512][105692] Updated weights for policy 0, policy_version 863373 (0.0009) [2023-12-26 21:38:52,566][105692] Updated weights for policy 0, policy_version 863383 (0.0009) [2023-12-26 21:38:52,626][105620] Updated weights for policy 1, policy_version 863393 (0.0007) [2023-12-26 21:38:52,684][105620] Updated weights for policy 1, policy_version 863403 (0.0009) [2023-12-26 21:38:52,743][105620] Updated weights for policy 1, policy_version 863413 (0.0009) [2023-12-26 21:38:53,270][105692] Updated weights for policy 0, policy_version 863393 (0.0007) [2023-12-26 21:38:53,322][105692] Updated weights for policy 0, policy_version 863403 (0.0010) [2023-12-26 21:38:53,388][105692] Updated weights for policy 0, policy_version 863413 (0.0011) [2023-12-26 21:38:53,443][105692] Updated weights for policy 0, policy_version 863423 (0.0010) [2023-12-26 21:38:53,506][105620] Updated weights for policy 1, policy_version 863423 (0.0010) [2023-12-26 21:38:53,565][105620] Updated weights for policy 1, policy_version 863433 (0.0010) [2023-12-26 21:38:53,630][105620] Updated weights for policy 1, policy_version 863443 (0.0011) [2023-12-26 21:38:54,197][105692] Updated weights for policy 0, policy_version 863433 (0.0010) [2023-12-26 21:38:54,252][105692] Updated weights for policy 0, policy_version 863443 (0.0010) [2023-12-26 21:38:54,303][105692] Updated weights for policy 0, policy_version 863453 (0.0010) [2023-12-26 21:38:54,318][105620] Updated weights for policy 1, policy_version 863453 (0.0008) [2023-12-26 21:38:54,376][105620] Updated weights for policy 1, policy_version 863463 (0.0008) [2023-12-26 21:38:54,435][105620] Updated weights for policy 1, policy_version 863473 (0.0008) [2023-12-26 21:38:55,043][105692] Updated weights for policy 0, policy_version 863463 (0.0010) [2023-12-26 21:38:55,106][105692] Updated weights for policy 0, policy_version 863473 (0.0010) [2023-12-26 21:38:55,168][105692] Updated weights for policy 0, policy_version 863483 (0.0010) [2023-12-26 21:38:55,201][105620] Updated weights for policy 1, policy_version 863483 (0.0007) [2023-12-26 21:38:55,252][105620] Updated weights for policy 1, policy_version 863493 (0.0005) [2023-12-26 21:38:55,307][105620] Updated weights for policy 1, policy_version 863503 (0.0005) [2023-12-26 21:38:55,825][105692] Updated weights for policy 0, policy_version 863493 (0.0008) [2023-12-26 21:38:55,873][105692] Updated weights for policy 0, policy_version 863503 (0.0010) [2023-12-26 21:38:55,928][105692] Updated weights for policy 0, policy_version 863513 (0.0010) [2023-12-26 21:38:55,958][105620] Updated weights for policy 1, policy_version 863513 (0.0006) [2023-12-26 21:38:56,011][105620] Updated weights for policy 1, policy_version 863523 (0.0008) [2023-12-26 21:38:56,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 442179584. Throughput: 0: 9610.4, 1: 9603.8. Samples: 442188372. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-12-26 21:38:56,063][104569] Avg episode reward: [(0, '8569.443'), (1, '8551.813')] [2023-12-26 21:38:56,067][105620] Updated weights for policy 1, policy_version 863533 (0.0008) [2023-12-26 21:38:56,124][105620] Updated weights for policy 1, policy_version 863543 (0.0008) [2023-12-26 21:38:56,625][105692] Updated weights for policy 0, policy_version 863523 (0.0009) [2023-12-26 21:38:56,679][105692] Updated weights for policy 0, policy_version 863533 (0.0005) [2023-12-26 21:38:56,729][105692] Updated weights for policy 0, policy_version 863543 (0.0006) [2023-12-26 21:38:56,980][105620] Updated weights for policy 1, policy_version 863553 (0.0006) [2023-12-26 21:38:57,031][105620] Updated weights for policy 1, policy_version 863563 (0.0005) [2023-12-26 21:38:57,096][105620] Updated weights for policy 1, policy_version 863573 (0.0005) [2023-12-26 21:38:57,269][105692] Updated weights for policy 0, policy_version 863553 (0.0009) [2023-12-26 21:38:57,330][105692] Updated weights for policy 0, policy_version 863563 (0.0008) [2023-12-26 21:38:57,385][105692] Updated weights for policy 0, policy_version 863573 (0.0007) [2023-12-26 21:38:57,443][105692] Updated weights for policy 0, policy_version 863583 (0.0008) [2023-12-26 21:38:57,697][105620] Updated weights for policy 1, policy_version 863583 (0.0009) [2023-12-26 21:38:57,751][105620] Updated weights for policy 1, policy_version 863593 (0.0010) [2023-12-26 21:38:57,802][105620] Updated weights for policy 1, policy_version 863603 (0.0010) [2023-12-26 21:38:58,118][105692] Updated weights for policy 0, policy_version 863593 (0.0010) [2023-12-26 21:38:58,175][105692] Updated weights for policy 0, policy_version 863603 (0.0009) [2023-12-26 21:38:58,237][105692] Updated weights for policy 0, policy_version 863613 (0.0008) [2023-12-26 21:38:58,515][105620] Updated weights for policy 1, policy_version 863613 (0.0011) [2023-12-26 21:38:58,584][105620] Updated weights for policy 1, policy_version 863623 (0.0010) [2023-12-26 21:38:58,650][105620] Updated weights for policy 1, policy_version 863633 (0.0008) [2023-12-26 21:38:59,070][105692] Updated weights for policy 0, policy_version 863623 (0.0008) [2023-12-26 21:38:59,125][105692] Updated weights for policy 0, policy_version 863633 (0.0008) [2023-12-26 21:38:59,183][105692] Updated weights for policy 0, policy_version 863643 (0.0009) [2023-12-26 21:38:59,462][105620] Updated weights for policy 1, policy_version 863643 (0.0007) [2023-12-26 21:38:59,533][105620] Updated weights for policy 1, policy_version 863653 (0.0008) [2023-12-26 21:38:59,597][105620] Updated weights for policy 1, policy_version 863663 (0.0006) [2023-12-26 21:38:59,930][105692] Updated weights for policy 0, policy_version 863653 (0.0009) [2023-12-26 21:38:59,989][105692] Updated weights for policy 0, policy_version 863663 (0.0008) [2023-12-26 21:39:00,054][105692] Updated weights for policy 0, policy_version 863673 (0.0009) [2023-12-26 21:39:00,318][105620] Updated weights for policy 1, policy_version 863673 (0.0006) [2023-12-26 21:39:00,381][105620] Updated weights for policy 1, policy_version 863683 (0.0009) [2023-12-26 21:39:00,436][105620] Updated weights for policy 1, policy_version 863693 (0.0008) [2023-12-26 21:39:00,501][105620] Updated weights for policy 1, policy_version 863703 (0.0009) [2023-12-26 21:39:00,722][105692] Updated weights for policy 0, policy_version 863683 (0.0009) [2023-12-26 21:39:00,773][105692] Updated weights for policy 0, policy_version 863693 (0.0009) [2023-12-26 21:39:00,821][105692] Updated weights for policy 0, policy_version 863703 (0.0009) [2023-12-26 21:39:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 442277888. Throughput: 0: 9699.5, 1: 9558.6. Samples: 442247620. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-12-26 21:39:01,062][104569] Avg episode reward: [(0, '8291.236'), (1, '8503.586')] [2023-12-26 21:39:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000863712_221143040.pth... [2023-12-26 21:39:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000863704_221134848.pth... [2023-12-26 21:39:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000862560_220848128.pth [2023-12-26 21:39:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000862584_220848128.pth [2023-12-26 21:39:01,275][105620] Updated weights for policy 1, policy_version 863713 (0.0009) [2023-12-26 21:39:01,337][105620] Updated weights for policy 1, policy_version 863723 (0.0009) [2023-12-26 21:39:01,403][105620] Updated weights for policy 1, policy_version 863733 (0.0008) [2023-12-26 21:39:01,594][105692] Updated weights for policy 0, policy_version 863713 (0.0009) [2023-12-26 21:39:01,664][105692] Updated weights for policy 0, policy_version 863723 (0.0008) [2023-12-26 21:39:01,732][105692] Updated weights for policy 0, policy_version 863733 (0.0008) [2023-12-26 21:39:01,784][105692] Updated weights for policy 0, policy_version 863743 (0.0008) [2023-12-26 21:39:02,232][105620] Updated weights for policy 1, policy_version 863743 (0.0006) [2023-12-26 21:39:02,296][105620] Updated weights for policy 1, policy_version 863753 (0.0006) [2023-12-26 21:39:02,362][105620] Updated weights for policy 1, policy_version 863763 (0.0007) [2023-12-26 21:39:02,442][105692] Updated weights for policy 0, policy_version 863753 (0.0008) [2023-12-26 21:39:02,501][105692] Updated weights for policy 0, policy_version 863763 (0.0008) [2023-12-26 21:39:02,556][105692] Updated weights for policy 0, policy_version 863773 (0.0008) [2023-12-26 21:39:02,927][105620] Updated weights for policy 1, policy_version 863773 (0.0007) [2023-12-26 21:39:02,984][105620] Updated weights for policy 1, policy_version 863783 (0.0005) [2023-12-26 21:39:03,032][105620] Updated weights for policy 1, policy_version 863793 (0.0005) [2023-12-26 21:39:03,374][105692] Updated weights for policy 0, policy_version 863783 (0.0009) [2023-12-26 21:39:03,422][105692] Updated weights for policy 0, policy_version 863793 (0.0009) [2023-12-26 21:39:03,473][105692] Updated weights for policy 0, policy_version 863803 (0.0009) [2023-12-26 21:39:03,624][105620] Updated weights for policy 1, policy_version 863803 (0.0005) [2023-12-26 21:39:03,677][105620] Updated weights for policy 1, policy_version 863813 (0.0005) [2023-12-26 21:39:03,732][105620] Updated weights for policy 1, policy_version 863823 (0.0005) [2023-12-26 21:39:04,318][105692] Updated weights for policy 0, policy_version 863813 (0.0009) [2023-12-26 21:39:04,378][105692] Updated weights for policy 0, policy_version 863823 (0.0009) [2023-12-26 21:39:04,413][105620] Updated weights for policy 1, policy_version 863833 (0.0006) [2023-12-26 21:39:04,434][105692] Updated weights for policy 0, policy_version 863833 (0.0008) [2023-12-26 21:39:04,473][105620] Updated weights for policy 1, policy_version 863843 (0.0009) [2023-12-26 21:39:04,528][105620] Updated weights for policy 1, policy_version 863853 (0.0009) [2023-12-26 21:39:04,590][105620] Updated weights for policy 1, policy_version 863863 (0.0010) [2023-12-26 21:39:05,048][105692] Updated weights for policy 0, policy_version 863843 (0.0007) [2023-12-26 21:39:05,106][105692] Updated weights for policy 0, policy_version 863853 (0.0007) [2023-12-26 21:39:05,161][105692] Updated weights for policy 0, policy_version 863863 (0.0009) [2023-12-26 21:39:05,416][105620] Updated weights for policy 1, policy_version 863873 (0.0009) [2023-12-26 21:39:05,477][105620] Updated weights for policy 1, policy_version 863883 (0.0009) [2023-12-26 21:39:05,535][105620] Updated weights for policy 1, policy_version 863893 (0.0009) [2023-12-26 21:39:05,830][105692] Updated weights for policy 0, policy_version 863873 (0.0009) [2023-12-26 21:39:05,884][105692] Updated weights for policy 0, policy_version 863883 (0.0005) [2023-12-26 21:39:05,945][105692] Updated weights for policy 0, policy_version 863893 (0.0005) [2023-12-26 21:39:06,006][105692] Updated weights for policy 0, policy_version 863903 (0.0005) [2023-12-26 21:39:06,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 442376192. Throughput: 0: 9533.5, 1: 9641.5. Samples: 442360780. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-12-26 21:39:06,063][104569] Avg episode reward: [(0, '8004.150'), (1, '8988.288')] [2023-12-26 21:39:06,410][105620] Updated weights for policy 1, policy_version 863903 (0.0010) [2023-12-26 21:39:06,464][105620] Updated weights for policy 1, policy_version 863913 (0.0010) [2023-12-26 21:39:06,519][105620] Updated weights for policy 1, policy_version 863923 (0.0009) [2023-12-26 21:39:06,554][105692] Updated weights for policy 0, policy_version 863913 (0.0007) [2023-12-26 21:39:06,611][105692] Updated weights for policy 0, policy_version 863923 (0.0006) [2023-12-26 21:39:06,667][105692] Updated weights for policy 0, policy_version 863933 (0.0006) [2023-12-26 21:39:07,300][105620] Updated weights for policy 1, policy_version 863933 (0.0006) [2023-12-26 21:39:07,337][105692] Updated weights for policy 0, policy_version 863943 (0.0007) [2023-12-26 21:39:07,360][105620] Updated weights for policy 1, policy_version 863943 (0.0005) [2023-12-26 21:39:07,387][105692] Updated weights for policy 0, policy_version 863953 (0.0006) [2023-12-26 21:39:07,422][105620] Updated weights for policy 1, policy_version 863953 (0.0008) [2023-12-26 21:39:07,449][105692] Updated weights for policy 0, policy_version 863963 (0.0006) [2023-12-26 21:39:08,018][105620] Updated weights for policy 1, policy_version 863963 (0.0008) [2023-12-26 21:39:08,072][105620] Updated weights for policy 1, policy_version 863974 (0.0010) [2023-12-26 21:39:08,125][105620] Updated weights for policy 1, policy_version 863984 (0.0009) [2023-12-26 21:39:08,136][105692] Updated weights for policy 0, policy_version 863973 (0.0007) [2023-12-26 21:39:08,193][105692] Updated weights for policy 0, policy_version 863983 (0.0007) [2023-12-26 21:39:08,253][105692] Updated weights for policy 0, policy_version 863993 (0.0009) [2023-12-26 21:39:08,850][105620] Updated weights for policy 1, policy_version 863994 (0.0008) [2023-12-26 21:39:08,910][105620] Updated weights for policy 1, policy_version 864004 (0.0011) [2023-12-26 21:39:08,976][105620] Updated weights for policy 1, policy_version 864014 (0.0007) [2023-12-26 21:39:09,020][105692] Updated weights for policy 0, policy_version 864004 (0.0013) [2023-12-26 21:39:09,033][105620] Updated weights for policy 1, policy_version 864024 (0.0006) [2023-12-26 21:39:09,090][105692] Updated weights for policy 0, policy_version 864014 (0.0010) [2023-12-26 21:39:09,142][105692] Updated weights for policy 0, policy_version 864024 (0.0010) [2023-12-26 21:39:09,767][105620] Updated weights for policy 1, policy_version 864034 (0.0008) [2023-12-26 21:39:09,838][105692] Updated weights for policy 0, policy_version 864034 (0.0011) [2023-12-26 21:39:09,840][105620] Updated weights for policy 1, policy_version 864044 (0.0008) [2023-12-26 21:39:09,896][105692] Updated weights for policy 0, policy_version 864044 (0.0010) [2023-12-26 21:39:09,899][105620] Updated weights for policy 1, policy_version 864054 (0.0008) [2023-12-26 21:39:09,961][105692] Updated weights for policy 0, policy_version 864054 (0.0008) [2023-12-26 21:39:10,015][105692] Updated weights for policy 0, policy_version 864064 (0.0008) [2023-12-26 21:39:10,594][105620] Updated weights for policy 1, policy_version 864064 (0.0006) [2023-12-26 21:39:10,662][105620] Updated weights for policy 1, policy_version 864074 (0.0006) [2023-12-26 21:39:10,723][105620] Updated weights for policy 1, policy_version 864084 (0.0009) [2023-12-26 21:39:10,787][105692] Updated weights for policy 0, policy_version 864074 (0.0011) [2023-12-26 21:39:10,851][105692] Updated weights for policy 0, policy_version 864084 (0.0011) [2023-12-26 21:39:10,920][105692] Updated weights for policy 0, policy_version 864094 (0.0006) [2023-12-26 21:39:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 442474496. Throughput: 0: 9691.0, 1: 9619.1. Samples: 442478892. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-12-26 21:39:11,062][104569] Avg episode reward: [(0, '8626.896'), (1, '9261.673')] [2023-12-26 21:39:11,437][105620] Updated weights for policy 1, policy_version 864094 (0.0008) [2023-12-26 21:39:11,493][105620] Updated weights for policy 1, policy_version 864104 (0.0009) [2023-12-26 21:39:11,550][105620] Updated weights for policy 1, policy_version 864114 (0.0009) [2023-12-26 21:39:11,682][105692] Updated weights for policy 0, policy_version 864104 (0.0009) [2023-12-26 21:39:11,746][105692] Updated weights for policy 0, policy_version 864114 (0.0008) [2023-12-26 21:39:11,805][105692] Updated weights for policy 0, policy_version 864124 (0.0007) [2023-12-26 21:39:12,360][105620] Updated weights for policy 1, policy_version 864124 (0.0010) [2023-12-26 21:39:12,420][105620] Updated weights for policy 1, policy_version 864134 (0.0008) [2023-12-26 21:39:12,476][105692] Updated weights for policy 0, policy_version 864134 (0.0009) [2023-12-26 21:39:12,478][105620] Updated weights for policy 1, policy_version 864144 (0.0007) [2023-12-26 21:39:12,529][105692] Updated weights for policy 0, policy_version 864144 (0.0011) [2023-12-26 21:39:12,588][105692] Updated weights for policy 0, policy_version 864154 (0.0010) [2023-12-26 21:39:13,227][105620] Updated weights for policy 1, policy_version 864154 (0.0006) [2023-12-26 21:39:13,282][105620] Updated weights for policy 1, policy_version 864164 (0.0010) [2023-12-26 21:39:13,293][105692] Updated weights for policy 0, policy_version 864164 (0.0009) [2023-12-26 21:39:13,345][105692] Updated weights for policy 0, policy_version 864174 (0.0005) [2023-12-26 21:39:13,349][105620] Updated weights for policy 1, policy_version 864174 (0.0006) [2023-12-26 21:39:13,402][105692] Updated weights for policy 0, policy_version 864184 (0.0006) [2023-12-26 21:39:13,413][105620] Updated weights for policy 1, policy_version 864184 (0.0008) [2023-12-26 21:39:13,946][105692] Updated weights for policy 0, policy_version 864194 (0.0006) [2023-12-26 21:39:14,001][105692] Updated weights for policy 0, policy_version 864204 (0.0010) [2023-12-26 21:39:14,020][105620] Updated weights for policy 1, policy_version 864194 (0.0010) [2023-12-26 21:39:14,050][105692] Updated weights for policy 0, policy_version 864214 (0.0009) [2023-12-26 21:39:14,079][105620] Updated weights for policy 1, policy_version 864204 (0.0010) [2023-12-26 21:39:14,113][105692] Updated weights for policy 0, policy_version 864224 (0.0007) [2023-12-26 21:39:14,134][105620] Updated weights for policy 1, policy_version 864214 (0.0010) [2023-12-26 21:39:14,844][105692] Updated weights for policy 0, policy_version 864234 (0.0006) [2023-12-26 21:39:14,882][105620] Updated weights for policy 1, policy_version 864224 (0.0010) [2023-12-26 21:39:14,906][105692] Updated weights for policy 0, policy_version 864244 (0.0006) [2023-12-26 21:39:14,946][105620] Updated weights for policy 1, policy_version 864234 (0.0011) [2023-12-26 21:39:14,968][105692] Updated weights for policy 0, policy_version 864254 (0.0009) [2023-12-26 21:39:15,013][105620] Updated weights for policy 1, policy_version 864244 (0.0011) [2023-12-26 21:39:15,671][105692] Updated weights for policy 0, policy_version 864264 (0.0008) [2023-12-26 21:39:15,730][105692] Updated weights for policy 0, policy_version 864274 (0.0008) [2023-12-26 21:39:15,743][105620] Updated weights for policy 1, policy_version 864254 (0.0010) [2023-12-26 21:39:15,789][105692] Updated weights for policy 0, policy_version 864284 (0.0007) [2023-12-26 21:39:15,795][105620] Updated weights for policy 1, policy_version 864264 (0.0010) [2023-12-26 21:39:15,846][105620] Updated weights for policy 1, policy_version 864274 (0.0010) [2023-12-26 21:39:16,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 442572800. Throughput: 0: 9763.5, 1: 9569.9. Samples: 442537772. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-12-26 21:39:16,062][104569] Avg episode reward: [(0, '8449.920'), (1, '8719.003')] [2023-12-26 21:39:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000864288_221290496.pth... [2023-12-26 21:39:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000864280_221282304.pth... [2023-12-26 21:39:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000863136_220995584.pth [2023-12-26 21:39:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000863160_220995584.pth [2023-12-26 21:39:16,531][105692] Updated weights for policy 0, policy_version 864294 (0.0005) [2023-12-26 21:39:16,590][105692] Updated weights for policy 0, policy_version 864304 (0.0005) [2023-12-26 21:39:16,601][105620] Updated weights for policy 1, policy_version 864284 (0.0010) [2023-12-26 21:39:16,648][105692] Updated weights for policy 0, policy_version 864314 (0.0010) [2023-12-26 21:39:16,659][105620] Updated weights for policy 1, policy_version 864294 (0.0010) [2023-12-26 21:39:16,714][105620] Updated weights for policy 1, policy_version 864304 (0.0010) [2023-12-26 21:39:17,270][105692] Updated weights for policy 0, policy_version 864324 (0.0008) [2023-12-26 21:39:17,325][105692] Updated weights for policy 0, policy_version 864334 (0.0010) [2023-12-26 21:39:17,375][105692] Updated weights for policy 0, policy_version 864344 (0.0010) [2023-12-26 21:39:17,461][105620] Updated weights for policy 1, policy_version 864314 (0.0010) [2023-12-26 21:39:17,519][105620] Updated weights for policy 1, policy_version 864324 (0.0010) [2023-12-26 21:39:17,567][105620] Updated weights for policy 1, policy_version 864334 (0.0010) [2023-12-26 21:39:17,623][105620] Updated weights for policy 1, policy_version 864344 (0.0010) [2023-12-26 21:39:17,952][105692] Updated weights for policy 0, policy_version 864354 (0.0009) [2023-12-26 21:39:17,998][105692] Updated weights for policy 0, policy_version 864364 (0.0005) [2023-12-26 21:39:18,045][105692] Updated weights for policy 0, policy_version 864374 (0.0005) [2023-12-26 21:39:18,101][105692] Updated weights for policy 0, policy_version 864384 (0.0005) [2023-12-26 21:39:18,392][105620] Updated weights for policy 1, policy_version 864354 (0.0010) [2023-12-26 21:39:18,448][105620] Updated weights for policy 1, policy_version 864364 (0.0006) [2023-12-26 21:39:18,500][105620] Updated weights for policy 1, policy_version 864374 (0.0008) [2023-12-26 21:39:18,691][105692] Updated weights for policy 0, policy_version 864394 (0.0009) [2023-12-26 21:39:18,748][105692] Updated weights for policy 0, policy_version 864404 (0.0009) [2023-12-26 21:39:18,801][105692] Updated weights for policy 0, policy_version 864414 (0.0009) [2023-12-26 21:39:19,259][105620] Updated weights for policy 1, policy_version 864384 (0.0009) [2023-12-26 21:39:19,323][105620] Updated weights for policy 1, policy_version 864394 (0.0008) [2023-12-26 21:39:19,384][105620] Updated weights for policy 1, policy_version 864404 (0.0009) [2023-12-26 21:39:19,485][105692] Updated weights for policy 0, policy_version 864424 (0.0008) [2023-12-26 21:39:19,547][105692] Updated weights for policy 0, policy_version 864434 (0.0008) [2023-12-26 21:39:19,610][105692] Updated weights for policy 0, policy_version 864444 (0.0008) [2023-12-26 21:39:20,162][105620] Updated weights for policy 1, policy_version 864414 (0.0008) [2023-12-26 21:39:20,216][105620] Updated weights for policy 1, policy_version 864424 (0.0008) [2023-12-26 21:39:20,269][105620] Updated weights for policy 1, policy_version 864434 (0.0008) [2023-12-26 21:39:20,378][105692] Updated weights for policy 0, policy_version 864454 (0.0010) [2023-12-26 21:39:20,433][105692] Updated weights for policy 0, policy_version 864464 (0.0011) [2023-12-26 21:39:20,486][105692] Updated weights for policy 0, policy_version 864474 (0.0011) [2023-12-26 21:39:20,932][105620] Updated weights for policy 1, policy_version 864444 (0.0008) [2023-12-26 21:39:20,988][105620] Updated weights for policy 1, policy_version 864454 (0.0008) [2023-12-26 21:39:21,057][105620] Updated weights for policy 1, policy_version 864464 (0.0007) [2023-12-26 21:39:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 442662912. Throughput: 0: 9930.2, 1: 9492.3. Samples: 442656640. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-12-26 21:39:21,063][104569] Avg episode reward: [(0, '3337.794'), (1, '8379.387')] [2023-12-26 21:39:21,259][105692] Updated weights for policy 0, policy_version 864484 (0.0009) [2023-12-26 21:39:21,319][105692] Updated weights for policy 0, policy_version 864494 (0.0008) [2023-12-26 21:39:21,390][105692] Updated weights for policy 0, policy_version 864504 (0.0008) [2023-12-26 21:39:21,869][105620] Updated weights for policy 1, policy_version 864474 (0.0008) [2023-12-26 21:39:21,936][105620] Updated weights for policy 1, policy_version 864484 (0.0010) [2023-12-26 21:39:21,990][105620] Updated weights for policy 1, policy_version 864494 (0.0009) [2023-12-26 21:39:22,052][105620] Updated weights for policy 1, policy_version 864504 (0.0009) [2023-12-26 21:39:22,112][105692] Updated weights for policy 0, policy_version 864514 (0.0007) [2023-12-26 21:39:22,167][105692] Updated weights for policy 0, policy_version 864524 (0.0009) [2023-12-26 21:39:22,216][105692] Updated weights for policy 0, policy_version 864534 (0.0007) [2023-12-26 21:39:22,272][105692] Updated weights for policy 0, policy_version 864544 (0.0007) [2023-12-26 21:39:22,840][105620] Updated weights for policy 1, policy_version 864514 (0.0009) [2023-12-26 21:39:22,903][105620] Updated weights for policy 1, policy_version 864524 (0.0009) [2023-12-26 21:39:22,965][105620] Updated weights for policy 1, policy_version 864534 (0.0009) [2023-12-26 21:39:23,061][105692] Updated weights for policy 0, policy_version 864554 (0.0009) [2023-12-26 21:39:23,124][105692] Updated weights for policy 0, policy_version 864564 (0.0009) [2023-12-26 21:39:23,185][105692] Updated weights for policy 0, policy_version 864574 (0.0010) [2023-12-26 21:39:23,732][105620] Updated weights for policy 1, policy_version 864544 (0.0010) [2023-12-26 21:39:23,776][105620] Updated weights for policy 1, policy_version 864554 (0.0010) [2023-12-26 21:39:23,828][105620] Updated weights for policy 1, policy_version 864564 (0.0010) [2023-12-26 21:39:23,912][105692] Updated weights for policy 0, policy_version 864584 (0.0009) [2023-12-26 21:39:23,972][105692] Updated weights for policy 0, policy_version 864594 (0.0006) [2023-12-26 21:39:24,019][105692] Updated weights for policy 0, policy_version 864604 (0.0005) [2023-12-26 21:39:24,619][105620] Updated weights for policy 1, policy_version 864574 (0.0009) [2023-12-26 21:39:24,682][105620] Updated weights for policy 1, policy_version 864584 (0.0008) [2023-12-26 21:39:24,708][105692] Updated weights for policy 0, policy_version 864614 (0.0008) [2023-12-26 21:39:24,743][105620] Updated weights for policy 1, policy_version 864594 (0.0006) [2023-12-26 21:39:24,771][105692] Updated weights for policy 0, policy_version 864624 (0.0010) [2023-12-26 21:39:24,833][105692] Updated weights for policy 0, policy_version 864634 (0.0010) [2023-12-26 21:39:25,298][105620] Updated weights for policy 1, policy_version 864604 (0.0009) [2023-12-26 21:39:25,346][105620] Updated weights for policy 1, policy_version 864614 (0.0010) [2023-12-26 21:39:25,394][105620] Updated weights for policy 1, policy_version 864624 (0.0010) [2023-12-26 21:39:25,475][105692] Updated weights for policy 0, policy_version 864644 (0.0008) [2023-12-26 21:39:25,529][105692] Updated weights for policy 0, policy_version 864654 (0.0009) [2023-12-26 21:39:25,578][105692] Updated weights for policy 0, policy_version 864664 (0.0005) [2023-12-26 21:39:26,057][105620] Updated weights for policy 1, policy_version 864634 (0.0009) [2023-12-26 21:39:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 442761216. Throughput: 0: 9917.3, 1: 9547.6. Samples: 442771980. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-12-26 21:39:26,063][104569] Avg episode reward: [(0, '914.997'), (1, '8865.215')] [2023-12-26 21:39:26,113][105620] Updated weights for policy 1, policy_version 864644 (0.0005) [2023-12-26 21:39:26,176][105620] Updated weights for policy 1, policy_version 864654 (0.0006) [2023-12-26 21:39:26,226][105620] Updated weights for policy 1, policy_version 864664 (0.0005) [2023-12-26 21:39:26,307][105692] Updated weights for policy 0, policy_version 864674 (0.0008) [2023-12-26 21:39:26,360][105692] Updated weights for policy 0, policy_version 864684 (0.0010) [2023-12-26 21:39:26,418][105692] Updated weights for policy 0, policy_version 864694 (0.0009) [2023-12-26 21:39:26,483][105692] Updated weights for policy 0, policy_version 864704 (0.0009) [2023-12-26 21:39:26,811][105620] Updated weights for policy 1, policy_version 864674 (0.0010) [2023-12-26 21:39:26,874][105620] Updated weights for policy 1, policy_version 864684 (0.0011) [2023-12-26 21:39:26,923][105620] Updated weights for policy 1, policy_version 864694 (0.0010) [2023-12-26 21:39:27,216][105692] Updated weights for policy 0, policy_version 864714 (0.0008) [2023-12-26 21:39:27,259][105692] Updated weights for policy 0, policy_version 864724 (0.0008) [2023-12-26 21:39:27,314][105692] Updated weights for policy 0, policy_version 864734 (0.0008) [2023-12-26 21:39:27,641][105620] Updated weights for policy 1, policy_version 864704 (0.0009) [2023-12-26 21:39:27,689][105620] Updated weights for policy 1, policy_version 864714 (0.0010) [2023-12-26 21:39:27,737][105620] Updated weights for policy 1, policy_version 864724 (0.0010) [2023-12-26 21:39:28,011][105692] Updated weights for policy 0, policy_version 864744 (0.0008) [2023-12-26 21:39:28,075][105692] Updated weights for policy 0, policy_version 864754 (0.0010) [2023-12-26 21:39:28,133][105692] Updated weights for policy 0, policy_version 864764 (0.0009) [2023-12-26 21:39:28,399][105620] Updated weights for policy 1, policy_version 864734 (0.0007) [2023-12-26 21:39:28,454][105620] Updated weights for policy 1, policy_version 864744 (0.0005) [2023-12-26 21:39:28,515][105620] Updated weights for policy 1, policy_version 864754 (0.0005) [2023-12-26 21:39:28,834][105692] Updated weights for policy 0, policy_version 864774 (0.0007) [2023-12-26 21:39:28,888][105692] Updated weights for policy 0, policy_version 864784 (0.0006) [2023-12-26 21:39:28,949][105692] Updated weights for policy 0, policy_version 864794 (0.0006) [2023-12-26 21:39:29,179][105620] Updated weights for policy 1, policy_version 864764 (0.0009) [2023-12-26 21:39:29,245][105620] Updated weights for policy 1, policy_version 864774 (0.0011) [2023-12-26 21:39:29,297][105620] Updated weights for policy 1, policy_version 864784 (0.0010) [2023-12-26 21:39:29,625][105692] Updated weights for policy 0, policy_version 864804 (0.0007) [2023-12-26 21:39:29,670][105692] Updated weights for policy 0, policy_version 864814 (0.0008) [2023-12-26 21:39:29,714][105692] Updated weights for policy 0, policy_version 864824 (0.0007) [2023-12-26 21:39:30,038][105620] Updated weights for policy 1, policy_version 864794 (0.0011) [2023-12-26 21:39:30,098][105620] Updated weights for policy 1, policy_version 864804 (0.0008) [2023-12-26 21:39:30,149][105620] Updated weights for policy 1, policy_version 864814 (0.0010) [2023-12-26 21:39:30,201][105620] Updated weights for policy 1, policy_version 864824 (0.0010) [2023-12-26 21:39:30,482][105692] Updated weights for policy 0, policy_version 864834 (0.0008) [2023-12-26 21:39:30,529][105692] Updated weights for policy 0, policy_version 864844 (0.0007) [2023-12-26 21:39:30,576][105692] Updated weights for policy 0, policy_version 864854 (0.0008) [2023-12-26 21:39:30,637][105692] Updated weights for policy 0, policy_version 864864 (0.0007) [2023-12-26 21:39:30,949][105620] Updated weights for policy 1, policy_version 864834 (0.0010) [2023-12-26 21:39:30,996][105620] Updated weights for policy 1, policy_version 864844 (0.0010) [2023-12-26 21:39:31,050][105620] Updated weights for policy 1, policy_version 864854 (0.0010) [2023-12-26 21:39:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 442859520. Throughput: 0: 9909.8, 1: 9603.0. Samples: 442832912. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-12-26 21:39:31,063][104569] Avg episode reward: [(0, '946.977'), (1, '7285.023')] [2023-12-26 21:39:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000864864_221437952.pth... [2023-12-26 21:39:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000864856_221429760.pth... [2023-12-26 21:39:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000863712_221143040.pth [2023-12-26 21:39:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000863704_221134848.pth [2023-12-26 21:39:31,325][105692] Updated weights for policy 0, policy_version 864874 (0.0005) [2023-12-26 21:39:31,391][105692] Updated weights for policy 0, policy_version 864884 (0.0007) [2023-12-26 21:39:31,454][105692] Updated weights for policy 0, policy_version 864894 (0.0008) [2023-12-26 21:39:31,810][105620] Updated weights for policy 1, policy_version 864864 (0.0009) [2023-12-26 21:39:31,872][105620] Updated weights for policy 1, policy_version 864874 (0.0010) [2023-12-26 21:39:31,929][105620] Updated weights for policy 1, policy_version 864884 (0.0007) [2023-12-26 21:39:32,080][105692] Updated weights for policy 0, policy_version 864904 (0.0010) [2023-12-26 21:39:32,138][105692] Updated weights for policy 0, policy_version 864914 (0.0010) [2023-12-26 21:39:32,194][105692] Updated weights for policy 0, policy_version 864924 (0.0007) [2023-12-26 21:39:32,607][105620] Updated weights for policy 1, policy_version 864894 (0.0007) [2023-12-26 21:39:32,669][105620] Updated weights for policy 1, policy_version 864904 (0.0008) [2023-12-26 21:39:32,726][105620] Updated weights for policy 1, policy_version 864914 (0.0009) [2023-12-26 21:39:32,888][105692] Updated weights for policy 0, policy_version 864934 (0.0006) [2023-12-26 21:39:32,938][105692] Updated weights for policy 0, policy_version 864944 (0.0005) [2023-12-26 21:39:32,992][105692] Updated weights for policy 0, policy_version 864954 (0.0006) [2023-12-26 21:39:33,536][105692] Updated weights for policy 0, policy_version 864964 (0.0005) [2023-12-26 21:39:33,590][105692] Updated weights for policy 0, policy_version 864974 (0.0005) [2023-12-26 21:39:33,594][105620] Updated weights for policy 1, policy_version 864924 (0.0009) [2023-12-26 21:39:33,637][105692] Updated weights for policy 0, policy_version 864984 (0.0005) [2023-12-26 21:39:33,655][105620] Updated weights for policy 1, policy_version 864934 (0.0009) [2023-12-26 21:39:33,713][105620] Updated weights for policy 1, policy_version 864944 (0.0010) [2023-12-26 21:39:34,322][105692] Updated weights for policy 0, policy_version 864994 (0.0006) [2023-12-26 21:39:34,382][105692] Updated weights for policy 0, policy_version 865004 (0.0009) [2023-12-26 21:39:34,430][105692] Updated weights for policy 0, policy_version 865014 (0.0009) [2023-12-26 21:39:34,490][105692] Updated weights for policy 0, policy_version 865024 (0.0009) [2023-12-26 21:39:34,499][105620] Updated weights for policy 1, policy_version 864954 (0.0009) [2023-12-26 21:39:34,563][105620] Updated weights for policy 1, policy_version 864964 (0.0009) [2023-12-26 21:39:34,624][105620] Updated weights for policy 1, policy_version 864974 (0.0008) [2023-12-26 21:39:34,687][105620] Updated weights for policy 1, policy_version 864984 (0.0007) [2023-12-26 21:39:35,259][105692] Updated weights for policy 0, policy_version 865034 (0.0005) [2023-12-26 21:39:35,306][105692] Updated weights for policy 0, policy_version 865044 (0.0005) [2023-12-26 21:39:35,364][105692] Updated weights for policy 0, policy_version 865054 (0.0006) [2023-12-26 21:39:35,438][105620] Updated weights for policy 1, policy_version 864994 (0.0009) [2023-12-26 21:39:35,493][105620] Updated weights for policy 1, policy_version 865004 (0.0008) [2023-12-26 21:39:35,545][105620] Updated weights for policy 1, policy_version 865014 (0.0008) [2023-12-26 21:39:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 442957824. Throughput: 0: 9943.2, 1: 9548.2. Samples: 442950432. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-12-26 21:39:36,063][104569] Avg episode reward: [(0, '6339.286'), (1, '5692.127')] [2023-12-26 21:39:36,074][105692] Updated weights for policy 0, policy_version 865064 (0.0010) [2023-12-26 21:39:36,134][105692] Updated weights for policy 0, policy_version 865074 (0.0010) [2023-12-26 21:39:36,186][105692] Updated weights for policy 0, policy_version 865084 (0.0011) [2023-12-26 21:39:36,326][105620] Updated weights for policy 1, policy_version 865024 (0.0008) [2023-12-26 21:39:36,385][105620] Updated weights for policy 1, policy_version 865034 (0.0009) [2023-12-26 21:39:36,442][105620] Updated weights for policy 1, policy_version 865044 (0.0013) [2023-12-26 21:39:36,915][105692] Updated weights for policy 0, policy_version 865094 (0.0011) [2023-12-26 21:39:36,977][105692] Updated weights for policy 0, policy_version 865104 (0.0010) [2023-12-26 21:39:37,046][105692] Updated weights for policy 0, policy_version 865114 (0.0010) [2023-12-26 21:39:37,130][105620] Updated weights for policy 1, policy_version 865054 (0.0008) [2023-12-26 21:39:37,193][105620] Updated weights for policy 1, policy_version 865064 (0.0010) [2023-12-26 21:39:37,253][105620] Updated weights for policy 1, policy_version 865074 (0.0011) [2023-12-26 21:39:37,770][105692] Updated weights for policy 0, policy_version 865124 (0.0008) [2023-12-26 21:39:37,836][105692] Updated weights for policy 0, policy_version 865134 (0.0005) [2023-12-26 21:39:37,891][105692] Updated weights for policy 0, policy_version 865144 (0.0010) [2023-12-26 21:39:37,926][105620] Updated weights for policy 1, policy_version 865084 (0.0010) [2023-12-26 21:39:37,971][105620] Updated weights for policy 1, policy_version 865094 (0.0010) [2023-12-26 21:39:38,019][105620] Updated weights for policy 1, policy_version 865104 (0.0009) [2023-12-26 21:39:38,549][105692] Updated weights for policy 0, policy_version 865154 (0.0009) [2023-12-26 21:39:38,615][105692] Updated weights for policy 0, policy_version 865164 (0.0009) [2023-12-26 21:39:38,677][105692] Updated weights for policy 0, policy_version 865174 (0.0006) [2023-12-26 21:39:38,737][105692] Updated weights for policy 0, policy_version 865184 (0.0008) [2023-12-26 21:39:38,810][105620] Updated weights for policy 1, policy_version 865114 (0.0008) [2023-12-26 21:39:38,870][105620] Updated weights for policy 1, policy_version 865124 (0.0005) [2023-12-26 21:39:38,927][105620] Updated weights for policy 1, policy_version 865134 (0.0006) [2023-12-26 21:39:38,997][105620] Updated weights for policy 1, policy_version 865144 (0.0007) [2023-12-26 21:39:39,362][105692] Updated weights for policy 0, policy_version 865194 (0.0009) [2023-12-26 21:39:39,422][105692] Updated weights for policy 0, policy_version 865204 (0.0008) [2023-12-26 21:39:39,489][105692] Updated weights for policy 0, policy_version 865214 (0.0009) [2023-12-26 21:39:39,666][105620] Updated weights for policy 1, policy_version 865154 (0.0009) [2023-12-26 21:39:39,721][105620] Updated weights for policy 1, policy_version 865164 (0.0010) [2023-12-26 21:39:39,789][105620] Updated weights for policy 1, policy_version 865174 (0.0009) [2023-12-26 21:39:40,253][105692] Updated weights for policy 0, policy_version 865224 (0.0008) [2023-12-26 21:39:40,302][105692] Updated weights for policy 0, policy_version 865234 (0.0009) [2023-12-26 21:39:40,359][105692] Updated weights for policy 0, policy_version 865244 (0.0008) [2023-12-26 21:39:40,544][105620] Updated weights for policy 1, policy_version 865184 (0.0006) [2023-12-26 21:39:40,601][105620] Updated weights for policy 1, policy_version 865194 (0.0005) [2023-12-26 21:39:40,651][105620] Updated weights for policy 1, policy_version 865204 (0.0007) [2023-12-26 21:39:41,000][105692] Updated weights for policy 0, policy_version 865254 (0.0007) [2023-12-26 21:39:41,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 443056128. Throughput: 0: 9958.2, 1: 9571.2. Samples: 443067188. Policy #0 lag: (min: 21.0, avg: 43.4, max: 48.0) [2023-12-26 21:39:41,062][104569] Avg episode reward: [(0, '8734.738'), (1, '7564.627')] [2023-12-26 21:39:41,068][105692] Updated weights for policy 0, policy_version 865264 (0.0008) [2023-12-26 21:39:41,127][105692] Updated weights for policy 0, policy_version 865274 (0.0007) [2023-12-26 21:39:41,437][105620] Updated weights for policy 1, policy_version 865214 (0.0009) [2023-12-26 21:39:41,488][105620] Updated weights for policy 1, policy_version 865224 (0.0009) [2023-12-26 21:39:41,544][105620] Updated weights for policy 1, policy_version 865234 (0.0009) [2023-12-26 21:39:41,931][105692] Updated weights for policy 0, policy_version 865284 (0.0009) [2023-12-26 21:39:41,995][105692] Updated weights for policy 0, policy_version 865294 (0.0009) [2023-12-26 21:39:42,055][105692] Updated weights for policy 0, policy_version 865304 (0.0009) [2023-12-26 21:39:42,264][105620] Updated weights for policy 1, policy_version 865244 (0.0009) [2023-12-26 21:39:42,332][105620] Updated weights for policy 1, policy_version 865254 (0.0009) [2023-12-26 21:39:42,398][105620] Updated weights for policy 1, policy_version 865264 (0.0009) [2023-12-26 21:39:42,798][105692] Updated weights for policy 0, policy_version 865314 (0.0008) [2023-12-26 21:39:42,854][105692] Updated weights for policy 0, policy_version 865324 (0.0007) [2023-12-26 21:39:42,909][105692] Updated weights for policy 0, policy_version 865334 (0.0009) [2023-12-26 21:39:42,962][105692] Updated weights for policy 0, policy_version 865344 (0.0008) [2023-12-26 21:39:43,138][105620] Updated weights for policy 1, policy_version 865274 (0.0008) [2023-12-26 21:39:43,204][105620] Updated weights for policy 1, policy_version 865284 (0.0005) [2023-12-26 21:39:43,255][105620] Updated weights for policy 1, policy_version 865294 (0.0005) [2023-12-26 21:39:43,313][105620] Updated weights for policy 1, policy_version 865304 (0.0008) [2023-12-26 21:39:43,586][105692] Updated weights for policy 0, policy_version 865354 (0.0005) [2023-12-26 21:39:43,646][105692] Updated weights for policy 0, policy_version 865364 (0.0010) [2023-12-26 21:39:43,699][105692] Updated weights for policy 0, policy_version 865374 (0.0010) [2023-12-26 21:39:43,858][105620] Updated weights for policy 1, policy_version 865314 (0.0006) [2023-12-26 21:39:43,913][105620] Updated weights for policy 1, policy_version 865325 (0.0010) [2023-12-26 21:39:43,966][105620] Updated weights for policy 1, policy_version 865335 (0.0010) [2023-12-26 21:39:44,335][105692] Updated weights for policy 0, policy_version 865384 (0.0010) [2023-12-26 21:39:44,397][105692] Updated weights for policy 0, policy_version 865394 (0.0010) [2023-12-26 21:39:44,466][105692] Updated weights for policy 0, policy_version 865404 (0.0010) [2023-12-26 21:39:44,537][105620] Updated weights for policy 1, policy_version 865345 (0.0008) [2023-12-26 21:39:44,588][105620] Updated weights for policy 1, policy_version 865355 (0.0006) [2023-12-26 21:39:44,643][105620] Updated weights for policy 1, policy_version 865365 (0.0005) [2023-12-26 21:39:45,233][105692] Updated weights for policy 0, policy_version 865414 (0.0008) [2023-12-26 21:39:45,291][105692] Updated weights for policy 0, policy_version 865424 (0.0006) [2023-12-26 21:39:45,344][105692] Updated weights for policy 0, policy_version 865434 (0.0006) [2023-12-26 21:39:45,382][105620] Updated weights for policy 1, policy_version 865375 (0.0008) [2023-12-26 21:39:45,441][105620] Updated weights for policy 1, policy_version 865385 (0.0010) [2023-12-26 21:39:45,496][105620] Updated weights for policy 1, policy_version 865395 (0.0010) [2023-12-26 21:39:45,948][105692] Updated weights for policy 0, policy_version 865444 (0.0006) [2023-12-26 21:39:46,005][105692] Updated weights for policy 0, policy_version 865454 (0.0005) [2023-12-26 21:39:46,054][105692] Updated weights for policy 0, policy_version 865464 (0.0005) [2023-12-26 21:39:46,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 443154432. Throughput: 0: 9934.5, 1: 9616.9. Samples: 443127432. Policy #0 lag: (min: 21.0, avg: 43.4, max: 48.0) [2023-12-26 21:39:46,062][104569] Avg episode reward: [(0, '8997.588'), (1, '8987.626')] [2023-12-26 21:39:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000865400_221569024.pth... [2023-12-26 21:39:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000864280_221282304.pth [2023-12-26 21:39:46,089][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000865472_221593600.pth... [2023-12-26 21:39:46,092][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000864288_221290496.pth [2023-12-26 21:39:46,326][105620] Updated weights for policy 1, policy_version 865405 (0.0008) [2023-12-26 21:39:46,380][105620] Updated weights for policy 1, policy_version 865415 (0.0007) [2023-12-26 21:39:46,439][105620] Updated weights for policy 1, policy_version 865425 (0.0008) [2023-12-26 21:39:46,710][105692] Updated weights for policy 0, policy_version 865474 (0.0006) [2023-12-26 21:39:46,761][105692] Updated weights for policy 0, policy_version 865484 (0.0010) [2023-12-26 21:39:46,822][105692] Updated weights for policy 0, policy_version 865494 (0.0009) [2023-12-26 21:39:46,872][105692] Updated weights for policy 0, policy_version 865504 (0.0005) [2023-12-26 21:39:47,237][105620] Updated weights for policy 1, policy_version 865435 (0.0008) [2023-12-26 21:39:47,298][105620] Updated weights for policy 1, policy_version 865445 (0.0008) [2023-12-26 21:39:47,356][105620] Updated weights for policy 1, policy_version 865455 (0.0006) [2023-12-26 21:39:47,530][105692] Updated weights for policy 0, policy_version 865514 (0.0009) [2023-12-26 21:39:47,582][105692] Updated weights for policy 0, policy_version 865524 (0.0010) [2023-12-26 21:39:47,640][105692] Updated weights for policy 0, policy_version 865534 (0.0010) [2023-12-26 21:39:48,117][105620] Updated weights for policy 1, policy_version 865465 (0.0005) [2023-12-26 21:39:48,171][105620] Updated weights for policy 1, policy_version 865475 (0.0008) [2023-12-26 21:39:48,228][105620] Updated weights for policy 1, policy_version 865485 (0.0009) [2023-12-26 21:39:48,284][105620] Updated weights for policy 1, policy_version 865495 (0.0010) [2023-12-26 21:39:48,292][105692] Updated weights for policy 0, policy_version 865544 (0.0010) [2023-12-26 21:39:48,360][105692] Updated weights for policy 0, policy_version 865554 (0.0008) [2023-12-26 21:39:48,423][105692] Updated weights for policy 0, policy_version 865564 (0.0009) [2023-12-26 21:39:49,073][105692] Updated weights for policy 0, policy_version 865574 (0.0008) [2023-12-26 21:39:49,100][105620] Updated weights for policy 1, policy_version 865505 (0.0009) [2023-12-26 21:39:49,127][105692] Updated weights for policy 0, policy_version 865584 (0.0005) [2023-12-26 21:39:49,152][105620] Updated weights for policy 1, policy_version 865515 (0.0008) [2023-12-26 21:39:49,179][105692] Updated weights for policy 0, policy_version 865594 (0.0005) [2023-12-26 21:39:49,200][105620] Updated weights for policy 1, policy_version 865525 (0.0009) [2023-12-26 21:39:49,798][105692] Updated weights for policy 0, policy_version 865604 (0.0008) [2023-12-26 21:39:49,866][105692] Updated weights for policy 0, policy_version 865614 (0.0007) [2023-12-26 21:39:49,932][105692] Updated weights for policy 0, policy_version 865624 (0.0006) [2023-12-26 21:39:49,976][105620] Updated weights for policy 1, policy_version 865535 (0.0009) [2023-12-26 21:39:50,039][105620] Updated weights for policy 1, policy_version 865545 (0.0006) [2023-12-26 21:39:50,109][105620] Updated weights for policy 1, policy_version 865555 (0.0006) [2023-12-26 21:39:50,626][105692] Updated weights for policy 0, policy_version 865634 (0.0009) [2023-12-26 21:39:50,685][105692] Updated weights for policy 0, policy_version 865644 (0.0011) [2023-12-26 21:39:50,734][105620] Updated weights for policy 1, policy_version 865565 (0.0005) [2023-12-26 21:39:50,747][105692] Updated weights for policy 0, policy_version 865654 (0.0010) [2023-12-26 21:39:50,794][105620] Updated weights for policy 1, policy_version 865575 (0.0006) [2023-12-26 21:39:50,808][105692] Updated weights for policy 0, policy_version 865664 (0.0010) [2023-12-26 21:39:50,841][105620] Updated weights for policy 1, policy_version 865585 (0.0007) [2023-12-26 21:39:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 443260928. Throughput: 0: 10091.8, 1: 9566.5. Samples: 443245396. Policy #0 lag: (min: 21.0, avg: 43.4, max: 48.0) [2023-12-26 21:39:51,062][104569] Avg episode reward: [(0, '8908.130'), (1, '8430.108')] [2023-12-26 21:39:51,518][105620] Updated weights for policy 1, policy_version 865595 (0.0009) [2023-12-26 21:39:51,542][105692] Updated weights for policy 0, policy_version 865674 (0.0006) [2023-12-26 21:39:51,586][105620] Updated weights for policy 1, policy_version 865605 (0.0008) [2023-12-26 21:39:51,609][105692] Updated weights for policy 0, policy_version 865684 (0.0006) [2023-12-26 21:39:51,656][105620] Updated weights for policy 1, policy_version 865615 (0.0008) [2023-12-26 21:39:51,678][105692] Updated weights for policy 0, policy_version 865694 (0.0007) [2023-12-26 21:39:52,280][105692] Updated weights for policy 0, policy_version 865704 (0.0006) [2023-12-26 21:39:52,335][105692] Updated weights for policy 0, policy_version 865714 (0.0006) [2023-12-26 21:39:52,364][105620] Updated weights for policy 1, policy_version 865625 (0.0007) [2023-12-26 21:39:52,400][105692] Updated weights for policy 0, policy_version 865724 (0.0008) [2023-12-26 21:39:52,424][105620] Updated weights for policy 1, policy_version 865635 (0.0008) [2023-12-26 21:39:52,486][105620] Updated weights for policy 1, policy_version 865645 (0.0008) [2023-12-26 21:39:52,552][105620] Updated weights for policy 1, policy_version 865655 (0.0008) [2023-12-26 21:39:53,029][105692] Updated weights for policy 0, policy_version 865734 (0.0009) [2023-12-26 21:39:53,086][105692] Updated weights for policy 0, policy_version 865744 (0.0011) [2023-12-26 21:39:53,144][105692] Updated weights for policy 0, policy_version 865754 (0.0008) [2023-12-26 21:39:53,215][105620] Updated weights for policy 1, policy_version 865665 (0.0010) [2023-12-26 21:39:53,263][105620] Updated weights for policy 1, policy_version 865675 (0.0010) [2023-12-26 21:39:53,317][105620] Updated weights for policy 1, policy_version 865685 (0.0010) [2023-12-26 21:39:53,775][105692] Updated weights for policy 0, policy_version 865764 (0.0008) [2023-12-26 21:39:53,820][105692] Updated weights for policy 0, policy_version 865774 (0.0008) [2023-12-26 21:39:53,867][105692] Updated weights for policy 0, policy_version 865784 (0.0006) [2023-12-26 21:39:53,961][105620] Updated weights for policy 1, policy_version 865695 (0.0008) [2023-12-26 21:39:54,025][105620] Updated weights for policy 1, policy_version 865705 (0.0005) [2023-12-26 21:39:54,084][105620] Updated weights for policy 1, policy_version 865715 (0.0009) [2023-12-26 21:39:54,580][105692] Updated weights for policy 0, policy_version 865794 (0.0010) [2023-12-26 21:39:54,635][105692] Updated weights for policy 0, policy_version 865804 (0.0010) [2023-12-26 21:39:54,699][105692] Updated weights for policy 0, policy_version 865814 (0.0011) [2023-12-26 21:39:54,761][105692] Updated weights for policy 0, policy_version 865824 (0.0011) [2023-12-26 21:39:54,836][105620] Updated weights for policy 1, policy_version 865725 (0.0008) [2023-12-26 21:39:54,885][105620] Updated weights for policy 1, policy_version 865735 (0.0008) [2023-12-26 21:39:54,939][105620] Updated weights for policy 1, policy_version 865745 (0.0007) [2023-12-26 21:39:55,489][105692] Updated weights for policy 0, policy_version 865834 (0.0005) [2023-12-26 21:39:55,540][105692] Updated weights for policy 0, policy_version 865844 (0.0005) [2023-12-26 21:39:55,587][105692] Updated weights for policy 0, policy_version 865854 (0.0009) [2023-12-26 21:39:55,753][105620] Updated weights for policy 1, policy_version 865755 (0.0009) [2023-12-26 21:39:55,806][105620] Updated weights for policy 1, policy_version 865766 (0.0009) [2023-12-26 21:39:55,858][105620] Updated weights for policy 1, policy_version 865777 (0.0010) [2023-12-26 21:39:56,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.9, 300 sec: 19522.0). Total num frames: 443359232. Throughput: 0: 10101.8, 1: 9598.1. Samples: 443365388. Policy #0 lag: (min: 21.0, avg: 43.4, max: 48.0) [2023-12-26 21:39:56,063][104569] Avg episode reward: [(0, '8905.720'), (1, '8618.831')] [2023-12-26 21:39:56,155][105692] Updated weights for policy 0, policy_version 865864 (0.0008) [2023-12-26 21:39:56,217][105692] Updated weights for policy 0, policy_version 865874 (0.0009) [2023-12-26 21:39:56,276][105692] Updated weights for policy 0, policy_version 865884 (0.0009) [2023-12-26 21:39:56,710][105620] Updated weights for policy 1, policy_version 865788 (0.0010) [2023-12-26 21:39:56,758][105620] Updated weights for policy 1, policy_version 865798 (0.0009) [2023-12-26 21:39:56,807][105620] Updated weights for policy 1, policy_version 865808 (0.0009) [2023-12-26 21:39:56,920][105692] Updated weights for policy 0, policy_version 865894 (0.0008) [2023-12-26 21:39:56,978][105692] Updated weights for policy 0, policy_version 865904 (0.0009) [2023-12-26 21:39:57,040][105692] Updated weights for policy 0, policy_version 865914 (0.0009) [2023-12-26 21:39:57,569][105620] Updated weights for policy 1, policy_version 865818 (0.0010) [2023-12-26 21:39:57,623][105620] Updated weights for policy 1, policy_version 865828 (0.0009) [2023-12-26 21:39:57,692][105620] Updated weights for policy 1, policy_version 865838 (0.0008) [2023-12-26 21:39:57,747][105620] Updated weights for policy 1, policy_version 865848 (0.0010) [2023-12-26 21:39:57,756][105692] Updated weights for policy 0, policy_version 865924 (0.0009) [2023-12-26 21:39:57,810][105692] Updated weights for policy 0, policy_version 865934 (0.0007) [2023-12-26 21:39:57,870][105692] Updated weights for policy 0, policy_version 865944 (0.0008) [2023-12-26 21:39:58,502][105620] Updated weights for policy 1, policy_version 865858 (0.0011) [2023-12-26 21:39:58,563][105620] Updated weights for policy 1, policy_version 865868 (0.0011) [2023-12-26 21:39:58,625][105620] Updated weights for policy 1, policy_version 865878 (0.0007) [2023-12-26 21:39:58,665][105692] Updated weights for policy 0, policy_version 865954 (0.0008) [2023-12-26 21:39:58,730][105692] Updated weights for policy 0, policy_version 865964 (0.0008) [2023-12-26 21:39:58,796][105692] Updated weights for policy 0, policy_version 865974 (0.0009) [2023-12-26 21:39:58,859][105692] Updated weights for policy 0, policy_version 865984 (0.0009) [2023-12-26 21:39:59,416][105620] Updated weights for policy 1, policy_version 865888 (0.0008) [2023-12-26 21:39:59,482][105620] Updated weights for policy 1, policy_version 865898 (0.0009) [2023-12-26 21:39:59,547][105620] Updated weights for policy 1, policy_version 865908 (0.0008) [2023-12-26 21:39:59,663][105692] Updated weights for policy 0, policy_version 865994 (0.0008) [2023-12-26 21:39:59,726][105692] Updated weights for policy 0, policy_version 866004 (0.0008) [2023-12-26 21:39:59,784][105692] Updated weights for policy 0, policy_version 866014 (0.0010) [2023-12-26 21:40:00,268][105620] Updated weights for policy 1, policy_version 865918 (0.0007) [2023-12-26 21:40:00,321][105620] Updated weights for policy 1, policy_version 865928 (0.0005) [2023-12-26 21:40:00,387][105620] Updated weights for policy 1, policy_version 865938 (0.0005) [2023-12-26 21:40:00,480][105692] Updated weights for policy 0, policy_version 866024 (0.0010) [2023-12-26 21:40:00,538][105692] Updated weights for policy 0, policy_version 866034 (0.0010) [2023-12-26 21:40:00,594][105692] Updated weights for policy 0, policy_version 866044 (0.0008) [2023-12-26 21:40:01,046][105620] Updated weights for policy 1, policy_version 865948 (0.0011) [2023-12-26 21:40:01,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 443449344. Throughput: 0: 10113.7, 1: 9565.1. Samples: 443423320. Policy #0 lag: (min: 21.0, avg: 43.4, max: 48.0) [2023-12-26 21:40:01,063][104569] Avg episode reward: [(0, '8902.222'), (1, '9084.249')] [2023-12-26 21:40:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000866048_221741056.pth... [2023-12-26 21:40:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000864864_221437952.pth [2023-12-26 21:40:01,112][105620] Updated weights for policy 1, policy_version 865958 (0.0011) [2023-12-26 21:40:01,175][105620] Updated weights for policy 1, policy_version 865968 (0.0010) [2023-12-26 21:40:01,227][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000865976_221716480.pth... [2023-12-26 21:40:01,231][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000864856_221429760.pth [2023-12-26 21:40:01,265][105692] Updated weights for policy 0, policy_version 866054 (0.0007) [2023-12-26 21:40:01,327][105692] Updated weights for policy 0, policy_version 866064 (0.0005) [2023-12-26 21:40:01,394][105692] Updated weights for policy 0, policy_version 866074 (0.0009) [2023-12-26 21:40:01,997][105620] Updated weights for policy 1, policy_version 865978 (0.0009) [2023-12-26 21:40:02,041][105692] Updated weights for policy 0, policy_version 866084 (0.0008) [2023-12-26 21:40:02,055][105620] Updated weights for policy 1, policy_version 865988 (0.0008) [2023-12-26 21:40:02,095][105692] Updated weights for policy 0, policy_version 866094 (0.0005) [2023-12-26 21:40:02,104][105620] Updated weights for policy 1, policy_version 865998 (0.0009) [2023-12-26 21:40:02,150][105692] Updated weights for policy 0, policy_version 866104 (0.0005) [2023-12-26 21:40:02,160][105620] Updated weights for policy 1, policy_version 866008 (0.0009) [2023-12-26 21:40:02,736][105692] Updated weights for policy 0, policy_version 866114 (0.0006) [2023-12-26 21:40:02,788][105692] Updated weights for policy 0, policy_version 866124 (0.0010) [2023-12-26 21:40:02,846][105692] Updated weights for policy 0, policy_version 866134 (0.0010) [2023-12-26 21:40:02,900][105692] Updated weights for policy 0, policy_version 866144 (0.0010) [2023-12-26 21:40:02,966][105620] Updated weights for policy 1, policy_version 866018 (0.0010) [2023-12-26 21:40:03,010][105620] Updated weights for policy 1, policy_version 866028 (0.0010) [2023-12-26 21:40:03,062][105620] Updated weights for policy 1, policy_version 866038 (0.0010) [2023-12-26 21:40:03,625][105692] Updated weights for policy 0, policy_version 866154 (0.0008) [2023-12-26 21:40:03,689][105692] Updated weights for policy 0, policy_version 866164 (0.0007) [2023-12-26 21:40:03,741][105692] Updated weights for policy 0, policy_version 866174 (0.0010) [2023-12-26 21:40:03,765][105620] Updated weights for policy 1, policy_version 866048 (0.0010) [2023-12-26 21:40:03,831][105620] Updated weights for policy 1, policy_version 866058 (0.0010) [2023-12-26 21:40:03,896][105620] Updated weights for policy 1, policy_version 866068 (0.0011) [2023-12-26 21:40:04,438][105692] Updated weights for policy 0, policy_version 866184 (0.0008) [2023-12-26 21:40:04,496][105692] Updated weights for policy 0, policy_version 866194 (0.0008) [2023-12-26 21:40:04,554][105692] Updated weights for policy 0, policy_version 866204 (0.0009) [2023-12-26 21:40:04,683][105620] Updated weights for policy 1, policy_version 866078 (0.0007) [2023-12-26 21:40:04,742][105620] Updated weights for policy 1, policy_version 866088 (0.0006) [2023-12-26 21:40:04,804][105620] Updated weights for policy 1, policy_version 866098 (0.0010) [2023-12-26 21:40:05,272][105692] Updated weights for policy 0, policy_version 866214 (0.0006) [2023-12-26 21:40:05,321][105692] Updated weights for policy 0, policy_version 866224 (0.0007) [2023-12-26 21:40:05,379][105692] Updated weights for policy 0, policy_version 866234 (0.0010) [2023-12-26 21:40:05,443][105620] Updated weights for policy 1, policy_version 866108 (0.0009) [2023-12-26 21:40:05,497][105620] Updated weights for policy 1, policy_version 866118 (0.0006) [2023-12-26 21:40:05,548][105620] Updated weights for policy 1, policy_version 866128 (0.0005) [2023-12-26 21:40:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 443547648. Throughput: 0: 10036.9, 1: 9572.3. Samples: 443539052. Policy #0 lag: (min: 21.0, avg: 43.4, max: 48.0) [2023-12-26 21:40:06,063][104569] Avg episode reward: [(0, '8994.260'), (1, '9062.319')] [2023-12-26 21:40:06,069][105692] Updated weights for policy 0, policy_version 866244 (0.0010) [2023-12-26 21:40:06,139][105692] Updated weights for policy 0, policy_version 866254 (0.0008) [2023-12-26 21:40:06,148][105620] Updated weights for policy 1, policy_version 866138 (0.0006) [2023-12-26 21:40:06,208][105692] Updated weights for policy 0, policy_version 866264 (0.0008) [2023-12-26 21:40:06,214][105620] Updated weights for policy 1, policy_version 866148 (0.0008) [2023-12-26 21:40:06,273][105620] Updated weights for policy 1, policy_version 866158 (0.0008) [2023-12-26 21:40:06,339][105620] Updated weights for policy 1, policy_version 866168 (0.0008) [2023-12-26 21:40:06,851][105692] Updated weights for policy 0, policy_version 866274 (0.0010) [2023-12-26 21:40:06,919][105692] Updated weights for policy 0, policy_version 866284 (0.0006) [2023-12-26 21:40:06,986][105692] Updated weights for policy 0, policy_version 866294 (0.0006) [2023-12-26 21:40:07,045][105692] Updated weights for policy 0, policy_version 866304 (0.0007) [2023-12-26 21:40:07,183][105620] Updated weights for policy 1, policy_version 866178 (0.0008) [2023-12-26 21:40:07,244][105620] Updated weights for policy 1, policy_version 866188 (0.0010) [2023-12-26 21:40:07,301][105620] Updated weights for policy 1, policy_version 866198 (0.0009) [2023-12-26 21:40:07,623][105692] Updated weights for policy 0, policy_version 866314 (0.0010) [2023-12-26 21:40:07,682][105692] Updated weights for policy 0, policy_version 866324 (0.0010) [2023-12-26 21:40:07,734][105692] Updated weights for policy 0, policy_version 866334 (0.0010) [2023-12-26 21:40:08,128][105620] Updated weights for policy 1, policy_version 866208 (0.0010) [2023-12-26 21:40:08,185][105620] Updated weights for policy 1, policy_version 866218 (0.0006) [2023-12-26 21:40:08,236][105620] Updated weights for policy 1, policy_version 866228 (0.0005) [2023-12-26 21:40:08,346][105692] Updated weights for policy 0, policy_version 866344 (0.0009) [2023-12-26 21:40:08,413][105692] Updated weights for policy 0, policy_version 866354 (0.0008) [2023-12-26 21:40:08,482][105692] Updated weights for policy 0, policy_version 866364 (0.0006) [2023-12-26 21:40:08,922][105620] Updated weights for policy 1, policy_version 866238 (0.0007) [2023-12-26 21:40:08,978][105620] Updated weights for policy 1, policy_version 866248 (0.0008) [2023-12-26 21:40:09,026][105620] Updated weights for policy 1, policy_version 866258 (0.0008) [2023-12-26 21:40:09,186][105692] Updated weights for policy 0, policy_version 866374 (0.0010) [2023-12-26 21:40:09,246][105692] Updated weights for policy 0, policy_version 866384 (0.0011) [2023-12-26 21:40:09,318][105692] Updated weights for policy 0, policy_version 866394 (0.0010) [2023-12-26 21:40:09,831][105620] Updated weights for policy 1, policy_version 866268 (0.0008) [2023-12-26 21:40:09,886][105620] Updated weights for policy 1, policy_version 866278 (0.0008) [2023-12-26 21:40:09,938][105620] Updated weights for policy 1, policy_version 866288 (0.0008) [2023-12-26 21:40:10,078][105692] Updated weights for policy 0, policy_version 866404 (0.0008) [2023-12-26 21:40:10,138][105692] Updated weights for policy 0, policy_version 866414 (0.0007) [2023-12-26 21:40:10,202][105692] Updated weights for policy 0, policy_version 866424 (0.0011) [2023-12-26 21:40:10,673][105620] Updated weights for policy 1, policy_version 866298 (0.0009) [2023-12-26 21:40:10,739][105620] Updated weights for policy 1, policy_version 866308 (0.0010) [2023-12-26 21:40:10,788][105620] Updated weights for policy 1, policy_version 866318 (0.0008) [2023-12-26 21:40:10,836][105620] Updated weights for policy 1, policy_version 866328 (0.0008) [2023-12-26 21:40:10,920][105692] Updated weights for policy 0, policy_version 866434 (0.0009) [2023-12-26 21:40:10,975][105692] Updated weights for policy 0, policy_version 866444 (0.0009) [2023-12-26 21:40:11,042][105692] Updated weights for policy 0, policy_version 866454 (0.0011) [2023-12-26 21:40:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 443645952. Throughput: 0: 10108.0, 1: 9553.3. Samples: 443656736. Policy #0 lag: (min: 21.0, avg: 43.4, max: 48.0) [2023-12-26 21:40:11,062][104569] Avg episode reward: [(0, '8906.729'), (1, '9063.013')] [2023-12-26 21:40:11,105][105692] Updated weights for policy 0, policy_version 866464 (0.0011) [2023-12-26 21:40:11,550][105620] Updated weights for policy 1, policy_version 866338 (0.0010) [2023-12-26 21:40:11,605][105620] Updated weights for policy 1, policy_version 866348 (0.0009) [2023-12-26 21:40:11,673][105620] Updated weights for policy 1, policy_version 866358 (0.0009) [2023-12-26 21:40:11,813][105692] Updated weights for policy 0, policy_version 866474 (0.0011) [2023-12-26 21:40:11,876][105692] Updated weights for policy 0, policy_version 866484 (0.0010) [2023-12-26 21:40:11,933][105692] Updated weights for policy 0, policy_version 866494 (0.0010) [2023-12-26 21:40:12,392][105620] Updated weights for policy 1, policy_version 866368 (0.0008) [2023-12-26 21:40:12,455][105620] Updated weights for policy 1, policy_version 866378 (0.0008) [2023-12-26 21:40:12,515][105620] Updated weights for policy 1, policy_version 866388 (0.0008) [2023-12-26 21:40:12,654][105692] Updated weights for policy 0, policy_version 866504 (0.0011) [2023-12-26 21:40:12,719][105692] Updated weights for policy 0, policy_version 866514 (0.0010) [2023-12-26 21:40:12,785][105692] Updated weights for policy 0, policy_version 866524 (0.0011) [2023-12-26 21:40:13,207][105620] Updated weights for policy 1, policy_version 866398 (0.0006) [2023-12-26 21:40:13,275][105620] Updated weights for policy 1, policy_version 866408 (0.0005) [2023-12-26 21:40:13,330][105620] Updated weights for policy 1, policy_version 866418 (0.0005) [2023-12-26 21:40:13,467][105692] Updated weights for policy 0, policy_version 866534 (0.0010) [2023-12-26 21:40:13,525][105692] Updated weights for policy 0, policy_version 866544 (0.0010) [2023-12-26 21:40:13,583][105692] Updated weights for policy 0, policy_version 866554 (0.0010) [2023-12-26 21:40:13,909][105620] Updated weights for policy 1, policy_version 866428 (0.0007) [2023-12-26 21:40:13,956][105620] Updated weights for policy 1, policy_version 866438 (0.0006) [2023-12-26 21:40:14,013][105620] Updated weights for policy 1, policy_version 866448 (0.0007) [2023-12-26 21:40:14,288][105692] Updated weights for policy 0, policy_version 866564 (0.0007) [2023-12-26 21:40:14,345][105692] Updated weights for policy 0, policy_version 866574 (0.0008) [2023-12-26 21:40:14,391][105692] Updated weights for policy 0, policy_version 866584 (0.0010) [2023-12-26 21:40:14,690][105620] Updated weights for policy 1, policy_version 866458 (0.0005) [2023-12-26 21:40:14,748][105620] Updated weights for policy 1, policy_version 866468 (0.0005) [2023-12-26 21:40:14,813][105620] Updated weights for policy 1, policy_version 866478 (0.0006) [2023-12-26 21:40:14,875][105620] Updated weights for policy 1, policy_version 866488 (0.0007) [2023-12-26 21:40:15,115][105692] Updated weights for policy 0, policy_version 866594 (0.0010) [2023-12-26 21:40:15,170][105692] Updated weights for policy 0, policy_version 866604 (0.0009) [2023-12-26 21:40:15,219][105692] Updated weights for policy 0, policy_version 866614 (0.0009) [2023-12-26 21:40:15,272][105692] Updated weights for policy 0, policy_version 866624 (0.0009) [2023-12-26 21:40:15,482][105620] Updated weights for policy 1, policy_version 866498 (0.0008) [2023-12-26 21:40:15,539][105620] Updated weights for policy 1, policy_version 866508 (0.0009) [2023-12-26 21:40:15,597][105620] Updated weights for policy 1, policy_version 866518 (0.0009) [2023-12-26 21:40:15,935][105692] Updated weights for policy 0, policy_version 866634 (0.0006) [2023-12-26 21:40:15,998][105692] Updated weights for policy 0, policy_version 866644 (0.0007) [2023-12-26 21:40:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 443744256. Throughput: 0: 10111.3, 1: 9530.7. Samples: 443716800. Policy #0 lag: (min: 21.0, avg: 43.4, max: 48.0) [2023-12-26 21:40:16,062][104569] Avg episode reward: [(0, '9084.704'), (1, '9077.393')] [2023-12-26 21:40:16,064][105692] Updated weights for policy 0, policy_version 866654 (0.0010) [2023-12-26 21:40:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000866520_221855744.pth... [2023-12-26 21:40:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000865400_221569024.pth [2023-12-26 21:40:16,078][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000866656_221896704.pth... [2023-12-26 21:40:16,084][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000865472_221593600.pth [2023-12-26 21:40:16,334][105620] Updated weights for policy 1, policy_version 866528 (0.0008) [2023-12-26 21:40:16,394][105620] Updated weights for policy 1, policy_version 866538 (0.0009) [2023-12-26 21:40:16,451][105620] Updated weights for policy 1, policy_version 866548 (0.0010) [2023-12-26 21:40:16,711][105692] Updated weights for policy 0, policy_version 866664 (0.0006) [2023-12-26 21:40:16,768][105692] Updated weights for policy 0, policy_version 866674 (0.0005) [2023-12-26 21:40:16,829][105692] Updated weights for policy 0, policy_version 866684 (0.0005) [2023-12-26 21:40:17,128][105620] Updated weights for policy 1, policy_version 866558 (0.0007) [2023-12-26 21:40:17,174][105620] Updated weights for policy 1, policy_version 866568 (0.0005) [2023-12-26 21:40:17,224][105620] Updated weights for policy 1, policy_version 866578 (0.0008) [2023-12-26 21:40:17,530][105692] Updated weights for policy 0, policy_version 866694 (0.0008) [2023-12-26 21:40:17,577][105692] Updated weights for policy 0, policy_version 866704 (0.0008) [2023-12-26 21:40:17,624][105692] Updated weights for policy 0, policy_version 866714 (0.0009) [2023-12-26 21:40:17,902][105620] Updated weights for policy 1, policy_version 866588 (0.0008) [2023-12-26 21:40:17,957][105620] Updated weights for policy 1, policy_version 866598 (0.0005) [2023-12-26 21:40:18,015][105620] Updated weights for policy 1, policy_version 866608 (0.0005) [2023-12-26 21:40:18,392][105692] Updated weights for policy 0, policy_version 866724 (0.0009) [2023-12-26 21:40:18,446][105692] Updated weights for policy 0, policy_version 866734 (0.0008) [2023-12-26 21:40:18,496][105692] Updated weights for policy 0, policy_version 866744 (0.0008) [2023-12-26 21:40:18,699][105620] Updated weights for policy 1, policy_version 866618 (0.0009) [2023-12-26 21:40:18,762][105620] Updated weights for policy 1, policy_version 866628 (0.0009) [2023-12-26 21:40:18,822][105620] Updated weights for policy 1, policy_version 866638 (0.0009) [2023-12-26 21:40:18,879][105620] Updated weights for policy 1, policy_version 866648 (0.0009) [2023-12-26 21:40:19,320][105692] Updated weights for policy 0, policy_version 866754 (0.0009) [2023-12-26 21:40:19,392][105692] Updated weights for policy 0, policy_version 866764 (0.0008) [2023-12-26 21:40:19,446][105692] Updated weights for policy 0, policy_version 866774 (0.0008) [2023-12-26 21:40:19,507][105692] Updated weights for policy 0, policy_version 866784 (0.0009) [2023-12-26 21:40:19,597][105620] Updated weights for policy 1, policy_version 866658 (0.0009) [2023-12-26 21:40:19,660][105620] Updated weights for policy 1, policy_version 866668 (0.0009) [2023-12-26 21:40:19,722][105620] Updated weights for policy 1, policy_version 866678 (0.0009) [2023-12-26 21:40:20,346][105692] Updated weights for policy 0, policy_version 866794 (0.0010) [2023-12-26 21:40:20,385][105620] Updated weights for policy 1, policy_version 866688 (0.0006) [2023-12-26 21:40:20,398][105692] Updated weights for policy 0, policy_version 866804 (0.0009) [2023-12-26 21:40:20,438][105620] Updated weights for policy 1, policy_version 866698 (0.0007) [2023-12-26 21:40:20,453][105692] Updated weights for policy 0, policy_version 866814 (0.0007) [2023-12-26 21:40:20,486][105620] Updated weights for policy 1, policy_version 866708 (0.0008) [2023-12-26 21:40:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 443842560. Throughput: 0: 10021.7, 1: 9619.5. Samples: 443834284. Policy #0 lag: (min: 21.0, avg: 43.4, max: 48.0) [2023-12-26 21:40:21,063][104569] Avg episode reward: [(0, '9174.632'), (1, '8986.284')] [2023-12-26 21:40:21,218][105620] Updated weights for policy 1, policy_version 866718 (0.0007) [2023-12-26 21:40:21,287][105620] Updated weights for policy 1, policy_version 866728 (0.0007) [2023-12-26 21:40:21,304][105692] Updated weights for policy 0, policy_version 866824 (0.0007) [2023-12-26 21:40:21,347][105620] Updated weights for policy 1, policy_version 866738 (0.0008) [2023-12-26 21:40:21,367][105692] Updated weights for policy 0, policy_version 866834 (0.0007) [2023-12-26 21:40:21,431][105692] Updated weights for policy 0, policy_version 866844 (0.0008) [2023-12-26 21:40:22,016][105620] Updated weights for policy 1, policy_version 866748 (0.0007) [2023-12-26 21:40:22,068][105620] Updated weights for policy 1, policy_version 866758 (0.0006) [2023-12-26 21:40:22,122][105620] Updated weights for policy 1, policy_version 866768 (0.0006) [2023-12-26 21:40:22,305][105692] Updated weights for policy 0, policy_version 866854 (0.0008) [2023-12-26 21:40:22,369][105692] Updated weights for policy 0, policy_version 866864 (0.0008) [2023-12-26 21:40:22,425][105692] Updated weights for policy 0, policy_version 866874 (0.0009) [2023-12-26 21:40:22,814][105620] Updated weights for policy 1, policy_version 866778 (0.0005) [2023-12-26 21:40:22,873][105620] Updated weights for policy 1, policy_version 866788 (0.0007) [2023-12-26 21:40:22,932][105620] Updated weights for policy 1, policy_version 866798 (0.0009) [2023-12-26 21:40:22,996][105620] Updated weights for policy 1, policy_version 866808 (0.0008) [2023-12-26 21:40:23,176][105692] Updated weights for policy 0, policy_version 866884 (0.0008) [2023-12-26 21:40:23,233][105692] Updated weights for policy 0, policy_version 866894 (0.0009) [2023-12-26 21:40:23,299][105692] Updated weights for policy 0, policy_version 866904 (0.0009) [2023-12-26 21:40:23,646][105620] Updated weights for policy 1, policy_version 866818 (0.0009) [2023-12-26 21:40:23,697][105620] Updated weights for policy 1, policy_version 866829 (0.0010) [2023-12-26 21:40:23,742][105620] Updated weights for policy 1, policy_version 866839 (0.0007) [2023-12-26 21:40:24,029][105692] Updated weights for policy 0, policy_version 866914 (0.0008) [2023-12-26 21:40:24,081][105692] Updated weights for policy 0, policy_version 866924 (0.0006) [2023-12-26 21:40:24,130][105692] Updated weights for policy 0, policy_version 866934 (0.0005) [2023-12-26 21:40:24,183][105692] Updated weights for policy 0, policy_version 866944 (0.0007) [2023-12-26 21:40:24,455][105620] Updated weights for policy 1, policy_version 866849 (0.0010) [2023-12-26 21:40:24,515][105620] Updated weights for policy 1, policy_version 866859 (0.0011) [2023-12-26 21:40:24,565][105620] Updated weights for policy 1, policy_version 866869 (0.0011) [2023-12-26 21:40:24,891][105585] KL-divergence is very high: 154.2727 [2023-12-26 21:40:24,902][105692] Updated weights for policy 0, policy_version 866954 (0.0008) [2023-12-26 21:40:24,937][105585] KL-divergence is very high: 305.1768 [2023-12-26 21:40:24,958][105692] Updated weights for policy 0, policy_version 866964 (0.0008) [2023-12-26 21:40:24,985][105585] KL-divergence is very high: 317.8646 [2023-12-26 21:40:25,020][105692] Updated weights for policy 0, policy_version 866974 (0.0009) [2023-12-26 21:40:25,292][105620] Updated weights for policy 1, policy_version 866879 (0.0009) [2023-12-26 21:40:25,365][105620] Updated weights for policy 1, policy_version 866889 (0.0011) [2023-12-26 21:40:25,438][105620] Updated weights for policy 1, policy_version 866899 (0.0011) [2023-12-26 21:40:25,734][105692] Updated weights for policy 0, policy_version 866984 (0.0006) [2023-12-26 21:40:25,788][105692] Updated weights for policy 0, policy_version 866994 (0.0005) [2023-12-26 21:40:25,842][105692] Updated weights for policy 0, policy_version 867004 (0.0005) [2023-12-26 21:40:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 443940864. Throughput: 0: 9903.8, 1: 9707.1. Samples: 443949680. Policy #0 lag: (min: 21.0, avg: 43.4, max: 48.0) [2023-12-26 21:40:26,062][104569] Avg episode reward: [(0, '8996.663'), (1, '9260.575')] [2023-12-26 21:40:26,069][105620] Updated weights for policy 1, policy_version 866909 (0.0010) [2023-12-26 21:40:26,130][105620] Updated weights for policy 1, policy_version 866919 (0.0010) [2023-12-26 21:40:26,188][105620] Updated weights for policy 1, policy_version 866929 (0.0010) [2023-12-26 21:40:26,509][105692] Updated weights for policy 0, policy_version 867014 (0.0008) [2023-12-26 21:40:26,574][105692] Updated weights for policy 0, policy_version 867025 (0.0008) [2023-12-26 21:40:26,638][105692] Updated weights for policy 0, policy_version 867035 (0.0006) [2023-12-26 21:40:26,775][105620] Updated weights for policy 1, policy_version 866939 (0.0009) [2023-12-26 21:40:26,828][105620] Updated weights for policy 1, policy_version 866949 (0.0006) [2023-12-26 21:40:26,883][105620] Updated weights for policy 1, policy_version 866959 (0.0006) [2023-12-26 21:40:27,203][105692] Updated weights for policy 0, policy_version 867045 (0.0010) [2023-12-26 21:40:27,251][105692] Updated weights for policy 0, policy_version 867055 (0.0010) [2023-12-26 21:40:27,312][105692] Updated weights for policy 0, policy_version 867065 (0.0010) [2023-12-26 21:40:27,512][105620] Updated weights for policy 1, policy_version 866969 (0.0010) [2023-12-26 21:40:27,567][105620] Updated weights for policy 1, policy_version 866979 (0.0006) [2023-12-26 21:40:27,630][105620] Updated weights for policy 1, policy_version 866989 (0.0010) [2023-12-26 21:40:27,685][105620] Updated weights for policy 1, policy_version 866999 (0.0008) [2023-12-26 21:40:27,914][105692] Updated weights for policy 0, policy_version 867075 (0.0007) [2023-12-26 21:40:27,977][105692] Updated weights for policy 0, policy_version 867085 (0.0005) [2023-12-26 21:40:28,034][105692] Updated weights for policy 0, policy_version 867095 (0.0005) [2023-12-26 21:40:28,331][105620] Updated weights for policy 1, policy_version 867009 (0.0010) [2023-12-26 21:40:28,388][105620] Updated weights for policy 1, policy_version 867019 (0.0010) [2023-12-26 21:40:28,444][105620] Updated weights for policy 1, policy_version 867029 (0.0010) [2023-12-26 21:40:28,630][105692] Updated weights for policy 0, policy_version 867105 (0.0008) [2023-12-26 21:40:28,692][105692] Updated weights for policy 0, policy_version 867115 (0.0011) [2023-12-26 21:40:28,743][105692] Updated weights for policy 0, policy_version 867125 (0.0007) [2023-12-26 21:40:28,795][105692] Updated weights for policy 0, policy_version 867135 (0.0005) [2023-12-26 21:40:29,153][105620] Updated weights for policy 1, policy_version 867039 (0.0009) [2023-12-26 21:40:29,209][105620] Updated weights for policy 1, policy_version 867049 (0.0008) [2023-12-26 21:40:29,270][105620] Updated weights for policy 1, policy_version 867059 (0.0008) [2023-12-26 21:40:29,415][105692] Updated weights for policy 0, policy_version 867145 (0.0006) [2023-12-26 21:40:29,470][105692] Updated weights for policy 0, policy_version 867155 (0.0006) [2023-12-26 21:40:29,524][105692] Updated weights for policy 0, policy_version 867165 (0.0006) [2023-12-26 21:40:30,107][105620] Updated weights for policy 1, policy_version 867069 (0.0007) [2023-12-26 21:40:30,121][105692] Updated weights for policy 0, policy_version 867175 (0.0009) [2023-12-26 21:40:30,167][105620] Updated weights for policy 1, policy_version 867079 (0.0007) [2023-12-26 21:40:30,171][105692] Updated weights for policy 0, policy_version 867185 (0.0010) [2023-12-26 21:40:30,226][105692] Updated weights for policy 0, policy_version 867195 (0.0010) [2023-12-26 21:40:30,227][105620] Updated weights for policy 1, policy_version 867089 (0.0010) [2023-12-26 21:40:30,820][105692] Updated weights for policy 0, policy_version 867205 (0.0008) [2023-12-26 21:40:30,867][105692] Updated weights for policy 0, policy_version 867215 (0.0007) [2023-12-26 21:40:30,919][105692] Updated weights for policy 0, policy_version 867225 (0.0007) [2023-12-26 21:40:31,027][105620] Updated weights for policy 1, policy_version 867099 (0.0006) [2023-12-26 21:40:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 444047360. Throughput: 0: 9990.8, 1: 9734.8. Samples: 444015084. Policy #0 lag: (min: 21.0, avg: 43.4, max: 48.0) [2023-12-26 21:40:31,062][104569] Avg episode reward: [(0, '8820.867'), (1, '9260.568')] [2023-12-26 21:40:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000867232_222044160.pth... [2023-12-26 21:40:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000866048_221741056.pth [2023-12-26 21:40:31,088][105620] Updated weights for policy 1, policy_version 867109 (0.0008) [2023-12-26 21:40:31,145][105620] Updated weights for policy 1, policy_version 867119 (0.0009) [2023-12-26 21:40:31,205][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000867128_222011392.pth... [2023-12-26 21:40:31,209][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000865976_221716480.pth [2023-12-26 21:40:31,644][105692] Updated weights for policy 0, policy_version 867235 (0.0009) [2023-12-26 21:40:31,703][105692] Updated weights for policy 0, policy_version 867245 (0.0006) [2023-12-26 21:40:31,773][105692] Updated weights for policy 0, policy_version 867255 (0.0009) [2023-12-26 21:40:31,840][105620] Updated weights for policy 1, policy_version 867129 (0.0006) [2023-12-26 21:40:31,912][105620] Updated weights for policy 1, policy_version 867139 (0.0010) [2023-12-26 21:40:31,966][105620] Updated weights for policy 1, policy_version 867149 (0.0005) [2023-12-26 21:40:32,033][105620] Updated weights for policy 1, policy_version 867159 (0.0009) [2023-12-26 21:40:32,391][105692] Updated weights for policy 0, policy_version 867265 (0.0008) [2023-12-26 21:40:32,442][105692] Updated weights for policy 0, policy_version 867275 (0.0006) [2023-12-26 21:40:32,501][105692] Updated weights for policy 0, policy_version 867285 (0.0009) [2023-12-26 21:40:32,552][105692] Updated weights for policy 0, policy_version 867295 (0.0010) [2023-12-26 21:40:32,752][105620] Updated weights for policy 1, policy_version 867169 (0.0008) [2023-12-26 21:40:32,808][105620] Updated weights for policy 1, policy_version 867179 (0.0009) [2023-12-26 21:40:32,869][105620] Updated weights for policy 1, policy_version 867189 (0.0009) [2023-12-26 21:40:33,182][105692] Updated weights for policy 0, policy_version 867305 (0.0006) [2023-12-26 21:40:33,236][105692] Updated weights for policy 0, policy_version 867315 (0.0006) [2023-12-26 21:40:33,295][105692] Updated weights for policy 0, policy_version 867325 (0.0005) [2023-12-26 21:40:33,746][105620] Updated weights for policy 1, policy_version 867199 (0.0009) [2023-12-26 21:40:33,804][105620] Updated weights for policy 1, policy_version 867209 (0.0008) [2023-12-26 21:40:33,812][105692] Updated weights for policy 0, policy_version 867335 (0.0005) [2023-12-26 21:40:33,864][105620] Updated weights for policy 1, policy_version 867219 (0.0008) [2023-12-26 21:40:33,879][105692] Updated weights for policy 0, policy_version 867345 (0.0005) [2023-12-26 21:40:33,939][105692] Updated weights for policy 0, policy_version 867355 (0.0005) [2023-12-26 21:40:34,598][105620] Updated weights for policy 1, policy_version 867230 (0.0007) [2023-12-26 21:40:34,607][105692] Updated weights for policy 0, policy_version 867365 (0.0007) [2023-12-26 21:40:34,651][105620] Updated weights for policy 1, policy_version 867240 (0.0005) [2023-12-26 21:40:34,673][105692] Updated weights for policy 0, policy_version 867375 (0.0009) [2023-12-26 21:40:34,702][105620] Updated weights for policy 1, policy_version 867250 (0.0006) [2023-12-26 21:40:34,736][105692] Updated weights for policy 0, policy_version 867385 (0.0009) [2023-12-26 21:40:35,339][105620] Updated weights for policy 1, policy_version 867260 (0.0007) [2023-12-26 21:40:35,400][105620] Updated weights for policy 1, policy_version 867270 (0.0009) [2023-12-26 21:40:35,459][105620] Updated weights for policy 1, policy_version 867280 (0.0007) [2023-12-26 21:40:35,524][105692] Updated weights for policy 0, policy_version 867395 (0.0009) [2023-12-26 21:40:35,574][105692] Updated weights for policy 0, policy_version 867405 (0.0009) [2023-12-26 21:40:35,632][105692] Updated weights for policy 0, policy_version 867415 (0.0008) [2023-12-26 21:40:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 444145664. Throughput: 0: 10075.6, 1: 9738.0. Samples: 444137008. Policy #0 lag: (min: 21.0, avg: 43.4, max: 48.0) [2023-12-26 21:40:36,063][104569] Avg episode reward: [(0, '8906.423'), (1, '9178.854')] [2023-12-26 21:40:36,263][105620] Updated weights for policy 1, policy_version 867290 (0.0008) [2023-12-26 21:40:36,296][105692] Updated weights for policy 0, policy_version 867425 (0.0009) [2023-12-26 21:40:36,315][105620] Updated weights for policy 1, policy_version 867300 (0.0007) [2023-12-26 21:40:36,351][105692] Updated weights for policy 0, policy_version 867435 (0.0007) [2023-12-26 21:40:36,371][105620] Updated weights for policy 1, policy_version 867310 (0.0007) [2023-12-26 21:40:36,402][105692] Updated weights for policy 0, policy_version 867445 (0.0010) [2023-12-26 21:40:36,421][105620] Updated weights for policy 1, policy_version 867320 (0.0007) [2023-12-26 21:40:36,450][105692] Updated weights for policy 0, policy_version 867455 (0.0007) [2023-12-26 21:40:37,106][105620] Updated weights for policy 1, policy_version 867330 (0.0009) [2023-12-26 21:40:37,179][105620] Updated weights for policy 1, policy_version 867340 (0.0007) [2023-12-26 21:40:37,238][105620] Updated weights for policy 1, policy_version 867350 (0.0008) [2023-12-26 21:40:37,276][105692] Updated weights for policy 0, policy_version 867465 (0.0010) [2023-12-26 21:40:37,328][105692] Updated weights for policy 0, policy_version 867475 (0.0010) [2023-12-26 21:40:37,390][105692] Updated weights for policy 0, policy_version 867485 (0.0010) [2023-12-26 21:40:37,944][105620] Updated weights for policy 1, policy_version 867360 (0.0009) [2023-12-26 21:40:37,996][105620] Updated weights for policy 1, policy_version 867370 (0.0010) [2023-12-26 21:40:38,054][105620] Updated weights for policy 1, policy_version 867380 (0.0011) [2023-12-26 21:40:38,107][105692] Updated weights for policy 0, policy_version 867495 (0.0010) [2023-12-26 21:40:38,164][105692] Updated weights for policy 0, policy_version 867505 (0.0011) [2023-12-26 21:40:38,213][105692] Updated weights for policy 0, policy_version 867515 (0.0010) [2023-12-26 21:40:38,795][105620] Updated weights for policy 1, policy_version 867390 (0.0008) [2023-12-26 21:40:38,847][105620] Updated weights for policy 1, policy_version 867400 (0.0010) [2023-12-26 21:40:38,902][105620] Updated weights for policy 1, policy_version 867410 (0.0010) [2023-12-26 21:40:38,972][105692] Updated weights for policy 0, policy_version 867525 (0.0010) [2023-12-26 21:40:39,020][105692] Updated weights for policy 0, policy_version 867535 (0.0010) [2023-12-26 21:40:39,068][105692] Updated weights for policy 0, policy_version 867545 (0.0010) [2023-12-26 21:40:39,571][105620] Updated weights for policy 1, policy_version 867420 (0.0010) [2023-12-26 21:40:39,629][105620] Updated weights for policy 1, policy_version 867430 (0.0008) [2023-12-26 21:40:39,692][105620] Updated weights for policy 1, policy_version 867440 (0.0008) [2023-12-26 21:40:39,895][105692] Updated weights for policy 0, policy_version 867555 (0.0010) [2023-12-26 21:40:39,959][105692] Updated weights for policy 0, policy_version 867565 (0.0010) [2023-12-26 21:40:40,013][105692] Updated weights for policy 0, policy_version 867575 (0.0011) [2023-12-26 21:40:40,472][105620] Updated weights for policy 1, policy_version 867450 (0.0008) [2023-12-26 21:40:40,537][105620] Updated weights for policy 1, policy_version 867460 (0.0011) [2023-12-26 21:40:40,601][105620] Updated weights for policy 1, policy_version 867470 (0.0011) [2023-12-26 21:40:40,669][105620] Updated weights for policy 1, policy_version 867480 (0.0011) [2023-12-26 21:40:40,798][105692] Updated weights for policy 0, policy_version 867585 (0.0011) [2023-12-26 21:40:40,861][105692] Updated weights for policy 0, policy_version 867595 (0.0011) [2023-12-26 21:40:40,921][105692] Updated weights for policy 0, policy_version 867605 (0.0010) [2023-12-26 21:40:40,966][105692] Updated weights for policy 0, policy_version 867615 (0.0011) [2023-12-26 21:40:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 444243968. Throughput: 0: 9937.4, 1: 9717.8. Samples: 444249872. Policy #0 lag: (min: 21.0, avg: 43.4, max: 48.0) [2023-12-26 21:40:41,062][104569] Avg episode reward: [(0, '9085.957'), (1, '8904.592')] [2023-12-26 21:40:41,408][105620] Updated weights for policy 1, policy_version 867490 (0.0013) [2023-12-26 21:40:41,467][105620] Updated weights for policy 1, policy_version 867500 (0.0008) [2023-12-26 21:40:41,530][105620] Updated weights for policy 1, policy_version 867510 (0.0008) [2023-12-26 21:40:41,768][105692] Updated weights for policy 0, policy_version 867625 (0.0008) [2023-12-26 21:40:41,826][105692] Updated weights for policy 0, policy_version 867635 (0.0009) [2023-12-26 21:40:41,888][105692] Updated weights for policy 0, policy_version 867645 (0.0006) [2023-12-26 21:40:42,354][105620] Updated weights for policy 1, policy_version 867520 (0.0009) [2023-12-26 21:40:42,419][105620] Updated weights for policy 1, policy_version 867530 (0.0006) [2023-12-26 21:40:42,477][105620] Updated weights for policy 1, policy_version 867540 (0.0008) [2023-12-26 21:40:42,544][105692] Updated weights for policy 0, policy_version 867655 (0.0008) [2023-12-26 21:40:42,608][105692] Updated weights for policy 0, policy_version 867665 (0.0006) [2023-12-26 21:40:42,680][105692] Updated weights for policy 0, policy_version 867675 (0.0006) [2023-12-26 21:40:43,223][105692] Updated weights for policy 0, policy_version 867685 (0.0008) [2023-12-26 21:40:43,270][105692] Updated weights for policy 0, policy_version 867695 (0.0009) [2023-12-26 21:40:43,318][105692] Updated weights for policy 0, policy_version 867705 (0.0007) [2023-12-26 21:40:43,320][105620] Updated weights for policy 1, policy_version 867550 (0.0008) [2023-12-26 21:40:43,366][105620] Updated weights for policy 1, policy_version 867560 (0.0006) [2023-12-26 21:40:43,418][105620] Updated weights for policy 1, policy_version 867570 (0.0009) [2023-12-26 21:40:44,031][105692] Updated weights for policy 0, policy_version 867715 (0.0007) [2023-12-26 21:40:44,085][105692] Updated weights for policy 0, policy_version 867725 (0.0005) [2023-12-26 21:40:44,138][105692] Updated weights for policy 0, policy_version 867735 (0.0006) [2023-12-26 21:40:44,155][105620] Updated weights for policy 1, policy_version 867580 (0.0008) [2023-12-26 21:40:44,214][105620] Updated weights for policy 1, policy_version 867590 (0.0009) [2023-12-26 21:40:44,283][105620] Updated weights for policy 1, policy_version 867600 (0.0010) [2023-12-26 21:40:44,767][105692] Updated weights for policy 0, policy_version 867745 (0.0006) [2023-12-26 21:40:44,822][105692] Updated weights for policy 0, policy_version 867755 (0.0009) [2023-12-26 21:40:44,878][105692] Updated weights for policy 0, policy_version 867765 (0.0007) [2023-12-26 21:40:44,933][105692] Updated weights for policy 0, policy_version 867775 (0.0009) [2023-12-26 21:40:44,976][105620] Updated weights for policy 1, policy_version 867610 (0.0006) [2023-12-26 21:40:45,039][105620] Updated weights for policy 1, policy_version 867620 (0.0010) [2023-12-26 21:40:45,100][105620] Updated weights for policy 1, policy_version 867630 (0.0010) [2023-12-26 21:40:45,163][105620] Updated weights for policy 1, policy_version 867640 (0.0008) [2023-12-26 21:40:45,685][105692] Updated weights for policy 0, policy_version 867785 (0.0008) [2023-12-26 21:40:45,737][105692] Updated weights for policy 0, policy_version 867795 (0.0008) [2023-12-26 21:40:45,794][105692] Updated weights for policy 0, policy_version 867805 (0.0008) [2023-12-26 21:40:45,905][105620] Updated weights for policy 1, policy_version 867650 (0.0011) [2023-12-26 21:40:45,969][105620] Updated weights for policy 1, policy_version 867660 (0.0008) [2023-12-26 21:40:46,030][105620] Updated weights for policy 1, policy_version 867670 (0.0008) [2023-12-26 21:40:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 444342272. Throughput: 0: 9921.7, 1: 9702.4. Samples: 444306404. Policy #0 lag: (min: 21.0, avg: 43.4, max: 48.0) [2023-12-26 21:40:46,063][104569] Avg episode reward: [(0, '9263.586'), (1, '9077.981')] [2023-12-26 21:40:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000867808_222191616.pth... [2023-12-26 21:40:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000867672_222150656.pth... [2023-12-26 21:40:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000866656_221896704.pth [2023-12-26 21:40:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000866520_221855744.pth [2023-12-26 21:40:46,598][105620] Updated weights for policy 1, policy_version 867680 (0.0010) [2023-12-26 21:40:46,639][105692] Updated weights for policy 0, policy_version 867815 (0.0006) [2023-12-26 21:40:46,656][105620] Updated weights for policy 1, policy_version 867690 (0.0010) [2023-12-26 21:40:46,694][105692] Updated weights for policy 0, policy_version 867825 (0.0006) [2023-12-26 21:40:46,703][105620] Updated weights for policy 1, policy_version 867700 (0.0010) [2023-12-26 21:40:46,751][105692] Updated weights for policy 0, policy_version 867835 (0.0007) [2023-12-26 21:40:47,442][105620] Updated weights for policy 1, policy_version 867710 (0.0010) [2023-12-26 21:40:47,503][105620] Updated weights for policy 1, policy_version 867720 (0.0010) [2023-12-26 21:40:47,517][105692] Updated weights for policy 0, policy_version 867845 (0.0007) [2023-12-26 21:40:47,565][105620] Updated weights for policy 1, policy_version 867730 (0.0010) [2023-12-26 21:40:47,575][105692] Updated weights for policy 0, policy_version 867855 (0.0007) [2023-12-26 21:40:47,635][105692] Updated weights for policy 0, policy_version 867865 (0.0008) [2023-12-26 21:40:48,302][105620] Updated weights for policy 1, policy_version 867740 (0.0010) [2023-12-26 21:40:48,372][105620] Updated weights for policy 1, policy_version 867750 (0.0012) [2023-12-26 21:40:48,396][105692] Updated weights for policy 0, policy_version 867875 (0.0008) [2023-12-26 21:40:48,433][105620] Updated weights for policy 1, policy_version 867760 (0.0009) [2023-12-26 21:40:48,449][105692] Updated weights for policy 0, policy_version 867885 (0.0009) [2023-12-26 21:40:48,495][105692] Updated weights for policy 0, policy_version 867895 (0.0008) [2023-12-26 21:40:49,129][105620] Updated weights for policy 1, policy_version 867770 (0.0008) [2023-12-26 21:40:49,184][105620] Updated weights for policy 1, policy_version 867780 (0.0008) [2023-12-26 21:40:49,263][105620] Updated weights for policy 1, policy_version 867790 (0.0008) [2023-12-26 21:40:49,316][105692] Updated weights for policy 0, policy_version 867905 (0.0010) [2023-12-26 21:40:49,330][105620] Updated weights for policy 1, policy_version 867800 (0.0008) [2023-12-26 21:40:49,388][105692] Updated weights for policy 0, policy_version 867915 (0.0012) [2023-12-26 21:40:49,448][105692] Updated weights for policy 0, policy_version 867925 (0.0010) [2023-12-26 21:40:49,506][105692] Updated weights for policy 0, policy_version 867935 (0.0011) [2023-12-26 21:40:49,982][105620] Updated weights for policy 1, policy_version 867810 (0.0005) [2023-12-26 21:40:50,046][105620] Updated weights for policy 1, policy_version 867820 (0.0006) [2023-12-26 21:40:50,105][105620] Updated weights for policy 1, policy_version 867830 (0.0006) [2023-12-26 21:40:50,270][105692] Updated weights for policy 0, policy_version 867945 (0.0011) [2023-12-26 21:40:50,338][105692] Updated weights for policy 0, policy_version 867955 (0.0011) [2023-12-26 21:40:50,401][105692] Updated weights for policy 0, policy_version 867965 (0.0011) [2023-12-26 21:40:50,685][105620] Updated weights for policy 1, policy_version 867840 (0.0007) [2023-12-26 21:40:50,744][105620] Updated weights for policy 1, policy_version 867850 (0.0009) [2023-12-26 21:40:50,801][105620] Updated weights for policy 1, policy_version 867860 (0.0012) [2023-12-26 21:40:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 444432384. Throughput: 0: 9846.2, 1: 9760.1. Samples: 444421332. Policy #0 lag: (min: 21.0, avg: 43.4, max: 48.0) [2023-12-26 21:40:51,062][104569] Avg episode reward: [(0, '9174.564'), (1, '9078.845')] [2023-12-26 21:40:51,072][105692] Updated weights for policy 0, policy_version 867975 (0.0009) [2023-12-26 21:40:51,147][105692] Updated weights for policy 0, policy_version 867985 (0.0007) [2023-12-26 21:40:51,212][105692] Updated weights for policy 0, policy_version 867995 (0.0006) [2023-12-26 21:40:51,525][105620] Updated weights for policy 1, policy_version 867870 (0.0007) [2023-12-26 21:40:51,579][105620] Updated weights for policy 1, policy_version 867880 (0.0008) [2023-12-26 21:40:51,640][105620] Updated weights for policy 1, policy_version 867890 (0.0008) [2023-12-26 21:40:51,964][105692] Updated weights for policy 0, policy_version 868005 (0.0006) [2023-12-26 21:40:52,021][105692] Updated weights for policy 0, policy_version 868015 (0.0008) [2023-12-26 21:40:52,074][105692] Updated weights for policy 0, policy_version 868025 (0.0009) [2023-12-26 21:40:52,355][105620] Updated weights for policy 1, policy_version 867900 (0.0008) [2023-12-26 21:40:52,416][105620] Updated weights for policy 1, policy_version 867910 (0.0009) [2023-12-26 21:40:52,481][105620] Updated weights for policy 1, policy_version 867920 (0.0009) [2023-12-26 21:40:52,844][105692] Updated weights for policy 0, policy_version 868035 (0.0009) [2023-12-26 21:40:52,892][105692] Updated weights for policy 0, policy_version 868045 (0.0009) [2023-12-26 21:40:52,943][105692] Updated weights for policy 0, policy_version 868055 (0.0009) [2023-12-26 21:40:53,283][105620] Updated weights for policy 1, policy_version 867930 (0.0009) [2023-12-26 21:40:53,334][105620] Updated weights for policy 1, policy_version 867940 (0.0009) [2023-12-26 21:40:53,402][105620] Updated weights for policy 1, policy_version 867950 (0.0009) [2023-12-26 21:40:53,468][105620] Updated weights for policy 1, policy_version 867960 (0.0010) [2023-12-26 21:40:53,524][105692] Updated weights for policy 0, policy_version 868065 (0.0006) [2023-12-26 21:40:53,581][105692] Updated weights for policy 0, policy_version 868075 (0.0005) [2023-12-26 21:40:53,625][105692] Updated weights for policy 0, policy_version 868085 (0.0008) [2023-12-26 21:40:53,671][105692] Updated weights for policy 0, policy_version 868095 (0.0005) [2023-12-26 21:40:54,289][105692] Updated weights for policy 0, policy_version 868105 (0.0008) [2023-12-26 21:40:54,319][105620] Updated weights for policy 1, policy_version 867970 (0.0007) [2023-12-26 21:40:54,341][105692] Updated weights for policy 0, policy_version 868115 (0.0008) [2023-12-26 21:40:54,371][105620] Updated weights for policy 1, policy_version 867980 (0.0008) [2023-12-26 21:40:54,393][105692] Updated weights for policy 0, policy_version 868125 (0.0005) [2023-12-26 21:40:54,430][105620] Updated weights for policy 1, policy_version 867990 (0.0010) [2023-12-26 21:40:55,076][105692] Updated weights for policy 0, policy_version 868135 (0.0006) [2023-12-26 21:40:55,139][105692] Updated weights for policy 0, policy_version 868145 (0.0006) [2023-12-26 21:40:55,159][105620] Updated weights for policy 1, policy_version 868000 (0.0011) [2023-12-26 21:40:55,198][105692] Updated weights for policy 0, policy_version 868155 (0.0006) [2023-12-26 21:40:55,220][105620] Updated weights for policy 1, policy_version 868010 (0.0008) [2023-12-26 21:40:55,279][105620] Updated weights for policy 1, policy_version 868020 (0.0009) [2023-12-26 21:40:55,829][105692] Updated weights for policy 0, policy_version 868165 (0.0005) [2023-12-26 21:40:55,878][105692] Updated weights for policy 0, policy_version 868175 (0.0005) [2023-12-26 21:40:55,926][105692] Updated weights for policy 0, policy_version 868185 (0.0006) [2023-12-26 21:40:55,954][105620] Updated weights for policy 1, policy_version 868030 (0.0007) [2023-12-26 21:40:56,013][105620] Updated weights for policy 1, policy_version 868040 (0.0005) [2023-12-26 21:40:56,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 444530688. Throughput: 0: 9847.4, 1: 9767.3. Samples: 444539396. Policy #0 lag: (min: 21.0, avg: 43.4, max: 48.0) [2023-12-26 21:40:56,062][104569] Avg episode reward: [(0, '9084.809'), (1, '9170.740')] [2023-12-26 21:40:56,074][105620] Updated weights for policy 1, policy_version 868050 (0.0007) [2023-12-26 21:40:56,674][105692] Updated weights for policy 0, policy_version 868195 (0.0009) [2023-12-26 21:40:56,719][105620] Updated weights for policy 1, policy_version 868060 (0.0007) [2023-12-26 21:40:56,726][105692] Updated weights for policy 0, policy_version 868205 (0.0010) [2023-12-26 21:40:56,771][105692] Updated weights for policy 0, policy_version 868215 (0.0009) [2023-12-26 21:40:56,772][105620] Updated weights for policy 1, policy_version 868070 (0.0006) [2023-12-26 21:40:56,825][105620] Updated weights for policy 1, policy_version 868080 (0.0008) [2023-12-26 21:40:57,374][105692] Updated weights for policy 0, policy_version 868225 (0.0006) [2023-12-26 21:40:57,421][105692] Updated weights for policy 0, policy_version 868235 (0.0010) [2023-12-26 21:40:57,466][105692] Updated weights for policy 0, policy_version 868245 (0.0010) [2023-12-26 21:40:57,484][105620] Updated weights for policy 1, policy_version 868091 (0.0009) [2023-12-26 21:40:57,517][105692] Updated weights for policy 0, policy_version 868255 (0.0010) [2023-12-26 21:40:57,540][105620] Updated weights for policy 1, policy_version 868101 (0.0006) [2023-12-26 21:40:57,591][105620] Updated weights for policy 1, policy_version 868111 (0.0008) [2023-12-26 21:40:58,268][105692] Updated weights for policy 0, policy_version 868265 (0.0008) [2023-12-26 21:40:58,335][105692] Updated weights for policy 0, policy_version 868275 (0.0010) [2023-12-26 21:40:58,343][105620] Updated weights for policy 1, policy_version 868121 (0.0007) [2023-12-26 21:40:58,404][105692] Updated weights for policy 0, policy_version 868285 (0.0010) [2023-12-26 21:40:58,405][105620] Updated weights for policy 1, policy_version 868131 (0.0009) [2023-12-26 21:40:58,473][105620] Updated weights for policy 1, policy_version 868141 (0.0010) [2023-12-26 21:40:58,539][105620] Updated weights for policy 1, policy_version 868151 (0.0011) [2023-12-26 21:40:59,137][105692] Updated weights for policy 0, policy_version 868295 (0.0010) [2023-12-26 21:40:59,181][105692] Updated weights for policy 0, policy_version 868305 (0.0010) [2023-12-26 21:40:59,230][105692] Updated weights for policy 0, policy_version 868315 (0.0010) [2023-12-26 21:40:59,265][105620] Updated weights for policy 1, policy_version 868161 (0.0007) [2023-12-26 21:40:59,318][105620] Updated weights for policy 1, policy_version 868171 (0.0010) [2023-12-26 21:40:59,385][105620] Updated weights for policy 1, policy_version 868181 (0.0009) [2023-12-26 21:40:59,982][105692] Updated weights for policy 0, policy_version 868325 (0.0009) [2023-12-26 21:41:00,033][105692] Updated weights for policy 0, policy_version 868335 (0.0010) [2023-12-26 21:41:00,094][105692] Updated weights for policy 0, policy_version 868345 (0.0011) [2023-12-26 21:41:00,145][105620] Updated weights for policy 1, policy_version 868191 (0.0010) [2023-12-26 21:41:00,200][105620] Updated weights for policy 1, policy_version 868201 (0.0010) [2023-12-26 21:41:00,257][105620] Updated weights for policy 1, policy_version 868211 (0.0010) [2023-12-26 21:41:00,797][105692] Updated weights for policy 0, policy_version 868355 (0.0010) [2023-12-26 21:41:00,844][105692] Updated weights for policy 0, policy_version 868365 (0.0010) [2023-12-26 21:41:00,895][105692] Updated weights for policy 0, policy_version 868375 (0.0010) [2023-12-26 21:41:01,001][105620] Updated weights for policy 1, policy_version 868221 (0.0009) [2023-12-26 21:41:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 444628992. Throughput: 0: 9866.3, 1: 9735.8. Samples: 444598896. Policy #0 lag: (min: 21.0, avg: 43.4, max: 48.0) [2023-12-26 21:41:01,062][104569] Avg episode reward: [(0, '9084.784'), (1, '8918.369')] [2023-12-26 21:41:01,063][105620] Updated weights for policy 1, policy_version 868231 (0.0008) [2023-12-26 21:41:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000868384_222339072.pth... [2023-12-26 21:41:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000867232_222044160.pth [2023-12-26 21:41:01,125][105620] Updated weights for policy 1, policy_version 868241 (0.0010) [2023-12-26 21:41:01,168][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000868248_222298112.pth... [2023-12-26 21:41:01,172][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000867128_222011392.pth [2023-12-26 21:41:01,676][105692] Updated weights for policy 0, policy_version 868385 (0.0010) [2023-12-26 21:41:01,743][105692] Updated weights for policy 0, policy_version 868395 (0.0009) [2023-12-26 21:41:01,800][105620] Updated weights for policy 1, policy_version 868251 (0.0009) [2023-12-26 21:41:01,803][105692] Updated weights for policy 0, policy_version 868405 (0.0009) [2023-12-26 21:41:01,858][105620] Updated weights for policy 1, policy_version 868261 (0.0008) [2023-12-26 21:41:01,861][105692] Updated weights for policy 0, policy_version 868415 (0.0006) [2023-12-26 21:41:01,917][105620] Updated weights for policy 1, policy_version 868271 (0.0008) [2023-12-26 21:41:02,580][105620] Updated weights for policy 1, policy_version 868281 (0.0009) [2023-12-26 21:41:02,639][105620] Updated weights for policy 1, policy_version 868291 (0.0011) [2023-12-26 21:41:02,688][105692] Updated weights for policy 0, policy_version 868425 (0.0006) [2023-12-26 21:41:02,705][105620] Updated weights for policy 1, policy_version 868301 (0.0010) [2023-12-26 21:41:02,751][105692] Updated weights for policy 0, policy_version 868435 (0.0005) [2023-12-26 21:41:02,760][105620] Updated weights for policy 1, policy_version 868311 (0.0010) [2023-12-26 21:41:02,804][105692] Updated weights for policy 0, policy_version 868445 (0.0007) [2023-12-26 21:41:03,445][105620] Updated weights for policy 1, policy_version 868321 (0.0010) [2023-12-26 21:41:03,478][105692] Updated weights for policy 0, policy_version 868455 (0.0008) [2023-12-26 21:41:03,493][105620] Updated weights for policy 1, policy_version 868331 (0.0007) [2023-12-26 21:41:03,528][105692] Updated weights for policy 0, policy_version 868465 (0.0008) [2023-12-26 21:41:03,539][105620] Updated weights for policy 1, policy_version 868341 (0.0005) [2023-12-26 21:41:03,576][105692] Updated weights for policy 0, policy_version 868476 (0.0009) [2023-12-26 21:41:04,214][105620] Updated weights for policy 1, policy_version 868351 (0.0006) [2023-12-26 21:41:04,280][105620] Updated weights for policy 1, policy_version 868361 (0.0009) [2023-12-26 21:41:04,347][105620] Updated weights for policy 1, policy_version 868371 (0.0006) [2023-12-26 21:41:04,374][105692] Updated weights for policy 0, policy_version 868486 (0.0008) [2023-12-26 21:41:04,439][105692] Updated weights for policy 0, policy_version 868496 (0.0008) [2023-12-26 21:41:04,503][105692] Updated weights for policy 0, policy_version 868506 (0.0006) [2023-12-26 21:41:05,044][105620] Updated weights for policy 1, policy_version 868381 (0.0010) [2023-12-26 21:41:05,099][105620] Updated weights for policy 1, policy_version 868391 (0.0010) [2023-12-26 21:41:05,168][105620] Updated weights for policy 1, policy_version 868401 (0.0010) [2023-12-26 21:41:05,171][105692] Updated weights for policy 0, policy_version 868516 (0.0006) [2023-12-26 21:41:05,234][105692] Updated weights for policy 0, policy_version 868526 (0.0007) [2023-12-26 21:41:05,292][105692] Updated weights for policy 0, policy_version 868536 (0.0008) [2023-12-26 21:41:05,828][105620] Updated weights for policy 1, policy_version 868411 (0.0010) [2023-12-26 21:41:05,876][105620] Updated weights for policy 1, policy_version 868421 (0.0010) [2023-12-26 21:41:05,909][105692] Updated weights for policy 0, policy_version 868546 (0.0009) [2023-12-26 21:41:05,928][105620] Updated weights for policy 1, policy_version 868431 (0.0010) [2023-12-26 21:41:05,958][105692] Updated weights for policy 0, policy_version 868556 (0.0007) [2023-12-26 21:41:06,010][105692] Updated weights for policy 0, policy_version 868566 (0.0007) [2023-12-26 21:41:06,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 444735488. Throughput: 0: 9838.3, 1: 9739.3. Samples: 444715276. Policy #0 lag: (min: 21.0, avg: 43.4, max: 48.0) [2023-12-26 21:41:06,062][104569] Avg episode reward: [(0, '8992.878'), (1, '8744.611')] [2023-12-26 21:41:06,062][105692] Updated weights for policy 0, policy_version 868576 (0.0008) [2023-12-26 21:41:06,667][105620] Updated weights for policy 1, policy_version 868441 (0.0010) [2023-12-26 21:41:06,729][105620] Updated weights for policy 1, policy_version 868451 (0.0009) [2023-12-26 21:41:06,799][105620] Updated weights for policy 1, policy_version 868461 (0.0011) [2023-12-26 21:41:06,845][105692] Updated weights for policy 0, policy_version 868586 (0.0006) [2023-12-26 21:41:06,865][105620] Updated weights for policy 1, policy_version 868471 (0.0011) [2023-12-26 21:41:06,906][105692] Updated weights for policy 0, policy_version 868596 (0.0009) [2023-12-26 21:41:06,974][105692] Updated weights for policy 0, policy_version 868606 (0.0008) [2023-12-26 21:41:07,566][105620] Updated weights for policy 1, policy_version 868481 (0.0010) [2023-12-26 21:41:07,621][105620] Updated weights for policy 1, policy_version 868491 (0.0010) [2023-12-26 21:41:07,683][105620] Updated weights for policy 1, policy_version 868501 (0.0010) [2023-12-26 21:41:07,726][105692] Updated weights for policy 0, policy_version 868616 (0.0007) [2023-12-26 21:41:07,777][105692] Updated weights for policy 0, policy_version 868626 (0.0008) [2023-12-26 21:41:07,825][105692] Updated weights for policy 0, policy_version 868636 (0.0008) [2023-12-26 21:41:08,366][105620] Updated weights for policy 1, policy_version 868511 (0.0009) [2023-12-26 21:41:08,418][105620] Updated weights for policy 1, policy_version 868521 (0.0006) [2023-12-26 21:41:08,476][105620] Updated weights for policy 1, policy_version 868531 (0.0005) [2023-12-26 21:41:08,570][105692] Updated weights for policy 0, policy_version 868646 (0.0009) [2023-12-26 21:41:08,627][105692] Updated weights for policy 0, policy_version 868656 (0.0010) [2023-12-26 21:41:08,686][105692] Updated weights for policy 0, policy_version 868666 (0.0009) [2023-12-26 21:41:09,135][105620] Updated weights for policy 1, policy_version 868541 (0.0007) [2023-12-26 21:41:09,201][105620] Updated weights for policy 1, policy_version 868551 (0.0008) [2023-12-26 21:41:09,268][105620] Updated weights for policy 1, policy_version 868561 (0.0010) [2023-12-26 21:41:09,492][105692] Updated weights for policy 0, policy_version 868676 (0.0008) [2023-12-26 21:41:09,555][105692] Updated weights for policy 0, policy_version 868686 (0.0009) [2023-12-26 21:41:09,615][105692] Updated weights for policy 0, policy_version 868696 (0.0009) [2023-12-26 21:41:10,111][105620] Updated weights for policy 1, policy_version 868571 (0.0010) [2023-12-26 21:41:10,180][105620] Updated weights for policy 1, policy_version 868581 (0.0010) [2023-12-26 21:41:10,252][105620] Updated weights for policy 1, policy_version 868591 (0.0009) [2023-12-26 21:41:10,257][105692] Updated weights for policy 0, policy_version 868706 (0.0006) [2023-12-26 21:41:10,322][105692] Updated weights for policy 0, policy_version 868716 (0.0007) [2023-12-26 21:41:10,375][105692] Updated weights for policy 0, policy_version 868726 (0.0008) [2023-12-26 21:41:10,438][105692] Updated weights for policy 0, policy_version 868736 (0.0008) [2023-12-26 21:41:10,997][105692] Updated weights for policy 0, policy_version 868746 (0.0009) [2023-12-26 21:41:11,011][105620] Updated weights for policy 1, policy_version 868601 (0.0008) [2023-12-26 21:41:11,061][105692] Updated weights for policy 0, policy_version 868756 (0.0008) [2023-12-26 21:41:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 444817408. Throughput: 0: 9936.4, 1: 9660.5. Samples: 444831540. Policy #0 lag: (min: 21.0, avg: 43.4, max: 48.0) [2023-12-26 21:41:11,062][104569] Avg episode reward: [(0, '9170.966'), (1, '9038.521')] [2023-12-26 21:41:11,080][105620] Updated weights for policy 1, policy_version 868611 (0.0008) [2023-12-26 21:41:11,125][105692] Updated weights for policy 0, policy_version 868766 (0.0008) [2023-12-26 21:41:11,145][105620] Updated weights for policy 1, policy_version 868621 (0.0008) [2023-12-26 21:41:11,198][105620] Updated weights for policy 1, policy_version 868631 (0.0009) [2023-12-26 21:41:11,835][105692] Updated weights for policy 0, policy_version 868776 (0.0007) [2023-12-26 21:41:11,891][105692] Updated weights for policy 0, policy_version 868786 (0.0009) [2023-12-26 21:41:11,944][105692] Updated weights for policy 0, policy_version 868796 (0.0008) [2023-12-26 21:41:11,966][105620] Updated weights for policy 1, policy_version 868641 (0.0009) [2023-12-26 21:41:12,028][105620] Updated weights for policy 1, policy_version 868651 (0.0009) [2023-12-26 21:41:12,083][105620] Updated weights for policy 1, policy_version 868661 (0.0009) [2023-12-26 21:41:12,699][105692] Updated weights for policy 0, policy_version 868806 (0.0008) [2023-12-26 21:41:12,754][105692] Updated weights for policy 0, policy_version 868816 (0.0009) [2023-12-26 21:41:12,813][105692] Updated weights for policy 0, policy_version 868826 (0.0008) [2023-12-26 21:41:12,820][105620] Updated weights for policy 1, policy_version 868671 (0.0008) [2023-12-26 21:41:12,867][105620] Updated weights for policy 1, policy_version 868681 (0.0006) [2023-12-26 21:41:12,915][105620] Updated weights for policy 1, policy_version 868691 (0.0009) [2023-12-26 21:41:13,578][105620] Updated weights for policy 1, policy_version 868701 (0.0009) [2023-12-26 21:41:13,606][105692] Updated weights for policy 0, policy_version 868836 (0.0006) [2023-12-26 21:41:13,637][105620] Updated weights for policy 1, policy_version 868711 (0.0009) [2023-12-26 21:41:13,663][105692] Updated weights for policy 0, policy_version 868846 (0.0007) [2023-12-26 21:41:13,685][105620] Updated weights for policy 1, policy_version 868721 (0.0006) [2023-12-26 21:41:13,716][105692] Updated weights for policy 0, policy_version 868856 (0.0008) [2023-12-26 21:41:14,360][105620] Updated weights for policy 1, policy_version 868731 (0.0006) [2023-12-26 21:41:14,410][105692] Updated weights for policy 0, policy_version 868866 (0.0008) [2023-12-26 21:41:14,413][105620] Updated weights for policy 1, policy_version 868741 (0.0006) [2023-12-26 21:41:14,470][105620] Updated weights for policy 1, policy_version 868751 (0.0006) [2023-12-26 21:41:14,471][105692] Updated weights for policy 0, policy_version 868876 (0.0006) [2023-12-26 21:41:14,536][105692] Updated weights for policy 0, policy_version 868886 (0.0006) [2023-12-26 21:41:14,594][105692] Updated weights for policy 0, policy_version 868896 (0.0008) [2023-12-26 21:41:15,095][105620] Updated weights for policy 1, policy_version 868761 (0.0007) [2023-12-26 21:41:15,149][105620] Updated weights for policy 1, policy_version 868771 (0.0009) [2023-12-26 21:41:15,202][105620] Updated weights for policy 1, policy_version 868781 (0.0009) [2023-12-26 21:41:15,260][105620] Updated weights for policy 1, policy_version 868791 (0.0010) [2023-12-26 21:41:15,359][105692] Updated weights for policy 0, policy_version 868906 (0.0008) [2023-12-26 21:41:15,419][105692] Updated weights for policy 0, policy_version 868916 (0.0009) [2023-12-26 21:41:15,486][105692] Updated weights for policy 0, policy_version 868926 (0.0008) [2023-12-26 21:41:16,038][105620] Updated weights for policy 1, policy_version 868801 (0.0010) [2023-12-26 21:41:16,062][104569] Fps is (10 sec: 18022.1, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 444915712. Throughput: 0: 9829.3, 1: 9586.6. Samples: 444888800. Policy #0 lag: (min: 21.0, avg: 43.4, max: 48.0) [2023-12-26 21:41:16,063][104569] Avg episode reward: [(0, '9079.154'), (1, '9262.321')] [2023-12-26 21:41:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000868928_222478336.pth... [2023-12-26 21:41:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000867808_222191616.pth [2023-12-26 21:41:16,106][105620] Updated weights for policy 1, policy_version 868811 (0.0010) [2023-12-26 21:41:16,167][105620] Updated weights for policy 1, policy_version 868821 (0.0010) [2023-12-26 21:41:16,177][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000868824_222445568.pth... [2023-12-26 21:41:16,180][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000867672_222150656.pth [2023-12-26 21:41:16,238][105692] Updated weights for policy 0, policy_version 868936 (0.0008) [2023-12-26 21:41:16,286][105692] Updated weights for policy 0, policy_version 868946 (0.0008) [2023-12-26 21:41:16,334][105692] Updated weights for policy 0, policy_version 868956 (0.0008) [2023-12-26 21:41:16,884][105620] Updated weights for policy 1, policy_version 868831 (0.0010) [2023-12-26 21:41:16,929][105620] Updated weights for policy 1, policy_version 868841 (0.0010) [2023-12-26 21:41:16,977][105620] Updated weights for policy 1, policy_version 868851 (0.0010) [2023-12-26 21:41:17,116][105692] Updated weights for policy 0, policy_version 868966 (0.0008) [2023-12-26 21:41:17,168][105692] Updated weights for policy 0, policy_version 868976 (0.0008) [2023-12-26 21:41:17,214][105692] Updated weights for policy 0, policy_version 868986 (0.0008) [2023-12-26 21:41:17,739][105620] Updated weights for policy 1, policy_version 868861 (0.0010) [2023-12-26 21:41:17,804][105620] Updated weights for policy 1, policy_version 868871 (0.0010) [2023-12-26 21:41:17,855][105620] Updated weights for policy 1, policy_version 868881 (0.0010) [2023-12-26 21:41:17,991][105692] Updated weights for policy 0, policy_version 868996 (0.0008) [2023-12-26 21:41:18,043][105692] Updated weights for policy 0, policy_version 869006 (0.0008) [2023-12-26 21:41:18,092][105692] Updated weights for policy 0, policy_version 869016 (0.0008) [2023-12-26 21:41:18,605][105620] Updated weights for policy 1, policy_version 868891 (0.0010) [2023-12-26 21:41:18,657][105620] Updated weights for policy 1, policy_version 868901 (0.0010) [2023-12-26 21:41:18,711][105620] Updated weights for policy 1, policy_version 868911 (0.0009) [2023-12-26 21:41:18,894][105692] Updated weights for policy 0, policy_version 869026 (0.0008) [2023-12-26 21:41:18,963][105692] Updated weights for policy 0, policy_version 869036 (0.0009) [2023-12-26 21:41:19,032][105692] Updated weights for policy 0, policy_version 869046 (0.0009) [2023-12-26 21:41:19,091][105692] Updated weights for policy 0, policy_version 869056 (0.0009) [2023-12-26 21:41:19,476][105620] Updated weights for policy 1, policy_version 868921 (0.0009) [2023-12-26 21:41:19,542][105620] Updated weights for policy 1, policy_version 868931 (0.0011) [2023-12-26 21:41:19,591][105620] Updated weights for policy 1, policy_version 868941 (0.0010) [2023-12-26 21:41:19,644][105620] Updated weights for policy 1, policy_version 868951 (0.0010) [2023-12-26 21:41:19,844][105692] Updated weights for policy 0, policy_version 869066 (0.0008) [2023-12-26 21:41:19,902][105692] Updated weights for policy 0, policy_version 869076 (0.0009) [2023-12-26 21:41:19,965][105692] Updated weights for policy 0, policy_version 869086 (0.0009) [2023-12-26 21:41:20,455][105620] Updated weights for policy 1, policy_version 868961 (0.0009) [2023-12-26 21:41:20,511][105620] Updated weights for policy 1, policy_version 868971 (0.0008) [2023-12-26 21:41:20,560][105620] Updated weights for policy 1, policy_version 868981 (0.0008) [2023-12-26 21:41:20,792][105692] Updated weights for policy 0, policy_version 869096 (0.0010) [2023-12-26 21:41:20,856][105692] Updated weights for policy 0, policy_version 869106 (0.0011) [2023-12-26 21:41:20,918][105692] Updated weights for policy 0, policy_version 869116 (0.0009) [2023-12-26 21:41:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 445014016. Throughput: 0: 9599.0, 1: 9639.5. Samples: 445002740. Policy #0 lag: (min: 21.0, avg: 43.4, max: 48.0) [2023-12-26 21:41:21,062][104569] Avg episode reward: [(0, '9079.157'), (1, '9262.671')] [2023-12-26 21:41:21,371][105620] Updated weights for policy 1, policy_version 868991 (0.0009) [2023-12-26 21:41:21,426][105620] Updated weights for policy 1, policy_version 869001 (0.0009) [2023-12-26 21:41:21,485][105620] Updated weights for policy 1, policy_version 869011 (0.0009) [2023-12-26 21:41:21,660][105692] Updated weights for policy 0, policy_version 869126 (0.0009) [2023-12-26 21:41:21,728][105692] Updated weights for policy 0, policy_version 869136 (0.0009) [2023-12-26 21:41:21,801][105692] Updated weights for policy 0, policy_version 869146 (0.0009) [2023-12-26 21:41:22,200][105620] Updated weights for policy 1, policy_version 869021 (0.0009) [2023-12-26 21:41:22,257][105620] Updated weights for policy 1, policy_version 869031 (0.0009) [2023-12-26 21:41:22,323][105620] Updated weights for policy 1, policy_version 869041 (0.0007) [2023-12-26 21:41:22,548][105692] Updated weights for policy 0, policy_version 869156 (0.0009) [2023-12-26 21:41:22,613][105692] Updated weights for policy 0, policy_version 869166 (0.0008) [2023-12-26 21:41:22,673][105692] Updated weights for policy 0, policy_version 869176 (0.0009) [2023-12-26 21:41:23,091][105620] Updated weights for policy 1, policy_version 869051 (0.0008) [2023-12-26 21:41:23,160][105620] Updated weights for policy 1, policy_version 869061 (0.0008) [2023-12-26 21:41:23,214][105620] Updated weights for policy 1, policy_version 869071 (0.0009) [2023-12-26 21:41:23,308][105692] Updated weights for policy 0, policy_version 869186 (0.0008) [2023-12-26 21:41:23,360][105692] Updated weights for policy 0, policy_version 869196 (0.0005) [2023-12-26 21:41:23,412][105692] Updated weights for policy 0, policy_version 869206 (0.0005) [2023-12-26 21:41:23,469][105692] Updated weights for policy 0, policy_version 869216 (0.0005) [2023-12-26 21:41:23,829][105620] Updated weights for policy 1, policy_version 869081 (0.0006) [2023-12-26 21:41:23,890][105620] Updated weights for policy 1, policy_version 869091 (0.0005) [2023-12-26 21:41:23,950][105620] Updated weights for policy 1, policy_version 869101 (0.0005) [2023-12-26 21:41:24,013][105620] Updated weights for policy 1, policy_version 869111 (0.0005) [2023-12-26 21:41:24,044][105692] Updated weights for policy 0, policy_version 869226 (0.0011) [2023-12-26 21:41:24,106][105692] Updated weights for policy 0, policy_version 869236 (0.0011) [2023-12-26 21:41:24,169][105692] Updated weights for policy 0, policy_version 869246 (0.0010) [2023-12-26 21:41:24,613][105620] Updated weights for policy 1, policy_version 869121 (0.0005) [2023-12-26 21:41:24,665][105620] Updated weights for policy 1, policy_version 869131 (0.0005) [2023-12-26 21:41:24,729][105620] Updated weights for policy 1, policy_version 869141 (0.0005) [2023-12-26 21:41:24,880][105692] Updated weights for policy 0, policy_version 869256 (0.0010) [2023-12-26 21:41:24,939][105692] Updated weights for policy 0, policy_version 869266 (0.0010) [2023-12-26 21:41:25,008][105692] Updated weights for policy 0, policy_version 869276 (0.0011) [2023-12-26 21:41:25,324][105620] Updated weights for policy 1, policy_version 869151 (0.0009) [2023-12-26 21:41:25,379][105620] Updated weights for policy 1, policy_version 869161 (0.0010) [2023-12-26 21:41:25,438][105620] Updated weights for policy 1, policy_version 869171 (0.0011) [2023-12-26 21:41:25,741][105692] Updated weights for policy 0, policy_version 869286 (0.0010) [2023-12-26 21:41:25,798][105692] Updated weights for policy 0, policy_version 869296 (0.0008) [2023-12-26 21:41:25,852][105692] Updated weights for policy 0, policy_version 869306 (0.0010) [2023-12-26 21:41:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 445112320. Throughput: 0: 9648.9, 1: 9688.0. Samples: 445120040. Policy #0 lag: (min: 21.0, avg: 43.4, max: 48.0) [2023-12-26 21:41:26,063][104569] Avg episode reward: [(0, '9080.162'), (1, '9082.699')] [2023-12-26 21:41:26,192][105620] Updated weights for policy 1, policy_version 869181 (0.0011) [2023-12-26 21:41:26,251][105620] Updated weights for policy 1, policy_version 869191 (0.0010) [2023-12-26 21:41:26,316][105620] Updated weights for policy 1, policy_version 869201 (0.0010) [2023-12-26 21:41:26,524][105692] Updated weights for policy 0, policy_version 869316 (0.0010) [2023-12-26 21:41:26,569][105692] Updated weights for policy 0, policy_version 869326 (0.0010) [2023-12-26 21:41:26,614][105692] Updated weights for policy 0, policy_version 869336 (0.0010) [2023-12-26 21:41:26,943][105620] Updated weights for policy 1, policy_version 869211 (0.0009) [2023-12-26 21:41:27,004][105620] Updated weights for policy 1, policy_version 869221 (0.0005) [2023-12-26 21:41:27,065][105620] Updated weights for policy 1, policy_version 869231 (0.0005) [2023-12-26 21:41:27,379][105692] Updated weights for policy 0, policy_version 869346 (0.0010) [2023-12-26 21:41:27,431][105692] Updated weights for policy 0, policy_version 869356 (0.0008) [2023-12-26 21:41:27,489][105692] Updated weights for policy 0, policy_version 869366 (0.0008) [2023-12-26 21:41:27,553][105692] Updated weights for policy 0, policy_version 869376 (0.0009) [2023-12-26 21:41:27,611][105620] Updated weights for policy 1, policy_version 869241 (0.0005) [2023-12-26 21:41:27,670][105620] Updated weights for policy 1, policy_version 869251 (0.0005) [2023-12-26 21:41:27,723][105620] Updated weights for policy 1, policy_version 869261 (0.0006) [2023-12-26 21:41:27,778][105620] Updated weights for policy 1, policy_version 869271 (0.0010) [2023-12-26 21:41:28,283][105692] Updated weights for policy 0, policy_version 869386 (0.0008) [2023-12-26 21:41:28,332][105692] Updated weights for policy 0, policy_version 869396 (0.0007) [2023-12-26 21:41:28,400][105692] Updated weights for policy 0, policy_version 869406 (0.0007) [2023-12-26 21:41:28,407][105620] Updated weights for policy 1, policy_version 869281 (0.0010) [2023-12-26 21:41:28,460][105620] Updated weights for policy 1, policy_version 869291 (0.0009) [2023-12-26 21:41:28,516][105620] Updated weights for policy 1, policy_version 869301 (0.0011) [2023-12-26 21:41:29,059][105692] Updated weights for policy 0, policy_version 869416 (0.0006) [2023-12-26 21:41:29,116][105692] Updated weights for policy 0, policy_version 869426 (0.0006) [2023-12-26 21:41:29,178][105692] Updated weights for policy 0, policy_version 869436 (0.0009) [2023-12-26 21:41:29,291][105620] Updated weights for policy 1, policy_version 869311 (0.0009) [2023-12-26 21:41:29,355][105620] Updated weights for policy 1, policy_version 869321 (0.0008) [2023-12-26 21:41:29,416][105620] Updated weights for policy 1, policy_version 869331 (0.0009) [2023-12-26 21:41:29,915][105692] Updated weights for policy 0, policy_version 869446 (0.0009) [2023-12-26 21:41:29,976][105692] Updated weights for policy 0, policy_version 869456 (0.0005) [2023-12-26 21:41:30,002][105585] KL-divergence is very high: 115.7559 [2023-12-26 21:41:30,029][105692] Updated weights for policy 0, policy_version 869466 (0.0005) [2023-12-26 21:41:30,047][105585] KL-divergence is very high: 100.3874 [2023-12-26 21:41:30,145][105620] Updated weights for policy 1, policy_version 869341 (0.0007) [2023-12-26 21:41:30,204][105620] Updated weights for policy 1, policy_version 869351 (0.0005) [2023-12-26 21:41:30,255][105620] Updated weights for policy 1, policy_version 869361 (0.0008) [2023-12-26 21:41:30,679][105692] Updated weights for policy 0, policy_version 869476 (0.0006) [2023-12-26 21:41:30,733][105692] Updated weights for policy 0, policy_version 869486 (0.0006) [2023-12-26 21:41:30,787][105692] Updated weights for policy 0, policy_version 869496 (0.0008) [2023-12-26 21:41:30,967][105620] Updated weights for policy 1, policy_version 869371 (0.0008) [2023-12-26 21:41:31,025][105620] Updated weights for policy 1, policy_version 869381 (0.0006) [2023-12-26 21:41:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 445210624. Throughput: 0: 9628.0, 1: 9806.1. Samples: 445180932. Policy #0 lag: (min: 31.0, avg: 31.8, max: 52.0) [2023-12-26 21:41:31,062][104569] Avg episode reward: [(0, '8819.369'), (1, '8993.503')] [2023-12-26 21:41:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000869504_222625792.pth... [2023-12-26 21:41:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000868384_222339072.pth [2023-12-26 21:41:31,121][105620] Updated weights for policy 1, policy_version 869391 (0.0008) [2023-12-26 21:41:31,174][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000869400_222593024.pth... [2023-12-26 21:41:31,182][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000868248_222298112.pth [2023-12-26 21:41:31,591][105692] Updated weights for policy 0, policy_version 869506 (0.0010) [2023-12-26 21:41:31,652][105692] Updated weights for policy 0, policy_version 869516 (0.0009) [2023-12-26 21:41:31,715][105692] Updated weights for policy 0, policy_version 869526 (0.0008) [2023-12-26 21:41:31,784][105692] Updated weights for policy 0, policy_version 869536 (0.0008) [2023-12-26 21:41:31,792][105620] Updated weights for policy 1, policy_version 869401 (0.0007) [2023-12-26 21:41:31,851][105620] Updated weights for policy 1, policy_version 869411 (0.0007) [2023-12-26 21:41:31,909][105620] Updated weights for policy 1, policy_version 869421 (0.0005) [2023-12-26 21:41:31,970][105620] Updated weights for policy 1, policy_version 869431 (0.0005) [2023-12-26 21:41:32,497][105692] Updated weights for policy 0, policy_version 869546 (0.0010) [2023-12-26 21:41:32,556][105692] Updated weights for policy 0, policy_version 869556 (0.0010) [2023-12-26 21:41:32,610][105692] Updated weights for policy 0, policy_version 869566 (0.0010) [2023-12-26 21:41:32,654][105620] Updated weights for policy 1, policy_version 869441 (0.0009) [2023-12-26 21:41:32,711][105620] Updated weights for policy 1, policy_version 869451 (0.0008) [2023-12-26 21:41:32,763][105620] Updated weights for policy 1, policy_version 869461 (0.0008) [2023-12-26 21:41:33,257][105692] Updated weights for policy 0, policy_version 869576 (0.0008) [2023-12-26 21:41:33,309][105692] Updated weights for policy 0, policy_version 869586 (0.0006) [2023-12-26 21:41:33,365][105692] Updated weights for policy 0, policy_version 869597 (0.0007) [2023-12-26 21:41:33,498][105620] Updated weights for policy 1, policy_version 869471 (0.0006) [2023-12-26 21:41:33,551][105620] Updated weights for policy 1, policy_version 869481 (0.0005) [2023-12-26 21:41:33,599][105620] Updated weights for policy 1, policy_version 869491 (0.0005) [2023-12-26 21:41:34,089][105692] Updated weights for policy 0, policy_version 869607 (0.0010) [2023-12-26 21:41:34,114][105620] Updated weights for policy 1, policy_version 869501 (0.0005) [2023-12-26 21:41:34,156][105692] Updated weights for policy 0, policy_version 869617 (0.0009) [2023-12-26 21:41:34,180][105620] Updated weights for policy 1, policy_version 869511 (0.0007) [2023-12-26 21:41:34,217][105692] Updated weights for policy 0, policy_version 869627 (0.0007) [2023-12-26 21:41:34,245][105620] Updated weights for policy 1, policy_version 869521 (0.0008) [2023-12-26 21:41:34,923][105620] Updated weights for policy 1, policy_version 869531 (0.0009) [2023-12-26 21:41:34,944][105692] Updated weights for policy 0, policy_version 869637 (0.0007) [2023-12-26 21:41:34,983][105620] Updated weights for policy 1, policy_version 869541 (0.0010) [2023-12-26 21:41:35,002][105692] Updated weights for policy 0, policy_version 869647 (0.0007) [2023-12-26 21:41:35,039][105620] Updated weights for policy 1, policy_version 869551 (0.0011) [2023-12-26 21:41:35,059][105692] Updated weights for policy 0, policy_version 869657 (0.0007) [2023-12-26 21:41:35,798][105620] Updated weights for policy 1, policy_version 869561 (0.0011) [2023-12-26 21:41:35,819][105692] Updated weights for policy 0, policy_version 869667 (0.0007) [2023-12-26 21:41:35,859][105620] Updated weights for policy 1, policy_version 869571 (0.0010) [2023-12-26 21:41:35,874][105692] Updated weights for policy 0, policy_version 869677 (0.0008) [2023-12-26 21:41:35,911][105620] Updated weights for policy 1, policy_version 869581 (0.0010) [2023-12-26 21:41:35,925][105692] Updated weights for policy 0, policy_version 869687 (0.0007) [2023-12-26 21:41:35,962][105620] Updated weights for policy 1, policy_version 869591 (0.0010) [2023-12-26 21:41:36,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 445317120. Throughput: 0: 9714.4, 1: 9844.6. Samples: 445301488. Policy #0 lag: (min: 31.0, avg: 31.8, max: 52.0) [2023-12-26 21:41:36,062][104569] Avg episode reward: [(0, '9086.157'), (1, '8813.930')] [2023-12-26 21:41:36,611][105692] Updated weights for policy 0, policy_version 869697 (0.0006) [2023-12-26 21:41:36,651][105620] Updated weights for policy 1, policy_version 869601 (0.0011) [2023-12-26 21:41:36,678][105692] Updated weights for policy 0, policy_version 869707 (0.0006) [2023-12-26 21:41:36,711][105620] Updated weights for policy 1, policy_version 869611 (0.0011) [2023-12-26 21:41:36,749][105692] Updated weights for policy 0, policy_version 869717 (0.0006) [2023-12-26 21:41:36,768][105620] Updated weights for policy 1, policy_version 869621 (0.0011) [2023-12-26 21:41:36,814][105692] Updated weights for policy 0, policy_version 869727 (0.0007) [2023-12-26 21:41:37,376][105692] Updated weights for policy 0, policy_version 869737 (0.0005) [2023-12-26 21:41:37,435][105692] Updated weights for policy 0, policy_version 869747 (0.0005) [2023-12-26 21:41:37,494][105692] Updated weights for policy 0, policy_version 869757 (0.0005) [2023-12-26 21:41:37,517][105620] Updated weights for policy 1, policy_version 869631 (0.0009) [2023-12-26 21:41:37,563][105620] Updated weights for policy 1, policy_version 869641 (0.0005) [2023-12-26 21:41:37,610][105620] Updated weights for policy 1, policy_version 869651 (0.0005) [2023-12-26 21:41:37,991][105692] Updated weights for policy 0, policy_version 869767 (0.0005) [2023-12-26 21:41:38,038][105692] Updated weights for policy 0, policy_version 869777 (0.0010) [2023-12-26 21:41:38,090][105692] Updated weights for policy 0, policy_version 869787 (0.0009) [2023-12-26 21:41:38,404][105620] Updated weights for policy 1, policy_version 869661 (0.0007) [2023-12-26 21:41:38,461][105620] Updated weights for policy 1, policy_version 869671 (0.0006) [2023-12-26 21:41:38,520][105620] Updated weights for policy 1, policy_version 869681 (0.0006) [2023-12-26 21:41:38,763][105692] Updated weights for policy 0, policy_version 869797 (0.0008) [2023-12-26 21:41:38,815][105692] Updated weights for policy 0, policy_version 869807 (0.0010) [2023-12-26 21:41:38,869][105692] Updated weights for policy 0, policy_version 869817 (0.0010) [2023-12-26 21:41:39,205][105620] Updated weights for policy 1, policy_version 869691 (0.0008) [2023-12-26 21:41:39,275][105620] Updated weights for policy 1, policy_version 869701 (0.0008) [2023-12-26 21:41:39,338][105620] Updated weights for policy 1, policy_version 869711 (0.0008) [2023-12-26 21:41:39,645][105692] Updated weights for policy 0, policy_version 869827 (0.0010) [2023-12-26 21:41:39,705][105692] Updated weights for policy 0, policy_version 869837 (0.0011) [2023-12-26 21:41:39,755][105692] Updated weights for policy 0, policy_version 869847 (0.0010) [2023-12-26 21:41:40,139][105620] Updated weights for policy 1, policy_version 869721 (0.0009) [2023-12-26 21:41:40,192][105620] Updated weights for policy 1, policy_version 869731 (0.0008) [2023-12-26 21:41:40,245][105620] Updated weights for policy 1, policy_version 869741 (0.0008) [2023-12-26 21:41:40,298][105620] Updated weights for policy 1, policy_version 869751 (0.0008) [2023-12-26 21:41:40,482][105692] Updated weights for policy 0, policy_version 869857 (0.0010) [2023-12-26 21:41:40,551][105692] Updated weights for policy 0, policy_version 869867 (0.0006) [2023-12-26 21:41:40,606][105692] Updated weights for policy 0, policy_version 869877 (0.0005) [2023-12-26 21:41:40,661][105692] Updated weights for policy 0, policy_version 869887 (0.0005) [2023-12-26 21:41:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 445407232. Throughput: 0: 9733.8, 1: 9798.5. Samples: 445418352. Policy #0 lag: (min: 31.0, avg: 31.8, max: 52.0) [2023-12-26 21:41:41,063][104569] Avg episode reward: [(0, '9170.541'), (1, '8994.378')] [2023-12-26 21:41:41,191][105620] Updated weights for policy 1, policy_version 869761 (0.0008) [2023-12-26 21:41:41,255][105620] Updated weights for policy 1, policy_version 869771 (0.0008) [2023-12-26 21:41:41,284][105692] Updated weights for policy 0, policy_version 869897 (0.0009) [2023-12-26 21:41:41,323][105620] Updated weights for policy 1, policy_version 869781 (0.0006) [2023-12-26 21:41:41,352][105692] Updated weights for policy 0, policy_version 869907 (0.0010) [2023-12-26 21:41:41,427][105692] Updated weights for policy 0, policy_version 869917 (0.0009) [2023-12-26 21:41:42,087][105620] Updated weights for policy 1, policy_version 869791 (0.0008) [2023-12-26 21:41:42,113][105692] Updated weights for policy 0, policy_version 869927 (0.0009) [2023-12-26 21:41:42,135][105620] Updated weights for policy 1, policy_version 869801 (0.0005) [2023-12-26 21:41:42,165][105692] Updated weights for policy 0, policy_version 869937 (0.0010) [2023-12-26 21:41:42,196][105620] Updated weights for policy 1, policy_version 869811 (0.0005) [2023-12-26 21:41:42,217][105692] Updated weights for policy 0, policy_version 869947 (0.0010) [2023-12-26 21:41:42,955][105620] Updated weights for policy 1, policy_version 869821 (0.0007) [2023-12-26 21:41:42,989][105692] Updated weights for policy 0, policy_version 869957 (0.0010) [2023-12-26 21:41:43,008][105620] Updated weights for policy 1, policy_version 869831 (0.0008) [2023-12-26 21:41:43,048][105692] Updated weights for policy 0, policy_version 869967 (0.0008) [2023-12-26 21:41:43,064][105620] Updated weights for policy 1, policy_version 869841 (0.0008) [2023-12-26 21:41:43,109][105692] Updated weights for policy 0, policy_version 869977 (0.0005) [2023-12-26 21:41:43,697][105692] Updated weights for policy 0, policy_version 869987 (0.0005) [2023-12-26 21:41:43,753][105692] Updated weights for policy 0, policy_version 869997 (0.0005) [2023-12-26 21:41:43,787][105620] Updated weights for policy 1, policy_version 869851 (0.0007) [2023-12-26 21:41:43,810][105692] Updated weights for policy 0, policy_version 870007 (0.0006) [2023-12-26 21:41:43,844][105620] Updated weights for policy 1, policy_version 869861 (0.0007) [2023-12-26 21:41:43,896][105620] Updated weights for policy 1, policy_version 869871 (0.0007) [2023-12-26 21:41:44,469][105692] Updated weights for policy 0, policy_version 870017 (0.0007) [2023-12-26 21:41:44,525][105692] Updated weights for policy 0, policy_version 870027 (0.0005) [2023-12-26 21:41:44,573][105692] Updated weights for policy 0, policy_version 870037 (0.0005) [2023-12-26 21:41:44,628][105692] Updated weights for policy 0, policy_version 870047 (0.0005) [2023-12-26 21:41:44,715][105620] Updated weights for policy 1, policy_version 869881 (0.0009) [2023-12-26 21:41:44,779][105620] Updated weights for policy 1, policy_version 869891 (0.0007) [2023-12-26 21:41:44,837][105620] Updated weights for policy 1, policy_version 869901 (0.0008) [2023-12-26 21:41:44,894][105620] Updated weights for policy 1, policy_version 869911 (0.0008) [2023-12-26 21:41:45,313][105692] Updated weights for policy 0, policy_version 870057 (0.0010) [2023-12-26 21:41:45,365][105692] Updated weights for policy 0, policy_version 870067 (0.0010) [2023-12-26 21:41:45,421][105692] Updated weights for policy 0, policy_version 870077 (0.0010) [2023-12-26 21:41:45,677][105620] Updated weights for policy 1, policy_version 869921 (0.0008) [2023-12-26 21:41:45,728][105620] Updated weights for policy 1, policy_version 869932 (0.0009) [2023-12-26 21:41:45,777][105620] Updated weights for policy 1, policy_version 869942 (0.0008) [2023-12-26 21:41:46,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 445505536. Throughput: 0: 9732.1, 1: 9759.4. Samples: 445476020. Policy #0 lag: (min: 31.0, avg: 31.8, max: 52.0) [2023-12-26 21:41:46,063][104569] Avg episode reward: [(0, '9351.406'), (1, '9173.381')] [2023-12-26 21:41:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000870080_222773248.pth... [2023-12-26 21:41:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000869944_222732288.pth... [2023-12-26 21:41:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000868928_222478336.pth [2023-12-26 21:41:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000868824_222445568.pth [2023-12-26 21:41:46,073][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_000870080_222773248.pth [2023-12-26 21:41:46,073][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_000869944_222732288.pth [2023-12-26 21:41:46,189][105692] Updated weights for policy 0, policy_version 870087 (0.0010) [2023-12-26 21:41:46,242][105692] Updated weights for policy 0, policy_version 870098 (0.0010) [2023-12-26 21:41:46,302][105692] Updated weights for policy 0, policy_version 870109 (0.0010) [2023-12-26 21:41:46,430][105620] Updated weights for policy 1, policy_version 869952 (0.0006) [2023-12-26 21:41:46,483][105620] Updated weights for policy 1, policy_version 869962 (0.0005) [2023-12-26 21:41:46,530][105620] Updated weights for policy 1, policy_version 869972 (0.0005) [2023-12-26 21:41:46,946][105692] Updated weights for policy 0, policy_version 870119 (0.0008) [2023-12-26 21:41:47,003][105692] Updated weights for policy 0, policy_version 870129 (0.0008) [2023-12-26 21:41:47,075][105692] Updated weights for policy 0, policy_version 870139 (0.0010) [2023-12-26 21:41:47,111][105620] Updated weights for policy 1, policy_version 869982 (0.0006) [2023-12-26 21:41:47,167][105620] Updated weights for policy 1, policy_version 869992 (0.0008) [2023-12-26 21:41:47,222][105620] Updated weights for policy 1, policy_version 870002 (0.0008) [2023-12-26 21:41:47,738][105692] Updated weights for policy 0, policy_version 870149 (0.0010) [2023-12-26 21:41:47,803][105692] Updated weights for policy 0, policy_version 870159 (0.0010) [2023-12-26 21:41:47,847][105620] Updated weights for policy 1, policy_version 870012 (0.0008) [2023-12-26 21:41:47,868][105692] Updated weights for policy 0, policy_version 870169 (0.0010) [2023-12-26 21:41:47,896][105620] Updated weights for policy 1, policy_version 870022 (0.0005) [2023-12-26 21:41:47,950][105620] Updated weights for policy 1, policy_version 870032 (0.0005) [2023-12-26 21:41:48,519][105620] Updated weights for policy 1, policy_version 870042 (0.0005) [2023-12-26 21:41:48,536][105692] Updated weights for policy 0, policy_version 870179 (0.0010) [2023-12-26 21:41:48,572][105620] Updated weights for policy 1, policy_version 870052 (0.0005) [2023-12-26 21:41:48,598][105692] Updated weights for policy 0, policy_version 870189 (0.0008) [2023-12-26 21:41:48,635][105620] Updated weights for policy 1, policy_version 870062 (0.0006) [2023-12-26 21:41:48,659][105692] Updated weights for policy 0, policy_version 870199 (0.0010) [2023-12-26 21:41:48,696][105620] Updated weights for policy 1, policy_version 870072 (0.0006) [2023-12-26 21:41:49,244][105692] Updated weights for policy 0, policy_version 870209 (0.0009) [2023-12-26 21:41:49,303][105692] Updated weights for policy 0, policy_version 870219 (0.0007) [2023-12-26 21:41:49,371][105692] Updated weights for policy 0, policy_version 870229 (0.0007) [2023-12-26 21:41:49,429][105692] Updated weights for policy 0, policy_version 870239 (0.0008) [2023-12-26 21:41:49,452][105620] Updated weights for policy 1, policy_version 870082 (0.0008) [2023-12-26 21:41:49,503][105620] Updated weights for policy 1, policy_version 870092 (0.0009) [2023-12-26 21:41:49,550][105620] Updated weights for policy 1, policy_version 870102 (0.0008) [2023-12-26 21:41:50,148][105692] Updated weights for policy 0, policy_version 870249 (0.0009) [2023-12-26 21:41:50,211][105692] Updated weights for policy 0, policy_version 870259 (0.0009) [2023-12-26 21:41:50,269][105692] Updated weights for policy 0, policy_version 870269 (0.0009) [2023-12-26 21:41:50,331][105620] Updated weights for policy 1, policy_version 870112 (0.0006) [2023-12-26 21:41:50,393][105620] Updated weights for policy 1, policy_version 870122 (0.0009) [2023-12-26 21:41:50,447][105620] Updated weights for policy 1, policy_version 870132 (0.0009) [2023-12-26 21:41:51,000][105692] Updated weights for policy 0, policy_version 870279 (0.0010) [2023-12-26 21:41:51,060][105692] Updated weights for policy 0, policy_version 870289 (0.0008) [2023-12-26 21:41:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 445603840. Throughput: 0: 9843.1, 1: 9777.9. Samples: 445598224. Policy #0 lag: (min: 31.0, avg: 31.8, max: 52.0) [2023-12-26 21:41:51,063][104569] Avg episode reward: [(0, '9261.832'), (1, '8990.390')] [2023-12-26 21:41:51,118][105692] Updated weights for policy 0, policy_version 870299 (0.0009) [2023-12-26 21:41:51,190][105620] Updated weights for policy 1, policy_version 870142 (0.0009) [2023-12-26 21:41:51,238][105620] Updated weights for policy 1, policy_version 870152 (0.0009) [2023-12-26 21:41:51,305][105620] Updated weights for policy 1, policy_version 870162 (0.0010) [2023-12-26 21:41:51,872][105692] Updated weights for policy 0, policy_version 870309 (0.0009) [2023-12-26 21:41:51,935][105692] Updated weights for policy 0, policy_version 870319 (0.0009) [2023-12-26 21:41:52,003][105692] Updated weights for policy 0, policy_version 870329 (0.0009) [2023-12-26 21:41:52,088][105620] Updated weights for policy 1, policy_version 870172 (0.0009) [2023-12-26 21:41:52,147][105620] Updated weights for policy 1, policy_version 870182 (0.0007) [2023-12-26 21:41:52,212][105620] Updated weights for policy 1, policy_version 870192 (0.0007) [2023-12-26 21:41:52,794][105692] Updated weights for policy 0, policy_version 870339 (0.0008) [2023-12-26 21:41:52,860][105692] Updated weights for policy 0, policy_version 870349 (0.0008) [2023-12-26 21:41:52,918][105620] Updated weights for policy 1, policy_version 870202 (0.0009) [2023-12-26 21:41:52,921][105692] Updated weights for policy 0, policy_version 870359 (0.0007) [2023-12-26 21:41:52,969][105620] Updated weights for policy 1, policy_version 870212 (0.0007) [2023-12-26 21:41:53,025][105620] Updated weights for policy 1, policy_version 870222 (0.0008) [2023-12-26 21:41:53,072][105620] Updated weights for policy 1, policy_version 870232 (0.0009) [2023-12-26 21:41:53,707][105620] Updated weights for policy 1, policy_version 870242 (0.0010) [2023-12-26 21:41:53,714][105692] Updated weights for policy 0, policy_version 870369 (0.0006) [2023-12-26 21:41:53,759][105620] Updated weights for policy 1, policy_version 870252 (0.0006) [2023-12-26 21:41:53,774][105692] Updated weights for policy 0, policy_version 870379 (0.0011) [2023-12-26 21:41:53,818][105620] Updated weights for policy 1, policy_version 870262 (0.0005) [2023-12-26 21:41:53,828][105692] Updated weights for policy 0, policy_version 870389 (0.0010) [2023-12-26 21:41:53,877][105692] Updated weights for policy 0, policy_version 870399 (0.0009) [2023-12-26 21:41:54,491][105620] Updated weights for policy 1, policy_version 870272 (0.0006) [2023-12-26 21:41:54,520][105692] Updated weights for policy 0, policy_version 870409 (0.0008) [2023-12-26 21:41:54,555][105620] Updated weights for policy 1, policy_version 870282 (0.0006) [2023-12-26 21:41:54,577][105692] Updated weights for policy 0, policy_version 870419 (0.0007) [2023-12-26 21:41:54,617][105620] Updated weights for policy 1, policy_version 870292 (0.0007) [2023-12-26 21:41:54,630][105692] Updated weights for policy 0, policy_version 870429 (0.0007) [2023-12-26 21:41:55,293][105620] Updated weights for policy 1, policy_version 870302 (0.0009) [2023-12-26 21:41:55,311][105692] Updated weights for policy 0, policy_version 870439 (0.0005) [2023-12-26 21:41:55,349][105620] Updated weights for policy 1, policy_version 870312 (0.0009) [2023-12-26 21:41:55,368][105692] Updated weights for policy 0, policy_version 870449 (0.0007) [2023-12-26 21:41:55,407][105620] Updated weights for policy 1, policy_version 870322 (0.0007) [2023-12-26 21:41:55,427][105692] Updated weights for policy 0, policy_version 870459 (0.0007) [2023-12-26 21:41:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 445702144. Throughput: 0: 9806.7, 1: 9809.6. Samples: 445714272. Policy #0 lag: (min: 31.0, avg: 31.8, max: 52.0) [2023-12-26 21:41:56,062][104569] Avg episode reward: [(0, '9082.483'), (1, '8988.803')] [2023-12-26 21:41:56,073][105692] Updated weights for policy 0, policy_version 870469 (0.0008) [2023-12-26 21:41:56,128][105692] Updated weights for policy 0, policy_version 870479 (0.0009) [2023-12-26 21:41:56,182][105692] Updated weights for policy 0, policy_version 870489 (0.0010) [2023-12-26 21:41:56,187][105620] Updated weights for policy 1, policy_version 870332 (0.0009) [2023-12-26 21:41:56,240][105620] Updated weights for policy 1, policy_version 870342 (0.0007) [2023-12-26 21:41:56,299][105620] Updated weights for policy 1, policy_version 870352 (0.0007) [2023-12-26 21:41:56,832][105692] Updated weights for policy 0, policy_version 870499 (0.0008) [2023-12-26 21:41:56,891][105692] Updated weights for policy 0, policy_version 870509 (0.0005) [2023-12-26 21:41:56,953][105692] Updated weights for policy 0, policy_version 870519 (0.0008) [2023-12-26 21:41:56,978][105620] Updated weights for policy 1, policy_version 870362 (0.0007) [2023-12-26 21:41:57,030][105620] Updated weights for policy 1, policy_version 870372 (0.0005) [2023-12-26 21:41:57,082][105620] Updated weights for policy 1, policy_version 870382 (0.0005) [2023-12-26 21:41:57,140][105620] Updated weights for policy 1, policy_version 870392 (0.0005) [2023-12-26 21:41:57,578][105692] Updated weights for policy 0, policy_version 870529 (0.0010) [2023-12-26 21:41:57,624][105692] Updated weights for policy 0, policy_version 870539 (0.0010) [2023-12-26 21:41:57,668][105692] Updated weights for policy 0, policy_version 870549 (0.0010) [2023-12-26 21:41:57,729][105692] Updated weights for policy 0, policy_version 870559 (0.0010) [2023-12-26 21:41:57,798][105620] Updated weights for policy 1, policy_version 870402 (0.0006) [2023-12-26 21:41:57,846][105620] Updated weights for policy 1, policy_version 870412 (0.0005) [2023-12-26 21:41:57,897][105620] Updated weights for policy 1, policy_version 870422 (0.0005) [2023-12-26 21:41:58,398][105692] Updated weights for policy 0, policy_version 870569 (0.0009) [2023-12-26 21:41:58,460][105692] Updated weights for policy 0, policy_version 870579 (0.0010) [2023-12-26 21:41:58,520][105692] Updated weights for policy 0, policy_version 870589 (0.0009) [2023-12-26 21:41:58,553][105620] Updated weights for policy 1, policy_version 870432 (0.0008) [2023-12-26 21:41:58,617][105620] Updated weights for policy 1, policy_version 870442 (0.0009) [2023-12-26 21:41:58,678][105620] Updated weights for policy 1, policy_version 870452 (0.0009) [2023-12-26 21:41:59,320][105692] Updated weights for policy 0, policy_version 870599 (0.0008) [2023-12-26 21:41:59,380][105692] Updated weights for policy 0, policy_version 870609 (0.0009) [2023-12-26 21:41:59,435][105620] Updated weights for policy 1, policy_version 870462 (0.0007) [2023-12-26 21:41:59,440][105692] Updated weights for policy 0, policy_version 870619 (0.0009) [2023-12-26 21:41:59,488][105620] Updated weights for policy 1, policy_version 870472 (0.0009) [2023-12-26 21:41:59,552][105620] Updated weights for policy 1, policy_version 870482 (0.0008) [2023-12-26 21:42:00,256][105692] Updated weights for policy 0, policy_version 870629 (0.0009) [2023-12-26 21:42:00,315][105620] Updated weights for policy 1, policy_version 870492 (0.0007) [2023-12-26 21:42:00,320][105692] Updated weights for policy 0, policy_version 870639 (0.0009) [2023-12-26 21:42:00,368][105620] Updated weights for policy 1, policy_version 870502 (0.0007) [2023-12-26 21:42:00,386][105692] Updated weights for policy 0, policy_version 870649 (0.0009) [2023-12-26 21:42:00,422][105620] Updated weights for policy 1, policy_version 870512 (0.0006) [2023-12-26 21:42:01,026][105620] Updated weights for policy 1, policy_version 870522 (0.0010) [2023-12-26 21:42:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 445800448. Throughput: 0: 9867.2, 1: 9855.5. Samples: 445776320. Policy #0 lag: (min: 31.0, avg: 31.8, max: 52.0) [2023-12-26 21:42:01,063][104569] Avg episode reward: [(0, '9171.866'), (1, '9171.675')] [2023-12-26 21:42:01,087][105620] Updated weights for policy 1, policy_version 870532 (0.0006) [2023-12-26 21:42:01,092][105692] Updated weights for policy 0, policy_version 870659 (0.0009) [2023-12-26 21:42:01,145][105620] Updated weights for policy 1, policy_version 870542 (0.0007) [2023-12-26 21:42:01,156][105692] Updated weights for policy 0, policy_version 870669 (0.0008) [2023-12-26 21:42:01,209][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000870552_222887936.pth... [2023-12-26 21:42:01,211][105620] Updated weights for policy 1, policy_version 870552 (0.0008) [2023-12-26 21:42:01,214][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000869400_222593024.pth [2023-12-26 21:42:01,224][105692] Updated weights for policy 0, policy_version 870679 (0.0007) [2023-12-26 21:42:01,281][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000870688_222928896.pth... [2023-12-26 21:42:01,286][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000869504_222625792.pth [2023-12-26 21:42:01,903][105692] Updated weights for policy 0, policy_version 870689 (0.0006) [2023-12-26 21:42:01,925][105620] Updated weights for policy 1, policy_version 870562 (0.0009) [2023-12-26 21:42:01,965][105692] Updated weights for policy 0, policy_version 870699 (0.0007) [2023-12-26 21:42:01,976][105620] Updated weights for policy 1, policy_version 870572 (0.0006) [2023-12-26 21:42:02,027][105692] Updated weights for policy 0, policy_version 870709 (0.0008) [2023-12-26 21:42:02,029][105620] Updated weights for policy 1, policy_version 870582 (0.0008) [2023-12-26 21:42:02,080][105692] Updated weights for policy 0, policy_version 870719 (0.0008) [2023-12-26 21:42:02,632][105620] Updated weights for policy 1, policy_version 870592 (0.0006) [2023-12-26 21:42:02,681][105620] Updated weights for policy 1, policy_version 870602 (0.0006) [2023-12-26 21:42:02,735][105620] Updated weights for policy 1, policy_version 870612 (0.0008) [2023-12-26 21:42:02,882][105692] Updated weights for policy 0, policy_version 870729 (0.0007) [2023-12-26 21:42:02,942][105692] Updated weights for policy 0, policy_version 870739 (0.0006) [2023-12-26 21:42:03,002][105692] Updated weights for policy 0, policy_version 870749 (0.0005) [2023-12-26 21:42:03,345][105620] Updated weights for policy 1, policy_version 870622 (0.0007) [2023-12-26 21:42:03,405][105620] Updated weights for policy 1, policy_version 870632 (0.0005) [2023-12-26 21:42:03,463][105620] Updated weights for policy 1, policy_version 870642 (0.0005) [2023-12-26 21:42:03,599][105692] Updated weights for policy 0, policy_version 870760 (0.0009) [2023-12-26 21:42:03,652][105692] Updated weights for policy 0, policy_version 870772 (0.0010) [2023-12-26 21:42:03,703][105692] Updated weights for policy 0, policy_version 870782 (0.0009) [2023-12-26 21:42:04,066][105620] Updated weights for policy 1, policy_version 870652 (0.0006) [2023-12-26 21:42:04,121][105620] Updated weights for policy 1, policy_version 870662 (0.0009) [2023-12-26 21:42:04,177][105620] Updated weights for policy 1, policy_version 870672 (0.0009) [2023-12-26 21:42:04,512][105692] Updated weights for policy 0, policy_version 870792 (0.0009) [2023-12-26 21:42:04,566][105692] Updated weights for policy 0, policy_version 870802 (0.0009) [2023-12-26 21:42:04,625][105692] Updated weights for policy 0, policy_version 870812 (0.0007) [2023-12-26 21:42:04,955][105620] Updated weights for policy 1, policy_version 870682 (0.0009) [2023-12-26 21:42:05,012][105620] Updated weights for policy 1, policy_version 870692 (0.0009) [2023-12-26 21:42:05,065][105620] Updated weights for policy 1, policy_version 870702 (0.0009) [2023-12-26 21:42:05,120][105620] Updated weights for policy 1, policy_version 870712 (0.0008) [2023-12-26 21:42:05,239][105692] Updated weights for policy 0, policy_version 870822 (0.0005) [2023-12-26 21:42:05,288][105692] Updated weights for policy 0, policy_version 870832 (0.0005) [2023-12-26 21:42:05,337][105692] Updated weights for policy 0, policy_version 870842 (0.0007) [2023-12-26 21:42:05,913][105692] Updated weights for policy 0, policy_version 870852 (0.0008) [2023-12-26 21:42:05,971][105692] Updated weights for policy 0, policy_version 870862 (0.0009) [2023-12-26 21:42:06,000][105620] Updated weights for policy 1, policy_version 870722 (0.0008) [2023-12-26 21:42:06,023][105692] Updated weights for policy 0, policy_version 870872 (0.0010) [2023-12-26 21:42:06,061][105620] Updated weights for policy 1, policy_version 870732 (0.0006) [2023-12-26 21:42:06,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 445906944. Throughput: 0: 9896.2, 1: 9942.1. Samples: 445895464. Policy #0 lag: (min: 31.0, avg: 31.8, max: 52.0) [2023-12-26 21:42:06,062][104569] Avg episode reward: [(0, '9171.896'), (1, '9079.638')] [2023-12-26 21:42:06,120][105620] Updated weights for policy 1, policy_version 870742 (0.0008) [2023-12-26 21:42:06,784][105692] Updated weights for policy 0, policy_version 870882 (0.0010) [2023-12-26 21:42:06,848][105692] Updated weights for policy 0, policy_version 870892 (0.0009) [2023-12-26 21:42:06,875][105620] Updated weights for policy 1, policy_version 870752 (0.0006) [2023-12-26 21:42:06,908][105692] Updated weights for policy 0, policy_version 870902 (0.0011) [2023-12-26 21:42:06,941][105620] Updated weights for policy 1, policy_version 870762 (0.0006) [2023-12-26 21:42:06,972][105692] Updated weights for policy 0, policy_version 870912 (0.0011) [2023-12-26 21:42:07,003][105620] Updated weights for policy 1, policy_version 870772 (0.0005) [2023-12-26 21:42:07,570][105620] Updated weights for policy 1, policy_version 870782 (0.0008) [2023-12-26 21:42:07,631][105620] Updated weights for policy 1, policy_version 870792 (0.0009) [2023-12-26 21:42:07,632][105692] Updated weights for policy 0, policy_version 870922 (0.0005) [2023-12-26 21:42:07,685][105692] Updated weights for policy 0, policy_version 870932 (0.0005) [2023-12-26 21:42:07,688][105620] Updated weights for policy 1, policy_version 870802 (0.0009) [2023-12-26 21:42:07,745][105692] Updated weights for policy 0, policy_version 870942 (0.0005) [2023-12-26 21:42:08,381][105692] Updated weights for policy 0, policy_version 870952 (0.0006) [2023-12-26 21:42:08,439][105692] Updated weights for policy 0, policy_version 870962 (0.0006) [2023-12-26 21:42:08,451][105620] Updated weights for policy 1, policy_version 870812 (0.0008) [2023-12-26 21:42:08,495][105692] Updated weights for policy 0, policy_version 870972 (0.0007) [2023-12-26 21:42:08,516][105620] Updated weights for policy 1, policy_version 870822 (0.0008) [2023-12-26 21:42:08,571][105620] Updated weights for policy 1, policy_version 870832 (0.0008) [2023-12-26 21:42:09,099][105692] Updated weights for policy 0, policy_version 870982 (0.0006) [2023-12-26 21:42:09,160][105692] Updated weights for policy 0, policy_version 870992 (0.0006) [2023-12-26 21:42:09,224][105692] Updated weights for policy 0, policy_version 871002 (0.0009) [2023-12-26 21:42:09,353][105620] Updated weights for policy 1, policy_version 870842 (0.0008) [2023-12-26 21:42:09,423][105620] Updated weights for policy 1, policy_version 870852 (0.0008) [2023-12-26 21:42:09,477][105620] Updated weights for policy 1, policy_version 870862 (0.0008) [2023-12-26 21:42:09,529][105620] Updated weights for policy 1, policy_version 870872 (0.0008) [2023-12-26 21:42:09,987][105692] Updated weights for policy 0, policy_version 871012 (0.0009) [2023-12-26 21:42:10,046][105692] Updated weights for policy 0, policy_version 871022 (0.0010) [2023-12-26 21:42:10,107][105692] Updated weights for policy 0, policy_version 871032 (0.0011) [2023-12-26 21:42:10,330][105620] Updated weights for policy 1, policy_version 870882 (0.0009) [2023-12-26 21:42:10,391][105620] Updated weights for policy 1, policy_version 870892 (0.0008) [2023-12-26 21:42:10,452][105620] Updated weights for policy 1, policy_version 870902 (0.0008) [2023-12-26 21:42:10,823][105692] Updated weights for policy 0, policy_version 871042 (0.0010) [2023-12-26 21:42:10,879][105692] Updated weights for policy 0, policy_version 871052 (0.0010) [2023-12-26 21:42:10,935][105692] Updated weights for policy 0, policy_version 871062 (0.0010) [2023-12-26 21:42:10,995][105692] Updated weights for policy 0, policy_version 871072 (0.0010) [2023-12-26 21:42:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 446005248. Throughput: 0: 9988.0, 1: 9867.2. Samples: 446013524. Policy #0 lag: (min: 31.0, avg: 31.8, max: 52.0) [2023-12-26 21:42:11,063][104569] Avg episode reward: [(0, '9351.140'), (1, '8908.435')] [2023-12-26 21:42:11,081][105620] Updated weights for policy 1, policy_version 870912 (0.0008) [2023-12-26 21:42:11,142][105620] Updated weights for policy 1, policy_version 870922 (0.0007) [2023-12-26 21:42:11,197][105620] Updated weights for policy 1, policy_version 870932 (0.0008) [2023-12-26 21:42:11,809][105692] Updated weights for policy 0, policy_version 871082 (0.0008) [2023-12-26 21:42:11,864][105692] Updated weights for policy 0, policy_version 871092 (0.0009) [2023-12-26 21:42:11,952][105692] Updated weights for policy 0, policy_version 871102 (0.0008) [2023-12-26 21:42:11,980][105620] Updated weights for policy 1, policy_version 870942 (0.0008) [2023-12-26 21:42:12,043][105620] Updated weights for policy 1, policy_version 870952 (0.0009) [2023-12-26 21:42:12,109][105620] Updated weights for policy 1, policy_version 870962 (0.0009) [2023-12-26 21:42:12,741][105692] Updated weights for policy 0, policy_version 871112 (0.0008) [2023-12-26 21:42:12,777][105620] Updated weights for policy 1, policy_version 870972 (0.0006) [2023-12-26 21:42:12,799][105692] Updated weights for policy 0, policy_version 871122 (0.0006) [2023-12-26 21:42:12,846][105620] Updated weights for policy 1, policy_version 870982 (0.0005) [2023-12-26 21:42:12,850][105692] Updated weights for policy 0, policy_version 871132 (0.0009) [2023-12-26 21:42:12,915][105620] Updated weights for policy 1, policy_version 870992 (0.0005) [2023-12-26 21:42:13,557][105692] Updated weights for policy 0, policy_version 871142 (0.0007) [2023-12-26 21:42:13,590][105620] Updated weights for policy 1, policy_version 871002 (0.0007) [2023-12-26 21:42:13,610][105692] Updated weights for policy 0, policy_version 871152 (0.0006) [2023-12-26 21:42:13,650][105620] Updated weights for policy 1, policy_version 871012 (0.0008) [2023-12-26 21:42:13,669][105692] Updated weights for policy 0, policy_version 871162 (0.0007) [2023-12-26 21:42:13,708][105620] Updated weights for policy 1, policy_version 871022 (0.0006) [2023-12-26 21:42:13,764][105620] Updated weights for policy 1, policy_version 871032 (0.0009) [2023-12-26 21:42:14,408][105692] Updated weights for policy 0, policy_version 871172 (0.0009) [2023-12-26 21:42:14,461][105692] Updated weights for policy 0, policy_version 871182 (0.0008) [2023-12-26 21:42:14,500][105620] Updated weights for policy 1, policy_version 871042 (0.0008) [2023-12-26 21:42:14,514][105692] Updated weights for policy 0, policy_version 871192 (0.0006) [2023-12-26 21:42:14,556][105620] Updated weights for policy 1, policy_version 871052 (0.0007) [2023-12-26 21:42:14,601][105620] Updated weights for policy 1, policy_version 871062 (0.0005) [2023-12-26 21:42:15,236][105692] Updated weights for policy 0, policy_version 871202 (0.0008) [2023-12-26 21:42:15,281][105620] Updated weights for policy 1, policy_version 871072 (0.0007) [2023-12-26 21:42:15,288][105692] Updated weights for policy 0, policy_version 871212 (0.0008) [2023-12-26 21:42:15,339][105620] Updated weights for policy 1, policy_version 871082 (0.0008) [2023-12-26 21:42:15,349][105692] Updated weights for policy 0, policy_version 871222 (0.0006) [2023-12-26 21:42:15,399][105620] Updated weights for policy 1, policy_version 871092 (0.0006) [2023-12-26 21:42:15,413][105692] Updated weights for policy 0, policy_version 871232 (0.0008) [2023-12-26 21:42:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 446095360. Throughput: 0: 9960.3, 1: 9800.5. Samples: 446070164. Policy #0 lag: (min: 31.0, avg: 31.8, max: 52.0) [2023-12-26 21:42:16,062][104569] Avg episode reward: [(0, '9087.081'), (1, '8908.557')] [2023-12-26 21:42:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000871232_223068160.pth... [2023-12-26 21:42:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000870080_222773248.pth [2023-12-26 21:42:16,110][105620] Updated weights for policy 1, policy_version 871102 (0.0008) [2023-12-26 21:42:16,166][105620] Updated weights for policy 1, policy_version 871112 (0.0010) [2023-12-26 21:42:16,168][105692] Updated weights for policy 0, policy_version 871242 (0.0006) [2023-12-26 21:42:16,221][105692] Updated weights for policy 0, policy_version 871252 (0.0007) [2023-12-26 21:42:16,222][105620] Updated weights for policy 1, policy_version 871122 (0.0009) [2023-12-26 21:42:16,258][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000871128_223035392.pth... [2023-12-26 21:42:16,262][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000869944_222732288.pth [2023-12-26 21:42:16,272][105692] Updated weights for policy 0, policy_version 871262 (0.0008) [2023-12-26 21:42:16,896][105620] Updated weights for policy 1, policy_version 871132 (0.0010) [2023-12-26 21:42:16,950][105620] Updated weights for policy 1, policy_version 871142 (0.0009) [2023-12-26 21:42:17,003][105620] Updated weights for policy 1, policy_version 871152 (0.0005) [2023-12-26 21:42:17,079][105692] Updated weights for policy 0, policy_version 871272 (0.0006) [2023-12-26 21:42:17,148][105692] Updated weights for policy 0, policy_version 871282 (0.0007) [2023-12-26 21:42:17,197][105692] Updated weights for policy 0, policy_version 871292 (0.0007) [2023-12-26 21:42:17,652][105620] Updated weights for policy 1, policy_version 871162 (0.0006) [2023-12-26 21:42:17,718][105620] Updated weights for policy 1, policy_version 871172 (0.0010) [2023-12-26 21:42:17,789][105620] Updated weights for policy 1, policy_version 871182 (0.0010) [2023-12-26 21:42:17,847][105620] Updated weights for policy 1, policy_version 871192 (0.0010) [2023-12-26 21:42:17,883][105692] Updated weights for policy 0, policy_version 871303 (0.0010) [2023-12-26 21:42:17,928][105692] Updated weights for policy 0, policy_version 871313 (0.0010) [2023-12-26 21:42:17,987][105692] Updated weights for policy 0, policy_version 871323 (0.0005) [2023-12-26 21:42:18,523][105620] Updated weights for policy 1, policy_version 871202 (0.0008) [2023-12-26 21:42:18,579][105620] Updated weights for policy 1, policy_version 871212 (0.0008) [2023-12-26 21:42:18,639][105620] Updated weights for policy 1, policy_version 871222 (0.0008) [2023-12-26 21:42:18,703][105692] Updated weights for policy 0, policy_version 871333 (0.0010) [2023-12-26 21:42:18,769][105692] Updated weights for policy 0, policy_version 871343 (0.0011) [2023-12-26 21:42:18,835][105692] Updated weights for policy 0, policy_version 871353 (0.0011) [2023-12-26 21:42:19,387][105620] Updated weights for policy 1, policy_version 871232 (0.0008) [2023-12-26 21:42:19,450][105620] Updated weights for policy 1, policy_version 871242 (0.0008) [2023-12-26 21:42:19,517][105620] Updated weights for policy 1, policy_version 871252 (0.0008) [2023-12-26 21:42:19,574][105692] Updated weights for policy 0, policy_version 871363 (0.0011) [2023-12-26 21:42:19,626][105692] Updated weights for policy 0, policy_version 871373 (0.0010) [2023-12-26 21:42:19,685][105692] Updated weights for policy 0, policy_version 871383 (0.0010) [2023-12-26 21:42:20,295][105620] Updated weights for policy 1, policy_version 871262 (0.0009) [2023-12-26 21:42:20,353][105620] Updated weights for policy 1, policy_version 871272 (0.0009) [2023-12-26 21:42:20,414][105620] Updated weights for policy 1, policy_version 871282 (0.0009) [2023-12-26 21:42:20,452][105692] Updated weights for policy 0, policy_version 871393 (0.0010) [2023-12-26 21:42:20,503][105692] Updated weights for policy 0, policy_version 871403 (0.0009) [2023-12-26 21:42:20,561][105692] Updated weights for policy 0, policy_version 871413 (0.0009) [2023-12-26 21:42:20,632][105692] Updated weights for policy 0, policy_version 871423 (0.0009) [2023-12-26 21:42:21,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 446193664. Throughput: 0: 9888.9, 1: 9772.7. Samples: 446186260. Policy #0 lag: (min: 31.0, avg: 31.8, max: 52.0) [2023-12-26 21:42:21,063][104569] Avg episode reward: [(0, '9085.804'), (1, '9262.273')] [2023-12-26 21:42:21,171][105620] Updated weights for policy 1, policy_version 871292 (0.0009) [2023-12-26 21:42:21,226][105620] Updated weights for policy 1, policy_version 871302 (0.0009) [2023-12-26 21:42:21,293][105620] Updated weights for policy 1, policy_version 871312 (0.0008) [2023-12-26 21:42:21,473][105692] Updated weights for policy 0, policy_version 871433 (0.0009) [2023-12-26 21:42:21,521][105692] Updated weights for policy 0, policy_version 871443 (0.0009) [2023-12-26 21:42:21,569][105692] Updated weights for policy 0, policy_version 871453 (0.0009) [2023-12-26 21:42:22,055][105620] Updated weights for policy 1, policy_version 871322 (0.0008) [2023-12-26 21:42:22,117][105620] Updated weights for policy 1, policy_version 871332 (0.0008) [2023-12-26 21:42:22,179][105620] Updated weights for policy 1, policy_version 871342 (0.0006) [2023-12-26 21:42:22,238][105620] Updated weights for policy 1, policy_version 871352 (0.0006) [2023-12-26 21:42:22,390][105692] Updated weights for policy 0, policy_version 871463 (0.0010) [2023-12-26 21:42:22,445][105692] Updated weights for policy 0, policy_version 871473 (0.0006) [2023-12-26 21:42:22,497][105692] Updated weights for policy 0, policy_version 871483 (0.0011) [2023-12-26 21:42:22,944][105620] Updated weights for policy 1, policy_version 871362 (0.0010) [2023-12-26 21:42:22,997][105620] Updated weights for policy 1, policy_version 871372 (0.0010) [2023-12-26 21:42:23,057][105620] Updated weights for policy 1, policy_version 871382 (0.0011) [2023-12-26 21:42:23,240][105692] Updated weights for policy 0, policy_version 871493 (0.0011) [2023-12-26 21:42:23,292][105692] Updated weights for policy 0, policy_version 871503 (0.0010) [2023-12-26 21:42:23,361][105692] Updated weights for policy 0, policy_version 871513 (0.0010) [2023-12-26 21:42:23,640][105620] Updated weights for policy 1, policy_version 871392 (0.0006) [2023-12-26 21:42:23,694][105620] Updated weights for policy 1, policy_version 871402 (0.0007) [2023-12-26 21:42:23,742][105620] Updated weights for policy 1, policy_version 871412 (0.0010) [2023-12-26 21:42:24,047][105692] Updated weights for policy 0, policy_version 871523 (0.0010) [2023-12-26 21:42:24,105][105692] Updated weights for policy 0, policy_version 871533 (0.0006) [2023-12-26 21:42:24,165][105692] Updated weights for policy 0, policy_version 871543 (0.0005) [2023-12-26 21:42:24,375][105620] Updated weights for policy 1, policy_version 871422 (0.0010) [2023-12-26 21:42:24,440][105620] Updated weights for policy 1, policy_version 871432 (0.0009) [2023-12-26 21:42:24,493][105620] Updated weights for policy 1, policy_version 871442 (0.0009) [2023-12-26 21:42:24,718][105692] Updated weights for policy 0, policy_version 871553 (0.0006) [2023-12-26 21:42:24,773][105692] Updated weights for policy 0, policy_version 871564 (0.0010) [2023-12-26 21:42:24,828][105692] Updated weights for policy 0, policy_version 871575 (0.0010) [2023-12-26 21:42:25,135][105620] Updated weights for policy 1, policy_version 871452 (0.0009) [2023-12-26 21:42:25,194][105620] Updated weights for policy 1, policy_version 871462 (0.0007) [2023-12-26 21:42:25,254][105620] Updated weights for policy 1, policy_version 871472 (0.0005) [2023-12-26 21:42:25,707][105692] Updated weights for policy 0, policy_version 871585 (0.0010) [2023-12-26 21:42:25,757][105692] Updated weights for policy 0, policy_version 871595 (0.0008) [2023-12-26 21:42:25,811][105692] Updated weights for policy 0, policy_version 871605 (0.0009) [2023-12-26 21:42:25,857][105692] Updated weights for policy 0, policy_version 871615 (0.0010) [2023-12-26 21:42:25,903][105620] Updated weights for policy 1, policy_version 871482 (0.0006) [2023-12-26 21:42:25,952][105620] Updated weights for policy 1, policy_version 871492 (0.0008) [2023-12-26 21:42:26,012][105620] Updated weights for policy 1, policy_version 871502 (0.0010) [2023-12-26 21:42:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.9, 300 sec: 19522.0). Total num frames: 446291968. Throughput: 0: 9773.5, 1: 9882.3. Samples: 446302864. Policy #0 lag: (min: 31.0, avg: 31.8, max: 52.0) [2023-12-26 21:42:26,062][104569] Avg episode reward: [(0, '9080.697'), (1, '9353.185')] [2023-12-26 21:42:26,078][105620] Updated weights for policy 1, policy_version 871512 (0.0010) [2023-12-26 21:42:26,505][105692] Updated weights for policy 0, policy_version 871625 (0.0010) [2023-12-26 21:42:26,556][105692] Updated weights for policy 0, policy_version 871635 (0.0010) [2023-12-26 21:42:26,614][105692] Updated weights for policy 0, policy_version 871645 (0.0010) [2023-12-26 21:42:26,735][105620] Updated weights for policy 1, policy_version 871522 (0.0005) [2023-12-26 21:42:26,799][105620] Updated weights for policy 1, policy_version 871532 (0.0008) [2023-12-26 21:42:26,862][105620] Updated weights for policy 1, policy_version 871542 (0.0011) [2023-12-26 21:42:27,244][105692] Updated weights for policy 0, policy_version 871655 (0.0007) [2023-12-26 21:42:27,288][105692] Updated weights for policy 0, policy_version 871665 (0.0009) [2023-12-26 21:42:27,341][105692] Updated weights for policy 0, policy_version 871675 (0.0006) [2023-12-26 21:42:27,533][105620] Updated weights for policy 1, policy_version 871552 (0.0010) [2023-12-26 21:42:27,594][105620] Updated weights for policy 1, policy_version 871562 (0.0010) [2023-12-26 21:42:27,646][105620] Updated weights for policy 1, policy_version 871572 (0.0010) [2023-12-26 21:42:27,953][105692] Updated weights for policy 0, policy_version 871685 (0.0005) [2023-12-26 21:42:27,994][105692] Updated weights for policy 0, policy_version 871695 (0.0005) [2023-12-26 21:42:28,050][105692] Updated weights for policy 0, policy_version 871705 (0.0009) [2023-12-26 21:42:28,384][105620] Updated weights for policy 1, policy_version 871582 (0.0010) [2023-12-26 21:42:28,439][105620] Updated weights for policy 1, policy_version 871592 (0.0010) [2023-12-26 21:42:28,484][105620] Updated weights for policy 1, policy_version 871602 (0.0010) [2023-12-26 21:42:28,681][105692] Updated weights for policy 0, policy_version 871715 (0.0010) [2023-12-26 21:42:28,747][105692] Updated weights for policy 0, policy_version 871725 (0.0006) [2023-12-26 21:42:28,801][105692] Updated weights for policy 0, policy_version 871735 (0.0005) [2023-12-26 21:42:29,251][105620] Updated weights for policy 1, policy_version 871612 (0.0009) [2023-12-26 21:42:29,306][105620] Updated weights for policy 1, policy_version 871622 (0.0007) [2023-12-26 21:42:29,373][105620] Updated weights for policy 1, policy_version 871632 (0.0008) [2023-12-26 21:42:29,439][105692] Updated weights for policy 0, policy_version 871745 (0.0010) [2023-12-26 21:42:29,505][105692] Updated weights for policy 0, policy_version 871755 (0.0009) [2023-12-26 21:42:29,560][105692] Updated weights for policy 0, policy_version 871765 (0.0006) [2023-12-26 21:42:29,611][105692] Updated weights for policy 0, policy_version 871775 (0.0010) [2023-12-26 21:42:30,107][105620] Updated weights for policy 1, policy_version 871642 (0.0006) [2023-12-26 21:42:30,155][105620] Updated weights for policy 1, policy_version 871652 (0.0008) [2023-12-26 21:42:30,203][105620] Updated weights for policy 1, policy_version 871662 (0.0008) [2023-12-26 21:42:30,257][105620] Updated weights for policy 1, policy_version 871672 (0.0007) [2023-12-26 21:42:30,322][105692] Updated weights for policy 0, policy_version 871785 (0.0009) [2023-12-26 21:42:30,373][105692] Updated weights for policy 0, policy_version 871795 (0.0009) [2023-12-26 21:42:30,434][105692] Updated weights for policy 0, policy_version 871805 (0.0009) [2023-12-26 21:42:31,051][105620] Updated weights for policy 1, policy_version 871682 (0.0009) [2023-12-26 21:42:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 446390272. Throughput: 0: 9834.6, 1: 9924.0. Samples: 446365152. Policy #0 lag: (min: 31.0, avg: 31.8, max: 52.0) [2023-12-26 21:42:31,063][104569] Avg episode reward: [(0, '9083.603'), (1, '9082.388')] [2023-12-26 21:42:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000871808_223215616.pth... [2023-12-26 21:42:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000870688_222928896.pth [2023-12-26 21:42:31,109][105620] Updated weights for policy 1, policy_version 871692 (0.0009) [2023-12-26 21:42:31,141][105692] Updated weights for policy 0, policy_version 871815 (0.0009) [2023-12-26 21:42:31,173][105620] Updated weights for policy 1, policy_version 871702 (0.0007) [2023-12-26 21:42:31,186][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000871704_223182848.pth... [2023-12-26 21:42:31,189][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000870552_222887936.pth [2023-12-26 21:42:31,197][105692] Updated weights for policy 0, policy_version 871825 (0.0007) [2023-12-26 21:42:31,244][105692] Updated weights for policy 0, policy_version 871835 (0.0009) [2023-12-26 21:42:31,887][105692] Updated weights for policy 0, policy_version 871845 (0.0009) [2023-12-26 21:42:31,949][105692] Updated weights for policy 0, policy_version 871855 (0.0009) [2023-12-26 21:42:31,992][105620] Updated weights for policy 1, policy_version 871712 (0.0008) [2023-12-26 21:42:32,010][105692] Updated weights for policy 0, policy_version 871865 (0.0008) [2023-12-26 21:42:32,046][105620] Updated weights for policy 1, policy_version 871722 (0.0008) [2023-12-26 21:42:32,105][105620] Updated weights for policy 1, policy_version 871732 (0.0008) [2023-12-26 21:42:32,751][105692] Updated weights for policy 0, policy_version 871875 (0.0008) [2023-12-26 21:42:32,812][105692] Updated weights for policy 0, policy_version 871885 (0.0010) [2023-12-26 21:42:32,861][105620] Updated weights for policy 1, policy_version 871742 (0.0006) [2023-12-26 21:42:32,863][105692] Updated weights for policy 0, policy_version 871895 (0.0010) [2023-12-26 21:42:32,918][105620] Updated weights for policy 1, policy_version 871752 (0.0007) [2023-12-26 21:42:32,976][105620] Updated weights for policy 1, policy_version 871762 (0.0008) [2023-12-26 21:42:33,521][105692] Updated weights for policy 0, policy_version 871905 (0.0010) [2023-12-26 21:42:33,584][105692] Updated weights for policy 0, policy_version 871915 (0.0010) [2023-12-26 21:42:33,641][105692] Updated weights for policy 0, policy_version 871925 (0.0010) [2023-12-26 21:42:33,704][105585] KL-divergence is very high: 107.1650 [2023-12-26 21:42:33,705][105692] Updated weights for policy 0, policy_version 871935 (0.0010) [2023-12-26 21:42:33,760][105620] Updated weights for policy 1, policy_version 871772 (0.0008) [2023-12-26 21:42:33,811][105620] Updated weights for policy 1, policy_version 871782 (0.0005) [2023-12-26 21:42:33,863][105620] Updated weights for policy 1, policy_version 871792 (0.0005) [2023-12-26 21:42:34,436][105692] Updated weights for policy 0, policy_version 871945 (0.0010) [2023-12-26 21:42:34,466][105620] Updated weights for policy 1, policy_version 871802 (0.0005) [2023-12-26 21:42:34,499][105692] Updated weights for policy 0, policy_version 871955 (0.0009) [2023-12-26 21:42:34,536][105620] Updated weights for policy 1, policy_version 871812 (0.0006) [2023-12-26 21:42:34,553][105692] Updated weights for policy 0, policy_version 871965 (0.0007) [2023-12-26 21:42:34,606][105620] Updated weights for policy 1, policy_version 871822 (0.0009) [2023-12-26 21:42:34,673][105620] Updated weights for policy 1, policy_version 871832 (0.0010) [2023-12-26 21:42:35,249][105620] Updated weights for policy 1, policy_version 871842 (0.0008) [2023-12-26 21:42:35,270][105692] Updated weights for policy 0, policy_version 871975 (0.0006) [2023-12-26 21:42:35,303][105620] Updated weights for policy 1, policy_version 871852 (0.0010) [2023-12-26 21:42:35,314][105692] Updated weights for policy 0, policy_version 871985 (0.0005) [2023-12-26 21:42:35,358][105692] Updated weights for policy 0, policy_version 871995 (0.0005) [2023-12-26 21:42:35,365][105620] Updated weights for policy 1, policy_version 871862 (0.0010) [2023-12-26 21:42:35,897][105620] Updated weights for policy 1, policy_version 871872 (0.0005) [2023-12-26 21:42:35,950][105620] Updated weights for policy 1, policy_version 871882 (0.0009) [2023-12-26 21:42:35,961][105692] Updated weights for policy 0, policy_version 872005 (0.0005) [2023-12-26 21:42:36,001][105620] Updated weights for policy 1, policy_version 871892 (0.0010) [2023-12-26 21:42:36,007][105692] Updated weights for policy 0, policy_version 872015 (0.0006) [2023-12-26 21:42:36,054][105692] Updated weights for policy 0, policy_version 872025 (0.0007) [2023-12-26 21:42:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 446496768. Throughput: 0: 9799.7, 1: 9840.9. Samples: 446482052. Policy #0 lag: (min: 31.0, avg: 31.8, max: 52.0) [2023-12-26 21:42:36,062][104569] Avg episode reward: [(0, '8987.735'), (1, '9174.795')] [2023-12-26 21:42:36,670][105620] Updated weights for policy 1, policy_version 871902 (0.0010) [2023-12-26 21:42:36,734][105620] Updated weights for policy 1, policy_version 871912 (0.0009) [2023-12-26 21:42:36,797][105620] Updated weights for policy 1, policy_version 871922 (0.0009) [2023-12-26 21:42:36,815][105692] Updated weights for policy 0, policy_version 872035 (0.0008) [2023-12-26 21:42:36,866][105692] Updated weights for policy 0, policy_version 872045 (0.0006) [2023-12-26 21:42:36,918][105692] Updated weights for policy 0, policy_version 872055 (0.0005) [2023-12-26 21:42:37,368][105620] Updated weights for policy 1, policy_version 871932 (0.0008) [2023-12-26 21:42:37,424][105620] Updated weights for policy 1, policy_version 871942 (0.0005) [2023-12-26 21:42:37,480][105620] Updated weights for policy 1, policy_version 871952 (0.0005) [2023-12-26 21:42:37,724][105692] Updated weights for policy 0, policy_version 872065 (0.0006) [2023-12-26 21:42:37,781][105692] Updated weights for policy 0, policy_version 872075 (0.0008) [2023-12-26 21:42:37,842][105692] Updated weights for policy 0, policy_version 872085 (0.0008) [2023-12-26 21:42:37,898][105692] Updated weights for policy 0, policy_version 872095 (0.0008) [2023-12-26 21:42:38,175][105620] Updated weights for policy 1, policy_version 871962 (0.0006) [2023-12-26 21:42:38,240][105620] Updated weights for policy 1, policy_version 871972 (0.0010) [2023-12-26 21:42:38,306][105620] Updated weights for policy 1, policy_version 871982 (0.0010) [2023-12-26 21:42:38,372][105620] Updated weights for policy 1, policy_version 871992 (0.0010) [2023-12-26 21:42:38,640][105692] Updated weights for policy 0, policy_version 872105 (0.0009) [2023-12-26 21:42:38,686][105692] Updated weights for policy 0, policy_version 872115 (0.0010) [2023-12-26 21:42:38,735][105692] Updated weights for policy 0, policy_version 872125 (0.0011) [2023-12-26 21:42:39,024][105620] Updated weights for policy 1, policy_version 872002 (0.0007) [2023-12-26 21:42:39,086][105620] Updated weights for policy 1, policy_version 872012 (0.0010) [2023-12-26 21:42:39,145][105620] Updated weights for policy 1, policy_version 872022 (0.0005) [2023-12-26 21:42:39,506][105692] Updated weights for policy 0, policy_version 872135 (0.0009) [2023-12-26 21:42:39,566][105692] Updated weights for policy 0, policy_version 872145 (0.0008) [2023-12-26 21:42:39,621][105692] Updated weights for policy 0, policy_version 872155 (0.0008) [2023-12-26 21:42:39,842][105620] Updated weights for policy 1, policy_version 872032 (0.0009) [2023-12-26 21:42:39,911][105620] Updated weights for policy 1, policy_version 872042 (0.0011) [2023-12-26 21:42:39,983][105620] Updated weights for policy 1, policy_version 872052 (0.0010) [2023-12-26 21:42:40,397][105692] Updated weights for policy 0, policy_version 872165 (0.0008) [2023-12-26 21:42:40,450][105692] Updated weights for policy 0, policy_version 872175 (0.0008) [2023-12-26 21:42:40,453][105585] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000010 [2023-12-26 21:42:40,763][105620] Updated weights for policy 1, policy_version 872062 (0.0007) [2023-12-26 21:42:40,828][105620] Updated weights for policy 1, policy_version 872072 (0.0007) [2023-12-26 21:42:40,900][105620] Updated weights for policy 1, policy_version 872082 (0.0006) [2023-12-26 21:42:41,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19797.4, 300 sec: 19549.8). Total num frames: 446595072. Throughput: 0: 9808.2, 1: 9929.7. Samples: 446602476. Policy #0 lag: (min: 31.0, avg: 31.8, max: 52.0) [2023-12-26 21:42:41,062][104569] Avg episode reward: [(0, '8984.800'), (1, '9080.416')] [2023-12-26 21:42:41,350][105692] Updated weights for policy 0, policy_version 872185 (0.0009) [2023-12-26 21:42:41,413][105692] Updated weights for policy 0, policy_version 872195 (0.0009) [2023-12-26 21:42:41,473][105692] Updated weights for policy 0, policy_version 872205 (0.0005) [2023-12-26 21:42:41,583][105620] Updated weights for policy 1, policy_version 872092 (0.0009) [2023-12-26 21:42:41,653][105620] Updated weights for policy 1, policy_version 872102 (0.0008) [2023-12-26 21:42:41,712][105620] Updated weights for policy 1, policy_version 872112 (0.0006) [2023-12-26 21:42:42,212][105692] Updated weights for policy 0, policy_version 872215 (0.0009) [2023-12-26 21:42:42,271][105692] Updated weights for policy 0, policy_version 872225 (0.0009) [2023-12-26 21:42:42,337][105692] Updated weights for policy 0, policy_version 872235 (0.0009) [2023-12-26 21:42:42,442][105620] Updated weights for policy 1, policy_version 872122 (0.0007) [2023-12-26 21:42:42,490][105620] Updated weights for policy 1, policy_version 872132 (0.0008) [2023-12-26 21:42:42,551][105620] Updated weights for policy 1, policy_version 872142 (0.0005) [2023-12-26 21:42:42,603][105620] Updated weights for policy 1, policy_version 872152 (0.0005) [2023-12-26 21:42:43,076][105692] Updated weights for policy 0, policy_version 872245 (0.0007) [2023-12-26 21:42:43,138][105692] Updated weights for policy 0, policy_version 872255 (0.0009) [2023-12-26 21:42:43,188][105692] Updated weights for policy 0, policy_version 872265 (0.0009) [2023-12-26 21:42:43,240][105620] Updated weights for policy 1, policy_version 872162 (0.0007) [2023-12-26 21:42:43,309][105620] Updated weights for policy 1, policy_version 872172 (0.0009) [2023-12-26 21:42:43,370][105620] Updated weights for policy 1, policy_version 872182 (0.0008) [2023-12-26 21:42:43,970][105692] Updated weights for policy 0, policy_version 872275 (0.0009) [2023-12-26 21:42:44,016][105692] Updated weights for policy 0, policy_version 872285 (0.0008) [2023-12-26 21:42:44,061][105620] Updated weights for policy 1, policy_version 872192 (0.0007) [2023-12-26 21:42:44,067][105692] Updated weights for policy 0, policy_version 872295 (0.0008) [2023-12-26 21:42:44,110][105620] Updated weights for policy 1, policy_version 872202 (0.0007) [2023-12-26 21:42:44,159][105620] Updated weights for policy 1, policy_version 872212 (0.0008) [2023-12-26 21:42:44,858][105692] Updated weights for policy 0, policy_version 872305 (0.0006) [2023-12-26 21:42:44,925][105692] Updated weights for policy 0, policy_version 872315 (0.0008) [2023-12-26 21:42:44,945][105620] Updated weights for policy 1, policy_version 872222 (0.0009) [2023-12-26 21:42:44,985][105692] Updated weights for policy 0, policy_version 872325 (0.0006) [2023-12-26 21:42:45,012][105620] Updated weights for policy 1, policy_version 872232 (0.0011) [2023-12-26 21:42:45,050][105692] Updated weights for policy 0, policy_version 872335 (0.0008) [2023-12-26 21:42:45,074][105620] Updated weights for policy 1, policy_version 872242 (0.0011) [2023-12-26 21:42:45,676][105620] Updated weights for policy 1, policy_version 872252 (0.0008) [2023-12-26 21:42:45,725][105620] Updated weights for policy 1, policy_version 872262 (0.0008) [2023-12-26 21:42:45,774][105620] Updated weights for policy 1, policy_version 872272 (0.0007) [2023-12-26 21:42:45,855][105692] Updated weights for policy 0, policy_version 872345 (0.0010) [2023-12-26 21:42:45,910][105692] Updated weights for policy 0, policy_version 872356 (0.0010) [2023-12-26 21:42:45,967][105692] Updated weights for policy 0, policy_version 872367 (0.0009) [2023-12-26 21:42:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 446693376. Throughput: 0: 9711.0, 1: 9919.3. Samples: 446659684. Policy #0 lag: (min: 31.0, avg: 31.8, max: 52.0) [2023-12-26 21:42:46,063][104569] Avg episode reward: [(0, '8767.175'), (1, '8721.437')] [2023-12-26 21:42:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000872368_223363072.pth... [2023-12-26 21:42:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000872280_223330304.pth... [2023-12-26 21:42:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000871232_223068160.pth [2023-12-26 21:42:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000871128_223035392.pth [2023-12-26 21:42:46,377][105620] Updated weights for policy 1, policy_version 872282 (0.0006) [2023-12-26 21:42:46,437][105620] Updated weights for policy 1, policy_version 872292 (0.0005) [2023-12-26 21:42:46,494][105620] Updated weights for policy 1, policy_version 872302 (0.0005) [2023-12-26 21:42:46,552][105620] Updated weights for policy 1, policy_version 872312 (0.0005) [2023-12-26 21:42:46,684][105692] Updated weights for policy 0, policy_version 872377 (0.0010) [2023-12-26 21:42:46,741][105692] Updated weights for policy 0, policy_version 872387 (0.0005) [2023-12-26 21:42:46,798][105692] Updated weights for policy 0, policy_version 872397 (0.0005) [2023-12-26 21:42:47,136][105620] Updated weights for policy 1, policy_version 872322 (0.0009) [2023-12-26 21:42:47,191][105620] Updated weights for policy 1, policy_version 872332 (0.0009) [2023-12-26 21:42:47,246][105620] Updated weights for policy 1, policy_version 872342 (0.0009) [2023-12-26 21:42:47,471][105692] Updated weights for policy 0, policy_version 872408 (0.0009) [2023-12-26 21:42:47,525][105692] Updated weights for policy 0, policy_version 872419 (0.0010) [2023-12-26 21:42:47,592][105692] Updated weights for policy 0, policy_version 872430 (0.0010) [2023-12-26 21:42:47,866][105620] Updated weights for policy 1, policy_version 872352 (0.0006) [2023-12-26 21:42:47,919][105620] Updated weights for policy 1, policy_version 872362 (0.0005) [2023-12-26 21:42:47,980][105620] Updated weights for policy 1, policy_version 872372 (0.0005) [2023-12-26 21:42:48,458][105692] Updated weights for policy 0, policy_version 872440 (0.0009) [2023-12-26 21:42:48,521][105692] Updated weights for policy 0, policy_version 872450 (0.0008) [2023-12-26 21:42:48,545][105585] KL-divergence is very high: 117.0519 [2023-12-26 21:42:48,565][105585] KL-divergence is very high: 114.6844 [2023-12-26 21:42:48,582][105692] Updated weights for policy 0, policy_version 872460 (0.0007) [2023-12-26 21:42:48,595][105585] KL-divergence is very high: 106.1626 [2023-12-26 21:42:48,598][105620] Updated weights for policy 1, policy_version 872382 (0.0008) [2023-12-26 21:42:48,662][105620] Updated weights for policy 1, policy_version 872392 (0.0006) [2023-12-26 21:42:48,727][105620] Updated weights for policy 1, policy_version 872402 (0.0010) [2023-12-26 21:42:49,338][105585] KL-divergence is very high: 109.7748 [2023-12-26 21:42:49,372][105692] Updated weights for policy 0, policy_version 872470 (0.0009) [2023-12-26 21:42:49,390][105585] KL-divergence is very high: 271.8494 [2023-12-26 21:42:49,418][105620] Updated weights for policy 1, policy_version 872412 (0.0008) [2023-12-26 21:42:49,427][105692] Updated weights for policy 0, policy_version 872480 (0.0009) [2023-12-26 21:42:49,431][105585] KL-divergence is very high: 417.2172 [2023-12-26 21:42:49,472][105585] KL-divergence is very high: 418.1451 [2023-12-26 21:42:49,477][105692] Updated weights for policy 0, policy_version 872490 (0.0008) [2023-12-26 21:42:49,480][105620] Updated weights for policy 1, policy_version 872422 (0.0006) [2023-12-26 21:42:49,542][105620] Updated weights for policy 1, policy_version 872432 (0.0005) [2023-12-26 21:42:50,175][105620] Updated weights for policy 1, policy_version 872442 (0.0007) [2023-12-26 21:42:50,207][105692] Updated weights for policy 0, policy_version 872500 (0.0010) [2023-12-26 21:42:50,239][105620] Updated weights for policy 1, policy_version 872452 (0.0011) [2023-12-26 21:42:50,269][105692] Updated weights for policy 0, policy_version 872510 (0.0011) [2023-12-26 21:42:50,303][105620] Updated weights for policy 1, policy_version 872462 (0.0011) [2023-12-26 21:42:50,329][105692] Updated weights for policy 0, policy_version 872520 (0.0011) [2023-12-26 21:42:50,359][105620] Updated weights for policy 1, policy_version 872472 (0.0006) [2023-12-26 21:42:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 446783488. Throughput: 0: 9652.9, 1: 9948.7. Samples: 446777536. Policy #0 lag: (min: 31.0, avg: 31.8, max: 52.0) [2023-12-26 21:42:51,062][104569] Avg episode reward: [(0, '8332.168'), (1, '8719.847')] [2023-12-26 21:42:51,077][105692] Updated weights for policy 0, policy_version 872530 (0.0010) [2023-12-26 21:42:51,097][105620] Updated weights for policy 1, policy_version 872482 (0.0008) [2023-12-26 21:42:51,134][105692] Updated weights for policy 0, policy_version 872540 (0.0008) [2023-12-26 21:42:51,159][105620] Updated weights for policy 1, policy_version 872492 (0.0007) [2023-12-26 21:42:51,186][105692] Updated weights for policy 0, policy_version 872550 (0.0009) [2023-12-26 21:42:51,209][105620] Updated weights for policy 1, policy_version 872502 (0.0007) [2023-12-26 21:42:51,241][105692] Updated weights for policy 0, policy_version 872560 (0.0006) [2023-12-26 21:42:51,968][105620] Updated weights for policy 1, policy_version 872512 (0.0010) [2023-12-26 21:42:52,014][105586] KL-divergence is very high: 194.6518 [2023-12-26 21:42:52,025][105620] Updated weights for policy 1, policy_version 872522 (0.0007) [2023-12-26 21:42:52,043][105692] Updated weights for policy 0, policy_version 872570 (0.0009) [2023-12-26 21:42:52,061][105586] KL-divergence is very high: 189.4200 [2023-12-26 21:42:52,082][105620] Updated weights for policy 1, policy_version 872532 (0.0007) [2023-12-26 21:42:52,100][105692] Updated weights for policy 0, policy_version 872580 (0.0008) [2023-12-26 21:42:52,150][105692] Updated weights for policy 0, policy_version 872590 (0.0009) [2023-12-26 21:42:52,834][105692] Updated weights for policy 0, policy_version 872600 (0.0006) [2023-12-26 21:42:52,889][105692] Updated weights for policy 0, policy_version 872610 (0.0007) [2023-12-26 21:42:52,891][105620] Updated weights for policy 1, policy_version 872542 (0.0007) [2023-12-26 21:42:52,948][105692] Updated weights for policy 0, policy_version 872620 (0.0007) [2023-12-26 21:42:52,954][105620] Updated weights for policy 1, policy_version 872552 (0.0005) [2023-12-26 21:42:53,024][105620] Updated weights for policy 1, policy_version 872562 (0.0008) [2023-12-26 21:42:53,587][105692] Updated weights for policy 0, policy_version 872630 (0.0008) [2023-12-26 21:42:53,635][105692] Updated weights for policy 0, policy_version 872640 (0.0007) [2023-12-26 21:42:53,686][105692] Updated weights for policy 0, policy_version 872650 (0.0008) [2023-12-26 21:42:53,708][105620] Updated weights for policy 1, policy_version 872572 (0.0009) [2023-12-26 21:42:53,769][105620] Updated weights for policy 1, policy_version 872582 (0.0010) [2023-12-26 21:42:53,815][105620] Updated weights for policy 1, policy_version 872592 (0.0006) [2023-12-26 21:42:54,465][105692] Updated weights for policy 0, policy_version 872660 (0.0007) [2023-12-26 21:42:54,492][105620] Updated weights for policy 1, policy_version 872602 (0.0005) [2023-12-26 21:42:54,520][105692] Updated weights for policy 0, policy_version 872670 (0.0007) [2023-12-26 21:42:54,546][105620] Updated weights for policy 1, policy_version 872612 (0.0007) [2023-12-26 21:42:54,572][105692] Updated weights for policy 0, policy_version 872680 (0.0007) [2023-12-26 21:42:54,605][105620] Updated weights for policy 1, policy_version 872622 (0.0008) [2023-12-26 21:42:54,668][105620] Updated weights for policy 1, policy_version 872632 (0.0008) [2023-12-26 21:42:55,321][105692] Updated weights for policy 0, policy_version 872690 (0.0007) [2023-12-26 21:42:55,370][105692] Updated weights for policy 0, policy_version 872700 (0.0008) [2023-12-26 21:42:55,412][105620] Updated weights for policy 1, policy_version 872642 (0.0007) [2023-12-26 21:42:55,426][105692] Updated weights for policy 0, policy_version 872710 (0.0009) [2023-12-26 21:42:55,468][105620] Updated weights for policy 1, policy_version 872652 (0.0007) [2023-12-26 21:42:55,481][105692] Updated weights for policy 0, policy_version 872720 (0.0005) [2023-12-26 21:42:55,529][105620] Updated weights for policy 1, policy_version 872662 (0.0009) [2023-12-26 21:42:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 446881792. Throughput: 0: 9568.2, 1: 9963.1. Samples: 446892432. Policy #0 lag: (min: 31.0, avg: 31.8, max: 52.0) [2023-12-26 21:42:56,063][104569] Avg episode reward: [(0, '8391.253'), (1, '9078.593')] [2023-12-26 21:42:56,257][105692] Updated weights for policy 0, policy_version 872730 (0.0005) [2023-12-26 21:42:56,287][105620] Updated weights for policy 1, policy_version 872672 (0.0008) [2023-12-26 21:42:56,306][105692] Updated weights for policy 0, policy_version 872740 (0.0006) [2023-12-26 21:42:56,345][105620] Updated weights for policy 1, policy_version 872682 (0.0008) [2023-12-26 21:42:56,355][105692] Updated weights for policy 0, policy_version 872750 (0.0009) [2023-12-26 21:42:56,402][105620] Updated weights for policy 1, policy_version 872692 (0.0008) [2023-12-26 21:42:57,082][105620] Updated weights for policy 1, policy_version 872702 (0.0006) [2023-12-26 21:42:57,087][105692] Updated weights for policy 0, policy_version 872760 (0.0010) [2023-12-26 21:42:57,136][105620] Updated weights for policy 1, policy_version 872712 (0.0005) [2023-12-26 21:42:57,146][105692] Updated weights for policy 0, policy_version 872770 (0.0009) [2023-12-26 21:42:57,192][105692] Updated weights for policy 0, policy_version 872780 (0.0008) [2023-12-26 21:42:57,193][105620] Updated weights for policy 1, policy_version 872722 (0.0005) [2023-12-26 21:42:57,758][105620] Updated weights for policy 1, policy_version 872732 (0.0007) [2023-12-26 21:42:57,767][105692] Updated weights for policy 0, policy_version 872790 (0.0007) [2023-12-26 21:42:57,815][105620] Updated weights for policy 1, policy_version 872742 (0.0005) [2023-12-26 21:42:57,819][105692] Updated weights for policy 0, policy_version 872800 (0.0005) [2023-12-26 21:42:57,866][105620] Updated weights for policy 1, policy_version 872752 (0.0005) [2023-12-26 21:42:57,872][105692] Updated weights for policy 0, policy_version 872810 (0.0006) [2023-12-26 21:42:58,554][105692] Updated weights for policy 0, policy_version 872820 (0.0009) [2023-12-26 21:42:58,573][105620] Updated weights for policy 1, policy_version 872762 (0.0006) [2023-12-26 21:42:58,617][105692] Updated weights for policy 0, policy_version 872830 (0.0009) [2023-12-26 21:42:58,630][105620] Updated weights for policy 1, policy_version 872772 (0.0009) [2023-12-26 21:42:58,677][105692] Updated weights for policy 0, policy_version 872840 (0.0011) [2023-12-26 21:42:58,687][105620] Updated weights for policy 1, policy_version 872782 (0.0006) [2023-12-26 21:42:58,749][105620] Updated weights for policy 1, policy_version 872792 (0.0007) [2023-12-26 21:42:59,481][105692] Updated weights for policy 0, policy_version 872850 (0.0008) [2023-12-26 21:42:59,516][105620] Updated weights for policy 1, policy_version 872802 (0.0006) [2023-12-26 21:42:59,539][105692] Updated weights for policy 0, policy_version 872860 (0.0010) [2023-12-26 21:42:59,567][105620] Updated weights for policy 1, policy_version 872812 (0.0008) [2023-12-26 21:42:59,594][105692] Updated weights for policy 0, policy_version 872870 (0.0010) [2023-12-26 21:42:59,619][105620] Updated weights for policy 1, policy_version 872822 (0.0005) [2023-12-26 21:42:59,652][105692] Updated weights for policy 0, policy_version 872880 (0.0011) [2023-12-26 21:43:00,327][105620] Updated weights for policy 1, policy_version 872832 (0.0008) [2023-12-26 21:43:00,363][105692] Updated weights for policy 0, policy_version 872890 (0.0008) [2023-12-26 21:43:00,390][105620] Updated weights for policy 1, policy_version 872842 (0.0008) [2023-12-26 21:43:00,416][105692] Updated weights for policy 0, policy_version 872900 (0.0007) [2023-12-26 21:43:00,445][105620] Updated weights for policy 1, policy_version 872852 (0.0006) [2023-12-26 21:43:00,476][105692] Updated weights for policy 0, policy_version 872910 (0.0008) [2023-12-26 21:43:01,040][105620] Updated weights for policy 1, policy_version 872862 (0.0007) [2023-12-26 21:43:01,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 446980096. Throughput: 0: 9640.3, 1: 9991.6. Samples: 446953604. Policy #0 lag: (min: 31.0, avg: 31.8, max: 52.0) [2023-12-26 21:43:01,063][104569] Avg episode reward: [(0, '8473.290'), (1, '9262.084')] [2023-12-26 21:43:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000872912_223502336.pth... [2023-12-26 21:43:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000871808_223215616.pth [2023-12-26 21:43:01,103][105620] Updated weights for policy 1, policy_version 872872 (0.0009) [2023-12-26 21:43:01,172][105620] Updated weights for policy 1, policy_version 872882 (0.0008) [2023-12-26 21:43:01,209][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000872888_223485952.pth... [2023-12-26 21:43:01,213][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000871704_223182848.pth [2023-12-26 21:43:01,328][105692] Updated weights for policy 0, policy_version 872920 (0.0010) [2023-12-26 21:43:01,393][105692] Updated weights for policy 0, policy_version 872930 (0.0009) [2023-12-26 21:43:01,456][105692] Updated weights for policy 0, policy_version 872940 (0.0009) [2023-12-26 21:43:01,849][105620] Updated weights for policy 1, policy_version 872892 (0.0009) [2023-12-26 21:43:01,895][105620] Updated weights for policy 1, policy_version 872902 (0.0008) [2023-12-26 21:43:01,944][105620] Updated weights for policy 1, policy_version 872912 (0.0008) [2023-12-26 21:43:02,299][105692] Updated weights for policy 0, policy_version 872950 (0.0008) [2023-12-26 21:43:02,361][105692] Updated weights for policy 0, policy_version 872960 (0.0008) [2023-12-26 21:43:02,361][105585] KL-divergence is very high: 135.9908 [2023-12-26 21:43:02,414][105585] KL-divergence is very high: 109.2421 [2023-12-26 21:43:02,425][105692] Updated weights for policy 0, policy_version 872970 (0.0006) [2023-12-26 21:43:02,568][105620] Updated weights for policy 1, policy_version 872922 (0.0007) [2023-12-26 21:43:02,626][105620] Updated weights for policy 1, policy_version 872932 (0.0005) [2023-12-26 21:43:02,685][105620] Updated weights for policy 1, policy_version 872942 (0.0005) [2023-12-26 21:43:02,744][105620] Updated weights for policy 1, policy_version 872952 (0.0005) [2023-12-26 21:43:03,099][105692] Updated weights for policy 0, policy_version 872980 (0.0005) [2023-12-26 21:43:03,148][105692] Updated weights for policy 0, policy_version 872990 (0.0005) [2023-12-26 21:43:03,202][105692] Updated weights for policy 0, policy_version 873000 (0.0006) [2023-12-26 21:43:03,417][105620] Updated weights for policy 1, policy_version 872962 (0.0007) [2023-12-26 21:43:03,461][105620] Updated weights for policy 1, policy_version 872972 (0.0006) [2023-12-26 21:43:03,512][105620] Updated weights for policy 1, policy_version 872982 (0.0009) [2023-12-26 21:43:03,748][105692] Updated weights for policy 0, policy_version 873010 (0.0006) [2023-12-26 21:43:03,799][105692] Updated weights for policy 0, policy_version 873020 (0.0005) [2023-12-26 21:43:03,850][105692] Updated weights for policy 0, policy_version 873030 (0.0006) [2023-12-26 21:43:03,910][105692] Updated weights for policy 0, policy_version 873040 (0.0007) [2023-12-26 21:43:04,339][105620] Updated weights for policy 1, policy_version 872992 (0.0009) [2023-12-26 21:43:04,398][105620] Updated weights for policy 1, policy_version 873002 (0.0007) [2023-12-26 21:43:04,463][105620] Updated weights for policy 1, policy_version 873012 (0.0009) [2023-12-26 21:43:04,578][105692] Updated weights for policy 0, policy_version 873050 (0.0008) [2023-12-26 21:43:04,636][105692] Updated weights for policy 0, policy_version 873060 (0.0009) [2023-12-26 21:43:04,691][105692] Updated weights for policy 0, policy_version 873070 (0.0009) [2023-12-26 21:43:05,183][105620] Updated weights for policy 1, policy_version 873022 (0.0009) [2023-12-26 21:43:05,237][105620] Updated weights for policy 1, policy_version 873032 (0.0008) [2023-12-26 21:43:05,294][105620] Updated weights for policy 1, policy_version 873042 (0.0008) [2023-12-26 21:43:05,446][105692] Updated weights for policy 0, policy_version 873080 (0.0008) [2023-12-26 21:43:05,501][105692] Updated weights for policy 0, policy_version 873090 (0.0008) [2023-12-26 21:43:05,560][105692] Updated weights for policy 0, policy_version 873100 (0.0009) [2023-12-26 21:43:05,939][105620] Updated weights for policy 1, policy_version 873052 (0.0007) [2023-12-26 21:43:05,985][105620] Updated weights for policy 1, policy_version 873062 (0.0005) [2023-12-26 21:43:06,028][105620] Updated weights for policy 1, policy_version 873072 (0.0005) [2023-12-26 21:43:06,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 447078400. Throughput: 0: 9650.2, 1: 10013.1. Samples: 447071108. Policy #0 lag: (min: 31.0, avg: 31.8, max: 52.0) [2023-12-26 21:43:06,062][104569] Avg episode reward: [(0, '8382.356'), (1, '9008.704')] [2023-12-26 21:43:06,430][105692] Updated weights for policy 0, policy_version 873110 (0.0008) [2023-12-26 21:43:06,488][105692] Updated weights for policy 0, policy_version 873120 (0.0009) [2023-12-26 21:43:06,546][105692] Updated weights for policy 0, policy_version 873130 (0.0010) [2023-12-26 21:43:06,631][105620] Updated weights for policy 1, policy_version 873082 (0.0006) [2023-12-26 21:43:06,673][105586] KL-divergence is very high: 109.6108 [2023-12-26 21:43:06,687][105620] Updated weights for policy 1, policy_version 873092 (0.0006) [2023-12-26 21:43:06,723][105586] KL-divergence is very high: 185.9206 [2023-12-26 21:43:06,747][105620] Updated weights for policy 1, policy_version 873102 (0.0011) [2023-12-26 21:43:06,768][105586] KL-divergence is very high: 189.5432 [2023-12-26 21:43:06,797][105620] Updated weights for policy 1, policy_version 873112 (0.0007) [2023-12-26 21:43:07,373][105692] Updated weights for policy 0, policy_version 873140 (0.0008) [2023-12-26 21:43:07,391][105620] Updated weights for policy 1, policy_version 873122 (0.0010) [2023-12-26 21:43:07,432][105692] Updated weights for policy 0, policy_version 873150 (0.0007) [2023-12-26 21:43:07,449][105620] Updated weights for policy 1, policy_version 873132 (0.0008) [2023-12-26 21:43:07,492][105692] Updated weights for policy 0, policy_version 873160 (0.0006) [2023-12-26 21:43:07,498][105620] Updated weights for policy 1, policy_version 873142 (0.0007) [2023-12-26 21:43:08,172][105620] Updated weights for policy 1, policy_version 873152 (0.0008) [2023-12-26 21:43:08,231][105620] Updated weights for policy 1, policy_version 873162 (0.0006) [2023-12-26 21:43:08,233][105692] Updated weights for policy 0, policy_version 873170 (0.0009) [2023-12-26 21:43:08,280][105620] Updated weights for policy 1, policy_version 873172 (0.0006) [2023-12-26 21:43:08,287][105692] Updated weights for policy 0, policy_version 873180 (0.0008) [2023-12-26 21:43:08,342][105692] Updated weights for policy 0, policy_version 873190 (0.0009) [2023-12-26 21:43:08,393][105692] Updated weights for policy 0, policy_version 873200 (0.0008) [2023-12-26 21:43:08,935][105620] Updated weights for policy 1, policy_version 873182 (0.0008) [2023-12-26 21:43:08,985][105620] Updated weights for policy 1, policy_version 873192 (0.0007) [2023-12-26 21:43:09,044][105620] Updated weights for policy 1, policy_version 873202 (0.0005) [2023-12-26 21:43:09,232][105692] Updated weights for policy 0, policy_version 873210 (0.0009) [2023-12-26 21:43:09,293][105692] Updated weights for policy 0, policy_version 873220 (0.0008) [2023-12-26 21:43:09,321][105585] KL-divergence is very high: 107.7008 [2023-12-26 21:43:09,355][105692] Updated weights for policy 0, policy_version 873230 (0.0009) [2023-12-26 21:43:09,755][105620] Updated weights for policy 1, policy_version 873212 (0.0007) [2023-12-26 21:43:09,827][105620] Updated weights for policy 1, policy_version 873222 (0.0009) [2023-12-26 21:43:09,885][105620] Updated weights for policy 1, policy_version 873232 (0.0009) [2023-12-26 21:43:10,151][105692] Updated weights for policy 0, policy_version 873240 (0.0008) [2023-12-26 21:43:10,211][105692] Updated weights for policy 0, policy_version 873250 (0.0008) [2023-12-26 21:43:10,260][105692] Updated weights for policy 0, policy_version 873260 (0.0008) [2023-12-26 21:43:10,630][105620] Updated weights for policy 1, policy_version 873242 (0.0008) [2023-12-26 21:43:10,687][105620] Updated weights for policy 1, policy_version 873252 (0.0007) [2023-12-26 21:43:10,744][105620] Updated weights for policy 1, policy_version 873262 (0.0005) [2023-12-26 21:43:10,804][105620] Updated weights for policy 1, policy_version 873272 (0.0005) [2023-12-26 21:43:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 447176704. Throughput: 0: 9572.4, 1: 10069.2. Samples: 447186732. Policy #0 lag: (min: 31.0, avg: 31.8, max: 52.0) [2023-12-26 21:43:11,062][104569] Avg episode reward: [(0, '8550.491'), (1, '8739.445')] [2023-12-26 21:43:11,064][105692] Updated weights for policy 0, policy_version 873270 (0.0008) [2023-12-26 21:43:11,130][105692] Updated weights for policy 0, policy_version 873280 (0.0009) [2023-12-26 21:43:11,191][105692] Updated weights for policy 0, policy_version 873290 (0.0009) [2023-12-26 21:43:11,512][105620] Updated weights for policy 1, policy_version 873282 (0.0008) [2023-12-26 21:43:11,572][105620] Updated weights for policy 1, policy_version 873292 (0.0008) [2023-12-26 21:43:11,636][105620] Updated weights for policy 1, policy_version 873302 (0.0007) [2023-12-26 21:43:11,990][105692] Updated weights for policy 0, policy_version 873300 (0.0009) [2023-12-26 21:43:12,037][105692] Updated weights for policy 0, policy_version 873310 (0.0008) [2023-12-26 21:43:12,092][105692] Updated weights for policy 0, policy_version 873320 (0.0009) [2023-12-26 21:43:12,378][105620] Updated weights for policy 1, policy_version 873312 (0.0008) [2023-12-26 21:43:12,433][105620] Updated weights for policy 1, policy_version 873322 (0.0008) [2023-12-26 21:43:12,493][105620] Updated weights for policy 1, policy_version 873332 (0.0008) [2023-12-26 21:43:12,875][105692] Updated weights for policy 0, policy_version 873330 (0.0008) [2023-12-26 21:43:12,944][105692] Updated weights for policy 0, policy_version 873340 (0.0005) [2023-12-26 21:43:13,003][105692] Updated weights for policy 0, policy_version 873350 (0.0005) [2023-12-26 21:43:13,057][105692] Updated weights for policy 0, policy_version 873360 (0.0005) [2023-12-26 21:43:13,220][105620] Updated weights for policy 1, policy_version 873342 (0.0009) [2023-12-26 21:43:13,270][105620] Updated weights for policy 1, policy_version 873353 (0.0006) [2023-12-26 21:43:13,337][105620] Updated weights for policy 1, policy_version 873363 (0.0005) [2023-12-26 21:43:13,574][105692] Updated weights for policy 0, policy_version 873370 (0.0008) [2023-12-26 21:43:13,640][105692] Updated weights for policy 0, policy_version 873380 (0.0007) [2023-12-26 21:43:13,696][105692] Updated weights for policy 0, policy_version 873390 (0.0009) [2023-12-26 21:43:13,998][105620] Updated weights for policy 1, policy_version 873373 (0.0005) [2023-12-26 21:43:14,049][105620] Updated weights for policy 1, policy_version 873383 (0.0005) [2023-12-26 21:43:14,098][105620] Updated weights for policy 1, policy_version 873393 (0.0005) [2023-12-26 21:43:14,446][105692] Updated weights for policy 0, policy_version 873400 (0.0006) [2023-12-26 21:43:14,501][105692] Updated weights for policy 0, policy_version 873410 (0.0008) [2023-12-26 21:43:14,561][105692] Updated weights for policy 0, policy_version 873420 (0.0009) [2023-12-26 21:43:14,805][105620] Updated weights for policy 1, policy_version 873403 (0.0006) [2023-12-26 21:43:14,859][105620] Updated weights for policy 1, policy_version 873413 (0.0010) [2023-12-26 21:43:14,917][105620] Updated weights for policy 1, policy_version 873423 (0.0010) [2023-12-26 21:43:15,188][105692] Updated weights for policy 0, policy_version 873430 (0.0010) [2023-12-26 21:43:15,228][105585] KL-divergence is very high: 112.3123 [2023-12-26 21:43:15,246][105692] Updated weights for policy 0, policy_version 873440 (0.0011) [2023-12-26 21:43:15,274][105585] KL-divergence is very high: 225.5478 [2023-12-26 21:43:15,302][105692] Updated weights for policy 0, policy_version 873450 (0.0011) [2023-12-26 21:43:15,321][105585] KL-divergence is very high: 234.8017 [2023-12-26 21:43:15,655][105620] Updated weights for policy 1, policy_version 873434 (0.0009) [2023-12-26 21:43:15,715][105620] Updated weights for policy 1, policy_version 873444 (0.0005) [2023-12-26 21:43:15,767][105620] Updated weights for policy 1, policy_version 873454 (0.0005) [2023-12-26 21:43:15,818][105620] Updated weights for policy 1, policy_version 873464 (0.0010) [2023-12-26 21:43:15,952][105692] Updated weights for policy 0, policy_version 873460 (0.0008) [2023-12-26 21:43:16,011][105692] Updated weights for policy 0, policy_version 873470 (0.0009) [2023-12-26 21:43:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 447275008. Throughput: 0: 9492.6, 1: 10053.6. Samples: 447244732. Policy #0 lag: (min: 31.0, avg: 31.8, max: 52.0) [2023-12-26 21:43:16,063][104569] Avg episode reward: [(0, '8728.573'), (1, '7092.778')] [2023-12-26 21:43:16,069][105692] Updated weights for policy 0, policy_version 873480 (0.0010) [2023-12-26 21:43:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000873464_223633408.pth... [2023-12-26 21:43:16,095][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000872280_223330304.pth [2023-12-26 21:43:16,120][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000873488_223649792.pth... [2023-12-26 21:43:16,125][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000872368_223363072.pth [2023-12-26 21:43:16,487][105620] Updated weights for policy 1, policy_version 873474 (0.0010) [2023-12-26 21:43:16,533][105620] Updated weights for policy 1, policy_version 873484 (0.0005) [2023-12-26 21:43:16,580][105620] Updated weights for policy 1, policy_version 873494 (0.0005) [2023-12-26 21:43:16,652][105692] Updated weights for policy 0, policy_version 873490 (0.0007) [2023-12-26 21:43:16,703][105692] Updated weights for policy 0, policy_version 873500 (0.0005) [2023-12-26 21:43:16,749][105692] Updated weights for policy 0, policy_version 873510 (0.0005) [2023-12-26 21:43:16,797][105692] Updated weights for policy 0, policy_version 873520 (0.0005) [2023-12-26 21:43:17,128][105620] Updated weights for policy 1, policy_version 873504 (0.0005) [2023-12-26 21:43:17,190][105620] Updated weights for policy 1, policy_version 873514 (0.0005) [2023-12-26 21:43:17,251][105620] Updated weights for policy 1, policy_version 873524 (0.0005) [2023-12-26 21:43:17,311][105692] Updated weights for policy 0, policy_version 873530 (0.0005) [2023-12-26 21:43:17,365][105692] Updated weights for policy 0, policy_version 873540 (0.0005) [2023-12-26 21:43:17,410][105692] Updated weights for policy 0, policy_version 873550 (0.0005) [2023-12-26 21:43:17,744][105620] Updated weights for policy 1, policy_version 873534 (0.0005) [2023-12-26 21:43:17,805][105620] Updated weights for policy 1, policy_version 873544 (0.0008) [2023-12-26 21:43:17,860][105620] Updated weights for policy 1, policy_version 873554 (0.0010) [2023-12-26 21:43:18,007][105692] Updated weights for policy 0, policy_version 873560 (0.0009) [2023-12-26 21:43:18,061][105692] Updated weights for policy 0, policy_version 873570 (0.0008) [2023-12-26 21:43:18,110][105692] Updated weights for policy 0, policy_version 873580 (0.0008) [2023-12-26 21:43:18,578][105620] Updated weights for policy 1, policy_version 873564 (0.0011) [2023-12-26 21:43:18,639][105620] Updated weights for policy 1, policy_version 873574 (0.0009) [2023-12-26 21:43:18,697][105620] Updated weights for policy 1, policy_version 873584 (0.0008) [2023-12-26 21:43:18,884][105692] Updated weights for policy 0, policy_version 873590 (0.0008) [2023-12-26 21:43:18,937][105692] Updated weights for policy 0, policy_version 873600 (0.0007) [2023-12-26 21:43:18,994][105692] Updated weights for policy 0, policy_version 873610 (0.0005) [2023-12-26 21:43:19,448][105620] Updated weights for policy 1, policy_version 873594 (0.0010) [2023-12-26 21:43:19,512][105620] Updated weights for policy 1, policy_version 873604 (0.0008) [2023-12-26 21:43:19,562][105620] Updated weights for policy 1, policy_version 873614 (0.0008) [2023-12-26 21:43:19,630][105620] Updated weights for policy 1, policy_version 873624 (0.0007) [2023-12-26 21:43:19,728][105692] Updated weights for policy 0, policy_version 873620 (0.0007) [2023-12-26 21:43:19,789][105692] Updated weights for policy 0, policy_version 873630 (0.0009) [2023-12-26 21:43:19,859][105692] Updated weights for policy 0, policy_version 873640 (0.0008) [2023-12-26 21:43:20,313][105620] Updated weights for policy 1, policy_version 873634 (0.0007) [2023-12-26 21:43:20,382][105620] Updated weights for policy 1, policy_version 873644 (0.0007) [2023-12-26 21:43:20,443][105620] Updated weights for policy 1, policy_version 873654 (0.0008) [2023-12-26 21:43:20,603][105692] Updated weights for policy 0, policy_version 873650 (0.0006) [2023-12-26 21:43:20,667][105692] Updated weights for policy 0, policy_version 873660 (0.0009) [2023-12-26 21:43:20,731][105692] Updated weights for policy 0, policy_version 873670 (0.0009) [2023-12-26 21:43:20,787][105692] Updated weights for policy 0, policy_version 873680 (0.0009) [2023-12-26 21:43:21,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 447381504. Throughput: 0: 9564.3, 1: 10185.9. Samples: 447370816. Policy #0 lag: (min: 31.0, avg: 31.8, max: 52.0) [2023-12-26 21:43:21,063][104569] Avg episode reward: [(0, '8915.092'), (1, '6844.378')] [2023-12-26 21:43:21,122][105620] Updated weights for policy 1, policy_version 873664 (0.0007) [2023-12-26 21:43:21,187][105620] Updated weights for policy 1, policy_version 873674 (0.0009) [2023-12-26 21:43:21,239][105620] Updated weights for policy 1, policy_version 873684 (0.0009) [2023-12-26 21:43:21,604][105692] Updated weights for policy 0, policy_version 873690 (0.0009) [2023-12-26 21:43:21,677][105692] Updated weights for policy 0, policy_version 873700 (0.0010) [2023-12-26 21:43:21,745][105692] Updated weights for policy 0, policy_version 873710 (0.0009) [2023-12-26 21:43:21,985][105620] Updated weights for policy 1, policy_version 873694 (0.0007) [2023-12-26 21:43:22,042][105620] Updated weights for policy 1, policy_version 873704 (0.0007) [2023-12-26 21:43:22,097][105620] Updated weights for policy 1, policy_version 873714 (0.0010) [2023-12-26 21:43:22,521][105692] Updated weights for policy 0, policy_version 873720 (0.0008) [2023-12-26 21:43:22,573][105692] Updated weights for policy 0, policy_version 873730 (0.0006) [2023-12-26 21:43:22,634][105692] Updated weights for policy 0, policy_version 873740 (0.0008) [2023-12-26 21:43:22,876][105620] Updated weights for policy 1, policy_version 873724 (0.0008) [2023-12-26 21:43:22,936][105620] Updated weights for policy 1, policy_version 873734 (0.0007) [2023-12-26 21:43:22,989][105620] Updated weights for policy 1, policy_version 873744 (0.0010) [2023-12-26 21:43:23,332][105692] Updated weights for policy 0, policy_version 873750 (0.0007) [2023-12-26 21:43:23,385][105692] Updated weights for policy 0, policy_version 873760 (0.0008) [2023-12-26 21:43:23,430][105692] Updated weights for policy 0, policy_version 873770 (0.0008) [2023-12-26 21:43:23,673][105620] Updated weights for policy 1, policy_version 873754 (0.0010) [2023-12-26 21:43:23,727][105620] Updated weights for policy 1, policy_version 873764 (0.0010) [2023-12-26 21:43:23,786][105620] Updated weights for policy 1, policy_version 873774 (0.0012) [2023-12-26 21:43:23,844][105620] Updated weights for policy 1, policy_version 873784 (0.0009) [2023-12-26 21:43:24,115][105692] Updated weights for policy 0, policy_version 873780 (0.0009) [2023-12-26 21:43:24,167][105692] Updated weights for policy 0, policy_version 873790 (0.0009) [2023-12-26 21:43:24,223][105692] Updated weights for policy 0, policy_version 873800 (0.0005) [2023-12-26 21:43:24,578][105620] Updated weights for policy 1, policy_version 873794 (0.0005) [2023-12-26 21:43:24,629][105620] Updated weights for policy 1, policy_version 873804 (0.0005) [2023-12-26 21:43:24,685][105620] Updated weights for policy 1, policy_version 873814 (0.0006) [2023-12-26 21:43:24,878][105692] Updated weights for policy 0, policy_version 873810 (0.0006) [2023-12-26 21:43:24,939][105692] Updated weights for policy 0, policy_version 873820 (0.0006) [2023-12-26 21:43:24,998][105692] Updated weights for policy 0, policy_version 873830 (0.0005) [2023-12-26 21:43:25,055][105692] Updated weights for policy 0, policy_version 873840 (0.0008) [2023-12-26 21:43:25,375][105620] Updated weights for policy 1, policy_version 873824 (0.0008) [2023-12-26 21:43:25,430][105620] Updated weights for policy 1, policy_version 873834 (0.0008) [2023-12-26 21:43:25,488][105620] Updated weights for policy 1, policy_version 873844 (0.0008) [2023-12-26 21:43:25,624][105692] Updated weights for policy 0, policy_version 873850 (0.0010) [2023-12-26 21:43:25,686][105692] Updated weights for policy 0, policy_version 873860 (0.0010) [2023-12-26 21:43:25,745][105692] Updated weights for policy 0, policy_version 873870 (0.0010) [2023-12-26 21:43:26,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 447479808. Throughput: 0: 9605.2, 1: 10077.7. Samples: 447488208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:43:26,063][104569] Avg episode reward: [(0, '8916.300'), (1, '8263.170')] [2023-12-26 21:43:26,255][105620] Updated weights for policy 1, policy_version 873854 (0.0009) [2023-12-26 21:43:26,321][105620] Updated weights for policy 1, policy_version 873864 (0.0010) [2023-12-26 21:43:26,380][105620] Updated weights for policy 1, policy_version 873874 (0.0010) [2023-12-26 21:43:26,517][105692] Updated weights for policy 0, policy_version 873880 (0.0008) [2023-12-26 21:43:26,537][105585] KL-divergence is very high: 422.2250 [2023-12-26 21:43:26,572][105692] Updated weights for policy 0, policy_version 873890 (0.0009) [2023-12-26 21:43:26,582][105585] KL-divergence is very high: 705.5555 [2023-12-26 21:43:26,628][105585] KL-divergence is very high: 736.6534 [2023-12-26 21:43:26,630][105692] Updated weights for policy 0, policy_version 873900 (0.0009) [2023-12-26 21:43:27,004][105620] Updated weights for policy 1, policy_version 873884 (0.0008) [2023-12-26 21:43:27,065][105620] Updated weights for policy 1, policy_version 873894 (0.0010) [2023-12-26 21:43:27,112][105620] Updated weights for policy 1, policy_version 873904 (0.0010) [2023-12-26 21:43:27,494][105585] KL-divergence is very high: 612.7646 [2023-12-26 21:43:27,514][105692] Updated weights for policy 0, policy_version 873910 (0.0008) [2023-12-26 21:43:27,535][105585] KL-divergence is very high: 787.2739 [2023-12-26 21:43:27,569][105692] Updated weights for policy 0, policy_version 873920 (0.0009) [2023-12-26 21:43:27,576][105585] KL-divergence is very high: 726.5724 [2023-12-26 21:43:27,620][105585] KL-divergence is very high: 667.2203 [2023-12-26 21:43:27,621][105692] Updated weights for policy 0, policy_version 873930 (0.0010) [2023-12-26 21:43:27,677][105620] Updated weights for policy 1, policy_version 873914 (0.0009) [2023-12-26 21:43:27,743][105620] Updated weights for policy 1, policy_version 873924 (0.0005) [2023-12-26 21:43:27,801][105620] Updated weights for policy 1, policy_version 873934 (0.0005) [2023-12-26 21:43:27,856][105620] Updated weights for policy 1, policy_version 873944 (0.0006) [2023-12-26 21:43:28,297][105692] Updated weights for policy 0, policy_version 873940 (0.0010) [2023-12-26 21:43:28,356][105692] Updated weights for policy 0, policy_version 873950 (0.0008) [2023-12-26 21:43:28,414][105692] Updated weights for policy 0, policy_version 873960 (0.0008) [2023-12-26 21:43:28,445][105620] Updated weights for policy 1, policy_version 873954 (0.0005) [2023-12-26 21:43:28,508][105620] Updated weights for policy 1, policy_version 873964 (0.0005) [2023-12-26 21:43:28,576][105620] Updated weights for policy 1, policy_version 873974 (0.0009) [2023-12-26 21:43:29,133][105692] Updated weights for policy 0, policy_version 873970 (0.0007) [2023-12-26 21:43:29,187][105692] Updated weights for policy 0, policy_version 873980 (0.0005) [2023-12-26 21:43:29,215][105620] Updated weights for policy 1, policy_version 873984 (0.0011) [2023-12-26 21:43:29,254][105692] Updated weights for policy 0, policy_version 873990 (0.0008) [2023-12-26 21:43:29,286][105620] Updated weights for policy 1, policy_version 873994 (0.0008) [2023-12-26 21:43:29,314][105692] Updated weights for policy 0, policy_version 874000 (0.0008) [2023-12-26 21:43:29,348][105620] Updated weights for policy 1, policy_version 874004 (0.0007) [2023-12-26 21:43:30,004][105585] KL-divergence is very high: 111.9754 [2023-12-26 21:43:30,022][105585] KL-divergence is very high: 180.6039 [2023-12-26 21:43:30,029][105585] KL-divergence is very high: 155.7980 [2023-12-26 21:43:30,034][105692] Updated weights for policy 0, policy_version 874010 (0.0007) [2023-12-26 21:43:30,034][105585] KL-divergence is very high: 134.0419 [2023-12-26 21:43:30,040][105585] KL-divergence is very high: 132.7753 [2023-12-26 21:43:30,044][105620] Updated weights for policy 1, policy_version 874014 (0.0008) [2023-12-26 21:43:30,047][105585] KL-divergence is very high: 120.7826 [2023-12-26 21:43:30,053][105585] KL-divergence is very high: 126.2436 [2023-12-26 21:43:30,071][105585] KL-divergence is very high: 152.6893 [2023-12-26 21:43:30,078][105585] KL-divergence is very high: 117.3177 [2023-12-26 21:43:30,097][105692] Updated weights for policy 0, policy_version 874020 (0.0009) [2023-12-26 21:43:30,099][105620] Updated weights for policy 1, policy_version 874024 (0.0006) [2023-12-26 21:43:30,152][105620] Updated weights for policy 1, policy_version 874034 (0.0008) [2023-12-26 21:43:30,155][105692] Updated weights for policy 0, policy_version 874030 (0.0008) [2023-12-26 21:43:30,766][105692] Updated weights for policy 0, policy_version 874040 (0.0007) [2023-12-26 21:43:30,825][105692] Updated weights for policy 0, policy_version 874050 (0.0009) [2023-12-26 21:43:30,885][105692] Updated weights for policy 0, policy_version 874060 (0.0007) [2023-12-26 21:43:30,997][105620] Updated weights for policy 1, policy_version 874044 (0.0009) [2023-12-26 21:43:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 447578112. Throughput: 0: 9600.7, 1: 10145.7. Samples: 447548272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:43:31,062][104569] Avg episode reward: [(0, '1985.128'), (1, '9268.012')] [2023-12-26 21:43:31,066][105620] Updated weights for policy 1, policy_version 874054 (0.0009) [2023-12-26 21:43:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000874064_223797248.pth... [2023-12-26 21:43:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000872912_223502336.pth [2023-12-26 21:43:31,139][105620] Updated weights for policy 1, policy_version 874064 (0.0009) [2023-12-26 21:43:31,178][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000874072_223789056.pth... [2023-12-26 21:43:31,182][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000872888_223485952.pth [2023-12-26 21:43:31,596][105692] Updated weights for policy 0, policy_version 874070 (0.0007) [2023-12-26 21:43:31,664][105692] Updated weights for policy 0, policy_version 874080 (0.0009) [2023-12-26 21:43:31,730][105692] Updated weights for policy 0, policy_version 874090 (0.0008) [2023-12-26 21:43:31,875][105620] Updated weights for policy 1, policy_version 874074 (0.0009) [2023-12-26 21:43:31,931][105620] Updated weights for policy 1, policy_version 874085 (0.0010) [2023-12-26 21:43:31,986][105620] Updated weights for policy 1, policy_version 874095 (0.0009) [2023-12-26 21:43:32,540][105692] Updated weights for policy 0, policy_version 874100 (0.0008) [2023-12-26 21:43:32,604][105692] Updated weights for policy 0, policy_version 874110 (0.0006) [2023-12-26 21:43:32,667][105692] Updated weights for policy 0, policy_version 874120 (0.0007) [2023-12-26 21:43:32,680][105620] Updated weights for policy 1, policy_version 874105 (0.0009) [2023-12-26 21:43:32,738][105620] Updated weights for policy 1, policy_version 874115 (0.0008) [2023-12-26 21:43:32,793][105620] Updated weights for policy 1, policy_version 874125 (0.0009) [2023-12-26 21:43:32,850][105620] Updated weights for policy 1, policy_version 874135 (0.0008) [2023-12-26 21:43:33,303][105692] Updated weights for policy 0, policy_version 874130 (0.0006) [2023-12-26 21:43:33,359][105692] Updated weights for policy 0, policy_version 874140 (0.0005) [2023-12-26 21:43:33,410][105692] Updated weights for policy 0, policy_version 874150 (0.0005) [2023-12-26 21:43:33,464][105692] Updated weights for policy 0, policy_version 874160 (0.0006) [2023-12-26 21:43:33,555][105620] Updated weights for policy 1, policy_version 874145 (0.0008) [2023-12-26 21:43:33,619][105620] Updated weights for policy 1, policy_version 874155 (0.0009) [2023-12-26 21:43:33,673][105620] Updated weights for policy 1, policy_version 874165 (0.0009) [2023-12-26 21:43:34,082][105692] Updated weights for policy 0, policy_version 874170 (0.0009) [2023-12-26 21:43:34,135][105692] Updated weights for policy 0, policy_version 874180 (0.0006) [2023-12-26 21:43:34,198][105692] Updated weights for policy 0, policy_version 874190 (0.0007) [2023-12-26 21:43:34,453][105620] Updated weights for policy 1, policy_version 874175 (0.0009) [2023-12-26 21:43:34,509][105620] Updated weights for policy 1, policy_version 874185 (0.0009) [2023-12-26 21:43:34,561][105620] Updated weights for policy 1, policy_version 874195 (0.0008) [2023-12-26 21:43:34,809][105692] Updated weights for policy 0, policy_version 874200 (0.0006) [2023-12-26 21:43:34,870][105692] Updated weights for policy 0, policy_version 874210 (0.0006) [2023-12-26 21:43:34,927][105692] Updated weights for policy 0, policy_version 874220 (0.0006) [2023-12-26 21:43:35,384][105620] Updated weights for policy 1, policy_version 874205 (0.0008) [2023-12-26 21:43:35,434][105620] Updated weights for policy 1, policy_version 874215 (0.0008) [2023-12-26 21:43:35,470][105692] Updated weights for policy 0, policy_version 874230 (0.0005) [2023-12-26 21:43:35,493][105620] Updated weights for policy 1, policy_version 874225 (0.0009) [2023-12-26 21:43:35,528][105692] Updated weights for policy 0, policy_version 874240 (0.0007) [2023-12-26 21:43:35,589][105692] Updated weights for policy 0, policy_version 874250 (0.0009) [2023-12-26 21:43:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 447676416. Throughput: 0: 9737.7, 1: 9993.7. Samples: 447665452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:43:36,063][104569] Avg episode reward: [(0, '4039.507'), (1, '9261.234')] [2023-12-26 21:43:36,270][105620] Updated weights for policy 1, policy_version 874235 (0.0009) [2023-12-26 21:43:36,272][105692] Updated weights for policy 0, policy_version 874260 (0.0006) [2023-12-26 21:43:36,296][105585] KL-divergence is very high: 198.3262 [2023-12-26 21:43:36,326][105620] Updated weights for policy 1, policy_version 874245 (0.0006) [2023-12-26 21:43:36,334][105692] Updated weights for policy 0, policy_version 874270 (0.0008) [2023-12-26 21:43:36,348][105585] KL-divergence is very high: 304.6350 [2023-12-26 21:43:36,378][105620] Updated weights for policy 1, policy_version 874255 (0.0005) [2023-12-26 21:43:36,398][105692] Updated weights for policy 0, policy_version 874280 (0.0008) [2023-12-26 21:43:36,400][105585] KL-divergence is very high: 306.4602 [2023-12-26 21:43:37,072][105620] Updated weights for policy 1, policy_version 874265 (0.0006) [2023-12-26 21:43:37,113][105692] Updated weights for policy 0, policy_version 874290 (0.0009) [2023-12-26 21:43:37,123][105620] Updated weights for policy 1, policy_version 874275 (0.0009) [2023-12-26 21:43:37,176][105620] Updated weights for policy 1, policy_version 874285 (0.0006) [2023-12-26 21:43:37,177][105692] Updated weights for policy 0, policy_version 874300 (0.0007) [2023-12-26 21:43:37,235][105620] Updated weights for policy 1, policy_version 874295 (0.0006) [2023-12-26 21:43:37,237][105692] Updated weights for policy 0, policy_version 874310 (0.0007) [2023-12-26 21:43:37,295][105692] Updated weights for policy 0, policy_version 874320 (0.0009) [2023-12-26 21:43:37,988][105620] Updated weights for policy 1, policy_version 874305 (0.0009) [2023-12-26 21:43:38,034][105692] Updated weights for policy 0, policy_version 874330 (0.0005) [2023-12-26 21:43:38,041][105620] Updated weights for policy 1, policy_version 874315 (0.0009) [2023-12-26 21:43:38,094][105620] Updated weights for policy 1, policy_version 874325 (0.0008) [2023-12-26 21:43:38,098][105692] Updated weights for policy 0, policy_version 874340 (0.0005) [2023-12-26 21:43:38,150][105692] Updated weights for policy 0, policy_version 874350 (0.0005) [2023-12-26 21:43:38,812][105692] Updated weights for policy 0, policy_version 874360 (0.0009) [2023-12-26 21:43:38,870][105692] Updated weights for policy 0, policy_version 874370 (0.0006) [2023-12-26 21:43:38,910][105620] Updated weights for policy 1, policy_version 874335 (0.0007) [2023-12-26 21:43:38,932][105692] Updated weights for policy 0, policy_version 874380 (0.0005) [2023-12-26 21:43:38,965][105620] Updated weights for policy 1, policy_version 874345 (0.0008) [2023-12-26 21:43:39,029][105620] Updated weights for policy 1, policy_version 874355 (0.0005) [2023-12-26 21:43:39,695][105692] Updated weights for policy 0, policy_version 874390 (0.0008) [2023-12-26 21:43:39,706][105620] Updated weights for policy 1, policy_version 874365 (0.0007) [2023-12-26 21:43:39,747][105692] Updated weights for policy 0, policy_version 874400 (0.0007) [2023-12-26 21:43:39,773][105620] Updated weights for policy 1, policy_version 874375 (0.0006) [2023-12-26 21:43:39,807][105692] Updated weights for policy 0, policy_version 874410 (0.0008) [2023-12-26 21:43:39,844][105620] Updated weights for policy 1, policy_version 874385 (0.0007) [2023-12-26 21:43:40,540][105692] Updated weights for policy 0, policy_version 874420 (0.0008) [2023-12-26 21:43:40,590][105692] Updated weights for policy 0, policy_version 874430 (0.0010) [2023-12-26 21:43:40,620][105620] Updated weights for policy 1, policy_version 874395 (0.0007) [2023-12-26 21:43:40,642][105692] Updated weights for policy 0, policy_version 874440 (0.0011) [2023-12-26 21:43:40,673][105620] Updated weights for policy 1, policy_version 874405 (0.0006) [2023-12-26 21:43:40,720][105620] Updated weights for policy 1, policy_version 874415 (0.0007) [2023-12-26 21:43:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 447774720. Throughput: 0: 9787.3, 1: 9967.1. Samples: 447781372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:43:41,062][104569] Avg episode reward: [(0, '6422.440'), (1, '9260.845')] [2023-12-26 21:43:41,459][105692] Updated weights for policy 0, policy_version 874450 (0.0010) [2023-12-26 21:43:41,471][105620] Updated weights for policy 1, policy_version 874425 (0.0005) [2023-12-26 21:43:41,522][105692] Updated weights for policy 0, policy_version 874460 (0.0010) [2023-12-26 21:43:41,533][105620] Updated weights for policy 1, policy_version 874435 (0.0007) [2023-12-26 21:43:41,578][105692] Updated weights for policy 0, policy_version 874470 (0.0011) [2023-12-26 21:43:41,592][105620] Updated weights for policy 1, policy_version 874445 (0.0008) [2023-12-26 21:43:41,642][105692] Updated weights for policy 0, policy_version 874480 (0.0011) [2023-12-26 21:43:41,657][105620] Updated weights for policy 1, policy_version 874455 (0.0007) [2023-12-26 21:43:42,432][105620] Updated weights for policy 1, policy_version 874465 (0.0007) [2023-12-26 21:43:42,467][105692] Updated weights for policy 0, policy_version 874490 (0.0010) [2023-12-26 21:43:42,493][105620] Updated weights for policy 1, policy_version 874475 (0.0005) [2023-12-26 21:43:42,529][105692] Updated weights for policy 0, policy_version 874500 (0.0009) [2023-12-26 21:43:42,552][105620] Updated weights for policy 1, policy_version 874485 (0.0007) [2023-12-26 21:43:42,588][105692] Updated weights for policy 0, policy_version 874510 (0.0011) [2023-12-26 21:43:43,256][105620] Updated weights for policy 1, policy_version 874495 (0.0005) [2023-12-26 21:43:43,257][105692] Updated weights for policy 0, policy_version 874520 (0.0008) [2023-12-26 21:43:43,307][105620] Updated weights for policy 1, policy_version 874505 (0.0006) [2023-12-26 21:43:43,311][105692] Updated weights for policy 0, policy_version 874530 (0.0010) [2023-12-26 21:43:43,359][105620] Updated weights for policy 1, policy_version 874515 (0.0007) [2023-12-26 21:43:43,362][105692] Updated weights for policy 0, policy_version 874540 (0.0007) [2023-12-26 21:43:43,903][105620] Updated weights for policy 1, policy_version 874525 (0.0006) [2023-12-26 21:43:43,961][105620] Updated weights for policy 1, policy_version 874535 (0.0005) [2023-12-26 21:43:44,022][105620] Updated weights for policy 1, policy_version 874545 (0.0006) [2023-12-26 21:43:44,089][105692] Updated weights for policy 0, policy_version 874550 (0.0009) [2023-12-26 21:43:44,147][105692] Updated weights for policy 0, policy_version 874560 (0.0010) [2023-12-26 21:43:44,207][105692] Updated weights for policy 0, policy_version 874570 (0.0009) [2023-12-26 21:43:44,603][105620] Updated weights for policy 1, policy_version 874555 (0.0007) [2023-12-26 21:43:44,660][105620] Updated weights for policy 1, policy_version 874565 (0.0009) [2023-12-26 21:43:44,720][105620] Updated weights for policy 1, policy_version 874575 (0.0008) [2023-12-26 21:43:44,992][105692] Updated weights for policy 0, policy_version 874580 (0.0008) [2023-12-26 21:43:45,060][105692] Updated weights for policy 0, policy_version 874590 (0.0010) [2023-12-26 21:43:45,117][105692] Updated weights for policy 0, policy_version 874600 (0.0009) [2023-12-26 21:43:45,413][105620] Updated weights for policy 1, policy_version 874585 (0.0008) [2023-12-26 21:43:45,471][105620] Updated weights for policy 1, policy_version 874595 (0.0005) [2023-12-26 21:43:45,528][105620] Updated weights for policy 1, policy_version 874605 (0.0005) [2023-12-26 21:43:45,587][105620] Updated weights for policy 1, policy_version 874615 (0.0006) [2023-12-26 21:43:45,971][105692] Updated weights for policy 0, policy_version 874610 (0.0008) [2023-12-26 21:43:46,025][105692] Updated weights for policy 0, policy_version 874620 (0.0005) [2023-12-26 21:43:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 447864832. Throughput: 0: 9706.9, 1: 9970.9. Samples: 447839100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:43:46,063][104569] Avg episode reward: [(0, '8063.097'), (1, '9260.560')] [2023-12-26 21:43:46,074][105692] Updated weights for policy 0, policy_version 874630 (0.0005) [2023-12-26 21:43:46,124][105620] Updated weights for policy 1, policy_version 874625 (0.0009) [2023-12-26 21:43:46,133][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000874640_223944704.pth... [2023-12-26 21:43:46,134][105692] Updated weights for policy 0, policy_version 874640 (0.0005) [2023-12-26 21:43:46,137][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000873488_223649792.pth [2023-12-26 21:43:46,186][105620] Updated weights for policy 1, policy_version 874635 (0.0008) [2023-12-26 21:43:46,238][105620] Updated weights for policy 1, policy_version 874645 (0.0009) [2023-12-26 21:43:46,252][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000874648_223936512.pth... [2023-12-26 21:43:46,256][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000873464_223633408.pth [2023-12-26 21:43:46,785][105692] Updated weights for policy 0, policy_version 874650 (0.0010) [2023-12-26 21:43:46,846][105692] Updated weights for policy 0, policy_version 874660 (0.0010) [2023-12-26 21:43:46,905][105692] Updated weights for policy 0, policy_version 874670 (0.0010) [2023-12-26 21:43:47,022][105620] Updated weights for policy 1, policy_version 874655 (0.0008) [2023-12-26 21:43:47,085][105620] Updated weights for policy 1, policy_version 874665 (0.0008) [2023-12-26 21:43:47,140][105620] Updated weights for policy 1, policy_version 874675 (0.0008) [2023-12-26 21:43:47,575][105692] Updated weights for policy 0, policy_version 874680 (0.0008) [2023-12-26 21:43:47,629][105692] Updated weights for policy 0, policy_version 874690 (0.0010) [2023-12-26 21:43:47,686][105692] Updated weights for policy 0, policy_version 874700 (0.0010) [2023-12-26 21:43:47,929][105620] Updated weights for policy 1, policy_version 874685 (0.0009) [2023-12-26 21:43:47,986][105620] Updated weights for policy 1, policy_version 874696 (0.0010) [2023-12-26 21:43:48,052][105620] Updated weights for policy 1, policy_version 874706 (0.0011) [2023-12-26 21:43:48,324][105692] Updated weights for policy 0, policy_version 874710 (0.0008) [2023-12-26 21:43:48,383][105692] Updated weights for policy 0, policy_version 874720 (0.0006) [2023-12-26 21:43:48,439][105692] Updated weights for policy 0, policy_version 874730 (0.0006) [2023-12-26 21:43:48,816][105620] Updated weights for policy 1, policy_version 874716 (0.0010) [2023-12-26 21:43:48,875][105620] Updated weights for policy 1, policy_version 874726 (0.0010) [2023-12-26 21:43:48,935][105620] Updated weights for policy 1, policy_version 874736 (0.0010) [2023-12-26 21:43:49,024][105692] Updated weights for policy 0, policy_version 874740 (0.0006) [2023-12-26 21:43:49,080][105692] Updated weights for policy 0, policy_version 874750 (0.0008) [2023-12-26 21:43:49,143][105692] Updated weights for policy 0, policy_version 874760 (0.0008) [2023-12-26 21:43:49,700][105620] Updated weights for policy 1, policy_version 874746 (0.0011) [2023-12-26 21:43:49,754][105620] Updated weights for policy 1, policy_version 874756 (0.0010) [2023-12-26 21:43:49,806][105620] Updated weights for policy 1, policy_version 874766 (0.0010) [2023-12-26 21:43:49,864][105620] Updated weights for policy 1, policy_version 874776 (0.0010) [2023-12-26 21:43:49,893][105692] Updated weights for policy 0, policy_version 874770 (0.0009) [2023-12-26 21:43:49,956][105692] Updated weights for policy 0, policy_version 874780 (0.0007) [2023-12-26 21:43:50,019][105692] Updated weights for policy 0, policy_version 874790 (0.0007) [2023-12-26 21:43:50,076][105692] Updated weights for policy 0, policy_version 874800 (0.0006) [2023-12-26 21:43:50,623][105620] Updated weights for policy 1, policy_version 874786 (0.0010) [2023-12-26 21:43:50,687][105620] Updated weights for policy 1, policy_version 874796 (0.0011) [2023-12-26 21:43:50,747][105620] Updated weights for policy 1, policy_version 874806 (0.0010) [2023-12-26 21:43:50,748][105692] Updated weights for policy 0, policy_version 874810 (0.0010) [2023-12-26 21:43:50,812][105692] Updated weights for policy 0, policy_version 874820 (0.0008) [2023-12-26 21:43:50,872][105692] Updated weights for policy 0, policy_version 874830 (0.0008) [2023-12-26 21:43:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19633.1). Total num frames: 447971328. Throughput: 0: 9761.7, 1: 9937.2. Samples: 447957556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:43:51,062][104569] Avg episode reward: [(0, '8775.756'), (1, '8993.438')] [2023-12-26 21:43:51,520][105620] Updated weights for policy 1, policy_version 874816 (0.0010) [2023-12-26 21:43:51,590][105620] Updated weights for policy 1, policy_version 874826 (0.0010) [2023-12-26 21:43:51,627][105692] Updated weights for policy 0, policy_version 874840 (0.0007) [2023-12-26 21:43:51,656][105620] Updated weights for policy 1, policy_version 874836 (0.0008) [2023-12-26 21:43:51,695][105692] Updated weights for policy 0, policy_version 874850 (0.0008) [2023-12-26 21:43:51,765][105692] Updated weights for policy 0, policy_version 874860 (0.0010) [2023-12-26 21:43:52,335][105692] Updated weights for policy 0, policy_version 874870 (0.0006) [2023-12-26 21:43:52,336][105620] Updated weights for policy 1, policy_version 874846 (0.0007) [2023-12-26 21:43:52,401][105692] Updated weights for policy 0, policy_version 874880 (0.0009) [2023-12-26 21:43:52,403][105620] Updated weights for policy 1, policy_version 874856 (0.0007) [2023-12-26 21:43:52,462][105692] Updated weights for policy 0, policy_version 874890 (0.0007) [2023-12-26 21:43:52,468][105620] Updated weights for policy 1, policy_version 874866 (0.0008) [2023-12-26 21:43:53,060][105692] Updated weights for policy 0, policy_version 874900 (0.0006) [2023-12-26 21:43:53,118][105692] Updated weights for policy 0, policy_version 874910 (0.0006) [2023-12-26 21:43:53,168][105692] Updated weights for policy 0, policy_version 874920 (0.0005) [2023-12-26 21:43:53,265][105620] Updated weights for policy 1, policy_version 874876 (0.0008) [2023-12-26 21:43:53,325][105620] Updated weights for policy 1, policy_version 874886 (0.0009) [2023-12-26 21:43:53,382][105620] Updated weights for policy 1, policy_version 874896 (0.0010) [2023-12-26 21:43:53,763][105692] Updated weights for policy 0, policy_version 874930 (0.0006) [2023-12-26 21:43:53,807][105692] Updated weights for policy 0, policy_version 874940 (0.0008) [2023-12-26 21:43:53,851][105692] Updated weights for policy 0, policy_version 874950 (0.0007) [2023-12-26 21:43:53,895][105692] Updated weights for policy 0, policy_version 874960 (0.0008) [2023-12-26 21:43:54,167][105620] Updated weights for policy 1, policy_version 874906 (0.0010) [2023-12-26 21:43:54,219][105620] Updated weights for policy 1, policy_version 874916 (0.0010) [2023-12-26 21:43:54,263][105620] Updated weights for policy 1, policy_version 874926 (0.0009) [2023-12-26 21:43:54,317][105620] Updated weights for policy 1, policy_version 874936 (0.0010) [2023-12-26 21:43:54,677][105692] Updated weights for policy 0, policy_version 874970 (0.0008) [2023-12-26 21:43:54,721][105692] Updated weights for policy 0, policy_version 874980 (0.0008) [2023-12-26 21:43:54,765][105692] Updated weights for policy 0, policy_version 874990 (0.0008) [2023-12-26 21:43:55,066][105620] Updated weights for policy 1, policy_version 874946 (0.0010) [2023-12-26 21:43:55,127][105620] Updated weights for policy 1, policy_version 874956 (0.0010) [2023-12-26 21:43:55,188][105620] Updated weights for policy 1, policy_version 874966 (0.0010) [2023-12-26 21:43:55,571][105692] Updated weights for policy 0, policy_version 875000 (0.0006) [2023-12-26 21:43:55,629][105692] Updated weights for policy 0, policy_version 875010 (0.0005) [2023-12-26 21:43:55,688][105692] Updated weights for policy 0, policy_version 875020 (0.0006) [2023-12-26 21:43:55,938][105620] Updated weights for policy 1, policy_version 874976 (0.0006) [2023-12-26 21:43:55,997][105620] Updated weights for policy 1, policy_version 874986 (0.0005) [2023-12-26 21:43:56,054][105620] Updated weights for policy 1, policy_version 874996 (0.0009) [2023-12-26 21:43:56,063][104569] Fps is (10 sec: 19658.2, 60 sec: 19660.4, 300 sec: 19605.2). Total num frames: 448061440. Throughput: 0: 9936.5, 1: 9779.0. Samples: 448073956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:43:56,064][104569] Avg episode reward: [(0, '8911.693'), (1, '9006.795')] [2023-12-26 21:43:56,320][105692] Updated weights for policy 0, policy_version 875030 (0.0010) [2023-12-26 21:43:56,382][105692] Updated weights for policy 0, policy_version 875040 (0.0010) [2023-12-26 21:43:56,445][105692] Updated weights for policy 0, policy_version 875050 (0.0008) [2023-12-26 21:43:56,698][105620] Updated weights for policy 1, policy_version 875006 (0.0011) [2023-12-26 21:43:56,764][105620] Updated weights for policy 1, policy_version 875016 (0.0008) [2023-12-26 21:43:56,828][105620] Updated weights for policy 1, policy_version 875026 (0.0010) [2023-12-26 21:43:57,130][105692] Updated weights for policy 0, policy_version 875060 (0.0009) [2023-12-26 21:43:57,181][105692] Updated weights for policy 0, policy_version 875070 (0.0010) [2023-12-26 21:43:57,235][105692] Updated weights for policy 0, policy_version 875080 (0.0010) [2023-12-26 21:43:57,474][105620] Updated weights for policy 1, policy_version 875036 (0.0008) [2023-12-26 21:43:57,525][105620] Updated weights for policy 1, policy_version 875046 (0.0005) [2023-12-26 21:43:57,576][105620] Updated weights for policy 1, policy_version 875056 (0.0006) [2023-12-26 21:43:57,953][105692] Updated weights for policy 0, policy_version 875090 (0.0010) [2023-12-26 21:43:57,996][105692] Updated weights for policy 0, policy_version 875100 (0.0010) [2023-12-26 21:43:58,044][105692] Updated weights for policy 0, policy_version 875110 (0.0010) [2023-12-26 21:43:58,091][105692] Updated weights for policy 0, policy_version 875120 (0.0010) [2023-12-26 21:43:58,257][105620] Updated weights for policy 1, policy_version 875066 (0.0010) [2023-12-26 21:43:58,314][105620] Updated weights for policy 1, policy_version 875076 (0.0009) [2023-12-26 21:43:58,385][105620] Updated weights for policy 1, policy_version 875086 (0.0007) [2023-12-26 21:43:58,447][105620] Updated weights for policy 1, policy_version 875096 (0.0007) [2023-12-26 21:43:59,003][105692] Updated weights for policy 0, policy_version 875130 (0.0008) [2023-12-26 21:43:59,066][105692] Updated weights for policy 0, policy_version 875140 (0.0008) [2023-12-26 21:43:59,128][105692] Updated weights for policy 0, policy_version 875150 (0.0005) [2023-12-26 21:43:59,180][105620] Updated weights for policy 1, policy_version 875106 (0.0009) [2023-12-26 21:43:59,257][105620] Updated weights for policy 1, policy_version 875116 (0.0009) [2023-12-26 21:43:59,324][105620] Updated weights for policy 1, policy_version 875126 (0.0008) [2023-12-26 21:43:59,805][105692] Updated weights for policy 0, policy_version 875160 (0.0006) [2023-12-26 21:43:59,872][105692] Updated weights for policy 0, policy_version 875170 (0.0007) [2023-12-26 21:43:59,935][105692] Updated weights for policy 0, policy_version 875180 (0.0008) [2023-12-26 21:44:00,030][105620] Updated weights for policy 1, policy_version 875136 (0.0006) [2023-12-26 21:44:00,076][105620] Updated weights for policy 1, policy_version 875146 (0.0006) [2023-12-26 21:44:00,127][105620] Updated weights for policy 1, policy_version 875156 (0.0005) [2023-12-26 21:44:00,660][105692] Updated weights for policy 0, policy_version 875190 (0.0010) [2023-12-26 21:44:00,665][105620] Updated weights for policy 1, policy_version 875166 (0.0005) [2023-12-26 21:44:00,704][105692] Updated weights for policy 0, policy_version 875200 (0.0010) [2023-12-26 21:44:00,723][105620] Updated weights for policy 1, policy_version 875176 (0.0006) [2023-12-26 21:44:00,751][105692] Updated weights for policy 0, policy_version 875210 (0.0010) [2023-12-26 21:44:00,781][105620] Updated weights for policy 1, policy_version 875186 (0.0006) [2023-12-26 21:44:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 448167936. Throughput: 0: 9943.3, 1: 9818.7. Samples: 448134020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:44:01,063][104569] Avg episode reward: [(0, '8805.557'), (1, '9176.063')] [2023-12-26 21:44:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000875216_224092160.pth... [2023-12-26 21:44:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000875192_224075776.pth... [2023-12-26 21:44:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000874064_223797248.pth [2023-12-26 21:44:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000874072_223789056.pth [2023-12-26 21:44:01,390][105620] Updated weights for policy 1, policy_version 875196 (0.0006) [2023-12-26 21:44:01,437][105620] Updated weights for policy 1, policy_version 875206 (0.0010) [2023-12-26 21:44:01,493][105620] Updated weights for policy 1, policy_version 875216 (0.0008) [2023-12-26 21:44:01,520][105692] Updated weights for policy 0, policy_version 875220 (0.0010) [2023-12-26 21:44:01,579][105692] Updated weights for policy 0, policy_version 875230 (0.0010) [2023-12-26 21:44:01,646][105692] Updated weights for policy 0, policy_version 875240 (0.0008) [2023-12-26 21:44:02,265][105620] Updated weights for policy 1, policy_version 875226 (0.0006) [2023-12-26 21:44:02,323][105620] Updated weights for policy 1, policy_version 875236 (0.0009) [2023-12-26 21:44:02,331][105692] Updated weights for policy 0, policy_version 875250 (0.0007) [2023-12-26 21:44:02,391][105620] Updated weights for policy 1, policy_version 875246 (0.0008) [2023-12-26 21:44:02,398][105692] Updated weights for policy 0, policy_version 875260 (0.0007) [2023-12-26 21:44:02,454][105620] Updated weights for policy 1, policy_version 875256 (0.0007) [2023-12-26 21:44:02,460][105692] Updated weights for policy 0, policy_version 875270 (0.0007) [2023-12-26 21:44:02,515][105692] Updated weights for policy 0, policy_version 875280 (0.0009) [2023-12-26 21:44:03,099][105620] Updated weights for policy 1, policy_version 875266 (0.0006) [2023-12-26 21:44:03,166][105620] Updated weights for policy 1, policy_version 875276 (0.0005) [2023-12-26 21:44:03,226][105620] Updated weights for policy 1, policy_version 875286 (0.0005) [2023-12-26 21:44:03,324][105692] Updated weights for policy 0, policy_version 875290 (0.0009) [2023-12-26 21:44:03,371][105692] Updated weights for policy 0, policy_version 875300 (0.0009) [2023-12-26 21:44:03,422][105692] Updated weights for policy 0, policy_version 875310 (0.0009) [2023-12-26 21:44:03,780][105620] Updated weights for policy 1, policy_version 875296 (0.0008) [2023-12-26 21:44:03,839][105620] Updated weights for policy 1, policy_version 875306 (0.0010) [2023-12-26 21:44:03,902][105620] Updated weights for policy 1, policy_version 875316 (0.0011) [2023-12-26 21:44:04,247][105692] Updated weights for policy 0, policy_version 875320 (0.0008) [2023-12-26 21:44:04,308][105692] Updated weights for policy 0, policy_version 875330 (0.0008) [2023-12-26 21:44:04,365][105692] Updated weights for policy 0, policy_version 875340 (0.0009) [2023-12-26 21:44:04,602][105620] Updated weights for policy 1, policy_version 875326 (0.0007) [2023-12-26 21:44:04,666][105620] Updated weights for policy 1, policy_version 875336 (0.0009) [2023-12-26 21:44:04,726][105620] Updated weights for policy 1, policy_version 875346 (0.0008) [2023-12-26 21:44:05,196][105692] Updated weights for policy 0, policy_version 875350 (0.0009) [2023-12-26 21:44:05,248][105692] Updated weights for policy 0, policy_version 875360 (0.0009) [2023-12-26 21:44:05,297][105692] Updated weights for policy 0, policy_version 875370 (0.0009) [2023-12-26 21:44:05,322][105620] Updated weights for policy 1, policy_version 875356 (0.0008) [2023-12-26 21:44:05,374][105620] Updated weights for policy 1, policy_version 875366 (0.0006) [2023-12-26 21:44:05,440][105620] Updated weights for policy 1, policy_version 875376 (0.0005) [2023-12-26 21:44:06,062][104569] Fps is (10 sec: 19663.2, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 448258048. Throughput: 0: 9769.6, 1: 9817.7. Samples: 448252244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:44:06,063][104569] Avg episode reward: [(0, '8723.174'), (1, '8993.369')] [2023-12-26 21:44:06,111][105620] Updated weights for policy 1, policy_version 875386 (0.0009) [2023-12-26 21:44:06,128][105692] Updated weights for policy 0, policy_version 875380 (0.0009) [2023-12-26 21:44:06,175][105620] Updated weights for policy 1, policy_version 875396 (0.0006) [2023-12-26 21:44:06,193][105692] Updated weights for policy 0, policy_version 875390 (0.0011) [2023-12-26 21:44:06,240][105620] Updated weights for policy 1, policy_version 875406 (0.0006) [2023-12-26 21:44:06,258][105692] Updated weights for policy 0, policy_version 875400 (0.0011) [2023-12-26 21:44:06,300][105620] Updated weights for policy 1, policy_version 875416 (0.0006) [2023-12-26 21:44:06,979][105692] Updated weights for policy 0, policy_version 875410 (0.0011) [2023-12-26 21:44:07,034][105692] Updated weights for policy 0, policy_version 875420 (0.0011) [2023-12-26 21:44:07,037][105620] Updated weights for policy 1, policy_version 875426 (0.0006) [2023-12-26 21:44:07,086][105692] Updated weights for policy 0, policy_version 875430 (0.0010) [2023-12-26 21:44:07,088][105620] Updated weights for policy 1, policy_version 875436 (0.0005) [2023-12-26 21:44:07,144][105620] Updated weights for policy 1, policy_version 875446 (0.0005) [2023-12-26 21:44:07,145][105692] Updated weights for policy 0, policy_version 875440 (0.0011) [2023-12-26 21:44:07,786][105692] Updated weights for policy 0, policy_version 875450 (0.0010) [2023-12-26 21:44:07,827][105620] Updated weights for policy 1, policy_version 875456 (0.0006) [2023-12-26 21:44:07,841][105692] Updated weights for policy 0, policy_version 875460 (0.0010) [2023-12-26 21:44:07,890][105620] Updated weights for policy 1, policy_version 875466 (0.0005) [2023-12-26 21:44:07,896][105692] Updated weights for policy 0, policy_version 875470 (0.0010) [2023-12-26 21:44:07,943][105620] Updated weights for policy 1, policy_version 875477 (0.0007) [2023-12-26 21:44:08,665][105692] Updated weights for policy 0, policy_version 875480 (0.0011) [2023-12-26 21:44:08,676][105620] Updated weights for policy 1, policy_version 875487 (0.0007) [2023-12-26 21:44:08,728][105692] Updated weights for policy 0, policy_version 875490 (0.0010) [2023-12-26 21:44:08,743][105620] Updated weights for policy 1, policy_version 875497 (0.0006) [2023-12-26 21:44:08,787][105692] Updated weights for policy 0, policy_version 875500 (0.0010) [2023-12-26 21:44:08,805][105620] Updated weights for policy 1, policy_version 875507 (0.0006) [2023-12-26 21:44:09,530][105692] Updated weights for policy 0, policy_version 875510 (0.0011) [2023-12-26 21:44:09,565][105620] Updated weights for policy 1, policy_version 875517 (0.0008) [2023-12-26 21:44:09,597][105692] Updated weights for policy 0, policy_version 875520 (0.0011) [2023-12-26 21:44:09,623][105620] Updated weights for policy 1, policy_version 875527 (0.0007) [2023-12-26 21:44:09,657][105692] Updated weights for policy 0, policy_version 875530 (0.0010) [2023-12-26 21:44:09,683][105620] Updated weights for policy 1, policy_version 875537 (0.0007) [2023-12-26 21:44:10,416][105692] Updated weights for policy 0, policy_version 875540 (0.0011) [2023-12-26 21:44:10,471][105620] Updated weights for policy 1, policy_version 875547 (0.0007) [2023-12-26 21:44:10,479][105692] Updated weights for policy 0, policy_version 875550 (0.0011) [2023-12-26 21:44:10,531][105620] Updated weights for policy 1, policy_version 875557 (0.0007) [2023-12-26 21:44:10,535][105692] Updated weights for policy 0, policy_version 875560 (0.0010) [2023-12-26 21:44:10,581][105620] Updated weights for policy 1, policy_version 875567 (0.0009) [2023-12-26 21:44:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 448356352. Throughput: 0: 9697.1, 1: 9798.6. Samples: 448365512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:44:11,062][104569] Avg episode reward: [(0, '2020.460'), (1, '8906.593')] [2023-12-26 21:44:11,305][105692] Updated weights for policy 0, policy_version 875570 (0.0010) [2023-12-26 21:44:11,318][105620] Updated weights for policy 1, policy_version 875577 (0.0008) [2023-12-26 21:44:11,372][105692] Updated weights for policy 0, policy_version 875580 (0.0011) [2023-12-26 21:44:11,391][105620] Updated weights for policy 1, policy_version 875587 (0.0008) [2023-12-26 21:44:11,438][105692] Updated weights for policy 0, policy_version 875590 (0.0008) [2023-12-26 21:44:11,456][105620] Updated weights for policy 1, policy_version 875597 (0.0008) [2023-12-26 21:44:11,502][105692] Updated weights for policy 0, policy_version 875600 (0.0008) [2023-12-26 21:44:11,523][105620] Updated weights for policy 1, policy_version 875607 (0.0009) [2023-12-26 21:44:12,257][105692] Updated weights for policy 0, policy_version 875610 (0.0007) [2023-12-26 21:44:12,280][105620] Updated weights for policy 1, policy_version 875617 (0.0006) [2023-12-26 21:44:12,320][105692] Updated weights for policy 0, policy_version 875620 (0.0008) [2023-12-26 21:44:12,349][105620] Updated weights for policy 1, policy_version 875627 (0.0007) [2023-12-26 21:44:12,396][105692] Updated weights for policy 0, policy_version 875630 (0.0007) [2023-12-26 21:44:12,416][105620] Updated weights for policy 1, policy_version 875637 (0.0010) [2023-12-26 21:44:13,056][105692] Updated weights for policy 0, policy_version 875640 (0.0008) [2023-12-26 21:44:13,121][105692] Updated weights for policy 0, policy_version 875650 (0.0005) [2023-12-26 21:44:13,179][105692] Updated weights for policy 0, policy_version 875660 (0.0007) [2023-12-26 21:44:13,184][105620] Updated weights for policy 1, policy_version 875647 (0.0007) [2023-12-26 21:44:13,235][105620] Updated weights for policy 1, policy_version 875657 (0.0006) [2023-12-26 21:44:13,286][105620] Updated weights for policy 1, policy_version 875667 (0.0007) [2023-12-26 21:44:13,751][105692] Updated weights for policy 0, policy_version 875670 (0.0006) [2023-12-26 21:44:13,800][105692] Updated weights for policy 0, policy_version 875680 (0.0010) [2023-12-26 21:44:13,844][105692] Updated weights for policy 0, policy_version 875690 (0.0010) [2023-12-26 21:44:13,902][105620] Updated weights for policy 1, policy_version 875677 (0.0008) [2023-12-26 21:44:13,957][105620] Updated weights for policy 1, policy_version 875687 (0.0005) [2023-12-26 21:44:14,010][105620] Updated weights for policy 1, policy_version 875697 (0.0010) [2023-12-26 21:44:14,412][105692] Updated weights for policy 0, policy_version 875700 (0.0008) [2023-12-26 21:44:14,462][105692] Updated weights for policy 0, policy_version 875710 (0.0005) [2023-12-26 21:44:14,516][105692] Updated weights for policy 0, policy_version 875720 (0.0005) [2023-12-26 21:44:14,731][105620] Updated weights for policy 1, policy_version 875707 (0.0010) [2023-12-26 21:44:14,797][105620] Updated weights for policy 1, policy_version 875717 (0.0011) [2023-12-26 21:44:14,855][105620] Updated weights for policy 1, policy_version 875727 (0.0010) [2023-12-26 21:44:15,157][105692] Updated weights for policy 0, policy_version 875730 (0.0006) [2023-12-26 21:44:15,220][105692] Updated weights for policy 0, policy_version 875740 (0.0011) [2023-12-26 21:44:15,273][105692] Updated weights for policy 0, policy_version 875750 (0.0010) [2023-12-26 21:44:15,332][105692] Updated weights for policy 0, policy_version 875760 (0.0010) [2023-12-26 21:44:15,626][105620] Updated weights for policy 1, policy_version 875737 (0.0011) [2023-12-26 21:44:15,681][105620] Updated weights for policy 1, policy_version 875747 (0.0010) [2023-12-26 21:44:15,743][105620] Updated weights for policy 1, policy_version 875757 (0.0010) [2023-12-26 21:44:15,804][105620] Updated weights for policy 1, policy_version 875767 (0.0010) [2023-12-26 21:44:16,052][105692] Updated weights for policy 0, policy_version 875770 (0.0007) [2023-12-26 21:44:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 448454656. Throughput: 0: 9751.3, 1: 9714.6. Samples: 448424236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:44:16,062][104569] Avg episode reward: [(0, '3283.667'), (1, '8899.231')] [2023-12-26 21:44:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000875768_224223232.pth... [2023-12-26 21:44:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000874648_223936512.pth [2023-12-26 21:44:16,105][105692] Updated weights for policy 0, policy_version 875780 (0.0007) [2023-12-26 21:44:16,157][105692] Updated weights for policy 0, policy_version 875790 (0.0008) [2023-12-26 21:44:16,166][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000875792_224239616.pth... [2023-12-26 21:44:16,169][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000874640_223944704.pth [2023-12-26 21:44:16,538][105620] Updated weights for policy 1, policy_version 875777 (0.0011) [2023-12-26 21:44:16,589][105620] Updated weights for policy 1, policy_version 875787 (0.0010) [2023-12-26 21:44:16,655][105620] Updated weights for policy 1, policy_version 875797 (0.0011) [2023-12-26 21:44:16,828][105692] Updated weights for policy 0, policy_version 875800 (0.0006) [2023-12-26 21:44:16,893][105692] Updated weights for policy 0, policy_version 875810 (0.0006) [2023-12-26 21:44:16,924][105585] KL-divergence is very high: 101.4642 [2023-12-26 21:44:16,957][105692] Updated weights for policy 0, policy_version 875820 (0.0007) [2023-12-26 21:44:17,372][105620] Updated weights for policy 1, policy_version 875807 (0.0011) [2023-12-26 21:44:17,441][105620] Updated weights for policy 1, policy_version 875817 (0.0011) [2023-12-26 21:44:17,506][105692] Updated weights for policy 0, policy_version 875830 (0.0007) [2023-12-26 21:44:17,507][105620] Updated weights for policy 1, policy_version 875827 (0.0011) [2023-12-26 21:44:17,562][105692] Updated weights for policy 0, policy_version 875840 (0.0006) [2023-12-26 21:44:17,614][105692] Updated weights for policy 0, policy_version 875850 (0.0008) [2023-12-26 21:44:18,142][105620] Updated weights for policy 1, policy_version 875837 (0.0011) [2023-12-26 21:44:18,197][105620] Updated weights for policy 1, policy_version 875847 (0.0010) [2023-12-26 21:44:18,256][105620] Updated weights for policy 1, policy_version 875857 (0.0007) [2023-12-26 21:44:18,315][105692] Updated weights for policy 0, policy_version 875860 (0.0007) [2023-12-26 21:44:18,377][105692] Updated weights for policy 0, policy_version 875870 (0.0009) [2023-12-26 21:44:18,432][105692] Updated weights for policy 0, policy_version 875880 (0.0008) [2023-12-26 21:44:18,931][105620] Updated weights for policy 1, policy_version 875867 (0.0008) [2023-12-26 21:44:19,000][105620] Updated weights for policy 1, policy_version 875877 (0.0011) [2023-12-26 21:44:19,061][105620] Updated weights for policy 1, policy_version 875887 (0.0010) [2023-12-26 21:44:19,100][105692] Updated weights for policy 0, policy_version 875891 (0.0010) [2023-12-26 21:44:19,149][105692] Updated weights for policy 0, policy_version 875901 (0.0008) [2023-12-26 21:44:19,204][105692] Updated weights for policy 0, policy_version 875911 (0.0007) [2023-12-26 21:44:19,780][105620] Updated weights for policy 1, policy_version 875897 (0.0010) [2023-12-26 21:44:19,840][105620] Updated weights for policy 1, policy_version 875907 (0.0006) [2023-12-26 21:44:19,902][105620] Updated weights for policy 1, policy_version 875917 (0.0009) [2023-12-26 21:44:19,943][105692] Updated weights for policy 0, policy_version 875921 (0.0008) [2023-12-26 21:44:19,962][105620] Updated weights for policy 1, policy_version 875927 (0.0007) [2023-12-26 21:44:20,012][105692] Updated weights for policy 0, policy_version 875931 (0.0009) [2023-12-26 21:44:20,072][105692] Updated weights for policy 0, policy_version 875941 (0.0008) [2023-12-26 21:44:20,128][105692] Updated weights for policy 0, policy_version 875951 (0.0009) [2023-12-26 21:44:20,615][105620] Updated weights for policy 1, policy_version 875937 (0.0009) [2023-12-26 21:44:20,677][105620] Updated weights for policy 1, policy_version 875947 (0.0008) [2023-12-26 21:44:20,750][105620] Updated weights for policy 1, policy_version 875957 (0.0006) [2023-12-26 21:44:20,878][105692] Updated weights for policy 0, policy_version 875961 (0.0009) [2023-12-26 21:44:20,932][105692] Updated weights for policy 0, policy_version 875971 (0.0008) [2023-12-26 21:44:20,984][105692] Updated weights for policy 0, policy_version 875981 (0.0009) [2023-12-26 21:44:21,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 448561152. Throughput: 0: 9817.3, 1: 9751.4. Samples: 448546040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:44:21,063][104569] Avg episode reward: [(0, '6600.102'), (1, '8156.715')] [2023-12-26 21:44:21,540][105620] Updated weights for policy 1, policy_version 875967 (0.0006) [2023-12-26 21:44:21,606][105620] Updated weights for policy 1, policy_version 875977 (0.0006) [2023-12-26 21:44:21,644][105692] Updated weights for policy 0, policy_version 875991 (0.0007) [2023-12-26 21:44:21,683][105620] Updated weights for policy 1, policy_version 875987 (0.0007) [2023-12-26 21:44:21,710][105692] Updated weights for policy 0, policy_version 876001 (0.0009) [2023-12-26 21:44:21,789][105692] Updated weights for policy 0, policy_version 876011 (0.0008) [2023-12-26 21:44:22,388][105620] Updated weights for policy 1, policy_version 875997 (0.0009) [2023-12-26 21:44:22,450][105620] Updated weights for policy 1, policy_version 876007 (0.0009) [2023-12-26 21:44:22,521][105620] Updated weights for policy 1, policy_version 876017 (0.0009) [2023-12-26 21:44:22,530][105692] Updated weights for policy 0, policy_version 876021 (0.0008) [2023-12-26 21:44:22,591][105692] Updated weights for policy 0, policy_version 876031 (0.0006) [2023-12-26 21:44:22,657][105692] Updated weights for policy 0, policy_version 876041 (0.0009) [2023-12-26 21:44:23,252][105620] Updated weights for policy 1, policy_version 876027 (0.0008) [2023-12-26 21:44:23,306][105620] Updated weights for policy 1, policy_version 876037 (0.0009) [2023-12-26 21:44:23,353][105620] Updated weights for policy 1, policy_version 876047 (0.0009) [2023-12-26 21:44:23,400][105692] Updated weights for policy 0, policy_version 876051 (0.0009) [2023-12-26 21:44:23,457][105692] Updated weights for policy 0, policy_version 876061 (0.0009) [2023-12-26 21:44:23,503][105692] Updated weights for policy 0, policy_version 876071 (0.0008) [2023-12-26 21:44:24,104][105620] Updated weights for policy 1, policy_version 876057 (0.0008) [2023-12-26 21:44:24,150][105620] Updated weights for policy 1, policy_version 876067 (0.0009) [2023-12-26 21:44:24,199][105620] Updated weights for policy 1, policy_version 876077 (0.0010) [2023-12-26 21:44:24,254][105620] Updated weights for policy 1, policy_version 876087 (0.0010) [2023-12-26 21:44:24,274][105692] Updated weights for policy 0, policy_version 876081 (0.0009) [2023-12-26 21:44:24,339][105692] Updated weights for policy 0, policy_version 876091 (0.0008) [2023-12-26 21:44:24,399][105692] Updated weights for policy 0, policy_version 876101 (0.0009) [2023-12-26 21:44:24,458][105692] Updated weights for policy 0, policy_version 876111 (0.0008) [2023-12-26 21:44:24,892][105620] Updated weights for policy 1, policy_version 876097 (0.0006) [2023-12-26 21:44:24,956][105620] Updated weights for policy 1, policy_version 876107 (0.0008) [2023-12-26 21:44:25,024][105620] Updated weights for policy 1, policy_version 876117 (0.0010) [2023-12-26 21:44:25,110][105692] Updated weights for policy 0, policy_version 876121 (0.0008) [2023-12-26 21:44:25,168][105692] Updated weights for policy 0, policy_version 876131 (0.0006) [2023-12-26 21:44:25,221][105692] Updated weights for policy 0, policy_version 876141 (0.0005) [2023-12-26 21:44:25,645][105620] Updated weights for policy 1, policy_version 876127 (0.0011) [2023-12-26 21:44:25,704][105620] Updated weights for policy 1, policy_version 876137 (0.0010) [2023-12-26 21:44:25,755][105620] Updated weights for policy 1, policy_version 876147 (0.0010) [2023-12-26 21:44:25,921][105692] Updated weights for policy 0, policy_version 876151 (0.0008) [2023-12-26 21:44:25,970][105692] Updated weights for policy 0, policy_version 876161 (0.0010) [2023-12-26 21:44:26,015][105692] Updated weights for policy 0, policy_version 876171 (0.0010) [2023-12-26 21:44:26,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 448659456. Throughput: 0: 9760.4, 1: 9831.2. Samples: 448662996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:44:26,062][104569] Avg episode reward: [(0, '8686.818'), (1, '7918.297')] [2023-12-26 21:44:26,446][105620] Updated weights for policy 1, policy_version 876157 (0.0010) [2023-12-26 21:44:26,498][105620] Updated weights for policy 1, policy_version 876167 (0.0011) [2023-12-26 21:44:26,549][105620] Updated weights for policy 1, policy_version 876177 (0.0010) [2023-12-26 21:44:26,699][105692] Updated weights for policy 0, policy_version 876181 (0.0008) [2023-12-26 21:44:26,759][105692] Updated weights for policy 0, policy_version 876191 (0.0006) [2023-12-26 21:44:26,819][105692] Updated weights for policy 0, policy_version 876201 (0.0008) [2023-12-26 21:44:27,310][105620] Updated weights for policy 1, policy_version 876187 (0.0010) [2023-12-26 21:44:27,372][105620] Updated weights for policy 1, policy_version 876197 (0.0011) [2023-12-26 21:44:27,431][105620] Updated weights for policy 1, policy_version 876207 (0.0010) [2023-12-26 21:44:27,517][105692] Updated weights for policy 0, policy_version 876211 (0.0009) [2023-12-26 21:44:27,568][105692] Updated weights for policy 0, policy_version 876221 (0.0010) [2023-12-26 21:44:27,626][105692] Updated weights for policy 0, policy_version 876231 (0.0010) [2023-12-26 21:44:28,129][105620] Updated weights for policy 1, policy_version 876217 (0.0008) [2023-12-26 21:44:28,176][105620] Updated weights for policy 1, policy_version 876227 (0.0010) [2023-12-26 21:44:28,220][105620] Updated weights for policy 1, policy_version 876237 (0.0010) [2023-12-26 21:44:28,274][105620] Updated weights for policy 1, policy_version 876247 (0.0010) [2023-12-26 21:44:28,377][105692] Updated weights for policy 0, policy_version 876241 (0.0010) [2023-12-26 21:44:28,436][105692] Updated weights for policy 0, policy_version 876251 (0.0010) [2023-12-26 21:44:28,487][105692] Updated weights for policy 0, policy_version 876261 (0.0010) [2023-12-26 21:44:28,532][105692] Updated weights for policy 0, policy_version 876271 (0.0010) [2023-12-26 21:44:28,884][105620] Updated weights for policy 1, policy_version 876257 (0.0006) [2023-12-26 21:44:28,930][105620] Updated weights for policy 1, policy_version 876267 (0.0009) [2023-12-26 21:44:28,978][105620] Updated weights for policy 1, policy_version 876277 (0.0011) [2023-12-26 21:44:29,239][105692] Updated weights for policy 0, policy_version 876281 (0.0008) [2023-12-26 21:44:29,294][105692] Updated weights for policy 0, policy_version 876291 (0.0008) [2023-12-26 21:44:29,362][105692] Updated weights for policy 0, policy_version 876301 (0.0008) [2023-12-26 21:44:29,675][105620] Updated weights for policy 1, policy_version 876287 (0.0011) [2023-12-26 21:44:29,731][105620] Updated weights for policy 1, policy_version 876297 (0.0010) [2023-12-26 21:44:29,786][105620] Updated weights for policy 1, policy_version 876307 (0.0010) [2023-12-26 21:44:29,957][105692] Updated weights for policy 0, policy_version 876311 (0.0007) [2023-12-26 21:44:30,019][105692] Updated weights for policy 0, policy_version 876321 (0.0006) [2023-12-26 21:44:30,076][105692] Updated weights for policy 0, policy_version 876331 (0.0006) [2023-12-26 21:44:30,601][105620] Updated weights for policy 1, policy_version 876317 (0.0010) [2023-12-26 21:44:30,656][105620] Updated weights for policy 1, policy_version 876327 (0.0008) [2023-12-26 21:44:30,670][105692] Updated weights for policy 0, policy_version 876341 (0.0007) [2023-12-26 21:44:30,710][105620] Updated weights for policy 1, policy_version 876337 (0.0008) [2023-12-26 21:44:30,716][105692] Updated weights for policy 0, policy_version 876351 (0.0008) [2023-12-26 21:44:30,762][105692] Updated weights for policy 0, policy_version 876361 (0.0006) [2023-12-26 21:44:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 448757760. Throughput: 0: 9809.8, 1: 9834.0. Samples: 448723076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:44:31,063][104569] Avg episode reward: [(0, '8830.829'), (1, '8239.237')] [2023-12-26 21:44:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000876368_224387072.pth... [2023-12-26 21:44:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000876344_224370688.pth... [2023-12-26 21:44:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000875192_224075776.pth [2023-12-26 21:44:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000875216_224092160.pth [2023-12-26 21:44:31,358][105620] Updated weights for policy 1, policy_version 876347 (0.0008) [2023-12-26 21:44:31,412][105620] Updated weights for policy 1, policy_version 876357 (0.0009) [2023-12-26 21:44:31,477][105620] Updated weights for policy 1, policy_version 876367 (0.0009) [2023-12-26 21:44:31,547][105692] Updated weights for policy 0, policy_version 876371 (0.0008) [2023-12-26 21:44:31,602][105692] Updated weights for policy 0, policy_version 876381 (0.0005) [2023-12-26 21:44:31,662][105692] Updated weights for policy 0, policy_version 876391 (0.0009) [2023-12-26 21:44:32,225][105620] Updated weights for policy 1, policy_version 876377 (0.0009) [2023-12-26 21:44:32,284][105620] Updated weights for policy 1, policy_version 876388 (0.0010) [2023-12-26 21:44:32,351][105620] Updated weights for policy 1, policy_version 876398 (0.0009) [2023-12-26 21:44:32,392][105692] Updated weights for policy 0, policy_version 876401 (0.0007) [2023-12-26 21:44:32,417][105620] Updated weights for policy 1, policy_version 876408 (0.0009) [2023-12-26 21:44:32,459][105692] Updated weights for policy 0, policy_version 876411 (0.0006) [2023-12-26 21:44:32,525][105692] Updated weights for policy 0, policy_version 876421 (0.0009) [2023-12-26 21:44:32,594][105692] Updated weights for policy 0, policy_version 876431 (0.0009) [2023-12-26 21:44:33,153][105620] Updated weights for policy 1, policy_version 876418 (0.0009) [2023-12-26 21:44:33,203][105620] Updated weights for policy 1, policy_version 876428 (0.0009) [2023-12-26 21:44:33,252][105620] Updated weights for policy 1, policy_version 876438 (0.0008) [2023-12-26 21:44:33,293][105692] Updated weights for policy 0, policy_version 876441 (0.0009) [2023-12-26 21:44:33,346][105692] Updated weights for policy 0, policy_version 876451 (0.0009) [2023-12-26 21:44:33,406][105692] Updated weights for policy 0, policy_version 876461 (0.0008) [2023-12-26 21:44:33,940][105620] Updated weights for policy 1, policy_version 876448 (0.0005) [2023-12-26 21:44:33,995][105620] Updated weights for policy 1, policy_version 876458 (0.0006) [2023-12-26 21:44:34,043][105620] Updated weights for policy 1, policy_version 876468 (0.0008) [2023-12-26 21:44:34,069][105692] Updated weights for policy 0, policy_version 876471 (0.0009) [2023-12-26 21:44:34,123][105692] Updated weights for policy 0, policy_version 876481 (0.0009) [2023-12-26 21:44:34,183][105692] Updated weights for policy 0, policy_version 876491 (0.0009) [2023-12-26 21:44:34,684][105620] Updated weights for policy 1, policy_version 876478 (0.0008) [2023-12-26 21:44:34,751][105620] Updated weights for policy 1, policy_version 876488 (0.0010) [2023-12-26 21:44:34,805][105620] Updated weights for policy 1, policy_version 876498 (0.0006) [2023-12-26 21:44:34,996][105692] Updated weights for policy 0, policy_version 876501 (0.0010) [2023-12-26 21:44:35,048][105692] Updated weights for policy 0, policy_version 876511 (0.0009) [2023-12-26 21:44:35,103][105692] Updated weights for policy 0, policy_version 876521 (0.0009) [2023-12-26 21:44:35,586][105620] Updated weights for policy 1, policy_version 876508 (0.0007) [2023-12-26 21:44:35,641][105620] Updated weights for policy 1, policy_version 876518 (0.0007) [2023-12-26 21:44:35,699][105620] Updated weights for policy 1, policy_version 876528 (0.0005) [2023-12-26 21:44:35,772][105692] Updated weights for policy 0, policy_version 876531 (0.0009) [2023-12-26 21:44:35,837][105692] Updated weights for policy 0, policy_version 876541 (0.0008) [2023-12-26 21:44:35,896][105692] Updated weights for policy 0, policy_version 876551 (0.0008) [2023-12-26 21:44:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 448856064. Throughput: 0: 9817.6, 1: 9827.1. Samples: 448841572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:44:36,063][104569] Avg episode reward: [(0, '8754.364'), (1, '9082.841')] [2023-12-26 21:44:36,332][105620] Updated weights for policy 1, policy_version 876538 (0.0009) [2023-12-26 21:44:36,397][105620] Updated weights for policy 1, policy_version 876548 (0.0010) [2023-12-26 21:44:36,468][105620] Updated weights for policy 1, policy_version 876558 (0.0009) [2023-12-26 21:44:36,531][105620] Updated weights for policy 1, policy_version 876568 (0.0009) [2023-12-26 21:44:36,556][105692] Updated weights for policy 0, policy_version 876561 (0.0008) [2023-12-26 21:44:36,612][105692] Updated weights for policy 0, policy_version 876571 (0.0007) [2023-12-26 21:44:36,662][105692] Updated weights for policy 0, policy_version 876581 (0.0008) [2023-12-26 21:44:36,715][105692] Updated weights for policy 0, policy_version 876591 (0.0010) [2023-12-26 21:44:37,308][105620] Updated weights for policy 1, policy_version 876578 (0.0008) [2023-12-26 21:44:37,363][105620] Updated weights for policy 1, policy_version 876588 (0.0008) [2023-12-26 21:44:37,402][105692] Updated weights for policy 0, policy_version 876601 (0.0010) [2023-12-26 21:44:37,413][105620] Updated weights for policy 1, policy_version 876598 (0.0008) [2023-12-26 21:44:37,463][105692] Updated weights for policy 0, policy_version 876611 (0.0008) [2023-12-26 21:44:37,522][105692] Updated weights for policy 0, policy_version 876621 (0.0006) [2023-12-26 21:44:38,069][105692] Updated weights for policy 0, policy_version 876631 (0.0005) [2023-12-26 21:44:38,132][105692] Updated weights for policy 0, policy_version 876641 (0.0006) [2023-12-26 21:44:38,185][105692] Updated weights for policy 0, policy_version 876651 (0.0007) [2023-12-26 21:44:38,303][105620] Updated weights for policy 1, policy_version 876608 (0.0008) [2023-12-26 21:44:38,368][105620] Updated weights for policy 1, policy_version 876618 (0.0009) [2023-12-26 21:44:38,424][105620] Updated weights for policy 1, policy_version 876628 (0.0008) [2023-12-26 21:44:38,881][105692] Updated weights for policy 0, policy_version 876661 (0.0010) [2023-12-26 21:44:38,940][105692] Updated weights for policy 0, policy_version 876671 (0.0010) [2023-12-26 21:44:38,996][105692] Updated weights for policy 0, policy_version 876681 (0.0010) [2023-12-26 21:44:39,137][105620] Updated weights for policy 1, policy_version 876638 (0.0008) [2023-12-26 21:44:39,199][105620] Updated weights for policy 1, policy_version 876648 (0.0008) [2023-12-26 21:44:39,263][105620] Updated weights for policy 1, policy_version 876658 (0.0009) [2023-12-26 21:44:39,754][105692] Updated weights for policy 0, policy_version 876691 (0.0010) [2023-12-26 21:44:39,821][105692] Updated weights for policy 0, policy_version 876701 (0.0011) [2023-12-26 21:44:39,883][105692] Updated weights for policy 0, policy_version 876711 (0.0011) [2023-12-26 21:44:39,989][105620] Updated weights for policy 1, policy_version 876668 (0.0010) [2023-12-26 21:44:40,044][105620] Updated weights for policy 1, policy_version 876678 (0.0009) [2023-12-26 21:44:40,112][105620] Updated weights for policy 1, policy_version 876688 (0.0008) [2023-12-26 21:44:40,628][105692] Updated weights for policy 0, policy_version 876721 (0.0009) [2023-12-26 21:44:40,692][105692] Updated weights for policy 0, policy_version 876731 (0.0005) [2023-12-26 21:44:40,758][105692] Updated weights for policy 0, policy_version 876741 (0.0005) [2023-12-26 21:44:40,828][105692] Updated weights for policy 0, policy_version 876751 (0.0005) [2023-12-26 21:44:40,869][105620] Updated weights for policy 1, policy_version 876698 (0.0008) [2023-12-26 21:44:40,925][105620] Updated weights for policy 1, policy_version 876708 (0.0009) [2023-12-26 21:44:40,994][105620] Updated weights for policy 1, policy_version 876718 (0.0010) [2023-12-26 21:44:41,056][105620] Updated weights for policy 1, policy_version 876728 (0.0009) [2023-12-26 21:44:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 448954368. Throughput: 0: 9822.9, 1: 9818.4. Samples: 448957788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:44:41,062][104569] Avg episode reward: [(0, '8662.856'), (1, '9181.592')] [2023-12-26 21:44:41,415][105692] Updated weights for policy 0, policy_version 876761 (0.0007) [2023-12-26 21:44:41,476][105692] Updated weights for policy 0, policy_version 876771 (0.0006) [2023-12-26 21:44:41,538][105692] Updated weights for policy 0, policy_version 876781 (0.0006) [2023-12-26 21:44:41,949][105620] Updated weights for policy 1, policy_version 876738 (0.0009) [2023-12-26 21:44:42,012][105620] Updated weights for policy 1, policy_version 876748 (0.0009) [2023-12-26 21:44:42,084][105620] Updated weights for policy 1, policy_version 876758 (0.0010) [2023-12-26 21:44:42,194][105692] Updated weights for policy 0, policy_version 876791 (0.0007) [2023-12-26 21:44:42,255][105692] Updated weights for policy 0, policy_version 876801 (0.0006) [2023-12-26 21:44:42,308][105692] Updated weights for policy 0, policy_version 876811 (0.0006) [2023-12-26 21:44:42,873][105620] Updated weights for policy 1, policy_version 876768 (0.0007) [2023-12-26 21:44:42,934][105620] Updated weights for policy 1, policy_version 876778 (0.0009) [2023-12-26 21:44:42,995][105620] Updated weights for policy 1, policy_version 876788 (0.0009) [2023-12-26 21:44:43,035][105692] Updated weights for policy 0, policy_version 876821 (0.0010) [2023-12-26 21:44:43,101][105692] Updated weights for policy 0, policy_version 876831 (0.0009) [2023-12-26 21:44:43,152][105692] Updated weights for policy 0, policy_version 876841 (0.0005) [2023-12-26 21:44:43,755][105620] Updated weights for policy 1, policy_version 876798 (0.0009) [2023-12-26 21:44:43,811][105620] Updated weights for policy 1, policy_version 876808 (0.0009) [2023-12-26 21:44:43,856][105692] Updated weights for policy 0, policy_version 876851 (0.0007) [2023-12-26 21:44:43,865][105620] Updated weights for policy 1, policy_version 876818 (0.0009) [2023-12-26 21:44:43,907][105692] Updated weights for policy 0, policy_version 876861 (0.0010) [2023-12-26 21:44:43,962][105692] Updated weights for policy 0, policy_version 876871 (0.0008) [2023-12-26 21:44:44,596][105620] Updated weights for policy 1, policy_version 876828 (0.0005) [2023-12-26 21:44:44,650][105692] Updated weights for policy 0, policy_version 876881 (0.0006) [2023-12-26 21:44:44,660][105620] Updated weights for policy 1, policy_version 876838 (0.0007) [2023-12-26 21:44:44,701][105692] Updated weights for policy 0, policy_version 876891 (0.0010) [2023-12-26 21:44:44,715][105620] Updated weights for policy 1, policy_version 876848 (0.0005) [2023-12-26 21:44:44,752][105692] Updated weights for policy 0, policy_version 876901 (0.0010) [2023-12-26 21:44:44,815][105692] Updated weights for policy 0, policy_version 876911 (0.0010) [2023-12-26 21:44:45,444][105620] Updated weights for policy 1, policy_version 876858 (0.0008) [2023-12-26 21:44:45,516][105620] Updated weights for policy 1, policy_version 876868 (0.0006) [2023-12-26 21:44:45,573][105692] Updated weights for policy 0, policy_version 876921 (0.0010) [2023-12-26 21:44:45,575][105620] Updated weights for policy 1, policy_version 876878 (0.0006) [2023-12-26 21:44:45,623][105620] Updated weights for policy 1, policy_version 876888 (0.0006) [2023-12-26 21:44:45,628][105692] Updated weights for policy 0, policy_version 876931 (0.0010) [2023-12-26 21:44:45,691][105692] Updated weights for policy 0, policy_version 876941 (0.0010) [2023-12-26 21:44:46,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 449044480. Throughput: 0: 9842.2, 1: 9728.8. Samples: 449014716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:44:46,062][104569] Avg episode reward: [(0, '9081.602'), (1, '8743.178')] [2023-12-26 21:44:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000876944_224534528.pth... [2023-12-26 21:44:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000876888_224509952.pth... [2023-12-26 21:44:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000875792_224239616.pth [2023-12-26 21:44:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000875768_224223232.pth [2023-12-26 21:44:46,310][105620] Updated weights for policy 1, policy_version 876898 (0.0005) [2023-12-26 21:44:46,330][105692] Updated weights for policy 0, policy_version 876951 (0.0009) [2023-12-26 21:44:46,359][105620] Updated weights for policy 1, policy_version 876908 (0.0005) [2023-12-26 21:44:46,380][105692] Updated weights for policy 0, policy_version 876961 (0.0008) [2023-12-26 21:44:46,406][105620] Updated weights for policy 1, policy_version 876918 (0.0006) [2023-12-26 21:44:46,438][105692] Updated weights for policy 0, policy_version 876971 (0.0006) [2023-12-26 21:44:47,137][105692] Updated weights for policy 0, policy_version 876981 (0.0007) [2023-12-26 21:44:47,169][105620] Updated weights for policy 1, policy_version 876928 (0.0009) [2023-12-26 21:44:47,204][105692] Updated weights for policy 0, policy_version 876991 (0.0005) [2023-12-26 21:44:47,217][105620] Updated weights for policy 1, policy_version 876938 (0.0008) [2023-12-26 21:44:47,257][105692] Updated weights for policy 0, policy_version 877001 (0.0005) [2023-12-26 21:44:47,270][105620] Updated weights for policy 1, policy_version 876948 (0.0008) [2023-12-26 21:44:47,951][105692] Updated weights for policy 0, policy_version 877011 (0.0006) [2023-12-26 21:44:48,002][105620] Updated weights for policy 1, policy_version 876958 (0.0008) [2023-12-26 21:44:48,006][105692] Updated weights for policy 0, policy_version 877021 (0.0010) [2023-12-26 21:44:48,054][105620] Updated weights for policy 1, policy_version 876968 (0.0007) [2023-12-26 21:44:48,061][105692] Updated weights for policy 0, policy_version 877031 (0.0008) [2023-12-26 21:44:48,107][105620] Updated weights for policy 1, policy_version 876978 (0.0008) [2023-12-26 21:44:48,825][105692] Updated weights for policy 0, policy_version 877041 (0.0006) [2023-12-26 21:44:48,881][105692] Updated weights for policy 0, policy_version 877051 (0.0009) [2023-12-26 21:44:48,940][105620] Updated weights for policy 1, policy_version 876988 (0.0009) [2023-12-26 21:44:48,942][105692] Updated weights for policy 0, policy_version 877061 (0.0009) [2023-12-26 21:44:49,005][105692] Updated weights for policy 0, policy_version 877071 (0.0009) [2023-12-26 21:44:49,007][105620] Updated weights for policy 1, policy_version 876998 (0.0008) [2023-12-26 21:44:49,065][105620] Updated weights for policy 1, policy_version 877008 (0.0008) [2023-12-26 21:44:49,821][105620] Updated weights for policy 1, policy_version 877018 (0.0009) [2023-12-26 21:44:49,832][105692] Updated weights for policy 0, policy_version 877081 (0.0007) [2023-12-26 21:44:49,884][105620] Updated weights for policy 1, policy_version 877028 (0.0008) [2023-12-26 21:44:49,900][105692] Updated weights for policy 0, policy_version 877091 (0.0008) [2023-12-26 21:44:49,900][105585] KL-divergence is very high: 143.7509 [2023-12-26 21:44:49,950][105620] Updated weights for policy 1, policy_version 877038 (0.0007) [2023-12-26 21:44:49,958][105585] KL-divergence is very high: 170.0770 [2023-12-26 21:44:49,970][105692] Updated weights for policy 0, policy_version 877101 (0.0007) [2023-12-26 21:44:50,014][105620] Updated weights for policy 1, policy_version 877048 (0.0009) [2023-12-26 21:44:50,677][105692] Updated weights for policy 0, policy_version 877111 (0.0007) [2023-12-26 21:44:50,695][105620] Updated weights for policy 1, policy_version 877058 (0.0007) [2023-12-26 21:44:50,738][105692] Updated weights for policy 0, policy_version 877121 (0.0006) [2023-12-26 21:44:50,763][105620] Updated weights for policy 1, policy_version 877068 (0.0009) [2023-12-26 21:44:50,796][105692] Updated weights for policy 0, policy_version 877131 (0.0009) [2023-12-26 21:44:50,826][105620] Updated weights for policy 1, policy_version 877078 (0.0005) [2023-12-26 21:44:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 449142784. Throughput: 0: 9906.8, 1: 9600.3. Samples: 449130060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:44:51,062][104569] Avg episode reward: [(0, '9002.958'), (1, '8411.105')] [2023-12-26 21:44:51,522][105620] Updated weights for policy 1, policy_version 877088 (0.0009) [2023-12-26 21:44:51,579][105620] Updated weights for policy 1, policy_version 877098 (0.0009) [2023-12-26 21:44:51,589][105692] Updated weights for policy 0, policy_version 877141 (0.0008) [2023-12-26 21:44:51,645][105620] Updated weights for policy 1, policy_version 877108 (0.0007) [2023-12-26 21:44:51,651][105692] Updated weights for policy 0, policy_version 877151 (0.0007) [2023-12-26 21:44:51,710][105692] Updated weights for policy 0, policy_version 877161 (0.0007) [2023-12-26 21:44:52,425][105620] Updated weights for policy 1, policy_version 877118 (0.0008) [2023-12-26 21:44:52,434][105692] Updated weights for policy 0, policy_version 877171 (0.0009) [2023-12-26 21:44:52,478][105620] Updated weights for policy 1, policy_version 877128 (0.0006) [2023-12-26 21:44:52,496][105692] Updated weights for policy 0, policy_version 877181 (0.0008) [2023-12-26 21:44:52,538][105620] Updated weights for policy 1, policy_version 877138 (0.0008) [2023-12-26 21:44:52,546][105692] Updated weights for policy 0, policy_version 877191 (0.0006) [2023-12-26 21:44:53,238][105692] Updated weights for policy 0, policy_version 877201 (0.0007) [2023-12-26 21:44:53,296][105692] Updated weights for policy 0, policy_version 877211 (0.0005) [2023-12-26 21:44:53,359][105692] Updated weights for policy 0, policy_version 877221 (0.0005) [2023-12-26 21:44:53,366][105620] Updated weights for policy 1, policy_version 877148 (0.0009) [2023-12-26 21:44:53,424][105692] Updated weights for policy 0, policy_version 877231 (0.0005) [2023-12-26 21:44:53,432][105620] Updated weights for policy 1, policy_version 877158 (0.0006) [2023-12-26 21:44:53,505][105620] Updated weights for policy 1, policy_version 877168 (0.0005) [2023-12-26 21:44:53,917][105692] Updated weights for policy 0, policy_version 877241 (0.0005) [2023-12-26 21:44:53,968][105692] Updated weights for policy 0, policy_version 877251 (0.0006) [2023-12-26 21:44:54,023][105692] Updated weights for policy 0, policy_version 877261 (0.0005) [2023-12-26 21:44:54,128][105620] Updated weights for policy 1, policy_version 877178 (0.0006) [2023-12-26 21:44:54,184][105620] Updated weights for policy 1, policy_version 877188 (0.0009) [2023-12-26 21:44:54,246][105620] Updated weights for policy 1, policy_version 877198 (0.0009) [2023-12-26 21:44:54,311][105620] Updated weights for policy 1, policy_version 877208 (0.0010) [2023-12-26 21:44:54,590][105692] Updated weights for policy 0, policy_version 877271 (0.0009) [2023-12-26 21:44:54,642][105692] Updated weights for policy 0, policy_version 877281 (0.0010) [2023-12-26 21:44:54,697][105692] Updated weights for policy 0, policy_version 877291 (0.0010) [2023-12-26 21:44:55,079][105620] Updated weights for policy 1, policy_version 877218 (0.0008) [2023-12-26 21:44:55,138][105620] Updated weights for policy 1, policy_version 877228 (0.0009) [2023-12-26 21:44:55,190][105620] Updated weights for policy 1, policy_version 877238 (0.0008) [2023-12-26 21:44:55,444][105692] Updated weights for policy 0, policy_version 877301 (0.0009) [2023-12-26 21:44:55,496][105692] Updated weights for policy 0, policy_version 877311 (0.0010) [2023-12-26 21:44:55,553][105692] Updated weights for policy 0, policy_version 877321 (0.0010) [2023-12-26 21:44:55,951][105620] Updated weights for policy 1, policy_version 877248 (0.0008) [2023-12-26 21:44:56,002][105620] Updated weights for policy 1, policy_version 877259 (0.0010) [2023-12-26 21:44:56,059][105620] Updated weights for policy 1, policy_version 877269 (0.0009) [2023-12-26 21:44:56,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.7, 300 sec: 19605.3). Total num frames: 449232896. Throughput: 0: 10014.7, 1: 9574.3. Samples: 449247012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:44:56,062][104569] Avg episode reward: [(0, '8809.398'), (1, '8853.479')] [2023-12-26 21:44:56,209][105692] Updated weights for policy 0, policy_version 877331 (0.0009) [2023-12-26 21:44:56,272][105692] Updated weights for policy 0, policy_version 877341 (0.0011) [2023-12-26 21:44:56,327][105692] Updated weights for policy 0, policy_version 877351 (0.0010) [2023-12-26 21:44:56,772][105620] Updated weights for policy 1, policy_version 877279 (0.0009) [2023-12-26 21:44:56,833][105620] Updated weights for policy 1, policy_version 877289 (0.0009) [2023-12-26 21:44:56,889][105620] Updated weights for policy 1, policy_version 877299 (0.0008) [2023-12-26 21:44:56,966][105692] Updated weights for policy 0, policy_version 877361 (0.0010) [2023-12-26 21:44:57,027][105692] Updated weights for policy 0, policy_version 877371 (0.0005) [2023-12-26 21:44:57,096][105692] Updated weights for policy 0, policy_version 877381 (0.0006) [2023-12-26 21:44:57,159][105692] Updated weights for policy 0, policy_version 877391 (0.0006) [2023-12-26 21:44:57,679][105692] Updated weights for policy 0, policy_version 877401 (0.0005) [2023-12-26 21:44:57,734][105692] Updated weights for policy 0, policy_version 877411 (0.0006) [2023-12-26 21:44:57,753][105620] Updated weights for policy 1, policy_version 877309 (0.0007) [2023-12-26 21:44:57,783][105692] Updated weights for policy 0, policy_version 877421 (0.0009) [2023-12-26 21:44:57,801][105620] Updated weights for policy 1, policy_version 877319 (0.0005) [2023-12-26 21:44:57,866][105620] Updated weights for policy 1, policy_version 877329 (0.0006) [2023-12-26 21:44:58,533][105692] Updated weights for policy 0, policy_version 877431 (0.0008) [2023-12-26 21:44:58,579][105620] Updated weights for policy 1, policy_version 877339 (0.0009) [2023-12-26 21:44:58,602][105692] Updated weights for policy 0, policy_version 877441 (0.0007) [2023-12-26 21:44:58,644][105620] Updated weights for policy 1, policy_version 877349 (0.0008) [2023-12-26 21:44:58,662][105692] Updated weights for policy 0, policy_version 877451 (0.0006) [2023-12-26 21:44:58,707][105620] Updated weights for policy 1, policy_version 877360 (0.0008) [2023-12-26 21:44:59,512][105620] Updated weights for policy 1, policy_version 877370 (0.0011) [2023-12-26 21:44:59,566][105692] Updated weights for policy 0, policy_version 877461 (0.0008) [2023-12-26 21:44:59,573][105620] Updated weights for policy 1, policy_version 877380 (0.0008) [2023-12-26 21:44:59,619][105692] Updated weights for policy 0, policy_version 877471 (0.0008) [2023-12-26 21:44:59,634][105620] Updated weights for policy 1, policy_version 877390 (0.0006) [2023-12-26 21:44:59,675][105692] Updated weights for policy 0, policy_version 877481 (0.0006) [2023-12-26 21:44:59,692][105620] Updated weights for policy 1, policy_version 877400 (0.0010) [2023-12-26 21:45:00,339][105620] Updated weights for policy 1, policy_version 877410 (0.0007) [2023-12-26 21:45:00,389][105620] Updated weights for policy 1, policy_version 877420 (0.0009) [2023-12-26 21:45:00,436][105692] Updated weights for policy 0, policy_version 877491 (0.0006) [2023-12-26 21:45:00,449][105620] Updated weights for policy 1, policy_version 877430 (0.0009) [2023-12-26 21:45:00,486][105692] Updated weights for policy 0, policy_version 877501 (0.0007) [2023-12-26 21:45:00,544][105692] Updated weights for policy 0, policy_version 877512 (0.0010) [2023-12-26 21:45:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 449331200. Throughput: 0: 10065.9, 1: 9527.7. Samples: 449305944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:45:01,062][104569] Avg episode reward: [(0, '9082.735'), (1, '8837.241')] [2023-12-26 21:45:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000877520_224681984.pth... [2023-12-26 21:45:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000877432_224649216.pth... [2023-12-26 21:45:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000876344_224370688.pth [2023-12-26 21:45:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000876368_224387072.pth [2023-12-26 21:45:01,192][105620] Updated weights for policy 1, policy_version 877440 (0.0005) [2023-12-26 21:45:01,224][105692] Updated weights for policy 0, policy_version 877523 (0.0008) [2023-12-26 21:45:01,253][105620] Updated weights for policy 1, policy_version 877450 (0.0006) [2023-12-26 21:45:01,283][105692] Updated weights for policy 0, policy_version 877533 (0.0008) [2023-12-26 21:45:01,316][105620] Updated weights for policy 1, policy_version 877460 (0.0009) [2023-12-26 21:45:01,337][105692] Updated weights for policy 0, policy_version 877543 (0.0006) [2023-12-26 21:45:01,962][105620] Updated weights for policy 1, policy_version 877470 (0.0006) [2023-12-26 21:45:02,015][105620] Updated weights for policy 1, policy_version 877480 (0.0007) [2023-12-26 21:45:02,075][105620] Updated weights for policy 1, policy_version 877490 (0.0008) [2023-12-26 21:45:02,171][105692] Updated weights for policy 0, policy_version 877553 (0.0008) [2023-12-26 21:45:02,228][105692] Updated weights for policy 0, policy_version 877563 (0.0005) [2023-12-26 21:45:02,295][105692] Updated weights for policy 0, policy_version 877573 (0.0007) [2023-12-26 21:45:02,365][105692] Updated weights for policy 0, policy_version 877583 (0.0008) [2023-12-26 21:45:02,780][105620] Updated weights for policy 1, policy_version 877500 (0.0009) [2023-12-26 21:45:02,842][105620] Updated weights for policy 1, policy_version 877510 (0.0009) [2023-12-26 21:45:02,903][105620] Updated weights for policy 1, policy_version 877520 (0.0009) [2023-12-26 21:45:03,008][105692] Updated weights for policy 0, policy_version 877593 (0.0008) [2023-12-26 21:45:03,055][105692] Updated weights for policy 0, policy_version 877603 (0.0009) [2023-12-26 21:45:03,107][105692] Updated weights for policy 0, policy_version 877613 (0.0007) [2023-12-26 21:45:03,690][105620] Updated weights for policy 1, policy_version 877530 (0.0009) [2023-12-26 21:45:03,743][105620] Updated weights for policy 1, policy_version 877540 (0.0008) [2023-12-26 21:45:03,773][105692] Updated weights for policy 0, policy_version 877623 (0.0007) [2023-12-26 21:45:03,799][105620] Updated weights for policy 1, policy_version 877550 (0.0005) [2023-12-26 21:45:03,833][105692] Updated weights for policy 0, policy_version 877633 (0.0008) [2023-12-26 21:45:03,868][105620] Updated weights for policy 1, policy_version 877560 (0.0012) [2023-12-26 21:45:03,897][105692] Updated weights for policy 0, policy_version 877643 (0.0007) [2023-12-26 21:45:04,549][105620] Updated weights for policy 1, policy_version 877570 (0.0011) [2023-12-26 21:45:04,613][105620] Updated weights for policy 1, policy_version 877580 (0.0011) [2023-12-26 21:45:04,669][105620] Updated weights for policy 1, policy_version 877590 (0.0010) [2023-12-26 21:45:04,712][105692] Updated weights for policy 0, policy_version 877653 (0.0010) [2023-12-26 21:45:04,771][105692] Updated weights for policy 0, policy_version 877663 (0.0009) [2023-12-26 21:45:04,843][105692] Updated weights for policy 0, policy_version 877673 (0.0006) [2023-12-26 21:45:05,373][105692] Updated weights for policy 0, policy_version 877683 (0.0006) [2023-12-26 21:45:05,373][105620] Updated weights for policy 1, policy_version 877600 (0.0009) [2023-12-26 21:45:05,422][105692] Updated weights for policy 0, policy_version 877693 (0.0006) [2023-12-26 21:45:05,446][105620] Updated weights for policy 1, policy_version 877610 (0.0009) [2023-12-26 21:45:05,467][105692] Updated weights for policy 0, policy_version 877703 (0.0005) [2023-12-26 21:45:05,507][105620] Updated weights for policy 1, policy_version 877620 (0.0008) [2023-12-26 21:45:06,020][105620] Updated weights for policy 1, policy_version 877630 (0.0005) [2023-12-26 21:45:06,054][105692] Updated weights for policy 0, policy_version 877713 (0.0006) [2023-12-26 21:45:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 449429504. Throughput: 0: 9894.8, 1: 9543.2. Samples: 449420748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:45:06,062][104569] Avg episode reward: [(0, '9084.560'), (1, '8566.068')] [2023-12-26 21:45:06,074][105620] Updated weights for policy 1, policy_version 877640 (0.0005) [2023-12-26 21:45:06,119][105692] Updated weights for policy 0, policy_version 877723 (0.0007) [2023-12-26 21:45:06,139][105620] Updated weights for policy 1, policy_version 877650 (0.0007) [2023-12-26 21:45:06,179][105692] Updated weights for policy 0, policy_version 877733 (0.0008) [2023-12-26 21:45:06,242][105692] Updated weights for policy 0, policy_version 877743 (0.0009) [2023-12-26 21:45:06,793][105620] Updated weights for policy 1, policy_version 877660 (0.0006) [2023-12-26 21:45:06,840][105620] Updated weights for policy 1, policy_version 877670 (0.0005) [2023-12-26 21:45:06,894][105620] Updated weights for policy 1, policy_version 877680 (0.0006) [2023-12-26 21:45:07,025][105692] Updated weights for policy 0, policy_version 877753 (0.0009) [2023-12-26 21:45:07,082][105692] Updated weights for policy 0, policy_version 877763 (0.0010) [2023-12-26 21:45:07,136][105692] Updated weights for policy 0, policy_version 877773 (0.0010) [2023-12-26 21:45:07,453][105620] Updated weights for policy 1, policy_version 877690 (0.0006) [2023-12-26 21:45:07,507][105620] Updated weights for policy 1, policy_version 877700 (0.0009) [2023-12-26 21:45:07,565][105620] Updated weights for policy 1, policy_version 877710 (0.0010) [2023-12-26 21:45:07,622][105620] Updated weights for policy 1, policy_version 877720 (0.0010) [2023-12-26 21:45:07,975][105692] Updated weights for policy 0, policy_version 877783 (0.0009) [2023-12-26 21:45:08,036][105692] Updated weights for policy 0, policy_version 877793 (0.0009) [2023-12-26 21:45:08,103][105692] Updated weights for policy 0, policy_version 877803 (0.0010) [2023-12-26 21:45:08,278][105620] Updated weights for policy 1, policy_version 877730 (0.0005) [2023-12-26 21:45:08,332][105620] Updated weights for policy 1, policy_version 877740 (0.0006) [2023-12-26 21:45:08,401][105620] Updated weights for policy 1, policy_version 877750 (0.0008) [2023-12-26 21:45:08,944][105692] Updated weights for policy 0, policy_version 877813 (0.0009) [2023-12-26 21:45:09,002][105692] Updated weights for policy 0, policy_version 877823 (0.0009) [2023-12-26 21:45:09,048][105620] Updated weights for policy 1, policy_version 877760 (0.0006) [2023-12-26 21:45:09,058][105692] Updated weights for policy 0, policy_version 877833 (0.0008) [2023-12-26 21:45:09,106][105620] Updated weights for policy 1, policy_version 877770 (0.0007) [2023-12-26 21:45:09,171][105620] Updated weights for policy 1, policy_version 877780 (0.0009) [2023-12-26 21:45:09,884][105692] Updated weights for policy 0, policy_version 877843 (0.0006) [2023-12-26 21:45:09,899][105620] Updated weights for policy 1, policy_version 877790 (0.0008) [2023-12-26 21:45:09,949][105692] Updated weights for policy 0, policy_version 877853 (0.0008) [2023-12-26 21:45:09,967][105620] Updated weights for policy 1, policy_version 877800 (0.0007) [2023-12-26 21:45:10,015][105692] Updated weights for policy 0, policy_version 877863 (0.0009) [2023-12-26 21:45:10,021][105620] Updated weights for policy 1, policy_version 877810 (0.0008) [2023-12-26 21:45:10,701][105620] Updated weights for policy 1, policy_version 877820 (0.0009) [2023-12-26 21:45:10,756][105620] Updated weights for policy 1, policy_version 877830 (0.0010) [2023-12-26 21:45:10,804][105692] Updated weights for policy 0, policy_version 877873 (0.0009) [2023-12-26 21:45:10,814][105620] Updated weights for policy 1, policy_version 877840 (0.0007) [2023-12-26 21:45:10,851][105692] Updated weights for policy 0, policy_version 877883 (0.0009) [2023-12-26 21:45:10,904][105692] Updated weights for policy 0, policy_version 877893 (0.0009) [2023-12-26 21:45:10,953][105692] Updated weights for policy 0, policy_version 877903 (0.0009) [2023-12-26 21:45:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 449536000. Throughput: 0: 9860.5, 1: 9636.7. Samples: 449540372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:45:11,063][104569] Avg episode reward: [(0, '8905.473'), (1, '8400.486')] [2023-12-26 21:45:11,485][105620] Updated weights for policy 1, policy_version 877850 (0.0009) [2023-12-26 21:45:11,535][105620] Updated weights for policy 1, policy_version 877860 (0.0008) [2023-12-26 21:45:11,589][105620] Updated weights for policy 1, policy_version 877870 (0.0009) [2023-12-26 21:45:11,646][105620] Updated weights for policy 1, policy_version 877880 (0.0010) [2023-12-26 21:45:11,799][105692] Updated weights for policy 0, policy_version 877913 (0.0010) [2023-12-26 21:45:11,868][105692] Updated weights for policy 0, policy_version 877923 (0.0009) [2023-12-26 21:45:11,922][105692] Updated weights for policy 0, policy_version 877933 (0.0010) [2023-12-26 21:45:12,282][105620] Updated weights for policy 1, policy_version 877890 (0.0006) [2023-12-26 21:45:12,353][105620] Updated weights for policy 1, policy_version 877900 (0.0008) [2023-12-26 21:45:12,416][105620] Updated weights for policy 1, policy_version 877910 (0.0007) [2023-12-26 21:45:12,809][105692] Updated weights for policy 0, policy_version 877943 (0.0009) [2023-12-26 21:45:12,869][105692] Updated weights for policy 0, policy_version 877953 (0.0009) [2023-12-26 21:45:12,930][105692] Updated weights for policy 0, policy_version 877963 (0.0009) [2023-12-26 21:45:13,052][105620] Updated weights for policy 1, policy_version 877920 (0.0007) [2023-12-26 21:45:13,111][105620] Updated weights for policy 1, policy_version 877930 (0.0010) [2023-12-26 21:45:13,173][105620] Updated weights for policy 1, policy_version 877940 (0.0010) [2023-12-26 21:45:13,736][105692] Updated weights for policy 0, policy_version 877973 (0.0009) [2023-12-26 21:45:13,799][105692] Updated weights for policy 0, policy_version 877983 (0.0008) [2023-12-26 21:45:13,801][105620] Updated weights for policy 1, policy_version 877950 (0.0010) [2023-12-26 21:45:13,860][105620] Updated weights for policy 1, policy_version 877960 (0.0010) [2023-12-26 21:45:13,863][105692] Updated weights for policy 0, policy_version 877993 (0.0005) [2023-12-26 21:45:13,926][105620] Updated weights for policy 1, policy_version 877970 (0.0010) [2023-12-26 21:45:14,542][105620] Updated weights for policy 1, policy_version 877980 (0.0006) [2023-12-26 21:45:14,600][105620] Updated weights for policy 1, policy_version 877990 (0.0005) [2023-12-26 21:45:14,665][105620] Updated weights for policy 1, policy_version 878000 (0.0006) [2023-12-26 21:45:14,701][105692] Updated weights for policy 0, policy_version 878003 (0.0007) [2023-12-26 21:45:14,752][105692] Updated weights for policy 0, policy_version 878013 (0.0009) [2023-12-26 21:45:14,807][105692] Updated weights for policy 0, policy_version 878023 (0.0008) [2023-12-26 21:45:15,353][105620] Updated weights for policy 1, policy_version 878010 (0.0010) [2023-12-26 21:45:15,408][105620] Updated weights for policy 1, policy_version 878020 (0.0009) [2023-12-26 21:45:15,464][105620] Updated weights for policy 1, policy_version 878030 (0.0009) [2023-12-26 21:45:15,511][105620] Updated weights for policy 1, policy_version 878040 (0.0008) [2023-12-26 21:45:15,596][105692] Updated weights for policy 0, policy_version 878033 (0.0009) [2023-12-26 21:45:15,654][105692] Updated weights for policy 0, policy_version 878043 (0.0009) [2023-12-26 21:45:15,712][105692] Updated weights for policy 0, policy_version 878054 (0.0010) [2023-12-26 21:45:16,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 449626112. Throughput: 0: 9774.1, 1: 9657.9. Samples: 449597520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:45:16,063][104569] Avg episode reward: [(0, '8018.046'), (1, '7915.576')] [2023-12-26 21:45:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000878064_224821248.pth... [2023-12-26 21:45:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000878040_224804864.pth... [2023-12-26 21:45:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000876944_224534528.pth [2023-12-26 21:45:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000876888_224509952.pth [2023-12-26 21:45:16,167][105620] Updated weights for policy 1, policy_version 878050 (0.0009) [2023-12-26 21:45:16,232][105620] Updated weights for policy 1, policy_version 878060 (0.0009) [2023-12-26 21:45:16,296][105620] Updated weights for policy 1, policy_version 878070 (0.0009) [2023-12-26 21:45:16,503][105692] Updated weights for policy 0, policy_version 878065 (0.0010) [2023-12-26 21:45:16,553][105692] Updated weights for policy 0, policy_version 878076 (0.0008) [2023-12-26 21:45:16,577][105585] KL-divergence is very high: 267.7838 [2023-12-26 21:45:16,588][105585] KL-divergence is very high: 316.8167 [2023-12-26 21:45:16,600][105585] KL-divergence is very high: 265.0409 [2023-12-26 21:45:16,608][105692] Updated weights for policy 0, policy_version 878086 (0.0009) [2023-12-26 21:45:16,623][105585] KL-divergence is very high: 578.9981 [2023-12-26 21:45:16,634][105585] KL-divergence is very high: 503.4969 [2023-12-26 21:45:16,644][105585] KL-divergence is very high: 338.6383 [2023-12-26 21:45:16,662][105692] Updated weights for policy 0, policy_version 878096 (0.0010) [2023-12-26 21:45:16,983][105620] Updated weights for policy 1, policy_version 878080 (0.0009) [2023-12-26 21:45:17,034][105620] Updated weights for policy 1, policy_version 878090 (0.0009) [2023-12-26 21:45:17,084][105620] Updated weights for policy 1, policy_version 878100 (0.0008) [2023-12-26 21:45:17,395][105585] KL-divergence is very high: 451.4156 [2023-12-26 21:45:17,443][105585] KL-divergence is very high: 375.4411 [2023-12-26 21:45:17,456][105692] Updated weights for policy 0, policy_version 878106 (0.0009) [2023-12-26 21:45:17,487][105585] KL-divergence is very high: 306.2254 [2023-12-26 21:45:17,510][105692] Updated weights for policy 0, policy_version 878116 (0.0009) [2023-12-26 21:45:17,529][105585] KL-divergence is very high: 251.7045 [2023-12-26 21:45:17,563][105692] Updated weights for policy 0, policy_version 878126 (0.0010) [2023-12-26 21:45:17,813][105620] Updated weights for policy 1, policy_version 878110 (0.0007) [2023-12-26 21:45:17,871][105620] Updated weights for policy 1, policy_version 878120 (0.0007) [2023-12-26 21:45:17,917][105620] Updated weights for policy 1, policy_version 878130 (0.0005) [2023-12-26 21:45:18,405][105692] Updated weights for policy 0, policy_version 878136 (0.0010) [2023-12-26 21:45:18,463][105692] Updated weights for policy 0, policy_version 878146 (0.0010) [2023-12-26 21:45:18,517][105692] Updated weights for policy 0, policy_version 878156 (0.0009) [2023-12-26 21:45:18,540][105620] Updated weights for policy 1, policy_version 878140 (0.0005) [2023-12-26 21:45:18,600][105620] Updated weights for policy 1, policy_version 878150 (0.0005) [2023-12-26 21:45:18,663][105620] Updated weights for policy 1, policy_version 878160 (0.0006) [2023-12-26 21:45:19,245][105620] Updated weights for policy 1, policy_version 878170 (0.0006) [2023-12-26 21:45:19,296][105620] Updated weights for policy 1, policy_version 878180 (0.0009) [2023-12-26 21:45:19,354][105620] Updated weights for policy 1, policy_version 878190 (0.0009) [2023-12-26 21:45:19,376][105692] Updated weights for policy 0, policy_version 878166 (0.0008) [2023-12-26 21:45:19,420][105620] Updated weights for policy 1, policy_version 878200 (0.0008) [2023-12-26 21:45:19,434][105692] Updated weights for policy 0, policy_version 878176 (0.0007) [2023-12-26 21:45:19,498][105692] Updated weights for policy 0, policy_version 878186 (0.0006) [2023-12-26 21:45:20,185][105692] Updated weights for policy 0, policy_version 878196 (0.0007) [2023-12-26 21:45:20,253][105692] Updated weights for policy 0, policy_version 878206 (0.0007) [2023-12-26 21:45:20,277][105620] Updated weights for policy 1, policy_version 878210 (0.0009) [2023-12-26 21:45:20,319][105692] Updated weights for policy 0, policy_version 878216 (0.0007) [2023-12-26 21:45:20,345][105620] Updated weights for policy 1, policy_version 878220 (0.0006) [2023-12-26 21:45:20,416][105620] Updated weights for policy 1, policy_version 878230 (0.0007) [2023-12-26 21:45:20,994][105620] Updated weights for policy 1, policy_version 878240 (0.0005) [2023-12-26 21:45:21,055][105620] Updated weights for policy 1, policy_version 878250 (0.0008) [2023-12-26 21:45:21,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 449716224. Throughput: 0: 9604.2, 1: 9729.9. Samples: 449711604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:45:21,063][104569] Avg episode reward: [(0, '7674.701'), (1, '8594.803')] [2023-12-26 21:45:21,116][105620] Updated weights for policy 1, policy_version 878260 (0.0010) [2023-12-26 21:45:21,135][105692] Updated weights for policy 0, policy_version 878226 (0.0010) [2023-12-26 21:45:21,192][105692] Updated weights for policy 0, policy_version 878236 (0.0009) [2023-12-26 21:45:21,256][105692] Updated weights for policy 0, policy_version 878246 (0.0009) [2023-12-26 21:45:21,316][105692] Updated weights for policy 0, policy_version 878256 (0.0008) [2023-12-26 21:45:21,894][105620] Updated weights for policy 1, policy_version 878270 (0.0008) [2023-12-26 21:45:21,955][105620] Updated weights for policy 1, policy_version 878280 (0.0007) [2023-12-26 21:45:22,016][105620] Updated weights for policy 1, policy_version 878290 (0.0009) [2023-12-26 21:45:22,139][105692] Updated weights for policy 0, policy_version 878266 (0.0008) [2023-12-26 21:45:22,198][105692] Updated weights for policy 0, policy_version 878276 (0.0009) [2023-12-26 21:45:22,264][105692] Updated weights for policy 0, policy_version 878286 (0.0008) [2023-12-26 21:45:22,794][105620] Updated weights for policy 1, policy_version 878300 (0.0010) [2023-12-26 21:45:22,846][105620] Updated weights for policy 1, policy_version 878310 (0.0010) [2023-12-26 21:45:22,902][105620] Updated weights for policy 1, policy_version 878320 (0.0011) [2023-12-26 21:45:22,956][105692] Updated weights for policy 0, policy_version 878296 (0.0009) [2023-12-26 21:45:23,020][105692] Updated weights for policy 0, policy_version 878306 (0.0011) [2023-12-26 21:45:23,085][105692] Updated weights for policy 0, policy_version 878316 (0.0010) [2023-12-26 21:45:23,588][105620] Updated weights for policy 1, policy_version 878330 (0.0010) [2023-12-26 21:45:23,639][105620] Updated weights for policy 1, policy_version 878340 (0.0009) [2023-12-26 21:45:23,700][105620] Updated weights for policy 1, policy_version 878350 (0.0007) [2023-12-26 21:45:23,761][105620] Updated weights for policy 1, policy_version 878360 (0.0005) [2023-12-26 21:45:23,799][105692] Updated weights for policy 0, policy_version 878326 (0.0009) [2023-12-26 21:45:23,857][105692] Updated weights for policy 0, policy_version 878336 (0.0011) [2023-12-26 21:45:23,913][105692] Updated weights for policy 0, policy_version 878346 (0.0010) [2023-12-26 21:45:24,354][105620] Updated weights for policy 1, policy_version 878371 (0.0010) [2023-12-26 21:45:24,407][105620] Updated weights for policy 1, policy_version 878382 (0.0009) [2023-12-26 21:45:24,465][105620] Updated weights for policy 1, policy_version 878392 (0.0007) [2023-12-26 21:45:24,570][105692] Updated weights for policy 0, policy_version 878356 (0.0010) [2023-12-26 21:45:24,628][105692] Updated weights for policy 0, policy_version 878366 (0.0011) [2023-12-26 21:45:24,685][105692] Updated weights for policy 0, policy_version 878376 (0.0008) [2023-12-26 21:45:25,200][105620] Updated weights for policy 1, policy_version 878402 (0.0010) [2023-12-26 21:45:25,261][105620] Updated weights for policy 1, policy_version 878412 (0.0010) [2023-12-26 21:45:25,316][105620] Updated weights for policy 1, policy_version 878422 (0.0010) [2023-12-26 21:45:25,332][105692] Updated weights for policy 0, policy_version 878386 (0.0009) [2023-12-26 21:45:25,385][105692] Updated weights for policy 0, policy_version 878396 (0.0007) [2023-12-26 21:45:25,443][105692] Updated weights for policy 0, policy_version 878406 (0.0006) [2023-12-26 21:45:25,504][105692] Updated weights for policy 0, policy_version 878416 (0.0006) [2023-12-26 21:45:25,985][105620] Updated weights for policy 1, policy_version 878432 (0.0008) [2023-12-26 21:45:26,033][105620] Updated weights for policy 1, policy_version 878442 (0.0007) [2023-12-26 21:45:26,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 449814528. Throughput: 0: 9540.6, 1: 9823.2. Samples: 449829160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:45:26,062][104569] Avg episode reward: [(0, '8060.395'), (1, '8985.162')] [2023-12-26 21:45:26,081][105620] Updated weights for policy 1, policy_version 878452 (0.0008) [2023-12-26 21:45:26,171][105692] Updated weights for policy 0, policy_version 878426 (0.0010) [2023-12-26 21:45:26,226][105692] Updated weights for policy 0, policy_version 878436 (0.0010) [2023-12-26 21:45:26,273][105692] Updated weights for policy 0, policy_version 878446 (0.0010) [2023-12-26 21:45:26,718][105620] Updated weights for policy 1, policy_version 878462 (0.0006) [2023-12-26 21:45:26,784][105620] Updated weights for policy 1, policy_version 878472 (0.0005) [2023-12-26 21:45:26,845][105620] Updated weights for policy 1, policy_version 878482 (0.0007) [2023-12-26 21:45:27,028][105692] Updated weights for policy 0, policy_version 878456 (0.0011) [2023-12-26 21:45:27,076][105692] Updated weights for policy 0, policy_version 878466 (0.0010) [2023-12-26 21:45:27,124][105692] Updated weights for policy 0, policy_version 878476 (0.0010) [2023-12-26 21:45:27,489][105620] Updated weights for policy 1, policy_version 878492 (0.0009) [2023-12-26 21:45:27,535][105620] Updated weights for policy 1, policy_version 878502 (0.0005) [2023-12-26 21:45:27,585][105620] Updated weights for policy 1, policy_version 878512 (0.0005) [2023-12-26 21:45:27,856][105692] Updated weights for policy 0, policy_version 878486 (0.0010) [2023-12-26 21:45:27,900][105692] Updated weights for policy 0, policy_version 878496 (0.0010) [2023-12-26 21:45:27,944][105692] Updated weights for policy 0, policy_version 878506 (0.0010) [2023-12-26 21:45:28,127][105620] Updated weights for policy 1, policy_version 878522 (0.0005) [2023-12-26 21:45:28,183][105620] Updated weights for policy 1, policy_version 878532 (0.0007) [2023-12-26 21:45:28,234][105620] Updated weights for policy 1, policy_version 878542 (0.0010) [2023-12-26 21:45:28,292][105620] Updated weights for policy 1, policy_version 878552 (0.0010) [2023-12-26 21:45:28,633][105692] Updated weights for policy 0, policy_version 878516 (0.0008) [2023-12-26 21:45:28,698][105692] Updated weights for policy 0, policy_version 878526 (0.0005) [2023-12-26 21:45:28,753][105692] Updated weights for policy 0, policy_version 878536 (0.0005) [2023-12-26 21:45:29,020][105620] Updated weights for policy 1, policy_version 878562 (0.0010) [2023-12-26 21:45:29,080][105620] Updated weights for policy 1, policy_version 878572 (0.0010) [2023-12-26 21:45:29,134][105620] Updated weights for policy 1, policy_version 878582 (0.0010) [2023-12-26 21:45:29,376][105692] Updated weights for policy 0, policy_version 878546 (0.0006) [2023-12-26 21:45:29,429][105692] Updated weights for policy 0, policy_version 878556 (0.0010) [2023-12-26 21:45:29,483][105692] Updated weights for policy 0, policy_version 878566 (0.0010) [2023-12-26 21:45:29,534][105692] Updated weights for policy 0, policy_version 878576 (0.0010) [2023-12-26 21:45:29,878][105620] Updated weights for policy 1, policy_version 878592 (0.0009) [2023-12-26 21:45:29,941][105620] Updated weights for policy 1, policy_version 878602 (0.0009) [2023-12-26 21:45:29,994][105620] Updated weights for policy 1, policy_version 878612 (0.0008) [2023-12-26 21:45:30,267][105692] Updated weights for policy 0, policy_version 878586 (0.0010) [2023-12-26 21:45:30,315][105692] Updated weights for policy 0, policy_version 878596 (0.0010) [2023-12-26 21:45:30,359][105692] Updated weights for policy 0, policy_version 878606 (0.0010) [2023-12-26 21:45:30,746][105620] Updated weights for policy 1, policy_version 878622 (0.0009) [2023-12-26 21:45:30,795][105620] Updated weights for policy 1, policy_version 878632 (0.0008) [2023-12-26 21:45:30,843][105620] Updated weights for policy 1, policy_version 878642 (0.0008) [2023-12-26 21:45:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 449921024. Throughput: 0: 9524.8, 1: 9961.0. Samples: 449891580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:45:31,063][104569] Avg episode reward: [(0, '8311.869'), (1, '9077.796')] [2023-12-26 21:45:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000878648_224960512.pth... [2023-12-26 21:45:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000877432_224649216.pth [2023-12-26 21:45:31,106][105692] Updated weights for policy 0, policy_version 878616 (0.0010) [2023-12-26 21:45:31,176][105692] Updated weights for policy 0, policy_version 878626 (0.0010) [2023-12-26 21:45:31,253][105692] Updated weights for policy 0, policy_version 878636 (0.0010) [2023-12-26 21:45:31,280][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000878640_224968704.pth... [2023-12-26 21:45:31,286][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000877520_224681984.pth [2023-12-26 21:45:31,560][105620] Updated weights for policy 1, policy_version 878652 (0.0007) [2023-12-26 21:45:31,613][105620] Updated weights for policy 1, policy_version 878662 (0.0008) [2023-12-26 21:45:31,670][105620] Updated weights for policy 1, policy_version 878672 (0.0008) [2023-12-26 21:45:31,982][105692] Updated weights for policy 0, policy_version 878646 (0.0009) [2023-12-26 21:45:32,044][105692] Updated weights for policy 0, policy_version 878656 (0.0007) [2023-12-26 21:45:32,111][105692] Updated weights for policy 0, policy_version 878666 (0.0008) [2023-12-26 21:45:32,479][105620] Updated weights for policy 1, policy_version 878682 (0.0009) [2023-12-26 21:45:32,540][105620] Updated weights for policy 1, policy_version 878692 (0.0010) [2023-12-26 21:45:32,603][105620] Updated weights for policy 1, policy_version 878702 (0.0010) [2023-12-26 21:45:32,654][105620] Updated weights for policy 1, policy_version 878712 (0.0010) [2023-12-26 21:45:32,808][105692] Updated weights for policy 0, policy_version 878676 (0.0008) [2023-12-26 21:45:32,864][105692] Updated weights for policy 0, policy_version 878686 (0.0008) [2023-12-26 21:45:32,908][105692] Updated weights for policy 0, policy_version 878696 (0.0007) [2023-12-26 21:45:33,337][105620] Updated weights for policy 1, policy_version 878722 (0.0008) [2023-12-26 21:45:33,402][105620] Updated weights for policy 1, policy_version 878732 (0.0008) [2023-12-26 21:45:33,463][105620] Updated weights for policy 1, policy_version 878742 (0.0009) [2023-12-26 21:45:33,578][105692] Updated weights for policy 0, policy_version 878706 (0.0007) [2023-12-26 21:45:33,635][105692] Updated weights for policy 0, policy_version 878716 (0.0005) [2023-12-26 21:45:33,694][105692] Updated weights for policy 0, policy_version 878726 (0.0005) [2023-12-26 21:45:33,752][105692] Updated weights for policy 0, policy_version 878736 (0.0007) [2023-12-26 21:45:34,249][105620] Updated weights for policy 1, policy_version 878752 (0.0008) [2023-12-26 21:45:34,307][105620] Updated weights for policy 1, policy_version 878762 (0.0010) [2023-12-26 21:45:34,373][105620] Updated weights for policy 1, policy_version 878772 (0.0009) [2023-12-26 21:45:34,417][105692] Updated weights for policy 0, policy_version 878746 (0.0006) [2023-12-26 21:45:34,481][105692] Updated weights for policy 0, policy_version 878756 (0.0009) [2023-12-26 21:45:34,537][105692] Updated weights for policy 0, policy_version 878766 (0.0009) [2023-12-26 21:45:35,163][105620] Updated weights for policy 1, policy_version 878782 (0.0009) [2023-12-26 21:45:35,193][105692] Updated weights for policy 0, policy_version 878776 (0.0007) [2023-12-26 21:45:35,220][105620] Updated weights for policy 1, policy_version 878792 (0.0008) [2023-12-26 21:45:35,238][105692] Updated weights for policy 0, policy_version 878786 (0.0008) [2023-12-26 21:45:35,280][105620] Updated weights for policy 1, policy_version 878802 (0.0008) [2023-12-26 21:45:35,320][105692] Updated weights for policy 0, policy_version 878796 (0.0006) [2023-12-26 21:45:35,976][105620] Updated weights for policy 1, policy_version 878812 (0.0009) [2023-12-26 21:45:36,042][105620] Updated weights for policy 1, policy_version 878822 (0.0009) [2023-12-26 21:45:36,052][105586] KL-divergence is very high: 152.3821 [2023-12-26 21:45:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 450011136. Throughput: 0: 9550.1, 1: 9930.3. Samples: 450006680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:45:36,063][104569] Avg episode reward: [(0, '8281.911'), (1, '8732.368')] [2023-12-26 21:45:36,088][105692] Updated weights for policy 0, policy_version 878806 (0.0010) [2023-12-26 21:45:36,104][105620] Updated weights for policy 1, policy_version 878832 (0.0008) [2023-12-26 21:45:36,106][105586] KL-divergence is very high: 151.8114 [2023-12-26 21:45:36,152][105692] Updated weights for policy 0, policy_version 878816 (0.0008) [2023-12-26 21:45:36,213][105692] Updated weights for policy 0, policy_version 878826 (0.0005) [2023-12-26 21:45:36,884][105620] Updated weights for policy 1, policy_version 878842 (0.0008) [2023-12-26 21:45:36,895][105692] Updated weights for policy 0, policy_version 878836 (0.0006) [2023-12-26 21:45:36,943][105620] Updated weights for policy 1, policy_version 878852 (0.0007) [2023-12-26 21:45:36,953][105692] Updated weights for policy 0, policy_version 878846 (0.0006) [2023-12-26 21:45:37,002][105620] Updated weights for policy 1, policy_version 878862 (0.0008) [2023-12-26 21:45:37,013][105692] Updated weights for policy 0, policy_version 878856 (0.0005) [2023-12-26 21:45:37,064][105620] Updated weights for policy 1, policy_version 878872 (0.0008) [2023-12-26 21:45:37,661][105620] Updated weights for policy 1, policy_version 878882 (0.0008) [2023-12-26 21:45:37,711][105620] Updated weights for policy 1, policy_version 878892 (0.0008) [2023-12-26 21:45:37,766][105620] Updated weights for policy 1, policy_version 878902 (0.0008) [2023-12-26 21:45:37,819][105692] Updated weights for policy 0, policy_version 878866 (0.0006) [2023-12-26 21:45:37,874][105692] Updated weights for policy 0, policy_version 878876 (0.0010) [2023-12-26 21:45:37,928][105692] Updated weights for policy 0, policy_version 878886 (0.0010) [2023-12-26 21:45:37,989][105692] Updated weights for policy 0, policy_version 878896 (0.0009) [2023-12-26 21:45:38,494][105620] Updated weights for policy 1, policy_version 878912 (0.0009) [2023-12-26 21:45:38,544][105620] Updated weights for policy 1, policy_version 878922 (0.0009) [2023-12-26 21:45:38,602][105620] Updated weights for policy 1, policy_version 878932 (0.0009) [2023-12-26 21:45:38,747][105692] Updated weights for policy 0, policy_version 878906 (0.0005) [2023-12-26 21:45:38,810][105692] Updated weights for policy 0, policy_version 878916 (0.0005) [2023-12-26 21:45:38,874][105692] Updated weights for policy 0, policy_version 878926 (0.0008) [2023-12-26 21:45:39,442][105620] Updated weights for policy 1, policy_version 878942 (0.0009) [2023-12-26 21:45:39,503][105620] Updated weights for policy 1, policy_version 878952 (0.0009) [2023-12-26 21:45:39,550][105692] Updated weights for policy 0, policy_version 878936 (0.0006) [2023-12-26 21:45:39,568][105620] Updated weights for policy 1, policy_version 878962 (0.0009) [2023-12-26 21:45:39,600][105692] Updated weights for policy 0, policy_version 878946 (0.0009) [2023-12-26 21:45:39,655][105692] Updated weights for policy 0, policy_version 878956 (0.0007) [2023-12-26 21:45:40,344][105620] Updated weights for policy 1, policy_version 878972 (0.0007) [2023-12-26 21:45:40,402][105620] Updated weights for policy 1, policy_version 878982 (0.0009) [2023-12-26 21:45:40,446][105692] Updated weights for policy 0, policy_version 878966 (0.0009) [2023-12-26 21:45:40,460][105620] Updated weights for policy 1, policy_version 878992 (0.0007) [2023-12-26 21:45:40,495][105692] Updated weights for policy 0, policy_version 878976 (0.0006) [2023-12-26 21:45:40,543][105692] Updated weights for policy 0, policy_version 878986 (0.0005) [2023-12-26 21:45:41,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 450109440. Throughput: 0: 9465.4, 1: 9940.5. Samples: 450120280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:45:41,062][104569] Avg episode reward: [(0, '8273.918'), (1, '8729.344')] [2023-12-26 21:45:41,227][105620] Updated weights for policy 1, policy_version 879002 (0.0008) [2023-12-26 21:45:41,296][105620] Updated weights for policy 1, policy_version 879012 (0.0008) [2023-12-26 21:45:41,308][105692] Updated weights for policy 0, policy_version 878996 (0.0006) [2023-12-26 21:45:41,366][105620] Updated weights for policy 1, policy_version 879022 (0.0008) [2023-12-26 21:45:41,381][105692] Updated weights for policy 0, policy_version 879006 (0.0010) [2023-12-26 21:45:41,433][105620] Updated weights for policy 1, policy_version 879032 (0.0008) [2023-12-26 21:45:41,449][105692] Updated weights for policy 0, policy_version 879016 (0.0007) [2023-12-26 21:45:42,149][105692] Updated weights for policy 0, policy_version 879026 (0.0008) [2023-12-26 21:45:42,212][105692] Updated weights for policy 0, policy_version 879036 (0.0008) [2023-12-26 21:45:42,226][105620] Updated weights for policy 1, policy_version 879042 (0.0009) [2023-12-26 21:45:42,280][105692] Updated weights for policy 0, policy_version 879046 (0.0006) [2023-12-26 21:45:42,293][105620] Updated weights for policy 1, policy_version 879052 (0.0011) [2023-12-26 21:45:42,343][105692] Updated weights for policy 0, policy_version 879056 (0.0009) [2023-12-26 21:45:42,356][105620] Updated weights for policy 1, policy_version 879062 (0.0010) [2023-12-26 21:45:43,118][105620] Updated weights for policy 1, policy_version 879072 (0.0010) [2023-12-26 21:45:43,120][105692] Updated weights for policy 0, policy_version 879066 (0.0008) [2023-12-26 21:45:43,136][105586] KL-divergence is very high: 255.9422 [2023-12-26 21:45:43,174][105692] Updated weights for policy 0, policy_version 879076 (0.0008) [2023-12-26 21:45:43,176][105620] Updated weights for policy 1, policy_version 879082 (0.0010) [2023-12-26 21:45:43,183][105586] KL-divergence is very high: 538.6227 [2023-12-26 21:45:43,224][105692] Updated weights for policy 0, policy_version 879086 (0.0007) [2023-12-26 21:45:43,231][105586] KL-divergence is very high: 637.8445 [2023-12-26 21:45:43,238][105620] Updated weights for policy 1, policy_version 879092 (0.0008) [2023-12-26 21:45:43,868][105620] Updated weights for policy 1, policy_version 879102 (0.0008) [2023-12-26 21:45:43,924][105620] Updated weights for policy 1, policy_version 879112 (0.0010) [2023-12-26 21:45:43,972][105620] Updated weights for policy 1, policy_version 879122 (0.0010) [2023-12-26 21:45:44,022][105692] Updated weights for policy 0, policy_version 879096 (0.0006) [2023-12-26 21:45:44,084][105692] Updated weights for policy 0, policy_version 879106 (0.0005) [2023-12-26 21:45:44,141][105692] Updated weights for policy 0, policy_version 879116 (0.0005) [2023-12-26 21:45:44,736][105620] Updated weights for policy 1, policy_version 879132 (0.0010) [2023-12-26 21:45:44,788][105692] Updated weights for policy 0, policy_version 879126 (0.0007) [2023-12-26 21:45:44,798][105620] Updated weights for policy 1, policy_version 879142 (0.0010) [2023-12-26 21:45:44,846][105692] Updated weights for policy 0, policy_version 879136 (0.0006) [2023-12-26 21:45:44,855][105620] Updated weights for policy 1, policy_version 879152 (0.0008) [2023-12-26 21:45:44,892][105692] Updated weights for policy 0, policy_version 879146 (0.0008) [2023-12-26 21:45:45,424][105620] Updated weights for policy 1, policy_version 879162 (0.0008) [2023-12-26 21:45:45,493][105620] Updated weights for policy 1, policy_version 879172 (0.0006) [2023-12-26 21:45:45,558][105620] Updated weights for policy 1, policy_version 879182 (0.0011) [2023-12-26 21:45:45,622][105620] Updated weights for policy 1, policy_version 879192 (0.0011) [2023-12-26 21:45:45,767][105692] Updated weights for policy 0, policy_version 879156 (0.0009) [2023-12-26 21:45:45,825][105692] Updated weights for policy 0, policy_version 879166 (0.0008) [2023-12-26 21:45:45,878][105692] Updated weights for policy 0, policy_version 879176 (0.0008) [2023-12-26 21:45:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 450207744. Throughput: 0: 9361.0, 1: 9968.4. Samples: 450175772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:45:46,063][104569] Avg episode reward: [(0, '8899.723'), (1, '8734.567')] [2023-12-26 21:45:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000879184_225107968.pth... [2023-12-26 21:45:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000879192_225099776.pth... [2023-12-26 21:45:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000878064_224821248.pth [2023-12-26 21:45:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000878040_224804864.pth [2023-12-26 21:45:46,273][105620] Updated weights for policy 1, policy_version 879202 (0.0010) [2023-12-26 21:45:46,320][105620] Updated weights for policy 1, policy_version 879212 (0.0010) [2023-12-26 21:45:46,368][105620] Updated weights for policy 1, policy_version 879222 (0.0010) [2023-12-26 21:45:46,489][105692] Updated weights for policy 0, policy_version 879186 (0.0007) [2023-12-26 21:45:46,544][105692] Updated weights for policy 0, policy_version 879196 (0.0010) [2023-12-26 21:45:46,603][105692] Updated weights for policy 0, policy_version 879206 (0.0009) [2023-12-26 21:45:46,670][105692] Updated weights for policy 0, policy_version 879216 (0.0011) [2023-12-26 21:45:47,036][105620] Updated weights for policy 1, policy_version 879232 (0.0006) [2023-12-26 21:45:47,100][105620] Updated weights for policy 1, policy_version 879242 (0.0006) [2023-12-26 21:45:47,167][105620] Updated weights for policy 1, policy_version 879252 (0.0005) [2023-12-26 21:45:47,367][105692] Updated weights for policy 0, policy_version 879226 (0.0010) [2023-12-26 21:45:47,426][105692] Updated weights for policy 0, policy_version 879236 (0.0010) [2023-12-26 21:45:47,485][105692] Updated weights for policy 0, policy_version 879246 (0.0010) [2023-12-26 21:45:47,804][105620] Updated weights for policy 1, policy_version 879262 (0.0007) [2023-12-26 21:45:47,869][105620] Updated weights for policy 1, policy_version 879272 (0.0008) [2023-12-26 21:45:47,931][105620] Updated weights for policy 1, policy_version 879282 (0.0010) [2023-12-26 21:45:48,191][105692] Updated weights for policy 0, policy_version 879256 (0.0010) [2023-12-26 21:45:48,243][105692] Updated weights for policy 0, policy_version 879266 (0.0010) [2023-12-26 21:45:48,313][105692] Updated weights for policy 0, policy_version 879276 (0.0011) [2023-12-26 21:45:48,632][105620] Updated weights for policy 1, policy_version 879292 (0.0011) [2023-12-26 21:45:48,695][105620] Updated weights for policy 1, policy_version 879302 (0.0010) [2023-12-26 21:45:48,758][105620] Updated weights for policy 1, policy_version 879312 (0.0010) [2023-12-26 21:45:49,068][105692] Updated weights for policy 0, policy_version 879286 (0.0009) [2023-12-26 21:45:49,124][105692] Updated weights for policy 0, policy_version 879296 (0.0008) [2023-12-26 21:45:49,175][105692] Updated weights for policy 0, policy_version 879306 (0.0007) [2023-12-26 21:45:49,512][105620] Updated weights for policy 1, policy_version 879322 (0.0010) [2023-12-26 21:45:49,572][105620] Updated weights for policy 1, policy_version 879332 (0.0006) [2023-12-26 21:45:49,628][105620] Updated weights for policy 1, policy_version 879342 (0.0006) [2023-12-26 21:45:49,689][105620] Updated weights for policy 1, policy_version 879352 (0.0005) [2023-12-26 21:45:50,032][105692] Updated weights for policy 0, policy_version 879316 (0.0008) [2023-12-26 21:45:50,088][105692] Updated weights for policy 0, policy_version 879326 (0.0011) [2023-12-26 21:45:50,147][105692] Updated weights for policy 0, policy_version 879336 (0.0009) [2023-12-26 21:45:50,378][105620] Updated weights for policy 1, policy_version 879362 (0.0008) [2023-12-26 21:45:50,444][105620] Updated weights for policy 1, policy_version 879372 (0.0008) [2023-12-26 21:45:50,498][105620] Updated weights for policy 1, policy_version 879382 (0.0008) [2023-12-26 21:45:50,956][105692] Updated weights for policy 0, policy_version 879346 (0.0009) [2023-12-26 21:45:51,021][105692] Updated weights for policy 0, policy_version 879356 (0.0009) [2023-12-26 21:45:51,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 450297856. Throughput: 0: 9402.7, 1: 10003.4. Samples: 450294028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:45:51,063][104569] Avg episode reward: [(0, '8363.990'), (1, '8318.950')] [2023-12-26 21:45:51,091][105692] Updated weights for policy 0, policy_version 879366 (0.0009) [2023-12-26 21:45:51,167][105692] Updated weights for policy 0, policy_version 879376 (0.0008) [2023-12-26 21:45:51,319][105620] Updated weights for policy 1, policy_version 879392 (0.0009) [2023-12-26 21:45:51,393][105620] Updated weights for policy 1, policy_version 879402 (0.0010) [2023-12-26 21:45:51,495][105620] Updated weights for policy 1, policy_version 879412 (0.0008) [2023-12-26 21:45:52,035][105692] Updated weights for policy 0, policy_version 879386 (0.0008) [2023-12-26 21:45:52,098][105692] Updated weights for policy 0, policy_version 879396 (0.0009) [2023-12-26 21:45:52,164][105692] Updated weights for policy 0, policy_version 879406 (0.0010) [2023-12-26 21:45:52,199][105620] Updated weights for policy 1, policy_version 879422 (0.0008) [2023-12-26 21:45:52,260][105620] Updated weights for policy 1, policy_version 879432 (0.0009) [2023-12-26 21:45:52,323][105620] Updated weights for policy 1, policy_version 879442 (0.0008) [2023-12-26 21:45:52,892][105692] Updated weights for policy 0, policy_version 879416 (0.0008) [2023-12-26 21:45:52,953][105692] Updated weights for policy 0, policy_version 879426 (0.0009) [2023-12-26 21:45:53,012][105692] Updated weights for policy 0, policy_version 879436 (0.0009) [2023-12-26 21:45:53,101][105620] Updated weights for policy 1, policy_version 879452 (0.0009) [2023-12-26 21:45:53,164][105620] Updated weights for policy 1, policy_version 879462 (0.0009) [2023-12-26 21:45:53,219][105620] Updated weights for policy 1, policy_version 879472 (0.0009) [2023-12-26 21:45:53,796][105692] Updated weights for policy 0, policy_version 879446 (0.0009) [2023-12-26 21:45:53,846][105692] Updated weights for policy 0, policy_version 879456 (0.0009) [2023-12-26 21:45:53,886][105620] Updated weights for policy 1, policy_version 879482 (0.0009) [2023-12-26 21:45:53,903][105692] Updated weights for policy 0, policy_version 879466 (0.0008) [2023-12-26 21:45:53,950][105620] Updated weights for policy 1, policy_version 879492 (0.0006) [2023-12-26 21:45:54,015][105620] Updated weights for policy 1, policy_version 879502 (0.0006) [2023-12-26 21:45:54,083][105620] Updated weights for policy 1, policy_version 879512 (0.0005) [2023-12-26 21:45:54,688][105692] Updated weights for policy 0, policy_version 879476 (0.0009) [2023-12-26 21:45:54,736][105692] Updated weights for policy 0, policy_version 879486 (0.0008) [2023-12-26 21:45:54,763][105620] Updated weights for policy 1, policy_version 879522 (0.0009) [2023-12-26 21:45:54,786][105692] Updated weights for policy 0, policy_version 879496 (0.0007) [2023-12-26 21:45:54,822][105620] Updated weights for policy 1, policy_version 879532 (0.0007) [2023-12-26 21:45:54,883][105620] Updated weights for policy 1, policy_version 879542 (0.0009) [2023-12-26 21:45:55,488][105620] Updated weights for policy 1, policy_version 879552 (0.0009) [2023-12-26 21:45:55,540][105620] Updated weights for policy 1, policy_version 879562 (0.0008) [2023-12-26 21:45:55,596][105620] Updated weights for policy 1, policy_version 879572 (0.0005) [2023-12-26 21:45:55,650][105692] Updated weights for policy 0, policy_version 879506 (0.0008) [2023-12-26 21:45:55,699][105692] Updated weights for policy 0, policy_version 879516 (0.0009) [2023-12-26 21:45:55,748][105692] Updated weights for policy 0, policy_version 879526 (0.0009) [2023-12-26 21:45:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 450396160. Throughput: 0: 9320.8, 1: 9880.7. Samples: 450404440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:45:56,063][104569] Avg episode reward: [(0, '8173.532'), (1, '8713.975')] [2023-12-26 21:45:56,254][105620] Updated weights for policy 1, policy_version 879582 (0.0006) [2023-12-26 21:45:56,317][105620] Updated weights for policy 1, policy_version 879592 (0.0007) [2023-12-26 21:45:56,375][105620] Updated weights for policy 1, policy_version 879602 (0.0010) [2023-12-26 21:45:56,592][105692] Updated weights for policy 0, policy_version 879537 (0.0010) [2023-12-26 21:45:56,640][105692] Updated weights for policy 0, policy_version 879547 (0.0009) [2023-12-26 21:45:56,689][105692] Updated weights for policy 0, policy_version 879557 (0.0009) [2023-12-26 21:45:56,737][105692] Updated weights for policy 0, policy_version 879567 (0.0009) [2023-12-26 21:45:57,053][105620] Updated weights for policy 1, policy_version 879612 (0.0008) [2023-12-26 21:45:57,110][105620] Updated weights for policy 1, policy_version 879622 (0.0005) [2023-12-26 21:45:57,183][105620] Updated weights for policy 1, policy_version 879632 (0.0006) [2023-12-26 21:45:57,604][105692] Updated weights for policy 0, policy_version 879577 (0.0007) [2023-12-26 21:45:57,661][105692] Updated weights for policy 0, policy_version 879587 (0.0006) [2023-12-26 21:45:57,711][105692] Updated weights for policy 0, policy_version 879597 (0.0005) [2023-12-26 21:45:57,758][105620] Updated weights for policy 1, policy_version 879642 (0.0006) [2023-12-26 21:45:57,817][105620] Updated weights for policy 1, policy_version 879652 (0.0005) [2023-12-26 21:45:57,873][105620] Updated weights for policy 1, policy_version 879662 (0.0005) [2023-12-26 21:45:57,937][105620] Updated weights for policy 1, policy_version 879672 (0.0007) [2023-12-26 21:45:58,331][105692] Updated weights for policy 0, policy_version 879607 (0.0007) [2023-12-26 21:45:58,393][105692] Updated weights for policy 0, policy_version 879617 (0.0007) [2023-12-26 21:45:58,455][105692] Updated weights for policy 0, policy_version 879627 (0.0008) [2023-12-26 21:45:58,662][105620] Updated weights for policy 1, policy_version 879682 (0.0011) [2023-12-26 21:45:58,725][105620] Updated weights for policy 1, policy_version 879692 (0.0010) [2023-12-26 21:45:58,793][105620] Updated weights for policy 1, policy_version 879702 (0.0010) [2023-12-26 21:45:59,299][105692] Updated weights for policy 0, policy_version 879637 (0.0009) [2023-12-26 21:45:59,367][105692] Updated weights for policy 0, policy_version 879647 (0.0010) [2023-12-26 21:45:59,422][105692] Updated weights for policy 0, policy_version 879657 (0.0007) [2023-12-26 21:45:59,567][105620] Updated weights for policy 1, policy_version 879712 (0.0011) [2023-12-26 21:45:59,628][105620] Updated weights for policy 1, policy_version 879722 (0.0011) [2023-12-26 21:45:59,687][105620] Updated weights for policy 1, policy_version 879732 (0.0010) [2023-12-26 21:46:00,176][105692] Updated weights for policy 0, policy_version 879667 (0.0009) [2023-12-26 21:46:00,227][105692] Updated weights for policy 0, policy_version 879677 (0.0010) [2023-12-26 21:46:00,288][105692] Updated weights for policy 0, policy_version 879687 (0.0010) [2023-12-26 21:46:00,439][105620] Updated weights for policy 1, policy_version 879742 (0.0009) [2023-12-26 21:46:00,491][105620] Updated weights for policy 1, policy_version 879752 (0.0006) [2023-12-26 21:46:00,546][105620] Updated weights for policy 1, policy_version 879762 (0.0005) [2023-12-26 21:46:01,023][105692] Updated weights for policy 0, policy_version 879697 (0.0010) [2023-12-26 21:46:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.1, 300 sec: 19494.2). Total num frames: 450486272. Throughput: 0: 9359.8, 1: 9850.4. Samples: 450461980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:46:01,063][104569] Avg episode reward: [(0, '7802.012'), (1, '8545.464')] [2023-12-26 21:46:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000879768_225247232.pth... [2023-12-26 21:46:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000878648_224960512.pth [2023-12-26 21:46:01,092][105692] Updated weights for policy 0, policy_version 879707 (0.0010) [2023-12-26 21:46:01,153][105692] Updated weights for policy 0, policy_version 879717 (0.0010) [2023-12-26 21:46:01,215][105692] Updated weights for policy 0, policy_version 879727 (0.0010) [2023-12-26 21:46:01,217][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000879728_225247232.pth... [2023-12-26 21:46:01,218][105620] Updated weights for policy 1, policy_version 879772 (0.0007) [2023-12-26 21:46:01,221][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000878640_224968704.pth [2023-12-26 21:46:01,278][105620] Updated weights for policy 1, policy_version 879782 (0.0008) [2023-12-26 21:46:01,346][105620] Updated weights for policy 1, policy_version 879792 (0.0008) [2023-12-26 21:46:01,912][105692] Updated weights for policy 0, policy_version 879737 (0.0008) [2023-12-26 21:46:01,960][105692] Updated weights for policy 0, policy_version 879747 (0.0008) [2023-12-26 21:46:01,981][105620] Updated weights for policy 1, policy_version 879802 (0.0008) [2023-12-26 21:46:02,018][105692] Updated weights for policy 0, policy_version 879757 (0.0008) [2023-12-26 21:46:02,044][105620] Updated weights for policy 1, policy_version 879812 (0.0008) [2023-12-26 21:46:02,096][105620] Updated weights for policy 1, policy_version 879822 (0.0007) [2023-12-26 21:46:02,162][105620] Updated weights for policy 1, policy_version 879832 (0.0006) [2023-12-26 21:46:02,770][105692] Updated weights for policy 0, policy_version 879767 (0.0008) [2023-12-26 21:46:02,830][105692] Updated weights for policy 0, policy_version 879777 (0.0008) [2023-12-26 21:46:02,855][105620] Updated weights for policy 1, policy_version 879842 (0.0011) [2023-12-26 21:46:02,885][105692] Updated weights for policy 0, policy_version 879787 (0.0009) [2023-12-26 21:46:02,907][105620] Updated weights for policy 1, policy_version 879852 (0.0010) [2023-12-26 21:46:02,935][105586] KL-divergence is very high: 113.5655 [2023-12-26 21:46:02,962][105620] Updated weights for policy 1, policy_version 879862 (0.0011) [2023-12-26 21:46:03,479][105692] Updated weights for policy 0, policy_version 879797 (0.0010) [2023-12-26 21:46:03,535][105692] Updated weights for policy 0, policy_version 879807 (0.0005) [2023-12-26 21:46:03,592][105692] Updated weights for policy 0, policy_version 879817 (0.0006) [2023-12-26 21:46:03,712][105620] Updated weights for policy 1, policy_version 879872 (0.0010) [2023-12-26 21:46:03,773][105620] Updated weights for policy 1, policy_version 879882 (0.0010) [2023-12-26 21:46:03,821][105620] Updated weights for policy 1, policy_version 879892 (0.0010) [2023-12-26 21:46:04,367][105692] Updated weights for policy 0, policy_version 879827 (0.0009) [2023-12-26 21:46:04,431][105692] Updated weights for policy 0, policy_version 879837 (0.0008) [2023-12-26 21:46:04,498][105692] Updated weights for policy 0, policy_version 879847 (0.0010) [2023-12-26 21:46:04,530][105620] Updated weights for policy 1, policy_version 879902 (0.0008) [2023-12-26 21:46:04,592][105620] Updated weights for policy 1, policy_version 879912 (0.0006) [2023-12-26 21:46:04,651][105620] Updated weights for policy 1, policy_version 879922 (0.0007) [2023-12-26 21:46:05,238][105692] Updated weights for policy 0, policy_version 879857 (0.0009) [2023-12-26 21:46:05,293][105692] Updated weights for policy 0, policy_version 879867 (0.0008) [2023-12-26 21:46:05,358][105620] Updated weights for policy 1, policy_version 879932 (0.0009) [2023-12-26 21:46:05,359][105692] Updated weights for policy 0, policy_version 879877 (0.0009) [2023-12-26 21:46:05,407][105692] Updated weights for policy 0, policy_version 879887 (0.0009) [2023-12-26 21:46:05,414][105620] Updated weights for policy 1, policy_version 879942 (0.0007) [2023-12-26 21:46:05,469][105620] Updated weights for policy 1, policy_version 879952 (0.0010) [2023-12-26 21:46:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 450584576. Throughput: 0: 9456.5, 1: 9796.3. Samples: 450577980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:46:06,062][104569] Avg episode reward: [(0, '8420.253'), (1, '8367.850')] [2023-12-26 21:46:06,129][105692] Updated weights for policy 0, policy_version 879897 (0.0009) [2023-12-26 21:46:06,158][105620] Updated weights for policy 1, policy_version 879962 (0.0009) [2023-12-26 21:46:06,195][105692] Updated weights for policy 0, policy_version 879907 (0.0010) [2023-12-26 21:46:06,218][105620] Updated weights for policy 1, policy_version 879972 (0.0011) [2023-12-26 21:46:06,257][105692] Updated weights for policy 0, policy_version 879917 (0.0011) [2023-12-26 21:46:06,271][105620] Updated weights for policy 1, policy_version 879982 (0.0010) [2023-12-26 21:46:06,326][105620] Updated weights for policy 1, policy_version 879992 (0.0010) [2023-12-26 21:46:06,932][105692] Updated weights for policy 0, policy_version 879927 (0.0010) [2023-12-26 21:46:06,980][105620] Updated weights for policy 1, policy_version 880002 (0.0011) [2023-12-26 21:46:06,997][105692] Updated weights for policy 0, policy_version 879937 (0.0008) [2023-12-26 21:46:07,040][105620] Updated weights for policy 1, policy_version 880012 (0.0011) [2023-12-26 21:46:07,060][105692] Updated weights for policy 0, policy_version 879947 (0.0006) [2023-12-26 21:46:07,102][105620] Updated weights for policy 1, policy_version 880022 (0.0009) [2023-12-26 21:46:07,727][105692] Updated weights for policy 0, policy_version 879957 (0.0010) [2023-12-26 21:46:07,783][105692] Updated weights for policy 0, policy_version 879967 (0.0009) [2023-12-26 21:46:07,836][105620] Updated weights for policy 1, policy_version 880032 (0.0010) [2023-12-26 21:46:07,842][105692] Updated weights for policy 0, policy_version 879977 (0.0008) [2023-12-26 21:46:07,891][105620] Updated weights for policy 1, policy_version 880042 (0.0010) [2023-12-26 21:46:07,944][105620] Updated weights for policy 1, policy_version 880052 (0.0006) [2023-12-26 21:46:08,491][105692] Updated weights for policy 0, policy_version 879987 (0.0008) [2023-12-26 21:46:08,516][105620] Updated weights for policy 1, policy_version 880062 (0.0005) [2023-12-26 21:46:08,551][105692] Updated weights for policy 0, policy_version 879997 (0.0008) [2023-12-26 21:46:08,573][105620] Updated weights for policy 1, policy_version 880072 (0.0005) [2023-12-26 21:46:08,617][105692] Updated weights for policy 0, policy_version 880007 (0.0008) [2023-12-26 21:46:08,634][105620] Updated weights for policy 1, policy_version 880082 (0.0005) [2023-12-26 21:46:09,175][105620] Updated weights for policy 1, policy_version 880092 (0.0006) [2023-12-26 21:46:09,240][105620] Updated weights for policy 1, policy_version 880102 (0.0008) [2023-12-26 21:46:09,293][105620] Updated weights for policy 1, policy_version 880112 (0.0006) [2023-12-26 21:46:09,490][105692] Updated weights for policy 0, policy_version 880017 (0.0009) [2023-12-26 21:46:09,542][105692] Updated weights for policy 0, policy_version 880027 (0.0008) [2023-12-26 21:46:09,602][105692] Updated weights for policy 0, policy_version 880037 (0.0008) [2023-12-26 21:46:09,662][105692] Updated weights for policy 0, policy_version 880047 (0.0009) [2023-12-26 21:46:10,026][105620] Updated weights for policy 1, policy_version 880122 (0.0008) [2023-12-26 21:46:10,075][105620] Updated weights for policy 1, policy_version 880132 (0.0010) [2023-12-26 21:46:10,131][105620] Updated weights for policy 1, policy_version 880142 (0.0010) [2023-12-26 21:46:10,194][105620] Updated weights for policy 1, policy_version 880152 (0.0010) [2023-12-26 21:46:10,467][105692] Updated weights for policy 0, policy_version 880057 (0.0009) [2023-12-26 21:46:10,516][105692] Updated weights for policy 0, policy_version 880067 (0.0007) [2023-12-26 21:46:10,569][105692] Updated weights for policy 0, policy_version 880077 (0.0010) [2023-12-26 21:46:10,976][105620] Updated weights for policy 1, policy_version 880162 (0.0009) [2023-12-26 21:46:11,032][105620] Updated weights for policy 1, policy_version 880172 (0.0008) [2023-12-26 21:46:11,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19114.7, 300 sec: 19549.7). Total num frames: 450682880. Throughput: 0: 9440.4, 1: 9834.4. Samples: 450696524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:46:11,063][104569] Avg episode reward: [(0, '9171.831'), (1, '8539.788')] [2023-12-26 21:46:11,097][105620] Updated weights for policy 1, policy_version 880182 (0.0008) [2023-12-26 21:46:11,339][105692] Updated weights for policy 0, policy_version 880087 (0.0011) [2023-12-26 21:46:11,413][105692] Updated weights for policy 0, policy_version 880097 (0.0011) [2023-12-26 21:46:11,479][105692] Updated weights for policy 0, policy_version 880107 (0.0011) [2023-12-26 21:46:11,891][105620] Updated weights for policy 1, policy_version 880192 (0.0008) [2023-12-26 21:46:11,944][105620] Updated weights for policy 1, policy_version 880202 (0.0008) [2023-12-26 21:46:12,001][105620] Updated weights for policy 1, policy_version 880212 (0.0008) [2023-12-26 21:46:12,213][105692] Updated weights for policy 0, policy_version 880117 (0.0011) [2023-12-26 21:46:12,272][105692] Updated weights for policy 0, policy_version 880127 (0.0010) [2023-12-26 21:46:12,313][105585] KL-divergence is very high: 126.9038 [2023-12-26 21:46:12,319][105585] KL-divergence is very high: 146.5736 [2023-12-26 21:46:12,328][105692] Updated weights for policy 0, policy_version 880137 (0.0008) [2023-12-26 21:46:12,361][105585] KL-divergence is very high: 243.3969 [2023-12-26 21:46:12,369][105585] KL-divergence is very high: 253.2943 [2023-12-26 21:46:12,791][105620] Updated weights for policy 1, policy_version 880222 (0.0008) [2023-12-26 21:46:12,850][105620] Updated weights for policy 1, policy_version 880232 (0.0008) [2023-12-26 21:46:12,909][105620] Updated weights for policy 1, policy_version 880242 (0.0008) [2023-12-26 21:46:13,104][105585] KL-divergence is very high: 261.9716 [2023-12-26 21:46:13,104][105692] Updated weights for policy 0, policy_version 880147 (0.0010) [2023-12-26 21:46:13,143][105585] KL-divergence is very high: 270.6836 [2023-12-26 21:46:13,156][105692] Updated weights for policy 0, policy_version 880157 (0.0010) [2023-12-26 21:46:13,188][105585] KL-divergence is very high: 266.6396 [2023-12-26 21:46:13,208][105692] Updated weights for policy 0, policy_version 880167 (0.0010) [2023-12-26 21:46:13,225][105585] KL-divergence is very high: 250.0814 [2023-12-26 21:46:13,661][105620] Updated weights for policy 1, policy_version 880252 (0.0009) [2023-12-26 21:46:13,725][105620] Updated weights for policy 1, policy_version 880262 (0.0009) [2023-12-26 21:46:13,783][105620] Updated weights for policy 1, policy_version 880272 (0.0009) [2023-12-26 21:46:13,915][105692] Updated weights for policy 0, policy_version 880177 (0.0010) [2023-12-26 21:46:13,978][105692] Updated weights for policy 0, policy_version 880187 (0.0007) [2023-12-26 21:46:14,036][105692] Updated weights for policy 0, policy_version 880197 (0.0009) [2023-12-26 21:46:14,091][105692] Updated weights for policy 0, policy_version 880207 (0.0009) [2023-12-26 21:46:14,455][105620] Updated weights for policy 1, policy_version 880282 (0.0010) [2023-12-26 21:46:14,513][105620] Updated weights for policy 1, policy_version 880292 (0.0009) [2023-12-26 21:46:14,568][105620] Updated weights for policy 1, policy_version 880302 (0.0009) [2023-12-26 21:46:14,626][105620] Updated weights for policy 1, policy_version 880312 (0.0009) [2023-12-26 21:46:14,836][105692] Updated weights for policy 0, policy_version 880217 (0.0009) [2023-12-26 21:46:14,894][105692] Updated weights for policy 0, policy_version 880227 (0.0008) [2023-12-26 21:46:14,957][105692] Updated weights for policy 0, policy_version 880237 (0.0009) [2023-12-26 21:46:15,411][105620] Updated weights for policy 1, policy_version 880322 (0.0009) [2023-12-26 21:46:15,468][105620] Updated weights for policy 1, policy_version 880332 (0.0009) [2023-12-26 21:46:15,526][105620] Updated weights for policy 1, policy_version 880342 (0.0010) [2023-12-26 21:46:15,636][105692] Updated weights for policy 0, policy_version 880247 (0.0006) [2023-12-26 21:46:15,707][105692] Updated weights for policy 0, policy_version 880257 (0.0005) [2023-12-26 21:46:15,777][105692] Updated weights for policy 0, policy_version 880267 (0.0005) [2023-12-26 21:46:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 450781184. Throughput: 0: 9389.8, 1: 9701.0. Samples: 450750664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:46:16,062][104569] Avg episode reward: [(0, '8995.894'), (1, '8602.623')] [2023-12-26 21:46:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000880272_225386496.pth... [2023-12-26 21:46:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000880344_225394688.pth... [2023-12-26 21:46:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000879184_225107968.pth [2023-12-26 21:46:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000879192_225099776.pth [2023-12-26 21:46:16,285][105692] Updated weights for policy 0, policy_version 880277 (0.0005) [2023-12-26 21:46:16,341][105692] Updated weights for policy 0, policy_version 880287 (0.0005) [2023-12-26 21:46:16,400][105692] Updated weights for policy 0, policy_version 880297 (0.0005) [2023-12-26 21:46:16,423][105620] Updated weights for policy 1, policy_version 880352 (0.0008) [2023-12-26 21:46:16,480][105620] Updated weights for policy 1, policy_version 880362 (0.0009) [2023-12-26 21:46:16,531][105620] Updated weights for policy 1, policy_version 880372 (0.0009) [2023-12-26 21:46:16,983][105692] Updated weights for policy 0, policy_version 880307 (0.0006) [2023-12-26 21:46:17,037][105692] Updated weights for policy 0, policy_version 880317 (0.0007) [2023-12-26 21:46:17,089][105692] Updated weights for policy 0, policy_version 880327 (0.0010) [2023-12-26 21:46:17,267][105620] Updated weights for policy 1, policy_version 880382 (0.0008) [2023-12-26 21:46:17,314][105620] Updated weights for policy 1, policy_version 880392 (0.0007) [2023-12-26 21:46:17,369][105620] Updated weights for policy 1, policy_version 880402 (0.0008) [2023-12-26 21:46:17,702][105692] Updated weights for policy 0, policy_version 880337 (0.0010) [2023-12-26 21:46:17,763][105692] Updated weights for policy 0, policy_version 880347 (0.0010) [2023-12-26 21:46:17,829][105692] Updated weights for policy 0, policy_version 880357 (0.0011) [2023-12-26 21:46:17,897][105692] Updated weights for policy 0, policy_version 880367 (0.0010) [2023-12-26 21:46:18,103][105620] Updated weights for policy 1, policy_version 880412 (0.0009) [2023-12-26 21:46:18,154][105620] Updated weights for policy 1, policy_version 880422 (0.0010) [2023-12-26 21:46:18,209][105620] Updated weights for policy 1, policy_version 880432 (0.0010) [2023-12-26 21:46:18,639][105692] Updated weights for policy 0, policy_version 880377 (0.0011) [2023-12-26 21:46:18,697][105692] Updated weights for policy 0, policy_version 880387 (0.0011) [2023-12-26 21:46:18,759][105692] Updated weights for policy 0, policy_version 880397 (0.0010) [2023-12-26 21:46:18,975][105620] Updated weights for policy 1, policy_version 880442 (0.0010) [2023-12-26 21:46:19,033][105620] Updated weights for policy 1, policy_version 880452 (0.0010) [2023-12-26 21:46:19,095][105620] Updated weights for policy 1, policy_version 880462 (0.0010) [2023-12-26 21:46:19,156][105620] Updated weights for policy 1, policy_version 880472 (0.0010) [2023-12-26 21:46:19,527][105692] Updated weights for policy 0, policy_version 880407 (0.0009) [2023-12-26 21:46:19,592][105692] Updated weights for policy 0, policy_version 880417 (0.0009) [2023-12-26 21:46:19,647][105692] Updated weights for policy 0, policy_version 880427 (0.0008) [2023-12-26 21:46:19,860][105620] Updated weights for policy 1, policy_version 880482 (0.0007) [2023-12-26 21:46:19,927][105620] Updated weights for policy 1, policy_version 880492 (0.0008) [2023-12-26 21:46:19,991][105620] Updated weights for policy 1, policy_version 880502 (0.0007) [2023-12-26 21:46:20,535][105692] Updated weights for policy 0, policy_version 880437 (0.0009) [2023-12-26 21:46:20,583][105620] Updated weights for policy 1, policy_version 880512 (0.0007) [2023-12-26 21:46:20,605][105692] Updated weights for policy 0, policy_version 880447 (0.0009) [2023-12-26 21:46:20,644][105620] Updated weights for policy 1, policy_version 880522 (0.0007) [2023-12-26 21:46:20,674][105692] Updated weights for policy 0, policy_version 880457 (0.0009) [2023-12-26 21:46:20,706][105620] Updated weights for policy 1, policy_version 880532 (0.0006) [2023-12-26 21:46:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 450879488. Throughput: 0: 9447.6, 1: 9711.5. Samples: 450868836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:46:21,063][104569] Avg episode reward: [(0, '9084.516'), (1, '8787.394')] [2023-12-26 21:46:21,429][105620] Updated weights for policy 1, policy_version 880542 (0.0007) [2023-12-26 21:46:21,485][105620] Updated weights for policy 1, policy_version 880552 (0.0006) [2023-12-26 21:46:21,511][105692] Updated weights for policy 0, policy_version 880467 (0.0008) [2023-12-26 21:46:21,550][105620] Updated weights for policy 1, policy_version 880562 (0.0008) [2023-12-26 21:46:21,570][105692] Updated weights for policy 0, policy_version 880477 (0.0007) [2023-12-26 21:46:21,640][105692] Updated weights for policy 0, policy_version 880487 (0.0007) [2023-12-26 21:46:22,252][105692] Updated weights for policy 0, policy_version 880497 (0.0010) [2023-12-26 21:46:22,324][105692] Updated weights for policy 0, policy_version 880507 (0.0010) [2023-12-26 21:46:22,340][105620] Updated weights for policy 1, policy_version 880572 (0.0008) [2023-12-26 21:46:22,394][105692] Updated weights for policy 0, policy_version 880517 (0.0008) [2023-12-26 21:46:22,404][105620] Updated weights for policy 1, policy_version 880582 (0.0008) [2023-12-26 21:46:22,458][105692] Updated weights for policy 0, policy_version 880527 (0.0011) [2023-12-26 21:46:22,464][105620] Updated weights for policy 1, policy_version 880592 (0.0007) [2023-12-26 21:46:23,208][105692] Updated weights for policy 0, policy_version 880537 (0.0011) [2023-12-26 21:46:23,238][105620] Updated weights for policy 1, policy_version 880602 (0.0008) [2023-12-26 21:46:23,265][105692] Updated weights for policy 0, policy_version 880547 (0.0010) [2023-12-26 21:46:23,295][105620] Updated weights for policy 1, policy_version 880612 (0.0006) [2023-12-26 21:46:23,321][105692] Updated weights for policy 0, policy_version 880557 (0.0010) [2023-12-26 21:46:23,356][105620] Updated weights for policy 1, policy_version 880622 (0.0006) [2023-12-26 21:46:23,415][105620] Updated weights for policy 1, policy_version 880632 (0.0008) [2023-12-26 21:46:24,032][105692] Updated weights for policy 0, policy_version 880567 (0.0007) [2023-12-26 21:46:24,051][105620] Updated weights for policy 1, policy_version 880642 (0.0009) [2023-12-26 21:46:24,092][105692] Updated weights for policy 0, policy_version 880577 (0.0010) [2023-12-26 21:46:24,107][105620] Updated weights for policy 1, policy_version 880652 (0.0006) [2023-12-26 21:46:24,151][105692] Updated weights for policy 0, policy_version 880587 (0.0010) [2023-12-26 21:46:24,161][105620] Updated weights for policy 1, policy_version 880662 (0.0006) [2023-12-26 21:46:24,812][105692] Updated weights for policy 0, policy_version 880597 (0.0008) [2023-12-26 21:46:24,820][105620] Updated weights for policy 1, policy_version 880672 (0.0008) [2023-12-26 21:46:24,869][105692] Updated weights for policy 0, policy_version 880607 (0.0005) [2023-12-26 21:46:24,884][105620] Updated weights for policy 1, policy_version 880682 (0.0009) [2023-12-26 21:46:24,924][105692] Updated weights for policy 0, policy_version 880617 (0.0005) [2023-12-26 21:46:24,944][105620] Updated weights for policy 1, policy_version 880692 (0.0009) [2023-12-26 21:46:25,524][105692] Updated weights for policy 0, policy_version 880627 (0.0007) [2023-12-26 21:46:25,531][105620] Updated weights for policy 1, policy_version 880702 (0.0006) [2023-12-26 21:46:25,572][105692] Updated weights for policy 0, policy_version 880638 (0.0010) [2023-12-26 21:46:25,605][105620] Updated weights for policy 1, policy_version 880712 (0.0005) [2023-12-26 21:46:25,615][105692] Updated weights for policy 0, policy_version 880648 (0.0009) [2023-12-26 21:46:25,665][105620] Updated weights for policy 1, policy_version 880722 (0.0005) [2023-12-26 21:46:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 450977792. Throughput: 0: 9459.7, 1: 9822.4. Samples: 450987976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:46:26,062][104569] Avg episode reward: [(0, '8994.925'), (1, '8992.276')] [2023-12-26 21:46:26,145][105620] Updated weights for policy 1, policy_version 880732 (0.0005) [2023-12-26 21:46:26,181][105692] Updated weights for policy 0, policy_version 880658 (0.0005) [2023-12-26 21:46:26,198][105620] Updated weights for policy 1, policy_version 880742 (0.0009) [2023-12-26 21:46:26,230][105692] Updated weights for policy 0, policy_version 880668 (0.0008) [2023-12-26 21:46:26,250][105620] Updated weights for policy 1, policy_version 880752 (0.0010) [2023-12-26 21:46:26,282][105692] Updated weights for policy 0, policy_version 880678 (0.0010) [2023-12-26 21:46:26,330][105692] Updated weights for policy 0, policy_version 880688 (0.0010) [2023-12-26 21:46:26,869][105620] Updated weights for policy 1, policy_version 880762 (0.0009) [2023-12-26 21:46:26,902][105692] Updated weights for policy 0, policy_version 880698 (0.0005) [2023-12-26 21:46:26,919][105620] Updated weights for policy 1, policy_version 880772 (0.0009) [2023-12-26 21:46:26,957][105692] Updated weights for policy 0, policy_version 880708 (0.0005) [2023-12-26 21:46:26,979][105620] Updated weights for policy 1, policy_version 880782 (0.0010) [2023-12-26 21:46:27,011][105692] Updated weights for policy 0, policy_version 880718 (0.0005) [2023-12-26 21:46:27,031][105620] Updated weights for policy 1, policy_version 880792 (0.0008) [2023-12-26 21:46:27,656][105692] Updated weights for policy 0, policy_version 880728 (0.0005) [2023-12-26 21:46:27,723][105692] Updated weights for policy 0, policy_version 880738 (0.0005) [2023-12-26 21:46:27,742][105620] Updated weights for policy 1, policy_version 880802 (0.0009) [2023-12-26 21:46:27,780][105692] Updated weights for policy 0, policy_version 880748 (0.0005) [2023-12-26 21:46:27,797][105620] Updated weights for policy 1, policy_version 880812 (0.0009) [2023-12-26 21:46:27,854][105620] Updated weights for policy 1, policy_version 880822 (0.0008) [2023-12-26 21:46:28,279][105692] Updated weights for policy 0, policy_version 880758 (0.0005) [2023-12-26 21:46:28,341][105692] Updated weights for policy 0, policy_version 880768 (0.0006) [2023-12-26 21:46:28,403][105692] Updated weights for policy 0, policy_version 880778 (0.0006) [2023-12-26 21:46:28,745][105620] Updated weights for policy 1, policy_version 880832 (0.0008) [2023-12-26 21:46:28,809][105620] Updated weights for policy 1, policy_version 880842 (0.0008) [2023-12-26 21:46:28,863][105620] Updated weights for policy 1, policy_version 880852 (0.0008) [2023-12-26 21:46:28,988][105692] Updated weights for policy 0, policy_version 880788 (0.0007) [2023-12-26 21:46:29,049][105692] Updated weights for policy 0, policy_version 880798 (0.0010) [2023-12-26 21:46:29,109][105692] Updated weights for policy 0, policy_version 880808 (0.0010) [2023-12-26 21:46:29,596][105620] Updated weights for policy 1, policy_version 880862 (0.0009) [2023-12-26 21:46:29,647][105620] Updated weights for policy 1, policy_version 880872 (0.0010) [2023-12-26 21:46:29,704][105620] Updated weights for policy 1, policy_version 880882 (0.0006) [2023-12-26 21:46:29,825][105692] Updated weights for policy 0, policy_version 880818 (0.0010) [2023-12-26 21:46:29,887][105692] Updated weights for policy 0, policy_version 880828 (0.0008) [2023-12-26 21:46:29,951][105692] Updated weights for policy 0, policy_version 880838 (0.0006) [2023-12-26 21:46:30,014][105692] Updated weights for policy 0, policy_version 880848 (0.0005) [2023-12-26 21:46:30,452][105620] Updated weights for policy 1, policy_version 880892 (0.0010) [2023-12-26 21:46:30,514][105620] Updated weights for policy 1, policy_version 880902 (0.0010) [2023-12-26 21:46:30,576][105620] Updated weights for policy 1, policy_version 880912 (0.0010) [2023-12-26 21:46:30,618][105692] Updated weights for policy 0, policy_version 880858 (0.0006) [2023-12-26 21:46:30,670][105692] Updated weights for policy 0, policy_version 880868 (0.0008) [2023-12-26 21:46:30,714][105692] Updated weights for policy 0, policy_version 880878 (0.0008) [2023-12-26 21:46:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 451084288. Throughput: 0: 9655.1, 1: 9841.3. Samples: 451053108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:46:31,063][104569] Avg episode reward: [(0, '8832.555'), (1, '8724.547')] [2023-12-26 21:46:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000880880_225542144.pth... [2023-12-26 21:46:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000880920_225542144.pth... [2023-12-26 21:46:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000879768_225247232.pth [2023-12-26 21:46:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000879728_225247232.pth [2023-12-26 21:46:31,333][105620] Updated weights for policy 1, policy_version 880922 (0.0010) [2023-12-26 21:46:31,391][105620] Updated weights for policy 1, policy_version 880932 (0.0012) [2023-12-26 21:46:31,447][105620] Updated weights for policy 1, policy_version 880942 (0.0009) [2023-12-26 21:46:31,464][105692] Updated weights for policy 0, policy_version 880888 (0.0006) [2023-12-26 21:46:31,503][105620] Updated weights for policy 1, policy_version 880952 (0.0007) [2023-12-26 21:46:31,529][105692] Updated weights for policy 0, policy_version 880898 (0.0005) [2023-12-26 21:46:31,588][105692] Updated weights for policy 0, policy_version 880908 (0.0006) [2023-12-26 21:46:32,192][105620] Updated weights for policy 1, policy_version 880962 (0.0006) [2023-12-26 21:46:32,236][105692] Updated weights for policy 0, policy_version 880918 (0.0010) [2023-12-26 21:46:32,254][105620] Updated weights for policy 1, policy_version 880972 (0.0010) [2023-12-26 21:46:32,305][105692] Updated weights for policy 0, policy_version 880928 (0.0010) [2023-12-26 21:46:32,312][105620] Updated weights for policy 1, policy_version 880982 (0.0007) [2023-12-26 21:46:32,371][105692] Updated weights for policy 0, policy_version 880938 (0.0011) [2023-12-26 21:46:32,940][105620] Updated weights for policy 1, policy_version 880992 (0.0005) [2023-12-26 21:46:33,004][105620] Updated weights for policy 1, policy_version 881002 (0.0005) [2023-12-26 21:46:33,057][105620] Updated weights for policy 1, policy_version 881012 (0.0005) [2023-12-26 21:46:33,084][105692] Updated weights for policy 0, policy_version 880948 (0.0007) [2023-12-26 21:46:33,137][105692] Updated weights for policy 0, policy_version 880958 (0.0009) [2023-12-26 21:46:33,186][105692] Updated weights for policy 0, policy_version 880968 (0.0008) [2023-12-26 21:46:33,639][105620] Updated weights for policy 1, policy_version 881022 (0.0009) [2023-12-26 21:46:33,692][105620] Updated weights for policy 1, policy_version 881032 (0.0010) [2023-12-26 21:46:33,745][105620] Updated weights for policy 1, policy_version 881043 (0.0010) [2023-12-26 21:46:33,753][105692] Updated weights for policy 0, policy_version 880978 (0.0006) [2023-12-26 21:46:33,803][105692] Updated weights for policy 0, policy_version 880988 (0.0005) [2023-12-26 21:46:33,853][105692] Updated weights for policy 0, policy_version 880998 (0.0006) [2023-12-26 21:46:33,899][105692] Updated weights for policy 0, policy_version 881008 (0.0005) [2023-12-26 21:46:34,466][105620] Updated weights for policy 1, policy_version 881053 (0.0009) [2023-12-26 21:46:34,532][105620] Updated weights for policy 1, policy_version 881063 (0.0008) [2023-12-26 21:46:34,569][105692] Updated weights for policy 0, policy_version 881018 (0.0008) [2023-12-26 21:46:34,595][105620] Updated weights for policy 1, policy_version 881073 (0.0008) [2023-12-26 21:46:34,635][105692] Updated weights for policy 0, policy_version 881028 (0.0007) [2023-12-26 21:46:34,696][105692] Updated weights for policy 0, policy_version 881038 (0.0009) [2023-12-26 21:46:35,299][105620] Updated weights for policy 1, policy_version 881083 (0.0007) [2023-12-26 21:46:35,367][105620] Updated weights for policy 1, policy_version 881093 (0.0009) [2023-12-26 21:46:35,410][105692] Updated weights for policy 0, policy_version 881048 (0.0007) [2023-12-26 21:46:35,416][105620] Updated weights for policy 1, policy_version 881103 (0.0007) [2023-12-26 21:46:35,467][105692] Updated weights for policy 0, policy_version 881058 (0.0007) [2023-12-26 21:46:35,530][105692] Updated weights for policy 0, policy_version 881068 (0.0009) [2023-12-26 21:46:36,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 451182592. Throughput: 0: 9755.7, 1: 9833.0. Samples: 451175520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:46:36,063][104569] Avg episode reward: [(0, '9008.813'), (1, '8634.878')] [2023-12-26 21:46:36,178][105620] Updated weights for policy 1, policy_version 881113 (0.0006) [2023-12-26 21:46:36,244][105620] Updated weights for policy 1, policy_version 881123 (0.0009) [2023-12-26 21:46:36,283][105692] Updated weights for policy 0, policy_version 881078 (0.0007) [2023-12-26 21:46:36,305][105620] Updated weights for policy 1, policy_version 881133 (0.0008) [2023-12-26 21:46:36,348][105692] Updated weights for policy 0, policy_version 881088 (0.0007) [2023-12-26 21:46:36,371][105620] Updated weights for policy 1, policy_version 881143 (0.0008) [2023-12-26 21:46:36,411][105692] Updated weights for policy 0, policy_version 881098 (0.0007) [2023-12-26 21:46:37,000][105692] Updated weights for policy 0, policy_version 881108 (0.0008) [2023-12-26 21:46:37,064][105692] Updated weights for policy 0, policy_version 881118 (0.0007) [2023-12-26 21:46:37,118][105692] Updated weights for policy 0, policy_version 881128 (0.0009) [2023-12-26 21:46:37,205][105620] Updated weights for policy 1, policy_version 881153 (0.0008) [2023-12-26 21:46:37,266][105620] Updated weights for policy 1, policy_version 881163 (0.0009) [2023-12-26 21:46:37,320][105620] Updated weights for policy 1, policy_version 881173 (0.0008) [2023-12-26 21:46:37,792][105692] Updated weights for policy 0, policy_version 881138 (0.0008) [2023-12-26 21:46:37,848][105692] Updated weights for policy 0, policy_version 881148 (0.0005) [2023-12-26 21:46:37,894][105692] Updated weights for policy 0, policy_version 881158 (0.0005) [2023-12-26 21:46:37,938][105692] Updated weights for policy 0, policy_version 881168 (0.0005) [2023-12-26 21:46:38,070][105620] Updated weights for policy 1, policy_version 881183 (0.0010) [2023-12-26 21:46:38,120][105620] Updated weights for policy 1, policy_version 881193 (0.0009) [2023-12-26 21:46:38,167][105620] Updated weights for policy 1, policy_version 881203 (0.0008) [2023-12-26 21:46:38,668][105692] Updated weights for policy 0, policy_version 881178 (0.0010) [2023-12-26 21:46:38,730][105692] Updated weights for policy 0, policy_version 881188 (0.0010) [2023-12-26 21:46:38,789][105692] Updated weights for policy 0, policy_version 881198 (0.0010) [2023-12-26 21:46:38,883][105620] Updated weights for policy 1, policy_version 881213 (0.0009) [2023-12-26 21:46:38,942][105620] Updated weights for policy 1, policy_version 881223 (0.0008) [2023-12-26 21:46:39,001][105620] Updated weights for policy 1, policy_version 881233 (0.0008) [2023-12-26 21:46:39,521][105692] Updated weights for policy 0, policy_version 881208 (0.0009) [2023-12-26 21:46:39,579][105692] Updated weights for policy 0, policy_version 881218 (0.0009) [2023-12-26 21:46:39,649][105692] Updated weights for policy 0, policy_version 881228 (0.0006) [2023-12-26 21:46:39,812][105620] Updated weights for policy 1, policy_version 881243 (0.0009) [2023-12-26 21:46:39,883][105620] Updated weights for policy 1, policy_version 881253 (0.0008) [2023-12-26 21:46:39,955][105620] Updated weights for policy 1, policy_version 881263 (0.0009) [2023-12-26 21:46:40,297][105692] Updated weights for policy 0, policy_version 881238 (0.0008) [2023-12-26 21:46:40,346][105692] Updated weights for policy 0, policy_version 881248 (0.0009) [2023-12-26 21:46:40,399][105692] Updated weights for policy 0, policy_version 881258 (0.0010) [2023-12-26 21:46:40,671][105620] Updated weights for policy 1, policy_version 881273 (0.0009) [2023-12-26 21:46:40,724][105620] Updated weights for policy 1, policy_version 881283 (0.0009) [2023-12-26 21:46:40,774][105620] Updated weights for policy 1, policy_version 881293 (0.0008) [2023-12-26 21:46:40,821][105620] Updated weights for policy 1, policy_version 881303 (0.0009) [2023-12-26 21:46:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 451280896. Throughput: 0: 9897.7, 1: 9767.9. Samples: 451289388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:46:41,062][104569] Avg episode reward: [(0, '9090.215'), (1, '8911.897')] [2023-12-26 21:46:41,191][105692] Updated weights for policy 0, policy_version 881268 (0.0009) [2023-12-26 21:46:41,251][105692] Updated weights for policy 0, policy_version 881278 (0.0009) [2023-12-26 21:46:41,310][105692] Updated weights for policy 0, policy_version 881288 (0.0009) [2023-12-26 21:46:41,562][105620] Updated weights for policy 1, policy_version 881313 (0.0007) [2023-12-26 21:46:41,621][105620] Updated weights for policy 1, policy_version 881323 (0.0007) [2023-12-26 21:46:41,693][105620] Updated weights for policy 1, policy_version 881333 (0.0009) [2023-12-26 21:46:42,119][105692] Updated weights for policy 0, policy_version 881298 (0.0008) [2023-12-26 21:46:42,187][105692] Updated weights for policy 0, policy_version 881308 (0.0009) [2023-12-26 21:46:42,250][105692] Updated weights for policy 0, policy_version 881318 (0.0009) [2023-12-26 21:46:42,314][105692] Updated weights for policy 0, policy_version 881328 (0.0009) [2023-12-26 21:46:42,416][105620] Updated weights for policy 1, policy_version 881343 (0.0009) [2023-12-26 21:46:42,479][105620] Updated weights for policy 1, policy_version 881353 (0.0009) [2023-12-26 21:46:42,538][105620] Updated weights for policy 1, policy_version 881363 (0.0009) [2023-12-26 21:46:43,014][105692] Updated weights for policy 0, policy_version 881338 (0.0009) [2023-12-26 21:46:43,077][105692] Updated weights for policy 0, policy_version 881348 (0.0009) [2023-12-26 21:46:43,147][105692] Updated weights for policy 0, policy_version 881358 (0.0009) [2023-12-26 21:46:43,327][105620] Updated weights for policy 1, policy_version 881373 (0.0009) [2023-12-26 21:46:43,388][105620] Updated weights for policy 1, policy_version 881383 (0.0009) [2023-12-26 21:46:43,439][105620] Updated weights for policy 1, policy_version 881393 (0.0009) [2023-12-26 21:46:43,778][105692] Updated weights for policy 0, policy_version 881368 (0.0006) [2023-12-26 21:46:43,823][105692] Updated weights for policy 0, policy_version 881378 (0.0005) [2023-12-26 21:46:43,871][105692] Updated weights for policy 0, policy_version 881388 (0.0005) [2023-12-26 21:46:44,330][105620] Updated weights for policy 1, policy_version 881403 (0.0010) [2023-12-26 21:46:44,378][105620] Updated weights for policy 1, policy_version 881413 (0.0008) [2023-12-26 21:46:44,431][105692] Updated weights for policy 0, policy_version 881398 (0.0008) [2023-12-26 21:46:44,433][105620] Updated weights for policy 1, policy_version 881423 (0.0006) [2023-12-26 21:46:44,482][105692] Updated weights for policy 0, policy_version 881408 (0.0010) [2023-12-26 21:46:44,541][105692] Updated weights for policy 0, policy_version 881418 (0.0010) [2023-12-26 21:46:45,213][105692] Updated weights for policy 0, policy_version 881428 (0.0008) [2023-12-26 21:46:45,252][105620] Updated weights for policy 1, policy_version 881433 (0.0006) [2023-12-26 21:46:45,281][105692] Updated weights for policy 0, policy_version 881438 (0.0005) [2023-12-26 21:46:45,315][105620] Updated weights for policy 1, policy_version 881443 (0.0008) [2023-12-26 21:46:45,344][105692] Updated weights for policy 0, policy_version 881448 (0.0007) [2023-12-26 21:46:45,377][105620] Updated weights for policy 1, policy_version 881453 (0.0009) [2023-12-26 21:46:45,447][105620] Updated weights for policy 1, policy_version 881463 (0.0006) [2023-12-26 21:46:45,930][105692] Updated weights for policy 0, policy_version 881458 (0.0007) [2023-12-26 21:46:45,985][105692] Updated weights for policy 0, policy_version 881468 (0.0011) [2023-12-26 21:46:46,043][105692] Updated weights for policy 0, policy_version 881478 (0.0006) [2023-12-26 21:46:46,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 451371008. Throughput: 0: 9916.3, 1: 9705.6. Samples: 451344964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:46:46,063][104569] Avg episode reward: [(0, '9078.600'), (1, '9091.260')] [2023-12-26 21:46:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000881464_225681408.pth... [2023-12-26 21:46:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000880344_225394688.pth [2023-12-26 21:46:46,098][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000881488_225697792.pth... [2023-12-26 21:46:46,100][105692] Updated weights for policy 0, policy_version 881488 (0.0005) [2023-12-26 21:46:46,101][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000880272_225386496.pth [2023-12-26 21:46:46,221][105620] Updated weights for policy 1, policy_version 881473 (0.0009) [2023-12-26 21:46:46,279][105620] Updated weights for policy 1, policy_version 881484 (0.0010) [2023-12-26 21:46:46,336][105620] Updated weights for policy 1, policy_version 881494 (0.0010) [2023-12-26 21:46:46,635][105692] Updated weights for policy 0, policy_version 881498 (0.0007) [2023-12-26 21:46:46,694][105692] Updated weights for policy 0, policy_version 881508 (0.0011) [2023-12-26 21:46:46,759][105692] Updated weights for policy 0, policy_version 881518 (0.0010) [2023-12-26 21:46:46,983][105620] Updated weights for policy 1, policy_version 881504 (0.0007) [2023-12-26 21:46:47,040][105620] Updated weights for policy 1, policy_version 881514 (0.0006) [2023-12-26 21:46:47,093][105620] Updated weights for policy 1, policy_version 881524 (0.0005) [2023-12-26 21:46:47,339][105692] Updated weights for policy 0, policy_version 881528 (0.0007) [2023-12-26 21:46:47,399][105692] Updated weights for policy 0, policy_version 881538 (0.0009) [2023-12-26 21:46:47,465][105692] Updated weights for policy 0, policy_version 881548 (0.0011) [2023-12-26 21:46:47,683][105620] Updated weights for policy 1, policy_version 881534 (0.0007) [2023-12-26 21:46:47,735][105620] Updated weights for policy 1, policy_version 881544 (0.0008) [2023-12-26 21:46:47,759][105586] KL-divergence is very high: 101.9687 [2023-12-26 21:46:47,787][105620] Updated weights for policy 1, policy_version 881554 (0.0008) [2023-12-26 21:46:48,166][105692] Updated weights for policy 0, policy_version 881558 (0.0011) [2023-12-26 21:46:48,225][105692] Updated weights for policy 0, policy_version 881568 (0.0010) [2023-12-26 21:46:48,283][105692] Updated weights for policy 0, policy_version 881578 (0.0010) [2023-12-26 21:46:48,553][105620] Updated weights for policy 1, policy_version 881564 (0.0008) [2023-12-26 21:46:48,612][105620] Updated weights for policy 1, policy_version 881574 (0.0008) [2023-12-26 21:46:48,653][105586] KL-divergence is very high: 110.7952 [2023-12-26 21:46:48,668][105620] Updated weights for policy 1, policy_version 881584 (0.0009) [2023-12-26 21:46:48,685][105586] KL-divergence is very high: 158.2153 [2023-12-26 21:46:48,695][105586] KL-divergence is very high: 183.6156 [2023-12-26 21:46:49,000][105692] Updated weights for policy 0, policy_version 881588 (0.0011) [2023-12-26 21:46:49,055][105692] Updated weights for policy 0, policy_version 881598 (0.0010) [2023-12-26 21:46:49,120][105692] Updated weights for policy 0, policy_version 881608 (0.0011) [2023-12-26 21:46:49,453][105620] Updated weights for policy 1, policy_version 881594 (0.0009) [2023-12-26 21:46:49,507][105620] Updated weights for policy 1, policy_version 881604 (0.0008) [2023-12-26 21:46:49,564][105620] Updated weights for policy 1, policy_version 881614 (0.0006) [2023-12-26 21:46:49,622][105620] Updated weights for policy 1, policy_version 881624 (0.0008) [2023-12-26 21:46:49,864][105692] Updated weights for policy 0, policy_version 881619 (0.0011) [2023-12-26 21:46:49,927][105692] Updated weights for policy 0, policy_version 881629 (0.0008) [2023-12-26 21:46:49,990][105692] Updated weights for policy 0, policy_version 881639 (0.0010) [2023-12-26 21:46:50,376][105620] Updated weights for policy 1, policy_version 881634 (0.0008) [2023-12-26 21:46:50,431][105620] Updated weights for policy 1, policy_version 881644 (0.0008) [2023-12-26 21:46:50,487][105620] Updated weights for policy 1, policy_version 881654 (0.0007) [2023-12-26 21:46:50,760][105692] Updated weights for policy 0, policy_version 881649 (0.0011) [2023-12-26 21:46:50,824][105692] Updated weights for policy 0, policy_version 881659 (0.0011) [2023-12-26 21:46:50,878][105692] Updated weights for policy 0, policy_version 881669 (0.0011) [2023-12-26 21:46:50,938][105692] Updated weights for policy 0, policy_version 881679 (0.0011) [2023-12-26 21:46:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 451477504. Throughput: 0: 10096.7, 1: 9643.0. Samples: 451466268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:46:51,063][104569] Avg episode reward: [(0, '9167.329'), (1, '7831.744')] [2023-12-26 21:46:51,251][105620] Updated weights for policy 1, policy_version 881664 (0.0006) [2023-12-26 21:46:51,314][105620] Updated weights for policy 1, policy_version 881674 (0.0008) [2023-12-26 21:46:51,391][105620] Updated weights for policy 1, policy_version 881684 (0.0009) [2023-12-26 21:46:51,681][105692] Updated weights for policy 0, policy_version 881689 (0.0006) [2023-12-26 21:46:51,758][105692] Updated weights for policy 0, policy_version 881699 (0.0009) [2023-12-26 21:46:51,825][105692] Updated weights for policy 0, policy_version 881709 (0.0011) [2023-12-26 21:46:52,130][105620] Updated weights for policy 1, policy_version 881694 (0.0008) [2023-12-26 21:46:52,185][105620] Updated weights for policy 1, policy_version 881704 (0.0008) [2023-12-26 21:46:52,233][105620] Updated weights for policy 1, policy_version 881714 (0.0007) [2023-12-26 21:46:52,513][105692] Updated weights for policy 0, policy_version 881719 (0.0011) [2023-12-26 21:46:52,579][105692] Updated weights for policy 0, policy_version 881729 (0.0010) [2023-12-26 21:46:52,645][105692] Updated weights for policy 0, policy_version 881739 (0.0011) [2023-12-26 21:46:53,003][105620] Updated weights for policy 1, policy_version 881724 (0.0009) [2023-12-26 21:46:53,058][105620] Updated weights for policy 1, policy_version 881734 (0.0010) [2023-12-26 21:46:53,113][105620] Updated weights for policy 1, policy_version 881744 (0.0010) [2023-12-26 21:46:53,215][105692] Updated weights for policy 0, policy_version 881749 (0.0008) [2023-12-26 21:46:53,272][105692] Updated weights for policy 0, policy_version 881759 (0.0009) [2023-12-26 21:46:53,330][105692] Updated weights for policy 0, policy_version 881769 (0.0009) [2023-12-26 21:46:53,861][105620] Updated weights for policy 1, policy_version 881754 (0.0009) [2023-12-26 21:46:53,909][105620] Updated weights for policy 1, policy_version 881764 (0.0006) [2023-12-26 21:46:53,960][105692] Updated weights for policy 0, policy_version 881779 (0.0006) [2023-12-26 21:46:53,970][105620] Updated weights for policy 1, policy_version 881774 (0.0008) [2023-12-26 21:46:54,009][105692] Updated weights for policy 0, policy_version 881789 (0.0009) [2023-12-26 21:46:54,020][105620] Updated weights for policy 1, policy_version 881784 (0.0008) [2023-12-26 21:46:54,067][105692] Updated weights for policy 0, policy_version 881799 (0.0006) [2023-12-26 21:46:54,721][105692] Updated weights for policy 0, policy_version 881809 (0.0007) [2023-12-26 21:46:54,777][105692] Updated weights for policy 0, policy_version 881819 (0.0005) [2023-12-26 21:46:54,827][105620] Updated weights for policy 1, policy_version 881794 (0.0010) [2023-12-26 21:46:54,834][105692] Updated weights for policy 0, policy_version 881829 (0.0006) [2023-12-26 21:46:54,882][105620] Updated weights for policy 1, policy_version 881804 (0.0010) [2023-12-26 21:46:54,898][105692] Updated weights for policy 0, policy_version 881839 (0.0009) [2023-12-26 21:46:54,936][105620] Updated weights for policy 1, policy_version 881814 (0.0010) [2023-12-26 21:46:55,465][105692] Updated weights for policy 0, policy_version 881849 (0.0006) [2023-12-26 21:46:55,513][105692] Updated weights for policy 0, policy_version 881859 (0.0007) [2023-12-26 21:46:55,567][105692] Updated weights for policy 0, policy_version 881869 (0.0008) [2023-12-26 21:46:55,610][105620] Updated weights for policy 1, policy_version 881824 (0.0006) [2023-12-26 21:46:55,661][105620] Updated weights for policy 1, policy_version 881834 (0.0005) [2023-12-26 21:46:55,714][105620] Updated weights for policy 1, policy_version 881844 (0.0005) [2023-12-26 21:46:56,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 451575808. Throughput: 0: 10181.4, 1: 9561.2. Samples: 451584940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:46:56,062][104569] Avg episode reward: [(0, '9076.741'), (1, '8228.893')] [2023-12-26 21:46:56,183][105692] Updated weights for policy 0, policy_version 881879 (0.0009) [2023-12-26 21:46:56,239][105692] Updated weights for policy 0, policy_version 881889 (0.0007) [2023-12-26 21:46:56,297][105692] Updated weights for policy 0, policy_version 881899 (0.0007) [2023-12-26 21:46:56,304][105620] Updated weights for policy 1, policy_version 881854 (0.0006) [2023-12-26 21:46:56,363][105620] Updated weights for policy 1, policy_version 881864 (0.0008) [2023-12-26 21:46:56,426][105620] Updated weights for policy 1, policy_version 881874 (0.0010) [2023-12-26 21:46:56,985][105692] Updated weights for policy 0, policy_version 881909 (0.0005) [2023-12-26 21:46:57,037][105692] Updated weights for policy 0, policy_version 881919 (0.0005) [2023-12-26 21:46:57,066][105620] Updated weights for policy 1, policy_version 881884 (0.0013) [2023-12-26 21:46:57,088][105692] Updated weights for policy 0, policy_version 881929 (0.0005) [2023-12-26 21:46:57,118][105620] Updated weights for policy 1, policy_version 881894 (0.0008) [2023-12-26 21:46:57,177][105620] Updated weights for policy 1, policy_version 881904 (0.0005) [2023-12-26 21:46:57,615][105692] Updated weights for policy 0, policy_version 881939 (0.0007) [2023-12-26 21:46:57,669][105692] Updated weights for policy 0, policy_version 881949 (0.0010) [2023-12-26 21:46:57,737][105692] Updated weights for policy 0, policy_version 881959 (0.0009) [2023-12-26 21:46:57,746][105620] Updated weights for policy 1, policy_version 881914 (0.0006) [2023-12-26 21:46:57,795][105620] Updated weights for policy 1, policy_version 881924 (0.0010) [2023-12-26 21:46:57,857][105620] Updated weights for policy 1, policy_version 881934 (0.0010) [2023-12-26 21:46:57,912][105620] Updated weights for policy 1, policy_version 881944 (0.0010) [2023-12-26 21:46:58,485][105692] Updated weights for policy 0, policy_version 881969 (0.0010) [2023-12-26 21:46:58,549][105692] Updated weights for policy 0, policy_version 881979 (0.0008) [2023-12-26 21:46:58,615][105692] Updated weights for policy 0, policy_version 881989 (0.0007) [2023-12-26 21:46:58,679][105692] Updated weights for policy 0, policy_version 881999 (0.0009) [2023-12-26 21:46:58,714][105620] Updated weights for policy 1, policy_version 881954 (0.0006) [2023-12-26 21:46:58,774][105620] Updated weights for policy 1, policy_version 881964 (0.0006) [2023-12-26 21:46:58,844][105620] Updated weights for policy 1, policy_version 881974 (0.0007) [2023-12-26 21:46:59,530][105692] Updated weights for policy 0, policy_version 882009 (0.0010) [2023-12-26 21:46:59,585][105620] Updated weights for policy 1, policy_version 881984 (0.0010) [2023-12-26 21:46:59,590][105692] Updated weights for policy 0, policy_version 882019 (0.0010) [2023-12-26 21:46:59,641][105620] Updated weights for policy 1, policy_version 881994 (0.0010) [2023-12-26 21:46:59,644][105692] Updated weights for policy 0, policy_version 882029 (0.0010) [2023-12-26 21:46:59,692][105620] Updated weights for policy 1, policy_version 882004 (0.0009) [2023-12-26 21:47:00,374][105692] Updated weights for policy 0, policy_version 882039 (0.0007) [2023-12-26 21:47:00,425][105692] Updated weights for policy 0, policy_version 882049 (0.0005) [2023-12-26 21:47:00,459][105620] Updated weights for policy 1, policy_version 882014 (0.0009) [2023-12-26 21:47:00,479][105692] Updated weights for policy 0, policy_version 882059 (0.0006) [2023-12-26 21:47:00,516][105620] Updated weights for policy 1, policy_version 882024 (0.0008) [2023-12-26 21:47:00,572][105620] Updated weights for policy 1, policy_version 882034 (0.0010) [2023-12-26 21:47:01,017][105692] Updated weights for policy 0, policy_version 882069 (0.0005) [2023-12-26 21:47:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 451674112. Throughput: 0: 10286.9, 1: 9649.3. Samples: 451647792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:47:01,063][104569] Avg episode reward: [(0, '9165.792'), (1, '8648.887')] [2023-12-26 21:47:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000882040_225828864.pth... [2023-12-26 21:47:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000880920_225542144.pth [2023-12-26 21:47:01,087][105692] Updated weights for policy 0, policy_version 882079 (0.0008) [2023-12-26 21:47:01,155][105692] Updated weights for policy 0, policy_version 882089 (0.0009) [2023-12-26 21:47:01,195][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000882096_225853440.pth... [2023-12-26 21:47:01,200][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000880880_225542144.pth [2023-12-26 21:47:01,371][105620] Updated weights for policy 1, policy_version 882044 (0.0010) [2023-12-26 21:47:01,433][105620] Updated weights for policy 1, policy_version 882054 (0.0010) [2023-12-26 21:47:01,477][105620] Updated weights for policy 1, policy_version 882064 (0.0010) [2023-12-26 21:47:01,846][105692] Updated weights for policy 0, policy_version 882099 (0.0009) [2023-12-26 21:47:01,910][105692] Updated weights for policy 0, policy_version 882109 (0.0008) [2023-12-26 21:47:01,975][105692] Updated weights for policy 0, policy_version 882119 (0.0009) [2023-12-26 21:47:02,121][105620] Updated weights for policy 1, policy_version 882074 (0.0009) [2023-12-26 21:47:02,184][105620] Updated weights for policy 1, policy_version 882084 (0.0005) [2023-12-26 21:47:02,247][105620] Updated weights for policy 1, policy_version 882094 (0.0006) [2023-12-26 21:47:02,307][105620] Updated weights for policy 1, policy_version 882104 (0.0010) [2023-12-26 21:47:02,697][105692] Updated weights for policy 0, policy_version 882129 (0.0008) [2023-12-26 21:47:02,750][105692] Updated weights for policy 0, policy_version 882139 (0.0005) [2023-12-26 21:47:02,820][105692] Updated weights for policy 0, policy_version 882149 (0.0008) [2023-12-26 21:47:02,867][105692] Updated weights for policy 0, policy_version 882159 (0.0009) [2023-12-26 21:47:03,020][105620] Updated weights for policy 1, policy_version 882114 (0.0009) [2023-12-26 21:47:03,079][105620] Updated weights for policy 1, policy_version 882124 (0.0009) [2023-12-26 21:47:03,140][105620] Updated weights for policy 1, policy_version 882134 (0.0009) [2023-12-26 21:47:03,522][105692] Updated weights for policy 0, policy_version 882169 (0.0006) [2023-12-26 21:47:03,570][105692] Updated weights for policy 0, policy_version 882179 (0.0005) [2023-12-26 21:47:03,621][105692] Updated weights for policy 0, policy_version 882189 (0.0005) [2023-12-26 21:47:03,906][105620] Updated weights for policy 1, policy_version 882144 (0.0007) [2023-12-26 21:47:03,966][105620] Updated weights for policy 1, policy_version 882154 (0.0007) [2023-12-26 21:47:04,021][105620] Updated weights for policy 1, policy_version 882164 (0.0006) [2023-12-26 21:47:04,350][105692] Updated weights for policy 0, policy_version 882199 (0.0008) [2023-12-26 21:47:04,410][105692] Updated weights for policy 0, policy_version 882209 (0.0009) [2023-12-26 21:47:04,458][105692] Updated weights for policy 0, policy_version 882219 (0.0009) [2023-12-26 21:47:04,691][105620] Updated weights for policy 1, policy_version 882174 (0.0008) [2023-12-26 21:47:04,740][105620] Updated weights for policy 1, policy_version 882184 (0.0007) [2023-12-26 21:47:04,801][105620] Updated weights for policy 1, policy_version 882194 (0.0005) [2023-12-26 21:47:05,252][105692] Updated weights for policy 0, policy_version 882229 (0.0008) [2023-12-26 21:47:05,299][105692] Updated weights for policy 0, policy_version 882239 (0.0008) [2023-12-26 21:47:05,348][105692] Updated weights for policy 0, policy_version 882249 (0.0009) [2023-12-26 21:47:05,467][105620] Updated weights for policy 1, policy_version 882204 (0.0010) [2023-12-26 21:47:05,525][105620] Updated weights for policy 1, policy_version 882214 (0.0010) [2023-12-26 21:47:05,584][105620] Updated weights for policy 1, policy_version 882224 (0.0007) [2023-12-26 21:47:06,057][105692] Updated weights for policy 0, policy_version 882259 (0.0009) [2023-12-26 21:47:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 451772416. Throughput: 0: 10212.0, 1: 9685.5. Samples: 451764224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:47:06,063][104569] Avg episode reward: [(0, '8988.318'), (1, '8465.495')] [2023-12-26 21:47:06,117][105692] Updated weights for policy 0, policy_version 882269 (0.0008) [2023-12-26 21:47:06,169][105692] Updated weights for policy 0, policy_version 882279 (0.0008) [2023-12-26 21:47:06,252][105620] Updated weights for policy 1, policy_version 882234 (0.0009) [2023-12-26 21:47:06,315][105620] Updated weights for policy 1, policy_version 882244 (0.0005) [2023-12-26 21:47:06,376][105620] Updated weights for policy 1, policy_version 882254 (0.0005) [2023-12-26 21:47:06,438][105620] Updated weights for policy 1, policy_version 882264 (0.0008) [2023-12-26 21:47:06,860][105692] Updated weights for policy 0, policy_version 882289 (0.0008) [2023-12-26 21:47:06,926][105692] Updated weights for policy 0, policy_version 882299 (0.0005) [2023-12-26 21:47:06,984][105692] Updated weights for policy 0, policy_version 882309 (0.0005) [2023-12-26 21:47:07,044][105692] Updated weights for policy 0, policy_version 882319 (0.0005) [2023-12-26 21:47:07,096][105620] Updated weights for policy 1, policy_version 882274 (0.0009) [2023-12-26 21:47:07,155][105620] Updated weights for policy 1, policy_version 882284 (0.0009) [2023-12-26 21:47:07,213][105620] Updated weights for policy 1, policy_version 882294 (0.0006) [2023-12-26 21:47:07,699][105692] Updated weights for policy 0, policy_version 882329 (0.0005) [2023-12-26 21:47:07,763][105692] Updated weights for policy 0, policy_version 882339 (0.0005) [2023-12-26 21:47:07,820][105620] Updated weights for policy 1, policy_version 882304 (0.0006) [2023-12-26 21:47:07,823][105692] Updated weights for policy 0, policy_version 882349 (0.0005) [2023-12-26 21:47:07,868][105620] Updated weights for policy 1, policy_version 882314 (0.0008) [2023-12-26 21:47:07,915][105620] Updated weights for policy 1, policy_version 882324 (0.0009) [2023-12-26 21:47:08,571][105620] Updated weights for policy 1, policy_version 882334 (0.0007) [2023-12-26 21:47:08,578][105692] Updated weights for policy 0, policy_version 882359 (0.0007) [2023-12-26 21:47:08,627][105620] Updated weights for policy 1, policy_version 882344 (0.0008) [2023-12-26 21:47:08,638][105692] Updated weights for policy 0, policy_version 882369 (0.0006) [2023-12-26 21:47:08,688][105620] Updated weights for policy 1, policy_version 882354 (0.0006) [2023-12-26 21:47:08,704][105692] Updated weights for policy 0, policy_version 882379 (0.0008) [2023-12-26 21:47:09,330][105620] Updated weights for policy 1, policy_version 882364 (0.0006) [2023-12-26 21:47:09,400][105620] Updated weights for policy 1, policy_version 882374 (0.0008) [2023-12-26 21:47:09,463][105620] Updated weights for policy 1, policy_version 882384 (0.0007) [2023-12-26 21:47:09,512][105692] Updated weights for policy 0, policy_version 882389 (0.0008) [2023-12-26 21:47:09,580][105692] Updated weights for policy 0, policy_version 882399 (0.0006) [2023-12-26 21:47:09,641][105692] Updated weights for policy 0, policy_version 882409 (0.0006) [2023-12-26 21:47:10,233][105620] Updated weights for policy 1, policy_version 882394 (0.0009) [2023-12-26 21:47:10,291][105620] Updated weights for policy 1, policy_version 882404 (0.0008) [2023-12-26 21:47:10,301][105692] Updated weights for policy 0, policy_version 882419 (0.0006) [2023-12-26 21:47:10,349][105620] Updated weights for policy 1, policy_version 882414 (0.0007) [2023-12-26 21:47:10,358][105692] Updated weights for policy 0, policy_version 882429 (0.0006) [2023-12-26 21:47:10,412][105620] Updated weights for policy 1, policy_version 882424 (0.0006) [2023-12-26 21:47:10,425][105692] Updated weights for policy 0, policy_version 882439 (0.0008) [2023-12-26 21:47:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 451870720. Throughput: 0: 10211.2, 1: 9679.0. Samples: 451883036. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-26 21:47:11,063][104569] Avg episode reward: [(0, '8658.798'), (1, '8715.233')] [2023-12-26 21:47:11,148][105620] Updated weights for policy 1, policy_version 882434 (0.0008) [2023-12-26 21:47:11,214][105620] Updated weights for policy 1, policy_version 882444 (0.0007) [2023-12-26 21:47:11,231][105692] Updated weights for policy 0, policy_version 882449 (0.0009) [2023-12-26 21:47:11,277][105620] Updated weights for policy 1, policy_version 882454 (0.0010) [2023-12-26 21:47:11,297][105692] Updated weights for policy 0, policy_version 882459 (0.0008) [2023-12-26 21:47:11,358][105692] Updated weights for policy 0, policy_version 882469 (0.0010) [2023-12-26 21:47:11,421][105692] Updated weights for policy 0, policy_version 882479 (0.0008) [2023-12-26 21:47:12,074][105620] Updated weights for policy 1, policy_version 882464 (0.0008) [2023-12-26 21:47:12,123][105692] Updated weights for policy 0, policy_version 882489 (0.0008) [2023-12-26 21:47:12,142][105620] Updated weights for policy 1, policy_version 882474 (0.0007) [2023-12-26 21:47:12,184][105692] Updated weights for policy 0, policy_version 882499 (0.0008) [2023-12-26 21:47:12,206][105620] Updated weights for policy 1, policy_version 882484 (0.0009) [2023-12-26 21:47:12,249][105692] Updated weights for policy 0, policy_version 882509 (0.0007) [2023-12-26 21:47:12,876][105620] Updated weights for policy 1, policy_version 882494 (0.0008) [2023-12-26 21:47:12,931][105620] Updated weights for policy 1, policy_version 882504 (0.0008) [2023-12-26 21:47:12,941][105692] Updated weights for policy 0, policy_version 882519 (0.0008) [2023-12-26 21:47:12,983][105620] Updated weights for policy 1, policy_version 882514 (0.0008) [2023-12-26 21:47:13,003][105692] Updated weights for policy 0, policy_version 882529 (0.0007) [2023-12-26 21:47:13,058][105692] Updated weights for policy 0, policy_version 882539 (0.0008) [2023-12-26 21:47:13,597][105620] Updated weights for policy 1, policy_version 882524 (0.0009) [2023-12-26 21:47:13,652][105620] Updated weights for policy 1, policy_version 882534 (0.0010) [2023-12-26 21:47:13,707][105620] Updated weights for policy 1, policy_version 882544 (0.0010) [2023-12-26 21:47:13,890][105692] Updated weights for policy 0, policy_version 882549 (0.0009) [2023-12-26 21:47:13,949][105692] Updated weights for policy 0, policy_version 882559 (0.0010) [2023-12-26 21:47:14,001][105692] Updated weights for policy 0, policy_version 882570 (0.0009) [2023-12-26 21:47:14,299][105620] Updated weights for policy 1, policy_version 882554 (0.0008) [2023-12-26 21:47:14,359][105620] Updated weights for policy 1, policy_version 882564 (0.0008) [2023-12-26 21:47:14,423][105620] Updated weights for policy 1, policy_version 882574 (0.0005) [2023-12-26 21:47:14,474][105620] Updated weights for policy 1, policy_version 882584 (0.0009) [2023-12-26 21:47:14,855][105692] Updated weights for policy 0, policy_version 882580 (0.0009) [2023-12-26 21:47:14,900][105692] Updated weights for policy 0, policy_version 882590 (0.0008) [2023-12-26 21:47:14,945][105692] Updated weights for policy 0, policy_version 882600 (0.0008) [2023-12-26 21:47:15,153][105620] Updated weights for policy 1, policy_version 882594 (0.0011) [2023-12-26 21:47:15,213][105620] Updated weights for policy 1, policy_version 882604 (0.0011) [2023-12-26 21:47:15,276][105620] Updated weights for policy 1, policy_version 882614 (0.0011) [2023-12-26 21:47:15,742][105692] Updated weights for policy 0, policy_version 882610 (0.0008) [2023-12-26 21:47:15,790][105692] Updated weights for policy 0, policy_version 882620 (0.0008) [2023-12-26 21:47:15,849][105692] Updated weights for policy 0, policy_version 882630 (0.0008) [2023-12-26 21:47:15,917][105692] Updated weights for policy 0, policy_version 882640 (0.0008) [2023-12-26 21:47:16,037][105620] Updated weights for policy 1, policy_version 882624 (0.0011) [2023-12-26 21:47:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 451969024. Throughput: 0: 10024.4, 1: 9703.0. Samples: 451940840. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-26 21:47:16,062][104569] Avg episode reward: [(0, '8659.717'), (1, '8718.346')] [2023-12-26 21:47:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000882640_225992704.pth... [2023-12-26 21:47:16,085][105620] Updated weights for policy 1, policy_version 882634 (0.0010) [2023-12-26 21:47:16,088][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000881488_225697792.pth [2023-12-26 21:47:16,141][105620] Updated weights for policy 1, policy_version 882644 (0.0010) [2023-12-26 21:47:16,165][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000882648_225984512.pth... [2023-12-26 21:47:16,168][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000881464_225681408.pth [2023-12-26 21:47:16,620][105692] Updated weights for policy 0, policy_version 882650 (0.0008) [2023-12-26 21:47:16,675][105692] Updated weights for policy 0, policy_version 882660 (0.0008) [2023-12-26 21:47:16,733][105692] Updated weights for policy 0, policy_version 882670 (0.0008) [2023-12-26 21:47:16,913][105620] Updated weights for policy 1, policy_version 882654 (0.0011) [2023-12-26 21:47:16,968][105620] Updated weights for policy 1, policy_version 882665 (0.0009) [2023-12-26 21:47:17,036][105620] Updated weights for policy 1, policy_version 882675 (0.0009) [2023-12-26 21:47:17,526][105692] Updated weights for policy 0, policy_version 882680 (0.0009) [2023-12-26 21:47:17,575][105692] Updated weights for policy 0, policy_version 882690 (0.0008) [2023-12-26 21:47:17,626][105692] Updated weights for policy 0, policy_version 882700 (0.0009) [2023-12-26 21:47:17,670][105620] Updated weights for policy 1, policy_version 882685 (0.0009) [2023-12-26 21:47:17,721][105620] Updated weights for policy 1, policy_version 882695 (0.0009) [2023-12-26 21:47:17,775][105620] Updated weights for policy 1, policy_version 882705 (0.0008) [2023-12-26 21:47:18,341][105692] Updated weights for policy 0, policy_version 882710 (0.0008) [2023-12-26 21:47:18,406][105692] Updated weights for policy 0, policy_version 882720 (0.0010) [2023-12-26 21:47:18,438][105620] Updated weights for policy 1, policy_version 882715 (0.0008) [2023-12-26 21:47:18,469][105692] Updated weights for policy 0, policy_version 882730 (0.0008) [2023-12-26 21:47:18,496][105620] Updated weights for policy 1, policy_version 882725 (0.0006) [2023-12-26 21:47:18,554][105620] Updated weights for policy 1, policy_version 882735 (0.0008) [2023-12-26 21:47:19,192][105692] Updated weights for policy 0, policy_version 882740 (0.0008) [2023-12-26 21:47:19,259][105692] Updated weights for policy 0, policy_version 882750 (0.0009) [2023-12-26 21:47:19,314][105620] Updated weights for policy 1, policy_version 882745 (0.0009) [2023-12-26 21:47:19,316][105692] Updated weights for policy 0, policy_version 882760 (0.0010) [2023-12-26 21:47:19,380][105620] Updated weights for policy 1, policy_version 882755 (0.0009) [2023-12-26 21:47:19,432][105620] Updated weights for policy 1, policy_version 882765 (0.0009) [2023-12-26 21:47:19,493][105620] Updated weights for policy 1, policy_version 882775 (0.0009) [2023-12-26 21:47:20,041][105692] Updated weights for policy 0, policy_version 882770 (0.0007) [2023-12-26 21:47:20,104][105692] Updated weights for policy 0, policy_version 882780 (0.0008) [2023-12-26 21:47:20,157][105692] Updated weights for policy 0, policy_version 882790 (0.0005) [2023-12-26 21:47:20,212][105692] Updated weights for policy 0, policy_version 882800 (0.0005) [2023-12-26 21:47:20,300][105620] Updated weights for policy 1, policy_version 882785 (0.0009) [2023-12-26 21:47:20,362][105620] Updated weights for policy 1, policy_version 882795 (0.0009) [2023-12-26 21:47:20,423][105620] Updated weights for policy 1, policy_version 882805 (0.0010) [2023-12-26 21:47:20,837][105692] Updated weights for policy 0, policy_version 882810 (0.0009) [2023-12-26 21:47:20,895][105692] Updated weights for policy 0, policy_version 882820 (0.0008) [2023-12-26 21:47:20,956][105692] Updated weights for policy 0, policy_version 882830 (0.0006) [2023-12-26 21:47:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 452067328. Throughput: 0: 9874.2, 1: 9671.7. Samples: 452055080. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-26 21:47:21,063][104569] Avg episode reward: [(0, '8905.654'), (1, '9171.777')] [2023-12-26 21:47:21,315][105620] Updated weights for policy 1, policy_version 882815 (0.0010) [2023-12-26 21:47:21,386][105620] Updated weights for policy 1, policy_version 882825 (0.0009) [2023-12-26 21:47:21,446][105620] Updated weights for policy 1, policy_version 882835 (0.0009) [2023-12-26 21:47:21,695][105692] Updated weights for policy 0, policy_version 882840 (0.0007) [2023-12-26 21:47:21,766][105692] Updated weights for policy 0, policy_version 882850 (0.0009) [2023-12-26 21:47:21,829][105692] Updated weights for policy 0, policy_version 882860 (0.0007) [2023-12-26 21:47:22,132][105620] Updated weights for policy 1, policy_version 882845 (0.0009) [2023-12-26 21:47:22,192][105620] Updated weights for policy 1, policy_version 882855 (0.0010) [2023-12-26 21:47:22,250][105620] Updated weights for policy 1, policy_version 882865 (0.0008) [2023-12-26 21:47:22,518][105692] Updated weights for policy 0, policy_version 882870 (0.0008) [2023-12-26 21:47:22,573][105692] Updated weights for policy 0, policy_version 882880 (0.0009) [2023-12-26 21:47:22,626][105692] Updated weights for policy 0, policy_version 882890 (0.0009) [2023-12-26 21:47:23,075][105620] Updated weights for policy 1, policy_version 882875 (0.0009) [2023-12-26 21:47:23,134][105620] Updated weights for policy 1, policy_version 882885 (0.0011) [2023-12-26 21:47:23,193][105620] Updated weights for policy 1, policy_version 882895 (0.0010) [2023-12-26 21:47:23,358][105692] Updated weights for policy 0, policy_version 882900 (0.0007) [2023-12-26 21:47:23,416][105692] Updated weights for policy 0, policy_version 882910 (0.0008) [2023-12-26 21:47:23,477][105692] Updated weights for policy 0, policy_version 882920 (0.0010) [2023-12-26 21:47:23,929][105620] Updated weights for policy 1, policy_version 882905 (0.0009) [2023-12-26 21:47:23,998][105620] Updated weights for policy 1, policy_version 882915 (0.0010) [2023-12-26 21:47:24,062][105620] Updated weights for policy 1, policy_version 882925 (0.0010) [2023-12-26 21:47:24,123][105692] Updated weights for policy 0, policy_version 882930 (0.0009) [2023-12-26 21:47:24,124][105620] Updated weights for policy 1, policy_version 882935 (0.0010) [2023-12-26 21:47:24,188][105692] Updated weights for policy 0, policy_version 882940 (0.0006) [2023-12-26 21:47:24,242][105692] Updated weights for policy 0, policy_version 882950 (0.0005) [2023-12-26 21:47:24,300][105692] Updated weights for policy 0, policy_version 882960 (0.0006) [2023-12-26 21:47:24,789][105620] Updated weights for policy 1, policy_version 882945 (0.0010) [2023-12-26 21:47:24,856][105620] Updated weights for policy 1, policy_version 882955 (0.0010) [2023-12-26 21:47:24,873][105692] Updated weights for policy 0, policy_version 882970 (0.0006) [2023-12-26 21:47:24,912][105620] Updated weights for policy 1, policy_version 882965 (0.0010) [2023-12-26 21:47:24,922][105692] Updated weights for policy 0, policy_version 882980 (0.0007) [2023-12-26 21:47:24,979][105692] Updated weights for policy 0, policy_version 882990 (0.0011) [2023-12-26 21:47:25,640][105620] Updated weights for policy 1, policy_version 882975 (0.0010) [2023-12-26 21:47:25,694][105620] Updated weights for policy 1, policy_version 882985 (0.0007) [2023-12-26 21:47:25,704][105692] Updated weights for policy 0, policy_version 883000 (0.0006) [2023-12-26 21:47:25,752][105620] Updated weights for policy 1, policy_version 882995 (0.0010) [2023-12-26 21:47:25,761][105692] Updated weights for policy 0, policy_version 883010 (0.0005) [2023-12-26 21:47:25,814][105692] Updated weights for policy 0, policy_version 883020 (0.0005) [2023-12-26 21:47:26,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 452165632. Throughput: 0: 9925.2, 1: 9677.6. Samples: 452171516. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-26 21:47:26,063][104569] Avg episode reward: [(0, '8903.981'), (1, '8914.964')] [2023-12-26 21:47:26,316][105692] Updated weights for policy 0, policy_version 883030 (0.0005) [2023-12-26 21:47:26,364][105692] Updated weights for policy 0, policy_version 883040 (0.0005) [2023-12-26 21:47:26,420][105692] Updated weights for policy 0, policy_version 883050 (0.0005) [2023-12-26 21:47:26,496][105620] Updated weights for policy 1, policy_version 883005 (0.0010) [2023-12-26 21:47:26,541][105620] Updated weights for policy 1, policy_version 883015 (0.0010) [2023-12-26 21:47:26,589][105620] Updated weights for policy 1, policy_version 883025 (0.0010) [2023-12-26 21:47:27,096][105692] Updated weights for policy 0, policy_version 883060 (0.0007) [2023-12-26 21:47:27,147][105692] Updated weights for policy 0, policy_version 883070 (0.0010) [2023-12-26 21:47:27,205][105692] Updated weights for policy 0, policy_version 883080 (0.0010) [2023-12-26 21:47:27,345][105620] Updated weights for policy 1, policy_version 883035 (0.0010) [2023-12-26 21:47:27,399][105620] Updated weights for policy 1, policy_version 883045 (0.0010) [2023-12-26 21:47:27,453][105620] Updated weights for policy 1, policy_version 883055 (0.0010) [2023-12-26 21:47:27,848][105692] Updated weights for policy 0, policy_version 883090 (0.0009) [2023-12-26 21:47:27,906][105692] Updated weights for policy 0, policy_version 883100 (0.0005) [2023-12-26 21:47:27,953][105692] Updated weights for policy 0, policy_version 883110 (0.0005) [2023-12-26 21:47:28,011][105692] Updated weights for policy 0, policy_version 883120 (0.0007) [2023-12-26 21:47:28,205][105620] Updated weights for policy 1, policy_version 883065 (0.0010) [2023-12-26 21:47:28,270][105620] Updated weights for policy 1, policy_version 883075 (0.0010) [2023-12-26 21:47:28,332][105620] Updated weights for policy 1, policy_version 883085 (0.0010) [2023-12-26 21:47:28,395][105620] Updated weights for policy 1, policy_version 883095 (0.0010) [2023-12-26 21:47:28,681][105692] Updated weights for policy 0, policy_version 883130 (0.0010) [2023-12-26 21:47:28,728][105692] Updated weights for policy 0, policy_version 883140 (0.0010) [2023-12-26 21:47:28,772][105692] Updated weights for policy 0, policy_version 883150 (0.0010) [2023-12-26 21:47:29,118][105620] Updated weights for policy 1, policy_version 883105 (0.0010) [2023-12-26 21:47:29,172][105620] Updated weights for policy 1, policy_version 883115 (0.0010) [2023-12-26 21:47:29,224][105620] Updated weights for policy 1, policy_version 883125 (0.0010) [2023-12-26 21:47:29,463][105692] Updated weights for policy 0, policy_version 883160 (0.0007) [2023-12-26 21:47:29,514][105692] Updated weights for policy 0, policy_version 883170 (0.0005) [2023-12-26 21:47:29,568][105692] Updated weights for policy 0, policy_version 883180 (0.0005) [2023-12-26 21:47:29,958][105620] Updated weights for policy 1, policy_version 883135 (0.0009) [2023-12-26 21:47:30,013][105620] Updated weights for policy 1, policy_version 883145 (0.0006) [2023-12-26 21:47:30,067][105620] Updated weights for policy 1, policy_version 883155 (0.0006) [2023-12-26 21:47:30,297][105692] Updated weights for policy 0, policy_version 883190 (0.0006) [2023-12-26 21:47:30,359][105692] Updated weights for policy 0, policy_version 883200 (0.0006) [2023-12-26 21:47:30,417][105692] Updated weights for policy 0, policy_version 883210 (0.0007) [2023-12-26 21:47:30,631][105620] Updated weights for policy 1, policy_version 883165 (0.0005) [2023-12-26 21:47:30,677][105620] Updated weights for policy 1, policy_version 883175 (0.0008) [2023-12-26 21:47:30,726][105620] Updated weights for policy 1, policy_version 883185 (0.0008) [2023-12-26 21:47:31,053][105692] Updated weights for policy 0, policy_version 883220 (0.0009) [2023-12-26 21:47:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 452263936. Throughput: 0: 10027.2, 1: 9699.1. Samples: 452232652. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-26 21:47:31,063][104569] Avg episode reward: [(0, '8993.408'), (1, '8913.170')] [2023-12-26 21:47:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000883192_226123776.pth... [2023-12-26 21:47:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000882040_225828864.pth [2023-12-26 21:47:31,116][105692] Updated weights for policy 0, policy_version 883230 (0.0008) [2023-12-26 21:47:31,179][105692] Updated weights for policy 0, policy_version 883240 (0.0008) [2023-12-26 21:47:31,224][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000883248_226148352.pth... [2023-12-26 21:47:31,228][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000882096_225853440.pth [2023-12-26 21:47:31,498][105620] Updated weights for policy 1, policy_version 883195 (0.0007) [2023-12-26 21:47:31,562][105620] Updated weights for policy 1, policy_version 883205 (0.0009) [2023-12-26 21:47:31,624][105620] Updated weights for policy 1, policy_version 883215 (0.0009) [2023-12-26 21:47:31,906][105692] Updated weights for policy 0, policy_version 883250 (0.0007) [2023-12-26 21:47:31,956][105692] Updated weights for policy 0, policy_version 883260 (0.0005) [2023-12-26 21:47:32,003][105692] Updated weights for policy 0, policy_version 883270 (0.0005) [2023-12-26 21:47:32,049][105692] Updated weights for policy 0, policy_version 883280 (0.0005) [2023-12-26 21:47:32,423][105620] Updated weights for policy 1, policy_version 883225 (0.0009) [2023-12-26 21:47:32,484][105620] Updated weights for policy 1, policy_version 883235 (0.0009) [2023-12-26 21:47:32,532][105620] Updated weights for policy 1, policy_version 883245 (0.0009) [2023-12-26 21:47:32,578][105620] Updated weights for policy 1, policy_version 883255 (0.0008) [2023-12-26 21:47:32,754][105692] Updated weights for policy 0, policy_version 883290 (0.0009) [2023-12-26 21:47:32,801][105692] Updated weights for policy 0, policy_version 883300 (0.0008) [2023-12-26 21:47:32,848][105692] Updated weights for policy 0, policy_version 883310 (0.0009) [2023-12-26 21:47:33,292][105620] Updated weights for policy 1, policy_version 883265 (0.0009) [2023-12-26 21:47:33,339][105620] Updated weights for policy 1, policy_version 883275 (0.0009) [2023-12-26 21:47:33,392][105620] Updated weights for policy 1, policy_version 883285 (0.0006) [2023-12-26 21:47:33,714][105692] Updated weights for policy 0, policy_version 883320 (0.0008) [2023-12-26 21:47:33,765][105692] Updated weights for policy 0, policy_version 883330 (0.0009) [2023-12-26 21:47:33,819][105692] Updated weights for policy 0, policy_version 883340 (0.0008) [2023-12-26 21:47:34,011][105620] Updated weights for policy 1, policy_version 883295 (0.0007) [2023-12-26 21:47:34,089][105620] Updated weights for policy 1, policy_version 883305 (0.0010) [2023-12-26 21:47:34,141][105620] Updated weights for policy 1, policy_version 883315 (0.0010) [2023-12-26 21:47:34,625][105692] Updated weights for policy 0, policy_version 883350 (0.0008) [2023-12-26 21:47:34,674][105692] Updated weights for policy 0, policy_version 883360 (0.0008) [2023-12-26 21:47:34,724][105692] Updated weights for policy 0, policy_version 883370 (0.0008) [2023-12-26 21:47:34,897][105620] Updated weights for policy 1, policy_version 883325 (0.0010) [2023-12-26 21:47:34,952][105620] Updated weights for policy 1, policy_version 883335 (0.0010) [2023-12-26 21:47:35,014][105620] Updated weights for policy 1, policy_version 883345 (0.0010) [2023-12-26 21:47:35,554][105692] Updated weights for policy 0, policy_version 883380 (0.0008) [2023-12-26 21:47:35,605][105692] Updated weights for policy 0, policy_version 883391 (0.0010) [2023-12-26 21:47:35,648][105620] Updated weights for policy 1, policy_version 883355 (0.0009) [2023-12-26 21:47:35,661][105692] Updated weights for policy 0, policy_version 883402 (0.0011) [2023-12-26 21:47:35,705][105620] Updated weights for policy 1, policy_version 883365 (0.0010) [2023-12-26 21:47:35,763][105620] Updated weights for policy 1, policy_version 883375 (0.0009) [2023-12-26 21:47:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 452362240. Throughput: 0: 9871.7, 1: 9757.5. Samples: 452349584. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-26 21:47:36,063][104569] Avg episode reward: [(0, '8993.753'), (1, '9076.853')] [2023-12-26 21:47:36,468][105620] Updated weights for policy 1, policy_version 883385 (0.0009) [2023-12-26 21:47:36,520][105692] Updated weights for policy 0, policy_version 883412 (0.0009) [2023-12-26 21:47:36,539][105620] Updated weights for policy 1, policy_version 883395 (0.0009) [2023-12-26 21:47:36,576][105692] Updated weights for policy 0, policy_version 883422 (0.0010) [2023-12-26 21:47:36,605][105620] Updated weights for policy 1, policy_version 883405 (0.0007) [2023-12-26 21:47:36,636][105692] Updated weights for policy 0, policy_version 883432 (0.0008) [2023-12-26 21:47:36,668][105620] Updated weights for policy 1, policy_version 883415 (0.0006) [2023-12-26 21:47:37,371][105620] Updated weights for policy 1, policy_version 883425 (0.0005) [2023-12-26 21:47:37,412][105692] Updated weights for policy 0, policy_version 883442 (0.0007) [2023-12-26 21:47:37,425][105620] Updated weights for policy 1, policy_version 883435 (0.0006) [2023-12-26 21:47:37,472][105692] Updated weights for policy 0, policy_version 883452 (0.0008) [2023-12-26 21:47:37,479][105620] Updated weights for policy 1, policy_version 883445 (0.0007) [2023-12-26 21:47:37,524][105692] Updated weights for policy 0, policy_version 883463 (0.0010) [2023-12-26 21:47:38,106][105620] Updated weights for policy 1, policy_version 883455 (0.0005) [2023-12-26 21:47:38,173][105620] Updated weights for policy 1, policy_version 883465 (0.0005) [2023-12-26 21:47:38,223][105620] Updated weights for policy 1, policy_version 883475 (0.0005) [2023-12-26 21:47:38,384][105692] Updated weights for policy 0, policy_version 883473 (0.0008) [2023-12-26 21:47:38,433][105692] Updated weights for policy 0, policy_version 883483 (0.0009) [2023-12-26 21:47:38,483][105692] Updated weights for policy 0, policy_version 883493 (0.0007) [2023-12-26 21:47:38,540][105692] Updated weights for policy 0, policy_version 883503 (0.0010) [2023-12-26 21:47:38,919][105620] Updated weights for policy 1, policy_version 883485 (0.0009) [2023-12-26 21:47:38,985][105620] Updated weights for policy 1, policy_version 883495 (0.0011) [2023-12-26 21:47:39,037][105620] Updated weights for policy 1, policy_version 883505 (0.0010) [2023-12-26 21:47:39,314][105692] Updated weights for policy 0, policy_version 883513 (0.0009) [2023-12-26 21:47:39,382][105692] Updated weights for policy 0, policy_version 883523 (0.0010) [2023-12-26 21:47:39,443][105692] Updated weights for policy 0, policy_version 883533 (0.0007) [2023-12-26 21:47:39,807][105620] Updated weights for policy 1, policy_version 883515 (0.0011) [2023-12-26 21:47:39,874][105620] Updated weights for policy 1, policy_version 883525 (0.0011) [2023-12-26 21:47:39,943][105620] Updated weights for policy 1, policy_version 883535 (0.0011) [2023-12-26 21:47:40,257][105692] Updated weights for policy 0, policy_version 883543 (0.0009) [2023-12-26 21:47:40,314][105692] Updated weights for policy 0, policy_version 883553 (0.0010) [2023-12-26 21:47:40,373][105692] Updated weights for policy 0, policy_version 883563 (0.0006) [2023-12-26 21:47:40,709][105620] Updated weights for policy 1, policy_version 883545 (0.0011) [2023-12-26 21:47:40,760][105620] Updated weights for policy 1, policy_version 883555 (0.0007) [2023-12-26 21:47:40,804][105620] Updated weights for policy 1, policy_version 883565 (0.0008) [2023-12-26 21:47:40,857][105620] Updated weights for policy 1, policy_version 883575 (0.0008) [2023-12-26 21:47:41,049][105692] Updated weights for policy 0, policy_version 883573 (0.0007) [2023-12-26 21:47:41,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 452452352. Throughput: 0: 9693.9, 1: 9785.3. Samples: 452461508. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-26 21:47:41,062][104569] Avg episode reward: [(0, '8819.412'), (1, '8896.878')] [2023-12-26 21:47:41,104][105692] Updated weights for policy 0, policy_version 883583 (0.0006) [2023-12-26 21:47:41,167][105692] Updated weights for policy 0, policy_version 883593 (0.0011) [2023-12-26 21:47:41,651][105620] Updated weights for policy 1, policy_version 883585 (0.0008) [2023-12-26 21:47:41,702][105620] Updated weights for policy 1, policy_version 883595 (0.0010) [2023-12-26 21:47:41,767][105620] Updated weights for policy 1, policy_version 883605 (0.0008) [2023-12-26 21:47:41,944][105692] Updated weights for policy 0, policy_version 883603 (0.0007) [2023-12-26 21:47:42,010][105692] Updated weights for policy 0, policy_version 883613 (0.0006) [2023-12-26 21:47:42,075][105692] Updated weights for policy 0, policy_version 883623 (0.0008) [2023-12-26 21:47:42,613][105620] Updated weights for policy 1, policy_version 883615 (0.0008) [2023-12-26 21:47:42,668][105620] Updated weights for policy 1, policy_version 883625 (0.0007) [2023-12-26 21:47:42,720][105692] Updated weights for policy 0, policy_version 883633 (0.0009) [2023-12-26 21:47:42,721][105620] Updated weights for policy 1, policy_version 883635 (0.0005) [2023-12-26 21:47:42,789][105692] Updated weights for policy 0, policy_version 883643 (0.0006) [2023-12-26 21:47:42,850][105692] Updated weights for policy 0, policy_version 883653 (0.0006) [2023-12-26 21:47:42,913][105692] Updated weights for policy 0, policy_version 883663 (0.0007) [2023-12-26 21:47:43,416][105620] Updated weights for policy 1, policy_version 883645 (0.0009) [2023-12-26 21:47:43,474][105620] Updated weights for policy 1, policy_version 883655 (0.0009) [2023-12-26 21:47:43,525][105620] Updated weights for policy 1, policy_version 883665 (0.0007) [2023-12-26 21:47:43,548][105692] Updated weights for policy 0, policy_version 883673 (0.0007) [2023-12-26 21:47:43,613][105692] Updated weights for policy 0, policy_version 883683 (0.0010) [2023-12-26 21:47:43,667][105692] Updated weights for policy 0, policy_version 883693 (0.0009) [2023-12-26 21:47:44,286][105620] Updated weights for policy 1, policy_version 883675 (0.0010) [2023-12-26 21:47:44,341][105620] Updated weights for policy 1, policy_version 883685 (0.0010) [2023-12-26 21:47:44,396][105620] Updated weights for policy 1, policy_version 883695 (0.0010) [2023-12-26 21:47:44,398][105692] Updated weights for policy 0, policy_version 883703 (0.0006) [2023-12-26 21:47:44,454][105692] Updated weights for policy 0, policy_version 883713 (0.0006) [2023-12-26 21:47:44,498][105692] Updated weights for policy 0, policy_version 883723 (0.0008) [2023-12-26 21:47:45,166][105692] Updated weights for policy 0, policy_version 883733 (0.0007) [2023-12-26 21:47:45,175][105620] Updated weights for policy 1, policy_version 883705 (0.0010) [2023-12-26 21:47:45,229][105692] Updated weights for policy 0, policy_version 883743 (0.0006) [2023-12-26 21:47:45,239][105620] Updated weights for policy 1, policy_version 883715 (0.0011) [2023-12-26 21:47:45,292][105692] Updated weights for policy 0, policy_version 883753 (0.0006) [2023-12-26 21:47:45,302][105620] Updated weights for policy 1, policy_version 883725 (0.0011) [2023-12-26 21:47:45,358][105620] Updated weights for policy 1, policy_version 883735 (0.0011) [2023-12-26 21:47:45,953][105692] Updated weights for policy 0, policy_version 883763 (0.0006) [2023-12-26 21:47:46,015][105692] Updated weights for policy 0, policy_version 883773 (0.0008) [2023-12-26 21:47:46,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 452542464. Throughput: 0: 9642.2, 1: 9719.2. Samples: 452519056. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-26 21:47:46,062][104569] Avg episode reward: [(0, '8820.414'), (1, '8805.140')] [2023-12-26 21:47:46,080][105692] Updated weights for policy 0, policy_version 883783 (0.0007) [2023-12-26 21:47:46,094][105620] Updated weights for policy 1, policy_version 883745 (0.0011) [2023-12-26 21:47:46,130][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000883792_226287616.pth... [2023-12-26 21:47:46,134][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000882640_225992704.pth [2023-12-26 21:47:46,153][105620] Updated weights for policy 1, policy_version 883755 (0.0011) [2023-12-26 21:47:46,212][105620] Updated weights for policy 1, policy_version 883765 (0.0010) [2023-12-26 21:47:46,229][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000883768_226271232.pth... [2023-12-26 21:47:46,233][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000882648_225984512.pth [2023-12-26 21:47:46,688][105692] Updated weights for policy 0, policy_version 883793 (0.0005) [2023-12-26 21:47:46,753][105692] Updated weights for policy 0, policy_version 883803 (0.0007) [2023-12-26 21:47:46,809][105692] Updated weights for policy 0, policy_version 883813 (0.0009) [2023-12-26 21:47:46,871][105692] Updated weights for policy 0, policy_version 883823 (0.0009) [2023-12-26 21:47:47,009][105620] Updated weights for policy 1, policy_version 883775 (0.0007) [2023-12-26 21:47:47,068][105620] Updated weights for policy 1, policy_version 883785 (0.0005) [2023-12-26 21:47:47,119][105620] Updated weights for policy 1, policy_version 883795 (0.0006) [2023-12-26 21:47:47,455][105692] Updated weights for policy 0, policy_version 883833 (0.0006) [2023-12-26 21:47:47,509][105692] Updated weights for policy 0, policy_version 883843 (0.0005) [2023-12-26 21:47:47,561][105692] Updated weights for policy 0, policy_version 883853 (0.0006) [2023-12-26 21:47:47,702][105620] Updated weights for policy 1, policy_version 883805 (0.0007) [2023-12-26 21:47:47,772][105620] Updated weights for policy 1, policy_version 883815 (0.0009) [2023-12-26 21:47:47,832][105620] Updated weights for policy 1, policy_version 883825 (0.0006) [2023-12-26 21:47:48,150][105692] Updated weights for policy 0, policy_version 883863 (0.0010) [2023-12-26 21:47:48,169][105585] KL-divergence is very high: 135.5855 [2023-12-26 21:47:48,184][105585] KL-divergence is very high: 131.4489 [2023-12-26 21:47:48,205][105692] Updated weights for policy 0, policy_version 883873 (0.0010) [2023-12-26 21:47:48,209][105585] KL-divergence is very high: 232.0476 [2023-12-26 21:47:48,223][105585] KL-divergence is very high: 146.6862 [2023-12-26 21:47:48,256][105585] KL-divergence is very high: 162.0586 [2023-12-26 21:47:48,261][105692] Updated weights for policy 0, policy_version 883883 (0.0010) [2023-12-26 21:47:48,442][105620] Updated weights for policy 1, policy_version 883835 (0.0006) [2023-12-26 21:47:48,497][105620] Updated weights for policy 1, policy_version 883845 (0.0005) [2023-12-26 21:47:48,551][105620] Updated weights for policy 1, policy_version 883855 (0.0007) [2023-12-26 21:47:49,013][105692] Updated weights for policy 0, policy_version 883893 (0.0010) [2023-12-26 21:47:49,061][105692] Updated weights for policy 0, policy_version 883903 (0.0010) [2023-12-26 21:47:49,117][105692] Updated weights for policy 0, policy_version 883913 (0.0010) [2023-12-26 21:47:49,165][105620] Updated weights for policy 1, policy_version 883865 (0.0008) [2023-12-26 21:47:49,229][105620] Updated weights for policy 1, policy_version 883875 (0.0008) [2023-12-26 21:47:49,295][105620] Updated weights for policy 1, policy_version 883885 (0.0007) [2023-12-26 21:47:49,359][105620] Updated weights for policy 1, policy_version 883895 (0.0008) [2023-12-26 21:47:49,868][105692] Updated weights for policy 0, policy_version 883923 (0.0010) [2023-12-26 21:47:49,932][105692] Updated weights for policy 0, policy_version 883933 (0.0008) [2023-12-26 21:47:49,996][105692] Updated weights for policy 0, policy_version 883943 (0.0006) [2023-12-26 21:47:50,050][105620] Updated weights for policy 1, policy_version 883905 (0.0010) [2023-12-26 21:47:50,110][105620] Updated weights for policy 1, policy_version 883915 (0.0011) [2023-12-26 21:47:50,172][105620] Updated weights for policy 1, policy_version 883925 (0.0010) [2023-12-26 21:47:50,718][105692] Updated weights for policy 0, policy_version 883953 (0.0006) [2023-12-26 21:47:50,768][105692] Updated weights for policy 0, policy_version 883963 (0.0005) [2023-12-26 21:47:50,820][105692] Updated weights for policy 0, policy_version 883973 (0.0006) [2023-12-26 21:47:50,888][105692] Updated weights for policy 0, policy_version 883983 (0.0007) [2023-12-26 21:47:50,922][105620] Updated weights for policy 1, policy_version 883935 (0.0011) [2023-12-26 21:47:50,986][105620] Updated weights for policy 1, policy_version 883945 (0.0011) [2023-12-26 21:47:51,058][105620] Updated weights for policy 1, policy_version 883955 (0.0009) [2023-12-26 21:47:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 452648960. Throughput: 0: 9714.7, 1: 9772.3. Samples: 452641136. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-26 21:47:51,062][104569] Avg episode reward: [(0, '8211.622'), (1, '8803.583')] [2023-12-26 21:47:51,627][105692] Updated weights for policy 0, policy_version 883993 (0.0009) [2023-12-26 21:47:51,689][105692] Updated weights for policy 0, policy_version 884003 (0.0008) [2023-12-26 21:47:51,769][105692] Updated weights for policy 0, policy_version 884013 (0.0010) [2023-12-26 21:47:51,789][105620] Updated weights for policy 1, policy_version 883965 (0.0007) [2023-12-26 21:47:51,850][105620] Updated weights for policy 1, policy_version 883975 (0.0007) [2023-12-26 21:47:51,912][105620] Updated weights for policy 1, policy_version 883985 (0.0005) [2023-12-26 21:47:52,478][105620] Updated weights for policy 1, policy_version 883995 (0.0006) [2023-12-26 21:47:52,538][105620] Updated weights for policy 1, policy_version 884005 (0.0008) [2023-12-26 21:47:52,601][105620] Updated weights for policy 1, policy_version 884015 (0.0009) [2023-12-26 21:47:52,611][105692] Updated weights for policy 0, policy_version 884023 (0.0007) [2023-12-26 21:47:52,665][105692] Updated weights for policy 0, policy_version 884033 (0.0008) [2023-12-26 21:47:52,726][105692] Updated weights for policy 0, policy_version 884043 (0.0010) [2023-12-26 21:47:53,209][105620] Updated weights for policy 1, policy_version 884025 (0.0010) [2023-12-26 21:47:53,260][105620] Updated weights for policy 1, policy_version 884035 (0.0005) [2023-12-26 21:47:53,306][105620] Updated weights for policy 1, policy_version 884045 (0.0009) [2023-12-26 21:47:53,354][105620] Updated weights for policy 1, policy_version 884055 (0.0010) [2023-12-26 21:47:53,538][105692] Updated weights for policy 0, policy_version 884053 (0.0008) [2023-12-26 21:47:53,588][105692] Updated weights for policy 0, policy_version 884063 (0.0007) [2023-12-26 21:47:53,635][105692] Updated weights for policy 0, policy_version 884073 (0.0005) [2023-12-26 21:47:54,108][105620] Updated weights for policy 1, policy_version 884065 (0.0010) [2023-12-26 21:47:54,182][105620] Updated weights for policy 1, policy_version 884075 (0.0008) [2023-12-26 21:47:54,247][105620] Updated weights for policy 1, policy_version 884085 (0.0005) [2023-12-26 21:47:54,262][105692] Updated weights for policy 0, policy_version 884083 (0.0005) [2023-12-26 21:47:54,319][105692] Updated weights for policy 0, policy_version 884093 (0.0005) [2023-12-26 21:47:54,385][105692] Updated weights for policy 0, policy_version 884103 (0.0007) [2023-12-26 21:47:54,841][105620] Updated weights for policy 1, policy_version 884095 (0.0006) [2023-12-26 21:47:54,901][105620] Updated weights for policy 1, policy_version 884105 (0.0005) [2023-12-26 21:47:54,959][105620] Updated weights for policy 1, policy_version 884115 (0.0006) [2023-12-26 21:47:55,137][105692] Updated weights for policy 0, policy_version 884113 (0.0009) [2023-12-26 21:47:55,202][105692] Updated weights for policy 0, policy_version 884123 (0.0006) [2023-12-26 21:47:55,258][105692] Updated weights for policy 0, policy_version 884133 (0.0007) [2023-12-26 21:47:55,321][105692] Updated weights for policy 0, policy_version 884143 (0.0006) [2023-12-26 21:47:55,625][105620] Updated weights for policy 1, policy_version 884125 (0.0007) [2023-12-26 21:47:55,690][105620] Updated weights for policy 1, policy_version 884135 (0.0008) [2023-12-26 21:47:55,740][105620] Updated weights for policy 1, policy_version 884145 (0.0008) [2023-12-26 21:47:55,902][105692] Updated weights for policy 0, policy_version 884153 (0.0007) [2023-12-26 21:47:55,949][105692] Updated weights for policy 0, policy_version 884163 (0.0005) [2023-12-26 21:47:55,997][105692] Updated weights for policy 0, policy_version 884173 (0.0005) [2023-12-26 21:47:56,062][104569] Fps is (10 sec: 21298.6, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 452755456. Throughput: 0: 9707.2, 1: 9754.3. Samples: 452758808. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-26 21:47:56,063][104569] Avg episode reward: [(0, '8474.327'), (1, '8896.724')] [2023-12-26 21:47:56,456][105620] Updated weights for policy 1, policy_version 884155 (0.0009) [2023-12-26 21:47:56,513][105620] Updated weights for policy 1, policy_version 884165 (0.0009) [2023-12-26 21:47:56,567][105620] Updated weights for policy 1, policy_version 884175 (0.0009) [2023-12-26 21:47:56,659][105692] Updated weights for policy 0, policy_version 884183 (0.0005) [2023-12-26 21:47:56,705][105692] Updated weights for policy 0, policy_version 884193 (0.0006) [2023-12-26 21:47:56,759][105692] Updated weights for policy 0, policy_version 884203 (0.0009) [2023-12-26 21:47:57,371][105620] Updated weights for policy 1, policy_version 884185 (0.0010) [2023-12-26 21:47:57,389][105692] Updated weights for policy 0, policy_version 884213 (0.0008) [2023-12-26 21:47:57,423][105620] Updated weights for policy 1, policy_version 884195 (0.0006) [2023-12-26 21:47:57,444][105692] Updated weights for policy 0, policy_version 884223 (0.0008) [2023-12-26 21:47:57,471][105620] Updated weights for policy 1, policy_version 884205 (0.0007) [2023-12-26 21:47:57,490][105692] Updated weights for policy 0, policy_version 884233 (0.0007) [2023-12-26 21:47:57,516][105620] Updated weights for policy 1, policy_version 884215 (0.0007) [2023-12-26 21:47:58,263][105692] Updated weights for policy 0, policy_version 884243 (0.0006) [2023-12-26 21:47:58,273][105620] Updated weights for policy 1, policy_version 884225 (0.0009) [2023-12-26 21:47:58,329][105692] Updated weights for policy 0, policy_version 884253 (0.0008) [2023-12-26 21:47:58,350][105620] Updated weights for policy 1, policy_version 884235 (0.0009) [2023-12-26 21:47:58,394][105692] Updated weights for policy 0, policy_version 884263 (0.0008) [2023-12-26 21:47:58,416][105620] Updated weights for policy 1, policy_version 884245 (0.0008) [2023-12-26 21:47:59,230][105620] Updated weights for policy 1, policy_version 884255 (0.0008) [2023-12-26 21:47:59,238][105692] Updated weights for policy 0, policy_version 884273 (0.0008) [2023-12-26 21:47:59,301][105620] Updated weights for policy 1, policy_version 884265 (0.0007) [2023-12-26 21:47:59,305][105692] Updated weights for policy 0, policy_version 884283 (0.0008) [2023-12-26 21:47:59,362][105620] Updated weights for policy 1, policy_version 884275 (0.0007) [2023-12-26 21:47:59,367][105692] Updated weights for policy 0, policy_version 884293 (0.0008) [2023-12-26 21:47:59,422][105692] Updated weights for policy 0, policy_version 884303 (0.0009) [2023-12-26 21:47:59,957][105620] Updated weights for policy 1, policy_version 884285 (0.0009) [2023-12-26 21:48:00,009][105620] Updated weights for policy 1, policy_version 884295 (0.0010) [2023-12-26 21:48:00,063][105620] Updated weights for policy 1, policy_version 884305 (0.0010) [2023-12-26 21:48:00,254][105692] Updated weights for policy 0, policy_version 884313 (0.0008) [2023-12-26 21:48:00,308][105692] Updated weights for policy 0, policy_version 884323 (0.0008) [2023-12-26 21:48:00,362][105692] Updated weights for policy 0, policy_version 884333 (0.0006) [2023-12-26 21:48:00,679][105620] Updated weights for policy 1, policy_version 884315 (0.0007) [2023-12-26 21:48:00,734][105620] Updated weights for policy 1, policy_version 884325 (0.0005) [2023-12-26 21:48:00,792][105620] Updated weights for policy 1, policy_version 884335 (0.0005) [2023-12-26 21:48:01,015][105692] Updated weights for policy 0, policy_version 884343 (0.0009) [2023-12-26 21:48:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 452845568. Throughput: 0: 9760.1, 1: 9692.6. Samples: 452816212. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-26 21:48:01,062][104569] Avg episode reward: [(0, '8903.106'), (1, '8351.037')] [2023-12-26 21:48:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000884344_226418688.pth... [2023-12-26 21:48:01,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000883192_226123776.pth [2023-12-26 21:48:01,078][105692] Updated weights for policy 0, policy_version 884353 (0.0009) [2023-12-26 21:48:01,139][105692] Updated weights for policy 0, policy_version 884363 (0.0008) [2023-12-26 21:48:01,171][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000884368_226435072.pth... [2023-12-26 21:48:01,174][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000883248_226148352.pth [2023-12-26 21:48:01,423][105620] Updated weights for policy 1, policy_version 884345 (0.0005) [2023-12-26 21:48:01,481][105620] Updated weights for policy 1, policy_version 884355 (0.0005) [2023-12-26 21:48:01,539][105620] Updated weights for policy 1, policy_version 884365 (0.0006) [2023-12-26 21:48:01,587][105620] Updated weights for policy 1, policy_version 884375 (0.0010) [2023-12-26 21:48:01,965][105692] Updated weights for policy 0, policy_version 884373 (0.0009) [2023-12-26 21:48:02,013][105692] Updated weights for policy 0, policy_version 884383 (0.0010) [2023-12-26 21:48:02,075][105692] Updated weights for policy 0, policy_version 884393 (0.0009) [2023-12-26 21:48:02,274][105620] Updated weights for policy 1, policy_version 884385 (0.0010) [2023-12-26 21:48:02,342][105620] Updated weights for policy 1, policy_version 884395 (0.0010) [2023-12-26 21:48:02,400][105620] Updated weights for policy 1, policy_version 884405 (0.0011) [2023-12-26 21:48:02,688][105692] Updated weights for policy 0, policy_version 884403 (0.0008) [2023-12-26 21:48:02,736][105692] Updated weights for policy 0, policy_version 884413 (0.0008) [2023-12-26 21:48:02,787][105692] Updated weights for policy 0, policy_version 884423 (0.0008) [2023-12-26 21:48:03,131][105620] Updated weights for policy 1, policy_version 884415 (0.0010) [2023-12-26 21:48:03,193][105620] Updated weights for policy 1, policy_version 884425 (0.0008) [2023-12-26 21:48:03,249][105620] Updated weights for policy 1, policy_version 884435 (0.0005) [2023-12-26 21:48:03,378][105692] Updated weights for policy 0, policy_version 884433 (0.0007) [2023-12-26 21:48:03,449][105692] Updated weights for policy 0, policy_version 884443 (0.0005) [2023-12-26 21:48:03,508][105692] Updated weights for policy 0, policy_version 884453 (0.0006) [2023-12-26 21:48:03,566][105692] Updated weights for policy 0, policy_version 884463 (0.0005) [2023-12-26 21:48:03,788][105620] Updated weights for policy 1, policy_version 884445 (0.0005) [2023-12-26 21:48:03,840][105620] Updated weights for policy 1, policy_version 884455 (0.0006) [2023-12-26 21:48:03,896][105620] Updated weights for policy 1, policy_version 884465 (0.0010) [2023-12-26 21:48:04,204][105692] Updated weights for policy 0, policy_version 884473 (0.0010) [2023-12-26 21:48:04,265][105692] Updated weights for policy 0, policy_version 884483 (0.0011) [2023-12-26 21:48:04,329][105692] Updated weights for policy 0, policy_version 884493 (0.0011) [2023-12-26 21:48:04,640][105620] Updated weights for policy 1, policy_version 884475 (0.0010) [2023-12-26 21:48:04,699][105620] Updated weights for policy 1, policy_version 884485 (0.0010) [2023-12-26 21:48:04,758][105620] Updated weights for policy 1, policy_version 884495 (0.0011) [2023-12-26 21:48:05,053][105692] Updated weights for policy 0, policy_version 884503 (0.0007) [2023-12-26 21:48:05,096][105692] Updated weights for policy 0, policy_version 884513 (0.0005) [2023-12-26 21:48:05,143][105692] Updated weights for policy 0, policy_version 884523 (0.0005) [2023-12-26 21:48:05,516][105620] Updated weights for policy 1, policy_version 884505 (0.0010) [2023-12-26 21:48:05,578][105620] Updated weights for policy 1, policy_version 884515 (0.0009) [2023-12-26 21:48:05,637][105620] Updated weights for policy 1, policy_version 884525 (0.0009) [2023-12-26 21:48:05,701][105620] Updated weights for policy 1, policy_version 884535 (0.0007) [2023-12-26 21:48:05,903][105692] Updated weights for policy 0, policy_version 884533 (0.0007) [2023-12-26 21:48:05,969][105692] Updated weights for policy 0, policy_version 884543 (0.0006) [2023-12-26 21:48:06,016][105692] Updated weights for policy 0, policy_version 884553 (0.0008) [2023-12-26 21:48:06,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 452952064. Throughput: 0: 9830.8, 1: 9760.8. Samples: 452936704. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-26 21:48:06,063][104569] Avg episode reward: [(0, '9083.786'), (1, '8189.557')] [2023-12-26 21:48:06,447][105620] Updated weights for policy 1, policy_version 884545 (0.0010) [2023-12-26 21:48:06,512][105620] Updated weights for policy 1, policy_version 884555 (0.0010) [2023-12-26 21:48:06,570][105620] Updated weights for policy 1, policy_version 884565 (0.0009) [2023-12-26 21:48:06,776][105692] Updated weights for policy 0, policy_version 884563 (0.0009) [2023-12-26 21:48:06,825][105692] Updated weights for policy 0, policy_version 884573 (0.0010) [2023-12-26 21:48:06,890][105692] Updated weights for policy 0, policy_version 884583 (0.0009) [2023-12-26 21:48:07,278][105620] Updated weights for policy 1, policy_version 884575 (0.0010) [2023-12-26 21:48:07,330][105620] Updated weights for policy 1, policy_version 884585 (0.0010) [2023-12-26 21:48:07,396][105620] Updated weights for policy 1, policy_version 884595 (0.0010) [2023-12-26 21:48:07,588][105692] Updated weights for policy 0, policy_version 884593 (0.0010) [2023-12-26 21:48:07,637][105692] Updated weights for policy 0, policy_version 884603 (0.0005) [2023-12-26 21:48:07,683][105692] Updated weights for policy 0, policy_version 884613 (0.0005) [2023-12-26 21:48:07,734][105692] Updated weights for policy 0, policy_version 884623 (0.0007) [2023-12-26 21:48:08,056][105620] Updated weights for policy 1, policy_version 884605 (0.0008) [2023-12-26 21:48:08,113][105620] Updated weights for policy 1, policy_version 884615 (0.0006) [2023-12-26 21:48:08,169][105620] Updated weights for policy 1, policy_version 884625 (0.0007) [2023-12-26 21:48:08,408][105692] Updated weights for policy 0, policy_version 884633 (0.0007) [2023-12-26 21:48:08,472][105692] Updated weights for policy 0, policy_version 884643 (0.0007) [2023-12-26 21:48:08,531][105692] Updated weights for policy 0, policy_version 884653 (0.0009) [2023-12-26 21:48:08,877][105620] Updated weights for policy 1, policy_version 884635 (0.0006) [2023-12-26 21:48:08,933][105620] Updated weights for policy 1, policy_version 884645 (0.0008) [2023-12-26 21:48:08,993][105620] Updated weights for policy 1, policy_version 884655 (0.0008) [2023-12-26 21:48:09,263][105692] Updated weights for policy 0, policy_version 884663 (0.0011) [2023-12-26 21:48:09,329][105692] Updated weights for policy 0, policy_version 884673 (0.0011) [2023-12-26 21:48:09,400][105692] Updated weights for policy 0, policy_version 884683 (0.0012) [2023-12-26 21:48:09,719][105620] Updated weights for policy 1, policy_version 884665 (0.0008) [2023-12-26 21:48:09,781][105620] Updated weights for policy 1, policy_version 884675 (0.0008) [2023-12-26 21:48:09,836][105620] Updated weights for policy 1, policy_version 884685 (0.0010) [2023-12-26 21:48:09,900][105620] Updated weights for policy 1, policy_version 884695 (0.0007) [2023-12-26 21:48:10,146][105692] Updated weights for policy 0, policy_version 884693 (0.0009) [2023-12-26 21:48:10,209][105692] Updated weights for policy 0, policy_version 884703 (0.0008) [2023-12-26 21:48:10,276][105692] Updated weights for policy 0, policy_version 884713 (0.0008) [2023-12-26 21:48:10,581][105620] Updated weights for policy 1, policy_version 884705 (0.0007) [2023-12-26 21:48:10,629][105620] Updated weights for policy 1, policy_version 884715 (0.0006) [2023-12-26 21:48:10,684][105620] Updated weights for policy 1, policy_version 884725 (0.0010) [2023-12-26 21:48:11,056][105692] Updated weights for policy 0, policy_version 884723 (0.0008) [2023-12-26 21:48:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 453042176. Throughput: 0: 9754.2, 1: 9840.1. Samples: 453053256. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-26 21:48:11,062][104569] Avg episode reward: [(0, '9172.690'), (1, '8795.626')] [2023-12-26 21:48:11,112][105692] Updated weights for policy 0, policy_version 884733 (0.0009) [2023-12-26 21:48:11,173][105692] Updated weights for policy 0, policy_version 884743 (0.0007) [2023-12-26 21:48:11,356][105620] Updated weights for policy 1, policy_version 884735 (0.0009) [2023-12-26 21:48:11,428][105620] Updated weights for policy 1, policy_version 884745 (0.0009) [2023-12-26 21:48:11,482][105620] Updated weights for policy 1, policy_version 884755 (0.0007) [2023-12-26 21:48:11,960][105692] Updated weights for policy 0, policy_version 884753 (0.0007) [2023-12-26 21:48:12,015][105692] Updated weights for policy 0, policy_version 884763 (0.0009) [2023-12-26 21:48:12,063][105692] Updated weights for policy 0, policy_version 884773 (0.0008) [2023-12-26 21:48:12,110][105692] Updated weights for policy 0, policy_version 884783 (0.0009) [2023-12-26 21:48:12,181][105620] Updated weights for policy 1, policy_version 884765 (0.0007) [2023-12-26 21:48:12,236][105620] Updated weights for policy 1, policy_version 884775 (0.0006) [2023-12-26 21:48:12,299][105620] Updated weights for policy 1, policy_version 884785 (0.0007) [2023-12-26 21:48:12,884][105692] Updated weights for policy 0, policy_version 884793 (0.0010) [2023-12-26 21:48:12,932][105692] Updated weights for policy 0, policy_version 884803 (0.0010) [2023-12-26 21:48:12,985][105692] Updated weights for policy 0, policy_version 884813 (0.0010) [2023-12-26 21:48:13,066][105620] Updated weights for policy 1, policy_version 884795 (0.0008) [2023-12-26 21:48:13,114][105620] Updated weights for policy 1, policy_version 884805 (0.0008) [2023-12-26 21:48:13,171][105620] Updated weights for policy 1, policy_version 884815 (0.0007) [2023-12-26 21:48:13,738][105692] Updated weights for policy 0, policy_version 884823 (0.0007) [2023-12-26 21:48:13,789][105692] Updated weights for policy 0, policy_version 884833 (0.0008) [2023-12-26 21:48:13,848][105692] Updated weights for policy 0, policy_version 884843 (0.0008) [2023-12-26 21:48:13,865][105620] Updated weights for policy 1, policy_version 884825 (0.0007) [2023-12-26 21:48:13,914][105620] Updated weights for policy 1, policy_version 884835 (0.0008) [2023-12-26 21:48:13,959][105620] Updated weights for policy 1, policy_version 884845 (0.0006) [2023-12-26 21:48:14,014][105620] Updated weights for policy 1, policy_version 884855 (0.0006) [2023-12-26 21:48:14,511][105692] Updated weights for policy 0, policy_version 884853 (0.0008) [2023-12-26 21:48:14,562][105692] Updated weights for policy 0, policy_version 884863 (0.0005) [2023-12-26 21:48:14,613][105692] Updated weights for policy 0, policy_version 884873 (0.0007) [2023-12-26 21:48:14,666][105620] Updated weights for policy 1, policy_version 884865 (0.0006) [2023-12-26 21:48:14,722][105620] Updated weights for policy 1, policy_version 884875 (0.0009) [2023-12-26 21:48:14,797][105620] Updated weights for policy 1, policy_version 884885 (0.0008) [2023-12-26 21:48:15,257][105692] Updated weights for policy 0, policy_version 884883 (0.0010) [2023-12-26 21:48:15,321][105692] Updated weights for policy 0, policy_version 884893 (0.0011) [2023-12-26 21:48:15,378][105692] Updated weights for policy 0, policy_version 884903 (0.0011) [2023-12-26 21:48:15,612][105620] Updated weights for policy 1, policy_version 884895 (0.0008) [2023-12-26 21:48:15,661][105620] Updated weights for policy 1, policy_version 884905 (0.0008) [2023-12-26 21:48:15,717][105620] Updated weights for policy 1, policy_version 884915 (0.0007) [2023-12-26 21:48:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 453140480. Throughput: 0: 9641.4, 1: 9862.2. Samples: 453110316. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-26 21:48:16,063][104569] Avg episode reward: [(0, '8815.770'), (1, '9263.933')] [2023-12-26 21:48:16,063][105692] Updated weights for policy 0, policy_version 884913 (0.0010) [2023-12-26 21:48:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000884920_226566144.pth... [2023-12-26 21:48:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000883768_226271232.pth [2023-12-26 21:48:16,125][105692] Updated weights for policy 0, policy_version 884923 (0.0005) [2023-12-26 21:48:16,187][105692] Updated weights for policy 0, policy_version 884933 (0.0006) [2023-12-26 21:48:16,238][105692] Updated weights for policy 0, policy_version 884943 (0.0005) [2023-12-26 21:48:16,242][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000884944_226582528.pth... [2023-12-26 21:48:16,244][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000883792_226287616.pth [2023-12-26 21:48:16,375][105620] Updated weights for policy 1, policy_version 884925 (0.0007) [2023-12-26 21:48:16,426][105620] Updated weights for policy 1, policy_version 884935 (0.0008) [2023-12-26 21:48:16,472][105620] Updated weights for policy 1, policy_version 884945 (0.0009) [2023-12-26 21:48:16,811][105692] Updated weights for policy 0, policy_version 884953 (0.0009) [2023-12-26 21:48:16,873][105692] Updated weights for policy 0, policy_version 884963 (0.0010) [2023-12-26 21:48:16,934][105692] Updated weights for policy 0, policy_version 884973 (0.0010) [2023-12-26 21:48:17,284][105620] Updated weights for policy 1, policy_version 884955 (0.0009) [2023-12-26 21:48:17,343][105620] Updated weights for policy 1, policy_version 884965 (0.0008) [2023-12-26 21:48:17,398][105620] Updated weights for policy 1, policy_version 884975 (0.0008) [2023-12-26 21:48:17,684][105692] Updated weights for policy 0, policy_version 884983 (0.0010) [2023-12-26 21:48:17,738][105692] Updated weights for policy 0, policy_version 884993 (0.0010) [2023-12-26 21:48:17,793][105692] Updated weights for policy 0, policy_version 885003 (0.0010) [2023-12-26 21:48:18,157][105620] Updated weights for policy 1, policy_version 884985 (0.0009) [2023-12-26 21:48:18,214][105620] Updated weights for policy 1, policy_version 884995 (0.0008) [2023-12-26 21:48:18,266][105620] Updated weights for policy 1, policy_version 885005 (0.0009) [2023-12-26 21:48:18,313][105620] Updated weights for policy 1, policy_version 885015 (0.0008) [2023-12-26 21:48:18,528][105692] Updated weights for policy 0, policy_version 885013 (0.0010) [2023-12-26 21:48:18,576][105692] Updated weights for policy 0, policy_version 885023 (0.0010) [2023-12-26 21:48:18,631][105692] Updated weights for policy 0, policy_version 885033 (0.0010) [2023-12-26 21:48:19,106][105620] Updated weights for policy 1, policy_version 885025 (0.0008) [2023-12-26 21:48:19,154][105620] Updated weights for policy 1, policy_version 885035 (0.0008) [2023-12-26 21:48:19,205][105620] Updated weights for policy 1, policy_version 885045 (0.0007) [2023-12-26 21:48:19,413][105692] Updated weights for policy 0, policy_version 885043 (0.0011) [2023-12-26 21:48:19,469][105692] Updated weights for policy 0, policy_version 885053 (0.0011) [2023-12-26 21:48:19,533][105692] Updated weights for policy 0, policy_version 885063 (0.0011) [2023-12-26 21:48:20,004][105620] Updated weights for policy 1, policy_version 885055 (0.0008) [2023-12-26 21:48:20,057][105620] Updated weights for policy 1, policy_version 885065 (0.0008) [2023-12-26 21:48:20,113][105620] Updated weights for policy 1, policy_version 885075 (0.0008) [2023-12-26 21:48:20,304][105692] Updated weights for policy 0, policy_version 885073 (0.0011) [2023-12-26 21:48:20,368][105692] Updated weights for policy 0, policy_version 885083 (0.0011) [2023-12-26 21:48:20,424][105692] Updated weights for policy 0, policy_version 885093 (0.0011) [2023-12-26 21:48:20,484][105692] Updated weights for policy 0, policy_version 885103 (0.0011) [2023-12-26 21:48:20,886][105620] Updated weights for policy 1, policy_version 885085 (0.0008) [2023-12-26 21:48:20,955][105620] Updated weights for policy 1, policy_version 885095 (0.0008) [2023-12-26 21:48:21,016][105620] Updated weights for policy 1, policy_version 885105 (0.0008) [2023-12-26 21:48:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 453230592. Throughput: 0: 9695.3, 1: 9814.8. Samples: 453227536. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-26 21:48:21,063][104569] Avg episode reward: [(0, '9083.777'), (1, '9173.251')] [2023-12-26 21:48:21,274][105692] Updated weights for policy 0, policy_version 885113 (0.0011) [2023-12-26 21:48:21,338][105692] Updated weights for policy 0, policy_version 885123 (0.0011) [2023-12-26 21:48:21,407][105692] Updated weights for policy 0, policy_version 885133 (0.0011) [2023-12-26 21:48:21,818][105620] Updated weights for policy 1, policy_version 885115 (0.0009) [2023-12-26 21:48:21,870][105620] Updated weights for policy 1, policy_version 885125 (0.0008) [2023-12-26 21:48:21,925][105620] Updated weights for policy 1, policy_version 885135 (0.0010) [2023-12-26 21:48:22,185][105692] Updated weights for policy 0, policy_version 885143 (0.0011) [2023-12-26 21:48:22,248][105692] Updated weights for policy 0, policy_version 885153 (0.0011) [2023-12-26 21:48:22,316][105692] Updated weights for policy 0, policy_version 885163 (0.0011) [2023-12-26 21:48:22,608][105620] Updated weights for policy 1, policy_version 885145 (0.0007) [2023-12-26 21:48:22,667][105620] Updated weights for policy 1, policy_version 885155 (0.0010) [2023-12-26 21:48:22,731][105620] Updated weights for policy 1, policy_version 885165 (0.0009) [2023-12-26 21:48:22,780][105620] Updated weights for policy 1, policy_version 885175 (0.0010) [2023-12-26 21:48:23,006][105692] Updated weights for policy 0, policy_version 885173 (0.0009) [2023-12-26 21:48:23,066][105692] Updated weights for policy 0, policy_version 885183 (0.0008) [2023-12-26 21:48:23,122][105692] Updated weights for policy 0, policy_version 885193 (0.0008) [2023-12-26 21:48:23,553][105620] Updated weights for policy 1, policy_version 885185 (0.0010) [2023-12-26 21:48:23,602][105620] Updated weights for policy 1, policy_version 885195 (0.0010) [2023-12-26 21:48:23,646][105620] Updated weights for policy 1, policy_version 885205 (0.0010) [2023-12-26 21:48:23,825][105692] Updated weights for policy 0, policy_version 885203 (0.0009) [2023-12-26 21:48:23,888][105692] Updated weights for policy 0, policy_version 885213 (0.0009) [2023-12-26 21:48:23,940][105692] Updated weights for policy 0, policy_version 885223 (0.0009) [2023-12-26 21:48:24,219][105620] Updated weights for policy 1, policy_version 885215 (0.0007) [2023-12-26 21:48:24,276][105620] Updated weights for policy 1, policy_version 885225 (0.0006) [2023-12-26 21:48:24,341][105620] Updated weights for policy 1, policy_version 885235 (0.0009) [2023-12-26 21:48:24,804][105692] Updated weights for policy 0, policy_version 885233 (0.0010) [2023-12-26 21:48:24,863][105692] Updated weights for policy 0, policy_version 885243 (0.0009) [2023-12-26 21:48:24,925][105692] Updated weights for policy 0, policy_version 885253 (0.0009) [2023-12-26 21:48:24,980][105620] Updated weights for policy 1, policy_version 885245 (0.0009) [2023-12-26 21:48:24,983][105692] Updated weights for policy 0, policy_version 885263 (0.0007) [2023-12-26 21:48:25,045][105620] Updated weights for policy 1, policy_version 885255 (0.0009) [2023-12-26 21:48:25,109][105620] Updated weights for policy 1, policy_version 885265 (0.0008) [2023-12-26 21:48:25,767][105620] Updated weights for policy 1, policy_version 885275 (0.0008) [2023-12-26 21:48:25,770][105692] Updated weights for policy 0, policy_version 885273 (0.0010) [2023-12-26 21:48:25,814][105620] Updated weights for policy 1, policy_version 885285 (0.0007) [2023-12-26 21:48:25,823][105692] Updated weights for policy 0, policy_version 885283 (0.0009) [2023-12-26 21:48:25,866][105620] Updated weights for policy 1, policy_version 885295 (0.0011) [2023-12-26 21:48:25,876][105692] Updated weights for policy 0, policy_version 885293 (0.0006) [2023-12-26 21:48:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 453337088. Throughput: 0: 9732.2, 1: 9819.0. Samples: 453341308. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-26 21:48:26,062][104569] Avg episode reward: [(0, '9083.162'), (1, '9173.702')] [2023-12-26 21:48:26,498][105620] Updated weights for policy 1, policy_version 885305 (0.0010) [2023-12-26 21:48:26,565][105620] Updated weights for policy 1, policy_version 885315 (0.0008) [2023-12-26 21:48:26,603][105692] Updated weights for policy 0, policy_version 885303 (0.0005) [2023-12-26 21:48:26,627][105620] Updated weights for policy 1, policy_version 885325 (0.0006) [2023-12-26 21:48:26,668][105692] Updated weights for policy 0, policy_version 885313 (0.0005) [2023-12-26 21:48:26,685][105620] Updated weights for policy 1, policy_version 885335 (0.0006) [2023-12-26 21:48:26,731][105692] Updated weights for policy 0, policy_version 885323 (0.0006) [2023-12-26 21:48:27,256][105620] Updated weights for policy 1, policy_version 885345 (0.0007) [2023-12-26 21:48:27,304][105692] Updated weights for policy 0, policy_version 885333 (0.0007) [2023-12-26 21:48:27,310][105620] Updated weights for policy 1, policy_version 885355 (0.0007) [2023-12-26 21:48:27,360][105620] Updated weights for policy 1, policy_version 885365 (0.0006) [2023-12-26 21:48:27,368][105692] Updated weights for policy 0, policy_version 885343 (0.0006) [2023-12-26 21:48:27,431][105692] Updated weights for policy 0, policy_version 885353 (0.0005) [2023-12-26 21:48:27,993][105692] Updated weights for policy 0, policy_version 885363 (0.0007) [2023-12-26 21:48:28,043][105692] Updated weights for policy 0, policy_version 885373 (0.0008) [2023-12-26 21:48:28,056][105620] Updated weights for policy 1, policy_version 885375 (0.0005) [2023-12-26 21:48:28,102][105692] Updated weights for policy 0, policy_version 885383 (0.0007) [2023-12-26 21:48:28,108][105620] Updated weights for policy 1, policy_version 885385 (0.0007) [2023-12-26 21:48:28,160][105620] Updated weights for policy 1, policy_version 885395 (0.0007) [2023-12-26 21:48:28,870][105692] Updated weights for policy 0, policy_version 885393 (0.0007) [2023-12-26 21:48:28,906][105620] Updated weights for policy 1, policy_version 885405 (0.0008) [2023-12-26 21:48:28,926][105692] Updated weights for policy 0, policy_version 885403 (0.0009) [2023-12-26 21:48:28,969][105620] Updated weights for policy 1, policy_version 885415 (0.0007) [2023-12-26 21:48:28,983][105692] Updated weights for policy 0, policy_version 885413 (0.0009) [2023-12-26 21:48:29,025][105620] Updated weights for policy 1, policy_version 885425 (0.0006) [2023-12-26 21:48:29,039][105692] Updated weights for policy 0, policy_version 885423 (0.0006) [2023-12-26 21:48:29,742][105620] Updated weights for policy 1, policy_version 885435 (0.0006) [2023-12-26 21:48:29,793][105692] Updated weights for policy 0, policy_version 885433 (0.0008) [2023-12-26 21:48:29,807][105620] Updated weights for policy 1, policy_version 885445 (0.0006) [2023-12-26 21:48:29,854][105692] Updated weights for policy 0, policy_version 885443 (0.0006) [2023-12-26 21:48:29,863][105620] Updated weights for policy 1, policy_version 885455 (0.0008) [2023-12-26 21:48:29,934][105692] Updated weights for policy 0, policy_version 885453 (0.0007) [2023-12-26 21:48:30,462][105620] Updated weights for policy 1, policy_version 885465 (0.0007) [2023-12-26 21:48:30,519][105620] Updated weights for policy 1, policy_version 885475 (0.0010) [2023-12-26 21:48:30,574][105620] Updated weights for policy 1, policy_version 885485 (0.0010) [2023-12-26 21:48:30,626][105620] Updated weights for policy 1, policy_version 885495 (0.0010) [2023-12-26 21:48:30,729][105692] Updated weights for policy 0, policy_version 885463 (0.0007) [2023-12-26 21:48:30,786][105692] Updated weights for policy 0, policy_version 885473 (0.0010) [2023-12-26 21:48:30,839][105692] Updated weights for policy 0, policy_version 885483 (0.0010) [2023-12-26 21:48:31,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 453435392. Throughput: 0: 9753.6, 1: 9903.1. Samples: 453403612. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-26 21:48:31,063][104569] Avg episode reward: [(0, '9079.218'), (1, '9083.821')] [2023-12-26 21:48:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000885488_226721792.pth... [2023-12-26 21:48:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000885496_226713600.pth... [2023-12-26 21:48:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000884344_226418688.pth [2023-12-26 21:48:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000884368_226435072.pth [2023-12-26 21:48:31,417][105620] Updated weights for policy 1, policy_version 885505 (0.0008) [2023-12-26 21:48:31,459][105692] Updated weights for policy 0, policy_version 885493 (0.0009) [2023-12-26 21:48:31,474][105620] Updated weights for policy 1, policy_version 885515 (0.0006) [2023-12-26 21:48:31,508][105692] Updated weights for policy 0, policy_version 885503 (0.0009) [2023-12-26 21:48:31,531][105620] Updated weights for policy 1, policy_version 885525 (0.0006) [2023-12-26 21:48:31,556][105692] Updated weights for policy 0, policy_version 885513 (0.0007) [2023-12-26 21:48:32,275][105620] Updated weights for policy 1, policy_version 885535 (0.0008) [2023-12-26 21:48:32,300][105692] Updated weights for policy 0, policy_version 885523 (0.0008) [2023-12-26 21:48:32,332][105620] Updated weights for policy 1, policy_version 885545 (0.0008) [2023-12-26 21:48:32,357][105692] Updated weights for policy 0, policy_version 885533 (0.0008) [2023-12-26 21:48:32,397][105620] Updated weights for policy 1, policy_version 885555 (0.0008) [2023-12-26 21:48:32,415][105692] Updated weights for policy 0, policy_version 885543 (0.0006) [2023-12-26 21:48:33,021][105692] Updated weights for policy 0, policy_version 885553 (0.0006) [2023-12-26 21:48:33,068][105692] Updated weights for policy 0, policy_version 885563 (0.0007) [2023-12-26 21:48:33,115][105692] Updated weights for policy 0, policy_version 885573 (0.0010) [2023-12-26 21:48:33,159][105692] Updated weights for policy 0, policy_version 885583 (0.0010) [2023-12-26 21:48:33,199][105620] Updated weights for policy 1, policy_version 885565 (0.0008) [2023-12-26 21:48:33,251][105620] Updated weights for policy 1, policy_version 885575 (0.0008) [2023-12-26 21:48:33,294][105620] Updated weights for policy 1, policy_version 885585 (0.0008) [2023-12-26 21:48:33,934][105692] Updated weights for policy 0, policy_version 885593 (0.0010) [2023-12-26 21:48:33,985][105692] Updated weights for policy 0, policy_version 885603 (0.0010) [2023-12-26 21:48:34,038][105692] Updated weights for policy 0, policy_version 885613 (0.0010) [2023-12-26 21:48:34,072][105620] Updated weights for policy 1, policy_version 885595 (0.0008) [2023-12-26 21:48:34,123][105620] Updated weights for policy 1, policy_version 885605 (0.0007) [2023-12-26 21:48:34,182][105620] Updated weights for policy 1, policy_version 885615 (0.0008) [2023-12-26 21:48:34,808][105692] Updated weights for policy 0, policy_version 885623 (0.0010) [2023-12-26 21:48:34,871][105692] Updated weights for policy 0, policy_version 885633 (0.0010) [2023-12-26 21:48:34,898][105585] KL-divergence is very high: 198.7701 [2023-12-26 21:48:34,936][105692] Updated weights for policy 0, policy_version 885643 (0.0010) [2023-12-26 21:48:34,951][105585] KL-divergence is very high: 301.8994 [2023-12-26 21:48:34,963][105620] Updated weights for policy 1, policy_version 885625 (0.0008) [2023-12-26 21:48:35,018][105620] Updated weights for policy 1, policy_version 885635 (0.0008) [2023-12-26 21:48:35,073][105620] Updated weights for policy 1, policy_version 885645 (0.0008) [2023-12-26 21:48:35,127][105620] Updated weights for policy 1, policy_version 885655 (0.0008) [2023-12-26 21:48:35,673][105692] Updated weights for policy 0, policy_version 885653 (0.0010) [2023-12-26 21:48:35,729][105692] Updated weights for policy 0, policy_version 885663 (0.0010) [2023-12-26 21:48:35,785][105692] Updated weights for policy 0, policy_version 885673 (0.0010) [2023-12-26 21:48:35,915][105620] Updated weights for policy 1, policy_version 885665 (0.0009) [2023-12-26 21:48:35,967][105620] Updated weights for policy 1, policy_version 885675 (0.0008) [2023-12-26 21:48:36,019][105620] Updated weights for policy 1, policy_version 885685 (0.0008) [2023-12-26 21:48:36,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 453533696. Throughput: 0: 9678.0, 1: 9834.0. Samples: 453519180. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-26 21:48:36,063][104569] Avg episode reward: [(0, '8729.248'), (1, '9083.658')] [2023-12-26 21:48:36,551][105692] Updated weights for policy 0, policy_version 885683 (0.0010) [2023-12-26 21:48:36,607][105692] Updated weights for policy 0, policy_version 885693 (0.0011) [2023-12-26 21:48:36,671][105692] Updated weights for policy 0, policy_version 885703 (0.0011) [2023-12-26 21:48:36,815][105620] Updated weights for policy 1, policy_version 885695 (0.0008) [2023-12-26 21:48:36,859][105620] Updated weights for policy 1, policy_version 885705 (0.0008) [2023-12-26 21:48:36,911][105620] Updated weights for policy 1, policy_version 885715 (0.0008) [2023-12-26 21:48:37,413][105692] Updated weights for policy 0, policy_version 885713 (0.0011) [2023-12-26 21:48:37,475][105692] Updated weights for policy 0, policy_version 885723 (0.0008) [2023-12-26 21:48:37,530][105692] Updated weights for policy 0, policy_version 885733 (0.0009) [2023-12-26 21:48:37,582][105692] Updated weights for policy 0, policy_version 885743 (0.0009) [2023-12-26 21:48:37,642][105620] Updated weights for policy 1, policy_version 885725 (0.0008) [2023-12-26 21:48:37,701][105620] Updated weights for policy 1, policy_version 885735 (0.0010) [2023-12-26 21:48:37,772][105620] Updated weights for policy 1, policy_version 885745 (0.0010) [2023-12-26 21:48:38,227][105692] Updated weights for policy 0, policy_version 885753 (0.0010) [2023-12-26 21:48:38,290][105692] Updated weights for policy 0, policy_version 885763 (0.0007) [2023-12-26 21:48:38,349][105692] Updated weights for policy 0, policy_version 885773 (0.0009) [2023-12-26 21:48:38,523][105620] Updated weights for policy 1, policy_version 885755 (0.0009) [2023-12-26 21:48:38,588][105620] Updated weights for policy 1, policy_version 885765 (0.0009) [2023-12-26 21:48:38,656][105620] Updated weights for policy 1, policy_version 885775 (0.0011) [2023-12-26 21:48:39,076][105692] Updated weights for policy 0, policy_version 885783 (0.0010) [2023-12-26 21:48:39,135][105692] Updated weights for policy 0, policy_version 885793 (0.0010) [2023-12-26 21:48:39,186][105692] Updated weights for policy 0, policy_version 885803 (0.0010) [2023-12-26 21:48:39,376][105620] Updated weights for policy 1, policy_version 885785 (0.0010) [2023-12-26 21:48:39,442][105620] Updated weights for policy 1, policy_version 885795 (0.0012) [2023-12-26 21:48:39,506][105620] Updated weights for policy 1, policy_version 885805 (0.0011) [2023-12-26 21:48:39,569][105620] Updated weights for policy 1, policy_version 885815 (0.0010) [2023-12-26 21:48:39,892][105692] Updated weights for policy 0, policy_version 885813 (0.0008) [2023-12-26 21:48:39,967][105692] Updated weights for policy 0, policy_version 885823 (0.0007) [2023-12-26 21:48:40,024][105692] Updated weights for policy 0, policy_version 885833 (0.0011) [2023-12-26 21:48:40,258][105620] Updated weights for policy 1, policy_version 885825 (0.0007) [2023-12-26 21:48:40,313][105620] Updated weights for policy 1, policy_version 885835 (0.0005) [2023-12-26 21:48:40,368][105620] Updated weights for policy 1, policy_version 885845 (0.0009) [2023-12-26 21:48:40,707][105692] Updated weights for policy 0, policy_version 885843 (0.0011) [2023-12-26 21:48:40,756][105692] Updated weights for policy 0, policy_version 885853 (0.0010) [2023-12-26 21:48:40,805][105692] Updated weights for policy 0, policy_version 885863 (0.0009) [2023-12-26 21:48:41,012][105620] Updated weights for policy 1, policy_version 885855 (0.0007) [2023-12-26 21:48:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 453623808. Throughput: 0: 9696.7, 1: 9752.3. Samples: 453634008. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-26 21:48:41,063][104569] Avg episode reward: [(0, '8743.153'), (1, '8903.558')] [2023-12-26 21:48:41,083][105620] Updated weights for policy 1, policy_version 885865 (0.0008) [2023-12-26 21:48:41,156][105620] Updated weights for policy 1, policy_version 885876 (0.0008) [2023-12-26 21:48:41,614][105692] Updated weights for policy 0, policy_version 885873 (0.0010) [2023-12-26 21:48:41,682][105692] Updated weights for policy 0, policy_version 885883 (0.0008) [2023-12-26 21:48:41,752][105692] Updated weights for policy 0, policy_version 885893 (0.0007) [2023-12-26 21:48:41,809][105692] Updated weights for policy 0, policy_version 885903 (0.0006) [2023-12-26 21:48:41,878][105620] Updated weights for policy 1, policy_version 885886 (0.0007) [2023-12-26 21:48:41,948][105620] Updated weights for policy 1, policy_version 885896 (0.0008) [2023-12-26 21:48:42,004][105620] Updated weights for policy 1, policy_version 885906 (0.0009) [2023-12-26 21:48:42,459][105692] Updated weights for policy 0, policy_version 885913 (0.0009) [2023-12-26 21:48:42,517][105692] Updated weights for policy 0, policy_version 885923 (0.0008) [2023-12-26 21:48:42,579][105692] Updated weights for policy 0, policy_version 885933 (0.0009) [2023-12-26 21:48:42,771][105620] Updated weights for policy 1, policy_version 885916 (0.0009) [2023-12-26 21:48:42,822][105620] Updated weights for policy 1, policy_version 885926 (0.0008) [2023-12-26 21:48:42,876][105620] Updated weights for policy 1, policy_version 885936 (0.0005) [2023-12-26 21:48:43,386][105692] Updated weights for policy 0, policy_version 885943 (0.0009) [2023-12-26 21:48:43,422][105620] Updated weights for policy 1, policy_version 885946 (0.0005) [2023-12-26 21:48:43,437][105692] Updated weights for policy 0, policy_version 885953 (0.0010) [2023-12-26 21:48:43,487][105620] Updated weights for policy 1, policy_version 885956 (0.0008) [2023-12-26 21:48:43,492][105692] Updated weights for policy 0, policy_version 885963 (0.0008) [2023-12-26 21:48:43,541][105620] Updated weights for policy 1, policy_version 885966 (0.0009) [2023-12-26 21:48:43,592][105620] Updated weights for policy 1, policy_version 885976 (0.0009) [2023-12-26 21:48:44,268][105692] Updated weights for policy 0, policy_version 885973 (0.0008) [2023-12-26 21:48:44,310][105620] Updated weights for policy 1, policy_version 885986 (0.0006) [2023-12-26 21:48:44,319][105692] Updated weights for policy 0, policy_version 885983 (0.0009) [2023-12-26 21:48:44,364][105620] Updated weights for policy 1, policy_version 885996 (0.0005) [2023-12-26 21:48:44,370][105586] KL-divergence is very high: 101.5064 [2023-12-26 21:48:44,379][105692] Updated weights for policy 0, policy_version 885993 (0.0009) [2023-12-26 21:48:44,421][105620] Updated weights for policy 1, policy_version 886006 (0.0006) [2023-12-26 21:48:45,045][105620] Updated weights for policy 1, policy_version 886016 (0.0009) [2023-12-26 21:48:45,082][105692] Updated weights for policy 0, policy_version 886003 (0.0007) [2023-12-26 21:48:45,096][105620] Updated weights for policy 1, policy_version 886026 (0.0008) [2023-12-26 21:48:45,132][105692] Updated weights for policy 0, policy_version 886013 (0.0006) [2023-12-26 21:48:45,146][105620] Updated weights for policy 1, policy_version 886036 (0.0007) [2023-12-26 21:48:45,189][105692] Updated weights for policy 0, policy_version 886023 (0.0007) [2023-12-26 21:48:45,859][105620] Updated weights for policy 1, policy_version 886046 (0.0008) [2023-12-26 21:48:45,905][105620] Updated weights for policy 1, policy_version 886056 (0.0009) [2023-12-26 21:48:45,959][105620] Updated weights for policy 1, policy_version 886066 (0.0008) [2023-12-26 21:48:45,975][105692] Updated weights for policy 0, policy_version 886033 (0.0008) [2023-12-26 21:48:46,032][105692] Updated weights for policy 0, policy_version 886043 (0.0008) [2023-12-26 21:48:46,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 453722112. Throughput: 0: 9641.3, 1: 9810.1. Samples: 453691528. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-26 21:48:46,062][104569] Avg episode reward: [(0, '8390.527'), (1, '8819.147')] [2023-12-26 21:48:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000886072_226861056.pth... [2023-12-26 21:48:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000884920_226566144.pth [2023-12-26 21:48:46,087][105692] Updated weights for policy 0, policy_version 886053 (0.0006) [2023-12-26 21:48:46,143][105692] Updated weights for policy 0, policy_version 886063 (0.0005) [2023-12-26 21:48:46,150][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000886064_226869248.pth... [2023-12-26 21:48:46,154][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000884944_226582528.pth [2023-12-26 21:48:46,690][105620] Updated weights for policy 1, policy_version 886076 (0.0006) [2023-12-26 21:48:46,747][105620] Updated weights for policy 1, policy_version 886086 (0.0006) [2023-12-26 21:48:46,793][105620] Updated weights for policy 1, policy_version 886096 (0.0009) [2023-12-26 21:48:46,895][105692] Updated weights for policy 0, policy_version 886073 (0.0009) [2023-12-26 21:48:46,942][105692] Updated weights for policy 0, policy_version 886083 (0.0009) [2023-12-26 21:48:46,988][105692] Updated weights for policy 0, policy_version 886093 (0.0008) [2023-12-26 21:48:47,400][105620] Updated weights for policy 1, policy_version 886106 (0.0008) [2023-12-26 21:48:47,465][105620] Updated weights for policy 1, policy_version 886116 (0.0005) [2023-12-26 21:48:47,518][105620] Updated weights for policy 1, policy_version 886126 (0.0007) [2023-12-26 21:48:47,572][105620] Updated weights for policy 1, policy_version 886136 (0.0009) [2023-12-26 21:48:47,870][105692] Updated weights for policy 0, policy_version 886103 (0.0009) [2023-12-26 21:48:47,925][105692] Updated weights for policy 0, policy_version 886113 (0.0009) [2023-12-26 21:48:47,978][105692] Updated weights for policy 0, policy_version 886124 (0.0009) [2023-12-26 21:48:48,231][105620] Updated weights for policy 1, policy_version 886146 (0.0007) [2023-12-26 21:48:48,284][105620] Updated weights for policy 1, policy_version 886156 (0.0006) [2023-12-26 21:48:48,342][105620] Updated weights for policy 1, policy_version 886166 (0.0008) [2023-12-26 21:48:48,667][105692] Updated weights for policy 0, policy_version 886134 (0.0010) [2023-12-26 21:48:48,732][105692] Updated weights for policy 0, policy_version 886144 (0.0011) [2023-12-26 21:48:48,801][105692] Updated weights for policy 0, policy_version 886154 (0.0010) [2023-12-26 21:48:49,003][105620] Updated weights for policy 1, policy_version 886176 (0.0006) [2023-12-26 21:48:49,060][105620] Updated weights for policy 1, policy_version 886186 (0.0006) [2023-12-26 21:48:49,120][105620] Updated weights for policy 1, policy_version 886196 (0.0005) [2023-12-26 21:48:49,504][105692] Updated weights for policy 0, policy_version 886164 (0.0009) [2023-12-26 21:48:49,565][105692] Updated weights for policy 0, policy_version 886174 (0.0006) [2023-12-26 21:48:49,617][105692] Updated weights for policy 0, policy_version 886184 (0.0010) [2023-12-26 21:48:49,773][105620] Updated weights for policy 1, policy_version 886206 (0.0008) [2023-12-26 21:48:49,836][105620] Updated weights for policy 1, policy_version 886216 (0.0010) [2023-12-26 21:48:49,898][105620] Updated weights for policy 1, policy_version 886226 (0.0010) [2023-12-26 21:48:50,360][105692] Updated weights for policy 0, policy_version 886194 (0.0009) [2023-12-26 21:48:50,419][105692] Updated weights for policy 0, policy_version 886204 (0.0009) [2023-12-26 21:48:50,487][105692] Updated weights for policy 0, policy_version 886214 (0.0008) [2023-12-26 21:48:50,545][105692] Updated weights for policy 0, policy_version 886224 (0.0009) [2023-12-26 21:48:50,591][105620] Updated weights for policy 1, policy_version 886236 (0.0009) [2023-12-26 21:48:50,648][105620] Updated weights for policy 1, policy_version 886246 (0.0009) [2023-12-26 21:48:50,714][105620] Updated weights for policy 1, policy_version 886256 (0.0009) [2023-12-26 21:48:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 453820416. Throughput: 0: 9593.4, 1: 9811.0. Samples: 453809900. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-26 21:48:51,063][104569] Avg episode reward: [(0, '8129.180'), (1, '8815.452')] [2023-12-26 21:48:51,282][105692] Updated weights for policy 0, policy_version 886234 (0.0008) [2023-12-26 21:48:51,336][105692] Updated weights for policy 0, policy_version 886244 (0.0008) [2023-12-26 21:48:51,393][105692] Updated weights for policy 0, policy_version 886254 (0.0008) [2023-12-26 21:48:51,500][105620] Updated weights for policy 1, policy_version 886266 (0.0009) [2023-12-26 21:48:51,552][105620] Updated weights for policy 1, policy_version 886276 (0.0008) [2023-12-26 21:48:51,619][105620] Updated weights for policy 1, policy_version 886286 (0.0008) [2023-12-26 21:48:51,678][105620] Updated weights for policy 1, policy_version 886296 (0.0008) [2023-12-26 21:48:52,129][105692] Updated weights for policy 0, policy_version 886264 (0.0008) [2023-12-26 21:48:52,188][105692] Updated weights for policy 0, policy_version 886274 (0.0009) [2023-12-26 21:48:52,251][105692] Updated weights for policy 0, policy_version 886284 (0.0009) [2023-12-26 21:48:52,447][105620] Updated weights for policy 1, policy_version 886306 (0.0009) [2023-12-26 21:48:52,502][105620] Updated weights for policy 1, policy_version 886316 (0.0009) [2023-12-26 21:48:52,552][105620] Updated weights for policy 1, policy_version 886326 (0.0008) [2023-12-26 21:48:52,983][105692] Updated weights for policy 0, policy_version 886294 (0.0007) [2023-12-26 21:48:53,032][105692] Updated weights for policy 0, policy_version 886304 (0.0005) [2023-12-26 21:48:53,085][105692] Updated weights for policy 0, policy_version 886314 (0.0006) [2023-12-26 21:48:53,357][105620] Updated weights for policy 1, policy_version 886336 (0.0010) [2023-12-26 21:48:53,412][105620] Updated weights for policy 1, policy_version 886346 (0.0009) [2023-12-26 21:48:53,455][105620] Updated weights for policy 1, policy_version 886356 (0.0005) [2023-12-26 21:48:53,671][105692] Updated weights for policy 0, policy_version 886324 (0.0008) [2023-12-26 21:48:53,725][105692] Updated weights for policy 0, policy_version 886334 (0.0005) [2023-12-26 21:48:53,774][105692] Updated weights for policy 0, policy_version 886344 (0.0007) [2023-12-26 21:48:54,103][105620] Updated weights for policy 1, policy_version 886366 (0.0008) [2023-12-26 21:48:54,158][105620] Updated weights for policy 1, policy_version 886376 (0.0006) [2023-12-26 21:48:54,224][105620] Updated weights for policy 1, policy_version 886386 (0.0005) [2023-12-26 21:48:54,458][105692] Updated weights for policy 0, policy_version 886354 (0.0010) [2023-12-26 21:48:54,516][105692] Updated weights for policy 0, policy_version 886364 (0.0008) [2023-12-26 21:48:54,576][105692] Updated weights for policy 0, policy_version 886374 (0.0009) [2023-12-26 21:48:54,627][105692] Updated weights for policy 0, policy_version 886384 (0.0008) [2023-12-26 21:48:54,928][105620] Updated weights for policy 1, policy_version 886396 (0.0010) [2023-12-26 21:48:54,985][105620] Updated weights for policy 1, policy_version 886406 (0.0011) [2023-12-26 21:48:55,034][105620] Updated weights for policy 1, policy_version 886416 (0.0011) [2023-12-26 21:48:55,404][105692] Updated weights for policy 0, policy_version 886394 (0.0010) [2023-12-26 21:48:55,463][105692] Updated weights for policy 0, policy_version 886404 (0.0007) [2023-12-26 21:48:55,516][105692] Updated weights for policy 0, policy_version 886414 (0.0005) [2023-12-26 21:48:55,761][105620] Updated weights for policy 1, policy_version 886426 (0.0011) [2023-12-26 21:48:55,830][105620] Updated weights for policy 1, policy_version 886436 (0.0006) [2023-12-26 21:48:55,895][105620] Updated weights for policy 1, policy_version 886446 (0.0005) [2023-12-26 21:48:55,953][105620] Updated weights for policy 1, policy_version 886456 (0.0010) [2023-12-26 21:48:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 453918720. Throughput: 0: 9618.9, 1: 9790.4. Samples: 453926676. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-26 21:48:56,062][104569] Avg episode reward: [(0, '8849.520'), (1, '8904.371')] [2023-12-26 21:48:56,109][105692] Updated weights for policy 0, policy_version 886424 (0.0009) [2023-12-26 21:48:56,160][105692] Updated weights for policy 0, policy_version 886434 (0.0010) [2023-12-26 21:48:56,214][105692] Updated weights for policy 0, policy_version 886444 (0.0010) [2023-12-26 21:48:56,642][105620] Updated weights for policy 1, policy_version 886466 (0.0011) [2023-12-26 21:48:56,700][105620] Updated weights for policy 1, policy_version 886476 (0.0010) [2023-12-26 21:48:56,760][105620] Updated weights for policy 1, policy_version 886486 (0.0009) [2023-12-26 21:48:56,851][105692] Updated weights for policy 0, policy_version 886454 (0.0008) [2023-12-26 21:48:56,905][105692] Updated weights for policy 0, policy_version 886464 (0.0005) [2023-12-26 21:48:56,959][105692] Updated weights for policy 0, policy_version 886474 (0.0007) [2023-12-26 21:48:57,495][105620] Updated weights for policy 1, policy_version 886496 (0.0010) [2023-12-26 21:48:57,539][105620] Updated weights for policy 1, policy_version 886506 (0.0010) [2023-12-26 21:48:57,544][105586] KL-divergence is very high: 114.3753 [2023-12-26 21:48:57,580][105586] KL-divergence is very high: 104.9895 [2023-12-26 21:48:57,582][105692] Updated weights for policy 0, policy_version 886484 (0.0006) [2023-12-26 21:48:57,587][105620] Updated weights for policy 1, policy_version 886516 (0.0010) [2023-12-26 21:48:57,627][105692] Updated weights for policy 0, policy_version 886494 (0.0005) [2023-12-26 21:48:57,671][105692] Updated weights for policy 0, policy_version 886504 (0.0005) [2023-12-26 21:48:58,268][105620] Updated weights for policy 1, policy_version 886526 (0.0009) [2023-12-26 21:48:58,327][105620] Updated weights for policy 1, policy_version 886536 (0.0007) [2023-12-26 21:48:58,401][105620] Updated weights for policy 1, policy_version 886546 (0.0008) [2023-12-26 21:48:58,412][105692] Updated weights for policy 0, policy_version 886514 (0.0006) [2023-12-26 21:48:58,475][105692] Updated weights for policy 0, policy_version 886524 (0.0008) [2023-12-26 21:48:58,532][105692] Updated weights for policy 0, policy_version 886534 (0.0007) [2023-12-26 21:48:59,291][105620] Updated weights for policy 1, policy_version 886556 (0.0008) [2023-12-26 21:48:59,353][105620] Updated weights for policy 1, policy_version 886566 (0.0008) [2023-12-26 21:48:59,364][105692] Updated weights for policy 0, policy_version 886545 (0.0008) [2023-12-26 21:48:59,418][105620] Updated weights for policy 1, policy_version 886576 (0.0007) [2023-12-26 21:48:59,424][105692] Updated weights for policy 0, policy_version 886555 (0.0007) [2023-12-26 21:48:59,481][105692] Updated weights for policy 0, policy_version 886565 (0.0008) [2023-12-26 21:48:59,537][105692] Updated weights for policy 0, policy_version 886575 (0.0009) [2023-12-26 21:49:00,118][105620] Updated weights for policy 1, policy_version 886586 (0.0006) [2023-12-26 21:49:00,175][105620] Updated weights for policy 1, policy_version 886596 (0.0009) [2023-12-26 21:49:00,223][105620] Updated weights for policy 1, policy_version 886606 (0.0010) [2023-12-26 21:49:00,274][105620] Updated weights for policy 1, policy_version 886616 (0.0008) [2023-12-26 21:49:00,302][105692] Updated weights for policy 0, policy_version 886585 (0.0008) [2023-12-26 21:49:00,363][105692] Updated weights for policy 0, policy_version 886595 (0.0010) [2023-12-26 21:49:00,414][105692] Updated weights for policy 0, policy_version 886605 (0.0010) [2023-12-26 21:49:00,957][105620] Updated weights for policy 1, policy_version 886626 (0.0010) [2023-12-26 21:49:00,978][105692] Updated weights for policy 0, policy_version 886615 (0.0010) [2023-12-26 21:49:01,008][105620] Updated weights for policy 1, policy_version 886636 (0.0010) [2023-12-26 21:49:01,034][105692] Updated weights for policy 0, policy_version 886625 (0.0008) [2023-12-26 21:49:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 454008832. Throughput: 0: 9708.8, 1: 9758.1. Samples: 453986324. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-26 21:49:01,062][104569] Avg episode reward: [(0, '9099.056'), (1, '9090.723')] [2023-12-26 21:49:01,064][105620] Updated weights for policy 1, policy_version 886646 (0.0010) [2023-12-26 21:49:01,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000886648_227008512.pth... [2023-12-26 21:49:01,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000885496_226713600.pth [2023-12-26 21:49:01,098][105692] Updated weights for policy 0, policy_version 886635 (0.0008) [2023-12-26 21:49:01,130][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000886640_227016704.pth... [2023-12-26 21:49:01,135][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000885488_226721792.pth [2023-12-26 21:49:01,722][105620] Updated weights for policy 1, policy_version 886656 (0.0009) [2023-12-26 21:49:01,779][105620] Updated weights for policy 1, policy_version 886666 (0.0009) [2023-12-26 21:49:01,838][105620] Updated weights for policy 1, policy_version 886676 (0.0008) [2023-12-26 21:49:01,848][105692] Updated weights for policy 0, policy_version 886645 (0.0007) [2023-12-26 21:49:01,904][105692] Updated weights for policy 0, policy_version 886655 (0.0008) [2023-12-26 21:49:01,959][105692] Updated weights for policy 0, policy_version 886665 (0.0009) [2023-12-26 21:49:02,530][105620] Updated weights for policy 1, policy_version 886686 (0.0009) [2023-12-26 21:49:02,593][105620] Updated weights for policy 1, policy_version 886696 (0.0008) [2023-12-26 21:49:02,649][105620] Updated weights for policy 1, policy_version 886706 (0.0009) [2023-12-26 21:49:02,751][105692] Updated weights for policy 0, policy_version 886675 (0.0008) [2023-12-26 21:49:02,796][105692] Updated weights for policy 0, policy_version 886685 (0.0008) [2023-12-26 21:49:02,839][105692] Updated weights for policy 0, policy_version 886695 (0.0008) [2023-12-26 21:49:03,409][105620] Updated weights for policy 1, policy_version 886716 (0.0010) [2023-12-26 21:49:03,416][105692] Updated weights for policy 0, policy_version 886705 (0.0008) [2023-12-26 21:49:03,470][105692] Updated weights for policy 0, policy_version 886715 (0.0005) [2023-12-26 21:49:03,472][105620] Updated weights for policy 1, policy_version 886726 (0.0010) [2023-12-26 21:49:03,519][105692] Updated weights for policy 0, policy_version 886725 (0.0005) [2023-12-26 21:49:03,534][105620] Updated weights for policy 1, policy_version 886736 (0.0010) [2023-12-26 21:49:03,570][105692] Updated weights for policy 0, policy_version 886735 (0.0007) [2023-12-26 21:49:04,123][105692] Updated weights for policy 0, policy_version 886745 (0.0006) [2023-12-26 21:49:04,189][105692] Updated weights for policy 0, policy_version 886755 (0.0009) [2023-12-26 21:49:04,256][105692] Updated weights for policy 0, policy_version 886765 (0.0011) [2023-12-26 21:49:04,256][105620] Updated weights for policy 1, policy_version 886746 (0.0009) [2023-12-26 21:49:04,316][105620] Updated weights for policy 1, policy_version 886756 (0.0007) [2023-12-26 21:49:04,379][105620] Updated weights for policy 1, policy_version 886766 (0.0011) [2023-12-26 21:49:04,444][105620] Updated weights for policy 1, policy_version 886776 (0.0011) [2023-12-26 21:49:04,892][105692] Updated weights for policy 0, policy_version 886775 (0.0007) [2023-12-26 21:49:04,953][105692] Updated weights for policy 0, policy_version 886785 (0.0005) [2023-12-26 21:49:05,012][105692] Updated weights for policy 0, policy_version 886795 (0.0005) [2023-12-26 21:49:05,156][105620] Updated weights for policy 1, policy_version 886786 (0.0010) [2023-12-26 21:49:05,217][105620] Updated weights for policy 1, policy_version 886796 (0.0010) [2023-12-26 21:49:05,275][105620] Updated weights for policy 1, policy_version 886806 (0.0010) [2023-12-26 21:49:05,602][105692] Updated weights for policy 0, policy_version 886805 (0.0006) [2023-12-26 21:49:05,656][105692] Updated weights for policy 0, policy_version 886815 (0.0005) [2023-12-26 21:49:05,705][105692] Updated weights for policy 0, policy_version 886825 (0.0005) [2023-12-26 21:49:06,016][105620] Updated weights for policy 1, policy_version 886816 (0.0006) [2023-12-26 21:49:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 454115328. Throughput: 0: 9710.3, 1: 9791.4. Samples: 454105112. Policy #0 lag: (min: 16.0, avg: 43.7, max: 48.0) [2023-12-26 21:49:06,062][104569] Avg episode reward: [(0, '8655.979'), (1, '8809.222')] [2023-12-26 21:49:06,072][105620] Updated weights for policy 1, policy_version 886826 (0.0005) [2023-12-26 21:49:06,134][105620] Updated weights for policy 1, policy_version 886836 (0.0007) [2023-12-26 21:49:06,317][105692] Updated weights for policy 0, policy_version 886835 (0.0007) [2023-12-26 21:49:06,383][105692] Updated weights for policy 0, policy_version 886845 (0.0007) [2023-12-26 21:49:06,450][105692] Updated weights for policy 0, policy_version 886855 (0.0008) [2023-12-26 21:49:06,857][105620] Updated weights for policy 1, policy_version 886846 (0.0010) [2023-12-26 21:49:06,928][105620] Updated weights for policy 1, policy_version 886856 (0.0011) [2023-12-26 21:49:06,992][105620] Updated weights for policy 1, policy_version 886866 (0.0011) [2023-12-26 21:49:07,068][105692] Updated weights for policy 0, policy_version 886865 (0.0010) [2023-12-26 21:49:07,120][105692] Updated weights for policy 0, policy_version 886875 (0.0005) [2023-12-26 21:49:07,179][105692] Updated weights for policy 0, policy_version 886885 (0.0005) [2023-12-26 21:49:07,249][105692] Updated weights for policy 0, policy_version 886895 (0.0005) [2023-12-26 21:49:07,658][105620] Updated weights for policy 1, policy_version 886876 (0.0010) [2023-12-26 21:49:07,722][105620] Updated weights for policy 1, policy_version 886886 (0.0010) [2023-12-26 21:49:07,771][105620] Updated weights for policy 1, policy_version 886896 (0.0008) [2023-12-26 21:49:07,888][105692] Updated weights for policy 0, policy_version 886905 (0.0010) [2023-12-26 21:49:07,951][105692] Updated weights for policy 0, policy_version 886915 (0.0011) [2023-12-26 21:49:08,014][105692] Updated weights for policy 0, policy_version 886925 (0.0009) [2023-12-26 21:49:08,595][105620] Updated weights for policy 1, policy_version 886906 (0.0008) [2023-12-26 21:49:08,645][105620] Updated weights for policy 1, policy_version 886916 (0.0008) [2023-12-26 21:49:08,703][105620] Updated weights for policy 1, policy_version 886926 (0.0006) [2023-12-26 21:49:08,705][105692] Updated weights for policy 0, policy_version 886935 (0.0007) [2023-12-26 21:49:08,762][105620] Updated weights for policy 1, policy_version 886936 (0.0006) [2023-12-26 21:49:08,764][105692] Updated weights for policy 0, policy_version 886945 (0.0009) [2023-12-26 21:49:08,830][105692] Updated weights for policy 0, policy_version 886955 (0.0009) [2023-12-26 21:49:09,471][105620] Updated weights for policy 1, policy_version 886946 (0.0007) [2023-12-26 21:49:09,534][105620] Updated weights for policy 1, policy_version 886956 (0.0009) [2023-12-26 21:49:09,570][105692] Updated weights for policy 0, policy_version 886965 (0.0009) [2023-12-26 21:49:09,596][105620] Updated weights for policy 1, policy_version 886966 (0.0008) [2023-12-26 21:49:09,623][105692] Updated weights for policy 0, policy_version 886975 (0.0008) [2023-12-26 21:49:09,690][105692] Updated weights for policy 0, policy_version 886985 (0.0007) [2023-12-26 21:49:10,294][105620] Updated weights for policy 1, policy_version 886976 (0.0007) [2023-12-26 21:49:10,337][105692] Updated weights for policy 0, policy_version 886995 (0.0006) [2023-12-26 21:49:10,358][105620] Updated weights for policy 1, policy_version 886986 (0.0007) [2023-12-26 21:49:10,392][105692] Updated weights for policy 0, policy_version 887005 (0.0008) [2023-12-26 21:49:10,414][105620] Updated weights for policy 1, policy_version 886996 (0.0006) [2023-12-26 21:49:10,440][105692] Updated weights for policy 0, policy_version 887015 (0.0007) [2023-12-26 21:49:11,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 454213632. Throughput: 0: 9893.2, 1: 9750.3. Samples: 454225268. Policy #0 lag: (min: 16.0, avg: 43.7, max: 48.0) [2023-12-26 21:49:11,063][104569] Avg episode reward: [(0, '8571.226'), (1, '8627.188')] [2023-12-26 21:49:11,099][105692] Updated weights for policy 0, policy_version 887025 (0.0009) [2023-12-26 21:49:11,130][105620] Updated weights for policy 1, policy_version 887006 (0.0008) [2023-12-26 21:49:11,175][105692] Updated weights for policy 0, policy_version 887035 (0.0007) [2023-12-26 21:49:11,189][105620] Updated weights for policy 1, policy_version 887016 (0.0007) [2023-12-26 21:49:11,237][105692] Updated weights for policy 0, policy_version 887045 (0.0007) [2023-12-26 21:49:11,248][105620] Updated weights for policy 1, policy_version 887026 (0.0006) [2023-12-26 21:49:11,302][105692] Updated weights for policy 0, policy_version 887055 (0.0008) [2023-12-26 21:49:12,003][105620] Updated weights for policy 1, policy_version 887036 (0.0007) [2023-12-26 21:49:12,062][105620] Updated weights for policy 1, policy_version 887046 (0.0006) [2023-12-26 21:49:12,090][105692] Updated weights for policy 0, policy_version 887065 (0.0008) [2023-12-26 21:49:12,113][105620] Updated weights for policy 1, policy_version 887056 (0.0005) [2023-12-26 21:49:12,145][105692] Updated weights for policy 0, policy_version 887075 (0.0007) [2023-12-26 21:49:12,213][105692] Updated weights for policy 0, policy_version 887085 (0.0005) [2023-12-26 21:49:12,849][105620] Updated weights for policy 1, policy_version 887066 (0.0008) [2023-12-26 21:49:12,915][105620] Updated weights for policy 1, policy_version 887076 (0.0005) [2023-12-26 21:49:12,915][105692] Updated weights for policy 0, policy_version 887095 (0.0010) [2023-12-26 21:49:12,928][105586] KL-divergence is very high: 187.9168 [2023-12-26 21:49:12,969][105620] Updated weights for policy 1, policy_version 887086 (0.0005) [2023-12-26 21:49:12,970][105692] Updated weights for policy 0, policy_version 887105 (0.0007) [2023-12-26 21:49:12,971][105586] KL-divergence is very high: 302.3857 [2023-12-26 21:49:13,010][105586] KL-divergence is very high: 270.6655 [2023-12-26 21:49:13,019][105692] Updated weights for policy 0, policy_version 887115 (0.0005) [2023-12-26 21:49:13,021][105620] Updated weights for policy 1, policy_version 887096 (0.0005) [2023-12-26 21:49:13,621][105692] Updated weights for policy 0, policy_version 887125 (0.0005) [2023-12-26 21:49:13,677][105692] Updated weights for policy 0, policy_version 887135 (0.0005) [2023-12-26 21:49:13,730][105692] Updated weights for policy 0, policy_version 887145 (0.0008) [2023-12-26 21:49:13,761][105620] Updated weights for policy 1, policy_version 887106 (0.0006) [2023-12-26 21:49:13,811][105620] Updated weights for policy 1, policy_version 887116 (0.0007) [2023-12-26 21:49:13,859][105620] Updated weights for policy 1, policy_version 887126 (0.0008) [2023-12-26 21:49:14,387][105692] Updated weights for policy 0, policy_version 887155 (0.0010) [2023-12-26 21:49:14,445][105692] Updated weights for policy 0, policy_version 887165 (0.0010) [2023-12-26 21:49:14,507][105692] Updated weights for policy 0, policy_version 887175 (0.0010) [2023-12-26 21:49:14,654][105620] Updated weights for policy 1, policy_version 887136 (0.0009) [2023-12-26 21:49:14,701][105620] Updated weights for policy 1, policy_version 887147 (0.0008) [2023-12-26 21:49:14,764][105620] Updated weights for policy 1, policy_version 887157 (0.0009) [2023-12-26 21:49:15,283][105692] Updated weights for policy 0, policy_version 887185 (0.0008) [2023-12-26 21:49:15,341][105692] Updated weights for policy 0, policy_version 887195 (0.0009) [2023-12-26 21:49:15,389][105692] Updated weights for policy 0, policy_version 887205 (0.0009) [2023-12-26 21:49:15,436][105692] Updated weights for policy 0, policy_version 887215 (0.0009) [2023-12-26 21:49:15,533][105620] Updated weights for policy 1, policy_version 887167 (0.0006) [2023-12-26 21:49:15,582][105620] Updated weights for policy 1, policy_version 887177 (0.0007) [2023-12-26 21:49:15,635][105620] Updated weights for policy 1, policy_version 887187 (0.0009) [2023-12-26 21:49:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 454311936. Throughput: 0: 9878.9, 1: 9669.9. Samples: 454283304. Policy #0 lag: (min: 16.0, avg: 43.7, max: 48.0) [2023-12-26 21:49:16,063][104569] Avg episode reward: [(0, '9084.736'), (1, '8630.576')] [2023-12-26 21:49:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000887216_227164160.pth... [2023-12-26 21:49:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000887192_227147776.pth... [2023-12-26 21:49:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000886072_226861056.pth [2023-12-26 21:49:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000886064_226869248.pth [2023-12-26 21:49:16,192][105692] Updated weights for policy 0, policy_version 887225 (0.0009) [2023-12-26 21:49:16,246][105692] Updated weights for policy 0, policy_version 887235 (0.0009) [2023-12-26 21:49:16,305][105692] Updated weights for policy 0, policy_version 887245 (0.0009) [2023-12-26 21:49:16,378][105620] Updated weights for policy 1, policy_version 887197 (0.0010) [2023-12-26 21:49:16,425][105620] Updated weights for policy 1, policy_version 887207 (0.0009) [2023-12-26 21:49:16,483][105620] Updated weights for policy 1, policy_version 887217 (0.0009) [2023-12-26 21:49:16,957][105692] Updated weights for policy 0, policy_version 887255 (0.0009) [2023-12-26 21:49:17,016][105692] Updated weights for policy 0, policy_version 887265 (0.0009) [2023-12-26 21:49:17,074][105692] Updated weights for policy 0, policy_version 887275 (0.0009) [2023-12-26 21:49:17,251][105620] Updated weights for policy 1, policy_version 887227 (0.0009) [2023-12-26 21:49:17,298][105620] Updated weights for policy 1, policy_version 887237 (0.0009) [2023-12-26 21:49:17,344][105620] Updated weights for policy 1, policy_version 887247 (0.0008) [2023-12-26 21:49:17,809][105692] Updated weights for policy 0, policy_version 887285 (0.0008) [2023-12-26 21:49:17,853][105692] Updated weights for policy 0, policy_version 887295 (0.0008) [2023-12-26 21:49:17,905][105692] Updated weights for policy 0, policy_version 887305 (0.0008) [2023-12-26 21:49:18,121][105620] Updated weights for policy 1, policy_version 887257 (0.0008) [2023-12-26 21:49:18,174][105620] Updated weights for policy 1, policy_version 887267 (0.0011) [2023-12-26 21:49:18,222][105620] Updated weights for policy 1, policy_version 887277 (0.0010) [2023-12-26 21:49:18,274][105620] Updated weights for policy 1, policy_version 887287 (0.0010) [2023-12-26 21:49:18,698][105692] Updated weights for policy 0, policy_version 887315 (0.0008) [2023-12-26 21:49:18,754][105692] Updated weights for policy 0, policy_version 887325 (0.0008) [2023-12-26 21:49:18,814][105692] Updated weights for policy 0, policy_version 887335 (0.0008) [2023-12-26 21:49:19,067][105620] Updated weights for policy 1, policy_version 887297 (0.0011) [2023-12-26 21:49:19,122][105620] Updated weights for policy 1, policy_version 887307 (0.0011) [2023-12-26 21:49:19,184][105620] Updated weights for policy 1, policy_version 887317 (0.0010) [2023-12-26 21:49:19,591][105692] Updated weights for policy 0, policy_version 887345 (0.0008) [2023-12-26 21:49:19,647][105692] Updated weights for policy 0, policy_version 887355 (0.0008) [2023-12-26 21:49:19,703][105692] Updated weights for policy 0, policy_version 887365 (0.0008) [2023-12-26 21:49:19,760][105692] Updated weights for policy 0, policy_version 887375 (0.0009) [2023-12-26 21:49:19,961][105620] Updated weights for policy 1, policy_version 887327 (0.0011) [2023-12-26 21:49:20,029][105620] Updated weights for policy 1, policy_version 887337 (0.0011) [2023-12-26 21:49:20,120][105620] Updated weights for policy 1, policy_version 887347 (0.0011) [2023-12-26 21:49:20,552][105692] Updated weights for policy 0, policy_version 887385 (0.0008) [2023-12-26 21:49:20,616][105692] Updated weights for policy 0, policy_version 887395 (0.0008) [2023-12-26 21:49:20,680][105692] Updated weights for policy 0, policy_version 887405 (0.0009) [2023-12-26 21:49:20,868][105620] Updated weights for policy 1, policy_version 887357 (0.0011) [2023-12-26 21:49:20,936][105620] Updated weights for policy 1, policy_version 887367 (0.0011) [2023-12-26 21:49:21,006][105620] Updated weights for policy 1, policy_version 887377 (0.0011) [2023-12-26 21:49:21,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 454410240. Throughput: 0: 9858.1, 1: 9640.7. Samples: 454396620. Policy #0 lag: (min: 16.0, avg: 43.7, max: 48.0) [2023-12-26 21:49:21,062][104569] Avg episode reward: [(0, '9171.502'), (1, '8995.760')] [2023-12-26 21:49:21,478][105692] Updated weights for policy 0, policy_version 887415 (0.0009) [2023-12-26 21:49:21,535][105692] Updated weights for policy 0, policy_version 887425 (0.0009) [2023-12-26 21:49:21,585][105692] Updated weights for policy 0, policy_version 887435 (0.0009) [2023-12-26 21:49:21,828][105620] Updated weights for policy 1, policy_version 887387 (0.0011) [2023-12-26 21:49:21,879][105620] Updated weights for policy 1, policy_version 887397 (0.0009) [2023-12-26 21:49:21,927][105620] Updated weights for policy 1, policy_version 887407 (0.0010) [2023-12-26 21:49:22,421][105692] Updated weights for policy 0, policy_version 887445 (0.0009) [2023-12-26 21:49:22,485][105692] Updated weights for policy 0, policy_version 887455 (0.0006) [2023-12-26 21:49:22,549][105692] Updated weights for policy 0, policy_version 887465 (0.0007) [2023-12-26 21:49:22,724][105620] Updated weights for policy 1, policy_version 887417 (0.0010) [2023-12-26 21:49:22,779][105620] Updated weights for policy 1, policy_version 887427 (0.0009) [2023-12-26 21:49:22,844][105620] Updated weights for policy 1, policy_version 887437 (0.0008) [2023-12-26 21:49:22,904][105620] Updated weights for policy 1, policy_version 887447 (0.0008) [2023-12-26 21:49:23,223][105692] Updated weights for policy 0, policy_version 887475 (0.0009) [2023-12-26 21:49:23,276][105692] Updated weights for policy 0, policy_version 887485 (0.0010) [2023-12-26 21:49:23,335][105692] Updated weights for policy 0, policy_version 887496 (0.0009) [2023-12-26 21:49:23,594][105620] Updated weights for policy 1, policy_version 887457 (0.0008) [2023-12-26 21:49:23,655][105620] Updated weights for policy 1, policy_version 887467 (0.0009) [2023-12-26 21:49:23,702][105620] Updated weights for policy 1, policy_version 887477 (0.0008) [2023-12-26 21:49:23,993][105692] Updated weights for policy 0, policy_version 887506 (0.0008) [2023-12-26 21:49:24,049][105692] Updated weights for policy 0, policy_version 887516 (0.0005) [2023-12-26 21:49:24,102][105692] Updated weights for policy 0, policy_version 887526 (0.0006) [2023-12-26 21:49:24,148][105692] Updated weights for policy 0, policy_version 887536 (0.0007) [2023-12-26 21:49:24,477][105620] Updated weights for policy 1, policy_version 887487 (0.0009) [2023-12-26 21:49:24,524][105620] Updated weights for policy 1, policy_version 887497 (0.0009) [2023-12-26 21:49:24,577][105620] Updated weights for policy 1, policy_version 887507 (0.0008) [2023-12-26 21:49:24,798][105692] Updated weights for policy 0, policy_version 887546 (0.0009) [2023-12-26 21:49:24,844][105692] Updated weights for policy 0, policy_version 887556 (0.0010) [2023-12-26 21:49:24,892][105692] Updated weights for policy 0, policy_version 887566 (0.0007) [2023-12-26 21:49:25,158][105620] Updated weights for policy 1, policy_version 887517 (0.0008) [2023-12-26 21:49:25,216][105620] Updated weights for policy 1, policy_version 887527 (0.0008) [2023-12-26 21:49:25,282][105620] Updated weights for policy 1, policy_version 887537 (0.0005) [2023-12-26 21:49:25,483][105692] Updated weights for policy 0, policy_version 887576 (0.0009) [2023-12-26 21:49:25,543][105692] Updated weights for policy 0, policy_version 887586 (0.0010) [2023-12-26 21:49:25,602][105692] Updated weights for policy 0, policy_version 887596 (0.0010) [2023-12-26 21:49:25,935][105620] Updated weights for policy 1, policy_version 887547 (0.0008) [2023-12-26 21:49:25,984][105620] Updated weights for policy 1, policy_version 887557 (0.0006) [2023-12-26 21:49:26,035][105620] Updated weights for policy 1, policy_version 887567 (0.0010) [2023-12-26 21:49:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 454500352. Throughput: 0: 9876.5, 1: 9658.3. Samples: 454513072. Policy #0 lag: (min: 16.0, avg: 43.7, max: 48.0) [2023-12-26 21:49:26,062][104569] Avg episode reward: [(0, '8998.290'), (1, '9174.894')] [2023-12-26 21:49:26,237][105692] Updated weights for policy 0, policy_version 887606 (0.0007) [2023-12-26 21:49:26,286][105692] Updated weights for policy 0, policy_version 887616 (0.0005) [2023-12-26 21:49:26,332][105692] Updated weights for policy 0, policy_version 887626 (0.0009) [2023-12-26 21:49:26,580][105620] Updated weights for policy 1, policy_version 887577 (0.0007) [2023-12-26 21:49:26,631][105620] Updated weights for policy 1, policy_version 887587 (0.0005) [2023-12-26 21:49:26,679][105620] Updated weights for policy 1, policy_version 887597 (0.0005) [2023-12-26 21:49:26,724][105620] Updated weights for policy 1, policy_version 887607 (0.0005) [2023-12-26 21:49:27,063][105692] Updated weights for policy 0, policy_version 887636 (0.0011) [2023-12-26 21:49:27,117][105692] Updated weights for policy 0, policy_version 887646 (0.0010) [2023-12-26 21:49:27,168][105692] Updated weights for policy 0, policy_version 887656 (0.0010) [2023-12-26 21:49:27,239][105620] Updated weights for policy 1, policy_version 887617 (0.0005) [2023-12-26 21:49:27,295][105620] Updated weights for policy 1, policy_version 887627 (0.0005) [2023-12-26 21:49:27,345][105620] Updated weights for policy 1, policy_version 887637 (0.0006) [2023-12-26 21:49:27,859][105620] Updated weights for policy 1, policy_version 887647 (0.0005) [2023-12-26 21:49:27,917][105620] Updated weights for policy 1, policy_version 887657 (0.0005) [2023-12-26 21:49:27,922][105692] Updated weights for policy 0, policy_version 887666 (0.0010) [2023-12-26 21:49:27,976][105620] Updated weights for policy 1, policy_version 887667 (0.0005) [2023-12-26 21:49:27,979][105692] Updated weights for policy 0, policy_version 887676 (0.0010) [2023-12-26 21:49:28,025][105692] Updated weights for policy 0, policy_version 887686 (0.0007) [2023-12-26 21:49:28,071][105692] Updated weights for policy 0, policy_version 887696 (0.0005) [2023-12-26 21:49:28,557][105620] Updated weights for policy 1, policy_version 887677 (0.0006) [2023-12-26 21:49:28,628][105620] Updated weights for policy 1, policy_version 887687 (0.0006) [2023-12-26 21:49:28,689][105620] Updated weights for policy 1, policy_version 887697 (0.0005) [2023-12-26 21:49:28,773][105692] Updated weights for policy 0, policy_version 887706 (0.0010) [2023-12-26 21:49:28,835][105692] Updated weights for policy 0, policy_version 887716 (0.0005) [2023-12-26 21:49:28,884][105692] Updated weights for policy 0, policy_version 887726 (0.0005) [2023-12-26 21:49:29,381][105620] Updated weights for policy 1, policy_version 887707 (0.0009) [2023-12-26 21:49:29,448][105620] Updated weights for policy 1, policy_version 887717 (0.0008) [2023-12-26 21:49:29,456][105692] Updated weights for policy 0, policy_version 887736 (0.0005) [2023-12-26 21:49:29,509][105692] Updated weights for policy 0, policy_version 887746 (0.0008) [2023-12-26 21:49:29,516][105620] Updated weights for policy 1, policy_version 887727 (0.0008) [2023-12-26 21:49:29,565][105692] Updated weights for policy 0, policy_version 887756 (0.0010) [2023-12-26 21:49:30,229][105620] Updated weights for policy 1, policy_version 887737 (0.0009) [2023-12-26 21:49:30,288][105620] Updated weights for policy 1, policy_version 887747 (0.0010) [2023-12-26 21:49:30,318][105692] Updated weights for policy 0, policy_version 887766 (0.0008) [2023-12-26 21:49:30,347][105620] Updated weights for policy 1, policy_version 887757 (0.0010) [2023-12-26 21:49:30,369][105692] Updated weights for policy 0, policy_version 887776 (0.0007) [2023-12-26 21:49:30,409][105620] Updated weights for policy 1, policy_version 887767 (0.0010) [2023-12-26 21:49:30,419][105692] Updated weights for policy 0, policy_version 887786 (0.0007) [2023-12-26 21:49:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 454606848. Throughput: 0: 9929.5, 1: 9817.6. Samples: 454580148. Policy #0 lag: (min: 16.0, avg: 43.7, max: 48.0) [2023-12-26 21:49:31,062][104569] Avg episode reward: [(0, '8909.796'), (1, '9174.653')] [2023-12-26 21:49:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000887792_227311616.pth... [2023-12-26 21:49:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000887768_227295232.pth... [2023-12-26 21:49:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000886640_227016704.pth [2023-12-26 21:49:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000886648_227008512.pth [2023-12-26 21:49:31,157][105620] Updated weights for policy 1, policy_version 887777 (0.0011) [2023-12-26 21:49:31,210][105620] Updated weights for policy 1, policy_version 887787 (0.0011) [2023-12-26 21:49:31,240][105692] Updated weights for policy 0, policy_version 887796 (0.0009) [2023-12-26 21:49:31,269][105620] Updated weights for policy 1, policy_version 887797 (0.0010) [2023-12-26 21:49:31,299][105692] Updated weights for policy 0, policy_version 887806 (0.0007) [2023-12-26 21:49:31,360][105692] Updated weights for policy 0, policy_version 887816 (0.0007) [2023-12-26 21:49:31,924][105620] Updated weights for policy 1, policy_version 887807 (0.0006) [2023-12-26 21:49:31,978][105620] Updated weights for policy 1, policy_version 887817 (0.0005) [2023-12-26 21:49:32,042][105620] Updated weights for policy 1, policy_version 887827 (0.0007) [2023-12-26 21:49:32,065][105692] Updated weights for policy 0, policy_version 887826 (0.0009) [2023-12-26 21:49:32,127][105692] Updated weights for policy 0, policy_version 887836 (0.0009) [2023-12-26 21:49:32,182][105692] Updated weights for policy 0, policy_version 887846 (0.0009) [2023-12-26 21:49:32,244][105692] Updated weights for policy 0, policy_version 887856 (0.0009) [2023-12-26 21:49:32,657][105620] Updated weights for policy 1, policy_version 887837 (0.0010) [2023-12-26 21:49:32,709][105620] Updated weights for policy 1, policy_version 887847 (0.0010) [2023-12-26 21:49:32,754][105620] Updated weights for policy 1, policy_version 887857 (0.0010) [2023-12-26 21:49:32,990][105692] Updated weights for policy 0, policy_version 887866 (0.0008) [2023-12-26 21:49:33,045][105692] Updated weights for policy 0, policy_version 887876 (0.0008) [2023-12-26 21:49:33,103][105692] Updated weights for policy 0, policy_version 887886 (0.0008) [2023-12-26 21:49:33,467][105620] Updated weights for policy 1, policy_version 887867 (0.0010) [2023-12-26 21:49:33,511][105620] Updated weights for policy 1, policy_version 887877 (0.0010) [2023-12-26 21:49:33,562][105620] Updated weights for policy 1, policy_version 887887 (0.0010) [2023-12-26 21:49:33,885][105692] Updated weights for policy 0, policy_version 887896 (0.0008) [2023-12-26 21:49:33,931][105692] Updated weights for policy 0, policy_version 887906 (0.0008) [2023-12-26 21:49:33,980][105692] Updated weights for policy 0, policy_version 887916 (0.0008) [2023-12-26 21:49:34,308][105620] Updated weights for policy 1, policy_version 887897 (0.0010) [2023-12-26 21:49:34,356][105620] Updated weights for policy 1, policy_version 887907 (0.0010) [2023-12-26 21:49:34,411][105620] Updated weights for policy 1, policy_version 887917 (0.0010) [2023-12-26 21:49:34,464][105620] Updated weights for policy 1, policy_version 887927 (0.0009) [2023-12-26 21:49:34,798][105692] Updated weights for policy 0, policy_version 887926 (0.0009) [2023-12-26 21:49:34,857][105692] Updated weights for policy 0, policy_version 887936 (0.0008) [2023-12-26 21:49:34,912][105692] Updated weights for policy 0, policy_version 887946 (0.0008) [2023-12-26 21:49:35,220][105620] Updated weights for policy 1, policy_version 887937 (0.0010) [2023-12-26 21:49:35,285][105620] Updated weights for policy 1, policy_version 887947 (0.0010) [2023-12-26 21:49:35,342][105620] Updated weights for policy 1, policy_version 887957 (0.0010) [2023-12-26 21:49:35,668][105692] Updated weights for policy 0, policy_version 887956 (0.0009) [2023-12-26 21:49:35,722][105692] Updated weights for policy 0, policy_version 887966 (0.0009) [2023-12-26 21:49:35,775][105692] Updated weights for policy 0, policy_version 887976 (0.0008) [2023-12-26 21:49:36,031][105620] Updated weights for policy 1, policy_version 887967 (0.0009) [2023-12-26 21:49:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 454705152. Throughput: 0: 9957.7, 1: 9743.0. Samples: 454696432. Policy #0 lag: (min: 16.0, avg: 43.7, max: 48.0) [2023-12-26 21:49:36,062][104569] Avg episode reward: [(0, '8899.855'), (1, '9174.950')] [2023-12-26 21:49:36,090][105620] Updated weights for policy 1, policy_version 887977 (0.0009) [2023-12-26 21:49:36,156][105620] Updated weights for policy 1, policy_version 887987 (0.0007) [2023-12-26 21:49:36,553][105692] Updated weights for policy 0, policy_version 887986 (0.0009) [2023-12-26 21:49:36,612][105692] Updated weights for policy 0, policy_version 887996 (0.0009) [2023-12-26 21:49:36,671][105692] Updated weights for policy 0, policy_version 888006 (0.0009) [2023-12-26 21:49:36,732][105692] Updated weights for policy 0, policy_version 888016 (0.0008) [2023-12-26 21:49:36,917][105620] Updated weights for policy 1, policy_version 887997 (0.0009) [2023-12-26 21:49:36,972][105620] Updated weights for policy 1, policy_version 888007 (0.0009) [2023-12-26 21:49:37,018][105620] Updated weights for policy 1, policy_version 888017 (0.0008) [2023-12-26 21:49:37,021][105586] KL-divergence is very high: 182.4274 [2023-12-26 21:49:37,516][105692] Updated weights for policy 0, policy_version 888026 (0.0010) [2023-12-26 21:49:37,569][105692] Updated weights for policy 0, policy_version 888036 (0.0010) [2023-12-26 21:49:37,631][105692] Updated weights for policy 0, policy_version 888046 (0.0008) [2023-12-26 21:49:37,701][105620] Updated weights for policy 1, policy_version 888027 (0.0009) [2023-12-26 21:49:37,756][105620] Updated weights for policy 1, policy_version 888037 (0.0008) [2023-12-26 21:49:37,822][105620] Updated weights for policy 1, policy_version 888047 (0.0005) [2023-12-26 21:49:38,410][105692] Updated weights for policy 0, policy_version 888056 (0.0008) [2023-12-26 21:49:38,460][105692] Updated weights for policy 0, policy_version 888066 (0.0010) [2023-12-26 21:49:38,515][105692] Updated weights for policy 0, policy_version 888076 (0.0009) [2023-12-26 21:49:38,548][105620] Updated weights for policy 1, policy_version 888057 (0.0007) [2023-12-26 21:49:38,603][105620] Updated weights for policy 1, policy_version 888067 (0.0008) [2023-12-26 21:49:38,624][105586] KL-divergence is very high: 223.3924 [2023-12-26 21:49:38,670][105620] Updated weights for policy 1, policy_version 888077 (0.0009) [2023-12-26 21:49:38,676][105586] KL-divergence is very high: 314.5714 [2023-12-26 21:49:38,726][105586] KL-divergence is very high: 240.2726 [2023-12-26 21:49:38,732][105620] Updated weights for policy 1, policy_version 888087 (0.0008) [2023-12-26 21:49:39,262][105692] Updated weights for policy 0, policy_version 888086 (0.0008) [2023-12-26 21:49:39,313][105692] Updated weights for policy 0, policy_version 888096 (0.0005) [2023-12-26 21:49:39,378][105692] Updated weights for policy 0, policy_version 888106 (0.0007) [2023-12-26 21:49:39,558][105620] Updated weights for policy 1, policy_version 888097 (0.0010) [2023-12-26 21:49:39,620][105620] Updated weights for policy 1, policy_version 888107 (0.0010) [2023-12-26 21:49:39,671][105620] Updated weights for policy 1, policy_version 888117 (0.0009) [2023-12-26 21:49:40,062][105692] Updated weights for policy 0, policy_version 888116 (0.0008) [2023-12-26 21:49:40,129][105692] Updated weights for policy 0, policy_version 888126 (0.0010) [2023-12-26 21:49:40,192][105692] Updated weights for policy 0, policy_version 888136 (0.0010) [2023-12-26 21:49:40,405][105620] Updated weights for policy 1, policy_version 888127 (0.0007) [2023-12-26 21:49:40,478][105620] Updated weights for policy 1, policy_version 888137 (0.0006) [2023-12-26 21:49:40,532][105620] Updated weights for policy 1, policy_version 888147 (0.0008) [2023-12-26 21:49:40,934][105692] Updated weights for policy 0, policy_version 888146 (0.0009) [2023-12-26 21:49:40,981][105692] Updated weights for policy 0, policy_version 888156 (0.0009) [2023-12-26 21:49:41,042][105692] Updated weights for policy 0, policy_version 888166 (0.0009) [2023-12-26 21:49:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 454795264. Throughput: 0: 9870.1, 1: 9738.1. Samples: 454809048. Policy #0 lag: (min: 16.0, avg: 43.7, max: 48.0) [2023-12-26 21:49:41,063][104569] Avg episode reward: [(0, '9169.651'), (1, '8900.399')] [2023-12-26 21:49:41,113][105692] Updated weights for policy 0, policy_version 888176 (0.0008) [2023-12-26 21:49:41,135][105620] Updated weights for policy 1, policy_version 888157 (0.0007) [2023-12-26 21:49:41,202][105620] Updated weights for policy 1, policy_version 888167 (0.0010) [2023-12-26 21:49:41,259][105620] Updated weights for policy 1, policy_version 888177 (0.0011) [2023-12-26 21:49:41,902][105692] Updated weights for policy 0, policy_version 888186 (0.0011) [2023-12-26 21:49:41,955][105692] Updated weights for policy 0, policy_version 888196 (0.0011) [2023-12-26 21:49:42,002][105620] Updated weights for policy 1, policy_version 888187 (0.0011) [2023-12-26 21:49:42,012][105692] Updated weights for policy 0, policy_version 888206 (0.0011) [2023-12-26 21:49:42,066][105620] Updated weights for policy 1, policy_version 888197 (0.0011) [2023-12-26 21:49:42,129][105620] Updated weights for policy 1, policy_version 888207 (0.0011) [2023-12-26 21:49:42,786][105692] Updated weights for policy 0, policy_version 888216 (0.0011) [2023-12-26 21:49:42,838][105692] Updated weights for policy 0, policy_version 888226 (0.0011) [2023-12-26 21:49:42,884][105620] Updated weights for policy 1, policy_version 888217 (0.0010) [2023-12-26 21:49:42,891][105692] Updated weights for policy 0, policy_version 888236 (0.0011) [2023-12-26 21:49:42,936][105620] Updated weights for policy 1, policy_version 888227 (0.0010) [2023-12-26 21:49:42,984][105620] Updated weights for policy 1, policy_version 888237 (0.0010) [2023-12-26 21:49:43,054][105620] Updated weights for policy 1, policy_version 888247 (0.0010) [2023-12-26 21:49:43,566][105692] Updated weights for policy 0, policy_version 888246 (0.0010) [2023-12-26 21:49:43,617][105692] Updated weights for policy 0, policy_version 888256 (0.0006) [2023-12-26 21:49:43,679][105692] Updated weights for policy 0, policy_version 888266 (0.0006) [2023-12-26 21:49:43,806][105620] Updated weights for policy 1, policy_version 888257 (0.0010) [2023-12-26 21:49:43,861][105620] Updated weights for policy 1, policy_version 888267 (0.0010) [2023-12-26 21:49:43,919][105620] Updated weights for policy 1, policy_version 888277 (0.0010) [2023-12-26 21:49:44,420][105692] Updated weights for policy 0, policy_version 888276 (0.0009) [2023-12-26 21:49:44,483][105692] Updated weights for policy 0, policy_version 888286 (0.0008) [2023-12-26 21:49:44,547][105692] Updated weights for policy 0, policy_version 888296 (0.0008) [2023-12-26 21:49:44,589][105620] Updated weights for policy 1, policy_version 888287 (0.0011) [2023-12-26 21:49:44,650][105620] Updated weights for policy 1, policy_version 888297 (0.0011) [2023-12-26 21:49:44,703][105620] Updated weights for policy 1, policy_version 888307 (0.0010) [2023-12-26 21:49:45,308][105692] Updated weights for policy 0, policy_version 888306 (0.0009) [2023-12-26 21:49:45,360][105692] Updated weights for policy 0, policy_version 888316 (0.0010) [2023-12-26 21:49:45,419][105692] Updated weights for policy 0, policy_version 888326 (0.0010) [2023-12-26 21:49:45,475][105692] Updated weights for policy 0, policy_version 888336 (0.0010) [2023-12-26 21:49:45,480][105620] Updated weights for policy 1, policy_version 888317 (0.0010) [2023-12-26 21:49:45,548][105620] Updated weights for policy 1, policy_version 888327 (0.0009) [2023-12-26 21:49:45,617][105620] Updated weights for policy 1, policy_version 888337 (0.0006) [2023-12-26 21:49:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 454893568. Throughput: 0: 9802.1, 1: 9738.4. Samples: 454865648. Policy #0 lag: (min: 16.0, avg: 43.7, max: 48.0) [2023-12-26 21:49:46,062][104569] Avg episode reward: [(0, '9176.548'), (1, '8807.562')] [2023-12-26 21:49:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000888344_227442688.pth... [2023-12-26 21:49:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000887192_227147776.pth [2023-12-26 21:49:46,085][105692] Updated weights for policy 0, policy_version 888346 (0.0005) [2023-12-26 21:49:46,130][105692] Updated weights for policy 0, policy_version 888356 (0.0005) [2023-12-26 21:49:46,176][105692] Updated weights for policy 0, policy_version 888366 (0.0005) [2023-12-26 21:49:46,185][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000888368_227459072.pth... [2023-12-26 21:49:46,188][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000887216_227164160.pth [2023-12-26 21:49:46,338][105620] Updated weights for policy 1, policy_version 888347 (0.0010) [2023-12-26 21:49:46,396][105620] Updated weights for policy 1, policy_version 888357 (0.0006) [2023-12-26 21:49:46,453][105620] Updated weights for policy 1, policy_version 888367 (0.0005) [2023-12-26 21:49:46,716][105692] Updated weights for policy 0, policy_version 888376 (0.0005) [2023-12-26 21:49:46,770][105692] Updated weights for policy 0, policy_version 888386 (0.0005) [2023-12-26 21:49:46,816][105692] Updated weights for policy 0, policy_version 888396 (0.0005) [2023-12-26 21:49:47,107][105620] Updated weights for policy 1, policy_version 888377 (0.0006) [2023-12-26 21:49:47,158][105620] Updated weights for policy 1, policy_version 888387 (0.0009) [2023-12-26 21:49:47,208][105620] Updated weights for policy 1, policy_version 888397 (0.0007) [2023-12-26 21:49:47,268][105620] Updated weights for policy 1, policy_version 888407 (0.0008) [2023-12-26 21:49:47,374][105692] Updated weights for policy 0, policy_version 888406 (0.0006) [2023-12-26 21:49:47,433][105692] Updated weights for policy 0, policy_version 888416 (0.0005) [2023-12-26 21:49:47,487][105692] Updated weights for policy 0, policy_version 888426 (0.0005) [2023-12-26 21:49:47,902][105620] Updated weights for policy 1, policy_version 888417 (0.0006) [2023-12-26 21:49:47,975][105620] Updated weights for policy 1, policy_version 888427 (0.0006) [2023-12-26 21:49:48,004][105692] Updated weights for policy 0, policy_version 888436 (0.0005) [2023-12-26 21:49:48,028][105620] Updated weights for policy 1, policy_version 888437 (0.0008) [2023-12-26 21:49:48,058][105692] Updated weights for policy 0, policy_version 888446 (0.0005) [2023-12-26 21:49:48,118][105692] Updated weights for policy 0, policy_version 888456 (0.0006) [2023-12-26 21:49:48,629][105620] Updated weights for policy 1, policy_version 888447 (0.0008) [2023-12-26 21:49:48,679][105620] Updated weights for policy 1, policy_version 888457 (0.0008) [2023-12-26 21:49:48,724][105620] Updated weights for policy 1, policy_version 888467 (0.0007) [2023-12-26 21:49:48,801][105692] Updated weights for policy 0, policy_version 888466 (0.0006) [2023-12-26 21:49:48,854][105692] Updated weights for policy 0, policy_version 888476 (0.0010) [2023-12-26 21:49:48,902][105692] Updated weights for policy 0, policy_version 888486 (0.0010) [2023-12-26 21:49:48,955][105692] Updated weights for policy 0, policy_version 888496 (0.0010) [2023-12-26 21:49:49,358][105620] Updated weights for policy 1, policy_version 888477 (0.0008) [2023-12-26 21:49:49,428][105620] Updated weights for policy 1, policy_version 888487 (0.0006) [2023-12-26 21:49:49,493][105620] Updated weights for policy 1, policy_version 888497 (0.0006) [2023-12-26 21:49:49,722][105692] Updated weights for policy 0, policy_version 888506 (0.0009) [2023-12-26 21:49:49,785][105692] Updated weights for policy 0, policy_version 888516 (0.0010) [2023-12-26 21:49:49,848][105692] Updated weights for policy 0, policy_version 888526 (0.0011) [2023-12-26 21:49:50,269][105620] Updated weights for policy 1, policy_version 888507 (0.0007) [2023-12-26 21:49:50,326][105620] Updated weights for policy 1, policy_version 888517 (0.0008) [2023-12-26 21:49:50,383][105620] Updated weights for policy 1, policy_version 888527 (0.0008) [2023-12-26 21:49:50,499][105692] Updated weights for policy 0, policy_version 888536 (0.0010) [2023-12-26 21:49:50,555][105692] Updated weights for policy 0, policy_version 888546 (0.0010) [2023-12-26 21:49:50,619][105692] Updated weights for policy 0, policy_version 888556 (0.0009) [2023-12-26 21:49:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 455000064. Throughput: 0: 9892.7, 1: 9805.0. Samples: 454991508. Policy #0 lag: (min: 16.0, avg: 43.7, max: 48.0) [2023-12-26 21:49:51,063][104569] Avg episode reward: [(0, '9005.027'), (1, '8991.157')] [2023-12-26 21:49:51,232][105620] Updated weights for policy 1, policy_version 888537 (0.0008) [2023-12-26 21:49:51,302][105620] Updated weights for policy 1, policy_version 888547 (0.0007) [2023-12-26 21:49:51,347][105692] Updated weights for policy 0, policy_version 888566 (0.0007) [2023-12-26 21:49:51,367][105620] Updated weights for policy 1, policy_version 888557 (0.0007) [2023-12-26 21:49:51,410][105692] Updated weights for policy 0, policy_version 888576 (0.0012) [2023-12-26 21:49:51,436][105620] Updated weights for policy 1, policy_version 888567 (0.0007) [2023-12-26 21:49:51,494][105692] Updated weights for policy 0, policy_version 888586 (0.0011) [2023-12-26 21:49:52,064][105620] Updated weights for policy 1, policy_version 888577 (0.0008) [2023-12-26 21:49:52,133][105620] Updated weights for policy 1, policy_version 888587 (0.0010) [2023-12-26 21:49:52,188][105692] Updated weights for policy 0, policy_version 888596 (0.0009) [2023-12-26 21:49:52,196][105620] Updated weights for policy 1, policy_version 888597 (0.0008) [2023-12-26 21:49:52,233][105692] Updated weights for policy 0, policy_version 888606 (0.0010) [2023-12-26 21:49:52,305][105692] Updated weights for policy 0, policy_version 888616 (0.0011) [2023-12-26 21:49:52,800][105620] Updated weights for policy 1, policy_version 888607 (0.0008) [2023-12-26 21:49:52,861][105586] KL-divergence is very high: 185.2871 [2023-12-26 21:49:52,867][105620] Updated weights for policy 1, policy_version 888617 (0.0009) [2023-12-26 21:49:52,913][105586] KL-divergence is very high: 185.1244 [2023-12-26 21:49:52,933][105620] Updated weights for policy 1, policy_version 888627 (0.0007) [2023-12-26 21:49:53,042][105692] Updated weights for policy 0, policy_version 888626 (0.0010) [2023-12-26 21:49:53,096][105692] Updated weights for policy 0, policy_version 888636 (0.0006) [2023-12-26 21:49:53,149][105692] Updated weights for policy 0, policy_version 888646 (0.0010) [2023-12-26 21:49:53,201][105692] Updated weights for policy 0, policy_version 888656 (0.0010) [2023-12-26 21:49:53,663][105620] Updated weights for policy 1, policy_version 888637 (0.0008) [2023-12-26 21:49:53,710][105620] Updated weights for policy 1, policy_version 888647 (0.0008) [2023-12-26 21:49:53,768][105620] Updated weights for policy 1, policy_version 888657 (0.0008) [2023-12-26 21:49:53,939][105692] Updated weights for policy 0, policy_version 888666 (0.0011) [2023-12-26 21:49:53,985][105692] Updated weights for policy 0, policy_version 888676 (0.0008) [2023-12-26 21:49:54,047][105692] Updated weights for policy 0, policy_version 888686 (0.0005) [2023-12-26 21:49:54,534][105620] Updated weights for policy 1, policy_version 888667 (0.0008) [2023-12-26 21:49:54,590][105620] Updated weights for policy 1, policy_version 888677 (0.0008) [2023-12-26 21:49:54,650][105620] Updated weights for policy 1, policy_version 888687 (0.0008) [2023-12-26 21:49:54,717][105692] Updated weights for policy 0, policy_version 888696 (0.0009) [2023-12-26 21:49:54,771][105692] Updated weights for policy 0, policy_version 888706 (0.0010) [2023-12-26 21:49:54,829][105692] Updated weights for policy 0, policy_version 888716 (0.0010) [2023-12-26 21:49:55,399][105620] Updated weights for policy 1, policy_version 888697 (0.0010) [2023-12-26 21:49:55,447][105620] Updated weights for policy 1, policy_version 888707 (0.0010) [2023-12-26 21:49:55,481][105692] Updated weights for policy 0, policy_version 888726 (0.0006) [2023-12-26 21:49:55,502][105620] Updated weights for policy 1, policy_version 888717 (0.0010) [2023-12-26 21:49:55,534][105692] Updated weights for policy 0, policy_version 888736 (0.0007) [2023-12-26 21:49:55,565][105620] Updated weights for policy 1, policy_version 888727 (0.0010) [2023-12-26 21:49:55,577][105692] Updated weights for policy 0, policy_version 888746 (0.0008) [2023-12-26 21:49:56,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 455098368. Throughput: 0: 9798.8, 1: 9821.0. Samples: 455108156. Policy #0 lag: (min: 16.0, avg: 43.7, max: 48.0) [2023-12-26 21:49:56,062][104569] Avg episode reward: [(0, '9009.680'), (1, '8727.776')] [2023-12-26 21:49:56,203][105692] Updated weights for policy 0, policy_version 888756 (0.0008) [2023-12-26 21:49:56,219][105620] Updated weights for policy 1, policy_version 888737 (0.0009) [2023-12-26 21:49:56,256][105692] Updated weights for policy 0, policy_version 888766 (0.0006) [2023-12-26 21:49:56,278][105620] Updated weights for policy 1, policy_version 888747 (0.0006) [2023-12-26 21:49:56,315][105692] Updated weights for policy 0, policy_version 888776 (0.0005) [2023-12-26 21:49:56,334][105620] Updated weights for policy 1, policy_version 888757 (0.0010) [2023-12-26 21:49:56,892][105692] Updated weights for policy 0, policy_version 888786 (0.0006) [2023-12-26 21:49:56,954][105692] Updated weights for policy 0, policy_version 888796 (0.0006) [2023-12-26 21:49:56,964][105620] Updated weights for policy 1, policy_version 888767 (0.0010) [2023-12-26 21:49:57,005][105692] Updated weights for policy 0, policy_version 888806 (0.0007) [2023-12-26 21:49:57,012][105620] Updated weights for policy 1, policy_version 888777 (0.0010) [2023-12-26 21:49:57,053][105692] Updated weights for policy 0, policy_version 888816 (0.0005) [2023-12-26 21:49:57,056][105620] Updated weights for policy 1, policy_version 888787 (0.0010) [2023-12-26 21:49:57,614][105692] Updated weights for policy 0, policy_version 888826 (0.0005) [2023-12-26 21:49:57,668][105692] Updated weights for policy 0, policy_version 888836 (0.0005) [2023-12-26 21:49:57,690][105620] Updated weights for policy 1, policy_version 888797 (0.0008) [2023-12-26 21:49:57,725][105692] Updated weights for policy 0, policy_version 888846 (0.0006) [2023-12-26 21:49:57,746][105620] Updated weights for policy 1, policy_version 888807 (0.0009) [2023-12-26 21:49:57,795][105620] Updated weights for policy 1, policy_version 888817 (0.0010) [2023-12-26 21:49:58,446][105620] Updated weights for policy 1, policy_version 888827 (0.0008) [2023-12-26 21:49:58,494][105692] Updated weights for policy 0, policy_version 888856 (0.0010) [2023-12-26 21:49:58,510][105620] Updated weights for policy 1, policy_version 888837 (0.0008) [2023-12-26 21:49:58,548][105692] Updated weights for policy 0, policy_version 888866 (0.0010) [2023-12-26 21:49:58,570][105586] KL-divergence is very high: 102.0833 [2023-12-26 21:49:58,576][105620] Updated weights for policy 1, policy_version 888847 (0.0008) [2023-12-26 21:49:58,577][105586] KL-divergence is very high: 116.1165 [2023-12-26 21:49:58,583][105586] KL-divergence is very high: 161.3307 [2023-12-26 21:49:58,597][105586] KL-divergence is very high: 236.6909 [2023-12-26 21:49:58,603][105586] KL-divergence is very high: 253.5233 [2023-12-26 21:49:58,612][105692] Updated weights for policy 0, policy_version 888876 (0.0011) [2023-12-26 21:49:58,620][105586] KL-divergence is very high: 286.4307 [2023-12-26 21:49:58,626][105586] KL-divergence is very high: 206.6497 [2023-12-26 21:49:59,382][105692] Updated weights for policy 0, policy_version 888886 (0.0009) [2023-12-26 21:49:59,386][105620] Updated weights for policy 1, policy_version 888857 (0.0011) [2023-12-26 21:49:59,387][105586] KL-divergence is very high: 186.0598 [2023-12-26 21:49:59,406][105586] KL-divergence is very high: 162.8939 [2023-12-26 21:49:59,438][105586] KL-divergence is very high: 108.3314 [2023-12-26 21:49:59,447][105692] Updated weights for policy 0, policy_version 888896 (0.0008) [2023-12-26 21:49:59,449][105620] Updated weights for policy 1, policy_version 888867 (0.0008) [2023-12-26 21:49:59,455][105586] KL-divergence is very high: 116.6264 [2023-12-26 21:49:59,499][105586] KL-divergence is very high: 128.1602 [2023-12-26 21:49:59,505][105620] Updated weights for policy 1, policy_version 888877 (0.0006) [2023-12-26 21:49:59,510][105692] Updated weights for policy 0, policy_version 888906 (0.0008) [2023-12-26 21:49:59,527][105586] KL-divergence is very high: 102.9740 [2023-12-26 21:49:59,543][105586] KL-divergence is very high: 144.5473 [2023-12-26 21:49:59,560][105620] Updated weights for policy 1, policy_version 888887 (0.0010) [2023-12-26 21:50:00,188][105692] Updated weights for policy 0, policy_version 888916 (0.0009) [2023-12-26 21:50:00,251][105692] Updated weights for policy 0, policy_version 888926 (0.0008) [2023-12-26 21:50:00,312][105692] Updated weights for policy 0, policy_version 888936 (0.0006) [2023-12-26 21:50:00,365][105620] Updated weights for policy 1, policy_version 888897 (0.0009) [2023-12-26 21:50:00,424][105620] Updated weights for policy 1, policy_version 888907 (0.0009) [2023-12-26 21:50:00,482][105620] Updated weights for policy 1, policy_version 888917 (0.0009) [2023-12-26 21:50:01,042][105692] Updated weights for policy 0, policy_version 888946 (0.0006) [2023-12-26 21:50:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 455196672. Throughput: 0: 9861.3, 1: 9895.4. Samples: 455172356. Policy #0 lag: (min: 16.0, avg: 43.7, max: 48.0) [2023-12-26 21:50:01,063][104569] Avg episode reward: [(0, '8731.116'), (1, '6908.362')] [2023-12-26 21:50:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000888920_227590144.pth... [2023-12-26 21:50:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000887768_227295232.pth [2023-12-26 21:50:01,098][105692] Updated weights for policy 0, policy_version 888956 (0.0008) [2023-12-26 21:50:01,154][105692] Updated weights for policy 0, policy_version 888966 (0.0009) [2023-12-26 21:50:01,206][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000888976_227614720.pth... [2023-12-26 21:50:01,209][105692] Updated weights for policy 0, policy_version 888976 (0.0008) [2023-12-26 21:50:01,210][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000887792_227311616.pth [2023-12-26 21:50:01,253][105620] Updated weights for policy 1, policy_version 888927 (0.0008) [2023-12-26 21:50:01,326][105620] Updated weights for policy 1, policy_version 888937 (0.0008) [2023-12-26 21:50:01,397][105620] Updated weights for policy 1, policy_version 888947 (0.0008) [2023-12-26 21:50:01,881][105692] Updated weights for policy 0, policy_version 888986 (0.0006) [2023-12-26 21:50:01,949][105692] Updated weights for policy 0, policy_version 888996 (0.0005) [2023-12-26 21:50:02,017][105692] Updated weights for policy 0, policy_version 889006 (0.0007) [2023-12-26 21:50:02,079][105620] Updated weights for policy 1, policy_version 888957 (0.0007) [2023-12-26 21:50:02,145][105620] Updated weights for policy 1, policy_version 888967 (0.0008) [2023-12-26 21:50:02,212][105620] Updated weights for policy 1, policy_version 888977 (0.0008) [2023-12-26 21:50:02,608][105692] Updated weights for policy 0, policy_version 889016 (0.0006) [2023-12-26 21:50:02,659][105692] Updated weights for policy 0, policy_version 889026 (0.0005) [2023-12-26 21:50:02,705][105692] Updated weights for policy 0, policy_version 889036 (0.0005) [2023-12-26 21:50:03,023][105620] Updated weights for policy 1, policy_version 888987 (0.0008) [2023-12-26 21:50:03,081][105620] Updated weights for policy 1, policy_version 888997 (0.0005) [2023-12-26 21:50:03,126][105620] Updated weights for policy 1, policy_version 889007 (0.0005) [2023-12-26 21:50:03,348][105692] Updated weights for policy 0, policy_version 889046 (0.0008) [2023-12-26 21:50:03,398][105692] Updated weights for policy 0, policy_version 889056 (0.0009) [2023-12-26 21:50:03,453][105692] Updated weights for policy 0, policy_version 889066 (0.0008) [2023-12-26 21:50:03,732][105620] Updated weights for policy 1, policy_version 889017 (0.0006) [2023-12-26 21:50:03,784][105620] Updated weights for policy 1, policy_version 889027 (0.0009) [2023-12-26 21:50:03,845][105620] Updated weights for policy 1, policy_version 889037 (0.0007) [2023-12-26 21:50:03,911][105620] Updated weights for policy 1, policy_version 889048 (0.0007) [2023-12-26 21:50:04,078][105692] Updated weights for policy 0, policy_version 889076 (0.0006) [2023-12-26 21:50:04,138][105692] Updated weights for policy 0, policy_version 889086 (0.0008) [2023-12-26 21:50:04,197][105692] Updated weights for policy 0, policy_version 889096 (0.0008) [2023-12-26 21:50:04,545][105620] Updated weights for policy 1, policy_version 889058 (0.0011) [2023-12-26 21:50:04,608][105620] Updated weights for policy 1, policy_version 889068 (0.0010) [2023-12-26 21:50:04,670][105620] Updated weights for policy 1, policy_version 889078 (0.0010) [2023-12-26 21:50:04,940][105692] Updated weights for policy 0, policy_version 889106 (0.0008) [2023-12-26 21:50:05,005][105692] Updated weights for policy 0, policy_version 889116 (0.0009) [2023-12-26 21:50:05,055][105692] Updated weights for policy 0, policy_version 889126 (0.0007) [2023-12-26 21:50:05,107][105692] Updated weights for policy 0, policy_version 889136 (0.0005) [2023-12-26 21:50:05,356][105620] Updated weights for policy 1, policy_version 889088 (0.0006) [2023-12-26 21:50:05,421][105620] Updated weights for policy 1, policy_version 889098 (0.0006) [2023-12-26 21:50:05,473][105620] Updated weights for policy 1, policy_version 889108 (0.0009) [2023-12-26 21:50:05,724][105692] Updated weights for policy 0, policy_version 889146 (0.0009) [2023-12-26 21:50:05,776][105692] Updated weights for policy 0, policy_version 889156 (0.0006) [2023-12-26 21:50:05,829][105692] Updated weights for policy 0, policy_version 889166 (0.0005) [2023-12-26 21:50:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 455303168. Throughput: 0: 9946.6, 1: 9952.4. Samples: 455292076. Policy #0 lag: (min: 16.0, avg: 43.7, max: 48.0) [2023-12-26 21:50:06,062][104569] Avg episode reward: [(0, '8643.106'), (1, '7188.281')] [2023-12-26 21:50:06,095][105620] Updated weights for policy 1, policy_version 889118 (0.0010) [2023-12-26 21:50:06,151][105620] Updated weights for policy 1, policy_version 889128 (0.0009) [2023-12-26 21:50:06,206][105620] Updated weights for policy 1, policy_version 889138 (0.0009) [2023-12-26 21:50:06,646][105692] Updated weights for policy 0, policy_version 889176 (0.0009) [2023-12-26 21:50:06,709][105692] Updated weights for policy 0, policy_version 889186 (0.0009) [2023-12-26 21:50:06,775][105692] Updated weights for policy 0, policy_version 889196 (0.0009) [2023-12-26 21:50:06,879][105620] Updated weights for policy 1, policy_version 889148 (0.0009) [2023-12-26 21:50:06,943][105620] Updated weights for policy 1, policy_version 889158 (0.0007) [2023-12-26 21:50:06,994][105620] Updated weights for policy 1, policy_version 889168 (0.0008) [2023-12-26 21:50:07,503][105692] Updated weights for policy 0, policy_version 889206 (0.0007) [2023-12-26 21:50:07,553][105692] Updated weights for policy 0, policy_version 889216 (0.0007) [2023-12-26 21:50:07,601][105692] Updated weights for policy 0, policy_version 889226 (0.0008) [2023-12-26 21:50:07,766][105620] Updated weights for policy 1, policy_version 889178 (0.0008) [2023-12-26 21:50:07,821][105620] Updated weights for policy 1, policy_version 889188 (0.0006) [2023-12-26 21:50:07,875][105620] Updated weights for policy 1, policy_version 889198 (0.0006) [2023-12-26 21:50:07,936][105620] Updated weights for policy 1, policy_version 889208 (0.0009) [2023-12-26 21:50:08,364][105692] Updated weights for policy 0, policy_version 889236 (0.0008) [2023-12-26 21:50:08,424][105692] Updated weights for policy 0, policy_version 889246 (0.0009) [2023-12-26 21:50:08,483][105692] Updated weights for policy 0, policy_version 889256 (0.0009) [2023-12-26 21:50:08,593][105620] Updated weights for policy 1, policy_version 889218 (0.0005) [2023-12-26 21:50:08,653][105620] Updated weights for policy 1, policy_version 889228 (0.0006) [2023-12-26 21:50:08,702][105620] Updated weights for policy 1, policy_version 889238 (0.0005) [2023-12-26 21:50:09,289][105692] Updated weights for policy 0, policy_version 889266 (0.0009) [2023-12-26 21:50:09,336][105620] Updated weights for policy 1, policy_version 889248 (0.0007) [2023-12-26 21:50:09,351][105692] Updated weights for policy 0, policy_version 889276 (0.0008) [2023-12-26 21:50:09,411][105620] Updated weights for policy 1, policy_version 889258 (0.0008) [2023-12-26 21:50:09,425][105692] Updated weights for policy 0, policy_version 889286 (0.0009) [2023-12-26 21:50:09,478][105620] Updated weights for policy 1, policy_version 889268 (0.0007) [2023-12-26 21:50:09,485][105692] Updated weights for policy 0, policy_version 889296 (0.0008) [2023-12-26 21:50:10,164][105620] Updated weights for policy 1, policy_version 889278 (0.0009) [2023-12-26 21:50:10,226][105620] Updated weights for policy 1, policy_version 889288 (0.0007) [2023-12-26 21:50:10,244][105692] Updated weights for policy 0, policy_version 889306 (0.0008) [2023-12-26 21:50:10,283][105620] Updated weights for policy 1, policy_version 889298 (0.0007) [2023-12-26 21:50:10,305][105692] Updated weights for policy 0, policy_version 889316 (0.0007) [2023-12-26 21:50:10,364][105692] Updated weights for policy 0, policy_version 889326 (0.0009) [2023-12-26 21:50:11,038][105620] Updated weights for policy 1, policy_version 889308 (0.0007) [2023-12-26 21:50:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 455393280. Throughput: 0: 9878.0, 1: 10002.5. Samples: 455407692. Policy #0 lag: (min: 16.0, avg: 43.7, max: 48.0) [2023-12-26 21:50:11,062][104569] Avg episode reward: [(0, '8949.179'), (1, '8720.825')] [2023-12-26 21:50:11,089][105692] Updated weights for policy 0, policy_version 889336 (0.0009) [2023-12-26 21:50:11,109][105620] Updated weights for policy 1, policy_version 889318 (0.0007) [2023-12-26 21:50:11,155][105586] KL-divergence is very high: 134.7780 [2023-12-26 21:50:11,160][105692] Updated weights for policy 0, policy_version 889346 (0.0008) [2023-12-26 21:50:11,183][105620] Updated weights for policy 1, policy_version 889328 (0.0007) [2023-12-26 21:50:11,207][105586] KL-divergence is very high: 103.5693 [2023-12-26 21:50:11,219][105692] Updated weights for policy 0, policy_version 889356 (0.0008) [2023-12-26 21:50:11,896][105692] Updated weights for policy 0, policy_version 889366 (0.0007) [2023-12-26 21:50:11,955][105692] Updated weights for policy 0, policy_version 889376 (0.0007) [2023-12-26 21:50:12,011][105692] Updated weights for policy 0, policy_version 889386 (0.0007) [2023-12-26 21:50:12,022][105620] Updated weights for policy 1, policy_version 889338 (0.0007) [2023-12-26 21:50:12,080][105620] Updated weights for policy 1, policy_version 889348 (0.0008) [2023-12-26 21:50:12,147][105620] Updated weights for policy 1, policy_version 889358 (0.0007) [2023-12-26 21:50:12,195][105620] Updated weights for policy 1, policy_version 889368 (0.0006) [2023-12-26 21:50:12,807][105692] Updated weights for policy 0, policy_version 889396 (0.0007) [2023-12-26 21:50:12,869][105692] Updated weights for policy 0, policy_version 889406 (0.0005) [2023-12-26 21:50:12,874][105620] Updated weights for policy 1, policy_version 889378 (0.0009) [2023-12-26 21:50:12,926][105692] Updated weights for policy 0, policy_version 889416 (0.0007) [2023-12-26 21:50:12,927][105620] Updated weights for policy 1, policy_version 889388 (0.0006) [2023-12-26 21:50:12,984][105620] Updated weights for policy 1, policy_version 889398 (0.0006) [2023-12-26 21:50:13,669][105692] Updated weights for policy 0, policy_version 889426 (0.0008) [2023-12-26 21:50:13,716][105620] Updated weights for policy 1, policy_version 889408 (0.0007) [2023-12-26 21:50:13,725][105692] Updated weights for policy 0, policy_version 889436 (0.0008) [2023-12-26 21:50:13,771][105620] Updated weights for policy 1, policy_version 889418 (0.0006) [2023-12-26 21:50:13,784][105692] Updated weights for policy 0, policy_version 889446 (0.0008) [2023-12-26 21:50:13,835][105620] Updated weights for policy 1, policy_version 889428 (0.0007) [2023-12-26 21:50:13,848][105692] Updated weights for policy 0, policy_version 889456 (0.0007) [2023-12-26 21:50:14,531][105620] Updated weights for policy 1, policy_version 889438 (0.0008) [2023-12-26 21:50:14,592][105620] Updated weights for policy 1, policy_version 889448 (0.0008) [2023-12-26 21:50:14,613][105692] Updated weights for policy 0, policy_version 889466 (0.0006) [2023-12-26 21:50:14,644][105620] Updated weights for policy 1, policy_version 889458 (0.0008) [2023-12-26 21:50:14,662][105692] Updated weights for policy 0, policy_version 889476 (0.0006) [2023-12-26 21:50:14,715][105692] Updated weights for policy 0, policy_version 889486 (0.0008) [2023-12-26 21:50:15,440][105620] Updated weights for policy 1, policy_version 889468 (0.0006) [2023-12-26 21:50:15,442][105692] Updated weights for policy 0, policy_version 889496 (0.0009) [2023-12-26 21:50:15,502][105620] Updated weights for policy 1, policy_version 889478 (0.0009) [2023-12-26 21:50:15,506][105692] Updated weights for policy 0, policy_version 889506 (0.0006) [2023-12-26 21:50:15,564][105620] Updated weights for policy 1, policy_version 889488 (0.0008) [2023-12-26 21:50:15,570][105692] Updated weights for policy 0, policy_version 889516 (0.0006) [2023-12-26 21:50:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 455491584. Throughput: 0: 9850.4, 1: 9781.5. Samples: 455463584. Policy #0 lag: (min: 16.0, avg: 43.7, max: 48.0) [2023-12-26 21:50:16,062][104569] Avg episode reward: [(0, '8778.566'), (1, '8995.645')] [2023-12-26 21:50:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000889520_227753984.pth... [2023-12-26 21:50:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000889496_227737600.pth... [2023-12-26 21:50:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000888368_227459072.pth [2023-12-26 21:50:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000888344_227442688.pth [2023-12-26 21:50:16,164][105692] Updated weights for policy 0, policy_version 889526 (0.0008) [2023-12-26 21:50:16,216][105692] Updated weights for policy 0, policy_version 889536 (0.0009) [2023-12-26 21:50:16,274][105692] Updated weights for policy 0, policy_version 889546 (0.0009) [2023-12-26 21:50:16,368][105620] Updated weights for policy 1, policy_version 889498 (0.0009) [2023-12-26 21:50:16,429][105620] Updated weights for policy 1, policy_version 889508 (0.0009) [2023-12-26 21:50:16,490][105620] Updated weights for policy 1, policy_version 889518 (0.0009) [2023-12-26 21:50:16,551][105620] Updated weights for policy 1, policy_version 889528 (0.0008) [2023-12-26 21:50:17,111][105692] Updated weights for policy 0, policy_version 889556 (0.0009) [2023-12-26 21:50:17,163][105692] Updated weights for policy 0, policy_version 889566 (0.0009) [2023-12-26 21:50:17,165][105620] Updated weights for policy 1, policy_version 889538 (0.0005) [2023-12-26 21:50:17,207][105692] Updated weights for policy 0, policy_version 889576 (0.0009) [2023-12-26 21:50:17,214][105620] Updated weights for policy 1, policy_version 889548 (0.0010) [2023-12-26 21:50:17,268][105620] Updated weights for policy 1, policy_version 889558 (0.0007) [2023-12-26 21:50:17,888][105620] Updated weights for policy 1, policy_version 889568 (0.0008) [2023-12-26 21:50:17,935][105620] Updated weights for policy 1, policy_version 889578 (0.0009) [2023-12-26 21:50:17,995][105620] Updated weights for policy 1, policy_version 889588 (0.0006) [2023-12-26 21:50:18,027][105692] Updated weights for policy 0, policy_version 889586 (0.0007) [2023-12-26 21:50:18,079][105692] Updated weights for policy 0, policy_version 889596 (0.0009) [2023-12-26 21:50:18,132][105692] Updated weights for policy 0, policy_version 889607 (0.0010) [2023-12-26 21:50:18,657][105620] Updated weights for policy 1, policy_version 889598 (0.0009) [2023-12-26 21:50:18,713][105620] Updated weights for policy 1, policy_version 889608 (0.0010) [2023-12-26 21:50:18,775][105620] Updated weights for policy 1, policy_version 889618 (0.0010) [2023-12-26 21:50:18,941][105692] Updated weights for policy 0, policy_version 889618 (0.0009) [2023-12-26 21:50:19,001][105692] Updated weights for policy 0, policy_version 889628 (0.0008) [2023-12-26 21:50:19,067][105692] Updated weights for policy 0, policy_version 889638 (0.0008) [2023-12-26 21:50:19,129][105692] Updated weights for policy 0, policy_version 889648 (0.0008) [2023-12-26 21:50:19,526][105620] Updated weights for policy 1, policy_version 889628 (0.0010) [2023-12-26 21:50:19,586][105620] Updated weights for policy 1, policy_version 889638 (0.0008) [2023-12-26 21:50:19,649][105620] Updated weights for policy 1, policy_version 889648 (0.0008) [2023-12-26 21:50:19,896][105692] Updated weights for policy 0, policy_version 889658 (0.0008) [2023-12-26 21:50:19,959][105692] Updated weights for policy 0, policy_version 889668 (0.0008) [2023-12-26 21:50:20,032][105692] Updated weights for policy 0, policy_version 889678 (0.0010) [2023-12-26 21:50:20,341][105620] Updated weights for policy 1, policy_version 889658 (0.0009) [2023-12-26 21:50:20,390][105620] Updated weights for policy 1, policy_version 889668 (0.0011) [2023-12-26 21:50:20,445][105620] Updated weights for policy 1, policy_version 889678 (0.0009) [2023-12-26 21:50:20,498][105620] Updated weights for policy 1, policy_version 889688 (0.0011) [2023-12-26 21:50:20,723][105692] Updated weights for policy 0, policy_version 889688 (0.0009) [2023-12-26 21:50:20,785][105692] Updated weights for policy 0, policy_version 889698 (0.0006) [2023-12-26 21:50:20,842][105692] Updated weights for policy 0, policy_version 889708 (0.0006) [2023-12-26 21:50:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 455589888. Throughput: 0: 9802.8, 1: 9780.4. Samples: 455577680. Policy #0 lag: (min: 16.0, avg: 43.7, max: 48.0) [2023-12-26 21:50:21,062][104569] Avg episode reward: [(0, '8918.209'), (1, '8648.780')] [2023-12-26 21:50:21,297][105620] Updated weights for policy 1, policy_version 889698 (0.0011) [2023-12-26 21:50:21,372][105620] Updated weights for policy 1, policy_version 889708 (0.0011) [2023-12-26 21:50:21,432][105620] Updated weights for policy 1, policy_version 889718 (0.0009) [2023-12-26 21:50:21,605][105692] Updated weights for policy 0, policy_version 889718 (0.0009) [2023-12-26 21:50:21,674][105692] Updated weights for policy 0, policy_version 889728 (0.0008) [2023-12-26 21:50:21,745][105692] Updated weights for policy 0, policy_version 889738 (0.0010) [2023-12-26 21:50:22,161][105620] Updated weights for policy 1, policy_version 889728 (0.0006) [2023-12-26 21:50:22,207][105620] Updated weights for policy 1, policy_version 889738 (0.0008) [2023-12-26 21:50:22,269][105620] Updated weights for policy 1, policy_version 889748 (0.0009) [2023-12-26 21:50:22,522][105692] Updated weights for policy 0, policy_version 889748 (0.0008) [2023-12-26 21:50:22,587][105692] Updated weights for policy 0, policy_version 889758 (0.0007) [2023-12-26 21:50:22,645][105692] Updated weights for policy 0, policy_version 889768 (0.0006) [2023-12-26 21:50:23,009][105620] Updated weights for policy 1, policy_version 889758 (0.0008) [2023-12-26 21:50:23,067][105620] Updated weights for policy 1, policy_version 889768 (0.0008) [2023-12-26 21:50:23,115][105620] Updated weights for policy 1, policy_version 889778 (0.0009) [2023-12-26 21:50:23,259][105692] Updated weights for policy 0, policy_version 889778 (0.0005) [2023-12-26 21:50:23,325][105692] Updated weights for policy 0, policy_version 889788 (0.0005) [2023-12-26 21:50:23,382][105692] Updated weights for policy 0, policy_version 889798 (0.0005) [2023-12-26 21:50:23,436][105692] Updated weights for policy 0, policy_version 889808 (0.0005) [2023-12-26 21:50:23,958][105620] Updated weights for policy 1, policy_version 889788 (0.0008) [2023-12-26 21:50:23,973][105692] Updated weights for policy 0, policy_version 889818 (0.0008) [2023-12-26 21:50:24,019][105620] Updated weights for policy 1, policy_version 889798 (0.0007) [2023-12-26 21:50:24,031][105692] Updated weights for policy 0, policy_version 889828 (0.0005) [2023-12-26 21:50:24,067][105620] Updated weights for policy 1, policy_version 889808 (0.0008) [2023-12-26 21:50:24,088][105692] Updated weights for policy 0, policy_version 889838 (0.0005) [2023-12-26 21:50:24,688][105692] Updated weights for policy 0, policy_version 889848 (0.0006) [2023-12-26 21:50:24,744][105692] Updated weights for policy 0, policy_version 889858 (0.0005) [2023-12-26 21:50:24,802][105692] Updated weights for policy 0, policy_version 889868 (0.0005) [2023-12-26 21:50:24,863][105620] Updated weights for policy 1, policy_version 889818 (0.0010) [2023-12-26 21:50:24,932][105620] Updated weights for policy 1, policy_version 889828 (0.0008) [2023-12-26 21:50:24,998][105620] Updated weights for policy 1, policy_version 889838 (0.0006) [2023-12-26 21:50:25,063][105620] Updated weights for policy 1, policy_version 889848 (0.0010) [2023-12-26 21:50:25,380][105692] Updated weights for policy 0, policy_version 889878 (0.0008) [2023-12-26 21:50:25,435][105692] Updated weights for policy 0, policy_version 889888 (0.0012) [2023-12-26 21:50:25,497][105692] Updated weights for policy 0, policy_version 889898 (0.0008) [2023-12-26 21:50:25,698][105620] Updated weights for policy 1, policy_version 889858 (0.0006) [2023-12-26 21:50:25,759][105620] Updated weights for policy 1, policy_version 889868 (0.0010) [2023-12-26 21:50:25,825][105620] Updated weights for policy 1, policy_version 889878 (0.0010) [2023-12-26 21:50:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 455688192. Throughput: 0: 9950.9, 1: 9742.8. Samples: 455695264. Policy #0 lag: (min: 16.0, avg: 43.7, max: 48.0) [2023-12-26 21:50:26,062][104569] Avg episode reward: [(0, '9088.093'), (1, '8557.808')] [2023-12-26 21:50:26,082][105692] Updated weights for policy 0, policy_version 889908 (0.0006) [2023-12-26 21:50:26,137][105692] Updated weights for policy 0, policy_version 889918 (0.0005) [2023-12-26 21:50:26,190][105692] Updated weights for policy 0, policy_version 889928 (0.0009) [2023-12-26 21:50:26,662][105620] Updated weights for policy 1, policy_version 889888 (0.0009) [2023-12-26 21:50:26,718][105620] Updated weights for policy 1, policy_version 889898 (0.0009) [2023-12-26 21:50:26,753][105692] Updated weights for policy 0, policy_version 889938 (0.0008) [2023-12-26 21:50:26,772][105620] Updated weights for policy 1, policy_version 889908 (0.0009) [2023-12-26 21:50:26,811][105692] Updated weights for policy 0, policy_version 889948 (0.0008) [2023-12-26 21:50:26,869][105692] Updated weights for policy 0, policy_version 889958 (0.0010) [2023-12-26 21:50:26,920][105692] Updated weights for policy 0, policy_version 889968 (0.0010) [2023-12-26 21:50:27,531][105620] Updated weights for policy 1, policy_version 889918 (0.0006) [2023-12-26 21:50:27,537][105692] Updated weights for policy 0, policy_version 889978 (0.0010) [2023-12-26 21:50:27,581][105692] Updated weights for policy 0, policy_version 889988 (0.0010) [2023-12-26 21:50:27,588][105620] Updated weights for policy 1, policy_version 889928 (0.0007) [2023-12-26 21:50:27,640][105692] Updated weights for policy 0, policy_version 889998 (0.0007) [2023-12-26 21:50:27,651][105620] Updated weights for policy 1, policy_version 889938 (0.0007) [2023-12-26 21:50:28,196][105692] Updated weights for policy 0, policy_version 890008 (0.0005) [2023-12-26 21:50:28,240][105692] Updated weights for policy 0, policy_version 890018 (0.0005) [2023-12-26 21:50:28,296][105692] Updated weights for policy 0, policy_version 890028 (0.0005) [2023-12-26 21:50:28,410][105620] Updated weights for policy 1, policy_version 889948 (0.0009) [2023-12-26 21:50:28,469][105620] Updated weights for policy 1, policy_version 889958 (0.0008) [2023-12-26 21:50:28,517][105620] Updated weights for policy 1, policy_version 889968 (0.0008) [2023-12-26 21:50:28,955][105692] Updated weights for policy 0, policy_version 890038 (0.0005) [2023-12-26 21:50:29,002][105692] Updated weights for policy 0, policy_version 890048 (0.0005) [2023-12-26 21:50:29,053][105692] Updated weights for policy 0, policy_version 890058 (0.0008) [2023-12-26 21:50:29,302][105620] Updated weights for policy 1, policy_version 889978 (0.0008) [2023-12-26 21:50:29,359][105620] Updated weights for policy 1, policy_version 889988 (0.0010) [2023-12-26 21:50:29,424][105620] Updated weights for policy 1, policy_version 889998 (0.0009) [2023-12-26 21:50:29,482][105620] Updated weights for policy 1, policy_version 890008 (0.0009) [2023-12-26 21:50:29,746][105692] Updated weights for policy 0, policy_version 890068 (0.0009) [2023-12-26 21:50:29,794][105692] Updated weights for policy 0, policy_version 890078 (0.0009) [2023-12-26 21:50:29,857][105692] Updated weights for policy 0, policy_version 890088 (0.0007) [2023-12-26 21:50:30,234][105620] Updated weights for policy 1, policy_version 890018 (0.0005) [2023-12-26 21:50:30,303][105620] Updated weights for policy 1, policy_version 890028 (0.0005) [2023-12-26 21:50:30,303][105586] KL-divergence is very high: 104.8083 [2023-12-26 21:50:30,349][105586] KL-divergence is very high: 107.0661 [2023-12-26 21:50:30,360][105620] Updated weights for policy 1, policy_version 890038 (0.0005) [2023-12-26 21:50:30,486][105692] Updated weights for policy 0, policy_version 890098 (0.0007) [2023-12-26 21:50:30,535][105692] Updated weights for policy 0, policy_version 890108 (0.0006) [2023-12-26 21:50:30,583][105692] Updated weights for policy 0, policy_version 890118 (0.0006) [2023-12-26 21:50:30,643][105692] Updated weights for policy 0, policy_version 890128 (0.0008) [2023-12-26 21:50:30,945][105620] Updated weights for policy 1, policy_version 890048 (0.0005) [2023-12-26 21:50:30,993][105620] Updated weights for policy 1, policy_version 890058 (0.0005) [2023-12-26 21:50:31,050][105620] Updated weights for policy 1, policy_version 890068 (0.0006) [2023-12-26 21:50:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 455786496. Throughput: 0: 10101.4, 1: 9722.0. Samples: 455757704. Policy #0 lag: (min: 16.0, avg: 43.7, max: 48.0) [2023-12-26 21:50:31,062][104569] Avg episode reward: [(0, '8815.597'), (1, '8733.504')] [2023-12-26 21:50:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000890128_227909632.pth... [2023-12-26 21:50:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000888976_227614720.pth [2023-12-26 21:50:31,078][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000890072_227885056.pth... [2023-12-26 21:50:31,083][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000888920_227590144.pth [2023-12-26 21:50:31,274][105692] Updated weights for policy 0, policy_version 890138 (0.0008) [2023-12-26 21:50:31,329][105692] Updated weights for policy 0, policy_version 890148 (0.0009) [2023-12-26 21:50:31,409][105692] Updated weights for policy 0, policy_version 890158 (0.0009) [2023-12-26 21:50:31,759][105620] Updated weights for policy 1, policy_version 890078 (0.0007) [2023-12-26 21:50:31,823][105620] Updated weights for policy 1, policy_version 890088 (0.0009) [2023-12-26 21:50:31,888][105620] Updated weights for policy 1, policy_version 890098 (0.0010) [2023-12-26 21:50:32,084][105692] Updated weights for policy 0, policy_version 890168 (0.0006) [2023-12-26 21:50:32,144][105692] Updated weights for policy 0, policy_version 890178 (0.0005) [2023-12-26 21:50:32,210][105692] Updated weights for policy 0, policy_version 890188 (0.0007) [2023-12-26 21:50:32,733][105620] Updated weights for policy 1, policy_version 890108 (0.0008) [2023-12-26 21:50:32,786][105620] Updated weights for policy 1, policy_version 890118 (0.0008) [2023-12-26 21:50:32,796][105692] Updated weights for policy 0, policy_version 890198 (0.0008) [2023-12-26 21:50:32,846][105692] Updated weights for policy 0, policy_version 890208 (0.0008) [2023-12-26 21:50:32,848][105620] Updated weights for policy 1, policy_version 890128 (0.0008) [2023-12-26 21:50:32,895][105692] Updated weights for policy 0, policy_version 890218 (0.0008) [2023-12-26 21:50:33,431][105620] Updated weights for policy 1, policy_version 890138 (0.0008) [2023-12-26 21:50:33,478][105620] Updated weights for policy 1, policy_version 890148 (0.0009) [2023-12-26 21:50:33,524][105620] Updated weights for policy 1, policy_version 890158 (0.0009) [2023-12-26 21:50:33,574][105620] Updated weights for policy 1, policy_version 890168 (0.0008) [2023-12-26 21:50:33,739][105692] Updated weights for policy 0, policy_version 890228 (0.0009) [2023-12-26 21:50:33,790][105692] Updated weights for policy 0, policy_version 890238 (0.0009) [2023-12-26 21:50:33,842][105692] Updated weights for policy 0, policy_version 890248 (0.0009) [2023-12-26 21:50:34,284][105620] Updated weights for policy 1, policy_version 890178 (0.0008) [2023-12-26 21:50:34,349][105620] Updated weights for policy 1, policy_version 890188 (0.0008) [2023-12-26 21:50:34,417][105620] Updated weights for policy 1, policy_version 890198 (0.0009) [2023-12-26 21:50:34,667][105692] Updated weights for policy 0, policy_version 890258 (0.0009) [2023-12-26 21:50:34,714][105692] Updated weights for policy 0, policy_version 890268 (0.0009) [2023-12-26 21:50:34,772][105692] Updated weights for policy 0, policy_version 890278 (0.0009) [2023-12-26 21:50:34,831][105692] Updated weights for policy 0, policy_version 890288 (0.0009) [2023-12-26 21:50:35,157][105620] Updated weights for policy 1, policy_version 890208 (0.0009) [2023-12-26 21:50:35,214][105620] Updated weights for policy 1, policy_version 890218 (0.0008) [2023-12-26 21:50:35,271][105620] Updated weights for policy 1, policy_version 890228 (0.0009) [2023-12-26 21:50:35,543][105692] Updated weights for policy 0, policy_version 890298 (0.0005) [2023-12-26 21:50:35,589][105692] Updated weights for policy 0, policy_version 890308 (0.0005) [2023-12-26 21:50:35,635][105692] Updated weights for policy 0, policy_version 890318 (0.0005) [2023-12-26 21:50:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 455884800. Throughput: 0: 10014.4, 1: 9664.7. Samples: 455877068. Policy #0 lag: (min: 16.0, avg: 43.7, max: 48.0) [2023-12-26 21:50:36,062][104569] Avg episode reward: [(0, '8962.170'), (1, '8900.928')] [2023-12-26 21:50:36,076][105620] Updated weights for policy 1, policy_version 890238 (0.0009) [2023-12-26 21:50:36,130][105620] Updated weights for policy 1, policy_version 890248 (0.0009) [2023-12-26 21:50:36,180][105620] Updated weights for policy 1, policy_version 890258 (0.0010) [2023-12-26 21:50:36,311][105692] Updated weights for policy 0, policy_version 890328 (0.0009) [2023-12-26 21:50:36,375][105692] Updated weights for policy 0, policy_version 890338 (0.0009) [2023-12-26 21:50:36,431][105692] Updated weights for policy 0, policy_version 890348 (0.0009) [2023-12-26 21:50:36,831][105620] Updated weights for policy 1, policy_version 890268 (0.0008) [2023-12-26 21:50:36,881][105620] Updated weights for policy 1, policy_version 890278 (0.0005) [2023-12-26 21:50:36,936][105620] Updated weights for policy 1, policy_version 890288 (0.0005) [2023-12-26 21:50:37,154][105692] Updated weights for policy 0, policy_version 890358 (0.0010) [2023-12-26 21:50:37,203][105692] Updated weights for policy 0, policy_version 890368 (0.0009) [2023-12-26 21:50:37,263][105692] Updated weights for policy 0, policy_version 890378 (0.0010) [2023-12-26 21:50:37,616][105620] Updated weights for policy 1, policy_version 890298 (0.0007) [2023-12-26 21:50:37,671][105620] Updated weights for policy 1, policy_version 890308 (0.0009) [2023-12-26 21:50:37,726][105620] Updated weights for policy 1, policy_version 890318 (0.0009) [2023-12-26 21:50:37,774][105620] Updated weights for policy 1, policy_version 890328 (0.0009) [2023-12-26 21:50:37,989][105692] Updated weights for policy 0, policy_version 890388 (0.0010) [2023-12-26 21:50:38,038][105692] Updated weights for policy 0, policy_version 890398 (0.0011) [2023-12-26 21:50:38,091][105692] Updated weights for policy 0, policy_version 890408 (0.0010) [2023-12-26 21:50:38,498][105620] Updated weights for policy 1, policy_version 890338 (0.0009) [2023-12-26 21:50:38,559][105620] Updated weights for policy 1, policy_version 890348 (0.0009) [2023-12-26 21:50:38,620][105620] Updated weights for policy 1, policy_version 890358 (0.0008) [2023-12-26 21:50:38,855][105692] Updated weights for policy 0, policy_version 890418 (0.0010) [2023-12-26 21:50:38,917][105692] Updated weights for policy 0, policy_version 890428 (0.0009) [2023-12-26 21:50:38,972][105692] Updated weights for policy 0, policy_version 890438 (0.0009) [2023-12-26 21:50:39,031][105692] Updated weights for policy 0, policy_version 890448 (0.0009) [2023-12-26 21:50:39,370][105620] Updated weights for policy 1, policy_version 890368 (0.0008) [2023-12-26 21:50:39,442][105620] Updated weights for policy 1, policy_version 890378 (0.0008) [2023-12-26 21:50:39,508][105620] Updated weights for policy 1, policy_version 890388 (0.0008) [2023-12-26 21:50:39,788][105692] Updated weights for policy 0, policy_version 890458 (0.0010) [2023-12-26 21:50:39,847][105692] Updated weights for policy 0, policy_version 890468 (0.0008) [2023-12-26 21:50:39,904][105692] Updated weights for policy 0, policy_version 890478 (0.0009) [2023-12-26 21:50:40,299][105620] Updated weights for policy 1, policy_version 890398 (0.0008) [2023-12-26 21:50:40,364][105620] Updated weights for policy 1, policy_version 890408 (0.0008) [2023-12-26 21:50:40,425][105620] Updated weights for policy 1, policy_version 890418 (0.0006) [2023-12-26 21:50:40,702][105692] Updated weights for policy 0, policy_version 890488 (0.0008) [2023-12-26 21:50:40,758][105692] Updated weights for policy 0, policy_version 890498 (0.0009) [2023-12-26 21:50:40,817][105692] Updated weights for policy 0, policy_version 890508 (0.0009) [2023-12-26 21:50:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 455983104. Throughput: 0: 9980.7, 1: 9652.5. Samples: 455991652. Policy #0 lag: (min: 16.0, avg: 43.7, max: 48.0) [2023-12-26 21:50:41,062][104569] Avg episode reward: [(0, '8988.406'), (1, '8716.448')] [2023-12-26 21:50:41,154][105620] Updated weights for policy 1, policy_version 890428 (0.0008) [2023-12-26 21:50:41,216][105620] Updated weights for policy 1, policy_version 890438 (0.0009) [2023-12-26 21:50:41,275][105620] Updated weights for policy 1, policy_version 890448 (0.0009) [2023-12-26 21:50:41,620][105692] Updated weights for policy 0, policy_version 890518 (0.0008) [2023-12-26 21:50:41,690][105692] Updated weights for policy 0, policy_version 890528 (0.0009) [2023-12-26 21:50:41,755][105692] Updated weights for policy 0, policy_version 890538 (0.0009) [2023-12-26 21:50:42,064][105620] Updated weights for policy 1, policy_version 890458 (0.0009) [2023-12-26 21:50:42,117][105620] Updated weights for policy 1, policy_version 890468 (0.0010) [2023-12-26 21:50:42,169][105620] Updated weights for policy 1, policy_version 890478 (0.0009) [2023-12-26 21:50:42,236][105620] Updated weights for policy 1, policy_version 890488 (0.0010) [2023-12-26 21:50:42,407][105692] Updated weights for policy 0, policy_version 890548 (0.0008) [2023-12-26 21:50:42,469][105692] Updated weights for policy 0, policy_version 890558 (0.0006) [2023-12-26 21:50:42,522][105692] Updated weights for policy 0, policy_version 890568 (0.0009) [2023-12-26 21:50:43,083][105620] Updated weights for policy 1, policy_version 890498 (0.0010) [2023-12-26 21:50:43,133][105620] Updated weights for policy 1, policy_version 890508 (0.0006) [2023-12-26 21:50:43,192][105620] Updated weights for policy 1, policy_version 890518 (0.0008) [2023-12-26 21:50:43,258][105692] Updated weights for policy 0, policy_version 890578 (0.0009) [2023-12-26 21:50:43,306][105692] Updated weights for policy 0, policy_version 890588 (0.0008) [2023-12-26 21:50:43,327][105585] KL-divergence is very high: 121.9498 [2023-12-26 21:50:43,333][105585] KL-divergence is very high: 107.0851 [2023-12-26 21:50:43,343][105585] KL-divergence is very high: 109.0125 [2023-12-26 21:50:43,350][105585] KL-divergence is very high: 171.0013 [2023-12-26 21:50:43,360][105692] Updated weights for policy 0, policy_version 890598 (0.0007) [2023-12-26 21:50:43,403][105585] KL-divergence is very high: 149.9692 [2023-12-26 21:50:43,411][105692] Updated weights for policy 0, policy_version 890608 (0.0009) [2023-12-26 21:50:43,907][105620] Updated weights for policy 1, policy_version 890528 (0.0010) [2023-12-26 21:50:43,963][105620] Updated weights for policy 1, policy_version 890538 (0.0010) [2023-12-26 21:50:44,015][105620] Updated weights for policy 1, policy_version 890548 (0.0010) [2023-12-26 21:50:44,102][105585] KL-divergence is very high: 318.2085 [2023-12-26 21:50:44,125][105585] KL-divergence is very high: 110.5711 [2023-12-26 21:50:44,148][105692] Updated weights for policy 0, policy_version 890618 (0.0007) [2023-12-26 21:50:44,149][105585] KL-divergence is very high: 616.7704 [2023-12-26 21:50:44,174][105585] KL-divergence is very high: 211.2134 [2023-12-26 21:50:44,197][105585] KL-divergence is very high: 665.6569 [2023-12-26 21:50:44,207][105692] Updated weights for policy 0, policy_version 890628 (0.0008) [2023-12-26 21:50:44,221][105585] KL-divergence is very high: 230.4424 [2023-12-26 21:50:44,245][105585] KL-divergence is very high: 625.1046 [2023-12-26 21:50:44,267][105692] Updated weights for policy 0, policy_version 890638 (0.0008) [2023-12-26 21:50:44,269][105585] KL-divergence is very high: 193.3488 [2023-12-26 21:50:44,707][105620] Updated weights for policy 1, policy_version 890558 (0.0007) [2023-12-26 21:50:44,774][105620] Updated weights for policy 1, policy_version 890568 (0.0009) [2023-12-26 21:50:44,832][105620] Updated weights for policy 1, policy_version 890578 (0.0008) [2023-12-26 21:50:44,959][105692] Updated weights for policy 0, policy_version 890648 (0.0008) [2023-12-26 21:50:45,015][105692] Updated weights for policy 0, policy_version 890658 (0.0008) [2023-12-26 21:50:45,069][105692] Updated weights for policy 0, policy_version 890668 (0.0008) [2023-12-26 21:50:45,574][105620] Updated weights for policy 1, policy_version 890588 (0.0009) [2023-12-26 21:50:45,630][105620] Updated weights for policy 1, policy_version 890598 (0.0009) [2023-12-26 21:50:45,685][105620] Updated weights for policy 1, policy_version 890608 (0.0009) [2023-12-26 21:50:45,688][105692] Updated weights for policy 0, policy_version 890678 (0.0008) [2023-12-26 21:50:45,743][105692] Updated weights for policy 0, policy_version 890688 (0.0008) [2023-12-26 21:50:45,802][105692] Updated weights for policy 0, policy_version 890698 (0.0009) [2023-12-26 21:50:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 456081408. Throughput: 0: 9871.1, 1: 9549.4. Samples: 456046280. Policy #0 lag: (min: 16.0, avg: 43.7, max: 48.0) [2023-12-26 21:50:46,063][104569] Avg episode reward: [(0, '6865.201'), (1, '8352.992')] [2023-12-26 21:50:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000890616_228024320.pth... [2023-12-26 21:50:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000890704_228057088.pth... [2023-12-26 21:50:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000889496_227737600.pth [2023-12-26 21:50:46,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000889520_227753984.pth [2023-12-26 21:50:46,498][105620] Updated weights for policy 1, policy_version 890618 (0.0008) [2023-12-26 21:50:46,502][105692] Updated weights for policy 0, policy_version 890708 (0.0007) [2023-12-26 21:50:46,559][105620] Updated weights for policy 1, policy_version 890628 (0.0009) [2023-12-26 21:50:46,563][105692] Updated weights for policy 0, policy_version 890718 (0.0005) [2023-12-26 21:50:46,608][105620] Updated weights for policy 1, policy_version 890638 (0.0009) [2023-12-26 21:50:46,618][105692] Updated weights for policy 0, policy_version 890728 (0.0005) [2023-12-26 21:50:46,659][105620] Updated weights for policy 1, policy_version 890648 (0.0009) [2023-12-26 21:50:47,281][105692] Updated weights for policy 0, policy_version 890738 (0.0006) [2023-12-26 21:50:47,331][105692] Updated weights for policy 0, policy_version 890748 (0.0009) [2023-12-26 21:50:47,393][105692] Updated weights for policy 0, policy_version 890758 (0.0009) [2023-12-26 21:50:47,443][105620] Updated weights for policy 1, policy_version 890658 (0.0006) [2023-12-26 21:50:47,455][105692] Updated weights for policy 0, policy_version 890768 (0.0008) [2023-12-26 21:50:47,493][105620] Updated weights for policy 1, policy_version 890668 (0.0009) [2023-12-26 21:50:47,547][105620] Updated weights for policy 1, policy_version 890678 (0.0008) [2023-12-26 21:50:48,157][105692] Updated weights for policy 0, policy_version 890778 (0.0005) [2023-12-26 21:50:48,215][105692] Updated weights for policy 0, policy_version 890788 (0.0005) [2023-12-26 21:50:48,242][105620] Updated weights for policy 1, policy_version 890688 (0.0008) [2023-12-26 21:50:48,273][105692] Updated weights for policy 0, policy_version 890798 (0.0006) [2023-12-26 21:50:48,292][105620] Updated weights for policy 1, policy_version 890698 (0.0007) [2023-12-26 21:50:48,358][105620] Updated weights for policy 1, policy_version 890708 (0.0008) [2023-12-26 21:50:48,926][105692] Updated weights for policy 0, policy_version 890808 (0.0006) [2023-12-26 21:50:48,987][105692] Updated weights for policy 0, policy_version 890818 (0.0006) [2023-12-26 21:50:49,042][105692] Updated weights for policy 0, policy_version 890828 (0.0008) [2023-12-26 21:50:49,198][105620] Updated weights for policy 1, policy_version 890718 (0.0009) [2023-12-26 21:50:49,256][105620] Updated weights for policy 1, policy_version 890728 (0.0008) [2023-12-26 21:50:49,319][105620] Updated weights for policy 1, policy_version 890738 (0.0008) [2023-12-26 21:50:49,703][105692] Updated weights for policy 0, policy_version 890838 (0.0007) [2023-12-26 21:50:49,757][105692] Updated weights for policy 0, policy_version 890848 (0.0007) [2023-12-26 21:50:49,816][105692] Updated weights for policy 0, policy_version 890858 (0.0007) [2023-12-26 21:50:50,090][105620] Updated weights for policy 1, policy_version 890748 (0.0008) [2023-12-26 21:50:50,142][105620] Updated weights for policy 1, policy_version 890758 (0.0008) [2023-12-26 21:50:50,197][105620] Updated weights for policy 1, policy_version 890768 (0.0009) [2023-12-26 21:50:50,506][105692] Updated weights for policy 0, policy_version 890868 (0.0008) [2023-12-26 21:50:50,561][105692] Updated weights for policy 0, policy_version 890878 (0.0009) [2023-12-26 21:50:50,617][105692] Updated weights for policy 0, policy_version 890888 (0.0009) [2023-12-26 21:50:51,019][105620] Updated weights for policy 1, policy_version 890778 (0.0009) [2023-12-26 21:50:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 456171520. Throughput: 0: 9881.5, 1: 9488.1. Samples: 456163712. Policy #0 lag: (min: 16.0, avg: 43.7, max: 48.0) [2023-12-26 21:50:51,063][104569] Avg episode reward: [(0, '7965.090'), (1, '8897.475')] [2023-12-26 21:50:51,084][105620] Updated weights for policy 1, policy_version 890788 (0.0009) [2023-12-26 21:50:51,141][105620] Updated weights for policy 1, policy_version 890798 (0.0008) [2023-12-26 21:50:51,197][105620] Updated weights for policy 1, policy_version 890808 (0.0008) [2023-12-26 21:50:51,420][105692] Updated weights for policy 0, policy_version 890898 (0.0008) [2023-12-26 21:50:51,476][105692] Updated weights for policy 0, policy_version 890908 (0.0008) [2023-12-26 21:50:51,524][105692] Updated weights for policy 0, policy_version 890918 (0.0008) [2023-12-26 21:50:51,574][105692] Updated weights for policy 0, policy_version 890928 (0.0009) [2023-12-26 21:50:51,929][105620] Updated weights for policy 1, policy_version 890818 (0.0009) [2023-12-26 21:50:51,977][105620] Updated weights for policy 1, policy_version 890828 (0.0007) [2023-12-26 21:50:52,034][105620] Updated weights for policy 1, policy_version 890838 (0.0008) [2023-12-26 21:50:52,398][105692] Updated weights for policy 0, policy_version 890938 (0.0010) [2023-12-26 21:50:52,457][105692] Updated weights for policy 0, policy_version 890949 (0.0009) [2023-12-26 21:50:52,520][105692] Updated weights for policy 0, policy_version 890959 (0.0009) [2023-12-26 21:50:52,796][105620] Updated weights for policy 1, policy_version 890848 (0.0010) [2023-12-26 21:50:52,859][105620] Updated weights for policy 1, policy_version 890858 (0.0009) [2023-12-26 21:50:52,917][105620] Updated weights for policy 1, policy_version 890868 (0.0007) [2023-12-26 21:50:53,259][105692] Updated weights for policy 0, policy_version 890969 (0.0009) [2023-12-26 21:50:53,306][105692] Updated weights for policy 0, policy_version 890979 (0.0008) [2023-12-26 21:50:53,354][105692] Updated weights for policy 0, policy_version 890989 (0.0005) [2023-12-26 21:50:53,681][105620] Updated weights for policy 1, policy_version 890878 (0.0010) [2023-12-26 21:50:53,739][105620] Updated weights for policy 1, policy_version 890888 (0.0009) [2023-12-26 21:50:53,787][105620] Updated weights for policy 1, policy_version 890898 (0.0009) [2023-12-26 21:50:54,062][105692] Updated weights for policy 0, policy_version 890999 (0.0007) [2023-12-26 21:50:54,121][105692] Updated weights for policy 0, policy_version 891009 (0.0009) [2023-12-26 21:50:54,168][105692] Updated weights for policy 0, policy_version 891019 (0.0009) [2023-12-26 21:50:54,555][105620] Updated weights for policy 1, policy_version 890908 (0.0009) [2023-12-26 21:50:54,609][105620] Updated weights for policy 1, policy_version 890918 (0.0010) [2023-12-26 21:50:54,667][105620] Updated weights for policy 1, policy_version 890929 (0.0010) [2023-12-26 21:50:54,878][105692] Updated weights for policy 0, policy_version 891029 (0.0009) [2023-12-26 21:50:54,939][105692] Updated weights for policy 0, policy_version 891039 (0.0009) [2023-12-26 21:50:54,999][105692] Updated weights for policy 0, policy_version 891049 (0.0008) [2023-12-26 21:50:55,448][105620] Updated weights for policy 1, policy_version 890939 (0.0008) [2023-12-26 21:50:55,496][105620] Updated weights for policy 1, policy_version 890949 (0.0009) [2023-12-26 21:50:55,550][105620] Updated weights for policy 1, policy_version 890959 (0.0008) [2023-12-26 21:50:55,698][105692] Updated weights for policy 0, policy_version 891059 (0.0006) [2023-12-26 21:50:55,751][105692] Updated weights for policy 0, policy_version 891069 (0.0005) [2023-12-26 21:50:55,796][105692] Updated weights for policy 0, policy_version 891079 (0.0006) [2023-12-26 21:50:56,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 456269824. Throughput: 0: 9910.6, 1: 9368.2. Samples: 456275236. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 21:50:56,062][104569] Avg episode reward: [(0, '8988.863'), (1, '8992.063')] [2023-12-26 21:50:56,336][105620] Updated weights for policy 1, policy_version 890969 (0.0010) [2023-12-26 21:50:56,399][105620] Updated weights for policy 1, policy_version 890979 (0.0010) [2023-12-26 21:50:56,457][105620] Updated weights for policy 1, policy_version 890989 (0.0010) [2023-12-26 21:50:56,459][105692] Updated weights for policy 0, policy_version 891089 (0.0008) [2023-12-26 21:50:56,516][105692] Updated weights for policy 0, policy_version 891099 (0.0006) [2023-12-26 21:50:56,522][105620] Updated weights for policy 1, policy_version 890999 (0.0009) [2023-12-26 21:50:56,569][105692] Updated weights for policy 0, policy_version 891109 (0.0008) [2023-12-26 21:50:56,632][105692] Updated weights for policy 0, policy_version 891119 (0.0009) [2023-12-26 21:50:57,294][105620] Updated weights for policy 1, policy_version 891009 (0.0009) [2023-12-26 21:50:57,353][105620] Updated weights for policy 1, policy_version 891019 (0.0006) [2023-12-26 21:50:57,355][105692] Updated weights for policy 0, policy_version 891129 (0.0010) [2023-12-26 21:50:57,403][105692] Updated weights for policy 0, policy_version 891139 (0.0007) [2023-12-26 21:50:57,405][105620] Updated weights for policy 1, policy_version 891029 (0.0006) [2023-12-26 21:50:57,459][105692] Updated weights for policy 0, policy_version 891149 (0.0008) [2023-12-26 21:50:58,112][105620] Updated weights for policy 1, policy_version 891039 (0.0005) [2023-12-26 21:50:58,181][105620] Updated weights for policy 1, policy_version 891049 (0.0009) [2023-12-26 21:50:58,193][105692] Updated weights for policy 0, policy_version 891159 (0.0008) [2023-12-26 21:50:58,240][105620] Updated weights for policy 1, policy_version 891059 (0.0009) [2023-12-26 21:50:58,260][105692] Updated weights for policy 0, policy_version 891169 (0.0008) [2023-12-26 21:50:58,322][105692] Updated weights for policy 0, policy_version 891179 (0.0008) [2023-12-26 21:50:59,067][105620] Updated weights for policy 1, policy_version 891069 (0.0009) [2023-12-26 21:50:59,090][105692] Updated weights for policy 0, policy_version 891189 (0.0008) [2023-12-26 21:50:59,127][105620] Updated weights for policy 1, policy_version 891079 (0.0008) [2023-12-26 21:50:59,150][105692] Updated weights for policy 0, policy_version 891199 (0.0007) [2023-12-26 21:50:59,195][105620] Updated weights for policy 1, policy_version 891089 (0.0008) [2023-12-26 21:50:59,210][105692] Updated weights for policy 0, policy_version 891209 (0.0007) [2023-12-26 21:50:59,883][105620] Updated weights for policy 1, policy_version 891099 (0.0009) [2023-12-26 21:50:59,907][105692] Updated weights for policy 0, policy_version 891219 (0.0007) [2023-12-26 21:50:59,946][105620] Updated weights for policy 1, policy_version 891109 (0.0009) [2023-12-26 21:50:59,991][105692] Updated weights for policy 0, policy_version 891229 (0.0008) [2023-12-26 21:51:00,002][105620] Updated weights for policy 1, policy_version 891119 (0.0008) [2023-12-26 21:51:00,057][105692] Updated weights for policy 0, policy_version 891239 (0.0006) [2023-12-26 21:51:00,590][105692] Updated weights for policy 0, policy_version 891249 (0.0005) [2023-12-26 21:51:00,666][105692] Updated weights for policy 0, policy_version 891259 (0.0005) [2023-12-26 21:51:00,688][105620] Updated weights for policy 1, policy_version 891129 (0.0008) [2023-12-26 21:51:00,725][105692] Updated weights for policy 0, policy_version 891269 (0.0005) [2023-12-26 21:51:00,753][105620] Updated weights for policy 1, policy_version 891139 (0.0005) [2023-12-26 21:51:00,788][105692] Updated weights for policy 0, policy_version 891279 (0.0005) [2023-12-26 21:51:00,816][105620] Updated weights for policy 1, policy_version 891149 (0.0008) [2023-12-26 21:51:00,875][105620] Updated weights for policy 1, policy_version 891159 (0.0009) [2023-12-26 21:51:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 456368128. Throughput: 0: 9932.8, 1: 9378.6. Samples: 456332596. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 21:51:01,062][104569] Avg episode reward: [(0, '9078.063'), (1, '8989.416')] [2023-12-26 21:51:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000891280_228204544.pth... [2023-12-26 21:51:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000891160_228163584.pth... [2023-12-26 21:51:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000890128_227909632.pth [2023-12-26 21:51:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000890072_227885056.pth [2023-12-26 21:51:01,327][105692] Updated weights for policy 0, policy_version 891289 (0.0010) [2023-12-26 21:51:01,391][105692] Updated weights for policy 0, policy_version 891299 (0.0010) [2023-12-26 21:51:01,440][105692] Updated weights for policy 0, policy_version 891309 (0.0010) [2023-12-26 21:51:01,567][105620] Updated weights for policy 1, policy_version 891169 (0.0007) [2023-12-26 21:51:01,627][105620] Updated weights for policy 1, policy_version 891179 (0.0007) [2023-12-26 21:51:01,688][105620] Updated weights for policy 1, policy_version 891189 (0.0008) [2023-12-26 21:51:02,207][105692] Updated weights for policy 0, policy_version 891319 (0.0011) [2023-12-26 21:51:02,271][105692] Updated weights for policy 0, policy_version 891329 (0.0011) [2023-12-26 21:51:02,341][105692] Updated weights for policy 0, policy_version 891339 (0.0011) [2023-12-26 21:51:02,429][105620] Updated weights for policy 1, policy_version 891199 (0.0007) [2023-12-26 21:51:02,485][105620] Updated weights for policy 1, policy_version 891209 (0.0008) [2023-12-26 21:51:02,537][105620] Updated weights for policy 1, policy_version 891219 (0.0008) [2023-12-26 21:51:03,051][105692] Updated weights for policy 0, policy_version 891349 (0.0010) [2023-12-26 21:51:03,102][105692] Updated weights for policy 0, policy_version 891359 (0.0010) [2023-12-26 21:51:03,153][105692] Updated weights for policy 0, policy_version 891369 (0.0010) [2023-12-26 21:51:03,188][105620] Updated weights for policy 1, policy_version 891229 (0.0007) [2023-12-26 21:51:03,247][105620] Updated weights for policy 1, policy_version 891239 (0.0005) [2023-12-26 21:51:03,296][105620] Updated weights for policy 1, policy_version 891249 (0.0005) [2023-12-26 21:51:03,816][105620] Updated weights for policy 1, policy_version 891259 (0.0005) [2023-12-26 21:51:03,852][105692] Updated weights for policy 0, policy_version 891379 (0.0010) [2023-12-26 21:51:03,883][105620] Updated weights for policy 1, policy_version 891269 (0.0008) [2023-12-26 21:51:03,910][105692] Updated weights for policy 0, policy_version 891389 (0.0011) [2023-12-26 21:51:03,945][105620] Updated weights for policy 1, policy_version 891279 (0.0008) [2023-12-26 21:51:03,963][105692] Updated weights for policy 0, policy_version 891399 (0.0008) [2023-12-26 21:51:04,684][105692] Updated weights for policy 0, policy_version 891409 (0.0007) [2023-12-26 21:51:04,687][105620] Updated weights for policy 1, policy_version 891289 (0.0008) [2023-12-26 21:51:04,733][105692] Updated weights for policy 0, policy_version 891419 (0.0010) [2023-12-26 21:51:04,747][105620] Updated weights for policy 1, policy_version 891299 (0.0011) [2023-12-26 21:51:04,786][105692] Updated weights for policy 0, policy_version 891429 (0.0011) [2023-12-26 21:51:04,803][105620] Updated weights for policy 1, policy_version 891309 (0.0011) [2023-12-26 21:51:04,846][105692] Updated weights for policy 0, policy_version 891439 (0.0011) [2023-12-26 21:51:04,857][105620] Updated weights for policy 1, policy_version 891319 (0.0011) [2023-12-26 21:51:05,582][105692] Updated weights for policy 0, policy_version 891449 (0.0008) [2023-12-26 21:51:05,620][105620] Updated weights for policy 1, policy_version 891329 (0.0011) [2023-12-26 21:51:05,638][105692] Updated weights for policy 0, policy_version 891459 (0.0005) [2023-12-26 21:51:05,671][105620] Updated weights for policy 1, policy_version 891339 (0.0010) [2023-12-26 21:51:05,692][105692] Updated weights for policy 0, policy_version 891469 (0.0005) [2023-12-26 21:51:05,722][105620] Updated weights for policy 1, policy_version 891349 (0.0010) [2023-12-26 21:51:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.6, 300 sec: 19605.2). Total num frames: 456466432. Throughput: 0: 10044.7, 1: 9422.2. Samples: 456453696. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 21:51:06,063][104569] Avg episode reward: [(0, '9261.325'), (1, '8714.135')] [2023-12-26 21:51:06,445][105692] Updated weights for policy 0, policy_version 891479 (0.0007) [2023-12-26 21:51:06,487][105620] Updated weights for policy 1, policy_version 891359 (0.0010) [2023-12-26 21:51:06,506][105692] Updated weights for policy 0, policy_version 891489 (0.0006) [2023-12-26 21:51:06,552][105620] Updated weights for policy 1, policy_version 891369 (0.0009) [2023-12-26 21:51:06,570][105692] Updated weights for policy 0, policy_version 891499 (0.0005) [2023-12-26 21:51:06,623][105620] Updated weights for policy 1, policy_version 891379 (0.0009) [2023-12-26 21:51:07,202][105692] Updated weights for policy 0, policy_version 891509 (0.0005) [2023-12-26 21:51:07,255][105692] Updated weights for policy 0, policy_version 891519 (0.0007) [2023-12-26 21:51:07,311][105692] Updated weights for policy 0, policy_version 891529 (0.0008) [2023-12-26 21:51:07,373][105620] Updated weights for policy 1, policy_version 891389 (0.0010) [2023-12-26 21:51:07,436][105620] Updated weights for policy 1, policy_version 891399 (0.0010) [2023-12-26 21:51:07,494][105620] Updated weights for policy 1, policy_version 891409 (0.0010) [2023-12-26 21:51:08,086][105620] Updated weights for policy 1, policy_version 891419 (0.0009) [2023-12-26 21:51:08,123][105692] Updated weights for policy 0, policy_version 891539 (0.0008) [2023-12-26 21:51:08,137][105620] Updated weights for policy 1, policy_version 891429 (0.0005) [2023-12-26 21:51:08,176][105692] Updated weights for policy 0, policy_version 891549 (0.0005) [2023-12-26 21:51:08,196][105620] Updated weights for policy 1, policy_version 891439 (0.0005) [2023-12-26 21:51:08,229][105692] Updated weights for policy 0, policy_version 891559 (0.0005) [2023-12-26 21:51:08,819][105620] Updated weights for policy 1, policy_version 891449 (0.0009) [2023-12-26 21:51:08,879][105620] Updated weights for policy 1, policy_version 891459 (0.0009) [2023-12-26 21:51:08,939][105620] Updated weights for policy 1, policy_version 891469 (0.0010) [2023-12-26 21:51:08,999][105692] Updated weights for policy 0, policy_version 891569 (0.0005) [2023-12-26 21:51:09,001][105620] Updated weights for policy 1, policy_version 891479 (0.0009) [2023-12-26 21:51:09,059][105692] Updated weights for policy 0, policy_version 891579 (0.0009) [2023-12-26 21:51:09,117][105692] Updated weights for policy 0, policy_version 891589 (0.0009) [2023-12-26 21:51:09,176][105692] Updated weights for policy 0, policy_version 891599 (0.0009) [2023-12-26 21:51:09,751][105620] Updated weights for policy 1, policy_version 891489 (0.0007) [2023-12-26 21:51:09,808][105620] Updated weights for policy 1, policy_version 891499 (0.0009) [2023-12-26 21:51:09,873][105620] Updated weights for policy 1, policy_version 891509 (0.0009) [2023-12-26 21:51:09,965][105692] Updated weights for policy 0, policy_version 891609 (0.0009) [2023-12-26 21:51:10,037][105692] Updated weights for policy 0, policy_version 891619 (0.0008) [2023-12-26 21:51:10,101][105692] Updated weights for policy 0, policy_version 891629 (0.0007) [2023-12-26 21:51:10,668][105620] Updated weights for policy 1, policy_version 891519 (0.0009) [2023-12-26 21:51:10,723][105620] Updated weights for policy 1, policy_version 891529 (0.0008) [2023-12-26 21:51:10,749][105692] Updated weights for policy 0, policy_version 891639 (0.0007) [2023-12-26 21:51:10,776][105620] Updated weights for policy 1, policy_version 891539 (0.0006) [2023-12-26 21:51:10,806][105692] Updated weights for policy 0, policy_version 891649 (0.0007) [2023-12-26 21:51:10,863][105692] Updated weights for policy 0, policy_version 891659 (0.0008) [2023-12-26 21:51:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 456564736. Throughput: 0: 9946.8, 1: 9473.0. Samples: 456569152. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 21:51:11,063][104569] Avg episode reward: [(0, '9352.840'), (1, '8716.153')] [2023-12-26 21:51:11,572][105620] Updated weights for policy 1, policy_version 891549 (0.0008) [2023-12-26 21:51:11,578][105692] Updated weights for policy 0, policy_version 891669 (0.0007) [2023-12-26 21:51:11,634][105620] Updated weights for policy 1, policy_version 891559 (0.0008) [2023-12-26 21:51:11,644][105692] Updated weights for policy 0, policy_version 891679 (0.0008) [2023-12-26 21:51:11,702][105692] Updated weights for policy 0, policy_version 891689 (0.0008) [2023-12-26 21:51:11,702][105620] Updated weights for policy 1, policy_version 891569 (0.0009) [2023-12-26 21:51:12,496][105620] Updated weights for policy 1, policy_version 891579 (0.0008) [2023-12-26 21:51:12,523][105692] Updated weights for policy 0, policy_version 891699 (0.0010) [2023-12-26 21:51:12,546][105620] Updated weights for policy 1, policy_version 891589 (0.0006) [2023-12-26 21:51:12,586][105692] Updated weights for policy 0, policy_version 891709 (0.0008) [2023-12-26 21:51:12,601][105620] Updated weights for policy 1, policy_version 891599 (0.0008) [2023-12-26 21:51:12,644][105692] Updated weights for policy 0, policy_version 891719 (0.0006) [2023-12-26 21:51:13,216][105620] Updated weights for policy 1, policy_version 891609 (0.0007) [2023-12-26 21:51:13,230][105692] Updated weights for policy 0, policy_version 891729 (0.0007) [2023-12-26 21:51:13,280][105620] Updated weights for policy 1, policy_version 891619 (0.0005) [2023-12-26 21:51:13,287][105692] Updated weights for policy 0, policy_version 891739 (0.0008) [2023-12-26 21:51:13,340][105620] Updated weights for policy 1, policy_version 891629 (0.0009) [2023-12-26 21:51:13,342][105692] Updated weights for policy 0, policy_version 891749 (0.0006) [2023-12-26 21:51:13,391][105620] Updated weights for policy 1, policy_version 891639 (0.0005) [2023-12-26 21:51:13,400][105692] Updated weights for policy 0, policy_version 891759 (0.0008) [2023-12-26 21:51:14,060][105692] Updated weights for policy 0, policy_version 891769 (0.0009) [2023-12-26 21:51:14,103][105620] Updated weights for policy 1, policy_version 891649 (0.0008) [2023-12-26 21:51:14,114][105692] Updated weights for policy 0, policy_version 891779 (0.0006) [2023-12-26 21:51:14,167][105620] Updated weights for policy 1, policy_version 891659 (0.0009) [2023-12-26 21:51:14,174][105692] Updated weights for policy 0, policy_version 891789 (0.0006) [2023-12-26 21:51:14,216][105620] Updated weights for policy 1, policy_version 891669 (0.0008) [2023-12-26 21:51:14,933][105692] Updated weights for policy 0, policy_version 891799 (0.0008) [2023-12-26 21:51:14,952][105620] Updated weights for policy 1, policy_version 891679 (0.0007) [2023-12-26 21:51:14,983][105692] Updated weights for policy 0, policy_version 891809 (0.0006) [2023-12-26 21:51:15,016][105620] Updated weights for policy 1, policy_version 891689 (0.0008) [2023-12-26 21:51:15,051][105692] Updated weights for policy 0, policy_version 891819 (0.0006) [2023-12-26 21:51:15,070][105620] Updated weights for policy 1, policy_version 891699 (0.0006) [2023-12-26 21:51:15,707][105692] Updated weights for policy 0, policy_version 891829 (0.0008) [2023-12-26 21:51:15,718][105620] Updated weights for policy 1, policy_version 891709 (0.0005) [2023-12-26 21:51:15,762][105692] Updated weights for policy 0, policy_version 891839 (0.0007) [2023-12-26 21:51:15,779][105620] Updated weights for policy 1, policy_version 891719 (0.0009) [2023-12-26 21:51:15,811][105692] Updated weights for policy 0, policy_version 891849 (0.0006) [2023-12-26 21:51:15,832][105620] Updated weights for policy 1, policy_version 891729 (0.0008) [2023-12-26 21:51:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 456663040. Throughput: 0: 9801.7, 1: 9513.4. Samples: 456626884. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 21:51:16,063][104569] Avg episode reward: [(0, '9267.316'), (1, '8458.456')] [2023-12-26 21:51:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000891736_228311040.pth... [2023-12-26 21:51:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000891856_228352000.pth... [2023-12-26 21:51:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000890616_228024320.pth [2023-12-26 21:51:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000890704_228057088.pth [2023-12-26 21:51:16,563][105620] Updated weights for policy 1, policy_version 891739 (0.0008) [2023-12-26 21:51:16,570][105692] Updated weights for policy 0, policy_version 891859 (0.0009) [2023-12-26 21:51:16,623][105620] Updated weights for policy 1, policy_version 891749 (0.0007) [2023-12-26 21:51:16,625][105692] Updated weights for policy 0, policy_version 891869 (0.0011) [2023-12-26 21:51:16,677][105692] Updated weights for policy 0, policy_version 891879 (0.0010) [2023-12-26 21:51:16,682][105620] Updated weights for policy 1, policy_version 891759 (0.0006) [2023-12-26 21:51:17,433][105620] Updated weights for policy 1, policy_version 891769 (0.0007) [2023-12-26 21:51:17,433][105692] Updated weights for policy 0, policy_version 891889 (0.0011) [2023-12-26 21:51:17,481][105620] Updated weights for policy 1, policy_version 891779 (0.0009) [2023-12-26 21:51:17,489][105692] Updated weights for policy 0, policy_version 891899 (0.0011) [2023-12-26 21:51:17,533][105620] Updated weights for policy 1, policy_version 891789 (0.0008) [2023-12-26 21:51:17,540][105692] Updated weights for policy 0, policy_version 891909 (0.0010) [2023-12-26 21:51:17,591][105692] Updated weights for policy 0, policy_version 891919 (0.0010) [2023-12-26 21:51:17,598][105620] Updated weights for policy 1, policy_version 891799 (0.0008) [2023-12-26 21:51:18,362][105692] Updated weights for policy 0, policy_version 891929 (0.0010) [2023-12-26 21:51:18,365][105620] Updated weights for policy 1, policy_version 891809 (0.0008) [2023-12-26 21:51:18,425][105620] Updated weights for policy 1, policy_version 891819 (0.0006) [2023-12-26 21:51:18,428][105692] Updated weights for policy 0, policy_version 891939 (0.0008) [2023-12-26 21:51:18,484][105620] Updated weights for policy 1, policy_version 891829 (0.0009) [2023-12-26 21:51:18,499][105692] Updated weights for policy 0, policy_version 891949 (0.0007) [2023-12-26 21:51:19,176][105692] Updated weights for policy 0, policy_version 891959 (0.0010) [2023-12-26 21:51:19,181][105620] Updated weights for policy 1, policy_version 891839 (0.0006) [2023-12-26 21:51:19,239][105620] Updated weights for policy 1, policy_version 891849 (0.0007) [2023-12-26 21:51:19,240][105692] Updated weights for policy 0, policy_version 891969 (0.0011) [2023-12-26 21:51:19,301][105692] Updated weights for policy 0, policy_version 891979 (0.0009) [2023-12-26 21:51:19,307][105620] Updated weights for policy 1, policy_version 891859 (0.0007) [2023-12-26 21:51:20,015][105620] Updated weights for policy 1, policy_version 891869 (0.0010) [2023-12-26 21:51:20,085][105620] Updated weights for policy 1, policy_version 891879 (0.0011) [2023-12-26 21:51:20,095][105692] Updated weights for policy 0, policy_version 891989 (0.0006) [2023-12-26 21:51:20,148][105620] Updated weights for policy 1, policy_version 891889 (0.0010) [2023-12-26 21:51:20,160][105692] Updated weights for policy 0, policy_version 891999 (0.0006) [2023-12-26 21:51:20,215][105692] Updated weights for policy 0, policy_version 892009 (0.0008) [2023-12-26 21:51:20,855][105620] Updated weights for policy 1, policy_version 891899 (0.0010) [2023-12-26 21:51:20,915][105620] Updated weights for policy 1, policy_version 891909 (0.0010) [2023-12-26 21:51:20,933][105586] KL-divergence is very high: 135.1664 [2023-12-26 21:51:20,975][105620] Updated weights for policy 1, policy_version 891919 (0.0011) [2023-12-26 21:51:20,982][105586] KL-divergence is very high: 135.3610 [2023-12-26 21:51:20,988][105692] Updated weights for policy 0, policy_version 892019 (0.0009) [2023-12-26 21:51:21,050][105692] Updated weights for policy 0, policy_version 892029 (0.0009) [2023-12-26 21:51:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 456753152. Throughput: 0: 9747.4, 1: 9499.2. Samples: 456743164. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 21:51:21,062][104569] Avg episode reward: [(0, '9267.169'), (1, '8643.048')] [2023-12-26 21:51:21,112][105692] Updated weights for policy 0, policy_version 892039 (0.0008) [2023-12-26 21:51:21,742][105620] Updated weights for policy 1, policy_version 891929 (0.0011) [2023-12-26 21:51:21,799][105620] Updated weights for policy 1, policy_version 891939 (0.0011) [2023-12-26 21:51:21,855][105620] Updated weights for policy 1, policy_version 891949 (0.0011) [2023-12-26 21:51:21,887][105692] Updated weights for policy 0, policy_version 892049 (0.0009) [2023-12-26 21:51:21,921][105620] Updated weights for policy 1, policy_version 891959 (0.0011) [2023-12-26 21:51:21,952][105692] Updated weights for policy 0, policy_version 892059 (0.0006) [2023-12-26 21:51:22,015][105692] Updated weights for policy 0, policy_version 892069 (0.0008) [2023-12-26 21:51:22,075][105692] Updated weights for policy 0, policy_version 892079 (0.0008) [2023-12-26 21:51:22,683][105692] Updated weights for policy 0, policy_version 892089 (0.0006) [2023-12-26 21:51:22,686][105620] Updated weights for policy 1, policy_version 891969 (0.0010) [2023-12-26 21:51:22,740][105692] Updated weights for policy 0, policy_version 892099 (0.0006) [2023-12-26 21:51:22,742][105620] Updated weights for policy 1, policy_version 891979 (0.0011) [2023-12-26 21:51:22,792][105692] Updated weights for policy 0, policy_version 892109 (0.0007) [2023-12-26 21:51:22,804][105620] Updated weights for policy 1, policy_version 891989 (0.0010) [2023-12-26 21:51:23,459][105692] Updated weights for policy 0, policy_version 892119 (0.0009) [2023-12-26 21:51:23,511][105692] Updated weights for policy 0, policy_version 892129 (0.0010) [2023-12-26 21:51:23,546][105620] Updated weights for policy 1, policy_version 891999 (0.0010) [2023-12-26 21:51:23,559][105692] Updated weights for policy 0, policy_version 892139 (0.0010) [2023-12-26 21:51:23,595][105620] Updated weights for policy 1, policy_version 892009 (0.0010) [2023-12-26 21:51:23,639][105620] Updated weights for policy 1, policy_version 892019 (0.0010) [2023-12-26 21:51:24,163][105692] Updated weights for policy 0, policy_version 892149 (0.0008) [2023-12-26 21:51:24,211][105692] Updated weights for policy 0, policy_version 892159 (0.0005) [2023-12-26 21:51:24,267][105692] Updated weights for policy 0, policy_version 892169 (0.0006) [2023-12-26 21:51:24,434][105620] Updated weights for policy 1, policy_version 892029 (0.0010) [2023-12-26 21:51:24,485][105620] Updated weights for policy 1, policy_version 892039 (0.0010) [2023-12-26 21:51:24,550][105620] Updated weights for policy 1, policy_version 892049 (0.0010) [2023-12-26 21:51:24,885][105692] Updated weights for policy 0, policy_version 892179 (0.0007) [2023-12-26 21:51:24,946][105692] Updated weights for policy 0, policy_version 892189 (0.0008) [2023-12-26 21:51:25,008][105692] Updated weights for policy 0, policy_version 892199 (0.0011) [2023-12-26 21:51:25,238][105620] Updated weights for policy 1, policy_version 892059 (0.0009) [2023-12-26 21:51:25,298][105620] Updated weights for policy 1, policy_version 892069 (0.0007) [2023-12-26 21:51:25,360][105620] Updated weights for policy 1, policy_version 892079 (0.0010) [2023-12-26 21:51:25,730][105692] Updated weights for policy 0, policy_version 892209 (0.0011) [2023-12-26 21:51:25,794][105692] Updated weights for policy 0, policy_version 892219 (0.0010) [2023-12-26 21:51:25,852][105692] Updated weights for policy 0, policy_version 892229 (0.0010) [2023-12-26 21:51:25,917][105692] Updated weights for policy 0, policy_version 892239 (0.0006) [2023-12-26 21:51:25,997][105620] Updated weights for policy 1, policy_version 892089 (0.0010) [2023-12-26 21:51:26,057][105620] Updated weights for policy 1, policy_version 892099 (0.0009) [2023-12-26 21:51:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 456851456. Throughput: 0: 9822.8, 1: 9498.6. Samples: 456861112. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 21:51:26,063][104569] Avg episode reward: [(0, '9266.812'), (1, '8539.603')] [2023-12-26 21:51:26,116][105620] Updated weights for policy 1, policy_version 892109 (0.0010) [2023-12-26 21:51:26,175][105620] Updated weights for policy 1, policy_version 892119 (0.0009) [2023-12-26 21:51:26,533][105692] Updated weights for policy 0, policy_version 892249 (0.0005) [2023-12-26 21:51:26,598][105692] Updated weights for policy 0, policy_version 892259 (0.0007) [2023-12-26 21:51:26,656][105692] Updated weights for policy 0, policy_version 892269 (0.0008) [2023-12-26 21:51:26,912][105620] Updated weights for policy 1, policy_version 892129 (0.0010) [2023-12-26 21:51:26,965][105620] Updated weights for policy 1, policy_version 892139 (0.0010) [2023-12-26 21:51:27,018][105620] Updated weights for policy 1, policy_version 892149 (0.0010) [2023-12-26 21:51:27,195][105692] Updated weights for policy 0, policy_version 892279 (0.0006) [2023-12-26 21:51:27,247][105692] Updated weights for policy 0, policy_version 892289 (0.0008) [2023-12-26 21:51:27,303][105692] Updated weights for policy 0, policy_version 892299 (0.0006) [2023-12-26 21:51:27,727][105620] Updated weights for policy 1, policy_version 892159 (0.0010) [2023-12-26 21:51:27,781][105620] Updated weights for policy 1, policy_version 892169 (0.0010) [2023-12-26 21:51:27,824][105620] Updated weights for policy 1, policy_version 892179 (0.0010) [2023-12-26 21:51:27,989][105692] Updated weights for policy 0, policy_version 892309 (0.0007) [2023-12-26 21:51:28,041][105692] Updated weights for policy 0, policy_version 892319 (0.0008) [2023-12-26 21:51:28,087][105692] Updated weights for policy 0, policy_version 892329 (0.0008) [2023-12-26 21:51:28,483][105620] Updated weights for policy 1, policy_version 892189 (0.0009) [2023-12-26 21:51:28,548][105620] Updated weights for policy 1, policy_version 892199 (0.0005) [2023-12-26 21:51:28,615][105620] Updated weights for policy 1, policy_version 892209 (0.0005) [2023-12-26 21:51:28,823][105692] Updated weights for policy 0, policy_version 892339 (0.0010) [2023-12-26 21:51:28,881][105692] Updated weights for policy 0, policy_version 892349 (0.0011) [2023-12-26 21:51:28,943][105692] Updated weights for policy 0, policy_version 892359 (0.0010) [2023-12-26 21:51:29,291][105620] Updated weights for policy 1, policy_version 892219 (0.0009) [2023-12-26 21:51:29,356][105620] Updated weights for policy 1, policy_version 892229 (0.0010) [2023-12-26 21:51:29,420][105620] Updated weights for policy 1, policy_version 892239 (0.0011) [2023-12-26 21:51:29,629][105692] Updated weights for policy 0, policy_version 892369 (0.0010) [2023-12-26 21:51:29,689][105692] Updated weights for policy 0, policy_version 892379 (0.0008) [2023-12-26 21:51:29,744][105692] Updated weights for policy 0, policy_version 892389 (0.0008) [2023-12-26 21:51:29,807][105692] Updated weights for policy 0, policy_version 892399 (0.0008) [2023-12-26 21:51:30,212][105620] Updated weights for policy 1, policy_version 892249 (0.0010) [2023-12-26 21:51:30,276][105620] Updated weights for policy 1, policy_version 892259 (0.0006) [2023-12-26 21:51:30,331][105620] Updated weights for policy 1, policy_version 892269 (0.0005) [2023-12-26 21:51:30,388][105620] Updated weights for policy 1, policy_version 892279 (0.0005) [2023-12-26 21:51:30,563][105692] Updated weights for policy 0, policy_version 892410 (0.0010) [2023-12-26 21:51:30,624][105692] Updated weights for policy 0, policy_version 892420 (0.0010) [2023-12-26 21:51:30,681][105692] Updated weights for policy 0, policy_version 892431 (0.0010) [2023-12-26 21:51:30,899][105620] Updated weights for policy 1, policy_version 892289 (0.0009) [2023-12-26 21:51:30,953][105620] Updated weights for policy 1, policy_version 892299 (0.0009) [2023-12-26 21:51:31,009][105620] Updated weights for policy 1, policy_version 892310 (0.0009) [2023-12-26 21:51:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 456957952. Throughput: 0: 9907.6, 1: 9574.2. Samples: 456922956. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 21:51:31,062][104569] Avg episode reward: [(0, '7109.732'), (1, '8262.095')] [2023-12-26 21:51:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000892312_228458496.pth... [2023-12-26 21:51:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000892432_228499456.pth... [2023-12-26 21:51:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000891160_228163584.pth [2023-12-26 21:51:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000891280_228204544.pth [2023-12-26 21:51:31,431][105692] Updated weights for policy 0, policy_version 892441 (0.0007) [2023-12-26 21:51:31,479][105585] KL-divergence is very high: 120.6413 [2023-12-26 21:51:31,490][105585] KL-divergence is very high: 119.7916 [2023-12-26 21:51:31,495][105692] Updated weights for policy 0, policy_version 892451 (0.0006) [2023-12-26 21:51:31,507][105585] KL-divergence is very high: 145.3313 [2023-12-26 21:51:31,521][105585] KL-divergence is very high: 148.5124 [2023-12-26 21:51:31,531][105585] KL-divergence is very high: 119.0040 [2023-12-26 21:51:31,543][105692] Updated weights for policy 0, policy_version 892461 (0.0006) [2023-12-26 21:51:31,545][105585] KL-divergence is very high: 108.6938 [2023-12-26 21:51:31,730][105620] Updated weights for policy 1, policy_version 892320 (0.0008) [2023-12-26 21:51:31,794][105620] Updated weights for policy 1, policy_version 892330 (0.0007) [2023-12-26 21:51:31,852][105620] Updated weights for policy 1, policy_version 892340 (0.0005) [2023-12-26 21:51:32,289][105692] Updated weights for policy 0, policy_version 892471 (0.0009) [2023-12-26 21:51:32,343][105692] Updated weights for policy 0, policy_version 892481 (0.0008) [2023-12-26 21:51:32,407][105692] Updated weights for policy 0, policy_version 892491 (0.0007) [2023-12-26 21:51:32,517][105620] Updated weights for policy 1, policy_version 892350 (0.0008) [2023-12-26 21:51:32,578][105620] Updated weights for policy 1, policy_version 892360 (0.0009) [2023-12-26 21:51:32,640][105620] Updated weights for policy 1, policy_version 892370 (0.0008) [2023-12-26 21:51:33,177][105692] Updated weights for policy 0, policy_version 892501 (0.0008) [2023-12-26 21:51:33,224][105692] Updated weights for policy 0, policy_version 892511 (0.0007) [2023-12-26 21:51:33,279][105692] Updated weights for policy 0, policy_version 892521 (0.0008) [2023-12-26 21:51:33,344][105620] Updated weights for policy 1, policy_version 892380 (0.0010) [2023-12-26 21:51:33,402][105620] Updated weights for policy 1, policy_version 892390 (0.0010) [2023-12-26 21:51:33,450][105620] Updated weights for policy 1, policy_version 892400 (0.0010) [2023-12-26 21:51:34,051][105692] Updated weights for policy 0, policy_version 892531 (0.0008) [2023-12-26 21:51:34,106][105692] Updated weights for policy 0, policy_version 892541 (0.0008) [2023-12-26 21:51:34,164][105692] Updated weights for policy 0, policy_version 892551 (0.0008) [2023-12-26 21:51:34,204][105620] Updated weights for policy 1, policy_version 892410 (0.0010) [2023-12-26 21:51:34,257][105620] Updated weights for policy 1, policy_version 892420 (0.0010) [2023-12-26 21:51:34,316][105620] Updated weights for policy 1, policy_version 892430 (0.0010) [2023-12-26 21:51:34,376][105620] Updated weights for policy 1, policy_version 892440 (0.0010) [2023-12-26 21:51:34,934][105692] Updated weights for policy 0, policy_version 892561 (0.0008) [2023-12-26 21:51:34,990][105692] Updated weights for policy 0, policy_version 892571 (0.0008) [2023-12-26 21:51:35,041][105692] Updated weights for policy 0, policy_version 892581 (0.0008) [2023-12-26 21:51:35,085][105620] Updated weights for policy 1, policy_version 892450 (0.0009) [2023-12-26 21:51:35,088][105692] Updated weights for policy 0, policy_version 892591 (0.0007) [2023-12-26 21:51:35,141][105620] Updated weights for policy 1, policy_version 892460 (0.0010) [2023-12-26 21:51:35,193][105620] Updated weights for policy 1, policy_version 892470 (0.0006) [2023-12-26 21:51:35,898][105692] Updated weights for policy 0, policy_version 892601 (0.0008) [2023-12-26 21:51:35,914][105620] Updated weights for policy 1, policy_version 892480 (0.0007) [2023-12-26 21:51:35,959][105692] Updated weights for policy 0, policy_version 892611 (0.0007) [2023-12-26 21:51:35,961][105620] Updated weights for policy 1, policy_version 892490 (0.0007) [2023-12-26 21:51:36,014][105620] Updated weights for policy 1, policy_version 892500 (0.0007) [2023-12-26 21:51:36,021][105692] Updated weights for policy 0, policy_version 892621 (0.0008) [2023-12-26 21:51:36,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 457056256. Throughput: 0: 9773.6, 1: 9668.0. Samples: 457038584. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 21:51:36,062][104569] Avg episode reward: [(0, '1695.405'), (1, '8199.420')] [2023-12-26 21:51:36,738][105620] Updated weights for policy 1, policy_version 892510 (0.0008) [2023-12-26 21:51:36,792][105620] Updated weights for policy 1, policy_version 892520 (0.0009) [2023-12-26 21:51:36,815][105692] Updated weights for policy 0, policy_version 892631 (0.0008) [2023-12-26 21:51:36,841][105620] Updated weights for policy 1, policy_version 892530 (0.0006) [2023-12-26 21:51:36,868][105692] Updated weights for policy 0, policy_version 892641 (0.0008) [2023-12-26 21:51:36,930][105692] Updated weights for policy 0, policy_version 892651 (0.0008) [2023-12-26 21:51:37,561][105692] Updated weights for policy 0, policy_version 892661 (0.0007) [2023-12-26 21:51:37,613][105692] Updated weights for policy 0, policy_version 892671 (0.0005) [2023-12-26 21:51:37,670][105692] Updated weights for policy 0, policy_version 892681 (0.0005) [2023-12-26 21:51:37,690][105620] Updated weights for policy 1, policy_version 892540 (0.0007) [2023-12-26 21:51:37,742][105620] Updated weights for policy 1, policy_version 892550 (0.0008) [2023-12-26 21:51:37,792][105620] Updated weights for policy 1, policy_version 892560 (0.0009) [2023-12-26 21:51:38,249][105692] Updated weights for policy 0, policy_version 892691 (0.0010) [2023-12-26 21:51:38,304][105692] Updated weights for policy 0, policy_version 892701 (0.0009) [2023-12-26 21:51:38,370][105692] Updated weights for policy 0, policy_version 892711 (0.0008) [2023-12-26 21:51:38,674][105620] Updated weights for policy 1, policy_version 892570 (0.0009) [2023-12-26 21:51:38,743][105620] Updated weights for policy 1, policy_version 892580 (0.0009) [2023-12-26 21:51:38,817][105620] Updated weights for policy 1, policy_version 892590 (0.0010) [2023-12-26 21:51:38,892][105620] Updated weights for policy 1, policy_version 892600 (0.0010) [2023-12-26 21:51:39,016][105692] Updated weights for policy 0, policy_version 892721 (0.0009) [2023-12-26 21:51:39,072][105692] Updated weights for policy 0, policy_version 892731 (0.0006) [2023-12-26 21:51:39,122][105692] Updated weights for policy 0, policy_version 892741 (0.0005) [2023-12-26 21:51:39,177][105692] Updated weights for policy 0, policy_version 892751 (0.0008) [2023-12-26 21:51:39,669][105620] Updated weights for policy 1, policy_version 892610 (0.0009) [2023-12-26 21:51:39,735][105620] Updated weights for policy 1, policy_version 892620 (0.0008) [2023-12-26 21:51:39,796][105620] Updated weights for policy 1, policy_version 892630 (0.0008) [2023-12-26 21:51:39,943][105692] Updated weights for policy 0, policy_version 892761 (0.0009) [2023-12-26 21:51:39,999][105692] Updated weights for policy 0, policy_version 892771 (0.0009) [2023-12-26 21:51:40,063][105692] Updated weights for policy 0, policy_version 892781 (0.0008) [2023-12-26 21:51:40,461][105620] Updated weights for policy 1, policy_version 892640 (0.0009) [2023-12-26 21:51:40,512][105620] Updated weights for policy 1, policy_version 892650 (0.0009) [2023-12-26 21:51:40,567][105620] Updated weights for policy 1, policy_version 892660 (0.0009) [2023-12-26 21:51:40,858][105692] Updated weights for policy 0, policy_version 892791 (0.0010) [2023-12-26 21:51:40,911][105692] Updated weights for policy 0, policy_version 892801 (0.0009) [2023-12-26 21:51:40,973][105692] Updated weights for policy 0, policy_version 892811 (0.0009) [2023-12-26 21:51:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 457146368. Throughput: 0: 9814.5, 1: 9694.1. Samples: 457153120. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 21:51:41,062][104569] Avg episode reward: [(0, '1375.996'), (1, '7938.318')] [2023-12-26 21:51:41,272][105620] Updated weights for policy 1, policy_version 892670 (0.0008) [2023-12-26 21:51:41,340][105620] Updated weights for policy 1, policy_version 892680 (0.0010) [2023-12-26 21:51:41,416][105620] Updated weights for policy 1, policy_version 892690 (0.0009) [2023-12-26 21:51:41,873][105692] Updated weights for policy 0, policy_version 892822 (0.0010) [2023-12-26 21:51:41,944][105692] Updated weights for policy 0, policy_version 892832 (0.0011) [2023-12-26 21:51:42,004][105692] Updated weights for policy 0, policy_version 892842 (0.0011) [2023-12-26 21:51:42,126][105620] Updated weights for policy 1, policy_version 892700 (0.0009) [2023-12-26 21:51:42,189][105620] Updated weights for policy 1, policy_version 892710 (0.0008) [2023-12-26 21:51:42,248][105620] Updated weights for policy 1, policy_version 892720 (0.0008) [2023-12-26 21:51:42,763][105692] Updated weights for policy 0, policy_version 892852 (0.0010) [2023-12-26 21:51:42,826][105692] Updated weights for policy 0, policy_version 892862 (0.0009) [2023-12-26 21:51:42,888][105692] Updated weights for policy 0, policy_version 892872 (0.0009) [2023-12-26 21:51:42,967][105620] Updated weights for policy 1, policy_version 892730 (0.0009) [2023-12-26 21:51:43,019][105620] Updated weights for policy 1, policy_version 892740 (0.0009) [2023-12-26 21:51:43,081][105620] Updated weights for policy 1, policy_version 892750 (0.0008) [2023-12-26 21:51:43,144][105620] Updated weights for policy 1, policy_version 892760 (0.0008) [2023-12-26 21:51:43,642][105692] Updated weights for policy 0, policy_version 892882 (0.0009) [2023-12-26 21:51:43,696][105692] Updated weights for policy 0, policy_version 892892 (0.0010) [2023-12-26 21:51:43,753][105692] Updated weights for policy 0, policy_version 892902 (0.0010) [2023-12-26 21:51:43,797][105692] Updated weights for policy 0, policy_version 892912 (0.0010) [2023-12-26 21:51:43,902][105620] Updated weights for policy 1, policy_version 892770 (0.0008) [2023-12-26 21:51:43,952][105620] Updated weights for policy 1, policy_version 892781 (0.0008) [2023-12-26 21:51:43,996][105620] Updated weights for policy 1, policy_version 892791 (0.0008) [2023-12-26 21:51:44,540][105692] Updated weights for policy 0, policy_version 892922 (0.0010) [2023-12-26 21:51:44,604][105692] Updated weights for policy 0, policy_version 892932 (0.0010) [2023-12-26 21:51:44,663][105692] Updated weights for policy 0, policy_version 892942 (0.0010) [2023-12-26 21:51:44,784][105620] Updated weights for policy 1, policy_version 892801 (0.0008) [2023-12-26 21:51:44,850][105620] Updated weights for policy 1, policy_version 892811 (0.0008) [2023-12-26 21:51:44,911][105620] Updated weights for policy 1, policy_version 892821 (0.0008) [2023-12-26 21:51:45,424][105692] Updated weights for policy 0, policy_version 892952 (0.0011) [2023-12-26 21:51:45,487][105692] Updated weights for policy 0, policy_version 892962 (0.0010) [2023-12-26 21:51:45,546][105692] Updated weights for policy 0, policy_version 892972 (0.0010) [2023-12-26 21:51:45,695][105620] Updated weights for policy 1, policy_version 892831 (0.0008) [2023-12-26 21:51:45,750][105620] Updated weights for policy 1, policy_version 892841 (0.0008) [2023-12-26 21:51:45,799][105620] Updated weights for policy 1, policy_version 892851 (0.0008) [2023-12-26 21:51:46,062][104569] Fps is (10 sec: 18021.9, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 457236480. Throughput: 0: 9733.8, 1: 9712.9. Samples: 457207700. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 21:51:46,063][104569] Avg episode reward: [(0, '2572.298'), (1, '7848.950')] [2023-12-26 21:51:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000892976_228638720.pth... [2023-12-26 21:51:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000892856_228597760.pth... [2023-12-26 21:51:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000891856_228352000.pth [2023-12-26 21:51:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000891736_228311040.pth [2023-12-26 21:51:46,238][105692] Updated weights for policy 0, policy_version 892982 (0.0010) [2023-12-26 21:51:46,283][105692] Updated weights for policy 0, policy_version 892992 (0.0010) [2023-12-26 21:51:46,327][105692] Updated weights for policy 0, policy_version 893002 (0.0010) [2023-12-26 21:51:46,540][105620] Updated weights for policy 1, policy_version 892862 (0.0010) [2023-12-26 21:51:46,597][105620] Updated weights for policy 1, policy_version 892872 (0.0009) [2023-12-26 21:51:46,655][105620] Updated weights for policy 1, policy_version 892882 (0.0008) [2023-12-26 21:51:47,015][105692] Updated weights for policy 0, policy_version 893012 (0.0010) [2023-12-26 21:51:47,071][105692] Updated weights for policy 0, policy_version 893022 (0.0010) [2023-12-26 21:51:47,126][105692] Updated weights for policy 0, policy_version 893032 (0.0010) [2023-12-26 21:51:47,438][105620] Updated weights for policy 1, policy_version 892892 (0.0009) [2023-12-26 21:51:47,498][105620] Updated weights for policy 1, policy_version 892902 (0.0008) [2023-12-26 21:51:47,558][105620] Updated weights for policy 1, policy_version 892912 (0.0009) [2023-12-26 21:51:47,854][105692] Updated weights for policy 0, policy_version 893042 (0.0010) [2023-12-26 21:51:47,912][105692] Updated weights for policy 0, policy_version 893052 (0.0010) [2023-12-26 21:51:47,966][105692] Updated weights for policy 0, policy_version 893063 (0.0010) [2023-12-26 21:51:48,202][105620] Updated weights for policy 1, policy_version 892922 (0.0009) [2023-12-26 21:51:48,265][105620] Updated weights for policy 1, policy_version 892932 (0.0009) [2023-12-26 21:51:48,323][105620] Updated weights for policy 1, policy_version 892942 (0.0009) [2023-12-26 21:51:48,381][105620] Updated weights for policy 1, policy_version 892952 (0.0008) [2023-12-26 21:51:48,804][105692] Updated weights for policy 0, policy_version 893073 (0.0010) [2023-12-26 21:51:48,859][105692] Updated weights for policy 0, policy_version 893083 (0.0009) [2023-12-26 21:51:48,912][105692] Updated weights for policy 0, policy_version 893093 (0.0008) [2023-12-26 21:51:48,980][105692] Updated weights for policy 0, policy_version 893103 (0.0009) [2023-12-26 21:51:49,058][105620] Updated weights for policy 1, policy_version 892962 (0.0009) [2023-12-26 21:51:49,122][105620] Updated weights for policy 1, policy_version 892972 (0.0009) [2023-12-26 21:51:49,192][105620] Updated weights for policy 1, policy_version 892982 (0.0006) [2023-12-26 21:51:49,806][105692] Updated weights for policy 0, policy_version 893113 (0.0009) [2023-12-26 21:51:49,871][105692] Updated weights for policy 0, policy_version 893123 (0.0008) [2023-12-26 21:51:49,896][105620] Updated weights for policy 1, policy_version 892992 (0.0006) [2023-12-26 21:51:49,933][105692] Updated weights for policy 0, policy_version 893133 (0.0008) [2023-12-26 21:51:49,963][105620] Updated weights for policy 1, policy_version 893002 (0.0008) [2023-12-26 21:51:50,030][105620] Updated weights for policy 1, policy_version 893012 (0.0009) [2023-12-26 21:51:50,659][105692] Updated weights for policy 0, policy_version 893143 (0.0007) [2023-12-26 21:51:50,715][105692] Updated weights for policy 0, policy_version 893153 (0.0006) [2023-12-26 21:51:50,781][105692] Updated weights for policy 0, policy_version 893163 (0.0008) [2023-12-26 21:51:50,804][105620] Updated weights for policy 1, policy_version 893022 (0.0007) [2023-12-26 21:51:50,869][105620] Updated weights for policy 1, policy_version 893032 (0.0009) [2023-12-26 21:51:50,927][105620] Updated weights for policy 1, policy_version 893042 (0.0009) [2023-12-26 21:51:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 457334784. Throughput: 0: 9643.6, 1: 9652.4. Samples: 457322012. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 21:51:51,062][104569] Avg episode reward: [(0, '6514.818'), (1, '8472.182')] [2023-12-26 21:51:51,545][105692] Updated weights for policy 0, policy_version 893173 (0.0009) [2023-12-26 21:51:51,615][105692] Updated weights for policy 0, policy_version 893183 (0.0011) [2023-12-26 21:51:51,671][105692] Updated weights for policy 0, policy_version 893193 (0.0011) [2023-12-26 21:51:51,682][105620] Updated weights for policy 1, policy_version 893052 (0.0007) [2023-12-26 21:51:51,745][105620] Updated weights for policy 1, policy_version 893062 (0.0007) [2023-12-26 21:51:51,808][105620] Updated weights for policy 1, policy_version 893072 (0.0008) [2023-12-26 21:51:52,305][105692] Updated weights for policy 0, policy_version 893203 (0.0009) [2023-12-26 21:51:52,364][105692] Updated weights for policy 0, policy_version 893213 (0.0007) [2023-12-26 21:51:52,426][105692] Updated weights for policy 0, policy_version 893223 (0.0009) [2023-12-26 21:51:52,631][105620] Updated weights for policy 1, policy_version 893082 (0.0008) [2023-12-26 21:51:52,664][105586] KL-divergence is very high: 114.8626 [2023-12-26 21:51:52,680][105620] Updated weights for policy 1, policy_version 893092 (0.0008) [2023-12-26 21:51:52,704][105586] KL-divergence is very high: 179.9017 [2023-12-26 21:51:52,729][105620] Updated weights for policy 1, policy_version 893102 (0.0008) [2023-12-26 21:51:52,744][105586] KL-divergence is very high: 151.6567 [2023-12-26 21:51:52,778][105620] Updated weights for policy 1, policy_version 893112 (0.0008) [2023-12-26 21:51:53,162][105692] Updated weights for policy 0, policy_version 893233 (0.0008) [2023-12-26 21:51:53,218][105692] Updated weights for policy 0, policy_version 893243 (0.0011) [2023-12-26 21:51:53,276][105692] Updated weights for policy 0, policy_version 893253 (0.0010) [2023-12-26 21:51:53,323][105692] Updated weights for policy 0, policy_version 893263 (0.0010) [2023-12-26 21:51:53,549][105620] Updated weights for policy 1, policy_version 893122 (0.0005) [2023-12-26 21:51:53,595][105620] Updated weights for policy 1, policy_version 893132 (0.0005) [2023-12-26 21:51:53,642][105620] Updated weights for policy 1, policy_version 893142 (0.0008) [2023-12-26 21:51:53,952][105692] Updated weights for policy 0, policy_version 893273 (0.0006) [2023-12-26 21:51:53,999][105692] Updated weights for policy 0, policy_version 893283 (0.0006) [2023-12-26 21:51:54,051][105692] Updated weights for policy 0, policy_version 893293 (0.0006) [2023-12-26 21:51:54,273][105620] Updated weights for policy 1, policy_version 893152 (0.0010) [2023-12-26 21:51:54,331][105620] Updated weights for policy 1, policy_version 893162 (0.0010) [2023-12-26 21:51:54,388][105620] Updated weights for policy 1, policy_version 893172 (0.0008) [2023-12-26 21:51:54,724][105692] Updated weights for policy 0, policy_version 893303 (0.0010) [2023-12-26 21:51:54,776][105692] Updated weights for policy 0, policy_version 893313 (0.0010) [2023-12-26 21:51:54,834][105692] Updated weights for policy 0, policy_version 893323 (0.0011) [2023-12-26 21:51:55,022][105620] Updated weights for policy 1, policy_version 893182 (0.0006) [2023-12-26 21:51:55,071][105620] Updated weights for policy 1, policy_version 893192 (0.0005) [2023-12-26 21:51:55,138][105620] Updated weights for policy 1, policy_version 893202 (0.0006) [2023-12-26 21:51:55,578][105692] Updated weights for policy 0, policy_version 893333 (0.0010) [2023-12-26 21:51:55,629][105692] Updated weights for policy 0, policy_version 893343 (0.0010) [2023-12-26 21:51:55,687][105692] Updated weights for policy 0, policy_version 893353 (0.0010) [2023-12-26 21:51:55,782][105620] Updated weights for policy 1, policy_version 893212 (0.0008) [2023-12-26 21:51:55,835][105620] Updated weights for policy 1, policy_version 893222 (0.0008) [2023-12-26 21:51:55,886][105620] Updated weights for policy 1, policy_version 893232 (0.0008) [2023-12-26 21:51:56,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 457433088. Throughput: 0: 9685.0, 1: 9640.1. Samples: 457438780. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 21:51:56,062][104569] Avg episode reward: [(0, '8554.612'), (1, '8642.742')] [2023-12-26 21:51:56,337][105692] Updated weights for policy 0, policy_version 893363 (0.0010) [2023-12-26 21:51:56,381][105692] Updated weights for policy 0, policy_version 893373 (0.0010) [2023-12-26 21:51:56,437][105692] Updated weights for policy 0, policy_version 893383 (0.0010) [2023-12-26 21:51:56,637][105620] Updated weights for policy 1, policy_version 893242 (0.0007) [2023-12-26 21:51:56,693][105620] Updated weights for policy 1, policy_version 893252 (0.0006) [2023-12-26 21:51:56,748][105620] Updated weights for policy 1, policy_version 893262 (0.0008) [2023-12-26 21:51:56,805][105620] Updated weights for policy 1, policy_version 893272 (0.0007) [2023-12-26 21:51:57,206][105692] Updated weights for policy 0, policy_version 893393 (0.0010) [2023-12-26 21:51:57,257][105692] Updated weights for policy 0, policy_version 893403 (0.0010) [2023-12-26 21:51:57,314][105692] Updated weights for policy 0, policy_version 893413 (0.0010) [2023-12-26 21:51:57,375][105692] Updated weights for policy 0, policy_version 893423 (0.0010) [2023-12-26 21:51:57,529][105620] Updated weights for policy 1, policy_version 893282 (0.0008) [2023-12-26 21:51:57,543][105586] KL-divergence is very high: 222.8920 [2023-12-26 21:51:57,556][105586] KL-divergence is very high: 271.2247 [2023-12-26 21:51:57,576][105620] Updated weights for policy 1, policy_version 893292 (0.0008) [2023-12-26 21:51:57,579][105586] KL-divergence is very high: 300.5072 [2023-12-26 21:51:57,592][105586] KL-divergence is very high: 279.9220 [2023-12-26 21:51:57,614][105586] KL-divergence is very high: 237.8000 [2023-12-26 21:51:57,619][105620] Updated weights for policy 1, policy_version 893302 (0.0007) [2023-12-26 21:51:58,111][105692] Updated weights for policy 0, policy_version 893433 (0.0010) [2023-12-26 21:51:58,174][105692] Updated weights for policy 0, policy_version 893443 (0.0009) [2023-12-26 21:51:58,236][105692] Updated weights for policy 0, policy_version 893453 (0.0011) [2023-12-26 21:51:58,326][105586] KL-divergence is very high: 220.9853 [2023-12-26 21:51:58,381][105620] Updated weights for policy 1, policy_version 893312 (0.0007) [2023-12-26 21:51:58,383][105586] KL-divergence is very high: 140.5637 [2023-12-26 21:51:58,437][105586] KL-divergence is very high: 105.8658 [2023-12-26 21:51:58,448][105620] Updated weights for policy 1, policy_version 893322 (0.0008) [2023-12-26 21:51:58,518][105620] Updated weights for policy 1, policy_version 893332 (0.0009) [2023-12-26 21:51:59,040][105692] Updated weights for policy 0, policy_version 893463 (0.0009) [2023-12-26 21:51:59,098][105692] Updated weights for policy 0, policy_version 893473 (0.0008) [2023-12-26 21:51:59,158][105692] Updated weights for policy 0, policy_version 893483 (0.0007) [2023-12-26 21:51:59,300][105620] Updated weights for policy 1, policy_version 893342 (0.0009) [2023-12-26 21:51:59,366][105620] Updated weights for policy 1, policy_version 893352 (0.0008) [2023-12-26 21:51:59,424][105620] Updated weights for policy 1, policy_version 893362 (0.0006) [2023-12-26 21:52:00,003][105692] Updated weights for policy 0, policy_version 893493 (0.0008) [2023-12-26 21:52:00,034][105620] Updated weights for policy 1, policy_version 893372 (0.0007) [2023-12-26 21:52:00,053][105692] Updated weights for policy 0, policy_version 893503 (0.0006) [2023-12-26 21:52:00,091][105620] Updated weights for policy 1, policy_version 893382 (0.0008) [2023-12-26 21:52:00,097][105692] Updated weights for policy 0, policy_version 893513 (0.0009) [2023-12-26 21:52:00,133][105586] KL-divergence is very high: 126.1112 [2023-12-26 21:52:00,152][105620] Updated weights for policy 1, policy_version 893392 (0.0007) [2023-12-26 21:52:00,175][105586] KL-divergence is very high: 139.3905 [2023-12-26 21:52:00,786][105692] Updated weights for policy 0, policy_version 893523 (0.0007) [2023-12-26 21:52:00,845][105692] Updated weights for policy 0, policy_version 893533 (0.0009) [2023-12-26 21:52:00,907][105692] Updated weights for policy 0, policy_version 893543 (0.0009) [2023-12-26 21:52:00,951][105620] Updated weights for policy 1, policy_version 893402 (0.0009) [2023-12-26 21:52:01,000][105620] Updated weights for policy 1, policy_version 893412 (0.0007) [2023-12-26 21:52:01,062][105620] Updated weights for policy 1, policy_version 893422 (0.0009) [2023-12-26 21:52:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 457523200. Throughput: 0: 9682.5, 1: 9621.4. Samples: 457495560. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 21:52:01,063][104569] Avg episode reward: [(0, '8805.501'), (1, '7700.734')] [2023-12-26 21:52:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000893552_228786176.pth... [2023-12-26 21:52:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000892432_228499456.pth [2023-12-26 21:52:01,115][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000893432_228745216.pth... [2023-12-26 21:52:01,118][105620] Updated weights for policy 1, policy_version 893432 (0.0008) [2023-12-26 21:52:01,120][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000892312_228458496.pth [2023-12-26 21:52:01,601][105692] Updated weights for policy 0, policy_version 893553 (0.0010) [2023-12-26 21:52:01,666][105692] Updated weights for policy 0, policy_version 893563 (0.0007) [2023-12-26 21:52:01,735][105692] Updated weights for policy 0, policy_version 893573 (0.0008) [2023-12-26 21:52:01,797][105692] Updated weights for policy 0, policy_version 893583 (0.0009) [2023-12-26 21:52:01,872][105620] Updated weights for policy 1, policy_version 893442 (0.0009) [2023-12-26 21:52:01,937][105620] Updated weights for policy 1, policy_version 893452 (0.0007) [2023-12-26 21:52:01,999][105620] Updated weights for policy 1, policy_version 893462 (0.0010) [2023-12-26 21:52:02,441][105692] Updated weights for policy 0, policy_version 893593 (0.0007) [2023-12-26 21:52:02,507][105692] Updated weights for policy 0, policy_version 893603 (0.0006) [2023-12-26 21:52:02,566][105692] Updated weights for policy 0, policy_version 893613 (0.0009) [2023-12-26 21:52:02,676][105620] Updated weights for policy 1, policy_version 893472 (0.0008) [2023-12-26 21:52:02,723][105620] Updated weights for policy 1, policy_version 893482 (0.0005) [2023-12-26 21:52:02,783][105620] Updated weights for policy 1, policy_version 893492 (0.0006) [2023-12-26 21:52:03,254][105692] Updated weights for policy 0, policy_version 893623 (0.0007) [2023-12-26 21:52:03,302][105692] Updated weights for policy 0, policy_version 893633 (0.0005) [2023-12-26 21:52:03,356][105692] Updated weights for policy 0, policy_version 893643 (0.0007) [2023-12-26 21:52:03,502][105620] Updated weights for policy 1, policy_version 893502 (0.0008) [2023-12-26 21:52:03,568][105620] Updated weights for policy 1, policy_version 893512 (0.0006) [2023-12-26 21:52:03,626][105620] Updated weights for policy 1, policy_version 893522 (0.0009) [2023-12-26 21:52:04,073][105692] Updated weights for policy 0, policy_version 893653 (0.0010) [2023-12-26 21:52:04,128][105692] Updated weights for policy 0, policy_version 893663 (0.0010) [2023-12-26 21:52:04,187][105692] Updated weights for policy 0, policy_version 893673 (0.0010) [2023-12-26 21:52:04,350][105620] Updated weights for policy 1, policy_version 893532 (0.0009) [2023-12-26 21:52:04,417][105620] Updated weights for policy 1, policy_version 893542 (0.0010) [2023-12-26 21:52:04,489][105620] Updated weights for policy 1, policy_version 893552 (0.0010) [2023-12-26 21:52:04,836][105692] Updated weights for policy 0, policy_version 893683 (0.0009) [2023-12-26 21:52:04,890][105692] Updated weights for policy 0, policy_version 893693 (0.0009) [2023-12-26 21:52:04,941][105692] Updated weights for policy 0, policy_version 893703 (0.0008) [2023-12-26 21:52:05,181][105620] Updated weights for policy 1, policy_version 893562 (0.0009) [2023-12-26 21:52:05,248][105620] Updated weights for policy 1, policy_version 893572 (0.0009) [2023-12-26 21:52:05,304][105620] Updated weights for policy 1, policy_version 893582 (0.0009) [2023-12-26 21:52:05,366][105620] Updated weights for policy 1, policy_version 893592 (0.0009) [2023-12-26 21:52:05,622][105692] Updated weights for policy 0, policy_version 893713 (0.0009) [2023-12-26 21:52:05,678][105692] Updated weights for policy 0, policy_version 893723 (0.0005) [2023-12-26 21:52:05,727][105692] Updated weights for policy 0, policy_version 893733 (0.0005) [2023-12-26 21:52:05,783][105692] Updated weights for policy 0, policy_version 893743 (0.0005) [2023-12-26 21:52:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.3, 300 sec: 19494.2). Total num frames: 457621504. Throughput: 0: 9689.1, 1: 9617.7. Samples: 457611968. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 21:52:06,062][104569] Avg episode reward: [(0, '9171.473'), (1, '8059.361')] [2023-12-26 21:52:06,179][105586] KL-divergence is very high: 130.8762 [2023-12-26 21:52:06,185][105586] KL-divergence is very high: 131.6695 [2023-12-26 21:52:06,191][105620] Updated weights for policy 1, policy_version 893602 (0.0008) [2023-12-26 21:52:06,226][105586] KL-divergence is very high: 227.5148 [2023-12-26 21:52:06,231][105586] KL-divergence is very high: 196.9009 [2023-12-26 21:52:06,247][105620] Updated weights for policy 1, policy_version 893612 (0.0007) [2023-12-26 21:52:06,269][105586] KL-divergence is very high: 201.7046 [2023-12-26 21:52:06,275][105586] KL-divergence is very high: 152.7163 [2023-12-26 21:52:06,303][105620] Updated weights for policy 1, policy_version 893622 (0.0008) [2023-12-26 21:52:06,455][105692] Updated weights for policy 0, policy_version 893753 (0.0008) [2023-12-26 21:52:06,504][105692] Updated weights for policy 0, policy_version 893763 (0.0010) [2023-12-26 21:52:06,563][105692] Updated weights for policy 0, policy_version 893773 (0.0011) [2023-12-26 21:52:07,108][105620] Updated weights for policy 1, policy_version 893632 (0.0008) [2023-12-26 21:52:07,172][105620] Updated weights for policy 1, policy_version 893642 (0.0008) [2023-12-26 21:52:07,243][105620] Updated weights for policy 1, policy_version 893652 (0.0006) [2023-12-26 21:52:07,303][105692] Updated weights for policy 0, policy_version 893783 (0.0009) [2023-12-26 21:52:07,365][105692] Updated weights for policy 0, policy_version 893793 (0.0010) [2023-12-26 21:52:07,437][105692] Updated weights for policy 0, policy_version 893803 (0.0008) [2023-12-26 21:52:07,884][105620] Updated weights for policy 1, policy_version 893662 (0.0006) [2023-12-26 21:52:07,930][105620] Updated weights for policy 1, policy_version 893672 (0.0007) [2023-12-26 21:52:07,983][105620] Updated weights for policy 1, policy_version 893682 (0.0009) [2023-12-26 21:52:08,227][105692] Updated weights for policy 0, policy_version 893813 (0.0008) [2023-12-26 21:52:08,281][105692] Updated weights for policy 0, policy_version 893823 (0.0005) [2023-12-26 21:52:08,343][105692] Updated weights for policy 0, policy_version 893833 (0.0006) [2023-12-26 21:52:08,735][105620] Updated weights for policy 1, policy_version 893692 (0.0009) [2023-12-26 21:52:08,784][105620] Updated weights for policy 1, policy_version 893702 (0.0009) [2023-12-26 21:52:08,845][105620] Updated weights for policy 1, policy_version 893712 (0.0009) [2023-12-26 21:52:09,071][105692] Updated weights for policy 0, policy_version 893843 (0.0008) [2023-12-26 21:52:09,119][105692] Updated weights for policy 0, policy_version 893853 (0.0009) [2023-12-26 21:52:09,172][105692] Updated weights for policy 0, policy_version 893863 (0.0009) [2023-12-26 21:52:09,582][105620] Updated weights for policy 1, policy_version 893722 (0.0009) [2023-12-26 21:52:09,640][105620] Updated weights for policy 1, policy_version 893732 (0.0006) [2023-12-26 21:52:09,700][105620] Updated weights for policy 1, policy_version 893742 (0.0006) [2023-12-26 21:52:09,764][105620] Updated weights for policy 1, policy_version 893752 (0.0006) [2023-12-26 21:52:10,009][105692] Updated weights for policy 0, policy_version 893873 (0.0009) [2023-12-26 21:52:10,064][105692] Updated weights for policy 0, policy_version 893883 (0.0009) [2023-12-26 21:52:10,116][105692] Updated weights for policy 0, policy_version 893893 (0.0009) [2023-12-26 21:52:10,165][105692] Updated weights for policy 0, policy_version 893903 (0.0009) [2023-12-26 21:52:10,449][105620] Updated weights for policy 1, policy_version 893762 (0.0010) [2023-12-26 21:52:10,518][105620] Updated weights for policy 1, policy_version 893772 (0.0009) [2023-12-26 21:52:10,579][105620] Updated weights for policy 1, policy_version 893782 (0.0009) [2023-12-26 21:52:10,908][105692] Updated weights for policy 0, policy_version 893913 (0.0006) [2023-12-26 21:52:10,968][105692] Updated weights for policy 0, policy_version 893923 (0.0007) [2023-12-26 21:52:11,030][105692] Updated weights for policy 0, policy_version 893933 (0.0006) [2023-12-26 21:52:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 457719808. Throughput: 0: 9611.4, 1: 9591.5. Samples: 457725240. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 21:52:11,062][104569] Avg episode reward: [(0, '9080.400'), (1, '7945.962')] [2023-12-26 21:52:11,410][105620] Updated weights for policy 1, policy_version 893792 (0.0007) [2023-12-26 21:52:11,481][105620] Updated weights for policy 1, policy_version 893802 (0.0006) [2023-12-26 21:52:11,544][105620] Updated weights for policy 1, policy_version 893812 (0.0008) [2023-12-26 21:52:11,681][105692] Updated weights for policy 0, policy_version 893943 (0.0009) [2023-12-26 21:52:11,745][105692] Updated weights for policy 0, policy_version 893953 (0.0009) [2023-12-26 21:52:11,816][105692] Updated weights for policy 0, policy_version 893963 (0.0009) [2023-12-26 21:52:12,260][105620] Updated weights for policy 1, policy_version 893822 (0.0009) [2023-12-26 21:52:12,333][105620] Updated weights for policy 1, policy_version 893832 (0.0009) [2023-12-26 21:52:12,410][105620] Updated weights for policy 1, policy_version 893842 (0.0008) [2023-12-26 21:52:12,536][105692] Updated weights for policy 0, policy_version 893973 (0.0009) [2023-12-26 21:52:12,587][105692] Updated weights for policy 0, policy_version 893983 (0.0009) [2023-12-26 21:52:12,640][105692] Updated weights for policy 0, policy_version 893993 (0.0009) [2023-12-26 21:52:13,086][105620] Updated weights for policy 1, policy_version 893852 (0.0008) [2023-12-26 21:52:13,144][105620] Updated weights for policy 1, policy_version 893862 (0.0006) [2023-12-26 21:52:13,202][105620] Updated weights for policy 1, policy_version 893872 (0.0008) [2023-12-26 21:52:13,512][105692] Updated weights for policy 0, policy_version 894003 (0.0009) [2023-12-26 21:52:13,571][105692] Updated weights for policy 0, policy_version 894013 (0.0009) [2023-12-26 21:52:13,625][105692] Updated weights for policy 0, policy_version 894023 (0.0008) [2023-12-26 21:52:13,801][105620] Updated weights for policy 1, policy_version 893882 (0.0008) [2023-12-26 21:52:13,866][105620] Updated weights for policy 1, policy_version 893892 (0.0008) [2023-12-26 21:52:13,939][105620] Updated weights for policy 1, policy_version 893902 (0.0009) [2023-12-26 21:52:14,008][105620] Updated weights for policy 1, policy_version 893912 (0.0011) [2023-12-26 21:52:14,280][105692] Updated weights for policy 0, policy_version 894033 (0.0006) [2023-12-26 21:52:14,346][105692] Updated weights for policy 0, policy_version 894043 (0.0009) [2023-12-26 21:52:14,403][105692] Updated weights for policy 0, policy_version 894053 (0.0009) [2023-12-26 21:52:14,461][105692] Updated weights for policy 0, policy_version 894063 (0.0009) [2023-12-26 21:52:14,645][105620] Updated weights for policy 1, policy_version 893922 (0.0008) [2023-12-26 21:52:14,692][105620] Updated weights for policy 1, policy_version 893932 (0.0009) [2023-12-26 21:52:14,748][105620] Updated weights for policy 1, policy_version 893942 (0.0008) [2023-12-26 21:52:15,252][105692] Updated weights for policy 0, policy_version 894073 (0.0010) [2023-12-26 21:52:15,311][105692] Updated weights for policy 0, policy_version 894083 (0.0010) [2023-12-26 21:52:15,368][105692] Updated weights for policy 0, policy_version 894093 (0.0010) [2023-12-26 21:52:15,453][105620] Updated weights for policy 1, policy_version 893952 (0.0008) [2023-12-26 21:52:15,514][105620] Updated weights for policy 1, policy_version 893962 (0.0008) [2023-12-26 21:52:15,591][105620] Updated weights for policy 1, policy_version 893972 (0.0009) [2023-12-26 21:52:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 457809920. Throughput: 0: 9526.8, 1: 9582.9. Samples: 457782888. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 21:52:16,062][104569] Avg episode reward: [(0, '9079.787'), (1, '7433.366')] [2023-12-26 21:52:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000894096_228925440.pth... [2023-12-26 21:52:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000893976_228884480.pth... [2023-12-26 21:52:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000892976_228638720.pth [2023-12-26 21:52:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000892856_228597760.pth [2023-12-26 21:52:16,179][105692] Updated weights for policy 0, policy_version 894103 (0.0009) [2023-12-26 21:52:16,235][105692] Updated weights for policy 0, policy_version 894113 (0.0008) [2023-12-26 21:52:16,284][105692] Updated weights for policy 0, policy_version 894123 (0.0007) [2023-12-26 21:52:16,326][105620] Updated weights for policy 1, policy_version 893982 (0.0010) [2023-12-26 21:52:16,381][105620] Updated weights for policy 1, policy_version 893992 (0.0010) [2023-12-26 21:52:16,440][105620] Updated weights for policy 1, policy_version 894002 (0.0009) [2023-12-26 21:52:16,969][105692] Updated weights for policy 0, policy_version 894133 (0.0006) [2023-12-26 21:52:17,028][105692] Updated weights for policy 0, policy_version 894143 (0.0005) [2023-12-26 21:52:17,078][105692] Updated weights for policy 0, policy_version 894153 (0.0005) [2023-12-26 21:52:17,188][105620] Updated weights for policy 1, policy_version 894012 (0.0008) [2023-12-26 21:52:17,246][105620] Updated weights for policy 1, policy_version 894022 (0.0010) [2023-12-26 21:52:17,300][105620] Updated weights for policy 1, policy_version 894032 (0.0010) [2023-12-26 21:52:17,688][105692] Updated weights for policy 0, policy_version 894163 (0.0007) [2023-12-26 21:52:17,755][105692] Updated weights for policy 0, policy_version 894173 (0.0011) [2023-12-26 21:52:17,810][105692] Updated weights for policy 0, policy_version 894183 (0.0011) [2023-12-26 21:52:18,047][105620] Updated weights for policy 1, policy_version 894042 (0.0008) [2023-12-26 21:52:18,101][105620] Updated weights for policy 1, policy_version 894052 (0.0008) [2023-12-26 21:52:18,148][105620] Updated weights for policy 1, policy_version 894062 (0.0008) [2023-12-26 21:52:18,192][105620] Updated weights for policy 1, policy_version 894072 (0.0007) [2023-12-26 21:52:18,508][105692] Updated weights for policy 0, policy_version 894193 (0.0010) [2023-12-26 21:52:18,574][105692] Updated weights for policy 0, policy_version 894203 (0.0006) [2023-12-26 21:52:18,621][105692] Updated weights for policy 0, policy_version 894213 (0.0005) [2023-12-26 21:52:18,677][105692] Updated weights for policy 0, policy_version 894223 (0.0008) [2023-12-26 21:52:19,037][105620] Updated weights for policy 1, policy_version 894082 (0.0008) [2023-12-26 21:52:19,104][105620] Updated weights for policy 1, policy_version 894092 (0.0009) [2023-12-26 21:52:19,172][105620] Updated weights for policy 1, policy_version 894102 (0.0009) [2023-12-26 21:52:19,289][105692] Updated weights for policy 0, policy_version 894233 (0.0008) [2023-12-26 21:52:19,348][105692] Updated weights for policy 0, policy_version 894243 (0.0009) [2023-12-26 21:52:19,407][105692] Updated weights for policy 0, policy_version 894253 (0.0010) [2023-12-26 21:52:19,938][105620] Updated weights for policy 1, policy_version 894112 (0.0009) [2023-12-26 21:52:20,006][105620] Updated weights for policy 1, policy_version 894122 (0.0008) [2023-12-26 21:52:20,081][105620] Updated weights for policy 1, policy_version 894132 (0.0009) [2023-12-26 21:52:20,193][105692] Updated weights for policy 0, policy_version 894263 (0.0008) [2023-12-26 21:52:20,254][105692] Updated weights for policy 0, policy_version 894273 (0.0008) [2023-12-26 21:52:20,318][105692] Updated weights for policy 0, policy_version 894283 (0.0008) [2023-12-26 21:52:20,830][105620] Updated weights for policy 1, policy_version 894142 (0.0010) [2023-12-26 21:52:20,881][105586] KL-divergence is very high: 100.6417 [2023-12-26 21:52:20,888][105586] KL-divergence is very high: 115.1531 [2023-12-26 21:52:20,894][105620] Updated weights for policy 1, policy_version 894152 (0.0010) [2023-12-26 21:52:20,953][105620] Updated weights for policy 1, policy_version 894162 (0.0009) [2023-12-26 21:52:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 457908224. Throughput: 0: 9604.7, 1: 9500.0. Samples: 457898296. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 21:52:21,062][104569] Avg episode reward: [(0, '5616.261'), (1, '6370.300')] [2023-12-26 21:52:21,067][105692] Updated weights for policy 0, policy_version 894293 (0.0009) [2023-12-26 21:52:21,134][105692] Updated weights for policy 0, policy_version 894303 (0.0009) [2023-12-26 21:52:21,199][105692] Updated weights for policy 0, policy_version 894313 (0.0009) [2023-12-26 21:52:21,743][105620] Updated weights for policy 1, policy_version 894172 (0.0009) [2023-12-26 21:52:21,807][105620] Updated weights for policy 1, policy_version 894182 (0.0007) [2023-12-26 21:52:21,870][105620] Updated weights for policy 1, policy_version 894192 (0.0009) [2023-12-26 21:52:21,982][105692] Updated weights for policy 0, policy_version 894323 (0.0008) [2023-12-26 21:52:22,051][105692] Updated weights for policy 0, policy_version 894333 (0.0009) [2023-12-26 21:52:22,117][105692] Updated weights for policy 0, policy_version 894343 (0.0008) [2023-12-26 21:52:22,615][105620] Updated weights for policy 1, policy_version 894202 (0.0008) [2023-12-26 21:52:22,676][105620] Updated weights for policy 1, policy_version 894212 (0.0007) [2023-12-26 21:52:22,735][105620] Updated weights for policy 1, policy_version 894222 (0.0007) [2023-12-26 21:52:22,783][105620] Updated weights for policy 1, policy_version 894232 (0.0009) [2023-12-26 21:52:22,868][105692] Updated weights for policy 0, policy_version 894353 (0.0009) [2023-12-26 21:52:22,926][105692] Updated weights for policy 0, policy_version 894363 (0.0008) [2023-12-26 21:52:22,989][105692] Updated weights for policy 0, policy_version 894373 (0.0009) [2023-12-26 21:52:23,056][105692] Updated weights for policy 0, policy_version 894383 (0.0009) [2023-12-26 21:52:23,547][105620] Updated weights for policy 1, policy_version 894242 (0.0009) [2023-12-26 21:52:23,594][105620] Updated weights for policy 1, policy_version 894252 (0.0009) [2023-12-26 21:52:23,641][105620] Updated weights for policy 1, policy_version 894262 (0.0008) [2023-12-26 21:52:23,811][105692] Updated weights for policy 0, policy_version 894393 (0.0010) [2023-12-26 21:52:23,875][105692] Updated weights for policy 0, policy_version 894403 (0.0009) [2023-12-26 21:52:23,937][105692] Updated weights for policy 0, policy_version 894413 (0.0009) [2023-12-26 21:52:24,337][105620] Updated weights for policy 1, policy_version 894272 (0.0006) [2023-12-26 21:52:24,389][105620] Updated weights for policy 1, policy_version 894282 (0.0006) [2023-12-26 21:52:24,438][105620] Updated weights for policy 1, policy_version 894292 (0.0009) [2023-12-26 21:52:24,668][105692] Updated weights for policy 0, policy_version 894423 (0.0008) [2023-12-26 21:52:24,721][105692] Updated weights for policy 0, policy_version 894433 (0.0008) [2023-12-26 21:52:24,770][105692] Updated weights for policy 0, policy_version 894443 (0.0009) [2023-12-26 21:52:25,216][105620] Updated weights for policy 1, policy_version 894302 (0.0010) [2023-12-26 21:52:25,267][105620] Updated weights for policy 1, policy_version 894312 (0.0009) [2023-12-26 21:52:25,320][105620] Updated weights for policy 1, policy_version 894322 (0.0009) [2023-12-26 21:52:25,520][105692] Updated weights for policy 0, policy_version 894453 (0.0009) [2023-12-26 21:52:25,581][105692] Updated weights for policy 0, policy_version 894463 (0.0010) [2023-12-26 21:52:25,637][105692] Updated weights for policy 0, policy_version 894473 (0.0009) [2023-12-26 21:52:25,967][105620] Updated weights for policy 1, policy_version 894332 (0.0009) [2023-12-26 21:52:26,026][105620] Updated weights for policy 1, policy_version 894342 (0.0008) [2023-12-26 21:52:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19438.7). Total num frames: 457998336. Throughput: 0: 9534.7, 1: 9523.9. Samples: 458010756. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 21:52:26,062][104569] Avg episode reward: [(0, '3081.723'), (1, '4860.905')] [2023-12-26 21:52:26,089][105620] Updated weights for policy 1, policy_version 894352 (0.0009) [2023-12-26 21:52:26,450][105692] Updated weights for policy 0, policy_version 894483 (0.0010) [2023-12-26 21:52:26,513][105692] Updated weights for policy 0, policy_version 894493 (0.0010) [2023-12-26 21:52:26,568][105692] Updated weights for policy 0, policy_version 894503 (0.0010) [2023-12-26 21:52:26,720][105620] Updated weights for policy 1, policy_version 894362 (0.0009) [2023-12-26 21:52:26,788][105620] Updated weights for policy 1, policy_version 894372 (0.0005) [2023-12-26 21:52:26,858][105620] Updated weights for policy 1, policy_version 894382 (0.0005) [2023-12-26 21:52:26,918][105620] Updated weights for policy 1, policy_version 894392 (0.0008) [2023-12-26 21:52:27,400][105692] Updated weights for policy 0, policy_version 894513 (0.0010) [2023-12-26 21:52:27,450][105692] Updated weights for policy 0, policy_version 894524 (0.0009) [2023-12-26 21:52:27,487][105620] Updated weights for policy 1, policy_version 894402 (0.0010) [2023-12-26 21:52:27,521][105692] Updated weights for policy 0, policy_version 894534 (0.0005) [2023-12-26 21:52:27,544][105620] Updated weights for policy 1, policy_version 894412 (0.0008) [2023-12-26 21:52:27,568][105692] Updated weights for policy 0, policy_version 894544 (0.0009) [2023-12-26 21:52:27,592][105620] Updated weights for policy 1, policy_version 894422 (0.0005) [2023-12-26 21:52:28,296][105620] Updated weights for policy 1, policy_version 894432 (0.0010) [2023-12-26 21:52:28,355][105692] Updated weights for policy 0, policy_version 894554 (0.0006) [2023-12-26 21:52:28,356][105620] Updated weights for policy 1, policy_version 894442 (0.0009) [2023-12-26 21:52:28,414][105692] Updated weights for policy 0, policy_version 894564 (0.0006) [2023-12-26 21:52:28,416][105620] Updated weights for policy 1, policy_version 894452 (0.0011) [2023-12-26 21:52:28,466][105692] Updated weights for policy 0, policy_version 894574 (0.0007) [2023-12-26 21:52:29,045][105620] Updated weights for policy 1, policy_version 894462 (0.0007) [2023-12-26 21:52:29,095][105620] Updated weights for policy 1, policy_version 894472 (0.0010) [2023-12-26 21:52:29,139][105620] Updated weights for policy 1, policy_version 894482 (0.0010) [2023-12-26 21:52:29,278][105692] Updated weights for policy 0, policy_version 894584 (0.0008) [2023-12-26 21:52:29,337][105692] Updated weights for policy 0, policy_version 894594 (0.0008) [2023-12-26 21:52:29,393][105692] Updated weights for policy 0, policy_version 894604 (0.0008) [2023-12-26 21:52:29,888][105620] Updated weights for policy 1, policy_version 894492 (0.0010) [2023-12-26 21:52:29,952][105620] Updated weights for policy 1, policy_version 894502 (0.0010) [2023-12-26 21:52:30,007][105620] Updated weights for policy 1, policy_version 894512 (0.0010) [2023-12-26 21:52:30,141][105692] Updated weights for policy 0, policy_version 894614 (0.0008) [2023-12-26 21:52:30,206][105692] Updated weights for policy 0, policy_version 894624 (0.0008) [2023-12-26 21:52:30,263][105692] Updated weights for policy 0, policy_version 894634 (0.0008) [2023-12-26 21:52:30,697][105620] Updated weights for policy 1, policy_version 894522 (0.0011) [2023-12-26 21:52:30,746][105620] Updated weights for policy 1, policy_version 894532 (0.0010) [2023-12-26 21:52:30,794][105620] Updated weights for policy 1, policy_version 894542 (0.0010) [2023-12-26 21:52:30,841][105620] Updated weights for policy 1, policy_version 894552 (0.0010) [2023-12-26 21:52:30,987][105692] Updated weights for policy 0, policy_version 894644 (0.0008) [2023-12-26 21:52:31,045][105692] Updated weights for policy 0, policy_version 894654 (0.0009) [2023-12-26 21:52:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 18978.1, 300 sec: 19438.6). Total num frames: 458096640. Throughput: 0: 9539.1, 1: 9595.4. Samples: 458068748. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 21:52:31,062][104569] Avg episode reward: [(0, '5205.552'), (1, '7425.277')] [2023-12-26 21:52:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000894552_229031936.pth... [2023-12-26 21:52:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000893432_228745216.pth [2023-12-26 21:52:31,108][105692] Updated weights for policy 0, policy_version 894664 (0.0009) [2023-12-26 21:52:31,160][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000894672_229072896.pth... [2023-12-26 21:52:31,165][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000893552_228786176.pth [2023-12-26 21:52:31,611][105620] Updated weights for policy 1, policy_version 894562 (0.0010) [2023-12-26 21:52:31,673][105620] Updated weights for policy 1, policy_version 894572 (0.0010) [2023-12-26 21:52:31,741][105620] Updated weights for policy 1, policy_version 894582 (0.0010) [2023-12-26 21:52:31,898][105692] Updated weights for policy 0, policy_version 894674 (0.0008) [2023-12-26 21:52:31,964][105692] Updated weights for policy 0, policy_version 894684 (0.0008) [2023-12-26 21:52:32,016][105692] Updated weights for policy 0, policy_version 894694 (0.0008) [2023-12-26 21:52:32,060][105692] Updated weights for policy 0, policy_version 894704 (0.0007) [2023-12-26 21:52:32,476][105620] Updated weights for policy 1, policy_version 894592 (0.0010) [2023-12-26 21:52:32,533][105620] Updated weights for policy 1, policy_version 894602 (0.0010) [2023-12-26 21:52:32,592][105620] Updated weights for policy 1, policy_version 894612 (0.0010) [2023-12-26 21:52:32,831][105692] Updated weights for policy 0, policy_version 894714 (0.0008) [2023-12-26 21:52:32,883][105692] Updated weights for policy 0, policy_version 894724 (0.0008) [2023-12-26 21:52:32,934][105692] Updated weights for policy 0, policy_version 894734 (0.0008) [2023-12-26 21:52:33,326][105620] Updated weights for policy 1, policy_version 894622 (0.0010) [2023-12-26 21:52:33,386][105620] Updated weights for policy 1, policy_version 894632 (0.0010) [2023-12-26 21:52:33,438][105620] Updated weights for policy 1, policy_version 894642 (0.0010) [2023-12-26 21:52:33,691][105692] Updated weights for policy 0, policy_version 894744 (0.0009) [2023-12-26 21:52:33,745][105692] Updated weights for policy 0, policy_version 894754 (0.0008) [2023-12-26 21:52:33,803][105692] Updated weights for policy 0, policy_version 894764 (0.0008) [2023-12-26 21:52:34,184][105620] Updated weights for policy 1, policy_version 894652 (0.0010) [2023-12-26 21:52:34,236][105620] Updated weights for policy 1, policy_version 894662 (0.0009) [2023-12-26 21:52:34,292][105620] Updated weights for policy 1, policy_version 894672 (0.0010) [2023-12-26 21:52:34,584][105692] Updated weights for policy 0, policy_version 894774 (0.0008) [2023-12-26 21:52:34,644][105692] Updated weights for policy 0, policy_version 894784 (0.0008) [2023-12-26 21:52:34,702][105692] Updated weights for policy 0, policy_version 894794 (0.0008) [2023-12-26 21:52:35,059][105620] Updated weights for policy 1, policy_version 894682 (0.0010) [2023-12-26 21:52:35,107][105620] Updated weights for policy 1, policy_version 894692 (0.0010) [2023-12-26 21:52:35,165][105620] Updated weights for policy 1, policy_version 894702 (0.0010) [2023-12-26 21:52:35,226][105620] Updated weights for policy 1, policy_version 894712 (0.0010) [2023-12-26 21:52:35,469][105692] Updated weights for policy 0, policy_version 894804 (0.0008) [2023-12-26 21:52:35,517][105692] Updated weights for policy 0, policy_version 894814 (0.0008) [2023-12-26 21:52:35,566][105692] Updated weights for policy 0, policy_version 894824 (0.0008) [2023-12-26 21:52:35,909][105620] Updated weights for policy 1, policy_version 894722 (0.0008) [2023-12-26 21:52:35,963][105620] Updated weights for policy 1, policy_version 894732 (0.0006) [2023-12-26 21:52:36,011][105620] Updated weights for policy 1, policy_version 894742 (0.0005) [2023-12-26 21:52:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 18978.1, 300 sec: 19466.4). Total num frames: 458194944. Throughput: 0: 9511.3, 1: 9578.7. Samples: 458181064. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 21:52:36,063][104569] Avg episode reward: [(0, '8253.630'), (1, '8198.292')] [2023-12-26 21:52:36,242][105692] Updated weights for policy 0, policy_version 894834 (0.0009) [2023-12-26 21:52:36,307][105692] Updated weights for policy 0, policy_version 894844 (0.0010) [2023-12-26 21:52:36,373][105692] Updated weights for policy 0, policy_version 894854 (0.0010) [2023-12-26 21:52:36,438][105692] Updated weights for policy 0, policy_version 894864 (0.0010) [2023-12-26 21:52:36,698][105620] Updated weights for policy 1, policy_version 894752 (0.0009) [2023-12-26 21:52:36,757][105620] Updated weights for policy 1, policy_version 894762 (0.0010) [2023-12-26 21:52:36,809][105620] Updated weights for policy 1, policy_version 894772 (0.0010) [2023-12-26 21:52:37,123][105692] Updated weights for policy 0, policy_version 894874 (0.0010) [2023-12-26 21:52:37,172][105692] Updated weights for policy 0, policy_version 894884 (0.0010) [2023-12-26 21:52:37,220][105692] Updated weights for policy 0, policy_version 894894 (0.0010) [2023-12-26 21:52:37,516][105620] Updated weights for policy 1, policy_version 894782 (0.0007) [2023-12-26 21:52:37,580][105620] Updated weights for policy 1, policy_version 894792 (0.0005) [2023-12-26 21:52:37,637][105620] Updated weights for policy 1, policy_version 894802 (0.0005) [2023-12-26 21:52:37,981][105692] Updated weights for policy 0, policy_version 894904 (0.0010) [2023-12-26 21:52:38,046][105692] Updated weights for policy 0, policy_version 894914 (0.0010) [2023-12-26 21:52:38,116][105692] Updated weights for policy 0, policy_version 894924 (0.0011) [2023-12-26 21:52:38,147][105620] Updated weights for policy 1, policy_version 894812 (0.0008) [2023-12-26 21:52:38,192][105620] Updated weights for policy 1, policy_version 894822 (0.0010) [2023-12-26 21:52:38,236][105620] Updated weights for policy 1, policy_version 894832 (0.0010) [2023-12-26 21:52:38,868][105692] Updated weights for policy 0, policy_version 894934 (0.0010) [2023-12-26 21:52:38,928][105692] Updated weights for policy 0, policy_version 894944 (0.0011) [2023-12-26 21:52:38,981][105692] Updated weights for policy 0, policy_version 894954 (0.0010) [2023-12-26 21:52:38,984][105620] Updated weights for policy 1, policy_version 894842 (0.0010) [2023-12-26 21:52:39,033][105620] Updated weights for policy 1, policy_version 894852 (0.0010) [2023-12-26 21:52:39,082][105620] Updated weights for policy 1, policy_version 894862 (0.0010) [2023-12-26 21:52:39,134][105620] Updated weights for policy 1, policy_version 894872 (0.0010) [2023-12-26 21:52:39,691][105692] Updated weights for policy 0, policy_version 894964 (0.0010) [2023-12-26 21:52:39,755][105692] Updated weights for policy 0, policy_version 894974 (0.0008) [2023-12-26 21:52:39,814][105692] Updated weights for policy 0, policy_version 894984 (0.0010) [2023-12-26 21:52:39,950][105620] Updated weights for policy 1, policy_version 894882 (0.0010) [2023-12-26 21:52:40,004][105620] Updated weights for policy 1, policy_version 894892 (0.0008) [2023-12-26 21:52:40,053][105620] Updated weights for policy 1, policy_version 894902 (0.0011) [2023-12-26 21:52:40,514][105692] Updated weights for policy 0, policy_version 894994 (0.0009) [2023-12-26 21:52:40,582][105692] Updated weights for policy 0, policy_version 895004 (0.0007) [2023-12-26 21:52:40,646][105692] Updated weights for policy 0, policy_version 895014 (0.0006) [2023-12-26 21:52:40,720][105692] Updated weights for policy 0, policy_version 895024 (0.0006) [2023-12-26 21:52:40,779][105620] Updated weights for policy 1, policy_version 894912 (0.0006) [2023-12-26 21:52:40,849][105620] Updated weights for policy 1, policy_version 894922 (0.0005) [2023-12-26 21:52:40,906][105620] Updated weights for policy 1, policy_version 894932 (0.0007) [2023-12-26 21:52:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19114.6, 300 sec: 19494.2). Total num frames: 458293248. Throughput: 0: 9478.6, 1: 9646.1. Samples: 458299396. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 21:52:41,063][104569] Avg episode reward: [(0, '9353.423'), (1, '8815.898')] [2023-12-26 21:52:41,392][105692] Updated weights for policy 0, policy_version 895034 (0.0009) [2023-12-26 21:52:41,452][105692] Updated weights for policy 0, policy_version 895044 (0.0006) [2023-12-26 21:52:41,509][105692] Updated weights for policy 0, policy_version 895054 (0.0006) [2023-12-26 21:52:41,575][105620] Updated weights for policy 1, policy_version 894942 (0.0009) [2023-12-26 21:52:41,636][105620] Updated weights for policy 1, policy_version 894952 (0.0009) [2023-12-26 21:52:41,696][105620] Updated weights for policy 1, policy_version 894962 (0.0009) [2023-12-26 21:52:42,130][105692] Updated weights for policy 0, policy_version 895064 (0.0009) [2023-12-26 21:52:42,197][105692] Updated weights for policy 0, policy_version 895074 (0.0009) [2023-12-26 21:52:42,256][105692] Updated weights for policy 0, policy_version 895084 (0.0009) [2023-12-26 21:52:42,439][105620] Updated weights for policy 1, policy_version 894972 (0.0010) [2023-12-26 21:52:42,488][105620] Updated weights for policy 1, policy_version 894982 (0.0009) [2023-12-26 21:52:42,549][105620] Updated weights for policy 1, policy_version 894992 (0.0010) [2023-12-26 21:52:43,009][105692] Updated weights for policy 0, policy_version 895094 (0.0009) [2023-12-26 21:52:43,062][105692] Updated weights for policy 0, policy_version 895104 (0.0009) [2023-12-26 21:52:43,121][105692] Updated weights for policy 0, policy_version 895114 (0.0009) [2023-12-26 21:52:43,336][105620] Updated weights for policy 1, policy_version 895002 (0.0009) [2023-12-26 21:52:43,394][105620] Updated weights for policy 1, policy_version 895012 (0.0010) [2023-12-26 21:52:43,454][105620] Updated weights for policy 1, policy_version 895022 (0.0008) [2023-12-26 21:52:43,517][105620] Updated weights for policy 1, policy_version 895032 (0.0008) [2023-12-26 21:52:43,845][105692] Updated weights for policy 0, policy_version 895124 (0.0009) [2023-12-26 21:52:43,904][105692] Updated weights for policy 0, policy_version 895134 (0.0006) [2023-12-26 21:52:43,970][105692] Updated weights for policy 0, policy_version 895144 (0.0005) [2023-12-26 21:52:44,324][105620] Updated weights for policy 1, policy_version 895042 (0.0010) [2023-12-26 21:52:44,379][105620] Updated weights for policy 1, policy_version 895053 (0.0009) [2023-12-26 21:52:44,439][105620] Updated weights for policy 1, policy_version 895063 (0.0009) [2023-12-26 21:52:44,530][105692] Updated weights for policy 0, policy_version 895154 (0.0005) [2023-12-26 21:52:44,598][105692] Updated weights for policy 0, policy_version 895164 (0.0007) [2023-12-26 21:52:44,652][105692] Updated weights for policy 0, policy_version 895174 (0.0009) [2023-12-26 21:52:44,710][105692] Updated weights for policy 0, policy_version 895184 (0.0009) [2023-12-26 21:52:45,228][105620] Updated weights for policy 1, policy_version 895073 (0.0007) [2023-12-26 21:52:45,285][105620] Updated weights for policy 1, policy_version 895083 (0.0009) [2023-12-26 21:52:45,340][105620] Updated weights for policy 1, policy_version 895093 (0.0008) [2023-12-26 21:52:45,475][105692] Updated weights for policy 0, policy_version 895194 (0.0009) [2023-12-26 21:52:45,523][105692] Updated weights for policy 0, policy_version 895204 (0.0009) [2023-12-26 21:52:45,578][105692] Updated weights for policy 0, policy_version 895214 (0.0009) [2023-12-26 21:52:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19438.6). Total num frames: 458383360. Throughput: 0: 9496.9, 1: 9636.6. Samples: 458356564. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) [2023-12-26 21:52:46,062][104569] Avg episode reward: [(0, '9263.205'), (1, '8813.882')] [2023-12-26 21:52:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000895216_229212160.pth... [2023-12-26 21:52:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000894096_228925440.pth [2023-12-26 21:52:46,109][105620] Updated weights for policy 1, policy_version 895103 (0.0009) [2023-12-26 21:52:46,170][105620] Updated weights for policy 1, policy_version 895113 (0.0009) [2023-12-26 21:52:46,225][105620] Updated weights for policy 1, policy_version 895123 (0.0009) [2023-12-26 21:52:46,254][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000895128_229179392.pth... [2023-12-26 21:52:46,257][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000893976_228884480.pth [2023-12-26 21:52:46,366][105692] Updated weights for policy 0, policy_version 895224 (0.0009) [2023-12-26 21:52:46,429][105692] Updated weights for policy 0, policy_version 895234 (0.0009) [2023-12-26 21:52:46,490][105692] Updated weights for policy 0, policy_version 895244 (0.0008) [2023-12-26 21:52:47,034][105620] Updated weights for policy 1, policy_version 895133 (0.0008) [2023-12-26 21:52:47,091][105620] Updated weights for policy 1, policy_version 895143 (0.0009) [2023-12-26 21:52:47,117][105692] Updated weights for policy 0, policy_version 895254 (0.0005) [2023-12-26 21:52:47,145][105620] Updated weights for policy 1, policy_version 895153 (0.0007) [2023-12-26 21:52:47,166][105692] Updated weights for policy 0, policy_version 895264 (0.0009) [2023-12-26 21:52:47,211][105692] Updated weights for policy 0, policy_version 895274 (0.0006) [2023-12-26 21:52:47,881][105620] Updated weights for policy 1, policy_version 895163 (0.0007) [2023-12-26 21:52:47,916][105692] Updated weights for policy 0, policy_version 895284 (0.0006) [2023-12-26 21:52:47,938][105620] Updated weights for policy 1, policy_version 895173 (0.0009) [2023-12-26 21:52:47,969][105692] Updated weights for policy 0, policy_version 895294 (0.0009) [2023-12-26 21:52:47,991][105620] Updated weights for policy 1, policy_version 895183 (0.0005) [2023-12-26 21:52:48,024][105692] Updated weights for policy 0, policy_version 895304 (0.0010) [2023-12-26 21:52:48,751][105692] Updated weights for policy 0, policy_version 895314 (0.0009) [2023-12-26 21:52:48,801][105620] Updated weights for policy 1, policy_version 895193 (0.0006) [2023-12-26 21:52:48,805][105692] Updated weights for policy 0, policy_version 895324 (0.0007) [2023-12-26 21:52:48,851][105620] Updated weights for policy 1, policy_version 895203 (0.0008) [2023-12-26 21:52:48,858][105692] Updated weights for policy 0, policy_version 895334 (0.0005) [2023-12-26 21:52:48,899][105620] Updated weights for policy 1, policy_version 895213 (0.0009) [2023-12-26 21:52:48,912][105692] Updated weights for policy 0, policy_version 895344 (0.0005) [2023-12-26 21:52:48,953][105620] Updated weights for policy 1, policy_version 895223 (0.0009) [2023-12-26 21:52:49,556][105692] Updated weights for policy 0, policy_version 895354 (0.0011) [2023-12-26 21:52:49,626][105692] Updated weights for policy 0, policy_version 895364 (0.0011) [2023-12-26 21:52:49,649][105620] Updated weights for policy 1, policy_version 895233 (0.0007) [2023-12-26 21:52:49,695][105692] Updated weights for policy 0, policy_version 895374 (0.0010) [2023-12-26 21:52:49,706][105620] Updated weights for policy 1, policy_version 895243 (0.0007) [2023-12-26 21:52:49,769][105620] Updated weights for policy 1, policy_version 895253 (0.0005) [2023-12-26 21:52:50,317][105620] Updated weights for policy 1, policy_version 895263 (0.0007) [2023-12-26 21:52:50,373][105620] Updated weights for policy 1, policy_version 895273 (0.0006) [2023-12-26 21:52:50,420][105620] Updated weights for policy 1, policy_version 895283 (0.0007) [2023-12-26 21:52:50,441][105692] Updated weights for policy 0, policy_version 895384 (0.0010) [2023-12-26 21:52:50,492][105692] Updated weights for policy 0, policy_version 895394 (0.0010) [2023-12-26 21:52:50,559][105692] Updated weights for policy 0, policy_version 895405 (0.0011) [2023-12-26 21:52:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.6, 300 sec: 19410.9). Total num frames: 458481664. Throughput: 0: 9544.9, 1: 9581.4. Samples: 458472652. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-26 21:52:51,063][104569] Avg episode reward: [(0, '9263.517'), (1, '8811.644')] [2023-12-26 21:52:51,228][105620] Updated weights for policy 1, policy_version 895293 (0.0007) [2023-12-26 21:52:51,256][105692] Updated weights for policy 0, policy_version 895415 (0.0007) [2023-12-26 21:52:51,296][105620] Updated weights for policy 1, policy_version 895303 (0.0006) [2023-12-26 21:52:51,323][105692] Updated weights for policy 0, policy_version 895425 (0.0007) [2023-12-26 21:52:51,343][105620] Updated weights for policy 1, policy_version 895313 (0.0006) [2023-12-26 21:52:51,396][105692] Updated weights for policy 0, policy_version 895435 (0.0008) [2023-12-26 21:52:52,032][105620] Updated weights for policy 1, policy_version 895323 (0.0007) [2023-12-26 21:52:52,091][105620] Updated weights for policy 1, policy_version 895333 (0.0006) [2023-12-26 21:52:52,139][105692] Updated weights for policy 0, policy_version 895445 (0.0009) [2023-12-26 21:52:52,149][105620] Updated weights for policy 1, policy_version 895343 (0.0005) [2023-12-26 21:52:52,199][105692] Updated weights for policy 0, policy_version 895455 (0.0009) [2023-12-26 21:52:52,256][105692] Updated weights for policy 0, policy_version 895465 (0.0010) [2023-12-26 21:52:52,886][105620] Updated weights for policy 1, policy_version 895353 (0.0006) [2023-12-26 21:52:52,944][105620] Updated weights for policy 1, policy_version 895363 (0.0009) [2023-12-26 21:52:52,997][105620] Updated weights for policy 1, policy_version 895373 (0.0008) [2023-12-26 21:52:53,011][105692] Updated weights for policy 0, policy_version 895475 (0.0010) [2023-12-26 21:52:53,060][105692] Updated weights for policy 0, policy_version 895485 (0.0007) [2023-12-26 21:52:53,062][105620] Updated weights for policy 1, policy_version 895383 (0.0009) [2023-12-26 21:52:53,107][105692] Updated weights for policy 0, policy_version 895495 (0.0009) [2023-12-26 21:52:53,806][105620] Updated weights for policy 1, policy_version 895393 (0.0010) [2023-12-26 21:52:53,863][105620] Updated weights for policy 1, policy_version 895403 (0.0008) [2023-12-26 21:52:53,886][105692] Updated weights for policy 0, policy_version 895505 (0.0010) [2023-12-26 21:52:53,923][105620] Updated weights for policy 1, policy_version 895413 (0.0008) [2023-12-26 21:52:53,939][105692] Updated weights for policy 0, policy_version 895515 (0.0008) [2023-12-26 21:52:53,997][105692] Updated weights for policy 0, policy_version 895525 (0.0009) [2023-12-26 21:52:54,059][105692] Updated weights for policy 0, policy_version 895535 (0.0009) [2023-12-26 21:52:54,697][105620] Updated weights for policy 1, policy_version 895423 (0.0009) [2023-12-26 21:52:54,744][105620] Updated weights for policy 1, policy_version 895433 (0.0008) [2023-12-26 21:52:54,790][105692] Updated weights for policy 0, policy_version 895545 (0.0008) [2023-12-26 21:52:54,793][105620] Updated weights for policy 1, policy_version 895443 (0.0006) [2023-12-26 21:52:54,858][105692] Updated weights for policy 0, policy_version 895555 (0.0009) [2023-12-26 21:52:54,913][105692] Updated weights for policy 0, policy_version 895565 (0.0009) [2023-12-26 21:52:55,550][105620] Updated weights for policy 1, policy_version 895453 (0.0008) [2023-12-26 21:52:55,613][105620] Updated weights for policy 1, policy_version 895463 (0.0010) [2023-12-26 21:52:55,657][105692] Updated weights for policy 0, policy_version 895575 (0.0008) [2023-12-26 21:52:55,660][105620] Updated weights for policy 1, policy_version 895473 (0.0010) [2023-12-26 21:52:55,719][105692] Updated weights for policy 0, policy_version 895585 (0.0007) [2023-12-26 21:52:55,784][105692] Updated weights for policy 0, policy_version 895595 (0.0007) [2023-12-26 21:52:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19114.6, 300 sec: 19438.6). Total num frames: 458579968. Throughput: 0: 9522.9, 1: 9611.7. Samples: 458586300. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-26 21:52:56,063][104569] Avg episode reward: [(0, '9355.088'), (1, '8992.042')] [2023-12-26 21:52:56,342][105692] Updated weights for policy 0, policy_version 895605 (0.0005) [2023-12-26 21:52:56,391][105692] Updated weights for policy 0, policy_version 895615 (0.0005) [2023-12-26 21:52:56,395][105620] Updated weights for policy 1, policy_version 895483 (0.0010) [2023-12-26 21:52:56,446][105692] Updated weights for policy 0, policy_version 895625 (0.0005) [2023-12-26 21:52:56,447][105620] Updated weights for policy 1, policy_version 895493 (0.0010) [2023-12-26 21:52:56,499][105620] Updated weights for policy 1, policy_version 895503 (0.0010) [2023-12-26 21:52:57,022][105692] Updated weights for policy 0, policy_version 895635 (0.0005) [2023-12-26 21:52:57,087][105692] Updated weights for policy 0, policy_version 895645 (0.0005) [2023-12-26 21:52:57,136][105692] Updated weights for policy 0, policy_version 895655 (0.0010) [2023-12-26 21:52:57,138][105620] Updated weights for policy 1, policy_version 895513 (0.0010) [2023-12-26 21:52:57,182][105620] Updated weights for policy 1, policy_version 895523 (0.0005) [2023-12-26 21:52:57,233][105620] Updated weights for policy 1, policy_version 895533 (0.0008) [2023-12-26 21:52:57,287][105620] Updated weights for policy 1, policy_version 895543 (0.0007) [2023-12-26 21:52:57,812][105692] Updated weights for policy 0, policy_version 895665 (0.0010) [2023-12-26 21:52:57,868][105692] Updated weights for policy 0, policy_version 895675 (0.0006) [2023-12-26 21:52:57,924][105692] Updated weights for policy 0, policy_version 895685 (0.0007) [2023-12-26 21:52:57,978][105692] Updated weights for policy 0, policy_version 895695 (0.0010) [2023-12-26 21:52:58,010][105620] Updated weights for policy 1, policy_version 895553 (0.0006) [2023-12-26 21:52:58,056][105620] Updated weights for policy 1, policy_version 895563 (0.0006) [2023-12-26 21:52:58,100][105620] Updated weights for policy 1, policy_version 895573 (0.0008) [2023-12-26 21:52:58,720][105692] Updated weights for policy 0, policy_version 895705 (0.0010) [2023-12-26 21:52:58,784][105692] Updated weights for policy 0, policy_version 895715 (0.0010) [2023-12-26 21:52:58,860][105692] Updated weights for policy 0, policy_version 895725 (0.0008) [2023-12-26 21:52:59,005][105620] Updated weights for policy 1, policy_version 895583 (0.0008) [2023-12-26 21:52:59,064][105620] Updated weights for policy 1, policy_version 895593 (0.0008) [2023-12-26 21:52:59,127][105620] Updated weights for policy 1, policy_version 895603 (0.0008) [2023-12-26 21:52:59,625][105692] Updated weights for policy 0, policy_version 895735 (0.0009) [2023-12-26 21:52:59,671][105692] Updated weights for policy 0, policy_version 895745 (0.0009) [2023-12-26 21:52:59,729][105692] Updated weights for policy 0, policy_version 895755 (0.0009) [2023-12-26 21:52:59,909][105620] Updated weights for policy 1, policy_version 895613 (0.0009) [2023-12-26 21:52:59,973][105620] Updated weights for policy 1, policy_version 895623 (0.0009) [2023-12-26 21:53:00,030][105620] Updated weights for policy 1, policy_version 895633 (0.0010) [2023-12-26 21:53:00,511][105692] Updated weights for policy 0, policy_version 895765 (0.0009) [2023-12-26 21:53:00,572][105692] Updated weights for policy 0, policy_version 895775 (0.0009) [2023-12-26 21:53:00,629][105692] Updated weights for policy 0, policy_version 895785 (0.0009) [2023-12-26 21:53:00,664][105620] Updated weights for policy 1, policy_version 895643 (0.0008) [2023-12-26 21:53:00,725][105620] Updated weights for policy 1, policy_version 895653 (0.0005) [2023-12-26 21:53:00,780][105620] Updated weights for policy 1, policy_version 895663 (0.0005) [2023-12-26 21:53:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 458678272. Throughput: 0: 9617.5, 1: 9610.9. Samples: 458648168. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-26 21:53:01,062][104569] Avg episode reward: [(0, '9281.372'), (1, '8876.375')] [2023-12-26 21:53:01,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000895792_229359616.pth... [2023-12-26 21:53:01,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000895672_229318656.pth... [2023-12-26 21:53:01,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000894672_229072896.pth [2023-12-26 21:53:01,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000894552_229031936.pth [2023-12-26 21:53:01,424][105692] Updated weights for policy 0, policy_version 895795 (0.0009) [2023-12-26 21:53:01,452][105620] Updated weights for policy 1, policy_version 895673 (0.0005) [2023-12-26 21:53:01,482][105692] Updated weights for policy 0, policy_version 895805 (0.0008) [2023-12-26 21:53:01,510][105620] Updated weights for policy 1, policy_version 895683 (0.0005) [2023-12-26 21:53:01,543][105692] Updated weights for policy 0, policy_version 895815 (0.0009) [2023-12-26 21:53:01,569][105620] Updated weights for policy 1, policy_version 895693 (0.0005) [2023-12-26 21:53:01,630][105620] Updated weights for policy 1, policy_version 895703 (0.0006) [2023-12-26 21:53:02,285][105620] Updated weights for policy 1, policy_version 895713 (0.0008) [2023-12-26 21:53:02,346][105620] Updated weights for policy 1, policy_version 895723 (0.0008) [2023-12-26 21:53:02,348][105692] Updated weights for policy 0, policy_version 895825 (0.0009) [2023-12-26 21:53:02,408][105620] Updated weights for policy 1, policy_version 895733 (0.0009) [2023-12-26 21:53:02,409][105692] Updated weights for policy 0, policy_version 895835 (0.0011) [2023-12-26 21:53:02,464][105692] Updated weights for policy 0, policy_version 895845 (0.0010) [2023-12-26 21:53:02,519][105692] Updated weights for policy 0, policy_version 895855 (0.0010) [2023-12-26 21:53:03,157][105620] Updated weights for policy 1, policy_version 895743 (0.0009) [2023-12-26 21:53:03,202][105620] Updated weights for policy 1, policy_version 895753 (0.0008) [2023-12-26 21:53:03,247][105620] Updated weights for policy 1, policy_version 895763 (0.0006) [2023-12-26 21:53:03,257][105692] Updated weights for policy 0, policy_version 895865 (0.0008) [2023-12-26 21:53:03,308][105692] Updated weights for policy 0, policy_version 895875 (0.0007) [2023-12-26 21:53:03,358][105692] Updated weights for policy 0, policy_version 895885 (0.0009) [2023-12-26 21:53:03,971][105692] Updated weights for policy 0, policy_version 895895 (0.0006) [2023-12-26 21:53:04,030][105692] Updated weights for policy 0, policy_version 895905 (0.0005) [2023-12-26 21:53:04,092][105692] Updated weights for policy 0, policy_version 895915 (0.0006) [2023-12-26 21:53:04,099][105620] Updated weights for policy 1, policy_version 895773 (0.0008) [2023-12-26 21:53:04,164][105620] Updated weights for policy 1, policy_version 895783 (0.0008) [2023-12-26 21:53:04,224][105620] Updated weights for policy 1, policy_version 895793 (0.0009) [2023-12-26 21:53:04,786][105692] Updated weights for policy 0, policy_version 895925 (0.0006) [2023-12-26 21:53:04,831][105692] Updated weights for policy 0, policy_version 895935 (0.0005) [2023-12-26 21:53:04,892][105692] Updated weights for policy 0, policy_version 895945 (0.0005) [2023-12-26 21:53:04,909][105620] Updated weights for policy 1, policy_version 895803 (0.0008) [2023-12-26 21:53:04,968][105620] Updated weights for policy 1, policy_version 895813 (0.0010) [2023-12-26 21:53:05,025][105620] Updated weights for policy 1, policy_version 895823 (0.0009) [2023-12-26 21:53:05,495][105692] Updated weights for policy 0, policy_version 895955 (0.0005) [2023-12-26 21:53:05,545][105692] Updated weights for policy 0, policy_version 895965 (0.0005) [2023-12-26 21:53:05,595][105692] Updated weights for policy 0, policy_version 895975 (0.0005) [2023-12-26 21:53:05,890][105620] Updated weights for policy 1, policy_version 895834 (0.0010) [2023-12-26 21:53:05,947][105620] Updated weights for policy 1, policy_version 895844 (0.0010) [2023-12-26 21:53:05,998][105620] Updated weights for policy 1, policy_version 895854 (0.0010) [2023-12-26 21:53:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 458776576. Throughput: 0: 9545.3, 1: 9652.1. Samples: 458762180. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-26 21:53:06,062][104569] Avg episode reward: [(0, '6360.485'), (1, '8871.671')] [2023-12-26 21:53:06,107][105692] Updated weights for policy 0, policy_version 895985 (0.0006) [2023-12-26 21:53:06,167][105692] Updated weights for policy 0, policy_version 895995 (0.0009) [2023-12-26 21:53:06,229][105692] Updated weights for policy 0, policy_version 896005 (0.0009) [2023-12-26 21:53:06,288][105692] Updated weights for policy 0, policy_version 896015 (0.0009) [2023-12-26 21:53:06,695][105620] Updated weights for policy 1, policy_version 895865 (0.0010) [2023-12-26 21:53:06,765][105620] Updated weights for policy 1, policy_version 895875 (0.0007) [2023-12-26 21:53:06,833][105620] Updated weights for policy 1, policy_version 895885 (0.0007) [2023-12-26 21:53:06,901][105620] Updated weights for policy 1, policy_version 895895 (0.0006) [2023-12-26 21:53:07,050][105692] Updated weights for policy 0, policy_version 896025 (0.0005) [2023-12-26 21:53:07,113][105692] Updated weights for policy 0, policy_version 896035 (0.0006) [2023-12-26 21:53:07,168][105692] Updated weights for policy 0, policy_version 896045 (0.0009) [2023-12-26 21:53:07,584][105620] Updated weights for policy 1, policy_version 895905 (0.0009) [2023-12-26 21:53:07,649][105620] Updated weights for policy 1, policy_version 895915 (0.0009) [2023-12-26 21:53:07,716][105620] Updated weights for policy 1, policy_version 895925 (0.0008) [2023-12-26 21:53:07,850][105692] Updated weights for policy 0, policy_version 896055 (0.0008) [2023-12-26 21:53:07,897][105692] Updated weights for policy 0, policy_version 896065 (0.0009) [2023-12-26 21:53:07,952][105692] Updated weights for policy 0, policy_version 896075 (0.0010) [2023-12-26 21:53:08,390][105620] Updated weights for policy 1, policy_version 895935 (0.0007) [2023-12-26 21:53:08,448][105620] Updated weights for policy 1, policy_version 895945 (0.0007) [2023-12-26 21:53:08,510][105620] Updated weights for policy 1, policy_version 895955 (0.0008) [2023-12-26 21:53:08,726][105692] Updated weights for policy 0, policy_version 896085 (0.0008) [2023-12-26 21:53:08,791][105692] Updated weights for policy 0, policy_version 896095 (0.0005) [2023-12-26 21:53:08,845][105692] Updated weights for policy 0, policy_version 896105 (0.0009) [2023-12-26 21:53:09,290][105620] Updated weights for policy 1, policy_version 895965 (0.0008) [2023-12-26 21:53:09,353][105620] Updated weights for policy 1, policy_version 895975 (0.0008) [2023-12-26 21:53:09,421][105620] Updated weights for policy 1, policy_version 895985 (0.0008) [2023-12-26 21:53:09,536][105692] Updated weights for policy 0, policy_version 896115 (0.0010) [2023-12-26 21:53:09,604][105692] Updated weights for policy 0, policy_version 896125 (0.0007) [2023-12-26 21:53:09,667][105692] Updated weights for policy 0, policy_version 896135 (0.0006) [2023-12-26 21:53:10,202][105620] Updated weights for policy 1, policy_version 895995 (0.0007) [2023-12-26 21:53:10,265][105620] Updated weights for policy 1, policy_version 896005 (0.0006) [2023-12-26 21:53:10,327][105620] Updated weights for policy 1, policy_version 896015 (0.0009) [2023-12-26 21:53:10,357][105692] Updated weights for policy 0, policy_version 896145 (0.0009) [2023-12-26 21:53:10,418][105692] Updated weights for policy 0, policy_version 896155 (0.0011) [2023-12-26 21:53:10,476][105692] Updated weights for policy 0, policy_version 896165 (0.0008) [2023-12-26 21:53:10,541][105692] Updated weights for policy 0, policy_version 896175 (0.0005) [2023-12-26 21:53:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19410.9). Total num frames: 458866688. Throughput: 0: 9666.5, 1: 9623.7. Samples: 458878816. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-26 21:53:11,063][104569] Avg episode reward: [(0, '3226.091'), (1, '8987.997')] [2023-12-26 21:53:11,105][105620] Updated weights for policy 1, policy_version 896025 (0.0007) [2023-12-26 21:53:11,173][105620] Updated weights for policy 1, policy_version 896035 (0.0008) [2023-12-26 21:53:11,216][105692] Updated weights for policy 0, policy_version 896185 (0.0007) [2023-12-26 21:53:11,233][105620] Updated weights for policy 1, policy_version 896045 (0.0006) [2023-12-26 21:53:11,280][105692] Updated weights for policy 0, policy_version 896195 (0.0009) [2023-12-26 21:53:11,304][105620] Updated weights for policy 1, policy_version 896055 (0.0007) [2023-12-26 21:53:11,340][105692] Updated weights for policy 0, policy_version 896205 (0.0009) [2023-12-26 21:53:12,077][105620] Updated weights for policy 1, policy_version 896065 (0.0009) [2023-12-26 21:53:12,136][105620] Updated weights for policy 1, policy_version 896075 (0.0008) [2023-12-26 21:53:12,151][105692] Updated weights for policy 0, policy_version 896215 (0.0007) [2023-12-26 21:53:12,199][105620] Updated weights for policy 1, policy_version 896085 (0.0008) [2023-12-26 21:53:12,204][105692] Updated weights for policy 0, policy_version 896225 (0.0007) [2023-12-26 21:53:12,265][105692] Updated weights for policy 0, policy_version 896235 (0.0008) [2023-12-26 21:53:12,912][105620] Updated weights for policy 1, policy_version 896095 (0.0009) [2023-12-26 21:53:12,974][105620] Updated weights for policy 1, policy_version 896105 (0.0007) [2023-12-26 21:53:13,026][105620] Updated weights for policy 1, policy_version 896115 (0.0006) [2023-12-26 21:53:13,060][105692] Updated weights for policy 0, policy_version 896245 (0.0010) [2023-12-26 21:53:13,111][105692] Updated weights for policy 0, policy_version 896255 (0.0009) [2023-12-26 21:53:13,171][105692] Updated weights for policy 0, policy_version 896265 (0.0007) [2023-12-26 21:53:13,758][105620] Updated weights for policy 1, policy_version 896125 (0.0007) [2023-12-26 21:53:13,818][105620] Updated weights for policy 1, policy_version 896135 (0.0008) [2023-12-26 21:53:13,886][105620] Updated weights for policy 1, policy_version 896145 (0.0010) [2023-12-26 21:53:13,907][105692] Updated weights for policy 0, policy_version 896275 (0.0005) [2023-12-26 21:53:13,967][105692] Updated weights for policy 0, policy_version 896285 (0.0009) [2023-12-26 21:53:14,026][105692] Updated weights for policy 0, policy_version 896295 (0.0010) [2023-12-26 21:53:14,668][105620] Updated weights for policy 1, policy_version 896155 (0.0008) [2023-12-26 21:53:14,694][105692] Updated weights for policy 0, policy_version 896305 (0.0011) [2023-12-26 21:53:14,728][105620] Updated weights for policy 1, policy_version 896165 (0.0008) [2023-12-26 21:53:14,743][105692] Updated weights for policy 0, policy_version 896315 (0.0007) [2023-12-26 21:53:14,788][105620] Updated weights for policy 1, policy_version 896175 (0.0008) [2023-12-26 21:53:14,801][105692] Updated weights for policy 0, policy_version 896325 (0.0008) [2023-12-26 21:53:14,859][105692] Updated weights for policy 0, policy_version 896335 (0.0008) [2023-12-26 21:53:15,540][105620] Updated weights for policy 1, policy_version 896185 (0.0008) [2023-12-26 21:53:15,600][105620] Updated weights for policy 1, policy_version 896195 (0.0007) [2023-12-26 21:53:15,615][105692] Updated weights for policy 0, policy_version 896346 (0.0008) [2023-12-26 21:53:15,656][105620] Updated weights for policy 1, policy_version 896205 (0.0009) [2023-12-26 21:53:15,672][105692] Updated weights for policy 0, policy_version 896356 (0.0005) [2023-12-26 21:53:15,715][105620] Updated weights for policy 1, policy_version 896216 (0.0010) [2023-12-26 21:53:15,730][105692] Updated weights for policy 0, policy_version 896366 (0.0006) [2023-12-26 21:53:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.1, 300 sec: 19438.6). Total num frames: 458964992. Throughput: 0: 9702.5, 1: 9528.1. Samples: 458934132. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-26 21:53:16,063][104569] Avg episode reward: [(0, '7125.975'), (1, '8244.153')] [2023-12-26 21:53:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000896368_229507072.pth... [2023-12-26 21:53:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000896216_229457920.pth... [2023-12-26 21:53:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000895216_229212160.pth [2023-12-26 21:53:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000895128_229179392.pth [2023-12-26 21:53:16,395][105692] Updated weights for policy 0, policy_version 896376 (0.0008) [2023-12-26 21:53:16,457][105692] Updated weights for policy 0, policy_version 896386 (0.0009) [2023-12-26 21:53:16,518][105692] Updated weights for policy 0, policy_version 896396 (0.0008) [2023-12-26 21:53:16,533][105620] Updated weights for policy 1, policy_version 896226 (0.0009) [2023-12-26 21:53:16,584][105620] Updated weights for policy 1, policy_version 896236 (0.0008) [2023-12-26 21:53:16,642][105620] Updated weights for policy 1, policy_version 896246 (0.0009) [2023-12-26 21:53:17,276][105692] Updated weights for policy 0, policy_version 896406 (0.0008) [2023-12-26 21:53:17,337][105692] Updated weights for policy 0, policy_version 896416 (0.0009) [2023-12-26 21:53:17,391][105692] Updated weights for policy 0, policy_version 896426 (0.0007) [2023-12-26 21:53:17,404][105620] Updated weights for policy 1, policy_version 896256 (0.0008) [2023-12-26 21:53:17,466][105620] Updated weights for policy 1, policy_version 896266 (0.0009) [2023-12-26 21:53:17,523][105620] Updated weights for policy 1, policy_version 896276 (0.0009) [2023-12-26 21:53:18,008][105692] Updated weights for policy 0, policy_version 896436 (0.0006) [2023-12-26 21:53:18,063][105692] Updated weights for policy 0, policy_version 896446 (0.0009) [2023-12-26 21:53:18,112][105692] Updated weights for policy 0, policy_version 896456 (0.0009) [2023-12-26 21:53:18,310][105620] Updated weights for policy 1, policy_version 896286 (0.0010) [2023-12-26 21:53:18,374][105620] Updated weights for policy 1, policy_version 896296 (0.0009) [2023-12-26 21:53:18,438][105620] Updated weights for policy 1, policy_version 896306 (0.0009) [2023-12-26 21:53:18,866][105692] Updated weights for policy 0, policy_version 896466 (0.0010) [2023-12-26 21:53:18,929][105692] Updated weights for policy 0, policy_version 896476 (0.0009) [2023-12-26 21:53:18,993][105692] Updated weights for policy 0, policy_version 896486 (0.0009) [2023-12-26 21:53:19,055][105692] Updated weights for policy 0, policy_version 896496 (0.0009) [2023-12-26 21:53:19,191][105620] Updated weights for policy 1, policy_version 896316 (0.0009) [2023-12-26 21:53:19,257][105620] Updated weights for policy 1, policy_version 896326 (0.0009) [2023-12-26 21:53:19,325][105620] Updated weights for policy 1, policy_version 896336 (0.0008) [2023-12-26 21:53:19,853][105692] Updated weights for policy 0, policy_version 896506 (0.0009) [2023-12-26 21:53:19,923][105692] Updated weights for policy 0, policy_version 896516 (0.0010) [2023-12-26 21:53:19,987][105692] Updated weights for policy 0, policy_version 896526 (0.0009) [2023-12-26 21:53:20,033][105620] Updated weights for policy 1, policy_version 896346 (0.0007) [2023-12-26 21:53:20,090][105620] Updated weights for policy 1, policy_version 896356 (0.0005) [2023-12-26 21:53:20,148][105620] Updated weights for policy 1, policy_version 896366 (0.0009) [2023-12-26 21:53:20,206][105620] Updated weights for policy 1, policy_version 896376 (0.0009) [2023-12-26 21:53:20,826][105692] Updated weights for policy 0, policy_version 896536 (0.0010) [2023-12-26 21:53:20,885][105692] Updated weights for policy 0, policy_version 896546 (0.0010) [2023-12-26 21:53:20,911][105620] Updated weights for policy 1, policy_version 896386 (0.0007) [2023-12-26 21:53:20,938][105692] Updated weights for policy 0, policy_version 896556 (0.0008) [2023-12-26 21:53:20,971][105620] Updated weights for policy 1, policy_version 896396 (0.0006) [2023-12-26 21:53:21,036][105620] Updated weights for policy 1, policy_version 896406 (0.0009) [2023-12-26 21:53:21,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 459063296. Throughput: 0: 9777.8, 1: 9474.0. Samples: 459047392. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-26 21:53:21,062][104569] Avg episode reward: [(0, '2107.129'), (1, '8458.173')] [2023-12-26 21:53:21,804][105620] Updated weights for policy 1, policy_version 896416 (0.0008) [2023-12-26 21:53:21,822][105692] Updated weights for policy 0, policy_version 896566 (0.0009) [2023-12-26 21:53:21,857][105620] Updated weights for policy 1, policy_version 896426 (0.0006) [2023-12-26 21:53:21,885][105692] Updated weights for policy 0, policy_version 896576 (0.0009) [2023-12-26 21:53:21,911][105620] Updated weights for policy 1, policy_version 896436 (0.0009) [2023-12-26 21:53:21,947][105692] Updated weights for policy 0, policy_version 896586 (0.0008) [2023-12-26 21:53:22,706][105620] Updated weights for policy 1, policy_version 896446 (0.0008) [2023-12-26 21:53:22,736][105692] Updated weights for policy 0, policy_version 896596 (0.0008) [2023-12-26 21:53:22,774][105620] Updated weights for policy 1, policy_version 896456 (0.0007) [2023-12-26 21:53:22,803][105692] Updated weights for policy 0, policy_version 896606 (0.0007) [2023-12-26 21:53:22,845][105620] Updated weights for policy 1, policy_version 896466 (0.0008) [2023-12-26 21:53:22,871][105692] Updated weights for policy 0, policy_version 896616 (0.0007) [2023-12-26 21:53:23,430][105692] Updated weights for policy 0, policy_version 896626 (0.0005) [2023-12-26 21:53:23,493][105692] Updated weights for policy 0, policy_version 896636 (0.0005) [2023-12-26 21:53:23,551][105692] Updated weights for policy 0, policy_version 896646 (0.0005) [2023-12-26 21:53:23,600][105692] Updated weights for policy 0, policy_version 896656 (0.0005) [2023-12-26 21:53:23,653][105620] Updated weights for policy 1, policy_version 896476 (0.0008) [2023-12-26 21:53:23,713][105620] Updated weights for policy 1, policy_version 896486 (0.0009) [2023-12-26 21:53:23,774][105620] Updated weights for policy 1, policy_version 896496 (0.0009) [2023-12-26 21:53:24,168][105692] Updated weights for policy 0, policy_version 896666 (0.0007) [2023-12-26 21:53:24,230][105692] Updated weights for policy 0, policy_version 896676 (0.0009) [2023-12-26 21:53:24,288][105692] Updated weights for policy 0, policy_version 896686 (0.0009) [2023-12-26 21:53:24,546][105620] Updated weights for policy 1, policy_version 896506 (0.0009) [2023-12-26 21:53:24,623][105620] Updated weights for policy 1, policy_version 896516 (0.0009) [2023-12-26 21:53:24,695][105620] Updated weights for policy 1, policy_version 896526 (0.0009) [2023-12-26 21:53:24,767][105620] Updated weights for policy 1, policy_version 896536 (0.0009) [2023-12-26 21:53:24,933][105692] Updated weights for policy 0, policy_version 896696 (0.0006) [2023-12-26 21:53:24,989][105692] Updated weights for policy 0, policy_version 896706 (0.0008) [2023-12-26 21:53:25,046][105692] Updated weights for policy 0, policy_version 896716 (0.0009) [2023-12-26 21:53:25,411][105620] Updated weights for policy 1, policy_version 896546 (0.0005) [2023-12-26 21:53:25,471][105620] Updated weights for policy 1, policy_version 896556 (0.0005) [2023-12-26 21:53:25,526][105620] Updated weights for policy 1, policy_version 896566 (0.0009) [2023-12-26 21:53:25,809][105692] Updated weights for policy 0, policy_version 896726 (0.0009) [2023-12-26 21:53:25,879][105692] Updated weights for policy 0, policy_version 896736 (0.0006) [2023-12-26 21:53:25,944][105692] Updated weights for policy 0, policy_version 896746 (0.0008) [2023-12-26 21:53:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 459153408. Throughput: 0: 9774.9, 1: 9386.7. Samples: 459161664. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-26 21:53:26,062][104569] Avg episode reward: [(0, '6079.895'), (1, '8717.841')] [2023-12-26 21:53:26,169][105620] Updated weights for policy 1, policy_version 896576 (0.0009) [2023-12-26 21:53:26,229][105620] Updated weights for policy 1, policy_version 896587 (0.0010) [2023-12-26 21:53:26,289][105620] Updated weights for policy 1, policy_version 896597 (0.0009) [2023-12-26 21:53:26,568][105692] Updated weights for policy 0, policy_version 896756 (0.0008) [2023-12-26 21:53:26,624][105692] Updated weights for policy 0, policy_version 896766 (0.0005) [2023-12-26 21:53:26,679][105692] Updated weights for policy 0, policy_version 896776 (0.0006) [2023-12-26 21:53:27,038][105620] Updated weights for policy 1, policy_version 896607 (0.0009) [2023-12-26 21:53:27,090][105620] Updated weights for policy 1, policy_version 896617 (0.0010) [2023-12-26 21:53:27,150][105620] Updated weights for policy 1, policy_version 896627 (0.0008) [2023-12-26 21:53:27,242][105692] Updated weights for policy 0, policy_version 896786 (0.0007) [2023-12-26 21:53:27,299][105692] Updated weights for policy 0, policy_version 896796 (0.0009) [2023-12-26 21:53:27,359][105692] Updated weights for policy 0, policy_version 896806 (0.0005) [2023-12-26 21:53:27,424][105692] Updated weights for policy 0, policy_version 896816 (0.0007) [2023-12-26 21:53:27,879][105620] Updated weights for policy 1, policy_version 896637 (0.0007) [2023-12-26 21:53:27,929][105620] Updated weights for policy 1, policy_version 896647 (0.0005) [2023-12-26 21:53:27,988][105620] Updated weights for policy 1, policy_version 896657 (0.0005) [2023-12-26 21:53:28,209][105692] Updated weights for policy 0, policy_version 896826 (0.0009) [2023-12-26 21:53:28,276][105692] Updated weights for policy 0, policy_version 896836 (0.0008) [2023-12-26 21:53:28,334][105692] Updated weights for policy 0, policy_version 896846 (0.0006) [2023-12-26 21:53:28,598][105620] Updated weights for policy 1, policy_version 896667 (0.0006) [2023-12-26 21:53:28,653][105620] Updated weights for policy 1, policy_version 896677 (0.0009) [2023-12-26 21:53:28,717][105620] Updated weights for policy 1, policy_version 896687 (0.0009) [2023-12-26 21:53:28,969][105692] Updated weights for policy 0, policy_version 896856 (0.0009) [2023-12-26 21:53:29,023][105692] Updated weights for policy 0, policy_version 896866 (0.0009) [2023-12-26 21:53:29,070][105692] Updated weights for policy 0, policy_version 896876 (0.0009) [2023-12-26 21:53:29,478][105620] Updated weights for policy 1, policy_version 896697 (0.0009) [2023-12-26 21:53:29,535][105620] Updated weights for policy 1, policy_version 896707 (0.0009) [2023-12-26 21:53:29,597][105620] Updated weights for policy 1, policy_version 896717 (0.0009) [2023-12-26 21:53:29,649][105620] Updated weights for policy 1, policy_version 896727 (0.0009) [2023-12-26 21:53:29,832][105692] Updated weights for policy 0, policy_version 896886 (0.0008) [2023-12-26 21:53:29,898][105692] Updated weights for policy 0, policy_version 896896 (0.0008) [2023-12-26 21:53:29,968][105692] Updated weights for policy 0, policy_version 896906 (0.0008) [2023-12-26 21:53:30,347][105620] Updated weights for policy 1, policy_version 896737 (0.0009) [2023-12-26 21:53:30,357][105586] KL-divergence is very high: 147.6368 [2023-12-26 21:53:30,376][105586] KL-divergence is very high: 120.7753 [2023-12-26 21:53:30,407][105620] Updated weights for policy 1, policy_version 896747 (0.0009) [2023-12-26 21:53:30,408][105586] KL-divergence is very high: 224.4738 [2023-12-26 21:53:30,424][105586] KL-divergence is very high: 138.1115 [2023-12-26 21:53:30,454][105586] KL-divergence is very high: 188.4355 [2023-12-26 21:53:30,464][105620] Updated weights for policy 1, policy_version 896757 (0.0008) [2023-12-26 21:53:30,729][105692] Updated weights for policy 0, policy_version 896916 (0.0008) [2023-12-26 21:53:30,776][105692] Updated weights for policy 0, policy_version 896926 (0.0007) [2023-12-26 21:53:30,820][105692] Updated weights for policy 0, policy_version 896936 (0.0005) [2023-12-26 21:53:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 459251712. Throughput: 0: 9803.0, 1: 9429.5. Samples: 459222028. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-26 21:53:31,062][104569] Avg episode reward: [(0, '7228.946'), (1, '8087.617')] [2023-12-26 21:53:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000896944_229654528.pth... [2023-12-26 21:53:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000896760_229597184.pth... [2023-12-26 21:53:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000895672_229318656.pth [2023-12-26 21:53:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000895792_229359616.pth [2023-12-26 21:53:31,281][105620] Updated weights for policy 1, policy_version 896767 (0.0010) [2023-12-26 21:53:31,347][105620] Updated weights for policy 1, policy_version 896777 (0.0009) [2023-12-26 21:53:31,415][105620] Updated weights for policy 1, policy_version 896787 (0.0008) [2023-12-26 21:53:31,468][105692] Updated weights for policy 0, policy_version 896946 (0.0006) [2023-12-26 21:53:31,532][105692] Updated weights for policy 0, policy_version 896956 (0.0009) [2023-12-26 21:53:31,592][105692] Updated weights for policy 0, policy_version 896966 (0.0008) [2023-12-26 21:53:31,657][105692] Updated weights for policy 0, policy_version 896976 (0.0010) [2023-12-26 21:53:32,202][105620] Updated weights for policy 1, policy_version 896797 (0.0009) [2023-12-26 21:53:32,262][105620] Updated weights for policy 1, policy_version 896807 (0.0008) [2023-12-26 21:53:32,324][105620] Updated weights for policy 1, policy_version 896817 (0.0008) [2023-12-26 21:53:32,348][105692] Updated weights for policy 0, policy_version 896986 (0.0006) [2023-12-26 21:53:32,400][105692] Updated weights for policy 0, policy_version 896996 (0.0009) [2023-12-26 21:53:32,453][105692] Updated weights for policy 0, policy_version 897006 (0.0009) [2023-12-26 21:53:33,023][105620] Updated weights for policy 1, policy_version 896827 (0.0006) [2023-12-26 21:53:33,088][105620] Updated weights for policy 1, policy_version 896837 (0.0005) [2023-12-26 21:53:33,160][105620] Updated weights for policy 1, policy_version 896847 (0.0007) [2023-12-26 21:53:33,245][105692] Updated weights for policy 0, policy_version 897016 (0.0006) [2023-12-26 21:53:33,301][105692] Updated weights for policy 0, policy_version 897026 (0.0005) [2023-12-26 21:53:33,359][105692] Updated weights for policy 0, policy_version 897036 (0.0005) [2023-12-26 21:53:33,838][105620] Updated weights for policy 1, policy_version 896857 (0.0008) [2023-12-26 21:53:33,881][105692] Updated weights for policy 0, policy_version 897046 (0.0007) [2023-12-26 21:53:33,882][105620] Updated weights for policy 1, policy_version 896867 (0.0010) [2023-12-26 21:53:33,930][105620] Updated weights for policy 1, policy_version 896877 (0.0010) [2023-12-26 21:53:33,943][105692] Updated weights for policy 0, policy_version 897056 (0.0006) [2023-12-26 21:53:33,971][105620] Updated weights for policy 1, policy_version 896887 (0.0010) [2023-12-26 21:53:34,007][105692] Updated weights for policy 0, policy_version 897066 (0.0005) [2023-12-26 21:53:34,594][105692] Updated weights for policy 0, policy_version 897076 (0.0007) [2023-12-26 21:53:34,660][105692] Updated weights for policy 0, policy_version 897086 (0.0010) [2023-12-26 21:53:34,726][105692] Updated weights for policy 0, policy_version 897096 (0.0010) [2023-12-26 21:53:34,769][105620] Updated weights for policy 1, policy_version 896897 (0.0011) [2023-12-26 21:53:34,824][105620] Updated weights for policy 1, policy_version 896907 (0.0010) [2023-12-26 21:53:34,873][105620] Updated weights for policy 1, policy_version 896917 (0.0009) [2023-12-26 21:53:35,396][105692] Updated weights for policy 0, policy_version 897106 (0.0011) [2023-12-26 21:53:35,450][105692] Updated weights for policy 0, policy_version 897116 (0.0010) [2023-12-26 21:53:35,504][105692] Updated weights for policy 0, policy_version 897126 (0.0010) [2023-12-26 21:53:35,552][105692] Updated weights for policy 0, policy_version 897136 (0.0010) [2023-12-26 21:53:35,608][105620] Updated weights for policy 1, policy_version 896927 (0.0010) [2023-12-26 21:53:35,655][105620] Updated weights for policy 1, policy_version 896937 (0.0010) [2023-12-26 21:53:35,710][105620] Updated weights for policy 1, policy_version 896947 (0.0010) [2023-12-26 21:53:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 459350016. Throughput: 0: 9820.7, 1: 9448.4. Samples: 459339760. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-26 21:53:36,062][104569] Avg episode reward: [(0, '9176.234'), (1, '8266.458')] [2023-12-26 21:53:36,316][105692] Updated weights for policy 0, policy_version 897146 (0.0007) [2023-12-26 21:53:36,368][105692] Updated weights for policy 0, policy_version 897156 (0.0010) [2023-12-26 21:53:36,417][105692] Updated weights for policy 0, policy_version 897166 (0.0010) [2023-12-26 21:53:36,454][105620] Updated weights for policy 1, policy_version 896957 (0.0010) [2023-12-26 21:53:36,510][105620] Updated weights for policy 1, policy_version 896967 (0.0010) [2023-12-26 21:53:36,570][105620] Updated weights for policy 1, policy_version 896977 (0.0010) [2023-12-26 21:53:37,187][105692] Updated weights for policy 0, policy_version 897176 (0.0008) [2023-12-26 21:53:37,239][105692] Updated weights for policy 0, policy_version 897186 (0.0008) [2023-12-26 21:53:37,287][105692] Updated weights for policy 0, policy_version 897196 (0.0008) [2023-12-26 21:53:37,294][105620] Updated weights for policy 1, policy_version 896987 (0.0011) [2023-12-26 21:53:37,359][105620] Updated weights for policy 1, policy_version 896997 (0.0011) [2023-12-26 21:53:37,418][105620] Updated weights for policy 1, policy_version 897007 (0.0010) [2023-12-26 21:53:38,012][105692] Updated weights for policy 0, policy_version 897206 (0.0008) [2023-12-26 21:53:38,074][105692] Updated weights for policy 0, policy_version 897216 (0.0009) [2023-12-26 21:53:38,127][105620] Updated weights for policy 1, policy_version 897017 (0.0006) [2023-12-26 21:53:38,131][105692] Updated weights for policy 0, policy_version 897226 (0.0008) [2023-12-26 21:53:38,187][105620] Updated weights for policy 1, policy_version 897027 (0.0009) [2023-12-26 21:53:38,246][105620] Updated weights for policy 1, policy_version 897037 (0.0009) [2023-12-26 21:53:38,297][105620] Updated weights for policy 1, policy_version 897047 (0.0009) [2023-12-26 21:53:38,871][105692] Updated weights for policy 0, policy_version 897236 (0.0010) [2023-12-26 21:53:38,933][105692] Updated weights for policy 0, policy_version 897246 (0.0007) [2023-12-26 21:53:38,988][105692] Updated weights for policy 0, policy_version 897256 (0.0007) [2023-12-26 21:53:39,069][105620] Updated weights for policy 1, policy_version 897057 (0.0008) [2023-12-26 21:53:39,123][105620] Updated weights for policy 1, policy_version 897067 (0.0009) [2023-12-26 21:53:39,175][105620] Updated weights for policy 1, policy_version 897077 (0.0009) [2023-12-26 21:53:39,727][105692] Updated weights for policy 0, policy_version 897266 (0.0009) [2023-12-26 21:53:39,787][105692] Updated weights for policy 0, policy_version 897276 (0.0008) [2023-12-26 21:53:39,857][105692] Updated weights for policy 0, policy_version 897286 (0.0007) [2023-12-26 21:53:39,928][105692] Updated weights for policy 0, policy_version 897296 (0.0007) [2023-12-26 21:53:40,015][105620] Updated weights for policy 1, policy_version 897087 (0.0009) [2023-12-26 21:53:40,085][105620] Updated weights for policy 1, policy_version 897097 (0.0009) [2023-12-26 21:53:40,147][105620] Updated weights for policy 1, policy_version 897107 (0.0009) [2023-12-26 21:53:40,580][105692] Updated weights for policy 0, policy_version 897306 (0.0009) [2023-12-26 21:53:40,636][105692] Updated weights for policy 0, policy_version 897316 (0.0010) [2023-12-26 21:53:40,688][105692] Updated weights for policy 0, policy_version 897326 (0.0010) [2023-12-26 21:53:40,811][105620] Updated weights for policy 1, policy_version 897117 (0.0006) [2023-12-26 21:53:40,860][105620] Updated weights for policy 1, policy_version 897127 (0.0009) [2023-12-26 21:53:40,924][105620] Updated weights for policy 1, policy_version 897137 (0.0009) [2023-12-26 21:53:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 459448320. Throughput: 0: 9861.2, 1: 9434.0. Samples: 459454580. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-26 21:53:41,062][104569] Avg episode reward: [(0, '9261.957'), (1, '8624.669')] [2023-12-26 21:53:41,493][105692] Updated weights for policy 0, policy_version 897336 (0.0009) [2023-12-26 21:53:41,552][105692] Updated weights for policy 0, policy_version 897346 (0.0009) [2023-12-26 21:53:41,616][105692] Updated weights for policy 0, policy_version 897356 (0.0009) [2023-12-26 21:53:41,728][105620] Updated weights for policy 1, policy_version 897147 (0.0009) [2023-12-26 21:53:41,788][105620] Updated weights for policy 1, policy_version 897157 (0.0009) [2023-12-26 21:53:41,854][105620] Updated weights for policy 1, policy_version 897167 (0.0009) [2023-12-26 21:53:42,386][105692] Updated weights for policy 0, policy_version 897366 (0.0009) [2023-12-26 21:53:42,447][105692] Updated weights for policy 0, policy_version 897376 (0.0009) [2023-12-26 21:53:42,501][105692] Updated weights for policy 0, policy_version 897386 (0.0009) [2023-12-26 21:53:42,615][105620] Updated weights for policy 1, policy_version 897177 (0.0009) [2023-12-26 21:53:42,670][105620] Updated weights for policy 1, policy_version 897187 (0.0008) [2023-12-26 21:53:42,722][105620] Updated weights for policy 1, policy_version 897197 (0.0006) [2023-12-26 21:53:42,780][105620] Updated weights for policy 1, policy_version 897207 (0.0009) [2023-12-26 21:53:43,266][105692] Updated weights for policy 0, policy_version 897396 (0.0009) [2023-12-26 21:53:43,328][105692] Updated weights for policy 0, policy_version 897406 (0.0008) [2023-12-26 21:53:43,372][105692] Updated weights for policy 0, policy_version 897416 (0.0008) [2023-12-26 21:53:43,510][105620] Updated weights for policy 1, policy_version 897217 (0.0009) [2023-12-26 21:53:43,564][105620] Updated weights for policy 1, policy_version 897227 (0.0010) [2023-12-26 21:53:43,615][105620] Updated weights for policy 1, policy_version 897237 (0.0009) [2023-12-26 21:53:44,200][105620] Updated weights for policy 1, policy_version 897247 (0.0005) [2023-12-26 21:53:44,229][105692] Updated weights for policy 0, policy_version 897426 (0.0008) [2023-12-26 21:53:44,263][105620] Updated weights for policy 1, policy_version 897257 (0.0006) [2023-12-26 21:53:44,285][105692] Updated weights for policy 0, policy_version 897436 (0.0009) [2023-12-26 21:53:44,320][105620] Updated weights for policy 1, policy_version 897267 (0.0006) [2023-12-26 21:53:44,340][105692] Updated weights for policy 0, policy_version 897446 (0.0009) [2023-12-26 21:53:44,404][105692] Updated weights for policy 0, policy_version 897456 (0.0010) [2023-12-26 21:53:44,869][105620] Updated weights for policy 1, policy_version 897277 (0.0006) [2023-12-26 21:53:44,927][105620] Updated weights for policy 1, policy_version 897287 (0.0007) [2023-12-26 21:53:44,980][105620] Updated weights for policy 1, policy_version 897297 (0.0010) [2023-12-26 21:53:45,261][105692] Updated weights for policy 0, policy_version 897466 (0.0008) [2023-12-26 21:53:45,332][105692] Updated weights for policy 0, policy_version 897476 (0.0008) [2023-12-26 21:53:45,398][105692] Updated weights for policy 0, policy_version 897486 (0.0008) [2023-12-26 21:53:45,656][105620] Updated weights for policy 1, policy_version 897307 (0.0010) [2023-12-26 21:53:45,710][105620] Updated weights for policy 1, policy_version 897317 (0.0008) [2023-12-26 21:53:45,758][105620] Updated weights for policy 1, policy_version 897327 (0.0009) [2023-12-26 21:53:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 459538432. Throughput: 0: 9737.5, 1: 9412.6. Samples: 459509924. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-26 21:53:46,063][104569] Avg episode reward: [(0, '9262.512'), (1, '8533.604')] [2023-12-26 21:53:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000897488_229793792.pth... [2023-12-26 21:53:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000897336_229744640.pth... [2023-12-26 21:53:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000896368_229507072.pth [2023-12-26 21:53:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000896216_229457920.pth [2023-12-26 21:53:46,142][105692] Updated weights for policy 0, policy_version 897496 (0.0009) [2023-12-26 21:53:46,200][105692] Updated weights for policy 0, policy_version 897506 (0.0009) [2023-12-26 21:53:46,261][105692] Updated weights for policy 0, policy_version 897516 (0.0009) [2023-12-26 21:53:46,527][105620] Updated weights for policy 1, policy_version 897337 (0.0009) [2023-12-26 21:53:46,602][105620] Updated weights for policy 1, policy_version 897347 (0.0006) [2023-12-26 21:53:46,666][105620] Updated weights for policy 1, policy_version 897357 (0.0005) [2023-12-26 21:53:46,726][105620] Updated weights for policy 1, policy_version 897367 (0.0007) [2023-12-26 21:53:47,077][105692] Updated weights for policy 0, policy_version 897526 (0.0009) [2023-12-26 21:53:47,130][105692] Updated weights for policy 0, policy_version 897536 (0.0008) [2023-12-26 21:53:47,181][105692] Updated weights for policy 0, policy_version 897546 (0.0009) [2023-12-26 21:53:47,311][105620] Updated weights for policy 1, policy_version 897377 (0.0008) [2023-12-26 21:53:47,366][105620] Updated weights for policy 1, policy_version 897388 (0.0009) [2023-12-26 21:53:47,387][105586] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000001 [2023-12-26 21:53:47,998][105692] Updated weights for policy 0, policy_version 897556 (0.0009) [2023-12-26 21:53:48,050][105692] Updated weights for policy 0, policy_version 897566 (0.0007) [2023-12-26 21:53:48,100][105692] Updated weights for policy 0, policy_version 897576 (0.0009) [2023-12-26 21:53:48,126][105620] Updated weights for policy 1, policy_version 897398 (0.0009) [2023-12-26 21:53:48,187][105620] Updated weights for policy 1, policy_version 897408 (0.0008) [2023-12-26 21:53:48,237][105620] Updated weights for policy 1, policy_version 897418 (0.0009) [2023-12-26 21:53:48,877][105692] Updated weights for policy 0, policy_version 897586 (0.0007) [2023-12-26 21:53:48,941][105692] Updated weights for policy 0, policy_version 897596 (0.0007) [2023-12-26 21:53:48,980][105620] Updated weights for policy 1, policy_version 897428 (0.0008) [2023-12-26 21:53:48,999][105692] Updated weights for policy 0, policy_version 897606 (0.0009) [2023-12-26 21:53:49,040][105620] Updated weights for policy 1, policy_version 897438 (0.0005) [2023-12-26 21:53:49,062][105692] Updated weights for policy 0, policy_version 897616 (0.0009) [2023-12-26 21:53:49,094][105620] Updated weights for policy 1, policy_version 897448 (0.0009) [2023-12-26 21:53:49,711][105692] Updated weights for policy 0, policy_version 897626 (0.0009) [2023-12-26 21:53:49,765][105692] Updated weights for policy 0, policy_version 897636 (0.0008) [2023-12-26 21:53:49,817][105692] Updated weights for policy 0, policy_version 897646 (0.0010) [2023-12-26 21:53:49,865][105620] Updated weights for policy 1, policy_version 897458 (0.0009) [2023-12-26 21:53:49,930][105620] Updated weights for policy 1, policy_version 897468 (0.0009) [2023-12-26 21:53:49,990][105620] Updated weights for policy 1, policy_version 897478 (0.0009) [2023-12-26 21:53:50,055][105620] Updated weights for policy 1, policy_version 897488 (0.0008) [2023-12-26 21:53:50,639][105692] Updated weights for policy 0, policy_version 897656 (0.0006) [2023-12-26 21:53:50,700][105692] Updated weights for policy 0, policy_version 897666 (0.0006) [2023-12-26 21:53:50,733][105620] Updated weights for policy 1, policy_version 897498 (0.0009) [2023-12-26 21:53:50,758][105692] Updated weights for policy 0, policy_version 897676 (0.0005) [2023-12-26 21:53:50,803][105620] Updated weights for policy 1, policy_version 897508 (0.0009) [2023-12-26 21:53:50,862][105620] Updated weights for policy 1, policy_version 897518 (0.0009) [2023-12-26 21:53:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 459636736. Throughput: 0: 9678.9, 1: 9479.8. Samples: 459624320. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-26 21:53:51,062][104569] Avg episode reward: [(0, '9174.708'), (1, '8355.718')] [2023-12-26 21:53:51,366][105692] Updated weights for policy 0, policy_version 897686 (0.0008) [2023-12-26 21:53:51,430][105692] Updated weights for policy 0, policy_version 897696 (0.0009) [2023-12-26 21:53:51,492][105692] Updated weights for policy 0, policy_version 897706 (0.0010) [2023-12-26 21:53:51,604][105620] Updated weights for policy 1, policy_version 897528 (0.0010) [2023-12-26 21:53:51,668][105620] Updated weights for policy 1, policy_version 897538 (0.0010) [2023-12-26 21:53:51,737][105620] Updated weights for policy 1, policy_version 897548 (0.0010) [2023-12-26 21:53:52,304][105692] Updated weights for policy 0, policy_version 897716 (0.0010) [2023-12-26 21:53:52,367][105692] Updated weights for policy 0, policy_version 897726 (0.0009) [2023-12-26 21:53:52,429][105692] Updated weights for policy 0, policy_version 897736 (0.0007) [2023-12-26 21:53:52,476][105620] Updated weights for policy 1, policy_version 897558 (0.0008) [2023-12-26 21:53:52,533][105620] Updated weights for policy 1, policy_version 897568 (0.0008) [2023-12-26 21:53:52,594][105620] Updated weights for policy 1, policy_version 897578 (0.0008) [2023-12-26 21:53:53,207][105692] Updated weights for policy 0, policy_version 897746 (0.0008) [2023-12-26 21:53:53,265][105692] Updated weights for policy 0, policy_version 897756 (0.0010) [2023-12-26 21:53:53,320][105620] Updated weights for policy 1, policy_version 897588 (0.0008) [2023-12-26 21:53:53,325][105692] Updated weights for policy 0, policy_version 897767 (0.0009) [2023-12-26 21:53:53,382][105620] Updated weights for policy 1, policy_version 897598 (0.0006) [2023-12-26 21:53:53,442][105620] Updated weights for policy 1, policy_version 897608 (0.0008) [2023-12-26 21:53:54,081][105692] Updated weights for policy 0, policy_version 897777 (0.0009) [2023-12-26 21:53:54,135][105692] Updated weights for policy 0, policy_version 897787 (0.0009) [2023-12-26 21:53:54,161][105620] Updated weights for policy 1, policy_version 897618 (0.0009) [2023-12-26 21:53:54,198][105692] Updated weights for policy 0, policy_version 897797 (0.0009) [2023-12-26 21:53:54,220][105620] Updated weights for policy 1, policy_version 897628 (0.0007) [2023-12-26 21:53:54,259][105692] Updated weights for policy 0, policy_version 897807 (0.0008) [2023-12-26 21:53:54,272][105620] Updated weights for policy 1, policy_version 897638 (0.0007) [2023-12-26 21:53:54,318][105620] Updated weights for policy 1, policy_version 897648 (0.0008) [2023-12-26 21:53:55,018][105692] Updated weights for policy 0, policy_version 897817 (0.0009) [2023-12-26 21:53:55,061][105620] Updated weights for policy 1, policy_version 897658 (0.0008) [2023-12-26 21:53:55,075][105692] Updated weights for policy 0, policy_version 897827 (0.0007) [2023-12-26 21:53:55,114][105620] Updated weights for policy 1, policy_version 897668 (0.0007) [2023-12-26 21:53:55,125][105692] Updated weights for policy 0, policy_version 897837 (0.0008) [2023-12-26 21:53:55,177][105620] Updated weights for policy 1, policy_version 897678 (0.0008) [2023-12-26 21:53:55,901][105620] Updated weights for policy 1, policy_version 897688 (0.0008) [2023-12-26 21:53:55,904][105692] Updated weights for policy 0, policy_version 897847 (0.0006) [2023-12-26 21:53:55,961][105620] Updated weights for policy 1, policy_version 897698 (0.0007) [2023-12-26 21:53:55,963][105692] Updated weights for policy 0, policy_version 897857 (0.0006) [2023-12-26 21:53:56,009][105620] Updated weights for policy 1, policy_version 897708 (0.0005) [2023-12-26 21:53:56,020][105692] Updated weights for policy 0, policy_version 897867 (0.0008) [2023-12-26 21:53:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 459735040. Throughput: 0: 9563.2, 1: 9529.0. Samples: 459737964. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-26 21:53:56,063][104569] Avg episode reward: [(0, '8991.808'), (1, '8358.093')] [2023-12-26 21:53:56,561][105620] Updated weights for policy 1, policy_version 897718 (0.0007) [2023-12-26 21:53:56,615][105620] Updated weights for policy 1, policy_version 897728 (0.0006) [2023-12-26 21:53:56,666][105620] Updated weights for policy 1, policy_version 897738 (0.0005) [2023-12-26 21:53:56,817][105692] Updated weights for policy 0, policy_version 897877 (0.0009) [2023-12-26 21:53:56,870][105692] Updated weights for policy 0, policy_version 897887 (0.0007) [2023-12-26 21:53:56,922][105692] Updated weights for policy 0, policy_version 897897 (0.0007) [2023-12-26 21:53:57,276][105620] Updated weights for policy 1, policy_version 897748 (0.0007) [2023-12-26 21:53:57,325][105620] Updated weights for policy 1, policy_version 897758 (0.0008) [2023-12-26 21:53:57,379][105620] Updated weights for policy 1, policy_version 897768 (0.0009) [2023-12-26 21:53:57,694][105692] Updated weights for policy 0, policy_version 897907 (0.0009) [2023-12-26 21:53:57,741][105692] Updated weights for policy 0, policy_version 897917 (0.0008) [2023-12-26 21:53:57,805][105692] Updated weights for policy 0, policy_version 897927 (0.0008) [2023-12-26 21:53:58,129][105620] Updated weights for policy 1, policy_version 897778 (0.0009) [2023-12-26 21:53:58,195][105620] Updated weights for policy 1, policy_version 897788 (0.0007) [2023-12-26 21:53:58,257][105620] Updated weights for policy 1, policy_version 897798 (0.0006) [2023-12-26 21:53:58,313][105620] Updated weights for policy 1, policy_version 897808 (0.0006) [2023-12-26 21:53:58,551][105692] Updated weights for policy 0, policy_version 897937 (0.0008) [2023-12-26 21:53:58,623][105692] Updated weights for policy 0, policy_version 897947 (0.0008) [2023-12-26 21:53:58,679][105692] Updated weights for policy 0, policy_version 897957 (0.0009) [2023-12-26 21:53:58,737][105692] Updated weights for policy 0, policy_version 897967 (0.0008) [2023-12-26 21:53:59,036][105620] Updated weights for policy 1, policy_version 897818 (0.0006) [2023-12-26 21:53:59,100][105620] Updated weights for policy 1, policy_version 897828 (0.0009) [2023-12-26 21:53:59,155][105620] Updated weights for policy 1, policy_version 897838 (0.0009) [2023-12-26 21:53:59,565][105692] Updated weights for policy 0, policy_version 897977 (0.0009) [2023-12-26 21:53:59,623][105692] Updated weights for policy 0, policy_version 897987 (0.0009) [2023-12-26 21:53:59,673][105692] Updated weights for policy 0, policy_version 897997 (0.0009) [2023-12-26 21:53:59,948][105620] Updated weights for policy 1, policy_version 897848 (0.0008) [2023-12-26 21:54:00,001][105620] Updated weights for policy 1, policy_version 897858 (0.0006) [2023-12-26 21:54:00,057][105620] Updated weights for policy 1, policy_version 897868 (0.0008) [2023-12-26 21:54:00,444][105692] Updated weights for policy 0, policy_version 898007 (0.0010) [2023-12-26 21:54:00,500][105692] Updated weights for policy 0, policy_version 898017 (0.0010) [2023-12-26 21:54:00,558][105692] Updated weights for policy 0, policy_version 898027 (0.0010) [2023-12-26 21:54:00,787][105620] Updated weights for policy 1, policy_version 897878 (0.0008) [2023-12-26 21:54:00,837][105620] Updated weights for policy 1, policy_version 897888 (0.0008) [2023-12-26 21:54:00,888][105620] Updated weights for policy 1, policy_version 897898 (0.0008) [2023-12-26 21:54:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19114.7, 300 sec: 19355.3). Total num frames: 459825152. Throughput: 0: 9557.9, 1: 9610.2. Samples: 459796696. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-26 21:54:01,063][104569] Avg episode reward: [(0, '9078.775'), (1, '8356.790')] [2023-12-26 21:54:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000898032_229933056.pth... [2023-12-26 21:54:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000897904_229892096.pth... [2023-12-26 21:54:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000896944_229654528.pth [2023-12-26 21:54:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000896760_229597184.pth [2023-12-26 21:54:01,329][105692] Updated weights for policy 0, policy_version 898037 (0.0009) [2023-12-26 21:54:01,404][105692] Updated weights for policy 0, policy_version 898047 (0.0007) [2023-12-26 21:54:01,464][105692] Updated weights for policy 0, policy_version 898057 (0.0009) [2023-12-26 21:54:01,652][105620] Updated weights for policy 1, policy_version 897908 (0.0008) [2023-12-26 21:54:01,711][105620] Updated weights for policy 1, policy_version 897918 (0.0007) [2023-12-26 21:54:01,775][105620] Updated weights for policy 1, policy_version 897928 (0.0009) [2023-12-26 21:54:02,259][105692] Updated weights for policy 0, policy_version 898067 (0.0009) [2023-12-26 21:54:02,308][105692] Updated weights for policy 0, policy_version 898077 (0.0009) [2023-12-26 21:54:02,368][105692] Updated weights for policy 0, policy_version 898088 (0.0009) [2023-12-26 21:54:02,438][105620] Updated weights for policy 1, policy_version 897938 (0.0009) [2023-12-26 21:54:02,495][105620] Updated weights for policy 1, policy_version 897948 (0.0009) [2023-12-26 21:54:02,559][105620] Updated weights for policy 1, policy_version 897958 (0.0009) [2023-12-26 21:54:02,618][105620] Updated weights for policy 1, policy_version 897968 (0.0006) [2023-12-26 21:54:03,146][105692] Updated weights for policy 0, policy_version 898098 (0.0009) [2023-12-26 21:54:03,196][105692] Updated weights for policy 0, policy_version 898108 (0.0005) [2023-12-26 21:54:03,255][105692] Updated weights for policy 0, policy_version 898118 (0.0005) [2023-12-26 21:54:03,314][105692] Updated weights for policy 0, policy_version 898128 (0.0007) [2023-12-26 21:54:03,351][105620] Updated weights for policy 1, policy_version 897978 (0.0010) [2023-12-26 21:54:03,403][105620] Updated weights for policy 1, policy_version 897988 (0.0009) [2023-12-26 21:54:03,455][105620] Updated weights for policy 1, policy_version 897998 (0.0009) [2023-12-26 21:54:04,074][105620] Updated weights for policy 1, policy_version 898008 (0.0010) [2023-12-26 21:54:04,086][105692] Updated weights for policy 0, policy_version 898138 (0.0009) [2023-12-26 21:54:04,127][105620] Updated weights for policy 1, policy_version 898018 (0.0010) [2023-12-26 21:54:04,146][105692] Updated weights for policy 0, policy_version 898148 (0.0010) [2023-12-26 21:54:04,188][105620] Updated weights for policy 1, policy_version 898028 (0.0010) [2023-12-26 21:54:04,207][105692] Updated weights for policy 0, policy_version 898158 (0.0010) [2023-12-26 21:54:04,846][105620] Updated weights for policy 1, policy_version 898038 (0.0007) [2023-12-26 21:54:04,899][105620] Updated weights for policy 1, policy_version 898048 (0.0005) [2023-12-26 21:54:04,955][105620] Updated weights for policy 1, policy_version 898058 (0.0006) [2023-12-26 21:54:05,028][105692] Updated weights for policy 0, policy_version 898168 (0.0010) [2023-12-26 21:54:05,081][105692] Updated weights for policy 0, policy_version 898178 (0.0009) [2023-12-26 21:54:05,135][105692] Updated weights for policy 0, policy_version 898188 (0.0010) [2023-12-26 21:54:05,519][105620] Updated weights for policy 1, policy_version 898068 (0.0005) [2023-12-26 21:54:05,573][105620] Updated weights for policy 1, policy_version 898078 (0.0005) [2023-12-26 21:54:05,625][105620] Updated weights for policy 1, policy_version 898088 (0.0007) [2023-12-26 21:54:05,984][105692] Updated weights for policy 0, policy_version 898198 (0.0008) [2023-12-26 21:54:06,043][105692] Updated weights for policy 0, policy_version 898208 (0.0010) [2023-12-26 21:54:06,062][104569] Fps is (10 sec: 18022.6, 60 sec: 18978.1, 300 sec: 19327.6). Total num frames: 459915264. Throughput: 0: 9445.1, 1: 9714.7. Samples: 459909584. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-26 21:54:06,062][104569] Avg episode reward: [(0, '9169.785'), (1, '8629.812')] [2023-12-26 21:54:06,096][105692] Updated weights for policy 0, policy_version 898218 (0.0010) [2023-12-26 21:54:06,174][105620] Updated weights for policy 1, policy_version 898098 (0.0008) [2023-12-26 21:54:06,245][105620] Updated weights for policy 1, policy_version 898108 (0.0006) [2023-12-26 21:54:06,315][105620] Updated weights for policy 1, policy_version 898118 (0.0006) [2023-12-26 21:54:06,384][105620] Updated weights for policy 1, policy_version 898128 (0.0006) [2023-12-26 21:54:06,807][105692] Updated weights for policy 0, policy_version 898228 (0.0006) [2023-12-26 21:54:06,868][105692] Updated weights for policy 0, policy_version 898238 (0.0009) [2023-12-26 21:54:06,924][105692] Updated weights for policy 0, policy_version 898248 (0.0009) [2023-12-26 21:54:07,064][105620] Updated weights for policy 1, policy_version 898138 (0.0009) [2023-12-26 21:54:07,121][105620] Updated weights for policy 1, policy_version 898148 (0.0009) [2023-12-26 21:54:07,177][105620] Updated weights for policy 1, policy_version 898158 (0.0009) [2023-12-26 21:54:07,608][105692] Updated weights for policy 0, policy_version 898258 (0.0009) [2023-12-26 21:54:07,662][105692] Updated weights for policy 0, policy_version 898268 (0.0009) [2023-12-26 21:54:07,713][105692] Updated weights for policy 0, policy_version 898278 (0.0005) [2023-12-26 21:54:07,779][105692] Updated weights for policy 0, policy_version 898288 (0.0006) [2023-12-26 21:54:07,983][105620] Updated weights for policy 1, policy_version 898168 (0.0009) [2023-12-26 21:54:08,039][105620] Updated weights for policy 1, policy_version 898178 (0.0008) [2023-12-26 21:54:08,096][105620] Updated weights for policy 1, policy_version 898188 (0.0009) [2023-12-26 21:54:08,518][105692] Updated weights for policy 0, policy_version 898298 (0.0009) [2023-12-26 21:54:08,586][105692] Updated weights for policy 0, policy_version 898308 (0.0009) [2023-12-26 21:54:08,646][105692] Updated weights for policy 0, policy_version 898318 (0.0009) [2023-12-26 21:54:08,834][105620] Updated weights for policy 1, policy_version 898198 (0.0010) [2023-12-26 21:54:08,891][105620] Updated weights for policy 1, policy_version 898208 (0.0008) [2023-12-26 21:54:08,951][105620] Updated weights for policy 1, policy_version 898218 (0.0008) [2023-12-26 21:54:09,430][105692] Updated weights for policy 0, policy_version 898328 (0.0008) [2023-12-26 21:54:09,481][105692] Updated weights for policy 0, policy_version 898338 (0.0008) [2023-12-26 21:54:09,538][105692] Updated weights for policy 0, policy_version 898348 (0.0008) [2023-12-26 21:54:09,696][105620] Updated weights for policy 1, policy_version 898228 (0.0007) [2023-12-26 21:54:09,744][105620] Updated weights for policy 1, policy_version 898238 (0.0009) [2023-12-26 21:54:09,791][105620] Updated weights for policy 1, policy_version 898248 (0.0009) [2023-12-26 21:54:10,246][105692] Updated weights for policy 0, policy_version 898358 (0.0007) [2023-12-26 21:54:10,309][105692] Updated weights for policy 0, policy_version 898368 (0.0009) [2023-12-26 21:54:10,370][105692] Updated weights for policy 0, policy_version 898378 (0.0009) [2023-12-26 21:54:10,600][105620] Updated weights for policy 1, policy_version 898258 (0.0009) [2023-12-26 21:54:10,648][105620] Updated weights for policy 1, policy_version 898268 (0.0008) [2023-12-26 21:54:10,707][105620] Updated weights for policy 1, policy_version 898278 (0.0005) [2023-12-26 21:54:10,760][105620] Updated weights for policy 1, policy_version 898288 (0.0007) [2023-12-26 21:54:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19114.7, 300 sec: 19327.6). Total num frames: 460013568. Throughput: 0: 9402.0, 1: 9766.0. Samples: 460024220. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-26 21:54:11,062][104569] Avg episode reward: [(0, '9351.963'), (1, '8451.907')] [2023-12-26 21:54:11,136][105692] Updated weights for policy 0, policy_version 898388 (0.0009) [2023-12-26 21:54:11,203][105692] Updated weights for policy 0, policy_version 898398 (0.0010) [2023-12-26 21:54:11,263][105692] Updated weights for policy 0, policy_version 898408 (0.0011) [2023-12-26 21:54:11,500][105620] Updated weights for policy 1, policy_version 898298 (0.0008) [2023-12-26 21:54:11,564][105620] Updated weights for policy 1, policy_version 898308 (0.0008) [2023-12-26 21:54:11,566][105586] KL-divergence is very high: 146.8681 [2023-12-26 21:54:11,620][105586] KL-divergence is very high: 160.1855 [2023-12-26 21:54:11,633][105620] Updated weights for policy 1, policy_version 898318 (0.0008) [2023-12-26 21:54:12,050][105692] Updated weights for policy 0, policy_version 898418 (0.0011) [2023-12-26 21:54:12,106][105692] Updated weights for policy 0, policy_version 898428 (0.0010) [2023-12-26 21:54:12,170][105692] Updated weights for policy 0, policy_version 898438 (0.0011) [2023-12-26 21:54:12,234][105692] Updated weights for policy 0, policy_version 898448 (0.0011) [2023-12-26 21:54:12,453][105620] Updated weights for policy 1, policy_version 898328 (0.0008) [2023-12-26 21:54:12,521][105620] Updated weights for policy 1, policy_version 898338 (0.0008) [2023-12-26 21:54:12,588][105620] Updated weights for policy 1, policy_version 898348 (0.0008) [2023-12-26 21:54:13,028][105692] Updated weights for policy 0, policy_version 898458 (0.0010) [2023-12-26 21:54:13,091][105692] Updated weights for policy 0, policy_version 898468 (0.0010) [2023-12-26 21:54:13,101][105585] KL-divergence is very high: 106.5045 [2023-12-26 21:54:13,139][105692] Updated weights for policy 0, policy_version 898478 (0.0010) [2023-12-26 21:54:13,139][105585] KL-divergence is very high: 103.6088 [2023-12-26 21:54:13,331][105620] Updated weights for policy 1, policy_version 898358 (0.0008) [2023-12-26 21:54:13,390][105620] Updated weights for policy 1, policy_version 898368 (0.0008) [2023-12-26 21:54:13,447][105620] Updated weights for policy 1, policy_version 898378 (0.0008) [2023-12-26 21:54:13,846][105692] Updated weights for policy 0, policy_version 898488 (0.0007) [2023-12-26 21:54:13,904][105692] Updated weights for policy 0, policy_version 898498 (0.0007) [2023-12-26 21:54:13,969][105692] Updated weights for policy 0, policy_version 898508 (0.0007) [2023-12-26 21:54:14,305][105620] Updated weights for policy 1, policy_version 898388 (0.0009) [2023-12-26 21:54:14,368][105620] Updated weights for policy 1, policy_version 898398 (0.0010) [2023-12-26 21:54:14,421][105620] Updated weights for policy 1, policy_version 898408 (0.0008) [2023-12-26 21:54:14,511][105692] Updated weights for policy 0, policy_version 898518 (0.0010) [2023-12-26 21:54:14,570][105692] Updated weights for policy 0, policy_version 898528 (0.0010) [2023-12-26 21:54:14,633][105692] Updated weights for policy 0, policy_version 898538 (0.0011) [2023-12-26 21:54:15,243][105620] Updated weights for policy 1, policy_version 898418 (0.0008) [2023-12-26 21:54:15,291][105620] Updated weights for policy 1, policy_version 898428 (0.0010) [2023-12-26 21:54:15,350][105620] Updated weights for policy 1, policy_version 898438 (0.0009) [2023-12-26 21:54:15,374][105692] Updated weights for policy 0, policy_version 898548 (0.0009) [2023-12-26 21:54:15,404][105620] Updated weights for policy 1, policy_version 898448 (0.0006) [2023-12-26 21:54:15,429][105692] Updated weights for policy 0, policy_version 898558 (0.0007) [2023-12-26 21:54:15,476][105692] Updated weights for policy 0, policy_version 898568 (0.0009) [2023-12-26 21:54:16,062][104569] Fps is (10 sec: 18841.2, 60 sec: 18978.1, 300 sec: 19299.8). Total num frames: 460103680. Throughput: 0: 9316.1, 1: 9699.3. Samples: 460077724. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-26 21:54:16,063][104569] Avg episode reward: [(0, '6348.555'), (1, '8541.075')] [2023-12-26 21:54:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000898576_230072320.pth... [2023-12-26 21:54:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000898448_230031360.pth... [2023-12-26 21:54:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000897488_229793792.pth [2023-12-26 21:54:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000897336_229744640.pth [2023-12-26 21:54:16,173][105692] Updated weights for policy 0, policy_version 898578 (0.0008) [2023-12-26 21:54:16,220][105620] Updated weights for policy 1, policy_version 898458 (0.0009) [2023-12-26 21:54:16,228][105692] Updated weights for policy 0, policy_version 898588 (0.0006) [2023-12-26 21:54:16,280][105620] Updated weights for policy 1, policy_version 898468 (0.0009) [2023-12-26 21:54:16,283][105692] Updated weights for policy 0, policy_version 898598 (0.0006) [2023-12-26 21:54:16,340][105620] Updated weights for policy 1, policy_version 898478 (0.0007) [2023-12-26 21:54:16,344][105692] Updated weights for policy 0, policy_version 898608 (0.0006) [2023-12-26 21:54:16,958][105692] Updated weights for policy 0, policy_version 898618 (0.0011) [2023-12-26 21:54:17,010][105692] Updated weights for policy 0, policy_version 898628 (0.0011) [2023-12-26 21:54:17,068][105692] Updated weights for policy 0, policy_version 898638 (0.0010) [2023-12-26 21:54:17,138][105620] Updated weights for policy 1, policy_version 898488 (0.0010) [2023-12-26 21:54:17,200][105620] Updated weights for policy 1, policy_version 898498 (0.0010) [2023-12-26 21:54:17,261][105620] Updated weights for policy 1, policy_version 898508 (0.0010) [2023-12-26 21:54:17,653][105692] Updated weights for policy 0, policy_version 898648 (0.0006) [2023-12-26 21:54:17,701][105692] Updated weights for policy 0, policy_version 898658 (0.0005) [2023-12-26 21:54:17,747][105692] Updated weights for policy 0, policy_version 898668 (0.0005) [2023-12-26 21:54:17,979][105620] Updated weights for policy 1, policy_version 898518 (0.0010) [2023-12-26 21:54:18,031][105620] Updated weights for policy 1, policy_version 898528 (0.0010) [2023-12-26 21:54:18,087][105620] Updated weights for policy 1, policy_version 898538 (0.0010) [2023-12-26 21:54:18,390][105692] Updated weights for policy 0, policy_version 898678 (0.0006) [2023-12-26 21:54:18,450][105692] Updated weights for policy 0, policy_version 898688 (0.0006) [2023-12-26 21:54:18,519][105692] Updated weights for policy 0, policy_version 898698 (0.0006) [2023-12-26 21:54:18,875][105620] Updated weights for policy 1, policy_version 898548 (0.0010) [2023-12-26 21:54:18,937][105620] Updated weights for policy 1, policy_version 898558 (0.0010) [2023-12-26 21:54:18,990][105620] Updated weights for policy 1, policy_version 898568 (0.0010) [2023-12-26 21:54:19,109][105692] Updated weights for policy 0, policy_version 898708 (0.0007) [2023-12-26 21:54:19,168][105692] Updated weights for policy 0, policy_version 898718 (0.0008) [2023-12-26 21:54:19,228][105692] Updated weights for policy 0, policy_version 898728 (0.0008) [2023-12-26 21:54:19,777][105620] Updated weights for policy 1, policy_version 898578 (0.0010) [2023-12-26 21:54:19,849][105620] Updated weights for policy 1, policy_version 898588 (0.0012) [2023-12-26 21:54:19,918][105620] Updated weights for policy 1, policy_version 898598 (0.0009) [2023-12-26 21:54:19,956][105692] Updated weights for policy 0, policy_version 898738 (0.0009) [2023-12-26 21:54:19,981][105620] Updated weights for policy 1, policy_version 898608 (0.0008) [2023-12-26 21:54:20,019][105692] Updated weights for policy 0, policy_version 898748 (0.0008) [2023-12-26 21:54:20,079][105692] Updated weights for policy 0, policy_version 898758 (0.0007) [2023-12-26 21:54:20,146][105692] Updated weights for policy 0, policy_version 898768 (0.0006) [2023-12-26 21:54:20,681][105620] Updated weights for policy 1, policy_version 898618 (0.0009) [2023-12-26 21:54:20,748][105620] Updated weights for policy 1, policy_version 898628 (0.0009) [2023-12-26 21:54:20,807][105620] Updated weights for policy 1, policy_version 898638 (0.0009) [2023-12-26 21:54:20,891][105692] Updated weights for policy 0, policy_version 898778 (0.0009) [2023-12-26 21:54:20,947][105692] Updated weights for policy 0, policy_version 898788 (0.0010) [2023-12-26 21:54:21,013][105692] Updated weights for policy 0, policy_version 898798 (0.0009) [2023-12-26 21:54:21,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19114.5, 300 sec: 19355.3). Total num frames: 460210176. Throughput: 0: 9384.7, 1: 9638.0. Samples: 460195788. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-26 21:54:21,063][104569] Avg episode reward: [(0, '6181.823'), (1, '8267.709')] [2023-12-26 21:54:21,607][105620] Updated weights for policy 1, policy_version 898648 (0.0009) [2023-12-26 21:54:21,676][105620] Updated weights for policy 1, policy_version 898658 (0.0009) [2023-12-26 21:54:21,739][105620] Updated weights for policy 1, policy_version 898668 (0.0008) [2023-12-26 21:54:21,756][105692] Updated weights for policy 0, policy_version 898808 (0.0009) [2023-12-26 21:54:21,810][105692] Updated weights for policy 0, policy_version 898818 (0.0009) [2023-12-26 21:54:21,859][105692] Updated weights for policy 0, policy_version 898828 (0.0009) [2023-12-26 21:54:22,419][105620] Updated weights for policy 1, policy_version 898678 (0.0007) [2023-12-26 21:54:22,477][105620] Updated weights for policy 1, policy_version 898688 (0.0009) [2023-12-26 21:54:22,539][105620] Updated weights for policy 1, policy_version 898698 (0.0009) [2023-12-26 21:54:22,679][105692] Updated weights for policy 0, policy_version 898838 (0.0009) [2023-12-26 21:54:22,738][105692] Updated weights for policy 0, policy_version 898848 (0.0010) [2023-12-26 21:54:22,803][105692] Updated weights for policy 0, policy_version 898858 (0.0010) [2023-12-26 21:54:23,261][105620] Updated weights for policy 1, policy_version 898708 (0.0009) [2023-12-26 21:54:23,323][105620] Updated weights for policy 1, policy_version 898718 (0.0009) [2023-12-26 21:54:23,384][105620] Updated weights for policy 1, policy_version 898728 (0.0008) [2023-12-26 21:54:23,585][105692] Updated weights for policy 0, policy_version 898868 (0.0009) [2023-12-26 21:54:23,647][105692] Updated weights for policy 0, policy_version 898878 (0.0009) [2023-12-26 21:54:23,710][105692] Updated weights for policy 0, policy_version 898888 (0.0008) [2023-12-26 21:54:24,131][105620] Updated weights for policy 1, policy_version 898738 (0.0008) [2023-12-26 21:54:24,190][105620] Updated weights for policy 1, policy_version 898748 (0.0009) [2023-12-26 21:54:24,257][105620] Updated weights for policy 1, policy_version 898758 (0.0007) [2023-12-26 21:54:24,323][105620] Updated weights for policy 1, policy_version 898768 (0.0006) [2023-12-26 21:54:24,507][105692] Updated weights for policy 0, policy_version 898898 (0.0008) [2023-12-26 21:54:24,567][105692] Updated weights for policy 0, policy_version 898908 (0.0009) [2023-12-26 21:54:24,625][105692] Updated weights for policy 0, policy_version 898918 (0.0009) [2023-12-26 21:54:24,684][105692] Updated weights for policy 0, policy_version 898928 (0.0009) [2023-12-26 21:54:24,942][105620] Updated weights for policy 1, policy_version 898778 (0.0005) [2023-12-26 21:54:24,993][105620] Updated weights for policy 1, policy_version 898788 (0.0005) [2023-12-26 21:54:25,054][105620] Updated weights for policy 1, policy_version 898798 (0.0005) [2023-12-26 21:54:25,566][105692] Updated weights for policy 0, policy_version 898938 (0.0009) [2023-12-26 21:54:25,596][105620] Updated weights for policy 1, policy_version 898808 (0.0006) [2023-12-26 21:54:25,615][105692] Updated weights for policy 0, policy_version 898948 (0.0008) [2023-12-26 21:54:25,642][105620] Updated weights for policy 1, policy_version 898818 (0.0006) [2023-12-26 21:54:25,665][105692] Updated weights for policy 0, policy_version 898958 (0.0006) [2023-12-26 21:54:25,698][105620] Updated weights for policy 1, policy_version 898828 (0.0009) [2023-12-26 21:54:26,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19114.7, 300 sec: 19299.8). Total num frames: 460300288. Throughput: 0: 9278.0, 1: 9691.2. Samples: 460308192. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-26 21:54:26,062][104569] Avg episode reward: [(0, '7341.820'), (1, '8182.544')] [2023-12-26 21:54:26,365][105620] Updated weights for policy 1, policy_version 898838 (0.0007) [2023-12-26 21:54:26,411][105620] Updated weights for policy 1, policy_version 898848 (0.0005) [2023-12-26 21:54:26,466][105620] Updated weights for policy 1, policy_version 898858 (0.0008) [2023-12-26 21:54:26,504][105692] Updated weights for policy 0, policy_version 898968 (0.0009) [2023-12-26 21:54:26,558][105692] Updated weights for policy 0, policy_version 898979 (0.0009) [2023-12-26 21:54:26,614][105692] Updated weights for policy 0, policy_version 898989 (0.0009) [2023-12-26 21:54:27,250][105692] Updated weights for policy 0, policy_version 898999 (0.0009) [2023-12-26 21:54:27,264][105620] Updated weights for policy 1, policy_version 898868 (0.0008) [2023-12-26 21:54:27,309][105692] Updated weights for policy 0, policy_version 899009 (0.0009) [2023-12-26 21:54:27,312][105620] Updated weights for policy 1, policy_version 898878 (0.0005) [2023-12-26 21:54:27,364][105692] Updated weights for policy 0, policy_version 899019 (0.0008) [2023-12-26 21:54:27,368][105620] Updated weights for policy 1, policy_version 898888 (0.0005) [2023-12-26 21:54:27,913][105620] Updated weights for policy 1, policy_version 898898 (0.0006) [2023-12-26 21:54:27,972][105620] Updated weights for policy 1, policy_version 898908 (0.0008) [2023-12-26 21:54:28,032][105620] Updated weights for policy 1, policy_version 898918 (0.0008) [2023-12-26 21:54:28,091][105620] Updated weights for policy 1, policy_version 898928 (0.0008) [2023-12-26 21:54:28,189][105692] Updated weights for policy 0, policy_version 899029 (0.0009) [2023-12-26 21:54:28,244][105692] Updated weights for policy 0, policy_version 899039 (0.0009) [2023-12-26 21:54:28,307][105692] Updated weights for policy 0, policy_version 899049 (0.0009) [2023-12-26 21:54:28,797][105620] Updated weights for policy 1, policy_version 898938 (0.0005) [2023-12-26 21:54:28,857][105620] Updated weights for policy 1, policy_version 898948 (0.0009) [2023-12-26 21:54:28,912][105620] Updated weights for policy 1, policy_version 898958 (0.0007) [2023-12-26 21:54:29,087][105692] Updated weights for policy 0, policy_version 899059 (0.0008) [2023-12-26 21:54:29,140][105692] Updated weights for policy 0, policy_version 899069 (0.0005) [2023-12-26 21:54:29,187][105692] Updated weights for policy 0, policy_version 899079 (0.0005) [2023-12-26 21:54:29,630][105620] Updated weights for policy 1, policy_version 898968 (0.0008) [2023-12-26 21:54:29,698][105620] Updated weights for policy 1, policy_version 898978 (0.0008) [2023-12-26 21:54:29,745][105620] Updated weights for policy 1, policy_version 898988 (0.0008) [2023-12-26 21:54:29,905][105692] Updated weights for policy 0, policy_version 899089 (0.0007) [2023-12-26 21:54:29,962][105692] Updated weights for policy 0, policy_version 899099 (0.0011) [2023-12-26 21:54:30,024][105692] Updated weights for policy 0, policy_version 899109 (0.0011) [2023-12-26 21:54:30,082][105692] Updated weights for policy 0, policy_version 899119 (0.0010) [2023-12-26 21:54:30,511][105620] Updated weights for policy 1, policy_version 898998 (0.0008) [2023-12-26 21:54:30,559][105620] Updated weights for policy 1, policy_version 899008 (0.0007) [2023-12-26 21:54:30,605][105620] Updated weights for policy 1, policy_version 899018 (0.0008) [2023-12-26 21:54:30,833][105692] Updated weights for policy 0, policy_version 899129 (0.0010) [2023-12-26 21:54:30,887][105692] Updated weights for policy 0, policy_version 899139 (0.0010) [2023-12-26 21:54:30,941][105692] Updated weights for policy 0, policy_version 899149 (0.0010) [2023-12-26 21:54:31,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19114.6, 300 sec: 19299.8). Total num frames: 460398592. Throughput: 0: 9296.2, 1: 9746.9. Samples: 460366864. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-26 21:54:31,063][104569] Avg episode reward: [(0, '9171.359'), (1, '8091.209')] [2023-12-26 21:54:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000899024_230178816.pth... [2023-12-26 21:54:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000899152_230219776.pth... [2023-12-26 21:54:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000897904_229892096.pth [2023-12-26 21:54:31,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000898032_229933056.pth [2023-12-26 21:54:31,386][105620] Updated weights for policy 1, policy_version 899028 (0.0007) [2023-12-26 21:54:31,446][105620] Updated weights for policy 1, policy_version 899038 (0.0008) [2023-12-26 21:54:31,501][105620] Updated weights for policy 1, policy_version 899048 (0.0008) [2023-12-26 21:54:31,736][105692] Updated weights for policy 0, policy_version 899159 (0.0010) [2023-12-26 21:54:31,789][105692] Updated weights for policy 0, policy_version 899169 (0.0007) [2023-12-26 21:54:31,839][105692] Updated weights for policy 0, policy_version 899179 (0.0006) [2023-12-26 21:54:32,343][105620] Updated weights for policy 1, policy_version 899058 (0.0008) [2023-12-26 21:54:32,398][105620] Updated weights for policy 1, policy_version 899068 (0.0008) [2023-12-26 21:54:32,457][105620] Updated weights for policy 1, policy_version 899078 (0.0009) [2023-12-26 21:54:32,493][105692] Updated weights for policy 0, policy_version 899189 (0.0008) [2023-12-26 21:54:32,517][105620] Updated weights for policy 1, policy_version 899088 (0.0007) [2023-12-26 21:54:32,537][105692] Updated weights for policy 0, policy_version 899199 (0.0010) [2023-12-26 21:54:32,591][105692] Updated weights for policy 0, policy_version 899209 (0.0009) [2023-12-26 21:54:33,260][105620] Updated weights for policy 1, policy_version 899098 (0.0007) [2023-12-26 21:54:33,308][105620] Updated weights for policy 1, policy_version 899108 (0.0008) [2023-12-26 21:54:33,338][105692] Updated weights for policy 0, policy_version 899219 (0.0010) [2023-12-26 21:54:33,360][105620] Updated weights for policy 1, policy_version 899118 (0.0006) [2023-12-26 21:54:33,392][105692] Updated weights for policy 0, policy_version 899229 (0.0010) [2023-12-26 21:54:33,439][105692] Updated weights for policy 0, policy_version 899239 (0.0010) [2023-12-26 21:54:34,077][105620] Updated weights for policy 1, policy_version 899128 (0.0008) [2023-12-26 21:54:34,119][105692] Updated weights for policy 0, policy_version 899249 (0.0010) [2023-12-26 21:54:34,130][105620] Updated weights for policy 1, policy_version 899138 (0.0006) [2023-12-26 21:54:34,186][105692] Updated weights for policy 0, policy_version 899259 (0.0010) [2023-12-26 21:54:34,197][105620] Updated weights for policy 1, policy_version 899148 (0.0007) [2023-12-26 21:54:34,250][105692] Updated weights for policy 0, policy_version 899269 (0.0007) [2023-12-26 21:54:34,306][105692] Updated weights for policy 0, policy_version 899279 (0.0009) [2023-12-26 21:54:34,985][105620] Updated weights for policy 1, policy_version 899158 (0.0007) [2023-12-26 21:54:34,986][105692] Updated weights for policy 0, policy_version 899289 (0.0010) [2023-12-26 21:54:35,047][105620] Updated weights for policy 1, policy_version 899168 (0.0006) [2023-12-26 21:54:35,048][105692] Updated weights for policy 0, policy_version 899299 (0.0008) [2023-12-26 21:54:35,102][105692] Updated weights for policy 0, policy_version 899309 (0.0007) [2023-12-26 21:54:35,104][105620] Updated weights for policy 1, policy_version 899178 (0.0008) [2023-12-26 21:54:35,802][105620] Updated weights for policy 1, policy_version 899188 (0.0007) [2023-12-26 21:54:35,851][105692] Updated weights for policy 0, policy_version 899319 (0.0008) [2023-12-26 21:54:35,853][105620] Updated weights for policy 1, policy_version 899198 (0.0005) [2023-12-26 21:54:35,905][105692] Updated weights for policy 0, policy_version 899329 (0.0008) [2023-12-26 21:54:35,907][105620] Updated weights for policy 1, policy_version 899208 (0.0009) [2023-12-26 21:54:35,957][105692] Updated weights for policy 0, policy_version 899339 (0.0006) [2023-12-26 21:54:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19114.7, 300 sec: 19327.6). Total num frames: 460496896. Throughput: 0: 9400.2, 1: 9626.3. Samples: 460480512. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-26 21:54:36,062][104569] Avg episode reward: [(0, '9260.911'), (1, '8087.713')] [2023-12-26 21:54:36,556][105620] Updated weights for policy 1, policy_version 899218 (0.0009) [2023-12-26 21:54:36,641][105620] Updated weights for policy 1, policy_version 899228 (0.0009) [2023-12-26 21:54:36,695][105620] Updated weights for policy 1, policy_version 899238 (0.0011) [2023-12-26 21:54:36,756][105620] Updated weights for policy 1, policy_version 899248 (0.0011) [2023-12-26 21:54:36,764][105692] Updated weights for policy 0, policy_version 899349 (0.0008) [2023-12-26 21:54:36,817][105692] Updated weights for policy 0, policy_version 899359 (0.0006) [2023-12-26 21:54:36,863][105692] Updated weights for policy 0, policy_version 899369 (0.0005) [2023-12-26 21:54:37,450][105620] Updated weights for policy 1, policy_version 899258 (0.0011) [2023-12-26 21:54:37,499][105620] Updated weights for policy 1, policy_version 899268 (0.0010) [2023-12-26 21:54:37,552][105620] Updated weights for policy 1, policy_version 899278 (0.0010) [2023-12-26 21:54:37,576][105692] Updated weights for policy 0, policy_version 899379 (0.0007) [2023-12-26 21:54:37,636][105692] Updated weights for policy 0, policy_version 899389 (0.0008) [2023-12-26 21:54:37,688][105692] Updated weights for policy 0, policy_version 899399 (0.0008) [2023-12-26 21:54:38,320][105620] Updated weights for policy 1, policy_version 899288 (0.0011) [2023-12-26 21:54:38,385][105620] Updated weights for policy 1, policy_version 899298 (0.0009) [2023-12-26 21:54:38,448][105620] Updated weights for policy 1, policy_version 899308 (0.0009) [2023-12-26 21:54:38,449][105692] Updated weights for policy 0, policy_version 899409 (0.0008) [2023-12-26 21:54:38,499][105692] Updated weights for policy 0, policy_version 899419 (0.0007) [2023-12-26 21:54:38,551][105692] Updated weights for policy 0, policy_version 899429 (0.0010) [2023-12-26 21:54:38,603][105692] Updated weights for policy 0, policy_version 899439 (0.0010) [2023-12-26 21:54:39,080][105620] Updated weights for policy 1, policy_version 899318 (0.0009) [2023-12-26 21:54:39,138][105620] Updated weights for policy 1, policy_version 899328 (0.0010) [2023-12-26 21:54:39,196][105620] Updated weights for policy 1, policy_version 899338 (0.0010) [2023-12-26 21:54:39,379][105692] Updated weights for policy 0, policy_version 899449 (0.0009) [2023-12-26 21:54:39,439][105692] Updated weights for policy 0, policy_version 899459 (0.0008) [2023-12-26 21:54:39,501][105692] Updated weights for policy 0, policy_version 899469 (0.0008) [2023-12-26 21:54:39,945][105620] Updated weights for policy 1, policy_version 899348 (0.0009) [2023-12-26 21:54:40,017][105620] Updated weights for policy 1, policy_version 899358 (0.0009) [2023-12-26 21:54:40,084][105620] Updated weights for policy 1, policy_version 899368 (0.0009) [2023-12-26 21:54:40,316][105692] Updated weights for policy 0, policy_version 899479 (0.0010) [2023-12-26 21:54:40,380][105692] Updated weights for policy 0, policy_version 899489 (0.0009) [2023-12-26 21:54:40,449][105692] Updated weights for policy 0, policy_version 899499 (0.0006) [2023-12-26 21:54:40,772][105620] Updated weights for policy 1, policy_version 899378 (0.0008) [2023-12-26 21:54:40,821][105620] Updated weights for policy 1, policy_version 899388 (0.0005) [2023-12-26 21:54:40,880][105620] Updated weights for policy 1, policy_version 899398 (0.0005) [2023-12-26 21:54:40,942][105620] Updated weights for policy 1, policy_version 899408 (0.0005) [2023-12-26 21:54:41,062][104569] Fps is (10 sec: 18841.9, 60 sec: 18978.2, 300 sec: 19299.8). Total num frames: 460587008. Throughput: 0: 9406.9, 1: 9667.1. Samples: 460596288. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-26 21:54:41,063][104569] Avg episode reward: [(0, '9260.309'), (1, '8000.482')] [2023-12-26 21:54:41,071][105692] Updated weights for policy 0, policy_version 899509 (0.0009) [2023-12-26 21:54:41,157][105692] Updated weights for policy 0, policy_version 899519 (0.0011) [2023-12-26 21:54:41,210][105692] Updated weights for policy 0, policy_version 899529 (0.0010) [2023-12-26 21:54:41,627][105620] Updated weights for policy 1, policy_version 899418 (0.0008) [2023-12-26 21:54:41,686][105620] Updated weights for policy 1, policy_version 899428 (0.0009) [2023-12-26 21:54:41,748][105620] Updated weights for policy 1, policy_version 899438 (0.0009) [2023-12-26 21:54:41,905][105692] Updated weights for policy 0, policy_version 899539 (0.0010) [2023-12-26 21:54:41,973][105692] Updated weights for policy 0, policy_version 899549 (0.0007) [2023-12-26 21:54:42,034][105692] Updated weights for policy 0, policy_version 899559 (0.0006) [2023-12-26 21:54:42,552][105620] Updated weights for policy 1, policy_version 899448 (0.0009) [2023-12-26 21:54:42,599][105620] Updated weights for policy 1, policy_version 899458 (0.0008) [2023-12-26 21:54:42,650][105620] Updated weights for policy 1, policy_version 899468 (0.0009) [2023-12-26 21:54:42,791][105692] Updated weights for policy 0, policy_version 899569 (0.0007) [2023-12-26 21:54:42,849][105692] Updated weights for policy 0, policy_version 899579 (0.0009) [2023-12-26 21:54:42,911][105692] Updated weights for policy 0, policy_version 899589 (0.0008) [2023-12-26 21:54:42,969][105692] Updated weights for policy 0, policy_version 899599 (0.0009) [2023-12-26 21:54:43,425][105620] Updated weights for policy 1, policy_version 899478 (0.0008) [2023-12-26 21:54:43,472][105620] Updated weights for policy 1, policy_version 899488 (0.0008) [2023-12-26 21:54:43,529][105620] Updated weights for policy 1, policy_version 899498 (0.0009) [2023-12-26 21:54:43,704][105692] Updated weights for policy 0, policy_version 899609 (0.0010) [2023-12-26 21:54:43,748][105692] Updated weights for policy 0, policy_version 899619 (0.0010) [2023-12-26 21:54:43,800][105692] Updated weights for policy 0, policy_version 899629 (0.0010) [2023-12-26 21:54:44,137][105620] Updated weights for policy 1, policy_version 899508 (0.0007) [2023-12-26 21:54:44,195][105620] Updated weights for policy 1, policy_version 899518 (0.0005) [2023-12-26 21:54:44,255][105620] Updated weights for policy 1, policy_version 899528 (0.0005) [2023-12-26 21:54:44,575][105692] Updated weights for policy 0, policy_version 899639 (0.0011) [2023-12-26 21:54:44,630][105692] Updated weights for policy 0, policy_version 899649 (0.0010) [2023-12-26 21:54:44,697][105692] Updated weights for policy 0, policy_version 899659 (0.0008) [2023-12-26 21:54:44,903][105620] Updated weights for policy 1, policy_version 899538 (0.0006) [2023-12-26 21:54:44,967][105620] Updated weights for policy 1, policy_version 899548 (0.0008) [2023-12-26 21:54:45,028][105620] Updated weights for policy 1, policy_version 899558 (0.0008) [2023-12-26 21:54:45,092][105620] Updated weights for policy 1, policy_version 899568 (0.0008) [2023-12-26 21:54:45,454][105692] Updated weights for policy 0, policy_version 899669 (0.0008) [2023-12-26 21:54:45,520][105692] Updated weights for policy 0, policy_version 899679 (0.0008) [2023-12-26 21:54:45,586][105692] Updated weights for policy 0, policy_version 899689 (0.0008) [2023-12-26 21:54:45,883][105620] Updated weights for policy 1, policy_version 899578 (0.0007) [2023-12-26 21:54:45,947][105620] Updated weights for policy 1, policy_version 899588 (0.0005) [2023-12-26 21:54:46,015][105620] Updated weights for policy 1, policy_version 899598 (0.0006) [2023-12-26 21:54:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19114.7, 300 sec: 19272.0). Total num frames: 460685312. Throughput: 0: 9441.6, 1: 9600.1. Samples: 460653572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:54:46,063][104569] Avg episode reward: [(0, '9077.204'), (1, '8361.487')] [2023-12-26 21:54:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000899696_230359040.pth... [2023-12-26 21:54:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000899600_230326272.pth... [2023-12-26 21:54:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000898576_230072320.pth [2023-12-26 21:54:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000898448_230031360.pth [2023-12-26 21:54:46,256][105692] Updated weights for policy 0, policy_version 899699 (0.0007) [2023-12-26 21:54:46,312][105692] Updated weights for policy 0, policy_version 899709 (0.0007) [2023-12-26 21:54:46,366][105692] Updated weights for policy 0, policy_version 899719 (0.0006) [2023-12-26 21:54:46,755][105620] Updated weights for policy 1, policy_version 899608 (0.0009) [2023-12-26 21:54:46,818][105620] Updated weights for policy 1, policy_version 899618 (0.0008) [2023-12-26 21:54:46,886][105620] Updated weights for policy 1, policy_version 899628 (0.0009) [2023-12-26 21:54:46,942][105692] Updated weights for policy 0, policy_version 899729 (0.0006) [2023-12-26 21:54:47,005][105692] Updated weights for policy 0, policy_version 899739 (0.0008) [2023-12-26 21:54:47,070][105692] Updated weights for policy 0, policy_version 899749 (0.0008) [2023-12-26 21:54:47,134][105692] Updated weights for policy 0, policy_version 899759 (0.0009) [2023-12-26 21:54:47,681][105620] Updated weights for policy 1, policy_version 899638 (0.0010) [2023-12-26 21:54:47,739][105620] Updated weights for policy 1, policy_version 899648 (0.0009) [2023-12-26 21:54:47,749][105692] Updated weights for policy 0, policy_version 899769 (0.0006) [2023-12-26 21:54:47,785][105620] Updated weights for policy 1, policy_version 899658 (0.0009) [2023-12-26 21:54:47,803][105692] Updated weights for policy 0, policy_version 899779 (0.0009) [2023-12-26 21:54:47,858][105692] Updated weights for policy 0, policy_version 899789 (0.0009) [2023-12-26 21:54:48,469][105692] Updated weights for policy 0, policy_version 899799 (0.0006) [2023-12-26 21:54:48,479][105620] Updated weights for policy 1, policy_version 899668 (0.0006) [2023-12-26 21:54:48,528][105692] Updated weights for policy 0, policy_version 899809 (0.0006) [2023-12-26 21:54:48,532][105620] Updated weights for policy 1, policy_version 899678 (0.0007) [2023-12-26 21:54:48,579][105692] Updated weights for policy 0, policy_version 899819 (0.0006) [2023-12-26 21:54:48,585][105620] Updated weights for policy 1, policy_version 899688 (0.0008) [2023-12-26 21:54:49,144][105692] Updated weights for policy 0, policy_version 899829 (0.0006) [2023-12-26 21:54:49,206][105692] Updated weights for policy 0, policy_version 899839 (0.0006) [2023-12-26 21:54:49,275][105692] Updated weights for policy 0, policy_version 899849 (0.0008) [2023-12-26 21:54:49,446][105620] Updated weights for policy 1, policy_version 899698 (0.0009) [2023-12-26 21:54:49,497][105620] Updated weights for policy 1, policy_version 899708 (0.0006) [2023-12-26 21:54:49,522][105586] KL-divergence is very high: 133.3797 [2023-12-26 21:54:49,555][105620] Updated weights for policy 1, policy_version 899718 (0.0005) [2023-12-26 21:54:49,565][105586] KL-divergence is very high: 150.8375 [2023-12-26 21:54:49,607][105620] Updated weights for policy 1, policy_version 899728 (0.0007) [2023-12-26 21:54:50,003][105692] Updated weights for policy 0, policy_version 899859 (0.0009) [2023-12-26 21:54:50,063][105692] Updated weights for policy 0, policy_version 899869 (0.0005) [2023-12-26 21:54:50,127][105692] Updated weights for policy 0, policy_version 899879 (0.0011) [2023-12-26 21:54:50,199][105586] KL-divergence is very high: 126.9856 [2023-12-26 21:54:50,263][105620] Updated weights for policy 1, policy_version 899738 (0.0007) [2023-12-26 21:54:50,324][105620] Updated weights for policy 1, policy_version 899748 (0.0007) [2023-12-26 21:54:50,384][105620] Updated weights for policy 1, policy_version 899758 (0.0010) [2023-12-26 21:54:50,792][105692] Updated weights for policy 0, policy_version 899889 (0.0011) [2023-12-26 21:54:50,862][105692] Updated weights for policy 0, policy_version 899899 (0.0009) [2023-12-26 21:54:50,929][105692] Updated weights for policy 0, policy_version 899909 (0.0011) [2023-12-26 21:54:50,989][105692] Updated weights for policy 0, policy_version 899919 (0.0011) [2023-12-26 21:54:51,041][105620] Updated weights for policy 1, policy_version 899768 (0.0008) [2023-12-26 21:54:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19114.7, 300 sec: 19272.0). Total num frames: 460783616. Throughput: 0: 9635.9, 1: 9549.6. Samples: 460772932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:54:51,062][104569] Avg episode reward: [(0, '8987.269'), (1, '8001.065')] [2023-12-26 21:54:51,104][105620] Updated weights for policy 1, policy_version 899778 (0.0008) [2023-12-26 21:54:51,169][105620] Updated weights for policy 1, policy_version 899788 (0.0006) [2023-12-26 21:54:51,713][105692] Updated weights for policy 0, policy_version 899929 (0.0011) [2023-12-26 21:54:51,773][105692] Updated weights for policy 0, policy_version 899939 (0.0006) [2023-12-26 21:54:51,830][105692] Updated weights for policy 0, policy_version 899949 (0.0008) [2023-12-26 21:54:51,931][105620] Updated weights for policy 1, policy_version 899798 (0.0008) [2023-12-26 21:54:51,997][105620] Updated weights for policy 1, policy_version 899808 (0.0010) [2023-12-26 21:54:52,066][105620] Updated weights for policy 1, policy_version 899818 (0.0011) [2023-12-26 21:54:52,521][105692] Updated weights for policy 0, policy_version 899959 (0.0007) [2023-12-26 21:54:52,573][105692] Updated weights for policy 0, policy_version 899969 (0.0005) [2023-12-26 21:54:52,627][105692] Updated weights for policy 0, policy_version 899979 (0.0008) [2023-12-26 21:54:52,831][105620] Updated weights for policy 1, policy_version 899828 (0.0010) [2023-12-26 21:54:52,890][105620] Updated weights for policy 1, policy_version 899838 (0.0010) [2023-12-26 21:54:52,950][105620] Updated weights for policy 1, policy_version 899848 (0.0010) [2023-12-26 21:54:53,196][105692] Updated weights for policy 0, policy_version 899989 (0.0009) [2023-12-26 21:54:53,257][105692] Updated weights for policy 0, policy_version 899999 (0.0008) [2023-12-26 21:54:53,319][105692] Updated weights for policy 0, policy_version 900009 (0.0010) [2023-12-26 21:54:53,607][105620] Updated weights for policy 1, policy_version 899858 (0.0009) [2023-12-26 21:54:53,658][105620] Updated weights for policy 1, policy_version 899868 (0.0005) [2023-12-26 21:54:53,716][105586] KL-divergence is very high: 126.7732 [2023-12-26 21:54:53,716][105620] Updated weights for policy 1, policy_version 899878 (0.0010) [2023-12-26 21:54:53,762][105586] KL-divergence is very high: 138.1411 [2023-12-26 21:54:53,775][105620] Updated weights for policy 1, policy_version 899888 (0.0010) [2023-12-26 21:54:53,946][105692] Updated weights for policy 0, policy_version 900019 (0.0009) [2023-12-26 21:54:53,998][105692] Updated weights for policy 0, policy_version 900029 (0.0005) [2023-12-26 21:54:54,046][105692] Updated weights for policy 0, policy_version 900039 (0.0005) [2023-12-26 21:54:54,490][105620] Updated weights for policy 1, policy_version 899898 (0.0011) [2023-12-26 21:54:54,545][105620] Updated weights for policy 1, policy_version 899908 (0.0010) [2023-12-26 21:54:54,597][105620] Updated weights for policy 1, policy_version 899918 (0.0010) [2023-12-26 21:54:54,646][105692] Updated weights for policy 0, policy_version 900049 (0.0006) [2023-12-26 21:54:54,707][105692] Updated weights for policy 0, policy_version 900059 (0.0010) [2023-12-26 21:54:54,719][105585] KL-divergence is very high: 316.3620 [2023-12-26 21:54:54,737][105585] KL-divergence is very high: 114.2128 [2023-12-26 21:54:54,765][105692] Updated weights for policy 0, policy_version 900069 (0.0010) [2023-12-26 21:54:54,766][105585] KL-divergence is very high: 570.3778 [2023-12-26 21:54:54,784][105585] KL-divergence is very high: 153.0509 [2023-12-26 21:54:54,812][105585] KL-divergence is very high: 638.5396 [2023-12-26 21:54:54,826][105692] Updated weights for policy 0, policy_version 900079 (0.0010) [2023-12-26 21:54:55,356][105620] Updated weights for policy 1, policy_version 899928 (0.0010) [2023-12-26 21:54:55,414][105620] Updated weights for policy 1, policy_version 899938 (0.0010) [2023-12-26 21:54:55,424][105692] Updated weights for policy 0, policy_version 900089 (0.0010) [2023-12-26 21:54:55,467][105620] Updated weights for policy 1, policy_version 899948 (0.0010) [2023-12-26 21:54:55,472][105692] Updated weights for policy 0, policy_version 900099 (0.0005) [2023-12-26 21:54:55,520][105692] Updated weights for policy 0, policy_version 900109 (0.0005) [2023-12-26 21:54:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19114.7, 300 sec: 19272.0). Total num frames: 460881920. Throughput: 0: 9803.4, 1: 9534.2. Samples: 460894412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:54:56,062][104569] Avg episode reward: [(0, '8911.932'), (1, '7914.340')] [2023-12-26 21:54:56,105][105620] Updated weights for policy 1, policy_version 899958 (0.0008) [2023-12-26 21:54:56,166][105620] Updated weights for policy 1, policy_version 899968 (0.0006) [2023-12-26 21:54:56,224][105620] Updated weights for policy 1, policy_version 899978 (0.0005) [2023-12-26 21:54:56,257][105692] Updated weights for policy 0, policy_version 900119 (0.0009) [2023-12-26 21:54:56,305][105692] Updated weights for policy 0, policy_version 900129 (0.0010) [2023-12-26 21:54:56,351][105692] Updated weights for policy 0, policy_version 900139 (0.0010) [2023-12-26 21:54:56,823][105620] Updated weights for policy 1, policy_version 899988 (0.0007) [2023-12-26 21:54:56,880][105620] Updated weights for policy 1, policy_version 899998 (0.0010) [2023-12-26 21:54:56,931][105620] Updated weights for policy 1, policy_version 900008 (0.0010) [2023-12-26 21:54:57,125][105692] Updated weights for policy 0, policy_version 900149 (0.0011) [2023-12-26 21:54:57,179][105692] Updated weights for policy 0, policy_version 900159 (0.0010) [2023-12-26 21:54:57,236][105692] Updated weights for policy 0, policy_version 900169 (0.0010) [2023-12-26 21:54:57,670][105620] Updated weights for policy 1, policy_version 900018 (0.0010) [2023-12-26 21:54:57,724][105620] Updated weights for policy 1, policy_version 900028 (0.0010) [2023-12-26 21:54:57,778][105620] Updated weights for policy 1, policy_version 900038 (0.0010) [2023-12-26 21:54:57,836][105620] Updated weights for policy 1, policy_version 900048 (0.0010) [2023-12-26 21:54:57,966][105692] Updated weights for policy 0, policy_version 900179 (0.0010) [2023-12-26 21:54:58,030][105692] Updated weights for policy 0, policy_version 900189 (0.0010) [2023-12-26 21:54:58,090][105692] Updated weights for policy 0, policy_version 900199 (0.0010) [2023-12-26 21:54:58,547][105620] Updated weights for policy 1, policy_version 900058 (0.0009) [2023-12-26 21:54:58,615][105620] Updated weights for policy 1, policy_version 900068 (0.0008) [2023-12-26 21:54:58,677][105620] Updated weights for policy 1, policy_version 900078 (0.0008) [2023-12-26 21:54:58,842][105692] Updated weights for policy 0, policy_version 900209 (0.0010) [2023-12-26 21:54:58,913][105692] Updated weights for policy 0, policy_version 900219 (0.0010) [2023-12-26 21:54:58,979][105692] Updated weights for policy 0, policy_version 900229 (0.0011) [2023-12-26 21:54:59,044][105692] Updated weights for policy 0, policy_version 900239 (0.0009) [2023-12-26 21:54:59,496][105620] Updated weights for policy 1, policy_version 900088 (0.0007) [2023-12-26 21:54:59,548][105620] Updated weights for policy 1, policy_version 900098 (0.0008) [2023-12-26 21:54:59,604][105620] Updated weights for policy 1, policy_version 900108 (0.0008) [2023-12-26 21:54:59,841][105692] Updated weights for policy 0, policy_version 900249 (0.0010) [2023-12-26 21:54:59,905][105692] Updated weights for policy 0, policy_version 900259 (0.0008) [2023-12-26 21:54:59,968][105692] Updated weights for policy 0, policy_version 900269 (0.0009) [2023-12-26 21:55:00,295][105620] Updated weights for policy 1, policy_version 900118 (0.0007) [2023-12-26 21:55:00,353][105620] Updated weights for policy 1, policy_version 900128 (0.0008) [2023-12-26 21:55:00,411][105620] Updated weights for policy 1, policy_version 900138 (0.0007) [2023-12-26 21:55:00,700][105692] Updated weights for policy 0, policy_version 900279 (0.0010) [2023-12-26 21:55:00,749][105692] Updated weights for policy 0, policy_version 900289 (0.0008) [2023-12-26 21:55:00,795][105692] Updated weights for policy 0, policy_version 900299 (0.0005) [2023-12-26 21:55:01,056][105620] Updated weights for policy 1, policy_version 900148 (0.0008) [2023-12-26 21:55:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19244.2). Total num frames: 460980224. Throughput: 0: 9839.2, 1: 9603.1. Samples: 460952628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:55:01,063][104569] Avg episode reward: [(0, '9093.287'), (1, '8368.460')] [2023-12-26 21:55:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000900304_230514688.pth... [2023-12-26 21:55:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000899152_230219776.pth [2023-12-26 21:55:01,113][105620] Updated weights for policy 1, policy_version 900158 (0.0008) [2023-12-26 21:55:01,181][105620] Updated weights for policy 1, policy_version 900168 (0.0008) [2023-12-26 21:55:01,222][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000900176_230473728.pth... [2023-12-26 21:55:01,226][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000899024_230178816.pth [2023-12-26 21:55:01,482][105692] Updated weights for policy 0, policy_version 900309 (0.0007) [2023-12-26 21:55:01,532][105692] Updated weights for policy 0, policy_version 900319 (0.0010) [2023-12-26 21:55:01,592][105692] Updated weights for policy 0, policy_version 900331 (0.0010) [2023-12-26 21:55:01,835][105620] Updated weights for policy 1, policy_version 900178 (0.0008) [2023-12-26 21:55:01,883][105620] Updated weights for policy 1, policy_version 900188 (0.0007) [2023-12-26 21:55:01,941][105620] Updated weights for policy 1, policy_version 900198 (0.0009) [2023-12-26 21:55:01,994][105620] Updated weights for policy 1, policy_version 900208 (0.0008) [2023-12-26 21:55:02,407][105692] Updated weights for policy 0, policy_version 900341 (0.0010) [2023-12-26 21:55:02,467][105692] Updated weights for policy 0, policy_version 900351 (0.0009) [2023-12-26 21:55:02,529][105692] Updated weights for policy 0, policy_version 900361 (0.0009) [2023-12-26 21:55:02,749][105620] Updated weights for policy 1, policy_version 900218 (0.0008) [2023-12-26 21:55:02,799][105620] Updated weights for policy 1, policy_version 900228 (0.0009) [2023-12-26 21:55:02,852][105620] Updated weights for policy 1, policy_version 900238 (0.0008) [2023-12-26 21:55:03,280][105692] Updated weights for policy 0, policy_version 900371 (0.0009) [2023-12-26 21:55:03,333][105692] Updated weights for policy 0, policy_version 900381 (0.0009) [2023-12-26 21:55:03,386][105692] Updated weights for policy 0, policy_version 900391 (0.0009) [2023-12-26 21:55:03,538][105620] Updated weights for policy 1, policy_version 900248 (0.0007) [2023-12-26 21:55:03,595][105620] Updated weights for policy 1, policy_version 900258 (0.0010) [2023-12-26 21:55:03,649][105620] Updated weights for policy 1, policy_version 900268 (0.0010) [2023-12-26 21:55:04,229][105692] Updated weights for policy 0, policy_version 900401 (0.0009) [2023-12-26 21:55:04,247][105620] Updated weights for policy 1, policy_version 900278 (0.0007) [2023-12-26 21:55:04,286][105692] Updated weights for policy 0, policy_version 900411 (0.0009) [2023-12-26 21:55:04,301][105620] Updated weights for policy 1, policy_version 900288 (0.0005) [2023-12-26 21:55:04,343][105692] Updated weights for policy 0, policy_version 900421 (0.0009) [2023-12-26 21:55:04,375][105620] Updated weights for policy 1, policy_version 900298 (0.0008) [2023-12-26 21:55:04,394][105692] Updated weights for policy 0, policy_version 900431 (0.0006) [2023-12-26 21:55:04,928][105620] Updated weights for policy 1, policy_version 900308 (0.0009) [2023-12-26 21:55:04,985][105620] Updated weights for policy 1, policy_version 900318 (0.0008) [2023-12-26 21:55:05,044][105620] Updated weights for policy 1, policy_version 900328 (0.0009) [2023-12-26 21:55:05,217][105692] Updated weights for policy 0, policy_version 900441 (0.0006) [2023-12-26 21:55:05,274][105692] Updated weights for policy 0, policy_version 900451 (0.0005) [2023-12-26 21:55:05,343][105692] Updated weights for policy 0, policy_version 900461 (0.0010) [2023-12-26 21:55:05,672][105620] Updated weights for policy 1, policy_version 900338 (0.0006) [2023-12-26 21:55:05,723][105620] Updated weights for policy 1, policy_version 900348 (0.0010) [2023-12-26 21:55:05,775][105620] Updated weights for policy 1, policy_version 900358 (0.0010) [2023-12-26 21:55:05,830][105620] Updated weights for policy 1, policy_version 900368 (0.0011) [2023-12-26 21:55:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19272.0). Total num frames: 461078528. Throughput: 0: 9604.7, 1: 9795.0. Samples: 461068772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:55:06,063][104569] Avg episode reward: [(0, '9079.233'), (1, '8634.290')] [2023-12-26 21:55:06,077][105692] Updated weights for policy 0, policy_version 900471 (0.0008) [2023-12-26 21:55:06,137][105692] Updated weights for policy 0, policy_version 900481 (0.0009) [2023-12-26 21:55:06,196][105692] Updated weights for policy 0, policy_version 900491 (0.0010) [2023-12-26 21:55:06,560][105620] Updated weights for policy 1, policy_version 900378 (0.0009) [2023-12-26 21:55:06,627][105620] Updated weights for policy 1, policy_version 900388 (0.0009) [2023-12-26 21:55:06,689][105620] Updated weights for policy 1, policy_version 900398 (0.0008) [2023-12-26 21:55:06,993][105692] Updated weights for policy 0, policy_version 900501 (0.0009) [2023-12-26 21:55:07,058][105692] Updated weights for policy 0, policy_version 900511 (0.0009) [2023-12-26 21:55:07,120][105692] Updated weights for policy 0, policy_version 900521 (0.0009) [2023-12-26 21:55:07,491][105620] Updated weights for policy 1, policy_version 900408 (0.0009) [2023-12-26 21:55:07,549][105620] Updated weights for policy 1, policy_version 900419 (0.0010) [2023-12-26 21:55:07,600][105620] Updated weights for policy 1, policy_version 900430 (0.0010) [2023-12-26 21:55:07,679][105692] Updated weights for policy 0, policy_version 900531 (0.0007) [2023-12-26 21:55:07,733][105692] Updated weights for policy 0, policy_version 900541 (0.0008) [2023-12-26 21:55:07,784][105692] Updated weights for policy 0, policy_version 900551 (0.0009) [2023-12-26 21:55:08,381][105620] Updated weights for policy 1, policy_version 900440 (0.0010) [2023-12-26 21:55:08,442][105620] Updated weights for policy 1, policy_version 900450 (0.0011) [2023-12-26 21:55:08,505][105620] Updated weights for policy 1, policy_version 900460 (0.0011) [2023-12-26 21:55:08,530][105692] Updated weights for policy 0, policy_version 900561 (0.0008) [2023-12-26 21:55:08,582][105692] Updated weights for policy 0, policy_version 900571 (0.0008) [2023-12-26 21:55:08,641][105692] Updated weights for policy 0, policy_version 900581 (0.0008) [2023-12-26 21:55:08,701][105692] Updated weights for policy 0, policy_version 900591 (0.0008) [2023-12-26 21:55:09,207][105620] Updated weights for policy 1, policy_version 900470 (0.0011) [2023-12-26 21:55:09,275][105620] Updated weights for policy 1, policy_version 900480 (0.0008) [2023-12-26 21:55:09,329][105620] Updated weights for policy 1, policy_version 900490 (0.0010) [2023-12-26 21:55:09,516][105692] Updated weights for policy 0, policy_version 900601 (0.0008) [2023-12-26 21:55:09,583][105692] Updated weights for policy 0, policy_version 900611 (0.0009) [2023-12-26 21:55:09,644][105692] Updated weights for policy 0, policy_version 900621 (0.0005) [2023-12-26 21:55:10,092][105620] Updated weights for policy 1, policy_version 900500 (0.0010) [2023-12-26 21:55:10,155][105620] Updated weights for policy 1, policy_version 900510 (0.0010) [2023-12-26 21:55:10,212][105620] Updated weights for policy 1, policy_version 900520 (0.0009) [2023-12-26 21:55:10,372][105692] Updated weights for policy 0, policy_version 900631 (0.0008) [2023-12-26 21:55:10,437][105692] Updated weights for policy 0, policy_version 900641 (0.0009) [2023-12-26 21:55:10,494][105692] Updated weights for policy 0, policy_version 900651 (0.0008) [2023-12-26 21:55:10,919][105620] Updated weights for policy 1, policy_version 900530 (0.0008) [2023-12-26 21:55:10,978][105620] Updated weights for policy 1, policy_version 900540 (0.0011) [2023-12-26 21:55:11,041][105620] Updated weights for policy 1, policy_version 900550 (0.0011) [2023-12-26 21:55:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19244.3). Total num frames: 461168640. Throughput: 0: 9673.7, 1: 9750.3. Samples: 461182272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:55:11,063][104569] Avg episode reward: [(0, '8809.258'), (1, '8266.777')] [2023-12-26 21:55:11,094][105620] Updated weights for policy 1, policy_version 900560 (0.0010) [2023-12-26 21:55:11,236][105692] Updated weights for policy 0, policy_version 900661 (0.0008) [2023-12-26 21:55:11,298][105692] Updated weights for policy 0, policy_version 900671 (0.0009) [2023-12-26 21:55:11,362][105692] Updated weights for policy 0, policy_version 900681 (0.0007) [2023-12-26 21:55:11,837][105620] Updated weights for policy 1, policy_version 900570 (0.0010) [2023-12-26 21:55:11,890][105620] Updated weights for policy 1, policy_version 900580 (0.0010) [2023-12-26 21:55:11,946][105620] Updated weights for policy 1, policy_version 900590 (0.0010) [2023-12-26 21:55:12,117][105692] Updated weights for policy 0, policy_version 900691 (0.0007) [2023-12-26 21:55:12,179][105692] Updated weights for policy 0, policy_version 900701 (0.0008) [2023-12-26 21:55:12,241][105692] Updated weights for policy 0, policy_version 900711 (0.0008) [2023-12-26 21:55:12,697][105620] Updated weights for policy 1, policy_version 900600 (0.0010) [2023-12-26 21:55:12,762][105620] Updated weights for policy 1, policy_version 900610 (0.0010) [2023-12-26 21:55:12,827][105620] Updated weights for policy 1, policy_version 900620 (0.0010) [2023-12-26 21:55:12,982][105692] Updated weights for policy 0, policy_version 900721 (0.0008) [2023-12-26 21:55:13,044][105692] Updated weights for policy 0, policy_version 900731 (0.0008) [2023-12-26 21:55:13,103][105692] Updated weights for policy 0, policy_version 900741 (0.0008) [2023-12-26 21:55:13,151][105692] Updated weights for policy 0, policy_version 900751 (0.0008) [2023-12-26 21:55:13,509][105620] Updated weights for policy 1, policy_version 900630 (0.0009) [2023-12-26 21:55:13,560][105620] Updated weights for policy 1, policy_version 900640 (0.0010) [2023-12-26 21:55:13,604][105620] Updated weights for policy 1, policy_version 900650 (0.0010) [2023-12-26 21:55:13,921][105692] Updated weights for policy 0, policy_version 900761 (0.0009) [2023-12-26 21:55:13,969][105692] Updated weights for policy 0, policy_version 900771 (0.0008) [2023-12-26 21:55:14,022][105692] Updated weights for policy 0, policy_version 900781 (0.0008) [2023-12-26 21:55:14,366][105620] Updated weights for policy 1, policy_version 900660 (0.0010) [2023-12-26 21:55:14,424][105620] Updated weights for policy 1, policy_version 900670 (0.0010) [2023-12-26 21:55:14,488][105620] Updated weights for policy 1, policy_version 900680 (0.0010) [2023-12-26 21:55:14,799][105692] Updated weights for policy 0, policy_version 900791 (0.0008) [2023-12-26 21:55:14,863][105692] Updated weights for policy 0, policy_version 900801 (0.0008) [2023-12-26 21:55:14,931][105692] Updated weights for policy 0, policy_version 900811 (0.0008) [2023-12-26 21:55:15,252][105620] Updated weights for policy 1, policy_version 900690 (0.0010) [2023-12-26 21:55:15,314][105620] Updated weights for policy 1, policy_version 900700 (0.0011) [2023-12-26 21:55:15,376][105586] KL-divergence is very high: 219.2499 [2023-12-26 21:55:15,376][105620] Updated weights for policy 1, policy_version 900710 (0.0010) [2023-12-26 21:55:15,421][105586] KL-divergence is very high: 428.5684 [2023-12-26 21:55:15,434][105620] Updated weights for policy 1, policy_version 900720 (0.0010) [2023-12-26 21:55:15,666][105692] Updated weights for policy 0, policy_version 900821 (0.0008) [2023-12-26 21:55:15,715][105692] Updated weights for policy 0, policy_version 900831 (0.0008) [2023-12-26 21:55:15,764][105692] Updated weights for policy 0, policy_version 900841 (0.0008) [2023-12-26 21:55:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19244.3). Total num frames: 461266944. Throughput: 0: 9689.7, 1: 9696.0. Samples: 461239220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:55:16,063][104569] Avg episode reward: [(0, '8587.305'), (1, '7919.212')] [2023-12-26 21:55:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000900848_230653952.pth... [2023-12-26 21:55:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000900720_230612992.pth... [2023-12-26 21:55:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000899696_230359040.pth [2023-12-26 21:55:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000899600_230326272.pth [2023-12-26 21:55:16,181][105586] KL-divergence is very high: 103.0814 [2023-12-26 21:55:16,187][105620] Updated weights for policy 1, policy_version 900730 (0.0010) [2023-12-26 21:55:16,219][105586] KL-divergence is very high: 124.5061 [2023-12-26 21:55:16,233][105620] Updated weights for policy 1, policy_version 900740 (0.0010) [2023-12-26 21:55:16,233][105586] KL-divergence is very high: 110.7569 [2023-12-26 21:55:16,267][105586] KL-divergence is very high: 126.1524 [2023-12-26 21:55:16,295][105620] Updated weights for policy 1, policy_version 900750 (0.0010) [2023-12-26 21:55:16,602][105692] Updated weights for policy 0, policy_version 900851 (0.0009) [2023-12-26 21:55:16,647][105692] Updated weights for policy 0, policy_version 900861 (0.0008) [2023-12-26 21:55:16,703][105692] Updated weights for policy 0, policy_version 900871 (0.0008) [2023-12-26 21:55:16,951][105620] Updated weights for policy 1, policy_version 900760 (0.0009) [2023-12-26 21:55:17,015][105620] Updated weights for policy 1, policy_version 900770 (0.0011) [2023-12-26 21:55:17,081][105620] Updated weights for policy 1, policy_version 900780 (0.0011) [2023-12-26 21:55:17,495][105692] Updated weights for policy 0, policy_version 900881 (0.0009) [2023-12-26 21:55:17,547][105692] Updated weights for policy 0, policy_version 900891 (0.0010) [2023-12-26 21:55:17,598][105692] Updated weights for policy 0, policy_version 900901 (0.0010) [2023-12-26 21:55:17,664][105692] Updated weights for policy 0, policy_version 900911 (0.0008) [2023-12-26 21:55:17,859][105620] Updated weights for policy 1, policy_version 900790 (0.0007) [2023-12-26 21:55:17,930][105620] Updated weights for policy 1, policy_version 900800 (0.0005) [2023-12-26 21:55:17,991][105620] Updated weights for policy 1, policy_version 900810 (0.0006) [2023-12-26 21:55:18,341][105692] Updated weights for policy 0, policy_version 900921 (0.0008) [2023-12-26 21:55:18,401][105692] Updated weights for policy 0, policy_version 900931 (0.0009) [2023-12-26 21:55:18,464][105692] Updated weights for policy 0, policy_version 900941 (0.0009) [2023-12-26 21:55:18,539][105620] Updated weights for policy 1, policy_version 900820 (0.0005) [2023-12-26 21:55:18,599][105620] Updated weights for policy 1, policy_version 900830 (0.0005) [2023-12-26 21:55:18,650][105620] Updated weights for policy 1, policy_version 900840 (0.0006) [2023-12-26 21:55:19,179][105692] Updated weights for policy 0, policy_version 900951 (0.0007) [2023-12-26 21:55:19,245][105692] Updated weights for policy 0, policy_version 900961 (0.0006) [2023-12-26 21:55:19,306][105692] Updated weights for policy 0, policy_version 900971 (0.0008) [2023-12-26 21:55:19,345][105620] Updated weights for policy 1, policy_version 900850 (0.0010) [2023-12-26 21:55:19,406][105620] Updated weights for policy 1, policy_version 900860 (0.0009) [2023-12-26 21:55:19,468][105620] Updated weights for policy 1, policy_version 900870 (0.0010) [2023-12-26 21:55:19,533][105620] Updated weights for policy 1, policy_version 900880 (0.0008) [2023-12-26 21:55:20,016][105692] Updated weights for policy 0, policy_version 900981 (0.0007) [2023-12-26 21:55:20,078][105692] Updated weights for policy 0, policy_version 900991 (0.0009) [2023-12-26 21:55:20,141][105692] Updated weights for policy 0, policy_version 901001 (0.0009) [2023-12-26 21:55:20,284][105620] Updated weights for policy 1, policy_version 900890 (0.0009) [2023-12-26 21:55:20,351][105620] Updated weights for policy 1, policy_version 900900 (0.0009) [2023-12-26 21:55:20,415][105620] Updated weights for policy 1, policy_version 900910 (0.0010) [2023-12-26 21:55:20,855][105692] Updated weights for policy 0, policy_version 901011 (0.0008) [2023-12-26 21:55:20,920][105692] Updated weights for policy 0, policy_version 901021 (0.0007) [2023-12-26 21:55:20,975][105585] KL-divergence is very high: 119.6289 [2023-12-26 21:55:20,982][105692] Updated weights for policy 0, policy_version 901031 (0.0009) [2023-12-26 21:55:21,030][105585] KL-divergence is very high: 135.2088 [2023-12-26 21:55:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.3, 300 sec: 19244.3). Total num frames: 461365248. Throughput: 0: 9652.0, 1: 9774.9. Samples: 461354724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:55:21,063][104569] Avg episode reward: [(0, '8352.181'), (1, '8023.956')] [2023-12-26 21:55:21,194][105620] Updated weights for policy 1, policy_version 900920 (0.0007) [2023-12-26 21:55:21,266][105620] Updated weights for policy 1, policy_version 900930 (0.0006) [2023-12-26 21:55:21,336][105620] Updated weights for policy 1, policy_version 900940 (0.0008) [2023-12-26 21:55:21,777][105692] Updated weights for policy 0, policy_version 901041 (0.0009) [2023-12-26 21:55:21,840][105692] Updated weights for policy 0, policy_version 901051 (0.0008) [2023-12-26 21:55:21,896][105692] Updated weights for policy 0, policy_version 901061 (0.0007) [2023-12-26 21:55:21,949][105692] Updated weights for policy 0, policy_version 901071 (0.0005) [2023-12-26 21:55:22,053][105620] Updated weights for policy 1, policy_version 900950 (0.0010) [2023-12-26 21:55:22,101][105620] Updated weights for policy 1, policy_version 900960 (0.0009) [2023-12-26 21:55:22,149][105620] Updated weights for policy 1, policy_version 900970 (0.0009) [2023-12-26 21:55:22,711][105692] Updated weights for policy 0, policy_version 901081 (0.0009) [2023-12-26 21:55:22,766][105692] Updated weights for policy 0, policy_version 901091 (0.0008) [2023-12-26 21:55:22,821][105692] Updated weights for policy 0, policy_version 901101 (0.0009) [2023-12-26 21:55:22,984][105620] Updated weights for policy 1, policy_version 900980 (0.0009) [2023-12-26 21:55:23,031][105620] Updated weights for policy 1, policy_version 900990 (0.0009) [2023-12-26 21:55:23,089][105620] Updated weights for policy 1, policy_version 901000 (0.0009) [2023-12-26 21:55:23,563][105692] Updated weights for policy 0, policy_version 901111 (0.0010) [2023-12-26 21:55:23,620][105692] Updated weights for policy 0, policy_version 901121 (0.0005) [2023-12-26 21:55:23,672][105692] Updated weights for policy 0, policy_version 901131 (0.0005) [2023-12-26 21:55:23,810][105620] Updated weights for policy 1, policy_version 901010 (0.0008) [2023-12-26 21:55:23,859][105620] Updated weights for policy 1, policy_version 901020 (0.0008) [2023-12-26 21:55:23,915][105620] Updated weights for policy 1, policy_version 901030 (0.0009) [2023-12-26 21:55:23,965][105620] Updated weights for policy 1, policy_version 901040 (0.0009) [2023-12-26 21:55:24,288][105692] Updated weights for policy 0, policy_version 901141 (0.0005) [2023-12-26 21:55:24,341][105692] Updated weights for policy 0, policy_version 901151 (0.0005) [2023-12-26 21:55:24,398][105692] Updated weights for policy 0, policy_version 901161 (0.0007) [2023-12-26 21:55:24,661][105620] Updated weights for policy 1, policy_version 901050 (0.0007) [2023-12-26 21:55:24,724][105620] Updated weights for policy 1, policy_version 901060 (0.0006) [2023-12-26 21:55:24,786][105620] Updated weights for policy 1, policy_version 901070 (0.0010) [2023-12-26 21:55:24,980][105692] Updated weights for policy 0, policy_version 901171 (0.0007) [2023-12-26 21:55:25,036][105692] Updated weights for policy 0, policy_version 901181 (0.0005) [2023-12-26 21:55:25,097][105692] Updated weights for policy 0, policy_version 901191 (0.0005) [2023-12-26 21:55:25,449][105620] Updated weights for policy 1, policy_version 901080 (0.0006) [2023-12-26 21:55:25,508][105620] Updated weights for policy 1, policy_version 901090 (0.0005) [2023-12-26 21:55:25,561][105620] Updated weights for policy 1, policy_version 901100 (0.0005) [2023-12-26 21:55:25,636][105692] Updated weights for policy 0, policy_version 901201 (0.0005) [2023-12-26 21:55:25,706][105692] Updated weights for policy 0, policy_version 901211 (0.0006) [2023-12-26 21:55:25,776][105692] Updated weights for policy 0, policy_version 901221 (0.0005) [2023-12-26 21:55:25,837][105692] Updated weights for policy 0, policy_version 901231 (0.0005) [2023-12-26 21:55:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19244.3). Total num frames: 461463552. Throughput: 0: 9743.7, 1: 9732.5. Samples: 461472720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:55:26,062][104569] Avg episode reward: [(0, '8417.432'), (1, '8463.182')] [2023-12-26 21:55:26,224][105620] Updated weights for policy 1, policy_version 901110 (0.0008) [2023-12-26 21:55:26,278][105620] Updated weights for policy 1, policy_version 901120 (0.0010) [2023-12-26 21:55:26,330][105620] Updated weights for policy 1, policy_version 901130 (0.0011) [2023-12-26 21:55:26,335][105692] Updated weights for policy 0, policy_version 901241 (0.0008) [2023-12-26 21:55:26,390][105692] Updated weights for policy 0, policy_version 901251 (0.0009) [2023-12-26 21:55:26,441][105692] Updated weights for policy 0, policy_version 901261 (0.0005) [2023-12-26 21:55:26,991][105692] Updated weights for policy 0, policy_version 901271 (0.0006) [2023-12-26 21:55:27,036][105692] Updated weights for policy 0, policy_version 901281 (0.0010) [2023-12-26 21:55:27,073][105620] Updated weights for policy 1, policy_version 901140 (0.0008) [2023-12-26 21:55:27,086][105692] Updated weights for policy 0, policy_version 901291 (0.0009) [2023-12-26 21:55:27,120][105620] Updated weights for policy 1, policy_version 901150 (0.0008) [2023-12-26 21:55:27,169][105620] Updated weights for policy 1, policy_version 901160 (0.0006) [2023-12-26 21:55:27,649][105692] Updated weights for policy 0, policy_version 901301 (0.0006) [2023-12-26 21:55:27,705][105692] Updated weights for policy 0, policy_version 901311 (0.0010) [2023-12-26 21:55:27,758][105692] Updated weights for policy 0, policy_version 901321 (0.0008) [2023-12-26 21:55:27,891][105620] Updated weights for policy 1, policy_version 901170 (0.0006) [2023-12-26 21:55:27,943][105620] Updated weights for policy 1, policy_version 901180 (0.0005) [2023-12-26 21:55:28,003][105620] Updated weights for policy 1, policy_version 901190 (0.0007) [2023-12-26 21:55:28,065][105620] Updated weights for policy 1, policy_version 901200 (0.0007) [2023-12-26 21:55:28,320][105692] Updated weights for policy 0, policy_version 901331 (0.0007) [2023-12-26 21:55:28,379][105692] Updated weights for policy 0, policy_version 901341 (0.0008) [2023-12-26 21:55:28,434][105692] Updated weights for policy 0, policy_version 901351 (0.0008) [2023-12-26 21:55:28,751][105620] Updated weights for policy 1, policy_version 901210 (0.0009) [2023-12-26 21:55:28,809][105620] Updated weights for policy 1, policy_version 901220 (0.0009) [2023-12-26 21:55:28,871][105620] Updated weights for policy 1, policy_version 901230 (0.0008) [2023-12-26 21:55:29,182][105692] Updated weights for policy 0, policy_version 901361 (0.0008) [2023-12-26 21:55:29,241][105692] Updated weights for policy 0, policy_version 901371 (0.0009) [2023-12-26 21:55:29,294][105692] Updated weights for policy 0, policy_version 901381 (0.0008) [2023-12-26 21:55:29,353][105692] Updated weights for policy 0, policy_version 901391 (0.0008) [2023-12-26 21:55:29,586][105620] Updated weights for policy 1, policy_version 901240 (0.0009) [2023-12-26 21:55:29,608][105586] KL-divergence is very high: 133.7181 [2023-12-26 21:55:29,644][105620] Updated weights for policy 1, policy_version 901250 (0.0009) [2023-12-26 21:55:29,654][105586] KL-divergence is very high: 235.8910 [2023-12-26 21:55:29,704][105586] KL-divergence is very high: 252.7134 [2023-12-26 21:55:29,705][105620] Updated weights for policy 1, policy_version 901260 (0.0009) [2023-12-26 21:55:30,086][105692] Updated weights for policy 0, policy_version 901401 (0.0009) [2023-12-26 21:55:30,153][105692] Updated weights for policy 0, policy_version 901411 (0.0007) [2023-12-26 21:55:30,216][105692] Updated weights for policy 0, policy_version 901421 (0.0005) [2023-12-26 21:55:30,489][105586] KL-divergence is very high: 216.2501 [2023-12-26 21:55:30,507][105620] Updated weights for policy 1, policy_version 901270 (0.0009) [2023-12-26 21:55:30,527][105586] KL-divergence is very high: 161.9089 [2023-12-26 21:55:30,554][105620] Updated weights for policy 1, policy_version 901280 (0.0009) [2023-12-26 21:55:30,566][105586] KL-divergence is very high: 107.4295 [2023-12-26 21:55:30,613][105620] Updated weights for policy 1, policy_version 901290 (0.0009) [2023-12-26 21:55:30,889][105692] Updated weights for policy 0, policy_version 901431 (0.0005) [2023-12-26 21:55:30,950][105692] Updated weights for policy 0, policy_version 901441 (0.0006) [2023-12-26 21:55:31,013][105692] Updated weights for policy 0, policy_version 901451 (0.0009) [2023-12-26 21:55:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19272.0). Total num frames: 461570048. Throughput: 0: 9891.7, 1: 9749.6. Samples: 461537428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:55:31,062][104569] Avg episode reward: [(0, '8999.566'), (1, '8192.981')] [2023-12-26 21:55:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000901456_230809600.pth... [2023-12-26 21:55:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000901296_230760448.pth... [2023-12-26 21:55:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000900304_230514688.pth [2023-12-26 21:55:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000900176_230473728.pth [2023-12-26 21:55:31,273][105620] Updated weights for policy 1, policy_version 901300 (0.0011) [2023-12-26 21:55:31,329][105620] Updated weights for policy 1, policy_version 901310 (0.0011) [2023-12-26 21:55:31,389][105620] Updated weights for policy 1, policy_version 901320 (0.0008) [2023-12-26 21:55:31,771][105692] Updated weights for policy 0, policy_version 901461 (0.0009) [2023-12-26 21:55:31,819][105692] Updated weights for policy 0, policy_version 901471 (0.0008) [2023-12-26 21:55:31,882][105692] Updated weights for policy 0, policy_version 901481 (0.0008) [2023-12-26 21:55:32,089][105620] Updated weights for policy 1, policy_version 901330 (0.0006) [2023-12-26 21:55:32,151][105620] Updated weights for policy 1, policy_version 901340 (0.0010) [2023-12-26 21:55:32,216][105620] Updated weights for policy 1, policy_version 901350 (0.0009) [2023-12-26 21:55:32,283][105620] Updated weights for policy 1, policy_version 901360 (0.0008) [2023-12-26 21:55:32,678][105692] Updated weights for policy 0, policy_version 901491 (0.0008) [2023-12-26 21:55:32,733][105692] Updated weights for policy 0, policy_version 901501 (0.0008) [2023-12-26 21:55:32,790][105692] Updated weights for policy 0, policy_version 901511 (0.0006) [2023-12-26 21:55:32,950][105620] Updated weights for policy 1, policy_version 901370 (0.0008) [2023-12-26 21:55:33,012][105620] Updated weights for policy 1, policy_version 901380 (0.0008) [2023-12-26 21:55:33,083][105620] Updated weights for policy 1, policy_version 901390 (0.0006) [2023-12-26 21:55:33,501][105692] Updated weights for policy 0, policy_version 901521 (0.0006) [2023-12-26 21:55:33,554][105692] Updated weights for policy 0, policy_version 901531 (0.0009) [2023-12-26 21:55:33,600][105692] Updated weights for policy 0, policy_version 901541 (0.0008) [2023-12-26 21:55:33,646][105692] Updated weights for policy 0, policy_version 901551 (0.0009) [2023-12-26 21:55:33,729][105620] Updated weights for policy 1, policy_version 901400 (0.0006) [2023-12-26 21:55:33,779][105620] Updated weights for policy 1, policy_version 901410 (0.0005) [2023-12-26 21:55:33,832][105620] Updated weights for policy 1, policy_version 901420 (0.0006) [2023-12-26 21:55:34,375][105620] Updated weights for policy 1, policy_version 901430 (0.0007) [2023-12-26 21:55:34,432][105620] Updated weights for policy 1, policy_version 901440 (0.0008) [2023-12-26 21:55:34,488][105620] Updated weights for policy 1, policy_version 901450 (0.0009) [2023-12-26 21:55:34,514][105692] Updated weights for policy 0, policy_version 901561 (0.0009) [2023-12-26 21:55:34,568][105692] Updated weights for policy 0, policy_version 901571 (0.0008) [2023-12-26 21:55:34,622][105692] Updated weights for policy 0, policy_version 901581 (0.0010) [2023-12-26 21:55:35,170][105620] Updated weights for policy 1, policy_version 901460 (0.0008) [2023-12-26 21:55:35,227][105620] Updated weights for policy 1, policy_version 901470 (0.0009) [2023-12-26 21:55:35,288][105620] Updated weights for policy 1, policy_version 901480 (0.0009) [2023-12-26 21:55:35,352][105692] Updated weights for policy 0, policy_version 901591 (0.0010) [2023-12-26 21:55:35,417][105692] Updated weights for policy 0, policy_version 901601 (0.0009) [2023-12-26 21:55:35,467][105692] Updated weights for policy 0, policy_version 901611 (0.0009) [2023-12-26 21:55:36,005][105620] Updated weights for policy 1, policy_version 901490 (0.0008) [2023-12-26 21:55:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19244.3). Total num frames: 461660160. Throughput: 0: 9753.2, 1: 9837.3. Samples: 461654504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:55:36,062][104569] Avg episode reward: [(0, '9085.437'), (1, '7819.491')] [2023-12-26 21:55:36,070][105620] Updated weights for policy 1, policy_version 901500 (0.0009) [2023-12-26 21:55:36,132][105620] Updated weights for policy 1, policy_version 901510 (0.0009) [2023-12-26 21:55:36,188][105620] Updated weights for policy 1, policy_version 901520 (0.0009) [2023-12-26 21:55:36,251][105692] Updated weights for policy 0, policy_version 901621 (0.0009) [2023-12-26 21:55:36,299][105692] Updated weights for policy 0, policy_version 901631 (0.0008) [2023-12-26 21:55:36,351][105692] Updated weights for policy 0, policy_version 901641 (0.0009) [2023-12-26 21:55:36,947][105620] Updated weights for policy 1, policy_version 901530 (0.0009) [2023-12-26 21:55:37,012][105620] Updated weights for policy 1, policy_version 901540 (0.0009) [2023-12-26 21:55:37,066][105620] Updated weights for policy 1, policy_version 901550 (0.0009) [2023-12-26 21:55:37,124][105692] Updated weights for policy 0, policy_version 901651 (0.0008) [2023-12-26 21:55:37,178][105692] Updated weights for policy 0, policy_version 901661 (0.0009) [2023-12-26 21:55:37,236][105692] Updated weights for policy 0, policy_version 901671 (0.0008) [2023-12-26 21:55:37,757][105620] Updated weights for policy 1, policy_version 901560 (0.0008) [2023-12-26 21:55:37,808][105620] Updated weights for policy 1, policy_version 901570 (0.0009) [2023-12-26 21:55:37,858][105620] Updated weights for policy 1, policy_version 901580 (0.0007) [2023-12-26 21:55:38,029][105692] Updated weights for policy 0, policy_version 901681 (0.0010) [2023-12-26 21:55:38,080][105692] Updated weights for policy 0, policy_version 901691 (0.0008) [2023-12-26 21:55:38,136][105692] Updated weights for policy 0, policy_version 901701 (0.0008) [2023-12-26 21:55:38,198][105692] Updated weights for policy 0, policy_version 901711 (0.0008) [2023-12-26 21:55:38,579][105620] Updated weights for policy 1, policy_version 901590 (0.0008) [2023-12-26 21:55:38,634][105620] Updated weights for policy 1, policy_version 901600 (0.0005) [2023-12-26 21:55:38,690][105620] Updated weights for policy 1, policy_version 901610 (0.0007) [2023-12-26 21:55:39,014][105692] Updated weights for policy 0, policy_version 901721 (0.0008) [2023-12-26 21:55:39,071][105692] Updated weights for policy 0, policy_version 901731 (0.0005) [2023-12-26 21:55:39,134][105692] Updated weights for policy 0, policy_version 901741 (0.0005) [2023-12-26 21:55:39,482][105620] Updated weights for policy 1, policy_version 901620 (0.0007) [2023-12-26 21:55:39,546][105620] Updated weights for policy 1, policy_version 901630 (0.0008) [2023-12-26 21:55:39,600][105620] Updated weights for policy 1, policy_version 901640 (0.0009) [2023-12-26 21:55:39,815][105692] Updated weights for policy 0, policy_version 901751 (0.0008) [2023-12-26 21:55:39,878][105692] Updated weights for policy 0, policy_version 901761 (0.0008) [2023-12-26 21:55:39,938][105692] Updated weights for policy 0, policy_version 901771 (0.0009) [2023-12-26 21:55:40,362][105620] Updated weights for policy 1, policy_version 901650 (0.0008) [2023-12-26 21:55:40,433][105620] Updated weights for policy 1, policy_version 901660 (0.0006) [2023-12-26 21:55:40,497][105620] Updated weights for policy 1, policy_version 901670 (0.0006) [2023-12-26 21:55:40,560][105620] Updated weights for policy 1, policy_version 901680 (0.0006) [2023-12-26 21:55:40,655][105692] Updated weights for policy 0, policy_version 901781 (0.0009) [2023-12-26 21:55:40,702][105692] Updated weights for policy 0, policy_version 901791 (0.0009) [2023-12-26 21:55:40,750][105692] Updated weights for policy 0, policy_version 901801 (0.0010) [2023-12-26 21:55:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19244.3). Total num frames: 461758464. Throughput: 0: 9598.7, 1: 9819.3. Samples: 461768220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:55:41,062][104569] Avg episode reward: [(0, '9085.346'), (1, '8446.957')] [2023-12-26 21:55:41,200][105620] Updated weights for policy 1, policy_version 901690 (0.0006) [2023-12-26 21:55:41,262][105620] Updated weights for policy 1, policy_version 901700 (0.0008) [2023-12-26 21:55:41,316][105620] Updated weights for policy 1, policy_version 901711 (0.0010) [2023-12-26 21:55:41,640][105692] Updated weights for policy 0, policy_version 901811 (0.0008) [2023-12-26 21:55:41,698][105692] Updated weights for policy 0, policy_version 901821 (0.0009) [2023-12-26 21:55:41,762][105692] Updated weights for policy 0, policy_version 901831 (0.0008) [2023-12-26 21:55:42,018][105620] Updated weights for policy 1, policy_version 901721 (0.0010) [2023-12-26 21:55:42,082][105620] Updated weights for policy 1, policy_version 901731 (0.0011) [2023-12-26 21:55:42,150][105620] Updated weights for policy 1, policy_version 901741 (0.0011) [2023-12-26 21:55:42,560][105692] Updated weights for policy 0, policy_version 901841 (0.0007) [2023-12-26 21:55:42,628][105692] Updated weights for policy 0, policy_version 901851 (0.0007) [2023-12-26 21:55:42,698][105692] Updated weights for policy 0, policy_version 901861 (0.0008) [2023-12-26 21:55:42,763][105692] Updated weights for policy 0, policy_version 901871 (0.0008) [2023-12-26 21:55:42,763][105620] Updated weights for policy 1, policy_version 901751 (0.0011) [2023-12-26 21:55:42,831][105620] Updated weights for policy 1, policy_version 901761 (0.0011) [2023-12-26 21:55:42,887][105620] Updated weights for policy 1, policy_version 901771 (0.0010) [2023-12-26 21:55:43,472][105692] Updated weights for policy 0, policy_version 901881 (0.0009) [2023-12-26 21:55:43,535][105692] Updated weights for policy 0, policy_version 901891 (0.0008) [2023-12-26 21:55:43,595][105692] Updated weights for policy 0, policy_version 901902 (0.0007) [2023-12-26 21:55:43,635][105620] Updated weights for policy 1, policy_version 901781 (0.0010) [2023-12-26 21:55:43,701][105620] Updated weights for policy 1, policy_version 901791 (0.0010) [2023-12-26 21:55:43,760][105620] Updated weights for policy 1, policy_version 901801 (0.0011) [2023-12-26 21:55:44,329][105692] Updated weights for policy 0, policy_version 901912 (0.0008) [2023-12-26 21:55:44,377][105692] Updated weights for policy 0, policy_version 901922 (0.0007) [2023-12-26 21:55:44,430][105692] Updated weights for policy 0, policy_version 901932 (0.0005) [2023-12-26 21:55:44,489][105620] Updated weights for policy 1, policy_version 901811 (0.0010) [2023-12-26 21:55:44,545][105620] Updated weights for policy 1, policy_version 901821 (0.0010) [2023-12-26 21:55:44,605][105620] Updated weights for policy 1, policy_version 901831 (0.0008) [2023-12-26 21:55:45,182][105692] Updated weights for policy 0, policy_version 901942 (0.0008) [2023-12-26 21:55:45,243][105692] Updated weights for policy 0, policy_version 901952 (0.0009) [2023-12-26 21:55:45,299][105692] Updated weights for policy 0, policy_version 901962 (0.0009) [2023-12-26 21:55:45,321][105620] Updated weights for policy 1, policy_version 901841 (0.0006) [2023-12-26 21:55:45,382][105620] Updated weights for policy 1, policy_version 901851 (0.0009) [2023-12-26 21:55:45,445][105620] Updated weights for policy 1, policy_version 901861 (0.0009) [2023-12-26 21:55:45,507][105620] Updated weights for policy 1, policy_version 901871 (0.0009) [2023-12-26 21:55:46,045][105692] Updated weights for policy 0, policy_version 901972 (0.0008) [2023-12-26 21:55:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19244.3). Total num frames: 461848576. Throughput: 0: 9548.1, 1: 9816.0. Samples: 461824012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:55:46,062][104569] Avg episode reward: [(0, '9174.579'), (1, '8543.115')] [2023-12-26 21:55:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000901872_230907904.pth... [2023-12-26 21:55:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000900720_230612992.pth [2023-12-26 21:55:46,102][105692] Updated weights for policy 0, policy_version 901982 (0.0009) [2023-12-26 21:55:46,162][105692] Updated weights for policy 0, policy_version 901992 (0.0008) [2023-12-26 21:55:46,206][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000902000_230948864.pth... [2023-12-26 21:55:46,209][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000900848_230653952.pth [2023-12-26 21:55:46,255][105620] Updated weights for policy 1, policy_version 901881 (0.0009) [2023-12-26 21:55:46,325][105620] Updated weights for policy 1, policy_version 901891 (0.0009) [2023-12-26 21:55:46,387][105620] Updated weights for policy 1, policy_version 901901 (0.0009) [2023-12-26 21:55:46,923][105692] Updated weights for policy 0, policy_version 902002 (0.0008) [2023-12-26 21:55:46,982][105692] Updated weights for policy 0, policy_version 902012 (0.0009) [2023-12-26 21:55:47,042][105692] Updated weights for policy 0, policy_version 902022 (0.0008) [2023-12-26 21:55:47,070][105620] Updated weights for policy 1, policy_version 901911 (0.0008) [2023-12-26 21:55:47,097][105692] Updated weights for policy 0, policy_version 902032 (0.0007) [2023-12-26 21:55:47,133][105620] Updated weights for policy 1, policy_version 901921 (0.0009) [2023-12-26 21:55:47,157][105586] KL-divergence is very high: 105.2211 [2023-12-26 21:55:47,195][105620] Updated weights for policy 1, policy_version 901931 (0.0009) [2023-12-26 21:55:47,206][105586] KL-divergence is very high: 192.7248 [2023-12-26 21:55:47,885][105692] Updated weights for policy 0, policy_version 902042 (0.0009) [2023-12-26 21:55:47,888][105620] Updated weights for policy 1, policy_version 901941 (0.0007) [2023-12-26 21:55:47,946][105620] Updated weights for policy 1, policy_version 901951 (0.0006) [2023-12-26 21:55:47,948][105692] Updated weights for policy 0, policy_version 902052 (0.0007) [2023-12-26 21:55:47,991][105620] Updated weights for policy 1, policy_version 901961 (0.0007) [2023-12-26 21:55:48,003][105692] Updated weights for policy 0, policy_version 902062 (0.0009) [2023-12-26 21:55:48,719][105692] Updated weights for policy 0, policy_version 902072 (0.0008) [2023-12-26 21:55:48,757][105620] Updated weights for policy 1, policy_version 901971 (0.0009) [2023-12-26 21:55:48,779][105692] Updated weights for policy 0, policy_version 902082 (0.0006) [2023-12-26 21:55:48,824][105620] Updated weights for policy 1, policy_version 901981 (0.0010) [2023-12-26 21:55:48,842][105692] Updated weights for policy 0, policy_version 902092 (0.0008) [2023-12-26 21:55:48,884][105620] Updated weights for policy 1, policy_version 901991 (0.0010) [2023-12-26 21:55:49,618][105692] Updated weights for policy 0, policy_version 902102 (0.0007) [2023-12-26 21:55:49,627][105620] Updated weights for policy 1, policy_version 902001 (0.0011) [2023-12-26 21:55:49,678][105692] Updated weights for policy 0, policy_version 902112 (0.0006) [2023-12-26 21:55:49,680][105620] Updated weights for policy 1, policy_version 902011 (0.0009) [2023-12-26 21:55:49,738][105692] Updated weights for policy 0, policy_version 902122 (0.0006) [2023-12-26 21:55:49,743][105620] Updated weights for policy 1, policy_version 902021 (0.0010) [2023-12-26 21:55:49,797][105620] Updated weights for policy 1, policy_version 902031 (0.0005) [2023-12-26 21:55:50,385][105692] Updated weights for policy 0, policy_version 902132 (0.0007) [2023-12-26 21:55:50,436][105692] Updated weights for policy 0, policy_version 902142 (0.0005) [2023-12-26 21:55:50,495][105692] Updated weights for policy 0, policy_version 902152 (0.0006) [2023-12-26 21:55:50,532][105620] Updated weights for policy 1, policy_version 902041 (0.0007) [2023-12-26 21:55:50,594][105620] Updated weights for policy 1, policy_version 902051 (0.0008) [2023-12-26 21:55:50,647][105620] Updated weights for policy 1, policy_version 902061 (0.0008) [2023-12-26 21:55:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19244.3). Total num frames: 461946880. Throughput: 0: 9588.6, 1: 9705.5. Samples: 461937008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:55:51,063][104569] Avg episode reward: [(0, '9173.092'), (1, '8004.112')] [2023-12-26 21:55:51,297][105692] Updated weights for policy 0, policy_version 902162 (0.0006) [2023-12-26 21:55:51,315][105620] Updated weights for policy 1, policy_version 902071 (0.0007) [2023-12-26 21:55:51,358][105692] Updated weights for policy 0, policy_version 902172 (0.0009) [2023-12-26 21:55:51,378][105620] Updated weights for policy 1, policy_version 902081 (0.0010) [2023-12-26 21:55:51,421][105692] Updated weights for policy 0, policy_version 902182 (0.0008) [2023-12-26 21:55:51,438][105620] Updated weights for policy 1, policy_version 902091 (0.0011) [2023-12-26 21:55:51,480][105692] Updated weights for policy 0, policy_version 902192 (0.0006) [2023-12-26 21:55:52,081][105620] Updated weights for policy 1, policy_version 902101 (0.0010) [2023-12-26 21:55:52,145][105620] Updated weights for policy 1, policy_version 902111 (0.0008) [2023-12-26 21:55:52,210][105620] Updated weights for policy 1, policy_version 902121 (0.0010) [2023-12-26 21:55:52,300][105692] Updated weights for policy 0, policy_version 902202 (0.0009) [2023-12-26 21:55:52,366][105692] Updated weights for policy 0, policy_version 902212 (0.0008) [2023-12-26 21:55:52,420][105692] Updated weights for policy 0, policy_version 902222 (0.0009) [2023-12-26 21:55:52,920][105620] Updated weights for policy 1, policy_version 902131 (0.0009) [2023-12-26 21:55:52,985][105620] Updated weights for policy 1, policy_version 902141 (0.0009) [2023-12-26 21:55:53,044][105620] Updated weights for policy 1, policy_version 902151 (0.0009) [2023-12-26 21:55:53,173][105692] Updated weights for policy 0, policy_version 902232 (0.0009) [2023-12-26 21:55:53,223][105692] Updated weights for policy 0, policy_version 902242 (0.0008) [2023-12-26 21:55:53,271][105692] Updated weights for policy 0, policy_version 902252 (0.0009) [2023-12-26 21:55:53,719][105620] Updated weights for policy 1, policy_version 902161 (0.0008) [2023-12-26 21:55:53,777][105620] Updated weights for policy 1, policy_version 902171 (0.0009) [2023-12-26 21:55:53,841][105620] Updated weights for policy 1, policy_version 902181 (0.0008) [2023-12-26 21:55:53,905][105620] Updated weights for policy 1, policy_version 902191 (0.0008) [2023-12-26 21:55:54,075][105692] Updated weights for policy 0, policy_version 902262 (0.0009) [2023-12-26 21:55:54,135][105692] Updated weights for policy 0, policy_version 902272 (0.0009) [2023-12-26 21:55:54,201][105692] Updated weights for policy 0, policy_version 902282 (0.0010) [2023-12-26 21:55:54,668][105620] Updated weights for policy 1, policy_version 902201 (0.0009) [2023-12-26 21:55:54,720][105620] Updated weights for policy 1, policy_version 902211 (0.0009) [2023-12-26 21:55:54,776][105620] Updated weights for policy 1, policy_version 902221 (0.0009) [2023-12-26 21:55:54,865][105692] Updated weights for policy 0, policy_version 902292 (0.0009) [2023-12-26 21:55:54,913][105692] Updated weights for policy 0, policy_version 902302 (0.0008) [2023-12-26 21:55:54,962][105692] Updated weights for policy 0, policy_version 902312 (0.0009) [2023-12-26 21:55:55,614][105620] Updated weights for policy 1, policy_version 902231 (0.0009) [2023-12-26 21:55:55,626][105692] Updated weights for policy 0, policy_version 902322 (0.0007) [2023-12-26 21:55:55,657][105620] Updated weights for policy 1, policy_version 902241 (0.0006) [2023-12-26 21:55:55,670][105692] Updated weights for policy 0, policy_version 902332 (0.0010) [2023-12-26 21:55:55,703][105620] Updated weights for policy 1, policy_version 902251 (0.0006) [2023-12-26 21:55:55,716][105692] Updated weights for policy 0, policy_version 902342 (0.0008) [2023-12-26 21:55:55,757][105692] Updated weights for policy 0, policy_version 902352 (0.0005) [2023-12-26 21:55:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19244.3). Total num frames: 462045184. Throughput: 0: 9610.2, 1: 9705.8. Samples: 462051492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:55:56,062][104569] Avg episode reward: [(0, '9084.790'), (1, '7725.119')] [2023-12-26 21:55:56,460][105692] Updated weights for policy 0, policy_version 902362 (0.0011) [2023-12-26 21:55:56,508][105620] Updated weights for policy 1, policy_version 902261 (0.0006) [2023-12-26 21:55:56,516][105692] Updated weights for policy 0, policy_version 902372 (0.0010) [2023-12-26 21:55:56,565][105620] Updated weights for policy 1, policy_version 902271 (0.0005) [2023-12-26 21:55:56,575][105692] Updated weights for policy 0, policy_version 902382 (0.0011) [2023-12-26 21:55:56,615][105620] Updated weights for policy 1, policy_version 902281 (0.0007) [2023-12-26 21:55:57,205][105692] Updated weights for policy 0, policy_version 902392 (0.0007) [2023-12-26 21:55:57,268][105692] Updated weights for policy 0, policy_version 902402 (0.0005) [2023-12-26 21:55:57,284][105620] Updated weights for policy 1, policy_version 902291 (0.0008) [2023-12-26 21:55:57,322][105692] Updated weights for policy 0, policy_version 902412 (0.0006) [2023-12-26 21:55:57,330][105620] Updated weights for policy 1, policy_version 902301 (0.0009) [2023-12-26 21:55:57,387][105620] Updated weights for policy 1, policy_version 902311 (0.0006) [2023-12-26 21:55:57,842][105692] Updated weights for policy 0, policy_version 902422 (0.0006) [2023-12-26 21:55:57,885][105692] Updated weights for policy 0, policy_version 902432 (0.0006) [2023-12-26 21:55:57,931][105692] Updated weights for policy 0, policy_version 902442 (0.0005) [2023-12-26 21:55:58,034][105620] Updated weights for policy 1, policy_version 902321 (0.0006) [2023-12-26 21:55:58,079][105620] Updated weights for policy 1, policy_version 902331 (0.0009) [2023-12-26 21:55:58,131][105620] Updated weights for policy 1, policy_version 902341 (0.0009) [2023-12-26 21:55:58,199][105620] Updated weights for policy 1, policy_version 902351 (0.0009) [2023-12-26 21:55:58,634][105692] Updated weights for policy 0, policy_version 902452 (0.0006) [2023-12-26 21:55:58,703][105692] Updated weights for policy 0, policy_version 902462 (0.0010) [2023-12-26 21:55:58,772][105692] Updated weights for policy 0, policy_version 902472 (0.0009) [2023-12-26 21:55:59,019][105620] Updated weights for policy 1, policy_version 902361 (0.0009) [2023-12-26 21:55:59,079][105620] Updated weights for policy 1, policy_version 902371 (0.0009) [2023-12-26 21:55:59,132][105620] Updated weights for policy 1, policy_version 902381 (0.0009) [2023-12-26 21:55:59,552][105692] Updated weights for policy 0, policy_version 902482 (0.0009) [2023-12-26 21:55:59,618][105692] Updated weights for policy 0, policy_version 902492 (0.0006) [2023-12-26 21:55:59,681][105692] Updated weights for policy 0, policy_version 902502 (0.0006) [2023-12-26 21:55:59,735][105692] Updated weights for policy 0, policy_version 902512 (0.0007) [2023-12-26 21:55:59,928][105620] Updated weights for policy 1, policy_version 902391 (0.0009) [2023-12-26 21:55:59,989][105620] Updated weights for policy 1, policy_version 902401 (0.0011) [2023-12-26 21:56:00,037][105620] Updated weights for policy 1, policy_version 902411 (0.0010) [2023-12-26 21:56:00,448][105692] Updated weights for policy 0, policy_version 902522 (0.0008) [2023-12-26 21:56:00,492][105692] Updated weights for policy 0, policy_version 902532 (0.0008) [2023-12-26 21:56:00,551][105692] Updated weights for policy 0, policy_version 902542 (0.0008) [2023-12-26 21:56:00,782][105620] Updated weights for policy 1, policy_version 902421 (0.0010) [2023-12-26 21:56:00,826][105620] Updated weights for policy 1, policy_version 902431 (0.0010) [2023-12-26 21:56:00,873][105620] Updated weights for policy 1, policy_version 902441 (0.0010) [2023-12-26 21:56:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19244.3). Total num frames: 462143488. Throughput: 0: 9710.4, 1: 9727.7. Samples: 462113932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:56:01,062][104569] Avg episode reward: [(0, '9174.706'), (1, '8007.828')] [2023-12-26 21:56:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000902544_231088128.pth... [2023-12-26 21:56:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000902448_231055360.pth... [2023-12-26 21:56:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000901456_230809600.pth [2023-12-26 21:56:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000901296_230760448.pth [2023-12-26 21:56:01,317][105692] Updated weights for policy 0, policy_version 902552 (0.0007) [2023-12-26 21:56:01,386][105692] Updated weights for policy 0, policy_version 902562 (0.0008) [2023-12-26 21:56:01,453][105692] Updated weights for policy 0, policy_version 902572 (0.0010) [2023-12-26 21:56:01,638][105620] Updated weights for policy 1, policy_version 902451 (0.0010) [2023-12-26 21:56:01,701][105620] Updated weights for policy 1, policy_version 902461 (0.0010) [2023-12-26 21:56:01,760][105620] Updated weights for policy 1, policy_version 902471 (0.0008) [2023-12-26 21:56:02,162][105692] Updated weights for policy 0, policy_version 902582 (0.0008) [2023-12-26 21:56:02,220][105692] Updated weights for policy 0, policy_version 902592 (0.0008) [2023-12-26 21:56:02,276][105692] Updated weights for policy 0, policy_version 902602 (0.0008) [2023-12-26 21:56:02,532][105620] Updated weights for policy 1, policy_version 902481 (0.0008) [2023-12-26 21:56:02,581][105620] Updated weights for policy 1, policy_version 902491 (0.0010) [2023-12-26 21:56:02,629][105620] Updated weights for policy 1, policy_version 902501 (0.0010) [2023-12-26 21:56:02,685][105620] Updated weights for policy 1, policy_version 902511 (0.0011) [2023-12-26 21:56:02,973][105692] Updated weights for policy 0, policy_version 902612 (0.0007) [2023-12-26 21:56:03,042][105692] Updated weights for policy 0, policy_version 902622 (0.0006) [2023-12-26 21:56:03,095][105692] Updated weights for policy 0, policy_version 902632 (0.0006) [2023-12-26 21:56:03,333][105620] Updated weights for policy 1, policy_version 902521 (0.0006) [2023-12-26 21:56:03,380][105620] Updated weights for policy 1, policy_version 902531 (0.0005) [2023-12-26 21:56:03,426][105620] Updated weights for policy 1, policy_version 902541 (0.0005) [2023-12-26 21:56:03,626][105692] Updated weights for policy 0, policy_version 902642 (0.0008) [2023-12-26 21:56:03,685][105692] Updated weights for policy 0, policy_version 902652 (0.0005) [2023-12-26 21:56:03,743][105692] Updated weights for policy 0, policy_version 902662 (0.0005) [2023-12-26 21:56:03,792][105692] Updated weights for policy 0, policy_version 902672 (0.0005) [2023-12-26 21:56:03,961][105620] Updated weights for policy 1, policy_version 902551 (0.0006) [2023-12-26 21:56:04,022][105620] Updated weights for policy 1, policy_version 902561 (0.0007) [2023-12-26 21:56:04,083][105620] Updated weights for policy 1, policy_version 902571 (0.0006) [2023-12-26 21:56:04,431][105692] Updated weights for policy 0, policy_version 902682 (0.0007) [2023-12-26 21:56:04,488][105692] Updated weights for policy 0, policy_version 902692 (0.0010) [2023-12-26 21:56:04,545][105692] Updated weights for policy 0, policy_version 902702 (0.0006) [2023-12-26 21:56:04,781][105620] Updated weights for policy 1, policy_version 902581 (0.0008) [2023-12-26 21:56:04,836][105620] Updated weights for policy 1, policy_version 902591 (0.0007) [2023-12-26 21:56:04,884][105620] Updated weights for policy 1, policy_version 902601 (0.0007) [2023-12-26 21:56:05,294][105692] Updated weights for policy 0, policy_version 902712 (0.0008) [2023-12-26 21:56:05,348][105692] Updated weights for policy 0, policy_version 902722 (0.0008) [2023-12-26 21:56:05,394][105692] Updated weights for policy 0, policy_version 902732 (0.0008) [2023-12-26 21:56:05,600][105620] Updated weights for policy 1, policy_version 902611 (0.0005) [2023-12-26 21:56:05,664][105620] Updated weights for policy 1, policy_version 902621 (0.0008) [2023-12-26 21:56:05,720][105620] Updated weights for policy 1, policy_version 902631 (0.0009) [2023-12-26 21:56:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19244.3). Total num frames: 462241792. Throughput: 0: 9768.2, 1: 9725.8. Samples: 462231952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:56:06,062][104569] Avg episode reward: [(0, '9352.139'), (1, '7484.966')] [2023-12-26 21:56:06,100][105692] Updated weights for policy 0, policy_version 902742 (0.0007) [2023-12-26 21:56:06,161][105692] Updated weights for policy 0, policy_version 902752 (0.0007) [2023-12-26 21:56:06,214][105692] Updated weights for policy 0, policy_version 902762 (0.0005) [2023-12-26 21:56:06,404][105620] Updated weights for policy 1, policy_version 902642 (0.0010) [2023-12-26 21:56:06,463][105620] Updated weights for policy 1, policy_version 902652 (0.0009) [2023-12-26 21:56:06,521][105620] Updated weights for policy 1, policy_version 902662 (0.0010) [2023-12-26 21:56:06,581][105620] Updated weights for policy 1, policy_version 902672 (0.0009) [2023-12-26 21:56:06,853][105692] Updated weights for policy 0, policy_version 902772 (0.0008) [2023-12-26 21:56:06,909][105692] Updated weights for policy 0, policy_version 902782 (0.0011) [2023-12-26 21:56:06,969][105692] Updated weights for policy 0, policy_version 902792 (0.0011) [2023-12-26 21:56:07,373][105620] Updated weights for policy 1, policy_version 902682 (0.0008) [2023-12-26 21:56:07,437][105620] Updated weights for policy 1, policy_version 902692 (0.0008) [2023-12-26 21:56:07,499][105620] Updated weights for policy 1, policy_version 902702 (0.0006) [2023-12-26 21:56:07,651][105692] Updated weights for policy 0, policy_version 902802 (0.0010) [2023-12-26 21:56:07,707][105692] Updated weights for policy 0, policy_version 902812 (0.0005) [2023-12-26 21:56:07,764][105692] Updated weights for policy 0, policy_version 902822 (0.0005) [2023-12-26 21:56:07,810][105692] Updated weights for policy 0, policy_version 902832 (0.0005) [2023-12-26 21:56:08,220][105620] Updated weights for policy 1, policy_version 902712 (0.0008) [2023-12-26 21:56:08,277][105620] Updated weights for policy 1, policy_version 902722 (0.0009) [2023-12-26 21:56:08,339][105620] Updated weights for policy 1, policy_version 902732 (0.0008) [2023-12-26 21:56:08,427][105692] Updated weights for policy 0, policy_version 902842 (0.0007) [2023-12-26 21:56:08,496][105692] Updated weights for policy 0, policy_version 902852 (0.0006) [2023-12-26 21:56:08,552][105692] Updated weights for policy 0, policy_version 902862 (0.0005) [2023-12-26 21:56:09,039][105620] Updated weights for policy 1, policy_version 902742 (0.0007) [2023-12-26 21:56:09,095][105620] Updated weights for policy 1, policy_version 902752 (0.0009) [2023-12-26 21:56:09,150][105620] Updated weights for policy 1, policy_version 902762 (0.0005) [2023-12-26 21:56:09,158][105692] Updated weights for policy 0, policy_version 902872 (0.0008) [2023-12-26 21:56:09,207][105692] Updated weights for policy 0, policy_version 902882 (0.0007) [2023-12-26 21:56:09,266][105692] Updated weights for policy 0, policy_version 902892 (0.0008) [2023-12-26 21:56:09,726][105620] Updated weights for policy 1, policy_version 902772 (0.0009) [2023-12-26 21:56:09,778][105620] Updated weights for policy 1, policy_version 902782 (0.0010) [2023-12-26 21:56:09,841][105620] Updated weights for policy 1, policy_version 902792 (0.0011) [2023-12-26 21:56:10,087][105692] Updated weights for policy 0, policy_version 902902 (0.0007) [2023-12-26 21:56:10,151][105692] Updated weights for policy 0, policy_version 902912 (0.0008) [2023-12-26 21:56:10,211][105692] Updated weights for policy 0, policy_version 902922 (0.0009) [2023-12-26 21:56:10,594][105620] Updated weights for policy 1, policy_version 902802 (0.0011) [2023-12-26 21:56:10,642][105620] Updated weights for policy 1, policy_version 902812 (0.0010) [2023-12-26 21:56:10,691][105620] Updated weights for policy 1, policy_version 902822 (0.0010) [2023-12-26 21:56:10,746][105620] Updated weights for policy 1, policy_version 902832 (0.0010) [2023-12-26 21:56:10,972][105692] Updated weights for policy 0, policy_version 902932 (0.0007) [2023-12-26 21:56:11,041][105692] Updated weights for policy 0, policy_version 902942 (0.0007) [2023-12-26 21:56:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19244.3). Total num frames: 462340096. Throughput: 0: 9759.7, 1: 9746.3. Samples: 462350492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:56:11,063][104569] Avg episode reward: [(0, '9084.876'), (1, '7657.209')] [2023-12-26 21:56:11,107][105692] Updated weights for policy 0, policy_version 902952 (0.0007) [2023-12-26 21:56:11,527][105620] Updated weights for policy 1, policy_version 902842 (0.0005) [2023-12-26 21:56:11,592][105620] Updated weights for policy 1, policy_version 902852 (0.0006) [2023-12-26 21:56:11,658][105620] Updated weights for policy 1, policy_version 902862 (0.0009) [2023-12-26 21:56:11,897][105692] Updated weights for policy 0, policy_version 902962 (0.0009) [2023-12-26 21:56:11,946][105692] Updated weights for policy 0, policy_version 902972 (0.0008) [2023-12-26 21:56:12,003][105692] Updated weights for policy 0, policy_version 902982 (0.0008) [2023-12-26 21:56:12,061][105692] Updated weights for policy 0, policy_version 902992 (0.0008) [2023-12-26 21:56:12,381][105620] Updated weights for policy 1, policy_version 902872 (0.0011) [2023-12-26 21:56:12,444][105620] Updated weights for policy 1, policy_version 902882 (0.0010) [2023-12-26 21:56:12,511][105620] Updated weights for policy 1, policy_version 902892 (0.0008) [2023-12-26 21:56:12,802][105692] Updated weights for policy 0, policy_version 903002 (0.0010) [2023-12-26 21:56:12,867][105692] Updated weights for policy 0, policy_version 903012 (0.0010) [2023-12-26 21:56:12,938][105692] Updated weights for policy 0, policy_version 903022 (0.0010) [2023-12-26 21:56:13,105][105620] Updated weights for policy 1, policy_version 902902 (0.0008) [2023-12-26 21:56:13,165][105620] Updated weights for policy 1, policy_version 902912 (0.0008) [2023-12-26 21:56:13,220][105620] Updated weights for policy 1, policy_version 902922 (0.0008) [2023-12-26 21:56:13,613][105692] Updated weights for policy 0, policy_version 903032 (0.0006) [2023-12-26 21:56:13,683][105692] Updated weights for policy 0, policy_version 903042 (0.0006) [2023-12-26 21:56:13,736][105692] Updated weights for policy 0, policy_version 903052 (0.0011) [2023-12-26 21:56:13,966][105620] Updated weights for policy 1, policy_version 902932 (0.0009) [2023-12-26 21:56:14,021][105620] Updated weights for policy 1, policy_version 902942 (0.0010) [2023-12-26 21:56:14,083][105620] Updated weights for policy 1, policy_version 902952 (0.0010) [2023-12-26 21:56:14,431][105692] Updated weights for policy 0, policy_version 903062 (0.0010) [2023-12-26 21:56:14,479][105692] Updated weights for policy 0, policy_version 903072 (0.0009) [2023-12-26 21:56:14,540][105692] Updated weights for policy 0, policy_version 903082 (0.0008) [2023-12-26 21:56:14,735][105620] Updated weights for policy 1, policy_version 902962 (0.0009) [2023-12-26 21:56:14,795][105620] Updated weights for policy 1, policy_version 902972 (0.0007) [2023-12-26 21:56:14,846][105620] Updated weights for policy 1, policy_version 902982 (0.0006) [2023-12-26 21:56:14,906][105620] Updated weights for policy 1, policy_version 902992 (0.0009) [2023-12-26 21:56:15,291][105692] Updated weights for policy 0, policy_version 903092 (0.0010) [2023-12-26 21:56:15,359][105692] Updated weights for policy 0, policy_version 903102 (0.0009) [2023-12-26 21:56:15,425][105692] Updated weights for policy 0, policy_version 903112 (0.0009) [2023-12-26 21:56:15,598][105620] Updated weights for policy 1, policy_version 903002 (0.0006) [2023-12-26 21:56:15,631][105586] KL-divergence is very high: 227.1274 [2023-12-26 21:56:15,649][105586] KL-divergence is very high: 188.0349 [2023-12-26 21:56:15,659][105620] Updated weights for policy 1, policy_version 903012 (0.0009) [2023-12-26 21:56:15,679][105586] KL-divergence is very high: 415.4237 [2023-12-26 21:56:15,697][105586] KL-divergence is very high: 230.5968 [2023-12-26 21:56:15,721][105620] Updated weights for policy 1, policy_version 903022 (0.0009) [2023-12-26 21:56:15,728][105586] KL-divergence is very high: 438.6000 [2023-12-26 21:56:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19272.0). Total num frames: 462438400. Throughput: 0: 9598.1, 1: 9760.3. Samples: 462408560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:56:16,063][104569] Avg episode reward: [(0, '8825.213'), (1, '8182.117')] [2023-12-26 21:56:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000903024_231202816.pth... [2023-12-26 21:56:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000903120_231235584.pth... [2023-12-26 21:56:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000901872_230907904.pth [2023-12-26 21:56:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000902000_230948864.pth [2023-12-26 21:56:16,204][105692] Updated weights for policy 0, policy_version 903122 (0.0009) [2023-12-26 21:56:16,258][105692] Updated weights for policy 0, policy_version 903132 (0.0009) [2023-12-26 21:56:16,313][105692] Updated weights for policy 0, policy_version 903142 (0.0009) [2023-12-26 21:56:16,364][105692] Updated weights for policy 0, policy_version 903152 (0.0009) [2023-12-26 21:56:16,418][105620] Updated weights for policy 1, policy_version 903032 (0.0008) [2023-12-26 21:56:16,465][105620] Updated weights for policy 1, policy_version 903042 (0.0009) [2023-12-26 21:56:16,511][105620] Updated weights for policy 1, policy_version 903052 (0.0008) [2023-12-26 21:56:17,011][105692] Updated weights for policy 0, policy_version 903162 (0.0007) [2023-12-26 21:56:17,081][105692] Updated weights for policy 0, policy_version 903172 (0.0005) [2023-12-26 21:56:17,147][105692] Updated weights for policy 0, policy_version 903182 (0.0008) [2023-12-26 21:56:17,365][105620] Updated weights for policy 1, policy_version 903062 (0.0008) [2023-12-26 21:56:17,423][105620] Updated weights for policy 1, policy_version 903072 (0.0007) [2023-12-26 21:56:17,488][105620] Updated weights for policy 1, policy_version 903082 (0.0007) [2023-12-26 21:56:17,828][105692] Updated weights for policy 0, policy_version 903192 (0.0008) [2023-12-26 21:56:17,876][105692] Updated weights for policy 0, policy_version 903202 (0.0009) [2023-12-26 21:56:17,937][105692] Updated weights for policy 0, policy_version 903212 (0.0008) [2023-12-26 21:56:18,224][105620] Updated weights for policy 1, policy_version 903092 (0.0009) [2023-12-26 21:56:18,285][105620] Updated weights for policy 1, policy_version 903102 (0.0009) [2023-12-26 21:56:18,352][105620] Updated weights for policy 1, policy_version 903112 (0.0009) [2023-12-26 21:56:18,731][105692] Updated weights for policy 0, policy_version 903222 (0.0009) [2023-12-26 21:56:18,782][105692] Updated weights for policy 0, policy_version 903232 (0.0008) [2023-12-26 21:56:18,851][105692] Updated weights for policy 0, policy_version 903242 (0.0010) [2023-12-26 21:56:19,058][105620] Updated weights for policy 1, policy_version 903122 (0.0009) [2023-12-26 21:56:19,118][105620] Updated weights for policy 1, policy_version 903132 (0.0009) [2023-12-26 21:56:19,174][105620] Updated weights for policy 1, policy_version 903142 (0.0008) [2023-12-26 21:56:19,239][105620] Updated weights for policy 1, policy_version 903152 (0.0009) [2023-12-26 21:56:19,621][105692] Updated weights for policy 0, policy_version 903252 (0.0009) [2023-12-26 21:56:19,673][105692] Updated weights for policy 0, policy_version 903262 (0.0009) [2023-12-26 21:56:19,725][105692] Updated weights for policy 0, policy_version 903272 (0.0009) [2023-12-26 21:56:20,008][105620] Updated weights for policy 1, policy_version 903162 (0.0006) [2023-12-26 21:56:20,075][105620] Updated weights for policy 1, policy_version 903172 (0.0006) [2023-12-26 21:56:20,141][105620] Updated weights for policy 1, policy_version 903182 (0.0007) [2023-12-26 21:56:20,549][105692] Updated weights for policy 0, policy_version 903282 (0.0009) [2023-12-26 21:56:20,619][105692] Updated weights for policy 0, policy_version 903292 (0.0009) [2023-12-26 21:56:20,678][105692] Updated weights for policy 0, policy_version 903302 (0.0010) [2023-12-26 21:56:20,739][105692] Updated weights for policy 0, policy_version 903312 (0.0009) [2023-12-26 21:56:20,822][105620] Updated weights for policy 1, policy_version 903192 (0.0006) [2023-12-26 21:56:20,891][105620] Updated weights for policy 1, policy_version 903202 (0.0007) [2023-12-26 21:56:20,947][105620] Updated weights for policy 1, policy_version 903212 (0.0009) [2023-12-26 21:56:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19272.0). Total num frames: 462536704. Throughput: 0: 9615.5, 1: 9676.9. Samples: 462522664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:56:21,063][104569] Avg episode reward: [(0, '8915.254'), (1, '8456.737')] [2023-12-26 21:56:21,577][105692] Updated weights for policy 0, policy_version 903322 (0.0009) [2023-12-26 21:56:21,633][105692] Updated weights for policy 0, policy_version 903332 (0.0009) [2023-12-26 21:56:21,672][105620] Updated weights for policy 1, policy_version 903222 (0.0010) [2023-12-26 21:56:21,687][105692] Updated weights for policy 0, policy_version 903342 (0.0008) [2023-12-26 21:56:21,737][105620] Updated weights for policy 1, policy_version 903232 (0.0008) [2023-12-26 21:56:21,800][105620] Updated weights for policy 1, policy_version 903242 (0.0008) [2023-12-26 21:56:22,521][105620] Updated weights for policy 1, policy_version 903252 (0.0007) [2023-12-26 21:56:22,543][105692] Updated weights for policy 0, policy_version 903352 (0.0007) [2023-12-26 21:56:22,585][105620] Updated weights for policy 1, policy_version 903262 (0.0008) [2023-12-26 21:56:22,604][105692] Updated weights for policy 0, policy_version 903362 (0.0009) [2023-12-26 21:56:22,648][105620] Updated weights for policy 1, policy_version 903272 (0.0008) [2023-12-26 21:56:22,674][105692] Updated weights for policy 0, policy_version 903372 (0.0009) [2023-12-26 21:56:23,355][105692] Updated weights for policy 0, policy_version 903382 (0.0010) [2023-12-26 21:56:23,382][105620] Updated weights for policy 1, policy_version 903282 (0.0009) [2023-12-26 21:56:23,408][105692] Updated weights for policy 0, policy_version 903392 (0.0008) [2023-12-26 21:56:23,435][105620] Updated weights for policy 1, policy_version 903292 (0.0007) [2023-12-26 21:56:23,456][105692] Updated weights for policy 0, policy_version 903402 (0.0007) [2023-12-26 21:56:23,483][105620] Updated weights for policy 1, policy_version 903302 (0.0005) [2023-12-26 21:56:23,534][105620] Updated weights for policy 1, policy_version 903312 (0.0008) [2023-12-26 21:56:24,216][105692] Updated weights for policy 0, policy_version 903412 (0.0007) [2023-12-26 21:56:24,273][105692] Updated weights for policy 0, policy_version 903422 (0.0007) [2023-12-26 21:56:24,284][105620] Updated weights for policy 1, policy_version 903322 (0.0008) [2023-12-26 21:56:24,334][105620] Updated weights for policy 1, policy_version 903332 (0.0008) [2023-12-26 21:56:24,335][105692] Updated weights for policy 0, policy_version 903432 (0.0008) [2023-12-26 21:56:24,381][105620] Updated weights for policy 1, policy_version 903342 (0.0008) [2023-12-26 21:56:24,930][105692] Updated weights for policy 0, policy_version 903442 (0.0010) [2023-12-26 21:56:24,980][105692] Updated weights for policy 0, policy_version 903452 (0.0009) [2023-12-26 21:56:25,039][105692] Updated weights for policy 0, policy_version 903462 (0.0009) [2023-12-26 21:56:25,090][105692] Updated weights for policy 0, policy_version 903472 (0.0009) [2023-12-26 21:56:25,225][105620] Updated weights for policy 1, policy_version 903352 (0.0009) [2023-12-26 21:56:25,283][105620] Updated weights for policy 1, policy_version 903362 (0.0009) [2023-12-26 21:56:25,340][105620] Updated weights for policy 1, policy_version 903372 (0.0009) [2023-12-26 21:56:25,816][105692] Updated weights for policy 0, policy_version 903482 (0.0007) [2023-12-26 21:56:25,876][105692] Updated weights for policy 0, policy_version 903492 (0.0010) [2023-12-26 21:56:25,933][105692] Updated weights for policy 0, policy_version 903502 (0.0006) [2023-12-26 21:56:25,935][105620] Updated weights for policy 1, policy_version 903382 (0.0009) [2023-12-26 21:56:25,985][105620] Updated weights for policy 1, policy_version 903392 (0.0008) [2023-12-26 21:56:26,036][105620] Updated weights for policy 1, policy_version 903402 (0.0009) [2023-12-26 21:56:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19216.5). Total num frames: 462626816. Throughput: 0: 9604.1, 1: 9682.6. Samples: 462636120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:56:26,062][104569] Avg episode reward: [(0, '9089.877'), (1, '8372.569')] [2023-12-26 21:56:26,502][105692] Updated weights for policy 0, policy_version 903512 (0.0008) [2023-12-26 21:56:26,552][105692] Updated weights for policy 0, policy_version 903522 (0.0008) [2023-12-26 21:56:26,609][105692] Updated weights for policy 0, policy_version 903532 (0.0009) [2023-12-26 21:56:26,877][105620] Updated weights for policy 1, policy_version 903412 (0.0009) [2023-12-26 21:56:26,939][105620] Updated weights for policy 1, policy_version 903422 (0.0009) [2023-12-26 21:56:27,000][105620] Updated weights for policy 1, policy_version 903432 (0.0009) [2023-12-26 21:56:27,276][105692] Updated weights for policy 0, policy_version 903542 (0.0007) [2023-12-26 21:56:27,335][105692] Updated weights for policy 0, policy_version 903552 (0.0006) [2023-12-26 21:56:27,392][105692] Updated weights for policy 0, policy_version 903562 (0.0005) [2023-12-26 21:56:27,876][105620] Updated weights for policy 1, policy_version 903443 (0.0010) [2023-12-26 21:56:27,938][105692] Updated weights for policy 0, policy_version 903572 (0.0007) [2023-12-26 21:56:27,941][105620] Updated weights for policy 1, policy_version 903453 (0.0008) [2023-12-26 21:56:28,000][105620] Updated weights for policy 1, policy_version 903463 (0.0007) [2023-12-26 21:56:28,001][105692] Updated weights for policy 0, policy_version 903582 (0.0009) [2023-12-26 21:56:28,053][105692] Updated weights for policy 0, policy_version 903592 (0.0005) [2023-12-26 21:56:28,770][105620] Updated weights for policy 1, policy_version 903473 (0.0008) [2023-12-26 21:56:28,771][105692] Updated weights for policy 0, policy_version 903602 (0.0006) [2023-12-26 21:56:28,821][105692] Updated weights for policy 0, policy_version 903612 (0.0007) [2023-12-26 21:56:28,822][105620] Updated weights for policy 1, policy_version 903483 (0.0006) [2023-12-26 21:56:28,875][105620] Updated weights for policy 1, policy_version 903493 (0.0006) [2023-12-26 21:56:28,881][105692] Updated weights for policy 0, policy_version 903622 (0.0008) [2023-12-26 21:56:28,928][105692] Updated weights for policy 0, policy_version 903632 (0.0007) [2023-12-26 21:56:28,933][105620] Updated weights for policy 1, policy_version 903503 (0.0006) [2023-12-26 21:56:29,692][105620] Updated weights for policy 1, policy_version 903513 (0.0006) [2023-12-26 21:56:29,736][105692] Updated weights for policy 0, policy_version 903642 (0.0009) [2023-12-26 21:56:29,752][105620] Updated weights for policy 1, policy_version 903523 (0.0006) [2023-12-26 21:56:29,799][105692] Updated weights for policy 0, policy_version 903652 (0.0008) [2023-12-26 21:56:29,808][105620] Updated weights for policy 1, policy_version 903533 (0.0006) [2023-12-26 21:56:29,856][105692] Updated weights for policy 0, policy_version 903662 (0.0009) [2023-12-26 21:56:30,487][105620] Updated weights for policy 1, policy_version 903543 (0.0007) [2023-12-26 21:56:30,544][105620] Updated weights for policy 1, policy_version 903553 (0.0009) [2023-12-26 21:56:30,590][105692] Updated weights for policy 0, policy_version 903672 (0.0009) [2023-12-26 21:56:30,601][105620] Updated weights for policy 1, policy_version 903563 (0.0008) [2023-12-26 21:56:30,638][105692] Updated weights for policy 0, policy_version 903682 (0.0006) [2023-12-26 21:56:30,697][105692] Updated weights for policy 0, policy_version 903692 (0.0009) [2023-12-26 21:56:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19216.5). Total num frames: 462725120. Throughput: 0: 9749.7, 1: 9602.0. Samples: 462694836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:56:31,062][104569] Avg episode reward: [(0, '9170.423'), (1, '8274.438')] [2023-12-26 21:56:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000903696_231383040.pth... [2023-12-26 21:56:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000903568_231342080.pth... [2023-12-26 21:56:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000902448_231055360.pth [2023-12-26 21:56:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000902544_231088128.pth [2023-12-26 21:56:31,259][105620] Updated weights for policy 1, policy_version 903573 (0.0009) [2023-12-26 21:56:31,319][105620] Updated weights for policy 1, policy_version 903583 (0.0009) [2023-12-26 21:56:31,387][105620] Updated weights for policy 1, policy_version 903593 (0.0008) [2023-12-26 21:56:31,542][105692] Updated weights for policy 0, policy_version 903702 (0.0010) [2023-12-26 21:56:31,595][105692] Updated weights for policy 0, policy_version 903712 (0.0010) [2023-12-26 21:56:31,662][105692] Updated weights for policy 0, policy_version 903722 (0.0010) [2023-12-26 21:56:31,989][105620] Updated weights for policy 1, policy_version 903603 (0.0007) [2023-12-26 21:56:32,053][105620] Updated weights for policy 1, policy_version 903613 (0.0008) [2023-12-26 21:56:32,125][105620] Updated weights for policy 1, policy_version 903623 (0.0010) [2023-12-26 21:56:32,446][105692] Updated weights for policy 0, policy_version 903732 (0.0007) [2023-12-26 21:56:32,501][105692] Updated weights for policy 0, policy_version 903742 (0.0006) [2023-12-26 21:56:32,563][105692] Updated weights for policy 0, policy_version 903752 (0.0005) [2023-12-26 21:56:32,891][105620] Updated weights for policy 1, policy_version 903633 (0.0009) [2023-12-26 21:56:32,957][105620] Updated weights for policy 1, policy_version 903643 (0.0005) [2023-12-26 21:56:33,017][105620] Updated weights for policy 1, policy_version 903653 (0.0008) [2023-12-26 21:56:33,082][105620] Updated weights for policy 1, policy_version 903663 (0.0006) [2023-12-26 21:56:33,124][105692] Updated weights for policy 0, policy_version 903762 (0.0006) [2023-12-26 21:56:33,186][105692] Updated weights for policy 0, policy_version 903772 (0.0009) [2023-12-26 21:56:33,242][105692] Updated weights for policy 0, policy_version 903782 (0.0008) [2023-12-26 21:56:33,301][105692] Updated weights for policy 0, policy_version 903792 (0.0006) [2023-12-26 21:56:33,759][105620] Updated weights for policy 1, policy_version 903673 (0.0007) [2023-12-26 21:56:33,813][105620] Updated weights for policy 1, policy_version 903683 (0.0008) [2023-12-26 21:56:33,863][105620] Updated weights for policy 1, policy_version 903693 (0.0008) [2023-12-26 21:56:34,028][105692] Updated weights for policy 0, policy_version 903802 (0.0010) [2023-12-26 21:56:34,076][105692] Updated weights for policy 0, policy_version 903812 (0.0010) [2023-12-26 21:56:34,131][105692] Updated weights for policy 0, policy_version 903822 (0.0011) [2023-12-26 21:56:34,635][105620] Updated weights for policy 1, policy_version 903703 (0.0008) [2023-12-26 21:56:34,702][105620] Updated weights for policy 1, policy_version 903713 (0.0008) [2023-12-26 21:56:34,761][105620] Updated weights for policy 1, policy_version 903723 (0.0008) [2023-12-26 21:56:34,913][105692] Updated weights for policy 0, policy_version 903832 (0.0006) [2023-12-26 21:56:34,980][105692] Updated weights for policy 0, policy_version 903842 (0.0007) [2023-12-26 21:56:35,044][105692] Updated weights for policy 0, policy_version 903852 (0.0008) [2023-12-26 21:56:35,534][105620] Updated weights for policy 1, policy_version 903733 (0.0007) [2023-12-26 21:56:35,600][105620] Updated weights for policy 1, policy_version 903743 (0.0005) [2023-12-26 21:56:35,658][105620] Updated weights for policy 1, policy_version 903753 (0.0006) [2023-12-26 21:56:35,706][105692] Updated weights for policy 0, policy_version 903862 (0.0008) [2023-12-26 21:56:35,775][105692] Updated weights for policy 0, policy_version 903872 (0.0008) [2023-12-26 21:56:35,835][105692] Updated weights for policy 0, policy_version 903882 (0.0007) [2023-12-26 21:56:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19244.3). Total num frames: 462823424. Throughput: 0: 9765.0, 1: 9635.0. Samples: 462810004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 21:56:36,062][104569] Avg episode reward: [(0, '8991.952'), (1, '8291.559')] [2023-12-26 21:56:36,263][105620] Updated weights for policy 1, policy_version 903763 (0.0005) [2023-12-26 21:56:36,316][105620] Updated weights for policy 1, policy_version 903773 (0.0005) [2023-12-26 21:56:36,385][105620] Updated weights for policy 1, policy_version 903783 (0.0008) [2023-12-26 21:56:36,530][105692] Updated weights for policy 0, policy_version 903892 (0.0009) [2023-12-26 21:56:36,588][105692] Updated weights for policy 0, policy_version 903902 (0.0009) [2023-12-26 21:56:36,643][105692] Updated weights for policy 0, policy_version 903912 (0.0008) [2023-12-26 21:56:37,071][105620] Updated weights for policy 1, policy_version 903793 (0.0006) [2023-12-26 21:56:37,138][105620] Updated weights for policy 1, policy_version 903803 (0.0005) [2023-12-26 21:56:37,203][105620] Updated weights for policy 1, policy_version 903813 (0.0008) [2023-12-26 21:56:37,211][105586] KL-divergence is very high: 126.9574 [2023-12-26 21:56:37,263][105586] KL-divergence is very high: 133.4242 [2023-12-26 21:56:37,270][105620] Updated weights for policy 1, policy_version 903823 (0.0007) [2023-12-26 21:56:37,352][105692] Updated weights for policy 0, policy_version 903922 (0.0009) [2023-12-26 21:56:37,407][105692] Updated weights for policy 0, policy_version 903932 (0.0009) [2023-12-26 21:56:37,469][105692] Updated weights for policy 0, policy_version 903942 (0.0009) [2023-12-26 21:56:37,525][105692] Updated weights for policy 0, policy_version 903952 (0.0009) [2023-12-26 21:56:37,861][105620] Updated weights for policy 1, policy_version 903833 (0.0009) [2023-12-26 21:56:37,910][105620] Updated weights for policy 1, policy_version 903843 (0.0010) [2023-12-26 21:56:37,961][105620] Updated weights for policy 1, policy_version 903853 (0.0010) [2023-12-26 21:56:38,380][105692] Updated weights for policy 0, policy_version 903962 (0.0009) [2023-12-26 21:56:38,449][105692] Updated weights for policy 0, policy_version 903972 (0.0009) [2023-12-26 21:56:38,516][105692] Updated weights for policy 0, policy_version 903982 (0.0010) [2023-12-26 21:56:38,668][105620] Updated weights for policy 1, policy_version 903863 (0.0007) [2023-12-26 21:56:38,728][105620] Updated weights for policy 1, policy_version 903873 (0.0009) [2023-12-26 21:56:38,779][105620] Updated weights for policy 1, policy_version 903883 (0.0006) [2023-12-26 21:56:39,330][105692] Updated weights for policy 0, policy_version 903992 (0.0010) [2023-12-26 21:56:39,395][105692] Updated weights for policy 0, policy_version 904002 (0.0008) [2023-12-26 21:56:39,430][105620] Updated weights for policy 1, policy_version 903893 (0.0006) [2023-12-26 21:56:39,458][105692] Updated weights for policy 0, policy_version 904012 (0.0008) [2023-12-26 21:56:39,489][105620] Updated weights for policy 1, policy_version 903903 (0.0008) [2023-12-26 21:56:39,548][105620] Updated weights for policy 1, policy_version 903913 (0.0009) [2023-12-26 21:56:40,228][105620] Updated weights for policy 1, policy_version 903923 (0.0009) [2023-12-26 21:56:40,287][105620] Updated weights for policy 1, policy_version 903933 (0.0008) [2023-12-26 21:56:40,293][105692] Updated weights for policy 0, policy_version 904022 (0.0009) [2023-12-26 21:56:40,350][105620] Updated weights for policy 1, policy_version 903943 (0.0007) [2023-12-26 21:56:40,356][105692] Updated weights for policy 0, policy_version 904032 (0.0007) [2023-12-26 21:56:40,418][105692] Updated weights for policy 0, policy_version 904042 (0.0007) [2023-12-26 21:56:41,042][105620] Updated weights for policy 1, policy_version 903953 (0.0008) [2023-12-26 21:56:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19244.3). Total num frames: 462913536. Throughput: 0: 9700.2, 1: 9731.9. Samples: 462925936. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 21:56:41,062][104569] Avg episode reward: [(0, '9080.754'), (1, '8205.806')] [2023-12-26 21:56:41,106][105620] Updated weights for policy 1, policy_version 903963 (0.0009) [2023-12-26 21:56:41,170][105620] Updated weights for policy 1, policy_version 903973 (0.0009) [2023-12-26 21:56:41,182][105692] Updated weights for policy 0, policy_version 904052 (0.0010) [2023-12-26 21:56:41,226][105620] Updated weights for policy 1, policy_version 903983 (0.0007) [2023-12-26 21:56:41,238][105692] Updated weights for policy 0, policy_version 904062 (0.0008) [2023-12-26 21:56:41,297][105692] Updated weights for policy 0, policy_version 904072 (0.0008) [2023-12-26 21:56:42,021][105620] Updated weights for policy 1, policy_version 903993 (0.0005) [2023-12-26 21:56:42,083][105620] Updated weights for policy 1, policy_version 904003 (0.0006) [2023-12-26 21:56:42,137][105620] Updated weights for policy 1, policy_version 904013 (0.0008) [2023-12-26 21:56:42,140][105692] Updated weights for policy 0, policy_version 904082 (0.0008) [2023-12-26 21:56:42,197][105692] Updated weights for policy 0, policy_version 904092 (0.0009) [2023-12-26 21:56:42,251][105692] Updated weights for policy 0, policy_version 904102 (0.0009) [2023-12-26 21:56:42,312][105692] Updated weights for policy 0, policy_version 904112 (0.0009) [2023-12-26 21:56:42,773][105620] Updated weights for policy 1, policy_version 904023 (0.0008) [2023-12-26 21:56:42,835][105620] Updated weights for policy 1, policy_version 904033 (0.0009) [2023-12-26 21:56:42,896][105620] Updated weights for policy 1, policy_version 904043 (0.0008) [2023-12-26 21:56:43,123][105692] Updated weights for policy 0, policy_version 904122 (0.0009) [2023-12-26 21:56:43,174][105692] Updated weights for policy 0, policy_version 904132 (0.0009) [2023-12-26 21:56:43,236][105692] Updated weights for policy 0, policy_version 904142 (0.0009) [2023-12-26 21:56:43,507][105620] Updated weights for policy 1, policy_version 904053 (0.0008) [2023-12-26 21:56:43,564][105620] Updated weights for policy 1, policy_version 904063 (0.0009) [2023-12-26 21:56:43,619][105620] Updated weights for policy 1, policy_version 904073 (0.0009) [2023-12-26 21:56:44,040][105692] Updated weights for policy 0, policy_version 904152 (0.0009) [2023-12-26 21:56:44,096][105692] Updated weights for policy 0, policy_version 904162 (0.0009) [2023-12-26 21:56:44,152][105692] Updated weights for policy 0, policy_version 904172 (0.0008) [2023-12-26 21:56:44,354][105620] Updated weights for policy 1, policy_version 904083 (0.0009) [2023-12-26 21:56:44,412][105620] Updated weights for policy 1, policy_version 904093 (0.0009) [2023-12-26 21:56:44,474][105620] Updated weights for policy 1, policy_version 904103 (0.0009) [2023-12-26 21:56:44,961][105692] Updated weights for policy 0, policy_version 904182 (0.0009) [2023-12-26 21:56:45,025][105692] Updated weights for policy 0, policy_version 904192 (0.0009) [2023-12-26 21:56:45,093][105692] Updated weights for policy 0, policy_version 904202 (0.0009) [2023-12-26 21:56:45,228][105620] Updated weights for policy 1, policy_version 904113 (0.0010) [2023-12-26 21:56:45,296][105620] Updated weights for policy 1, policy_version 904123 (0.0009) [2023-12-26 21:56:45,352][105620] Updated weights for policy 1, policy_version 904133 (0.0007) [2023-12-26 21:56:45,407][105620] Updated weights for policy 1, policy_version 904143 (0.0009) [2023-12-26 21:56:45,848][105692] Updated weights for policy 0, policy_version 904212 (0.0008) [2023-12-26 21:56:45,896][105692] Updated weights for policy 0, policy_version 904222 (0.0009) [2023-12-26 21:56:45,948][105692] Updated weights for policy 0, policy_version 904232 (0.0009) [2023-12-26 21:56:46,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19387.6, 300 sec: 19244.2). Total num frames: 463011840. Throughput: 0: 9546.5, 1: 9741.1. Samples: 462981880. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 21:56:46,063][104569] Avg episode reward: [(0, '9258.471'), (1, '8101.495')] [2023-12-26 21:56:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000904144_231489536.pth... [2023-12-26 21:56:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000904240_231522304.pth... [2023-12-26 21:56:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000903024_231202816.pth [2023-12-26 21:56:46,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000903120_231235584.pth [2023-12-26 21:56:46,146][105620] Updated weights for policy 1, policy_version 904153 (0.0009) [2023-12-26 21:56:46,193][105620] Updated weights for policy 1, policy_version 904163 (0.0009) [2023-12-26 21:56:46,239][105620] Updated weights for policy 1, policy_version 904173 (0.0008) [2023-12-26 21:56:46,726][105692] Updated weights for policy 0, policy_version 904242 (0.0009) [2023-12-26 21:56:46,785][105692] Updated weights for policy 0, policy_version 904252 (0.0010) [2023-12-26 21:56:46,843][105692] Updated weights for policy 0, policy_version 904262 (0.0010) [2023-12-26 21:56:46,895][105692] Updated weights for policy 0, policy_version 904272 (0.0010) [2023-12-26 21:56:47,001][105620] Updated weights for policy 1, policy_version 904183 (0.0009) [2023-12-26 21:56:47,061][105620] Updated weights for policy 1, policy_version 904193 (0.0008) [2023-12-26 21:56:47,126][105620] Updated weights for policy 1, policy_version 904203 (0.0009) [2023-12-26 21:56:47,600][105692] Updated weights for policy 0, policy_version 904282 (0.0005) [2023-12-26 21:56:47,661][105692] Updated weights for policy 0, policy_version 904292 (0.0009) [2023-12-26 21:56:47,715][105692] Updated weights for policy 0, policy_version 904302 (0.0010) [2023-12-26 21:56:47,866][105620] Updated weights for policy 1, policy_version 904213 (0.0007) [2023-12-26 21:56:47,936][105620] Updated weights for policy 1, policy_version 904223 (0.0008) [2023-12-26 21:56:47,999][105620] Updated weights for policy 1, policy_version 904233 (0.0008) [2023-12-26 21:56:48,434][105692] Updated weights for policy 0, policy_version 904312 (0.0008) [2023-12-26 21:56:48,493][105692] Updated weights for policy 0, policy_version 904322 (0.0007) [2023-12-26 21:56:48,558][105692] Updated weights for policy 0, policy_version 904332 (0.0005) [2023-12-26 21:56:48,664][105620] Updated weights for policy 1, policy_version 904243 (0.0007) [2023-12-26 21:56:48,712][105620] Updated weights for policy 1, policy_version 904253 (0.0006) [2023-12-26 21:56:48,760][105620] Updated weights for policy 1, policy_version 904263 (0.0005) [2023-12-26 21:56:49,218][105692] Updated weights for policy 0, policy_version 904342 (0.0008) [2023-12-26 21:56:49,282][105692] Updated weights for policy 0, policy_version 904352 (0.0008) [2023-12-26 21:56:49,335][105692] Updated weights for policy 0, policy_version 904362 (0.0009) [2023-12-26 21:56:49,427][105620] Updated weights for policy 1, policy_version 904273 (0.0006) [2023-12-26 21:56:49,498][105620] Updated weights for policy 1, policy_version 904283 (0.0009) [2023-12-26 21:56:49,556][105620] Updated weights for policy 1, policy_version 904293 (0.0009) [2023-12-26 21:56:49,617][105620] Updated weights for policy 1, policy_version 904303 (0.0009) [2023-12-26 21:56:50,024][105692] Updated weights for policy 0, policy_version 904372 (0.0007) [2023-12-26 21:56:50,085][105692] Updated weights for policy 0, policy_version 904382 (0.0008) [2023-12-26 21:56:50,152][105692] Updated weights for policy 0, policy_version 904392 (0.0006) [2023-12-26 21:56:50,479][105620] Updated weights for policy 1, policy_version 904313 (0.0008) [2023-12-26 21:56:50,531][105620] Updated weights for policy 1, policy_version 904323 (0.0009) [2023-12-26 21:56:50,590][105620] Updated weights for policy 1, policy_version 904333 (0.0008) [2023-12-26 21:56:50,791][105692] Updated weights for policy 0, policy_version 904402 (0.0006) [2023-12-26 21:56:50,838][105692] Updated weights for policy 0, policy_version 904412 (0.0008) [2023-12-26 21:56:50,900][105692] Updated weights for policy 0, policy_version 904422 (0.0009) [2023-12-26 21:56:50,952][105692] Updated weights for policy 0, policy_version 904432 (0.0009) [2023-12-26 21:56:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19244.3). Total num frames: 463110144. Throughput: 0: 9499.6, 1: 9706.2. Samples: 463096216. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 21:56:51,062][104569] Avg episode reward: [(0, '8917.193'), (1, '8366.318')] [2023-12-26 21:56:51,316][105620] Updated weights for policy 1, policy_version 904343 (0.0007) [2023-12-26 21:56:51,389][105620] Updated weights for policy 1, policy_version 904353 (0.0008) [2023-12-26 21:56:51,444][105620] Updated weights for policy 1, policy_version 904363 (0.0008) [2023-12-26 21:56:51,798][105692] Updated weights for policy 0, policy_version 904442 (0.0009) [2023-12-26 21:56:51,847][105692] Updated weights for policy 0, policy_version 904452 (0.0007) [2023-12-26 21:56:51,892][105692] Updated weights for policy 0, policy_version 904462 (0.0005) [2023-12-26 21:56:52,165][105620] Updated weights for policy 1, policy_version 904373 (0.0008) [2023-12-26 21:56:52,221][105620] Updated weights for policy 1, policy_version 904383 (0.0009) [2023-12-26 21:56:52,250][105586] KL-divergence is very high: 172.7835 [2023-12-26 21:56:52,293][105620] Updated weights for policy 1, policy_version 904393 (0.0008) [2023-12-26 21:56:52,303][105586] KL-divergence is very high: 209.3208 [2023-12-26 21:56:52,682][105692] Updated weights for policy 0, policy_version 904472 (0.0005) [2023-12-26 21:56:52,750][105692] Updated weights for policy 0, policy_version 904482 (0.0006) [2023-12-26 21:56:52,821][105692] Updated weights for policy 0, policy_version 904492 (0.0006) [2023-12-26 21:56:52,930][105620] Updated weights for policy 1, policy_version 904403 (0.0010) [2023-12-26 21:56:52,980][105620] Updated weights for policy 1, policy_version 904413 (0.0010) [2023-12-26 21:56:53,025][105620] Updated weights for policy 1, policy_version 904423 (0.0010) [2023-12-26 21:56:53,416][105692] Updated weights for policy 0, policy_version 904502 (0.0008) [2023-12-26 21:56:53,477][105692] Updated weights for policy 0, policy_version 904512 (0.0009) [2023-12-26 21:56:53,546][105692] Updated weights for policy 0, policy_version 904522 (0.0011) [2023-12-26 21:56:53,758][105620] Updated weights for policy 1, policy_version 904433 (0.0010) [2023-12-26 21:56:53,826][105620] Updated weights for policy 1, policy_version 904443 (0.0010) [2023-12-26 21:56:53,880][105620] Updated weights for policy 1, policy_version 904454 (0.0010) [2023-12-26 21:56:53,933][105620] Updated weights for policy 1, policy_version 904464 (0.0010) [2023-12-26 21:56:54,111][105692] Updated weights for policy 0, policy_version 904532 (0.0009) [2023-12-26 21:56:54,170][105692] Updated weights for policy 0, policy_version 904542 (0.0010) [2023-12-26 21:56:54,228][105692] Updated weights for policy 0, policy_version 904552 (0.0010) [2023-12-26 21:56:54,682][105620] Updated weights for policy 1, policy_version 904474 (0.0010) [2023-12-26 21:56:54,727][105620] Updated weights for policy 1, policy_version 904484 (0.0010) [2023-12-26 21:56:54,779][105620] Updated weights for policy 1, policy_version 904494 (0.0010) [2023-12-26 21:56:54,936][105692] Updated weights for policy 0, policy_version 904562 (0.0010) [2023-12-26 21:56:54,998][105692] Updated weights for policy 0, policy_version 904572 (0.0010) [2023-12-26 21:56:55,058][105692] Updated weights for policy 0, policy_version 904582 (0.0011) [2023-12-26 21:56:55,123][105692] Updated weights for policy 0, policy_version 904592 (0.0010) [2023-12-26 21:56:55,414][105620] Updated weights for policy 1, policy_version 904504 (0.0006) [2023-12-26 21:56:55,466][105620] Updated weights for policy 1, policy_version 904514 (0.0005) [2023-12-26 21:56:55,518][105620] Updated weights for policy 1, policy_version 904524 (0.0005) [2023-12-26 21:56:55,807][105692] Updated weights for policy 0, policy_version 904602 (0.0010) [2023-12-26 21:56:55,858][105692] Updated weights for policy 0, policy_version 904612 (0.0010) [2023-12-26 21:56:55,945][105692] Updated weights for policy 0, policy_version 904622 (0.0010) [2023-12-26 21:56:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.7, 300 sec: 19272.0). Total num frames: 463208448. Throughput: 0: 9505.0, 1: 9732.6. Samples: 463216188. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 21:56:56,062][104569] Avg episode reward: [(0, '8909.615'), (1, '8634.110')] [2023-12-26 21:56:56,126][105620] Updated weights for policy 1, policy_version 904534 (0.0008) [2023-12-26 21:56:56,183][105620] Updated weights for policy 1, policy_version 904544 (0.0008) [2023-12-26 21:56:56,234][105620] Updated weights for policy 1, policy_version 904554 (0.0010) [2023-12-26 21:56:56,645][105692] Updated weights for policy 0, policy_version 904632 (0.0010) [2023-12-26 21:56:56,692][105692] Updated weights for policy 0, policy_version 904642 (0.0010) [2023-12-26 21:56:56,736][105692] Updated weights for policy 0, policy_version 904652 (0.0010) [2023-12-26 21:56:56,894][105620] Updated weights for policy 1, policy_version 904564 (0.0008) [2023-12-26 21:56:56,939][105620] Updated weights for policy 1, policy_version 904574 (0.0005) [2023-12-26 21:56:56,985][105620] Updated weights for policy 1, policy_version 904584 (0.0005) [2023-12-26 21:56:57,484][105692] Updated weights for policy 0, policy_version 904662 (0.0010) [2023-12-26 21:56:57,536][105692] Updated weights for policy 0, policy_version 904672 (0.0010) [2023-12-26 21:56:57,584][105692] Updated weights for policy 0, policy_version 904682 (0.0010) [2023-12-26 21:56:57,661][105620] Updated weights for policy 1, policy_version 904594 (0.0006) [2023-12-26 21:56:57,715][105620] Updated weights for policy 1, policy_version 904605 (0.0010) [2023-12-26 21:56:57,768][105620] Updated weights for policy 1, policy_version 904616 (0.0009) [2023-12-26 21:56:58,160][105692] Updated weights for policy 0, policy_version 904692 (0.0009) [2023-12-26 21:56:58,219][105692] Updated weights for policy 0, policy_version 904702 (0.0008) [2023-12-26 21:56:58,282][105692] Updated weights for policy 0, policy_version 904712 (0.0009) [2023-12-26 21:56:58,582][105620] Updated weights for policy 1, policy_version 904627 (0.0009) [2023-12-26 21:56:58,645][105620] Updated weights for policy 1, policy_version 904637 (0.0008) [2023-12-26 21:56:58,710][105620] Updated weights for policy 1, policy_version 904647 (0.0007) [2023-12-26 21:56:59,033][105692] Updated weights for policy 0, policy_version 904722 (0.0010) [2023-12-26 21:56:59,088][105692] Updated weights for policy 0, policy_version 904732 (0.0005) [2023-12-26 21:56:59,143][105692] Updated weights for policy 0, policy_version 904742 (0.0005) [2023-12-26 21:56:59,193][105692] Updated weights for policy 0, policy_version 904752 (0.0009) [2023-12-26 21:56:59,416][105620] Updated weights for policy 1, policy_version 904657 (0.0006) [2023-12-26 21:56:59,468][105620] Updated weights for policy 1, policy_version 904667 (0.0005) [2023-12-26 21:56:59,514][105620] Updated weights for policy 1, policy_version 904677 (0.0005) [2023-12-26 21:56:59,566][105620] Updated weights for policy 1, policy_version 904687 (0.0007) [2023-12-26 21:56:59,846][105692] Updated weights for policy 0, policy_version 904762 (0.0009) [2023-12-26 21:56:59,907][105692] Updated weights for policy 0, policy_version 904772 (0.0010) [2023-12-26 21:56:59,967][105692] Updated weights for policy 0, policy_version 904782 (0.0007) [2023-12-26 21:57:00,259][105620] Updated weights for policy 1, policy_version 904697 (0.0009) [2023-12-26 21:57:00,317][105620] Updated weights for policy 1, policy_version 904707 (0.0005) [2023-12-26 21:57:00,376][105620] Updated weights for policy 1, policy_version 904717 (0.0007) [2023-12-26 21:57:00,697][105692] Updated weights for policy 0, policy_version 904792 (0.0009) [2023-12-26 21:57:00,747][105692] Updated weights for policy 0, policy_version 904802 (0.0010) [2023-12-26 21:57:00,794][105692] Updated weights for policy 0, policy_version 904812 (0.0010) [2023-12-26 21:57:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19272.0). Total num frames: 463306752. Throughput: 0: 9549.1, 1: 9733.8. Samples: 463276288. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 21:57:01,063][104569] Avg episode reward: [(0, '9170.521'), (1, '8282.552')] [2023-12-26 21:57:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000904816_231669760.pth... [2023-12-26 21:57:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000903696_231383040.pth [2023-12-26 21:57:01,074][105620] Updated weights for policy 1, policy_version 904727 (0.0008) [2023-12-26 21:57:01,142][105620] Updated weights for policy 1, policy_version 904737 (0.0009) [2023-12-26 21:57:01,197][105620] Updated weights for policy 1, policy_version 904747 (0.0008) [2023-12-26 21:57:01,221][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000904752_231645184.pth... [2023-12-26 21:57:01,224][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000903568_231342080.pth [2023-12-26 21:57:01,554][105692] Updated weights for policy 0, policy_version 904822 (0.0010) [2023-12-26 21:57:01,603][105692] Updated weights for policy 0, policy_version 904832 (0.0010) [2023-12-26 21:57:01,666][105692] Updated weights for policy 0, policy_version 904842 (0.0010) [2023-12-26 21:57:01,955][105620] Updated weights for policy 1, policy_version 904757 (0.0008) [2023-12-26 21:57:02,023][105620] Updated weights for policy 1, policy_version 904767 (0.0007) [2023-12-26 21:57:02,089][105620] Updated weights for policy 1, policy_version 904777 (0.0008) [2023-12-26 21:57:02,339][105692] Updated weights for policy 0, policy_version 904852 (0.0010) [2023-12-26 21:57:02,400][105692] Updated weights for policy 0, policy_version 904862 (0.0008) [2023-12-26 21:57:02,447][105692] Updated weights for policy 0, policy_version 904872 (0.0008) [2023-12-26 21:57:02,831][105620] Updated weights for policy 1, policy_version 904787 (0.0007) [2023-12-26 21:57:02,882][105620] Updated weights for policy 1, policy_version 904797 (0.0009) [2023-12-26 21:57:02,936][105620] Updated weights for policy 1, policy_version 904807 (0.0009) [2023-12-26 21:57:03,112][105692] Updated weights for policy 0, policy_version 904882 (0.0007) [2023-12-26 21:57:03,173][105692] Updated weights for policy 0, policy_version 904892 (0.0008) [2023-12-26 21:57:03,224][105692] Updated weights for policy 0, policy_version 904902 (0.0008) [2023-12-26 21:57:03,269][105692] Updated weights for policy 0, policy_version 904912 (0.0008) [2023-12-26 21:57:03,689][105620] Updated weights for policy 1, policy_version 904817 (0.0010) [2023-12-26 21:57:03,734][105620] Updated weights for policy 1, policy_version 904827 (0.0010) [2023-12-26 21:57:03,785][105620] Updated weights for policy 1, policy_version 904837 (0.0009) [2023-12-26 21:57:03,832][105620] Updated weights for policy 1, policy_version 904847 (0.0008) [2023-12-26 21:57:04,015][105692] Updated weights for policy 0, policy_version 904922 (0.0008) [2023-12-26 21:57:04,079][105692] Updated weights for policy 0, policy_version 904932 (0.0008) [2023-12-26 21:57:04,143][105692] Updated weights for policy 0, policy_version 904942 (0.0008) [2023-12-26 21:57:04,599][105620] Updated weights for policy 1, policy_version 904857 (0.0010) [2023-12-26 21:57:04,657][105620] Updated weights for policy 1, policy_version 904867 (0.0010) [2023-12-26 21:57:04,716][105620] Updated weights for policy 1, policy_version 904877 (0.0010) [2023-12-26 21:57:04,734][105692] Updated weights for policy 0, policy_version 904952 (0.0005) [2023-12-26 21:57:04,791][105692] Updated weights for policy 0, policy_version 904962 (0.0009) [2023-12-26 21:57:04,848][105692] Updated weights for policy 0, policy_version 904972 (0.0010) [2023-12-26 21:57:05,265][105620] Updated weights for policy 1, policy_version 904887 (0.0009) [2023-12-26 21:57:05,314][105620] Updated weights for policy 1, policy_version 904897 (0.0006) [2023-12-26 21:57:05,362][105620] Updated weights for policy 1, policy_version 904907 (0.0005) [2023-12-26 21:57:05,710][105692] Updated weights for policy 0, policy_version 904982 (0.0010) [2023-12-26 21:57:05,772][105692] Updated weights for policy 0, policy_version 904992 (0.0010) [2023-12-26 21:57:05,828][105692] Updated weights for policy 0, policy_version 905002 (0.0014) [2023-12-26 21:57:05,898][105620] Updated weights for policy 1, policy_version 904917 (0.0005) [2023-12-26 21:57:05,956][105620] Updated weights for policy 1, policy_version 904927 (0.0005) [2023-12-26 21:57:06,007][105620] Updated weights for policy 1, policy_version 904937 (0.0005) [2023-12-26 21:57:06,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.3, 300 sec: 19299.8). Total num frames: 463413248. Throughput: 0: 9612.6, 1: 9745.2. Samples: 463393764. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 21:57:06,062][104569] Avg episode reward: [(0, '9171.813'), (1, '8015.337')] [2023-12-26 21:57:06,663][105620] Updated weights for policy 1, policy_version 904947 (0.0007) [2023-12-26 21:57:06,681][105692] Updated weights for policy 0, policy_version 905012 (0.0011) [2023-12-26 21:57:06,722][105620] Updated weights for policy 1, policy_version 904957 (0.0010) [2023-12-26 21:57:06,749][105692] Updated weights for policy 0, policy_version 905022 (0.0006) [2023-12-26 21:57:06,783][105620] Updated weights for policy 1, policy_version 904967 (0.0011) [2023-12-26 21:57:06,808][105692] Updated weights for policy 0, policy_version 905032 (0.0005) [2023-12-26 21:57:07,496][105692] Updated weights for policy 0, policy_version 905042 (0.0007) [2023-12-26 21:57:07,532][105620] Updated weights for policy 1, policy_version 904977 (0.0011) [2023-12-26 21:57:07,546][105692] Updated weights for policy 0, policy_version 905052 (0.0008) [2023-12-26 21:57:07,593][105620] Updated weights for policy 1, policy_version 904987 (0.0010) [2023-12-26 21:57:07,606][105692] Updated weights for policy 0, policy_version 905062 (0.0007) [2023-12-26 21:57:07,650][105620] Updated weights for policy 1, policy_version 904997 (0.0008) [2023-12-26 21:57:07,665][105692] Updated weights for policy 0, policy_version 905072 (0.0010) [2023-12-26 21:57:07,701][105620] Updated weights for policy 1, policy_version 905007 (0.0010) [2023-12-26 21:57:08,353][105620] Updated weights for policy 1, policy_version 905017 (0.0008) [2023-12-26 21:57:08,363][105692] Updated weights for policy 0, policy_version 905082 (0.0009) [2023-12-26 21:57:08,416][105692] Updated weights for policy 0, policy_version 905092 (0.0010) [2023-12-26 21:57:08,417][105620] Updated weights for policy 1, policy_version 905027 (0.0008) [2023-12-26 21:57:08,476][105692] Updated weights for policy 0, policy_version 905102 (0.0011) [2023-12-26 21:57:08,481][105620] Updated weights for policy 1, policy_version 905037 (0.0006) [2023-12-26 21:57:09,028][105620] Updated weights for policy 1, policy_version 905047 (0.0009) [2023-12-26 21:57:09,088][105620] Updated weights for policy 1, policy_version 905057 (0.0010) [2023-12-26 21:57:09,140][105620] Updated weights for policy 1, policy_version 905067 (0.0010) [2023-12-26 21:57:09,157][105692] Updated weights for policy 0, policy_version 905112 (0.0010) [2023-12-26 21:57:09,210][105692] Updated weights for policy 0, policy_version 905122 (0.0011) [2023-12-26 21:57:09,273][105692] Updated weights for policy 0, policy_version 905132 (0.0009) [2023-12-26 21:57:09,880][105620] Updated weights for policy 1, policy_version 905077 (0.0009) [2023-12-26 21:57:09,952][105620] Updated weights for policy 1, policy_version 905087 (0.0008) [2023-12-26 21:57:10,019][105620] Updated weights for policy 1, policy_version 905097 (0.0008) [2023-12-26 21:57:10,087][105692] Updated weights for policy 0, policy_version 905142 (0.0007) [2023-12-26 21:57:10,155][105692] Updated weights for policy 0, policy_version 905152 (0.0008) [2023-12-26 21:57:10,217][105692] Updated weights for policy 0, policy_version 905162 (0.0008) [2023-12-26 21:57:10,658][105620] Updated weights for policy 1, policy_version 905107 (0.0009) [2023-12-26 21:57:10,710][105620] Updated weights for policy 1, policy_version 905117 (0.0010) [2023-12-26 21:57:10,765][105620] Updated weights for policy 1, policy_version 905127 (0.0010) [2023-12-26 21:57:10,988][105692] Updated weights for policy 0, policy_version 905172 (0.0009) [2023-12-26 21:57:11,056][105692] Updated weights for policy 0, policy_version 905182 (0.0009) [2023-12-26 21:57:11,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.7, 300 sec: 19299.8). Total num frames: 463503360. Throughput: 0: 9615.7, 1: 9877.4. Samples: 463513308. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 21:57:11,062][104569] Avg episode reward: [(0, '9259.698'), (1, '7231.181')] [2023-12-26 21:57:11,109][105692] Updated weights for policy 0, policy_version 905192 (0.0008) [2023-12-26 21:57:11,545][105620] Updated weights for policy 1, policy_version 905137 (0.0010) [2023-12-26 21:57:11,607][105620] Updated weights for policy 1, policy_version 905147 (0.0011) [2023-12-26 21:57:11,679][105620] Updated weights for policy 1, policy_version 905157 (0.0009) [2023-12-26 21:57:11,773][105620] Updated weights for policy 1, policy_version 905167 (0.0010) [2023-12-26 21:57:11,882][105692] Updated weights for policy 0, policy_version 905202 (0.0008) [2023-12-26 21:57:11,931][105692] Updated weights for policy 0, policy_version 905212 (0.0008) [2023-12-26 21:57:11,992][105692] Updated weights for policy 0, policy_version 905222 (0.0008) [2023-12-26 21:57:12,052][105692] Updated weights for policy 0, policy_version 905232 (0.0008) [2023-12-26 21:57:12,527][105620] Updated weights for policy 1, policy_version 905177 (0.0010) [2023-12-26 21:57:12,548][105586] KL-divergence is very high: 123.8310 [2023-12-26 21:57:12,590][105620] Updated weights for policy 1, policy_version 905187 (0.0011) [2023-12-26 21:57:12,597][105586] KL-divergence is very high: 122.8079 [2023-12-26 21:57:12,648][105620] Updated weights for policy 1, policy_version 905197 (0.0010) [2023-12-26 21:57:12,844][105692] Updated weights for policy 0, policy_version 905242 (0.0009) [2023-12-26 21:57:12,904][105692] Updated weights for policy 0, policy_version 905252 (0.0008) [2023-12-26 21:57:12,966][105692] Updated weights for policy 0, policy_version 905262 (0.0008) [2023-12-26 21:57:13,309][105620] Updated weights for policy 1, policy_version 905207 (0.0011) [2023-12-26 21:57:13,371][105620] Updated weights for policy 1, policy_version 905217 (0.0011) [2023-12-26 21:57:13,437][105620] Updated weights for policy 1, policy_version 905227 (0.0010) [2023-12-26 21:57:13,775][105692] Updated weights for policy 0, policy_version 905272 (0.0008) [2023-12-26 21:57:13,823][105692] Updated weights for policy 0, policy_version 905282 (0.0008) [2023-12-26 21:57:13,870][105692] Updated weights for policy 0, policy_version 905292 (0.0008) [2023-12-26 21:57:14,126][105620] Updated weights for policy 1, policy_version 905237 (0.0011) [2023-12-26 21:57:14,192][105620] Updated weights for policy 1, policy_version 905247 (0.0010) [2023-12-26 21:57:14,260][105620] Updated weights for policy 1, policy_version 905257 (0.0010) [2023-12-26 21:57:14,678][105692] Updated weights for policy 0, policy_version 905302 (0.0009) [2023-12-26 21:57:14,732][105692] Updated weights for policy 0, policy_version 905312 (0.0010) [2023-12-26 21:57:14,799][105692] Updated weights for policy 0, policy_version 905322 (0.0007) [2023-12-26 21:57:14,870][105620] Updated weights for policy 1, policy_version 905267 (0.0011) [2023-12-26 21:57:14,918][105620] Updated weights for policy 1, policy_version 905277 (0.0010) [2023-12-26 21:57:14,948][105586] KL-divergence is very high: 109.6707 [2023-12-26 21:57:14,981][105620] Updated weights for policy 1, policy_version 905287 (0.0011) [2023-12-26 21:57:14,996][105586] KL-divergence is very high: 113.8991 [2023-12-26 21:57:15,654][105692] Updated weights for policy 0, policy_version 905332 (0.0009) [2023-12-26 21:57:15,663][105620] Updated weights for policy 1, policy_version 905297 (0.0011) [2023-12-26 21:57:15,713][105692] Updated weights for policy 0, policy_version 905342 (0.0006) [2023-12-26 21:57:15,715][105620] Updated weights for policy 1, policy_version 905307 (0.0010) [2023-12-26 21:57:15,764][105692] Updated weights for policy 0, policy_version 905352 (0.0006) [2023-12-26 21:57:15,766][105620] Updated weights for policy 1, policy_version 905317 (0.0010) [2023-12-26 21:57:15,817][105620] Updated weights for policy 1, policy_version 905327 (0.0010) [2023-12-26 21:57:16,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19387.7, 300 sec: 19299.8). Total num frames: 463601664. Throughput: 0: 9479.1, 1: 9945.3. Samples: 463568936. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 21:57:16,063][104569] Avg episode reward: [(0, '8783.640'), (1, '7240.277')] [2023-12-26 21:57:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000905360_231809024.pth... [2023-12-26 21:57:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000905328_231792640.pth... [2023-12-26 21:57:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000904144_231489536.pth [2023-12-26 21:57:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000904240_231522304.pth [2023-12-26 21:57:16,527][105692] Updated weights for policy 0, policy_version 905362 (0.0006) [2023-12-26 21:57:16,571][105620] Updated weights for policy 1, policy_version 905337 (0.0010) [2023-12-26 21:57:16,586][105692] Updated weights for policy 0, policy_version 905372 (0.0007) [2023-12-26 21:57:16,621][105620] Updated weights for policy 1, policy_version 905347 (0.0005) [2023-12-26 21:57:16,651][105692] Updated weights for policy 0, policy_version 905382 (0.0009) [2023-12-26 21:57:16,674][105620] Updated weights for policy 1, policy_version 905357 (0.0005) [2023-12-26 21:57:16,714][105692] Updated weights for policy 0, policy_version 905392 (0.0010) [2023-12-26 21:57:17,250][105620] Updated weights for policy 1, policy_version 905367 (0.0009) [2023-12-26 21:57:17,301][105620] Updated weights for policy 1, policy_version 905377 (0.0010) [2023-12-26 21:57:17,348][105620] Updated weights for policy 1, policy_version 905387 (0.0010) [2023-12-26 21:57:17,528][105692] Updated weights for policy 0, policy_version 905402 (0.0008) [2023-12-26 21:57:17,587][105692] Updated weights for policy 0, policy_version 905412 (0.0008) [2023-12-26 21:57:17,645][105692] Updated weights for policy 0, policy_version 905422 (0.0008) [2023-12-26 21:57:18,111][105620] Updated weights for policy 1, policy_version 905397 (0.0010) [2023-12-26 21:57:18,179][105620] Updated weights for policy 1, policy_version 905407 (0.0010) [2023-12-26 21:57:18,247][105620] Updated weights for policy 1, policy_version 905417 (0.0009) [2023-12-26 21:57:18,407][105692] Updated weights for policy 0, policy_version 905432 (0.0008) [2023-12-26 21:57:18,464][105692] Updated weights for policy 0, policy_version 905442 (0.0008) [2023-12-26 21:57:18,519][105692] Updated weights for policy 0, policy_version 905452 (0.0008) [2023-12-26 21:57:19,062][105620] Updated weights for policy 1, policy_version 905427 (0.0010) [2023-12-26 21:57:19,114][105620] Updated weights for policy 1, policy_version 905437 (0.0010) [2023-12-26 21:57:19,138][105586] KL-divergence is very high: 145.9599 [2023-12-26 21:57:19,174][105620] Updated weights for policy 1, policy_version 905447 (0.0005) [2023-12-26 21:57:19,186][105586] KL-divergence is very high: 144.8834 [2023-12-26 21:57:19,305][105692] Updated weights for policy 0, policy_version 905462 (0.0009) [2023-12-26 21:57:19,364][105692] Updated weights for policy 0, policy_version 905472 (0.0008) [2023-12-26 21:57:19,426][105692] Updated weights for policy 0, policy_version 905482 (0.0010) [2023-12-26 21:57:19,883][105620] Updated weights for policy 1, policy_version 905457 (0.0007) [2023-12-26 21:57:19,949][105620] Updated weights for policy 1, policy_version 905467 (0.0009) [2023-12-26 21:57:20,002][105620] Updated weights for policy 1, policy_version 905477 (0.0009) [2023-12-26 21:57:20,065][105620] Updated weights for policy 1, policy_version 905487 (0.0009) [2023-12-26 21:57:20,164][105692] Updated weights for policy 0, policy_version 905492 (0.0009) [2023-12-26 21:57:20,226][105692] Updated weights for policy 0, policy_version 905502 (0.0009) [2023-12-26 21:57:20,289][105692] Updated weights for policy 0, policy_version 905512 (0.0008) [2023-12-26 21:57:20,845][105620] Updated weights for policy 1, policy_version 905497 (0.0010) [2023-12-26 21:57:20,913][105620] Updated weights for policy 1, policy_version 905507 (0.0009) [2023-12-26 21:57:20,983][105620] Updated weights for policy 1, policy_version 905517 (0.0009) [2023-12-26 21:57:21,039][105692] Updated weights for policy 0, policy_version 905522 (0.0009) [2023-12-26 21:57:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19299.8). Total num frames: 463691776. Throughput: 0: 9401.7, 1: 9968.3. Samples: 463681652. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 21:57:21,062][104569] Avg episode reward: [(0, '8609.866'), (1, '7857.920')] [2023-12-26 21:57:21,097][105692] Updated weights for policy 0, policy_version 905532 (0.0009) [2023-12-26 21:57:21,161][105692] Updated weights for policy 0, policy_version 905542 (0.0008) [2023-12-26 21:57:21,226][105692] Updated weights for policy 0, policy_version 905552 (0.0009) [2023-12-26 21:57:21,775][105620] Updated weights for policy 1, policy_version 905527 (0.0010) [2023-12-26 21:57:21,823][105620] Updated weights for policy 1, policy_version 905537 (0.0008) [2023-12-26 21:57:21,876][105620] Updated weights for policy 1, policy_version 905547 (0.0009) [2023-12-26 21:57:21,950][105692] Updated weights for policy 0, policy_version 905562 (0.0009) [2023-12-26 21:57:22,010][105692] Updated weights for policy 0, policy_version 905572 (0.0009) [2023-12-26 21:57:22,067][105692] Updated weights for policy 0, policy_version 905582 (0.0008) [2023-12-26 21:57:22,695][105620] Updated weights for policy 1, policy_version 905557 (0.0009) [2023-12-26 21:57:22,749][105620] Updated weights for policy 1, policy_version 905567 (0.0010) [2023-12-26 21:57:22,752][105692] Updated weights for policy 0, policy_version 905592 (0.0008) [2023-12-26 21:57:22,754][105586] KL-divergence is very high: 104.4964 [2023-12-26 21:57:22,804][105586] KL-divergence is very high: 181.9199 [2023-12-26 21:57:22,811][105620] Updated weights for policy 1, policy_version 905577 (0.0010) [2023-12-26 21:57:22,818][105692] Updated weights for policy 0, policy_version 905602 (0.0008) [2023-12-26 21:57:22,878][105692] Updated weights for policy 0, policy_version 905612 (0.0008) [2023-12-26 21:57:23,479][105692] Updated weights for policy 0, policy_version 905622 (0.0007) [2023-12-26 21:57:23,528][105692] Updated weights for policy 0, policy_version 905632 (0.0005) [2023-12-26 21:57:23,578][105692] Updated weights for policy 0, policy_version 905642 (0.0007) [2023-12-26 21:57:23,626][105620] Updated weights for policy 1, policy_version 905587 (0.0006) [2023-12-26 21:57:23,681][105620] Updated weights for policy 1, policy_version 905597 (0.0005) [2023-12-26 21:57:23,747][105620] Updated weights for policy 1, policy_version 905607 (0.0005) [2023-12-26 21:57:24,320][105692] Updated weights for policy 0, policy_version 905653 (0.0010) [2023-12-26 21:57:24,392][105692] Updated weights for policy 0, policy_version 905663 (0.0010) [2023-12-26 21:57:24,426][105620] Updated weights for policy 1, policy_version 905617 (0.0007) [2023-12-26 21:57:24,449][105692] Updated weights for policy 0, policy_version 905673 (0.0010) [2023-12-26 21:57:24,478][105620] Updated weights for policy 1, policy_version 905627 (0.0011) [2023-12-26 21:57:24,530][105620] Updated weights for policy 1, policy_version 905637 (0.0010) [2023-12-26 21:57:24,582][105620] Updated weights for policy 1, policy_version 905647 (0.0010) [2023-12-26 21:57:25,170][105692] Updated weights for policy 0, policy_version 905683 (0.0009) [2023-12-26 21:57:25,220][105692] Updated weights for policy 0, policy_version 905693 (0.0005) [2023-12-26 21:57:25,287][105692] Updated weights for policy 0, policy_version 905703 (0.0009) [2023-12-26 21:57:25,329][105620] Updated weights for policy 1, policy_version 905657 (0.0010) [2023-12-26 21:57:25,387][105620] Updated weights for policy 1, policy_version 905667 (0.0010) [2023-12-26 21:57:25,438][105620] Updated weights for policy 1, policy_version 905677 (0.0010) [2023-12-26 21:57:25,948][105692] Updated weights for policy 0, policy_version 905713 (0.0010) [2023-12-26 21:57:26,009][105692] Updated weights for policy 0, policy_version 905723 (0.0010) [2023-12-26 21:57:26,062][104569] Fps is (10 sec: 18022.7, 60 sec: 19251.2, 300 sec: 19272.0). Total num frames: 463781888. Throughput: 0: 9504.0, 1: 9825.1. Samples: 463795744. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 21:57:26,062][104569] Avg episode reward: [(0, '8799.176'), (1, '7433.451')] [2023-12-26 21:57:26,064][105692] Updated weights for policy 0, policy_version 905733 (0.0010) [2023-12-26 21:57:26,078][105620] Updated weights for policy 1, policy_version 905687 (0.0007) [2023-12-26 21:57:26,120][105692] Updated weights for policy 0, policy_version 905743 (0.0007) [2023-12-26 21:57:26,133][105620] Updated weights for policy 1, policy_version 905697 (0.0009) [2023-12-26 21:57:26,185][105620] Updated weights for policy 1, policy_version 905707 (0.0010) [2023-12-26 21:57:26,660][105692] Updated weights for policy 0, policy_version 905753 (0.0005) [2023-12-26 21:57:26,725][105692] Updated weights for policy 0, policy_version 905763 (0.0005) [2023-12-26 21:57:26,778][105692] Updated weights for policy 0, policy_version 905773 (0.0005) [2023-12-26 21:57:26,913][105620] Updated weights for policy 1, policy_version 905717 (0.0010) [2023-12-26 21:57:26,974][105620] Updated weights for policy 1, policy_version 905727 (0.0010) [2023-12-26 21:57:27,007][105586] KL-divergence is very high: 127.8854 [2023-12-26 21:57:27,025][105586] KL-divergence is very high: 117.0886 [2023-12-26 21:57:27,031][105620] Updated weights for policy 1, policy_version 905737 (0.0006) [2023-12-26 21:57:27,053][105586] KL-divergence is very high: 122.6516 [2023-12-26 21:57:27,371][105692] Updated weights for policy 0, policy_version 905783 (0.0006) [2023-12-26 21:57:27,436][105692] Updated weights for policy 0, policy_version 905793 (0.0005) [2023-12-26 21:57:27,495][105692] Updated weights for policy 0, policy_version 905803 (0.0005) [2023-12-26 21:57:27,560][105620] Updated weights for policy 1, policy_version 905747 (0.0005) [2023-12-26 21:57:27,610][105620] Updated weights for policy 1, policy_version 905757 (0.0005) [2023-12-26 21:57:27,675][105620] Updated weights for policy 1, policy_version 905767 (0.0005) [2023-12-26 21:57:28,040][105692] Updated weights for policy 0, policy_version 905813 (0.0008) [2023-12-26 21:57:28,088][105692] Updated weights for policy 0, policy_version 905823 (0.0006) [2023-12-26 21:57:28,145][105692] Updated weights for policy 0, policy_version 905833 (0.0010) [2023-12-26 21:57:28,267][105620] Updated weights for policy 1, policy_version 905777 (0.0006) [2023-12-26 21:57:28,325][105620] Updated weights for policy 1, policy_version 905787 (0.0010) [2023-12-26 21:57:28,392][105620] Updated weights for policy 1, policy_version 905797 (0.0005) [2023-12-26 21:57:28,459][105620] Updated weights for policy 1, policy_version 905807 (0.0007) [2023-12-26 21:57:28,902][105692] Updated weights for policy 0, policy_version 905843 (0.0009) [2023-12-26 21:57:28,961][105692] Updated weights for policy 0, policy_version 905853 (0.0008) [2023-12-26 21:57:29,021][105692] Updated weights for policy 0, policy_version 905863 (0.0006) [2023-12-26 21:57:29,159][105620] Updated weights for policy 1, policy_version 905817 (0.0010) [2023-12-26 21:57:29,228][105620] Updated weights for policy 1, policy_version 905827 (0.0010) [2023-12-26 21:57:29,293][105620] Updated weights for policy 1, policy_version 905837 (0.0011) [2023-12-26 21:57:29,754][105692] Updated weights for policy 0, policy_version 905873 (0.0007) [2023-12-26 21:57:29,829][105692] Updated weights for policy 0, policy_version 905883 (0.0009) [2023-12-26 21:57:29,892][105692] Updated weights for policy 0, policy_version 905893 (0.0010) [2023-12-26 21:57:29,956][105692] Updated weights for policy 0, policy_version 905903 (0.0008) [2023-12-26 21:57:30,008][105620] Updated weights for policy 1, policy_version 905847 (0.0008) [2023-12-26 21:57:30,061][105620] Updated weights for policy 1, policy_version 905857 (0.0008) [2023-12-26 21:57:30,115][105620] Updated weights for policy 1, policy_version 905867 (0.0009) [2023-12-26 21:57:30,529][105692] Updated weights for policy 0, policy_version 905913 (0.0009) [2023-12-26 21:57:30,583][105692] Updated weights for policy 0, policy_version 905923 (0.0005) [2023-12-26 21:57:30,636][105692] Updated weights for policy 0, policy_version 905933 (0.0005) [2023-12-26 21:57:31,002][105620] Updated weights for policy 1, policy_version 905878 (0.0009) [2023-12-26 21:57:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19299.8). Total num frames: 463888384. Throughput: 0: 9679.8, 1: 9876.4. Samples: 463861904. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 21:57:31,063][104569] Avg episode reward: [(0, '8894.028'), (1, '7762.480')] [2023-12-26 21:57:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000905936_231956480.pth... [2023-12-26 21:57:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000904816_231669760.pth [2023-12-26 21:57:31,074][105620] Updated weights for policy 1, policy_version 905888 (0.0010) [2023-12-26 21:57:31,140][105620] Updated weights for policy 1, policy_version 905898 (0.0010) [2023-12-26 21:57:31,180][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000905904_231940096.pth... [2023-12-26 21:57:31,185][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000904752_231645184.pth [2023-12-26 21:57:31,230][105692] Updated weights for policy 0, policy_version 905943 (0.0009) [2023-12-26 21:57:31,289][105692] Updated weights for policy 0, policy_version 905953 (0.0009) [2023-12-26 21:57:31,351][105692] Updated weights for policy 0, policy_version 905963 (0.0008) [2023-12-26 21:57:31,914][105620] Updated weights for policy 1, policy_version 905908 (0.0009) [2023-12-26 21:57:31,972][105620] Updated weights for policy 1, policy_version 905918 (0.0009) [2023-12-26 21:57:32,029][105692] Updated weights for policy 0, policy_version 905973 (0.0007) [2023-12-26 21:57:32,031][105620] Updated weights for policy 1, policy_version 905928 (0.0008) [2023-12-26 21:57:32,087][105692] Updated weights for policy 0, policy_version 905983 (0.0006) [2023-12-26 21:57:32,151][105692] Updated weights for policy 0, policy_version 905993 (0.0008) [2023-12-26 21:57:32,664][105620] Updated weights for policy 1, policy_version 905938 (0.0007) [2023-12-26 21:57:32,730][105620] Updated weights for policy 1, policy_version 905948 (0.0005) [2023-12-26 21:57:32,783][105620] Updated weights for policy 1, policy_version 905958 (0.0005) [2023-12-26 21:57:32,837][105620] Updated weights for policy 1, policy_version 905968 (0.0005) [2023-12-26 21:57:32,982][105692] Updated weights for policy 0, policy_version 906003 (0.0009) [2023-12-26 21:57:33,030][105692] Updated weights for policy 0, policy_version 906013 (0.0009) [2023-12-26 21:57:33,081][105692] Updated weights for policy 0, policy_version 906023 (0.0009) [2023-12-26 21:57:33,546][105620] Updated weights for policy 1, policy_version 905978 (0.0009) [2023-12-26 21:57:33,551][105586] KL-divergence is very high: 107.0148 [2023-12-26 21:57:33,556][105586] KL-divergence is very high: 127.9008 [2023-12-26 21:57:33,597][105586] KL-divergence is very high: 218.1366 [2023-12-26 21:57:33,603][105586] KL-divergence is very high: 231.4492 [2023-12-26 21:57:33,604][105620] Updated weights for policy 1, policy_version 905988 (0.0009) [2023-12-26 21:57:33,644][105586] KL-divergence is very high: 224.4571 [2023-12-26 21:57:33,649][105586] KL-divergence is very high: 232.1389 [2023-12-26 21:57:33,661][105620] Updated weights for policy 1, policy_version 905998 (0.0009) [2023-12-26 21:57:33,828][105692] Updated weights for policy 0, policy_version 906033 (0.0009) [2023-12-26 21:57:33,889][105692] Updated weights for policy 0, policy_version 906043 (0.0009) [2023-12-26 21:57:33,945][105692] Updated weights for policy 0, policy_version 906053 (0.0009) [2023-12-26 21:57:33,999][105692] Updated weights for policy 0, policy_version 906064 (0.0010) [2023-12-26 21:57:34,304][105586] KL-divergence is very high: 180.4254 [2023-12-26 21:57:34,349][105586] KL-divergence is very high: 120.6668 [2023-12-26 21:57:34,350][105620] Updated weights for policy 1, policy_version 906008 (0.0009) [2023-12-26 21:57:34,409][105620] Updated weights for policy 1, policy_version 906018 (0.0007) [2023-12-26 21:57:34,472][105620] Updated weights for policy 1, policy_version 906028 (0.0007) [2023-12-26 21:57:34,740][105692] Updated weights for policy 0, policy_version 906074 (0.0007) [2023-12-26 21:57:34,799][105692] Updated weights for policy 0, policy_version 906085 (0.0010) [2023-12-26 21:57:34,857][105692] Updated weights for policy 0, policy_version 906095 (0.0010) [2023-12-26 21:57:35,068][105620] Updated weights for policy 1, policy_version 906038 (0.0009) [2023-12-26 21:57:35,130][105620] Updated weights for policy 1, policy_version 906048 (0.0009) [2023-12-26 21:57:35,187][105620] Updated weights for policy 1, policy_version 906058 (0.0009) [2023-12-26 21:57:35,620][105692] Updated weights for policy 0, policy_version 906105 (0.0009) [2023-12-26 21:57:35,667][105692] Updated weights for policy 0, policy_version 906115 (0.0009) [2023-12-26 21:57:35,714][105692] Updated weights for policy 0, policy_version 906125 (0.0008) [2023-12-26 21:57:35,922][105620] Updated weights for policy 1, policy_version 906068 (0.0008) [2023-12-26 21:57:35,972][105620] Updated weights for policy 1, policy_version 906078 (0.0009) [2023-12-26 21:57:36,026][105620] Updated weights for policy 1, policy_version 906088 (0.0009) [2023-12-26 21:57:36,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19387.7, 300 sec: 19299.8). Total num frames: 463986688. Throughput: 0: 9724.2, 1: 9885.6. Samples: 463978656. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 21:57:36,063][104569] Avg episode reward: [(0, '8984.960'), (1, '7755.245')] [2023-12-26 21:57:36,447][105692] Updated weights for policy 0, policy_version 906135 (0.0009) [2023-12-26 21:57:36,507][105692] Updated weights for policy 0, policy_version 906145 (0.0009) [2023-12-26 21:57:36,565][105692] Updated weights for policy 0, policy_version 906155 (0.0009) [2023-12-26 21:57:36,835][105620] Updated weights for policy 1, policy_version 906098 (0.0008) [2023-12-26 21:57:36,882][105620] Updated weights for policy 1, policy_version 906108 (0.0009) [2023-12-26 21:57:36,929][105620] Updated weights for policy 1, policy_version 906118 (0.0009) [2023-12-26 21:57:36,979][105620] Updated weights for policy 1, policy_version 906128 (0.0009) [2023-12-26 21:57:37,319][105692] Updated weights for policy 0, policy_version 906165 (0.0009) [2023-12-26 21:57:37,367][105692] Updated weights for policy 0, policy_version 906176 (0.0009) [2023-12-26 21:57:37,421][105692] Updated weights for policy 0, policy_version 906186 (0.0009) [2023-12-26 21:57:37,701][105620] Updated weights for policy 1, policy_version 906138 (0.0009) [2023-12-26 21:57:37,763][105620] Updated weights for policy 1, policy_version 906148 (0.0006) [2023-12-26 21:57:37,832][105620] Updated weights for policy 1, policy_version 906158 (0.0005) [2023-12-26 21:57:38,139][105692] Updated weights for policy 0, policy_version 906196 (0.0008) [2023-12-26 21:57:38,185][105692] Updated weights for policy 0, policy_version 906206 (0.0005) [2023-12-26 21:57:38,238][105692] Updated weights for policy 0, policy_version 906216 (0.0005) [2023-12-26 21:57:38,439][105620] Updated weights for policy 1, policy_version 906168 (0.0007) [2023-12-26 21:57:38,507][105620] Updated weights for policy 1, policy_version 906178 (0.0010) [2023-12-26 21:57:38,572][105620] Updated weights for policy 1, policy_version 906188 (0.0010) [2023-12-26 21:57:38,985][105692] Updated weights for policy 0, policy_version 906226 (0.0009) [2023-12-26 21:57:39,043][105692] Updated weights for policy 0, policy_version 906236 (0.0008) [2023-12-26 21:57:39,107][105692] Updated weights for policy 0, policy_version 906246 (0.0005) [2023-12-26 21:57:39,161][105692] Updated weights for policy 0, policy_version 906256 (0.0007) [2023-12-26 21:57:39,236][105620] Updated weights for policy 1, policy_version 906198 (0.0008) [2023-12-26 21:57:39,300][105620] Updated weights for policy 1, policy_version 906209 (0.0008) [2023-12-26 21:57:39,370][105620] Updated weights for policy 1, policy_version 906219 (0.0008) [2023-12-26 21:57:39,911][105692] Updated weights for policy 0, policy_version 906266 (0.0009) [2023-12-26 21:57:39,973][105692] Updated weights for policy 0, policy_version 906276 (0.0009) [2023-12-26 21:57:40,038][105692] Updated weights for policy 0, policy_version 906286 (0.0009) [2023-12-26 21:57:40,075][105620] Updated weights for policy 1, policy_version 906229 (0.0008) [2023-12-26 21:57:40,136][105620] Updated weights for policy 1, policy_version 906239 (0.0009) [2023-12-26 21:57:40,191][105620] Updated weights for policy 1, policy_version 906249 (0.0010) [2023-12-26 21:57:40,799][105692] Updated weights for policy 0, policy_version 906296 (0.0009) [2023-12-26 21:57:40,854][105692] Updated weights for policy 0, policy_version 906306 (0.0009) [2023-12-26 21:57:40,914][105692] Updated weights for policy 0, policy_version 906316 (0.0008) [2023-12-26 21:57:40,960][105620] Updated weights for policy 1, policy_version 906259 (0.0009) [2023-12-26 21:57:41,020][105620] Updated weights for policy 1, policy_version 906269 (0.0009) [2023-12-26 21:57:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19327.6). Total num frames: 464084992. Throughput: 0: 9640.5, 1: 9852.8. Samples: 464093384. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 21:57:41,062][104569] Avg episode reward: [(0, '9018.425'), (1, '7318.746')] [2023-12-26 21:57:41,085][105620] Updated weights for policy 1, policy_version 906279 (0.0007) [2023-12-26 21:57:41,688][105692] Updated weights for policy 0, policy_version 906326 (0.0009) [2023-12-26 21:57:41,754][105692] Updated weights for policy 0, policy_version 906336 (0.0009) [2023-12-26 21:57:41,814][105692] Updated weights for policy 0, policy_version 906346 (0.0010) [2023-12-26 21:57:41,888][105620] Updated weights for policy 1, policy_version 906289 (0.0008) [2023-12-26 21:57:41,941][105620] Updated weights for policy 1, policy_version 906299 (0.0007) [2023-12-26 21:57:41,965][105586] KL-divergence is very high: 386.8873 [2023-12-26 21:57:42,002][105620] Updated weights for policy 1, policy_version 906309 (0.0009) [2023-12-26 21:57:42,015][105586] KL-divergence is very high: 703.8198 [2023-12-26 21:57:42,061][105620] Updated weights for policy 1, policy_version 906319 (0.0009) [2023-12-26 21:57:42,062][105586] KL-divergence is very high: 777.6853 [2023-12-26 21:57:42,581][105692] Updated weights for policy 0, policy_version 906356 (0.0008) [2023-12-26 21:57:42,638][105692] Updated weights for policy 0, policy_version 906366 (0.0006) [2023-12-26 21:57:42,701][105692] Updated weights for policy 0, policy_version 906376 (0.0005) [2023-12-26 21:57:42,852][105620] Updated weights for policy 1, policy_version 906329 (0.0006) [2023-12-26 21:57:42,908][105620] Updated weights for policy 1, policy_version 906339 (0.0005) [2023-12-26 21:57:42,971][105620] Updated weights for policy 1, policy_version 906349 (0.0008) [2023-12-26 21:57:43,352][105692] Updated weights for policy 0, policy_version 906386 (0.0006) [2023-12-26 21:57:43,400][105692] Updated weights for policy 0, policy_version 906396 (0.0008) [2023-12-26 21:57:43,448][105692] Updated weights for policy 0, policy_version 906406 (0.0008) [2023-12-26 21:57:43,500][105692] Updated weights for policy 0, policy_version 906416 (0.0008) [2023-12-26 21:57:43,538][105620] Updated weights for policy 1, policy_version 906359 (0.0009) [2023-12-26 21:57:43,605][105620] Updated weights for policy 1, policy_version 906369 (0.0010) [2023-12-26 21:57:43,671][105620] Updated weights for policy 1, policy_version 906379 (0.0005) [2023-12-26 21:57:44,153][105692] Updated weights for policy 0, policy_version 906426 (0.0010) [2023-12-26 21:57:44,207][105692] Updated weights for policy 0, policy_version 906436 (0.0009) [2023-12-26 21:57:44,263][105692] Updated weights for policy 0, policy_version 906446 (0.0008) [2023-12-26 21:57:44,304][105620] Updated weights for policy 1, policy_version 906389 (0.0008) [2023-12-26 21:57:44,352][105620] Updated weights for policy 1, policy_version 906399 (0.0010) [2023-12-26 21:57:44,359][105586] KL-divergence is very high: 163.7369 [2023-12-26 21:57:44,384][105586] KL-divergence is very high: 117.6477 [2023-12-26 21:57:44,399][105586] KL-divergence is very high: 172.7578 [2023-12-26 21:57:44,401][105620] Updated weights for policy 1, policy_version 906409 (0.0009) [2023-12-26 21:57:44,425][105586] KL-divergence is very high: 108.0757 [2023-12-26 21:57:45,053][105620] Updated weights for policy 1, policy_version 906419 (0.0010) [2023-12-26 21:57:45,117][105692] Updated weights for policy 0, policy_version 906456 (0.0009) [2023-12-26 21:57:45,117][105620] Updated weights for policy 1, policy_version 906429 (0.0010) [2023-12-26 21:57:45,174][105620] Updated weights for policy 1, policy_version 906439 (0.0010) [2023-12-26 21:57:45,176][105692] Updated weights for policy 0, policy_version 906466 (0.0009) [2023-12-26 21:57:45,235][105692] Updated weights for policy 0, policy_version 906476 (0.0006) [2023-12-26 21:57:45,898][105620] Updated weights for policy 1, policy_version 906449 (0.0011) [2023-12-26 21:57:45,901][105692] Updated weights for policy 0, policy_version 906486 (0.0006) [2023-12-26 21:57:45,953][105620] Updated weights for policy 1, policy_version 906459 (0.0010) [2023-12-26 21:57:45,961][105692] Updated weights for policy 0, policy_version 906496 (0.0005) [2023-12-26 21:57:46,011][105620] Updated weights for policy 1, policy_version 906469 (0.0010) [2023-12-26 21:57:46,017][105692] Updated weights for policy 0, policy_version 906506 (0.0005) [2023-12-26 21:57:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19327.6). Total num frames: 464183296. Throughput: 0: 9588.7, 1: 9849.8. Samples: 464151020. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 21:57:46,062][104569] Avg episode reward: [(0, '8574.425'), (1, '8199.402')] [2023-12-26 21:57:46,066][105620] Updated weights for policy 1, policy_version 906479 (0.0010) [2023-12-26 21:57:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000906512_232103936.pth... [2023-12-26 21:57:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000906480_232087552.pth... [2023-12-26 21:57:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000905360_231809024.pth [2023-12-26 21:57:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000905328_231792640.pth [2023-12-26 21:57:46,589][105692] Updated weights for policy 0, policy_version 906516 (0.0007) [2023-12-26 21:57:46,635][105692] Updated weights for policy 0, policy_version 906526 (0.0008) [2023-12-26 21:57:46,686][105692] Updated weights for policy 0, policy_version 906536 (0.0008) [2023-12-26 21:57:46,781][105620] Updated weights for policy 1, policy_version 906489 (0.0006) [2023-12-26 21:57:46,830][105620] Updated weights for policy 1, policy_version 906499 (0.0006) [2023-12-26 21:57:46,892][105620] Updated weights for policy 1, policy_version 906509 (0.0010) [2023-12-26 21:57:47,434][105692] Updated weights for policy 0, policy_version 906546 (0.0009) [2023-12-26 21:57:47,492][105692] Updated weights for policy 0, policy_version 906556 (0.0007) [2023-12-26 21:57:47,558][105692] Updated weights for policy 0, policy_version 906566 (0.0007) [2023-12-26 21:57:47,621][105692] Updated weights for policy 0, policy_version 906576 (0.0007) [2023-12-26 21:57:47,625][105620] Updated weights for policy 1, policy_version 906519 (0.0007) [2023-12-26 21:57:47,669][105620] Updated weights for policy 1, policy_version 906529 (0.0005) [2023-12-26 21:57:47,726][105620] Updated weights for policy 1, policy_version 906539 (0.0008) [2023-12-26 21:57:48,256][105692] Updated weights for policy 0, policy_version 906586 (0.0009) [2023-12-26 21:57:48,307][105692] Updated weights for policy 0, policy_version 906596 (0.0008) [2023-12-26 21:57:48,349][105585] KL-divergence is very high: 132.9837 [2023-12-26 21:57:48,369][105692] Updated weights for policy 0, policy_version 906606 (0.0008) [2023-12-26 21:57:48,503][105620] Updated weights for policy 1, policy_version 906549 (0.0009) [2023-12-26 21:57:48,554][105620] Updated weights for policy 1, policy_version 906559 (0.0009) [2023-12-26 21:57:48,601][105620] Updated weights for policy 1, policy_version 906569 (0.0009) [2023-12-26 21:57:49,070][105692] Updated weights for policy 0, policy_version 906616 (0.0006) [2023-12-26 21:57:49,130][105692] Updated weights for policy 0, policy_version 906626 (0.0005) [2023-12-26 21:57:49,184][105692] Updated weights for policy 0, policy_version 906636 (0.0005) [2023-12-26 21:57:49,510][105620] Updated weights for policy 1, policy_version 906579 (0.0009) [2023-12-26 21:57:49,563][105620] Updated weights for policy 1, policy_version 906589 (0.0008) [2023-12-26 21:57:49,613][105620] Updated weights for policy 1, policy_version 906599 (0.0009) [2023-12-26 21:57:49,803][105692] Updated weights for policy 0, policy_version 906646 (0.0007) [2023-12-26 21:57:49,867][105692] Updated weights for policy 0, policy_version 906656 (0.0009) [2023-12-26 21:57:49,926][105692] Updated weights for policy 0, policy_version 906666 (0.0008) [2023-12-26 21:57:50,422][105620] Updated weights for policy 1, policy_version 906609 (0.0008) [2023-12-26 21:57:50,484][105620] Updated weights for policy 1, policy_version 906619 (0.0010) [2023-12-26 21:57:50,543][105620] Updated weights for policy 1, policy_version 906629 (0.0010) [2023-12-26 21:57:50,588][105692] Updated weights for policy 0, policy_version 906676 (0.0009) [2023-12-26 21:57:50,607][105620] Updated weights for policy 1, policy_version 906639 (0.0008) [2023-12-26 21:57:50,646][105692] Updated weights for policy 0, policy_version 906686 (0.0008) [2023-12-26 21:57:50,700][105692] Updated weights for policy 0, policy_version 906696 (0.0009) [2023-12-26 21:57:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19327.6). Total num frames: 464281600. Throughput: 0: 9635.8, 1: 9834.8. Samples: 464269944. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 21:57:51,062][104569] Avg episode reward: [(0, '8217.572'), (1, '8557.991')] [2023-12-26 21:57:51,415][105620] Updated weights for policy 1, policy_version 906649 (0.0009) [2023-12-26 21:57:51,478][105620] Updated weights for policy 1, policy_version 906659 (0.0007) [2023-12-26 21:57:51,493][105692] Updated weights for policy 0, policy_version 906706 (0.0007) [2023-12-26 21:57:51,533][105620] Updated weights for policy 1, policy_version 906669 (0.0008) [2023-12-26 21:57:51,540][105692] Updated weights for policy 0, policy_version 906716 (0.0006) [2023-12-26 21:57:51,586][105692] Updated weights for policy 0, policy_version 906726 (0.0009) [2023-12-26 21:57:51,644][105692] Updated weights for policy 0, policy_version 906736 (0.0009) [2023-12-26 21:57:52,162][105620] Updated weights for policy 1, policy_version 906679 (0.0007) [2023-12-26 21:57:52,211][105620] Updated weights for policy 1, policy_version 906689 (0.0005) [2023-12-26 21:57:52,268][105620] Updated weights for policy 1, policy_version 906699 (0.0007) [2023-12-26 21:57:52,541][105692] Updated weights for policy 0, policy_version 906746 (0.0008) [2023-12-26 21:57:52,589][105692] Updated weights for policy 0, policy_version 906756 (0.0008) [2023-12-26 21:57:52,634][105692] Updated weights for policy 0, policy_version 906766 (0.0008) [2023-12-26 21:57:52,970][105620] Updated weights for policy 1, policy_version 906709 (0.0010) [2023-12-26 21:57:53,021][105620] Updated weights for policy 1, policy_version 906719 (0.0010) [2023-12-26 21:57:53,076][105620] Updated weights for policy 1, policy_version 906729 (0.0010) [2023-12-26 21:57:53,336][105692] Updated weights for policy 0, policy_version 906776 (0.0006) [2023-12-26 21:57:53,387][105692] Updated weights for policy 0, policy_version 906786 (0.0007) [2023-12-26 21:57:53,447][105692] Updated weights for policy 0, policy_version 906796 (0.0008) [2023-12-26 21:57:53,835][105620] Updated weights for policy 1, policy_version 906739 (0.0010) [2023-12-26 21:57:53,893][105620] Updated weights for policy 1, policy_version 906749 (0.0010) [2023-12-26 21:57:53,940][105620] Updated weights for policy 1, policy_version 906759 (0.0010) [2023-12-26 21:57:54,197][105692] Updated weights for policy 0, policy_version 906806 (0.0008) [2023-12-26 21:57:54,260][105692] Updated weights for policy 0, policy_version 906816 (0.0008) [2023-12-26 21:57:54,319][105692] Updated weights for policy 0, policy_version 906826 (0.0008) [2023-12-26 21:57:54,684][105620] Updated weights for policy 1, policy_version 906769 (0.0010) [2023-12-26 21:57:54,734][105620] Updated weights for policy 1, policy_version 906779 (0.0010) [2023-12-26 21:57:54,782][105620] Updated weights for policy 1, policy_version 906789 (0.0010) [2023-12-26 21:57:54,829][105620] Updated weights for policy 1, policy_version 906799 (0.0010) [2023-12-26 21:57:55,073][105692] Updated weights for policy 0, policy_version 906836 (0.0008) [2023-12-26 21:57:55,132][105692] Updated weights for policy 0, policy_version 906846 (0.0007) [2023-12-26 21:57:55,178][105692] Updated weights for policy 0, policy_version 906856 (0.0008) [2023-12-26 21:57:55,587][105620] Updated weights for policy 1, policy_version 906809 (0.0010) [2023-12-26 21:57:55,646][105620] Updated weights for policy 1, policy_version 906819 (0.0010) [2023-12-26 21:57:55,699][105620] Updated weights for policy 1, policy_version 906829 (0.0011) [2023-12-26 21:57:55,947][105692] Updated weights for policy 0, policy_version 906866 (0.0008) [2023-12-26 21:57:56,000][105692] Updated weights for policy 0, policy_version 906876 (0.0009) [2023-12-26 21:57:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19299.8). Total num frames: 464371712. Throughput: 0: 9647.9, 1: 9687.1. Samples: 464383388. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 21:57:56,063][104569] Avg episode reward: [(0, '8392.543'), (1, '8376.563')] [2023-12-26 21:57:56,064][105692] Updated weights for policy 0, policy_version 906886 (0.0008) [2023-12-26 21:57:56,126][105692] Updated weights for policy 0, policy_version 906896 (0.0010) [2023-12-26 21:57:56,425][105620] Updated weights for policy 1, policy_version 906839 (0.0007) [2023-12-26 21:57:56,487][105620] Updated weights for policy 1, policy_version 906849 (0.0006) [2023-12-26 21:57:56,534][105620] Updated weights for policy 1, policy_version 906859 (0.0005) [2023-12-26 21:57:56,800][105692] Updated weights for policy 0, policy_version 906906 (0.0009) [2023-12-26 21:57:56,852][105692] Updated weights for policy 0, policy_version 906916 (0.0006) [2023-12-26 21:57:56,899][105692] Updated weights for policy 0, policy_version 906926 (0.0005) [2023-12-26 21:57:57,079][105620] Updated weights for policy 1, policy_version 906869 (0.0005) [2023-12-26 21:57:57,136][105620] Updated weights for policy 1, policy_version 906879 (0.0005) [2023-12-26 21:57:57,202][105620] Updated weights for policy 1, policy_version 906889 (0.0005) [2023-12-26 21:57:57,507][105692] Updated weights for policy 0, policy_version 906936 (0.0006) [2023-12-26 21:57:57,558][105692] Updated weights for policy 0, policy_version 906946 (0.0005) [2023-12-26 21:57:57,609][105692] Updated weights for policy 0, policy_version 906956 (0.0005) [2023-12-26 21:57:57,704][105620] Updated weights for policy 1, policy_version 906899 (0.0005) [2023-12-26 21:57:57,755][105620] Updated weights for policy 1, policy_version 906909 (0.0005) [2023-12-26 21:57:57,806][105620] Updated weights for policy 1, policy_version 906919 (0.0005) [2023-12-26 21:57:58,287][105692] Updated weights for policy 0, policy_version 906966 (0.0008) [2023-12-26 21:57:58,358][105692] Updated weights for policy 0, policy_version 906976 (0.0010) [2023-12-26 21:57:58,414][105620] Updated weights for policy 1, policy_version 906929 (0.0005) [2023-12-26 21:57:58,428][105692] Updated weights for policy 0, policy_version 906986 (0.0010) [2023-12-26 21:57:58,475][105620] Updated weights for policy 1, policy_version 906939 (0.0006) [2023-12-26 21:57:58,539][105620] Updated weights for policy 1, policy_version 906949 (0.0008) [2023-12-26 21:57:58,607][105620] Updated weights for policy 1, policy_version 906959 (0.0007) [2023-12-26 21:57:59,276][105692] Updated weights for policy 0, policy_version 906996 (0.0010) [2023-12-26 21:57:59,341][105692] Updated weights for policy 0, policy_version 907006 (0.0008) [2023-12-26 21:57:59,403][105620] Updated weights for policy 1, policy_version 906969 (0.0008) [2023-12-26 21:57:59,407][105692] Updated weights for policy 0, policy_version 907016 (0.0008) [2023-12-26 21:57:59,466][105620] Updated weights for policy 1, policy_version 906979 (0.0006) [2023-12-26 21:57:59,520][105620] Updated weights for policy 1, policy_version 906989 (0.0005) [2023-12-26 21:58:00,162][105692] Updated weights for policy 0, policy_version 907026 (0.0009) [2023-12-26 21:58:00,196][105620] Updated weights for policy 1, policy_version 906999 (0.0009) [2023-12-26 21:58:00,217][105692] Updated weights for policy 0, policy_version 907036 (0.0010) [2023-12-26 21:58:00,246][105620] Updated weights for policy 1, policy_version 907009 (0.0009) [2023-12-26 21:58:00,272][105692] Updated weights for policy 0, policy_version 907046 (0.0010) [2023-12-26 21:58:00,298][105620] Updated weights for policy 1, policy_version 907019 (0.0010) [2023-12-26 21:58:00,321][105692] Updated weights for policy 0, policy_version 907056 (0.0010) [2023-12-26 21:58:01,038][105692] Updated weights for policy 0, policy_version 907066 (0.0007) [2023-12-26 21:58:01,056][105620] Updated weights for policy 1, policy_version 907029 (0.0010) [2023-12-26 21:58:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19299.8). Total num frames: 464470016. Throughput: 0: 9730.4, 1: 9790.2. Samples: 464447360. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 21:58:01,062][104569] Avg episode reward: [(0, '8844.649'), (1, '8282.970')] [2023-12-26 21:58:01,098][105692] Updated weights for policy 0, policy_version 907076 (0.0011) [2023-12-26 21:58:01,115][105620] Updated weights for policy 1, policy_version 907039 (0.0010) [2023-12-26 21:58:01,156][105692] Updated weights for policy 0, policy_version 907086 (0.0010) [2023-12-26 21:58:01,164][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000907088_232251392.pth... [2023-12-26 21:58:01,167][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000905936_231956480.pth [2023-12-26 21:58:01,183][105620] Updated weights for policy 1, policy_version 907049 (0.0009) [2023-12-26 21:58:01,212][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000907056_232235008.pth... [2023-12-26 21:58:01,215][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000905904_231940096.pth [2023-12-26 21:58:01,874][105692] Updated weights for policy 0, policy_version 907096 (0.0010) [2023-12-26 21:58:01,897][105620] Updated weights for policy 1, policy_version 907059 (0.0009) [2023-12-26 21:58:01,939][105692] Updated weights for policy 0, policy_version 907106 (0.0005) [2023-12-26 21:58:01,960][105620] Updated weights for policy 1, policy_version 907069 (0.0005) [2023-12-26 21:58:01,990][105692] Updated weights for policy 0, policy_version 907116 (0.0005) [2023-12-26 21:58:02,024][105620] Updated weights for policy 1, policy_version 907079 (0.0008) [2023-12-26 21:58:02,601][105692] Updated weights for policy 0, policy_version 907126 (0.0008) [2023-12-26 21:58:02,645][105620] Updated weights for policy 1, policy_version 907089 (0.0010) [2023-12-26 21:58:02,664][105692] Updated weights for policy 0, policy_version 907136 (0.0011) [2023-12-26 21:58:02,712][105620] Updated weights for policy 1, policy_version 907099 (0.0005) [2023-12-26 21:58:02,721][105692] Updated weights for policy 0, policy_version 907146 (0.0010) [2023-12-26 21:58:02,763][105620] Updated weights for policy 1, policy_version 907109 (0.0006) [2023-12-26 21:58:02,814][105620] Updated weights for policy 1, policy_version 907119 (0.0009) [2023-12-26 21:58:03,381][105692] Updated weights for policy 0, policy_version 907156 (0.0010) [2023-12-26 21:58:03,435][105692] Updated weights for policy 0, policy_version 907166 (0.0010) [2023-12-26 21:58:03,484][105692] Updated weights for policy 0, policy_version 907176 (0.0008) [2023-12-26 21:58:03,490][105620] Updated weights for policy 1, policy_version 907129 (0.0006) [2023-12-26 21:58:03,545][105620] Updated weights for policy 1, policy_version 907139 (0.0008) [2023-12-26 21:58:03,597][105620] Updated weights for policy 1, policy_version 907149 (0.0008) [2023-12-26 21:58:04,128][105692] Updated weights for policy 0, policy_version 907186 (0.0008) [2023-12-26 21:58:04,191][105692] Updated weights for policy 0, policy_version 907196 (0.0011) [2023-12-26 21:58:04,258][105692] Updated weights for policy 0, policy_version 907206 (0.0009) [2023-12-26 21:58:04,317][105692] Updated weights for policy 0, policy_version 907216 (0.0010) [2023-12-26 21:58:04,355][105620] Updated weights for policy 1, policy_version 907159 (0.0008) [2023-12-26 21:58:04,420][105620] Updated weights for policy 1, policy_version 907169 (0.0007) [2023-12-26 21:58:04,483][105620] Updated weights for policy 1, policy_version 907179 (0.0005) [2023-12-26 21:58:05,044][105692] Updated weights for policy 0, policy_version 907226 (0.0010) [2023-12-26 21:58:05,109][105692] Updated weights for policy 0, policy_version 907236 (0.0007) [2023-12-26 21:58:05,159][105620] Updated weights for policy 1, policy_version 907189 (0.0007) [2023-12-26 21:58:05,171][105692] Updated weights for policy 0, policy_version 907246 (0.0005) [2023-12-26 21:58:05,209][105620] Updated weights for policy 1, policy_version 907199 (0.0009) [2023-12-26 21:58:05,266][105620] Updated weights for policy 1, policy_version 907210 (0.0010) [2023-12-26 21:58:05,807][105692] Updated weights for policy 0, policy_version 907256 (0.0009) [2023-12-26 21:58:05,866][105692] Updated weights for policy 0, policy_version 907266 (0.0011) [2023-12-26 21:58:05,924][105692] Updated weights for policy 0, policy_version 907276 (0.0010) [2023-12-26 21:58:06,048][105620] Updated weights for policy 1, policy_version 907220 (0.0008) [2023-12-26 21:58:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 464576512. Throughput: 0: 9864.8, 1: 9777.8. Samples: 464565568. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 21:58:06,063][104569] Avg episode reward: [(0, '8747.655'), (1, '8644.989')] [2023-12-26 21:58:06,108][105620] Updated weights for policy 1, policy_version 907230 (0.0007) [2023-12-26 21:58:06,172][105620] Updated weights for policy 1, policy_version 907241 (0.0009) [2023-12-26 21:58:06,654][105692] Updated weights for policy 0, policy_version 907286 (0.0011) [2023-12-26 21:58:06,718][105692] Updated weights for policy 0, policy_version 907296 (0.0011) [2023-12-26 21:58:06,780][105692] Updated weights for policy 0, policy_version 907306 (0.0010) [2023-12-26 21:58:06,943][105620] Updated weights for policy 1, policy_version 907252 (0.0009) [2023-12-26 21:58:07,005][105620] Updated weights for policy 1, policy_version 907262 (0.0010) [2023-12-26 21:58:07,070][105620] Updated weights for policy 1, policy_version 907272 (0.0009) [2023-12-26 21:58:07,438][105692] Updated weights for policy 0, policy_version 907316 (0.0011) [2023-12-26 21:58:07,486][105692] Updated weights for policy 0, policy_version 907326 (0.0010) [2023-12-26 21:58:07,536][105692] Updated weights for policy 0, policy_version 907336 (0.0010) [2023-12-26 21:58:07,830][105620] Updated weights for policy 1, policy_version 907282 (0.0008) [2023-12-26 21:58:07,899][105620] Updated weights for policy 1, policy_version 907292 (0.0009) [2023-12-26 21:58:07,952][105620] Updated weights for policy 1, policy_version 907302 (0.0008) [2023-12-26 21:58:08,004][105620] Updated weights for policy 1, policy_version 907312 (0.0009) [2023-12-26 21:58:08,210][105692] Updated weights for policy 0, policy_version 907346 (0.0009) [2023-12-26 21:58:08,266][105692] Updated weights for policy 0, policy_version 907356 (0.0005) [2023-12-26 21:58:08,312][105692] Updated weights for policy 0, policy_version 907366 (0.0005) [2023-12-26 21:58:08,373][105692] Updated weights for policy 0, policy_version 907376 (0.0008) [2023-12-26 21:58:08,825][105620] Updated weights for policy 1, policy_version 907322 (0.0008) [2023-12-26 21:58:08,884][105620] Updated weights for policy 1, policy_version 907332 (0.0008) [2023-12-26 21:58:08,946][105620] Updated weights for policy 1, policy_version 907342 (0.0008) [2023-12-26 21:58:09,070][105692] Updated weights for policy 0, policy_version 907386 (0.0007) [2023-12-26 21:58:09,118][105692] Updated weights for policy 0, policy_version 907396 (0.0010) [2023-12-26 21:58:09,166][105692] Updated weights for policy 0, policy_version 907406 (0.0010) [2023-12-26 21:58:09,680][105620] Updated weights for policy 1, policy_version 907352 (0.0008) [2023-12-26 21:58:09,736][105620] Updated weights for policy 1, policy_version 907362 (0.0008) [2023-12-26 21:58:09,795][105620] Updated weights for policy 1, policy_version 907372 (0.0008) [2023-12-26 21:58:09,961][105692] Updated weights for policy 0, policy_version 907416 (0.0009) [2023-12-26 21:58:10,026][105692] Updated weights for policy 0, policy_version 907426 (0.0008) [2023-12-26 21:58:10,082][105692] Updated weights for policy 0, policy_version 907436 (0.0011) [2023-12-26 21:58:10,607][105620] Updated weights for policy 1, policy_version 907382 (0.0009) [2023-12-26 21:58:10,659][105620] Updated weights for policy 1, policy_version 907392 (0.0008) [2023-12-26 21:58:10,708][105620] Updated weights for policy 1, policy_version 907402 (0.0008) [2023-12-26 21:58:10,819][105692] Updated weights for policy 0, policy_version 907446 (0.0010) [2023-12-26 21:58:10,864][105692] Updated weights for policy 0, policy_version 907456 (0.0010) [2023-12-26 21:58:10,921][105692] Updated weights for policy 0, policy_version 907466 (0.0010) [2023-12-26 21:58:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19355.3). Total num frames: 464674816. Throughput: 0: 9871.4, 1: 9761.2. Samples: 464679212. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 21:58:11,062][104569] Avg episode reward: [(0, '8989.755'), (1, '8207.851')] [2023-12-26 21:58:11,522][105620] Updated weights for policy 1, policy_version 907412 (0.0008) [2023-12-26 21:58:11,591][105620] Updated weights for policy 1, policy_version 907422 (0.0010) [2023-12-26 21:58:11,654][105692] Updated weights for policy 0, policy_version 907476 (0.0010) [2023-12-26 21:58:11,660][105620] Updated weights for policy 1, policy_version 907432 (0.0009) [2023-12-26 21:58:11,722][105692] Updated weights for policy 0, policy_version 907486 (0.0008) [2023-12-26 21:58:11,792][105692] Updated weights for policy 0, policy_version 907496 (0.0007) [2023-12-26 21:58:12,436][105620] Updated weights for policy 1, policy_version 907442 (0.0009) [2023-12-26 21:58:12,450][105692] Updated weights for policy 0, policy_version 907506 (0.0009) [2023-12-26 21:58:12,492][105620] Updated weights for policy 1, policy_version 907452 (0.0005) [2023-12-26 21:58:12,509][105692] Updated weights for policy 0, policy_version 907516 (0.0011) [2023-12-26 21:58:12,551][105620] Updated weights for policy 1, policy_version 907462 (0.0006) [2023-12-26 21:58:12,568][105692] Updated weights for policy 0, policy_version 907526 (0.0011) [2023-12-26 21:58:12,604][105620] Updated weights for policy 1, policy_version 907472 (0.0007) [2023-12-26 21:58:12,630][105692] Updated weights for policy 0, policy_version 907536 (0.0011) [2023-12-26 21:58:13,241][105692] Updated weights for policy 0, policy_version 907546 (0.0006) [2023-12-26 21:58:13,289][105692] Updated weights for policy 0, policy_version 907556 (0.0005) [2023-12-26 21:58:13,348][105692] Updated weights for policy 0, policy_version 907566 (0.0005) [2023-12-26 21:58:13,417][105620] Updated weights for policy 1, policy_version 907482 (0.0009) [2023-12-26 21:58:13,474][105620] Updated weights for policy 1, policy_version 907492 (0.0009) [2023-12-26 21:58:13,525][105620] Updated weights for policy 1, policy_version 907502 (0.0009) [2023-12-26 21:58:13,874][105692] Updated weights for policy 0, policy_version 907576 (0.0005) [2023-12-26 21:58:13,926][105692] Updated weights for policy 0, policy_version 907586 (0.0005) [2023-12-26 21:58:13,987][105692] Updated weights for policy 0, policy_version 907596 (0.0007) [2023-12-26 21:58:14,381][105620] Updated weights for policy 1, policy_version 907512 (0.0009) [2023-12-26 21:58:14,437][105620] Updated weights for policy 1, policy_version 907522 (0.0005) [2023-12-26 21:58:14,489][105620] Updated weights for policy 1, policy_version 907532 (0.0009) [2023-12-26 21:58:14,589][105692] Updated weights for policy 0, policy_version 907606 (0.0005) [2023-12-26 21:58:14,635][105692] Updated weights for policy 0, policy_version 907616 (0.0005) [2023-12-26 21:58:14,681][105692] Updated weights for policy 0, policy_version 907626 (0.0005) [2023-12-26 21:58:15,228][105620] Updated weights for policy 1, policy_version 907542 (0.0009) [2023-12-26 21:58:15,291][105620] Updated weights for policy 1, policy_version 907552 (0.0007) [2023-12-26 21:58:15,340][105620] Updated weights for policy 1, policy_version 907562 (0.0008) [2023-12-26 21:58:15,433][105692] Updated weights for policy 0, policy_version 907636 (0.0009) [2023-12-26 21:58:15,489][105692] Updated weights for policy 0, policy_version 907646 (0.0010) [2023-12-26 21:58:15,552][105692] Updated weights for policy 0, policy_version 907656 (0.0011) [2023-12-26 21:58:16,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19327.6). Total num frames: 464764928. Throughput: 0: 9816.2, 1: 9607.9. Samples: 464735988. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 21:58:16,062][104569] Avg episode reward: [(0, '9172.264'), (1, '7939.992')] [2023-12-26 21:58:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000907664_232398848.pth... [2023-12-26 21:58:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000906512_232103936.pth [2023-12-26 21:58:16,106][105620] Updated weights for policy 1, policy_version 907572 (0.0008) [2023-12-26 21:58:16,166][105620] Updated weights for policy 1, policy_version 907582 (0.0008) [2023-12-26 21:58:16,218][105620] Updated weights for policy 1, policy_version 907592 (0.0008) [2023-12-26 21:58:16,222][105586] KL-divergence is very high: 174.6913 [2023-12-26 21:58:16,249][105586] KL-divergence is very high: 120.4890 [2023-12-26 21:58:16,253][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000907600_232374272.pth... [2023-12-26 21:58:16,256][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000906480_232087552.pth [2023-12-26 21:58:16,289][105692] Updated weights for policy 0, policy_version 907666 (0.0011) [2023-12-26 21:58:16,347][105692] Updated weights for policy 0, policy_version 907676 (0.0010) [2023-12-26 21:58:16,399][105692] Updated weights for policy 0, policy_version 907686 (0.0010) [2023-12-26 21:58:16,448][105692] Updated weights for policy 0, policy_version 907696 (0.0010) [2023-12-26 21:58:16,968][105620] Updated weights for policy 1, policy_version 907602 (0.0008) [2023-12-26 21:58:17,039][105620] Updated weights for policy 1, policy_version 907612 (0.0010) [2023-12-26 21:58:17,106][105620] Updated weights for policy 1, policy_version 907622 (0.0009) [2023-12-26 21:58:17,127][105692] Updated weights for policy 0, policy_version 907706 (0.0007) [2023-12-26 21:58:17,163][105620] Updated weights for policy 1, policy_version 907632 (0.0006) [2023-12-26 21:58:17,177][105692] Updated weights for policy 0, policy_version 907716 (0.0008) [2023-12-26 21:58:17,225][105692] Updated weights for policy 0, policy_version 907726 (0.0009) [2023-12-26 21:58:17,892][105620] Updated weights for policy 1, policy_version 907642 (0.0005) [2023-12-26 21:58:17,910][105692] Updated weights for policy 0, policy_version 907736 (0.0006) [2023-12-26 21:58:17,947][105620] Updated weights for policy 1, policy_version 907652 (0.0005) [2023-12-26 21:58:17,969][105692] Updated weights for policy 0, policy_version 907746 (0.0005) [2023-12-26 21:58:18,006][105620] Updated weights for policy 1, policy_version 907662 (0.0006) [2023-12-26 21:58:18,031][105692] Updated weights for policy 0, policy_version 907756 (0.0005) [2023-12-26 21:58:18,619][105620] Updated weights for policy 1, policy_version 907672 (0.0007) [2023-12-26 21:58:18,690][105620] Updated weights for policy 1, policy_version 907682 (0.0007) [2023-12-26 21:58:18,720][105692] Updated weights for policy 0, policy_version 907766 (0.0007) [2023-12-26 21:58:18,747][105620] Updated weights for policy 1, policy_version 907692 (0.0007) [2023-12-26 21:58:18,786][105692] Updated weights for policy 0, policy_version 907776 (0.0008) [2023-12-26 21:58:18,844][105692] Updated weights for policy 0, policy_version 907786 (0.0010) [2023-12-26 21:58:19,509][105620] Updated weights for policy 1, policy_version 907702 (0.0008) [2023-12-26 21:58:19,558][105692] Updated weights for policy 0, policy_version 907796 (0.0008) [2023-12-26 21:58:19,577][105620] Updated weights for policy 1, policy_version 907712 (0.0008) [2023-12-26 21:58:19,621][105692] Updated weights for policy 0, policy_version 907806 (0.0006) [2023-12-26 21:58:19,640][105620] Updated weights for policy 1, policy_version 907722 (0.0008) [2023-12-26 21:58:19,678][105692] Updated weights for policy 0, policy_version 907816 (0.0007) [2023-12-26 21:58:20,309][105692] Updated weights for policy 0, policy_version 907826 (0.0006) [2023-12-26 21:58:20,369][105692] Updated weights for policy 0, policy_version 907836 (0.0009) [2023-12-26 21:58:20,424][105692] Updated weights for policy 0, policy_version 907846 (0.0005) [2023-12-26 21:58:20,471][105620] Updated weights for policy 1, policy_version 907732 (0.0009) [2023-12-26 21:58:20,484][105692] Updated weights for policy 0, policy_version 907856 (0.0006) [2023-12-26 21:58:20,525][105620] Updated weights for policy 1, policy_version 907742 (0.0009) [2023-12-26 21:58:20,584][105620] Updated weights for policy 1, policy_version 907752 (0.0009) [2023-12-26 21:58:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19355.3). Total num frames: 464863232. Throughput: 0: 9869.5, 1: 9587.8. Samples: 464854236. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 21:58:21,062][104569] Avg episode reward: [(0, '9259.327'), (1, '8208.957')] [2023-12-26 21:58:21,218][105692] Updated weights for policy 0, policy_version 907866 (0.0008) [2023-12-26 21:58:21,283][105692] Updated weights for policy 0, policy_version 907876 (0.0008) [2023-12-26 21:58:21,349][105692] Updated weights for policy 0, policy_version 907886 (0.0009) [2023-12-26 21:58:21,427][105620] Updated weights for policy 1, policy_version 907762 (0.0009) [2023-12-26 21:58:21,481][105620] Updated weights for policy 1, policy_version 907772 (0.0010) [2023-12-26 21:58:21,544][105620] Updated weights for policy 1, policy_version 907782 (0.0007) [2023-12-26 21:58:21,598][105620] Updated weights for policy 1, policy_version 907792 (0.0006) [2023-12-26 21:58:22,062][105692] Updated weights for policy 0, policy_version 907896 (0.0008) [2023-12-26 21:58:22,127][105692] Updated weights for policy 0, policy_version 907906 (0.0008) [2023-12-26 21:58:22,184][105692] Updated weights for policy 0, policy_version 907916 (0.0009) [2023-12-26 21:58:22,344][105620] Updated weights for policy 1, policy_version 907802 (0.0008) [2023-12-26 21:58:22,411][105620] Updated weights for policy 1, policy_version 907812 (0.0008) [2023-12-26 21:58:22,470][105620] Updated weights for policy 1, policy_version 907822 (0.0008) [2023-12-26 21:58:22,964][105692] Updated weights for policy 0, policy_version 907926 (0.0009) [2023-12-26 21:58:23,019][105692] Updated weights for policy 0, policy_version 907936 (0.0009) [2023-12-26 21:58:23,069][105692] Updated weights for policy 0, policy_version 907946 (0.0009) [2023-12-26 21:58:23,231][105620] Updated weights for policy 1, policy_version 907832 (0.0009) [2023-12-26 21:58:23,282][105620] Updated weights for policy 1, policy_version 907842 (0.0009) [2023-12-26 21:58:23,337][105620] Updated weights for policy 1, policy_version 907852 (0.0009) [2023-12-26 21:58:23,829][105692] Updated weights for policy 0, policy_version 907956 (0.0009) [2023-12-26 21:58:23,890][105692] Updated weights for policy 0, policy_version 907966 (0.0009) [2023-12-26 21:58:23,948][105692] Updated weights for policy 0, policy_version 907976 (0.0009) [2023-12-26 21:58:24,109][105620] Updated weights for policy 1, policy_version 907862 (0.0009) [2023-12-26 21:58:24,164][105620] Updated weights for policy 1, policy_version 907872 (0.0009) [2023-12-26 21:58:24,225][105620] Updated weights for policy 1, policy_version 907882 (0.0009) [2023-12-26 21:58:24,606][105692] Updated weights for policy 0, policy_version 907986 (0.0009) [2023-12-26 21:58:24,666][105692] Updated weights for policy 0, policy_version 907996 (0.0008) [2023-12-26 21:58:24,718][105692] Updated weights for policy 0, policy_version 908006 (0.0009) [2023-12-26 21:58:24,767][105692] Updated weights for policy 0, policy_version 908016 (0.0009) [2023-12-26 21:58:24,984][105620] Updated weights for policy 1, policy_version 907892 (0.0009) [2023-12-26 21:58:25,039][105620] Updated weights for policy 1, policy_version 907902 (0.0008) [2023-12-26 21:58:25,094][105620] Updated weights for policy 1, policy_version 907912 (0.0008) [2023-12-26 21:58:25,544][105692] Updated weights for policy 0, policy_version 908026 (0.0008) [2023-12-26 21:58:25,601][105692] Updated weights for policy 0, policy_version 908036 (0.0005) [2023-12-26 21:58:25,658][105692] Updated weights for policy 0, policy_version 908046 (0.0005) [2023-12-26 21:58:25,894][105620] Updated weights for policy 1, policy_version 907922 (0.0008) [2023-12-26 21:58:25,959][105620] Updated weights for policy 1, policy_version 907932 (0.0008) [2023-12-26 21:58:26,020][105620] Updated weights for policy 1, policy_version 907942 (0.0010) [2023-12-26 21:58:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19327.6). Total num frames: 464953344. Throughput: 0: 9903.1, 1: 9495.4. Samples: 464966316. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 21:58:26,062][104569] Avg episode reward: [(0, '9257.926'), (1, '8731.525')] [2023-12-26 21:58:26,081][105620] Updated weights for policy 1, policy_version 907952 (0.0009) [2023-12-26 21:58:26,220][105692] Updated weights for policy 0, policy_version 908056 (0.0008) [2023-12-26 21:58:26,267][105692] Updated weights for policy 0, policy_version 908066 (0.0006) [2023-12-26 21:58:26,315][105692] Updated weights for policy 0, policy_version 908076 (0.0005) [2023-12-26 21:58:26,789][105620] Updated weights for policy 1, policy_version 907962 (0.0006) [2023-12-26 21:58:26,851][105620] Updated weights for policy 1, policy_version 907972 (0.0006) [2023-12-26 21:58:26,917][105620] Updated weights for policy 1, policy_version 907982 (0.0006) [2023-12-26 21:58:27,038][105692] Updated weights for policy 0, policy_version 908086 (0.0005) [2023-12-26 21:58:27,100][105692] Updated weights for policy 0, policy_version 908096 (0.0006) [2023-12-26 21:58:27,148][105692] Updated weights for policy 0, policy_version 908106 (0.0005) [2023-12-26 21:58:27,640][105620] Updated weights for policy 1, policy_version 907992 (0.0009) [2023-12-26 21:58:27,685][105692] Updated weights for policy 0, policy_version 908116 (0.0005) [2023-12-26 21:58:27,692][105620] Updated weights for policy 1, policy_version 908002 (0.0008) [2023-12-26 21:58:27,735][105692] Updated weights for policy 0, policy_version 908126 (0.0005) [2023-12-26 21:58:27,743][105620] Updated weights for policy 1, policy_version 908012 (0.0008) [2023-12-26 21:58:27,779][105692] Updated weights for policy 0, policy_version 908136 (0.0008) [2023-12-26 21:58:28,392][105692] Updated weights for policy 0, policy_version 908146 (0.0010) [2023-12-26 21:58:28,453][105692] Updated weights for policy 0, policy_version 908156 (0.0008) [2023-12-26 21:58:28,515][105692] Updated weights for policy 0, policy_version 908166 (0.0008) [2023-12-26 21:58:28,560][105620] Updated weights for policy 1, policy_version 908022 (0.0007) [2023-12-26 21:58:28,570][105692] Updated weights for policy 0, policy_version 908176 (0.0008) [2023-12-26 21:58:28,614][105620] Updated weights for policy 1, policy_version 908032 (0.0008) [2023-12-26 21:58:28,662][105620] Updated weights for policy 1, policy_version 908042 (0.0008) [2023-12-26 21:58:29,311][105692] Updated weights for policy 0, policy_version 908186 (0.0008) [2023-12-26 21:58:29,351][105620] Updated weights for policy 1, policy_version 908052 (0.0007) [2023-12-26 21:58:29,373][105692] Updated weights for policy 0, policy_version 908196 (0.0008) [2023-12-26 21:58:29,416][105620] Updated weights for policy 1, policy_version 908062 (0.0007) [2023-12-26 21:58:29,431][105692] Updated weights for policy 0, policy_version 908206 (0.0008) [2023-12-26 21:58:29,473][105620] Updated weights for policy 1, policy_version 908072 (0.0007) [2023-12-26 21:58:30,131][105620] Updated weights for policy 1, policy_version 908082 (0.0008) [2023-12-26 21:58:30,189][105620] Updated weights for policy 1, policy_version 908092 (0.0006) [2023-12-26 21:58:30,194][105692] Updated weights for policy 0, policy_version 908216 (0.0010) [2023-12-26 21:58:30,247][105620] Updated weights for policy 1, policy_version 908102 (0.0006) [2023-12-26 21:58:30,257][105692] Updated weights for policy 0, policy_version 908226 (0.0011) [2023-12-26 21:58:30,287][105586] KL-divergence is very high: 120.7986 [2023-12-26 21:58:30,309][105692] Updated weights for policy 0, policy_version 908236 (0.0010) [2023-12-26 21:58:30,313][105620] Updated weights for policy 1, policy_version 908112 (0.0007) [2023-12-26 21:58:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19327.6). Total num frames: 465051648. Throughput: 0: 10016.6, 1: 9469.8. Samples: 465027908. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-26 21:58:31,063][104569] Avg episode reward: [(0, '9164.818'), (1, '8365.423')] [2023-12-26 21:58:31,064][105692] Updated weights for policy 0, policy_version 908246 (0.0009) [2023-12-26 21:58:31,067][105620] Updated weights for policy 1, policy_version 908122 (0.0008) [2023-12-26 21:58:31,117][105620] Updated weights for policy 1, policy_version 908132 (0.0006) [2023-12-26 21:58:31,123][105692] Updated weights for policy 0, policy_version 908256 (0.0010) [2023-12-26 21:58:31,180][105620] Updated weights for policy 1, policy_version 908142 (0.0006) [2023-12-26 21:58:31,190][105692] Updated weights for policy 0, policy_version 908266 (0.0011) [2023-12-26 21:58:31,192][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000908144_232513536.pth... [2023-12-26 21:58:31,197][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000907056_232235008.pth [2023-12-26 21:58:31,223][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000908272_232554496.pth... [2023-12-26 21:58:31,226][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000907088_232251392.pth [2023-12-26 21:58:31,919][105620] Updated weights for policy 1, policy_version 908152 (0.0008) [2023-12-26 21:58:31,925][105692] Updated weights for policy 0, policy_version 908276 (0.0010) [2023-12-26 21:58:31,960][105586] KL-divergence is very high: 124.7727 [2023-12-26 21:58:31,969][105620] Updated weights for policy 1, policy_version 908162 (0.0006) [2023-12-26 21:58:31,988][105692] Updated weights for policy 0, policy_version 908286 (0.0010) [2023-12-26 21:58:32,008][105586] KL-divergence is very high: 150.1142 [2023-12-26 21:58:32,032][105620] Updated weights for policy 1, policy_version 908172 (0.0005) [2023-12-26 21:58:32,040][105692] Updated weights for policy 0, policy_version 908296 (0.0010) [2023-12-26 21:58:32,681][105620] Updated weights for policy 1, policy_version 908182 (0.0008) [2023-12-26 21:58:32,740][105620] Updated weights for policy 1, policy_version 908192 (0.0009) [2023-12-26 21:58:32,794][105620] Updated weights for policy 1, policy_version 908202 (0.0007) [2023-12-26 21:58:32,797][105692] Updated weights for policy 0, policy_version 908306 (0.0008) [2023-12-26 21:58:32,854][105692] Updated weights for policy 0, policy_version 908316 (0.0006) [2023-12-26 21:58:32,909][105692] Updated weights for policy 0, policy_version 908326 (0.0007) [2023-12-26 21:58:32,963][105692] Updated weights for policy 0, policy_version 908336 (0.0005) [2023-12-26 21:58:33,550][105620] Updated weights for policy 1, policy_version 908212 (0.0008) [2023-12-26 21:58:33,610][105620] Updated weights for policy 1, policy_version 908222 (0.0009) [2023-12-26 21:58:33,670][105620] Updated weights for policy 1, policy_version 908232 (0.0007) [2023-12-26 21:58:33,675][105692] Updated weights for policy 0, policy_version 908346 (0.0007) [2023-12-26 21:58:33,735][105692] Updated weights for policy 0, policy_version 908356 (0.0007) [2023-12-26 21:58:33,794][105692] Updated weights for policy 0, policy_version 908366 (0.0009) [2023-12-26 21:58:34,432][105620] Updated weights for policy 1, policy_version 908242 (0.0007) [2023-12-26 21:58:34,489][105620] Updated weights for policy 1, policy_version 908252 (0.0009) [2023-12-26 21:58:34,545][105620] Updated weights for policy 1, policy_version 908262 (0.0006) [2023-12-26 21:58:34,562][105692] Updated weights for policy 0, policy_version 908376 (0.0009) [2023-12-26 21:58:34,598][105620] Updated weights for policy 1, policy_version 908272 (0.0006) [2023-12-26 21:58:34,617][105692] Updated weights for policy 0, policy_version 908386 (0.0008) [2023-12-26 21:58:34,664][105692] Updated weights for policy 0, policy_version 908396 (0.0009) [2023-12-26 21:58:35,337][105692] Updated weights for policy 0, policy_version 908406 (0.0010) [2023-12-26 21:58:35,378][105620] Updated weights for policy 1, policy_version 908282 (0.0007) [2023-12-26 21:58:35,382][105692] Updated weights for policy 0, policy_version 908416 (0.0010) [2023-12-26 21:58:35,435][105620] Updated weights for policy 1, policy_version 908292 (0.0007) [2023-12-26 21:58:35,439][105692] Updated weights for policy 0, policy_version 908426 (0.0009) [2023-12-26 21:58:35,492][105620] Updated weights for policy 1, policy_version 908302 (0.0009) [2023-12-26 21:58:35,993][105692] Updated weights for policy 0, policy_version 908436 (0.0005) [2023-12-26 21:58:36,051][105692] Updated weights for policy 0, policy_version 908446 (0.0005) [2023-12-26 21:58:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19327.6). Total num frames: 465149952. Throughput: 0: 9895.1, 1: 9496.6. Samples: 465142572. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 21:58:36,063][104569] Avg episode reward: [(0, '9163.956'), (1, '8450.279')] [2023-12-26 21:58:36,113][105692] Updated weights for policy 0, policy_version 908456 (0.0006) [2023-12-26 21:58:36,343][105620] Updated weights for policy 1, policy_version 908312 (0.0009) [2023-12-26 21:58:36,410][105620] Updated weights for policy 1, policy_version 908322 (0.0008) [2023-12-26 21:58:36,431][105586] KL-divergence is very high: 136.7385 [2023-12-26 21:58:36,477][105620] Updated weights for policy 1, policy_version 908332 (0.0008) [2023-12-26 21:58:36,481][105586] KL-divergence is very high: 155.1012 [2023-12-26 21:58:36,814][105692] Updated weights for policy 0, policy_version 908466 (0.0009) [2023-12-26 21:58:36,862][105692] Updated weights for policy 0, policy_version 908476 (0.0010) [2023-12-26 21:58:36,911][105692] Updated weights for policy 0, policy_version 908486 (0.0010) [2023-12-26 21:58:36,959][105692] Updated weights for policy 0, policy_version 908496 (0.0010) [2023-12-26 21:58:37,246][105620] Updated weights for policy 1, policy_version 908342 (0.0009) [2023-12-26 21:58:37,302][105620] Updated weights for policy 1, policy_version 908352 (0.0009) [2023-12-26 21:58:37,357][105620] Updated weights for policy 1, policy_version 908362 (0.0008) [2023-12-26 21:58:37,670][105692] Updated weights for policy 0, policy_version 908506 (0.0005) [2023-12-26 21:58:37,728][105692] Updated weights for policy 0, policy_version 908516 (0.0005) [2023-12-26 21:58:37,786][105692] Updated weights for policy 0, policy_version 908526 (0.0008) [2023-12-26 21:58:38,224][105620] Updated weights for policy 1, policy_version 908372 (0.0009) [2023-12-26 21:58:38,285][105620] Updated weights for policy 1, policy_version 908382 (0.0007) [2023-12-26 21:58:38,342][105620] Updated weights for policy 1, policy_version 908392 (0.0007) [2023-12-26 21:58:38,426][105692] Updated weights for policy 0, policy_version 908536 (0.0008) [2023-12-26 21:58:38,488][105692] Updated weights for policy 0, policy_version 908546 (0.0008) [2023-12-26 21:58:38,543][105692] Updated weights for policy 0, policy_version 908556 (0.0008) [2023-12-26 21:58:38,974][105620] Updated weights for policy 1, policy_version 908402 (0.0006) [2023-12-26 21:58:39,044][105620] Updated weights for policy 1, policy_version 908412 (0.0005) [2023-12-26 21:58:39,108][105620] Updated weights for policy 1, policy_version 908422 (0.0006) [2023-12-26 21:58:39,161][105620] Updated weights for policy 1, policy_version 908432 (0.0005) [2023-12-26 21:58:39,343][105692] Updated weights for policy 0, policy_version 908566 (0.0008) [2023-12-26 21:58:39,404][105692] Updated weights for policy 0, policy_version 908576 (0.0008) [2023-12-26 21:58:39,468][105692] Updated weights for policy 0, policy_version 908586 (0.0009) [2023-12-26 21:58:39,791][105620] Updated weights for policy 1, policy_version 908442 (0.0009) [2023-12-26 21:58:39,856][105620] Updated weights for policy 1, policy_version 908452 (0.0009) [2023-12-26 21:58:39,914][105620] Updated weights for policy 1, policy_version 908462 (0.0009) [2023-12-26 21:58:40,237][105692] Updated weights for policy 0, policy_version 908596 (0.0010) [2023-12-26 21:58:40,289][105692] Updated weights for policy 0, policy_version 908606 (0.0009) [2023-12-26 21:58:40,345][105692] Updated weights for policy 0, policy_version 908616 (0.0009) [2023-12-26 21:58:40,715][105620] Updated weights for policy 1, policy_version 908472 (0.0006) [2023-12-26 21:58:40,775][105620] Updated weights for policy 1, policy_version 908482 (0.0008) [2023-12-26 21:58:40,833][105620] Updated weights for policy 1, policy_version 908492 (0.0009) [2023-12-26 21:58:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 465248256. Throughput: 0: 9986.1, 1: 9447.8. Samples: 465257912. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 21:58:41,063][104569] Avg episode reward: [(0, '9167.391'), (1, '8725.031')] [2023-12-26 21:58:41,097][105692] Updated weights for policy 0, policy_version 908626 (0.0010) [2023-12-26 21:58:41,163][105692] Updated weights for policy 0, policy_version 908636 (0.0009) [2023-12-26 21:58:41,218][105692] Updated weights for policy 0, policy_version 908646 (0.0009) [2023-12-26 21:58:41,279][105692] Updated weights for policy 0, policy_version 908656 (0.0009) [2023-12-26 21:58:41,548][105620] Updated weights for policy 1, policy_version 908502 (0.0007) [2023-12-26 21:58:41,612][105620] Updated weights for policy 1, policy_version 908512 (0.0009) [2023-12-26 21:58:41,679][105620] Updated weights for policy 1, policy_version 908522 (0.0009) [2023-12-26 21:58:42,089][105692] Updated weights for policy 0, policy_version 908666 (0.0011) [2023-12-26 21:58:42,161][105692] Updated weights for policy 0, policy_version 908676 (0.0010) [2023-12-26 21:58:42,221][105692] Updated weights for policy 0, policy_version 908686 (0.0011) [2023-12-26 21:58:42,301][105620] Updated weights for policy 1, policy_version 908532 (0.0007) [2023-12-26 21:58:42,370][105620] Updated weights for policy 1, policy_version 908542 (0.0009) [2023-12-26 21:58:42,429][105620] Updated weights for policy 1, policy_version 908552 (0.0010) [2023-12-26 21:58:42,891][105692] Updated weights for policy 0, policy_version 908696 (0.0010) [2023-12-26 21:58:42,948][105692] Updated weights for policy 0, policy_version 908706 (0.0010) [2023-12-26 21:58:42,996][105692] Updated weights for policy 0, policy_version 908716 (0.0010) [2023-12-26 21:58:43,197][105620] Updated weights for policy 1, policy_version 908562 (0.0011) [2023-12-26 21:58:43,260][105620] Updated weights for policy 1, policy_version 908572 (0.0011) [2023-12-26 21:58:43,313][105620] Updated weights for policy 1, policy_version 908582 (0.0011) [2023-12-26 21:58:43,365][105620] Updated weights for policy 1, policy_version 908592 (0.0010) [2023-12-26 21:58:43,716][105692] Updated weights for policy 0, policy_version 908726 (0.0010) [2023-12-26 21:58:43,773][105692] Updated weights for policy 0, policy_version 908736 (0.0010) [2023-12-26 21:58:43,840][105692] Updated weights for policy 0, policy_version 908746 (0.0008) [2023-12-26 21:58:44,085][105620] Updated weights for policy 1, policy_version 908602 (0.0008) [2023-12-26 21:58:44,148][105620] Updated weights for policy 1, policy_version 908612 (0.0010) [2023-12-26 21:58:44,213][105620] Updated weights for policy 1, policy_version 908622 (0.0010) [2023-12-26 21:58:44,510][105692] Updated weights for policy 0, policy_version 908756 (0.0008) [2023-12-26 21:58:44,563][105692] Updated weights for policy 0, policy_version 908766 (0.0008) [2023-12-26 21:58:44,619][105692] Updated weights for policy 0, policy_version 908776 (0.0008) [2023-12-26 21:58:44,945][105620] Updated weights for policy 1, policy_version 908632 (0.0010) [2023-12-26 21:58:45,004][105620] Updated weights for policy 1, policy_version 908642 (0.0005) [2023-12-26 21:58:45,065][105620] Updated weights for policy 1, policy_version 908652 (0.0006) [2023-12-26 21:58:45,372][105692] Updated weights for policy 0, policy_version 908786 (0.0008) [2023-12-26 21:58:45,435][105692] Updated weights for policy 0, policy_version 908796 (0.0008) [2023-12-26 21:58:45,495][105692] Updated weights for policy 0, policy_version 908806 (0.0008) [2023-12-26 21:58:45,544][105692] Updated weights for policy 0, policy_version 908816 (0.0008) [2023-12-26 21:58:45,761][105620] Updated weights for policy 1, policy_version 908662 (0.0006) [2023-12-26 21:58:45,827][105620] Updated weights for policy 1, policy_version 908672 (0.0007) [2023-12-26 21:58:45,898][105620] Updated weights for policy 1, policy_version 908682 (0.0010) [2023-12-26 21:58:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 465346560. Throughput: 0: 9939.4, 1: 9349.4. Samples: 465315360. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 21:58:46,063][104569] Avg episode reward: [(0, '9171.678'), (1, '8633.621')] [2023-12-26 21:58:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000908816_232693760.pth... [2023-12-26 21:58:46,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000908688_232652800.pth... [2023-12-26 21:58:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000907664_232398848.pth [2023-12-26 21:58:46,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000907600_232374272.pth [2023-12-26 21:58:46,182][105692] Updated weights for policy 0, policy_version 908826 (0.0005) [2023-12-26 21:58:46,236][105692] Updated weights for policy 0, policy_version 908836 (0.0005) [2023-12-26 21:58:46,300][105692] Updated weights for policy 0, policy_version 908846 (0.0008) [2023-12-26 21:58:46,599][105620] Updated weights for policy 1, policy_version 908692 (0.0010) [2023-12-26 21:58:46,660][105620] Updated weights for policy 1, policy_version 908702 (0.0010) [2023-12-26 21:58:46,722][105620] Updated weights for policy 1, policy_version 908712 (0.0009) [2023-12-26 21:58:46,955][105692] Updated weights for policy 0, policy_version 908856 (0.0009) [2023-12-26 21:58:47,007][105692] Updated weights for policy 0, policy_version 908866 (0.0009) [2023-12-26 21:58:47,058][105692] Updated weights for policy 0, policy_version 908876 (0.0009) [2023-12-26 21:58:47,456][105620] Updated weights for policy 1, policy_version 908722 (0.0008) [2023-12-26 21:58:47,513][105620] Updated weights for policy 1, policy_version 908732 (0.0006) [2023-12-26 21:58:47,585][105620] Updated weights for policy 1, policy_version 908742 (0.0005) [2023-12-26 21:58:47,639][105620] Updated weights for policy 1, policy_version 908752 (0.0005) [2023-12-26 21:58:47,762][105692] Updated weights for policy 0, policy_version 908886 (0.0007) [2023-12-26 21:58:47,807][105692] Updated weights for policy 0, policy_version 908896 (0.0005) [2023-12-26 21:58:47,856][105692] Updated weights for policy 0, policy_version 908906 (0.0005) [2023-12-26 21:58:48,200][105620] Updated weights for policy 1, policy_version 908762 (0.0010) [2023-12-26 21:58:48,270][105620] Updated weights for policy 1, policy_version 908772 (0.0009) [2023-12-26 21:58:48,335][105620] Updated weights for policy 1, policy_version 908782 (0.0009) [2023-12-26 21:58:48,411][105692] Updated weights for policy 0, policy_version 908916 (0.0006) [2023-12-26 21:58:48,472][105692] Updated weights for policy 0, policy_version 908926 (0.0009) [2023-12-26 21:58:48,535][105692] Updated weights for policy 0, policy_version 908936 (0.0009) [2023-12-26 21:58:49,083][105620] Updated weights for policy 1, policy_version 908792 (0.0010) [2023-12-26 21:58:49,149][105620] Updated weights for policy 1, policy_version 908802 (0.0010) [2023-12-26 21:58:49,201][105620] Updated weights for policy 1, policy_version 908812 (0.0010) [2023-12-26 21:58:49,216][105692] Updated weights for policy 0, policy_version 908946 (0.0010) [2023-12-26 21:58:49,283][105692] Updated weights for policy 0, policy_version 908956 (0.0007) [2023-12-26 21:58:49,349][105692] Updated weights for policy 0, policy_version 908966 (0.0006) [2023-12-26 21:58:49,415][105692] Updated weights for policy 0, policy_version 908976 (0.0008) [2023-12-26 21:58:49,945][105620] Updated weights for policy 1, policy_version 908822 (0.0010) [2023-12-26 21:58:50,012][105620] Updated weights for policy 1, policy_version 908832 (0.0009) [2023-12-26 21:58:50,071][105620] Updated weights for policy 1, policy_version 908842 (0.0010) [2023-12-26 21:58:50,115][105692] Updated weights for policy 0, policy_version 908986 (0.0007) [2023-12-26 21:58:50,172][105692] Updated weights for policy 0, policy_version 908996 (0.0008) [2023-12-26 21:58:50,228][105692] Updated weights for policy 0, policy_version 909006 (0.0008) [2023-12-26 21:58:50,810][105620] Updated weights for policy 1, policy_version 908852 (0.0009) [2023-12-26 21:58:50,866][105620] Updated weights for policy 1, policy_version 908862 (0.0006) [2023-12-26 21:58:50,930][105620] Updated weights for policy 1, policy_version 908872 (0.0007) [2023-12-26 21:58:51,023][105692] Updated weights for policy 0, policy_version 909016 (0.0009) [2023-12-26 21:58:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 465444864. Throughput: 0: 9991.3, 1: 9337.6. Samples: 465435368. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 21:58:51,063][104569] Avg episode reward: [(0, '9078.744'), (1, '8813.175')] [2023-12-26 21:58:51,082][105692] Updated weights for policy 0, policy_version 909026 (0.0009) [2023-12-26 21:58:51,147][105692] Updated weights for policy 0, policy_version 909036 (0.0009) [2023-12-26 21:58:51,534][105620] Updated weights for policy 1, policy_version 908882 (0.0006) [2023-12-26 21:58:51,597][105620] Updated weights for policy 1, policy_version 908892 (0.0005) [2023-12-26 21:58:51,670][105620] Updated weights for policy 1, policy_version 908902 (0.0007) [2023-12-26 21:58:51,733][105620] Updated weights for policy 1, policy_version 908912 (0.0009) [2023-12-26 21:58:52,045][105692] Updated weights for policy 0, policy_version 909046 (0.0009) [2023-12-26 21:58:52,095][105692] Updated weights for policy 0, policy_version 909056 (0.0009) [2023-12-26 21:58:52,148][105692] Updated weights for policy 0, policy_version 909066 (0.0009) [2023-12-26 21:58:52,316][105620] Updated weights for policy 1, policy_version 908922 (0.0008) [2023-12-26 21:58:52,382][105620] Updated weights for policy 1, policy_version 908932 (0.0011) [2023-12-26 21:58:52,444][105620] Updated weights for policy 1, policy_version 908942 (0.0010) [2023-12-26 21:58:52,901][105692] Updated weights for policy 0, policy_version 909076 (0.0008) [2023-12-26 21:58:52,957][105692] Updated weights for policy 0, policy_version 909086 (0.0005) [2023-12-26 21:58:53,023][105692] Updated weights for policy 0, policy_version 909096 (0.0010) [2023-12-26 21:58:53,133][105620] Updated weights for policy 1, policy_version 908952 (0.0009) [2023-12-26 21:58:53,192][105620] Updated weights for policy 1, policy_version 908962 (0.0008) [2023-12-26 21:58:53,253][105620] Updated weights for policy 1, policy_version 908972 (0.0008) [2023-12-26 21:58:53,732][105692] Updated weights for policy 0, policy_version 909106 (0.0010) [2023-12-26 21:58:53,798][105692] Updated weights for policy 0, policy_version 909116 (0.0006) [2023-12-26 21:58:53,869][105692] Updated weights for policy 0, policy_version 909126 (0.0005) [2023-12-26 21:58:53,924][105620] Updated weights for policy 1, policy_version 908982 (0.0009) [2023-12-26 21:58:53,926][105692] Updated weights for policy 0, policy_version 909136 (0.0006) [2023-12-26 21:58:53,993][105620] Updated weights for policy 1, policy_version 908992 (0.0011) [2023-12-26 21:58:54,058][105620] Updated weights for policy 1, policy_version 909002 (0.0009) [2023-12-26 21:58:54,474][105692] Updated weights for policy 0, policy_version 909146 (0.0010) [2023-12-26 21:58:54,532][105692] Updated weights for policy 0, policy_version 909156 (0.0008) [2023-12-26 21:58:54,588][105692] Updated weights for policy 0, policy_version 909166 (0.0008) [2023-12-26 21:58:54,654][105620] Updated weights for policy 1, policy_version 909012 (0.0007) [2023-12-26 21:58:54,718][105620] Updated weights for policy 1, policy_version 909022 (0.0009) [2023-12-26 21:58:54,780][105620] Updated weights for policy 1, policy_version 909032 (0.0010) [2023-12-26 21:58:55,294][105692] Updated weights for policy 0, policy_version 909176 (0.0006) [2023-12-26 21:58:55,344][105692] Updated weights for policy 0, policy_version 909186 (0.0006) [2023-12-26 21:58:55,394][105620] Updated weights for policy 1, policy_version 909042 (0.0010) [2023-12-26 21:58:55,399][105692] Updated weights for policy 0, policy_version 909196 (0.0008) [2023-12-26 21:58:55,453][105620] Updated weights for policy 1, policy_version 909052 (0.0011) [2023-12-26 21:58:55,519][105620] Updated weights for policy 1, policy_version 909062 (0.0011) [2023-12-26 21:58:55,585][105620] Updated weights for policy 1, policy_version 909072 (0.0010) [2023-12-26 21:58:56,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 465543168. Throughput: 0: 9941.8, 1: 9511.6. Samples: 465554612. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 21:58:56,062][104569] Avg episode reward: [(0, '8987.162'), (1, '9087.322')] [2023-12-26 21:58:56,097][105692] Updated weights for policy 0, policy_version 909206 (0.0005) [2023-12-26 21:58:56,157][105692] Updated weights for policy 0, policy_version 909216 (0.0006) [2023-12-26 21:58:56,222][105692] Updated weights for policy 0, policy_version 909226 (0.0006) [2023-12-26 21:58:56,311][105620] Updated weights for policy 1, policy_version 909082 (0.0010) [2023-12-26 21:58:56,357][105620] Updated weights for policy 1, policy_version 909092 (0.0005) [2023-12-26 21:58:56,406][105620] Updated weights for policy 1, policy_version 909102 (0.0005) [2023-12-26 21:58:56,915][105692] Updated weights for policy 0, policy_version 909236 (0.0007) [2023-12-26 21:58:56,959][105692] Updated weights for policy 0, policy_version 909246 (0.0007) [2023-12-26 21:58:57,003][105692] Updated weights for policy 0, policy_version 909256 (0.0008) [2023-12-26 21:58:57,127][105620] Updated weights for policy 1, policy_version 909112 (0.0009) [2023-12-26 21:58:57,171][105620] Updated weights for policy 1, policy_version 909122 (0.0010) [2023-12-26 21:58:57,222][105620] Updated weights for policy 1, policy_version 909132 (0.0010) [2023-12-26 21:58:57,778][105692] Updated weights for policy 0, policy_version 909266 (0.0007) [2023-12-26 21:58:57,825][105692] Updated weights for policy 0, policy_version 909276 (0.0007) [2023-12-26 21:58:57,868][105620] Updated weights for policy 1, policy_version 909142 (0.0007) [2023-12-26 21:58:57,879][105692] Updated weights for policy 0, policy_version 909286 (0.0008) [2023-12-26 21:58:57,919][105620] Updated weights for policy 1, policy_version 909152 (0.0010) [2023-12-26 21:58:57,934][105692] Updated weights for policy 0, policy_version 909296 (0.0007) [2023-12-26 21:58:57,964][105620] Updated weights for policy 1, policy_version 909162 (0.0010) [2023-12-26 21:58:58,672][105692] Updated weights for policy 0, policy_version 909306 (0.0008) [2023-12-26 21:58:58,736][105692] Updated weights for policy 0, policy_version 909316 (0.0007) [2023-12-26 21:58:58,760][105620] Updated weights for policy 1, policy_version 909172 (0.0009) [2023-12-26 21:58:58,805][105692] Updated weights for policy 0, policy_version 909326 (0.0007) [2023-12-26 21:58:58,832][105620] Updated weights for policy 1, policy_version 909182 (0.0007) [2023-12-26 21:58:58,904][105620] Updated weights for policy 1, policy_version 909192 (0.0007) [2023-12-26 21:58:59,582][105692] Updated weights for policy 0, policy_version 909336 (0.0008) [2023-12-26 21:58:59,642][105692] Updated weights for policy 0, policy_version 909346 (0.0008) [2023-12-26 21:58:59,704][105692] Updated weights for policy 0, policy_version 909356 (0.0007) [2023-12-26 21:58:59,719][105620] Updated weights for policy 1, policy_version 909202 (0.0008) [2023-12-26 21:58:59,781][105620] Updated weights for policy 1, policy_version 909212 (0.0009) [2023-12-26 21:58:59,847][105620] Updated weights for policy 1, policy_version 909222 (0.0011) [2023-12-26 21:58:59,904][105620] Updated weights for policy 1, policy_version 909232 (0.0010) [2023-12-26 21:59:00,366][105692] Updated weights for policy 0, policy_version 909366 (0.0006) [2023-12-26 21:59:00,429][105692] Updated weights for policy 0, policy_version 909376 (0.0006) [2023-12-26 21:59:00,496][105692] Updated weights for policy 0, policy_version 909386 (0.0008) [2023-12-26 21:59:00,617][105620] Updated weights for policy 1, policy_version 909242 (0.0008) [2023-12-26 21:59:00,671][105620] Updated weights for policy 1, policy_version 909252 (0.0009) [2023-12-26 21:59:00,725][105620] Updated weights for policy 1, policy_version 909262 (0.0010) [2023-12-26 21:59:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 465641472. Throughput: 0: 9895.5, 1: 9593.9. Samples: 465613016. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 21:59:01,063][104569] Avg episode reward: [(0, '9075.874'), (1, '8910.002')] [2023-12-26 21:59:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000909264_232800256.pth... [2023-12-26 21:59:01,074][105692] Updated weights for policy 0, policy_version 909396 (0.0008) [2023-12-26 21:59:01,093][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000908144_232513536.pth [2023-12-26 21:59:01,138][105692] Updated weights for policy 0, policy_version 909406 (0.0009) [2023-12-26 21:59:01,199][105692] Updated weights for policy 0, policy_version 909416 (0.0009) [2023-12-26 21:59:01,239][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000909424_232849408.pth... [2023-12-26 21:59:01,242][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000908272_232554496.pth [2023-12-26 21:59:01,398][105620] Updated weights for policy 1, policy_version 909272 (0.0007) [2023-12-26 21:59:01,457][105620] Updated weights for policy 1, policy_version 909282 (0.0006) [2023-12-26 21:59:01,515][105620] Updated weights for policy 1, policy_version 909292 (0.0005) [2023-12-26 21:59:02,064][105692] Updated weights for policy 0, policy_version 909426 (0.0008) [2023-12-26 21:59:02,084][105620] Updated weights for policy 1, policy_version 909302 (0.0006) [2023-12-26 21:59:02,121][105692] Updated weights for policy 0, policy_version 909436 (0.0010) [2023-12-26 21:59:02,137][105620] Updated weights for policy 1, policy_version 909312 (0.0005) [2023-12-26 21:59:02,169][105692] Updated weights for policy 0, policy_version 909446 (0.0007) [2023-12-26 21:59:02,201][105620] Updated weights for policy 1, policy_version 909322 (0.0005) [2023-12-26 21:59:02,220][105692] Updated weights for policy 0, policy_version 909456 (0.0005) [2023-12-26 21:59:02,815][105620] Updated weights for policy 1, policy_version 909332 (0.0007) [2023-12-26 21:59:02,866][105620] Updated weights for policy 1, policy_version 909343 (0.0009) [2023-12-26 21:59:02,911][105620] Updated weights for policy 1, policy_version 909353 (0.0008) [2023-12-26 21:59:03,000][105692] Updated weights for policy 0, policy_version 909466 (0.0009) [2023-12-26 21:59:03,050][105692] Updated weights for policy 0, policy_version 909476 (0.0009) [2023-12-26 21:59:03,096][105692] Updated weights for policy 0, policy_version 909486 (0.0008) [2023-12-26 21:59:03,670][105620] Updated weights for policy 1, policy_version 909363 (0.0009) [2023-12-26 21:59:03,720][105620] Updated weights for policy 1, policy_version 909373 (0.0009) [2023-12-26 21:59:03,774][105620] Updated weights for policy 1, policy_version 909383 (0.0008) [2023-12-26 21:59:03,881][105692] Updated weights for policy 0, policy_version 909496 (0.0008) [2023-12-26 21:59:03,947][105692] Updated weights for policy 0, policy_version 909506 (0.0009) [2023-12-26 21:59:04,007][105692] Updated weights for policy 0, policy_version 909516 (0.0009) [2023-12-26 21:59:04,579][105620] Updated weights for policy 1, policy_version 909393 (0.0006) [2023-12-26 21:59:04,638][105620] Updated weights for policy 1, policy_version 909403 (0.0009) [2023-12-26 21:59:04,688][105620] Updated weights for policy 1, policy_version 909413 (0.0008) [2023-12-26 21:59:04,735][105620] Updated weights for policy 1, policy_version 909423 (0.0009) [2023-12-26 21:59:04,797][105692] Updated weights for policy 0, policy_version 909526 (0.0009) [2023-12-26 21:59:04,851][105692] Updated weights for policy 0, policy_version 909536 (0.0008) [2023-12-26 21:59:04,905][105692] Updated weights for policy 0, policy_version 909546 (0.0009) [2023-12-26 21:59:05,512][105620] Updated weights for policy 1, policy_version 909433 (0.0009) [2023-12-26 21:59:05,566][105620] Updated weights for policy 1, policy_version 909443 (0.0008) [2023-12-26 21:59:05,622][105620] Updated weights for policy 1, policy_version 909453 (0.0010) [2023-12-26 21:59:05,680][105692] Updated weights for policy 0, policy_version 909556 (0.0009) [2023-12-26 21:59:05,741][105692] Updated weights for policy 0, policy_version 909566 (0.0008) [2023-12-26 21:59:05,808][105692] Updated weights for policy 0, policy_version 909576 (0.0010) [2023-12-26 21:59:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 465739776. Throughput: 0: 9773.4, 1: 9656.7. Samples: 465728592. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 21:59:06,062][104569] Avg episode reward: [(0, '9164.903'), (1, '8648.272')] [2023-12-26 21:59:06,320][105620] Updated weights for policy 1, policy_version 909463 (0.0011) [2023-12-26 21:59:06,383][105620] Updated weights for policy 1, policy_version 909473 (0.0010) [2023-12-26 21:59:06,441][105620] Updated weights for policy 1, policy_version 909483 (0.0010) [2023-12-26 21:59:06,597][105692] Updated weights for policy 0, policy_version 909586 (0.0009) [2023-12-26 21:59:06,661][105692] Updated weights for policy 0, policy_version 909596 (0.0008) [2023-12-26 21:59:06,726][105692] Updated weights for policy 0, policy_version 909606 (0.0009) [2023-12-26 21:59:06,784][105692] Updated weights for policy 0, policy_version 909616 (0.0010) [2023-12-26 21:59:07,064][105620] Updated weights for policy 1, policy_version 909493 (0.0008) [2023-12-26 21:59:07,130][105620] Updated weights for policy 1, policy_version 909503 (0.0009) [2023-12-26 21:59:07,190][105620] Updated weights for policy 1, policy_version 909513 (0.0008) [2023-12-26 21:59:07,523][105692] Updated weights for policy 0, policy_version 909626 (0.0005) [2023-12-26 21:59:07,584][105692] Updated weights for policy 0, policy_version 909636 (0.0005) [2023-12-26 21:59:07,653][105692] Updated weights for policy 0, policy_version 909646 (0.0005) [2023-12-26 21:59:07,792][105620] Updated weights for policy 1, policy_version 909523 (0.0006) [2023-12-26 21:59:07,847][105620] Updated weights for policy 1, policy_version 909533 (0.0005) [2023-12-26 21:59:07,905][105620] Updated weights for policy 1, policy_version 909543 (0.0005) [2023-12-26 21:59:08,297][105692] Updated weights for policy 0, policy_version 909656 (0.0008) [2023-12-26 21:59:08,356][105692] Updated weights for policy 0, policy_version 909666 (0.0009) [2023-12-26 21:59:08,422][105692] Updated weights for policy 0, policy_version 909676 (0.0009) [2023-12-26 21:59:08,510][105620] Updated weights for policy 1, policy_version 909553 (0.0005) [2023-12-26 21:59:08,569][105620] Updated weights for policy 1, policy_version 909563 (0.0009) [2023-12-26 21:59:08,587][105586] KL-divergence is very high: 139.2449 [2023-12-26 21:59:08,632][105620] Updated weights for policy 1, policy_version 909573 (0.0010) [2023-12-26 21:59:08,638][105586] KL-divergence is very high: 220.0489 [2023-12-26 21:59:08,692][105586] KL-divergence is very high: 208.7154 [2023-12-26 21:59:08,696][105620] Updated weights for policy 1, policy_version 909583 (0.0009) [2023-12-26 21:59:09,191][105692] Updated weights for policy 0, policy_version 909686 (0.0007) [2023-12-26 21:59:09,254][105692] Updated weights for policy 0, policy_version 909696 (0.0011) [2023-12-26 21:59:09,310][105692] Updated weights for policy 0, policy_version 909706 (0.0011) [2023-12-26 21:59:09,371][105620] Updated weights for policy 1, policy_version 909593 (0.0007) [2023-12-26 21:59:09,443][105620] Updated weights for policy 1, policy_version 909603 (0.0009) [2023-12-26 21:59:09,500][105620] Updated weights for policy 1, policy_version 909613 (0.0010) [2023-12-26 21:59:10,031][105692] Updated weights for policy 0, policy_version 909716 (0.0009) [2023-12-26 21:59:10,098][105692] Updated weights for policy 0, policy_version 909726 (0.0006) [2023-12-26 21:59:10,160][105692] Updated weights for policy 0, policy_version 909736 (0.0010) [2023-12-26 21:59:10,303][105620] Updated weights for policy 1, policy_version 909623 (0.0008) [2023-12-26 21:59:10,362][105620] Updated weights for policy 1, policy_version 909633 (0.0007) [2023-12-26 21:59:10,423][105620] Updated weights for policy 1, policy_version 909643 (0.0006) [2023-12-26 21:59:10,852][105692] Updated weights for policy 0, policy_version 909746 (0.0011) [2023-12-26 21:59:10,911][105692] Updated weights for policy 0, policy_version 909756 (0.0010) [2023-12-26 21:59:10,977][105692] Updated weights for policy 0, policy_version 909766 (0.0011) [2023-12-26 21:59:11,032][105692] Updated weights for policy 0, policy_version 909776 (0.0010) [2023-12-26 21:59:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 465838080. Throughput: 0: 9751.7, 1: 9793.8. Samples: 465845868. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 21:59:11,063][104569] Avg episode reward: [(0, '9347.885'), (1, '8561.812')] [2023-12-26 21:59:11,146][105620] Updated weights for policy 1, policy_version 909653 (0.0009) [2023-12-26 21:59:11,208][105620] Updated weights for policy 1, policy_version 909663 (0.0009) [2023-12-26 21:59:11,265][105620] Updated weights for policy 1, policy_version 909673 (0.0006) [2023-12-26 21:59:11,823][105692] Updated weights for policy 0, policy_version 909786 (0.0009) [2023-12-26 21:59:11,883][105692] Updated weights for policy 0, policy_version 909796 (0.0009) [2023-12-26 21:59:11,941][105692] Updated weights for policy 0, policy_version 909806 (0.0009) [2023-12-26 21:59:11,981][105620] Updated weights for policy 1, policy_version 909683 (0.0006) [2023-12-26 21:59:12,050][105620] Updated weights for policy 1, policy_version 909693 (0.0005) [2023-12-26 21:59:12,116][105620] Updated weights for policy 1, policy_version 909703 (0.0006) [2023-12-26 21:59:12,767][105620] Updated weights for policy 1, policy_version 909713 (0.0008) [2023-12-26 21:59:12,786][105692] Updated weights for policy 0, policy_version 909816 (0.0010) [2023-12-26 21:59:12,820][105620] Updated weights for policy 1, policy_version 909723 (0.0006) [2023-12-26 21:59:12,845][105692] Updated weights for policy 0, policy_version 909826 (0.0010) [2023-12-26 21:59:12,871][105620] Updated weights for policy 1, policy_version 909733 (0.0006) [2023-12-26 21:59:12,900][105692] Updated weights for policy 0, policy_version 909836 (0.0009) [2023-12-26 21:59:12,921][105620] Updated weights for policy 1, policy_version 909743 (0.0007) [2023-12-26 21:59:13,557][105692] Updated weights for policy 0, policy_version 909846 (0.0007) [2023-12-26 21:59:13,621][105692] Updated weights for policy 0, policy_version 909856 (0.0006) [2023-12-26 21:59:13,686][105692] Updated weights for policy 0, policy_version 909866 (0.0006) [2023-12-26 21:59:13,693][105620] Updated weights for policy 1, policy_version 909753 (0.0006) [2023-12-26 21:59:13,757][105620] Updated weights for policy 1, policy_version 909763 (0.0005) [2023-12-26 21:59:13,809][105620] Updated weights for policy 1, policy_version 909773 (0.0005) [2023-12-26 21:59:14,312][105692] Updated weights for policy 0, policy_version 909876 (0.0009) [2023-12-26 21:59:14,341][105620] Updated weights for policy 1, policy_version 909783 (0.0005) [2023-12-26 21:59:14,369][105692] Updated weights for policy 0, policy_version 909886 (0.0005) [2023-12-26 21:59:14,395][105620] Updated weights for policy 1, policy_version 909793 (0.0006) [2023-12-26 21:59:14,436][105692] Updated weights for policy 0, policy_version 909896 (0.0005) [2023-12-26 21:59:14,460][105620] Updated weights for policy 1, policy_version 909803 (0.0005) [2023-12-26 21:59:15,111][105692] Updated weights for policy 0, policy_version 909906 (0.0007) [2023-12-26 21:59:15,123][105620] Updated weights for policy 1, policy_version 909813 (0.0007) [2023-12-26 21:59:15,169][105692] Updated weights for policy 0, policy_version 909916 (0.0011) [2023-12-26 21:59:15,179][105620] Updated weights for policy 1, policy_version 909823 (0.0008) [2023-12-26 21:59:15,229][105692] Updated weights for policy 0, policy_version 909926 (0.0011) [2023-12-26 21:59:15,239][105620] Updated weights for policy 1, policy_version 909833 (0.0006) [2023-12-26 21:59:15,296][105692] Updated weights for policy 0, policy_version 909936 (0.0010) [2023-12-26 21:59:15,980][105692] Updated weights for policy 0, policy_version 909946 (0.0010) [2023-12-26 21:59:16,034][105620] Updated weights for policy 1, policy_version 909843 (0.0005) [2023-12-26 21:59:16,035][105692] Updated weights for policy 0, policy_version 909956 (0.0010) [2023-12-26 21:59:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 465928192. Throughput: 0: 9624.2, 1: 9828.9. Samples: 465903296. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 21:59:16,063][104569] Avg episode reward: [(0, '9076.220'), (1, '8729.629')] [2023-12-26 21:59:16,088][105620] Updated weights for policy 1, policy_version 909853 (0.0005) [2023-12-26 21:59:16,090][105692] Updated weights for policy 0, policy_version 909966 (0.0011) [2023-12-26 21:59:16,102][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000909968_232988672.pth... [2023-12-26 21:59:16,105][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000908816_232693760.pth [2023-12-26 21:59:16,140][105620] Updated weights for policy 1, policy_version 909863 (0.0006) [2023-12-26 21:59:16,194][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000909872_232955904.pth... [2023-12-26 21:59:16,197][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000908688_232652800.pth [2023-12-26 21:59:16,781][105620] Updated weights for policy 1, policy_version 909873 (0.0006) [2023-12-26 21:59:16,828][105620] Updated weights for policy 1, policy_version 909883 (0.0008) [2023-12-26 21:59:16,837][105692] Updated weights for policy 0, policy_version 909976 (0.0011) [2023-12-26 21:59:16,875][105620] Updated weights for policy 1, policy_version 909893 (0.0006) [2023-12-26 21:59:16,899][105692] Updated weights for policy 0, policy_version 909986 (0.0011) [2023-12-26 21:59:16,919][105620] Updated weights for policy 1, policy_version 909903 (0.0005) [2023-12-26 21:59:16,963][105692] Updated weights for policy 0, policy_version 909996 (0.0009) [2023-12-26 21:59:17,524][105692] Updated weights for policy 0, policy_version 910006 (0.0005) [2023-12-26 21:59:17,580][105692] Updated weights for policy 0, policy_version 910016 (0.0007) [2023-12-26 21:59:17,636][105692] Updated weights for policy 0, policy_version 910026 (0.0005) [2023-12-26 21:59:17,652][105620] Updated weights for policy 1, policy_version 909913 (0.0008) [2023-12-26 21:59:17,719][105620] Updated weights for policy 1, policy_version 909923 (0.0007) [2023-12-26 21:59:17,787][105620] Updated weights for policy 1, policy_version 909933 (0.0009) [2023-12-26 21:59:18,246][105692] Updated weights for policy 0, policy_version 910036 (0.0006) [2023-12-26 21:59:18,300][105692] Updated weights for policy 0, policy_version 910046 (0.0005) [2023-12-26 21:59:18,361][105692] Updated weights for policy 0, policy_version 910056 (0.0008) [2023-12-26 21:59:18,519][105620] Updated weights for policy 1, policy_version 909943 (0.0008) [2023-12-26 21:59:18,582][105620] Updated weights for policy 1, policy_version 909953 (0.0005) [2023-12-26 21:59:18,644][105620] Updated weights for policy 1, policy_version 909963 (0.0009) [2023-12-26 21:59:19,083][105692] Updated weights for policy 0, policy_version 910066 (0.0007) [2023-12-26 21:59:19,141][105692] Updated weights for policy 0, policy_version 910076 (0.0007) [2023-12-26 21:59:19,206][105692] Updated weights for policy 0, policy_version 910086 (0.0011) [2023-12-26 21:59:19,279][105692] Updated weights for policy 0, policy_version 910096 (0.0012) [2023-12-26 21:59:19,358][105620] Updated weights for policy 1, policy_version 909973 (0.0010) [2023-12-26 21:59:19,409][105620] Updated weights for policy 1, policy_version 909983 (0.0011) [2023-12-26 21:59:19,468][105620] Updated weights for policy 1, policy_version 909993 (0.0009) [2023-12-26 21:59:20,051][105692] Updated weights for policy 0, policy_version 910106 (0.0011) [2023-12-26 21:59:20,111][105692] Updated weights for policy 0, policy_version 910116 (0.0011) [2023-12-26 21:59:20,171][105692] Updated weights for policy 0, policy_version 910126 (0.0011) [2023-12-26 21:59:20,245][105620] Updated weights for policy 1, policy_version 910003 (0.0010) [2023-12-26 21:59:20,304][105620] Updated weights for policy 1, policy_version 910013 (0.0008) [2023-12-26 21:59:20,364][105620] Updated weights for policy 1, policy_version 910023 (0.0008) [2023-12-26 21:59:20,942][105692] Updated weights for policy 0, policy_version 910136 (0.0011) [2023-12-26 21:59:21,000][105692] Updated weights for policy 0, policy_version 910146 (0.0011) [2023-12-26 21:59:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 466026496. Throughput: 0: 9747.7, 1: 9845.2. Samples: 466024248. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 21:59:21,063][104569] Avg episode reward: [(0, '8897.229'), (1, '9001.128')] [2023-12-26 21:59:21,069][105692] Updated weights for policy 0, policy_version 910156 (0.0011) [2023-12-26 21:59:21,186][105620] Updated weights for policy 1, policy_version 910033 (0.0008) [2023-12-26 21:59:21,241][105620] Updated weights for policy 1, policy_version 910043 (0.0008) [2023-12-26 21:59:21,307][105620] Updated weights for policy 1, policy_version 910053 (0.0008) [2023-12-26 21:59:21,372][105620] Updated weights for policy 1, policy_version 910063 (0.0008) [2023-12-26 21:59:21,898][105692] Updated weights for policy 0, policy_version 910166 (0.0009) [2023-12-26 21:59:21,952][105692] Updated weights for policy 0, policy_version 910176 (0.0008) [2023-12-26 21:59:22,013][105692] Updated weights for policy 0, policy_version 910186 (0.0008) [2023-12-26 21:59:22,140][105620] Updated weights for policy 1, policy_version 910073 (0.0010) [2023-12-26 21:59:22,200][105620] Updated weights for policy 1, policy_version 910083 (0.0010) [2023-12-26 21:59:22,260][105620] Updated weights for policy 1, policy_version 910093 (0.0010) [2023-12-26 21:59:22,693][105692] Updated weights for policy 0, policy_version 910196 (0.0008) [2023-12-26 21:59:22,737][105692] Updated weights for policy 0, policy_version 910206 (0.0008) [2023-12-26 21:59:22,791][105692] Updated weights for policy 0, policy_version 910216 (0.0009) [2023-12-26 21:59:22,966][105620] Updated weights for policy 1, policy_version 910103 (0.0009) [2023-12-26 21:59:23,029][105620] Updated weights for policy 1, policy_version 910113 (0.0008) [2023-12-26 21:59:23,084][105620] Updated weights for policy 1, policy_version 910123 (0.0009) [2023-12-26 21:59:23,606][105692] Updated weights for policy 0, policy_version 910226 (0.0008) [2023-12-26 21:59:23,662][105692] Updated weights for policy 0, policy_version 910236 (0.0005) [2023-12-26 21:59:23,718][105692] Updated weights for policy 0, policy_version 910246 (0.0006) [2023-12-26 21:59:23,756][105620] Updated weights for policy 1, policy_version 910133 (0.0009) [2023-12-26 21:59:23,776][105692] Updated weights for policy 0, policy_version 910256 (0.0008) [2023-12-26 21:59:23,810][105620] Updated weights for policy 1, policy_version 910143 (0.0007) [2023-12-26 21:59:23,872][105620] Updated weights for policy 1, policy_version 910153 (0.0009) [2023-12-26 21:59:24,500][105692] Updated weights for policy 0, policy_version 910266 (0.0009) [2023-12-26 21:59:24,551][105692] Updated weights for policy 0, policy_version 910276 (0.0009) [2023-12-26 21:59:24,602][105692] Updated weights for policy 0, policy_version 910286 (0.0008) [2023-12-26 21:59:24,622][105620] Updated weights for policy 1, policy_version 910163 (0.0008) [2023-12-26 21:59:24,679][105620] Updated weights for policy 1, policy_version 910173 (0.0008) [2023-12-26 21:59:24,733][105620] Updated weights for policy 1, policy_version 910183 (0.0010) [2023-12-26 21:59:25,396][105692] Updated weights for policy 0, policy_version 910296 (0.0009) [2023-12-26 21:59:25,454][105692] Updated weights for policy 0, policy_version 910306 (0.0008) [2023-12-26 21:59:25,461][105620] Updated weights for policy 1, policy_version 910193 (0.0010) [2023-12-26 21:59:25,500][105692] Updated weights for policy 0, policy_version 910316 (0.0007) [2023-12-26 21:59:25,515][105620] Updated weights for policy 1, policy_version 910203 (0.0007) [2023-12-26 21:59:25,578][105620] Updated weights for policy 1, policy_version 910213 (0.0009) [2023-12-26 21:59:25,636][105620] Updated weights for policy 1, policy_version 910223 (0.0008) [2023-12-26 21:59:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 466124800. Throughput: 0: 9628.4, 1: 9874.3. Samples: 466135532. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 21:59:26,062][104569] Avg episode reward: [(0, '9169.353'), (1, '9183.051')] [2023-12-26 21:59:26,254][105692] Updated weights for policy 0, policy_version 910326 (0.0008) [2023-12-26 21:59:26,307][105692] Updated weights for policy 0, policy_version 910336 (0.0008) [2023-12-26 21:59:26,356][105692] Updated weights for policy 0, policy_version 910346 (0.0006) [2023-12-26 21:59:26,395][105620] Updated weights for policy 1, policy_version 910233 (0.0010) [2023-12-26 21:59:26,461][105620] Updated weights for policy 1, policy_version 910243 (0.0010) [2023-12-26 21:59:26,523][105620] Updated weights for policy 1, policy_version 910253 (0.0010) [2023-12-26 21:59:26,988][105692] Updated weights for policy 0, policy_version 910356 (0.0007) [2023-12-26 21:59:27,055][105692] Updated weights for policy 0, policy_version 910366 (0.0005) [2023-12-26 21:59:27,107][105692] Updated weights for policy 0, policy_version 910376 (0.0005) [2023-12-26 21:59:27,135][105620] Updated weights for policy 1, policy_version 910263 (0.0007) [2023-12-26 21:59:27,186][105620] Updated weights for policy 1, policy_version 910273 (0.0006) [2023-12-26 21:59:27,232][105620] Updated weights for policy 1, policy_version 910283 (0.0005) [2023-12-26 21:59:27,795][105692] Updated weights for policy 0, policy_version 910386 (0.0007) [2023-12-26 21:59:27,847][105620] Updated weights for policy 1, policy_version 910293 (0.0008) [2023-12-26 21:59:27,848][105692] Updated weights for policy 0, policy_version 910396 (0.0007) [2023-12-26 21:59:27,901][105692] Updated weights for policy 0, policy_version 910406 (0.0005) [2023-12-26 21:59:27,915][105620] Updated weights for policy 1, policy_version 910303 (0.0010) [2023-12-26 21:59:27,956][105692] Updated weights for policy 0, policy_version 910416 (0.0005) [2023-12-26 21:59:27,976][105620] Updated weights for policy 1, policy_version 910313 (0.0010) [2023-12-26 21:59:28,626][105692] Updated weights for policy 0, policy_version 910426 (0.0011) [2023-12-26 21:59:28,688][105692] Updated weights for policy 0, policy_version 910436 (0.0011) [2023-12-26 21:59:28,703][105620] Updated weights for policy 1, policy_version 910323 (0.0010) [2023-12-26 21:59:28,749][105692] Updated weights for policy 0, policy_version 910446 (0.0011) [2023-12-26 21:59:28,758][105620] Updated weights for policy 1, policy_version 910333 (0.0010) [2023-12-26 21:59:28,809][105620] Updated weights for policy 1, policy_version 910343 (0.0010) [2023-12-26 21:59:29,418][105692] Updated weights for policy 0, policy_version 910456 (0.0011) [2023-12-26 21:59:29,476][105692] Updated weights for policy 0, policy_version 910466 (0.0010) [2023-12-26 21:59:29,526][105692] Updated weights for policy 0, policy_version 910476 (0.0007) [2023-12-26 21:59:29,591][105620] Updated weights for policy 1, policy_version 910353 (0.0010) [2023-12-26 21:59:29,647][105620] Updated weights for policy 1, policy_version 910363 (0.0010) [2023-12-26 21:59:29,697][105620] Updated weights for policy 1, policy_version 910373 (0.0009) [2023-12-26 21:59:29,756][105620] Updated weights for policy 1, policy_version 910383 (0.0006) [2023-12-26 21:59:30,134][105692] Updated weights for policy 0, policy_version 910486 (0.0008) [2023-12-26 21:59:30,186][105692] Updated weights for policy 0, policy_version 910496 (0.0011) [2023-12-26 21:59:30,234][105692] Updated weights for policy 0, policy_version 910506 (0.0011) [2023-12-26 21:59:30,380][105620] Updated weights for policy 1, policy_version 910393 (0.0006) [2023-12-26 21:59:30,441][105620] Updated weights for policy 1, policy_version 910403 (0.0006) [2023-12-26 21:59:30,485][105620] Updated weights for policy 1, policy_version 910413 (0.0005) [2023-12-26 21:59:31,000][105692] Updated weights for policy 0, policy_version 910516 (0.0010) [2023-12-26 21:59:31,059][105692] Updated weights for policy 0, policy_version 910526 (0.0012) [2023-12-26 21:59:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 466223104. Throughput: 0: 9686.1, 1: 9910.4. Samples: 466197196. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 21:59:31,063][104569] Avg episode reward: [(0, '9076.050'), (1, '8907.161')] [2023-12-26 21:59:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000910416_233095168.pth... [2023-12-26 21:59:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000909264_232800256.pth [2023-12-26 21:59:31,118][105692] Updated weights for policy 0, policy_version 910536 (0.0011) [2023-12-26 21:59:31,161][105620] Updated weights for policy 1, policy_version 910423 (0.0006) [2023-12-26 21:59:31,166][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000910544_233136128.pth... [2023-12-26 21:59:31,172][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000909424_232849408.pth [2023-12-26 21:59:31,215][105620] Updated weights for policy 1, policy_version 910433 (0.0007) [2023-12-26 21:59:31,269][105620] Updated weights for policy 1, policy_version 910443 (0.0006) [2023-12-26 21:59:31,913][105692] Updated weights for policy 0, policy_version 910546 (0.0010) [2023-12-26 21:59:31,974][105692] Updated weights for policy 0, policy_version 910556 (0.0006) [2023-12-26 21:59:31,986][105620] Updated weights for policy 1, policy_version 910453 (0.0006) [2023-12-26 21:59:32,045][105620] Updated weights for policy 1, policy_version 910463 (0.0006) [2023-12-26 21:59:32,056][105692] Updated weights for policy 0, policy_version 910566 (0.0006) [2023-12-26 21:59:32,095][105620] Updated weights for policy 1, policy_version 910473 (0.0006) [2023-12-26 21:59:32,111][105692] Updated weights for policy 0, policy_version 910576 (0.0007) [2023-12-26 21:59:32,721][105692] Updated weights for policy 0, policy_version 910586 (0.0006) [2023-12-26 21:59:32,782][105692] Updated weights for policy 0, policy_version 910596 (0.0009) [2023-12-26 21:59:32,839][105692] Updated weights for policy 0, policy_version 910606 (0.0009) [2023-12-26 21:59:32,878][105620] Updated weights for policy 1, policy_version 910483 (0.0009) [2023-12-26 21:59:32,943][105620] Updated weights for policy 1, policy_version 910493 (0.0007) [2023-12-26 21:59:33,002][105620] Updated weights for policy 1, policy_version 910503 (0.0008) [2023-12-26 21:59:33,485][105692] Updated weights for policy 0, policy_version 910616 (0.0008) [2023-12-26 21:59:33,530][105692] Updated weights for policy 0, policy_version 910626 (0.0008) [2023-12-26 21:59:33,574][105692] Updated weights for policy 0, policy_version 910636 (0.0008) [2023-12-26 21:59:33,613][105620] Updated weights for policy 1, policy_version 910513 (0.0007) [2023-12-26 21:59:33,661][105620] Updated weights for policy 1, policy_version 910523 (0.0010) [2023-12-26 21:59:33,715][105620] Updated weights for policy 1, policy_version 910533 (0.0010) [2023-12-26 21:59:33,758][105620] Updated weights for policy 1, policy_version 910543 (0.0010) [2023-12-26 21:59:34,345][105692] Updated weights for policy 0, policy_version 910646 (0.0008) [2023-12-26 21:59:34,405][105692] Updated weights for policy 0, policy_version 910656 (0.0008) [2023-12-26 21:59:34,469][105692] Updated weights for policy 0, policy_version 910666 (0.0008) [2023-12-26 21:59:34,522][105620] Updated weights for policy 1, policy_version 910553 (0.0008) [2023-12-26 21:59:34,577][105620] Updated weights for policy 1, policy_version 910563 (0.0008) [2023-12-26 21:59:34,640][105620] Updated weights for policy 1, policy_version 910573 (0.0008) [2023-12-26 21:59:35,151][105692] Updated weights for policy 0, policy_version 910676 (0.0006) [2023-12-26 21:59:35,214][105692] Updated weights for policy 0, policy_version 910686 (0.0006) [2023-12-26 21:59:35,273][105692] Updated weights for policy 0, policy_version 910696 (0.0009) [2023-12-26 21:59:35,409][105620] Updated weights for policy 1, policy_version 910583 (0.0009) [2023-12-26 21:59:35,466][105620] Updated weights for policy 1, policy_version 910593 (0.0009) [2023-12-26 21:59:35,520][105620] Updated weights for policy 1, policy_version 910603 (0.0010) [2023-12-26 21:59:35,872][105692] Updated weights for policy 0, policy_version 910706 (0.0009) [2023-12-26 21:59:35,927][105692] Updated weights for policy 0, policy_version 910716 (0.0009) [2023-12-26 21:59:35,981][105692] Updated weights for policy 0, policy_version 910726 (0.0009) [2023-12-26 21:59:36,039][105692] Updated weights for policy 0, policy_version 910736 (0.0005) [2023-12-26 21:59:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 466329600. Throughput: 0: 9646.1, 1: 9918.5. Samples: 466315772. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 21:59:36,062][104569] Avg episode reward: [(0, '9166.810'), (1, '8739.015')] [2023-12-26 21:59:36,356][105620] Updated weights for policy 1, policy_version 910613 (0.0008) [2023-12-26 21:59:36,412][105620] Updated weights for policy 1, policy_version 910623 (0.0005) [2023-12-26 21:59:36,471][105620] Updated weights for policy 1, policy_version 910633 (0.0008) [2023-12-26 21:59:36,502][105586] KL-divergence is very high: 146.2853 [2023-12-26 21:59:36,809][105692] Updated weights for policy 0, policy_version 910746 (0.0010) [2023-12-26 21:59:36,876][105692] Updated weights for policy 0, policy_version 910756 (0.0010) [2023-12-26 21:59:36,931][105692] Updated weights for policy 0, policy_version 910767 (0.0010) [2023-12-26 21:59:37,078][105620] Updated weights for policy 1, policy_version 910643 (0.0009) [2023-12-26 21:59:37,130][105620] Updated weights for policy 1, policy_version 910653 (0.0009) [2023-12-26 21:59:37,176][105620] Updated weights for policy 1, policy_version 910663 (0.0008) [2023-12-26 21:59:37,787][105692] Updated weights for policy 0, policy_version 910777 (0.0007) [2023-12-26 21:59:37,822][105620] Updated weights for policy 1, policy_version 910673 (0.0009) [2023-12-26 21:59:37,845][105692] Updated weights for policy 0, policy_version 910787 (0.0007) [2023-12-26 21:59:37,878][105620] Updated weights for policy 1, policy_version 910683 (0.0009) [2023-12-26 21:59:37,904][105692] Updated weights for policy 0, policy_version 910797 (0.0008) [2023-12-26 21:59:37,921][105620] Updated weights for policy 1, policy_version 910693 (0.0007) [2023-12-26 21:59:37,968][105620] Updated weights for policy 1, policy_version 910703 (0.0008) [2023-12-26 21:59:38,665][105692] Updated weights for policy 0, policy_version 910807 (0.0008) [2023-12-26 21:59:38,703][105620] Updated weights for policy 1, policy_version 910713 (0.0006) [2023-12-26 21:59:38,714][105692] Updated weights for policy 0, policy_version 910817 (0.0009) [2023-12-26 21:59:38,765][105620] Updated weights for policy 1, policy_version 910723 (0.0006) [2023-12-26 21:59:38,768][105692] Updated weights for policy 0, policy_version 910827 (0.0008) [2023-12-26 21:59:38,825][105620] Updated weights for policy 1, policy_version 910733 (0.0007) [2023-12-26 21:59:39,483][105692] Updated weights for policy 0, policy_version 910837 (0.0009) [2023-12-26 21:59:39,535][105692] Updated weights for policy 0, policy_version 910847 (0.0009) [2023-12-26 21:59:39,591][105692] Updated weights for policy 0, policy_version 910857 (0.0007) [2023-12-26 21:59:39,596][105620] Updated weights for policy 1, policy_version 910743 (0.0009) [2023-12-26 21:59:39,654][105620] Updated weights for policy 1, policy_version 910753 (0.0008) [2023-12-26 21:59:39,705][105620] Updated weights for policy 1, policy_version 910763 (0.0009) [2023-12-26 21:59:40,366][105692] Updated weights for policy 0, policy_version 910867 (0.0006) [2023-12-26 21:59:40,407][105620] Updated weights for policy 1, policy_version 910773 (0.0008) [2023-12-26 21:59:40,429][105692] Updated weights for policy 0, policy_version 910877 (0.0006) [2023-12-26 21:59:40,467][105620] Updated weights for policy 1, policy_version 910783 (0.0009) [2023-12-26 21:59:40,484][105692] Updated weights for policy 0, policy_version 910887 (0.0006) [2023-12-26 21:59:40,523][105620] Updated weights for policy 1, policy_version 910793 (0.0007) [2023-12-26 21:59:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 466419712. Throughput: 0: 9651.5, 1: 9823.5. Samples: 466430984. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 21:59:41,062][104569] Avg episode reward: [(0, '9347.800'), (1, '8294.220')] [2023-12-26 21:59:41,129][105692] Updated weights for policy 0, policy_version 910897 (0.0007) [2023-12-26 21:59:41,197][105692] Updated weights for policy 0, policy_version 910907 (0.0009) [2023-12-26 21:59:41,258][105692] Updated weights for policy 0, policy_version 910917 (0.0007) [2023-12-26 21:59:41,314][105692] Updated weights for policy 0, policy_version 910927 (0.0008) [2023-12-26 21:59:41,338][105620] Updated weights for policy 1, policy_version 910803 (0.0009) [2023-12-26 21:59:41,409][105620] Updated weights for policy 1, policy_version 910813 (0.0008) [2023-12-26 21:59:41,473][105620] Updated weights for policy 1, policy_version 910823 (0.0009) [2023-12-26 21:59:42,123][105620] Updated weights for policy 1, policy_version 910833 (0.0008) [2023-12-26 21:59:42,137][105692] Updated weights for policy 0, policy_version 910937 (0.0009) [2023-12-26 21:59:42,184][105620] Updated weights for policy 1, policy_version 910843 (0.0006) [2023-12-26 21:59:42,197][105692] Updated weights for policy 0, policy_version 910947 (0.0007) [2023-12-26 21:59:42,237][105620] Updated weights for policy 1, policy_version 910853 (0.0006) [2023-12-26 21:59:42,252][105692] Updated weights for policy 0, policy_version 910957 (0.0007) [2023-12-26 21:59:42,298][105620] Updated weights for policy 1, policy_version 910863 (0.0007) [2023-12-26 21:59:42,965][105620] Updated weights for policy 1, policy_version 910873 (0.0008) [2023-12-26 21:59:43,028][105620] Updated weights for policy 1, policy_version 910883 (0.0009) [2023-12-26 21:59:43,058][105692] Updated weights for policy 0, policy_version 910967 (0.0006) [2023-12-26 21:59:43,081][105620] Updated weights for policy 1, policy_version 910893 (0.0008) [2023-12-26 21:59:43,118][105692] Updated weights for policy 0, policy_version 910977 (0.0005) [2023-12-26 21:59:43,184][105692] Updated weights for policy 0, policy_version 910987 (0.0005) [2023-12-26 21:59:43,683][105620] Updated weights for policy 1, policy_version 910903 (0.0006) [2023-12-26 21:59:43,736][105620] Updated weights for policy 1, policy_version 910913 (0.0005) [2023-12-26 21:59:43,754][105692] Updated weights for policy 0, policy_version 910997 (0.0007) [2023-12-26 21:59:43,794][105620] Updated weights for policy 1, policy_version 910923 (0.0005) [2023-12-26 21:59:43,817][105692] Updated weights for policy 0, policy_version 911007 (0.0009) [2023-12-26 21:59:43,873][105692] Updated weights for policy 0, policy_version 911017 (0.0010) [2023-12-26 21:59:44,308][105620] Updated weights for policy 1, policy_version 910933 (0.0007) [2023-12-26 21:59:44,366][105620] Updated weights for policy 1, policy_version 910943 (0.0010) [2023-12-26 21:59:44,414][105620] Updated weights for policy 1, policy_version 910953 (0.0010) [2023-12-26 21:59:44,771][105692] Updated weights for policy 0, policy_version 911028 (0.0010) [2023-12-26 21:59:44,832][105692] Updated weights for policy 0, policy_version 911038 (0.0009) [2023-12-26 21:59:44,885][105692] Updated weights for policy 0, policy_version 911048 (0.0009) [2023-12-26 21:59:45,031][105620] Updated weights for policy 1, policy_version 910963 (0.0010) [2023-12-26 21:59:45,094][105620] Updated weights for policy 1, policy_version 910973 (0.0007) [2023-12-26 21:59:45,154][105620] Updated weights for policy 1, policy_version 910983 (0.0007) [2023-12-26 21:59:45,718][105692] Updated weights for policy 0, policy_version 911058 (0.0009) [2023-12-26 21:59:45,766][105620] Updated weights for policy 1, policy_version 910993 (0.0006) [2023-12-26 21:59:45,782][105692] Updated weights for policy 0, policy_version 911068 (0.0009) [2023-12-26 21:59:45,830][105620] Updated weights for policy 1, policy_version 911003 (0.0006) [2023-12-26 21:59:45,837][105692] Updated weights for policy 0, policy_version 911078 (0.0005) [2023-12-26 21:59:45,878][105620] Updated weights for policy 1, policy_version 911013 (0.0008) [2023-12-26 21:59:45,890][105692] Updated weights for policy 0, policy_version 911088 (0.0005) [2023-12-26 21:59:45,930][105620] Updated weights for policy 1, policy_version 911023 (0.0010) [2023-12-26 21:59:46,062][104569] Fps is (10 sec: 19659.9, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 466526208. Throughput: 0: 9645.8, 1: 9883.6. Samples: 466491844. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 21:59:46,063][104569] Avg episode reward: [(0, '9348.237'), (1, '8375.262')] [2023-12-26 21:59:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000911088_233275392.pth... [2023-12-26 21:59:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000911024_233250816.pth... [2023-12-26 21:59:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000909968_232988672.pth [2023-12-26 21:59:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000909872_232955904.pth [2023-12-26 21:59:46,484][105692] Updated weights for policy 0, policy_version 911098 (0.0009) [2023-12-26 21:59:46,538][105692] Updated weights for policy 0, policy_version 911108 (0.0009) [2023-12-26 21:59:46,593][105692] Updated weights for policy 0, policy_version 911118 (0.0008) [2023-12-26 21:59:46,689][105620] Updated weights for policy 1, policy_version 911033 (0.0009) [2023-12-26 21:59:46,741][105620] Updated weights for policy 1, policy_version 911043 (0.0010) [2023-12-26 21:59:46,794][105620] Updated weights for policy 1, policy_version 911053 (0.0010) [2023-12-26 21:59:47,253][105692] Updated weights for policy 0, policy_version 911128 (0.0008) [2023-12-26 21:59:47,305][105692] Updated weights for policy 0, policy_version 911138 (0.0008) [2023-12-26 21:59:47,366][105692] Updated weights for policy 0, policy_version 911148 (0.0009) [2023-12-26 21:59:47,553][105620] Updated weights for policy 1, policy_version 911063 (0.0007) [2023-12-26 21:59:47,599][105620] Updated weights for policy 1, policy_version 911073 (0.0005) [2023-12-26 21:59:47,663][105620] Updated weights for policy 1, policy_version 911083 (0.0005) [2023-12-26 21:59:48,203][105692] Updated weights for policy 0, policy_version 911158 (0.0009) [2023-12-26 21:59:48,249][105692] Updated weights for policy 0, policy_version 911168 (0.0008) [2023-12-26 21:59:48,285][105620] Updated weights for policy 1, policy_version 911093 (0.0008) [2023-12-26 21:59:48,307][105692] Updated weights for policy 0, policy_version 911178 (0.0006) [2023-12-26 21:59:48,345][105620] Updated weights for policy 1, policy_version 911103 (0.0011) [2023-12-26 21:59:48,409][105620] Updated weights for policy 1, policy_version 911113 (0.0011) [2023-12-26 21:59:49,063][105692] Updated weights for policy 0, policy_version 911188 (0.0006) [2023-12-26 21:59:49,123][105692] Updated weights for policy 0, policy_version 911198 (0.0005) [2023-12-26 21:59:49,160][105620] Updated weights for policy 1, policy_version 911123 (0.0011) [2023-12-26 21:59:49,178][105692] Updated weights for policy 0, policy_version 911208 (0.0006) [2023-12-26 21:59:49,227][105620] Updated weights for policy 1, policy_version 911133 (0.0010) [2023-12-26 21:59:49,285][105620] Updated weights for policy 1, policy_version 911143 (0.0010) [2023-12-26 21:59:49,880][105692] Updated weights for policy 0, policy_version 911218 (0.0007) [2023-12-26 21:59:49,942][105692] Updated weights for policy 0, policy_version 911228 (0.0008) [2023-12-26 21:59:50,004][105692] Updated weights for policy 0, policy_version 911238 (0.0009) [2023-12-26 21:59:50,059][105620] Updated weights for policy 1, policy_version 911153 (0.0011) [2023-12-26 21:59:50,062][105692] Updated weights for policy 0, policy_version 911248 (0.0009) [2023-12-26 21:59:50,121][105620] Updated weights for policy 1, policy_version 911163 (0.0010) [2023-12-26 21:59:50,179][105620] Updated weights for policy 1, policy_version 911173 (0.0009) [2023-12-26 21:59:50,234][105620] Updated weights for policy 1, policy_version 911183 (0.0009) [2023-12-26 21:59:50,822][105692] Updated weights for policy 0, policy_version 911258 (0.0008) [2023-12-26 21:59:50,882][105692] Updated weights for policy 0, policy_version 911268 (0.0010) [2023-12-26 21:59:50,936][105692] Updated weights for policy 0, policy_version 911278 (0.0009) [2023-12-26 21:59:50,979][105620] Updated weights for policy 1, policy_version 911193 (0.0009) [2023-12-26 21:59:51,042][105620] Updated weights for policy 1, policy_version 911203 (0.0008) [2023-12-26 21:59:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 466616320. Throughput: 0: 9641.9, 1: 9910.7. Samples: 466608464. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 21:59:51,062][104569] Avg episode reward: [(0, '9256.813'), (1, '8914.113')] [2023-12-26 21:59:51,109][105620] Updated weights for policy 1, policy_version 911213 (0.0009) [2023-12-26 21:59:51,759][105692] Updated weights for policy 0, policy_version 911288 (0.0009) [2023-12-26 21:59:51,822][105692] Updated weights for policy 0, policy_version 911298 (0.0010) [2023-12-26 21:59:51,881][105692] Updated weights for policy 0, policy_version 911308 (0.0009) [2023-12-26 21:59:51,916][105620] Updated weights for policy 1, policy_version 911223 (0.0009) [2023-12-26 21:59:51,979][105620] Updated weights for policy 1, policy_version 911233 (0.0008) [2023-12-26 21:59:52,044][105620] Updated weights for policy 1, policy_version 911243 (0.0009) [2023-12-26 21:59:52,551][105692] Updated weights for policy 0, policy_version 911318 (0.0009) [2023-12-26 21:59:52,609][105692] Updated weights for policy 0, policy_version 911328 (0.0010) [2023-12-26 21:59:52,661][105692] Updated weights for policy 0, policy_version 911338 (0.0010) [2023-12-26 21:59:52,823][105620] Updated weights for policy 1, policy_version 911253 (0.0009) [2023-12-26 21:59:52,878][105620] Updated weights for policy 1, policy_version 911263 (0.0008) [2023-12-26 21:59:52,936][105620] Updated weights for policy 1, policy_version 911273 (0.0008) [2023-12-26 21:59:53,349][105692] Updated weights for policy 0, policy_version 911348 (0.0008) [2023-12-26 21:59:53,398][105692] Updated weights for policy 0, policy_version 911359 (0.0007) [2023-12-26 21:59:53,458][105692] Updated weights for policy 0, policy_version 911369 (0.0007) [2023-12-26 21:59:53,731][105620] Updated weights for policy 1, policy_version 911283 (0.0009) [2023-12-26 21:59:53,796][105620] Updated weights for policy 1, policy_version 911293 (0.0010) [2023-12-26 21:59:53,865][105620] Updated weights for policy 1, policy_version 911303 (0.0009) [2023-12-26 21:59:54,056][105692] Updated weights for policy 0, policy_version 911379 (0.0008) [2023-12-26 21:59:54,115][105692] Updated weights for policy 0, policy_version 911389 (0.0006) [2023-12-26 21:59:54,182][105692] Updated weights for policy 0, policy_version 911399 (0.0005) [2023-12-26 21:59:54,541][105620] Updated weights for policy 1, policy_version 911313 (0.0010) [2023-12-26 21:59:54,589][105620] Updated weights for policy 1, policy_version 911323 (0.0010) [2023-12-26 21:59:54,637][105620] Updated weights for policy 1, policy_version 911333 (0.0010) [2023-12-26 21:59:54,695][105620] Updated weights for policy 1, policy_version 911343 (0.0010) [2023-12-26 21:59:54,871][105692] Updated weights for policy 0, policy_version 911409 (0.0005) [2023-12-26 21:59:54,935][105692] Updated weights for policy 0, policy_version 911419 (0.0006) [2023-12-26 21:59:54,987][105692] Updated weights for policy 0, policy_version 911429 (0.0010) [2023-12-26 21:59:55,032][105692] Updated weights for policy 0, policy_version 911439 (0.0008) [2023-12-26 21:59:55,419][105620] Updated weights for policy 1, policy_version 911353 (0.0006) [2023-12-26 21:59:55,474][105620] Updated weights for policy 1, policy_version 911363 (0.0005) [2023-12-26 21:59:55,537][105620] Updated weights for policy 1, policy_version 911373 (0.0006) [2023-12-26 21:59:55,760][105692] Updated weights for policy 0, policy_version 911449 (0.0010) [2023-12-26 21:59:55,825][105692] Updated weights for policy 0, policy_version 911459 (0.0010) [2023-12-26 21:59:55,885][105692] Updated weights for policy 0, policy_version 911469 (0.0010) [2023-12-26 21:59:56,062][104569] Fps is (10 sec: 18842.3, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 466714624. Throughput: 0: 9690.5, 1: 9828.1. Samples: 466724204. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 21:59:56,063][104569] Avg episode reward: [(0, '8991.652'), (1, '9095.265')] [2023-12-26 21:59:56,141][105620] Updated weights for policy 1, policy_version 911383 (0.0007) [2023-12-26 21:59:56,194][105620] Updated weights for policy 1, policy_version 911393 (0.0008) [2023-12-26 21:59:56,250][105620] Updated weights for policy 1, policy_version 911403 (0.0006) [2023-12-26 21:59:56,532][105692] Updated weights for policy 0, policy_version 911479 (0.0010) [2023-12-26 21:59:56,596][105692] Updated weights for policy 0, policy_version 911489 (0.0010) [2023-12-26 21:59:56,653][105692] Updated weights for policy 0, policy_version 911499 (0.0007) [2023-12-26 21:59:56,813][105620] Updated weights for policy 1, policy_version 911413 (0.0005) [2023-12-26 21:59:56,861][105620] Updated weights for policy 1, policy_version 911423 (0.0005) [2023-12-26 21:59:56,912][105620] Updated weights for policy 1, policy_version 911433 (0.0005) [2023-12-26 21:59:57,274][105692] Updated weights for policy 0, policy_version 911509 (0.0007) [2023-12-26 21:59:57,329][105692] Updated weights for policy 0, policy_version 911519 (0.0007) [2023-12-26 21:59:57,394][105692] Updated weights for policy 0, policy_version 911529 (0.0006) [2023-12-26 21:59:57,542][105620] Updated weights for policy 1, policy_version 911443 (0.0006) [2023-12-26 21:59:57,598][105620] Updated weights for policy 1, policy_version 911453 (0.0005) [2023-12-26 21:59:57,663][105620] Updated weights for policy 1, policy_version 911463 (0.0005) [2023-12-26 21:59:57,987][105692] Updated weights for policy 0, policy_version 911539 (0.0007) [2023-12-26 21:59:58,046][105692] Updated weights for policy 0, policy_version 911549 (0.0010) [2023-12-26 21:59:58,091][105692] Updated weights for policy 0, policy_version 911559 (0.0010) [2023-12-26 21:59:58,231][105620] Updated weights for policy 1, policy_version 911473 (0.0007) [2023-12-26 21:59:58,294][105620] Updated weights for policy 1, policy_version 911483 (0.0011) [2023-12-26 21:59:58,361][105620] Updated weights for policy 1, policy_version 911493 (0.0011) [2023-12-26 21:59:58,419][105620] Updated weights for policy 1, policy_version 911503 (0.0007) [2023-12-26 21:59:58,866][105692] Updated weights for policy 0, policy_version 911569 (0.0008) [2023-12-26 21:59:58,928][105692] Updated weights for policy 0, policy_version 911579 (0.0008) [2023-12-26 21:59:58,983][105692] Updated weights for policy 0, policy_version 911589 (0.0008) [2023-12-26 21:59:59,033][105692] Updated weights for policy 0, policy_version 911599 (0.0008) [2023-12-26 21:59:59,164][105620] Updated weights for policy 1, policy_version 911513 (0.0009) [2023-12-26 21:59:59,230][105620] Updated weights for policy 1, policy_version 911523 (0.0007) [2023-12-26 21:59:59,280][105620] Updated weights for policy 1, policy_version 911533 (0.0006) [2023-12-26 21:59:59,699][105692] Updated weights for policy 0, policy_version 911609 (0.0008) [2023-12-26 21:59:59,757][105692] Updated weights for policy 0, policy_version 911619 (0.0011) [2023-12-26 21:59:59,815][105692] Updated weights for policy 0, policy_version 911629 (0.0010) [2023-12-26 21:59:59,904][105620] Updated weights for policy 1, policy_version 911543 (0.0007) [2023-12-26 21:59:59,963][105620] Updated weights for policy 1, policy_version 911553 (0.0008) [2023-12-26 22:00:00,017][105620] Updated weights for policy 1, policy_version 911563 (0.0008) [2023-12-26 22:00:00,509][105692] Updated weights for policy 0, policy_version 911639 (0.0010) [2023-12-26 22:00:00,560][105692] Updated weights for policy 0, policy_version 911649 (0.0010) [2023-12-26 22:00:00,614][105692] Updated weights for policy 0, policy_version 911659 (0.0010) [2023-12-26 22:00:00,705][105620] Updated weights for policy 1, policy_version 911573 (0.0007) [2023-12-26 22:00:00,758][105620] Updated weights for policy 1, policy_version 911583 (0.0005) [2023-12-26 22:00:00,813][105620] Updated weights for policy 1, policy_version 911593 (0.0009) [2023-12-26 22:00:01,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 466821120. Throughput: 0: 9774.2, 1: 9892.5. Samples: 466788296. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 22:00:01,062][104569] Avg episode reward: [(0, '8996.088'), (1, '8919.035')] [2023-12-26 22:00:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000911664_233422848.pth... [2023-12-26 22:00:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000911600_233398272.pth... [2023-12-26 22:00:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000910544_233136128.pth [2023-12-26 22:00:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000910416_233095168.pth [2023-12-26 22:00:01,273][105692] Updated weights for policy 0, policy_version 911669 (0.0010) [2023-12-26 22:00:01,327][105692] Updated weights for policy 0, policy_version 911679 (0.0010) [2023-12-26 22:00:01,395][105692] Updated weights for policy 0, policy_version 911689 (0.0010) [2023-12-26 22:00:01,508][105620] Updated weights for policy 1, policy_version 911603 (0.0010) [2023-12-26 22:00:01,558][105620] Updated weights for policy 1, policy_version 911613 (0.0009) [2023-12-26 22:00:01,606][105620] Updated weights for policy 1, policy_version 911623 (0.0009) [2023-12-26 22:00:02,118][105692] Updated weights for policy 0, policy_version 911699 (0.0009) [2023-12-26 22:00:02,181][105692] Updated weights for policy 0, policy_version 911709 (0.0009) [2023-12-26 22:00:02,229][105692] Updated weights for policy 0, policy_version 911719 (0.0009) [2023-12-26 22:00:02,408][105620] Updated weights for policy 1, policy_version 911633 (0.0010) [2023-12-26 22:00:02,487][105620] Updated weights for policy 1, policy_version 911643 (0.0009) [2023-12-26 22:00:02,554][105620] Updated weights for policy 1, policy_version 911653 (0.0008) [2023-12-26 22:00:02,619][105620] Updated weights for policy 1, policy_version 911663 (0.0009) [2023-12-26 22:00:02,961][105692] Updated weights for policy 0, policy_version 911729 (0.0008) [2023-12-26 22:00:03,031][105692] Updated weights for policy 0, policy_version 911739 (0.0007) [2023-12-26 22:00:03,091][105692] Updated weights for policy 0, policy_version 911749 (0.0009) [2023-12-26 22:00:03,167][105692] Updated weights for policy 0, policy_version 911759 (0.0009) [2023-12-26 22:00:03,445][105620] Updated weights for policy 1, policy_version 911673 (0.0008) [2023-12-26 22:00:03,499][105620] Updated weights for policy 1, policy_version 911683 (0.0008) [2023-12-26 22:00:03,557][105620] Updated weights for policy 1, policy_version 911693 (0.0007) [2023-12-26 22:00:03,823][105692] Updated weights for policy 0, policy_version 911769 (0.0006) [2023-12-26 22:00:03,883][105692] Updated weights for policy 0, policy_version 911779 (0.0007) [2023-12-26 22:00:03,943][105692] Updated weights for policy 0, policy_version 911789 (0.0008) [2023-12-26 22:00:04,326][105620] Updated weights for policy 1, policy_version 911703 (0.0010) [2023-12-26 22:00:04,388][105620] Updated weights for policy 1, policy_version 911713 (0.0011) [2023-12-26 22:00:04,458][105620] Updated weights for policy 1, policy_version 911723 (0.0011) [2023-12-26 22:00:04,695][105692] Updated weights for policy 0, policy_version 911799 (0.0011) [2023-12-26 22:00:04,754][105692] Updated weights for policy 0, policy_version 911809 (0.0010) [2023-12-26 22:00:04,820][105692] Updated weights for policy 0, policy_version 911819 (0.0011) [2023-12-26 22:00:05,136][105620] Updated weights for policy 1, policy_version 911733 (0.0010) [2023-12-26 22:00:05,188][105620] Updated weights for policy 1, policy_version 911743 (0.0011) [2023-12-26 22:00:05,257][105620] Updated weights for policy 1, policy_version 911753 (0.0011) [2023-12-26 22:00:05,545][105692] Updated weights for policy 0, policy_version 911829 (0.0011) [2023-12-26 22:00:05,599][105692] Updated weights for policy 0, policy_version 911839 (0.0011) [2023-12-26 22:00:05,653][105692] Updated weights for policy 0, policy_version 911849 (0.0011) [2023-12-26 22:00:06,024][105620] Updated weights for policy 1, policy_version 911763 (0.0011) [2023-12-26 22:00:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 466911232. Throughput: 0: 9709.6, 1: 9844.2. Samples: 466904168. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 22:00:06,062][104569] Avg episode reward: [(0, '9083.999'), (1, '8390.952')] [2023-12-26 22:00:06,070][105620] Updated weights for policy 1, policy_version 911773 (0.0011) [2023-12-26 22:00:06,130][105620] Updated weights for policy 1, policy_version 911783 (0.0011) [2023-12-26 22:00:06,416][105692] Updated weights for policy 0, policy_version 911859 (0.0011) [2023-12-26 22:00:06,481][105692] Updated weights for policy 0, policy_version 911869 (0.0011) [2023-12-26 22:00:06,553][105692] Updated weights for policy 0, policy_version 911879 (0.0006) [2023-12-26 22:00:06,971][105620] Updated weights for policy 1, policy_version 911793 (0.0011) [2023-12-26 22:00:07,033][105620] Updated weights for policy 1, policy_version 911803 (0.0011) [2023-12-26 22:00:07,093][105620] Updated weights for policy 1, policy_version 911813 (0.0011) [2023-12-26 22:00:07,148][105620] Updated weights for policy 1, policy_version 911823 (0.0011) [2023-12-26 22:00:07,218][105692] Updated weights for policy 0, policy_version 911889 (0.0011) [2023-12-26 22:00:07,273][105692] Updated weights for policy 0, policy_version 911899 (0.0010) [2023-12-26 22:00:07,328][105692] Updated weights for policy 0, policy_version 911909 (0.0010) [2023-12-26 22:00:07,381][105692] Updated weights for policy 0, policy_version 911919 (0.0009) [2023-12-26 22:00:07,880][105620] Updated weights for policy 1, policy_version 911833 (0.0011) [2023-12-26 22:00:07,937][105620] Updated weights for policy 1, policy_version 911843 (0.0010) [2023-12-26 22:00:08,001][105620] Updated weights for policy 1, policy_version 911853 (0.0011) [2023-12-26 22:00:08,050][105692] Updated weights for policy 0, policy_version 911929 (0.0007) [2023-12-26 22:00:08,115][105692] Updated weights for policy 0, policy_version 911939 (0.0008) [2023-12-26 22:00:08,178][105692] Updated weights for policy 0, policy_version 911949 (0.0008) [2023-12-26 22:00:08,807][105620] Updated weights for policy 1, policy_version 911863 (0.0011) [2023-12-26 22:00:08,873][105620] Updated weights for policy 1, policy_version 911873 (0.0011) [2023-12-26 22:00:08,916][105692] Updated weights for policy 0, policy_version 911959 (0.0010) [2023-12-26 22:00:08,935][105620] Updated weights for policy 1, policy_version 911883 (0.0010) [2023-12-26 22:00:08,968][105692] Updated weights for policy 0, policy_version 911969 (0.0010) [2023-12-26 22:00:09,037][105692] Updated weights for policy 0, policy_version 911979 (0.0008) [2023-12-26 22:00:09,730][105620] Updated weights for policy 1, policy_version 911893 (0.0011) [2023-12-26 22:00:09,799][105620] Updated weights for policy 1, policy_version 911903 (0.0011) [2023-12-26 22:00:09,874][105620] Updated weights for policy 1, policy_version 911913 (0.0010) [2023-12-26 22:00:09,906][105692] Updated weights for policy 0, policy_version 911989 (0.0010) [2023-12-26 22:00:09,965][105692] Updated weights for policy 0, policy_version 911999 (0.0009) [2023-12-26 22:00:10,021][105692] Updated weights for policy 0, policy_version 912009 (0.0009) [2023-12-26 22:00:10,637][105620] Updated weights for policy 1, policy_version 911923 (0.0010) [2023-12-26 22:00:10,705][105620] Updated weights for policy 1, policy_version 911933 (0.0009) [2023-12-26 22:00:10,764][105620] Updated weights for policy 1, policy_version 911943 (0.0009) [2023-12-26 22:00:10,779][105692] Updated weights for policy 0, policy_version 912019 (0.0008) [2023-12-26 22:00:10,847][105692] Updated weights for policy 0, policy_version 912029 (0.0007) [2023-12-26 22:00:10,915][105692] Updated weights for policy 0, policy_version 912039 (0.0010) [2023-12-26 22:00:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 467009536. Throughput: 0: 9742.6, 1: 9787.0. Samples: 467014364. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 22:00:11,062][104569] Avg episode reward: [(0, '8897.630'), (1, '8467.605')] [2023-12-26 22:00:11,542][105620] Updated weights for policy 1, policy_version 911953 (0.0009) [2023-12-26 22:00:11,609][105620] Updated weights for policy 1, policy_version 911963 (0.0011) [2023-12-26 22:00:11,690][105620] Updated weights for policy 1, policy_version 911973 (0.0010) [2023-12-26 22:00:11,767][105620] Updated weights for policy 1, policy_version 911983 (0.0010) [2023-12-26 22:00:11,775][105692] Updated weights for policy 0, policy_version 912049 (0.0009) [2023-12-26 22:00:11,839][105692] Updated weights for policy 0, policy_version 912059 (0.0009) [2023-12-26 22:00:11,896][105692] Updated weights for policy 0, policy_version 912069 (0.0008) [2023-12-26 22:00:11,960][105692] Updated weights for policy 0, policy_version 912079 (0.0008) [2023-12-26 22:00:12,552][105620] Updated weights for policy 1, policy_version 911993 (0.0009) [2023-12-26 22:00:12,603][105620] Updated weights for policy 1, policy_version 912003 (0.0009) [2023-12-26 22:00:12,659][105620] Updated weights for policy 1, policy_version 912013 (0.0009) [2023-12-26 22:00:12,766][105692] Updated weights for policy 0, policy_version 912089 (0.0009) [2023-12-26 22:00:12,831][105692] Updated weights for policy 0, policy_version 912099 (0.0009) [2023-12-26 22:00:12,895][105692] Updated weights for policy 0, policy_version 912109 (0.0009) [2023-12-26 22:00:13,423][105620] Updated weights for policy 1, policy_version 912023 (0.0007) [2023-12-26 22:00:13,495][105620] Updated weights for policy 1, policy_version 912033 (0.0006) [2023-12-26 22:00:13,566][105620] Updated weights for policy 1, policy_version 912043 (0.0006) [2023-12-26 22:00:13,664][105692] Updated weights for policy 0, policy_version 912119 (0.0009) [2023-12-26 22:00:13,720][105692] Updated weights for policy 0, policy_version 912129 (0.0009) [2023-12-26 22:00:13,778][105692] Updated weights for policy 0, policy_version 912139 (0.0010) [2023-12-26 22:00:14,097][105620] Updated weights for policy 1, policy_version 912053 (0.0006) [2023-12-26 22:00:14,163][105620] Updated weights for policy 1, policy_version 912063 (0.0008) [2023-12-26 22:00:14,227][105620] Updated weights for policy 1, policy_version 912073 (0.0009) [2023-12-26 22:00:14,597][105692] Updated weights for policy 0, policy_version 912150 (0.0009) [2023-12-26 22:00:14,664][105692] Updated weights for policy 0, policy_version 912160 (0.0009) [2023-12-26 22:00:14,731][105692] Updated weights for policy 0, policy_version 912170 (0.0010) [2023-12-26 22:00:15,001][105620] Updated weights for policy 1, policy_version 912083 (0.0010) [2023-12-26 22:00:15,067][105620] Updated weights for policy 1, policy_version 912093 (0.0009) [2023-12-26 22:00:15,122][105620] Updated weights for policy 1, policy_version 912103 (0.0009) [2023-12-26 22:00:15,538][105692] Updated weights for policy 0, policy_version 912180 (0.0009) [2023-12-26 22:00:15,610][105692] Updated weights for policy 0, policy_version 912190 (0.0009) [2023-12-26 22:00:15,683][105692] Updated weights for policy 0, policy_version 912200 (0.0009) [2023-12-26 22:00:15,905][105620] Updated weights for policy 1, policy_version 912113 (0.0009) [2023-12-26 22:00:15,958][105620] Updated weights for policy 1, policy_version 912123 (0.0009) [2023-12-26 22:00:16,008][105620] Updated weights for policy 1, policy_version 912133 (0.0008) [2023-12-26 22:00:16,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 467091456. Throughput: 0: 9623.0, 1: 9749.0. Samples: 467068936. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 22:00:16,063][104569] Avg episode reward: [(0, '8990.586'), (1, '7421.715')] [2023-12-26 22:00:16,065][105620] Updated weights for policy 1, policy_version 912143 (0.0009) [2023-12-26 22:00:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000912208_233562112.pth... [2023-12-26 22:00:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000912144_233537536.pth... [2023-12-26 22:00:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000911088_233275392.pth [2023-12-26 22:00:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000911024_233250816.pth [2023-12-26 22:00:16,529][105692] Updated weights for policy 0, policy_version 912210 (0.0010) [2023-12-26 22:00:16,586][105692] Updated weights for policy 0, policy_version 912220 (0.0011) [2023-12-26 22:00:16,644][105692] Updated weights for policy 0, policy_version 912230 (0.0009) [2023-12-26 22:00:16,677][105620] Updated weights for policy 1, policy_version 912153 (0.0008) [2023-12-26 22:00:16,709][105692] Updated weights for policy 0, policy_version 912240 (0.0008) [2023-12-26 22:00:16,739][105620] Updated weights for policy 1, policy_version 912163 (0.0010) [2023-12-26 22:00:16,787][105620] Updated weights for policy 1, policy_version 912173 (0.0009) [2023-12-26 22:00:17,495][105620] Updated weights for policy 1, policy_version 912183 (0.0008) [2023-12-26 22:00:17,524][105692] Updated weights for policy 0, policy_version 912250 (0.0010) [2023-12-26 22:00:17,540][105620] Updated weights for policy 1, policy_version 912193 (0.0009) [2023-12-26 22:00:17,582][105692] Updated weights for policy 0, policy_version 912260 (0.0009) [2023-12-26 22:00:17,585][105620] Updated weights for policy 1, policy_version 912203 (0.0007) [2023-12-26 22:00:17,641][105692] Updated weights for policy 0, policy_version 912270 (0.0007) [2023-12-26 22:00:18,388][105692] Updated weights for policy 0, policy_version 912280 (0.0009) [2023-12-26 22:00:18,430][105620] Updated weights for policy 1, policy_version 912213 (0.0007) [2023-12-26 22:00:18,445][105692] Updated weights for policy 0, policy_version 912290 (0.0007) [2023-12-26 22:00:18,489][105620] Updated weights for policy 1, policy_version 912223 (0.0007) [2023-12-26 22:00:18,507][105692] Updated weights for policy 0, policy_version 912300 (0.0007) [2023-12-26 22:00:18,540][105620] Updated weights for policy 1, policy_version 912233 (0.0006) [2023-12-26 22:00:19,288][105620] Updated weights for policy 1, policy_version 912243 (0.0009) [2023-12-26 22:00:19,303][105692] Updated weights for policy 0, policy_version 912310 (0.0006) [2023-12-26 22:00:19,348][105586] KL-divergence is very high: 118.5369 [2023-12-26 22:00:19,364][105620] Updated weights for policy 1, policy_version 912253 (0.0008) [2023-12-26 22:00:19,375][105692] Updated weights for policy 0, policy_version 912320 (0.0007) [2023-12-26 22:00:19,403][105586] KL-divergence is very high: 303.5320 [2023-12-26 22:00:19,427][105620] Updated weights for policy 1, policy_version 912263 (0.0008) [2023-12-26 22:00:19,434][105692] Updated weights for policy 0, policy_version 912330 (0.0006) [2023-12-26 22:00:19,450][105586] KL-divergence is very high: 302.7091 [2023-12-26 22:00:20,159][105620] Updated weights for policy 1, policy_version 912273 (0.0008) [2023-12-26 22:00:20,216][105620] Updated weights for policy 1, policy_version 912283 (0.0009) [2023-12-26 22:00:20,243][105586] KL-divergence is very high: 188.2094 [2023-12-26 22:00:20,275][105620] Updated weights for policy 1, policy_version 912293 (0.0008) [2023-12-26 22:00:20,291][105586] KL-divergence is very high: 344.6849 [2023-12-26 22:00:20,309][105692] Updated weights for policy 0, policy_version 912340 (0.0009) [2023-12-26 22:00:20,332][105620] Updated weights for policy 1, policy_version 912303 (0.0006) [2023-12-26 22:00:20,371][105692] Updated weights for policy 0, policy_version 912350 (0.0011) [2023-12-26 22:00:20,435][105692] Updated weights for policy 0, policy_version 912360 (0.0011) [2023-12-26 22:00:20,994][105620] Updated weights for policy 1, policy_version 912313 (0.0008) [2023-12-26 22:00:21,062][104569] Fps is (10 sec: 17203.1, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 467181568. Throughput: 0: 9457.0, 1: 9698.1. Samples: 467177756. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 22:00:21,063][104569] Avg episode reward: [(0, '9263.963'), (1, '7525.580')] [2023-12-26 22:00:21,073][105620] Updated weights for policy 1, policy_version 912323 (0.0008) [2023-12-26 22:00:21,142][105620] Updated weights for policy 1, policy_version 912333 (0.0009) [2023-12-26 22:00:21,248][105692] Updated weights for policy 0, policy_version 912370 (0.0010) [2023-12-26 22:00:21,323][105692] Updated weights for policy 0, policy_version 912380 (0.0007) [2023-12-26 22:00:21,395][105692] Updated weights for policy 0, policy_version 912390 (0.0009) [2023-12-26 22:00:21,457][105692] Updated weights for policy 0, policy_version 912400 (0.0011) [2023-12-26 22:00:21,981][105620] Updated weights for policy 1, policy_version 912343 (0.0010) [2023-12-26 22:00:22,048][105620] Updated weights for policy 1, policy_version 912353 (0.0009) [2023-12-26 22:00:22,127][105620] Updated weights for policy 1, policy_version 912363 (0.0009) [2023-12-26 22:00:22,268][105692] Updated weights for policy 0, policy_version 912410 (0.0009) [2023-12-26 22:00:22,339][105692] Updated weights for policy 0, policy_version 912420 (0.0011) [2023-12-26 22:00:22,401][105692] Updated weights for policy 0, policy_version 912430 (0.0008) [2023-12-26 22:00:22,839][105620] Updated weights for policy 1, policy_version 912373 (0.0007) [2023-12-26 22:00:22,886][105620] Updated weights for policy 1, policy_version 912383 (0.0005) [2023-12-26 22:00:22,941][105620] Updated weights for policy 1, policy_version 912393 (0.0005) [2023-12-26 22:00:23,274][105692] Updated weights for policy 0, policy_version 912440 (0.0009) [2023-12-26 22:00:23,333][105692] Updated weights for policy 0, policy_version 912450 (0.0009) [2023-12-26 22:00:23,391][105692] Updated weights for policy 0, policy_version 912460 (0.0010) [2023-12-26 22:00:23,560][105620] Updated weights for policy 1, policy_version 912403 (0.0006) [2023-12-26 22:00:23,612][105620] Updated weights for policy 1, policy_version 912413 (0.0006) [2023-12-26 22:00:23,659][105620] Updated weights for policy 1, policy_version 912423 (0.0009) [2023-12-26 22:00:24,279][105692] Updated weights for policy 0, policy_version 912470 (0.0009) [2023-12-26 22:00:24,301][105620] Updated weights for policy 1, policy_version 912433 (0.0007) [2023-12-26 22:00:24,337][105692] Updated weights for policy 0, policy_version 912480 (0.0008) [2023-12-26 22:00:24,361][105620] Updated weights for policy 1, policy_version 912443 (0.0007) [2023-12-26 22:00:24,392][105692] Updated weights for policy 0, policy_version 912490 (0.0006) [2023-12-26 22:00:24,415][105620] Updated weights for policy 1, policy_version 912453 (0.0009) [2023-12-26 22:00:24,465][105620] Updated weights for policy 1, policy_version 912463 (0.0008) [2023-12-26 22:00:25,175][105692] Updated weights for policy 0, policy_version 912500 (0.0008) [2023-12-26 22:00:25,223][105692] Updated weights for policy 0, policy_version 912510 (0.0007) [2023-12-26 22:00:25,249][105620] Updated weights for policy 1, policy_version 912473 (0.0009) [2023-12-26 22:00:25,280][105692] Updated weights for policy 0, policy_version 912520 (0.0006) [2023-12-26 22:00:25,295][105620] Updated weights for policy 1, policy_version 912483 (0.0007) [2023-12-26 22:00:25,344][105620] Updated weights for policy 1, policy_version 912493 (0.0007) [2023-12-26 22:00:26,055][105692] Updated weights for policy 0, policy_version 912530 (0.0008) [2023-12-26 22:00:26,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19114.6, 300 sec: 19327.6). Total num frames: 467271680. Throughput: 0: 9301.2, 1: 9700.4. Samples: 467286060. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-26 22:00:26,062][104569] Avg episode reward: [(0, '9256.615'), (1, '8230.888')] [2023-12-26 22:00:26,109][105692] Updated weights for policy 0, policy_version 912540 (0.0006) [2023-12-26 22:00:26,115][105620] Updated weights for policy 1, policy_version 912503 (0.0009) [2023-12-26 22:00:26,171][105692] Updated weights for policy 0, policy_version 912550 (0.0006) [2023-12-26 22:00:26,175][105620] Updated weights for policy 1, policy_version 912513 (0.0008) [2023-12-26 22:00:26,238][105620] Updated weights for policy 1, policy_version 912523 (0.0006) [2023-12-26 22:00:26,242][105692] Updated weights for policy 0, policy_version 912560 (0.0008) [2023-12-26 22:00:26,883][105620] Updated weights for policy 1, policy_version 912533 (0.0009) [2023-12-26 22:00:26,952][105620] Updated weights for policy 1, policy_version 912543 (0.0009) [2023-12-26 22:00:27,002][105692] Updated weights for policy 0, policy_version 912570 (0.0007) [2023-12-26 22:00:27,008][105620] Updated weights for policy 1, policy_version 912553 (0.0009) [2023-12-26 22:00:27,057][105692] Updated weights for policy 0, policy_version 912580 (0.0008) [2023-12-26 22:00:27,105][105692] Updated weights for policy 0, policy_version 912590 (0.0009) [2023-12-26 22:00:27,747][105620] Updated weights for policy 1, policy_version 912563 (0.0007) [2023-12-26 22:00:27,800][105620] Updated weights for policy 1, policy_version 912573 (0.0006) [2023-12-26 22:00:27,812][105692] Updated weights for policy 0, policy_version 912600 (0.0006) [2023-12-26 22:00:27,859][105620] Updated weights for policy 1, policy_version 912583 (0.0008) [2023-12-26 22:00:27,870][105692] Updated weights for policy 0, policy_version 912610 (0.0005) [2023-12-26 22:00:27,926][105692] Updated weights for policy 0, policy_version 912620 (0.0005) [2023-12-26 22:00:28,569][105620] Updated weights for policy 1, policy_version 912593 (0.0009) [2023-12-26 22:00:28,587][105692] Updated weights for policy 0, policy_version 912630 (0.0008) [2023-12-26 22:00:28,623][105620] Updated weights for policy 1, policy_version 912603 (0.0007) [2023-12-26 22:00:28,650][105692] Updated weights for policy 0, policy_version 912640 (0.0007) [2023-12-26 22:00:28,681][105620] Updated weights for policy 1, policy_version 912613 (0.0007) [2023-12-26 22:00:28,713][105692] Updated weights for policy 0, policy_version 912650 (0.0007) [2023-12-26 22:00:28,742][105620] Updated weights for policy 1, policy_version 912623 (0.0006) [2023-12-26 22:00:29,500][105692] Updated weights for policy 0, policy_version 912660 (0.0006) [2023-12-26 22:00:29,531][105620] Updated weights for policy 1, policy_version 912633 (0.0008) [2023-12-26 22:00:29,555][105692] Updated weights for policy 0, policy_version 912670 (0.0006) [2023-12-26 22:00:29,589][105620] Updated weights for policy 1, policy_version 912643 (0.0008) [2023-12-26 22:00:29,611][105692] Updated weights for policy 0, policy_version 912680 (0.0008) [2023-12-26 22:00:29,648][105620] Updated weights for policy 1, policy_version 912653 (0.0008) [2023-12-26 22:00:30,397][105692] Updated weights for policy 0, policy_version 912690 (0.0008) [2023-12-26 22:00:30,441][105620] Updated weights for policy 1, policy_version 912663 (0.0010) [2023-12-26 22:00:30,454][105692] Updated weights for policy 0, policy_version 912700 (0.0005) [2023-12-26 22:00:30,503][105620] Updated weights for policy 1, policy_version 912673 (0.0010) [2023-12-26 22:00:30,508][105692] Updated weights for policy 0, policy_version 912710 (0.0005) [2023-12-26 22:00:30,560][105620] Updated weights for policy 1, policy_version 912683 (0.0010) [2023-12-26 22:00:30,560][105692] Updated weights for policy 0, policy_version 912720 (0.0006) [2023-12-26 22:00:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19355.3). Total num frames: 467369984. Throughput: 0: 9306.6, 1: 9635.6. Samples: 467344236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:00:31,062][104569] Avg episode reward: [(0, '9256.626'), (1, '8398.872')] [2023-12-26 22:00:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000912720_233693184.pth... [2023-12-26 22:00:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000912688_233676800.pth... [2023-12-26 22:00:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000911600_233398272.pth [2023-12-26 22:00:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000911664_233422848.pth [2023-12-26 22:00:31,271][105620] Updated weights for policy 1, policy_version 912693 (0.0010) [2023-12-26 22:00:31,271][105692] Updated weights for policy 0, policy_version 912730 (0.0009) [2023-12-26 22:00:31,348][105620] Updated weights for policy 1, policy_version 912703 (0.0007) [2023-12-26 22:00:31,349][105692] Updated weights for policy 0, policy_version 912740 (0.0008) [2023-12-26 22:00:31,418][105620] Updated weights for policy 1, policy_version 912713 (0.0008) [2023-12-26 22:00:31,419][105692] Updated weights for policy 0, policy_version 912750 (0.0009) [2023-12-26 22:00:32,020][105692] Updated weights for policy 0, policy_version 912760 (0.0008) [2023-12-26 22:00:32,086][105692] Updated weights for policy 0, policy_version 912770 (0.0011) [2023-12-26 22:00:32,151][105620] Updated weights for policy 1, policy_version 912723 (0.0008) [2023-12-26 22:00:32,153][105692] Updated weights for policy 0, policy_version 912780 (0.0011) [2023-12-26 22:00:32,214][105620] Updated weights for policy 1, policy_version 912733 (0.0007) [2023-12-26 22:00:32,278][105620] Updated weights for policy 1, policy_version 912743 (0.0009) [2023-12-26 22:00:32,909][105692] Updated weights for policy 0, policy_version 912790 (0.0010) [2023-12-26 22:00:32,952][105692] Updated weights for policy 0, policy_version 912800 (0.0010) [2023-12-26 22:00:33,004][105692] Updated weights for policy 0, policy_version 912810 (0.0009) [2023-12-26 22:00:33,055][105620] Updated weights for policy 1, policy_version 912753 (0.0008) [2023-12-26 22:00:33,116][105620] Updated weights for policy 1, policy_version 912763 (0.0009) [2023-12-26 22:00:33,175][105620] Updated weights for policy 1, policy_version 912773 (0.0009) [2023-12-26 22:00:33,235][105620] Updated weights for policy 1, policy_version 912783 (0.0009) [2023-12-26 22:00:33,779][105692] Updated weights for policy 0, policy_version 912820 (0.0009) [2023-12-26 22:00:33,841][105692] Updated weights for policy 0, policy_version 912830 (0.0009) [2023-12-26 22:00:33,903][105692] Updated weights for policy 0, policy_version 912840 (0.0009) [2023-12-26 22:00:33,981][105620] Updated weights for policy 1, policy_version 912793 (0.0008) [2023-12-26 22:00:34,033][105620] Updated weights for policy 1, policy_version 912803 (0.0009) [2023-12-26 22:00:34,084][105620] Updated weights for policy 1, policy_version 912813 (0.0009) [2023-12-26 22:00:34,709][105692] Updated weights for policy 0, policy_version 912850 (0.0008) [2023-12-26 22:00:34,768][105692] Updated weights for policy 0, policy_version 912860 (0.0009) [2023-12-26 22:00:34,834][105692] Updated weights for policy 0, policy_version 912870 (0.0009) [2023-12-26 22:00:34,837][105620] Updated weights for policy 1, policy_version 912823 (0.0009) [2023-12-26 22:00:34,897][105692] Updated weights for policy 0, policy_version 912880 (0.0007) [2023-12-26 22:00:34,903][105620] Updated weights for policy 1, policy_version 912833 (0.0008) [2023-12-26 22:00:34,967][105620] Updated weights for policy 1, policy_version 912843 (0.0009) [2023-12-26 22:00:35,651][105620] Updated weights for policy 1, policy_version 912853 (0.0009) [2023-12-26 22:00:35,711][105620] Updated weights for policy 1, policy_version 912863 (0.0008) [2023-12-26 22:00:35,725][105692] Updated weights for policy 0, policy_version 912890 (0.0008) [2023-12-26 22:00:35,770][105620] Updated weights for policy 1, policy_version 912873 (0.0006) [2023-12-26 22:00:35,780][105692] Updated weights for policy 0, policy_version 912900 (0.0006) [2023-12-26 22:00:35,835][105692] Updated weights for policy 0, policy_version 912910 (0.0007) [2023-12-26 22:00:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 18978.1, 300 sec: 19355.3). Total num frames: 467468288. Throughput: 0: 9319.6, 1: 9516.9. Samples: 467456104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:00:36,063][104569] Avg episode reward: [(0, '9347.737'), (1, '8388.114')] [2023-12-26 22:00:36,584][105620] Updated weights for policy 1, policy_version 912883 (0.0008) [2023-12-26 22:00:36,591][105692] Updated weights for policy 0, policy_version 912920 (0.0009) [2023-12-26 22:00:36,650][105620] Updated weights for policy 1, policy_version 912893 (0.0006) [2023-12-26 22:00:36,660][105692] Updated weights for policy 0, policy_version 912930 (0.0009) [2023-12-26 22:00:36,713][105620] Updated weights for policy 1, policy_version 912903 (0.0006) [2023-12-26 22:00:36,723][105692] Updated weights for policy 0, policy_version 912940 (0.0009) [2023-12-26 22:00:37,383][105620] Updated weights for policy 1, policy_version 912913 (0.0006) [2023-12-26 22:00:37,455][105620] Updated weights for policy 1, policy_version 912923 (0.0009) [2023-12-26 22:00:37,483][105692] Updated weights for policy 0, policy_version 912950 (0.0007) [2023-12-26 22:00:37,525][105620] Updated weights for policy 1, policy_version 912933 (0.0007) [2023-12-26 22:00:37,552][105692] Updated weights for policy 0, policy_version 912960 (0.0007) [2023-12-26 22:00:37,579][105620] Updated weights for policy 1, policy_version 912943 (0.0008) [2023-12-26 22:00:37,618][105692] Updated weights for policy 0, policy_version 912970 (0.0007) [2023-12-26 22:00:38,169][105692] Updated weights for policy 0, policy_version 912980 (0.0009) [2023-12-26 22:00:38,180][105620] Updated weights for policy 1, policy_version 912953 (0.0007) [2023-12-26 22:00:38,232][105692] Updated weights for policy 0, policy_version 912990 (0.0006) [2023-12-26 22:00:38,240][105620] Updated weights for policy 1, policy_version 912963 (0.0009) [2023-12-26 22:00:38,292][105692] Updated weights for policy 0, policy_version 913000 (0.0008) [2023-12-26 22:00:38,307][105620] Updated weights for policy 1, policy_version 912973 (0.0007) [2023-12-26 22:00:38,980][105620] Updated weights for policy 1, policy_version 912983 (0.0010) [2023-12-26 22:00:38,985][105692] Updated weights for policy 0, policy_version 913010 (0.0009) [2023-12-26 22:00:39,045][105620] Updated weights for policy 1, policy_version 912993 (0.0011) [2023-12-26 22:00:39,048][105692] Updated weights for policy 0, policy_version 913020 (0.0007) [2023-12-26 22:00:39,101][105692] Updated weights for policy 0, policy_version 913030 (0.0005) [2023-12-26 22:00:39,102][105620] Updated weights for policy 1, policy_version 913003 (0.0011) [2023-12-26 22:00:39,155][105692] Updated weights for policy 0, policy_version 913040 (0.0008) [2023-12-26 22:00:39,847][105620] Updated weights for policy 1, policy_version 913013 (0.0009) [2023-12-26 22:00:39,910][105620] Updated weights for policy 1, policy_version 913023 (0.0011) [2023-12-26 22:00:39,979][105620] Updated weights for policy 1, policy_version 913033 (0.0011) [2023-12-26 22:00:39,982][105692] Updated weights for policy 0, policy_version 913050 (0.0008) [2023-12-26 22:00:40,052][105692] Updated weights for policy 0, policy_version 913060 (0.0008) [2023-12-26 22:00:40,114][105692] Updated weights for policy 0, policy_version 913070 (0.0008) [2023-12-26 22:00:40,686][105620] Updated weights for policy 1, policy_version 913043 (0.0010) [2023-12-26 22:00:40,751][105620] Updated weights for policy 1, policy_version 913053 (0.0007) [2023-12-26 22:00:40,814][105620] Updated weights for policy 1, policy_version 913063 (0.0010) [2023-12-26 22:00:40,930][105692] Updated weights for policy 0, policy_version 913080 (0.0009) [2023-12-26 22:00:40,993][105692] Updated weights for policy 0, policy_version 913090 (0.0009) [2023-12-26 22:00:41,060][105692] Updated weights for policy 0, policy_version 913100 (0.0009) [2023-12-26 22:00:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 18978.1, 300 sec: 19355.3). Total num frames: 467558400. Throughput: 0: 9244.8, 1: 9559.7. Samples: 467570408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:00:41,062][104569] Avg episode reward: [(0, '9347.593'), (1, '8642.604')] [2023-12-26 22:00:41,542][105620] Updated weights for policy 1, policy_version 913073 (0.0011) [2023-12-26 22:00:41,603][105620] Updated weights for policy 1, policy_version 913083 (0.0006) [2023-12-26 22:00:41,676][105620] Updated weights for policy 1, policy_version 913093 (0.0007) [2023-12-26 22:00:41,744][105620] Updated weights for policy 1, policy_version 913103 (0.0007) [2023-12-26 22:00:42,002][105692] Updated weights for policy 0, policy_version 913110 (0.0007) [2023-12-26 22:00:42,068][105692] Updated weights for policy 0, policy_version 913120 (0.0006) [2023-12-26 22:00:42,138][105692] Updated weights for policy 0, policy_version 913130 (0.0006) [2023-12-26 22:00:42,530][105620] Updated weights for policy 1, policy_version 913113 (0.0008) [2023-12-26 22:00:42,594][105620] Updated weights for policy 1, policy_version 913123 (0.0009) [2023-12-26 22:00:42,653][105620] Updated weights for policy 1, policy_version 913133 (0.0008) [2023-12-26 22:00:42,809][105692] Updated weights for policy 0, policy_version 913140 (0.0006) [2023-12-26 22:00:42,868][105692] Updated weights for policy 0, policy_version 913150 (0.0005) [2023-12-26 22:00:42,930][105692] Updated weights for policy 0, policy_version 913160 (0.0005) [2023-12-26 22:00:43,298][105620] Updated weights for policy 1, policy_version 913143 (0.0006) [2023-12-26 22:00:43,359][105620] Updated weights for policy 1, policy_version 913153 (0.0010) [2023-12-26 22:00:43,408][105620] Updated weights for policy 1, policy_version 913163 (0.0011) [2023-12-26 22:00:43,487][105692] Updated weights for policy 0, policy_version 913170 (0.0006) [2023-12-26 22:00:43,555][105692] Updated weights for policy 0, policy_version 913180 (0.0011) [2023-12-26 22:00:43,621][105692] Updated weights for policy 0, policy_version 913190 (0.0011) [2023-12-26 22:00:43,686][105692] Updated weights for policy 0, policy_version 913200 (0.0011) [2023-12-26 22:00:44,120][105620] Updated weights for policy 1, policy_version 913173 (0.0009) [2023-12-26 22:00:44,180][105620] Updated weights for policy 1, policy_version 913183 (0.0008) [2023-12-26 22:00:44,225][105620] Updated weights for policy 1, policy_version 913193 (0.0010) [2023-12-26 22:00:44,417][105692] Updated weights for policy 0, policy_version 913210 (0.0010) [2023-12-26 22:00:44,481][105692] Updated weights for policy 0, policy_version 913220 (0.0010) [2023-12-26 22:00:44,542][105692] Updated weights for policy 0, policy_version 913230 (0.0010) [2023-12-26 22:00:44,938][105620] Updated weights for policy 1, policy_version 913203 (0.0011) [2023-12-26 22:00:45,004][105620] Updated weights for policy 1, policy_version 913213 (0.0011) [2023-12-26 22:00:45,074][105620] Updated weights for policy 1, policy_version 913223 (0.0011) [2023-12-26 22:00:45,310][105692] Updated weights for policy 0, policy_version 913240 (0.0011) [2023-12-26 22:00:45,367][105692] Updated weights for policy 0, policy_version 913250 (0.0011) [2023-12-26 22:00:45,431][105692] Updated weights for policy 0, policy_version 913260 (0.0011) [2023-12-26 22:00:45,767][105620] Updated weights for policy 1, policy_version 913233 (0.0009) [2023-12-26 22:00:45,825][105620] Updated weights for policy 1, policy_version 913243 (0.0009) [2023-12-26 22:00:45,878][105620] Updated weights for policy 1, policy_version 913253 (0.0011) [2023-12-26 22:00:45,941][105620] Updated weights for policy 1, policy_version 913263 (0.0011) [2023-12-26 22:00:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18841.7, 300 sec: 19355.3). Total num frames: 467656704. Throughput: 0: 9176.6, 1: 9470.3. Samples: 467627408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:00:46,062][104569] Avg episode reward: [(0, '9256.327'), (1, '8646.105')] [2023-12-26 22:00:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000913264_233824256.pth... [2023-12-26 22:00:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000913264_233832448.pth... [2023-12-26 22:00:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000912144_233537536.pth [2023-12-26 22:00:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000912208_233562112.pth [2023-12-26 22:00:46,186][105692] Updated weights for policy 0, policy_version 913270 (0.0011) [2023-12-26 22:00:46,240][105692] Updated weights for policy 0, policy_version 913280 (0.0010) [2023-12-26 22:00:46,302][105692] Updated weights for policy 0, policy_version 913290 (0.0010) [2023-12-26 22:00:46,670][105620] Updated weights for policy 1, policy_version 913273 (0.0010) [2023-12-26 22:00:46,728][105620] Updated weights for policy 1, policy_version 913283 (0.0010) [2023-12-26 22:00:46,783][105620] Updated weights for policy 1, policy_version 913293 (0.0010) [2023-12-26 22:00:46,966][105692] Updated weights for policy 0, policy_version 913300 (0.0010) [2023-12-26 22:00:47,032][105692] Updated weights for policy 0, policy_version 913310 (0.0011) [2023-12-26 22:00:47,076][105692] Updated weights for policy 0, policy_version 913320 (0.0010) [2023-12-26 22:00:47,407][105620] Updated weights for policy 1, policy_version 913303 (0.0007) [2023-12-26 22:00:47,466][105620] Updated weights for policy 1, policy_version 913313 (0.0006) [2023-12-26 22:00:47,530][105620] Updated weights for policy 1, policy_version 913323 (0.0010) [2023-12-26 22:00:47,839][105692] Updated weights for policy 0, policy_version 913330 (0.0010) [2023-12-26 22:00:47,906][105692] Updated weights for policy 0, policy_version 913340 (0.0009) [2023-12-26 22:00:47,960][105692] Updated weights for policy 0, policy_version 913350 (0.0009) [2023-12-26 22:00:48,019][105692] Updated weights for policy 0, policy_version 913360 (0.0009) [2023-12-26 22:00:48,233][105620] Updated weights for policy 1, policy_version 913333 (0.0011) [2023-12-26 22:00:48,281][105620] Updated weights for policy 1, policy_version 913343 (0.0010) [2023-12-26 22:00:48,337][105620] Updated weights for policy 1, policy_version 913353 (0.0010) [2023-12-26 22:00:48,775][105692] Updated weights for policy 0, policy_version 913370 (0.0007) [2023-12-26 22:00:48,844][105692] Updated weights for policy 0, policy_version 913380 (0.0008) [2023-12-26 22:00:48,915][105692] Updated weights for policy 0, policy_version 913390 (0.0007) [2023-12-26 22:00:49,126][105620] Updated weights for policy 1, policy_version 913363 (0.0010) [2023-12-26 22:00:49,194][105620] Updated weights for policy 1, policy_version 913373 (0.0010) [2023-12-26 22:00:49,270][105620] Updated weights for policy 1, policy_version 913383 (0.0010) [2023-12-26 22:00:49,762][105692] Updated weights for policy 0, policy_version 913400 (0.0009) [2023-12-26 22:00:49,819][105692] Updated weights for policy 0, policy_version 913410 (0.0009) [2023-12-26 22:00:49,896][105692] Updated weights for policy 0, policy_version 913420 (0.0009) [2023-12-26 22:00:50,046][105620] Updated weights for policy 1, policy_version 913393 (0.0010) [2023-12-26 22:00:50,106][105620] Updated weights for policy 1, policy_version 913403 (0.0009) [2023-12-26 22:00:50,172][105620] Updated weights for policy 1, policy_version 913413 (0.0008) [2023-12-26 22:00:50,240][105620] Updated weights for policy 1, policy_version 913423 (0.0009) [2023-12-26 22:00:50,671][105692] Updated weights for policy 0, policy_version 913430 (0.0009) [2023-12-26 22:00:50,737][105692] Updated weights for policy 0, policy_version 913440 (0.0009) [2023-12-26 22:00:50,800][105692] Updated weights for policy 0, policy_version 913450 (0.0009) [2023-12-26 22:00:51,013][105620] Updated weights for policy 1, policy_version 913433 (0.0009) [2023-12-26 22:00:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 18841.6, 300 sec: 19327.6). Total num frames: 467746816. Throughput: 0: 9091.4, 1: 9483.9. Samples: 467740056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:00:51,063][104569] Avg episode reward: [(0, '8909.119'), (1, '8911.855')] [2023-12-26 22:00:51,082][105620] Updated weights for policy 1, policy_version 913443 (0.0009) [2023-12-26 22:00:51,151][105620] Updated weights for policy 1, policy_version 913453 (0.0008) [2023-12-26 22:00:51,604][105692] Updated weights for policy 0, policy_version 913460 (0.0009) [2023-12-26 22:00:51,677][105692] Updated weights for policy 0, policy_version 913470 (0.0009) [2023-12-26 22:00:51,747][105692] Updated weights for policy 0, policy_version 913480 (0.0009) [2023-12-26 22:00:51,931][105620] Updated weights for policy 1, policy_version 913463 (0.0009) [2023-12-26 22:00:51,993][105620] Updated weights for policy 1, policy_version 913473 (0.0010) [2023-12-26 22:00:52,062][105620] Updated weights for policy 1, policy_version 913483 (0.0010) [2023-12-26 22:00:52,471][105692] Updated weights for policy 0, policy_version 913490 (0.0009) [2023-12-26 22:00:52,527][105692] Updated weights for policy 0, policy_version 913500 (0.0007) [2023-12-26 22:00:52,595][105692] Updated weights for policy 0, policy_version 913510 (0.0009) [2023-12-26 22:00:52,663][105692] Updated weights for policy 0, policy_version 913520 (0.0008) [2023-12-26 22:00:52,867][105620] Updated weights for policy 1, policy_version 913493 (0.0007) [2023-12-26 22:00:52,923][105620] Updated weights for policy 1, policy_version 913503 (0.0008) [2023-12-26 22:00:52,987][105620] Updated weights for policy 1, policy_version 913513 (0.0008) [2023-12-26 22:00:53,429][105692] Updated weights for policy 0, policy_version 913530 (0.0009) [2023-12-26 22:00:53,493][105692] Updated weights for policy 0, policy_version 913540 (0.0009) [2023-12-26 22:00:53,556][105692] Updated weights for policy 0, policy_version 913550 (0.0010) [2023-12-26 22:00:53,722][105620] Updated weights for policy 1, policy_version 913523 (0.0009) [2023-12-26 22:00:53,783][105620] Updated weights for policy 1, policy_version 913533 (0.0008) [2023-12-26 22:00:53,846][105620] Updated weights for policy 1, policy_version 913543 (0.0008) [2023-12-26 22:00:54,284][105692] Updated weights for policy 0, policy_version 913560 (0.0010) [2023-12-26 22:00:54,343][105692] Updated weights for policy 0, policy_version 913570 (0.0008) [2023-12-26 22:00:54,402][105692] Updated weights for policy 0, policy_version 913580 (0.0009) [2023-12-26 22:00:54,571][105620] Updated weights for policy 1, policy_version 913553 (0.0008) [2023-12-26 22:00:54,623][105620] Updated weights for policy 1, policy_version 913563 (0.0006) [2023-12-26 22:00:54,677][105620] Updated weights for policy 1, policy_version 913573 (0.0008) [2023-12-26 22:00:54,735][105620] Updated weights for policy 1, policy_version 913583 (0.0009) [2023-12-26 22:00:55,133][105692] Updated weights for policy 0, policy_version 913590 (0.0009) [2023-12-26 22:00:55,201][105692] Updated weights for policy 0, policy_version 913600 (0.0008) [2023-12-26 22:00:55,274][105692] Updated weights for policy 0, policy_version 913610 (0.0010) [2023-12-26 22:00:55,528][105620] Updated weights for policy 1, policy_version 913593 (0.0008) [2023-12-26 22:00:55,592][105620] Updated weights for policy 1, policy_version 913603 (0.0009) [2023-12-26 22:00:55,652][105620] Updated weights for policy 1, policy_version 913613 (0.0010) [2023-12-26 22:00:56,019][105692] Updated weights for policy 0, policy_version 913620 (0.0009) [2023-12-26 22:00:56,062][104569] Fps is (10 sec: 18022.5, 60 sec: 18705.1, 300 sec: 19299.8). Total num frames: 467836928. Throughput: 0: 9031.2, 1: 9487.4. Samples: 467847700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:00:56,062][104569] Avg episode reward: [(0, '8856.447'), (1, '8729.874')] [2023-12-26 22:00:56,074][105692] Updated weights for policy 0, policy_version 913630 (0.0009) [2023-12-26 22:00:56,130][105692] Updated weights for policy 0, policy_version 913640 (0.0009) [2023-12-26 22:00:56,402][105620] Updated weights for policy 1, policy_version 913623 (0.0009) [2023-12-26 22:00:56,453][105620] Updated weights for policy 1, policy_version 913633 (0.0008) [2023-12-26 22:00:56,511][105620] Updated weights for policy 1, policy_version 913643 (0.0009) [2023-12-26 22:00:56,873][105692] Updated weights for policy 0, policy_version 913650 (0.0009) [2023-12-26 22:00:56,931][105692] Updated weights for policy 0, policy_version 913660 (0.0009) [2023-12-26 22:00:56,994][105692] Updated weights for policy 0, policy_version 913670 (0.0009) [2023-12-26 22:00:57,048][105692] Updated weights for policy 0, policy_version 913680 (0.0009) [2023-12-26 22:00:57,303][105620] Updated weights for policy 1, policy_version 913653 (0.0009) [2023-12-26 22:00:57,365][105620] Updated weights for policy 1, policy_version 913663 (0.0009) [2023-12-26 22:00:57,427][105620] Updated weights for policy 1, policy_version 913673 (0.0008) [2023-12-26 22:00:57,785][105692] Updated weights for policy 0, policy_version 913690 (0.0009) [2023-12-26 22:00:57,850][105692] Updated weights for policy 0, policy_version 913700 (0.0008) [2023-12-26 22:00:57,915][105692] Updated weights for policy 0, policy_version 913710 (0.0008) [2023-12-26 22:00:58,206][105620] Updated weights for policy 1, policy_version 913683 (0.0010) [2023-12-26 22:00:58,269][105620] Updated weights for policy 1, policy_version 913693 (0.0009) [2023-12-26 22:00:58,358][105620] Updated weights for policy 1, policy_version 913705 (0.0009) [2023-12-26 22:00:58,774][105692] Updated weights for policy 0, policy_version 913720 (0.0009) [2023-12-26 22:00:58,838][105692] Updated weights for policy 0, policy_version 913730 (0.0009) [2023-12-26 22:00:58,912][105692] Updated weights for policy 0, policy_version 913740 (0.0009) [2023-12-26 22:00:59,196][105620] Updated weights for policy 1, policy_version 913715 (0.0008) [2023-12-26 22:00:59,261][105620] Updated weights for policy 1, policy_version 913725 (0.0008) [2023-12-26 22:00:59,325][105620] Updated weights for policy 1, policy_version 913735 (0.0008) [2023-12-26 22:00:59,672][105692] Updated weights for policy 0, policy_version 913750 (0.0007) [2023-12-26 22:00:59,732][105692] Updated weights for policy 0, policy_version 913760 (0.0007) [2023-12-26 22:00:59,796][105692] Updated weights for policy 0, policy_version 913770 (0.0009) [2023-12-26 22:01:00,130][105620] Updated weights for policy 1, policy_version 913745 (0.0008) [2023-12-26 22:01:00,186][105620] Updated weights for policy 1, policy_version 913755 (0.0009) [2023-12-26 22:01:00,249][105620] Updated weights for policy 1, policy_version 913765 (0.0009) [2023-12-26 22:01:00,309][105620] Updated weights for policy 1, policy_version 913775 (0.0009) [2023-12-26 22:01:00,499][105692] Updated weights for policy 0, policy_version 913780 (0.0007) [2023-12-26 22:01:00,565][105692] Updated weights for policy 0, policy_version 913790 (0.0008) [2023-12-26 22:01:00,625][105692] Updated weights for policy 0, policy_version 913800 (0.0009) [2023-12-26 22:01:01,062][104569] Fps is (10 sec: 18022.6, 60 sec: 18432.0, 300 sec: 19272.0). Total num frames: 467927040. Throughput: 0: 9082.5, 1: 9429.6. Samples: 467901980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:01:01,062][104569] Avg episode reward: [(0, '8302.988'), (1, '8910.181')] [2023-12-26 22:01:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000913808_233971712.pth... [2023-12-26 22:01:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000913776_233955328.pth... [2023-12-26 22:01:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000912688_233676800.pth [2023-12-26 22:01:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000912720_233693184.pth [2023-12-26 22:01:01,151][105620] Updated weights for policy 1, policy_version 913785 (0.0009) [2023-12-26 22:01:01,216][105620] Updated weights for policy 1, policy_version 913795 (0.0009) [2023-12-26 22:01:01,278][105620] Updated weights for policy 1, policy_version 913805 (0.0009) [2023-12-26 22:01:01,286][105692] Updated weights for policy 0, policy_version 913810 (0.0008) [2023-12-26 22:01:01,353][105692] Updated weights for policy 0, policy_version 913820 (0.0009) [2023-12-26 22:01:01,418][105692] Updated weights for policy 0, policy_version 913830 (0.0009) [2023-12-26 22:01:01,480][105692] Updated weights for policy 0, policy_version 913840 (0.0009) [2023-12-26 22:01:02,089][105620] Updated weights for policy 1, policy_version 913815 (0.0009) [2023-12-26 22:01:02,149][105620] Updated weights for policy 1, policy_version 913825 (0.0008) [2023-12-26 22:01:02,206][105620] Updated weights for policy 1, policy_version 913835 (0.0005) [2023-12-26 22:01:02,246][105692] Updated weights for policy 0, policy_version 913850 (0.0008) [2023-12-26 22:01:02,306][105692] Updated weights for policy 0, policy_version 913860 (0.0008) [2023-12-26 22:01:02,372][105692] Updated weights for policy 0, policy_version 913870 (0.0009) [2023-12-26 22:01:02,947][105620] Updated weights for policy 1, policy_version 913845 (0.0007) [2023-12-26 22:01:03,003][105620] Updated weights for policy 1, policy_version 913855 (0.0009) [2023-12-26 22:01:03,067][105620] Updated weights for policy 1, policy_version 913865 (0.0008) [2023-12-26 22:01:03,094][105692] Updated weights for policy 0, policy_version 913880 (0.0007) [2023-12-26 22:01:03,157][105692] Updated weights for policy 0, policy_version 913890 (0.0008) [2023-12-26 22:01:03,217][105692] Updated weights for policy 0, policy_version 913900 (0.0006) [2023-12-26 22:01:03,824][105620] Updated weights for policy 1, policy_version 913875 (0.0007) [2023-12-26 22:01:03,891][105620] Updated weights for policy 1, policy_version 913885 (0.0008) [2023-12-26 22:01:03,942][105692] Updated weights for policy 0, policy_version 913910 (0.0008) [2023-12-26 22:01:03,950][105620] Updated weights for policy 1, policy_version 913895 (0.0006) [2023-12-26 22:01:04,002][105692] Updated weights for policy 0, policy_version 913920 (0.0008) [2023-12-26 22:01:04,068][105692] Updated weights for policy 0, policy_version 913930 (0.0009) [2023-12-26 22:01:04,616][105620] Updated weights for policy 1, policy_version 913905 (0.0006) [2023-12-26 22:01:04,680][105620] Updated weights for policy 1, policy_version 913915 (0.0006) [2023-12-26 22:01:04,742][105620] Updated weights for policy 1, policy_version 913925 (0.0009) [2023-12-26 22:01:04,802][105620] Updated weights for policy 1, policy_version 913935 (0.0009) [2023-12-26 22:01:04,892][105692] Updated weights for policy 0, policy_version 913940 (0.0008) [2023-12-26 22:01:04,943][105692] Updated weights for policy 0, policy_version 913950 (0.0009) [2023-12-26 22:01:04,993][105692] Updated weights for policy 0, policy_version 913960 (0.0008) [2023-12-26 22:01:05,552][105620] Updated weights for policy 1, policy_version 913945 (0.0009) [2023-12-26 22:01:05,613][105620] Updated weights for policy 1, policy_version 913955 (0.0009) [2023-12-26 22:01:05,668][105620] Updated weights for policy 1, policy_version 913965 (0.0008) [2023-12-26 22:01:05,772][105692] Updated weights for policy 0, policy_version 913970 (0.0009) [2023-12-26 22:01:05,823][105692] Updated weights for policy 0, policy_version 913980 (0.0008) [2023-12-26 22:01:05,879][105692] Updated weights for policy 0, policy_version 913990 (0.0009) [2023-12-26 22:01:05,926][105692] Updated weights for policy 0, policy_version 914000 (0.0009) [2023-12-26 22:01:06,062][104569] Fps is (10 sec: 18841.4, 60 sec: 18568.5, 300 sec: 19272.0). Total num frames: 468025344. Throughput: 0: 9164.8, 1: 9371.9. Samples: 468011908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:01:06,063][104569] Avg episode reward: [(0, '8445.469'), (1, '9268.581')] [2023-12-26 22:01:06,424][105620] Updated weights for policy 1, policy_version 913975 (0.0009) [2023-12-26 22:01:06,486][105620] Updated weights for policy 1, policy_version 913985 (0.0009) [2023-12-26 22:01:06,542][105620] Updated weights for policy 1, policy_version 913995 (0.0006) [2023-12-26 22:01:06,775][105692] Updated weights for policy 0, policy_version 914010 (0.0011) [2023-12-26 22:01:06,835][105692] Updated weights for policy 0, policy_version 914020 (0.0009) [2023-12-26 22:01:06,895][105692] Updated weights for policy 0, policy_version 914030 (0.0009) [2023-12-26 22:01:07,150][105620] Updated weights for policy 1, policy_version 914005 (0.0008) [2023-12-26 22:01:07,217][105620] Updated weights for policy 1, policy_version 914015 (0.0009) [2023-12-26 22:01:07,279][105620] Updated weights for policy 1, policy_version 914025 (0.0009) [2023-12-26 22:01:07,721][105692] Updated weights for policy 0, policy_version 914040 (0.0009) [2023-12-26 22:01:07,778][105692] Updated weights for policy 0, policy_version 914050 (0.0009) [2023-12-26 22:01:07,832][105692] Updated weights for policy 0, policy_version 914060 (0.0009) [2023-12-26 22:01:08,001][105620] Updated weights for policy 1, policy_version 914035 (0.0006) [2023-12-26 22:01:08,058][105620] Updated weights for policy 1, policy_version 914045 (0.0010) [2023-12-26 22:01:08,124][105620] Updated weights for policy 1, policy_version 914055 (0.0010) [2023-12-26 22:01:08,572][105692] Updated weights for policy 0, policy_version 914070 (0.0009) [2023-12-26 22:01:08,619][105692] Updated weights for policy 0, policy_version 914080 (0.0009) [2023-12-26 22:01:08,677][105692] Updated weights for policy 0, policy_version 914090 (0.0008) [2023-12-26 22:01:08,925][105620] Updated weights for policy 1, policy_version 914065 (0.0010) [2023-12-26 22:01:08,977][105620] Updated weights for policy 1, policy_version 914075 (0.0009) [2023-12-26 22:01:09,033][105620] Updated weights for policy 1, policy_version 914085 (0.0009) [2023-12-26 22:01:09,088][105620] Updated weights for policy 1, policy_version 914095 (0.0009) [2023-12-26 22:01:09,481][105692] Updated weights for policy 0, policy_version 914100 (0.0009) [2023-12-26 22:01:09,542][105692] Updated weights for policy 0, policy_version 914110 (0.0008) [2023-12-26 22:01:09,603][105692] Updated weights for policy 0, policy_version 914120 (0.0010) [2023-12-26 22:01:09,849][105620] Updated weights for policy 1, policy_version 914105 (0.0008) [2023-12-26 22:01:09,911][105620] Updated weights for policy 1, policy_version 914115 (0.0008) [2023-12-26 22:01:09,975][105620] Updated weights for policy 1, policy_version 914125 (0.0009) [2023-12-26 22:01:10,376][105692] Updated weights for policy 0, policy_version 914130 (0.0010) [2023-12-26 22:01:10,424][105692] Updated weights for policy 0, policy_version 914140 (0.0008) [2023-12-26 22:01:10,484][105692] Updated weights for policy 0, policy_version 914150 (0.0007) [2023-12-26 22:01:10,532][105692] Updated weights for policy 0, policy_version 914160 (0.0005) [2023-12-26 22:01:10,775][105620] Updated weights for policy 1, policy_version 914135 (0.0009) [2023-12-26 22:01:10,832][105620] Updated weights for policy 1, policy_version 914145 (0.0009) [2023-12-26 22:01:10,898][105620] Updated weights for policy 1, policy_version 914155 (0.0009) [2023-12-26 22:01:11,062][104569] Fps is (10 sec: 18841.3, 60 sec: 18432.0, 300 sec: 19244.3). Total num frames: 468115456. Throughput: 0: 9246.3, 1: 9326.5. Samples: 468121836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:01:11,063][104569] Avg episode reward: [(0, '8688.407'), (1, '8999.128')] [2023-12-26 22:01:11,315][105692] Updated weights for policy 0, policy_version 914170 (0.0009) [2023-12-26 22:01:11,386][105692] Updated weights for policy 0, policy_version 914180 (0.0010) [2023-12-26 22:01:11,454][105692] Updated weights for policy 0, policy_version 914190 (0.0009) [2023-12-26 22:01:11,692][105620] Updated weights for policy 1, policy_version 914165 (0.0009) [2023-12-26 22:01:11,759][105620] Updated weights for policy 1, policy_version 914175 (0.0009) [2023-12-26 22:01:11,811][105620] Updated weights for policy 1, policy_version 914185 (0.0008) [2023-12-26 22:01:12,266][105692] Updated weights for policy 0, policy_version 914200 (0.0010) [2023-12-26 22:01:12,335][105692] Updated weights for policy 0, policy_version 914210 (0.0008) [2023-12-26 22:01:12,403][105692] Updated weights for policy 0, policy_version 914220 (0.0007) [2023-12-26 22:01:12,567][105620] Updated weights for policy 1, policy_version 914195 (0.0009) [2023-12-26 22:01:12,621][105620] Updated weights for policy 1, policy_version 914205 (0.0009) [2023-12-26 22:01:12,690][105620] Updated weights for policy 1, policy_version 914215 (0.0009) [2023-12-26 22:01:13,143][105692] Updated weights for policy 0, policy_version 914230 (0.0009) [2023-12-26 22:01:13,205][105692] Updated weights for policy 0, policy_version 914240 (0.0009) [2023-12-26 22:01:13,267][105692] Updated weights for policy 0, policy_version 914250 (0.0009) [2023-12-26 22:01:13,439][105620] Updated weights for policy 1, policy_version 914225 (0.0009) [2023-12-26 22:01:13,490][105620] Updated weights for policy 1, policy_version 914235 (0.0009) [2023-12-26 22:01:13,545][105620] Updated weights for policy 1, policy_version 914245 (0.0009) [2023-12-26 22:01:13,598][105620] Updated weights for policy 1, policy_version 914255 (0.0008) [2023-12-26 22:01:14,005][105692] Updated weights for policy 0, policy_version 914260 (0.0007) [2023-12-26 22:01:14,071][105692] Updated weights for policy 0, policy_version 914270 (0.0010) [2023-12-26 22:01:14,130][105692] Updated weights for policy 0, policy_version 914280 (0.0009) [2023-12-26 22:01:14,294][105620] Updated weights for policy 1, policy_version 914265 (0.0008) [2023-12-26 22:01:14,347][105620] Updated weights for policy 1, policy_version 914275 (0.0005) [2023-12-26 22:01:14,405][105620] Updated weights for policy 1, policy_version 914285 (0.0005) [2023-12-26 22:01:14,890][105692] Updated weights for policy 0, policy_version 914290 (0.0009) [2023-12-26 22:01:14,940][105692] Updated weights for policy 0, policy_version 914300 (0.0011) [2023-12-26 22:01:14,997][105692] Updated weights for policy 0, policy_version 914310 (0.0011) [2023-12-26 22:01:15,055][105692] Updated weights for policy 0, policy_version 914320 (0.0011) [2023-12-26 22:01:15,071][105620] Updated weights for policy 1, policy_version 914295 (0.0007) [2023-12-26 22:01:15,134][105620] Updated weights for policy 1, policy_version 914305 (0.0008) [2023-12-26 22:01:15,192][105620] Updated weights for policy 1, policy_version 914315 (0.0008) [2023-12-26 22:01:15,814][105620] Updated weights for policy 1, policy_version 914325 (0.0007) [2023-12-26 22:01:15,827][105692] Updated weights for policy 0, policy_version 914330 (0.0011) [2023-12-26 22:01:15,862][105620] Updated weights for policy 1, policy_version 914335 (0.0006) [2023-12-26 22:01:15,886][105692] Updated weights for policy 0, policy_version 914340 (0.0011) [2023-12-26 22:01:15,912][105620] Updated weights for policy 1, policy_version 914345 (0.0009) [2023-12-26 22:01:15,942][105692] Updated weights for policy 0, policy_version 914350 (0.0010) [2023-12-26 22:01:16,062][104569] Fps is (10 sec: 18841.8, 60 sec: 18705.1, 300 sec: 19244.3). Total num frames: 468213760. Throughput: 0: 9194.1, 1: 9297.5. Samples: 468176360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:01:16,062][104569] Avg episode reward: [(0, '8689.095'), (1, '9003.945')] [2023-12-26 22:01:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000914352_234110976.pth... [2023-12-26 22:01:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000914352_234102784.pth... [2023-12-26 22:01:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000913264_233832448.pth [2023-12-26 22:01:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000913264_233824256.pth [2023-12-26 22:01:16,598][105620] Updated weights for policy 1, policy_version 914355 (0.0006) [2023-12-26 22:01:16,635][105692] Updated weights for policy 0, policy_version 914360 (0.0009) [2023-12-26 22:01:16,652][105620] Updated weights for policy 1, policy_version 914365 (0.0009) [2023-12-26 22:01:16,682][105692] Updated weights for policy 0, policy_version 914370 (0.0008) [2023-12-26 22:01:16,697][105620] Updated weights for policy 1, policy_version 914375 (0.0007) [2023-12-26 22:01:16,735][105692] Updated weights for policy 0, policy_version 914380 (0.0008) [2023-12-26 22:01:17,348][105620] Updated weights for policy 1, policy_version 914385 (0.0005) [2023-12-26 22:01:17,403][105620] Updated weights for policy 1, policy_version 914395 (0.0005) [2023-12-26 22:01:17,460][105620] Updated weights for policy 1, policy_version 914405 (0.0006) [2023-12-26 22:01:17,504][105692] Updated weights for policy 0, policy_version 914390 (0.0010) [2023-12-26 22:01:17,514][105620] Updated weights for policy 1, policy_version 914415 (0.0006) [2023-12-26 22:01:17,552][105692] Updated weights for policy 0, policy_version 914400 (0.0010) [2023-12-26 22:01:17,609][105692] Updated weights for policy 0, policy_version 914410 (0.0010) [2023-12-26 22:01:18,115][105620] Updated weights for policy 1, policy_version 914425 (0.0007) [2023-12-26 22:01:18,176][105620] Updated weights for policy 1, policy_version 914435 (0.0010) [2023-12-26 22:01:18,220][105620] Updated weights for policy 1, policy_version 914445 (0.0010) [2023-12-26 22:01:18,388][105692] Updated weights for policy 0, policy_version 914420 (0.0010) [2023-12-26 22:01:18,441][105692] Updated weights for policy 0, policy_version 914430 (0.0010) [2023-12-26 22:01:18,504][105692] Updated weights for policy 0, policy_version 914440 (0.0010) [2023-12-26 22:01:18,961][105620] Updated weights for policy 1, policy_version 914455 (0.0007) [2023-12-26 22:01:19,029][105620] Updated weights for policy 1, policy_version 914465 (0.0006) [2023-12-26 22:01:19,096][105620] Updated weights for policy 1, policy_version 914475 (0.0005) [2023-12-26 22:01:19,232][105692] Updated weights for policy 0, policy_version 914450 (0.0011) [2023-12-26 22:01:19,290][105692] Updated weights for policy 0, policy_version 914460 (0.0010) [2023-12-26 22:01:19,361][105692] Updated weights for policy 0, policy_version 914470 (0.0010) [2023-12-26 22:01:19,427][105692] Updated weights for policy 0, policy_version 914480 (0.0009) [2023-12-26 22:01:19,798][105620] Updated weights for policy 1, policy_version 914485 (0.0008) [2023-12-26 22:01:19,867][105620] Updated weights for policy 1, policy_version 914495 (0.0011) [2023-12-26 22:01:19,939][105620] Updated weights for policy 1, policy_version 914505 (0.0009) [2023-12-26 22:01:20,229][105692] Updated weights for policy 0, policy_version 914490 (0.0009) [2023-12-26 22:01:20,296][105692] Updated weights for policy 0, policy_version 914500 (0.0009) [2023-12-26 22:01:20,361][105692] Updated weights for policy 0, policy_version 914510 (0.0008) [2023-12-26 22:01:20,716][105620] Updated weights for policy 1, policy_version 914515 (0.0007) [2023-12-26 22:01:20,774][105620] Updated weights for policy 1, policy_version 914525 (0.0009) [2023-12-26 22:01:20,837][105620] Updated weights for policy 1, policy_version 914535 (0.0009) [2023-12-26 22:01:21,062][104569] Fps is (10 sec: 18841.9, 60 sec: 18705.1, 300 sec: 19244.3). Total num frames: 468303872. Throughput: 0: 9192.9, 1: 9446.9. Samples: 468294896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:01:21,062][104569] Avg episode reward: [(0, '9075.765'), (1, '9184.189')] [2023-12-26 22:01:21,175][105692] Updated weights for policy 0, policy_version 914520 (0.0009) [2023-12-26 22:01:21,238][105692] Updated weights for policy 0, policy_version 914530 (0.0009) [2023-12-26 22:01:21,307][105692] Updated weights for policy 0, policy_version 914540 (0.0008) [2023-12-26 22:01:21,724][105620] Updated weights for policy 1, policy_version 914545 (0.0009) [2023-12-26 22:01:21,791][105620] Updated weights for policy 1, policy_version 914555 (0.0009) [2023-12-26 22:01:21,856][105620] Updated weights for policy 1, policy_version 914565 (0.0009) [2023-12-26 22:01:21,918][105620] Updated weights for policy 1, policy_version 914575 (0.0008) [2023-12-26 22:01:22,095][105692] Updated weights for policy 0, policy_version 914550 (0.0009) [2023-12-26 22:01:22,149][105692] Updated weights for policy 0, policy_version 914560 (0.0007) [2023-12-26 22:01:22,201][105692] Updated weights for policy 0, policy_version 914570 (0.0008) [2023-12-26 22:01:22,689][105620] Updated weights for policy 1, policy_version 914585 (0.0008) [2023-12-26 22:01:22,749][105620] Updated weights for policy 1, policy_version 914595 (0.0009) [2023-12-26 22:01:22,808][105620] Updated weights for policy 1, policy_version 914605 (0.0009) [2023-12-26 22:01:22,981][105692] Updated weights for policy 0, policy_version 914580 (0.0009) [2023-12-26 22:01:23,036][105692] Updated weights for policy 0, policy_version 914590 (0.0008) [2023-12-26 22:01:23,085][105692] Updated weights for policy 0, policy_version 914600 (0.0006) [2023-12-26 22:01:23,617][105620] Updated weights for policy 1, policy_version 914615 (0.0009) [2023-12-26 22:01:23,674][105620] Updated weights for policy 1, policy_version 914625 (0.0007) [2023-12-26 22:01:23,741][105620] Updated weights for policy 1, policy_version 914635 (0.0005) [2023-12-26 22:01:23,804][105692] Updated weights for policy 0, policy_version 914610 (0.0008) [2023-12-26 22:01:23,866][105692] Updated weights for policy 0, policy_version 914620 (0.0009) [2023-12-26 22:01:23,931][105692] Updated weights for policy 0, policy_version 914630 (0.0009) [2023-12-26 22:01:23,990][105692] Updated weights for policy 0, policy_version 914640 (0.0006) [2023-12-26 22:01:24,426][105620] Updated weights for policy 1, policy_version 914645 (0.0010) [2023-12-26 22:01:24,468][105586] KL-divergence is very high: 118.8241 [2023-12-26 22:01:24,475][105620] Updated weights for policy 1, policy_version 914655 (0.0009) [2023-12-26 22:01:24,514][105586] KL-divergence is very high: 204.6096 [2023-12-26 22:01:24,533][105620] Updated weights for policy 1, policy_version 914665 (0.0009) [2023-12-26 22:01:24,565][105586] KL-divergence is very high: 181.7457 [2023-12-26 22:01:24,715][105692] Updated weights for policy 0, policy_version 914650 (0.0009) [2023-12-26 22:01:24,769][105692] Updated weights for policy 0, policy_version 914660 (0.0009) [2023-12-26 22:01:24,816][105692] Updated weights for policy 0, policy_version 914670 (0.0009) [2023-12-26 22:01:25,394][105620] Updated weights for policy 1, policy_version 914675 (0.0009) [2023-12-26 22:01:25,447][105692] Updated weights for policy 0, policy_version 914680 (0.0007) [2023-12-26 22:01:25,460][105620] Updated weights for policy 1, policy_version 914685 (0.0007) [2023-12-26 22:01:25,495][105692] Updated weights for policy 0, policy_version 914690 (0.0006) [2023-12-26 22:01:25,520][105620] Updated weights for policy 1, policy_version 914695 (0.0008) [2023-12-26 22:01:25,552][105692] Updated weights for policy 0, policy_version 914700 (0.0006) [2023-12-26 22:01:26,062][104569] Fps is (10 sec: 18022.2, 60 sec: 18705.1, 300 sec: 19216.5). Total num frames: 468393984. Throughput: 0: 9202.0, 1: 9326.4. Samples: 468404188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:01:26,063][104569] Avg episode reward: [(0, '8894.605'), (1, '9000.573')] [2023-12-26 22:01:26,273][105620] Updated weights for policy 1, policy_version 914705 (0.0006) [2023-12-26 22:01:26,283][105692] Updated weights for policy 0, policy_version 914710 (0.0009) [2023-12-26 22:01:26,329][105620] Updated weights for policy 1, policy_version 914715 (0.0007) [2023-12-26 22:01:26,339][105692] Updated weights for policy 0, policy_version 914720 (0.0006) [2023-12-26 22:01:26,386][105620] Updated weights for policy 1, policy_version 914725 (0.0007) [2023-12-26 22:01:26,403][105692] Updated weights for policy 0, policy_version 914730 (0.0008) [2023-12-26 22:01:26,457][105620] Updated weights for policy 1, policy_version 914735 (0.0008) [2023-12-26 22:01:27,133][105692] Updated weights for policy 0, policy_version 914740 (0.0008) [2023-12-26 22:01:27,185][105620] Updated weights for policy 1, policy_version 914745 (0.0006) [2023-12-26 22:01:27,190][105692] Updated weights for policy 0, policy_version 914750 (0.0009) [2023-12-26 22:01:27,241][105620] Updated weights for policy 1, policy_version 914755 (0.0007) [2023-12-26 22:01:27,253][105692] Updated weights for policy 0, policy_version 914760 (0.0007) [2023-12-26 22:01:27,300][105620] Updated weights for policy 1, policy_version 914765 (0.0010) [2023-12-26 22:01:27,859][105620] Updated weights for policy 1, policy_version 914775 (0.0009) [2023-12-26 22:01:27,921][105620] Updated weights for policy 1, policy_version 914785 (0.0010) [2023-12-26 22:01:27,972][105692] Updated weights for policy 0, policy_version 914770 (0.0007) [2023-12-26 22:01:27,979][105620] Updated weights for policy 1, policy_version 914795 (0.0010) [2023-12-26 22:01:28,034][105692] Updated weights for policy 0, policy_version 914780 (0.0010) [2023-12-26 22:01:28,091][105692] Updated weights for policy 0, policy_version 914790 (0.0008) [2023-12-26 22:01:28,141][105692] Updated weights for policy 0, policy_version 914800 (0.0008) [2023-12-26 22:01:28,757][105620] Updated weights for policy 1, policy_version 914805 (0.0011) [2023-12-26 22:01:28,813][105620] Updated weights for policy 1, policy_version 914815 (0.0010) [2023-12-26 22:01:28,869][105620] Updated weights for policy 1, policy_version 914825 (0.0010) [2023-12-26 22:01:28,896][105692] Updated weights for policy 0, policy_version 914810 (0.0006) [2023-12-26 22:01:28,954][105692] Updated weights for policy 0, policy_version 914820 (0.0008) [2023-12-26 22:01:29,001][105692] Updated weights for policy 0, policy_version 914830 (0.0008) [2023-12-26 22:01:29,668][105620] Updated weights for policy 1, policy_version 914835 (0.0010) [2023-12-26 22:01:29,724][105620] Updated weights for policy 1, policy_version 914845 (0.0008) [2023-12-26 22:01:29,729][105692] Updated weights for policy 0, policy_version 914840 (0.0010) [2023-12-26 22:01:29,780][105620] Updated weights for policy 1, policy_version 914855 (0.0006) [2023-12-26 22:01:29,786][105692] Updated weights for policy 0, policy_version 914850 (0.0011) [2023-12-26 22:01:29,857][105692] Updated weights for policy 0, policy_version 914860 (0.0010) [2023-12-26 22:01:30,479][105620] Updated weights for policy 1, policy_version 914865 (0.0008) [2023-12-26 22:01:30,538][105620] Updated weights for policy 1, policy_version 914875 (0.0007) [2023-12-26 22:01:30,598][105692] Updated weights for policy 0, policy_version 914870 (0.0008) [2023-12-26 22:01:30,600][105620] Updated weights for policy 1, policy_version 914885 (0.0006) [2023-12-26 22:01:30,653][105692] Updated weights for policy 0, policy_version 914880 (0.0006) [2023-12-26 22:01:30,665][105620] Updated weights for policy 1, policy_version 914895 (0.0006) [2023-12-26 22:01:30,708][105692] Updated weights for policy 0, policy_version 914890 (0.0005) [2023-12-26 22:01:31,062][104569] Fps is (10 sec: 18840.9, 60 sec: 18705.0, 300 sec: 19216.5). Total num frames: 468492288. Throughput: 0: 9214.7, 1: 9350.2. Samples: 468462832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:01:31,063][104569] Avg episode reward: [(0, '9076.856'), (1, '8725.572')] [2023-12-26 22:01:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000914896_234242048.pth... [2023-12-26 22:01:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000914896_234250240.pth... [2023-12-26 22:01:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000913776_233955328.pth [2023-12-26 22:01:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000913808_233971712.pth [2023-12-26 22:01:31,252][105620] Updated weights for policy 1, policy_version 914905 (0.0008) [2023-12-26 22:01:31,311][105620] Updated weights for policy 1, policy_version 914915 (0.0008) [2023-12-26 22:01:31,371][105620] Updated weights for policy 1, policy_version 914925 (0.0008) [2023-12-26 22:01:31,396][105692] Updated weights for policy 0, policy_version 914900 (0.0008) [2023-12-26 22:01:31,463][105692] Updated weights for policy 0, policy_version 914910 (0.0011) [2023-12-26 22:01:31,512][105692] Updated weights for policy 0, policy_version 914920 (0.0010) [2023-12-26 22:01:32,128][105620] Updated weights for policy 1, policy_version 914935 (0.0010) [2023-12-26 22:01:32,188][105620] Updated weights for policy 1, policy_version 914945 (0.0008) [2023-12-26 22:01:32,256][105620] Updated weights for policy 1, policy_version 914955 (0.0008) [2023-12-26 22:01:32,262][105692] Updated weights for policy 0, policy_version 914930 (0.0010) [2023-12-26 22:01:32,324][105692] Updated weights for policy 0, policy_version 914940 (0.0007) [2023-12-26 22:01:32,393][105692] Updated weights for policy 0, policy_version 914950 (0.0008) [2023-12-26 22:01:32,446][105692] Updated weights for policy 0, policy_version 914960 (0.0008) [2023-12-26 22:01:33,001][105620] Updated weights for policy 1, policy_version 914965 (0.0011) [2023-12-26 22:01:33,054][105620] Updated weights for policy 1, policy_version 914975 (0.0010) [2023-12-26 22:01:33,104][105692] Updated weights for policy 0, policy_version 914970 (0.0009) [2023-12-26 22:01:33,112][105620] Updated weights for policy 1, policy_version 914985 (0.0010) [2023-12-26 22:01:33,159][105692] Updated weights for policy 0, policy_version 914980 (0.0009) [2023-12-26 22:01:33,208][105692] Updated weights for policy 0, policy_version 914990 (0.0010) [2023-12-26 22:01:33,714][105620] Updated weights for policy 1, policy_version 914995 (0.0009) [2023-12-26 22:01:33,770][105620] Updated weights for policy 1, policy_version 915005 (0.0005) [2023-12-26 22:01:33,836][105620] Updated weights for policy 1, policy_version 915015 (0.0005) [2023-12-26 22:01:33,944][105692] Updated weights for policy 0, policy_version 915000 (0.0006) [2023-12-26 22:01:33,994][105692] Updated weights for policy 0, policy_version 915010 (0.0005) [2023-12-26 22:01:34,052][105692] Updated weights for policy 0, policy_version 915020 (0.0006) [2023-12-26 22:01:34,580][105620] Updated weights for policy 1, policy_version 915025 (0.0006) [2023-12-26 22:01:34,643][105620] Updated weights for policy 1, policy_version 915035 (0.0009) [2023-12-26 22:01:34,710][105620] Updated weights for policy 1, policy_version 915045 (0.0008) [2023-12-26 22:01:34,727][105692] Updated weights for policy 0, policy_version 915030 (0.0008) [2023-12-26 22:01:34,767][105620] Updated weights for policy 1, policy_version 915055 (0.0007) [2023-12-26 22:01:34,778][105692] Updated weights for policy 0, policy_version 915040 (0.0007) [2023-12-26 22:01:34,836][105692] Updated weights for policy 0, policy_version 915050 (0.0006) [2023-12-26 22:01:35,421][105692] Updated weights for policy 0, policy_version 915060 (0.0007) [2023-12-26 22:01:35,482][105692] Updated weights for policy 0, policy_version 915070 (0.0009) [2023-12-26 22:01:35,530][105692] Updated weights for policy 0, policy_version 915080 (0.0008) [2023-12-26 22:01:35,590][105620] Updated weights for policy 1, policy_version 915065 (0.0008) [2023-12-26 22:01:35,638][105620] Updated weights for policy 1, policy_version 915075 (0.0008) [2023-12-26 22:01:35,686][105620] Updated weights for policy 1, policy_version 915085 (0.0008) [2023-12-26 22:01:36,062][104569] Fps is (10 sec: 19660.4, 60 sec: 18705.0, 300 sec: 19244.2). Total num frames: 468590592. Throughput: 0: 9283.5, 1: 9378.5. Samples: 468579848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:01:36,063][104569] Avg episode reward: [(0, '9258.827'), (1, '8640.402')] [2023-12-26 22:01:36,327][105692] Updated weights for policy 0, policy_version 915090 (0.0008) [2023-12-26 22:01:36,392][105692] Updated weights for policy 0, policy_version 915100 (0.0009) [2023-12-26 22:01:36,455][105692] Updated weights for policy 0, policy_version 915110 (0.0008) [2023-12-26 22:01:36,481][105620] Updated weights for policy 1, policy_version 915095 (0.0010) [2023-12-26 22:01:36,519][105692] Updated weights for policy 0, policy_version 915120 (0.0006) [2023-12-26 22:01:36,542][105620] Updated weights for policy 1, policy_version 915105 (0.0011) [2023-12-26 22:01:36,606][105620] Updated weights for policy 1, policy_version 915115 (0.0008) [2023-12-26 22:01:37,212][105692] Updated weights for policy 0, policy_version 915130 (0.0008) [2023-12-26 22:01:37,273][105692] Updated weights for policy 0, policy_version 915140 (0.0008) [2023-12-26 22:01:37,310][105620] Updated weights for policy 1, policy_version 915125 (0.0008) [2023-12-26 22:01:37,333][105692] Updated weights for policy 0, policy_version 915150 (0.0008) [2023-12-26 22:01:37,370][105620] Updated weights for policy 1, policy_version 915135 (0.0011) [2023-12-26 22:01:37,419][105620] Updated weights for policy 1, policy_version 915145 (0.0011) [2023-12-26 22:01:38,010][105692] Updated weights for policy 0, policy_version 915160 (0.0007) [2023-12-26 22:01:38,064][105692] Updated weights for policy 0, policy_version 915170 (0.0008) [2023-12-26 22:01:38,123][105692] Updated weights for policy 0, policy_version 915180 (0.0008) [2023-12-26 22:01:38,168][105620] Updated weights for policy 1, policy_version 915155 (0.0009) [2023-12-26 22:01:38,238][105620] Updated weights for policy 1, policy_version 915165 (0.0005) [2023-12-26 22:01:38,297][105620] Updated weights for policy 1, policy_version 915175 (0.0010) [2023-12-26 22:01:38,902][105692] Updated weights for policy 0, policy_version 915190 (0.0008) [2023-12-26 22:01:38,951][105692] Updated weights for policy 0, policy_version 915200 (0.0009) [2023-12-26 22:01:39,008][105692] Updated weights for policy 0, policy_version 915210 (0.0008) [2023-12-26 22:01:39,021][105620] Updated weights for policy 1, policy_version 915185 (0.0009) [2023-12-26 22:01:39,084][105620] Updated weights for policy 1, policy_version 915195 (0.0011) [2023-12-26 22:01:39,171][105620] Updated weights for policy 1, policy_version 915205 (0.0011) [2023-12-26 22:01:39,237][105620] Updated weights for policy 1, policy_version 915215 (0.0009) [2023-12-26 22:01:39,837][105692] Updated weights for policy 0, policy_version 915220 (0.0010) [2023-12-26 22:01:39,902][105692] Updated weights for policy 0, policy_version 915230 (0.0008) [2023-12-26 22:01:39,969][105692] Updated weights for policy 0, policy_version 915240 (0.0008) [2023-12-26 22:01:39,989][105620] Updated weights for policy 1, policy_version 915225 (0.0009) [2023-12-26 22:01:40,057][105620] Updated weights for policy 1, policy_version 915235 (0.0011) [2023-12-26 22:01:40,120][105620] Updated weights for policy 1, policy_version 915245 (0.0010) [2023-12-26 22:01:40,637][105692] Updated weights for policy 0, policy_version 915250 (0.0006) [2023-12-26 22:01:40,698][105692] Updated weights for policy 0, policy_version 915260 (0.0009) [2023-12-26 22:01:40,750][105692] Updated weights for policy 0, policy_version 915270 (0.0009) [2023-12-26 22:01:40,801][105692] Updated weights for policy 0, policy_version 915280 (0.0009) [2023-12-26 22:01:40,839][105620] Updated weights for policy 1, policy_version 915255 (0.0009) [2023-12-26 22:01:40,897][105620] Updated weights for policy 1, policy_version 915265 (0.0009) [2023-12-26 22:01:40,958][105620] Updated weights for policy 1, policy_version 915275 (0.0009) [2023-12-26 22:01:41,062][104569] Fps is (10 sec: 19661.5, 60 sec: 18841.6, 300 sec: 19244.3). Total num frames: 468688896. Throughput: 0: 9378.9, 1: 9405.3. Samples: 468692992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:01:41,062][104569] Avg episode reward: [(0, '9350.108'), (1, '8822.384')] [2023-12-26 22:01:41,585][105692] Updated weights for policy 0, policy_version 915290 (0.0008) [2023-12-26 22:01:41,654][105692] Updated weights for policy 0, policy_version 915300 (0.0008) [2023-12-26 22:01:41,717][105692] Updated weights for policy 0, policy_version 915310 (0.0008) [2023-12-26 22:01:41,753][105620] Updated weights for policy 1, policy_version 915285 (0.0009) [2023-12-26 22:01:41,813][105620] Updated weights for policy 1, policy_version 915295 (0.0009) [2023-12-26 22:01:41,872][105620] Updated weights for policy 1, policy_version 915305 (0.0005) [2023-12-26 22:01:42,464][105692] Updated weights for policy 0, policy_version 915320 (0.0008) [2023-12-26 22:01:42,504][105620] Updated weights for policy 1, policy_version 915315 (0.0007) [2023-12-26 22:01:42,525][105692] Updated weights for policy 0, policy_version 915330 (0.0011) [2023-12-26 22:01:42,569][105620] Updated weights for policy 1, policy_version 915325 (0.0011) [2023-12-26 22:01:42,587][105692] Updated weights for policy 0, policy_version 915340 (0.0008) [2023-12-26 22:01:42,632][105620] Updated weights for policy 1, policy_version 915335 (0.0008) [2023-12-26 22:01:43,245][105620] Updated weights for policy 1, policy_version 915345 (0.0006) [2023-12-26 22:01:43,304][105620] Updated weights for policy 1, policy_version 915355 (0.0007) [2023-12-26 22:01:43,370][105620] Updated weights for policy 1, policy_version 915365 (0.0007) [2023-12-26 22:01:43,402][105692] Updated weights for policy 0, policy_version 915350 (0.0007) [2023-12-26 22:01:43,425][105620] Updated weights for policy 1, policy_version 915375 (0.0006) [2023-12-26 22:01:43,460][105692] Updated weights for policy 0, policy_version 915360 (0.0009) [2023-12-26 22:01:43,525][105692] Updated weights for policy 0, policy_version 915370 (0.0008) [2023-12-26 22:01:44,095][105620] Updated weights for policy 1, policy_version 915385 (0.0010) [2023-12-26 22:01:44,144][105620] Updated weights for policy 1, policy_version 915395 (0.0009) [2023-12-26 22:01:44,199][105620] Updated weights for policy 1, policy_version 915405 (0.0010) [2023-12-26 22:01:44,238][105692] Updated weights for policy 0, policy_version 915380 (0.0007) [2023-12-26 22:01:44,297][105692] Updated weights for policy 0, policy_version 915390 (0.0008) [2023-12-26 22:01:44,346][105692] Updated weights for policy 0, policy_version 915400 (0.0008) [2023-12-26 22:01:44,933][105620] Updated weights for policy 1, policy_version 915415 (0.0007) [2023-12-26 22:01:44,993][105620] Updated weights for policy 1, policy_version 915425 (0.0007) [2023-12-26 22:01:45,021][105692] Updated weights for policy 0, policy_version 915410 (0.0009) [2023-12-26 22:01:45,054][105620] Updated weights for policy 1, policy_version 915435 (0.0007) [2023-12-26 22:01:45,085][105692] Updated weights for policy 0, policy_version 915420 (0.0011) [2023-12-26 22:01:45,152][105692] Updated weights for policy 0, policy_version 915430 (0.0011) [2023-12-26 22:01:45,222][105692] Updated weights for policy 0, policy_version 915440 (0.0011) [2023-12-26 22:01:45,661][105620] Updated weights for policy 1, policy_version 915445 (0.0007) [2023-12-26 22:01:45,714][105620] Updated weights for policy 1, policy_version 915455 (0.0007) [2023-12-26 22:01:45,764][105620] Updated weights for policy 1, policy_version 915465 (0.0011) [2023-12-26 22:01:45,935][105692] Updated weights for policy 0, policy_version 915450 (0.0008) [2023-12-26 22:01:46,009][105692] Updated weights for policy 0, policy_version 915460 (0.0011) [2023-12-26 22:01:46,062][104569] Fps is (10 sec: 18842.0, 60 sec: 18705.1, 300 sec: 19216.5). Total num frames: 468779008. Throughput: 0: 9349.3, 1: 9497.0. Samples: 468750068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:01:46,062][104569] Avg episode reward: [(0, '9350.531'), (1, '9001.138')] [2023-12-26 22:01:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000915472_234389504.pth... [2023-12-26 22:01:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000914352_234102784.pth [2023-12-26 22:01:46,074][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_000915472_234389504.pth [2023-12-26 22:01:46,083][105692] Updated weights for policy 0, policy_version 915470 (0.0010) [2023-12-26 22:01:46,095][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000915472_234397696.pth... [2023-12-26 22:01:46,101][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000914352_234110976.pth [2023-12-26 22:01:46,102][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_000915472_234397696.pth [2023-12-26 22:01:46,364][105620] Updated weights for policy 1, policy_version 915475 (0.0011) [2023-12-26 22:01:46,416][105620] Updated weights for policy 1, policy_version 915485 (0.0010) [2023-12-26 22:01:46,468][105620] Updated weights for policy 1, policy_version 915495 (0.0010) [2023-12-26 22:01:46,773][105692] Updated weights for policy 0, policy_version 915480 (0.0009) [2023-12-26 22:01:46,832][105692] Updated weights for policy 0, policy_version 915490 (0.0008) [2023-12-26 22:01:46,888][105692] Updated weights for policy 0, policy_version 915500 (0.0008) [2023-12-26 22:01:47,237][105620] Updated weights for policy 1, policy_version 915505 (0.0010) [2023-12-26 22:01:47,303][105620] Updated weights for policy 1, policy_version 915515 (0.0010) [2023-12-26 22:01:47,351][105620] Updated weights for policy 1, policy_version 915525 (0.0010) [2023-12-26 22:01:47,403][105620] Updated weights for policy 1, policy_version 915535 (0.0010) [2023-12-26 22:01:47,662][105692] Updated weights for policy 0, policy_version 915510 (0.0008) [2023-12-26 22:01:47,710][105692] Updated weights for policy 0, policy_version 915520 (0.0008) [2023-12-26 22:01:47,758][105692] Updated weights for policy 0, policy_version 915530 (0.0008) [2023-12-26 22:01:48,161][105620] Updated weights for policy 1, policy_version 915545 (0.0011) [2023-12-26 22:01:48,214][105620] Updated weights for policy 1, policy_version 915555 (0.0011) [2023-12-26 22:01:48,263][105620] Updated weights for policy 1, policy_version 915565 (0.0010) [2023-12-26 22:01:48,574][105692] Updated weights for policy 0, policy_version 915540 (0.0008) [2023-12-26 22:01:48,631][105692] Updated weights for policy 0, policy_version 915550 (0.0008) [2023-12-26 22:01:48,692][105692] Updated weights for policy 0, policy_version 915560 (0.0008) [2023-12-26 22:01:49,083][105620] Updated weights for policy 1, policy_version 915575 (0.0011) [2023-12-26 22:01:49,147][105620] Updated weights for policy 1, policy_version 915585 (0.0011) [2023-12-26 22:01:49,210][105620] Updated weights for policy 1, policy_version 915595 (0.0011) [2023-12-26 22:01:49,474][105692] Updated weights for policy 0, policy_version 915570 (0.0008) [2023-12-26 22:01:49,533][105692] Updated weights for policy 0, policy_version 915580 (0.0008) [2023-12-26 22:01:49,597][105692] Updated weights for policy 0, policy_version 915590 (0.0008) [2023-12-26 22:01:49,650][105692] Updated weights for policy 0, policy_version 915600 (0.0008) [2023-12-26 22:01:49,987][105620] Updated weights for policy 1, policy_version 915605 (0.0011) [2023-12-26 22:01:50,040][105620] Updated weights for policy 1, policy_version 915615 (0.0010) [2023-12-26 22:01:50,098][105620] Updated weights for policy 1, policy_version 915625 (0.0010) [2023-12-26 22:01:50,450][105692] Updated weights for policy 0, policy_version 915610 (0.0008) [2023-12-26 22:01:50,516][105692] Updated weights for policy 0, policy_version 915620 (0.0008) [2023-12-26 22:01:50,587][105692] Updated weights for policy 0, policy_version 915630 (0.0009) [2023-12-26 22:01:50,917][105620] Updated weights for policy 1, policy_version 915635 (0.0011) [2023-12-26 22:01:50,980][105620] Updated weights for policy 1, policy_version 915645 (0.0010) [2023-12-26 22:01:51,043][105620] Updated weights for policy 1, policy_version 915655 (0.0011) [2023-12-26 22:01:51,062][104569] Fps is (10 sec: 18022.5, 60 sec: 18705.1, 300 sec: 19188.7). Total num frames: 468869120. Throughput: 0: 9368.2, 1: 9596.1. Samples: 468865300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:01:51,062][104569] Avg episode reward: [(0, '9260.669'), (1, '8650.843')] [2023-12-26 22:01:51,401][105692] Updated weights for policy 0, policy_version 915640 (0.0009) [2023-12-26 22:01:51,467][105692] Updated weights for policy 0, policy_version 915650 (0.0009) [2023-12-26 22:01:51,534][105692] Updated weights for policy 0, policy_version 915660 (0.0010) [2023-12-26 22:01:51,744][105620] Updated weights for policy 1, policy_version 915665 (0.0010) [2023-12-26 22:01:51,804][105620] Updated weights for policy 1, policy_version 915675 (0.0011) [2023-12-26 22:01:51,868][105620] Updated weights for policy 1, policy_version 915685 (0.0008) [2023-12-26 22:01:51,936][105620] Updated weights for policy 1, policy_version 915695 (0.0008) [2023-12-26 22:01:52,283][105692] Updated weights for policy 0, policy_version 915670 (0.0008) [2023-12-26 22:01:52,353][105692] Updated weights for policy 0, policy_version 915680 (0.0007) [2023-12-26 22:01:52,420][105692] Updated weights for policy 0, policy_version 915690 (0.0008) [2023-12-26 22:01:52,675][105620] Updated weights for policy 1, policy_version 915705 (0.0010) [2023-12-26 22:01:52,738][105620] Updated weights for policy 1, policy_version 915715 (0.0011) [2023-12-26 22:01:52,787][105620] Updated weights for policy 1, policy_version 915725 (0.0010) [2023-12-26 22:01:53,032][105692] Updated weights for policy 0, policy_version 915700 (0.0009) [2023-12-26 22:01:53,091][105692] Updated weights for policy 0, policy_version 915710 (0.0009) [2023-12-26 22:01:53,156][105692] Updated weights for policy 0, policy_version 915720 (0.0008) [2023-12-26 22:01:53,537][105620] Updated weights for policy 1, policy_version 915735 (0.0009) [2023-12-26 22:01:53,594][105620] Updated weights for policy 1, policy_version 915745 (0.0008) [2023-12-26 22:01:53,658][105620] Updated weights for policy 1, policy_version 915755 (0.0008) [2023-12-26 22:01:53,888][105692] Updated weights for policy 0, policy_version 915730 (0.0008) [2023-12-26 22:01:53,934][105692] Updated weights for policy 0, policy_version 915740 (0.0005) [2023-12-26 22:01:53,989][105692] Updated weights for policy 0, policy_version 915750 (0.0006) [2023-12-26 22:01:54,049][105692] Updated weights for policy 0, policy_version 915760 (0.0007) [2023-12-26 22:01:54,410][105620] Updated weights for policy 1, policy_version 915765 (0.0008) [2023-12-26 22:01:54,472][105620] Updated weights for policy 1, policy_version 915775 (0.0007) [2023-12-26 22:01:54,532][105620] Updated weights for policy 1, policy_version 915785 (0.0008) [2023-12-26 22:01:54,800][105692] Updated weights for policy 0, policy_version 915770 (0.0005) [2023-12-26 22:01:54,854][105692] Updated weights for policy 0, policy_version 915780 (0.0005) [2023-12-26 22:01:54,918][105692] Updated weights for policy 0, policy_version 915790 (0.0006) [2023-12-26 22:01:55,313][105620] Updated weights for policy 1, policy_version 915795 (0.0009) [2023-12-26 22:01:55,363][105620] Updated weights for policy 1, policy_version 915805 (0.0008) [2023-12-26 22:01:55,411][105620] Updated weights for policy 1, policy_version 915815 (0.0008) [2023-12-26 22:01:55,636][105692] Updated weights for policy 0, policy_version 915800 (0.0010) [2023-12-26 22:01:55,693][105692] Updated weights for policy 0, policy_version 915810 (0.0010) [2023-12-26 22:01:55,745][105692] Updated weights for policy 0, policy_version 915820 (0.0010) [2023-12-26 22:01:56,062][104569] Fps is (10 sec: 18841.9, 60 sec: 18841.6, 300 sec: 19188.7). Total num frames: 468967424. Throughput: 0: 9427.2, 1: 9586.5. Samples: 468977448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:01:56,062][104569] Avg episode reward: [(0, '9260.350'), (1, '8294.489')] [2023-12-26 22:01:56,095][105620] Updated weights for policy 1, policy_version 915825 (0.0006) [2023-12-26 22:01:56,165][105620] Updated weights for policy 1, policy_version 915835 (0.0005) [2023-12-26 22:01:56,234][105620] Updated weights for policy 1, policy_version 915845 (0.0005) [2023-12-26 22:01:56,307][105620] Updated weights for policy 1, policy_version 915855 (0.0005) [2023-12-26 22:01:56,387][105692] Updated weights for policy 0, policy_version 915830 (0.0011) [2023-12-26 22:01:56,453][105692] Updated weights for policy 0, policy_version 915840 (0.0011) [2023-12-26 22:01:56,501][105692] Updated weights for policy 0, policy_version 915850 (0.0008) [2023-12-26 22:01:56,911][105620] Updated weights for policy 1, policy_version 915865 (0.0005) [2023-12-26 22:01:56,950][105586] KL-divergence is very high: 131.0501 [2023-12-26 22:01:56,959][105620] Updated weights for policy 1, policy_version 915875 (0.0005) [2023-12-26 22:01:56,993][105586] KL-divergence is very high: 100.5286 [2023-12-26 22:01:57,018][105620] Updated weights for policy 1, policy_version 915885 (0.0005) [2023-12-26 22:01:57,178][105692] Updated weights for policy 0, policy_version 915860 (0.0007) [2023-12-26 22:01:57,236][105692] Updated weights for policy 0, policy_version 915870 (0.0010) [2023-12-26 22:01:57,308][105692] Updated weights for policy 0, policy_version 915880 (0.0010) [2023-12-26 22:01:57,640][105620] Updated weights for policy 1, policy_version 915895 (0.0008) [2023-12-26 22:01:57,687][105620] Updated weights for policy 1, policy_version 915905 (0.0007) [2023-12-26 22:01:57,738][105620] Updated weights for policy 1, policy_version 915915 (0.0008) [2023-12-26 22:01:57,981][105692] Updated weights for policy 0, policy_version 915890 (0.0010) [2023-12-26 22:01:58,033][105692] Updated weights for policy 0, policy_version 915900 (0.0011) [2023-12-26 22:01:58,078][105692] Updated weights for policy 0, policy_version 915910 (0.0010) [2023-12-26 22:01:58,141][105692] Updated weights for policy 0, policy_version 915920 (0.0009) [2023-12-26 22:01:58,589][105620] Updated weights for policy 1, policy_version 915925 (0.0007) [2023-12-26 22:01:58,660][105620] Updated weights for policy 1, policy_version 915935 (0.0009) [2023-12-26 22:01:58,730][105620] Updated weights for policy 1, policy_version 915945 (0.0007) [2023-12-26 22:01:58,930][105692] Updated weights for policy 0, policy_version 915930 (0.0015) [2023-12-26 22:01:59,013][105692] Updated weights for policy 0, policy_version 915940 (0.0011) [2023-12-26 22:01:59,072][105692] Updated weights for policy 0, policy_version 915950 (0.0009) [2023-12-26 22:01:59,602][105620] Updated weights for policy 1, policy_version 915955 (0.0008) [2023-12-26 22:01:59,656][105620] Updated weights for policy 1, policy_version 915965 (0.0006) [2023-12-26 22:01:59,713][105620] Updated weights for policy 1, policy_version 915975 (0.0006) [2023-12-26 22:01:59,898][105692] Updated weights for policy 0, policy_version 915960 (0.0008) [2023-12-26 22:01:59,962][105692] Updated weights for policy 0, policy_version 915970 (0.0008) [2023-12-26 22:02:00,018][105692] Updated weights for policy 0, policy_version 915980 (0.0009) [2023-12-26 22:02:00,477][105620] Updated weights for policy 1, policy_version 915985 (0.0009) [2023-12-26 22:02:00,527][105620] Updated weights for policy 1, policy_version 915995 (0.0009) [2023-12-26 22:02:00,575][105620] Updated weights for policy 1, policy_version 916005 (0.0009) [2023-12-26 22:02:00,631][105620] Updated weights for policy 1, policy_version 916015 (0.0008) [2023-12-26 22:02:00,806][105692] Updated weights for policy 0, policy_version 915990 (0.0008) [2023-12-26 22:02:00,857][105692] Updated weights for policy 0, policy_version 916000 (0.0009) [2023-12-26 22:02:00,920][105692] Updated weights for policy 0, policy_version 916010 (0.0009) [2023-12-26 22:02:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 18978.1, 300 sec: 19160.9). Total num frames: 469065728. Throughput: 0: 9503.8, 1: 9629.5. Samples: 469037360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:02:01,062][104569] Avg episode reward: [(0, '9349.937'), (1, '7843.516')] [2023-12-26 22:02:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000916016_234536960.pth... [2023-12-26 22:02:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000916016_234528768.pth... [2023-12-26 22:02:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000914896_234250240.pth [2023-12-26 22:02:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000914896_234242048.pth [2023-12-26 22:02:01,403][105620] Updated weights for policy 1, policy_version 916025 (0.0008) [2023-12-26 22:02:01,451][105620] Updated weights for policy 1, policy_version 916035 (0.0005) [2023-12-26 22:02:01,498][105620] Updated weights for policy 1, policy_version 916045 (0.0009) [2023-12-26 22:02:01,748][105692] Updated weights for policy 0, policy_version 916020 (0.0008) [2023-12-26 22:02:01,819][105692] Updated weights for policy 0, policy_version 916030 (0.0008) [2023-12-26 22:02:01,881][105692] Updated weights for policy 0, policy_version 916040 (0.0008) [2023-12-26 22:02:02,255][105620] Updated weights for policy 1, policy_version 916055 (0.0007) [2023-12-26 22:02:02,322][105620] Updated weights for policy 1, policy_version 916065 (0.0006) [2023-12-26 22:02:02,391][105620] Updated weights for policy 1, policy_version 916075 (0.0008) [2023-12-26 22:02:02,579][105692] Updated weights for policy 0, policy_version 916050 (0.0007) [2023-12-26 22:02:02,636][105692] Updated weights for policy 0, policy_version 916060 (0.0005) [2023-12-26 22:02:02,696][105692] Updated weights for policy 0, policy_version 916070 (0.0009) [2023-12-26 22:02:02,755][105692] Updated weights for policy 0, policy_version 916080 (0.0010) [2023-12-26 22:02:03,057][105620] Updated weights for policy 1, policy_version 916085 (0.0008) [2023-12-26 22:02:03,108][105620] Updated weights for policy 1, policy_version 916095 (0.0010) [2023-12-26 22:02:03,162][105620] Updated weights for policy 1, policy_version 916105 (0.0008) [2023-12-26 22:02:03,338][105692] Updated weights for policy 0, policy_version 916090 (0.0005) [2023-12-26 22:02:03,402][105692] Updated weights for policy 0, policy_version 916100 (0.0005) [2023-12-26 22:02:03,469][105692] Updated weights for policy 0, policy_version 916110 (0.0005) [2023-12-26 22:02:03,784][105620] Updated weights for policy 1, policy_version 916115 (0.0005) [2023-12-26 22:02:03,850][105620] Updated weights for policy 1, policy_version 916125 (0.0006) [2023-12-26 22:02:03,918][105620] Updated weights for policy 1, policy_version 916135 (0.0008) [2023-12-26 22:02:04,028][105692] Updated weights for policy 0, policy_version 916120 (0.0009) [2023-12-26 22:02:04,091][105692] Updated weights for policy 0, policy_version 916130 (0.0011) [2023-12-26 22:02:04,156][105692] Updated weights for policy 0, policy_version 916140 (0.0011) [2023-12-26 22:02:04,693][105620] Updated weights for policy 1, policy_version 916145 (0.0008) [2023-12-26 22:02:04,755][105620] Updated weights for policy 1, policy_version 916155 (0.0005) [2023-12-26 22:02:04,822][105620] Updated weights for policy 1, policy_version 916165 (0.0005) [2023-12-26 22:02:04,882][105620] Updated weights for policy 1, policy_version 916175 (0.0006) [2023-12-26 22:02:04,891][105692] Updated weights for policy 0, policy_version 916150 (0.0010) [2023-12-26 22:02:04,953][105692] Updated weights for policy 0, policy_version 916160 (0.0010) [2023-12-26 22:02:05,008][105692] Updated weights for policy 0, policy_version 916170 (0.0008) [2023-12-26 22:02:05,515][105620] Updated weights for policy 1, policy_version 916185 (0.0010) [2023-12-26 22:02:05,566][105620] Updated weights for policy 1, policy_version 916195 (0.0010) [2023-12-26 22:02:05,628][105620] Updated weights for policy 1, policy_version 916205 (0.0010) [2023-12-26 22:02:05,672][105692] Updated weights for policy 0, policy_version 916180 (0.0007) [2023-12-26 22:02:05,742][105692] Updated weights for policy 0, policy_version 916190 (0.0010) [2023-12-26 22:02:05,811][105692] Updated weights for policy 0, policy_version 916200 (0.0010) [2023-12-26 22:02:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 18978.1, 300 sec: 19188.7). Total num frames: 469164032. Throughput: 0: 9514.3, 1: 9523.6. Samples: 469151604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:02:06,064][104569] Avg episode reward: [(0, '9170.604'), (1, '8380.729')] [2023-12-26 22:02:06,296][105620] Updated weights for policy 1, policy_version 916215 (0.0009) [2023-12-26 22:02:06,360][105620] Updated weights for policy 1, policy_version 916225 (0.0008) [2023-12-26 22:02:06,427][105620] Updated weights for policy 1, policy_version 916235 (0.0006) [2023-12-26 22:02:06,553][105692] Updated weights for policy 0, policy_version 916210 (0.0010) [2023-12-26 22:02:06,619][105692] Updated weights for policy 0, policy_version 916220 (0.0010) [2023-12-26 22:02:06,684][105692] Updated weights for policy 0, policy_version 916230 (0.0010) [2023-12-26 22:02:06,756][105692] Updated weights for policy 0, policy_version 916240 (0.0010) [2023-12-26 22:02:07,127][105620] Updated weights for policy 1, policy_version 916245 (0.0008) [2023-12-26 22:02:07,178][105620] Updated weights for policy 1, policy_version 916255 (0.0007) [2023-12-26 22:02:07,230][105620] Updated weights for policy 1, policy_version 916265 (0.0008) [2023-12-26 22:02:07,488][105692] Updated weights for policy 0, policy_version 916250 (0.0010) [2023-12-26 22:02:07,551][105692] Updated weights for policy 0, policy_version 916260 (0.0010) [2023-12-26 22:02:07,621][105692] Updated weights for policy 0, policy_version 916270 (0.0011) [2023-12-26 22:02:07,961][105620] Updated weights for policy 1, policy_version 916275 (0.0008) [2023-12-26 22:02:08,021][105620] Updated weights for policy 1, policy_version 916285 (0.0007) [2023-12-26 22:02:08,083][105620] Updated weights for policy 1, policy_version 916295 (0.0006) [2023-12-26 22:02:08,348][105692] Updated weights for policy 0, policy_version 916280 (0.0010) [2023-12-26 22:02:08,412][105692] Updated weights for policy 0, policy_version 916290 (0.0010) [2023-12-26 22:02:08,474][105692] Updated weights for policy 0, policy_version 916300 (0.0010) [2023-12-26 22:02:08,828][105620] Updated weights for policy 1, policy_version 916305 (0.0007) [2023-12-26 22:02:08,885][105620] Updated weights for policy 1, policy_version 916315 (0.0008) [2023-12-26 22:02:08,947][105620] Updated weights for policy 1, policy_version 916325 (0.0009) [2023-12-26 22:02:09,006][105620] Updated weights for policy 1, policy_version 916335 (0.0009) [2023-12-26 22:02:09,215][105692] Updated weights for policy 0, policy_version 916310 (0.0009) [2023-12-26 22:02:09,286][105692] Updated weights for policy 0, policy_version 916320 (0.0009) [2023-12-26 22:02:09,360][105692] Updated weights for policy 0, policy_version 916330 (0.0009) [2023-12-26 22:02:09,706][105620] Updated weights for policy 1, policy_version 916345 (0.0011) [2023-12-26 22:02:09,759][105620] Updated weights for policy 1, policy_version 916355 (0.0010) [2023-12-26 22:02:09,813][105620] Updated weights for policy 1, policy_version 916365 (0.0010) [2023-12-26 22:02:10,207][105692] Updated weights for policy 0, policy_version 916340 (0.0009) [2023-12-26 22:02:10,261][105692] Updated weights for policy 0, policy_version 916350 (0.0010) [2023-12-26 22:02:10,318][105692] Updated weights for policy 0, policy_version 916360 (0.0010) [2023-12-26 22:02:10,463][105620] Updated weights for policy 1, policy_version 916375 (0.0008) [2023-12-26 22:02:10,524][105620] Updated weights for policy 1, policy_version 916385 (0.0006) [2023-12-26 22:02:10,573][105620] Updated weights for policy 1, policy_version 916395 (0.0006) [2023-12-26 22:02:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 18978.1, 300 sec: 19161.0). Total num frames: 469254144. Throughput: 0: 9479.8, 1: 9674.2. Samples: 469266120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:02:11,063][104569] Avg episode reward: [(0, '9037.100'), (1, '9004.140')] [2023-12-26 22:02:11,189][105692] Updated weights for policy 0, policy_version 916370 (0.0008) [2023-12-26 22:02:11,251][105620] Updated weights for policy 1, policy_version 916405 (0.0008) [2023-12-26 22:02:11,252][105692] Updated weights for policy 0, policy_version 916380 (0.0007) [2023-12-26 22:02:11,324][105692] Updated weights for policy 0, policy_version 916390 (0.0007) [2023-12-26 22:02:11,328][105620] Updated weights for policy 1, policy_version 916415 (0.0010) [2023-12-26 22:02:11,393][105692] Updated weights for policy 0, policy_version 916400 (0.0009) [2023-12-26 22:02:11,399][105620] Updated weights for policy 1, policy_version 916425 (0.0009) [2023-12-26 22:02:12,128][105692] Updated weights for policy 0, policy_version 916410 (0.0008) [2023-12-26 22:02:12,194][105692] Updated weights for policy 0, policy_version 916420 (0.0008) [2023-12-26 22:02:12,203][105620] Updated weights for policy 1, policy_version 916435 (0.0011) [2023-12-26 22:02:12,254][105692] Updated weights for policy 0, policy_version 916430 (0.0008) [2023-12-26 22:02:12,266][105620] Updated weights for policy 1, policy_version 916445 (0.0011) [2023-12-26 22:02:12,304][105586] KL-divergence is very high: 100.3126 [2023-12-26 22:02:12,331][105620] Updated weights for policy 1, policy_version 916455 (0.0010) [2023-12-26 22:02:13,076][105692] Updated weights for policy 0, policy_version 916440 (0.0007) [2023-12-26 22:02:13,082][105620] Updated weights for policy 1, policy_version 916465 (0.0011) [2023-12-26 22:02:13,131][105620] Updated weights for policy 1, policy_version 916475 (0.0010) [2023-12-26 22:02:13,148][105692] Updated weights for policy 0, policy_version 916450 (0.0005) [2023-12-26 22:02:13,190][105620] Updated weights for policy 1, policy_version 916485 (0.0011) [2023-12-26 22:02:13,198][105692] Updated weights for policy 0, policy_version 916460 (0.0005) [2023-12-26 22:02:13,252][105620] Updated weights for policy 1, policy_version 916495 (0.0010) [2023-12-26 22:02:13,775][105692] Updated weights for policy 0, policy_version 916470 (0.0008) [2023-12-26 22:02:13,833][105692] Updated weights for policy 0, policy_version 916480 (0.0009) [2023-12-26 22:02:13,887][105692] Updated weights for policy 0, policy_version 916490 (0.0009) [2023-12-26 22:02:13,932][105620] Updated weights for policy 1, policy_version 916505 (0.0006) [2023-12-26 22:02:13,999][105620] Updated weights for policy 1, policy_version 916515 (0.0009) [2023-12-26 22:02:14,051][105620] Updated weights for policy 1, policy_version 916525 (0.0010) [2023-12-26 22:02:14,677][105692] Updated weights for policy 0, policy_version 916500 (0.0009) [2023-12-26 22:02:14,765][105692] Updated weights for policy 0, policy_version 916510 (0.0008) [2023-12-26 22:02:14,787][105620] Updated weights for policy 1, policy_version 916535 (0.0010) [2023-12-26 22:02:14,830][105692] Updated weights for policy 0, policy_version 916520 (0.0007) [2023-12-26 22:02:14,852][105620] Updated weights for policy 1, policy_version 916545 (0.0009) [2023-12-26 22:02:14,910][105620] Updated weights for policy 1, policy_version 916555 (0.0011) [2023-12-26 22:02:15,573][105692] Updated weights for policy 0, policy_version 916530 (0.0006) [2023-12-26 22:02:15,622][105692] Updated weights for policy 0, policy_version 916540 (0.0009) [2023-12-26 22:02:15,677][105692] Updated weights for policy 0, policy_version 916550 (0.0008) [2023-12-26 22:02:15,680][105620] Updated weights for policy 1, policy_version 916565 (0.0011) [2023-12-26 22:02:15,732][105692] Updated weights for policy 0, policy_version 916560 (0.0005) [2023-12-26 22:02:15,738][105620] Updated weights for policy 1, policy_version 916575 (0.0010) [2023-12-26 22:02:15,761][105586] KL-divergence is very high: 149.0649 [2023-12-26 22:02:15,810][105620] Updated weights for policy 1, policy_version 916585 (0.0010) [2023-12-26 22:02:15,815][105586] KL-divergence is very high: 155.2894 [2023-12-26 22:02:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 18978.1, 300 sec: 19188.7). Total num frames: 469352448. Throughput: 0: 9454.9, 1: 9639.2. Samples: 469322060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:02:16,062][104569] Avg episode reward: [(0, '8915.207'), (1, '8558.294')] [2023-12-26 22:02:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000916592_234676224.pth... [2023-12-26 22:02:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000916560_234676224.pth... [2023-12-26 22:02:16,069][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000915472_234389504.pth [2023-12-26 22:02:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000915472_234397696.pth [2023-12-26 22:02:16,520][105692] Updated weights for policy 0, policy_version 916570 (0.0009) [2023-12-26 22:02:16,535][105620] Updated weights for policy 1, policy_version 916595 (0.0009) [2023-12-26 22:02:16,577][105692] Updated weights for policy 0, policy_version 916580 (0.0009) [2023-12-26 22:02:16,597][105620] Updated weights for policy 1, policy_version 916605 (0.0005) [2023-12-26 22:02:16,636][105692] Updated weights for policy 0, policy_version 916590 (0.0008) [2023-12-26 22:02:16,663][105620] Updated weights for policy 1, policy_version 916615 (0.0007) [2023-12-26 22:02:17,330][105620] Updated weights for policy 1, policy_version 916625 (0.0009) [2023-12-26 22:02:17,384][105620] Updated weights for policy 1, policy_version 916635 (0.0005) [2023-12-26 22:02:17,423][105692] Updated weights for policy 0, policy_version 916600 (0.0008) [2023-12-26 22:02:17,443][105620] Updated weights for policy 1, policy_version 916645 (0.0006) [2023-12-26 22:02:17,478][105692] Updated weights for policy 0, policy_version 916610 (0.0006) [2023-12-26 22:02:17,504][105620] Updated weights for policy 1, policy_version 916655 (0.0009) [2023-12-26 22:02:17,533][105692] Updated weights for policy 0, policy_version 916620 (0.0007) [2023-12-26 22:02:18,193][105620] Updated weights for policy 1, policy_version 916665 (0.0009) [2023-12-26 22:02:18,243][105620] Updated weights for policy 1, policy_version 916675 (0.0009) [2023-12-26 22:02:18,295][105620] Updated weights for policy 1, policy_version 916685 (0.0007) [2023-12-26 22:02:18,309][105692] Updated weights for policy 0, policy_version 916630 (0.0008) [2023-12-26 22:02:18,377][105692] Updated weights for policy 0, policy_version 916640 (0.0008) [2023-12-26 22:02:18,433][105692] Updated weights for policy 0, policy_version 916650 (0.0009) [2023-12-26 22:02:19,097][105692] Updated weights for policy 0, policy_version 916660 (0.0009) [2023-12-26 22:02:19,151][105620] Updated weights for policy 1, policy_version 916695 (0.0006) [2023-12-26 22:02:19,153][105692] Updated weights for policy 0, policy_version 916670 (0.0006) [2023-12-26 22:02:19,208][105692] Updated weights for policy 0, policy_version 916680 (0.0006) [2023-12-26 22:02:19,209][105620] Updated weights for policy 1, policy_version 916705 (0.0008) [2023-12-26 22:02:19,275][105620] Updated weights for policy 1, policy_version 916715 (0.0007) [2023-12-26 22:02:20,010][105620] Updated weights for policy 1, policy_version 916725 (0.0007) [2023-12-26 22:02:20,075][105620] Updated weights for policy 1, policy_version 916735 (0.0008) [2023-12-26 22:02:20,093][105692] Updated weights for policy 0, policy_version 916690 (0.0008) [2023-12-26 22:02:20,137][105620] Updated weights for policy 1, policy_version 916745 (0.0009) [2023-12-26 22:02:20,157][105692] Updated weights for policy 0, policy_version 916700 (0.0009) [2023-12-26 22:02:20,225][105692] Updated weights for policy 0, policy_version 916710 (0.0009) [2023-12-26 22:02:20,284][105692] Updated weights for policy 0, policy_version 916720 (0.0009) [2023-12-26 22:02:20,925][105620] Updated weights for policy 1, policy_version 916755 (0.0008) [2023-12-26 22:02:20,987][105620] Updated weights for policy 1, policy_version 916765 (0.0007) [2023-12-26 22:02:21,056][105620] Updated weights for policy 1, policy_version 916775 (0.0009) [2023-12-26 22:02:21,062][104569] Fps is (10 sec: 18022.4, 60 sec: 18841.6, 300 sec: 19160.9). Total num frames: 469434368. Throughput: 0: 9384.7, 1: 9588.5. Samples: 469433636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:02:21,063][104569] Avg episode reward: [(0, '8863.235'), (1, '8473.602')] [2023-12-26 22:02:21,110][105692] Updated weights for policy 0, policy_version 916730 (0.0008) [2023-12-26 22:02:21,174][105692] Updated weights for policy 0, policy_version 916740 (0.0009) [2023-12-26 22:02:21,236][105692] Updated weights for policy 0, policy_version 916750 (0.0007) [2023-12-26 22:02:21,812][105620] Updated weights for policy 1, policy_version 916785 (0.0008) [2023-12-26 22:02:21,877][105620] Updated weights for policy 1, policy_version 916795 (0.0008) [2023-12-26 22:02:21,935][105620] Updated weights for policy 1, policy_version 916805 (0.0006) [2023-12-26 22:02:21,999][105620] Updated weights for policy 1, policy_version 916815 (0.0006) [2023-12-26 22:02:22,085][105692] Updated weights for policy 0, policy_version 916760 (0.0010) [2023-12-26 22:02:22,151][105692] Updated weights for policy 0, policy_version 916770 (0.0009) [2023-12-26 22:02:22,212][105692] Updated weights for policy 0, policy_version 916780 (0.0009) [2023-12-26 22:02:22,668][105620] Updated weights for policy 1, policy_version 916825 (0.0009) [2023-12-26 22:02:22,743][105620] Updated weights for policy 1, policy_version 916835 (0.0008) [2023-12-26 22:02:22,795][105620] Updated weights for policy 1, policy_version 916845 (0.0009) [2023-12-26 22:02:23,000][105692] Updated weights for policy 0, policy_version 916790 (0.0007) [2023-12-26 22:02:23,069][105692] Updated weights for policy 0, policy_version 916800 (0.0008) [2023-12-26 22:02:23,142][105692] Updated weights for policy 0, policy_version 916810 (0.0010) [2023-12-26 22:02:23,444][105620] Updated weights for policy 1, policy_version 916855 (0.0008) [2023-12-26 22:02:23,494][105620] Updated weights for policy 1, policy_version 916865 (0.0008) [2023-12-26 22:02:23,544][105620] Updated weights for policy 1, policy_version 916875 (0.0008) [2023-12-26 22:02:23,852][105692] Updated weights for policy 0, policy_version 916820 (0.0009) [2023-12-26 22:02:23,912][105692] Updated weights for policy 0, policy_version 916830 (0.0006) [2023-12-26 22:02:23,975][105692] Updated weights for policy 0, policy_version 916840 (0.0007) [2023-12-26 22:02:24,222][105620] Updated weights for policy 1, policy_version 916885 (0.0006) [2023-12-26 22:02:24,289][105620] Updated weights for policy 1, policy_version 916895 (0.0009) [2023-12-26 22:02:24,358][105620] Updated weights for policy 1, policy_version 916905 (0.0008) [2023-12-26 22:02:24,690][105692] Updated weights for policy 0, policy_version 916850 (0.0011) [2023-12-26 22:02:24,753][105692] Updated weights for policy 0, policy_version 916860 (0.0010) [2023-12-26 22:02:24,813][105692] Updated weights for policy 0, policy_version 916870 (0.0010) [2023-12-26 22:02:24,866][105692] Updated weights for policy 0, policy_version 916880 (0.0011) [2023-12-26 22:02:25,089][105620] Updated weights for policy 1, policy_version 916915 (0.0008) [2023-12-26 22:02:25,154][105620] Updated weights for policy 1, policy_version 916925 (0.0009) [2023-12-26 22:02:25,221][105620] Updated weights for policy 1, policy_version 916935 (0.0008) [2023-12-26 22:02:25,636][105692] Updated weights for policy 0, policy_version 916890 (0.0010) [2023-12-26 22:02:25,686][105692] Updated weights for policy 0, policy_version 916900 (0.0010) [2023-12-26 22:02:25,734][105692] Updated weights for policy 0, policy_version 916910 (0.0010) [2023-12-26 22:02:25,918][105620] Updated weights for policy 1, policy_version 916945 (0.0008) [2023-12-26 22:02:25,985][105620] Updated weights for policy 1, policy_version 916955 (0.0006) [2023-12-26 22:02:26,050][105620] Updated weights for policy 1, policy_version 916965 (0.0009) [2023-12-26 22:02:26,062][104569] Fps is (10 sec: 18022.3, 60 sec: 18978.1, 300 sec: 19133.2). Total num frames: 469532672. Throughput: 0: 9290.6, 1: 9644.4. Samples: 469545068. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 22:02:26,063][104569] Avg episode reward: [(0, '8901.385'), (1, '8304.354')] [2023-12-26 22:02:26,110][105620] Updated weights for policy 1, policy_version 916975 (0.0010) [2023-12-26 22:02:26,525][105692] Updated weights for policy 0, policy_version 916920 (0.0011) [2023-12-26 22:02:26,591][105692] Updated weights for policy 0, policy_version 916930 (0.0011) [2023-12-26 22:02:26,655][105692] Updated weights for policy 0, policy_version 916940 (0.0011) [2023-12-26 22:02:26,698][105620] Updated weights for policy 1, policy_version 916985 (0.0007) [2023-12-26 22:02:26,770][105620] Updated weights for policy 1, policy_version 916995 (0.0008) [2023-12-26 22:02:26,830][105620] Updated weights for policy 1, policy_version 917005 (0.0008) [2023-12-26 22:02:27,305][105692] Updated weights for policy 0, policy_version 916950 (0.0008) [2023-12-26 22:02:27,357][105692] Updated weights for policy 0, policy_version 916960 (0.0010) [2023-12-26 22:02:27,402][105692] Updated weights for policy 0, policy_version 916970 (0.0005) [2023-12-26 22:02:27,596][105620] Updated weights for policy 1, policy_version 917015 (0.0010) [2023-12-26 22:02:27,652][105620] Updated weights for policy 1, policy_version 917025 (0.0010) [2023-12-26 22:02:27,716][105620] Updated weights for policy 1, policy_version 917035 (0.0010) [2023-12-26 22:02:28,060][105692] Updated weights for policy 0, policy_version 916980 (0.0007) [2023-12-26 22:02:28,112][105692] Updated weights for policy 0, policy_version 916990 (0.0011) [2023-12-26 22:02:28,157][105692] Updated weights for policy 0, policy_version 917000 (0.0010) [2023-12-26 22:02:28,361][105620] Updated weights for policy 1, policy_version 917045 (0.0007) [2023-12-26 22:02:28,431][105620] Updated weights for policy 1, policy_version 917055 (0.0008) [2023-12-26 22:02:28,494][105620] Updated weights for policy 1, policy_version 917065 (0.0008) [2023-12-26 22:02:28,942][105692] Updated weights for policy 0, policy_version 917010 (0.0011) [2023-12-26 22:02:28,990][105692] Updated weights for policy 0, policy_version 917020 (0.0010) [2023-12-26 22:02:29,046][105692] Updated weights for policy 0, policy_version 917030 (0.0009) [2023-12-26 22:02:29,103][105692] Updated weights for policy 0, policy_version 917040 (0.0005) [2023-12-26 22:02:29,285][105620] Updated weights for policy 1, policy_version 917075 (0.0008) [2023-12-26 22:02:29,360][105620] Updated weights for policy 1, policy_version 917085 (0.0007) [2023-12-26 22:02:29,409][105586] KL-divergence is very high: 242.5504 [2023-12-26 22:02:29,432][105620] Updated weights for policy 1, policy_version 917095 (0.0008) [2023-12-26 22:02:29,466][105586] KL-divergence is very high: 358.5998 [2023-12-26 22:02:29,787][105692] Updated weights for policy 0, policy_version 917050 (0.0008) [2023-12-26 22:02:29,850][105692] Updated weights for policy 0, policy_version 917060 (0.0009) [2023-12-26 22:02:29,922][105692] Updated weights for policy 0, policy_version 917070 (0.0009) [2023-12-26 22:02:30,137][105620] Updated weights for policy 1, policy_version 917105 (0.0010) [2023-12-26 22:02:30,195][105620] Updated weights for policy 1, policy_version 917115 (0.0009) [2023-12-26 22:02:30,257][105620] Updated weights for policy 1, policy_version 917125 (0.0009) [2023-12-26 22:02:30,316][105620] Updated weights for policy 1, policy_version 917135 (0.0009) [2023-12-26 22:02:30,729][105692] Updated weights for policy 0, policy_version 917080 (0.0009) [2023-12-26 22:02:30,794][105692] Updated weights for policy 0, policy_version 917090 (0.0009) [2023-12-26 22:02:30,859][105692] Updated weights for policy 0, policy_version 917100 (0.0008) [2023-12-26 22:02:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 18978.2, 300 sec: 19133.2). Total num frames: 469630976. Throughput: 0: 9353.7, 1: 9632.8. Samples: 469604460. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 22:02:31,062][104569] Avg episode reward: [(0, '8993.156'), (1, '7863.743')] [2023-12-26 22:02:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000917104_234815488.pth... [2023-12-26 22:02:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000916016_234536960.pth [2023-12-26 22:02:31,085][105620] Updated weights for policy 1, policy_version 917145 (0.0009) [2023-12-26 22:02:31,153][105620] Updated weights for policy 1, policy_version 917155 (0.0008) [2023-12-26 22:02:31,220][105620] Updated weights for policy 1, policy_version 917165 (0.0008) [2023-12-26 22:02:31,238][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000917168_234823680.pth... [2023-12-26 22:02:31,243][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000916016_234528768.pth [2023-12-26 22:02:31,635][105692] Updated weights for policy 0, policy_version 917110 (0.0010) [2023-12-26 22:02:31,699][105692] Updated weights for policy 0, policy_version 917120 (0.0010) [2023-12-26 22:02:31,762][105692] Updated weights for policy 0, policy_version 917130 (0.0009) [2023-12-26 22:02:31,928][105620] Updated weights for policy 1, policy_version 917175 (0.0009) [2023-12-26 22:02:31,993][105620] Updated weights for policy 1, policy_version 917185 (0.0009) [2023-12-26 22:02:32,043][105620] Updated weights for policy 1, policy_version 917195 (0.0009) [2023-12-26 22:02:32,518][105692] Updated weights for policy 0, policy_version 917140 (0.0009) [2023-12-26 22:02:32,573][105692] Updated weights for policy 0, policy_version 917150 (0.0008) [2023-12-26 22:02:32,634][105692] Updated weights for policy 0, policy_version 917160 (0.0008) [2023-12-26 22:02:32,799][105620] Updated weights for policy 1, policy_version 917205 (0.0009) [2023-12-26 22:02:32,845][105620] Updated weights for policy 1, policy_version 917215 (0.0008) [2023-12-26 22:02:32,898][105620] Updated weights for policy 1, policy_version 917225 (0.0008) [2023-12-26 22:02:33,281][105692] Updated weights for policy 0, policy_version 917170 (0.0009) [2023-12-26 22:02:33,344][105692] Updated weights for policy 0, policy_version 917180 (0.0008) [2023-12-26 22:02:33,413][105692] Updated weights for policy 0, policy_version 917190 (0.0008) [2023-12-26 22:02:33,475][105692] Updated weights for policy 0, policy_version 917200 (0.0008) [2023-12-26 22:02:33,726][105620] Updated weights for policy 1, policy_version 917235 (0.0009) [2023-12-26 22:02:33,786][105620] Updated weights for policy 1, policy_version 917245 (0.0010) [2023-12-26 22:02:33,840][105620] Updated weights for policy 1, policy_version 917255 (0.0009) [2023-12-26 22:02:34,059][105692] Updated weights for policy 0, policy_version 917210 (0.0010) [2023-12-26 22:02:34,114][105692] Updated weights for policy 0, policy_version 917220 (0.0010) [2023-12-26 22:02:34,182][105692] Updated weights for policy 0, policy_version 917230 (0.0007) [2023-12-26 22:02:34,640][105620] Updated weights for policy 1, policy_version 917265 (0.0010) [2023-12-26 22:02:34,707][105620] Updated weights for policy 1, policy_version 917275 (0.0009) [2023-12-26 22:02:34,764][105620] Updated weights for policy 1, policy_version 917285 (0.0008) [2023-12-26 22:02:34,829][105620] Updated weights for policy 1, policy_version 917295 (0.0008) [2023-12-26 22:02:34,933][105692] Updated weights for policy 0, policy_version 917240 (0.0009) [2023-12-26 22:02:34,987][105692] Updated weights for policy 0, policy_version 917250 (0.0010) [2023-12-26 22:02:35,048][105692] Updated weights for policy 0, policy_version 917260 (0.0010) [2023-12-26 22:02:35,604][105620] Updated weights for policy 1, policy_version 917305 (0.0008) [2023-12-26 22:02:35,657][105620] Updated weights for policy 1, policy_version 917315 (0.0008) [2023-12-26 22:02:35,717][105620] Updated weights for policy 1, policy_version 917325 (0.0008) [2023-12-26 22:02:35,842][105692] Updated weights for policy 0, policy_version 917270 (0.0011) [2023-12-26 22:02:35,907][105692] Updated weights for policy 0, policy_version 917280 (0.0011) [2023-12-26 22:02:35,972][105692] Updated weights for policy 0, policy_version 917290 (0.0011) [2023-12-26 22:02:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 18978.2, 300 sec: 19133.2). Total num frames: 469729280. Throughput: 0: 9374.1, 1: 9525.2. Samples: 469715768. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 22:02:36,062][104569] Avg episode reward: [(0, '9096.930'), (1, '7963.837')] [2023-12-26 22:02:36,520][105620] Updated weights for policy 1, policy_version 917335 (0.0008) [2023-12-26 22:02:36,577][105620] Updated weights for policy 1, policy_version 917345 (0.0008) [2023-12-26 22:02:36,641][105620] Updated weights for policy 1, policy_version 917355 (0.0008) [2023-12-26 22:02:36,760][105692] Updated weights for policy 0, policy_version 917300 (0.0011) [2023-12-26 22:02:36,822][105692] Updated weights for policy 0, policy_version 917310 (0.0011) [2023-12-26 22:02:36,885][105692] Updated weights for policy 0, policy_version 917320 (0.0011) [2023-12-26 22:02:37,428][105620] Updated weights for policy 1, policy_version 917365 (0.0008) [2023-12-26 22:02:37,489][105620] Updated weights for policy 1, policy_version 917375 (0.0008) [2023-12-26 22:02:37,552][105620] Updated weights for policy 1, policy_version 917385 (0.0008) [2023-12-26 22:02:37,652][105692] Updated weights for policy 0, policy_version 917330 (0.0010) [2023-12-26 22:02:37,701][105692] Updated weights for policy 0, policy_version 917340 (0.0010) [2023-12-26 22:02:37,757][105692] Updated weights for policy 0, policy_version 917350 (0.0010) [2023-12-26 22:02:37,825][105692] Updated weights for policy 0, policy_version 917360 (0.0011) [2023-12-26 22:02:38,323][105620] Updated weights for policy 1, policy_version 917395 (0.0008) [2023-12-26 22:02:38,384][105620] Updated weights for policy 1, policy_version 917405 (0.0008) [2023-12-26 22:02:38,447][105620] Updated weights for policy 1, policy_version 917415 (0.0010) [2023-12-26 22:02:38,612][105692] Updated weights for policy 0, policy_version 917370 (0.0010) [2023-12-26 22:02:38,673][105692] Updated weights for policy 0, policy_version 917380 (0.0011) [2023-12-26 22:02:38,726][105692] Updated weights for policy 0, policy_version 917390 (0.0010) [2023-12-26 22:02:39,100][105620] Updated weights for policy 1, policy_version 917425 (0.0010) [2023-12-26 22:02:39,161][105620] Updated weights for policy 1, policy_version 917435 (0.0007) [2023-12-26 22:02:39,225][105620] Updated weights for policy 1, policy_version 917445 (0.0007) [2023-12-26 22:02:39,289][105620] Updated weights for policy 1, policy_version 917455 (0.0010) [2023-12-26 22:02:39,525][105692] Updated weights for policy 0, policy_version 917400 (0.0010) [2023-12-26 22:02:39,585][105692] Updated weights for policy 0, policy_version 917410 (0.0009) [2023-12-26 22:02:39,651][105692] Updated weights for policy 0, policy_version 917420 (0.0009) [2023-12-26 22:02:40,002][105620] Updated weights for policy 1, policy_version 917465 (0.0009) [2023-12-26 22:02:40,075][105620] Updated weights for policy 1, policy_version 917475 (0.0009) [2023-12-26 22:02:40,140][105620] Updated weights for policy 1, policy_version 917485 (0.0010) [2023-12-26 22:02:40,394][105692] Updated weights for policy 0, policy_version 917430 (0.0010) [2023-12-26 22:02:40,462][105692] Updated weights for policy 0, policy_version 917440 (0.0010) [2023-12-26 22:02:40,535][105692] Updated weights for policy 0, policy_version 917450 (0.0005) [2023-12-26 22:02:40,792][105620] Updated weights for policy 1, policy_version 917495 (0.0008) [2023-12-26 22:02:40,850][105620] Updated weights for policy 1, policy_version 917505 (0.0010) [2023-12-26 22:02:40,916][105620] Updated weights for policy 1, policy_version 917515 (0.0010) [2023-12-26 22:02:40,923][105586] KL-divergence is very high: 102.8797 [2023-12-26 22:02:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18841.6, 300 sec: 19105.4). Total num frames: 469819392. Throughput: 0: 9315.2, 1: 9540.3. Samples: 469825944. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 22:02:41,062][104569] Avg episode reward: [(0, '9184.150'), (1, '8669.202')] [2023-12-26 22:02:41,215][105692] Updated weights for policy 0, policy_version 917460 (0.0006) [2023-12-26 22:02:41,282][105692] Updated weights for policy 0, policy_version 917470 (0.0010) [2023-12-26 22:02:41,359][105692] Updated weights for policy 0, policy_version 917480 (0.0011) [2023-12-26 22:02:41,743][105620] Updated weights for policy 1, policy_version 917525 (0.0010) [2023-12-26 22:02:41,809][105620] Updated weights for policy 1, policy_version 917535 (0.0009) [2023-12-26 22:02:41,867][105620] Updated weights for policy 1, policy_version 917545 (0.0008) [2023-12-26 22:02:42,156][105692] Updated weights for policy 0, policy_version 917490 (0.0009) [2023-12-26 22:02:42,220][105692] Updated weights for policy 0, policy_version 917500 (0.0010) [2023-12-26 22:02:42,280][105692] Updated weights for policy 0, policy_version 917510 (0.0009) [2023-12-26 22:02:42,348][105692] Updated weights for policy 0, policy_version 917520 (0.0009) [2023-12-26 22:02:42,633][105620] Updated weights for policy 1, policy_version 917555 (0.0008) [2023-12-26 22:02:42,692][105620] Updated weights for policy 1, policy_version 917565 (0.0008) [2023-12-26 22:02:42,754][105620] Updated weights for policy 1, policy_version 917575 (0.0009) [2023-12-26 22:02:43,079][105692] Updated weights for policy 0, policy_version 917530 (0.0009) [2023-12-26 22:02:43,137][105692] Updated weights for policy 0, policy_version 917540 (0.0009) [2023-12-26 22:02:43,189][105692] Updated weights for policy 0, policy_version 917550 (0.0009) [2023-12-26 22:02:43,495][105620] Updated weights for policy 1, policy_version 917585 (0.0009) [2023-12-26 22:02:43,551][105620] Updated weights for policy 1, policy_version 917595 (0.0009) [2023-12-26 22:02:43,601][105620] Updated weights for policy 1, policy_version 917605 (0.0009) [2023-12-26 22:02:43,661][105620] Updated weights for policy 1, policy_version 917615 (0.0009) [2023-12-26 22:02:43,960][105692] Updated weights for policy 0, policy_version 917560 (0.0009) [2023-12-26 22:02:44,008][105692] Updated weights for policy 0, policy_version 917570 (0.0009) [2023-12-26 22:02:44,067][105692] Updated weights for policy 0, policy_version 917580 (0.0009) [2023-12-26 22:02:44,441][105620] Updated weights for policy 1, policy_version 917625 (0.0010) [2023-12-26 22:02:44,495][105620] Updated weights for policy 1, policy_version 917635 (0.0010) [2023-12-26 22:02:44,556][105620] Updated weights for policy 1, policy_version 917645 (0.0009) [2023-12-26 22:02:44,868][105692] Updated weights for policy 0, policy_version 917590 (0.0009) [2023-12-26 22:02:44,932][105692] Updated weights for policy 0, policy_version 917600 (0.0009) [2023-12-26 22:02:44,991][105692] Updated weights for policy 0, policy_version 917610 (0.0009) [2023-12-26 22:02:45,336][105620] Updated weights for policy 1, policy_version 917655 (0.0010) [2023-12-26 22:02:45,392][105620] Updated weights for policy 1, policy_version 917665 (0.0009) [2023-12-26 22:02:45,456][105620] Updated weights for policy 1, policy_version 917675 (0.0009) [2023-12-26 22:02:45,765][105692] Updated weights for policy 0, policy_version 917620 (0.0009) [2023-12-26 22:02:45,833][105692] Updated weights for policy 0, policy_version 917630 (0.0009) [2023-12-26 22:02:45,904][105692] Updated weights for policy 0, policy_version 917640 (0.0010) [2023-12-26 22:02:46,062][104569] Fps is (10 sec: 18022.3, 60 sec: 18841.6, 300 sec: 19077.6). Total num frames: 469909504. Throughput: 0: 9247.3, 1: 9478.8. Samples: 469880032. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 22:02:46,063][104569] Avg episode reward: [(0, '3364.002'), (1, '7874.004')] [2023-12-26 22:02:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000917680_234954752.pth... [2023-12-26 22:02:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000917648_234954752.pth... [2023-12-26 22:02:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000916592_234676224.pth [2023-12-26 22:02:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000916560_234676224.pth [2023-12-26 22:02:46,132][105620] Updated weights for policy 1, policy_version 917685 (0.0007) [2023-12-26 22:02:46,191][105620] Updated weights for policy 1, policy_version 917695 (0.0005) [2023-12-26 22:02:46,261][105620] Updated weights for policy 1, policy_version 917705 (0.0006) [2023-12-26 22:02:46,607][105692] Updated weights for policy 0, policy_version 917650 (0.0010) [2023-12-26 22:02:46,657][105692] Updated weights for policy 0, policy_version 917660 (0.0006) [2023-12-26 22:02:46,708][105692] Updated weights for policy 0, policy_version 917670 (0.0005) [2023-12-26 22:02:46,765][105692] Updated weights for policy 0, policy_version 917680 (0.0007) [2023-12-26 22:02:47,036][105620] Updated weights for policy 1, policy_version 917715 (0.0007) [2023-12-26 22:02:47,097][105620] Updated weights for policy 1, policy_version 917725 (0.0009) [2023-12-26 22:02:47,157][105620] Updated weights for policy 1, policy_version 917735 (0.0009) [2023-12-26 22:02:47,346][105692] Updated weights for policy 0, policy_version 917690 (0.0008) [2023-12-26 22:02:47,394][105692] Updated weights for policy 0, policy_version 917700 (0.0006) [2023-12-26 22:02:47,449][105692] Updated weights for policy 0, policy_version 917710 (0.0006) [2023-12-26 22:02:48,004][105620] Updated weights for policy 1, policy_version 917745 (0.0009) [2023-12-26 22:02:48,052][105692] Updated weights for policy 0, policy_version 917720 (0.0007) [2023-12-26 22:02:48,064][105620] Updated weights for policy 1, policy_version 917755 (0.0010) [2023-12-26 22:02:48,112][105692] Updated weights for policy 0, policy_version 917730 (0.0010) [2023-12-26 22:02:48,122][105620] Updated weights for policy 1, policy_version 917765 (0.0009) [2023-12-26 22:02:48,166][105692] Updated weights for policy 0, policy_version 917740 (0.0005) [2023-12-26 22:02:48,184][105620] Updated weights for policy 1, policy_version 917775 (0.0008) [2023-12-26 22:02:48,898][105620] Updated weights for policy 1, policy_version 917785 (0.0009) [2023-12-26 22:02:48,961][105692] Updated weights for policy 0, policy_version 917750 (0.0008) [2023-12-26 22:02:48,962][105620] Updated weights for policy 1, policy_version 917795 (0.0009) [2023-12-26 22:02:49,015][105692] Updated weights for policy 0, policy_version 917760 (0.0006) [2023-12-26 22:02:49,020][105620] Updated weights for policy 1, policy_version 917805 (0.0008) [2023-12-26 22:02:49,074][105692] Updated weights for policy 0, policy_version 917770 (0.0008) [2023-12-26 22:02:49,760][105620] Updated weights for policy 1, policy_version 917815 (0.0008) [2023-12-26 22:02:49,830][105620] Updated weights for policy 1, policy_version 917825 (0.0007) [2023-12-26 22:02:49,850][105692] Updated weights for policy 0, policy_version 917780 (0.0009) [2023-12-26 22:02:49,895][105620] Updated weights for policy 1, policy_version 917835 (0.0008) [2023-12-26 22:02:49,913][105692] Updated weights for policy 0, policy_version 917790 (0.0009) [2023-12-26 22:02:49,978][105692] Updated weights for policy 0, policy_version 917800 (0.0008) [2023-12-26 22:02:50,612][105620] Updated weights for policy 1, policy_version 917845 (0.0008) [2023-12-26 22:02:50,649][105586] KL-divergence is very high: 221.9681 [2023-12-26 22:02:50,676][105620] Updated weights for policy 1, policy_version 917855 (0.0009) [2023-12-26 22:02:50,688][105692] Updated weights for policy 0, policy_version 917810 (0.0008) [2023-12-26 22:02:50,698][105586] KL-divergence is very high: 408.4305 [2023-12-26 22:02:50,737][105620] Updated weights for policy 1, policy_version 917865 (0.0007) [2023-12-26 22:02:50,747][105692] Updated weights for policy 0, policy_version 917820 (0.0007) [2023-12-26 22:02:50,749][105586] KL-divergence is very high: 465.9493 [2023-12-26 22:02:50,799][105692] Updated weights for policy 0, policy_version 917830 (0.0008) [2023-12-26 22:02:50,849][105692] Updated weights for policy 0, policy_version 917840 (0.0008) [2023-12-26 22:02:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 18978.1, 300 sec: 19105.4). Total num frames: 470007808. Throughput: 0: 9288.1, 1: 9427.5. Samples: 469993804. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 22:02:51,062][104569] Avg episode reward: [(0, '2702.612'), (1, '7275.417')] [2023-12-26 22:02:51,536][105620] Updated weights for policy 1, policy_version 917875 (0.0010) [2023-12-26 22:02:51,601][105620] Updated weights for policy 1, policy_version 917885 (0.0009) [2023-12-26 22:02:51,667][105620] Updated weights for policy 1, policy_version 917895 (0.0008) [2023-12-26 22:02:51,739][105692] Updated weights for policy 0, policy_version 917850 (0.0009) [2023-12-26 22:02:51,810][105692] Updated weights for policy 0, policy_version 917860 (0.0009) [2023-12-26 22:02:51,879][105692] Updated weights for policy 0, policy_version 917870 (0.0010) [2023-12-26 22:02:52,385][105620] Updated weights for policy 1, policy_version 917905 (0.0009) [2023-12-26 22:02:52,452][105620] Updated weights for policy 1, policy_version 917915 (0.0008) [2023-12-26 22:02:52,514][105620] Updated weights for policy 1, policy_version 917925 (0.0009) [2023-12-26 22:02:52,572][105620] Updated weights for policy 1, policy_version 917935 (0.0009) [2023-12-26 22:02:52,654][105692] Updated weights for policy 0, policy_version 917880 (0.0009) [2023-12-26 22:02:52,717][105692] Updated weights for policy 0, policy_version 917890 (0.0009) [2023-12-26 22:02:52,776][105692] Updated weights for policy 0, policy_version 917900 (0.0009) [2023-12-26 22:02:53,263][105620] Updated weights for policy 1, policy_version 917945 (0.0010) [2023-12-26 22:02:53,326][105620] Updated weights for policy 1, policy_version 917955 (0.0009) [2023-12-26 22:02:53,385][105620] Updated weights for policy 1, policy_version 917965 (0.0008) [2023-12-26 22:02:53,470][105692] Updated weights for policy 0, policy_version 917910 (0.0009) [2023-12-26 22:02:53,530][105692] Updated weights for policy 0, policy_version 917920 (0.0009) [2023-12-26 22:02:53,577][105692] Updated weights for policy 0, policy_version 917930 (0.0009) [2023-12-26 22:02:54,133][105620] Updated weights for policy 1, policy_version 917975 (0.0010) [2023-12-26 22:02:54,197][105620] Updated weights for policy 1, policy_version 917985 (0.0011) [2023-12-26 22:02:54,256][105620] Updated weights for policy 1, policy_version 917995 (0.0011) [2023-12-26 22:02:54,292][105692] Updated weights for policy 0, policy_version 917940 (0.0007) [2023-12-26 22:02:54,349][105692] Updated weights for policy 0, policy_version 917950 (0.0005) [2023-12-26 22:02:54,403][105692] Updated weights for policy 0, policy_version 917960 (0.0005) [2023-12-26 22:02:55,013][105620] Updated weights for policy 1, policy_version 918005 (0.0010) [2023-12-26 22:02:55,047][105692] Updated weights for policy 0, policy_version 917970 (0.0006) [2023-12-26 22:02:55,080][105620] Updated weights for policy 1, policy_version 918015 (0.0008) [2023-12-26 22:02:55,111][105692] Updated weights for policy 0, policy_version 917980 (0.0008) [2023-12-26 22:02:55,142][105620] Updated weights for policy 1, policy_version 918025 (0.0007) [2023-12-26 22:02:55,173][105692] Updated weights for policy 0, policy_version 917990 (0.0007) [2023-12-26 22:02:55,234][105692] Updated weights for policy 0, policy_version 918000 (0.0009) [2023-12-26 22:02:55,834][105620] Updated weights for policy 1, policy_version 918035 (0.0008) [2023-12-26 22:02:55,887][105620] Updated weights for policy 1, policy_version 918045 (0.0008) [2023-12-26 22:02:55,937][105620] Updated weights for policy 1, policy_version 918055 (0.0010) [2023-12-26 22:02:55,984][105692] Updated weights for policy 0, policy_version 918010 (0.0006) [2023-12-26 22:02:56,034][105692] Updated weights for policy 0, policy_version 918020 (0.0009) [2023-12-26 22:02:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 18841.6, 300 sec: 19077.6). Total num frames: 470097920. Throughput: 0: 9323.1, 1: 9345.4. Samples: 470106196. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 22:02:56,062][104569] Avg episode reward: [(0, '6200.643'), (1, '7790.553')] [2023-12-26 22:02:56,093][105692] Updated weights for policy 0, policy_version 918031 (0.0010) [2023-12-26 22:02:56,557][105620] Updated weights for policy 1, policy_version 918065 (0.0010) [2023-12-26 22:02:56,617][105620] Updated weights for policy 1, policy_version 918075 (0.0008) [2023-12-26 22:02:56,663][105620] Updated weights for policy 1, policy_version 918085 (0.0005) [2023-12-26 22:02:56,711][105620] Updated weights for policy 1, policy_version 918095 (0.0006) [2023-12-26 22:02:56,869][105692] Updated weights for policy 0, policy_version 918041 (0.0010) [2023-12-26 22:02:56,925][105692] Updated weights for policy 0, policy_version 918051 (0.0008) [2023-12-26 22:02:56,991][105692] Updated weights for policy 0, policy_version 918061 (0.0006) [2023-12-26 22:02:57,421][105620] Updated weights for policy 1, policy_version 918105 (0.0009) [2023-12-26 22:02:57,477][105620] Updated weights for policy 1, policy_version 918115 (0.0008) [2023-12-26 22:02:57,537][105620] Updated weights for policy 1, policy_version 918125 (0.0009) [2023-12-26 22:02:57,652][105692] Updated weights for policy 0, policy_version 918071 (0.0009) [2023-12-26 22:02:57,698][105692] Updated weights for policy 0, policy_version 918081 (0.0008) [2023-12-26 22:02:57,749][105692] Updated weights for policy 0, policy_version 918091 (0.0008) [2023-12-26 22:02:58,342][105620] Updated weights for policy 1, policy_version 918135 (0.0008) [2023-12-26 22:02:58,411][105620] Updated weights for policy 1, policy_version 918145 (0.0008) [2023-12-26 22:02:58,424][105692] Updated weights for policy 0, policy_version 918101 (0.0007) [2023-12-26 22:02:58,474][105620] Updated weights for policy 1, policy_version 918155 (0.0009) [2023-12-26 22:02:58,493][105692] Updated weights for policy 0, policy_version 918111 (0.0008) [2023-12-26 22:02:58,563][105692] Updated weights for policy 0, policy_version 918121 (0.0008) [2023-12-26 22:02:59,258][105620] Updated weights for policy 1, policy_version 918165 (0.0008) [2023-12-26 22:02:59,321][105620] Updated weights for policy 1, policy_version 918175 (0.0008) [2023-12-26 22:02:59,388][105620] Updated weights for policy 1, policy_version 918185 (0.0009) [2023-12-26 22:02:59,487][105692] Updated weights for policy 0, policy_version 918131 (0.0010) [2023-12-26 22:02:59,546][105692] Updated weights for policy 0, policy_version 918141 (0.0009) [2023-12-26 22:02:59,596][105692] Updated weights for policy 0, policy_version 918151 (0.0008) [2023-12-26 22:03:00,107][105620] Updated weights for policy 1, policy_version 918195 (0.0007) [2023-12-26 22:03:00,169][105620] Updated weights for policy 1, policy_version 918205 (0.0007) [2023-12-26 22:03:00,232][105620] Updated weights for policy 1, policy_version 918215 (0.0010) [2023-12-26 22:03:00,464][105692] Updated weights for policy 0, policy_version 918161 (0.0009) [2023-12-26 22:03:00,526][105692] Updated weights for policy 0, policy_version 918171 (0.0009) [2023-12-26 22:03:00,586][105692] Updated weights for policy 0, policy_version 918181 (0.0009) [2023-12-26 22:03:00,642][105692] Updated weights for policy 0, policy_version 918191 (0.0008) [2023-12-26 22:03:00,848][105620] Updated weights for policy 1, policy_version 918225 (0.0006) [2023-12-26 22:03:00,904][105620] Updated weights for policy 1, policy_version 918235 (0.0005) [2023-12-26 22:03:00,960][105620] Updated weights for policy 1, policy_version 918245 (0.0007) [2023-12-26 22:03:01,017][105620] Updated weights for policy 1, policy_version 918255 (0.0009) [2023-12-26 22:03:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 18841.6, 300 sec: 19049.9). Total num frames: 470196224. Throughput: 0: 9360.3, 1: 9351.3. Samples: 470164080. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 22:03:01,062][104569] Avg episode reward: [(0, '8045.857'), (1, '7853.962')] [2023-12-26 22:03:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000918192_235094016.pth... [2023-12-26 22:03:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000918256_235102208.pth... [2023-12-26 22:03:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000917168_234823680.pth [2023-12-26 22:03:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000917104_234815488.pth [2023-12-26 22:03:01,480][105692] Updated weights for policy 0, policy_version 918201 (0.0009) [2023-12-26 22:03:01,531][105692] Updated weights for policy 0, policy_version 918211 (0.0009) [2023-12-26 22:03:01,592][105692] Updated weights for policy 0, policy_version 918221 (0.0009) [2023-12-26 22:03:01,772][105620] Updated weights for policy 1, policy_version 918265 (0.0008) [2023-12-26 22:03:01,830][105620] Updated weights for policy 1, policy_version 918275 (0.0009) [2023-12-26 22:03:01,887][105620] Updated weights for policy 1, policy_version 918285 (0.0009) [2023-12-26 22:03:02,402][105692] Updated weights for policy 0, policy_version 918231 (0.0007) [2023-12-26 22:03:02,465][105692] Updated weights for policy 0, policy_version 918241 (0.0007) [2023-12-26 22:03:02,526][105692] Updated weights for policy 0, policy_version 918251 (0.0005) [2023-12-26 22:03:02,648][105620] Updated weights for policy 1, policy_version 918295 (0.0007) [2023-12-26 22:03:02,715][105620] Updated weights for policy 1, policy_version 918305 (0.0006) [2023-12-26 22:03:02,778][105620] Updated weights for policy 1, policy_version 918315 (0.0007) [2023-12-26 22:03:03,136][105692] Updated weights for policy 0, policy_version 918261 (0.0005) [2023-12-26 22:03:03,193][105692] Updated weights for policy 0, policy_version 918271 (0.0007) [2023-12-26 22:03:03,250][105692] Updated weights for policy 0, policy_version 918281 (0.0009) [2023-12-26 22:03:03,464][105620] Updated weights for policy 1, policy_version 918325 (0.0010) [2023-12-26 22:03:03,507][105620] Updated weights for policy 1, policy_version 918335 (0.0008) [2023-12-26 22:03:03,553][105620] Updated weights for policy 1, policy_version 918345 (0.0005) [2023-12-26 22:03:04,026][105692] Updated weights for policy 0, policy_version 918291 (0.0008) [2023-12-26 22:03:04,099][105692] Updated weights for policy 0, policy_version 918301 (0.0011) [2023-12-26 22:03:04,141][105620] Updated weights for policy 1, policy_version 918355 (0.0005) [2023-12-26 22:03:04,171][105692] Updated weights for policy 0, policy_version 918311 (0.0011) [2023-12-26 22:03:04,202][105620] Updated weights for policy 1, policy_version 918365 (0.0006) [2023-12-26 22:03:04,264][105620] Updated weights for policy 1, policy_version 918375 (0.0007) [2023-12-26 22:03:04,874][105692] Updated weights for policy 0, policy_version 918321 (0.0010) [2023-12-26 22:03:04,946][105692] Updated weights for policy 0, policy_version 918331 (0.0010) [2023-12-26 22:03:05,013][105692] Updated weights for policy 0, policy_version 918341 (0.0010) [2023-12-26 22:03:05,039][105620] Updated weights for policy 1, policy_version 918385 (0.0008) [2023-12-26 22:03:05,074][105692] Updated weights for policy 0, policy_version 918351 (0.0005) [2023-12-26 22:03:05,093][105620] Updated weights for policy 1, policy_version 918395 (0.0009) [2023-12-26 22:03:05,156][105620] Updated weights for policy 1, policy_version 918405 (0.0009) [2023-12-26 22:03:05,208][105620] Updated weights for policy 1, policy_version 918415 (0.0008) [2023-12-26 22:03:05,670][105692] Updated weights for policy 0, policy_version 918361 (0.0007) [2023-12-26 22:03:05,721][105692] Updated weights for policy 0, policy_version 918371 (0.0010) [2023-12-26 22:03:05,771][105692] Updated weights for policy 0, policy_version 918381 (0.0008) [2023-12-26 22:03:06,032][105620] Updated weights for policy 1, policy_version 918425 (0.0009) [2023-12-26 22:03:06,062][104569] Fps is (10 sec: 18841.4, 60 sec: 18705.1, 300 sec: 19022.1). Total num frames: 470286336. Throughput: 0: 9315.0, 1: 9420.4. Samples: 470276728. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 22:03:06,063][104569] Avg episode reward: [(0, '9100.727'), (1, '7760.643')] [2023-12-26 22:03:06,082][105620] Updated weights for policy 1, policy_version 918435 (0.0009) [2023-12-26 22:03:06,155][105620] Updated weights for policy 1, policy_version 918445 (0.0010) [2023-12-26 22:03:06,587][105692] Updated weights for policy 0, policy_version 918391 (0.0010) [2023-12-26 22:03:06,656][105692] Updated weights for policy 0, policy_version 918401 (0.0011) [2023-12-26 22:03:06,724][105692] Updated weights for policy 0, policy_version 918411 (0.0011) [2023-12-26 22:03:06,986][105620] Updated weights for policy 1, policy_version 918455 (0.0009) [2023-12-26 22:03:07,056][105620] Updated weights for policy 1, policy_version 918465 (0.0009) [2023-12-26 22:03:07,122][105620] Updated weights for policy 1, policy_version 918475 (0.0008) [2023-12-26 22:03:07,481][105692] Updated weights for policy 0, policy_version 918421 (0.0010) [2023-12-26 22:03:07,536][105692] Updated weights for policy 0, policy_version 918431 (0.0010) [2023-12-26 22:03:07,597][105692] Updated weights for policy 0, policy_version 918441 (0.0006) [2023-12-26 22:03:07,890][105620] Updated weights for policy 1, policy_version 918485 (0.0010) [2023-12-26 22:03:07,944][105620] Updated weights for policy 1, policy_version 918495 (0.0007) [2023-12-26 22:03:07,999][105620] Updated weights for policy 1, policy_version 918505 (0.0005) [2023-12-26 22:03:08,223][105692] Updated weights for policy 0, policy_version 918451 (0.0007) [2023-12-26 22:03:08,276][105692] Updated weights for policy 0, policy_version 918461 (0.0008) [2023-12-26 22:03:08,334][105692] Updated weights for policy 0, policy_version 918471 (0.0009) [2023-12-26 22:03:08,650][105620] Updated weights for policy 1, policy_version 918515 (0.0007) [2023-12-26 22:03:08,718][105620] Updated weights for policy 1, policy_version 918525 (0.0007) [2023-12-26 22:03:08,772][105620] Updated weights for policy 1, policy_version 918535 (0.0008) [2023-12-26 22:03:09,163][105692] Updated weights for policy 0, policy_version 918481 (0.0010) [2023-12-26 22:03:09,222][105692] Updated weights for policy 0, policy_version 918491 (0.0009) [2023-12-26 22:03:09,291][105692] Updated weights for policy 0, policy_version 918501 (0.0009) [2023-12-26 22:03:09,366][105692] Updated weights for policy 0, policy_version 918511 (0.0009) [2023-12-26 22:03:09,473][105620] Updated weights for policy 1, policy_version 918545 (0.0007) [2023-12-26 22:03:09,537][105620] Updated weights for policy 1, policy_version 918555 (0.0008) [2023-12-26 22:03:09,601][105620] Updated weights for policy 1, policy_version 918565 (0.0009) [2023-12-26 22:03:09,663][105620] Updated weights for policy 1, policy_version 918575 (0.0008) [2023-12-26 22:03:10,192][105692] Updated weights for policy 0, policy_version 918521 (0.0008) [2023-12-26 22:03:10,252][105692] Updated weights for policy 0, policy_version 918531 (0.0008) [2023-12-26 22:03:10,313][105692] Updated weights for policy 0, policy_version 918541 (0.0008) [2023-12-26 22:03:10,379][105620] Updated weights for policy 1, policy_version 918585 (0.0010) [2023-12-26 22:03:10,443][105620] Updated weights for policy 1, policy_version 918595 (0.0011) [2023-12-26 22:03:10,509][105620] Updated weights for policy 1, policy_version 918605 (0.0010) [2023-12-26 22:03:11,062][104569] Fps is (10 sec: 18022.3, 60 sec: 18705.1, 300 sec: 19022.1). Total num frames: 470376448. Throughput: 0: 9368.0, 1: 9377.4. Samples: 470388612. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 22:03:11,062][104569] Avg episode reward: [(0, '8387.469'), (1, '8129.892')] [2023-12-26 22:03:11,130][105692] Updated weights for policy 0, policy_version 918551 (0.0009) [2023-12-26 22:03:11,194][105692] Updated weights for policy 0, policy_version 918561 (0.0008) [2023-12-26 22:03:11,259][105692] Updated weights for policy 0, policy_version 918571 (0.0009) [2023-12-26 22:03:11,287][105620] Updated weights for policy 1, policy_version 918615 (0.0009) [2023-12-26 22:03:11,357][105620] Updated weights for policy 1, policy_version 918625 (0.0009) [2023-12-26 22:03:11,422][105620] Updated weights for policy 1, policy_version 918635 (0.0008) [2023-12-26 22:03:12,076][105692] Updated weights for policy 0, policy_version 918581 (0.0009) [2023-12-26 22:03:12,127][105620] Updated weights for policy 1, policy_version 918645 (0.0007) [2023-12-26 22:03:12,133][105692] Updated weights for policy 0, policy_version 918591 (0.0008) [2023-12-26 22:03:12,189][105620] Updated weights for policy 1, policy_version 918655 (0.0007) [2023-12-26 22:03:12,200][105692] Updated weights for policy 0, policy_version 918601 (0.0008) [2023-12-26 22:03:12,251][105620] Updated weights for policy 1, policy_version 918665 (0.0008) [2023-12-26 22:03:12,962][105692] Updated weights for policy 0, policy_version 918611 (0.0006) [2023-12-26 22:03:13,019][105692] Updated weights for policy 0, policy_version 918621 (0.0006) [2023-12-26 22:03:13,064][105620] Updated weights for policy 1, policy_version 918675 (0.0009) [2023-12-26 22:03:13,078][105692] Updated weights for policy 0, policy_version 918631 (0.0005) [2023-12-26 22:03:13,124][105620] Updated weights for policy 1, policy_version 918685 (0.0008) [2023-12-26 22:03:13,190][105620] Updated weights for policy 1, policy_version 918695 (0.0008) [2023-12-26 22:03:13,655][105692] Updated weights for policy 0, policy_version 918641 (0.0006) [2023-12-26 22:03:13,708][105692] Updated weights for policy 0, policy_version 918651 (0.0009) [2023-12-26 22:03:13,755][105692] Updated weights for policy 0, policy_version 918661 (0.0009) [2023-12-26 22:03:13,804][105692] Updated weights for policy 0, policy_version 918671 (0.0009) [2023-12-26 22:03:13,950][105620] Updated weights for policy 1, policy_version 918705 (0.0008) [2023-12-26 22:03:14,015][105620] Updated weights for policy 1, policy_version 918715 (0.0006) [2023-12-26 22:03:14,082][105620] Updated weights for policy 1, policy_version 918725 (0.0005) [2023-12-26 22:03:14,145][105620] Updated weights for policy 1, policy_version 918735 (0.0006) [2023-12-26 22:03:14,621][105692] Updated weights for policy 0, policy_version 918681 (0.0009) [2023-12-26 22:03:14,689][105692] Updated weights for policy 0, policy_version 918691 (0.0010) [2023-12-26 22:03:14,753][105692] Updated weights for policy 0, policy_version 918701 (0.0009) [2023-12-26 22:03:14,839][105620] Updated weights for policy 1, policy_version 918745 (0.0010) [2023-12-26 22:03:14,897][105620] Updated weights for policy 1, policy_version 918755 (0.0010) [2023-12-26 22:03:14,951][105620] Updated weights for policy 1, policy_version 918765 (0.0011) [2023-12-26 22:03:15,598][105692] Updated weights for policy 0, policy_version 918711 (0.0009) [2023-12-26 22:03:15,659][105692] Updated weights for policy 0, policy_version 918721 (0.0008) [2023-12-26 22:03:15,681][105620] Updated weights for policy 1, policy_version 918775 (0.0009) [2023-12-26 22:03:15,716][105692] Updated weights for policy 0, policy_version 918731 (0.0006) [2023-12-26 22:03:15,734][105620] Updated weights for policy 1, policy_version 918785 (0.0006) [2023-12-26 22:03:15,794][105620] Updated weights for policy 1, policy_version 918795 (0.0007) [2023-12-26 22:03:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18705.1, 300 sec: 19022.1). Total num frames: 470474752. Throughput: 0: 9342.7, 1: 9318.3. Samples: 470444208. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 22:03:16,063][104569] Avg episode reward: [(0, '7915.576'), (1, '8295.809')] [2023-12-26 22:03:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000918736_235233280.pth... [2023-12-26 22:03:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000918800_235241472.pth... [2023-12-26 22:03:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000917680_234954752.pth [2023-12-26 22:03:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000917648_234954752.pth [2023-12-26 22:03:16,463][105692] Updated weights for policy 0, policy_version 918741 (0.0008) [2023-12-26 22:03:16,511][105692] Updated weights for policy 0, policy_version 918751 (0.0009) [2023-12-26 22:03:16,561][105620] Updated weights for policy 1, policy_version 918805 (0.0009) [2023-12-26 22:03:16,562][105692] Updated weights for policy 0, policy_version 918761 (0.0007) [2023-12-26 22:03:16,620][105620] Updated weights for policy 1, policy_version 918815 (0.0008) [2023-12-26 22:03:16,673][105620] Updated weights for policy 1, policy_version 918825 (0.0008) [2023-12-26 22:03:17,221][105692] Updated weights for policy 0, policy_version 918771 (0.0007) [2023-12-26 22:03:17,276][105692] Updated weights for policy 0, policy_version 918781 (0.0007) [2023-12-26 22:03:17,336][105692] Updated weights for policy 0, policy_version 918791 (0.0009) [2023-12-26 22:03:17,506][105620] Updated weights for policy 1, policy_version 918835 (0.0009) [2023-12-26 22:03:17,568][105620] Updated weights for policy 1, policy_version 918845 (0.0009) [2023-12-26 22:03:17,628][105620] Updated weights for policy 1, policy_version 918855 (0.0009) [2023-12-26 22:03:18,044][105692] Updated weights for policy 0, policy_version 918801 (0.0010) [2023-12-26 22:03:18,094][105692] Updated weights for policy 0, policy_version 918811 (0.0010) [2023-12-26 22:03:18,147][105692] Updated weights for policy 0, policy_version 918821 (0.0010) [2023-12-26 22:03:18,201][105692] Updated weights for policy 0, policy_version 918831 (0.0011) [2023-12-26 22:03:18,414][105620] Updated weights for policy 1, policy_version 918865 (0.0009) [2023-12-26 22:03:18,477][105620] Updated weights for policy 1, policy_version 918875 (0.0008) [2023-12-26 22:03:18,536][105620] Updated weights for policy 1, policy_version 918885 (0.0007) [2023-12-26 22:03:18,592][105620] Updated weights for policy 1, policy_version 918895 (0.0009) [2023-12-26 22:03:19,014][105692] Updated weights for policy 0, policy_version 918841 (0.0009) [2023-12-26 22:03:19,078][105692] Updated weights for policy 0, policy_version 918851 (0.0009) [2023-12-26 22:03:19,139][105692] Updated weights for policy 0, policy_version 918861 (0.0007) [2023-12-26 22:03:19,400][105620] Updated weights for policy 1, policy_version 918905 (0.0010) [2023-12-26 22:03:19,456][105620] Updated weights for policy 1, policy_version 918915 (0.0009) [2023-12-26 22:03:19,525][105620] Updated weights for policy 1, policy_version 918925 (0.0009) [2023-12-26 22:03:19,919][105692] Updated weights for policy 0, policy_version 918871 (0.0008) [2023-12-26 22:03:19,994][105692] Updated weights for policy 0, policy_version 918881 (0.0008) [2023-12-26 22:03:20,061][105692] Updated weights for policy 0, policy_version 918891 (0.0007) [2023-12-26 22:03:20,305][105620] Updated weights for policy 1, policy_version 918935 (0.0008) [2023-12-26 22:03:20,377][105620] Updated weights for policy 1, policy_version 918945 (0.0008) [2023-12-26 22:03:20,443][105620] Updated weights for policy 1, policy_version 918955 (0.0008) [2023-12-26 22:03:20,772][105692] Updated weights for policy 0, policy_version 918901 (0.0008) [2023-12-26 22:03:20,838][105692] Updated weights for policy 0, policy_version 918911 (0.0009) [2023-12-26 22:03:20,892][105692] Updated weights for policy 0, policy_version 918921 (0.0009) [2023-12-26 22:03:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 18841.6, 300 sec: 19022.1). Total num frames: 470564864. Throughput: 0: 9304.8, 1: 9328.2. Samples: 470554256. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 22:03:21,063][104569] Avg episode reward: [(0, '5976.950'), (1, '8382.684')] [2023-12-26 22:03:21,069][105620] Updated weights for policy 1, policy_version 918965 (0.0008) [2023-12-26 22:03:21,135][105620] Updated weights for policy 1, policy_version 918975 (0.0010) [2023-12-26 22:03:21,205][105620] Updated weights for policy 1, policy_version 918985 (0.0009) [2023-12-26 22:03:21,702][105692] Updated weights for policy 0, policy_version 918931 (0.0009) [2023-12-26 22:03:21,779][105692] Updated weights for policy 0, policy_version 918941 (0.0008) [2023-12-26 22:03:21,850][105692] Updated weights for policy 0, policy_version 918951 (0.0006) [2023-12-26 22:03:22,043][105620] Updated weights for policy 1, policy_version 918995 (0.0010) [2023-12-26 22:03:22,105][105620] Updated weights for policy 1, policy_version 919005 (0.0007) [2023-12-26 22:03:22,171][105620] Updated weights for policy 1, policy_version 919015 (0.0007) [2023-12-26 22:03:22,541][105692] Updated weights for policy 0, policy_version 918961 (0.0006) [2023-12-26 22:03:22,599][105692] Updated weights for policy 0, policy_version 918971 (0.0009) [2023-12-26 22:03:22,659][105692] Updated weights for policy 0, policy_version 918981 (0.0009) [2023-12-26 22:03:22,725][105692] Updated weights for policy 0, policy_version 918991 (0.0009) [2023-12-26 22:03:22,891][105620] Updated weights for policy 1, policy_version 919025 (0.0008) [2023-12-26 22:03:22,952][105620] Updated weights for policy 1, policy_version 919035 (0.0009) [2023-12-26 22:03:22,999][105620] Updated weights for policy 1, policy_version 919045 (0.0009) [2023-12-26 22:03:23,047][105620] Updated weights for policy 1, policy_version 919055 (0.0009) [2023-12-26 22:03:23,496][105692] Updated weights for policy 0, policy_version 919001 (0.0008) [2023-12-26 22:03:23,552][105692] Updated weights for policy 0, policy_version 919011 (0.0008) [2023-12-26 22:03:23,616][105692] Updated weights for policy 0, policy_version 919021 (0.0008) [2023-12-26 22:03:23,821][105620] Updated weights for policy 1, policy_version 919065 (0.0006) [2023-12-26 22:03:23,854][105586] KL-divergence is very high: 209.6312 [2023-12-26 22:03:23,867][105620] Updated weights for policy 1, policy_version 919075 (0.0005) [2023-12-26 22:03:23,899][105586] KL-divergence is very high: 404.1241 [2023-12-26 22:03:23,925][105620] Updated weights for policy 1, policy_version 919085 (0.0007) [2023-12-26 22:03:24,292][105692] Updated weights for policy 0, policy_version 919031 (0.0007) [2023-12-26 22:03:24,344][105692] Updated weights for policy 0, policy_version 919041 (0.0005) [2023-12-26 22:03:24,391][105692] Updated weights for policy 0, policy_version 919051 (0.0005) [2023-12-26 22:03:24,578][105620] Updated weights for policy 1, policy_version 919095 (0.0010) [2023-12-26 22:03:24,623][105620] Updated weights for policy 1, policy_version 919105 (0.0010) [2023-12-26 22:03:24,671][105620] Updated weights for policy 1, policy_version 919115 (0.0010) [2023-12-26 22:03:25,008][105692] Updated weights for policy 0, policy_version 919061 (0.0008) [2023-12-26 22:03:25,065][105692] Updated weights for policy 0, policy_version 919071 (0.0007) [2023-12-26 22:03:25,111][105692] Updated weights for policy 0, policy_version 919081 (0.0005) [2023-12-26 22:03:25,415][105620] Updated weights for policy 1, policy_version 919125 (0.0010) [2023-12-26 22:03:25,466][105620] Updated weights for policy 1, policy_version 919135 (0.0007) [2023-12-26 22:03:25,529][105620] Updated weights for policy 1, policy_version 919145 (0.0006) [2023-12-26 22:03:25,771][105692] Updated weights for policy 0, policy_version 919091 (0.0007) [2023-12-26 22:03:25,820][105692] Updated weights for policy 0, policy_version 919101 (0.0011) [2023-12-26 22:03:25,881][105692] Updated weights for policy 0, policy_version 919111 (0.0010) [2023-12-26 22:03:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 18841.6, 300 sec: 19022.1). Total num frames: 470663168. Throughput: 0: 9389.4, 1: 9375.9. Samples: 470670384. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 22:03:26,062][104569] Avg episode reward: [(0, '5319.710'), (1, '8017.431')] [2023-12-26 22:03:26,132][105620] Updated weights for policy 1, policy_version 919155 (0.0007) [2023-12-26 22:03:26,184][105620] Updated weights for policy 1, policy_version 919165 (0.0007) [2023-12-26 22:03:26,241][105620] Updated weights for policy 1, policy_version 919175 (0.0009) [2023-12-26 22:03:26,661][105692] Updated weights for policy 0, policy_version 919121 (0.0010) [2023-12-26 22:03:26,720][105692] Updated weights for policy 0, policy_version 919132 (0.0011) [2023-12-26 22:03:26,775][105692] Updated weights for policy 0, policy_version 919142 (0.0010) [2023-12-26 22:03:26,839][105692] Updated weights for policy 0, policy_version 919152 (0.0007) [2023-12-26 22:03:26,909][105620] Updated weights for policy 1, policy_version 919185 (0.0008) [2023-12-26 22:03:26,964][105620] Updated weights for policy 1, policy_version 919195 (0.0009) [2023-12-26 22:03:27,030][105620] Updated weights for policy 1, policy_version 919205 (0.0008) [2023-12-26 22:03:27,096][105620] Updated weights for policy 1, policy_version 919215 (0.0008) [2023-12-26 22:03:27,537][105692] Updated weights for policy 0, policy_version 919162 (0.0009) [2023-12-26 22:03:27,596][105692] Updated weights for policy 0, policy_version 919172 (0.0010) [2023-12-26 22:03:27,654][105692] Updated weights for policy 0, policy_version 919182 (0.0010) [2023-12-26 22:03:27,730][105620] Updated weights for policy 1, policy_version 919225 (0.0010) [2023-12-26 22:03:27,789][105620] Updated weights for policy 1, policy_version 919235 (0.0010) [2023-12-26 22:03:27,845][105620] Updated weights for policy 1, policy_version 919245 (0.0010) [2023-12-26 22:03:28,381][105692] Updated weights for policy 0, policy_version 919192 (0.0010) [2023-12-26 22:03:28,437][105692] Updated weights for policy 0, policy_version 919202 (0.0011) [2023-12-26 22:03:28,497][105692] Updated weights for policy 0, policy_version 919212 (0.0011) [2023-12-26 22:03:28,511][105620] Updated weights for policy 1, policy_version 919255 (0.0007) [2023-12-26 22:03:28,574][105620] Updated weights for policy 1, policy_version 919265 (0.0008) [2023-12-26 22:03:28,641][105620] Updated weights for policy 1, policy_version 919275 (0.0008) [2023-12-26 22:03:29,297][105692] Updated weights for policy 0, policy_version 919222 (0.0009) [2023-12-26 22:03:29,366][105692] Updated weights for policy 0, policy_version 919232 (0.0008) [2023-12-26 22:03:29,388][105620] Updated weights for policy 1, policy_version 919285 (0.0008) [2023-12-26 22:03:29,428][105692] Updated weights for policy 0, policy_version 919242 (0.0006) [2023-12-26 22:03:29,451][105620] Updated weights for policy 1, policy_version 919295 (0.0008) [2023-12-26 22:03:29,517][105620] Updated weights for policy 1, policy_version 919305 (0.0008) [2023-12-26 22:03:30,198][105620] Updated weights for policy 1, policy_version 919315 (0.0009) [2023-12-26 22:03:30,235][105692] Updated weights for policy 0, policy_version 919252 (0.0007) [2023-12-26 22:03:30,254][105620] Updated weights for policy 1, policy_version 919325 (0.0008) [2023-12-26 22:03:30,297][105692] Updated weights for policy 0, policy_version 919262 (0.0007) [2023-12-26 22:03:30,308][105620] Updated weights for policy 1, policy_version 919335 (0.0007) [2023-12-26 22:03:30,362][105692] Updated weights for policy 0, policy_version 919272 (0.0007) [2023-12-26 22:03:31,006][105620] Updated weights for policy 1, policy_version 919345 (0.0008) [2023-12-26 22:03:31,062][104569] Fps is (10 sec: 18841.8, 60 sec: 18705.1, 300 sec: 18994.3). Total num frames: 470753280. Throughput: 0: 9423.3, 1: 9465.6. Samples: 470730032. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 22:03:31,062][104569] Avg episode reward: [(0, '7175.564'), (1, '8279.498')] [2023-12-26 22:03:31,076][105620] Updated weights for policy 1, policy_version 919355 (0.0009) [2023-12-26 22:03:31,083][105692] Updated weights for policy 0, policy_version 919282 (0.0009) [2023-12-26 22:03:31,136][105620] Updated weights for policy 1, policy_version 919365 (0.0009) [2023-12-26 22:03:31,144][105692] Updated weights for policy 0, policy_version 919292 (0.0007) [2023-12-26 22:03:31,202][105620] Updated weights for policy 1, policy_version 919375 (0.0009) [2023-12-26 22:03:31,205][105692] Updated weights for policy 0, policy_version 919302 (0.0008) [2023-12-26 22:03:31,207][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000919376_235388928.pth... [2023-12-26 22:03:31,210][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000918256_235102208.pth [2023-12-26 22:03:31,269][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000919312_235380736.pth... [2023-12-26 22:03:31,270][105692] Updated weights for policy 0, policy_version 919312 (0.0006) [2023-12-26 22:03:31,275][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000918192_235094016.pth [2023-12-26 22:03:31,862][105620] Updated weights for policy 1, policy_version 919385 (0.0007) [2023-12-26 22:03:31,916][105620] Updated weights for policy 1, policy_version 919395 (0.0007) [2023-12-26 22:03:31,977][105620] Updated weights for policy 1, policy_version 919405 (0.0009) [2023-12-26 22:03:32,018][105692] Updated weights for policy 0, policy_version 919322 (0.0008) [2023-12-26 22:03:32,069][105692] Updated weights for policy 0, policy_version 919332 (0.0009) [2023-12-26 22:03:32,123][105692] Updated weights for policy 0, policy_version 919342 (0.0009) [2023-12-26 22:03:32,743][105692] Updated weights for policy 0, policy_version 919352 (0.0007) [2023-12-26 22:03:32,757][105620] Updated weights for policy 1, policy_version 919415 (0.0008) [2023-12-26 22:03:32,801][105692] Updated weights for policy 0, policy_version 919362 (0.0008) [2023-12-26 22:03:32,808][105620] Updated weights for policy 1, policy_version 919425 (0.0008) [2023-12-26 22:03:32,858][105692] Updated weights for policy 0, policy_version 919372 (0.0007) [2023-12-26 22:03:32,864][105620] Updated weights for policy 1, policy_version 919435 (0.0006) [2023-12-26 22:03:33,597][105692] Updated weights for policy 0, policy_version 919382 (0.0008) [2023-12-26 22:03:33,627][105620] Updated weights for policy 1, policy_version 919445 (0.0008) [2023-12-26 22:03:33,646][105692] Updated weights for policy 0, policy_version 919392 (0.0006) [2023-12-26 22:03:33,684][105620] Updated weights for policy 1, policy_version 919455 (0.0007) [2023-12-26 22:03:33,697][105692] Updated weights for policy 0, policy_version 919402 (0.0007) [2023-12-26 22:03:33,733][105620] Updated weights for policy 1, policy_version 919465 (0.0006) [2023-12-26 22:03:34,416][105692] Updated weights for policy 0, policy_version 919412 (0.0005) [2023-12-26 22:03:34,468][105692] Updated weights for policy 0, policy_version 919422 (0.0006) [2023-12-26 22:03:34,524][105620] Updated weights for policy 1, policy_version 919475 (0.0009) [2023-12-26 22:03:34,524][105692] Updated weights for policy 0, policy_version 919432 (0.0005) [2023-12-26 22:03:34,586][105620] Updated weights for policy 1, policy_version 919485 (0.0009) [2023-12-26 22:03:34,640][105620] Updated weights for policy 1, policy_version 919495 (0.0009) [2023-12-26 22:03:35,135][105692] Updated weights for policy 0, policy_version 919442 (0.0006) [2023-12-26 22:03:35,184][105692] Updated weights for policy 0, policy_version 919452 (0.0008) [2023-12-26 22:03:35,232][105692] Updated weights for policy 0, policy_version 919462 (0.0008) [2023-12-26 22:03:35,286][105692] Updated weights for policy 0, policy_version 919472 (0.0009) [2023-12-26 22:03:35,409][105620] Updated weights for policy 1, policy_version 919505 (0.0009) [2023-12-26 22:03:35,466][105620] Updated weights for policy 1, policy_version 919515 (0.0006) [2023-12-26 22:03:35,521][105620] Updated weights for policy 1, policy_version 919525 (0.0005) [2023-12-26 22:03:35,585][105620] Updated weights for policy 1, policy_version 919535 (0.0010) [2023-12-26 22:03:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18705.1, 300 sec: 18994.3). Total num frames: 470851584. Throughput: 0: 9394.7, 1: 9496.1. Samples: 470843888. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 22:03:36,063][104569] Avg episode reward: [(0, '7587.955'), (1, '8551.522')] [2023-12-26 22:03:36,167][105620] Updated weights for policy 1, policy_version 919545 (0.0010) [2023-12-26 22:03:36,171][105692] Updated weights for policy 0, policy_version 919482 (0.0007) [2023-12-26 22:03:36,223][105620] Updated weights for policy 1, policy_version 919555 (0.0009) [2023-12-26 22:03:36,237][105692] Updated weights for policy 0, policy_version 919492 (0.0006) [2023-12-26 22:03:36,283][105620] Updated weights for policy 1, policy_version 919565 (0.0007) [2023-12-26 22:03:36,302][105692] Updated weights for policy 0, policy_version 919502 (0.0007) [2023-12-26 22:03:37,018][105692] Updated weights for policy 0, policy_version 919512 (0.0008) [2023-12-26 22:03:37,058][105620] Updated weights for policy 1, policy_version 919575 (0.0011) [2023-12-26 22:03:37,072][105692] Updated weights for policy 0, policy_version 919522 (0.0006) [2023-12-26 22:03:37,117][105620] Updated weights for policy 1, policy_version 919585 (0.0011) [2023-12-26 22:03:37,128][105692] Updated weights for policy 0, policy_version 919532 (0.0007) [2023-12-26 22:03:37,187][105620] Updated weights for policy 1, policy_version 919595 (0.0010) [2023-12-26 22:03:37,910][105692] Updated weights for policy 0, policy_version 919542 (0.0007) [2023-12-26 22:03:37,936][105620] Updated weights for policy 1, policy_version 919605 (0.0010) [2023-12-26 22:03:37,975][105692] Updated weights for policy 0, policy_version 919552 (0.0006) [2023-12-26 22:03:37,989][105620] Updated weights for policy 1, policy_version 919615 (0.0011) [2023-12-26 22:03:38,037][105692] Updated weights for policy 0, policy_version 919562 (0.0006) [2023-12-26 22:03:38,042][105620] Updated weights for policy 1, policy_version 919625 (0.0010) [2023-12-26 22:03:38,829][105692] Updated weights for policy 0, policy_version 919572 (0.0006) [2023-12-26 22:03:38,830][105620] Updated weights for policy 1, policy_version 919635 (0.0010) [2023-12-26 22:03:38,887][105692] Updated weights for policy 0, policy_version 919582 (0.0006) [2023-12-26 22:03:38,889][105620] Updated weights for policy 1, policy_version 919645 (0.0010) [2023-12-26 22:03:38,943][105692] Updated weights for policy 0, policy_version 919592 (0.0006) [2023-12-26 22:03:38,954][105620] Updated weights for policy 1, policy_version 919655 (0.0010) [2023-12-26 22:03:39,715][105620] Updated weights for policy 1, policy_version 919665 (0.0010) [2023-12-26 22:03:39,778][105692] Updated weights for policy 0, policy_version 919602 (0.0006) [2023-12-26 22:03:39,784][105620] Updated weights for policy 1, policy_version 919675 (0.0010) [2023-12-26 22:03:39,839][105692] Updated weights for policy 0, policy_version 919612 (0.0006) [2023-12-26 22:03:39,855][105620] Updated weights for policy 1, policy_version 919685 (0.0010) [2023-12-26 22:03:39,906][105692] Updated weights for policy 0, policy_version 919622 (0.0007) [2023-12-26 22:03:39,926][105620] Updated weights for policy 1, policy_version 919695 (0.0011) [2023-12-26 22:03:39,973][105692] Updated weights for policy 0, policy_version 919632 (0.0007) [2023-12-26 22:03:40,675][105620] Updated weights for policy 1, policy_version 919705 (0.0008) [2023-12-26 22:03:40,730][105692] Updated weights for policy 0, policy_version 919642 (0.0010) [2023-12-26 22:03:40,734][105620] Updated weights for policy 1, policy_version 919715 (0.0005) [2023-12-26 22:03:40,786][105692] Updated weights for policy 0, policy_version 919652 (0.0010) [2023-12-26 22:03:40,790][105620] Updated weights for policy 1, policy_version 919725 (0.0008) [2023-12-26 22:03:40,841][105692] Updated weights for policy 0, policy_version 919662 (0.0010) [2023-12-26 22:03:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 18841.6, 300 sec: 18994.4). Total num frames: 470949888. Throughput: 0: 9353.8, 1: 9520.4. Samples: 470955536. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 22:03:41,062][104569] Avg episode reward: [(0, '8904.190'), (1, '8453.816')] [2023-12-26 22:03:41,539][105620] Updated weights for policy 1, policy_version 919735 (0.0010) [2023-12-26 22:03:41,576][105692] Updated weights for policy 0, policy_version 919672 (0.0008) [2023-12-26 22:03:41,608][105620] Updated weights for policy 1, policy_version 919745 (0.0008) [2023-12-26 22:03:41,642][105692] Updated weights for policy 0, policy_version 919682 (0.0009) [2023-12-26 22:03:41,677][105620] Updated weights for policy 1, policy_version 919755 (0.0007) [2023-12-26 22:03:41,708][105692] Updated weights for policy 0, policy_version 919693 (0.0010) [2023-12-26 22:03:42,401][105620] Updated weights for policy 1, policy_version 919765 (0.0009) [2023-12-26 22:03:42,454][105620] Updated weights for policy 1, policy_version 919775 (0.0011) [2023-12-26 22:03:42,473][105692] Updated weights for policy 0, policy_version 919703 (0.0011) [2023-12-26 22:03:42,506][105620] Updated weights for policy 1, policy_version 919785 (0.0010) [2023-12-26 22:03:42,532][105692] Updated weights for policy 0, policy_version 919713 (0.0010) [2023-12-26 22:03:42,588][105692] Updated weights for policy 0, policy_version 919723 (0.0010) [2023-12-26 22:03:43,210][105620] Updated weights for policy 1, policy_version 919795 (0.0010) [2023-12-26 22:03:43,257][105620] Updated weights for policy 1, policy_version 919805 (0.0005) [2023-12-26 22:03:43,310][105620] Updated weights for policy 1, policy_version 919815 (0.0005) [2023-12-26 22:03:43,335][105692] Updated weights for policy 0, policy_version 919733 (0.0011) [2023-12-26 22:03:43,381][105692] Updated weights for policy 0, policy_version 919743 (0.0010) [2023-12-26 22:03:43,426][105692] Updated weights for policy 0, policy_version 919753 (0.0005) [2023-12-26 22:03:43,897][105620] Updated weights for policy 1, policy_version 919825 (0.0006) [2023-12-26 22:03:43,953][105620] Updated weights for policy 1, policy_version 919835 (0.0010) [2023-12-26 22:03:44,011][105620] Updated weights for policy 1, policy_version 919845 (0.0010) [2023-12-26 22:03:44,077][105620] Updated weights for policy 1, policy_version 919855 (0.0010) [2023-12-26 22:03:44,122][105692] Updated weights for policy 0, policy_version 919763 (0.0006) [2023-12-26 22:03:44,182][105692] Updated weights for policy 0, policy_version 919773 (0.0008) [2023-12-26 22:03:44,242][105692] Updated weights for policy 0, policy_version 919783 (0.0009) [2023-12-26 22:03:44,743][105620] Updated weights for policy 1, policy_version 919865 (0.0006) [2023-12-26 22:03:44,808][105620] Updated weights for policy 1, policy_version 919875 (0.0009) [2023-12-26 22:03:44,875][105620] Updated weights for policy 1, policy_version 919885 (0.0008) [2023-12-26 22:03:45,068][105692] Updated weights for policy 0, policy_version 919793 (0.0010) [2023-12-26 22:03:45,128][105692] Updated weights for policy 0, policy_version 919803 (0.0009) [2023-12-26 22:03:45,181][105692] Updated weights for policy 0, policy_version 919813 (0.0009) [2023-12-26 22:03:45,241][105692] Updated weights for policy 0, policy_version 919823 (0.0009) [2023-12-26 22:03:45,591][105620] Updated weights for policy 1, policy_version 919895 (0.0010) [2023-12-26 22:03:45,641][105620] Updated weights for policy 1, policy_version 919905 (0.0009) [2023-12-26 22:03:45,686][105620] Updated weights for policy 1, policy_version 919915 (0.0009) [2023-12-26 22:03:45,954][105692] Updated weights for policy 0, policy_version 919833 (0.0009) [2023-12-26 22:03:46,009][105692] Updated weights for policy 0, policy_version 919843 (0.0011) [2023-12-26 22:03:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 18841.6, 300 sec: 18966.6). Total num frames: 471040000. Throughput: 0: 9320.1, 1: 9551.7. Samples: 471013312. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 22:03:46,063][104569] Avg episode reward: [(0, '9085.204'), (1, '8363.254')] [2023-12-26 22:03:46,067][105692] Updated weights for policy 0, policy_version 919853 (0.0010) [2023-12-26 22:03:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000919920_235528192.pth... [2023-12-26 22:03:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000918800_235241472.pth [2023-12-26 22:03:46,086][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000919856_235520000.pth... [2023-12-26 22:03:46,091][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000918736_235233280.pth [2023-12-26 22:03:46,444][105620] Updated weights for policy 1, policy_version 919925 (0.0008) [2023-12-26 22:03:46,505][105620] Updated weights for policy 1, policy_version 919935 (0.0008) [2023-12-26 22:03:46,566][105620] Updated weights for policy 1, policy_version 919945 (0.0009) [2023-12-26 22:03:46,772][105692] Updated weights for policy 0, policy_version 919863 (0.0009) [2023-12-26 22:03:46,835][105692] Updated weights for policy 0, policy_version 919873 (0.0007) [2023-12-26 22:03:46,894][105692] Updated weights for policy 0, policy_version 919883 (0.0005) [2023-12-26 22:03:47,383][105620] Updated weights for policy 1, policy_version 919955 (0.0008) [2023-12-26 22:03:47,433][105620] Updated weights for policy 1, policy_version 919965 (0.0008) [2023-12-26 22:03:47,474][105692] Updated weights for policy 0, policy_version 919893 (0.0005) [2023-12-26 22:03:47,481][105620] Updated weights for policy 1, policy_version 919975 (0.0007) [2023-12-26 22:03:47,521][105692] Updated weights for policy 0, policy_version 919903 (0.0008) [2023-12-26 22:03:47,578][105692] Updated weights for policy 0, policy_version 919913 (0.0005) [2023-12-26 22:03:48,175][105620] Updated weights for policy 1, policy_version 919985 (0.0006) [2023-12-26 22:03:48,233][105620] Updated weights for policy 1, policy_version 919995 (0.0009) [2023-12-26 22:03:48,275][105692] Updated weights for policy 0, policy_version 919923 (0.0005) [2023-12-26 22:03:48,281][105620] Updated weights for policy 1, policy_version 920005 (0.0007) [2023-12-26 22:03:48,321][105692] Updated weights for policy 0, policy_version 919933 (0.0005) [2023-12-26 22:03:48,342][105620] Updated weights for policy 1, policy_version 920015 (0.0008) [2023-12-26 22:03:48,382][105692] Updated weights for policy 0, policy_version 919943 (0.0008) [2023-12-26 22:03:49,117][105692] Updated weights for policy 0, policy_version 919953 (0.0008) [2023-12-26 22:03:49,135][105620] Updated weights for policy 1, policy_version 920025 (0.0008) [2023-12-26 22:03:49,171][105692] Updated weights for policy 0, policy_version 919963 (0.0007) [2023-12-26 22:03:49,192][105620] Updated weights for policy 1, policy_version 920035 (0.0007) [2023-12-26 22:03:49,227][105692] Updated weights for policy 0, policy_version 919973 (0.0007) [2023-12-26 22:03:49,257][105620] Updated weights for policy 1, policy_version 920045 (0.0007) [2023-12-26 22:03:49,292][105692] Updated weights for policy 0, policy_version 919983 (0.0008) [2023-12-26 22:03:50,060][105620] Updated weights for policy 1, policy_version 920055 (0.0009) [2023-12-26 22:03:50,067][105692] Updated weights for policy 0, policy_version 919993 (0.0008) [2023-12-26 22:03:50,121][105692] Updated weights for policy 0, policy_version 920003 (0.0007) [2023-12-26 22:03:50,124][105620] Updated weights for policy 1, policy_version 920065 (0.0008) [2023-12-26 22:03:50,172][105692] Updated weights for policy 0, policy_version 920013 (0.0007) [2023-12-26 22:03:50,190][105620] Updated weights for policy 1, policy_version 920075 (0.0008) [2023-12-26 22:03:50,892][105692] Updated weights for policy 0, policy_version 920023 (0.0006) [2023-12-26 22:03:50,954][105692] Updated weights for policy 0, policy_version 920033 (0.0007) [2023-12-26 22:03:50,995][105620] Updated weights for policy 1, policy_version 920085 (0.0009) [2023-12-26 22:03:51,022][105692] Updated weights for policy 0, policy_version 920043 (0.0007) [2023-12-26 22:03:51,056][105620] Updated weights for policy 1, policy_version 920095 (0.0007) [2023-12-26 22:03:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18841.6, 300 sec: 18966.6). Total num frames: 471138304. Throughput: 0: 9442.7, 1: 9489.2. Samples: 471128660. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 22:03:51,062][104569] Avg episode reward: [(0, '9261.704'), (1, '8820.074')] [2023-12-26 22:03:51,117][105620] Updated weights for policy 1, policy_version 920105 (0.0010) [2023-12-26 22:03:51,746][105692] Updated weights for policy 0, policy_version 920053 (0.0008) [2023-12-26 22:03:51,805][105692] Updated weights for policy 0, policy_version 920063 (0.0006) [2023-12-26 22:03:51,868][105692] Updated weights for policy 0, policy_version 920073 (0.0006) [2023-12-26 22:03:51,878][105620] Updated weights for policy 1, policy_version 920115 (0.0007) [2023-12-26 22:03:51,946][105620] Updated weights for policy 1, policy_version 920125 (0.0007) [2023-12-26 22:03:52,019][105620] Updated weights for policy 1, policy_version 920135 (0.0006) [2023-12-26 22:03:52,562][105692] Updated weights for policy 0, policy_version 920083 (0.0007) [2023-12-26 22:03:52,618][105692] Updated weights for policy 0, policy_version 920093 (0.0009) [2023-12-26 22:03:52,659][105620] Updated weights for policy 1, policy_version 920145 (0.0008) [2023-12-26 22:03:52,670][105692] Updated weights for policy 0, policy_version 920103 (0.0010) [2023-12-26 22:03:52,716][105620] Updated weights for policy 1, policy_version 920155 (0.0007) [2023-12-26 22:03:52,767][105620] Updated weights for policy 1, policy_version 920165 (0.0008) [2023-12-26 22:03:52,840][105620] Updated weights for policy 1, policy_version 920175 (0.0010) [2023-12-26 22:03:53,413][105692] Updated weights for policy 0, policy_version 920113 (0.0007) [2023-12-26 22:03:53,462][105692] Updated weights for policy 0, policy_version 920123 (0.0007) [2023-12-26 22:03:53,520][105692] Updated weights for policy 0, policy_version 920133 (0.0006) [2023-12-26 22:03:53,574][105620] Updated weights for policy 1, policy_version 920185 (0.0010) [2023-12-26 22:03:53,574][105692] Updated weights for policy 0, policy_version 920143 (0.0005) [2023-12-26 22:03:53,646][105620] Updated weights for policy 1, policy_version 920195 (0.0010) [2023-12-26 22:03:53,711][105620] Updated weights for policy 1, policy_version 920205 (0.0010) [2023-12-26 22:03:54,129][105692] Updated weights for policy 0, policy_version 920153 (0.0006) [2023-12-26 22:03:54,195][105692] Updated weights for policy 0, policy_version 920163 (0.0006) [2023-12-26 22:03:54,255][105692] Updated weights for policy 0, policy_version 920173 (0.0006) [2023-12-26 22:03:54,437][105620] Updated weights for policy 1, policy_version 920215 (0.0010) [2023-12-26 22:03:54,492][105620] Updated weights for policy 1, policy_version 920225 (0.0010) [2023-12-26 22:03:54,554][105620] Updated weights for policy 1, policy_version 920235 (0.0010) [2023-12-26 22:03:54,893][105692] Updated weights for policy 0, policy_version 920183 (0.0010) [2023-12-26 22:03:54,951][105692] Updated weights for policy 0, policy_version 920193 (0.0010) [2023-12-26 22:03:55,017][105692] Updated weights for policy 0, policy_version 920203 (0.0007) [2023-12-26 22:03:55,277][105620] Updated weights for policy 1, policy_version 920245 (0.0010) [2023-12-26 22:03:55,336][105620] Updated weights for policy 1, policy_version 920255 (0.0010) [2023-12-26 22:03:55,401][105620] Updated weights for policy 1, policy_version 920265 (0.0010) [2023-12-26 22:03:55,742][105692] Updated weights for policy 0, policy_version 920213 (0.0010) [2023-12-26 22:03:55,803][105692] Updated weights for policy 0, policy_version 920223 (0.0010) [2023-12-26 22:03:55,865][105692] Updated weights for policy 0, policy_version 920233 (0.0011) [2023-12-26 22:03:56,031][105620] Updated weights for policy 1, policy_version 920275 (0.0009) [2023-12-26 22:03:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 18978.1, 300 sec: 18966.6). Total num frames: 471236608. Throughput: 0: 9540.6, 1: 9515.3. Samples: 471246128. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 22:03:56,063][104569] Avg episode reward: [(0, '9353.312'), (1, '8020.236')] [2023-12-26 22:03:56,101][105620] Updated weights for policy 1, policy_version 920285 (0.0006) [2023-12-26 22:03:56,165][105620] Updated weights for policy 1, policy_version 920295 (0.0007) [2023-12-26 22:03:56,613][105692] Updated weights for policy 0, policy_version 920243 (0.0010) [2023-12-26 22:03:56,662][105692] Updated weights for policy 0, policy_version 920253 (0.0011) [2023-12-26 22:03:56,707][105692] Updated weights for policy 0, policy_version 920263 (0.0010) [2023-12-26 22:03:56,846][105620] Updated weights for policy 1, policy_version 920305 (0.0007) [2023-12-26 22:03:56,899][105620] Updated weights for policy 1, policy_version 920315 (0.0005) [2023-12-26 22:03:56,956][105620] Updated weights for policy 1, policy_version 920325 (0.0005) [2023-12-26 22:03:57,001][105620] Updated weights for policy 1, policy_version 920335 (0.0005) [2023-12-26 22:03:57,458][105692] Updated weights for policy 0, policy_version 920273 (0.0010) [2023-12-26 22:03:57,517][105692] Updated weights for policy 0, policy_version 920283 (0.0006) [2023-12-26 22:03:57,579][105692] Updated weights for policy 0, policy_version 920293 (0.0005) [2023-12-26 22:03:57,641][105692] Updated weights for policy 0, policy_version 920303 (0.0007) [2023-12-26 22:03:57,667][105620] Updated weights for policy 1, policy_version 920345 (0.0007) [2023-12-26 22:03:57,707][105586] KL-divergence is very high: 109.7652 [2023-12-26 22:03:57,726][105620] Updated weights for policy 1, policy_version 920355 (0.0006) [2023-12-26 22:03:57,760][105586] KL-divergence is very high: 114.9130 [2023-12-26 22:03:57,786][105620] Updated weights for policy 1, policy_version 920365 (0.0006) [2023-12-26 22:03:58,234][105692] Updated weights for policy 0, policy_version 920313 (0.0009) [2023-12-26 22:03:58,297][105692] Updated weights for policy 0, policy_version 920323 (0.0008) [2023-12-26 22:03:58,371][105692] Updated weights for policy 0, policy_version 920333 (0.0008) [2023-12-26 22:03:58,463][105620] Updated weights for policy 1, policy_version 920375 (0.0008) [2023-12-26 22:03:58,526][105620] Updated weights for policy 1, policy_version 920385 (0.0010) [2023-12-26 22:03:58,598][105620] Updated weights for policy 1, policy_version 920396 (0.0011) [2023-12-26 22:03:59,192][105692] Updated weights for policy 0, policy_version 920343 (0.0007) [2023-12-26 22:03:59,262][105692] Updated weights for policy 0, policy_version 920353 (0.0008) [2023-12-26 22:03:59,287][105620] Updated weights for policy 1, policy_version 920406 (0.0009) [2023-12-26 22:03:59,325][105692] Updated weights for policy 0, policy_version 920363 (0.0008) [2023-12-26 22:03:59,348][105620] Updated weights for policy 1, policy_version 920416 (0.0009) [2023-12-26 22:03:59,415][105620] Updated weights for policy 1, policy_version 920426 (0.0008) [2023-12-26 22:03:59,980][105692] Updated weights for policy 0, policy_version 920373 (0.0008) [2023-12-26 22:04:00,044][105692] Updated weights for policy 0, policy_version 920383 (0.0008) [2023-12-26 22:04:00,099][105692] Updated weights for policy 0, policy_version 920393 (0.0009) [2023-12-26 22:04:00,195][105620] Updated weights for policy 1, policy_version 920436 (0.0008) [2023-12-26 22:04:00,257][105620] Updated weights for policy 1, policy_version 920446 (0.0007) [2023-12-26 22:04:00,320][105620] Updated weights for policy 1, policy_version 920456 (0.0010) [2023-12-26 22:04:00,849][105692] Updated weights for policy 0, policy_version 920403 (0.0009) [2023-12-26 22:04:00,901][105692] Updated weights for policy 0, policy_version 920413 (0.0008) [2023-12-26 22:04:00,946][105692] Updated weights for policy 0, policy_version 920423 (0.0008) [2023-12-26 22:04:01,019][105620] Updated weights for policy 1, policy_version 920466 (0.0009) [2023-12-26 22:04:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 18978.1, 300 sec: 18966.6). Total num frames: 471334912. Throughput: 0: 9554.9, 1: 9571.2. Samples: 471304884. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 22:04:01,063][104569] Avg episode reward: [(0, '9265.836'), (1, '8119.472')] [2023-12-26 22:04:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000920432_235667456.pth... [2023-12-26 22:04:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000919312_235380736.pth [2023-12-26 22:04:01,078][105620] Updated weights for policy 1, policy_version 920476 (0.0011) [2023-12-26 22:04:01,142][105620] Updated weights for policy 1, policy_version 920486 (0.0011) [2023-12-26 22:04:01,206][105620] Updated weights for policy 1, policy_version 920496 (0.0008) [2023-12-26 22:04:01,206][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000920496_235675648.pth... [2023-12-26 22:04:01,210][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000919376_235388928.pth [2023-12-26 22:04:01,771][105692] Updated weights for policy 0, policy_version 920433 (0.0008) [2023-12-26 22:04:01,834][105692] Updated weights for policy 0, policy_version 920443 (0.0009) [2023-12-26 22:04:01,889][105620] Updated weights for policy 1, policy_version 920506 (0.0007) [2023-12-26 22:04:01,891][105692] Updated weights for policy 0, policy_version 920453 (0.0006) [2023-12-26 22:04:01,950][105692] Updated weights for policy 0, policy_version 920463 (0.0007) [2023-12-26 22:04:01,952][105620] Updated weights for policy 1, policy_version 920516 (0.0007) [2023-12-26 22:04:02,013][105620] Updated weights for policy 1, policy_version 920526 (0.0009) [2023-12-26 22:04:02,681][105692] Updated weights for policy 0, policy_version 920473 (0.0009) [2023-12-26 22:04:02,740][105692] Updated weights for policy 0, policy_version 920483 (0.0007) [2023-12-26 22:04:02,754][105620] Updated weights for policy 1, policy_version 920536 (0.0007) [2023-12-26 22:04:02,801][105692] Updated weights for policy 0, policy_version 920493 (0.0008) [2023-12-26 22:04:02,807][105620] Updated weights for policy 1, policy_version 920546 (0.0006) [2023-12-26 22:04:02,867][105620] Updated weights for policy 1, policy_version 920556 (0.0008) [2023-12-26 22:04:03,563][105620] Updated weights for policy 1, policy_version 920566 (0.0008) [2023-12-26 22:04:03,594][105692] Updated weights for policy 0, policy_version 920503 (0.0008) [2023-12-26 22:04:03,621][105620] Updated weights for policy 1, policy_version 920576 (0.0009) [2023-12-26 22:04:03,652][105692] Updated weights for policy 0, policy_version 920513 (0.0008) [2023-12-26 22:04:03,680][105620] Updated weights for policy 1, policy_version 920586 (0.0010) [2023-12-26 22:04:03,710][105692] Updated weights for policy 0, policy_version 920523 (0.0009) [2023-12-26 22:04:04,408][105692] Updated weights for policy 0, policy_version 920533 (0.0007) [2023-12-26 22:04:04,410][105620] Updated weights for policy 1, policy_version 920596 (0.0008) [2023-12-26 22:04:04,467][105692] Updated weights for policy 0, policy_version 920543 (0.0005) [2023-12-26 22:04:04,473][105620] Updated weights for policy 1, policy_version 920606 (0.0011) [2023-12-26 22:04:04,533][105692] Updated weights for policy 0, policy_version 920553 (0.0006) [2023-12-26 22:04:04,537][105620] Updated weights for policy 1, policy_version 920616 (0.0008) [2023-12-26 22:04:05,139][105620] Updated weights for policy 1, policy_version 920626 (0.0006) [2023-12-26 22:04:05,185][105620] Updated weights for policy 1, policy_version 920636 (0.0005) [2023-12-26 22:04:05,240][105692] Updated weights for policy 0, policy_version 920563 (0.0009) [2023-12-26 22:04:05,240][105620] Updated weights for policy 1, policy_version 920646 (0.0005) [2023-12-26 22:04:05,287][105620] Updated weights for policy 1, policy_version 920656 (0.0007) [2023-12-26 22:04:05,299][105692] Updated weights for policy 0, policy_version 920573 (0.0009) [2023-12-26 22:04:05,351][105692] Updated weights for policy 0, policy_version 920584 (0.0010) [2023-12-26 22:04:05,900][105620] Updated weights for policy 1, policy_version 920666 (0.0005) [2023-12-26 22:04:05,960][105620] Updated weights for policy 1, policy_version 920676 (0.0006) [2023-12-26 22:04:06,016][105620] Updated weights for policy 1, policy_version 920686 (0.0005) [2023-12-26 22:04:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19114.7, 300 sec: 18966.6). Total num frames: 471433216. Throughput: 0: 9557.7, 1: 9678.9. Samples: 471419904. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 22:04:06,063][104569] Avg episode reward: [(0, '9265.825'), (1, '8297.398')] [2023-12-26 22:04:06,259][105692] Updated weights for policy 0, policy_version 920594 (0.0009) [2023-12-26 22:04:06,318][105692] Updated weights for policy 0, policy_version 920604 (0.0010) [2023-12-26 22:04:06,381][105692] Updated weights for policy 0, policy_version 920614 (0.0009) [2023-12-26 22:04:06,447][105692] Updated weights for policy 0, policy_version 920624 (0.0009) [2023-12-26 22:04:06,633][105620] Updated weights for policy 1, policy_version 920696 (0.0006) [2023-12-26 22:04:06,702][105620] Updated weights for policy 1, policy_version 920706 (0.0007) [2023-12-26 22:04:06,770][105620] Updated weights for policy 1, policy_version 920716 (0.0009) [2023-12-26 22:04:07,213][105692] Updated weights for policy 0, policy_version 920634 (0.0010) [2023-12-26 22:04:07,269][105692] Updated weights for policy 0, policy_version 920644 (0.0009) [2023-12-26 22:04:07,331][105692] Updated weights for policy 0, policy_version 920654 (0.0009) [2023-12-26 22:04:07,415][105620] Updated weights for policy 1, policy_version 920726 (0.0009) [2023-12-26 22:04:07,477][105620] Updated weights for policy 1, policy_version 920736 (0.0009) [2023-12-26 22:04:07,534][105620] Updated weights for policy 1, policy_version 920746 (0.0009) [2023-12-26 22:04:08,105][105692] Updated weights for policy 0, policy_version 920664 (0.0009) [2023-12-26 22:04:08,162][105692] Updated weights for policy 0, policy_version 920674 (0.0009) [2023-12-26 22:04:08,180][105620] Updated weights for policy 1, policy_version 920756 (0.0007) [2023-12-26 22:04:08,215][105692] Updated weights for policy 0, policy_version 920684 (0.0008) [2023-12-26 22:04:08,250][105620] Updated weights for policy 1, policy_version 920766 (0.0006) [2023-12-26 22:04:08,313][105620] Updated weights for policy 1, policy_version 920776 (0.0007) [2023-12-26 22:04:08,890][105620] Updated weights for policy 1, policy_version 920786 (0.0009) [2023-12-26 22:04:08,958][105620] Updated weights for policy 1, policy_version 920796 (0.0008) [2023-12-26 22:04:09,022][105620] Updated weights for policy 1, policy_version 920806 (0.0011) [2023-12-26 22:04:09,089][105620] Updated weights for policy 1, policy_version 920816 (0.0007) [2023-12-26 22:04:09,089][105692] Updated weights for policy 0, policy_version 920694 (0.0008) [2023-12-26 22:04:09,149][105692] Updated weights for policy 0, policy_version 920704 (0.0008) [2023-12-26 22:04:09,216][105692] Updated weights for policy 0, policy_version 920714 (0.0008) [2023-12-26 22:04:09,879][105620] Updated weights for policy 1, policy_version 920826 (0.0008) [2023-12-26 22:04:09,936][105620] Updated weights for policy 1, policy_version 920836 (0.0010) [2023-12-26 22:04:09,987][105692] Updated weights for policy 0, policy_version 920724 (0.0008) [2023-12-26 22:04:10,001][105620] Updated weights for policy 1, policy_version 920846 (0.0008) [2023-12-26 22:04:10,049][105692] Updated weights for policy 0, policy_version 920734 (0.0009) [2023-12-26 22:04:10,100][105692] Updated weights for policy 0, policy_version 920744 (0.0009) [2023-12-26 22:04:10,759][105620] Updated weights for policy 1, policy_version 920856 (0.0006) [2023-12-26 22:04:10,826][105620] Updated weights for policy 1, policy_version 920866 (0.0006) [2023-12-26 22:04:10,882][105692] Updated weights for policy 0, policy_version 920754 (0.0009) [2023-12-26 22:04:10,890][105620] Updated weights for policy 1, policy_version 920876 (0.0006) [2023-12-26 22:04:10,951][105692] Updated weights for policy 0, policy_version 920764 (0.0009) [2023-12-26 22:04:11,011][105692] Updated weights for policy 0, policy_version 920774 (0.0008) [2023-12-26 22:04:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19114.7, 300 sec: 18966.6). Total num frames: 471523328. Throughput: 0: 9443.4, 1: 9749.2. Samples: 471534048. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 22:04:11,062][104569] Avg episode reward: [(0, '9170.164'), (1, '7769.694')] [2023-12-26 22:04:11,075][105692] Updated weights for policy 0, policy_version 920784 (0.0010) [2023-12-26 22:04:11,632][105620] Updated weights for policy 1, policy_version 920886 (0.0009) [2023-12-26 22:04:11,702][105620] Updated weights for policy 1, policy_version 920896 (0.0011) [2023-12-26 22:04:11,777][105620] Updated weights for policy 1, policy_version 920906 (0.0010) [2023-12-26 22:04:11,863][105692] Updated weights for policy 0, policy_version 920794 (0.0008) [2023-12-26 22:04:11,930][105692] Updated weights for policy 0, policy_version 920804 (0.0008) [2023-12-26 22:04:11,997][105692] Updated weights for policy 0, policy_version 920814 (0.0008) [2023-12-26 22:04:12,563][105620] Updated weights for policy 1, policy_version 920916 (0.0010) [2023-12-26 22:04:12,616][105620] Updated weights for policy 1, policy_version 920926 (0.0009) [2023-12-26 22:04:12,670][105620] Updated weights for policy 1, policy_version 920936 (0.0011) [2023-12-26 22:04:12,728][105692] Updated weights for policy 0, policy_version 920824 (0.0006) [2023-12-26 22:04:12,787][105692] Updated weights for policy 0, policy_version 920834 (0.0007) [2023-12-26 22:04:12,846][105692] Updated weights for policy 0, policy_version 920844 (0.0008) [2023-12-26 22:04:13,449][105620] Updated weights for policy 1, policy_version 920946 (0.0011) [2023-12-26 22:04:13,515][105620] Updated weights for policy 1, policy_version 920956 (0.0010) [2023-12-26 22:04:13,552][105692] Updated weights for policy 0, policy_version 920854 (0.0007) [2023-12-26 22:04:13,567][105620] Updated weights for policy 1, policy_version 920966 (0.0010) [2023-12-26 22:04:13,608][105692] Updated weights for policy 0, policy_version 920864 (0.0005) [2023-12-26 22:04:13,629][105620] Updated weights for policy 1, policy_version 920976 (0.0010) [2023-12-26 22:04:13,671][105692] Updated weights for policy 0, policy_version 920874 (0.0006) [2023-12-26 22:04:14,248][105692] Updated weights for policy 0, policy_version 920884 (0.0008) [2023-12-26 22:04:14,298][105692] Updated weights for policy 0, policy_version 920894 (0.0009) [2023-12-26 22:04:14,356][105692] Updated weights for policy 0, policy_version 920904 (0.0008) [2023-12-26 22:04:14,404][105620] Updated weights for policy 1, policy_version 920986 (0.0009) [2023-12-26 22:04:14,465][105586] KL-divergence is very high: 116.9482 [2023-12-26 22:04:14,466][105620] Updated weights for policy 1, policy_version 920996 (0.0010) [2023-12-26 22:04:14,491][105586] KL-divergence is very high: 122.6198 [2023-12-26 22:04:14,518][105586] KL-divergence is very high: 136.3134 [2023-12-26 22:04:14,531][105620] Updated weights for policy 1, policy_version 921006 (0.0008) [2023-12-26 22:04:15,142][105620] Updated weights for policy 1, policy_version 921016 (0.0010) [2023-12-26 22:04:15,193][105692] Updated weights for policy 0, policy_version 920914 (0.0010) [2023-12-26 22:04:15,203][105620] Updated weights for policy 1, policy_version 921026 (0.0010) [2023-12-26 22:04:15,251][105692] Updated weights for policy 0, policy_version 920924 (0.0007) [2023-12-26 22:04:15,261][105620] Updated weights for policy 1, policy_version 921036 (0.0011) [2023-12-26 22:04:15,315][105692] Updated weights for policy 0, policy_version 920934 (0.0008) [2023-12-26 22:04:15,380][105692] Updated weights for policy 0, policy_version 920944 (0.0009) [2023-12-26 22:04:16,008][105620] Updated weights for policy 1, policy_version 921046 (0.0010) [2023-12-26 22:04:16,020][105586] KL-divergence is very high: 258.1851 [2023-12-26 22:04:16,062][104569] Fps is (10 sec: 18022.6, 60 sec: 18978.2, 300 sec: 18938.8). Total num frames: 471613440. Throughput: 0: 9419.0, 1: 9665.4. Samples: 471588832. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-26 22:04:16,062][104569] Avg episode reward: [(0, '7563.298'), (1, '7521.330')] [2023-12-26 22:04:16,070][105692] Updated weights for policy 0, policy_version 920954 (0.0005) [2023-12-26 22:04:16,072][105620] Updated weights for policy 1, policy_version 921056 (0.0009) [2023-12-26 22:04:16,073][105586] KL-divergence is very high: 450.8438 [2023-12-26 22:04:16,126][105586] KL-divergence is very high: 533.9449 [2023-12-26 22:04:16,130][105692] Updated weights for policy 0, policy_version 920964 (0.0006) [2023-12-26 22:04:16,136][105620] Updated weights for policy 1, policy_version 921066 (0.0009) [2023-12-26 22:04:16,171][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000921072_235823104.pth... [2023-12-26 22:04:16,175][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000919920_235528192.pth [2023-12-26 22:04:16,189][105692] Updated weights for policy 0, policy_version 920974 (0.0006) [2023-12-26 22:04:16,199][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000920976_235806720.pth... [2023-12-26 22:04:16,202][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000919856_235520000.pth [2023-12-26 22:04:16,870][105692] Updated weights for policy 0, policy_version 920984 (0.0008) [2023-12-26 22:04:16,922][105620] Updated weights for policy 1, policy_version 921076 (0.0008) [2023-12-26 22:04:16,930][105692] Updated weights for policy 0, policy_version 920994 (0.0009) [2023-12-26 22:04:16,974][105620] Updated weights for policy 1, policy_version 921086 (0.0010) [2023-12-26 22:04:16,977][105692] Updated weights for policy 0, policy_version 921004 (0.0009) [2023-12-26 22:04:17,028][105620] Updated weights for policy 1, policy_version 921096 (0.0009) [2023-12-26 22:04:17,626][105692] Updated weights for policy 0, policy_version 921014 (0.0008) [2023-12-26 22:04:17,684][105692] Updated weights for policy 0, policy_version 921024 (0.0009) [2023-12-26 22:04:17,742][105692] Updated weights for policy 0, policy_version 921034 (0.0009) [2023-12-26 22:04:17,811][105620] Updated weights for policy 1, policy_version 921106 (0.0009) [2023-12-26 22:04:17,868][105620] Updated weights for policy 1, policy_version 921116 (0.0008) [2023-12-26 22:04:17,928][105620] Updated weights for policy 1, policy_version 921126 (0.0009) [2023-12-26 22:04:17,991][105620] Updated weights for policy 1, policy_version 921136 (0.0008) [2023-12-26 22:04:18,448][105692] Updated weights for policy 0, policy_version 921044 (0.0008) [2023-12-26 22:04:18,509][105692] Updated weights for policy 0, policy_version 921054 (0.0008) [2023-12-26 22:04:18,576][105692] Updated weights for policy 0, policy_version 921064 (0.0006) [2023-12-26 22:04:18,807][105620] Updated weights for policy 1, policy_version 921146 (0.0009) [2023-12-26 22:04:18,873][105620] Updated weights for policy 1, policy_version 921156 (0.0010) [2023-12-26 22:04:18,944][105620] Updated weights for policy 1, policy_version 921166 (0.0007) [2023-12-26 22:04:19,318][105692] Updated weights for policy 0, policy_version 921074 (0.0007) [2023-12-26 22:04:19,392][105692] Updated weights for policy 0, policy_version 921084 (0.0009) [2023-12-26 22:04:19,451][105692] Updated weights for policy 0, policy_version 921094 (0.0009) [2023-12-26 22:04:19,513][105692] Updated weights for policy 0, policy_version 921104 (0.0008) [2023-12-26 22:04:19,667][105620] Updated weights for policy 1, policy_version 921176 (0.0007) [2023-12-26 22:04:19,735][105620] Updated weights for policy 1, policy_version 921186 (0.0011) [2023-12-26 22:04:19,798][105620] Updated weights for policy 1, policy_version 921196 (0.0009) [2023-12-26 22:04:20,275][105692] Updated weights for policy 0, policy_version 921114 (0.0009) [2023-12-26 22:04:20,338][105692] Updated weights for policy 0, policy_version 921124 (0.0008) [2023-12-26 22:04:20,406][105692] Updated weights for policy 0, policy_version 921134 (0.0010) [2023-12-26 22:04:20,436][105620] Updated weights for policy 1, policy_version 921206 (0.0007) [2023-12-26 22:04:20,504][105620] Updated weights for policy 1, policy_version 921216 (0.0009) [2023-12-26 22:04:20,572][105620] Updated weights for policy 1, policy_version 921226 (0.0008) [2023-12-26 22:04:21,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19114.7, 300 sec: 18938.8). Total num frames: 471711744. Throughput: 0: 9470.9, 1: 9650.7. Samples: 471704364. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-26 22:04:21,063][104569] Avg episode reward: [(0, '7398.971'), (1, '8318.068')] [2023-12-26 22:04:21,163][105692] Updated weights for policy 0, policy_version 921144 (0.0009) [2023-12-26 22:04:21,220][105620] Updated weights for policy 1, policy_version 921236 (0.0010) [2023-12-26 22:04:21,225][105692] Updated weights for policy 0, policy_version 921154 (0.0009) [2023-12-26 22:04:21,283][105692] Updated weights for policy 0, policy_version 921164 (0.0009) [2023-12-26 22:04:21,286][105620] Updated weights for policy 1, policy_version 921246 (0.0011) [2023-12-26 22:04:21,350][105620] Updated weights for policy 1, policy_version 921256 (0.0011) [2023-12-26 22:04:22,072][105692] Updated weights for policy 0, policy_version 921174 (0.0009) [2023-12-26 22:04:22,132][105692] Updated weights for policy 0, policy_version 921184 (0.0011) [2023-12-26 22:04:22,151][105620] Updated weights for policy 1, policy_version 921266 (0.0010) [2023-12-26 22:04:22,178][105692] Updated weights for policy 0, policy_version 921194 (0.0011) [2023-12-26 22:04:22,210][105620] Updated weights for policy 1, policy_version 921276 (0.0011) [2023-12-26 22:04:22,273][105620] Updated weights for policy 1, policy_version 921286 (0.0011) [2023-12-26 22:04:22,340][105620] Updated weights for policy 1, policy_version 921296 (0.0011) [2023-12-26 22:04:22,947][105692] Updated weights for policy 0, policy_version 921204 (0.0008) [2023-12-26 22:04:23,007][105692] Updated weights for policy 0, policy_version 921214 (0.0005) [2023-12-26 22:04:23,071][105692] Updated weights for policy 0, policy_version 921224 (0.0009) [2023-12-26 22:04:23,099][105620] Updated weights for policy 1, policy_version 921306 (0.0011) [2023-12-26 22:04:23,153][105620] Updated weights for policy 1, policy_version 921316 (0.0009) [2023-12-26 22:04:23,200][105620] Updated weights for policy 1, policy_version 921326 (0.0005) [2023-12-26 22:04:23,738][105692] Updated weights for policy 0, policy_version 921234 (0.0010) [2023-12-26 22:04:23,753][105620] Updated weights for policy 1, policy_version 921336 (0.0008) [2023-12-26 22:04:23,793][105692] Updated weights for policy 0, policy_version 921244 (0.0006) [2023-12-26 22:04:23,800][105620] Updated weights for policy 1, policy_version 921347 (0.0009) [2023-12-26 22:04:23,842][105692] Updated weights for policy 0, policy_version 921254 (0.0011) [2023-12-26 22:04:23,852][105620] Updated weights for policy 1, policy_version 921357 (0.0008) [2023-12-26 22:04:23,894][105692] Updated weights for policy 0, policy_version 921264 (0.0007) [2023-12-26 22:04:24,599][105692] Updated weights for policy 0, policy_version 921274 (0.0010) [2023-12-26 22:04:24,603][105620] Updated weights for policy 1, policy_version 921367 (0.0009) [2023-12-26 22:04:24,653][105620] Updated weights for policy 1, policy_version 921377 (0.0010) [2023-12-26 22:04:24,653][105692] Updated weights for policy 0, policy_version 921284 (0.0010) [2023-12-26 22:04:24,702][105692] Updated weights for policy 0, policy_version 921294 (0.0010) [2023-12-26 22:04:24,706][105620] Updated weights for policy 1, policy_version 921387 (0.0006) [2023-12-26 22:04:25,358][105620] Updated weights for policy 1, policy_version 921397 (0.0008) [2023-12-26 22:04:25,419][105620] Updated weights for policy 1, policy_version 921407 (0.0010) [2023-12-26 22:04:25,464][105692] Updated weights for policy 0, policy_version 921304 (0.0009) [2023-12-26 22:04:25,468][105620] Updated weights for policy 1, policy_version 921417 (0.0010) [2023-12-26 22:04:25,513][105692] Updated weights for policy 0, policy_version 921314 (0.0005) [2023-12-26 22:04:25,557][105692] Updated weights for policy 0, policy_version 921324 (0.0005) [2023-12-26 22:04:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19114.7, 300 sec: 18938.8). Total num frames: 471810048. Throughput: 0: 9514.1, 1: 9703.4. Samples: 471820324. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-26 22:04:26,062][104569] Avg episode reward: [(0, '8150.375'), (1, '8760.106')] [2023-12-26 22:04:26,192][105620] Updated weights for policy 1, policy_version 921427 (0.0010) [2023-12-26 22:04:26,240][105620] Updated weights for policy 1, policy_version 921437 (0.0010) [2023-12-26 22:04:26,282][105692] Updated weights for policy 0, policy_version 921334 (0.0010) [2023-12-26 22:04:26,292][105620] Updated weights for policy 1, policy_version 921447 (0.0010) [2023-12-26 22:04:26,337][105692] Updated weights for policy 0, policy_version 921344 (0.0011) [2023-12-26 22:04:26,390][105692] Updated weights for policy 0, policy_version 921354 (0.0007) [2023-12-26 22:04:27,060][105620] Updated weights for policy 1, policy_version 921457 (0.0010) [2023-12-26 22:04:27,118][105620] Updated weights for policy 1, policy_version 921467 (0.0010) [2023-12-26 22:04:27,131][105692] Updated weights for policy 0, policy_version 921364 (0.0009) [2023-12-26 22:04:27,172][105620] Updated weights for policy 1, policy_version 921477 (0.0007) [2023-12-26 22:04:27,190][105692] Updated weights for policy 0, policy_version 921374 (0.0010) [2023-12-26 22:04:27,227][105620] Updated weights for policy 1, policy_version 921487 (0.0010) [2023-12-26 22:04:27,248][105692] Updated weights for policy 0, policy_version 921384 (0.0010) [2023-12-26 22:04:27,843][105620] Updated weights for policy 1, policy_version 921497 (0.0007) [2023-12-26 22:04:27,853][105692] Updated weights for policy 0, policy_version 921394 (0.0009) [2023-12-26 22:04:27,887][105620] Updated weights for policy 1, policy_version 921507 (0.0010) [2023-12-26 22:04:27,911][105692] Updated weights for policy 0, policy_version 921404 (0.0006) [2023-12-26 22:04:27,942][105620] Updated weights for policy 1, policy_version 921517 (0.0010) [2023-12-26 22:04:27,961][105692] Updated weights for policy 0, policy_version 921414 (0.0010) [2023-12-26 22:04:28,024][105692] Updated weights for policy 0, policy_version 921424 (0.0011) [2023-12-26 22:04:28,575][105692] Updated weights for policy 0, policy_version 921434 (0.0005) [2023-12-26 22:04:28,636][105692] Updated weights for policy 0, policy_version 921444 (0.0006) [2023-12-26 22:04:28,690][105692] Updated weights for policy 0, policy_version 921454 (0.0006) [2023-12-26 22:04:28,702][105620] Updated weights for policy 1, policy_version 921527 (0.0009) [2023-12-26 22:04:28,760][105620] Updated weights for policy 1, policy_version 921537 (0.0008) [2023-12-26 22:04:28,823][105620] Updated weights for policy 1, policy_version 921547 (0.0008) [2023-12-26 22:04:29,310][105692] Updated weights for policy 0, policy_version 921464 (0.0008) [2023-12-26 22:04:29,381][105692] Updated weights for policy 0, policy_version 921474 (0.0008) [2023-12-26 22:04:29,444][105692] Updated weights for policy 0, policy_version 921484 (0.0008) [2023-12-26 22:04:29,612][105620] Updated weights for policy 1, policy_version 921557 (0.0008) [2023-12-26 22:04:29,672][105620] Updated weights for policy 1, policy_version 921567 (0.0009) [2023-12-26 22:04:29,734][105620] Updated weights for policy 1, policy_version 921577 (0.0009) [2023-12-26 22:04:30,195][105692] Updated weights for policy 0, policy_version 921494 (0.0009) [2023-12-26 22:04:30,262][105692] Updated weights for policy 0, policy_version 921504 (0.0010) [2023-12-26 22:04:30,329][105692] Updated weights for policy 0, policy_version 921514 (0.0010) [2023-12-26 22:04:30,391][105620] Updated weights for policy 1, policy_version 921587 (0.0007) [2023-12-26 22:04:30,444][105620] Updated weights for policy 1, policy_version 921597 (0.0005) [2023-12-26 22:04:30,499][105620] Updated weights for policy 1, policy_version 921607 (0.0005) [2023-12-26 22:04:31,041][105692] Updated weights for policy 0, policy_version 921524 (0.0010) [2023-12-26 22:04:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 18911.0). Total num frames: 471908352. Throughput: 0: 9600.4, 1: 9687.7. Samples: 471881276. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-26 22:04:31,063][104569] Avg episode reward: [(0, '8992.513'), (1, '8580.563')] [2023-12-26 22:04:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000921616_235962368.pth... [2023-12-26 22:04:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000920496_235675648.pth [2023-12-26 22:04:31,107][105692] Updated weights for policy 0, policy_version 921534 (0.0009) [2023-12-26 22:04:31,169][105692] Updated weights for policy 0, policy_version 921544 (0.0006) [2023-12-26 22:04:31,216][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000921552_235954176.pth... [2023-12-26 22:04:31,220][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000920432_235667456.pth [2023-12-26 22:04:31,270][105620] Updated weights for policy 1, policy_version 921617 (0.0009) [2023-12-26 22:04:31,328][105620] Updated weights for policy 1, policy_version 921627 (0.0010) [2023-12-26 22:04:31,384][105620] Updated weights for policy 1, policy_version 921637 (0.0010) [2023-12-26 22:04:31,449][105620] Updated weights for policy 1, policy_version 921647 (0.0009) [2023-12-26 22:04:31,817][105692] Updated weights for policy 0, policy_version 921554 (0.0006) [2023-12-26 22:04:31,868][105692] Updated weights for policy 0, policy_version 921564 (0.0008) [2023-12-26 22:04:31,924][105692] Updated weights for policy 0, policy_version 921574 (0.0008) [2023-12-26 22:04:31,984][105692] Updated weights for policy 0, policy_version 921584 (0.0009) [2023-12-26 22:04:32,252][105620] Updated weights for policy 1, policy_version 921657 (0.0009) [2023-12-26 22:04:32,313][105620] Updated weights for policy 1, policy_version 921667 (0.0008) [2023-12-26 22:04:32,382][105620] Updated weights for policy 1, policy_version 921677 (0.0009) [2023-12-26 22:04:32,687][105692] Updated weights for policy 0, policy_version 921594 (0.0009) [2023-12-26 22:04:32,748][105692] Updated weights for policy 0, policy_version 921605 (0.0010) [2023-12-26 22:04:32,802][105692] Updated weights for policy 0, policy_version 921615 (0.0009) [2023-12-26 22:04:33,165][105620] Updated weights for policy 1, policy_version 921687 (0.0009) [2023-12-26 22:04:33,223][105620] Updated weights for policy 1, policy_version 921697 (0.0009) [2023-12-26 22:04:33,281][105620] Updated weights for policy 1, policy_version 921707 (0.0009) [2023-12-26 22:04:33,551][105692] Updated weights for policy 0, policy_version 921625 (0.0007) [2023-12-26 22:04:33,607][105692] Updated weights for policy 0, policy_version 921635 (0.0008) [2023-12-26 22:04:33,667][105692] Updated weights for policy 0, policy_version 921645 (0.0009) [2023-12-26 22:04:34,033][105620] Updated weights for policy 1, policy_version 921717 (0.0009) [2023-12-26 22:04:34,079][105620] Updated weights for policy 1, policy_version 921727 (0.0008) [2023-12-26 22:04:34,137][105620] Updated weights for policy 1, policy_version 921737 (0.0009) [2023-12-26 22:04:34,410][105692] Updated weights for policy 0, policy_version 921655 (0.0009) [2023-12-26 22:04:34,472][105692] Updated weights for policy 0, policy_version 921665 (0.0009) [2023-12-26 22:04:34,540][105692] Updated weights for policy 0, policy_version 921675 (0.0009) [2023-12-26 22:04:34,917][105620] Updated weights for policy 1, policy_version 921747 (0.0007) [2023-12-26 22:04:34,971][105620] Updated weights for policy 1, policy_version 921757 (0.0008) [2023-12-26 22:04:35,030][105620] Updated weights for policy 1, policy_version 921767 (0.0009) [2023-12-26 22:04:35,313][105692] Updated weights for policy 0, policy_version 921685 (0.0010) [2023-12-26 22:04:35,376][105692] Updated weights for policy 0, policy_version 921695 (0.0010) [2023-12-26 22:04:35,439][105692] Updated weights for policy 0, policy_version 921705 (0.0010) [2023-12-26 22:04:35,717][105620] Updated weights for policy 1, policy_version 921777 (0.0008) [2023-12-26 22:04:35,772][105620] Updated weights for policy 1, policy_version 921787 (0.0006) [2023-12-26 22:04:35,832][105620] Updated weights for policy 1, policy_version 921797 (0.0005) [2023-12-26 22:04:35,888][105620] Updated weights for policy 1, policy_version 921807 (0.0005) [2023-12-26 22:04:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 18938.8). Total num frames: 472006656. Throughput: 0: 9595.6, 1: 9670.3. Samples: 471995628. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-26 22:04:36,062][104569] Avg episode reward: [(0, '8718.351'), (1, '7303.525')] [2023-12-26 22:04:36,254][105692] Updated weights for policy 0, policy_version 921715 (0.0008) [2023-12-26 22:04:36,315][105692] Updated weights for policy 0, policy_version 921725 (0.0006) [2023-12-26 22:04:36,374][105692] Updated weights for policy 0, policy_version 921735 (0.0006) [2023-12-26 22:04:36,569][105620] Updated weights for policy 1, policy_version 921817 (0.0008) [2023-12-26 22:04:36,641][105620] Updated weights for policy 1, policy_version 921827 (0.0006) [2023-12-26 22:04:36,710][105620] Updated weights for policy 1, policy_version 921837 (0.0005) [2023-12-26 22:04:37,103][105692] Updated weights for policy 0, policy_version 921745 (0.0008) [2023-12-26 22:04:37,176][105692] Updated weights for policy 0, policy_version 921755 (0.0010) [2023-12-26 22:04:37,246][105692] Updated weights for policy 0, policy_version 921765 (0.0009) [2023-12-26 22:04:37,308][105692] Updated weights for policy 0, policy_version 921775 (0.0008) [2023-12-26 22:04:37,319][105620] Updated weights for policy 1, policy_version 921847 (0.0008) [2023-12-26 22:04:37,385][105620] Updated weights for policy 1, policy_version 921857 (0.0010) [2023-12-26 22:04:37,448][105620] Updated weights for policy 1, policy_version 921867 (0.0008) [2023-12-26 22:04:38,036][105692] Updated weights for policy 0, policy_version 921785 (0.0009) [2023-12-26 22:04:38,091][105692] Updated weights for policy 0, policy_version 921795 (0.0008) [2023-12-26 22:04:38,142][105692] Updated weights for policy 0, policy_version 921805 (0.0009) [2023-12-26 22:04:38,214][105620] Updated weights for policy 1, policy_version 921877 (0.0010) [2023-12-26 22:04:38,272][105620] Updated weights for policy 1, policy_version 921887 (0.0009) [2023-12-26 22:04:38,336][105620] Updated weights for policy 1, policy_version 921897 (0.0009) [2023-12-26 22:04:38,884][105692] Updated weights for policy 0, policy_version 921815 (0.0009) [2023-12-26 22:04:38,951][105692] Updated weights for policy 0, policy_version 921825 (0.0007) [2023-12-26 22:04:39,010][105692] Updated weights for policy 0, policy_version 921835 (0.0009) [2023-12-26 22:04:39,142][105620] Updated weights for policy 1, policy_version 921907 (0.0009) [2023-12-26 22:04:39,204][105620] Updated weights for policy 1, policy_version 921917 (0.0009) [2023-12-26 22:04:39,274][105620] Updated weights for policy 1, policy_version 921927 (0.0008) [2023-12-26 22:04:39,760][105692] Updated weights for policy 0, policy_version 921845 (0.0011) [2023-12-26 22:04:39,820][105692] Updated weights for policy 0, policy_version 921855 (0.0011) [2023-12-26 22:04:39,903][105692] Updated weights for policy 0, policy_version 921865 (0.0009) [2023-12-26 22:04:40,060][105620] Updated weights for policy 1, policy_version 921937 (0.0008) [2023-12-26 22:04:40,125][105620] Updated weights for policy 1, policy_version 921947 (0.0009) [2023-12-26 22:04:40,190][105620] Updated weights for policy 1, policy_version 921957 (0.0009) [2023-12-26 22:04:40,251][105620] Updated weights for policy 1, policy_version 921967 (0.0008) [2023-12-26 22:04:40,674][105692] Updated weights for policy 0, policy_version 921875 (0.0011) [2023-12-26 22:04:40,723][105692] Updated weights for policy 0, policy_version 921885 (0.0010) [2023-12-26 22:04:40,775][105692] Updated weights for policy 0, policy_version 921895 (0.0010) [2023-12-26 22:04:40,967][105620] Updated weights for policy 1, policy_version 921977 (0.0008) [2023-12-26 22:04:41,026][105620] Updated weights for policy 1, policy_version 921987 (0.0009) [2023-12-26 22:04:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 18883.3). Total num frames: 472096768. Throughput: 0: 9479.7, 1: 9678.8. Samples: 472108256. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-26 22:04:41,063][104569] Avg episode reward: [(0, '8898.263'), (1, '7652.339')] [2023-12-26 22:04:41,083][105620] Updated weights for policy 1, policy_version 921997 (0.0008) [2023-12-26 22:04:41,556][105692] Updated weights for policy 0, policy_version 921905 (0.0010) [2023-12-26 22:04:41,608][105692] Updated weights for policy 0, policy_version 921915 (0.0009) [2023-12-26 22:04:41,675][105692] Updated weights for policy 0, policy_version 921925 (0.0008) [2023-12-26 22:04:41,733][105692] Updated weights for policy 0, policy_version 921935 (0.0008) [2023-12-26 22:04:41,907][105620] Updated weights for policy 1, policy_version 922007 (0.0009) [2023-12-26 22:04:41,963][105620] Updated weights for policy 1, policy_version 922017 (0.0009) [2023-12-26 22:04:42,027][105620] Updated weights for policy 1, policy_version 922027 (0.0010) [2023-12-26 22:04:42,452][105692] Updated weights for policy 0, policy_version 921945 (0.0007) [2023-12-26 22:04:42,453][105585] KL-divergence is very high: 103.9259 [2023-12-26 22:04:42,460][105585] KL-divergence is very high: 129.4706 [2023-12-26 22:04:42,473][105585] KL-divergence is very high: 156.3314 [2023-12-26 22:04:42,481][105585] KL-divergence is very high: 192.7613 [2023-12-26 22:04:42,488][105585] KL-divergence is very high: 197.2929 [2023-12-26 22:04:42,507][105585] KL-divergence is very high: 181.2343 [2023-12-26 22:04:42,514][105585] KL-divergence is very high: 202.6241 [2023-12-26 22:04:42,518][105692] Updated weights for policy 0, policy_version 921955 (0.0009) [2023-12-26 22:04:42,527][105585] KL-divergence is very high: 180.7732 [2023-12-26 22:04:42,534][105585] KL-divergence is very high: 251.3061 [2023-12-26 22:04:42,541][105585] KL-divergence is very high: 193.6533 [2023-12-26 22:04:42,560][105585] KL-divergence is very high: 151.8381 [2023-12-26 22:04:42,565][105585] KL-divergence is very high: 162.5054 [2023-12-26 22:04:42,578][105585] KL-divergence is very high: 123.1561 [2023-12-26 22:04:42,584][105585] KL-divergence is very high: 236.0293 [2023-12-26 22:04:42,584][105692] Updated weights for policy 0, policy_version 921965 (0.0009) [2023-12-26 22:04:42,829][105620] Updated weights for policy 1, policy_version 922037 (0.0010) [2023-12-26 22:04:42,883][105620] Updated weights for policy 1, policy_version 922047 (0.0009) [2023-12-26 22:04:42,938][105620] Updated weights for policy 1, policy_version 922057 (0.0009) [2023-12-26 22:04:43,302][105692] Updated weights for policy 0, policy_version 921975 (0.0009) [2023-12-26 22:04:43,360][105692] Updated weights for policy 0, policy_version 921985 (0.0009) [2023-12-26 22:04:43,406][105692] Updated weights for policy 0, policy_version 921995 (0.0009) [2023-12-26 22:04:43,664][105620] Updated weights for policy 1, policy_version 922067 (0.0008) [2023-12-26 22:04:43,712][105620] Updated weights for policy 1, policy_version 922077 (0.0009) [2023-12-26 22:04:43,765][105620] Updated weights for policy 1, policy_version 922087 (0.0009) [2023-12-26 22:04:44,139][105692] Updated weights for policy 0, policy_version 922005 (0.0010) [2023-12-26 22:04:44,198][105692] Updated weights for policy 0, policy_version 922015 (0.0006) [2023-12-26 22:04:44,251][105692] Updated weights for policy 0, policy_version 922025 (0.0005) [2023-12-26 22:04:44,581][105620] Updated weights for policy 1, policy_version 922097 (0.0009) [2023-12-26 22:04:44,644][105620] Updated weights for policy 1, policy_version 922107 (0.0009) [2023-12-26 22:04:44,710][105620] Updated weights for policy 1, policy_version 922117 (0.0006) [2023-12-26 22:04:44,785][105620] Updated weights for policy 1, policy_version 922127 (0.0007) [2023-12-26 22:04:44,825][105692] Updated weights for policy 0, policy_version 922035 (0.0007) [2023-12-26 22:04:44,888][105692] Updated weights for policy 0, policy_version 922045 (0.0010) [2023-12-26 22:04:44,930][105585] KL-divergence is very high: 102.2495 [2023-12-26 22:04:44,957][105692] Updated weights for policy 0, policy_version 922055 (0.0006) [2023-12-26 22:04:45,499][105620] Updated weights for policy 1, policy_version 922137 (0.0009) [2023-12-26 22:04:45,565][105620] Updated weights for policy 1, policy_version 922147 (0.0009) [2023-12-26 22:04:45,620][105620] Updated weights for policy 1, policy_version 922157 (0.0009) [2023-12-26 22:04:45,667][105692] Updated weights for policy 0, policy_version 922065 (0.0007) [2023-12-26 22:04:45,725][105692] Updated weights for policy 0, policy_version 922075 (0.0005) [2023-12-26 22:04:45,781][105692] Updated weights for policy 0, policy_version 922085 (0.0006) [2023-12-26 22:04:45,832][105692] Updated weights for policy 0, policy_version 922095 (0.0009) [2023-12-26 22:04:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 18911.0). Total num frames: 472195072. Throughput: 0: 9449.9, 1: 9609.6. Samples: 472162560. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-26 22:04:46,062][104569] Avg episode reward: [(0, '6018.995'), (1, '8504.812')] [2023-12-26 22:04:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000922096_236093440.pth... [2023-12-26 22:04:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000922160_236101632.pth... [2023-12-26 22:04:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000920976_235806720.pth [2023-12-26 22:04:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000921072_235823104.pth [2023-12-26 22:04:46,300][105620] Updated weights for policy 1, policy_version 922167 (0.0009) [2023-12-26 22:04:46,351][105620] Updated weights for policy 1, policy_version 922177 (0.0009) [2023-12-26 22:04:46,405][105620] Updated weights for policy 1, policy_version 922187 (0.0009) [2023-12-26 22:04:46,569][105692] Updated weights for policy 0, policy_version 922105 (0.0009) [2023-12-26 22:04:46,641][105692] Updated weights for policy 0, policy_version 922115 (0.0009) [2023-12-26 22:04:46,711][105692] Updated weights for policy 0, policy_version 922125 (0.0009) [2023-12-26 22:04:47,213][105620] Updated weights for policy 1, policy_version 922197 (0.0009) [2023-12-26 22:04:47,261][105620] Updated weights for policy 1, policy_version 922207 (0.0009) [2023-12-26 22:04:47,307][105620] Updated weights for policy 1, policy_version 922217 (0.0009) [2023-12-26 22:04:47,353][105692] Updated weights for policy 0, policy_version 922135 (0.0007) [2023-12-26 22:04:47,400][105692] Updated weights for policy 0, policy_version 922145 (0.0008) [2023-12-26 22:04:47,446][105692] Updated weights for policy 0, policy_version 922155 (0.0008) [2023-12-26 22:04:47,957][105620] Updated weights for policy 1, policy_version 922227 (0.0007) [2023-12-26 22:04:48,022][105620] Updated weights for policy 1, policy_version 922237 (0.0007) [2023-12-26 22:04:48,088][105620] Updated weights for policy 1, policy_version 922247 (0.0008) [2023-12-26 22:04:48,297][105692] Updated weights for policy 0, policy_version 922165 (0.0009) [2023-12-26 22:04:48,352][105692] Updated weights for policy 0, policy_version 922175 (0.0009) [2023-12-26 22:04:48,407][105692] Updated weights for policy 0, policy_version 922185 (0.0008) [2023-12-26 22:04:48,737][105620] Updated weights for policy 1, policy_version 922257 (0.0008) [2023-12-26 22:04:48,804][105620] Updated weights for policy 1, policy_version 922267 (0.0007) [2023-12-26 22:04:48,867][105620] Updated weights for policy 1, policy_version 922277 (0.0009) [2023-12-26 22:04:48,923][105620] Updated weights for policy 1, policy_version 922287 (0.0009) [2023-12-26 22:04:49,201][105692] Updated weights for policy 0, policy_version 922195 (0.0007) [2023-12-26 22:04:49,268][105692] Updated weights for policy 0, policy_version 922205 (0.0009) [2023-12-26 22:04:49,329][105692] Updated weights for policy 0, policy_version 922215 (0.0008) [2023-12-26 22:04:49,695][105620] Updated weights for policy 1, policy_version 922297 (0.0009) [2023-12-26 22:04:49,742][105620] Updated weights for policy 1, policy_version 922307 (0.0009) [2023-12-26 22:04:49,789][105620] Updated weights for policy 1, policy_version 922317 (0.0009) [2023-12-26 22:04:50,025][105692] Updated weights for policy 0, policy_version 922225 (0.0009) [2023-12-26 22:04:50,082][105692] Updated weights for policy 0, policy_version 922235 (0.0006) [2023-12-26 22:04:50,146][105692] Updated weights for policy 0, policy_version 922245 (0.0006) [2023-12-26 22:04:50,213][105692] Updated weights for policy 0, policy_version 922255 (0.0006) [2023-12-26 22:04:50,554][105620] Updated weights for policy 1, policy_version 922327 (0.0007) [2023-12-26 22:04:50,614][105620] Updated weights for policy 1, policy_version 922337 (0.0008) [2023-12-26 22:04:50,674][105620] Updated weights for policy 1, policy_version 922347 (0.0009) [2023-12-26 22:04:50,890][105692] Updated weights for policy 0, policy_version 922265 (0.0009) [2023-12-26 22:04:50,941][105692] Updated weights for policy 0, policy_version 922275 (0.0008) [2023-12-26 22:04:51,004][105692] Updated weights for policy 0, policy_version 922285 (0.0009) [2023-12-26 22:04:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 18911.0). Total num frames: 472293376. Throughput: 0: 9511.2, 1: 9567.3. Samples: 472278436. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-26 22:04:51,063][104569] Avg episode reward: [(0, '6416.408'), (1, '8585.879')] [2023-12-26 22:04:51,434][105620] Updated weights for policy 1, policy_version 922357 (0.0009) [2023-12-26 22:04:51,490][105620] Updated weights for policy 1, policy_version 922367 (0.0009) [2023-12-26 22:04:51,538][105620] Updated weights for policy 1, policy_version 922377 (0.0009) [2023-12-26 22:04:51,788][105692] Updated weights for policy 0, policy_version 922295 (0.0009) [2023-12-26 22:04:51,849][105692] Updated weights for policy 0, policy_version 922305 (0.0008) [2023-12-26 22:04:51,911][105692] Updated weights for policy 0, policy_version 922315 (0.0009) [2023-12-26 22:04:52,393][105620] Updated weights for policy 1, policy_version 922387 (0.0009) [2023-12-26 22:04:52,457][105620] Updated weights for policy 1, policy_version 922397 (0.0009) [2023-12-26 22:04:52,519][105620] Updated weights for policy 1, policy_version 922407 (0.0010) [2023-12-26 22:04:52,562][105692] Updated weights for policy 0, policy_version 922325 (0.0006) [2023-12-26 22:04:52,618][105692] Updated weights for policy 0, policy_version 922335 (0.0009) [2023-12-26 22:04:52,668][105692] Updated weights for policy 0, policy_version 922345 (0.0008) [2023-12-26 22:04:53,237][105620] Updated weights for policy 1, policy_version 922417 (0.0007) [2023-12-26 22:04:53,306][105620] Updated weights for policy 1, policy_version 922427 (0.0005) [2023-12-26 22:04:53,357][105620] Updated weights for policy 1, policy_version 922437 (0.0005) [2023-12-26 22:04:53,406][105620] Updated weights for policy 1, policy_version 922447 (0.0005) [2023-12-26 22:04:53,480][105692] Updated weights for policy 0, policy_version 922355 (0.0009) [2023-12-26 22:04:53,538][105692] Updated weights for policy 0, policy_version 922365 (0.0010) [2023-12-26 22:04:53,592][105692] Updated weights for policy 0, policy_version 922375 (0.0009) [2023-12-26 22:04:53,918][105620] Updated weights for policy 1, policy_version 922457 (0.0008) [2023-12-26 22:04:53,967][105620] Updated weights for policy 1, policy_version 922467 (0.0008) [2023-12-26 22:04:54,020][105620] Updated weights for policy 1, policy_version 922478 (0.0006) [2023-12-26 22:04:54,312][105692] Updated weights for policy 0, policy_version 922385 (0.0009) [2023-12-26 22:04:54,372][105692] Updated weights for policy 0, policy_version 922395 (0.0006) [2023-12-26 22:04:54,420][105692] Updated weights for policy 0, policy_version 922405 (0.0009) [2023-12-26 22:04:54,479][105692] Updated weights for policy 0, policy_version 922415 (0.0009) [2023-12-26 22:04:54,834][105620] Updated weights for policy 1, policy_version 922488 (0.0009) [2023-12-26 22:04:54,897][105620] Updated weights for policy 1, policy_version 922498 (0.0008) [2023-12-26 22:04:54,944][105620] Updated weights for policy 1, policy_version 922508 (0.0008) [2023-12-26 22:04:55,194][105692] Updated weights for policy 0, policy_version 922425 (0.0007) [2023-12-26 22:04:55,245][105692] Updated weights for policy 0, policy_version 922435 (0.0005) [2023-12-26 22:04:55,326][105692] Updated weights for policy 0, policy_version 922445 (0.0006) [2023-12-26 22:04:55,639][105620] Updated weights for policy 1, policy_version 922518 (0.0007) [2023-12-26 22:04:55,698][105620] Updated weights for policy 1, policy_version 922528 (0.0010) [2023-12-26 22:04:55,767][105620] Updated weights for policy 1, policy_version 922538 (0.0010) [2023-12-26 22:04:56,008][105692] Updated weights for policy 0, policy_version 922455 (0.0008) [2023-12-26 22:04:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 18855.5). Total num frames: 472383488. Throughput: 0: 9624.2, 1: 9504.7. Samples: 472394848. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-26 22:04:56,062][104569] Avg episode reward: [(0, '8435.535'), (1, '8319.582')] [2023-12-26 22:04:56,064][105692] Updated weights for policy 0, policy_version 922465 (0.0008) [2023-12-26 22:04:56,124][105692] Updated weights for policy 0, policy_version 922475 (0.0008) [2023-12-26 22:04:56,473][105620] Updated weights for policy 1, policy_version 922548 (0.0010) [2023-12-26 22:04:56,537][105620] Updated weights for policy 1, policy_version 922558 (0.0010) [2023-12-26 22:04:56,596][105620] Updated weights for policy 1, policy_version 922568 (0.0010) [2023-12-26 22:04:56,894][105692] Updated weights for policy 0, policy_version 922485 (0.0008) [2023-12-26 22:04:56,948][105692] Updated weights for policy 0, policy_version 922495 (0.0007) [2023-12-26 22:04:57,003][105692] Updated weights for policy 0, policy_version 922505 (0.0008) [2023-12-26 22:04:57,336][105620] Updated weights for policy 1, policy_version 922578 (0.0010) [2023-12-26 22:04:57,391][105620] Updated weights for policy 1, policy_version 922588 (0.0010) [2023-12-26 22:04:57,454][105620] Updated weights for policy 1, policy_version 922598 (0.0011) [2023-12-26 22:04:57,503][105620] Updated weights for policy 1, policy_version 922608 (0.0010) [2023-12-26 22:04:57,773][105692] Updated weights for policy 0, policy_version 922515 (0.0008) [2023-12-26 22:04:57,830][105692] Updated weights for policy 0, policy_version 922525 (0.0009) [2023-12-26 22:04:57,882][105692] Updated weights for policy 0, policy_version 922535 (0.0009) [2023-12-26 22:04:58,134][105620] Updated weights for policy 1, policy_version 922618 (0.0010) [2023-12-26 22:04:58,200][105620] Updated weights for policy 1, policy_version 922628 (0.0009) [2023-12-26 22:04:58,265][105620] Updated weights for policy 1, policy_version 922638 (0.0009) [2023-12-26 22:04:58,723][105692] Updated weights for policy 0, policy_version 922545 (0.0010) [2023-12-26 22:04:58,787][105692] Updated weights for policy 0, policy_version 922555 (0.0009) [2023-12-26 22:04:58,857][105692] Updated weights for policy 0, policy_version 922565 (0.0008) [2023-12-26 22:04:58,932][105692] Updated weights for policy 0, policy_version 922575 (0.0008) [2023-12-26 22:04:59,120][105620] Updated weights for policy 1, policy_version 922648 (0.0008) [2023-12-26 22:04:59,179][105620] Updated weights for policy 1, policy_version 922658 (0.0008) [2023-12-26 22:04:59,248][105620] Updated weights for policy 1, policy_version 922668 (0.0007) [2023-12-26 22:04:59,656][105692] Updated weights for policy 0, policy_version 922585 (0.0009) [2023-12-26 22:04:59,707][105692] Updated weights for policy 0, policy_version 922595 (0.0006) [2023-12-26 22:04:59,761][105692] Updated weights for policy 0, policy_version 922605 (0.0008) [2023-12-26 22:04:59,999][105620] Updated weights for policy 1, policy_version 922678 (0.0009) [2023-12-26 22:05:00,052][105620] Updated weights for policy 1, policy_version 922688 (0.0010) [2023-12-26 22:05:00,061][105586] KL-divergence is very high: 115.4624 [2023-12-26 22:05:00,108][105586] KL-divergence is very high: 203.9144 [2023-12-26 22:05:00,110][105620] Updated weights for policy 1, policy_version 922698 (0.0008) [2023-12-26 22:05:00,475][105692] Updated weights for policy 0, policy_version 922615 (0.0009) [2023-12-26 22:05:00,529][105692] Updated weights for policy 0, policy_version 922625 (0.0009) [2023-12-26 22:05:00,594][105692] Updated weights for policy 0, policy_version 922635 (0.0007) [2023-12-26 22:05:00,938][105620] Updated weights for policy 1, policy_version 922708 (0.0009) [2023-12-26 22:05:01,010][105620] Updated weights for policy 1, policy_version 922718 (0.0010) [2023-12-26 22:05:01,062][104569] Fps is (10 sec: 18022.4, 60 sec: 18978.1, 300 sec: 18855.5). Total num frames: 472473600. Throughput: 0: 9618.8, 1: 9526.4. Samples: 472450368. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-26 22:05:01,063][104569] Avg episode reward: [(0, '9171.856'), (1, '8501.251')] [2023-12-26 22:05:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000922640_236232704.pth... [2023-12-26 22:05:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000921552_235954176.pth [2023-12-26 22:05:01,075][105620] Updated weights for policy 1, policy_version 922728 (0.0009) [2023-12-26 22:05:01,126][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000922736_236249088.pth... [2023-12-26 22:05:01,131][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000921616_235962368.pth [2023-12-26 22:05:01,212][105692] Updated weights for policy 0, policy_version 922645 (0.0009) [2023-12-26 22:05:01,273][105692] Updated weights for policy 0, policy_version 922655 (0.0008) [2023-12-26 22:05:01,332][105692] Updated weights for policy 0, policy_version 922665 (0.0009) [2023-12-26 22:05:01,872][105620] Updated weights for policy 1, policy_version 922738 (0.0008) [2023-12-26 22:05:01,925][105620] Updated weights for policy 1, policy_version 922748 (0.0009) [2023-12-26 22:05:01,973][105620] Updated weights for policy 1, policy_version 922758 (0.0009) [2023-12-26 22:05:02,024][105620] Updated weights for policy 1, policy_version 922768 (0.0009) [2023-12-26 22:05:02,074][105692] Updated weights for policy 0, policy_version 922675 (0.0009) [2023-12-26 22:05:02,128][105692] Updated weights for policy 0, policy_version 922685 (0.0008) [2023-12-26 22:05:02,180][105692] Updated weights for policy 0, policy_version 922695 (0.0009) [2023-12-26 22:05:02,779][105620] Updated weights for policy 1, policy_version 922778 (0.0009) [2023-12-26 22:05:02,833][105620] Updated weights for policy 1, policy_version 922788 (0.0009) [2023-12-26 22:05:02,893][105620] Updated weights for policy 1, policy_version 922798 (0.0008) [2023-12-26 22:05:02,982][105692] Updated weights for policy 0, policy_version 922705 (0.0009) [2023-12-26 22:05:03,043][105692] Updated weights for policy 0, policy_version 922715 (0.0009) [2023-12-26 22:05:03,103][105692] Updated weights for policy 0, policy_version 922725 (0.0010) [2023-12-26 22:05:03,167][105692] Updated weights for policy 0, policy_version 922735 (0.0009) [2023-12-26 22:05:03,511][105620] Updated weights for policy 1, policy_version 922808 (0.0005) [2023-12-26 22:05:03,576][105620] Updated weights for policy 1, policy_version 922818 (0.0008) [2023-12-26 22:05:03,631][105620] Updated weights for policy 1, policy_version 922828 (0.0006) [2023-12-26 22:05:04,009][105692] Updated weights for policy 0, policy_version 922745 (0.0011) [2023-12-26 22:05:04,073][105692] Updated weights for policy 0, policy_version 922755 (0.0010) [2023-12-26 22:05:04,130][105692] Updated weights for policy 0, policy_version 922765 (0.0009) [2023-12-26 22:05:04,283][105620] Updated weights for policy 1, policy_version 922838 (0.0008) [2023-12-26 22:05:04,350][105620] Updated weights for policy 1, policy_version 922848 (0.0009) [2023-12-26 22:05:04,411][105620] Updated weights for policy 1, policy_version 922858 (0.0009) [2023-12-26 22:05:04,884][105692] Updated weights for policy 0, policy_version 922775 (0.0008) [2023-12-26 22:05:04,949][105692] Updated weights for policy 0, policy_version 922785 (0.0006) [2023-12-26 22:05:05,008][105692] Updated weights for policy 0, policy_version 922795 (0.0010) [2023-12-26 22:05:05,184][105620] Updated weights for policy 1, policy_version 922868 (0.0009) [2023-12-26 22:05:05,236][105620] Updated weights for policy 1, policy_version 922878 (0.0009) [2023-12-26 22:05:05,291][105620] Updated weights for policy 1, policy_version 922890 (0.0010) [2023-12-26 22:05:05,603][105692] Updated weights for policy 0, policy_version 922805 (0.0006) [2023-12-26 22:05:05,668][105692] Updated weights for policy 0, policy_version 922815 (0.0006) [2023-12-26 22:05:05,737][105692] Updated weights for policy 0, policy_version 922825 (0.0006) [2023-12-26 22:05:05,944][105620] Updated weights for policy 1, policy_version 922900 (0.0009) [2023-12-26 22:05:05,991][105620] Updated weights for policy 1, policy_version 922910 (0.0008) [2023-12-26 22:05:06,044][105620] Updated weights for policy 1, policy_version 922920 (0.0009) [2023-12-26 22:05:06,062][104569] Fps is (10 sec: 18841.4, 60 sec: 18978.1, 300 sec: 18855.5). Total num frames: 472571904. Throughput: 0: 9533.5, 1: 9529.8. Samples: 472562212. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-26 22:05:06,063][104569] Avg episode reward: [(0, '8899.173'), (1, '8847.245')] [2023-12-26 22:05:06,336][105692] Updated weights for policy 0, policy_version 922835 (0.0006) [2023-12-26 22:05:06,401][105692] Updated weights for policy 0, policy_version 922845 (0.0008) [2023-12-26 22:05:06,464][105692] Updated weights for policy 0, policy_version 922855 (0.0008) [2023-12-26 22:05:06,845][105620] Updated weights for policy 1, policy_version 922930 (0.0009) [2023-12-26 22:05:06,900][105620] Updated weights for policy 1, policy_version 922940 (0.0008) [2023-12-26 22:05:06,962][105620] Updated weights for policy 1, policy_version 922950 (0.0009) [2023-12-26 22:05:07,025][105620] Updated weights for policy 1, policy_version 922960 (0.0009) [2023-12-26 22:05:07,194][105692] Updated weights for policy 0, policy_version 922865 (0.0008) [2023-12-26 22:05:07,254][105692] Updated weights for policy 0, policy_version 922875 (0.0007) [2023-12-26 22:05:07,319][105692] Updated weights for policy 0, policy_version 922885 (0.0006) [2023-12-26 22:05:07,383][105692] Updated weights for policy 0, policy_version 922895 (0.0006) [2023-12-26 22:05:07,805][105620] Updated weights for policy 1, policy_version 922970 (0.0005) [2023-12-26 22:05:07,859][105620] Updated weights for policy 1, policy_version 922980 (0.0005) [2023-12-26 22:05:07,927][105620] Updated weights for policy 1, policy_version 922990 (0.0006) [2023-12-26 22:05:07,975][105692] Updated weights for policy 0, policy_version 922905 (0.0005) [2023-12-26 22:05:08,044][105692] Updated weights for policy 0, policy_version 922915 (0.0007) [2023-12-26 22:05:08,095][105692] Updated weights for policy 0, policy_version 922925 (0.0009) [2023-12-26 22:05:08,635][105620] Updated weights for policy 1, policy_version 923000 (0.0010) [2023-12-26 22:05:08,689][105620] Updated weights for policy 1, policy_version 923010 (0.0010) [2023-12-26 22:05:08,743][105620] Updated weights for policy 1, policy_version 923020 (0.0009) [2023-12-26 22:05:08,745][105692] Updated weights for policy 0, policy_version 922935 (0.0007) [2023-12-26 22:05:08,811][105692] Updated weights for policy 0, policy_version 922945 (0.0008) [2023-12-26 22:05:08,877][105692] Updated weights for policy 0, policy_version 922955 (0.0008) [2023-12-26 22:05:09,492][105620] Updated weights for policy 1, policy_version 923030 (0.0007) [2023-12-26 22:05:09,560][105620] Updated weights for policy 1, policy_version 923040 (0.0006) [2023-12-26 22:05:09,620][105620] Updated weights for policy 1, policy_version 923050 (0.0009) [2023-12-26 22:05:09,676][105692] Updated weights for policy 0, policy_version 922965 (0.0009) [2023-12-26 22:05:09,738][105692] Updated weights for policy 0, policy_version 922975 (0.0010) [2023-12-26 22:05:09,801][105692] Updated weights for policy 0, policy_version 922985 (0.0009) [2023-12-26 22:05:10,313][105620] Updated weights for policy 1, policy_version 923060 (0.0007) [2023-12-26 22:05:10,373][105620] Updated weights for policy 1, policy_version 923070 (0.0009) [2023-12-26 22:05:10,420][105620] Updated weights for policy 1, policy_version 923080 (0.0008) [2023-12-26 22:05:10,566][105692] Updated weights for policy 0, policy_version 922995 (0.0008) [2023-12-26 22:05:10,623][105692] Updated weights for policy 0, policy_version 923005 (0.0005) [2023-12-26 22:05:10,670][105692] Updated weights for policy 0, policy_version 923015 (0.0006) [2023-12-26 22:05:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19114.6, 300 sec: 18911.0). Total num frames: 472670208. Throughput: 0: 9618.4, 1: 9496.3. Samples: 472680488. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-26 22:05:11,063][104569] Avg episode reward: [(0, '9077.963'), (1, '8480.544')] [2023-12-26 22:05:11,113][105620] Updated weights for policy 1, policy_version 923090 (0.0009) [2023-12-26 22:05:11,182][105620] Updated weights for policy 1, policy_version 923100 (0.0009) [2023-12-26 22:05:11,240][105620] Updated weights for policy 1, policy_version 923110 (0.0008) [2023-12-26 22:05:11,305][105620] Updated weights for policy 1, policy_version 923120 (0.0008) [2023-12-26 22:05:11,386][105692] Updated weights for policy 0, policy_version 923025 (0.0006) [2023-12-26 22:05:11,462][105692] Updated weights for policy 0, policy_version 923035 (0.0008) [2023-12-26 22:05:11,529][105692] Updated weights for policy 0, policy_version 923045 (0.0008) [2023-12-26 22:05:11,598][105692] Updated weights for policy 0, policy_version 923055 (0.0008) [2023-12-26 22:05:12,098][105620] Updated weights for policy 1, policy_version 923130 (0.0007) [2023-12-26 22:05:12,160][105620] Updated weights for policy 1, policy_version 923140 (0.0009) [2023-12-26 22:05:12,217][105620] Updated weights for policy 1, policy_version 923150 (0.0009) [2023-12-26 22:05:12,358][105692] Updated weights for policy 0, policy_version 923065 (0.0009) [2023-12-26 22:05:12,420][105692] Updated weights for policy 0, policy_version 923075 (0.0010) [2023-12-26 22:05:12,485][105692] Updated weights for policy 0, policy_version 923085 (0.0010) [2023-12-26 22:05:12,994][105620] Updated weights for policy 1, policy_version 923160 (0.0009) [2023-12-26 22:05:13,051][105620] Updated weights for policy 1, policy_version 923170 (0.0009) [2023-12-26 22:05:13,104][105620] Updated weights for policy 1, policy_version 923181 (0.0009) [2023-12-26 22:05:13,198][105692] Updated weights for policy 0, policy_version 923095 (0.0008) [2023-12-26 22:05:13,250][105692] Updated weights for policy 0, policy_version 923105 (0.0006) [2023-12-26 22:05:13,313][105692] Updated weights for policy 0, policy_version 923115 (0.0008) [2023-12-26 22:05:13,891][105620] Updated weights for policy 1, policy_version 923191 (0.0009) [2023-12-26 22:05:13,955][105620] Updated weights for policy 1, policy_version 923201 (0.0008) [2023-12-26 22:05:13,977][105692] Updated weights for policy 0, policy_version 923125 (0.0009) [2023-12-26 22:05:14,018][105620] Updated weights for policy 1, policy_version 923211 (0.0008) [2023-12-26 22:05:14,025][105692] Updated weights for policy 0, policy_version 923135 (0.0008) [2023-12-26 22:05:14,079][105692] Updated weights for policy 0, policy_version 923145 (0.0006) [2023-12-26 22:05:14,697][105620] Updated weights for policy 1, policy_version 923221 (0.0006) [2023-12-26 22:05:14,745][105620] Updated weights for policy 1, policy_version 923231 (0.0009) [2023-12-26 22:05:14,763][105692] Updated weights for policy 0, policy_version 923155 (0.0009) [2023-12-26 22:05:14,810][105620] Updated weights for policy 1, policy_version 923241 (0.0010) [2023-12-26 22:05:14,825][105692] Updated weights for policy 0, policy_version 923165 (0.0006) [2023-12-26 22:05:14,883][105692] Updated weights for policy 0, policy_version 923175 (0.0008) [2023-12-26 22:05:15,471][105620] Updated weights for policy 1, policy_version 923251 (0.0010) [2023-12-26 22:05:15,527][105620] Updated weights for policy 1, policy_version 923261 (0.0010) [2023-12-26 22:05:15,580][105620] Updated weights for policy 1, policy_version 923271 (0.0009) [2023-12-26 22:05:15,596][105692] Updated weights for policy 0, policy_version 923185 (0.0008) [2023-12-26 22:05:15,659][105692] Updated weights for policy 0, policy_version 923195 (0.0007) [2023-12-26 22:05:15,717][105692] Updated weights for policy 0, policy_version 923205 (0.0009) [2023-12-26 22:05:15,780][105692] Updated weights for policy 0, policy_version 923215 (0.0009) [2023-12-26 22:05:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.1, 300 sec: 18938.8). Total num frames: 472768512. Throughput: 0: 9548.0, 1: 9437.9. Samples: 472735640. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-26 22:05:16,063][104569] Avg episode reward: [(0, '9168.510'), (1, '7840.776')] [2023-12-26 22:05:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000923216_236380160.pth... [2023-12-26 22:05:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000923280_236388352.pth... [2023-12-26 22:05:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000922096_236093440.pth [2023-12-26 22:05:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000922160_236101632.pth [2023-12-26 22:05:16,205][105620] Updated weights for policy 1, policy_version 923281 (0.0007) [2023-12-26 22:05:16,267][105620] Updated weights for policy 1, policy_version 923291 (0.0009) [2023-12-26 22:05:16,318][105620] Updated weights for policy 1, policy_version 923301 (0.0009) [2023-12-26 22:05:16,368][105620] Updated weights for policy 1, policy_version 923311 (0.0008) [2023-12-26 22:05:16,571][105692] Updated weights for policy 0, policy_version 923225 (0.0008) [2023-12-26 22:05:16,618][105692] Updated weights for policy 0, policy_version 923235 (0.0009) [2023-12-26 22:05:16,671][105692] Updated weights for policy 0, policy_version 923246 (0.0009) [2023-12-26 22:05:17,126][105620] Updated weights for policy 1, policy_version 923321 (0.0009) [2023-12-26 22:05:17,188][105620] Updated weights for policy 1, policy_version 923331 (0.0009) [2023-12-26 22:05:17,238][105620] Updated weights for policy 1, policy_version 923341 (0.0008) [2023-12-26 22:05:17,442][105692] Updated weights for policy 0, policy_version 923256 (0.0009) [2023-12-26 22:05:17,489][105692] Updated weights for policy 0, policy_version 923266 (0.0008) [2023-12-26 22:05:17,544][105692] Updated weights for policy 0, policy_version 923276 (0.0009) [2023-12-26 22:05:18,003][105620] Updated weights for policy 1, policy_version 923351 (0.0009) [2023-12-26 22:05:18,050][105620] Updated weights for policy 1, policy_version 923361 (0.0009) [2023-12-26 22:05:18,102][105620] Updated weights for policy 1, policy_version 923371 (0.0009) [2023-12-26 22:05:18,254][105692] Updated weights for policy 0, policy_version 923286 (0.0009) [2023-12-26 22:05:18,305][105692] Updated weights for policy 0, policy_version 923296 (0.0009) [2023-12-26 22:05:18,363][105692] Updated weights for policy 0, policy_version 923306 (0.0009) [2023-12-26 22:05:18,867][105620] Updated weights for policy 1, policy_version 923382 (0.0008) [2023-12-26 22:05:18,934][105620] Updated weights for policy 1, policy_version 923392 (0.0008) [2023-12-26 22:05:18,989][105620] Updated weights for policy 1, policy_version 923402 (0.0010) [2023-12-26 22:05:19,046][105692] Updated weights for policy 0, policy_version 923316 (0.0009) [2023-12-26 22:05:19,110][105692] Updated weights for policy 0, policy_version 923326 (0.0009) [2023-12-26 22:05:19,172][105692] Updated weights for policy 0, policy_version 923336 (0.0009) [2023-12-26 22:05:19,697][105620] Updated weights for policy 1, policy_version 923412 (0.0008) [2023-12-26 22:05:19,756][105620] Updated weights for policy 1, policy_version 923422 (0.0006) [2023-12-26 22:05:19,813][105620] Updated weights for policy 1, policy_version 923432 (0.0006) [2023-12-26 22:05:19,915][105692] Updated weights for policy 0, policy_version 923346 (0.0009) [2023-12-26 22:05:19,986][105692] Updated weights for policy 0, policy_version 923356 (0.0007) [2023-12-26 22:05:20,049][105692] Updated weights for policy 0, policy_version 923366 (0.0006) [2023-12-26 22:05:20,118][105692] Updated weights for policy 0, policy_version 923376 (0.0006) [2023-12-26 22:05:20,544][105620] Updated weights for policy 1, policy_version 923442 (0.0007) [2023-12-26 22:05:20,619][105620] Updated weights for policy 1, policy_version 923452 (0.0009) [2023-12-26 22:05:20,679][105620] Updated weights for policy 1, policy_version 923462 (0.0007) [2023-12-26 22:05:20,717][105692] Updated weights for policy 0, policy_version 923386 (0.0011) [2023-12-26 22:05:20,736][105620] Updated weights for policy 1, policy_version 923472 (0.0006) [2023-12-26 22:05:20,775][105692] Updated weights for policy 0, policy_version 923396 (0.0009) [2023-12-26 22:05:20,839][105692] Updated weights for policy 0, policy_version 923406 (0.0005) [2023-12-26 22:05:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 18966.6). Total num frames: 472866816. Throughput: 0: 9548.1, 1: 9515.4. Samples: 472853484. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-26 22:05:21,062][104569] Avg episode reward: [(0, '8658.843'), (1, '8196.144')] [2023-12-26 22:05:21,405][105620] Updated weights for policy 1, policy_version 923482 (0.0008) [2023-12-26 22:05:21,468][105620] Updated weights for policy 1, policy_version 923492 (0.0009) [2023-12-26 22:05:21,509][105692] Updated weights for policy 0, policy_version 923416 (0.0009) [2023-12-26 22:05:21,529][105620] Updated weights for policy 1, policy_version 923502 (0.0010) [2023-12-26 22:05:21,571][105692] Updated weights for policy 0, policy_version 923426 (0.0010) [2023-12-26 22:05:21,635][105692] Updated weights for policy 0, policy_version 923436 (0.0009) [2023-12-26 22:05:22,305][105620] Updated weights for policy 1, policy_version 923512 (0.0009) [2023-12-26 22:05:22,379][105620] Updated weights for policy 1, policy_version 923522 (0.0008) [2023-12-26 22:05:22,408][105692] Updated weights for policy 0, policy_version 923446 (0.0008) [2023-12-26 22:05:22,435][105620] Updated weights for policy 1, policy_version 923532 (0.0006) [2023-12-26 22:05:22,471][105692] Updated weights for policy 0, policy_version 923456 (0.0008) [2023-12-26 22:05:22,526][105692] Updated weights for policy 0, policy_version 923466 (0.0009) [2023-12-26 22:05:23,113][105620] Updated weights for policy 1, policy_version 923542 (0.0008) [2023-12-26 22:05:23,165][105620] Updated weights for policy 1, policy_version 923552 (0.0009) [2023-12-26 22:05:23,224][105620] Updated weights for policy 1, policy_version 923562 (0.0009) [2023-12-26 22:05:23,316][105692] Updated weights for policy 0, policy_version 923476 (0.0010) [2023-12-26 22:05:23,375][105692] Updated weights for policy 0, policy_version 923486 (0.0010) [2023-12-26 22:05:23,436][105692] Updated weights for policy 0, policy_version 923496 (0.0009) [2023-12-26 22:05:23,890][105620] Updated weights for policy 1, policy_version 923572 (0.0007) [2023-12-26 22:05:23,936][105620] Updated weights for policy 1, policy_version 923582 (0.0005) [2023-12-26 22:05:23,993][105620] Updated weights for policy 1, policy_version 923592 (0.0005) [2023-12-26 22:05:24,326][105692] Updated weights for policy 0, policy_version 923506 (0.0009) [2023-12-26 22:05:24,383][105692] Updated weights for policy 0, policy_version 923516 (0.0008) [2023-12-26 22:05:24,445][105692] Updated weights for policy 0, policy_version 923527 (0.0010) [2023-12-26 22:05:24,552][105620] Updated weights for policy 1, policy_version 923602 (0.0005) [2023-12-26 22:05:24,613][105620] Updated weights for policy 1, policy_version 923612 (0.0005) [2023-12-26 22:05:24,677][105620] Updated weights for policy 1, policy_version 923622 (0.0009) [2023-12-26 22:05:24,731][105620] Updated weights for policy 1, policy_version 923632 (0.0008) [2023-12-26 22:05:25,240][105692] Updated weights for policy 0, policy_version 923537 (0.0010) [2023-12-26 22:05:25,294][105692] Updated weights for policy 0, policy_version 923547 (0.0008) [2023-12-26 22:05:25,342][105692] Updated weights for policy 0, policy_version 923557 (0.0009) [2023-12-26 22:05:25,396][105692] Updated weights for policy 0, policy_version 923567 (0.0009) [2023-12-26 22:05:25,441][105620] Updated weights for policy 1, policy_version 923642 (0.0008) [2023-12-26 22:05:25,491][105620] Updated weights for policy 1, policy_version 923652 (0.0008) [2023-12-26 22:05:25,538][105620] Updated weights for policy 1, policy_version 923662 (0.0009) [2023-12-26 22:05:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.6, 300 sec: 18938.8). Total num frames: 472956928. Throughput: 0: 9548.7, 1: 9563.2. Samples: 472968292. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-26 22:05:26,063][104569] Avg episode reward: [(0, '8836.091'), (1, '8207.178')] [2023-12-26 22:05:26,163][105692] Updated weights for policy 0, policy_version 923577 (0.0010) [2023-12-26 22:05:26,224][105692] Updated weights for policy 0, policy_version 923587 (0.0008) [2023-12-26 22:05:26,287][105692] Updated weights for policy 0, policy_version 923597 (0.0008) [2023-12-26 22:05:26,324][105620] Updated weights for policy 1, policy_version 923672 (0.0008) [2023-12-26 22:05:26,375][105620] Updated weights for policy 1, policy_version 923682 (0.0008) [2023-12-26 22:05:26,423][105620] Updated weights for policy 1, policy_version 923692 (0.0009) [2023-12-26 22:05:27,039][105692] Updated weights for policy 0, policy_version 923607 (0.0009) [2023-12-26 22:05:27,101][105692] Updated weights for policy 0, policy_version 923617 (0.0009) [2023-12-26 22:05:27,160][105692] Updated weights for policy 0, policy_version 923627 (0.0008) [2023-12-26 22:05:27,171][105620] Updated weights for policy 1, policy_version 923702 (0.0007) [2023-12-26 22:05:27,223][105620] Updated weights for policy 1, policy_version 923712 (0.0008) [2023-12-26 22:05:27,270][105620] Updated weights for policy 1, policy_version 923722 (0.0008) [2023-12-26 22:05:27,852][105692] Updated weights for policy 0, policy_version 923637 (0.0009) [2023-12-26 22:05:27,902][105692] Updated weights for policy 0, policy_version 923647 (0.0009) [2023-12-26 22:05:27,961][105692] Updated weights for policy 0, policy_version 923657 (0.0009) [2023-12-26 22:05:28,018][105620] Updated weights for policy 1, policy_version 923732 (0.0008) [2023-12-26 22:05:28,068][105620] Updated weights for policy 1, policy_version 923742 (0.0009) [2023-12-26 22:05:28,119][105620] Updated weights for policy 1, policy_version 923752 (0.0009) [2023-12-26 22:05:28,718][105692] Updated weights for policy 0, policy_version 923667 (0.0009) [2023-12-26 22:05:28,775][105692] Updated weights for policy 0, policy_version 923677 (0.0010) [2023-12-26 22:05:28,832][105692] Updated weights for policy 0, policy_version 923687 (0.0010) [2023-12-26 22:05:28,886][105620] Updated weights for policy 1, policy_version 923762 (0.0008) [2023-12-26 22:05:28,951][105620] Updated weights for policy 1, policy_version 923772 (0.0006) [2023-12-26 22:05:29,008][105620] Updated weights for policy 1, policy_version 923782 (0.0005) [2023-12-26 22:05:29,065][105620] Updated weights for policy 1, policy_version 923792 (0.0006) [2023-12-26 22:05:29,641][105692] Updated weights for policy 0, policy_version 923697 (0.0008) [2023-12-26 22:05:29,692][105692] Updated weights for policy 0, policy_version 923707 (0.0009) [2023-12-26 22:05:29,752][105692] Updated weights for policy 0, policy_version 923717 (0.0007) [2023-12-26 22:05:29,762][105620] Updated weights for policy 1, policy_version 923802 (0.0008) [2023-12-26 22:05:29,812][105692] Updated weights for policy 0, policy_version 923727 (0.0007) [2023-12-26 22:05:29,823][105620] Updated weights for policy 1, policy_version 923812 (0.0006) [2023-12-26 22:05:29,888][105620] Updated weights for policy 1, policy_version 923822 (0.0008) [2023-12-26 22:05:30,576][105692] Updated weights for policy 0, policy_version 923737 (0.0009) [2023-12-26 22:05:30,614][105620] Updated weights for policy 1, policy_version 923832 (0.0007) [2023-12-26 22:05:30,629][105692] Updated weights for policy 0, policy_version 923747 (0.0007) [2023-12-26 22:05:30,671][105620] Updated weights for policy 1, policy_version 923842 (0.0007) [2023-12-26 22:05:30,689][105692] Updated weights for policy 0, policy_version 923757 (0.0007) [2023-12-26 22:05:30,724][105620] Updated weights for policy 1, policy_version 923852 (0.0007) [2023-12-26 22:05:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.7, 300 sec: 18938.8). Total num frames: 473055232. Throughput: 0: 9565.7, 1: 9606.6. Samples: 473025312. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-26 22:05:31,062][104569] Avg episode reward: [(0, '9174.872'), (1, '8390.578')] [2023-12-26 22:05:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000923760_236519424.pth... [2023-12-26 22:05:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000923856_236535808.pth... [2023-12-26 22:05:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000922736_236249088.pth [2023-12-26 22:05:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000922640_236232704.pth [2023-12-26 22:05:31,424][105692] Updated weights for policy 0, policy_version 923767 (0.0007) [2023-12-26 22:05:31,468][105620] Updated weights for policy 1, policy_version 923862 (0.0009) [2023-12-26 22:05:31,484][105692] Updated weights for policy 0, policy_version 923777 (0.0008) [2023-12-26 22:05:31,527][105620] Updated weights for policy 1, policy_version 923872 (0.0007) [2023-12-26 22:05:31,538][105692] Updated weights for policy 0, policy_version 923787 (0.0008) [2023-12-26 22:05:31,556][105586] KL-divergence is very high: 114.3765 [2023-12-26 22:05:31,586][105620] Updated weights for policy 1, policy_version 923882 (0.0007) [2023-12-26 22:05:31,607][105586] KL-divergence is very high: 122.1169 [2023-12-26 22:05:32,306][105692] Updated weights for policy 0, policy_version 923797 (0.0008) [2023-12-26 22:05:32,363][105620] Updated weights for policy 1, policy_version 923892 (0.0008) [2023-12-26 22:05:32,364][105692] Updated weights for policy 0, policy_version 923807 (0.0009) [2023-12-26 22:05:32,423][105620] Updated weights for policy 1, policy_version 923902 (0.0005) [2023-12-26 22:05:32,424][105692] Updated weights for policy 0, policy_version 923817 (0.0009) [2023-12-26 22:05:32,487][105620] Updated weights for policy 1, policy_version 923912 (0.0006) [2023-12-26 22:05:33,036][105620] Updated weights for policy 1, policy_version 923922 (0.0008) [2023-12-26 22:05:33,089][105620] Updated weights for policy 1, policy_version 923933 (0.0009) [2023-12-26 22:05:33,141][105620] Updated weights for policy 1, policy_version 923943 (0.0009) [2023-12-26 22:05:33,220][105692] Updated weights for policy 0, policy_version 923827 (0.0008) [2023-12-26 22:05:33,277][105692] Updated weights for policy 0, policy_version 923837 (0.0009) [2023-12-26 22:05:33,338][105692] Updated weights for policy 0, policy_version 923847 (0.0009) [2023-12-26 22:05:33,945][105620] Updated weights for policy 1, policy_version 923953 (0.0009) [2023-12-26 22:05:34,006][105692] Updated weights for policy 0, policy_version 923857 (0.0009) [2023-12-26 22:05:34,012][105620] Updated weights for policy 1, policy_version 923963 (0.0007) [2023-12-26 22:05:34,051][105692] Updated weights for policy 0, policy_version 923867 (0.0006) [2023-12-26 22:05:34,064][105620] Updated weights for policy 1, policy_version 923973 (0.0007) [2023-12-26 22:05:34,097][105692] Updated weights for policy 0, policy_version 923877 (0.0008) [2023-12-26 22:05:34,115][105620] Updated weights for policy 1, policy_version 923983 (0.0009) [2023-12-26 22:05:34,147][105692] Updated weights for policy 0, policy_version 923887 (0.0008) [2023-12-26 22:05:34,758][105620] Updated weights for policy 1, policy_version 923993 (0.0005) [2023-12-26 22:05:34,819][105620] Updated weights for policy 1, policy_version 924003 (0.0005) [2023-12-26 22:05:34,876][105620] Updated weights for policy 1, policy_version 924013 (0.0010) [2023-12-26 22:05:34,949][105692] Updated weights for policy 0, policy_version 923897 (0.0009) [2023-12-26 22:05:35,007][105692] Updated weights for policy 0, policy_version 923907 (0.0008) [2023-12-26 22:05:35,062][105692] Updated weights for policy 0, policy_version 923917 (0.0008) [2023-12-26 22:05:35,539][105620] Updated weights for policy 1, policy_version 924023 (0.0010) [2023-12-26 22:05:35,583][105620] Updated weights for policy 1, policy_version 924033 (0.0010) [2023-12-26 22:05:35,631][105620] Updated weights for policy 1, policy_version 924043 (0.0010) [2023-12-26 22:05:35,683][105692] Updated weights for policy 0, policy_version 923927 (0.0006) [2023-12-26 22:05:35,738][105692] Updated weights for policy 0, policy_version 923937 (0.0005) [2023-12-26 22:05:35,799][105692] Updated weights for policy 0, policy_version 923947 (0.0006) [2023-12-26 22:05:36,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19114.7, 300 sec: 18966.6). Total num frames: 473153536. Throughput: 0: 9498.2, 1: 9653.4. Samples: 473140252. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-26 22:05:36,062][104569] Avg episode reward: [(0, '9081.472'), (1, '7611.247')] [2023-12-26 22:05:36,346][105692] Updated weights for policy 0, policy_version 923957 (0.0005) [2023-12-26 22:05:36,370][105620] Updated weights for policy 1, policy_version 924053 (0.0011) [2023-12-26 22:05:36,408][105692] Updated weights for policy 0, policy_version 923967 (0.0006) [2023-12-26 22:05:36,430][105620] Updated weights for policy 1, policy_version 924063 (0.0011) [2023-12-26 22:05:36,472][105692] Updated weights for policy 0, policy_version 923977 (0.0006) [2023-12-26 22:05:36,493][105620] Updated weights for policy 1, policy_version 924073 (0.0011) [2023-12-26 22:05:37,059][105692] Updated weights for policy 0, policy_version 923987 (0.0007) [2023-12-26 22:05:37,105][105692] Updated weights for policy 0, policy_version 923997 (0.0006) [2023-12-26 22:05:37,153][105692] Updated weights for policy 0, policy_version 924007 (0.0005) [2023-12-26 22:05:37,187][105620] Updated weights for policy 1, policy_version 924083 (0.0011) [2023-12-26 22:05:37,239][105620] Updated weights for policy 1, policy_version 924093 (0.0010) [2023-12-26 22:05:37,301][105620] Updated weights for policy 1, policy_version 924103 (0.0010) [2023-12-26 22:05:37,876][105692] Updated weights for policy 0, policy_version 924017 (0.0010) [2023-12-26 22:05:37,938][105692] Updated weights for policy 0, policy_version 924027 (0.0010) [2023-12-26 22:05:37,987][105692] Updated weights for policy 0, policy_version 924037 (0.0010) [2023-12-26 22:05:38,056][105692] Updated weights for policy 0, policy_version 924047 (0.0010) [2023-12-26 22:05:38,093][105620] Updated weights for policy 1, policy_version 924113 (0.0010) [2023-12-26 22:05:38,149][105620] Updated weights for policy 1, policy_version 924123 (0.0008) [2023-12-26 22:05:38,198][105620] Updated weights for policy 1, policy_version 924133 (0.0009) [2023-12-26 22:05:38,250][105620] Updated weights for policy 1, policy_version 924143 (0.0008) [2023-12-26 22:05:38,833][105692] Updated weights for policy 0, policy_version 924057 (0.0010) [2023-12-26 22:05:38,886][105692] Updated weights for policy 0, policy_version 924067 (0.0011) [2023-12-26 22:05:38,947][105692] Updated weights for policy 0, policy_version 924077 (0.0011) [2023-12-26 22:05:38,955][105620] Updated weights for policy 1, policy_version 924153 (0.0010) [2023-12-26 22:05:39,026][105620] Updated weights for policy 1, policy_version 924163 (0.0011) [2023-12-26 22:05:39,084][105620] Updated weights for policy 1, policy_version 924173 (0.0007) [2023-12-26 22:05:39,626][105692] Updated weights for policy 0, policy_version 924087 (0.0007) [2023-12-26 22:05:39,683][105692] Updated weights for policy 0, policy_version 924097 (0.0006) [2023-12-26 22:05:39,753][105692] Updated weights for policy 0, policy_version 924107 (0.0006) [2023-12-26 22:05:39,861][105620] Updated weights for policy 1, policy_version 924184 (0.0008) [2023-12-26 22:05:39,927][105620] Updated weights for policy 1, policy_version 924194 (0.0010) [2023-12-26 22:05:39,995][105620] Updated weights for policy 1, policy_version 924204 (0.0011) [2023-12-26 22:05:40,407][105692] Updated weights for policy 0, policy_version 924117 (0.0008) [2023-12-26 22:05:40,471][105692] Updated weights for policy 0, policy_version 924127 (0.0011) [2023-12-26 22:05:40,528][105692] Updated weights for policy 0, policy_version 924137 (0.0010) [2023-12-26 22:05:40,731][105620] Updated weights for policy 1, policy_version 924214 (0.0009) [2023-12-26 22:05:40,787][105620] Updated weights for policy 1, policy_version 924224 (0.0008) [2023-12-26 22:05:40,844][105620] Updated weights for policy 1, policy_version 924234 (0.0008) [2023-12-26 22:05:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 18966.6). Total num frames: 473251840. Throughput: 0: 9592.0, 1: 9610.4. Samples: 473258956. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-26 22:05:41,062][104569] Avg episode reward: [(0, '9080.670'), (1, '7884.014')] [2023-12-26 22:05:41,314][105692] Updated weights for policy 0, policy_version 924147 (0.0009) [2023-12-26 22:05:41,375][105692] Updated weights for policy 0, policy_version 924157 (0.0009) [2023-12-26 22:05:41,437][105692] Updated weights for policy 0, policy_version 924167 (0.0009) [2023-12-26 22:05:41,638][105620] Updated weights for policy 1, policy_version 924244 (0.0009) [2023-12-26 22:05:41,703][105620] Updated weights for policy 1, policy_version 924254 (0.0010) [2023-12-26 22:05:41,772][105620] Updated weights for policy 1, policy_version 924264 (0.0010) [2023-12-26 22:05:42,131][105692] Updated weights for policy 0, policy_version 924177 (0.0009) [2023-12-26 22:05:42,191][105692] Updated weights for policy 0, policy_version 924187 (0.0006) [2023-12-26 22:05:42,255][105692] Updated weights for policy 0, policy_version 924197 (0.0007) [2023-12-26 22:05:42,323][105692] Updated weights for policy 0, policy_version 924207 (0.0008) [2023-12-26 22:05:42,533][105620] Updated weights for policy 1, policy_version 924274 (0.0010) [2023-12-26 22:05:42,600][105620] Updated weights for policy 1, policy_version 924284 (0.0010) [2023-12-26 22:05:42,675][105620] Updated weights for policy 1, policy_version 924295 (0.0009) [2023-12-26 22:05:43,074][105692] Updated weights for policy 0, policy_version 924217 (0.0009) [2023-12-26 22:05:43,137][105692] Updated weights for policy 0, policy_version 924227 (0.0008) [2023-12-26 22:05:43,206][105692] Updated weights for policy 0, policy_version 924237 (0.0006) [2023-12-26 22:05:43,475][105620] Updated weights for policy 1, policy_version 924305 (0.0009) [2023-12-26 22:05:43,536][105620] Updated weights for policy 1, policy_version 924315 (0.0009) [2023-12-26 22:05:43,598][105620] Updated weights for policy 1, policy_version 924325 (0.0008) [2023-12-26 22:05:43,669][105620] Updated weights for policy 1, policy_version 924335 (0.0007) [2023-12-26 22:05:43,790][105692] Updated weights for policy 0, policy_version 924247 (0.0007) [2023-12-26 22:05:43,854][105692] Updated weights for policy 0, policy_version 924257 (0.0009) [2023-12-26 22:05:43,921][105692] Updated weights for policy 0, policy_version 924267 (0.0007) [2023-12-26 22:05:44,372][105620] Updated weights for policy 1, policy_version 924345 (0.0008) [2023-12-26 22:05:44,423][105620] Updated weights for policy 1, policy_version 924355 (0.0009) [2023-12-26 22:05:44,469][105620] Updated weights for policy 1, policy_version 924365 (0.0009) [2023-12-26 22:05:44,622][105692] Updated weights for policy 0, policy_version 924277 (0.0008) [2023-12-26 22:05:44,685][105692] Updated weights for policy 0, policy_version 924287 (0.0009) [2023-12-26 22:05:44,747][105692] Updated weights for policy 0, policy_version 924297 (0.0009) [2023-12-26 22:05:45,242][105620] Updated weights for policy 1, policy_version 924375 (0.0010) [2023-12-26 22:05:45,309][105620] Updated weights for policy 1, policy_version 924385 (0.0011) [2023-12-26 22:05:45,376][105620] Updated weights for policy 1, policy_version 924395 (0.0010) [2023-12-26 22:05:45,570][105692] Updated weights for policy 0, policy_version 924307 (0.0009) [2023-12-26 22:05:45,627][105692] Updated weights for policy 0, policy_version 924317 (0.0008) [2023-12-26 22:05:45,687][105692] Updated weights for policy 0, policy_version 924327 (0.0008) [2023-12-26 22:05:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19114.6, 300 sec: 18966.6). Total num frames: 473341952. Throughput: 0: 9624.1, 1: 9585.3. Samples: 473314792. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-26 22:05:46,063][104569] Avg episode reward: [(0, '9170.938'), (1, '8394.820')] [2023-12-26 22:05:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000924336_236666880.pth... [2023-12-26 22:05:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000923216_236380160.pth [2023-12-26 22:05:46,096][105620] Updated weights for policy 1, policy_version 924405 (0.0010) [2023-12-26 22:05:46,158][105620] Updated weights for policy 1, policy_version 924415 (0.0010) [2023-12-26 22:05:46,223][105620] Updated weights for policy 1, policy_version 924425 (0.0010) [2023-12-26 22:05:46,261][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000924432_236683264.pth... [2023-12-26 22:05:46,264][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000923280_236388352.pth [2023-12-26 22:05:46,472][105692] Updated weights for policy 0, policy_version 924337 (0.0009) [2023-12-26 22:05:46,522][105692] Updated weights for policy 0, policy_version 924347 (0.0008) [2023-12-26 22:05:46,578][105692] Updated weights for policy 0, policy_version 924357 (0.0006) [2023-12-26 22:05:46,632][105692] Updated weights for policy 0, policy_version 924367 (0.0006) [2023-12-26 22:05:46,963][105620] Updated weights for policy 1, policy_version 924435 (0.0010) [2023-12-26 22:05:47,023][105620] Updated weights for policy 1, policy_version 924445 (0.0007) [2023-12-26 22:05:47,077][105620] Updated weights for policy 1, policy_version 924455 (0.0007) [2023-12-26 22:05:47,349][105692] Updated weights for policy 0, policy_version 924377 (0.0007) [2023-12-26 22:05:47,411][105692] Updated weights for policy 0, policy_version 924387 (0.0006) [2023-12-26 22:05:47,476][105692] Updated weights for policy 0, policy_version 924397 (0.0010) [2023-12-26 22:05:47,798][105620] Updated weights for policy 1, policy_version 924465 (0.0007) [2023-12-26 22:05:47,849][105620] Updated weights for policy 1, policy_version 924475 (0.0005) [2023-12-26 22:05:47,903][105620] Updated weights for policy 1, policy_version 924485 (0.0005) [2023-12-26 22:05:47,951][105620] Updated weights for policy 1, policy_version 924495 (0.0005) [2023-12-26 22:05:48,039][105692] Updated weights for policy 0, policy_version 924407 (0.0007) [2023-12-26 22:05:48,099][105692] Updated weights for policy 0, policy_version 924417 (0.0007) [2023-12-26 22:05:48,170][105692] Updated weights for policy 0, policy_version 924427 (0.0006) [2023-12-26 22:05:48,561][105620] Updated weights for policy 1, policy_version 924505 (0.0008) [2023-12-26 22:05:48,622][105620] Updated weights for policy 1, policy_version 924515 (0.0008) [2023-12-26 22:05:48,692][105620] Updated weights for policy 1, policy_version 924525 (0.0007) [2023-12-26 22:05:48,752][105692] Updated weights for policy 0, policy_version 924437 (0.0008) [2023-12-26 22:05:48,813][105692] Updated weights for policy 0, policy_version 924447 (0.0009) [2023-12-26 22:05:48,872][105692] Updated weights for policy 0, policy_version 924457 (0.0010) [2023-12-26 22:05:49,404][105620] Updated weights for policy 1, policy_version 924535 (0.0010) [2023-12-26 22:05:49,460][105620] Updated weights for policy 1, policy_version 924545 (0.0010) [2023-12-26 22:05:49,519][105620] Updated weights for policy 1, policy_version 924555 (0.0010) [2023-12-26 22:05:49,604][105692] Updated weights for policy 0, policy_version 924467 (0.0009) [2023-12-26 22:05:49,652][105692] Updated weights for policy 0, policy_version 924477 (0.0008) [2023-12-26 22:05:49,698][105692] Updated weights for policy 0, policy_version 924487 (0.0008) [2023-12-26 22:05:50,299][105620] Updated weights for policy 1, policy_version 924565 (0.0010) [2023-12-26 22:05:50,359][105620] Updated weights for policy 1, policy_version 924575 (0.0010) [2023-12-26 22:05:50,415][105620] Updated weights for policy 1, policy_version 924585 (0.0009) [2023-12-26 22:05:50,497][105692] Updated weights for policy 0, policy_version 924497 (0.0008) [2023-12-26 22:05:50,556][105692] Updated weights for policy 0, policy_version 924507 (0.0008) [2023-12-26 22:05:50,622][105692] Updated weights for policy 0, policy_version 924517 (0.0007) [2023-12-26 22:05:50,683][105692] Updated weights for policy 0, policy_version 924527 (0.0008) [2023-12-26 22:05:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.7, 300 sec: 18994.3). Total num frames: 473440256. Throughput: 0: 9712.1, 1: 9624.5. Samples: 473432356. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-26 22:05:51,062][104569] Avg episode reward: [(0, '9171.112'), (1, '8136.281')] [2023-12-26 22:05:51,183][105620] Updated weights for policy 1, policy_version 924595 (0.0010) [2023-12-26 22:05:51,243][105620] Updated weights for policy 1, policy_version 924605 (0.0010) [2023-12-26 22:05:51,311][105620] Updated weights for policy 1, policy_version 924615 (0.0011) [2023-12-26 22:05:51,502][105692] Updated weights for policy 0, policy_version 924537 (0.0009) [2023-12-26 22:05:51,559][105692] Updated weights for policy 0, policy_version 924547 (0.0008) [2023-12-26 22:05:51,621][105692] Updated weights for policy 0, policy_version 924557 (0.0008) [2023-12-26 22:05:52,116][105620] Updated weights for policy 1, policy_version 924625 (0.0011) [2023-12-26 22:05:52,179][105620] Updated weights for policy 1, policy_version 924635 (0.0010) [2023-12-26 22:05:52,242][105620] Updated weights for policy 1, policy_version 924645 (0.0010) [2023-12-26 22:05:52,310][105620] Updated weights for policy 1, policy_version 924655 (0.0011) [2023-12-26 22:05:52,414][105692] Updated weights for policy 0, policy_version 924567 (0.0008) [2023-12-26 22:05:52,474][105692] Updated weights for policy 0, policy_version 924577 (0.0009) [2023-12-26 22:05:52,542][105692] Updated weights for policy 0, policy_version 924587 (0.0008) [2023-12-26 22:05:52,568][105585] KL-divergence is very high: 104.1500 [2023-12-26 22:05:53,057][105620] Updated weights for policy 1, policy_version 924665 (0.0010) [2023-12-26 22:05:53,108][105620] Updated weights for policy 1, policy_version 924675 (0.0010) [2023-12-26 22:05:53,163][105620] Updated weights for policy 1, policy_version 924685 (0.0010) [2023-12-26 22:05:53,294][105692] Updated weights for policy 0, policy_version 924597 (0.0008) [2023-12-26 22:05:53,338][105692] Updated weights for policy 0, policy_version 924607 (0.0008) [2023-12-26 22:05:53,382][105692] Updated weights for policy 0, policy_version 924617 (0.0008) [2023-12-26 22:05:53,932][105620] Updated weights for policy 1, policy_version 924695 (0.0010) [2023-12-26 22:05:53,980][105620] Updated weights for policy 1, policy_version 924705 (0.0010) [2023-12-26 22:05:54,042][105620] Updated weights for policy 1, policy_version 924715 (0.0010) [2023-12-26 22:05:54,167][105692] Updated weights for policy 0, policy_version 924627 (0.0008) [2023-12-26 22:05:54,226][105692] Updated weights for policy 0, policy_version 924637 (0.0008) [2023-12-26 22:05:54,284][105692] Updated weights for policy 0, policy_version 924647 (0.0008) [2023-12-26 22:05:54,789][105620] Updated weights for policy 1, policy_version 924725 (0.0010) [2023-12-26 22:05:54,859][105620] Updated weights for policy 1, policy_version 924735 (0.0009) [2023-12-26 22:05:54,923][105620] Updated weights for policy 1, policy_version 924745 (0.0008) [2023-12-26 22:05:55,062][105692] Updated weights for policy 0, policy_version 924657 (0.0009) [2023-12-26 22:05:55,124][105692] Updated weights for policy 0, policy_version 924667 (0.0006) [2023-12-26 22:05:55,183][105692] Updated weights for policy 0, policy_version 924677 (0.0009) [2023-12-26 22:05:55,240][105692] Updated weights for policy 0, policy_version 924687 (0.0010) [2023-12-26 22:05:55,535][105620] Updated weights for policy 1, policy_version 924755 (0.0008) [2023-12-26 22:05:55,581][105620] Updated weights for policy 1, policy_version 924765 (0.0008) [2023-12-26 22:05:55,627][105620] Updated weights for policy 1, policy_version 924775 (0.0008) [2023-12-26 22:05:56,029][105692] Updated weights for policy 0, policy_version 924697 (0.0009) [2023-12-26 22:05:56,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19114.7, 300 sec: 18994.3). Total num frames: 473530368. Throughput: 0: 9572.1, 1: 9583.3. Samples: 473542480. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-26 22:05:56,062][104569] Avg episode reward: [(0, '9084.855'), (1, '7697.018')] [2023-12-26 22:05:56,087][105692] Updated weights for policy 0, policy_version 924707 (0.0009) [2023-12-26 22:05:56,146][105692] Updated weights for policy 0, policy_version 924717 (0.0009) [2023-12-26 22:05:56,337][105620] Updated weights for policy 1, policy_version 924785 (0.0008) [2023-12-26 22:05:56,401][105620] Updated weights for policy 1, policy_version 924795 (0.0005) [2023-12-26 22:05:56,458][105620] Updated weights for policy 1, policy_version 924805 (0.0009) [2023-12-26 22:05:56,520][105620] Updated weights for policy 1, policy_version 924815 (0.0009) [2023-12-26 22:05:56,937][105692] Updated weights for policy 0, policy_version 924727 (0.0009) [2023-12-26 22:05:56,987][105692] Updated weights for policy 0, policy_version 924737 (0.0008) [2023-12-26 22:05:57,041][105692] Updated weights for policy 0, policy_version 924747 (0.0009) [2023-12-26 22:05:57,218][105620] Updated weights for policy 1, policy_version 924825 (0.0008) [2023-12-26 22:05:57,269][105620] Updated weights for policy 1, policy_version 924835 (0.0009) [2023-12-26 22:05:57,323][105620] Updated weights for policy 1, policy_version 924845 (0.0008) [2023-12-26 22:05:57,787][105692] Updated weights for policy 0, policy_version 924757 (0.0009) [2023-12-26 22:05:57,845][105692] Updated weights for policy 0, policy_version 924767 (0.0010) [2023-12-26 22:05:57,912][105692] Updated weights for policy 0, policy_version 924778 (0.0010) [2023-12-26 22:05:57,946][105620] Updated weights for policy 1, policy_version 924855 (0.0006) [2023-12-26 22:05:58,008][105620] Updated weights for policy 1, policy_version 924865 (0.0005) [2023-12-26 22:05:58,066][105620] Updated weights for policy 1, policy_version 924875 (0.0005) [2023-12-26 22:05:58,735][105692] Updated weights for policy 0, policy_version 924788 (0.0009) [2023-12-26 22:05:58,812][105692] Updated weights for policy 0, policy_version 924798 (0.0009) [2023-12-26 22:05:58,841][105620] Updated weights for policy 1, policy_version 924885 (0.0007) [2023-12-26 22:05:58,885][105692] Updated weights for policy 0, policy_version 924808 (0.0008) [2023-12-26 22:05:58,916][105620] Updated weights for policy 1, policy_version 924895 (0.0011) [2023-12-26 22:05:58,979][105620] Updated weights for policy 1, policy_version 924905 (0.0008) [2023-12-26 22:05:59,677][105692] Updated weights for policy 0, policy_version 924818 (0.0008) [2023-12-26 22:05:59,720][105620] Updated weights for policy 1, policy_version 924915 (0.0008) [2023-12-26 22:05:59,735][105692] Updated weights for policy 0, policy_version 924828 (0.0007) [2023-12-26 22:05:59,781][105620] Updated weights for policy 1, policy_version 924925 (0.0009) [2023-12-26 22:05:59,807][105692] Updated weights for policy 0, policy_version 924838 (0.0005) [2023-12-26 22:05:59,848][105620] Updated weights for policy 1, policy_version 924935 (0.0009) [2023-12-26 22:05:59,866][105692] Updated weights for policy 0, policy_version 924848 (0.0008) [2023-12-26 22:06:00,474][105692] Updated weights for policy 0, policy_version 924858 (0.0009) [2023-12-26 22:06:00,536][105692] Updated weights for policy 0, policy_version 924868 (0.0009) [2023-12-26 22:06:00,593][105692] Updated weights for policy 0, policy_version 924878 (0.0009) [2023-12-26 22:06:00,653][105620] Updated weights for policy 1, policy_version 924945 (0.0008) [2023-12-26 22:06:00,699][105620] Updated weights for policy 1, policy_version 924955 (0.0008) [2023-12-26 22:06:00,748][105620] Updated weights for policy 1, policy_version 924965 (0.0009) [2023-12-26 22:06:00,809][105620] Updated weights for policy 1, policy_version 924975 (0.0009) [2023-12-26 22:06:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 18994.3). Total num frames: 473628672. Throughput: 0: 9543.8, 1: 9666.2. Samples: 473600084. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-26 22:06:01,063][104569] Avg episode reward: [(0, '8990.954'), (1, '7683.660')] [2023-12-26 22:06:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000924880_236806144.pth... [2023-12-26 22:06:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000924976_236822528.pth... [2023-12-26 22:06:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000923760_236519424.pth [2023-12-26 22:06:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000923856_236535808.pth [2023-12-26 22:06:01,202][105692] Updated weights for policy 0, policy_version 924888 (0.0006) [2023-12-26 22:06:01,265][105692] Updated weights for policy 0, policy_version 924898 (0.0009) [2023-12-26 22:06:01,322][105692] Updated weights for policy 0, policy_version 924908 (0.0006) [2023-12-26 22:06:01,635][105620] Updated weights for policy 1, policy_version 924985 (0.0010) [2023-12-26 22:06:01,697][105620] Updated weights for policy 1, policy_version 924995 (0.0009) [2023-12-26 22:06:01,764][105620] Updated weights for policy 1, policy_version 925005 (0.0009) [2023-12-26 22:06:02,023][105692] Updated weights for policy 0, policy_version 924918 (0.0007) [2023-12-26 22:06:02,081][105692] Updated weights for policy 0, policy_version 924928 (0.0009) [2023-12-26 22:06:02,132][105692] Updated weights for policy 0, policy_version 924938 (0.0009) [2023-12-26 22:06:02,469][105620] Updated weights for policy 1, policy_version 925015 (0.0008) [2023-12-26 22:06:02,534][105620] Updated weights for policy 1, policy_version 925025 (0.0009) [2023-12-26 22:06:02,586][105620] Updated weights for policy 1, policy_version 925035 (0.0009) [2023-12-26 22:06:02,913][105692] Updated weights for policy 0, policy_version 924949 (0.0010) [2023-12-26 22:06:02,958][105692] Updated weights for policy 0, policy_version 924959 (0.0007) [2023-12-26 22:06:03,003][105692] Updated weights for policy 0, policy_version 924969 (0.0005) [2023-12-26 22:06:03,417][105620] Updated weights for policy 1, policy_version 925046 (0.0009) [2023-12-26 22:06:03,469][105620] Updated weights for policy 1, policy_version 925056 (0.0010) [2023-12-26 22:06:03,521][105620] Updated weights for policy 1, policy_version 925066 (0.0009) [2023-12-26 22:06:03,600][105692] Updated weights for policy 0, policy_version 924979 (0.0005) [2023-12-26 22:06:03,657][105692] Updated weights for policy 0, policy_version 924989 (0.0005) [2023-12-26 22:06:03,715][105692] Updated weights for policy 0, policy_version 924999 (0.0005) [2023-12-26 22:06:04,280][105620] Updated weights for policy 1, policy_version 925076 (0.0010) [2023-12-26 22:06:04,348][105620] Updated weights for policy 1, policy_version 925086 (0.0011) [2023-12-26 22:06:04,367][105692] Updated weights for policy 0, policy_version 925009 (0.0006) [2023-12-26 22:06:04,414][105620] Updated weights for policy 1, policy_version 925096 (0.0008) [2023-12-26 22:06:04,433][105692] Updated weights for policy 0, policy_version 925019 (0.0007) [2023-12-26 22:06:04,504][105692] Updated weights for policy 0, policy_version 925029 (0.0006) [2023-12-26 22:06:04,569][105692] Updated weights for policy 0, policy_version 925039 (0.0010) [2023-12-26 22:06:05,065][105620] Updated weights for policy 1, policy_version 925106 (0.0006) [2023-12-26 22:06:05,117][105620] Updated weights for policy 1, policy_version 925116 (0.0008) [2023-12-26 22:06:05,170][105620] Updated weights for policy 1, policy_version 925126 (0.0008) [2023-12-26 22:06:05,225][105620] Updated weights for policy 1, policy_version 925136 (0.0009) [2023-12-26 22:06:05,300][105692] Updated weights for policy 0, policy_version 925049 (0.0009) [2023-12-26 22:06:05,354][105692] Updated weights for policy 0, policy_version 925059 (0.0009) [2023-12-26 22:06:05,412][105692] Updated weights for policy 0, policy_version 925069 (0.0008) [2023-12-26 22:06:05,907][105620] Updated weights for policy 1, policy_version 925146 (0.0005) [2023-12-26 22:06:05,967][105620] Updated weights for policy 1, policy_version 925156 (0.0007) [2023-12-26 22:06:06,031][105620] Updated weights for policy 1, policy_version 925166 (0.0008) [2023-12-26 22:06:06,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19251.2, 300 sec: 19022.1). Total num frames: 473726976. Throughput: 0: 9579.9, 1: 9560.7. Samples: 473714812. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-26 22:06:06,063][104569] Avg episode reward: [(0, '9167.160'), (1, '8218.387')] [2023-12-26 22:06:06,123][105692] Updated weights for policy 0, policy_version 925079 (0.0010) [2023-12-26 22:06:06,186][105692] Updated weights for policy 0, policy_version 925089 (0.0007) [2023-12-26 22:06:06,243][105692] Updated weights for policy 0, policy_version 925099 (0.0009) [2023-12-26 22:06:06,681][105620] Updated weights for policy 1, policy_version 925176 (0.0009) [2023-12-26 22:06:06,755][105620] Updated weights for policy 1, policy_version 925186 (0.0010) [2023-12-26 22:06:06,820][105620] Updated weights for policy 1, policy_version 925196 (0.0009) [2023-12-26 22:06:06,976][105692] Updated weights for policy 0, policy_version 925109 (0.0010) [2023-12-26 22:06:07,039][105692] Updated weights for policy 0, policy_version 925119 (0.0009) [2023-12-26 22:06:07,090][105692] Updated weights for policy 0, policy_version 925129 (0.0009) [2023-12-26 22:06:07,593][105620] Updated weights for policy 1, policy_version 925206 (0.0008) [2023-12-26 22:06:07,659][105620] Updated weights for policy 1, policy_version 925216 (0.0009) [2023-12-26 22:06:07,724][105620] Updated weights for policy 1, policy_version 925226 (0.0008) [2023-12-26 22:06:07,835][105692] Updated weights for policy 0, policy_version 925139 (0.0009) [2023-12-26 22:06:07,894][105692] Updated weights for policy 0, policy_version 925149 (0.0009) [2023-12-26 22:06:07,964][105692] Updated weights for policy 0, policy_version 925159 (0.0009) [2023-12-26 22:06:08,512][105620] Updated weights for policy 1, policy_version 925236 (0.0007) [2023-12-26 22:06:08,576][105620] Updated weights for policy 1, policy_version 925246 (0.0008) [2023-12-26 22:06:08,633][105620] Updated weights for policy 1, policy_version 925256 (0.0008) [2023-12-26 22:06:08,714][105692] Updated weights for policy 0, policy_version 925169 (0.0009) [2023-12-26 22:06:08,774][105692] Updated weights for policy 0, policy_version 925179 (0.0009) [2023-12-26 22:06:08,838][105692] Updated weights for policy 0, policy_version 925189 (0.0008) [2023-12-26 22:06:08,902][105692] Updated weights for policy 0, policy_version 925199 (0.0008) [2023-12-26 22:06:09,347][105620] Updated weights for policy 1, policy_version 925266 (0.0009) [2023-12-26 22:06:09,417][105620] Updated weights for policy 1, policy_version 925276 (0.0010) [2023-12-26 22:06:09,488][105620] Updated weights for policy 1, policy_version 925286 (0.0008) [2023-12-26 22:06:09,556][105620] Updated weights for policy 1, policy_version 925296 (0.0008) [2023-12-26 22:06:09,663][105692] Updated weights for policy 0, policy_version 925209 (0.0008) [2023-12-26 22:06:09,724][105692] Updated weights for policy 0, policy_version 925219 (0.0008) [2023-12-26 22:06:09,782][105692] Updated weights for policy 0, policy_version 925229 (0.0009) [2023-12-26 22:06:10,227][105620] Updated weights for policy 1, policy_version 925306 (0.0008) [2023-12-26 22:06:10,282][105620] Updated weights for policy 1, policy_version 925317 (0.0009) [2023-12-26 22:06:10,342][105620] Updated weights for policy 1, policy_version 925327 (0.0006) [2023-12-26 22:06:10,583][105692] Updated weights for policy 0, policy_version 925239 (0.0009) [2023-12-26 22:06:10,642][105692] Updated weights for policy 0, policy_version 925249 (0.0009) [2023-12-26 22:06:10,690][105692] Updated weights for policy 0, policy_version 925259 (0.0009) [2023-12-26 22:06:11,028][105620] Updated weights for policy 1, policy_version 925337 (0.0008) [2023-12-26 22:06:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 18994.3). Total num frames: 473817088. Throughput: 0: 9580.7, 1: 9543.4. Samples: 473828872. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-26 22:06:11,062][104569] Avg episode reward: [(0, '9258.453'), (1, '8215.076')] [2023-12-26 22:06:11,095][105620] Updated weights for policy 1, policy_version 925347 (0.0009) [2023-12-26 22:06:11,165][105620] Updated weights for policy 1, policy_version 925357 (0.0008) [2023-12-26 22:06:11,553][105692] Updated weights for policy 0, policy_version 925269 (0.0009) [2023-12-26 22:06:11,610][105692] Updated weights for policy 0, policy_version 925279 (0.0009) [2023-12-26 22:06:11,675][105692] Updated weights for policy 0, policy_version 925289 (0.0009) [2023-12-26 22:06:11,903][105620] Updated weights for policy 1, policy_version 925367 (0.0008) [2023-12-26 22:06:11,962][105620] Updated weights for policy 1, policy_version 925377 (0.0009) [2023-12-26 22:06:12,030][105620] Updated weights for policy 1, policy_version 925387 (0.0009) [2023-12-26 22:06:12,473][105692] Updated weights for policy 0, policy_version 925299 (0.0008) [2023-12-26 22:06:12,528][105692] Updated weights for policy 0, policy_version 925309 (0.0008) [2023-12-26 22:06:12,587][105692] Updated weights for policy 0, policy_version 925319 (0.0009) [2023-12-26 22:06:12,827][105620] Updated weights for policy 1, policy_version 925397 (0.0008) [2023-12-26 22:06:12,881][105620] Updated weights for policy 1, policy_version 925407 (0.0009) [2023-12-26 22:06:12,945][105620] Updated weights for policy 1, policy_version 925417 (0.0009) [2023-12-26 22:06:13,333][105692] Updated weights for policy 0, policy_version 925329 (0.0009) [2023-12-26 22:06:13,388][105692] Updated weights for policy 0, policy_version 925339 (0.0008) [2023-12-26 22:06:13,448][105692] Updated weights for policy 0, policy_version 925349 (0.0008) [2023-12-26 22:06:13,504][105692] Updated weights for policy 0, policy_version 925359 (0.0008) [2023-12-26 22:06:13,707][105620] Updated weights for policy 1, policy_version 925427 (0.0009) [2023-12-26 22:06:13,761][105620] Updated weights for policy 1, policy_version 925437 (0.0008) [2023-12-26 22:06:13,815][105620] Updated weights for policy 1, policy_version 925447 (0.0009) [2023-12-26 22:06:14,187][105692] Updated weights for policy 0, policy_version 925369 (0.0010) [2023-12-26 22:06:14,237][105692] Updated weights for policy 0, policy_version 925379 (0.0009) [2023-12-26 22:06:14,295][105692] Updated weights for policy 0, policy_version 925389 (0.0009) [2023-12-26 22:06:14,584][105620] Updated weights for policy 1, policy_version 925457 (0.0010) [2023-12-26 22:06:14,639][105620] Updated weights for policy 1, policy_version 925467 (0.0008) [2023-12-26 22:06:14,688][105620] Updated weights for policy 1, policy_version 925477 (0.0009) [2023-12-26 22:06:14,748][105620] Updated weights for policy 1, policy_version 925487 (0.0008) [2023-12-26 22:06:15,124][105692] Updated weights for policy 0, policy_version 925399 (0.0009) [2023-12-26 22:06:15,194][105692] Updated weights for policy 0, policy_version 925409 (0.0009) [2023-12-26 22:06:15,260][105692] Updated weights for policy 0, policy_version 925419 (0.0009) [2023-12-26 22:06:15,452][105620] Updated weights for policy 1, policy_version 925497 (0.0009) [2023-12-26 22:06:15,516][105620] Updated weights for policy 1, policy_version 925507 (0.0010) [2023-12-26 22:06:15,569][105620] Updated weights for policy 1, policy_version 925517 (0.0007) [2023-12-26 22:06:16,037][105692] Updated weights for policy 0, policy_version 925429 (0.0009) [2023-12-26 22:06:16,062][104569] Fps is (10 sec: 18022.6, 60 sec: 18978.2, 300 sec: 18994.3). Total num frames: 473907200. Throughput: 0: 9534.2, 1: 9501.8. Samples: 473881932. Policy #0 lag: (min: 31.0, avg: 31.6, max: 54.0) [2023-12-26 22:06:16,062][104569] Avg episode reward: [(0, '8896.560'), (1, '8127.156')] [2023-12-26 22:06:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000925520_236961792.pth... [2023-12-26 22:06:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000924432_236683264.pth [2023-12-26 22:06:16,086][105692] Updated weights for policy 0, policy_version 925439 (0.0008) [2023-12-26 22:06:16,133][105692] Updated weights for policy 0, policy_version 925449 (0.0008) [2023-12-26 22:06:16,165][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000925456_236953600.pth... [2023-12-26 22:06:16,169][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000924336_236666880.pth [2023-12-26 22:06:16,235][105620] Updated weights for policy 1, policy_version 925527 (0.0008) [2023-12-26 22:06:16,285][105620] Updated weights for policy 1, policy_version 925537 (0.0006) [2023-12-26 22:06:16,338][105620] Updated weights for policy 1, policy_version 925547 (0.0009) [2023-12-26 22:06:16,882][105692] Updated weights for policy 0, policy_version 925459 (0.0009) [2023-12-26 22:06:16,946][105692] Updated weights for policy 0, policy_version 925469 (0.0009) [2023-12-26 22:06:17,009][105692] Updated weights for policy 0, policy_version 925479 (0.0009) [2023-12-26 22:06:17,066][105620] Updated weights for policy 1, policy_version 925557 (0.0009) [2023-12-26 22:06:17,113][105620] Updated weights for policy 1, policy_version 925567 (0.0009) [2023-12-26 22:06:17,161][105620] Updated weights for policy 1, policy_version 925577 (0.0009) [2023-12-26 22:06:17,748][105692] Updated weights for policy 0, policy_version 925489 (0.0008) [2023-12-26 22:06:17,807][105692] Updated weights for policy 0, policy_version 925499 (0.0009) [2023-12-26 22:06:17,868][105692] Updated weights for policy 0, policy_version 925509 (0.0009) [2023-12-26 22:06:17,918][105692] Updated weights for policy 0, policy_version 925519 (0.0009) [2023-12-26 22:06:17,945][105620] Updated weights for policy 1, policy_version 925587 (0.0008) [2023-12-26 22:06:18,004][105620] Updated weights for policy 1, policy_version 925597 (0.0009) [2023-12-26 22:06:18,059][105620] Updated weights for policy 1, policy_version 925607 (0.0009) [2023-12-26 22:06:18,657][105620] Updated weights for policy 1, policy_version 925617 (0.0005) [2023-12-26 22:06:18,694][105692] Updated weights for policy 0, policy_version 925529 (0.0007) [2023-12-26 22:06:18,727][105620] Updated weights for policy 1, policy_version 925627 (0.0008) [2023-12-26 22:06:18,755][105692] Updated weights for policy 0, policy_version 925539 (0.0007) [2023-12-26 22:06:18,788][105620] Updated weights for policy 1, policy_version 925637 (0.0009) [2023-12-26 22:06:18,816][105692] Updated weights for policy 0, policy_version 925549 (0.0006) [2023-12-26 22:06:18,850][105620] Updated weights for policy 1, policy_version 925647 (0.0008) [2023-12-26 22:06:19,545][105692] Updated weights for policy 0, policy_version 925559 (0.0009) [2023-12-26 22:06:19,600][105692] Updated weights for policy 0, policy_version 925569 (0.0007) [2023-12-26 22:06:19,602][105620] Updated weights for policy 1, policy_version 925657 (0.0007) [2023-12-26 22:06:19,660][105620] Updated weights for policy 1, policy_version 925667 (0.0007) [2023-12-26 22:06:19,662][105692] Updated weights for policy 0, policy_version 925579 (0.0006) [2023-12-26 22:06:19,717][105620] Updated weights for policy 1, policy_version 925677 (0.0009) [2023-12-26 22:06:20,321][105692] Updated weights for policy 0, policy_version 925589 (0.0008) [2023-12-26 22:06:20,370][105692] Updated weights for policy 0, policy_version 925599 (0.0009) [2023-12-26 22:06:20,421][105692] Updated weights for policy 0, policy_version 925609 (0.0007) [2023-12-26 22:06:20,610][105620] Updated weights for policy 1, policy_version 925687 (0.0010) [2023-12-26 22:06:20,680][105620] Updated weights for policy 1, policy_version 925697 (0.0010) [2023-12-26 22:06:20,733][105620] Updated weights for policy 1, policy_version 925707 (0.0009) [2023-12-26 22:06:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 18978.1, 300 sec: 19022.1). Total num frames: 474005504. Throughput: 0: 9539.6, 1: 9479.8. Samples: 473996128. Policy #0 lag: (min: 31.0, avg: 31.6, max: 54.0) [2023-12-26 22:06:21,062][104569] Avg episode reward: [(0, '8808.353'), (1, '7965.671')] [2023-12-26 22:06:21,084][105692] Updated weights for policy 0, policy_version 925619 (0.0007) [2023-12-26 22:06:21,149][105692] Updated weights for policy 0, policy_version 925629 (0.0008) [2023-12-26 22:06:21,208][105692] Updated weights for policy 0, policy_version 925639 (0.0008) [2023-12-26 22:06:21,602][105620] Updated weights for policy 1, policy_version 925717 (0.0009) [2023-12-26 22:06:21,671][105620] Updated weights for policy 1, policy_version 925727 (0.0010) [2023-12-26 22:06:21,735][105620] Updated weights for policy 1, policy_version 925737 (0.0009) [2023-12-26 22:06:21,900][105692] Updated weights for policy 0, policy_version 925649 (0.0008) [2023-12-26 22:06:21,963][105692] Updated weights for policy 0, policy_version 925659 (0.0009) [2023-12-26 22:06:22,026][105692] Updated weights for policy 0, policy_version 925669 (0.0007) [2023-12-26 22:06:22,083][105692] Updated weights for policy 0, policy_version 925679 (0.0009) [2023-12-26 22:06:22,588][105620] Updated weights for policy 1, policy_version 925747 (0.0010) [2023-12-26 22:06:22,655][105620] Updated weights for policy 1, policy_version 925757 (0.0009) [2023-12-26 22:06:22,726][105620] Updated weights for policy 1, policy_version 925767 (0.0010) [2023-12-26 22:06:22,827][105692] Updated weights for policy 0, policy_version 925689 (0.0009) [2023-12-26 22:06:22,883][105692] Updated weights for policy 0, policy_version 925699 (0.0009) [2023-12-26 22:06:22,957][105692] Updated weights for policy 0, policy_version 925709 (0.0009) [2023-12-26 22:06:23,477][105620] Updated weights for policy 1, policy_version 925777 (0.0008) [2023-12-26 22:06:23,529][105620] Updated weights for policy 1, policy_version 925787 (0.0010) [2023-12-26 22:06:23,580][105620] Updated weights for policy 1, policy_version 925797 (0.0010) [2023-12-26 22:06:23,638][105620] Updated weights for policy 1, policy_version 925807 (0.0010) [2023-12-26 22:06:23,718][105692] Updated weights for policy 0, policy_version 925719 (0.0007) [2023-12-26 22:06:23,772][105692] Updated weights for policy 0, policy_version 925729 (0.0009) [2023-12-26 22:06:23,831][105692] Updated weights for policy 0, policy_version 925739 (0.0011) [2023-12-26 22:06:24,342][105620] Updated weights for policy 1, policy_version 925817 (0.0010) [2023-12-26 22:06:24,408][105620] Updated weights for policy 1, policy_version 925827 (0.0010) [2023-12-26 22:06:24,469][105620] Updated weights for policy 1, policy_version 925837 (0.0010) [2023-12-26 22:06:24,514][105692] Updated weights for policy 0, policy_version 925749 (0.0009) [2023-12-26 22:06:24,575][105692] Updated weights for policy 0, policy_version 925759 (0.0005) [2023-12-26 22:06:24,640][105692] Updated weights for policy 0, policy_version 925769 (0.0009) [2023-12-26 22:06:25,047][105620] Updated weights for policy 1, policy_version 925847 (0.0010) [2023-12-26 22:06:25,116][105620] Updated weights for policy 1, policy_version 925857 (0.0010) [2023-12-26 22:06:25,184][105620] Updated weights for policy 1, policy_version 925867 (0.0010) [2023-12-26 22:06:25,361][105692] Updated weights for policy 0, policy_version 925779 (0.0009) [2023-12-26 22:06:25,413][105692] Updated weights for policy 0, policy_version 925789 (0.0008) [2023-12-26 22:06:25,465][105692] Updated weights for policy 0, policy_version 925799 (0.0007) [2023-12-26 22:06:25,885][105620] Updated weights for policy 1, policy_version 925877 (0.0010) [2023-12-26 22:06:25,947][105620] Updated weights for policy 1, policy_version 925887 (0.0010) [2023-12-26 22:06:26,012][105620] Updated weights for policy 1, policy_version 925897 (0.0010) [2023-12-26 22:06:26,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19114.7, 300 sec: 19022.1). Total num frames: 474103808. Throughput: 0: 9471.2, 1: 9447.1. Samples: 474110284. Policy #0 lag: (min: 31.0, avg: 31.6, max: 54.0) [2023-12-26 22:06:26,063][104569] Avg episode reward: [(0, '9079.694'), (1, '7223.709')] [2023-12-26 22:06:26,215][105692] Updated weights for policy 0, policy_version 925809 (0.0007) [2023-12-26 22:06:26,284][105692] Updated weights for policy 0, policy_version 925819 (0.0007) [2023-12-26 22:06:26,333][105692] Updated weights for policy 0, policy_version 925829 (0.0007) [2023-12-26 22:06:26,384][105692] Updated weights for policy 0, policy_version 925839 (0.0009) [2023-12-26 22:06:26,655][105620] Updated weights for policy 1, policy_version 925907 (0.0010) [2023-12-26 22:06:26,709][105620] Updated weights for policy 1, policy_version 925917 (0.0010) [2023-12-26 22:06:26,774][105620] Updated weights for policy 1, policy_version 925927 (0.0009) [2023-12-26 22:06:26,927][105692] Updated weights for policy 0, policy_version 925849 (0.0005) [2023-12-26 22:06:26,978][105692] Updated weights for policy 0, policy_version 925859 (0.0005) [2023-12-26 22:06:27,029][105692] Updated weights for policy 0, policy_version 925869 (0.0009) [2023-12-26 22:06:27,457][105620] Updated weights for policy 1, policy_version 925937 (0.0010) [2023-12-26 22:06:27,514][105620] Updated weights for policy 1, policy_version 925947 (0.0010) [2023-12-26 22:06:27,567][105620] Updated weights for policy 1, policy_version 925957 (0.0010) [2023-12-26 22:06:27,623][105620] Updated weights for policy 1, policy_version 925967 (0.0008) [2023-12-26 22:06:27,726][105692] Updated weights for policy 0, policy_version 925879 (0.0009) [2023-12-26 22:06:27,779][105692] Updated weights for policy 0, policy_version 925889 (0.0008) [2023-12-26 22:06:27,831][105692] Updated weights for policy 0, policy_version 925899 (0.0009) [2023-12-26 22:06:28,341][105620] Updated weights for policy 1, policy_version 925977 (0.0007) [2023-12-26 22:06:28,405][105620] Updated weights for policy 1, policy_version 925987 (0.0007) [2023-12-26 22:06:28,466][105620] Updated weights for policy 1, policy_version 925997 (0.0009) [2023-12-26 22:06:28,655][105692] Updated weights for policy 0, policy_version 925909 (0.0009) [2023-12-26 22:06:28,720][105692] Updated weights for policy 0, policy_version 925919 (0.0009) [2023-12-26 22:06:28,775][105692] Updated weights for policy 0, policy_version 925929 (0.0009) [2023-12-26 22:06:29,105][105620] Updated weights for policy 1, policy_version 926007 (0.0009) [2023-12-26 22:06:29,153][105620] Updated weights for policy 1, policy_version 926017 (0.0009) [2023-12-26 22:06:29,211][105620] Updated weights for policy 1, policy_version 926027 (0.0009) [2023-12-26 22:06:29,567][105692] Updated weights for policy 0, policy_version 925939 (0.0008) [2023-12-26 22:06:29,626][105692] Updated weights for policy 0, policy_version 925949 (0.0009) [2023-12-26 22:06:29,693][105692] Updated weights for policy 0, policy_version 925959 (0.0009) [2023-12-26 22:06:30,004][105620] Updated weights for policy 1, policy_version 926037 (0.0008) [2023-12-26 22:06:30,069][105620] Updated weights for policy 1, policy_version 926047 (0.0010) [2023-12-26 22:06:30,129][105620] Updated weights for policy 1, policy_version 926057 (0.0009) [2023-12-26 22:06:30,402][105692] Updated weights for policy 0, policy_version 925969 (0.0009) [2023-12-26 22:06:30,459][105692] Updated weights for policy 0, policy_version 925979 (0.0009) [2023-12-26 22:06:30,516][105692] Updated weights for policy 0, policy_version 925989 (0.0009) [2023-12-26 22:06:30,576][105692] Updated weights for policy 0, policy_version 925999 (0.0009) [2023-12-26 22:06:30,771][105620] Updated weights for policy 1, policy_version 926067 (0.0006) [2023-12-26 22:06:30,823][105620] Updated weights for policy 1, policy_version 926077 (0.0005) [2023-12-26 22:06:30,879][105620] Updated weights for policy 1, policy_version 926087 (0.0007) [2023-12-26 22:06:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19114.7, 300 sec: 19022.1). Total num frames: 474202112. Throughput: 0: 9495.1, 1: 9524.6. Samples: 474170676. Policy #0 lag: (min: 31.0, avg: 31.6, max: 54.0) [2023-12-26 22:06:31,062][104569] Avg episode reward: [(0, '9175.121'), (1, '7636.581')] [2023-12-26 22:06:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000926000_237092864.pth... [2023-12-26 22:06:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000926096_237109248.pth... [2023-12-26 22:06:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000924880_236806144.pth [2023-12-26 22:06:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000924976_236822528.pth [2023-12-26 22:06:31,450][105692] Updated weights for policy 0, policy_version 926009 (0.0010) [2023-12-26 22:06:31,514][105692] Updated weights for policy 0, policy_version 926019 (0.0010) [2023-12-26 22:06:31,551][105620] Updated weights for policy 1, policy_version 926097 (0.0010) [2023-12-26 22:06:31,572][105692] Updated weights for policy 0, policy_version 926029 (0.0010) [2023-12-26 22:06:31,608][105620] Updated weights for policy 1, policy_version 926107 (0.0005) [2023-12-26 22:06:31,681][105620] Updated weights for policy 1, policy_version 926117 (0.0008) [2023-12-26 22:06:31,760][105620] Updated weights for policy 1, policy_version 926127 (0.0007) [2023-12-26 22:06:32,275][105620] Updated weights for policy 1, policy_version 926137 (0.0009) [2023-12-26 22:06:32,338][105620] Updated weights for policy 1, policy_version 926147 (0.0011) [2023-12-26 22:06:32,400][105620] Updated weights for policy 1, policy_version 926157 (0.0009) [2023-12-26 22:06:32,426][105692] Updated weights for policy 0, policy_version 926039 (0.0007) [2023-12-26 22:06:32,478][105692] Updated weights for policy 0, policy_version 926049 (0.0008) [2023-12-26 22:06:32,530][105692] Updated weights for policy 0, policy_version 926059 (0.0008) [2023-12-26 22:06:33,123][105620] Updated weights for policy 1, policy_version 926167 (0.0010) [2023-12-26 22:06:33,178][105620] Updated weights for policy 1, policy_version 926177 (0.0010) [2023-12-26 22:06:33,234][105692] Updated weights for policy 0, policy_version 926069 (0.0008) [2023-12-26 22:06:33,240][105620] Updated weights for policy 1, policy_version 926187 (0.0011) [2023-12-26 22:06:33,281][105692] Updated weights for policy 0, policy_version 926079 (0.0007) [2023-12-26 22:06:33,333][105692] Updated weights for policy 0, policy_version 926089 (0.0008) [2023-12-26 22:06:33,851][105620] Updated weights for policy 1, policy_version 926197 (0.0008) [2023-12-26 22:06:33,909][105620] Updated weights for policy 1, policy_version 926207 (0.0005) [2023-12-26 22:06:33,953][105620] Updated weights for policy 1, policy_version 926217 (0.0005) [2023-12-26 22:06:34,149][105692] Updated weights for policy 0, policy_version 926099 (0.0007) [2023-12-26 22:06:34,214][105692] Updated weights for policy 0, policy_version 926109 (0.0008) [2023-12-26 22:06:34,280][105692] Updated weights for policy 0, policy_version 926119 (0.0010) [2023-12-26 22:06:34,568][105620] Updated weights for policy 1, policy_version 926227 (0.0007) [2023-12-26 22:06:34,636][105620] Updated weights for policy 1, policy_version 926237 (0.0011) [2023-12-26 22:06:34,704][105620] Updated weights for policy 1, policy_version 926247 (0.0008) [2023-12-26 22:06:35,013][105692] Updated weights for policy 0, policy_version 926129 (0.0008) [2023-12-26 22:06:35,067][105692] Updated weights for policy 0, policy_version 926139 (0.0010) [2023-12-26 22:06:35,125][105692] Updated weights for policy 0, policy_version 926149 (0.0009) [2023-12-26 22:06:35,187][105692] Updated weights for policy 0, policy_version 926159 (0.0009) [2023-12-26 22:06:35,273][105620] Updated weights for policy 1, policy_version 926257 (0.0011) [2023-12-26 22:06:35,328][105620] Updated weights for policy 1, policy_version 926267 (0.0010) [2023-12-26 22:06:35,387][105620] Updated weights for policy 1, policy_version 926277 (0.0011) [2023-12-26 22:06:35,451][105620] Updated weights for policy 1, policy_version 926287 (0.0007) [2023-12-26 22:06:35,887][105692] Updated weights for policy 0, policy_version 926169 (0.0010) [2023-12-26 22:06:35,939][105692] Updated weights for policy 0, policy_version 926179 (0.0008) [2023-12-26 22:06:35,994][105692] Updated weights for policy 0, policy_version 926189 (0.0010) [2023-12-26 22:06:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19114.6, 300 sec: 19022.1). Total num frames: 474300416. Throughput: 0: 9375.9, 1: 9623.9. Samples: 474287348. Policy #0 lag: (min: 31.0, avg: 31.6, max: 54.0) [2023-12-26 22:06:36,063][104569] Avg episode reward: [(0, '9173.739'), (1, '7951.723')] [2023-12-26 22:06:36,107][105620] Updated weights for policy 1, policy_version 926297 (0.0011) [2023-12-26 22:06:36,170][105620] Updated weights for policy 1, policy_version 926307 (0.0007) [2023-12-26 22:06:36,236][105620] Updated weights for policy 1, policy_version 926317 (0.0007) [2023-12-26 22:06:36,721][105692] Updated weights for policy 0, policy_version 926199 (0.0010) [2023-12-26 22:06:36,787][105692] Updated weights for policy 0, policy_version 926209 (0.0010) [2023-12-26 22:06:36,850][105692] Updated weights for policy 0, policy_version 926219 (0.0011) [2023-12-26 22:06:36,893][105620] Updated weights for policy 1, policy_version 926327 (0.0007) [2023-12-26 22:06:36,962][105620] Updated weights for policy 1, policy_version 926337 (0.0005) [2023-12-26 22:06:37,032][105620] Updated weights for policy 1, policy_version 926347 (0.0005) [2023-12-26 22:06:37,558][105692] Updated weights for policy 0, policy_version 926229 (0.0011) [2023-12-26 22:06:37,588][105620] Updated weights for policy 1, policy_version 926357 (0.0007) [2023-12-26 22:06:37,611][105692] Updated weights for policy 0, policy_version 926239 (0.0010) [2023-12-26 22:06:37,645][105620] Updated weights for policy 1, policy_version 926367 (0.0007) [2023-12-26 22:06:37,660][105692] Updated weights for policy 0, policy_version 926249 (0.0010) [2023-12-26 22:06:37,703][105620] Updated weights for policy 1, policy_version 926377 (0.0006) [2023-12-26 22:06:38,420][105620] Updated weights for policy 1, policy_version 926387 (0.0008) [2023-12-26 22:06:38,432][105692] Updated weights for policy 0, policy_version 926259 (0.0009) [2023-12-26 22:06:38,475][105620] Updated weights for policy 1, policy_version 926397 (0.0006) [2023-12-26 22:06:38,495][105692] Updated weights for policy 0, policy_version 926269 (0.0007) [2023-12-26 22:06:38,539][105620] Updated weights for policy 1, policy_version 926407 (0.0007) [2023-12-26 22:06:38,559][105692] Updated weights for policy 0, policy_version 926279 (0.0007) [2023-12-26 22:06:39,124][105692] Updated weights for policy 0, policy_version 926289 (0.0007) [2023-12-26 22:06:39,183][105692] Updated weights for policy 0, policy_version 926299 (0.0009) [2023-12-26 22:06:39,252][105692] Updated weights for policy 0, policy_version 926309 (0.0009) [2023-12-26 22:06:39,309][105692] Updated weights for policy 0, policy_version 926319 (0.0008) [2023-12-26 22:06:39,343][105620] Updated weights for policy 1, policy_version 926417 (0.0007) [2023-12-26 22:06:39,409][105620] Updated weights for policy 1, policy_version 926427 (0.0009) [2023-12-26 22:06:39,475][105620] Updated weights for policy 1, policy_version 926437 (0.0008) [2023-12-26 22:06:39,545][105620] Updated weights for policy 1, policy_version 926447 (0.0010) [2023-12-26 22:06:39,974][105692] Updated weights for policy 0, policy_version 926329 (0.0009) [2023-12-26 22:06:40,031][105692] Updated weights for policy 0, policy_version 926339 (0.0006) [2023-12-26 22:06:40,097][105692] Updated weights for policy 0, policy_version 926349 (0.0009) [2023-12-26 22:06:40,351][105620] Updated weights for policy 1, policy_version 926457 (0.0006) [2023-12-26 22:06:40,409][105620] Updated weights for policy 1, policy_version 926467 (0.0009) [2023-12-26 22:06:40,479][105620] Updated weights for policy 1, policy_version 926477 (0.0009) [2023-12-26 22:06:40,870][105692] Updated weights for policy 0, policy_version 926359 (0.0010) [2023-12-26 22:06:40,923][105692] Updated weights for policy 0, policy_version 926369 (0.0009) [2023-12-26 22:06:40,979][105692] Updated weights for policy 0, policy_version 926379 (0.0007) [2023-12-26 22:06:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19114.6, 300 sec: 19049.9). Total num frames: 474398720. Throughput: 0: 9487.9, 1: 9697.5. Samples: 474405824. Policy #0 lag: (min: 31.0, avg: 31.6, max: 54.0) [2023-12-26 22:06:41,063][104569] Avg episode reward: [(0, '9166.537'), (1, '7944.601')] [2023-12-26 22:06:41,148][105620] Updated weights for policy 1, policy_version 926487 (0.0008) [2023-12-26 22:06:41,208][105620] Updated weights for policy 1, policy_version 926497 (0.0008) [2023-12-26 22:06:41,275][105620] Updated weights for policy 1, policy_version 926507 (0.0008) [2023-12-26 22:06:41,738][105692] Updated weights for policy 0, policy_version 926389 (0.0008) [2023-12-26 22:06:41,803][105692] Updated weights for policy 0, policy_version 926399 (0.0008) [2023-12-26 22:06:41,863][105692] Updated weights for policy 0, policy_version 926409 (0.0008) [2023-12-26 22:06:42,095][105620] Updated weights for policy 1, policy_version 926517 (0.0009) [2023-12-26 22:06:42,156][105620] Updated weights for policy 1, policy_version 926527 (0.0009) [2023-12-26 22:06:42,189][105586] KL-divergence is very high: 124.2970 [2023-12-26 22:06:42,209][105620] Updated weights for policy 1, policy_version 926537 (0.0009) [2023-12-26 22:06:42,251][105586] KL-divergence is very high: 128.0655 [2023-12-26 22:06:42,627][105692] Updated weights for policy 0, policy_version 926419 (0.0008) [2023-12-26 22:06:42,682][105692] Updated weights for policy 0, policy_version 926429 (0.0009) [2023-12-26 22:06:42,729][105692] Updated weights for policy 0, policy_version 926439 (0.0008) [2023-12-26 22:06:43,032][105620] Updated weights for policy 1, policy_version 926547 (0.0009) [2023-12-26 22:06:43,089][105620] Updated weights for policy 1, policy_version 926557 (0.0009) [2023-12-26 22:06:43,150][105620] Updated weights for policy 1, policy_version 926567 (0.0009) [2023-12-26 22:06:43,439][105692] Updated weights for policy 0, policy_version 926449 (0.0009) [2023-12-26 22:06:43,484][105692] Updated weights for policy 0, policy_version 926459 (0.0008) [2023-12-26 22:06:43,552][105692] Updated weights for policy 0, policy_version 926469 (0.0007) [2023-12-26 22:06:43,610][105692] Updated weights for policy 0, policy_version 926479 (0.0009) [2023-12-26 22:06:43,870][105620] Updated weights for policy 1, policy_version 926577 (0.0009) [2023-12-26 22:06:43,927][105620] Updated weights for policy 1, policy_version 926587 (0.0009) [2023-12-26 22:06:43,978][105620] Updated weights for policy 1, policy_version 926597 (0.0007) [2023-12-26 22:06:44,048][105620] Updated weights for policy 1, policy_version 926607 (0.0008) [2023-12-26 22:06:44,325][105692] Updated weights for policy 0, policy_version 926489 (0.0008) [2023-12-26 22:06:44,391][105692] Updated weights for policy 0, policy_version 926499 (0.0009) [2023-12-26 22:06:44,457][105692] Updated weights for policy 0, policy_version 926509 (0.0009) [2023-12-26 22:06:44,746][105620] Updated weights for policy 1, policy_version 926617 (0.0010) [2023-12-26 22:06:44,813][105620] Updated weights for policy 1, policy_version 926627 (0.0010) [2023-12-26 22:06:44,878][105620] Updated weights for policy 1, policy_version 926637 (0.0011) [2023-12-26 22:06:45,230][105692] Updated weights for policy 0, policy_version 926519 (0.0009) [2023-12-26 22:06:45,291][105692] Updated weights for policy 0, policy_version 926529 (0.0008) [2023-12-26 22:06:45,352][105692] Updated weights for policy 0, policy_version 926539 (0.0008) [2023-12-26 22:06:45,651][105620] Updated weights for policy 1, policy_version 926647 (0.0011) [2023-12-26 22:06:45,714][105620] Updated weights for policy 1, policy_version 926657 (0.0010) [2023-12-26 22:06:45,778][105620] Updated weights for policy 1, policy_version 926667 (0.0011) [2023-12-26 22:06:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19114.6, 300 sec: 19049.8). Total num frames: 474488832. Throughput: 0: 9501.1, 1: 9633.4. Samples: 474461144. Policy #0 lag: (min: 31.0, avg: 31.6, max: 54.0) [2023-12-26 22:06:46,063][104569] Avg episode reward: [(0, '9166.112'), (1, '7696.979')] [2023-12-26 22:06:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000926544_237232128.pth... [2023-12-26 22:06:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000926672_237256704.pth... [2023-12-26 22:06:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000925456_236953600.pth [2023-12-26 22:06:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000925520_236961792.pth [2023-12-26 22:06:46,118][105692] Updated weights for policy 0, policy_version 926549 (0.0007) [2023-12-26 22:06:46,177][105692] Updated weights for policy 0, policy_version 926559 (0.0006) [2023-12-26 22:06:46,232][105692] Updated weights for policy 0, policy_version 926569 (0.0005) [2023-12-26 22:06:46,509][105620] Updated weights for policy 1, policy_version 926677 (0.0010) [2023-12-26 22:06:46,558][105620] Updated weights for policy 1, policy_version 926687 (0.0010) [2023-12-26 22:06:46,613][105620] Updated weights for policy 1, policy_version 926697 (0.0010) [2023-12-26 22:06:46,874][105692] Updated weights for policy 0, policy_version 926579 (0.0009) [2023-12-26 22:06:46,933][105692] Updated weights for policy 0, policy_version 926589 (0.0010) [2023-12-26 22:06:46,990][105692] Updated weights for policy 0, policy_version 926599 (0.0010) [2023-12-26 22:06:47,347][105620] Updated weights for policy 1, policy_version 926707 (0.0010) [2023-12-26 22:06:47,415][105620] Updated weights for policy 1, policy_version 926717 (0.0010) [2023-12-26 22:06:47,476][105620] Updated weights for policy 1, policy_version 926727 (0.0010) [2023-12-26 22:06:47,781][105692] Updated weights for policy 0, policy_version 926609 (0.0009) [2023-12-26 22:06:47,829][105692] Updated weights for policy 0, policy_version 926619 (0.0008) [2023-12-26 22:06:47,883][105692] Updated weights for policy 0, policy_version 926629 (0.0006) [2023-12-26 22:06:47,951][105692] Updated weights for policy 0, policy_version 926639 (0.0006) [2023-12-26 22:06:48,217][105620] Updated weights for policy 1, policy_version 926737 (0.0010) [2023-12-26 22:06:48,278][105620] Updated weights for policy 1, policy_version 926747 (0.0010) [2023-12-26 22:06:48,348][105620] Updated weights for policy 1, policy_version 926757 (0.0010) [2023-12-26 22:06:48,418][105620] Updated weights for policy 1, policy_version 926767 (0.0010) [2023-12-26 22:06:48,542][105692] Updated weights for policy 0, policy_version 926649 (0.0009) [2023-12-26 22:06:48,594][105692] Updated weights for policy 0, policy_version 926659 (0.0009) [2023-12-26 22:06:48,655][105692] Updated weights for policy 0, policy_version 926669 (0.0009) [2023-12-26 22:06:49,135][105620] Updated weights for policy 1, policy_version 926777 (0.0008) [2023-12-26 22:06:49,197][105620] Updated weights for policy 1, policy_version 926787 (0.0009) [2023-12-26 22:06:49,273][105620] Updated weights for policy 1, policy_version 926797 (0.0009) [2023-12-26 22:06:49,423][105692] Updated weights for policy 0, policy_version 926679 (0.0008) [2023-12-26 22:06:49,479][105692] Updated weights for policy 0, policy_version 926689 (0.0008) [2023-12-26 22:06:49,540][105692] Updated weights for policy 0, policy_version 926699 (0.0009) [2023-12-26 22:06:50,050][105620] Updated weights for policy 1, policy_version 926807 (0.0009) [2023-12-26 22:06:50,116][105620] Updated weights for policy 1, policy_version 926817 (0.0006) [2023-12-26 22:06:50,179][105620] Updated weights for policy 1, policy_version 926827 (0.0006) [2023-12-26 22:06:50,327][105692] Updated weights for policy 0, policy_version 926709 (0.0009) [2023-12-26 22:06:50,384][105692] Updated weights for policy 0, policy_version 926719 (0.0010) [2023-12-26 22:06:50,439][105692] Updated weights for policy 0, policy_version 926729 (0.0009) [2023-12-26 22:06:50,769][105620] Updated weights for policy 1, policy_version 926837 (0.0007) [2023-12-26 22:06:50,826][105620] Updated weights for policy 1, policy_version 926847 (0.0009) [2023-12-26 22:06:50,869][105586] KL-divergence is very high: 105.4396 [2023-12-26 22:06:50,883][105620] Updated weights for policy 1, policy_version 926857 (0.0009) [2023-12-26 22:06:50,920][105586] KL-divergence is very high: 113.2375 [2023-12-26 22:06:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.6, 300 sec: 19049.9). Total num frames: 474587136. Throughput: 0: 9443.7, 1: 9666.5. Samples: 474574768. Policy #0 lag: (min: 31.0, avg: 31.6, max: 54.0) [2023-12-26 22:06:51,062][104569] Avg episode reward: [(0, '8984.612'), (1, '7971.386')] [2023-12-26 22:06:51,280][105692] Updated weights for policy 0, policy_version 926739 (0.0009) [2023-12-26 22:06:51,339][105692] Updated weights for policy 0, policy_version 926749 (0.0010) [2023-12-26 22:06:51,412][105692] Updated weights for policy 0, policy_version 926759 (0.0009) [2023-12-26 22:06:51,623][105620] Updated weights for policy 1, policy_version 926867 (0.0009) [2023-12-26 22:06:51,686][105620] Updated weights for policy 1, policy_version 926877 (0.0008) [2023-12-26 22:06:51,754][105620] Updated weights for policy 1, policy_version 926887 (0.0009) [2023-12-26 22:06:52,138][105692] Updated weights for policy 0, policy_version 926769 (0.0010) [2023-12-26 22:06:52,202][105692] Updated weights for policy 0, policy_version 926779 (0.0009) [2023-12-26 22:06:52,269][105692] Updated weights for policy 0, policy_version 926789 (0.0008) [2023-12-26 22:06:52,334][105692] Updated weights for policy 0, policy_version 926799 (0.0008) [2023-12-26 22:06:52,437][105620] Updated weights for policy 1, policy_version 926897 (0.0006) [2023-12-26 22:06:52,489][105620] Updated weights for policy 1, policy_version 926907 (0.0005) [2023-12-26 22:06:52,556][105620] Updated weights for policy 1, policy_version 926917 (0.0005) [2023-12-26 22:06:52,624][105620] Updated weights for policy 1, policy_version 926927 (0.0006) [2023-12-26 22:06:53,155][105692] Updated weights for policy 0, policy_version 926809 (0.0009) [2023-12-26 22:06:53,208][105692] Updated weights for policy 0, policy_version 926819 (0.0006) [2023-12-26 22:06:53,210][105620] Updated weights for policy 1, policy_version 926937 (0.0009) [2023-12-26 22:06:53,265][105692] Updated weights for policy 0, policy_version 926829 (0.0006) [2023-12-26 22:06:53,271][105620] Updated weights for policy 1, policy_version 926947 (0.0008) [2023-12-26 22:06:53,329][105620] Updated weights for policy 1, policy_version 926957 (0.0008) [2023-12-26 22:06:54,028][105620] Updated weights for policy 1, policy_version 926967 (0.0008) [2023-12-26 22:06:54,058][105692] Updated weights for policy 0, policy_version 926839 (0.0008) [2023-12-26 22:06:54,084][105620] Updated weights for policy 1, policy_version 926977 (0.0006) [2023-12-26 22:06:54,117][105692] Updated weights for policy 0, policy_version 926849 (0.0008) [2023-12-26 22:06:54,141][105620] Updated weights for policy 1, policy_version 926987 (0.0008) [2023-12-26 22:06:54,183][105692] Updated weights for policy 0, policy_version 926859 (0.0006) [2023-12-26 22:06:54,725][105620] Updated weights for policy 1, policy_version 926997 (0.0006) [2023-12-26 22:06:54,786][105620] Updated weights for policy 1, policy_version 927007 (0.0005) [2023-12-26 22:06:54,839][105620] Updated weights for policy 1, policy_version 927017 (0.0008) [2023-12-26 22:06:55,001][105692] Updated weights for policy 0, policy_version 926869 (0.0009) [2023-12-26 22:06:55,055][105692] Updated weights for policy 0, policy_version 926879 (0.0009) [2023-12-26 22:06:55,107][105692] Updated weights for policy 0, policy_version 926889 (0.0009) [2023-12-26 22:06:55,523][105620] Updated weights for policy 1, policy_version 927027 (0.0008) [2023-12-26 22:06:55,581][105620] Updated weights for policy 1, policy_version 927037 (0.0005) [2023-12-26 22:06:55,630][105620] Updated weights for policy 1, policy_version 927047 (0.0005) [2023-12-26 22:06:55,795][105692] Updated weights for policy 0, policy_version 926899 (0.0009) [2023-12-26 22:06:55,853][105692] Updated weights for policy 0, policy_version 926909 (0.0010) [2023-12-26 22:06:55,904][105692] Updated weights for policy 0, policy_version 926919 (0.0010) [2023-12-26 22:06:56,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19251.2, 300 sec: 19049.9). Total num frames: 474685440. Throughput: 0: 9404.6, 1: 9752.3. Samples: 474690932. Policy #0 lag: (min: 31.0, avg: 31.6, max: 54.0) [2023-12-26 22:06:56,062][104569] Avg episode reward: [(0, '8988.023'), (1, '8136.850')] [2023-12-26 22:06:56,244][105620] Updated weights for policy 1, policy_version 927057 (0.0009) [2023-12-26 22:06:56,294][105620] Updated weights for policy 1, policy_version 927067 (0.0005) [2023-12-26 22:06:56,353][105620] Updated weights for policy 1, policy_version 927077 (0.0006) [2023-12-26 22:06:56,439][105620] Updated weights for policy 1, policy_version 927087 (0.0005) [2023-12-26 22:06:56,492][105692] Updated weights for policy 0, policy_version 926929 (0.0007) [2023-12-26 22:06:56,560][105692] Updated weights for policy 0, policy_version 926939 (0.0011) [2023-12-26 22:06:56,632][105692] Updated weights for policy 0, policy_version 926949 (0.0011) [2023-12-26 22:06:56,691][105692] Updated weights for policy 0, policy_version 926959 (0.0010) [2023-12-26 22:06:57,028][105620] Updated weights for policy 1, policy_version 927097 (0.0005) [2023-12-26 22:06:57,079][105620] Updated weights for policy 1, policy_version 927107 (0.0005) [2023-12-26 22:06:57,131][105620] Updated weights for policy 1, policy_version 927117 (0.0005) [2023-12-26 22:06:57,239][105692] Updated weights for policy 0, policy_version 926969 (0.0006) [2023-12-26 22:06:57,300][105692] Updated weights for policy 0, policy_version 926979 (0.0006) [2023-12-26 22:06:57,351][105692] Updated weights for policy 0, policy_version 926989 (0.0010) [2023-12-26 22:06:57,751][105620] Updated weights for policy 1, policy_version 927127 (0.0008) [2023-12-26 22:06:57,812][105620] Updated weights for policy 1, policy_version 927137 (0.0010) [2023-12-26 22:06:57,849][105586] KL-divergence is very high: 121.8716 [2023-12-26 22:06:57,885][105620] Updated weights for policy 1, policy_version 927147 (0.0010) [2023-12-26 22:06:57,902][105586] KL-divergence is very high: 122.0577 [2023-12-26 22:06:58,054][105692] Updated weights for policy 0, policy_version 926999 (0.0007) [2023-12-26 22:06:58,110][105692] Updated weights for policy 0, policy_version 927009 (0.0005) [2023-12-26 22:06:58,174][105692] Updated weights for policy 0, policy_version 927019 (0.0008) [2023-12-26 22:06:58,602][105620] Updated weights for policy 1, policy_version 927157 (0.0010) [2023-12-26 22:06:58,665][105620] Updated weights for policy 1, policy_version 927167 (0.0011) [2023-12-26 22:06:58,734][105620] Updated weights for policy 1, policy_version 927177 (0.0010) [2023-12-26 22:06:58,967][105692] Updated weights for policy 0, policy_version 927029 (0.0009) [2023-12-26 22:06:59,027][105692] Updated weights for policy 0, policy_version 927039 (0.0009) [2023-12-26 22:06:59,089][105692] Updated weights for policy 0, policy_version 927049 (0.0008) [2023-12-26 22:06:59,525][105620] Updated weights for policy 1, policy_version 927187 (0.0011) [2023-12-26 22:06:59,583][105620] Updated weights for policy 1, policy_version 927197 (0.0011) [2023-12-26 22:06:59,644][105620] Updated weights for policy 1, policy_version 927207 (0.0010) [2023-12-26 22:06:59,825][105692] Updated weights for policy 0, policy_version 927059 (0.0007) [2023-12-26 22:06:59,883][105692] Updated weights for policy 0, policy_version 927069 (0.0009) [2023-12-26 22:06:59,949][105692] Updated weights for policy 0, policy_version 927079 (0.0009) [2023-12-26 22:07:00,421][105620] Updated weights for policy 1, policy_version 927217 (0.0009) [2023-12-26 22:07:00,481][105620] Updated weights for policy 1, policy_version 927227 (0.0009) [2023-12-26 22:07:00,540][105620] Updated weights for policy 1, policy_version 927237 (0.0009) [2023-12-26 22:07:00,592][105620] Updated weights for policy 1, policy_version 927247 (0.0009) [2023-12-26 22:07:00,711][105692] Updated weights for policy 0, policy_version 927089 (0.0009) [2023-12-26 22:07:00,764][105692] Updated weights for policy 0, policy_version 927099 (0.0009) [2023-12-26 22:07:00,815][105692] Updated weights for policy 0, policy_version 927110 (0.0010) [2023-12-26 22:07:00,872][105692] Updated weights for policy 0, policy_version 927120 (0.0005) [2023-12-26 22:07:01,062][104569] Fps is (10 sec: 19660.0, 60 sec: 19251.1, 300 sec: 19049.8). Total num frames: 474783744. Throughput: 0: 9540.7, 1: 9840.3. Samples: 474754088. Policy #0 lag: (min: 31.0, avg: 31.6, max: 54.0) [2023-12-26 22:07:01,064][104569] Avg episode reward: [(0, '8897.895'), (1, '7910.469')] [2023-12-26 22:07:01,073][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000927120_237379584.pth... [2023-12-26 22:07:01,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000927248_237404160.pth... [2023-12-26 22:07:01,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000926096_237109248.pth [2023-12-26 22:07:01,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000926000_237092864.pth [2023-12-26 22:07:01,301][105620] Updated weights for policy 1, policy_version 927257 (0.0010) [2023-12-26 22:07:01,363][105620] Updated weights for policy 1, policy_version 927267 (0.0009) [2023-12-26 22:07:01,435][105620] Updated weights for policy 1, policy_version 927277 (0.0007) [2023-12-26 22:07:01,515][105692] Updated weights for policy 0, policy_version 927130 (0.0009) [2023-12-26 22:07:01,572][105692] Updated weights for policy 0, policy_version 927140 (0.0006) [2023-12-26 22:07:01,644][105692] Updated weights for policy 0, policy_version 927150 (0.0006) [2023-12-26 22:07:02,183][105620] Updated weights for policy 1, policy_version 927287 (0.0009) [2023-12-26 22:07:02,232][105620] Updated weights for policy 1, policy_version 927297 (0.0008) [2023-12-26 22:07:02,260][105692] Updated weights for policy 0, policy_version 927160 (0.0007) [2023-12-26 22:07:02,287][105620] Updated weights for policy 1, policy_version 927307 (0.0007) [2023-12-26 22:07:02,319][105692] Updated weights for policy 0, policy_version 927170 (0.0009) [2023-12-26 22:07:02,381][105692] Updated weights for policy 0, policy_version 927180 (0.0006) [2023-12-26 22:07:02,990][105692] Updated weights for policy 0, policy_version 927190 (0.0007) [2023-12-26 22:07:03,041][105692] Updated weights for policy 0, policy_version 927200 (0.0005) [2023-12-26 22:07:03,088][105692] Updated weights for policy 0, policy_version 927210 (0.0005) [2023-12-26 22:07:03,139][105620] Updated weights for policy 1, policy_version 927317 (0.0008) [2023-12-26 22:07:03,189][105620] Updated weights for policy 1, policy_version 927327 (0.0009) [2023-12-26 22:07:03,241][105620] Updated weights for policy 1, policy_version 927337 (0.0010) [2023-12-26 22:07:03,653][105692] Updated weights for policy 0, policy_version 927220 (0.0005) [2023-12-26 22:07:03,703][105692] Updated weights for policy 0, policy_version 927230 (0.0007) [2023-12-26 22:07:03,761][105692] Updated weights for policy 0, policy_version 927240 (0.0010) [2023-12-26 22:07:04,066][105620] Updated weights for policy 1, policy_version 927347 (0.0009) [2023-12-26 22:07:04,125][105620] Updated weights for policy 1, policy_version 927357 (0.0009) [2023-12-26 22:07:04,185][105620] Updated weights for policy 1, policy_version 927367 (0.0008) [2023-12-26 22:07:04,453][105692] Updated weights for policy 0, policy_version 927250 (0.0010) [2023-12-26 22:07:04,518][105692] Updated weights for policy 0, policy_version 927260 (0.0008) [2023-12-26 22:07:04,582][105692] Updated weights for policy 0, policy_version 927270 (0.0009) [2023-12-26 22:07:04,633][105692] Updated weights for policy 0, policy_version 927280 (0.0009) [2023-12-26 22:07:04,956][105620] Updated weights for policy 1, policy_version 927377 (0.0009) [2023-12-26 22:07:05,018][105620] Updated weights for policy 1, policy_version 927387 (0.0009) [2023-12-26 22:07:05,076][105620] Updated weights for policy 1, policy_version 927397 (0.0008) [2023-12-26 22:07:05,132][105620] Updated weights for policy 1, policy_version 927407 (0.0007) [2023-12-26 22:07:05,374][105692] Updated weights for policy 0, policy_version 927290 (0.0009) [2023-12-26 22:07:05,424][105692] Updated weights for policy 0, policy_version 927300 (0.0008) [2023-12-26 22:07:05,484][105692] Updated weights for policy 0, policy_version 927310 (0.0009) [2023-12-26 22:07:05,830][105620] Updated weights for policy 1, policy_version 927417 (0.0007) [2023-12-26 22:07:05,887][105620] Updated weights for policy 1, policy_version 927427 (0.0009) [2023-12-26 22:07:05,941][105620] Updated weights for policy 1, policy_version 927437 (0.0009) [2023-12-26 22:07:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19077.6). Total num frames: 474882048. Throughput: 0: 9663.4, 1: 9750.5. Samples: 474869756. Policy #0 lag: (min: 31.0, avg: 31.6, max: 54.0) [2023-12-26 22:07:06,063][104569] Avg episode reward: [(0, '9078.045'), (1, '7821.119')] [2023-12-26 22:07:06,288][105692] Updated weights for policy 0, policy_version 927320 (0.0008) [2023-12-26 22:07:06,353][105692] Updated weights for policy 0, policy_version 927330 (0.0006) [2023-12-26 22:07:06,419][105692] Updated weights for policy 0, policy_version 927340 (0.0009) [2023-12-26 22:07:06,730][105620] Updated weights for policy 1, policy_version 927447 (0.0006) [2023-12-26 22:07:06,794][105620] Updated weights for policy 1, policy_version 927457 (0.0008) [2023-12-26 22:07:06,850][105620] Updated weights for policy 1, policy_version 927467 (0.0005) [2023-12-26 22:07:07,135][105692] Updated weights for policy 0, policy_version 927350 (0.0009) [2023-12-26 22:07:07,197][105692] Updated weights for policy 0, policy_version 927360 (0.0009) [2023-12-26 22:07:07,259][105692] Updated weights for policy 0, policy_version 927370 (0.0009) [2023-12-26 22:07:07,567][105620] Updated weights for policy 1, policy_version 927477 (0.0009) [2023-12-26 22:07:07,632][105620] Updated weights for policy 1, policy_version 927487 (0.0009) [2023-12-26 22:07:07,691][105620] Updated weights for policy 1, policy_version 927497 (0.0009) [2023-12-26 22:07:07,998][105692] Updated weights for policy 0, policy_version 927380 (0.0009) [2023-12-26 22:07:08,056][105692] Updated weights for policy 0, policy_version 927390 (0.0009) [2023-12-26 22:07:08,114][105692] Updated weights for policy 0, policy_version 927400 (0.0009) [2023-12-26 22:07:08,417][105620] Updated weights for policy 1, policy_version 927507 (0.0009) [2023-12-26 22:07:08,471][105620] Updated weights for policy 1, policy_version 927517 (0.0009) [2023-12-26 22:07:08,531][105620] Updated weights for policy 1, policy_version 927527 (0.0009) [2023-12-26 22:07:08,886][105692] Updated weights for policy 0, policy_version 927410 (0.0009) [2023-12-26 22:07:08,944][105692] Updated weights for policy 0, policy_version 927420 (0.0009) [2023-12-26 22:07:08,991][105692] Updated weights for policy 0, policy_version 927430 (0.0008) [2023-12-26 22:07:09,053][105692] Updated weights for policy 0, policy_version 927440 (0.0008) [2023-12-26 22:07:09,300][105620] Updated weights for policy 1, policy_version 927537 (0.0009) [2023-12-26 22:07:09,366][105620] Updated weights for policy 1, policy_version 927547 (0.0009) [2023-12-26 22:07:09,437][105620] Updated weights for policy 1, policy_version 927557 (0.0008) [2023-12-26 22:07:09,502][105620] Updated weights for policy 1, policy_version 927567 (0.0008) [2023-12-26 22:07:09,847][105692] Updated weights for policy 0, policy_version 927450 (0.0006) [2023-12-26 22:07:09,914][105692] Updated weights for policy 0, policy_version 927460 (0.0007) [2023-12-26 22:07:09,984][105692] Updated weights for policy 0, policy_version 927470 (0.0007) [2023-12-26 22:07:10,277][105620] Updated weights for policy 1, policy_version 927577 (0.0009) [2023-12-26 22:07:10,340][105620] Updated weights for policy 1, policy_version 927587 (0.0009) [2023-12-26 22:07:10,400][105620] Updated weights for policy 1, policy_version 927597 (0.0009) [2023-12-26 22:07:10,668][105692] Updated weights for policy 0, policy_version 927480 (0.0009) [2023-12-26 22:07:10,726][105692] Updated weights for policy 0, policy_version 927490 (0.0009) [2023-12-26 22:07:10,795][105692] Updated weights for policy 0, policy_version 927500 (0.0009) [2023-12-26 22:07:11,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19251.2, 300 sec: 19049.9). Total num frames: 474972160. Throughput: 0: 9591.6, 1: 9759.6. Samples: 474981084. Policy #0 lag: (min: 31.0, avg: 31.6, max: 54.0) [2023-12-26 22:07:11,063][104569] Avg episode reward: [(0, '9075.935'), (1, '8326.994')] [2023-12-26 22:07:11,197][105620] Updated weights for policy 1, policy_version 927607 (0.0009) [2023-12-26 22:07:11,272][105620] Updated weights for policy 1, policy_version 927617 (0.0010) [2023-12-26 22:07:11,341][105620] Updated weights for policy 1, policy_version 927627 (0.0009) [2023-12-26 22:07:11,586][105692] Updated weights for policy 0, policy_version 927510 (0.0009) [2023-12-26 22:07:11,650][105692] Updated weights for policy 0, policy_version 927520 (0.0009) [2023-12-26 22:07:11,710][105692] Updated weights for policy 0, policy_version 927530 (0.0008) [2023-12-26 22:07:12,112][105620] Updated weights for policy 1, policy_version 927637 (0.0009) [2023-12-26 22:07:12,171][105620] Updated weights for policy 1, policy_version 927647 (0.0008) [2023-12-26 22:07:12,234][105620] Updated weights for policy 1, policy_version 927657 (0.0009) [2023-12-26 22:07:12,517][105692] Updated weights for policy 0, policy_version 927540 (0.0010) [2023-12-26 22:07:12,585][105692] Updated weights for policy 0, policy_version 927550 (0.0011) [2023-12-26 22:07:12,650][105692] Updated weights for policy 0, policy_version 927560 (0.0010) [2023-12-26 22:07:13,005][105620] Updated weights for policy 1, policy_version 927667 (0.0007) [2023-12-26 22:07:13,067][105620] Updated weights for policy 1, policy_version 927677 (0.0010) [2023-12-26 22:07:13,128][105620] Updated weights for policy 1, policy_version 927687 (0.0010) [2023-12-26 22:07:13,408][105692] Updated weights for policy 0, policy_version 927570 (0.0010) [2023-12-26 22:07:13,466][105692] Updated weights for policy 0, policy_version 927580 (0.0005) [2023-12-26 22:07:13,524][105692] Updated weights for policy 0, policy_version 927590 (0.0008) [2023-12-26 22:07:13,573][105692] Updated weights for policy 0, policy_version 927600 (0.0007) [2023-12-26 22:07:13,793][105620] Updated weights for policy 1, policy_version 927697 (0.0007) [2023-12-26 22:07:13,855][105620] Updated weights for policy 1, policy_version 927707 (0.0010) [2023-12-26 22:07:13,903][105620] Updated weights for policy 1, policy_version 927717 (0.0010) [2023-12-26 22:07:13,962][105620] Updated weights for policy 1, policy_version 927727 (0.0010) [2023-12-26 22:07:14,204][105692] Updated weights for policy 0, policy_version 927610 (0.0008) [2023-12-26 22:07:14,265][105692] Updated weights for policy 0, policy_version 927620 (0.0008) [2023-12-26 22:07:14,319][105692] Updated weights for policy 0, policy_version 927630 (0.0008) [2023-12-26 22:07:14,712][105620] Updated weights for policy 1, policy_version 927737 (0.0010) [2023-12-26 22:07:14,760][105620] Updated weights for policy 1, policy_version 927747 (0.0010) [2023-12-26 22:07:14,825][105620] Updated weights for policy 1, policy_version 927757 (0.0008) [2023-12-26 22:07:15,032][105692] Updated weights for policy 0, policy_version 927640 (0.0008) [2023-12-26 22:07:15,096][105692] Updated weights for policy 0, policy_version 927650 (0.0008) [2023-12-26 22:07:15,160][105692] Updated weights for policy 0, policy_version 927660 (0.0009) [2023-12-26 22:07:15,608][105620] Updated weights for policy 1, policy_version 927767 (0.0007) [2023-12-26 22:07:15,670][105620] Updated weights for policy 1, policy_version 927777 (0.0005) [2023-12-26 22:07:15,727][105620] Updated weights for policy 1, policy_version 927787 (0.0005) [2023-12-26 22:07:15,965][105692] Updated weights for policy 0, policy_version 927670 (0.0009) [2023-12-26 22:07:16,024][105692] Updated weights for policy 0, policy_version 927681 (0.0010) [2023-12-26 22:07:16,062][104569] Fps is (10 sec: 18021.9, 60 sec: 19251.1, 300 sec: 19077.6). Total num frames: 475062272. Throughput: 0: 9532.5, 1: 9697.5. Samples: 475036032. Policy #0 lag: (min: 31.0, avg: 31.6, max: 54.0) [2023-12-26 22:07:16,063][104569] Avg episode reward: [(0, '8990.985'), (1, '8414.222')] [2023-12-26 22:07:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000927792_237543424.pth... [2023-12-26 22:07:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000926672_237256704.pth [2023-12-26 22:07:16,078][105692] Updated weights for policy 0, policy_version 927691 (0.0010) [2023-12-26 22:07:16,099][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000927696_237527040.pth... [2023-12-26 22:07:16,102][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000926544_237232128.pth [2023-12-26 22:07:16,268][105620] Updated weights for policy 1, policy_version 927797 (0.0008) [2023-12-26 22:07:16,337][105620] Updated weights for policy 1, policy_version 927807 (0.0010) [2023-12-26 22:07:16,392][105620] Updated weights for policy 1, policy_version 927817 (0.0010) [2023-12-26 22:07:16,848][105692] Updated weights for policy 0, policy_version 927702 (0.0011) [2023-12-26 22:07:16,903][105692] Updated weights for policy 0, policy_version 927712 (0.0010) [2023-12-26 22:07:16,961][105692] Updated weights for policy 0, policy_version 927722 (0.0010) [2023-12-26 22:07:17,133][105620] Updated weights for policy 1, policy_version 927827 (0.0010) [2023-12-26 22:07:17,184][105620] Updated weights for policy 1, policy_version 927837 (0.0008) [2023-12-26 22:07:17,239][105620] Updated weights for policy 1, policy_version 927847 (0.0008) [2023-12-26 22:07:17,718][105692] Updated weights for policy 0, policy_version 927732 (0.0010) [2023-12-26 22:07:17,766][105692] Updated weights for policy 0, policy_version 927742 (0.0009) [2023-12-26 22:07:17,818][105692] Updated weights for policy 0, policy_version 927752 (0.0009) [2023-12-26 22:07:17,974][105620] Updated weights for policy 1, policy_version 927857 (0.0008) [2023-12-26 22:07:18,028][105620] Updated weights for policy 1, policy_version 927867 (0.0009) [2023-12-26 22:07:18,087][105620] Updated weights for policy 1, policy_version 927877 (0.0009) [2023-12-26 22:07:18,139][105620] Updated weights for policy 1, policy_version 927887 (0.0009) [2023-12-26 22:07:18,490][105692] Updated weights for policy 0, policy_version 927762 (0.0009) [2023-12-26 22:07:18,550][105692] Updated weights for policy 0, policy_version 927772 (0.0010) [2023-12-26 22:07:18,606][105692] Updated weights for policy 0, policy_version 927782 (0.0009) [2023-12-26 22:07:18,673][105692] Updated weights for policy 0, policy_version 927792 (0.0010) [2023-12-26 22:07:18,934][105620] Updated weights for policy 1, policy_version 927897 (0.0009) [2023-12-26 22:07:18,988][105620] Updated weights for policy 1, policy_version 927907 (0.0009) [2023-12-26 22:07:19,043][105620] Updated weights for policy 1, policy_version 927917 (0.0009) [2023-12-26 22:07:19,423][105692] Updated weights for policy 0, policy_version 927802 (0.0009) [2023-12-26 22:07:19,477][105692] Updated weights for policy 0, policy_version 927812 (0.0009) [2023-12-26 22:07:19,546][105692] Updated weights for policy 0, policy_version 927822 (0.0010) [2023-12-26 22:07:19,868][105620] Updated weights for policy 1, policy_version 927927 (0.0009) [2023-12-26 22:07:19,933][105620] Updated weights for policy 1, policy_version 927937 (0.0008) [2023-12-26 22:07:19,992][105620] Updated weights for policy 1, policy_version 927947 (0.0009) [2023-12-26 22:07:20,270][105692] Updated weights for policy 0, policy_version 927832 (0.0009) [2023-12-26 22:07:20,333][105692] Updated weights for policy 0, policy_version 927842 (0.0008) [2023-12-26 22:07:20,392][105692] Updated weights for policy 0, policy_version 927852 (0.0009) [2023-12-26 22:07:20,715][105620] Updated weights for policy 1, policy_version 927957 (0.0009) [2023-12-26 22:07:20,778][105620] Updated weights for policy 1, policy_version 927967 (0.0009) [2023-12-26 22:07:20,845][105620] Updated weights for policy 1, policy_version 927977 (0.0008) [2023-12-26 22:07:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19077.6). Total num frames: 475160576. Throughput: 0: 9598.8, 1: 9578.5. Samples: 475150324. Policy #0 lag: (min: 31.0, avg: 31.6, max: 54.0) [2023-12-26 22:07:21,063][104569] Avg episode reward: [(0, '8723.360'), (1, '8219.706')] [2023-12-26 22:07:21,151][105692] Updated weights for policy 0, policy_version 927862 (0.0008) [2023-12-26 22:07:21,223][105692] Updated weights for policy 0, policy_version 927872 (0.0007) [2023-12-26 22:07:21,278][105692] Updated weights for policy 0, policy_version 927882 (0.0008) [2023-12-26 22:07:21,598][105620] Updated weights for policy 1, policy_version 927987 (0.0009) [2023-12-26 22:07:21,667][105620] Updated weights for policy 1, policy_version 927997 (0.0009) [2023-12-26 22:07:21,728][105620] Updated weights for policy 1, policy_version 928007 (0.0010) [2023-12-26 22:07:22,001][105692] Updated weights for policy 0, policy_version 927892 (0.0010) [2023-12-26 22:07:22,064][105692] Updated weights for policy 0, policy_version 927902 (0.0009) [2023-12-26 22:07:22,125][105692] Updated weights for policy 0, policy_version 927912 (0.0009) [2023-12-26 22:07:22,545][105620] Updated weights for policy 1, policy_version 928017 (0.0009) [2023-12-26 22:07:22,611][105620] Updated weights for policy 1, policy_version 928027 (0.0009) [2023-12-26 22:07:22,679][105620] Updated weights for policy 1, policy_version 928037 (0.0007) [2023-12-26 22:07:22,748][105620] Updated weights for policy 1, policy_version 928047 (0.0009) [2023-12-26 22:07:22,876][105692] Updated weights for policy 0, policy_version 927922 (0.0009) [2023-12-26 22:07:22,940][105692] Updated weights for policy 0, policy_version 927932 (0.0008) [2023-12-26 22:07:23,006][105692] Updated weights for policy 0, policy_version 927942 (0.0009) [2023-12-26 22:07:23,061][105692] Updated weights for policy 0, policy_version 927952 (0.0009) [2023-12-26 22:07:23,475][105620] Updated weights for policy 1, policy_version 928057 (0.0010) [2023-12-26 22:07:23,523][105620] Updated weights for policy 1, policy_version 928067 (0.0009) [2023-12-26 22:07:23,581][105620] Updated weights for policy 1, policy_version 928077 (0.0008) [2023-12-26 22:07:23,819][105692] Updated weights for policy 0, policy_version 927962 (0.0005) [2023-12-26 22:07:23,873][105692] Updated weights for policy 0, policy_version 927972 (0.0005) [2023-12-26 22:07:23,919][105692] Updated weights for policy 0, policy_version 927982 (0.0007) [2023-12-26 22:07:24,289][105620] Updated weights for policy 1, policy_version 928087 (0.0005) [2023-12-26 22:07:24,347][105620] Updated weights for policy 1, policy_version 928097 (0.0008) [2023-12-26 22:07:24,404][105620] Updated weights for policy 1, policy_version 928107 (0.0010) [2023-12-26 22:07:24,615][105692] Updated weights for policy 0, policy_version 927992 (0.0008) [2023-12-26 22:07:24,664][105692] Updated weights for policy 0, policy_version 928002 (0.0009) [2023-12-26 22:07:24,712][105692] Updated weights for policy 0, policy_version 928012 (0.0009) [2023-12-26 22:07:25,138][105620] Updated weights for policy 1, policy_version 928118 (0.0009) [2023-12-26 22:07:25,204][105620] Updated weights for policy 1, policy_version 928128 (0.0008) [2023-12-26 22:07:25,258][105620] Updated weights for policy 1, policy_version 928138 (0.0009) [2023-12-26 22:07:25,463][105692] Updated weights for policy 0, policy_version 928022 (0.0009) [2023-12-26 22:07:25,517][105692] Updated weights for policy 0, policy_version 928032 (0.0009) [2023-12-26 22:07:25,571][105692] Updated weights for policy 0, policy_version 928042 (0.0008) [2023-12-26 22:07:26,016][105620] Updated weights for policy 1, policy_version 928148 (0.0009) [2023-12-26 22:07:26,062][104569] Fps is (10 sec: 18842.3, 60 sec: 19114.8, 300 sec: 19049.9). Total num frames: 475250688. Throughput: 0: 9548.3, 1: 9487.5. Samples: 475262432. Policy #0 lag: (min: 31.0, avg: 31.6, max: 54.0) [2023-12-26 22:07:26,062][104569] Avg episode reward: [(0, '8903.238'), (1, '8304.647')] [2023-12-26 22:07:26,071][105620] Updated weights for policy 1, policy_version 928158 (0.0009) [2023-12-26 22:07:26,134][105620] Updated weights for policy 1, policy_version 928168 (0.0009) [2023-12-26 22:07:26,342][105692] Updated weights for policy 0, policy_version 928052 (0.0008) [2023-12-26 22:07:26,398][105692] Updated weights for policy 0, policy_version 928062 (0.0009) [2023-12-26 22:07:26,447][105692] Updated weights for policy 0, policy_version 928072 (0.0008) [2023-12-26 22:07:26,867][105620] Updated weights for policy 1, policy_version 928178 (0.0009) [2023-12-26 22:07:26,918][105620] Updated weights for policy 1, policy_version 928188 (0.0009) [2023-12-26 22:07:26,976][105620] Updated weights for policy 1, policy_version 928198 (0.0009) [2023-12-26 22:07:27,037][105620] Updated weights for policy 1, policy_version 928208 (0.0008) [2023-12-26 22:07:27,207][105692] Updated weights for policy 0, policy_version 928082 (0.0008) [2023-12-26 22:07:27,265][105692] Updated weights for policy 0, policy_version 928092 (0.0009) [2023-12-26 22:07:27,326][105692] Updated weights for policy 0, policy_version 928102 (0.0008) [2023-12-26 22:07:27,382][105692] Updated weights for policy 0, policy_version 928112 (0.0008) [2023-12-26 22:07:27,757][105620] Updated weights for policy 1, policy_version 928218 (0.0005) [2023-12-26 22:07:27,813][105620] Updated weights for policy 1, policy_version 928228 (0.0006) [2023-12-26 22:07:27,868][105620] Updated weights for policy 1, policy_version 928238 (0.0008) [2023-12-26 22:07:28,144][105692] Updated weights for policy 0, policy_version 928122 (0.0008) [2023-12-26 22:07:28,192][105692] Updated weights for policy 0, policy_version 928132 (0.0010) [2023-12-26 22:07:28,248][105692] Updated weights for policy 0, policy_version 928142 (0.0009) [2023-12-26 22:07:28,486][105620] Updated weights for policy 1, policy_version 928248 (0.0008) [2023-12-26 22:07:28,547][105620] Updated weights for policy 1, policy_version 928258 (0.0008) [2023-12-26 22:07:28,609][105620] Updated weights for policy 1, policy_version 928268 (0.0007) [2023-12-26 22:07:29,094][105692] Updated weights for policy 0, policy_version 928152 (0.0009) [2023-12-26 22:07:29,165][105692] Updated weights for policy 0, policy_version 928162 (0.0009) [2023-12-26 22:07:29,234][105692] Updated weights for policy 0, policy_version 928172 (0.0009) [2023-12-26 22:07:29,268][105620] Updated weights for policy 1, policy_version 928278 (0.0006) [2023-12-26 22:07:29,336][105620] Updated weights for policy 1, policy_version 928288 (0.0008) [2023-12-26 22:07:29,401][105620] Updated weights for policy 1, policy_version 928298 (0.0009) [2023-12-26 22:07:29,976][105692] Updated weights for policy 0, policy_version 928182 (0.0009) [2023-12-26 22:07:30,027][105692] Updated weights for policy 0, policy_version 928192 (0.0009) [2023-12-26 22:07:30,078][105692] Updated weights for policy 0, policy_version 928202 (0.0009) [2023-12-26 22:07:30,156][105620] Updated weights for policy 1, policy_version 928308 (0.0009) [2023-12-26 22:07:30,203][105620] Updated weights for policy 1, policy_version 928318 (0.0009) [2023-12-26 22:07:30,254][105620] Updated weights for policy 1, policy_version 928328 (0.0009) [2023-12-26 22:07:30,846][105692] Updated weights for policy 0, policy_version 928212 (0.0009) [2023-12-26 22:07:30,903][105692] Updated weights for policy 0, policy_version 928222 (0.0009) [2023-12-26 22:07:30,961][105692] Updated weights for policy 0, policy_version 928232 (0.0009) [2023-12-26 22:07:31,021][105620] Updated weights for policy 1, policy_version 928338 (0.0008) [2023-12-26 22:07:31,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19114.7, 300 sec: 19049.9). Total num frames: 475348992. Throughput: 0: 9534.5, 1: 9552.8. Samples: 475320064. Policy #0 lag: (min: 31.0, avg: 31.6, max: 54.0) [2023-12-26 22:07:31,062][104569] Avg episode reward: [(0, '8992.256'), (1, '8673.142')] [2023-12-26 22:07:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000928240_237666304.pth... [2023-12-26 22:07:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000927120_237379584.pth [2023-12-26 22:07:31,091][105620] Updated weights for policy 1, policy_version 928348 (0.0009) [2023-12-26 22:07:31,159][105620] Updated weights for policy 1, policy_version 928358 (0.0009) [2023-12-26 22:07:31,225][105620] Updated weights for policy 1, policy_version 928368 (0.0009) [2023-12-26 22:07:31,226][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000928368_237690880.pth... [2023-12-26 22:07:31,231][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000927248_237404160.pth [2023-12-26 22:07:31,752][105692] Updated weights for policy 0, policy_version 928242 (0.0009) [2023-12-26 22:07:31,805][105692] Updated weights for policy 0, policy_version 928252 (0.0009) [2023-12-26 22:07:31,861][105692] Updated weights for policy 0, policy_version 928262 (0.0009) [2023-12-26 22:07:31,911][105692] Updated weights for policy 0, policy_version 928272 (0.0009) [2023-12-26 22:07:32,016][105620] Updated weights for policy 1, policy_version 928378 (0.0009) [2023-12-26 22:07:32,067][105620] Updated weights for policy 1, policy_version 928388 (0.0009) [2023-12-26 22:07:32,115][105620] Updated weights for policy 1, policy_version 928398 (0.0009) [2023-12-26 22:07:32,614][105692] Updated weights for policy 0, policy_version 928282 (0.0010) [2023-12-26 22:07:32,660][105692] Updated weights for policy 0, policy_version 928292 (0.0008) [2023-12-26 22:07:32,726][105692] Updated weights for policy 0, policy_version 928302 (0.0008) [2023-12-26 22:07:32,906][105620] Updated weights for policy 1, policy_version 928408 (0.0008) [2023-12-26 22:07:32,967][105620] Updated weights for policy 1, policy_version 928418 (0.0009) [2023-12-26 22:07:33,024][105620] Updated weights for policy 1, policy_version 928428 (0.0009) [2023-12-26 22:07:33,481][105692] Updated weights for policy 0, policy_version 928312 (0.0009) [2023-12-26 22:07:33,538][105692] Updated weights for policy 0, policy_version 928322 (0.0009) [2023-12-26 22:07:33,591][105692] Updated weights for policy 0, policy_version 928332 (0.0009) [2023-12-26 22:07:33,756][105620] Updated weights for policy 1, policy_version 928438 (0.0009) [2023-12-26 22:07:33,816][105620] Updated weights for policy 1, policy_version 928448 (0.0009) [2023-12-26 22:07:33,863][105620] Updated weights for policy 1, policy_version 928458 (0.0008) [2023-12-26 22:07:34,371][105692] Updated weights for policy 0, policy_version 928342 (0.0009) [2023-12-26 22:07:34,434][105692] Updated weights for policy 0, policy_version 928352 (0.0009) [2023-12-26 22:07:34,487][105692] Updated weights for policy 0, policy_version 928362 (0.0009) [2023-12-26 22:07:34,567][105620] Updated weights for policy 1, policy_version 928468 (0.0009) [2023-12-26 22:07:34,633][105620] Updated weights for policy 1, policy_version 928478 (0.0009) [2023-12-26 22:07:34,692][105620] Updated weights for policy 1, policy_version 928488 (0.0009) [2023-12-26 22:07:35,265][105620] Updated weights for policy 1, policy_version 928498 (0.0008) [2023-12-26 22:07:35,319][105620] Updated weights for policy 1, policy_version 928508 (0.0009) [2023-12-26 22:07:35,340][105692] Updated weights for policy 0, policy_version 928372 (0.0009) [2023-12-26 22:07:35,379][105620] Updated weights for policy 1, policy_version 928518 (0.0008) [2023-12-26 22:07:35,390][105692] Updated weights for policy 0, policy_version 928382 (0.0006) [2023-12-26 22:07:35,431][105620] Updated weights for policy 1, policy_version 928528 (0.0008) [2023-12-26 22:07:35,469][105692] Updated weights for policy 0, policy_version 928392 (0.0008) [2023-12-26 22:07:36,062][104569] Fps is (10 sec: 18841.5, 60 sec: 18978.2, 300 sec: 19049.9). Total num frames: 475439104. Throughput: 0: 9485.5, 1: 9546.9. Samples: 475431224. Policy #0 lag: (min: 31.0, avg: 31.6, max: 54.0) [2023-12-26 22:07:36,062][104569] Avg episode reward: [(0, '9078.571'), (1, '8854.648')] [2023-12-26 22:07:36,112][105620] Updated weights for policy 1, policy_version 928538 (0.0007) [2023-12-26 22:07:36,175][105620] Updated weights for policy 1, policy_version 928548 (0.0009) [2023-12-26 22:07:36,209][105692] Updated weights for policy 0, policy_version 928402 (0.0009) [2023-12-26 22:07:36,241][105620] Updated weights for policy 1, policy_version 928558 (0.0010) [2023-12-26 22:07:36,271][105692] Updated weights for policy 0, policy_version 928412 (0.0010) [2023-12-26 22:07:36,330][105692] Updated weights for policy 0, policy_version 928422 (0.0009) [2023-12-26 22:07:36,398][105692] Updated weights for policy 0, policy_version 928432 (0.0010) [2023-12-26 22:07:36,934][105620] Updated weights for policy 1, policy_version 928568 (0.0009) [2023-12-26 22:07:36,989][105620] Updated weights for policy 1, policy_version 928578 (0.0009) [2023-12-26 22:07:37,047][105620] Updated weights for policy 1, policy_version 928588 (0.0009) [2023-12-26 22:07:37,163][105692] Updated weights for policy 0, policy_version 928442 (0.0008) [2023-12-26 22:07:37,214][105692] Updated weights for policy 0, policy_version 928452 (0.0009) [2023-12-26 22:07:37,269][105692] Updated weights for policy 0, policy_version 928462 (0.0009) [2023-12-26 22:07:37,817][105620] Updated weights for policy 1, policy_version 928598 (0.0009) [2023-12-26 22:07:37,864][105620] Updated weights for policy 1, policy_version 928608 (0.0009) [2023-12-26 22:07:37,923][105620] Updated weights for policy 1, policy_version 928618 (0.0008) [2023-12-26 22:07:38,038][105692] Updated weights for policy 0, policy_version 928472 (0.0008) [2023-12-26 22:07:38,092][105692] Updated weights for policy 0, policy_version 928482 (0.0009) [2023-12-26 22:07:38,141][105692] Updated weights for policy 0, policy_version 928492 (0.0008) [2023-12-26 22:07:38,737][105620] Updated weights for policy 1, policy_version 928628 (0.0009) [2023-12-26 22:07:38,801][105620] Updated weights for policy 1, policy_version 928638 (0.0009) [2023-12-26 22:07:38,866][105620] Updated weights for policy 1, policy_version 928648 (0.0007) [2023-12-26 22:07:38,926][105692] Updated weights for policy 0, policy_version 928502 (0.0009) [2023-12-26 22:07:38,988][105692] Updated weights for policy 0, policy_version 928512 (0.0009) [2023-12-26 22:07:39,049][105692] Updated weights for policy 0, policy_version 928522 (0.0008) [2023-12-26 22:07:39,556][105620] Updated weights for policy 1, policy_version 928658 (0.0006) [2023-12-26 22:07:39,616][105620] Updated weights for policy 1, policy_version 928668 (0.0007) [2023-12-26 22:07:39,676][105620] Updated weights for policy 1, policy_version 928678 (0.0010) [2023-12-26 22:07:39,736][105620] Updated weights for policy 1, policy_version 928688 (0.0009) [2023-12-26 22:07:39,851][105692] Updated weights for policy 0, policy_version 928532 (0.0010) [2023-12-26 22:07:39,910][105692] Updated weights for policy 0, policy_version 928542 (0.0009) [2023-12-26 22:07:39,974][105692] Updated weights for policy 0, policy_version 928552 (0.0009) [2023-12-26 22:07:40,496][105620] Updated weights for policy 1, policy_version 928698 (0.0010) [2023-12-26 22:07:40,558][105620] Updated weights for policy 1, policy_version 928708 (0.0009) [2023-12-26 22:07:40,606][105620] Updated weights for policy 1, policy_version 928718 (0.0008) [2023-12-26 22:07:40,721][105692] Updated weights for policy 0, policy_version 928562 (0.0010) [2023-12-26 22:07:40,783][105692] Updated weights for policy 0, policy_version 928572 (0.0009) [2023-12-26 22:07:40,844][105692] Updated weights for policy 0, policy_version 928582 (0.0008) [2023-12-26 22:07:40,907][105692] Updated weights for policy 0, policy_version 928592 (0.0009) [2023-12-26 22:07:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 18978.1, 300 sec: 19077.6). Total num frames: 475537408. Throughput: 0: 9497.5, 1: 9443.4. Samples: 475543272. Policy #0 lag: (min: 31.0, avg: 31.6, max: 54.0) [2023-12-26 22:07:41,063][104569] Avg episode reward: [(0, '8227.619'), (1, '9106.536')] [2023-12-26 22:07:41,403][105620] Updated weights for policy 1, policy_version 928728 (0.0009) [2023-12-26 22:07:41,478][105620] Updated weights for policy 1, policy_version 928738 (0.0008) [2023-12-26 22:07:41,552][105620] Updated weights for policy 1, policy_version 928748 (0.0008) [2023-12-26 22:07:41,743][105692] Updated weights for policy 0, policy_version 928602 (0.0008) [2023-12-26 22:07:41,813][105692] Updated weights for policy 0, policy_version 928612 (0.0009) [2023-12-26 22:07:41,873][105692] Updated weights for policy 0, policy_version 928622 (0.0010) [2023-12-26 22:07:42,339][105620] Updated weights for policy 1, policy_version 928758 (0.0009) [2023-12-26 22:07:42,408][105620] Updated weights for policy 1, policy_version 928768 (0.0009) [2023-12-26 22:07:42,475][105620] Updated weights for policy 1, policy_version 928778 (0.0009) [2023-12-26 22:07:42,601][105692] Updated weights for policy 0, policy_version 928632 (0.0009) [2023-12-26 22:07:42,653][105692] Updated weights for policy 0, policy_version 928642 (0.0009) [2023-12-26 22:07:42,701][105692] Updated weights for policy 0, policy_version 928652 (0.0009) [2023-12-26 22:07:43,211][105620] Updated weights for policy 1, policy_version 928788 (0.0009) [2023-12-26 22:07:43,260][105620] Updated weights for policy 1, policy_version 928798 (0.0007) [2023-12-26 22:07:43,310][105620] Updated weights for policy 1, policy_version 928808 (0.0008) [2023-12-26 22:07:43,472][105692] Updated weights for policy 0, policy_version 928662 (0.0009) [2023-12-26 22:07:43,519][105692] Updated weights for policy 0, policy_version 928672 (0.0009) [2023-12-26 22:07:43,575][105692] Updated weights for policy 0, policy_version 928682 (0.0008) [2023-12-26 22:07:44,062][105620] Updated weights for policy 1, policy_version 928818 (0.0008) [2023-12-26 22:07:44,126][105620] Updated weights for policy 1, policy_version 928828 (0.0006) [2023-12-26 22:07:44,184][105620] Updated weights for policy 1, policy_version 928838 (0.0005) [2023-12-26 22:07:44,246][105620] Updated weights for policy 1, policy_version 928848 (0.0007) [2023-12-26 22:07:44,368][105692] Updated weights for policy 0, policy_version 928692 (0.0009) [2023-12-26 22:07:44,420][105692] Updated weights for policy 0, policy_version 928702 (0.0008) [2023-12-26 22:07:44,471][105692] Updated weights for policy 0, policy_version 928712 (0.0005) [2023-12-26 22:07:44,972][105620] Updated weights for policy 1, policy_version 928858 (0.0010) [2023-12-26 22:07:45,039][105620] Updated weights for policy 1, policy_version 928868 (0.0010) [2023-12-26 22:07:45,087][105692] Updated weights for policy 0, policy_version 928722 (0.0006) [2023-12-26 22:07:45,102][105620] Updated weights for policy 1, policy_version 928878 (0.0011) [2023-12-26 22:07:45,146][105692] Updated weights for policy 0, policy_version 928732 (0.0007) [2023-12-26 22:07:45,211][105692] Updated weights for policy 0, policy_version 928742 (0.0008) [2023-12-26 22:07:45,276][105692] Updated weights for policy 0, policy_version 928752 (0.0009) [2023-12-26 22:07:45,836][105620] Updated weights for policy 1, policy_version 928888 (0.0010) [2023-12-26 22:07:45,894][105620] Updated weights for policy 1, policy_version 928898 (0.0010) [2023-12-26 22:07:45,950][105620] Updated weights for policy 1, policy_version 928908 (0.0010) [2023-12-26 22:07:46,042][105692] Updated weights for policy 0, policy_version 928762 (0.0006) [2023-12-26 22:07:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 18978.2, 300 sec: 19049.9). Total num frames: 475627520. Throughput: 0: 9375.1, 1: 9361.2. Samples: 475597216. Policy #0 lag: (min: 31.0, avg: 31.6, max: 54.0) [2023-12-26 22:07:46,063][104569] Avg episode reward: [(0, '8227.428'), (1, '9002.478')] [2023-12-26 22:07:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000928912_237830144.pth... [2023-12-26 22:07:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000927792_237543424.pth [2023-12-26 22:07:46,098][105692] Updated weights for policy 0, policy_version 928772 (0.0006) [2023-12-26 22:07:46,146][105692] Updated weights for policy 0, policy_version 928782 (0.0009) [2023-12-26 22:07:46,155][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000928784_237805568.pth... [2023-12-26 22:07:46,159][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000927696_237527040.pth [2023-12-26 22:07:46,551][105620] Updated weights for policy 1, policy_version 928918 (0.0007) [2023-12-26 22:07:46,603][105620] Updated weights for policy 1, policy_version 928928 (0.0005) [2023-12-26 22:07:46,650][105620] Updated weights for policy 1, policy_version 928938 (0.0005) [2023-12-26 22:07:46,842][105692] Updated weights for policy 0, policy_version 928792 (0.0009) [2023-12-26 22:07:46,902][105692] Updated weights for policy 0, policy_version 928802 (0.0008) [2023-12-26 22:07:46,964][105692] Updated weights for policy 0, policy_version 928812 (0.0006) [2023-12-26 22:07:47,301][105620] Updated weights for policy 1, policy_version 928948 (0.0007) [2023-12-26 22:07:47,360][105620] Updated weights for policy 1, policy_version 928958 (0.0008) [2023-12-26 22:07:47,421][105620] Updated weights for policy 1, policy_version 928968 (0.0007) [2023-12-26 22:07:47,618][105692] Updated weights for policy 0, policy_version 928822 (0.0008) [2023-12-26 22:07:47,678][105692] Updated weights for policy 0, policy_version 928832 (0.0006) [2023-12-26 22:07:47,732][105692] Updated weights for policy 0, policy_version 928842 (0.0006) [2023-12-26 22:07:48,109][105620] Updated weights for policy 1, policy_version 928978 (0.0009) [2023-12-26 22:07:48,170][105620] Updated weights for policy 1, policy_version 928988 (0.0009) [2023-12-26 22:07:48,227][105620] Updated weights for policy 1, policy_version 928998 (0.0009) [2023-12-26 22:07:48,274][105620] Updated weights for policy 1, policy_version 929008 (0.0009) [2023-12-26 22:07:48,484][105692] Updated weights for policy 0, policy_version 928852 (0.0008) [2023-12-26 22:07:48,547][105692] Updated weights for policy 0, policy_version 928862 (0.0008) [2023-12-26 22:07:48,602][105692] Updated weights for policy 0, policy_version 928872 (0.0008) [2023-12-26 22:07:49,064][105620] Updated weights for policy 1, policy_version 929018 (0.0008) [2023-12-26 22:07:49,126][105620] Updated weights for policy 1, policy_version 929028 (0.0009) [2023-12-26 22:07:49,183][105620] Updated weights for policy 1, policy_version 929038 (0.0008) [2023-12-26 22:07:49,349][105692] Updated weights for policy 0, policy_version 928882 (0.0006) [2023-12-26 22:07:49,417][105692] Updated weights for policy 0, policy_version 928892 (0.0009) [2023-12-26 22:07:49,480][105692] Updated weights for policy 0, policy_version 928902 (0.0009) [2023-12-26 22:07:49,538][105692] Updated weights for policy 0, policy_version 928912 (0.0008) [2023-12-26 22:07:49,992][105620] Updated weights for policy 1, policy_version 929048 (0.0007) [2023-12-26 22:07:50,052][105620] Updated weights for policy 1, policy_version 929058 (0.0008) [2023-12-26 22:07:50,106][105620] Updated weights for policy 1, policy_version 929068 (0.0009) [2023-12-26 22:07:50,319][105692] Updated weights for policy 0, policy_version 928922 (0.0009) [2023-12-26 22:07:50,371][105692] Updated weights for policy 0, policy_version 928932 (0.0009) [2023-12-26 22:07:50,423][105692] Updated weights for policy 0, policy_version 928942 (0.0009) [2023-12-26 22:07:50,873][105620] Updated weights for policy 1, policy_version 929078 (0.0009) [2023-12-26 22:07:50,931][105620] Updated weights for policy 1, policy_version 929088 (0.0009) [2023-12-26 22:07:50,993][105620] Updated weights for policy 1, policy_version 929098 (0.0009) [2023-12-26 22:07:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 18978.1, 300 sec: 19077.6). Total num frames: 475725824. Throughput: 0: 9298.0, 1: 9453.7. Samples: 475713580. Policy #0 lag: (min: 31.0, avg: 31.6, max: 54.0) [2023-12-26 22:07:51,063][104569] Avg episode reward: [(0, '8724.258'), (1, '8415.817')] [2023-12-26 22:07:51,184][105692] Updated weights for policy 0, policy_version 928952 (0.0009) [2023-12-26 22:07:51,243][105692] Updated weights for policy 0, policy_version 928962 (0.0009) [2023-12-26 22:07:51,302][105692] Updated weights for policy 0, policy_version 928972 (0.0009) [2023-12-26 22:07:51,784][105620] Updated weights for policy 1, policy_version 929108 (0.0007) [2023-12-26 22:07:51,841][105620] Updated weights for policy 1, policy_version 929118 (0.0005) [2023-12-26 22:07:51,892][105620] Updated weights for policy 1, policy_version 929128 (0.0008) [2023-12-26 22:07:52,068][105692] Updated weights for policy 0, policy_version 928982 (0.0009) [2023-12-26 22:07:52,139][105692] Updated weights for policy 0, policy_version 928992 (0.0009) [2023-12-26 22:07:52,210][105692] Updated weights for policy 0, policy_version 929002 (0.0009) [2023-12-26 22:07:52,570][105620] Updated weights for policy 1, policy_version 929138 (0.0007) [2023-12-26 22:07:52,632][105620] Updated weights for policy 1, policy_version 929148 (0.0007) [2023-12-26 22:07:52,692][105620] Updated weights for policy 1, policy_version 929158 (0.0008) [2023-12-26 22:07:52,749][105620] Updated weights for policy 1, policy_version 929168 (0.0006) [2023-12-26 22:07:52,956][105692] Updated weights for policy 0, policy_version 929012 (0.0009) [2023-12-26 22:07:53,015][105692] Updated weights for policy 0, policy_version 929022 (0.0009) [2023-12-26 22:07:53,083][105692] Updated weights for policy 0, policy_version 929032 (0.0009) [2023-12-26 22:07:53,426][105620] Updated weights for policy 1, policy_version 929178 (0.0009) [2023-12-26 22:07:53,478][105620] Updated weights for policy 1, policy_version 929188 (0.0008) [2023-12-26 22:07:53,535][105620] Updated weights for policy 1, policy_version 929198 (0.0005) [2023-12-26 22:07:53,900][105692] Updated weights for policy 0, policy_version 929042 (0.0008) [2023-12-26 22:07:53,971][105692] Updated weights for policy 0, policy_version 929052 (0.0010) [2023-12-26 22:07:54,030][105692] Updated weights for policy 0, policy_version 929062 (0.0009) [2023-12-26 22:07:54,090][105692] Updated weights for policy 0, policy_version 929072 (0.0009) [2023-12-26 22:07:54,242][105620] Updated weights for policy 1, policy_version 929208 (0.0008) [2023-12-26 22:07:54,305][105620] Updated weights for policy 1, policy_version 929218 (0.0009) [2023-12-26 22:07:54,366][105620] Updated weights for policy 1, policy_version 929228 (0.0009) [2023-12-26 22:07:54,876][105692] Updated weights for policy 0, policy_version 929082 (0.0010) [2023-12-26 22:07:54,932][105692] Updated weights for policy 0, policy_version 929093 (0.0012) [2023-12-26 22:07:54,987][105692] Updated weights for policy 0, policy_version 929103 (0.0010) [2023-12-26 22:07:55,049][105620] Updated weights for policy 1, policy_version 929238 (0.0006) [2023-12-26 22:07:55,096][105620] Updated weights for policy 1, policy_version 929248 (0.0006) [2023-12-26 22:07:55,156][105620] Updated weights for policy 1, policy_version 929258 (0.0008) [2023-12-26 22:07:55,830][105620] Updated weights for policy 1, policy_version 929268 (0.0008) [2023-12-26 22:07:55,875][105692] Updated weights for policy 0, policy_version 929113 (0.0008) [2023-12-26 22:07:55,879][105620] Updated weights for policy 1, policy_version 929278 (0.0005) [2023-12-26 22:07:55,924][105692] Updated weights for policy 0, policy_version 929123 (0.0008) [2023-12-26 22:07:55,944][105620] Updated weights for policy 1, policy_version 929288 (0.0005) [2023-12-26 22:07:55,978][105692] Updated weights for policy 0, policy_version 929133 (0.0009) [2023-12-26 22:07:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 18978.1, 300 sec: 19077.6). Total num frames: 475824128. Throughput: 0: 9259.8, 1: 9555.0. Samples: 475827752. Policy #0 lag: (min: 31.0, avg: 31.6, max: 54.0) [2023-12-26 22:07:56,063][104569] Avg episode reward: [(0, '9074.301'), (1, '8254.082')] [2023-12-26 22:07:56,534][105620] Updated weights for policy 1, policy_version 929298 (0.0005) [2023-12-26 22:07:56,600][105620] Updated weights for policy 1, policy_version 929308 (0.0005) [2023-12-26 22:07:56,662][105620] Updated weights for policy 1, policy_version 929318 (0.0008) [2023-12-26 22:07:56,708][105620] Updated weights for policy 1, policy_version 929328 (0.0009) [2023-12-26 22:07:56,839][105692] Updated weights for policy 0, policy_version 929143 (0.0008) [2023-12-26 22:07:56,887][105692] Updated weights for policy 0, policy_version 929153 (0.0009) [2023-12-26 22:07:56,934][105692] Updated weights for policy 0, policy_version 929163 (0.0009) [2023-12-26 22:07:57,402][105620] Updated weights for policy 1, policy_version 929338 (0.0009) [2023-12-26 22:07:57,463][105620] Updated weights for policy 1, policy_version 929348 (0.0009) [2023-12-26 22:07:57,521][105620] Updated weights for policy 1, policy_version 929358 (0.0009) [2023-12-26 22:07:57,709][105692] Updated weights for policy 0, policy_version 929173 (0.0009) [2023-12-26 22:07:57,757][105692] Updated weights for policy 0, policy_version 929183 (0.0009) [2023-12-26 22:07:57,805][105692] Updated weights for policy 0, policy_version 929193 (0.0009) [2023-12-26 22:07:58,228][105620] Updated weights for policy 1, policy_version 929368 (0.0008) [2023-12-26 22:07:58,283][105586] KL-divergence is very high: 148.4890 [2023-12-26 22:07:58,289][105620] Updated weights for policy 1, policy_version 929378 (0.0008) [2023-12-26 22:07:58,336][105586] KL-divergence is very high: 162.7144 [2023-12-26 22:07:58,363][105620] Updated weights for policy 1, policy_version 929388 (0.0009) [2023-12-26 22:07:58,602][105692] Updated weights for policy 0, policy_version 929203 (0.0010) [2023-12-26 22:07:58,665][105692] Updated weights for policy 0, policy_version 929213 (0.0011) [2023-12-26 22:07:58,727][105692] Updated weights for policy 0, policy_version 929223 (0.0011) [2023-12-26 22:07:59,161][105620] Updated weights for policy 1, policy_version 929398 (0.0007) [2023-12-26 22:07:59,231][105620] Updated weights for policy 1, policy_version 929408 (0.0007) [2023-12-26 22:07:59,311][105620] Updated weights for policy 1, policy_version 929418 (0.0008) [2023-12-26 22:07:59,829][105692] Updated weights for policy 0, policy_version 929233 (0.0010) [2023-12-26 22:07:59,890][105692] Updated weights for policy 0, policy_version 929243 (0.0009) [2023-12-26 22:07:59,961][105692] Updated weights for policy 0, policy_version 929253 (0.0009) [2023-12-26 22:07:59,976][105620] Updated weights for policy 1, policy_version 929428 (0.0008) [2023-12-26 22:08:00,016][105692] Updated weights for policy 0, policy_version 929263 (0.0007) [2023-12-26 22:08:00,031][105620] Updated weights for policy 1, policy_version 929438 (0.0006) [2023-12-26 22:08:00,084][105620] Updated weights for policy 1, policy_version 929448 (0.0009) [2023-12-26 22:08:00,743][105692] Updated weights for policy 0, policy_version 929273 (0.0006) [2023-12-26 22:08:00,796][105620] Updated weights for policy 1, policy_version 929458 (0.0008) [2023-12-26 22:08:00,798][105692] Updated weights for policy 0, policy_version 929283 (0.0005) [2023-12-26 22:08:00,844][105620] Updated weights for policy 1, policy_version 929468 (0.0005) [2023-12-26 22:08:00,850][105692] Updated weights for policy 0, policy_version 929293 (0.0005) [2023-12-26 22:08:00,895][105620] Updated weights for policy 1, policy_version 929478 (0.0007) [2023-12-26 22:08:00,959][105620] Updated weights for policy 1, policy_version 929488 (0.0006) [2023-12-26 22:08:01,062][104569] Fps is (10 sec: 18841.8, 60 sec: 18841.8, 300 sec: 19077.6). Total num frames: 475914240. Throughput: 0: 9237.0, 1: 9596.3. Samples: 475883520. Policy #0 lag: (min: 31.0, avg: 31.6, max: 54.0) [2023-12-26 22:08:01,063][104569] Avg episode reward: [(0, '8990.668'), (1, '8845.921')] [2023-12-26 22:08:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000929296_237936640.pth... [2023-12-26 22:08:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000929488_237977600.pth... [2023-12-26 22:08:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000928240_237666304.pth [2023-12-26 22:08:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000928368_237690880.pth [2023-12-26 22:08:01,505][105692] Updated weights for policy 0, policy_version 929303 (0.0009) [2023-12-26 22:08:01,559][105692] Updated weights for policy 0, policy_version 929313 (0.0010) [2023-12-26 22:08:01,619][105692] Updated weights for policy 0, policy_version 929323 (0.0010) [2023-12-26 22:08:01,620][105620] Updated weights for policy 1, policy_version 929498 (0.0010) [2023-12-26 22:08:01,685][105620] Updated weights for policy 1, policy_version 929508 (0.0013) [2023-12-26 22:08:01,756][105620] Updated weights for policy 1, policy_version 929518 (0.0009) [2023-12-26 22:08:02,386][105692] Updated weights for policy 0, policy_version 929333 (0.0010) [2023-12-26 22:08:02,444][105692] Updated weights for policy 0, policy_version 929343 (0.0008) [2023-12-26 22:08:02,483][105620] Updated weights for policy 1, policy_version 929528 (0.0010) [2023-12-26 22:08:02,505][105692] Updated weights for policy 0, policy_version 929353 (0.0007) [2023-12-26 22:08:02,544][105620] Updated weights for policy 1, policy_version 929538 (0.0010) [2023-12-26 22:08:02,592][105620] Updated weights for policy 1, policy_version 929548 (0.0010) [2023-12-26 22:08:03,115][105692] Updated weights for policy 0, policy_version 929363 (0.0010) [2023-12-26 22:08:03,173][105692] Updated weights for policy 0, policy_version 929373 (0.0007) [2023-12-26 22:08:03,223][105692] Updated weights for policy 0, policy_version 929384 (0.0010) [2023-12-26 22:08:03,303][105620] Updated weights for policy 1, policy_version 929558 (0.0007) [2023-12-26 22:08:03,363][105620] Updated weights for policy 1, policy_version 929568 (0.0005) [2023-12-26 22:08:03,410][105620] Updated weights for policy 1, policy_version 929578 (0.0005) [2023-12-26 22:08:03,993][105692] Updated weights for policy 0, policy_version 929394 (0.0009) [2023-12-26 22:08:04,058][105692] Updated weights for policy 0, policy_version 929404 (0.0007) [2023-12-26 22:08:04,058][105620] Updated weights for policy 1, policy_version 929588 (0.0007) [2023-12-26 22:08:04,119][105692] Updated weights for policy 0, policy_version 929414 (0.0007) [2023-12-26 22:08:04,124][105620] Updated weights for policy 1, policy_version 929598 (0.0008) [2023-12-26 22:08:04,175][105692] Updated weights for policy 0, policy_version 929424 (0.0006) [2023-12-26 22:08:04,190][105620] Updated weights for policy 1, policy_version 929608 (0.0007) [2023-12-26 22:08:04,898][105620] Updated weights for policy 1, policy_version 929618 (0.0006) [2023-12-26 22:08:04,907][105692] Updated weights for policy 0, policy_version 929434 (0.0008) [2023-12-26 22:08:04,944][105620] Updated weights for policy 1, policy_version 929628 (0.0006) [2023-12-26 22:08:04,955][105692] Updated weights for policy 0, policy_version 929444 (0.0010) [2023-12-26 22:08:05,000][105620] Updated weights for policy 1, policy_version 929638 (0.0008) [2023-12-26 22:08:05,014][105692] Updated weights for policy 0, policy_version 929454 (0.0007) [2023-12-26 22:08:05,058][105620] Updated weights for policy 1, policy_version 929648 (0.0009) [2023-12-26 22:08:05,606][105692] Updated weights for policy 0, policy_version 929464 (0.0009) [2023-12-26 22:08:05,670][105692] Updated weights for policy 0, policy_version 929474 (0.0009) [2023-12-26 22:08:05,729][105692] Updated weights for policy 0, policy_version 929484 (0.0008) [2023-12-26 22:08:05,873][105620] Updated weights for policy 1, policy_version 929658 (0.0010) [2023-12-26 22:08:05,932][105620] Updated weights for policy 1, policy_version 929668 (0.0010) [2023-12-26 22:08:05,984][105620] Updated weights for policy 1, policy_version 929678 (0.0010) [2023-12-26 22:08:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 18841.6, 300 sec: 19105.4). Total num frames: 476012544. Throughput: 0: 9187.6, 1: 9642.8. Samples: 475997688. Policy #0 lag: (min: 31.0, avg: 31.6, max: 54.0) [2023-12-26 22:08:06,063][104569] Avg episode reward: [(0, '8999.453'), (1, '8860.105')] [2023-12-26 22:08:06,520][105692] Updated weights for policy 0, policy_version 929494 (0.0008) [2023-12-26 22:08:06,581][105692] Updated weights for policy 0, policy_version 929504 (0.0009) [2023-12-26 22:08:06,638][105692] Updated weights for policy 0, policy_version 929514 (0.0008) [2023-12-26 22:08:06,777][105620] Updated weights for policy 1, policy_version 929688 (0.0011) [2023-12-26 22:08:06,837][105620] Updated weights for policy 1, policy_version 929698 (0.0010) [2023-12-26 22:08:06,903][105620] Updated weights for policy 1, policy_version 929708 (0.0011) [2023-12-26 22:08:07,407][105692] Updated weights for policy 0, policy_version 929524 (0.0008) [2023-12-26 22:08:07,471][105692] Updated weights for policy 0, policy_version 929534 (0.0009) [2023-12-26 22:08:07,527][105692] Updated weights for policy 0, policy_version 929544 (0.0008) [2023-12-26 22:08:07,641][105620] Updated weights for policy 1, policy_version 929718 (0.0010) [2023-12-26 22:08:07,692][105620] Updated weights for policy 1, policy_version 929728 (0.0010) [2023-12-26 22:08:07,740][105620] Updated weights for policy 1, policy_version 929738 (0.0010) [2023-12-26 22:08:08,234][105692] Updated weights for policy 0, policy_version 929554 (0.0008) [2023-12-26 22:08:08,288][105692] Updated weights for policy 0, policy_version 929564 (0.0005) [2023-12-26 22:08:08,347][105692] Updated weights for policy 0, policy_version 929574 (0.0007) [2023-12-26 22:08:08,408][105692] Updated weights for policy 0, policy_version 929584 (0.0009) [2023-12-26 22:08:08,468][105620] Updated weights for policy 1, policy_version 929748 (0.0010) [2023-12-26 22:08:08,526][105620] Updated weights for policy 1, policy_version 929758 (0.0010) [2023-12-26 22:08:08,592][105620] Updated weights for policy 1, policy_version 929768 (0.0010) [2023-12-26 22:08:09,128][105692] Updated weights for policy 0, policy_version 929594 (0.0006) [2023-12-26 22:08:09,191][105692] Updated weights for policy 0, policy_version 929604 (0.0006) [2023-12-26 22:08:09,256][105692] Updated weights for policy 0, policy_version 929614 (0.0008) [2023-12-26 22:08:09,358][105620] Updated weights for policy 1, policy_version 929778 (0.0010) [2023-12-26 22:08:09,427][105620] Updated weights for policy 1, policy_version 929788 (0.0009) [2023-12-26 22:08:09,480][105620] Updated weights for policy 1, policy_version 929798 (0.0009) [2023-12-26 22:08:09,531][105620] Updated weights for policy 1, policy_version 929808 (0.0009) [2023-12-26 22:08:09,950][105692] Updated weights for policy 0, policy_version 929624 (0.0008) [2023-12-26 22:08:10,016][105692] Updated weights for policy 0, policy_version 929634 (0.0008) [2023-12-26 22:08:10,081][105692] Updated weights for policy 0, policy_version 929644 (0.0008) [2023-12-26 22:08:10,320][105620] Updated weights for policy 1, policy_version 929818 (0.0009) [2023-12-26 22:08:10,382][105620] Updated weights for policy 1, policy_version 929828 (0.0009) [2023-12-26 22:08:10,445][105620] Updated weights for policy 1, policy_version 929838 (0.0009) [2023-12-26 22:08:10,829][105692] Updated weights for policy 0, policy_version 929654 (0.0009) [2023-12-26 22:08:10,900][105692] Updated weights for policy 0, policy_version 929664 (0.0009) [2023-12-26 22:08:10,965][105692] Updated weights for policy 0, policy_version 929674 (0.0009) [2023-12-26 22:08:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18841.6, 300 sec: 19077.6). Total num frames: 476102656. Throughput: 0: 9214.4, 1: 9624.2. Samples: 476110168. Policy #0 lag: (min: 11.0, avg: 15.5, max: 43.0) [2023-12-26 22:08:11,062][104569] Avg episode reward: [(0, '8998.625'), (1, '9043.166')] [2023-12-26 22:08:11,233][105620] Updated weights for policy 1, policy_version 929848 (0.0010) [2023-12-26 22:08:11,292][105620] Updated weights for policy 1, policy_version 929858 (0.0009) [2023-12-26 22:08:11,361][105620] Updated weights for policy 1, policy_version 929868 (0.0009) [2023-12-26 22:08:11,661][105692] Updated weights for policy 0, policy_version 929684 (0.0008) [2023-12-26 22:08:11,729][105692] Updated weights for policy 0, policy_version 929694 (0.0008) [2023-12-26 22:08:11,787][105692] Updated weights for policy 0, policy_version 929704 (0.0009) [2023-12-26 22:08:12,166][105620] Updated weights for policy 1, policy_version 929878 (0.0010) [2023-12-26 22:08:12,238][105620] Updated weights for policy 1, policy_version 929888 (0.0009) [2023-12-26 22:08:12,254][105586] KL-divergence is very high: 105.8721 [2023-12-26 22:08:12,312][105620] Updated weights for policy 1, policy_version 929898 (0.0009) [2023-12-26 22:08:12,314][105586] KL-divergence is very high: 110.2316 [2023-12-26 22:08:12,589][105692] Updated weights for policy 0, policy_version 929714 (0.0009) [2023-12-26 22:08:12,646][105692] Updated weights for policy 0, policy_version 929724 (0.0008) [2023-12-26 22:08:12,702][105692] Updated weights for policy 0, policy_version 929734 (0.0009) [2023-12-26 22:08:12,750][105692] Updated weights for policy 0, policy_version 929744 (0.0009) [2023-12-26 22:08:13,101][105620] Updated weights for policy 1, policy_version 929908 (0.0008) [2023-12-26 22:08:13,163][105620] Updated weights for policy 1, policy_version 929918 (0.0009) [2023-12-26 22:08:13,230][105620] Updated weights for policy 1, policy_version 929928 (0.0010) [2023-12-26 22:08:13,466][105692] Updated weights for policy 0, policy_version 929754 (0.0009) [2023-12-26 22:08:13,517][105692] Updated weights for policy 0, policy_version 929764 (0.0009) [2023-12-26 22:08:13,569][105692] Updated weights for policy 0, policy_version 929774 (0.0009) [2023-12-26 22:08:13,989][105620] Updated weights for policy 1, policy_version 929938 (0.0009) [2023-12-26 22:08:14,035][105620] Updated weights for policy 1, policy_version 929948 (0.0008) [2023-12-26 22:08:14,085][105620] Updated weights for policy 1, policy_version 929958 (0.0009) [2023-12-26 22:08:14,143][105620] Updated weights for policy 1, policy_version 929968 (0.0009) [2023-12-26 22:08:14,305][105692] Updated weights for policy 0, policy_version 929784 (0.0008) [2023-12-26 22:08:14,359][105692] Updated weights for policy 0, policy_version 929794 (0.0009) [2023-12-26 22:08:14,416][105692] Updated weights for policy 0, policy_version 929804 (0.0008) [2023-12-26 22:08:14,944][105620] Updated weights for policy 1, policy_version 929978 (0.0011) [2023-12-26 22:08:15,012][105620] Updated weights for policy 1, policy_version 929988 (0.0011) [2023-12-26 22:08:15,074][105620] Updated weights for policy 1, policy_version 929998 (0.0005) [2023-12-26 22:08:15,173][105692] Updated weights for policy 0, policy_version 929814 (0.0009) [2023-12-26 22:08:15,241][105692] Updated weights for policy 0, policy_version 929824 (0.0007) [2023-12-26 22:08:15,309][105692] Updated weights for policy 0, policy_version 929834 (0.0008) [2023-12-26 22:08:15,736][105620] Updated weights for policy 1, policy_version 930008 (0.0005) [2023-12-26 22:08:15,796][105620] Updated weights for policy 1, policy_version 930018 (0.0006) [2023-12-26 22:08:15,848][105620] Updated weights for policy 1, policy_version 930028 (0.0009) [2023-12-26 22:08:16,062][104569] Fps is (10 sec: 18022.5, 60 sec: 18841.7, 300 sec: 19077.6). Total num frames: 476192768. Throughput: 0: 9228.8, 1: 9538.9. Samples: 476164612. Policy #0 lag: (min: 11.0, avg: 15.5, max: 43.0) [2023-12-26 22:08:16,062][104569] Avg episode reward: [(0, '8764.037'), (1, '9127.960')] [2023-12-26 22:08:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000930032_238116864.pth... [2023-12-26 22:08:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000928912_237830144.pth [2023-12-26 22:08:16,079][105692] Updated weights for policy 0, policy_version 929844 (0.0008) [2023-12-26 22:08:16,133][105692] Updated weights for policy 0, policy_version 929854 (0.0009) [2023-12-26 22:08:16,194][105692] Updated weights for policy 0, policy_version 929864 (0.0008) [2023-12-26 22:08:16,240][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000929872_238084096.pth... [2023-12-26 22:08:16,244][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000928784_237805568.pth [2023-12-26 22:08:16,471][105620] Updated weights for policy 1, policy_version 930038 (0.0005) [2023-12-26 22:08:16,525][105620] Updated weights for policy 1, policy_version 930048 (0.0006) [2023-12-26 22:08:16,577][105620] Updated weights for policy 1, policy_version 930058 (0.0005) [2023-12-26 22:08:17,092][105692] Updated weights for policy 0, policy_version 929874 (0.0009) [2023-12-26 22:08:17,122][105620] Updated weights for policy 1, policy_version 930068 (0.0006) [2023-12-26 22:08:17,145][105692] Updated weights for policy 0, policy_version 929884 (0.0005) [2023-12-26 22:08:17,170][105620] Updated weights for policy 1, policy_version 930078 (0.0008) [2023-12-26 22:08:17,192][105692] Updated weights for policy 0, policy_version 929894 (0.0005) [2023-12-26 22:08:17,225][105620] Updated weights for policy 1, policy_version 930088 (0.0008) [2023-12-26 22:08:17,244][105692] Updated weights for policy 0, policy_version 929904 (0.0007) [2023-12-26 22:08:17,945][105620] Updated weights for policy 1, policy_version 930098 (0.0008) [2023-12-26 22:08:18,008][105620] Updated weights for policy 1, policy_version 930108 (0.0010) [2023-12-26 22:08:18,057][105692] Updated weights for policy 0, policy_version 929914 (0.0007) [2023-12-26 22:08:18,063][105620] Updated weights for policy 1, policy_version 930118 (0.0011) [2023-12-26 22:08:18,116][105620] Updated weights for policy 1, policy_version 930128 (0.0010) [2023-12-26 22:08:18,118][105692] Updated weights for policy 0, policy_version 929924 (0.0006) [2023-12-26 22:08:18,174][105692] Updated weights for policy 0, policy_version 929934 (0.0008) [2023-12-26 22:08:18,858][105620] Updated weights for policy 1, policy_version 930138 (0.0010) [2023-12-26 22:08:18,931][105620] Updated weights for policy 1, policy_version 930148 (0.0011) [2023-12-26 22:08:18,964][105692] Updated weights for policy 0, policy_version 929944 (0.0010) [2023-12-26 22:08:19,001][105620] Updated weights for policy 1, policy_version 930158 (0.0011) [2023-12-26 22:08:19,024][105692] Updated weights for policy 0, policy_version 929954 (0.0011) [2023-12-26 22:08:19,090][105692] Updated weights for policy 0, policy_version 929964 (0.0010) [2023-12-26 22:08:19,754][105620] Updated weights for policy 1, policy_version 930168 (0.0011) [2023-12-26 22:08:19,821][105620] Updated weights for policy 1, policy_version 930178 (0.0011) [2023-12-26 22:08:19,867][105692] Updated weights for policy 0, policy_version 929974 (0.0011) [2023-12-26 22:08:19,884][105620] Updated weights for policy 1, policy_version 930188 (0.0011) [2023-12-26 22:08:19,932][105692] Updated weights for policy 0, policy_version 929984 (0.0011) [2023-12-26 22:08:19,993][105692] Updated weights for policy 0, policy_version 929994 (0.0011) [2023-12-26 22:08:20,629][105692] Updated weights for policy 0, policy_version 930004 (0.0008) [2023-12-26 22:08:20,650][105620] Updated weights for policy 1, policy_version 930198 (0.0010) [2023-12-26 22:08:20,694][105692] Updated weights for policy 0, policy_version 930014 (0.0008) [2023-12-26 22:08:20,714][105620] Updated weights for policy 1, policy_version 930208 (0.0008) [2023-12-26 22:08:20,753][105692] Updated weights for policy 0, policy_version 930024 (0.0008) [2023-12-26 22:08:20,779][105620] Updated weights for policy 1, policy_version 930218 (0.0008) [2023-12-26 22:08:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 18841.6, 300 sec: 19077.6). Total num frames: 476291072. Throughput: 0: 9193.9, 1: 9627.0. Samples: 476278164. Policy #0 lag: (min: 11.0, avg: 15.5, max: 43.0) [2023-12-26 22:08:21,063][104569] Avg episode reward: [(0, '8575.541'), (1, '8858.938')] [2023-12-26 22:08:21,477][105692] Updated weights for policy 0, policy_version 930034 (0.0007) [2023-12-26 22:08:21,535][105620] Updated weights for policy 1, policy_version 930228 (0.0009) [2023-12-26 22:08:21,549][105692] Updated weights for policy 0, policy_version 930044 (0.0008) [2023-12-26 22:08:21,595][105620] Updated weights for policy 1, policy_version 930238 (0.0011) [2023-12-26 22:08:21,614][105692] Updated weights for policy 0, policy_version 930054 (0.0006) [2023-12-26 22:08:21,673][105620] Updated weights for policy 1, policy_version 930248 (0.0010) [2023-12-26 22:08:21,681][105692] Updated weights for policy 0, policy_version 930064 (0.0006) [2023-12-26 22:08:22,403][105692] Updated weights for policy 0, policy_version 930074 (0.0009) [2023-12-26 22:08:22,431][105620] Updated weights for policy 1, policy_version 930258 (0.0008) [2023-12-26 22:08:22,462][105692] Updated weights for policy 0, policy_version 930084 (0.0009) [2023-12-26 22:08:22,494][105620] Updated weights for policy 1, policy_version 930268 (0.0006) [2023-12-26 22:08:22,516][105692] Updated weights for policy 0, policy_version 930094 (0.0007) [2023-12-26 22:08:22,558][105620] Updated weights for policy 1, policy_version 930278 (0.0011) [2023-12-26 22:08:22,621][105620] Updated weights for policy 1, policy_version 930288 (0.0011) [2023-12-26 22:08:23,240][105620] Updated weights for policy 1, policy_version 930298 (0.0011) [2023-12-26 22:08:23,302][105620] Updated weights for policy 1, policy_version 930308 (0.0011) [2023-12-26 22:08:23,349][105692] Updated weights for policy 0, policy_version 930104 (0.0006) [2023-12-26 22:08:23,363][105620] Updated weights for policy 1, policy_version 930318 (0.0011) [2023-12-26 22:08:23,399][105692] Updated weights for policy 0, policy_version 930114 (0.0007) [2023-12-26 22:08:23,450][105692] Updated weights for policy 0, policy_version 930125 (0.0009) [2023-12-26 22:08:24,102][105692] Updated weights for policy 0, policy_version 930135 (0.0009) [2023-12-26 22:08:24,104][105620] Updated weights for policy 1, policy_version 930328 (0.0010) [2023-12-26 22:08:24,166][105692] Updated weights for policy 0, policy_version 930145 (0.0007) [2023-12-26 22:08:24,169][105620] Updated weights for policy 1, policy_version 930338 (0.0010) [2023-12-26 22:08:24,224][105692] Updated weights for policy 0, policy_version 930155 (0.0007) [2023-12-26 22:08:24,228][105620] Updated weights for policy 1, policy_version 930348 (0.0011) [2023-12-26 22:08:24,957][105692] Updated weights for policy 0, policy_version 930165 (0.0008) [2023-12-26 22:08:24,971][105620] Updated weights for policy 1, policy_version 930358 (0.0010) [2023-12-26 22:08:25,020][105692] Updated weights for policy 0, policy_version 930175 (0.0009) [2023-12-26 22:08:25,023][105620] Updated weights for policy 1, policy_version 930368 (0.0010) [2023-12-26 22:08:25,078][105620] Updated weights for policy 1, policy_version 930378 (0.0010) [2023-12-26 22:08:25,081][105692] Updated weights for policy 0, policy_version 930185 (0.0008) [2023-12-26 22:08:25,693][105692] Updated weights for policy 0, policy_version 930195 (0.0006) [2023-12-26 22:08:25,747][105692] Updated weights for policy 0, policy_version 930205 (0.0005) [2023-12-26 22:08:25,795][105692] Updated weights for policy 0, policy_version 930215 (0.0005) [2023-12-26 22:08:25,835][105620] Updated weights for policy 1, policy_version 930388 (0.0010) [2023-12-26 22:08:25,893][105620] Updated weights for policy 1, policy_version 930398 (0.0011) [2023-12-26 22:08:25,956][105620] Updated weights for policy 1, policy_version 930408 (0.0010) [2023-12-26 22:08:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 18978.1, 300 sec: 19105.4). Total num frames: 476389376. Throughput: 0: 9292.7, 1: 9597.8. Samples: 476393344. Policy #0 lag: (min: 11.0, avg: 15.5, max: 43.0) [2023-12-26 22:08:26,063][104569] Avg episode reward: [(0, '8625.725'), (1, '8764.513')] [2023-12-26 22:08:26,406][105692] Updated weights for policy 0, policy_version 930225 (0.0005) [2023-12-26 22:08:26,459][105692] Updated weights for policy 0, policy_version 930235 (0.0008) [2023-12-26 22:08:26,507][105692] Updated weights for policy 0, policy_version 930245 (0.0008) [2023-12-26 22:08:26,557][105692] Updated weights for policy 0, policy_version 930255 (0.0006) [2023-12-26 22:08:26,703][105620] Updated weights for policy 1, policy_version 930418 (0.0010) [2023-12-26 22:08:26,762][105620] Updated weights for policy 1, policy_version 930428 (0.0011) [2023-12-26 22:08:26,814][105620] Updated weights for policy 1, policy_version 930438 (0.0011) [2023-12-26 22:08:26,869][105620] Updated weights for policy 1, policy_version 930448 (0.0010) [2023-12-26 22:08:27,174][105692] Updated weights for policy 0, policy_version 930265 (0.0008) [2023-12-26 22:08:27,225][105692] Updated weights for policy 0, policy_version 930275 (0.0008) [2023-12-26 22:08:27,281][105692] Updated weights for policy 0, policy_version 930285 (0.0008) [2023-12-26 22:08:27,597][105620] Updated weights for policy 1, policy_version 930458 (0.0009) [2023-12-26 22:08:27,650][105620] Updated weights for policy 1, policy_version 930468 (0.0008) [2023-12-26 22:08:27,708][105620] Updated weights for policy 1, policy_version 930478 (0.0009) [2023-12-26 22:08:28,049][105692] Updated weights for policy 0, policy_version 930295 (0.0007) [2023-12-26 22:08:28,102][105692] Updated weights for policy 0, policy_version 930305 (0.0005) [2023-12-26 22:08:28,152][105692] Updated weights for policy 0, policy_version 930315 (0.0005) [2023-12-26 22:08:28,530][105620] Updated weights for policy 1, policy_version 930488 (0.0010) [2023-12-26 22:08:28,580][105620] Updated weights for policy 1, policy_version 930498 (0.0008) [2023-12-26 22:08:28,627][105620] Updated weights for policy 1, policy_version 930508 (0.0008) [2023-12-26 22:08:28,751][105692] Updated weights for policy 0, policy_version 930325 (0.0007) [2023-12-26 22:08:28,809][105692] Updated weights for policy 0, policy_version 930335 (0.0008) [2023-12-26 22:08:28,861][105692] Updated weights for policy 0, policy_version 930345 (0.0009) [2023-12-26 22:08:29,435][105620] Updated weights for policy 1, policy_version 930518 (0.0010) [2023-12-26 22:08:29,494][105620] Updated weights for policy 1, policy_version 930528 (0.0011) [2023-12-26 22:08:29,556][105620] Updated weights for policy 1, policy_version 930538 (0.0011) [2023-12-26 22:08:29,648][105692] Updated weights for policy 0, policy_version 930355 (0.0009) [2023-12-26 22:08:29,708][105692] Updated weights for policy 0, policy_version 930365 (0.0008) [2023-12-26 22:08:29,766][105692] Updated weights for policy 0, policy_version 930375 (0.0008) [2023-12-26 22:08:30,298][105620] Updated weights for policy 1, policy_version 930548 (0.0010) [2023-12-26 22:08:30,352][105620] Updated weights for policy 1, policy_version 930558 (0.0010) [2023-12-26 22:08:30,414][105620] Updated weights for policy 1, policy_version 930568 (0.0010) [2023-12-26 22:08:30,518][105692] Updated weights for policy 0, policy_version 930385 (0.0008) [2023-12-26 22:08:30,586][105692] Updated weights for policy 0, policy_version 930395 (0.0005) [2023-12-26 22:08:30,637][105692] Updated weights for policy 0, policy_version 930405 (0.0008) [2023-12-26 22:08:30,688][105692] Updated weights for policy 0, policy_version 930415 (0.0010) [2023-12-26 22:08:31,008][105620] Updated weights for policy 1, policy_version 930578 (0.0010) [2023-12-26 22:08:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18841.6, 300 sec: 19077.6). Total num frames: 476479488. Throughput: 0: 9416.0, 1: 9607.8. Samples: 476453288. Policy #0 lag: (min: 11.0, avg: 15.5, max: 43.0) [2023-12-26 22:08:31,062][104569] Avg episode reward: [(0, '8990.750'), (1, '8847.927')] [2023-12-26 22:08:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000930416_238223360.pth... [2023-12-26 22:08:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000929296_237936640.pth [2023-12-26 22:08:31,072][105620] Updated weights for policy 1, policy_version 930588 (0.0008) [2023-12-26 22:08:31,136][105620] Updated weights for policy 1, policy_version 930598 (0.0008) [2023-12-26 22:08:31,198][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000930608_238264320.pth... [2023-12-26 22:08:31,200][105620] Updated weights for policy 1, policy_version 930608 (0.0008) [2023-12-26 22:08:31,203][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000929488_237977600.pth [2023-12-26 22:08:31,348][105692] Updated weights for policy 0, policy_version 930425 (0.0009) [2023-12-26 22:08:31,414][105692] Updated weights for policy 0, policy_version 930435 (0.0006) [2023-12-26 22:08:31,468][105692] Updated weights for policy 0, policy_version 930445 (0.0005) [2023-12-26 22:08:31,899][105620] Updated weights for policy 1, policy_version 930618 (0.0009) [2023-12-26 22:08:31,959][105620] Updated weights for policy 1, policy_version 930629 (0.0009) [2023-12-26 22:08:32,027][105620] Updated weights for policy 1, policy_version 930639 (0.0008) [2023-12-26 22:08:32,224][105692] Updated weights for policy 0, policy_version 930456 (0.0009) [2023-12-26 22:08:32,286][105692] Updated weights for policy 0, policy_version 930467 (0.0009) [2023-12-26 22:08:32,348][105692] Updated weights for policy 0, policy_version 930477 (0.0009) [2023-12-26 22:08:32,671][105620] Updated weights for policy 1, policy_version 930649 (0.0006) [2023-12-26 22:08:32,725][105620] Updated weights for policy 1, policy_version 930659 (0.0007) [2023-12-26 22:08:32,785][105620] Updated weights for policy 1, policy_version 930669 (0.0009) [2023-12-26 22:08:33,136][105692] Updated weights for policy 0, policy_version 930487 (0.0008) [2023-12-26 22:08:33,187][105692] Updated weights for policy 0, policy_version 930497 (0.0008) [2023-12-26 22:08:33,237][105692] Updated weights for policy 0, policy_version 930508 (0.0007) [2023-12-26 22:08:33,549][105620] Updated weights for policy 1, policy_version 930679 (0.0009) [2023-12-26 22:08:33,596][105620] Updated weights for policy 1, policy_version 930689 (0.0009) [2023-12-26 22:08:33,642][105620] Updated weights for policy 1, policy_version 930699 (0.0008) [2023-12-26 22:08:33,936][105692] Updated weights for policy 0, policy_version 930518 (0.0007) [2023-12-26 22:08:33,990][105692] Updated weights for policy 0, policy_version 930528 (0.0006) [2023-12-26 22:08:34,054][105692] Updated weights for policy 0, policy_version 930538 (0.0005) [2023-12-26 22:08:34,314][105620] Updated weights for policy 1, policy_version 930709 (0.0009) [2023-12-26 22:08:34,375][105620] Updated weights for policy 1, policy_version 930719 (0.0007) [2023-12-26 22:08:34,436][105620] Updated weights for policy 1, policy_version 930729 (0.0006) [2023-12-26 22:08:34,654][105692] Updated weights for policy 0, policy_version 930548 (0.0008) [2023-12-26 22:08:34,712][105692] Updated weights for policy 0, policy_version 930558 (0.0010) [2023-12-26 22:08:34,768][105692] Updated weights for policy 0, policy_version 930568 (0.0010) [2023-12-26 22:08:35,211][105620] Updated weights for policy 1, policy_version 930739 (0.0008) [2023-12-26 22:08:35,271][105620] Updated weights for policy 1, policy_version 930749 (0.0008) [2023-12-26 22:08:35,335][105620] Updated weights for policy 1, policy_version 930759 (0.0009) [2023-12-26 22:08:35,352][105692] Updated weights for policy 0, policy_version 930578 (0.0010) [2023-12-26 22:08:35,403][105692] Updated weights for policy 0, policy_version 930588 (0.0007) [2023-12-26 22:08:35,451][105692] Updated weights for policy 0, policy_version 930598 (0.0008) [2023-12-26 22:08:35,510][105692] Updated weights for policy 0, policy_version 930608 (0.0009) [2023-12-26 22:08:36,062][104569] Fps is (10 sec: 18842.0, 60 sec: 18978.1, 300 sec: 19077.6). Total num frames: 476577792. Throughput: 0: 9420.7, 1: 9618.8. Samples: 476570352. Policy #0 lag: (min: 11.0, avg: 15.5, max: 43.0) [2023-12-26 22:08:36,062][104569] Avg episode reward: [(0, '9258.141'), (1, '8928.653')] [2023-12-26 22:08:36,073][105620] Updated weights for policy 1, policy_version 930769 (0.0008) [2023-12-26 22:08:36,149][105620] Updated weights for policy 1, policy_version 930779 (0.0006) [2023-12-26 22:08:36,212][105620] Updated weights for policy 1, policy_version 930789 (0.0006) [2023-12-26 22:08:36,276][105620] Updated weights for policy 1, policy_version 930799 (0.0009) [2023-12-26 22:08:36,319][105692] Updated weights for policy 0, policy_version 930618 (0.0009) [2023-12-26 22:08:36,393][105692] Updated weights for policy 0, policy_version 930628 (0.0009) [2023-12-26 22:08:36,461][105692] Updated weights for policy 0, policy_version 930638 (0.0009) [2023-12-26 22:08:37,008][105620] Updated weights for policy 1, policy_version 930809 (0.0009) [2023-12-26 22:08:37,065][105620] Updated weights for policy 1, policy_version 930819 (0.0009) [2023-12-26 22:08:37,116][105620] Updated weights for policy 1, policy_version 930829 (0.0009) [2023-12-26 22:08:37,192][105692] Updated weights for policy 0, policy_version 930648 (0.0009) [2023-12-26 22:08:37,244][105692] Updated weights for policy 0, policy_version 930658 (0.0009) [2023-12-26 22:08:37,296][105692] Updated weights for policy 0, policy_version 930668 (0.0009) [2023-12-26 22:08:37,781][105620] Updated weights for policy 1, policy_version 930839 (0.0006) [2023-12-26 22:08:37,831][105620] Updated weights for policy 1, policy_version 930849 (0.0005) [2023-12-26 22:08:37,899][105620] Updated weights for policy 1, policy_version 930859 (0.0005) [2023-12-26 22:08:38,168][105692] Updated weights for policy 0, policy_version 930678 (0.0009) [2023-12-26 22:08:38,234][105692] Updated weights for policy 0, policy_version 930688 (0.0010) [2023-12-26 22:08:38,293][105692] Updated weights for policy 0, policy_version 930699 (0.0010) [2023-12-26 22:08:38,451][105620] Updated weights for policy 1, policy_version 930869 (0.0008) [2023-12-26 22:08:38,516][105620] Updated weights for policy 1, policy_version 930879 (0.0010) [2023-12-26 22:08:38,573][105620] Updated weights for policy 1, policy_version 930889 (0.0005) [2023-12-26 22:08:39,049][105692] Updated weights for policy 0, policy_version 930709 (0.0010) [2023-12-26 22:08:39,107][105692] Updated weights for policy 0, policy_version 930719 (0.0009) [2023-12-26 22:08:39,161][105692] Updated weights for policy 0, policy_version 930729 (0.0009) [2023-12-26 22:08:39,321][105620] Updated weights for policy 1, policy_version 930899 (0.0010) [2023-12-26 22:08:39,393][105620] Updated weights for policy 1, policy_version 930909 (0.0007) [2023-12-26 22:08:39,462][105620] Updated weights for policy 1, policy_version 930919 (0.0010) [2023-12-26 22:08:39,950][105692] Updated weights for policy 0, policy_version 930739 (0.0008) [2023-12-26 22:08:40,015][105692] Updated weights for policy 0, policy_version 930749 (0.0008) [2023-12-26 22:08:40,086][105692] Updated weights for policy 0, policy_version 930759 (0.0008) [2023-12-26 22:08:40,217][105620] Updated weights for policy 1, policy_version 930929 (0.0011) [2023-12-26 22:08:40,287][105620] Updated weights for policy 1, policy_version 930939 (0.0011) [2023-12-26 22:08:40,359][105620] Updated weights for policy 1, policy_version 930949 (0.0011) [2023-12-26 22:08:40,428][105620] Updated weights for policy 1, policy_version 930959 (0.0010) [2023-12-26 22:08:40,831][105692] Updated weights for policy 0, policy_version 930769 (0.0007) [2023-12-26 22:08:40,891][105692] Updated weights for policy 0, policy_version 930779 (0.0009) [2023-12-26 22:08:40,951][105692] Updated weights for policy 0, policy_version 930789 (0.0009) [2023-12-26 22:08:41,009][105692] Updated weights for policy 0, policy_version 930799 (0.0010) [2023-12-26 22:08:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 18978.1, 300 sec: 19105.4). Total num frames: 476676096. Throughput: 0: 9450.0, 1: 9586.6. Samples: 476684396. Policy #0 lag: (min: 11.0, avg: 15.5, max: 43.0) [2023-12-26 22:08:41,062][104569] Avg episode reward: [(0, '9348.970'), (1, '8764.018')] [2023-12-26 22:08:41,095][105620] Updated weights for policy 1, policy_version 930969 (0.0008) [2023-12-26 22:08:41,168][105620] Updated weights for policy 1, policy_version 930979 (0.0009) [2023-12-26 22:08:41,217][105620] Updated weights for policy 1, policy_version 930989 (0.0009) [2023-12-26 22:08:41,913][105692] Updated weights for policy 0, policy_version 930809 (0.0008) [2023-12-26 22:08:41,981][105692] Updated weights for policy 0, policy_version 930819 (0.0008) [2023-12-26 22:08:41,987][105620] Updated weights for policy 1, policy_version 930999 (0.0008) [2023-12-26 22:08:42,040][105692] Updated weights for policy 0, policy_version 930829 (0.0006) [2023-12-26 22:08:42,046][105620] Updated weights for policy 1, policy_version 931009 (0.0008) [2023-12-26 22:08:42,107][105620] Updated weights for policy 1, policy_version 931019 (0.0009) [2023-12-26 22:08:42,809][105692] Updated weights for policy 0, policy_version 930839 (0.0008) [2023-12-26 22:08:42,867][105692] Updated weights for policy 0, policy_version 930849 (0.0008) [2023-12-26 22:08:42,878][105620] Updated weights for policy 1, policy_version 931029 (0.0009) [2023-12-26 22:08:42,926][105692] Updated weights for policy 0, policy_version 930859 (0.0006) [2023-12-26 22:08:42,940][105620] Updated weights for policy 1, policy_version 931039 (0.0008) [2023-12-26 22:08:43,000][105620] Updated weights for policy 1, policy_version 931049 (0.0008) [2023-12-26 22:08:43,683][105692] Updated weights for policy 0, policy_version 930869 (0.0007) [2023-12-26 22:08:43,737][105692] Updated weights for policy 0, policy_version 930879 (0.0006) [2023-12-26 22:08:43,768][105620] Updated weights for policy 1, policy_version 931059 (0.0008) [2023-12-26 22:08:43,795][105692] Updated weights for policy 0, policy_version 930889 (0.0007) [2023-12-26 22:08:43,828][105620] Updated weights for policy 1, policy_version 931069 (0.0006) [2023-12-26 22:08:43,881][105620] Updated weights for policy 1, policy_version 931079 (0.0008) [2023-12-26 22:08:44,403][105692] Updated weights for policy 0, policy_version 930899 (0.0007) [2023-12-26 22:08:44,457][105692] Updated weights for policy 0, policy_version 930910 (0.0010) [2023-12-26 22:08:44,516][105692] Updated weights for policy 0, policy_version 930920 (0.0009) [2023-12-26 22:08:44,673][105620] Updated weights for policy 1, policy_version 931089 (0.0009) [2023-12-26 22:08:44,733][105620] Updated weights for policy 1, policy_version 931099 (0.0009) [2023-12-26 22:08:44,792][105620] Updated weights for policy 1, policy_version 931109 (0.0009) [2023-12-26 22:08:44,858][105620] Updated weights for policy 1, policy_version 931119 (0.0008) [2023-12-26 22:08:45,302][105692] Updated weights for policy 0, policy_version 930930 (0.0009) [2023-12-26 22:08:45,367][105692] Updated weights for policy 0, policy_version 930940 (0.0009) [2023-12-26 22:08:45,427][105692] Updated weights for policy 0, policy_version 930950 (0.0009) [2023-12-26 22:08:45,488][105692] Updated weights for policy 0, policy_version 930960 (0.0009) [2023-12-26 22:08:45,651][105620] Updated weights for policy 1, policy_version 931129 (0.0009) [2023-12-26 22:08:45,717][105620] Updated weights for policy 1, policy_version 931139 (0.0009) [2023-12-26 22:08:45,764][105620] Updated weights for policy 1, policy_version 931149 (0.0010) [2023-12-26 22:08:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 18978.2, 300 sec: 19077.6). Total num frames: 476766208. Throughput: 0: 9454.1, 1: 9532.3. Samples: 476737912. Policy #0 lag: (min: 11.0, avg: 15.5, max: 43.0) [2023-12-26 22:08:46,062][104569] Avg episode reward: [(0, '9259.659'), (1, '8686.346')] [2023-12-26 22:08:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000930960_238362624.pth... [2023-12-26 22:08:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000931152_238403584.pth... [2023-12-26 22:08:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000929872_238084096.pth [2023-12-26 22:08:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000930032_238116864.pth [2023-12-26 22:08:46,253][105692] Updated weights for policy 0, policy_version 930970 (0.0009) [2023-12-26 22:08:46,297][105692] Updated weights for policy 0, policy_version 930980 (0.0008) [2023-12-26 22:08:46,349][105692] Updated weights for policy 0, policy_version 930990 (0.0008) [2023-12-26 22:08:46,490][105620] Updated weights for policy 1, policy_version 931159 (0.0007) [2023-12-26 22:08:46,549][105620] Updated weights for policy 1, policy_version 931169 (0.0008) [2023-12-26 22:08:46,615][105620] Updated weights for policy 1, policy_version 931179 (0.0010) [2023-12-26 22:08:46,984][105692] Updated weights for policy 0, policy_version 931000 (0.0006) [2023-12-26 22:08:47,046][105692] Updated weights for policy 0, policy_version 931010 (0.0010) [2023-12-26 22:08:47,109][105692] Updated weights for policy 0, policy_version 931020 (0.0010) [2023-12-26 22:08:47,178][105620] Updated weights for policy 1, policy_version 931189 (0.0008) [2023-12-26 22:08:47,239][105620] Updated weights for policy 1, policy_version 931199 (0.0006) [2023-12-26 22:08:47,298][105620] Updated weights for policy 1, policy_version 931209 (0.0006) [2023-12-26 22:08:47,795][105692] Updated weights for policy 0, policy_version 931030 (0.0007) [2023-12-26 22:08:47,853][105692] Updated weights for policy 0, policy_version 931040 (0.0005) [2023-12-26 22:08:47,903][105620] Updated weights for policy 1, policy_version 931219 (0.0011) [2023-12-26 22:08:47,909][105692] Updated weights for policy 0, policy_version 931050 (0.0006) [2023-12-26 22:08:47,957][105620] Updated weights for policy 1, policy_version 931229 (0.0009) [2023-12-26 22:08:48,012][105620] Updated weights for policy 1, policy_version 931239 (0.0005) [2023-12-26 22:08:48,576][105692] Updated weights for policy 0, policy_version 931060 (0.0008) [2023-12-26 22:08:48,636][105692] Updated weights for policy 0, policy_version 931070 (0.0005) [2023-12-26 22:08:48,699][105692] Updated weights for policy 0, policy_version 931080 (0.0006) [2023-12-26 22:08:48,717][105620] Updated weights for policy 1, policy_version 931249 (0.0007) [2023-12-26 22:08:48,771][105620] Updated weights for policy 1, policy_version 931259 (0.0009) [2023-12-26 22:08:48,833][105620] Updated weights for policy 1, policy_version 931269 (0.0009) [2023-12-26 22:08:48,892][105620] Updated weights for policy 1, policy_version 931279 (0.0009) [2023-12-26 22:08:49,367][105692] Updated weights for policy 0, policy_version 931090 (0.0007) [2023-12-26 22:08:49,423][105692] Updated weights for policy 0, policy_version 931100 (0.0009) [2023-12-26 22:08:49,478][105692] Updated weights for policy 0, policy_version 931110 (0.0009) [2023-12-26 22:08:49,537][105692] Updated weights for policy 0, policy_version 931120 (0.0009) [2023-12-26 22:08:49,715][105620] Updated weights for policy 1, policy_version 931289 (0.0010) [2023-12-26 22:08:49,773][105620] Updated weights for policy 1, policy_version 931299 (0.0006) [2023-12-26 22:08:49,832][105620] Updated weights for policy 1, policy_version 931309 (0.0006) [2023-12-26 22:08:50,304][105692] Updated weights for policy 0, policy_version 931130 (0.0010) [2023-12-26 22:08:50,358][105692] Updated weights for policy 0, policy_version 931141 (0.0010) [2023-12-26 22:08:50,406][105692] Updated weights for policy 0, policy_version 931151 (0.0007) [2023-12-26 22:08:50,440][105620] Updated weights for policy 1, policy_version 931319 (0.0007) [2023-12-26 22:08:50,497][105620] Updated weights for policy 1, policy_version 931329 (0.0005) [2023-12-26 22:08:50,557][105620] Updated weights for policy 1, policy_version 931339 (0.0007) [2023-12-26 22:08:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18978.2, 300 sec: 19077.6). Total num frames: 476864512. Throughput: 0: 9557.9, 1: 9522.3. Samples: 476856296. Policy #0 lag: (min: 11.0, avg: 15.5, max: 43.0) [2023-12-26 22:08:51,063][104569] Avg episode reward: [(0, '8991.501'), (1, '8932.557')] [2023-12-26 22:08:51,213][105620] Updated weights for policy 1, policy_version 931349 (0.0009) [2023-12-26 22:08:51,251][105692] Updated weights for policy 0, policy_version 931161 (0.0006) [2023-12-26 22:08:51,276][105620] Updated weights for policy 1, policy_version 931359 (0.0012) [2023-12-26 22:08:51,318][105692] Updated weights for policy 0, policy_version 931171 (0.0006) [2023-12-26 22:08:51,335][105620] Updated weights for policy 1, policy_version 931369 (0.0010) [2023-12-26 22:08:51,383][105692] Updated weights for policy 0, policy_version 931181 (0.0008) [2023-12-26 22:08:52,068][105692] Updated weights for policy 0, policy_version 931191 (0.0008) [2023-12-26 22:08:52,101][105620] Updated weights for policy 1, policy_version 931379 (0.0010) [2023-12-26 22:08:52,136][105692] Updated weights for policy 0, policy_version 931201 (0.0007) [2023-12-26 22:08:52,159][105620] Updated weights for policy 1, policy_version 931389 (0.0011) [2023-12-26 22:08:52,205][105692] Updated weights for policy 0, policy_version 931211 (0.0006) [2023-12-26 22:08:52,215][105620] Updated weights for policy 1, policy_version 931399 (0.0011) [2023-12-26 22:08:52,945][105692] Updated weights for policy 0, policy_version 931221 (0.0007) [2023-12-26 22:08:52,987][105620] Updated weights for policy 1, policy_version 931409 (0.0010) [2023-12-26 22:08:53,001][105692] Updated weights for policy 0, policy_version 931231 (0.0008) [2023-12-26 22:08:53,052][105620] Updated weights for policy 1, policy_version 931419 (0.0007) [2023-12-26 22:08:53,064][105692] Updated weights for policy 0, policy_version 931241 (0.0009) [2023-12-26 22:08:53,110][105620] Updated weights for policy 1, policy_version 931429 (0.0007) [2023-12-26 22:08:53,172][105620] Updated weights for policy 1, policy_version 931439 (0.0005) [2023-12-26 22:08:53,645][105692] Updated weights for policy 0, policy_version 931251 (0.0008) [2023-12-26 22:08:53,709][105692] Updated weights for policy 0, policy_version 931261 (0.0007) [2023-12-26 22:08:53,779][105692] Updated weights for policy 0, policy_version 931271 (0.0005) [2023-12-26 22:08:53,842][105620] Updated weights for policy 1, policy_version 931449 (0.0009) [2023-12-26 22:08:53,895][105620] Updated weights for policy 1, policy_version 931459 (0.0010) [2023-12-26 22:08:53,953][105620] Updated weights for policy 1, policy_version 931469 (0.0010) [2023-12-26 22:08:54,397][105692] Updated weights for policy 0, policy_version 931281 (0.0006) [2023-12-26 22:08:54,446][105692] Updated weights for policy 0, policy_version 931291 (0.0009) [2023-12-26 22:08:54,499][105692] Updated weights for policy 0, policy_version 931301 (0.0009) [2023-12-26 22:08:54,550][105692] Updated weights for policy 0, policy_version 931311 (0.0009) [2023-12-26 22:08:54,729][105620] Updated weights for policy 1, policy_version 931479 (0.0007) [2023-12-26 22:08:54,785][105620] Updated weights for policy 1, policy_version 931489 (0.0008) [2023-12-26 22:08:54,839][105620] Updated weights for policy 1, policy_version 931499 (0.0009) [2023-12-26 22:08:55,390][105692] Updated weights for policy 0, policy_version 931321 (0.0009) [2023-12-26 22:08:55,452][105692] Updated weights for policy 0, policy_version 931331 (0.0009) [2023-12-26 22:08:55,513][105692] Updated weights for policy 0, policy_version 931341 (0.0008) [2023-12-26 22:08:55,519][105620] Updated weights for policy 1, policy_version 931509 (0.0007) [2023-12-26 22:08:55,567][105620] Updated weights for policy 1, policy_version 931519 (0.0008) [2023-12-26 22:08:55,623][105620] Updated weights for policy 1, policy_version 931529 (0.0008) [2023-12-26 22:08:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 18978.2, 300 sec: 19077.6). Total num frames: 476962816. Throughput: 0: 9550.1, 1: 9615.3. Samples: 476972612. Policy #0 lag: (min: 11.0, avg: 15.5, max: 43.0) [2023-12-26 22:08:56,063][104569] Avg episode reward: [(0, '8998.960'), (1, '9185.862')] [2023-12-26 22:08:56,269][105692] Updated weights for policy 0, policy_version 931351 (0.0009) [2023-12-26 22:08:56,331][105692] Updated weights for policy 0, policy_version 931361 (0.0008) [2023-12-26 22:08:56,391][105692] Updated weights for policy 0, policy_version 931371 (0.0009) [2023-12-26 22:08:56,397][105620] Updated weights for policy 1, policy_version 931539 (0.0008) [2023-12-26 22:08:56,453][105620] Updated weights for policy 1, policy_version 931549 (0.0008) [2023-12-26 22:08:56,511][105620] Updated weights for policy 1, policy_version 931559 (0.0007) [2023-12-26 22:08:57,157][105692] Updated weights for policy 0, policy_version 931381 (0.0009) [2023-12-26 22:08:57,186][105620] Updated weights for policy 1, policy_version 931569 (0.0005) [2023-12-26 22:08:57,211][105692] Updated weights for policy 0, policy_version 931391 (0.0010) [2023-12-26 22:08:57,248][105620] Updated weights for policy 1, policy_version 931579 (0.0005) [2023-12-26 22:08:57,269][105692] Updated weights for policy 0, policy_version 931401 (0.0010) [2023-12-26 22:08:57,307][105620] Updated weights for policy 1, policy_version 931589 (0.0006) [2023-12-26 22:08:57,366][105620] Updated weights for policy 1, policy_version 931599 (0.0008) [2023-12-26 22:08:57,916][105692] Updated weights for policy 0, policy_version 931411 (0.0009) [2023-12-26 22:08:57,972][105692] Updated weights for policy 0, policy_version 931421 (0.0006) [2023-12-26 22:08:58,030][105692] Updated weights for policy 0, policy_version 931431 (0.0007) [2023-12-26 22:08:58,157][105620] Updated weights for policy 1, policy_version 931609 (0.0009) [2023-12-26 22:08:58,219][105620] Updated weights for policy 1, policy_version 931619 (0.0008) [2023-12-26 22:08:58,284][105620] Updated weights for policy 1, policy_version 931629 (0.0008) [2023-12-26 22:08:58,781][105692] Updated weights for policy 0, policy_version 931441 (0.0008) [2023-12-26 22:08:58,841][105692] Updated weights for policy 0, policy_version 931451 (0.0009) [2023-12-26 22:08:58,906][105692] Updated weights for policy 0, policy_version 931461 (0.0008) [2023-12-26 22:08:58,969][105692] Updated weights for policy 0, policy_version 931471 (0.0008) [2023-12-26 22:08:59,092][105620] Updated weights for policy 1, policy_version 931639 (0.0010) [2023-12-26 22:08:59,147][105620] Updated weights for policy 1, policy_version 931649 (0.0010) [2023-12-26 22:08:59,221][105620] Updated weights for policy 1, policy_version 931660 (0.0008) [2023-12-26 22:08:59,784][105692] Updated weights for policy 0, policy_version 931481 (0.0009) [2023-12-26 22:08:59,839][105692] Updated weights for policy 0, policy_version 931491 (0.0009) [2023-12-26 22:08:59,895][105692] Updated weights for policy 0, policy_version 931501 (0.0008) [2023-12-26 22:09:00,014][105620] Updated weights for policy 1, policy_version 931670 (0.0007) [2023-12-26 22:09:00,069][105620] Updated weights for policy 1, policy_version 931680 (0.0007) [2023-12-26 22:09:00,136][105620] Updated weights for policy 1, policy_version 931690 (0.0007) [2023-12-26 22:09:00,573][105692] Updated weights for policy 0, policy_version 931511 (0.0008) [2023-12-26 22:09:00,623][105692] Updated weights for policy 0, policy_version 931521 (0.0009) [2023-12-26 22:09:00,674][105692] Updated weights for policy 0, policy_version 931531 (0.0009) [2023-12-26 22:09:00,851][105620] Updated weights for policy 1, policy_version 931700 (0.0007) [2023-12-26 22:09:00,908][105620] Updated weights for policy 1, policy_version 931710 (0.0010) [2023-12-26 22:09:00,973][105620] Updated weights for policy 1, policy_version 931720 (0.0010) [2023-12-26 22:09:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19114.6, 300 sec: 19077.6). Total num frames: 477061120. Throughput: 0: 9580.6, 1: 9640.6. Samples: 477029568. Policy #0 lag: (min: 11.0, avg: 15.5, max: 43.0) [2023-12-26 22:09:01,063][104569] Avg episode reward: [(0, '8816.834'), (1, '9270.056')] [2023-12-26 22:09:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000931536_238510080.pth... [2023-12-26 22:09:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000931728_238551040.pth... [2023-12-26 22:09:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000930416_238223360.pth [2023-12-26 22:09:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000930608_238264320.pth [2023-12-26 22:09:01,473][105692] Updated weights for policy 0, policy_version 931541 (0.0009) [2023-12-26 22:09:01,529][105692] Updated weights for policy 0, policy_version 931551 (0.0008) [2023-12-26 22:09:01,580][105692] Updated weights for policy 0, policy_version 931561 (0.0009) [2023-12-26 22:09:01,694][105620] Updated weights for policy 1, policy_version 931730 (0.0010) [2023-12-26 22:09:01,755][105620] Updated weights for policy 1, policy_version 931740 (0.0009) [2023-12-26 22:09:01,803][105620] Updated weights for policy 1, policy_version 931750 (0.0009) [2023-12-26 22:09:02,260][105692] Updated weights for policy 0, policy_version 931571 (0.0008) [2023-12-26 22:09:02,327][105692] Updated weights for policy 0, policy_version 931581 (0.0006) [2023-12-26 22:09:02,395][105692] Updated weights for policy 0, policy_version 931591 (0.0008) [2023-12-26 22:09:02,421][105620] Updated weights for policy 1, policy_version 931761 (0.0009) [2023-12-26 22:09:02,485][105620] Updated weights for policy 1, policy_version 931771 (0.0008) [2023-12-26 22:09:02,551][105620] Updated weights for policy 1, policy_version 931781 (0.0009) [2023-12-26 22:09:02,612][105620] Updated weights for policy 1, policy_version 931791 (0.0009) [2023-12-26 22:09:03,108][105692] Updated weights for policy 0, policy_version 931601 (0.0007) [2023-12-26 22:09:03,157][105692] Updated weights for policy 0, policy_version 931611 (0.0009) [2023-12-26 22:09:03,209][105692] Updated weights for policy 0, policy_version 931621 (0.0009) [2023-12-26 22:09:03,264][105692] Updated weights for policy 0, policy_version 931631 (0.0008) [2023-12-26 22:09:03,290][105620] Updated weights for policy 1, policy_version 931801 (0.0007) [2023-12-26 22:09:03,340][105620] Updated weights for policy 1, policy_version 931811 (0.0009) [2023-12-26 22:09:03,400][105620] Updated weights for policy 1, policy_version 931821 (0.0008) [2023-12-26 22:09:03,971][105620] Updated weights for policy 1, policy_version 931831 (0.0008) [2023-12-26 22:09:04,022][105620] Updated weights for policy 1, policy_version 931841 (0.0009) [2023-12-26 22:09:04,082][105620] Updated weights for policy 1, policy_version 931851 (0.0008) [2023-12-26 22:09:04,120][105692] Updated weights for policy 0, policy_version 931641 (0.0008) [2023-12-26 22:09:04,183][105692] Updated weights for policy 0, policy_version 931651 (0.0009) [2023-12-26 22:09:04,237][105692] Updated weights for policy 0, policy_version 931661 (0.0009) [2023-12-26 22:09:04,844][105620] Updated weights for policy 1, policy_version 931861 (0.0007) [2023-12-26 22:09:04,899][105620] Updated weights for policy 1, policy_version 931871 (0.0008) [2023-12-26 22:09:04,946][105692] Updated weights for policy 0, policy_version 931671 (0.0008) [2023-12-26 22:09:04,957][105620] Updated weights for policy 1, policy_version 931881 (0.0009) [2023-12-26 22:09:05,003][105692] Updated weights for policy 0, policy_version 931681 (0.0007) [2023-12-26 22:09:05,051][105692] Updated weights for policy 0, policy_version 931691 (0.0008) [2023-12-26 22:09:05,632][105620] Updated weights for policy 1, policy_version 931891 (0.0009) [2023-12-26 22:09:05,696][105620] Updated weights for policy 1, policy_version 931901 (0.0008) [2023-12-26 22:09:05,732][105692] Updated weights for policy 0, policy_version 931701 (0.0007) [2023-12-26 22:09:05,758][105620] Updated weights for policy 1, policy_version 931911 (0.0008) [2023-12-26 22:09:05,794][105692] Updated weights for policy 0, policy_version 931711 (0.0006) [2023-12-26 22:09:05,845][105692] Updated weights for policy 0, policy_version 931721 (0.0005) [2023-12-26 22:09:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19114.7, 300 sec: 19105.4). Total num frames: 477159424. Throughput: 0: 9620.6, 1: 9616.6. Samples: 477143840. Policy #0 lag: (min: 11.0, avg: 15.5, max: 43.0) [2023-12-26 22:09:06,063][104569] Avg episode reward: [(0, '8721.812'), (1, '9270.019')] [2023-12-26 22:09:06,402][105692] Updated weights for policy 0, policy_version 931731 (0.0006) [2023-12-26 22:09:06,471][105692] Updated weights for policy 0, policy_version 931741 (0.0011) [2023-12-26 22:09:06,475][105620] Updated weights for policy 1, policy_version 931921 (0.0007) [2023-12-26 22:09:06,532][105692] Updated weights for policy 0, policy_version 931751 (0.0010) [2023-12-26 22:09:06,545][105620] Updated weights for policy 1, policy_version 931931 (0.0007) [2023-12-26 22:09:06,612][105620] Updated weights for policy 1, policy_version 931941 (0.0007) [2023-12-26 22:09:06,673][105620] Updated weights for policy 1, policy_version 931951 (0.0008) [2023-12-26 22:09:07,257][105692] Updated weights for policy 0, policy_version 931761 (0.0011) [2023-12-26 22:09:07,306][105692] Updated weights for policy 0, policy_version 931771 (0.0011) [2023-12-26 22:09:07,344][105620] Updated weights for policy 1, policy_version 931961 (0.0007) [2023-12-26 22:09:07,359][105692] Updated weights for policy 0, policy_version 931781 (0.0010) [2023-12-26 22:09:07,400][105620] Updated weights for policy 1, policy_version 931971 (0.0005) [2023-12-26 22:09:07,422][105692] Updated weights for policy 0, policy_version 931791 (0.0011) [2023-12-26 22:09:07,458][105620] Updated weights for policy 1, policy_version 931981 (0.0007) [2023-12-26 22:09:08,181][105620] Updated weights for policy 1, policy_version 931991 (0.0006) [2023-12-26 22:09:08,184][105692] Updated weights for policy 0, policy_version 931801 (0.0011) [2023-12-26 22:09:08,240][105620] Updated weights for policy 1, policy_version 932001 (0.0006) [2023-12-26 22:09:08,243][105692] Updated weights for policy 0, policy_version 931811 (0.0011) [2023-12-26 22:09:08,295][105620] Updated weights for policy 1, policy_version 932011 (0.0006) [2023-12-26 22:09:08,298][105692] Updated weights for policy 0, policy_version 931821 (0.0010) [2023-12-26 22:09:08,887][105692] Updated weights for policy 0, policy_version 931831 (0.0008) [2023-12-26 22:09:08,953][105692] Updated weights for policy 0, policy_version 931841 (0.0008) [2023-12-26 22:09:09,020][105692] Updated weights for policy 0, policy_version 931851 (0.0008) [2023-12-26 22:09:09,032][105620] Updated weights for policy 1, policy_version 932021 (0.0009) [2023-12-26 22:09:09,085][105620] Updated weights for policy 1, policy_version 932031 (0.0010) [2023-12-26 22:09:09,141][105620] Updated weights for policy 1, policy_version 932041 (0.0010) [2023-12-26 22:09:09,714][105692] Updated weights for policy 0, policy_version 931861 (0.0009) [2023-12-26 22:09:09,763][105692] Updated weights for policy 0, policy_version 931871 (0.0010) [2023-12-26 22:09:09,820][105692] Updated weights for policy 0, policy_version 931881 (0.0011) [2023-12-26 22:09:09,895][105620] Updated weights for policy 1, policy_version 932051 (0.0010) [2023-12-26 22:09:09,961][105620] Updated weights for policy 1, policy_version 932061 (0.0011) [2023-12-26 22:09:10,026][105620] Updated weights for policy 1, policy_version 932071 (0.0011) [2023-12-26 22:09:10,613][105692] Updated weights for policy 0, policy_version 931891 (0.0011) [2023-12-26 22:09:10,670][105692] Updated weights for policy 0, policy_version 931901 (0.0011) [2023-12-26 22:09:10,730][105692] Updated weights for policy 0, policy_version 931911 (0.0011) [2023-12-26 22:09:10,785][105620] Updated weights for policy 1, policy_version 932081 (0.0010) [2023-12-26 22:09:10,847][105620] Updated weights for policy 1, policy_version 932091 (0.0010) [2023-12-26 22:09:10,898][105620] Updated weights for policy 1, policy_version 932102 (0.0009) [2023-12-26 22:09:10,944][105620] Updated weights for policy 1, policy_version 932112 (0.0008) [2023-12-26 22:09:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19133.2). Total num frames: 477257728. Throughput: 0: 9665.0, 1: 9646.8. Samples: 477262372. Policy #0 lag: (min: 11.0, avg: 15.5, max: 43.0) [2023-12-26 22:09:11,063][104569] Avg episode reward: [(0, '9077.629'), (1, '9268.406')] [2023-12-26 22:09:11,493][105692] Updated weights for policy 0, policy_version 931921 (0.0011) [2023-12-26 22:09:11,553][105692] Updated weights for policy 0, policy_version 931931 (0.0011) [2023-12-26 22:09:11,613][105692] Updated weights for policy 0, policy_version 931941 (0.0010) [2023-12-26 22:09:11,686][105692] Updated weights for policy 0, policy_version 931951 (0.0009) [2023-12-26 22:09:11,784][105620] Updated weights for policy 1, policy_version 932122 (0.0008) [2023-12-26 22:09:11,849][105620] Updated weights for policy 1, policy_version 932132 (0.0006) [2023-12-26 22:09:11,911][105620] Updated weights for policy 1, policy_version 932142 (0.0007) [2023-12-26 22:09:12,423][105692] Updated weights for policy 0, policy_version 931961 (0.0008) [2023-12-26 22:09:12,485][105692] Updated weights for policy 0, policy_version 931971 (0.0006) [2023-12-26 22:09:12,539][105692] Updated weights for policy 0, policy_version 931981 (0.0006) [2023-12-26 22:09:12,709][105620] Updated weights for policy 1, policy_version 932152 (0.0009) [2023-12-26 22:09:12,767][105620] Updated weights for policy 1, policy_version 932162 (0.0010) [2023-12-26 22:09:12,820][105620] Updated weights for policy 1, policy_version 932172 (0.0009) [2023-12-26 22:09:13,135][105692] Updated weights for policy 0, policy_version 931991 (0.0008) [2023-12-26 22:09:13,183][105692] Updated weights for policy 0, policy_version 932001 (0.0006) [2023-12-26 22:09:13,232][105692] Updated weights for policy 0, policy_version 932011 (0.0007) [2023-12-26 22:09:13,650][105620] Updated weights for policy 1, policy_version 932182 (0.0009) [2023-12-26 22:09:13,705][105620] Updated weights for policy 1, policy_version 932192 (0.0008) [2023-12-26 22:09:13,757][105620] Updated weights for policy 1, policy_version 932202 (0.0008) [2023-12-26 22:09:13,949][105692] Updated weights for policy 0, policy_version 932021 (0.0006) [2023-12-26 22:09:14,008][105692] Updated weights for policy 0, policy_version 932031 (0.0005) [2023-12-26 22:09:14,064][105692] Updated weights for policy 0, policy_version 932041 (0.0005) [2023-12-26 22:09:14,616][105692] Updated weights for policy 0, policy_version 932051 (0.0007) [2023-12-26 22:09:14,626][105620] Updated weights for policy 1, policy_version 932212 (0.0007) [2023-12-26 22:09:14,661][105692] Updated weights for policy 0, policy_version 932061 (0.0007) [2023-12-26 22:09:14,688][105620] Updated weights for policy 1, policy_version 932222 (0.0007) [2023-12-26 22:09:14,714][105692] Updated weights for policy 0, policy_version 932071 (0.0009) [2023-12-26 22:09:14,737][105620] Updated weights for policy 1, policy_version 932232 (0.0006) [2023-12-26 22:09:15,472][105620] Updated weights for policy 1, policy_version 932242 (0.0007) [2023-12-26 22:09:15,480][105692] Updated weights for policy 0, policy_version 932081 (0.0007) [2023-12-26 22:09:15,530][105692] Updated weights for policy 0, policy_version 932091 (0.0006) [2023-12-26 22:09:15,532][105620] Updated weights for policy 1, policy_version 932252 (0.0009) [2023-12-26 22:09:15,583][105620] Updated weights for policy 1, policy_version 932262 (0.0009) [2023-12-26 22:09:15,585][105692] Updated weights for policy 0, policy_version 932101 (0.0006) [2023-12-26 22:09:15,641][105620] Updated weights for policy 1, policy_version 932272 (0.0008) [2023-12-26 22:09:15,644][105692] Updated weights for policy 0, policy_version 932111 (0.0006) [2023-12-26 22:09:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19105.4). Total num frames: 477347840. Throughput: 0: 9597.8, 1: 9604.5. Samples: 477317388. Policy #0 lag: (min: 11.0, avg: 15.5, max: 43.0) [2023-12-26 22:09:16,062][104569] Avg episode reward: [(0, '9351.788'), (1, '9176.616')] [2023-12-26 22:09:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000932112_238657536.pth... [2023-12-26 22:09:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000932272_238690304.pth... [2023-12-26 22:09:16,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000930960_238362624.pth [2023-12-26 22:09:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000931152_238403584.pth [2023-12-26 22:09:16,269][105692] Updated weights for policy 0, policy_version 932121 (0.0007) [2023-12-26 22:09:16,320][105692] Updated weights for policy 0, policy_version 932131 (0.0007) [2023-12-26 22:09:16,372][105692] Updated weights for policy 0, policy_version 932141 (0.0006) [2023-12-26 22:09:16,461][105620] Updated weights for policy 1, policy_version 932282 (0.0008) [2023-12-26 22:09:16,511][105620] Updated weights for policy 1, policy_version 932292 (0.0008) [2023-12-26 22:09:16,557][105620] Updated weights for policy 1, policy_version 932302 (0.0009) [2023-12-26 22:09:17,122][105692] Updated weights for policy 0, policy_version 932151 (0.0008) [2023-12-26 22:09:17,178][105692] Updated weights for policy 0, policy_version 932161 (0.0009) [2023-12-26 22:09:17,239][105692] Updated weights for policy 0, policy_version 932171 (0.0009) [2023-12-26 22:09:17,295][105620] Updated weights for policy 1, policy_version 932312 (0.0008) [2023-12-26 22:09:17,353][105620] Updated weights for policy 1, policy_version 932322 (0.0009) [2023-12-26 22:09:17,413][105620] Updated weights for policy 1, policy_version 932332 (0.0010) [2023-12-26 22:09:18,047][105692] Updated weights for policy 0, policy_version 932181 (0.0008) [2023-12-26 22:09:18,062][105620] Updated weights for policy 1, policy_version 932342 (0.0009) [2023-12-26 22:09:18,108][105692] Updated weights for policy 0, policy_version 932191 (0.0007) [2023-12-26 22:09:18,114][105620] Updated weights for policy 1, policy_version 932352 (0.0007) [2023-12-26 22:09:18,161][105692] Updated weights for policy 0, policy_version 932201 (0.0008) [2023-12-26 22:09:18,163][105620] Updated weights for policy 1, policy_version 932362 (0.0006) [2023-12-26 22:09:18,812][105620] Updated weights for policy 1, policy_version 932372 (0.0006) [2023-12-26 22:09:18,879][105620] Updated weights for policy 1, policy_version 932382 (0.0007) [2023-12-26 22:09:18,886][105692] Updated weights for policy 0, policy_version 932211 (0.0006) [2023-12-26 22:09:18,945][105620] Updated weights for policy 1, policy_version 932392 (0.0006) [2023-12-26 22:09:18,956][105692] Updated weights for policy 0, policy_version 932221 (0.0007) [2023-12-26 22:09:19,010][105692] Updated weights for policy 0, policy_version 932231 (0.0009) [2023-12-26 22:09:19,553][105620] Updated weights for policy 1, policy_version 932402 (0.0006) [2023-12-26 22:09:19,609][105620] Updated weights for policy 1, policy_version 932412 (0.0008) [2023-12-26 22:09:19,672][105620] Updated weights for policy 1, policy_version 932422 (0.0008) [2023-12-26 22:09:19,743][105620] Updated weights for policy 1, policy_version 932432 (0.0008) [2023-12-26 22:09:19,768][105692] Updated weights for policy 0, policy_version 932241 (0.0008) [2023-12-26 22:09:19,834][105692] Updated weights for policy 0, policy_version 932251 (0.0006) [2023-12-26 22:09:19,901][105692] Updated weights for policy 0, policy_version 932261 (0.0006) [2023-12-26 22:09:19,964][105692] Updated weights for policy 0, policy_version 932271 (0.0009) [2023-12-26 22:09:20,394][105620] Updated weights for policy 1, policy_version 932442 (0.0007) [2023-12-26 22:09:20,457][105620] Updated weights for policy 1, policy_version 932452 (0.0010) [2023-12-26 22:09:20,516][105620] Updated weights for policy 1, policy_version 932462 (0.0010) [2023-12-26 22:09:20,710][105692] Updated weights for policy 0, policy_version 932281 (0.0009) [2023-12-26 22:09:20,772][105692] Updated weights for policy 0, policy_version 932291 (0.0009) [2023-12-26 22:09:20,834][105692] Updated weights for policy 0, policy_version 932301 (0.0009) [2023-12-26 22:09:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19105.4). Total num frames: 477446144. Throughput: 0: 9610.1, 1: 9586.0. Samples: 477434180. Policy #0 lag: (min: 11.0, avg: 15.5, max: 43.0) [2023-12-26 22:09:21,062][104569] Avg episode reward: [(0, '5279.915'), (1, '9181.793')] [2023-12-26 22:09:21,347][105620] Updated weights for policy 1, policy_version 932472 (0.0009) [2023-12-26 22:09:21,415][105620] Updated weights for policy 1, policy_version 932482 (0.0009) [2023-12-26 22:09:21,484][105620] Updated weights for policy 1, policy_version 932492 (0.0008) [2023-12-26 22:09:21,557][105692] Updated weights for policy 0, policy_version 932311 (0.0007) [2023-12-26 22:09:21,630][105692] Updated weights for policy 0, policy_version 932321 (0.0007) [2023-12-26 22:09:21,685][105692] Updated weights for policy 0, policy_version 932331 (0.0009) [2023-12-26 22:09:22,266][105585] KL-divergence is very high: 109.8890 [2023-12-26 22:09:22,291][105692] Updated weights for policy 0, policy_version 932341 (0.0008) [2023-12-26 22:09:22,292][105620] Updated weights for policy 1, policy_version 932502 (0.0009) [2023-12-26 22:09:22,358][105620] Updated weights for policy 1, policy_version 932512 (0.0007) [2023-12-26 22:09:22,359][105692] Updated weights for policy 0, policy_version 932351 (0.0008) [2023-12-26 22:09:22,427][105620] Updated weights for policy 1, policy_version 932522 (0.0008) [2023-12-26 22:09:22,429][105692] Updated weights for policy 0, policy_version 932361 (0.0008) [2023-12-26 22:09:23,066][105692] Updated weights for policy 0, policy_version 932371 (0.0007) [2023-12-26 22:09:23,135][105692] Updated weights for policy 0, policy_version 932381 (0.0007) [2023-12-26 22:09:23,194][105692] Updated weights for policy 0, policy_version 932391 (0.0009) [2023-12-26 22:09:23,272][105620] Updated weights for policy 1, policy_version 932532 (0.0008) [2023-12-26 22:09:23,338][105620] Updated weights for policy 1, policy_version 932542 (0.0009) [2023-12-26 22:09:23,396][105620] Updated weights for policy 1, policy_version 932552 (0.0008) [2023-12-26 22:09:23,799][105692] Updated weights for policy 0, policy_version 932401 (0.0008) [2023-12-26 22:09:23,845][105692] Updated weights for policy 0, policy_version 932411 (0.0005) [2023-12-26 22:09:23,872][105585] KL-divergence is very high: 104.6715 [2023-12-26 22:09:23,893][105692] Updated weights for policy 0, policy_version 932421 (0.0005) [2023-12-26 22:09:23,914][105585] KL-divergence is very high: 129.4802 [2023-12-26 22:09:23,942][105692] Updated weights for policy 0, policy_version 932431 (0.0005) [2023-12-26 22:09:24,268][105620] Updated weights for policy 1, policy_version 932562 (0.0008) [2023-12-26 22:09:24,328][105620] Updated weights for policy 1, policy_version 932572 (0.0008) [2023-12-26 22:09:24,398][105620] Updated weights for policy 1, policy_version 932582 (0.0009) [2023-12-26 22:09:24,457][105620] Updated weights for policy 1, policy_version 932592 (0.0010) [2023-12-26 22:09:24,588][105692] Updated weights for policy 0, policy_version 932441 (0.0009) [2023-12-26 22:09:24,652][105692] Updated weights for policy 0, policy_version 932451 (0.0011) [2023-12-26 22:09:24,709][105692] Updated weights for policy 0, policy_version 932461 (0.0011) [2023-12-26 22:09:25,238][105620] Updated weights for policy 1, policy_version 932602 (0.0005) [2023-12-26 22:09:25,298][105620] Updated weights for policy 1, policy_version 932612 (0.0005) [2023-12-26 22:09:25,353][105620] Updated weights for policy 1, policy_version 932622 (0.0006) [2023-12-26 22:09:25,423][105692] Updated weights for policy 0, policy_version 932471 (0.0007) [2023-12-26 22:09:25,482][105692] Updated weights for policy 0, policy_version 932481 (0.0005) [2023-12-26 22:09:25,546][105692] Updated weights for policy 0, policy_version 932491 (0.0006) [2023-12-26 22:09:25,940][105620] Updated weights for policy 1, policy_version 932632 (0.0009) [2023-12-26 22:09:25,998][105620] Updated weights for policy 1, policy_version 932642 (0.0010) [2023-12-26 22:09:26,053][105620] Updated weights for policy 1, policy_version 932652 (0.0010) [2023-12-26 22:09:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19077.6). Total num frames: 477536256. Throughput: 0: 9753.0, 1: 9486.2. Samples: 477550160. Policy #0 lag: (min: 11.0, avg: 15.5, max: 43.0) [2023-12-26 22:09:26,062][104569] Avg episode reward: [(0, '4553.218'), (1, '9009.632')] [2023-12-26 22:09:26,192][105692] Updated weights for policy 0, policy_version 932501 (0.0008) [2023-12-26 22:09:26,247][105692] Updated weights for policy 0, policy_version 932511 (0.0008) [2023-12-26 22:09:26,292][105692] Updated weights for policy 0, policy_version 932521 (0.0008) [2023-12-26 22:09:26,819][105620] Updated weights for policy 1, policy_version 932662 (0.0009) [2023-12-26 22:09:26,881][105620] Updated weights for policy 1, policy_version 932672 (0.0008) [2023-12-26 22:09:26,944][105620] Updated weights for policy 1, policy_version 932682 (0.0007) [2023-12-26 22:09:26,968][105692] Updated weights for policy 0, policy_version 932531 (0.0009) [2023-12-26 22:09:27,017][105692] Updated weights for policy 0, policy_version 932541 (0.0010) [2023-12-26 22:09:27,069][105692] Updated weights for policy 0, policy_version 932551 (0.0010) [2023-12-26 22:09:27,606][105620] Updated weights for policy 1, policy_version 932692 (0.0007) [2023-12-26 22:09:27,654][105620] Updated weights for policy 1, policy_version 932702 (0.0010) [2023-12-26 22:09:27,705][105620] Updated weights for policy 1, policy_version 932712 (0.0009) [2023-12-26 22:09:27,755][105692] Updated weights for policy 0, policy_version 932561 (0.0010) [2023-12-26 22:09:27,808][105692] Updated weights for policy 0, policy_version 932571 (0.0007) [2023-12-26 22:09:27,853][105692] Updated weights for policy 0, policy_version 932581 (0.0010) [2023-12-26 22:09:27,904][105692] Updated weights for policy 0, policy_version 932591 (0.0010) [2023-12-26 22:09:28,357][105620] Updated weights for policy 1, policy_version 932722 (0.0010) [2023-12-26 22:09:28,425][105620] Updated weights for policy 1, policy_version 932732 (0.0006) [2023-12-26 22:09:28,490][105620] Updated weights for policy 1, policy_version 932742 (0.0007) [2023-12-26 22:09:28,549][105620] Updated weights for policy 1, policy_version 932752 (0.0007) [2023-12-26 22:09:28,668][105692] Updated weights for policy 0, policy_version 932601 (0.0006) [2023-12-26 22:09:28,729][105692] Updated weights for policy 0, policy_version 932611 (0.0009) [2023-12-26 22:09:28,797][105692] Updated weights for policy 0, policy_version 932621 (0.0010) [2023-12-26 22:09:29,111][105620] Updated weights for policy 1, policy_version 932762 (0.0006) [2023-12-26 22:09:29,168][105620] Updated weights for policy 1, policy_version 932772 (0.0006) [2023-12-26 22:09:29,228][105620] Updated weights for policy 1, policy_version 932782 (0.0006) [2023-12-26 22:09:29,525][105692] Updated weights for policy 0, policy_version 932631 (0.0010) [2023-12-26 22:09:29,584][105692] Updated weights for policy 0, policy_version 932641 (0.0007) [2023-12-26 22:09:29,644][105692] Updated weights for policy 0, policy_version 932651 (0.0005) [2023-12-26 22:09:29,941][105620] Updated weights for policy 1, policy_version 932792 (0.0008) [2023-12-26 22:09:30,003][105620] Updated weights for policy 1, policy_version 932802 (0.0008) [2023-12-26 22:09:30,062][105620] Updated weights for policy 1, policy_version 932812 (0.0008) [2023-12-26 22:09:30,382][105692] Updated weights for policy 0, policy_version 932661 (0.0008) [2023-12-26 22:09:30,441][105692] Updated weights for policy 0, policy_version 932671 (0.0010) [2023-12-26 22:09:30,500][105692] Updated weights for policy 0, policy_version 932681 (0.0010) [2023-12-26 22:09:30,810][105620] Updated weights for policy 1, policy_version 932822 (0.0009) [2023-12-26 22:09:30,858][105620] Updated weights for policy 1, policy_version 932832 (0.0008) [2023-12-26 22:09:30,903][105620] Updated weights for policy 1, policy_version 932842 (0.0005) [2023-12-26 22:09:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19105.4). Total num frames: 477642752. Throughput: 0: 9835.7, 1: 9577.6. Samples: 477611512. Policy #0 lag: (min: 11.0, avg: 15.5, max: 43.0) [2023-12-26 22:09:31,062][104569] Avg episode reward: [(0, '7047.229'), (1, '8829.789')] [2023-12-26 22:09:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000932688_238804992.pth... [2023-12-26 22:09:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000932848_238837760.pth... [2023-12-26 22:09:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000931728_238551040.pth [2023-12-26 22:09:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000931536_238510080.pth [2023-12-26 22:09:31,258][105692] Updated weights for policy 0, policy_version 932691 (0.0010) [2023-12-26 22:09:31,323][105692] Updated weights for policy 0, policy_version 932701 (0.0009) [2023-12-26 22:09:31,394][105692] Updated weights for policy 0, policy_version 932711 (0.0010) [2023-12-26 22:09:31,619][105620] Updated weights for policy 1, policy_version 932852 (0.0007) [2023-12-26 22:09:31,683][105620] Updated weights for policy 1, policy_version 932862 (0.0011) [2023-12-26 22:09:31,751][105620] Updated weights for policy 1, policy_version 932872 (0.0009) [2023-12-26 22:09:32,132][105692] Updated weights for policy 0, policy_version 932721 (0.0010) [2023-12-26 22:09:32,195][105692] Updated weights for policy 0, policy_version 932731 (0.0005) [2023-12-26 22:09:32,261][105692] Updated weights for policy 0, policy_version 932741 (0.0007) [2023-12-26 22:09:32,324][105692] Updated weights for policy 0, policy_version 932751 (0.0006) [2023-12-26 22:09:32,494][105620] Updated weights for policy 1, policy_version 932882 (0.0011) [2023-12-26 22:09:32,559][105620] Updated weights for policy 1, policy_version 932892 (0.0010) [2023-12-26 22:09:32,621][105620] Updated weights for policy 1, policy_version 932902 (0.0010) [2023-12-26 22:09:32,686][105620] Updated weights for policy 1, policy_version 932912 (0.0010) [2023-12-26 22:09:32,986][105692] Updated weights for policy 0, policy_version 932761 (0.0008) [2023-12-26 22:09:33,044][105692] Updated weights for policy 0, policy_version 932771 (0.0007) [2023-12-26 22:09:33,113][105692] Updated weights for policy 0, policy_version 932781 (0.0005) [2023-12-26 22:09:33,424][105620] Updated weights for policy 1, policy_version 932922 (0.0011) [2023-12-26 22:09:33,482][105620] Updated weights for policy 1, policy_version 932932 (0.0011) [2023-12-26 22:09:33,537][105620] Updated weights for policy 1, policy_version 932942 (0.0010) [2023-12-26 22:09:33,675][105692] Updated weights for policy 0, policy_version 932791 (0.0008) [2023-12-26 22:09:33,729][105692] Updated weights for policy 0, policy_version 932801 (0.0010) [2023-12-26 22:09:33,777][105692] Updated weights for policy 0, policy_version 932811 (0.0008) [2023-12-26 22:09:34,121][105620] Updated weights for policy 1, policy_version 932952 (0.0008) [2023-12-26 22:09:34,199][105620] Updated weights for policy 1, policy_version 932962 (0.0010) [2023-12-26 22:09:34,269][105620] Updated weights for policy 1, policy_version 932972 (0.0011) [2023-12-26 22:09:34,616][105692] Updated weights for policy 0, policy_version 932821 (0.0007) [2023-12-26 22:09:34,681][105692] Updated weights for policy 0, policy_version 932831 (0.0009) [2023-12-26 22:09:34,747][105692] Updated weights for policy 0, policy_version 932841 (0.0008) [2023-12-26 22:09:34,988][105620] Updated weights for policy 1, policy_version 932982 (0.0010) [2023-12-26 22:09:35,050][105620] Updated weights for policy 1, policy_version 932992 (0.0009) [2023-12-26 22:09:35,105][105620] Updated weights for policy 1, policy_version 933002 (0.0009) [2023-12-26 22:09:35,525][105692] Updated weights for policy 0, policy_version 932851 (0.0008) [2023-12-26 22:09:35,593][105692] Updated weights for policy 0, policy_version 932861 (0.0008) [2023-12-26 22:09:35,653][105692] Updated weights for policy 0, policy_version 932871 (0.0009) [2023-12-26 22:09:35,750][105620] Updated weights for policy 1, policy_version 933012 (0.0009) [2023-12-26 22:09:35,809][105620] Updated weights for policy 1, policy_version 933022 (0.0009) [2023-12-26 22:09:35,868][105620] Updated weights for policy 1, policy_version 933032 (0.0009) [2023-12-26 22:09:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19133.2). Total num frames: 477741056. Throughput: 0: 9800.1, 1: 9577.0. Samples: 477728264. Policy #0 lag: (min: 11.0, avg: 15.5, max: 43.0) [2023-12-26 22:09:36,062][104569] Avg episode reward: [(0, '8845.657'), (1, '9009.113')] [2023-12-26 22:09:36,343][105692] Updated weights for policy 0, policy_version 932881 (0.0009) [2023-12-26 22:09:36,418][105692] Updated weights for policy 0, policy_version 932891 (0.0010) [2023-12-26 22:09:36,483][105692] Updated weights for policy 0, policy_version 932901 (0.0008) [2023-12-26 22:09:36,541][105692] Updated weights for policy 0, policy_version 932911 (0.0009) [2023-12-26 22:09:36,678][105620] Updated weights for policy 1, policy_version 933042 (0.0009) [2023-12-26 22:09:36,739][105620] Updated weights for policy 1, policy_version 933052 (0.0007) [2023-12-26 22:09:36,804][105620] Updated weights for policy 1, policy_version 933062 (0.0008) [2023-12-26 22:09:36,865][105620] Updated weights for policy 1, policy_version 933072 (0.0009) [2023-12-26 22:09:37,264][105692] Updated weights for policy 0, policy_version 932921 (0.0010) [2023-12-26 22:09:37,319][105692] Updated weights for policy 0, policy_version 932931 (0.0010) [2023-12-26 22:09:37,379][105692] Updated weights for policy 0, policy_version 932941 (0.0011) [2023-12-26 22:09:37,499][105620] Updated weights for policy 1, policy_version 933082 (0.0009) [2023-12-26 22:09:37,552][105620] Updated weights for policy 1, policy_version 933092 (0.0009) [2023-12-26 22:09:37,608][105620] Updated weights for policy 1, policy_version 933102 (0.0008) [2023-12-26 22:09:38,076][105692] Updated weights for policy 0, policy_version 932951 (0.0010) [2023-12-26 22:09:38,139][105692] Updated weights for policy 0, policy_version 932961 (0.0010) [2023-12-26 22:09:38,201][105692] Updated weights for policy 0, policy_version 932971 (0.0010) [2023-12-26 22:09:38,294][105620] Updated weights for policy 1, policy_version 933112 (0.0007) [2023-12-26 22:09:38,364][105620] Updated weights for policy 1, policy_version 933122 (0.0008) [2023-12-26 22:09:38,424][105620] Updated weights for policy 1, policy_version 933132 (0.0007) [2023-12-26 22:09:38,890][105692] Updated weights for policy 0, policy_version 932981 (0.0009) [2023-12-26 22:09:38,956][105692] Updated weights for policy 0, policy_version 932991 (0.0007) [2023-12-26 22:09:39,028][105692] Updated weights for policy 0, policy_version 933001 (0.0006) [2023-12-26 22:09:39,192][105620] Updated weights for policy 1, policy_version 933142 (0.0009) [2023-12-26 22:09:39,256][105620] Updated weights for policy 1, policy_version 933152 (0.0008) [2023-12-26 22:09:39,305][105620] Updated weights for policy 1, policy_version 933162 (0.0008) [2023-12-26 22:09:39,744][105692] Updated weights for policy 0, policy_version 933011 (0.0007) [2023-12-26 22:09:39,805][105692] Updated weights for policy 0, policy_version 933021 (0.0009) [2023-12-26 22:09:39,861][105692] Updated weights for policy 0, policy_version 933031 (0.0009) [2023-12-26 22:09:40,084][105620] Updated weights for policy 1, policy_version 933172 (0.0008) [2023-12-26 22:09:40,141][105620] Updated weights for policy 1, policy_version 933182 (0.0009) [2023-12-26 22:09:40,206][105620] Updated weights for policy 1, policy_version 933192 (0.0007) [2023-12-26 22:09:40,574][105692] Updated weights for policy 0, policy_version 933041 (0.0009) [2023-12-26 22:09:40,632][105692] Updated weights for policy 0, policy_version 933051 (0.0005) [2023-12-26 22:09:40,696][105692] Updated weights for policy 0, policy_version 933061 (0.0006) [2023-12-26 22:09:40,766][105692] Updated weights for policy 0, policy_version 933071 (0.0006) [2023-12-26 22:09:40,929][105620] Updated weights for policy 1, policy_version 933202 (0.0008) [2023-12-26 22:09:40,991][105620] Updated weights for policy 1, policy_version 933212 (0.0010) [2023-12-26 22:09:41,060][105620] Updated weights for policy 1, policy_version 933222 (0.0007) [2023-12-26 22:09:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19105.4). Total num frames: 477831168. Throughput: 0: 9802.0, 1: 9555.1. Samples: 477843684. Policy #0 lag: (min: 11.0, avg: 15.5, max: 43.0) [2023-12-26 22:09:41,062][104569] Avg episode reward: [(0, '8995.890'), (1, '9104.178')] [2023-12-26 22:09:41,124][105620] Updated weights for policy 1, policy_version 933232 (0.0006) [2023-12-26 22:09:41,415][105692] Updated weights for policy 0, policy_version 933081 (0.0008) [2023-12-26 22:09:41,474][105692] Updated weights for policy 0, policy_version 933091 (0.0008) [2023-12-26 22:09:41,534][105692] Updated weights for policy 0, policy_version 933101 (0.0008) [2023-12-26 22:09:41,853][105620] Updated weights for policy 1, policy_version 933242 (0.0009) [2023-12-26 22:09:41,913][105620] Updated weights for policy 1, policy_version 933252 (0.0008) [2023-12-26 22:09:41,966][105620] Updated weights for policy 1, policy_version 933262 (0.0007) [2023-12-26 22:09:42,306][105692] Updated weights for policy 0, policy_version 933111 (0.0008) [2023-12-26 22:09:42,376][105692] Updated weights for policy 0, policy_version 933121 (0.0008) [2023-12-26 22:09:42,445][105692] Updated weights for policy 0, policy_version 933131 (0.0009) [2023-12-26 22:09:42,779][105620] Updated weights for policy 1, policy_version 933272 (0.0010) [2023-12-26 22:09:42,848][105620] Updated weights for policy 1, policy_version 933282 (0.0010) [2023-12-26 22:09:42,917][105620] Updated weights for policy 1, policy_version 933292 (0.0010) [2023-12-26 22:09:43,088][105692] Updated weights for policy 0, policy_version 933141 (0.0007) [2023-12-26 22:09:43,156][105692] Updated weights for policy 0, policy_version 933151 (0.0005) [2023-12-26 22:09:43,221][105692] Updated weights for policy 0, policy_version 933161 (0.0006) [2023-12-26 22:09:43,629][105620] Updated weights for policy 1, policy_version 933302 (0.0010) [2023-12-26 22:09:43,677][105620] Updated weights for policy 1, policy_version 933312 (0.0010) [2023-12-26 22:09:43,724][105620] Updated weights for policy 1, policy_version 933322 (0.0007) [2023-12-26 22:09:43,818][105692] Updated weights for policy 0, policy_version 933171 (0.0007) [2023-12-26 22:09:43,880][105692] Updated weights for policy 0, policy_version 933181 (0.0009) [2023-12-26 22:09:43,934][105692] Updated weights for policy 0, policy_version 933191 (0.0009) [2023-12-26 22:09:44,407][105620] Updated weights for policy 1, policy_version 933332 (0.0007) [2023-12-26 22:09:44,462][105620] Updated weights for policy 1, policy_version 933342 (0.0009) [2023-12-26 22:09:44,525][105620] Updated weights for policy 1, policy_version 933352 (0.0006) [2023-12-26 22:09:44,707][105692] Updated weights for policy 0, policy_version 933201 (0.0009) [2023-12-26 22:09:44,774][105692] Updated weights for policy 0, policy_version 933211 (0.0010) [2023-12-26 22:09:44,833][105692] Updated weights for policy 0, policy_version 933221 (0.0010) [2023-12-26 22:09:44,889][105692] Updated weights for policy 0, policy_version 933231 (0.0010) [2023-12-26 22:09:45,225][105620] Updated weights for policy 1, policy_version 933362 (0.0006) [2023-12-26 22:09:45,286][105620] Updated weights for policy 1, policy_version 933372 (0.0008) [2023-12-26 22:09:45,343][105620] Updated weights for policy 1, policy_version 933382 (0.0008) [2023-12-26 22:09:45,403][105620] Updated weights for policy 1, policy_version 933392 (0.0009) [2023-12-26 22:09:45,658][105692] Updated weights for policy 0, policy_version 933241 (0.0009) [2023-12-26 22:09:45,706][105692] Updated weights for policy 0, policy_version 933251 (0.0009) [2023-12-26 22:09:45,753][105692] Updated weights for policy 0, policy_version 933261 (0.0008) [2023-12-26 22:09:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19105.4). Total num frames: 477929472. Throughput: 0: 9830.3, 1: 9565.3. Samples: 477902372. Policy #0 lag: (min: 11.0, avg: 15.5, max: 43.0) [2023-12-26 22:09:46,062][104569] Avg episode reward: [(0, '8827.573'), (1, '8975.767')] [2023-12-26 22:09:46,067][105620] Updated weights for policy 1, policy_version 933402 (0.0008) [2023-12-26 22:09:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000933264_238952448.pth... [2023-12-26 22:09:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000932112_238657536.pth [2023-12-26 22:09:46,126][105620] Updated weights for policy 1, policy_version 933412 (0.0007) [2023-12-26 22:09:46,180][105620] Updated weights for policy 1, policy_version 933422 (0.0009) [2023-12-26 22:09:46,186][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000933424_238985216.pth... [2023-12-26 22:09:46,191][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000932272_238690304.pth [2023-12-26 22:09:46,512][105692] Updated weights for policy 0, policy_version 933271 (0.0009) [2023-12-26 22:09:46,561][105692] Updated weights for policy 0, policy_version 933281 (0.0009) [2023-12-26 22:09:46,620][105692] Updated weights for policy 0, policy_version 933291 (0.0008) [2023-12-26 22:09:46,855][105620] Updated weights for policy 1, policy_version 933432 (0.0006) [2023-12-26 22:09:46,904][105620] Updated weights for policy 1, policy_version 933442 (0.0005) [2023-12-26 22:09:46,956][105620] Updated weights for policy 1, policy_version 933452 (0.0005) [2023-12-26 22:09:47,277][105692] Updated weights for policy 0, policy_version 933301 (0.0009) [2023-12-26 22:09:47,327][105692] Updated weights for policy 0, policy_version 933311 (0.0009) [2023-12-26 22:09:47,375][105692] Updated weights for policy 0, policy_version 933321 (0.0009) [2023-12-26 22:09:47,633][105620] Updated weights for policy 1, policy_version 933462 (0.0007) [2023-12-26 22:09:47,680][105620] Updated weights for policy 1, policy_version 933472 (0.0009) [2023-12-26 22:09:47,731][105620] Updated weights for policy 1, policy_version 933482 (0.0009) [2023-12-26 22:09:48,177][105692] Updated weights for policy 0, policy_version 933331 (0.0008) [2023-12-26 22:09:48,234][105692] Updated weights for policy 0, policy_version 933341 (0.0005) [2023-12-26 22:09:48,290][105692] Updated weights for policy 0, policy_version 933351 (0.0006) [2023-12-26 22:09:48,499][105620] Updated weights for policy 1, policy_version 933492 (0.0007) [2023-12-26 22:09:48,562][105620] Updated weights for policy 1, policy_version 933502 (0.0005) [2023-12-26 22:09:48,624][105620] Updated weights for policy 1, policy_version 933512 (0.0006) [2023-12-26 22:09:49,005][105692] Updated weights for policy 0, policy_version 933361 (0.0007) [2023-12-26 22:09:49,059][105692] Updated weights for policy 0, policy_version 933371 (0.0009) [2023-12-26 22:09:49,076][105585] KL-divergence is very high: 111.6042 [2023-12-26 22:09:49,109][105692] Updated weights for policy 0, policy_version 933381 (0.0008) [2023-12-26 22:09:49,114][105585] KL-divergence is very high: 195.1011 [2023-12-26 22:09:49,158][105585] KL-divergence is very high: 193.8658 [2023-12-26 22:09:49,164][105692] Updated weights for policy 0, policy_version 933391 (0.0009) [2023-12-26 22:09:49,325][105620] Updated weights for policy 1, policy_version 933522 (0.0007) [2023-12-26 22:09:49,395][105620] Updated weights for policy 1, policy_version 933532 (0.0009) [2023-12-26 22:09:49,452][105620] Updated weights for policy 1, policy_version 933542 (0.0008) [2023-12-26 22:09:49,503][105620] Updated weights for policy 1, policy_version 933552 (0.0008) [2023-12-26 22:09:49,983][105692] Updated weights for policy 0, policy_version 933401 (0.0009) [2023-12-26 22:09:50,045][105692] Updated weights for policy 0, policy_version 933411 (0.0009) [2023-12-26 22:09:50,111][105692] Updated weights for policy 0, policy_version 933421 (0.0009) [2023-12-26 22:09:50,280][105620] Updated weights for policy 1, policy_version 933562 (0.0008) [2023-12-26 22:09:50,338][105620] Updated weights for policy 1, policy_version 933572 (0.0009) [2023-12-26 22:09:50,396][105620] Updated weights for policy 1, policy_version 933582 (0.0010) [2023-12-26 22:09:50,819][105692] Updated weights for policy 0, policy_version 933431 (0.0008) [2023-12-26 22:09:50,894][105692] Updated weights for policy 0, policy_version 933441 (0.0008) [2023-12-26 22:09:50,953][105692] Updated weights for policy 0, policy_version 933451 (0.0008) [2023-12-26 22:09:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19133.2). Total num frames: 478027776. Throughput: 0: 9842.9, 1: 9572.4. Samples: 478017528. Policy #0 lag: (min: 11.0, avg: 15.5, max: 43.0) [2023-12-26 22:09:51,062][104569] Avg episode reward: [(0, '8733.922'), (1, '9053.416')] [2023-12-26 22:09:51,177][105620] Updated weights for policy 1, policy_version 933592 (0.0011) [2023-12-26 22:09:51,235][105620] Updated weights for policy 1, policy_version 933602 (0.0011) [2023-12-26 22:09:51,298][105620] Updated weights for policy 1, policy_version 933612 (0.0011) [2023-12-26 22:09:51,781][105692] Updated weights for policy 0, policy_version 933461 (0.0008) [2023-12-26 22:09:51,841][105692] Updated weights for policy 0, policy_version 933471 (0.0008) [2023-12-26 22:09:51,896][105692] Updated weights for policy 0, policy_version 933481 (0.0007) [2023-12-26 22:09:52,091][105620] Updated weights for policy 1, policy_version 933622 (0.0010) [2023-12-26 22:09:52,157][105620] Updated weights for policy 1, policy_version 933632 (0.0009) [2023-12-26 22:09:52,227][105620] Updated weights for policy 1, policy_version 933642 (0.0010) [2023-12-26 22:09:52,563][105692] Updated weights for policy 0, policy_version 933491 (0.0006) [2023-12-26 22:09:52,619][105692] Updated weights for policy 0, policy_version 933501 (0.0010) [2023-12-26 22:09:52,678][105692] Updated weights for policy 0, policy_version 933511 (0.0010) [2023-12-26 22:09:52,898][105620] Updated weights for policy 1, policy_version 933652 (0.0007) [2023-12-26 22:09:52,955][105620] Updated weights for policy 1, policy_version 933662 (0.0006) [2023-12-26 22:09:53,014][105620] Updated weights for policy 1, policy_version 933672 (0.0009) [2023-12-26 22:09:53,456][105692] Updated weights for policy 0, policy_version 933521 (0.0009) [2023-12-26 22:09:53,511][105692] Updated weights for policy 0, policy_version 933531 (0.0009) [2023-12-26 22:09:53,566][105692] Updated weights for policy 0, policy_version 933541 (0.0009) [2023-12-26 22:09:53,621][105692] Updated weights for policy 0, policy_version 933551 (0.0009) [2023-12-26 22:09:53,725][105620] Updated weights for policy 1, policy_version 933682 (0.0009) [2023-12-26 22:09:53,786][105620] Updated weights for policy 1, policy_version 933692 (0.0010) [2023-12-26 22:09:53,844][105620] Updated weights for policy 1, policy_version 933702 (0.0010) [2023-12-26 22:09:53,898][105620] Updated weights for policy 1, policy_version 933712 (0.0010) [2023-12-26 22:09:54,395][105692] Updated weights for policy 0, policy_version 933561 (0.0008) [2023-12-26 22:09:54,444][105692] Updated weights for policy 0, policy_version 933571 (0.0008) [2023-12-26 22:09:54,497][105692] Updated weights for policy 0, policy_version 933581 (0.0009) [2023-12-26 22:09:54,585][105620] Updated weights for policy 1, policy_version 933722 (0.0006) [2023-12-26 22:09:54,639][105620] Updated weights for policy 1, policy_version 933732 (0.0006) [2023-12-26 22:09:54,703][105620] Updated weights for policy 1, policy_version 933742 (0.0009) [2023-12-26 22:09:55,334][105692] Updated weights for policy 0, policy_version 933591 (0.0008) [2023-12-26 22:09:55,396][105692] Updated weights for policy 0, policy_version 933601 (0.0008) [2023-12-26 22:09:55,409][105620] Updated weights for policy 1, policy_version 933752 (0.0010) [2023-12-26 22:09:55,452][105692] Updated weights for policy 0, policy_version 933611 (0.0005) [2023-12-26 22:09:55,465][105620] Updated weights for policy 1, policy_version 933762 (0.0010) [2023-12-26 22:09:55,532][105620] Updated weights for policy 1, policy_version 933772 (0.0010) [2023-12-26 22:09:56,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19251.1, 300 sec: 19133.2). Total num frames: 478117888. Throughput: 0: 9712.6, 1: 9552.9. Samples: 478129324. Policy #0 lag: (min: 11.0, avg: 15.5, max: 43.0) [2023-12-26 22:09:56,063][104569] Avg episode reward: [(0, '8900.955'), (1, '9012.631')] [2023-12-26 22:09:56,230][105692] Updated weights for policy 0, policy_version 933621 (0.0007) [2023-12-26 22:09:56,281][105620] Updated weights for policy 1, policy_version 933782 (0.0010) [2023-12-26 22:09:56,284][105692] Updated weights for policy 0, policy_version 933631 (0.0006) [2023-12-26 22:09:56,329][105620] Updated weights for policy 1, policy_version 933792 (0.0010) [2023-12-26 22:09:56,335][105692] Updated weights for policy 0, policy_version 933641 (0.0006) [2023-12-26 22:09:56,391][105620] Updated weights for policy 1, policy_version 933802 (0.0010) [2023-12-26 22:09:57,080][105692] Updated weights for policy 0, policy_version 933651 (0.0006) [2023-12-26 22:09:57,100][105620] Updated weights for policy 1, policy_version 933812 (0.0010) [2023-12-26 22:09:57,148][105620] Updated weights for policy 1, policy_version 933822 (0.0010) [2023-12-26 22:09:57,150][105692] Updated weights for policy 0, policy_version 933661 (0.0006) [2023-12-26 22:09:57,196][105620] Updated weights for policy 1, policy_version 933832 (0.0006) [2023-12-26 22:09:57,205][105692] Updated weights for policy 0, policy_version 933671 (0.0008) [2023-12-26 22:09:57,761][105620] Updated weights for policy 1, policy_version 933842 (0.0006) [2023-12-26 22:09:57,825][105692] Updated weights for policy 0, policy_version 933681 (0.0009) [2023-12-26 22:09:57,829][105620] Updated weights for policy 1, policy_version 933852 (0.0010) [2023-12-26 22:09:57,880][105692] Updated weights for policy 0, policy_version 933691 (0.0007) [2023-12-26 22:09:57,893][105620] Updated weights for policy 1, policy_version 933862 (0.0009) [2023-12-26 22:09:57,930][105692] Updated weights for policy 0, policy_version 933702 (0.0009) [2023-12-26 22:09:57,943][105620] Updated weights for policy 1, policy_version 933872 (0.0005) [2023-12-26 22:09:58,593][105620] Updated weights for policy 1, policy_version 933882 (0.0012) [2023-12-26 22:09:58,652][105620] Updated weights for policy 1, policy_version 933892 (0.0010) [2023-12-26 22:09:58,723][105620] Updated weights for policy 1, policy_version 933902 (0.0011) [2023-12-26 22:09:58,754][105692] Updated weights for policy 0, policy_version 933713 (0.0010) [2023-12-26 22:09:58,822][105692] Updated weights for policy 0, policy_version 933723 (0.0007) [2023-12-26 22:09:58,893][105692] Updated weights for policy 0, policy_version 933733 (0.0010) [2023-12-26 22:09:58,960][105692] Updated weights for policy 0, policy_version 933743 (0.0011) [2023-12-26 22:09:59,696][105692] Updated weights for policy 0, policy_version 933753 (0.0010) [2023-12-26 22:09:59,749][105620] Updated weights for policy 1, policy_version 933912 (0.0009) [2023-12-26 22:09:59,761][105692] Updated weights for policy 0, policy_version 933763 (0.0010) [2023-12-26 22:09:59,807][105620] Updated weights for policy 1, policy_version 933922 (0.0009) [2023-12-26 22:09:59,815][105692] Updated weights for policy 0, policy_version 933773 (0.0010) [2023-12-26 22:09:59,868][105620] Updated weights for policy 1, policy_version 933932 (0.0009) [2023-12-26 22:10:00,526][105692] Updated weights for policy 0, policy_version 933783 (0.0010) [2023-12-26 22:10:00,585][105692] Updated weights for policy 0, policy_version 933793 (0.0010) [2023-12-26 22:10:00,599][105620] Updated weights for policy 1, policy_version 933942 (0.0007) [2023-12-26 22:10:00,643][105692] Updated weights for policy 0, policy_version 933803 (0.0010) [2023-12-26 22:10:00,658][105620] Updated weights for policy 1, policy_version 933952 (0.0006) [2023-12-26 22:10:00,712][105620] Updated weights for policy 1, policy_version 933962 (0.0007) [2023-12-26 22:10:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19133.2). Total num frames: 478216192. Throughput: 0: 9700.8, 1: 9645.4. Samples: 478187968. Policy #0 lag: (min: 11.0, avg: 15.5, max: 43.0) [2023-12-26 22:10:01,062][104569] Avg episode reward: [(0, '9171.029'), (1, '8837.539')] [2023-12-26 22:10:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000933808_239091712.pth... [2023-12-26 22:10:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000933968_239124480.pth... [2023-12-26 22:10:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000932688_238804992.pth [2023-12-26 22:10:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000932848_238837760.pth [2023-12-26 22:10:01,411][105692] Updated weights for policy 0, policy_version 933813 (0.0009) [2023-12-26 22:10:01,474][105692] Updated weights for policy 0, policy_version 933823 (0.0010) [2023-12-26 22:10:01,500][105620] Updated weights for policy 1, policy_version 933972 (0.0007) [2023-12-26 22:10:01,535][105692] Updated weights for policy 0, policy_version 933833 (0.0010) [2023-12-26 22:10:01,558][105620] Updated weights for policy 1, policy_version 933982 (0.0007) [2023-12-26 22:10:01,625][105620] Updated weights for policy 1, policy_version 933992 (0.0006) [2023-12-26 22:10:02,228][105692] Updated weights for policy 0, policy_version 933843 (0.0010) [2023-12-26 22:10:02,287][105692] Updated weights for policy 0, policy_version 933853 (0.0008) [2023-12-26 22:10:02,357][105692] Updated weights for policy 0, policy_version 933863 (0.0006) [2023-12-26 22:10:02,367][105620] Updated weights for policy 1, policy_version 934002 (0.0009) [2023-12-26 22:10:02,423][105620] Updated weights for policy 1, policy_version 934012 (0.0008) [2023-12-26 22:10:02,478][105620] Updated weights for policy 1, policy_version 934022 (0.0012) [2023-12-26 22:10:02,544][105620] Updated weights for policy 1, policy_version 934032 (0.0009) [2023-12-26 22:10:02,913][105692] Updated weights for policy 0, policy_version 933873 (0.0008) [2023-12-26 22:10:02,967][105692] Updated weights for policy 0, policy_version 933883 (0.0007) [2023-12-26 22:10:03,014][105692] Updated weights for policy 0, policy_version 933893 (0.0009) [2023-12-26 22:10:03,061][105692] Updated weights for policy 0, policy_version 933903 (0.0010) [2023-12-26 22:10:03,412][105620] Updated weights for policy 1, policy_version 934042 (0.0010) [2023-12-26 22:10:03,474][105620] Updated weights for policy 1, policy_version 934052 (0.0010) [2023-12-26 22:10:03,530][105620] Updated weights for policy 1, policy_version 934062 (0.0008) [2023-12-26 22:10:03,772][105692] Updated weights for policy 0, policy_version 933913 (0.0008) [2023-12-26 22:10:03,830][105692] Updated weights for policy 0, policy_version 933923 (0.0009) [2023-12-26 22:10:03,902][105692] Updated weights for policy 0, policy_version 933933 (0.0009) [2023-12-26 22:10:04,306][105620] Updated weights for policy 1, policy_version 934072 (0.0008) [2023-12-26 22:10:04,372][105620] Updated weights for policy 1, policy_version 934082 (0.0009) [2023-12-26 22:10:04,430][105620] Updated weights for policy 1, policy_version 934092 (0.0009) [2023-12-26 22:10:04,606][105692] Updated weights for policy 0, policy_version 933943 (0.0009) [2023-12-26 22:10:04,665][105692] Updated weights for policy 0, policy_version 933953 (0.0008) [2023-12-26 22:10:04,724][105692] Updated weights for policy 0, policy_version 933963 (0.0006) [2023-12-26 22:10:05,196][105620] Updated weights for policy 1, policy_version 934102 (0.0009) [2023-12-26 22:10:05,253][105620] Updated weights for policy 1, policy_version 934112 (0.0008) [2023-12-26 22:10:05,310][105620] Updated weights for policy 1, policy_version 934122 (0.0009) [2023-12-26 22:10:05,443][105692] Updated weights for policy 0, policy_version 933973 (0.0006) [2023-12-26 22:10:05,489][105692] Updated weights for policy 0, policy_version 933983 (0.0005) [2023-12-26 22:10:05,534][105692] Updated weights for policy 0, policy_version 933993 (0.0005) [2023-12-26 22:10:06,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19114.7, 300 sec: 19105.4). Total num frames: 478306304. Throughput: 0: 9697.3, 1: 9528.0. Samples: 478299316. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-26 22:10:06,062][104569] Avg episode reward: [(0, '8992.074'), (1, '9090.497')] [2023-12-26 22:10:06,111][105620] Updated weights for policy 1, policy_version 934132 (0.0009) [2023-12-26 22:10:06,145][105692] Updated weights for policy 0, policy_version 934003 (0.0006) [2023-12-26 22:10:06,170][105620] Updated weights for policy 1, policy_version 934142 (0.0007) [2023-12-26 22:10:06,203][105692] Updated weights for policy 0, policy_version 934013 (0.0007) [2023-12-26 22:10:06,231][105620] Updated weights for policy 1, policy_version 934152 (0.0006) [2023-12-26 22:10:06,263][105692] Updated weights for policy 0, policy_version 934023 (0.0007) [2023-12-26 22:10:06,911][105620] Updated weights for policy 1, policy_version 934162 (0.0007) [2023-12-26 22:10:06,966][105620] Updated weights for policy 1, policy_version 934172 (0.0009) [2023-12-26 22:10:07,020][105620] Updated weights for policy 1, policy_version 934182 (0.0008) [2023-12-26 22:10:07,064][105692] Updated weights for policy 0, policy_version 934033 (0.0008) [2023-12-26 22:10:07,078][105620] Updated weights for policy 1, policy_version 934192 (0.0009) [2023-12-26 22:10:07,113][105692] Updated weights for policy 0, policy_version 934043 (0.0009) [2023-12-26 22:10:07,169][105692] Updated weights for policy 0, policy_version 934053 (0.0009) [2023-12-26 22:10:07,218][105692] Updated weights for policy 0, policy_version 934063 (0.0006) [2023-12-26 22:10:07,867][105620] Updated weights for policy 1, policy_version 934202 (0.0008) [2023-12-26 22:10:07,927][105620] Updated weights for policy 1, policy_version 934212 (0.0006) [2023-12-26 22:10:07,977][105692] Updated weights for policy 0, policy_version 934073 (0.0006) [2023-12-26 22:10:07,981][105620] Updated weights for policy 1, policy_version 934222 (0.0008) [2023-12-26 22:10:08,039][105692] Updated weights for policy 0, policy_version 934083 (0.0009) [2023-12-26 22:10:08,102][105692] Updated weights for policy 0, policy_version 934093 (0.0008) [2023-12-26 22:10:08,684][105620] Updated weights for policy 1, policy_version 934232 (0.0010) [2023-12-26 22:10:08,737][105620] Updated weights for policy 1, policy_version 934242 (0.0010) [2023-12-26 22:10:08,790][105620] Updated weights for policy 1, policy_version 934252 (0.0011) [2023-12-26 22:10:08,792][105692] Updated weights for policy 0, policy_version 934103 (0.0007) [2023-12-26 22:10:08,847][105692] Updated weights for policy 0, policy_version 934113 (0.0007) [2023-12-26 22:10:08,899][105692] Updated weights for policy 0, policy_version 934123 (0.0008) [2023-12-26 22:10:09,564][105620] Updated weights for policy 1, policy_version 934262 (0.0011) [2023-12-26 22:10:09,633][105620] Updated weights for policy 1, policy_version 934272 (0.0011) [2023-12-26 22:10:09,655][105692] Updated weights for policy 0, policy_version 934133 (0.0007) [2023-12-26 22:10:09,699][105620] Updated weights for policy 1, policy_version 934282 (0.0010) [2023-12-26 22:10:09,720][105692] Updated weights for policy 0, policy_version 934143 (0.0006) [2023-12-26 22:10:09,794][105692] Updated weights for policy 0, policy_version 934153 (0.0007) [2023-12-26 22:10:10,477][105620] Updated weights for policy 1, policy_version 934292 (0.0010) [2023-12-26 22:10:10,527][105692] Updated weights for policy 0, policy_version 934163 (0.0008) [2023-12-26 22:10:10,546][105620] Updated weights for policy 1, policy_version 934302 (0.0011) [2023-12-26 22:10:10,582][105692] Updated weights for policy 0, policy_version 934173 (0.0008) [2023-12-26 22:10:10,606][105620] Updated weights for policy 1, policy_version 934312 (0.0011) [2023-12-26 22:10:10,636][105692] Updated weights for policy 0, policy_version 934183 (0.0008) [2023-12-26 22:10:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.7, 300 sec: 19105.4). Total num frames: 478404608. Throughput: 0: 9609.6, 1: 9579.4. Samples: 478413664. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-26 22:10:11,063][104569] Avg episode reward: [(0, '9012.206'), (1, '9173.555')] [2023-12-26 22:10:11,299][105620] Updated weights for policy 1, policy_version 934322 (0.0009) [2023-12-26 22:10:11,329][105692] Updated weights for policy 0, policy_version 934193 (0.0007) [2023-12-26 22:10:11,364][105620] Updated weights for policy 1, policy_version 934332 (0.0009) [2023-12-26 22:10:11,398][105692] Updated weights for policy 0, policy_version 934203 (0.0007) [2023-12-26 22:10:11,424][105620] Updated weights for policy 1, policy_version 934342 (0.0009) [2023-12-26 22:10:11,451][105692] Updated weights for policy 0, policy_version 934213 (0.0006) [2023-12-26 22:10:11,487][105620] Updated weights for policy 1, policy_version 934352 (0.0008) [2023-12-26 22:10:11,503][105692] Updated weights for policy 0, policy_version 934223 (0.0008) [2023-12-26 22:10:12,243][105620] Updated weights for policy 1, policy_version 934362 (0.0006) [2023-12-26 22:10:12,286][105692] Updated weights for policy 0, policy_version 934233 (0.0009) [2023-12-26 22:10:12,300][105620] Updated weights for policy 1, policy_version 934372 (0.0007) [2023-12-26 22:10:12,354][105692] Updated weights for policy 0, policy_version 934243 (0.0008) [2023-12-26 22:10:12,368][105620] Updated weights for policy 1, policy_version 934382 (0.0008) [2023-12-26 22:10:12,416][105692] Updated weights for policy 0, policy_version 934253 (0.0008) [2023-12-26 22:10:13,062][105620] Updated weights for policy 1, policy_version 934392 (0.0008) [2023-12-26 22:10:13,126][105620] Updated weights for policy 1, policy_version 934402 (0.0008) [2023-12-26 22:10:13,181][105692] Updated weights for policy 0, policy_version 934263 (0.0010) [2023-12-26 22:10:13,184][105620] Updated weights for policy 1, policy_version 934412 (0.0008) [2023-12-26 22:10:13,240][105692] Updated weights for policy 0, policy_version 934273 (0.0011) [2023-12-26 22:10:13,303][105692] Updated weights for policy 0, policy_version 934283 (0.0011) [2023-12-26 22:10:13,922][105620] Updated weights for policy 1, policy_version 934422 (0.0009) [2023-12-26 22:10:13,969][105620] Updated weights for policy 1, policy_version 934432 (0.0008) [2023-12-26 22:10:14,024][105620] Updated weights for policy 1, policy_version 934442 (0.0008) [2023-12-26 22:10:14,045][105692] Updated weights for policy 0, policy_version 934293 (0.0008) [2023-12-26 22:10:14,100][105692] Updated weights for policy 0, policy_version 934303 (0.0008) [2023-12-26 22:10:14,158][105692] Updated weights for policy 0, policy_version 934313 (0.0006) [2023-12-26 22:10:14,786][105692] Updated weights for policy 0, policy_version 934323 (0.0007) [2023-12-26 22:10:14,821][105620] Updated weights for policy 1, policy_version 934452 (0.0008) [2023-12-26 22:10:14,869][105692] Updated weights for policy 0, policy_version 934333 (0.0007) [2023-12-26 22:10:14,891][105620] Updated weights for policy 1, policy_version 934462 (0.0009) [2023-12-26 22:10:14,931][105692] Updated weights for policy 0, policy_version 934343 (0.0006) [2023-12-26 22:10:14,949][105620] Updated weights for policy 1, policy_version 934472 (0.0007) [2023-12-26 22:10:15,659][105692] Updated weights for policy 0, policy_version 934353 (0.0008) [2023-12-26 22:10:15,720][105692] Updated weights for policy 0, policy_version 934363 (0.0009) [2023-12-26 22:10:15,733][105620] Updated weights for policy 1, policy_version 934482 (0.0009) [2023-12-26 22:10:15,774][105692] Updated weights for policy 0, policy_version 934373 (0.0007) [2023-12-26 22:10:15,781][105620] Updated weights for policy 1, policy_version 934492 (0.0005) [2023-12-26 22:10:15,824][105692] Updated weights for policy 0, policy_version 934383 (0.0006) [2023-12-26 22:10:15,830][105620] Updated weights for policy 1, policy_version 934502 (0.0006) [2023-12-26 22:10:15,883][105620] Updated weights for policy 1, policy_version 934512 (0.0009) [2023-12-26 22:10:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19105.4). Total num frames: 478502912. Throughput: 0: 9561.4, 1: 9510.5. Samples: 478469748. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-26 22:10:16,062][104569] Avg episode reward: [(0, '8573.379'), (1, '9175.414')] [2023-12-26 22:10:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000934384_239239168.pth... [2023-12-26 22:10:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000934512_239263744.pth... [2023-12-26 22:10:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000933424_238985216.pth [2023-12-26 22:10:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000933264_238952448.pth [2023-12-26 22:10:16,575][105692] Updated weights for policy 0, policy_version 934393 (0.0005) [2023-12-26 22:10:16,628][105692] Updated weights for policy 0, policy_version 934403 (0.0006) [2023-12-26 22:10:16,645][105620] Updated weights for policy 1, policy_version 934522 (0.0009) [2023-12-26 22:10:16,676][105692] Updated weights for policy 0, policy_version 934413 (0.0005) [2023-12-26 22:10:16,702][105620] Updated weights for policy 1, policy_version 934532 (0.0008) [2023-12-26 22:10:16,760][105620] Updated weights for policy 1, policy_version 934542 (0.0009) [2023-12-26 22:10:17,369][105692] Updated weights for policy 0, policy_version 934423 (0.0008) [2023-12-26 22:10:17,414][105692] Updated weights for policy 0, policy_version 934433 (0.0008) [2023-12-26 22:10:17,461][105692] Updated weights for policy 0, policy_version 934443 (0.0009) [2023-12-26 22:10:17,528][105620] Updated weights for policy 1, policy_version 934552 (0.0008) [2023-12-26 22:10:17,575][105620] Updated weights for policy 1, policy_version 934562 (0.0008) [2023-12-26 22:10:17,628][105620] Updated weights for policy 1, policy_version 934572 (0.0010) [2023-12-26 22:10:18,193][105692] Updated weights for policy 0, policy_version 934453 (0.0009) [2023-12-26 22:10:18,241][105692] Updated weights for policy 0, policy_version 934463 (0.0009) [2023-12-26 22:10:18,286][105692] Updated weights for policy 0, policy_version 934473 (0.0008) [2023-12-26 22:10:18,414][105620] Updated weights for policy 1, policy_version 934582 (0.0009) [2023-12-26 22:10:18,473][105620] Updated weights for policy 1, policy_version 934592 (0.0009) [2023-12-26 22:10:18,538][105620] Updated weights for policy 1, policy_version 934602 (0.0009) [2023-12-26 22:10:19,087][105692] Updated weights for policy 0, policy_version 934483 (0.0009) [2023-12-26 22:10:19,144][105692] Updated weights for policy 0, policy_version 934493 (0.0009) [2023-12-26 22:10:19,202][105692] Updated weights for policy 0, policy_version 934503 (0.0009) [2023-12-26 22:10:19,287][105620] Updated weights for policy 1, policy_version 934612 (0.0009) [2023-12-26 22:10:19,354][105620] Updated weights for policy 1, policy_version 934622 (0.0008) [2023-12-26 22:10:19,415][105620] Updated weights for policy 1, policy_version 934632 (0.0006) [2023-12-26 22:10:20,006][105692] Updated weights for policy 0, policy_version 934513 (0.0009) [2023-12-26 22:10:20,062][105692] Updated weights for policy 0, policy_version 934523 (0.0008) [2023-12-26 22:10:20,122][105692] Updated weights for policy 0, policy_version 934533 (0.0009) [2023-12-26 22:10:20,164][105620] Updated weights for policy 1, policy_version 934642 (0.0006) [2023-12-26 22:10:20,183][105692] Updated weights for policy 0, policy_version 934543 (0.0009) [2023-12-26 22:10:20,224][105620] Updated weights for policy 1, policy_version 934652 (0.0010) [2023-12-26 22:10:20,283][105620] Updated weights for policy 1, policy_version 934662 (0.0010) [2023-12-26 22:10:20,350][105620] Updated weights for policy 1, policy_version 934672 (0.0011) [2023-12-26 22:10:20,940][105692] Updated weights for policy 0, policy_version 934553 (0.0009) [2023-12-26 22:10:21,004][105692] Updated weights for policy 0, policy_version 934563 (0.0008) [2023-12-26 22:10:21,070][104569] Fps is (10 sec: 18008.5, 60 sec: 18975.7, 300 sec: 19077.1). Total num frames: 478584832. Throughput: 0: 9549.0, 1: 9427.7. Samples: 478582364. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-26 22:10:21,070][104569] Avg episode reward: [(0, '8643.575'), (1, '9175.541')] [2023-12-26 22:10:21,092][105692] Updated weights for policy 0, policy_version 934573 (0.0007) [2023-12-26 22:10:21,125][105620] Updated weights for policy 1, policy_version 934682 (0.0011) [2023-12-26 22:10:21,190][105620] Updated weights for policy 1, policy_version 934692 (0.0008) [2023-12-26 22:10:21,259][105620] Updated weights for policy 1, policy_version 934702 (0.0009) [2023-12-26 22:10:21,960][105620] Updated weights for policy 1, policy_version 934712 (0.0011) [2023-12-26 22:10:21,962][105692] Updated weights for policy 0, policy_version 934583 (0.0007) [2023-12-26 22:10:22,013][105620] Updated weights for policy 1, policy_version 934722 (0.0011) [2023-12-26 22:10:22,033][105692] Updated weights for policy 0, policy_version 934593 (0.0007) [2023-12-26 22:10:22,069][105620] Updated weights for policy 1, policy_version 934732 (0.0010) [2023-12-26 22:10:22,097][105692] Updated weights for policy 0, policy_version 934603 (0.0007) [2023-12-26 22:10:22,881][105692] Updated weights for policy 0, policy_version 934613 (0.0009) [2023-12-26 22:10:22,897][105620] Updated weights for policy 1, policy_version 934742 (0.0008) [2023-12-26 22:10:22,931][105692] Updated weights for policy 0, policy_version 934623 (0.0009) [2023-12-26 22:10:22,947][105620] Updated weights for policy 1, policy_version 934752 (0.0008) [2023-12-26 22:10:22,981][105692] Updated weights for policy 0, policy_version 934633 (0.0009) [2023-12-26 22:10:23,000][105620] Updated weights for policy 1, policy_version 934762 (0.0009) [2023-12-26 22:10:23,735][105620] Updated weights for policy 1, policy_version 934772 (0.0008) [2023-12-26 22:10:23,787][105692] Updated weights for policy 0, policy_version 934643 (0.0007) [2023-12-26 22:10:23,801][105620] Updated weights for policy 1, policy_version 934782 (0.0006) [2023-12-26 22:10:23,840][105692] Updated weights for policy 0, policy_version 934653 (0.0006) [2023-12-26 22:10:23,854][105620] Updated weights for policy 1, policy_version 934792 (0.0007) [2023-12-26 22:10:23,896][105692] Updated weights for policy 0, policy_version 934663 (0.0007) [2023-12-26 22:10:24,522][105620] Updated weights for policy 1, policy_version 934802 (0.0007) [2023-12-26 22:10:24,587][105620] Updated weights for policy 1, policy_version 934812 (0.0010) [2023-12-26 22:10:24,645][105620] Updated weights for policy 1, policy_version 934822 (0.0010) [2023-12-26 22:10:24,683][105692] Updated weights for policy 0, policy_version 934673 (0.0008) [2023-12-26 22:10:24,701][105620] Updated weights for policy 1, policy_version 934832 (0.0010) [2023-12-26 22:10:24,733][105692] Updated weights for policy 0, policy_version 934683 (0.0007) [2023-12-26 22:10:24,798][105692] Updated weights for policy 0, policy_version 934693 (0.0008) [2023-12-26 22:10:24,860][105692] Updated weights for policy 0, policy_version 934703 (0.0008) [2023-12-26 22:10:25,438][105620] Updated weights for policy 1, policy_version 934842 (0.0011) [2023-12-26 22:10:25,504][105620] Updated weights for policy 1, policy_version 934852 (0.0011) [2023-12-26 22:10:25,566][105620] Updated weights for policy 1, policy_version 934862 (0.0010) [2023-12-26 22:10:25,587][105692] Updated weights for policy 0, policy_version 934713 (0.0006) [2023-12-26 22:10:25,648][105692] Updated weights for policy 0, policy_version 934723 (0.0009) [2023-12-26 22:10:25,715][105692] Updated weights for policy 0, policy_version 934733 (0.0009) [2023-12-26 22:10:26,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19114.7, 300 sec: 19077.6). Total num frames: 478683136. Throughput: 0: 9460.8, 1: 9400.2. Samples: 478692428. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-26 22:10:26,063][104569] Avg episode reward: [(0, '8931.216'), (1, '9264.747')] [2023-12-26 22:10:26,178][105620] Updated weights for policy 1, policy_version 934872 (0.0007) [2023-12-26 22:10:26,235][105620] Updated weights for policy 1, policy_version 934882 (0.0005) [2023-12-26 22:10:26,282][105620] Updated weights for policy 1, policy_version 934892 (0.0006) [2023-12-26 22:10:26,531][105692] Updated weights for policy 0, policy_version 934743 (0.0009) [2023-12-26 22:10:26,585][105692] Updated weights for policy 0, policy_version 934753 (0.0008) [2023-12-26 22:10:26,643][105692] Updated weights for policy 0, policy_version 934763 (0.0009) [2023-12-26 22:10:26,921][105620] Updated weights for policy 1, policy_version 934902 (0.0009) [2023-12-26 22:10:26,975][105620] Updated weights for policy 1, policy_version 934912 (0.0009) [2023-12-26 22:10:27,023][105620] Updated weights for policy 1, policy_version 934922 (0.0009) [2023-12-26 22:10:27,487][105692] Updated weights for policy 0, policy_version 934773 (0.0009) [2023-12-26 22:10:27,537][105692] Updated weights for policy 0, policy_version 934783 (0.0009) [2023-12-26 22:10:27,594][105692] Updated weights for policy 0, policy_version 934793 (0.0008) [2023-12-26 22:10:27,670][105620] Updated weights for policy 1, policy_version 934932 (0.0009) [2023-12-26 22:10:27,716][105620] Updated weights for policy 1, policy_version 934942 (0.0010) [2023-12-26 22:10:27,767][105620] Updated weights for policy 1, policy_version 934952 (0.0010) [2023-12-26 22:10:28,169][105692] Updated weights for policy 0, policy_version 934803 (0.0007) [2023-12-26 22:10:28,214][105692] Updated weights for policy 0, policy_version 934813 (0.0005) [2023-12-26 22:10:28,259][105692] Updated weights for policy 0, policy_version 934823 (0.0005) [2023-12-26 22:10:28,435][105620] Updated weights for policy 1, policy_version 934962 (0.0010) [2023-12-26 22:10:28,493][105620] Updated weights for policy 1, policy_version 934972 (0.0009) [2023-12-26 22:10:28,550][105620] Updated weights for policy 1, policy_version 934982 (0.0010) [2023-12-26 22:10:28,612][105620] Updated weights for policy 1, policy_version 934992 (0.0010) [2023-12-26 22:10:28,883][105692] Updated weights for policy 0, policy_version 934833 (0.0006) [2023-12-26 22:10:28,936][105692] Updated weights for policy 0, policy_version 934843 (0.0009) [2023-12-26 22:10:28,983][105692] Updated weights for policy 0, policy_version 934853 (0.0008) [2023-12-26 22:10:29,030][105692] Updated weights for policy 0, policy_version 934863 (0.0009) [2023-12-26 22:10:29,305][105620] Updated weights for policy 1, policy_version 935002 (0.0008) [2023-12-26 22:10:29,362][105620] Updated weights for policy 1, policy_version 935012 (0.0011) [2023-12-26 22:10:29,414][105620] Updated weights for policy 1, policy_version 935022 (0.0011) [2023-12-26 22:10:29,877][105692] Updated weights for policy 0, policy_version 934873 (0.0007) [2023-12-26 22:10:29,938][105692] Updated weights for policy 0, policy_version 934883 (0.0010) [2023-12-26 22:10:29,998][105692] Updated weights for policy 0, policy_version 934893 (0.0006) [2023-12-26 22:10:30,131][105620] Updated weights for policy 1, policy_version 935032 (0.0009) [2023-12-26 22:10:30,192][105620] Updated weights for policy 1, policy_version 935042 (0.0008) [2023-12-26 22:10:30,253][105620] Updated weights for policy 1, policy_version 935052 (0.0009) [2023-12-26 22:10:30,653][105692] Updated weights for policy 0, policy_version 934903 (0.0005) [2023-12-26 22:10:30,699][105692] Updated weights for policy 0, policy_version 934913 (0.0005) [2023-12-26 22:10:30,742][105692] Updated weights for policy 0, policy_version 934923 (0.0005) [2023-12-26 22:10:31,038][105620] Updated weights for policy 1, policy_version 935062 (0.0009) [2023-12-26 22:10:31,066][104569] Fps is (10 sec: 19667.2, 60 sec: 18976.7, 300 sec: 19077.3). Total num frames: 478781440. Throughput: 0: 9429.6, 1: 9487.1. Samples: 478753712. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-26 22:10:31,067][104569] Avg episode reward: [(0, '8408.825'), (1, '9356.369')] [2023-12-26 22:10:31,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000934928_239378432.pth... [2023-12-26 22:10:31,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000933808_239091712.pth [2023-12-26 22:10:31,101][105620] Updated weights for policy 1, policy_version 935072 (0.0008) [2023-12-26 22:10:31,171][105620] Updated weights for policy 1, policy_version 935082 (0.0008) [2023-12-26 22:10:31,210][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000935088_239411200.pth... [2023-12-26 22:10:31,215][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000933968_239124480.pth [2023-12-26 22:10:31,420][105692] Updated weights for policy 0, policy_version 934933 (0.0006) [2023-12-26 22:10:31,482][105692] Updated weights for policy 0, policy_version 934943 (0.0008) [2023-12-26 22:10:31,544][105692] Updated weights for policy 0, policy_version 934953 (0.0007) [2023-12-26 22:10:31,959][105620] Updated weights for policy 1, policy_version 935092 (0.0009) [2023-12-26 22:10:32,015][105620] Updated weights for policy 1, policy_version 935102 (0.0009) [2023-12-26 22:10:32,062][105620] Updated weights for policy 1, policy_version 935112 (0.0009) [2023-12-26 22:10:32,238][105692] Updated weights for policy 0, policy_version 934963 (0.0007) [2023-12-26 22:10:32,298][105692] Updated weights for policy 0, policy_version 934973 (0.0009) [2023-12-26 22:10:32,362][105692] Updated weights for policy 0, policy_version 934983 (0.0007) [2023-12-26 22:10:32,930][105620] Updated weights for policy 1, policy_version 935122 (0.0009) [2023-12-26 22:10:32,969][105692] Updated weights for policy 0, policy_version 934993 (0.0005) [2023-12-26 22:10:32,979][105620] Updated weights for policy 1, policy_version 935132 (0.0008) [2023-12-26 22:10:33,029][105620] Updated weights for policy 1, policy_version 935142 (0.0008) [2023-12-26 22:10:33,040][105692] Updated weights for policy 0, policy_version 935003 (0.0007) [2023-12-26 22:10:33,080][105620] Updated weights for policy 1, policy_version 935152 (0.0008) [2023-12-26 22:10:33,110][105692] Updated weights for policy 0, policy_version 935013 (0.0006) [2023-12-26 22:10:33,175][105692] Updated weights for policy 0, policy_version 935023 (0.0005) [2023-12-26 22:10:33,689][105692] Updated weights for policy 0, policy_version 935033 (0.0006) [2023-12-26 22:10:33,757][105692] Updated weights for policy 0, policy_version 935043 (0.0005) [2023-12-26 22:10:33,825][105692] Updated weights for policy 0, policy_version 935053 (0.0005) [2023-12-26 22:10:33,977][105620] Updated weights for policy 1, policy_version 935162 (0.0009) [2023-12-26 22:10:34,047][105620] Updated weights for policy 1, policy_version 935172 (0.0009) [2023-12-26 22:10:34,119][105620] Updated weights for policy 1, policy_version 935182 (0.0009) [2023-12-26 22:10:34,399][105692] Updated weights for policy 0, policy_version 935063 (0.0009) [2023-12-26 22:10:34,466][105692] Updated weights for policy 0, policy_version 935073 (0.0011) [2023-12-26 22:10:34,532][105692] Updated weights for policy 0, policy_version 935083 (0.0010) [2023-12-26 22:10:34,913][105620] Updated weights for policy 1, policy_version 935192 (0.0009) [2023-12-26 22:10:34,969][105620] Updated weights for policy 1, policy_version 935202 (0.0008) [2023-12-26 22:10:35,037][105620] Updated weights for policy 1, policy_version 935212 (0.0009) [2023-12-26 22:10:35,255][105692] Updated weights for policy 0, policy_version 935093 (0.0008) [2023-12-26 22:10:35,311][105692] Updated weights for policy 0, policy_version 935103 (0.0007) [2023-12-26 22:10:35,369][105692] Updated weights for policy 0, policy_version 935113 (0.0010) [2023-12-26 22:10:35,817][105620] Updated weights for policy 1, policy_version 935222 (0.0007) [2023-12-26 22:10:35,865][105620] Updated weights for policy 1, policy_version 935232 (0.0005) [2023-12-26 22:10:35,928][105620] Updated weights for policy 1, policy_version 935242 (0.0005) [2023-12-26 22:10:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 18978.1, 300 sec: 19077.6). Total num frames: 478879744. Throughput: 0: 9569.9, 1: 9357.7. Samples: 478869272. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-26 22:10:36,063][104569] Avg episode reward: [(0, '8651.865'), (1, '9356.477')] [2023-12-26 22:10:36,087][105692] Updated weights for policy 0, policy_version 935123 (0.0010) [2023-12-26 22:10:36,149][105692] Updated weights for policy 0, policy_version 935133 (0.0008) [2023-12-26 22:10:36,215][105692] Updated weights for policy 0, policy_version 935143 (0.0010) [2023-12-26 22:10:36,583][105620] Updated weights for policy 1, policy_version 935252 (0.0007) [2023-12-26 22:10:36,644][105620] Updated weights for policy 1, policy_version 935262 (0.0009) [2023-12-26 22:10:36,702][105620] Updated weights for policy 1, policy_version 935272 (0.0008) [2023-12-26 22:10:36,965][105692] Updated weights for policy 0, policy_version 935153 (0.0009) [2023-12-26 22:10:37,020][105692] Updated weights for policy 0, policy_version 935163 (0.0008) [2023-12-26 22:10:37,069][105692] Updated weights for policy 0, policy_version 935173 (0.0009) [2023-12-26 22:10:37,129][105692] Updated weights for policy 0, policy_version 935183 (0.0009) [2023-12-26 22:10:37,488][105620] Updated weights for policy 1, policy_version 935282 (0.0009) [2023-12-26 22:10:37,552][105620] Updated weights for policy 1, policy_version 935292 (0.0009) [2023-12-26 22:10:37,617][105620] Updated weights for policy 1, policy_version 935302 (0.0009) [2023-12-26 22:10:37,678][105620] Updated weights for policy 1, policy_version 935312 (0.0006) [2023-12-26 22:10:37,875][105692] Updated weights for policy 0, policy_version 935193 (0.0008) [2023-12-26 22:10:37,937][105692] Updated weights for policy 0, policy_version 935203 (0.0005) [2023-12-26 22:10:38,006][105692] Updated weights for policy 0, policy_version 935213 (0.0006) [2023-12-26 22:10:38,490][105620] Updated weights for policy 1, policy_version 935322 (0.0008) [2023-12-26 22:10:38,538][105620] Updated weights for policy 1, policy_version 935332 (0.0008) [2023-12-26 22:10:38,584][105692] Updated weights for policy 0, policy_version 935223 (0.0008) [2023-12-26 22:10:38,598][105620] Updated weights for policy 1, policy_version 935342 (0.0007) [2023-12-26 22:10:38,633][105692] Updated weights for policy 0, policy_version 935233 (0.0010) [2023-12-26 22:10:38,684][105692] Updated weights for policy 0, policy_version 935243 (0.0010) [2023-12-26 22:10:39,281][105620] Updated weights for policy 1, policy_version 935352 (0.0008) [2023-12-26 22:10:39,345][105620] Updated weights for policy 1, policy_version 935362 (0.0008) [2023-12-26 22:10:39,404][105620] Updated weights for policy 1, policy_version 935372 (0.0009) [2023-12-26 22:10:39,574][105692] Updated weights for policy 0, policy_version 935253 (0.0011) [2023-12-26 22:10:39,644][105692] Updated weights for policy 0, policy_version 935263 (0.0011) [2023-12-26 22:10:39,710][105692] Updated weights for policy 0, policy_version 935273 (0.0011) [2023-12-26 22:10:40,173][105620] Updated weights for policy 1, policy_version 935382 (0.0007) [2023-12-26 22:10:40,224][105620] Updated weights for policy 1, policy_version 935392 (0.0007) [2023-12-26 22:10:40,281][105620] Updated weights for policy 1, policy_version 935402 (0.0009) [2023-12-26 22:10:40,459][105692] Updated weights for policy 0, policy_version 935283 (0.0010) [2023-12-26 22:10:40,534][105692] Updated weights for policy 0, policy_version 935293 (0.0008) [2023-12-26 22:10:40,598][105692] Updated weights for policy 0, policy_version 935303 (0.0008) [2023-12-26 22:10:41,019][105620] Updated weights for policy 1, policy_version 935412 (0.0007) [2023-12-26 22:10:41,062][104569] Fps is (10 sec: 18850.2, 60 sec: 18978.1, 300 sec: 19077.6). Total num frames: 478969856. Throughput: 0: 9612.7, 1: 9342.7. Samples: 478982312. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-26 22:10:41,062][104569] Avg episode reward: [(0, '8825.280'), (1, '9356.390')] [2023-12-26 22:10:41,085][105620] Updated weights for policy 1, policy_version 935422 (0.0009) [2023-12-26 22:10:41,156][105620] Updated weights for policy 1, policy_version 935432 (0.0008) [2023-12-26 22:10:41,343][105692] Updated weights for policy 0, policy_version 935313 (0.0008) [2023-12-26 22:10:41,427][105692] Updated weights for policy 0, policy_version 935323 (0.0008) [2023-12-26 22:10:41,494][105692] Updated weights for policy 0, policy_version 935333 (0.0008) [2023-12-26 22:10:41,565][105692] Updated weights for policy 0, policy_version 935343 (0.0008) [2023-12-26 22:10:41,952][105620] Updated weights for policy 1, policy_version 935442 (0.0009) [2023-12-26 22:10:42,022][105620] Updated weights for policy 1, policy_version 935452 (0.0008) [2023-12-26 22:10:42,086][105620] Updated weights for policy 1, policy_version 935462 (0.0008) [2023-12-26 22:10:42,148][105620] Updated weights for policy 1, policy_version 935472 (0.0009) [2023-12-26 22:10:42,288][105692] Updated weights for policy 0, policy_version 935353 (0.0007) [2023-12-26 22:10:42,351][105692] Updated weights for policy 0, policy_version 935363 (0.0006) [2023-12-26 22:10:42,422][105692] Updated weights for policy 0, policy_version 935373 (0.0008) [2023-12-26 22:10:42,849][105620] Updated weights for policy 1, policy_version 935482 (0.0011) [2023-12-26 22:10:42,899][105620] Updated weights for policy 1, policy_version 935492 (0.0011) [2023-12-26 22:10:42,943][105620] Updated weights for policy 1, policy_version 935502 (0.0009) [2023-12-26 22:10:43,132][105692] Updated weights for policy 0, policy_version 935383 (0.0008) [2023-12-26 22:10:43,185][105692] Updated weights for policy 0, policy_version 935393 (0.0006) [2023-12-26 22:10:43,239][105692] Updated weights for policy 0, policy_version 935403 (0.0006) [2023-12-26 22:10:43,639][105620] Updated weights for policy 1, policy_version 935512 (0.0010) [2023-12-26 22:10:43,691][105620] Updated weights for policy 1, policy_version 935522 (0.0010) [2023-12-26 22:10:43,742][105620] Updated weights for policy 1, policy_version 935532 (0.0010) [2023-12-26 22:10:43,978][105692] Updated weights for policy 0, policy_version 935413 (0.0008) [2023-12-26 22:10:44,035][105692] Updated weights for policy 0, policy_version 935423 (0.0007) [2023-12-26 22:10:44,091][105692] Updated weights for policy 0, policy_version 935433 (0.0007) [2023-12-26 22:10:44,518][105620] Updated weights for policy 1, policy_version 935542 (0.0011) [2023-12-26 22:10:44,567][105620] Updated weights for policy 1, policy_version 935552 (0.0010) [2023-12-26 22:10:44,621][105620] Updated weights for policy 1, policy_version 935562 (0.0010) [2023-12-26 22:10:44,855][105692] Updated weights for policy 0, policy_version 935443 (0.0008) [2023-12-26 22:10:44,917][105692] Updated weights for policy 0, policy_version 935453 (0.0006) [2023-12-26 22:10:44,975][105692] Updated weights for policy 0, policy_version 935463 (0.0006) [2023-12-26 22:10:45,441][105620] Updated weights for policy 1, policy_version 935572 (0.0010) [2023-12-26 22:10:45,490][105620] Updated weights for policy 1, policy_version 935582 (0.0009) [2023-12-26 22:10:45,544][105620] Updated weights for policy 1, policy_version 935592 (0.0009) [2023-12-26 22:10:45,785][105692] Updated weights for policy 0, policy_version 935473 (0.0007) [2023-12-26 22:10:45,850][105692] Updated weights for policy 0, policy_version 935483 (0.0005) [2023-12-26 22:10:45,919][105692] Updated weights for policy 0, policy_version 935493 (0.0005) [2023-12-26 22:10:45,987][105692] Updated weights for policy 0, policy_version 935503 (0.0005) [2023-12-26 22:10:46,062][104569] Fps is (10 sec: 18841.8, 60 sec: 18978.1, 300 sec: 19077.6). Total num frames: 479068160. Throughput: 0: 9581.9, 1: 9308.3. Samples: 479038024. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-26 22:10:46,062][104569] Avg episode reward: [(0, '8738.714'), (1, '9356.077')] [2023-12-26 22:10:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000935504_239525888.pth... [2023-12-26 22:10:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000935600_239542272.pth... [2023-12-26 22:10:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000934384_239239168.pth [2023-12-26 22:10:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000934512_239263744.pth [2023-12-26 22:10:46,266][105620] Updated weights for policy 1, policy_version 935602 (0.0009) [2023-12-26 22:10:46,318][105620] Updated weights for policy 1, policy_version 935612 (0.0010) [2023-12-26 22:10:46,363][105620] Updated weights for policy 1, policy_version 935622 (0.0010) [2023-12-26 22:10:46,417][105620] Updated weights for policy 1, policy_version 935632 (0.0010) [2023-12-26 22:10:46,557][105692] Updated weights for policy 0, policy_version 935513 (0.0006) [2023-12-26 22:10:46,625][105692] Updated weights for policy 0, policy_version 935523 (0.0008) [2023-12-26 22:10:46,679][105692] Updated weights for policy 0, policy_version 935533 (0.0008) [2023-12-26 22:10:47,171][105620] Updated weights for policy 1, policy_version 935642 (0.0010) [2023-12-26 22:10:47,225][105620] Updated weights for policy 1, policy_version 935652 (0.0010) [2023-12-26 22:10:47,284][105620] Updated weights for policy 1, policy_version 935662 (0.0010) [2023-12-26 22:10:47,321][105692] Updated weights for policy 0, policy_version 935543 (0.0008) [2023-12-26 22:10:47,373][105692] Updated weights for policy 0, policy_version 935553 (0.0008) [2023-12-26 22:10:47,433][105692] Updated weights for policy 0, policy_version 935563 (0.0008) [2023-12-26 22:10:48,033][105620] Updated weights for policy 1, policy_version 935672 (0.0010) [2023-12-26 22:10:48,099][105620] Updated weights for policy 1, policy_version 935682 (0.0010) [2023-12-26 22:10:48,157][105620] Updated weights for policy 1, policy_version 935692 (0.0010) [2023-12-26 22:10:48,179][105692] Updated weights for policy 0, policy_version 935573 (0.0008) [2023-12-26 22:10:48,239][105692] Updated weights for policy 0, policy_version 935583 (0.0008) [2023-12-26 22:10:48,299][105692] Updated weights for policy 0, policy_version 935593 (0.0008) [2023-12-26 22:10:48,939][105620] Updated weights for policy 1, policy_version 935702 (0.0009) [2023-12-26 22:10:48,956][105692] Updated weights for policy 0, policy_version 935603 (0.0007) [2023-12-26 22:10:48,998][105620] Updated weights for policy 1, policy_version 935712 (0.0010) [2023-12-26 22:10:49,024][105692] Updated weights for policy 0, policy_version 935613 (0.0006) [2023-12-26 22:10:49,061][105620] Updated weights for policy 1, policy_version 935722 (0.0010) [2023-12-26 22:10:49,084][105692] Updated weights for policy 0, policy_version 935623 (0.0009) [2023-12-26 22:10:49,808][105620] Updated weights for policy 1, policy_version 935732 (0.0010) [2023-12-26 22:10:49,875][105620] Updated weights for policy 1, policy_version 935742 (0.0010) [2023-12-26 22:10:49,886][105692] Updated weights for policy 0, policy_version 935633 (0.0009) [2023-12-26 22:10:49,944][105620] Updated weights for policy 1, policy_version 935752 (0.0011) [2023-12-26 22:10:49,954][105692] Updated weights for policy 0, policy_version 935643 (0.0007) [2023-12-26 22:10:50,017][105692] Updated weights for policy 0, policy_version 935653 (0.0007) [2023-12-26 22:10:50,081][105692] Updated weights for policy 0, policy_version 935663 (0.0008) [2023-12-26 22:10:50,603][105620] Updated weights for policy 1, policy_version 935762 (0.0011) [2023-12-26 22:10:50,656][105620] Updated weights for policy 1, policy_version 935772 (0.0010) [2023-12-26 22:10:50,712][105620] Updated weights for policy 1, policy_version 935782 (0.0011) [2023-12-26 22:10:50,775][105620] Updated weights for policy 1, policy_version 935792 (0.0011) [2023-12-26 22:10:50,891][105692] Updated weights for policy 0, policy_version 935673 (0.0008) [2023-12-26 22:10:50,955][105692] Updated weights for policy 0, policy_version 935683 (0.0008) [2023-12-26 22:10:51,024][105692] Updated weights for policy 0, policy_version 935693 (0.0008) [2023-12-26 22:10:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 18978.1, 300 sec: 19105.4). Total num frames: 479166464. Throughput: 0: 9571.6, 1: 9371.7. Samples: 479151768. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-26 22:10:51,062][104569] Avg episode reward: [(0, '8383.296'), (1, '9267.229')] [2023-12-26 22:10:51,595][105620] Updated weights for policy 1, policy_version 935802 (0.0011) [2023-12-26 22:10:51,665][105620] Updated weights for policy 1, policy_version 935812 (0.0011) [2023-12-26 22:10:51,730][105620] Updated weights for policy 1, policy_version 935822 (0.0011) [2023-12-26 22:10:51,825][105692] Updated weights for policy 0, policy_version 935703 (0.0009) [2023-12-26 22:10:51,893][105692] Updated weights for policy 0, policy_version 935713 (0.0008) [2023-12-26 22:10:51,957][105692] Updated weights for policy 0, policy_version 935723 (0.0009) [2023-12-26 22:10:52,460][105620] Updated weights for policy 1, policy_version 935832 (0.0010) [2023-12-26 22:10:52,525][105620] Updated weights for policy 1, policy_version 935842 (0.0010) [2023-12-26 22:10:52,584][105620] Updated weights for policy 1, policy_version 935852 (0.0010) [2023-12-26 22:10:52,730][105692] Updated weights for policy 0, policy_version 935733 (0.0008) [2023-12-26 22:10:52,791][105692] Updated weights for policy 0, policy_version 935743 (0.0008) [2023-12-26 22:10:52,851][105692] Updated weights for policy 0, policy_version 935753 (0.0008) [2023-12-26 22:10:53,266][105620] Updated weights for policy 1, policy_version 935862 (0.0009) [2023-12-26 22:10:53,323][105620] Updated weights for policy 1, policy_version 935872 (0.0010) [2023-12-26 22:10:53,383][105620] Updated weights for policy 1, policy_version 935882 (0.0005) [2023-12-26 22:10:53,606][105692] Updated weights for policy 0, policy_version 935763 (0.0008) [2023-12-26 22:10:53,655][105692] Updated weights for policy 0, policy_version 935773 (0.0008) [2023-12-26 22:10:53,701][105692] Updated weights for policy 0, policy_version 935783 (0.0008) [2023-12-26 22:10:54,105][105620] Updated weights for policy 1, policy_version 935892 (0.0009) [2023-12-26 22:10:54,157][105620] Updated weights for policy 1, policy_version 935902 (0.0010) [2023-12-26 22:10:54,216][105620] Updated weights for policy 1, policy_version 935912 (0.0010) [2023-12-26 22:10:54,465][105692] Updated weights for policy 0, policy_version 935793 (0.0008) [2023-12-26 22:10:54,532][105692] Updated weights for policy 0, policy_version 935803 (0.0009) [2023-12-26 22:10:54,599][105692] Updated weights for policy 0, policy_version 935813 (0.0008) [2023-12-26 22:10:54,664][105692] Updated weights for policy 0, policy_version 935823 (0.0010) [2023-12-26 22:10:54,886][105620] Updated weights for policy 1, policy_version 935922 (0.0009) [2023-12-26 22:10:54,958][105620] Updated weights for policy 1, policy_version 935932 (0.0010) [2023-12-26 22:10:55,018][105620] Updated weights for policy 1, policy_version 935942 (0.0009) [2023-12-26 22:10:55,071][105620] Updated weights for policy 1, policy_version 935952 (0.0008) [2023-12-26 22:10:55,441][105692] Updated weights for policy 0, policy_version 935833 (0.0009) [2023-12-26 22:10:55,507][105692] Updated weights for policy 0, policy_version 935843 (0.0009) [2023-12-26 22:10:55,570][105692] Updated weights for policy 0, policy_version 935853 (0.0009) [2023-12-26 22:10:55,838][105620] Updated weights for policy 1, policy_version 935962 (0.0009) [2023-12-26 22:10:55,897][105620] Updated weights for policy 1, policy_version 935972 (0.0009) [2023-12-26 22:10:55,952][105620] Updated weights for policy 1, policy_version 935982 (0.0009) [2023-12-26 22:10:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18978.2, 300 sec: 19077.6). Total num frames: 479256576. Throughput: 0: 9488.0, 1: 9388.6. Samples: 479263112. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-26 22:10:56,063][104569] Avg episode reward: [(0, '8741.521'), (1, '8994.048')] [2023-12-26 22:10:56,318][105692] Updated weights for policy 0, policy_version 935863 (0.0008) [2023-12-26 22:10:56,365][105692] Updated weights for policy 0, policy_version 935873 (0.0009) [2023-12-26 22:10:56,413][105692] Updated weights for policy 0, policy_version 935883 (0.0008) [2023-12-26 22:10:56,711][105620] Updated weights for policy 1, policy_version 935992 (0.0009) [2023-12-26 22:10:56,761][105620] Updated weights for policy 1, policy_version 936002 (0.0008) [2023-12-26 22:10:56,819][105620] Updated weights for policy 1, policy_version 936012 (0.0009) [2023-12-26 22:10:57,187][105692] Updated weights for policy 0, policy_version 935893 (0.0009) [2023-12-26 22:10:57,244][105692] Updated weights for policy 0, policy_version 935903 (0.0010) [2023-12-26 22:10:57,299][105692] Updated weights for policy 0, policy_version 935913 (0.0009) [2023-12-26 22:10:57,552][105620] Updated weights for policy 1, policy_version 936022 (0.0009) [2023-12-26 22:10:57,607][105620] Updated weights for policy 1, policy_version 936032 (0.0009) [2023-12-26 22:10:57,665][105620] Updated weights for policy 1, policy_version 936042 (0.0009) [2023-12-26 22:10:58,083][105692] Updated weights for policy 0, policy_version 935923 (0.0007) [2023-12-26 22:10:58,148][105692] Updated weights for policy 0, policy_version 935933 (0.0006) [2023-12-26 22:10:58,207][105692] Updated weights for policy 0, policy_version 935943 (0.0009) [2023-12-26 22:10:58,328][105620] Updated weights for policy 1, policy_version 936052 (0.0008) [2023-12-26 22:10:58,395][105620] Updated weights for policy 1, policy_version 936062 (0.0010) [2023-12-26 22:10:58,463][105620] Updated weights for policy 1, policy_version 936072 (0.0011) [2023-12-26 22:10:59,036][105692] Updated weights for policy 0, policy_version 935953 (0.0008) [2023-12-26 22:10:59,085][105692] Updated weights for policy 0, policy_version 935963 (0.0009) [2023-12-26 22:10:59,139][105692] Updated weights for policy 0, policy_version 935973 (0.0009) [2023-12-26 22:10:59,197][105692] Updated weights for policy 0, policy_version 935983 (0.0009) [2023-12-26 22:10:59,353][105620] Updated weights for policy 1, policy_version 936082 (0.0010) [2023-12-26 22:10:59,422][105620] Updated weights for policy 1, policy_version 936092 (0.0008) [2023-12-26 22:10:59,471][105620] Updated weights for policy 1, policy_version 936102 (0.0010) [2023-12-26 22:10:59,524][105620] Updated weights for policy 1, policy_version 936112 (0.0011) [2023-12-26 22:11:00,061][105692] Updated weights for policy 0, policy_version 935993 (0.0006) [2023-12-26 22:11:00,120][105692] Updated weights for policy 0, policy_version 936003 (0.0009) [2023-12-26 22:11:00,192][105692] Updated weights for policy 0, policy_version 936013 (0.0005) [2023-12-26 22:11:00,290][105620] Updated weights for policy 1, policy_version 936122 (0.0005) [2023-12-26 22:11:00,350][105620] Updated weights for policy 1, policy_version 936132 (0.0006) [2023-12-26 22:11:00,407][105620] Updated weights for policy 1, policy_version 936142 (0.0008) [2023-12-26 22:11:00,892][105692] Updated weights for policy 0, policy_version 936023 (0.0009) [2023-12-26 22:11:00,947][105692] Updated weights for policy 0, policy_version 936033 (0.0009) [2023-12-26 22:11:01,002][105620] Updated weights for policy 1, policy_version 936152 (0.0009) [2023-12-26 22:11:01,008][105692] Updated weights for policy 0, policy_version 936043 (0.0007) [2023-12-26 22:11:01,062][104569] Fps is (10 sec: 18022.2, 60 sec: 18841.6, 300 sec: 19049.9). Total num frames: 479346688. Throughput: 0: 9484.3, 1: 9390.6. Samples: 479319120. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-26 22:11:01,063][104569] Avg episode reward: [(0, '8527.904'), (1, '9083.121')] [2023-12-26 22:11:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000936048_239665152.pth... [2023-12-26 22:11:01,069][105620] Updated weights for policy 1, policy_version 936162 (0.0007) [2023-12-26 22:11:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000934928_239378432.pth [2023-12-26 22:11:01,139][105620] Updated weights for policy 1, policy_version 936172 (0.0007) [2023-12-26 22:11:01,164][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000936176_239689728.pth... [2023-12-26 22:11:01,169][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000935088_239411200.pth [2023-12-26 22:11:01,786][105692] Updated weights for policy 0, policy_version 936053 (0.0008) [2023-12-26 22:11:01,841][105692] Updated weights for policy 0, policy_version 936063 (0.0006) [2023-12-26 22:11:01,873][105620] Updated weights for policy 1, policy_version 936182 (0.0009) [2023-12-26 22:11:01,896][105692] Updated weights for policy 0, policy_version 936073 (0.0006) [2023-12-26 22:11:01,931][105620] Updated weights for policy 1, policy_version 936192 (0.0007) [2023-12-26 22:11:01,995][105620] Updated weights for policy 1, policy_version 936202 (0.0007) [2023-12-26 22:11:02,577][105692] Updated weights for policy 0, policy_version 936083 (0.0007) [2023-12-26 22:11:02,635][105692] Updated weights for policy 0, policy_version 936093 (0.0009) [2023-12-26 22:11:02,691][105692] Updated weights for policy 0, policy_version 936103 (0.0009) [2023-12-26 22:11:02,761][105620] Updated weights for policy 1, policy_version 936212 (0.0008) [2023-12-26 22:11:02,830][105620] Updated weights for policy 1, policy_version 936222 (0.0006) [2023-12-26 22:11:02,898][105620] Updated weights for policy 1, policy_version 936232 (0.0005) [2023-12-26 22:11:03,505][105692] Updated weights for policy 0, policy_version 936113 (0.0009) [2023-12-26 22:11:03,526][105620] Updated weights for policy 1, policy_version 936242 (0.0006) [2023-12-26 22:11:03,563][105692] Updated weights for policy 0, policy_version 936123 (0.0009) [2023-12-26 22:11:03,582][105620] Updated weights for policy 1, policy_version 936252 (0.0008) [2023-12-26 22:11:03,615][105692] Updated weights for policy 0, policy_version 936133 (0.0006) [2023-12-26 22:11:03,638][105620] Updated weights for policy 1, policy_version 936262 (0.0008) [2023-12-26 22:11:03,667][105692] Updated weights for policy 0, policy_version 936143 (0.0008) [2023-12-26 22:11:03,698][105620] Updated weights for policy 1, policy_version 936272 (0.0006) [2023-12-26 22:11:04,354][105620] Updated weights for policy 1, policy_version 936282 (0.0011) [2023-12-26 22:11:04,422][105620] Updated weights for policy 1, policy_version 936292 (0.0011) [2023-12-26 22:11:04,469][105692] Updated weights for policy 0, policy_version 936153 (0.0007) [2023-12-26 22:11:04,483][105620] Updated weights for policy 1, policy_version 936302 (0.0011) [2023-12-26 22:11:04,532][105692] Updated weights for policy 0, policy_version 936163 (0.0007) [2023-12-26 22:11:04,600][105692] Updated weights for policy 0, policy_version 936173 (0.0008) [2023-12-26 22:11:05,262][105620] Updated weights for policy 1, policy_version 936312 (0.0011) [2023-12-26 22:11:05,303][105692] Updated weights for policy 0, policy_version 936183 (0.0006) [2023-12-26 22:11:05,321][105620] Updated weights for policy 1, policy_version 936322 (0.0010) [2023-12-26 22:11:05,355][105692] Updated weights for policy 0, policy_version 936193 (0.0006) [2023-12-26 22:11:05,380][105620] Updated weights for policy 1, policy_version 936332 (0.0011) [2023-12-26 22:11:05,414][105692] Updated weights for policy 0, policy_version 936203 (0.0007) [2023-12-26 22:11:06,045][105620] Updated weights for policy 1, policy_version 936342 (0.0009) [2023-12-26 22:11:06,062][104569] Fps is (10 sec: 18022.4, 60 sec: 18841.6, 300 sec: 19049.9). Total num frames: 479436800. Throughput: 0: 9398.5, 1: 9462.3. Samples: 479430952. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-26 22:11:06,062][104569] Avg episode reward: [(0, '6988.629'), (1, '9268.814')] [2023-12-26 22:11:06,112][105620] Updated weights for policy 1, policy_version 936352 (0.0006) [2023-12-26 22:11:06,175][105620] Updated weights for policy 1, policy_version 936362 (0.0007) [2023-12-26 22:11:06,235][105692] Updated weights for policy 0, policy_version 936213 (0.0009) [2023-12-26 22:11:06,291][105692] Updated weights for policy 0, policy_version 936223 (0.0009) [2023-12-26 22:11:06,351][105692] Updated weights for policy 0, policy_version 936233 (0.0008) [2023-12-26 22:11:06,850][105620] Updated weights for policy 1, policy_version 936372 (0.0008) [2023-12-26 22:11:06,907][105620] Updated weights for policy 1, policy_version 936382 (0.0011) [2023-12-26 22:11:06,974][105620] Updated weights for policy 1, policy_version 936392 (0.0011) [2023-12-26 22:11:07,124][105692] Updated weights for policy 0, policy_version 936243 (0.0009) [2023-12-26 22:11:07,178][105692] Updated weights for policy 0, policy_version 936253 (0.0009) [2023-12-26 22:11:07,245][105692] Updated weights for policy 0, policy_version 936263 (0.0009) [2023-12-26 22:11:07,739][105620] Updated weights for policy 1, policy_version 936402 (0.0010) [2023-12-26 22:11:07,787][105620] Updated weights for policy 1, policy_version 936412 (0.0009) [2023-12-26 22:11:07,837][105620] Updated weights for policy 1, policy_version 936422 (0.0005) [2023-12-26 22:11:07,892][105620] Updated weights for policy 1, policy_version 936432 (0.0006) [2023-12-26 22:11:07,938][105692] Updated weights for policy 0, policy_version 936273 (0.0008) [2023-12-26 22:11:08,005][105692] Updated weights for policy 0, policy_version 936283 (0.0010) [2023-12-26 22:11:08,070][105692] Updated weights for policy 0, policy_version 936293 (0.0010) [2023-12-26 22:11:08,132][105692] Updated weights for policy 0, policy_version 936303 (0.0009) [2023-12-26 22:11:08,527][105620] Updated weights for policy 1, policy_version 936442 (0.0008) [2023-12-26 22:11:08,579][105620] Updated weights for policy 1, policy_version 936452 (0.0008) [2023-12-26 22:11:08,631][105620] Updated weights for policy 1, policy_version 936462 (0.0008) [2023-12-26 22:11:08,927][105692] Updated weights for policy 0, policy_version 936313 (0.0008) [2023-12-26 22:11:08,994][105692] Updated weights for policy 0, policy_version 936323 (0.0011) [2023-12-26 22:11:09,053][105692] Updated weights for policy 0, policy_version 936333 (0.0010) [2023-12-26 22:11:09,439][105620] Updated weights for policy 1, policy_version 936472 (0.0007) [2023-12-26 22:11:09,500][105620] Updated weights for policy 1, policy_version 936482 (0.0006) [2023-12-26 22:11:09,557][105620] Updated weights for policy 1, policy_version 936492 (0.0008) [2023-12-26 22:11:09,817][105692] Updated weights for policy 0, policy_version 936343 (0.0009) [2023-12-26 22:11:09,883][105692] Updated weights for policy 0, policy_version 936353 (0.0008) [2023-12-26 22:11:09,946][105692] Updated weights for policy 0, policy_version 936363 (0.0009) [2023-12-26 22:11:10,293][105620] Updated weights for policy 1, policy_version 936502 (0.0008) [2023-12-26 22:11:10,355][105620] Updated weights for policy 1, policy_version 936512 (0.0008) [2023-12-26 22:11:10,424][105620] Updated weights for policy 1, policy_version 936522 (0.0009) [2023-12-26 22:11:10,661][105692] Updated weights for policy 0, policy_version 936373 (0.0010) [2023-12-26 22:11:10,721][105692] Updated weights for policy 0, policy_version 936383 (0.0010) [2023-12-26 22:11:10,788][105692] Updated weights for policy 0, policy_version 936393 (0.0006) [2023-12-26 22:11:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 18841.6, 300 sec: 19077.6). Total num frames: 479535104. Throughput: 0: 9439.6, 1: 9486.2. Samples: 479544092. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-26 22:11:11,062][104569] Avg episode reward: [(0, '7165.534'), (1, '9268.907')] [2023-12-26 22:11:11,209][105620] Updated weights for policy 1, policy_version 936532 (0.0007) [2023-12-26 22:11:11,278][105620] Updated weights for policy 1, policy_version 936542 (0.0009) [2023-12-26 22:11:11,343][105620] Updated weights for policy 1, policy_version 936552 (0.0008) [2023-12-26 22:11:11,477][105692] Updated weights for policy 0, policy_version 936403 (0.0007) [2023-12-26 22:11:11,530][105692] Updated weights for policy 0, policy_version 936413 (0.0011) [2023-12-26 22:11:11,577][105692] Updated weights for policy 0, policy_version 936423 (0.0011) [2023-12-26 22:11:12,159][105620] Updated weights for policy 1, policy_version 936562 (0.0008) [2023-12-26 22:11:12,211][105620] Updated weights for policy 1, policy_version 936572 (0.0008) [2023-12-26 22:11:12,263][105620] Updated weights for policy 1, policy_version 936582 (0.0008) [2023-12-26 22:11:12,328][105620] Updated weights for policy 1, policy_version 936592 (0.0008) [2023-12-26 22:11:12,371][105692] Updated weights for policy 0, policy_version 936433 (0.0010) [2023-12-26 22:11:12,426][105692] Updated weights for policy 0, policy_version 936443 (0.0006) [2023-12-26 22:11:12,475][105692] Updated weights for policy 0, policy_version 936453 (0.0005) [2023-12-26 22:11:12,524][105692] Updated weights for policy 0, policy_version 936463 (0.0008) [2023-12-26 22:11:13,112][105620] Updated weights for policy 1, policy_version 936602 (0.0008) [2023-12-26 22:11:13,174][105620] Updated weights for policy 1, policy_version 936612 (0.0007) [2023-12-26 22:11:13,220][105692] Updated weights for policy 0, policy_version 936473 (0.0008) [2023-12-26 22:11:13,239][105620] Updated weights for policy 1, policy_version 936622 (0.0007) [2023-12-26 22:11:13,270][105692] Updated weights for policy 0, policy_version 936483 (0.0011) [2023-12-26 22:11:13,319][105692] Updated weights for policy 0, policy_version 936493 (0.0010) [2023-12-26 22:11:13,980][105692] Updated weights for policy 0, policy_version 936503 (0.0007) [2023-12-26 22:11:14,003][105620] Updated weights for policy 1, policy_version 936632 (0.0005) [2023-12-26 22:11:14,036][105692] Updated weights for policy 0, policy_version 936513 (0.0006) [2023-12-26 22:11:14,049][105620] Updated weights for policy 1, policy_version 936642 (0.0005) [2023-12-26 22:11:14,099][105620] Updated weights for policy 1, policy_version 936652 (0.0006) [2023-12-26 22:11:14,099][105692] Updated weights for policy 0, policy_version 936523 (0.0008) [2023-12-26 22:11:14,689][105620] Updated weights for policy 1, policy_version 936662 (0.0006) [2023-12-26 22:11:14,758][105620] Updated weights for policy 1, policy_version 936672 (0.0010) [2023-12-26 22:11:14,820][105620] Updated weights for policy 1, policy_version 936682 (0.0008) [2023-12-26 22:11:14,864][105692] Updated weights for policy 0, policy_version 936533 (0.0008) [2023-12-26 22:11:14,919][105692] Updated weights for policy 0, policy_version 936543 (0.0010) [2023-12-26 22:11:14,974][105692] Updated weights for policy 0, policy_version 936553 (0.0010) [2023-12-26 22:11:15,501][105620] Updated weights for policy 1, policy_version 936692 (0.0008) [2023-12-26 22:11:15,565][105620] Updated weights for policy 1, policy_version 936702 (0.0009) [2023-12-26 22:11:15,629][105620] Updated weights for policy 1, policy_version 936712 (0.0009) [2023-12-26 22:11:15,739][105692] Updated weights for policy 0, policy_version 936563 (0.0009) [2023-12-26 22:11:15,801][105692] Updated weights for policy 0, policy_version 936573 (0.0006) [2023-12-26 22:11:15,855][105692] Updated weights for policy 0, policy_version 936583 (0.0006) [2023-12-26 22:11:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 18841.6, 300 sec: 19077.6). Total num frames: 479633408. Throughput: 0: 9433.8, 1: 9366.2. Samples: 479599624. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-26 22:11:16,062][104569] Avg episode reward: [(0, '8063.519'), (1, '9356.846')] [2023-12-26 22:11:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000936592_239804416.pth... [2023-12-26 22:11:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000936720_239828992.pth... [2023-12-26 22:11:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000935600_239542272.pth [2023-12-26 22:11:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000935504_239525888.pth [2023-12-26 22:11:16,437][105692] Updated weights for policy 0, policy_version 936593 (0.0006) [2023-12-26 22:11:16,446][105620] Updated weights for policy 1, policy_version 936722 (0.0009) [2023-12-26 22:11:16,499][105692] Updated weights for policy 0, policy_version 936603 (0.0006) [2023-12-26 22:11:16,502][105620] Updated weights for policy 1, policy_version 936732 (0.0009) [2023-12-26 22:11:16,553][105620] Updated weights for policy 1, policy_version 936742 (0.0006) [2023-12-26 22:11:16,559][105692] Updated weights for policy 0, policy_version 936613 (0.0006) [2023-12-26 22:11:16,602][105620] Updated weights for policy 1, policy_version 936752 (0.0006) [2023-12-26 22:11:16,617][105692] Updated weights for policy 0, policy_version 936623 (0.0007) [2023-12-26 22:11:17,241][105620] Updated weights for policy 1, policy_version 936762 (0.0009) [2023-12-26 22:11:17,286][105620] Updated weights for policy 1, policy_version 936772 (0.0009) [2023-12-26 22:11:17,336][105620] Updated weights for policy 1, policy_version 936782 (0.0009) [2023-12-26 22:11:17,381][105692] Updated weights for policy 0, policy_version 936633 (0.0008) [2023-12-26 22:11:17,428][105692] Updated weights for policy 0, policy_version 936643 (0.0009) [2023-12-26 22:11:17,483][105692] Updated weights for policy 0, policy_version 936653 (0.0009) [2023-12-26 22:11:18,109][105620] Updated weights for policy 1, policy_version 936792 (0.0008) [2023-12-26 22:11:18,163][105620] Updated weights for policy 1, policy_version 936802 (0.0009) [2023-12-26 22:11:18,216][105620] Updated weights for policy 1, policy_version 936812 (0.0008) [2023-12-26 22:11:18,268][105692] Updated weights for policy 0, policy_version 936663 (0.0007) [2023-12-26 22:11:18,324][105692] Updated weights for policy 0, policy_version 936673 (0.0008) [2023-12-26 22:11:18,388][105692] Updated weights for policy 0, policy_version 936683 (0.0009) [2023-12-26 22:11:18,998][105620] Updated weights for policy 1, policy_version 936822 (0.0008) [2023-12-26 22:11:19,055][105620] Updated weights for policy 1, policy_version 936832 (0.0009) [2023-12-26 22:11:19,111][105620] Updated weights for policy 1, policy_version 936842 (0.0008) [2023-12-26 22:11:19,134][105692] Updated weights for policy 0, policy_version 936693 (0.0007) [2023-12-26 22:11:19,187][105692] Updated weights for policy 0, policy_version 936703 (0.0009) [2023-12-26 22:11:19,254][105692] Updated weights for policy 0, policy_version 936713 (0.0009) [2023-12-26 22:11:19,914][105620] Updated weights for policy 1, policy_version 936852 (0.0009) [2023-12-26 22:11:19,986][105620] Updated weights for policy 1, policy_version 936862 (0.0009) [2023-12-26 22:11:19,998][105692] Updated weights for policy 0, policy_version 936723 (0.0008) [2023-12-26 22:11:20,047][105620] Updated weights for policy 1, policy_version 936872 (0.0009) [2023-12-26 22:11:20,058][105692] Updated weights for policy 0, policy_version 936733 (0.0008) [2023-12-26 22:11:20,114][105692] Updated weights for policy 0, policy_version 936743 (0.0009) [2023-12-26 22:11:20,729][105620] Updated weights for policy 1, policy_version 936882 (0.0009) [2023-12-26 22:11:20,788][105620] Updated weights for policy 1, policy_version 936892 (0.0006) [2023-12-26 22:11:20,841][105620] Updated weights for policy 1, policy_version 936902 (0.0005) [2023-12-26 22:11:20,903][105620] Updated weights for policy 1, policy_version 936912 (0.0006) [2023-12-26 22:11:20,972][105692] Updated weights for policy 0, policy_version 936753 (0.0009) [2023-12-26 22:11:21,033][105692] Updated weights for policy 0, policy_version 936763 (0.0010) [2023-12-26 22:11:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18980.6, 300 sec: 19049.9). Total num frames: 479723520. Throughput: 0: 9331.4, 1: 9458.1. Samples: 479714800. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-26 22:11:21,063][104569] Avg episode reward: [(0, '8375.012'), (1, '9356.749')] [2023-12-26 22:11:21,101][105692] Updated weights for policy 0, policy_version 936773 (0.0007) [2023-12-26 22:11:21,166][105692] Updated weights for policy 0, policy_version 936783 (0.0008) [2023-12-26 22:11:21,645][105620] Updated weights for policy 1, policy_version 936922 (0.0009) [2023-12-26 22:11:21,711][105620] Updated weights for policy 1, policy_version 936932 (0.0009) [2023-12-26 22:11:21,784][105620] Updated weights for policy 1, policy_version 936942 (0.0010) [2023-12-26 22:11:21,849][105692] Updated weights for policy 0, policy_version 936793 (0.0006) [2023-12-26 22:11:21,900][105692] Updated weights for policy 0, policy_version 936803 (0.0007) [2023-12-26 22:11:21,957][105692] Updated weights for policy 0, policy_version 936813 (0.0009) [2023-12-26 22:11:22,629][105620] Updated weights for policy 1, policy_version 936952 (0.0009) [2023-12-26 22:11:22,653][105692] Updated weights for policy 0, policy_version 936823 (0.0006) [2023-12-26 22:11:22,693][105620] Updated weights for policy 1, policy_version 936962 (0.0008) [2023-12-26 22:11:22,712][105692] Updated weights for policy 0, policy_version 936833 (0.0007) [2023-12-26 22:11:22,751][105620] Updated weights for policy 1, policy_version 936972 (0.0009) [2023-12-26 22:11:22,777][105692] Updated weights for policy 0, policy_version 936843 (0.0007) [2023-12-26 22:11:23,450][105620] Updated weights for policy 1, policy_version 936982 (0.0007) [2023-12-26 22:11:23,508][105620] Updated weights for policy 1, policy_version 936992 (0.0008) [2023-12-26 22:11:23,537][105692] Updated weights for policy 0, policy_version 936853 (0.0006) [2023-12-26 22:11:23,556][105620] Updated weights for policy 1, policy_version 937002 (0.0010) [2023-12-26 22:11:23,605][105692] Updated weights for policy 0, policy_version 936863 (0.0005) [2023-12-26 22:11:23,673][105692] Updated weights for policy 0, policy_version 936873 (0.0007) [2023-12-26 22:11:24,295][105692] Updated weights for policy 0, policy_version 936883 (0.0008) [2023-12-26 22:11:24,306][105620] Updated weights for policy 1, policy_version 937012 (0.0009) [2023-12-26 22:11:24,345][105692] Updated weights for policy 0, policy_version 936893 (0.0006) [2023-12-26 22:11:24,369][105620] Updated weights for policy 1, policy_version 937022 (0.0008) [2023-12-26 22:11:24,401][105692] Updated weights for policy 0, policy_version 936903 (0.0009) [2023-12-26 22:11:24,421][105620] Updated weights for policy 1, policy_version 937032 (0.0008) [2023-12-26 22:11:25,050][105620] Updated weights for policy 1, policy_version 937042 (0.0005) [2023-12-26 22:11:25,111][105620] Updated weights for policy 1, policy_version 937052 (0.0005) [2023-12-26 22:11:25,179][105620] Updated weights for policy 1, policy_version 937062 (0.0006) [2023-12-26 22:11:25,238][105620] Updated weights for policy 1, policy_version 937072 (0.0009) [2023-12-26 22:11:25,238][105692] Updated weights for policy 0, policy_version 936913 (0.0009) [2023-12-26 22:11:25,295][105692] Updated weights for policy 0, policy_version 936923 (0.0009) [2023-12-26 22:11:25,355][105692] Updated weights for policy 0, policy_version 936933 (0.0007) [2023-12-26 22:11:25,410][105692] Updated weights for policy 0, policy_version 936943 (0.0009) [2023-12-26 22:11:25,909][105620] Updated weights for policy 1, policy_version 937082 (0.0008) [2023-12-26 22:11:25,968][105620] Updated weights for policy 1, policy_version 937092 (0.0009) [2023-12-26 22:11:26,029][105620] Updated weights for policy 1, policy_version 937102 (0.0009) [2023-12-26 22:11:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18978.1, 300 sec: 19049.9). Total num frames: 479821824. Throughput: 0: 9306.0, 1: 9500.5. Samples: 479828604. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-26 22:11:26,062][104569] Avg episode reward: [(0, '8816.157'), (1, '9356.568')] [2023-12-26 22:11:26,195][105692] Updated weights for policy 0, policy_version 936953 (0.0009) [2023-12-26 22:11:26,247][105692] Updated weights for policy 0, policy_version 936963 (0.0009) [2023-12-26 22:11:26,298][105692] Updated weights for policy 0, policy_version 936973 (0.0009) [2023-12-26 22:11:26,696][105620] Updated weights for policy 1, policy_version 937113 (0.0010) [2023-12-26 22:11:26,743][105620] Updated weights for policy 1, policy_version 937123 (0.0008) [2023-12-26 22:11:26,792][105620] Updated weights for policy 1, policy_version 937133 (0.0008) [2023-12-26 22:11:26,996][105692] Updated weights for policy 0, policy_version 936984 (0.0010) [2023-12-26 22:11:27,048][105692] Updated weights for policy 0, policy_version 936994 (0.0009) [2023-12-26 22:11:27,105][105692] Updated weights for policy 0, policy_version 937004 (0.0009) [2023-12-26 22:11:27,527][105620] Updated weights for policy 1, policy_version 937143 (0.0009) [2023-12-26 22:11:27,579][105620] Updated weights for policy 1, policy_version 937153 (0.0010) [2023-12-26 22:11:27,632][105620] Updated weights for policy 1, policy_version 937164 (0.0010) [2023-12-26 22:11:27,781][105692] Updated weights for policy 0, policy_version 937014 (0.0007) [2023-12-26 22:11:27,839][105692] Updated weights for policy 0, policy_version 937024 (0.0005) [2023-12-26 22:11:27,886][105692] Updated weights for policy 0, policy_version 937034 (0.0008) [2023-12-26 22:11:28,468][105620] Updated weights for policy 1, policy_version 937174 (0.0007) [2023-12-26 22:11:28,491][105692] Updated weights for policy 0, policy_version 937044 (0.0006) [2023-12-26 22:11:28,521][105620] Updated weights for policy 1, policy_version 937184 (0.0007) [2023-12-26 22:11:28,559][105692] Updated weights for policy 0, policy_version 937054 (0.0006) [2023-12-26 22:11:28,578][105620] Updated weights for policy 1, policy_version 937194 (0.0007) [2023-12-26 22:11:28,615][105692] Updated weights for policy 0, policy_version 937064 (0.0008) [2023-12-26 22:11:29,263][105692] Updated weights for policy 0, policy_version 937074 (0.0008) [2023-12-26 22:11:29,269][105620] Updated weights for policy 1, policy_version 937204 (0.0008) [2023-12-26 22:11:29,325][105692] Updated weights for policy 0, policy_version 937084 (0.0008) [2023-12-26 22:11:29,341][105620] Updated weights for policy 1, policy_version 937214 (0.0008) [2023-12-26 22:11:29,393][105692] Updated weights for policy 0, policy_version 937094 (0.0009) [2023-12-26 22:11:29,403][105620] Updated weights for policy 1, policy_version 937224 (0.0009) [2023-12-26 22:11:29,459][105692] Updated weights for policy 0, policy_version 937104 (0.0007) [2023-12-26 22:11:30,090][105692] Updated weights for policy 0, policy_version 937114 (0.0007) [2023-12-26 22:11:30,153][105692] Updated weights for policy 0, policy_version 937124 (0.0006) [2023-12-26 22:11:30,221][105692] Updated weights for policy 0, policy_version 937134 (0.0008) [2023-12-26 22:11:30,246][105620] Updated weights for policy 1, policy_version 937234 (0.0007) [2023-12-26 22:11:30,309][105620] Updated weights for policy 1, policy_version 937244 (0.0011) [2023-12-26 22:11:30,371][105620] Updated weights for policy 1, policy_version 937254 (0.0010) [2023-12-26 22:11:30,434][105620] Updated weights for policy 1, policy_version 937264 (0.0011) [2023-12-26 22:11:30,872][105692] Updated weights for policy 0, policy_version 937144 (0.0009) [2023-12-26 22:11:30,926][105692] Updated weights for policy 0, policy_version 937156 (0.0010) [2023-12-26 22:11:30,980][105692] Updated weights for policy 0, policy_version 937166 (0.0010) [2023-12-26 22:11:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 18979.6, 300 sec: 19049.9). Total num frames: 479920128. Throughput: 0: 9379.1, 1: 9502.4. Samples: 479887692. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-26 22:11:31,062][104569] Avg episode reward: [(0, '8809.938'), (1, '9265.441')] [2023-12-26 22:11:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000937168_239951872.pth... [2023-12-26 22:11:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000936048_239665152.pth [2023-12-26 22:11:31,100][105620] Updated weights for policy 1, policy_version 937274 (0.0008) [2023-12-26 22:11:31,167][105620] Updated weights for policy 1, policy_version 937284 (0.0009) [2023-12-26 22:11:31,230][105620] Updated weights for policy 1, policy_version 937294 (0.0006) [2023-12-26 22:11:31,246][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000937296_239976448.pth... [2023-12-26 22:11:31,252][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000936176_239689728.pth [2023-12-26 22:11:31,866][105692] Updated weights for policy 0, policy_version 937176 (0.0007) [2023-12-26 22:11:31,907][105620] Updated weights for policy 1, policy_version 937304 (0.0007) [2023-12-26 22:11:31,924][105692] Updated weights for policy 0, policy_version 937186 (0.0006) [2023-12-26 22:11:31,958][105620] Updated weights for policy 1, policy_version 937314 (0.0008) [2023-12-26 22:11:31,984][105692] Updated weights for policy 0, policy_version 937196 (0.0007) [2023-12-26 22:11:32,025][105620] Updated weights for policy 1, policy_version 937324 (0.0007) [2023-12-26 22:11:32,771][105620] Updated weights for policy 1, policy_version 937334 (0.0007) [2023-12-26 22:11:32,783][105692] Updated weights for policy 0, policy_version 937206 (0.0007) [2023-12-26 22:11:32,835][105620] Updated weights for policy 1, policy_version 937344 (0.0008) [2023-12-26 22:11:32,846][105692] Updated weights for policy 0, policy_version 937216 (0.0006) [2023-12-26 22:11:32,883][105620] Updated weights for policy 1, policy_version 937354 (0.0007) [2023-12-26 22:11:32,912][105692] Updated weights for policy 0, policy_version 937226 (0.0006) [2023-12-26 22:11:33,496][105620] Updated weights for policy 1, policy_version 937364 (0.0008) [2023-12-26 22:11:33,543][105620] Updated weights for policy 1, policy_version 937374 (0.0008) [2023-12-26 22:11:33,593][105620] Updated weights for policy 1, policy_version 937384 (0.0009) [2023-12-26 22:11:33,672][105692] Updated weights for policy 0, policy_version 937236 (0.0007) [2023-12-26 22:11:33,727][105692] Updated weights for policy 0, policy_version 937246 (0.0005) [2023-12-26 22:11:33,775][105692] Updated weights for policy 0, policy_version 937256 (0.0005) [2023-12-26 22:11:34,422][105620] Updated weights for policy 1, policy_version 937395 (0.0010) [2023-12-26 22:11:34,447][105692] Updated weights for policy 0, policy_version 937266 (0.0006) [2023-12-26 22:11:34,485][105620] Updated weights for policy 1, policy_version 937405 (0.0007) [2023-12-26 22:11:34,507][105692] Updated weights for policy 0, policy_version 937276 (0.0006) [2023-12-26 22:11:34,545][105620] Updated weights for policy 1, policy_version 937415 (0.0008) [2023-12-26 22:11:34,564][105692] Updated weights for policy 0, policy_version 937286 (0.0008) [2023-12-26 22:11:34,619][105692] Updated weights for policy 0, policy_version 937296 (0.0009) [2023-12-26 22:11:35,209][105620] Updated weights for policy 1, policy_version 937425 (0.0007) [2023-12-26 22:11:35,258][105620] Updated weights for policy 1, policy_version 937435 (0.0009) [2023-12-26 22:11:35,306][105620] Updated weights for policy 1, policy_version 937445 (0.0009) [2023-12-26 22:11:35,353][105620] Updated weights for policy 1, policy_version 937455 (0.0008) [2023-12-26 22:11:35,397][105692] Updated weights for policy 0, policy_version 937306 (0.0008) [2023-12-26 22:11:35,459][105692] Updated weights for policy 0, policy_version 937316 (0.0009) [2023-12-26 22:11:35,532][105692] Updated weights for policy 0, policy_version 937326 (0.0009) [2023-12-26 22:11:36,062][104569] Fps is (10 sec: 18841.5, 60 sec: 18841.6, 300 sec: 19022.1). Total num frames: 480010240. Throughput: 0: 9372.8, 1: 9556.3. Samples: 480003576. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-26 22:11:36,062][104569] Avg episode reward: [(0, '9082.107'), (1, '9175.101')] [2023-12-26 22:11:36,119][105620] Updated weights for policy 1, policy_version 937465 (0.0009) [2023-12-26 22:11:36,187][105620] Updated weights for policy 1, policy_version 937475 (0.0007) [2023-12-26 22:11:36,250][105620] Updated weights for policy 1, policy_version 937485 (0.0009) [2023-12-26 22:11:36,299][105692] Updated weights for policy 0, policy_version 937336 (0.0008) [2023-12-26 22:11:36,356][105692] Updated weights for policy 0, policy_version 937346 (0.0009) [2023-12-26 22:11:36,419][105692] Updated weights for policy 0, policy_version 937356 (0.0010) [2023-12-26 22:11:36,994][105620] Updated weights for policy 1, policy_version 937495 (0.0009) [2023-12-26 22:11:37,058][105620] Updated weights for policy 1, policy_version 937505 (0.0008) [2023-12-26 22:11:37,110][105620] Updated weights for policy 1, policy_version 937515 (0.0005) [2023-12-26 22:11:37,209][105692] Updated weights for policy 0, policy_version 937366 (0.0010) [2023-12-26 22:11:37,263][105692] Updated weights for policy 0, policy_version 937376 (0.0010) [2023-12-26 22:11:37,332][105692] Updated weights for policy 0, policy_version 937386 (0.0010) [2023-12-26 22:11:37,741][105620] Updated weights for policy 1, policy_version 937525 (0.0008) [2023-12-26 22:11:37,798][105620] Updated weights for policy 1, policy_version 937535 (0.0011) [2023-12-26 22:11:37,856][105620] Updated weights for policy 1, policy_version 937545 (0.0010) [2023-12-26 22:11:38,125][105692] Updated weights for policy 0, policy_version 937396 (0.0008) [2023-12-26 22:11:38,180][105692] Updated weights for policy 0, policy_version 937406 (0.0007) [2023-12-26 22:11:38,232][105692] Updated weights for policy 0, policy_version 937416 (0.0008) [2023-12-26 22:11:38,580][105620] Updated weights for policy 1, policy_version 937555 (0.0010) [2023-12-26 22:11:38,631][105620] Updated weights for policy 1, policy_version 937565 (0.0009) [2023-12-26 22:11:38,690][105620] Updated weights for policy 1, policy_version 937575 (0.0009) [2023-12-26 22:11:39,013][105692] Updated weights for policy 0, policy_version 937426 (0.0008) [2023-12-26 22:11:39,064][105692] Updated weights for policy 0, policy_version 937436 (0.0009) [2023-12-26 22:11:39,116][105692] Updated weights for policy 0, policy_version 937446 (0.0009) [2023-12-26 22:11:39,170][105692] Updated weights for policy 0, policy_version 937456 (0.0007) [2023-12-26 22:11:39,473][105620] Updated weights for policy 1, policy_version 937585 (0.0008) [2023-12-26 22:11:39,538][105620] Updated weights for policy 1, policy_version 937595 (0.0009) [2023-12-26 22:11:39,601][105620] Updated weights for policy 1, policy_version 937605 (0.0009) [2023-12-26 22:11:39,663][105620] Updated weights for policy 1, policy_version 937615 (0.0009) [2023-12-26 22:11:39,950][105692] Updated weights for policy 0, policy_version 937466 (0.0009) [2023-12-26 22:11:40,013][105692] Updated weights for policy 0, policy_version 937476 (0.0008) [2023-12-26 22:11:40,080][105692] Updated weights for policy 0, policy_version 937486 (0.0009) [2023-12-26 22:11:40,425][105620] Updated weights for policy 1, policy_version 937625 (0.0009) [2023-12-26 22:11:40,482][105620] Updated weights for policy 1, policy_version 937635 (0.0007) [2023-12-26 22:11:40,536][105620] Updated weights for policy 1, policy_version 937645 (0.0005) [2023-12-26 22:11:40,897][105692] Updated weights for policy 0, policy_version 937496 (0.0010) [2023-12-26 22:11:40,957][105692] Updated weights for policy 0, policy_version 937506 (0.0010) [2023-12-26 22:11:41,022][105692] Updated weights for policy 0, policy_version 937516 (0.0011) [2023-12-26 22:11:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18978.2, 300 sec: 19049.9). Total num frames: 480108544. Throughput: 0: 9369.3, 1: 9553.2. Samples: 480114624. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-26 22:11:41,063][104569] Avg episode reward: [(0, '8955.142'), (1, '9266.166')] [2023-12-26 22:11:41,297][105620] Updated weights for policy 1, policy_version 937655 (0.0007) [2023-12-26 22:11:41,352][105620] Updated weights for policy 1, policy_version 937665 (0.0008) [2023-12-26 22:11:41,415][105620] Updated weights for policy 1, policy_version 937675 (0.0008) [2023-12-26 22:11:41,812][105692] Updated weights for policy 0, policy_version 937526 (0.0011) [2023-12-26 22:11:41,868][105692] Updated weights for policy 0, policy_version 937536 (0.0011) [2023-12-26 22:11:41,917][105692] Updated weights for policy 0, policy_version 937546 (0.0010) [2023-12-26 22:11:42,252][105620] Updated weights for policy 1, policy_version 937685 (0.0009) [2023-12-26 22:11:42,313][105620] Updated weights for policy 1, policy_version 937695 (0.0008) [2023-12-26 22:11:42,376][105620] Updated weights for policy 1, policy_version 937705 (0.0008) [2023-12-26 22:11:42,701][105692] Updated weights for policy 0, policy_version 937556 (0.0011) [2023-12-26 22:11:42,758][105692] Updated weights for policy 0, policy_version 937566 (0.0011) [2023-12-26 22:11:42,820][105692] Updated weights for policy 0, policy_version 937576 (0.0011) [2023-12-26 22:11:43,158][105620] Updated weights for policy 1, policy_version 937715 (0.0008) [2023-12-26 22:11:43,210][105620] Updated weights for policy 1, policy_version 937725 (0.0008) [2023-12-26 22:11:43,262][105620] Updated weights for policy 1, policy_version 937735 (0.0008) [2023-12-26 22:11:43,562][105692] Updated weights for policy 0, policy_version 937586 (0.0010) [2023-12-26 22:11:43,610][105692] Updated weights for policy 0, policy_version 937596 (0.0010) [2023-12-26 22:11:43,661][105692] Updated weights for policy 0, policy_version 937606 (0.0010) [2023-12-26 22:11:43,713][105692] Updated weights for policy 0, policy_version 937616 (0.0010) [2023-12-26 22:11:44,040][105620] Updated weights for policy 1, policy_version 937745 (0.0008) [2023-12-26 22:11:44,084][105620] Updated weights for policy 1, policy_version 937755 (0.0008) [2023-12-26 22:11:44,144][105620] Updated weights for policy 1, policy_version 937765 (0.0008) [2023-12-26 22:11:44,196][105620] Updated weights for policy 1, policy_version 937775 (0.0008) [2023-12-26 22:11:44,434][105692] Updated weights for policy 0, policy_version 937626 (0.0005) [2023-12-26 22:11:44,492][105692] Updated weights for policy 0, policy_version 937636 (0.0007) [2023-12-26 22:11:44,540][105692] Updated weights for policy 0, policy_version 937646 (0.0010) [2023-12-26 22:11:45,014][105620] Updated weights for policy 1, policy_version 937785 (0.0007) [2023-12-26 22:11:45,075][105620] Updated weights for policy 1, policy_version 937795 (0.0008) [2023-12-26 22:11:45,131][105620] Updated weights for policy 1, policy_version 937805 (0.0006) [2023-12-26 22:11:45,288][105692] Updated weights for policy 0, policy_version 937656 (0.0010) [2023-12-26 22:11:45,354][105692] Updated weights for policy 0, policy_version 937666 (0.0006) [2023-12-26 22:11:45,421][105692] Updated weights for policy 0, policy_version 937676 (0.0008) [2023-12-26 22:11:45,855][105620] Updated weights for policy 1, policy_version 937815 (0.0008) [2023-12-26 22:11:45,908][105620] Updated weights for policy 1, policy_version 937825 (0.0009) [2023-12-26 22:11:45,964][105620] Updated weights for policy 1, policy_version 937835 (0.0010) [2023-12-26 22:11:45,996][105692] Updated weights for policy 0, policy_version 937686 (0.0009) [2023-12-26 22:11:46,041][105692] Updated weights for policy 0, policy_version 937696 (0.0010) [2023-12-26 22:11:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 18841.5, 300 sec: 19022.1). Total num frames: 480198656. Throughput: 0: 9351.9, 1: 9520.8. Samples: 480168392. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-26 22:11:46,062][104569] Avg episode reward: [(0, '5292.944'), (1, '9356.593')] [2023-12-26 22:11:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000937840_240115712.pth... [2023-12-26 22:11:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000936720_239828992.pth [2023-12-26 22:11:46,094][105692] Updated weights for policy 0, policy_version 937706 (0.0011) [2023-12-26 22:11:46,120][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000937712_240091136.pth... [2023-12-26 22:11:46,123][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000936592_239804416.pth [2023-12-26 22:11:46,758][105620] Updated weights for policy 1, policy_version 937845 (0.0008) [2023-12-26 22:11:46,777][105692] Updated weights for policy 0, policy_version 937716 (0.0009) [2023-12-26 22:11:46,818][105620] Updated weights for policy 1, policy_version 937855 (0.0008) [2023-12-26 22:11:46,826][105692] Updated weights for policy 0, policy_version 937726 (0.0006) [2023-12-26 22:11:46,874][105620] Updated weights for policy 1, policy_version 937865 (0.0009) [2023-12-26 22:11:46,877][105692] Updated weights for policy 0, policy_version 937736 (0.0005) [2023-12-26 22:11:47,471][105692] Updated weights for policy 0, policy_version 937746 (0.0005) [2023-12-26 22:11:47,530][105692] Updated weights for policy 0, policy_version 937756 (0.0006) [2023-12-26 22:11:47,597][105692] Updated weights for policy 0, policy_version 937766 (0.0005) [2023-12-26 22:11:47,663][105692] Updated weights for policy 0, policy_version 937776 (0.0005) [2023-12-26 22:11:47,719][105620] Updated weights for policy 1, policy_version 937875 (0.0009) [2023-12-26 22:11:47,772][105620] Updated weights for policy 1, policy_version 937885 (0.0009) [2023-12-26 22:11:47,830][105620] Updated weights for policy 1, policy_version 937895 (0.0010) [2023-12-26 22:11:48,184][105692] Updated weights for policy 0, policy_version 937786 (0.0005) [2023-12-26 22:11:48,251][105692] Updated weights for policy 0, policy_version 937796 (0.0005) [2023-12-26 22:11:48,316][105692] Updated weights for policy 0, policy_version 937806 (0.0005) [2023-12-26 22:11:48,613][105620] Updated weights for policy 1, policy_version 937905 (0.0009) [2023-12-26 22:11:48,672][105620] Updated weights for policy 1, policy_version 937915 (0.0010) [2023-12-26 22:11:48,731][105620] Updated weights for policy 1, policy_version 937925 (0.0010) [2023-12-26 22:11:48,794][105620] Updated weights for policy 1, policy_version 937935 (0.0010) [2023-12-26 22:11:49,022][105692] Updated weights for policy 0, policy_version 937816 (0.0008) [2023-12-26 22:11:49,087][105692] Updated weights for policy 0, policy_version 937826 (0.0008) [2023-12-26 22:11:49,134][105692] Updated weights for policy 0, policy_version 937836 (0.0008) [2023-12-26 22:11:49,564][105620] Updated weights for policy 1, policy_version 937945 (0.0010) [2023-12-26 22:11:49,621][105620] Updated weights for policy 1, policy_version 937955 (0.0010) [2023-12-26 22:11:49,670][105620] Updated weights for policy 1, policy_version 937965 (0.0010) [2023-12-26 22:11:49,984][105692] Updated weights for policy 0, policy_version 937846 (0.0008) [2023-12-26 22:11:50,050][105692] Updated weights for policy 0, policy_version 937856 (0.0010) [2023-12-26 22:11:50,114][105692] Updated weights for policy 0, policy_version 937866 (0.0010) [2023-12-26 22:11:50,409][105620] Updated weights for policy 1, policy_version 937975 (0.0010) [2023-12-26 22:11:50,461][105620] Updated weights for policy 1, policy_version 937985 (0.0009) [2023-12-26 22:11:50,514][105620] Updated weights for policy 1, policy_version 937995 (0.0009) [2023-12-26 22:11:50,898][105692] Updated weights for policy 0, policy_version 937876 (0.0010) [2023-12-26 22:11:50,961][105692] Updated weights for policy 0, policy_version 937886 (0.0011) [2023-12-26 22:11:51,014][105692] Updated weights for policy 0, policy_version 937896 (0.0011) [2023-12-26 22:11:51,062][104569] Fps is (10 sec: 18022.3, 60 sec: 18705.1, 300 sec: 18994.3). Total num frames: 480288768. Throughput: 0: 9547.2, 1: 9422.3. Samples: 480284580. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-26 22:11:51,062][104569] Avg episode reward: [(0, '3602.273'), (1, '9356.472')] [2023-12-26 22:11:51,290][105620] Updated weights for policy 1, policy_version 938005 (0.0009) [2023-12-26 22:11:51,351][105620] Updated weights for policy 1, policy_version 938015 (0.0007) [2023-12-26 22:11:51,413][105620] Updated weights for policy 1, policy_version 938025 (0.0009) [2023-12-26 22:11:51,743][105692] Updated weights for policy 0, policy_version 937906 (0.0010) [2023-12-26 22:11:51,800][105692] Updated weights for policy 0, policy_version 937916 (0.0010) [2023-12-26 22:11:51,863][105692] Updated weights for policy 0, policy_version 937926 (0.0010) [2023-12-26 22:11:51,923][105692] Updated weights for policy 0, policy_version 937936 (0.0010) [2023-12-26 22:11:52,109][105620] Updated weights for policy 1, policy_version 938035 (0.0010) [2023-12-26 22:11:52,160][105620] Updated weights for policy 1, policy_version 938045 (0.0009) [2023-12-26 22:11:52,217][105620] Updated weights for policy 1, policy_version 938055 (0.0008) [2023-12-26 22:11:52,656][105692] Updated weights for policy 0, policy_version 937946 (0.0009) [2023-12-26 22:11:52,708][105692] Updated weights for policy 0, policy_version 937956 (0.0009) [2023-12-26 22:11:52,766][105692] Updated weights for policy 0, policy_version 937966 (0.0007) [2023-12-26 22:11:52,996][105620] Updated weights for policy 1, policy_version 938065 (0.0009) [2023-12-26 22:11:53,061][105620] Updated weights for policy 1, policy_version 938075 (0.0008) [2023-12-26 22:11:53,116][105620] Updated weights for policy 1, policy_version 938085 (0.0008) [2023-12-26 22:11:53,168][105620] Updated weights for policy 1, policy_version 938095 (0.0008) [2023-12-26 22:11:53,547][105692] Updated weights for policy 0, policy_version 937976 (0.0009) [2023-12-26 22:11:53,602][105692] Updated weights for policy 0, policy_version 937986 (0.0011) [2023-12-26 22:11:53,657][105692] Updated weights for policy 0, policy_version 937996 (0.0010) [2023-12-26 22:11:53,921][105620] Updated weights for policy 1, policy_version 938105 (0.0008) [2023-12-26 22:11:53,973][105620] Updated weights for policy 1, policy_version 938115 (0.0009) [2023-12-26 22:11:54,030][105620] Updated weights for policy 1, policy_version 938125 (0.0008) [2023-12-26 22:11:54,324][105692] Updated weights for policy 0, policy_version 938006 (0.0007) [2023-12-26 22:11:54,378][105692] Updated weights for policy 0, policy_version 938016 (0.0008) [2023-12-26 22:11:54,430][105692] Updated weights for policy 0, policy_version 938026 (0.0010) [2023-12-26 22:11:54,853][105620] Updated weights for policy 1, policy_version 938135 (0.0008) [2023-12-26 22:11:54,904][105620] Updated weights for policy 1, policy_version 938145 (0.0005) [2023-12-26 22:11:54,962][105620] Updated weights for policy 1, policy_version 938155 (0.0007) [2023-12-26 22:11:55,141][105692] Updated weights for policy 0, policy_version 938036 (0.0010) [2023-12-26 22:11:55,198][105692] Updated weights for policy 0, policy_version 938046 (0.0010) [2023-12-26 22:11:55,263][105692] Updated weights for policy 0, policy_version 938056 (0.0010) [2023-12-26 22:11:55,658][105620] Updated weights for policy 1, policy_version 938165 (0.0007) [2023-12-26 22:11:55,713][105620] Updated weights for policy 1, policy_version 938175 (0.0005) [2023-12-26 22:11:55,769][105620] Updated weights for policy 1, policy_version 938185 (0.0007) [2023-12-26 22:11:56,013][105692] Updated weights for policy 0, policy_version 938066 (0.0010) [2023-12-26 22:11:56,062][104569] Fps is (10 sec: 18842.0, 60 sec: 18841.6, 300 sec: 18994.4). Total num frames: 480387072. Throughput: 0: 9567.8, 1: 9392.1. Samples: 480397288. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-26 22:11:56,062][104569] Avg episode reward: [(0, '7049.065'), (1, '9264.876')] [2023-12-26 22:11:56,072][105692] Updated weights for policy 0, policy_version 938076 (0.0010) [2023-12-26 22:11:56,138][105692] Updated weights for policy 0, policy_version 938086 (0.0011) [2023-12-26 22:11:56,207][105692] Updated weights for policy 0, policy_version 938096 (0.0005) [2023-12-26 22:11:56,486][105620] Updated weights for policy 1, policy_version 938195 (0.0009) [2023-12-26 22:11:56,541][105620] Updated weights for policy 1, policy_version 938205 (0.0010) [2023-12-26 22:11:56,602][105620] Updated weights for policy 1, policy_version 938215 (0.0010) [2023-12-26 22:11:56,881][105692] Updated weights for policy 0, policy_version 938106 (0.0010) [2023-12-26 22:11:56,925][105692] Updated weights for policy 0, policy_version 938116 (0.0010) [2023-12-26 22:11:56,979][105692] Updated weights for policy 0, policy_version 938126 (0.0010) [2023-12-26 22:11:57,320][105620] Updated weights for policy 1, policy_version 938225 (0.0010) [2023-12-26 22:11:57,377][105620] Updated weights for policy 1, policy_version 938235 (0.0008) [2023-12-26 22:11:57,441][105620] Updated weights for policy 1, policy_version 938245 (0.0006) [2023-12-26 22:11:57,507][105620] Updated weights for policy 1, policy_version 938255 (0.0005) [2023-12-26 22:11:57,738][105692] Updated weights for policy 0, policy_version 938136 (0.0010) [2023-12-26 22:11:57,785][105692] Updated weights for policy 0, policy_version 938146 (0.0010) [2023-12-26 22:11:57,829][105692] Updated weights for policy 0, policy_version 938156 (0.0010) [2023-12-26 22:11:58,202][105620] Updated weights for policy 1, policy_version 938265 (0.0008) [2023-12-26 22:11:58,258][105620] Updated weights for policy 1, policy_version 938275 (0.0008) [2023-12-26 22:11:58,321][105620] Updated weights for policy 1, policy_version 938285 (0.0008) [2023-12-26 22:11:58,644][105692] Updated weights for policy 0, policy_version 938166 (0.0010) [2023-12-26 22:11:58,711][105692] Updated weights for policy 0, policy_version 938176 (0.0010) [2023-12-26 22:11:58,779][105692] Updated weights for policy 0, policy_version 938186 (0.0010) [2023-12-26 22:11:59,196][105620] Updated weights for policy 1, policy_version 938295 (0.0009) [2023-12-26 22:11:59,267][105620] Updated weights for policy 1, policy_version 938305 (0.0009) [2023-12-26 22:11:59,337][105620] Updated weights for policy 1, policy_version 938315 (0.0008) [2023-12-26 22:11:59,583][105692] Updated weights for policy 0, policy_version 938196 (0.0009) [2023-12-26 22:11:59,630][105692] Updated weights for policy 0, policy_version 938206 (0.0009) [2023-12-26 22:11:59,685][105692] Updated weights for policy 0, policy_version 938217 (0.0009) [2023-12-26 22:12:00,090][105620] Updated weights for policy 1, policy_version 938325 (0.0007) [2023-12-26 22:12:00,139][105620] Updated weights for policy 1, policy_version 938335 (0.0005) [2023-12-26 22:12:00,198][105620] Updated weights for policy 1, policy_version 938345 (0.0007) [2023-12-26 22:12:00,461][105692] Updated weights for policy 0, policy_version 938227 (0.0008) [2023-12-26 22:12:00,523][105692] Updated weights for policy 0, policy_version 938237 (0.0007) [2023-12-26 22:12:00,598][105692] Updated weights for policy 0, policy_version 938247 (0.0007) [2023-12-26 22:12:00,871][105620] Updated weights for policy 1, policy_version 938355 (0.0008) [2023-12-26 22:12:00,926][105620] Updated weights for policy 1, policy_version 938365 (0.0005) [2023-12-26 22:12:00,974][105620] Updated weights for policy 1, policy_version 938375 (0.0005) [2023-12-26 22:12:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 18978.1, 300 sec: 18994.3). Total num frames: 480485376. Throughput: 0: 9558.3, 1: 9420.0. Samples: 480453648. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:12:01,062][104569] Avg episode reward: [(0, '8994.146'), (1, '8924.666')] [2023-12-26 22:12:01,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000938256_240230400.pth... [2023-12-26 22:12:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000938384_240254976.pth... [2023-12-26 22:12:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000937168_239951872.pth [2023-12-26 22:12:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000937296_239976448.pth [2023-12-26 22:12:01,360][105692] Updated weights for policy 0, policy_version 938257 (0.0008) [2023-12-26 22:12:01,431][105692] Updated weights for policy 0, policy_version 938267 (0.0008) [2023-12-26 22:12:01,499][105692] Updated weights for policy 0, policy_version 938277 (0.0008) [2023-12-26 22:12:01,564][105692] Updated weights for policy 0, policy_version 938287 (0.0008) [2023-12-26 22:12:01,664][105620] Updated weights for policy 1, policy_version 938385 (0.0007) [2023-12-26 22:12:01,732][105620] Updated weights for policy 1, policy_version 938395 (0.0009) [2023-12-26 22:12:01,790][105620] Updated weights for policy 1, policy_version 938405 (0.0006) [2023-12-26 22:12:01,853][105620] Updated weights for policy 1, policy_version 938415 (0.0006) [2023-12-26 22:12:02,281][105692] Updated weights for policy 0, policy_version 938297 (0.0009) [2023-12-26 22:12:02,341][105692] Updated weights for policy 0, policy_version 938307 (0.0008) [2023-12-26 22:12:02,396][105692] Updated weights for policy 0, policy_version 938317 (0.0008) [2023-12-26 22:12:02,583][105620] Updated weights for policy 1, policy_version 938425 (0.0010) [2023-12-26 22:12:02,639][105620] Updated weights for policy 1, policy_version 938435 (0.0010) [2023-12-26 22:12:02,665][105586] KL-divergence is very high: 103.2822 [2023-12-26 22:12:02,699][105620] Updated weights for policy 1, policy_version 938445 (0.0009) [2023-12-26 22:12:03,081][105692] Updated weights for policy 0, policy_version 938327 (0.0008) [2023-12-26 22:12:03,139][105692] Updated weights for policy 0, policy_version 938337 (0.0008) [2023-12-26 22:12:03,185][105692] Updated weights for policy 0, policy_version 938347 (0.0006) [2023-12-26 22:12:03,485][105620] Updated weights for policy 1, policy_version 938455 (0.0010) [2023-12-26 22:12:03,537][105620] Updated weights for policy 1, policy_version 938465 (0.0010) [2023-12-26 22:12:03,589][105620] Updated weights for policy 1, policy_version 938475 (0.0010) [2023-12-26 22:12:03,749][105692] Updated weights for policy 0, policy_version 938357 (0.0005) [2023-12-26 22:12:03,796][105692] Updated weights for policy 0, policy_version 938367 (0.0005) [2023-12-26 22:12:03,850][105692] Updated weights for policy 0, policy_version 938377 (0.0006) [2023-12-26 22:12:04,320][105620] Updated weights for policy 1, policy_version 938485 (0.0010) [2023-12-26 22:12:04,386][105620] Updated weights for policy 1, policy_version 938495 (0.0010) [2023-12-26 22:12:04,448][105620] Updated weights for policy 1, policy_version 938505 (0.0010) [2023-12-26 22:12:04,531][105692] Updated weights for policy 0, policy_version 938387 (0.0008) [2023-12-26 22:12:04,578][105692] Updated weights for policy 0, policy_version 938397 (0.0008) [2023-12-26 22:12:04,638][105692] Updated weights for policy 0, policy_version 938407 (0.0008) [2023-12-26 22:12:05,164][105620] Updated weights for policy 1, policy_version 938515 (0.0010) [2023-12-26 22:12:05,211][105620] Updated weights for policy 1, policy_version 938525 (0.0010) [2023-12-26 22:12:05,272][105620] Updated weights for policy 1, policy_version 938535 (0.0010) [2023-12-26 22:12:05,417][105692] Updated weights for policy 0, policy_version 938417 (0.0008) [2023-12-26 22:12:05,479][105692] Updated weights for policy 0, policy_version 938427 (0.0008) [2023-12-26 22:12:05,546][105692] Updated weights for policy 0, policy_version 938437 (0.0008) [2023-12-26 22:12:05,612][105692] Updated weights for policy 0, policy_version 938447 (0.0009) [2023-12-26 22:12:06,024][105620] Updated weights for policy 1, policy_version 938545 (0.0010) [2023-12-26 22:12:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18978.1, 300 sec: 18994.3). Total num frames: 480575488. Throughput: 0: 9564.0, 1: 9405.9. Samples: 480568444. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:12:06,062][104569] Avg episode reward: [(0, '8887.892'), (1, '8759.227')] [2023-12-26 22:12:06,087][105620] Updated weights for policy 1, policy_version 938555 (0.0010) [2023-12-26 22:12:06,163][105620] Updated weights for policy 1, policy_version 938566 (0.0008) [2023-12-26 22:12:06,232][105620] Updated weights for policy 1, policy_version 938576 (0.0007) [2023-12-26 22:12:06,386][105692] Updated weights for policy 0, policy_version 938457 (0.0009) [2023-12-26 22:12:06,449][105692] Updated weights for policy 0, policy_version 938467 (0.0009) [2023-12-26 22:12:06,510][105692] Updated weights for policy 0, policy_version 938477 (0.0009) [2023-12-26 22:12:06,883][105620] Updated weights for policy 1, policy_version 938586 (0.0009) [2023-12-26 22:12:06,945][105620] Updated weights for policy 1, policy_version 938596 (0.0009) [2023-12-26 22:12:07,008][105620] Updated weights for policy 1, policy_version 938606 (0.0009) [2023-12-26 22:12:07,254][105692] Updated weights for policy 0, policy_version 938487 (0.0006) [2023-12-26 22:12:07,307][105692] Updated weights for policy 0, policy_version 938497 (0.0005) [2023-12-26 22:12:07,366][105692] Updated weights for policy 0, policy_version 938507 (0.0007) [2023-12-26 22:12:07,756][105620] Updated weights for policy 1, policy_version 938616 (0.0007) [2023-12-26 22:12:07,800][105620] Updated weights for policy 1, policy_version 938626 (0.0005) [2023-12-26 22:12:07,847][105620] Updated weights for policy 1, policy_version 938636 (0.0007) [2023-12-26 22:12:08,090][105692] Updated weights for policy 0, policy_version 938517 (0.0010) [2023-12-26 22:12:08,158][105692] Updated weights for policy 0, policy_version 938527 (0.0010) [2023-12-26 22:12:08,213][105692] Updated weights for policy 0, policy_version 938537 (0.0010) [2023-12-26 22:12:08,534][105620] Updated weights for policy 1, policy_version 938646 (0.0008) [2023-12-26 22:12:08,596][105620] Updated weights for policy 1, policy_version 938656 (0.0005) [2023-12-26 22:12:08,654][105620] Updated weights for policy 1, policy_version 938666 (0.0005) [2023-12-26 22:12:08,958][105692] Updated weights for policy 0, policy_version 938547 (0.0010) [2023-12-26 22:12:09,006][105692] Updated weights for policy 0, policy_version 938557 (0.0010) [2023-12-26 22:12:09,062][105692] Updated weights for policy 0, policy_version 938567 (0.0010) [2023-12-26 22:12:09,287][105620] Updated weights for policy 1, policy_version 938676 (0.0006) [2023-12-26 22:12:09,356][105620] Updated weights for policy 1, policy_version 938686 (0.0008) [2023-12-26 22:12:09,424][105620] Updated weights for policy 1, policy_version 938696 (0.0010) [2023-12-26 22:12:09,948][105692] Updated weights for policy 0, policy_version 938577 (0.0010) [2023-12-26 22:12:10,014][105692] Updated weights for policy 0, policy_version 938587 (0.0008) [2023-12-26 22:12:10,071][105620] Updated weights for policy 1, policy_version 938706 (0.0010) [2023-12-26 22:12:10,081][105692] Updated weights for policy 0, policy_version 938597 (0.0007) [2023-12-26 22:12:10,127][105620] Updated weights for policy 1, policy_version 938716 (0.0011) [2023-12-26 22:12:10,141][105692] Updated weights for policy 0, policy_version 938607 (0.0006) [2023-12-26 22:12:10,194][105620] Updated weights for policy 1, policy_version 938726 (0.0007) [2023-12-26 22:12:10,260][105620] Updated weights for policy 1, policy_version 938736 (0.0010) [2023-12-26 22:12:10,861][105620] Updated weights for policy 1, policy_version 938746 (0.0008) [2023-12-26 22:12:10,919][105620] Updated weights for policy 1, policy_version 938756 (0.0009) [2023-12-26 22:12:10,946][105692] Updated weights for policy 0, policy_version 938617 (0.0007) [2023-12-26 22:12:11,005][105620] Updated weights for policy 1, policy_version 938766 (0.0008) [2023-12-26 22:12:11,013][105692] Updated weights for policy 0, policy_version 938627 (0.0010) [2023-12-26 22:12:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18978.1, 300 sec: 19022.1). Total num frames: 480673792. Throughput: 0: 9530.9, 1: 9461.0. Samples: 480683240. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:12:11,063][104569] Avg episode reward: [(0, '8888.660'), (1, '8848.381')] [2023-12-26 22:12:11,083][105692] Updated weights for policy 0, policy_version 938637 (0.0009) [2023-12-26 22:12:11,768][105620] Updated weights for policy 1, policy_version 938776 (0.0009) [2023-12-26 22:12:11,846][105620] Updated weights for policy 1, policy_version 938786 (0.0007) [2023-12-26 22:12:11,899][105692] Updated weights for policy 0, policy_version 938647 (0.0010) [2023-12-26 22:12:11,924][105620] Updated weights for policy 1, policy_version 938796 (0.0006) [2023-12-26 22:12:11,964][105692] Updated weights for policy 0, policy_version 938657 (0.0008) [2023-12-26 22:12:12,023][105692] Updated weights for policy 0, policy_version 938667 (0.0009) [2023-12-26 22:12:12,561][105620] Updated weights for policy 1, policy_version 938806 (0.0007) [2023-12-26 22:12:12,628][105620] Updated weights for policy 1, policy_version 938816 (0.0008) [2023-12-26 22:12:12,686][105620] Updated weights for policy 1, policy_version 938826 (0.0008) [2023-12-26 22:12:12,791][105692] Updated weights for policy 0, policy_version 938678 (0.0010) [2023-12-26 22:12:12,856][105692] Updated weights for policy 0, policy_version 938688 (0.0009) [2023-12-26 22:12:12,920][105692] Updated weights for policy 0, policy_version 938698 (0.0010) [2023-12-26 22:12:13,332][105620] Updated weights for policy 1, policy_version 938836 (0.0007) [2023-12-26 22:12:13,394][105620] Updated weights for policy 1, policy_version 938846 (0.0007) [2023-12-26 22:12:13,447][105586] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000000 [2023-12-26 22:12:13,448][105620] Updated weights for policy 1, policy_version 938856 (0.0010) [2023-12-26 22:12:13,618][105692] Updated weights for policy 0, policy_version 938708 (0.0008) [2023-12-26 22:12:13,670][105692] Updated weights for policy 0, policy_version 938718 (0.0005) [2023-12-26 22:12:13,725][105692] Updated weights for policy 0, policy_version 938728 (0.0005) [2023-12-26 22:12:14,210][105620] Updated weights for policy 1, policy_version 938866 (0.0010) [2023-12-26 22:12:14,275][105620] Updated weights for policy 1, policy_version 938876 (0.0010) [2023-12-26 22:12:14,338][105620] Updated weights for policy 1, policy_version 938886 (0.0010) [2023-12-26 22:12:14,352][105692] Updated weights for policy 0, policy_version 938738 (0.0006) [2023-12-26 22:12:14,414][105692] Updated weights for policy 0, policy_version 938748 (0.0008) [2023-12-26 22:12:14,462][105692] Updated weights for policy 0, policy_version 938758 (0.0008) [2023-12-26 22:12:14,518][105692] Updated weights for policy 0, policy_version 938768 (0.0005) [2023-12-26 22:12:15,053][105620] Updated weights for policy 1, policy_version 938896 (0.0008) [2023-12-26 22:12:15,112][105620] Updated weights for policy 1, policy_version 938906 (0.0008) [2023-12-26 22:12:15,169][105620] Updated weights for policy 1, policy_version 938916 (0.0008) [2023-12-26 22:12:15,201][105692] Updated weights for policy 0, policy_version 938778 (0.0011) [2023-12-26 22:12:15,265][105692] Updated weights for policy 0, policy_version 938788 (0.0011) [2023-12-26 22:12:15,332][105692] Updated weights for policy 0, policy_version 938798 (0.0011) [2023-12-26 22:12:15,899][105620] Updated weights for policy 1, policy_version 938926 (0.0009) [2023-12-26 22:12:15,948][105620] Updated weights for policy 1, policy_version 938936 (0.0010) [2023-12-26 22:12:16,010][105620] Updated weights for policy 1, policy_version 938946 (0.0010) [2023-12-26 22:12:16,061][105692] Updated weights for policy 0, policy_version 938808 (0.0009) [2023-12-26 22:12:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 18978.1, 300 sec: 19022.1). Total num frames: 480772096. Throughput: 0: 9449.8, 1: 9483.1. Samples: 480739672. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:12:16,062][104569] Avg episode reward: [(0, '9082.822'), (1, '8655.258')] [2023-12-26 22:12:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000938952_240402432.pth... [2023-12-26 22:12:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000937840_240115712.pth [2023-12-26 22:12:16,122][105692] Updated weights for policy 0, policy_version 938818 (0.0009) [2023-12-26 22:12:16,185][105692] Updated weights for policy 0, policy_version 938828 (0.0010) [2023-12-26 22:12:16,210][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000938832_240377856.pth... [2023-12-26 22:12:16,215][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000937712_240091136.pth [2023-12-26 22:12:16,604][105620] Updated weights for policy 1, policy_version 938956 (0.0008) [2023-12-26 22:12:16,677][105620] Updated weights for policy 1, policy_version 938966 (0.0006) [2023-12-26 22:12:16,748][105620] Updated weights for policy 1, policy_version 938976 (0.0007) [2023-12-26 22:12:16,982][105692] Updated weights for policy 0, policy_version 938838 (0.0008) [2023-12-26 22:12:17,042][105692] Updated weights for policy 0, policy_version 938848 (0.0005) [2023-12-26 22:12:17,105][105692] Updated weights for policy 0, policy_version 938858 (0.0005) [2023-12-26 22:12:17,409][105620] Updated weights for policy 1, policy_version 938986 (0.0009) [2023-12-26 22:12:17,474][105620] Updated weights for policy 1, policy_version 938996 (0.0008) [2023-12-26 22:12:17,533][105620] Updated weights for policy 1, policy_version 939006 (0.0007) [2023-12-26 22:12:17,586][105620] Updated weights for policy 1, policy_version 939016 (0.0008) [2023-12-26 22:12:17,795][105692] Updated weights for policy 0, policy_version 938868 (0.0006) [2023-12-26 22:12:17,843][105692] Updated weights for policy 0, policy_version 938878 (0.0005) [2023-12-26 22:12:17,897][105692] Updated weights for policy 0, policy_version 938888 (0.0005) [2023-12-26 22:12:18,271][105620] Updated weights for policy 1, policy_version 939026 (0.0005) [2023-12-26 22:12:18,329][105620] Updated weights for policy 1, policy_version 939036 (0.0006) [2023-12-26 22:12:18,397][105620] Updated weights for policy 1, policy_version 939047 (0.0006) [2023-12-26 22:12:18,507][105692] Updated weights for policy 0, policy_version 938898 (0.0005) [2023-12-26 22:12:18,572][105692] Updated weights for policy 0, policy_version 938908 (0.0005) [2023-12-26 22:12:18,639][105692] Updated weights for policy 0, policy_version 938918 (0.0006) [2023-12-26 22:12:18,696][105692] Updated weights for policy 0, policy_version 938928 (0.0008) [2023-12-26 22:12:19,072][105620] Updated weights for policy 1, policy_version 939057 (0.0010) [2023-12-26 22:12:19,128][105620] Updated weights for policy 1, policy_version 939067 (0.0010) [2023-12-26 22:12:19,200][105620] Updated weights for policy 1, policy_version 939077 (0.0010) [2023-12-26 22:12:19,240][105692] Updated weights for policy 0, policy_version 938938 (0.0007) [2023-12-26 22:12:19,302][105692] Updated weights for policy 0, policy_version 938948 (0.0009) [2023-12-26 22:12:19,371][105692] Updated weights for policy 0, policy_version 938958 (0.0011) [2023-12-26 22:12:19,946][105620] Updated weights for policy 1, policy_version 939087 (0.0011) [2023-12-26 22:12:20,010][105620] Updated weights for policy 1, policy_version 939097 (0.0009) [2023-12-26 22:12:20,073][105620] Updated weights for policy 1, policy_version 939107 (0.0009) [2023-12-26 22:12:20,138][105692] Updated weights for policy 0, policy_version 938968 (0.0009) [2023-12-26 22:12:20,197][105692] Updated weights for policy 0, policy_version 938978 (0.0011) [2023-12-26 22:12:20,254][105692] Updated weights for policy 0, policy_version 938988 (0.0010) [2023-12-26 22:12:20,834][105620] Updated weights for policy 1, policy_version 939117 (0.0009) [2023-12-26 22:12:20,891][105620] Updated weights for policy 1, policy_version 939127 (0.0011) [2023-12-26 22:12:20,956][105620] Updated weights for policy 1, policy_version 939137 (0.0011) [2023-12-26 22:12:20,978][105692] Updated weights for policy 0, policy_version 938998 (0.0010) [2023-12-26 22:12:21,038][105692] Updated weights for policy 0, policy_version 939008 (0.0011) [2023-12-26 22:12:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19114.7, 300 sec: 19049.9). Total num frames: 480870400. Throughput: 0: 9531.4, 1: 9519.6. Samples: 480860868. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:12:21,062][104569] Avg episode reward: [(0, '9173.477'), (1, '8565.461')] [2023-12-26 22:12:21,102][105692] Updated weights for policy 0, policy_version 939018 (0.0008) [2023-12-26 22:12:21,757][105620] Updated weights for policy 1, policy_version 939147 (0.0009) [2023-12-26 22:12:21,791][105692] Updated weights for policy 0, policy_version 939028 (0.0009) [2023-12-26 22:12:21,826][105620] Updated weights for policy 1, policy_version 939157 (0.0006) [2023-12-26 22:12:21,853][105692] Updated weights for policy 0, policy_version 939038 (0.0010) [2023-12-26 22:12:21,891][105620] Updated weights for policy 1, policy_version 939167 (0.0007) [2023-12-26 22:12:21,914][105692] Updated weights for policy 0, policy_version 939048 (0.0011) [2023-12-26 22:12:22,699][105620] Updated weights for policy 1, policy_version 939177 (0.0006) [2023-12-26 22:12:22,700][105692] Updated weights for policy 0, policy_version 939058 (0.0011) [2023-12-26 22:12:22,760][105620] Updated weights for policy 1, policy_version 939187 (0.0009) [2023-12-26 22:12:22,763][105692] Updated weights for policy 0, policy_version 939068 (0.0010) [2023-12-26 22:12:22,818][105620] Updated weights for policy 1, policy_version 939197 (0.0006) [2023-12-26 22:12:22,824][105692] Updated weights for policy 0, policy_version 939078 (0.0010) [2023-12-26 22:12:22,878][105620] Updated weights for policy 1, policy_version 939207 (0.0007) [2023-12-26 22:12:22,883][105692] Updated weights for policy 0, policy_version 939088 (0.0010) [2023-12-26 22:12:23,620][105692] Updated weights for policy 0, policy_version 939098 (0.0010) [2023-12-26 22:12:23,650][105620] Updated weights for policy 1, policy_version 939217 (0.0006) [2023-12-26 22:12:23,678][105692] Updated weights for policy 0, policy_version 939108 (0.0010) [2023-12-26 22:12:23,700][105620] Updated weights for policy 1, policy_version 939227 (0.0007) [2023-12-26 22:12:23,739][105692] Updated weights for policy 0, policy_version 939118 (0.0010) [2023-12-26 22:12:23,766][105620] Updated weights for policy 1, policy_version 939237 (0.0007) [2023-12-26 22:12:24,311][105620] Updated weights for policy 1, policy_version 939247 (0.0008) [2023-12-26 22:12:24,365][105620] Updated weights for policy 1, policy_version 939257 (0.0009) [2023-12-26 22:12:24,413][105620] Updated weights for policy 1, policy_version 939267 (0.0008) [2023-12-26 22:12:24,577][105692] Updated weights for policy 0, policy_version 939128 (0.0010) [2023-12-26 22:12:24,631][105692] Updated weights for policy 0, policy_version 939138 (0.0010) [2023-12-26 22:12:24,688][105692] Updated weights for policy 0, policy_version 939148 (0.0010) [2023-12-26 22:12:25,014][105620] Updated weights for policy 1, policy_version 939277 (0.0007) [2023-12-26 22:12:25,077][105620] Updated weights for policy 1, policy_version 939287 (0.0006) [2023-12-26 22:12:25,131][105620] Updated weights for policy 1, policy_version 939297 (0.0005) [2023-12-26 22:12:25,604][105692] Updated weights for policy 0, policy_version 939159 (0.0010) [2023-12-26 22:12:25,662][105692] Updated weights for policy 0, policy_version 939169 (0.0010) [2023-12-26 22:12:25,694][105620] Updated weights for policy 1, policy_version 939307 (0.0005) [2023-12-26 22:12:25,711][105692] Updated weights for policy 0, policy_version 939180 (0.0008) [2023-12-26 22:12:25,745][105620] Updated weights for policy 1, policy_version 939317 (0.0005) [2023-12-26 22:12:25,793][105620] Updated weights for policy 1, policy_version 939327 (0.0006) [2023-12-26 22:12:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19114.7, 300 sec: 19049.9). Total num frames: 480968704. Throughput: 0: 9542.2, 1: 9581.7. Samples: 480975200. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:12:26,062][104569] Avg episode reward: [(0, '9173.386'), (1, '8603.613')] [2023-12-26 22:12:26,439][105620] Updated weights for policy 1, policy_version 939337 (0.0010) [2023-12-26 22:12:26,441][105692] Updated weights for policy 0, policy_version 939190 (0.0009) [2023-12-26 22:12:26,492][105620] Updated weights for policy 1, policy_version 939347 (0.0008) [2023-12-26 22:12:26,494][105692] Updated weights for policy 0, policy_version 939200 (0.0007) [2023-12-26 22:12:26,548][105620] Updated weights for policy 1, policy_version 939357 (0.0010) [2023-12-26 22:12:26,554][105692] Updated weights for policy 0, policy_version 939210 (0.0006) [2023-12-26 22:12:26,597][105620] Updated weights for policy 1, policy_version 939367 (0.0010) [2023-12-26 22:12:27,312][105692] Updated weights for policy 0, policy_version 939220 (0.0006) [2023-12-26 22:12:27,316][105620] Updated weights for policy 1, policy_version 939377 (0.0010) [2023-12-26 22:12:27,366][105692] Updated weights for policy 0, policy_version 939230 (0.0006) [2023-12-26 22:12:27,375][105620] Updated weights for policy 1, policy_version 939387 (0.0009) [2023-12-26 22:12:27,408][105692] Updated weights for policy 0, policy_version 939240 (0.0008) [2023-12-26 22:12:27,433][105620] Updated weights for policy 1, policy_version 939397 (0.0010) [2023-12-26 22:12:28,092][105692] Updated weights for policy 0, policy_version 939250 (0.0006) [2023-12-26 22:12:28,148][105692] Updated weights for policy 0, policy_version 939260 (0.0008) [2023-12-26 22:12:28,151][105620] Updated weights for policy 1, policy_version 939407 (0.0007) [2023-12-26 22:12:28,198][105692] Updated weights for policy 0, policy_version 939270 (0.0007) [2023-12-26 22:12:28,200][105620] Updated weights for policy 1, policy_version 939417 (0.0006) [2023-12-26 22:12:28,249][105620] Updated weights for policy 1, policy_version 939427 (0.0006) [2023-12-26 22:12:28,250][105692] Updated weights for policy 0, policy_version 939280 (0.0006) [2023-12-26 22:12:28,947][105620] Updated weights for policy 1, policy_version 939437 (0.0006) [2023-12-26 22:12:28,996][105620] Updated weights for policy 1, policy_version 939447 (0.0005) [2023-12-26 22:12:29,020][105692] Updated weights for policy 0, policy_version 939290 (0.0009) [2023-12-26 22:12:29,049][105620] Updated weights for policy 1, policy_version 939457 (0.0005) [2023-12-26 22:12:29,077][105692] Updated weights for policy 0, policy_version 939300 (0.0008) [2023-12-26 22:12:29,129][105692] Updated weights for policy 0, policy_version 939310 (0.0010) [2023-12-26 22:12:29,640][105620] Updated weights for policy 1, policy_version 939467 (0.0006) [2023-12-26 22:12:29,694][105620] Updated weights for policy 1, policy_version 939477 (0.0008) [2023-12-26 22:12:29,749][105620] Updated weights for policy 1, policy_version 939487 (0.0010) [2023-12-26 22:12:29,974][105692] Updated weights for policy 0, policy_version 939320 (0.0010) [2023-12-26 22:12:30,037][105692] Updated weights for policy 0, policy_version 939330 (0.0009) [2023-12-26 22:12:30,087][105692] Updated weights for policy 0, policy_version 939340 (0.0009) [2023-12-26 22:12:30,466][105620] Updated weights for policy 1, policy_version 939497 (0.0009) [2023-12-26 22:12:30,525][105620] Updated weights for policy 1, policy_version 939507 (0.0010) [2023-12-26 22:12:30,586][105620] Updated weights for policy 1, policy_version 939517 (0.0010) [2023-12-26 22:12:30,645][105620] Updated weights for policy 1, policy_version 939527 (0.0006) [2023-12-26 22:12:30,823][105692] Updated weights for policy 0, policy_version 939350 (0.0010) [2023-12-26 22:12:30,885][105692] Updated weights for policy 0, policy_version 939360 (0.0009) [2023-12-26 22:12:30,940][105692] Updated weights for policy 0, policy_version 939370 (0.0010) [2023-12-26 22:12:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19114.7, 300 sec: 19077.6). Total num frames: 481067008. Throughput: 0: 9585.5, 1: 9662.7. Samples: 481034560. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:12:31,062][104569] Avg episode reward: [(0, '9172.994'), (1, '8290.678')] [2023-12-26 22:12:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000939376_240517120.pth... [2023-12-26 22:12:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000939528_240549888.pth... [2023-12-26 22:12:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000938256_240230400.pth [2023-12-26 22:12:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000938384_240254976.pth [2023-12-26 22:12:31,202][105620] Updated weights for policy 1, policy_version 939537 (0.0009) [2023-12-26 22:12:31,267][105620] Updated weights for policy 1, policy_version 939547 (0.0008) [2023-12-26 22:12:31,333][105620] Updated weights for policy 1, policy_version 939557 (0.0007) [2023-12-26 22:12:31,830][105692] Updated weights for policy 0, policy_version 939381 (0.0009) [2023-12-26 22:12:31,886][105692] Updated weights for policy 0, policy_version 939391 (0.0008) [2023-12-26 22:12:31,942][105692] Updated weights for policy 0, policy_version 939401 (0.0007) [2023-12-26 22:12:31,966][105620] Updated weights for policy 1, policy_version 939567 (0.0010) [2023-12-26 22:12:32,026][105620] Updated weights for policy 1, policy_version 939577 (0.0010) [2023-12-26 22:12:32,096][105620] Updated weights for policy 1, policy_version 939587 (0.0011) [2023-12-26 22:12:32,586][105692] Updated weights for policy 0, policy_version 939411 (0.0007) [2023-12-26 22:12:32,641][105692] Updated weights for policy 0, policy_version 939421 (0.0008) [2023-12-26 22:12:32,687][105692] Updated weights for policy 0, policy_version 939431 (0.0008) [2023-12-26 22:12:32,817][105620] Updated weights for policy 1, policy_version 939597 (0.0011) [2023-12-26 22:12:32,880][105620] Updated weights for policy 1, policy_version 939607 (0.0010) [2023-12-26 22:12:32,943][105620] Updated weights for policy 1, policy_version 939617 (0.0011) [2023-12-26 22:12:33,361][105692] Updated weights for policy 0, policy_version 939441 (0.0008) [2023-12-26 22:12:33,427][105692] Updated weights for policy 0, policy_version 939451 (0.0005) [2023-12-26 22:12:33,486][105692] Updated weights for policy 0, policy_version 939461 (0.0007) [2023-12-26 22:12:33,538][105692] Updated weights for policy 0, policy_version 939471 (0.0008) [2023-12-26 22:12:33,664][105620] Updated weights for policy 1, policy_version 939627 (0.0009) [2023-12-26 22:12:33,724][105620] Updated weights for policy 1, policy_version 939637 (0.0009) [2023-12-26 22:12:33,781][105620] Updated weights for policy 1, policy_version 939647 (0.0007) [2023-12-26 22:12:34,221][105692] Updated weights for policy 0, policy_version 939481 (0.0007) [2023-12-26 22:12:34,284][105692] Updated weights for policy 0, policy_version 939491 (0.0007) [2023-12-26 22:12:34,349][105692] Updated weights for policy 0, policy_version 939501 (0.0006) [2023-12-26 22:12:34,450][105620] Updated weights for policy 1, policy_version 939657 (0.0006) [2023-12-26 22:12:34,519][105620] Updated weights for policy 1, policy_version 939667 (0.0011) [2023-12-26 22:12:34,587][105620] Updated weights for policy 1, policy_version 939677 (0.0006) [2023-12-26 22:12:34,644][105620] Updated weights for policy 1, policy_version 939687 (0.0006) [2023-12-26 22:12:35,056][105692] Updated weights for policy 0, policy_version 939512 (0.0008) [2023-12-26 22:12:35,119][105692] Updated weights for policy 0, policy_version 939522 (0.0010) [2023-12-26 22:12:35,177][105692] Updated weights for policy 0, policy_version 939532 (0.0008) [2023-12-26 22:12:35,182][105620] Updated weights for policy 1, policy_version 939697 (0.0009) [2023-12-26 22:12:35,231][105620] Updated weights for policy 1, policy_version 939707 (0.0010) [2023-12-26 22:12:35,276][105620] Updated weights for policy 1, policy_version 939717 (0.0010) [2023-12-26 22:12:35,827][105692] Updated weights for policy 0, policy_version 939542 (0.0006) [2023-12-26 22:12:35,890][105692] Updated weights for policy 0, policy_version 939552 (0.0005) [2023-12-26 22:12:35,946][105692] Updated weights for policy 0, policy_version 939562 (0.0007) [2023-12-26 22:12:36,043][105620] Updated weights for policy 1, policy_version 939727 (0.0010) [2023-12-26 22:12:36,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19251.1, 300 sec: 19077.6). Total num frames: 481165312. Throughput: 0: 9458.1, 1: 9858.4. Samples: 481153824. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:12:36,063][104569] Avg episode reward: [(0, '9262.479'), (1, '8695.489')] [2023-12-26 22:12:36,091][105620] Updated weights for policy 1, policy_version 939737 (0.0010) [2023-12-26 22:12:36,151][105620] Updated weights for policy 1, policy_version 939747 (0.0007) [2023-12-26 22:12:36,657][105692] Updated weights for policy 0, policy_version 939572 (0.0008) [2023-12-26 22:12:36,714][105692] Updated weights for policy 0, policy_version 939582 (0.0008) [2023-12-26 22:12:36,771][105692] Updated weights for policy 0, policy_version 939592 (0.0008) [2023-12-26 22:12:36,928][105620] Updated weights for policy 1, policy_version 939757 (0.0006) [2023-12-26 22:12:36,989][105620] Updated weights for policy 1, policy_version 939767 (0.0009) [2023-12-26 22:12:37,041][105620] Updated weights for policy 1, policy_version 939777 (0.0009) [2023-12-26 22:12:37,636][105692] Updated weights for policy 0, policy_version 939602 (0.0008) [2023-12-26 22:12:37,651][105620] Updated weights for policy 1, policy_version 939787 (0.0007) [2023-12-26 22:12:37,694][105692] Updated weights for policy 0, policy_version 939612 (0.0008) [2023-12-26 22:12:37,712][105620] Updated weights for policy 1, policy_version 939797 (0.0008) [2023-12-26 22:12:37,748][105692] Updated weights for policy 0, policy_version 939622 (0.0006) [2023-12-26 22:12:37,774][105620] Updated weights for policy 1, policy_version 939807 (0.0007) [2023-12-26 22:12:37,808][105692] Updated weights for policy 0, policy_version 939632 (0.0007) [2023-12-26 22:12:38,436][105620] Updated weights for policy 1, policy_version 939817 (0.0006) [2023-12-26 22:12:38,494][105620] Updated weights for policy 1, policy_version 939827 (0.0006) [2023-12-26 22:12:38,551][105620] Updated weights for policy 1, policy_version 939837 (0.0008) [2023-12-26 22:12:38,607][105620] Updated weights for policy 1, policy_version 939847 (0.0005) [2023-12-26 22:12:38,636][105692] Updated weights for policy 0, policy_version 939642 (0.0010) [2023-12-26 22:12:38,692][105692] Updated weights for policy 0, policy_version 939652 (0.0010) [2023-12-26 22:12:38,739][105692] Updated weights for policy 0, policy_version 939662 (0.0011) [2023-12-26 22:12:39,213][105620] Updated weights for policy 1, policy_version 939857 (0.0006) [2023-12-26 22:12:39,276][105620] Updated weights for policy 1, policy_version 939867 (0.0008) [2023-12-26 22:12:39,337][105620] Updated weights for policy 1, policy_version 939878 (0.0011) [2023-12-26 22:12:39,545][105692] Updated weights for policy 0, policy_version 939672 (0.0009) [2023-12-26 22:12:39,593][105692] Updated weights for policy 0, policy_version 939682 (0.0009) [2023-12-26 22:12:39,644][105692] Updated weights for policy 0, policy_version 939692 (0.0008) [2023-12-26 22:12:40,057][105620] Updated weights for policy 1, policy_version 939888 (0.0008) [2023-12-26 22:12:40,109][105620] Updated weights for policy 1, policy_version 939898 (0.0009) [2023-12-26 22:12:40,156][105620] Updated weights for policy 1, policy_version 939908 (0.0009) [2023-12-26 22:12:40,484][105692] Updated weights for policy 0, policy_version 939702 (0.0009) [2023-12-26 22:12:40,540][105692] Updated weights for policy 0, policy_version 939712 (0.0009) [2023-12-26 22:12:40,591][105692] Updated weights for policy 0, policy_version 939722 (0.0009) [2023-12-26 22:12:40,958][105620] Updated weights for policy 1, policy_version 939918 (0.0009) [2023-12-26 22:12:41,010][105620] Updated weights for policy 1, policy_version 939928 (0.0009) [2023-12-26 22:12:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19077.7). Total num frames: 481255424. Throughput: 0: 9407.5, 1: 9945.5. Samples: 481268172. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:12:41,062][104569] Avg episode reward: [(0, '9261.481'), (1, '8283.678')] [2023-12-26 22:12:41,077][105620] Updated weights for policy 1, policy_version 939938 (0.0009) [2023-12-26 22:12:41,396][105692] Updated weights for policy 0, policy_version 939732 (0.0009) [2023-12-26 22:12:41,458][105692] Updated weights for policy 0, policy_version 939742 (0.0009) [2023-12-26 22:12:41,509][105692] Updated weights for policy 0, policy_version 939752 (0.0008) [2023-12-26 22:12:41,902][105620] Updated weights for policy 1, policy_version 939948 (0.0009) [2023-12-26 22:12:41,957][105620] Updated weights for policy 1, policy_version 939958 (0.0009) [2023-12-26 22:12:42,021][105620] Updated weights for policy 1, policy_version 939968 (0.0009) [2023-12-26 22:12:42,314][105692] Updated weights for policy 0, policy_version 939762 (0.0009) [2023-12-26 22:12:42,382][105692] Updated weights for policy 0, policy_version 939772 (0.0008) [2023-12-26 22:12:42,453][105692] Updated weights for policy 0, policy_version 939782 (0.0008) [2023-12-26 22:12:42,512][105692] Updated weights for policy 0, policy_version 939792 (0.0009) [2023-12-26 22:12:42,833][105620] Updated weights for policy 1, policy_version 939978 (0.0009) [2023-12-26 22:12:42,895][105620] Updated weights for policy 1, policy_version 939988 (0.0008) [2023-12-26 22:12:42,963][105620] Updated weights for policy 1, policy_version 939998 (0.0009) [2023-12-26 22:12:43,025][105620] Updated weights for policy 1, policy_version 940008 (0.0009) [2023-12-26 22:12:43,257][105692] Updated weights for policy 0, policy_version 939802 (0.0009) [2023-12-26 22:12:43,307][105692] Updated weights for policy 0, policy_version 939812 (0.0009) [2023-12-26 22:12:43,368][105692] Updated weights for policy 0, policy_version 939822 (0.0009) [2023-12-26 22:12:43,743][105620] Updated weights for policy 1, policy_version 940018 (0.0008) [2023-12-26 22:12:43,790][105620] Updated weights for policy 1, policy_version 940028 (0.0007) [2023-12-26 22:12:43,845][105620] Updated weights for policy 1, policy_version 940038 (0.0008) [2023-12-26 22:12:44,095][105692] Updated weights for policy 0, policy_version 939832 (0.0007) [2023-12-26 22:12:44,153][105692] Updated weights for policy 0, policy_version 939842 (0.0006) [2023-12-26 22:12:44,211][105692] Updated weights for policy 0, policy_version 939852 (0.0010) [2023-12-26 22:12:44,701][105620] Updated weights for policy 1, policy_version 940048 (0.0009) [2023-12-26 22:12:44,758][105620] Updated weights for policy 1, policy_version 940058 (0.0009) [2023-12-26 22:12:44,824][105620] Updated weights for policy 1, policy_version 940068 (0.0007) [2023-12-26 22:12:44,833][105692] Updated weights for policy 0, policy_version 939862 (0.0011) [2023-12-26 22:12:44,893][105692] Updated weights for policy 0, policy_version 939872 (0.0007) [2023-12-26 22:12:44,957][105692] Updated weights for policy 0, policy_version 939882 (0.0007) [2023-12-26 22:12:45,640][105620] Updated weights for policy 1, policy_version 940078 (0.0006) [2023-12-26 22:12:45,645][105692] Updated weights for policy 0, policy_version 939892 (0.0007) [2023-12-26 22:12:45,696][105620] Updated weights for policy 1, policy_version 940088 (0.0005) [2023-12-26 22:12:45,703][105692] Updated weights for policy 0, policy_version 939902 (0.0009) [2023-12-26 22:12:45,750][105620] Updated weights for policy 1, policy_version 940098 (0.0006) [2023-12-26 22:12:45,760][105692] Updated weights for policy 0, policy_version 939912 (0.0007) [2023-12-26 22:12:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19077.6). Total num frames: 481353728. Throughput: 0: 9367.1, 1: 9918.0. Samples: 481321484. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:12:46,063][104569] Avg episode reward: [(0, '9352.255'), (1, '8212.071')] [2023-12-26 22:12:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000939920_240656384.pth... [2023-12-26 22:12:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000940104_240697344.pth... [2023-12-26 22:12:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000938832_240377856.pth [2023-12-26 22:12:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000938952_240402432.pth [2023-12-26 22:12:46,377][105620] Updated weights for policy 1, policy_version 940108 (0.0006) [2023-12-26 22:12:46,443][105620] Updated weights for policy 1, policy_version 940118 (0.0005) [2023-12-26 22:12:46,500][105620] Updated weights for policy 1, policy_version 940128 (0.0005) [2023-12-26 22:12:46,520][105692] Updated weights for policy 0, policy_version 939922 (0.0008) [2023-12-26 22:12:46,575][105692] Updated weights for policy 0, policy_version 939932 (0.0010) [2023-12-26 22:12:46,627][105692] Updated weights for policy 0, policy_version 939942 (0.0010) [2023-12-26 22:12:46,675][105692] Updated weights for policy 0, policy_version 939952 (0.0010) [2023-12-26 22:12:47,007][105620] Updated weights for policy 1, policy_version 940138 (0.0005) [2023-12-26 22:12:47,065][105620] Updated weights for policy 1, policy_version 940148 (0.0005) [2023-12-26 22:12:47,121][105620] Updated weights for policy 1, policy_version 940158 (0.0006) [2023-12-26 22:12:47,178][105620] Updated weights for policy 1, policy_version 940168 (0.0009) [2023-12-26 22:12:47,251][105692] Updated weights for policy 0, policy_version 939962 (0.0005) [2023-12-26 22:12:47,312][105692] Updated weights for policy 0, policy_version 939972 (0.0005) [2023-12-26 22:12:47,374][105692] Updated weights for policy 0, policy_version 939982 (0.0005) [2023-12-26 22:12:47,960][105620] Updated weights for policy 1, policy_version 940178 (0.0007) [2023-12-26 22:12:47,961][105692] Updated weights for policy 0, policy_version 939992 (0.0009) [2023-12-26 22:12:48,009][105620] Updated weights for policy 1, policy_version 940188 (0.0006) [2023-12-26 22:12:48,019][105692] Updated weights for policy 0, policy_version 940002 (0.0007) [2023-12-26 22:12:48,057][105620] Updated weights for policy 1, policy_version 940198 (0.0006) [2023-12-26 22:12:48,076][105692] Updated weights for policy 0, policy_version 940012 (0.0008) [2023-12-26 22:12:48,809][105620] Updated weights for policy 1, policy_version 940208 (0.0008) [2023-12-26 22:12:48,864][105620] Updated weights for policy 1, policy_version 940218 (0.0008) [2023-12-26 22:12:48,868][105692] Updated weights for policy 0, policy_version 940022 (0.0007) [2023-12-26 22:12:48,928][105620] Updated weights for policy 1, policy_version 940228 (0.0008) [2023-12-26 22:12:48,931][105692] Updated weights for policy 0, policy_version 940032 (0.0006) [2023-12-26 22:12:48,996][105692] Updated weights for policy 0, policy_version 940042 (0.0009) [2023-12-26 22:12:49,711][105620] Updated weights for policy 1, policy_version 940238 (0.0007) [2023-12-26 22:12:49,712][105692] Updated weights for policy 0, policy_version 940052 (0.0009) [2023-12-26 22:12:49,776][105620] Updated weights for policy 1, policy_version 940248 (0.0009) [2023-12-26 22:12:49,780][105692] Updated weights for policy 0, policy_version 940062 (0.0008) [2023-12-26 22:12:49,843][105620] Updated weights for policy 1, policy_version 940258 (0.0011) [2023-12-26 22:12:49,844][105692] Updated weights for policy 0, policy_version 940072 (0.0011) [2023-12-26 22:12:50,484][105692] Updated weights for policy 0, policy_version 940082 (0.0008) [2023-12-26 22:12:50,542][105620] Updated weights for policy 1, policy_version 940268 (0.0010) [2023-12-26 22:12:50,552][105692] Updated weights for policy 0, policy_version 940092 (0.0006) [2023-12-26 22:12:50,607][105620] Updated weights for policy 1, policy_version 940278 (0.0011) [2023-12-26 22:12:50,617][105692] Updated weights for policy 0, policy_version 940102 (0.0008) [2023-12-26 22:12:50,664][105620] Updated weights for policy 1, policy_version 940288 (0.0010) [2023-12-26 22:12:50,682][105692] Updated weights for policy 0, policy_version 940112 (0.0007) [2023-12-26 22:12:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19077.6). Total num frames: 481452032. Throughput: 0: 9440.1, 1: 9943.6. Samples: 481440712. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:12:51,062][104569] Avg episode reward: [(0, '9080.725'), (1, '8768.683')] [2023-12-26 22:12:51,383][105620] Updated weights for policy 1, policy_version 940298 (0.0011) [2023-12-26 22:12:51,446][105620] Updated weights for policy 1, policy_version 940308 (0.0011) [2023-12-26 22:12:51,461][105692] Updated weights for policy 0, policy_version 940122 (0.0007) [2023-12-26 22:12:51,506][105620] Updated weights for policy 1, policy_version 940318 (0.0011) [2023-12-26 22:12:51,523][105692] Updated weights for policy 0, policy_version 940132 (0.0010) [2023-12-26 22:12:51,567][105620] Updated weights for policy 1, policy_version 940328 (0.0011) [2023-12-26 22:12:51,581][105692] Updated weights for policy 0, policy_version 940142 (0.0010) [2023-12-26 22:12:52,307][105620] Updated weights for policy 1, policy_version 940338 (0.0008) [2023-12-26 22:12:52,338][105692] Updated weights for policy 0, policy_version 940152 (0.0011) [2023-12-26 22:12:52,368][105620] Updated weights for policy 1, policy_version 940348 (0.0008) [2023-12-26 22:12:52,401][105692] Updated weights for policy 0, policy_version 940162 (0.0011) [2023-12-26 22:12:52,421][105620] Updated weights for policy 1, policy_version 940358 (0.0006) [2023-12-26 22:12:52,466][105692] Updated weights for policy 0, policy_version 940172 (0.0009) [2023-12-26 22:12:53,005][105620] Updated weights for policy 1, policy_version 940368 (0.0005) [2023-12-26 22:12:53,063][105620] Updated weights for policy 1, policy_version 940378 (0.0006) [2023-12-26 22:12:53,121][105620] Updated weights for policy 1, policy_version 940388 (0.0010) [2023-12-26 22:12:53,181][105692] Updated weights for policy 0, policy_version 940182 (0.0007) [2023-12-26 22:12:53,248][105692] Updated weights for policy 0, policy_version 940192 (0.0005) [2023-12-26 22:12:53,316][105692] Updated weights for policy 0, policy_version 940202 (0.0011) [2023-12-26 22:12:53,794][105620] Updated weights for policy 1, policy_version 940398 (0.0010) [2023-12-26 22:12:53,841][105620] Updated weights for policy 1, policy_version 940408 (0.0010) [2023-12-26 22:12:53,890][105620] Updated weights for policy 1, policy_version 940418 (0.0009) [2023-12-26 22:12:53,997][105692] Updated weights for policy 0, policy_version 940212 (0.0010) [2023-12-26 22:12:54,053][105692] Updated weights for policy 0, policy_version 940222 (0.0008) [2023-12-26 22:12:54,097][105692] Updated weights for policy 0, policy_version 940232 (0.0008) [2023-12-26 22:12:54,648][105620] Updated weights for policy 1, policy_version 940428 (0.0008) [2023-12-26 22:12:54,705][105620] Updated weights for policy 1, policy_version 940438 (0.0005) [2023-12-26 22:12:54,761][105620] Updated weights for policy 1, policy_version 940448 (0.0005) [2023-12-26 22:12:54,878][105692] Updated weights for policy 0, policy_version 940242 (0.0009) [2023-12-26 22:12:54,929][105692] Updated weights for policy 0, policy_version 940252 (0.0010) [2023-12-26 22:12:54,985][105692] Updated weights for policy 0, policy_version 940262 (0.0011) [2023-12-26 22:12:55,038][105692] Updated weights for policy 0, policy_version 940272 (0.0011) [2023-12-26 22:12:55,316][105620] Updated weights for policy 1, policy_version 940458 (0.0005) [2023-12-26 22:12:55,364][105620] Updated weights for policy 1, policy_version 940468 (0.0005) [2023-12-26 22:12:55,410][105620] Updated weights for policy 1, policy_version 940478 (0.0005) [2023-12-26 22:12:55,460][105620] Updated weights for policy 1, policy_version 940488 (0.0005) [2023-12-26 22:12:55,723][105692] Updated weights for policy 0, policy_version 940282 (0.0005) [2023-12-26 22:12:55,778][105692] Updated weights for policy 0, policy_version 940292 (0.0005) [2023-12-26 22:12:55,863][105692] Updated weights for policy 0, policy_version 940302 (0.0005) [2023-12-26 22:12:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.7, 300 sec: 19105.4). Total num frames: 481550336. Throughput: 0: 9507.1, 1: 9945.8. Samples: 481558620. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:12:56,063][104569] Avg episode reward: [(0, '8643.262'), (1, '8593.026')] [2023-12-26 22:12:56,271][105620] Updated weights for policy 1, policy_version 940498 (0.0008) [2023-12-26 22:12:56,330][105620] Updated weights for policy 1, policy_version 940508 (0.0008) [2023-12-26 22:12:56,388][105620] Updated weights for policy 1, policy_version 940518 (0.0007) [2023-12-26 22:12:56,415][105692] Updated weights for policy 0, policy_version 940312 (0.0009) [2023-12-26 22:12:56,471][105692] Updated weights for policy 0, policy_version 940322 (0.0006) [2023-12-26 22:12:56,538][105692] Updated weights for policy 0, policy_version 940332 (0.0006) [2023-12-26 22:12:57,068][105692] Updated weights for policy 0, policy_version 940342 (0.0005) [2023-12-26 22:12:57,120][105692] Updated weights for policy 0, policy_version 940352 (0.0005) [2023-12-26 22:12:57,147][105620] Updated weights for policy 1, policy_version 940528 (0.0009) [2023-12-26 22:12:57,174][105692] Updated weights for policy 0, policy_version 940362 (0.0005) [2023-12-26 22:12:57,209][105620] Updated weights for policy 1, policy_version 940538 (0.0010) [2023-12-26 22:12:57,253][105620] Updated weights for policy 1, policy_version 940548 (0.0010) [2023-12-26 22:12:57,724][105692] Updated weights for policy 0, policy_version 940372 (0.0006) [2023-12-26 22:12:57,780][105692] Updated weights for policy 0, policy_version 940382 (0.0005) [2023-12-26 22:12:57,840][105692] Updated weights for policy 0, policy_version 940392 (0.0009) [2023-12-26 22:12:58,013][105620] Updated weights for policy 1, policy_version 940558 (0.0010) [2023-12-26 22:12:58,077][105620] Updated weights for policy 1, policy_version 940568 (0.0010) [2023-12-26 22:12:58,141][105620] Updated weights for policy 1, policy_version 940578 (0.0010) [2023-12-26 22:12:58,572][105692] Updated weights for policy 0, policy_version 940402 (0.0010) [2023-12-26 22:12:58,648][105692] Updated weights for policy 0, policy_version 940412 (0.0011) [2023-12-26 22:12:58,710][105692] Updated weights for policy 0, policy_version 940422 (0.0009) [2023-12-26 22:12:58,776][105692] Updated weights for policy 0, policy_version 940432 (0.0011) [2023-12-26 22:12:58,959][105620] Updated weights for policy 1, policy_version 940588 (0.0010) [2023-12-26 22:12:59,033][105620] Updated weights for policy 1, policy_version 940598 (0.0007) [2023-12-26 22:12:59,086][105620] Updated weights for policy 1, policy_version 940608 (0.0009) [2023-12-26 22:12:59,660][105692] Updated weights for policy 0, policy_version 940442 (0.0007) [2023-12-26 22:12:59,726][105692] Updated weights for policy 0, policy_version 940452 (0.0008) [2023-12-26 22:12:59,781][105620] Updated weights for policy 1, policy_version 940618 (0.0007) [2023-12-26 22:12:59,790][105692] Updated weights for policy 0, policy_version 940462 (0.0009) [2023-12-26 22:12:59,852][105620] Updated weights for policy 1, policy_version 940628 (0.0009) [2023-12-26 22:12:59,910][105620] Updated weights for policy 1, policy_version 940638 (0.0010) [2023-12-26 22:12:59,974][105620] Updated weights for policy 1, policy_version 940648 (0.0009) [2023-12-26 22:13:00,423][105692] Updated weights for policy 0, policy_version 940472 (0.0006) [2023-12-26 22:13:00,479][105692] Updated weights for policy 0, policy_version 940482 (0.0005) [2023-12-26 22:13:00,541][105692] Updated weights for policy 0, policy_version 940492 (0.0005) [2023-12-26 22:13:00,711][105620] Updated weights for policy 1, policy_version 940658 (0.0006) [2023-12-26 22:13:00,766][105620] Updated weights for policy 1, policy_version 940668 (0.0005) [2023-12-26 22:13:00,824][105620] Updated weights for policy 1, policy_version 940678 (0.0005) [2023-12-26 22:13:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19105.4). Total num frames: 481648640. Throughput: 0: 9661.6, 1: 9891.5. Samples: 481619564. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:13:01,062][104569] Avg episode reward: [(0, '8645.671'), (1, '8411.343')] [2023-12-26 22:13:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000940680_240844800.pth... [2023-12-26 22:13:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000939528_240549888.pth [2023-12-26 22:13:01,080][105692] Updated weights for policy 0, policy_version 940502 (0.0007) [2023-12-26 22:13:01,142][105692] Updated weights for policy 0, policy_version 940512 (0.0008) [2023-12-26 22:13:01,211][105692] Updated weights for policy 0, policy_version 940522 (0.0008) [2023-12-26 22:13:01,246][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000940528_240812032.pth... [2023-12-26 22:13:01,251][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000939376_240517120.pth [2023-12-26 22:13:01,489][105620] Updated weights for policy 1, policy_version 940688 (0.0009) [2023-12-26 22:13:01,543][105620] Updated weights for policy 1, policy_version 940698 (0.0010) [2023-12-26 22:13:01,598][105620] Updated weights for policy 1, policy_version 940708 (0.0010) [2023-12-26 22:13:01,897][105692] Updated weights for policy 0, policy_version 940532 (0.0008) [2023-12-26 22:13:01,955][105692] Updated weights for policy 0, policy_version 940542 (0.0009) [2023-12-26 22:13:02,015][105692] Updated weights for policy 0, policy_version 940552 (0.0010) [2023-12-26 22:13:02,203][105620] Updated weights for policy 1, policy_version 940718 (0.0007) [2023-12-26 22:13:02,260][105620] Updated weights for policy 1, policy_version 940728 (0.0006) [2023-12-26 22:13:02,318][105620] Updated weights for policy 1, policy_version 940738 (0.0009) [2023-12-26 22:13:02,780][105692] Updated weights for policy 0, policy_version 940562 (0.0009) [2023-12-26 22:13:02,840][105692] Updated weights for policy 0, policy_version 940572 (0.0008) [2023-12-26 22:13:02,891][105692] Updated weights for policy 0, policy_version 940582 (0.0007) [2023-12-26 22:13:02,945][105692] Updated weights for policy 0, policy_version 940592 (0.0005) [2023-12-26 22:13:03,022][105620] Updated weights for policy 1, policy_version 940748 (0.0010) [2023-12-26 22:13:03,073][105620] Updated weights for policy 1, policy_version 940758 (0.0010) [2023-12-26 22:13:03,130][105620] Updated weights for policy 1, policy_version 940768 (0.0010) [2023-12-26 22:13:03,558][105692] Updated weights for policy 0, policy_version 940602 (0.0006) [2023-12-26 22:13:03,618][105692] Updated weights for policy 0, policy_version 940612 (0.0006) [2023-12-26 22:13:03,668][105692] Updated weights for policy 0, policy_version 940622 (0.0008) [2023-12-26 22:13:03,799][105620] Updated weights for policy 1, policy_version 940778 (0.0010) [2023-12-26 22:13:03,867][105620] Updated weights for policy 1, policy_version 940788 (0.0008) [2023-12-26 22:13:03,940][105620] Updated weights for policy 1, policy_version 940798 (0.0009) [2023-12-26 22:13:04,004][105620] Updated weights for policy 1, policy_version 940808 (0.0010) [2023-12-26 22:13:04,445][105692] Updated weights for policy 0, policy_version 940632 (0.0008) [2023-12-26 22:13:04,511][105692] Updated weights for policy 0, policy_version 940642 (0.0008) [2023-12-26 22:13:04,569][105692] Updated weights for policy 0, policy_version 940652 (0.0008) [2023-12-26 22:13:04,775][105620] Updated weights for policy 1, policy_version 940818 (0.0010) [2023-12-26 22:13:04,836][105620] Updated weights for policy 1, policy_version 940828 (0.0010) [2023-12-26 22:13:04,892][105620] Updated weights for policy 1, policy_version 940838 (0.0010) [2023-12-26 22:13:05,307][105692] Updated weights for policy 0, policy_version 940662 (0.0008) [2023-12-26 22:13:05,356][105692] Updated weights for policy 0, policy_version 940672 (0.0008) [2023-12-26 22:13:05,400][105692] Updated weights for policy 0, policy_version 940682 (0.0008) [2023-12-26 22:13:05,608][105620] Updated weights for policy 1, policy_version 940848 (0.0006) [2023-12-26 22:13:05,671][105620] Updated weights for policy 1, policy_version 940858 (0.0005) [2023-12-26 22:13:05,730][105620] Updated weights for policy 1, policy_version 940868 (0.0005) [2023-12-26 22:13:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19133.2). Total num frames: 481746944. Throughput: 0: 9616.5, 1: 9882.1. Samples: 481738308. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:13:06,063][104569] Avg episode reward: [(0, '8988.951'), (1, '8563.122')] [2023-12-26 22:13:06,260][105692] Updated weights for policy 0, policy_version 940692 (0.0008) [2023-12-26 22:13:06,320][105692] Updated weights for policy 0, policy_version 940702 (0.0008) [2023-12-26 22:13:06,324][105620] Updated weights for policy 1, policy_version 940878 (0.0008) [2023-12-26 22:13:06,379][105692] Updated weights for policy 0, policy_version 940712 (0.0008) [2023-12-26 22:13:06,383][105620] Updated weights for policy 1, policy_version 940888 (0.0010) [2023-12-26 22:13:06,443][105620] Updated weights for policy 1, policy_version 940898 (0.0010) [2023-12-26 22:13:07,162][105692] Updated weights for policy 0, policy_version 940722 (0.0009) [2023-12-26 22:13:07,183][105620] Updated weights for policy 1, policy_version 940908 (0.0010) [2023-12-26 22:13:07,227][105692] Updated weights for policy 0, policy_version 940732 (0.0006) [2023-12-26 22:13:07,244][105620] Updated weights for policy 1, policy_version 940918 (0.0011) [2023-12-26 22:13:07,288][105692] Updated weights for policy 0, policy_version 940742 (0.0007) [2023-12-26 22:13:07,302][105620] Updated weights for policy 1, policy_version 940928 (0.0010) [2023-12-26 22:13:07,352][105692] Updated weights for policy 0, policy_version 940752 (0.0007) [2023-12-26 22:13:08,020][105620] Updated weights for policy 1, policy_version 940938 (0.0009) [2023-12-26 22:13:08,079][105620] Updated weights for policy 1, policy_version 940948 (0.0005) [2023-12-26 22:13:08,103][105692] Updated weights for policy 0, policy_version 940762 (0.0008) [2023-12-26 22:13:08,125][105620] Updated weights for policy 1, policy_version 940958 (0.0005) [2023-12-26 22:13:08,161][105692] Updated weights for policy 0, policy_version 940772 (0.0008) [2023-12-26 22:13:08,172][105620] Updated weights for policy 1, policy_version 940968 (0.0006) [2023-12-26 22:13:08,217][105692] Updated weights for policy 0, policy_version 940782 (0.0008) [2023-12-26 22:13:08,856][105620] Updated weights for policy 1, policy_version 940978 (0.0006) [2023-12-26 22:13:08,915][105620] Updated weights for policy 1, policy_version 940988 (0.0010) [2023-12-26 22:13:08,963][105620] Updated weights for policy 1, policy_version 940998 (0.0010) [2023-12-26 22:13:09,037][105692] Updated weights for policy 0, policy_version 940792 (0.0010) [2023-12-26 22:13:09,096][105692] Updated weights for policy 0, policy_version 940802 (0.0010) [2023-12-26 22:13:09,149][105692] Updated weights for policy 0, policy_version 940812 (0.0009) [2023-12-26 22:13:09,668][105620] Updated weights for policy 1, policy_version 941008 (0.0010) [2023-12-26 22:13:09,727][105620] Updated weights for policy 1, policy_version 941018 (0.0006) [2023-12-26 22:13:09,788][105620] Updated weights for policy 1, policy_version 941028 (0.0006) [2023-12-26 22:13:09,971][105692] Updated weights for policy 0, policy_version 940822 (0.0006) [2023-12-26 22:13:10,039][105692] Updated weights for policy 0, policy_version 940832 (0.0008) [2023-12-26 22:13:10,113][105692] Updated weights for policy 0, policy_version 940842 (0.0007) [2023-12-26 22:13:10,417][105620] Updated weights for policy 1, policy_version 941038 (0.0011) [2023-12-26 22:13:10,473][105620] Updated weights for policy 1, policy_version 941048 (0.0010) [2023-12-26 22:13:10,541][105620] Updated weights for policy 1, policy_version 941058 (0.0011) [2023-12-26 22:13:10,890][105692] Updated weights for policy 0, policy_version 940852 (0.0009) [2023-12-26 22:13:10,942][105692] Updated weights for policy 0, policy_version 940862 (0.0007) [2023-12-26 22:13:10,999][105692] Updated weights for policy 0, policy_version 940872 (0.0006) [2023-12-26 22:13:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19160.9). Total num frames: 481845248. Throughput: 0: 9594.4, 1: 9898.8. Samples: 481852396. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:13:11,063][104569] Avg episode reward: [(0, '8897.238'), (1, '8906.590')] [2023-12-26 22:13:11,241][105620] Updated weights for policy 1, policy_version 941068 (0.0011) [2023-12-26 22:13:11,310][105620] Updated weights for policy 1, policy_version 941078 (0.0010) [2023-12-26 22:13:11,369][105620] Updated weights for policy 1, policy_version 941088 (0.0011) [2023-12-26 22:13:11,737][105692] Updated weights for policy 0, policy_version 940882 (0.0010) [2023-12-26 22:13:11,797][105692] Updated weights for policy 0, policy_version 940892 (0.0009) [2023-12-26 22:13:11,859][105692] Updated weights for policy 0, policy_version 940902 (0.0009) [2023-12-26 22:13:11,912][105692] Updated weights for policy 0, policy_version 940912 (0.0008) [2023-12-26 22:13:12,168][105620] Updated weights for policy 1, policy_version 941098 (0.0010) [2023-12-26 22:13:12,230][105620] Updated weights for policy 1, policy_version 941108 (0.0009) [2023-12-26 22:13:12,298][105620] Updated weights for policy 1, policy_version 941118 (0.0007) [2023-12-26 22:13:12,360][105620] Updated weights for policy 1, policy_version 941128 (0.0010) [2023-12-26 22:13:12,720][105692] Updated weights for policy 0, policy_version 940922 (0.0009) [2023-12-26 22:13:12,780][105692] Updated weights for policy 0, policy_version 940932 (0.0008) [2023-12-26 22:13:12,831][105692] Updated weights for policy 0, policy_version 940942 (0.0007) [2023-12-26 22:13:13,046][105620] Updated weights for policy 1, policy_version 941138 (0.0010) [2023-12-26 22:13:13,105][105620] Updated weights for policy 1, policy_version 941148 (0.0010) [2023-12-26 22:13:13,160][105620] Updated weights for policy 1, policy_version 941158 (0.0010) [2023-12-26 22:13:13,406][105692] Updated weights for policy 0, policy_version 940952 (0.0005) [2023-12-26 22:13:13,468][105692] Updated weights for policy 0, policy_version 940962 (0.0006) [2023-12-26 22:13:13,529][105692] Updated weights for policy 0, policy_version 940972 (0.0006) [2023-12-26 22:13:13,785][105620] Updated weights for policy 1, policy_version 941168 (0.0010) [2023-12-26 22:13:13,841][105620] Updated weights for policy 1, policy_version 941178 (0.0010) [2023-12-26 22:13:13,902][105620] Updated weights for policy 1, policy_version 941188 (0.0010) [2023-12-26 22:13:14,157][105692] Updated weights for policy 0, policy_version 940982 (0.0007) [2023-12-26 22:13:14,222][105692] Updated weights for policy 0, policy_version 940992 (0.0010) [2023-12-26 22:13:14,290][105692] Updated weights for policy 0, policy_version 941002 (0.0006) [2023-12-26 22:13:14,556][105620] Updated weights for policy 1, policy_version 941198 (0.0009) [2023-12-26 22:13:14,617][105620] Updated weights for policy 1, policy_version 941208 (0.0006) [2023-12-26 22:13:14,679][105620] Updated weights for policy 1, policy_version 941218 (0.0007) [2023-12-26 22:13:14,994][105692] Updated weights for policy 0, policy_version 941012 (0.0006) [2023-12-26 22:13:15,070][105692] Updated weights for policy 0, policy_version 941022 (0.0010) [2023-12-26 22:13:15,136][105692] Updated weights for policy 0, policy_version 941032 (0.0010) [2023-12-26 22:13:15,324][105620] Updated weights for policy 1, policy_version 941228 (0.0011) [2023-12-26 22:13:15,391][105620] Updated weights for policy 1, policy_version 941238 (0.0009) [2023-12-26 22:13:15,460][105620] Updated weights for policy 1, policy_version 941248 (0.0006) [2023-12-26 22:13:15,851][105692] Updated weights for policy 0, policy_version 941042 (0.0009) [2023-12-26 22:13:15,921][105692] Updated weights for policy 0, policy_version 941052 (0.0005) [2023-12-26 22:13:15,982][105692] Updated weights for policy 0, policy_version 941062 (0.0005) [2023-12-26 22:13:16,041][105620] Updated weights for policy 1, policy_version 941258 (0.0007) [2023-12-26 22:13:16,045][105692] Updated weights for policy 0, policy_version 941072 (0.0006) [2023-12-26 22:13:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19161.0). Total num frames: 481943552. Throughput: 0: 9597.0, 1: 9870.0. Samples: 481910576. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:13:16,062][104569] Avg episode reward: [(0, '9079.745'), (1, '9100.885')] [2023-12-26 22:13:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000941072_240951296.pth... [2023-12-26 22:13:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000939920_240656384.pth [2023-12-26 22:13:16,107][105620] Updated weights for policy 1, policy_version 941268 (0.0010) [2023-12-26 22:13:16,165][105620] Updated weights for policy 1, policy_version 941278 (0.0010) [2023-12-26 22:13:16,219][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000941288_241000448.pth... [2023-12-26 22:13:16,220][105620] Updated weights for policy 1, policy_version 941288 (0.0010) [2023-12-26 22:13:16,222][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000940104_240697344.pth [2023-12-26 22:13:16,666][105692] Updated weights for policy 0, policy_version 941082 (0.0010) [2023-12-26 22:13:16,724][105692] Updated weights for policy 0, policy_version 941092 (0.0010) [2023-12-26 22:13:16,773][105692] Updated weights for policy 0, policy_version 941102 (0.0010) [2023-12-26 22:13:16,954][105620] Updated weights for policy 1, policy_version 941298 (0.0006) [2023-12-26 22:13:17,013][105620] Updated weights for policy 1, policy_version 941308 (0.0006) [2023-12-26 22:13:17,066][105620] Updated weights for policy 1, policy_version 941318 (0.0009) [2023-12-26 22:13:17,340][105692] Updated weights for policy 0, policy_version 941112 (0.0006) [2023-12-26 22:13:17,386][105692] Updated weights for policy 0, policy_version 941122 (0.0005) [2023-12-26 22:13:17,429][105692] Updated weights for policy 0, policy_version 941132 (0.0005) [2023-12-26 22:13:17,930][105620] Updated weights for policy 1, policy_version 941328 (0.0009) [2023-12-26 22:13:17,982][105692] Updated weights for policy 0, policy_version 941142 (0.0006) [2023-12-26 22:13:17,996][105620] Updated weights for policy 1, policy_version 941338 (0.0008) [2023-12-26 22:13:18,036][105692] Updated weights for policy 0, policy_version 941152 (0.0008) [2023-12-26 22:13:18,059][105620] Updated weights for policy 1, policy_version 941348 (0.0006) [2023-12-26 22:13:18,091][105692] Updated weights for policy 0, policy_version 941162 (0.0008) [2023-12-26 22:13:18,772][105620] Updated weights for policy 1, policy_version 941358 (0.0007) [2023-12-26 22:13:18,839][105620] Updated weights for policy 1, policy_version 941368 (0.0006) [2023-12-26 22:13:18,909][105692] Updated weights for policy 0, policy_version 941172 (0.0009) [2023-12-26 22:13:18,909][105620] Updated weights for policy 1, policy_version 941378 (0.0007) [2023-12-26 22:13:18,970][105692] Updated weights for policy 0, policy_version 941182 (0.0010) [2023-12-26 22:13:19,042][105692] Updated weights for policy 0, policy_version 941192 (0.0010) [2023-12-26 22:13:19,496][105620] Updated weights for policy 1, policy_version 941388 (0.0008) [2023-12-26 22:13:19,558][105620] Updated weights for policy 1, policy_version 941398 (0.0009) [2023-12-26 22:13:19,620][105620] Updated weights for policy 1, policy_version 941408 (0.0009) [2023-12-26 22:13:19,874][105692] Updated weights for policy 0, policy_version 941202 (0.0010) [2023-12-26 22:13:19,941][105692] Updated weights for policy 0, policy_version 941212 (0.0008) [2023-12-26 22:13:20,006][105692] Updated weights for policy 0, policy_version 941222 (0.0010) [2023-12-26 22:13:20,073][105692] Updated weights for policy 0, policy_version 941232 (0.0010) [2023-12-26 22:13:20,333][105620] Updated weights for policy 1, policy_version 941418 (0.0008) [2023-12-26 22:13:20,398][105620] Updated weights for policy 1, policy_version 941428 (0.0006) [2023-12-26 22:13:20,467][105620] Updated weights for policy 1, policy_version 941438 (0.0007) [2023-12-26 22:13:20,537][105620] Updated weights for policy 1, policy_version 941448 (0.0007) [2023-12-26 22:13:20,915][105692] Updated weights for policy 0, policy_version 941242 (0.0009) [2023-12-26 22:13:20,979][105692] Updated weights for policy 0, policy_version 941252 (0.0009) [2023-12-26 22:13:21,046][105692] Updated weights for policy 0, policy_version 941262 (0.0007) [2023-12-26 22:13:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19161.0). Total num frames: 482041856. Throughput: 0: 9680.2, 1: 9806.6. Samples: 482030728. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:13:21,062][104569] Avg episode reward: [(0, '9263.257'), (1, '8757.957')] [2023-12-26 22:13:21,246][105620] Updated weights for policy 1, policy_version 941458 (0.0010) [2023-12-26 22:13:21,312][105620] Updated weights for policy 1, policy_version 941468 (0.0011) [2023-12-26 22:13:21,383][105620] Updated weights for policy 1, policy_version 941478 (0.0010) [2023-12-26 22:13:21,904][105692] Updated weights for policy 0, policy_version 941272 (0.0010) [2023-12-26 22:13:21,965][105692] Updated weights for policy 0, policy_version 941282 (0.0008) [2023-12-26 22:13:22,019][105692] Updated weights for policy 0, policy_version 941292 (0.0008) [2023-12-26 22:13:22,120][105620] Updated weights for policy 1, policy_version 941488 (0.0009) [2023-12-26 22:13:22,171][105620] Updated weights for policy 1, policy_version 941498 (0.0008) [2023-12-26 22:13:22,233][105620] Updated weights for policy 1, policy_version 941508 (0.0008) [2023-12-26 22:13:22,789][105692] Updated weights for policy 0, policy_version 941302 (0.0008) [2023-12-26 22:13:22,850][105692] Updated weights for policy 0, policy_version 941312 (0.0009) [2023-12-26 22:13:22,899][105692] Updated weights for policy 0, policy_version 941322 (0.0010) [2023-12-26 22:13:22,950][105620] Updated weights for policy 1, policy_version 941518 (0.0007) [2023-12-26 22:13:23,005][105620] Updated weights for policy 1, policy_version 941528 (0.0005) [2023-12-26 22:13:23,068][105620] Updated weights for policy 1, policy_version 941538 (0.0006) [2023-12-26 22:13:23,604][105692] Updated weights for policy 0, policy_version 941332 (0.0007) [2023-12-26 22:13:23,658][105692] Updated weights for policy 0, policy_version 941342 (0.0008) [2023-12-26 22:13:23,702][105692] Updated weights for policy 0, policy_version 941352 (0.0008) [2023-12-26 22:13:23,708][105620] Updated weights for policy 1, policy_version 941548 (0.0009) [2023-12-26 22:13:23,769][105620] Updated weights for policy 1, policy_version 941558 (0.0006) [2023-12-26 22:13:23,833][105620] Updated weights for policy 1, policy_version 941568 (0.0005) [2023-12-26 22:13:24,374][105620] Updated weights for policy 1, policy_version 941578 (0.0005) [2023-12-26 22:13:24,405][105692] Updated weights for policy 0, policy_version 941362 (0.0007) [2023-12-26 22:13:24,443][105620] Updated weights for policy 1, policy_version 941588 (0.0006) [2023-12-26 22:13:24,465][105692] Updated weights for policy 0, policy_version 941372 (0.0007) [2023-12-26 22:13:24,501][105620] Updated weights for policy 1, policy_version 941598 (0.0006) [2023-12-26 22:13:24,530][105692] Updated weights for policy 0, policy_version 941382 (0.0008) [2023-12-26 22:13:24,559][105620] Updated weights for policy 1, policy_version 941608 (0.0005) [2023-12-26 22:13:24,584][105692] Updated weights for policy 0, policy_version 941392 (0.0008) [2023-12-26 22:13:25,171][105620] Updated weights for policy 1, policy_version 941618 (0.0005) [2023-12-26 22:13:25,238][105620] Updated weights for policy 1, policy_version 941628 (0.0009) [2023-12-26 22:13:25,247][105692] Updated weights for policy 0, policy_version 941402 (0.0010) [2023-12-26 22:13:25,304][105620] Updated weights for policy 1, policy_version 941638 (0.0011) [2023-12-26 22:13:25,306][105692] Updated weights for policy 0, policy_version 941412 (0.0010) [2023-12-26 22:13:25,365][105692] Updated weights for policy 0, policy_version 941422 (0.0010) [2023-12-26 22:13:25,949][105692] Updated weights for policy 0, policy_version 941432 (0.0010) [2023-12-26 22:13:25,991][105620] Updated weights for policy 1, policy_version 941648 (0.0010) [2023-12-26 22:13:26,004][105692] Updated weights for policy 0, policy_version 941442 (0.0010) [2023-12-26 22:13:26,049][105620] Updated weights for policy 1, policy_version 941658 (0.0010) [2023-12-26 22:13:26,059][105692] Updated weights for policy 0, policy_version 941452 (0.0010) [2023-12-26 22:13:26,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19387.6, 300 sec: 19160.9). Total num frames: 482131968. Throughput: 0: 9717.1, 1: 9832.8. Samples: 482147924. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:13:26,063][104569] Avg episode reward: [(0, '9172.760'), (1, '8924.875')] [2023-12-26 22:13:26,107][105620] Updated weights for policy 1, policy_version 941668 (0.0010) [2023-12-26 22:13:26,743][105620] Updated weights for policy 1, policy_version 941678 (0.0008) [2023-12-26 22:13:26,759][105692] Updated weights for policy 0, policy_version 941462 (0.0009) [2023-12-26 22:13:26,806][105620] Updated weights for policy 1, policy_version 941688 (0.0010) [2023-12-26 22:13:26,816][105692] Updated weights for policy 0, policy_version 941472 (0.0005) [2023-12-26 22:13:26,851][105620] Updated weights for policy 1, policy_version 941698 (0.0010) [2023-12-26 22:13:26,873][105692] Updated weights for policy 0, policy_version 941482 (0.0006) [2023-12-26 22:13:27,454][105620] Updated weights for policy 1, policy_version 941708 (0.0010) [2023-12-26 22:13:27,499][105620] Updated weights for policy 1, policy_version 941718 (0.0009) [2023-12-26 22:13:27,556][105620] Updated weights for policy 1, policy_version 941728 (0.0008) [2023-12-26 22:13:27,558][105692] Updated weights for policy 0, policy_version 941492 (0.0009) [2023-12-26 22:13:27,605][105692] Updated weights for policy 0, policy_version 941502 (0.0007) [2023-12-26 22:13:27,651][105692] Updated weights for policy 0, policy_version 941512 (0.0008) [2023-12-26 22:13:28,259][105620] Updated weights for policy 1, policy_version 941738 (0.0009) [2023-12-26 22:13:28,317][105620] Updated weights for policy 1, policy_version 941748 (0.0010) [2023-12-26 22:13:28,380][105620] Updated weights for policy 1, policy_version 941758 (0.0009) [2023-12-26 22:13:28,399][105692] Updated weights for policy 0, policy_version 941523 (0.0010) [2023-12-26 22:13:28,446][105620] Updated weights for policy 1, policy_version 941768 (0.0010) [2023-12-26 22:13:28,454][105692] Updated weights for policy 0, policy_version 941533 (0.0007) [2023-12-26 22:13:28,516][105692] Updated weights for policy 0, policy_version 941543 (0.0006) [2023-12-26 22:13:29,155][105620] Updated weights for policy 1, policy_version 941778 (0.0005) [2023-12-26 22:13:29,216][105620] Updated weights for policy 1, policy_version 941788 (0.0007) [2023-12-26 22:13:29,227][105692] Updated weights for policy 0, policy_version 941553 (0.0008) [2023-12-26 22:13:29,277][105620] Updated weights for policy 1, policy_version 941798 (0.0009) [2023-12-26 22:13:29,287][105692] Updated weights for policy 0, policy_version 941563 (0.0008) [2023-12-26 22:13:29,358][105692] Updated weights for policy 0, policy_version 941573 (0.0009) [2023-12-26 22:13:29,427][105692] Updated weights for policy 0, policy_version 941583 (0.0006) [2023-12-26 22:13:29,999][105620] Updated weights for policy 1, policy_version 941808 (0.0010) [2023-12-26 22:13:30,061][105620] Updated weights for policy 1, policy_version 941818 (0.0010) [2023-12-26 22:13:30,080][105692] Updated weights for policy 0, policy_version 941593 (0.0009) [2023-12-26 22:13:30,123][105620] Updated weights for policy 1, policy_version 941828 (0.0010) [2023-12-26 22:13:30,137][105692] Updated weights for policy 0, policy_version 941603 (0.0007) [2023-12-26 22:13:30,191][105692] Updated weights for policy 0, policy_version 941613 (0.0008) [2023-12-26 22:13:30,834][105620] Updated weights for policy 1, policy_version 941838 (0.0010) [2023-12-26 22:13:30,898][105620] Updated weights for policy 1, policy_version 941848 (0.0010) [2023-12-26 22:13:30,921][105692] Updated weights for policy 0, policy_version 941623 (0.0006) [2023-12-26 22:13:30,946][105620] Updated weights for policy 1, policy_version 941858 (0.0006) [2023-12-26 22:13:30,989][105692] Updated weights for policy 0, policy_version 941633 (0.0005) [2023-12-26 22:13:31,061][105692] Updated weights for policy 0, policy_version 941643 (0.0014) [2023-12-26 22:13:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19188.7). Total num frames: 482238464. Throughput: 0: 9808.0, 1: 9931.2. Samples: 482209740. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:13:31,062][104569] Avg episode reward: [(0, '9084.857'), (1, '8911.171')] [2023-12-26 22:13:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000941864_241147904.pth... [2023-12-26 22:13:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000940680_240844800.pth [2023-12-26 22:13:31,092][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000941648_241098752.pth... [2023-12-26 22:13:31,098][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000940528_240812032.pth [2023-12-26 22:13:31,595][105620] Updated weights for policy 1, policy_version 941868 (0.0006) [2023-12-26 22:13:31,660][105620] Updated weights for policy 1, policy_version 941878 (0.0009) [2023-12-26 22:13:31,729][105620] Updated weights for policy 1, policy_version 941888 (0.0009) [2023-12-26 22:13:31,742][105692] Updated weights for policy 0, policy_version 941653 (0.0009) [2023-12-26 22:13:31,804][105692] Updated weights for policy 0, policy_version 941663 (0.0011) [2023-12-26 22:13:31,869][105692] Updated weights for policy 0, policy_version 941673 (0.0011) [2023-12-26 22:13:32,389][105620] Updated weights for policy 1, policy_version 941898 (0.0008) [2023-12-26 22:13:32,451][105620] Updated weights for policy 1, policy_version 941908 (0.0009) [2023-12-26 22:13:32,511][105620] Updated weights for policy 1, policy_version 941918 (0.0008) [2023-12-26 22:13:32,575][105620] Updated weights for policy 1, policy_version 941928 (0.0008) [2023-12-26 22:13:32,606][105692] Updated weights for policy 0, policy_version 941683 (0.0010) [2023-12-26 22:13:32,671][105692] Updated weights for policy 0, policy_version 941693 (0.0010) [2023-12-26 22:13:32,720][105692] Updated weights for policy 0, policy_version 941703 (0.0007) [2023-12-26 22:13:33,275][105692] Updated weights for policy 0, policy_version 941713 (0.0006) [2023-12-26 22:13:33,307][105620] Updated weights for policy 1, policy_version 941938 (0.0006) [2023-12-26 22:13:33,337][105692] Updated weights for policy 0, policy_version 941723 (0.0010) [2023-12-26 22:13:33,366][105620] Updated weights for policy 1, policy_version 941948 (0.0006) [2023-12-26 22:13:33,401][105692] Updated weights for policy 0, policy_version 941733 (0.0010) [2023-12-26 22:13:33,424][105620] Updated weights for policy 1, policy_version 941958 (0.0010) [2023-12-26 22:13:33,462][105692] Updated weights for policy 0, policy_version 941743 (0.0010) [2023-12-26 22:13:34,115][105692] Updated weights for policy 0, policy_version 941753 (0.0010) [2023-12-26 22:13:34,148][105620] Updated weights for policy 1, policy_version 941968 (0.0007) [2023-12-26 22:13:34,185][105692] Updated weights for policy 0, policy_version 941763 (0.0010) [2023-12-26 22:13:34,213][105620] Updated weights for policy 1, policy_version 941978 (0.0007) [2023-12-26 22:13:34,252][105692] Updated weights for policy 0, policy_version 941773 (0.0010) [2023-12-26 22:13:34,280][105620] Updated weights for policy 1, policy_version 941988 (0.0008) [2023-12-26 22:13:34,959][105620] Updated weights for policy 1, policy_version 941998 (0.0006) [2023-12-26 22:13:35,014][105620] Updated weights for policy 1, policy_version 942008 (0.0005) [2023-12-26 22:13:35,022][105692] Updated weights for policy 0, policy_version 941783 (0.0011) [2023-12-26 22:13:35,073][105620] Updated weights for policy 1, policy_version 942018 (0.0005) [2023-12-26 22:13:35,085][105692] Updated weights for policy 0, policy_version 941793 (0.0011) [2023-12-26 22:13:35,154][105692] Updated weights for policy 0, policy_version 941803 (0.0011) [2023-12-26 22:13:35,644][105620] Updated weights for policy 1, policy_version 942028 (0.0006) [2023-12-26 22:13:35,698][105620] Updated weights for policy 1, policy_version 942038 (0.0008) [2023-12-26 22:13:35,751][105620] Updated weights for policy 1, policy_version 942048 (0.0008) [2023-12-26 22:13:35,867][105692] Updated weights for policy 0, policy_version 941813 (0.0008) [2023-12-26 22:13:35,929][105692] Updated weights for policy 0, policy_version 941823 (0.0008) [2023-12-26 22:13:35,988][105692] Updated weights for policy 0, policy_version 941833 (0.0010) [2023-12-26 22:13:36,062][104569] Fps is (10 sec: 21299.6, 60 sec: 19660.9, 300 sec: 19216.5). Total num frames: 482344960. Throughput: 0: 9786.0, 1: 9963.5. Samples: 482329444. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:13:36,062][104569] Avg episode reward: [(0, '9262.812'), (1, '8820.572')] [2023-12-26 22:13:36,499][105620] Updated weights for policy 1, policy_version 942058 (0.0009) [2023-12-26 22:13:36,569][105620] Updated weights for policy 1, policy_version 942068 (0.0006) [2023-12-26 22:13:36,635][105620] Updated weights for policy 1, policy_version 942078 (0.0008) [2023-12-26 22:13:36,699][105620] Updated weights for policy 1, policy_version 942088 (0.0008) [2023-12-26 22:13:36,701][105692] Updated weights for policy 0, policy_version 941843 (0.0009) [2023-12-26 22:13:36,770][105692] Updated weights for policy 0, policy_version 941853 (0.0011) [2023-12-26 22:13:36,832][105692] Updated weights for policy 0, policy_version 941863 (0.0010) [2023-12-26 22:13:37,417][105620] Updated weights for policy 1, policy_version 942098 (0.0008) [2023-12-26 22:13:37,477][105620] Updated weights for policy 1, policy_version 942108 (0.0008) [2023-12-26 22:13:37,530][105620] Updated weights for policy 1, policy_version 942118 (0.0008) [2023-12-26 22:13:37,565][105692] Updated weights for policy 0, policy_version 941873 (0.0010) [2023-12-26 22:13:37,632][105692] Updated weights for policy 0, policy_version 941883 (0.0009) [2023-12-26 22:13:37,692][105692] Updated weights for policy 0, policy_version 941893 (0.0011) [2023-12-26 22:13:37,760][105692] Updated weights for policy 0, policy_version 941903 (0.0007) [2023-12-26 22:13:38,361][105692] Updated weights for policy 0, policy_version 941913 (0.0008) [2023-12-26 22:13:38,365][105620] Updated weights for policy 1, policy_version 942128 (0.0009) [2023-12-26 22:13:38,427][105692] Updated weights for policy 0, policy_version 941923 (0.0009) [2023-12-26 22:13:38,430][105620] Updated weights for policy 1, policy_version 942138 (0.0007) [2023-12-26 22:13:38,484][105692] Updated weights for policy 0, policy_version 941933 (0.0007) [2023-12-26 22:13:38,499][105620] Updated weights for policy 1, policy_version 942148 (0.0006) [2023-12-26 22:13:39,093][105692] Updated weights for policy 0, policy_version 941943 (0.0007) [2023-12-26 22:13:39,145][105692] Updated weights for policy 0, policy_version 941953 (0.0009) [2023-12-26 22:13:39,198][105692] Updated weights for policy 0, policy_version 941963 (0.0011) [2023-12-26 22:13:39,280][105620] Updated weights for policy 1, policy_version 942158 (0.0008) [2023-12-26 22:13:39,341][105620] Updated weights for policy 1, policy_version 942168 (0.0009) [2023-12-26 22:13:39,414][105620] Updated weights for policy 1, policy_version 942178 (0.0009) [2023-12-26 22:13:39,886][105692] Updated weights for policy 0, policy_version 941973 (0.0010) [2023-12-26 22:13:39,951][105692] Updated weights for policy 0, policy_version 941983 (0.0009) [2023-12-26 22:13:40,018][105692] Updated weights for policy 0, policy_version 941993 (0.0009) [2023-12-26 22:13:40,276][105620] Updated weights for policy 1, policy_version 942188 (0.0009) [2023-12-26 22:13:40,334][105620] Updated weights for policy 1, policy_version 942198 (0.0006) [2023-12-26 22:13:40,401][105620] Updated weights for policy 1, policy_version 942208 (0.0008) [2023-12-26 22:13:40,736][105692] Updated weights for policy 0, policy_version 942003 (0.0009) [2023-12-26 22:13:40,791][105692] Updated weights for policy 0, policy_version 942013 (0.0009) [2023-12-26 22:13:40,846][105692] Updated weights for policy 0, policy_version 942023 (0.0006) [2023-12-26 22:13:41,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.7, 300 sec: 19216.5). Total num frames: 482435072. Throughput: 0: 9827.4, 1: 9853.4. Samples: 482444256. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:13:41,063][104569] Avg episode reward: [(0, '9259.582'), (1, '8828.521')] [2023-12-26 22:13:41,152][105620] Updated weights for policy 1, policy_version 942218 (0.0009) [2023-12-26 22:13:41,214][105620] Updated weights for policy 1, policy_version 942228 (0.0010) [2023-12-26 22:13:41,277][105620] Updated weights for policy 1, policy_version 942238 (0.0009) [2023-12-26 22:13:41,341][105620] Updated weights for policy 1, policy_version 942248 (0.0008) [2023-12-26 22:13:41,617][105692] Updated weights for policy 0, policy_version 942033 (0.0006) [2023-12-26 22:13:41,687][105692] Updated weights for policy 0, policy_version 942043 (0.0009) [2023-12-26 22:13:41,757][105692] Updated weights for policy 0, policy_version 942053 (0.0009) [2023-12-26 22:13:41,823][105692] Updated weights for policy 0, policy_version 942063 (0.0008) [2023-12-26 22:13:42,056][105620] Updated weights for policy 1, policy_version 942258 (0.0009) [2023-12-26 22:13:42,121][105620] Updated weights for policy 1, policy_version 942268 (0.0009) [2023-12-26 22:13:42,181][105620] Updated weights for policy 1, policy_version 942278 (0.0009) [2023-12-26 22:13:42,539][105692] Updated weights for policy 0, policy_version 942073 (0.0007) [2023-12-26 22:13:42,595][105692] Updated weights for policy 0, policy_version 942083 (0.0006) [2023-12-26 22:13:42,654][105692] Updated weights for policy 0, policy_version 942093 (0.0009) [2023-12-26 22:13:42,928][105620] Updated weights for policy 1, policy_version 942288 (0.0006) [2023-12-26 22:13:42,985][105620] Updated weights for policy 1, policy_version 942298 (0.0010) [2023-12-26 22:13:43,036][105620] Updated weights for policy 1, policy_version 942308 (0.0010) [2023-12-26 22:13:43,507][105692] Updated weights for policy 0, policy_version 942103 (0.0008) [2023-12-26 22:13:43,564][105692] Updated weights for policy 0, policy_version 942113 (0.0008) [2023-12-26 22:13:43,619][105620] Updated weights for policy 1, policy_version 942318 (0.0008) [2023-12-26 22:13:43,625][105692] Updated weights for policy 0, policy_version 942123 (0.0007) [2023-12-26 22:13:43,668][105620] Updated weights for policy 1, policy_version 942328 (0.0007) [2023-12-26 22:13:43,719][105620] Updated weights for policy 1, policy_version 942338 (0.0010) [2023-12-26 22:13:44,383][105692] Updated weights for policy 0, policy_version 942133 (0.0008) [2023-12-26 22:13:44,433][105692] Updated weights for policy 0, policy_version 942143 (0.0010) [2023-12-26 22:13:44,478][105692] Updated weights for policy 0, policy_version 942153 (0.0010) [2023-12-26 22:13:44,484][105620] Updated weights for policy 1, policy_version 942348 (0.0010) [2023-12-26 22:13:44,546][105620] Updated weights for policy 1, policy_version 942358 (0.0010) [2023-12-26 22:13:44,601][105620] Updated weights for policy 1, policy_version 942368 (0.0010) [2023-12-26 22:13:45,207][105692] Updated weights for policy 0, policy_version 942163 (0.0011) [2023-12-26 22:13:45,266][105692] Updated weights for policy 0, policy_version 942173 (0.0011) [2023-12-26 22:13:45,329][105692] Updated weights for policy 0, policy_version 942183 (0.0011) [2023-12-26 22:13:45,348][105620] Updated weights for policy 1, policy_version 942378 (0.0010) [2023-12-26 22:13:45,412][105620] Updated weights for policy 1, policy_version 942388 (0.0011) [2023-12-26 22:13:45,482][105620] Updated weights for policy 1, policy_version 942398 (0.0011) [2023-12-26 22:13:45,548][105620] Updated weights for policy 1, policy_version 942408 (0.0011) [2023-12-26 22:13:45,961][105692] Updated weights for policy 0, policy_version 942193 (0.0009) [2023-12-26 22:13:46,020][105692] Updated weights for policy 0, policy_version 942203 (0.0006) [2023-12-26 22:13:46,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19524.4, 300 sec: 19188.7). Total num frames: 482525184. Throughput: 0: 9674.9, 1: 9902.8. Samples: 482500556. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:13:46,062][104569] Avg episode reward: [(0, '8987.445'), (1, '8926.443')] [2023-12-26 22:13:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000942408_241287168.pth... [2023-12-26 22:13:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000941288_241000448.pth [2023-12-26 22:13:46,075][105692] Updated weights for policy 0, policy_version 942213 (0.0008) [2023-12-26 22:13:46,141][105692] Updated weights for policy 0, policy_version 942223 (0.0006) [2023-12-26 22:13:46,143][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000942224_241246208.pth... [2023-12-26 22:13:46,146][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000941072_240951296.pth [2023-12-26 22:13:46,238][105620] Updated weights for policy 1, policy_version 942418 (0.0006) [2023-12-26 22:13:46,299][105620] Updated weights for policy 1, policy_version 942428 (0.0007) [2023-12-26 22:13:46,358][105620] Updated weights for policy 1, policy_version 942438 (0.0010) [2023-12-26 22:13:46,848][105692] Updated weights for policy 0, policy_version 942233 (0.0008) [2023-12-26 22:13:46,899][105692] Updated weights for policy 0, policy_version 942243 (0.0009) [2023-12-26 22:13:46,949][105692] Updated weights for policy 0, policy_version 942253 (0.0009) [2023-12-26 22:13:46,955][105620] Updated weights for policy 1, policy_version 942448 (0.0006) [2023-12-26 22:13:47,001][105620] Updated weights for policy 1, policy_version 942458 (0.0005) [2023-12-26 22:13:47,053][105620] Updated weights for policy 1, policy_version 942468 (0.0006) [2023-12-26 22:13:47,617][105620] Updated weights for policy 1, policy_version 942478 (0.0008) [2023-12-26 22:13:47,664][105620] Updated weights for policy 1, policy_version 942488 (0.0009) [2023-12-26 22:13:47,711][105620] Updated weights for policy 1, policy_version 942498 (0.0009) [2023-12-26 22:13:47,814][105692] Updated weights for policy 0, policy_version 942263 (0.0009) [2023-12-26 22:13:47,866][105692] Updated weights for policy 0, policy_version 942273 (0.0008) [2023-12-26 22:13:47,937][105692] Updated weights for policy 0, policy_version 942283 (0.0005) [2023-12-26 22:13:48,404][105620] Updated weights for policy 1, policy_version 942508 (0.0009) [2023-12-26 22:13:48,467][105620] Updated weights for policy 1, policy_version 942518 (0.0010) [2023-12-26 22:13:48,534][105620] Updated weights for policy 1, policy_version 942528 (0.0011) [2023-12-26 22:13:48,565][105692] Updated weights for policy 0, policy_version 942293 (0.0006) [2023-12-26 22:13:48,623][105692] Updated weights for policy 0, policy_version 942303 (0.0007) [2023-12-26 22:13:48,683][105692] Updated weights for policy 0, policy_version 942313 (0.0009) [2023-12-26 22:13:49,232][105620] Updated weights for policy 1, policy_version 942538 (0.0011) [2023-12-26 22:13:49,297][105620] Updated weights for policy 1, policy_version 942548 (0.0011) [2023-12-26 22:13:49,364][105620] Updated weights for policy 1, policy_version 942558 (0.0011) [2023-12-26 22:13:49,431][105620] Updated weights for policy 1, policy_version 942568 (0.0011) [2023-12-26 22:13:49,473][105692] Updated weights for policy 0, policy_version 942323 (0.0009) [2023-12-26 22:13:49,534][105692] Updated weights for policy 0, policy_version 942333 (0.0006) [2023-12-26 22:13:49,596][105692] Updated weights for policy 0, policy_version 942343 (0.0005) [2023-12-26 22:13:50,189][105620] Updated weights for policy 1, policy_version 942578 (0.0011) [2023-12-26 22:13:50,248][105620] Updated weights for policy 1, policy_version 942588 (0.0011) [2023-12-26 22:13:50,272][105692] Updated weights for policy 0, policy_version 942353 (0.0008) [2023-12-26 22:13:50,320][105620] Updated weights for policy 1, policy_version 942598 (0.0010) [2023-12-26 22:13:50,341][105692] Updated weights for policy 0, policy_version 942363 (0.0010) [2023-12-26 22:13:50,404][105692] Updated weights for policy 0, policy_version 942373 (0.0008) [2023-12-26 22:13:50,463][105692] Updated weights for policy 0, policy_version 942383 (0.0010) [2023-12-26 22:13:51,046][105620] Updated weights for policy 1, policy_version 942608 (0.0010) [2023-12-26 22:13:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19188.7). Total num frames: 482623488. Throughput: 0: 9636.3, 1: 9927.5. Samples: 482618676. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:13:51,063][104569] Avg episode reward: [(0, '8809.069'), (1, '9185.405')] [2023-12-26 22:13:51,114][105620] Updated weights for policy 1, policy_version 942618 (0.0011) [2023-12-26 22:13:51,174][105620] Updated weights for policy 1, policy_version 942628 (0.0011) [2023-12-26 22:13:51,211][105692] Updated weights for policy 0, policy_version 942393 (0.0007) [2023-12-26 22:13:51,278][105692] Updated weights for policy 0, policy_version 942403 (0.0007) [2023-12-26 22:13:51,337][105692] Updated weights for policy 0, policy_version 942413 (0.0008) [2023-12-26 22:13:52,004][105620] Updated weights for policy 1, policy_version 942638 (0.0010) [2023-12-26 22:13:52,039][105692] Updated weights for policy 0, policy_version 942423 (0.0006) [2023-12-26 22:13:52,052][105620] Updated weights for policy 1, policy_version 942648 (0.0009) [2023-12-26 22:13:52,088][105692] Updated weights for policy 0, policy_version 942433 (0.0006) [2023-12-26 22:13:52,107][105620] Updated weights for policy 1, policy_version 942658 (0.0008) [2023-12-26 22:13:52,153][105692] Updated weights for policy 0, policy_version 942443 (0.0009) [2023-12-26 22:13:52,786][105620] Updated weights for policy 1, policy_version 942668 (0.0006) [2023-12-26 22:13:52,848][105620] Updated weights for policy 1, policy_version 942678 (0.0009) [2023-12-26 22:13:52,907][105692] Updated weights for policy 0, policy_version 942453 (0.0008) [2023-12-26 22:13:52,912][105620] Updated weights for policy 1, policy_version 942688 (0.0007) [2023-12-26 22:13:52,962][105692] Updated weights for policy 0, policy_version 942463 (0.0009) [2023-12-26 22:13:53,026][105692] Updated weights for policy 0, policy_version 942473 (0.0009) [2023-12-26 22:13:53,640][105692] Updated weights for policy 0, policy_version 942483 (0.0008) [2023-12-26 22:13:53,703][105692] Updated weights for policy 0, policy_version 942493 (0.0008) [2023-12-26 22:13:53,711][105620] Updated weights for policy 1, policy_version 942698 (0.0007) [2023-12-26 22:13:53,753][105692] Updated weights for policy 0, policy_version 942503 (0.0005) [2023-12-26 22:13:53,768][105620] Updated weights for policy 1, policy_version 942708 (0.0009) [2023-12-26 22:13:53,820][105620] Updated weights for policy 1, policy_version 942718 (0.0010) [2023-12-26 22:13:53,878][105620] Updated weights for policy 1, policy_version 942728 (0.0010) [2023-12-26 22:13:54,396][105692] Updated weights for policy 0, policy_version 942513 (0.0006) [2023-12-26 22:13:54,453][105692] Updated weights for policy 0, policy_version 942523 (0.0009) [2023-12-26 22:13:54,518][105692] Updated weights for policy 0, policy_version 942533 (0.0008) [2023-12-26 22:13:54,534][105620] Updated weights for policy 1, policy_version 942738 (0.0009) [2023-12-26 22:13:54,577][105692] Updated weights for policy 0, policy_version 942543 (0.0008) [2023-12-26 22:13:54,589][105620] Updated weights for policy 1, policy_version 942748 (0.0010) [2023-12-26 22:13:54,647][105620] Updated weights for policy 1, policy_version 942758 (0.0010) [2023-12-26 22:13:55,264][105692] Updated weights for policy 0, policy_version 942553 (0.0007) [2023-12-26 22:13:55,286][105620] Updated weights for policy 1, policy_version 942768 (0.0006) [2023-12-26 22:13:55,310][105692] Updated weights for policy 0, policy_version 942563 (0.0009) [2023-12-26 22:13:55,344][105620] Updated weights for policy 1, policy_version 942778 (0.0010) [2023-12-26 22:13:55,356][105692] Updated weights for policy 0, policy_version 942573 (0.0006) [2023-12-26 22:13:55,405][105620] Updated weights for policy 1, policy_version 942788 (0.0010) [2023-12-26 22:13:55,954][105692] Updated weights for policy 0, policy_version 942583 (0.0006) [2023-12-26 22:13:56,020][105692] Updated weights for policy 0, policy_version 942593 (0.0009) [2023-12-26 22:13:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19188.7). Total num frames: 482721792. Throughput: 0: 9800.6, 1: 9859.7. Samples: 482737108. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-26 22:13:56,063][104569] Avg episode reward: [(0, '8907.296'), (1, '9003.113')] [2023-12-26 22:13:56,082][105692] Updated weights for policy 0, policy_version 942603 (0.0008) [2023-12-26 22:13:56,123][105620] Updated weights for policy 1, policy_version 942798 (0.0010) [2023-12-26 22:13:56,175][105620] Updated weights for policy 1, policy_version 942808 (0.0010) [2023-12-26 22:13:56,220][105620] Updated weights for policy 1, policy_version 942818 (0.0010) [2023-12-26 22:13:56,704][105692] Updated weights for policy 0, policy_version 942613 (0.0007) [2023-12-26 22:13:56,769][105692] Updated weights for policy 0, policy_version 942623 (0.0008) [2023-12-26 22:13:56,831][105692] Updated weights for policy 0, policy_version 942633 (0.0008) [2023-12-26 22:13:56,972][105620] Updated weights for policy 1, policy_version 942828 (0.0010) [2023-12-26 22:13:57,035][105620] Updated weights for policy 1, policy_version 942838 (0.0011) [2023-12-26 22:13:57,099][105620] Updated weights for policy 1, policy_version 942848 (0.0011) [2023-12-26 22:13:57,615][105692] Updated weights for policy 0, policy_version 942643 (0.0008) [2023-12-26 22:13:57,676][105692] Updated weights for policy 0, policy_version 942653 (0.0008) [2023-12-26 22:13:57,739][105692] Updated weights for policy 0, policy_version 942663 (0.0008) [2023-12-26 22:13:57,859][105620] Updated weights for policy 1, policy_version 942858 (0.0010) [2023-12-26 22:13:57,918][105620] Updated weights for policy 1, policy_version 942868 (0.0008) [2023-12-26 22:13:57,972][105620] Updated weights for policy 1, policy_version 942878 (0.0007) [2023-12-26 22:13:58,028][105620] Updated weights for policy 1, policy_version 942888 (0.0007) [2023-12-26 22:13:58,508][105692] Updated weights for policy 0, policy_version 942673 (0.0008) [2023-12-26 22:13:58,575][105692] Updated weights for policy 0, policy_version 942683 (0.0009) [2023-12-26 22:13:58,640][105692] Updated weights for policy 0, policy_version 942693 (0.0008) [2023-12-26 22:13:58,690][105692] Updated weights for policy 0, policy_version 942703 (0.0008) [2023-12-26 22:13:58,797][105620] Updated weights for policy 1, policy_version 942898 (0.0007) [2023-12-26 22:13:58,867][105620] Updated weights for policy 1, policy_version 942908 (0.0007) [2023-12-26 22:13:58,937][105620] Updated weights for policy 1, policy_version 942918 (0.0008) [2023-12-26 22:13:59,558][105692] Updated weights for policy 0, policy_version 942713 (0.0005) [2023-12-26 22:13:59,620][105692] Updated weights for policy 0, policy_version 942723 (0.0006) [2023-12-26 22:13:59,676][105620] Updated weights for policy 1, policy_version 942928 (0.0006) [2023-12-26 22:13:59,679][105692] Updated weights for policy 0, policy_version 942733 (0.0009) [2023-12-26 22:13:59,749][105620] Updated weights for policy 1, policy_version 942938 (0.0006) [2023-12-26 22:13:59,806][105620] Updated weights for policy 1, policy_version 942948 (0.0009) [2023-12-26 22:14:00,281][105692] Updated weights for policy 0, policy_version 942743 (0.0009) [2023-12-26 22:14:00,344][105692] Updated weights for policy 0, policy_version 942753 (0.0009) [2023-12-26 22:14:00,406][105692] Updated weights for policy 0, policy_version 942763 (0.0009) [2023-12-26 22:14:00,553][105620] Updated weights for policy 1, policy_version 942958 (0.0008) [2023-12-26 22:14:00,616][105620] Updated weights for policy 1, policy_version 942968 (0.0008) [2023-12-26 22:14:00,674][105620] Updated weights for policy 1, policy_version 942978 (0.0005) [2023-12-26 22:14:01,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19188.7). Total num frames: 482820096. Throughput: 0: 9798.3, 1: 9821.8. Samples: 482793488. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 22:14:01,063][104569] Avg episode reward: [(0, '9086.173'), (1, '8917.457')] [2023-12-26 22:14:01,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000942768_241385472.pth... [2023-12-26 22:14:01,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000942984_241434624.pth... [2023-12-26 22:14:01,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000941648_241098752.pth [2023-12-26 22:14:01,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000941864_241147904.pth [2023-12-26 22:14:01,271][105692] Updated weights for policy 0, policy_version 942773 (0.0008) [2023-12-26 22:14:01,273][105620] Updated weights for policy 1, policy_version 942988 (0.0007) [2023-12-26 22:14:01,327][105620] Updated weights for policy 1, policy_version 942998 (0.0006) [2023-12-26 22:14:01,334][105692] Updated weights for policy 0, policy_version 942783 (0.0008) [2023-12-26 22:14:01,395][105620] Updated weights for policy 1, policy_version 943008 (0.0008) [2023-12-26 22:14:01,397][105692] Updated weights for policy 0, policy_version 942793 (0.0007) [2023-12-26 22:14:02,157][105692] Updated weights for policy 0, policy_version 942803 (0.0006) [2023-12-26 22:14:02,172][105620] Updated weights for policy 1, policy_version 943018 (0.0007) [2023-12-26 22:14:02,218][105620] Updated weights for policy 1, policy_version 943028 (0.0007) [2023-12-26 22:14:02,223][105692] Updated weights for policy 0, policy_version 942813 (0.0008) [2023-12-26 22:14:02,279][105620] Updated weights for policy 1, policy_version 943038 (0.0007) [2023-12-26 22:14:02,285][105692] Updated weights for policy 0, policy_version 942823 (0.0007) [2023-12-26 22:14:02,339][105620] Updated weights for policy 1, policy_version 943048 (0.0007) [2023-12-26 22:14:03,006][105692] Updated weights for policy 0, policy_version 942833 (0.0006) [2023-12-26 22:14:03,064][105692] Updated weights for policy 0, policy_version 942843 (0.0008) [2023-12-26 22:14:03,113][105692] Updated weights for policy 0, policy_version 942853 (0.0007) [2023-12-26 22:14:03,125][105620] Updated weights for policy 1, policy_version 943058 (0.0010) [2023-12-26 22:14:03,169][105692] Updated weights for policy 0, policy_version 942863 (0.0008) [2023-12-26 22:14:03,199][105620] Updated weights for policy 1, policy_version 943068 (0.0010) [2023-12-26 22:14:03,266][105620] Updated weights for policy 1, policy_version 943078 (0.0010) [2023-12-26 22:14:03,892][105620] Updated weights for policy 1, policy_version 943088 (0.0008) [2023-12-26 22:14:03,917][105692] Updated weights for policy 0, policy_version 942873 (0.0009) [2023-12-26 22:14:03,945][105620] Updated weights for policy 1, policy_version 943098 (0.0006) [2023-12-26 22:14:03,971][105692] Updated weights for policy 0, policy_version 942883 (0.0007) [2023-12-26 22:14:04,004][105620] Updated weights for policy 1, policy_version 943108 (0.0010) [2023-12-26 22:14:04,032][105692] Updated weights for policy 0, policy_version 942893 (0.0007) [2023-12-26 22:14:04,763][105620] Updated weights for policy 1, policy_version 943118 (0.0008) [2023-12-26 22:14:04,814][105620] Updated weights for policy 1, policy_version 943128 (0.0008) [2023-12-26 22:14:04,820][105692] Updated weights for policy 0, policy_version 942903 (0.0007) [2023-12-26 22:14:04,870][105620] Updated weights for policy 1, policy_version 943138 (0.0006) [2023-12-26 22:14:04,881][105692] Updated weights for policy 0, policy_version 942913 (0.0006) [2023-12-26 22:14:04,939][105692] Updated weights for policy 0, policy_version 942923 (0.0007) [2023-12-26 22:14:05,603][105620] Updated weights for policy 1, policy_version 943148 (0.0007) [2023-12-26 22:14:05,662][105620] Updated weights for policy 1, policy_version 943158 (0.0005) [2023-12-26 22:14:05,701][105692] Updated weights for policy 0, policy_version 942933 (0.0007) [2023-12-26 22:14:05,731][105620] Updated weights for policy 1, policy_version 943168 (0.0007) [2023-12-26 22:14:05,772][105692] Updated weights for policy 0, policy_version 942943 (0.0005) [2023-12-26 22:14:05,832][105692] Updated weights for policy 0, policy_version 942953 (0.0009) [2023-12-26 22:14:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19188.7). Total num frames: 482918400. Throughput: 0: 9675.3, 1: 9776.9. Samples: 482906080. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 22:14:06,063][104569] Avg episode reward: [(0, '8901.581'), (1, '9004.347')] [2023-12-26 22:14:06,400][105620] Updated weights for policy 1, policy_version 943178 (0.0010) [2023-12-26 22:14:06,464][105620] Updated weights for policy 1, policy_version 943188 (0.0010) [2023-12-26 22:14:06,466][105692] Updated weights for policy 0, policy_version 942963 (0.0009) [2023-12-26 22:14:06,528][105620] Updated weights for policy 1, policy_version 943198 (0.0011) [2023-12-26 22:14:06,531][105692] Updated weights for policy 0, policy_version 942973 (0.0006) [2023-12-26 22:14:06,595][105620] Updated weights for policy 1, policy_version 943208 (0.0011) [2023-12-26 22:14:06,599][105692] Updated weights for policy 0, policy_version 942983 (0.0006) [2023-12-26 22:14:07,311][105692] Updated weights for policy 0, policy_version 942993 (0.0006) [2023-12-26 22:14:07,340][105620] Updated weights for policy 1, policy_version 943218 (0.0010) [2023-12-26 22:14:07,371][105692] Updated weights for policy 0, policy_version 943003 (0.0006) [2023-12-26 22:14:07,393][105620] Updated weights for policy 1, policy_version 943228 (0.0010) [2023-12-26 22:14:07,420][105692] Updated weights for policy 0, policy_version 943013 (0.0006) [2023-12-26 22:14:07,451][105620] Updated weights for policy 1, policy_version 943238 (0.0007) [2023-12-26 22:14:07,474][105692] Updated weights for policy 0, policy_version 943023 (0.0008) [2023-12-26 22:14:08,196][105692] Updated weights for policy 0, policy_version 943033 (0.0009) [2023-12-26 22:14:08,207][105620] Updated weights for policy 1, policy_version 943248 (0.0010) [2023-12-26 22:14:08,243][105692] Updated weights for policy 0, policy_version 943043 (0.0006) [2023-12-26 22:14:08,267][105620] Updated weights for policy 1, policy_version 943258 (0.0010) [2023-12-26 22:14:08,289][105692] Updated weights for policy 0, policy_version 943053 (0.0007) [2023-12-26 22:14:08,325][105620] Updated weights for policy 1, policy_version 943268 (0.0011) [2023-12-26 22:14:09,053][105692] Updated weights for policy 0, policy_version 943063 (0.0009) [2023-12-26 22:14:09,084][105620] Updated weights for policy 1, policy_version 943278 (0.0008) [2023-12-26 22:14:09,100][105692] Updated weights for policy 0, policy_version 943073 (0.0008) [2023-12-26 22:14:09,135][105620] Updated weights for policy 1, policy_version 943288 (0.0006) [2023-12-26 22:14:09,143][105692] Updated weights for policy 0, policy_version 943083 (0.0006) [2023-12-26 22:14:09,187][105620] Updated weights for policy 1, policy_version 943298 (0.0006) [2023-12-26 22:14:10,008][105692] Updated weights for policy 0, policy_version 943093 (0.0008) [2023-12-26 22:14:10,024][105620] Updated weights for policy 1, policy_version 943308 (0.0008) [2023-12-26 22:14:10,070][105692] Updated weights for policy 0, policy_version 943103 (0.0010) [2023-12-26 22:14:10,088][105620] Updated weights for policy 1, policy_version 943318 (0.0009) [2023-12-26 22:14:10,127][105692] Updated weights for policy 0, policy_version 943113 (0.0006) [2023-12-26 22:14:10,146][105620] Updated weights for policy 1, policy_version 943328 (0.0007) [2023-12-26 22:14:10,859][105692] Updated weights for policy 0, policy_version 943123 (0.0008) [2023-12-26 22:14:10,910][105692] Updated weights for policy 0, policy_version 943133 (0.0009) [2023-12-26 22:14:10,928][105620] Updated weights for policy 1, policy_version 943338 (0.0008) [2023-12-26 22:14:10,957][105692] Updated weights for policy 0, policy_version 943143 (0.0009) [2023-12-26 22:14:10,984][105620] Updated weights for policy 1, policy_version 943348 (0.0007) [2023-12-26 22:14:11,055][105620] Updated weights for policy 1, policy_version 943358 (0.0009) [2023-12-26 22:14:11,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19387.8, 300 sec: 19188.7). Total num frames: 483008512. Throughput: 0: 9698.5, 1: 9657.0. Samples: 483018916. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 22:14:11,062][104569] Avg episode reward: [(0, '8810.160'), (1, '9265.673')] [2023-12-26 22:14:11,114][105620] Updated weights for policy 1, policy_version 943368 (0.0009) [2023-12-26 22:14:11,705][105692] Updated weights for policy 0, policy_version 943153 (0.0006) [2023-12-26 22:14:11,781][105692] Updated weights for policy 0, policy_version 943163 (0.0008) [2023-12-26 22:14:11,835][105692] Updated weights for policy 0, policy_version 943173 (0.0009) [2023-12-26 22:14:11,887][105692] Updated weights for policy 0, policy_version 943183 (0.0009) [2023-12-26 22:14:11,952][105620] Updated weights for policy 1, policy_version 943378 (0.0009) [2023-12-26 22:14:12,004][105620] Updated weights for policy 1, policy_version 943388 (0.0009) [2023-12-26 22:14:12,064][105620] Updated weights for policy 1, policy_version 943398 (0.0009) [2023-12-26 22:14:12,585][105692] Updated weights for policy 0, policy_version 943193 (0.0008) [2023-12-26 22:14:12,642][105692] Updated weights for policy 0, policy_version 943203 (0.0009) [2023-12-26 22:14:12,694][105692] Updated weights for policy 0, policy_version 943213 (0.0009) [2023-12-26 22:14:12,896][105620] Updated weights for policy 1, policy_version 943408 (0.0010) [2023-12-26 22:14:12,959][105620] Updated weights for policy 1, policy_version 943418 (0.0009) [2023-12-26 22:14:13,026][105620] Updated weights for policy 1, policy_version 943428 (0.0009) [2023-12-26 22:14:13,388][105692] Updated weights for policy 0, policy_version 943223 (0.0006) [2023-12-26 22:14:13,435][105692] Updated weights for policy 0, policy_version 943233 (0.0005) [2023-12-26 22:14:13,494][105692] Updated weights for policy 0, policy_version 943243 (0.0005) [2023-12-26 22:14:13,869][105620] Updated weights for policy 1, policy_version 943438 (0.0008) [2023-12-26 22:14:13,934][105620] Updated weights for policy 1, policy_version 943448 (0.0011) [2023-12-26 22:14:13,993][105620] Updated weights for policy 1, policy_version 943458 (0.0010) [2023-12-26 22:14:14,122][105692] Updated weights for policy 0, policy_version 943253 (0.0007) [2023-12-26 22:14:14,177][105585] KL-divergence is very high: 189.4731 [2023-12-26 22:14:14,180][105692] Updated weights for policy 0, policy_version 943263 (0.0009) [2023-12-26 22:14:14,219][105585] KL-divergence is very high: 330.5895 [2023-12-26 22:14:14,236][105692] Updated weights for policy 0, policy_version 943274 (0.0010) [2023-12-26 22:14:14,263][105585] KL-divergence is very high: 333.7749 [2023-12-26 22:14:14,601][105620] Updated weights for policy 1, policy_version 943468 (0.0008) [2023-12-26 22:14:14,654][105620] Updated weights for policy 1, policy_version 943478 (0.0006) [2023-12-26 22:14:14,707][105620] Updated weights for policy 1, policy_version 943488 (0.0006) [2023-12-26 22:14:14,978][105692] Updated weights for policy 0, policy_version 943284 (0.0010) [2023-12-26 22:14:15,038][105692] Updated weights for policy 0, policy_version 943294 (0.0011) [2023-12-26 22:14:15,102][105692] Updated weights for policy 0, policy_version 943304 (0.0011) [2023-12-26 22:14:15,376][105620] Updated weights for policy 1, policy_version 943498 (0.0006) [2023-12-26 22:14:15,439][105620] Updated weights for policy 1, policy_version 943508 (0.0011) [2023-12-26 22:14:15,495][105620] Updated weights for policy 1, policy_version 943518 (0.0007) [2023-12-26 22:14:15,551][105620] Updated weights for policy 1, policy_version 943528 (0.0005) [2023-12-26 22:14:15,852][105692] Updated weights for policy 0, policy_version 943314 (0.0011) [2023-12-26 22:14:15,905][105692] Updated weights for policy 0, policy_version 943324 (0.0010) [2023-12-26 22:14:15,961][105692] Updated weights for policy 0, policy_version 943334 (0.0010) [2023-12-26 22:14:16,012][105692] Updated weights for policy 0, policy_version 943344 (0.0010) [2023-12-26 22:14:16,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.7, 300 sec: 19188.7). Total num frames: 483106816. Throughput: 0: 9666.8, 1: 9535.2. Samples: 483073832. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 22:14:16,062][104569] Avg episode reward: [(0, '8990.184'), (1, '9086.725')] [2023-12-26 22:14:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000943344_241532928.pth... [2023-12-26 22:14:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000942224_241246208.pth [2023-12-26 22:14:16,134][105620] Updated weights for policy 1, policy_version 943538 (0.0006) [2023-12-26 22:14:16,188][105620] Updated weights for policy 1, policy_version 943548 (0.0006) [2023-12-26 22:14:16,250][105620] Updated weights for policy 1, policy_version 943558 (0.0006) [2023-12-26 22:14:16,259][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000943560_241582080.pth... [2023-12-26 22:14:16,262][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000942408_241287168.pth [2023-12-26 22:14:16,758][105692] Updated weights for policy 0, policy_version 943354 (0.0010) [2023-12-26 22:14:16,806][105692] Updated weights for policy 0, policy_version 943364 (0.0010) [2023-12-26 22:14:16,852][105692] Updated weights for policy 0, policy_version 943374 (0.0006) [2023-12-26 22:14:16,946][105620] Updated weights for policy 1, policy_version 943568 (0.0009) [2023-12-26 22:14:17,003][105620] Updated weights for policy 1, policy_version 943578 (0.0009) [2023-12-26 22:14:17,064][105620] Updated weights for policy 1, policy_version 943588 (0.0009) [2023-12-26 22:14:17,580][105692] Updated weights for policy 0, policy_version 943384 (0.0005) [2023-12-26 22:14:17,640][105692] Updated weights for policy 0, policy_version 943394 (0.0005) [2023-12-26 22:14:17,696][105692] Updated weights for policy 0, policy_version 943404 (0.0005) [2023-12-26 22:14:17,804][105620] Updated weights for policy 1, policy_version 943598 (0.0007) [2023-12-26 22:14:17,853][105620] Updated weights for policy 1, policy_version 943608 (0.0005) [2023-12-26 22:14:17,905][105620] Updated weights for policy 1, policy_version 943618 (0.0005) [2023-12-26 22:14:18,409][105692] Updated weights for policy 0, policy_version 943414 (0.0007) [2023-12-26 22:14:18,465][105692] Updated weights for policy 0, policy_version 943424 (0.0009) [2023-12-26 22:14:18,526][105692] Updated weights for policy 0, policy_version 943434 (0.0009) [2023-12-26 22:14:18,612][105620] Updated weights for policy 1, policy_version 943628 (0.0008) [2023-12-26 22:14:18,667][105620] Updated weights for policy 1, policy_version 943638 (0.0011) [2023-12-26 22:14:18,702][105586] KL-divergence is very high: 103.1492 [2023-12-26 22:14:18,716][105620] Updated weights for policy 1, policy_version 943648 (0.0009) [2023-12-26 22:14:18,740][105586] KL-divergence is very high: 115.9222 [2023-12-26 22:14:19,194][105692] Updated weights for policy 0, policy_version 943444 (0.0006) [2023-12-26 22:14:19,260][105692] Updated weights for policy 0, policy_version 943454 (0.0007) [2023-12-26 22:14:19,332][105692] Updated weights for policy 0, policy_version 943464 (0.0007) [2023-12-26 22:14:19,612][105620] Updated weights for policy 1, policy_version 943658 (0.0008) [2023-12-26 22:14:19,678][105620] Updated weights for policy 1, policy_version 943668 (0.0009) [2023-12-26 22:14:19,748][105620] Updated weights for policy 1, policy_version 943678 (0.0006) [2023-12-26 22:14:19,818][105620] Updated weights for policy 1, policy_version 943688 (0.0006) [2023-12-26 22:14:20,087][105692] Updated weights for policy 0, policy_version 943474 (0.0008) [2023-12-26 22:14:20,145][105692] Updated weights for policy 0, policy_version 943484 (0.0009) [2023-12-26 22:14:20,208][105692] Updated weights for policy 0, policy_version 943494 (0.0009) [2023-12-26 22:14:20,267][105692] Updated weights for policy 0, policy_version 943504 (0.0009) [2023-12-26 22:14:20,517][105620] Updated weights for policy 1, policy_version 943698 (0.0007) [2023-12-26 22:14:20,583][105620] Updated weights for policy 1, policy_version 943708 (0.0008) [2023-12-26 22:14:20,651][105620] Updated weights for policy 1, policy_version 943718 (0.0008) [2023-12-26 22:14:21,055][105692] Updated weights for policy 0, policy_version 943514 (0.0009) [2023-12-26 22:14:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19188.7). Total num frames: 483196928. Throughput: 0: 9628.4, 1: 9533.0. Samples: 483191704. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 22:14:21,063][104569] Avg episode reward: [(0, '9080.428'), (1, '8729.657')] [2023-12-26 22:14:21,124][105692] Updated weights for policy 0, policy_version 943524 (0.0009) [2023-12-26 22:14:21,192][105692] Updated weights for policy 0, policy_version 943534 (0.0008) [2023-12-26 22:14:21,422][105620] Updated weights for policy 1, policy_version 943728 (0.0008) [2023-12-26 22:14:21,489][105620] Updated weights for policy 1, policy_version 943738 (0.0009) [2023-12-26 22:14:21,557][105620] Updated weights for policy 1, policy_version 943748 (0.0009) [2023-12-26 22:14:21,933][105692] Updated weights for policy 0, policy_version 943544 (0.0009) [2023-12-26 22:14:21,994][105692] Updated weights for policy 0, policy_version 943554 (0.0007) [2023-12-26 22:14:22,065][105692] Updated weights for policy 0, policy_version 943564 (0.0007) [2023-12-26 22:14:22,342][105620] Updated weights for policy 1, policy_version 943758 (0.0008) [2023-12-26 22:14:22,408][105620] Updated weights for policy 1, policy_version 943768 (0.0009) [2023-12-26 22:14:22,482][105620] Updated weights for policy 1, policy_version 943778 (0.0009) [2023-12-26 22:14:22,775][105692] Updated weights for policy 0, policy_version 943574 (0.0009) [2023-12-26 22:14:22,825][105692] Updated weights for policy 0, policy_version 943584 (0.0008) [2023-12-26 22:14:22,877][105692] Updated weights for policy 0, policy_version 943594 (0.0009) [2023-12-26 22:14:23,234][105620] Updated weights for policy 1, policy_version 943788 (0.0009) [2023-12-26 22:14:23,296][105620] Updated weights for policy 1, policy_version 943798 (0.0009) [2023-12-26 22:14:23,363][105620] Updated weights for policy 1, policy_version 943808 (0.0010) [2023-12-26 22:14:23,615][105692] Updated weights for policy 0, policy_version 943604 (0.0009) [2023-12-26 22:14:23,673][105692] Updated weights for policy 0, policy_version 943614 (0.0009) [2023-12-26 22:14:23,725][105692] Updated weights for policy 0, policy_version 943624 (0.0009) [2023-12-26 22:14:24,162][105620] Updated weights for policy 1, policy_version 943818 (0.0009) [2023-12-26 22:14:24,229][105620] Updated weights for policy 1, policy_version 943828 (0.0005) [2023-12-26 22:14:24,276][105620] Updated weights for policy 1, policy_version 943838 (0.0005) [2023-12-26 22:14:24,331][105620] Updated weights for policy 1, policy_version 943848 (0.0005) [2023-12-26 22:14:24,385][105692] Updated weights for policy 0, policy_version 943634 (0.0009) [2023-12-26 22:14:24,447][105692] Updated weights for policy 0, policy_version 943644 (0.0009) [2023-12-26 22:14:24,511][105692] Updated weights for policy 0, policy_version 943654 (0.0006) [2023-12-26 22:14:24,586][105692] Updated weights for policy 0, policy_version 943664 (0.0005) [2023-12-26 22:14:24,970][105620] Updated weights for policy 1, policy_version 943858 (0.0009) [2023-12-26 22:14:25,018][105620] Updated weights for policy 1, policy_version 943868 (0.0009) [2023-12-26 22:14:25,065][105620] Updated weights for policy 1, policy_version 943878 (0.0008) [2023-12-26 22:14:25,283][105692] Updated weights for policy 0, policy_version 943674 (0.0009) [2023-12-26 22:14:25,336][105692] Updated weights for policy 0, policy_version 943684 (0.0009) [2023-12-26 22:14:25,394][105692] Updated weights for policy 0, policy_version 943694 (0.0009) [2023-12-26 22:14:25,768][105620] Updated weights for policy 1, policy_version 943888 (0.0006) [2023-12-26 22:14:25,820][105620] Updated weights for policy 1, policy_version 943898 (0.0005) [2023-12-26 22:14:25,844][105586] KL-divergence is very high: 110.3759 [2023-12-26 22:14:25,879][105620] Updated weights for policy 1, policy_version 943908 (0.0006) [2023-12-26 22:14:26,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.8, 300 sec: 19160.9). Total num frames: 483295232. Throughput: 0: 9575.8, 1: 9562.6. Samples: 483305484. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 22:14:26,063][104569] Avg episode reward: [(0, '9349.604'), (1, '8732.562')] [2023-12-26 22:14:26,204][105692] Updated weights for policy 0, policy_version 943704 (0.0006) [2023-12-26 22:14:26,261][105692] Updated weights for policy 0, policy_version 943714 (0.0008) [2023-12-26 22:14:26,326][105692] Updated weights for policy 0, policy_version 943724 (0.0009) [2023-12-26 22:14:26,585][105620] Updated weights for policy 1, policy_version 943918 (0.0009) [2023-12-26 22:14:26,639][105620] Updated weights for policy 1, policy_version 943928 (0.0010) [2023-12-26 22:14:26,688][105620] Updated weights for policy 1, policy_version 943938 (0.0009) [2023-12-26 22:14:26,693][105586] KL-divergence is very high: 106.7211 [2023-12-26 22:14:26,978][105692] Updated weights for policy 0, policy_version 943734 (0.0007) [2023-12-26 22:14:27,033][105692] Updated weights for policy 0, policy_version 943744 (0.0005) [2023-12-26 22:14:27,086][105692] Updated weights for policy 0, policy_version 943754 (0.0005) [2023-12-26 22:14:27,550][105620] Updated weights for policy 1, policy_version 943948 (0.0009) [2023-12-26 22:14:27,604][105620] Updated weights for policy 1, policy_version 943959 (0.0011) [2023-12-26 22:14:27,658][105620] Updated weights for policy 1, policy_version 943969 (0.0010) [2023-12-26 22:14:27,674][105692] Updated weights for policy 0, policy_version 943764 (0.0008) [2023-12-26 22:14:27,727][105692] Updated weights for policy 0, policy_version 943774 (0.0009) [2023-12-26 22:14:27,779][105692] Updated weights for policy 0, policy_version 943784 (0.0009) [2023-12-26 22:14:28,428][105692] Updated weights for policy 0, policy_version 943794 (0.0009) [2023-12-26 22:14:28,484][105692] Updated weights for policy 0, policy_version 943804 (0.0006) [2023-12-26 22:14:28,507][105620] Updated weights for policy 1, policy_version 943979 (0.0008) [2023-12-26 22:14:28,537][105692] Updated weights for policy 0, policy_version 943814 (0.0011) [2023-12-26 22:14:28,560][105620] Updated weights for policy 1, policy_version 943989 (0.0005) [2023-12-26 22:14:28,587][105692] Updated weights for policy 0, policy_version 943824 (0.0010) [2023-12-26 22:14:28,618][105620] Updated weights for policy 1, policy_version 943999 (0.0007) [2023-12-26 22:14:29,352][105692] Updated weights for policy 0, policy_version 943834 (0.0007) [2023-12-26 22:14:29,402][105620] Updated weights for policy 1, policy_version 944009 (0.0007) [2023-12-26 22:14:29,413][105692] Updated weights for policy 0, policy_version 943844 (0.0007) [2023-12-26 22:14:29,468][105620] Updated weights for policy 1, policy_version 944019 (0.0008) [2023-12-26 22:14:29,474][105692] Updated weights for policy 0, policy_version 943854 (0.0007) [2023-12-26 22:14:29,536][105620] Updated weights for policy 1, policy_version 944029 (0.0007) [2023-12-26 22:14:29,601][105620] Updated weights for policy 1, policy_version 944039 (0.0008) [2023-12-26 22:14:30,148][105692] Updated weights for policy 0, policy_version 943864 (0.0005) [2023-12-26 22:14:30,206][105692] Updated weights for policy 0, policy_version 943874 (0.0005) [2023-12-26 22:14:30,255][105620] Updated weights for policy 1, policy_version 944049 (0.0008) [2023-12-26 22:14:30,267][105692] Updated weights for policy 0, policy_version 943884 (0.0006) [2023-12-26 22:14:30,317][105620] Updated weights for policy 1, policy_version 944059 (0.0007) [2023-12-26 22:14:30,375][105620] Updated weights for policy 1, policy_version 944070 (0.0010) [2023-12-26 22:14:30,794][105692] Updated weights for policy 0, policy_version 943894 (0.0006) [2023-12-26 22:14:30,842][105692] Updated weights for policy 0, policy_version 943904 (0.0005) [2023-12-26 22:14:30,891][105692] Updated weights for policy 0, policy_version 943914 (0.0005) [2023-12-26 22:14:31,017][105620] Updated weights for policy 1, policy_version 944080 (0.0006) [2023-12-26 22:14:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.1, 300 sec: 19160.9). Total num frames: 483393536. Throughput: 0: 9666.0, 1: 9496.8. Samples: 483362884. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 22:14:31,063][104569] Avg episode reward: [(0, '9170.801'), (1, '8475.604')] [2023-12-26 22:14:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000943920_241680384.pth... [2023-12-26 22:14:31,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000942768_241385472.pth [2023-12-26 22:14:31,081][105620] Updated weights for policy 1, policy_version 944090 (0.0008) [2023-12-26 22:14:31,139][105620] Updated weights for policy 1, policy_version 944100 (0.0007) [2023-12-26 22:14:31,163][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000944104_241721344.pth... [2023-12-26 22:14:31,167][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000942984_241434624.pth [2023-12-26 22:14:31,674][105692] Updated weights for policy 0, policy_version 943924 (0.0006) [2023-12-26 22:14:31,741][105692] Updated weights for policy 0, policy_version 943934 (0.0009) [2023-12-26 22:14:31,779][105620] Updated weights for policy 1, policy_version 944110 (0.0006) [2023-12-26 22:14:31,803][105692] Updated weights for policy 0, policy_version 943945 (0.0010) [2023-12-26 22:14:31,839][105620] Updated weights for policy 1, policy_version 944120 (0.0005) [2023-12-26 22:14:31,901][105620] Updated weights for policy 1, policy_version 944130 (0.0005) [2023-12-26 22:14:32,466][105620] Updated weights for policy 1, policy_version 944140 (0.0005) [2023-12-26 22:14:32,513][105620] Updated weights for policy 1, policy_version 944150 (0.0006) [2023-12-26 22:14:32,574][105620] Updated weights for policy 1, policy_version 944160 (0.0007) [2023-12-26 22:14:32,787][105692] Updated weights for policy 0, policy_version 943955 (0.0010) [2023-12-26 22:14:32,849][105692] Updated weights for policy 0, policy_version 943965 (0.0007) [2023-12-26 22:14:32,914][105692] Updated weights for policy 0, policy_version 943975 (0.0006) [2023-12-26 22:14:33,272][105620] Updated weights for policy 1, policy_version 944170 (0.0007) [2023-12-26 22:14:33,321][105620] Updated weights for policy 1, policy_version 944180 (0.0008) [2023-12-26 22:14:33,367][105620] Updated weights for policy 1, policy_version 944190 (0.0008) [2023-12-26 22:14:33,413][105620] Updated weights for policy 1, policy_version 944200 (0.0009) [2023-12-26 22:14:33,612][105692] Updated weights for policy 0, policy_version 943985 (0.0007) [2023-12-26 22:14:33,665][105692] Updated weights for policy 0, policy_version 943995 (0.0007) [2023-12-26 22:14:33,715][105692] Updated weights for policy 0, policy_version 944005 (0.0005) [2023-12-26 22:14:33,763][105692] Updated weights for policy 0, policy_version 944015 (0.0006) [2023-12-26 22:14:34,217][105620] Updated weights for policy 1, policy_version 944210 (0.0009) [2023-12-26 22:14:34,276][105620] Updated weights for policy 1, policy_version 944220 (0.0009) [2023-12-26 22:14:34,338][105620] Updated weights for policy 1, policy_version 944230 (0.0009) [2023-12-26 22:14:34,506][105692] Updated weights for policy 0, policy_version 944025 (0.0010) [2023-12-26 22:14:34,567][105692] Updated weights for policy 0, policy_version 944035 (0.0009) [2023-12-26 22:14:34,631][105692] Updated weights for policy 0, policy_version 944045 (0.0008) [2023-12-26 22:14:35,108][105620] Updated weights for policy 1, policy_version 944240 (0.0007) [2023-12-26 22:14:35,164][105620] Updated weights for policy 1, policy_version 944250 (0.0005) [2023-12-26 22:14:35,218][105620] Updated weights for policy 1, policy_version 944260 (0.0005) [2023-12-26 22:14:35,376][105692] Updated weights for policy 0, policy_version 944055 (0.0009) [2023-12-26 22:14:35,435][105692] Updated weights for policy 0, policy_version 944065 (0.0009) [2023-12-26 22:14:35,499][105692] Updated weights for policy 0, policy_version 944075 (0.0008) [2023-12-26 22:14:35,958][105620] Updated weights for policy 1, policy_version 944270 (0.0007) [2023-12-26 22:14:36,023][105620] Updated weights for policy 1, policy_version 944280 (0.0009) [2023-12-26 22:14:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18978.1, 300 sec: 19160.9). Total num frames: 483483648. Throughput: 0: 9643.4, 1: 9488.2. Samples: 483479596. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 22:14:36,063][104569] Avg episode reward: [(0, '8990.013'), (1, '8474.550')] [2023-12-26 22:14:36,081][105620] Updated weights for policy 1, policy_version 944290 (0.0009) [2023-12-26 22:14:36,227][105692] Updated weights for policy 0, policy_version 944085 (0.0009) [2023-12-26 22:14:36,277][105692] Updated weights for policy 0, policy_version 944095 (0.0009) [2023-12-26 22:14:36,326][105692] Updated weights for policy 0, policy_version 944105 (0.0009) [2023-12-26 22:14:36,810][105620] Updated weights for policy 1, policy_version 944300 (0.0010) [2023-12-26 22:14:36,860][105620] Updated weights for policy 1, policy_version 944310 (0.0010) [2023-12-26 22:14:36,912][105620] Updated weights for policy 1, policy_version 944320 (0.0010) [2023-12-26 22:14:37,190][105692] Updated weights for policy 0, policy_version 944115 (0.0008) [2023-12-26 22:14:37,249][105692] Updated weights for policy 0, policy_version 944125 (0.0008) [2023-12-26 22:14:37,298][105692] Updated weights for policy 0, policy_version 944135 (0.0008) [2023-12-26 22:14:37,669][105620] Updated weights for policy 1, policy_version 944330 (0.0010) [2023-12-26 22:14:37,729][105620] Updated weights for policy 1, policy_version 944340 (0.0010) [2023-12-26 22:14:37,786][105620] Updated weights for policy 1, policy_version 944350 (0.0007) [2023-12-26 22:14:37,847][105620] Updated weights for policy 1, policy_version 944360 (0.0005) [2023-12-26 22:14:38,072][105692] Updated weights for policy 0, policy_version 944145 (0.0008) [2023-12-26 22:14:38,131][105692] Updated weights for policy 0, policy_version 944155 (0.0010) [2023-12-26 22:14:38,194][105692] Updated weights for policy 0, policy_version 944165 (0.0007) [2023-12-26 22:14:38,256][105692] Updated weights for policy 0, policy_version 944175 (0.0006) [2023-12-26 22:14:38,560][105620] Updated weights for policy 1, policy_version 944370 (0.0011) [2023-12-26 22:14:38,620][105620] Updated weights for policy 1, policy_version 944380 (0.0011) [2023-12-26 22:14:38,678][105620] Updated weights for policy 1, policy_version 944390 (0.0011) [2023-12-26 22:14:38,980][105692] Updated weights for policy 0, policy_version 944185 (0.0010) [2023-12-26 22:14:39,041][105692] Updated weights for policy 0, policy_version 944195 (0.0009) [2023-12-26 22:14:39,094][105692] Updated weights for policy 0, policy_version 944205 (0.0011) [2023-12-26 22:14:39,431][105620] Updated weights for policy 1, policy_version 944400 (0.0008) [2023-12-26 22:14:39,491][105620] Updated weights for policy 1, policy_version 944410 (0.0010) [2023-12-26 22:14:39,559][105620] Updated weights for policy 1, policy_version 944420 (0.0009) [2023-12-26 22:14:39,827][105692] Updated weights for policy 0, policy_version 944215 (0.0009) [2023-12-26 22:14:39,896][105692] Updated weights for policy 0, policy_version 944225 (0.0008) [2023-12-26 22:14:39,959][105692] Updated weights for policy 0, policy_version 944235 (0.0008) [2023-12-26 22:14:40,342][105620] Updated weights for policy 1, policy_version 944430 (0.0008) [2023-12-26 22:14:40,400][105620] Updated weights for policy 1, policy_version 944440 (0.0009) [2023-12-26 22:14:40,459][105620] Updated weights for policy 1, policy_version 944450 (0.0009) [2023-12-26 22:14:40,725][105692] Updated weights for policy 0, policy_version 944245 (0.0009) [2023-12-26 22:14:40,773][105692] Updated weights for policy 0, policy_version 944255 (0.0009) [2023-12-26 22:14:40,825][105692] Updated weights for policy 0, policy_version 944265 (0.0009) [2023-12-26 22:14:41,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19114.7, 300 sec: 19161.0). Total num frames: 483581952. Throughput: 0: 9521.2, 1: 9458.3. Samples: 483591184. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 22:14:41,062][104569] Avg episode reward: [(0, '8985.270'), (1, '8745.341')] [2023-12-26 22:14:41,212][105620] Updated weights for policy 1, policy_version 944460 (0.0008) [2023-12-26 22:14:41,272][105620] Updated weights for policy 1, policy_version 944470 (0.0007) [2023-12-26 22:14:41,337][105620] Updated weights for policy 1, policy_version 944480 (0.0007) [2023-12-26 22:14:41,345][105586] KL-divergence is very high: 105.1729 [2023-12-26 22:14:41,623][105692] Updated weights for policy 0, policy_version 944275 (0.0009) [2023-12-26 22:14:41,685][105692] Updated weights for policy 0, policy_version 944285 (0.0010) [2023-12-26 22:14:41,750][105692] Updated weights for policy 0, policy_version 944295 (0.0008) [2023-12-26 22:14:42,115][105620] Updated weights for policy 1, policy_version 944490 (0.0007) [2023-12-26 22:14:42,166][105620] Updated weights for policy 1, policy_version 944500 (0.0008) [2023-12-26 22:14:42,237][105620] Updated weights for policy 1, policy_version 944510 (0.0010) [2023-12-26 22:14:42,299][105620] Updated weights for policy 1, policy_version 944520 (0.0009) [2023-12-26 22:14:42,472][105692] Updated weights for policy 0, policy_version 944305 (0.0009) [2023-12-26 22:14:42,538][105692] Updated weights for policy 0, policy_version 944315 (0.0009) [2023-12-26 22:14:42,605][105692] Updated weights for policy 0, policy_version 944325 (0.0009) [2023-12-26 22:14:42,667][105692] Updated weights for policy 0, policy_version 944335 (0.0009) [2023-12-26 22:14:43,007][105620] Updated weights for policy 1, policy_version 944530 (0.0009) [2023-12-26 22:14:43,065][105620] Updated weights for policy 1, policy_version 944540 (0.0007) [2023-12-26 22:14:43,128][105620] Updated weights for policy 1, policy_version 944550 (0.0008) [2023-12-26 22:14:43,445][105692] Updated weights for policy 0, policy_version 944345 (0.0008) [2023-12-26 22:14:43,507][105692] Updated weights for policy 0, policy_version 944355 (0.0005) [2023-12-26 22:14:43,573][105692] Updated weights for policy 0, policy_version 944365 (0.0005) [2023-12-26 22:14:43,719][105620] Updated weights for policy 1, policy_version 944560 (0.0010) [2023-12-26 22:14:43,785][105620] Updated weights for policy 1, policy_version 944570 (0.0010) [2023-12-26 22:14:43,843][105620] Updated weights for policy 1, policy_version 944580 (0.0010) [2023-12-26 22:14:44,107][105692] Updated weights for policy 0, policy_version 944375 (0.0008) [2023-12-26 22:14:44,166][105692] Updated weights for policy 0, policy_version 944385 (0.0006) [2023-12-26 22:14:44,219][105692] Updated weights for policy 0, policy_version 944395 (0.0008) [2023-12-26 22:14:44,578][105620] Updated weights for policy 1, policy_version 944590 (0.0011) [2023-12-26 22:14:44,641][105620] Updated weights for policy 1, policy_version 944600 (0.0010) [2023-12-26 22:14:44,703][105620] Updated weights for policy 1, policy_version 944610 (0.0011) [2023-12-26 22:14:44,910][105692] Updated weights for policy 0, policy_version 944405 (0.0009) [2023-12-26 22:14:44,967][105692] Updated weights for policy 0, policy_version 944415 (0.0010) [2023-12-26 22:14:45,030][105692] Updated weights for policy 0, policy_version 944425 (0.0011) [2023-12-26 22:14:45,377][105620] Updated weights for policy 1, policy_version 944620 (0.0010) [2023-12-26 22:14:45,435][105620] Updated weights for policy 1, policy_version 944630 (0.0008) [2023-12-26 22:14:45,494][105620] Updated weights for policy 1, policy_version 944640 (0.0009) [2023-12-26 22:14:45,744][105692] Updated weights for policy 0, policy_version 944435 (0.0009) [2023-12-26 22:14:45,812][105692] Updated weights for policy 0, policy_version 944445 (0.0005) [2023-12-26 22:14:45,874][105692] Updated weights for policy 0, policy_version 944455 (0.0008) [2023-12-26 22:14:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19251.1, 300 sec: 19160.9). Total num frames: 483680256. Throughput: 0: 9481.4, 1: 9490.8. Samples: 483647236. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 22:14:46,063][104569] Avg episode reward: [(0, '9075.925'), (1, '8748.171')] [2023-12-26 22:14:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000944464_241819648.pth... [2023-12-26 22:14:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000944648_241860608.pth... [2023-12-26 22:14:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000943344_241532928.pth [2023-12-26 22:14:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000943560_241582080.pth [2023-12-26 22:14:46,185][105620] Updated weights for policy 1, policy_version 944650 (0.0009) [2023-12-26 22:14:46,242][105620] Updated weights for policy 1, policy_version 944660 (0.0009) [2023-12-26 22:14:46,288][105620] Updated weights for policy 1, policy_version 944670 (0.0009) [2023-12-26 22:14:46,334][105620] Updated weights for policy 1, policy_version 944680 (0.0008) [2023-12-26 22:14:46,567][105692] Updated weights for policy 0, policy_version 944465 (0.0010) [2023-12-26 22:14:46,625][105692] Updated weights for policy 0, policy_version 944475 (0.0008) [2023-12-26 22:14:46,677][105692] Updated weights for policy 0, policy_version 944485 (0.0009) [2023-12-26 22:14:46,739][105692] Updated weights for policy 0, policy_version 944495 (0.0009) [2023-12-26 22:14:47,026][105620] Updated weights for policy 1, policy_version 944690 (0.0008) [2023-12-26 22:14:47,088][105620] Updated weights for policy 1, policy_version 944700 (0.0009) [2023-12-26 22:14:47,155][105620] Updated weights for policy 1, policy_version 944710 (0.0009) [2023-12-26 22:14:47,402][105692] Updated weights for policy 0, policy_version 944505 (0.0006) [2023-12-26 22:14:47,453][105692] Updated weights for policy 0, policy_version 944515 (0.0007) [2023-12-26 22:14:47,509][105692] Updated weights for policy 0, policy_version 944525 (0.0009) [2023-12-26 22:14:47,889][105620] Updated weights for policy 1, policy_version 944720 (0.0008) [2023-12-26 22:14:47,952][105620] Updated weights for policy 1, policy_version 944730 (0.0008) [2023-12-26 22:14:48,017][105620] Updated weights for policy 1, policy_version 944740 (0.0008) [2023-12-26 22:14:48,217][105692] Updated weights for policy 0, policy_version 944535 (0.0010) [2023-12-26 22:14:48,279][105692] Updated weights for policy 0, policy_version 944545 (0.0010) [2023-12-26 22:14:48,347][105692] Updated weights for policy 0, policy_version 944555 (0.0009) [2023-12-26 22:14:48,610][105620] Updated weights for policy 1, policy_version 944750 (0.0009) [2023-12-26 22:14:48,658][105620] Updated weights for policy 1, policy_version 944760 (0.0010) [2023-12-26 22:14:48,703][105620] Updated weights for policy 1, policy_version 944770 (0.0010) [2023-12-26 22:14:49,048][105692] Updated weights for policy 0, policy_version 944565 (0.0008) [2023-12-26 22:14:49,097][105692] Updated weights for policy 0, policy_version 944575 (0.0010) [2023-12-26 22:14:49,146][105692] Updated weights for policy 0, policy_version 944585 (0.0010) [2023-12-26 22:14:49,483][105620] Updated weights for policy 1, policy_version 944780 (0.0010) [2023-12-26 22:14:49,541][105620] Updated weights for policy 1, policy_version 944790 (0.0010) [2023-12-26 22:14:49,599][105620] Updated weights for policy 1, policy_version 944800 (0.0010) [2023-12-26 22:14:49,936][105692] Updated weights for policy 0, policy_version 944595 (0.0010) [2023-12-26 22:14:50,003][105692] Updated weights for policy 0, policy_version 944605 (0.0009) [2023-12-26 22:14:50,067][105692] Updated weights for policy 0, policy_version 944615 (0.0008) [2023-12-26 22:14:50,376][105620] Updated weights for policy 1, policy_version 944810 (0.0010) [2023-12-26 22:14:50,433][105620] Updated weights for policy 1, policy_version 944820 (0.0011) [2023-12-26 22:14:50,482][105620] Updated weights for policy 1, policy_version 944830 (0.0011) [2023-12-26 22:14:50,528][105620] Updated weights for policy 1, policy_version 944840 (0.0010) [2023-12-26 22:14:50,804][105692] Updated weights for policy 0, policy_version 944625 (0.0008) [2023-12-26 22:14:50,862][105692] Updated weights for policy 0, policy_version 944635 (0.0009) [2023-12-26 22:14:50,925][105692] Updated weights for policy 0, policy_version 944645 (0.0010) [2023-12-26 22:14:50,983][105692] Updated weights for policy 0, policy_version 944655 (0.0010) [2023-12-26 22:14:51,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19188.7). Total num frames: 483778560. Throughput: 0: 9604.3, 1: 9531.5. Samples: 483767188. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 22:14:51,063][104569] Avg episode reward: [(0, '9259.345'), (1, '8837.074')] [2023-12-26 22:14:51,197][105620] Updated weights for policy 1, policy_version 944850 (0.0006) [2023-12-26 22:14:51,261][105620] Updated weights for policy 1, policy_version 944860 (0.0008) [2023-12-26 22:14:51,317][105620] Updated weights for policy 1, policy_version 944870 (0.0010) [2023-12-26 22:14:51,832][105692] Updated weights for policy 0, policy_version 944665 (0.0009) [2023-12-26 22:14:51,879][105692] Updated weights for policy 0, policy_version 944675 (0.0008) [2023-12-26 22:14:51,927][105692] Updated weights for policy 0, policy_version 944685 (0.0009) [2023-12-26 22:14:52,046][105620] Updated weights for policy 1, policy_version 944880 (0.0006) [2023-12-26 22:14:52,103][105620] Updated weights for policy 1, policy_version 944890 (0.0005) [2023-12-26 22:14:52,162][105620] Updated weights for policy 1, policy_version 944900 (0.0008) [2023-12-26 22:14:52,787][105692] Updated weights for policy 0, policy_version 944695 (0.0009) [2023-12-26 22:14:52,830][105620] Updated weights for policy 1, policy_version 944910 (0.0009) [2023-12-26 22:14:52,849][105692] Updated weights for policy 0, policy_version 944705 (0.0006) [2023-12-26 22:14:52,888][105620] Updated weights for policy 1, policy_version 944920 (0.0010) [2023-12-26 22:14:52,910][105692] Updated weights for policy 0, policy_version 944715 (0.0007) [2023-12-26 22:14:52,946][105620] Updated weights for policy 1, policy_version 944930 (0.0007) [2023-12-26 22:14:53,563][105620] Updated weights for policy 1, policy_version 944940 (0.0007) [2023-12-26 22:14:53,616][105620] Updated weights for policy 1, policy_version 944950 (0.0008) [2023-12-26 22:14:53,666][105620] Updated weights for policy 1, policy_version 944960 (0.0008) [2023-12-26 22:14:53,705][105692] Updated weights for policy 0, policy_version 944725 (0.0007) [2023-12-26 22:14:53,763][105692] Updated weights for policy 0, policy_version 944735 (0.0008) [2023-12-26 22:14:53,827][105692] Updated weights for policy 0, policy_version 944745 (0.0005) [2023-12-26 22:14:54,430][105620] Updated weights for policy 1, policy_version 944970 (0.0008) [2023-12-26 22:14:54,449][105692] Updated weights for policy 0, policy_version 944755 (0.0005) [2023-12-26 22:14:54,493][105620] Updated weights for policy 1, policy_version 944980 (0.0011) [2023-12-26 22:14:54,501][105692] Updated weights for policy 0, policy_version 944765 (0.0007) [2023-12-26 22:14:54,549][105620] Updated weights for policy 1, policy_version 944990 (0.0010) [2023-12-26 22:14:54,560][105692] Updated weights for policy 0, policy_version 944775 (0.0006) [2023-12-26 22:14:54,601][105620] Updated weights for policy 1, policy_version 945000 (0.0010) [2023-12-26 22:14:55,215][105692] Updated weights for policy 0, policy_version 944785 (0.0006) [2023-12-26 22:14:55,271][105692] Updated weights for policy 0, policy_version 944795 (0.0007) [2023-12-26 22:14:55,300][105620] Updated weights for policy 1, policy_version 945010 (0.0011) [2023-12-26 22:14:55,326][105692] Updated weights for policy 0, policy_version 944805 (0.0005) [2023-12-26 22:14:55,356][105620] Updated weights for policy 1, policy_version 945020 (0.0010) [2023-12-26 22:14:55,386][105692] Updated weights for policy 0, policy_version 944815 (0.0006) [2023-12-26 22:14:55,412][105620] Updated weights for policy 1, policy_version 945030 (0.0010) [2023-12-26 22:14:56,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19114.7, 300 sec: 19161.0). Total num frames: 483868672. Throughput: 0: 9585.2, 1: 9610.7. Samples: 483882732. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 22:14:56,062][104569] Avg episode reward: [(0, '9168.508'), (1, '8915.231')] [2023-12-26 22:14:56,141][105692] Updated weights for policy 0, policy_version 944825 (0.0007) [2023-12-26 22:14:56,143][105620] Updated weights for policy 1, policy_version 945040 (0.0010) [2023-12-26 22:14:56,198][105620] Updated weights for policy 1, policy_version 945050 (0.0011) [2023-12-26 22:14:56,200][105692] Updated weights for policy 0, policy_version 944835 (0.0005) [2023-12-26 22:14:56,248][105692] Updated weights for policy 0, policy_version 944845 (0.0007) [2023-12-26 22:14:56,253][105620] Updated weights for policy 1, policy_version 945060 (0.0010) [2023-12-26 22:14:57,002][105620] Updated weights for policy 1, policy_version 945070 (0.0010) [2023-12-26 22:14:57,019][105692] Updated weights for policy 0, policy_version 944855 (0.0007) [2023-12-26 22:14:57,053][105620] Updated weights for policy 1, policy_version 945080 (0.0010) [2023-12-26 22:14:57,074][105692] Updated weights for policy 0, policy_version 944865 (0.0005) [2023-12-26 22:14:57,107][105620] Updated weights for policy 1, policy_version 945090 (0.0010) [2023-12-26 22:14:57,135][105692] Updated weights for policy 0, policy_version 944875 (0.0006) [2023-12-26 22:14:57,786][105620] Updated weights for policy 1, policy_version 945100 (0.0006) [2023-12-26 22:14:57,835][105620] Updated weights for policy 1, policy_version 945110 (0.0005) [2023-12-26 22:14:57,882][105620] Updated weights for policy 1, policy_version 945120 (0.0005) [2023-12-26 22:14:57,952][105692] Updated weights for policy 0, policy_version 944885 (0.0009) [2023-12-26 22:14:58,010][105692] Updated weights for policy 0, policy_version 944895 (0.0009) [2023-12-26 22:14:58,063][105692] Updated weights for policy 0, policy_version 944905 (0.0010) [2023-12-26 22:14:58,482][105620] Updated weights for policy 1, policy_version 945130 (0.0006) [2023-12-26 22:14:58,546][105620] Updated weights for policy 1, policy_version 945140 (0.0010) [2023-12-26 22:14:58,608][105620] Updated weights for policy 1, policy_version 945150 (0.0007) [2023-12-26 22:14:58,671][105620] Updated weights for policy 1, policy_version 945160 (0.0008) [2023-12-26 22:14:58,927][105692] Updated weights for policy 0, policy_version 944915 (0.0009) [2023-12-26 22:14:58,988][105692] Updated weights for policy 0, policy_version 944925 (0.0008) [2023-12-26 22:14:59,050][105692] Updated weights for policy 0, policy_version 944935 (0.0007) [2023-12-26 22:14:59,397][105620] Updated weights for policy 1, policy_version 945170 (0.0009) [2023-12-26 22:14:59,463][105620] Updated weights for policy 1, policy_version 945180 (0.0010) [2023-12-26 22:14:59,528][105620] Updated weights for policy 1, policy_version 945190 (0.0010) [2023-12-26 22:14:59,718][105692] Updated weights for policy 0, policy_version 944945 (0.0007) [2023-12-26 22:14:59,786][105692] Updated weights for policy 0, policy_version 944955 (0.0008) [2023-12-26 22:14:59,850][105692] Updated weights for policy 0, policy_version 944965 (0.0009) [2023-12-26 22:14:59,908][105692] Updated weights for policy 0, policy_version 944975 (0.0009) [2023-12-26 22:15:00,256][105620] Updated weights for policy 1, policy_version 945200 (0.0009) [2023-12-26 22:15:00,314][105620] Updated weights for policy 1, policy_version 945210 (0.0009) [2023-12-26 22:15:00,365][105620] Updated weights for policy 1, policy_version 945220 (0.0009) [2023-12-26 22:15:00,635][105692] Updated weights for policy 0, policy_version 944985 (0.0009) [2023-12-26 22:15:00,689][105692] Updated weights for policy 0, policy_version 944995 (0.0009) [2023-12-26 22:15:00,747][105692] Updated weights for policy 0, policy_version 945005 (0.0009) [2023-12-26 22:15:01,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19114.7, 300 sec: 19188.7). Total num frames: 483966976. Throughput: 0: 9517.0, 1: 9714.5. Samples: 483939248. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 22:15:01,062][104569] Avg episode reward: [(0, '8990.970'), (1, '8662.748')] [2023-12-26 22:15:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000945008_241958912.pth... [2023-12-26 22:15:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000943920_241680384.pth [2023-12-26 22:15:01,103][105620] Updated weights for policy 1, policy_version 945230 (0.0009) [2023-12-26 22:15:01,171][105620] Updated weights for policy 1, policy_version 945240 (0.0008) [2023-12-26 22:15:01,226][105620] Updated weights for policy 1, policy_version 945250 (0.0009) [2023-12-26 22:15:01,264][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000945256_242016256.pth... [2023-12-26 22:15:01,268][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000944104_241721344.pth [2023-12-26 22:15:01,556][105692] Updated weights for policy 0, policy_version 945016 (0.0009) [2023-12-26 22:15:01,612][105692] Updated weights for policy 0, policy_version 945026 (0.0010) [2023-12-26 22:15:01,668][105692] Updated weights for policy 0, policy_version 945036 (0.0009) [2023-12-26 22:15:01,933][105620] Updated weights for policy 1, policy_version 945260 (0.0007) [2023-12-26 22:15:01,995][105620] Updated weights for policy 1, policy_version 945270 (0.0005) [2023-12-26 22:15:02,052][105620] Updated weights for policy 1, policy_version 945280 (0.0005) [2023-12-26 22:15:02,554][105692] Updated weights for policy 0, policy_version 945046 (0.0008) [2023-12-26 22:15:02,617][105692] Updated weights for policy 0, policy_version 945056 (0.0006) [2023-12-26 22:15:02,655][105620] Updated weights for policy 1, policy_version 945290 (0.0006) [2023-12-26 22:15:02,685][105692] Updated weights for policy 0, policy_version 945066 (0.0005) [2023-12-26 22:15:02,716][105620] Updated weights for policy 1, policy_version 945300 (0.0009) [2023-12-26 22:15:02,773][105620] Updated weights for policy 1, policy_version 945310 (0.0010) [2023-12-26 22:15:02,835][105620] Updated weights for policy 1, policy_version 945320 (0.0011) [2023-12-26 22:15:03,361][105692] Updated weights for policy 0, policy_version 945076 (0.0006) [2023-12-26 22:15:03,409][105692] Updated weights for policy 0, policy_version 945086 (0.0008) [2023-12-26 22:15:03,456][105692] Updated weights for policy 0, policy_version 945096 (0.0007) [2023-12-26 22:15:03,587][105620] Updated weights for policy 1, policy_version 945330 (0.0009) [2023-12-26 22:15:03,635][105620] Updated weights for policy 1, policy_version 945340 (0.0010) [2023-12-26 22:15:03,684][105620] Updated weights for policy 1, policy_version 945350 (0.0010) [2023-12-26 22:15:04,154][105692] Updated weights for policy 0, policy_version 945106 (0.0008) [2023-12-26 22:15:04,208][105692] Updated weights for policy 0, policy_version 945116 (0.0008) [2023-12-26 22:15:04,280][105692] Updated weights for policy 0, policy_version 945126 (0.0006) [2023-12-26 22:15:04,345][105692] Updated weights for policy 0, policy_version 945136 (0.0009) [2023-12-26 22:15:04,498][105620] Updated weights for policy 1, policy_version 945360 (0.0007) [2023-12-26 22:15:04,555][105620] Updated weights for policy 1, policy_version 945370 (0.0008) [2023-12-26 22:15:04,618][105620] Updated weights for policy 1, policy_version 945380 (0.0010) [2023-12-26 22:15:05,071][105692] Updated weights for policy 0, policy_version 945146 (0.0009) [2023-12-26 22:15:05,119][105692] Updated weights for policy 0, policy_version 945156 (0.0009) [2023-12-26 22:15:05,174][105692] Updated weights for policy 0, policy_version 945166 (0.0010) [2023-12-26 22:15:05,239][105620] Updated weights for policy 1, policy_version 945390 (0.0007) [2023-12-26 22:15:05,292][105620] Updated weights for policy 1, policy_version 945400 (0.0008) [2023-12-26 22:15:05,353][105620] Updated weights for policy 1, policy_version 945410 (0.0008) [2023-12-26 22:15:05,984][105620] Updated weights for policy 1, policy_version 945420 (0.0009) [2023-12-26 22:15:06,012][105692] Updated weights for policy 0, policy_version 945176 (0.0006) [2023-12-26 22:15:06,040][105620] Updated weights for policy 1, policy_version 945430 (0.0007) [2023-12-26 22:15:06,062][104569] Fps is (10 sec: 18841.4, 60 sec: 18978.2, 300 sec: 19161.0). Total num frames: 484057088. Throughput: 0: 9456.2, 1: 9682.8. Samples: 484052956. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 22:15:06,063][104569] Avg episode reward: [(0, '9086.017'), (1, '8477.975')] [2023-12-26 22:15:06,072][105692] Updated weights for policy 0, policy_version 945186 (0.0006) [2023-12-26 22:15:06,098][105620] Updated weights for policy 1, policy_version 945440 (0.0006) [2023-12-26 22:15:06,134][105692] Updated weights for policy 0, policy_version 945196 (0.0009) [2023-12-26 22:15:06,877][105620] Updated weights for policy 1, policy_version 945450 (0.0007) [2023-12-26 22:15:06,880][105692] Updated weights for policy 0, policy_version 945206 (0.0008) [2023-12-26 22:15:06,932][105620] Updated weights for policy 1, policy_version 945460 (0.0005) [2023-12-26 22:15:06,947][105692] Updated weights for policy 0, policy_version 945216 (0.0008) [2023-12-26 22:15:06,993][105620] Updated weights for policy 1, policy_version 945470 (0.0006) [2023-12-26 22:15:07,009][105692] Updated weights for policy 0, policy_version 945226 (0.0011) [2023-12-26 22:15:07,045][105620] Updated weights for policy 1, policy_version 945480 (0.0007) [2023-12-26 22:15:07,576][105692] Updated weights for policy 0, policy_version 945236 (0.0009) [2023-12-26 22:15:07,627][105692] Updated weights for policy 0, policy_version 945246 (0.0007) [2023-12-26 22:15:07,676][105692] Updated weights for policy 0, policy_version 945256 (0.0006) [2023-12-26 22:15:07,825][105620] Updated weights for policy 1, policy_version 945490 (0.0009) [2023-12-26 22:15:07,886][105620] Updated weights for policy 1, policy_version 945500 (0.0009) [2023-12-26 22:15:07,940][105620] Updated weights for policy 1, policy_version 945510 (0.0009) [2023-12-26 22:15:08,240][105692] Updated weights for policy 0, policy_version 945266 (0.0006) [2023-12-26 22:15:08,285][105692] Updated weights for policy 0, policy_version 945276 (0.0005) [2023-12-26 22:15:08,347][105692] Updated weights for policy 0, policy_version 945286 (0.0007) [2023-12-26 22:15:08,413][105692] Updated weights for policy 0, policy_version 945296 (0.0008) [2023-12-26 22:15:08,765][105620] Updated weights for policy 1, policy_version 945520 (0.0007) [2023-12-26 22:15:08,829][105620] Updated weights for policy 1, policy_version 945530 (0.0006) [2023-12-26 22:15:08,891][105620] Updated weights for policy 1, policy_version 945540 (0.0006) [2023-12-26 22:15:09,195][105692] Updated weights for policy 0, policy_version 945306 (0.0009) [2023-12-26 22:15:09,263][105692] Updated weights for policy 0, policy_version 945316 (0.0008) [2023-12-26 22:15:09,332][105692] Updated weights for policy 0, policy_version 945326 (0.0006) [2023-12-26 22:15:09,526][105620] Updated weights for policy 1, policy_version 945550 (0.0009) [2023-12-26 22:15:09,594][105620] Updated weights for policy 1, policy_version 945560 (0.0009) [2023-12-26 22:15:09,661][105620] Updated weights for policy 1, policy_version 945570 (0.0008) [2023-12-26 22:15:10,107][105692] Updated weights for policy 0, policy_version 945336 (0.0009) [2023-12-26 22:15:10,169][105692] Updated weights for policy 0, policy_version 945346 (0.0008) [2023-12-26 22:15:10,235][105692] Updated weights for policy 0, policy_version 945356 (0.0010) [2023-12-26 22:15:10,430][105620] Updated weights for policy 1, policy_version 945580 (0.0009) [2023-12-26 22:15:10,494][105620] Updated weights for policy 1, policy_version 945590 (0.0008) [2023-12-26 22:15:10,552][105620] Updated weights for policy 1, policy_version 945600 (0.0009) [2023-12-26 22:15:10,881][105692] Updated weights for policy 0, policy_version 945366 (0.0007) [2023-12-26 22:15:10,930][105692] Updated weights for policy 0, policy_version 945376 (0.0005) [2023-12-26 22:15:10,983][105692] Updated weights for policy 0, policy_version 945386 (0.0005) [2023-12-26 22:15:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19188.7). Total num frames: 484163584. Throughput: 0: 9491.2, 1: 9673.3. Samples: 484167888. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 22:15:11,063][104569] Avg episode reward: [(0, '8910.429'), (1, '8468.657')] [2023-12-26 22:15:11,458][105620] Updated weights for policy 1, policy_version 945610 (0.0008) [2023-12-26 22:15:11,520][105620] Updated weights for policy 1, policy_version 945620 (0.0010) [2023-12-26 22:15:11,585][105620] Updated weights for policy 1, policy_version 945630 (0.0009) [2023-12-26 22:15:11,636][105692] Updated weights for policy 0, policy_version 945396 (0.0008) [2023-12-26 22:15:11,652][105620] Updated weights for policy 1, policy_version 945640 (0.0008) [2023-12-26 22:15:11,700][105692] Updated weights for policy 0, policy_version 945406 (0.0008) [2023-12-26 22:15:11,762][105692] Updated weights for policy 0, policy_version 945416 (0.0008) [2023-12-26 22:15:12,399][105620] Updated weights for policy 1, policy_version 945650 (0.0011) [2023-12-26 22:15:12,460][105620] Updated weights for policy 1, policy_version 945660 (0.0008) [2023-12-26 22:15:12,461][105692] Updated weights for policy 0, policy_version 945426 (0.0008) [2023-12-26 22:15:12,515][105692] Updated weights for policy 0, policy_version 945436 (0.0006) [2023-12-26 22:15:12,516][105620] Updated weights for policy 1, policy_version 945670 (0.0008) [2023-12-26 22:15:12,580][105692] Updated weights for policy 0, policy_version 945446 (0.0007) [2023-12-26 22:15:12,636][105692] Updated weights for policy 0, policy_version 945456 (0.0006) [2023-12-26 22:15:13,229][105620] Updated weights for policy 1, policy_version 945680 (0.0006) [2023-12-26 22:15:13,290][105692] Updated weights for policy 0, policy_version 945466 (0.0008) [2023-12-26 22:15:13,291][105620] Updated weights for policy 1, policy_version 945690 (0.0006) [2023-12-26 22:15:13,350][105620] Updated weights for policy 1, policy_version 945700 (0.0007) [2023-12-26 22:15:13,354][105692] Updated weights for policy 0, policy_version 945476 (0.0008) [2023-12-26 22:15:13,410][105692] Updated weights for policy 0, policy_version 945486 (0.0008) [2023-12-26 22:15:13,943][105620] Updated weights for policy 1, policy_version 945710 (0.0010) [2023-12-26 22:15:13,995][105620] Updated weights for policy 1, policy_version 945720 (0.0010) [2023-12-26 22:15:14,040][105620] Updated weights for policy 1, policy_version 945730 (0.0010) [2023-12-26 22:15:14,091][105692] Updated weights for policy 0, policy_version 945496 (0.0009) [2023-12-26 22:15:14,148][105692] Updated weights for policy 0, policy_version 945506 (0.0010) [2023-12-26 22:15:14,206][105692] Updated weights for policy 0, policy_version 945517 (0.0010) [2023-12-26 22:15:14,751][105620] Updated weights for policy 1, policy_version 945740 (0.0010) [2023-12-26 22:15:14,816][105620] Updated weights for policy 1, policy_version 945750 (0.0009) [2023-12-26 22:15:14,879][105620] Updated weights for policy 1, policy_version 945760 (0.0010) [2023-12-26 22:15:15,059][105692] Updated weights for policy 0, policy_version 945527 (0.0009) [2023-12-26 22:15:15,125][105692] Updated weights for policy 0, policy_version 945537 (0.0008) [2023-12-26 22:15:15,187][105692] Updated weights for policy 0, policy_version 945547 (0.0008) [2023-12-26 22:15:15,647][105620] Updated weights for policy 1, policy_version 945770 (0.0011) [2023-12-26 22:15:15,696][105620] Updated weights for policy 1, policy_version 945780 (0.0006) [2023-12-26 22:15:15,757][105620] Updated weights for policy 1, policy_version 945790 (0.0005) [2023-12-26 22:15:15,820][105620] Updated weights for policy 1, policy_version 945800 (0.0005) [2023-12-26 22:15:16,007][105692] Updated weights for policy 0, policy_version 945557 (0.0009) [2023-12-26 22:15:16,060][105692] Updated weights for policy 0, policy_version 945567 (0.0010) [2023-12-26 22:15:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19114.6, 300 sec: 19217.0). Total num frames: 484253696. Throughput: 0: 9481.1, 1: 9731.6. Samples: 484227452. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 22:15:16,062][104569] Avg episode reward: [(0, '9082.034'), (1, '8737.379')] [2023-12-26 22:15:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000945800_242155520.pth... [2023-12-26 22:15:16,095][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000944648_241860608.pth [2023-12-26 22:15:16,114][105692] Updated weights for policy 0, policy_version 945578 (0.0010) [2023-12-26 22:15:16,143][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000945584_242106368.pth... [2023-12-26 22:15:16,146][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000944464_241819648.pth [2023-12-26 22:15:16,341][105620] Updated weights for policy 1, policy_version 945810 (0.0005) [2023-12-26 22:15:16,395][105620] Updated weights for policy 1, policy_version 945820 (0.0005) [2023-12-26 22:15:16,450][105620] Updated weights for policy 1, policy_version 945830 (0.0005) [2023-12-26 22:15:17,000][105692] Updated weights for policy 0, policy_version 945589 (0.0012) [2023-12-26 22:15:17,054][105692] Updated weights for policy 0, policy_version 945599 (0.0008) [2023-12-26 22:15:17,081][105620] Updated weights for policy 1, policy_version 945840 (0.0007) [2023-12-26 22:15:17,111][105692] Updated weights for policy 0, policy_version 945609 (0.0008) [2023-12-26 22:15:17,138][105620] Updated weights for policy 1, policy_version 945850 (0.0006) [2023-12-26 22:15:17,201][105620] Updated weights for policy 1, policy_version 945860 (0.0008) [2023-12-26 22:15:17,871][105692] Updated weights for policy 0, policy_version 945619 (0.0007) [2023-12-26 22:15:17,918][105692] Updated weights for policy 0, policy_version 945629 (0.0009) [2023-12-26 22:15:17,937][105620] Updated weights for policy 1, policy_version 945870 (0.0008) [2023-12-26 22:15:17,968][105692] Updated weights for policy 0, policy_version 945639 (0.0008) [2023-12-26 22:15:17,990][105620] Updated weights for policy 1, policy_version 945880 (0.0008) [2023-12-26 22:15:18,047][105620] Updated weights for policy 1, policy_version 945890 (0.0009) [2023-12-26 22:15:18,766][105692] Updated weights for policy 0, policy_version 945649 (0.0008) [2023-12-26 22:15:18,796][105620] Updated weights for policy 1, policy_version 945900 (0.0007) [2023-12-26 22:15:18,826][105692] Updated weights for policy 0, policy_version 945659 (0.0011) [2023-12-26 22:15:18,867][105620] Updated weights for policy 1, policy_version 945910 (0.0005) [2023-12-26 22:15:18,884][105692] Updated weights for policy 0, policy_version 945669 (0.0010) [2023-12-26 22:15:18,931][105620] Updated weights for policy 1, policy_version 945920 (0.0006) [2023-12-26 22:15:18,937][105692] Updated weights for policy 0, policy_version 945679 (0.0010) [2023-12-26 22:15:19,650][105692] Updated weights for policy 0, policy_version 945689 (0.0011) [2023-12-26 22:15:19,697][105620] Updated weights for policy 1, policy_version 945930 (0.0008) [2023-12-26 22:15:19,717][105692] Updated weights for policy 0, policy_version 945699 (0.0011) [2023-12-26 22:15:19,764][105620] Updated weights for policy 1, policy_version 945940 (0.0009) [2023-12-26 22:15:19,784][105692] Updated weights for policy 0, policy_version 945709 (0.0010) [2023-12-26 22:15:19,827][105620] Updated weights for policy 1, policy_version 945950 (0.0008) [2023-12-26 22:15:19,889][105620] Updated weights for policy 1, policy_version 945960 (0.0009) [2023-12-26 22:15:20,546][105692] Updated weights for policy 0, policy_version 945719 (0.0008) [2023-12-26 22:15:20,613][105692] Updated weights for policy 0, policy_version 945729 (0.0008) [2023-12-26 22:15:20,650][105620] Updated weights for policy 1, policy_version 945970 (0.0011) [2023-12-26 22:15:20,681][105692] Updated weights for policy 0, policy_version 945739 (0.0006) [2023-12-26 22:15:20,711][105620] Updated weights for policy 1, policy_version 945980 (0.0011) [2023-12-26 22:15:20,778][105620] Updated weights for policy 1, policy_version 945990 (0.0011) [2023-12-26 22:15:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19216.5). Total num frames: 484352000. Throughput: 0: 9435.9, 1: 9715.2. Samples: 484341396. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 22:15:21,063][104569] Avg episode reward: [(0, '9168.681'), (1, '9095.289')] [2023-12-26 22:15:21,472][105692] Updated weights for policy 0, policy_version 945749 (0.0007) [2023-12-26 22:15:21,534][105692] Updated weights for policy 0, policy_version 945759 (0.0006) [2023-12-26 22:15:21,551][105620] Updated weights for policy 1, policy_version 946000 (0.0010) [2023-12-26 22:15:21,592][105692] Updated weights for policy 0, policy_version 945769 (0.0007) [2023-12-26 22:15:21,616][105620] Updated weights for policy 1, policy_version 946010 (0.0010) [2023-12-26 22:15:21,682][105620] Updated weights for policy 1, policy_version 946020 (0.0011) [2023-12-26 22:15:22,324][105692] Updated weights for policy 0, policy_version 945779 (0.0007) [2023-12-26 22:15:22,389][105692] Updated weights for policy 0, policy_version 945789 (0.0008) [2023-12-26 22:15:22,445][105692] Updated weights for policy 0, policy_version 945799 (0.0006) [2023-12-26 22:15:22,451][105620] Updated weights for policy 1, policy_version 946030 (0.0010) [2023-12-26 22:15:22,514][105620] Updated weights for policy 1, policy_version 946040 (0.0011) [2023-12-26 22:15:22,578][105620] Updated weights for policy 1, policy_version 946050 (0.0011) [2023-12-26 22:15:23,232][105620] Updated weights for policy 1, policy_version 946060 (0.0009) [2023-12-26 22:15:23,248][105692] Updated weights for policy 0, policy_version 945809 (0.0006) [2023-12-26 22:15:23,288][105620] Updated weights for policy 1, policy_version 946070 (0.0005) [2023-12-26 22:15:23,308][105692] Updated weights for policy 0, policy_version 945819 (0.0007) [2023-12-26 22:15:23,335][105620] Updated weights for policy 1, policy_version 946080 (0.0005) [2023-12-26 22:15:23,364][105692] Updated weights for policy 0, policy_version 945829 (0.0008) [2023-12-26 22:15:23,421][105692] Updated weights for policy 0, policy_version 945839 (0.0007) [2023-12-26 22:15:23,913][105620] Updated weights for policy 1, policy_version 946090 (0.0008) [2023-12-26 22:15:23,973][105620] Updated weights for policy 1, policy_version 946100 (0.0006) [2023-12-26 22:15:24,042][105620] Updated weights for policy 1, policy_version 946110 (0.0008) [2023-12-26 22:15:24,101][105620] Updated weights for policy 1, policy_version 946120 (0.0011) [2023-12-26 22:15:24,254][105692] Updated weights for policy 0, policy_version 945849 (0.0009) [2023-12-26 22:15:24,316][105692] Updated weights for policy 0, policy_version 945859 (0.0009) [2023-12-26 22:15:24,384][105692] Updated weights for policy 0, policy_version 945869 (0.0009) [2023-12-26 22:15:24,715][105620] Updated weights for policy 1, policy_version 946130 (0.0005) [2023-12-26 22:15:24,772][105620] Updated weights for policy 1, policy_version 946140 (0.0005) [2023-12-26 22:15:24,817][105620] Updated weights for policy 1, policy_version 946150 (0.0007) [2023-12-26 22:15:25,207][105692] Updated weights for policy 0, policy_version 945879 (0.0009) [2023-12-26 22:15:25,266][105692] Updated weights for policy 0, policy_version 945889 (0.0007) [2023-12-26 22:15:25,320][105692] Updated weights for policy 0, policy_version 945899 (0.0005) [2023-12-26 22:15:25,472][105620] Updated weights for policy 1, policy_version 946160 (0.0010) [2023-12-26 22:15:25,529][105620] Updated weights for policy 1, policy_version 946170 (0.0008) [2023-12-26 22:15:25,590][105620] Updated weights for policy 1, policy_version 946180 (0.0009) [2023-12-26 22:15:25,974][105692] Updated weights for policy 0, policy_version 945909 (0.0005) [2023-12-26 22:15:26,030][105692] Updated weights for policy 0, policy_version 945919 (0.0005) [2023-12-26 22:15:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19114.7, 300 sec: 19189.0). Total num frames: 484442112. Throughput: 0: 9416.3, 1: 9775.0. Samples: 484454792. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 22:15:26,062][104569] Avg episode reward: [(0, '9260.141'), (1, '9006.002')] [2023-12-26 22:15:26,080][105692] Updated weights for policy 0, policy_version 945929 (0.0008) [2023-12-26 22:15:26,373][105620] Updated weights for policy 1, policy_version 946190 (0.0009) [2023-12-26 22:15:26,431][105620] Updated weights for policy 1, policy_version 946200 (0.0008) [2023-12-26 22:15:26,485][105620] Updated weights for policy 1, policy_version 946210 (0.0008) [2023-12-26 22:15:26,799][105692] Updated weights for policy 0, policy_version 945939 (0.0008) [2023-12-26 22:15:26,858][105692] Updated weights for policy 0, policy_version 945949 (0.0009) [2023-12-26 22:15:26,915][105692] Updated weights for policy 0, policy_version 945959 (0.0009) [2023-12-26 22:15:27,220][105620] Updated weights for policy 1, policy_version 946220 (0.0009) [2023-12-26 22:15:27,272][105620] Updated weights for policy 1, policy_version 946230 (0.0008) [2023-12-26 22:15:27,327][105620] Updated weights for policy 1, policy_version 946240 (0.0006) [2023-12-26 22:15:27,674][105692] Updated weights for policy 0, policy_version 945969 (0.0009) [2023-12-26 22:15:27,737][105692] Updated weights for policy 0, policy_version 945979 (0.0010) [2023-12-26 22:15:27,792][105692] Updated weights for policy 0, policy_version 945989 (0.0009) [2023-12-26 22:15:27,858][105692] Updated weights for policy 0, policy_version 945999 (0.0010) [2023-12-26 22:15:27,919][105620] Updated weights for policy 1, policy_version 946250 (0.0006) [2023-12-26 22:15:27,987][105620] Updated weights for policy 1, policy_version 946260 (0.0009) [2023-12-26 22:15:28,048][105620] Updated weights for policy 1, policy_version 946270 (0.0009) [2023-12-26 22:15:28,109][105620] Updated weights for policy 1, policy_version 946280 (0.0009) [2023-12-26 22:15:28,584][105692] Updated weights for policy 0, policy_version 946009 (0.0007) [2023-12-26 22:15:28,649][105692] Updated weights for policy 0, policy_version 946019 (0.0009) [2023-12-26 22:15:28,698][105692] Updated weights for policy 0, policy_version 946029 (0.0008) [2023-12-26 22:15:28,890][105620] Updated weights for policy 1, policy_version 946290 (0.0009) [2023-12-26 22:15:28,937][105620] Updated weights for policy 1, policy_version 946300 (0.0009) [2023-12-26 22:15:28,984][105620] Updated weights for policy 1, policy_version 946310 (0.0008) [2023-12-26 22:15:29,393][105692] Updated weights for policy 0, policy_version 946039 (0.0008) [2023-12-26 22:15:29,453][105692] Updated weights for policy 0, policy_version 946049 (0.0005) [2023-12-26 22:15:29,509][105692] Updated weights for policy 0, policy_version 946059 (0.0006) [2023-12-26 22:15:29,790][105620] Updated weights for policy 1, policy_version 946320 (0.0006) [2023-12-26 22:15:29,860][105620] Updated weights for policy 1, policy_version 946330 (0.0008) [2023-12-26 22:15:29,913][105620] Updated weights for policy 1, policy_version 946340 (0.0009) [2023-12-26 22:15:30,201][105692] Updated weights for policy 0, policy_version 946069 (0.0008) [2023-12-26 22:15:30,261][105692] Updated weights for policy 0, policy_version 946079 (0.0010) [2023-12-26 22:15:30,323][105692] Updated weights for policy 0, policy_version 946089 (0.0010) [2023-12-26 22:15:30,592][105620] Updated weights for policy 1, policy_version 946350 (0.0010) [2023-12-26 22:15:30,648][105620] Updated weights for policy 1, policy_version 946360 (0.0009) [2023-12-26 22:15:30,707][105620] Updated weights for policy 1, policy_version 946370 (0.0010) [2023-12-26 22:15:30,928][105692] Updated weights for policy 0, policy_version 946099 (0.0009) [2023-12-26 22:15:30,988][105692] Updated weights for policy 0, policy_version 946109 (0.0010) [2023-12-26 22:15:31,039][105692] Updated weights for policy 0, policy_version 946119 (0.0010) [2023-12-26 22:15:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19114.7, 300 sec: 19188.7). Total num frames: 484540416. Throughput: 0: 9449.9, 1: 9783.8. Samples: 484512748. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 22:15:31,063][104569] Avg episode reward: [(0, '9171.887'), (1, '8922.032')] [2023-12-26 22:15:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000946376_242302976.pth... [2023-12-26 22:15:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000945256_242016256.pth [2023-12-26 22:15:31,091][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000946128_242245632.pth... [2023-12-26 22:15:31,095][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000945008_241958912.pth [2023-12-26 22:15:31,413][105620] Updated weights for policy 1, policy_version 946380 (0.0009) [2023-12-26 22:15:31,472][105620] Updated weights for policy 1, policy_version 946390 (0.0010) [2023-12-26 22:15:31,538][105620] Updated weights for policy 1, policy_version 946400 (0.0010) [2023-12-26 22:15:31,804][105692] Updated weights for policy 0, policy_version 946129 (0.0010) [2023-12-26 22:15:31,869][105692] Updated weights for policy 0, policy_version 946140 (0.0008) [2023-12-26 22:15:31,927][105692] Updated weights for policy 0, policy_version 946150 (0.0008) [2023-12-26 22:15:31,981][105692] Updated weights for policy 0, policy_version 946160 (0.0008) [2023-12-26 22:15:32,305][105620] Updated weights for policy 1, policy_version 946410 (0.0011) [2023-12-26 22:15:32,372][105620] Updated weights for policy 1, policy_version 946420 (0.0010) [2023-12-26 22:15:32,432][105620] Updated weights for policy 1, policy_version 946430 (0.0011) [2023-12-26 22:15:32,488][105620] Updated weights for policy 1, policy_version 946440 (0.0010) [2023-12-26 22:15:32,717][105692] Updated weights for policy 0, policy_version 946170 (0.0008) [2023-12-26 22:15:32,762][105692] Updated weights for policy 0, policy_version 946180 (0.0007) [2023-12-26 22:15:32,813][105692] Updated weights for policy 0, policy_version 946190 (0.0007) [2023-12-26 22:15:33,235][105620] Updated weights for policy 1, policy_version 946450 (0.0010) [2023-12-26 22:15:33,282][105620] Updated weights for policy 1, policy_version 946460 (0.0010) [2023-12-26 22:15:33,326][105620] Updated weights for policy 1, policy_version 946470 (0.0010) [2023-12-26 22:15:33,461][105692] Updated weights for policy 0, policy_version 946200 (0.0006) [2023-12-26 22:15:33,518][105692] Updated weights for policy 0, policy_version 946210 (0.0005) [2023-12-26 22:15:33,580][105692] Updated weights for policy 0, policy_version 946220 (0.0006) [2023-12-26 22:15:33,980][105620] Updated weights for policy 1, policy_version 946480 (0.0007) [2023-12-26 22:15:34,035][105620] Updated weights for policy 1, policy_version 946490 (0.0007) [2023-12-26 22:15:34,086][105620] Updated weights for policy 1, policy_version 946500 (0.0008) [2023-12-26 22:15:34,236][105692] Updated weights for policy 0, policy_version 946230 (0.0008) [2023-12-26 22:15:34,331][105692] Updated weights for policy 0, policy_version 946240 (0.0009) [2023-12-26 22:15:34,398][105692] Updated weights for policy 0, policy_version 946250 (0.0009) [2023-12-26 22:15:34,875][105620] Updated weights for policy 1, policy_version 946510 (0.0008) [2023-12-26 22:15:34,940][105620] Updated weights for policy 1, policy_version 946520 (0.0009) [2023-12-26 22:15:34,995][105620] Updated weights for policy 1, policy_version 946530 (0.0009) [2023-12-26 22:15:35,062][105692] Updated weights for policy 0, policy_version 946260 (0.0009) [2023-12-26 22:15:35,123][105692] Updated weights for policy 0, policy_version 946270 (0.0009) [2023-12-26 22:15:35,184][105692] Updated weights for policy 0, policy_version 946280 (0.0009) [2023-12-26 22:15:35,739][105620] Updated weights for policy 1, policy_version 946540 (0.0009) [2023-12-26 22:15:35,798][105620] Updated weights for policy 1, policy_version 946550 (0.0009) [2023-12-26 22:15:35,862][105620] Updated weights for policy 1, policy_version 946560 (0.0009) [2023-12-26 22:15:35,932][105692] Updated weights for policy 0, policy_version 946290 (0.0009) [2023-12-26 22:15:35,986][105692] Updated weights for policy 0, policy_version 946300 (0.0009) [2023-12-26 22:15:36,030][105692] Updated weights for policy 0, policy_version 946310 (0.0008) [2023-12-26 22:15:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19216.5). Total num frames: 484638720. Throughput: 0: 9432.8, 1: 9737.2. Samples: 484629836. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 22:15:36,063][104569] Avg episode reward: [(0, '9263.135'), (1, '8838.498')] [2023-12-26 22:15:36,081][105692] Updated weights for policy 0, policy_version 946320 (0.0009) [2023-12-26 22:15:36,609][105620] Updated weights for policy 1, policy_version 946570 (0.0009) [2023-12-26 22:15:36,675][105620] Updated weights for policy 1, policy_version 946580 (0.0008) [2023-12-26 22:15:36,722][105620] Updated weights for policy 1, policy_version 946590 (0.0005) [2023-12-26 22:15:36,784][105620] Updated weights for policy 1, policy_version 946600 (0.0006) [2023-12-26 22:15:36,908][105692] Updated weights for policy 0, policy_version 946330 (0.0009) [2023-12-26 22:15:36,962][105692] Updated weights for policy 0, policy_version 946340 (0.0010) [2023-12-26 22:15:37,019][105692] Updated weights for policy 0, policy_version 946351 (0.0010) [2023-12-26 22:15:37,444][105620] Updated weights for policy 1, policy_version 946610 (0.0009) [2023-12-26 22:15:37,519][105620] Updated weights for policy 1, policy_version 946620 (0.0010) [2023-12-26 22:15:37,586][105620] Updated weights for policy 1, policy_version 946630 (0.0010) [2023-12-26 22:15:37,758][105692] Updated weights for policy 0, policy_version 946361 (0.0009) [2023-12-26 22:15:37,822][105692] Updated weights for policy 0, policy_version 946371 (0.0009) [2023-12-26 22:15:37,846][105585] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000001 [2023-12-26 22:15:38,345][105620] Updated weights for policy 1, policy_version 946640 (0.0010) [2023-12-26 22:15:38,409][105620] Updated weights for policy 1, policy_version 946650 (0.0009) [2023-12-26 22:15:38,471][105620] Updated weights for policy 1, policy_version 946660 (0.0009) [2023-12-26 22:15:38,650][105692] Updated weights for policy 0, policy_version 946381 (0.0010) [2023-12-26 22:15:38,705][105692] Updated weights for policy 0, policy_version 946391 (0.0009) [2023-12-26 22:15:38,765][105692] Updated weights for policy 0, policy_version 946401 (0.0009) [2023-12-26 22:15:39,160][105620] Updated weights for policy 1, policy_version 946670 (0.0009) [2023-12-26 22:15:39,212][105620] Updated weights for policy 1, policy_version 946680 (0.0009) [2023-12-26 22:15:39,278][105620] Updated weights for policy 1, policy_version 946690 (0.0010) [2023-12-26 22:15:39,521][105692] Updated weights for policy 0, policy_version 946411 (0.0009) [2023-12-26 22:15:39,580][105692] Updated weights for policy 0, policy_version 946421 (0.0008) [2023-12-26 22:15:39,644][105692] Updated weights for policy 0, policy_version 946431 (0.0008) [2023-12-26 22:15:40,073][105620] Updated weights for policy 1, policy_version 946700 (0.0009) [2023-12-26 22:15:40,134][105620] Updated weights for policy 1, policy_version 946710 (0.0008) [2023-12-26 22:15:40,202][105620] Updated weights for policy 1, policy_version 946720 (0.0008) [2023-12-26 22:15:40,472][105692] Updated weights for policy 0, policy_version 946441 (0.0008) [2023-12-26 22:15:40,527][105692] Updated weights for policy 0, policy_version 946451 (0.0009) [2023-12-26 22:15:40,583][105692] Updated weights for policy 0, policy_version 946461 (0.0008) [2023-12-26 22:15:40,640][105692] Updated weights for policy 0, policy_version 946471 (0.0008) [2023-12-26 22:15:40,956][105620] Updated weights for policy 1, policy_version 946730 (0.0008) [2023-12-26 22:15:41,015][105620] Updated weights for policy 1, policy_version 946740 (0.0010) [2023-12-26 22:15:41,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19114.7, 300 sec: 19188.7). Total num frames: 484728832. Throughput: 0: 9417.9, 1: 9658.4. Samples: 484741164. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 22:15:41,062][104569] Avg episode reward: [(0, '9260.425'), (1, '8820.908')] [2023-12-26 22:15:41,083][105620] Updated weights for policy 1, policy_version 946750 (0.0011) [2023-12-26 22:15:41,151][105620] Updated weights for policy 1, policy_version 946760 (0.0011) [2023-12-26 22:15:41,491][105692] Updated weights for policy 0, policy_version 946481 (0.0011) [2023-12-26 22:15:41,556][105692] Updated weights for policy 0, policy_version 946491 (0.0011) [2023-12-26 22:15:41,626][105692] Updated weights for policy 0, policy_version 946501 (0.0011) [2023-12-26 22:15:41,987][105620] Updated weights for policy 1, policy_version 946770 (0.0011) [2023-12-26 22:15:42,047][105620] Updated weights for policy 1, policy_version 946780 (0.0011) [2023-12-26 22:15:42,115][105620] Updated weights for policy 1, policy_version 946790 (0.0011) [2023-12-26 22:15:42,312][105692] Updated weights for policy 0, policy_version 946511 (0.0008) [2023-12-26 22:15:42,381][105692] Updated weights for policy 0, policy_version 946521 (0.0009) [2023-12-26 22:15:42,447][105692] Updated weights for policy 0, policy_version 946531 (0.0008) [2023-12-26 22:15:42,864][105620] Updated weights for policy 1, policy_version 946800 (0.0008) [2023-12-26 22:15:42,925][105620] Updated weights for policy 1, policy_version 946810 (0.0007) [2023-12-26 22:15:42,983][105620] Updated weights for policy 1, policy_version 946820 (0.0009) [2023-12-26 22:15:43,109][105692] Updated weights for policy 0, policy_version 946541 (0.0009) [2023-12-26 22:15:43,171][105692] Updated weights for policy 0, policy_version 946551 (0.0010) [2023-12-26 22:15:43,234][105692] Updated weights for policy 0, policy_version 946561 (0.0010) [2023-12-26 22:15:43,752][105620] Updated weights for policy 1, policy_version 946830 (0.0006) [2023-12-26 22:15:43,816][105620] Updated weights for policy 1, policy_version 946840 (0.0005) [2023-12-26 22:15:43,870][105620] Updated weights for policy 1, policy_version 946850 (0.0005) [2023-12-26 22:15:43,967][105692] Updated weights for policy 0, policy_version 946571 (0.0006) [2023-12-26 22:15:44,034][105692] Updated weights for policy 0, policy_version 946581 (0.0008) [2023-12-26 22:15:44,100][105692] Updated weights for policy 0, policy_version 946591 (0.0010) [2023-12-26 22:15:44,534][105620] Updated weights for policy 1, policy_version 946860 (0.0006) [2023-12-26 22:15:44,581][105620] Updated weights for policy 1, policy_version 946870 (0.0007) [2023-12-26 22:15:44,633][105620] Updated weights for policy 1, policy_version 946880 (0.0008) [2023-12-26 22:15:44,771][105692] Updated weights for policy 0, policy_version 946601 (0.0010) [2023-12-26 22:15:44,836][105692] Updated weights for policy 0, policy_version 946611 (0.0006) [2023-12-26 22:15:44,895][105692] Updated weights for policy 0, policy_version 946621 (0.0008) [2023-12-26 22:15:44,959][105692] Updated weights for policy 0, policy_version 946631 (0.0006) [2023-12-26 22:15:45,397][105620] Updated weights for policy 1, policy_version 946890 (0.0008) [2023-12-26 22:15:45,464][105620] Updated weights for policy 1, policy_version 946900 (0.0008) [2023-12-26 22:15:45,531][105620] Updated weights for policy 1, policy_version 946910 (0.0009) [2023-12-26 22:15:45,594][105620] Updated weights for policy 1, policy_version 946920 (0.0008) [2023-12-26 22:15:45,684][105692] Updated weights for policy 0, policy_version 946641 (0.0008) [2023-12-26 22:15:45,734][105692] Updated weights for policy 0, policy_version 946651 (0.0007) [2023-12-26 22:15:45,786][105692] Updated weights for policy 0, policy_version 946661 (0.0006) [2023-12-26 22:15:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.8, 300 sec: 19188.7). Total num frames: 484827136. Throughput: 0: 9462.6, 1: 9599.8. Samples: 484797056. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 22:15:46,062][104569] Avg episode reward: [(0, '9260.211'), (1, '8820.483')] [2023-12-26 22:15:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000946664_242384896.pth... [2023-12-26 22:15:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000946920_242442240.pth... [2023-12-26 22:15:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000945800_242155520.pth [2023-12-26 22:15:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000945584_242106368.pth [2023-12-26 22:15:46,369][105620] Updated weights for policy 1, policy_version 946930 (0.0009) [2023-12-26 22:15:46,418][105620] Updated weights for policy 1, policy_version 946940 (0.0008) [2023-12-26 22:15:46,449][105692] Updated weights for policy 0, policy_version 946671 (0.0009) [2023-12-26 22:15:46,478][105620] Updated weights for policy 1, policy_version 946950 (0.0009) [2023-12-26 22:15:46,502][105692] Updated weights for policy 0, policy_version 946681 (0.0007) [2023-12-26 22:15:46,555][105692] Updated weights for policy 0, policy_version 946691 (0.0009) [2023-12-26 22:15:47,154][105692] Updated weights for policy 0, policy_version 946702 (0.0007) [2023-12-26 22:15:47,212][105692] Updated weights for policy 0, policy_version 946712 (0.0008) [2023-12-26 22:15:47,245][105620] Updated weights for policy 1, policy_version 946960 (0.0009) [2023-12-26 22:15:47,268][105692] Updated weights for policy 0, policy_version 946722 (0.0006) [2023-12-26 22:15:47,299][105620] Updated weights for policy 1, policy_version 946970 (0.0007) [2023-12-26 22:15:47,358][105620] Updated weights for policy 1, policy_version 946980 (0.0008) [2023-12-26 22:15:47,977][105692] Updated weights for policy 0, policy_version 946732 (0.0008) [2023-12-26 22:15:48,042][105692] Updated weights for policy 0, policy_version 946742 (0.0006) [2023-12-26 22:15:48,097][105692] Updated weights for policy 0, policy_version 946752 (0.0006) [2023-12-26 22:15:48,143][105620] Updated weights for policy 1, policy_version 946990 (0.0008) [2023-12-26 22:15:48,204][105620] Updated weights for policy 1, policy_version 947000 (0.0008) [2023-12-26 22:15:48,266][105620] Updated weights for policy 1, policy_version 947010 (0.0010) [2023-12-26 22:15:48,806][105692] Updated weights for policy 0, policy_version 946762 (0.0008) [2023-12-26 22:15:48,858][105692] Updated weights for policy 0, policy_version 946772 (0.0008) [2023-12-26 22:15:48,906][105692] Updated weights for policy 0, policy_version 946782 (0.0008) [2023-12-26 22:15:48,962][105692] Updated weights for policy 0, policy_version 946792 (0.0009) [2023-12-26 22:15:49,008][105620] Updated weights for policy 1, policy_version 947020 (0.0008) [2023-12-26 22:15:49,073][105620] Updated weights for policy 1, policy_version 947030 (0.0008) [2023-12-26 22:15:49,127][105620] Updated weights for policy 1, policy_version 947040 (0.0008) [2023-12-26 22:15:49,833][105620] Updated weights for policy 1, policy_version 947050 (0.0008) [2023-12-26 22:15:49,895][105620] Updated weights for policy 1, policy_version 947060 (0.0008) [2023-12-26 22:15:49,928][105692] Updated weights for policy 0, policy_version 946802 (0.0008) [2023-12-26 22:15:49,966][105620] Updated weights for policy 1, policy_version 947070 (0.0008) [2023-12-26 22:15:49,994][105692] Updated weights for policy 0, policy_version 946812 (0.0008) [2023-12-26 22:15:50,022][105620] Updated weights for policy 1, policy_version 947080 (0.0007) [2023-12-26 22:15:50,047][105692] Updated weights for policy 0, policy_version 946822 (0.0008) [2023-12-26 22:15:50,764][105692] Updated weights for policy 0, policy_version 946832 (0.0007) [2023-12-26 22:15:50,823][105692] Updated weights for policy 0, policy_version 946842 (0.0007) [2023-12-26 22:15:50,866][105620] Updated weights for policy 1, policy_version 947090 (0.0008) [2023-12-26 22:15:50,889][105692] Updated weights for policy 0, policy_version 946852 (0.0009) [2023-12-26 22:15:50,931][105620] Updated weights for policy 1, policy_version 947101 (0.0011) [2023-12-26 22:15:50,994][105620] Updated weights for policy 1, policy_version 947111 (0.0009) [2023-12-26 22:15:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19114.7, 300 sec: 19216.5). Total num frames: 484925440. Throughput: 0: 9520.3, 1: 9555.6. Samples: 484911368. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 22:15:51,063][104569] Avg episode reward: [(0, '9260.896'), (1, '9003.324')] [2023-12-26 22:15:51,752][105692] Updated weights for policy 0, policy_version 946862 (0.0008) [2023-12-26 22:15:51,814][105692] Updated weights for policy 0, policy_version 946872 (0.0009) [2023-12-26 22:15:51,875][105692] Updated weights for policy 0, policy_version 946882 (0.0008) [2023-12-26 22:15:51,906][105620] Updated weights for policy 1, policy_version 947121 (0.0008) [2023-12-26 22:15:51,971][105620] Updated weights for policy 1, policy_version 947131 (0.0009) [2023-12-26 22:15:52,038][105620] Updated weights for policy 1, policy_version 947141 (0.0009) [2023-12-26 22:15:52,646][105692] Updated weights for policy 0, policy_version 946892 (0.0007) [2023-12-26 22:15:52,700][105692] Updated weights for policy 0, policy_version 946902 (0.0009) [2023-12-26 22:15:52,759][105692] Updated weights for policy 0, policy_version 946912 (0.0009) [2023-12-26 22:15:52,840][105620] Updated weights for policy 1, policy_version 947151 (0.0009) [2023-12-26 22:15:52,896][105620] Updated weights for policy 1, policy_version 947161 (0.0008) [2023-12-26 22:15:52,952][105620] Updated weights for policy 1, policy_version 947171 (0.0009) [2023-12-26 22:15:53,537][105692] Updated weights for policy 0, policy_version 946922 (0.0009) [2023-12-26 22:15:53,600][105692] Updated weights for policy 0, policy_version 946932 (0.0010) [2023-12-26 22:15:53,657][105692] Updated weights for policy 0, policy_version 946942 (0.0009) [2023-12-26 22:15:53,711][105620] Updated weights for policy 1, policy_version 947181 (0.0009) [2023-12-26 22:15:53,720][105692] Updated weights for policy 0, policy_version 946952 (0.0009) [2023-12-26 22:15:53,773][105620] Updated weights for policy 1, policy_version 947191 (0.0009) [2023-12-26 22:15:53,837][105620] Updated weights for policy 1, policy_version 947201 (0.0009) [2023-12-26 22:15:54,479][105692] Updated weights for policy 0, policy_version 946962 (0.0009) [2023-12-26 22:15:54,541][105692] Updated weights for policy 0, policy_version 946972 (0.0009) [2023-12-26 22:15:54,604][105692] Updated weights for policy 0, policy_version 946982 (0.0009) [2023-12-26 22:15:54,613][105620] Updated weights for policy 1, policy_version 947211 (0.0008) [2023-12-26 22:15:54,666][105620] Updated weights for policy 1, policy_version 947221 (0.0009) [2023-12-26 22:15:54,721][105620] Updated weights for policy 1, policy_version 947231 (0.0009) [2023-12-26 22:15:55,334][105692] Updated weights for policy 0, policy_version 946992 (0.0011) [2023-12-26 22:15:55,394][105692] Updated weights for policy 0, policy_version 947002 (0.0011) [2023-12-26 22:15:55,459][105620] Updated weights for policy 1, policy_version 947241 (0.0009) [2023-12-26 22:15:55,461][105692] Updated weights for policy 0, policy_version 947012 (0.0011) [2023-12-26 22:15:55,523][105620] Updated weights for policy 1, policy_version 947251 (0.0011) [2023-12-26 22:15:55,592][105620] Updated weights for policy 1, policy_version 947261 (0.0011) [2023-12-26 22:15:55,662][105620] Updated weights for policy 1, policy_version 947271 (0.0011) [2023-12-26 22:15:56,062][104569] Fps is (10 sec: 18022.4, 60 sec: 18978.1, 300 sec: 19188.7). Total num frames: 485007360. Throughput: 0: 9430.7, 1: 9493.0. Samples: 485019452. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:15:56,063][104569] Avg episode reward: [(0, '8668.789'), (1, '9266.615')] [2023-12-26 22:15:56,096][105692] Updated weights for policy 0, policy_version 947022 (0.0010) [2023-12-26 22:15:56,159][105692] Updated weights for policy 0, policy_version 947032 (0.0010) [2023-12-26 22:15:56,224][105692] Updated weights for policy 0, policy_version 947042 (0.0008) [2023-12-26 22:15:56,264][105620] Updated weights for policy 1, policy_version 947281 (0.0006) [2023-12-26 22:15:56,330][105620] Updated weights for policy 1, policy_version 947291 (0.0009) [2023-12-26 22:15:56,390][105620] Updated weights for policy 1, policy_version 947301 (0.0011) [2023-12-26 22:15:56,876][105692] Updated weights for policy 0, policy_version 947052 (0.0010) [2023-12-26 22:15:56,925][105692] Updated weights for policy 0, policy_version 947062 (0.0005) [2023-12-26 22:15:56,982][105692] Updated weights for policy 0, policy_version 947072 (0.0005) [2023-12-26 22:15:57,072][105620] Updated weights for policy 1, policy_version 947311 (0.0007) [2023-12-26 22:15:57,119][105620] Updated weights for policy 1, policy_version 947321 (0.0005) [2023-12-26 22:15:57,164][105620] Updated weights for policy 1, policy_version 947331 (0.0005) [2023-12-26 22:15:57,657][105692] Updated weights for policy 0, policy_version 947082 (0.0007) [2023-12-26 22:15:57,704][105620] Updated weights for policy 1, policy_version 947341 (0.0007) [2023-12-26 22:15:57,717][105692] Updated weights for policy 0, policy_version 947092 (0.0005) [2023-12-26 22:15:57,760][105620] Updated weights for policy 1, policy_version 947351 (0.0010) [2023-12-26 22:15:57,779][105692] Updated weights for policy 0, policy_version 947102 (0.0005) [2023-12-26 22:15:57,811][105620] Updated weights for policy 1, policy_version 947361 (0.0010) [2023-12-26 22:15:57,843][105692] Updated weights for policy 0, policy_version 947112 (0.0005) [2023-12-26 22:15:58,538][105620] Updated weights for policy 1, policy_version 947371 (0.0010) [2023-12-26 22:15:58,566][105692] Updated weights for policy 0, policy_version 947122 (0.0009) [2023-12-26 22:15:58,609][105620] Updated weights for policy 1, policy_version 947381 (0.0007) [2023-12-26 22:15:58,631][105692] Updated weights for policy 0, policy_version 947132 (0.0009) [2023-12-26 22:15:58,640][105586] KL-divergence is very high: 105.4155 [2023-12-26 22:15:58,670][105620] Updated weights for policy 1, policy_version 947391 (0.0006) [2023-12-26 22:15:58,693][105586] KL-divergence is very high: 116.8280 [2023-12-26 22:15:58,694][105692] Updated weights for policy 0, policy_version 947142 (0.0008) [2023-12-26 22:15:59,470][105620] Updated weights for policy 1, policy_version 947401 (0.0007) [2023-12-26 22:15:59,477][105692] Updated weights for policy 0, policy_version 947152 (0.0009) [2023-12-26 22:15:59,531][105620] Updated weights for policy 1, policy_version 947411 (0.0008) [2023-12-26 22:15:59,533][105692] Updated weights for policy 0, policy_version 947162 (0.0006) [2023-12-26 22:15:59,591][105620] Updated weights for policy 1, policy_version 947421 (0.0010) [2023-12-26 22:15:59,597][105692] Updated weights for policy 0, policy_version 947172 (0.0006) [2023-12-26 22:15:59,650][105620] Updated weights for policy 1, policy_version 947431 (0.0010) [2023-12-26 22:16:00,339][105620] Updated weights for policy 1, policy_version 947441 (0.0010) [2023-12-26 22:16:00,341][105692] Updated weights for policy 0, policy_version 947182 (0.0007) [2023-12-26 22:16:00,395][105620] Updated weights for policy 1, policy_version 947451 (0.0010) [2023-12-26 22:16:00,398][105692] Updated weights for policy 0, policy_version 947192 (0.0006) [2023-12-26 22:16:00,452][105620] Updated weights for policy 1, policy_version 947461 (0.0010) [2023-12-26 22:16:00,455][105692] Updated weights for policy 0, policy_version 947202 (0.0005) [2023-12-26 22:16:01,062][104569] Fps is (10 sec: 18022.5, 60 sec: 18978.1, 300 sec: 19216.5). Total num frames: 485105664. Throughput: 0: 9422.1, 1: 9546.5. Samples: 485081036. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:16:01,062][104569] Avg episode reward: [(0, '8583.379'), (1, '9178.100')] [2023-12-26 22:16:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000947464_242581504.pth... [2023-12-26 22:16:01,072][105692] Updated weights for policy 0, policy_version 947212 (0.0007) [2023-12-26 22:16:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000946376_242302976.pth [2023-12-26 22:16:01,147][105692] Updated weights for policy 0, policy_version 947222 (0.0009) [2023-12-26 22:16:01,199][105620] Updated weights for policy 1, policy_version 947471 (0.0009) [2023-12-26 22:16:01,205][105692] Updated weights for policy 0, policy_version 947232 (0.0007) [2023-12-26 22:16:01,250][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000947240_242532352.pth... [2023-12-26 22:16:01,254][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000946128_242245632.pth [2023-12-26 22:16:01,261][105620] Updated weights for policy 1, policy_version 947481 (0.0009) [2023-12-26 22:16:01,316][105620] Updated weights for policy 1, policy_version 947491 (0.0009) [2023-12-26 22:16:02,015][105620] Updated weights for policy 1, policy_version 947501 (0.0009) [2023-12-26 22:16:02,044][105692] Updated weights for policy 0, policy_version 947242 (0.0007) [2023-12-26 22:16:02,078][105620] Updated weights for policy 1, policy_version 947511 (0.0007) [2023-12-26 22:16:02,101][105692] Updated weights for policy 0, policy_version 947252 (0.0006) [2023-12-26 22:16:02,132][105620] Updated weights for policy 1, policy_version 947521 (0.0007) [2023-12-26 22:16:02,159][105692] Updated weights for policy 0, policy_version 947262 (0.0006) [2023-12-26 22:16:02,219][105692] Updated weights for policy 0, policy_version 947272 (0.0006) [2023-12-26 22:16:02,818][105692] Updated weights for policy 0, policy_version 947282 (0.0009) [2023-12-26 22:16:02,865][105692] Updated weights for policy 0, policy_version 947292 (0.0009) [2023-12-26 22:16:02,912][105692] Updated weights for policy 0, policy_version 947302 (0.0008) [2023-12-26 22:16:02,932][105620] Updated weights for policy 1, policy_version 947531 (0.0007) [2023-12-26 22:16:02,985][105620] Updated weights for policy 1, policy_version 947541 (0.0008) [2023-12-26 22:16:03,043][105620] Updated weights for policy 1, policy_version 947551 (0.0009) [2023-12-26 22:16:03,609][105692] Updated weights for policy 0, policy_version 947312 (0.0007) [2023-12-26 22:16:03,671][105692] Updated weights for policy 0, policy_version 947322 (0.0005) [2023-12-26 22:16:03,729][105692] Updated weights for policy 0, policy_version 947332 (0.0005) [2023-12-26 22:16:03,886][105620] Updated weights for policy 1, policy_version 947561 (0.0009) [2023-12-26 22:16:03,951][105620] Updated weights for policy 1, policy_version 947571 (0.0008) [2023-12-26 22:16:04,015][105620] Updated weights for policy 1, policy_version 947581 (0.0009) [2023-12-26 22:16:04,074][105620] Updated weights for policy 1, policy_version 947591 (0.0010) [2023-12-26 22:16:04,381][105692] Updated weights for policy 0, policy_version 947342 (0.0006) [2023-12-26 22:16:04,449][105692] Updated weights for policy 0, policy_version 947352 (0.0006) [2023-12-26 22:16:04,509][105692] Updated weights for policy 0, policy_version 947362 (0.0009) [2023-12-26 22:16:04,868][105620] Updated weights for policy 1, policy_version 947601 (0.0009) [2023-12-26 22:16:04,932][105620] Updated weights for policy 1, policy_version 947611 (0.0010) [2023-12-26 22:16:04,994][105620] Updated weights for policy 1, policy_version 947621 (0.0010) [2023-12-26 22:16:05,133][105692] Updated weights for policy 0, policy_version 947372 (0.0008) [2023-12-26 22:16:05,188][105692] Updated weights for policy 0, policy_version 947382 (0.0006) [2023-12-26 22:16:05,235][105692] Updated weights for policy 0, policy_version 947392 (0.0009) [2023-12-26 22:16:05,716][105620] Updated weights for policy 1, policy_version 947631 (0.0007) [2023-12-26 22:16:05,772][105620] Updated weights for policy 1, policy_version 947641 (0.0005) [2023-12-26 22:16:05,818][105620] Updated weights for policy 1, policy_version 947651 (0.0005) [2023-12-26 22:16:06,036][105692] Updated weights for policy 0, policy_version 947402 (0.0009) [2023-12-26 22:16:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19114.7, 300 sec: 19216.5). Total num frames: 485203968. Throughput: 0: 9524.4, 1: 9450.3. Samples: 485195260. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:16:06,063][104569] Avg episode reward: [(0, '8675.058'), (1, '9094.551')] [2023-12-26 22:16:06,101][105692] Updated weights for policy 0, policy_version 947412 (0.0010) [2023-12-26 22:16:06,160][105692] Updated weights for policy 0, policy_version 947422 (0.0008) [2023-12-26 22:16:06,223][105692] Updated weights for policy 0, policy_version 947432 (0.0006) [2023-12-26 22:16:06,503][105620] Updated weights for policy 1, policy_version 947661 (0.0006) [2023-12-26 22:16:06,571][105620] Updated weights for policy 1, policy_version 947671 (0.0005) [2023-12-26 22:16:06,634][105586] KL-divergence is very high: 102.7318 [2023-12-26 22:16:06,641][105620] Updated weights for policy 1, policy_version 947681 (0.0005) [2023-12-26 22:16:06,925][105692] Updated weights for policy 0, policy_version 947442 (0.0008) [2023-12-26 22:16:06,991][105692] Updated weights for policy 0, policy_version 947452 (0.0008) [2023-12-26 22:16:07,048][105692] Updated weights for policy 0, policy_version 947462 (0.0008) [2023-12-26 22:16:07,308][105620] Updated weights for policy 1, policy_version 947691 (0.0011) [2023-12-26 22:16:07,372][105620] Updated weights for policy 1, policy_version 947701 (0.0011) [2023-12-26 22:16:07,438][105620] Updated weights for policy 1, policy_version 947711 (0.0011) [2023-12-26 22:16:07,846][105692] Updated weights for policy 0, policy_version 947472 (0.0010) [2023-12-26 22:16:07,886][105585] KL-divergence is very high: 105.0723 [2023-12-26 22:16:07,903][105692] Updated weights for policy 0, policy_version 947482 (0.0010) [2023-12-26 22:16:07,929][105585] KL-divergence is very high: 178.9830 [2023-12-26 22:16:07,951][105692] Updated weights for policy 0, policy_version 947492 (0.0010) [2023-12-26 22:16:07,965][105585] KL-divergence is very high: 170.1253 [2023-12-26 22:16:08,131][105620] Updated weights for policy 1, policy_version 947721 (0.0007) [2023-12-26 22:16:08,193][105620] Updated weights for policy 1, policy_version 947731 (0.0010) [2023-12-26 22:16:08,245][105620] Updated weights for policy 1, policy_version 947741 (0.0010) [2023-12-26 22:16:08,297][105620] Updated weights for policy 1, policy_version 947751 (0.0010) [2023-12-26 22:16:08,607][105585] KL-divergence is very high: 204.1292 [2023-12-26 22:16:08,636][105585] KL-divergence is very high: 172.1349 [2023-12-26 22:16:08,648][105692] Updated weights for policy 0, policy_version 947502 (0.0007) [2023-12-26 22:16:08,661][105585] KL-divergence is very high: 145.9937 [2023-12-26 22:16:08,686][105585] KL-divergence is very high: 119.9143 [2023-12-26 22:16:08,714][105585] KL-divergence is very high: 103.3793 [2023-12-26 22:16:08,714][105692] Updated weights for policy 0, policy_version 947512 (0.0006) [2023-12-26 22:16:08,773][105692] Updated weights for policy 0, policy_version 947522 (0.0010) [2023-12-26 22:16:09,067][105620] Updated weights for policy 1, policy_version 947761 (0.0011) [2023-12-26 22:16:09,135][105620] Updated weights for policy 1, policy_version 947771 (0.0010) [2023-12-26 22:16:09,187][105620] Updated weights for policy 1, policy_version 947781 (0.0008) [2023-12-26 22:16:09,363][105692] Updated weights for policy 0, policy_version 947532 (0.0007) [2023-12-26 22:16:09,435][105692] Updated weights for policy 0, policy_version 947542 (0.0009) [2023-12-26 22:16:09,494][105692] Updated weights for policy 0, policy_version 947552 (0.0006) [2023-12-26 22:16:09,955][105620] Updated weights for policy 1, policy_version 947791 (0.0008) [2023-12-26 22:16:10,007][105620] Updated weights for policy 1, policy_version 947801 (0.0008) [2023-12-26 22:16:10,077][105620] Updated weights for policy 1, policy_version 947811 (0.0008) [2023-12-26 22:16:10,116][105692] Updated weights for policy 0, policy_version 947562 (0.0007) [2023-12-26 22:16:10,181][105692] Updated weights for policy 0, policy_version 947572 (0.0011) [2023-12-26 22:16:10,241][105692] Updated weights for policy 0, policy_version 947582 (0.0011) [2023-12-26 22:16:10,298][105692] Updated weights for policy 0, policy_version 947592 (0.0009) [2023-12-26 22:16:10,763][105620] Updated weights for policy 1, policy_version 947821 (0.0009) [2023-12-26 22:16:10,826][105620] Updated weights for policy 1, policy_version 947831 (0.0011) [2023-12-26 22:16:10,891][105620] Updated weights for policy 1, policy_version 947841 (0.0011) [2023-12-26 22:16:10,978][105692] Updated weights for policy 0, policy_version 947602 (0.0011) [2023-12-26 22:16:11,043][105692] Updated weights for policy 0, policy_version 947612 (0.0010) [2023-12-26 22:16:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 18978.2, 300 sec: 19216.5). Total num frames: 485302272. Throughput: 0: 9654.7, 1: 9423.6. Samples: 485313316. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:16:11,062][104569] Avg episode reward: [(0, '8824.468'), (1, '9182.943')] [2023-12-26 22:16:11,115][105692] Updated weights for policy 0, policy_version 947622 (0.0009) [2023-12-26 22:16:11,673][105620] Updated weights for policy 1, policy_version 947851 (0.0010) [2023-12-26 22:16:11,736][105620] Updated weights for policy 1, policy_version 947861 (0.0008) [2023-12-26 22:16:11,802][105620] Updated weights for policy 1, policy_version 947871 (0.0009) [2023-12-26 22:16:11,898][105692] Updated weights for policy 0, policy_version 947632 (0.0007) [2023-12-26 22:16:11,971][105692] Updated weights for policy 0, policy_version 947642 (0.0008) [2023-12-26 22:16:12,043][105692] Updated weights for policy 0, policy_version 947652 (0.0006) [2023-12-26 22:16:12,638][105620] Updated weights for policy 1, policy_version 947881 (0.0006) [2023-12-26 22:16:12,673][105692] Updated weights for policy 0, policy_version 947662 (0.0007) [2023-12-26 22:16:12,697][105620] Updated weights for policy 1, policy_version 947891 (0.0010) [2023-12-26 22:16:12,735][105692] Updated weights for policy 0, policy_version 947672 (0.0006) [2023-12-26 22:16:12,758][105620] Updated weights for policy 1, policy_version 947901 (0.0011) [2023-12-26 22:16:12,796][105692] Updated weights for policy 0, policy_version 947682 (0.0009) [2023-12-26 22:16:12,818][105620] Updated weights for policy 1, policy_version 947911 (0.0011) [2023-12-26 22:16:13,419][105692] Updated weights for policy 0, policy_version 947692 (0.0008) [2023-12-26 22:16:13,477][105692] Updated weights for policy 0, policy_version 947702 (0.0009) [2023-12-26 22:16:13,539][105692] Updated weights for policy 0, policy_version 947712 (0.0009) [2023-12-26 22:16:13,644][105620] Updated weights for policy 1, policy_version 947921 (0.0010) [2023-12-26 22:16:13,707][105620] Updated weights for policy 1, policy_version 947931 (0.0011) [2023-12-26 22:16:13,770][105620] Updated weights for policy 1, policy_version 947941 (0.0011) [2023-12-26 22:16:14,334][105692] Updated weights for policy 0, policy_version 947722 (0.0009) [2023-12-26 22:16:14,395][105692] Updated weights for policy 0, policy_version 947732 (0.0008) [2023-12-26 22:16:14,461][105692] Updated weights for policy 0, policy_version 947742 (0.0008) [2023-12-26 22:16:14,521][105620] Updated weights for policy 1, policy_version 947951 (0.0010) [2023-12-26 22:16:14,523][105692] Updated weights for policy 0, policy_version 947752 (0.0006) [2023-12-26 22:16:14,580][105620] Updated weights for policy 1, policy_version 947961 (0.0010) [2023-12-26 22:16:14,643][105620] Updated weights for policy 1, policy_version 947971 (0.0010) [2023-12-26 22:16:15,265][105692] Updated weights for policy 0, policy_version 947762 (0.0009) [2023-12-26 22:16:15,325][105692] Updated weights for policy 0, policy_version 947772 (0.0008) [2023-12-26 22:16:15,392][105692] Updated weights for policy 0, policy_version 947782 (0.0008) [2023-12-26 22:16:15,398][105620] Updated weights for policy 1, policy_version 947981 (0.0011) [2023-12-26 22:16:15,458][105620] Updated weights for policy 1, policy_version 947991 (0.0011) [2023-12-26 22:16:15,523][105620] Updated weights for policy 1, policy_version 948001 (0.0010) [2023-12-26 22:16:16,062][104569] Fps is (10 sec: 18840.8, 60 sec: 18978.0, 300 sec: 19216.5). Total num frames: 485392384. Throughput: 0: 9663.0, 1: 9354.5. Samples: 485368544. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:16:16,063][104569] Avg episode reward: [(0, '9080.551'), (1, '8913.127')] [2023-12-26 22:16:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000947784_242671616.pth... [2023-12-26 22:16:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000948008_242720768.pth... [2023-12-26 22:16:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000946920_242442240.pth [2023-12-26 22:16:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000946664_242384896.pth [2023-12-26 22:16:16,151][105692] Updated weights for policy 0, policy_version 947792 (0.0010) [2023-12-26 22:16:16,207][105692] Updated weights for policy 0, policy_version 947802 (0.0010) [2023-12-26 22:16:16,251][105620] Updated weights for policy 1, policy_version 948011 (0.0010) [2023-12-26 22:16:16,267][105692] Updated weights for policy 0, policy_version 947812 (0.0011) [2023-12-26 22:16:16,314][105620] Updated weights for policy 1, policy_version 948021 (0.0008) [2023-12-26 22:16:16,380][105620] Updated weights for policy 1, policy_version 948031 (0.0009) [2023-12-26 22:16:16,979][105692] Updated weights for policy 0, policy_version 947822 (0.0008) [2023-12-26 22:16:17,026][105692] Updated weights for policy 0, policy_version 947832 (0.0005) [2023-12-26 22:16:17,043][105620] Updated weights for policy 1, policy_version 948041 (0.0008) [2023-12-26 22:16:17,085][105692] Updated weights for policy 0, policy_version 947842 (0.0007) [2023-12-26 22:16:17,091][105620] Updated weights for policy 1, policy_version 948051 (0.0010) [2023-12-26 22:16:17,150][105620] Updated weights for policy 1, policy_version 948061 (0.0010) [2023-12-26 22:16:17,215][105620] Updated weights for policy 1, policy_version 948071 (0.0011) [2023-12-26 22:16:17,696][105692] Updated weights for policy 0, policy_version 947852 (0.0008) [2023-12-26 22:16:17,761][105692] Updated weights for policy 0, policy_version 947862 (0.0010) [2023-12-26 22:16:17,825][105692] Updated weights for policy 0, policy_version 947872 (0.0010) [2023-12-26 22:16:17,967][105620] Updated weights for policy 1, policy_version 948081 (0.0010) [2023-12-26 22:16:18,032][105620] Updated weights for policy 1, policy_version 948091 (0.0009) [2023-12-26 22:16:18,097][105620] Updated weights for policy 1, policy_version 948101 (0.0007) [2023-12-26 22:16:18,569][105692] Updated weights for policy 0, policy_version 947882 (0.0010) [2023-12-26 22:16:18,616][105692] Updated weights for policy 0, policy_version 947892 (0.0008) [2023-12-26 22:16:18,676][105692] Updated weights for policy 0, policy_version 947902 (0.0008) [2023-12-26 22:16:18,710][105620] Updated weights for policy 1, policy_version 948111 (0.0007) [2023-12-26 22:16:18,736][105692] Updated weights for policy 0, policy_version 947912 (0.0006) [2023-12-26 22:16:18,771][105620] Updated weights for policy 1, policy_version 948121 (0.0010) [2023-12-26 22:16:18,836][105620] Updated weights for policy 1, policy_version 948131 (0.0010) [2023-12-26 22:16:19,501][105692] Updated weights for policy 0, policy_version 947922 (0.0009) [2023-12-26 22:16:19,560][105692] Updated weights for policy 0, policy_version 947932 (0.0009) [2023-12-26 22:16:19,567][105620] Updated weights for policy 1, policy_version 948141 (0.0010) [2023-12-26 22:16:19,614][105692] Updated weights for policy 0, policy_version 947942 (0.0006) [2023-12-26 22:16:19,627][105620] Updated weights for policy 1, policy_version 948151 (0.0011) [2023-12-26 22:16:19,688][105620] Updated weights for policy 1, policy_version 948161 (0.0011) [2023-12-26 22:16:20,397][105692] Updated weights for policy 0, policy_version 947952 (0.0009) [2023-12-26 22:16:20,430][105620] Updated weights for policy 1, policy_version 948171 (0.0007) [2023-12-26 22:16:20,462][105692] Updated weights for policy 0, policy_version 947962 (0.0008) [2023-12-26 22:16:20,495][105620] Updated weights for policy 1, policy_version 948181 (0.0007) [2023-12-26 22:16:20,530][105692] Updated weights for policy 0, policy_version 947972 (0.0008) [2023-12-26 22:16:20,558][105620] Updated weights for policy 1, policy_version 948191 (0.0008) [2023-12-26 22:16:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18978.2, 300 sec: 19216.5). Total num frames: 485490688. Throughput: 0: 9599.7, 1: 9368.5. Samples: 485483404. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:16:21,062][104569] Avg episode reward: [(0, '9258.347'), (1, '8822.078')] [2023-12-26 22:16:21,212][105620] Updated weights for policy 1, policy_version 948201 (0.0009) [2023-12-26 22:16:21,279][105620] Updated weights for policy 1, policy_version 948211 (0.0008) [2023-12-26 22:16:21,345][105620] Updated weights for policy 1, policy_version 948221 (0.0009) [2023-12-26 22:16:21,366][105692] Updated weights for policy 0, policy_version 947982 (0.0008) [2023-12-26 22:16:21,415][105620] Updated weights for policy 1, policy_version 948231 (0.0008) [2023-12-26 22:16:21,424][105692] Updated weights for policy 0, policy_version 947992 (0.0011) [2023-12-26 22:16:21,487][105692] Updated weights for policy 0, policy_version 948002 (0.0011) [2023-12-26 22:16:22,089][105620] Updated weights for policy 1, policy_version 948241 (0.0010) [2023-12-26 22:16:22,145][105620] Updated weights for policy 1, policy_version 948251 (0.0009) [2023-12-26 22:16:22,207][105620] Updated weights for policy 1, policy_version 948261 (0.0009) [2023-12-26 22:16:22,359][105692] Updated weights for policy 0, policy_version 948012 (0.0012) [2023-12-26 22:16:22,423][105692] Updated weights for policy 0, policy_version 948022 (0.0009) [2023-12-26 22:16:22,493][105692] Updated weights for policy 0, policy_version 948032 (0.0011) [2023-12-26 22:16:23,021][105620] Updated weights for policy 1, policy_version 948271 (0.0008) [2023-12-26 22:16:23,074][105620] Updated weights for policy 1, policy_version 948281 (0.0008) [2023-12-26 22:16:23,122][105620] Updated weights for policy 1, policy_version 948291 (0.0008) [2023-12-26 22:16:23,236][105692] Updated weights for policy 0, policy_version 948042 (0.0011) [2023-12-26 22:16:23,292][105692] Updated weights for policy 0, policy_version 948052 (0.0011) [2023-12-26 22:16:23,348][105692] Updated weights for policy 0, policy_version 948062 (0.0010) [2023-12-26 22:16:23,402][105692] Updated weights for policy 0, policy_version 948072 (0.0010) [2023-12-26 22:16:23,923][105620] Updated weights for policy 1, policy_version 948301 (0.0007) [2023-12-26 22:16:23,980][105692] Updated weights for policy 0, policy_version 948082 (0.0007) [2023-12-26 22:16:23,990][105620] Updated weights for policy 1, policy_version 948311 (0.0005) [2023-12-26 22:16:24,033][105692] Updated weights for policy 0, policy_version 948092 (0.0006) [2023-12-26 22:16:24,056][105620] Updated weights for policy 1, policy_version 948321 (0.0006) [2023-12-26 22:16:24,092][105692] Updated weights for policy 0, policy_version 948102 (0.0010) [2023-12-26 22:16:24,719][105620] Updated weights for policy 1, policy_version 948331 (0.0007) [2023-12-26 22:16:24,768][105692] Updated weights for policy 0, policy_version 948112 (0.0006) [2023-12-26 22:16:24,775][105620] Updated weights for policy 1, policy_version 948341 (0.0008) [2023-12-26 22:16:24,823][105692] Updated weights for policy 0, policy_version 948122 (0.0006) [2023-12-26 22:16:24,832][105620] Updated weights for policy 1, policy_version 948351 (0.0008) [2023-12-26 22:16:24,883][105692] Updated weights for policy 0, policy_version 948132 (0.0009) [2023-12-26 22:16:25,499][105692] Updated weights for policy 0, policy_version 948142 (0.0010) [2023-12-26 22:16:25,562][105692] Updated weights for policy 0, policy_version 948152 (0.0010) [2023-12-26 22:16:25,617][105692] Updated weights for policy 0, policy_version 948162 (0.0010) [2023-12-26 22:16:25,646][105620] Updated weights for policy 1, policy_version 948361 (0.0006) [2023-12-26 22:16:25,704][105620] Updated weights for policy 1, policy_version 948371 (0.0005) [2023-12-26 22:16:25,764][105620] Updated weights for policy 1, policy_version 948381 (0.0007) [2023-12-26 22:16:25,820][105620] Updated weights for policy 1, policy_version 948391 (0.0008) [2023-12-26 22:16:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19114.6, 300 sec: 19216.5). Total num frames: 485588992. Throughput: 0: 9647.3, 1: 9365.4. Samples: 485596740. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:16:26,063][104569] Avg episode reward: [(0, '9167.577'), (1, '8910.142')] [2023-12-26 22:16:26,303][105692] Updated weights for policy 0, policy_version 948172 (0.0010) [2023-12-26 22:16:26,348][105692] Updated weights for policy 0, policy_version 948182 (0.0010) [2023-12-26 22:16:26,406][105692] Updated weights for policy 0, policy_version 948192 (0.0008) [2023-12-26 22:16:26,574][105620] Updated weights for policy 1, policy_version 948401 (0.0008) [2023-12-26 22:16:26,625][105620] Updated weights for policy 1, policy_version 948411 (0.0008) [2023-12-26 22:16:26,670][105620] Updated weights for policy 1, policy_version 948421 (0.0008) [2023-12-26 22:16:27,043][105692] Updated weights for policy 0, policy_version 948202 (0.0010) [2023-12-26 22:16:27,091][105692] Updated weights for policy 0, policy_version 948212 (0.0005) [2023-12-26 22:16:27,142][105692] Updated weights for policy 0, policy_version 948222 (0.0005) [2023-12-26 22:16:27,190][105692] Updated weights for policy 0, policy_version 948232 (0.0005) [2023-12-26 22:16:27,566][105620] Updated weights for policy 1, policy_version 948431 (0.0009) [2023-12-26 22:16:27,621][105620] Updated weights for policy 1, policy_version 948441 (0.0008) [2023-12-26 22:16:27,678][105620] Updated weights for policy 1, policy_version 948451 (0.0007) [2023-12-26 22:16:27,758][105692] Updated weights for policy 0, policy_version 948242 (0.0011) [2023-12-26 22:16:27,819][105692] Updated weights for policy 0, policy_version 948252 (0.0010) [2023-12-26 22:16:27,874][105692] Updated weights for policy 0, policy_version 948262 (0.0010) [2023-12-26 22:16:28,426][105620] Updated weights for policy 1, policy_version 948461 (0.0007) [2023-12-26 22:16:28,488][105620] Updated weights for policy 1, policy_version 948471 (0.0008) [2023-12-26 22:16:28,551][105620] Updated weights for policy 1, policy_version 948481 (0.0009) [2023-12-26 22:16:28,620][105692] Updated weights for policy 0, policy_version 948272 (0.0010) [2023-12-26 22:16:28,679][105692] Updated weights for policy 0, policy_version 948282 (0.0010) [2023-12-26 22:16:28,745][105692] Updated weights for policy 0, policy_version 948292 (0.0010) [2023-12-26 22:16:29,275][105620] Updated weights for policy 1, policy_version 948491 (0.0008) [2023-12-26 22:16:29,340][105620] Updated weights for policy 1, policy_version 948501 (0.0009) [2023-12-26 22:16:29,402][105620] Updated weights for policy 1, policy_version 948511 (0.0008) [2023-12-26 22:16:29,463][105692] Updated weights for policy 0, policy_version 948302 (0.0008) [2023-12-26 22:16:29,521][105692] Updated weights for policy 0, policy_version 948312 (0.0011) [2023-12-26 22:16:29,570][105692] Updated weights for policy 0, policy_version 948322 (0.0011) [2023-12-26 22:16:30,211][105620] Updated weights for policy 1, policy_version 948521 (0.0010) [2023-12-26 22:16:30,235][105692] Updated weights for policy 0, policy_version 948332 (0.0008) [2023-12-26 22:16:30,271][105620] Updated weights for policy 1, policy_version 948531 (0.0009) [2023-12-26 22:16:30,292][105692] Updated weights for policy 0, policy_version 948342 (0.0008) [2023-12-26 22:16:30,319][105620] Updated weights for policy 1, policy_version 948541 (0.0007) [2023-12-26 22:16:30,353][105692] Updated weights for policy 0, policy_version 948352 (0.0008) [2023-12-26 22:16:30,368][105620] Updated weights for policy 1, policy_version 948551 (0.0007) [2023-12-26 22:16:30,965][105692] Updated weights for policy 0, policy_version 948362 (0.0009) [2023-12-26 22:16:31,020][105692] Updated weights for policy 0, policy_version 948372 (0.0006) [2023-12-26 22:16:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 18978.1, 300 sec: 19216.5). Total num frames: 485679104. Throughput: 0: 9723.3, 1: 9348.5. Samples: 485655288. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:16:31,063][104569] Avg episode reward: [(0, '9259.535'), (1, '8833.912')] [2023-12-26 22:16:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000948552_242860032.pth... [2023-12-26 22:16:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000947464_242581504.pth [2023-12-26 22:16:31,086][105692] Updated weights for policy 0, policy_version 948382 (0.0009) [2023-12-26 22:16:31,148][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000948392_242827264.pth... [2023-12-26 22:16:31,151][105692] Updated weights for policy 0, policy_version 948392 (0.0007) [2023-12-26 22:16:31,153][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000947240_242532352.pth [2023-12-26 22:16:31,218][105620] Updated weights for policy 1, policy_version 948561 (0.0009) [2023-12-26 22:16:31,297][105620] Updated weights for policy 1, policy_version 948572 (0.0009) [2023-12-26 22:16:31,358][105620] Updated weights for policy 1, policy_version 948582 (0.0008) [2023-12-26 22:16:31,814][105692] Updated weights for policy 0, policy_version 948402 (0.0011) [2023-12-26 22:16:31,862][105692] Updated weights for policy 0, policy_version 948412 (0.0010) [2023-12-26 22:16:31,913][105692] Updated weights for policy 0, policy_version 948422 (0.0010) [2023-12-26 22:16:32,038][105620] Updated weights for policy 1, policy_version 948592 (0.0008) [2023-12-26 22:16:32,098][105620] Updated weights for policy 1, policy_version 948602 (0.0009) [2023-12-26 22:16:32,165][105620] Updated weights for policy 1, policy_version 948612 (0.0008) [2023-12-26 22:16:32,622][105692] Updated weights for policy 0, policy_version 948432 (0.0006) [2023-12-26 22:16:32,686][105692] Updated weights for policy 0, policy_version 948442 (0.0006) [2023-12-26 22:16:32,742][105692] Updated weights for policy 0, policy_version 948452 (0.0010) [2023-12-26 22:16:32,881][105620] Updated weights for policy 1, policy_version 948622 (0.0006) [2023-12-26 22:16:32,944][105620] Updated weights for policy 1, policy_version 948632 (0.0005) [2023-12-26 22:16:33,018][105620] Updated weights for policy 1, policy_version 948642 (0.0005) [2023-12-26 22:16:33,413][105692] Updated weights for policy 0, policy_version 948462 (0.0010) [2023-12-26 22:16:33,463][105692] Updated weights for policy 0, policy_version 948472 (0.0010) [2023-12-26 22:16:33,519][105692] Updated weights for policy 0, policy_version 948482 (0.0009) [2023-12-26 22:16:33,560][105620] Updated weights for policy 1, policy_version 948652 (0.0007) [2023-12-26 22:16:33,619][105620] Updated weights for policy 1, policy_version 948662 (0.0010) [2023-12-26 22:16:33,678][105620] Updated weights for policy 1, policy_version 948673 (0.0011) [2023-12-26 22:16:34,197][105692] Updated weights for policy 0, policy_version 948492 (0.0007) [2023-12-26 22:16:34,253][105692] Updated weights for policy 0, policy_version 948502 (0.0010) [2023-12-26 22:16:34,312][105692] Updated weights for policy 0, policy_version 948512 (0.0005) [2023-12-26 22:16:34,490][105620] Updated weights for policy 1, policy_version 948684 (0.0010) [2023-12-26 22:16:34,551][105620] Updated weights for policy 1, policy_version 948694 (0.0008) [2023-12-26 22:16:34,604][105620] Updated weights for policy 1, policy_version 948704 (0.0008) [2023-12-26 22:16:35,065][105692] Updated weights for policy 0, policy_version 948522 (0.0009) [2023-12-26 22:16:35,123][105692] Updated weights for policy 0, policy_version 948532 (0.0010) [2023-12-26 22:16:35,184][105692] Updated weights for policy 0, policy_version 948542 (0.0010) [2023-12-26 22:16:35,235][105692] Updated weights for policy 0, policy_version 948552 (0.0010) [2023-12-26 22:16:35,390][105620] Updated weights for policy 1, policy_version 948714 (0.0008) [2023-12-26 22:16:35,438][105620] Updated weights for policy 1, policy_version 948724 (0.0008) [2023-12-26 22:16:35,487][105620] Updated weights for policy 1, policy_version 948734 (0.0008) [2023-12-26 22:16:35,859][105692] Updated weights for policy 0, policy_version 948562 (0.0006) [2023-12-26 22:16:35,912][105692] Updated weights for policy 0, policy_version 948572 (0.0009) [2023-12-26 22:16:35,964][105692] Updated weights for policy 0, policy_version 948582 (0.0008) [2023-12-26 22:16:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19114.6, 300 sec: 19244.2). Total num frames: 485785600. Throughput: 0: 9781.4, 1: 9362.6. Samples: 485772852. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:16:36,063][104569] Avg episode reward: [(0, '9260.874'), (1, '8836.369')] [2023-12-26 22:16:36,340][105620] Updated weights for policy 1, policy_version 948745 (0.0010) [2023-12-26 22:16:36,399][105620] Updated weights for policy 1, policy_version 948755 (0.0010) [2023-12-26 22:16:36,466][105620] Updated weights for policy 1, policy_version 948765 (0.0011) [2023-12-26 22:16:36,534][105620] Updated weights for policy 1, policy_version 948775 (0.0011) [2023-12-26 22:16:36,586][105692] Updated weights for policy 0, policy_version 948592 (0.0008) [2023-12-26 22:16:36,655][105692] Updated weights for policy 0, policy_version 948602 (0.0008) [2023-12-26 22:16:36,724][105692] Updated weights for policy 0, policy_version 948612 (0.0008) [2023-12-26 22:16:37,281][105620] Updated weights for policy 1, policy_version 948785 (0.0008) [2023-12-26 22:16:37,337][105620] Updated weights for policy 1, policy_version 948795 (0.0008) [2023-12-26 22:16:37,385][105620] Updated weights for policy 1, policy_version 948805 (0.0008) [2023-12-26 22:16:37,447][105692] Updated weights for policy 0, policy_version 948622 (0.0010) [2023-12-26 22:16:37,507][105692] Updated weights for policy 0, policy_version 948632 (0.0008) [2023-12-26 22:16:37,569][105692] Updated weights for policy 0, policy_version 948642 (0.0005) [2023-12-26 22:16:38,167][105692] Updated weights for policy 0, policy_version 948652 (0.0007) [2023-12-26 22:16:38,223][105692] Updated weights for policy 0, policy_version 948662 (0.0011) [2023-12-26 22:16:38,256][105620] Updated weights for policy 1, policy_version 948815 (0.0006) [2023-12-26 22:16:38,279][105692] Updated weights for policy 0, policy_version 948672 (0.0009) [2023-12-26 22:16:38,306][105620] Updated weights for policy 1, policy_version 948825 (0.0008) [2023-12-26 22:16:38,376][105620] Updated weights for policy 1, policy_version 948835 (0.0010) [2023-12-26 22:16:39,001][105692] Updated weights for policy 0, policy_version 948682 (0.0007) [2023-12-26 22:16:39,063][105692] Updated weights for policy 0, policy_version 948692 (0.0010) [2023-12-26 22:16:39,121][105692] Updated weights for policy 0, policy_version 948702 (0.0010) [2023-12-26 22:16:39,176][105692] Updated weights for policy 0, policy_version 948712 (0.0010) [2023-12-26 22:16:39,179][105620] Updated weights for policy 1, policy_version 948845 (0.0009) [2023-12-26 22:16:39,236][105620] Updated weights for policy 1, policy_version 948855 (0.0008) [2023-12-26 22:16:39,294][105620] Updated weights for policy 1, policy_version 948865 (0.0009) [2023-12-26 22:16:39,805][105692] Updated weights for policy 0, policy_version 948722 (0.0011) [2023-12-26 22:16:39,870][105692] Updated weights for policy 0, policy_version 948732 (0.0009) [2023-12-26 22:16:39,929][105692] Updated weights for policy 0, policy_version 948742 (0.0008) [2023-12-26 22:16:40,136][105620] Updated weights for policy 1, policy_version 948875 (0.0009) [2023-12-26 22:16:40,195][105620] Updated weights for policy 1, policy_version 948885 (0.0009) [2023-12-26 22:16:40,262][105620] Updated weights for policy 1, policy_version 948895 (0.0009) [2023-12-26 22:16:40,583][105692] Updated weights for policy 0, policy_version 948752 (0.0009) [2023-12-26 22:16:40,643][105692] Updated weights for policy 0, policy_version 948762 (0.0005) [2023-12-26 22:16:40,701][105692] Updated weights for policy 0, policy_version 948772 (0.0009) [2023-12-26 22:16:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19114.7, 300 sec: 19244.3). Total num frames: 485875712. Throughput: 0: 9965.2, 1: 9323.1. Samples: 485887428. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:16:41,062][104569] Avg episode reward: [(0, '9261.223'), (1, '8917.791')] [2023-12-26 22:16:41,108][105620] Updated weights for policy 1, policy_version 948905 (0.0010) [2023-12-26 22:16:41,183][105620] Updated weights for policy 1, policy_version 948915 (0.0008) [2023-12-26 22:16:41,248][105620] Updated weights for policy 1, policy_version 948925 (0.0007) [2023-12-26 22:16:41,324][105620] Updated weights for policy 1, policy_version 948935 (0.0006) [2023-12-26 22:16:41,445][105692] Updated weights for policy 0, policy_version 948782 (0.0011) [2023-12-26 22:16:41,514][105692] Updated weights for policy 0, policy_version 948792 (0.0011) [2023-12-26 22:16:41,576][105692] Updated weights for policy 0, policy_version 948802 (0.0011) [2023-12-26 22:16:42,029][105620] Updated weights for policy 1, policy_version 948945 (0.0006) [2023-12-26 22:16:42,092][105620] Updated weights for policy 1, policy_version 948955 (0.0006) [2023-12-26 22:16:42,160][105620] Updated weights for policy 1, policy_version 948965 (0.0008) [2023-12-26 22:16:42,461][105692] Updated weights for policy 0, policy_version 948812 (0.0010) [2023-12-26 22:16:42,509][105692] Updated weights for policy 0, policy_version 948822 (0.0008) [2023-12-26 22:16:42,570][105692] Updated weights for policy 0, policy_version 948832 (0.0010) [2023-12-26 22:16:42,780][105620] Updated weights for policy 1, policy_version 948975 (0.0007) [2023-12-26 22:16:42,844][105620] Updated weights for policy 1, policy_version 948985 (0.0005) [2023-12-26 22:16:42,910][105620] Updated weights for policy 1, policy_version 948995 (0.0005) [2023-12-26 22:16:43,271][105692] Updated weights for policy 0, policy_version 948842 (0.0006) [2023-12-26 22:16:43,326][105692] Updated weights for policy 0, policy_version 948852 (0.0009) [2023-12-26 22:16:43,380][105692] Updated weights for policy 0, policy_version 948863 (0.0010) [2023-12-26 22:16:43,471][105620] Updated weights for policy 1, policy_version 949005 (0.0007) [2023-12-26 22:16:43,534][105620] Updated weights for policy 1, policy_version 949015 (0.0008) [2023-12-26 22:16:43,598][105620] Updated weights for policy 1, policy_version 949025 (0.0009) [2023-12-26 22:16:44,146][105692] Updated weights for policy 0, policy_version 948873 (0.0008) [2023-12-26 22:16:44,205][105692] Updated weights for policy 0, policy_version 948883 (0.0011) [2023-12-26 22:16:44,254][105692] Updated weights for policy 0, policy_version 948893 (0.0011) [2023-12-26 22:16:44,311][105692] Updated weights for policy 0, policy_version 948903 (0.0011) [2023-12-26 22:16:44,322][105620] Updated weights for policy 1, policy_version 949035 (0.0007) [2023-12-26 22:16:44,373][105620] Updated weights for policy 1, policy_version 949045 (0.0007) [2023-12-26 22:16:44,430][105620] Updated weights for policy 1, policy_version 949055 (0.0009) [2023-12-26 22:16:45,016][105692] Updated weights for policy 0, policy_version 948913 (0.0009) [2023-12-26 22:16:45,078][105692] Updated weights for policy 0, policy_version 948923 (0.0009) [2023-12-26 22:16:45,146][105692] Updated weights for policy 0, policy_version 948933 (0.0009) [2023-12-26 22:16:45,207][105620] Updated weights for policy 1, policy_version 949065 (0.0009) [2023-12-26 22:16:45,272][105620] Updated weights for policy 1, policy_version 949075 (0.0005) [2023-12-26 22:16:45,338][105620] Updated weights for policy 1, policy_version 949085 (0.0006) [2023-12-26 22:16:45,403][105620] Updated weights for policy 1, policy_version 949095 (0.0009) [2023-12-26 22:16:45,865][105692] Updated weights for policy 0, policy_version 948943 (0.0007) [2023-12-26 22:16:45,913][105692] Updated weights for policy 0, policy_version 948953 (0.0005) [2023-12-26 22:16:45,966][105692] Updated weights for policy 0, policy_version 948964 (0.0010) [2023-12-26 22:16:46,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19114.7, 300 sec: 19272.0). Total num frames: 485974016. Throughput: 0: 9878.0, 1: 9309.7. Samples: 485944480. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:16:46,062][104569] Avg episode reward: [(0, '9262.235'), (1, '9098.646')] [2023-12-26 22:16:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000948968_242974720.pth... [2023-12-26 22:16:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000947784_242671616.pth [2023-12-26 22:16:46,085][105620] Updated weights for policy 1, policy_version 949105 (0.0006) [2023-12-26 22:16:46,139][105620] Updated weights for policy 1, policy_version 949115 (0.0009) [2023-12-26 22:16:46,190][105620] Updated weights for policy 1, policy_version 949125 (0.0009) [2023-12-26 22:16:46,204][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000949128_243007488.pth... [2023-12-26 22:16:46,207][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000948008_242720768.pth [2023-12-26 22:16:46,746][105692] Updated weights for policy 0, policy_version 948974 (0.0009) [2023-12-26 22:16:46,807][105692] Updated weights for policy 0, policy_version 948984 (0.0009) [2023-12-26 22:16:46,867][105692] Updated weights for policy 0, policy_version 948994 (0.0009) [2023-12-26 22:16:46,903][105620] Updated weights for policy 1, policy_version 949135 (0.0007) [2023-12-26 22:16:46,956][105620] Updated weights for policy 1, policy_version 949145 (0.0009) [2023-12-26 22:16:47,007][105620] Updated weights for policy 1, policy_version 949155 (0.0009) [2023-12-26 22:16:47,640][105692] Updated weights for policy 0, policy_version 949004 (0.0009) [2023-12-26 22:16:47,697][105692] Updated weights for policy 0, policy_version 949014 (0.0009) [2023-12-26 22:16:47,746][105620] Updated weights for policy 1, policy_version 949165 (0.0008) [2023-12-26 22:16:47,748][105692] Updated weights for policy 0, policy_version 949024 (0.0007) [2023-12-26 22:16:47,803][105620] Updated weights for policy 1, policy_version 949175 (0.0007) [2023-12-26 22:16:47,855][105620] Updated weights for policy 1, policy_version 949185 (0.0009) [2023-12-26 22:16:48,485][105692] Updated weights for policy 0, policy_version 949034 (0.0007) [2023-12-26 22:16:48,549][105692] Updated weights for policy 0, policy_version 949044 (0.0008) [2023-12-26 22:16:48,558][105620] Updated weights for policy 1, policy_version 949195 (0.0009) [2023-12-26 22:16:48,615][105692] Updated weights for policy 0, policy_version 949054 (0.0007) [2023-12-26 22:16:48,620][105620] Updated weights for policy 1, policy_version 949205 (0.0010) [2023-12-26 22:16:48,680][105620] Updated weights for policy 1, policy_version 949215 (0.0011) [2023-12-26 22:16:48,684][105692] Updated weights for policy 0, policy_version 949064 (0.0006) [2023-12-26 22:16:49,262][105692] Updated weights for policy 0, policy_version 949074 (0.0008) [2023-12-26 22:16:49,322][105692] Updated weights for policy 0, policy_version 949084 (0.0008) [2023-12-26 22:16:49,388][105692] Updated weights for policy 0, policy_version 949094 (0.0008) [2023-12-26 22:16:49,431][105620] Updated weights for policy 1, policy_version 949225 (0.0010) [2023-12-26 22:16:49,490][105620] Updated weights for policy 1, policy_version 949235 (0.0007) [2023-12-26 22:16:49,559][105620] Updated weights for policy 1, policy_version 949245 (0.0010) [2023-12-26 22:16:49,622][105620] Updated weights for policy 1, policy_version 949255 (0.0011) [2023-12-26 22:16:50,122][105692] Updated weights for policy 0, policy_version 949104 (0.0008) [2023-12-26 22:16:50,173][105692] Updated weights for policy 0, policy_version 949114 (0.0009) [2023-12-26 22:16:50,225][105692] Updated weights for policy 0, policy_version 949124 (0.0009) [2023-12-26 22:16:50,325][105620] Updated weights for policy 1, policy_version 949265 (0.0006) [2023-12-26 22:16:50,383][105620] Updated weights for policy 1, policy_version 949275 (0.0006) [2023-12-26 22:16:50,441][105620] Updated weights for policy 1, policy_version 949285 (0.0006) [2023-12-26 22:16:51,060][105692] Updated weights for policy 0, policy_version 949134 (0.0009) [2023-12-26 22:16:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18978.1, 300 sec: 19244.3). Total num frames: 486064128. Throughput: 0: 9840.8, 1: 9360.0. Samples: 486059296. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:16:51,062][104569] Avg episode reward: [(0, '9180.562'), (1, '9098.103')] [2023-12-26 22:16:51,120][105620] Updated weights for policy 1, policy_version 949295 (0.0009) [2023-12-26 22:16:51,125][105692] Updated weights for policy 0, policy_version 949144 (0.0007) [2023-12-26 22:16:51,181][105620] Updated weights for policy 1, policy_version 949305 (0.0011) [2023-12-26 22:16:51,190][105692] Updated weights for policy 0, policy_version 949154 (0.0006) [2023-12-26 22:16:51,250][105620] Updated weights for policy 1, policy_version 949315 (0.0008) [2023-12-26 22:16:51,872][105692] Updated weights for policy 0, policy_version 949164 (0.0008) [2023-12-26 22:16:51,934][105692] Updated weights for policy 0, policy_version 949174 (0.0007) [2023-12-26 22:16:51,997][105692] Updated weights for policy 0, policy_version 949184 (0.0006) [2023-12-26 22:16:52,049][105620] Updated weights for policy 1, policy_version 949325 (0.0011) [2023-12-26 22:16:52,112][105620] Updated weights for policy 1, policy_version 949335 (0.0010) [2023-12-26 22:16:52,182][105620] Updated weights for policy 1, policy_version 949345 (0.0010) [2023-12-26 22:16:52,707][105692] Updated weights for policy 0, policy_version 949194 (0.0006) [2023-12-26 22:16:52,756][105692] Updated weights for policy 0, policy_version 949204 (0.0005) [2023-12-26 22:16:52,809][105692] Updated weights for policy 0, policy_version 949214 (0.0007) [2023-12-26 22:16:52,858][105692] Updated weights for policy 0, policy_version 949224 (0.0005) [2023-12-26 22:16:52,959][105620] Updated weights for policy 1, policy_version 949355 (0.0010) [2023-12-26 22:16:53,009][105620] Updated weights for policy 1, policy_version 949365 (0.0009) [2023-12-26 22:16:53,056][105620] Updated weights for policy 1, policy_version 949375 (0.0009) [2023-12-26 22:16:53,439][105692] Updated weights for policy 0, policy_version 949234 (0.0009) [2023-12-26 22:16:53,485][105692] Updated weights for policy 0, policy_version 949244 (0.0009) [2023-12-26 22:16:53,530][105692] Updated weights for policy 0, policy_version 949254 (0.0008) [2023-12-26 22:16:53,900][105620] Updated weights for policy 1, policy_version 949385 (0.0010) [2023-12-26 22:16:53,962][105620] Updated weights for policy 1, policy_version 949395 (0.0010) [2023-12-26 22:16:54,023][105620] Updated weights for policy 1, policy_version 949405 (0.0010) [2023-12-26 22:16:54,090][105620] Updated weights for policy 1, policy_version 949415 (0.0010) [2023-12-26 22:16:54,220][105692] Updated weights for policy 0, policy_version 949264 (0.0009) [2023-12-26 22:16:54,279][105692] Updated weights for policy 0, policy_version 949274 (0.0010) [2023-12-26 22:16:54,332][105692] Updated weights for policy 0, policy_version 949285 (0.0010) [2023-12-26 22:16:54,648][105620] Updated weights for policy 1, policy_version 949425 (0.0008) [2023-12-26 22:16:54,709][105620] Updated weights for policy 1, policy_version 949435 (0.0008) [2023-12-26 22:16:54,768][105620] Updated weights for policy 1, policy_version 949445 (0.0008) [2023-12-26 22:16:55,192][105692] Updated weights for policy 0, policy_version 949295 (0.0009) [2023-12-26 22:16:55,244][105692] Updated weights for policy 0, policy_version 949305 (0.0009) [2023-12-26 22:16:55,299][105692] Updated weights for policy 0, policy_version 949316 (0.0010) [2023-12-26 22:16:55,338][105620] Updated weights for policy 1, policy_version 949455 (0.0006) [2023-12-26 22:16:55,394][105620] Updated weights for policy 1, policy_version 949465 (0.0005) [2023-12-26 22:16:55,445][105620] Updated weights for policy 1, policy_version 949475 (0.0005) [2023-12-26 22:16:55,959][105692] Updated weights for policy 0, policy_version 949326 (0.0007) [2023-12-26 22:16:56,014][105692] Updated weights for policy 0, policy_version 949336 (0.0006) [2023-12-26 22:16:56,021][105620] Updated weights for policy 1, policy_version 949485 (0.0007) [2023-12-26 22:16:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19244.3). Total num frames: 486162432. Throughput: 0: 9796.3, 1: 9413.1. Samples: 486177736. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:16:56,062][104569] Avg episode reward: [(0, '9089.679'), (1, '8917.029')] [2023-12-26 22:16:56,071][105692] Updated weights for policy 0, policy_version 949346 (0.0006) [2023-12-26 22:16:56,085][105620] Updated weights for policy 1, policy_version 949495 (0.0010) [2023-12-26 22:16:56,150][105620] Updated weights for policy 1, policy_version 949505 (0.0010) [2023-12-26 22:16:56,719][105692] Updated weights for policy 0, policy_version 949356 (0.0007) [2023-12-26 22:16:56,768][105692] Updated weights for policy 0, policy_version 949366 (0.0010) [2023-12-26 22:16:56,831][105692] Updated weights for policy 0, policy_version 949376 (0.0011) [2023-12-26 22:16:56,873][105620] Updated weights for policy 1, policy_version 949515 (0.0009) [2023-12-26 22:16:56,921][105620] Updated weights for policy 1, policy_version 949525 (0.0008) [2023-12-26 22:16:56,978][105620] Updated weights for policy 1, policy_version 949535 (0.0008) [2023-12-26 22:16:57,553][105692] Updated weights for policy 0, policy_version 949386 (0.0009) [2023-12-26 22:16:57,611][105692] Updated weights for policy 0, policy_version 949396 (0.0005) [2023-12-26 22:16:57,670][105692] Updated weights for policy 0, policy_version 949406 (0.0005) [2023-12-26 22:16:57,728][105692] Updated weights for policy 0, policy_version 949416 (0.0005) [2023-12-26 22:16:57,760][105620] Updated weights for policy 1, policy_version 949545 (0.0008) [2023-12-26 22:16:57,831][105620] Updated weights for policy 1, policy_version 949555 (0.0005) [2023-12-26 22:16:57,895][105620] Updated weights for policy 1, policy_version 949565 (0.0005) [2023-12-26 22:16:57,946][105620] Updated weights for policy 1, policy_version 949575 (0.0005) [2023-12-26 22:16:58,300][105692] Updated weights for policy 0, policy_version 949426 (0.0011) [2023-12-26 22:16:58,377][105692] Updated weights for policy 0, policy_version 949436 (0.0010) [2023-12-26 22:16:58,441][105692] Updated weights for policy 0, policy_version 949446 (0.0009) [2023-12-26 22:16:58,610][105620] Updated weights for policy 1, policy_version 949585 (0.0008) [2023-12-26 22:16:58,669][105620] Updated weights for policy 1, policy_version 949595 (0.0008) [2023-12-26 22:16:58,740][105620] Updated weights for policy 1, policy_version 949605 (0.0009) [2023-12-26 22:16:59,244][105692] Updated weights for policy 0, policy_version 949456 (0.0008) [2023-12-26 22:16:59,315][105692] Updated weights for policy 0, policy_version 949466 (0.0008) [2023-12-26 22:16:59,392][105692] Updated weights for policy 0, policy_version 949476 (0.0008) [2023-12-26 22:16:59,621][105620] Updated weights for policy 1, policy_version 949615 (0.0009) [2023-12-26 22:16:59,683][105620] Updated weights for policy 1, policy_version 949625 (0.0009) [2023-12-26 22:16:59,741][105620] Updated weights for policy 1, policy_version 949635 (0.0009) [2023-12-26 22:17:00,148][105692] Updated weights for policy 0, policy_version 949486 (0.0007) [2023-12-26 22:17:00,196][105692] Updated weights for policy 0, policy_version 949496 (0.0009) [2023-12-26 22:17:00,247][105692] Updated weights for policy 0, policy_version 949506 (0.0009) [2023-12-26 22:17:00,495][105620] Updated weights for policy 1, policy_version 949645 (0.0009) [2023-12-26 22:17:00,540][105620] Updated weights for policy 1, policy_version 949655 (0.0008) [2023-12-26 22:17:00,586][105620] Updated weights for policy 1, policy_version 949665 (0.0009) [2023-12-26 22:17:01,014][105692] Updated weights for policy 0, policy_version 949516 (0.0008) [2023-12-26 22:17:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19272.0). Total num frames: 486260736. Throughput: 0: 9850.8, 1: 9451.5. Samples: 486237136. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:17:01,063][104569] Avg episode reward: [(0, '9172.589'), (1, '9012.197')] [2023-12-26 22:17:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000949672_243146752.pth... [2023-12-26 22:17:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000948552_242860032.pth [2023-12-26 22:17:01,079][105692] Updated weights for policy 0, policy_version 949526 (0.0009) [2023-12-26 22:17:01,132][105692] Updated weights for policy 0, policy_version 949536 (0.0008) [2023-12-26 22:17:01,180][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000949544_243122176.pth... [2023-12-26 22:17:01,183][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000948392_242827264.pth [2023-12-26 22:17:01,406][105620] Updated weights for policy 1, policy_version 949675 (0.0009) [2023-12-26 22:17:01,454][105620] Updated weights for policy 1, policy_version 949685 (0.0009) [2023-12-26 22:17:01,504][105620] Updated weights for policy 1, policy_version 949695 (0.0008) [2023-12-26 22:17:01,842][105692] Updated weights for policy 0, policy_version 949546 (0.0007) [2023-12-26 22:17:01,908][105692] Updated weights for policy 0, policy_version 949556 (0.0009) [2023-12-26 22:17:01,968][105692] Updated weights for policy 0, policy_version 949566 (0.0009) [2023-12-26 22:17:02,030][105692] Updated weights for policy 0, policy_version 949576 (0.0009) [2023-12-26 22:17:02,313][105620] Updated weights for policy 1, policy_version 949705 (0.0009) [2023-12-26 22:17:02,378][105620] Updated weights for policy 1, policy_version 949715 (0.0009) [2023-12-26 22:17:02,432][105620] Updated weights for policy 1, policy_version 949725 (0.0008) [2023-12-26 22:17:02,486][105620] Updated weights for policy 1, policy_version 949735 (0.0009) [2023-12-26 22:17:02,760][105692] Updated weights for policy 0, policy_version 949586 (0.0009) [2023-12-26 22:17:02,814][105692] Updated weights for policy 0, policy_version 949596 (0.0009) [2023-12-26 22:17:02,865][105692] Updated weights for policy 0, policy_version 949606 (0.0009) [2023-12-26 22:17:03,245][105620] Updated weights for policy 1, policy_version 949745 (0.0008) [2023-12-26 22:17:03,292][105620] Updated weights for policy 1, policy_version 949755 (0.0009) [2023-12-26 22:17:03,337][105620] Updated weights for policy 1, policy_version 949765 (0.0008) [2023-12-26 22:17:03,642][105692] Updated weights for policy 0, policy_version 949616 (0.0009) [2023-12-26 22:17:03,709][105692] Updated weights for policy 0, policy_version 949626 (0.0010) [2023-12-26 22:17:03,765][105692] Updated weights for policy 0, policy_version 949636 (0.0009) [2023-12-26 22:17:03,978][105620] Updated weights for policy 1, policy_version 949775 (0.0009) [2023-12-26 22:17:04,026][105620] Updated weights for policy 1, policy_version 949785 (0.0008) [2023-12-26 22:17:04,073][105620] Updated weights for policy 1, policy_version 949795 (0.0008) [2023-12-26 22:17:04,568][105692] Updated weights for policy 0, policy_version 949647 (0.0009) [2023-12-26 22:17:04,627][105692] Updated weights for policy 0, policy_version 949657 (0.0008) [2023-12-26 22:17:04,678][105692] Updated weights for policy 0, policy_version 949667 (0.0008) [2023-12-26 22:17:04,817][105620] Updated weights for policy 1, policy_version 949805 (0.0009) [2023-12-26 22:17:04,867][105620] Updated weights for policy 1, policy_version 949815 (0.0010) [2023-12-26 22:17:04,920][105620] Updated weights for policy 1, policy_version 949825 (0.0005) [2023-12-26 22:17:05,482][105692] Updated weights for policy 0, policy_version 949677 (0.0009) [2023-12-26 22:17:05,539][105692] Updated weights for policy 0, policy_version 949687 (0.0010) [2023-12-26 22:17:05,591][105692] Updated weights for policy 0, policy_version 949698 (0.0010) [2023-12-26 22:17:05,605][105620] Updated weights for policy 1, policy_version 949835 (0.0005) [2023-12-26 22:17:05,656][105620] Updated weights for policy 1, policy_version 949845 (0.0007) [2023-12-26 22:17:05,711][105620] Updated weights for policy 1, policy_version 949855 (0.0009) [2023-12-26 22:17:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19272.0). Total num frames: 486359040. Throughput: 0: 9812.1, 1: 9404.7. Samples: 486348160. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:17:06,062][104569] Avg episode reward: [(0, '9264.878'), (1, '8924.817')] [2023-12-26 22:17:06,407][105692] Updated weights for policy 0, policy_version 949708 (0.0009) [2023-12-26 22:17:06,416][105620] Updated weights for policy 1, policy_version 949865 (0.0007) [2023-12-26 22:17:06,463][105692] Updated weights for policy 0, policy_version 949718 (0.0008) [2023-12-26 22:17:06,477][105620] Updated weights for policy 1, policy_version 949875 (0.0006) [2023-12-26 22:17:06,520][105692] Updated weights for policy 0, policy_version 949728 (0.0006) [2023-12-26 22:17:06,539][105620] Updated weights for policy 1, policy_version 949885 (0.0007) [2023-12-26 22:17:06,602][105620] Updated weights for policy 1, policy_version 949895 (0.0008) [2023-12-26 22:17:07,307][105692] Updated weights for policy 0, policy_version 949738 (0.0007) [2023-12-26 22:17:07,333][105620] Updated weights for policy 1, policy_version 949905 (0.0010) [2023-12-26 22:17:07,363][105692] Updated weights for policy 0, policy_version 949748 (0.0007) [2023-12-26 22:17:07,393][105620] Updated weights for policy 1, policy_version 949915 (0.0008) [2023-12-26 22:17:07,421][105692] Updated weights for policy 0, policy_version 949758 (0.0008) [2023-12-26 22:17:07,447][105620] Updated weights for policy 1, policy_version 949925 (0.0010) [2023-12-26 22:17:07,484][105692] Updated weights for policy 0, policy_version 949768 (0.0008) [2023-12-26 22:17:08,103][105620] Updated weights for policy 1, policy_version 949935 (0.0007) [2023-12-26 22:17:08,114][105692] Updated weights for policy 0, policy_version 949778 (0.0006) [2023-12-26 22:17:08,157][105620] Updated weights for policy 1, policy_version 949945 (0.0008) [2023-12-26 22:17:08,173][105692] Updated weights for policy 0, policy_version 949788 (0.0005) [2023-12-26 22:17:08,210][105620] Updated weights for policy 1, policy_version 949955 (0.0006) [2023-12-26 22:17:08,238][105692] Updated weights for policy 0, policy_version 949798 (0.0005) [2023-12-26 22:17:08,932][105620] Updated weights for policy 1, policy_version 949965 (0.0006) [2023-12-26 22:17:08,946][105692] Updated weights for policy 0, policy_version 949808 (0.0007) [2023-12-26 22:17:08,991][105620] Updated weights for policy 1, policy_version 949975 (0.0008) [2023-12-26 22:17:08,998][105692] Updated weights for policy 0, policy_version 949818 (0.0006) [2023-12-26 22:17:09,052][105620] Updated weights for policy 1, policy_version 949985 (0.0009) [2023-12-26 22:17:09,055][105692] Updated weights for policy 0, policy_version 949828 (0.0008) [2023-12-26 22:17:09,800][105692] Updated weights for policy 0, policy_version 949838 (0.0009) [2023-12-26 22:17:09,871][105692] Updated weights for policy 0, policy_version 949848 (0.0011) [2023-12-26 22:17:09,890][105620] Updated weights for policy 1, policy_version 949995 (0.0009) [2023-12-26 22:17:09,932][105692] Updated weights for policy 0, policy_version 949858 (0.0011) [2023-12-26 22:17:09,946][105620] Updated weights for policy 1, policy_version 950005 (0.0009) [2023-12-26 22:17:10,000][105620] Updated weights for policy 1, policy_version 950015 (0.0008) [2023-12-26 22:17:10,684][105692] Updated weights for policy 0, policy_version 949868 (0.0010) [2023-12-26 22:17:10,743][105692] Updated weights for policy 0, policy_version 949878 (0.0010) [2023-12-26 22:17:10,785][105620] Updated weights for policy 1, policy_version 950025 (0.0008) [2023-12-26 22:17:10,793][105692] Updated weights for policy 0, policy_version 949888 (0.0010) [2023-12-26 22:17:10,847][105620] Updated weights for policy 1, policy_version 950035 (0.0010) [2023-12-26 22:17:10,921][105620] Updated weights for policy 1, policy_version 950045 (0.0011) [2023-12-26 22:17:10,983][105620] Updated weights for policy 1, policy_version 950055 (0.0010) [2023-12-26 22:17:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19272.0). Total num frames: 486457344. Throughput: 0: 9788.3, 1: 9441.4. Samples: 486462072. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:17:11,062][104569] Avg episode reward: [(0, '9084.479'), (1, '8752.086')] [2023-12-26 22:17:11,517][105692] Updated weights for policy 0, policy_version 949898 (0.0010) [2023-12-26 22:17:11,576][105692] Updated weights for policy 0, policy_version 949908 (0.0010) [2023-12-26 22:17:11,639][105692] Updated weights for policy 0, policy_version 949918 (0.0010) [2023-12-26 22:17:11,695][105692] Updated weights for policy 0, policy_version 949928 (0.0011) [2023-12-26 22:17:11,695][105620] Updated weights for policy 1, policy_version 950065 (0.0011) [2023-12-26 22:17:11,761][105620] Updated weights for policy 1, policy_version 950075 (0.0010) [2023-12-26 22:17:11,827][105620] Updated weights for policy 1, policy_version 950085 (0.0007) [2023-12-26 22:17:12,497][105692] Updated weights for policy 0, policy_version 949938 (0.0011) [2023-12-26 22:17:12,507][105620] Updated weights for policy 1, policy_version 950095 (0.0006) [2023-12-26 22:17:12,554][105692] Updated weights for policy 0, policy_version 949948 (0.0011) [2023-12-26 22:17:12,566][105620] Updated weights for policy 1, policy_version 950105 (0.0007) [2023-12-26 22:17:12,614][105692] Updated weights for policy 0, policy_version 949958 (0.0011) [2023-12-26 22:17:12,629][105620] Updated weights for policy 1, policy_version 950115 (0.0008) [2023-12-26 22:17:13,291][105620] Updated weights for policy 1, policy_version 950125 (0.0008) [2023-12-26 22:17:13,357][105620] Updated weights for policy 1, policy_version 950135 (0.0008) [2023-12-26 22:17:13,386][105692] Updated weights for policy 0, policy_version 949968 (0.0011) [2023-12-26 22:17:13,413][105620] Updated weights for policy 1, policy_version 950145 (0.0006) [2023-12-26 22:17:13,441][105692] Updated weights for policy 0, policy_version 949978 (0.0010) [2023-12-26 22:17:13,500][105692] Updated weights for policy 0, policy_version 949988 (0.0010) [2023-12-26 22:17:14,185][105692] Updated weights for policy 0, policy_version 949998 (0.0009) [2023-12-26 22:17:14,199][105620] Updated weights for policy 1, policy_version 950155 (0.0006) [2023-12-26 22:17:14,254][105692] Updated weights for policy 0, policy_version 950008 (0.0007) [2023-12-26 22:17:14,263][105620] Updated weights for policy 1, policy_version 950165 (0.0007) [2023-12-26 22:17:14,315][105692] Updated weights for policy 0, policy_version 950018 (0.0007) [2023-12-26 22:17:14,317][105620] Updated weights for policy 1, policy_version 950175 (0.0006) [2023-12-26 22:17:15,054][105692] Updated weights for policy 0, policy_version 950028 (0.0008) [2023-12-26 22:17:15,086][105620] Updated weights for policy 1, policy_version 950185 (0.0007) [2023-12-26 22:17:15,128][105692] Updated weights for policy 0, policy_version 950038 (0.0011) [2023-12-26 22:17:15,146][105620] Updated weights for policy 1, policy_version 950195 (0.0006) [2023-12-26 22:17:15,193][105692] Updated weights for policy 0, policy_version 950048 (0.0010) [2023-12-26 22:17:15,210][105620] Updated weights for policy 1, policy_version 950205 (0.0008) [2023-12-26 22:17:15,275][105620] Updated weights for policy 1, policy_version 950215 (0.0006) [2023-12-26 22:17:15,922][105692] Updated weights for policy 0, policy_version 950058 (0.0011) [2023-12-26 22:17:15,931][105620] Updated weights for policy 1, policy_version 950225 (0.0008) [2023-12-26 22:17:15,985][105692] Updated weights for policy 0, policy_version 950068 (0.0011) [2023-12-26 22:17:15,990][105620] Updated weights for policy 1, policy_version 950235 (0.0011) [2023-12-26 22:17:16,043][105692] Updated weights for policy 0, policy_version 950078 (0.0010) [2023-12-26 22:17:16,049][105620] Updated weights for policy 1, policy_version 950245 (0.0010) [2023-12-26 22:17:16,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19114.8, 300 sec: 19216.5). Total num frames: 486539264. Throughput: 0: 9690.3, 1: 9489.4. Samples: 486518376. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:17:16,062][104569] Avg episode reward: [(0, '9085.519'), (1, '9102.065')] [2023-12-26 22:17:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000950248_243294208.pth... [2023-12-26 22:17:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000949128_243007488.pth [2023-12-26 22:17:16,106][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000950088_243261440.pth... [2023-12-26 22:17:16,109][105692] Updated weights for policy 0, policy_version 950088 (0.0011) [2023-12-26 22:17:16,111][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000948968_242974720.pth [2023-12-26 22:17:16,626][105620] Updated weights for policy 1, policy_version 950255 (0.0010) [2023-12-26 22:17:16,682][105620] Updated weights for policy 1, policy_version 950265 (0.0011) [2023-12-26 22:17:16,730][105620] Updated weights for policy 1, policy_version 950275 (0.0010) [2023-12-26 22:17:16,835][105692] Updated weights for policy 0, policy_version 950098 (0.0005) [2023-12-26 22:17:16,883][105692] Updated weights for policy 0, policy_version 950108 (0.0005) [2023-12-26 22:17:16,929][105692] Updated weights for policy 0, policy_version 950118 (0.0005) [2023-12-26 22:17:17,462][105692] Updated weights for policy 0, policy_version 950128 (0.0006) [2023-12-26 22:17:17,463][105620] Updated weights for policy 1, policy_version 950285 (0.0012) [2023-12-26 22:17:17,527][105692] Updated weights for policy 0, policy_version 950138 (0.0007) [2023-12-26 22:17:17,529][105620] Updated weights for policy 1, policy_version 950295 (0.0008) [2023-12-26 22:17:17,590][105692] Updated weights for policy 0, policy_version 950148 (0.0005) [2023-12-26 22:17:17,591][105620] Updated weights for policy 1, policy_version 950305 (0.0009) [2023-12-26 22:17:18,292][105692] Updated weights for policy 0, policy_version 950158 (0.0009) [2023-12-26 22:17:18,352][105620] Updated weights for policy 1, policy_version 950315 (0.0007) [2023-12-26 22:17:18,357][105692] Updated weights for policy 0, policy_version 950168 (0.0008) [2023-12-26 22:17:18,421][105620] Updated weights for policy 1, policy_version 950325 (0.0008) [2023-12-26 22:17:18,424][105692] Updated weights for policy 0, policy_version 950178 (0.0007) [2023-12-26 22:17:18,484][105620] Updated weights for policy 1, policy_version 950335 (0.0008) [2023-12-26 22:17:19,155][105692] Updated weights for policy 0, policy_version 950188 (0.0007) [2023-12-26 22:17:19,210][105692] Updated weights for policy 0, policy_version 950198 (0.0009) [2023-12-26 22:17:19,255][105620] Updated weights for policy 1, policy_version 950345 (0.0008) [2023-12-26 22:17:19,276][105692] Updated weights for policy 0, policy_version 950208 (0.0009) [2023-12-26 22:17:19,315][105620] Updated weights for policy 1, policy_version 950355 (0.0007) [2023-12-26 22:17:19,380][105620] Updated weights for policy 1, policy_version 950365 (0.0009) [2023-12-26 22:17:19,445][105620] Updated weights for policy 1, policy_version 950375 (0.0009) [2023-12-26 22:17:20,076][105692] Updated weights for policy 0, policy_version 950218 (0.0007) [2023-12-26 22:17:20,127][105692] Updated weights for policy 0, policy_version 950228 (0.0009) [2023-12-26 22:17:20,179][105692] Updated weights for policy 0, policy_version 950238 (0.0009) [2023-12-26 22:17:20,239][105692] Updated weights for policy 0, policy_version 950248 (0.0008) [2023-12-26 22:17:20,258][105620] Updated weights for policy 1, policy_version 950385 (0.0007) [2023-12-26 22:17:20,319][105620] Updated weights for policy 1, policy_version 950395 (0.0006) [2023-12-26 22:17:20,385][105620] Updated weights for policy 1, policy_version 950405 (0.0007) [2023-12-26 22:17:20,956][105692] Updated weights for policy 0, policy_version 950258 (0.0009) [2023-12-26 22:17:21,019][105692] Updated weights for policy 0, policy_version 950268 (0.0008) [2023-12-26 22:17:21,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19114.6, 300 sec: 19216.5). Total num frames: 486637568. Throughput: 0: 9642.2, 1: 9507.5. Samples: 486634584. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:17:21,063][104569] Avg episode reward: [(0, '9357.565'), (1, '9095.356')] [2023-12-26 22:17:21,085][105692] Updated weights for policy 0, policy_version 950278 (0.0009) [2023-12-26 22:17:21,123][105620] Updated weights for policy 1, policy_version 950415 (0.0008) [2023-12-26 22:17:21,190][105620] Updated weights for policy 1, policy_version 950425 (0.0008) [2023-12-26 22:17:21,254][105620] Updated weights for policy 1, policy_version 950435 (0.0009) [2023-12-26 22:17:21,856][105692] Updated weights for policy 0, policy_version 950288 (0.0007) [2023-12-26 22:17:21,912][105692] Updated weights for policy 0, policy_version 950298 (0.0007) [2023-12-26 22:17:21,974][105692] Updated weights for policy 0, policy_version 950308 (0.0011) [2023-12-26 22:17:22,033][105620] Updated weights for policy 1, policy_version 950445 (0.0009) [2023-12-26 22:17:22,096][105620] Updated weights for policy 1, policy_version 950455 (0.0008) [2023-12-26 22:17:22,158][105620] Updated weights for policy 1, policy_version 950465 (0.0008) [2023-12-26 22:17:22,737][105692] Updated weights for policy 0, policy_version 950318 (0.0010) [2023-12-26 22:17:22,796][105692] Updated weights for policy 0, policy_version 950328 (0.0009) [2023-12-26 22:17:22,863][105692] Updated weights for policy 0, policy_version 950338 (0.0009) [2023-12-26 22:17:22,883][105620] Updated weights for policy 1, policy_version 950475 (0.0007) [2023-12-26 22:17:22,945][105620] Updated weights for policy 1, policy_version 950485 (0.0009) [2023-12-26 22:17:23,013][105620] Updated weights for policy 1, policy_version 950495 (0.0009) [2023-12-26 22:17:23,622][105692] Updated weights for policy 0, policy_version 950348 (0.0009) [2023-12-26 22:17:23,672][105692] Updated weights for policy 0, policy_version 950358 (0.0009) [2023-12-26 22:17:23,720][105692] Updated weights for policy 0, policy_version 950368 (0.0008) [2023-12-26 22:17:23,815][105620] Updated weights for policy 1, policy_version 950505 (0.0010) [2023-12-26 22:17:23,883][105620] Updated weights for policy 1, policy_version 950515 (0.0010) [2023-12-26 22:17:23,951][105620] Updated weights for policy 1, policy_version 950525 (0.0007) [2023-12-26 22:17:24,016][105620] Updated weights for policy 1, policy_version 950535 (0.0008) [2023-12-26 22:17:24,338][105692] Updated weights for policy 0, policy_version 950378 (0.0006) [2023-12-26 22:17:24,407][105692] Updated weights for policy 0, policy_version 950388 (0.0006) [2023-12-26 22:17:24,468][105692] Updated weights for policy 0, policy_version 950398 (0.0007) [2023-12-26 22:17:24,523][105692] Updated weights for policy 0, policy_version 950408 (0.0009) [2023-12-26 22:17:24,597][105620] Updated weights for policy 1, policy_version 950545 (0.0005) [2023-12-26 22:17:24,662][105620] Updated weights for policy 1, policy_version 950555 (0.0008) [2023-12-26 22:17:24,721][105620] Updated weights for policy 1, policy_version 950565 (0.0008) [2023-12-26 22:17:25,140][105692] Updated weights for policy 0, policy_version 950418 (0.0005) [2023-12-26 22:17:25,205][105692] Updated weights for policy 0, policy_version 950428 (0.0005) [2023-12-26 22:17:25,274][105692] Updated weights for policy 0, policy_version 950438 (0.0005) [2023-12-26 22:17:25,359][105620] Updated weights for policy 1, policy_version 950575 (0.0009) [2023-12-26 22:17:25,426][105620] Updated weights for policy 1, policy_version 950585 (0.0005) [2023-12-26 22:17:25,491][105620] Updated weights for policy 1, policy_version 950595 (0.0006) [2023-12-26 22:17:25,863][105692] Updated weights for policy 0, policy_version 950448 (0.0006) [2023-12-26 22:17:25,919][105692] Updated weights for policy 0, policy_version 950458 (0.0005) [2023-12-26 22:17:25,969][105692] Updated weights for policy 0, policy_version 950468 (0.0005) [2023-12-26 22:17:26,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19251.3, 300 sec: 19244.3). Total num frames: 486744064. Throughput: 0: 9579.3, 1: 9633.0. Samples: 486751980. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:17:26,063][104569] Avg episode reward: [(0, '9357.642'), (1, '9095.338')] [2023-12-26 22:17:26,158][105620] Updated weights for policy 1, policy_version 950605 (0.0010) [2023-12-26 22:17:26,219][105620] Updated weights for policy 1, policy_version 950615 (0.0010) [2023-12-26 22:17:26,270][105620] Updated weights for policy 1, policy_version 950625 (0.0010) [2023-12-26 22:17:26,538][105692] Updated weights for policy 0, policy_version 950478 (0.0007) [2023-12-26 22:17:26,596][105692] Updated weights for policy 0, policy_version 950488 (0.0005) [2023-12-26 22:17:26,647][105692] Updated weights for policy 0, policy_version 950498 (0.0005) [2023-12-26 22:17:27,011][105620] Updated weights for policy 1, policy_version 950635 (0.0010) [2023-12-26 22:17:27,063][105620] Updated weights for policy 1, policy_version 950645 (0.0010) [2023-12-26 22:17:27,118][105620] Updated weights for policy 1, policy_version 950655 (0.0010) [2023-12-26 22:17:27,314][105692] Updated weights for policy 0, policy_version 950508 (0.0007) [2023-12-26 22:17:27,372][105692] Updated weights for policy 0, policy_version 950518 (0.0010) [2023-12-26 22:17:27,425][105692] Updated weights for policy 0, policy_version 950528 (0.0006) [2023-12-26 22:17:27,875][105620] Updated weights for policy 1, policy_version 950665 (0.0010) [2023-12-26 22:17:27,923][105620] Updated weights for policy 1, policy_version 950675 (0.0010) [2023-12-26 22:17:27,971][105620] Updated weights for policy 1, policy_version 950685 (0.0010) [2023-12-26 22:17:27,997][105692] Updated weights for policy 0, policy_version 950538 (0.0005) [2023-12-26 22:17:28,019][105620] Updated weights for policy 1, policy_version 950695 (0.0010) [2023-12-26 22:17:28,058][105692] Updated weights for policy 0, policy_version 950548 (0.0005) [2023-12-26 22:17:28,126][105692] Updated weights for policy 0, policy_version 950558 (0.0006) [2023-12-26 22:17:28,188][105692] Updated weights for policy 0, policy_version 950568 (0.0005) [2023-12-26 22:17:28,743][105620] Updated weights for policy 1, policy_version 950705 (0.0009) [2023-12-26 22:17:28,801][105620] Updated weights for policy 1, policy_version 950715 (0.0008) [2023-12-26 22:17:28,832][105692] Updated weights for policy 0, policy_version 950578 (0.0011) [2023-12-26 22:17:28,869][105620] Updated weights for policy 1, policy_version 950725 (0.0008) [2023-12-26 22:17:28,892][105692] Updated weights for policy 0, policy_version 950588 (0.0011) [2023-12-26 22:17:28,955][105692] Updated weights for policy 0, policy_version 950598 (0.0010) [2023-12-26 22:17:29,611][105620] Updated weights for policy 1, policy_version 950735 (0.0007) [2023-12-26 22:17:29,667][105620] Updated weights for policy 1, policy_version 950745 (0.0008) [2023-12-26 22:17:29,705][105692] Updated weights for policy 0, policy_version 950608 (0.0011) [2023-12-26 22:17:29,724][105620] Updated weights for policy 1, policy_version 950755 (0.0006) [2023-12-26 22:17:29,758][105692] Updated weights for policy 0, policy_version 950618 (0.0010) [2023-12-26 22:17:29,806][105692] Updated weights for policy 0, policy_version 950628 (0.0010) [2023-12-26 22:17:30,496][105620] Updated weights for policy 1, policy_version 950765 (0.0010) [2023-12-26 22:17:30,558][105620] Updated weights for policy 1, policy_version 950775 (0.0010) [2023-12-26 22:17:30,577][105692] Updated weights for policy 0, policy_version 950638 (0.0011) [2023-12-26 22:17:30,613][105620] Updated weights for policy 1, policy_version 950785 (0.0010) [2023-12-26 22:17:30,632][105692] Updated weights for policy 0, policy_version 950648 (0.0010) [2023-12-26 22:17:30,694][105692] Updated weights for policy 0, policy_version 950658 (0.0010) [2023-12-26 22:17:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19244.3). Total num frames: 486842368. Throughput: 0: 9723.6, 1: 9585.4. Samples: 486813388. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:17:31,062][104569] Avg episode reward: [(0, '9357.604'), (1, '9356.075')] [2023-12-26 22:17:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000950664_243408896.pth... [2023-12-26 22:17:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000950792_243433472.pth... [2023-12-26 22:17:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000949544_243122176.pth [2023-12-26 22:17:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000949672_243146752.pth [2023-12-26 22:17:31,242][105620] Updated weights for policy 1, policy_version 950795 (0.0009) [2023-12-26 22:17:31,308][105620] Updated weights for policy 1, policy_version 950805 (0.0007) [2023-12-26 22:17:31,376][105620] Updated weights for policy 1, policy_version 950815 (0.0010) [2023-12-26 22:17:31,459][105692] Updated weights for policy 0, policy_version 950668 (0.0008) [2023-12-26 22:17:31,510][105692] Updated weights for policy 0, policy_version 950678 (0.0006) [2023-12-26 22:17:31,565][105692] Updated weights for policy 0, policy_version 950688 (0.0010) [2023-12-26 22:17:32,123][105620] Updated weights for policy 1, policy_version 950825 (0.0007) [2023-12-26 22:17:32,175][105620] Updated weights for policy 1, policy_version 950835 (0.0007) [2023-12-26 22:17:32,223][105620] Updated weights for policy 1, policy_version 950845 (0.0008) [2023-12-26 22:17:32,283][105620] Updated weights for policy 1, policy_version 950855 (0.0008) [2023-12-26 22:17:32,316][105692] Updated weights for policy 0, policy_version 950698 (0.0009) [2023-12-26 22:17:32,393][105692] Updated weights for policy 0, policy_version 950708 (0.0010) [2023-12-26 22:17:32,455][105692] Updated weights for policy 0, policy_version 950718 (0.0010) [2023-12-26 22:17:32,520][105692] Updated weights for policy 0, policy_version 950728 (0.0010) [2023-12-26 22:17:33,093][105692] Updated weights for policy 0, policy_version 950738 (0.0005) [2023-12-26 22:17:33,145][105620] Updated weights for policy 1, policy_version 950865 (0.0007) [2023-12-26 22:17:33,162][105692] Updated weights for policy 0, policy_version 950748 (0.0010) [2023-12-26 22:17:33,196][105620] Updated weights for policy 1, policy_version 950875 (0.0005) [2023-12-26 22:17:33,217][105692] Updated weights for policy 0, policy_version 950758 (0.0010) [2023-12-26 22:17:33,251][105620] Updated weights for policy 1, policy_version 950885 (0.0006) [2023-12-26 22:17:33,902][105620] Updated weights for policy 1, policy_version 950895 (0.0010) [2023-12-26 22:17:33,904][105692] Updated weights for policy 0, policy_version 950768 (0.0006) [2023-12-26 22:17:33,966][105620] Updated weights for policy 1, policy_version 950905 (0.0011) [2023-12-26 22:17:33,966][105692] Updated weights for policy 0, policy_version 950778 (0.0006) [2023-12-26 22:17:34,022][105692] Updated weights for policy 0, policy_version 950788 (0.0011) [2023-12-26 22:17:34,027][105620] Updated weights for policy 1, policy_version 950915 (0.0011) [2023-12-26 22:17:34,675][105692] Updated weights for policy 0, policy_version 950798 (0.0010) [2023-12-26 22:17:34,725][105692] Updated weights for policy 0, policy_version 950808 (0.0008) [2023-12-26 22:17:34,757][105620] Updated weights for policy 1, policy_version 950925 (0.0010) [2023-12-26 22:17:34,772][105692] Updated weights for policy 0, policy_version 950818 (0.0006) [2023-12-26 22:17:34,805][105620] Updated weights for policy 1, policy_version 950935 (0.0010) [2023-12-26 22:17:34,854][105620] Updated weights for policy 1, policy_version 950945 (0.0010) [2023-12-26 22:17:35,527][105692] Updated weights for policy 0, policy_version 950828 (0.0006) [2023-12-26 22:17:35,590][105620] Updated weights for policy 1, policy_version 950955 (0.0010) [2023-12-26 22:17:35,593][105692] Updated weights for policy 0, policy_version 950838 (0.0008) [2023-12-26 22:17:35,638][105620] Updated weights for policy 1, policy_version 950965 (0.0010) [2023-12-26 22:17:35,655][105692] Updated weights for policy 0, policy_version 950848 (0.0006) [2023-12-26 22:17:35,686][105620] Updated weights for policy 1, policy_version 950975 (0.0010) [2023-12-26 22:17:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19272.0). Total num frames: 486940672. Throughput: 0: 9750.8, 1: 9574.0. Samples: 486928916. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:17:36,063][104569] Avg episode reward: [(0, '9185.284'), (1, '9356.136')] [2023-12-26 22:17:36,401][105692] Updated weights for policy 0, policy_version 950858 (0.0006) [2023-12-26 22:17:36,462][105620] Updated weights for policy 1, policy_version 950985 (0.0010) [2023-12-26 22:17:36,468][105692] Updated weights for policy 0, policy_version 950868 (0.0006) [2023-12-26 22:17:36,525][105620] Updated weights for policy 1, policy_version 950995 (0.0011) [2023-12-26 22:17:36,531][105692] Updated weights for policy 0, policy_version 950878 (0.0006) [2023-12-26 22:17:36,588][105620] Updated weights for policy 1, policy_version 951005 (0.0011) [2023-12-26 22:17:36,598][105692] Updated weights for policy 0, policy_version 950888 (0.0007) [2023-12-26 22:17:36,655][105620] Updated weights for policy 1, policy_version 951015 (0.0011) [2023-12-26 22:17:37,285][105692] Updated weights for policy 0, policy_version 950898 (0.0007) [2023-12-26 22:17:37,343][105692] Updated weights for policy 0, policy_version 950908 (0.0006) [2023-12-26 22:17:37,368][105620] Updated weights for policy 1, policy_version 951025 (0.0010) [2023-12-26 22:17:37,398][105692] Updated weights for policy 0, policy_version 950918 (0.0005) [2023-12-26 22:17:37,423][105620] Updated weights for policy 1, policy_version 951035 (0.0010) [2023-12-26 22:17:37,484][105620] Updated weights for policy 1, policy_version 951045 (0.0010) [2023-12-26 22:17:38,141][105692] Updated weights for policy 0, policy_version 950928 (0.0008) [2023-12-26 22:17:38,201][105692] Updated weights for policy 0, policy_version 950938 (0.0008) [2023-12-26 22:17:38,233][105620] Updated weights for policy 1, policy_version 951055 (0.0010) [2023-12-26 22:17:38,259][105692] Updated weights for policy 0, policy_version 950948 (0.0006) [2023-12-26 22:17:38,284][105620] Updated weights for policy 1, policy_version 951065 (0.0010) [2023-12-26 22:17:38,340][105620] Updated weights for policy 1, policy_version 951075 (0.0010) [2023-12-26 22:17:39,040][105692] Updated weights for policy 0, policy_version 950958 (0.0007) [2023-12-26 22:17:39,050][105620] Updated weights for policy 1, policy_version 951085 (0.0009) [2023-12-26 22:17:39,096][105692] Updated weights for policy 0, policy_version 950968 (0.0006) [2023-12-26 22:17:39,109][105620] Updated weights for policy 1, policy_version 951095 (0.0010) [2023-12-26 22:17:39,149][105692] Updated weights for policy 0, policy_version 950978 (0.0009) [2023-12-26 22:17:39,161][105620] Updated weights for policy 1, policy_version 951105 (0.0010) [2023-12-26 22:17:39,945][105692] Updated weights for policy 0, policy_version 950988 (0.0009) [2023-12-26 22:17:39,956][105620] Updated weights for policy 1, policy_version 951115 (0.0010) [2023-12-26 22:17:40,009][105692] Updated weights for policy 0, policy_version 950998 (0.0007) [2023-12-26 22:17:40,015][105620] Updated weights for policy 1, policy_version 951125 (0.0009) [2023-12-26 22:17:40,025][105585] KL-divergence is very high: 134.8892 [2023-12-26 22:17:40,073][105585] KL-divergence is very high: 121.3532 [2023-12-26 22:17:40,074][105692] Updated weights for policy 0, policy_version 951008 (0.0006) [2023-12-26 22:17:40,079][105620] Updated weights for policy 1, policy_version 951135 (0.0008) [2023-12-26 22:17:40,691][105692] Updated weights for policy 0, policy_version 951018 (0.0008) [2023-12-26 22:17:40,745][105692] Updated weights for policy 0, policy_version 951028 (0.0007) [2023-12-26 22:17:40,797][105692] Updated weights for policy 0, policy_version 951038 (0.0009) [2023-12-26 22:17:40,858][105692] Updated weights for policy 0, policy_version 951048 (0.0009) [2023-12-26 22:17:40,918][105620] Updated weights for policy 1, policy_version 951145 (0.0007) [2023-12-26 22:17:40,982][105620] Updated weights for policy 1, policy_version 951155 (0.0010) [2023-12-26 22:17:41,042][105620] Updated weights for policy 1, policy_version 951165 (0.0009) [2023-12-26 22:17:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19244.3). Total num frames: 487030784. Throughput: 0: 9728.6, 1: 9468.8. Samples: 487041620. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:17:41,063][104569] Avg episode reward: [(0, '9090.685'), (1, '9089.271')] [2023-12-26 22:17:41,107][105620] Updated weights for policy 1, policy_version 951175 (0.0010) [2023-12-26 22:17:41,583][105692] Updated weights for policy 0, policy_version 951058 (0.0006) [2023-12-26 22:17:41,644][105692] Updated weights for policy 0, policy_version 951068 (0.0010) [2023-12-26 22:17:41,709][105692] Updated weights for policy 0, policy_version 951078 (0.0011) [2023-12-26 22:17:41,919][105620] Updated weights for policy 1, policy_version 951185 (0.0009) [2023-12-26 22:17:41,985][105620] Updated weights for policy 1, policy_version 951195 (0.0007) [2023-12-26 22:17:42,026][105586] KL-divergence is very high: 101.6196 [2023-12-26 22:17:42,052][105620] Updated weights for policy 1, policy_version 951205 (0.0008) [2023-12-26 22:17:42,496][105692] Updated weights for policy 0, policy_version 951088 (0.0010) [2023-12-26 22:17:42,559][105692] Updated weights for policy 0, policy_version 951098 (0.0011) [2023-12-26 22:17:42,631][105692] Updated weights for policy 0, policy_version 951108 (0.0011) [2023-12-26 22:17:42,834][105620] Updated weights for policy 1, policy_version 951215 (0.0008) [2023-12-26 22:17:42,894][105620] Updated weights for policy 1, policy_version 951225 (0.0008) [2023-12-26 22:17:42,962][105620] Updated weights for policy 1, policy_version 951235 (0.0009) [2023-12-26 22:17:43,341][105692] Updated weights for policy 0, policy_version 951118 (0.0008) [2023-12-26 22:17:43,396][105692] Updated weights for policy 0, policy_version 951128 (0.0005) [2023-12-26 22:17:43,444][105692] Updated weights for policy 0, policy_version 951138 (0.0005) [2023-12-26 22:17:43,752][105620] Updated weights for policy 1, policy_version 951245 (0.0009) [2023-12-26 22:17:43,806][105620] Updated weights for policy 1, policy_version 951255 (0.0009) [2023-12-26 22:17:43,868][105620] Updated weights for policy 1, policy_version 951265 (0.0009) [2023-12-26 22:17:44,088][105692] Updated weights for policy 0, policy_version 951148 (0.0005) [2023-12-26 22:17:44,151][105692] Updated weights for policy 0, policy_version 951158 (0.0006) [2023-12-26 22:17:44,216][105692] Updated weights for policy 0, policy_version 951168 (0.0006) [2023-12-26 22:17:44,628][105620] Updated weights for policy 1, policy_version 951275 (0.0009) [2023-12-26 22:17:44,686][105620] Updated weights for policy 1, policy_version 951285 (0.0009) [2023-12-26 22:17:44,754][105620] Updated weights for policy 1, policy_version 951295 (0.0009) [2023-12-26 22:17:44,907][105692] Updated weights for policy 0, policy_version 951178 (0.0008) [2023-12-26 22:17:44,965][105692] Updated weights for policy 0, policy_version 951188 (0.0008) [2023-12-26 22:17:45,018][105692] Updated weights for policy 0, policy_version 951198 (0.0009) [2023-12-26 22:17:45,069][105692] Updated weights for policy 0, policy_version 951208 (0.0006) [2023-12-26 22:17:45,530][105620] Updated weights for policy 1, policy_version 951305 (0.0008) [2023-12-26 22:17:45,596][105620] Updated weights for policy 1, policy_version 951315 (0.0009) [2023-12-26 22:17:45,651][105620] Updated weights for policy 1, policy_version 951325 (0.0009) [2023-12-26 22:17:45,697][105620] Updated weights for policy 1, policy_version 951335 (0.0008) [2023-12-26 22:17:45,807][105692] Updated weights for policy 0, policy_version 951218 (0.0009) [2023-12-26 22:17:45,862][105692] Updated weights for policy 0, policy_version 951228 (0.0009) [2023-12-26 22:17:45,863][105585] KL-divergence is very high: 306.7488 [2023-12-26 22:17:45,909][105585] KL-divergence is very high: 440.7603 [2023-12-26 22:17:45,920][105692] Updated weights for policy 0, policy_version 951238 (0.0009) [2023-12-26 22:17:46,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19244.3). Total num frames: 487129088. Throughput: 0: 9663.8, 1: 9423.5. Samples: 487096060. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:17:46,062][104569] Avg episode reward: [(0, '9175.009'), (1, '8832.676')] [2023-12-26 22:17:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000951336_243572736.pth... [2023-12-26 22:17:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000951240_243556352.pth... [2023-12-26 22:17:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000950248_243294208.pth [2023-12-26 22:17:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000950088_243261440.pth [2023-12-26 22:17:46,458][105620] Updated weights for policy 1, policy_version 951345 (0.0008) [2023-12-26 22:17:46,518][105620] Updated weights for policy 1, policy_version 951355 (0.0008) [2023-12-26 22:17:46,567][105620] Updated weights for policy 1, policy_version 951365 (0.0008) [2023-12-26 22:17:46,689][105692] Updated weights for policy 0, policy_version 951248 (0.0011) [2023-12-26 22:17:46,739][105692] Updated weights for policy 0, policy_version 951258 (0.0011) [2023-12-26 22:17:46,805][105692] Updated weights for policy 0, policy_version 951268 (0.0011) [2023-12-26 22:17:47,364][105620] Updated weights for policy 1, policy_version 951375 (0.0009) [2023-12-26 22:17:47,419][105620] Updated weights for policy 1, policy_version 951385 (0.0012) [2023-12-26 22:17:47,453][105692] Updated weights for policy 0, policy_version 951278 (0.0010) [2023-12-26 22:17:47,467][105620] Updated weights for policy 1, policy_version 951395 (0.0006) [2023-12-26 22:17:47,515][105692] Updated weights for policy 0, policy_version 951288 (0.0011) [2023-12-26 22:17:47,574][105692] Updated weights for policy 0, policy_version 951298 (0.0010) [2023-12-26 22:17:48,212][105692] Updated weights for policy 0, policy_version 951308 (0.0010) [2023-12-26 22:17:48,270][105692] Updated weights for policy 0, policy_version 951318 (0.0009) [2023-12-26 22:17:48,297][105620] Updated weights for policy 1, policy_version 951405 (0.0008) [2023-12-26 22:17:48,342][105692] Updated weights for policy 0, policy_version 951328 (0.0007) [2023-12-26 22:17:48,365][105620] Updated weights for policy 1, policy_version 951415 (0.0009) [2023-12-26 22:17:48,420][105620] Updated weights for policy 1, policy_version 951425 (0.0009) [2023-12-26 22:17:49,108][105692] Updated weights for policy 0, policy_version 951338 (0.0008) [2023-12-26 22:17:49,162][105620] Updated weights for policy 1, policy_version 951435 (0.0008) [2023-12-26 22:17:49,168][105692] Updated weights for policy 0, policy_version 951348 (0.0008) [2023-12-26 22:17:49,224][105620] Updated weights for policy 1, policy_version 951445 (0.0008) [2023-12-26 22:17:49,231][105692] Updated weights for policy 0, policy_version 951358 (0.0007) [2023-12-26 22:17:49,289][105620] Updated weights for policy 1, policy_version 951455 (0.0009) [2023-12-26 22:17:49,291][105692] Updated weights for policy 0, policy_version 951368 (0.0006) [2023-12-26 22:17:50,011][105692] Updated weights for policy 0, policy_version 951378 (0.0008) [2023-12-26 22:17:50,063][105692] Updated weights for policy 0, policy_version 951388 (0.0008) [2023-12-26 22:17:50,082][105620] Updated weights for policy 1, policy_version 951465 (0.0008) [2023-12-26 22:17:50,117][105692] Updated weights for policy 0, policy_version 951398 (0.0008) [2023-12-26 22:17:50,142][105620] Updated weights for policy 1, policy_version 951475 (0.0009) [2023-12-26 22:17:50,218][105620] Updated weights for policy 1, policy_version 951485 (0.0010) [2023-12-26 22:17:50,281][105620] Updated weights for policy 1, policy_version 951495 (0.0008) [2023-12-26 22:17:50,805][105692] Updated weights for policy 0, policy_version 951408 (0.0010) [2023-12-26 22:17:50,858][105692] Updated weights for policy 0, policy_version 951418 (0.0011) [2023-12-26 22:17:50,909][105692] Updated weights for policy 0, policy_version 951428 (0.0009) [2023-12-26 22:17:51,046][105620] Updated weights for policy 1, policy_version 951505 (0.0009) [2023-12-26 22:17:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19216.5). Total num frames: 487219200. Throughput: 0: 9742.6, 1: 9394.6. Samples: 487209336. Policy #0 lag: (min: 5.0, avg: 20.7, max: 37.0) [2023-12-26 22:17:51,063][104569] Avg episode reward: [(0, '9093.098'), (1, '9008.487')] [2023-12-26 22:17:51,109][105620] Updated weights for policy 1, policy_version 951515 (0.0008) [2023-12-26 22:17:51,174][105620] Updated weights for policy 1, policy_version 951526 (0.0008) [2023-12-26 22:17:51,638][105692] Updated weights for policy 0, policy_version 951438 (0.0007) [2023-12-26 22:17:51,702][105692] Updated weights for policy 0, policy_version 951448 (0.0008) [2023-12-26 22:17:51,778][105692] Updated weights for policy 0, policy_version 951458 (0.0007) [2023-12-26 22:17:51,935][105620] Updated weights for policy 1, policy_version 951536 (0.0007) [2023-12-26 22:17:51,993][105620] Updated weights for policy 1, policy_version 951547 (0.0010) [2023-12-26 22:17:52,063][105620] Updated weights for policy 1, policy_version 951557 (0.0009) [2023-12-26 22:17:52,392][105692] Updated weights for policy 0, policy_version 951468 (0.0006) [2023-12-26 22:17:52,453][105692] Updated weights for policy 0, policy_version 951478 (0.0008) [2023-12-26 22:17:52,512][105692] Updated weights for policy 0, policy_version 951488 (0.0009) [2023-12-26 22:17:52,863][105620] Updated weights for policy 1, policy_version 951567 (0.0008) [2023-12-26 22:17:52,927][105620] Updated weights for policy 1, policy_version 951577 (0.0009) [2023-12-26 22:17:52,982][105620] Updated weights for policy 1, policy_version 951587 (0.0009) [2023-12-26 22:17:53,159][105692] Updated weights for policy 0, policy_version 951498 (0.0006) [2023-12-26 22:17:53,212][105692] Updated weights for policy 0, policy_version 951508 (0.0006) [2023-12-26 22:17:53,257][105585] KL-divergence is very high: 112.0037 [2023-12-26 22:17:53,263][105692] Updated weights for policy 0, policy_version 951518 (0.0009) [2023-12-26 22:17:53,302][105585] KL-divergence is very high: 235.1859 [2023-12-26 22:17:53,316][105692] Updated weights for policy 0, policy_version 951528 (0.0007) [2023-12-26 22:17:53,778][105620] Updated weights for policy 1, policy_version 951597 (0.0007) [2023-12-26 22:17:53,842][105620] Updated weights for policy 1, policy_version 951607 (0.0006) [2023-12-26 22:17:53,908][105620] Updated weights for policy 1, policy_version 951617 (0.0009) [2023-12-26 22:17:54,061][105692] Updated weights for policy 0, policy_version 951538 (0.0005) [2023-12-26 22:17:54,112][105692] Updated weights for policy 0, policy_version 951548 (0.0006) [2023-12-26 22:17:54,164][105692] Updated weights for policy 0, policy_version 951558 (0.0009) [2023-12-26 22:17:54,612][105620] Updated weights for policy 1, policy_version 951627 (0.0009) [2023-12-26 22:17:54,674][105620] Updated weights for policy 1, policy_version 951637 (0.0009) [2023-12-26 22:17:54,738][105620] Updated weights for policy 1, policy_version 951647 (0.0009) [2023-12-26 22:17:54,891][105692] Updated weights for policy 0, policy_version 951568 (0.0009) [2023-12-26 22:17:54,939][105692] Updated weights for policy 0, policy_version 951578 (0.0009) [2023-12-26 22:17:55,001][105692] Updated weights for policy 0, policy_version 951588 (0.0008) [2023-12-26 22:17:55,486][105620] Updated weights for policy 1, policy_version 951657 (0.0009) [2023-12-26 22:17:55,551][105620] Updated weights for policy 1, policy_version 951667 (0.0009) [2023-12-26 22:17:55,621][105620] Updated weights for policy 1, policy_version 951677 (0.0010) [2023-12-26 22:17:55,689][105620] Updated weights for policy 1, policy_version 951687 (0.0010) [2023-12-26 22:17:55,724][105692] Updated weights for policy 0, policy_version 951598 (0.0007) [2023-12-26 22:17:55,783][105692] Updated weights for policy 0, policy_version 951608 (0.0007) [2023-12-26 22:17:55,848][105692] Updated weights for policy 0, policy_version 951618 (0.0006) [2023-12-26 22:17:56,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.1, 300 sec: 19216.5). Total num frames: 487317504. Throughput: 0: 9810.9, 1: 9335.8. Samples: 487323676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:17:56,063][104569] Avg episode reward: [(0, '8918.377'), (1, '9176.037')] [2023-12-26 22:17:56,385][105692] Updated weights for policy 0, policy_version 951628 (0.0008) [2023-12-26 22:17:56,439][105692] Updated weights for policy 0, policy_version 951638 (0.0005) [2023-12-26 22:17:56,463][105620] Updated weights for policy 1, policy_version 951697 (0.0009) [2023-12-26 22:17:56,489][105692] Updated weights for policy 0, policy_version 951648 (0.0008) [2023-12-26 22:17:56,519][105620] Updated weights for policy 1, policy_version 951707 (0.0006) [2023-12-26 22:17:56,573][105620] Updated weights for policy 1, policy_version 951717 (0.0008) [2023-12-26 22:17:57,179][105692] Updated weights for policy 0, policy_version 951658 (0.0009) [2023-12-26 22:17:57,229][105692] Updated weights for policy 0, policy_version 951668 (0.0010) [2023-12-26 22:17:57,277][105692] Updated weights for policy 0, policy_version 951678 (0.0010) [2023-12-26 22:17:57,328][105692] Updated weights for policy 0, policy_version 951688 (0.0010) [2023-12-26 22:17:57,342][105620] Updated weights for policy 1, policy_version 951727 (0.0007) [2023-12-26 22:17:57,397][105620] Updated weights for policy 1, policy_version 951737 (0.0008) [2023-12-26 22:17:57,444][105620] Updated weights for policy 1, policy_version 951747 (0.0008) [2023-12-26 22:17:57,993][105692] Updated weights for policy 0, policy_version 951698 (0.0010) [2023-12-26 22:17:58,047][105692] Updated weights for policy 0, policy_version 951708 (0.0010) [2023-12-26 22:17:58,094][105692] Updated weights for policy 0, policy_version 951718 (0.0010) [2023-12-26 22:17:58,251][105620] Updated weights for policy 1, policy_version 951757 (0.0008) [2023-12-26 22:17:58,313][105620] Updated weights for policy 1, policy_version 951767 (0.0007) [2023-12-26 22:17:58,382][105620] Updated weights for policy 1, policy_version 951777 (0.0008) [2023-12-26 22:17:58,964][105692] Updated weights for policy 0, policy_version 951728 (0.0007) [2023-12-26 22:17:59,033][105692] Updated weights for policy 0, policy_version 951738 (0.0009) [2023-12-26 22:17:59,101][105692] Updated weights for policy 0, policy_version 951748 (0.0009) [2023-12-26 22:17:59,204][105620] Updated weights for policy 1, policy_version 951787 (0.0008) [2023-12-26 22:17:59,278][105620] Updated weights for policy 1, policy_version 951797 (0.0008) [2023-12-26 22:17:59,345][105620] Updated weights for policy 1, policy_version 951807 (0.0008) [2023-12-26 22:17:59,776][105692] Updated weights for policy 0, policy_version 951758 (0.0006) [2023-12-26 22:17:59,848][105692] Updated weights for policy 0, policy_version 951768 (0.0007) [2023-12-26 22:17:59,902][105692] Updated weights for policy 0, policy_version 951778 (0.0008) [2023-12-26 22:18:00,075][105620] Updated weights for policy 1, policy_version 951817 (0.0009) [2023-12-26 22:18:00,140][105620] Updated weights for policy 1, policy_version 951827 (0.0007) [2023-12-26 22:18:00,210][105620] Updated weights for policy 1, policy_version 951837 (0.0006) [2023-12-26 22:18:00,278][105620] Updated weights for policy 1, policy_version 951847 (0.0009) [2023-12-26 22:18:00,640][105692] Updated weights for policy 0, policy_version 951788 (0.0009) [2023-12-26 22:18:00,694][105692] Updated weights for policy 0, policy_version 951798 (0.0006) [2023-12-26 22:18:00,745][105692] Updated weights for policy 0, policy_version 951808 (0.0005) [2023-12-26 22:18:00,981][105620] Updated weights for policy 1, policy_version 951857 (0.0010) [2023-12-26 22:18:01,043][105620] Updated weights for policy 1, policy_version 951867 (0.0011) [2023-12-26 22:18:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19188.7). Total num frames: 487407616. Throughput: 0: 9909.9, 1: 9273.2. Samples: 487381612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:18:01,062][104569] Avg episode reward: [(0, '8820.036'), (1, '9089.362')] [2023-12-26 22:18:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000951816_243703808.pth... [2023-12-26 22:18:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000950664_243408896.pth [2023-12-26 22:18:01,103][105620] Updated weights for policy 1, policy_version 951877 (0.0011) [2023-12-26 22:18:01,126][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000951880_243712000.pth... [2023-12-26 22:18:01,130][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000950792_243433472.pth [2023-12-26 22:18:01,437][105692] Updated weights for policy 0, policy_version 951818 (0.0005) [2023-12-26 22:18:01,501][105692] Updated weights for policy 0, policy_version 951828 (0.0005) [2023-12-26 22:18:01,567][105692] Updated weights for policy 0, policy_version 951838 (0.0007) [2023-12-26 22:18:01,636][105692] Updated weights for policy 0, policy_version 951848 (0.0008) [2023-12-26 22:18:01,902][105620] Updated weights for policy 1, policy_version 951887 (0.0007) [2023-12-26 22:18:01,955][105620] Updated weights for policy 1, policy_version 951897 (0.0009) [2023-12-26 22:18:02,011][105620] Updated weights for policy 1, policy_version 951907 (0.0010) [2023-12-26 22:18:02,198][105692] Updated weights for policy 0, policy_version 951858 (0.0006) [2023-12-26 22:18:02,261][105692] Updated weights for policy 0, policy_version 951868 (0.0009) [2023-12-26 22:18:02,328][105692] Updated weights for policy 0, policy_version 951878 (0.0011) [2023-12-26 22:18:02,750][105620] Updated weights for policy 1, policy_version 951918 (0.0010) [2023-12-26 22:18:02,806][105620] Updated weights for policy 1, policy_version 951928 (0.0009) [2023-12-26 22:18:02,858][105620] Updated weights for policy 1, policy_version 951938 (0.0009) [2023-12-26 22:18:02,917][105692] Updated weights for policy 0, policy_version 951888 (0.0009) [2023-12-26 22:18:02,977][105692] Updated weights for policy 0, policy_version 951898 (0.0008) [2023-12-26 22:18:03,048][105692] Updated weights for policy 0, policy_version 951908 (0.0005) [2023-12-26 22:18:03,515][105620] Updated weights for policy 1, policy_version 951948 (0.0007) [2023-12-26 22:18:03,574][105620] Updated weights for policy 1, policy_version 951958 (0.0008) [2023-12-26 22:18:03,630][105620] Updated weights for policy 1, policy_version 951969 (0.0010) [2023-12-26 22:18:03,709][105692] Updated weights for policy 0, policy_version 951918 (0.0009) [2023-12-26 22:18:03,768][105692] Updated weights for policy 0, policy_version 951928 (0.0009) [2023-12-26 22:18:03,829][105692] Updated weights for policy 0, policy_version 951938 (0.0010) [2023-12-26 22:18:04,269][105620] Updated weights for policy 1, policy_version 951979 (0.0009) [2023-12-26 22:18:04,339][105620] Updated weights for policy 1, policy_version 951989 (0.0010) [2023-12-26 22:18:04,403][105620] Updated weights for policy 1, policy_version 951999 (0.0009) [2023-12-26 22:18:04,605][105692] Updated weights for policy 0, policy_version 951948 (0.0010) [2023-12-26 22:18:04,663][105692] Updated weights for policy 0, policy_version 951958 (0.0009) [2023-12-26 22:18:04,713][105692] Updated weights for policy 0, policy_version 951968 (0.0005) [2023-12-26 22:18:05,203][105620] Updated weights for policy 1, policy_version 952009 (0.0010) [2023-12-26 22:18:05,267][105620] Updated weights for policy 1, policy_version 952019 (0.0008) [2023-12-26 22:18:05,331][105620] Updated weights for policy 1, policy_version 952029 (0.0009) [2023-12-26 22:18:05,391][105620] Updated weights for policy 1, policy_version 952039 (0.0009) [2023-12-26 22:18:05,391][105692] Updated weights for policy 0, policy_version 951978 (0.0006) [2023-12-26 22:18:05,441][105692] Updated weights for policy 0, policy_version 951988 (0.0009) [2023-12-26 22:18:05,498][105692] Updated weights for policy 0, policy_version 951998 (0.0008) [2023-12-26 22:18:05,558][105692] Updated weights for policy 0, policy_version 952008 (0.0007) [2023-12-26 22:18:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.6, 300 sec: 19188.7). Total num frames: 487505920. Throughput: 0: 9934.8, 1: 9257.3. Samples: 487498232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:18:06,063][104569] Avg episode reward: [(0, '8995.567'), (1, '8999.527')] [2023-12-26 22:18:06,153][105620] Updated weights for policy 1, policy_version 952049 (0.0009) [2023-12-26 22:18:06,206][105620] Updated weights for policy 1, policy_version 952059 (0.0007) [2023-12-26 22:18:06,276][105620] Updated weights for policy 1, policy_version 952069 (0.0006) [2023-12-26 22:18:06,314][105692] Updated weights for policy 0, policy_version 952018 (0.0008) [2023-12-26 22:18:06,381][105692] Updated weights for policy 0, policy_version 952028 (0.0009) [2023-12-26 22:18:06,444][105692] Updated weights for policy 0, policy_version 952038 (0.0009) [2023-12-26 22:18:06,940][105620] Updated weights for policy 1, policy_version 952079 (0.0006) [2023-12-26 22:18:07,002][105620] Updated weights for policy 1, policy_version 952089 (0.0011) [2023-12-26 22:18:07,065][105620] Updated weights for policy 1, policy_version 952099 (0.0010) [2023-12-26 22:18:07,243][105692] Updated weights for policy 0, policy_version 952048 (0.0007) [2023-12-26 22:18:07,292][105692] Updated weights for policy 0, policy_version 952058 (0.0008) [2023-12-26 22:18:07,348][105692] Updated weights for policy 0, policy_version 952068 (0.0008) [2023-12-26 22:18:07,702][105620] Updated weights for policy 1, policy_version 952109 (0.0007) [2023-12-26 22:18:07,763][105620] Updated weights for policy 1, policy_version 952119 (0.0010) [2023-12-26 22:18:07,825][105620] Updated weights for policy 1, policy_version 952129 (0.0011) [2023-12-26 22:18:08,157][105692] Updated weights for policy 0, policy_version 952078 (0.0009) [2023-12-26 22:18:08,212][105692] Updated weights for policy 0, policy_version 952089 (0.0010) [2023-12-26 22:18:08,265][105692] Updated weights for policy 0, policy_version 952099 (0.0010) [2023-12-26 22:18:08,428][105620] Updated weights for policy 1, policy_version 952139 (0.0009) [2023-12-26 22:18:08,485][105620] Updated weights for policy 1, policy_version 952149 (0.0005) [2023-12-26 22:18:08,547][105620] Updated weights for policy 1, policy_version 952159 (0.0009) [2023-12-26 22:18:09,073][105692] Updated weights for policy 0, policy_version 952109 (0.0010) [2023-12-26 22:18:09,123][105692] Updated weights for policy 0, policy_version 952119 (0.0009) [2023-12-26 22:18:09,175][105692] Updated weights for policy 0, policy_version 952129 (0.0009) [2023-12-26 22:18:09,234][105620] Updated weights for policy 1, policy_version 952169 (0.0010) [2023-12-26 22:18:09,302][105620] Updated weights for policy 1, policy_version 952179 (0.0009) [2023-12-26 22:18:09,373][105620] Updated weights for policy 1, policy_version 952189 (0.0009) [2023-12-26 22:18:09,446][105620] Updated weights for policy 1, policy_version 952199 (0.0009) [2023-12-26 22:18:10,042][105692] Updated weights for policy 0, policy_version 952139 (0.0009) [2023-12-26 22:18:10,099][105692] Updated weights for policy 0, policy_version 952149 (0.0009) [2023-12-26 22:18:10,145][105620] Updated weights for policy 1, policy_version 952209 (0.0008) [2023-12-26 22:18:10,157][105692] Updated weights for policy 0, policy_version 952159 (0.0008) [2023-12-26 22:18:10,196][105620] Updated weights for policy 1, policy_version 952219 (0.0007) [2023-12-26 22:18:10,262][105620] Updated weights for policy 1, policy_version 952229 (0.0008) [2023-12-26 22:18:10,889][105692] Updated weights for policy 0, policy_version 952169 (0.0008) [2023-12-26 22:18:10,952][105692] Updated weights for policy 0, policy_version 952179 (0.0009) [2023-12-26 22:18:11,040][105692] Updated weights for policy 0, policy_version 952189 (0.0011) [2023-12-26 22:18:11,055][105620] Updated weights for policy 1, policy_version 952239 (0.0010) [2023-12-26 22:18:11,062][104569] Fps is (10 sec: 18841.4, 60 sec: 18978.1, 300 sec: 19160.9). Total num frames: 487596032. Throughput: 0: 9822.7, 1: 9286.7. Samples: 487611908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:18:11,063][104569] Avg episode reward: [(0, '9173.506'), (1, '8999.095')] [2023-12-26 22:18:11,101][105692] Updated weights for policy 0, policy_version 952199 (0.0008) [2023-12-26 22:18:11,116][105620] Updated weights for policy 1, policy_version 952249 (0.0010) [2023-12-26 22:18:11,184][105620] Updated weights for policy 1, policy_version 952259 (0.0011) [2023-12-26 22:18:11,844][105692] Updated weights for policy 0, policy_version 952209 (0.0008) [2023-12-26 22:18:11,912][105692] Updated weights for policy 0, policy_version 952219 (0.0009) [2023-12-26 22:18:11,983][105692] Updated weights for policy 0, policy_version 952229 (0.0009) [2023-12-26 22:18:12,008][105620] Updated weights for policy 1, policy_version 952269 (0.0010) [2023-12-26 22:18:12,073][105620] Updated weights for policy 1, policy_version 952279 (0.0007) [2023-12-26 22:18:12,137][105620] Updated weights for policy 1, policy_version 952289 (0.0009) [2023-12-26 22:18:12,644][105692] Updated weights for policy 0, policy_version 952239 (0.0008) [2023-12-26 22:18:12,692][105692] Updated weights for policy 0, policy_version 952249 (0.0005) [2023-12-26 22:18:12,738][105692] Updated weights for policy 0, policy_version 952259 (0.0006) [2023-12-26 22:18:12,940][105620] Updated weights for policy 1, policy_version 952299 (0.0008) [2023-12-26 22:18:12,994][105620] Updated weights for policy 1, policy_version 952309 (0.0008) [2023-12-26 22:18:13,052][105620] Updated weights for policy 1, policy_version 952319 (0.0009) [2023-12-26 22:18:13,472][105692] Updated weights for policy 0, policy_version 952269 (0.0007) [2023-12-26 22:18:13,530][105692] Updated weights for policy 0, policy_version 952279 (0.0010) [2023-12-26 22:18:13,590][105692] Updated weights for policy 0, policy_version 952290 (0.0010) [2023-12-26 22:18:13,740][105620] Updated weights for policy 1, policy_version 952329 (0.0009) [2023-12-26 22:18:13,801][105620] Updated weights for policy 1, policy_version 952339 (0.0009) [2023-12-26 22:18:13,858][105620] Updated weights for policy 1, policy_version 952349 (0.0010) [2023-12-26 22:18:13,911][105620] Updated weights for policy 1, policy_version 952359 (0.0008) [2023-12-26 22:18:14,389][105692] Updated weights for policy 0, policy_version 952301 (0.0010) [2023-12-26 22:18:14,450][105692] Updated weights for policy 0, policy_version 952311 (0.0009) [2023-12-26 22:18:14,503][105692] Updated weights for policy 0, policy_version 952321 (0.0010) [2023-12-26 22:18:14,648][105620] Updated weights for policy 1, policy_version 952369 (0.0009) [2023-12-26 22:18:14,713][105620] Updated weights for policy 1, policy_version 952379 (0.0007) [2023-12-26 22:18:14,775][105620] Updated weights for policy 1, policy_version 952389 (0.0008) [2023-12-26 22:18:15,220][105692] Updated weights for policy 0, policy_version 952331 (0.0009) [2023-12-26 22:18:15,287][105692] Updated weights for policy 0, policy_version 952341 (0.0009) [2023-12-26 22:18:15,347][105692] Updated weights for policy 0, policy_version 952351 (0.0009) [2023-12-26 22:18:15,495][105620] Updated weights for policy 1, policy_version 952399 (0.0008) [2023-12-26 22:18:15,556][105620] Updated weights for policy 1, policy_version 952409 (0.0009) [2023-12-26 22:18:15,622][105620] Updated weights for policy 1, policy_version 952419 (0.0008) [2023-12-26 22:18:16,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19251.2, 300 sec: 19161.0). Total num frames: 487694336. Throughput: 0: 9710.6, 1: 9262.1. Samples: 487667156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:18:16,062][104569] Avg episode reward: [(0, '8991.511'), (1, '9172.803')] [2023-12-26 22:18:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000952360_243843072.pth... [2023-12-26 22:18:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000952424_243851264.pth... [2023-12-26 22:18:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000951240_243556352.pth [2023-12-26 22:18:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000951336_243572736.pth [2023-12-26 22:18:16,151][105692] Updated weights for policy 0, policy_version 952361 (0.0007) [2023-12-26 22:18:16,209][105692] Updated weights for policy 0, policy_version 952371 (0.0009) [2023-12-26 22:18:16,264][105692] Updated weights for policy 0, policy_version 952381 (0.0007) [2023-12-26 22:18:16,265][105620] Updated weights for policy 1, policy_version 952429 (0.0008) [2023-12-26 22:18:16,311][105692] Updated weights for policy 0, policy_version 952391 (0.0007) [2023-12-26 22:18:16,326][105620] Updated weights for policy 1, policy_version 952439 (0.0007) [2023-12-26 22:18:16,387][105620] Updated weights for policy 1, policy_version 952449 (0.0009) [2023-12-26 22:18:17,051][105620] Updated weights for policy 1, policy_version 952459 (0.0007) [2023-12-26 22:18:17,117][105620] Updated weights for policy 1, policy_version 952469 (0.0009) [2023-12-26 22:18:17,127][105692] Updated weights for policy 0, policy_version 952401 (0.0007) [2023-12-26 22:18:17,177][105620] Updated weights for policy 1, policy_version 952479 (0.0008) [2023-12-26 22:18:17,183][105692] Updated weights for policy 0, policy_version 952411 (0.0006) [2023-12-26 22:18:17,239][105692] Updated weights for policy 0, policy_version 952421 (0.0006) [2023-12-26 22:18:17,879][105692] Updated weights for policy 0, policy_version 952431 (0.0009) [2023-12-26 22:18:17,943][105692] Updated weights for policy 0, policy_version 952441 (0.0008) [2023-12-26 22:18:17,945][105620] Updated weights for policy 1, policy_version 952489 (0.0007) [2023-12-26 22:18:18,002][105692] Updated weights for policy 0, policy_version 952451 (0.0006) [2023-12-26 22:18:18,007][105620] Updated weights for policy 1, policy_version 952499 (0.0007) [2023-12-26 22:18:18,075][105620] Updated weights for policy 1, policy_version 952509 (0.0010) [2023-12-26 22:18:18,124][105620] Updated weights for policy 1, policy_version 952519 (0.0009) [2023-12-26 22:18:18,666][105692] Updated weights for policy 0, policy_version 952461 (0.0007) [2023-12-26 22:18:18,695][105585] KL-divergence is very high: 105.0147 [2023-12-26 22:18:18,722][105692] Updated weights for policy 0, policy_version 952471 (0.0010) [2023-12-26 22:18:18,739][105585] KL-divergence is very high: 114.0979 [2023-12-26 22:18:18,782][105692] Updated weights for policy 0, policy_version 952481 (0.0009) [2023-12-26 22:18:18,921][105620] Updated weights for policy 1, policy_version 952529 (0.0009) [2023-12-26 22:18:18,976][105620] Updated weights for policy 1, policy_version 952539 (0.0009) [2023-12-26 22:18:19,027][105620] Updated weights for policy 1, policy_version 952549 (0.0008) [2023-12-26 22:18:19,597][105692] Updated weights for policy 0, policy_version 952491 (0.0009) [2023-12-26 22:18:19,656][105692] Updated weights for policy 0, policy_version 952501 (0.0009) [2023-12-26 22:18:19,721][105692] Updated weights for policy 0, policy_version 952511 (0.0009) [2023-12-26 22:18:19,813][105620] Updated weights for policy 1, policy_version 952559 (0.0009) [2023-12-26 22:18:19,880][105620] Updated weights for policy 1, policy_version 952569 (0.0009) [2023-12-26 22:18:19,938][105620] Updated weights for policy 1, policy_version 952579 (0.0009) [2023-12-26 22:18:20,521][105692] Updated weights for policy 0, policy_version 952521 (0.0009) [2023-12-26 22:18:20,579][105692] Updated weights for policy 0, policy_version 952531 (0.0009) [2023-12-26 22:18:20,641][105692] Updated weights for policy 0, policy_version 952541 (0.0009) [2023-12-26 22:18:20,674][105620] Updated weights for policy 1, policy_version 952589 (0.0007) [2023-12-26 22:18:20,705][105692] Updated weights for policy 0, policy_version 952551 (0.0008) [2023-12-26 22:18:20,726][105620] Updated weights for policy 1, policy_version 952599 (0.0006) [2023-12-26 22:18:20,779][105620] Updated weights for policy 1, policy_version 952609 (0.0009) [2023-12-26 22:18:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19188.7). Total num frames: 487792640. Throughput: 0: 9656.0, 1: 9268.5. Samples: 487780516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:18:21,063][104569] Avg episode reward: [(0, '8728.130'), (1, '9173.317')] [2023-12-26 22:18:21,544][105692] Updated weights for policy 0, policy_version 952561 (0.0009) [2023-12-26 22:18:21,582][105620] Updated weights for policy 1, policy_version 952619 (0.0008) [2023-12-26 22:18:21,605][105692] Updated weights for policy 0, policy_version 952571 (0.0009) [2023-12-26 22:18:21,660][105620] Updated weights for policy 1, policy_version 952629 (0.0007) [2023-12-26 22:18:21,670][105692] Updated weights for policy 0, policy_version 952581 (0.0008) [2023-12-26 22:18:21,719][105620] Updated weights for policy 1, policy_version 952639 (0.0008) [2023-12-26 22:18:22,456][105692] Updated weights for policy 0, policy_version 952591 (0.0008) [2023-12-26 22:18:22,478][105620] Updated weights for policy 1, policy_version 952649 (0.0008) [2023-12-26 22:18:22,523][105692] Updated weights for policy 0, policy_version 952601 (0.0008) [2023-12-26 22:18:22,537][105620] Updated weights for policy 1, policy_version 952659 (0.0008) [2023-12-26 22:18:22,588][105692] Updated weights for policy 0, policy_version 952611 (0.0007) [2023-12-26 22:18:22,594][105620] Updated weights for policy 1, policy_version 952669 (0.0008) [2023-12-26 22:18:22,656][105620] Updated weights for policy 1, policy_version 952679 (0.0008) [2023-12-26 22:18:23,256][105620] Updated weights for policy 1, policy_version 952689 (0.0006) [2023-12-26 22:18:23,312][105620] Updated weights for policy 1, policy_version 952699 (0.0007) [2023-12-26 22:18:23,373][105620] Updated weights for policy 1, policy_version 952709 (0.0005) [2023-12-26 22:18:23,456][105692] Updated weights for policy 0, policy_version 952621 (0.0008) [2023-12-26 22:18:23,508][105692] Updated weights for policy 0, policy_version 952631 (0.0009) [2023-12-26 22:18:23,555][105692] Updated weights for policy 0, policy_version 952641 (0.0009) [2023-12-26 22:18:23,992][105620] Updated weights for policy 1, policy_version 952719 (0.0008) [2023-12-26 22:18:24,051][105620] Updated weights for policy 1, policy_version 952729 (0.0009) [2023-12-26 22:18:24,102][105620] Updated weights for policy 1, policy_version 952739 (0.0009) [2023-12-26 22:18:24,340][105692] Updated weights for policy 0, policy_version 952651 (0.0009) [2023-12-26 22:18:24,404][105692] Updated weights for policy 0, policy_version 952661 (0.0009) [2023-12-26 22:18:24,468][105692] Updated weights for policy 0, policy_version 952671 (0.0009) [2023-12-26 22:18:24,871][105620] Updated weights for policy 1, policy_version 952749 (0.0010) [2023-12-26 22:18:24,920][105620] Updated weights for policy 1, policy_version 952759 (0.0008) [2023-12-26 22:18:24,970][105620] Updated weights for policy 1, policy_version 952769 (0.0009) [2023-12-26 22:18:25,177][105692] Updated weights for policy 0, policy_version 952681 (0.0010) [2023-12-26 22:18:25,236][105692] Updated weights for policy 0, policy_version 952691 (0.0009) [2023-12-26 22:18:25,294][105692] Updated weights for policy 0, policy_version 952701 (0.0009) [2023-12-26 22:18:25,346][105692] Updated weights for policy 0, policy_version 952711 (0.0009) [2023-12-26 22:18:25,662][105620] Updated weights for policy 1, policy_version 952779 (0.0009) [2023-12-26 22:18:25,712][105620] Updated weights for policy 1, policy_version 952789 (0.0008) [2023-12-26 22:18:25,762][105620] Updated weights for policy 1, policy_version 952799 (0.0009) [2023-12-26 22:18:26,062][104569] Fps is (10 sec: 18841.0, 60 sec: 18978.1, 300 sec: 19133.2). Total num frames: 487882752. Throughput: 0: 9551.3, 1: 9336.7. Samples: 487891584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:18:26,063][104569] Avg episode reward: [(0, '8823.129'), (1, '9085.095')] [2023-12-26 22:18:26,132][105692] Updated weights for policy 0, policy_version 952721 (0.0008) [2023-12-26 22:18:26,201][105692] Updated weights for policy 0, policy_version 952731 (0.0009) [2023-12-26 22:18:26,275][105692] Updated weights for policy 0, policy_version 952741 (0.0010) [2023-12-26 22:18:26,548][105620] Updated weights for policy 1, policy_version 952809 (0.0009) [2023-12-26 22:18:26,607][105620] Updated weights for policy 1, policy_version 952819 (0.0009) [2023-12-26 22:18:26,675][105620] Updated weights for policy 1, policy_version 952829 (0.0009) [2023-12-26 22:18:26,736][105620] Updated weights for policy 1, policy_version 952839 (0.0009) [2023-12-26 22:18:26,960][105692] Updated weights for policy 0, policy_version 952751 (0.0009) [2023-12-26 22:18:27,007][105692] Updated weights for policy 0, policy_version 952761 (0.0009) [2023-12-26 22:18:27,060][105692] Updated weights for policy 0, policy_version 952771 (0.0009) [2023-12-26 22:18:27,468][105620] Updated weights for policy 1, policy_version 952849 (0.0008) [2023-12-26 22:18:27,531][105620] Updated weights for policy 1, policy_version 952859 (0.0009) [2023-12-26 22:18:27,588][105620] Updated weights for policy 1, policy_version 952869 (0.0008) [2023-12-26 22:18:27,829][105692] Updated weights for policy 0, policy_version 952781 (0.0009) [2023-12-26 22:18:27,883][105692] Updated weights for policy 0, policy_version 952792 (0.0010) [2023-12-26 22:18:27,930][105692] Updated weights for policy 0, policy_version 952802 (0.0009) [2023-12-26 22:18:28,308][105620] Updated weights for policy 1, policy_version 952879 (0.0009) [2023-12-26 22:18:28,377][105620] Updated weights for policy 1, policy_version 952889 (0.0009) [2023-12-26 22:18:28,440][105620] Updated weights for policy 1, policy_version 952899 (0.0009) [2023-12-26 22:18:28,718][105692] Updated weights for policy 0, policy_version 952812 (0.0010) [2023-12-26 22:18:28,778][105692] Updated weights for policy 0, policy_version 952823 (0.0009) [2023-12-26 22:18:28,829][105692] Updated weights for policy 0, policy_version 952833 (0.0009) [2023-12-26 22:18:29,130][105620] Updated weights for policy 1, policy_version 952909 (0.0008) [2023-12-26 22:18:29,190][105620] Updated weights for policy 1, policy_version 952919 (0.0009) [2023-12-26 22:18:29,254][105620] Updated weights for policy 1, policy_version 952929 (0.0008) [2023-12-26 22:18:29,650][105692] Updated weights for policy 0, policy_version 952843 (0.0009) [2023-12-26 22:18:29,713][105692] Updated weights for policy 0, policy_version 952853 (0.0009) [2023-12-26 22:18:29,771][105692] Updated weights for policy 0, policy_version 952863 (0.0009) [2023-12-26 22:18:29,978][105620] Updated weights for policy 1, policy_version 952939 (0.0008) [2023-12-26 22:18:30,029][105620] Updated weights for policy 1, policy_version 952949 (0.0009) [2023-12-26 22:18:30,079][105620] Updated weights for policy 1, policy_version 952959 (0.0008) [2023-12-26 22:18:30,527][105692] Updated weights for policy 0, policy_version 952873 (0.0009) [2023-12-26 22:18:30,586][105692] Updated weights for policy 0, policy_version 952883 (0.0009) [2023-12-26 22:18:30,650][105692] Updated weights for policy 0, policy_version 952893 (0.0010) [2023-12-26 22:18:30,704][105692] Updated weights for policy 0, policy_version 952903 (0.0009) [2023-12-26 22:18:30,840][105620] Updated weights for policy 1, policy_version 952969 (0.0008) [2023-12-26 22:18:30,897][105620] Updated weights for policy 1, policy_version 952979 (0.0009) [2023-12-26 22:18:30,951][105620] Updated weights for policy 1, policy_version 952989 (0.0010) [2023-12-26 22:18:31,008][105620] Updated weights for policy 1, policy_version 952999 (0.0009) [2023-12-26 22:18:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 18978.2, 300 sec: 19105.4). Total num frames: 487981056. Throughput: 0: 9547.9, 1: 9393.7. Samples: 487948432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:18:31,063][104569] Avg episode reward: [(0, '8999.923'), (1, '8993.091')] [2023-12-26 22:18:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000952904_243982336.pth... [2023-12-26 22:18:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000953000_243998720.pth... [2023-12-26 22:18:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000951816_243703808.pth [2023-12-26 22:18:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000951880_243712000.pth [2023-12-26 22:18:31,363][105692] Updated weights for policy 0, policy_version 952913 (0.0010) [2023-12-26 22:18:31,430][105692] Updated weights for policy 0, policy_version 952923 (0.0008) [2023-12-26 22:18:31,492][105692] Updated weights for policy 0, policy_version 952933 (0.0009) [2023-12-26 22:18:31,835][105620] Updated weights for policy 1, policy_version 953009 (0.0009) [2023-12-26 22:18:31,897][105620] Updated weights for policy 1, policy_version 953019 (0.0009) [2023-12-26 22:18:31,957][105620] Updated weights for policy 1, policy_version 953029 (0.0009) [2023-12-26 22:18:32,279][105692] Updated weights for policy 0, policy_version 952943 (0.0009) [2023-12-26 22:18:32,338][105692] Updated weights for policy 0, policy_version 952953 (0.0009) [2023-12-26 22:18:32,401][105692] Updated weights for policy 0, policy_version 952963 (0.0008) [2023-12-26 22:18:32,754][105620] Updated weights for policy 1, policy_version 953039 (0.0008) [2023-12-26 22:18:32,801][105620] Updated weights for policy 1, policy_version 953049 (0.0008) [2023-12-26 22:18:32,851][105620] Updated weights for policy 1, policy_version 953059 (0.0009) [2023-12-26 22:18:33,150][105692] Updated weights for policy 0, policy_version 952973 (0.0010) [2023-12-26 22:18:33,201][105692] Updated weights for policy 0, policy_version 952983 (0.0010) [2023-12-26 22:18:33,219][105585] KL-divergence is very high: 105.2818 [2023-12-26 22:18:33,259][105692] Updated weights for policy 0, policy_version 952993 (0.0010) [2023-12-26 22:18:33,263][105585] KL-divergence is very high: 101.5418 [2023-12-26 22:18:33,608][105620] Updated weights for policy 1, policy_version 953069 (0.0008) [2023-12-26 22:18:33,662][105620] Updated weights for policy 1, policy_version 953079 (0.0005) [2023-12-26 22:18:33,713][105620] Updated weights for policy 1, policy_version 953089 (0.0005) [2023-12-26 22:18:33,985][105692] Updated weights for policy 0, policy_version 953003 (0.0009) [2023-12-26 22:18:34,040][105692] Updated weights for policy 0, policy_version 953013 (0.0006) [2023-12-26 22:18:34,090][105692] Updated weights for policy 0, policy_version 953023 (0.0009) [2023-12-26 22:18:34,277][105620] Updated weights for policy 1, policy_version 953099 (0.0006) [2023-12-26 22:18:34,333][105620] Updated weights for policy 1, policy_version 953109 (0.0008) [2023-12-26 22:18:34,385][105620] Updated weights for policy 1, policy_version 953119 (0.0008) [2023-12-26 22:18:34,771][105692] Updated weights for policy 0, policy_version 953033 (0.0010) [2023-12-26 22:18:34,826][105692] Updated weights for policy 0, policy_version 953043 (0.0011) [2023-12-26 22:18:34,878][105692] Updated weights for policy 0, policy_version 953053 (0.0010) [2023-12-26 22:18:34,933][105692] Updated weights for policy 0, policy_version 953063 (0.0008) [2023-12-26 22:18:35,136][105620] Updated weights for policy 1, policy_version 953129 (0.0008) [2023-12-26 22:18:35,207][105620] Updated weights for policy 1, policy_version 953139 (0.0008) [2023-12-26 22:18:35,267][105620] Updated weights for policy 1, policy_version 953149 (0.0008) [2023-12-26 22:18:35,282][105586] KL-divergence is very high: 116.1377 [2023-12-26 22:18:35,322][105586] KL-divergence is very high: 116.1090 [2023-12-26 22:18:35,322][105620] Updated weights for policy 1, policy_version 953159 (0.0006) [2023-12-26 22:18:35,668][105692] Updated weights for policy 0, policy_version 953073 (0.0011) [2023-12-26 22:18:35,712][105692] Updated weights for policy 0, policy_version 953083 (0.0010) [2023-12-26 22:18:35,764][105692] Updated weights for policy 0, policy_version 953093 (0.0010) [2023-12-26 22:18:35,966][105620] Updated weights for policy 1, policy_version 953169 (0.0005) [2023-12-26 22:18:36,020][105620] Updated weights for policy 1, policy_version 953179 (0.0005) [2023-12-26 22:18:36,062][104569] Fps is (10 sec: 18841.8, 60 sec: 18841.6, 300 sec: 19105.4). Total num frames: 488071168. Throughput: 0: 9489.3, 1: 9468.4. Samples: 488062436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:18:36,063][104569] Avg episode reward: [(0, '9172.619'), (1, '8815.371')] [2023-12-26 22:18:36,069][105620] Updated weights for policy 1, policy_version 953189 (0.0005) [2023-12-26 22:18:36,528][105692] Updated weights for policy 0, policy_version 953103 (0.0011) [2023-12-26 22:18:36,591][105692] Updated weights for policy 0, policy_version 953113 (0.0011) [2023-12-26 22:18:36,651][105692] Updated weights for policy 0, policy_version 953123 (0.0011) [2023-12-26 22:18:36,715][105620] Updated weights for policy 1, policy_version 953199 (0.0009) [2023-12-26 22:18:36,773][105620] Updated weights for policy 1, policy_version 953209 (0.0008) [2023-12-26 22:18:36,832][105620] Updated weights for policy 1, policy_version 953219 (0.0010) [2023-12-26 22:18:37,327][105692] Updated weights for policy 0, policy_version 953133 (0.0011) [2023-12-26 22:18:37,379][105692] Updated weights for policy 0, policy_version 953143 (0.0010) [2023-12-26 22:18:37,435][105692] Updated weights for policy 0, policy_version 953153 (0.0011) [2023-12-26 22:18:37,566][105620] Updated weights for policy 1, policy_version 953229 (0.0009) [2023-12-26 22:18:37,625][105620] Updated weights for policy 1, policy_version 953239 (0.0008) [2023-12-26 22:18:37,690][105620] Updated weights for policy 1, policy_version 953249 (0.0008) [2023-12-26 22:18:38,201][105692] Updated weights for policy 0, policy_version 953163 (0.0009) [2023-12-26 22:18:38,260][105692] Updated weights for policy 0, policy_version 953173 (0.0009) [2023-12-26 22:18:38,308][105692] Updated weights for policy 0, policy_version 953183 (0.0009) [2023-12-26 22:18:38,452][105620] Updated weights for policy 1, policy_version 953259 (0.0009) [2023-12-26 22:18:38,517][105620] Updated weights for policy 1, policy_version 953269 (0.0008) [2023-12-26 22:18:38,582][105620] Updated weights for policy 1, policy_version 953279 (0.0010) [2023-12-26 22:18:39,081][105692] Updated weights for policy 0, policy_version 953193 (0.0008) [2023-12-26 22:18:39,142][105692] Updated weights for policy 0, policy_version 953203 (0.0009) [2023-12-26 22:18:39,201][105692] Updated weights for policy 0, policy_version 953213 (0.0007) [2023-12-26 22:18:39,266][105692] Updated weights for policy 0, policy_version 953223 (0.0007) [2023-12-26 22:18:39,393][105620] Updated weights for policy 1, policy_version 953289 (0.0009) [2023-12-26 22:18:39,456][105620] Updated weights for policy 1, policy_version 953299 (0.0009) [2023-12-26 22:18:39,516][105620] Updated weights for policy 1, policy_version 953309 (0.0009) [2023-12-26 22:18:39,585][105620] Updated weights for policy 1, policy_version 953319 (0.0008) [2023-12-26 22:18:40,026][105692] Updated weights for policy 0, policy_version 953233 (0.0008) [2023-12-26 22:18:40,089][105692] Updated weights for policy 0, policy_version 953243 (0.0009) [2023-12-26 22:18:40,151][105692] Updated weights for policy 0, policy_version 953253 (0.0011) [2023-12-26 22:18:40,344][105620] Updated weights for policy 1, policy_version 953329 (0.0008) [2023-12-26 22:18:40,396][105620] Updated weights for policy 1, policy_version 953339 (0.0008) [2023-12-26 22:18:40,453][105620] Updated weights for policy 1, policy_version 953349 (0.0006) [2023-12-26 22:18:40,914][105692] Updated weights for policy 0, policy_version 953263 (0.0010) [2023-12-26 22:18:40,976][105692] Updated weights for policy 0, policy_version 953273 (0.0010) [2023-12-26 22:18:41,033][105692] Updated weights for policy 0, policy_version 953283 (0.0010) [2023-12-26 22:18:41,062][104569] Fps is (10 sec: 18022.5, 60 sec: 18841.6, 300 sec: 19105.4). Total num frames: 488161280. Throughput: 0: 9446.4, 1: 9506.3. Samples: 488176544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:18:41,063][104569] Avg episode reward: [(0, '9177.772'), (1, '8728.785')] [2023-12-26 22:18:41,197][105620] Updated weights for policy 1, policy_version 953359 (0.0008) [2023-12-26 22:18:41,263][105620] Updated weights for policy 1, policy_version 953369 (0.0008) [2023-12-26 22:18:41,329][105620] Updated weights for policy 1, policy_version 953379 (0.0007) [2023-12-26 22:18:41,838][105692] Updated weights for policy 0, policy_version 953293 (0.0010) [2023-12-26 22:18:41,898][105692] Updated weights for policy 0, policy_version 953303 (0.0011) [2023-12-26 22:18:41,962][105692] Updated weights for policy 0, policy_version 953313 (0.0011) [2023-12-26 22:18:42,056][105620] Updated weights for policy 1, policy_version 953389 (0.0008) [2023-12-26 22:18:42,114][105620] Updated weights for policy 1, policy_version 953399 (0.0008) [2023-12-26 22:18:42,171][105620] Updated weights for policy 1, policy_version 953409 (0.0009) [2023-12-26 22:18:42,670][105692] Updated weights for policy 0, policy_version 953323 (0.0010) [2023-12-26 22:18:42,726][105692] Updated weights for policy 0, policy_version 953333 (0.0009) [2023-12-26 22:18:42,782][105692] Updated weights for policy 0, policy_version 953343 (0.0009) [2023-12-26 22:18:42,975][105620] Updated weights for policy 1, policy_version 953419 (0.0009) [2023-12-26 22:18:43,032][105620] Updated weights for policy 1, policy_version 953429 (0.0009) [2023-12-26 22:18:43,098][105620] Updated weights for policy 1, policy_version 953439 (0.0008) [2023-12-26 22:18:43,612][105692] Updated weights for policy 0, policy_version 953353 (0.0009) [2023-12-26 22:18:43,666][105692] Updated weights for policy 0, policy_version 953363 (0.0008) [2023-12-26 22:18:43,723][105620] Updated weights for policy 1, policy_version 953449 (0.0008) [2023-12-26 22:18:43,736][105692] Updated weights for policy 0, policy_version 953373 (0.0009) [2023-12-26 22:18:43,778][105620] Updated weights for policy 1, policy_version 953459 (0.0006) [2023-12-26 22:18:43,804][105692] Updated weights for policy 0, policy_version 953383 (0.0009) [2023-12-26 22:18:43,838][105620] Updated weights for policy 1, policy_version 953469 (0.0005) [2023-12-26 22:18:43,902][105620] Updated weights for policy 1, policy_version 953479 (0.0009) [2023-12-26 22:18:44,610][105692] Updated weights for policy 0, policy_version 953393 (0.0008) [2023-12-26 22:18:44,630][105620] Updated weights for policy 1, policy_version 953489 (0.0010) [2023-12-26 22:18:44,668][105692] Updated weights for policy 0, policy_version 953403 (0.0007) [2023-12-26 22:18:44,693][105620] Updated weights for policy 1, policy_version 953499 (0.0011) [2023-12-26 22:18:44,724][105692] Updated weights for policy 0, policy_version 953413 (0.0006) [2023-12-26 22:18:44,756][105620] Updated weights for policy 1, policy_version 953509 (0.0010) [2023-12-26 22:18:45,435][105620] Updated weights for policy 1, policy_version 953519 (0.0010) [2023-12-26 22:18:45,490][105692] Updated weights for policy 0, policy_version 953423 (0.0007) [2023-12-26 22:18:45,492][105620] Updated weights for policy 1, policy_version 953529 (0.0011) [2023-12-26 22:18:45,539][105692] Updated weights for policy 0, policy_version 953433 (0.0005) [2023-12-26 22:18:45,541][105620] Updated weights for policy 1, policy_version 953539 (0.0010) [2023-12-26 22:18:45,597][105692] Updated weights for policy 0, policy_version 953443 (0.0005) [2023-12-26 22:18:46,062][104569] Fps is (10 sec: 18842.0, 60 sec: 18841.6, 300 sec: 19105.4). Total num frames: 488259584. Throughput: 0: 9347.6, 1: 9567.4. Samples: 488232784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:18:46,062][104569] Avg episode reward: [(0, '9086.075'), (1, '8565.616')] [2023-12-26 22:18:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000953448_244121600.pth... [2023-12-26 22:18:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000953544_244137984.pth... [2023-12-26 22:18:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000952424_243851264.pth [2023-12-26 22:18:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000952360_243843072.pth [2023-12-26 22:18:46,284][105620] Updated weights for policy 1, policy_version 953549 (0.0010) [2023-12-26 22:18:46,319][105692] Updated weights for policy 0, policy_version 953453 (0.0005) [2023-12-26 22:18:46,337][105620] Updated weights for policy 1, policy_version 953559 (0.0010) [2023-12-26 22:18:46,376][105692] Updated weights for policy 0, policy_version 953463 (0.0006) [2023-12-26 22:18:46,389][105620] Updated weights for policy 1, policy_version 953569 (0.0010) [2023-12-26 22:18:46,435][105692] Updated weights for policy 0, policy_version 953473 (0.0007) [2023-12-26 22:18:47,094][105692] Updated weights for policy 0, policy_version 953483 (0.0006) [2023-12-26 22:18:47,151][105692] Updated weights for policy 0, policy_version 953493 (0.0007) [2023-12-26 22:18:47,156][105620] Updated weights for policy 1, policy_version 953579 (0.0011) [2023-12-26 22:18:47,204][105620] Updated weights for policy 1, policy_version 953589 (0.0010) [2023-12-26 22:18:47,207][105692] Updated weights for policy 0, policy_version 953503 (0.0005) [2023-12-26 22:18:47,249][105620] Updated weights for policy 1, policy_version 953599 (0.0010) [2023-12-26 22:18:47,823][105692] Updated weights for policy 0, policy_version 953513 (0.0005) [2023-12-26 22:18:47,888][105692] Updated weights for policy 0, policy_version 953523 (0.0009) [2023-12-26 22:18:47,949][105692] Updated weights for policy 0, policy_version 953533 (0.0009) [2023-12-26 22:18:48,007][105692] Updated weights for policy 0, policy_version 953543 (0.0008) [2023-12-26 22:18:48,011][105620] Updated weights for policy 1, policy_version 953609 (0.0009) [2023-12-26 22:18:48,062][105620] Updated weights for policy 1, policy_version 953619 (0.0007) [2023-12-26 22:18:48,114][105620] Updated weights for policy 1, policy_version 953629 (0.0006) [2023-12-26 22:18:48,183][105620] Updated weights for policy 1, policy_version 953639 (0.0005) [2023-12-26 22:18:48,629][105692] Updated weights for policy 0, policy_version 953553 (0.0005) [2023-12-26 22:18:48,694][105692] Updated weights for policy 0, policy_version 953563 (0.0006) [2023-12-26 22:18:48,749][105692] Updated weights for policy 0, policy_version 953573 (0.0006) [2023-12-26 22:18:48,926][105620] Updated weights for policy 1, policy_version 953649 (0.0009) [2023-12-26 22:18:48,986][105620] Updated weights for policy 1, policy_version 953659 (0.0009) [2023-12-26 22:18:49,052][105620] Updated weights for policy 1, policy_version 953669 (0.0008) [2023-12-26 22:18:49,351][105692] Updated weights for policy 0, policy_version 953583 (0.0008) [2023-12-26 22:18:49,415][105692] Updated weights for policy 0, policy_version 953593 (0.0009) [2023-12-26 22:18:49,486][105692] Updated weights for policy 0, policy_version 953603 (0.0009) [2023-12-26 22:18:49,901][105620] Updated weights for policy 1, policy_version 953679 (0.0007) [2023-12-26 22:18:49,972][105620] Updated weights for policy 1, policy_version 953689 (0.0008) [2023-12-26 22:18:50,045][105620] Updated weights for policy 1, policy_version 953699 (0.0008) [2023-12-26 22:18:50,212][105692] Updated weights for policy 0, policy_version 953613 (0.0009) [2023-12-26 22:18:50,277][105692] Updated weights for policy 0, policy_version 953623 (0.0008) [2023-12-26 22:18:50,347][105692] Updated weights for policy 0, policy_version 953633 (0.0009) [2023-12-26 22:18:50,763][105620] Updated weights for policy 1, policy_version 953709 (0.0009) [2023-12-26 22:18:50,826][105620] Updated weights for policy 1, policy_version 953719 (0.0009) [2023-12-26 22:18:50,888][105620] Updated weights for policy 1, policy_version 953729 (0.0009) [2023-12-26 22:18:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 18978.1, 300 sec: 19105.4). Total num frames: 488357888. Throughput: 0: 9336.1, 1: 9547.3. Samples: 488347984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:18:51,063][104569] Avg episode reward: [(0, '8903.050'), (1, '8750.248')] [2023-12-26 22:18:51,070][105692] Updated weights for policy 0, policy_version 953643 (0.0010) [2023-12-26 22:18:51,133][105692] Updated weights for policy 0, policy_version 953653 (0.0009) [2023-12-26 22:18:51,199][105692] Updated weights for policy 0, policy_version 953663 (0.0009) [2023-12-26 22:18:51,702][105620] Updated weights for policy 1, policy_version 953739 (0.0009) [2023-12-26 22:18:51,767][105620] Updated weights for policy 1, policy_version 953749 (0.0007) [2023-12-26 22:18:51,829][105620] Updated weights for policy 1, policy_version 953759 (0.0009) [2023-12-26 22:18:51,982][105692] Updated weights for policy 0, policy_version 953673 (0.0008) [2023-12-26 22:18:52,041][105692] Updated weights for policy 0, policy_version 953683 (0.0009) [2023-12-26 22:18:52,105][105692] Updated weights for policy 0, policy_version 953693 (0.0009) [2023-12-26 22:18:52,172][105692] Updated weights for policy 0, policy_version 953703 (0.0009) [2023-12-26 22:18:52,554][105620] Updated weights for policy 1, policy_version 953769 (0.0009) [2023-12-26 22:18:52,621][105620] Updated weights for policy 1, policy_version 953779 (0.0006) [2023-12-26 22:18:52,687][105620] Updated weights for policy 1, policy_version 953789 (0.0008) [2023-12-26 22:18:52,748][105620] Updated weights for policy 1, policy_version 953799 (0.0006) [2023-12-26 22:18:52,951][105692] Updated weights for policy 0, policy_version 953713 (0.0010) [2023-12-26 22:18:53,018][105692] Updated weights for policy 0, policy_version 953723 (0.0010) [2023-12-26 22:18:53,087][105692] Updated weights for policy 0, policy_version 953733 (0.0008) [2023-12-26 22:18:53,384][105620] Updated weights for policy 1, policy_version 953809 (0.0006) [2023-12-26 22:18:53,431][105620] Updated weights for policy 1, policy_version 953819 (0.0007) [2023-12-26 22:18:53,479][105620] Updated weights for policy 1, policy_version 953829 (0.0010) [2023-12-26 22:18:53,878][105692] Updated weights for policy 0, policy_version 953743 (0.0008) [2023-12-26 22:18:53,922][105692] Updated weights for policy 0, policy_version 953753 (0.0008) [2023-12-26 22:18:53,981][105692] Updated weights for policy 0, policy_version 953763 (0.0008) [2023-12-26 22:18:54,172][105620] Updated weights for policy 1, policy_version 953839 (0.0009) [2023-12-26 22:18:54,232][105620] Updated weights for policy 1, policy_version 953849 (0.0005) [2023-12-26 22:18:54,284][105620] Updated weights for policy 1, policy_version 953859 (0.0006) [2023-12-26 22:18:54,745][105692] Updated weights for policy 0, policy_version 953773 (0.0007) [2023-12-26 22:18:54,805][105692] Updated weights for policy 0, policy_version 953783 (0.0005) [2023-12-26 22:18:54,863][105692] Updated weights for policy 0, policy_version 953793 (0.0005) [2023-12-26 22:18:55,030][105620] Updated weights for policy 1, policy_version 953869 (0.0007) [2023-12-26 22:18:55,090][105620] Updated weights for policy 1, policy_version 953879 (0.0008) [2023-12-26 22:18:55,152][105620] Updated weights for policy 1, policy_version 953889 (0.0006) [2023-12-26 22:18:55,574][105692] Updated weights for policy 0, policy_version 953803 (0.0005) [2023-12-26 22:18:55,634][105692] Updated weights for policy 0, policy_version 953813 (0.0005) [2023-12-26 22:18:55,688][105692] Updated weights for policy 0, policy_version 953823 (0.0008) [2023-12-26 22:18:55,801][105620] Updated weights for policy 1, policy_version 953899 (0.0007) [2023-12-26 22:18:55,857][105620] Updated weights for policy 1, policy_version 953909 (0.0010) [2023-12-26 22:18:55,913][105620] Updated weights for policy 1, policy_version 953919 (0.0010) [2023-12-26 22:18:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 18978.2, 300 sec: 19105.4). Total num frames: 488456192. Throughput: 0: 9347.1, 1: 9534.1. Samples: 488461564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:18:56,062][104569] Avg episode reward: [(0, '8899.053'), (1, '8906.240')] [2023-12-26 22:18:56,413][105692] Updated weights for policy 0, policy_version 953833 (0.0010) [2023-12-26 22:18:56,474][105692] Updated weights for policy 0, policy_version 953843 (0.0010) [2023-12-26 22:18:56,529][105620] Updated weights for policy 1, policy_version 953929 (0.0011) [2023-12-26 22:18:56,536][105692] Updated weights for policy 0, policy_version 953853 (0.0010) [2023-12-26 22:18:56,577][105620] Updated weights for policy 1, policy_version 953939 (0.0010) [2023-12-26 22:18:56,583][105692] Updated weights for policy 0, policy_version 953863 (0.0010) [2023-12-26 22:18:56,629][105620] Updated weights for policy 1, policy_version 953949 (0.0010) [2023-12-26 22:18:56,674][105620] Updated weights for policy 1, policy_version 953959 (0.0010) [2023-12-26 22:18:57,356][105692] Updated weights for policy 0, policy_version 953873 (0.0007) [2023-12-26 22:18:57,357][105620] Updated weights for policy 1, policy_version 953969 (0.0008) [2023-12-26 22:18:57,408][105692] Updated weights for policy 0, policy_version 953883 (0.0005) [2023-12-26 22:18:57,419][105620] Updated weights for policy 1, policy_version 953979 (0.0008) [2023-12-26 22:18:57,462][105692] Updated weights for policy 0, policy_version 953893 (0.0008) [2023-12-26 22:18:57,478][105620] Updated weights for policy 1, policy_version 953989 (0.0007) [2023-12-26 22:18:58,032][105692] Updated weights for policy 0, policy_version 953903 (0.0005) [2023-12-26 22:18:58,093][105692] Updated weights for policy 0, policy_version 953913 (0.0009) [2023-12-26 22:18:58,153][105692] Updated weights for policy 0, policy_version 953923 (0.0009) [2023-12-26 22:18:58,200][105620] Updated weights for policy 1, policy_version 953999 (0.0009) [2023-12-26 22:18:58,266][105620] Updated weights for policy 1, policy_version 954009 (0.0010) [2023-12-26 22:18:58,327][105620] Updated weights for policy 1, policy_version 954019 (0.0012) [2023-12-26 22:18:58,959][105692] Updated weights for policy 0, policy_version 953933 (0.0011) [2023-12-26 22:18:59,021][105692] Updated weights for policy 0, policy_version 953943 (0.0010) [2023-12-26 22:18:59,080][105692] Updated weights for policy 0, policy_version 953953 (0.0008) [2023-12-26 22:18:59,089][105620] Updated weights for policy 1, policy_version 954029 (0.0009) [2023-12-26 22:18:59,152][105620] Updated weights for policy 1, policy_version 954039 (0.0011) [2023-12-26 22:18:59,213][105620] Updated weights for policy 1, policy_version 954049 (0.0010) [2023-12-26 22:18:59,813][105692] Updated weights for policy 0, policy_version 953963 (0.0007) [2023-12-26 22:18:59,879][105692] Updated weights for policy 0, policy_version 953973 (0.0007) [2023-12-26 22:18:59,940][105692] Updated weights for policy 0, policy_version 953983 (0.0010) [2023-12-26 22:18:59,956][105620] Updated weights for policy 1, policy_version 954060 (0.0009) [2023-12-26 22:19:00,012][105620] Updated weights for policy 1, policy_version 954070 (0.0011) [2023-12-26 22:19:00,071][105620] Updated weights for policy 1, policy_version 954080 (0.0010) [2023-12-26 22:19:00,599][105692] Updated weights for policy 0, policy_version 953993 (0.0010) [2023-12-26 22:19:00,650][105692] Updated weights for policy 0, policy_version 954003 (0.0009) [2023-12-26 22:19:00,704][105692] Updated weights for policy 0, policy_version 954013 (0.0009) [2023-12-26 22:19:00,715][105620] Updated weights for policy 1, policy_version 954090 (0.0009) [2023-12-26 22:19:00,753][105692] Updated weights for policy 0, policy_version 954023 (0.0007) [2023-12-26 22:19:00,772][105620] Updated weights for policy 1, policy_version 954100 (0.0007) [2023-12-26 22:19:00,828][105620] Updated weights for policy 1, policy_version 954110 (0.0005) [2023-12-26 22:19:00,895][105620] Updated weights for policy 1, policy_version 954120 (0.0008) [2023-12-26 22:19:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19114.7, 300 sec: 19105.4). Total num frames: 488554496. Throughput: 0: 9380.6, 1: 9600.3. Samples: 488521300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:19:01,062][104569] Avg episode reward: [(0, '8991.973'), (1, '8814.121')] [2023-12-26 22:19:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000954120_244285440.pth... [2023-12-26 22:19:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000954024_244269056.pth... [2023-12-26 22:19:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000952904_243982336.pth [2023-12-26 22:19:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000953000_243998720.pth [2023-12-26 22:19:01,431][105692] Updated weights for policy 0, policy_version 954033 (0.0009) [2023-12-26 22:19:01,495][105692] Updated weights for policy 0, policy_version 954043 (0.0009) [2023-12-26 22:19:01,554][105692] Updated weights for policy 0, policy_version 954053 (0.0007) [2023-12-26 22:19:01,568][105620] Updated weights for policy 1, policy_version 954130 (0.0008) [2023-12-26 22:19:01,625][105620] Updated weights for policy 1, policy_version 954140 (0.0010) [2023-12-26 22:19:01,690][105620] Updated weights for policy 1, policy_version 954150 (0.0009) [2023-12-26 22:19:02,250][105692] Updated weights for policy 0, policy_version 954063 (0.0008) [2023-12-26 22:19:02,323][105692] Updated weights for policy 0, policy_version 954073 (0.0008) [2023-12-26 22:19:02,388][105692] Updated weights for policy 0, policy_version 954083 (0.0008) [2023-12-26 22:19:02,429][105620] Updated weights for policy 1, policy_version 954160 (0.0008) [2023-12-26 22:19:02,487][105620] Updated weights for policy 1, policy_version 954170 (0.0008) [2023-12-26 22:19:02,540][105620] Updated weights for policy 1, policy_version 954180 (0.0008) [2023-12-26 22:19:03,133][105692] Updated weights for policy 0, policy_version 954093 (0.0009) [2023-12-26 22:19:03,191][105692] Updated weights for policy 0, policy_version 954103 (0.0008) [2023-12-26 22:19:03,247][105692] Updated weights for policy 0, policy_version 954113 (0.0009) [2023-12-26 22:19:03,299][105620] Updated weights for policy 1, policy_version 954190 (0.0009) [2023-12-26 22:19:03,358][105620] Updated weights for policy 1, policy_version 954200 (0.0009) [2023-12-26 22:19:03,420][105620] Updated weights for policy 1, policy_version 954210 (0.0010) [2023-12-26 22:19:03,946][105692] Updated weights for policy 0, policy_version 954123 (0.0007) [2023-12-26 22:19:04,017][105692] Updated weights for policy 0, policy_version 954133 (0.0010) [2023-12-26 22:19:04,034][105620] Updated weights for policy 1, policy_version 954220 (0.0010) [2023-12-26 22:19:04,065][105692] Updated weights for policy 0, policy_version 954143 (0.0006) [2023-12-26 22:19:04,094][105620] Updated weights for policy 1, policy_version 954230 (0.0010) [2023-12-26 22:19:04,148][105620] Updated weights for policy 1, policy_version 954240 (0.0006) [2023-12-26 22:19:04,801][105692] Updated weights for policy 0, policy_version 954153 (0.0009) [2023-12-26 22:19:04,863][105692] Updated weights for policy 0, policy_version 954163 (0.0006) [2023-12-26 22:19:04,875][105620] Updated weights for policy 1, policy_version 954250 (0.0007) [2023-12-26 22:19:04,912][105692] Updated weights for policy 0, policy_version 954173 (0.0010) [2023-12-26 22:19:04,921][105620] Updated weights for policy 1, policy_version 954260 (0.0010) [2023-12-26 22:19:04,960][105692] Updated weights for policy 0, policy_version 954183 (0.0007) [2023-12-26 22:19:04,967][105620] Updated weights for policy 1, policy_version 954270 (0.0006) [2023-12-26 22:19:05,014][105620] Updated weights for policy 1, policy_version 954280 (0.0008) [2023-12-26 22:19:05,623][105692] Updated weights for policy 0, policy_version 954193 (0.0008) [2023-12-26 22:19:05,669][105585] KL-divergence is very high: 121.0616 [2023-12-26 22:19:05,681][105692] Updated weights for policy 0, policy_version 954203 (0.0005) [2023-12-26 22:19:05,719][105585] KL-divergence is very high: 112.7386 [2023-12-26 22:19:05,744][105692] Updated weights for policy 0, policy_version 954213 (0.0005) [2023-12-26 22:19:05,853][105620] Updated weights for policy 1, policy_version 954290 (0.0009) [2023-12-26 22:19:05,921][105620] Updated weights for policy 1, policy_version 954300 (0.0009) [2023-12-26 22:19:05,990][105620] Updated weights for policy 1, policy_version 954310 (0.0009) [2023-12-26 22:19:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19114.7, 300 sec: 19133.2). Total num frames: 488652800. Throughput: 0: 9406.2, 1: 9642.4. Samples: 488637704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:19:06,063][104569] Avg episode reward: [(0, '8900.923'), (1, '8828.487')] [2023-12-26 22:19:06,336][105692] Updated weights for policy 0, policy_version 954223 (0.0008) [2023-12-26 22:19:06,402][105692] Updated weights for policy 0, policy_version 954233 (0.0009) [2023-12-26 22:19:06,465][105692] Updated weights for policy 0, policy_version 954243 (0.0009) [2023-12-26 22:19:06,769][105620] Updated weights for policy 1, policy_version 954320 (0.0008) [2023-12-26 22:19:06,841][105620] Updated weights for policy 1, policy_version 954330 (0.0006) [2023-12-26 22:19:06,909][105620] Updated weights for policy 1, policy_version 954340 (0.0006) [2023-12-26 22:19:07,149][105692] Updated weights for policy 0, policy_version 954253 (0.0009) [2023-12-26 22:19:07,212][105692] Updated weights for policy 0, policy_version 954263 (0.0008) [2023-12-26 22:19:07,281][105692] Updated weights for policy 0, policy_version 954273 (0.0009) [2023-12-26 22:19:07,568][105620] Updated weights for policy 1, policy_version 954350 (0.0009) [2023-12-26 22:19:07,632][105620] Updated weights for policy 1, policy_version 954360 (0.0009) [2023-12-26 22:19:07,692][105620] Updated weights for policy 1, policy_version 954370 (0.0008) [2023-12-26 22:19:08,024][105692] Updated weights for policy 0, policy_version 954283 (0.0009) [2023-12-26 22:19:08,086][105692] Updated weights for policy 0, policy_version 954293 (0.0009) [2023-12-26 22:19:08,136][105692] Updated weights for policy 0, policy_version 954303 (0.0008) [2023-12-26 22:19:08,457][105620] Updated weights for policy 1, policy_version 954380 (0.0009) [2023-12-26 22:19:08,523][105620] Updated weights for policy 1, policy_version 954390 (0.0008) [2023-12-26 22:19:08,582][105620] Updated weights for policy 1, policy_version 954400 (0.0009) [2023-12-26 22:19:08,911][105692] Updated weights for policy 0, policy_version 954313 (0.0009) [2023-12-26 22:19:08,987][105692] Updated weights for policy 0, policy_version 954323 (0.0006) [2023-12-26 22:19:09,041][105692] Updated weights for policy 0, policy_version 954333 (0.0006) [2023-12-26 22:19:09,096][105692] Updated weights for policy 0, policy_version 954343 (0.0005) [2023-12-26 22:19:09,411][105620] Updated weights for policy 1, policy_version 954410 (0.0009) [2023-12-26 22:19:09,476][105620] Updated weights for policy 1, policy_version 954420 (0.0008) [2023-12-26 22:19:09,539][105620] Updated weights for policy 1, policy_version 954430 (0.0008) [2023-12-26 22:19:09,606][105620] Updated weights for policy 1, policy_version 954440 (0.0008) [2023-12-26 22:19:09,776][105692] Updated weights for policy 0, policy_version 954353 (0.0010) [2023-12-26 22:19:09,833][105692] Updated weights for policy 0, policy_version 954363 (0.0010) [2023-12-26 22:19:09,900][105692] Updated weights for policy 0, policy_version 954373 (0.0010) [2023-12-26 22:19:10,361][105620] Updated weights for policy 1, policy_version 954450 (0.0009) [2023-12-26 22:19:10,429][105620] Updated weights for policy 1, policy_version 954460 (0.0007) [2023-12-26 22:19:10,487][105620] Updated weights for policy 1, policy_version 954470 (0.0010) [2023-12-26 22:19:10,668][105692] Updated weights for policy 0, policy_version 954383 (0.0010) [2023-12-26 22:19:10,726][105692] Updated weights for policy 0, policy_version 954393 (0.0010) [2023-12-26 22:19:10,781][105692] Updated weights for policy 0, policy_version 954403 (0.0010) [2023-12-26 22:19:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19105.4). Total num frames: 488742912. Throughput: 0: 9567.8, 1: 9565.1. Samples: 488752560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:19:11,063][104569] Avg episode reward: [(0, '8809.925'), (1, '8916.055')] [2023-12-26 22:19:11,164][105620] Updated weights for policy 1, policy_version 954480 (0.0009) [2023-12-26 22:19:11,234][105620] Updated weights for policy 1, policy_version 954490 (0.0010) [2023-12-26 22:19:11,301][105620] Updated weights for policy 1, policy_version 954500 (0.0011) [2023-12-26 22:19:11,569][105692] Updated weights for policy 0, policy_version 954413 (0.0010) [2023-12-26 22:19:11,639][105692] Updated weights for policy 0, policy_version 954423 (0.0011) [2023-12-26 22:19:11,706][105692] Updated weights for policy 0, policy_version 954433 (0.0011) [2023-12-26 22:19:12,091][105620] Updated weights for policy 1, policy_version 954510 (0.0009) [2023-12-26 22:19:12,148][105620] Updated weights for policy 1, policy_version 954520 (0.0008) [2023-12-26 22:19:12,209][105620] Updated weights for policy 1, policy_version 954530 (0.0008) [2023-12-26 22:19:12,423][105692] Updated weights for policy 0, policy_version 954443 (0.0009) [2023-12-26 22:19:12,479][105692] Updated weights for policy 0, policy_version 954453 (0.0010) [2023-12-26 22:19:12,542][105692] Updated weights for policy 0, policy_version 954463 (0.0014) [2023-12-26 22:19:12,964][105620] Updated weights for policy 1, policy_version 954540 (0.0008) [2023-12-26 22:19:13,022][105620] Updated weights for policy 1, policy_version 954550 (0.0009) [2023-12-26 22:19:13,086][105620] Updated weights for policy 1, policy_version 954560 (0.0008) [2023-12-26 22:19:13,362][105692] Updated weights for policy 0, policy_version 954473 (0.0009) [2023-12-26 22:19:13,432][105692] Updated weights for policy 0, policy_version 954483 (0.0010) [2023-12-26 22:19:13,501][105692] Updated weights for policy 0, policy_version 954493 (0.0009) [2023-12-26 22:19:13,581][105692] Updated weights for policy 0, policy_version 954503 (0.0010) [2023-12-26 22:19:13,698][105620] Updated weights for policy 1, policy_version 954570 (0.0006) [2023-12-26 22:19:13,760][105620] Updated weights for policy 1, policy_version 954580 (0.0009) [2023-12-26 22:19:13,835][105620] Updated weights for policy 1, policy_version 954590 (0.0010) [2023-12-26 22:19:13,893][105620] Updated weights for policy 1, policy_version 954600 (0.0010) [2023-12-26 22:19:14,177][105692] Updated weights for policy 0, policy_version 954513 (0.0009) [2023-12-26 22:19:14,237][105692] Updated weights for policy 0, policy_version 954523 (0.0009) [2023-12-26 22:19:14,297][105692] Updated weights for policy 0, policy_version 954533 (0.0009) [2023-12-26 22:19:14,607][105620] Updated weights for policy 1, policy_version 954610 (0.0009) [2023-12-26 22:19:14,669][105620] Updated weights for policy 1, policy_version 954620 (0.0009) [2023-12-26 22:19:14,736][105620] Updated weights for policy 1, policy_version 954630 (0.0008) [2023-12-26 22:19:15,098][105692] Updated weights for policy 0, policy_version 954543 (0.0009) [2023-12-26 22:19:15,162][105692] Updated weights for policy 0, policy_version 954553 (0.0009) [2023-12-26 22:19:15,217][105692] Updated weights for policy 0, policy_version 954563 (0.0009) [2023-12-26 22:19:15,484][105620] Updated weights for policy 1, policy_version 954640 (0.0010) [2023-12-26 22:19:15,548][105620] Updated weights for policy 1, policy_version 954650 (0.0011) [2023-12-26 22:19:15,608][105620] Updated weights for policy 1, policy_version 954660 (0.0011) [2023-12-26 22:19:16,009][105692] Updated weights for policy 0, policy_version 954573 (0.0009) [2023-12-26 22:19:16,062][104569] Fps is (10 sec: 18022.2, 60 sec: 18978.1, 300 sec: 19105.4). Total num frames: 488833024. Throughput: 0: 9516.8, 1: 9556.6. Samples: 488806736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:19:16,063][104569] Avg episode reward: [(0, '9175.439'), (1, '8927.704')] [2023-12-26 22:19:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000954664_244424704.pth... [2023-12-26 22:19:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000953544_244137984.pth [2023-12-26 22:19:16,075][105692] Updated weights for policy 0, policy_version 954583 (0.0008) [2023-12-26 22:19:16,135][105692] Updated weights for policy 0, policy_version 954593 (0.0008) [2023-12-26 22:19:16,173][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000954600_244416512.pth... [2023-12-26 22:19:16,177][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000953448_244121600.pth [2023-12-26 22:19:16,354][105620] Updated weights for policy 1, policy_version 954670 (0.0010) [2023-12-26 22:19:16,402][105620] Updated weights for policy 1, policy_version 954680 (0.0010) [2023-12-26 22:19:16,447][105620] Updated weights for policy 1, policy_version 954690 (0.0010) [2023-12-26 22:19:16,922][105692] Updated weights for policy 0, policy_version 954603 (0.0007) [2023-12-26 22:19:16,979][105692] Updated weights for policy 0, policy_version 954613 (0.0009) [2023-12-26 22:19:17,040][105692] Updated weights for policy 0, policy_version 954623 (0.0010) [2023-12-26 22:19:17,093][105620] Updated weights for policy 1, policy_version 954700 (0.0008) [2023-12-26 22:19:17,146][105620] Updated weights for policy 1, policy_version 954710 (0.0007) [2023-12-26 22:19:17,198][105620] Updated weights for policy 1, policy_version 954720 (0.0006) [2023-12-26 22:19:17,825][105620] Updated weights for policy 1, policy_version 954730 (0.0009) [2023-12-26 22:19:17,851][105692] Updated weights for policy 0, policy_version 954633 (0.0008) [2023-12-26 22:19:17,879][105620] Updated weights for policy 1, policy_version 954740 (0.0007) [2023-12-26 22:19:17,908][105692] Updated weights for policy 0, policy_version 954643 (0.0010) [2023-12-26 22:19:17,935][105620] Updated weights for policy 1, policy_version 954750 (0.0008) [2023-12-26 22:19:17,965][105692] Updated weights for policy 0, policy_version 954653 (0.0007) [2023-12-26 22:19:17,995][105620] Updated weights for policy 1, policy_version 954760 (0.0007) [2023-12-26 22:19:18,020][105692] Updated weights for policy 0, policy_version 954663 (0.0007) [2023-12-26 22:19:18,748][105620] Updated weights for policy 1, policy_version 954770 (0.0009) [2023-12-26 22:19:18,802][105620] Updated weights for policy 1, policy_version 954780 (0.0007) [2023-12-26 22:19:18,804][105692] Updated weights for policy 0, policy_version 954673 (0.0007) [2023-12-26 22:19:18,864][105692] Updated weights for policy 0, policy_version 954683 (0.0008) [2023-12-26 22:19:18,868][105620] Updated weights for policy 1, policy_version 954790 (0.0007) [2023-12-26 22:19:18,914][105692] Updated weights for policy 0, policy_version 954693 (0.0008) [2023-12-26 22:19:19,653][105620] Updated weights for policy 1, policy_version 954800 (0.0007) [2023-12-26 22:19:19,671][105692] Updated weights for policy 0, policy_version 954703 (0.0007) [2023-12-26 22:19:19,706][105620] Updated weights for policy 1, policy_version 954810 (0.0008) [2023-12-26 22:19:19,737][105692] Updated weights for policy 0, policy_version 954713 (0.0007) [2023-12-26 22:19:19,776][105620] Updated weights for policy 1, policy_version 954820 (0.0006) [2023-12-26 22:19:19,794][105692] Updated weights for policy 0, policy_version 954723 (0.0011) [2023-12-26 22:19:20,395][105620] Updated weights for policy 1, policy_version 954830 (0.0006) [2023-12-26 22:19:20,442][105692] Updated weights for policy 0, policy_version 954733 (0.0007) [2023-12-26 22:19:20,462][105620] Updated weights for policy 1, policy_version 954840 (0.0006) [2023-12-26 22:19:20,514][105692] Updated weights for policy 0, policy_version 954743 (0.0008) [2023-12-26 22:19:20,523][105620] Updated weights for policy 1, policy_version 954850 (0.0007) [2023-12-26 22:19:20,582][105692] Updated weights for policy 0, policy_version 954753 (0.0008) [2023-12-26 22:19:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 18978.1, 300 sec: 19105.4). Total num frames: 488931328. Throughput: 0: 9492.1, 1: 9571.7. Samples: 488920304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:19:21,062][104569] Avg episode reward: [(0, '9089.527'), (1, '8844.282')] [2023-12-26 22:19:21,178][105620] Updated weights for policy 1, policy_version 954860 (0.0007) [2023-12-26 22:19:21,245][105620] Updated weights for policy 1, policy_version 954870 (0.0008) [2023-12-26 22:19:21,319][105620] Updated weights for policy 1, policy_version 954880 (0.0008) [2023-12-26 22:19:21,383][105692] Updated weights for policy 0, policy_version 954763 (0.0009) [2023-12-26 22:19:21,456][105692] Updated weights for policy 0, policy_version 954773 (0.0010) [2023-12-26 22:19:21,521][105692] Updated weights for policy 0, policy_version 954783 (0.0010) [2023-12-26 22:19:22,127][105620] Updated weights for policy 1, policy_version 954890 (0.0008) [2023-12-26 22:19:22,192][105620] Updated weights for policy 1, policy_version 954900 (0.0010) [2023-12-26 22:19:22,254][105692] Updated weights for policy 0, policy_version 954793 (0.0010) [2023-12-26 22:19:22,260][105620] Updated weights for policy 1, policy_version 954910 (0.0011) [2023-12-26 22:19:22,321][105692] Updated weights for policy 0, policy_version 954803 (0.0011) [2023-12-26 22:19:22,327][105620] Updated weights for policy 1, policy_version 954920 (0.0011) [2023-12-26 22:19:22,389][105692] Updated weights for policy 0, policy_version 954813 (0.0011) [2023-12-26 22:19:22,449][105692] Updated weights for policy 0, policy_version 954823 (0.0011) [2023-12-26 22:19:23,071][105620] Updated weights for policy 1, policy_version 954930 (0.0008) [2023-12-26 22:19:23,131][105620] Updated weights for policy 1, policy_version 954940 (0.0011) [2023-12-26 22:19:23,140][105692] Updated weights for policy 0, policy_version 954833 (0.0010) [2023-12-26 22:19:23,192][105620] Updated weights for policy 1, policy_version 954950 (0.0011) [2023-12-26 22:19:23,200][105692] Updated weights for policy 0, policy_version 954843 (0.0011) [2023-12-26 22:19:23,258][105692] Updated weights for policy 0, policy_version 954853 (0.0009) [2023-12-26 22:19:23,822][105692] Updated weights for policy 0, policy_version 954863 (0.0005) [2023-12-26 22:19:23,875][105692] Updated weights for policy 0, policy_version 954873 (0.0009) [2023-12-26 22:19:23,933][105692] Updated weights for policy 0, policy_version 954883 (0.0010) [2023-12-26 22:19:23,935][105620] Updated weights for policy 1, policy_version 954960 (0.0006) [2023-12-26 22:19:23,985][105620] Updated weights for policy 1, policy_version 954970 (0.0008) [2023-12-26 22:19:24,029][105620] Updated weights for policy 1, policy_version 954980 (0.0008) [2023-12-26 22:19:24,523][105692] Updated weights for policy 0, policy_version 954893 (0.0010) [2023-12-26 22:19:24,572][105692] Updated weights for policy 0, policy_version 954903 (0.0010) [2023-12-26 22:19:24,634][105692] Updated weights for policy 0, policy_version 954913 (0.0011) [2023-12-26 22:19:24,783][105620] Updated weights for policy 1, policy_version 954990 (0.0008) [2023-12-26 22:19:24,856][105620] Updated weights for policy 1, policy_version 955000 (0.0006) [2023-12-26 22:19:24,926][105620] Updated weights for policy 1, policy_version 955010 (0.0008) [2023-12-26 22:19:25,407][105692] Updated weights for policy 0, policy_version 954923 (0.0010) [2023-12-26 22:19:25,471][105692] Updated weights for policy 0, policy_version 954933 (0.0009) [2023-12-26 22:19:25,531][105692] Updated weights for policy 0, policy_version 954943 (0.0009) [2023-12-26 22:19:25,613][105620] Updated weights for policy 1, policy_version 955020 (0.0008) [2023-12-26 22:19:25,664][105620] Updated weights for policy 1, policy_version 955030 (0.0009) [2023-12-26 22:19:25,724][105620] Updated weights for policy 1, policy_version 955040 (0.0009) [2023-12-26 22:19:26,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19114.8, 300 sec: 19105.4). Total num frames: 489029632. Throughput: 0: 9537.6, 1: 9582.0. Samples: 489036928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:19:26,063][104569] Avg episode reward: [(0, '9089.354'), (1, '8931.916')] [2023-12-26 22:19:26,141][105692] Updated weights for policy 0, policy_version 954954 (0.0009) [2023-12-26 22:19:26,203][105692] Updated weights for policy 0, policy_version 954964 (0.0010) [2023-12-26 22:19:26,265][105692] Updated weights for policy 0, policy_version 954974 (0.0010) [2023-12-26 22:19:26,317][105692] Updated weights for policy 0, policy_version 954984 (0.0010) [2023-12-26 22:19:26,505][105620] Updated weights for policy 1, policy_version 955050 (0.0012) [2023-12-26 22:19:26,554][105620] Updated weights for policy 1, policy_version 955060 (0.0008) [2023-12-26 22:19:26,603][105620] Updated weights for policy 1, policy_version 955070 (0.0007) [2023-12-26 22:19:26,658][105620] Updated weights for policy 1, policy_version 955080 (0.0008) [2023-12-26 22:19:27,055][105692] Updated weights for policy 0, policy_version 954994 (0.0009) [2023-12-26 22:19:27,068][105585] KL-divergence is very high: 100.1057 [2023-12-26 22:19:27,116][105585] KL-divergence is very high: 153.3297 [2023-12-26 22:19:27,118][105692] Updated weights for policy 0, policy_version 955004 (0.0009) [2023-12-26 22:19:27,165][105585] KL-divergence is very high: 115.0881 [2023-12-26 22:19:27,176][105692] Updated weights for policy 0, policy_version 955014 (0.0008) [2023-12-26 22:19:27,382][105620] Updated weights for policy 1, policy_version 955090 (0.0006) [2023-12-26 22:19:27,441][105620] Updated weights for policy 1, policy_version 955100 (0.0005) [2023-12-26 22:19:27,497][105620] Updated weights for policy 1, policy_version 955110 (0.0007) [2023-12-26 22:19:27,809][105692] Updated weights for policy 0, policy_version 955024 (0.0006) [2023-12-26 22:19:27,873][105692] Updated weights for policy 0, policy_version 955034 (0.0007) [2023-12-26 22:19:27,939][105692] Updated weights for policy 0, policy_version 955044 (0.0005) [2023-12-26 22:19:28,324][105620] Updated weights for policy 1, policy_version 955120 (0.0009) [2023-12-26 22:19:28,390][105620] Updated weights for policy 1, policy_version 955130 (0.0008) [2023-12-26 22:19:28,450][105620] Updated weights for policy 1, policy_version 955140 (0.0007) [2023-12-26 22:19:28,482][105692] Updated weights for policy 0, policy_version 955054 (0.0007) [2023-12-26 22:19:28,545][105692] Updated weights for policy 0, policy_version 955064 (0.0008) [2023-12-26 22:19:28,606][105692] Updated weights for policy 0, policy_version 955074 (0.0009) [2023-12-26 22:19:29,153][105620] Updated weights for policy 1, policy_version 955150 (0.0007) [2023-12-26 22:19:29,208][105620] Updated weights for policy 1, policy_version 955160 (0.0009) [2023-12-26 22:19:29,277][105620] Updated weights for policy 1, policy_version 955170 (0.0009) [2023-12-26 22:19:29,349][105692] Updated weights for policy 0, policy_version 955084 (0.0009) [2023-12-26 22:19:29,410][105692] Updated weights for policy 0, policy_version 955094 (0.0007) [2023-12-26 22:19:29,468][105692] Updated weights for policy 0, policy_version 955104 (0.0007) [2023-12-26 22:19:30,116][105620] Updated weights for policy 1, policy_version 955180 (0.0009) [2023-12-26 22:19:30,181][105620] Updated weights for policy 1, policy_version 955190 (0.0005) [2023-12-26 22:19:30,235][105620] Updated weights for policy 1, policy_version 955200 (0.0005) [2023-12-26 22:19:30,264][105692] Updated weights for policy 0, policy_version 955114 (0.0007) [2023-12-26 22:19:30,314][105692] Updated weights for policy 0, policy_version 955124 (0.0010) [2023-12-26 22:19:30,365][105692] Updated weights for policy 0, policy_version 955134 (0.0009) [2023-12-26 22:19:30,426][105692] Updated weights for policy 0, policy_version 955144 (0.0009) [2023-12-26 22:19:30,943][105620] Updated weights for policy 1, policy_version 955210 (0.0005) [2023-12-26 22:19:31,000][105620] Updated weights for policy 1, policy_version 955220 (0.0006) [2023-12-26 22:19:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 18978.1, 300 sec: 19105.4). Total num frames: 489119744. Throughput: 0: 9650.4, 1: 9570.4. Samples: 489097720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:19:31,063][104569] Avg episode reward: [(0, '9173.528'), (1, '9019.988')] [2023-12-26 22:19:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000955144_244555776.pth... [2023-12-26 22:19:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000954024_244269056.pth [2023-12-26 22:19:31,073][105620] Updated weights for policy 1, policy_version 955230 (0.0006) [2023-12-26 22:19:31,145][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000955240_244572160.pth... [2023-12-26 22:19:31,147][105620] Updated weights for policy 1, policy_version 955240 (0.0007) [2023-12-26 22:19:31,149][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000954120_244285440.pth [2023-12-26 22:19:31,229][105692] Updated weights for policy 0, policy_version 955154 (0.0010) [2023-12-26 22:19:31,287][105692] Updated weights for policy 0, policy_version 955164 (0.0007) [2023-12-26 22:19:31,352][105692] Updated weights for policy 0, policy_version 955174 (0.0008) [2023-12-26 22:19:31,713][105620] Updated weights for policy 1, policy_version 955250 (0.0008) [2023-12-26 22:19:31,771][105620] Updated weights for policy 1, policy_version 955260 (0.0008) [2023-12-26 22:19:31,825][105620] Updated weights for policy 1, policy_version 955270 (0.0008) [2023-12-26 22:19:32,145][105692] Updated weights for policy 0, policy_version 955184 (0.0009) [2023-12-26 22:19:32,192][105692] Updated weights for policy 0, policy_version 955194 (0.0009) [2023-12-26 22:19:32,238][105692] Updated weights for policy 0, policy_version 955204 (0.0008) [2023-12-26 22:19:32,516][105620] Updated weights for policy 1, policy_version 955280 (0.0009) [2023-12-26 22:19:32,570][105620] Updated weights for policy 1, policy_version 955290 (0.0008) [2023-12-26 22:19:32,639][105620] Updated weights for policy 1, policy_version 955300 (0.0009) [2023-12-26 22:19:32,995][105692] Updated weights for policy 0, policy_version 955214 (0.0007) [2023-12-26 22:19:33,047][105692] Updated weights for policy 0, policy_version 955225 (0.0010) [2023-12-26 22:19:33,100][105692] Updated weights for policy 0, policy_version 955235 (0.0009) [2023-12-26 22:19:33,325][105620] Updated weights for policy 1, policy_version 955310 (0.0007) [2023-12-26 22:19:33,370][105620] Updated weights for policy 1, policy_version 955320 (0.0005) [2023-12-26 22:19:33,422][105620] Updated weights for policy 1, policy_version 955330 (0.0005) [2023-12-26 22:19:33,928][105692] Updated weights for policy 0, policy_version 955245 (0.0008) [2023-12-26 22:19:33,979][105692] Updated weights for policy 0, policy_version 955255 (0.0008) [2023-12-26 22:19:34,023][105692] Updated weights for policy 0, policy_version 955265 (0.0007) [2023-12-26 22:19:34,082][105620] Updated weights for policy 1, policy_version 955340 (0.0007) [2023-12-26 22:19:34,147][105620] Updated weights for policy 1, policy_version 955350 (0.0010) [2023-12-26 22:19:34,214][105620] Updated weights for policy 1, policy_version 955360 (0.0008) [2023-12-26 22:19:34,799][105692] Updated weights for policy 0, policy_version 955275 (0.0008) [2023-12-26 22:19:34,833][105585] KL-divergence is very high: 221.6578 [2023-12-26 22:19:34,855][105692] Updated weights for policy 0, policy_version 955285 (0.0010) [2023-12-26 22:19:34,880][105585] KL-divergence is very high: 457.0887 [2023-12-26 22:19:34,911][105692] Updated weights for policy 0, policy_version 955295 (0.0008) [2023-12-26 22:19:34,919][105585] KL-divergence is very high: 525.4554 [2023-12-26 22:19:34,942][105620] Updated weights for policy 1, policy_version 955370 (0.0008) [2023-12-26 22:19:34,991][105620] Updated weights for policy 1, policy_version 955380 (0.0007) [2023-12-26 22:19:35,041][105620] Updated weights for policy 1, policy_version 955390 (0.0005) [2023-12-26 22:19:35,093][105620] Updated weights for policy 1, policy_version 955400 (0.0006) [2023-12-26 22:19:35,744][105620] Updated weights for policy 1, policy_version 955410 (0.0005) [2023-12-26 22:19:35,745][105692] Updated weights for policy 0, policy_version 955305 (0.0009) [2023-12-26 22:19:35,797][105692] Updated weights for policy 0, policy_version 955315 (0.0009) [2023-12-26 22:19:35,798][105620] Updated weights for policy 1, policy_version 955420 (0.0005) [2023-12-26 22:19:35,846][105692] Updated weights for policy 0, policy_version 955325 (0.0009) [2023-12-26 22:19:35,868][105620] Updated weights for policy 1, policy_version 955430 (0.0005) [2023-12-26 22:19:35,900][105692] Updated weights for policy 0, policy_version 955335 (0.0009) [2023-12-26 22:19:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19133.2). Total num frames: 489226240. Throughput: 0: 9547.4, 1: 9644.6. Samples: 489211624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:19:36,063][104569] Avg episode reward: [(0, '9172.542'), (1, '8920.059')] [2023-12-26 22:19:36,423][105620] Updated weights for policy 1, policy_version 955440 (0.0008) [2023-12-26 22:19:36,491][105620] Updated weights for policy 1, policy_version 955450 (0.0008) [2023-12-26 22:19:36,551][105620] Updated weights for policy 1, policy_version 955460 (0.0008) [2023-12-26 22:19:36,749][105692] Updated weights for policy 0, policy_version 955345 (0.0010) [2023-12-26 22:19:36,798][105692] Updated weights for policy 0, policy_version 955355 (0.0011) [2023-12-26 22:19:36,851][105692] Updated weights for policy 0, policy_version 955365 (0.0010) [2023-12-26 22:19:37,191][105620] Updated weights for policy 1, policy_version 955470 (0.0008) [2023-12-26 22:19:37,245][105620] Updated weights for policy 1, policy_version 955480 (0.0009) [2023-12-26 22:19:37,300][105620] Updated weights for policy 1, policy_version 955490 (0.0010) [2023-12-26 22:19:37,521][105692] Updated weights for policy 0, policy_version 955375 (0.0007) [2023-12-26 22:19:37,588][105692] Updated weights for policy 0, policy_version 955385 (0.0010) [2023-12-26 22:19:37,651][105692] Updated weights for policy 0, policy_version 955395 (0.0011) [2023-12-26 22:19:38,024][105620] Updated weights for policy 1, policy_version 955501 (0.0010) [2023-12-26 22:19:38,086][105620] Updated weights for policy 1, policy_version 955511 (0.0010) [2023-12-26 22:19:38,154][105620] Updated weights for policy 1, policy_version 955521 (0.0010) [2023-12-26 22:19:38,255][105692] Updated weights for policy 0, policy_version 955405 (0.0008) [2023-12-26 22:19:38,320][105692] Updated weights for policy 0, policy_version 955415 (0.0006) [2023-12-26 22:19:38,382][105692] Updated weights for policy 0, policy_version 955425 (0.0008) [2023-12-26 22:19:38,935][105692] Updated weights for policy 0, policy_version 955435 (0.0009) [2023-12-26 22:19:38,993][105692] Updated weights for policy 0, policy_version 955445 (0.0005) [2023-12-26 22:19:39,044][105620] Updated weights for policy 1, policy_version 955531 (0.0010) [2023-12-26 22:19:39,056][105692] Updated weights for policy 0, policy_version 955455 (0.0006) [2023-12-26 22:19:39,106][105620] Updated weights for policy 1, policy_version 955541 (0.0008) [2023-12-26 22:19:39,162][105620] Updated weights for policy 1, policy_version 955551 (0.0010) [2023-12-26 22:19:39,633][105692] Updated weights for policy 0, policy_version 955465 (0.0006) [2023-12-26 22:19:39,692][105692] Updated weights for policy 0, policy_version 955475 (0.0007) [2023-12-26 22:19:39,746][105692] Updated weights for policy 0, policy_version 955485 (0.0009) [2023-12-26 22:19:39,803][105692] Updated weights for policy 0, policy_version 955495 (0.0011) [2023-12-26 22:19:40,035][105620] Updated weights for policy 1, policy_version 955561 (0.0009) [2023-12-26 22:19:40,091][105620] Updated weights for policy 1, policy_version 955571 (0.0008) [2023-12-26 22:19:40,154][105620] Updated weights for policy 1, policy_version 955581 (0.0008) [2023-12-26 22:19:40,215][105620] Updated weights for policy 1, policy_version 955591 (0.0008) [2023-12-26 22:19:40,591][105692] Updated weights for policy 0, policy_version 955505 (0.0010) [2023-12-26 22:19:40,651][105692] Updated weights for policy 0, policy_version 955515 (0.0011) [2023-12-26 22:19:40,710][105692] Updated weights for policy 0, policy_version 955525 (0.0011) [2023-12-26 22:19:40,939][105620] Updated weights for policy 1, policy_version 955601 (0.0008) [2023-12-26 22:19:41,001][105620] Updated weights for policy 1, policy_version 955611 (0.0008) [2023-12-26 22:19:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19105.4). Total num frames: 489316352. Throughput: 0: 9638.0, 1: 9622.0. Samples: 489328260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:19:41,063][104569] Avg episode reward: [(0, '9171.872'), (1, '8920.050')] [2023-12-26 22:19:41,068][105620] Updated weights for policy 1, policy_version 955621 (0.0009) [2023-12-26 22:19:41,515][105692] Updated weights for policy 0, policy_version 955535 (0.0010) [2023-12-26 22:19:41,566][105692] Updated weights for policy 0, policy_version 955545 (0.0010) [2023-12-26 22:19:41,612][105692] Updated weights for policy 0, policy_version 955555 (0.0011) [2023-12-26 22:19:41,823][105620] Updated weights for policy 1, policy_version 955631 (0.0009) [2023-12-26 22:19:41,876][105620] Updated weights for policy 1, policy_version 955641 (0.0008) [2023-12-26 22:19:41,932][105620] Updated weights for policy 1, policy_version 955651 (0.0008) [2023-12-26 22:19:42,430][105692] Updated weights for policy 0, policy_version 955565 (0.0012) [2023-12-26 22:19:42,482][105692] Updated weights for policy 0, policy_version 955575 (0.0011) [2023-12-26 22:19:42,500][105585] KL-divergence is very high: 145.4930 [2023-12-26 22:19:42,542][105692] Updated weights for policy 0, policy_version 955585 (0.0011) [2023-12-26 22:19:42,547][105585] KL-divergence is very high: 151.5846 [2023-12-26 22:19:42,714][105620] Updated weights for policy 1, policy_version 955661 (0.0009) [2023-12-26 22:19:42,778][105620] Updated weights for policy 1, policy_version 955671 (0.0008) [2023-12-26 22:19:42,846][105620] Updated weights for policy 1, policy_version 955681 (0.0009) [2023-12-26 22:19:43,354][105692] Updated weights for policy 0, policy_version 955595 (0.0009) [2023-12-26 22:19:43,406][105692] Updated weights for policy 0, policy_version 955605 (0.0005) [2023-12-26 22:19:43,462][105692] Updated weights for policy 0, policy_version 955615 (0.0006) [2023-12-26 22:19:43,548][105620] Updated weights for policy 1, policy_version 955691 (0.0008) [2023-12-26 22:19:43,610][105620] Updated weights for policy 1, policy_version 955701 (0.0005) [2023-12-26 22:19:43,674][105620] Updated weights for policy 1, policy_version 955711 (0.0005) [2023-12-26 22:19:44,043][105692] Updated weights for policy 0, policy_version 955625 (0.0005) [2023-12-26 22:19:44,106][105692] Updated weights for policy 0, policy_version 955635 (0.0009) [2023-12-26 22:19:44,160][105692] Updated weights for policy 0, policy_version 955645 (0.0010) [2023-12-26 22:19:44,215][105692] Updated weights for policy 0, policy_version 955655 (0.0010) [2023-12-26 22:19:44,301][105620] Updated weights for policy 1, policy_version 955721 (0.0006) [2023-12-26 22:19:44,354][105620] Updated weights for policy 1, policy_version 955731 (0.0007) [2023-12-26 22:19:44,407][105620] Updated weights for policy 1, policy_version 955741 (0.0005) [2023-12-26 22:19:44,468][105620] Updated weights for policy 1, policy_version 955751 (0.0005) [2023-12-26 22:19:44,942][105692] Updated weights for policy 0, policy_version 955665 (0.0010) [2023-12-26 22:19:45,013][105692] Updated weights for policy 0, policy_version 955675 (0.0009) [2023-12-26 22:19:45,048][105620] Updated weights for policy 1, policy_version 955761 (0.0007) [2023-12-26 22:19:45,078][105692] Updated weights for policy 0, policy_version 955685 (0.0009) [2023-12-26 22:19:45,110][105620] Updated weights for policy 1, policy_version 955771 (0.0006) [2023-12-26 22:19:45,164][105620] Updated weights for policy 1, policy_version 955781 (0.0008) [2023-12-26 22:19:45,891][105692] Updated weights for policy 0, policy_version 955695 (0.0008) [2023-12-26 22:19:45,910][105620] Updated weights for policy 1, policy_version 955791 (0.0008) [2023-12-26 22:19:45,947][105692] Updated weights for policy 0, policy_version 955705 (0.0007) [2023-12-26 22:19:45,967][105620] Updated weights for policy 1, policy_version 955801 (0.0007) [2023-12-26 22:19:46,008][105692] Updated weights for policy 0, policy_version 955715 (0.0007) [2023-12-26 22:19:46,021][105620] Updated weights for policy 1, policy_version 955811 (0.0007) [2023-12-26 22:19:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.6, 300 sec: 19133.2). Total num frames: 489422848. Throughput: 0: 9590.3, 1: 9602.9. Samples: 489384996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:19:46,063][104569] Avg episode reward: [(0, '9170.972'), (1, '9182.485')] [2023-12-26 22:19:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000955720_244703232.pth... [2023-12-26 22:19:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000955816_244719616.pth... [2023-12-26 22:19:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000954664_244424704.pth [2023-12-26 22:19:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000954600_244416512.pth [2023-12-26 22:19:46,616][105620] Updated weights for policy 1, policy_version 955821 (0.0008) [2023-12-26 22:19:46,666][105620] Updated weights for policy 1, policy_version 955831 (0.0008) [2023-12-26 22:19:46,721][105620] Updated weights for policy 1, policy_version 955841 (0.0009) [2023-12-26 22:19:46,834][105692] Updated weights for policy 0, policy_version 955725 (0.0008) [2023-12-26 22:19:46,889][105692] Updated weights for policy 0, policy_version 955735 (0.0009) [2023-12-26 22:19:46,941][105692] Updated weights for policy 0, policy_version 955745 (0.0009) [2023-12-26 22:19:47,374][105620] Updated weights for policy 1, policy_version 955851 (0.0008) [2023-12-26 22:19:47,430][105620] Updated weights for policy 1, policy_version 955861 (0.0009) [2023-12-26 22:19:47,489][105620] Updated weights for policy 1, policy_version 955871 (0.0008) [2023-12-26 22:19:47,738][105692] Updated weights for policy 0, policy_version 955755 (0.0010) [2023-12-26 22:19:47,793][105692] Updated weights for policy 0, policy_version 955765 (0.0010) [2023-12-26 22:19:47,840][105585] KL-divergence is very high: 111.0024 [2023-12-26 22:19:47,850][105692] Updated weights for policy 0, policy_version 955776 (0.0011) [2023-12-26 22:19:48,168][105620] Updated weights for policy 1, policy_version 955881 (0.0007) [2023-12-26 22:19:48,218][105620] Updated weights for policy 1, policy_version 955891 (0.0009) [2023-12-26 22:19:48,273][105620] Updated weights for policy 1, policy_version 955901 (0.0009) [2023-12-26 22:19:48,335][105620] Updated weights for policy 1, policy_version 955911 (0.0009) [2023-12-26 22:19:48,582][105692] Updated weights for policy 0, policy_version 955786 (0.0007) [2023-12-26 22:19:48,641][105692] Updated weights for policy 0, policy_version 955796 (0.0009) [2023-12-26 22:19:48,702][105692] Updated weights for policy 0, policy_version 955806 (0.0009) [2023-12-26 22:19:48,765][105692] Updated weights for policy 0, policy_version 955816 (0.0009) [2023-12-26 22:19:49,111][105620] Updated weights for policy 1, policy_version 955921 (0.0008) [2023-12-26 22:19:49,169][105620] Updated weights for policy 1, policy_version 955931 (0.0009) [2023-12-26 22:19:49,229][105620] Updated weights for policy 1, policy_version 955941 (0.0009) [2023-12-26 22:19:49,484][105692] Updated weights for policy 0, policy_version 955826 (0.0008) [2023-12-26 22:19:49,543][105692] Updated weights for policy 0, policy_version 955836 (0.0009) [2023-12-26 22:19:49,602][105692] Updated weights for policy 0, policy_version 955846 (0.0009) [2023-12-26 22:19:50,047][105620] Updated weights for policy 1, policy_version 955951 (0.0007) [2023-12-26 22:19:50,108][105620] Updated weights for policy 1, policy_version 955961 (0.0009) [2023-12-26 22:19:50,172][105620] Updated weights for policy 1, policy_version 955971 (0.0008) [2023-12-26 22:19:50,307][105692] Updated weights for policy 0, policy_version 955856 (0.0008) [2023-12-26 22:19:50,372][105692] Updated weights for policy 0, policy_version 955866 (0.0007) [2023-12-26 22:19:50,445][105692] Updated weights for policy 0, policy_version 955876 (0.0008) [2023-12-26 22:19:50,971][105620] Updated weights for policy 1, policy_version 955981 (0.0010) [2023-12-26 22:19:51,039][105620] Updated weights for policy 1, policy_version 955991 (0.0010) [2023-12-26 22:19:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19105.4). Total num frames: 489504768. Throughput: 0: 9555.8, 1: 9626.9. Samples: 489500920. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:19:51,062][104569] Avg episode reward: [(0, '9259.566'), (1, '9264.758')] [2023-12-26 22:19:51,104][105620] Updated weights for policy 1, policy_version 956001 (0.0011) [2023-12-26 22:19:51,131][105692] Updated weights for policy 0, policy_version 955886 (0.0007) [2023-12-26 22:19:51,201][105692] Updated weights for policy 0, policy_version 955896 (0.0009) [2023-12-26 22:19:51,267][105692] Updated weights for policy 0, policy_version 955906 (0.0009) [2023-12-26 22:19:51,842][105620] Updated weights for policy 1, policy_version 956011 (0.0008) [2023-12-26 22:19:51,908][105620] Updated weights for policy 1, policy_version 956021 (0.0007) [2023-12-26 22:19:51,987][105620] Updated weights for policy 1, policy_version 956031 (0.0006) [2023-12-26 22:19:51,997][105692] Updated weights for policy 0, policy_version 955916 (0.0008) [2023-12-26 22:19:52,064][105692] Updated weights for policy 0, policy_version 955926 (0.0008) [2023-12-26 22:19:52,125][105692] Updated weights for policy 0, policy_version 955936 (0.0009) [2023-12-26 22:19:52,599][105620] Updated weights for policy 1, policy_version 956041 (0.0006) [2023-12-26 22:19:52,657][105620] Updated weights for policy 1, policy_version 956051 (0.0005) [2023-12-26 22:19:52,715][105620] Updated weights for policy 1, policy_version 956061 (0.0008) [2023-12-26 22:19:52,775][105620] Updated weights for policy 1, policy_version 956071 (0.0008) [2023-12-26 22:19:52,934][105692] Updated weights for policy 0, policy_version 955946 (0.0008) [2023-12-26 22:19:53,003][105692] Updated weights for policy 0, policy_version 955956 (0.0007) [2023-12-26 22:19:53,065][105692] Updated weights for policy 0, policy_version 955966 (0.0010) [2023-12-26 22:19:53,128][105692] Updated weights for policy 0, policy_version 955976 (0.0010) [2023-12-26 22:19:53,439][105620] Updated weights for policy 1, policy_version 956081 (0.0005) [2023-12-26 22:19:53,496][105620] Updated weights for policy 1, policy_version 956091 (0.0008) [2023-12-26 22:19:53,548][105620] Updated weights for policy 1, policy_version 956101 (0.0008) [2023-12-26 22:19:53,798][105692] Updated weights for policy 0, policy_version 955986 (0.0010) [2023-12-26 22:19:53,839][105585] KL-divergence is very high: 166.3352 [2023-12-26 22:19:53,846][105692] Updated weights for policy 0, policy_version 955996 (0.0010) [2023-12-26 22:19:53,849][105585] KL-divergence is very high: 180.9532 [2023-12-26 22:19:53,858][105585] KL-divergence is very high: 134.4933 [2023-12-26 22:19:53,877][105585] KL-divergence is very high: 178.2448 [2023-12-26 22:19:53,886][105585] KL-divergence is very high: 174.7803 [2023-12-26 22:19:53,891][105692] Updated weights for policy 0, policy_version 956006 (0.0010) [2023-12-26 22:19:53,895][105585] KL-divergence is very high: 115.4268 [2023-12-26 22:19:54,106][105620] Updated weights for policy 1, policy_version 956111 (0.0008) [2023-12-26 22:19:54,154][105620] Updated weights for policy 1, policy_version 956121 (0.0008) [2023-12-26 22:19:54,206][105620] Updated weights for policy 1, policy_version 956131 (0.0008) [2023-12-26 22:19:54,595][105692] Updated weights for policy 0, policy_version 956016 (0.0006) [2023-12-26 22:19:54,641][105692] Updated weights for policy 0, policy_version 956026 (0.0005) [2023-12-26 22:19:54,705][105692] Updated weights for policy 0, policy_version 956036 (0.0006) [2023-12-26 22:19:54,869][105620] Updated weights for policy 1, policy_version 956141 (0.0007) [2023-12-26 22:19:54,924][105620] Updated weights for policy 1, policy_version 956151 (0.0005) [2023-12-26 22:19:54,995][105620] Updated weights for policy 1, policy_version 956161 (0.0006) [2023-12-26 22:19:55,259][105692] Updated weights for policy 0, policy_version 956046 (0.0008) [2023-12-26 22:19:55,314][105692] Updated weights for policy 0, policy_version 956056 (0.0010) [2023-12-26 22:19:55,379][105692] Updated weights for policy 0, policy_version 956066 (0.0010) [2023-12-26 22:19:55,694][105620] Updated weights for policy 1, policy_version 956171 (0.0008) [2023-12-26 22:19:55,752][105620] Updated weights for policy 1, policy_version 956181 (0.0006) [2023-12-26 22:19:55,810][105620] Updated weights for policy 1, policy_version 956191 (0.0005) [2023-12-26 22:19:56,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19251.2, 300 sec: 19133.2). Total num frames: 489611264. Throughput: 0: 9544.7, 1: 9756.2. Samples: 489621100. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:19:56,062][104569] Avg episode reward: [(0, '8984.534'), (1, '9000.822')] [2023-12-26 22:19:56,105][105692] Updated weights for policy 0, policy_version 956076 (0.0010) [2023-12-26 22:19:56,154][105692] Updated weights for policy 0, policy_version 956086 (0.0010) [2023-12-26 22:19:56,202][105692] Updated weights for policy 0, policy_version 956096 (0.0010) [2023-12-26 22:19:56,417][105620] Updated weights for policy 1, policy_version 956201 (0.0006) [2023-12-26 22:19:56,482][105620] Updated weights for policy 1, policy_version 956211 (0.0006) [2023-12-26 22:19:56,549][105620] Updated weights for policy 1, policy_version 956221 (0.0005) [2023-12-26 22:19:56,620][105620] Updated weights for policy 1, policy_version 956231 (0.0006) [2023-12-26 22:19:56,788][105692] Updated weights for policy 0, policy_version 956106 (0.0008) [2023-12-26 22:19:56,837][105692] Updated weights for policy 0, policy_version 956116 (0.0007) [2023-12-26 22:19:56,893][105692] Updated weights for policy 0, policy_version 956126 (0.0005) [2023-12-26 22:19:56,949][105585] KL-divergence is very high: 107.1763 [2023-12-26 22:19:56,954][105692] Updated weights for policy 0, policy_version 956136 (0.0005) [2023-12-26 22:19:57,236][105620] Updated weights for policy 1, policy_version 956241 (0.0010) [2023-12-26 22:19:57,287][105620] Updated weights for policy 1, policy_version 956251 (0.0007) [2023-12-26 22:19:57,352][105620] Updated weights for policy 1, policy_version 956261 (0.0006) [2023-12-26 22:19:57,558][105692] Updated weights for policy 0, policy_version 956146 (0.0005) [2023-12-26 22:19:57,612][105692] Updated weights for policy 0, policy_version 956156 (0.0005) [2023-12-26 22:19:57,677][105692] Updated weights for policy 0, policy_version 956166 (0.0005) [2023-12-26 22:19:58,046][105620] Updated weights for policy 1, policy_version 956271 (0.0009) [2023-12-26 22:19:58,105][105620] Updated weights for policy 1, policy_version 956281 (0.0011) [2023-12-26 22:19:58,171][105620] Updated weights for policy 1, policy_version 956291 (0.0010) [2023-12-26 22:19:58,288][105692] Updated weights for policy 0, policy_version 956176 (0.0008) [2023-12-26 22:19:58,363][105692] Updated weights for policy 0, policy_version 956186 (0.0008) [2023-12-26 22:19:58,432][105692] Updated weights for policy 0, policy_version 956196 (0.0009) [2023-12-26 22:19:59,054][105620] Updated weights for policy 1, policy_version 956301 (0.0011) [2023-12-26 22:19:59,125][105620] Updated weights for policy 1, policy_version 956311 (0.0010) [2023-12-26 22:19:59,189][105620] Updated weights for policy 1, policy_version 956321 (0.0011) [2023-12-26 22:19:59,235][105692] Updated weights for policy 0, policy_version 956206 (0.0009) [2023-12-26 22:19:59,302][105692] Updated weights for policy 0, policy_version 956216 (0.0009) [2023-12-26 22:19:59,372][105692] Updated weights for policy 0, policy_version 956226 (0.0008) [2023-12-26 22:19:59,918][105620] Updated weights for policy 1, policy_version 956331 (0.0011) [2023-12-26 22:19:59,989][105620] Updated weights for policy 1, policy_version 956341 (0.0011) [2023-12-26 22:20:00,046][105620] Updated weights for policy 1, policy_version 956351 (0.0011) [2023-12-26 22:20:00,051][105586] KL-divergence is very high: 160.3065 [2023-12-26 22:20:00,148][105692] Updated weights for policy 0, policy_version 956236 (0.0009) [2023-12-26 22:20:00,203][105692] Updated weights for policy 0, policy_version 956246 (0.0008) [2023-12-26 22:20:00,244][105585] KL-divergence is very high: 161.2123 [2023-12-26 22:20:00,263][105692] Updated weights for policy 0, policy_version 956256 (0.0008) [2023-12-26 22:20:00,295][105585] KL-divergence is very high: 180.6619 [2023-12-26 22:20:00,785][105620] Updated weights for policy 1, policy_version 956361 (0.0010) [2023-12-26 22:20:00,857][105620] Updated weights for policy 1, policy_version 956371 (0.0010) [2023-12-26 22:20:00,921][105620] Updated weights for policy 1, policy_version 956381 (0.0010) [2023-12-26 22:20:00,957][105692] Updated weights for policy 0, policy_version 956266 (0.0008) [2023-12-26 22:20:00,985][105620] Updated weights for policy 1, policy_version 956391 (0.0010) [2023-12-26 22:20:01,006][105692] Updated weights for policy 0, policy_version 956276 (0.0006) [2023-12-26 22:20:01,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19251.2, 300 sec: 19160.9). Total num frames: 489709568. Throughput: 0: 9676.1, 1: 9783.8. Samples: 489682432. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:20:01,063][104569] Avg episode reward: [(0, '8894.230'), (1, '8908.285')] [2023-12-26 22:20:01,067][105692] Updated weights for policy 0, policy_version 956286 (0.0011) [2023-12-26 22:20:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000956392_244867072.pth... [2023-12-26 22:20:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000955240_244572160.pth [2023-12-26 22:20:01,128][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000956296_244850688.pth... [2023-12-26 22:20:01,131][105692] Updated weights for policy 0, policy_version 956296 (0.0010) [2023-12-26 22:20:01,152][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000955144_244555776.pth [2023-12-26 22:20:01,738][105620] Updated weights for policy 1, policy_version 956401 (0.0009) [2023-12-26 22:20:01,802][105620] Updated weights for policy 1, policy_version 956411 (0.0007) [2023-12-26 22:20:01,864][105620] Updated weights for policy 1, policy_version 956421 (0.0007) [2023-12-26 22:20:01,878][105692] Updated weights for policy 0, policy_version 956306 (0.0011) [2023-12-26 22:20:01,934][105692] Updated weights for policy 0, policy_version 956316 (0.0009) [2023-12-26 22:20:01,994][105692] Updated weights for policy 0, policy_version 956326 (0.0006) [2023-12-26 22:20:02,590][105620] Updated weights for policy 1, policy_version 956431 (0.0005) [2023-12-26 22:20:02,664][105620] Updated weights for policy 1, policy_version 956441 (0.0005) [2023-12-26 22:20:02,736][105620] Updated weights for policy 1, policy_version 956451 (0.0007) [2023-12-26 22:20:02,747][105692] Updated weights for policy 0, policy_version 956336 (0.0010) [2023-12-26 22:20:02,807][105692] Updated weights for policy 0, policy_version 956346 (0.0008) [2023-12-26 22:20:02,854][105692] Updated weights for policy 0, policy_version 956356 (0.0006) [2023-12-26 22:20:03,267][105620] Updated weights for policy 1, policy_version 956461 (0.0007) [2023-12-26 22:20:03,315][105620] Updated weights for policy 1, policy_version 956471 (0.0008) [2023-12-26 22:20:03,366][105620] Updated weights for policy 1, policy_version 956481 (0.0005) [2023-12-26 22:20:03,490][105692] Updated weights for policy 0, policy_version 956366 (0.0005) [2023-12-26 22:20:03,539][105692] Updated weights for policy 0, policy_version 956376 (0.0006) [2023-12-26 22:20:03,600][105692] Updated weights for policy 0, policy_version 956386 (0.0005) [2023-12-26 22:20:03,900][105620] Updated weights for policy 1, policy_version 956491 (0.0007) [2023-12-26 22:20:03,964][105620] Updated weights for policy 1, policy_version 956501 (0.0009) [2023-12-26 22:20:04,030][105620] Updated weights for policy 1, policy_version 956511 (0.0010) [2023-12-26 22:20:04,241][105692] Updated weights for policy 0, policy_version 956396 (0.0007) [2023-12-26 22:20:04,298][105692] Updated weights for policy 0, policy_version 956406 (0.0011) [2023-12-26 22:20:04,363][105692] Updated weights for policy 0, policy_version 956416 (0.0011) [2023-12-26 22:20:04,661][105620] Updated weights for policy 1, policy_version 956521 (0.0007) [2023-12-26 22:20:04,721][105620] Updated weights for policy 1, policy_version 956531 (0.0006) [2023-12-26 22:20:04,788][105620] Updated weights for policy 1, policy_version 956541 (0.0006) [2023-12-26 22:20:04,861][105620] Updated weights for policy 1, policy_version 956551 (0.0005) [2023-12-26 22:20:05,046][105692] Updated weights for policy 0, policy_version 956426 (0.0010) [2023-12-26 22:20:05,116][105692] Updated weights for policy 0, policy_version 956436 (0.0006) [2023-12-26 22:20:05,172][105692] Updated weights for policy 0, policy_version 956446 (0.0006) [2023-12-26 22:20:05,229][105692] Updated weights for policy 0, policy_version 956456 (0.0005) [2023-12-26 22:20:05,438][105620] Updated weights for policy 1, policy_version 956561 (0.0005) [2023-12-26 22:20:05,490][105620] Updated weights for policy 1, policy_version 956571 (0.0005) [2023-12-26 22:20:05,554][105620] Updated weights for policy 1, policy_version 956581 (0.0005) [2023-12-26 22:20:05,843][105692] Updated weights for policy 0, policy_version 956466 (0.0006) [2023-12-26 22:20:05,898][105692] Updated weights for policy 0, policy_version 956476 (0.0008) [2023-12-26 22:20:05,952][105692] Updated weights for policy 0, policy_version 956486 (0.0010) [2023-12-26 22:20:06,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.8, 300 sec: 19161.0). Total num frames: 489816064. Throughput: 0: 9738.5, 1: 9834.5. Samples: 489801088. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:20:06,062][104569] Avg episode reward: [(0, '9078.008'), (1, '9173.384')] [2023-12-26 22:20:06,096][105620] Updated weights for policy 1, policy_version 956591 (0.0009) [2023-12-26 22:20:06,155][105620] Updated weights for policy 1, policy_version 956601 (0.0010) [2023-12-26 22:20:06,218][105620] Updated weights for policy 1, policy_version 956611 (0.0008) [2023-12-26 22:20:06,669][105692] Updated weights for policy 0, policy_version 956496 (0.0009) [2023-12-26 22:20:06,725][105692] Updated weights for policy 0, policy_version 956506 (0.0011) [2023-12-26 22:20:06,782][105692] Updated weights for policy 0, policy_version 956516 (0.0011) [2023-12-26 22:20:06,990][105620] Updated weights for policy 1, policy_version 956621 (0.0008) [2023-12-26 22:20:07,050][105620] Updated weights for policy 1, policy_version 956631 (0.0008) [2023-12-26 22:20:07,113][105620] Updated weights for policy 1, policy_version 956641 (0.0008) [2023-12-26 22:20:07,541][105692] Updated weights for policy 0, policy_version 956526 (0.0011) [2023-12-26 22:20:07,601][105692] Updated weights for policy 0, policy_version 956536 (0.0010) [2023-12-26 22:20:07,657][105692] Updated weights for policy 0, policy_version 956546 (0.0010) [2023-12-26 22:20:07,859][105620] Updated weights for policy 1, policy_version 956651 (0.0008) [2023-12-26 22:20:07,911][105620] Updated weights for policy 1, policy_version 956661 (0.0007) [2023-12-26 22:20:07,955][105620] Updated weights for policy 1, policy_version 956671 (0.0008) [2023-12-26 22:20:08,417][105692] Updated weights for policy 0, policy_version 956556 (0.0011) [2023-12-26 22:20:08,478][105692] Updated weights for policy 0, policy_version 956566 (0.0010) [2023-12-26 22:20:08,541][105692] Updated weights for policy 0, policy_version 956576 (0.0011) [2023-12-26 22:20:08,757][105620] Updated weights for policy 1, policy_version 956681 (0.0008) [2023-12-26 22:20:08,808][105620] Updated weights for policy 1, policy_version 956691 (0.0008) [2023-12-26 22:20:08,856][105620] Updated weights for policy 1, policy_version 956701 (0.0008) [2023-12-26 22:20:08,904][105620] Updated weights for policy 1, policy_version 956711 (0.0007) [2023-12-26 22:20:09,282][105692] Updated weights for policy 0, policy_version 956586 (0.0010) [2023-12-26 22:20:09,353][105692] Updated weights for policy 0, policy_version 956596 (0.0011) [2023-12-26 22:20:09,424][105692] Updated weights for policy 0, policy_version 956606 (0.0010) [2023-12-26 22:20:09,486][105692] Updated weights for policy 0, policy_version 956616 (0.0011) [2023-12-26 22:20:09,734][105620] Updated weights for policy 1, policy_version 956721 (0.0008) [2023-12-26 22:20:09,789][105620] Updated weights for policy 1, policy_version 956731 (0.0008) [2023-12-26 22:20:09,855][105620] Updated weights for policy 1, policy_version 956741 (0.0008) [2023-12-26 22:20:10,276][105692] Updated weights for policy 0, policy_version 956626 (0.0011) [2023-12-26 22:20:10,332][105692] Updated weights for policy 0, policy_version 956636 (0.0011) [2023-12-26 22:20:10,395][105692] Updated weights for policy 0, policy_version 956646 (0.0011) [2023-12-26 22:20:10,584][105620] Updated weights for policy 1, policy_version 956751 (0.0007) [2023-12-26 22:20:10,646][105620] Updated weights for policy 1, policy_version 956761 (0.0008) [2023-12-26 22:20:10,716][105620] Updated weights for policy 1, policy_version 956771 (0.0009) [2023-12-26 22:20:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.8, 300 sec: 19161.0). Total num frames: 489906176. Throughput: 0: 9697.4, 1: 9839.6. Samples: 489916092. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:20:11,062][104569] Avg episode reward: [(0, '9169.845'), (1, '9176.164')] [2023-12-26 22:20:11,067][105692] Updated weights for policy 0, policy_version 956656 (0.0009) [2023-12-26 22:20:11,123][105692] Updated weights for policy 0, policy_version 956666 (0.0008) [2023-12-26 22:20:11,189][105692] Updated weights for policy 0, policy_version 956676 (0.0008) [2023-12-26 22:20:11,529][105620] Updated weights for policy 1, policy_version 956781 (0.0009) [2023-12-26 22:20:11,599][105620] Updated weights for policy 1, policy_version 956791 (0.0008) [2023-12-26 22:20:11,668][105620] Updated weights for policy 1, policy_version 956801 (0.0008) [2023-12-26 22:20:11,933][105692] Updated weights for policy 0, policy_version 956686 (0.0008) [2023-12-26 22:20:11,986][105692] Updated weights for policy 0, policy_version 956696 (0.0008) [2023-12-26 22:20:12,042][105692] Updated weights for policy 0, policy_version 956706 (0.0008) [2023-12-26 22:20:12,377][105620] Updated weights for policy 1, policy_version 956811 (0.0008) [2023-12-26 22:20:12,442][105620] Updated weights for policy 1, policy_version 956821 (0.0007) [2023-12-26 22:20:12,505][105620] Updated weights for policy 1, policy_version 956831 (0.0009) [2023-12-26 22:20:12,846][105692] Updated weights for policy 0, policy_version 956716 (0.0007) [2023-12-26 22:20:12,916][105692] Updated weights for policy 0, policy_version 956726 (0.0006) [2023-12-26 22:20:12,973][105692] Updated weights for policy 0, policy_version 956736 (0.0009) [2023-12-26 22:20:13,332][105620] Updated weights for policy 1, policy_version 956841 (0.0008) [2023-12-26 22:20:13,398][105620] Updated weights for policy 1, policy_version 956851 (0.0007) [2023-12-26 22:20:13,459][105620] Updated weights for policy 1, policy_version 956861 (0.0008) [2023-12-26 22:20:13,509][105620] Updated weights for policy 1, policy_version 956871 (0.0008) [2023-12-26 22:20:13,545][105692] Updated weights for policy 0, policy_version 956746 (0.0009) [2023-12-26 22:20:13,599][105692] Updated weights for policy 0, policy_version 956756 (0.0006) [2023-12-26 22:20:13,651][105692] Updated weights for policy 0, policy_version 956766 (0.0010) [2023-12-26 22:20:13,703][105692] Updated weights for policy 0, policy_version 956776 (0.0010) [2023-12-26 22:20:14,192][105620] Updated weights for policy 1, policy_version 956881 (0.0006) [2023-12-26 22:20:14,246][105620] Updated weights for policy 1, policy_version 956891 (0.0007) [2023-12-26 22:20:14,306][105620] Updated weights for policy 1, policy_version 956901 (0.0006) [2023-12-26 22:20:14,371][105692] Updated weights for policy 0, policy_version 956786 (0.0005) [2023-12-26 22:20:14,430][105692] Updated weights for policy 0, policy_version 956796 (0.0007) [2023-12-26 22:20:14,494][105692] Updated weights for policy 0, policy_version 956806 (0.0008) [2023-12-26 22:20:15,040][105620] Updated weights for policy 1, policy_version 956911 (0.0007) [2023-12-26 22:20:15,097][105620] Updated weights for policy 1, policy_version 956921 (0.0006) [2023-12-26 22:20:15,125][105692] Updated weights for policy 0, policy_version 956816 (0.0010) [2023-12-26 22:20:15,160][105620] Updated weights for policy 1, policy_version 956931 (0.0006) [2023-12-26 22:20:15,189][105692] Updated weights for policy 0, policy_version 956826 (0.0011) [2023-12-26 22:20:15,250][105692] Updated weights for policy 0, policy_version 956836 (0.0010) [2023-12-26 22:20:15,757][105620] Updated weights for policy 1, policy_version 956941 (0.0006) [2023-12-26 22:20:15,827][105620] Updated weights for policy 1, policy_version 956951 (0.0008) [2023-12-26 22:20:15,892][105620] Updated weights for policy 1, policy_version 956961 (0.0008) [2023-12-26 22:20:15,943][105692] Updated weights for policy 0, policy_version 956846 (0.0010) [2023-12-26 22:20:15,999][105692] Updated weights for policy 0, policy_version 956856 (0.0010) [2023-12-26 22:20:16,048][105692] Updated weights for policy 0, policy_version 956866 (0.0010) [2023-12-26 22:20:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19161.0). Total num frames: 490004480. Throughput: 0: 9644.0, 1: 9812.7. Samples: 489973272. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:20:16,062][104569] Avg episode reward: [(0, '8897.845'), (1, '9176.145')] [2023-12-26 22:20:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000956968_245014528.pth... [2023-12-26 22:20:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000955816_244719616.pth [2023-12-26 22:20:16,083][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000956872_244998144.pth... [2023-12-26 22:20:16,087][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000955720_244703232.pth [2023-12-26 22:20:16,646][105692] Updated weights for policy 0, policy_version 956876 (0.0007) [2023-12-26 22:20:16,708][105692] Updated weights for policy 0, policy_version 956886 (0.0005) [2023-12-26 22:20:16,715][105620] Updated weights for policy 1, policy_version 956971 (0.0008) [2023-12-26 22:20:16,769][105692] Updated weights for policy 0, policy_version 956896 (0.0005) [2023-12-26 22:20:16,777][105620] Updated weights for policy 1, policy_version 956981 (0.0005) [2023-12-26 22:20:16,834][105620] Updated weights for policy 1, policy_version 956991 (0.0009) [2023-12-26 22:20:17,421][105692] Updated weights for policy 0, policy_version 956906 (0.0005) [2023-12-26 22:20:17,483][105692] Updated weights for policy 0, policy_version 956916 (0.0006) [2023-12-26 22:20:17,496][105620] Updated weights for policy 1, policy_version 957001 (0.0008) [2023-12-26 22:20:17,549][105692] Updated weights for policy 0, policy_version 956926 (0.0009) [2023-12-26 22:20:17,556][105620] Updated weights for policy 1, policy_version 957011 (0.0005) [2023-12-26 22:20:17,613][105620] Updated weights for policy 1, policy_version 957021 (0.0006) [2023-12-26 22:20:17,615][105692] Updated weights for policy 0, policy_version 956936 (0.0008) [2023-12-26 22:20:17,677][105620] Updated weights for policy 1, policy_version 957031 (0.0006) [2023-12-26 22:20:18,310][105692] Updated weights for policy 0, policy_version 956946 (0.0009) [2023-12-26 22:20:18,375][105692] Updated weights for policy 0, policy_version 956956 (0.0008) [2023-12-26 22:20:18,381][105620] Updated weights for policy 1, policy_version 957041 (0.0008) [2023-12-26 22:20:18,433][105692] Updated weights for policy 0, policy_version 956966 (0.0006) [2023-12-26 22:20:18,446][105620] Updated weights for policy 1, policy_version 957051 (0.0007) [2023-12-26 22:20:18,498][105620] Updated weights for policy 1, policy_version 957061 (0.0009) [2023-12-26 22:20:19,188][105692] Updated weights for policy 0, policy_version 956976 (0.0006) [2023-12-26 22:20:19,251][105692] Updated weights for policy 0, policy_version 956986 (0.0007) [2023-12-26 22:20:19,253][105585] KL-divergence is very high: 139.1633 [2023-12-26 22:20:19,259][105585] KL-divergence is very high: 183.0078 [2023-12-26 22:20:19,267][105620] Updated weights for policy 1, policy_version 957071 (0.0009) [2023-12-26 22:20:19,270][105585] KL-divergence is very high: 229.8515 [2023-12-26 22:20:19,301][105585] KL-divergence is very high: 238.6776 [2023-12-26 22:20:19,307][105585] KL-divergence is very high: 278.6364 [2023-12-26 22:20:19,311][105692] Updated weights for policy 0, policy_version 956996 (0.0007) [2023-12-26 22:20:19,320][105585] KL-divergence is very high: 286.8846 [2023-12-26 22:20:19,326][105620] Updated weights for policy 1, policy_version 957081 (0.0008) [2023-12-26 22:20:19,391][105620] Updated weights for policy 1, policy_version 957091 (0.0008) [2023-12-26 22:20:20,054][105585] KL-divergence is very high: 152.7975 [2023-12-26 22:20:20,070][105692] Updated weights for policy 0, policy_version 957006 (0.0009) [2023-12-26 22:20:20,072][105585] KL-divergence is very high: 171.0470 [2023-12-26 22:20:20,100][105585] KL-divergence is very high: 122.1009 [2023-12-26 22:20:20,109][105620] Updated weights for policy 1, policy_version 957101 (0.0009) [2023-12-26 22:20:20,119][105585] KL-divergence is very high: 141.2952 [2023-12-26 22:20:20,131][105692] Updated weights for policy 0, policy_version 957016 (0.0006) [2023-12-26 22:20:20,151][105585] KL-divergence is very high: 103.1230 [2023-12-26 22:20:20,169][105585] KL-divergence is very high: 122.7523 [2023-12-26 22:20:20,178][105620] Updated weights for policy 1, policy_version 957111 (0.0006) [2023-12-26 22:20:20,198][105692] Updated weights for policy 0, policy_version 957026 (0.0006) [2023-12-26 22:20:20,227][105585] KL-divergence is very high: 102.3563 [2023-12-26 22:20:20,244][105620] Updated weights for policy 1, policy_version 957121 (0.0006) [2023-12-26 22:20:20,902][105692] Updated weights for policy 0, policy_version 957036 (0.0007) [2023-12-26 22:20:20,966][105692] Updated weights for policy 0, policy_version 957046 (0.0011) [2023-12-26 22:20:20,980][105620] Updated weights for policy 1, policy_version 957131 (0.0009) [2023-12-26 22:20:21,023][105692] Updated weights for policy 0, policy_version 957056 (0.0011) [2023-12-26 22:20:21,040][105620] Updated weights for policy 1, policy_version 957141 (0.0010) [2023-12-26 22:20:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19160.9). Total num frames: 490094592. Throughput: 0: 9789.2, 1: 9789.8. Samples: 490092676. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:20:21,063][104569] Avg episode reward: [(0, '8990.282'), (1, '9081.931')] [2023-12-26 22:20:21,109][105620] Updated weights for policy 1, policy_version 957151 (0.0008) [2023-12-26 22:20:21,824][105692] Updated weights for policy 0, policy_version 957066 (0.0009) [2023-12-26 22:20:21,880][105692] Updated weights for policy 0, policy_version 957076 (0.0006) [2023-12-26 22:20:21,935][105620] Updated weights for policy 1, policy_version 957161 (0.0008) [2023-12-26 22:20:21,942][105692] Updated weights for policy 0, policy_version 957086 (0.0007) [2023-12-26 22:20:21,989][105620] Updated weights for policy 1, policy_version 957171 (0.0007) [2023-12-26 22:20:22,003][105692] Updated weights for policy 0, policy_version 957096 (0.0011) [2023-12-26 22:20:22,040][105620] Updated weights for policy 1, policy_version 957181 (0.0008) [2023-12-26 22:20:22,096][105620] Updated weights for policy 1, policy_version 957191 (0.0008) [2023-12-26 22:20:22,643][105692] Updated weights for policy 0, policy_version 957106 (0.0009) [2023-12-26 22:20:22,701][105692] Updated weights for policy 0, policy_version 957116 (0.0009) [2023-12-26 22:20:22,769][105692] Updated weights for policy 0, policy_version 957126 (0.0006) [2023-12-26 22:20:22,959][105620] Updated weights for policy 1, policy_version 957201 (0.0009) [2023-12-26 22:20:23,026][105620] Updated weights for policy 1, policy_version 957211 (0.0008) [2023-12-26 22:20:23,084][105620] Updated weights for policy 1, policy_version 957221 (0.0006) [2023-12-26 22:20:23,354][105692] Updated weights for policy 0, policy_version 957136 (0.0008) [2023-12-26 22:20:23,409][105692] Updated weights for policy 0, policy_version 957146 (0.0009) [2023-12-26 22:20:23,473][105692] Updated weights for policy 0, policy_version 957156 (0.0009) [2023-12-26 22:20:23,867][105620] Updated weights for policy 1, policy_version 957231 (0.0005) [2023-12-26 22:20:23,931][105620] Updated weights for policy 1, policy_version 957241 (0.0005) [2023-12-26 22:20:23,987][105620] Updated weights for policy 1, policy_version 957251 (0.0005) [2023-12-26 22:20:24,082][105692] Updated weights for policy 0, policy_version 957166 (0.0007) [2023-12-26 22:20:24,143][105692] Updated weights for policy 0, policy_version 957176 (0.0008) [2023-12-26 22:20:24,211][105692] Updated weights for policy 0, policy_version 957186 (0.0010) [2023-12-26 22:20:24,614][105620] Updated weights for policy 1, policy_version 957261 (0.0007) [2023-12-26 22:20:24,683][105620] Updated weights for policy 1, policy_version 957271 (0.0010) [2023-12-26 22:20:24,749][105620] Updated weights for policy 1, policy_version 957281 (0.0010) [2023-12-26 22:20:24,782][105692] Updated weights for policy 0, policy_version 957196 (0.0009) [2023-12-26 22:20:24,840][105692] Updated weights for policy 0, policy_version 957206 (0.0010) [2023-12-26 22:20:24,902][105692] Updated weights for policy 0, policy_version 957216 (0.0010) [2023-12-26 22:20:25,462][105620] Updated weights for policy 1, policy_version 957291 (0.0010) [2023-12-26 22:20:25,518][105620] Updated weights for policy 1, policy_version 957301 (0.0010) [2023-12-26 22:20:25,573][105620] Updated weights for policy 1, policy_version 957311 (0.0010) [2023-12-26 22:20:25,633][105692] Updated weights for policy 0, policy_version 957226 (0.0009) [2023-12-26 22:20:25,688][105692] Updated weights for policy 0, policy_version 957236 (0.0010) [2023-12-26 22:20:25,749][105692] Updated weights for policy 0, policy_version 957246 (0.0010) [2023-12-26 22:20:25,796][105692] Updated weights for policy 0, policy_version 957256 (0.0010) [2023-12-26 22:20:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19188.7). Total num frames: 490201088. Throughput: 0: 9821.1, 1: 9751.5. Samples: 490209028. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:20:26,063][104569] Avg episode reward: [(0, '8989.285'), (1, '8909.631')] [2023-12-26 22:20:26,256][105620] Updated weights for policy 1, policy_version 957321 (0.0010) [2023-12-26 22:20:26,315][105620] Updated weights for policy 1, policy_version 957331 (0.0010) [2023-12-26 22:20:26,374][105620] Updated weights for policy 1, policy_version 957341 (0.0010) [2023-12-26 22:20:26,425][105692] Updated weights for policy 0, policy_version 957266 (0.0011) [2023-12-26 22:20:26,434][105620] Updated weights for policy 1, policy_version 957351 (0.0011) [2023-12-26 22:20:26,465][105585] KL-divergence is very high: 105.0108 [2023-12-26 22:20:26,490][105692] Updated weights for policy 0, policy_version 957276 (0.0011) [2023-12-26 22:20:26,516][105585] KL-divergence is very high: 210.4701 [2023-12-26 22:20:26,552][105692] Updated weights for policy 0, policy_version 957286 (0.0010) [2023-12-26 22:20:27,076][105620] Updated weights for policy 1, policy_version 957361 (0.0010) [2023-12-26 22:20:27,127][105620] Updated weights for policy 1, policy_version 957371 (0.0010) [2023-12-26 22:20:27,171][105620] Updated weights for policy 1, policy_version 957381 (0.0010) [2023-12-26 22:20:27,297][105692] Updated weights for policy 0, policy_version 957296 (0.0009) [2023-12-26 22:20:27,349][105692] Updated weights for policy 0, policy_version 957306 (0.0009) [2023-12-26 22:20:27,400][105692] Updated weights for policy 0, policy_version 957316 (0.0009) [2023-12-26 22:20:27,908][105620] Updated weights for policy 1, policy_version 957391 (0.0007) [2023-12-26 22:20:27,965][105620] Updated weights for policy 1, policy_version 957401 (0.0005) [2023-12-26 22:20:28,027][105620] Updated weights for policy 1, policy_version 957411 (0.0006) [2023-12-26 22:20:28,099][105692] Updated weights for policy 0, policy_version 957326 (0.0010) [2023-12-26 22:20:28,160][105692] Updated weights for policy 0, policy_version 957336 (0.0010) [2023-12-26 22:20:28,224][105692] Updated weights for policy 0, policy_version 957346 (0.0010) [2023-12-26 22:20:28,695][105620] Updated weights for policy 1, policy_version 957421 (0.0008) [2023-12-26 22:20:28,746][105620] Updated weights for policy 1, policy_version 957431 (0.0009) [2023-12-26 22:20:28,803][105620] Updated weights for policy 1, policy_version 957441 (0.0009) [2023-12-26 22:20:28,881][105692] Updated weights for policy 0, policy_version 957356 (0.0010) [2023-12-26 22:20:28,940][105692] Updated weights for policy 0, policy_version 957366 (0.0008) [2023-12-26 22:20:29,000][105692] Updated weights for policy 0, policy_version 957376 (0.0009) [2023-12-26 22:20:29,606][105620] Updated weights for policy 1, policy_version 957451 (0.0010) [2023-12-26 22:20:29,652][105620] Updated weights for policy 1, policy_version 957461 (0.0008) [2023-12-26 22:20:29,699][105620] Updated weights for policy 1, policy_version 957471 (0.0009) [2023-12-26 22:20:29,740][105692] Updated weights for policy 0, policy_version 957386 (0.0009) [2023-12-26 22:20:29,791][105692] Updated weights for policy 0, policy_version 957396 (0.0009) [2023-12-26 22:20:29,854][105692] Updated weights for policy 0, policy_version 957406 (0.0009) [2023-12-26 22:20:29,916][105692] Updated weights for policy 0, policy_version 957416 (0.0006) [2023-12-26 22:20:30,484][105620] Updated weights for policy 1, policy_version 957481 (0.0008) [2023-12-26 22:20:30,539][105620] Updated weights for policy 1, policy_version 957491 (0.0008) [2023-12-26 22:20:30,607][105620] Updated weights for policy 1, policy_version 957501 (0.0008) [2023-12-26 22:20:30,666][105692] Updated weights for policy 0, policy_version 957426 (0.0011) [2023-12-26 22:20:30,668][105620] Updated weights for policy 1, policy_version 957511 (0.0006) [2023-12-26 22:20:30,722][105692] Updated weights for policy 0, policy_version 957436 (0.0011) [2023-12-26 22:20:30,777][105692] Updated weights for policy 0, policy_version 957446 (0.0011) [2023-12-26 22:20:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19188.7). Total num frames: 490299392. Throughput: 0: 9874.4, 1: 9757.6. Samples: 490268428. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:20:31,063][104569] Avg episode reward: [(0, '8897.663'), (1, '9002.007')] [2023-12-26 22:20:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000957448_245145600.pth... [2023-12-26 22:20:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000957512_245153792.pth... [2023-12-26 22:20:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000956296_244850688.pth [2023-12-26 22:20:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000956392_244867072.pth [2023-12-26 22:20:31,397][105692] Updated weights for policy 0, policy_version 957456 (0.0009) [2023-12-26 22:20:31,454][105692] Updated weights for policy 0, policy_version 957466 (0.0007) [2023-12-26 22:20:31,502][105620] Updated weights for policy 1, policy_version 957521 (0.0008) [2023-12-26 22:20:31,509][105692] Updated weights for policy 0, policy_version 957476 (0.0005) [2023-12-26 22:20:31,556][105620] Updated weights for policy 1, policy_version 957531 (0.0009) [2023-12-26 22:20:31,607][105620] Updated weights for policy 1, policy_version 957541 (0.0009) [2023-12-26 22:20:32,207][105692] Updated weights for policy 0, policy_version 957486 (0.0008) [2023-12-26 22:20:32,255][105692] Updated weights for policy 0, policy_version 957496 (0.0010) [2023-12-26 22:20:32,315][105692] Updated weights for policy 0, policy_version 957506 (0.0011) [2023-12-26 22:20:32,354][105620] Updated weights for policy 1, policy_version 957551 (0.0009) [2023-12-26 22:20:32,402][105620] Updated weights for policy 1, policy_version 957561 (0.0010) [2023-12-26 22:20:32,446][105620] Updated weights for policy 1, policy_version 957571 (0.0010) [2023-12-26 22:20:33,049][105692] Updated weights for policy 0, policy_version 957516 (0.0008) [2023-12-26 22:20:33,104][105692] Updated weights for policy 0, policy_version 957526 (0.0006) [2023-12-26 22:20:33,155][105692] Updated weights for policy 0, policy_version 957536 (0.0008) [2023-12-26 22:20:33,209][105620] Updated weights for policy 1, policy_version 957581 (0.0010) [2023-12-26 22:20:33,269][105620] Updated weights for policy 1, policy_version 957591 (0.0011) [2023-12-26 22:20:33,328][105620] Updated weights for policy 1, policy_version 957601 (0.0010) [2023-12-26 22:20:33,818][105692] Updated weights for policy 0, policy_version 957546 (0.0008) [2023-12-26 22:20:33,884][105692] Updated weights for policy 0, policy_version 957556 (0.0008) [2023-12-26 22:20:33,939][105692] Updated weights for policy 0, policy_version 957566 (0.0007) [2023-12-26 22:20:33,993][105692] Updated weights for policy 0, policy_version 957576 (0.0008) [2023-12-26 22:20:34,068][105620] Updated weights for policy 1, policy_version 957611 (0.0009) [2023-12-26 22:20:34,128][105620] Updated weights for policy 1, policy_version 957621 (0.0005) [2023-12-26 22:20:34,195][105620] Updated weights for policy 1, policy_version 957631 (0.0007) [2023-12-26 22:20:34,825][105692] Updated weights for policy 0, policy_version 957586 (0.0009) [2023-12-26 22:20:34,864][105620] Updated weights for policy 1, policy_version 957641 (0.0011) [2023-12-26 22:20:34,886][105692] Updated weights for policy 0, policy_version 957596 (0.0008) [2023-12-26 22:20:34,914][105620] Updated weights for policy 1, policy_version 957651 (0.0008) [2023-12-26 22:20:34,937][105692] Updated weights for policy 0, policy_version 957606 (0.0006) [2023-12-26 22:20:34,962][105620] Updated weights for policy 1, policy_version 957661 (0.0008) [2023-12-26 22:20:35,019][105620] Updated weights for policy 1, policy_version 957671 (0.0009) [2023-12-26 22:20:35,625][105692] Updated weights for policy 0, policy_version 957616 (0.0008) [2023-12-26 22:20:35,679][105692] Updated weights for policy 0, policy_version 957626 (0.0009) [2023-12-26 22:20:35,725][105692] Updated weights for policy 0, policy_version 957636 (0.0008) [2023-12-26 22:20:35,780][105620] Updated weights for policy 1, policy_version 957681 (0.0009) [2023-12-26 22:20:35,863][105620] Updated weights for policy 1, policy_version 957691 (0.0009) [2023-12-26 22:20:35,921][105620] Updated weights for policy 1, policy_version 957701 (0.0010) [2023-12-26 22:20:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19216.5). Total num frames: 490397696. Throughput: 0: 9937.9, 1: 9672.6. Samples: 490383396. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:20:36,062][104569] Avg episode reward: [(0, '8898.797'), (1, '9013.574')] [2023-12-26 22:20:36,356][105692] Updated weights for policy 0, policy_version 957646 (0.0008) [2023-12-26 22:20:36,420][105692] Updated weights for policy 0, policy_version 957656 (0.0008) [2023-12-26 22:20:36,477][105692] Updated weights for policy 0, policy_version 957666 (0.0008) [2023-12-26 22:20:36,582][105620] Updated weights for policy 1, policy_version 957711 (0.0011) [2023-12-26 22:20:36,631][105620] Updated weights for policy 1, policy_version 957721 (0.0011) [2023-12-26 22:20:36,690][105620] Updated weights for policy 1, policy_version 957731 (0.0011) [2023-12-26 22:20:37,204][105692] Updated weights for policy 0, policy_version 957676 (0.0007) [2023-12-26 22:20:37,270][105692] Updated weights for policy 0, policy_version 957686 (0.0005) [2023-12-26 22:20:37,327][105692] Updated weights for policy 0, policy_version 957696 (0.0005) [2023-12-26 22:20:37,451][105620] Updated weights for policy 1, policy_version 957741 (0.0011) [2023-12-26 22:20:37,514][105620] Updated weights for policy 1, policy_version 957751 (0.0010) [2023-12-26 22:20:37,569][105620] Updated weights for policy 1, policy_version 957761 (0.0010) [2023-12-26 22:20:37,894][105692] Updated weights for policy 0, policy_version 957706 (0.0006) [2023-12-26 22:20:37,950][105692] Updated weights for policy 0, policy_version 957716 (0.0006) [2023-12-26 22:20:37,999][105692] Updated weights for policy 0, policy_version 957726 (0.0006) [2023-12-26 22:20:38,057][105692] Updated weights for policy 0, policy_version 957736 (0.0010) [2023-12-26 22:20:38,159][105620] Updated weights for policy 1, policy_version 957771 (0.0010) [2023-12-26 22:20:38,218][105620] Updated weights for policy 1, policy_version 957781 (0.0007) [2023-12-26 22:20:38,269][105620] Updated weights for policy 1, policy_version 957791 (0.0009) [2023-12-26 22:20:38,813][105692] Updated weights for policy 0, policy_version 957746 (0.0009) [2023-12-26 22:20:38,871][105692] Updated weights for policy 0, policy_version 957756 (0.0009) [2023-12-26 22:20:38,926][105692] Updated weights for policy 0, policy_version 957766 (0.0009) [2023-12-26 22:20:39,019][105620] Updated weights for policy 1, policy_version 957801 (0.0010) [2023-12-26 22:20:39,083][105620] Updated weights for policy 1, policy_version 957811 (0.0008) [2023-12-26 22:20:39,144][105620] Updated weights for policy 1, policy_version 957821 (0.0009) [2023-12-26 22:20:39,201][105620] Updated weights for policy 1, policy_version 957831 (0.0009) [2023-12-26 22:20:39,718][105692] Updated weights for policy 0, policy_version 957776 (0.0009) [2023-12-26 22:20:39,782][105692] Updated weights for policy 0, policy_version 957786 (0.0007) [2023-12-26 22:20:39,851][105692] Updated weights for policy 0, policy_version 957796 (0.0007) [2023-12-26 22:20:39,984][105620] Updated weights for policy 1, policy_version 957841 (0.0008) [2023-12-26 22:20:40,044][105620] Updated weights for policy 1, policy_version 957851 (0.0008) [2023-12-26 22:20:40,093][105620] Updated weights for policy 1, policy_version 957861 (0.0008) [2023-12-26 22:20:40,571][105692] Updated weights for policy 0, policy_version 957806 (0.0008) [2023-12-26 22:20:40,622][105692] Updated weights for policy 0, policy_version 957816 (0.0009) [2023-12-26 22:20:40,678][105692] Updated weights for policy 0, policy_version 957826 (0.0009) [2023-12-26 22:20:40,890][105620] Updated weights for policy 1, policy_version 957872 (0.0009) [2023-12-26 22:20:40,945][105620] Updated weights for policy 1, policy_version 957882 (0.0010) [2023-12-26 22:20:40,997][105620] Updated weights for policy 1, policy_version 957892 (0.0009) [2023-12-26 22:20:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19216.5). Total num frames: 490496000. Throughput: 0: 9941.7, 1: 9583.4. Samples: 490499728. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:20:41,063][104569] Avg episode reward: [(0, '8899.843'), (1, '9189.235')] [2023-12-26 22:20:41,398][105692] Updated weights for policy 0, policy_version 957836 (0.0009) [2023-12-26 22:20:41,465][105692] Updated weights for policy 0, policy_version 957846 (0.0009) [2023-12-26 22:20:41,530][105692] Updated weights for policy 0, policy_version 957856 (0.0007) [2023-12-26 22:20:41,868][105620] Updated weights for policy 1, policy_version 957902 (0.0010) [2023-12-26 22:20:41,927][105620] Updated weights for policy 1, policy_version 957912 (0.0009) [2023-12-26 22:20:41,990][105620] Updated weights for policy 1, policy_version 957922 (0.0009) [2023-12-26 22:20:42,220][105692] Updated weights for policy 0, policy_version 957866 (0.0008) [2023-12-26 22:20:42,278][105692] Updated weights for policy 0, policy_version 957876 (0.0008) [2023-12-26 22:20:42,340][105692] Updated weights for policy 0, policy_version 957886 (0.0008) [2023-12-26 22:20:42,405][105692] Updated weights for policy 0, policy_version 957896 (0.0009) [2023-12-26 22:20:42,821][105620] Updated weights for policy 1, policy_version 957932 (0.0010) [2023-12-26 22:20:42,887][105620] Updated weights for policy 1, policy_version 957942 (0.0010) [2023-12-26 22:20:42,947][105620] Updated weights for policy 1, policy_version 957952 (0.0006) [2023-12-26 22:20:43,175][105692] Updated weights for policy 0, policy_version 957906 (0.0009) [2023-12-26 22:20:43,228][105692] Updated weights for policy 0, policy_version 957916 (0.0008) [2023-12-26 22:20:43,284][105692] Updated weights for policy 0, policy_version 957926 (0.0008) [2023-12-26 22:20:43,602][105620] Updated weights for policy 1, policy_version 957962 (0.0011) [2023-12-26 22:20:43,657][105620] Updated weights for policy 1, policy_version 957972 (0.0010) [2023-12-26 22:20:43,715][105620] Updated weights for policy 1, policy_version 957982 (0.0009) [2023-12-26 22:20:43,782][105620] Updated weights for policy 1, policy_version 957992 (0.0008) [2023-12-26 22:20:43,937][105692] Updated weights for policy 0, policy_version 957936 (0.0006) [2023-12-26 22:20:43,990][105692] Updated weights for policy 0, policy_version 957946 (0.0005) [2023-12-26 22:20:44,050][105692] Updated weights for policy 0, policy_version 957956 (0.0006) [2023-12-26 22:20:44,527][105620] Updated weights for policy 1, policy_version 958002 (0.0006) [2023-12-26 22:20:44,603][105620] Updated weights for policy 1, policy_version 958012 (0.0011) [2023-12-26 22:20:44,638][105692] Updated weights for policy 0, policy_version 957966 (0.0006) [2023-12-26 22:20:44,666][105620] Updated weights for policy 1, policy_version 958022 (0.0011) [2023-12-26 22:20:44,689][105692] Updated weights for policy 0, policy_version 957976 (0.0007) [2023-12-26 22:20:44,756][105692] Updated weights for policy 0, policy_version 957986 (0.0006) [2023-12-26 22:20:45,337][105620] Updated weights for policy 1, policy_version 958032 (0.0011) [2023-12-26 22:20:45,400][105620] Updated weights for policy 1, policy_version 958042 (0.0011) [2023-12-26 22:20:45,412][105692] Updated weights for policy 0, policy_version 957996 (0.0009) [2023-12-26 22:20:45,460][105620] Updated weights for policy 1, policy_version 958052 (0.0011) [2023-12-26 22:20:45,465][105692] Updated weights for policy 0, policy_version 958006 (0.0011) [2023-12-26 22:20:45,528][105692] Updated weights for policy 0, policy_version 958016 (0.0010) [2023-12-26 22:20:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.8, 300 sec: 19188.7). Total num frames: 490586112. Throughput: 0: 9864.7, 1: 9521.3. Samples: 490554804. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:20:46,063][104569] Avg episode reward: [(0, '9081.077'), (1, '9355.904')] [2023-12-26 22:20:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000958056_245293056.pth... [2023-12-26 22:20:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000958024_245293056.pth... [2023-12-26 22:20:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000956968_245014528.pth [2023-12-26 22:20:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000956872_244998144.pth [2023-12-26 22:20:46,237][105620] Updated weights for policy 1, policy_version 958062 (0.0010) [2023-12-26 22:20:46,272][105692] Updated weights for policy 0, policy_version 958026 (0.0010) [2023-12-26 22:20:46,299][105620] Updated weights for policy 1, policy_version 958072 (0.0010) [2023-12-26 22:20:46,325][105692] Updated weights for policy 0, policy_version 958036 (0.0011) [2023-12-26 22:20:46,356][105620] Updated weights for policy 1, policy_version 958082 (0.0011) [2023-12-26 22:20:46,382][105692] Updated weights for policy 0, policy_version 958046 (0.0011) [2023-12-26 22:20:46,448][105692] Updated weights for policy 0, policy_version 958056 (0.0011) [2023-12-26 22:20:47,119][105620] Updated weights for policy 1, policy_version 958092 (0.0011) [2023-12-26 22:20:47,174][105620] Updated weights for policy 1, policy_version 958102 (0.0010) [2023-12-26 22:20:47,180][105692] Updated weights for policy 0, policy_version 958066 (0.0005) [2023-12-26 22:20:47,232][105620] Updated weights for policy 1, policy_version 958112 (0.0010) [2023-12-26 22:20:47,235][105692] Updated weights for policy 0, policy_version 958076 (0.0007) [2023-12-26 22:20:47,291][105692] Updated weights for policy 0, policy_version 958086 (0.0006) [2023-12-26 22:20:47,923][105692] Updated weights for policy 0, policy_version 958096 (0.0007) [2023-12-26 22:20:47,979][105620] Updated weights for policy 1, policy_version 958122 (0.0010) [2023-12-26 22:20:47,987][105692] Updated weights for policy 0, policy_version 958106 (0.0008) [2023-12-26 22:20:48,034][105620] Updated weights for policy 1, policy_version 958132 (0.0009) [2023-12-26 22:20:48,036][105692] Updated weights for policy 0, policy_version 958116 (0.0009) [2023-12-26 22:20:48,090][105620] Updated weights for policy 1, policy_version 958142 (0.0008) [2023-12-26 22:20:48,140][105620] Updated weights for policy 1, policy_version 958152 (0.0007) [2023-12-26 22:20:48,748][105620] Updated weights for policy 1, policy_version 958162 (0.0007) [2023-12-26 22:20:48,800][105620] Updated weights for policy 1, policy_version 958172 (0.0010) [2023-12-26 22:20:48,853][105620] Updated weights for policy 1, policy_version 958182 (0.0008) [2023-12-26 22:20:48,918][105692] Updated weights for policy 0, policy_version 958126 (0.0006) [2023-12-26 22:20:48,978][105692] Updated weights for policy 0, policy_version 958136 (0.0008) [2023-12-26 22:20:49,047][105692] Updated weights for policy 0, policy_version 958146 (0.0008) [2023-12-26 22:20:49,620][105620] Updated weights for policy 1, policy_version 958192 (0.0007) [2023-12-26 22:20:49,677][105620] Updated weights for policy 1, policy_version 958202 (0.0006) [2023-12-26 22:20:49,703][105586] KL-divergence is very high: 160.4356 [2023-12-26 22:20:49,739][105620] Updated weights for policy 1, policy_version 958212 (0.0007) [2023-12-26 22:20:49,751][105586] KL-divergence is very high: 176.1793 [2023-12-26 22:20:49,778][105692] Updated weights for policy 0, policy_version 958156 (0.0008) [2023-12-26 22:20:49,843][105692] Updated weights for policy 0, policy_version 958166 (0.0008) [2023-12-26 22:20:49,915][105692] Updated weights for policy 0, policy_version 958176 (0.0007) [2023-12-26 22:20:50,441][105620] Updated weights for policy 1, policy_version 958222 (0.0007) [2023-12-26 22:20:50,506][105620] Updated weights for policy 1, policy_version 958232 (0.0009) [2023-12-26 22:20:50,530][105692] Updated weights for policy 0, policy_version 958186 (0.0007) [2023-12-26 22:20:50,566][105620] Updated weights for policy 1, policy_version 958242 (0.0007) [2023-12-26 22:20:50,591][105692] Updated weights for policy 0, policy_version 958196 (0.0007) [2023-12-26 22:20:50,663][105692] Updated weights for policy 0, policy_version 958206 (0.0006) [2023-12-26 22:20:50,727][105692] Updated weights for policy 0, policy_version 958216 (0.0006) [2023-12-26 22:20:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19244.3). Total num frames: 490684416. Throughput: 0: 9909.3, 1: 9467.7. Samples: 490673056. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:20:51,062][104569] Avg episode reward: [(0, '9262.656'), (1, '9007.438')] [2023-12-26 22:20:51,373][105692] Updated weights for policy 0, policy_version 958226 (0.0008) [2023-12-26 22:20:51,411][105620] Updated weights for policy 1, policy_version 958252 (0.0007) [2023-12-26 22:20:51,435][105692] Updated weights for policy 0, policy_version 958236 (0.0009) [2023-12-26 22:20:51,470][105620] Updated weights for policy 1, policy_version 958262 (0.0009) [2023-12-26 22:20:51,492][105692] Updated weights for policy 0, policy_version 958246 (0.0005) [2023-12-26 22:20:51,537][105620] Updated weights for policy 1, policy_version 958272 (0.0009) [2023-12-26 22:20:52,235][105692] Updated weights for policy 0, policy_version 958256 (0.0008) [2023-12-26 22:20:52,299][105692] Updated weights for policy 0, policy_version 958266 (0.0009) [2023-12-26 22:20:52,352][105620] Updated weights for policy 1, policy_version 958282 (0.0009) [2023-12-26 22:20:52,364][105692] Updated weights for policy 0, policy_version 958276 (0.0009) [2023-12-26 22:20:52,411][105620] Updated weights for policy 1, policy_version 958292 (0.0009) [2023-12-26 22:20:52,459][105620] Updated weights for policy 1, policy_version 958302 (0.0009) [2023-12-26 22:20:52,508][105620] Updated weights for policy 1, policy_version 958312 (0.0008) [2023-12-26 22:20:53,114][105692] Updated weights for policy 0, policy_version 958286 (0.0008) [2023-12-26 22:20:53,165][105692] Updated weights for policy 0, policy_version 958296 (0.0009) [2023-12-26 22:20:53,217][105692] Updated weights for policy 0, policy_version 958306 (0.0009) [2023-12-26 22:20:53,292][105620] Updated weights for policy 1, policy_version 958322 (0.0009) [2023-12-26 22:20:53,346][105620] Updated weights for policy 1, policy_version 958332 (0.0006) [2023-12-26 22:20:53,412][105620] Updated weights for policy 1, policy_version 958342 (0.0010) [2023-12-26 22:20:53,796][105692] Updated weights for policy 0, policy_version 958316 (0.0006) [2023-12-26 22:20:53,845][105692] Updated weights for policy 0, policy_version 958326 (0.0008) [2023-12-26 22:20:53,889][105692] Updated weights for policy 0, policy_version 958336 (0.0008) [2023-12-26 22:20:54,022][105620] Updated weights for policy 1, policy_version 958352 (0.0006) [2023-12-26 22:20:54,084][105620] Updated weights for policy 1, policy_version 958362 (0.0006) [2023-12-26 22:20:54,149][105620] Updated weights for policy 1, policy_version 958372 (0.0009) [2023-12-26 22:20:54,664][105692] Updated weights for policy 0, policy_version 958346 (0.0008) [2023-12-26 22:20:54,712][105692] Updated weights for policy 0, policy_version 958356 (0.0008) [2023-12-26 22:20:54,756][105692] Updated weights for policy 0, policy_version 958366 (0.0007) [2023-12-26 22:20:54,803][105692] Updated weights for policy 0, policy_version 958376 (0.0007) [2023-12-26 22:20:54,870][105620] Updated weights for policy 1, policy_version 958382 (0.0009) [2023-12-26 22:20:54,935][105620] Updated weights for policy 1, policy_version 958392 (0.0010) [2023-12-26 22:20:55,008][105620] Updated weights for policy 1, policy_version 958402 (0.0009) [2023-12-26 22:20:55,570][105692] Updated weights for policy 0, policy_version 958386 (0.0008) [2023-12-26 22:20:55,628][105692] Updated weights for policy 0, policy_version 958396 (0.0009) [2023-12-26 22:20:55,694][105692] Updated weights for policy 0, policy_version 958406 (0.0009) [2023-12-26 22:20:55,816][105620] Updated weights for policy 1, policy_version 958412 (0.0009) [2023-12-26 22:20:55,881][105620] Updated weights for policy 1, policy_version 958422 (0.0009) [2023-12-26 22:20:55,936][105620] Updated weights for policy 1, policy_version 958432 (0.0009) [2023-12-26 22:20:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19244.3). Total num frames: 490782720. Throughput: 0: 9952.3, 1: 9423.2. Samples: 490787992. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:20:56,063][104569] Avg episode reward: [(0, '9079.520'), (1, '8650.957')] [2023-12-26 22:20:56,307][105692] Updated weights for policy 0, policy_version 958416 (0.0005) [2023-12-26 22:20:56,363][105692] Updated weights for policy 0, policy_version 958426 (0.0005) [2023-12-26 22:20:56,417][105692] Updated weights for policy 0, policy_version 958436 (0.0008) [2023-12-26 22:20:56,690][105620] Updated weights for policy 1, policy_version 958442 (0.0009) [2023-12-26 22:20:56,735][105620] Updated weights for policy 1, policy_version 958452 (0.0005) [2023-12-26 22:20:56,784][105620] Updated weights for policy 1, policy_version 958462 (0.0006) [2023-12-26 22:20:56,856][105620] Updated weights for policy 1, policy_version 958472 (0.0010) [2023-12-26 22:20:57,024][105692] Updated weights for policy 0, policy_version 958446 (0.0007) [2023-12-26 22:20:57,087][105692] Updated weights for policy 0, policy_version 958456 (0.0006) [2023-12-26 22:20:57,140][105692] Updated weights for policy 0, policy_version 958466 (0.0010) [2023-12-26 22:20:57,528][105620] Updated weights for policy 1, policy_version 958482 (0.0006) [2023-12-26 22:20:57,585][105620] Updated weights for policy 1, policy_version 958492 (0.0009) [2023-12-26 22:20:57,642][105620] Updated weights for policy 1, policy_version 958502 (0.0008) [2023-12-26 22:20:57,900][105692] Updated weights for policy 0, policy_version 958476 (0.0008) [2023-12-26 22:20:57,963][105692] Updated weights for policy 0, policy_version 958486 (0.0005) [2023-12-26 22:20:58,018][105692] Updated weights for policy 0, policy_version 958496 (0.0005) [2023-12-26 22:20:58,415][105620] Updated weights for policy 1, policy_version 958512 (0.0007) [2023-12-26 22:20:58,474][105620] Updated weights for policy 1, policy_version 958522 (0.0008) [2023-12-26 22:20:58,542][105620] Updated weights for policy 1, policy_version 958532 (0.0008) [2023-12-26 22:20:58,701][105692] Updated weights for policy 0, policy_version 958506 (0.0006) [2023-12-26 22:20:58,772][105692] Updated weights for policy 0, policy_version 958517 (0.0008) [2023-12-26 22:20:58,839][105692] Updated weights for policy 0, policy_version 958527 (0.0008) [2023-12-26 22:20:59,378][105620] Updated weights for policy 1, policy_version 958542 (0.0010) [2023-12-26 22:20:59,436][105620] Updated weights for policy 1, policy_version 958552 (0.0008) [2023-12-26 22:20:59,492][105620] Updated weights for policy 1, policy_version 958562 (0.0008) [2023-12-26 22:20:59,635][105692] Updated weights for policy 0, policy_version 958537 (0.0008) [2023-12-26 22:20:59,695][105692] Updated weights for policy 0, policy_version 958547 (0.0008) [2023-12-26 22:20:59,763][105692] Updated weights for policy 0, policy_version 958557 (0.0009) [2023-12-26 22:20:59,819][105692] Updated weights for policy 0, policy_version 958567 (0.0008) [2023-12-26 22:21:00,216][105620] Updated weights for policy 1, policy_version 958572 (0.0008) [2023-12-26 22:21:00,280][105620] Updated weights for policy 1, policy_version 958582 (0.0009) [2023-12-26 22:21:00,340][105620] Updated weights for policy 1, policy_version 958592 (0.0007) [2023-12-26 22:21:00,610][105692] Updated weights for policy 0, policy_version 958577 (0.0010) [2023-12-26 22:21:00,673][105692] Updated weights for policy 0, policy_version 958587 (0.0011) [2023-12-26 22:21:00,743][105692] Updated weights for policy 0, policy_version 958597 (0.0011) [2023-12-26 22:21:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19216.5). Total num frames: 490872832. Throughput: 0: 9977.0, 1: 9440.0. Samples: 490847040. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:21:01,063][104569] Avg episode reward: [(0, '8987.056'), (1, '8385.209')] [2023-12-26 22:21:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000958600_245440512.pth... [2023-12-26 22:21:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000958600_245432320.pth... [2023-12-26 22:21:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000957448_245145600.pth [2023-12-26 22:21:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000957512_245153792.pth [2023-12-26 22:21:01,125][105620] Updated weights for policy 1, policy_version 958602 (0.0008) [2023-12-26 22:21:01,186][105620] Updated weights for policy 1, policy_version 958612 (0.0009) [2023-12-26 22:21:01,240][105620] Updated weights for policy 1, policy_version 958622 (0.0008) [2023-12-26 22:21:01,301][105620] Updated weights for policy 1, policy_version 958632 (0.0009) [2023-12-26 22:21:01,476][105692] Updated weights for policy 0, policy_version 958607 (0.0009) [2023-12-26 22:21:01,532][105692] Updated weights for policy 0, policy_version 958617 (0.0009) [2023-12-26 22:21:01,593][105692] Updated weights for policy 0, policy_version 958627 (0.0009) [2023-12-26 22:21:02,000][105620] Updated weights for policy 1, policy_version 958642 (0.0005) [2023-12-26 22:21:02,054][105620] Updated weights for policy 1, policy_version 958652 (0.0005) [2023-12-26 22:21:02,109][105620] Updated weights for policy 1, policy_version 958662 (0.0006) [2023-12-26 22:21:02,444][105692] Updated weights for policy 0, policy_version 958637 (0.0009) [2023-12-26 22:21:02,508][105692] Updated weights for policy 0, policy_version 958647 (0.0009) [2023-12-26 22:21:02,569][105692] Updated weights for policy 0, policy_version 958657 (0.0009) [2023-12-26 22:21:02,720][105620] Updated weights for policy 1, policy_version 958672 (0.0009) [2023-12-26 22:21:02,787][105620] Updated weights for policy 1, policy_version 958682 (0.0006) [2023-12-26 22:21:02,845][105620] Updated weights for policy 1, policy_version 958692 (0.0006) [2023-12-26 22:21:03,278][105692] Updated weights for policy 0, policy_version 958667 (0.0009) [2023-12-26 22:21:03,325][105692] Updated weights for policy 0, policy_version 958677 (0.0006) [2023-12-26 22:21:03,375][105692] Updated weights for policy 0, policy_version 958687 (0.0009) [2023-12-26 22:21:03,426][105620] Updated weights for policy 1, policy_version 958702 (0.0006) [2023-12-26 22:21:03,473][105620] Updated weights for policy 1, policy_version 958712 (0.0008) [2023-12-26 22:21:03,521][105620] Updated weights for policy 1, policy_version 958722 (0.0008) [2023-12-26 22:21:04,116][105692] Updated weights for policy 0, policy_version 958697 (0.0010) [2023-12-26 22:21:04,179][105692] Updated weights for policy 0, policy_version 958707 (0.0011) [2023-12-26 22:21:04,206][105620] Updated weights for policy 1, policy_version 958732 (0.0009) [2023-12-26 22:21:04,240][105692] Updated weights for policy 0, policy_version 958717 (0.0011) [2023-12-26 22:21:04,273][105620] Updated weights for policy 1, policy_version 958742 (0.0009) [2023-12-26 22:21:04,301][105692] Updated weights for policy 0, policy_version 958727 (0.0011) [2023-12-26 22:21:04,344][105620] Updated weights for policy 1, policy_version 958752 (0.0009) [2023-12-26 22:21:05,023][105620] Updated weights for policy 1, policy_version 958762 (0.0008) [2023-12-26 22:21:05,038][105692] Updated weights for policy 0, policy_version 958737 (0.0010) [2023-12-26 22:21:05,089][105692] Updated weights for policy 0, policy_version 958747 (0.0005) [2023-12-26 22:21:05,090][105620] Updated weights for policy 1, policy_version 958772 (0.0005) [2023-12-26 22:21:05,137][105692] Updated weights for policy 0, policy_version 958757 (0.0008) [2023-12-26 22:21:05,156][105620] Updated weights for policy 1, policy_version 958782 (0.0006) [2023-12-26 22:21:05,215][105620] Updated weights for policy 1, policy_version 958792 (0.0008) [2023-12-26 22:21:05,817][105692] Updated weights for policy 0, policy_version 958767 (0.0008) [2023-12-26 22:21:05,868][105620] Updated weights for policy 1, policy_version 958802 (0.0006) [2023-12-26 22:21:05,870][105692] Updated weights for policy 0, policy_version 958777 (0.0007) [2023-12-26 22:21:05,930][105692] Updated weights for policy 0, policy_version 958787 (0.0005) [2023-12-26 22:21:05,930][105620] Updated weights for policy 1, policy_version 958812 (0.0008) [2023-12-26 22:21:05,980][105620] Updated weights for policy 1, policy_version 958822 (0.0009) [2023-12-26 22:21:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19244.3). Total num frames: 490979328. Throughput: 0: 9815.7, 1: 9485.0. Samples: 490961204. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:21:06,062][104569] Avg episode reward: [(0, '8987.478'), (1, '8027.346')] [2023-12-26 22:21:06,603][105692] Updated weights for policy 0, policy_version 958797 (0.0007) [2023-12-26 22:21:06,663][105692] Updated weights for policy 0, policy_version 958807 (0.0008) [2023-12-26 22:21:06,722][105692] Updated weights for policy 0, policy_version 958817 (0.0007) [2023-12-26 22:21:06,740][105620] Updated weights for policy 1, policy_version 958832 (0.0009) [2023-12-26 22:21:06,799][105620] Updated weights for policy 1, policy_version 958842 (0.0010) [2023-12-26 22:21:06,858][105620] Updated weights for policy 1, policy_version 958852 (0.0009) [2023-12-26 22:21:07,483][105692] Updated weights for policy 0, policy_version 958827 (0.0010) [2023-12-26 22:21:07,507][105620] Updated weights for policy 1, policy_version 958862 (0.0007) [2023-12-26 22:21:07,540][105692] Updated weights for policy 0, policy_version 958837 (0.0009) [2023-12-26 22:21:07,566][105620] Updated weights for policy 1, policy_version 958872 (0.0006) [2023-12-26 22:21:07,599][105692] Updated weights for policy 0, policy_version 958847 (0.0008) [2023-12-26 22:21:07,632][105620] Updated weights for policy 1, policy_version 958882 (0.0010) [2023-12-26 22:21:08,286][105620] Updated weights for policy 1, policy_version 958892 (0.0010) [2023-12-26 22:21:08,319][105692] Updated weights for policy 0, policy_version 958857 (0.0006) [2023-12-26 22:21:08,349][105620] Updated weights for policy 1, policy_version 958902 (0.0010) [2023-12-26 22:21:08,380][105692] Updated weights for policy 0, policy_version 958867 (0.0007) [2023-12-26 22:21:08,407][105620] Updated weights for policy 1, policy_version 958912 (0.0010) [2023-12-26 22:21:08,441][105692] Updated weights for policy 0, policy_version 958877 (0.0007) [2023-12-26 22:21:08,498][105692] Updated weights for policy 0, policy_version 958887 (0.0008) [2023-12-26 22:21:09,093][105620] Updated weights for policy 1, policy_version 958922 (0.0011) [2023-12-26 22:21:09,155][105620] Updated weights for policy 1, policy_version 958932 (0.0010) [2023-12-26 22:21:09,215][105620] Updated weights for policy 1, policy_version 958942 (0.0011) [2023-12-26 22:21:09,282][105620] Updated weights for policy 1, policy_version 958952 (0.0008) [2023-12-26 22:21:09,308][105692] Updated weights for policy 0, policy_version 958897 (0.0008) [2023-12-26 22:21:09,376][105692] Updated weights for policy 0, policy_version 958907 (0.0009) [2023-12-26 22:21:09,443][105692] Updated weights for policy 0, policy_version 958917 (0.0009) [2023-12-26 22:21:10,057][105620] Updated weights for policy 1, policy_version 958962 (0.0009) [2023-12-26 22:21:10,123][105620] Updated weights for policy 1, policy_version 958972 (0.0008) [2023-12-26 22:21:10,186][105620] Updated weights for policy 1, policy_version 958982 (0.0008) [2023-12-26 22:21:10,216][105692] Updated weights for policy 0, policy_version 958927 (0.0009) [2023-12-26 22:21:10,282][105692] Updated weights for policy 0, policy_version 958937 (0.0009) [2023-12-26 22:21:10,344][105692] Updated weights for policy 0, policy_version 958947 (0.0009) [2023-12-26 22:21:10,919][105620] Updated weights for policy 1, policy_version 958992 (0.0009) [2023-12-26 22:21:10,980][105620] Updated weights for policy 1, policy_version 959002 (0.0008) [2023-12-26 22:21:11,035][105692] Updated weights for policy 0, policy_version 958957 (0.0009) [2023-12-26 22:21:11,043][105620] Updated weights for policy 1, policy_version 959012 (0.0008) [2023-12-26 22:21:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19216.5). Total num frames: 491061248. Throughput: 0: 9718.9, 1: 9547.3. Samples: 491076004. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:21:11,062][104569] Avg episode reward: [(0, '8897.678'), (1, '7678.952')] [2023-12-26 22:21:11,104][105692] Updated weights for policy 0, policy_version 958967 (0.0006) [2023-12-26 22:21:11,172][105692] Updated weights for policy 0, policy_version 958977 (0.0008) [2023-12-26 22:21:11,769][105620] Updated weights for policy 1, policy_version 959022 (0.0010) [2023-12-26 22:21:11,832][105620] Updated weights for policy 1, policy_version 959032 (0.0010) [2023-12-26 22:21:11,870][105692] Updated weights for policy 0, policy_version 958987 (0.0007) [2023-12-26 22:21:11,896][105620] Updated weights for policy 1, policy_version 959042 (0.0011) [2023-12-26 22:21:11,934][105692] Updated weights for policy 0, policy_version 958997 (0.0006) [2023-12-26 22:21:11,985][105692] Updated weights for policy 0, policy_version 959007 (0.0008) [2023-12-26 22:21:12,666][105620] Updated weights for policy 1, policy_version 959052 (0.0010) [2023-12-26 22:21:12,725][105620] Updated weights for policy 1, policy_version 959062 (0.0011) [2023-12-26 22:21:12,776][105692] Updated weights for policy 0, policy_version 959017 (0.0008) [2023-12-26 22:21:12,783][105620] Updated weights for policy 1, policy_version 959072 (0.0010) [2023-12-26 22:21:12,833][105692] Updated weights for policy 0, policy_version 959027 (0.0010) [2023-12-26 22:21:12,894][105692] Updated weights for policy 0, policy_version 959037 (0.0009) [2023-12-26 22:21:12,950][105692] Updated weights for policy 0, policy_version 959047 (0.0008) [2023-12-26 22:21:13,447][105620] Updated weights for policy 1, policy_version 959082 (0.0006) [2023-12-26 22:21:13,501][105620] Updated weights for policy 1, policy_version 959092 (0.0010) [2023-12-26 22:21:13,560][105620] Updated weights for policy 1, policy_version 959102 (0.0010) [2023-12-26 22:21:13,609][105620] Updated weights for policy 1, policy_version 959112 (0.0007) [2023-12-26 22:21:13,733][105692] Updated weights for policy 0, policy_version 959057 (0.0008) [2023-12-26 22:21:13,793][105692] Updated weights for policy 0, policy_version 959067 (0.0008) [2023-12-26 22:21:13,851][105692] Updated weights for policy 0, policy_version 959077 (0.0010) [2023-12-26 22:21:14,306][105620] Updated weights for policy 1, policy_version 959122 (0.0009) [2023-12-26 22:21:14,357][105620] Updated weights for policy 1, policy_version 959132 (0.0010) [2023-12-26 22:21:14,411][105620] Updated weights for policy 1, policy_version 959142 (0.0010) [2023-12-26 22:21:14,609][105692] Updated weights for policy 0, policy_version 959087 (0.0009) [2023-12-26 22:21:14,665][105692] Updated weights for policy 0, policy_version 959097 (0.0008) [2023-12-26 22:21:14,720][105692] Updated weights for policy 0, policy_version 959107 (0.0008) [2023-12-26 22:21:15,123][105620] Updated weights for policy 1, policy_version 959152 (0.0007) [2023-12-26 22:21:15,180][105620] Updated weights for policy 1, policy_version 959162 (0.0006) [2023-12-26 22:21:15,238][105620] Updated weights for policy 1, policy_version 959172 (0.0009) [2023-12-26 22:21:15,440][105692] Updated weights for policy 0, policy_version 959117 (0.0008) [2023-12-26 22:21:15,497][105692] Updated weights for policy 0, policy_version 959127 (0.0008) [2023-12-26 22:21:15,546][105692] Updated weights for policy 0, policy_version 959137 (0.0008) [2023-12-26 22:21:15,957][105620] Updated weights for policy 1, policy_version 959182 (0.0011) [2023-12-26 22:21:16,021][105620] Updated weights for policy 1, policy_version 959192 (0.0011) [2023-12-26 22:21:16,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19251.2, 300 sec: 19216.5). Total num frames: 491159552. Throughput: 0: 9693.4, 1: 9529.4. Samples: 491133452. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:21:16,062][104569] Avg episode reward: [(0, '8809.254'), (1, '7933.312')] [2023-12-26 22:21:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000959144_245579776.pth... [2023-12-26 22:21:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000958024_245293056.pth [2023-12-26 22:21:16,084][105620] Updated weights for policy 1, policy_version 959202 (0.0010) [2023-12-26 22:21:16,124][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000959208_245587968.pth... [2023-12-26 22:21:16,128][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000958056_245293056.pth [2023-12-26 22:21:16,241][105692] Updated weights for policy 0, policy_version 959147 (0.0008) [2023-12-26 22:21:16,290][105692] Updated weights for policy 0, policy_version 959157 (0.0008) [2023-12-26 22:21:16,336][105692] Updated weights for policy 0, policy_version 959167 (0.0008) [2023-12-26 22:21:16,755][105620] Updated weights for policy 1, policy_version 959212 (0.0010) [2023-12-26 22:21:16,825][105620] Updated weights for policy 1, policy_version 959222 (0.0011) [2023-12-26 22:21:16,894][105620] Updated weights for policy 1, policy_version 959232 (0.0010) [2023-12-26 22:21:17,122][105692] Updated weights for policy 0, policy_version 959177 (0.0008) [2023-12-26 22:21:17,180][105692] Updated weights for policy 0, policy_version 959187 (0.0010) [2023-12-26 22:21:17,241][105692] Updated weights for policy 0, policy_version 959197 (0.0008) [2023-12-26 22:21:17,301][105692] Updated weights for policy 0, policy_version 959207 (0.0008) [2023-12-26 22:21:17,554][105620] Updated weights for policy 1, policy_version 959242 (0.0009) [2023-12-26 22:21:17,607][105620] Updated weights for policy 1, policy_version 959252 (0.0007) [2023-12-26 22:21:17,670][105620] Updated weights for policy 1, policy_version 959262 (0.0007) [2023-12-26 22:21:17,734][105620] Updated weights for policy 1, policy_version 959272 (0.0009) [2023-12-26 22:21:18,033][105692] Updated weights for policy 0, policy_version 959217 (0.0007) [2023-12-26 22:21:18,088][105692] Updated weights for policy 0, policy_version 959227 (0.0006) [2023-12-26 22:21:18,153][105692] Updated weights for policy 0, policy_version 959237 (0.0006) [2023-12-26 22:21:18,425][105620] Updated weights for policy 1, policy_version 959282 (0.0006) [2023-12-26 22:21:18,497][105620] Updated weights for policy 1, policy_version 959292 (0.0006) [2023-12-26 22:21:18,566][105620] Updated weights for policy 1, policy_version 959302 (0.0005) [2023-12-26 22:21:18,807][105692] Updated weights for policy 0, policy_version 959247 (0.0006) [2023-12-26 22:21:18,873][105692] Updated weights for policy 0, policy_version 959257 (0.0007) [2023-12-26 22:21:18,925][105692] Updated weights for policy 0, policy_version 959267 (0.0010) [2023-12-26 22:21:19,258][105620] Updated weights for policy 1, policy_version 959312 (0.0008) [2023-12-26 22:21:19,318][105620] Updated weights for policy 1, policy_version 959322 (0.0008) [2023-12-26 22:21:19,385][105620] Updated weights for policy 1, policy_version 959332 (0.0008) [2023-12-26 22:21:19,626][105692] Updated weights for policy 0, policy_version 959277 (0.0008) [2023-12-26 22:21:19,692][105692] Updated weights for policy 0, policy_version 959287 (0.0007) [2023-12-26 22:21:19,752][105692] Updated weights for policy 0, policy_version 959297 (0.0007) [2023-12-26 22:21:20,168][105620] Updated weights for policy 1, policy_version 959342 (0.0008) [2023-12-26 22:21:20,223][105620] Updated weights for policy 1, policy_version 959352 (0.0009) [2023-12-26 22:21:20,272][105620] Updated weights for policy 1, policy_version 959362 (0.0009) [2023-12-26 22:21:20,459][105692] Updated weights for policy 0, policy_version 959307 (0.0008) [2023-12-26 22:21:20,521][105692] Updated weights for policy 0, policy_version 959317 (0.0005) [2023-12-26 22:21:20,588][105692] Updated weights for policy 0, policy_version 959327 (0.0007) [2023-12-26 22:21:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.8, 300 sec: 19216.5). Total num frames: 491257856. Throughput: 0: 9692.7, 1: 9580.6. Samples: 491250696. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:21:21,063][104569] Avg episode reward: [(0, '8990.950'), (1, '7843.682')] [2023-12-26 22:21:21,132][105620] Updated weights for policy 1, policy_version 959372 (0.0009) [2023-12-26 22:21:21,195][105620] Updated weights for policy 1, policy_version 959382 (0.0009) [2023-12-26 22:21:21,206][105692] Updated weights for policy 0, policy_version 959337 (0.0006) [2023-12-26 22:21:21,263][105620] Updated weights for policy 1, policy_version 959392 (0.0007) [2023-12-26 22:21:21,271][105692] Updated weights for policy 0, policy_version 959347 (0.0008) [2023-12-26 22:21:21,331][105692] Updated weights for policy 0, policy_version 959357 (0.0008) [2023-12-26 22:21:21,399][105692] Updated weights for policy 0, policy_version 959367 (0.0009) [2023-12-26 22:21:22,000][105620] Updated weights for policy 1, policy_version 959402 (0.0007) [2023-12-26 22:21:22,064][105620] Updated weights for policy 1, policy_version 959412 (0.0009) [2023-12-26 22:21:22,095][105692] Updated weights for policy 0, policy_version 959377 (0.0008) [2023-12-26 22:21:22,128][105620] Updated weights for policy 1, policy_version 959422 (0.0008) [2023-12-26 22:21:22,156][105692] Updated weights for policy 0, policy_version 959387 (0.0006) [2023-12-26 22:21:22,185][105620] Updated weights for policy 1, policy_version 959432 (0.0008) [2023-12-26 22:21:22,209][105692] Updated weights for policy 0, policy_version 959397 (0.0005) [2023-12-26 22:21:22,880][105692] Updated weights for policy 0, policy_version 959407 (0.0009) [2023-12-26 22:21:22,928][105620] Updated weights for policy 1, policy_version 959442 (0.0008) [2023-12-26 22:21:22,933][105692] Updated weights for policy 0, policy_version 959417 (0.0010) [2023-12-26 22:21:22,987][105692] Updated weights for policy 0, policy_version 959427 (0.0011) [2023-12-26 22:21:22,994][105620] Updated weights for policy 1, policy_version 959452 (0.0008) [2023-12-26 22:21:23,058][105620] Updated weights for policy 1, policy_version 959462 (0.0008) [2023-12-26 22:21:23,661][105620] Updated weights for policy 1, policy_version 959472 (0.0006) [2023-12-26 22:21:23,720][105620] Updated weights for policy 1, policy_version 959482 (0.0005) [2023-12-26 22:21:23,764][105692] Updated weights for policy 0, policy_version 959437 (0.0010) [2023-12-26 22:21:23,769][105620] Updated weights for policy 1, policy_version 959492 (0.0005) [2023-12-26 22:21:23,826][105692] Updated weights for policy 0, policy_version 959447 (0.0010) [2023-12-26 22:21:23,885][105692] Updated weights for policy 0, policy_version 959457 (0.0005) [2023-12-26 22:21:24,496][105620] Updated weights for policy 1, policy_version 959502 (0.0005) [2023-12-26 22:21:24,517][105692] Updated weights for policy 0, policy_version 959467 (0.0008) [2023-12-26 22:21:24,546][105620] Updated weights for policy 1, policy_version 959512 (0.0006) [2023-12-26 22:21:24,574][105692] Updated weights for policy 0, policy_version 959477 (0.0008) [2023-12-26 22:21:24,595][105620] Updated weights for policy 1, policy_version 959522 (0.0005) [2023-12-26 22:21:24,633][105692] Updated weights for policy 0, policy_version 959487 (0.0007) [2023-12-26 22:21:25,155][105620] Updated weights for policy 1, policy_version 959532 (0.0005) [2023-12-26 22:21:25,203][105620] Updated weights for policy 1, policy_version 959542 (0.0005) [2023-12-26 22:21:25,255][105620] Updated weights for policy 1, policy_version 959552 (0.0005) [2023-12-26 22:21:25,303][105692] Updated weights for policy 0, policy_version 959497 (0.0009) [2023-12-26 22:21:25,354][105692] Updated weights for policy 0, policy_version 959507 (0.0010) [2023-12-26 22:21:25,412][105692] Updated weights for policy 0, policy_version 959517 (0.0009) [2023-12-26 22:21:25,460][105692] Updated weights for policy 0, policy_version 959527 (0.0010) [2023-12-26 22:21:25,894][105620] Updated weights for policy 1, policy_version 959562 (0.0007) [2023-12-26 22:21:25,959][105620] Updated weights for policy 1, policy_version 959572 (0.0009) [2023-12-26 22:21:26,029][105620] Updated weights for policy 1, policy_version 959582 (0.0008) [2023-12-26 22:21:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19251.2, 300 sec: 19244.3). Total num frames: 491356160. Throughput: 0: 9714.7, 1: 9640.1. Samples: 491370700. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:21:26,063][104569] Avg episode reward: [(0, '8811.969'), (1, '7790.221')] [2023-12-26 22:21:26,095][105620] Updated weights for policy 1, policy_version 959592 (0.0007) [2023-12-26 22:21:26,225][105692] Updated weights for policy 0, policy_version 959537 (0.0010) [2023-12-26 22:21:26,274][105692] Updated weights for policy 0, policy_version 959547 (0.0010) [2023-12-26 22:21:26,334][105692] Updated weights for policy 0, policy_version 959557 (0.0011) [2023-12-26 22:21:26,685][105620] Updated weights for policy 1, policy_version 959602 (0.0010) [2023-12-26 22:21:26,733][105620] Updated weights for policy 1, policy_version 959612 (0.0010) [2023-12-26 22:21:26,778][105620] Updated weights for policy 1, policy_version 959622 (0.0010) [2023-12-26 22:21:27,054][105692] Updated weights for policy 0, policy_version 959567 (0.0010) [2023-12-26 22:21:27,106][105692] Updated weights for policy 0, policy_version 959577 (0.0010) [2023-12-26 22:21:27,154][105692] Updated weights for policy 0, policy_version 959587 (0.0010) [2023-12-26 22:21:27,351][105620] Updated weights for policy 1, policy_version 959632 (0.0006) [2023-12-26 22:21:27,413][105620] Updated weights for policy 1, policy_version 959642 (0.0005) [2023-12-26 22:21:27,471][105620] Updated weights for policy 1, policy_version 959652 (0.0005) [2023-12-26 22:21:27,925][105692] Updated weights for policy 0, policy_version 959597 (0.0010) [2023-12-26 22:21:27,990][105692] Updated weights for policy 0, policy_version 959607 (0.0007) [2023-12-26 22:21:28,003][105620] Updated weights for policy 1, policy_version 959662 (0.0008) [2023-12-26 22:21:28,049][105692] Updated weights for policy 0, policy_version 959617 (0.0005) [2023-12-26 22:21:28,065][105620] Updated weights for policy 1, policy_version 959672 (0.0010) [2023-12-26 22:21:28,127][105620] Updated weights for policy 1, policy_version 959682 (0.0010) [2023-12-26 22:21:28,749][105692] Updated weights for policy 0, policy_version 959627 (0.0009) [2023-12-26 22:21:28,797][105692] Updated weights for policy 0, policy_version 959637 (0.0010) [2023-12-26 22:21:28,841][105692] Updated weights for policy 0, policy_version 959647 (0.0010) [2023-12-26 22:21:28,873][105620] Updated weights for policy 1, policy_version 959692 (0.0010) [2023-12-26 22:21:28,932][105620] Updated weights for policy 1, policy_version 959702 (0.0010) [2023-12-26 22:21:28,984][105620] Updated weights for policy 1, policy_version 959712 (0.0010) [2023-12-26 22:21:29,620][105692] Updated weights for policy 0, policy_version 959657 (0.0010) [2023-12-26 22:21:29,677][105692] Updated weights for policy 0, policy_version 959667 (0.0006) [2023-12-26 22:21:29,717][105620] Updated weights for policy 1, policy_version 959722 (0.0010) [2023-12-26 22:21:29,739][105692] Updated weights for policy 0, policy_version 959677 (0.0006) [2023-12-26 22:21:29,775][105620] Updated weights for policy 1, policy_version 959732 (0.0005) [2023-12-26 22:21:29,800][105692] Updated weights for policy 0, policy_version 959687 (0.0006) [2023-12-26 22:21:29,837][105620] Updated weights for policy 1, policy_version 959742 (0.0009) [2023-12-26 22:21:29,901][105620] Updated weights for policy 1, policy_version 959752 (0.0009) [2023-12-26 22:21:30,387][105692] Updated weights for policy 0, policy_version 959697 (0.0009) [2023-12-26 22:21:30,449][105692] Updated weights for policy 0, policy_version 959707 (0.0006) [2023-12-26 22:21:30,501][105692] Updated weights for policy 0, policy_version 959717 (0.0005) [2023-12-26 22:21:30,703][105620] Updated weights for policy 1, policy_version 959762 (0.0007) [2023-12-26 22:21:30,758][105620] Updated weights for policy 1, policy_version 959772 (0.0007) [2023-12-26 22:21:30,806][105620] Updated weights for policy 1, policy_version 959782 (0.0009) [2023-12-26 22:21:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.8, 300 sec: 19244.3). Total num frames: 491462656. Throughput: 0: 9721.7, 1: 9786.4. Samples: 491432668. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:21:31,062][104569] Avg episode reward: [(0, '8724.124'), (1, '8149.184')] [2023-12-26 22:21:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000959720_245727232.pth... [2023-12-26 22:21:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000959784_245735424.pth... [2023-12-26 22:21:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000958600_245440512.pth [2023-12-26 22:21:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000958600_245432320.pth [2023-12-26 22:21:31,268][105692] Updated weights for policy 0, policy_version 959727 (0.0010) [2023-12-26 22:21:31,332][105692] Updated weights for policy 0, policy_version 959737 (0.0009) [2023-12-26 22:21:31,409][105692] Updated weights for policy 0, policy_version 959747 (0.0009) [2023-12-26 22:21:31,506][105620] Updated weights for policy 1, policy_version 959792 (0.0006) [2023-12-26 22:21:31,568][105620] Updated weights for policy 1, policy_version 959802 (0.0005) [2023-12-26 22:21:31,637][105620] Updated weights for policy 1, policy_version 959812 (0.0007) [2023-12-26 22:21:32,196][105692] Updated weights for policy 0, policy_version 959757 (0.0008) [2023-12-26 22:21:32,257][105692] Updated weights for policy 0, policy_version 959767 (0.0010) [2023-12-26 22:21:32,258][105620] Updated weights for policy 1, policy_version 959822 (0.0008) [2023-12-26 22:21:32,318][105692] Updated weights for policy 0, policy_version 959777 (0.0011) [2023-12-26 22:21:32,319][105620] Updated weights for policy 1, policy_version 959832 (0.0006) [2023-12-26 22:21:32,381][105620] Updated weights for policy 1, policy_version 959842 (0.0010) [2023-12-26 22:21:33,067][105692] Updated weights for policy 0, policy_version 959787 (0.0011) [2023-12-26 22:21:33,088][105620] Updated weights for policy 1, policy_version 959852 (0.0009) [2023-12-26 22:21:33,125][105692] Updated weights for policy 0, policy_version 959797 (0.0010) [2023-12-26 22:21:33,132][105620] Updated weights for policy 1, policy_version 959862 (0.0006) [2023-12-26 22:21:33,177][105620] Updated weights for policy 1, policy_version 959872 (0.0007) [2023-12-26 22:21:33,182][105692] Updated weights for policy 0, policy_version 959807 (0.0010) [2023-12-26 22:21:33,901][105620] Updated weights for policy 1, policy_version 959882 (0.0008) [2023-12-26 22:21:33,910][105692] Updated weights for policy 0, policy_version 959817 (0.0010) [2023-12-26 22:21:33,954][105692] Updated weights for policy 0, policy_version 959827 (0.0010) [2023-12-26 22:21:33,959][105620] Updated weights for policy 1, policy_version 959892 (0.0006) [2023-12-26 22:21:34,008][105692] Updated weights for policy 0, policy_version 959837 (0.0010) [2023-12-26 22:21:34,018][105620] Updated weights for policy 1, policy_version 959902 (0.0005) [2023-12-26 22:21:34,069][105692] Updated weights for policy 0, policy_version 959847 (0.0010) [2023-12-26 22:21:34,075][105620] Updated weights for policy 1, policy_version 959912 (0.0005) [2023-12-26 22:21:34,698][105692] Updated weights for policy 0, policy_version 959857 (0.0011) [2023-12-26 22:21:34,765][105692] Updated weights for policy 0, policy_version 959867 (0.0010) [2023-12-26 22:21:34,824][105692] Updated weights for policy 0, policy_version 959877 (0.0010) [2023-12-26 22:21:34,865][105620] Updated weights for policy 1, policy_version 959922 (0.0008) [2023-12-26 22:21:34,924][105620] Updated weights for policy 1, policy_version 959932 (0.0008) [2023-12-26 22:21:34,977][105620] Updated weights for policy 1, policy_version 959942 (0.0008) [2023-12-26 22:21:35,444][105692] Updated weights for policy 0, policy_version 959887 (0.0007) [2023-12-26 22:21:35,497][105692] Updated weights for policy 0, policy_version 959897 (0.0008) [2023-12-26 22:21:35,550][105692] Updated weights for policy 0, policy_version 959907 (0.0008) [2023-12-26 22:21:35,601][105620] Updated weights for policy 1, policy_version 959952 (0.0006) [2023-12-26 22:21:35,670][105620] Updated weights for policy 1, policy_version 959962 (0.0006) [2023-12-26 22:21:35,729][105620] Updated weights for policy 1, policy_version 959972 (0.0006) [2023-12-26 22:21:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.7, 300 sec: 19272.0). Total num frames: 491560960. Throughput: 0: 9677.6, 1: 9773.6. Samples: 491548364. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:21:36,063][104569] Avg episode reward: [(0, '8546.082'), (1, '8226.634')] [2023-12-26 22:21:36,237][105692] Updated weights for policy 0, policy_version 959917 (0.0007) [2023-12-26 22:21:36,252][105620] Updated weights for policy 1, policy_version 959982 (0.0006) [2023-12-26 22:21:36,301][105692] Updated weights for policy 0, policy_version 959927 (0.0007) [2023-12-26 22:21:36,317][105620] Updated weights for policy 1, policy_version 959992 (0.0007) [2023-12-26 22:21:36,364][105692] Updated weights for policy 0, policy_version 959937 (0.0007) [2023-12-26 22:21:36,377][105620] Updated weights for policy 1, policy_version 960002 (0.0009) [2023-12-26 22:21:37,057][105620] Updated weights for policy 1, policy_version 960012 (0.0008) [2023-12-26 22:21:37,107][105692] Updated weights for policy 0, policy_version 959947 (0.0006) [2023-12-26 22:21:37,113][105620] Updated weights for policy 1, policy_version 960022 (0.0010) [2023-12-26 22:21:37,163][105692] Updated weights for policy 0, policy_version 959957 (0.0006) [2023-12-26 22:21:37,169][105620] Updated weights for policy 1, policy_version 960032 (0.0010) [2023-12-26 22:21:37,224][105692] Updated weights for policy 0, policy_version 959967 (0.0006) [2023-12-26 22:21:37,864][105620] Updated weights for policy 1, policy_version 960042 (0.0009) [2023-12-26 22:21:37,925][105620] Updated weights for policy 1, policy_version 960052 (0.0009) [2023-12-26 22:21:37,982][105620] Updated weights for policy 1, policy_version 960062 (0.0011) [2023-12-26 22:21:37,992][105692] Updated weights for policy 0, policy_version 959977 (0.0007) [2023-12-26 22:21:38,039][105620] Updated weights for policy 1, policy_version 960072 (0.0011) [2023-12-26 22:21:38,043][105692] Updated weights for policy 0, policy_version 959987 (0.0006) [2023-12-26 22:21:38,093][105692] Updated weights for policy 0, policy_version 959997 (0.0005) [2023-12-26 22:21:38,150][105692] Updated weights for policy 0, policy_version 960007 (0.0005) [2023-12-26 22:21:38,779][105620] Updated weights for policy 1, policy_version 960082 (0.0010) [2023-12-26 22:21:38,832][105620] Updated weights for policy 1, policy_version 960092 (0.0010) [2023-12-26 22:21:38,858][105692] Updated weights for policy 0, policy_version 960017 (0.0006) [2023-12-26 22:21:38,890][105620] Updated weights for policy 1, policy_version 960102 (0.0010) [2023-12-26 22:21:38,914][105692] Updated weights for policy 0, policy_version 960027 (0.0006) [2023-12-26 22:21:38,964][105692] Updated weights for policy 0, policy_version 960037 (0.0007) [2023-12-26 22:21:39,551][105620] Updated weights for policy 1, policy_version 960112 (0.0010) [2023-12-26 22:21:39,617][105620] Updated weights for policy 1, policy_version 960122 (0.0010) [2023-12-26 22:21:39,677][105620] Updated weights for policy 1, policy_version 960132 (0.0010) [2023-12-26 22:21:39,687][105692] Updated weights for policy 0, policy_version 960047 (0.0007) [2023-12-26 22:21:39,748][105692] Updated weights for policy 0, policy_version 960057 (0.0008) [2023-12-26 22:21:39,808][105692] Updated weights for policy 0, policy_version 960067 (0.0007) [2023-12-26 22:21:40,401][105620] Updated weights for policy 1, policy_version 960142 (0.0008) [2023-12-26 22:21:40,454][105620] Updated weights for policy 1, policy_version 960152 (0.0005) [2023-12-26 22:21:40,507][105620] Updated weights for policy 1, policy_version 960162 (0.0006) [2023-12-26 22:21:40,613][105692] Updated weights for policy 0, policy_version 960077 (0.0007) [2023-12-26 22:21:40,675][105692] Updated weights for policy 0, policy_version 960087 (0.0005) [2023-12-26 22:21:40,735][105692] Updated weights for policy 0, policy_version 960097 (0.0007) [2023-12-26 22:21:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19272.0). Total num frames: 491659264. Throughput: 0: 9642.4, 1: 9940.0. Samples: 491669200. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:21:41,062][104569] Avg episode reward: [(0, '8725.334'), (1, '8150.046')] [2023-12-26 22:21:41,089][105620] Updated weights for policy 1, policy_version 960172 (0.0006) [2023-12-26 22:21:41,153][105620] Updated weights for policy 1, policy_version 960182 (0.0010) [2023-12-26 22:21:41,211][105620] Updated weights for policy 1, policy_version 960192 (0.0009) [2023-12-26 22:21:41,589][105692] Updated weights for policy 0, policy_version 960107 (0.0010) [2023-12-26 22:21:41,651][105692] Updated weights for policy 0, policy_version 960117 (0.0010) [2023-12-26 22:21:41,717][105692] Updated weights for policy 0, policy_version 960127 (0.0008) [2023-12-26 22:21:42,022][105620] Updated weights for policy 1, policy_version 960202 (0.0008) [2023-12-26 22:21:42,088][105620] Updated weights for policy 1, policy_version 960212 (0.0009) [2023-12-26 22:21:42,150][105620] Updated weights for policy 1, policy_version 960222 (0.0009) [2023-12-26 22:21:42,216][105620] Updated weights for policy 1, policy_version 960232 (0.0009) [2023-12-26 22:21:42,503][105692] Updated weights for policy 0, policy_version 960137 (0.0008) [2023-12-26 22:21:42,562][105692] Updated weights for policy 0, policy_version 960147 (0.0008) [2023-12-26 22:21:42,618][105692] Updated weights for policy 0, policy_version 960157 (0.0009) [2023-12-26 22:21:42,681][105692] Updated weights for policy 0, policy_version 960167 (0.0009) [2023-12-26 22:21:42,990][105620] Updated weights for policy 1, policy_version 960242 (0.0008) [2023-12-26 22:21:43,050][105620] Updated weights for policy 1, policy_version 960252 (0.0008) [2023-12-26 22:21:43,112][105620] Updated weights for policy 1, policy_version 960262 (0.0008) [2023-12-26 22:21:43,417][105692] Updated weights for policy 0, policy_version 960177 (0.0010) [2023-12-26 22:21:43,469][105692] Updated weights for policy 0, policy_version 960187 (0.0010) [2023-12-26 22:21:43,516][105692] Updated weights for policy 0, policy_version 960197 (0.0008) [2023-12-26 22:21:43,875][105620] Updated weights for policy 1, policy_version 960272 (0.0009) [2023-12-26 22:21:43,930][105620] Updated weights for policy 1, policy_version 960282 (0.0009) [2023-12-26 22:21:43,983][105620] Updated weights for policy 1, policy_version 960292 (0.0010) [2023-12-26 22:21:44,122][105692] Updated weights for policy 0, policy_version 960207 (0.0007) [2023-12-26 22:21:44,180][105692] Updated weights for policy 0, policy_version 960217 (0.0008) [2023-12-26 22:21:44,234][105692] Updated weights for policy 0, policy_version 960227 (0.0009) [2023-12-26 22:21:44,734][105620] Updated weights for policy 1, policy_version 960303 (0.0008) [2023-12-26 22:21:44,795][105620] Updated weights for policy 1, policy_version 960313 (0.0007) [2023-12-26 22:21:44,862][105620] Updated weights for policy 1, policy_version 960323 (0.0008) [2023-12-26 22:21:44,936][105692] Updated weights for policy 0, policy_version 960237 (0.0008) [2023-12-26 22:21:44,984][105692] Updated weights for policy 0, policy_version 960247 (0.0008) [2023-12-26 22:21:45,043][105692] Updated weights for policy 0, policy_version 960257 (0.0008) [2023-12-26 22:21:45,608][105620] Updated weights for policy 1, policy_version 960333 (0.0008) [2023-12-26 22:21:45,666][105620] Updated weights for policy 1, policy_version 960343 (0.0009) [2023-12-26 22:21:45,718][105620] Updated weights for policy 1, policy_version 960353 (0.0009) [2023-12-26 22:21:45,811][105692] Updated weights for policy 0, policy_version 960267 (0.0009) [2023-12-26 22:21:45,870][105692] Updated weights for policy 0, policy_version 960277 (0.0009) [2023-12-26 22:21:45,923][105692] Updated weights for policy 0, policy_version 960287 (0.0008) [2023-12-26 22:21:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19299.8). Total num frames: 491757568. Throughput: 0: 9553.2, 1: 9916.6. Samples: 491723184. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-26 22:21:46,063][104569] Avg episode reward: [(0, '7346.550'), (1, '8573.474')] [2023-12-26 22:21:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000960296_245874688.pth... [2023-12-26 22:21:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000960360_245882880.pth... [2023-12-26 22:21:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000959144_245579776.pth [2023-12-26 22:21:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000959208_245587968.pth [2023-12-26 22:21:46,073][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_000960296_245874688.pth [2023-12-26 22:21:46,074][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_000960360_245882880.pth [2023-12-26 22:21:46,460][105620] Updated weights for policy 1, policy_version 960363 (0.0008) [2023-12-26 22:21:46,518][105620] Updated weights for policy 1, policy_version 960373 (0.0005) [2023-12-26 22:21:46,566][105620] Updated weights for policy 1, policy_version 960383 (0.0005) [2023-12-26 22:21:46,669][105692] Updated weights for policy 0, policy_version 960297 (0.0006) [2023-12-26 22:21:46,725][105692] Updated weights for policy 0, policy_version 960307 (0.0011) [2023-12-26 22:21:46,752][105585] KL-divergence is very high: 157.3814 [2023-12-26 22:21:46,777][105692] Updated weights for policy 0, policy_version 960317 (0.0010) [2023-12-26 22:21:46,798][105585] KL-divergence is very high: 174.4210 [2023-12-26 22:21:46,835][105692] Updated weights for policy 0, policy_version 960327 (0.0010) [2023-12-26 22:21:47,259][105620] Updated weights for policy 1, policy_version 960393 (0.0007) [2023-12-26 22:21:47,323][105620] Updated weights for policy 1, policy_version 960403 (0.0005) [2023-12-26 22:21:47,388][105620] Updated weights for policy 1, policy_version 960413 (0.0008) [2023-12-26 22:21:47,445][105620] Updated weights for policy 1, policy_version 960423 (0.0008) [2023-12-26 22:21:47,560][105692] Updated weights for policy 0, policy_version 960337 (0.0006) [2023-12-26 22:21:47,627][105692] Updated weights for policy 0, policy_version 960347 (0.0009) [2023-12-26 22:21:47,679][105692] Updated weights for policy 0, policy_version 960357 (0.0010) [2023-12-26 22:21:48,168][105620] Updated weights for policy 1, policy_version 960433 (0.0006) [2023-12-26 22:21:48,233][105620] Updated weights for policy 1, policy_version 960443 (0.0008) [2023-12-26 22:21:48,296][105620] Updated weights for policy 1, policy_version 960453 (0.0008) [2023-12-26 22:21:48,395][105692] Updated weights for policy 0, policy_version 960367 (0.0011) [2023-12-26 22:21:48,448][105692] Updated weights for policy 0, policy_version 960377 (0.0011) [2023-12-26 22:21:48,494][105692] Updated weights for policy 0, policy_version 960387 (0.0010) [2023-12-26 22:21:49,041][105620] Updated weights for policy 1, policy_version 960463 (0.0009) [2023-12-26 22:21:49,104][105620] Updated weights for policy 1, policy_version 960473 (0.0009) [2023-12-26 22:21:49,168][105620] Updated weights for policy 1, policy_version 960483 (0.0008) [2023-12-26 22:21:49,204][105692] Updated weights for policy 0, policy_version 960397 (0.0010) [2023-12-26 22:21:49,272][105692] Updated weights for policy 0, policy_version 960407 (0.0009) [2023-12-26 22:21:49,343][105692] Updated weights for policy 0, policy_version 960417 (0.0010) [2023-12-26 22:21:49,858][105620] Updated weights for policy 1, policy_version 960493 (0.0008) [2023-12-26 22:21:49,926][105620] Updated weights for policy 1, policy_version 960503 (0.0008) [2023-12-26 22:21:49,987][105620] Updated weights for policy 1, policy_version 960513 (0.0008) [2023-12-26 22:21:50,057][105692] Updated weights for policy 0, policy_version 960427 (0.0009) [2023-12-26 22:21:50,117][105692] Updated weights for policy 0, policy_version 960437 (0.0011) [2023-12-26 22:21:50,165][105692] Updated weights for policy 0, policy_version 960447 (0.0010) [2023-12-26 22:21:50,731][105620] Updated weights for policy 1, policy_version 960523 (0.0007) [2023-12-26 22:21:50,780][105620] Updated weights for policy 1, policy_version 960533 (0.0008) [2023-12-26 22:21:50,833][105620] Updated weights for policy 1, policy_version 960543 (0.0008) [2023-12-26 22:21:50,954][105692] Updated weights for policy 0, policy_version 960457 (0.0011) [2023-12-26 22:21:51,017][105692] Updated weights for policy 0, policy_version 960467 (0.0011) [2023-12-26 22:21:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19272.0). Total num frames: 491847680. Throughput: 0: 9662.0, 1: 9856.9. Samples: 491839556. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:21:51,062][104569] Avg episode reward: [(0, '6893.471'), (1, '8468.798')] [2023-12-26 22:21:51,084][105692] Updated weights for policy 0, policy_version 960477 (0.0011) [2023-12-26 22:21:51,147][105692] Updated weights for policy 0, policy_version 960487 (0.0010) [2023-12-26 22:21:51,653][105620] Updated weights for policy 1, policy_version 960553 (0.0008) [2023-12-26 22:21:51,718][105620] Updated weights for policy 1, policy_version 960563 (0.0008) [2023-12-26 22:21:51,777][105620] Updated weights for policy 1, policy_version 960573 (0.0009) [2023-12-26 22:21:51,798][105692] Updated weights for policy 0, policy_version 960497 (0.0006) [2023-12-26 22:21:51,838][105620] Updated weights for policy 1, policy_version 960583 (0.0008) [2023-12-26 22:21:51,854][105692] Updated weights for policy 0, policy_version 960507 (0.0006) [2023-12-26 22:21:51,901][105692] Updated weights for policy 0, policy_version 960517 (0.0005) [2023-12-26 22:21:52,588][105692] Updated weights for policy 0, policy_version 960527 (0.0009) [2023-12-26 22:21:52,640][105692] Updated weights for policy 0, policy_version 960537 (0.0010) [2023-12-26 22:21:52,671][105620] Updated weights for policy 1, policy_version 960593 (0.0007) [2023-12-26 22:21:52,692][105692] Updated weights for policy 0, policy_version 960547 (0.0010) [2023-12-26 22:21:52,731][105620] Updated weights for policy 1, policy_version 960603 (0.0007) [2023-12-26 22:21:52,793][105620] Updated weights for policy 1, policy_version 960613 (0.0007) [2023-12-26 22:21:53,461][105692] Updated weights for policy 0, policy_version 960557 (0.0010) [2023-12-26 22:21:53,516][105692] Updated weights for policy 0, policy_version 960567 (0.0010) [2023-12-26 22:21:53,542][105620] Updated weights for policy 1, policy_version 960623 (0.0009) [2023-12-26 22:21:53,564][105692] Updated weights for policy 0, policy_version 960577 (0.0010) [2023-12-26 22:21:53,605][105620] Updated weights for policy 1, policy_version 960633 (0.0006) [2023-12-26 22:21:53,663][105620] Updated weights for policy 1, policy_version 960643 (0.0008) [2023-12-26 22:21:54,299][105620] Updated weights for policy 1, policy_version 960653 (0.0009) [2023-12-26 22:21:54,315][105692] Updated weights for policy 0, policy_version 960587 (0.0009) [2023-12-26 22:21:54,359][105620] Updated weights for policy 1, policy_version 960663 (0.0011) [2023-12-26 22:21:54,379][105692] Updated weights for policy 0, policy_version 960597 (0.0005) [2023-12-26 22:21:54,411][105620] Updated weights for policy 1, policy_version 960673 (0.0010) [2023-12-26 22:21:54,435][105692] Updated weights for policy 0, policy_version 960607 (0.0006) [2023-12-26 22:21:55,039][105692] Updated weights for policy 0, policy_version 960617 (0.0006) [2023-12-26 22:21:55,091][105692] Updated weights for policy 0, policy_version 960627 (0.0008) [2023-12-26 22:21:55,101][105620] Updated weights for policy 1, policy_version 960683 (0.0011) [2023-12-26 22:21:55,144][105692] Updated weights for policy 0, policy_version 960637 (0.0007) [2023-12-26 22:21:55,158][105620] Updated weights for policy 1, policy_version 960693 (0.0010) [2023-12-26 22:21:55,193][105692] Updated weights for policy 0, policy_version 960647 (0.0006) [2023-12-26 22:21:55,214][105620] Updated weights for policy 1, policy_version 960703 (0.0010) [2023-12-26 22:21:55,882][105692] Updated weights for policy 0, policy_version 960657 (0.0006) [2023-12-26 22:21:55,927][105620] Updated weights for policy 1, policy_version 960713 (0.0010) [2023-12-26 22:21:55,949][105692] Updated weights for policy 0, policy_version 960667 (0.0005) [2023-12-26 22:21:55,983][105620] Updated weights for policy 1, policy_version 960723 (0.0010) [2023-12-26 22:21:56,008][105692] Updated weights for policy 0, policy_version 960677 (0.0006) [2023-12-26 22:21:56,039][105620] Updated weights for policy 1, policy_version 960733 (0.0011) [2023-12-26 22:21:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19272.0). Total num frames: 491945984. Throughput: 0: 9728.2, 1: 9812.2. Samples: 491955328. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:21:56,063][104569] Avg episode reward: [(0, '7386.821'), (1, '7933.561')] [2023-12-26 22:21:56,094][105620] Updated weights for policy 1, policy_version 960743 (0.0010) [2023-12-26 22:21:56,624][105692] Updated weights for policy 0, policy_version 960687 (0.0008) [2023-12-26 22:21:56,678][105692] Updated weights for policy 0, policy_version 960697 (0.0005) [2023-12-26 22:21:56,745][105692] Updated weights for policy 0, policy_version 960707 (0.0006) [2023-12-26 22:21:56,822][105620] Updated weights for policy 1, policy_version 960753 (0.0008) [2023-12-26 22:21:56,873][105620] Updated weights for policy 1, policy_version 960763 (0.0010) [2023-12-26 22:21:56,921][105620] Updated weights for policy 1, policy_version 960773 (0.0010) [2023-12-26 22:21:57,427][105692] Updated weights for policy 0, policy_version 960717 (0.0006) [2023-12-26 22:21:57,482][105692] Updated weights for policy 0, policy_version 960727 (0.0005) [2023-12-26 22:21:57,529][105692] Updated weights for policy 0, policy_version 960737 (0.0005) [2023-12-26 22:21:57,661][105620] Updated weights for policy 1, policy_version 960783 (0.0010) [2023-12-26 22:21:57,717][105620] Updated weights for policy 1, policy_version 960793 (0.0011) [2023-12-26 22:21:57,774][105620] Updated weights for policy 1, policy_version 960803 (0.0011) [2023-12-26 22:21:58,184][105692] Updated weights for policy 0, policy_version 960747 (0.0006) [2023-12-26 22:21:58,250][105692] Updated weights for policy 0, policy_version 960757 (0.0008) [2023-12-26 22:21:58,314][105692] Updated weights for policy 0, policy_version 960767 (0.0008) [2023-12-26 22:21:58,436][105620] Updated weights for policy 1, policy_version 960813 (0.0010) [2023-12-26 22:21:58,502][105620] Updated weights for policy 1, policy_version 960823 (0.0008) [2023-12-26 22:21:58,571][105620] Updated weights for policy 1, policy_version 960833 (0.0008) [2023-12-26 22:21:59,137][105692] Updated weights for policy 0, policy_version 960777 (0.0011) [2023-12-26 22:21:59,199][105692] Updated weights for policy 0, policy_version 960787 (0.0010) [2023-12-26 22:21:59,262][105692] Updated weights for policy 0, policy_version 960797 (0.0008) [2023-12-26 22:21:59,323][105692] Updated weights for policy 0, policy_version 960807 (0.0009) [2023-12-26 22:21:59,527][105620] Updated weights for policy 1, policy_version 960843 (0.0010) [2023-12-26 22:21:59,575][105620] Updated weights for policy 1, policy_version 960853 (0.0008) [2023-12-26 22:21:59,635][105620] Updated weights for policy 1, policy_version 960863 (0.0006) [2023-12-26 22:22:00,057][105692] Updated weights for policy 0, policy_version 960817 (0.0008) [2023-12-26 22:22:00,123][105692] Updated weights for policy 0, policy_version 960827 (0.0005) [2023-12-26 22:22:00,183][105692] Updated weights for policy 0, policy_version 960837 (0.0005) [2023-12-26 22:22:00,392][105620] Updated weights for policy 1, policy_version 960873 (0.0009) [2023-12-26 22:22:00,453][105620] Updated weights for policy 1, policy_version 960883 (0.0008) [2023-12-26 22:22:00,515][105620] Updated weights for policy 1, policy_version 960893 (0.0009) [2023-12-26 22:22:00,577][105620] Updated weights for policy 1, policy_version 960903 (0.0009) [2023-12-26 22:22:00,861][105692] Updated weights for policy 0, policy_version 960847 (0.0008) [2023-12-26 22:22:00,918][105692] Updated weights for policy 0, policy_version 960857 (0.0008) [2023-12-26 22:22:00,970][105692] Updated weights for policy 0, policy_version 960867 (0.0010) [2023-12-26 22:22:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19272.0). Total num frames: 492044288. Throughput: 0: 9771.5, 1: 9800.4. Samples: 492014192. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:22:01,063][104569] Avg episode reward: [(0, '8197.746'), (1, '7774.745')] [2023-12-26 22:22:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000960872_246022144.pth... [2023-12-26 22:22:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000960904_246022144.pth... [2023-12-26 22:22:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000959720_245727232.pth [2023-12-26 22:22:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000959784_245735424.pth [2023-12-26 22:22:01,285][105620] Updated weights for policy 1, policy_version 960913 (0.0009) [2023-12-26 22:22:01,354][105620] Updated weights for policy 1, policy_version 960923 (0.0010) [2023-12-26 22:22:01,428][105620] Updated weights for policy 1, policy_version 960933 (0.0009) [2023-12-26 22:22:01,744][105692] Updated weights for policy 0, policy_version 960877 (0.0010) [2023-12-26 22:22:01,802][105692] Updated weights for policy 0, policy_version 960887 (0.0009) [2023-12-26 22:22:01,864][105692] Updated weights for policy 0, policy_version 960897 (0.0009) [2023-12-26 22:22:02,150][105620] Updated weights for policy 1, policy_version 960943 (0.0010) [2023-12-26 22:22:02,208][105620] Updated weights for policy 1, policy_version 960953 (0.0010) [2023-12-26 22:22:02,278][105620] Updated weights for policy 1, policy_version 960963 (0.0010) [2023-12-26 22:22:02,622][105692] Updated weights for policy 0, policy_version 960907 (0.0009) [2023-12-26 22:22:02,682][105692] Updated weights for policy 0, policy_version 960917 (0.0008) [2023-12-26 22:22:02,750][105692] Updated weights for policy 0, policy_version 960927 (0.0008) [2023-12-26 22:22:03,030][105620] Updated weights for policy 1, policy_version 960973 (0.0010) [2023-12-26 22:22:03,092][105620] Updated weights for policy 1, policy_version 960983 (0.0010) [2023-12-26 22:22:03,153][105620] Updated weights for policy 1, policy_version 960993 (0.0010) [2023-12-26 22:22:03,515][105692] Updated weights for policy 0, policy_version 960937 (0.0007) [2023-12-26 22:22:03,570][105692] Updated weights for policy 0, policy_version 960947 (0.0009) [2023-12-26 22:22:03,618][105692] Updated weights for policy 0, policy_version 960957 (0.0008) [2023-12-26 22:22:03,666][105692] Updated weights for policy 0, policy_version 960967 (0.0008) [2023-12-26 22:22:03,876][105620] Updated weights for policy 1, policy_version 961003 (0.0009) [2023-12-26 22:22:03,938][105620] Updated weights for policy 1, policy_version 961013 (0.0008) [2023-12-26 22:22:03,993][105620] Updated weights for policy 1, policy_version 961023 (0.0008) [2023-12-26 22:22:04,437][105692] Updated weights for policy 0, policy_version 960977 (0.0009) [2023-12-26 22:22:04,500][105692] Updated weights for policy 0, policy_version 960987 (0.0008) [2023-12-26 22:22:04,548][105692] Updated weights for policy 0, policy_version 960997 (0.0008) [2023-12-26 22:22:04,765][105620] Updated weights for policy 1, policy_version 961033 (0.0009) [2023-12-26 22:22:04,830][105620] Updated weights for policy 1, policy_version 961043 (0.0009) [2023-12-26 22:22:04,891][105620] Updated weights for policy 1, policy_version 961053 (0.0010) [2023-12-26 22:22:04,950][105620] Updated weights for policy 1, policy_version 961063 (0.0010) [2023-12-26 22:22:05,329][105692] Updated weights for policy 0, policy_version 961007 (0.0008) [2023-12-26 22:22:05,378][105692] Updated weights for policy 0, policy_version 961017 (0.0008) [2023-12-26 22:22:05,433][105692] Updated weights for policy 0, policy_version 961027 (0.0008) [2023-12-26 22:22:05,694][105620] Updated weights for policy 1, policy_version 961073 (0.0010) [2023-12-26 22:22:05,755][105620] Updated weights for policy 1, policy_version 961083 (0.0010) [2023-12-26 22:22:05,803][105620] Updated weights for policy 1, policy_version 961093 (0.0010) [2023-12-26 22:22:06,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19251.2, 300 sec: 19244.3). Total num frames: 492134400. Throughput: 0: 9677.6, 1: 9720.0. Samples: 492123588. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:22:06,062][104569] Avg episode reward: [(0, '8223.368'), (1, '7675.837')] [2023-12-26 22:22:06,223][105692] Updated weights for policy 0, policy_version 961037 (0.0009) [2023-12-26 22:22:06,286][105692] Updated weights for policy 0, policy_version 961047 (0.0009) [2023-12-26 22:22:06,346][105692] Updated weights for policy 0, policy_version 961057 (0.0007) [2023-12-26 22:22:06,530][105620] Updated weights for policy 1, policy_version 961103 (0.0010) [2023-12-26 22:22:06,593][105620] Updated weights for policy 1, policy_version 961113 (0.0011) [2023-12-26 22:22:06,656][105620] Updated weights for policy 1, policy_version 961123 (0.0010) [2023-12-26 22:22:07,072][105692] Updated weights for policy 0, policy_version 961067 (0.0011) [2023-12-26 22:22:07,128][105692] Updated weights for policy 0, policy_version 961077 (0.0011) [2023-12-26 22:22:07,188][105692] Updated weights for policy 0, policy_version 961087 (0.0011) [2023-12-26 22:22:07,403][105620] Updated weights for policy 1, policy_version 961133 (0.0011) [2023-12-26 22:22:07,471][105620] Updated weights for policy 1, policy_version 961143 (0.0011) [2023-12-26 22:22:07,538][105620] Updated weights for policy 1, policy_version 961153 (0.0011) [2023-12-26 22:22:07,941][105692] Updated weights for policy 0, policy_version 961097 (0.0011) [2023-12-26 22:22:07,997][105692] Updated weights for policy 0, policy_version 961107 (0.0006) [2023-12-26 22:22:08,057][105692] Updated weights for policy 0, policy_version 961117 (0.0010) [2023-12-26 22:22:08,126][105692] Updated weights for policy 0, policy_version 961127 (0.0010) [2023-12-26 22:22:08,178][105620] Updated weights for policy 1, policy_version 961163 (0.0009) [2023-12-26 22:22:08,248][105620] Updated weights for policy 1, policy_version 961173 (0.0008) [2023-12-26 22:22:08,314][105620] Updated weights for policy 1, policy_version 961183 (0.0011) [2023-12-26 22:22:08,902][105692] Updated weights for policy 0, policy_version 961137 (0.0009) [2023-12-26 22:22:08,950][105620] Updated weights for policy 1, policy_version 961193 (0.0009) [2023-12-26 22:22:08,960][105692] Updated weights for policy 0, policy_version 961147 (0.0008) [2023-12-26 22:22:09,019][105620] Updated weights for policy 1, policy_version 961203 (0.0008) [2023-12-26 22:22:09,023][105692] Updated weights for policy 0, policy_version 961157 (0.0009) [2023-12-26 22:22:09,087][105620] Updated weights for policy 1, policy_version 961213 (0.0008) [2023-12-26 22:22:09,154][105620] Updated weights for policy 1, policy_version 961223 (0.0008) [2023-12-26 22:22:09,864][105620] Updated weights for policy 1, policy_version 961233 (0.0008) [2023-12-26 22:22:09,867][105692] Updated weights for policy 0, policy_version 961167 (0.0009) [2023-12-26 22:22:09,926][105692] Updated weights for policy 0, policy_version 961177 (0.0007) [2023-12-26 22:22:09,934][105620] Updated weights for policy 1, policy_version 961243 (0.0009) [2023-12-26 22:22:09,989][105692] Updated weights for policy 0, policy_version 961187 (0.0007) [2023-12-26 22:22:09,995][105620] Updated weights for policy 1, policy_version 961253 (0.0007) [2023-12-26 22:22:10,753][105620] Updated weights for policy 1, policy_version 961263 (0.0010) [2023-12-26 22:22:10,795][105692] Updated weights for policy 0, policy_version 961197 (0.0007) [2023-12-26 22:22:10,816][105620] Updated weights for policy 1, policy_version 961273 (0.0010) [2023-12-26 22:22:10,859][105692] Updated weights for policy 0, policy_version 961207 (0.0006) [2023-12-26 22:22:10,868][105620] Updated weights for policy 1, policy_version 961283 (0.0010) [2023-12-26 22:22:10,925][105692] Updated weights for policy 0, policy_version 961217 (0.0005) [2023-12-26 22:22:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19299.8). Total num frames: 492232704. Throughput: 0: 9542.3, 1: 9680.8. Samples: 492235732. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:22:11,062][104569] Avg episode reward: [(0, '7770.883'), (1, '2698.558')] [2023-12-26 22:22:11,668][105620] Updated weights for policy 1, policy_version 961293 (0.0010) [2023-12-26 22:22:11,690][105692] Updated weights for policy 0, policy_version 961227 (0.0006) [2023-12-26 22:22:11,725][105620] Updated weights for policy 1, policy_version 961303 (0.0011) [2023-12-26 22:22:11,730][105585] KL-divergence is very high: 177.5023 [2023-12-26 22:22:11,737][105585] KL-divergence is very high: 147.6512 [2023-12-26 22:22:11,759][105692] Updated weights for policy 0, policy_version 961237 (0.0009) [2023-12-26 22:22:11,785][105585] KL-divergence is very high: 237.1340 [2023-12-26 22:22:11,792][105585] KL-divergence is very high: 166.2584 [2023-12-26 22:22:11,794][105620] Updated weights for policy 1, policy_version 961313 (0.0011) [2023-12-26 22:22:11,821][105692] Updated weights for policy 0, policy_version 961247 (0.0006) [2023-12-26 22:22:11,837][105585] KL-divergence is very high: 173.0269 [2023-12-26 22:22:11,845][105585] KL-divergence is very high: 109.2301 [2023-12-26 22:22:12,563][105620] Updated weights for policy 1, policy_version 961323 (0.0011) [2023-12-26 22:22:12,599][105692] Updated weights for policy 0, policy_version 961257 (0.0006) [2023-12-26 22:22:12,626][105620] Updated weights for policy 1, policy_version 961333 (0.0011) [2023-12-26 22:22:12,649][105692] Updated weights for policy 0, policy_version 961267 (0.0006) [2023-12-26 22:22:12,686][105620] Updated weights for policy 1, policy_version 961343 (0.0011) [2023-12-26 22:22:12,717][105692] Updated weights for policy 0, policy_version 961277 (0.0008) [2023-12-26 22:22:12,776][105692] Updated weights for policy 0, policy_version 961287 (0.0007) [2023-12-26 22:22:13,394][105620] Updated weights for policy 1, policy_version 961353 (0.0010) [2023-12-26 22:22:13,456][105620] Updated weights for policy 1, policy_version 961363 (0.0011) [2023-12-26 22:22:13,522][105620] Updated weights for policy 1, policy_version 961373 (0.0010) [2023-12-26 22:22:13,562][105692] Updated weights for policy 0, policy_version 961297 (0.0007) [2023-12-26 22:22:13,585][105620] Updated weights for policy 1, policy_version 961383 (0.0010) [2023-12-26 22:22:13,620][105692] Updated weights for policy 0, policy_version 961307 (0.0007) [2023-12-26 22:22:13,688][105692] Updated weights for policy 0, policy_version 961317 (0.0008) [2023-12-26 22:22:14,309][105620] Updated weights for policy 1, policy_version 961393 (0.0009) [2023-12-26 22:22:14,359][105620] Updated weights for policy 1, policy_version 961403 (0.0008) [2023-12-26 22:22:14,394][105692] Updated weights for policy 0, policy_version 961327 (0.0007) [2023-12-26 22:22:14,409][105620] Updated weights for policy 1, policy_version 961413 (0.0006) [2023-12-26 22:22:14,457][105692] Updated weights for policy 0, policy_version 961337 (0.0009) [2023-12-26 22:22:14,519][105692] Updated weights for policy 0, policy_version 961347 (0.0009) [2023-12-26 22:22:15,198][105620] Updated weights for policy 1, policy_version 961423 (0.0008) [2023-12-26 22:22:15,260][105620] Updated weights for policy 1, policy_version 961433 (0.0009) [2023-12-26 22:22:15,291][105692] Updated weights for policy 0, policy_version 961357 (0.0009) [2023-12-26 22:22:15,317][105620] Updated weights for policy 1, policy_version 961443 (0.0007) [2023-12-26 22:22:15,345][105692] Updated weights for policy 0, policy_version 961367 (0.0007) [2023-12-26 22:22:15,397][105692] Updated weights for policy 0, policy_version 961377 (0.0009) [2023-12-26 22:22:15,403][105585] KL-divergence is very high: 117.4647 [2023-12-26 22:22:16,062][104569] Fps is (10 sec: 18022.0, 60 sec: 19251.2, 300 sec: 19244.3). Total num frames: 492314624. Throughput: 0: 9503.0, 1: 9563.1. Samples: 492290644. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:22:16,062][104569] Avg episode reward: [(0, '7680.806'), (1, '2328.636')] [2023-12-26 22:22:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000961384_246153216.pth... [2023-12-26 22:22:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000960296_245874688.pth [2023-12-26 22:22:16,075][105620] Updated weights for policy 1, policy_version 961453 (0.0009) [2023-12-26 22:22:16,126][105620] Updated weights for policy 1, policy_version 961463 (0.0009) [2023-12-26 22:22:16,170][105692] Updated weights for policy 0, policy_version 961387 (0.0008) [2023-12-26 22:22:16,186][105620] Updated weights for policy 1, policy_version 961473 (0.0008) [2023-12-26 22:22:16,230][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000961480_246169600.pth... [2023-12-26 22:22:16,233][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000960360_245882880.pth [2023-12-26 22:22:16,234][105692] Updated weights for policy 0, policy_version 961397 (0.0008) [2023-12-26 22:22:16,285][105692] Updated weights for policy 0, policy_version 961407 (0.0009) [2023-12-26 22:22:16,944][105620] Updated weights for policy 1, policy_version 961483 (0.0007) [2023-12-26 22:22:17,009][105620] Updated weights for policy 1, policy_version 961493 (0.0009) [2023-12-26 22:22:17,049][105692] Updated weights for policy 0, policy_version 961417 (0.0010) [2023-12-26 22:22:17,073][105620] Updated weights for policy 1, policy_version 961503 (0.0008) [2023-12-26 22:22:17,096][105692] Updated weights for policy 0, policy_version 961427 (0.0007) [2023-12-26 22:22:17,144][105692] Updated weights for policy 0, policy_version 961437 (0.0009) [2023-12-26 22:22:17,198][105692] Updated weights for policy 0, policy_version 961447 (0.0009) [2023-12-26 22:22:17,727][105620] Updated weights for policy 1, policy_version 961513 (0.0008) [2023-12-26 22:22:17,773][105620] Updated weights for policy 1, policy_version 961523 (0.0007) [2023-12-26 22:22:17,833][105620] Updated weights for policy 1, policy_version 961533 (0.0007) [2023-12-26 22:22:17,880][105620] Updated weights for policy 1, policy_version 961543 (0.0009) [2023-12-26 22:22:18,035][105692] Updated weights for policy 0, policy_version 961457 (0.0006) [2023-12-26 22:22:18,097][105692] Updated weights for policy 0, policy_version 961467 (0.0006) [2023-12-26 22:22:18,167][105692] Updated weights for policy 0, policy_version 961477 (0.0006) [2023-12-26 22:22:18,769][105620] Updated weights for policy 1, policy_version 961553 (0.0011) [2023-12-26 22:22:18,830][105620] Updated weights for policy 1, policy_version 961563 (0.0011) [2023-12-26 22:22:18,859][105692] Updated weights for policy 0, policy_version 961487 (0.0009) [2023-12-26 22:22:18,893][105620] Updated weights for policy 1, policy_version 961573 (0.0010) [2023-12-26 22:22:18,922][105692] Updated weights for policy 0, policy_version 961497 (0.0009) [2023-12-26 22:22:18,981][105692] Updated weights for policy 0, policy_version 961507 (0.0008) [2023-12-26 22:22:19,638][105620] Updated weights for policy 1, policy_version 961583 (0.0011) [2023-12-26 22:22:19,705][105620] Updated weights for policy 1, policy_version 961593 (0.0011) [2023-12-26 22:22:19,752][105692] Updated weights for policy 0, policy_version 961517 (0.0007) [2023-12-26 22:22:19,765][105620] Updated weights for policy 1, policy_version 961603 (0.0011) [2023-12-26 22:22:19,814][105692] Updated weights for policy 0, policy_version 961527 (0.0010) [2023-12-26 22:22:19,882][105692] Updated weights for policy 0, policy_version 961537 (0.0008) [2023-12-26 22:22:20,525][105620] Updated weights for policy 1, policy_version 961613 (0.0011) [2023-12-26 22:22:20,588][105620] Updated weights for policy 1, policy_version 961623 (0.0011) [2023-12-26 22:22:20,653][105692] Updated weights for policy 0, policy_version 961547 (0.0009) [2023-12-26 22:22:20,660][105620] Updated weights for policy 1, policy_version 961633 (0.0011) [2023-12-26 22:22:20,714][105692] Updated weights for policy 0, policy_version 961557 (0.0011) [2023-12-26 22:22:20,775][105692] Updated weights for policy 0, policy_version 961567 (0.0011) [2023-12-26 22:22:21,062][104569] Fps is (10 sec: 18022.0, 60 sec: 19251.2, 300 sec: 19216.5). Total num frames: 492412928. Throughput: 0: 9446.8, 1: 9497.0. Samples: 492400832. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:22:21,063][104569] Avg episode reward: [(0, '6780.491'), (1, '2864.947')] [2023-12-26 22:22:21,516][105692] Updated weights for policy 0, policy_version 961577 (0.0009) [2023-12-26 22:22:21,526][105620] Updated weights for policy 1, policy_version 961643 (0.0010) [2023-12-26 22:22:21,577][105692] Updated weights for policy 0, policy_version 961587 (0.0011) [2023-12-26 22:22:21,583][105620] Updated weights for policy 1, policy_version 961653 (0.0006) [2023-12-26 22:22:21,641][105692] Updated weights for policy 0, policy_version 961597 (0.0012) [2023-12-26 22:22:21,642][105620] Updated weights for policy 1, policy_version 961663 (0.0007) [2023-12-26 22:22:21,704][105692] Updated weights for policy 0, policy_version 961607 (0.0010) [2023-12-26 22:22:22,405][105620] Updated weights for policy 1, policy_version 961673 (0.0009) [2023-12-26 22:22:22,409][105692] Updated weights for policy 0, policy_version 961617 (0.0008) [2023-12-26 22:22:22,464][105620] Updated weights for policy 1, policy_version 961683 (0.0009) [2023-12-26 22:22:22,473][105692] Updated weights for policy 0, policy_version 961627 (0.0008) [2023-12-26 22:22:22,520][105620] Updated weights for policy 1, policy_version 961693 (0.0007) [2023-12-26 22:22:22,536][105692] Updated weights for policy 0, policy_version 961637 (0.0008) [2023-12-26 22:22:22,580][105620] Updated weights for policy 1, policy_version 961703 (0.0008) [2023-12-26 22:22:23,193][105692] Updated weights for policy 0, policy_version 961647 (0.0008) [2023-12-26 22:22:23,239][105620] Updated weights for policy 1, policy_version 961713 (0.0008) [2023-12-26 22:22:23,265][105692] Updated weights for policy 0, policy_version 961657 (0.0009) [2023-12-26 22:22:23,311][105620] Updated weights for policy 1, policy_version 961723 (0.0006) [2023-12-26 22:22:23,335][105692] Updated weights for policy 0, policy_version 961667 (0.0011) [2023-12-26 22:22:23,373][105620] Updated weights for policy 1, policy_version 961733 (0.0007) [2023-12-26 22:22:23,942][105692] Updated weights for policy 0, policy_version 961677 (0.0008) [2023-12-26 22:22:23,979][105620] Updated weights for policy 1, policy_version 961743 (0.0006) [2023-12-26 22:22:24,006][105692] Updated weights for policy 0, policy_version 961687 (0.0010) [2023-12-26 22:22:24,041][105620] Updated weights for policy 1, policy_version 961753 (0.0005) [2023-12-26 22:22:24,068][105692] Updated weights for policy 0, policy_version 961697 (0.0011) [2023-12-26 22:22:24,098][105620] Updated weights for policy 1, policy_version 961763 (0.0005) [2023-12-26 22:22:24,760][105620] Updated weights for policy 1, policy_version 961773 (0.0007) [2023-12-26 22:22:24,771][105692] Updated weights for policy 0, policy_version 961707 (0.0010) [2023-12-26 22:22:24,816][105620] Updated weights for policy 1, policy_version 961783 (0.0006) [2023-12-26 22:22:24,833][105692] Updated weights for policy 0, policy_version 961717 (0.0010) [2023-12-26 22:22:24,871][105620] Updated weights for policy 1, policy_version 961793 (0.0008) [2023-12-26 22:22:24,885][105692] Updated weights for policy 0, policy_version 961727 (0.0010) [2023-12-26 22:22:25,537][105620] Updated weights for policy 1, policy_version 961803 (0.0007) [2023-12-26 22:22:25,590][105620] Updated weights for policy 1, policy_version 961813 (0.0009) [2023-12-26 22:22:25,630][105692] Updated weights for policy 0, policy_version 961737 (0.0008) [2023-12-26 22:22:25,636][105620] Updated weights for policy 1, policy_version 961823 (0.0008) [2023-12-26 22:22:25,679][105692] Updated weights for policy 0, policy_version 961747 (0.0006) [2023-12-26 22:22:25,730][105692] Updated weights for policy 0, policy_version 961757 (0.0008) [2023-12-26 22:22:25,788][105692] Updated weights for policy 0, policy_version 961767 (0.0005) [2023-12-26 22:22:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.3, 300 sec: 19216.5). Total num frames: 492511232. Throughput: 0: 9453.2, 1: 9390.7. Samples: 492517176. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:22:26,063][104569] Avg episode reward: [(0, '7379.395'), (1, '3680.304')] [2023-12-26 22:22:26,386][105620] Updated weights for policy 1, policy_version 961833 (0.0006) [2023-12-26 22:22:26,423][105692] Updated weights for policy 0, policy_version 961777 (0.0009) [2023-12-26 22:22:26,449][105620] Updated weights for policy 1, policy_version 961843 (0.0006) [2023-12-26 22:22:26,480][105692] Updated weights for policy 0, policy_version 961787 (0.0008) [2023-12-26 22:22:26,515][105620] Updated weights for policy 1, policy_version 961853 (0.0005) [2023-12-26 22:22:26,529][105692] Updated weights for policy 0, policy_version 961797 (0.0010) [2023-12-26 22:22:26,581][105620] Updated weights for policy 1, policy_version 961863 (0.0007) [2023-12-26 22:22:27,206][105620] Updated weights for policy 1, policy_version 961873 (0.0008) [2023-12-26 22:22:27,261][105692] Updated weights for policy 0, policy_version 961807 (0.0009) [2023-12-26 22:22:27,263][105620] Updated weights for policy 1, policy_version 961883 (0.0009) [2023-12-26 22:22:27,322][105620] Updated weights for policy 1, policy_version 961893 (0.0007) [2023-12-26 22:22:27,324][105692] Updated weights for policy 0, policy_version 961817 (0.0007) [2023-12-26 22:22:27,373][105692] Updated weights for policy 0, policy_version 961827 (0.0009) [2023-12-26 22:22:28,083][105620] Updated weights for policy 1, policy_version 961903 (0.0008) [2023-12-26 22:22:28,109][105692] Updated weights for policy 0, policy_version 961837 (0.0009) [2023-12-26 22:22:28,135][105620] Updated weights for policy 1, policy_version 961913 (0.0009) [2023-12-26 22:22:28,152][105692] Updated weights for policy 0, policy_version 961847 (0.0007) [2023-12-26 22:22:28,188][105620] Updated weights for policy 1, policy_version 961923 (0.0008) [2023-12-26 22:22:28,197][105692] Updated weights for policy 0, policy_version 961857 (0.0008) [2023-12-26 22:22:28,838][105620] Updated weights for policy 1, policy_version 961933 (0.0008) [2023-12-26 22:22:28,896][105620] Updated weights for policy 1, policy_version 961943 (0.0006) [2023-12-26 22:22:28,954][105620] Updated weights for policy 1, policy_version 961953 (0.0005) [2023-12-26 22:22:29,023][105692] Updated weights for policy 0, policy_version 961867 (0.0009) [2023-12-26 22:22:29,081][105692] Updated weights for policy 0, policy_version 961877 (0.0009) [2023-12-26 22:22:29,139][105692] Updated weights for policy 0, policy_version 961887 (0.0009) [2023-12-26 22:22:29,670][105620] Updated weights for policy 1, policy_version 961963 (0.0008) [2023-12-26 22:22:29,732][105620] Updated weights for policy 1, policy_version 961973 (0.0010) [2023-12-26 22:22:29,786][105620] Updated weights for policy 1, policy_version 961983 (0.0010) [2023-12-26 22:22:29,861][105692] Updated weights for policy 0, policy_version 961897 (0.0008) [2023-12-26 22:22:29,916][105692] Updated weights for policy 0, policy_version 961907 (0.0008) [2023-12-26 22:22:29,986][105692] Updated weights for policy 0, policy_version 961917 (0.0009) [2023-12-26 22:22:30,041][105692] Updated weights for policy 0, policy_version 961927 (0.0006) [2023-12-26 22:22:30,616][105620] Updated weights for policy 1, policy_version 961993 (0.0009) [2023-12-26 22:22:30,670][105620] Updated weights for policy 1, policy_version 962003 (0.0009) [2023-12-26 22:22:30,726][105620] Updated weights for policy 1, policy_version 962013 (0.0007) [2023-12-26 22:22:30,755][105692] Updated weights for policy 0, policy_version 961937 (0.0010) [2023-12-26 22:22:30,786][105620] Updated weights for policy 1, policy_version 962023 (0.0007) [2023-12-26 22:22:30,810][105692] Updated weights for policy 0, policy_version 961947 (0.0010) [2023-12-26 22:22:30,855][105692] Updated weights for policy 0, policy_version 961957 (0.0010) [2023-12-26 22:22:31,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19114.7, 300 sec: 19216.5). Total num frames: 492609536. Throughput: 0: 9505.3, 1: 9469.8. Samples: 492577060. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:22:31,062][104569] Avg episode reward: [(0, '8224.502'), (1, '5262.260')] [2023-12-26 22:22:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000961960_246300672.pth... [2023-12-26 22:22:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000962024_246308864.pth... [2023-12-26 22:22:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000960904_246022144.pth [2023-12-26 22:22:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000960872_246022144.pth [2023-12-26 22:22:31,546][105692] Updated weights for policy 0, policy_version 961967 (0.0006) [2023-12-26 22:22:31,604][105692] Updated weights for policy 0, policy_version 961977 (0.0006) [2023-12-26 22:22:31,629][105620] Updated weights for policy 1, policy_version 962033 (0.0007) [2023-12-26 22:22:31,672][105692] Updated weights for policy 0, policy_version 961987 (0.0006) [2023-12-26 22:22:31,692][105620] Updated weights for policy 1, policy_version 962043 (0.0006) [2023-12-26 22:22:31,761][105620] Updated weights for policy 1, policy_version 962053 (0.0007) [2023-12-26 22:22:32,296][105620] Updated weights for policy 1, policy_version 962063 (0.0006) [2023-12-26 22:22:32,356][105620] Updated weights for policy 1, policy_version 962073 (0.0007) [2023-12-26 22:22:32,422][105620] Updated weights for policy 1, policy_version 962083 (0.0007) [2023-12-26 22:22:32,479][105692] Updated weights for policy 0, policy_version 961997 (0.0006) [2023-12-26 22:22:32,532][105692] Updated weights for policy 0, policy_version 962007 (0.0005) [2023-12-26 22:22:32,603][105692] Updated weights for policy 0, policy_version 962017 (0.0006) [2023-12-26 22:22:32,998][105620] Updated weights for policy 1, policy_version 962093 (0.0008) [2023-12-26 22:22:33,053][105620] Updated weights for policy 1, policy_version 962104 (0.0010) [2023-12-26 22:22:33,111][105620] Updated weights for policy 1, policy_version 962114 (0.0009) [2023-12-26 22:22:33,145][105692] Updated weights for policy 0, policy_version 962027 (0.0005) [2023-12-26 22:22:33,205][105692] Updated weights for policy 0, policy_version 962037 (0.0007) [2023-12-26 22:22:33,259][105692] Updated weights for policy 0, policy_version 962047 (0.0005) [2023-12-26 22:22:33,870][105620] Updated weights for policy 1, policy_version 962124 (0.0009) [2023-12-26 22:22:33,929][105620] Updated weights for policy 1, policy_version 962134 (0.0010) [2023-12-26 22:22:33,982][105692] Updated weights for policy 0, policy_version 962057 (0.0005) [2023-12-26 22:22:33,984][105620] Updated weights for policy 1, policy_version 962144 (0.0010) [2023-12-26 22:22:34,045][105692] Updated weights for policy 0, policy_version 962067 (0.0007) [2023-12-26 22:22:34,109][105692] Updated weights for policy 0, policy_version 962077 (0.0010) [2023-12-26 22:22:34,172][105692] Updated weights for policy 0, policy_version 962087 (0.0009) [2023-12-26 22:22:34,679][105620] Updated weights for policy 1, policy_version 962154 (0.0010) [2023-12-26 22:22:34,739][105620] Updated weights for policy 1, policy_version 962164 (0.0011) [2023-12-26 22:22:34,798][105620] Updated weights for policy 1, policy_version 962174 (0.0010) [2023-12-26 22:22:34,856][105620] Updated weights for policy 1, policy_version 962184 (0.0010) [2023-12-26 22:22:34,943][105692] Updated weights for policy 0, policy_version 962097 (0.0008) [2023-12-26 22:22:34,991][105692] Updated weights for policy 0, policy_version 962107 (0.0008) [2023-12-26 22:22:35,042][105692] Updated weights for policy 0, policy_version 962117 (0.0008) [2023-12-26 22:22:35,592][105620] Updated weights for policy 1, policy_version 962194 (0.0010) [2023-12-26 22:22:35,651][105620] Updated weights for policy 1, policy_version 962204 (0.0009) [2023-12-26 22:22:35,698][105620] Updated weights for policy 1, policy_version 962214 (0.0008) [2023-12-26 22:22:35,715][105692] Updated weights for policy 0, policy_version 962127 (0.0008) [2023-12-26 22:22:35,774][105692] Updated weights for policy 0, policy_version 962137 (0.0009) [2023-12-26 22:22:35,832][105692] Updated weights for policy 0, policy_version 962147 (0.0008) [2023-12-26 22:22:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19114.7, 300 sec: 19244.3). Total num frames: 492707840. Throughput: 0: 9493.9, 1: 9496.2. Samples: 492694108. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:22:36,063][104569] Avg episode reward: [(0, '8724.507'), (1, '7588.978')] [2023-12-26 22:22:36,366][105620] Updated weights for policy 1, policy_version 962224 (0.0007) [2023-12-26 22:22:36,428][105620] Updated weights for policy 1, policy_version 962234 (0.0008) [2023-12-26 22:22:36,491][105620] Updated weights for policy 1, policy_version 962244 (0.0009) [2023-12-26 22:22:36,660][105692] Updated weights for policy 0, policy_version 962157 (0.0009) [2023-12-26 22:22:36,717][105692] Updated weights for policy 0, policy_version 962167 (0.0009) [2023-12-26 22:22:36,785][105692] Updated weights for policy 0, policy_version 962177 (0.0010) [2023-12-26 22:22:37,210][105620] Updated weights for policy 1, policy_version 962254 (0.0006) [2023-12-26 22:22:37,276][105620] Updated weights for policy 1, policy_version 962264 (0.0007) [2023-12-26 22:22:37,332][105620] Updated weights for policy 1, policy_version 962274 (0.0009) [2023-12-26 22:22:37,541][105692] Updated weights for policy 0, policy_version 962187 (0.0009) [2023-12-26 22:22:37,601][105692] Updated weights for policy 0, policy_version 962197 (0.0009) [2023-12-26 22:22:37,659][105692] Updated weights for policy 0, policy_version 962207 (0.0008) [2023-12-26 22:22:37,996][105620] Updated weights for policy 1, policy_version 962284 (0.0008) [2023-12-26 22:22:38,053][105620] Updated weights for policy 1, policy_version 962294 (0.0005) [2023-12-26 22:22:38,113][105620] Updated weights for policy 1, policy_version 962304 (0.0009) [2023-12-26 22:22:38,433][105692] Updated weights for policy 0, policy_version 962217 (0.0010) [2023-12-26 22:22:38,496][105692] Updated weights for policy 0, policy_version 962227 (0.0008) [2023-12-26 22:22:38,555][105692] Updated weights for policy 0, policy_version 962237 (0.0008) [2023-12-26 22:22:38,614][105692] Updated weights for policy 0, policy_version 962247 (0.0010) [2023-12-26 22:22:38,820][105620] Updated weights for policy 1, policy_version 962314 (0.0008) [2023-12-26 22:22:38,877][105620] Updated weights for policy 1, policy_version 962324 (0.0008) [2023-12-26 22:22:38,937][105620] Updated weights for policy 1, policy_version 962334 (0.0008) [2023-12-26 22:22:38,989][105620] Updated weights for policy 1, policy_version 962344 (0.0007) [2023-12-26 22:22:39,393][105692] Updated weights for policy 0, policy_version 962257 (0.0009) [2023-12-26 22:22:39,459][105692] Updated weights for policy 0, policy_version 962267 (0.0008) [2023-12-26 22:22:39,549][105692] Updated weights for policy 0, policy_version 962277 (0.0009) [2023-12-26 22:22:39,725][105620] Updated weights for policy 1, policy_version 962354 (0.0009) [2023-12-26 22:22:39,789][105620] Updated weights for policy 1, policy_version 962364 (0.0009) [2023-12-26 22:22:39,863][105620] Updated weights for policy 1, policy_version 962374 (0.0009) [2023-12-26 22:22:40,375][105692] Updated weights for policy 0, policy_version 962287 (0.0009) [2023-12-26 22:22:40,435][105692] Updated weights for policy 0, policy_version 962297 (0.0009) [2023-12-26 22:22:40,495][105692] Updated weights for policy 0, policy_version 962307 (0.0011) [2023-12-26 22:22:40,614][105620] Updated weights for policy 1, policy_version 962384 (0.0008) [2023-12-26 22:22:40,671][105620] Updated weights for policy 1, policy_version 962394 (0.0008) [2023-12-26 22:22:40,728][105620] Updated weights for policy 1, policy_version 962404 (0.0008) [2023-12-26 22:22:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 18978.1, 300 sec: 19216.5). Total num frames: 492797952. Throughput: 0: 9383.4, 1: 9530.6. Samples: 492806456. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:22:41,062][104569] Avg episode reward: [(0, '8717.010'), (1, '8650.687')] [2023-12-26 22:22:41,220][105692] Updated weights for policy 0, policy_version 962317 (0.0011) [2023-12-26 22:22:41,282][105692] Updated weights for policy 0, policy_version 962327 (0.0011) [2023-12-26 22:22:41,343][105692] Updated weights for policy 0, policy_version 962337 (0.0010) [2023-12-26 22:22:41,543][105620] Updated weights for policy 1, policy_version 962414 (0.0008) [2023-12-26 22:22:41,603][105620] Updated weights for policy 1, policy_version 962424 (0.0007) [2023-12-26 22:22:41,667][105620] Updated weights for policy 1, policy_version 962434 (0.0008) [2023-12-26 22:22:42,180][105692] Updated weights for policy 0, policy_version 962347 (0.0010) [2023-12-26 22:22:42,240][105692] Updated weights for policy 0, policy_version 962357 (0.0011) [2023-12-26 22:22:42,312][105692] Updated weights for policy 0, policy_version 962367 (0.0010) [2023-12-26 22:22:42,452][105620] Updated weights for policy 1, policy_version 962444 (0.0008) [2023-12-26 22:22:42,523][105620] Updated weights for policy 1, policy_version 962454 (0.0010) [2023-12-26 22:22:42,586][105620] Updated weights for policy 1, policy_version 962464 (0.0008) [2023-12-26 22:22:42,910][105692] Updated weights for policy 0, policy_version 962378 (0.0008) [2023-12-26 22:22:42,978][105692] Updated weights for policy 0, policy_version 962388 (0.0005) [2023-12-26 22:22:43,046][105692] Updated weights for policy 0, policy_version 962398 (0.0005) [2023-12-26 22:22:43,114][105692] Updated weights for policy 0, policy_version 962408 (0.0005) [2023-12-26 22:22:43,356][105620] Updated weights for policy 1, policy_version 962474 (0.0008) [2023-12-26 22:22:43,410][105620] Updated weights for policy 1, policy_version 962484 (0.0010) [2023-12-26 22:22:43,459][105620] Updated weights for policy 1, policy_version 962494 (0.0010) [2023-12-26 22:22:43,506][105620] Updated weights for policy 1, policy_version 962504 (0.0009) [2023-12-26 22:22:43,602][105692] Updated weights for policy 0, policy_version 962418 (0.0008) [2023-12-26 22:22:43,659][105692] Updated weights for policy 0, policy_version 962428 (0.0008) [2023-12-26 22:22:43,707][105692] Updated weights for policy 0, policy_version 962438 (0.0009) [2023-12-26 22:22:44,310][105620] Updated weights for policy 1, policy_version 962514 (0.0009) [2023-12-26 22:22:44,361][105620] Updated weights for policy 1, policy_version 962524 (0.0009) [2023-12-26 22:22:44,414][105620] Updated weights for policy 1, policy_version 962534 (0.0010) [2023-12-26 22:22:44,468][105692] Updated weights for policy 0, policy_version 962448 (0.0008) [2023-12-26 22:22:44,527][105692] Updated weights for policy 0, policy_version 962458 (0.0007) [2023-12-26 22:22:44,594][105692] Updated weights for policy 0, policy_version 962468 (0.0005) [2023-12-26 22:22:45,126][105620] Updated weights for policy 1, policy_version 962544 (0.0008) [2023-12-26 22:22:45,167][105692] Updated weights for policy 0, policy_version 962478 (0.0008) [2023-12-26 22:22:45,191][105620] Updated weights for policy 1, policy_version 962554 (0.0007) [2023-12-26 22:22:45,228][105692] Updated weights for policy 0, policy_version 962488 (0.0009) [2023-12-26 22:22:45,261][105620] Updated weights for policy 1, policy_version 962564 (0.0008) [2023-12-26 22:22:45,288][105692] Updated weights for policy 0, policy_version 962498 (0.0010) [2023-12-26 22:22:45,973][105620] Updated weights for policy 1, policy_version 962574 (0.0008) [2023-12-26 22:22:46,031][105620] Updated weights for policy 1, policy_version 962584 (0.0009) [2023-12-26 22:22:46,062][104569] Fps is (10 sec: 18022.4, 60 sec: 18841.6, 300 sec: 19216.5). Total num frames: 492888064. Throughput: 0: 9399.8, 1: 9497.0. Samples: 492864544. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:22:46,063][104569] Avg episode reward: [(0, '8899.209'), (1, '8648.130')] [2023-12-26 22:22:46,077][105692] Updated weights for policy 0, policy_version 962508 (0.0008) [2023-12-26 22:22:46,092][105620] Updated weights for policy 1, policy_version 962594 (0.0007) [2023-12-26 22:22:46,118][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000962600_246456320.pth... [2023-12-26 22:22:46,121][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000961480_246169600.pth [2023-12-26 22:22:46,140][105692] Updated weights for policy 0, policy_version 962518 (0.0008) [2023-12-26 22:22:46,208][105692] Updated weights for policy 0, policy_version 962528 (0.0010) [2023-12-26 22:22:46,262][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000962536_246448128.pth... [2023-12-26 22:22:46,268][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000961384_246153216.pth [2023-12-26 22:22:46,755][105620] Updated weights for policy 1, policy_version 962604 (0.0009) [2023-12-26 22:22:46,818][105620] Updated weights for policy 1, policy_version 962614 (0.0009) [2023-12-26 22:22:46,857][105692] Updated weights for policy 0, policy_version 962538 (0.0009) [2023-12-26 22:22:46,879][105620] Updated weights for policy 1, policy_version 962624 (0.0009) [2023-12-26 22:22:46,923][105692] Updated weights for policy 0, policy_version 962548 (0.0006) [2023-12-26 22:22:46,977][105692] Updated weights for policy 0, policy_version 962558 (0.0009) [2023-12-26 22:22:47,029][105692] Updated weights for policy 0, policy_version 962568 (0.0009) [2023-12-26 22:22:47,623][105620] Updated weights for policy 1, policy_version 962634 (0.0009) [2023-12-26 22:22:47,669][105692] Updated weights for policy 0, policy_version 962578 (0.0006) [2023-12-26 22:22:47,680][105620] Updated weights for policy 1, policy_version 962644 (0.0008) [2023-12-26 22:22:47,719][105692] Updated weights for policy 0, policy_version 962588 (0.0008) [2023-12-26 22:22:47,740][105620] Updated weights for policy 1, policy_version 962654 (0.0006) [2023-12-26 22:22:47,769][105692] Updated weights for policy 0, policy_version 962598 (0.0009) [2023-12-26 22:22:47,794][105620] Updated weights for policy 1, policy_version 962664 (0.0007) [2023-12-26 22:22:48,516][105692] Updated weights for policy 0, policy_version 962608 (0.0009) [2023-12-26 22:22:48,534][105620] Updated weights for policy 1, policy_version 962674 (0.0009) [2023-12-26 22:22:48,575][105692] Updated weights for policy 0, policy_version 962618 (0.0009) [2023-12-26 22:22:48,590][105620] Updated weights for policy 1, policy_version 962684 (0.0007) [2023-12-26 22:22:48,639][105692] Updated weights for policy 0, policy_version 962628 (0.0008) [2023-12-26 22:22:48,651][105620] Updated weights for policy 1, policy_version 962694 (0.0011) [2023-12-26 22:22:49,378][105620] Updated weights for policy 1, policy_version 962704 (0.0011) [2023-12-26 22:22:49,445][105620] Updated weights for policy 1, policy_version 962714 (0.0010) [2023-12-26 22:22:49,448][105692] Updated weights for policy 0, policy_version 962638 (0.0007) [2023-12-26 22:22:49,503][105620] Updated weights for policy 1, policy_version 962724 (0.0011) [2023-12-26 22:22:49,505][105692] Updated weights for policy 0, policy_version 962648 (0.0006) [2023-12-26 22:22:49,567][105692] Updated weights for policy 0, policy_version 962658 (0.0007) [2023-12-26 22:22:50,263][105620] Updated weights for policy 1, policy_version 962734 (0.0009) [2023-12-26 22:22:50,325][105620] Updated weights for policy 1, policy_version 962744 (0.0009) [2023-12-26 22:22:50,355][105692] Updated weights for policy 0, policy_version 962668 (0.0008) [2023-12-26 22:22:50,383][105620] Updated weights for policy 1, policy_version 962754 (0.0006) [2023-12-26 22:22:50,422][105692] Updated weights for policy 0, policy_version 962678 (0.0009) [2023-12-26 22:22:50,479][105692] Updated weights for policy 0, policy_version 962688 (0.0010) [2023-12-26 22:22:51,062][104569] Fps is (10 sec: 18841.3, 60 sec: 18978.1, 300 sec: 19216.5). Total num frames: 492986368. Throughput: 0: 9487.8, 1: 9563.1. Samples: 492980880. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:22:51,063][104569] Avg episode reward: [(0, '8698.075'), (1, '8733.130')] [2023-12-26 22:22:51,153][105620] Updated weights for policy 1, policy_version 962764 (0.0007) [2023-12-26 22:22:51,217][105620] Updated weights for policy 1, policy_version 962774 (0.0007) [2023-12-26 22:22:51,274][105692] Updated weights for policy 0, policy_version 962698 (0.0010) [2023-12-26 22:22:51,282][105620] Updated weights for policy 1, policy_version 962784 (0.0006) [2023-12-26 22:22:51,336][105692] Updated weights for policy 0, policy_version 962708 (0.0008) [2023-12-26 22:22:51,407][105692] Updated weights for policy 0, policy_version 962718 (0.0010) [2023-12-26 22:22:51,465][105692] Updated weights for policy 0, policy_version 962728 (0.0009) [2023-12-26 22:22:51,992][105620] Updated weights for policy 1, policy_version 962794 (0.0008) [2023-12-26 22:22:52,040][105620] Updated weights for policy 1, policy_version 962804 (0.0009) [2023-12-26 22:22:52,095][105620] Updated weights for policy 1, policy_version 962814 (0.0009) [2023-12-26 22:22:52,154][105620] Updated weights for policy 1, policy_version 962824 (0.0009) [2023-12-26 22:22:52,254][105692] Updated weights for policy 0, policy_version 962738 (0.0010) [2023-12-26 22:22:52,310][105692] Updated weights for policy 0, policy_version 962748 (0.0009) [2023-12-26 22:22:52,372][105692] Updated weights for policy 0, policy_version 962758 (0.0009) [2023-12-26 22:22:52,778][105620] Updated weights for policy 1, policy_version 962834 (0.0006) [2023-12-26 22:22:52,832][105620] Updated weights for policy 1, policy_version 962844 (0.0009) [2023-12-26 22:22:52,892][105620] Updated weights for policy 1, policy_version 962854 (0.0008) [2023-12-26 22:22:53,216][105692] Updated weights for policy 0, policy_version 962768 (0.0009) [2023-12-26 22:22:53,263][105692] Updated weights for policy 0, policy_version 962778 (0.0008) [2023-12-26 22:22:53,314][105692] Updated weights for policy 0, policy_version 962788 (0.0009) [2023-12-26 22:22:53,623][105620] Updated weights for policy 1, policy_version 962864 (0.0009) [2023-12-26 22:22:53,682][105620] Updated weights for policy 1, policy_version 962874 (0.0010) [2023-12-26 22:22:53,739][105620] Updated weights for policy 1, policy_version 962884 (0.0010) [2023-12-26 22:22:54,130][105692] Updated weights for policy 0, policy_version 962798 (0.0008) [2023-12-26 22:22:54,187][105692] Updated weights for policy 0, policy_version 962808 (0.0008) [2023-12-26 22:22:54,244][105692] Updated weights for policy 0, policy_version 962818 (0.0008) [2023-12-26 22:22:54,449][105620] Updated weights for policy 1, policy_version 962894 (0.0010) [2023-12-26 22:22:54,507][105620] Updated weights for policy 1, policy_version 962904 (0.0011) [2023-12-26 22:22:54,556][105620] Updated weights for policy 1, policy_version 962914 (0.0010) [2023-12-26 22:22:55,049][105692] Updated weights for policy 0, policy_version 962828 (0.0008) [2023-12-26 22:22:55,106][105692] Updated weights for policy 0, policy_version 962838 (0.0008) [2023-12-26 22:22:55,166][105692] Updated weights for policy 0, policy_version 962848 (0.0008) [2023-12-26 22:22:55,248][105620] Updated weights for policy 1, policy_version 962924 (0.0010) [2023-12-26 22:22:55,295][105620] Updated weights for policy 1, policy_version 962934 (0.0010) [2023-12-26 22:22:55,343][105620] Updated weights for policy 1, policy_version 962944 (0.0010) [2023-12-26 22:22:55,884][105692] Updated weights for policy 0, policy_version 962858 (0.0008) [2023-12-26 22:22:55,942][105692] Updated weights for policy 0, policy_version 962868 (0.0007) [2023-12-26 22:22:56,009][105692] Updated weights for policy 0, policy_version 962878 (0.0008) [2023-12-26 22:22:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 18841.7, 300 sec: 19216.5). Total num frames: 493076480. Throughput: 0: 9457.0, 1: 9582.0. Samples: 493092488. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:22:56,062][104569] Avg episode reward: [(0, '8944.761'), (1, '8481.141')] [2023-12-26 22:22:56,063][105692] Updated weights for policy 0, policy_version 962888 (0.0005) [2023-12-26 22:22:56,082][105620] Updated weights for policy 1, policy_version 962954 (0.0010) [2023-12-26 22:22:56,131][105620] Updated weights for policy 1, policy_version 962964 (0.0010) [2023-12-26 22:22:56,179][105620] Updated weights for policy 1, policy_version 962974 (0.0010) [2023-12-26 22:22:56,235][105620] Updated weights for policy 1, policy_version 962984 (0.0008) [2023-12-26 22:22:56,702][105692] Updated weights for policy 0, policy_version 962898 (0.0006) [2023-12-26 22:22:56,750][105692] Updated weights for policy 0, policy_version 962908 (0.0008) [2023-12-26 22:22:56,800][105692] Updated weights for policy 0, policy_version 962918 (0.0008) [2023-12-26 22:22:56,987][105620] Updated weights for policy 1, policy_version 962994 (0.0010) [2023-12-26 22:22:57,044][105620] Updated weights for policy 1, policy_version 963004 (0.0010) [2023-12-26 22:22:57,099][105620] Updated weights for policy 1, policy_version 963014 (0.0010) [2023-12-26 22:22:57,487][105692] Updated weights for policy 0, policy_version 962928 (0.0010) [2023-12-26 22:22:57,548][105692] Updated weights for policy 0, policy_version 962938 (0.0010) [2023-12-26 22:22:57,601][105692] Updated weights for policy 0, policy_version 962948 (0.0006) [2023-12-26 22:22:57,858][105620] Updated weights for policy 1, policy_version 963024 (0.0008) [2023-12-26 22:22:57,923][105620] Updated weights for policy 1, policy_version 963034 (0.0008) [2023-12-26 22:22:57,979][105620] Updated weights for policy 1, policy_version 963044 (0.0009) [2023-12-26 22:22:58,215][105692] Updated weights for policy 0, policy_version 962958 (0.0008) [2023-12-26 22:22:58,278][105692] Updated weights for policy 0, policy_version 962968 (0.0010) [2023-12-26 22:22:58,347][105692] Updated weights for policy 0, policy_version 962978 (0.0011) [2023-12-26 22:22:58,881][105620] Updated weights for policy 1, policy_version 963054 (0.0009) [2023-12-26 22:22:58,951][105620] Updated weights for policy 1, policy_version 963064 (0.0008) [2023-12-26 22:22:59,022][105620] Updated weights for policy 1, policy_version 963074 (0.0008) [2023-12-26 22:22:59,174][105692] Updated weights for policy 0, policy_version 962988 (0.0008) [2023-12-26 22:22:59,252][105692] Updated weights for policy 0, policy_version 962998 (0.0009) [2023-12-26 22:22:59,319][105692] Updated weights for policy 0, policy_version 963008 (0.0011) [2023-12-26 22:22:59,733][105620] Updated weights for policy 1, policy_version 963084 (0.0009) [2023-12-26 22:22:59,784][105620] Updated weights for policy 1, policy_version 963094 (0.0010) [2023-12-26 22:22:59,833][105620] Updated weights for policy 1, policy_version 963104 (0.0010) [2023-12-26 22:23:00,054][105692] Updated weights for policy 0, policy_version 963018 (0.0010) [2023-12-26 22:23:00,114][105692] Updated weights for policy 0, policy_version 963028 (0.0008) [2023-12-26 22:23:00,170][105692] Updated weights for policy 0, policy_version 963038 (0.0008) [2023-12-26 22:23:00,226][105692] Updated weights for policy 0, policy_version 963048 (0.0008) [2023-12-26 22:23:00,626][105620] Updated weights for policy 1, policy_version 963114 (0.0008) [2023-12-26 22:23:00,685][105620] Updated weights for policy 1, policy_version 963124 (0.0005) [2023-12-26 22:23:00,735][105620] Updated weights for policy 1, policy_version 963134 (0.0005) [2023-12-26 22:23:00,784][105620] Updated weights for policy 1, policy_version 963144 (0.0009) [2023-12-26 22:23:01,032][105692] Updated weights for policy 0, policy_version 963058 (0.0008) [2023-12-26 22:23:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18841.6, 300 sec: 19216.5). Total num frames: 493174784. Throughput: 0: 9536.7, 1: 9567.3. Samples: 493150324. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:23:01,063][104569] Avg episode reward: [(0, '8630.550'), (1, '8408.142')] [2023-12-26 22:23:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000963144_246595584.pth... [2023-12-26 22:23:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000962024_246308864.pth [2023-12-26 22:23:01,100][105692] Updated weights for policy 0, policy_version 963068 (0.0009) [2023-12-26 22:23:01,169][105692] Updated weights for policy 0, policy_version 963078 (0.0009) [2023-12-26 22:23:01,180][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000963080_246587392.pth... [2023-12-26 22:23:01,183][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000961960_246300672.pth [2023-12-26 22:23:01,450][105620] Updated weights for policy 1, policy_version 963154 (0.0011) [2023-12-26 22:23:01,499][105620] Updated weights for policy 1, policy_version 963164 (0.0010) [2023-12-26 22:23:01,555][105620] Updated weights for policy 1, policy_version 963174 (0.0010) [2023-12-26 22:23:01,967][105692] Updated weights for policy 0, policy_version 963088 (0.0008) [2023-12-26 22:23:01,973][105585] KL-divergence is very high: 111.7058 [2023-12-26 22:23:02,019][105585] KL-divergence is very high: 146.8937 [2023-12-26 22:23:02,024][105692] Updated weights for policy 0, policy_version 963098 (0.0008) [2023-12-26 22:23:02,065][105585] KL-divergence is very high: 129.6449 [2023-12-26 22:23:02,081][105692] Updated weights for policy 0, policy_version 963108 (0.0007) [2023-12-26 22:23:02,315][105620] Updated weights for policy 1, policy_version 963184 (0.0006) [2023-12-26 22:23:02,382][105620] Updated weights for policy 1, policy_version 963194 (0.0008) [2023-12-26 22:23:02,444][105620] Updated weights for policy 1, policy_version 963204 (0.0006) [2023-12-26 22:23:02,888][105692] Updated weights for policy 0, policy_version 963118 (0.0009) [2023-12-26 22:23:02,940][105692] Updated weights for policy 0, policy_version 963128 (0.0009) [2023-12-26 22:23:02,973][105620] Updated weights for policy 1, policy_version 963214 (0.0009) [2023-12-26 22:23:02,999][105692] Updated weights for policy 0, policy_version 963138 (0.0005) [2023-12-26 22:23:03,029][105620] Updated weights for policy 1, policy_version 963224 (0.0010) [2023-12-26 22:23:03,083][105620] Updated weights for policy 1, policy_version 963234 (0.0010) [2023-12-26 22:23:03,700][105692] Updated weights for policy 0, policy_version 963148 (0.0006) [2023-12-26 22:23:03,746][105620] Updated weights for policy 1, policy_version 963244 (0.0009) [2023-12-26 22:23:03,759][105692] Updated weights for policy 0, policy_version 963158 (0.0006) [2023-12-26 22:23:03,795][105620] Updated weights for policy 1, policy_version 963254 (0.0005) [2023-12-26 22:23:03,828][105692] Updated weights for policy 0, policy_version 963168 (0.0006) [2023-12-26 22:23:03,858][105620] Updated weights for policy 1, policy_version 963264 (0.0006) [2023-12-26 22:23:04,465][105620] Updated weights for policy 1, policy_version 963274 (0.0007) [2023-12-26 22:23:04,516][105620] Updated weights for policy 1, policy_version 963284 (0.0009) [2023-12-26 22:23:04,534][105692] Updated weights for policy 0, policy_version 963178 (0.0008) [2023-12-26 22:23:04,567][105620] Updated weights for policy 1, policy_version 963294 (0.0008) [2023-12-26 22:23:04,589][105692] Updated weights for policy 0, policy_version 963188 (0.0009) [2023-12-26 22:23:04,631][105620] Updated weights for policy 1, policy_version 963304 (0.0008) [2023-12-26 22:23:04,641][105692] Updated weights for policy 0, policy_version 963198 (0.0007) [2023-12-26 22:23:04,702][105692] Updated weights for policy 0, policy_version 963208 (0.0009) [2023-12-26 22:23:05,388][105620] Updated weights for policy 1, policy_version 963314 (0.0009) [2023-12-26 22:23:05,441][105620] Updated weights for policy 1, policy_version 963324 (0.0009) [2023-12-26 22:23:05,489][105692] Updated weights for policy 0, policy_version 963218 (0.0006) [2023-12-26 22:23:05,499][105620] Updated weights for policy 1, policy_version 963334 (0.0007) [2023-12-26 22:23:05,544][105692] Updated weights for policy 0, policy_version 963228 (0.0008) [2023-12-26 22:23:05,604][105692] Updated weights for policy 0, policy_version 963238 (0.0009) [2023-12-26 22:23:06,062][104569] Fps is (10 sec: 19660.4, 60 sec: 18978.0, 300 sec: 19244.3). Total num frames: 493273088. Throughput: 0: 9517.1, 1: 9691.4. Samples: 493265216. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:23:06,063][104569] Avg episode reward: [(0, '8446.926'), (1, '8570.902')] [2023-12-26 22:23:06,294][105620] Updated weights for policy 1, policy_version 963344 (0.0007) [2023-12-26 22:23:06,308][105692] Updated weights for policy 0, policy_version 963248 (0.0008) [2023-12-26 22:23:06,357][105620] Updated weights for policy 1, policy_version 963354 (0.0007) [2023-12-26 22:23:06,377][105692] Updated weights for policy 0, policy_version 963258 (0.0007) [2023-12-26 22:23:06,420][105620] Updated weights for policy 1, policy_version 963364 (0.0008) [2023-12-26 22:23:06,445][105692] Updated weights for policy 0, policy_version 963268 (0.0008) [2023-12-26 22:23:07,076][105692] Updated weights for policy 0, policy_version 963278 (0.0007) [2023-12-26 22:23:07,142][105692] Updated weights for policy 0, policy_version 963288 (0.0008) [2023-12-26 22:23:07,207][105692] Updated weights for policy 0, policy_version 963298 (0.0007) [2023-12-26 22:23:07,211][105620] Updated weights for policy 1, policy_version 963374 (0.0006) [2023-12-26 22:23:07,278][105620] Updated weights for policy 1, policy_version 963384 (0.0006) [2023-12-26 22:23:07,341][105620] Updated weights for policy 1, policy_version 963394 (0.0006) [2023-12-26 22:23:07,922][105620] Updated weights for policy 1, policy_version 963404 (0.0007) [2023-12-26 22:23:07,944][105692] Updated weights for policy 0, policy_version 963308 (0.0006) [2023-12-26 22:23:07,969][105620] Updated weights for policy 1, policy_version 963414 (0.0008) [2023-12-26 22:23:08,000][105692] Updated weights for policy 0, policy_version 963318 (0.0007) [2023-12-26 22:23:08,018][105620] Updated weights for policy 1, policy_version 963424 (0.0007) [2023-12-26 22:23:08,050][105692] Updated weights for policy 0, policy_version 963328 (0.0007) [2023-12-26 22:23:08,665][105620] Updated weights for policy 1, policy_version 963434 (0.0007) [2023-12-26 22:23:08,737][105620] Updated weights for policy 1, policy_version 963444 (0.0006) [2023-12-26 22:23:08,797][105620] Updated weights for policy 1, policy_version 963454 (0.0007) [2023-12-26 22:23:08,864][105620] Updated weights for policy 1, policy_version 963464 (0.0010) [2023-12-26 22:23:08,874][105692] Updated weights for policy 0, policy_version 963338 (0.0007) [2023-12-26 22:23:08,929][105692] Updated weights for policy 0, policy_version 963348 (0.0007) [2023-12-26 22:23:08,989][105692] Updated weights for policy 0, policy_version 963358 (0.0008) [2023-12-26 22:23:09,046][105692] Updated weights for policy 0, policy_version 963368 (0.0006) [2023-12-26 22:23:09,529][105620] Updated weights for policy 1, policy_version 963474 (0.0009) [2023-12-26 22:23:09,595][105620] Updated weights for policy 1, policy_version 963484 (0.0010) [2023-12-26 22:23:09,661][105620] Updated weights for policy 1, policy_version 963494 (0.0007) [2023-12-26 22:23:09,753][105692] Updated weights for policy 0, policy_version 963378 (0.0007) [2023-12-26 22:23:09,807][105692] Updated weights for policy 0, policy_version 963388 (0.0007) [2023-12-26 22:23:09,872][105692] Updated weights for policy 0, policy_version 963398 (0.0009) [2023-12-26 22:23:10,371][105620] Updated weights for policy 1, policy_version 963504 (0.0010) [2023-12-26 22:23:10,420][105620] Updated weights for policy 1, policy_version 963514 (0.0011) [2023-12-26 22:23:10,470][105620] Updated weights for policy 1, policy_version 963524 (0.0011) [2023-12-26 22:23:10,639][105692] Updated weights for policy 0, policy_version 963408 (0.0008) [2023-12-26 22:23:10,692][105692] Updated weights for policy 0, policy_version 963418 (0.0009) [2023-12-26 22:23:10,747][105692] Updated weights for policy 0, policy_version 963428 (0.0009) [2023-12-26 22:23:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 18978.1, 300 sec: 19244.3). Total num frames: 493371392. Throughput: 0: 9480.9, 1: 9706.3. Samples: 493380600. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:23:11,063][104569] Avg episode reward: [(0, '8538.996'), (1, '8731.356')] [2023-12-26 22:23:11,228][105620] Updated weights for policy 1, policy_version 963534 (0.0008) [2023-12-26 22:23:11,296][105620] Updated weights for policy 1, policy_version 963544 (0.0009) [2023-12-26 22:23:11,350][105620] Updated weights for policy 1, policy_version 963554 (0.0010) [2023-12-26 22:23:11,549][105692] Updated weights for policy 0, policy_version 963438 (0.0008) [2023-12-26 22:23:11,611][105692] Updated weights for policy 0, policy_version 963448 (0.0006) [2023-12-26 22:23:11,677][105692] Updated weights for policy 0, policy_version 963458 (0.0009) [2023-12-26 22:23:12,032][105620] Updated weights for policy 1, policy_version 963564 (0.0012) [2023-12-26 22:23:12,098][105620] Updated weights for policy 1, policy_version 963574 (0.0007) [2023-12-26 22:23:12,166][105620] Updated weights for policy 1, policy_version 963584 (0.0005) [2023-12-26 22:23:12,491][105692] Updated weights for policy 0, policy_version 963468 (0.0008) [2023-12-26 22:23:12,554][105692] Updated weights for policy 0, policy_version 963478 (0.0008) [2023-12-26 22:23:12,610][105692] Updated weights for policy 0, policy_version 963488 (0.0007) [2023-12-26 22:23:12,874][105620] Updated weights for policy 1, policy_version 963594 (0.0009) [2023-12-26 22:23:12,936][105620] Updated weights for policy 1, policy_version 963604 (0.0011) [2023-12-26 22:23:13,003][105620] Updated weights for policy 1, policy_version 963614 (0.0010) [2023-12-26 22:23:13,074][105620] Updated weights for policy 1, policy_version 963624 (0.0010) [2023-12-26 22:23:13,188][105692] Updated weights for policy 0, policy_version 963498 (0.0007) [2023-12-26 22:23:13,256][105692] Updated weights for policy 0, policy_version 963508 (0.0005) [2023-12-26 22:23:13,321][105692] Updated weights for policy 0, policy_version 963518 (0.0007) [2023-12-26 22:23:13,381][105692] Updated weights for policy 0, policy_version 963528 (0.0008) [2023-12-26 22:23:13,825][105620] Updated weights for policy 1, policy_version 963634 (0.0011) [2023-12-26 22:23:13,886][105620] Updated weights for policy 1, policy_version 963644 (0.0010) [2023-12-26 22:23:13,943][105620] Updated weights for policy 1, policy_version 963654 (0.0010) [2023-12-26 22:23:14,077][105692] Updated weights for policy 0, policy_version 963538 (0.0009) [2023-12-26 22:23:14,142][105692] Updated weights for policy 0, policy_version 963548 (0.0009) [2023-12-26 22:23:14,201][105692] Updated weights for policy 0, policy_version 963558 (0.0010) [2023-12-26 22:23:14,595][105620] Updated weights for policy 1, policy_version 963664 (0.0009) [2023-12-26 22:23:14,653][105620] Updated weights for policy 1, policy_version 963674 (0.0009) [2023-12-26 22:23:14,714][105620] Updated weights for policy 1, policy_version 963684 (0.0009) [2023-12-26 22:23:14,981][105692] Updated weights for policy 0, policy_version 963568 (0.0009) [2023-12-26 22:23:15,058][105692] Updated weights for policy 0, policy_version 963579 (0.0008) [2023-12-26 22:23:15,123][105692] Updated weights for policy 0, policy_version 963589 (0.0009) [2023-12-26 22:23:15,469][105620] Updated weights for policy 1, policy_version 963694 (0.0009) [2023-12-26 22:23:15,524][105620] Updated weights for policy 1, policy_version 963704 (0.0009) [2023-12-26 22:23:15,571][105620] Updated weights for policy 1, policy_version 963714 (0.0009) [2023-12-26 22:23:15,887][105692] Updated weights for policy 0, policy_version 963599 (0.0010) [2023-12-26 22:23:15,951][105692] Updated weights for policy 0, policy_version 963609 (0.0009) [2023-12-26 22:23:16,024][105692] Updated weights for policy 0, policy_version 963619 (0.0009) [2023-12-26 22:23:16,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19251.3, 300 sec: 19244.3). Total num frames: 493469696. Throughput: 0: 9467.6, 1: 9672.0. Samples: 493438340. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:23:16,062][104569] Avg episode reward: [(0, '8989.263'), (1, '8735.086')] [2023-12-26 22:23:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000963624_246726656.pth... [2023-12-26 22:23:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000963720_246743040.pth... [2023-12-26 22:23:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000962536_246448128.pth [2023-12-26 22:23:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000962600_246456320.pth [2023-12-26 22:23:16,197][105620] Updated weights for policy 1, policy_version 963724 (0.0009) [2023-12-26 22:23:16,262][105620] Updated weights for policy 1, policy_version 963734 (0.0009) [2023-12-26 22:23:16,316][105620] Updated weights for policy 1, policy_version 963744 (0.0007) [2023-12-26 22:23:16,846][105692] Updated weights for policy 0, policy_version 963629 (0.0009) [2023-12-26 22:23:16,892][105620] Updated weights for policy 1, policy_version 963754 (0.0006) [2023-12-26 22:23:16,907][105692] Updated weights for policy 0, policy_version 963639 (0.0008) [2023-12-26 22:23:16,960][105620] Updated weights for policy 1, policy_version 963764 (0.0009) [2023-12-26 22:23:16,968][105692] Updated weights for policy 0, policy_version 963649 (0.0008) [2023-12-26 22:23:17,021][105620] Updated weights for policy 1, policy_version 963774 (0.0006) [2023-12-26 22:23:17,086][105620] Updated weights for policy 1, policy_version 963784 (0.0008) [2023-12-26 22:23:17,737][105692] Updated weights for policy 0, policy_version 963659 (0.0009) [2023-12-26 22:23:17,768][105620] Updated weights for policy 1, policy_version 963794 (0.0007) [2023-12-26 22:23:17,794][105692] Updated weights for policy 0, policy_version 963669 (0.0008) [2023-12-26 22:23:17,831][105620] Updated weights for policy 1, policy_version 963804 (0.0005) [2023-12-26 22:23:17,849][105692] Updated weights for policy 0, policy_version 963679 (0.0009) [2023-12-26 22:23:17,884][105620] Updated weights for policy 1, policy_version 963814 (0.0005) [2023-12-26 22:23:18,576][105692] Updated weights for policy 0, policy_version 963689 (0.0008) [2023-12-26 22:23:18,581][105620] Updated weights for policy 1, policy_version 963824 (0.0009) [2023-12-26 22:23:18,635][105692] Updated weights for policy 0, policy_version 963699 (0.0006) [2023-12-26 22:23:18,638][105620] Updated weights for policy 1, policy_version 963834 (0.0008) [2023-12-26 22:23:18,695][105620] Updated weights for policy 1, policy_version 963844 (0.0006) [2023-12-26 22:23:18,696][105692] Updated weights for policy 0, policy_version 963709 (0.0008) [2023-12-26 22:23:18,751][105692] Updated weights for policy 0, policy_version 963719 (0.0008) [2023-12-26 22:23:19,490][105692] Updated weights for policy 0, policy_version 963729 (0.0009) [2023-12-26 22:23:19,512][105620] Updated weights for policy 1, policy_version 963854 (0.0007) [2023-12-26 22:23:19,554][105692] Updated weights for policy 0, policy_version 963739 (0.0007) [2023-12-26 22:23:19,581][105620] Updated weights for policy 1, policy_version 963864 (0.0008) [2023-12-26 22:23:19,620][105692] Updated weights for policy 0, policy_version 963749 (0.0008) [2023-12-26 22:23:19,639][105620] Updated weights for policy 1, policy_version 963874 (0.0007) [2023-12-26 22:23:20,309][105620] Updated weights for policy 1, policy_version 963884 (0.0008) [2023-12-26 22:23:20,377][105620] Updated weights for policy 1, policy_version 963894 (0.0006) [2023-12-26 22:23:20,449][105620] Updated weights for policy 1, policy_version 963904 (0.0007) [2023-12-26 22:23:20,461][105692] Updated weights for policy 0, policy_version 963759 (0.0007) [2023-12-26 22:23:20,525][105692] Updated weights for policy 0, policy_version 963769 (0.0010) [2023-12-26 22:23:20,595][105692] Updated weights for policy 0, policy_version 963779 (0.0009) [2023-12-26 22:23:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19244.3). Total num frames: 493559808. Throughput: 0: 9369.2, 1: 9695.6. Samples: 493552024. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:23:21,063][104569] Avg episode reward: [(0, '8879.510'), (1, '8555.011')] [2023-12-26 22:23:21,170][105620] Updated weights for policy 1, policy_version 963914 (0.0007) [2023-12-26 22:23:21,237][105620] Updated weights for policy 1, policy_version 963924 (0.0008) [2023-12-26 22:23:21,302][105620] Updated weights for policy 1, policy_version 963934 (0.0009) [2023-12-26 22:23:21,357][105620] Updated weights for policy 1, policy_version 963944 (0.0008) [2023-12-26 22:23:21,362][105692] Updated weights for policy 0, policy_version 963789 (0.0008) [2023-12-26 22:23:21,433][105692] Updated weights for policy 0, policy_version 963799 (0.0008) [2023-12-26 22:23:21,496][105692] Updated weights for policy 0, policy_version 963809 (0.0007) [2023-12-26 22:23:22,138][105620] Updated weights for policy 1, policy_version 963954 (0.0010) [2023-12-26 22:23:22,176][105692] Updated weights for policy 0, policy_version 963819 (0.0008) [2023-12-26 22:23:22,205][105620] Updated weights for policy 1, policy_version 963964 (0.0007) [2023-12-26 22:23:22,241][105692] Updated weights for policy 0, policy_version 963829 (0.0006) [2023-12-26 22:23:22,265][105620] Updated weights for policy 1, policy_version 963974 (0.0009) [2023-12-26 22:23:22,309][105692] Updated weights for policy 0, policy_version 963839 (0.0009) [2023-12-26 22:23:23,025][105692] Updated weights for policy 0, policy_version 963849 (0.0009) [2023-12-26 22:23:23,064][105620] Updated weights for policy 1, policy_version 963984 (0.0007) [2023-12-26 22:23:23,078][105692] Updated weights for policy 0, policy_version 963859 (0.0007) [2023-12-26 22:23:23,113][105620] Updated weights for policy 1, policy_version 963994 (0.0006) [2023-12-26 22:23:23,131][105692] Updated weights for policy 0, policy_version 963869 (0.0007) [2023-12-26 22:23:23,170][105620] Updated weights for policy 1, policy_version 964004 (0.0006) [2023-12-26 22:23:23,184][105692] Updated weights for policy 0, policy_version 963879 (0.0006) [2023-12-26 22:23:23,911][105692] Updated weights for policy 0, policy_version 963889 (0.0009) [2023-12-26 22:23:23,966][105620] Updated weights for policy 1, policy_version 964014 (0.0008) [2023-12-26 22:23:23,970][105692] Updated weights for policy 0, policy_version 963899 (0.0007) [2023-12-26 22:23:24,024][105620] Updated weights for policy 1, policy_version 964024 (0.0006) [2023-12-26 22:23:24,028][105692] Updated weights for policy 0, policy_version 963909 (0.0010) [2023-12-26 22:23:24,081][105620] Updated weights for policy 1, policy_version 964034 (0.0007) [2023-12-26 22:23:24,627][105692] Updated weights for policy 0, policy_version 963919 (0.0010) [2023-12-26 22:23:24,676][105692] Updated weights for policy 0, policy_version 963929 (0.0005) [2023-12-26 22:23:24,738][105692] Updated weights for policy 0, policy_version 963939 (0.0006) [2023-12-26 22:23:24,897][105620] Updated weights for policy 1, policy_version 964044 (0.0008) [2023-12-26 22:23:24,960][105620] Updated weights for policy 1, policy_version 964054 (0.0008) [2023-12-26 22:23:25,025][105620] Updated weights for policy 1, policy_version 964064 (0.0009) [2023-12-26 22:23:25,363][105692] Updated weights for policy 0, policy_version 963949 (0.0010) [2023-12-26 22:23:25,420][105692] Updated weights for policy 0, policy_version 963959 (0.0010) [2023-12-26 22:23:25,478][105692] Updated weights for policy 0, policy_version 963969 (0.0010) [2023-12-26 22:23:25,804][105620] Updated weights for policy 1, policy_version 964074 (0.0009) [2023-12-26 22:23:25,856][105620] Updated weights for policy 1, policy_version 964084 (0.0008) [2023-12-26 22:23:25,906][105620] Updated weights for policy 1, policy_version 964094 (0.0005) [2023-12-26 22:23:25,958][105620] Updated weights for policy 1, policy_version 964104 (0.0005) [2023-12-26 22:23:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.7, 300 sec: 19244.3). Total num frames: 493658112. Throughput: 0: 9450.7, 1: 9619.5. Samples: 493664616. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:23:26,062][104569] Avg episode reward: [(0, '8708.997'), (1, '8636.075')] [2023-12-26 22:23:26,174][105692] Updated weights for policy 0, policy_version 963979 (0.0009) [2023-12-26 22:23:26,221][105692] Updated weights for policy 0, policy_version 963989 (0.0005) [2023-12-26 22:23:26,267][105692] Updated weights for policy 0, policy_version 963999 (0.0005) [2023-12-26 22:23:26,594][105620] Updated weights for policy 1, policy_version 964114 (0.0008) [2023-12-26 22:23:26,652][105620] Updated weights for policy 1, policy_version 964124 (0.0008) [2023-12-26 22:23:26,711][105620] Updated weights for policy 1, policy_version 964134 (0.0008) [2023-12-26 22:23:26,867][105692] Updated weights for policy 0, policy_version 964009 (0.0006) [2023-12-26 22:23:26,929][105692] Updated weights for policy 0, policy_version 964019 (0.0010) [2023-12-26 22:23:26,987][105692] Updated weights for policy 0, policy_version 964029 (0.0010) [2023-12-26 22:23:27,042][105692] Updated weights for policy 0, policy_version 964039 (0.0010) [2023-12-26 22:23:27,420][105620] Updated weights for policy 1, policy_version 964144 (0.0008) [2023-12-26 22:23:27,485][105620] Updated weights for policy 1, policy_version 964154 (0.0009) [2023-12-26 22:23:27,546][105620] Updated weights for policy 1, policy_version 964164 (0.0010) [2023-12-26 22:23:27,711][105692] Updated weights for policy 0, policy_version 964049 (0.0010) [2023-12-26 22:23:27,779][105692] Updated weights for policy 0, policy_version 964059 (0.0010) [2023-12-26 22:23:27,841][105692] Updated weights for policy 0, policy_version 964069 (0.0010) [2023-12-26 22:23:28,307][105620] Updated weights for policy 1, policy_version 964174 (0.0009) [2023-12-26 22:23:28,368][105620] Updated weights for policy 1, policy_version 964184 (0.0008) [2023-12-26 22:23:28,416][105620] Updated weights for policy 1, policy_version 964194 (0.0008) [2023-12-26 22:23:28,495][105692] Updated weights for policy 0, policy_version 964079 (0.0007) [2023-12-26 22:23:28,551][105692] Updated weights for policy 0, policy_version 964089 (0.0005) [2023-12-26 22:23:28,616][105692] Updated weights for policy 0, policy_version 964099 (0.0007) [2023-12-26 22:23:29,174][105692] Updated weights for policy 0, policy_version 964109 (0.0008) [2023-12-26 22:23:29,229][105692] Updated weights for policy 0, policy_version 964119 (0.0008) [2023-12-26 22:23:29,247][105620] Updated weights for policy 1, policy_version 964204 (0.0009) [2023-12-26 22:23:29,296][105692] Updated weights for policy 0, policy_version 964129 (0.0010) [2023-12-26 22:23:29,306][105620] Updated weights for policy 1, policy_version 964214 (0.0006) [2023-12-26 22:23:29,375][105620] Updated weights for policy 1, policy_version 964224 (0.0008) [2023-12-26 22:23:29,990][105692] Updated weights for policy 0, policy_version 964139 (0.0008) [2023-12-26 22:23:30,044][105692] Updated weights for policy 0, policy_version 964149 (0.0007) [2023-12-26 22:23:30,101][105692] Updated weights for policy 0, policy_version 964159 (0.0005) [2023-12-26 22:23:30,175][105620] Updated weights for policy 1, policy_version 964234 (0.0009) [2023-12-26 22:23:30,230][105620] Updated weights for policy 1, policy_version 964244 (0.0011) [2023-12-26 22:23:30,296][105620] Updated weights for policy 1, policy_version 964254 (0.0010) [2023-12-26 22:23:30,358][105620] Updated weights for policy 1, policy_version 964264 (0.0010) [2023-12-26 22:23:30,800][105692] Updated weights for policy 0, policy_version 964169 (0.0007) [2023-12-26 22:23:30,844][105692] Updated weights for policy 0, policy_version 964179 (0.0010) [2023-12-26 22:23:30,885][105692] Updated weights for policy 0, policy_version 964189 (0.0010) [2023-12-26 22:23:30,932][105692] Updated weights for policy 0, policy_version 964199 (0.0010) [2023-12-26 22:23:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19114.7, 300 sec: 19272.0). Total num frames: 493756416. Throughput: 0: 9470.9, 1: 9663.3. Samples: 493725584. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:23:31,064][104569] Avg episode reward: [(0, '8807.970'), (1, '8904.859')] [2023-12-26 22:23:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000964200_246874112.pth... [2023-12-26 22:23:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000963080_246587392.pth [2023-12-26 22:23:31,100][105620] Updated weights for policy 1, policy_version 964274 (0.0010) [2023-12-26 22:23:31,171][105620] Updated weights for policy 1, policy_version 964284 (0.0010) [2023-12-26 22:23:31,244][105620] Updated weights for policy 1, policy_version 964294 (0.0010) [2023-12-26 22:23:31,257][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000964296_246890496.pth... [2023-12-26 22:23:31,262][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000963144_246595584.pth [2023-12-26 22:23:31,700][105692] Updated weights for policy 0, policy_version 964209 (0.0010) [2023-12-26 22:23:31,766][105692] Updated weights for policy 0, policy_version 964219 (0.0010) [2023-12-26 22:23:31,829][105692] Updated weights for policy 0, policy_version 964229 (0.0010) [2023-12-26 22:23:32,009][105620] Updated weights for policy 1, policy_version 964304 (0.0008) [2023-12-26 22:23:32,075][105620] Updated weights for policy 1, policy_version 964314 (0.0008) [2023-12-26 22:23:32,139][105620] Updated weights for policy 1, policy_version 964324 (0.0008) [2023-12-26 22:23:32,554][105692] Updated weights for policy 0, policy_version 964239 (0.0010) [2023-12-26 22:23:32,616][105692] Updated weights for policy 0, policy_version 964249 (0.0010) [2023-12-26 22:23:32,671][105692] Updated weights for policy 0, policy_version 964259 (0.0008) [2023-12-26 22:23:32,897][105620] Updated weights for policy 1, policy_version 964334 (0.0008) [2023-12-26 22:23:32,969][105620] Updated weights for policy 1, policy_version 964344 (0.0009) [2023-12-26 22:23:33,031][105620] Updated weights for policy 1, policy_version 964354 (0.0010) [2023-12-26 22:23:33,248][105692] Updated weights for policy 0, policy_version 964269 (0.0006) [2023-12-26 22:23:33,297][105692] Updated weights for policy 0, policy_version 964279 (0.0005) [2023-12-26 22:23:33,355][105692] Updated weights for policy 0, policy_version 964289 (0.0005) [2023-12-26 22:23:33,752][105620] Updated weights for policy 1, policy_version 964364 (0.0007) [2023-12-26 22:23:33,804][105620] Updated weights for policy 1, policy_version 964374 (0.0009) [2023-12-26 22:23:33,855][105620] Updated weights for policy 1, policy_version 964384 (0.0007) [2023-12-26 22:23:33,876][105692] Updated weights for policy 0, policy_version 964299 (0.0006) [2023-12-26 22:23:33,939][105692] Updated weights for policy 0, policy_version 964309 (0.0009) [2023-12-26 22:23:33,998][105692] Updated weights for policy 0, policy_version 964319 (0.0010) [2023-12-26 22:23:34,536][105620] Updated weights for policy 1, policy_version 964394 (0.0006) [2023-12-26 22:23:34,595][105620] Updated weights for policy 1, policy_version 964404 (0.0010) [2023-12-26 22:23:34,659][105620] Updated weights for policy 1, policy_version 964414 (0.0007) [2023-12-26 22:23:34,718][105620] Updated weights for policy 1, policy_version 964424 (0.0010) [2023-12-26 22:23:34,749][105692] Updated weights for policy 0, policy_version 964329 (0.0010) [2023-12-26 22:23:34,808][105692] Updated weights for policy 0, policy_version 964339 (0.0010) [2023-12-26 22:23:34,867][105692] Updated weights for policy 0, policy_version 964349 (0.0010) [2023-12-26 22:23:34,925][105692] Updated weights for policy 0, policy_version 964359 (0.0010) [2023-12-26 22:23:35,365][105620] Updated weights for policy 1, policy_version 964434 (0.0005) [2023-12-26 22:23:35,411][105620] Updated weights for policy 1, policy_version 964444 (0.0005) [2023-12-26 22:23:35,463][105620] Updated weights for policy 1, policy_version 964454 (0.0005) [2023-12-26 22:23:35,625][105692] Updated weights for policy 0, policy_version 964369 (0.0010) [2023-12-26 22:23:35,676][105692] Updated weights for policy 0, policy_version 964379 (0.0010) [2023-12-26 22:23:35,690][105585] KL-divergence is very high: 1044.9878 [2023-12-26 22:23:35,728][105692] Updated weights for policy 0, policy_version 964389 (0.0010) [2023-12-26 22:23:35,733][105585] KL-divergence is very high: 1648.2374 [2023-12-26 22:23:35,995][105620] Updated weights for policy 1, policy_version 964464 (0.0005) [2023-12-26 22:23:36,044][105620] Updated weights for policy 1, policy_version 964474 (0.0005) [2023-12-26 22:23:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19114.7, 300 sec: 19299.8). Total num frames: 493854720. Throughput: 0: 9565.2, 1: 9619.6. Samples: 493844192. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:23:36,063][104569] Avg episode reward: [(0, '7995.729'), (1, '8819.604')] [2023-12-26 22:23:36,111][105620] Updated weights for policy 1, policy_version 964484 (0.0006) [2023-12-26 22:23:36,533][105692] Updated weights for policy 0, policy_version 964399 (0.0011) [2023-12-26 22:23:36,600][105692] Updated weights for policy 0, policy_version 964409 (0.0006) [2023-12-26 22:23:36,671][105692] Updated weights for policy 0, policy_version 964419 (0.0008) [2023-12-26 22:23:36,711][105620] Updated weights for policy 1, policy_version 964494 (0.0010) [2023-12-26 22:23:36,780][105620] Updated weights for policy 1, policy_version 964504 (0.0007) [2023-12-26 22:23:36,843][105620] Updated weights for policy 1, policy_version 964514 (0.0005) [2023-12-26 22:23:37,369][105692] Updated weights for policy 0, policy_version 964429 (0.0011) [2023-12-26 22:23:37,425][105692] Updated weights for policy 0, policy_version 964439 (0.0010) [2023-12-26 22:23:37,440][105620] Updated weights for policy 1, policy_version 964524 (0.0008) [2023-12-26 22:23:37,474][105692] Updated weights for policy 0, policy_version 964449 (0.0010) [2023-12-26 22:23:37,502][105620] Updated weights for policy 1, policy_version 964534 (0.0010) [2023-12-26 22:23:37,557][105620] Updated weights for policy 1, policy_version 964544 (0.0010) [2023-12-26 22:23:38,226][105692] Updated weights for policy 0, policy_version 964459 (0.0009) [2023-12-26 22:23:38,299][105692] Updated weights for policy 0, policy_version 964469 (0.0005) [2023-12-26 22:23:38,311][105620] Updated weights for policy 1, policy_version 964554 (0.0010) [2023-12-26 22:23:38,370][105692] Updated weights for policy 0, policy_version 964479 (0.0007) [2023-12-26 22:23:38,380][105620] Updated weights for policy 1, policy_version 964564 (0.0011) [2023-12-26 22:23:38,437][105620] Updated weights for policy 1, policy_version 964574 (0.0011) [2023-12-26 22:23:38,506][105620] Updated weights for policy 1, policy_version 964584 (0.0011) [2023-12-26 22:23:38,997][105692] Updated weights for policy 0, policy_version 964489 (0.0007) [2023-12-26 22:23:39,053][105692] Updated weights for policy 0, policy_version 964499 (0.0010) [2023-12-26 22:23:39,111][105692] Updated weights for policy 0, policy_version 964509 (0.0010) [2023-12-26 22:23:39,174][105692] Updated weights for policy 0, policy_version 964519 (0.0011) [2023-12-26 22:23:39,262][105620] Updated weights for policy 1, policy_version 964594 (0.0011) [2023-12-26 22:23:39,315][105620] Updated weights for policy 1, policy_version 964604 (0.0010) [2023-12-26 22:23:39,394][105620] Updated weights for policy 1, policy_version 964614 (0.0010) [2023-12-26 22:23:39,942][105692] Updated weights for policy 0, policy_version 964529 (0.0009) [2023-12-26 22:23:39,999][105692] Updated weights for policy 0, policy_version 964539 (0.0008) [2023-12-26 22:23:40,063][105692] Updated weights for policy 0, policy_version 964549 (0.0008) [2023-12-26 22:23:40,219][105620] Updated weights for policy 1, policy_version 964624 (0.0011) [2023-12-26 22:23:40,276][105620] Updated weights for policy 1, policy_version 964634 (0.0011) [2023-12-26 22:23:40,349][105620] Updated weights for policy 1, policy_version 964644 (0.0011) [2023-12-26 22:23:40,743][105692] Updated weights for policy 0, policy_version 964559 (0.0008) [2023-12-26 22:23:40,812][105692] Updated weights for policy 0, policy_version 964569 (0.0009) [2023-12-26 22:23:40,865][105692] Updated weights for policy 0, policy_version 964579 (0.0009) [2023-12-26 22:23:41,020][105620] Updated weights for policy 1, policy_version 964654 (0.0008) [2023-12-26 22:23:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19299.8). Total num frames: 493953024. Throughput: 0: 9658.8, 1: 9686.7. Samples: 493963036. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:23:41,063][104569] Avg episode reward: [(0, '7470.865'), (1, '8909.404')] [2023-12-26 22:23:41,085][105620] Updated weights for policy 1, policy_version 964664 (0.0008) [2023-12-26 22:23:41,143][105620] Updated weights for policy 1, policy_version 964674 (0.0009) [2023-12-26 22:23:41,672][105692] Updated weights for policy 0, policy_version 964589 (0.0009) [2023-12-26 22:23:41,740][105692] Updated weights for policy 0, policy_version 964599 (0.0011) [2023-12-26 22:23:41,807][105692] Updated weights for policy 0, policy_version 964609 (0.0011) [2023-12-26 22:23:41,882][105620] Updated weights for policy 1, policy_version 964684 (0.0009) [2023-12-26 22:23:41,935][105620] Updated weights for policy 1, policy_version 964694 (0.0007) [2023-12-26 22:23:41,993][105620] Updated weights for policy 1, policy_version 964704 (0.0008) [2023-12-26 22:23:42,528][105692] Updated weights for policy 0, policy_version 964619 (0.0011) [2023-12-26 22:23:42,574][105692] Updated weights for policy 0, policy_version 964629 (0.0009) [2023-12-26 22:23:42,638][105692] Updated weights for policy 0, policy_version 964639 (0.0008) [2023-12-26 22:23:42,690][105620] Updated weights for policy 1, policy_version 964714 (0.0008) [2023-12-26 22:23:42,760][105620] Updated weights for policy 1, policy_version 964724 (0.0010) [2023-12-26 22:23:42,822][105620] Updated weights for policy 1, policy_version 964734 (0.0010) [2023-12-26 22:23:42,881][105620] Updated weights for policy 1, policy_version 964744 (0.0010) [2023-12-26 22:23:43,382][105692] Updated weights for policy 0, policy_version 964649 (0.0011) [2023-12-26 22:23:43,434][105692] Updated weights for policy 0, policy_version 964659 (0.0011) [2023-12-26 22:23:43,494][105692] Updated weights for policy 0, policy_version 964669 (0.0011) [2023-12-26 22:23:43,547][105692] Updated weights for policy 0, policy_version 964679 (0.0008) [2023-12-26 22:23:43,611][105620] Updated weights for policy 1, policy_version 964754 (0.0010) [2023-12-26 22:23:43,660][105620] Updated weights for policy 1, policy_version 964764 (0.0010) [2023-12-26 22:23:43,711][105620] Updated weights for policy 1, policy_version 964774 (0.0010) [2023-12-26 22:23:44,327][105692] Updated weights for policy 0, policy_version 964689 (0.0007) [2023-12-26 22:23:44,384][105692] Updated weights for policy 0, policy_version 964699 (0.0006) [2023-12-26 22:23:44,398][105620] Updated weights for policy 1, policy_version 964784 (0.0010) [2023-12-26 22:23:44,442][105692] Updated weights for policy 0, policy_version 964709 (0.0006) [2023-12-26 22:23:44,461][105620] Updated weights for policy 1, policy_version 964794 (0.0008) [2023-12-26 22:23:44,509][105620] Updated weights for policy 1, policy_version 964804 (0.0009) [2023-12-26 22:23:45,148][105692] Updated weights for policy 0, policy_version 964719 (0.0007) [2023-12-26 22:23:45,216][105692] Updated weights for policy 0, policy_version 964729 (0.0009) [2023-12-26 22:23:45,228][105585] KL-divergence is very high: 102.5971 [2023-12-26 22:23:45,235][105620] Updated weights for policy 1, policy_version 964814 (0.0010) [2023-12-26 22:23:45,265][105585] KL-divergence is very high: 132.1048 [2023-12-26 22:23:45,277][105692] Updated weights for policy 0, policy_version 964739 (0.0007) [2023-12-26 22:23:45,278][105585] KL-divergence is very high: 171.4573 [2023-12-26 22:23:45,299][105620] Updated weights for policy 1, policy_version 964824 (0.0011) [2023-12-26 22:23:45,362][105620] Updated weights for policy 1, policy_version 964834 (0.0006) [2023-12-26 22:23:45,983][105585] KL-divergence is very high: 134.1656 [2023-12-26 22:23:45,997][105585] KL-divergence is very high: 121.6638 [2023-12-26 22:23:46,017][105692] Updated weights for policy 0, policy_version 964749 (0.0007) [2023-12-26 22:23:46,036][105585] KL-divergence is very high: 151.4227 [2023-12-26 22:23:46,049][105585] KL-divergence is very high: 141.1732 [2023-12-26 22:23:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19272.0). Total num frames: 494043136. Throughput: 0: 9590.9, 1: 9721.4. Samples: 494019376. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-26 22:23:46,062][104569] Avg episode reward: [(0, '7401.476'), (1, '8289.932')] [2023-12-26 22:23:46,079][105692] Updated weights for policy 0, policy_version 964759 (0.0007) [2023-12-26 22:23:46,080][105620] Updated weights for policy 1, policy_version 964844 (0.0009) [2023-12-26 22:23:46,086][105585] KL-divergence is very high: 138.7749 [2023-12-26 22:23:46,098][105585] KL-divergence is very high: 126.5161 [2023-12-26 22:23:46,129][105620] Updated weights for policy 1, policy_version 964854 (0.0010) [2023-12-26 22:23:46,134][105585] KL-divergence is very high: 122.5028 [2023-12-26 22:23:46,139][105692] Updated weights for policy 0, policy_version 964769 (0.0006) [2023-12-26 22:23:46,146][105585] KL-divergence is very high: 108.0759 [2023-12-26 22:23:46,183][105620] Updated weights for policy 1, policy_version 964864 (0.0010) [2023-12-26 22:23:46,185][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000964776_247021568.pth... [2023-12-26 22:23:46,189][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000963624_246726656.pth [2023-12-26 22:23:46,231][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000964872_247037952.pth... [2023-12-26 22:23:46,235][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000963720_246743040.pth [2023-12-26 22:23:46,767][105620] Updated weights for policy 1, policy_version 964874 (0.0006) [2023-12-26 22:23:46,834][105620] Updated weights for policy 1, policy_version 964884 (0.0010) [2023-12-26 22:23:46,899][105620] Updated weights for policy 1, policy_version 964894 (0.0010) [2023-12-26 22:23:46,965][105620] Updated weights for policy 1, policy_version 964904 (0.0008) [2023-12-26 22:23:47,001][105692] Updated weights for policy 0, policy_version 964779 (0.0009) [2023-12-26 22:23:47,064][105692] Updated weights for policy 0, policy_version 964789 (0.0009) [2023-12-26 22:23:47,128][105692] Updated weights for policy 0, policy_version 964799 (0.0009) [2023-12-26 22:23:47,675][105620] Updated weights for policy 1, policy_version 964914 (0.0010) [2023-12-26 22:23:47,732][105620] Updated weights for policy 1, policy_version 964924 (0.0011) [2023-12-26 22:23:47,753][105692] Updated weights for policy 0, policy_version 964809 (0.0008) [2023-12-26 22:23:47,803][105620] Updated weights for policy 1, policy_version 964934 (0.0010) [2023-12-26 22:23:47,816][105692] Updated weights for policy 0, policy_version 964819 (0.0007) [2023-12-26 22:23:47,873][105692] Updated weights for policy 0, policy_version 964829 (0.0006) [2023-12-26 22:23:47,934][105692] Updated weights for policy 0, policy_version 964839 (0.0005) [2023-12-26 22:23:48,531][105620] Updated weights for policy 1, policy_version 964944 (0.0008) [2023-12-26 22:23:48,586][105620] Updated weights for policy 1, policy_version 964954 (0.0008) [2023-12-26 22:23:48,622][105692] Updated weights for policy 0, policy_version 964849 (0.0010) [2023-12-26 22:23:48,644][105620] Updated weights for policy 1, policy_version 964964 (0.0008) [2023-12-26 22:23:48,682][105692] Updated weights for policy 0, policy_version 964859 (0.0008) [2023-12-26 22:23:48,741][105692] Updated weights for policy 0, policy_version 964869 (0.0010) [2023-12-26 22:23:49,383][105620] Updated weights for policy 1, policy_version 964974 (0.0009) [2023-12-26 22:23:49,443][105620] Updated weights for policy 1, policy_version 964984 (0.0011) [2023-12-26 22:23:49,496][105692] Updated weights for policy 0, policy_version 964879 (0.0006) [2023-12-26 22:23:49,499][105620] Updated weights for policy 1, policy_version 964994 (0.0011) [2023-12-26 22:23:49,559][105692] Updated weights for policy 0, policy_version 964889 (0.0006) [2023-12-26 22:23:49,611][105692] Updated weights for policy 0, policy_version 964899 (0.0008) [2023-12-26 22:23:50,286][105620] Updated weights for policy 1, policy_version 965004 (0.0010) [2023-12-26 22:23:50,337][105620] Updated weights for policy 1, policy_version 965014 (0.0009) [2023-12-26 22:23:50,370][105692] Updated weights for policy 0, policy_version 964909 (0.0007) [2023-12-26 22:23:50,402][105620] Updated weights for policy 1, policy_version 965024 (0.0007) [2023-12-26 22:23:50,436][105692] Updated weights for policy 0, policy_version 964919 (0.0009) [2023-12-26 22:23:50,497][105692] Updated weights for policy 0, policy_version 964929 (0.0010) [2023-12-26 22:23:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19272.0). Total num frames: 494141440. Throughput: 0: 9650.3, 1: 9688.1. Samples: 494135436. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-26 22:23:51,062][104569] Avg episode reward: [(0, '7486.799'), (1, '8110.799')] [2023-12-26 22:23:51,086][105620] Updated weights for policy 1, policy_version 965034 (0.0008) [2023-12-26 22:23:51,147][105620] Updated weights for policy 1, policy_version 965044 (0.0009) [2023-12-26 22:23:51,216][105620] Updated weights for policy 1, policy_version 965054 (0.0007) [2023-12-26 22:23:51,284][105620] Updated weights for policy 1, policy_version 965064 (0.0007) [2023-12-26 22:23:51,317][105692] Updated weights for policy 0, policy_version 964939 (0.0008) [2023-12-26 22:23:51,396][105692] Updated weights for policy 0, policy_version 964949 (0.0009) [2023-12-26 22:23:51,465][105692] Updated weights for policy 0, policy_version 964959 (0.0008) [2023-12-26 22:23:52,099][105620] Updated weights for policy 1, policy_version 965074 (0.0009) [2023-12-26 22:23:52,111][105692] Updated weights for policy 0, policy_version 964969 (0.0007) [2023-12-26 22:23:52,150][105620] Updated weights for policy 1, policy_version 965084 (0.0007) [2023-12-26 22:23:52,169][105692] Updated weights for policy 0, policy_version 964979 (0.0007) [2023-12-26 22:23:52,211][105620] Updated weights for policy 1, policy_version 965094 (0.0006) [2023-12-26 22:23:52,226][105692] Updated weights for policy 0, policy_version 964989 (0.0011) [2023-12-26 22:23:52,290][105692] Updated weights for policy 0, policy_version 964999 (0.0011) [2023-12-26 22:23:52,995][105620] Updated weights for policy 1, policy_version 965104 (0.0006) [2023-12-26 22:23:52,997][105692] Updated weights for policy 0, policy_version 965009 (0.0011) [2023-12-26 22:23:53,049][105692] Updated weights for policy 0, policy_version 965019 (0.0010) [2023-12-26 22:23:53,051][105620] Updated weights for policy 1, policy_version 965114 (0.0005) [2023-12-26 22:23:53,104][105620] Updated weights for policy 1, policy_version 965124 (0.0005) [2023-12-26 22:23:53,105][105692] Updated weights for policy 0, policy_version 965029 (0.0010) [2023-12-26 22:23:53,830][105692] Updated weights for policy 0, policy_version 965039 (0.0009) [2023-12-26 22:23:53,858][105620] Updated weights for policy 1, policy_version 965134 (0.0007) [2023-12-26 22:23:53,889][105692] Updated weights for policy 0, policy_version 965049 (0.0008) [2023-12-26 22:23:53,919][105620] Updated weights for policy 1, policy_version 965144 (0.0008) [2023-12-26 22:23:53,942][105692] Updated weights for policy 0, policy_version 965059 (0.0007) [2023-12-26 22:23:53,976][105620] Updated weights for policy 1, policy_version 965154 (0.0008) [2023-12-26 22:23:54,644][105692] Updated weights for policy 0, policy_version 965069 (0.0006) [2023-12-26 22:23:54,691][105692] Updated weights for policy 0, policy_version 965079 (0.0005) [2023-12-26 22:23:54,699][105585] KL-divergence is very high: 115.0441 [2023-12-26 22:23:54,738][105692] Updated weights for policy 0, policy_version 965089 (0.0005) [2023-12-26 22:23:54,738][105585] KL-divergence is very high: 130.9474 [2023-12-26 22:23:54,766][105620] Updated weights for policy 1, policy_version 965164 (0.0009) [2023-12-26 22:23:54,817][105620] Updated weights for policy 1, policy_version 965174 (0.0009) [2023-12-26 22:23:54,869][105620] Updated weights for policy 1, policy_version 965184 (0.0009) [2023-12-26 22:23:55,358][105692] Updated weights for policy 0, policy_version 965099 (0.0008) [2023-12-26 22:23:55,424][105692] Updated weights for policy 0, policy_version 965109 (0.0010) [2023-12-26 22:23:55,483][105692] Updated weights for policy 0, policy_version 965119 (0.0010) [2023-12-26 22:23:55,688][105620] Updated weights for policy 1, policy_version 965194 (0.0009) [2023-12-26 22:23:55,745][105620] Updated weights for policy 1, policy_version 965204 (0.0010) [2023-12-26 22:23:55,802][105620] Updated weights for policy 1, policy_version 965214 (0.0009) [2023-12-26 22:23:55,870][105620] Updated weights for policy 1, policy_version 965224 (0.0009) [2023-12-26 22:23:56,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19272.0). Total num frames: 494239744. Throughput: 0: 9680.9, 1: 9579.2. Samples: 494247304. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-26 22:23:56,063][104569] Avg episode reward: [(0, '8457.318'), (1, '8377.401')] [2023-12-26 22:23:56,081][105692] Updated weights for policy 0, policy_version 965129 (0.0010) [2023-12-26 22:23:56,133][105692] Updated weights for policy 0, policy_version 965139 (0.0007) [2023-12-26 22:23:56,183][105692] Updated weights for policy 0, policy_version 965149 (0.0005) [2023-12-26 22:23:56,248][105692] Updated weights for policy 0, policy_version 965159 (0.0007) [2023-12-26 22:23:56,566][105620] Updated weights for policy 1, policy_version 965234 (0.0008) [2023-12-26 22:23:56,617][105620] Updated weights for policy 1, policy_version 965244 (0.0008) [2023-12-26 22:23:56,679][105620] Updated weights for policy 1, policy_version 965254 (0.0008) [2023-12-26 22:23:56,964][105692] Updated weights for policy 0, policy_version 965169 (0.0010) [2023-12-26 22:23:57,028][105692] Updated weights for policy 0, policy_version 965179 (0.0009) [2023-12-26 22:23:57,083][105692] Updated weights for policy 0, policy_version 965189 (0.0005) [2023-12-26 22:23:57,327][105620] Updated weights for policy 1, policy_version 965264 (0.0007) [2023-12-26 22:23:57,379][105620] Updated weights for policy 1, policy_version 965274 (0.0008) [2023-12-26 22:23:57,422][105620] Updated weights for policy 1, policy_version 965284 (0.0008) [2023-12-26 22:23:57,674][105692] Updated weights for policy 0, policy_version 965199 (0.0005) [2023-12-26 22:23:57,731][105692] Updated weights for policy 0, policy_version 965209 (0.0006) [2023-12-26 22:23:57,779][105692] Updated weights for policy 0, policy_version 965219 (0.0009) [2023-12-26 22:23:58,243][105620] Updated weights for policy 1, policy_version 965294 (0.0008) [2023-12-26 22:23:58,303][105620] Updated weights for policy 1, policy_version 965304 (0.0010) [2023-12-26 22:23:58,373][105620] Updated weights for policy 1, policy_version 965314 (0.0010) [2023-12-26 22:23:58,453][105692] Updated weights for policy 0, policy_version 965229 (0.0008) [2023-12-26 22:23:58,520][105692] Updated weights for policy 0, policy_version 965239 (0.0008) [2023-12-26 22:23:58,597][105692] Updated weights for policy 0, policy_version 965249 (0.0009) [2023-12-26 22:23:59,186][105620] Updated weights for policy 1, policy_version 965324 (0.0011) [2023-12-26 22:23:59,261][105620] Updated weights for policy 1, policy_version 965334 (0.0010) [2023-12-26 22:23:59,332][105620] Updated weights for policy 1, policy_version 965344 (0.0011) [2023-12-26 22:23:59,351][105692] Updated weights for policy 0, policy_version 965259 (0.0010) [2023-12-26 22:23:59,418][105692] Updated weights for policy 0, policy_version 965269 (0.0008) [2023-12-26 22:23:59,481][105692] Updated weights for policy 0, policy_version 965279 (0.0005) [2023-12-26 22:24:00,082][105620] Updated weights for policy 1, policy_version 965354 (0.0011) [2023-12-26 22:24:00,147][105620] Updated weights for policy 1, policy_version 965364 (0.0009) [2023-12-26 22:24:00,207][105620] Updated weights for policy 1, policy_version 965374 (0.0010) [2023-12-26 22:24:00,216][105692] Updated weights for policy 0, policy_version 965289 (0.0007) [2023-12-26 22:24:00,268][105692] Updated weights for policy 0, policy_version 965299 (0.0007) [2023-12-26 22:24:00,270][105620] Updated weights for policy 1, policy_version 965384 (0.0007) [2023-12-26 22:24:00,320][105692] Updated weights for policy 0, policy_version 965309 (0.0009) [2023-12-26 22:24:00,379][105692] Updated weights for policy 0, policy_version 965319 (0.0009) [2023-12-26 22:24:00,964][105620] Updated weights for policy 1, policy_version 965394 (0.0009) [2023-12-26 22:24:01,015][105620] Updated weights for policy 1, policy_version 965404 (0.0009) [2023-12-26 22:24:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19244.3). Total num frames: 494329856. Throughput: 0: 9760.3, 1: 9570.6. Samples: 494308236. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-26 22:24:01,062][104569] Avg episode reward: [(0, '7403.125'), (1, '8914.660')] [2023-12-26 22:24:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000965320_247160832.pth... [2023-12-26 22:24:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000964200_246874112.pth [2023-12-26 22:24:01,081][105620] Updated weights for policy 1, policy_version 965414 (0.0006) [2023-12-26 22:24:01,094][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000965416_247177216.pth... [2023-12-26 22:24:01,099][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000964296_246890496.pth [2023-12-26 22:24:01,174][105692] Updated weights for policy 0, policy_version 965329 (0.0010) [2023-12-26 22:24:01,233][105692] Updated weights for policy 0, policy_version 965339 (0.0009) [2023-12-26 22:24:01,297][105692] Updated weights for policy 0, policy_version 965349 (0.0009) [2023-12-26 22:24:01,907][105620] Updated weights for policy 1, policy_version 965424 (0.0008) [2023-12-26 22:24:01,957][105620] Updated weights for policy 1, policy_version 965434 (0.0008) [2023-12-26 22:24:02,017][105620] Updated weights for policy 1, policy_version 965444 (0.0008) [2023-12-26 22:24:02,042][105692] Updated weights for policy 0, policy_version 965359 (0.0006) [2023-12-26 22:24:02,085][105692] Updated weights for policy 0, policy_version 965369 (0.0005) [2023-12-26 22:24:02,152][105692] Updated weights for policy 0, policy_version 965379 (0.0010) [2023-12-26 22:24:02,761][105692] Updated weights for policy 0, policy_version 965389 (0.0008) [2023-12-26 22:24:02,812][105692] Updated weights for policy 0, policy_version 965399 (0.0010) [2023-12-26 22:24:02,863][105692] Updated weights for policy 0, policy_version 965409 (0.0010) [2023-12-26 22:24:02,874][105620] Updated weights for policy 1, policy_version 965454 (0.0007) [2023-12-26 22:24:02,934][105620] Updated weights for policy 1, policy_version 965464 (0.0009) [2023-12-26 22:24:03,001][105620] Updated weights for policy 1, policy_version 965474 (0.0006) [2023-12-26 22:24:03,531][105692] Updated weights for policy 0, policy_version 965419 (0.0010) [2023-12-26 22:24:03,532][105620] Updated weights for policy 1, policy_version 965484 (0.0005) [2023-12-26 22:24:03,582][105620] Updated weights for policy 1, policy_version 965494 (0.0005) [2023-12-26 22:24:03,591][105692] Updated weights for policy 0, policy_version 965429 (0.0010) [2023-12-26 22:24:03,627][105620] Updated weights for policy 1, policy_version 965504 (0.0005) [2023-12-26 22:24:03,636][105692] Updated weights for policy 0, policy_version 965439 (0.0010) [2023-12-26 22:24:04,257][105620] Updated weights for policy 1, policy_version 965514 (0.0006) [2023-12-26 22:24:04,323][105620] Updated weights for policy 1, policy_version 965524 (0.0008) [2023-12-26 22:24:04,363][105692] Updated weights for policy 0, policy_version 965449 (0.0010) [2023-12-26 22:24:04,388][105620] Updated weights for policy 1, policy_version 965534 (0.0011) [2023-12-26 22:24:04,426][105692] Updated weights for policy 0, policy_version 965459 (0.0009) [2023-12-26 22:24:04,454][105620] Updated weights for policy 1, policy_version 965544 (0.0011) [2023-12-26 22:24:04,486][105692] Updated weights for policy 0, policy_version 965469 (0.0010) [2023-12-26 22:24:04,543][105692] Updated weights for policy 0, policy_version 965479 (0.0010) [2023-12-26 22:24:05,133][105620] Updated weights for policy 1, policy_version 965554 (0.0011) [2023-12-26 22:24:05,200][105620] Updated weights for policy 1, policy_version 965564 (0.0010) [2023-12-26 22:24:05,260][105620] Updated weights for policy 1, policy_version 965574 (0.0010) [2023-12-26 22:24:05,334][105692] Updated weights for policy 0, policy_version 965489 (0.0010) [2023-12-26 22:24:05,392][105692] Updated weights for policy 0, policy_version 965499 (0.0010) [2023-12-26 22:24:05,446][105692] Updated weights for policy 0, policy_version 965509 (0.0007) [2023-12-26 22:24:05,998][105620] Updated weights for policy 1, policy_version 965584 (0.0010) [2023-12-26 22:24:06,051][105692] Updated weights for policy 0, policy_version 965519 (0.0007) [2023-12-26 22:24:06,057][105620] Updated weights for policy 1, policy_version 965594 (0.0010) [2023-12-26 22:24:06,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19251.3, 300 sec: 19272.0). Total num frames: 494428160. Throughput: 0: 9842.7, 1: 9543.3. Samples: 494424392. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-26 22:24:06,062][104569] Avg episode reward: [(0, '6834.656'), (1, '8841.932')] [2023-12-26 22:24:06,107][105692] Updated weights for policy 0, policy_version 965529 (0.0006) [2023-12-26 22:24:06,111][105620] Updated weights for policy 1, policy_version 965604 (0.0010) [2023-12-26 22:24:06,175][105692] Updated weights for policy 0, policy_version 965539 (0.0007) [2023-12-26 22:24:06,797][105692] Updated weights for policy 0, policy_version 965549 (0.0006) [2023-12-26 22:24:06,852][105692] Updated weights for policy 0, policy_version 965559 (0.0005) [2023-12-26 22:24:06,902][105620] Updated weights for policy 1, policy_version 965614 (0.0011) [2023-12-26 22:24:06,906][105692] Updated weights for policy 0, policy_version 965569 (0.0005) [2023-12-26 22:24:06,962][105620] Updated weights for policy 1, policy_version 965624 (0.0011) [2023-12-26 22:24:07,020][105620] Updated weights for policy 1, policy_version 965634 (0.0005) [2023-12-26 22:24:07,537][105692] Updated weights for policy 0, policy_version 965579 (0.0007) [2023-12-26 22:24:07,586][105620] Updated weights for policy 1, policy_version 965644 (0.0005) [2023-12-26 22:24:07,597][105692] Updated weights for policy 0, policy_version 965589 (0.0010) [2023-12-26 22:24:07,642][105620] Updated weights for policy 1, policy_version 965654 (0.0006) [2023-12-26 22:24:07,657][105692] Updated weights for policy 0, policy_version 965599 (0.0010) [2023-12-26 22:24:07,704][105620] Updated weights for policy 1, policy_version 965664 (0.0007) [2023-12-26 22:24:08,304][105620] Updated weights for policy 1, policy_version 965674 (0.0009) [2023-12-26 22:24:08,375][105620] Updated weights for policy 1, policy_version 965684 (0.0006) [2023-12-26 22:24:08,405][105692] Updated weights for policy 0, policy_version 965609 (0.0010) [2023-12-26 22:24:08,439][105620] Updated weights for policy 1, policy_version 965694 (0.0010) [2023-12-26 22:24:08,458][105692] Updated weights for policy 0, policy_version 965619 (0.0011) [2023-12-26 22:24:08,501][105620] Updated weights for policy 1, policy_version 965704 (0.0010) [2023-12-26 22:24:08,517][105692] Updated weights for policy 0, policy_version 965629 (0.0011) [2023-12-26 22:24:08,567][105692] Updated weights for policy 0, policy_version 965639 (0.0010) [2023-12-26 22:24:09,095][105620] Updated weights for policy 1, policy_version 965714 (0.0009) [2023-12-26 22:24:09,147][105620] Updated weights for policy 1, policy_version 965724 (0.0010) [2023-12-26 22:24:09,198][105620] Updated weights for policy 1, policy_version 965734 (0.0010) [2023-12-26 22:24:09,332][105692] Updated weights for policy 0, policy_version 965649 (0.0010) [2023-12-26 22:24:09,401][105692] Updated weights for policy 0, policy_version 965659 (0.0009) [2023-12-26 22:24:09,470][105692] Updated weights for policy 0, policy_version 965669 (0.0009) [2023-12-26 22:24:09,921][105620] Updated weights for policy 1, policy_version 965744 (0.0008) [2023-12-26 22:24:09,994][105620] Updated weights for policy 1, policy_version 965754 (0.0009) [2023-12-26 22:24:10,047][105620] Updated weights for policy 1, policy_version 965764 (0.0008) [2023-12-26 22:24:10,237][105692] Updated weights for policy 0, policy_version 965679 (0.0011) [2023-12-26 22:24:10,296][105692] Updated weights for policy 0, policy_version 965689 (0.0010) [2023-12-26 22:24:10,352][105692] Updated weights for policy 0, policy_version 965699 (0.0010) [2023-12-26 22:24:10,814][105620] Updated weights for policy 1, policy_version 965774 (0.0009) [2023-12-26 22:24:10,881][105620] Updated weights for policy 1, policy_version 965784 (0.0009) [2023-12-26 22:24:10,942][105620] Updated weights for policy 1, policy_version 965794 (0.0008) [2023-12-26 22:24:11,037][105692] Updated weights for policy 0, policy_version 965709 (0.0009) [2023-12-26 22:24:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.7, 300 sec: 19327.6). Total num frames: 494534656. Throughput: 0: 9875.9, 1: 9651.6. Samples: 494543356. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-26 22:24:11,062][104569] Avg episode reward: [(0, '7450.553'), (1, '8757.331')] [2023-12-26 22:24:11,106][105692] Updated weights for policy 0, policy_version 965719 (0.0009) [2023-12-26 22:24:11,166][105692] Updated weights for policy 0, policy_version 965729 (0.0009) [2023-12-26 22:24:11,699][105620] Updated weights for policy 1, policy_version 965804 (0.0006) [2023-12-26 22:24:11,767][105620] Updated weights for policy 1, policy_version 965814 (0.0009) [2023-12-26 22:24:11,823][105620] Updated weights for policy 1, policy_version 965824 (0.0008) [2023-12-26 22:24:11,951][105692] Updated weights for policy 0, policy_version 965739 (0.0010) [2023-12-26 22:24:12,017][105692] Updated weights for policy 0, policy_version 965749 (0.0009) [2023-12-26 22:24:12,065][105692] Updated weights for policy 0, policy_version 965759 (0.0009) [2023-12-26 22:24:12,553][105620] Updated weights for policy 1, policy_version 965834 (0.0008) [2023-12-26 22:24:12,629][105620] Updated weights for policy 1, policy_version 965844 (0.0006) [2023-12-26 22:24:12,697][105620] Updated weights for policy 1, policy_version 965854 (0.0006) [2023-12-26 22:24:12,765][105620] Updated weights for policy 1, policy_version 965864 (0.0007) [2023-12-26 22:24:12,880][105692] Updated weights for policy 0, policy_version 965769 (0.0010) [2023-12-26 22:24:12,946][105692] Updated weights for policy 0, policy_version 965779 (0.0010) [2023-12-26 22:24:13,006][105692] Updated weights for policy 0, policy_version 965789 (0.0009) [2023-12-26 22:24:13,069][105692] Updated weights for policy 0, policy_version 965799 (0.0009) [2023-12-26 22:24:13,425][105620] Updated weights for policy 1, policy_version 965874 (0.0009) [2023-12-26 22:24:13,488][105620] Updated weights for policy 1, policy_version 965884 (0.0009) [2023-12-26 22:24:13,550][105620] Updated weights for policy 1, policy_version 965894 (0.0008) [2023-12-26 22:24:13,794][105692] Updated weights for policy 0, policy_version 965809 (0.0010) [2023-12-26 22:24:13,848][105692] Updated weights for policy 0, policy_version 965820 (0.0010) [2023-12-26 22:24:13,919][105692] Updated weights for policy 0, policy_version 965830 (0.0010) [2023-12-26 22:24:14,139][105620] Updated weights for policy 1, policy_version 965904 (0.0009) [2023-12-26 22:24:14,194][105620] Updated weights for policy 1, policy_version 965914 (0.0009) [2023-12-26 22:24:14,252][105620] Updated weights for policy 1, policy_version 965924 (0.0009) [2023-12-26 22:24:14,629][105692] Updated weights for policy 0, policy_version 965840 (0.0006) [2023-12-26 22:24:14,699][105692] Updated weights for policy 0, policy_version 965850 (0.0005) [2023-12-26 22:24:14,766][105692] Updated weights for policy 0, policy_version 965860 (0.0009) [2023-12-26 22:24:15,081][105620] Updated weights for policy 1, policy_version 965934 (0.0009) [2023-12-26 22:24:15,151][105620] Updated weights for policy 1, policy_version 965944 (0.0008) [2023-12-26 22:24:15,214][105620] Updated weights for policy 1, policy_version 965954 (0.0009) [2023-12-26 22:24:15,374][105692] Updated weights for policy 0, policy_version 965870 (0.0008) [2023-12-26 22:24:15,435][105692] Updated weights for policy 0, policy_version 965880 (0.0009) [2023-12-26 22:24:15,504][105692] Updated weights for policy 0, policy_version 965890 (0.0006) [2023-12-26 22:24:15,889][105620] Updated weights for policy 1, policy_version 965964 (0.0010) [2023-12-26 22:24:15,938][105620] Updated weights for policy 1, policy_version 965974 (0.0010) [2023-12-26 22:24:15,987][105620] Updated weights for policy 1, policy_version 965984 (0.0010) [2023-12-26 22:24:16,056][105692] Updated weights for policy 0, policy_version 965900 (0.0005) [2023-12-26 22:24:16,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19327.6). Total num frames: 494632960. Throughput: 0: 9782.6, 1: 9661.3. Samples: 494600560. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-26 22:24:16,062][104569] Avg episode reward: [(0, '7501.426'), (1, '8839.009')] [2023-12-26 22:24:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000965992_247324672.pth... [2023-12-26 22:24:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000964872_247037952.pth [2023-12-26 22:24:16,115][105692] Updated weights for policy 0, policy_version 965910 (0.0005) [2023-12-26 22:24:16,172][105692] Updated weights for policy 0, policy_version 965920 (0.0005) [2023-12-26 22:24:16,219][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000965928_247316480.pth... [2023-12-26 22:24:16,223][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000964776_247021568.pth [2023-12-26 22:24:16,690][105692] Updated weights for policy 0, policy_version 965930 (0.0006) [2023-12-26 22:24:16,749][105620] Updated weights for policy 1, policy_version 965994 (0.0010) [2023-12-26 22:24:16,758][105692] Updated weights for policy 0, policy_version 965940 (0.0007) [2023-12-26 22:24:16,807][105620] Updated weights for policy 1, policy_version 966004 (0.0010) [2023-12-26 22:24:16,817][105692] Updated weights for policy 0, policy_version 965950 (0.0010) [2023-12-26 22:24:16,871][105620] Updated weights for policy 1, policy_version 966014 (0.0010) [2023-12-26 22:24:16,876][105692] Updated weights for policy 0, policy_version 965960 (0.0010) [2023-12-26 22:24:16,931][105620] Updated weights for policy 1, policy_version 966024 (0.0009) [2023-12-26 22:24:17,479][105692] Updated weights for policy 0, policy_version 965970 (0.0011) [2023-12-26 22:24:17,546][105692] Updated weights for policy 0, policy_version 965980 (0.0006) [2023-12-26 22:24:17,593][105620] Updated weights for policy 1, policy_version 966034 (0.0008) [2023-12-26 22:24:17,609][105692] Updated weights for policy 0, policy_version 965990 (0.0006) [2023-12-26 22:24:17,644][105620] Updated weights for policy 1, policy_version 966044 (0.0008) [2023-12-26 22:24:17,701][105620] Updated weights for policy 1, policy_version 966054 (0.0008) [2023-12-26 22:24:18,293][105692] Updated weights for policy 0, policy_version 966000 (0.0010) [2023-12-26 22:24:18,358][105692] Updated weights for policy 0, policy_version 966010 (0.0009) [2023-12-26 22:24:18,365][105620] Updated weights for policy 1, policy_version 966064 (0.0008) [2023-12-26 22:24:18,419][105692] Updated weights for policy 0, policy_version 966020 (0.0009) [2023-12-26 22:24:18,429][105620] Updated weights for policy 1, policy_version 966074 (0.0009) [2023-12-26 22:24:18,484][105620] Updated weights for policy 1, policy_version 966084 (0.0008) [2023-12-26 22:24:19,174][105692] Updated weights for policy 0, policy_version 966030 (0.0008) [2023-12-26 22:24:19,193][105620] Updated weights for policy 1, policy_version 966094 (0.0010) [2023-12-26 22:24:19,236][105692] Updated weights for policy 0, policy_version 966040 (0.0007) [2023-12-26 22:24:19,255][105620] Updated weights for policy 1, policy_version 966104 (0.0010) [2023-12-26 22:24:19,300][105692] Updated weights for policy 0, policy_version 966050 (0.0007) [2023-12-26 22:24:19,318][105620] Updated weights for policy 1, policy_version 966114 (0.0011) [2023-12-26 22:24:20,066][105692] Updated weights for policy 0, policy_version 966060 (0.0006) [2023-12-26 22:24:20,123][105620] Updated weights for policy 1, policy_version 966124 (0.0011) [2023-12-26 22:24:20,125][105692] Updated weights for policy 0, policy_version 966070 (0.0006) [2023-12-26 22:24:20,185][105692] Updated weights for policy 0, policy_version 966080 (0.0006) [2023-12-26 22:24:20,190][105620] Updated weights for policy 1, policy_version 966134 (0.0011) [2023-12-26 22:24:20,253][105620] Updated weights for policy 1, policy_version 966144 (0.0011) [2023-12-26 22:24:20,939][105692] Updated weights for policy 0, policy_version 966090 (0.0007) [2023-12-26 22:24:20,999][105692] Updated weights for policy 0, policy_version 966100 (0.0009) [2023-12-26 22:24:21,019][105620] Updated weights for policy 1, policy_version 966154 (0.0010) [2023-12-26 22:24:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19299.8). Total num frames: 494723072. Throughput: 0: 9773.7, 1: 9714.9. Samples: 494721180. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-26 22:24:21,063][104569] Avg episode reward: [(0, '7835.592'), (1, '8814.053')] [2023-12-26 22:24:21,064][105692] Updated weights for policy 0, policy_version 966110 (0.0008) [2023-12-26 22:24:21,090][105620] Updated weights for policy 1, policy_version 966164 (0.0007) [2023-12-26 22:24:21,135][105692] Updated weights for policy 0, policy_version 966120 (0.0009) [2023-12-26 22:24:21,155][105620] Updated weights for policy 1, policy_version 966174 (0.0008) [2023-12-26 22:24:21,207][105620] Updated weights for policy 1, policy_version 966184 (0.0008) [2023-12-26 22:24:21,847][105692] Updated weights for policy 0, policy_version 966130 (0.0008) [2023-12-26 22:24:21,913][105692] Updated weights for policy 0, policy_version 966140 (0.0008) [2023-12-26 22:24:21,969][105692] Updated weights for policy 0, policy_version 966150 (0.0006) [2023-12-26 22:24:22,011][105620] Updated weights for policy 1, policy_version 966194 (0.0008) [2023-12-26 22:24:22,082][105620] Updated weights for policy 1, policy_version 966204 (0.0010) [2023-12-26 22:24:22,153][105620] Updated weights for policy 1, policy_version 966214 (0.0010) [2023-12-26 22:24:22,607][105692] Updated weights for policy 0, policy_version 966160 (0.0008) [2023-12-26 22:24:22,655][105692] Updated weights for policy 0, policy_version 966170 (0.0009) [2023-12-26 22:24:22,707][105692] Updated weights for policy 0, policy_version 966180 (0.0009) [2023-12-26 22:24:22,897][105620] Updated weights for policy 1, policy_version 966224 (0.0008) [2023-12-26 22:24:22,966][105620] Updated weights for policy 1, policy_version 966234 (0.0008) [2023-12-26 22:24:23,029][105620] Updated weights for policy 1, policy_version 966244 (0.0008) [2023-12-26 22:24:23,416][105692] Updated weights for policy 0, policy_version 966190 (0.0009) [2023-12-26 22:24:23,478][105692] Updated weights for policy 0, policy_version 966200 (0.0009) [2023-12-26 22:24:23,542][105692] Updated weights for policy 0, policy_version 966210 (0.0010) [2023-12-26 22:24:23,797][105620] Updated weights for policy 1, policy_version 966254 (0.0009) [2023-12-26 22:24:23,854][105620] Updated weights for policy 1, policy_version 966264 (0.0008) [2023-12-26 22:24:23,913][105620] Updated weights for policy 1, policy_version 966274 (0.0008) [2023-12-26 22:24:24,238][105692] Updated weights for policy 0, policy_version 966220 (0.0008) [2023-12-26 22:24:24,284][105692] Updated weights for policy 0, policy_version 966230 (0.0005) [2023-12-26 22:24:24,331][105692] Updated weights for policy 0, policy_version 966240 (0.0005) [2023-12-26 22:24:24,702][105620] Updated weights for policy 1, policy_version 966284 (0.0009) [2023-12-26 22:24:24,758][105620] Updated weights for policy 1, policy_version 966294 (0.0006) [2023-12-26 22:24:24,810][105620] Updated weights for policy 1, policy_version 966304 (0.0005) [2023-12-26 22:24:24,950][105692] Updated weights for policy 0, policy_version 966250 (0.0005) [2023-12-26 22:24:25,001][105692] Updated weights for policy 0, policy_version 966260 (0.0005) [2023-12-26 22:24:25,068][105692] Updated weights for policy 0, policy_version 966270 (0.0006) [2023-12-26 22:24:25,124][105692] Updated weights for policy 0, policy_version 966280 (0.0007) [2023-12-26 22:24:25,416][105620] Updated weights for policy 1, policy_version 966314 (0.0006) [2023-12-26 22:24:25,482][105620] Updated weights for policy 1, policy_version 966324 (0.0008) [2023-12-26 22:24:25,540][105620] Updated weights for policy 1, policy_version 966334 (0.0008) [2023-12-26 22:24:25,596][105620] Updated weights for policy 1, policy_version 966344 (0.0008) [2023-12-26 22:24:25,817][105692] Updated weights for policy 0, policy_version 966290 (0.0011) [2023-12-26 22:24:25,879][105692] Updated weights for policy 0, policy_version 966300 (0.0011) [2023-12-26 22:24:25,941][105692] Updated weights for policy 0, policy_version 966310 (0.0010) [2023-12-26 22:24:26,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19355.3). Total num frames: 494829568. Throughput: 0: 9843.4, 1: 9556.9. Samples: 494836056. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-26 22:24:26,063][104569] Avg episode reward: [(0, '7309.154'), (1, '8904.074')] [2023-12-26 22:24:26,367][105620] Updated weights for policy 1, policy_version 966354 (0.0005) [2023-12-26 22:24:26,432][105620] Updated weights for policy 1, policy_version 966364 (0.0006) [2023-12-26 22:24:26,498][105620] Updated weights for policy 1, policy_version 966374 (0.0008) [2023-12-26 22:24:26,649][105692] Updated weights for policy 0, policy_version 966320 (0.0006) [2023-12-26 22:24:26,697][105692] Updated weights for policy 0, policy_version 966330 (0.0005) [2023-12-26 22:24:26,753][105692] Updated weights for policy 0, policy_version 966340 (0.0005) [2023-12-26 22:24:27,266][105692] Updated weights for policy 0, policy_version 966350 (0.0006) [2023-12-26 22:24:27,317][105692] Updated weights for policy 0, policy_version 966360 (0.0005) [2023-12-26 22:24:27,318][105620] Updated weights for policy 1, policy_version 966384 (0.0009) [2023-12-26 22:24:27,366][105620] Updated weights for policy 1, policy_version 966394 (0.0008) [2023-12-26 22:24:27,377][105692] Updated weights for policy 0, policy_version 966370 (0.0005) [2023-12-26 22:24:27,413][105620] Updated weights for policy 1, policy_version 966404 (0.0008) [2023-12-26 22:24:27,907][105692] Updated weights for policy 0, policy_version 966380 (0.0006) [2023-12-26 22:24:27,957][105692] Updated weights for policy 0, policy_version 966390 (0.0005) [2023-12-26 22:24:28,007][105692] Updated weights for policy 0, policy_version 966400 (0.0006) [2023-12-26 22:24:28,307][105620] Updated weights for policy 1, policy_version 966415 (0.0010) [2023-12-26 22:24:28,367][105620] Updated weights for policy 1, policy_version 966425 (0.0007) [2023-12-26 22:24:28,430][105620] Updated weights for policy 1, policy_version 966435 (0.0008) [2023-12-26 22:24:28,665][105692] Updated weights for policy 0, policy_version 966410 (0.0008) [2023-12-26 22:24:28,721][105692] Updated weights for policy 0, policy_version 966420 (0.0009) [2023-12-26 22:24:28,769][105692] Updated weights for policy 0, policy_version 966430 (0.0009) [2023-12-26 22:24:28,823][105692] Updated weights for policy 0, policy_version 966440 (0.0007) [2023-12-26 22:24:29,239][105620] Updated weights for policy 1, policy_version 966445 (0.0007) [2023-12-26 22:24:29,302][105620] Updated weights for policy 1, policy_version 966455 (0.0007) [2023-12-26 22:24:29,360][105620] Updated weights for policy 1, policy_version 966465 (0.0008) [2023-12-26 22:24:29,600][105692] Updated weights for policy 0, policy_version 966450 (0.0008) [2023-12-26 22:24:29,655][105692] Updated weights for policy 0, policy_version 966460 (0.0008) [2023-12-26 22:24:29,713][105692] Updated weights for policy 0, policy_version 966470 (0.0007) [2023-12-26 22:24:30,004][105620] Updated weights for policy 1, policy_version 966475 (0.0007) [2023-12-26 22:24:30,070][105620] Updated weights for policy 1, policy_version 966485 (0.0006) [2023-12-26 22:24:30,138][105620] Updated weights for policy 1, policy_version 966495 (0.0005) [2023-12-26 22:24:30,473][105692] Updated weights for policy 0, policy_version 966480 (0.0006) [2023-12-26 22:24:30,540][105692] Updated weights for policy 0, policy_version 966490 (0.0008) [2023-12-26 22:24:30,607][105692] Updated weights for policy 0, policy_version 966500 (0.0007) [2023-12-26 22:24:30,725][105620] Updated weights for policy 1, policy_version 966505 (0.0006) [2023-12-26 22:24:30,782][105620] Updated weights for policy 1, policy_version 966515 (0.0006) [2023-12-26 22:24:30,843][105620] Updated weights for policy 1, policy_version 966525 (0.0008) [2023-12-26 22:24:30,904][105620] Updated weights for policy 1, policy_version 966535 (0.0007) [2023-12-26 22:24:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.2, 300 sec: 19327.6). Total num frames: 494927872. Throughput: 0: 9980.1, 1: 9496.9. Samples: 494895840. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-26 22:24:31,063][104569] Avg episode reward: [(0, '7571.650'), (1, '9174.124')] [2023-12-26 22:24:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000966504_247463936.pth... [2023-12-26 22:24:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000966536_247463936.pth... [2023-12-26 22:24:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000965320_247160832.pth [2023-12-26 22:24:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000965416_247177216.pth [2023-12-26 22:24:31,360][105692] Updated weights for policy 0, policy_version 966510 (0.0007) [2023-12-26 22:24:31,414][105692] Updated weights for policy 0, policy_version 966520 (0.0008) [2023-12-26 22:24:31,475][105692] Updated weights for policy 0, policy_version 966530 (0.0008) [2023-12-26 22:24:31,580][105620] Updated weights for policy 1, policy_version 966545 (0.0006) [2023-12-26 22:24:31,644][105620] Updated weights for policy 1, policy_version 966555 (0.0007) [2023-12-26 22:24:31,709][105620] Updated weights for policy 1, policy_version 966565 (0.0009) [2023-12-26 22:24:32,281][105692] Updated weights for policy 0, policy_version 966540 (0.0008) [2023-12-26 22:24:32,341][105692] Updated weights for policy 0, policy_version 966550 (0.0007) [2023-12-26 22:24:32,342][105620] Updated weights for policy 1, policy_version 966575 (0.0010) [2023-12-26 22:24:32,403][105620] Updated weights for policy 1, policy_version 966585 (0.0009) [2023-12-26 22:24:32,406][105692] Updated weights for policy 0, policy_version 966560 (0.0007) [2023-12-26 22:24:32,459][105620] Updated weights for policy 1, policy_version 966595 (0.0008) [2023-12-26 22:24:33,148][105692] Updated weights for policy 0, policy_version 966570 (0.0006) [2023-12-26 22:24:33,195][105692] Updated weights for policy 0, policy_version 966580 (0.0008) [2023-12-26 22:24:33,200][105620] Updated weights for policy 1, policy_version 966605 (0.0007) [2023-12-26 22:24:33,250][105692] Updated weights for policy 0, policy_version 966590 (0.0009) [2023-12-26 22:24:33,251][105620] Updated weights for policy 1, policy_version 966615 (0.0005) [2023-12-26 22:24:33,296][105620] Updated weights for policy 1, policy_version 966625 (0.0005) [2023-12-26 22:24:33,307][105692] Updated weights for policy 0, policy_version 966600 (0.0010) [2023-12-26 22:24:33,863][105620] Updated weights for policy 1, policy_version 966635 (0.0006) [2023-12-26 22:24:33,909][105620] Updated weights for policy 1, policy_version 966645 (0.0005) [2023-12-26 22:24:33,956][105620] Updated weights for policy 1, policy_version 966655 (0.0009) [2023-12-26 22:24:34,165][105692] Updated weights for policy 0, policy_version 966610 (0.0010) [2023-12-26 22:24:34,223][105692] Updated weights for policy 0, policy_version 966620 (0.0006) [2023-12-26 22:24:34,279][105692] Updated weights for policy 0, policy_version 966630 (0.0006) [2023-12-26 22:24:34,644][105620] Updated weights for policy 1, policy_version 966665 (0.0009) [2023-12-26 22:24:34,710][105620] Updated weights for policy 1, policy_version 966675 (0.0010) [2023-12-26 22:24:34,766][105620] Updated weights for policy 1, policy_version 966685 (0.0011) [2023-12-26 22:24:34,818][105620] Updated weights for policy 1, policy_version 966695 (0.0010) [2023-12-26 22:24:34,916][105692] Updated weights for policy 0, policy_version 966640 (0.0007) [2023-12-26 22:24:34,974][105692] Updated weights for policy 0, policy_version 966650 (0.0006) [2023-12-26 22:24:35,027][105692] Updated weights for policy 0, policy_version 966660 (0.0006) [2023-12-26 22:24:35,509][105620] Updated weights for policy 1, policy_version 966705 (0.0009) [2023-12-26 22:24:35,574][105620] Updated weights for policy 1, policy_version 966715 (0.0005) [2023-12-26 22:24:35,586][105692] Updated weights for policy 0, policy_version 966670 (0.0008) [2023-12-26 22:24:35,632][105620] Updated weights for policy 1, policy_version 966725 (0.0006) [2023-12-26 22:24:35,645][105692] Updated weights for policy 0, policy_version 966680 (0.0009) [2023-12-26 22:24:35,709][105692] Updated weights for policy 0, policy_version 966690 (0.0008) [2023-12-26 22:24:36,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19355.3). Total num frames: 495026176. Throughput: 0: 9917.2, 1: 9564.1. Samples: 495012096. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-26 22:24:36,063][104569] Avg episode reward: [(0, '7680.467'), (1, '8998.046')] [2023-12-26 22:24:36,365][105620] Updated weights for policy 1, policy_version 966735 (0.0010) [2023-12-26 22:24:36,446][105620] Updated weights for policy 1, policy_version 966745 (0.0010) [2023-12-26 22:24:36,467][105692] Updated weights for policy 0, policy_version 966700 (0.0008) [2023-12-26 22:24:36,511][105620] Updated weights for policy 1, policy_version 966755 (0.0008) [2023-12-26 22:24:36,526][105692] Updated weights for policy 0, policy_version 966710 (0.0006) [2023-12-26 22:24:36,587][105692] Updated weights for policy 0, policy_version 966720 (0.0008) [2023-12-26 22:24:37,286][105620] Updated weights for policy 1, policy_version 966765 (0.0006) [2023-12-26 22:24:37,336][105692] Updated weights for policy 0, policy_version 966730 (0.0010) [2023-12-26 22:24:37,350][105620] Updated weights for policy 1, policy_version 966775 (0.0007) [2023-12-26 22:24:37,397][105692] Updated weights for policy 0, policy_version 966740 (0.0008) [2023-12-26 22:24:37,410][105620] Updated weights for policy 1, policy_version 966785 (0.0006) [2023-12-26 22:24:37,451][105692] Updated weights for policy 0, policy_version 966750 (0.0009) [2023-12-26 22:24:37,502][105692] Updated weights for policy 0, policy_version 966760 (0.0009) [2023-12-26 22:24:37,940][105620] Updated weights for policy 1, policy_version 966795 (0.0005) [2023-12-26 22:24:37,992][105620] Updated weights for policy 1, policy_version 966805 (0.0006) [2023-12-26 22:24:38,039][105620] Updated weights for policy 1, policy_version 966815 (0.0006) [2023-12-26 22:24:38,430][105692] Updated weights for policy 0, policy_version 966770 (0.0008) [2023-12-26 22:24:38,490][105692] Updated weights for policy 0, policy_version 966780 (0.0008) [2023-12-26 22:24:38,551][105692] Updated weights for policy 0, policy_version 966790 (0.0008) [2023-12-26 22:24:38,703][105620] Updated weights for policy 1, policy_version 966825 (0.0007) [2023-12-26 22:24:38,756][105620] Updated weights for policy 1, policy_version 966835 (0.0011) [2023-12-26 22:24:38,815][105620] Updated weights for policy 1, policy_version 966845 (0.0010) [2023-12-26 22:24:38,874][105620] Updated weights for policy 1, policy_version 966855 (0.0010) [2023-12-26 22:24:39,326][105692] Updated weights for policy 0, policy_version 966800 (0.0009) [2023-12-26 22:24:39,386][105692] Updated weights for policy 0, policy_version 966810 (0.0009) [2023-12-26 22:24:39,445][105692] Updated weights for policy 0, policy_version 966820 (0.0008) [2023-12-26 22:24:39,618][105620] Updated weights for policy 1, policy_version 966865 (0.0010) [2023-12-26 22:24:39,677][105620] Updated weights for policy 1, policy_version 966875 (0.0011) [2023-12-26 22:24:39,752][105620] Updated weights for policy 1, policy_version 966885 (0.0011) [2023-12-26 22:24:40,282][105692] Updated weights for policy 0, policy_version 966830 (0.0010) [2023-12-26 22:24:40,338][105692] Updated weights for policy 0, policy_version 966840 (0.0008) [2023-12-26 22:24:40,376][105620] Updated weights for policy 1, policy_version 966895 (0.0009) [2023-12-26 22:24:40,399][105692] Updated weights for policy 0, policy_version 966850 (0.0007) [2023-12-26 22:24:40,444][105620] Updated weights for policy 1, policy_version 966905 (0.0006) [2023-12-26 22:24:40,496][105620] Updated weights for policy 1, policy_version 966915 (0.0005) [2023-12-26 22:24:41,060][105620] Updated weights for policy 1, policy_version 966925 (0.0007) [2023-12-26 22:24:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19299.8). Total num frames: 495116288. Throughput: 0: 9849.8, 1: 9745.8. Samples: 495129104. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-26 22:24:41,062][104569] Avg episode reward: [(0, '7236.540'), (1, '8558.440')] [2023-12-26 22:24:41,089][105692] Updated weights for policy 0, policy_version 966860 (0.0009) [2023-12-26 22:24:41,126][105620] Updated weights for policy 1, policy_version 966935 (0.0008) [2023-12-26 22:24:41,161][105692] Updated weights for policy 0, policy_version 966870 (0.0011) [2023-12-26 22:24:41,187][105620] Updated weights for policy 1, policy_version 966945 (0.0007) [2023-12-26 22:24:41,225][105692] Updated weights for policy 0, policy_version 966880 (0.0008) [2023-12-26 22:24:41,967][105620] Updated weights for policy 1, policy_version 966955 (0.0006) [2023-12-26 22:24:42,027][105620] Updated weights for policy 1, policy_version 966965 (0.0008) [2023-12-26 22:24:42,058][105692] Updated weights for policy 0, policy_version 966890 (0.0009) [2023-12-26 22:24:42,088][105620] Updated weights for policy 1, policy_version 966975 (0.0006) [2023-12-26 22:24:42,121][105692] Updated weights for policy 0, policy_version 966900 (0.0010) [2023-12-26 22:24:42,184][105692] Updated weights for policy 0, policy_version 966910 (0.0011) [2023-12-26 22:24:42,236][105692] Updated weights for policy 0, policy_version 966920 (0.0010) [2023-12-26 22:24:42,879][105620] Updated weights for policy 1, policy_version 966985 (0.0008) [2023-12-26 22:24:42,944][105620] Updated weights for policy 1, policy_version 966995 (0.0006) [2023-12-26 22:24:42,989][105692] Updated weights for policy 0, policy_version 966930 (0.0007) [2023-12-26 22:24:42,998][105620] Updated weights for policy 1, policy_version 967005 (0.0008) [2023-12-26 22:24:43,039][105692] Updated weights for policy 0, policy_version 966940 (0.0008) [2023-12-26 22:24:43,059][105620] Updated weights for policy 1, policy_version 967015 (0.0008) [2023-12-26 22:24:43,087][105692] Updated weights for policy 0, policy_version 966950 (0.0008) [2023-12-26 22:24:43,768][105620] Updated weights for policy 1, policy_version 967025 (0.0006) [2023-12-26 22:24:43,826][105620] Updated weights for policy 1, policy_version 967035 (0.0005) [2023-12-26 22:24:43,826][105692] Updated weights for policy 0, policy_version 966960 (0.0006) [2023-12-26 22:24:43,885][105692] Updated weights for policy 0, policy_version 966970 (0.0005) [2023-12-26 22:24:43,893][105620] Updated weights for policy 1, policy_version 967045 (0.0007) [2023-12-26 22:24:43,943][105692] Updated weights for policy 0, policy_version 966980 (0.0005) [2023-12-26 22:24:44,521][105620] Updated weights for policy 1, policy_version 967055 (0.0010) [2023-12-26 22:24:44,539][105692] Updated weights for policy 0, policy_version 966990 (0.0007) [2023-12-26 22:24:44,579][105620] Updated weights for policy 1, policy_version 967065 (0.0010) [2023-12-26 22:24:44,593][105692] Updated weights for policy 0, policy_version 967000 (0.0010) [2023-12-26 22:24:44,622][105585] KL-divergence is very high: 190.6629 [2023-12-26 22:24:44,634][105585] KL-divergence is very high: 130.4718 [2023-12-26 22:24:44,638][105620] Updated weights for policy 1, policy_version 967075 (0.0010) [2023-12-26 22:24:44,640][105585] KL-divergence is very high: 183.2552 [2023-12-26 22:24:44,651][105585] KL-divergence is very high: 118.8264 [2023-12-26 22:24:44,652][105692] Updated weights for policy 0, policy_version 967010 (0.0006) [2023-12-26 22:24:44,668][105585] KL-divergence is very high: 317.1561 [2023-12-26 22:24:44,682][105585] KL-divergence is very high: 172.5543 [2023-12-26 22:24:45,288][105692] Updated weights for policy 0, policy_version 967020 (0.0008) [2023-12-26 22:24:45,310][105585] KL-divergence is very high: 101.8033 [2023-12-26 22:24:45,355][105692] Updated weights for policy 0, policy_version 967030 (0.0006) [2023-12-26 22:24:45,393][105620] Updated weights for policy 1, policy_version 967085 (0.0010) [2023-12-26 22:24:45,418][105692] Updated weights for policy 0, policy_version 967040 (0.0006) [2023-12-26 22:24:45,456][105620] Updated weights for policy 1, policy_version 967095 (0.0010) [2023-12-26 22:24:45,519][105620] Updated weights for policy 1, policy_version 967105 (0.0010) [2023-12-26 22:24:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19355.3). Total num frames: 495214592. Throughput: 0: 9759.5, 1: 9733.1. Samples: 495185400. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-26 22:24:46,063][104569] Avg episode reward: [(0, '6865.662'), (1, '8291.999')] [2023-12-26 22:24:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000967112_247611392.pth... [2023-12-26 22:24:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000967048_247603200.pth... [2023-12-26 22:24:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000965992_247324672.pth [2023-12-26 22:24:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000965928_247316480.pth [2023-12-26 22:24:46,121][105692] Updated weights for policy 0, policy_version 967050 (0.0006) [2023-12-26 22:24:46,167][105692] Updated weights for policy 0, policy_version 967060 (0.0008) [2023-12-26 22:24:46,195][105620] Updated weights for policy 1, policy_version 967115 (0.0010) [2023-12-26 22:24:46,214][105692] Updated weights for policy 0, policy_version 967070 (0.0008) [2023-12-26 22:24:46,241][105620] Updated weights for policy 1, policy_version 967125 (0.0006) [2023-12-26 22:24:46,268][105692] Updated weights for policy 0, policy_version 967080 (0.0007) [2023-12-26 22:24:46,294][105620] Updated weights for policy 1, policy_version 967135 (0.0006) [2023-12-26 22:24:46,938][105620] Updated weights for policy 1, policy_version 967145 (0.0005) [2023-12-26 22:24:46,975][105692] Updated weights for policy 0, policy_version 967090 (0.0009) [2023-12-26 22:24:47,002][105620] Updated weights for policy 1, policy_version 967155 (0.0005) [2023-12-26 22:24:47,025][105692] Updated weights for policy 0, policy_version 967100 (0.0009) [2023-12-26 22:24:47,069][105620] Updated weights for policy 1, policy_version 967165 (0.0005) [2023-12-26 22:24:47,077][105692] Updated weights for policy 0, policy_version 967110 (0.0008) [2023-12-26 22:24:47,140][105620] Updated weights for policy 1, policy_version 967175 (0.0006) [2023-12-26 22:24:47,738][105692] Updated weights for policy 0, policy_version 967120 (0.0009) [2023-12-26 22:24:47,801][105692] Updated weights for policy 0, policy_version 967130 (0.0009) [2023-12-26 22:24:47,856][105692] Updated weights for policy 0, policy_version 967140 (0.0006) [2023-12-26 22:24:47,867][105620] Updated weights for policy 1, policy_version 967185 (0.0008) [2023-12-26 22:24:47,918][105620] Updated weights for policy 1, policy_version 967195 (0.0007) [2023-12-26 22:24:47,975][105620] Updated weights for policy 1, policy_version 967205 (0.0007) [2023-12-26 22:24:48,649][105692] Updated weights for policy 0, policy_version 967150 (0.0009) [2023-12-26 22:24:48,707][105692] Updated weights for policy 0, policy_version 967160 (0.0009) [2023-12-26 22:24:48,733][105620] Updated weights for policy 1, policy_version 967215 (0.0006) [2023-12-26 22:24:48,771][105692] Updated weights for policy 0, policy_version 967170 (0.0008) [2023-12-26 22:24:48,794][105620] Updated weights for policy 1, policy_version 967225 (0.0006) [2023-12-26 22:24:48,846][105620] Updated weights for policy 1, policy_version 967235 (0.0008) [2023-12-26 22:24:49,404][105692] Updated weights for policy 0, policy_version 967180 (0.0008) [2023-12-26 22:24:49,470][105692] Updated weights for policy 0, policy_version 967190 (0.0009) [2023-12-26 22:24:49,528][105692] Updated weights for policy 0, policy_version 967200 (0.0009) [2023-12-26 22:24:49,647][105620] Updated weights for policy 1, policy_version 967245 (0.0010) [2023-12-26 22:24:49,696][105620] Updated weights for policy 1, policy_version 967255 (0.0008) [2023-12-26 22:24:49,762][105620] Updated weights for policy 1, policy_version 967265 (0.0006) [2023-12-26 22:24:50,362][105692] Updated weights for policy 0, policy_version 967210 (0.0009) [2023-12-26 22:24:50,399][105620] Updated weights for policy 1, policy_version 967275 (0.0006) [2023-12-26 22:24:50,421][105692] Updated weights for policy 0, policy_version 967220 (0.0008) [2023-12-26 22:24:50,456][105620] Updated weights for policy 1, policy_version 967285 (0.0007) [2023-12-26 22:24:50,478][105692] Updated weights for policy 0, policy_version 967230 (0.0007) [2023-12-26 22:24:50,509][105620] Updated weights for policy 1, policy_version 967295 (0.0006) [2023-12-26 22:24:50,531][105692] Updated weights for policy 0, policy_version 967240 (0.0008) [2023-12-26 22:24:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19327.6). Total num frames: 495312896. Throughput: 0: 9810.5, 1: 9742.8. Samples: 495304296. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-26 22:24:51,063][104569] Avg episode reward: [(0, '7395.243'), (1, '8752.229')] [2023-12-26 22:24:51,333][105620] Updated weights for policy 1, policy_version 967305 (0.0009) [2023-12-26 22:24:51,356][105692] Updated weights for policy 0, policy_version 967250 (0.0007) [2023-12-26 22:24:51,405][105620] Updated weights for policy 1, policy_version 967315 (0.0008) [2023-12-26 22:24:51,424][105692] Updated weights for policy 0, policy_version 967260 (0.0008) [2023-12-26 22:24:51,472][105620] Updated weights for policy 1, policy_version 967325 (0.0008) [2023-12-26 22:24:51,485][105692] Updated weights for policy 0, policy_version 967270 (0.0006) [2023-12-26 22:24:51,535][105620] Updated weights for policy 1, policy_version 967335 (0.0008) [2023-12-26 22:24:52,140][105692] Updated weights for policy 0, policy_version 967280 (0.0008) [2023-12-26 22:24:52,192][105692] Updated weights for policy 0, policy_version 967290 (0.0009) [2023-12-26 22:24:52,252][105692] Updated weights for policy 0, policy_version 967300 (0.0009) [2023-12-26 22:24:52,352][105620] Updated weights for policy 1, policy_version 967345 (0.0008) [2023-12-26 22:24:52,412][105620] Updated weights for policy 1, policy_version 967355 (0.0008) [2023-12-26 22:24:52,467][105620] Updated weights for policy 1, policy_version 967365 (0.0008) [2023-12-26 22:24:52,950][105692] Updated weights for policy 0, policy_version 967310 (0.0007) [2023-12-26 22:24:53,011][105692] Updated weights for policy 0, policy_version 967320 (0.0006) [2023-12-26 22:24:53,071][105692] Updated weights for policy 0, policy_version 967330 (0.0008) [2023-12-26 22:24:53,310][105620] Updated weights for policy 1, policy_version 967375 (0.0008) [2023-12-26 22:24:53,364][105620] Updated weights for policy 1, policy_version 967385 (0.0010) [2023-12-26 22:24:53,425][105620] Updated weights for policy 1, policy_version 967395 (0.0007) [2023-12-26 22:24:53,779][105692] Updated weights for policy 0, policy_version 967340 (0.0007) [2023-12-26 22:24:53,836][105692] Updated weights for policy 0, policy_version 967350 (0.0008) [2023-12-26 22:24:53,891][105692] Updated weights for policy 0, policy_version 967361 (0.0010) [2023-12-26 22:24:54,057][105620] Updated weights for policy 1, policy_version 967405 (0.0005) [2023-12-26 22:24:54,115][105620] Updated weights for policy 1, policy_version 967415 (0.0006) [2023-12-26 22:24:54,174][105620] Updated weights for policy 1, policy_version 967425 (0.0005) [2023-12-26 22:24:54,739][105620] Updated weights for policy 1, policy_version 967435 (0.0009) [2023-12-26 22:24:54,764][105692] Updated weights for policy 0, policy_version 967371 (0.0010) [2023-12-26 22:24:54,798][105620] Updated weights for policy 1, policy_version 967445 (0.0010) [2023-12-26 22:24:54,812][105692] Updated weights for policy 0, policy_version 967381 (0.0006) [2023-12-26 22:24:54,860][105620] Updated weights for policy 1, policy_version 967455 (0.0010) [2023-12-26 22:24:54,862][105692] Updated weights for policy 0, policy_version 967391 (0.0008) [2023-12-26 22:24:55,495][105620] Updated weights for policy 1, policy_version 967465 (0.0010) [2023-12-26 22:24:55,544][105620] Updated weights for policy 1, policy_version 967475 (0.0005) [2023-12-26 22:24:55,607][105620] Updated weights for policy 1, policy_version 967485 (0.0006) [2023-12-26 22:24:55,662][105620] Updated weights for policy 1, policy_version 967495 (0.0008) [2023-12-26 22:24:55,722][105692] Updated weights for policy 0, policy_version 967401 (0.0008) [2023-12-26 22:24:55,774][105692] Updated weights for policy 0, policy_version 967411 (0.0008) [2023-12-26 22:24:55,822][105692] Updated weights for policy 0, policy_version 967421 (0.0008) [2023-12-26 22:24:55,825][105585] KL-divergence is very high: 139.0465 [2023-12-26 22:24:55,861][105585] KL-divergence is very high: 143.1797 [2023-12-26 22:24:55,867][105692] Updated weights for policy 0, policy_version 967431 (0.0008) [2023-12-26 22:24:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19327.6). Total num frames: 495411200. Throughput: 0: 9708.1, 1: 9725.1. Samples: 495417848. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-26 22:24:56,062][104569] Avg episode reward: [(0, '7241.021'), (1, '8748.053')] [2023-12-26 22:24:56,339][105620] Updated weights for policy 1, policy_version 967505 (0.0010) [2023-12-26 22:24:56,388][105620] Updated weights for policy 1, policy_version 967515 (0.0005) [2023-12-26 22:24:56,439][105620] Updated weights for policy 1, policy_version 967525 (0.0006) [2023-12-26 22:24:56,555][105692] Updated weights for policy 0, policy_version 967441 (0.0006) [2023-12-26 22:24:56,619][105692] Updated weights for policy 0, policy_version 967451 (0.0006) [2023-12-26 22:24:56,679][105692] Updated weights for policy 0, policy_version 967461 (0.0009) [2023-12-26 22:24:57,032][105620] Updated weights for policy 1, policy_version 967535 (0.0008) [2023-12-26 22:24:57,097][105620] Updated weights for policy 1, policy_version 967545 (0.0009) [2023-12-26 22:24:57,153][105620] Updated weights for policy 1, policy_version 967555 (0.0009) [2023-12-26 22:24:57,302][105692] Updated weights for policy 0, policy_version 967471 (0.0009) [2023-12-26 22:24:57,355][105692] Updated weights for policy 0, policy_version 967481 (0.0009) [2023-12-26 22:24:57,413][105692] Updated weights for policy 0, policy_version 967491 (0.0009) [2023-12-26 22:24:57,952][105620] Updated weights for policy 1, policy_version 967566 (0.0009) [2023-12-26 22:24:58,009][105620] Updated weights for policy 1, policy_version 967576 (0.0009) [2023-12-26 22:24:58,038][105692] Updated weights for policy 0, policy_version 967501 (0.0010) [2023-12-26 22:24:58,056][105620] Updated weights for policy 1, policy_version 967586 (0.0006) [2023-12-26 22:24:58,090][105692] Updated weights for policy 0, policy_version 967511 (0.0010) [2023-12-26 22:24:58,152][105692] Updated weights for policy 0, policy_version 967521 (0.0010) [2023-12-26 22:24:58,880][105620] Updated weights for policy 1, policy_version 967596 (0.0006) [2023-12-26 22:24:58,939][105620] Updated weights for policy 1, policy_version 967606 (0.0008) [2023-12-26 22:24:58,972][105692] Updated weights for policy 0, policy_version 967531 (0.0011) [2023-12-26 22:24:58,999][105620] Updated weights for policy 1, policy_version 967616 (0.0006) [2023-12-26 22:24:59,027][105692] Updated weights for policy 0, policy_version 967541 (0.0010) [2023-12-26 22:24:59,072][105692] Updated weights for policy 0, policy_version 967551 (0.0010) [2023-12-26 22:24:59,697][105620] Updated weights for policy 1, policy_version 967626 (0.0006) [2023-12-26 22:24:59,746][105620] Updated weights for policy 1, policy_version 967636 (0.0009) [2023-12-26 22:24:59,757][105692] Updated weights for policy 0, policy_version 967561 (0.0010) [2023-12-26 22:24:59,802][105620] Updated weights for policy 1, policy_version 967646 (0.0006) [2023-12-26 22:24:59,815][105692] Updated weights for policy 0, policy_version 967571 (0.0011) [2023-12-26 22:24:59,860][105620] Updated weights for policy 1, policy_version 967656 (0.0007) [2023-12-26 22:24:59,877][105692] Updated weights for policy 0, policy_version 967581 (0.0009) [2023-12-26 22:24:59,943][105692] Updated weights for policy 0, policy_version 967591 (0.0008) [2023-12-26 22:25:00,543][105620] Updated weights for policy 1, policy_version 967666 (0.0005) [2023-12-26 22:25:00,599][105620] Updated weights for policy 1, policy_version 967676 (0.0005) [2023-12-26 22:25:00,653][105620] Updated weights for policy 1, policy_version 967686 (0.0005) [2023-12-26 22:25:00,680][105692] Updated weights for policy 0, policy_version 967601 (0.0006) [2023-12-26 22:25:00,740][105692] Updated weights for policy 0, policy_version 967611 (0.0008) [2023-12-26 22:25:00,801][105692] Updated weights for policy 0, policy_version 967621 (0.0009) [2023-12-26 22:25:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19299.8). Total num frames: 495509504. Throughput: 0: 9771.0, 1: 9719.4. Samples: 495477628. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-26 22:25:01,063][104569] Avg episode reward: [(0, '7785.092'), (1, '8464.356')] [2023-12-26 22:25:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000967624_247750656.pth... [2023-12-26 22:25:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000967688_247758848.pth... [2023-12-26 22:25:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000966536_247463936.pth [2023-12-26 22:25:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000966504_247463936.pth [2023-12-26 22:25:01,194][105620] Updated weights for policy 1, policy_version 967696 (0.0008) [2023-12-26 22:25:01,244][105620] Updated weights for policy 1, policy_version 967706 (0.0010) [2023-12-26 22:25:01,311][105620] Updated weights for policy 1, policy_version 967716 (0.0011) [2023-12-26 22:25:01,520][105692] Updated weights for policy 0, policy_version 967631 (0.0010) [2023-12-26 22:25:01,582][105692] Updated weights for policy 0, policy_version 967641 (0.0009) [2023-12-26 22:25:01,646][105692] Updated weights for policy 0, policy_version 967651 (0.0007) [2023-12-26 22:25:02,017][105620] Updated weights for policy 1, policy_version 967726 (0.0008) [2023-12-26 22:25:02,080][105620] Updated weights for policy 1, policy_version 967736 (0.0007) [2023-12-26 22:25:02,141][105620] Updated weights for policy 1, policy_version 967746 (0.0009) [2023-12-26 22:25:02,408][105692] Updated weights for policy 0, policy_version 967661 (0.0006) [2023-12-26 22:25:02,469][105692] Updated weights for policy 0, policy_version 967671 (0.0006) [2023-12-26 22:25:02,523][105692] Updated weights for policy 0, policy_version 967681 (0.0006) [2023-12-26 22:25:02,789][105620] Updated weights for policy 1, policy_version 967756 (0.0010) [2023-12-26 22:25:02,859][105620] Updated weights for policy 1, policy_version 967766 (0.0009) [2023-12-26 22:25:02,923][105620] Updated weights for policy 1, policy_version 967776 (0.0008) [2023-12-26 22:25:03,187][105692] Updated weights for policy 0, policy_version 967691 (0.0006) [2023-12-26 22:25:03,244][105692] Updated weights for policy 0, policy_version 967701 (0.0005) [2023-12-26 22:25:03,289][105692] Updated weights for policy 0, policy_version 967711 (0.0005) [2023-12-26 22:25:03,800][105620] Updated weights for policy 1, policy_version 967786 (0.0009) [2023-12-26 22:25:03,805][105692] Updated weights for policy 0, policy_version 967721 (0.0005) [2023-12-26 22:25:03,864][105620] Updated weights for policy 1, policy_version 967796 (0.0007) [2023-12-26 22:25:03,871][105692] Updated weights for policy 0, policy_version 967731 (0.0007) [2023-12-26 22:25:03,922][105620] Updated weights for policy 1, policy_version 967806 (0.0008) [2023-12-26 22:25:03,924][105692] Updated weights for policy 0, policy_version 967741 (0.0006) [2023-12-26 22:25:03,976][105620] Updated weights for policy 1, policy_version 967816 (0.0007) [2023-12-26 22:25:03,978][105692] Updated weights for policy 0, policy_version 967751 (0.0007) [2023-12-26 22:25:04,593][105692] Updated weights for policy 0, policy_version 967761 (0.0006) [2023-12-26 22:25:04,651][105692] Updated weights for policy 0, policy_version 967771 (0.0005) [2023-12-26 22:25:04,707][105692] Updated weights for policy 0, policy_version 967781 (0.0005) [2023-12-26 22:25:04,793][105620] Updated weights for policy 1, policy_version 967826 (0.0009) [2023-12-26 22:25:04,847][105620] Updated weights for policy 1, policy_version 967836 (0.0009) [2023-12-26 22:25:04,900][105620] Updated weights for policy 1, policy_version 967846 (0.0009) [2023-12-26 22:25:05,331][105692] Updated weights for policy 0, policy_version 967791 (0.0005) [2023-12-26 22:25:05,380][105692] Updated weights for policy 0, policy_version 967801 (0.0005) [2023-12-26 22:25:05,426][105692] Updated weights for policy 0, policy_version 967811 (0.0008) [2023-12-26 22:25:05,705][105620] Updated weights for policy 1, policy_version 967856 (0.0010) [2023-12-26 22:25:05,753][105620] Updated weights for policy 1, policy_version 967866 (0.0010) [2023-12-26 22:25:05,810][105620] Updated weights for policy 1, policy_version 967876 (0.0010) [2023-12-26 22:25:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19327.6). Total num frames: 495607808. Throughput: 0: 9737.6, 1: 9712.4. Samples: 495596428. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-26 22:25:06,062][104569] Avg episode reward: [(0, '8107.373'), (1, '8733.118')] [2023-12-26 22:25:06,199][105692] Updated weights for policy 0, policy_version 967821 (0.0008) [2023-12-26 22:25:06,263][105692] Updated weights for policy 0, policy_version 967831 (0.0008) [2023-12-26 22:25:06,328][105692] Updated weights for policy 0, policy_version 967841 (0.0008) [2023-12-26 22:25:06,507][105620] Updated weights for policy 1, policy_version 967886 (0.0011) [2023-12-26 22:25:06,576][105620] Updated weights for policy 1, policy_version 967896 (0.0010) [2023-12-26 22:25:06,636][105620] Updated weights for policy 1, policy_version 967906 (0.0010) [2023-12-26 22:25:07,107][105692] Updated weights for policy 0, policy_version 967851 (0.0009) [2023-12-26 22:25:07,164][105692] Updated weights for policy 0, policy_version 967861 (0.0011) [2023-12-26 22:25:07,217][105692] Updated weights for policy 0, policy_version 967871 (0.0010) [2023-12-26 22:25:07,349][105620] Updated weights for policy 1, policy_version 967916 (0.0009) [2023-12-26 22:25:07,410][105620] Updated weights for policy 1, policy_version 967926 (0.0011) [2023-12-26 22:25:07,466][105620] Updated weights for policy 1, policy_version 967936 (0.0010) [2023-12-26 22:25:07,871][105692] Updated weights for policy 0, policy_version 967881 (0.0010) [2023-12-26 22:25:07,934][105692] Updated weights for policy 0, policy_version 967891 (0.0007) [2023-12-26 22:25:07,990][105692] Updated weights for policy 0, policy_version 967901 (0.0008) [2023-12-26 22:25:08,054][105692] Updated weights for policy 0, policy_version 967911 (0.0008) [2023-12-26 22:25:08,240][105620] Updated weights for policy 1, policy_version 967946 (0.0010) [2023-12-26 22:25:08,299][105620] Updated weights for policy 1, policy_version 967956 (0.0010) [2023-12-26 22:25:08,367][105620] Updated weights for policy 1, policy_version 967966 (0.0010) [2023-12-26 22:25:08,432][105620] Updated weights for policy 1, policy_version 967976 (0.0010) [2023-12-26 22:25:08,706][105692] Updated weights for policy 0, policy_version 967921 (0.0010) [2023-12-26 22:25:08,772][105692] Updated weights for policy 0, policy_version 967931 (0.0009) [2023-12-26 22:25:08,842][105692] Updated weights for policy 0, policy_version 967941 (0.0006) [2023-12-26 22:25:09,148][105620] Updated weights for policy 1, policy_version 967986 (0.0010) [2023-12-26 22:25:09,210][105620] Updated weights for policy 1, policy_version 967996 (0.0007) [2023-12-26 22:25:09,273][105620] Updated weights for policy 1, policy_version 968006 (0.0007) [2023-12-26 22:25:09,570][105692] Updated weights for policy 0, policy_version 967951 (0.0009) [2023-12-26 22:25:09,624][105692] Updated weights for policy 0, policy_version 967961 (0.0011) [2023-12-26 22:25:09,676][105692] Updated weights for policy 0, policy_version 967971 (0.0010) [2023-12-26 22:25:10,035][105620] Updated weights for policy 1, policy_version 968016 (0.0008) [2023-12-26 22:25:10,099][105620] Updated weights for policy 1, policy_version 968026 (0.0008) [2023-12-26 22:25:10,154][105620] Updated weights for policy 1, policy_version 968036 (0.0008) [2023-12-26 22:25:10,444][105692] Updated weights for policy 0, policy_version 967981 (0.0008) [2023-12-26 22:25:10,505][105692] Updated weights for policy 0, policy_version 967991 (0.0006) [2023-12-26 22:25:10,566][105692] Updated weights for policy 0, policy_version 968001 (0.0006) [2023-12-26 22:25:10,874][105620] Updated weights for policy 1, policy_version 968046 (0.0008) [2023-12-26 22:25:10,926][105620] Updated weights for policy 1, policy_version 968056 (0.0006) [2023-12-26 22:25:10,984][105620] Updated weights for policy 1, policy_version 968066 (0.0006) [2023-12-26 22:25:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19327.6). Total num frames: 495706112. Throughput: 0: 9715.0, 1: 9767.8. Samples: 495712784. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-26 22:25:11,063][104569] Avg episode reward: [(0, '8314.807'), (1, '8910.897')] [2023-12-26 22:25:11,257][105692] Updated weights for policy 0, policy_version 968011 (0.0007) [2023-12-26 22:25:11,317][105692] Updated weights for policy 0, policy_version 968021 (0.0011) [2023-12-26 22:25:11,387][105692] Updated weights for policy 0, policy_version 968031 (0.0010) [2023-12-26 22:25:11,681][105620] Updated weights for policy 1, policy_version 968076 (0.0008) [2023-12-26 22:25:11,751][105620] Updated weights for policy 1, policy_version 968086 (0.0009) [2023-12-26 22:25:11,800][105620] Updated weights for policy 1, policy_version 968096 (0.0008) [2023-12-26 22:25:12,115][105692] Updated weights for policy 0, policy_version 968041 (0.0008) [2023-12-26 22:25:12,179][105692] Updated weights for policy 0, policy_version 968051 (0.0009) [2023-12-26 22:25:12,185][105585] KL-divergence is very high: 245.4294 [2023-12-26 22:25:12,219][105585] KL-divergence is very high: 217.8200 [2023-12-26 22:25:12,241][105585] KL-divergence is very high: 433.6106 [2023-12-26 22:25:12,246][105692] Updated weights for policy 0, policy_version 968061 (0.0009) [2023-12-26 22:25:12,275][105585] KL-divergence is very high: 187.7869 [2023-12-26 22:25:12,295][105585] KL-divergence is very high: 395.4300 [2023-12-26 22:25:12,312][105692] Updated weights for policy 0, policy_version 968071 (0.0010) [2023-12-26 22:25:12,594][105620] Updated weights for policy 1, policy_version 968106 (0.0008) [2023-12-26 22:25:12,649][105620] Updated weights for policy 1, policy_version 968116 (0.0009) [2023-12-26 22:25:12,718][105620] Updated weights for policy 1, policy_version 968126 (0.0009) [2023-12-26 22:25:12,781][105620] Updated weights for policy 1, policy_version 968136 (0.0008) [2023-12-26 22:25:13,004][105692] Updated weights for policy 0, policy_version 968081 (0.0006) [2023-12-26 22:25:13,056][105692] Updated weights for policy 0, policy_version 968091 (0.0008) [2023-12-26 22:25:13,107][105692] Updated weights for policy 0, policy_version 968101 (0.0009) [2023-12-26 22:25:13,582][105620] Updated weights for policy 1, policy_version 968146 (0.0008) [2023-12-26 22:25:13,643][105620] Updated weights for policy 1, policy_version 968156 (0.0008) [2023-12-26 22:25:13,694][105620] Updated weights for policy 1, policy_version 968166 (0.0009) [2023-12-26 22:25:13,798][105692] Updated weights for policy 0, policy_version 968111 (0.0010) [2023-12-26 22:25:13,843][105692] Updated weights for policy 0, policy_version 968121 (0.0010) [2023-12-26 22:25:13,901][105692] Updated weights for policy 0, policy_version 968131 (0.0010) [2023-12-26 22:25:14,438][105620] Updated weights for policy 1, policy_version 968176 (0.0006) [2023-12-26 22:25:14,499][105620] Updated weights for policy 1, policy_version 968186 (0.0009) [2023-12-26 22:25:14,554][105620] Updated weights for policy 1, policy_version 968196 (0.0009) [2023-12-26 22:25:14,655][105692] Updated weights for policy 0, policy_version 968141 (0.0010) [2023-12-26 22:25:14,712][105692] Updated weights for policy 0, policy_version 968151 (0.0009) [2023-12-26 22:25:14,765][105692] Updated weights for policy 0, policy_version 968161 (0.0009) [2023-12-26 22:25:15,252][105620] Updated weights for policy 1, policy_version 968206 (0.0008) [2023-12-26 22:25:15,315][105620] Updated weights for policy 1, policy_version 968216 (0.0009) [2023-12-26 22:25:15,384][105620] Updated weights for policy 1, policy_version 968226 (0.0008) [2023-12-26 22:25:15,552][105692] Updated weights for policy 0, policy_version 968171 (0.0010) [2023-12-26 22:25:15,611][105692] Updated weights for policy 0, policy_version 968181 (0.0009) [2023-12-26 22:25:15,674][105692] Updated weights for policy 0, policy_version 968191 (0.0009) [2023-12-26 22:25:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19327.6). Total num frames: 495796224. Throughput: 0: 9607.6, 1: 9789.6. Samples: 495768716. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-26 22:25:16,063][104569] Avg episode reward: [(0, '8048.137'), (1, '9090.063')] [2023-12-26 22:25:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000968232_247898112.pth... [2023-12-26 22:25:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000968200_247898112.pth... [2023-12-26 22:25:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000967112_247611392.pth [2023-12-26 22:25:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000967048_247603200.pth [2023-12-26 22:25:16,164][105620] Updated weights for policy 1, policy_version 968236 (0.0009) [2023-12-26 22:25:16,223][105620] Updated weights for policy 1, policy_version 968246 (0.0008) [2023-12-26 22:25:16,292][105620] Updated weights for policy 1, policy_version 968256 (0.0008) [2023-12-26 22:25:16,372][105692] Updated weights for policy 0, policy_version 968201 (0.0009) [2023-12-26 22:25:16,425][105692] Updated weights for policy 0, policy_version 968211 (0.0008) [2023-12-26 22:25:16,506][105692] Updated weights for policy 0, policy_version 968221 (0.0010) [2023-12-26 22:25:16,555][105692] Updated weights for policy 0, policy_version 968231 (0.0010) [2023-12-26 22:25:16,921][105620] Updated weights for policy 1, policy_version 968266 (0.0009) [2023-12-26 22:25:16,968][105620] Updated weights for policy 1, policy_version 968276 (0.0007) [2023-12-26 22:25:17,029][105620] Updated weights for policy 1, policy_version 968286 (0.0009) [2023-12-26 22:25:17,080][105620] Updated weights for policy 1, policy_version 968296 (0.0009) [2023-12-26 22:25:17,263][105692] Updated weights for policy 0, policy_version 968241 (0.0010) [2023-12-26 22:25:17,324][105692] Updated weights for policy 0, policy_version 968251 (0.0010) [2023-12-26 22:25:17,381][105692] Updated weights for policy 0, policy_version 968261 (0.0010) [2023-12-26 22:25:17,831][105620] Updated weights for policy 1, policy_version 968306 (0.0005) [2023-12-26 22:25:17,895][105620] Updated weights for policy 1, policy_version 968316 (0.0008) [2023-12-26 22:25:17,959][105620] Updated weights for policy 1, policy_version 968326 (0.0009) [2023-12-26 22:25:18,118][105692] Updated weights for policy 0, policy_version 968271 (0.0010) [2023-12-26 22:25:18,169][105692] Updated weights for policy 0, policy_version 968281 (0.0010) [2023-12-26 22:25:18,229][105692] Updated weights for policy 0, policy_version 968291 (0.0010) [2023-12-26 22:25:18,714][105620] Updated weights for policy 1, policy_version 968336 (0.0009) [2023-12-26 22:25:18,772][105620] Updated weights for policy 1, policy_version 968346 (0.0009) [2023-12-26 22:25:18,832][105620] Updated weights for policy 1, policy_version 968356 (0.0008) [2023-12-26 22:25:18,999][105692] Updated weights for policy 0, policy_version 968301 (0.0008) [2023-12-26 22:25:19,057][105692] Updated weights for policy 0, policy_version 968311 (0.0010) [2023-12-26 22:25:19,109][105692] Updated weights for policy 0, policy_version 968321 (0.0010) [2023-12-26 22:25:19,534][105620] Updated weights for policy 1, policy_version 968366 (0.0007) [2023-12-26 22:25:19,610][105620] Updated weights for policy 1, policy_version 968376 (0.0008) [2023-12-26 22:25:19,682][105620] Updated weights for policy 1, policy_version 968386 (0.0009) [2023-12-26 22:25:19,870][105692] Updated weights for policy 0, policy_version 968331 (0.0009) [2023-12-26 22:25:19,941][105692] Updated weights for policy 0, policy_version 968341 (0.0008) [2023-12-26 22:25:20,013][105692] Updated weights for policy 0, policy_version 968351 (0.0011) [2023-12-26 22:25:20,313][105620] Updated weights for policy 1, policy_version 968396 (0.0007) [2023-12-26 22:25:20,376][105620] Updated weights for policy 1, policy_version 968406 (0.0008) [2023-12-26 22:25:20,429][105620] Updated weights for policy 1, policy_version 968416 (0.0008) [2023-12-26 22:25:20,731][105692] Updated weights for policy 0, policy_version 968361 (0.0010) [2023-12-26 22:25:20,799][105692] Updated weights for policy 0, policy_version 968371 (0.0008) [2023-12-26 22:25:20,866][105692] Updated weights for policy 0, policy_version 968381 (0.0008) [2023-12-26 22:25:20,928][105692] Updated weights for policy 0, policy_version 968391 (0.0009) [2023-12-26 22:25:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19299.8). Total num frames: 495894528. Throughput: 0: 9652.9, 1: 9694.0. Samples: 495882704. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-26 22:25:21,062][104569] Avg episode reward: [(0, '7685.874'), (1, '9080.246')] [2023-12-26 22:25:21,143][105620] Updated weights for policy 1, policy_version 968426 (0.0008) [2023-12-26 22:25:21,214][105620] Updated weights for policy 1, policy_version 968436 (0.0009) [2023-12-26 22:25:21,284][105620] Updated weights for policy 1, policy_version 968446 (0.0008) [2023-12-26 22:25:21,349][105620] Updated weights for policy 1, policy_version 968456 (0.0008) [2023-12-26 22:25:21,679][105692] Updated weights for policy 0, policy_version 968401 (0.0010) [2023-12-26 22:25:21,753][105585] KL-divergence is very high: 388.4982 [2023-12-26 22:25:21,754][105692] Updated weights for policy 0, policy_version 968411 (0.0009) [2023-12-26 22:25:21,761][105585] KL-divergence is very high: 373.4593 [2023-12-26 22:25:21,809][105585] KL-divergence is very high: 619.5731 [2023-12-26 22:25:21,815][105585] KL-divergence is very high: 537.6223 [2023-12-26 22:25:21,821][105692] Updated weights for policy 0, policy_version 968421 (0.0011) [2023-12-26 22:25:22,111][105620] Updated weights for policy 1, policy_version 968466 (0.0007) [2023-12-26 22:25:22,162][105620] Updated weights for policy 1, policy_version 968476 (0.0006) [2023-12-26 22:25:22,220][105620] Updated weights for policy 1, policy_version 968486 (0.0007) [2023-12-26 22:25:22,529][105585] KL-divergence is very high: 213.4708 [2023-12-26 22:25:22,551][105585] KL-divergence is very high: 692.3180 [2023-12-26 22:25:22,557][105692] Updated weights for policy 0, policy_version 968431 (0.0010) [2023-12-26 22:25:22,570][105585] KL-divergence is very high: 189.7654 [2023-12-26 22:25:22,589][105585] KL-divergence is very high: 637.7004 [2023-12-26 22:25:22,606][105692] Updated weights for policy 0, policy_version 968441 (0.0010) [2023-12-26 22:25:22,610][105585] KL-divergence is very high: 152.6457 [2023-12-26 22:25:22,633][105585] KL-divergence is very high: 557.4701 [2023-12-26 22:25:22,656][105585] KL-divergence is very high: 118.4797 [2023-12-26 22:25:22,662][105692] Updated weights for policy 0, policy_version 968451 (0.0011) [2023-12-26 22:25:22,679][105585] KL-divergence is very high: 484.6993 [2023-12-26 22:25:23,040][105620] Updated weights for policy 1, policy_version 968496 (0.0010) [2023-12-26 22:25:23,104][105620] Updated weights for policy 1, policy_version 968506 (0.0009) [2023-12-26 22:25:23,169][105620] Updated weights for policy 1, policy_version 968516 (0.0009) [2023-12-26 22:25:23,453][105692] Updated weights for policy 0, policy_version 968461 (0.0010) [2023-12-26 22:25:23,506][105692] Updated weights for policy 0, policy_version 968471 (0.0010) [2023-12-26 22:25:23,558][105692] Updated weights for policy 0, policy_version 968481 (0.0011) [2023-12-26 22:25:23,818][105620] Updated weights for policy 1, policy_version 968526 (0.0010) [2023-12-26 22:25:23,871][105620] Updated weights for policy 1, policy_version 968536 (0.0010) [2023-12-26 22:25:23,929][105620] Updated weights for policy 1, policy_version 968546 (0.0005) [2023-12-26 22:25:24,276][105692] Updated weights for policy 0, policy_version 968491 (0.0009) [2023-12-26 22:25:24,334][105692] Updated weights for policy 0, policy_version 968501 (0.0005) [2023-12-26 22:25:24,394][105692] Updated weights for policy 0, policy_version 968511 (0.0006) [2023-12-26 22:25:24,547][105620] Updated weights for policy 1, policy_version 968556 (0.0007) [2023-12-26 22:25:24,602][105620] Updated weights for policy 1, policy_version 968566 (0.0010) [2023-12-26 22:25:24,664][105620] Updated weights for policy 1, policy_version 968576 (0.0010) [2023-12-26 22:25:24,961][105692] Updated weights for policy 0, policy_version 968521 (0.0006) [2023-12-26 22:25:25,022][105692] Updated weights for policy 0, policy_version 968531 (0.0010) [2023-12-26 22:25:25,081][105692] Updated weights for policy 0, policy_version 968541 (0.0010) [2023-12-26 22:25:25,129][105692] Updated weights for policy 0, policy_version 968551 (0.0010) [2023-12-26 22:25:25,410][105620] Updated weights for policy 1, policy_version 968586 (0.0010) [2023-12-26 22:25:25,461][105620] Updated weights for policy 1, policy_version 968596 (0.0010) [2023-12-26 22:25:25,519][105620] Updated weights for policy 1, policy_version 968606 (0.0010) [2023-12-26 22:25:25,579][105620] Updated weights for policy 1, policy_version 968616 (0.0010) [2023-12-26 22:25:25,884][105692] Updated weights for policy 0, policy_version 968561 (0.0010) [2023-12-26 22:25:25,942][105692] Updated weights for policy 0, policy_version 968571 (0.0010) [2023-12-26 22:25:26,007][105692] Updated weights for policy 0, policy_version 968581 (0.0010) [2023-12-26 22:25:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19299.8). Total num frames: 495992832. Throughput: 0: 9717.0, 1: 9604.9. Samples: 495998588. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-26 22:25:26,063][104569] Avg episode reward: [(0, '7892.872'), (1, '8899.035')] [2023-12-26 22:25:26,293][105620] Updated weights for policy 1, policy_version 968626 (0.0005) [2023-12-26 22:25:26,352][105620] Updated weights for policy 1, policy_version 968636 (0.0007) [2023-12-26 22:25:26,409][105620] Updated weights for policy 1, policy_version 968646 (0.0009) [2023-12-26 22:25:26,730][105692] Updated weights for policy 0, policy_version 968591 (0.0010) [2023-12-26 22:25:26,788][105692] Updated weights for policy 0, policy_version 968601 (0.0010) [2023-12-26 22:25:26,836][105692] Updated weights for policy 0, policy_version 968611 (0.0010) [2023-12-26 22:25:27,145][105620] Updated weights for policy 1, policy_version 968656 (0.0010) [2023-12-26 22:25:27,197][105620] Updated weights for policy 1, policy_version 968666 (0.0010) [2023-12-26 22:25:27,241][105620] Updated weights for policy 1, policy_version 968676 (0.0010) [2023-12-26 22:25:27,525][105692] Updated weights for policy 0, policy_version 968621 (0.0008) [2023-12-26 22:25:27,587][105692] Updated weights for policy 0, policy_version 968631 (0.0006) [2023-12-26 22:25:27,647][105692] Updated weights for policy 0, policy_version 968641 (0.0010) [2023-12-26 22:25:27,869][105620] Updated weights for policy 1, policy_version 968686 (0.0007) [2023-12-26 22:25:27,918][105620] Updated weights for policy 1, policy_version 968696 (0.0006) [2023-12-26 22:25:27,962][105620] Updated weights for policy 1, policy_version 968706 (0.0005) [2023-12-26 22:25:28,319][105692] Updated weights for policy 0, policy_version 968651 (0.0010) [2023-12-26 22:25:28,381][105692] Updated weights for policy 0, policy_version 968661 (0.0010) [2023-12-26 22:25:28,443][105692] Updated weights for policy 0, policy_version 968671 (0.0009) [2023-12-26 22:25:28,553][105620] Updated weights for policy 1, policy_version 968716 (0.0005) [2023-12-26 22:25:28,610][105620] Updated weights for policy 1, policy_version 968726 (0.0005) [2023-12-26 22:25:28,676][105620] Updated weights for policy 1, policy_version 968736 (0.0008) [2023-12-26 22:25:29,102][105692] Updated weights for policy 0, policy_version 968681 (0.0010) [2023-12-26 22:25:29,162][105692] Updated weights for policy 0, policy_version 968691 (0.0010) [2023-12-26 22:25:29,214][105692] Updated weights for policy 0, policy_version 968701 (0.0010) [2023-12-26 22:25:29,276][105692] Updated weights for policy 0, policy_version 968711 (0.0010) [2023-12-26 22:25:29,403][105620] Updated weights for policy 1, policy_version 968746 (0.0008) [2023-12-26 22:25:29,455][105620] Updated weights for policy 1, policy_version 968756 (0.0008) [2023-12-26 22:25:29,517][105620] Updated weights for policy 1, policy_version 968766 (0.0008) [2023-12-26 22:25:29,562][105620] Updated weights for policy 1, policy_version 968776 (0.0008) [2023-12-26 22:25:29,998][105692] Updated weights for policy 0, policy_version 968721 (0.0006) [2023-12-26 22:25:30,061][105692] Updated weights for policy 0, policy_version 968731 (0.0006) [2023-12-26 22:25:30,120][105692] Updated weights for policy 0, policy_version 968741 (0.0006) [2023-12-26 22:25:30,389][105620] Updated weights for policy 1, policy_version 968786 (0.0005) [2023-12-26 22:25:30,447][105620] Updated weights for policy 1, policy_version 968796 (0.0007) [2023-12-26 22:25:30,491][105620] Updated weights for policy 1, policy_version 968806 (0.0008) [2023-12-26 22:25:30,729][105692] Updated weights for policy 0, policy_version 968751 (0.0006) [2023-12-26 22:25:30,785][105692] Updated weights for policy 0, policy_version 968761 (0.0005) [2023-12-26 22:25:30,826][105692] Updated weights for policy 0, policy_version 968771 (0.0005) [2023-12-26 22:25:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19299.8). Total num frames: 496091136. Throughput: 0: 9738.9, 1: 9689.6. Samples: 496059680. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-26 22:25:31,062][104569] Avg episode reward: [(0, '7696.407'), (1, '8905.014')] [2023-12-26 22:25:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000968808_248045568.pth... [2023-12-26 22:25:31,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000968776_248045568.pth... [2023-12-26 22:25:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000967624_247750656.pth [2023-12-26 22:25:31,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000967688_247758848.pth [2023-12-26 22:25:31,249][105620] Updated weights for policy 1, policy_version 968816 (0.0008) [2023-12-26 22:25:31,311][105620] Updated weights for policy 1, policy_version 968826 (0.0008) [2023-12-26 22:25:31,380][105620] Updated weights for policy 1, policy_version 968836 (0.0008) [2023-12-26 22:25:31,476][105692] Updated weights for policy 0, policy_version 968781 (0.0008) [2023-12-26 22:25:31,538][105692] Updated weights for policy 0, policy_version 968791 (0.0008) [2023-12-26 22:25:31,598][105692] Updated weights for policy 0, policy_version 968801 (0.0009) [2023-12-26 22:25:32,156][105620] Updated weights for policy 1, policy_version 968846 (0.0010) [2023-12-26 22:25:32,214][105620] Updated weights for policy 1, policy_version 968856 (0.0010) [2023-12-26 22:25:32,280][105620] Updated weights for policy 1, policy_version 968866 (0.0011) [2023-12-26 22:25:32,381][105692] Updated weights for policy 0, policy_version 968811 (0.0010) [2023-12-26 22:25:32,433][105692] Updated weights for policy 0, policy_version 968821 (0.0008) [2023-12-26 22:25:32,478][105692] Updated weights for policy 0, policy_version 968831 (0.0008) [2023-12-26 22:25:33,041][105620] Updated weights for policy 1, policy_version 968876 (0.0010) [2023-12-26 22:25:33,100][105620] Updated weights for policy 1, policy_version 968886 (0.0010) [2023-12-26 22:25:33,151][105620] Updated weights for policy 1, policy_version 968896 (0.0010) [2023-12-26 22:25:33,194][105692] Updated weights for policy 0, policy_version 968841 (0.0008) [2023-12-26 22:25:33,249][105692] Updated weights for policy 0, policy_version 968851 (0.0010) [2023-12-26 22:25:33,312][105692] Updated weights for policy 0, policy_version 968861 (0.0010) [2023-12-26 22:25:33,367][105692] Updated weights for policy 0, policy_version 968871 (0.0010) [2023-12-26 22:25:33,888][105620] Updated weights for policy 1, policy_version 968906 (0.0010) [2023-12-26 22:25:33,946][105620] Updated weights for policy 1, policy_version 968916 (0.0010) [2023-12-26 22:25:34,004][105620] Updated weights for policy 1, policy_version 968926 (0.0010) [2023-12-26 22:25:34,022][105692] Updated weights for policy 0, policy_version 968881 (0.0010) [2023-12-26 22:25:34,048][105620] Updated weights for policy 1, policy_version 968936 (0.0010) [2023-12-26 22:25:34,090][105692] Updated weights for policy 0, policy_version 968891 (0.0010) [2023-12-26 22:25:34,156][105692] Updated weights for policy 0, policy_version 968901 (0.0011) [2023-12-26 22:25:34,809][105620] Updated weights for policy 1, policy_version 968946 (0.0010) [2023-12-26 22:25:34,847][105692] Updated weights for policy 0, policy_version 968911 (0.0008) [2023-12-26 22:25:34,858][105620] Updated weights for policy 1, policy_version 968956 (0.0010) [2023-12-26 22:25:34,895][105692] Updated weights for policy 0, policy_version 968921 (0.0010) [2023-12-26 22:25:34,906][105620] Updated weights for policy 1, policy_version 968966 (0.0010) [2023-12-26 22:25:34,939][105692] Updated weights for policy 0, policy_version 968931 (0.0008) [2023-12-26 22:25:35,488][105620] Updated weights for policy 1, policy_version 968976 (0.0007) [2023-12-26 22:25:35,507][105692] Updated weights for policy 0, policy_version 968941 (0.0005) [2023-12-26 22:25:35,545][105620] Updated weights for policy 1, policy_version 968986 (0.0005) [2023-12-26 22:25:35,562][105692] Updated weights for policy 0, policy_version 968951 (0.0005) [2023-12-26 22:25:35,604][105620] Updated weights for policy 1, policy_version 968996 (0.0005) [2023-12-26 22:25:35,620][105692] Updated weights for policy 0, policy_version 968961 (0.0005) [2023-12-26 22:25:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19299.8). Total num frames: 496189440. Throughput: 0: 9752.3, 1: 9603.7. Samples: 496175312. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-26 22:25:36,062][104569] Avg episode reward: [(0, '7732.666'), (1, '8725.657')] [2023-12-26 22:25:36,117][105620] Updated weights for policy 1, policy_version 969006 (0.0006) [2023-12-26 22:25:36,178][105620] Updated weights for policy 1, policy_version 969016 (0.0009) [2023-12-26 22:25:36,248][105620] Updated weights for policy 1, policy_version 969026 (0.0011) [2023-12-26 22:25:36,280][105692] Updated weights for policy 0, policy_version 968971 (0.0005) [2023-12-26 22:25:36,342][105692] Updated weights for policy 0, policy_version 968981 (0.0006) [2023-12-26 22:25:36,407][105692] Updated weights for policy 0, policy_version 968991 (0.0006) [2023-12-26 22:25:36,932][105620] Updated weights for policy 1, policy_version 969036 (0.0010) [2023-12-26 22:25:36,988][105620] Updated weights for policy 1, policy_version 969046 (0.0011) [2023-12-26 22:25:37,037][105620] Updated weights for policy 1, policy_version 969056 (0.0010) [2023-12-26 22:25:37,066][105692] Updated weights for policy 0, policy_version 969001 (0.0009) [2023-12-26 22:25:37,117][105692] Updated weights for policy 0, policy_version 969011 (0.0008) [2023-12-26 22:25:37,163][105692] Updated weights for policy 0, policy_version 969021 (0.0008) [2023-12-26 22:25:37,222][105692] Updated weights for policy 0, policy_version 969031 (0.0008) [2023-12-26 22:25:37,805][105620] Updated weights for policy 1, policy_version 969066 (0.0010) [2023-12-26 22:25:37,873][105620] Updated weights for policy 1, policy_version 969076 (0.0010) [2023-12-26 22:25:37,936][105620] Updated weights for policy 1, policy_version 969086 (0.0010) [2023-12-26 22:25:38,000][105620] Updated weights for policy 1, policy_version 969096 (0.0011) [2023-12-26 22:25:38,003][105692] Updated weights for policy 0, policy_version 969041 (0.0009) [2023-12-26 22:25:38,066][105692] Updated weights for policy 0, policy_version 969051 (0.0005) [2023-12-26 22:25:38,130][105692] Updated weights for policy 0, policy_version 969061 (0.0005) [2023-12-26 22:25:38,744][105620] Updated weights for policy 1, policy_version 969106 (0.0007) [2023-12-26 22:25:38,805][105620] Updated weights for policy 1, policy_version 969116 (0.0009) [2023-12-26 22:25:38,836][105692] Updated weights for policy 0, policy_version 969071 (0.0006) [2023-12-26 22:25:38,865][105620] Updated weights for policy 1, policy_version 969126 (0.0009) [2023-12-26 22:25:38,889][105692] Updated weights for policy 0, policy_version 969081 (0.0008) [2023-12-26 22:25:38,945][105692] Updated weights for policy 0, policy_version 969091 (0.0009) [2023-12-26 22:25:39,470][105620] Updated weights for policy 1, policy_version 969136 (0.0009) [2023-12-26 22:25:39,543][105620] Updated weights for policy 1, policy_version 969146 (0.0011) [2023-12-26 22:25:39,613][105620] Updated weights for policy 1, policy_version 969156 (0.0011) [2023-12-26 22:25:39,839][105692] Updated weights for policy 0, policy_version 969101 (0.0009) [2023-12-26 22:25:39,892][105692] Updated weights for policy 0, policy_version 969111 (0.0008) [2023-12-26 22:25:39,956][105692] Updated weights for policy 0, policy_version 969121 (0.0009) [2023-12-26 22:25:40,349][105620] Updated weights for policy 1, policy_version 969166 (0.0009) [2023-12-26 22:25:40,411][105620] Updated weights for policy 1, policy_version 969176 (0.0010) [2023-12-26 22:25:40,478][105620] Updated weights for policy 1, policy_version 969186 (0.0011) [2023-12-26 22:25:40,730][105692] Updated weights for policy 0, policy_version 969131 (0.0008) [2023-12-26 22:25:40,793][105692] Updated weights for policy 0, policy_version 969141 (0.0006) [2023-12-26 22:25:40,858][105692] Updated weights for policy 0, policy_version 969151 (0.0008) [2023-12-26 22:25:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19327.6). Total num frames: 496287744. Throughput: 0: 9849.4, 1: 9663.1. Samples: 496295912. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-26 22:25:41,062][104569] Avg episode reward: [(0, '8454.132'), (1, '8824.305')] [2023-12-26 22:25:41,219][105620] Updated weights for policy 1, policy_version 969196 (0.0010) [2023-12-26 22:25:41,281][105620] Updated weights for policy 1, policy_version 969206 (0.0007) [2023-12-26 22:25:41,340][105620] Updated weights for policy 1, policy_version 969216 (0.0006) [2023-12-26 22:25:41,656][105692] Updated weights for policy 0, policy_version 969161 (0.0008) [2023-12-26 22:25:41,720][105692] Updated weights for policy 0, policy_version 969171 (0.0009) [2023-12-26 22:25:41,781][105692] Updated weights for policy 0, policy_version 969181 (0.0007) [2023-12-26 22:25:41,842][105692] Updated weights for policy 0, policy_version 969191 (0.0006) [2023-12-26 22:25:42,044][105620] Updated weights for policy 1, policy_version 969226 (0.0008) [2023-12-26 22:25:42,096][105620] Updated weights for policy 1, policy_version 969236 (0.0008) [2023-12-26 22:25:42,151][105620] Updated weights for policy 1, policy_version 969246 (0.0009) [2023-12-26 22:25:42,205][105620] Updated weights for policy 1, policy_version 969256 (0.0010) [2023-12-26 22:25:42,506][105692] Updated weights for policy 0, policy_version 969201 (0.0008) [2023-12-26 22:25:42,561][105692] Updated weights for policy 0, policy_version 969212 (0.0008) [2023-12-26 22:25:42,615][105692] Updated weights for policy 0, policy_version 969222 (0.0005) [2023-12-26 22:25:42,966][105620] Updated weights for policy 1, policy_version 969266 (0.0009) [2023-12-26 22:25:43,022][105620] Updated weights for policy 1, policy_version 969276 (0.0005) [2023-12-26 22:25:43,083][105620] Updated weights for policy 1, policy_version 969286 (0.0008) [2023-12-26 22:25:43,270][105692] Updated weights for policy 0, policy_version 969232 (0.0009) [2023-12-26 22:25:43,318][105692] Updated weights for policy 0, policy_version 969242 (0.0009) [2023-12-26 22:25:43,368][105692] Updated weights for policy 0, policy_version 969252 (0.0009) [2023-12-26 22:25:43,789][105620] Updated weights for policy 1, policy_version 969296 (0.0008) [2023-12-26 22:25:43,852][105620] Updated weights for policy 1, policy_version 969306 (0.0008) [2023-12-26 22:25:43,914][105620] Updated weights for policy 1, policy_version 969316 (0.0008) [2023-12-26 22:25:44,138][105692] Updated weights for policy 0, policy_version 969262 (0.0007) [2023-12-26 22:25:44,194][105692] Updated weights for policy 0, policy_version 969272 (0.0006) [2023-12-26 22:25:44,253][105692] Updated weights for policy 0, policy_version 969282 (0.0009) [2023-12-26 22:25:44,659][105620] Updated weights for policy 1, policy_version 969326 (0.0007) [2023-12-26 22:25:44,730][105620] Updated weights for policy 1, policy_version 969336 (0.0005) [2023-12-26 22:25:44,798][105620] Updated weights for policy 1, policy_version 969346 (0.0007) [2023-12-26 22:25:44,903][105692] Updated weights for policy 0, policy_version 969292 (0.0009) [2023-12-26 22:25:44,939][105585] KL-divergence is very high: 323.8501 [2023-12-26 22:25:44,973][105692] Updated weights for policy 0, policy_version 969302 (0.0008) [2023-12-26 22:25:44,995][105585] KL-divergence is very high: 502.6442 [2023-12-26 22:25:45,044][105692] Updated weights for policy 0, policy_version 969312 (0.0006) [2023-12-26 22:25:45,054][105585] KL-divergence is very high: 525.5815 [2023-12-26 22:25:45,438][105620] Updated weights for policy 1, policy_version 969356 (0.0007) [2023-12-26 22:25:45,494][105620] Updated weights for policy 1, policy_version 969366 (0.0010) [2023-12-26 22:25:45,550][105620] Updated weights for policy 1, policy_version 969376 (0.0009) [2023-12-26 22:25:45,660][105692] Updated weights for policy 0, policy_version 969322 (0.0007) [2023-12-26 22:25:45,681][105585] KL-divergence is very high: 372.7997 [2023-12-26 22:25:45,724][105692] Updated weights for policy 0, policy_version 969332 (0.0009) [2023-12-26 22:25:45,728][105585] KL-divergence is very high: 322.3429 [2023-12-26 22:25:45,778][105585] KL-divergence is very high: 265.3587 [2023-12-26 22:25:45,786][105692] Updated weights for policy 0, policy_version 969342 (0.0009) [2023-12-26 22:25:45,829][105585] KL-divergence is very high: 219.8169 [2023-12-26 22:25:45,848][105692] Updated weights for policy 0, policy_version 969352 (0.0010) [2023-12-26 22:25:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.3, 300 sec: 19327.6). Total num frames: 496386048. Throughput: 0: 9817.3, 1: 9655.7. Samples: 496353916. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:25:46,063][104569] Avg episode reward: [(0, '8602.612'), (1, '9107.735')] [2023-12-26 22:25:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000969352_248193024.pth... [2023-12-26 22:25:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000969384_248193024.pth... [2023-12-26 22:25:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000968232_247898112.pth [2023-12-26 22:25:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000968200_247898112.pth [2023-12-26 22:25:46,285][105620] Updated weights for policy 1, policy_version 969386 (0.0009) [2023-12-26 22:25:46,339][105620] Updated weights for policy 1, policy_version 969396 (0.0008) [2023-12-26 22:25:46,385][105620] Updated weights for policy 1, policy_version 969406 (0.0008) [2023-12-26 22:25:46,443][105620] Updated weights for policy 1, policy_version 969416 (0.0007) [2023-12-26 22:25:46,591][105692] Updated weights for policy 0, policy_version 969362 (0.0009) [2023-12-26 22:25:46,649][105692] Updated weights for policy 0, policy_version 969372 (0.0009) [2023-12-26 22:25:46,697][105692] Updated weights for policy 0, policy_version 969382 (0.0009) [2023-12-26 22:25:47,160][105620] Updated weights for policy 1, policy_version 969426 (0.0009) [2023-12-26 22:25:47,210][105620] Updated weights for policy 1, policy_version 969436 (0.0008) [2023-12-26 22:25:47,271][105620] Updated weights for policy 1, policy_version 969446 (0.0009) [2023-12-26 22:25:47,472][105692] Updated weights for policy 0, policy_version 969392 (0.0008) [2023-12-26 22:25:47,535][105692] Updated weights for policy 0, policy_version 969402 (0.0008) [2023-12-26 22:25:47,596][105692] Updated weights for policy 0, policy_version 969412 (0.0008) [2023-12-26 22:25:48,073][105620] Updated weights for policy 1, policy_version 969456 (0.0009) [2023-12-26 22:25:48,135][105620] Updated weights for policy 1, policy_version 969466 (0.0008) [2023-12-26 22:25:48,206][105620] Updated weights for policy 1, policy_version 969476 (0.0009) [2023-12-26 22:25:48,309][105692] Updated weights for policy 0, policy_version 969422 (0.0007) [2023-12-26 22:25:48,379][105692] Updated weights for policy 0, policy_version 969432 (0.0008) [2023-12-26 22:25:48,444][105692] Updated weights for policy 0, policy_version 969442 (0.0007) [2023-12-26 22:25:48,920][105620] Updated weights for policy 1, policy_version 969486 (0.0007) [2023-12-26 22:25:48,974][105620] Updated weights for policy 1, policy_version 969496 (0.0005) [2023-12-26 22:25:49,031][105620] Updated weights for policy 1, policy_version 969506 (0.0007) [2023-12-26 22:25:49,181][105692] Updated weights for policy 0, policy_version 969452 (0.0008) [2023-12-26 22:25:49,254][105692] Updated weights for policy 0, policy_version 969462 (0.0009) [2023-12-26 22:25:49,320][105692] Updated weights for policy 0, policy_version 969472 (0.0008) [2023-12-26 22:25:49,649][105620] Updated weights for policy 1, policy_version 969516 (0.0007) [2023-12-26 22:25:49,699][105620] Updated weights for policy 1, policy_version 969526 (0.0005) [2023-12-26 22:25:49,758][105620] Updated weights for policy 1, policy_version 969536 (0.0008) [2023-12-26 22:25:49,972][105692] Updated weights for policy 0, policy_version 969482 (0.0008) [2023-12-26 22:25:50,039][105692] Updated weights for policy 0, policy_version 969492 (0.0009) [2023-12-26 22:25:50,095][105692] Updated weights for policy 0, policy_version 969502 (0.0009) [2023-12-26 22:25:50,147][105692] Updated weights for policy 0, policy_version 969512 (0.0010) [2023-12-26 22:25:50,484][105620] Updated weights for policy 1, policy_version 969546 (0.0009) [2023-12-26 22:25:50,545][105620] Updated weights for policy 1, policy_version 969556 (0.0009) [2023-12-26 22:25:50,612][105620] Updated weights for policy 1, policy_version 969566 (0.0007) [2023-12-26 22:25:50,695][105620] Updated weights for policy 1, policy_version 969576 (0.0006) [2023-12-26 22:25:50,890][105692] Updated weights for policy 0, policy_version 969522 (0.0006) [2023-12-26 22:25:50,963][105692] Updated weights for policy 0, policy_version 969532 (0.0006) [2023-12-26 22:25:51,023][105692] Updated weights for policy 0, policy_version 969542 (0.0008) [2023-12-26 22:25:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19327.6). Total num frames: 496484352. Throughput: 0: 9756.9, 1: 9671.4. Samples: 496470700. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:25:51,063][104569] Avg episode reward: [(0, '8578.077'), (1, '8663.619')] [2023-12-26 22:25:51,461][105620] Updated weights for policy 1, policy_version 969586 (0.0009) [2023-12-26 22:25:51,516][105620] Updated weights for policy 1, policy_version 969596 (0.0009) [2023-12-26 22:25:51,573][105620] Updated weights for policy 1, policy_version 969606 (0.0009) [2023-12-26 22:25:51,707][105692] Updated weights for policy 0, policy_version 969552 (0.0009) [2023-12-26 22:25:51,767][105692] Updated weights for policy 0, policy_version 969562 (0.0007) [2023-12-26 22:25:51,822][105692] Updated weights for policy 0, policy_version 969572 (0.0006) [2023-12-26 22:25:52,359][105620] Updated weights for policy 1, policy_version 969616 (0.0008) [2023-12-26 22:25:52,411][105620] Updated weights for policy 1, policy_version 969626 (0.0009) [2023-12-26 22:25:52,440][105692] Updated weights for policy 0, policy_version 969582 (0.0006) [2023-12-26 22:25:52,467][105620] Updated weights for policy 1, policy_version 969636 (0.0007) [2023-12-26 22:25:52,504][105692] Updated weights for policy 0, policy_version 969592 (0.0007) [2023-12-26 22:25:52,576][105692] Updated weights for policy 0, policy_version 969602 (0.0009) [2023-12-26 22:25:53,175][105620] Updated weights for policy 1, policy_version 969646 (0.0009) [2023-12-26 22:25:53,228][105620] Updated weights for policy 1, policy_version 969656 (0.0008) [2023-12-26 22:25:53,276][105620] Updated weights for policy 1, policy_version 969666 (0.0007) [2023-12-26 22:25:53,389][105692] Updated weights for policy 0, policy_version 969612 (0.0008) [2023-12-26 22:25:53,445][105692] Updated weights for policy 0, policy_version 969622 (0.0008) [2023-12-26 22:25:53,500][105692] Updated weights for policy 0, policy_version 969632 (0.0008) [2023-12-26 22:25:53,991][105620] Updated weights for policy 1, policy_version 969676 (0.0010) [2023-12-26 22:25:54,039][105620] Updated weights for policy 1, policy_version 969686 (0.0010) [2023-12-26 22:25:54,087][105620] Updated weights for policy 1, policy_version 969696 (0.0010) [2023-12-26 22:25:54,177][105692] Updated weights for policy 0, policy_version 969642 (0.0008) [2023-12-26 22:25:54,234][105692] Updated weights for policy 0, policy_version 969652 (0.0006) [2023-12-26 22:25:54,297][105692] Updated weights for policy 0, policy_version 969662 (0.0006) [2023-12-26 22:25:54,363][105692] Updated weights for policy 0, policy_version 969672 (0.0008) [2023-12-26 22:25:54,848][105620] Updated weights for policy 1, policy_version 969706 (0.0011) [2023-12-26 22:25:54,917][105620] Updated weights for policy 1, policy_version 969716 (0.0011) [2023-12-26 22:25:54,963][105692] Updated weights for policy 0, policy_version 969682 (0.0007) [2023-12-26 22:25:54,981][105620] Updated weights for policy 1, policy_version 969726 (0.0011) [2023-12-26 22:25:55,025][105692] Updated weights for policy 0, policy_version 969692 (0.0009) [2023-12-26 22:25:55,041][105620] Updated weights for policy 1, policy_version 969736 (0.0010) [2023-12-26 22:25:55,086][105692] Updated weights for policy 0, policy_version 969702 (0.0009) [2023-12-26 22:25:55,775][105620] Updated weights for policy 1, policy_version 969746 (0.0007) [2023-12-26 22:25:55,792][105692] Updated weights for policy 0, policy_version 969712 (0.0007) [2023-12-26 22:25:55,840][105620] Updated weights for policy 1, policy_version 969756 (0.0011) [2023-12-26 22:25:55,847][105692] Updated weights for policy 0, policy_version 969722 (0.0006) [2023-12-26 22:25:55,894][105692] Updated weights for policy 0, policy_version 969732 (0.0006) [2023-12-26 22:25:55,896][105620] Updated weights for policy 1, policy_version 969766 (0.0010) [2023-12-26 22:25:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19355.3). Total num frames: 496582656. Throughput: 0: 9767.0, 1: 9652.4. Samples: 496586656. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:25:56,063][104569] Avg episode reward: [(0, '8585.593'), (1, '8406.907')] [2023-12-26 22:25:56,518][105692] Updated weights for policy 0, policy_version 969742 (0.0007) [2023-12-26 22:25:56,562][105692] Updated weights for policy 0, policy_version 969752 (0.0008) [2023-12-26 22:25:56,621][105692] Updated weights for policy 0, policy_version 969762 (0.0008) [2023-12-26 22:25:56,623][105620] Updated weights for policy 1, policy_version 969776 (0.0010) [2023-12-26 22:25:56,674][105620] Updated weights for policy 1, policy_version 969786 (0.0010) [2023-12-26 22:25:56,721][105620] Updated weights for policy 1, policy_version 969796 (0.0010) [2023-12-26 22:25:57,225][105692] Updated weights for policy 0, policy_version 969772 (0.0006) [2023-12-26 22:25:57,278][105692] Updated weights for policy 0, policy_version 969782 (0.0006) [2023-12-26 22:25:57,327][105692] Updated weights for policy 0, policy_version 969792 (0.0008) [2023-12-26 22:25:57,469][105620] Updated weights for policy 1, policy_version 969806 (0.0007) [2023-12-26 22:25:57,527][105620] Updated weights for policy 1, policy_version 969816 (0.0005) [2023-12-26 22:25:57,587][105620] Updated weights for policy 1, policy_version 969826 (0.0005) [2023-12-26 22:25:57,935][105692] Updated weights for policy 0, policy_version 969802 (0.0007) [2023-12-26 22:25:57,996][105692] Updated weights for policy 0, policy_version 969812 (0.0007) [2023-12-26 22:25:58,047][105692] Updated weights for policy 0, policy_version 969822 (0.0007) [2023-12-26 22:25:58,102][105692] Updated weights for policy 0, policy_version 969832 (0.0008) [2023-12-26 22:25:58,241][105620] Updated weights for policy 1, policy_version 969836 (0.0007) [2023-12-26 22:25:58,304][105620] Updated weights for policy 1, policy_version 969846 (0.0011) [2023-12-26 22:25:58,377][105620] Updated weights for policy 1, policy_version 969856 (0.0010) [2023-12-26 22:25:58,949][105692] Updated weights for policy 0, policy_version 969842 (0.0010) [2023-12-26 22:25:59,012][105692] Updated weights for policy 0, policy_version 969852 (0.0009) [2023-12-26 22:25:59,013][105585] KL-divergence is very high: 136.2743 [2023-12-26 22:25:59,055][105585] KL-divergence is very high: 135.0973 [2023-12-26 22:25:59,064][105692] Updated weights for policy 0, policy_version 969862 (0.0008) [2023-12-26 22:25:59,150][105620] Updated weights for policy 1, policy_version 969866 (0.0011) [2023-12-26 22:25:59,201][105620] Updated weights for policy 1, policy_version 969876 (0.0009) [2023-12-26 22:25:59,270][105620] Updated weights for policy 1, policy_version 969886 (0.0009) [2023-12-26 22:25:59,336][105620] Updated weights for policy 1, policy_version 969896 (0.0010) [2023-12-26 22:25:59,846][105692] Updated weights for policy 0, policy_version 969872 (0.0009) [2023-12-26 22:25:59,908][105692] Updated weights for policy 0, policy_version 969882 (0.0008) [2023-12-26 22:25:59,974][105692] Updated weights for policy 0, policy_version 969892 (0.0008) [2023-12-26 22:26:00,020][105620] Updated weights for policy 1, policy_version 969906 (0.0011) [2023-12-26 22:26:00,079][105620] Updated weights for policy 1, policy_version 969916 (0.0010) [2023-12-26 22:26:00,138][105620] Updated weights for policy 1, policy_version 969926 (0.0008) [2023-12-26 22:26:00,616][105692] Updated weights for policy 0, policy_version 969902 (0.0006) [2023-12-26 22:26:00,670][105692] Updated weights for policy 0, policy_version 969912 (0.0006) [2023-12-26 22:26:00,717][105692] Updated weights for policy 0, policy_version 969922 (0.0006) [2023-12-26 22:26:00,906][105620] Updated weights for policy 1, policy_version 969936 (0.0008) [2023-12-26 22:26:00,959][105620] Updated weights for policy 1, policy_version 969947 (0.0009) [2023-12-26 22:26:01,002][105620] Updated weights for policy 1, policy_version 969957 (0.0007) [2023-12-26 22:26:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19327.6). Total num frames: 496680960. Throughput: 0: 9834.4, 1: 9692.4. Samples: 496647420. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:26:01,062][104569] Avg episode reward: [(0, '8102.998'), (1, '8053.216')] [2023-12-26 22:26:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000969928_248340480.pth... [2023-12-26 22:26:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000969960_248340480.pth... [2023-12-26 22:26:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000968776_248045568.pth [2023-12-26 22:26:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000968808_248045568.pth [2023-12-26 22:26:01,457][105692] Updated weights for policy 0, policy_version 969932 (0.0009) [2023-12-26 22:26:01,509][105692] Updated weights for policy 0, policy_version 969942 (0.0010) [2023-12-26 22:26:01,565][105692] Updated weights for policy 0, policy_version 969952 (0.0010) [2023-12-26 22:26:01,807][105620] Updated weights for policy 1, policy_version 969967 (0.0009) [2023-12-26 22:26:01,864][105620] Updated weights for policy 1, policy_version 969977 (0.0010) [2023-12-26 22:26:01,926][105620] Updated weights for policy 1, policy_version 969987 (0.0009) [2023-12-26 22:26:02,336][105692] Updated weights for policy 0, policy_version 969962 (0.0009) [2023-12-26 22:26:02,406][105692] Updated weights for policy 0, policy_version 969972 (0.0010) [2023-12-26 22:26:02,479][105692] Updated weights for policy 0, policy_version 969982 (0.0009) [2023-12-26 22:26:02,551][105692] Updated weights for policy 0, policy_version 969992 (0.0010) [2023-12-26 22:26:02,616][105620] Updated weights for policy 1, policy_version 969997 (0.0009) [2023-12-26 22:26:02,673][105620] Updated weights for policy 1, policy_version 970007 (0.0009) [2023-12-26 22:26:02,731][105620] Updated weights for policy 1, policy_version 970017 (0.0009) [2023-12-26 22:26:03,291][105692] Updated weights for policy 0, policy_version 970002 (0.0009) [2023-12-26 22:26:03,353][105692] Updated weights for policy 0, policy_version 970012 (0.0009) [2023-12-26 22:26:03,419][105692] Updated weights for policy 0, policy_version 970022 (0.0008) [2023-12-26 22:26:03,498][105620] Updated weights for policy 1, policy_version 970027 (0.0009) [2023-12-26 22:26:03,557][105620] Updated weights for policy 1, policy_version 970037 (0.0010) [2023-12-26 22:26:03,619][105620] Updated weights for policy 1, policy_version 970047 (0.0009) [2023-12-26 22:26:04,230][105692] Updated weights for policy 0, policy_version 970032 (0.0010) [2023-12-26 22:26:04,290][105692] Updated weights for policy 0, policy_version 970042 (0.0008) [2023-12-26 22:26:04,297][105620] Updated weights for policy 1, policy_version 970057 (0.0008) [2023-12-26 22:26:04,347][105620] Updated weights for policy 1, policy_version 970067 (0.0006) [2023-12-26 22:26:04,351][105692] Updated weights for policy 0, policy_version 970052 (0.0009) [2023-12-26 22:26:04,406][105620] Updated weights for policy 1, policy_version 970077 (0.0008) [2023-12-26 22:26:04,458][105620] Updated weights for policy 1, policy_version 970087 (0.0009) [2023-12-26 22:26:05,115][105692] Updated weights for policy 0, policy_version 970062 (0.0010) [2023-12-26 22:26:05,165][105692] Updated weights for policy 0, policy_version 970072 (0.0010) [2023-12-26 22:26:05,212][105692] Updated weights for policy 0, policy_version 970082 (0.0005) [2023-12-26 22:26:05,214][105620] Updated weights for policy 1, policy_version 970097 (0.0006) [2023-12-26 22:26:05,266][105620] Updated weights for policy 1, policy_version 970107 (0.0006) [2023-12-26 22:26:05,315][105620] Updated weights for policy 1, policy_version 970117 (0.0006) [2023-12-26 22:26:05,825][105692] Updated weights for policy 0, policy_version 970092 (0.0008) [2023-12-26 22:26:05,886][105692] Updated weights for policy 0, policy_version 970102 (0.0009) [2023-12-26 22:26:05,951][105692] Updated weights for policy 0, policy_version 970112 (0.0009) [2023-12-26 22:26:06,053][105620] Updated weights for policy 1, policy_version 970127 (0.0009) [2023-12-26 22:26:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 496771072. Throughput: 0: 9799.5, 1: 9667.6. Samples: 496758728. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:26:06,062][104569] Avg episode reward: [(0, '7655.606'), (1, '8118.080')] [2023-12-26 22:26:06,114][105620] Updated weights for policy 1, policy_version 970137 (0.0009) [2023-12-26 22:26:06,183][105620] Updated weights for policy 1, policy_version 970147 (0.0009) [2023-12-26 22:26:06,725][105692] Updated weights for policy 0, policy_version 970122 (0.0009) [2023-12-26 22:26:06,792][105692] Updated weights for policy 0, policy_version 970132 (0.0009) [2023-12-26 22:26:06,860][105692] Updated weights for policy 0, policy_version 970142 (0.0007) [2023-12-26 22:26:06,925][105692] Updated weights for policy 0, policy_version 970152 (0.0006) [2023-12-26 22:26:06,926][105620] Updated weights for policy 1, policy_version 970157 (0.0010) [2023-12-26 22:26:06,982][105620] Updated weights for policy 1, policy_version 970167 (0.0009) [2023-12-26 22:26:07,044][105620] Updated weights for policy 1, policy_version 970177 (0.0008) [2023-12-26 22:26:07,637][105692] Updated weights for policy 0, policy_version 970162 (0.0011) [2023-12-26 22:26:07,696][105692] Updated weights for policy 0, policy_version 970172 (0.0010) [2023-12-26 22:26:07,715][105620] Updated weights for policy 1, policy_version 970187 (0.0008) [2023-12-26 22:26:07,752][105692] Updated weights for policy 0, policy_version 970182 (0.0010) [2023-12-26 22:26:07,769][105620] Updated weights for policy 1, policy_version 970197 (0.0007) [2023-12-26 22:26:07,839][105620] Updated weights for policy 1, policy_version 970207 (0.0005) [2023-12-26 22:26:08,436][105620] Updated weights for policy 1, policy_version 970217 (0.0006) [2023-12-26 22:26:08,500][105620] Updated weights for policy 1, policy_version 970227 (0.0011) [2023-12-26 22:26:08,506][105692] Updated weights for policy 0, policy_version 970192 (0.0011) [2023-12-26 22:26:08,559][105620] Updated weights for policy 1, policy_version 970237 (0.0011) [2023-12-26 22:26:08,559][105692] Updated weights for policy 0, policy_version 970202 (0.0010) [2023-12-26 22:26:08,612][105692] Updated weights for policy 0, policy_version 970212 (0.0010) [2023-12-26 22:26:08,622][105620] Updated weights for policy 1, policy_version 970247 (0.0011) [2023-12-26 22:26:09,178][105620] Updated weights for policy 1, policy_version 970257 (0.0006) [2023-12-26 22:26:09,237][105620] Updated weights for policy 1, policy_version 970267 (0.0008) [2023-12-26 22:26:09,294][105620] Updated weights for policy 1, policy_version 970277 (0.0011) [2023-12-26 22:26:09,407][105692] Updated weights for policy 0, policy_version 970222 (0.0010) [2023-12-26 22:26:09,431][105585] KL-divergence is very high: 191.5264 [2023-12-26 22:26:09,467][105692] Updated weights for policy 0, policy_version 970232 (0.0010) [2023-12-26 22:26:09,481][105585] KL-divergence is very high: 371.8488 [2023-12-26 22:26:09,524][105585] KL-divergence is very high: 439.9160 [2023-12-26 22:26:09,525][105692] Updated weights for policy 0, policy_version 970242 (0.0009) [2023-12-26 22:26:10,027][105620] Updated weights for policy 1, policy_version 970287 (0.0008) [2023-12-26 22:26:10,098][105620] Updated weights for policy 1, policy_version 970297 (0.0006) [2023-12-26 22:26:10,163][105620] Updated weights for policy 1, policy_version 970307 (0.0007) [2023-12-26 22:26:10,288][105692] Updated weights for policy 0, policy_version 970252 (0.0008) [2023-12-26 22:26:10,358][105692] Updated weights for policy 0, policy_version 970262 (0.0009) [2023-12-26 22:26:10,422][105692] Updated weights for policy 0, policy_version 970272 (0.0011) [2023-12-26 22:26:10,892][105620] Updated weights for policy 1, policy_version 970317 (0.0008) [2023-12-26 22:26:10,956][105620] Updated weights for policy 1, policy_version 970327 (0.0009) [2023-12-26 22:26:11,012][105620] Updated weights for policy 1, policy_version 970337 (0.0008) [2023-12-26 22:26:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19355.3). Total num frames: 496869376. Throughput: 0: 9774.3, 1: 9722.9. Samples: 496875964. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:26:11,062][104569] Avg episode reward: [(0, '8367.584'), (1, '8391.169')] [2023-12-26 22:26:11,118][105692] Updated weights for policy 0, policy_version 970282 (0.0009) [2023-12-26 22:26:11,186][105692] Updated weights for policy 0, policy_version 970292 (0.0009) [2023-12-26 22:26:11,251][105692] Updated weights for policy 0, policy_version 970302 (0.0008) [2023-12-26 22:26:11,319][105692] Updated weights for policy 0, policy_version 970312 (0.0008) [2023-12-26 22:26:11,830][105620] Updated weights for policy 1, policy_version 970347 (0.0008) [2023-12-26 22:26:11,889][105620] Updated weights for policy 1, policy_version 970357 (0.0009) [2023-12-26 22:26:11,951][105620] Updated weights for policy 1, policy_version 970367 (0.0010) [2023-12-26 22:26:12,158][105692] Updated weights for policy 0, policy_version 970322 (0.0010) [2023-12-26 22:26:12,222][105692] Updated weights for policy 0, policy_version 970332 (0.0009) [2023-12-26 22:26:12,290][105692] Updated weights for policy 0, policy_version 970342 (0.0009) [2023-12-26 22:26:12,678][105620] Updated weights for policy 1, policy_version 970377 (0.0009) [2023-12-26 22:26:12,737][105620] Updated weights for policy 1, policy_version 970387 (0.0008) [2023-12-26 22:26:12,796][105620] Updated weights for policy 1, policy_version 970397 (0.0006) [2023-12-26 22:26:12,846][105620] Updated weights for policy 1, policy_version 970407 (0.0005) [2023-12-26 22:26:13,055][105692] Updated weights for policy 0, policy_version 970352 (0.0009) [2023-12-26 22:26:13,120][105692] Updated weights for policy 0, policy_version 970362 (0.0009) [2023-12-26 22:26:13,184][105692] Updated weights for policy 0, policy_version 970372 (0.0008) [2023-12-26 22:26:13,495][105620] Updated weights for policy 1, policy_version 970417 (0.0010) [2023-12-26 22:26:13,553][105620] Updated weights for policy 1, policy_version 970427 (0.0010) [2023-12-26 22:26:13,616][105620] Updated weights for policy 1, policy_version 970437 (0.0009) [2023-12-26 22:26:13,866][105692] Updated weights for policy 0, policy_version 970382 (0.0007) [2023-12-26 22:26:13,937][105692] Updated weights for policy 0, policy_version 970392 (0.0005) [2023-12-26 22:26:14,009][105692] Updated weights for policy 0, policy_version 970402 (0.0005) [2023-12-26 22:26:14,372][105620] Updated weights for policy 1, policy_version 970447 (0.0008) [2023-12-26 22:26:14,438][105620] Updated weights for policy 1, policy_version 970457 (0.0008) [2023-12-26 22:26:14,500][105620] Updated weights for policy 1, policy_version 970467 (0.0008) [2023-12-26 22:26:14,584][105692] Updated weights for policy 0, policy_version 970412 (0.0008) [2023-12-26 22:26:14,642][105692] Updated weights for policy 0, policy_version 970422 (0.0010) [2023-12-26 22:26:14,698][105692] Updated weights for policy 0, policy_version 970432 (0.0010) [2023-12-26 22:26:15,156][105620] Updated weights for policy 1, policy_version 970477 (0.0007) [2023-12-26 22:26:15,232][105620] Updated weights for policy 1, policy_version 970487 (0.0007) [2023-12-26 22:26:15,293][105620] Updated weights for policy 1, policy_version 970497 (0.0008) [2023-12-26 22:26:15,459][105692] Updated weights for policy 0, policy_version 970442 (0.0009) [2023-12-26 22:26:15,529][105692] Updated weights for policy 0, policy_version 970452 (0.0005) [2023-12-26 22:26:15,582][105692] Updated weights for policy 0, policy_version 970462 (0.0006) [2023-12-26 22:26:15,633][105692] Updated weights for policy 0, policy_version 970472 (0.0005) [2023-12-26 22:26:15,974][105620] Updated weights for policy 1, policy_version 970507 (0.0008) [2023-12-26 22:26:16,024][105620] Updated weights for policy 1, policy_version 970517 (0.0007) [2023-12-26 22:26:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19327.6). Total num frames: 496959488. Throughput: 0: 9724.5, 1: 9652.9. Samples: 496931664. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:26:16,063][104569] Avg episode reward: [(0, '8313.254'), (1, '8835.748')] [2023-12-26 22:26:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000970472_248479744.pth... [2023-12-26 22:26:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000969352_248193024.pth [2023-12-26 22:26:16,083][105620] Updated weights for policy 1, policy_version 970527 (0.0007) [2023-12-26 22:26:16,130][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000970536_248487936.pth... [2023-12-26 22:26:16,133][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000969384_248193024.pth [2023-12-26 22:26:16,231][105692] Updated weights for policy 0, policy_version 970482 (0.0010) [2023-12-26 22:26:16,286][105692] Updated weights for policy 0, policy_version 970492 (0.0010) [2023-12-26 22:26:16,344][105692] Updated weights for policy 0, policy_version 970502 (0.0010) [2023-12-26 22:26:16,842][105620] Updated weights for policy 1, policy_version 970537 (0.0008) [2023-12-26 22:26:16,898][105620] Updated weights for policy 1, policy_version 970547 (0.0008) [2023-12-26 22:26:16,951][105620] Updated weights for policy 1, policy_version 970557 (0.0008) [2023-12-26 22:26:16,996][105620] Updated weights for policy 1, policy_version 970567 (0.0008) [2023-12-26 22:26:17,101][105692] Updated weights for policy 0, policy_version 970512 (0.0010) [2023-12-26 22:26:17,166][105692] Updated weights for policy 0, policy_version 970522 (0.0010) [2023-12-26 22:26:17,230][105692] Updated weights for policy 0, policy_version 970532 (0.0010) [2023-12-26 22:26:17,767][105620] Updated weights for policy 1, policy_version 970577 (0.0007) [2023-12-26 22:26:17,816][105620] Updated weights for policy 1, policy_version 970587 (0.0008) [2023-12-26 22:26:17,889][105620] Updated weights for policy 1, policy_version 970597 (0.0008) [2023-12-26 22:26:17,960][105692] Updated weights for policy 0, policy_version 970542 (0.0010) [2023-12-26 22:26:18,009][105692] Updated weights for policy 0, policy_version 970552 (0.0010) [2023-12-26 22:26:18,064][105692] Updated weights for policy 0, policy_version 970562 (0.0010) [2023-12-26 22:26:18,617][105620] Updated weights for policy 1, policy_version 970607 (0.0008) [2023-12-26 22:26:18,680][105620] Updated weights for policy 1, policy_version 970617 (0.0006) [2023-12-26 22:26:18,750][105620] Updated weights for policy 1, policy_version 970627 (0.0007) [2023-12-26 22:26:18,795][105692] Updated weights for policy 0, policy_version 970572 (0.0010) [2023-12-26 22:26:18,853][105692] Updated weights for policy 0, policy_version 970582 (0.0011) [2023-12-26 22:26:18,916][105692] Updated weights for policy 0, policy_version 970592 (0.0011) [2023-12-26 22:26:19,476][105620] Updated weights for policy 1, policy_version 970637 (0.0006) [2023-12-26 22:26:19,543][105620] Updated weights for policy 1, policy_version 970647 (0.0011) [2023-12-26 22:26:19,609][105620] Updated weights for policy 1, policy_version 970657 (0.0011) [2023-12-26 22:26:19,659][105692] Updated weights for policy 0, policy_version 970602 (0.0009) [2023-12-26 22:26:19,705][105585] KL-divergence is very high: 188.3676 [2023-12-26 22:26:19,717][105692] Updated weights for policy 0, policy_version 970612 (0.0008) [2023-12-26 22:26:19,749][105585] KL-divergence is very high: 331.0952 [2023-12-26 22:26:19,772][105692] Updated weights for policy 0, policy_version 970622 (0.0006) [2023-12-26 22:26:19,797][105585] KL-divergence is very high: 327.2660 [2023-12-26 22:26:19,835][105692] Updated weights for policy 0, policy_version 970632 (0.0007) [2023-12-26 22:26:20,378][105620] Updated weights for policy 1, policy_version 970667 (0.0011) [2023-12-26 22:26:20,434][105620] Updated weights for policy 1, policy_version 970677 (0.0010) [2023-12-26 22:26:20,493][105620] Updated weights for policy 1, policy_version 970687 (0.0011) [2023-12-26 22:26:20,628][105692] Updated weights for policy 0, policy_version 970642 (0.0011) [2023-12-26 22:26:20,689][105692] Updated weights for policy 0, policy_version 970652 (0.0011) [2023-12-26 22:26:20,756][105692] Updated weights for policy 0, policy_version 970662 (0.0011) [2023-12-26 22:26:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19327.6). Total num frames: 497057792. Throughput: 0: 9694.5, 1: 9712.9. Samples: 497048644. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:26:21,062][104569] Avg episode reward: [(0, '8131.154'), (1, '9011.820')] [2023-12-26 22:26:21,263][105620] Updated weights for policy 1, policy_version 970697 (0.0010) [2023-12-26 22:26:21,327][105620] Updated weights for policy 1, policy_version 970707 (0.0011) [2023-12-26 22:26:21,397][105620] Updated weights for policy 1, policy_version 970717 (0.0008) [2023-12-26 22:26:21,461][105620] Updated weights for policy 1, policy_version 970727 (0.0008) [2023-12-26 22:26:21,508][105692] Updated weights for policy 0, policy_version 970672 (0.0011) [2023-12-26 22:26:21,564][105692] Updated weights for policy 0, policy_version 970682 (0.0009) [2023-12-26 22:26:21,625][105692] Updated weights for policy 0, policy_version 970692 (0.0010) [2023-12-26 22:26:22,239][105620] Updated weights for policy 1, policy_version 970737 (0.0010) [2023-12-26 22:26:22,306][105620] Updated weights for policy 1, policy_version 970747 (0.0009) [2023-12-26 22:26:22,369][105692] Updated weights for policy 0, policy_version 970702 (0.0009) [2023-12-26 22:26:22,373][105620] Updated weights for policy 1, policy_version 970757 (0.0008) [2023-12-26 22:26:22,434][105692] Updated weights for policy 0, policy_version 970712 (0.0010) [2023-12-26 22:26:22,487][105692] Updated weights for policy 0, policy_version 970722 (0.0009) [2023-12-26 22:26:23,151][105620] Updated weights for policy 1, policy_version 970767 (0.0009) [2023-12-26 22:26:23,213][105620] Updated weights for policy 1, policy_version 970777 (0.0009) [2023-12-26 22:26:23,272][105692] Updated weights for policy 0, policy_version 970732 (0.0009) [2023-12-26 22:26:23,274][105620] Updated weights for policy 1, policy_version 970787 (0.0008) [2023-12-26 22:26:23,321][105692] Updated weights for policy 0, policy_version 970742 (0.0007) [2023-12-26 22:26:23,376][105692] Updated weights for policy 0, policy_version 970753 (0.0010) [2023-12-26 22:26:23,970][105620] Updated weights for policy 1, policy_version 970797 (0.0008) [2023-12-26 22:26:24,032][105620] Updated weights for policy 1, policy_version 970807 (0.0008) [2023-12-26 22:26:24,089][105620] Updated weights for policy 1, policy_version 970817 (0.0006) [2023-12-26 22:26:24,198][105692] Updated weights for policy 0, policy_version 970763 (0.0009) [2023-12-26 22:26:24,256][105692] Updated weights for policy 0, policy_version 970773 (0.0009) [2023-12-26 22:26:24,313][105692] Updated weights for policy 0, policy_version 970783 (0.0009) [2023-12-26 22:26:24,662][105620] Updated weights for policy 1, policy_version 970827 (0.0006) [2023-12-26 22:26:24,716][105620] Updated weights for policy 1, policy_version 970837 (0.0005) [2023-12-26 22:26:24,771][105620] Updated weights for policy 1, policy_version 970847 (0.0005) [2023-12-26 22:26:24,992][105692] Updated weights for policy 0, policy_version 970793 (0.0008) [2023-12-26 22:26:25,047][105692] Updated weights for policy 0, policy_version 970803 (0.0005) [2023-12-26 22:26:25,101][105692] Updated weights for policy 0, policy_version 970813 (0.0005) [2023-12-26 22:26:25,157][105692] Updated weights for policy 0, policy_version 970823 (0.0005) [2023-12-26 22:26:25,416][105620] Updated weights for policy 1, policy_version 970857 (0.0006) [2023-12-26 22:26:25,477][105620] Updated weights for policy 1, policy_version 970867 (0.0010) [2023-12-26 22:26:25,525][105620] Updated weights for policy 1, policy_version 970877 (0.0010) [2023-12-26 22:26:25,569][105620] Updated weights for policy 1, policy_version 970887 (0.0010) [2023-12-26 22:26:25,706][105692] Updated weights for policy 0, policy_version 970833 (0.0010) [2023-12-26 22:26:25,760][105692] Updated weights for policy 0, policy_version 970843 (0.0007) [2023-12-26 22:26:25,816][105692] Updated weights for policy 0, policy_version 970853 (0.0005) [2023-12-26 22:26:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19299.8). Total num frames: 497156096. Throughput: 0: 9651.8, 1: 9635.2. Samples: 497163828. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:26:26,063][104569] Avg episode reward: [(0, '8547.413'), (1, '8731.155')] [2023-12-26 22:26:26,330][105620] Updated weights for policy 1, policy_version 970897 (0.0010) [2023-12-26 22:26:26,385][105620] Updated weights for policy 1, policy_version 970907 (0.0010) [2023-12-26 22:26:26,444][105620] Updated weights for policy 1, policy_version 970917 (0.0010) [2023-12-26 22:26:26,540][105692] Updated weights for policy 0, policy_version 970863 (0.0008) [2023-12-26 22:26:26,599][105692] Updated weights for policy 0, policy_version 970873 (0.0008) [2023-12-26 22:26:26,651][105692] Updated weights for policy 0, policy_version 970883 (0.0008) [2023-12-26 22:26:27,185][105620] Updated weights for policy 1, policy_version 970927 (0.0010) [2023-12-26 22:26:27,240][105620] Updated weights for policy 1, policy_version 970937 (0.0010) [2023-12-26 22:26:27,300][105620] Updated weights for policy 1, policy_version 970947 (0.0011) [2023-12-26 22:26:27,404][105692] Updated weights for policy 0, policy_version 970893 (0.0007) [2023-12-26 22:26:27,460][105692] Updated weights for policy 0, policy_version 970903 (0.0008) [2023-12-26 22:26:27,520][105692] Updated weights for policy 0, policy_version 970913 (0.0008) [2023-12-26 22:26:27,975][105620] Updated weights for policy 1, policy_version 970957 (0.0010) [2023-12-26 22:26:28,026][105620] Updated weights for policy 1, policy_version 970967 (0.0010) [2023-12-26 22:26:28,080][105620] Updated weights for policy 1, policy_version 970977 (0.0010) [2023-12-26 22:26:28,323][105692] Updated weights for policy 0, policy_version 970923 (0.0008) [2023-12-26 22:26:28,388][105692] Updated weights for policy 0, policy_version 970933 (0.0009) [2023-12-26 22:26:28,451][105692] Updated weights for policy 0, policy_version 970943 (0.0009) [2023-12-26 22:26:28,828][105620] Updated weights for policy 1, policy_version 970987 (0.0010) [2023-12-26 22:26:28,882][105620] Updated weights for policy 1, policy_version 970997 (0.0010) [2023-12-26 22:26:28,935][105620] Updated weights for policy 1, policy_version 971007 (0.0008) [2023-12-26 22:26:29,057][105692] Updated weights for policy 0, policy_version 970953 (0.0009) [2023-12-26 22:26:29,121][105692] Updated weights for policy 0, policy_version 970963 (0.0005) [2023-12-26 22:26:29,188][105692] Updated weights for policy 0, policy_version 970973 (0.0006) [2023-12-26 22:26:29,253][105692] Updated weights for policy 0, policy_version 970983 (0.0008) [2023-12-26 22:26:29,719][105620] Updated weights for policy 1, policy_version 971017 (0.0010) [2023-12-26 22:26:29,785][105620] Updated weights for policy 1, policy_version 971027 (0.0011) [2023-12-26 22:26:29,848][105620] Updated weights for policy 1, policy_version 971037 (0.0011) [2023-12-26 22:26:29,909][105620] Updated weights for policy 1, policy_version 971047 (0.0011) [2023-12-26 22:26:29,911][105692] Updated weights for policy 0, policy_version 970993 (0.0006) [2023-12-26 22:26:29,970][105692] Updated weights for policy 0, policy_version 971003 (0.0008) [2023-12-26 22:26:30,033][105692] Updated weights for policy 0, policy_version 971013 (0.0008) [2023-12-26 22:26:30,643][105620] Updated weights for policy 1, policy_version 971057 (0.0009) [2023-12-26 22:26:30,690][105620] Updated weights for policy 1, policy_version 971067 (0.0008) [2023-12-26 22:26:30,739][105692] Updated weights for policy 0, policy_version 971023 (0.0010) [2023-12-26 22:26:30,743][105620] Updated weights for policy 1, policy_version 971077 (0.0009) [2023-12-26 22:26:30,794][105692] Updated weights for policy 0, policy_version 971033 (0.0007) [2023-12-26 22:26:30,861][105692] Updated weights for policy 0, policy_version 971043 (0.0006) [2023-12-26 22:26:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19299.8). Total num frames: 497254400. Throughput: 0: 9609.2, 1: 9636.7. Samples: 497219980. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:26:31,063][104569] Avg episode reward: [(0, '8724.332'), (1, '8477.474')] [2023-12-26 22:26:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000971048_248627200.pth... [2023-12-26 22:26:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000971080_248627200.pth... [2023-12-26 22:26:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000969928_248340480.pth [2023-12-26 22:26:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000969960_248340480.pth [2023-12-26 22:26:31,469][105620] Updated weights for policy 1, policy_version 971087 (0.0009) [2023-12-26 22:26:31,520][105692] Updated weights for policy 0, policy_version 971053 (0.0005) [2023-12-26 22:26:31,528][105620] Updated weights for policy 1, policy_version 971097 (0.0009) [2023-12-26 22:26:31,577][105692] Updated weights for policy 0, policy_version 971063 (0.0006) [2023-12-26 22:26:31,578][105620] Updated weights for policy 1, policy_version 971107 (0.0008) [2023-12-26 22:26:31,646][105692] Updated weights for policy 0, policy_version 971073 (0.0006) [2023-12-26 22:26:32,298][105620] Updated weights for policy 1, policy_version 971117 (0.0009) [2023-12-26 22:26:32,334][105692] Updated weights for policy 0, policy_version 971083 (0.0007) [2023-12-26 22:26:32,356][105620] Updated weights for policy 1, policy_version 971127 (0.0008) [2023-12-26 22:26:32,381][105692] Updated weights for policy 0, policy_version 971093 (0.0007) [2023-12-26 22:26:32,420][105620] Updated weights for policy 1, policy_version 971137 (0.0008) [2023-12-26 22:26:32,446][105692] Updated weights for policy 0, policy_version 971103 (0.0008) [2023-12-26 22:26:33,132][105692] Updated weights for policy 0, policy_version 971113 (0.0008) [2023-12-26 22:26:33,186][105692] Updated weights for policy 0, policy_version 971123 (0.0005) [2023-12-26 22:26:33,232][105620] Updated weights for policy 1, policy_version 971147 (0.0008) [2023-12-26 22:26:33,240][105692] Updated weights for policy 0, policy_version 971133 (0.0005) [2023-12-26 22:26:33,292][105692] Updated weights for policy 0, policy_version 971143 (0.0007) [2023-12-26 22:26:33,296][105620] Updated weights for policy 1, policy_version 971157 (0.0008) [2023-12-26 22:26:33,365][105620] Updated weights for policy 1, policy_version 971167 (0.0009) [2023-12-26 22:26:33,913][105692] Updated weights for policy 0, policy_version 971153 (0.0006) [2023-12-26 22:26:33,973][105692] Updated weights for policy 0, policy_version 971163 (0.0007) [2023-12-26 22:26:34,025][105692] Updated weights for policy 0, policy_version 971173 (0.0006) [2023-12-26 22:26:34,173][105620] Updated weights for policy 1, policy_version 971177 (0.0008) [2023-12-26 22:26:34,226][105620] Updated weights for policy 1, policy_version 971187 (0.0010) [2023-12-26 22:26:34,287][105620] Updated weights for policy 1, policy_version 971197 (0.0010) [2023-12-26 22:26:34,346][105620] Updated weights for policy 1, policy_version 971207 (0.0009) [2023-12-26 22:26:34,640][105692] Updated weights for policy 0, policy_version 971183 (0.0006) [2023-12-26 22:26:34,699][105692] Updated weights for policy 0, policy_version 971193 (0.0006) [2023-12-26 22:26:34,759][105692] Updated weights for policy 0, policy_version 971203 (0.0006) [2023-12-26 22:26:35,080][105620] Updated weights for policy 1, policy_version 971217 (0.0006) [2023-12-26 22:26:35,147][105620] Updated weights for policy 1, policy_version 971227 (0.0005) [2023-12-26 22:26:35,206][105620] Updated weights for policy 1, policy_version 971237 (0.0005) [2023-12-26 22:26:35,423][105692] Updated weights for policy 0, policy_version 971213 (0.0008) [2023-12-26 22:26:35,478][105692] Updated weights for policy 0, policy_version 971223 (0.0010) [2023-12-26 22:26:35,526][105692] Updated weights for policy 0, policy_version 971233 (0.0010) [2023-12-26 22:26:35,788][105620] Updated weights for policy 1, policy_version 971247 (0.0006) [2023-12-26 22:26:35,851][105620] Updated weights for policy 1, policy_version 971257 (0.0006) [2023-12-26 22:26:35,908][105620] Updated weights for policy 1, policy_version 971267 (0.0010) [2023-12-26 22:26:36,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.7, 300 sec: 19299.8). Total num frames: 497352704. Throughput: 0: 9722.0, 1: 9565.0. Samples: 497338612. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:26:36,062][104569] Avg episode reward: [(0, '8907.457'), (1, '8660.591')] [2023-12-26 22:26:36,150][105692] Updated weights for policy 0, policy_version 971243 (0.0010) [2023-12-26 22:26:36,201][105692] Updated weights for policy 0, policy_version 971253 (0.0006) [2023-12-26 22:26:36,265][105692] Updated weights for policy 0, policy_version 971263 (0.0006) [2023-12-26 22:26:36,728][105620] Updated weights for policy 1, policy_version 971277 (0.0009) [2023-12-26 22:26:36,780][105620] Updated weights for policy 1, policy_version 971287 (0.0009) [2023-12-26 22:26:36,833][105620] Updated weights for policy 1, policy_version 971297 (0.0010) [2023-12-26 22:26:36,863][105692] Updated weights for policy 0, policy_version 971273 (0.0006) [2023-12-26 22:26:36,931][105692] Updated weights for policy 0, policy_version 971283 (0.0007) [2023-12-26 22:26:36,999][105692] Updated weights for policy 0, policy_version 971293 (0.0009) [2023-12-26 22:26:37,060][105692] Updated weights for policy 0, policy_version 971303 (0.0008) [2023-12-26 22:26:37,659][105620] Updated weights for policy 1, policy_version 971307 (0.0009) [2023-12-26 22:26:37,724][105692] Updated weights for policy 0, policy_version 971313 (0.0008) [2023-12-26 22:26:37,724][105620] Updated weights for policy 1, policy_version 971317 (0.0007) [2023-12-26 22:26:37,784][105692] Updated weights for policy 0, policy_version 971323 (0.0007) [2023-12-26 22:26:37,786][105620] Updated weights for policy 1, policy_version 971327 (0.0007) [2023-12-26 22:26:37,843][105692] Updated weights for policy 0, policy_version 971333 (0.0006) [2023-12-26 22:26:38,519][105620] Updated weights for policy 1, policy_version 971337 (0.0008) [2023-12-26 22:26:38,576][105620] Updated weights for policy 1, policy_version 971347 (0.0010) [2023-12-26 22:26:38,591][105692] Updated weights for policy 0, policy_version 971343 (0.0005) [2023-12-26 22:26:38,635][105620] Updated weights for policy 1, policy_version 971357 (0.0010) [2023-12-26 22:26:38,644][105692] Updated weights for policy 0, policy_version 971353 (0.0007) [2023-12-26 22:26:38,686][105620] Updated weights for policy 1, policy_version 971367 (0.0009) [2023-12-26 22:26:38,701][105692] Updated weights for policy 0, policy_version 971363 (0.0009) [2023-12-26 22:26:39,345][105692] Updated weights for policy 0, policy_version 971373 (0.0009) [2023-12-26 22:26:39,412][105692] Updated weights for policy 0, policy_version 971383 (0.0008) [2023-12-26 22:26:39,429][105620] Updated weights for policy 1, policy_version 971377 (0.0010) [2023-12-26 22:26:39,467][105692] Updated weights for policy 0, policy_version 971393 (0.0006) [2023-12-26 22:26:39,489][105620] Updated weights for policy 1, policy_version 971387 (0.0011) [2023-12-26 22:26:39,552][105620] Updated weights for policy 1, policy_version 971397 (0.0010) [2023-12-26 22:26:40,163][105692] Updated weights for policy 0, policy_version 971403 (0.0006) [2023-12-26 22:26:40,227][105692] Updated weights for policy 0, policy_version 971413 (0.0006) [2023-12-26 22:26:40,296][105692] Updated weights for policy 0, policy_version 971423 (0.0008) [2023-12-26 22:26:40,314][105620] Updated weights for policy 1, policy_version 971407 (0.0011) [2023-12-26 22:26:40,377][105620] Updated weights for policy 1, policy_version 971417 (0.0011) [2023-12-26 22:26:40,434][105620] Updated weights for policy 1, policy_version 971427 (0.0011) [2023-12-26 22:26:40,836][105692] Updated weights for policy 0, policy_version 971433 (0.0008) [2023-12-26 22:26:40,901][105692] Updated weights for policy 0, policy_version 971443 (0.0007) [2023-12-26 22:26:40,960][105692] Updated weights for policy 0, policy_version 971453 (0.0008) [2023-12-26 22:26:41,018][105692] Updated weights for policy 0, policy_version 971463 (0.0008) [2023-12-26 22:26:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19299.8). Total num frames: 497451008. Throughput: 0: 9783.5, 1: 9573.5. Samples: 497457720. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:26:41,063][104569] Avg episode reward: [(0, '9085.463'), (1, '9086.588')] [2023-12-26 22:26:41,125][105620] Updated weights for policy 1, policy_version 971437 (0.0010) [2023-12-26 22:26:41,195][105620] Updated weights for policy 1, policy_version 971447 (0.0009) [2023-12-26 22:26:41,262][105620] Updated weights for policy 1, policy_version 971457 (0.0009) [2023-12-26 22:26:41,808][105692] Updated weights for policy 0, policy_version 971473 (0.0008) [2023-12-26 22:26:41,869][105692] Updated weights for policy 0, policy_version 971483 (0.0009) [2023-12-26 22:26:41,926][105692] Updated weights for policy 0, policy_version 971493 (0.0008) [2023-12-26 22:26:42,022][105620] Updated weights for policy 1, policy_version 971467 (0.0011) [2023-12-26 22:26:42,089][105620] Updated weights for policy 1, policy_version 971477 (0.0011) [2023-12-26 22:26:42,151][105620] Updated weights for policy 1, policy_version 971487 (0.0007) [2023-12-26 22:26:42,737][105692] Updated weights for policy 0, policy_version 971503 (0.0008) [2023-12-26 22:26:42,787][105692] Updated weights for policy 0, policy_version 971513 (0.0009) [2023-12-26 22:26:42,839][105692] Updated weights for policy 0, policy_version 971523 (0.0008) [2023-12-26 22:26:42,894][105620] Updated weights for policy 1, policy_version 971497 (0.0006) [2023-12-26 22:26:42,958][105620] Updated weights for policy 1, policy_version 971507 (0.0008) [2023-12-26 22:26:43,012][105620] Updated weights for policy 1, policy_version 971517 (0.0009) [2023-12-26 22:26:43,073][105620] Updated weights for policy 1, policy_version 971527 (0.0008) [2023-12-26 22:26:43,547][105692] Updated weights for policy 0, policy_version 971533 (0.0009) [2023-12-26 22:26:43,604][105692] Updated weights for policy 0, policy_version 971543 (0.0009) [2023-12-26 22:26:43,667][105692] Updated weights for policy 0, policy_version 971553 (0.0010) [2023-12-26 22:26:43,696][105620] Updated weights for policy 1, policy_version 971537 (0.0006) [2023-12-26 22:26:43,754][105620] Updated weights for policy 1, policy_version 971547 (0.0005) [2023-12-26 22:26:43,807][105620] Updated weights for policy 1, policy_version 971557 (0.0005) [2023-12-26 22:26:44,338][105620] Updated weights for policy 1, policy_version 971567 (0.0005) [2023-12-26 22:26:44,392][105692] Updated weights for policy 0, policy_version 971563 (0.0008) [2023-12-26 22:26:44,406][105620] Updated weights for policy 1, policy_version 971577 (0.0007) [2023-12-26 22:26:44,455][105692] Updated weights for policy 0, policy_version 971573 (0.0005) [2023-12-26 22:26:44,466][105620] Updated weights for policy 1, policy_version 971587 (0.0008) [2023-12-26 22:26:44,512][105692] Updated weights for policy 0, policy_version 971583 (0.0005) [2023-12-26 22:26:45,110][105692] Updated weights for policy 0, policy_version 971593 (0.0006) [2023-12-26 22:26:45,172][105692] Updated weights for policy 0, policy_version 971603 (0.0009) [2023-12-26 22:26:45,231][105692] Updated weights for policy 0, policy_version 971613 (0.0008) [2023-12-26 22:26:45,245][105620] Updated weights for policy 1, policy_version 971597 (0.0009) [2023-12-26 22:26:45,295][105692] Updated weights for policy 0, policy_version 971623 (0.0005) [2023-12-26 22:26:45,308][105620] Updated weights for policy 1, policy_version 971607 (0.0010) [2023-12-26 22:26:45,378][105620] Updated weights for policy 1, policy_version 971617 (0.0010) [2023-12-26 22:26:46,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19251.2, 300 sec: 19299.8). Total num frames: 497541120. Throughput: 0: 9707.2, 1: 9602.1. Samples: 497516344. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:26:46,063][104569] Avg episode reward: [(0, '8903.852'), (1, '9171.939')] [2023-12-26 22:26:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000971624_248766464.pth... [2023-12-26 22:26:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000970536_248487936.pth [2023-12-26 22:26:46,080][105692] Updated weights for policy 0, policy_version 971633 (0.0008) [2023-12-26 22:26:46,122][105620] Updated weights for policy 1, policy_version 971627 (0.0011) [2023-12-26 22:26:46,137][105692] Updated weights for policy 0, policy_version 971643 (0.0008) [2023-12-26 22:26:46,173][105620] Updated weights for policy 1, policy_version 971637 (0.0010) [2023-12-26 22:26:46,184][105692] Updated weights for policy 0, policy_version 971653 (0.0007) [2023-12-26 22:26:46,197][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000971656_248782848.pth... [2023-12-26 22:26:46,201][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000970472_248479744.pth [2023-12-26 22:26:46,221][105620] Updated weights for policy 1, policy_version 971647 (0.0010) [2023-12-26 22:26:46,926][105620] Updated weights for policy 1, policy_version 971657 (0.0010) [2023-12-26 22:26:46,987][105692] Updated weights for policy 0, policy_version 971663 (0.0005) [2023-12-26 22:26:46,988][105620] Updated weights for policy 1, policy_version 971667 (0.0008) [2023-12-26 22:26:47,037][105620] Updated weights for policy 1, policy_version 971677 (0.0006) [2023-12-26 22:26:47,039][105692] Updated weights for policy 0, policy_version 971673 (0.0008) [2023-12-26 22:26:47,089][105692] Updated weights for policy 0, policy_version 971683 (0.0006) [2023-12-26 22:26:47,091][105620] Updated weights for policy 1, policy_version 971687 (0.0007) [2023-12-26 22:26:47,698][105692] Updated weights for policy 0, policy_version 971693 (0.0007) [2023-12-26 22:26:47,755][105692] Updated weights for policy 0, policy_version 971703 (0.0006) [2023-12-26 22:26:47,812][105692] Updated weights for policy 0, policy_version 971713 (0.0006) [2023-12-26 22:26:47,926][105620] Updated weights for policy 1, policy_version 971697 (0.0006) [2023-12-26 22:26:47,986][105620] Updated weights for policy 1, policy_version 971707 (0.0008) [2023-12-26 22:26:48,057][105620] Updated weights for policy 1, policy_version 971717 (0.0006) [2023-12-26 22:26:48,478][105692] Updated weights for policy 0, policy_version 971723 (0.0010) [2023-12-26 22:26:48,534][105692] Updated weights for policy 0, policy_version 971733 (0.0009) [2023-12-26 22:26:48,581][105692] Updated weights for policy 0, policy_version 971743 (0.0009) [2023-12-26 22:26:48,748][105620] Updated weights for policy 1, policy_version 971727 (0.0006) [2023-12-26 22:26:48,800][105620] Updated weights for policy 1, policy_version 971737 (0.0005) [2023-12-26 22:26:48,858][105620] Updated weights for policy 1, policy_version 971747 (0.0005) [2023-12-26 22:26:49,389][105692] Updated weights for policy 0, policy_version 971753 (0.0008) [2023-12-26 22:26:49,443][105692] Updated weights for policy 0, policy_version 971763 (0.0010) [2023-12-26 22:26:49,493][105620] Updated weights for policy 1, policy_version 971757 (0.0008) [2023-12-26 22:26:49,500][105692] Updated weights for policy 0, policy_version 971773 (0.0007) [2023-12-26 22:26:49,547][105692] Updated weights for policy 0, policy_version 971783 (0.0007) [2023-12-26 22:26:49,552][105620] Updated weights for policy 1, policy_version 971767 (0.0008) [2023-12-26 22:26:49,610][105620] Updated weights for policy 1, policy_version 971777 (0.0008) [2023-12-26 22:26:50,300][105620] Updated weights for policy 1, policy_version 971787 (0.0007) [2023-12-26 22:26:50,364][105692] Updated weights for policy 0, policy_version 971793 (0.0011) [2023-12-26 22:26:50,365][105620] Updated weights for policy 1, policy_version 971797 (0.0008) [2023-12-26 22:26:50,417][105692] Updated weights for policy 0, policy_version 971803 (0.0010) [2023-12-26 22:26:50,427][105620] Updated weights for policy 1, policy_version 971807 (0.0008) [2023-12-26 22:26:50,473][105692] Updated weights for policy 0, policy_version 971813 (0.0006) [2023-12-26 22:26:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19299.8). Total num frames: 497639424. Throughput: 0: 9798.7, 1: 9639.0. Samples: 497633424. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:26:51,063][104569] Avg episode reward: [(0, '8730.757'), (1, '9081.089')] [2023-12-26 22:26:51,098][105692] Updated weights for policy 0, policy_version 971823 (0.0007) [2023-12-26 22:26:51,166][105692] Updated weights for policy 0, policy_version 971833 (0.0010) [2023-12-26 22:26:51,207][105620] Updated weights for policy 1, policy_version 971817 (0.0006) [2023-12-26 22:26:51,222][105692] Updated weights for policy 0, policy_version 971843 (0.0010) [2023-12-26 22:26:51,268][105620] Updated weights for policy 1, policy_version 971827 (0.0009) [2023-12-26 22:26:51,325][105620] Updated weights for policy 1, policy_version 971837 (0.0010) [2023-12-26 22:26:51,352][105586] KL-divergence is very high: 102.7815 [2023-12-26 22:26:51,385][105620] Updated weights for policy 1, policy_version 971847 (0.0009) [2023-12-26 22:26:51,923][105692] Updated weights for policy 0, policy_version 971853 (0.0009) [2023-12-26 22:26:51,971][105692] Updated weights for policy 0, policy_version 971863 (0.0010) [2023-12-26 22:26:52,020][105692] Updated weights for policy 0, policy_version 971873 (0.0010) [2023-12-26 22:26:52,212][105620] Updated weights for policy 1, policy_version 971857 (0.0010) [2023-12-26 22:26:52,282][105620] Updated weights for policy 1, policy_version 971867 (0.0011) [2023-12-26 22:26:52,351][105620] Updated weights for policy 1, policy_version 971877 (0.0007) [2023-12-26 22:26:52,706][105692] Updated weights for policy 0, policy_version 971883 (0.0008) [2023-12-26 22:26:52,753][105692] Updated weights for policy 0, policy_version 971893 (0.0005) [2023-12-26 22:26:52,800][105692] Updated weights for policy 0, policy_version 971903 (0.0005) [2023-12-26 22:26:52,980][105620] Updated weights for policy 1, policy_version 971887 (0.0007) [2023-12-26 22:26:53,050][105620] Updated weights for policy 1, policy_version 971897 (0.0011) [2023-12-26 22:26:53,116][105620] Updated weights for policy 1, policy_version 971907 (0.0011) [2023-12-26 22:26:53,401][105692] Updated weights for policy 0, policy_version 971913 (0.0006) [2023-12-26 22:26:53,453][105692] Updated weights for policy 0, policy_version 971923 (0.0010) [2023-12-26 22:26:53,505][105692] Updated weights for policy 0, policy_version 971933 (0.0011) [2023-12-26 22:26:53,557][105692] Updated weights for policy 0, policy_version 971943 (0.0011) [2023-12-26 22:26:53,831][105620] Updated weights for policy 1, policy_version 971917 (0.0011) [2023-12-26 22:26:53,890][105620] Updated weights for policy 1, policy_version 971927 (0.0011) [2023-12-26 22:26:53,949][105620] Updated weights for policy 1, policy_version 971937 (0.0011) [2023-12-26 22:26:54,223][105692] Updated weights for policy 0, policy_version 971953 (0.0011) [2023-12-26 22:26:54,278][105692] Updated weights for policy 0, policy_version 971963 (0.0010) [2023-12-26 22:26:54,332][105692] Updated weights for policy 0, policy_version 971973 (0.0008) [2023-12-26 22:26:54,697][105620] Updated weights for policy 1, policy_version 971947 (0.0011) [2023-12-26 22:26:54,746][105620] Updated weights for policy 1, policy_version 971957 (0.0011) [2023-12-26 22:26:54,801][105620] Updated weights for policy 1, policy_version 971967 (0.0005) [2023-12-26 22:26:54,975][105692] Updated weights for policy 0, policy_version 971983 (0.0006) [2023-12-26 22:26:55,041][105692] Updated weights for policy 0, policy_version 971993 (0.0006) [2023-12-26 22:26:55,114][105692] Updated weights for policy 0, policy_version 972003 (0.0006) [2023-12-26 22:26:55,450][105620] Updated weights for policy 1, policy_version 971977 (0.0006) [2023-12-26 22:26:55,508][105620] Updated weights for policy 1, policy_version 971987 (0.0009) [2023-12-26 22:26:55,563][105620] Updated weights for policy 1, policy_version 971997 (0.0009) [2023-12-26 22:26:55,621][105620] Updated weights for policy 1, policy_version 972008 (0.0010) [2023-12-26 22:26:55,731][105692] Updated weights for policy 0, policy_version 972013 (0.0009) [2023-12-26 22:26:55,778][105692] Updated weights for policy 0, policy_version 972023 (0.0009) [2023-12-26 22:26:55,829][105692] Updated weights for policy 0, policy_version 972033 (0.0009) [2023-12-26 22:26:56,062][104569] Fps is (10 sec: 20480.7, 60 sec: 19387.8, 300 sec: 19327.6). Total num frames: 497745920. Throughput: 0: 9920.2, 1: 9575.0. Samples: 497753248. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:26:56,062][104569] Avg episode reward: [(0, '8640.206'), (1, '9172.487')] [2023-12-26 22:26:56,323][105620] Updated weights for policy 1, policy_version 972018 (0.0009) [2023-12-26 22:26:56,388][105620] Updated weights for policy 1, policy_version 972028 (0.0006) [2023-12-26 22:26:56,457][105620] Updated weights for policy 1, policy_version 972038 (0.0005) [2023-12-26 22:26:56,673][105692] Updated weights for policy 0, policy_version 972043 (0.0010) [2023-12-26 22:26:56,720][105692] Updated weights for policy 0, policy_version 972053 (0.0006) [2023-12-26 22:26:56,774][105692] Updated weights for policy 0, policy_version 972063 (0.0008) [2023-12-26 22:26:57,118][105620] Updated weights for policy 1, policy_version 972048 (0.0007) [2023-12-26 22:26:57,174][105620] Updated weights for policy 1, policy_version 972058 (0.0005) [2023-12-26 22:26:57,235][105620] Updated weights for policy 1, policy_version 972068 (0.0009) [2023-12-26 22:26:57,537][105692] Updated weights for policy 0, policy_version 972073 (0.0009) [2023-12-26 22:26:57,597][105692] Updated weights for policy 0, policy_version 972083 (0.0009) [2023-12-26 22:26:57,658][105692] Updated weights for policy 0, policy_version 972093 (0.0009) [2023-12-26 22:26:57,719][105692] Updated weights for policy 0, policy_version 972103 (0.0009) [2023-12-26 22:26:57,931][105620] Updated weights for policy 1, policy_version 972078 (0.0007) [2023-12-26 22:26:57,991][105620] Updated weights for policy 1, policy_version 972088 (0.0007) [2023-12-26 22:26:58,047][105620] Updated weights for policy 1, policy_version 972098 (0.0008) [2023-12-26 22:26:58,393][105692] Updated weights for policy 0, policy_version 972113 (0.0009) [2023-12-26 22:26:58,459][105692] Updated weights for policy 0, policy_version 972123 (0.0008) [2023-12-26 22:26:58,525][105692] Updated weights for policy 0, policy_version 972133 (0.0008) [2023-12-26 22:26:58,825][105620] Updated weights for policy 1, policy_version 972108 (0.0008) [2023-12-26 22:26:58,900][105620] Updated weights for policy 1, policy_version 972119 (0.0015) [2023-12-26 22:26:58,972][105620] Updated weights for policy 1, policy_version 972129 (0.0008) [2023-12-26 22:26:59,402][105692] Updated weights for policy 0, policy_version 972143 (0.0010) [2023-12-26 22:26:59,474][105692] Updated weights for policy 0, policy_version 972153 (0.0008) [2023-12-26 22:26:59,535][105692] Updated weights for policy 0, policy_version 972163 (0.0008) [2023-12-26 22:26:59,815][105620] Updated weights for policy 1, policy_version 972139 (0.0010) [2023-12-26 22:26:59,887][105620] Updated weights for policy 1, policy_version 972149 (0.0009) [2023-12-26 22:26:59,948][105620] Updated weights for policy 1, policy_version 972159 (0.0011) [2023-12-26 22:27:00,202][105692] Updated weights for policy 0, policy_version 972173 (0.0007) [2023-12-26 22:27:00,257][105692] Updated weights for policy 0, policy_version 972183 (0.0006) [2023-12-26 22:27:00,311][105692] Updated weights for policy 0, policy_version 972193 (0.0005) [2023-12-26 22:27:00,660][105620] Updated weights for policy 1, policy_version 972169 (0.0010) [2023-12-26 22:27:00,721][105620] Updated weights for policy 1, policy_version 972179 (0.0010) [2023-12-26 22:27:00,772][105620] Updated weights for policy 1, policy_version 972189 (0.0010) [2023-12-26 22:27:00,820][105620] Updated weights for policy 1, policy_version 972199 (0.0010) [2023-12-26 22:27:00,997][105692] Updated weights for policy 0, policy_version 972203 (0.0007) [2023-12-26 22:27:01,061][105692] Updated weights for policy 0, policy_version 972213 (0.0009) [2023-12-26 22:27:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19327.6). Total num frames: 497836032. Throughput: 0: 9937.3, 1: 9583.0. Samples: 497810072. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:27:01,062][104569] Avg episode reward: [(0, '8372.387'), (1, '9263.402')] [2023-12-26 22:27:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000972200_248913920.pth... [2023-12-26 22:27:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000971080_248627200.pth [2023-12-26 22:27:01,114][105692] Updated weights for policy 0, policy_version 972223 (0.0008) [2023-12-26 22:27:01,167][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000972232_248930304.pth... [2023-12-26 22:27:01,170][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000971048_248627200.pth [2023-12-26 22:27:01,594][105620] Updated weights for policy 1, policy_version 972209 (0.0010) [2023-12-26 22:27:01,661][105620] Updated weights for policy 1, policy_version 972219 (0.0008) [2023-12-26 22:27:01,723][105620] Updated weights for policy 1, policy_version 972229 (0.0008) [2023-12-26 22:27:01,837][105692] Updated weights for policy 0, policy_version 972233 (0.0009) [2023-12-26 22:27:01,890][105692] Updated weights for policy 0, policy_version 972243 (0.0009) [2023-12-26 22:27:01,943][105692] Updated weights for policy 0, policy_version 972253 (0.0008) [2023-12-26 22:27:02,001][105692] Updated weights for policy 0, policy_version 972263 (0.0009) [2023-12-26 22:27:02,357][105620] Updated weights for policy 1, policy_version 972239 (0.0007) [2023-12-26 22:27:02,416][105620] Updated weights for policy 1, policy_version 972249 (0.0009) [2023-12-26 22:27:02,477][105620] Updated weights for policy 1, policy_version 972259 (0.0010) [2023-12-26 22:27:02,792][105692] Updated weights for policy 0, policy_version 972273 (0.0008) [2023-12-26 22:27:02,840][105692] Updated weights for policy 0, policy_version 972283 (0.0009) [2023-12-26 22:27:02,888][105692] Updated weights for policy 0, policy_version 972293 (0.0009) [2023-12-26 22:27:03,205][105620] Updated weights for policy 1, policy_version 972269 (0.0010) [2023-12-26 22:27:03,253][105620] Updated weights for policy 1, policy_version 972279 (0.0010) [2023-12-26 22:27:03,312][105620] Updated weights for policy 1, policy_version 972289 (0.0010) [2023-12-26 22:27:03,723][105692] Updated weights for policy 0, policy_version 972303 (0.0009) [2023-12-26 22:27:03,785][105692] Updated weights for policy 0, policy_version 972314 (0.0010) [2023-12-26 22:27:03,845][105692] Updated weights for policy 0, policy_version 972324 (0.0009) [2023-12-26 22:27:03,947][105620] Updated weights for policy 1, policy_version 972299 (0.0010) [2023-12-26 22:27:04,000][105620] Updated weights for policy 1, policy_version 972309 (0.0010) [2023-12-26 22:27:04,059][105620] Updated weights for policy 1, policy_version 972319 (0.0011) [2023-12-26 22:27:04,640][105692] Updated weights for policy 0, policy_version 972334 (0.0010) [2023-12-26 22:27:04,699][105692] Updated weights for policy 0, policy_version 972344 (0.0010) [2023-12-26 22:27:04,754][105692] Updated weights for policy 0, policy_version 972354 (0.0011) [2023-12-26 22:27:04,841][105620] Updated weights for policy 1, policy_version 972329 (0.0011) [2023-12-26 22:27:04,896][105620] Updated weights for policy 1, policy_version 972339 (0.0010) [2023-12-26 22:27:04,946][105620] Updated weights for policy 1, policy_version 972349 (0.0010) [2023-12-26 22:27:04,995][105620] Updated weights for policy 1, policy_version 972359 (0.0010) [2023-12-26 22:27:05,518][105692] Updated weights for policy 0, policy_version 972364 (0.0009) [2023-12-26 22:27:05,580][105692] Updated weights for policy 0, policy_version 972374 (0.0008) [2023-12-26 22:27:05,634][105692] Updated weights for policy 0, policy_version 972384 (0.0008) [2023-12-26 22:27:05,734][105620] Updated weights for policy 1, policy_version 972369 (0.0010) [2023-12-26 22:27:05,782][105620] Updated weights for policy 1, policy_version 972379 (0.0010) [2023-12-26 22:27:05,835][105620] Updated weights for policy 1, policy_version 972389 (0.0006) [2023-12-26 22:27:06,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19327.6). Total num frames: 497934336. Throughput: 0: 9845.5, 1: 9575.3. Samples: 497922580. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:27:06,063][104569] Avg episode reward: [(0, '7768.938'), (1, '9355.638')] [2023-12-26 22:27:06,442][105692] Updated weights for policy 0, policy_version 972394 (0.0007) [2023-12-26 22:27:06,459][105620] Updated weights for policy 1, policy_version 972399 (0.0009) [2023-12-26 22:27:06,506][105692] Updated weights for policy 0, policy_version 972404 (0.0006) [2023-12-26 22:27:06,530][105620] Updated weights for policy 1, policy_version 972409 (0.0009) [2023-12-26 22:27:06,572][105692] Updated weights for policy 0, policy_version 972414 (0.0006) [2023-12-26 22:27:06,599][105620] Updated weights for policy 1, policy_version 972419 (0.0008) [2023-12-26 22:27:06,634][105692] Updated weights for policy 0, policy_version 972424 (0.0006) [2023-12-26 22:27:07,236][105692] Updated weights for policy 0, policy_version 972434 (0.0010) [2023-12-26 22:27:07,290][105620] Updated weights for policy 1, policy_version 972429 (0.0007) [2023-12-26 22:27:07,292][105692] Updated weights for policy 0, policy_version 972444 (0.0009) [2023-12-26 22:27:07,345][105692] Updated weights for policy 0, policy_version 972454 (0.0008) [2023-12-26 22:27:07,351][105620] Updated weights for policy 1, policy_version 972439 (0.0010) [2023-12-26 22:27:07,403][105620] Updated weights for policy 1, policy_version 972449 (0.0010) [2023-12-26 22:27:07,985][105692] Updated weights for policy 0, policy_version 972464 (0.0008) [2023-12-26 22:27:08,044][105692] Updated weights for policy 0, policy_version 972474 (0.0006) [2023-12-26 22:27:08,107][105692] Updated weights for policy 0, policy_version 972484 (0.0005) [2023-12-26 22:27:08,116][105620] Updated weights for policy 1, policy_version 972459 (0.0011) [2023-12-26 22:27:08,165][105620] Updated weights for policy 1, policy_version 972469 (0.0010) [2023-12-26 22:27:08,233][105620] Updated weights for policy 1, policy_version 972479 (0.0011) [2023-12-26 22:27:08,786][105692] Updated weights for policy 0, policy_version 972494 (0.0007) [2023-12-26 22:27:08,849][105692] Updated weights for policy 0, policy_version 972504 (0.0008) [2023-12-26 22:27:08,913][105692] Updated weights for policy 0, policy_version 972514 (0.0008) [2023-12-26 22:27:08,960][105620] Updated weights for policy 1, policy_version 972489 (0.0010) [2023-12-26 22:27:09,012][105620] Updated weights for policy 1, policy_version 972499 (0.0010) [2023-12-26 22:27:09,073][105620] Updated weights for policy 1, policy_version 972509 (0.0010) [2023-12-26 22:27:09,131][105620] Updated weights for policy 1, policy_version 972519 (0.0010) [2023-12-26 22:27:09,686][105692] Updated weights for policy 0, policy_version 972524 (0.0008) [2023-12-26 22:27:09,736][105692] Updated weights for policy 0, policy_version 972534 (0.0008) [2023-12-26 22:27:09,784][105692] Updated weights for policy 0, policy_version 972544 (0.0008) [2023-12-26 22:27:09,954][105620] Updated weights for policy 1, policy_version 972529 (0.0009) [2023-12-26 22:27:10,024][105620] Updated weights for policy 1, policy_version 972539 (0.0009) [2023-12-26 22:27:10,082][105620] Updated weights for policy 1, policy_version 972549 (0.0008) [2023-12-26 22:27:10,497][105692] Updated weights for policy 0, policy_version 972554 (0.0008) [2023-12-26 22:27:10,553][105692] Updated weights for policy 0, policy_version 972564 (0.0008) [2023-12-26 22:27:10,617][105692] Updated weights for policy 0, policy_version 972574 (0.0008) [2023-12-26 22:27:10,677][105692] Updated weights for policy 0, policy_version 972584 (0.0008) [2023-12-26 22:27:10,807][105620] Updated weights for policy 1, policy_version 972559 (0.0006) [2023-12-26 22:27:10,875][105620] Updated weights for policy 1, policy_version 972569 (0.0005) [2023-12-26 22:27:10,947][105620] Updated weights for policy 1, policy_version 972579 (0.0008) [2023-12-26 22:27:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 498032640. Throughput: 0: 9870.9, 1: 9589.2. Samples: 498039528. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:27:11,062][104569] Avg episode reward: [(0, '7784.612'), (1, '9355.513')] [2023-12-26 22:27:11,498][105692] Updated weights for policy 0, policy_version 972594 (0.0011) [2023-12-26 22:27:11,560][105692] Updated weights for policy 0, policy_version 972605 (0.0011) [2023-12-26 22:27:11,622][105692] Updated weights for policy 0, policy_version 972615 (0.0011) [2023-12-26 22:27:11,655][105620] Updated weights for policy 1, policy_version 972589 (0.0009) [2023-12-26 22:27:11,713][105620] Updated weights for policy 1, policy_version 972599 (0.0006) [2023-12-26 22:27:11,782][105620] Updated weights for policy 1, policy_version 972609 (0.0012) [2023-12-26 22:27:12,481][105692] Updated weights for policy 0, policy_version 972625 (0.0010) [2023-12-26 22:27:12,494][105620] Updated weights for policy 1, policy_version 972619 (0.0009) [2023-12-26 22:27:12,543][105692] Updated weights for policy 0, policy_version 972635 (0.0008) [2023-12-26 22:27:12,558][105620] Updated weights for policy 1, policy_version 972629 (0.0006) [2023-12-26 22:27:12,607][105692] Updated weights for policy 0, policy_version 972645 (0.0008) [2023-12-26 22:27:12,619][105620] Updated weights for policy 1, policy_version 972639 (0.0007) [2023-12-26 22:27:13,244][105620] Updated weights for policy 1, policy_version 972649 (0.0009) [2023-12-26 22:27:13,299][105620] Updated weights for policy 1, policy_version 972659 (0.0005) [2023-12-26 22:27:13,355][105620] Updated weights for policy 1, policy_version 972669 (0.0005) [2023-12-26 22:27:13,409][105620] Updated weights for policy 1, policy_version 972679 (0.0008) [2023-12-26 22:27:13,442][105692] Updated weights for policy 0, policy_version 972655 (0.0009) [2023-12-26 22:27:13,500][105692] Updated weights for policy 0, policy_version 972665 (0.0010) [2023-12-26 22:27:13,556][105692] Updated weights for policy 0, policy_version 972675 (0.0010) [2023-12-26 22:27:14,007][105620] Updated weights for policy 1, policy_version 972689 (0.0008) [2023-12-26 22:27:14,071][105620] Updated weights for policy 1, policy_version 972699 (0.0007) [2023-12-26 22:27:14,139][105620] Updated weights for policy 1, policy_version 972709 (0.0008) [2023-12-26 22:27:14,431][105692] Updated weights for policy 0, policy_version 972686 (0.0009) [2023-12-26 22:27:14,495][105692] Updated weights for policy 0, policy_version 972696 (0.0010) [2023-12-26 22:27:14,549][105692] Updated weights for policy 0, policy_version 972706 (0.0010) [2023-12-26 22:27:14,733][105620] Updated weights for policy 1, policy_version 972719 (0.0009) [2023-12-26 22:27:14,798][105620] Updated weights for policy 1, policy_version 972729 (0.0008) [2023-12-26 22:27:14,867][105620] Updated weights for policy 1, policy_version 972739 (0.0008) [2023-12-26 22:27:15,276][105692] Updated weights for policy 0, policy_version 972716 (0.0009) [2023-12-26 22:27:15,336][105692] Updated weights for policy 0, policy_version 972726 (0.0009) [2023-12-26 22:27:15,403][105692] Updated weights for policy 0, policy_version 972736 (0.0009) [2023-12-26 22:27:15,667][105620] Updated weights for policy 1, policy_version 972749 (0.0009) [2023-12-26 22:27:15,728][105620] Updated weights for policy 1, policy_version 972759 (0.0009) [2023-12-26 22:27:15,786][105620] Updated weights for policy 1, policy_version 972769 (0.0005) [2023-12-26 22:27:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19355.3). Total num frames: 498122752. Throughput: 0: 9822.2, 1: 9633.1. Samples: 498095464. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:27:16,062][104569] Avg episode reward: [(0, '8407.667'), (1, '9089.791')] [2023-12-26 22:27:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000972744_249061376.pth... [2023-12-26 22:27:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000972776_249061376.pth... [2023-12-26 22:27:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000971656_248782848.pth [2023-12-26 22:27:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000971624_248766464.pth [2023-12-26 22:27:16,212][105692] Updated weights for policy 0, policy_version 972746 (0.0009) [2023-12-26 22:27:16,273][105692] Updated weights for policy 0, policy_version 972756 (0.0008) [2023-12-26 22:27:16,326][105585] KL-divergence is very high: 137.4857 [2023-12-26 22:27:16,328][105692] Updated weights for policy 0, policy_version 972766 (0.0009) [2023-12-26 22:27:16,372][105585] KL-divergence is very high: 141.4641 [2023-12-26 22:27:16,386][105692] Updated weights for policy 0, policy_version 972776 (0.0009) [2023-12-26 22:27:16,424][105620] Updated weights for policy 1, policy_version 972779 (0.0006) [2023-12-26 22:27:16,482][105620] Updated weights for policy 1, policy_version 972789 (0.0008) [2023-12-26 22:27:16,538][105620] Updated weights for policy 1, policy_version 972799 (0.0009) [2023-12-26 22:27:17,053][105692] Updated weights for policy 0, policy_version 972786 (0.0005) [2023-12-26 22:27:17,120][105692] Updated weights for policy 0, policy_version 972796 (0.0007) [2023-12-26 22:27:17,182][105692] Updated weights for policy 0, policy_version 972806 (0.0008) [2023-12-26 22:27:17,295][105620] Updated weights for policy 1, policy_version 972809 (0.0010) [2023-12-26 22:27:17,367][105620] Updated weights for policy 1, policy_version 972819 (0.0010) [2023-12-26 22:27:17,438][105620] Updated weights for policy 1, policy_version 972829 (0.0010) [2023-12-26 22:27:17,516][105620] Updated weights for policy 1, policy_version 972839 (0.0011) [2023-12-26 22:27:17,703][105692] Updated weights for policy 0, policy_version 972816 (0.0006) [2023-12-26 22:27:17,772][105692] Updated weights for policy 0, policy_version 972826 (0.0006) [2023-12-26 22:27:17,826][105692] Updated weights for policy 0, policy_version 972836 (0.0005) [2023-12-26 22:27:18,076][105620] Updated weights for policy 1, policy_version 972849 (0.0006) [2023-12-26 22:27:18,126][105620] Updated weights for policy 1, policy_version 972859 (0.0005) [2023-12-26 22:27:18,185][105620] Updated weights for policy 1, policy_version 972869 (0.0005) [2023-12-26 22:27:18,377][105692] Updated weights for policy 0, policy_version 972846 (0.0006) [2023-12-26 22:27:18,439][105692] Updated weights for policy 0, policy_version 972856 (0.0008) [2023-12-26 22:27:18,502][105692] Updated weights for policy 0, policy_version 972866 (0.0009) [2023-12-26 22:27:18,794][105620] Updated weights for policy 1, policy_version 972879 (0.0006) [2023-12-26 22:27:18,854][105620] Updated weights for policy 1, policy_version 972889 (0.0005) [2023-12-26 22:27:18,908][105620] Updated weights for policy 1, policy_version 972899 (0.0006) [2023-12-26 22:27:19,242][105692] Updated weights for policy 0, policy_version 972876 (0.0009) [2023-12-26 22:27:19,308][105692] Updated weights for policy 0, policy_version 972886 (0.0009) [2023-12-26 22:27:19,375][105692] Updated weights for policy 0, policy_version 972896 (0.0008) [2023-12-26 22:27:19,636][105620] Updated weights for policy 1, policy_version 972909 (0.0009) [2023-12-26 22:27:19,696][105620] Updated weights for policy 1, policy_version 972919 (0.0010) [2023-12-26 22:27:19,759][105620] Updated weights for policy 1, policy_version 972929 (0.0010) [2023-12-26 22:27:20,127][105692] Updated weights for policy 0, policy_version 972906 (0.0007) [2023-12-26 22:27:20,181][105692] Updated weights for policy 0, policy_version 972916 (0.0009) [2023-12-26 22:27:20,208][105585] KL-divergence is very high: 157.2781 [2023-12-26 22:27:20,239][105692] Updated weights for policy 0, policy_version 972926 (0.0009) [2023-12-26 22:27:20,253][105585] KL-divergence is very high: 192.2154 [2023-12-26 22:27:20,266][105585] KL-divergence is very high: 100.7618 [2023-12-26 22:27:20,296][105692] Updated weights for policy 0, policy_version 972936 (0.0008) [2023-12-26 22:27:20,515][105620] Updated weights for policy 1, policy_version 972939 (0.0009) [2023-12-26 22:27:20,576][105620] Updated weights for policy 1, policy_version 972949 (0.0007) [2023-12-26 22:27:20,644][105620] Updated weights for policy 1, policy_version 972959 (0.0010) [2023-12-26 22:27:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 498221056. Throughput: 0: 9735.5, 1: 9744.5. Samples: 498215212. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:27:21,063][104569] Avg episode reward: [(0, '7987.819'), (1, '9003.253')] [2023-12-26 22:27:21,119][105692] Updated weights for policy 0, policy_version 972946 (0.0008) [2023-12-26 22:27:21,188][105692] Updated weights for policy 0, policy_version 972956 (0.0006) [2023-12-26 22:27:21,258][105692] Updated weights for policy 0, policy_version 972966 (0.0007) [2023-12-26 22:27:21,410][105620] Updated weights for policy 1, policy_version 972969 (0.0011) [2023-12-26 22:27:21,478][105620] Updated weights for policy 1, policy_version 972979 (0.0011) [2023-12-26 22:27:21,542][105620] Updated weights for policy 1, policy_version 972989 (0.0011) [2023-12-26 22:27:21,611][105620] Updated weights for policy 1, policy_version 972999 (0.0009) [2023-12-26 22:27:22,040][105692] Updated weights for policy 0, policy_version 972976 (0.0010) [2023-12-26 22:27:22,100][105692] Updated weights for policy 0, policy_version 972986 (0.0011) [2023-12-26 22:27:22,159][105692] Updated weights for policy 0, policy_version 972996 (0.0011) [2023-12-26 22:27:22,321][105620] Updated weights for policy 1, policy_version 973009 (0.0010) [2023-12-26 22:27:22,390][105620] Updated weights for policy 1, policy_version 973019 (0.0011) [2023-12-26 22:27:22,457][105620] Updated weights for policy 1, policy_version 973029 (0.0011) [2023-12-26 22:27:22,892][105692] Updated weights for policy 0, policy_version 973006 (0.0010) [2023-12-26 22:27:22,953][105585] KL-divergence is very high: 103.5383 [2023-12-26 22:27:22,969][105692] Updated weights for policy 0, policy_version 973016 (0.0010) [2023-12-26 22:27:22,983][105585] KL-divergence is very high: 107.5634 [2023-12-26 22:27:23,014][105585] KL-divergence is very high: 115.7256 [2023-12-26 22:27:23,042][105692] Updated weights for policy 0, policy_version 973026 (0.0010) [2023-12-26 22:27:23,044][105585] KL-divergence is very high: 102.5157 [2023-12-26 22:27:23,074][105585] KL-divergence is very high: 100.0133 [2023-12-26 22:27:23,156][105620] Updated weights for policy 1, policy_version 973039 (0.0011) [2023-12-26 22:27:23,219][105620] Updated weights for policy 1, policy_version 973049 (0.0011) [2023-12-26 22:27:23,278][105620] Updated weights for policy 1, policy_version 973059 (0.0011) [2023-12-26 22:27:23,749][105692] Updated weights for policy 0, policy_version 973036 (0.0009) [2023-12-26 22:27:23,797][105692] Updated weights for policy 0, policy_version 973046 (0.0010) [2023-12-26 22:27:23,853][105692] Updated weights for policy 0, policy_version 973056 (0.0010) [2023-12-26 22:27:23,992][105620] Updated weights for policy 1, policy_version 973069 (0.0008) [2023-12-26 22:27:24,043][105620] Updated weights for policy 1, policy_version 973079 (0.0005) [2023-12-26 22:27:24,095][105620] Updated weights for policy 1, policy_version 973089 (0.0009) [2023-12-26 22:27:24,631][105692] Updated weights for policy 0, policy_version 973066 (0.0008) [2023-12-26 22:27:24,693][105692] Updated weights for policy 0, policy_version 973076 (0.0006) [2023-12-26 22:27:24,751][105692] Updated weights for policy 0, policy_version 973086 (0.0006) [2023-12-26 22:27:24,812][105692] Updated weights for policy 0, policy_version 973096 (0.0005) [2023-12-26 22:27:24,820][105620] Updated weights for policy 1, policy_version 973099 (0.0009) [2023-12-26 22:27:24,896][105620] Updated weights for policy 1, policy_version 973109 (0.0011) [2023-12-26 22:27:24,973][105620] Updated weights for policy 1, policy_version 973119 (0.0010) [2023-12-26 22:27:25,546][105620] Updated weights for policy 1, policy_version 973129 (0.0011) [2023-12-26 22:27:25,557][105692] Updated weights for policy 0, policy_version 973106 (0.0006) [2023-12-26 22:27:25,600][105620] Updated weights for policy 1, policy_version 973139 (0.0009) [2023-12-26 22:27:25,606][105692] Updated weights for policy 0, policy_version 973116 (0.0006) [2023-12-26 22:27:25,657][105692] Updated weights for policy 0, policy_version 973126 (0.0007) [2023-12-26 22:27:25,659][105620] Updated weights for policy 1, policy_version 973149 (0.0007) [2023-12-26 22:27:25,728][105620] Updated weights for policy 1, policy_version 973159 (0.0006) [2023-12-26 22:27:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19355.3). Total num frames: 498319360. Throughput: 0: 9569.4, 1: 9779.9. Samples: 498328436. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:27:26,062][104569] Avg episode reward: [(0, '7001.534'), (1, '8994.355')] [2023-12-26 22:27:26,303][105620] Updated weights for policy 1, policy_version 973169 (0.0006) [2023-12-26 22:27:26,325][105692] Updated weights for policy 0, policy_version 973136 (0.0005) [2023-12-26 22:27:26,360][105620] Updated weights for policy 1, policy_version 973179 (0.0005) [2023-12-26 22:27:26,375][105692] Updated weights for policy 0, policy_version 973146 (0.0005) [2023-12-26 22:27:26,418][105620] Updated weights for policy 1, policy_version 973189 (0.0006) [2023-12-26 22:27:26,439][105692] Updated weights for policy 0, policy_version 973156 (0.0005) [2023-12-26 22:27:26,964][105620] Updated weights for policy 1, policy_version 973199 (0.0008) [2023-12-26 22:27:27,023][105692] Updated weights for policy 0, policy_version 973166 (0.0005) [2023-12-26 22:27:27,028][105620] Updated weights for policy 1, policy_version 973209 (0.0008) [2023-12-26 22:27:27,084][105692] Updated weights for policy 0, policy_version 973176 (0.0007) [2023-12-26 22:27:27,088][105620] Updated weights for policy 1, policy_version 973219 (0.0009) [2023-12-26 22:27:27,150][105692] Updated weights for policy 0, policy_version 973186 (0.0011) [2023-12-26 22:27:27,729][105692] Updated weights for policy 0, policy_version 973196 (0.0009) [2023-12-26 22:27:27,783][105692] Updated weights for policy 0, policy_version 973206 (0.0007) [2023-12-26 22:27:27,832][105692] Updated weights for policy 0, policy_version 973216 (0.0005) [2023-12-26 22:27:27,897][105620] Updated weights for policy 1, policy_version 973229 (0.0010) [2023-12-26 22:27:27,950][105620] Updated weights for policy 1, policy_version 973239 (0.0010) [2023-12-26 22:27:28,003][105620] Updated weights for policy 1, policy_version 973250 (0.0009) [2023-12-26 22:27:28,463][105692] Updated weights for policy 0, policy_version 973226 (0.0006) [2023-12-26 22:27:28,514][105692] Updated weights for policy 0, policy_version 973236 (0.0010) [2023-12-26 22:27:28,567][105692] Updated weights for policy 0, policy_version 973246 (0.0010) [2023-12-26 22:27:28,627][105692] Updated weights for policy 0, policy_version 973256 (0.0010) [2023-12-26 22:27:28,786][105620] Updated weights for policy 1, policy_version 973260 (0.0007) [2023-12-26 22:27:28,842][105620] Updated weights for policy 1, policy_version 973270 (0.0010) [2023-12-26 22:27:28,898][105620] Updated weights for policy 1, policy_version 973280 (0.0009) [2023-12-26 22:27:29,322][105692] Updated weights for policy 0, policy_version 973266 (0.0010) [2023-12-26 22:27:29,395][105692] Updated weights for policy 0, policy_version 973276 (0.0009) [2023-12-26 22:27:29,458][105692] Updated weights for policy 0, policy_version 973286 (0.0009) [2023-12-26 22:27:29,691][105620] Updated weights for policy 1, policy_version 973290 (0.0009) [2023-12-26 22:27:29,753][105620] Updated weights for policy 1, policy_version 973300 (0.0009) [2023-12-26 22:27:29,818][105620] Updated weights for policy 1, policy_version 973310 (0.0009) [2023-12-26 22:27:29,887][105620] Updated weights for policy 1, policy_version 973320 (0.0009) [2023-12-26 22:27:30,260][105692] Updated weights for policy 0, policy_version 973296 (0.0010) [2023-12-26 22:27:30,319][105692] Updated weights for policy 0, policy_version 973306 (0.0010) [2023-12-26 22:27:30,380][105692] Updated weights for policy 0, policy_version 973316 (0.0007) [2023-12-26 22:27:30,590][105620] Updated weights for policy 1, policy_version 973330 (0.0009) [2023-12-26 22:27:30,644][105620] Updated weights for policy 1, policy_version 973340 (0.0009) [2023-12-26 22:27:30,694][105620] Updated weights for policy 1, policy_version 973350 (0.0009) [2023-12-26 22:27:31,027][105692] Updated weights for policy 0, policy_version 973326 (0.0010) [2023-12-26 22:27:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19355.3). Total num frames: 498417664. Throughput: 0: 9661.2, 1: 9779.4. Samples: 498391164. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:27:31,062][104569] Avg episode reward: [(0, '7032.431'), (1, '8997.561')] [2023-12-26 22:27:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000973352_249208832.pth... [2023-12-26 22:27:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000972200_248913920.pth [2023-12-26 22:27:31,089][105692] Updated weights for policy 0, policy_version 973336 (0.0009) [2023-12-26 22:27:31,160][105692] Updated weights for policy 0, policy_version 973346 (0.0010) [2023-12-26 22:27:31,197][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000973352_249217024.pth... [2023-12-26 22:27:31,202][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000972232_248930304.pth [2023-12-26 22:27:31,465][105620] Updated weights for policy 1, policy_version 973360 (0.0009) [2023-12-26 22:27:31,516][105620] Updated weights for policy 1, policy_version 973370 (0.0009) [2023-12-26 22:27:31,569][105620] Updated weights for policy 1, policy_version 973380 (0.0010) [2023-12-26 22:27:31,855][105692] Updated weights for policy 0, policy_version 973356 (0.0009) [2023-12-26 22:27:31,906][105692] Updated weights for policy 0, policy_version 973366 (0.0009) [2023-12-26 22:27:31,911][105585] KL-divergence is very high: 129.4545 [2023-12-26 22:27:31,950][105585] KL-divergence is very high: 148.6279 [2023-12-26 22:27:31,957][105692] Updated weights for policy 0, policy_version 973376 (0.0009) [2023-12-26 22:27:32,304][105620] Updated weights for policy 1, policy_version 973390 (0.0009) [2023-12-26 22:27:32,356][105620] Updated weights for policy 1, policy_version 973400 (0.0009) [2023-12-26 22:27:32,421][105620] Updated weights for policy 1, policy_version 973410 (0.0008) [2023-12-26 22:27:32,743][105692] Updated weights for policy 0, policy_version 973386 (0.0008) [2023-12-26 22:27:32,798][105692] Updated weights for policy 0, policy_version 973396 (0.0007) [2023-12-26 22:27:32,849][105692] Updated weights for policy 0, policy_version 973406 (0.0009) [2023-12-26 22:27:32,911][105692] Updated weights for policy 0, policy_version 973416 (0.0009) [2023-12-26 22:27:33,119][105620] Updated weights for policy 1, policy_version 973420 (0.0007) [2023-12-26 22:27:33,169][105620] Updated weights for policy 1, policy_version 973430 (0.0005) [2023-12-26 22:27:33,217][105620] Updated weights for policy 1, policy_version 973440 (0.0005) [2023-12-26 22:27:33,683][105692] Updated weights for policy 0, policy_version 973426 (0.0009) [2023-12-26 22:27:33,729][105692] Updated weights for policy 0, policy_version 973436 (0.0008) [2023-12-26 22:27:33,787][105692] Updated weights for policy 0, policy_version 973446 (0.0009) [2023-12-26 22:27:33,892][105620] Updated weights for policy 1, policy_version 973450 (0.0006) [2023-12-26 22:27:33,949][105620] Updated weights for policy 1, policy_version 973460 (0.0009) [2023-12-26 22:27:34,001][105620] Updated weights for policy 1, policy_version 973470 (0.0008) [2023-12-26 22:27:34,059][105620] Updated weights for policy 1, policy_version 973480 (0.0010) [2023-12-26 22:27:34,492][105692] Updated weights for policy 0, policy_version 973456 (0.0009) [2023-12-26 22:27:34,555][105692] Updated weights for policy 0, policy_version 973466 (0.0007) [2023-12-26 22:27:34,619][105692] Updated weights for policy 0, policy_version 973476 (0.0009) [2023-12-26 22:27:34,888][105620] Updated weights for policy 1, policy_version 973490 (0.0010) [2023-12-26 22:27:34,951][105620] Updated weights for policy 1, policy_version 973500 (0.0010) [2023-12-26 22:27:35,002][105620] Updated weights for policy 1, policy_version 973510 (0.0009) [2023-12-26 22:27:35,253][105692] Updated weights for policy 0, policy_version 973486 (0.0007) [2023-12-26 22:27:35,306][105692] Updated weights for policy 0, policy_version 973496 (0.0006) [2023-12-26 22:27:35,354][105692] Updated weights for policy 0, policy_version 973506 (0.0009) [2023-12-26 22:27:35,797][105620] Updated weights for policy 1, policy_version 973520 (0.0009) [2023-12-26 22:27:35,869][105620] Updated weights for policy 1, policy_version 973530 (0.0008) [2023-12-26 22:27:35,922][105620] Updated weights for policy 1, policy_version 973540 (0.0008) [2023-12-26 22:27:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 498515968. Throughput: 0: 9634.0, 1: 9743.0. Samples: 498505392. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:27:36,063][104569] Avg episode reward: [(0, '7518.616'), (1, '8909.294')] [2023-12-26 22:27:36,111][105692] Updated weights for policy 0, policy_version 973516 (0.0009) [2023-12-26 22:27:36,177][105692] Updated weights for policy 0, policy_version 973526 (0.0008) [2023-12-26 22:27:36,235][105692] Updated weights for policy 0, policy_version 973536 (0.0008) [2023-12-26 22:27:36,685][105620] Updated weights for policy 1, policy_version 973550 (0.0009) [2023-12-26 22:27:36,736][105620] Updated weights for policy 1, policy_version 973560 (0.0009) [2023-12-26 22:27:36,789][105620] Updated weights for policy 1, policy_version 973570 (0.0008) [2023-12-26 22:27:36,986][105692] Updated weights for policy 0, policy_version 973546 (0.0009) [2023-12-26 22:27:37,049][105692] Updated weights for policy 0, policy_version 973556 (0.0009) [2023-12-26 22:27:37,100][105692] Updated weights for policy 0, policy_version 973566 (0.0009) [2023-12-26 22:27:37,156][105692] Updated weights for policy 0, policy_version 973576 (0.0010) [2023-12-26 22:27:37,529][105620] Updated weights for policy 1, policy_version 973580 (0.0009) [2023-12-26 22:27:37,579][105620] Updated weights for policy 1, policy_version 973590 (0.0009) [2023-12-26 22:27:37,637][105620] Updated weights for policy 1, policy_version 973600 (0.0009) [2023-12-26 22:27:37,940][105692] Updated weights for policy 0, policy_version 973586 (0.0009) [2023-12-26 22:27:38,000][105692] Updated weights for policy 0, policy_version 973596 (0.0009) [2023-12-26 22:27:38,056][105692] Updated weights for policy 0, policy_version 973606 (0.0009) [2023-12-26 22:27:38,410][105620] Updated weights for policy 1, policy_version 973610 (0.0009) [2023-12-26 22:27:38,475][105620] Updated weights for policy 1, policy_version 973620 (0.0009) [2023-12-26 22:27:38,537][105620] Updated weights for policy 1, policy_version 973630 (0.0009) [2023-12-26 22:27:38,587][105620] Updated weights for policy 1, policy_version 973640 (0.0008) [2023-12-26 22:27:38,821][105692] Updated weights for policy 0, policy_version 973616 (0.0009) [2023-12-26 22:27:38,885][105692] Updated weights for policy 0, policy_version 973626 (0.0009) [2023-12-26 22:27:38,942][105585] KL-divergence is very high: 103.9754 [2023-12-26 22:27:38,948][105692] Updated weights for policy 0, policy_version 973636 (0.0009) [2023-12-26 22:27:38,955][105585] KL-divergence is very high: 101.6864 [2023-12-26 22:27:38,967][105585] KL-divergence is very high: 100.2025 [2023-12-26 22:27:39,379][105620] Updated weights for policy 1, policy_version 973650 (0.0010) [2023-12-26 22:27:39,447][105620] Updated weights for policy 1, policy_version 973660 (0.0008) [2023-12-26 22:27:39,510][105620] Updated weights for policy 1, policy_version 973670 (0.0009) [2023-12-26 22:27:39,701][105692] Updated weights for policy 0, policy_version 973646 (0.0009) [2023-12-26 22:27:39,765][105692] Updated weights for policy 0, policy_version 973656 (0.0008) [2023-12-26 22:27:39,830][105692] Updated weights for policy 0, policy_version 973666 (0.0009) [2023-12-26 22:27:40,281][105620] Updated weights for policy 1, policy_version 973680 (0.0009) [2023-12-26 22:27:40,345][105620] Updated weights for policy 1, policy_version 973690 (0.0009) [2023-12-26 22:27:40,409][105620] Updated weights for policy 1, policy_version 973700 (0.0008) [2023-12-26 22:27:40,604][105692] Updated weights for policy 0, policy_version 973676 (0.0010) [2023-12-26 22:27:40,659][105692] Updated weights for policy 0, policy_version 973686 (0.0009) [2023-12-26 22:27:40,723][105692] Updated weights for policy 0, policy_version 973696 (0.0009) [2023-12-26 22:27:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 498606080. Throughput: 0: 9489.7, 1: 9683.4. Samples: 498616036. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-12-26 22:27:41,062][104569] Avg episode reward: [(0, '6976.277'), (1, '8991.834')] [2023-12-26 22:27:41,153][105620] Updated weights for policy 1, policy_version 973710 (0.0009) [2023-12-26 22:27:41,216][105620] Updated weights for policy 1, policy_version 973720 (0.0009) [2023-12-26 22:27:41,283][105620] Updated weights for policy 1, policy_version 973730 (0.0009) [2023-12-26 22:27:41,607][105692] Updated weights for policy 0, policy_version 973706 (0.0009) [2023-12-26 22:27:41,674][105692] Updated weights for policy 0, policy_version 973716 (0.0010) [2023-12-26 22:27:41,744][105692] Updated weights for policy 0, policy_version 973726 (0.0009) [2023-12-26 22:27:41,808][105692] Updated weights for policy 0, policy_version 973736 (0.0009) [2023-12-26 22:27:42,075][105620] Updated weights for policy 1, policy_version 973740 (0.0009) [2023-12-26 22:27:42,143][105620] Updated weights for policy 1, policy_version 973750 (0.0010) [2023-12-26 22:27:42,201][105620] Updated weights for policy 1, policy_version 973761 (0.0010) [2023-12-26 22:27:42,542][105692] Updated weights for policy 0, policy_version 973746 (0.0009) [2023-12-26 22:27:42,598][105692] Updated weights for policy 0, policy_version 973756 (0.0009) [2023-12-26 22:27:42,654][105692] Updated weights for policy 0, policy_version 973766 (0.0009) [2023-12-26 22:27:43,022][105620] Updated weights for policy 1, policy_version 973771 (0.0009) [2023-12-26 22:27:43,083][105620] Updated weights for policy 1, policy_version 973781 (0.0008) [2023-12-26 22:27:43,144][105620] Updated weights for policy 1, policy_version 973791 (0.0009) [2023-12-26 22:27:43,351][105692] Updated weights for policy 0, policy_version 973776 (0.0008) [2023-12-26 22:27:43,402][105692] Updated weights for policy 0, policy_version 973786 (0.0009) [2023-12-26 22:27:43,453][105692] Updated weights for policy 0, policy_version 973796 (0.0009) [2023-12-26 22:27:43,922][105620] Updated weights for policy 1, policy_version 973801 (0.0009) [2023-12-26 22:27:43,976][105620] Updated weights for policy 1, policy_version 973811 (0.0009) [2023-12-26 22:27:44,027][105620] Updated weights for policy 1, policy_version 973821 (0.0009) [2023-12-26 22:27:44,077][105620] Updated weights for policy 1, policy_version 973831 (0.0009) [2023-12-26 22:27:44,165][105692] Updated weights for policy 0, policy_version 973806 (0.0008) [2023-12-26 22:27:44,224][105692] Updated weights for policy 0, policy_version 973816 (0.0009) [2023-12-26 22:27:44,271][105692] Updated weights for policy 0, policy_version 973826 (0.0008) [2023-12-26 22:27:44,860][105620] Updated weights for policy 1, policy_version 973841 (0.0010) [2023-12-26 22:27:44,914][105620] Updated weights for policy 1, policy_version 973851 (0.0009) [2023-12-26 22:27:44,976][105620] Updated weights for policy 1, policy_version 973861 (0.0008) [2023-12-26 22:27:45,044][105692] Updated weights for policy 0, policy_version 973836 (0.0009) [2023-12-26 22:27:45,107][105692] Updated weights for policy 0, policy_version 973846 (0.0009) [2023-12-26 22:27:45,164][105692] Updated weights for policy 0, policy_version 973856 (0.0009) [2023-12-26 22:27:45,669][105620] Updated weights for policy 1, policy_version 973871 (0.0009) [2023-12-26 22:27:45,730][105620] Updated weights for policy 1, policy_version 973881 (0.0008) [2023-12-26 22:27:45,787][105620] Updated weights for policy 1, policy_version 973891 (0.0006) [2023-12-26 22:27:46,013][105692] Updated weights for policy 0, policy_version 973866 (0.0009) [2023-12-26 22:27:46,059][105692] Updated weights for policy 0, policy_version 973876 (0.0008) [2023-12-26 22:27:46,062][104569] Fps is (10 sec: 18022.0, 60 sec: 19251.2, 300 sec: 19355.3). Total num frames: 498696192. Throughput: 0: 9487.7, 1: 9631.8. Samples: 498670456. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:27:46,063][104569] Avg episode reward: [(0, '6980.038'), (1, '9080.943')] [2023-12-26 22:27:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000973896_249348096.pth... [2023-12-26 22:27:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000972776_249061376.pth [2023-12-26 22:27:46,111][105692] Updated weights for policy 0, policy_version 973886 (0.0009) [2023-12-26 22:27:46,153][105585] KL-divergence is very high: 133.4694 [2023-12-26 22:27:46,168][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000973896_249356288.pth... [2023-12-26 22:27:46,170][105692] Updated weights for policy 0, policy_version 973896 (0.0009) [2023-12-26 22:27:46,172][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000972744_249061376.pth [2023-12-26 22:27:46,431][105620] Updated weights for policy 1, policy_version 973901 (0.0007) [2023-12-26 22:27:46,493][105620] Updated weights for policy 1, policy_version 973911 (0.0009) [2023-12-26 22:27:46,551][105620] Updated weights for policy 1, policy_version 973921 (0.0009) [2023-12-26 22:27:46,964][105585] KL-divergence is very high: 131.4283 [2023-12-26 22:27:46,964][105692] Updated weights for policy 0, policy_version 973906 (0.0009) [2023-12-26 22:27:47,007][105585] KL-divergence is very high: 134.1455 [2023-12-26 22:27:47,018][105692] Updated weights for policy 0, policy_version 973916 (0.0009) [2023-12-26 22:27:47,049][105585] KL-divergence is very high: 130.9417 [2023-12-26 22:27:47,070][105692] Updated weights for policy 0, policy_version 973926 (0.0009) [2023-12-26 22:27:47,259][105620] Updated weights for policy 1, policy_version 973931 (0.0008) [2023-12-26 22:27:47,313][105620] Updated weights for policy 1, policy_version 973941 (0.0005) [2023-12-26 22:27:47,357][105620] Updated weights for policy 1, policy_version 973951 (0.0005) [2023-12-26 22:27:47,878][105692] Updated weights for policy 0, policy_version 973936 (0.0008) [2023-12-26 22:27:47,930][105692] Updated weights for policy 0, policy_version 973946 (0.0008) [2023-12-26 22:27:47,991][105692] Updated weights for policy 0, policy_version 973957 (0.0010) [2023-12-26 22:27:48,031][105620] Updated weights for policy 1, policy_version 973961 (0.0006) [2023-12-26 22:27:48,079][105620] Updated weights for policy 1, policy_version 973971 (0.0010) [2023-12-26 22:27:48,123][105620] Updated weights for policy 1, policy_version 973981 (0.0010) [2023-12-26 22:27:48,178][105620] Updated weights for policy 1, policy_version 973991 (0.0010) [2023-12-26 22:27:48,790][105692] Updated weights for policy 0, policy_version 973967 (0.0010) [2023-12-26 22:27:48,843][105692] Updated weights for policy 0, policy_version 973977 (0.0010) [2023-12-26 22:27:48,893][105692] Updated weights for policy 0, policy_version 973987 (0.0010) [2023-12-26 22:27:48,957][105620] Updated weights for policy 1, policy_version 974001 (0.0010) [2023-12-26 22:27:49,011][105620] Updated weights for policy 1, policy_version 974011 (0.0010) [2023-12-26 22:27:49,059][105620] Updated weights for policy 1, policy_version 974021 (0.0010) [2023-12-26 22:27:49,669][105692] Updated weights for policy 0, policy_version 973997 (0.0011) [2023-12-26 22:27:49,729][105692] Updated weights for policy 0, policy_version 974007 (0.0010) [2023-12-26 22:27:49,788][105692] Updated weights for policy 0, policy_version 974017 (0.0010) [2023-12-26 22:27:49,858][105620] Updated weights for policy 1, policy_version 974031 (0.0011) [2023-12-26 22:27:49,931][105620] Updated weights for policy 1, policy_version 974041 (0.0010) [2023-12-26 22:27:49,993][105620] Updated weights for policy 1, policy_version 974051 (0.0010) [2023-12-26 22:27:50,540][105692] Updated weights for policy 0, policy_version 974027 (0.0009) [2023-12-26 22:27:50,611][105692] Updated weights for policy 0, policy_version 974037 (0.0006) [2023-12-26 22:27:50,655][105620] Updated weights for policy 1, policy_version 974061 (0.0011) [2023-12-26 22:27:50,672][105692] Updated weights for policy 0, policy_version 974047 (0.0008) [2023-12-26 22:27:50,722][105620] Updated weights for policy 1, policy_version 974071 (0.0009) [2023-12-26 22:27:50,774][105620] Updated weights for policy 1, policy_version 974081 (0.0009) [2023-12-26 22:27:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 498794496. Throughput: 0: 9461.5, 1: 9647.2. Samples: 498782472. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:27:51,063][104569] Avg episode reward: [(0, '7684.886'), (1, '8993.829')] [2023-12-26 22:27:51,374][105692] Updated weights for policy 0, policy_version 974057 (0.0008) [2023-12-26 22:27:51,437][105692] Updated weights for policy 0, policy_version 974067 (0.0009) [2023-12-26 22:27:51,499][105692] Updated weights for policy 0, policy_version 974077 (0.0009) [2023-12-26 22:27:51,519][105620] Updated weights for policy 1, policy_version 974091 (0.0007) [2023-12-26 22:27:51,567][105692] Updated weights for policy 0, policy_version 974087 (0.0008) [2023-12-26 22:27:51,591][105620] Updated weights for policy 1, policy_version 974101 (0.0011) [2023-12-26 22:27:51,662][105620] Updated weights for policy 1, policy_version 974111 (0.0010) [2023-12-26 22:27:52,318][105692] Updated weights for policy 0, policy_version 974097 (0.0008) [2023-12-26 22:27:52,360][105620] Updated weights for policy 1, policy_version 974121 (0.0009) [2023-12-26 22:27:52,384][105692] Updated weights for policy 0, policy_version 974107 (0.0009) [2023-12-26 22:27:52,420][105620] Updated weights for policy 1, policy_version 974131 (0.0007) [2023-12-26 22:27:52,447][105692] Updated weights for policy 0, policy_version 974117 (0.0009) [2023-12-26 22:27:52,482][105620] Updated weights for policy 1, policy_version 974141 (0.0008) [2023-12-26 22:27:52,546][105620] Updated weights for policy 1, policy_version 974151 (0.0009) [2023-12-26 22:27:53,223][105692] Updated weights for policy 0, policy_version 974127 (0.0006) [2023-12-26 22:27:53,230][105620] Updated weights for policy 1, policy_version 974161 (0.0007) [2023-12-26 22:27:53,277][105620] Updated weights for policy 1, policy_version 974171 (0.0007) [2023-12-26 22:27:53,282][105692] Updated weights for policy 0, policy_version 974137 (0.0007) [2023-12-26 22:27:53,325][105620] Updated weights for policy 1, policy_version 974181 (0.0006) [2023-12-26 22:27:53,338][105692] Updated weights for policy 0, policy_version 974147 (0.0009) [2023-12-26 22:27:54,034][105692] Updated weights for policy 0, policy_version 974157 (0.0009) [2023-12-26 22:27:54,085][105692] Updated weights for policy 0, policy_version 974167 (0.0009) [2023-12-26 22:27:54,114][105620] Updated weights for policy 1, policy_version 974191 (0.0008) [2023-12-26 22:27:54,155][105692] Updated weights for policy 0, policy_version 974177 (0.0006) [2023-12-26 22:27:54,168][105620] Updated weights for policy 1, policy_version 974201 (0.0009) [2023-12-26 22:27:54,226][105620] Updated weights for policy 1, policy_version 974211 (0.0008) [2023-12-26 22:27:54,869][105692] Updated weights for policy 0, policy_version 974187 (0.0007) [2023-12-26 22:27:54,927][105692] Updated weights for policy 0, policy_version 974197 (0.0009) [2023-12-26 22:27:54,975][105620] Updated weights for policy 1, policy_version 974221 (0.0008) [2023-12-26 22:27:54,984][105692] Updated weights for policy 0, policy_version 974207 (0.0007) [2023-12-26 22:27:55,032][105620] Updated weights for policy 1, policy_version 974231 (0.0009) [2023-12-26 22:27:55,090][105620] Updated weights for policy 1, policy_version 974241 (0.0008) [2023-12-26 22:27:55,660][105692] Updated weights for policy 0, policy_version 974217 (0.0006) [2023-12-26 22:27:55,705][105692] Updated weights for policy 0, policy_version 974227 (0.0008) [2023-12-26 22:27:55,763][105692] Updated weights for policy 0, policy_version 974237 (0.0009) [2023-12-26 22:27:55,828][105692] Updated weights for policy 0, policy_version 974247 (0.0009) [2023-12-26 22:27:55,852][105620] Updated weights for policy 1, policy_version 974251 (0.0009) [2023-12-26 22:27:55,898][105620] Updated weights for policy 1, policy_version 974261 (0.0008) [2023-12-26 22:27:55,949][105620] Updated weights for policy 1, policy_version 974271 (0.0009) [2023-12-26 22:27:56,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19114.6, 300 sec: 19383.1). Total num frames: 498892800. Throughput: 0: 9429.6, 1: 9612.7. Samples: 498896432. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:27:56,063][104569] Avg episode reward: [(0, '7487.786'), (1, '8902.301')] [2023-12-26 22:27:56,590][105692] Updated weights for policy 0, policy_version 974257 (0.0009) [2023-12-26 22:27:56,645][105692] Updated weights for policy 0, policy_version 974267 (0.0009) [2023-12-26 22:27:56,695][105620] Updated weights for policy 1, policy_version 974281 (0.0008) [2023-12-26 22:27:56,699][105692] Updated weights for policy 0, policy_version 974277 (0.0009) [2023-12-26 22:27:56,746][105620] Updated weights for policy 1, policy_version 974291 (0.0005) [2023-12-26 22:27:56,797][105620] Updated weights for policy 1, policy_version 974301 (0.0005) [2023-12-26 22:27:56,850][105620] Updated weights for policy 1, policy_version 974311 (0.0007) [2023-12-26 22:27:57,507][105620] Updated weights for policy 1, policy_version 974321 (0.0008) [2023-12-26 22:27:57,529][105692] Updated weights for policy 0, policy_version 974287 (0.0007) [2023-12-26 22:27:57,568][105620] Updated weights for policy 1, policy_version 974331 (0.0007) [2023-12-26 22:27:57,574][105692] Updated weights for policy 0, policy_version 974297 (0.0007) [2023-12-26 22:27:57,625][105620] Updated weights for policy 1, policy_version 974341 (0.0006) [2023-12-26 22:27:57,627][105692] Updated weights for policy 0, policy_version 974307 (0.0008) [2023-12-26 22:27:58,234][105620] Updated weights for policy 1, policy_version 974351 (0.0009) [2023-12-26 22:27:58,300][105620] Updated weights for policy 1, policy_version 974361 (0.0008) [2023-12-26 22:27:58,374][105620] Updated weights for policy 1, policy_version 974371 (0.0008) [2023-12-26 22:27:58,487][105692] Updated weights for policy 0, policy_version 974317 (0.0008) [2023-12-26 22:27:58,546][105692] Updated weights for policy 0, policy_version 974327 (0.0008) [2023-12-26 22:27:58,616][105692] Updated weights for policy 0, policy_version 974337 (0.0009) [2023-12-26 22:27:59,144][105620] Updated weights for policy 1, policy_version 974381 (0.0009) [2023-12-26 22:27:59,199][105620] Updated weights for policy 1, policy_version 974391 (0.0010) [2023-12-26 22:27:59,271][105620] Updated weights for policy 1, policy_version 974401 (0.0008) [2023-12-26 22:27:59,425][105692] Updated weights for policy 0, policy_version 974347 (0.0009) [2023-12-26 22:27:59,486][105692] Updated weights for policy 0, policy_version 974357 (0.0006) [2023-12-26 22:27:59,546][105692] Updated weights for policy 0, policy_version 974367 (0.0008) [2023-12-26 22:28:00,064][105620] Updated weights for policy 1, policy_version 974411 (0.0010) [2023-12-26 22:28:00,119][105620] Updated weights for policy 1, policy_version 974421 (0.0010) [2023-12-26 22:28:00,177][105620] Updated weights for policy 1, policy_version 974431 (0.0011) [2023-12-26 22:28:00,253][105692] Updated weights for policy 0, policy_version 974377 (0.0009) [2023-12-26 22:28:00,308][105692] Updated weights for policy 0, policy_version 974387 (0.0007) [2023-12-26 22:28:00,370][105692] Updated weights for policy 0, policy_version 974397 (0.0008) [2023-12-26 22:28:00,426][105692] Updated weights for policy 0, policy_version 974407 (0.0009) [2023-12-26 22:28:00,896][105620] Updated weights for policy 1, policy_version 974441 (0.0010) [2023-12-26 22:28:00,951][105620] Updated weights for policy 1, policy_version 974451 (0.0010) [2023-12-26 22:28:01,011][105620] Updated weights for policy 1, policy_version 974461 (0.0010) [2023-12-26 22:28:01,062][104569] Fps is (10 sec: 18022.6, 60 sec: 18978.1, 300 sec: 19327.6). Total num frames: 498974720. Throughput: 0: 9452.0, 1: 9601.7. Samples: 498952880. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:28:01,062][104569] Avg episode reward: [(0, '7586.285'), (1, '8805.219')] [2023-12-26 22:28:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000974408_249487360.pth... [2023-12-26 22:28:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000973352_249217024.pth [2023-12-26 22:28:01,079][105620] Updated weights for policy 1, policy_version 974471 (0.0010) [2023-12-26 22:28:01,085][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000974472_249495552.pth... [2023-12-26 22:28:01,089][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000973352_249208832.pth [2023-12-26 22:28:01,248][105692] Updated weights for policy 0, policy_version 974417 (0.0009) [2023-12-26 22:28:01,312][105692] Updated weights for policy 0, policy_version 974427 (0.0010) [2023-12-26 22:28:01,375][105692] Updated weights for policy 0, policy_version 974437 (0.0009) [2023-12-26 22:28:01,775][105620] Updated weights for policy 1, policy_version 974481 (0.0011) [2023-12-26 22:28:01,840][105620] Updated weights for policy 1, policy_version 974491 (0.0010) [2023-12-26 22:28:01,905][105620] Updated weights for policy 1, policy_version 974501 (0.0010) [2023-12-26 22:28:02,182][105692] Updated weights for policy 0, policy_version 974447 (0.0010) [2023-12-26 22:28:02,242][105692] Updated weights for policy 0, policy_version 974457 (0.0011) [2023-12-26 22:28:02,305][105692] Updated weights for policy 0, policy_version 974467 (0.0011) [2023-12-26 22:28:02,650][105620] Updated weights for policy 1, policy_version 974511 (0.0010) [2023-12-26 22:28:02,718][105620] Updated weights for policy 1, policy_version 974521 (0.0010) [2023-12-26 22:28:02,782][105620] Updated weights for policy 1, policy_version 974531 (0.0010) [2023-12-26 22:28:03,007][105692] Updated weights for policy 0, policy_version 974477 (0.0008) [2023-12-26 22:28:03,053][105692] Updated weights for policy 0, policy_version 974487 (0.0005) [2023-12-26 22:28:03,063][105585] KL-divergence is very high: 412.8422 [2023-12-26 22:28:03,099][105692] Updated weights for policy 0, policy_version 974497 (0.0005) [2023-12-26 22:28:03,099][105585] KL-divergence is very high: 752.2627 [2023-12-26 22:28:03,486][105620] Updated weights for policy 1, policy_version 974541 (0.0010) [2023-12-26 22:28:03,534][105620] Updated weights for policy 1, policy_version 974551 (0.0010) [2023-12-26 22:28:03,582][105620] Updated weights for policy 1, policy_version 974561 (0.0010) [2023-12-26 22:28:03,669][105585] KL-divergence is very high: 296.1692 [2023-12-26 22:28:03,674][105692] Updated weights for policy 0, policy_version 974507 (0.0007) [2023-12-26 22:28:03,710][105585] KL-divergence is very high: 284.2118 [2023-12-26 22:28:03,731][105692] Updated weights for policy 0, policy_version 974517 (0.0009) [2023-12-26 22:28:03,758][105585] KL-divergence is very high: 280.4701 [2023-12-26 22:28:03,790][105692] Updated weights for policy 0, policy_version 974527 (0.0009) [2023-12-26 22:28:03,807][105585] KL-divergence is very high: 287.1311 [2023-12-26 22:28:04,296][105620] Updated weights for policy 1, policy_version 974571 (0.0010) [2023-12-26 22:28:04,370][105620] Updated weights for policy 1, policy_version 974581 (0.0011) [2023-12-26 22:28:04,437][105620] Updated weights for policy 1, policy_version 974591 (0.0011) [2023-12-26 22:28:04,537][105692] Updated weights for policy 0, policy_version 974537 (0.0008) [2023-12-26 22:28:04,602][105692] Updated weights for policy 0, policy_version 974547 (0.0009) [2023-12-26 22:28:04,664][105692] Updated weights for policy 0, policy_version 974557 (0.0008) [2023-12-26 22:28:04,727][105692] Updated weights for policy 0, policy_version 974567 (0.0008) [2023-12-26 22:28:05,169][105620] Updated weights for policy 1, policy_version 974601 (0.0010) [2023-12-26 22:28:05,231][105620] Updated weights for policy 1, policy_version 974611 (0.0009) [2023-12-26 22:28:05,290][105620] Updated weights for policy 1, policy_version 974621 (0.0009) [2023-12-26 22:28:05,338][105620] Updated weights for policy 1, policy_version 974631 (0.0009) [2023-12-26 22:28:05,481][105692] Updated weights for policy 0, policy_version 974577 (0.0009) [2023-12-26 22:28:05,528][105692] Updated weights for policy 0, policy_version 974587 (0.0009) [2023-12-26 22:28:05,580][105692] Updated weights for policy 0, policy_version 974597 (0.0009) [2023-12-26 22:28:06,025][105620] Updated weights for policy 1, policy_version 974641 (0.0009) [2023-12-26 22:28:06,062][104569] Fps is (10 sec: 18022.4, 60 sec: 18978.1, 300 sec: 19327.6). Total num frames: 499073024. Throughput: 0: 9391.3, 1: 9517.7. Samples: 499066116. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:28:06,063][104569] Avg episode reward: [(0, '7332.439'), (1, '9080.544')] [2023-12-26 22:28:06,086][105620] Updated weights for policy 1, policy_version 974651 (0.0008) [2023-12-26 22:28:06,146][105620] Updated weights for policy 1, policy_version 974661 (0.0008) [2023-12-26 22:28:06,394][105692] Updated weights for policy 0, policy_version 974607 (0.0009) [2023-12-26 22:28:06,460][105692] Updated weights for policy 0, policy_version 974617 (0.0009) [2023-12-26 22:28:06,522][105692] Updated weights for policy 0, policy_version 974627 (0.0008) [2023-12-26 22:28:06,895][105620] Updated weights for policy 1, policy_version 974671 (0.0008) [2023-12-26 22:28:06,954][105620] Updated weights for policy 1, policy_version 974681 (0.0009) [2023-12-26 22:28:07,006][105620] Updated weights for policy 1, policy_version 974691 (0.0009) [2023-12-26 22:28:07,301][105692] Updated weights for policy 0, policy_version 974637 (0.0009) [2023-12-26 22:28:07,356][105692] Updated weights for policy 0, policy_version 974647 (0.0009) [2023-12-26 22:28:07,407][105692] Updated weights for policy 0, policy_version 974657 (0.0009) [2023-12-26 22:28:07,694][105620] Updated weights for policy 1, policy_version 974701 (0.0009) [2023-12-26 22:28:07,753][105620] Updated weights for policy 1, policy_version 974711 (0.0009) [2023-12-26 22:28:07,818][105620] Updated weights for policy 1, policy_version 974721 (0.0005) [2023-12-26 22:28:08,289][105692] Updated weights for policy 0, policy_version 974667 (0.0009) [2023-12-26 22:28:08,353][105620] Updated weights for policy 1, policy_version 974731 (0.0006) [2023-12-26 22:28:08,358][105692] Updated weights for policy 0, policy_version 974677 (0.0009) [2023-12-26 22:28:08,417][105620] Updated weights for policy 1, policy_version 974741 (0.0008) [2023-12-26 22:28:08,419][105692] Updated weights for policy 0, policy_version 974687 (0.0007) [2023-12-26 22:28:08,477][105620] Updated weights for policy 1, policy_version 974751 (0.0006) [2023-12-26 22:28:09,177][105620] Updated weights for policy 1, policy_version 974761 (0.0009) [2023-12-26 22:28:09,212][105692] Updated weights for policy 0, policy_version 974697 (0.0007) [2023-12-26 22:28:09,237][105620] Updated weights for policy 1, policy_version 974771 (0.0012) [2023-12-26 22:28:09,278][105692] Updated weights for policy 0, policy_version 974707 (0.0007) [2023-12-26 22:28:09,303][105620] Updated weights for policy 1, policy_version 974781 (0.0007) [2023-12-26 22:28:09,347][105692] Updated weights for policy 0, policy_version 974717 (0.0009) [2023-12-26 22:28:09,371][105620] Updated weights for policy 1, policy_version 974791 (0.0008) [2023-12-26 22:28:09,412][105692] Updated weights for policy 0, policy_version 974727 (0.0008) [2023-12-26 22:28:10,142][105620] Updated weights for policy 1, policy_version 974801 (0.0009) [2023-12-26 22:28:10,196][105620] Updated weights for policy 1, policy_version 974811 (0.0006) [2023-12-26 22:28:10,213][105692] Updated weights for policy 0, policy_version 974737 (0.0009) [2023-12-26 22:28:10,262][105620] Updated weights for policy 1, policy_version 974821 (0.0006) [2023-12-26 22:28:10,275][105692] Updated weights for policy 0, policy_version 974747 (0.0009) [2023-12-26 22:28:10,335][105692] Updated weights for policy 0, policy_version 974757 (0.0008) [2023-12-26 22:28:10,910][105692] Updated weights for policy 0, policy_version 974767 (0.0008) [2023-12-26 22:28:10,958][105692] Updated weights for policy 0, policy_version 974777 (0.0009) [2023-12-26 22:28:11,019][105692] Updated weights for policy 0, policy_version 974787 (0.0008) [2023-12-26 22:28:11,029][105620] Updated weights for policy 1, policy_version 974831 (0.0007) [2023-12-26 22:28:11,062][104569] Fps is (10 sec: 19660.5, 60 sec: 18978.1, 300 sec: 19327.6). Total num frames: 499171328. Throughput: 0: 9340.4, 1: 9541.2. Samples: 499178112. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:28:11,063][104569] Avg episode reward: [(0, '7418.299'), (1, '9083.275')] [2023-12-26 22:28:11,092][105620] Updated weights for policy 1, policy_version 974841 (0.0009) [2023-12-26 22:28:11,172][105620] Updated weights for policy 1, policy_version 974851 (0.0009) [2023-12-26 22:28:11,752][105692] Updated weights for policy 0, policy_version 974797 (0.0009) [2023-12-26 22:28:11,810][105692] Updated weights for policy 0, policy_version 974807 (0.0009) [2023-12-26 22:28:11,876][105692] Updated weights for policy 0, policy_version 974817 (0.0009) [2023-12-26 22:28:11,945][105620] Updated weights for policy 1, policy_version 974861 (0.0009) [2023-12-26 22:28:11,998][105620] Updated weights for policy 1, policy_version 974871 (0.0011) [2023-12-26 22:28:12,061][105620] Updated weights for policy 1, policy_version 974881 (0.0011) [2023-12-26 22:28:12,649][105692] Updated weights for policy 0, policy_version 974827 (0.0008) [2023-12-26 22:28:12,713][105692] Updated weights for policy 0, policy_version 974837 (0.0008) [2023-12-26 22:28:12,770][105692] Updated weights for policy 0, policy_version 974847 (0.0009) [2023-12-26 22:28:12,837][105620] Updated weights for policy 1, policy_version 974891 (0.0011) [2023-12-26 22:28:12,900][105620] Updated weights for policy 1, policy_version 974901 (0.0010) [2023-12-26 22:28:12,961][105620] Updated weights for policy 1, policy_version 974911 (0.0010) [2023-12-26 22:28:13,532][105692] Updated weights for policy 0, policy_version 974857 (0.0009) [2023-12-26 22:28:13,592][105692] Updated weights for policy 0, policy_version 974868 (0.0010) [2023-12-26 22:28:13,643][105620] Updated weights for policy 1, policy_version 974921 (0.0010) [2023-12-26 22:28:13,644][105692] Updated weights for policy 0, policy_version 974878 (0.0010) [2023-12-26 22:28:13,698][105692] Updated weights for policy 0, policy_version 974888 (0.0008) [2023-12-26 22:28:13,700][105620] Updated weights for policy 1, policy_version 974931 (0.0006) [2023-12-26 22:28:13,754][105620] Updated weights for policy 1, policy_version 974941 (0.0007) [2023-12-26 22:28:13,804][105620] Updated weights for policy 1, policy_version 974951 (0.0009) [2023-12-26 22:28:14,438][105620] Updated weights for policy 1, policy_version 974961 (0.0009) [2023-12-26 22:28:14,495][105620] Updated weights for policy 1, policy_version 974971 (0.0009) [2023-12-26 22:28:14,502][105692] Updated weights for policy 0, policy_version 974898 (0.0005) [2023-12-26 22:28:14,555][105620] Updated weights for policy 1, policy_version 974981 (0.0008) [2023-12-26 22:28:14,555][105692] Updated weights for policy 0, policy_version 974908 (0.0005) [2023-12-26 22:28:14,608][105692] Updated weights for policy 0, policy_version 974918 (0.0006) [2023-12-26 22:28:15,244][105692] Updated weights for policy 0, policy_version 974928 (0.0007) [2023-12-26 22:28:15,312][105692] Updated weights for policy 0, policy_version 974938 (0.0006) [2023-12-26 22:28:15,343][105620] Updated weights for policy 1, policy_version 974991 (0.0010) [2023-12-26 22:28:15,368][105692] Updated weights for policy 0, policy_version 974948 (0.0008) [2023-12-26 22:28:15,404][105620] Updated weights for policy 1, policy_version 975001 (0.0011) [2023-12-26 22:28:15,457][105620] Updated weights for policy 1, policy_version 975011 (0.0010) [2023-12-26 22:28:16,060][105692] Updated weights for policy 0, policy_version 974958 (0.0009) [2023-12-26 22:28:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 18978.1, 300 sec: 19327.6). Total num frames: 499261440. Throughput: 0: 9252.4, 1: 9507.8. Samples: 499235372. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:28:16,062][104569] Avg episode reward: [(0, '7627.875'), (1, '9084.281')] [2023-12-26 22:28:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000975016_249634816.pth... [2023-12-26 22:28:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000973896_249348096.pth [2023-12-26 22:28:16,126][105692] Updated weights for policy 0, policy_version 974968 (0.0010) [2023-12-26 22:28:16,158][105620] Updated weights for policy 1, policy_version 975021 (0.0010) [2023-12-26 22:28:16,192][105692] Updated weights for policy 0, policy_version 974978 (0.0010) [2023-12-26 22:28:16,211][105620] Updated weights for policy 1, policy_version 975031 (0.0010) [2023-12-26 22:28:16,227][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000974984_249634816.pth... [2023-12-26 22:28:16,232][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000973896_249356288.pth [2023-12-26 22:28:16,260][105620] Updated weights for policy 1, policy_version 975041 (0.0010) [2023-12-26 22:28:16,842][105620] Updated weights for policy 1, policy_version 975051 (0.0009) [2023-12-26 22:28:16,894][105620] Updated weights for policy 1, policy_version 975061 (0.0010) [2023-12-26 22:28:16,947][105620] Updated weights for policy 1, policy_version 975071 (0.0007) [2023-12-26 22:28:16,952][105692] Updated weights for policy 0, policy_version 974988 (0.0009) [2023-12-26 22:28:17,015][105692] Updated weights for policy 0, policy_version 974998 (0.0006) [2023-12-26 22:28:17,070][105692] Updated weights for policy 0, policy_version 975008 (0.0006) [2023-12-26 22:28:17,684][105620] Updated weights for policy 1, policy_version 975081 (0.0008) [2023-12-26 22:28:17,751][105620] Updated weights for policy 1, policy_version 975091 (0.0010) [2023-12-26 22:28:17,754][105692] Updated weights for policy 0, policy_version 975018 (0.0006) [2023-12-26 22:28:17,810][105620] Updated weights for policy 1, policy_version 975101 (0.0010) [2023-12-26 22:28:17,812][105692] Updated weights for policy 0, policy_version 975028 (0.0005) [2023-12-26 22:28:17,863][105692] Updated weights for policy 0, policy_version 975038 (0.0005) [2023-12-26 22:28:17,869][105620] Updated weights for policy 1, policy_version 975111 (0.0010) [2023-12-26 22:28:17,922][105692] Updated weights for policy 0, policy_version 975048 (0.0006) [2023-12-26 22:28:18,535][105692] Updated weights for policy 0, policy_version 975058 (0.0010) [2023-12-26 22:28:18,595][105692] Updated weights for policy 0, policy_version 975068 (0.0010) [2023-12-26 22:28:18,612][105620] Updated weights for policy 1, policy_version 975121 (0.0010) [2023-12-26 22:28:18,654][105692] Updated weights for policy 0, policy_version 975078 (0.0011) [2023-12-26 22:28:18,671][105620] Updated weights for policy 1, policy_version 975131 (0.0010) [2023-12-26 22:28:18,723][105620] Updated weights for policy 1, policy_version 975141 (0.0010) [2023-12-26 22:28:19,386][105692] Updated weights for policy 0, policy_version 975088 (0.0010) [2023-12-26 22:28:19,449][105692] Updated weights for policy 0, policy_version 975098 (0.0010) [2023-12-26 22:28:19,519][105692] Updated weights for policy 0, policy_version 975108 (0.0010) [2023-12-26 22:28:19,531][105620] Updated weights for policy 1, policy_version 975151 (0.0009) [2023-12-26 22:28:19,594][105620] Updated weights for policy 1, policy_version 975161 (0.0007) [2023-12-26 22:28:19,661][105620] Updated weights for policy 1, policy_version 975171 (0.0008) [2023-12-26 22:28:20,268][105692] Updated weights for policy 0, policy_version 975118 (0.0010) [2023-12-26 22:28:20,334][105692] Updated weights for policy 0, policy_version 975128 (0.0011) [2023-12-26 22:28:20,401][105692] Updated weights for policy 0, policy_version 975138 (0.0011) [2023-12-26 22:28:20,427][105620] Updated weights for policy 1, policy_version 975181 (0.0010) [2023-12-26 22:28:20,490][105620] Updated weights for policy 1, policy_version 975191 (0.0009) [2023-12-26 22:28:20,557][105620] Updated weights for policy 1, policy_version 975201 (0.0008) [2023-12-26 22:28:21,062][104569] Fps is (10 sec: 18842.0, 60 sec: 18978.2, 300 sec: 19327.6). Total num frames: 499359744. Throughput: 0: 9279.4, 1: 9539.1. Samples: 499352220. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:28:21,062][104569] Avg episode reward: [(0, '7574.437'), (1, '9086.039')] [2023-12-26 22:28:21,180][105692] Updated weights for policy 0, policy_version 975148 (0.0011) [2023-12-26 22:28:21,245][105692] Updated weights for policy 0, policy_version 975158 (0.0010) [2023-12-26 22:28:21,256][105620] Updated weights for policy 1, policy_version 975211 (0.0009) [2023-12-26 22:28:21,306][105692] Updated weights for policy 0, policy_version 975168 (0.0007) [2023-12-26 22:28:21,323][105620] Updated weights for policy 1, policy_version 975221 (0.0011) [2023-12-26 22:28:21,391][105620] Updated weights for policy 1, policy_version 975231 (0.0009) [2023-12-26 22:28:22,010][105692] Updated weights for policy 0, policy_version 975178 (0.0008) [2023-12-26 22:28:22,039][105620] Updated weights for policy 1, policy_version 975241 (0.0006) [2023-12-26 22:28:22,077][105692] Updated weights for policy 0, policy_version 975188 (0.0009) [2023-12-26 22:28:22,103][105620] Updated weights for policy 1, policy_version 975251 (0.0007) [2023-12-26 22:28:22,138][105692] Updated weights for policy 0, policy_version 975198 (0.0010) [2023-12-26 22:28:22,168][105620] Updated weights for policy 1, policy_version 975261 (0.0011) [2023-12-26 22:28:22,202][105692] Updated weights for policy 0, policy_version 975208 (0.0011) [2023-12-26 22:28:22,230][105620] Updated weights for policy 1, policy_version 975271 (0.0010) [2023-12-26 22:28:22,958][105620] Updated weights for policy 1, policy_version 975281 (0.0006) [2023-12-26 22:28:22,975][105692] Updated weights for policy 0, policy_version 975218 (0.0011) [2023-12-26 22:28:23,012][105620] Updated weights for policy 1, policy_version 975291 (0.0009) [2023-12-26 22:28:23,035][105692] Updated weights for policy 0, policy_version 975228 (0.0011) [2023-12-26 22:28:23,065][105620] Updated weights for policy 1, policy_version 975301 (0.0005) [2023-12-26 22:28:23,094][105692] Updated weights for policy 0, policy_version 975238 (0.0011) [2023-12-26 22:28:23,833][105620] Updated weights for policy 1, policy_version 975311 (0.0007) [2023-12-26 22:28:23,835][105692] Updated weights for policy 0, policy_version 975248 (0.0010) [2023-12-26 22:28:23,884][105620] Updated weights for policy 1, policy_version 975321 (0.0005) [2023-12-26 22:28:23,886][105692] Updated weights for policy 0, policy_version 975258 (0.0010) [2023-12-26 22:28:23,932][105620] Updated weights for policy 1, policy_version 975331 (0.0005) [2023-12-26 22:28:23,934][105692] Updated weights for policy 0, policy_version 975268 (0.0010) [2023-12-26 22:28:24,602][105692] Updated weights for policy 0, policy_version 975278 (0.0010) [2023-12-26 22:28:24,660][105692] Updated weights for policy 0, policy_version 975288 (0.0010) [2023-12-26 22:28:24,682][105620] Updated weights for policy 1, policy_version 975341 (0.0005) [2023-12-26 22:28:24,715][105692] Updated weights for policy 0, policy_version 975298 (0.0010) [2023-12-26 22:28:24,737][105620] Updated weights for policy 1, policy_version 975351 (0.0006) [2023-12-26 22:28:24,799][105620] Updated weights for policy 1, policy_version 975361 (0.0007) [2023-12-26 22:28:25,420][105692] Updated weights for policy 0, policy_version 975308 (0.0010) [2023-12-26 22:28:25,475][105692] Updated weights for policy 0, policy_version 975318 (0.0010) [2023-12-26 22:28:25,526][105692] Updated weights for policy 0, policy_version 975329 (0.0009) [2023-12-26 22:28:25,535][105620] Updated weights for policy 1, policy_version 975371 (0.0010) [2023-12-26 22:28:25,595][105620] Updated weights for policy 1, policy_version 975381 (0.0007) [2023-12-26 22:28:25,656][105620] Updated weights for policy 1, policy_version 975391 (0.0005) [2023-12-26 22:28:26,062][104569] Fps is (10 sec: 19660.1, 60 sec: 18978.0, 300 sec: 19327.5). Total num frames: 499458048. Throughput: 0: 9320.6, 1: 9618.6. Samples: 499468304. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:28:26,063][104569] Avg episode reward: [(0, '8013.591'), (1, '8903.280')] [2023-12-26 22:28:26,226][105620] Updated weights for policy 1, policy_version 975401 (0.0006) [2023-12-26 22:28:26,280][105692] Updated weights for policy 0, policy_version 975339 (0.0008) [2023-12-26 22:28:26,282][105620] Updated weights for policy 1, policy_version 975411 (0.0008) [2023-12-26 22:28:26,331][105620] Updated weights for policy 1, policy_version 975421 (0.0005) [2023-12-26 22:28:26,338][105692] Updated weights for policy 0, policy_version 975349 (0.0008) [2023-12-26 22:28:26,388][105620] Updated weights for policy 1, policy_version 975431 (0.0005) [2023-12-26 22:28:26,392][105692] Updated weights for policy 0, policy_version 975359 (0.0009) [2023-12-26 22:28:26,949][105620] Updated weights for policy 1, policy_version 975441 (0.0008) [2023-12-26 22:28:27,004][105620] Updated weights for policy 1, policy_version 975451 (0.0006) [2023-12-26 22:28:27,063][105620] Updated weights for policy 1, policy_version 975461 (0.0006) [2023-12-26 22:28:27,132][105692] Updated weights for policy 0, policy_version 975369 (0.0009) [2023-12-26 22:28:27,194][105692] Updated weights for policy 0, policy_version 975379 (0.0009) [2023-12-26 22:28:27,265][105692] Updated weights for policy 0, policy_version 975389 (0.0010) [2023-12-26 22:28:27,338][105692] Updated weights for policy 0, policy_version 975400 (0.0008) [2023-12-26 22:28:27,657][105620] Updated weights for policy 1, policy_version 975471 (0.0010) [2023-12-26 22:28:27,712][105620] Updated weights for policy 1, policy_version 975481 (0.0010) [2023-12-26 22:28:27,770][105620] Updated weights for policy 1, policy_version 975491 (0.0010) [2023-12-26 22:28:28,157][105692] Updated weights for policy 0, policy_version 975410 (0.0010) [2023-12-26 22:28:28,209][105692] Updated weights for policy 0, policy_version 975420 (0.0006) [2023-12-26 22:28:28,267][105692] Updated weights for policy 0, policy_version 975430 (0.0005) [2023-12-26 22:28:28,380][105620] Updated weights for policy 1, policy_version 975501 (0.0009) [2023-12-26 22:28:28,445][105620] Updated weights for policy 1, policy_version 975511 (0.0008) [2023-12-26 22:28:28,540][105620] Updated weights for policy 1, policy_version 975521 (0.0008) [2023-12-26 22:28:29,020][105692] Updated weights for policy 0, policy_version 975440 (0.0009) [2023-12-26 22:28:29,087][105692] Updated weights for policy 0, policy_version 975450 (0.0010) [2023-12-26 22:28:29,130][105620] Updated weights for policy 1, policy_version 975531 (0.0008) [2023-12-26 22:28:29,152][105692] Updated weights for policy 0, policy_version 975460 (0.0006) [2023-12-26 22:28:29,185][105620] Updated weights for policy 1, policy_version 975541 (0.0010) [2023-12-26 22:28:29,243][105620] Updated weights for policy 1, policy_version 975551 (0.0007) [2023-12-26 22:28:29,903][105692] Updated weights for policy 0, policy_version 975470 (0.0008) [2023-12-26 22:28:29,928][105620] Updated weights for policy 1, policy_version 975561 (0.0006) [2023-12-26 22:28:29,966][105692] Updated weights for policy 0, policy_version 975480 (0.0008) [2023-12-26 22:28:29,999][105620] Updated weights for policy 1, policy_version 975571 (0.0010) [2023-12-26 22:28:30,021][105692] Updated weights for policy 0, policy_version 975490 (0.0007) [2023-12-26 22:28:30,065][105620] Updated weights for policy 1, policy_version 975581 (0.0010) [2023-12-26 22:28:30,113][105620] Updated weights for policy 1, policy_version 975591 (0.0010) [2023-12-26 22:28:30,756][105620] Updated weights for policy 1, policy_version 975601 (0.0008) [2023-12-26 22:28:30,819][105620] Updated weights for policy 1, policy_version 975611 (0.0007) [2023-12-26 22:28:30,826][105692] Updated weights for policy 0, policy_version 975500 (0.0006) [2023-12-26 22:28:30,879][105620] Updated weights for policy 1, policy_version 975621 (0.0005) [2023-12-26 22:28:30,883][105692] Updated weights for policy 0, policy_version 975510 (0.0005) [2023-12-26 22:28:30,933][105692] Updated weights for policy 0, policy_version 975520 (0.0006) [2023-12-26 22:28:31,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19114.6, 300 sec: 19355.3). Total num frames: 499564544. Throughput: 0: 9330.0, 1: 9778.2. Samples: 499530320. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:28:31,063][104569] Avg episode reward: [(0, '8755.631'), (1, '8810.996')] [2023-12-26 22:28:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000975624_249790464.pth... [2023-12-26 22:28:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000975528_249774080.pth... [2023-12-26 22:28:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000974408_249487360.pth [2023-12-26 22:28:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000974472_249495552.pth [2023-12-26 22:28:31,574][105620] Updated weights for policy 1, policy_version 975631 (0.0009) [2023-12-26 22:28:31,580][105692] Updated weights for policy 0, policy_version 975530 (0.0008) [2023-12-26 22:28:31,640][105692] Updated weights for policy 0, policy_version 975540 (0.0006) [2023-12-26 22:28:31,644][105620] Updated weights for policy 1, policy_version 975641 (0.0009) [2023-12-26 22:28:31,696][105692] Updated weights for policy 0, policy_version 975550 (0.0011) [2023-12-26 22:28:31,706][105620] Updated weights for policy 1, policy_version 975651 (0.0008) [2023-12-26 22:28:31,763][105692] Updated weights for policy 0, policy_version 975560 (0.0009) [2023-12-26 22:28:32,486][105620] Updated weights for policy 1, policy_version 975661 (0.0007) [2023-12-26 22:28:32,518][105692] Updated weights for policy 0, policy_version 975570 (0.0005) [2023-12-26 22:28:32,560][105620] Updated weights for policy 1, policy_version 975671 (0.0006) [2023-12-26 22:28:32,578][105692] Updated weights for policy 0, policy_version 975580 (0.0009) [2023-12-26 22:28:32,634][105620] Updated weights for policy 1, policy_version 975681 (0.0005) [2023-12-26 22:28:32,637][105692] Updated weights for policy 0, policy_version 975590 (0.0010) [2023-12-26 22:28:33,185][105620] Updated weights for policy 1, policy_version 975691 (0.0006) [2023-12-26 22:28:33,233][105620] Updated weights for policy 1, policy_version 975701 (0.0008) [2023-12-26 22:28:33,282][105692] Updated weights for policy 0, policy_version 975600 (0.0006) [2023-12-26 22:28:33,282][105620] Updated weights for policy 1, policy_version 975711 (0.0008) [2023-12-26 22:28:33,333][105692] Updated weights for policy 0, policy_version 975610 (0.0010) [2023-12-26 22:28:33,387][105692] Updated weights for policy 0, policy_version 975620 (0.0010) [2023-12-26 22:28:33,851][105620] Updated weights for policy 1, policy_version 975721 (0.0007) [2023-12-26 22:28:33,902][105620] Updated weights for policy 1, policy_version 975731 (0.0005) [2023-12-26 22:28:33,959][105620] Updated weights for policy 1, policy_version 975741 (0.0005) [2023-12-26 22:28:34,010][105620] Updated weights for policy 1, policy_version 975751 (0.0005) [2023-12-26 22:28:34,157][105692] Updated weights for policy 0, policy_version 975630 (0.0009) [2023-12-26 22:28:34,223][105692] Updated weights for policy 0, policy_version 975640 (0.0008) [2023-12-26 22:28:34,278][105692] Updated weights for policy 0, policy_version 975650 (0.0008) [2023-12-26 22:28:34,639][105620] Updated weights for policy 1, policy_version 975761 (0.0009) [2023-12-26 22:28:34,700][105620] Updated weights for policy 1, policy_version 975771 (0.0009) [2023-12-26 22:28:34,759][105620] Updated weights for policy 1, policy_version 975781 (0.0010) [2023-12-26 22:28:34,981][105692] Updated weights for policy 0, policy_version 975660 (0.0009) [2023-12-26 22:28:35,030][105692] Updated weights for policy 0, policy_version 975670 (0.0009) [2023-12-26 22:28:35,091][105692] Updated weights for policy 0, policy_version 975680 (0.0006) [2023-12-26 22:28:35,441][105620] Updated weights for policy 1, policy_version 975791 (0.0010) [2023-12-26 22:28:35,503][105620] Updated weights for policy 1, policy_version 975801 (0.0010) [2023-12-26 22:28:35,564][105620] Updated weights for policy 1, policy_version 975811 (0.0010) [2023-12-26 22:28:35,765][105692] Updated weights for policy 0, policy_version 975690 (0.0007) [2023-12-26 22:28:35,828][105692] Updated weights for policy 0, policy_version 975700 (0.0011) [2023-12-26 22:28:35,884][105692] Updated weights for policy 0, policy_version 975710 (0.0010) [2023-12-26 22:28:35,942][105692] Updated weights for policy 0, policy_version 975720 (0.0011) [2023-12-26 22:28:36,062][104569] Fps is (10 sec: 20480.7, 60 sec: 19114.7, 300 sec: 19355.3). Total num frames: 499662848. Throughput: 0: 9397.7, 1: 9882.2. Samples: 499650064. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:28:36,062][104569] Avg episode reward: [(0, '8544.124'), (1, '8993.296')] [2023-12-26 22:28:36,214][105620] Updated weights for policy 1, policy_version 975821 (0.0010) [2023-12-26 22:28:36,284][105620] Updated weights for policy 1, policy_version 975831 (0.0011) [2023-12-26 22:28:36,357][105620] Updated weights for policy 1, policy_version 975841 (0.0010) [2023-12-26 22:28:36,708][105692] Updated weights for policy 0, policy_version 975730 (0.0011) [2023-12-26 22:28:36,770][105692] Updated weights for policy 0, policy_version 975740 (0.0011) [2023-12-26 22:28:36,825][105692] Updated weights for policy 0, policy_version 975750 (0.0011) [2023-12-26 22:28:36,945][105620] Updated weights for policy 1, policy_version 975851 (0.0009) [2023-12-26 22:28:37,001][105620] Updated weights for policy 1, policy_version 975861 (0.0005) [2023-12-26 22:28:37,058][105620] Updated weights for policy 1, policy_version 975871 (0.0005) [2023-12-26 22:28:37,574][105692] Updated weights for policy 0, policy_version 975760 (0.0011) [2023-12-26 22:28:37,613][105620] Updated weights for policy 1, policy_version 975881 (0.0005) [2023-12-26 22:28:37,637][105692] Updated weights for policy 0, policy_version 975770 (0.0011) [2023-12-26 22:28:37,681][105620] Updated weights for policy 1, policy_version 975891 (0.0006) [2023-12-26 22:28:37,694][105692] Updated weights for policy 0, policy_version 975780 (0.0011) [2023-12-26 22:28:37,746][105620] Updated weights for policy 1, policy_version 975901 (0.0006) [2023-12-26 22:28:37,805][105620] Updated weights for policy 1, policy_version 975911 (0.0007) [2023-12-26 22:28:38,401][105620] Updated weights for policy 1, policy_version 975921 (0.0011) [2023-12-26 22:28:38,456][105692] Updated weights for policy 0, policy_version 975790 (0.0009) [2023-12-26 22:28:38,463][105620] Updated weights for policy 1, policy_version 975931 (0.0008) [2023-12-26 22:28:38,511][105692] Updated weights for policy 0, policy_version 975800 (0.0008) [2023-12-26 22:28:38,525][105620] Updated weights for policy 1, policy_version 975941 (0.0011) [2023-12-26 22:28:38,560][105692] Updated weights for policy 0, policy_version 975810 (0.0011) [2023-12-26 22:28:39,239][105620] Updated weights for policy 1, policy_version 975951 (0.0010) [2023-12-26 22:28:39,301][105620] Updated weights for policy 1, policy_version 975961 (0.0011) [2023-12-26 22:28:39,316][105692] Updated weights for policy 0, policy_version 975820 (0.0011) [2023-12-26 22:28:39,367][105620] Updated weights for policy 1, policy_version 975971 (0.0012) [2023-12-26 22:28:39,379][105692] Updated weights for policy 0, policy_version 975830 (0.0009) [2023-12-26 22:28:39,452][105692] Updated weights for policy 0, policy_version 975840 (0.0009) [2023-12-26 22:28:40,186][105692] Updated weights for policy 0, policy_version 975850 (0.0008) [2023-12-26 22:28:40,191][105620] Updated weights for policy 1, policy_version 975981 (0.0009) [2023-12-26 22:28:40,247][105692] Updated weights for policy 0, policy_version 975860 (0.0006) [2023-12-26 22:28:40,254][105620] Updated weights for policy 1, policy_version 975991 (0.0007) [2023-12-26 22:28:40,306][105692] Updated weights for policy 0, policy_version 975870 (0.0007) [2023-12-26 22:28:40,316][105620] Updated weights for policy 1, policy_version 976001 (0.0006) [2023-12-26 22:28:40,369][105692] Updated weights for policy 0, policy_version 975880 (0.0008) [2023-12-26 22:28:41,032][105692] Updated weights for policy 0, policy_version 975890 (0.0006) [2023-12-26 22:28:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.6, 300 sec: 19355.3). Total num frames: 499752960. Throughput: 0: 9387.3, 1: 9973.0. Samples: 499767644. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:28:41,063][104569] Avg episode reward: [(0, '8450.046'), (1, '9081.341')] [2023-12-26 22:28:41,103][105692] Updated weights for policy 0, policy_version 975900 (0.0007) [2023-12-26 22:28:41,145][105620] Updated weights for policy 1, policy_version 976011 (0.0007) [2023-12-26 22:28:41,174][105692] Updated weights for policy 0, policy_version 975910 (0.0007) [2023-12-26 22:28:41,206][105620] Updated weights for policy 1, policy_version 976021 (0.0009) [2023-12-26 22:28:41,270][105620] Updated weights for policy 1, policy_version 976031 (0.0009) [2023-12-26 22:28:41,792][105692] Updated weights for policy 0, policy_version 975920 (0.0008) [2023-12-26 22:28:41,848][105692] Updated weights for policy 0, policy_version 975930 (0.0008) [2023-12-26 22:28:41,905][105692] Updated weights for policy 0, policy_version 975940 (0.0008) [2023-12-26 22:28:42,096][105620] Updated weights for policy 1, policy_version 976041 (0.0008) [2023-12-26 22:28:42,162][105620] Updated weights for policy 1, policy_version 976051 (0.0009) [2023-12-26 22:28:42,227][105620] Updated weights for policy 1, policy_version 976061 (0.0009) [2023-12-26 22:28:42,284][105620] Updated weights for policy 1, policy_version 976071 (0.0009) [2023-12-26 22:28:42,628][105692] Updated weights for policy 0, policy_version 975950 (0.0009) [2023-12-26 22:28:42,695][105692] Updated weights for policy 0, policy_version 975960 (0.0009) [2023-12-26 22:28:42,756][105692] Updated weights for policy 0, policy_version 975970 (0.0008) [2023-12-26 22:28:43,055][105620] Updated weights for policy 1, policy_version 976081 (0.0009) [2023-12-26 22:28:43,113][105620] Updated weights for policy 1, policy_version 976091 (0.0009) [2023-12-26 22:28:43,158][105620] Updated weights for policy 1, policy_version 976101 (0.0006) [2023-12-26 22:28:43,488][105692] Updated weights for policy 0, policy_version 975980 (0.0009) [2023-12-26 22:28:43,544][105692] Updated weights for policy 0, policy_version 975990 (0.0009) [2023-12-26 22:28:43,599][105692] Updated weights for policy 0, policy_version 976000 (0.0009) [2023-12-26 22:28:43,834][105620] Updated weights for policy 1, policy_version 976111 (0.0008) [2023-12-26 22:28:43,905][105620] Updated weights for policy 1, policy_version 976121 (0.0007) [2023-12-26 22:28:43,967][105620] Updated weights for policy 1, policy_version 976131 (0.0008) [2023-12-26 22:28:44,344][105692] Updated weights for policy 0, policy_version 976010 (0.0010) [2023-12-26 22:28:44,403][105692] Updated weights for policy 0, policy_version 976020 (0.0009) [2023-12-26 22:28:44,462][105692] Updated weights for policy 0, policy_version 976030 (0.0009) [2023-12-26 22:28:44,525][105692] Updated weights for policy 0, policy_version 976040 (0.0009) [2023-12-26 22:28:44,685][105620] Updated weights for policy 1, policy_version 976141 (0.0009) [2023-12-26 22:28:44,749][105620] Updated weights for policy 1, policy_version 976151 (0.0009) [2023-12-26 22:28:44,811][105620] Updated weights for policy 1, policy_version 976161 (0.0009) [2023-12-26 22:28:45,318][105692] Updated weights for policy 0, policy_version 976050 (0.0009) [2023-12-26 22:28:45,377][105692] Updated weights for policy 0, policy_version 976060 (0.0009) [2023-12-26 22:28:45,444][105692] Updated weights for policy 0, policy_version 976070 (0.0009) [2023-12-26 22:28:45,536][105620] Updated weights for policy 1, policy_version 976171 (0.0009) [2023-12-26 22:28:45,591][105620] Updated weights for policy 1, policy_version 976181 (0.0009) [2023-12-26 22:28:45,646][105620] Updated weights for policy 1, policy_version 976191 (0.0008) [2023-12-26 22:28:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19251.3, 300 sec: 19355.3). Total num frames: 499851264. Throughput: 0: 9461.0, 1: 9911.2. Samples: 499824632. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:28:46,062][104569] Avg episode reward: [(0, '8636.461'), (1, '9171.987')] [2023-12-26 22:28:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000976072_249913344.pth... [2023-12-26 22:28:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000976200_249937920.pth... [2023-12-26 22:28:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000974984_249634816.pth [2023-12-26 22:28:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000975016_249634816.pth [2023-12-26 22:28:46,171][105692] Updated weights for policy 0, policy_version 976080 (0.0009) [2023-12-26 22:28:46,229][105692] Updated weights for policy 0, policy_version 976090 (0.0010) [2023-12-26 22:28:46,291][105692] Updated weights for policy 0, policy_version 976100 (0.0010) [2023-12-26 22:28:46,362][105620] Updated weights for policy 1, policy_version 976201 (0.0008) [2023-12-26 22:28:46,419][105620] Updated weights for policy 1, policy_version 976211 (0.0005) [2023-12-26 22:28:46,472][105620] Updated weights for policy 1, policy_version 976221 (0.0005) [2023-12-26 22:28:46,527][105620] Updated weights for policy 1, policy_version 976231 (0.0007) [2023-12-26 22:28:47,068][105620] Updated weights for policy 1, policy_version 976241 (0.0006) [2023-12-26 22:28:47,129][105620] Updated weights for policy 1, policy_version 976251 (0.0010) [2023-12-26 22:28:47,157][105692] Updated weights for policy 0, policy_version 976110 (0.0007) [2023-12-26 22:28:47,189][105620] Updated weights for policy 1, policy_version 976261 (0.0011) [2023-12-26 22:28:47,210][105692] Updated weights for policy 0, policy_version 976120 (0.0005) [2023-12-26 22:28:47,268][105692] Updated weights for policy 0, policy_version 976130 (0.0008) [2023-12-26 22:28:47,857][105620] Updated weights for policy 1, policy_version 976271 (0.0007) [2023-12-26 22:28:47,920][105620] Updated weights for policy 1, policy_version 976281 (0.0009) [2023-12-26 22:28:47,983][105620] Updated weights for policy 1, policy_version 976291 (0.0011) [2023-12-26 22:28:48,036][105692] Updated weights for policy 0, policy_version 976140 (0.0009) [2023-12-26 22:28:48,092][105692] Updated weights for policy 0, policy_version 976150 (0.0008) [2023-12-26 22:28:48,153][105692] Updated weights for policy 0, policy_version 976160 (0.0009) [2023-12-26 22:28:48,716][105620] Updated weights for policy 1, policy_version 976301 (0.0011) [2023-12-26 22:28:48,778][105620] Updated weights for policy 1, policy_version 976311 (0.0010) [2023-12-26 22:28:48,837][105620] Updated weights for policy 1, policy_version 976321 (0.0011) [2023-12-26 22:28:48,916][105692] Updated weights for policy 0, policy_version 976170 (0.0009) [2023-12-26 22:28:48,979][105692] Updated weights for policy 0, policy_version 976180 (0.0010) [2023-12-26 22:28:49,039][105692] Updated weights for policy 0, policy_version 976190 (0.0008) [2023-12-26 22:28:49,102][105692] Updated weights for policy 0, policy_version 976200 (0.0007) [2023-12-26 22:28:49,587][105620] Updated weights for policy 1, policy_version 976331 (0.0011) [2023-12-26 22:28:49,651][105620] Updated weights for policy 1, policy_version 976341 (0.0011) [2023-12-26 22:28:49,711][105620] Updated weights for policy 1, policy_version 976351 (0.0011) [2023-12-26 22:28:49,801][105692] Updated weights for policy 0, policy_version 976210 (0.0008) [2023-12-26 22:28:49,871][105692] Updated weights for policy 0, policy_version 976220 (0.0009) [2023-12-26 22:28:49,935][105692] Updated weights for policy 0, policy_version 976230 (0.0011) [2023-12-26 22:28:50,409][105620] Updated weights for policy 1, policy_version 976361 (0.0007) [2023-12-26 22:28:50,469][105620] Updated weights for policy 1, policy_version 976371 (0.0008) [2023-12-26 22:28:50,520][105620] Updated weights for policy 1, policy_version 976381 (0.0008) [2023-12-26 22:28:50,581][105620] Updated weights for policy 1, policy_version 976391 (0.0008) [2023-12-26 22:28:50,672][105692] Updated weights for policy 0, policy_version 976240 (0.0009) [2023-12-26 22:28:50,730][105692] Updated weights for policy 0, policy_version 976250 (0.0009) [2023-12-26 22:28:50,800][105692] Updated weights for policy 0, policy_version 976260 (0.0009) [2023-12-26 22:28:51,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.3, 300 sec: 19355.3). Total num frames: 499949568. Throughput: 0: 9429.4, 1: 9963.1. Samples: 499938780. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:28:51,062][104569] Avg episode reward: [(0, '8284.681'), (1, '8991.852')] [2023-12-26 22:28:51,439][105620] Updated weights for policy 1, policy_version 976401 (0.0009) [2023-12-26 22:28:51,494][105620] Updated weights for policy 1, policy_version 976411 (0.0008) [2023-12-26 22:28:51,558][105620] Updated weights for policy 1, policy_version 976421 (0.0009) [2023-12-26 22:28:51,646][105692] Updated weights for policy 0, policy_version 976270 (0.0009) [2023-12-26 22:28:51,715][105692] Updated weights for policy 0, policy_version 976280 (0.0009) [2023-12-26 22:28:51,779][105692] Updated weights for policy 0, policy_version 976290 (0.0006) [2023-12-26 22:28:52,244][105620] Updated weights for policy 1, policy_version 976431 (0.0007) [2023-12-26 22:28:52,314][105620] Updated weights for policy 1, policy_version 976441 (0.0009) [2023-12-26 22:28:52,393][105620] Updated weights for policy 1, policy_version 976451 (0.0009) [2023-12-26 22:28:52,530][105692] Updated weights for policy 0, policy_version 976300 (0.0009) [2023-12-26 22:28:52,590][105692] Updated weights for policy 0, policy_version 976310 (0.0011) [2023-12-26 22:28:52,658][105692] Updated weights for policy 0, policy_version 976320 (0.0011) [2023-12-26 22:28:53,114][105620] Updated weights for policy 1, policy_version 976461 (0.0008) [2023-12-26 22:28:53,165][105620] Updated weights for policy 1, policy_version 976471 (0.0008) [2023-12-26 22:28:53,217][105620] Updated weights for policy 1, policy_version 976481 (0.0006) [2023-12-26 22:28:53,395][105692] Updated weights for policy 0, policy_version 976330 (0.0011) [2023-12-26 22:28:53,464][105692] Updated weights for policy 0, policy_version 976340 (0.0011) [2023-12-26 22:28:53,520][105692] Updated weights for policy 0, policy_version 976350 (0.0009) [2023-12-26 22:28:53,585][105692] Updated weights for policy 0, policy_version 976360 (0.0010) [2023-12-26 22:28:53,872][105620] Updated weights for policy 1, policy_version 976491 (0.0007) [2023-12-26 22:28:53,937][105620] Updated weights for policy 1, policy_version 976501 (0.0010) [2023-12-26 22:28:53,993][105620] Updated weights for policy 1, policy_version 976511 (0.0010) [2023-12-26 22:28:54,302][105692] Updated weights for policy 0, policy_version 976370 (0.0010) [2023-12-26 22:28:54,363][105692] Updated weights for policy 0, policy_version 976380 (0.0010) [2023-12-26 22:28:54,418][105692] Updated weights for policy 0, policy_version 976390 (0.0011) [2023-12-26 22:28:54,733][105620] Updated weights for policy 1, policy_version 976521 (0.0010) [2023-12-26 22:28:54,791][105620] Updated weights for policy 1, policy_version 976531 (0.0009) [2023-12-26 22:28:54,839][105620] Updated weights for policy 1, policy_version 976541 (0.0010) [2023-12-26 22:28:54,902][105620] Updated weights for policy 1, policy_version 976551 (0.0011) [2023-12-26 22:28:55,090][105692] Updated weights for policy 0, policy_version 976400 (0.0010) [2023-12-26 22:28:55,145][105692] Updated weights for policy 0, policy_version 976410 (0.0008) [2023-12-26 22:28:55,204][105692] Updated weights for policy 0, policy_version 976420 (0.0006) [2023-12-26 22:28:55,568][105620] Updated weights for policy 1, policy_version 976561 (0.0006) [2023-12-26 22:28:55,631][105620] Updated weights for policy 1, policy_version 976571 (0.0006) [2023-12-26 22:28:55,700][105620] Updated weights for policy 1, policy_version 976581 (0.0010) [2023-12-26 22:28:55,813][105692] Updated weights for policy 0, policy_version 976430 (0.0007) [2023-12-26 22:28:55,878][105692] Updated weights for policy 0, policy_version 976440 (0.0009) [2023-12-26 22:28:55,943][105692] Updated weights for policy 0, policy_version 976450 (0.0009) [2023-12-26 22:28:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.1, 300 sec: 19383.1). Total num frames: 500047872. Throughput: 0: 9517.6, 1: 9941.8. Samples: 500053784. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:28:56,063][104569] Avg episode reward: [(0, '7580.123'), (1, '9085.210')] [2023-12-26 22:28:56,377][105620] Updated weights for policy 1, policy_version 976591 (0.0009) [2023-12-26 22:28:56,432][105620] Updated weights for policy 1, policy_version 976601 (0.0010) [2023-12-26 22:28:56,479][105620] Updated weights for policy 1, policy_version 976611 (0.0010) [2023-12-26 22:28:56,690][105692] Updated weights for policy 0, policy_version 976460 (0.0008) [2023-12-26 22:28:56,734][105692] Updated weights for policy 0, policy_version 976470 (0.0007) [2023-12-26 22:28:56,782][105692] Updated weights for policy 0, policy_version 976480 (0.0008) [2023-12-26 22:28:57,177][105620] Updated weights for policy 1, policy_version 976621 (0.0008) [2023-12-26 22:28:57,235][105620] Updated weights for policy 1, policy_version 976631 (0.0005) [2023-12-26 22:28:57,297][105620] Updated weights for policy 1, policy_version 976641 (0.0005) [2023-12-26 22:28:57,563][105692] Updated weights for policy 0, policy_version 976490 (0.0008) [2023-12-26 22:28:57,611][105692] Updated weights for policy 0, policy_version 976500 (0.0008) [2023-12-26 22:28:57,662][105692] Updated weights for policy 0, policy_version 976510 (0.0008) [2023-12-26 22:28:57,714][105692] Updated weights for policy 0, policy_version 976520 (0.0007) [2023-12-26 22:28:57,937][105620] Updated weights for policy 1, policy_version 976651 (0.0007) [2023-12-26 22:28:57,999][105620] Updated weights for policy 1, policy_version 976661 (0.0010) [2023-12-26 22:28:58,061][105620] Updated weights for policy 1, policy_version 976671 (0.0010) [2023-12-26 22:28:58,451][105692] Updated weights for policy 0, policy_version 976530 (0.0007) [2023-12-26 22:28:58,522][105692] Updated weights for policy 0, policy_version 976540 (0.0008) [2023-12-26 22:28:58,592][105692] Updated weights for policy 0, policy_version 976550 (0.0006) [2023-12-26 22:28:59,004][105620] Updated weights for policy 1, policy_version 976681 (0.0010) [2023-12-26 22:28:59,066][105620] Updated weights for policy 1, policy_version 976691 (0.0008) [2023-12-26 22:28:59,125][105620] Updated weights for policy 1, policy_version 976701 (0.0007) [2023-12-26 22:28:59,189][105620] Updated weights for policy 1, policy_version 976711 (0.0006) [2023-12-26 22:28:59,356][105692] Updated weights for policy 0, policy_version 976560 (0.0008) [2023-12-26 22:28:59,418][105692] Updated weights for policy 0, policy_version 976570 (0.0007) [2023-12-26 22:28:59,474][105692] Updated weights for policy 0, policy_version 976580 (0.0008) [2023-12-26 22:28:59,908][105620] Updated weights for policy 1, policy_version 976721 (0.0010) [2023-12-26 22:28:59,971][105620] Updated weights for policy 1, policy_version 976731 (0.0009) [2023-12-26 22:29:00,035][105620] Updated weights for policy 1, policy_version 976741 (0.0009) [2023-12-26 22:29:00,190][105692] Updated weights for policy 0, policy_version 976590 (0.0009) [2023-12-26 22:29:00,248][105692] Updated weights for policy 0, policy_version 976600 (0.0009) [2023-12-26 22:29:00,307][105692] Updated weights for policy 0, policy_version 976610 (0.0009) [2023-12-26 22:29:00,724][105620] Updated weights for policy 1, policy_version 976751 (0.0007) [2023-12-26 22:29:00,777][105620] Updated weights for policy 1, policy_version 976761 (0.0006) [2023-12-26 22:29:00,844][105620] Updated weights for policy 1, policy_version 976771 (0.0006) [2023-12-26 22:29:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 500137984. Throughput: 0: 9500.6, 1: 9929.1. Samples: 500109708. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:29:01,062][104569] Avg episode reward: [(0, '8023.360'), (1, '8998.917')] [2023-12-26 22:29:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000976776_250085376.pth... [2023-12-26 22:29:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000975624_249790464.pth [2023-12-26 22:29:01,088][105692] Updated weights for policy 0, policy_version 976620 (0.0009) [2023-12-26 22:29:01,152][105692] Updated weights for policy 0, policy_version 976630 (0.0009) [2023-12-26 22:29:01,208][105692] Updated weights for policy 0, policy_version 976640 (0.0009) [2023-12-26 22:29:01,252][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000976648_250060800.pth... [2023-12-26 22:29:01,257][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000975528_249774080.pth [2023-12-26 22:29:01,542][105620] Updated weights for policy 1, policy_version 976781 (0.0008) [2023-12-26 22:29:01,595][105620] Updated weights for policy 1, policy_version 976791 (0.0009) [2023-12-26 22:29:01,662][105620] Updated weights for policy 1, policy_version 976801 (0.0008) [2023-12-26 22:29:01,932][105692] Updated weights for policy 0, policy_version 976650 (0.0008) [2023-12-26 22:29:01,987][105692] Updated weights for policy 0, policy_version 976660 (0.0005) [2023-12-26 22:29:02,041][105692] Updated weights for policy 0, policy_version 976670 (0.0005) [2023-12-26 22:29:02,087][105692] Updated weights for policy 0, policy_version 976680 (0.0006) [2023-12-26 22:29:02,467][105620] Updated weights for policy 1, policy_version 976811 (0.0006) [2023-12-26 22:29:02,532][105620] Updated weights for policy 1, policy_version 976821 (0.0008) [2023-12-26 22:29:02,598][105620] Updated weights for policy 1, policy_version 976831 (0.0009) [2023-12-26 22:29:02,807][105692] Updated weights for policy 0, policy_version 976690 (0.0009) [2023-12-26 22:29:02,855][105692] Updated weights for policy 0, policy_version 976700 (0.0009) [2023-12-26 22:29:02,907][105692] Updated weights for policy 0, policy_version 976710 (0.0009) [2023-12-26 22:29:03,309][105620] Updated weights for policy 1, policy_version 976841 (0.0009) [2023-12-26 22:29:03,373][105620] Updated weights for policy 1, policy_version 976851 (0.0009) [2023-12-26 22:29:03,433][105620] Updated weights for policy 1, policy_version 976861 (0.0009) [2023-12-26 22:29:03,494][105620] Updated weights for policy 1, policy_version 976871 (0.0009) [2023-12-26 22:29:03,639][105692] Updated weights for policy 0, policy_version 976720 (0.0006) [2023-12-26 22:29:03,693][105692] Updated weights for policy 0, policy_version 976730 (0.0005) [2023-12-26 22:29:03,743][105692] Updated weights for policy 0, policy_version 976740 (0.0005) [2023-12-26 22:29:04,167][105620] Updated weights for policy 1, policy_version 976881 (0.0009) [2023-12-26 22:29:04,242][105620] Updated weights for policy 1, policy_version 976891 (0.0008) [2023-12-26 22:29:04,301][105620] Updated weights for policy 1, policy_version 976901 (0.0009) [2023-12-26 22:29:04,480][105692] Updated weights for policy 0, policy_version 976750 (0.0008) [2023-12-26 22:29:04,552][105692] Updated weights for policy 0, policy_version 976760 (0.0010) [2023-12-26 22:29:04,619][105692] Updated weights for policy 0, policy_version 976770 (0.0010) [2023-12-26 22:29:04,982][105620] Updated weights for policy 1, policy_version 976911 (0.0010) [2023-12-26 22:29:05,042][105620] Updated weights for policy 1, policy_version 976922 (0.0011) [2023-12-26 22:29:05,101][105620] Updated weights for policy 1, policy_version 976932 (0.0008) [2023-12-26 22:29:05,312][105692] Updated weights for policy 0, policy_version 976780 (0.0010) [2023-12-26 22:29:05,378][105692] Updated weights for policy 0, policy_version 976790 (0.0009) [2023-12-26 22:29:05,443][105692] Updated weights for policy 0, policy_version 976800 (0.0009) [2023-12-26 22:29:05,883][105620] Updated weights for policy 1, policy_version 976943 (0.0010) [2023-12-26 22:29:05,939][105620] Updated weights for policy 1, policy_version 976953 (0.0009) [2023-12-26 22:29:05,990][105620] Updated weights for policy 1, policy_version 976963 (0.0008) [2023-12-26 22:29:06,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.7, 300 sec: 19327.6). Total num frames: 500236288. Throughput: 0: 9450.9, 1: 9927.5. Samples: 500224252. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:29:06,063][104569] Avg episode reward: [(0, '8201.322'), (1, '9087.615')] [2023-12-26 22:29:06,119][105692] Updated weights for policy 0, policy_version 976810 (0.0009) [2023-12-26 22:29:06,186][105692] Updated weights for policy 0, policy_version 976820 (0.0009) [2023-12-26 22:29:06,248][105692] Updated weights for policy 0, policy_version 976830 (0.0008) [2023-12-26 22:29:06,311][105692] Updated weights for policy 0, policy_version 976840 (0.0007) [2023-12-26 22:29:06,903][105620] Updated weights for policy 1, policy_version 976973 (0.0009) [2023-12-26 22:29:06,957][105620] Updated weights for policy 1, policy_version 976983 (0.0006) [2023-12-26 22:29:06,959][105692] Updated weights for policy 0, policy_version 976850 (0.0008) [2023-12-26 22:29:07,016][105692] Updated weights for policy 0, policy_version 976860 (0.0007) [2023-12-26 22:29:07,018][105620] Updated weights for policy 1, policy_version 976993 (0.0006) [2023-12-26 22:29:07,067][105692] Updated weights for policy 0, policy_version 976870 (0.0007) [2023-12-26 22:29:07,750][105692] Updated weights for policy 0, policy_version 976880 (0.0009) [2023-12-26 22:29:07,798][105692] Updated weights for policy 0, policy_version 976890 (0.0007) [2023-12-26 22:29:07,823][105620] Updated weights for policy 1, policy_version 977003 (0.0007) [2023-12-26 22:29:07,846][105692] Updated weights for policy 0, policy_version 976900 (0.0006) [2023-12-26 22:29:07,869][105620] Updated weights for policy 1, policy_version 977013 (0.0007) [2023-12-26 22:29:07,930][105620] Updated weights for policy 1, policy_version 977023 (0.0009) [2023-12-26 22:29:08,550][105692] Updated weights for policy 0, policy_version 976910 (0.0009) [2023-12-26 22:29:08,609][105692] Updated weights for policy 0, policy_version 976920 (0.0011) [2023-12-26 22:29:08,670][105692] Updated weights for policy 0, policy_version 976930 (0.0010) [2023-12-26 22:29:08,728][105620] Updated weights for policy 1, policy_version 977033 (0.0009) [2023-12-26 22:29:08,785][105620] Updated weights for policy 1, policy_version 977043 (0.0008) [2023-12-26 22:29:08,845][105620] Updated weights for policy 1, policy_version 977053 (0.0009) [2023-12-26 22:29:08,910][105620] Updated weights for policy 1, policy_version 977063 (0.0008) [2023-12-26 22:29:09,434][105692] Updated weights for policy 0, policy_version 976940 (0.0009) [2023-12-26 22:29:09,496][105692] Updated weights for policy 0, policy_version 976950 (0.0009) [2023-12-26 22:29:09,552][105692] Updated weights for policy 0, policy_version 976960 (0.0009) [2023-12-26 22:29:09,739][105620] Updated weights for policy 1, policy_version 977073 (0.0009) [2023-12-26 22:29:09,796][105620] Updated weights for policy 1, policy_version 977083 (0.0008) [2023-12-26 22:29:09,861][105620] Updated weights for policy 1, policy_version 977093 (0.0007) [2023-12-26 22:29:10,268][105692] Updated weights for policy 0, policy_version 976970 (0.0009) [2023-12-26 22:29:10,324][105692] Updated weights for policy 0, policy_version 976980 (0.0011) [2023-12-26 22:29:10,378][105692] Updated weights for policy 0, policy_version 976990 (0.0009) [2023-12-26 22:29:10,451][105692] Updated weights for policy 0, policy_version 977000 (0.0008) [2023-12-26 22:29:10,587][105620] Updated weights for policy 1, policy_version 977103 (0.0008) [2023-12-26 22:29:10,648][105620] Updated weights for policy 1, policy_version 977113 (0.0006) [2023-12-26 22:29:10,710][105620] Updated weights for policy 1, policy_version 977123 (0.0009) [2023-12-26 22:29:11,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.2, 300 sec: 19299.8). Total num frames: 500326400. Throughput: 0: 9483.6, 1: 9813.2. Samples: 500336656. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:29:11,063][104569] Avg episode reward: [(0, '8113.915'), (1, '9268.852')] [2023-12-26 22:29:11,212][105692] Updated weights for policy 0, policy_version 977010 (0.0008) [2023-12-26 22:29:11,276][105692] Updated weights for policy 0, policy_version 977020 (0.0009) [2023-12-26 22:29:11,346][105692] Updated weights for policy 0, policy_version 977030 (0.0010) [2023-12-26 22:29:11,498][105620] Updated weights for policy 1, policy_version 977133 (0.0010) [2023-12-26 22:29:11,555][105620] Updated weights for policy 1, policy_version 977143 (0.0009) [2023-12-26 22:29:11,620][105620] Updated weights for policy 1, policy_version 977153 (0.0009) [2023-12-26 22:29:12,093][105692] Updated weights for policy 0, policy_version 977040 (0.0009) [2023-12-26 22:29:12,155][105692] Updated weights for policy 0, policy_version 977050 (0.0008) [2023-12-26 22:29:12,222][105692] Updated weights for policy 0, policy_version 977060 (0.0009) [2023-12-26 22:29:12,408][105620] Updated weights for policy 1, policy_version 977163 (0.0010) [2023-12-26 22:29:12,460][105620] Updated weights for policy 1, policy_version 977173 (0.0008) [2023-12-26 22:29:12,512][105620] Updated weights for policy 1, policy_version 977183 (0.0008) [2023-12-26 22:29:13,054][105692] Updated weights for policy 0, policy_version 977070 (0.0009) [2023-12-26 22:29:13,102][105692] Updated weights for policy 0, policy_version 977080 (0.0008) [2023-12-26 22:29:13,149][105692] Updated weights for policy 0, policy_version 977090 (0.0008) [2023-12-26 22:29:13,174][105620] Updated weights for policy 1, policy_version 977193 (0.0008) [2023-12-26 22:29:13,238][105620] Updated weights for policy 1, policy_version 977203 (0.0010) [2023-12-26 22:29:13,301][105620] Updated weights for policy 1, policy_version 977213 (0.0011) [2023-12-26 22:29:13,349][105620] Updated weights for policy 1, policy_version 977223 (0.0010) [2023-12-26 22:29:13,976][105692] Updated weights for policy 0, policy_version 977100 (0.0008) [2023-12-26 22:29:14,010][105620] Updated weights for policy 1, policy_version 977233 (0.0006) [2023-12-26 22:29:14,036][105692] Updated weights for policy 0, policy_version 977110 (0.0009) [2023-12-26 22:29:14,064][105620] Updated weights for policy 1, policy_version 977243 (0.0009) [2023-12-26 22:29:14,091][105692] Updated weights for policy 0, policy_version 977120 (0.0006) [2023-12-26 22:29:14,121][105620] Updated weights for policy 1, policy_version 977253 (0.0011) [2023-12-26 22:29:14,780][105692] Updated weights for policy 0, policy_version 977130 (0.0007) [2023-12-26 22:29:14,837][105692] Updated weights for policy 0, policy_version 977140 (0.0008) [2023-12-26 22:29:14,855][105620] Updated weights for policy 1, policy_version 977263 (0.0009) [2023-12-26 22:29:14,897][105692] Updated weights for policy 0, policy_version 977150 (0.0007) [2023-12-26 22:29:14,912][105620] Updated weights for policy 1, policy_version 977273 (0.0011) [2023-12-26 22:29:14,954][105692] Updated weights for policy 0, policy_version 977160 (0.0008) [2023-12-26 22:29:14,958][105620] Updated weights for policy 1, policy_version 977283 (0.0011) [2023-12-26 22:29:15,655][105620] Updated weights for policy 1, policy_version 977293 (0.0011) [2023-12-26 22:29:15,677][105692] Updated weights for policy 0, policy_version 977170 (0.0007) [2023-12-26 22:29:15,700][105620] Updated weights for policy 1, policy_version 977303 (0.0010) [2023-12-26 22:29:15,730][105692] Updated weights for policy 0, policy_version 977180 (0.0006) [2023-12-26 22:29:15,745][105620] Updated weights for policy 1, policy_version 977313 (0.0010) [2023-12-26 22:29:15,787][105692] Updated weights for policy 0, policy_version 977190 (0.0006) [2023-12-26 22:29:16,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19387.6, 300 sec: 19327.6). Total num frames: 500424704. Throughput: 0: 9453.5, 1: 9701.4. Samples: 500392292. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:29:16,063][104569] Avg episode reward: [(0, '8551.513'), (1, '9268.889')] [2023-12-26 22:29:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000977192_250200064.pth... [2023-12-26 22:29:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000977320_250224640.pth... [2023-12-26 22:29:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000976072_249913344.pth [2023-12-26 22:29:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000976200_249937920.pth [2023-12-26 22:29:16,503][105620] Updated weights for policy 1, policy_version 977323 (0.0010) [2023-12-26 22:29:16,560][105692] Updated weights for policy 0, policy_version 977200 (0.0008) [2023-12-26 22:29:16,567][105620] Updated weights for policy 1, policy_version 977333 (0.0010) [2023-12-26 22:29:16,608][105692] Updated weights for policy 0, policy_version 977210 (0.0007) [2023-12-26 22:29:16,633][105620] Updated weights for policy 1, policy_version 977343 (0.0011) [2023-12-26 22:29:16,657][105692] Updated weights for policy 0, policy_version 977220 (0.0007) [2023-12-26 22:29:17,295][105620] Updated weights for policy 1, policy_version 977353 (0.0010) [2023-12-26 22:29:17,315][105692] Updated weights for policy 0, policy_version 977230 (0.0008) [2023-12-26 22:29:17,356][105620] Updated weights for policy 1, policy_version 977363 (0.0006) [2023-12-26 22:29:17,375][105692] Updated weights for policy 0, policy_version 977240 (0.0006) [2023-12-26 22:29:17,405][105620] Updated weights for policy 1, policy_version 977373 (0.0005) [2023-12-26 22:29:17,433][105692] Updated weights for policy 0, policy_version 977250 (0.0006) [2023-12-26 22:29:17,454][105620] Updated weights for policy 1, policy_version 977383 (0.0008) [2023-12-26 22:29:18,112][105620] Updated weights for policy 1, policy_version 977393 (0.0010) [2023-12-26 22:29:18,146][105692] Updated weights for policy 0, policy_version 977260 (0.0008) [2023-12-26 22:29:18,161][105620] Updated weights for policy 1, policy_version 977403 (0.0010) [2023-12-26 22:29:18,205][105692] Updated weights for policy 0, policy_version 977270 (0.0008) [2023-12-26 22:29:18,215][105620] Updated weights for policy 1, policy_version 977413 (0.0008) [2023-12-26 22:29:18,273][105692] Updated weights for policy 0, policy_version 977280 (0.0011) [2023-12-26 22:29:18,933][105692] Updated weights for policy 0, policy_version 977290 (0.0008) [2023-12-26 22:29:18,987][105620] Updated weights for policy 1, policy_version 977423 (0.0010) [2023-12-26 22:29:18,994][105692] Updated weights for policy 0, policy_version 977300 (0.0006) [2023-12-26 22:29:19,047][105692] Updated weights for policy 0, policy_version 977310 (0.0007) [2023-12-26 22:29:19,050][105620] Updated weights for policy 1, policy_version 977433 (0.0011) [2023-12-26 22:29:19,113][105620] Updated weights for policy 1, policy_version 977443 (0.0011) [2023-12-26 22:29:19,115][105692] Updated weights for policy 0, policy_version 977320 (0.0006) [2023-12-26 22:29:19,834][105692] Updated weights for policy 0, policy_version 977330 (0.0008) [2023-12-26 22:29:19,888][105692] Updated weights for policy 0, policy_version 977340 (0.0008) [2023-12-26 22:29:19,909][105620] Updated weights for policy 1, policy_version 977453 (0.0010) [2023-12-26 22:29:19,948][105692] Updated weights for policy 0, policy_version 977350 (0.0006) [2023-12-26 22:29:19,983][105620] Updated weights for policy 1, policy_version 977463 (0.0008) [2023-12-26 22:29:20,048][105620] Updated weights for policy 1, policy_version 977473 (0.0009) [2023-12-26 22:29:20,690][105692] Updated weights for policy 0, policy_version 977360 (0.0008) [2023-12-26 22:29:20,755][105692] Updated weights for policy 0, policy_version 977370 (0.0008) [2023-12-26 22:29:20,798][105620] Updated weights for policy 1, policy_version 977483 (0.0009) [2023-12-26 22:29:20,820][105692] Updated weights for policy 0, policy_version 977380 (0.0009) [2023-12-26 22:29:20,857][105620] Updated weights for policy 1, policy_version 977493 (0.0007) [2023-12-26 22:29:20,924][105620] Updated weights for policy 1, policy_version 977503 (0.0009) [2023-12-26 22:29:21,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19299.8). Total num frames: 500523008. Throughput: 0: 9492.9, 1: 9600.3. Samples: 500509260. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:29:21,062][104569] Avg episode reward: [(0, '9087.760'), (1, '9356.657')] [2023-12-26 22:29:21,573][105692] Updated weights for policy 0, policy_version 977390 (0.0006) [2023-12-26 22:29:21,641][105692] Updated weights for policy 0, policy_version 977400 (0.0006) [2023-12-26 22:29:21,709][105692] Updated weights for policy 0, policy_version 977410 (0.0008) [2023-12-26 22:29:21,739][105620] Updated weights for policy 1, policy_version 977513 (0.0008) [2023-12-26 22:29:21,747][105585] KL-divergence is very high: 306.8556 [2023-12-26 22:29:21,806][105620] Updated weights for policy 1, policy_version 977523 (0.0008) [2023-12-26 22:29:21,875][105620] Updated weights for policy 1, policy_version 977533 (0.0008) [2023-12-26 22:29:21,940][105620] Updated weights for policy 1, policy_version 977543 (0.0008) [2023-12-26 22:29:22,445][105692] Updated weights for policy 0, policy_version 977420 (0.0008) [2023-12-26 22:29:22,501][105692] Updated weights for policy 0, policy_version 977430 (0.0009) [2023-12-26 22:29:22,550][105692] Updated weights for policy 0, policy_version 977440 (0.0008) [2023-12-26 22:29:22,660][105620] Updated weights for policy 1, policy_version 977553 (0.0009) [2023-12-26 22:29:22,730][105620] Updated weights for policy 1, policy_version 977563 (0.0009) [2023-12-26 22:29:22,784][105620] Updated weights for policy 1, policy_version 977573 (0.0009) [2023-12-26 22:29:23,317][105692] Updated weights for policy 0, policy_version 977450 (0.0008) [2023-12-26 22:29:23,380][105692] Updated weights for policy 0, policy_version 977460 (0.0010) [2023-12-26 22:29:23,422][105692] Updated weights for policy 0, policy_version 977470 (0.0008) [2023-12-26 22:29:23,484][105692] Updated weights for policy 0, policy_version 977480 (0.0010) [2023-12-26 22:29:23,559][105620] Updated weights for policy 1, policy_version 977583 (0.0008) [2023-12-26 22:29:23,615][105620] Updated weights for policy 1, policy_version 977593 (0.0009) [2023-12-26 22:29:23,675][105620] Updated weights for policy 1, policy_version 977603 (0.0009) [2023-12-26 22:29:24,089][105692] Updated weights for policy 0, policy_version 977490 (0.0005) [2023-12-26 22:29:24,144][105692] Updated weights for policy 0, policy_version 977500 (0.0006) [2023-12-26 22:29:24,196][105692] Updated weights for policy 0, policy_version 977510 (0.0010) [2023-12-26 22:29:24,546][105620] Updated weights for policy 1, policy_version 977613 (0.0009) [2023-12-26 22:29:24,597][105620] Updated weights for policy 1, policy_version 977623 (0.0010) [2023-12-26 22:29:24,649][105620] Updated weights for policy 1, policy_version 977633 (0.0009) [2023-12-26 22:29:24,752][105692] Updated weights for policy 0, policy_version 977520 (0.0006) [2023-12-26 22:29:24,812][105692] Updated weights for policy 0, policy_version 977530 (0.0005) [2023-12-26 22:29:24,870][105692] Updated weights for policy 0, policy_version 977540 (0.0005) [2023-12-26 22:29:25,335][105620] Updated weights for policy 1, policy_version 977643 (0.0009) [2023-12-26 22:29:25,400][105620] Updated weights for policy 1, policy_version 977653 (0.0011) [2023-12-26 22:29:25,463][105620] Updated weights for policy 1, policy_version 977663 (0.0010) [2023-12-26 22:29:25,588][105692] Updated weights for policy 0, policy_version 977550 (0.0008) [2023-12-26 22:29:25,645][105692] Updated weights for policy 0, policy_version 977560 (0.0005) [2023-12-26 22:29:25,699][105692] Updated weights for policy 0, policy_version 977570 (0.0010) [2023-12-26 22:29:26,062][104569] Fps is (10 sec: 18842.3, 60 sec: 19251.3, 300 sec: 19272.0). Total num frames: 500613120. Throughput: 0: 9558.2, 1: 9448.1. Samples: 500622924. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:29:26,062][104569] Avg episode reward: [(0, '8818.155'), (1, '9089.557')] [2023-12-26 22:29:26,189][105620] Updated weights for policy 1, policy_version 977673 (0.0010) [2023-12-26 22:29:26,252][105620] Updated weights for policy 1, policy_version 977683 (0.0010) [2023-12-26 22:29:26,312][105620] Updated weights for policy 1, policy_version 977693 (0.0010) [2023-12-26 22:29:26,375][105620] Updated weights for policy 1, policy_version 977703 (0.0010) [2023-12-26 22:29:26,450][105692] Updated weights for policy 0, policy_version 977580 (0.0011) [2023-12-26 22:29:26,515][105692] Updated weights for policy 0, policy_version 977590 (0.0009) [2023-12-26 22:29:26,576][105692] Updated weights for policy 0, policy_version 977600 (0.0008) [2023-12-26 22:29:27,113][105620] Updated weights for policy 1, policy_version 977713 (0.0010) [2023-12-26 22:29:27,160][105620] Updated weights for policy 1, policy_version 977723 (0.0010) [2023-12-26 22:29:27,204][105620] Updated weights for policy 1, policy_version 977733 (0.0010) [2023-12-26 22:29:27,231][105692] Updated weights for policy 0, policy_version 977610 (0.0010) [2023-12-26 22:29:27,279][105692] Updated weights for policy 0, policy_version 977620 (0.0010) [2023-12-26 22:29:27,342][105692] Updated weights for policy 0, policy_version 977630 (0.0006) [2023-12-26 22:29:27,392][105692] Updated weights for policy 0, policy_version 977640 (0.0005) [2023-12-26 22:29:27,938][105692] Updated weights for policy 0, policy_version 977650 (0.0011) [2023-12-26 22:29:27,962][105620] Updated weights for policy 1, policy_version 977743 (0.0010) [2023-12-26 22:29:27,990][105692] Updated weights for policy 0, policy_version 977660 (0.0011) [2023-12-26 22:29:28,017][105620] Updated weights for policy 1, policy_version 977753 (0.0010) [2023-12-26 22:29:28,039][105692] Updated weights for policy 0, policy_version 977670 (0.0010) [2023-12-26 22:29:28,071][105620] Updated weights for policy 1, policy_version 977763 (0.0010) [2023-12-26 22:29:28,775][105692] Updated weights for policy 0, policy_version 977680 (0.0010) [2023-12-26 22:29:28,823][105692] Updated weights for policy 0, policy_version 977690 (0.0010) [2023-12-26 22:29:28,835][105620] Updated weights for policy 1, policy_version 977773 (0.0010) [2023-12-26 22:29:28,875][105692] Updated weights for policy 0, policy_version 977700 (0.0010) [2023-12-26 22:29:28,894][105620] Updated weights for policy 1, policy_version 977783 (0.0010) [2023-12-26 22:29:28,954][105620] Updated weights for policy 1, policy_version 977793 (0.0010) [2023-12-26 22:29:29,640][105692] Updated weights for policy 0, policy_version 977710 (0.0009) [2023-12-26 22:29:29,699][105692] Updated weights for policy 0, policy_version 977720 (0.0010) [2023-12-26 22:29:29,726][105620] Updated weights for policy 1, policy_version 977803 (0.0009) [2023-12-26 22:29:29,754][105692] Updated weights for policy 0, policy_version 977730 (0.0011) [2023-12-26 22:29:29,781][105620] Updated weights for policy 1, policy_version 977813 (0.0006) [2023-12-26 22:29:29,836][105620] Updated weights for policy 1, policy_version 977823 (0.0008) [2023-12-26 22:29:30,428][105692] Updated weights for policy 0, policy_version 977740 (0.0009) [2023-12-26 22:29:30,496][105692] Updated weights for policy 0, policy_version 977750 (0.0008) [2023-12-26 22:29:30,571][105692] Updated weights for policy 0, policy_version 977760 (0.0006) [2023-12-26 22:29:30,646][105620] Updated weights for policy 1, policy_version 977833 (0.0006) [2023-12-26 22:29:30,713][105620] Updated weights for policy 1, policy_version 977843 (0.0009) [2023-12-26 22:29:30,767][105620] Updated weights for policy 1, policy_version 977853 (0.0009) [2023-12-26 22:29:30,824][105620] Updated weights for policy 1, policy_version 977863 (0.0009) [2023-12-26 22:29:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.7, 300 sec: 19272.0). Total num frames: 500711424. Throughput: 0: 9593.5, 1: 9463.9. Samples: 500682216. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:29:31,063][104569] Avg episode reward: [(0, '8034.299'), (1, '8994.933')] [2023-12-26 22:29:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000977768_250347520.pth... [2023-12-26 22:29:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000977864_250363904.pth... [2023-12-26 22:29:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000976648_250060800.pth [2023-12-26 22:29:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000976776_250085376.pth [2023-12-26 22:29:31,131][105692] Updated weights for policy 0, policy_version 977770 (0.0008) [2023-12-26 22:29:31,188][105692] Updated weights for policy 0, policy_version 977780 (0.0009) [2023-12-26 22:29:31,255][105692] Updated weights for policy 0, policy_version 977790 (0.0008) [2023-12-26 22:29:31,314][105692] Updated weights for policy 0, policy_version 977800 (0.0008) [2023-12-26 22:29:31,673][105620] Updated weights for policy 1, policy_version 977873 (0.0008) [2023-12-26 22:29:31,748][105620] Updated weights for policy 1, policy_version 977883 (0.0008) [2023-12-26 22:29:31,805][105620] Updated weights for policy 1, policy_version 977893 (0.0008) [2023-12-26 22:29:32,060][105692] Updated weights for policy 0, policy_version 977810 (0.0010) [2023-12-26 22:29:32,108][105692] Updated weights for policy 0, policy_version 977820 (0.0010) [2023-12-26 22:29:32,156][105692] Updated weights for policy 0, policy_version 977830 (0.0010) [2023-12-26 22:29:32,594][105620] Updated weights for policy 1, policy_version 977903 (0.0009) [2023-12-26 22:29:32,652][105620] Updated weights for policy 1, policy_version 977913 (0.0008) [2023-12-26 22:29:32,700][105620] Updated weights for policy 1, policy_version 977923 (0.0007) [2023-12-26 22:29:32,851][105692] Updated weights for policy 0, policy_version 977840 (0.0007) [2023-12-26 22:29:32,921][105692] Updated weights for policy 0, policy_version 977850 (0.0009) [2023-12-26 22:29:32,978][105692] Updated weights for policy 0, policy_version 977860 (0.0009) [2023-12-26 22:29:33,499][105620] Updated weights for policy 1, policy_version 977933 (0.0007) [2023-12-26 22:29:33,559][105620] Updated weights for policy 1, policy_version 977943 (0.0006) [2023-12-26 22:29:33,564][105692] Updated weights for policy 0, policy_version 977870 (0.0008) [2023-12-26 22:29:33,614][105620] Updated weights for policy 1, policy_version 977953 (0.0005) [2023-12-26 22:29:33,624][105692] Updated weights for policy 0, policy_version 977880 (0.0010) [2023-12-26 22:29:33,686][105692] Updated weights for policy 0, policy_version 977890 (0.0010) [2023-12-26 22:29:34,365][105620] Updated weights for policy 1, policy_version 977963 (0.0006) [2023-12-26 22:29:34,387][105692] Updated weights for policy 0, policy_version 977900 (0.0010) [2023-12-26 22:29:34,433][105620] Updated weights for policy 1, policy_version 977973 (0.0007) [2023-12-26 22:29:34,445][105692] Updated weights for policy 0, policy_version 977910 (0.0008) [2023-12-26 22:29:34,488][105620] Updated weights for policy 1, policy_version 977983 (0.0007) [2023-12-26 22:29:34,500][105692] Updated weights for policy 0, policy_version 977920 (0.0008) [2023-12-26 22:29:35,208][105692] Updated weights for policy 0, policy_version 977930 (0.0007) [2023-12-26 22:29:35,274][105692] Updated weights for policy 0, policy_version 977940 (0.0007) [2023-12-26 22:29:35,277][105620] Updated weights for policy 1, policy_version 977993 (0.0007) [2023-12-26 22:29:35,337][105620] Updated weights for policy 1, policy_version 978003 (0.0007) [2023-12-26 22:29:35,338][105692] Updated weights for policy 0, policy_version 977950 (0.0011) [2023-12-26 22:29:35,395][105620] Updated weights for policy 1, policy_version 978013 (0.0007) [2023-12-26 22:29:35,408][105692] Updated weights for policy 0, policy_version 977960 (0.0011) [2023-12-26 22:29:35,456][105620] Updated weights for policy 1, policy_version 978023 (0.0008) [2023-12-26 22:29:36,048][105692] Updated weights for policy 0, policy_version 977970 (0.0009) [2023-12-26 22:29:36,062][104569] Fps is (10 sec: 18841.4, 60 sec: 18978.1, 300 sec: 19272.0). Total num frames: 500801536. Throughput: 0: 9710.0, 1: 9332.9. Samples: 500795712. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:29:36,063][104569] Avg episode reward: [(0, '7660.339'), (1, '9079.899')] [2023-12-26 22:29:36,104][105692] Updated weights for policy 0, policy_version 977980 (0.0010) [2023-12-26 22:29:36,163][105692] Updated weights for policy 0, policy_version 977990 (0.0010) [2023-12-26 22:29:36,268][105620] Updated weights for policy 1, policy_version 978033 (0.0008) [2023-12-26 22:29:36,334][105620] Updated weights for policy 1, policy_version 978043 (0.0009) [2023-12-26 22:29:36,396][105620] Updated weights for policy 1, policy_version 978053 (0.0010) [2023-12-26 22:29:36,838][105692] Updated weights for policy 0, policy_version 978000 (0.0011) [2023-12-26 22:29:36,883][105692] Updated weights for policy 0, policy_version 978010 (0.0010) [2023-12-26 22:29:36,938][105692] Updated weights for policy 0, policy_version 978020 (0.0010) [2023-12-26 22:29:37,213][105620] Updated weights for policy 1, policy_version 978063 (0.0009) [2023-12-26 22:29:37,269][105620] Updated weights for policy 1, policy_version 978073 (0.0010) [2023-12-26 22:29:37,321][105620] Updated weights for policy 1, policy_version 978083 (0.0009) [2023-12-26 22:29:37,604][105692] Updated weights for policy 0, policy_version 978030 (0.0006) [2023-12-26 22:29:37,667][105692] Updated weights for policy 0, policy_version 978040 (0.0005) [2023-12-26 22:29:37,728][105692] Updated weights for policy 0, policy_version 978050 (0.0006) [2023-12-26 22:29:38,096][105620] Updated weights for policy 1, policy_version 978093 (0.0008) [2023-12-26 22:29:38,154][105620] Updated weights for policy 1, policy_version 978103 (0.0006) [2023-12-26 22:29:38,205][105620] Updated weights for policy 1, policy_version 978113 (0.0007) [2023-12-26 22:29:38,498][105692] Updated weights for policy 0, policy_version 978060 (0.0009) [2023-12-26 22:29:38,549][105692] Updated weights for policy 0, policy_version 978070 (0.0009) [2023-12-26 22:29:38,622][105692] Updated weights for policy 0, policy_version 978080 (0.0009) [2023-12-26 22:29:38,827][105620] Updated weights for policy 1, policy_version 978123 (0.0006) [2023-12-26 22:29:38,887][105620] Updated weights for policy 1, policy_version 978133 (0.0008) [2023-12-26 22:29:38,946][105620] Updated weights for policy 1, policy_version 978143 (0.0009) [2023-12-26 22:29:39,412][105692] Updated weights for policy 0, policy_version 978090 (0.0008) [2023-12-26 22:29:39,468][105692] Updated weights for policy 0, policy_version 978100 (0.0009) [2023-12-26 22:29:39,525][105692] Updated weights for policy 0, policy_version 978110 (0.0007) [2023-12-26 22:29:39,588][105692] Updated weights for policy 0, policy_version 978120 (0.0008) [2023-12-26 22:29:39,684][105620] Updated weights for policy 1, policy_version 978153 (0.0009) [2023-12-26 22:29:39,756][105620] Updated weights for policy 1, policy_version 978163 (0.0007) [2023-12-26 22:29:39,830][105620] Updated weights for policy 1, policy_version 978173 (0.0009) [2023-12-26 22:29:39,899][105620] Updated weights for policy 1, policy_version 978183 (0.0008) [2023-12-26 22:29:40,281][105692] Updated weights for policy 0, policy_version 978130 (0.0009) [2023-12-26 22:29:40,339][105692] Updated weights for policy 0, policy_version 978140 (0.0009) [2023-12-26 22:29:40,395][105692] Updated weights for policy 0, policy_version 978150 (0.0009) [2023-12-26 22:29:40,620][105620] Updated weights for policy 1, policy_version 978193 (0.0008) [2023-12-26 22:29:40,686][105620] Updated weights for policy 1, policy_version 978203 (0.0007) [2023-12-26 22:29:40,733][105620] Updated weights for policy 1, policy_version 978213 (0.0009) [2023-12-26 22:29:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19114.7, 300 sec: 19272.0). Total num frames: 500899840. Throughput: 0: 9754.4, 1: 9262.4. Samples: 500909532. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-12-26 22:29:41,062][104569] Avg episode reward: [(0, '8062.357'), (1, '9173.797')] [2023-12-26 22:29:41,190][105692] Updated weights for policy 0, policy_version 978160 (0.0010) [2023-12-26 22:29:41,252][105692] Updated weights for policy 0, policy_version 978170 (0.0010) [2023-12-26 22:29:41,316][105692] Updated weights for policy 0, policy_version 978180 (0.0009) [2023-12-26 22:29:41,560][105620] Updated weights for policy 1, policy_version 978223 (0.0007) [2023-12-26 22:29:41,622][105620] Updated weights for policy 1, policy_version 978233 (0.0008) [2023-12-26 22:29:41,701][105620] Updated weights for policy 1, policy_version 978243 (0.0007) [2023-12-26 22:29:42,106][105692] Updated weights for policy 0, policy_version 978190 (0.0009) [2023-12-26 22:29:42,171][105692] Updated weights for policy 0, policy_version 978200 (0.0008) [2023-12-26 22:29:42,237][105692] Updated weights for policy 0, policy_version 978210 (0.0009) [2023-12-26 22:29:42,442][105620] Updated weights for policy 1, policy_version 978253 (0.0008) [2023-12-26 22:29:42,502][105620] Updated weights for policy 1, policy_version 978263 (0.0010) [2023-12-26 22:29:42,555][105620] Updated weights for policy 1, policy_version 978273 (0.0009) [2023-12-26 22:29:42,998][105692] Updated weights for policy 0, policy_version 978220 (0.0007) [2023-12-26 22:29:43,052][105692] Updated weights for policy 0, policy_version 978230 (0.0009) [2023-12-26 22:29:43,115][105692] Updated weights for policy 0, policy_version 978240 (0.0006) [2023-12-26 22:29:43,332][105620] Updated weights for policy 1, policy_version 978283 (0.0008) [2023-12-26 22:29:43,389][105620] Updated weights for policy 1, policy_version 978293 (0.0006) [2023-12-26 22:29:43,454][105620] Updated weights for policy 1, policy_version 978303 (0.0011) [2023-12-26 22:29:43,792][105692] Updated weights for policy 0, policy_version 978250 (0.0006) [2023-12-26 22:29:43,847][105692] Updated weights for policy 0, policy_version 978260 (0.0008) [2023-12-26 22:29:43,902][105692] Updated weights for policy 0, policy_version 978270 (0.0008) [2023-12-26 22:29:43,960][105692] Updated weights for policy 0, policy_version 978280 (0.0009) [2023-12-26 22:29:44,074][105620] Updated weights for policy 1, policy_version 978313 (0.0010) [2023-12-26 22:29:44,123][105620] Updated weights for policy 1, policy_version 978323 (0.0011) [2023-12-26 22:29:44,172][105620] Updated weights for policy 1, policy_version 978333 (0.0010) [2023-12-26 22:29:44,221][105620] Updated weights for policy 1, policy_version 978343 (0.0010) [2023-12-26 22:29:44,655][105692] Updated weights for policy 0, policy_version 978290 (0.0010) [2023-12-26 22:29:44,713][105692] Updated weights for policy 0, policy_version 978300 (0.0010) [2023-12-26 22:29:44,766][105692] Updated weights for policy 0, policy_version 978310 (0.0011) [2023-12-26 22:29:44,960][105620] Updated weights for policy 1, policy_version 978353 (0.0009) [2023-12-26 22:29:45,030][105620] Updated weights for policy 1, policy_version 978363 (0.0008) [2023-12-26 22:29:45,094][105620] Updated weights for policy 1, policy_version 978373 (0.0011) [2023-12-26 22:29:45,551][105692] Updated weights for policy 0, policy_version 978320 (0.0009) [2023-12-26 22:29:45,608][105692] Updated weights for policy 0, policy_version 978330 (0.0008) [2023-12-26 22:29:45,668][105692] Updated weights for policy 0, policy_version 978340 (0.0008) [2023-12-26 22:29:45,719][105620] Updated weights for policy 1, policy_version 978383 (0.0009) [2023-12-26 22:29:45,767][105620] Updated weights for policy 1, policy_version 978393 (0.0010) [2023-12-26 22:29:45,819][105620] Updated weights for policy 1, policy_version 978403 (0.0009) [2023-12-26 22:29:46,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19114.7, 300 sec: 19272.0). Total num frames: 500998144. Throughput: 0: 9742.1, 1: 9274.1. Samples: 500965440. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:29:46,062][104569] Avg episode reward: [(0, '8180.338'), (1, '9265.940')] [2023-12-26 22:29:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000978344_250494976.pth... [2023-12-26 22:29:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000978408_250503168.pth... [2023-12-26 22:29:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000977192_250200064.pth [2023-12-26 22:29:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000977320_250224640.pth [2023-12-26 22:29:46,433][105620] Updated weights for policy 1, policy_version 978413 (0.0008) [2023-12-26 22:29:46,491][105620] Updated weights for policy 1, policy_version 978423 (0.0006) [2023-12-26 22:29:46,494][105692] Updated weights for policy 0, policy_version 978350 (0.0007) [2023-12-26 22:29:46,545][105620] Updated weights for policy 1, policy_version 978433 (0.0005) [2023-12-26 22:29:46,551][105692] Updated weights for policy 0, policy_version 978360 (0.0009) [2023-12-26 22:29:46,608][105692] Updated weights for policy 0, policy_version 978370 (0.0009) [2023-12-26 22:29:47,108][105620] Updated weights for policy 1, policy_version 978443 (0.0005) [2023-12-26 22:29:47,167][105620] Updated weights for policy 1, policy_version 978453 (0.0006) [2023-12-26 22:29:47,214][105620] Updated weights for policy 1, policy_version 978463 (0.0005) [2023-12-26 22:29:47,486][105692] Updated weights for policy 0, policy_version 978380 (0.0009) [2023-12-26 22:29:47,538][105692] Updated weights for policy 0, policy_version 978390 (0.0009) [2023-12-26 22:29:47,590][105692] Updated weights for policy 0, policy_version 978400 (0.0007) [2023-12-26 22:29:47,824][105620] Updated weights for policy 1, policy_version 978473 (0.0006) [2023-12-26 22:29:47,876][105620] Updated weights for policy 1, policy_version 978483 (0.0007) [2023-12-26 22:29:47,935][105620] Updated weights for policy 1, policy_version 978493 (0.0009) [2023-12-26 22:29:47,980][105620] Updated weights for policy 1, policy_version 978503 (0.0006) [2023-12-26 22:29:48,367][105692] Updated weights for policy 0, policy_version 978410 (0.0007) [2023-12-26 22:29:48,419][105692] Updated weights for policy 0, policy_version 978420 (0.0011) [2023-12-26 22:29:48,480][105692] Updated weights for policy 0, policy_version 978430 (0.0010) [2023-12-26 22:29:48,543][105692] Updated weights for policy 0, policy_version 978440 (0.0010) [2023-12-26 22:29:48,630][105620] Updated weights for policy 1, policy_version 978513 (0.0010) [2023-12-26 22:29:48,688][105620] Updated weights for policy 1, policy_version 978523 (0.0010) [2023-12-26 22:29:48,755][105620] Updated weights for policy 1, policy_version 978533 (0.0006) [2023-12-26 22:29:49,271][105692] Updated weights for policy 0, policy_version 978450 (0.0011) [2023-12-26 22:29:49,334][105692] Updated weights for policy 0, policy_version 978460 (0.0011) [2023-12-26 22:29:49,398][105692] Updated weights for policy 0, policy_version 978470 (0.0011) [2023-12-26 22:29:49,469][105620] Updated weights for policy 1, policy_version 978543 (0.0007) [2023-12-26 22:29:49,527][105620] Updated weights for policy 1, policy_version 978553 (0.0005) [2023-12-26 22:29:49,574][105620] Updated weights for policy 1, policy_version 978563 (0.0005) [2023-12-26 22:29:50,160][105692] Updated weights for policy 0, policy_version 978480 (0.0007) [2023-12-26 22:29:50,225][105692] Updated weights for policy 0, policy_version 978490 (0.0007) [2023-12-26 22:29:50,254][105620] Updated weights for policy 1, policy_version 978573 (0.0007) [2023-12-26 22:29:50,283][105692] Updated weights for policy 0, policy_version 978500 (0.0006) [2023-12-26 22:29:50,314][105620] Updated weights for policy 1, policy_version 978583 (0.0008) [2023-12-26 22:29:50,382][105620] Updated weights for policy 1, policy_version 978593 (0.0008) [2023-12-26 22:29:50,896][105692] Updated weights for policy 0, policy_version 978510 (0.0010) [2023-12-26 22:29:50,959][105692] Updated weights for policy 0, policy_version 978520 (0.0011) [2023-12-26 22:29:51,023][105692] Updated weights for policy 0, policy_version 978530 (0.0011) [2023-12-26 22:29:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19114.6, 300 sec: 19272.0). Total num frames: 501096448. Throughput: 0: 9698.0, 1: 9416.5. Samples: 501084404. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:29:51,062][104569] Avg episode reward: [(0, '7836.335'), (1, '9172.588')] [2023-12-26 22:29:51,177][105620] Updated weights for policy 1, policy_version 978603 (0.0008) [2023-12-26 22:29:51,236][105620] Updated weights for policy 1, policy_version 978613 (0.0010) [2023-12-26 22:29:51,310][105620] Updated weights for policy 1, policy_version 978623 (0.0008) [2023-12-26 22:29:51,757][105692] Updated weights for policy 0, policy_version 978540 (0.0009) [2023-12-26 22:29:51,809][105692] Updated weights for policy 0, policy_version 978550 (0.0005) [2023-12-26 22:29:51,874][105692] Updated weights for policy 0, policy_version 978560 (0.0008) [2023-12-26 22:29:52,052][105620] Updated weights for policy 1, policy_version 978633 (0.0010) [2023-12-26 22:29:52,121][105620] Updated weights for policy 1, policy_version 978643 (0.0006) [2023-12-26 22:29:52,179][105620] Updated weights for policy 1, policy_version 978653 (0.0008) [2023-12-26 22:29:52,254][105620] Updated weights for policy 1, policy_version 978663 (0.0007) [2023-12-26 22:29:52,566][105692] Updated weights for policy 0, policy_version 978570 (0.0008) [2023-12-26 22:29:52,626][105692] Updated weights for policy 0, policy_version 978580 (0.0008) [2023-12-26 22:29:52,686][105692] Updated weights for policy 0, policy_version 978590 (0.0008) [2023-12-26 22:29:52,754][105692] Updated weights for policy 0, policy_version 978600 (0.0008) [2023-12-26 22:29:52,944][105620] Updated weights for policy 1, policy_version 978673 (0.0010) [2023-12-26 22:29:53,004][105620] Updated weights for policy 1, policy_version 978683 (0.0011) [2023-12-26 22:29:53,063][105620] Updated weights for policy 1, policy_version 978693 (0.0010) [2023-12-26 22:29:53,430][105692] Updated weights for policy 0, policy_version 978610 (0.0008) [2023-12-26 22:29:53,494][105692] Updated weights for policy 0, policy_version 978620 (0.0008) [2023-12-26 22:29:53,557][105692] Updated weights for policy 0, policy_version 978630 (0.0008) [2023-12-26 22:29:53,800][105620] Updated weights for policy 1, policy_version 978703 (0.0010) [2023-12-26 22:29:53,848][105620] Updated weights for policy 1, policy_version 978713 (0.0010) [2023-12-26 22:29:53,892][105620] Updated weights for policy 1, policy_version 978723 (0.0010) [2023-12-26 22:29:54,216][105692] Updated weights for policy 0, policy_version 978640 (0.0007) [2023-12-26 22:29:54,268][105692] Updated weights for policy 0, policy_version 978650 (0.0008) [2023-12-26 22:29:54,329][105692] Updated weights for policy 0, policy_version 978660 (0.0008) [2023-12-26 22:29:54,657][105620] Updated weights for policy 1, policy_version 978733 (0.0010) [2023-12-26 22:29:54,712][105620] Updated weights for policy 1, policy_version 978743 (0.0010) [2023-12-26 22:29:54,763][105620] Updated weights for policy 1, policy_version 978753 (0.0010) [2023-12-26 22:29:55,038][105692] Updated weights for policy 0, policy_version 978670 (0.0007) [2023-12-26 22:29:55,103][105692] Updated weights for policy 0, policy_version 978680 (0.0005) [2023-12-26 22:29:55,170][105692] Updated weights for policy 0, policy_version 978690 (0.0005) [2023-12-26 22:29:55,494][105620] Updated weights for policy 1, policy_version 978763 (0.0009) [2023-12-26 22:29:55,554][105620] Updated weights for policy 1, policy_version 978773 (0.0007) [2023-12-26 22:29:55,611][105620] Updated weights for policy 1, policy_version 978783 (0.0010) [2023-12-26 22:29:55,751][105692] Updated weights for policy 0, policy_version 978700 (0.0005) [2023-12-26 22:29:55,812][105692] Updated weights for policy 0, policy_version 978710 (0.0006) [2023-12-26 22:29:55,874][105692] Updated weights for policy 0, policy_version 978720 (0.0008) [2023-12-26 22:29:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19114.8, 300 sec: 19272.0). Total num frames: 501194752. Throughput: 0: 9746.4, 1: 9473.5. Samples: 501201548. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:29:56,062][104569] Avg episode reward: [(0, '7228.605'), (1, '8989.543')] [2023-12-26 22:29:56,240][105620] Updated weights for policy 1, policy_version 978793 (0.0010) [2023-12-26 22:29:56,291][105620] Updated weights for policy 1, policy_version 978803 (0.0010) [2023-12-26 22:29:56,352][105620] Updated weights for policy 1, policy_version 978813 (0.0010) [2023-12-26 22:29:56,412][105620] Updated weights for policy 1, policy_version 978823 (0.0011) [2023-12-26 22:29:56,439][105692] Updated weights for policy 0, policy_version 978730 (0.0009) [2023-12-26 22:29:56,499][105692] Updated weights for policy 0, policy_version 978740 (0.0006) [2023-12-26 22:29:56,564][105692] Updated weights for policy 0, policy_version 978750 (0.0009) [2023-12-26 22:29:56,628][105692] Updated weights for policy 0, policy_version 978760 (0.0010) [2023-12-26 22:29:57,134][105620] Updated weights for policy 1, policy_version 978833 (0.0010) [2023-12-26 22:29:57,188][105620] Updated weights for policy 1, policy_version 978843 (0.0010) [2023-12-26 22:29:57,250][105620] Updated weights for policy 1, policy_version 978853 (0.0010) [2023-12-26 22:29:57,351][105692] Updated weights for policy 0, policy_version 978770 (0.0008) [2023-12-26 22:29:57,410][105692] Updated weights for policy 0, policy_version 978780 (0.0008) [2023-12-26 22:29:57,466][105692] Updated weights for policy 0, policy_version 978790 (0.0008) [2023-12-26 22:29:57,912][105620] Updated weights for policy 1, policy_version 978863 (0.0009) [2023-12-26 22:29:57,974][105620] Updated weights for policy 1, policy_version 978873 (0.0006) [2023-12-26 22:29:58,034][105620] Updated weights for policy 1, policy_version 978883 (0.0007) [2023-12-26 22:29:58,204][105692] Updated weights for policy 0, policy_version 978800 (0.0008) [2023-12-26 22:29:58,267][105692] Updated weights for policy 0, policy_version 978810 (0.0008) [2023-12-26 22:29:58,333][105692] Updated weights for policy 0, policy_version 978820 (0.0008) [2023-12-26 22:29:58,788][105620] Updated weights for policy 1, policy_version 978893 (0.0009) [2023-12-26 22:29:58,845][105620] Updated weights for policy 1, policy_version 978903 (0.0008) [2023-12-26 22:29:58,906][105620] Updated weights for policy 1, policy_version 978913 (0.0009) [2023-12-26 22:29:59,034][105692] Updated weights for policy 0, policy_version 978830 (0.0008) [2023-12-26 22:29:59,085][105692] Updated weights for policy 0, policy_version 978840 (0.0009) [2023-12-26 22:29:59,136][105692] Updated weights for policy 0, policy_version 978850 (0.0009) [2023-12-26 22:29:59,627][105620] Updated weights for policy 1, policy_version 978923 (0.0006) [2023-12-26 22:29:59,675][105620] Updated weights for policy 1, policy_version 978933 (0.0005) [2023-12-26 22:29:59,733][105620] Updated weights for policy 1, policy_version 978943 (0.0008) [2023-12-26 22:29:59,964][105692] Updated weights for policy 0, policy_version 978860 (0.0009) [2023-12-26 22:30:00,023][105692] Updated weights for policy 0, policy_version 978870 (0.0008) [2023-12-26 22:30:00,074][105692] Updated weights for policy 0, policy_version 978880 (0.0008) [2023-12-26 22:30:00,356][105620] Updated weights for policy 1, policy_version 978953 (0.0008) [2023-12-26 22:30:00,424][105620] Updated weights for policy 1, policy_version 978963 (0.0009) [2023-12-26 22:30:00,484][105620] Updated weights for policy 1, policy_version 978973 (0.0007) [2023-12-26 22:30:00,544][105620] Updated weights for policy 1, policy_version 978983 (0.0008) [2023-12-26 22:30:00,694][105692] Updated weights for policy 0, policy_version 978890 (0.0007) [2023-12-26 22:30:00,759][105692] Updated weights for policy 0, policy_version 978900 (0.0005) [2023-12-26 22:30:00,823][105692] Updated weights for policy 0, policy_version 978910 (0.0006) [2023-12-26 22:30:00,884][105692] Updated weights for policy 0, policy_version 978920 (0.0009) [2023-12-26 22:30:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19272.0). Total num frames: 501293056. Throughput: 0: 9826.1, 1: 9488.1. Samples: 501261424. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:30:01,062][104569] Avg episode reward: [(0, '8018.812'), (1, '9081.642')] [2023-12-26 22:30:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000978920_250642432.pth... [2023-12-26 22:30:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000978984_250650624.pth... [2023-12-26 22:30:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000977864_250363904.pth [2023-12-26 22:30:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000977768_250347520.pth [2023-12-26 22:30:01,322][105620] Updated weights for policy 1, policy_version 978993 (0.0009) [2023-12-26 22:30:01,384][105620] Updated weights for policy 1, policy_version 979003 (0.0009) [2023-12-26 22:30:01,457][105620] Updated weights for policy 1, policy_version 979013 (0.0007) [2023-12-26 22:30:01,571][105692] Updated weights for policy 0, policy_version 978931 (0.0010) [2023-12-26 22:30:01,631][105692] Updated weights for policy 0, policy_version 978941 (0.0010) [2023-12-26 22:30:01,694][105692] Updated weights for policy 0, policy_version 978951 (0.0008) [2023-12-26 22:30:02,194][105620] Updated weights for policy 1, policy_version 979023 (0.0009) [2023-12-26 22:30:02,240][105620] Updated weights for policy 1, policy_version 979033 (0.0009) [2023-12-26 22:30:02,291][105620] Updated weights for policy 1, policy_version 979043 (0.0009) [2023-12-26 22:30:02,475][105692] Updated weights for policy 0, policy_version 978961 (0.0009) [2023-12-26 22:30:02,536][105692] Updated weights for policy 0, policy_version 978971 (0.0009) [2023-12-26 22:30:02,595][105692] Updated weights for policy 0, policy_version 978981 (0.0006) [2023-12-26 22:30:03,060][105620] Updated weights for policy 1, policy_version 979053 (0.0008) [2023-12-26 22:30:03,114][105620] Updated weights for policy 1, policy_version 979063 (0.0005) [2023-12-26 22:30:03,168][105620] Updated weights for policy 1, policy_version 979073 (0.0005) [2023-12-26 22:30:03,345][105692] Updated weights for policy 0, policy_version 978991 (0.0009) [2023-12-26 22:30:03,400][105692] Updated weights for policy 0, policy_version 979001 (0.0009) [2023-12-26 22:30:03,454][105692] Updated weights for policy 0, policy_version 979011 (0.0009) [2023-12-26 22:30:03,689][105620] Updated weights for policy 1, policy_version 979083 (0.0006) [2023-12-26 22:30:03,754][105620] Updated weights for policy 1, policy_version 979093 (0.0009) [2023-12-26 22:30:03,815][105620] Updated weights for policy 1, policy_version 979103 (0.0006) [2023-12-26 22:30:04,311][105692] Updated weights for policy 0, policy_version 979021 (0.0009) [2023-12-26 22:30:04,376][105692] Updated weights for policy 0, policy_version 979031 (0.0008) [2023-12-26 22:30:04,444][105692] Updated weights for policy 0, policy_version 979041 (0.0010) [2023-12-26 22:30:04,479][105620] Updated weights for policy 1, policy_version 979113 (0.0007) [2023-12-26 22:30:04,543][105620] Updated weights for policy 1, policy_version 979123 (0.0005) [2023-12-26 22:30:04,602][105620] Updated weights for policy 1, policy_version 979133 (0.0008) [2023-12-26 22:30:04,658][105620] Updated weights for policy 1, policy_version 979143 (0.0011) [2023-12-26 22:30:05,213][105620] Updated weights for policy 1, policy_version 979153 (0.0010) [2023-12-26 22:30:05,267][105620] Updated weights for policy 1, policy_version 979163 (0.0010) [2023-12-26 22:30:05,321][105692] Updated weights for policy 0, policy_version 979051 (0.0008) [2023-12-26 22:30:05,325][105620] Updated weights for policy 1, policy_version 979173 (0.0010) [2023-12-26 22:30:05,384][105692] Updated weights for policy 0, policy_version 979061 (0.0007) [2023-12-26 22:30:05,448][105692] Updated weights for policy 0, policy_version 979071 (0.0006) [2023-12-26 22:30:06,027][105620] Updated weights for policy 1, policy_version 979183 (0.0008) [2023-12-26 22:30:06,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19114.6, 300 sec: 19244.3). Total num frames: 501383168. Throughput: 0: 9746.2, 1: 9548.7. Samples: 501377532. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:30:06,063][104569] Avg episode reward: [(0, '8110.818'), (1, '9172.853')] [2023-12-26 22:30:06,081][105620] Updated weights for policy 1, policy_version 979193 (0.0010) [2023-12-26 22:30:06,154][105620] Updated weights for policy 1, policy_version 979203 (0.0010) [2023-12-26 22:30:06,155][105692] Updated weights for policy 0, policy_version 979081 (0.0008) [2023-12-26 22:30:06,203][105692] Updated weights for policy 0, policy_version 979091 (0.0008) [2023-12-26 22:30:06,249][105692] Updated weights for policy 0, policy_version 979101 (0.0008) [2023-12-26 22:30:06,298][105692] Updated weights for policy 0, policy_version 979111 (0.0008) [2023-12-26 22:30:06,849][105620] Updated weights for policy 1, policy_version 979213 (0.0008) [2023-12-26 22:30:06,901][105620] Updated weights for policy 1, policy_version 979223 (0.0005) [2023-12-26 22:30:06,957][105620] Updated weights for policy 1, policy_version 979233 (0.0006) [2023-12-26 22:30:07,148][105692] Updated weights for policy 0, policy_version 979121 (0.0009) [2023-12-26 22:30:07,195][105692] Updated weights for policy 0, policy_version 979131 (0.0008) [2023-12-26 22:30:07,243][105692] Updated weights for policy 0, policy_version 979141 (0.0009) [2023-12-26 22:30:07,598][105620] Updated weights for policy 1, policy_version 979243 (0.0006) [2023-12-26 22:30:07,667][105620] Updated weights for policy 1, policy_version 979253 (0.0005) [2023-12-26 22:30:07,728][105620] Updated weights for policy 1, policy_version 979263 (0.0009) [2023-12-26 22:30:08,097][105692] Updated weights for policy 0, policy_version 979151 (0.0009) [2023-12-26 22:30:08,163][105692] Updated weights for policy 0, policy_version 979161 (0.0010) [2023-12-26 22:30:08,231][105692] Updated weights for policy 0, policy_version 979171 (0.0009) [2023-12-26 22:30:08,382][105620] Updated weights for policy 1, policy_version 979273 (0.0009) [2023-12-26 22:30:08,450][105620] Updated weights for policy 1, policy_version 979283 (0.0008) [2023-12-26 22:30:08,512][105620] Updated weights for policy 1, policy_version 979293 (0.0009) [2023-12-26 22:30:08,574][105620] Updated weights for policy 1, policy_version 979303 (0.0009) [2023-12-26 22:30:08,982][105692] Updated weights for policy 0, policy_version 979181 (0.0009) [2023-12-26 22:30:09,037][105692] Updated weights for policy 0, policy_version 979191 (0.0009) [2023-12-26 22:30:09,095][105692] Updated weights for policy 0, policy_version 979201 (0.0009) [2023-12-26 22:30:09,327][105620] Updated weights for policy 1, policy_version 979313 (0.0009) [2023-12-26 22:30:09,397][105620] Updated weights for policy 1, policy_version 979323 (0.0008) [2023-12-26 22:30:09,473][105620] Updated weights for policy 1, policy_version 979333 (0.0009) [2023-12-26 22:30:09,886][105692] Updated weights for policy 0, policy_version 979211 (0.0010) [2023-12-26 22:30:09,951][105692] Updated weights for policy 0, policy_version 979221 (0.0010) [2023-12-26 22:30:10,022][105692] Updated weights for policy 0, policy_version 979231 (0.0007) [2023-12-26 22:30:10,218][105620] Updated weights for policy 1, policy_version 979343 (0.0007) [2023-12-26 22:30:10,280][105620] Updated weights for policy 1, policy_version 979353 (0.0007) [2023-12-26 22:30:10,340][105620] Updated weights for policy 1, policy_version 979363 (0.0008) [2023-12-26 22:30:10,760][105692] Updated weights for policy 0, policy_version 979241 (0.0012) [2023-12-26 22:30:10,822][105692] Updated weights for policy 0, policy_version 979251 (0.0011) [2023-12-26 22:30:10,890][105692] Updated weights for policy 0, policy_version 979261 (0.0011) [2023-12-26 22:30:10,953][105692] Updated weights for policy 0, policy_version 979271 (0.0011) [2023-12-26 22:30:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19272.0). Total num frames: 501481472. Throughput: 0: 9613.5, 1: 9665.1. Samples: 501490464. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:30:11,063][104569] Avg episode reward: [(0, '7393.895'), (1, '9173.405')] [2023-12-26 22:30:11,072][105620] Updated weights for policy 1, policy_version 979373 (0.0008) [2023-12-26 22:30:11,144][105620] Updated weights for policy 1, policy_version 979383 (0.0008) [2023-12-26 22:30:11,217][105620] Updated weights for policy 1, policy_version 979393 (0.0009) [2023-12-26 22:30:11,737][105692] Updated weights for policy 0, policy_version 979281 (0.0012) [2023-12-26 22:30:11,800][105692] Updated weights for policy 0, policy_version 979291 (0.0009) [2023-12-26 22:30:11,864][105692] Updated weights for policy 0, policy_version 979301 (0.0009) [2023-12-26 22:30:11,981][105620] Updated weights for policy 1, policy_version 979403 (0.0008) [2023-12-26 22:30:12,048][105620] Updated weights for policy 1, policy_version 979413 (0.0009) [2023-12-26 22:30:12,114][105620] Updated weights for policy 1, policy_version 979423 (0.0009) [2023-12-26 22:30:12,609][105692] Updated weights for policy 0, policy_version 979311 (0.0008) [2023-12-26 22:30:12,659][105692] Updated weights for policy 0, policy_version 979321 (0.0007) [2023-12-26 22:30:12,726][105692] Updated weights for policy 0, policy_version 979331 (0.0006) [2023-12-26 22:30:12,912][105620] Updated weights for policy 1, policy_version 979433 (0.0009) [2023-12-26 22:30:12,969][105620] Updated weights for policy 1, policy_version 979443 (0.0009) [2023-12-26 22:30:13,027][105620] Updated weights for policy 1, policy_version 979453 (0.0009) [2023-12-26 22:30:13,081][105620] Updated weights for policy 1, policy_version 979463 (0.0008) [2023-12-26 22:30:13,362][105692] Updated weights for policy 0, policy_version 979341 (0.0007) [2023-12-26 22:30:13,409][105692] Updated weights for policy 0, policy_version 979351 (0.0009) [2023-12-26 22:30:13,458][105692] Updated weights for policy 0, policy_version 979361 (0.0009) [2023-12-26 22:30:13,909][105620] Updated weights for policy 1, policy_version 979473 (0.0009) [2023-12-26 22:30:13,960][105620] Updated weights for policy 1, policy_version 979483 (0.0009) [2023-12-26 22:30:14,012][105620] Updated weights for policy 1, policy_version 979493 (0.0009) [2023-12-26 22:30:14,101][105692] Updated weights for policy 0, policy_version 979371 (0.0007) [2023-12-26 22:30:14,156][105692] Updated weights for policy 0, policy_version 979381 (0.0010) [2023-12-26 22:30:14,225][105692] Updated weights for policy 0, policy_version 979391 (0.0008) [2023-12-26 22:30:14,794][105620] Updated weights for policy 1, policy_version 979503 (0.0010) [2023-12-26 22:30:14,811][105692] Updated weights for policy 0, policy_version 979401 (0.0006) [2023-12-26 22:30:14,849][105620] Updated weights for policy 1, policy_version 979513 (0.0010) [2023-12-26 22:30:14,871][105692] Updated weights for policy 0, policy_version 979411 (0.0010) [2023-12-26 22:30:14,909][105620] Updated weights for policy 1, policy_version 979523 (0.0010) [2023-12-26 22:30:14,934][105692] Updated weights for policy 0, policy_version 979421 (0.0011) [2023-12-26 22:30:14,986][105692] Updated weights for policy 0, policy_version 979431 (0.0011) [2023-12-26 22:30:15,630][105620] Updated weights for policy 1, policy_version 979533 (0.0008) [2023-12-26 22:30:15,683][105620] Updated weights for policy 1, policy_version 979543 (0.0005) [2023-12-26 22:30:15,735][105620] Updated weights for policy 1, policy_version 979553 (0.0006) [2023-12-26 22:30:15,737][105692] Updated weights for policy 0, policy_version 979441 (0.0007) [2023-12-26 22:30:15,787][105692] Updated weights for policy 0, policy_version 979451 (0.0008) [2023-12-26 22:30:15,842][105692] Updated weights for policy 0, policy_version 979461 (0.0008) [2023-12-26 22:30:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.3, 300 sec: 19272.0). Total num frames: 501579776. Throughput: 0: 9554.8, 1: 9620.9. Samples: 501545124. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:30:16,063][104569] Avg episode reward: [(0, '7920.141'), (1, '9173.980')] [2023-12-26 22:30:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000979464_250781696.pth... [2023-12-26 22:30:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000979560_250798080.pth... [2023-12-26 22:30:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000978408_250503168.pth [2023-12-26 22:30:16,098][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000978344_250494976.pth [2023-12-26 22:30:16,457][105620] Updated weights for policy 1, policy_version 979563 (0.0010) [2023-12-26 22:30:16,515][105620] Updated weights for policy 1, policy_version 979573 (0.0010) [2023-12-26 22:30:16,549][105692] Updated weights for policy 0, policy_version 979471 (0.0009) [2023-12-26 22:30:16,572][105620] Updated weights for policy 1, policy_version 979583 (0.0009) [2023-12-26 22:30:16,598][105692] Updated weights for policy 0, policy_version 979481 (0.0006) [2023-12-26 22:30:16,648][105692] Updated weights for policy 0, policy_version 979491 (0.0008) [2023-12-26 22:30:17,172][105620] Updated weights for policy 1, policy_version 979593 (0.0010) [2023-12-26 22:30:17,234][105620] Updated weights for policy 1, policy_version 979603 (0.0007) [2023-12-26 22:30:17,293][105620] Updated weights for policy 1, policy_version 979613 (0.0010) [2023-12-26 22:30:17,347][105620] Updated weights for policy 1, policy_version 979623 (0.0010) [2023-12-26 22:30:17,503][105692] Updated weights for policy 0, policy_version 979501 (0.0008) [2023-12-26 22:30:17,552][105692] Updated weights for policy 0, policy_version 979511 (0.0007) [2023-12-26 22:30:17,603][105692] Updated weights for policy 0, policy_version 979521 (0.0009) [2023-12-26 22:30:17,970][105620] Updated weights for policy 1, policy_version 979633 (0.0009) [2023-12-26 22:30:18,015][105620] Updated weights for policy 1, policy_version 979643 (0.0006) [2023-12-26 22:30:18,075][105620] Updated weights for policy 1, policy_version 979653 (0.0006) [2023-12-26 22:30:18,440][105692] Updated weights for policy 0, policy_version 979531 (0.0010) [2023-12-26 22:30:18,502][105692] Updated weights for policy 0, policy_version 979541 (0.0009) [2023-12-26 22:30:18,558][105692] Updated weights for policy 0, policy_version 979551 (0.0009) [2023-12-26 22:30:18,745][105620] Updated weights for policy 1, policy_version 979663 (0.0008) [2023-12-26 22:30:18,793][105620] Updated weights for policy 1, policy_version 979673 (0.0009) [2023-12-26 22:30:18,852][105620] Updated weights for policy 1, policy_version 979683 (0.0009) [2023-12-26 22:30:19,373][105692] Updated weights for policy 0, policy_version 979561 (0.0009) [2023-12-26 22:30:19,440][105692] Updated weights for policy 0, policy_version 979571 (0.0009) [2023-12-26 22:30:19,510][105692] Updated weights for policy 0, policy_version 979581 (0.0009) [2023-12-26 22:30:19,567][105620] Updated weights for policy 1, policy_version 979693 (0.0009) [2023-12-26 22:30:19,568][105692] Updated weights for policy 0, policy_version 979591 (0.0007) [2023-12-26 22:30:19,633][105620] Updated weights for policy 1, policy_version 979703 (0.0009) [2023-12-26 22:30:19,695][105620] Updated weights for policy 1, policy_version 979713 (0.0010) [2023-12-26 22:30:20,306][105692] Updated weights for policy 0, policy_version 979601 (0.0009) [2023-12-26 22:30:20,378][105692] Updated weights for policy 0, policy_version 979611 (0.0009) [2023-12-26 22:30:20,451][105692] Updated weights for policy 0, policy_version 979621 (0.0009) [2023-12-26 22:30:20,575][105620] Updated weights for policy 1, policy_version 979723 (0.0008) [2023-12-26 22:30:20,644][105620] Updated weights for policy 1, policy_version 979733 (0.0008) [2023-12-26 22:30:20,715][105620] Updated weights for policy 1, policy_version 979743 (0.0009) [2023-12-26 22:30:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19114.7, 300 sec: 19244.3). Total num frames: 501669888. Throughput: 0: 9469.6, 1: 9760.6. Samples: 501661072. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:30:21,063][104569] Avg episode reward: [(0, '8378.289'), (1, '9113.592')] [2023-12-26 22:30:21,098][105692] Updated weights for policy 0, policy_version 979631 (0.0008) [2023-12-26 22:30:21,165][105692] Updated weights for policy 0, policy_version 979641 (0.0009) [2023-12-26 22:30:21,234][105692] Updated weights for policy 0, policy_version 979651 (0.0010) [2023-12-26 22:30:21,533][105620] Updated weights for policy 1, policy_version 979753 (0.0009) [2023-12-26 22:30:21,597][105620] Updated weights for policy 1, policy_version 979763 (0.0008) [2023-12-26 22:30:21,666][105620] Updated weights for policy 1, policy_version 979773 (0.0008) [2023-12-26 22:30:21,742][105620] Updated weights for policy 1, policy_version 979783 (0.0009) [2023-12-26 22:30:22,103][105692] Updated weights for policy 0, policy_version 979661 (0.0010) [2023-12-26 22:30:22,162][105692] Updated weights for policy 0, policy_version 979671 (0.0011) [2023-12-26 22:30:22,185][105585] KL-divergence is very high: 174.0060 [2023-12-26 22:30:22,217][105692] Updated weights for policy 0, policy_version 979681 (0.0010) [2023-12-26 22:30:22,228][105585] KL-divergence is very high: 318.6073 [2023-12-26 22:30:22,431][105620] Updated weights for policy 1, policy_version 979793 (0.0008) [2023-12-26 22:30:22,496][105620] Updated weights for policy 1, policy_version 979803 (0.0009) [2023-12-26 22:30:22,572][105620] Updated weights for policy 1, policy_version 979813 (0.0008) [2023-12-26 22:30:23,122][105692] Updated weights for policy 0, policy_version 979691 (0.0009) [2023-12-26 22:30:23,180][105692] Updated weights for policy 0, policy_version 979701 (0.0010) [2023-12-26 22:30:23,236][105692] Updated weights for policy 0, policy_version 979711 (0.0008) [2023-12-26 22:30:23,313][105620] Updated weights for policy 1, policy_version 979823 (0.0008) [2023-12-26 22:30:23,359][105620] Updated weights for policy 1, policy_version 979833 (0.0006) [2023-12-26 22:30:23,403][105620] Updated weights for policy 1, policy_version 979843 (0.0005) [2023-12-26 22:30:24,016][105620] Updated weights for policy 1, policy_version 979853 (0.0007) [2023-12-26 22:30:24,022][105692] Updated weights for policy 0, policy_version 979721 (0.0008) [2023-12-26 22:30:24,072][105620] Updated weights for policy 1, policy_version 979863 (0.0007) [2023-12-26 22:30:24,086][105692] Updated weights for policy 0, policy_version 979731 (0.0007) [2023-12-26 22:30:24,122][105620] Updated weights for policy 1, policy_version 979873 (0.0007) [2023-12-26 22:30:24,141][105692] Updated weights for policy 0, policy_version 979741 (0.0006) [2023-12-26 22:30:24,205][105692] Updated weights for policy 0, policy_version 979751 (0.0008) [2023-12-26 22:30:24,837][105620] Updated weights for policy 1, policy_version 979883 (0.0008) [2023-12-26 22:30:24,890][105620] Updated weights for policy 1, policy_version 979894 (0.0010) [2023-12-26 22:30:24,905][105692] Updated weights for policy 0, policy_version 979761 (0.0008) [2023-12-26 22:30:24,948][105620] Updated weights for policy 1, policy_version 979904 (0.0007) [2023-12-26 22:30:24,954][105692] Updated weights for policy 0, policy_version 979771 (0.0008) [2023-12-26 22:30:25,015][105692] Updated weights for policy 0, policy_version 979781 (0.0007) [2023-12-26 22:30:25,681][105692] Updated weights for policy 0, policy_version 979791 (0.0006) [2023-12-26 22:30:25,745][105692] Updated weights for policy 0, policy_version 979801 (0.0005) [2023-12-26 22:30:25,772][105620] Updated weights for policy 1, policy_version 979914 (0.0008) [2023-12-26 22:30:25,800][105692] Updated weights for policy 0, policy_version 979811 (0.0006) [2023-12-26 22:30:25,825][105620] Updated weights for policy 1, policy_version 979924 (0.0008) [2023-12-26 22:30:25,880][105620] Updated weights for policy 1, policy_version 979934 (0.0008) [2023-12-26 22:30:25,942][105620] Updated weights for policy 1, policy_version 979944 (0.0009) [2023-12-26 22:30:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19244.3). Total num frames: 501768192. Throughput: 0: 9398.9, 1: 9778.1. Samples: 501772500. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:30:26,063][104569] Avg episode reward: [(0, '7663.057'), (1, '8837.095')] [2023-12-26 22:30:26,482][105692] Updated weights for policy 0, policy_version 979821 (0.0008) [2023-12-26 22:30:26,537][105692] Updated weights for policy 0, policy_version 979833 (0.0010) [2023-12-26 22:30:26,590][105692] Updated weights for policy 0, policy_version 979844 (0.0010) [2023-12-26 22:30:26,661][105620] Updated weights for policy 1, policy_version 979954 (0.0008) [2023-12-26 22:30:26,718][105620] Updated weights for policy 1, policy_version 979964 (0.0009) [2023-12-26 22:30:26,779][105620] Updated weights for policy 1, policy_version 979974 (0.0008) [2023-12-26 22:30:27,412][105692] Updated weights for policy 0, policy_version 979854 (0.0009) [2023-12-26 22:30:27,464][105620] Updated weights for policy 1, policy_version 979984 (0.0006) [2023-12-26 22:30:27,480][105692] Updated weights for policy 0, policy_version 979864 (0.0008) [2023-12-26 22:30:27,529][105620] Updated weights for policy 1, policy_version 979994 (0.0011) [2023-12-26 22:30:27,543][105692] Updated weights for policy 0, policy_version 979874 (0.0008) [2023-12-26 22:30:27,580][105620] Updated weights for policy 1, policy_version 980004 (0.0010) [2023-12-26 22:30:28,214][105620] Updated weights for policy 1, policy_version 980014 (0.0010) [2023-12-26 22:30:28,269][105692] Updated weights for policy 0, policy_version 979884 (0.0008) [2023-12-26 22:30:28,275][105620] Updated weights for policy 1, policy_version 980024 (0.0007) [2023-12-26 22:30:28,325][105692] Updated weights for policy 0, policy_version 979894 (0.0006) [2023-12-26 22:30:28,341][105620] Updated weights for policy 1, policy_version 980034 (0.0008) [2023-12-26 22:30:28,386][105692] Updated weights for policy 0, policy_version 979904 (0.0007) [2023-12-26 22:30:28,970][105620] Updated weights for policy 1, policy_version 980044 (0.0010) [2023-12-26 22:30:29,028][105620] Updated weights for policy 1, policy_version 980054 (0.0009) [2023-12-26 22:30:29,079][105620] Updated weights for policy 1, policy_version 980064 (0.0008) [2023-12-26 22:30:29,204][105692] Updated weights for policy 0, policy_version 979914 (0.0009) [2023-12-26 22:30:29,264][105692] Updated weights for policy 0, policy_version 979924 (0.0009) [2023-12-26 22:30:29,319][105692] Updated weights for policy 0, policy_version 979934 (0.0008) [2023-12-26 22:30:29,387][105692] Updated weights for policy 0, policy_version 979944 (0.0009) [2023-12-26 22:30:29,844][105620] Updated weights for policy 1, policy_version 980074 (0.0009) [2023-12-26 22:30:29,902][105620] Updated weights for policy 1, policy_version 980084 (0.0009) [2023-12-26 22:30:29,962][105620] Updated weights for policy 1, policy_version 980094 (0.0009) [2023-12-26 22:30:30,014][105620] Updated weights for policy 1, policy_version 980104 (0.0009) [2023-12-26 22:30:30,154][105692] Updated weights for policy 0, policy_version 979954 (0.0010) [2023-12-26 22:30:30,208][105692] Updated weights for policy 0, policy_version 979965 (0.0010) [2023-12-26 22:30:30,261][105692] Updated weights for policy 0, policy_version 979975 (0.0010) [2023-12-26 22:30:30,599][105620] Updated weights for policy 1, policy_version 980114 (0.0007) [2023-12-26 22:30:30,659][105620] Updated weights for policy 1, policy_version 980124 (0.0007) [2023-12-26 22:30:30,705][105620] Updated weights for policy 1, policy_version 980134 (0.0008) [2023-12-26 22:30:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19216.5). Total num frames: 501858304. Throughput: 0: 9405.8, 1: 9847.2. Samples: 501831824. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:30:31,062][104569] Avg episode reward: [(0, '7572.712'), (1, '8904.164')] [2023-12-26 22:30:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000980136_250945536.pth... [2023-12-26 22:30:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000978984_250650624.pth [2023-12-26 22:30:31,129][105692] Updated weights for policy 0, policy_version 979985 (0.0009) [2023-12-26 22:30:31,189][105692] Updated weights for policy 0, policy_version 979995 (0.0009) [2023-12-26 22:30:31,249][105692] Updated weights for policy 0, policy_version 980005 (0.0008) [2023-12-26 22:30:31,269][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000980008_250920960.pth... [2023-12-26 22:30:31,273][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000978920_250642432.pth [2023-12-26 22:30:31,385][105620] Updated weights for policy 1, policy_version 980144 (0.0009) [2023-12-26 22:30:31,442][105620] Updated weights for policy 1, policy_version 980154 (0.0009) [2023-12-26 22:30:31,500][105620] Updated weights for policy 1, policy_version 980164 (0.0007) [2023-12-26 22:30:32,006][105692] Updated weights for policy 0, policy_version 980015 (0.0010) [2023-12-26 22:30:32,075][105692] Updated weights for policy 0, policy_version 980025 (0.0010) [2023-12-26 22:30:32,138][105692] Updated weights for policy 0, policy_version 980035 (0.0008) [2023-12-26 22:30:32,157][105620] Updated weights for policy 1, policy_version 980174 (0.0009) [2023-12-26 22:30:32,223][105620] Updated weights for policy 1, policy_version 980184 (0.0008) [2023-12-26 22:30:32,285][105620] Updated weights for policy 1, policy_version 980194 (0.0008) [2023-12-26 22:30:32,870][105692] Updated weights for policy 0, policy_version 980045 (0.0010) [2023-12-26 22:30:32,914][105620] Updated weights for policy 1, policy_version 980204 (0.0006) [2023-12-26 22:30:32,931][105692] Updated weights for policy 0, policy_version 980055 (0.0010) [2023-12-26 22:30:32,966][105620] Updated weights for policy 1, policy_version 980214 (0.0006) [2023-12-26 22:30:32,990][105692] Updated weights for policy 0, policy_version 980065 (0.0008) [2023-12-26 22:30:33,013][105620] Updated weights for policy 1, policy_version 980224 (0.0005) [2023-12-26 22:30:33,738][105620] Updated weights for policy 1, policy_version 980234 (0.0007) [2023-12-26 22:30:33,748][105692] Updated weights for policy 0, policy_version 980075 (0.0008) [2023-12-26 22:30:33,789][105620] Updated weights for policy 1, policy_version 980244 (0.0008) [2023-12-26 22:30:33,796][105692] Updated weights for policy 0, policy_version 980085 (0.0006) [2023-12-26 22:30:33,840][105692] Updated weights for policy 0, policy_version 980095 (0.0006) [2023-12-26 22:30:33,841][105620] Updated weights for policy 1, policy_version 980254 (0.0008) [2023-12-26 22:30:33,898][105620] Updated weights for policy 1, policy_version 980264 (0.0008) [2023-12-26 22:30:34,591][105692] Updated weights for policy 0, policy_version 980105 (0.0006) [2023-12-26 22:30:34,651][105692] Updated weights for policy 0, policy_version 980115 (0.0007) [2023-12-26 22:30:34,675][105620] Updated weights for policy 1, policy_version 980274 (0.0010) [2023-12-26 22:30:34,714][105692] Updated weights for policy 0, policy_version 980125 (0.0009) [2023-12-26 22:30:34,728][105620] Updated weights for policy 1, policy_version 980284 (0.0007) [2023-12-26 22:30:34,770][105692] Updated weights for policy 0, policy_version 980135 (0.0007) [2023-12-26 22:30:34,789][105620] Updated weights for policy 1, policy_version 980294 (0.0007) [2023-12-26 22:30:35,426][105692] Updated weights for policy 0, policy_version 980145 (0.0009) [2023-12-26 22:30:35,476][105692] Updated weights for policy 0, policy_version 980155 (0.0010) [2023-12-26 22:30:35,521][105692] Updated weights for policy 0, policy_version 980165 (0.0010) [2023-12-26 22:30:35,597][105620] Updated weights for policy 1, policy_version 980304 (0.0008) [2023-12-26 22:30:35,662][105620] Updated weights for policy 1, policy_version 980314 (0.0007) [2023-12-26 22:30:35,716][105620] Updated weights for policy 1, policy_version 980324 (0.0008) [2023-12-26 22:30:36,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19251.3, 300 sec: 19216.5). Total num frames: 501956608. Throughput: 0: 9397.2, 1: 9750.0. Samples: 501946024. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:30:36,062][104569] Avg episode reward: [(0, '6437.185'), (1, '9087.973')] [2023-12-26 22:30:36,284][105692] Updated weights for policy 0, policy_version 980175 (0.0011) [2023-12-26 22:30:36,344][105692] Updated weights for policy 0, policy_version 980185 (0.0010) [2023-12-26 22:30:36,406][105692] Updated weights for policy 0, policy_version 980195 (0.0010) [2023-12-26 22:30:36,473][105620] Updated weights for policy 1, policy_version 980334 (0.0006) [2023-12-26 22:30:36,545][105620] Updated weights for policy 1, policy_version 980344 (0.0008) [2023-12-26 22:30:36,622][105620] Updated weights for policy 1, policy_version 980354 (0.0006) [2023-12-26 22:30:37,125][105692] Updated weights for policy 0, policy_version 980205 (0.0010) [2023-12-26 22:30:37,192][105692] Updated weights for policy 0, policy_version 980215 (0.0007) [2023-12-26 22:30:37,250][105692] Updated weights for policy 0, policy_version 980225 (0.0011) [2023-12-26 22:30:37,296][105620] Updated weights for policy 1, policy_version 980364 (0.0006) [2023-12-26 22:30:37,361][105620] Updated weights for policy 1, policy_version 980374 (0.0008) [2023-12-26 22:30:37,425][105620] Updated weights for policy 1, policy_version 980384 (0.0008) [2023-12-26 22:30:37,849][105692] Updated weights for policy 0, policy_version 980235 (0.0009) [2023-12-26 22:30:37,899][105692] Updated weights for policy 0, policy_version 980245 (0.0006) [2023-12-26 22:30:37,959][105692] Updated weights for policy 0, policy_version 980255 (0.0006) [2023-12-26 22:30:37,975][105620] Updated weights for policy 1, policy_version 980394 (0.0006) [2023-12-26 22:30:38,039][105620] Updated weights for policy 1, policy_version 980404 (0.0011) [2023-12-26 22:30:38,105][105620] Updated weights for policy 1, policy_version 980414 (0.0007) [2023-12-26 22:30:38,174][105620] Updated weights for policy 1, policy_version 980424 (0.0006) [2023-12-26 22:30:38,562][105692] Updated weights for policy 0, policy_version 980265 (0.0007) [2023-12-26 22:30:38,628][105692] Updated weights for policy 0, policy_version 980275 (0.0007) [2023-12-26 22:30:38,695][105692] Updated weights for policy 0, policy_version 980285 (0.0009) [2023-12-26 22:30:38,750][105692] Updated weights for policy 0, policy_version 980295 (0.0009) [2023-12-26 22:30:38,820][105620] Updated weights for policy 1, policy_version 980434 (0.0010) [2023-12-26 22:30:38,879][105620] Updated weights for policy 1, policy_version 980444 (0.0010) [2023-12-26 22:30:38,938][105620] Updated weights for policy 1, policy_version 980454 (0.0010) [2023-12-26 22:30:39,507][105692] Updated weights for policy 0, policy_version 980305 (0.0010) [2023-12-26 22:30:39,569][105692] Updated weights for policy 0, policy_version 980315 (0.0009) [2023-12-26 22:30:39,626][105692] Updated weights for policy 0, policy_version 980325 (0.0009) [2023-12-26 22:30:39,682][105620] Updated weights for policy 1, policy_version 980464 (0.0010) [2023-12-26 22:30:39,749][105620] Updated weights for policy 1, policy_version 980474 (0.0011) [2023-12-26 22:30:39,820][105620] Updated weights for policy 1, policy_version 980484 (0.0010) [2023-12-26 22:30:40,443][105692] Updated weights for policy 0, policy_version 980335 (0.0008) [2023-12-26 22:30:40,461][105620] Updated weights for policy 1, policy_version 980494 (0.0009) [2023-12-26 22:30:40,508][105692] Updated weights for policy 0, policy_version 980345 (0.0007) [2023-12-26 22:30:40,527][105620] Updated weights for policy 1, policy_version 980504 (0.0008) [2023-12-26 22:30:40,571][105692] Updated weights for policy 0, policy_version 980355 (0.0008) [2023-12-26 22:30:40,596][105620] Updated weights for policy 1, policy_version 980514 (0.0008) [2023-12-26 22:30:41,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19251.1, 300 sec: 19216.5). Total num frames: 502054912. Throughput: 0: 9333.7, 1: 9849.0. Samples: 502064776. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:30:41,063][104569] Avg episode reward: [(0, '7246.968'), (1, '9356.778')] [2023-12-26 22:30:41,223][105620] Updated weights for policy 1, policy_version 980524 (0.0009) [2023-12-26 22:30:41,225][105692] Updated weights for policy 0, policy_version 980365 (0.0007) [2023-12-26 22:30:41,292][105620] Updated weights for policy 1, policy_version 980534 (0.0009) [2023-12-26 22:30:41,296][105692] Updated weights for policy 0, policy_version 980375 (0.0009) [2023-12-26 22:30:41,360][105620] Updated weights for policy 1, policy_version 980544 (0.0008) [2023-12-26 22:30:41,365][105692] Updated weights for policy 0, policy_version 980385 (0.0007) [2023-12-26 22:30:42,125][105620] Updated weights for policy 1, policy_version 980554 (0.0009) [2023-12-26 22:30:42,183][105620] Updated weights for policy 1, policy_version 980564 (0.0010) [2023-12-26 22:30:42,201][105692] Updated weights for policy 0, policy_version 980395 (0.0008) [2023-12-26 22:30:42,249][105620] Updated weights for policy 1, policy_version 980574 (0.0007) [2023-12-26 22:30:42,262][105692] Updated weights for policy 0, policy_version 980405 (0.0010) [2023-12-26 22:30:42,306][105620] Updated weights for policy 1, policy_version 980584 (0.0009) [2023-12-26 22:30:42,320][105692] Updated weights for policy 0, policy_version 980415 (0.0009) [2023-12-26 22:30:43,058][105692] Updated weights for policy 0, policy_version 980425 (0.0009) [2023-12-26 22:30:43,113][105692] Updated weights for policy 0, policy_version 980435 (0.0007) [2023-12-26 22:30:43,120][105620] Updated weights for policy 1, policy_version 980594 (0.0007) [2023-12-26 22:30:43,171][105620] Updated weights for policy 1, policy_version 980604 (0.0006) [2023-12-26 22:30:43,173][105692] Updated weights for policy 0, policy_version 980445 (0.0009) [2023-12-26 22:30:43,228][105620] Updated weights for policy 1, policy_version 980614 (0.0006) [2023-12-26 22:30:43,230][105692] Updated weights for policy 0, policy_version 980455 (0.0006) [2023-12-26 22:30:43,826][105620] Updated weights for policy 1, policy_version 980624 (0.0010) [2023-12-26 22:30:43,871][105620] Updated weights for policy 1, policy_version 980634 (0.0010) [2023-12-26 22:30:43,917][105620] Updated weights for policy 1, policy_version 980644 (0.0010) [2023-12-26 22:30:44,019][105692] Updated weights for policy 0, policy_version 980465 (0.0009) [2023-12-26 22:30:44,079][105692] Updated weights for policy 0, policy_version 980476 (0.0011) [2023-12-26 22:30:44,138][105692] Updated weights for policy 0, policy_version 980487 (0.0010) [2023-12-26 22:30:44,593][105620] Updated weights for policy 1, policy_version 980654 (0.0007) [2023-12-26 22:30:44,647][105620] Updated weights for policy 1, policy_version 980664 (0.0006) [2023-12-26 22:30:44,707][105620] Updated weights for policy 1, policy_version 980674 (0.0009) [2023-12-26 22:30:44,921][105692] Updated weights for policy 0, policy_version 980497 (0.0009) [2023-12-26 22:30:44,986][105692] Updated weights for policy 0, policy_version 980507 (0.0010) [2023-12-26 22:30:45,044][105692] Updated weights for policy 0, policy_version 980517 (0.0010) [2023-12-26 22:30:45,309][105620] Updated weights for policy 1, policy_version 980684 (0.0008) [2023-12-26 22:30:45,375][105620] Updated weights for policy 1, policy_version 980694 (0.0006) [2023-12-26 22:30:45,434][105620] Updated weights for policy 1, policy_version 980704 (0.0009) [2023-12-26 22:30:45,866][105692] Updated weights for policy 0, policy_version 980527 (0.0009) [2023-12-26 22:30:45,926][105692] Updated weights for policy 0, policy_version 980537 (0.0009) [2023-12-26 22:30:45,982][105692] Updated weights for policy 0, policy_version 980547 (0.0009) [2023-12-26 22:30:46,053][105620] Updated weights for policy 1, policy_version 980714 (0.0009) [2023-12-26 22:30:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19251.1, 300 sec: 19216.5). Total num frames: 502153216. Throughput: 0: 9269.9, 1: 9833.2. Samples: 502121068. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:30:46,063][104569] Avg episode reward: [(0, '8039.775'), (1, '9356.825')] [2023-12-26 22:30:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000980552_251060224.pth... [2023-12-26 22:30:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000979464_250781696.pth [2023-12-26 22:30:46,119][105620] Updated weights for policy 1, policy_version 980724 (0.0005) [2023-12-26 22:30:46,183][105620] Updated weights for policy 1, policy_version 980734 (0.0005) [2023-12-26 22:30:46,251][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000980744_251101184.pth... [2023-12-26 22:30:46,253][105620] Updated weights for policy 1, policy_version 980744 (0.0005) [2023-12-26 22:30:46,256][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000979560_250798080.pth [2023-12-26 22:30:46,612][105692] Updated weights for policy 0, policy_version 980557 (0.0006) [2023-12-26 22:30:46,664][105692] Updated weights for policy 0, policy_version 980567 (0.0005) [2023-12-26 22:30:46,727][105692] Updated weights for policy 0, policy_version 980577 (0.0009) [2023-12-26 22:30:46,749][105620] Updated weights for policy 1, policy_version 980754 (0.0006) [2023-12-26 22:30:46,806][105620] Updated weights for policy 1, policy_version 980764 (0.0007) [2023-12-26 22:30:46,872][105620] Updated weights for policy 1, policy_version 980774 (0.0008) [2023-12-26 22:30:47,451][105620] Updated weights for policy 1, policy_version 980784 (0.0006) [2023-12-26 22:30:47,472][105692] Updated weights for policy 0, policy_version 980587 (0.0011) [2023-12-26 22:30:47,499][105620] Updated weights for policy 1, policy_version 980794 (0.0005) [2023-12-26 22:30:47,527][105692] Updated weights for policy 0, policy_version 980597 (0.0010) [2023-12-26 22:30:47,557][105620] Updated weights for policy 1, policy_version 980804 (0.0005) [2023-12-26 22:30:47,585][105692] Updated weights for policy 0, policy_version 980607 (0.0010) [2023-12-26 22:30:48,103][105620] Updated weights for policy 1, policy_version 980814 (0.0006) [2023-12-26 22:30:48,173][105620] Updated weights for policy 1, policy_version 980824 (0.0008) [2023-12-26 22:30:48,227][105620] Updated weights for policy 1, policy_version 980834 (0.0008) [2023-12-26 22:30:48,350][105692] Updated weights for policy 0, policy_version 980617 (0.0010) [2023-12-26 22:30:48,418][105692] Updated weights for policy 0, policy_version 980627 (0.0008) [2023-12-26 22:30:48,487][105692] Updated weights for policy 0, policy_version 980637 (0.0008) [2023-12-26 22:30:48,544][105692] Updated weights for policy 0, policy_version 980647 (0.0008) [2023-12-26 22:30:48,965][105620] Updated weights for policy 1, policy_version 980844 (0.0008) [2023-12-26 22:30:49,010][105620] Updated weights for policy 1, policy_version 980854 (0.0008) [2023-12-26 22:30:49,071][105620] Updated weights for policy 1, policy_version 980864 (0.0008) [2023-12-26 22:30:49,321][105692] Updated weights for policy 0, policy_version 980657 (0.0010) [2023-12-26 22:30:49,391][105692] Updated weights for policy 0, policy_version 980667 (0.0011) [2023-12-26 22:30:49,454][105692] Updated weights for policy 0, policy_version 980677 (0.0011) [2023-12-26 22:30:49,852][105620] Updated weights for policy 1, policy_version 980874 (0.0009) [2023-12-26 22:30:49,919][105620] Updated weights for policy 1, policy_version 980884 (0.0011) [2023-12-26 22:30:49,987][105620] Updated weights for policy 1, policy_version 980894 (0.0010) [2023-12-26 22:30:50,047][105620] Updated weights for policy 1, policy_version 980904 (0.0010) [2023-12-26 22:30:50,104][105692] Updated weights for policy 0, policy_version 980687 (0.0008) [2023-12-26 22:30:50,166][105692] Updated weights for policy 0, policy_version 980697 (0.0008) [2023-12-26 22:30:50,232][105692] Updated weights for policy 0, policy_version 980707 (0.0010) [2023-12-26 22:30:50,769][105620] Updated weights for policy 1, policy_version 980914 (0.0008) [2023-12-26 22:30:50,830][105620] Updated weights for policy 1, policy_version 980924 (0.0008) [2023-12-26 22:30:50,887][105620] Updated weights for policy 1, policy_version 980934 (0.0008) [2023-12-26 22:30:50,970][105692] Updated weights for policy 0, policy_version 980717 (0.0010) [2023-12-26 22:30:51,037][105692] Updated weights for policy 0, policy_version 980727 (0.0009) [2023-12-26 22:30:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19216.5). Total num frames: 502251520. Throughput: 0: 9278.1, 1: 9924.1. Samples: 502241632. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:30:51,063][104569] Avg episode reward: [(0, '7578.688'), (1, '8717.399')] [2023-12-26 22:30:51,103][105692] Updated weights for policy 0, policy_version 980737 (0.0009) [2023-12-26 22:30:51,713][105620] Updated weights for policy 1, policy_version 980944 (0.0008) [2023-12-26 22:30:51,782][105620] Updated weights for policy 1, policy_version 980954 (0.0008) [2023-12-26 22:30:51,848][105620] Updated weights for policy 1, policy_version 980964 (0.0008) [2023-12-26 22:30:51,862][105692] Updated weights for policy 0, policy_version 980747 (0.0007) [2023-12-26 22:30:51,921][105692] Updated weights for policy 0, policy_version 980757 (0.0009) [2023-12-26 22:30:51,984][105692] Updated weights for policy 0, policy_version 980767 (0.0011) [2023-12-26 22:30:52,637][105620] Updated weights for policy 1, policy_version 980974 (0.0008) [2023-12-26 22:30:52,704][105620] Updated weights for policy 1, policy_version 980984 (0.0007) [2023-12-26 22:30:52,724][105692] Updated weights for policy 0, policy_version 980777 (0.0010) [2023-12-26 22:30:52,765][105620] Updated weights for policy 1, policy_version 980994 (0.0010) [2023-12-26 22:30:52,782][105692] Updated weights for policy 0, policy_version 980787 (0.0008) [2023-12-26 22:30:52,841][105692] Updated weights for policy 0, policy_version 980797 (0.0008) [2023-12-26 22:30:52,888][105692] Updated weights for policy 0, policy_version 980807 (0.0008) [2023-12-26 22:30:53,458][105620] Updated weights for policy 1, policy_version 981004 (0.0011) [2023-12-26 22:30:53,521][105620] Updated weights for policy 1, policy_version 981014 (0.0010) [2023-12-26 22:30:53,556][105692] Updated weights for policy 0, policy_version 980817 (0.0007) [2023-12-26 22:30:53,587][105620] Updated weights for policy 1, policy_version 981024 (0.0010) [2023-12-26 22:30:53,609][105692] Updated weights for policy 0, policy_version 980827 (0.0007) [2023-12-26 22:30:53,675][105692] Updated weights for policy 0, policy_version 980837 (0.0010) [2023-12-26 22:30:54,216][105620] Updated weights for policy 1, policy_version 981034 (0.0008) [2023-12-26 22:30:54,275][105620] Updated weights for policy 1, policy_version 981044 (0.0010) [2023-12-26 22:30:54,319][105620] Updated weights for policy 1, policy_version 981054 (0.0010) [2023-12-26 22:30:54,368][105620] Updated weights for policy 1, policy_version 981064 (0.0010) [2023-12-26 22:30:54,459][105692] Updated weights for policy 0, policy_version 980847 (0.0009) [2023-12-26 22:30:54,522][105692] Updated weights for policy 0, policy_version 980857 (0.0008) [2023-12-26 22:30:54,583][105692] Updated weights for policy 0, policy_version 980867 (0.0009) [2023-12-26 22:30:55,136][105620] Updated weights for policy 1, policy_version 981074 (0.0010) [2023-12-26 22:30:55,185][105620] Updated weights for policy 1, policy_version 981084 (0.0010) [2023-12-26 22:30:55,250][105620] Updated weights for policy 1, policy_version 981094 (0.0010) [2023-12-26 22:30:55,294][105692] Updated weights for policy 0, policy_version 980877 (0.0008) [2023-12-26 22:30:55,345][105692] Updated weights for policy 0, policy_version 980887 (0.0008) [2023-12-26 22:30:55,396][105692] Updated weights for policy 0, policy_version 980897 (0.0006) [2023-12-26 22:30:56,012][105692] Updated weights for policy 0, policy_version 980907 (0.0006) [2023-12-26 22:30:56,013][105620] Updated weights for policy 1, policy_version 981104 (0.0010) [2023-12-26 22:30:56,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19114.6, 300 sec: 19188.7). Total num frames: 502341632. Throughput: 0: 9378.9, 1: 9845.4. Samples: 502355560. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:30:56,063][104569] Avg episode reward: [(0, '7406.702'), (1, '8209.881')] [2023-12-26 22:30:56,069][105692] Updated weights for policy 0, policy_version 980917 (0.0010) [2023-12-26 22:30:56,080][105620] Updated weights for policy 1, policy_version 981114 (0.0007) [2023-12-26 22:30:56,130][105692] Updated weights for policy 0, policy_version 980927 (0.0008) [2023-12-26 22:30:56,136][105620] Updated weights for policy 1, policy_version 981124 (0.0005) [2023-12-26 22:30:56,794][105620] Updated weights for policy 1, policy_version 981134 (0.0010) [2023-12-26 22:30:56,850][105620] Updated weights for policy 1, policy_version 981144 (0.0011) [2023-12-26 22:30:56,903][105620] Updated weights for policy 1, policy_version 981154 (0.0010) [2023-12-26 22:30:56,926][105692] Updated weights for policy 0, policy_version 980937 (0.0007) [2023-12-26 22:30:56,982][105692] Updated weights for policy 0, policy_version 980947 (0.0009) [2023-12-26 22:30:57,027][105692] Updated weights for policy 0, policy_version 980957 (0.0008) [2023-12-26 22:30:57,076][105692] Updated weights for policy 0, policy_version 980967 (0.0008) [2023-12-26 22:30:57,613][105620] Updated weights for policy 1, policy_version 981164 (0.0007) [2023-12-26 22:30:57,666][105620] Updated weights for policy 1, policy_version 981174 (0.0006) [2023-12-26 22:30:57,733][105620] Updated weights for policy 1, policy_version 981184 (0.0005) [2023-12-26 22:30:57,849][105692] Updated weights for policy 0, policy_version 980977 (0.0006) [2023-12-26 22:30:57,914][105692] Updated weights for policy 0, policy_version 980987 (0.0008) [2023-12-26 22:30:57,976][105692] Updated weights for policy 0, policy_version 980997 (0.0008) [2023-12-26 22:30:58,523][105620] Updated weights for policy 1, policy_version 981194 (0.0006) [2023-12-26 22:30:58,591][105620] Updated weights for policy 1, policy_version 981204 (0.0009) [2023-12-26 22:30:58,653][105620] Updated weights for policy 1, policy_version 981214 (0.0009) [2023-12-26 22:30:58,717][105620] Updated weights for policy 1, policy_version 981224 (0.0008) [2023-12-26 22:30:58,723][105692] Updated weights for policy 0, policy_version 981007 (0.0008) [2023-12-26 22:30:58,784][105692] Updated weights for policy 0, policy_version 981017 (0.0009) [2023-12-26 22:30:58,854][105692] Updated weights for policy 0, policy_version 981027 (0.0009) [2023-12-26 22:30:59,504][105620] Updated weights for policy 1, policy_version 981234 (0.0009) [2023-12-26 22:30:59,554][105620] Updated weights for policy 1, policy_version 981244 (0.0009) [2023-12-26 22:30:59,606][105692] Updated weights for policy 0, policy_version 981037 (0.0008) [2023-12-26 22:30:59,612][105620] Updated weights for policy 1, policy_version 981254 (0.0008) [2023-12-26 22:30:59,664][105692] Updated weights for policy 0, policy_version 981047 (0.0009) [2023-12-26 22:30:59,728][105692] Updated weights for policy 0, policy_version 981057 (0.0009) [2023-12-26 22:31:00,329][105620] Updated weights for policy 1, policy_version 981264 (0.0009) [2023-12-26 22:31:00,380][105620] Updated weights for policy 1, policy_version 981274 (0.0009) [2023-12-26 22:31:00,426][105620] Updated weights for policy 1, policy_version 981284 (0.0009) [2023-12-26 22:31:00,463][105692] Updated weights for policy 0, policy_version 981067 (0.0009) [2023-12-26 22:31:00,517][105692] Updated weights for policy 0, policy_version 981077 (0.0009) [2023-12-26 22:31:00,573][105692] Updated weights for policy 0, policy_version 981087 (0.0008) [2023-12-26 22:31:00,595][105585] KL-divergence is very high: 100.1610 [2023-12-26 22:31:00,600][105585] KL-divergence is very high: 121.5442 [2023-12-26 22:31:00,604][105585] KL-divergence is very high: 130.5564 [2023-12-26 22:31:01,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19114.7, 300 sec: 19216.5). Total num frames: 502439936. Throughput: 0: 9384.0, 1: 9910.1. Samples: 502413356. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:31:01,062][104569] Avg episode reward: [(0, '7494.131'), (1, '8402.385')] [2023-12-26 22:31:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000981096_251199488.pth... [2023-12-26 22:31:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000981288_251240448.pth... [2023-12-26 22:31:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000980008_250920960.pth [2023-12-26 22:31:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000980136_250945536.pth [2023-12-26 22:31:01,180][105620] Updated weights for policy 1, policy_version 981294 (0.0008) [2023-12-26 22:31:01,231][105620] Updated weights for policy 1, policy_version 981304 (0.0008) [2023-12-26 22:31:01,286][105620] Updated weights for policy 1, policy_version 981314 (0.0008) [2023-12-26 22:31:01,339][105692] Updated weights for policy 0, policy_version 981097 (0.0009) [2023-12-26 22:31:01,399][105692] Updated weights for policy 0, policy_version 981107 (0.0009) [2023-12-26 22:31:01,443][105692] Updated weights for policy 0, policy_version 981117 (0.0007) [2023-12-26 22:31:01,498][105692] Updated weights for policy 0, policy_version 981127 (0.0008) [2023-12-26 22:31:02,028][105620] Updated weights for policy 1, policy_version 981324 (0.0009) [2023-12-26 22:31:02,081][105620] Updated weights for policy 1, policy_version 981334 (0.0008) [2023-12-26 22:31:02,131][105620] Updated weights for policy 1, policy_version 981344 (0.0008) [2023-12-26 22:31:02,297][105692] Updated weights for policy 0, policy_version 981137 (0.0005) [2023-12-26 22:31:02,345][105692] Updated weights for policy 0, policy_version 981147 (0.0006) [2023-12-26 22:31:02,408][105692] Updated weights for policy 0, policy_version 981157 (0.0009) [2023-12-26 22:31:02,849][105620] Updated weights for policy 1, policy_version 981354 (0.0009) [2023-12-26 22:31:02,896][105620] Updated weights for policy 1, policy_version 981364 (0.0008) [2023-12-26 22:31:02,941][105620] Updated weights for policy 1, policy_version 981374 (0.0007) [2023-12-26 22:31:02,999][105620] Updated weights for policy 1, policy_version 981384 (0.0007) [2023-12-26 22:31:03,193][105692] Updated weights for policy 0, policy_version 981167 (0.0009) [2023-12-26 22:31:03,248][105692] Updated weights for policy 0, policy_version 981177 (0.0009) [2023-12-26 22:31:03,296][105692] Updated weights for policy 0, policy_version 981187 (0.0009) [2023-12-26 22:31:03,694][105620] Updated weights for policy 1, policy_version 981394 (0.0005) [2023-12-26 22:31:03,740][105620] Updated weights for policy 1, policy_version 981404 (0.0005) [2023-12-26 22:31:03,793][105620] Updated weights for policy 1, policy_version 981414 (0.0005) [2023-12-26 22:31:04,089][105692] Updated weights for policy 0, policy_version 981197 (0.0009) [2023-12-26 22:31:04,139][105692] Updated weights for policy 0, policy_version 981207 (0.0010) [2023-12-26 22:31:04,196][105692] Updated weights for policy 0, policy_version 981217 (0.0009) [2023-12-26 22:31:04,483][105620] Updated weights for policy 1, policy_version 981424 (0.0008) [2023-12-26 22:31:04,542][105620] Updated weights for policy 1, policy_version 981434 (0.0009) [2023-12-26 22:31:04,598][105620] Updated weights for policy 1, policy_version 981444 (0.0008) [2023-12-26 22:31:04,921][105692] Updated weights for policy 0, policy_version 981227 (0.0008) [2023-12-26 22:31:04,984][105692] Updated weights for policy 0, policy_version 981237 (0.0009) [2023-12-26 22:31:05,050][105692] Updated weights for policy 0, policy_version 981247 (0.0009) [2023-12-26 22:31:05,388][105620] Updated weights for policy 1, policy_version 981454 (0.0005) [2023-12-26 22:31:05,437][105620] Updated weights for policy 1, policy_version 981464 (0.0005) [2023-12-26 22:31:05,501][105620] Updated weights for policy 1, policy_version 981474 (0.0006) [2023-12-26 22:31:05,611][105692] Updated weights for policy 0, policy_version 981257 (0.0006) [2023-12-26 22:31:05,671][105692] Updated weights for policy 0, policy_version 981267 (0.0005) [2023-12-26 22:31:05,722][105692] Updated weights for policy 0, policy_version 981277 (0.0008) [2023-12-26 22:31:05,771][105692] Updated weights for policy 0, policy_version 981287 (0.0006) [2023-12-26 22:31:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19216.5). Total num frames: 502538240. Throughput: 0: 9340.8, 1: 9881.8. Samples: 502526088. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:31:06,063][104569] Avg episode reward: [(0, '7050.287'), (1, '8699.931')] [2023-12-26 22:31:06,162][105620] Updated weights for policy 1, policy_version 981484 (0.0009) [2023-12-26 22:31:06,226][105620] Updated weights for policy 1, policy_version 981494 (0.0008) [2023-12-26 22:31:06,291][105620] Updated weights for policy 1, policy_version 981504 (0.0007) [2023-12-26 22:31:06,473][105692] Updated weights for policy 0, policy_version 981297 (0.0007) [2023-12-26 22:31:06,536][105692] Updated weights for policy 0, policy_version 981307 (0.0008) [2023-12-26 22:31:06,592][105692] Updated weights for policy 0, policy_version 981317 (0.0008) [2023-12-26 22:31:07,038][105620] Updated weights for policy 1, policy_version 981514 (0.0009) [2023-12-26 22:31:07,101][105620] Updated weights for policy 1, policy_version 981524 (0.0011) [2023-12-26 22:31:07,168][105620] Updated weights for policy 1, policy_version 981534 (0.0011) [2023-12-26 22:31:07,223][105620] Updated weights for policy 1, policy_version 981544 (0.0010) [2023-12-26 22:31:07,322][105692] Updated weights for policy 0, policy_version 981327 (0.0008) [2023-12-26 22:31:07,373][105692] Updated weights for policy 0, policy_version 981337 (0.0008) [2023-12-26 22:31:07,436][105692] Updated weights for policy 0, policy_version 981347 (0.0008) [2023-12-26 22:31:07,969][105620] Updated weights for policy 1, policy_version 981554 (0.0010) [2023-12-26 22:31:08,039][105620] Updated weights for policy 1, policy_version 981564 (0.0010) [2023-12-26 22:31:08,094][105620] Updated weights for policy 1, policy_version 981574 (0.0010) [2023-12-26 22:31:08,142][105692] Updated weights for policy 0, policy_version 981357 (0.0008) [2023-12-26 22:31:08,201][105692] Updated weights for policy 0, policy_version 981367 (0.0009) [2023-12-26 22:31:08,262][105692] Updated weights for policy 0, policy_version 981377 (0.0006) [2023-12-26 22:31:08,830][105620] Updated weights for policy 1, policy_version 981584 (0.0011) [2023-12-26 22:31:08,879][105620] Updated weights for policy 1, policy_version 981594 (0.0010) [2023-12-26 22:31:08,907][105692] Updated weights for policy 0, policy_version 981387 (0.0009) [2023-12-26 22:31:08,934][105620] Updated weights for policy 1, policy_version 981604 (0.0010) [2023-12-26 22:31:08,964][105692] Updated weights for policy 0, policy_version 981397 (0.0009) [2023-12-26 22:31:09,011][105692] Updated weights for policy 0, policy_version 981407 (0.0008) [2023-12-26 22:31:09,672][105692] Updated weights for policy 0, policy_version 981417 (0.0009) [2023-12-26 22:31:09,700][105620] Updated weights for policy 1, policy_version 981614 (0.0011) [2023-12-26 22:31:09,739][105692] Updated weights for policy 0, policy_version 981427 (0.0007) [2023-12-26 22:31:09,765][105620] Updated weights for policy 1, policy_version 981624 (0.0011) [2023-12-26 22:31:09,800][105692] Updated weights for policy 0, policy_version 981437 (0.0007) [2023-12-26 22:31:09,832][105620] Updated weights for policy 1, policy_version 981634 (0.0011) [2023-12-26 22:31:09,865][105692] Updated weights for policy 0, policy_version 981447 (0.0009) [2023-12-26 22:31:10,592][105620] Updated weights for policy 1, policy_version 981644 (0.0011) [2023-12-26 22:31:10,619][105692] Updated weights for policy 0, policy_version 981457 (0.0006) [2023-12-26 22:31:10,648][105620] Updated weights for policy 1, policy_version 981654 (0.0010) [2023-12-26 22:31:10,675][105692] Updated weights for policy 0, policy_version 981467 (0.0005) [2023-12-26 22:31:10,705][105620] Updated weights for policy 1, policy_version 981664 (0.0011) [2023-12-26 22:31:10,723][105692] Updated weights for policy 0, policy_version 981477 (0.0006) [2023-12-26 22:31:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19244.3). Total num frames: 502636544. Throughput: 0: 9457.2, 1: 9879.7. Samples: 502642664. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:31:11,063][104569] Avg episode reward: [(0, '7331.855'), (1, '8356.250')] [2023-12-26 22:31:11,471][105620] Updated weights for policy 1, policy_version 981674 (0.0010) [2023-12-26 22:31:11,520][105620] Updated weights for policy 1, policy_version 981684 (0.0010) [2023-12-26 22:31:11,562][105692] Updated weights for policy 0, policy_version 981487 (0.0007) [2023-12-26 22:31:11,565][105620] Updated weights for policy 1, policy_version 981694 (0.0010) [2023-12-26 22:31:11,615][105692] Updated weights for policy 0, policy_version 981497 (0.0011) [2023-12-26 22:31:11,630][105620] Updated weights for policy 1, policy_version 981704 (0.0010) [2023-12-26 22:31:11,681][105692] Updated weights for policy 0, policy_version 981507 (0.0011) [2023-12-26 22:31:12,378][105620] Updated weights for policy 1, policy_version 981714 (0.0008) [2023-12-26 22:31:12,411][105692] Updated weights for policy 0, policy_version 981517 (0.0009) [2023-12-26 22:31:12,443][105620] Updated weights for policy 1, policy_version 981724 (0.0011) [2023-12-26 22:31:12,473][105692] Updated weights for policy 0, policy_version 981527 (0.0008) [2023-12-26 22:31:12,504][105620] Updated weights for policy 1, policy_version 981734 (0.0011) [2023-12-26 22:31:12,532][105692] Updated weights for policy 0, policy_version 981537 (0.0006) [2023-12-26 22:31:13,259][105692] Updated weights for policy 0, policy_version 981547 (0.0008) [2023-12-26 22:31:13,260][105620] Updated weights for policy 1, policy_version 981744 (0.0010) [2023-12-26 22:31:13,318][105692] Updated weights for policy 0, policy_version 981557 (0.0005) [2023-12-26 22:31:13,320][105620] Updated weights for policy 1, policy_version 981754 (0.0011) [2023-12-26 22:31:13,378][105692] Updated weights for policy 0, policy_version 981567 (0.0006) [2023-12-26 22:31:13,379][105620] Updated weights for policy 1, policy_version 981764 (0.0011) [2023-12-26 22:31:14,033][105692] Updated weights for policy 0, policy_version 981577 (0.0006) [2023-12-26 22:31:14,081][105692] Updated weights for policy 0, policy_version 981587 (0.0007) [2023-12-26 22:31:14,107][105620] Updated weights for policy 1, policy_version 981774 (0.0011) [2023-12-26 22:31:14,134][105692] Updated weights for policy 0, policy_version 981597 (0.0006) [2023-12-26 22:31:14,155][105620] Updated weights for policy 1, policy_version 981784 (0.0009) [2023-12-26 22:31:14,188][105692] Updated weights for policy 0, policy_version 981607 (0.0006) [2023-12-26 22:31:14,202][105620] Updated weights for policy 1, policy_version 981794 (0.0006) [2023-12-26 22:31:14,944][105620] Updated weights for policy 1, policy_version 981804 (0.0010) [2023-12-26 22:31:14,979][105692] Updated weights for policy 0, policy_version 981617 (0.0008) [2023-12-26 22:31:15,004][105620] Updated weights for policy 1, policy_version 981814 (0.0011) [2023-12-26 22:31:15,040][105692] Updated weights for policy 0, policy_version 981627 (0.0007) [2023-12-26 22:31:15,058][105620] Updated weights for policy 1, policy_version 981824 (0.0011) [2023-12-26 22:31:15,096][105692] Updated weights for policy 0, policy_version 981637 (0.0005) [2023-12-26 22:31:15,695][105692] Updated weights for policy 0, policy_version 981647 (0.0008) [2023-12-26 22:31:15,749][105692] Updated weights for policy 0, policy_version 981657 (0.0006) [2023-12-26 22:31:15,800][105692] Updated weights for policy 0, policy_version 981667 (0.0005) [2023-12-26 22:31:15,870][105620] Updated weights for policy 1, policy_version 981834 (0.0010) [2023-12-26 22:31:15,927][105620] Updated weights for policy 1, policy_version 981844 (0.0005) [2023-12-26 22:31:15,985][105620] Updated weights for policy 1, policy_version 981854 (0.0009) [2023-12-26 22:31:16,036][105620] Updated weights for policy 1, policy_version 981864 (0.0008) [2023-12-26 22:31:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.3, 300 sec: 19244.3). Total num frames: 502734848. Throughput: 0: 9465.9, 1: 9807.2. Samples: 502699112. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:31:16,062][104569] Avg episode reward: [(0, '7591.374'), (1, '7801.416')] [2023-12-26 22:31:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000981672_251346944.pth... [2023-12-26 22:31:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000981864_251387904.pth... [2023-12-26 22:31:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000980744_251101184.pth [2023-12-26 22:31:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000980552_251060224.pth [2023-12-26 22:31:16,392][105692] Updated weights for policy 0, policy_version 981677 (0.0005) [2023-12-26 22:31:16,457][105692] Updated weights for policy 0, policy_version 981687 (0.0008) [2023-12-26 22:31:16,514][105692] Updated weights for policy 0, policy_version 981697 (0.0009) [2023-12-26 22:31:16,641][105620] Updated weights for policy 1, policy_version 981874 (0.0005) [2023-12-26 22:31:16,714][105620] Updated weights for policy 1, policy_version 981884 (0.0005) [2023-12-26 22:31:16,760][105620] Updated weights for policy 1, policy_version 981894 (0.0005) [2023-12-26 22:31:17,272][105620] Updated weights for policy 1, policy_version 981904 (0.0005) [2023-12-26 22:31:17,329][105692] Updated weights for policy 0, policy_version 981707 (0.0010) [2023-12-26 22:31:17,339][105620] Updated weights for policy 1, policy_version 981914 (0.0005) [2023-12-26 22:31:17,394][105692] Updated weights for policy 0, policy_version 981717 (0.0009) [2023-12-26 22:31:17,399][105620] Updated weights for policy 1, policy_version 981924 (0.0005) [2023-12-26 22:31:17,450][105692] Updated weights for policy 0, policy_version 981727 (0.0008) [2023-12-26 22:31:17,906][105620] Updated weights for policy 1, policy_version 981934 (0.0008) [2023-12-26 22:31:17,961][105620] Updated weights for policy 1, policy_version 981944 (0.0009) [2023-12-26 22:31:18,009][105620] Updated weights for policy 1, policy_version 981954 (0.0008) [2023-12-26 22:31:18,199][105692] Updated weights for policy 0, policy_version 981737 (0.0009) [2023-12-26 22:31:18,250][105692] Updated weights for policy 0, policy_version 981747 (0.0009) [2023-12-26 22:31:18,303][105692] Updated weights for policy 0, policy_version 981757 (0.0009) [2023-12-26 22:31:18,403][105692] Updated weights for policy 0, policy_version 981767 (0.0008) [2023-12-26 22:31:18,757][105620] Updated weights for policy 1, policy_version 981964 (0.0007) [2023-12-26 22:31:18,818][105620] Updated weights for policy 1, policy_version 981974 (0.0006) [2023-12-26 22:31:18,887][105620] Updated weights for policy 1, policy_version 981984 (0.0008) [2023-12-26 22:31:19,032][105692] Updated weights for policy 0, policy_version 981777 (0.0009) [2023-12-26 22:31:19,083][105692] Updated weights for policy 0, policy_version 981787 (0.0009) [2023-12-26 22:31:19,148][105692] Updated weights for policy 0, policy_version 981797 (0.0010) [2023-12-26 22:31:19,610][105620] Updated weights for policy 1, policy_version 981994 (0.0009) [2023-12-26 22:31:19,669][105620] Updated weights for policy 1, policy_version 982004 (0.0006) [2023-12-26 22:31:19,729][105620] Updated weights for policy 1, policy_version 982014 (0.0005) [2023-12-26 22:31:19,797][105620] Updated weights for policy 1, policy_version 982024 (0.0006) [2023-12-26 22:31:19,980][105692] Updated weights for policy 0, policy_version 981807 (0.0009) [2023-12-26 22:31:20,041][105692] Updated weights for policy 0, policy_version 981817 (0.0008) [2023-12-26 22:31:20,097][105692] Updated weights for policy 0, policy_version 981827 (0.0008) [2023-12-26 22:31:20,398][105620] Updated weights for policy 1, policy_version 982034 (0.0009) [2023-12-26 22:31:20,454][105620] Updated weights for policy 1, policy_version 982044 (0.0008) [2023-12-26 22:31:20,502][105620] Updated weights for policy 1, policy_version 982054 (0.0009) [2023-12-26 22:31:20,939][105692] Updated weights for policy 0, policy_version 981837 (0.0008) [2023-12-26 22:31:20,995][105692] Updated weights for policy 0, policy_version 981847 (0.0006) [2023-12-26 22:31:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19216.5). Total num frames: 502824960. Throughput: 0: 9569.5, 1: 9855.5. Samples: 502820148. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:31:21,062][104569] Avg episode reward: [(0, '8025.844'), (1, '7424.685')] [2023-12-26 22:31:21,067][105692] Updated weights for policy 0, policy_version 981857 (0.0009) [2023-12-26 22:31:21,240][105620] Updated weights for policy 1, policy_version 982064 (0.0006) [2023-12-26 22:31:21,308][105620] Updated weights for policy 1, policy_version 982074 (0.0009) [2023-12-26 22:31:21,374][105620] Updated weights for policy 1, policy_version 982084 (0.0008) [2023-12-26 22:31:21,854][105692] Updated weights for policy 0, policy_version 981867 (0.0009) [2023-12-26 22:31:21,913][105692] Updated weights for policy 0, policy_version 981877 (0.0006) [2023-12-26 22:31:21,974][105692] Updated weights for policy 0, policy_version 981887 (0.0007) [2023-12-26 22:31:22,077][105620] Updated weights for policy 1, policy_version 982094 (0.0008) [2023-12-26 22:31:22,136][105620] Updated weights for policy 1, policy_version 982104 (0.0009) [2023-12-26 22:31:22,185][105620] Updated weights for policy 1, policy_version 982114 (0.0007) [2023-12-26 22:31:22,695][105692] Updated weights for policy 0, policy_version 981897 (0.0010) [2023-12-26 22:31:22,758][105692] Updated weights for policy 0, policy_version 981907 (0.0010) [2023-12-26 22:31:22,828][105692] Updated weights for policy 0, policy_version 981917 (0.0008) [2023-12-26 22:31:22,891][105692] Updated weights for policy 0, policy_version 981927 (0.0008) [2023-12-26 22:31:22,961][105620] Updated weights for policy 1, policy_version 982124 (0.0007) [2023-12-26 22:31:23,023][105620] Updated weights for policy 1, policy_version 982134 (0.0009) [2023-12-26 22:31:23,075][105620] Updated weights for policy 1, policy_version 982144 (0.0009) [2023-12-26 22:31:23,675][105692] Updated weights for policy 0, policy_version 981937 (0.0009) [2023-12-26 22:31:23,730][105692] Updated weights for policy 0, policy_version 981947 (0.0009) [2023-12-26 22:31:23,749][105620] Updated weights for policy 1, policy_version 982154 (0.0008) [2023-12-26 22:31:23,775][105692] Updated weights for policy 0, policy_version 981957 (0.0006) [2023-12-26 22:31:23,803][105620] Updated weights for policy 1, policy_version 982164 (0.0007) [2023-12-26 22:31:23,857][105620] Updated weights for policy 1, policy_version 982174 (0.0010) [2023-12-26 22:31:23,912][105620] Updated weights for policy 1, policy_version 982184 (0.0010) [2023-12-26 22:31:24,492][105692] Updated weights for policy 0, policy_version 981967 (0.0007) [2023-12-26 22:31:24,552][105692] Updated weights for policy 0, policy_version 981977 (0.0005) [2023-12-26 22:31:24,557][105585] KL-divergence is very high: 137.6345 [2023-12-26 22:31:24,564][105585] KL-divergence is very high: 130.3678 [2023-12-26 22:31:24,603][105585] KL-divergence is very high: 153.8977 [2023-12-26 22:31:24,610][105585] KL-divergence is very high: 141.9047 [2023-12-26 22:31:24,610][105692] Updated weights for policy 0, policy_version 981987 (0.0006) [2023-12-26 22:31:24,711][105620] Updated weights for policy 1, policy_version 982194 (0.0011) [2023-12-26 22:31:24,770][105620] Updated weights for policy 1, policy_version 982204 (0.0011) [2023-12-26 22:31:24,819][105620] Updated weights for policy 1, policy_version 982214 (0.0011) [2023-12-26 22:31:25,328][105692] Updated weights for policy 0, policy_version 981997 (0.0008) [2023-12-26 22:31:25,377][105692] Updated weights for policy 0, policy_version 982007 (0.0005) [2023-12-26 22:31:25,433][105692] Updated weights for policy 0, policy_version 982017 (0.0005) [2023-12-26 22:31:25,492][105620] Updated weights for policy 1, policy_version 982224 (0.0011) [2023-12-26 22:31:25,546][105586] KL-divergence is very high: 116.2052 [2023-12-26 22:31:25,551][105620] Updated weights for policy 1, policy_version 982234 (0.0011) [2023-12-26 22:31:25,594][105586] KL-divergence is very high: 139.2383 [2023-12-26 22:31:25,614][105620] Updated weights for policy 1, policy_version 982244 (0.0010) [2023-12-26 22:31:26,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19216.5). Total num frames: 502923264. Throughput: 0: 9499.8, 1: 9837.6. Samples: 502934956. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:31:26,062][104569] Avg episode reward: [(0, '8204.046'), (1, '6669.970')] [2023-12-26 22:31:26,179][105620] Updated weights for policy 1, policy_version 982254 (0.0009) [2023-12-26 22:31:26,234][105620] Updated weights for policy 1, policy_version 982264 (0.0010) [2023-12-26 22:31:26,235][105692] Updated weights for policy 0, policy_version 982027 (0.0005) [2023-12-26 22:31:26,286][105692] Updated weights for policy 0, policy_version 982037 (0.0005) [2023-12-26 22:31:26,293][105620] Updated weights for policy 1, policy_version 982274 (0.0010) [2023-12-26 22:31:26,342][105692] Updated weights for policy 0, policy_version 982047 (0.0005) [2023-12-26 22:31:26,969][105620] Updated weights for policy 1, policy_version 982284 (0.0010) [2023-12-26 22:31:27,017][105620] Updated weights for policy 1, policy_version 982294 (0.0008) [2023-12-26 22:31:27,058][105692] Updated weights for policy 0, policy_version 982057 (0.0007) [2023-12-26 22:31:27,063][105620] Updated weights for policy 1, policy_version 982304 (0.0005) [2023-12-26 22:31:27,122][105692] Updated weights for policy 0, policy_version 982067 (0.0009) [2023-12-26 22:31:27,169][105692] Updated weights for policy 0, policy_version 982077 (0.0010) [2023-12-26 22:31:27,223][105692] Updated weights for policy 0, policy_version 982087 (0.0009) [2023-12-26 22:31:27,739][105620] Updated weights for policy 1, policy_version 982314 (0.0006) [2023-12-26 22:31:27,787][105620] Updated weights for policy 1, policy_version 982324 (0.0008) [2023-12-26 22:31:27,837][105620] Updated weights for policy 1, policy_version 982334 (0.0007) [2023-12-26 22:31:27,874][105692] Updated weights for policy 0, policy_version 982097 (0.0010) [2023-12-26 22:31:27,885][105620] Updated weights for policy 1, policy_version 982344 (0.0006) [2023-12-26 22:31:27,934][105692] Updated weights for policy 0, policy_version 982107 (0.0010) [2023-12-26 22:31:28,006][105692] Updated weights for policy 0, policy_version 982117 (0.0009) [2023-12-26 22:31:28,601][105620] Updated weights for policy 1, policy_version 982354 (0.0007) [2023-12-26 22:31:28,664][105620] Updated weights for policy 1, policy_version 982364 (0.0011) [2023-12-26 22:31:28,677][105692] Updated weights for policy 0, policy_version 982127 (0.0010) [2023-12-26 22:31:28,730][105620] Updated weights for policy 1, policy_version 982374 (0.0009) [2023-12-26 22:31:28,741][105692] Updated weights for policy 0, policy_version 982137 (0.0011) [2023-12-26 22:31:28,803][105692] Updated weights for policy 0, policy_version 982147 (0.0008) [2023-12-26 22:31:29,387][105620] Updated weights for policy 1, policy_version 982384 (0.0011) [2023-12-26 22:31:29,452][105620] Updated weights for policy 1, policy_version 982394 (0.0010) [2023-12-26 22:31:29,481][105692] Updated weights for policy 0, policy_version 982157 (0.0007) [2023-12-26 22:31:29,517][105620] Updated weights for policy 1, policy_version 982404 (0.0010) [2023-12-26 22:31:29,539][105692] Updated weights for policy 0, policy_version 982167 (0.0005) [2023-12-26 22:31:29,598][105692] Updated weights for policy 0, policy_version 982177 (0.0005) [2023-12-26 22:31:30,166][105692] Updated weights for policy 0, policy_version 982187 (0.0010) [2023-12-26 22:31:30,221][105692] Updated weights for policy 0, policy_version 982197 (0.0010) [2023-12-26 22:31:30,229][105620] Updated weights for policy 1, policy_version 982414 (0.0010) [2023-12-26 22:31:30,282][105692] Updated weights for policy 0, policy_version 982207 (0.0010) [2023-12-26 22:31:30,284][105620] Updated weights for policy 1, policy_version 982424 (0.0010) [2023-12-26 22:31:30,339][105620] Updated weights for policy 1, policy_version 982434 (0.0010) [2023-12-26 22:31:31,002][105692] Updated weights for policy 0, policy_version 982217 (0.0009) [2023-12-26 22:31:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19216.5). Total num frames: 503021568. Throughput: 0: 9543.7, 1: 9878.7. Samples: 502995076. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:31:31,063][104569] Avg episode reward: [(0, '8032.978'), (1, '5427.101')] [2023-12-26 22:31:31,066][105692] Updated weights for policy 0, policy_version 982227 (0.0011) [2023-12-26 22:31:31,100][105620] Updated weights for policy 1, policy_version 982444 (0.0009) [2023-12-26 22:31:31,130][105692] Updated weights for policy 0, policy_version 982237 (0.0011) [2023-12-26 22:31:31,162][105620] Updated weights for policy 1, policy_version 982454 (0.0008) [2023-12-26 22:31:31,196][105692] Updated weights for policy 0, policy_version 982247 (0.0011) [2023-12-26 22:31:31,199][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000982248_251494400.pth... [2023-12-26 22:31:31,203][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000981096_251199488.pth [2023-12-26 22:31:31,216][105620] Updated weights for policy 1, policy_version 982464 (0.0007) [2023-12-26 22:31:31,254][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000982472_251543552.pth... [2023-12-26 22:31:31,259][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000981288_251240448.pth [2023-12-26 22:31:31,942][105692] Updated weights for policy 0, policy_version 982257 (0.0010) [2023-12-26 22:31:31,977][105620] Updated weights for policy 1, policy_version 982474 (0.0008) [2023-12-26 22:31:32,004][105692] Updated weights for policy 0, policy_version 982267 (0.0008) [2023-12-26 22:31:32,037][105620] Updated weights for policy 1, policy_version 982484 (0.0009) [2023-12-26 22:31:32,056][105692] Updated weights for policy 0, policy_version 982277 (0.0006) [2023-12-26 22:31:32,091][105620] Updated weights for policy 1, policy_version 982494 (0.0008) [2023-12-26 22:31:32,152][105620] Updated weights for policy 1, policy_version 982504 (0.0007) [2023-12-26 22:31:32,814][105692] Updated weights for policy 0, policy_version 982287 (0.0007) [2023-12-26 22:31:32,834][105620] Updated weights for policy 1, policy_version 982514 (0.0006) [2023-12-26 22:31:32,873][105692] Updated weights for policy 0, policy_version 982297 (0.0007) [2023-12-26 22:31:32,895][105620] Updated weights for policy 1, policy_version 982524 (0.0008) [2023-12-26 22:31:32,929][105692] Updated weights for policy 0, policy_version 982307 (0.0006) [2023-12-26 22:31:32,961][105620] Updated weights for policy 1, policy_version 982534 (0.0008) [2023-12-26 22:31:33,507][105692] Updated weights for policy 0, policy_version 982317 (0.0006) [2023-12-26 22:31:33,557][105692] Updated weights for policy 0, policy_version 982327 (0.0005) [2023-12-26 22:31:33,600][105692] Updated weights for policy 0, policy_version 982337 (0.0006) [2023-12-26 22:31:33,602][105620] Updated weights for policy 1, policy_version 982544 (0.0008) [2023-12-26 22:31:33,659][105620] Updated weights for policy 1, policy_version 982554 (0.0009) [2023-12-26 22:31:33,715][105620] Updated weights for policy 1, policy_version 982564 (0.0009) [2023-12-26 22:31:34,285][105692] Updated weights for policy 0, policy_version 982347 (0.0007) [2023-12-26 22:31:34,351][105692] Updated weights for policy 0, policy_version 982357 (0.0009) [2023-12-26 22:31:34,400][105692] Updated weights for policy 0, policy_version 982367 (0.0009) [2023-12-26 22:31:34,471][105620] Updated weights for policy 1, policy_version 982574 (0.0008) [2023-12-26 22:31:34,526][105620] Updated weights for policy 1, policy_version 982584 (0.0005) [2023-12-26 22:31:34,576][105620] Updated weights for policy 1, policy_version 982594 (0.0006) [2023-12-26 22:31:35,095][105692] Updated weights for policy 0, policy_version 982377 (0.0008) [2023-12-26 22:31:35,133][105620] Updated weights for policy 1, policy_version 982604 (0.0005) [2023-12-26 22:31:35,146][105692] Updated weights for policy 0, policy_version 982388 (0.0010) [2023-12-26 22:31:35,198][105692] Updated weights for policy 0, policy_version 982398 (0.0008) [2023-12-26 22:31:35,200][105620] Updated weights for policy 1, policy_version 982614 (0.0005) [2023-12-26 22:31:35,250][105692] Updated weights for policy 0, policy_version 982408 (0.0006) [2023-12-26 22:31:35,261][105620] Updated weights for policy 1, policy_version 982624 (0.0009) [2023-12-26 22:31:35,875][105620] Updated weights for policy 1, policy_version 982634 (0.0008) [2023-12-26 22:31:35,919][105620] Updated weights for policy 1, policy_version 982644 (0.0005) [2023-12-26 22:31:35,974][105620] Updated weights for policy 1, policy_version 982654 (0.0007) [2023-12-26 22:31:35,976][105692] Updated weights for policy 0, policy_version 982418 (0.0005) [2023-12-26 22:31:36,024][105692] Updated weights for policy 0, policy_version 982428 (0.0005) [2023-12-26 22:31:36,026][105620] Updated weights for policy 1, policy_version 982664 (0.0008) [2023-12-26 22:31:36,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19524.2, 300 sec: 19244.2). Total num frames: 503128064. Throughput: 0: 9643.3, 1: 9755.6. Samples: 503114584. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:31:36,063][104569] Avg episode reward: [(0, '7960.860'), (1, '6930.940')] [2023-12-26 22:31:36,077][105692] Updated weights for policy 0, policy_version 982438 (0.0008) [2023-12-26 22:31:36,744][105692] Updated weights for policy 0, policy_version 982448 (0.0007) [2023-12-26 22:31:36,766][105620] Updated weights for policy 1, policy_version 982674 (0.0009) [2023-12-26 22:31:36,801][105692] Updated weights for policy 0, policy_version 982458 (0.0007) [2023-12-26 22:31:36,823][105620] Updated weights for policy 1, policy_version 982684 (0.0008) [2023-12-26 22:31:36,846][105692] Updated weights for policy 0, policy_version 982468 (0.0006) [2023-12-26 22:31:36,884][105620] Updated weights for policy 1, policy_version 982694 (0.0009) [2023-12-26 22:31:37,488][105692] Updated weights for policy 0, policy_version 982478 (0.0008) [2023-12-26 22:31:37,550][105692] Updated weights for policy 0, policy_version 982488 (0.0009) [2023-12-26 22:31:37,591][105620] Updated weights for policy 1, policy_version 982704 (0.0009) [2023-12-26 22:31:37,612][105692] Updated weights for policy 0, policy_version 982498 (0.0007) [2023-12-26 22:31:37,652][105620] Updated weights for policy 1, policy_version 982714 (0.0007) [2023-12-26 22:31:37,713][105620] Updated weights for policy 1, policy_version 982724 (0.0009) [2023-12-26 22:31:38,160][105692] Updated weights for policy 0, policy_version 982508 (0.0007) [2023-12-26 22:31:38,220][105692] Updated weights for policy 0, policy_version 982518 (0.0008) [2023-12-26 22:31:38,269][105692] Updated weights for policy 0, policy_version 982528 (0.0008) [2023-12-26 22:31:38,585][105620] Updated weights for policy 1, policy_version 982734 (0.0010) [2023-12-26 22:31:38,646][105620] Updated weights for policy 1, policy_version 982744 (0.0010) [2023-12-26 22:31:38,706][105620] Updated weights for policy 1, policy_version 982754 (0.0010) [2023-12-26 22:31:39,050][105692] Updated weights for policy 0, policy_version 982538 (0.0007) [2023-12-26 22:31:39,099][105692] Updated weights for policy 0, policy_version 982548 (0.0007) [2023-12-26 22:31:39,146][105692] Updated weights for policy 0, policy_version 982558 (0.0007) [2023-12-26 22:31:39,204][105692] Updated weights for policy 0, policy_version 982568 (0.0008) [2023-12-26 22:31:39,444][105620] Updated weights for policy 1, policy_version 982764 (0.0009) [2023-12-26 22:31:39,504][105620] Updated weights for policy 1, policy_version 982774 (0.0008) [2023-12-26 22:31:39,565][105620] Updated weights for policy 1, policy_version 982784 (0.0011) [2023-12-26 22:31:40,041][105692] Updated weights for policy 0, policy_version 982578 (0.0008) [2023-12-26 22:31:40,104][105692] Updated weights for policy 0, policy_version 982588 (0.0008) [2023-12-26 22:31:40,161][105692] Updated weights for policy 0, policy_version 982598 (0.0008) [2023-12-26 22:31:40,321][105620] Updated weights for policy 1, policy_version 982794 (0.0011) [2023-12-26 22:31:40,380][105620] Updated weights for policy 1, policy_version 982804 (0.0010) [2023-12-26 22:31:40,440][105620] Updated weights for policy 1, policy_version 982814 (0.0010) [2023-12-26 22:31:40,503][105620] Updated weights for policy 1, policy_version 982824 (0.0010) [2023-12-26 22:31:40,972][105692] Updated weights for policy 0, policy_version 982608 (0.0008) [2023-12-26 22:31:41,025][105692] Updated weights for policy 0, policy_version 982618 (0.0008) [2023-12-26 22:31:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19244.3). Total num frames: 503218176. Throughput: 0: 9698.4, 1: 9802.8. Samples: 503233116. Policy #0 lag: (min: 7.0, avg: 9.8, max: 39.0) [2023-12-26 22:31:41,063][104569] Avg episode reward: [(0, '7890.530'), (1, '7966.134')] [2023-12-26 22:31:41,093][105692] Updated weights for policy 0, policy_version 982628 (0.0008) [2023-12-26 22:31:41,182][105620] Updated weights for policy 1, policy_version 982834 (0.0010) [2023-12-26 22:31:41,251][105620] Updated weights for policy 1, policy_version 982844 (0.0011) [2023-12-26 22:31:41,312][105620] Updated weights for policy 1, policy_version 982854 (0.0011) [2023-12-26 22:31:41,854][105692] Updated weights for policy 0, policy_version 982638 (0.0007) [2023-12-26 22:31:41,923][105692] Updated weights for policy 0, policy_version 982648 (0.0007) [2023-12-26 22:31:41,983][105692] Updated weights for policy 0, policy_version 982658 (0.0008) [2023-12-26 22:31:42,085][105620] Updated weights for policy 1, policy_version 982864 (0.0010) [2023-12-26 22:31:42,148][105620] Updated weights for policy 1, policy_version 982874 (0.0011) [2023-12-26 22:31:42,212][105620] Updated weights for policy 1, policy_version 982884 (0.0011) [2023-12-26 22:31:42,738][105692] Updated weights for policy 0, policy_version 982668 (0.0008) [2023-12-26 22:31:42,784][105692] Updated weights for policy 0, policy_version 982678 (0.0008) [2023-12-26 22:31:42,831][105692] Updated weights for policy 0, policy_version 982688 (0.0008) [2023-12-26 22:31:42,997][105620] Updated weights for policy 1, policy_version 982894 (0.0010) [2023-12-26 22:31:43,053][105620] Updated weights for policy 1, policy_version 982904 (0.0010) [2023-12-26 22:31:43,114][105620] Updated weights for policy 1, policy_version 982914 (0.0005) [2023-12-26 22:31:43,551][105692] Updated weights for policy 0, policy_version 982698 (0.0008) [2023-12-26 22:31:43,597][105692] Updated weights for policy 0, policy_version 982708 (0.0008) [2023-12-26 22:31:43,641][105692] Updated weights for policy 0, policy_version 982718 (0.0008) [2023-12-26 22:31:43,685][105692] Updated weights for policy 0, policy_version 982728 (0.0008) [2023-12-26 22:31:43,785][105620] Updated weights for policy 1, policy_version 982924 (0.0007) [2023-12-26 22:31:43,838][105620] Updated weights for policy 1, policy_version 982934 (0.0010) [2023-12-26 22:31:43,897][105620] Updated weights for policy 1, policy_version 982944 (0.0010) [2023-12-26 22:31:44,359][105692] Updated weights for policy 0, policy_version 982738 (0.0010) [2023-12-26 22:31:44,429][105692] Updated weights for policy 0, policy_version 982748 (0.0005) [2023-12-26 22:31:44,485][105692] Updated weights for policy 0, policy_version 982758 (0.0008) [2023-12-26 22:31:44,664][105620] Updated weights for policy 1, policy_version 982954 (0.0010) [2023-12-26 22:31:44,713][105620] Updated weights for policy 1, policy_version 982964 (0.0010) [2023-12-26 22:31:44,762][105620] Updated weights for policy 1, policy_version 982974 (0.0011) [2023-12-26 22:31:44,829][105620] Updated weights for policy 1, policy_version 982984 (0.0011) [2023-12-26 22:31:45,098][105692] Updated weights for policy 0, policy_version 982768 (0.0007) [2023-12-26 22:31:45,169][105692] Updated weights for policy 0, policy_version 982778 (0.0007) [2023-12-26 22:31:45,227][105692] Updated weights for policy 0, policy_version 982788 (0.0006) [2023-12-26 22:31:45,570][105620] Updated weights for policy 1, policy_version 982994 (0.0011) [2023-12-26 22:31:45,627][105620] Updated weights for policy 1, policy_version 983004 (0.0011) [2023-12-26 22:31:45,685][105620] Updated weights for policy 1, policy_version 983014 (0.0010) [2023-12-26 22:31:45,774][105692] Updated weights for policy 0, policy_version 982798 (0.0009) [2023-12-26 22:31:45,825][105692] Updated weights for policy 0, policy_version 982808 (0.0007) [2023-12-26 22:31:45,881][105692] Updated weights for policy 0, policy_version 982818 (0.0008) [2023-12-26 22:31:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19272.0). Total num frames: 503324672. Throughput: 0: 9674.9, 1: 9777.8. Samples: 503288732. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:31:46,063][104569] Avg episode reward: [(0, '7994.178'), (1, '8159.935')] [2023-12-26 22:31:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000982824_251641856.pth... [2023-12-26 22:31:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000983016_251682816.pth... [2023-12-26 22:31:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000981864_251387904.pth [2023-12-26 22:31:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000981672_251346944.pth [2023-12-26 22:31:46,425][105620] Updated weights for policy 1, policy_version 983024 (0.0010) [2023-12-26 22:31:46,472][105620] Updated weights for policy 1, policy_version 983034 (0.0008) [2023-12-26 22:31:46,523][105620] Updated weights for policy 1, policy_version 983044 (0.0005) [2023-12-26 22:31:46,555][105692] Updated weights for policy 0, policy_version 982828 (0.0009) [2023-12-26 22:31:46,599][105692] Updated weights for policy 0, policy_version 982838 (0.0010) [2023-12-26 22:31:46,647][105692] Updated weights for policy 0, policy_version 982848 (0.0010) [2023-12-26 22:31:47,279][105620] Updated weights for policy 1, policy_version 983054 (0.0008) [2023-12-26 22:31:47,324][105620] Updated weights for policy 1, policy_version 983064 (0.0010) [2023-12-26 22:31:47,340][105692] Updated weights for policy 0, policy_version 982858 (0.0010) [2023-12-26 22:31:47,376][105620] Updated weights for policy 1, policy_version 983074 (0.0010) [2023-12-26 22:31:47,392][105692] Updated weights for policy 0, policy_version 982868 (0.0010) [2023-12-26 22:31:47,449][105692] Updated weights for policy 0, policy_version 982878 (0.0010) [2023-12-26 22:31:47,503][105692] Updated weights for policy 0, policy_version 982888 (0.0010) [2023-12-26 22:31:48,022][105620] Updated weights for policy 1, policy_version 983084 (0.0010) [2023-12-26 22:31:48,081][105620] Updated weights for policy 1, policy_version 983094 (0.0009) [2023-12-26 22:31:48,125][105692] Updated weights for policy 0, policy_version 982898 (0.0011) [2023-12-26 22:31:48,137][105620] Updated weights for policy 1, policy_version 983104 (0.0010) [2023-12-26 22:31:48,187][105692] Updated weights for policy 0, policy_version 982908 (0.0010) [2023-12-26 22:31:48,241][105692] Updated weights for policy 0, policy_version 982918 (0.0010) [2023-12-26 22:31:48,871][105620] Updated weights for policy 1, policy_version 983114 (0.0010) [2023-12-26 22:31:48,920][105620] Updated weights for policy 1, policy_version 983124 (0.0010) [2023-12-26 22:31:48,947][105692] Updated weights for policy 0, policy_version 982928 (0.0010) [2023-12-26 22:31:48,979][105620] Updated weights for policy 1, policy_version 983134 (0.0010) [2023-12-26 22:31:49,007][105692] Updated weights for policy 0, policy_version 982938 (0.0011) [2023-12-26 22:31:49,033][105620] Updated weights for policy 1, policy_version 983144 (0.0010) [2023-12-26 22:31:49,070][105692] Updated weights for policy 0, policy_version 982948 (0.0010) [2023-12-26 22:31:49,780][105620] Updated weights for policy 1, policy_version 983154 (0.0007) [2023-12-26 22:31:49,849][105620] Updated weights for policy 1, policy_version 983164 (0.0008) [2023-12-26 22:31:49,858][105692] Updated weights for policy 0, policy_version 982958 (0.0009) [2023-12-26 22:31:49,906][105620] Updated weights for policy 1, policy_version 983174 (0.0006) [2023-12-26 22:31:49,926][105692] Updated weights for policy 0, policy_version 982968 (0.0009) [2023-12-26 22:31:49,990][105692] Updated weights for policy 0, policy_version 982978 (0.0010) [2023-12-26 22:31:50,559][105620] Updated weights for policy 1, policy_version 983184 (0.0008) [2023-12-26 22:31:50,621][105620] Updated weights for policy 1, policy_version 983194 (0.0008) [2023-12-26 22:31:50,674][105620] Updated weights for policy 1, policy_version 983204 (0.0009) [2023-12-26 22:31:50,763][105692] Updated weights for policy 0, policy_version 982988 (0.0010) [2023-12-26 22:31:50,830][105692] Updated weights for policy 0, policy_version 982998 (0.0010) [2023-12-26 22:31:50,892][105692] Updated weights for policy 0, policy_version 983008 (0.0009) [2023-12-26 22:31:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19244.3). Total num frames: 503422976. Throughput: 0: 9863.0, 1: 9774.1. Samples: 503409756. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:31:51,062][104569] Avg episode reward: [(0, '8133.960'), (1, '9112.731')] [2023-12-26 22:31:51,422][105620] Updated weights for policy 1, policy_version 983214 (0.0008) [2023-12-26 22:31:51,481][105620] Updated weights for policy 1, policy_version 983224 (0.0008) [2023-12-26 22:31:51,540][105620] Updated weights for policy 1, policy_version 983234 (0.0005) [2023-12-26 22:31:51,654][105692] Updated weights for policy 0, policy_version 983018 (0.0007) [2023-12-26 22:31:51,716][105692] Updated weights for policy 0, policy_version 983028 (0.0007) [2023-12-26 22:31:51,780][105692] Updated weights for policy 0, policy_version 983038 (0.0008) [2023-12-26 22:31:51,846][105692] Updated weights for policy 0, policy_version 983048 (0.0008) [2023-12-26 22:31:52,270][105620] Updated weights for policy 1, policy_version 983244 (0.0010) [2023-12-26 22:31:52,334][105620] Updated weights for policy 1, policy_version 983254 (0.0007) [2023-12-26 22:31:52,390][105620] Updated weights for policy 1, policy_version 983264 (0.0008) [2023-12-26 22:31:52,568][105692] Updated weights for policy 0, policy_version 983058 (0.0009) [2023-12-26 22:31:52,615][105692] Updated weights for policy 0, policy_version 983068 (0.0005) [2023-12-26 22:31:52,675][105692] Updated weights for policy 0, policy_version 983078 (0.0006) [2023-12-26 22:31:53,168][105620] Updated weights for policy 1, policy_version 983274 (0.0009) [2023-12-26 22:31:53,231][105620] Updated weights for policy 1, policy_version 983284 (0.0008) [2023-12-26 22:31:53,286][105620] Updated weights for policy 1, policy_version 983294 (0.0008) [2023-12-26 22:31:53,344][105620] Updated weights for policy 1, policy_version 983304 (0.0010) [2023-12-26 22:31:53,366][105692] Updated weights for policy 0, policy_version 983088 (0.0009) [2023-12-26 22:31:53,430][105692] Updated weights for policy 0, policy_version 983098 (0.0010) [2023-12-26 22:31:53,490][105692] Updated weights for policy 0, policy_version 983108 (0.0007) [2023-12-26 22:31:54,029][105620] Updated weights for policy 1, policy_version 983314 (0.0005) [2023-12-26 22:31:54,033][105692] Updated weights for policy 0, policy_version 983118 (0.0007) [2023-12-26 22:31:54,076][105620] Updated weights for policy 1, policy_version 983324 (0.0006) [2023-12-26 22:31:54,091][105692] Updated weights for policy 0, policy_version 983128 (0.0009) [2023-12-26 22:31:54,122][105620] Updated weights for policy 1, policy_version 983334 (0.0006) [2023-12-26 22:31:54,148][105692] Updated weights for policy 0, policy_version 983138 (0.0009) [2023-12-26 22:31:54,820][105620] Updated weights for policy 1, policy_version 983344 (0.0006) [2023-12-26 22:31:54,870][105620] Updated weights for policy 1, policy_version 983354 (0.0006) [2023-12-26 22:31:54,889][105692] Updated weights for policy 0, policy_version 983148 (0.0009) [2023-12-26 22:31:54,918][105620] Updated weights for policy 1, policy_version 983364 (0.0010) [2023-12-26 22:31:54,957][105692] Updated weights for policy 0, policy_version 983158 (0.0011) [2023-12-26 22:31:54,984][105585] KL-divergence is very high: 140.6287 [2023-12-26 22:31:55,012][105692] Updated weights for policy 0, policy_version 983168 (0.0007) [2023-12-26 22:31:55,027][105585] KL-divergence is very high: 137.0774 [2023-12-26 22:31:55,538][105620] Updated weights for policy 1, policy_version 983374 (0.0008) [2023-12-26 22:31:55,596][105620] Updated weights for policy 1, policy_version 983384 (0.0006) [2023-12-26 22:31:55,650][105692] Updated weights for policy 0, policy_version 983178 (0.0006) [2023-12-26 22:31:55,651][105620] Updated weights for policy 1, policy_version 983394 (0.0006) [2023-12-26 22:31:55,708][105692] Updated weights for policy 0, policy_version 983188 (0.0010) [2023-12-26 22:31:55,727][105585] KL-divergence is very high: 138.4654 [2023-12-26 22:31:55,734][105585] KL-divergence is very high: 128.7555 [2023-12-26 22:31:55,768][105692] Updated weights for policy 0, policy_version 983198 (0.0011) [2023-12-26 22:31:55,777][105585] KL-divergence is very high: 146.5311 [2023-12-26 22:31:55,783][105585] KL-divergence is very high: 133.0333 [2023-12-26 22:31:55,821][105692] Updated weights for policy 0, policy_version 983208 (0.0011) [2023-12-26 22:31:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19272.0). Total num frames: 503521280. Throughput: 0: 9820.6, 1: 9869.1. Samples: 503528696. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:31:56,062][104569] Avg episode reward: [(0, '8191.101'), (1, '8828.406')] [2023-12-26 22:31:56,232][105620] Updated weights for policy 1, policy_version 983404 (0.0006) [2023-12-26 22:31:56,300][105620] Updated weights for policy 1, policy_version 983414 (0.0006) [2023-12-26 22:31:56,361][105620] Updated weights for policy 1, policy_version 983424 (0.0005) [2023-12-26 22:31:56,591][105692] Updated weights for policy 0, policy_version 983218 (0.0007) [2023-12-26 22:31:56,651][105692] Updated weights for policy 0, policy_version 983228 (0.0005) [2023-12-26 22:31:56,696][105692] Updated weights for policy 0, policy_version 983238 (0.0005) [2023-12-26 22:31:56,881][105620] Updated weights for policy 1, policy_version 983434 (0.0005) [2023-12-26 22:31:56,940][105620] Updated weights for policy 1, policy_version 983444 (0.0005) [2023-12-26 22:31:57,002][105620] Updated weights for policy 1, policy_version 983454 (0.0005) [2023-12-26 22:31:57,062][105620] Updated weights for policy 1, policy_version 983464 (0.0005) [2023-12-26 22:31:57,306][105692] Updated weights for policy 0, policy_version 983248 (0.0006) [2023-12-26 22:31:57,358][105692] Updated weights for policy 0, policy_version 983258 (0.0010) [2023-12-26 22:31:57,422][105692] Updated weights for policy 0, policy_version 983268 (0.0010) [2023-12-26 22:31:57,618][105620] Updated weights for policy 1, policy_version 983474 (0.0008) [2023-12-26 22:31:57,679][105620] Updated weights for policy 1, policy_version 983484 (0.0010) [2023-12-26 22:31:57,733][105620] Updated weights for policy 1, policy_version 983494 (0.0010) [2023-12-26 22:31:58,083][105692] Updated weights for policy 0, policy_version 983278 (0.0007) [2023-12-26 22:31:58,138][105692] Updated weights for policy 0, policy_version 983288 (0.0005) [2023-12-26 22:31:58,202][105692] Updated weights for policy 0, policy_version 983298 (0.0011) [2023-12-26 22:31:58,420][105620] Updated weights for policy 1, policy_version 983504 (0.0009) [2023-12-26 22:31:58,480][105620] Updated weights for policy 1, policy_version 983514 (0.0011) [2023-12-26 22:31:58,540][105620] Updated weights for policy 1, policy_version 983524 (0.0011) [2023-12-26 22:31:59,017][105692] Updated weights for policy 0, policy_version 983308 (0.0009) [2023-12-26 22:31:59,083][105692] Updated weights for policy 0, policy_version 983318 (0.0007) [2023-12-26 22:31:59,146][105692] Updated weights for policy 0, policy_version 983328 (0.0007) [2023-12-26 22:31:59,367][105620] Updated weights for policy 1, policy_version 983534 (0.0009) [2023-12-26 22:31:59,422][105620] Updated weights for policy 1, policy_version 983544 (0.0010) [2023-12-26 22:31:59,487][105620] Updated weights for policy 1, policy_version 983554 (0.0010) [2023-12-26 22:31:59,866][105692] Updated weights for policy 0, policy_version 983338 (0.0008) [2023-12-26 22:31:59,928][105692] Updated weights for policy 0, policy_version 983348 (0.0007) [2023-12-26 22:31:59,993][105692] Updated weights for policy 0, policy_version 983358 (0.0007) [2023-12-26 22:32:00,058][105692] Updated weights for policy 0, policy_version 983368 (0.0008) [2023-12-26 22:32:00,245][105620] Updated weights for policy 1, policy_version 983564 (0.0008) [2023-12-26 22:32:00,305][105620] Updated weights for policy 1, policy_version 983574 (0.0006) [2023-12-26 22:32:00,351][105620] Updated weights for policy 1, policy_version 983584 (0.0009) [2023-12-26 22:32:00,851][105692] Updated weights for policy 0, policy_version 983378 (0.0011) [2023-12-26 22:32:00,908][105692] Updated weights for policy 0, policy_version 983388 (0.0010) [2023-12-26 22:32:00,962][105692] Updated weights for policy 0, policy_version 983398 (0.0008) [2023-12-26 22:32:00,965][105620] Updated weights for policy 1, policy_version 983594 (0.0008) [2023-12-26 22:32:01,029][105620] Updated weights for policy 1, policy_version 983604 (0.0008) [2023-12-26 22:32:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19272.0). Total num frames: 503619584. Throughput: 0: 9866.6, 1: 9963.6. Samples: 503591472. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:32:01,062][104569] Avg episode reward: [(0, '8015.530'), (1, '8482.785')] [2023-12-26 22:32:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000983400_251789312.pth... [2023-12-26 22:32:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000982248_251494400.pth [2023-12-26 22:32:01,093][105620] Updated weights for policy 1, policy_version 983614 (0.0008) [2023-12-26 22:32:01,161][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000983624_251838464.pth... [2023-12-26 22:32:01,163][105620] Updated weights for policy 1, policy_version 983624 (0.0008) [2023-12-26 22:32:01,166][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000982472_251543552.pth [2023-12-26 22:32:01,729][105692] Updated weights for policy 0, policy_version 983408 (0.0009) [2023-12-26 22:32:01,789][105692] Updated weights for policy 0, policy_version 983418 (0.0010) [2023-12-26 22:32:01,847][105692] Updated weights for policy 0, policy_version 983428 (0.0009) [2023-12-26 22:32:01,864][105620] Updated weights for policy 1, policy_version 983634 (0.0006) [2023-12-26 22:32:01,922][105620] Updated weights for policy 1, policy_version 983644 (0.0005) [2023-12-26 22:32:01,979][105620] Updated weights for policy 1, policy_version 983654 (0.0006) [2023-12-26 22:32:02,594][105620] Updated weights for policy 1, policy_version 983664 (0.0005) [2023-12-26 22:32:02,656][105620] Updated weights for policy 1, policy_version 983674 (0.0008) [2023-12-26 22:32:02,705][105620] Updated weights for policy 1, policy_version 983684 (0.0008) [2023-12-26 22:32:02,728][105692] Updated weights for policy 0, policy_version 983438 (0.0008) [2023-12-26 22:32:02,782][105692] Updated weights for policy 0, policy_version 983448 (0.0006) [2023-12-26 22:32:02,845][105692] Updated weights for policy 0, policy_version 983458 (0.0007) [2023-12-26 22:32:03,441][105620] Updated weights for policy 1, policy_version 983694 (0.0009) [2023-12-26 22:32:03,458][105692] Updated weights for policy 0, policy_version 983468 (0.0009) [2023-12-26 22:32:03,489][105620] Updated weights for policy 1, policy_version 983704 (0.0010) [2023-12-26 22:32:03,502][105692] Updated weights for policy 0, policy_version 983478 (0.0010) [2023-12-26 22:32:03,535][105620] Updated weights for policy 1, policy_version 983714 (0.0006) [2023-12-26 22:32:03,549][105692] Updated weights for policy 0, policy_version 983488 (0.0010) [2023-12-26 22:32:04,166][105620] Updated weights for policy 1, policy_version 983724 (0.0006) [2023-12-26 22:32:04,229][105620] Updated weights for policy 1, policy_version 983734 (0.0011) [2023-12-26 22:32:04,298][105620] Updated weights for policy 1, policy_version 983744 (0.0011) [2023-12-26 22:32:04,322][105692] Updated weights for policy 0, policy_version 983498 (0.0009) [2023-12-26 22:32:04,381][105692] Updated weights for policy 0, policy_version 983508 (0.0007) [2023-12-26 22:32:04,436][105692] Updated weights for policy 0, policy_version 983518 (0.0008) [2023-12-26 22:32:04,492][105692] Updated weights for policy 0, policy_version 983528 (0.0009) [2023-12-26 22:32:04,971][105620] Updated weights for policy 1, policy_version 983754 (0.0008) [2023-12-26 22:32:05,036][105620] Updated weights for policy 1, policy_version 983764 (0.0005) [2023-12-26 22:32:05,082][105620] Updated weights for policy 1, policy_version 983774 (0.0005) [2023-12-26 22:32:05,150][105620] Updated weights for policy 1, policy_version 983784 (0.0008) [2023-12-26 22:32:05,166][105692] Updated weights for policy 0, policy_version 983538 (0.0007) [2023-12-26 22:32:05,220][105692] Updated weights for policy 0, policy_version 983548 (0.0009) [2023-12-26 22:32:05,277][105692] Updated weights for policy 0, policy_version 983559 (0.0010) [2023-12-26 22:32:05,705][105620] Updated weights for policy 1, policy_version 983794 (0.0009) [2023-12-26 22:32:05,752][105620] Updated weights for policy 1, policy_version 983804 (0.0008) [2023-12-26 22:32:05,809][105620] Updated weights for policy 1, policy_version 983814 (0.0008) [2023-12-26 22:32:06,030][105692] Updated weights for policy 0, policy_version 983569 (0.0006) [2023-12-26 22:32:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19272.0). Total num frames: 503717888. Throughput: 0: 9777.9, 1: 9928.7. Samples: 503706948. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:32:06,063][104569] Avg episode reward: [(0, '8462.804'), (1, '8861.681')] [2023-12-26 22:32:06,079][105692] Updated weights for policy 0, policy_version 983579 (0.0005) [2023-12-26 22:32:06,148][105692] Updated weights for policy 0, policy_version 983589 (0.0006) [2023-12-26 22:32:06,585][105620] Updated weights for policy 1, policy_version 983824 (0.0010) [2023-12-26 22:32:06,649][105620] Updated weights for policy 1, policy_version 983834 (0.0011) [2023-12-26 22:32:06,719][105620] Updated weights for policy 1, policy_version 983844 (0.0011) [2023-12-26 22:32:06,801][105692] Updated weights for policy 0, policy_version 983599 (0.0007) [2023-12-26 22:32:06,856][105692] Updated weights for policy 0, policy_version 983609 (0.0008) [2023-12-26 22:32:06,916][105692] Updated weights for policy 0, policy_version 983619 (0.0007) [2023-12-26 22:32:07,363][105620] Updated weights for policy 1, policy_version 983854 (0.0008) [2023-12-26 22:32:07,419][105620] Updated weights for policy 1, policy_version 983864 (0.0006) [2023-12-26 22:32:07,470][105620] Updated weights for policy 1, policy_version 983874 (0.0010) [2023-12-26 22:32:07,503][105692] Updated weights for policy 0, policy_version 983629 (0.0007) [2023-12-26 22:32:07,551][105692] Updated weights for policy 0, policy_version 983639 (0.0007) [2023-12-26 22:32:07,610][105692] Updated weights for policy 0, policy_version 983649 (0.0008) [2023-12-26 22:32:08,180][105620] Updated weights for policy 1, policy_version 983884 (0.0011) [2023-12-26 22:32:08,245][105620] Updated weights for policy 1, policy_version 983894 (0.0011) [2023-12-26 22:32:08,301][105692] Updated weights for policy 0, policy_version 983659 (0.0009) [2023-12-26 22:32:08,303][105620] Updated weights for policy 1, policy_version 983904 (0.0010) [2023-12-26 22:32:08,360][105692] Updated weights for policy 0, policy_version 983669 (0.0008) [2023-12-26 22:32:08,423][105692] Updated weights for policy 0, policy_version 983679 (0.0008) [2023-12-26 22:32:09,029][105692] Updated weights for policy 0, policy_version 983689 (0.0007) [2023-12-26 22:32:09,047][105620] Updated weights for policy 1, policy_version 983914 (0.0011) [2023-12-26 22:32:09,094][105692] Updated weights for policy 0, policy_version 983699 (0.0006) [2023-12-26 22:32:09,107][105620] Updated weights for policy 1, policy_version 983924 (0.0010) [2023-12-26 22:32:09,165][105692] Updated weights for policy 0, policy_version 983709 (0.0005) [2023-12-26 22:32:09,175][105620] Updated weights for policy 1, policy_version 983934 (0.0009) [2023-12-26 22:32:09,234][105692] Updated weights for policy 0, policy_version 983719 (0.0006) [2023-12-26 22:32:09,240][105620] Updated weights for policy 1, policy_version 983944 (0.0008) [2023-12-26 22:32:09,854][105692] Updated weights for policy 0, policy_version 983729 (0.0009) [2023-12-26 22:32:09,914][105692] Updated weights for policy 0, policy_version 983739 (0.0009) [2023-12-26 22:32:09,951][105620] Updated weights for policy 1, policy_version 983954 (0.0008) [2023-12-26 22:32:09,980][105692] Updated weights for policy 0, policy_version 983749 (0.0009) [2023-12-26 22:32:10,018][105620] Updated weights for policy 1, policy_version 983964 (0.0007) [2023-12-26 22:32:10,081][105620] Updated weights for policy 1, policy_version 983974 (0.0009) [2023-12-26 22:32:10,742][105692] Updated weights for policy 0, policy_version 983759 (0.0010) [2023-12-26 22:32:10,799][105620] Updated weights for policy 1, policy_version 983984 (0.0007) [2023-12-26 22:32:10,801][105692] Updated weights for policy 0, policy_version 983769 (0.0006) [2023-12-26 22:32:10,853][105620] Updated weights for policy 1, policy_version 983994 (0.0005) [2023-12-26 22:32:10,854][105692] Updated weights for policy 0, policy_version 983779 (0.0008) [2023-12-26 22:32:10,906][105620] Updated weights for policy 1, policy_version 984004 (0.0005) [2023-12-26 22:32:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19327.6). Total num frames: 503824384. Throughput: 0: 9928.4, 1: 9918.0. Samples: 503828040. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:32:11,062][104569] Avg episode reward: [(0, '8561.697'), (1, '9031.469')] [2023-12-26 22:32:11,585][105620] Updated weights for policy 1, policy_version 984014 (0.0007) [2023-12-26 22:32:11,624][105692] Updated weights for policy 0, policy_version 983789 (0.0008) [2023-12-26 22:32:11,653][105620] Updated weights for policy 1, policy_version 984024 (0.0007) [2023-12-26 22:32:11,684][105692] Updated weights for policy 0, policy_version 983799 (0.0007) [2023-12-26 22:32:11,720][105620] Updated weights for policy 1, policy_version 984034 (0.0008) [2023-12-26 22:32:11,751][105692] Updated weights for policy 0, policy_version 983809 (0.0011) [2023-12-26 22:32:12,388][105692] Updated weights for policy 0, policy_version 983819 (0.0009) [2023-12-26 22:32:12,450][105692] Updated weights for policy 0, policy_version 983829 (0.0009) [2023-12-26 22:32:12,513][105692] Updated weights for policy 0, policy_version 983839 (0.0010) [2023-12-26 22:32:12,515][105620] Updated weights for policy 1, policy_version 984044 (0.0008) [2023-12-26 22:32:12,566][105620] Updated weights for policy 1, policy_version 984054 (0.0008) [2023-12-26 22:32:12,619][105620] Updated weights for policy 1, policy_version 984064 (0.0010) [2023-12-26 22:32:13,197][105692] Updated weights for policy 0, policy_version 983849 (0.0007) [2023-12-26 22:32:13,255][105692] Updated weights for policy 0, policy_version 983859 (0.0009) [2023-12-26 22:32:13,309][105692] Updated weights for policy 0, policy_version 983869 (0.0008) [2023-12-26 22:32:13,363][105692] Updated weights for policy 0, policy_version 983879 (0.0009) [2023-12-26 22:32:13,395][105620] Updated weights for policy 1, policy_version 984075 (0.0010) [2023-12-26 22:32:13,441][105620] Updated weights for policy 1, policy_version 984085 (0.0008) [2023-12-26 22:32:13,495][105620] Updated weights for policy 1, policy_version 984095 (0.0009) [2023-12-26 22:32:14,127][105692] Updated weights for policy 0, policy_version 983889 (0.0010) [2023-12-26 22:32:14,181][105692] Updated weights for policy 0, policy_version 983900 (0.0010) [2023-12-26 22:32:14,216][105620] Updated weights for policy 1, policy_version 984105 (0.0008) [2023-12-26 22:32:14,234][105692] Updated weights for policy 0, policy_version 983910 (0.0009) [2023-12-26 22:32:14,273][105620] Updated weights for policy 1, policy_version 984115 (0.0007) [2023-12-26 22:32:14,335][105620] Updated weights for policy 1, policy_version 984125 (0.0009) [2023-12-26 22:32:14,398][105620] Updated weights for policy 1, policy_version 984135 (0.0008) [2023-12-26 22:32:15,036][105692] Updated weights for policy 0, policy_version 983920 (0.0008) [2023-12-26 22:32:15,107][105692] Updated weights for policy 0, policy_version 983930 (0.0008) [2023-12-26 22:32:15,132][105620] Updated weights for policy 1, policy_version 984145 (0.0006) [2023-12-26 22:32:15,169][105692] Updated weights for policy 0, policy_version 983940 (0.0009) [2023-12-26 22:32:15,192][105620] Updated weights for policy 1, policy_version 984155 (0.0007) [2023-12-26 22:32:15,252][105620] Updated weights for policy 1, policy_version 984165 (0.0007) [2023-12-26 22:32:15,893][105692] Updated weights for policy 0, policy_version 983950 (0.0008) [2023-12-26 22:32:15,959][105692] Updated weights for policy 0, policy_version 983960 (0.0009) [2023-12-26 22:32:15,969][105620] Updated weights for policy 1, policy_version 984175 (0.0005) [2023-12-26 22:32:16,012][105692] Updated weights for policy 0, policy_version 983970 (0.0009) [2023-12-26 22:32:16,029][105620] Updated weights for policy 1, policy_version 984185 (0.0005) [2023-12-26 22:32:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.7, 300 sec: 19299.8). Total num frames: 503914496. Throughput: 0: 9917.7, 1: 9847.0. Samples: 503884488. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:32:16,063][104569] Avg episode reward: [(0, '7877.345'), (1, '9088.685')] [2023-12-26 22:32:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000983976_251936768.pth... [2023-12-26 22:32:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000982824_251641856.pth [2023-12-26 22:32:16,082][105620] Updated weights for policy 1, policy_version 984195 (0.0006) [2023-12-26 22:32:16,112][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000984200_251985920.pth... [2023-12-26 22:32:16,115][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000983016_251682816.pth [2023-12-26 22:32:16,659][105620] Updated weights for policy 1, policy_version 984205 (0.0007) [2023-12-26 22:32:16,710][105620] Updated weights for policy 1, policy_version 984215 (0.0008) [2023-12-26 22:32:16,767][105620] Updated weights for policy 1, policy_version 984225 (0.0005) [2023-12-26 22:32:16,821][105692] Updated weights for policy 0, policy_version 983980 (0.0007) [2023-12-26 22:32:16,879][105692] Updated weights for policy 0, policy_version 983990 (0.0005) [2023-12-26 22:32:16,935][105692] Updated weights for policy 0, policy_version 984000 (0.0007) [2023-12-26 22:32:17,425][105620] Updated weights for policy 1, policy_version 984235 (0.0005) [2023-12-26 22:32:17,478][105692] Updated weights for policy 0, policy_version 984010 (0.0005) [2023-12-26 22:32:17,487][105620] Updated weights for policy 1, policy_version 984245 (0.0005) [2023-12-26 22:32:17,533][105692] Updated weights for policy 0, policy_version 984020 (0.0005) [2023-12-26 22:32:17,540][105620] Updated weights for policy 1, policy_version 984255 (0.0005) [2023-12-26 22:32:17,582][105692] Updated weights for policy 0, policy_version 984030 (0.0005) [2023-12-26 22:32:17,645][105692] Updated weights for policy 0, policy_version 984040 (0.0005) [2023-12-26 22:32:18,226][105692] Updated weights for policy 0, policy_version 984050 (0.0005) [2023-12-26 22:32:18,262][105620] Updated weights for policy 1, policy_version 984265 (0.0006) [2023-12-26 22:32:18,281][105692] Updated weights for policy 0, policy_version 984060 (0.0005) [2023-12-26 22:32:18,311][105620] Updated weights for policy 1, policy_version 984275 (0.0009) [2023-12-26 22:32:18,334][105692] Updated weights for policy 0, policy_version 984070 (0.0006) [2023-12-26 22:32:18,364][105620] Updated weights for policy 1, policy_version 984285 (0.0008) [2023-12-26 22:32:18,420][105620] Updated weights for policy 1, policy_version 984295 (0.0009) [2023-12-26 22:32:19,023][105692] Updated weights for policy 0, policy_version 984080 (0.0006) [2023-12-26 22:32:19,078][105692] Updated weights for policy 0, policy_version 984090 (0.0006) [2023-12-26 22:32:19,130][105692] Updated weights for policy 0, policy_version 984100 (0.0005) [2023-12-26 22:32:19,163][105620] Updated weights for policy 1, policy_version 984305 (0.0010) [2023-12-26 22:32:19,216][105620] Updated weights for policy 1, policy_version 984315 (0.0010) [2023-12-26 22:32:19,277][105620] Updated weights for policy 1, policy_version 984325 (0.0008) [2023-12-26 22:32:19,791][105692] Updated weights for policy 0, policy_version 984110 (0.0006) [2023-12-26 22:32:19,861][105692] Updated weights for policy 0, policy_version 984120 (0.0011) [2023-12-26 22:32:19,922][105692] Updated weights for policy 0, policy_version 984130 (0.0011) [2023-12-26 22:32:20,091][105620] Updated weights for policy 1, policy_version 984335 (0.0009) [2023-12-26 22:32:20,148][105620] Updated weights for policy 1, policy_version 984345 (0.0009) [2023-12-26 22:32:20,207][105620] Updated weights for policy 1, policy_version 984355 (0.0009) [2023-12-26 22:32:20,568][105692] Updated weights for policy 0, policy_version 984140 (0.0009) [2023-12-26 22:32:20,632][105692] Updated weights for policy 0, policy_version 984150 (0.0008) [2023-12-26 22:32:20,695][105692] Updated weights for policy 0, policy_version 984160 (0.0009) [2023-12-26 22:32:21,053][105620] Updated weights for policy 1, policy_version 984365 (0.0008) [2023-12-26 22:32:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19797.3, 300 sec: 19299.8). Total num frames: 504012800. Throughput: 0: 9923.5, 1: 9852.8. Samples: 504004516. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:32:21,062][104569] Avg episode reward: [(0, '7885.223'), (1, '9264.687')] [2023-12-26 22:32:21,111][105620] Updated weights for policy 1, policy_version 984375 (0.0008) [2023-12-26 22:32:21,178][105620] Updated weights for policy 1, policy_version 984385 (0.0008) [2023-12-26 22:32:21,471][105692] Updated weights for policy 0, policy_version 984170 (0.0008) [2023-12-26 22:32:21,539][105585] KL-divergence is very high: 118.5686 [2023-12-26 22:32:21,540][105692] Updated weights for policy 0, policy_version 984180 (0.0010) [2023-12-26 22:32:21,591][105585] KL-divergence is very high: 198.0309 [2023-12-26 22:32:21,603][105692] Updated weights for policy 0, policy_version 984190 (0.0008) [2023-12-26 22:32:21,648][105585] KL-divergence is very high: 192.0896 [2023-12-26 22:32:21,675][105692] Updated weights for policy 0, policy_version 984200 (0.0008) [2023-12-26 22:32:21,915][105620] Updated weights for policy 1, policy_version 984395 (0.0009) [2023-12-26 22:32:21,979][105620] Updated weights for policy 1, policy_version 984405 (0.0011) [2023-12-26 22:32:22,039][105620] Updated weights for policy 1, policy_version 984415 (0.0011) [2023-12-26 22:32:22,378][105692] Updated weights for policy 0, policy_version 984210 (0.0009) [2023-12-26 22:32:22,431][105692] Updated weights for policy 0, policy_version 984220 (0.0009) [2023-12-26 22:32:22,485][105692] Updated weights for policy 0, policy_version 984230 (0.0010) [2023-12-26 22:32:22,707][105620] Updated weights for policy 1, policy_version 984425 (0.0010) [2023-12-26 22:32:22,767][105620] Updated weights for policy 1, policy_version 984435 (0.0006) [2023-12-26 22:32:22,816][105620] Updated weights for policy 1, policy_version 984445 (0.0011) [2023-12-26 22:32:22,872][105620] Updated weights for policy 1, policy_version 984455 (0.0010) [2023-12-26 22:32:23,247][105692] Updated weights for policy 0, policy_version 984240 (0.0007) [2023-12-26 22:32:23,307][105692] Updated weights for policy 0, policy_version 984250 (0.0008) [2023-12-26 22:32:23,351][105692] Updated weights for policy 0, policy_version 984260 (0.0008) [2023-12-26 22:32:23,475][105620] Updated weights for policy 1, policy_version 984465 (0.0007) [2023-12-26 22:32:23,525][105620] Updated weights for policy 1, policy_version 984475 (0.0007) [2023-12-26 22:32:23,576][105620] Updated weights for policy 1, policy_version 984485 (0.0010) [2023-12-26 22:32:24,056][105692] Updated weights for policy 0, policy_version 984270 (0.0007) [2023-12-26 22:32:24,102][105692] Updated weights for policy 0, policy_version 984280 (0.0008) [2023-12-26 22:32:24,160][105692] Updated weights for policy 0, policy_version 984290 (0.0008) [2023-12-26 22:32:24,250][105620] Updated weights for policy 1, policy_version 984495 (0.0007) [2023-12-26 22:32:24,297][105620] Updated weights for policy 1, policy_version 984505 (0.0005) [2023-12-26 22:32:24,357][105620] Updated weights for policy 1, policy_version 984515 (0.0005) [2023-12-26 22:32:24,954][105692] Updated weights for policy 0, policy_version 984300 (0.0008) [2023-12-26 22:32:24,994][105620] Updated weights for policy 1, policy_version 984525 (0.0006) [2023-12-26 22:32:25,017][105692] Updated weights for policy 0, policy_version 984310 (0.0008) [2023-12-26 22:32:25,051][105620] Updated weights for policy 1, policy_version 984535 (0.0007) [2023-12-26 22:32:25,080][105692] Updated weights for policy 0, policy_version 984320 (0.0008) [2023-12-26 22:32:25,098][105620] Updated weights for policy 1, policy_version 984545 (0.0006) [2023-12-26 22:32:25,696][105620] Updated weights for policy 1, policy_version 984555 (0.0007) [2023-12-26 22:32:25,763][105620] Updated weights for policy 1, policy_version 984565 (0.0008) [2023-12-26 22:32:25,831][105620] Updated weights for policy 1, policy_version 984575 (0.0008) [2023-12-26 22:32:25,881][105692] Updated weights for policy 0, policy_version 984330 (0.0009) [2023-12-26 22:32:25,940][105692] Updated weights for policy 0, policy_version 984340 (0.0008) [2023-12-26 22:32:26,002][105692] Updated weights for policy 0, policy_version 984350 (0.0009) [2023-12-26 22:32:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19299.8). Total num frames: 504111104. Throughput: 0: 9827.5, 1: 9909.3. Samples: 504121272. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:32:26,063][104569] Avg episode reward: [(0, '8025.845'), (1, '9264.901')] [2023-12-26 22:32:26,065][105692] Updated weights for policy 0, policy_version 984360 (0.0008) [2023-12-26 22:32:26,506][105620] Updated weights for policy 1, policy_version 984585 (0.0007) [2023-12-26 22:32:26,573][105620] Updated weights for policy 1, policy_version 984595 (0.0005) [2023-12-26 22:32:26,631][105620] Updated weights for policy 1, policy_version 984605 (0.0005) [2023-12-26 22:32:26,684][105620] Updated weights for policy 1, policy_version 984615 (0.0005) [2023-12-26 22:32:26,799][105692] Updated weights for policy 0, policy_version 984370 (0.0010) [2023-12-26 22:32:26,853][105692] Updated weights for policy 0, policy_version 984380 (0.0010) [2023-12-26 22:32:26,910][105692] Updated weights for policy 0, policy_version 984390 (0.0010) [2023-12-26 22:32:27,202][105620] Updated weights for policy 1, policy_version 984625 (0.0008) [2023-12-26 22:32:27,264][105620] Updated weights for policy 1, policy_version 984635 (0.0007) [2023-12-26 22:32:27,312][105620] Updated weights for policy 1, policy_version 984645 (0.0008) [2023-12-26 22:32:27,551][105692] Updated weights for policy 0, policy_version 984400 (0.0006) [2023-12-26 22:32:27,599][105692] Updated weights for policy 0, policy_version 984410 (0.0005) [2023-12-26 22:32:27,645][105692] Updated weights for policy 0, policy_version 984420 (0.0005) [2023-12-26 22:32:27,965][105620] Updated weights for policy 1, policy_version 984655 (0.0005) [2023-12-26 22:32:28,019][105620] Updated weights for policy 1, policy_version 984665 (0.0005) [2023-12-26 22:32:28,073][105620] Updated weights for policy 1, policy_version 984675 (0.0005) [2023-12-26 22:32:28,221][105692] Updated weights for policy 0, policy_version 984430 (0.0006) [2023-12-26 22:32:28,277][105692] Updated weights for policy 0, policy_version 984440 (0.0005) [2023-12-26 22:32:28,330][105692] Updated weights for policy 0, policy_version 984450 (0.0007) [2023-12-26 22:32:28,703][105620] Updated weights for policy 1, policy_version 984685 (0.0007) [2023-12-26 22:32:28,749][105620] Updated weights for policy 1, policy_version 984695 (0.0009) [2023-12-26 22:32:28,796][105620] Updated weights for policy 1, policy_version 984705 (0.0009) [2023-12-26 22:32:29,042][105692] Updated weights for policy 0, policy_version 984460 (0.0010) [2023-12-26 22:32:29,106][105692] Updated weights for policy 0, policy_version 984470 (0.0006) [2023-12-26 22:32:29,160][105692] Updated weights for policy 0, policy_version 984480 (0.0005) [2023-12-26 22:32:29,688][105620] Updated weights for policy 1, policy_version 984715 (0.0008) [2023-12-26 22:32:29,703][105692] Updated weights for policy 0, policy_version 984490 (0.0005) [2023-12-26 22:32:29,736][105620] Updated weights for policy 1, policy_version 984725 (0.0010) [2023-12-26 22:32:29,753][105692] Updated weights for policy 0, policy_version 984500 (0.0006) [2023-12-26 22:32:29,791][105620] Updated weights for policy 1, policy_version 984735 (0.0009) [2023-12-26 22:32:29,807][105692] Updated weights for policy 0, policy_version 984510 (0.0005) [2023-12-26 22:32:29,865][105692] Updated weights for policy 0, policy_version 984520 (0.0008) [2023-12-26 22:32:30,485][105620] Updated weights for policy 1, policy_version 984745 (0.0008) [2023-12-26 22:32:30,531][105620] Updated weights for policy 1, policy_version 984755 (0.0008) [2023-12-26 22:32:30,583][105620] Updated weights for policy 1, policy_version 984765 (0.0008) [2023-12-26 22:32:30,625][105692] Updated weights for policy 0, policy_version 984530 (0.0007) [2023-12-26 22:32:30,638][105620] Updated weights for policy 1, policy_version 984775 (0.0007) [2023-12-26 22:32:30,676][105692] Updated weights for policy 0, policy_version 984540 (0.0009) [2023-12-26 22:32:30,729][105692] Updated weights for policy 0, policy_version 984550 (0.0009) [2023-12-26 22:32:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19933.9, 300 sec: 19327.6). Total num frames: 504217600. Throughput: 0: 9906.3, 1: 10014.1. Samples: 504185152. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:32:31,063][104569] Avg episode reward: [(0, '8275.897'), (1, '9264.707')] [2023-12-26 22:32:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000984552_252084224.pth... [2023-12-26 22:32:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000984776_252133376.pth... [2023-12-26 22:32:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000983400_251789312.pth [2023-12-26 22:32:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000983624_251838464.pth [2023-12-26 22:32:31,411][105692] Updated weights for policy 0, policy_version 984560 (0.0009) [2023-12-26 22:32:31,463][105620] Updated weights for policy 1, policy_version 984785 (0.0008) [2023-12-26 22:32:31,468][105692] Updated weights for policy 0, policy_version 984570 (0.0006) [2023-12-26 22:32:31,519][105620] Updated weights for policy 1, policy_version 984795 (0.0005) [2023-12-26 22:32:31,525][105692] Updated weights for policy 0, policy_version 984580 (0.0009) [2023-12-26 22:32:31,571][105620] Updated weights for policy 1, policy_version 984805 (0.0007) [2023-12-26 22:32:32,147][105692] Updated weights for policy 0, policy_version 984590 (0.0006) [2023-12-26 22:32:32,210][105692] Updated weights for policy 0, policy_version 984600 (0.0005) [2023-12-26 22:32:32,277][105692] Updated weights for policy 0, policy_version 984610 (0.0007) [2023-12-26 22:32:32,388][105620] Updated weights for policy 1, policy_version 984815 (0.0008) [2023-12-26 22:32:32,442][105620] Updated weights for policy 1, policy_version 984825 (0.0010) [2023-12-26 22:32:32,512][105620] Updated weights for policy 1, policy_version 984835 (0.0010) [2023-12-26 22:32:32,872][105692] Updated weights for policy 0, policy_version 984620 (0.0006) [2023-12-26 22:32:32,934][105692] Updated weights for policy 0, policy_version 984630 (0.0007) [2023-12-26 22:32:32,988][105692] Updated weights for policy 0, policy_version 984640 (0.0009) [2023-12-26 22:32:33,259][105620] Updated weights for policy 1, policy_version 984845 (0.0008) [2023-12-26 22:32:33,308][105620] Updated weights for policy 1, policy_version 984855 (0.0006) [2023-12-26 22:32:33,379][105620] Updated weights for policy 1, policy_version 984865 (0.0008) [2023-12-26 22:32:33,596][105692] Updated weights for policy 0, policy_version 984650 (0.0008) [2023-12-26 22:32:33,654][105692] Updated weights for policy 0, policy_version 984660 (0.0005) [2023-12-26 22:32:33,704][105692] Updated weights for policy 0, policy_version 984670 (0.0005) [2023-12-26 22:32:33,758][105692] Updated weights for policy 0, policy_version 984680 (0.0005) [2023-12-26 22:32:34,006][105620] Updated weights for policy 1, policy_version 984875 (0.0007) [2023-12-26 22:32:34,060][105620] Updated weights for policy 1, policy_version 984885 (0.0005) [2023-12-26 22:32:34,134][105620] Updated weights for policy 1, policy_version 984895 (0.0005) [2023-12-26 22:32:34,341][105692] Updated weights for policy 0, policy_version 984690 (0.0009) [2023-12-26 22:32:34,392][105692] Updated weights for policy 0, policy_version 984700 (0.0009) [2023-12-26 22:32:34,447][105692] Updated weights for policy 0, policy_version 984710 (0.0009) [2023-12-26 22:32:34,834][105620] Updated weights for policy 1, policy_version 984905 (0.0008) [2023-12-26 22:32:34,886][105620] Updated weights for policy 1, policy_version 984915 (0.0009) [2023-12-26 22:32:34,956][105620] Updated weights for policy 1, policy_version 984925 (0.0006) [2023-12-26 22:32:35,025][105620] Updated weights for policy 1, policy_version 984935 (0.0007) [2023-12-26 22:32:35,131][105692] Updated weights for policy 0, policy_version 984720 (0.0006) [2023-12-26 22:32:35,187][105692] Updated weights for policy 0, policy_version 984730 (0.0005) [2023-12-26 22:32:35,251][105692] Updated weights for policy 0, policy_version 984740 (0.0005) [2023-12-26 22:32:35,587][105620] Updated weights for policy 1, policy_version 984945 (0.0008) [2023-12-26 22:32:35,649][105620] Updated weights for policy 1, policy_version 984955 (0.0010) [2023-12-26 22:32:35,706][105620] Updated weights for policy 1, policy_version 984965 (0.0013) [2023-12-26 22:32:35,800][105692] Updated weights for policy 0, policy_version 984750 (0.0005) [2023-12-26 22:32:35,853][105692] Updated weights for policy 0, policy_version 984760 (0.0005) [2023-12-26 22:32:35,932][105692] Updated weights for policy 0, policy_version 984770 (0.0008) [2023-12-26 22:32:36,062][104569] Fps is (10 sec: 21299.4, 60 sec: 19933.9, 300 sec: 19383.1). Total num frames: 504324096. Throughput: 0: 9928.8, 1: 9973.2. Samples: 504305344. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:32:36,063][104569] Avg episode reward: [(0, '8015.493'), (1, '9356.037')] [2023-12-26 22:32:36,446][105620] Updated weights for policy 1, policy_version 984975 (0.0009) [2023-12-26 22:32:36,505][105620] Updated weights for policy 1, policy_version 984985 (0.0009) [2023-12-26 22:32:36,567][105620] Updated weights for policy 1, policy_version 984995 (0.0009) [2023-12-26 22:32:36,594][105692] Updated weights for policy 0, policy_version 984780 (0.0008) [2023-12-26 22:32:36,663][105692] Updated weights for policy 0, policy_version 984790 (0.0008) [2023-12-26 22:32:36,730][105692] Updated weights for policy 0, policy_version 984800 (0.0009) [2023-12-26 22:32:37,192][105620] Updated weights for policy 1, policy_version 985005 (0.0008) [2023-12-26 22:32:37,243][105620] Updated weights for policy 1, policy_version 985015 (0.0009) [2023-12-26 22:32:37,298][105620] Updated weights for policy 1, policy_version 985025 (0.0009) [2023-12-26 22:32:37,619][105692] Updated weights for policy 0, policy_version 984810 (0.0009) [2023-12-26 22:32:37,671][105692] Updated weights for policy 0, policy_version 984820 (0.0008) [2023-12-26 22:32:37,723][105692] Updated weights for policy 0, policy_version 984830 (0.0009) [2023-12-26 22:32:37,772][105692] Updated weights for policy 0, policy_version 984840 (0.0009) [2023-12-26 22:32:37,958][105620] Updated weights for policy 1, policy_version 985035 (0.0007) [2023-12-26 22:32:38,028][105620] Updated weights for policy 1, policy_version 985045 (0.0009) [2023-12-26 22:32:38,091][105620] Updated weights for policy 1, policy_version 985055 (0.0009) [2023-12-26 22:32:38,475][105692] Updated weights for policy 0, policy_version 984850 (0.0008) [2023-12-26 22:32:38,527][105692] Updated weights for policy 0, policy_version 984860 (0.0008) [2023-12-26 22:32:38,576][105692] Updated weights for policy 0, policy_version 984870 (0.0008) [2023-12-26 22:32:38,837][105620] Updated weights for policy 1, policy_version 985065 (0.0011) [2023-12-26 22:32:38,893][105620] Updated weights for policy 1, policy_version 985075 (0.0010) [2023-12-26 22:32:38,949][105620] Updated weights for policy 1, policy_version 985085 (0.0010) [2023-12-26 22:32:39,001][105620] Updated weights for policy 1, policy_version 985095 (0.0010) [2023-12-26 22:32:39,373][105692] Updated weights for policy 0, policy_version 984881 (0.0008) [2023-12-26 22:32:39,437][105692] Updated weights for policy 0, policy_version 984891 (0.0009) [2023-12-26 22:32:39,489][105692] Updated weights for policy 0, policy_version 984901 (0.0010) [2023-12-26 22:32:39,726][105620] Updated weights for policy 1, policy_version 985105 (0.0010) [2023-12-26 22:32:39,785][105620] Updated weights for policy 1, policy_version 985115 (0.0009) [2023-12-26 22:32:39,857][105620] Updated weights for policy 1, policy_version 985125 (0.0009) [2023-12-26 22:32:40,280][105692] Updated weights for policy 0, policy_version 984911 (0.0009) [2023-12-26 22:32:40,340][105692] Updated weights for policy 0, policy_version 984921 (0.0009) [2023-12-26 22:32:40,410][105692] Updated weights for policy 0, policy_version 984931 (0.0007) [2023-12-26 22:32:40,614][105620] Updated weights for policy 1, policy_version 985135 (0.0008) [2023-12-26 22:32:40,669][105620] Updated weights for policy 1, policy_version 985145 (0.0008) [2023-12-26 22:32:40,726][105620] Updated weights for policy 1, policy_version 985155 (0.0009) [2023-12-26 22:32:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19933.9, 300 sec: 19383.1). Total num frames: 504414208. Throughput: 0: 9922.3, 1: 9960.1. Samples: 504423404. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:32:41,063][104569] Avg episode reward: [(0, '8025.619'), (1, '9356.043')] [2023-12-26 22:32:41,210][105692] Updated weights for policy 0, policy_version 984941 (0.0007) [2023-12-26 22:32:41,279][105692] Updated weights for policy 0, policy_version 984951 (0.0007) [2023-12-26 22:32:41,342][105692] Updated weights for policy 0, policy_version 984961 (0.0008) [2023-12-26 22:32:41,552][105620] Updated weights for policy 1, policy_version 985166 (0.0010) [2023-12-26 22:32:41,618][105620] Updated weights for policy 1, policy_version 985176 (0.0009) [2023-12-26 22:32:41,684][105620] Updated weights for policy 1, policy_version 985186 (0.0009) [2023-12-26 22:32:42,022][105692] Updated weights for policy 0, policy_version 984971 (0.0007) [2023-12-26 22:32:42,068][105692] Updated weights for policy 0, policy_version 984981 (0.0005) [2023-12-26 22:32:42,118][105692] Updated weights for policy 0, policy_version 984991 (0.0005) [2023-12-26 22:32:42,440][105620] Updated weights for policy 1, policy_version 985196 (0.0009) [2023-12-26 22:32:42,492][105620] Updated weights for policy 1, policy_version 985206 (0.0011) [2023-12-26 22:32:42,552][105620] Updated weights for policy 1, policy_version 985216 (0.0009) [2023-12-26 22:32:42,825][105692] Updated weights for policy 0, policy_version 985001 (0.0006) [2023-12-26 22:32:42,885][105692] Updated weights for policy 0, policy_version 985011 (0.0008) [2023-12-26 22:32:42,944][105692] Updated weights for policy 0, policy_version 985021 (0.0008) [2023-12-26 22:32:42,999][105692] Updated weights for policy 0, policy_version 985031 (0.0008) [2023-12-26 22:32:43,304][105620] Updated weights for policy 1, policy_version 985226 (0.0010) [2023-12-26 22:32:43,362][105620] Updated weights for policy 1, policy_version 985236 (0.0010) [2023-12-26 22:32:43,421][105620] Updated weights for policy 1, policy_version 985246 (0.0010) [2023-12-26 22:32:43,479][105620] Updated weights for policy 1, policy_version 985256 (0.0010) [2023-12-26 22:32:43,759][105692] Updated weights for policy 0, policy_version 985041 (0.0011) [2023-12-26 22:32:43,809][105692] Updated weights for policy 0, policy_version 985051 (0.0008) [2023-12-26 22:32:43,858][105692] Updated weights for policy 0, policy_version 985061 (0.0005) [2023-12-26 22:32:44,218][105620] Updated weights for policy 1, policy_version 985266 (0.0005) [2023-12-26 22:32:44,277][105620] Updated weights for policy 1, policy_version 985276 (0.0005) [2023-12-26 22:32:44,331][105620] Updated weights for policy 1, policy_version 985286 (0.0008) [2023-12-26 22:32:44,466][105692] Updated weights for policy 0, policy_version 985071 (0.0009) [2023-12-26 22:32:44,520][105692] Updated weights for policy 0, policy_version 985081 (0.0009) [2023-12-26 22:32:44,565][105692] Updated weights for policy 0, policy_version 985091 (0.0010) [2023-12-26 22:32:44,965][105620] Updated weights for policy 1, policy_version 985296 (0.0009) [2023-12-26 22:32:45,028][105620] Updated weights for policy 1, policy_version 985306 (0.0010) [2023-12-26 22:32:45,086][105620] Updated weights for policy 1, policy_version 985316 (0.0010) [2023-12-26 22:32:45,362][105692] Updated weights for policy 0, policy_version 985101 (0.0011) [2023-12-26 22:32:45,429][105692] Updated weights for policy 0, policy_version 985111 (0.0011) [2023-12-26 22:32:45,495][105692] Updated weights for policy 0, policy_version 985121 (0.0011) [2023-12-26 22:32:45,825][105620] Updated weights for policy 1, policy_version 985326 (0.0010) [2023-12-26 22:32:45,868][105620] Updated weights for policy 1, policy_version 985336 (0.0010) [2023-12-26 22:32:45,916][105620] Updated weights for policy 1, policy_version 985346 (0.0010) [2023-12-26 22:32:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19797.3, 300 sec: 19383.1). Total num frames: 504512512. Throughput: 0: 9892.3, 1: 9844.8. Samples: 504479648. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:32:46,063][104569] Avg episode reward: [(0, '8297.282'), (1, '9265.666')] [2023-12-26 22:32:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000985352_252280832.pth... [2023-12-26 22:32:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000985128_252231680.pth... [2023-12-26 22:32:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000984200_251985920.pth [2023-12-26 22:32:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000983976_251936768.pth [2023-12-26 22:32:46,214][105692] Updated weights for policy 0, policy_version 985131 (0.0011) [2023-12-26 22:32:46,272][105692] Updated weights for policy 0, policy_version 985141 (0.0010) [2023-12-26 22:32:46,341][105692] Updated weights for policy 0, policy_version 985151 (0.0011) [2023-12-26 22:32:46,594][105620] Updated weights for policy 1, policy_version 985356 (0.0008) [2023-12-26 22:32:46,648][105620] Updated weights for policy 1, policy_version 985366 (0.0005) [2023-12-26 22:32:46,703][105620] Updated weights for policy 1, policy_version 985376 (0.0005) [2023-12-26 22:32:47,036][105692] Updated weights for policy 0, policy_version 985161 (0.0010) [2023-12-26 22:32:47,094][105692] Updated weights for policy 0, policy_version 985171 (0.0008) [2023-12-26 22:32:47,148][105692] Updated weights for policy 0, policy_version 985181 (0.0010) [2023-12-26 22:32:47,207][105692] Updated weights for policy 0, policy_version 985192 (0.0011) [2023-12-26 22:32:47,234][105620] Updated weights for policy 1, policy_version 985386 (0.0006) [2023-12-26 22:32:47,290][105620] Updated weights for policy 1, policy_version 985396 (0.0009) [2023-12-26 22:32:47,336][105620] Updated weights for policy 1, policy_version 985406 (0.0008) [2023-12-26 22:32:47,385][105620] Updated weights for policy 1, policy_version 985416 (0.0007) [2023-12-26 22:32:47,959][105620] Updated weights for policy 1, policy_version 985426 (0.0005) [2023-12-26 22:32:48,031][105620] Updated weights for policy 1, policy_version 985436 (0.0005) [2023-12-26 22:32:48,064][105692] Updated weights for policy 0, policy_version 985202 (0.0009) [2023-12-26 22:32:48,087][105620] Updated weights for policy 1, policy_version 985446 (0.0005) [2023-12-26 22:32:48,113][105692] Updated weights for policy 0, policy_version 985212 (0.0009) [2023-12-26 22:32:48,165][105692] Updated weights for policy 0, policy_version 985223 (0.0010) [2023-12-26 22:32:48,730][105620] Updated weights for policy 1, policy_version 985456 (0.0009) [2023-12-26 22:32:48,781][105620] Updated weights for policy 1, policy_version 985466 (0.0009) [2023-12-26 22:32:48,843][105620] Updated weights for policy 1, policy_version 985476 (0.0009) [2023-12-26 22:32:48,917][105692] Updated weights for policy 0, policy_version 985233 (0.0009) [2023-12-26 22:32:48,964][105692] Updated weights for policy 0, policy_version 985243 (0.0009) [2023-12-26 22:32:49,012][105692] Updated weights for policy 0, policy_version 985253 (0.0009) [2023-12-26 22:32:49,551][105620] Updated weights for policy 1, policy_version 985486 (0.0007) [2023-12-26 22:32:49,619][105620] Updated weights for policy 1, policy_version 985496 (0.0006) [2023-12-26 22:32:49,678][105620] Updated weights for policy 1, policy_version 985506 (0.0007) [2023-12-26 22:32:49,773][105692] Updated weights for policy 0, policy_version 985263 (0.0009) [2023-12-26 22:32:49,826][105692] Updated weights for policy 0, policy_version 985273 (0.0009) [2023-12-26 22:32:49,891][105692] Updated weights for policy 0, policy_version 985283 (0.0008) [2023-12-26 22:32:50,378][105620] Updated weights for policy 1, policy_version 985516 (0.0008) [2023-12-26 22:32:50,439][105620] Updated weights for policy 1, policy_version 985526 (0.0009) [2023-12-26 22:32:50,500][105620] Updated weights for policy 1, policy_version 985536 (0.0009) [2023-12-26 22:32:50,603][105692] Updated weights for policy 0, policy_version 985293 (0.0009) [2023-12-26 22:32:50,661][105692] Updated weights for policy 0, policy_version 985303 (0.0008) [2023-12-26 22:32:50,724][105692] Updated weights for policy 0, policy_version 985313 (0.0008) [2023-12-26 22:32:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19383.1). Total num frames: 504610816. Throughput: 0: 9931.9, 1: 9913.5. Samples: 504599992. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:32:51,063][104569] Avg episode reward: [(0, '8650.648'), (1, '9265.418')] [2023-12-26 22:32:51,282][105620] Updated weights for policy 1, policy_version 985546 (0.0009) [2023-12-26 22:32:51,350][105620] Updated weights for policy 1, policy_version 985556 (0.0007) [2023-12-26 22:32:51,418][105620] Updated weights for policy 1, policy_version 985566 (0.0007) [2023-12-26 22:32:51,448][105692] Updated weights for policy 0, policy_version 985323 (0.0009) [2023-12-26 22:32:51,470][105620] Updated weights for policy 1, policy_version 985576 (0.0008) [2023-12-26 22:32:51,503][105692] Updated weights for policy 0, policy_version 985333 (0.0008) [2023-12-26 22:32:51,561][105692] Updated weights for policy 0, policy_version 985343 (0.0010) [2023-12-26 22:32:52,158][105620] Updated weights for policy 1, policy_version 985586 (0.0009) [2023-12-26 22:32:52,219][105620] Updated weights for policy 1, policy_version 985596 (0.0009) [2023-12-26 22:32:52,282][105620] Updated weights for policy 1, policy_version 985606 (0.0009) [2023-12-26 22:32:52,426][105692] Updated weights for policy 0, policy_version 985353 (0.0010) [2023-12-26 22:32:52,480][105692] Updated weights for policy 0, policy_version 985363 (0.0009) [2023-12-26 22:32:52,541][105692] Updated weights for policy 0, policy_version 985373 (0.0009) [2023-12-26 22:32:52,606][105692] Updated weights for policy 0, policy_version 985383 (0.0010) [2023-12-26 22:32:52,952][105620] Updated weights for policy 1, policy_version 985616 (0.0008) [2023-12-26 22:32:53,006][105620] Updated weights for policy 1, policy_version 985626 (0.0010) [2023-12-26 22:32:53,056][105620] Updated weights for policy 1, policy_version 985636 (0.0007) [2023-12-26 22:32:53,334][105692] Updated weights for policy 0, policy_version 985393 (0.0006) [2023-12-26 22:32:53,402][105692] Updated weights for policy 0, policy_version 985403 (0.0005) [2023-12-26 22:32:53,491][105692] Updated weights for policy 0, policy_version 985413 (0.0007) [2023-12-26 22:32:53,836][105620] Updated weights for policy 1, policy_version 985646 (0.0006) [2023-12-26 22:32:53,907][105620] Updated weights for policy 1, policy_version 985656 (0.0005) [2023-12-26 22:32:53,975][105620] Updated weights for policy 1, policy_version 985666 (0.0005) [2023-12-26 22:32:54,034][105692] Updated weights for policy 0, policy_version 985423 (0.0006) [2023-12-26 22:32:54,094][105692] Updated weights for policy 0, policy_version 985433 (0.0007) [2023-12-26 22:32:54,157][105692] Updated weights for policy 0, policy_version 985443 (0.0007) [2023-12-26 22:32:54,584][105620] Updated weights for policy 1, policy_version 985676 (0.0007) [2023-12-26 22:32:54,642][105620] Updated weights for policy 1, policy_version 985686 (0.0006) [2023-12-26 22:32:54,698][105620] Updated weights for policy 1, policy_version 985696 (0.0005) [2023-12-26 22:32:54,875][105692] Updated weights for policy 0, policy_version 985453 (0.0007) [2023-12-26 22:32:54,930][105692] Updated weights for policy 0, policy_version 985463 (0.0008) [2023-12-26 22:32:54,984][105692] Updated weights for policy 0, policy_version 985473 (0.0009) [2023-12-26 22:32:55,365][105620] Updated weights for policy 1, policy_version 985706 (0.0008) [2023-12-26 22:32:55,418][105620] Updated weights for policy 1, policy_version 985716 (0.0005) [2023-12-26 22:32:55,469][105620] Updated weights for policy 1, policy_version 985726 (0.0008) [2023-12-26 22:32:55,522][105620] Updated weights for policy 1, policy_version 985736 (0.0006) [2023-12-26 22:32:55,809][105692] Updated weights for policy 0, policy_version 985483 (0.0009) [2023-12-26 22:32:55,873][105692] Updated weights for policy 0, policy_version 985493 (0.0008) [2023-12-26 22:32:55,939][105692] Updated weights for policy 0, policy_version 985503 (0.0008) [2023-12-26 22:32:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19797.3, 300 sec: 19438.6). Total num frames: 504709120. Throughput: 0: 9835.0, 1: 9930.7. Samples: 504717496. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:32:56,062][104569] Avg episode reward: [(0, '8234.636'), (1, '9355.475')] [2023-12-26 22:32:56,148][105620] Updated weights for policy 1, policy_version 985746 (0.0006) [2023-12-26 22:32:56,205][105620] Updated weights for policy 1, policy_version 985756 (0.0009) [2023-12-26 22:32:56,265][105620] Updated weights for policy 1, policy_version 985766 (0.0011) [2023-12-26 22:32:56,645][105692] Updated weights for policy 0, policy_version 985513 (0.0008) [2023-12-26 22:32:56,697][105692] Updated weights for policy 0, policy_version 985523 (0.0009) [2023-12-26 22:32:56,755][105692] Updated weights for policy 0, policy_version 985533 (0.0010) [2023-12-26 22:32:56,807][105692] Updated weights for policy 0, policy_version 985543 (0.0010) [2023-12-26 22:32:56,868][105620] Updated weights for policy 1, policy_version 985776 (0.0010) [2023-12-26 22:32:56,921][105620] Updated weights for policy 1, policy_version 985786 (0.0010) [2023-12-26 22:32:56,979][105620] Updated weights for policy 1, policy_version 985796 (0.0010) [2023-12-26 22:32:57,456][105692] Updated weights for policy 0, policy_version 985553 (0.0006) [2023-12-26 22:32:57,501][105692] Updated weights for policy 0, policy_version 985563 (0.0005) [2023-12-26 22:32:57,553][105692] Updated weights for policy 0, policy_version 985573 (0.0005) [2023-12-26 22:32:57,653][105620] Updated weights for policy 1, policy_version 985806 (0.0009) [2023-12-26 22:32:57,710][105620] Updated weights for policy 1, policy_version 985816 (0.0008) [2023-12-26 22:32:57,762][105620] Updated weights for policy 1, policy_version 985826 (0.0005) [2023-12-26 22:32:58,217][105692] Updated weights for policy 0, policy_version 985583 (0.0009) [2023-12-26 22:32:58,280][105692] Updated weights for policy 0, policy_version 985593 (0.0010) [2023-12-26 22:32:58,342][105692] Updated weights for policy 0, policy_version 985603 (0.0009) [2023-12-26 22:32:58,411][105620] Updated weights for policy 1, policy_version 985836 (0.0007) [2023-12-26 22:32:58,472][105620] Updated weights for policy 1, policy_version 985846 (0.0011) [2023-12-26 22:32:58,532][105620] Updated weights for policy 1, policy_version 985856 (0.0011) [2023-12-26 22:32:59,211][105692] Updated weights for policy 0, policy_version 985613 (0.0010) [2023-12-26 22:32:59,279][105692] Updated weights for policy 0, policy_version 985623 (0.0010) [2023-12-26 22:32:59,351][105692] Updated weights for policy 0, policy_version 985633 (0.0010) [2023-12-26 22:32:59,356][105620] Updated weights for policy 1, policy_version 985866 (0.0009) [2023-12-26 22:32:59,418][105620] Updated weights for policy 1, policy_version 985876 (0.0009) [2023-12-26 22:32:59,470][105620] Updated weights for policy 1, policy_version 985886 (0.0010) [2023-12-26 22:32:59,522][105620] Updated weights for policy 1, policy_version 985896 (0.0010) [2023-12-26 22:32:59,978][105692] Updated weights for policy 0, policy_version 985643 (0.0009) [2023-12-26 22:33:00,034][105692] Updated weights for policy 0, policy_version 985653 (0.0010) [2023-12-26 22:33:00,082][105692] Updated weights for policy 0, policy_version 985663 (0.0009) [2023-12-26 22:33:00,303][105620] Updated weights for policy 1, policy_version 985906 (0.0008) [2023-12-26 22:33:00,357][105620] Updated weights for policy 1, policy_version 985916 (0.0009) [2023-12-26 22:33:00,411][105620] Updated weights for policy 1, policy_version 985926 (0.0007) [2023-12-26 22:33:00,850][105692] Updated weights for policy 0, policy_version 985673 (0.0008) [2023-12-26 22:33:00,905][105692] Updated weights for policy 0, policy_version 985683 (0.0009) [2023-12-26 22:33:00,962][105692] Updated weights for policy 0, policy_version 985693 (0.0009) [2023-12-26 22:33:01,027][105692] Updated weights for policy 0, policy_version 985703 (0.0008) [2023-12-26 22:33:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19438.6). Total num frames: 504807424. Throughput: 0: 9850.6, 1: 9998.2. Samples: 504777680. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:33:01,063][104569] Avg episode reward: [(0, '7781.744'), (1, '9171.200')] [2023-12-26 22:33:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000985704_252379136.pth... [2023-12-26 22:33:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000985928_252428288.pth... [2023-12-26 22:33:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000984552_252084224.pth [2023-12-26 22:33:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000984776_252133376.pth [2023-12-26 22:33:01,230][105620] Updated weights for policy 1, policy_version 985936 (0.0008) [2023-12-26 22:33:01,289][105620] Updated weights for policy 1, policy_version 985946 (0.0008) [2023-12-26 22:33:01,350][105620] Updated weights for policy 1, policy_version 985956 (0.0008) [2023-12-26 22:33:01,766][105692] Updated weights for policy 0, policy_version 985713 (0.0008) [2023-12-26 22:33:01,818][105692] Updated weights for policy 0, policy_version 985723 (0.0009) [2023-12-26 22:33:01,877][105692] Updated weights for policy 0, policy_version 985733 (0.0008) [2023-12-26 22:33:02,030][105620] Updated weights for policy 1, policy_version 985966 (0.0009) [2023-12-26 22:33:02,093][105620] Updated weights for policy 1, policy_version 985976 (0.0005) [2023-12-26 22:33:02,161][105620] Updated weights for policy 1, policy_version 985986 (0.0005) [2023-12-26 22:33:02,731][105692] Updated weights for policy 0, policy_version 985743 (0.0008) [2023-12-26 22:33:02,777][105620] Updated weights for policy 1, policy_version 985996 (0.0007) [2023-12-26 22:33:02,792][105692] Updated weights for policy 0, policy_version 985753 (0.0006) [2023-12-26 22:33:02,837][105620] Updated weights for policy 1, policy_version 986006 (0.0011) [2023-12-26 22:33:02,847][105692] Updated weights for policy 0, policy_version 985763 (0.0006) [2023-12-26 22:33:02,896][105620] Updated weights for policy 1, policy_version 986016 (0.0011) [2023-12-26 22:33:03,513][105692] Updated weights for policy 0, policy_version 985773 (0.0007) [2023-12-26 22:33:03,570][105620] Updated weights for policy 1, policy_version 986026 (0.0010) [2023-12-26 22:33:03,572][105692] Updated weights for policy 0, policy_version 985783 (0.0005) [2023-12-26 22:33:03,618][105692] Updated weights for policy 0, policy_version 985793 (0.0005) [2023-12-26 22:33:03,632][105620] Updated weights for policy 1, policy_version 986036 (0.0010) [2023-12-26 22:33:03,687][105620] Updated weights for policy 1, policy_version 986046 (0.0010) [2023-12-26 22:33:03,746][105620] Updated weights for policy 1, policy_version 986056 (0.0005) [2023-12-26 22:33:04,185][105692] Updated weights for policy 0, policy_version 985803 (0.0006) [2023-12-26 22:33:04,235][105692] Updated weights for policy 0, policy_version 985813 (0.0006) [2023-12-26 22:33:04,285][105692] Updated weights for policy 0, policy_version 985823 (0.0006) [2023-12-26 22:33:04,346][105620] Updated weights for policy 1, policy_version 986066 (0.0009) [2023-12-26 22:33:04,405][105620] Updated weights for policy 1, policy_version 986076 (0.0010) [2023-12-26 22:33:04,471][105620] Updated weights for policy 1, policy_version 986086 (0.0008) [2023-12-26 22:33:05,019][105692] Updated weights for policy 0, policy_version 985833 (0.0006) [2023-12-26 22:33:05,069][105692] Updated weights for policy 0, policy_version 985843 (0.0007) [2023-12-26 22:33:05,120][105692] Updated weights for policy 0, policy_version 985853 (0.0005) [2023-12-26 22:33:05,121][105620] Updated weights for policy 1, policy_version 986096 (0.0010) [2023-12-26 22:33:05,170][105620] Updated weights for policy 1, policy_version 986106 (0.0011) [2023-12-26 22:33:05,177][105692] Updated weights for policy 0, policy_version 985863 (0.0005) [2023-12-26 22:33:05,215][105620] Updated weights for policy 1, policy_version 986116 (0.0010) [2023-12-26 22:33:05,818][105692] Updated weights for policy 0, policy_version 985873 (0.0010) [2023-12-26 22:33:05,845][105620] Updated weights for policy 1, policy_version 986126 (0.0009) [2023-12-26 22:33:05,876][105692] Updated weights for policy 0, policy_version 985883 (0.0010) [2023-12-26 22:33:05,904][105620] Updated weights for policy 1, policy_version 986136 (0.0005) [2023-12-26 22:33:05,935][105692] Updated weights for policy 0, policy_version 985893 (0.0010) [2023-12-26 22:33:05,957][105620] Updated weights for policy 1, policy_version 986146 (0.0006) [2023-12-26 22:33:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19933.9, 300 sec: 19466.4). Total num frames: 504913920. Throughput: 0: 9812.2, 1: 10004.6. Samples: 504896272. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:33:06,062][104569] Avg episode reward: [(0, '7929.440'), (1, '9171.113')] [2023-12-26 22:33:06,564][105620] Updated weights for policy 1, policy_version 986156 (0.0008) [2023-12-26 22:33:06,570][105692] Updated weights for policy 0, policy_version 985903 (0.0007) [2023-12-26 22:33:06,617][105620] Updated weights for policy 1, policy_version 986166 (0.0007) [2023-12-26 22:33:06,634][105692] Updated weights for policy 0, policy_version 985913 (0.0007) [2023-12-26 22:33:06,684][105620] Updated weights for policy 1, policy_version 986176 (0.0005) [2023-12-26 22:33:06,693][105692] Updated weights for policy 0, policy_version 985923 (0.0007) [2023-12-26 22:33:07,347][105692] Updated weights for policy 0, policy_version 985933 (0.0009) [2023-12-26 22:33:07,395][105692] Updated weights for policy 0, policy_version 985943 (0.0010) [2023-12-26 22:33:07,439][105692] Updated weights for policy 0, policy_version 985953 (0.0010) [2023-12-26 22:33:07,473][105620] Updated weights for policy 1, policy_version 986186 (0.0006) [2023-12-26 22:33:07,520][105620] Updated weights for policy 1, policy_version 986196 (0.0007) [2023-12-26 22:33:07,565][105620] Updated weights for policy 1, policy_version 986206 (0.0007) [2023-12-26 22:33:07,620][105620] Updated weights for policy 1, policy_version 986216 (0.0006) [2023-12-26 22:33:08,138][105692] Updated weights for policy 0, policy_version 985963 (0.0009) [2023-12-26 22:33:08,201][105692] Updated weights for policy 0, policy_version 985973 (0.0005) [2023-12-26 22:33:08,256][105692] Updated weights for policy 0, policy_version 985983 (0.0006) [2023-12-26 22:33:08,287][105620] Updated weights for policy 1, policy_version 986226 (0.0008) [2023-12-26 22:33:08,347][105620] Updated weights for policy 1, policy_version 986236 (0.0008) [2023-12-26 22:33:08,400][105620] Updated weights for policy 1, policy_version 986246 (0.0009) [2023-12-26 22:33:08,963][105692] Updated weights for policy 0, policy_version 985993 (0.0005) [2023-12-26 22:33:09,016][105692] Updated weights for policy 0, policy_version 986003 (0.0005) [2023-12-26 22:33:09,070][105692] Updated weights for policy 0, policy_version 986013 (0.0005) [2023-12-26 22:33:09,120][105692] Updated weights for policy 0, policy_version 986023 (0.0007) [2023-12-26 22:33:09,213][105620] Updated weights for policy 1, policy_version 986256 (0.0010) [2023-12-26 22:33:09,274][105620] Updated weights for policy 1, policy_version 986266 (0.0011) [2023-12-26 22:33:09,324][105620] Updated weights for policy 1, policy_version 986276 (0.0011) [2023-12-26 22:33:09,875][105692] Updated weights for policy 0, policy_version 986033 (0.0008) [2023-12-26 22:33:09,947][105692] Updated weights for policy 0, policy_version 986043 (0.0007) [2023-12-26 22:33:10,001][105692] Updated weights for policy 0, policy_version 986053 (0.0007) [2023-12-26 22:33:10,010][105620] Updated weights for policy 1, policy_version 986286 (0.0011) [2023-12-26 22:33:10,070][105620] Updated weights for policy 1, policy_version 986296 (0.0010) [2023-12-26 22:33:10,129][105620] Updated weights for policy 1, policy_version 986306 (0.0008) [2023-12-26 22:33:10,701][105692] Updated weights for policy 0, policy_version 986063 (0.0008) [2023-12-26 22:33:10,755][105692] Updated weights for policy 0, policy_version 986073 (0.0009) [2023-12-26 22:33:10,815][105692] Updated weights for policy 0, policy_version 986083 (0.0008) [2023-12-26 22:33:10,880][105620] Updated weights for policy 1, policy_version 986316 (0.0010) [2023-12-26 22:33:10,934][105620] Updated weights for policy 1, policy_version 986326 (0.0009) [2023-12-26 22:33:10,998][105620] Updated weights for policy 1, policy_version 986336 (0.0009) [2023-12-26 22:33:11,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 505012224. Throughput: 0: 9906.0, 1: 9985.3. Samples: 505016380. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:33:11,063][104569] Avg episode reward: [(0, '8380.173'), (1, '9355.296')] [2023-12-26 22:33:11,581][105692] Updated weights for policy 0, policy_version 986093 (0.0009) [2023-12-26 22:33:11,650][105692] Updated weights for policy 0, policy_version 986103 (0.0009) [2023-12-26 22:33:11,715][105692] Updated weights for policy 0, policy_version 986113 (0.0009) [2023-12-26 22:33:11,827][105620] Updated weights for policy 1, policy_version 986346 (0.0008) [2023-12-26 22:33:11,889][105620] Updated weights for policy 1, policy_version 986356 (0.0008) [2023-12-26 22:33:11,955][105620] Updated weights for policy 1, policy_version 986366 (0.0008) [2023-12-26 22:33:12,022][105620] Updated weights for policy 1, policy_version 986376 (0.0008) [2023-12-26 22:33:12,460][105692] Updated weights for policy 0, policy_version 986123 (0.0008) [2023-12-26 22:33:12,534][105692] Updated weights for policy 0, policy_version 986133 (0.0009) [2023-12-26 22:33:12,599][105692] Updated weights for policy 0, policy_version 986143 (0.0008) [2023-12-26 22:33:12,718][105620] Updated weights for policy 1, policy_version 986386 (0.0006) [2023-12-26 22:33:12,777][105620] Updated weights for policy 1, policy_version 986396 (0.0005) [2023-12-26 22:33:12,836][105620] Updated weights for policy 1, policy_version 986406 (0.0005) [2023-12-26 22:33:13,262][105692] Updated weights for policy 0, policy_version 986153 (0.0009) [2023-12-26 22:33:13,314][105692] Updated weights for policy 0, policy_version 986163 (0.0005) [2023-12-26 22:33:13,369][105692] Updated weights for policy 0, policy_version 986173 (0.0005) [2023-12-26 22:33:13,409][105620] Updated weights for policy 1, policy_version 986416 (0.0006) [2023-12-26 22:33:13,422][105692] Updated weights for policy 0, policy_version 986183 (0.0006) [2023-12-26 22:33:13,454][105620] Updated weights for policy 1, policy_version 986426 (0.0006) [2023-12-26 22:33:13,506][105620] Updated weights for policy 1, policy_version 986436 (0.0008) [2023-12-26 22:33:14,073][105692] Updated weights for policy 0, policy_version 986193 (0.0010) [2023-12-26 22:33:14,132][105692] Updated weights for policy 0, policy_version 986203 (0.0010) [2023-12-26 22:33:14,148][105620] Updated weights for policy 1, policy_version 986446 (0.0006) [2023-12-26 22:33:14,184][105692] Updated weights for policy 0, policy_version 986213 (0.0010) [2023-12-26 22:33:14,200][105620] Updated weights for policy 1, policy_version 986456 (0.0006) [2023-12-26 22:33:14,256][105620] Updated weights for policy 1, policy_version 986466 (0.0005) [2023-12-26 22:33:14,895][105620] Updated weights for policy 1, policy_version 986476 (0.0007) [2023-12-26 22:33:14,949][105620] Updated weights for policy 1, policy_version 986486 (0.0011) [2023-12-26 22:33:14,951][105692] Updated weights for policy 0, policy_version 986223 (0.0011) [2023-12-26 22:33:14,999][105620] Updated weights for policy 1, policy_version 986496 (0.0011) [2023-12-26 22:33:15,012][105692] Updated weights for policy 0, policy_version 986233 (0.0011) [2023-12-26 22:33:15,074][105692] Updated weights for policy 0, policy_version 986243 (0.0010) [2023-12-26 22:33:15,746][105620] Updated weights for policy 1, policy_version 986506 (0.0011) [2023-12-26 22:33:15,797][105620] Updated weights for policy 1, policy_version 986516 (0.0010) [2023-12-26 22:33:15,812][105692] Updated weights for policy 0, policy_version 986253 (0.0010) [2023-12-26 22:33:15,859][105620] Updated weights for policy 1, policy_version 986526 (0.0008) [2023-12-26 22:33:15,859][105692] Updated weights for policy 0, policy_version 986263 (0.0010) [2023-12-26 22:33:15,908][105692] Updated weights for policy 0, policy_version 986273 (0.0010) [2023-12-26 22:33:15,915][105620] Updated weights for policy 1, policy_version 986536 (0.0011) [2023-12-26 22:33:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19933.9, 300 sec: 19494.2). Total num frames: 505110528. Throughput: 0: 9846.5, 1: 9923.9. Samples: 505074820. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:33:16,062][104569] Avg episode reward: [(0, '8191.686'), (1, '9355.312')] [2023-12-26 22:33:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000986536_252583936.pth... [2023-12-26 22:33:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000986280_252526592.pth... [2023-12-26 22:33:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000985352_252280832.pth [2023-12-26 22:33:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000985128_252231680.pth [2023-12-26 22:33:16,493][105620] Updated weights for policy 1, policy_version 986546 (0.0005) [2023-12-26 22:33:16,555][105620] Updated weights for policy 1, policy_version 986556 (0.0010) [2023-12-26 22:33:16,613][105620] Updated weights for policy 1, policy_version 986566 (0.0011) [2023-12-26 22:33:16,675][105692] Updated weights for policy 0, policy_version 986283 (0.0010) [2023-12-26 22:33:16,720][105692] Updated weights for policy 0, policy_version 986293 (0.0010) [2023-12-26 22:33:16,768][105692] Updated weights for policy 0, policy_version 986303 (0.0008) [2023-12-26 22:33:17,282][105620] Updated weights for policy 1, policy_version 986576 (0.0009) [2023-12-26 22:33:17,338][105620] Updated weights for policy 1, policy_version 986586 (0.0008) [2023-12-26 22:33:17,394][105620] Updated weights for policy 1, policy_version 986596 (0.0009) [2023-12-26 22:33:17,413][105692] Updated weights for policy 0, policy_version 986313 (0.0005) [2023-12-26 22:33:17,479][105692] Updated weights for policy 0, policy_version 986323 (0.0006) [2023-12-26 22:33:17,531][105692] Updated weights for policy 0, policy_version 986333 (0.0006) [2023-12-26 22:33:17,586][105692] Updated weights for policy 0, policy_version 986343 (0.0006) [2023-12-26 22:33:18,102][105620] Updated weights for policy 1, policy_version 986606 (0.0007) [2023-12-26 22:33:18,160][105620] Updated weights for policy 1, policy_version 986616 (0.0005) [2023-12-26 22:33:18,226][105620] Updated weights for policy 1, policy_version 986626 (0.0006) [2023-12-26 22:33:18,292][105692] Updated weights for policy 0, policy_version 986353 (0.0007) [2023-12-26 22:33:18,364][105692] Updated weights for policy 0, policy_version 986363 (0.0008) [2023-12-26 22:33:18,428][105692] Updated weights for policy 0, policy_version 986373 (0.0006) [2023-12-26 22:33:18,849][105620] Updated weights for policy 1, policy_version 986636 (0.0008) [2023-12-26 22:33:18,912][105620] Updated weights for policy 1, policy_version 986646 (0.0008) [2023-12-26 22:33:18,966][105620] Updated weights for policy 1, policy_version 986656 (0.0006) [2023-12-26 22:33:19,189][105692] Updated weights for policy 0, policy_version 986383 (0.0006) [2023-12-26 22:33:19,249][105692] Updated weights for policy 0, policy_version 986393 (0.0007) [2023-12-26 22:33:19,304][105692] Updated weights for policy 0, policy_version 986403 (0.0009) [2023-12-26 22:33:19,744][105620] Updated weights for policy 1, policy_version 986666 (0.0009) [2023-12-26 22:33:19,798][105620] Updated weights for policy 1, policy_version 986676 (0.0010) [2023-12-26 22:33:19,865][105620] Updated weights for policy 1, policy_version 986686 (0.0009) [2023-12-26 22:33:19,930][105620] Updated weights for policy 1, policy_version 986696 (0.0010) [2023-12-26 22:33:19,980][105692] Updated weights for policy 0, policy_version 986413 (0.0007) [2023-12-26 22:33:20,035][105692] Updated weights for policy 0, policy_version 986423 (0.0009) [2023-12-26 22:33:20,092][105692] Updated weights for policy 0, policy_version 986433 (0.0010) [2023-12-26 22:33:20,742][105620] Updated weights for policy 1, policy_version 986706 (0.0009) [2023-12-26 22:33:20,743][105692] Updated weights for policy 0, policy_version 986443 (0.0006) [2023-12-26 22:33:20,806][105692] Updated weights for policy 0, policy_version 986453 (0.0006) [2023-12-26 22:33:20,808][105620] Updated weights for policy 1, policy_version 986716 (0.0009) [2023-12-26 22:33:20,866][105620] Updated weights for policy 1, policy_version 986726 (0.0006) [2023-12-26 22:33:20,868][105692] Updated weights for policy 0, policy_version 986463 (0.0008) [2023-12-26 22:33:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19933.9, 300 sec: 19494.2). Total num frames: 505208832. Throughput: 0: 9709.0, 1: 10027.7. Samples: 505193492. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:33:21,062][104569] Avg episode reward: [(0, '8009.219'), (1, '9355.356')] [2023-12-26 22:33:21,628][105692] Updated weights for policy 0, policy_version 986473 (0.0008) [2023-12-26 22:33:21,662][105620] Updated weights for policy 1, policy_version 986736 (0.0010) [2023-12-26 22:33:21,688][105692] Updated weights for policy 0, policy_version 986483 (0.0007) [2023-12-26 22:33:21,727][105620] Updated weights for policy 1, policy_version 986746 (0.0009) [2023-12-26 22:33:21,762][105692] Updated weights for policy 0, policy_version 986493 (0.0007) [2023-12-26 22:33:21,794][105620] Updated weights for policy 1, policy_version 986756 (0.0008) [2023-12-26 22:33:21,825][105692] Updated weights for policy 0, policy_version 986503 (0.0006) [2023-12-26 22:33:22,541][105692] Updated weights for policy 0, policy_version 986513 (0.0008) [2023-12-26 22:33:22,583][105620] Updated weights for policy 1, policy_version 986766 (0.0007) [2023-12-26 22:33:22,605][105692] Updated weights for policy 0, policy_version 986523 (0.0009) [2023-12-26 22:33:22,638][105620] Updated weights for policy 1, policy_version 986776 (0.0008) [2023-12-26 22:33:22,667][105692] Updated weights for policy 0, policy_version 986533 (0.0008) [2023-12-26 22:33:22,695][105620] Updated weights for policy 1, policy_version 986786 (0.0008) [2023-12-26 22:33:23,331][105692] Updated weights for policy 0, policy_version 986543 (0.0007) [2023-12-26 22:33:23,393][105692] Updated weights for policy 0, policy_version 986553 (0.0009) [2023-12-26 22:33:23,445][105692] Updated weights for policy 0, policy_version 986563 (0.0009) [2023-12-26 22:33:23,493][105620] Updated weights for policy 1, policy_version 986796 (0.0009) [2023-12-26 22:33:23,547][105620] Updated weights for policy 1, policy_version 986806 (0.0009) [2023-12-26 22:33:23,593][105620] Updated weights for policy 1, policy_version 986816 (0.0009) [2023-12-26 22:33:24,053][105692] Updated weights for policy 0, policy_version 986573 (0.0008) [2023-12-26 22:33:24,109][105692] Updated weights for policy 0, policy_version 986583 (0.0009) [2023-12-26 22:33:24,165][105692] Updated weights for policy 0, policy_version 986593 (0.0009) [2023-12-26 22:33:24,424][105620] Updated weights for policy 1, policy_version 986826 (0.0009) [2023-12-26 22:33:24,477][105620] Updated weights for policy 1, policy_version 986836 (0.0006) [2023-12-26 22:33:24,525][105620] Updated weights for policy 1, policy_version 986846 (0.0009) [2023-12-26 22:33:24,587][105620] Updated weights for policy 1, policy_version 986856 (0.0006) [2023-12-26 22:33:24,830][105692] Updated weights for policy 0, policy_version 986603 (0.0008) [2023-12-26 22:33:24,900][105692] Updated weights for policy 0, policy_version 986613 (0.0006) [2023-12-26 22:33:24,969][105692] Updated weights for policy 0, policy_version 986623 (0.0005) [2023-12-26 22:33:25,254][105620] Updated weights for policy 1, policy_version 986866 (0.0008) [2023-12-26 22:33:25,307][105620] Updated weights for policy 1, policy_version 986876 (0.0005) [2023-12-26 22:33:25,366][105620] Updated weights for policy 1, policy_version 986886 (0.0005) [2023-12-26 22:33:25,624][105692] Updated weights for policy 0, policy_version 986633 (0.0010) [2023-12-26 22:33:25,672][105692] Updated weights for policy 0, policy_version 986643 (0.0009) [2023-12-26 22:33:25,728][105692] Updated weights for policy 0, policy_version 986653 (0.0005) [2023-12-26 22:33:25,786][105692] Updated weights for policy 0, policy_version 986663 (0.0006) [2023-12-26 22:33:25,958][105620] Updated weights for policy 1, policy_version 986896 (0.0005) [2023-12-26 22:33:26,012][105620] Updated weights for policy 1, policy_version 986906 (0.0006) [2023-12-26 22:33:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19797.4, 300 sec: 19438.7). Total num frames: 505298944. Throughput: 0: 9771.7, 1: 9951.0. Samples: 505310920. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:33:26,062][104569] Avg episode reward: [(0, '8279.922'), (1, '9263.551')] [2023-12-26 22:33:26,064][105620] Updated weights for policy 1, policy_version 986916 (0.0010) [2023-12-26 22:33:26,330][105692] Updated weights for policy 0, policy_version 986673 (0.0010) [2023-12-26 22:33:26,382][105692] Updated weights for policy 0, policy_version 986683 (0.0010) [2023-12-26 22:33:26,434][105692] Updated weights for policy 0, policy_version 986693 (0.0010) [2023-12-26 22:33:26,779][105620] Updated weights for policy 1, policy_version 986926 (0.0011) [2023-12-26 22:33:26,831][105620] Updated weights for policy 1, policy_version 986936 (0.0010) [2023-12-26 22:33:26,889][105620] Updated weights for policy 1, policy_version 986946 (0.0010) [2023-12-26 22:33:27,186][105692] Updated weights for policy 0, policy_version 986703 (0.0011) [2023-12-26 22:33:27,242][105692] Updated weights for policy 0, policy_version 986713 (0.0011) [2023-12-26 22:33:27,304][105692] Updated weights for policy 0, policy_version 986723 (0.0010) [2023-12-26 22:33:27,544][105620] Updated weights for policy 1, policy_version 986956 (0.0008) [2023-12-26 22:33:27,603][105620] Updated weights for policy 1, policy_version 986966 (0.0005) [2023-12-26 22:33:27,653][105620] Updated weights for policy 1, policy_version 986976 (0.0005) [2023-12-26 22:33:28,015][105692] Updated weights for policy 0, policy_version 986733 (0.0008) [2023-12-26 22:33:28,082][105692] Updated weights for policy 0, policy_version 986743 (0.0006) [2023-12-26 22:33:28,149][105692] Updated weights for policy 0, policy_version 986753 (0.0006) [2023-12-26 22:33:28,169][105620] Updated weights for policy 1, policy_version 986986 (0.0006) [2023-12-26 22:33:28,220][105620] Updated weights for policy 1, policy_version 986996 (0.0010) [2023-12-26 22:33:28,274][105620] Updated weights for policy 1, policy_version 987006 (0.0010) [2023-12-26 22:33:28,323][105620] Updated weights for policy 1, policy_version 987016 (0.0010) [2023-12-26 22:33:28,772][105692] Updated weights for policy 0, policy_version 986763 (0.0007) [2023-12-26 22:33:28,817][105692] Updated weights for policy 0, policy_version 986773 (0.0008) [2023-12-26 22:33:28,862][105692] Updated weights for policy 0, policy_version 986783 (0.0008) [2023-12-26 22:33:29,085][105620] Updated weights for policy 1, policy_version 987026 (0.0010) [2023-12-26 22:33:29,136][105620] Updated weights for policy 1, policy_version 987036 (0.0010) [2023-12-26 22:33:29,198][105620] Updated weights for policy 1, policy_version 987046 (0.0010) [2023-12-26 22:33:29,680][105692] Updated weights for policy 0, policy_version 986793 (0.0008) [2023-12-26 22:33:29,733][105692] Updated weights for policy 0, policy_version 986804 (0.0010) [2023-12-26 22:33:29,796][105692] Updated weights for policy 0, policy_version 986814 (0.0009) [2023-12-26 22:33:29,868][105692] Updated weights for policy 0, policy_version 986824 (0.0008) [2023-12-26 22:33:29,889][105620] Updated weights for policy 1, policy_version 987056 (0.0011) [2023-12-26 22:33:29,957][105620] Updated weights for policy 1, policy_version 987066 (0.0011) [2023-12-26 22:33:30,013][105620] Updated weights for policy 1, policy_version 987076 (0.0010) [2023-12-26 22:33:30,520][105692] Updated weights for policy 0, policy_version 986834 (0.0005) [2023-12-26 22:33:30,574][105692] Updated weights for policy 0, policy_version 986844 (0.0005) [2023-12-26 22:33:30,620][105692] Updated weights for policy 0, policy_version 986854 (0.0005) [2023-12-26 22:33:30,683][105620] Updated weights for policy 1, policy_version 987086 (0.0007) [2023-12-26 22:33:30,738][105620] Updated weights for policy 1, policy_version 987096 (0.0005) [2023-12-26 22:33:30,789][105620] Updated weights for policy 1, policy_version 987106 (0.0005) [2023-12-26 22:33:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 505405440. Throughput: 0: 9821.1, 1: 10049.3. Samples: 505373816. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:33:31,063][104569] Avg episode reward: [(0, '8549.158'), (1, '9263.562')] [2023-12-26 22:33:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000987112_252731392.pth... [2023-12-26 22:33:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000986856_252674048.pth... [2023-12-26 22:33:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000985928_252428288.pth [2023-12-26 22:33:31,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000985704_252379136.pth [2023-12-26 22:33:31,179][105692] Updated weights for policy 0, policy_version 986864 (0.0007) [2023-12-26 22:33:31,245][105692] Updated weights for policy 0, policy_version 986874 (0.0006) [2023-12-26 22:33:31,311][105692] Updated weights for policy 0, policy_version 986884 (0.0007) [2023-12-26 22:33:31,489][105620] Updated weights for policy 1, policy_version 987116 (0.0006) [2023-12-26 22:33:31,541][105620] Updated weights for policy 1, policy_version 987126 (0.0006) [2023-12-26 22:33:31,604][105620] Updated weights for policy 1, policy_version 987136 (0.0006) [2023-12-26 22:33:32,046][105692] Updated weights for policy 0, policy_version 986894 (0.0009) [2023-12-26 22:33:32,105][105692] Updated weights for policy 0, policy_version 986904 (0.0009) [2023-12-26 22:33:32,166][105692] Updated weights for policy 0, policy_version 986914 (0.0008) [2023-12-26 22:33:32,302][105620] Updated weights for policy 1, policy_version 987146 (0.0009) [2023-12-26 22:33:32,368][105620] Updated weights for policy 1, policy_version 987156 (0.0008) [2023-12-26 22:33:32,429][105620] Updated weights for policy 1, policy_version 987166 (0.0009) [2023-12-26 22:33:32,495][105620] Updated weights for policy 1, policy_version 987176 (0.0008) [2023-12-26 22:33:33,003][105692] Updated weights for policy 0, policy_version 986924 (0.0009) [2023-12-26 22:33:33,056][105692] Updated weights for policy 0, policy_version 986934 (0.0009) [2023-12-26 22:33:33,083][105620] Updated weights for policy 1, policy_version 987186 (0.0006) [2023-12-26 22:33:33,116][105692] Updated weights for policy 0, policy_version 986944 (0.0008) [2023-12-26 22:33:33,144][105620] Updated weights for policy 1, policy_version 987196 (0.0007) [2023-12-26 22:33:33,203][105620] Updated weights for policy 1, policy_version 987206 (0.0005) [2023-12-26 22:33:33,831][105620] Updated weights for policy 1, policy_version 987216 (0.0009) [2023-12-26 22:33:33,891][105620] Updated weights for policy 1, policy_version 987226 (0.0009) [2023-12-26 22:33:33,923][105692] Updated weights for policy 0, policy_version 986954 (0.0006) [2023-12-26 22:33:33,955][105620] Updated weights for policy 1, policy_version 987236 (0.0008) [2023-12-26 22:33:33,977][105692] Updated weights for policy 0, policy_version 986964 (0.0009) [2023-12-26 22:33:34,043][105692] Updated weights for policy 0, policy_version 986974 (0.0010) [2023-12-26 22:33:34,096][105692] Updated weights for policy 0, policy_version 986984 (0.0010) [2023-12-26 22:33:34,542][105620] Updated weights for policy 1, policy_version 987246 (0.0006) [2023-12-26 22:33:34,606][105620] Updated weights for policy 1, policy_version 987256 (0.0007) [2023-12-26 22:33:34,665][105620] Updated weights for policy 1, policy_version 987266 (0.0009) [2023-12-26 22:33:34,955][105692] Updated weights for policy 0, policy_version 986994 (0.0009) [2023-12-26 22:33:35,015][105692] Updated weights for policy 0, policy_version 987004 (0.0009) [2023-12-26 22:33:35,072][105692] Updated weights for policy 0, policy_version 987014 (0.0009) [2023-12-26 22:33:35,348][105620] Updated weights for policy 1, policy_version 987276 (0.0007) [2023-12-26 22:33:35,395][105620] Updated weights for policy 1, policy_version 987286 (0.0009) [2023-12-26 22:33:35,445][105620] Updated weights for policy 1, policy_version 987296 (0.0009) [2023-12-26 22:33:35,852][105692] Updated weights for policy 0, policy_version 987024 (0.0009) [2023-12-26 22:33:35,915][105692] Updated weights for policy 0, policy_version 987034 (0.0009) [2023-12-26 22:33:35,972][105692] Updated weights for policy 0, policy_version 987044 (0.0009) [2023-12-26 22:33:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 505503744. Throughput: 0: 9806.7, 1: 10022.6. Samples: 505492308. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:33:36,062][104569] Avg episode reward: [(0, '8372.239'), (1, '9263.372')] [2023-12-26 22:33:36,206][105620] Updated weights for policy 1, policy_version 987306 (0.0009) [2023-12-26 22:33:36,268][105620] Updated weights for policy 1, policy_version 987316 (0.0008) [2023-12-26 22:33:36,339][105620] Updated weights for policy 1, policy_version 987326 (0.0005) [2023-12-26 22:33:36,404][105620] Updated weights for policy 1, policy_version 987336 (0.0006) [2023-12-26 22:33:36,732][105692] Updated weights for policy 0, policy_version 987054 (0.0009) [2023-12-26 22:33:36,790][105692] Updated weights for policy 0, policy_version 987064 (0.0010) [2023-12-26 22:33:36,850][105692] Updated weights for policy 0, policy_version 987074 (0.0010) [2023-12-26 22:33:37,093][105620] Updated weights for policy 1, policy_version 987346 (0.0008) [2023-12-26 22:33:37,159][105620] Updated weights for policy 1, policy_version 987356 (0.0009) [2023-12-26 22:33:37,221][105620] Updated weights for policy 1, policy_version 987366 (0.0010) [2023-12-26 22:33:37,576][105692] Updated weights for policy 0, policy_version 987084 (0.0009) [2023-12-26 22:33:37,641][105692] Updated weights for policy 0, policy_version 987094 (0.0010) [2023-12-26 22:33:37,693][105692] Updated weights for policy 0, policy_version 987104 (0.0009) [2023-12-26 22:33:38,048][105620] Updated weights for policy 1, policy_version 987376 (0.0009) [2023-12-26 22:33:38,104][105620] Updated weights for policy 1, policy_version 987386 (0.0009) [2023-12-26 22:33:38,162][105620] Updated weights for policy 1, policy_version 987396 (0.0010) [2023-12-26 22:33:38,336][105692] Updated weights for policy 0, policy_version 987114 (0.0008) [2023-12-26 22:33:38,404][105692] Updated weights for policy 0, policy_version 987124 (0.0011) [2023-12-26 22:33:38,466][105692] Updated weights for policy 0, policy_version 987134 (0.0011) [2023-12-26 22:33:38,535][105692] Updated weights for policy 0, policy_version 987144 (0.0010) [2023-12-26 22:33:38,929][105620] Updated weights for policy 1, policy_version 987406 (0.0007) [2023-12-26 22:33:38,989][105620] Updated weights for policy 1, policy_version 987416 (0.0009) [2023-12-26 22:33:39,049][105620] Updated weights for policy 1, policy_version 987426 (0.0010) [2023-12-26 22:33:39,225][105692] Updated weights for policy 0, policy_version 987154 (0.0009) [2023-12-26 22:33:39,253][105585] KL-divergence is very high: 123.3359 [2023-12-26 22:33:39,293][105692] Updated weights for policy 0, policy_version 987164 (0.0011) [2023-12-26 22:33:39,302][105585] KL-divergence is very high: 227.6443 [2023-12-26 22:33:39,354][105585] KL-divergence is very high: 255.1516 [2023-12-26 22:33:39,354][105692] Updated weights for policy 0, policy_version 987174 (0.0010) [2023-12-26 22:33:39,798][105620] Updated weights for policy 1, policy_version 987436 (0.0010) [2023-12-26 22:33:39,859][105620] Updated weights for policy 1, policy_version 987446 (0.0009) [2023-12-26 22:33:39,914][105620] Updated weights for policy 1, policy_version 987456 (0.0009) [2023-12-26 22:33:39,984][105692] Updated weights for policy 0, policy_version 987184 (0.0010) [2023-12-26 22:33:40,048][105692] Updated weights for policy 0, policy_version 987194 (0.0011) [2023-12-26 22:33:40,108][105692] Updated weights for policy 0, policy_version 987204 (0.0011) [2023-12-26 22:33:40,713][105620] Updated weights for policy 1, policy_version 987466 (0.0007) [2023-12-26 22:33:40,780][105620] Updated weights for policy 1, policy_version 987476 (0.0007) [2023-12-26 22:33:40,783][105692] Updated weights for policy 0, policy_version 987214 (0.0010) [2023-12-26 22:33:40,840][105620] Updated weights for policy 1, policy_version 987486 (0.0008) [2023-12-26 22:33:40,843][105692] Updated weights for policy 0, policy_version 987224 (0.0011) [2023-12-26 22:33:40,889][105620] Updated weights for policy 1, policy_version 987496 (0.0010) [2023-12-26 22:33:40,895][105692] Updated weights for policy 0, policy_version 987234 (0.0010) [2023-12-26 22:33:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 505602048. Throughput: 0: 9826.5, 1: 9925.4. Samples: 505606332. Policy #0 lag: (min: 10.0, avg: 15.8, max: 42.0) [2023-12-26 22:33:41,063][104569] Avg episode reward: [(0, '8277.307'), (1, '9171.483')] [2023-12-26 22:33:41,661][105620] Updated weights for policy 1, policy_version 987506 (0.0011) [2023-12-26 22:33:41,676][105692] Updated weights for policy 0, policy_version 987244 (0.0010) [2023-12-26 22:33:41,722][105620] Updated weights for policy 1, policy_version 987516 (0.0010) [2023-12-26 22:33:41,741][105692] Updated weights for policy 0, policy_version 987254 (0.0008) [2023-12-26 22:33:41,785][105620] Updated weights for policy 1, policy_version 987526 (0.0011) [2023-12-26 22:33:41,800][105692] Updated weights for policy 0, policy_version 987264 (0.0007) [2023-12-26 22:33:42,465][105620] Updated weights for policy 1, policy_version 987536 (0.0009) [2023-12-26 22:33:42,518][105620] Updated weights for policy 1, policy_version 987546 (0.0009) [2023-12-26 22:33:42,536][105692] Updated weights for policy 0, policy_version 987274 (0.0008) [2023-12-26 22:33:42,584][105620] Updated weights for policy 1, policy_version 987556 (0.0006) [2023-12-26 22:33:42,591][105692] Updated weights for policy 0, policy_version 987284 (0.0008) [2023-12-26 22:33:42,651][105692] Updated weights for policy 0, policy_version 987294 (0.0008) [2023-12-26 22:33:42,713][105692] Updated weights for policy 0, policy_version 987304 (0.0009) [2023-12-26 22:33:43,331][105620] Updated weights for policy 1, policy_version 987566 (0.0009) [2023-12-26 22:33:43,391][105620] Updated weights for policy 1, policy_version 987576 (0.0009) [2023-12-26 22:33:43,444][105620] Updated weights for policy 1, policy_version 987586 (0.0008) [2023-12-26 22:33:43,467][105692] Updated weights for policy 0, policy_version 987314 (0.0006) [2023-12-26 22:33:43,527][105692] Updated weights for policy 0, policy_version 987324 (0.0008) [2023-12-26 22:33:43,588][105692] Updated weights for policy 0, policy_version 987334 (0.0008) [2023-12-26 22:33:44,086][105620] Updated weights for policy 1, policy_version 987596 (0.0008) [2023-12-26 22:33:44,148][105620] Updated weights for policy 1, policy_version 987606 (0.0007) [2023-12-26 22:33:44,209][105620] Updated weights for policy 1, policy_version 987616 (0.0008) [2023-12-26 22:33:44,362][105692] Updated weights for policy 0, policy_version 987344 (0.0006) [2023-12-26 22:33:44,429][105692] Updated weights for policy 0, policy_version 987354 (0.0005) [2023-12-26 22:33:44,483][105692] Updated weights for policy 0, policy_version 987364 (0.0005) [2023-12-26 22:33:44,844][105620] Updated weights for policy 1, policy_version 987626 (0.0009) [2023-12-26 22:33:44,908][105620] Updated weights for policy 1, policy_version 987636 (0.0006) [2023-12-26 22:33:44,980][105620] Updated weights for policy 1, policy_version 987646 (0.0006) [2023-12-26 22:33:45,021][105692] Updated weights for policy 0, policy_version 987374 (0.0006) [2023-12-26 22:33:45,036][105620] Updated weights for policy 1, policy_version 987656 (0.0006) [2023-12-26 22:33:45,089][105692] Updated weights for policy 0, policy_version 987384 (0.0007) [2023-12-26 22:33:45,156][105692] Updated weights for policy 0, policy_version 987394 (0.0010) [2023-12-26 22:33:45,601][105620] Updated weights for policy 1, policy_version 987666 (0.0005) [2023-12-26 22:33:45,645][105620] Updated weights for policy 1, policy_version 987676 (0.0005) [2023-12-26 22:33:45,692][105620] Updated weights for policy 1, policy_version 987686 (0.0005) [2023-12-26 22:33:45,792][105692] Updated weights for policy 0, policy_version 987404 (0.0006) [2023-12-26 22:33:45,854][105692] Updated weights for policy 0, policy_version 987414 (0.0005) [2023-12-26 22:33:45,931][105692] Updated weights for policy 0, policy_version 987424 (0.0007) [2023-12-26 22:33:46,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 505700352. Throughput: 0: 9786.9, 1: 9896.4. Samples: 505663436. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:33:46,063][104569] Avg episode reward: [(0, '8465.066'), (1, '9263.582')] [2023-12-26 22:33:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000987432_252821504.pth... [2023-12-26 22:33:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000987688_252878848.pth... [2023-12-26 22:33:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000986536_252583936.pth [2023-12-26 22:33:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000986280_252526592.pth [2023-12-26 22:33:46,331][105620] Updated weights for policy 1, policy_version 987696 (0.0007) [2023-12-26 22:33:46,378][105620] Updated weights for policy 1, policy_version 987706 (0.0008) [2023-12-26 22:33:46,427][105620] Updated weights for policy 1, policy_version 987716 (0.0008) [2023-12-26 22:33:46,505][105692] Updated weights for policy 0, policy_version 987434 (0.0006) [2023-12-26 22:33:46,564][105692] Updated weights for policy 0, policy_version 987444 (0.0010) [2023-12-26 22:33:46,619][105692] Updated weights for policy 0, policy_version 987454 (0.0008) [2023-12-26 22:33:46,672][105692] Updated weights for policy 0, policy_version 987464 (0.0005) [2023-12-26 22:33:47,113][105620] Updated weights for policy 1, policy_version 987726 (0.0009) [2023-12-26 22:33:47,174][105620] Updated weights for policy 1, policy_version 987736 (0.0010) [2023-12-26 22:33:47,240][105620] Updated weights for policy 1, policy_version 987746 (0.0011) [2023-12-26 22:33:47,365][105692] Updated weights for policy 0, policy_version 987474 (0.0010) [2023-12-26 22:33:47,429][105692] Updated weights for policy 0, policy_version 987484 (0.0010) [2023-12-26 22:33:47,492][105692] Updated weights for policy 0, policy_version 987494 (0.0011) [2023-12-26 22:33:47,843][105620] Updated weights for policy 1, policy_version 987756 (0.0010) [2023-12-26 22:33:47,892][105620] Updated weights for policy 1, policy_version 987766 (0.0010) [2023-12-26 22:33:47,946][105620] Updated weights for policy 1, policy_version 987776 (0.0010) [2023-12-26 22:33:48,178][105692] Updated weights for policy 0, policy_version 987504 (0.0006) [2023-12-26 22:33:48,233][105692] Updated weights for policy 0, policy_version 987514 (0.0007) [2023-12-26 22:33:48,287][105692] Updated weights for policy 0, policy_version 987524 (0.0008) [2023-12-26 22:33:48,648][105620] Updated weights for policy 1, policy_version 987786 (0.0010) [2023-12-26 22:33:48,710][105620] Updated weights for policy 1, policy_version 987796 (0.0010) [2023-12-26 22:33:48,769][105620] Updated weights for policy 1, policy_version 987806 (0.0007) [2023-12-26 22:33:48,830][105620] Updated weights for policy 1, policy_version 987816 (0.0006) [2023-12-26 22:33:49,057][105692] Updated weights for policy 0, policy_version 987534 (0.0008) [2023-12-26 22:33:49,123][105692] Updated weights for policy 0, policy_version 987544 (0.0008) [2023-12-26 22:33:49,186][105692] Updated weights for policy 0, policy_version 987554 (0.0008) [2023-12-26 22:33:49,504][105620] Updated weights for policy 1, policy_version 987826 (0.0011) [2023-12-26 22:33:49,557][105620] Updated weights for policy 1, policy_version 987836 (0.0010) [2023-12-26 22:33:49,606][105620] Updated weights for policy 1, policy_version 987846 (0.0010) [2023-12-26 22:33:49,912][105692] Updated weights for policy 0, policy_version 987564 (0.0009) [2023-12-26 22:33:49,966][105692] Updated weights for policy 0, policy_version 987574 (0.0007) [2023-12-26 22:33:50,024][105692] Updated weights for policy 0, policy_version 987584 (0.0005) [2023-12-26 22:33:50,415][105620] Updated weights for policy 1, policy_version 987856 (0.0010) [2023-12-26 22:33:50,477][105620] Updated weights for policy 1, policy_version 987866 (0.0011) [2023-12-26 22:33:50,544][105620] Updated weights for policy 1, policy_version 987876 (0.0006) [2023-12-26 22:33:50,834][105692] Updated weights for policy 0, policy_version 987594 (0.0008) [2023-12-26 22:33:50,893][105692] Updated weights for policy 0, policy_version 987604 (0.0008) [2023-12-26 22:33:50,943][105692] Updated weights for policy 0, policy_version 987614 (0.0009) [2023-12-26 22:33:51,003][105692] Updated weights for policy 0, policy_version 987624 (0.0008) [2023-12-26 22:33:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 505798656. Throughput: 0: 9834.8, 1: 9966.1. Samples: 505787312. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:33:51,062][104569] Avg episode reward: [(0, '8734.769'), (1, '9355.558')] [2023-12-26 22:33:51,243][105620] Updated weights for policy 1, policy_version 987886 (0.0009) [2023-12-26 22:33:51,306][105620] Updated weights for policy 1, policy_version 987896 (0.0011) [2023-12-26 22:33:51,365][105620] Updated weights for policy 1, policy_version 987906 (0.0011) [2023-12-26 22:33:51,798][105692] Updated weights for policy 0, policy_version 987634 (0.0009) [2023-12-26 22:33:51,856][105692] Updated weights for policy 0, policy_version 987644 (0.0010) [2023-12-26 22:33:51,909][105692] Updated weights for policy 0, policy_version 987654 (0.0010) [2023-12-26 22:33:52,047][105620] Updated weights for policy 1, policy_version 987916 (0.0009) [2023-12-26 22:33:52,106][105620] Updated weights for policy 1, policy_version 987926 (0.0010) [2023-12-26 22:33:52,158][105620] Updated weights for policy 1, policy_version 987936 (0.0010) [2023-12-26 22:33:52,727][105692] Updated weights for policy 0, policy_version 987664 (0.0008) [2023-12-26 22:33:52,779][105692] Updated weights for policy 0, policy_version 987674 (0.0008) [2023-12-26 22:33:52,824][105692] Updated weights for policy 0, policy_version 987684 (0.0008) [2023-12-26 22:33:52,902][105620] Updated weights for policy 1, policy_version 987946 (0.0010) [2023-12-26 22:33:52,954][105620] Updated weights for policy 1, policy_version 987956 (0.0010) [2023-12-26 22:33:53,012][105620] Updated weights for policy 1, policy_version 987966 (0.0010) [2023-12-26 22:33:53,070][105620] Updated weights for policy 1, policy_version 987976 (0.0010) [2023-12-26 22:33:53,582][105692] Updated weights for policy 0, policy_version 987694 (0.0008) [2023-12-26 22:33:53,637][105692] Updated weights for policy 0, policy_version 987704 (0.0009) [2023-12-26 22:33:53,693][105692] Updated weights for policy 0, policy_version 987717 (0.0011) [2023-12-26 22:33:53,779][105620] Updated weights for policy 1, policy_version 987986 (0.0008) [2023-12-26 22:33:53,843][105620] Updated weights for policy 1, policy_version 987996 (0.0009) [2023-12-26 22:33:53,904][105620] Updated weights for policy 1, policy_version 988006 (0.0009) [2023-12-26 22:33:54,330][105692] Updated weights for policy 0, policy_version 987727 (0.0006) [2023-12-26 22:33:54,383][105692] Updated weights for policy 0, policy_version 987737 (0.0007) [2023-12-26 22:33:54,432][105692] Updated weights for policy 0, policy_version 987747 (0.0008) [2023-12-26 22:33:54,579][105620] Updated weights for policy 1, policy_version 988016 (0.0007) [2023-12-26 22:33:54,638][105620] Updated weights for policy 1, policy_version 988026 (0.0010) [2023-12-26 22:33:54,704][105620] Updated weights for policy 1, policy_version 988036 (0.0010) [2023-12-26 22:33:55,150][105692] Updated weights for policy 0, policy_version 987757 (0.0008) [2023-12-26 22:33:55,198][105692] Updated weights for policy 0, policy_version 987767 (0.0008) [2023-12-26 22:33:55,242][105692] Updated weights for policy 0, policy_version 987777 (0.0007) [2023-12-26 22:33:55,419][105620] Updated weights for policy 1, policy_version 988046 (0.0010) [2023-12-26 22:33:55,465][105620] Updated weights for policy 1, policy_version 988056 (0.0010) [2023-12-26 22:33:55,513][105620] Updated weights for policy 1, policy_version 988066 (0.0010) [2023-12-26 22:33:56,023][105692] Updated weights for policy 0, policy_version 987787 (0.0008) [2023-12-26 22:33:56,062][104569] Fps is (10 sec: 18842.3, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 505888768. Throughput: 0: 9746.5, 1: 9931.9. Samples: 505901908. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:33:56,062][104569] Avg episode reward: [(0, '8818.801'), (1, '9355.604')] [2023-12-26 22:33:56,082][105692] Updated weights for policy 0, policy_version 987797 (0.0008) [2023-12-26 22:33:56,142][105692] Updated weights for policy 0, policy_version 987807 (0.0008) [2023-12-26 22:33:56,282][105620] Updated weights for policy 1, policy_version 988076 (0.0010) [2023-12-26 22:33:56,337][105620] Updated weights for policy 1, policy_version 988086 (0.0010) [2023-12-26 22:33:56,384][105620] Updated weights for policy 1, policy_version 988096 (0.0010) [2023-12-26 22:33:56,886][105692] Updated weights for policy 0, policy_version 987817 (0.0008) [2023-12-26 22:33:56,938][105692] Updated weights for policy 0, policy_version 987827 (0.0008) [2023-12-26 22:33:56,992][105692] Updated weights for policy 0, policy_version 987837 (0.0008) [2023-12-26 22:33:57,045][105692] Updated weights for policy 0, policy_version 987847 (0.0008) [2023-12-26 22:33:57,131][105620] Updated weights for policy 1, policy_version 988106 (0.0010) [2023-12-26 22:33:57,179][105620] Updated weights for policy 1, policy_version 988116 (0.0010) [2023-12-26 22:33:57,230][105620] Updated weights for policy 1, policy_version 988126 (0.0010) [2023-12-26 22:33:57,284][105620] Updated weights for policy 1, policy_version 988136 (0.0010) [2023-12-26 22:33:57,808][105692] Updated weights for policy 0, policy_version 987857 (0.0008) [2023-12-26 22:33:57,859][105692] Updated weights for policy 0, policy_version 987867 (0.0007) [2023-12-26 22:33:57,915][105692] Updated weights for policy 0, policy_version 987877 (0.0008) [2023-12-26 22:33:58,059][105620] Updated weights for policy 1, policy_version 988146 (0.0010) [2023-12-26 22:33:58,111][105620] Updated weights for policy 1, policy_version 988156 (0.0010) [2023-12-26 22:33:58,165][105620] Updated weights for policy 1, policy_version 988166 (0.0010) [2023-12-26 22:33:58,741][105692] Updated weights for policy 0, policy_version 987887 (0.0008) [2023-12-26 22:33:58,805][105692] Updated weights for policy 0, policy_version 987897 (0.0010) [2023-12-26 22:33:58,872][105692] Updated weights for policy 0, policy_version 987907 (0.0010) [2023-12-26 22:33:58,898][105620] Updated weights for policy 1, policy_version 988176 (0.0007) [2023-12-26 22:33:58,963][105620] Updated weights for policy 1, policy_version 988186 (0.0008) [2023-12-26 22:33:59,016][105620] Updated weights for policy 1, policy_version 988196 (0.0008) [2023-12-26 22:33:59,629][105692] Updated weights for policy 0, policy_version 987917 (0.0010) [2023-12-26 22:33:59,685][105692] Updated weights for policy 0, policy_version 987927 (0.0009) [2023-12-26 22:33:59,734][105692] Updated weights for policy 0, policy_version 987937 (0.0009) [2023-12-26 22:33:59,783][105620] Updated weights for policy 1, policy_version 988206 (0.0009) [2023-12-26 22:33:59,842][105620] Updated weights for policy 1, policy_version 988216 (0.0010) [2023-12-26 22:33:59,905][105620] Updated weights for policy 1, policy_version 988226 (0.0006) [2023-12-26 22:34:00,435][105692] Updated weights for policy 0, policy_version 987947 (0.0009) [2023-12-26 22:34:00,479][105620] Updated weights for policy 1, policy_version 988236 (0.0007) [2023-12-26 22:34:00,497][105692] Updated weights for policy 0, policy_version 987957 (0.0008) [2023-12-26 22:34:00,528][105620] Updated weights for policy 1, policy_version 988246 (0.0006) [2023-12-26 22:34:00,547][105692] Updated weights for policy 0, policy_version 987967 (0.0006) [2023-12-26 22:34:00,577][105620] Updated weights for policy 1, policy_version 988256 (0.0007) [2023-12-26 22:34:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 505987072. Throughput: 0: 9728.7, 1: 9898.6. Samples: 505958048. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:34:01,062][104569] Avg episode reward: [(0, '8821.659'), (1, '9355.522')] [2023-12-26 22:34:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000987976_252960768.pth... [2023-12-26 22:34:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000988264_253026304.pth... [2023-12-26 22:34:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000986856_252674048.pth [2023-12-26 22:34:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000987112_252731392.pth [2023-12-26 22:34:01,192][105620] Updated weights for policy 1, policy_version 988266 (0.0005) [2023-12-26 22:34:01,258][105620] Updated weights for policy 1, policy_version 988276 (0.0006) [2023-12-26 22:34:01,312][105620] Updated weights for policy 1, policy_version 988286 (0.0009) [2023-12-26 22:34:01,381][105620] Updated weights for policy 1, policy_version 988296 (0.0009) [2023-12-26 22:34:01,403][105692] Updated weights for policy 0, policy_version 987977 (0.0009) [2023-12-26 22:34:01,470][105692] Updated weights for policy 0, policy_version 987987 (0.0009) [2023-12-26 22:34:01,540][105692] Updated weights for policy 0, policy_version 987997 (0.0010) [2023-12-26 22:34:01,609][105692] Updated weights for policy 0, policy_version 988007 (0.0010) [2023-12-26 22:34:02,093][105620] Updated weights for policy 1, policy_version 988306 (0.0008) [2023-12-26 22:34:02,148][105620] Updated weights for policy 1, policy_version 988316 (0.0009) [2023-12-26 22:34:02,203][105620] Updated weights for policy 1, policy_version 988326 (0.0008) [2023-12-26 22:34:02,387][105692] Updated weights for policy 0, policy_version 988017 (0.0008) [2023-12-26 22:34:02,439][105692] Updated weights for policy 0, policy_version 988027 (0.0009) [2023-12-26 22:34:02,491][105692] Updated weights for policy 0, policy_version 988037 (0.0009) [2023-12-26 22:34:02,938][105620] Updated weights for policy 1, policy_version 988336 (0.0009) [2023-12-26 22:34:02,985][105620] Updated weights for policy 1, policy_version 988346 (0.0009) [2023-12-26 22:34:03,035][105620] Updated weights for policy 1, policy_version 988356 (0.0009) [2023-12-26 22:34:03,271][105692] Updated weights for policy 0, policy_version 988047 (0.0009) [2023-12-26 22:34:03,329][105692] Updated weights for policy 0, policy_version 988057 (0.0008) [2023-12-26 22:34:03,379][105692] Updated weights for policy 0, policy_version 988067 (0.0009) [2023-12-26 22:34:03,742][105620] Updated weights for policy 1, policy_version 988366 (0.0007) [2023-12-26 22:34:03,795][105620] Updated weights for policy 1, policy_version 988376 (0.0005) [2023-12-26 22:34:03,862][105620] Updated weights for policy 1, policy_version 988386 (0.0007) [2023-12-26 22:34:04,140][105692] Updated weights for policy 0, policy_version 988077 (0.0007) [2023-12-26 22:34:04,206][105692] Updated weights for policy 0, policy_version 988087 (0.0006) [2023-12-26 22:34:04,270][105692] Updated weights for policy 0, policy_version 988097 (0.0006) [2023-12-26 22:34:04,430][105620] Updated weights for policy 1, policy_version 988396 (0.0006) [2023-12-26 22:34:04,485][105620] Updated weights for policy 1, policy_version 988406 (0.0005) [2023-12-26 22:34:04,546][105620] Updated weights for policy 1, policy_version 988416 (0.0006) [2023-12-26 22:34:04,915][105692] Updated weights for policy 0, policy_version 988107 (0.0006) [2023-12-26 22:34:04,981][105692] Updated weights for policy 0, policy_version 988117 (0.0006) [2023-12-26 22:34:05,035][105692] Updated weights for policy 0, policy_version 988127 (0.0007) [2023-12-26 22:34:05,243][105620] Updated weights for policy 1, policy_version 988426 (0.0009) [2023-12-26 22:34:05,302][105620] Updated weights for policy 1, policy_version 988436 (0.0005) [2023-12-26 22:34:05,348][105620] Updated weights for policy 1, policy_version 988446 (0.0005) [2023-12-26 22:34:05,411][105620] Updated weights for policy 1, policy_version 988456 (0.0005) [2023-12-26 22:34:05,604][105692] Updated weights for policy 0, policy_version 988137 (0.0008) [2023-12-26 22:34:05,652][105692] Updated weights for policy 0, policy_version 988147 (0.0009) [2023-12-26 22:34:05,701][105692] Updated weights for policy 0, policy_version 988157 (0.0008) [2023-12-26 22:34:05,750][105692] Updated weights for policy 0, policy_version 988167 (0.0005) [2023-12-26 22:34:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 506085376. Throughput: 0: 9662.6, 1: 9917.1. Samples: 506074576. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:34:06,062][104569] Avg episode reward: [(0, '8191.295'), (1, '9263.392')] [2023-12-26 22:34:06,088][105620] Updated weights for policy 1, policy_version 988466 (0.0010) [2023-12-26 22:34:06,147][105620] Updated weights for policy 1, policy_version 988476 (0.0009) [2023-12-26 22:34:06,203][105620] Updated weights for policy 1, policy_version 988486 (0.0010) [2023-12-26 22:34:06,350][105692] Updated weights for policy 0, policy_version 988177 (0.0008) [2023-12-26 22:34:06,409][105692] Updated weights for policy 0, policy_version 988187 (0.0009) [2023-12-26 22:34:06,469][105692] Updated weights for policy 0, policy_version 988197 (0.0009) [2023-12-26 22:34:06,993][105620] Updated weights for policy 1, policy_version 988496 (0.0007) [2023-12-26 22:34:07,053][105620] Updated weights for policy 1, policy_version 988506 (0.0006) [2023-12-26 22:34:07,118][105620] Updated weights for policy 1, policy_version 988516 (0.0008) [2023-12-26 22:34:07,296][105692] Updated weights for policy 0, policy_version 988207 (0.0009) [2023-12-26 22:34:07,353][105692] Updated weights for policy 0, policy_version 988217 (0.0009) [2023-12-26 22:34:07,418][105692] Updated weights for policy 0, policy_version 988227 (0.0009) [2023-12-26 22:34:07,752][105620] Updated weights for policy 1, policy_version 988526 (0.0007) [2023-12-26 22:34:07,818][105620] Updated weights for policy 1, policy_version 988536 (0.0009) [2023-12-26 22:34:07,883][105620] Updated weights for policy 1, policy_version 988546 (0.0009) [2023-12-26 22:34:08,210][105692] Updated weights for policy 0, policy_version 988237 (0.0009) [2023-12-26 22:34:08,263][105692] Updated weights for policy 0, policy_version 988247 (0.0010) [2023-12-26 22:34:08,313][105692] Updated weights for policy 0, policy_version 988257 (0.0009) [2023-12-26 22:34:08,557][105620] Updated weights for policy 1, policy_version 988556 (0.0007) [2023-12-26 22:34:08,615][105620] Updated weights for policy 1, policy_version 988566 (0.0007) [2023-12-26 22:34:08,677][105620] Updated weights for policy 1, policy_version 988576 (0.0009) [2023-12-26 22:34:09,137][105692] Updated weights for policy 0, policy_version 988267 (0.0009) [2023-12-26 22:34:09,200][105692] Updated weights for policy 0, policy_version 988277 (0.0009) [2023-12-26 22:34:09,273][105692] Updated weights for policy 0, policy_version 988287 (0.0009) [2023-12-26 22:34:09,398][105620] Updated weights for policy 1, policy_version 988586 (0.0009) [2023-12-26 22:34:09,461][105620] Updated weights for policy 1, policy_version 988596 (0.0006) [2023-12-26 22:34:09,519][105620] Updated weights for policy 1, policy_version 988606 (0.0007) [2023-12-26 22:34:09,577][105620] Updated weights for policy 1, policy_version 988616 (0.0009) [2023-12-26 22:34:10,042][105692] Updated weights for policy 0, policy_version 988297 (0.0009) [2023-12-26 22:34:10,095][105692] Updated weights for policy 0, policy_version 988307 (0.0009) [2023-12-26 22:34:10,150][105692] Updated weights for policy 0, policy_version 988317 (0.0005) [2023-12-26 22:34:10,199][105692] Updated weights for policy 0, policy_version 988327 (0.0007) [2023-12-26 22:34:10,315][105620] Updated weights for policy 1, policy_version 988626 (0.0008) [2023-12-26 22:34:10,369][105620] Updated weights for policy 1, policy_version 988636 (0.0008) [2023-12-26 22:34:10,419][105620] Updated weights for policy 1, policy_version 988646 (0.0009) [2023-12-26 22:34:10,959][105692] Updated weights for policy 0, policy_version 988337 (0.0009) [2023-12-26 22:34:11,019][105692] Updated weights for policy 0, policy_version 988347 (0.0010) [2023-12-26 22:34:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 506175488. Throughput: 0: 9584.0, 1: 9970.1. Samples: 506190856. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:34:11,063][104569] Avg episode reward: [(0, '8366.747'), (1, '9263.321')] [2023-12-26 22:34:11,087][105692] Updated weights for policy 0, policy_version 988357 (0.0009) [2023-12-26 22:34:11,225][105620] Updated weights for policy 1, policy_version 988656 (0.0008) [2023-12-26 22:34:11,295][105620] Updated weights for policy 1, policy_version 988666 (0.0008) [2023-12-26 22:34:11,361][105620] Updated weights for policy 1, policy_version 988676 (0.0009) [2023-12-26 22:34:11,818][105692] Updated weights for policy 0, policy_version 988367 (0.0007) [2023-12-26 22:34:11,873][105692] Updated weights for policy 0, policy_version 988377 (0.0005) [2023-12-26 22:34:11,925][105692] Updated weights for policy 0, policy_version 988387 (0.0005) [2023-12-26 22:34:12,132][105620] Updated weights for policy 1, policy_version 988686 (0.0008) [2023-12-26 22:34:12,191][105620] Updated weights for policy 1, policy_version 988696 (0.0008) [2023-12-26 22:34:12,249][105620] Updated weights for policy 1, policy_version 988706 (0.0010) [2023-12-26 22:34:12,534][105692] Updated weights for policy 0, policy_version 988397 (0.0008) [2023-12-26 22:34:12,595][105692] Updated weights for policy 0, policy_version 988407 (0.0012) [2023-12-26 22:34:12,654][105692] Updated weights for policy 0, policy_version 988417 (0.0007) [2023-12-26 22:34:13,127][105620] Updated weights for policy 1, policy_version 988716 (0.0009) [2023-12-26 22:34:13,185][105620] Updated weights for policy 1, policy_version 988726 (0.0008) [2023-12-26 22:34:13,247][105620] Updated weights for policy 1, policy_version 988736 (0.0008) [2023-12-26 22:34:13,289][105692] Updated weights for policy 0, policy_version 988427 (0.0007) [2023-12-26 22:34:13,341][105692] Updated weights for policy 0, policy_version 988437 (0.0010) [2023-12-26 22:34:13,403][105692] Updated weights for policy 0, policy_version 988447 (0.0008) [2023-12-26 22:34:13,824][105620] Updated weights for policy 1, policy_version 988746 (0.0006) [2023-12-26 22:34:13,877][105620] Updated weights for policy 1, policy_version 988756 (0.0010) [2023-12-26 22:34:13,926][105620] Updated weights for policy 1, policy_version 988766 (0.0010) [2023-12-26 22:34:13,980][105620] Updated weights for policy 1, policy_version 988776 (0.0010) [2023-12-26 22:34:14,132][105692] Updated weights for policy 0, policy_version 988457 (0.0009) [2023-12-26 22:34:14,196][105692] Updated weights for policy 0, policy_version 988467 (0.0010) [2023-12-26 22:34:14,250][105692] Updated weights for policy 0, policy_version 988477 (0.0010) [2023-12-26 22:34:14,298][105692] Updated weights for policy 0, policy_version 988487 (0.0010) [2023-12-26 22:34:14,587][105620] Updated weights for policy 1, policy_version 988786 (0.0006) [2023-12-26 22:34:14,634][105620] Updated weights for policy 1, policy_version 988796 (0.0006) [2023-12-26 22:34:14,682][105620] Updated weights for policy 1, policy_version 988806 (0.0010) [2023-12-26 22:34:15,039][105692] Updated weights for policy 0, policy_version 988497 (0.0010) [2023-12-26 22:34:15,097][105692] Updated weights for policy 0, policy_version 988507 (0.0010) [2023-12-26 22:34:15,166][105692] Updated weights for policy 0, policy_version 988517 (0.0010) [2023-12-26 22:34:15,374][105620] Updated weights for policy 1, policy_version 988816 (0.0011) [2023-12-26 22:34:15,440][105620] Updated weights for policy 1, policy_version 988826 (0.0011) [2023-12-26 22:34:15,504][105620] Updated weights for policy 1, policy_version 988836 (0.0011) [2023-12-26 22:34:15,910][105692] Updated weights for policy 0, policy_version 988527 (0.0010) [2023-12-26 22:34:15,980][105692] Updated weights for policy 0, policy_version 988537 (0.0010) [2023-12-26 22:34:16,028][105692] Updated weights for policy 0, policy_version 988547 (0.0010) [2023-12-26 22:34:16,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 506281984. Throughput: 0: 9562.7, 1: 9885.0. Samples: 506248968. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:34:16,063][104569] Avg episode reward: [(0, '8639.867'), (1, '9355.451')] [2023-12-26 22:34:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000988552_253108224.pth... [2023-12-26 22:34:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000987432_252821504.pth [2023-12-26 22:34:16,102][105620] Updated weights for policy 1, policy_version 988846 (0.0009) [2023-12-26 22:34:16,154][105620] Updated weights for policy 1, policy_version 988856 (0.0008) [2023-12-26 22:34:16,203][105620] Updated weights for policy 1, policy_version 988866 (0.0008) [2023-12-26 22:34:16,232][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000988872_253181952.pth... [2023-12-26 22:34:16,237][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000987688_252878848.pth [2023-12-26 22:34:16,763][105692] Updated weights for policy 0, policy_version 988557 (0.0010) [2023-12-26 22:34:16,832][105692] Updated weights for policy 0, policy_version 988567 (0.0011) [2023-12-26 22:34:16,882][105620] Updated weights for policy 1, policy_version 988876 (0.0009) [2023-12-26 22:34:16,889][105692] Updated weights for policy 0, policy_version 988577 (0.0010) [2023-12-26 22:34:16,938][105620] Updated weights for policy 1, policy_version 988886 (0.0008) [2023-12-26 22:34:16,987][105620] Updated weights for policy 1, policy_version 988896 (0.0008) [2023-12-26 22:34:17,624][105692] Updated weights for policy 0, policy_version 988587 (0.0010) [2023-12-26 22:34:17,650][105620] Updated weights for policy 1, policy_version 988906 (0.0008) [2023-12-26 22:34:17,683][105692] Updated weights for policy 0, policy_version 988597 (0.0010) [2023-12-26 22:34:17,702][105620] Updated weights for policy 1, policy_version 988916 (0.0005) [2023-12-26 22:34:17,745][105692] Updated weights for policy 0, policy_version 988607 (0.0011) [2023-12-26 22:34:17,757][105620] Updated weights for policy 1, policy_version 988926 (0.0005) [2023-12-26 22:34:17,815][105620] Updated weights for policy 1, policy_version 988936 (0.0005) [2023-12-26 22:34:18,411][105620] Updated weights for policy 1, policy_version 988946 (0.0010) [2023-12-26 22:34:18,418][105692] Updated weights for policy 0, policy_version 988617 (0.0010) [2023-12-26 22:34:18,464][105620] Updated weights for policy 1, policy_version 988956 (0.0010) [2023-12-26 22:34:18,475][105692] Updated weights for policy 0, policy_version 988627 (0.0007) [2023-12-26 22:34:18,516][105620] Updated weights for policy 1, policy_version 988966 (0.0010) [2023-12-26 22:34:18,524][105692] Updated weights for policy 0, policy_version 988637 (0.0005) [2023-12-26 22:34:18,579][105692] Updated weights for policy 0, policy_version 988647 (0.0005) [2023-12-26 22:34:19,170][105692] Updated weights for policy 0, policy_version 988657 (0.0005) [2023-12-26 22:34:19,223][105692] Updated weights for policy 0, policy_version 988667 (0.0006) [2023-12-26 22:34:19,285][105692] Updated weights for policy 0, policy_version 988677 (0.0010) [2023-12-26 22:34:19,294][105620] Updated weights for policy 1, policy_version 988976 (0.0010) [2023-12-26 22:34:19,353][105620] Updated weights for policy 1, policy_version 988986 (0.0011) [2023-12-26 22:34:19,406][105620] Updated weights for policy 1, policy_version 988996 (0.0011) [2023-12-26 22:34:20,018][105692] Updated weights for policy 0, policy_version 988687 (0.0009) [2023-12-26 22:34:20,078][105692] Updated weights for policy 0, policy_version 988697 (0.0010) [2023-12-26 22:34:20,140][105620] Updated weights for policy 1, policy_version 989006 (0.0010) [2023-12-26 22:34:20,140][105692] Updated weights for policy 0, policy_version 988707 (0.0009) [2023-12-26 22:34:20,195][105620] Updated weights for policy 1, policy_version 989016 (0.0006) [2023-12-26 22:34:20,254][105620] Updated weights for policy 1, policy_version 989026 (0.0005) [2023-12-26 22:34:20,907][105620] Updated weights for policy 1, policy_version 989036 (0.0007) [2023-12-26 22:34:20,947][105692] Updated weights for policy 0, policy_version 988717 (0.0010) [2023-12-26 22:34:20,964][105620] Updated weights for policy 1, policy_version 989046 (0.0009) [2023-12-26 22:34:21,005][105692] Updated weights for policy 0, policy_version 988727 (0.0006) [2023-12-26 22:34:21,025][105620] Updated weights for policy 1, policy_version 989056 (0.0008) [2023-12-26 22:34:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 506372096. Throughput: 0: 9634.6, 1: 9882.9. Samples: 506370596. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:34:21,062][104569] Avg episode reward: [(0, '8284.333'), (1, '9172.759')] [2023-12-26 22:34:21,068][105692] Updated weights for policy 0, policy_version 988737 (0.0011) [2023-12-26 22:34:21,763][105620] Updated weights for policy 1, policy_version 989066 (0.0008) [2023-12-26 22:34:21,825][105620] Updated weights for policy 1, policy_version 989076 (0.0009) [2023-12-26 22:34:21,842][105692] Updated weights for policy 0, policy_version 988747 (0.0011) [2023-12-26 22:34:21,881][105620] Updated weights for policy 1, policy_version 989086 (0.0007) [2023-12-26 22:34:21,896][105692] Updated weights for policy 0, policy_version 988757 (0.0007) [2023-12-26 22:34:21,936][105620] Updated weights for policy 1, policy_version 989096 (0.0008) [2023-12-26 22:34:21,948][105692] Updated weights for policy 0, policy_version 988767 (0.0006) [2023-12-26 22:34:22,640][105692] Updated weights for policy 0, policy_version 988777 (0.0006) [2023-12-26 22:34:22,695][105692] Updated weights for policy 0, policy_version 988787 (0.0006) [2023-12-26 22:34:22,725][105620] Updated weights for policy 1, policy_version 989106 (0.0008) [2023-12-26 22:34:22,742][105692] Updated weights for policy 0, policy_version 988797 (0.0005) [2023-12-26 22:34:22,791][105620] Updated weights for policy 1, policy_version 989116 (0.0008) [2023-12-26 22:34:22,803][105692] Updated weights for policy 0, policy_version 988807 (0.0006) [2023-12-26 22:34:22,854][105620] Updated weights for policy 1, policy_version 989126 (0.0006) [2023-12-26 22:34:23,469][105692] Updated weights for policy 0, policy_version 988817 (0.0009) [2023-12-26 22:34:23,521][105692] Updated weights for policy 0, policy_version 988827 (0.0009) [2023-12-26 22:34:23,540][105620] Updated weights for policy 1, policy_version 989136 (0.0005) [2023-12-26 22:34:23,573][105692] Updated weights for policy 0, policy_version 988837 (0.0008) [2023-12-26 22:34:23,601][105620] Updated weights for policy 1, policy_version 989146 (0.0005) [2023-12-26 22:34:23,671][105620] Updated weights for policy 1, policy_version 989156 (0.0005) [2023-12-26 22:34:24,165][105620] Updated weights for policy 1, policy_version 989166 (0.0007) [2023-12-26 22:34:24,212][105620] Updated weights for policy 1, policy_version 989176 (0.0009) [2023-12-26 22:34:24,270][105620] Updated weights for policy 1, policy_version 989186 (0.0010) [2023-12-26 22:34:24,390][105692] Updated weights for policy 0, policy_version 988847 (0.0008) [2023-12-26 22:34:24,437][105692] Updated weights for policy 0, policy_version 988857 (0.0008) [2023-12-26 22:34:24,489][105692] Updated weights for policy 0, policy_version 988867 (0.0008) [2023-12-26 22:34:25,030][105620] Updated weights for policy 1, policy_version 989196 (0.0010) [2023-12-26 22:34:25,079][105620] Updated weights for policy 1, policy_version 989206 (0.0010) [2023-12-26 22:34:25,126][105620] Updated weights for policy 1, policy_version 989216 (0.0010) [2023-12-26 22:34:25,182][105692] Updated weights for policy 0, policy_version 988877 (0.0007) [2023-12-26 22:34:25,239][105692] Updated weights for policy 0, policy_version 988887 (0.0005) [2023-12-26 22:34:25,301][105692] Updated weights for policy 0, policy_version 988897 (0.0007) [2023-12-26 22:34:25,883][105620] Updated weights for policy 1, policy_version 989226 (0.0010) [2023-12-26 22:34:25,931][105620] Updated weights for policy 1, policy_version 989236 (0.0010) [2023-12-26 22:34:25,945][105692] Updated weights for policy 0, policy_version 988907 (0.0009) [2023-12-26 22:34:25,985][105620] Updated weights for policy 1, policy_version 989246 (0.0010) [2023-12-26 22:34:26,000][105692] Updated weights for policy 0, policy_version 988917 (0.0005) [2023-12-26 22:34:26,040][105620] Updated weights for policy 1, policy_version 989256 (0.0010) [2023-12-26 22:34:26,052][105692] Updated weights for policy 0, policy_version 988927 (0.0006) [2023-12-26 22:34:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 506478592. Throughput: 0: 9603.4, 1: 9982.5. Samples: 506487696. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:34:26,062][104569] Avg episode reward: [(0, '8281.984'), (1, '8988.619')] [2023-12-26 22:34:26,783][105620] Updated weights for policy 1, policy_version 989266 (0.0011) [2023-12-26 22:34:26,812][105692] Updated weights for policy 0, policy_version 988937 (0.0008) [2023-12-26 22:34:26,836][105620] Updated weights for policy 1, policy_version 989276 (0.0010) [2023-12-26 22:34:26,863][105692] Updated weights for policy 0, policy_version 988947 (0.0006) [2023-12-26 22:34:26,888][105620] Updated weights for policy 1, policy_version 989286 (0.0010) [2023-12-26 22:34:26,911][105692] Updated weights for policy 0, policy_version 988957 (0.0006) [2023-12-26 22:34:26,960][105692] Updated weights for policy 0, policy_version 988967 (0.0008) [2023-12-26 22:34:27,607][105620] Updated weights for policy 1, policy_version 989296 (0.0010) [2023-12-26 22:34:27,661][105620] Updated weights for policy 1, policy_version 989306 (0.0010) [2023-12-26 22:34:27,708][105620] Updated weights for policy 1, policy_version 989316 (0.0010) [2023-12-26 22:34:27,743][105692] Updated weights for policy 0, policy_version 988977 (0.0006) [2023-12-26 22:34:27,790][105692] Updated weights for policy 0, policy_version 988987 (0.0008) [2023-12-26 22:34:27,842][105692] Updated weights for policy 0, policy_version 988997 (0.0008) [2023-12-26 22:34:28,445][105620] Updated weights for policy 1, policy_version 989326 (0.0010) [2023-12-26 22:34:28,506][105620] Updated weights for policy 1, policy_version 989336 (0.0010) [2023-12-26 22:34:28,568][105620] Updated weights for policy 1, policy_version 989346 (0.0010) [2023-12-26 22:34:28,593][105692] Updated weights for policy 0, policy_version 989007 (0.0006) [2023-12-26 22:34:28,654][105692] Updated weights for policy 0, policy_version 989017 (0.0008) [2023-12-26 22:34:28,717][105692] Updated weights for policy 0, policy_version 989027 (0.0008) [2023-12-26 22:34:29,315][105620] Updated weights for policy 1, policy_version 989356 (0.0010) [2023-12-26 22:34:29,382][105620] Updated weights for policy 1, policy_version 989366 (0.0011) [2023-12-26 22:34:29,401][105692] Updated weights for policy 0, policy_version 989037 (0.0007) [2023-12-26 22:34:29,431][105620] Updated weights for policy 1, policy_version 989376 (0.0010) [2023-12-26 22:34:29,449][105692] Updated weights for policy 0, policy_version 989047 (0.0005) [2023-12-26 22:34:29,504][105692] Updated weights for policy 0, policy_version 989057 (0.0006) [2023-12-26 22:34:30,082][105620] Updated weights for policy 1, policy_version 989386 (0.0008) [2023-12-26 22:34:30,145][105620] Updated weights for policy 1, policy_version 989396 (0.0007) [2023-12-26 22:34:30,186][105692] Updated weights for policy 0, policy_version 989067 (0.0007) [2023-12-26 22:34:30,207][105620] Updated weights for policy 1, policy_version 989406 (0.0005) [2023-12-26 22:34:30,238][105692] Updated weights for policy 0, policy_version 989077 (0.0009) [2023-12-26 22:34:30,269][105620] Updated weights for policy 1, policy_version 989416 (0.0005) [2023-12-26 22:34:30,298][105692] Updated weights for policy 0, policy_version 989087 (0.0009) [2023-12-26 22:34:30,913][105620] Updated weights for policy 1, policy_version 989426 (0.0010) [2023-12-26 22:34:30,953][105692] Updated weights for policy 0, policy_version 989097 (0.0008) [2023-12-26 22:34:30,967][105620] Updated weights for policy 1, policy_version 989436 (0.0010) [2023-12-26 22:34:31,001][105692] Updated weights for policy 0, policy_version 989107 (0.0005) [2023-12-26 22:34:31,015][105620] Updated weights for policy 1, policy_version 989446 (0.0010) [2023-12-26 22:34:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 506576896. Throughput: 0: 9617.8, 1: 9973.3. Samples: 506545028. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:34:31,062][104569] Avg episode reward: [(0, '8645.982'), (1, '9079.548')] [2023-12-26 22:34:31,063][105692] Updated weights for policy 0, policy_version 989117 (0.0007) [2023-12-26 22:34:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000989448_253329408.pth... [2023-12-26 22:34:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000988264_253026304.pth [2023-12-26 22:34:31,124][105692] Updated weights for policy 0, policy_version 989127 (0.0008) [2023-12-26 22:34:31,130][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000989128_253255680.pth... [2023-12-26 22:34:31,135][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000987976_252960768.pth [2023-12-26 22:34:31,649][105620] Updated weights for policy 1, policy_version 989456 (0.0011) [2023-12-26 22:34:31,710][105620] Updated weights for policy 1, policy_version 989466 (0.0010) [2023-12-26 22:34:31,770][105620] Updated weights for policy 1, policy_version 989476 (0.0010) [2023-12-26 22:34:31,899][105692] Updated weights for policy 0, policy_version 989137 (0.0008) [2023-12-26 22:34:31,946][105692] Updated weights for policy 0, policy_version 989147 (0.0007) [2023-12-26 22:34:31,990][105692] Updated weights for policy 0, policy_version 989157 (0.0007) [2023-12-26 22:34:32,505][105620] Updated weights for policy 1, policy_version 989486 (0.0010) [2023-12-26 22:34:32,561][105620] Updated weights for policy 1, policy_version 989496 (0.0011) [2023-12-26 22:34:32,622][105620] Updated weights for policy 1, policy_version 989506 (0.0011) [2023-12-26 22:34:32,787][105692] Updated weights for policy 0, policy_version 989167 (0.0008) [2023-12-26 22:34:32,847][105692] Updated weights for policy 0, policy_version 989177 (0.0008) [2023-12-26 22:34:32,894][105692] Updated weights for policy 0, policy_version 989187 (0.0008) [2023-12-26 22:34:33,364][105620] Updated weights for policy 1, policy_version 989516 (0.0010) [2023-12-26 22:34:33,415][105620] Updated weights for policy 1, policy_version 989526 (0.0010) [2023-12-26 22:34:33,463][105620] Updated weights for policy 1, policy_version 989536 (0.0010) [2023-12-26 22:34:33,651][105692] Updated weights for policy 0, policy_version 989197 (0.0008) [2023-12-26 22:34:33,704][105692] Updated weights for policy 0, policy_version 989207 (0.0010) [2023-12-26 22:34:33,757][105692] Updated weights for policy 0, policy_version 989217 (0.0009) [2023-12-26 22:34:34,064][105620] Updated weights for policy 1, policy_version 989546 (0.0009) [2023-12-26 22:34:34,129][105620] Updated weights for policy 1, policy_version 989556 (0.0007) [2023-12-26 22:34:34,186][105620] Updated weights for policy 1, policy_version 989566 (0.0008) [2023-12-26 22:34:34,239][105620] Updated weights for policy 1, policy_version 989576 (0.0007) [2023-12-26 22:34:34,625][105692] Updated weights for policy 0, policy_version 989227 (0.0009) [2023-12-26 22:34:34,677][105692] Updated weights for policy 0, policy_version 989237 (0.0008) [2023-12-26 22:34:34,733][105692] Updated weights for policy 0, policy_version 989247 (0.0008) [2023-12-26 22:34:34,984][105620] Updated weights for policy 1, policy_version 989586 (0.0010) [2023-12-26 22:34:35,032][105620] Updated weights for policy 1, policy_version 989596 (0.0010) [2023-12-26 22:34:35,083][105620] Updated weights for policy 1, policy_version 989606 (0.0010) [2023-12-26 22:34:35,498][105692] Updated weights for policy 0, policy_version 989257 (0.0008) [2023-12-26 22:34:35,548][105692] Updated weights for policy 0, policy_version 989267 (0.0009) [2023-12-26 22:34:35,599][105692] Updated weights for policy 0, policy_version 989277 (0.0009) [2023-12-26 22:34:35,651][105692] Updated weights for policy 0, policy_version 989287 (0.0009) [2023-12-26 22:34:35,780][105620] Updated weights for policy 1, policy_version 989616 (0.0006) [2023-12-26 22:34:35,838][105620] Updated weights for policy 1, policy_version 989626 (0.0008) [2023-12-26 22:34:35,902][105620] Updated weights for policy 1, policy_version 989636 (0.0010) [2023-12-26 22:34:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 506675200. Throughput: 0: 9528.1, 1: 9923.1. Samples: 506662616. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:34:36,062][104569] Avg episode reward: [(0, '8736.101'), (1, '9172.270')] [2023-12-26 22:34:36,398][105692] Updated weights for policy 0, policy_version 989297 (0.0008) [2023-12-26 22:34:36,446][105692] Updated weights for policy 0, policy_version 989307 (0.0008) [2023-12-26 22:34:36,498][105692] Updated weights for policy 0, policy_version 989317 (0.0008) [2023-12-26 22:34:36,625][105620] Updated weights for policy 1, policy_version 989646 (0.0011) [2023-12-26 22:34:36,680][105620] Updated weights for policy 1, policy_version 989656 (0.0010) [2023-12-26 22:34:36,739][105620] Updated weights for policy 1, policy_version 989666 (0.0010) [2023-12-26 22:34:37,281][105692] Updated weights for policy 0, policy_version 989327 (0.0008) [2023-12-26 22:34:37,340][105692] Updated weights for policy 0, policy_version 989337 (0.0008) [2023-12-26 22:34:37,390][105692] Updated weights for policy 0, policy_version 989347 (0.0008) [2023-12-26 22:34:37,489][105620] Updated weights for policy 1, policy_version 989676 (0.0010) [2023-12-26 22:34:37,554][105620] Updated weights for policy 1, policy_version 989686 (0.0010) [2023-12-26 22:34:37,613][105620] Updated weights for policy 1, policy_version 989696 (0.0010) [2023-12-26 22:34:38,222][105692] Updated weights for policy 0, policy_version 989357 (0.0008) [2023-12-26 22:34:38,274][105692] Updated weights for policy 0, policy_version 989367 (0.0008) [2023-12-26 22:34:38,331][105692] Updated weights for policy 0, policy_version 989377 (0.0007) [2023-12-26 22:34:38,359][105620] Updated weights for policy 1, policy_version 989706 (0.0010) [2023-12-26 22:34:38,422][105620] Updated weights for policy 1, policy_version 989716 (0.0010) [2023-12-26 22:34:38,481][105620] Updated weights for policy 1, policy_version 989726 (0.0010) [2023-12-26 22:34:38,541][105620] Updated weights for policy 1, policy_version 989736 (0.0011) [2023-12-26 22:34:39,096][105692] Updated weights for policy 0, policy_version 989387 (0.0007) [2023-12-26 22:34:39,154][105692] Updated weights for policy 0, policy_version 989397 (0.0009) [2023-12-26 22:34:39,217][105692] Updated weights for policy 0, policy_version 989407 (0.0009) [2023-12-26 22:34:39,289][105620] Updated weights for policy 1, policy_version 989746 (0.0007) [2023-12-26 22:34:39,352][105620] Updated weights for policy 1, policy_version 989756 (0.0006) [2023-12-26 22:34:39,422][105620] Updated weights for policy 1, policy_version 989766 (0.0007) [2023-12-26 22:34:39,925][105692] Updated weights for policy 0, policy_version 989417 (0.0008) [2023-12-26 22:34:39,985][105692] Updated weights for policy 0, policy_version 989427 (0.0009) [2023-12-26 22:34:40,049][105692] Updated weights for policy 0, policy_version 989437 (0.0009) [2023-12-26 22:34:40,111][105692] Updated weights for policy 0, policy_version 989447 (0.0008) [2023-12-26 22:34:40,123][105620] Updated weights for policy 1, policy_version 989776 (0.0006) [2023-12-26 22:34:40,192][105620] Updated weights for policy 1, policy_version 989786 (0.0006) [2023-12-26 22:34:40,261][105620] Updated weights for policy 1, policy_version 989796 (0.0006) [2023-12-26 22:34:40,808][105620] Updated weights for policy 1, policy_version 989806 (0.0008) [2023-12-26 22:34:40,870][105620] Updated weights for policy 1, policy_version 989816 (0.0008) [2023-12-26 22:34:40,930][105620] Updated weights for policy 1, policy_version 989826 (0.0010) [2023-12-26 22:34:40,973][105692] Updated weights for policy 0, policy_version 989457 (0.0008) [2023-12-26 22:34:41,024][105692] Updated weights for policy 0, policy_version 989467 (0.0008) [2023-12-26 22:34:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 506765312. Throughput: 0: 9488.6, 1: 9932.9. Samples: 506775876. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:34:41,062][104569] Avg episode reward: [(0, '8906.122'), (1, '9263.894')] [2023-12-26 22:34:41,089][105692] Updated weights for policy 0, policy_version 989477 (0.0009) [2023-12-26 22:34:41,641][105620] Updated weights for policy 1, policy_version 989836 (0.0011) [2023-12-26 22:34:41,702][105620] Updated weights for policy 1, policy_version 989846 (0.0011) [2023-12-26 22:34:41,769][105620] Updated weights for policy 1, policy_version 989856 (0.0011) [2023-12-26 22:34:41,883][105692] Updated weights for policy 0, policy_version 989487 (0.0009) [2023-12-26 22:34:41,941][105692] Updated weights for policy 0, policy_version 989497 (0.0008) [2023-12-26 22:34:41,998][105692] Updated weights for policy 0, policy_version 989507 (0.0005) [2023-12-26 22:34:42,490][105620] Updated weights for policy 1, policy_version 989866 (0.0010) [2023-12-26 22:34:42,542][105620] Updated weights for policy 1, policy_version 989876 (0.0009) [2023-12-26 22:34:42,595][105620] Updated weights for policy 1, policy_version 989886 (0.0010) [2023-12-26 22:34:42,703][105692] Updated weights for policy 0, policy_version 989517 (0.0008) [2023-12-26 22:34:42,765][105692] Updated weights for policy 0, policy_version 989527 (0.0008) [2023-12-26 22:34:42,826][105692] Updated weights for policy 0, policy_version 989537 (0.0008) [2023-12-26 22:34:43,386][105620] Updated weights for policy 1, policy_version 989897 (0.0009) [2023-12-26 22:34:43,437][105620] Updated weights for policy 1, policy_version 989907 (0.0005) [2023-12-26 22:34:43,503][105620] Updated weights for policy 1, policy_version 989917 (0.0008) [2023-12-26 22:34:43,529][105692] Updated weights for policy 0, policy_version 989547 (0.0007) [2023-12-26 22:34:43,560][105620] Updated weights for policy 1, policy_version 989927 (0.0007) [2023-12-26 22:34:43,584][105692] Updated weights for policy 0, policy_version 989557 (0.0007) [2023-12-26 22:34:43,635][105692] Updated weights for policy 0, policy_version 989567 (0.0009) [2023-12-26 22:34:44,207][105692] Updated weights for policy 0, policy_version 989577 (0.0008) [2023-12-26 22:34:44,265][105692] Updated weights for policy 0, policy_version 989587 (0.0007) [2023-12-26 22:34:44,276][105620] Updated weights for policy 1, policy_version 989937 (0.0007) [2023-12-26 22:34:44,321][105692] Updated weights for policy 0, policy_version 989597 (0.0007) [2023-12-26 22:34:44,328][105620] Updated weights for policy 1, policy_version 989947 (0.0008) [2023-12-26 22:34:44,378][105692] Updated weights for policy 0, policy_version 989607 (0.0006) [2023-12-26 22:34:44,389][105620] Updated weights for policy 1, policy_version 989957 (0.0009) [2023-12-26 22:34:44,994][105692] Updated weights for policy 0, policy_version 989617 (0.0008) [2023-12-26 22:34:45,046][105692] Updated weights for policy 0, policy_version 989627 (0.0009) [2023-12-26 22:34:45,107][105692] Updated weights for policy 0, policy_version 989637 (0.0008) [2023-12-26 22:34:45,229][105620] Updated weights for policy 1, policy_version 989967 (0.0010) [2023-12-26 22:34:45,293][105620] Updated weights for policy 1, policy_version 989977 (0.0008) [2023-12-26 22:34:45,362][105620] Updated weights for policy 1, policy_version 989987 (0.0010) [2023-12-26 22:34:45,794][105692] Updated weights for policy 0, policy_version 989647 (0.0007) [2023-12-26 22:34:45,837][105692] Updated weights for policy 0, policy_version 989657 (0.0005) [2023-12-26 22:34:45,884][105692] Updated weights for policy 0, policy_version 989667 (0.0005) [2023-12-26 22:34:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 506863616. Throughput: 0: 9497.4, 1: 9927.7. Samples: 506832180. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:34:46,062][104569] Avg episode reward: [(0, '8997.125'), (1, '8988.718')] [2023-12-26 22:34:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000989672_253394944.pth... [2023-12-26 22:34:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000988552_253108224.pth [2023-12-26 22:34:46,102][105620] Updated weights for policy 1, policy_version 989997 (0.0007) [2023-12-26 22:34:46,162][105620] Updated weights for policy 1, policy_version 990007 (0.0006) [2023-12-26 22:34:46,219][105620] Updated weights for policy 1, policy_version 990017 (0.0005) [2023-12-26 22:34:46,258][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000990024_253476864.pth... [2023-12-26 22:34:46,263][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000988872_253181952.pth [2023-12-26 22:34:46,654][105692] Updated weights for policy 0, policy_version 989677 (0.0007) [2023-12-26 22:34:46,698][105692] Updated weights for policy 0, policy_version 989687 (0.0007) [2023-12-26 22:34:46,732][105620] Updated weights for policy 1, policy_version 990027 (0.0005) [2023-12-26 22:34:46,743][105692] Updated weights for policy 0, policy_version 989697 (0.0005) [2023-12-26 22:34:46,797][105620] Updated weights for policy 1, policy_version 990037 (0.0005) [2023-12-26 22:34:46,854][105620] Updated weights for policy 1, policy_version 990047 (0.0005) [2023-12-26 22:34:47,388][105692] Updated weights for policy 0, policy_version 989707 (0.0005) [2023-12-26 22:34:47,398][105620] Updated weights for policy 1, policy_version 990057 (0.0005) [2023-12-26 22:34:47,439][105692] Updated weights for policy 0, policy_version 989717 (0.0006) [2023-12-26 22:34:47,454][105620] Updated weights for policy 1, policy_version 990067 (0.0005) [2023-12-26 22:34:47,489][105692] Updated weights for policy 0, policy_version 989727 (0.0010) [2023-12-26 22:34:47,512][105620] Updated weights for policy 1, policy_version 990077 (0.0006) [2023-12-26 22:34:47,567][105620] Updated weights for policy 1, policy_version 990087 (0.0010) [2023-12-26 22:34:48,207][105620] Updated weights for policy 1, policy_version 990097 (0.0009) [2023-12-26 22:34:48,249][105692] Updated weights for policy 0, policy_version 989737 (0.0010) [2023-12-26 22:34:48,255][105620] Updated weights for policy 1, policy_version 990107 (0.0009) [2023-12-26 22:34:48,311][105692] Updated weights for policy 0, policy_version 989747 (0.0008) [2023-12-26 22:34:48,313][105620] Updated weights for policy 1, policy_version 990117 (0.0007) [2023-12-26 22:34:48,378][105692] Updated weights for policy 0, policy_version 989757 (0.0008) [2023-12-26 22:34:48,432][105692] Updated weights for policy 0, policy_version 989767 (0.0007) [2023-12-26 22:34:49,017][105620] Updated weights for policy 1, policy_version 990127 (0.0009) [2023-12-26 22:34:49,074][105620] Updated weights for policy 1, policy_version 990137 (0.0008) [2023-12-26 22:34:49,121][105692] Updated weights for policy 0, policy_version 989777 (0.0010) [2023-12-26 22:34:49,132][105620] Updated weights for policy 1, policy_version 990147 (0.0007) [2023-12-26 22:34:49,180][105692] Updated weights for policy 0, policy_version 989787 (0.0010) [2023-12-26 22:34:49,245][105692] Updated weights for policy 0, policy_version 989797 (0.0008) [2023-12-26 22:34:49,883][105620] Updated weights for policy 1, policy_version 990157 (0.0008) [2023-12-26 22:34:49,945][105620] Updated weights for policy 1, policy_version 990167 (0.0010) [2023-12-26 22:34:49,968][105692] Updated weights for policy 0, policy_version 989807 (0.0007) [2023-12-26 22:34:50,001][105620] Updated weights for policy 1, policy_version 990177 (0.0008) [2023-12-26 22:34:50,020][105692] Updated weights for policy 0, policy_version 989817 (0.0007) [2023-12-26 22:34:50,065][105692] Updated weights for policy 0, policy_version 989827 (0.0007) [2023-12-26 22:34:50,753][105692] Updated weights for policy 0, policy_version 989837 (0.0009) [2023-12-26 22:34:50,766][105620] Updated weights for policy 1, policy_version 990187 (0.0007) [2023-12-26 22:34:50,816][105692] Updated weights for policy 0, policy_version 989847 (0.0005) [2023-12-26 22:34:50,829][105620] Updated weights for policy 1, policy_version 990197 (0.0011) [2023-12-26 22:34:50,875][105692] Updated weights for policy 0, policy_version 989857 (0.0005) [2023-12-26 22:34:50,892][105620] Updated weights for policy 1, policy_version 990207 (0.0010) [2023-12-26 22:34:51,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 506970112. Throughput: 0: 9641.1, 1: 9909.8. Samples: 506954368. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:34:51,063][104569] Avg episode reward: [(0, '8997.218'), (1, '9080.072')] [2023-12-26 22:34:51,568][105692] Updated weights for policy 0, policy_version 989867 (0.0006) [2023-12-26 22:34:51,627][105692] Updated weights for policy 0, policy_version 989877 (0.0007) [2023-12-26 22:34:51,633][105620] Updated weights for policy 1, policy_version 990217 (0.0010) [2023-12-26 22:34:51,691][105692] Updated weights for policy 0, policy_version 989887 (0.0007) [2023-12-26 22:34:51,697][105620] Updated weights for policy 1, policy_version 990227 (0.0009) [2023-12-26 22:34:51,765][105620] Updated weights for policy 1, policy_version 990237 (0.0009) [2023-12-26 22:34:51,824][105620] Updated weights for policy 1, policy_version 990247 (0.0008) [2023-12-26 22:34:52,478][105692] Updated weights for policy 0, policy_version 989897 (0.0007) [2023-12-26 22:34:52,525][105692] Updated weights for policy 0, policy_version 989907 (0.0008) [2023-12-26 22:34:52,539][105620] Updated weights for policy 1, policy_version 990257 (0.0008) [2023-12-26 22:34:52,568][105692] Updated weights for policy 0, policy_version 989917 (0.0007) [2023-12-26 22:34:52,600][105620] Updated weights for policy 1, policy_version 990267 (0.0007) [2023-12-26 22:34:52,619][105692] Updated weights for policy 0, policy_version 989927 (0.0008) [2023-12-26 22:34:52,657][105620] Updated weights for policy 1, policy_version 990277 (0.0007) [2023-12-26 22:34:53,290][105620] Updated weights for policy 1, policy_version 990287 (0.0009) [2023-12-26 22:34:53,341][105620] Updated weights for policy 1, policy_version 990297 (0.0008) [2023-12-26 22:34:53,398][105620] Updated weights for policy 1, policy_version 990307 (0.0008) [2023-12-26 22:34:53,462][105692] Updated weights for policy 0, policy_version 989937 (0.0009) [2023-12-26 22:34:53,523][105692] Updated weights for policy 0, policy_version 989947 (0.0010) [2023-12-26 22:34:53,586][105692] Updated weights for policy 0, policy_version 989957 (0.0010) [2023-12-26 22:34:54,074][105620] Updated weights for policy 1, policy_version 990317 (0.0009) [2023-12-26 22:34:54,131][105620] Updated weights for policy 1, policy_version 990327 (0.0008) [2023-12-26 22:34:54,186][105620] Updated weights for policy 1, policy_version 990337 (0.0009) [2023-12-26 22:34:54,389][105692] Updated weights for policy 0, policy_version 989967 (0.0009) [2023-12-26 22:34:54,452][105692] Updated weights for policy 0, policy_version 989977 (0.0008) [2023-12-26 22:34:54,515][105692] Updated weights for policy 0, policy_version 989987 (0.0009) [2023-12-26 22:34:54,845][105620] Updated weights for policy 1, policy_version 990347 (0.0010) [2023-12-26 22:34:54,893][105620] Updated weights for policy 1, policy_version 990357 (0.0010) [2023-12-26 22:34:54,941][105620] Updated weights for policy 1, policy_version 990367 (0.0010) [2023-12-26 22:34:55,279][105692] Updated weights for policy 0, policy_version 989997 (0.0009) [2023-12-26 22:34:55,348][105692] Updated weights for policy 0, policy_version 990007 (0.0010) [2023-12-26 22:34:55,417][105692] Updated weights for policy 0, policy_version 990017 (0.0010) [2023-12-26 22:34:55,617][105620] Updated weights for policy 1, policy_version 990377 (0.0007) [2023-12-26 22:34:55,692][105620] Updated weights for policy 1, policy_version 990387 (0.0008) [2023-12-26 22:34:55,757][105620] Updated weights for policy 1, policy_version 990397 (0.0006) [2023-12-26 22:34:55,824][105620] Updated weights for policy 1, policy_version 990407 (0.0008) [2023-12-26 22:34:55,957][105692] Updated weights for policy 0, policy_version 990027 (0.0009) [2023-12-26 22:34:56,023][105692] Updated weights for policy 0, policy_version 990037 (0.0011) [2023-12-26 22:34:56,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 507060224. Throughput: 0: 9597.1, 1: 9952.1. Samples: 507070576. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:34:56,063][104569] Avg episode reward: [(0, '9095.254'), (1, '9355.422')] [2023-12-26 22:34:56,089][105692] Updated weights for policy 0, policy_version 990047 (0.0011) [2023-12-26 22:34:56,440][105620] Updated weights for policy 1, policy_version 990417 (0.0009) [2023-12-26 22:34:56,496][105620] Updated weights for policy 1, policy_version 990428 (0.0009) [2023-12-26 22:34:56,552][105620] Updated weights for policy 1, policy_version 990438 (0.0007) [2023-12-26 22:34:56,723][105692] Updated weights for policy 0, policy_version 990057 (0.0006) [2023-12-26 22:34:56,781][105692] Updated weights for policy 0, policy_version 990068 (0.0010) [2023-12-26 22:34:56,829][105692] Updated weights for policy 0, policy_version 990078 (0.0007) [2023-12-26 22:34:56,890][105692] Updated weights for policy 0, policy_version 990088 (0.0005) [2023-12-26 22:34:57,187][105620] Updated weights for policy 1, policy_version 990448 (0.0005) [2023-12-26 22:34:57,236][105620] Updated weights for policy 1, policy_version 990458 (0.0006) [2023-12-26 22:34:57,284][105620] Updated weights for policy 1, policy_version 990468 (0.0008) [2023-12-26 22:34:57,485][105692] Updated weights for policy 0, policy_version 990098 (0.0008) [2023-12-26 22:34:57,539][105692] Updated weights for policy 0, policy_version 990108 (0.0009) [2023-12-26 22:34:57,591][105692] Updated weights for policy 0, policy_version 990118 (0.0008) [2023-12-26 22:34:57,889][105620] Updated weights for policy 1, policy_version 990478 (0.0008) [2023-12-26 22:34:57,934][105620] Updated weights for policy 1, policy_version 990488 (0.0006) [2023-12-26 22:34:57,979][105620] Updated weights for policy 1, policy_version 990498 (0.0008) [2023-12-26 22:34:58,396][105692] Updated weights for policy 0, policy_version 990128 (0.0009) [2023-12-26 22:34:58,456][105692] Updated weights for policy 0, policy_version 990138 (0.0011) [2023-12-26 22:34:58,515][105692] Updated weights for policy 0, policy_version 990148 (0.0011) [2023-12-26 22:34:58,803][105620] Updated weights for policy 1, policy_version 990508 (0.0008) [2023-12-26 22:34:58,868][105620] Updated weights for policy 1, policy_version 990518 (0.0007) [2023-12-26 22:34:58,937][105620] Updated weights for policy 1, policy_version 990528 (0.0009) [2023-12-26 22:34:59,345][105692] Updated weights for policy 0, policy_version 990158 (0.0010) [2023-12-26 22:34:59,410][105692] Updated weights for policy 0, policy_version 990168 (0.0009) [2023-12-26 22:34:59,469][105692] Updated weights for policy 0, policy_version 990178 (0.0006) [2023-12-26 22:34:59,756][105620] Updated weights for policy 1, policy_version 990538 (0.0009) [2023-12-26 22:34:59,810][105620] Updated weights for policy 1, policy_version 990548 (0.0009) [2023-12-26 22:34:59,871][105620] Updated weights for policy 1, policy_version 990558 (0.0006) [2023-12-26 22:34:59,936][105620] Updated weights for policy 1, policy_version 990568 (0.0006) [2023-12-26 22:35:00,196][105692] Updated weights for policy 0, policy_version 990188 (0.0005) [2023-12-26 22:35:00,262][105692] Updated weights for policy 0, policy_version 990198 (0.0005) [2023-12-26 22:35:00,329][105692] Updated weights for policy 0, policy_version 990208 (0.0005) [2023-12-26 22:35:00,716][105620] Updated weights for policy 1, policy_version 990578 (0.0009) [2023-12-26 22:35:00,763][105620] Updated weights for policy 1, policy_version 990588 (0.0009) [2023-12-26 22:35:00,815][105620] Updated weights for policy 1, policy_version 990598 (0.0008) [2023-12-26 22:35:00,920][105692] Updated weights for policy 0, policy_version 990218 (0.0009) [2023-12-26 22:35:00,973][105692] Updated weights for policy 0, policy_version 990228 (0.0008) [2023-12-26 22:35:01,034][105692] Updated weights for policy 0, policy_version 990238 (0.0006) [2023-12-26 22:35:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 507158528. Throughput: 0: 9622.0, 1: 10002.0. Samples: 507132044. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:35:01,063][104569] Avg episode reward: [(0, '8824.133'), (1, '9355.465')] [2023-12-26 22:35:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000990600_253624320.pth... [2023-12-26 22:35:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000989448_253329408.pth [2023-12-26 22:35:01,098][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000990248_253542400.pth... [2023-12-26 22:35:01,099][105692] Updated weights for policy 0, policy_version 990248 (0.0007) [2023-12-26 22:35:01,101][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000989128_253255680.pth [2023-12-26 22:35:01,578][105620] Updated weights for policy 1, policy_version 990608 (0.0009) [2023-12-26 22:35:01,646][105620] Updated weights for policy 1, policy_version 990618 (0.0009) [2023-12-26 22:35:01,700][105620] Updated weights for policy 1, policy_version 990628 (0.0010) [2023-12-26 22:35:01,785][105692] Updated weights for policy 0, policy_version 990258 (0.0008) [2023-12-26 22:35:01,847][105692] Updated weights for policy 0, policy_version 990268 (0.0009) [2023-12-26 22:35:01,909][105692] Updated weights for policy 0, policy_version 990278 (0.0009) [2023-12-26 22:35:02,496][105620] Updated weights for policy 1, policy_version 990638 (0.0009) [2023-12-26 22:35:02,542][105620] Updated weights for policy 1, policy_version 990648 (0.0007) [2023-12-26 22:35:02,565][105692] Updated weights for policy 0, policy_version 990288 (0.0009) [2023-12-26 22:35:02,600][105620] Updated weights for policy 1, policy_version 990658 (0.0006) [2023-12-26 22:35:02,621][105692] Updated weights for policy 0, policy_version 990298 (0.0009) [2023-12-26 22:35:02,675][105692] Updated weights for policy 0, policy_version 990308 (0.0009) [2023-12-26 22:35:03,325][105620] Updated weights for policy 1, policy_version 990668 (0.0008) [2023-12-26 22:35:03,382][105620] Updated weights for policy 1, policy_version 990678 (0.0008) [2023-12-26 22:35:03,384][105692] Updated weights for policy 0, policy_version 990318 (0.0008) [2023-12-26 22:35:03,443][105620] Updated weights for policy 1, policy_version 990688 (0.0008) [2023-12-26 22:35:03,444][105692] Updated weights for policy 0, policy_version 990328 (0.0006) [2023-12-26 22:35:03,497][105692] Updated weights for policy 0, policy_version 990338 (0.0007) [2023-12-26 22:35:04,209][105620] Updated weights for policy 1, policy_version 990698 (0.0008) [2023-12-26 22:35:04,258][105692] Updated weights for policy 0, policy_version 990348 (0.0008) [2023-12-26 22:35:04,274][105620] Updated weights for policy 1, policy_version 990708 (0.0009) [2023-12-26 22:35:04,318][105692] Updated weights for policy 0, policy_version 990358 (0.0008) [2023-12-26 22:35:04,332][105620] Updated weights for policy 1, policy_version 990718 (0.0007) [2023-12-26 22:35:04,372][105692] Updated weights for policy 0, policy_version 990368 (0.0007) [2023-12-26 22:35:04,400][105620] Updated weights for policy 1, policy_version 990728 (0.0007) [2023-12-26 22:35:05,042][105692] Updated weights for policy 0, policy_version 990378 (0.0010) [2023-12-26 22:35:05,097][105692] Updated weights for policy 0, policy_version 990388 (0.0009) [2023-12-26 22:35:05,158][105692] Updated weights for policy 0, policy_version 990398 (0.0007) [2023-12-26 22:35:05,160][105620] Updated weights for policy 1, policy_version 990738 (0.0007) [2023-12-26 22:35:05,215][105620] Updated weights for policy 1, policy_version 990748 (0.0008) [2023-12-26 22:35:05,217][105692] Updated weights for policy 0, policy_version 990408 (0.0006) [2023-12-26 22:35:05,272][105620] Updated weights for policy 1, policy_version 990758 (0.0008) [2023-12-26 22:35:05,905][105692] Updated weights for policy 0, policy_version 990418 (0.0009) [2023-12-26 22:35:05,971][105692] Updated weights for policy 0, policy_version 990428 (0.0009) [2023-12-26 22:35:06,033][105692] Updated weights for policy 0, policy_version 990438 (0.0007) [2023-12-26 22:35:06,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 507256832. Throughput: 0: 9600.8, 1: 9828.4. Samples: 507244912. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:35:06,062][104569] Avg episode reward: [(0, '8905.128'), (1, '9263.053')] [2023-12-26 22:35:06,064][105620] Updated weights for policy 1, policy_version 990768 (0.0009) [2023-12-26 22:35:06,117][105620] Updated weights for policy 1, policy_version 990778 (0.0010) [2023-12-26 22:35:06,175][105620] Updated weights for policy 1, policy_version 990788 (0.0010) [2023-12-26 22:35:06,759][105692] Updated weights for policy 0, policy_version 990448 (0.0009) [2023-12-26 22:35:06,810][105692] Updated weights for policy 0, policy_version 990458 (0.0009) [2023-12-26 22:35:06,861][105692] Updated weights for policy 0, policy_version 990468 (0.0009) [2023-12-26 22:35:06,942][105620] Updated weights for policy 1, policy_version 990798 (0.0009) [2023-12-26 22:35:07,007][105620] Updated weights for policy 1, policy_version 990808 (0.0009) [2023-12-26 22:35:07,066][105620] Updated weights for policy 1, policy_version 990818 (0.0009) [2023-12-26 22:35:07,642][105692] Updated weights for policy 0, policy_version 990478 (0.0009) [2023-12-26 22:35:07,703][105692] Updated weights for policy 0, policy_version 990488 (0.0009) [2023-12-26 22:35:07,751][105692] Updated weights for policy 0, policy_version 990498 (0.0009) [2023-12-26 22:35:07,808][105620] Updated weights for policy 1, policy_version 990828 (0.0009) [2023-12-26 22:35:07,854][105620] Updated weights for policy 1, policy_version 990838 (0.0009) [2023-12-26 22:35:07,910][105620] Updated weights for policy 1, policy_version 990848 (0.0009) [2023-12-26 22:35:08,519][105692] Updated weights for policy 0, policy_version 990508 (0.0009) [2023-12-26 22:35:08,574][105692] Updated weights for policy 0, policy_version 990518 (0.0009) [2023-12-26 22:35:08,623][105692] Updated weights for policy 0, policy_version 990528 (0.0009) [2023-12-26 22:35:08,667][105620] Updated weights for policy 1, policy_version 990858 (0.0007) [2023-12-26 22:35:08,724][105620] Updated weights for policy 1, policy_version 990868 (0.0007) [2023-12-26 22:35:08,778][105620] Updated weights for policy 1, policy_version 990878 (0.0005) [2023-12-26 22:35:08,844][105620] Updated weights for policy 1, policy_version 990888 (0.0006) [2023-12-26 22:35:09,486][105692] Updated weights for policy 0, policy_version 990539 (0.0009) [2023-12-26 22:35:09,488][105620] Updated weights for policy 1, policy_version 990898 (0.0009) [2023-12-26 22:35:09,546][105692] Updated weights for policy 0, policy_version 990549 (0.0010) [2023-12-26 22:35:09,555][105620] Updated weights for policy 1, policy_version 990908 (0.0007) [2023-12-26 22:35:09,610][105692] Updated weights for policy 0, policy_version 990559 (0.0008) [2023-12-26 22:35:09,613][105620] Updated weights for policy 1, policy_version 990918 (0.0006) [2023-12-26 22:35:10,366][105620] Updated weights for policy 1, policy_version 990928 (0.0008) [2023-12-26 22:35:10,376][105692] Updated weights for policy 0, policy_version 990569 (0.0008) [2023-12-26 22:35:10,422][105620] Updated weights for policy 1, policy_version 990938 (0.0009) [2023-12-26 22:35:10,432][105692] Updated weights for policy 0, policy_version 990579 (0.0006) [2023-12-26 22:35:10,488][105620] Updated weights for policy 1, policy_version 990948 (0.0008) [2023-12-26 22:35:10,491][105692] Updated weights for policy 0, policy_version 990589 (0.0008) [2023-12-26 22:35:10,553][105692] Updated weights for policy 0, policy_version 990599 (0.0007) [2023-12-26 22:35:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 507346944. Throughput: 0: 9575.7, 1: 9753.8. Samples: 507357524. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:35:11,063][104569] Avg episode reward: [(0, '9173.854'), (1, '9263.068')] [2023-12-26 22:35:11,159][105692] Updated weights for policy 0, policy_version 990609 (0.0007) [2023-12-26 22:35:11,214][105620] Updated weights for policy 1, policy_version 990958 (0.0010) [2023-12-26 22:35:11,216][105692] Updated weights for policy 0, policy_version 990619 (0.0006) [2023-12-26 22:35:11,276][105692] Updated weights for policy 0, policy_version 990629 (0.0007) [2023-12-26 22:35:11,276][105620] Updated weights for policy 1, policy_version 990968 (0.0009) [2023-12-26 22:35:11,333][105620] Updated weights for policy 1, policy_version 990978 (0.0006) [2023-12-26 22:35:11,973][105692] Updated weights for policy 0, policy_version 990639 (0.0008) [2023-12-26 22:35:12,033][105692] Updated weights for policy 0, policy_version 990649 (0.0009) [2023-12-26 22:35:12,088][105692] Updated weights for policy 0, policy_version 990659 (0.0009) [2023-12-26 22:35:12,102][105620] Updated weights for policy 1, policy_version 990988 (0.0009) [2023-12-26 22:35:12,163][105620] Updated weights for policy 1, policy_version 990998 (0.0008) [2023-12-26 22:35:12,221][105620] Updated weights for policy 1, policy_version 991008 (0.0008) [2023-12-26 22:35:12,868][105692] Updated weights for policy 0, policy_version 990669 (0.0008) [2023-12-26 22:35:12,926][105692] Updated weights for policy 0, policy_version 990679 (0.0006) [2023-12-26 22:35:12,942][105620] Updated weights for policy 1, policy_version 991018 (0.0010) [2023-12-26 22:35:12,987][105692] Updated weights for policy 0, policy_version 990689 (0.0006) [2023-12-26 22:35:13,001][105620] Updated weights for policy 1, policy_version 991028 (0.0008) [2023-12-26 22:35:13,054][105620] Updated weights for policy 1, policy_version 991038 (0.0005) [2023-12-26 22:35:13,118][105620] Updated weights for policy 1, policy_version 991048 (0.0006) [2023-12-26 22:35:13,619][105692] Updated weights for policy 0, policy_version 990699 (0.0011) [2023-12-26 22:35:13,670][105692] Updated weights for policy 0, policy_version 990709 (0.0010) [2023-12-26 22:35:13,715][105692] Updated weights for policy 0, policy_version 990719 (0.0010) [2023-12-26 22:35:13,810][105620] Updated weights for policy 1, policy_version 991058 (0.0008) [2023-12-26 22:35:13,858][105620] Updated weights for policy 1, policy_version 991068 (0.0007) [2023-12-26 22:35:13,909][105620] Updated weights for policy 1, policy_version 991078 (0.0008) [2023-12-26 22:35:14,494][105692] Updated weights for policy 0, policy_version 990729 (0.0011) [2023-12-26 22:35:14,556][105692] Updated weights for policy 0, policy_version 990739 (0.0011) [2023-12-26 22:35:14,571][105620] Updated weights for policy 1, policy_version 991088 (0.0006) [2023-12-26 22:35:14,615][105692] Updated weights for policy 0, policy_version 990749 (0.0010) [2023-12-26 22:35:14,618][105620] Updated weights for policy 1, policy_version 991098 (0.0006) [2023-12-26 22:35:14,662][105620] Updated weights for policy 1, policy_version 991108 (0.0009) [2023-12-26 22:35:14,676][105692] Updated weights for policy 0, policy_version 990759 (0.0010) [2023-12-26 22:35:15,427][105692] Updated weights for policy 0, policy_version 990769 (0.0008) [2023-12-26 22:35:15,456][105620] Updated weights for policy 1, policy_version 991118 (0.0008) [2023-12-26 22:35:15,483][105692] Updated weights for policy 0, policy_version 990779 (0.0006) [2023-12-26 22:35:15,510][105620] Updated weights for policy 1, policy_version 991128 (0.0008) [2023-12-26 22:35:15,541][105692] Updated weights for policy 0, policy_version 990789 (0.0006) [2023-12-26 22:35:15,575][105620] Updated weights for policy 1, policy_version 991138 (0.0009) [2023-12-26 22:35:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 507445248. Throughput: 0: 9622.1, 1: 9739.8. Samples: 507416316. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:35:16,063][104569] Avg episode reward: [(0, '9086.324'), (1, '9173.744')] [2023-12-26 22:35:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000991144_253763584.pth... [2023-12-26 22:35:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000990792_253681664.pth... [2023-12-26 22:35:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000990024_253476864.pth [2023-12-26 22:35:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000989672_253394944.pth [2023-12-26 22:35:16,308][105692] Updated weights for policy 0, policy_version 990799 (0.0008) [2023-12-26 22:35:16,322][105620] Updated weights for policy 1, policy_version 991148 (0.0009) [2023-12-26 22:35:16,360][105692] Updated weights for policy 0, policy_version 990809 (0.0007) [2023-12-26 22:35:16,383][105620] Updated weights for policy 1, policy_version 991158 (0.0006) [2023-12-26 22:35:16,412][105692] Updated weights for policy 0, policy_version 990819 (0.0007) [2023-12-26 22:35:16,444][105620] Updated weights for policy 1, policy_version 991168 (0.0007) [2023-12-26 22:35:17,146][105692] Updated weights for policy 0, policy_version 990829 (0.0006) [2023-12-26 22:35:17,199][105620] Updated weights for policy 1, policy_version 991178 (0.0009) [2023-12-26 22:35:17,211][105692] Updated weights for policy 0, policy_version 990839 (0.0005) [2023-12-26 22:35:17,263][105620] Updated weights for policy 1, policy_version 991188 (0.0008) [2023-12-26 22:35:17,281][105692] Updated weights for policy 0, policy_version 990849 (0.0005) [2023-12-26 22:35:17,315][105620] Updated weights for policy 1, policy_version 991198 (0.0008) [2023-12-26 22:35:17,365][105620] Updated weights for policy 1, policy_version 991208 (0.0008) [2023-12-26 22:35:17,863][105692] Updated weights for policy 0, policy_version 990859 (0.0006) [2023-12-26 22:35:17,914][105692] Updated weights for policy 0, policy_version 990869 (0.0005) [2023-12-26 22:35:17,966][105692] Updated weights for policy 0, policy_version 990879 (0.0005) [2023-12-26 22:35:18,221][105620] Updated weights for policy 1, policy_version 991219 (0.0010) [2023-12-26 22:35:18,272][105620] Updated weights for policy 1, policy_version 991229 (0.0009) [2023-12-26 22:35:18,330][105620] Updated weights for policy 1, policy_version 991239 (0.0010) [2023-12-26 22:35:18,504][105692] Updated weights for policy 0, policy_version 990889 (0.0006) [2023-12-26 22:35:18,558][105692] Updated weights for policy 0, policy_version 990899 (0.0007) [2023-12-26 22:35:18,577][105585] KL-divergence is very high: 131.8776 [2023-12-26 22:35:18,606][105692] Updated weights for policy 0, policy_version 990909 (0.0006) [2023-12-26 22:35:18,615][105585] KL-divergence is very high: 256.6713 [2023-12-26 22:35:18,636][105585] KL-divergence is very high: 120.8333 [2023-12-26 22:35:18,649][105585] KL-divergence is very high: 175.5938 [2023-12-26 22:35:18,665][105692] Updated weights for policy 0, policy_version 990919 (0.0005) [2023-12-26 22:35:18,667][105585] KL-divergence is very high: 289.5809 [2023-12-26 22:35:19,224][105620] Updated weights for policy 1, policy_version 991249 (0.0008) [2023-12-26 22:35:19,290][105620] Updated weights for policy 1, policy_version 991259 (0.0008) [2023-12-26 22:35:19,344][105692] Updated weights for policy 0, policy_version 990929 (0.0010) [2023-12-26 22:35:19,352][105620] Updated weights for policy 1, policy_version 991269 (0.0007) [2023-12-26 22:35:19,411][105692] Updated weights for policy 0, policy_version 990939 (0.0011) [2023-12-26 22:35:19,459][105692] Updated weights for policy 0, policy_version 990949 (0.0006) [2023-12-26 22:35:20,111][105620] Updated weights for policy 1, policy_version 991279 (0.0007) [2023-12-26 22:35:20,166][105620] Updated weights for policy 1, policy_version 991289 (0.0009) [2023-12-26 22:35:20,226][105620] Updated weights for policy 1, policy_version 991299 (0.0009) [2023-12-26 22:35:20,245][105692] Updated weights for policy 0, policy_version 990959 (0.0008) [2023-12-26 22:35:20,300][105692] Updated weights for policy 0, policy_version 990969 (0.0009) [2023-12-26 22:35:20,353][105692] Updated weights for policy 0, policy_version 990979 (0.0009) [2023-12-26 22:35:20,956][105620] Updated weights for policy 1, policy_version 991309 (0.0009) [2023-12-26 22:35:20,971][105692] Updated weights for policy 0, policy_version 990989 (0.0006) [2023-12-26 22:35:21,007][105620] Updated weights for policy 1, policy_version 991319 (0.0007) [2023-12-26 22:35:21,026][105692] Updated weights for policy 0, policy_version 990999 (0.0008) [2023-12-26 22:35:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 507535360. Throughput: 0: 9696.1, 1: 9610.0. Samples: 507531392. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:35:21,063][104569] Avg episode reward: [(0, '8814.776'), (1, '9081.771')] [2023-12-26 22:35:21,084][105692] Updated weights for policy 0, policy_version 991009 (0.0009) [2023-12-26 22:35:21,087][105620] Updated weights for policy 1, policy_version 991329 (0.0011) [2023-12-26 22:35:21,836][105620] Updated weights for policy 1, policy_version 991339 (0.0011) [2023-12-26 22:35:21,893][105620] Updated weights for policy 1, policy_version 991349 (0.0011) [2023-12-26 22:35:21,908][105692] Updated weights for policy 0, policy_version 991019 (0.0008) [2023-12-26 22:35:21,949][105620] Updated weights for policy 1, policy_version 991359 (0.0011) [2023-12-26 22:35:21,971][105692] Updated weights for policy 0, policy_version 991029 (0.0009) [2023-12-26 22:35:22,025][105692] Updated weights for policy 0, policy_version 991039 (0.0007) [2023-12-26 22:35:22,718][105620] Updated weights for policy 1, policy_version 991369 (0.0011) [2023-12-26 22:35:22,739][105692] Updated weights for policy 0, policy_version 991049 (0.0008) [2023-12-26 22:35:22,774][105620] Updated weights for policy 1, policy_version 991379 (0.0011) [2023-12-26 22:35:22,795][105692] Updated weights for policy 0, policy_version 991059 (0.0010) [2023-12-26 22:35:22,835][105620] Updated weights for policy 1, policy_version 991389 (0.0011) [2023-12-26 22:35:22,850][105692] Updated weights for policy 0, policy_version 991069 (0.0006) [2023-12-26 22:35:22,899][105620] Updated weights for policy 1, policy_version 991399 (0.0011) [2023-12-26 22:35:22,914][105692] Updated weights for policy 0, policy_version 991079 (0.0007) [2023-12-26 22:35:23,601][105692] Updated weights for policy 0, policy_version 991089 (0.0009) [2023-12-26 22:35:23,662][105692] Updated weights for policy 0, policy_version 991099 (0.0006) [2023-12-26 22:35:23,681][105620] Updated weights for policy 1, policy_version 991409 (0.0011) [2023-12-26 22:35:23,721][105692] Updated weights for policy 0, policy_version 991109 (0.0006) [2023-12-26 22:35:23,737][105620] Updated weights for policy 1, policy_version 991419 (0.0011) [2023-12-26 22:35:23,799][105620] Updated weights for policy 1, policy_version 991429 (0.0008) [2023-12-26 22:35:24,392][105620] Updated weights for policy 1, policy_version 991439 (0.0005) [2023-12-26 22:35:24,420][105692] Updated weights for policy 0, policy_version 991119 (0.0007) [2023-12-26 22:35:24,453][105620] Updated weights for policy 1, policy_version 991449 (0.0009) [2023-12-26 22:35:24,478][105692] Updated weights for policy 0, policy_version 991129 (0.0008) [2023-12-26 22:35:24,519][105620] Updated weights for policy 1, policy_version 991459 (0.0010) [2023-12-26 22:35:24,538][105692] Updated weights for policy 0, policy_version 991139 (0.0006) [2023-12-26 22:35:25,125][105692] Updated weights for policy 0, policy_version 991149 (0.0009) [2023-12-26 22:35:25,179][105692] Updated weights for policy 0, policy_version 991159 (0.0009) [2023-12-26 22:35:25,226][105692] Updated weights for policy 0, policy_version 991169 (0.0009) [2023-12-26 22:35:25,263][105620] Updated weights for policy 1, policy_version 991469 (0.0008) [2023-12-26 22:35:25,319][105620] Updated weights for policy 1, policy_version 991479 (0.0010) [2023-12-26 22:35:25,367][105620] Updated weights for policy 1, policy_version 991489 (0.0007) [2023-12-26 22:35:25,901][105692] Updated weights for policy 0, policy_version 991179 (0.0007) [2023-12-26 22:35:25,957][105692] Updated weights for policy 0, policy_version 991189 (0.0007) [2023-12-26 22:35:26,006][105692] Updated weights for policy 0, policy_version 991199 (0.0008) [2023-12-26 22:35:26,017][105620] Updated weights for policy 1, policy_version 991499 (0.0010) [2023-12-26 22:35:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 507641856. Throughput: 0: 9799.9, 1: 9596.3. Samples: 507648708. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:35:26,063][104569] Avg episode reward: [(0, '8558.242'), (1, '9171.878')] [2023-12-26 22:35:26,074][105620] Updated weights for policy 1, policy_version 991509 (0.0006) [2023-12-26 22:35:26,135][105620] Updated weights for policy 1, policy_version 991519 (0.0005) [2023-12-26 22:35:26,644][105620] Updated weights for policy 1, policy_version 991529 (0.0005) [2023-12-26 22:35:26,702][105620] Updated weights for policy 1, policy_version 991539 (0.0005) [2023-12-26 22:35:26,723][105692] Updated weights for policy 0, policy_version 991209 (0.0008) [2023-12-26 22:35:26,765][105620] Updated weights for policy 1, policy_version 991549 (0.0005) [2023-12-26 22:35:26,782][105692] Updated weights for policy 0, policy_version 991219 (0.0009) [2023-12-26 22:35:26,816][105620] Updated weights for policy 1, policy_version 991559 (0.0010) [2023-12-26 22:35:26,834][105692] Updated weights for policy 0, policy_version 991229 (0.0006) [2023-12-26 22:35:26,885][105692] Updated weights for policy 0, policy_version 991239 (0.0007) [2023-12-26 22:35:27,449][105620] Updated weights for policy 1, policy_version 991569 (0.0010) [2023-12-26 22:35:27,516][105620] Updated weights for policy 1, policy_version 991579 (0.0010) [2023-12-26 22:35:27,564][105620] Updated weights for policy 1, policy_version 991589 (0.0010) [2023-12-26 22:35:27,673][105692] Updated weights for policy 0, policy_version 991249 (0.0009) [2023-12-26 22:35:27,735][105692] Updated weights for policy 0, policy_version 991259 (0.0009) [2023-12-26 22:35:27,796][105692] Updated weights for policy 0, policy_version 991269 (0.0009) [2023-12-26 22:35:28,278][105620] Updated weights for policy 1, policy_version 991599 (0.0007) [2023-12-26 22:35:28,349][105620] Updated weights for policy 1, policy_version 991609 (0.0007) [2023-12-26 22:35:28,418][105620] Updated weights for policy 1, policy_version 991619 (0.0009) [2023-12-26 22:35:28,547][105692] Updated weights for policy 0, policy_version 991279 (0.0009) [2023-12-26 22:35:28,615][105692] Updated weights for policy 0, policy_version 991289 (0.0010) [2023-12-26 22:35:28,676][105692] Updated weights for policy 0, policy_version 991299 (0.0008) [2023-12-26 22:35:29,080][105620] Updated weights for policy 1, policy_version 991629 (0.0009) [2023-12-26 22:35:29,133][105620] Updated weights for policy 1, policy_version 991639 (0.0009) [2023-12-26 22:35:29,182][105620] Updated weights for policy 1, policy_version 991649 (0.0008) [2023-12-26 22:35:29,409][105692] Updated weights for policy 0, policy_version 991309 (0.0007) [2023-12-26 22:35:29,480][105692] Updated weights for policy 0, policy_version 991319 (0.0008) [2023-12-26 22:35:29,546][105692] Updated weights for policy 0, policy_version 991329 (0.0010) [2023-12-26 22:35:29,793][105620] Updated weights for policy 1, policy_version 991659 (0.0006) [2023-12-26 22:35:29,856][105620] Updated weights for policy 1, policy_version 991669 (0.0011) [2023-12-26 22:35:29,913][105620] Updated weights for policy 1, policy_version 991679 (0.0010) [2023-12-26 22:35:30,318][105692] Updated weights for policy 0, policy_version 991339 (0.0009) [2023-12-26 22:35:30,374][105692] Updated weights for policy 0, policy_version 991349 (0.0005) [2023-12-26 22:35:30,441][105692] Updated weights for policy 0, policy_version 991359 (0.0005) [2023-12-26 22:35:30,643][105620] Updated weights for policy 1, policy_version 991689 (0.0010) [2023-12-26 22:35:30,698][105620] Updated weights for policy 1, policy_version 991699 (0.0010) [2023-12-26 22:35:30,763][105620] Updated weights for policy 1, policy_version 991709 (0.0010) [2023-12-26 22:35:30,818][105620] Updated weights for policy 1, policy_version 991719 (0.0010) [2023-12-26 22:35:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19605.2). Total num frames: 507740160. Throughput: 0: 9821.9, 1: 9684.6. Samples: 507709976. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:35:31,062][104569] Avg episode reward: [(0, '8472.074'), (1, '9263.597')] [2023-12-26 22:35:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000991720_253911040.pth... [2023-12-26 22:35:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000991368_253829120.pth... [2023-12-26 22:35:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000990600_253624320.pth [2023-12-26 22:35:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000990248_253542400.pth [2023-12-26 22:35:31,136][105692] Updated weights for policy 0, policy_version 991369 (0.0008) [2023-12-26 22:35:31,196][105692] Updated weights for policy 0, policy_version 991379 (0.0008) [2023-12-26 22:35:31,253][105692] Updated weights for policy 0, policy_version 991389 (0.0009) [2023-12-26 22:35:31,308][105692] Updated weights for policy 0, policy_version 991399 (0.0008) [2023-12-26 22:35:31,589][105620] Updated weights for policy 1, policy_version 991729 (0.0010) [2023-12-26 22:35:31,652][105620] Updated weights for policy 1, policy_version 991739 (0.0010) [2023-12-26 22:35:31,710][105620] Updated weights for policy 1, policy_version 991749 (0.0010) [2023-12-26 22:35:32,106][105692] Updated weights for policy 0, policy_version 991409 (0.0008) [2023-12-26 22:35:32,164][105692] Updated weights for policy 0, policy_version 991419 (0.0008) [2023-12-26 22:35:32,223][105692] Updated weights for policy 0, policy_version 991429 (0.0008) [2023-12-26 22:35:32,440][105620] Updated weights for policy 1, policy_version 991759 (0.0010) [2023-12-26 22:35:32,501][105620] Updated weights for policy 1, policy_version 991769 (0.0010) [2023-12-26 22:35:32,552][105620] Updated weights for policy 1, policy_version 991779 (0.0010) [2023-12-26 22:35:33,002][105692] Updated weights for policy 0, policy_version 991439 (0.0008) [2023-12-26 22:35:33,068][105692] Updated weights for policy 0, policy_version 991449 (0.0008) [2023-12-26 22:35:33,119][105692] Updated weights for policy 0, policy_version 991459 (0.0008) [2023-12-26 22:35:33,296][105620] Updated weights for policy 1, policy_version 991789 (0.0010) [2023-12-26 22:35:33,352][105620] Updated weights for policy 1, policy_version 991799 (0.0010) [2023-12-26 22:35:33,408][105620] Updated weights for policy 1, policy_version 991809 (0.0010) [2023-12-26 22:35:33,860][105692] Updated weights for policy 0, policy_version 991469 (0.0008) [2023-12-26 22:35:33,909][105692] Updated weights for policy 0, policy_version 991479 (0.0008) [2023-12-26 22:35:33,963][105692] Updated weights for policy 0, policy_version 991489 (0.0010) [2023-12-26 22:35:34,154][105620] Updated weights for policy 1, policy_version 991819 (0.0009) [2023-12-26 22:35:34,218][105620] Updated weights for policy 1, policy_version 991829 (0.0009) [2023-12-26 22:35:34,280][105620] Updated weights for policy 1, policy_version 991839 (0.0008) [2023-12-26 22:35:34,725][105692] Updated weights for policy 0, policy_version 991499 (0.0010) [2023-12-26 22:35:34,775][105692] Updated weights for policy 0, policy_version 991509 (0.0008) [2023-12-26 22:35:34,842][105692] Updated weights for policy 0, policy_version 991519 (0.0008) [2023-12-26 22:35:34,958][105620] Updated weights for policy 1, policy_version 991849 (0.0007) [2023-12-26 22:35:35,018][105620] Updated weights for policy 1, policy_version 991859 (0.0007) [2023-12-26 22:35:35,080][105620] Updated weights for policy 1, policy_version 991869 (0.0007) [2023-12-26 22:35:35,137][105620] Updated weights for policy 1, policy_version 991879 (0.0005) [2023-12-26 22:35:35,543][105692] Updated weights for policy 0, policy_version 991529 (0.0008) [2023-12-26 22:35:35,606][105692] Updated weights for policy 0, policy_version 991539 (0.0005) [2023-12-26 22:35:35,656][105692] Updated weights for policy 0, policy_version 991549 (0.0007) [2023-12-26 22:35:35,677][105620] Updated weights for policy 1, policy_version 991889 (0.0010) [2023-12-26 22:35:35,707][105692] Updated weights for policy 0, policy_version 991559 (0.0006) [2023-12-26 22:35:35,735][105620] Updated weights for policy 1, policy_version 991899 (0.0010) [2023-12-26 22:35:35,789][105620] Updated weights for policy 1, policy_version 991909 (0.0010) [2023-12-26 22:35:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 507838464. Throughput: 0: 9687.5, 1: 9623.9. Samples: 507823380. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:35:36,063][104569] Avg episode reward: [(0, '8378.832'), (1, '9263.012')] [2023-12-26 22:35:36,405][105692] Updated weights for policy 0, policy_version 991569 (0.0007) [2023-12-26 22:35:36,470][105692] Updated weights for policy 0, policy_version 991579 (0.0008) [2023-12-26 22:35:36,531][105692] Updated weights for policy 0, policy_version 991589 (0.0008) [2023-12-26 22:35:36,557][105620] Updated weights for policy 1, policy_version 991919 (0.0010) [2023-12-26 22:35:36,621][105620] Updated weights for policy 1, policy_version 991929 (0.0011) [2023-12-26 22:35:36,679][105620] Updated weights for policy 1, policy_version 991939 (0.0007) [2023-12-26 22:35:37,303][105692] Updated weights for policy 0, policy_version 991599 (0.0008) [2023-12-26 22:35:37,359][105692] Updated weights for policy 0, policy_version 991609 (0.0010) [2023-12-26 22:35:37,402][105620] Updated weights for policy 1, policy_version 991949 (0.0008) [2023-12-26 22:35:37,418][105692] Updated weights for policy 0, policy_version 991619 (0.0011) [2023-12-26 22:35:37,457][105620] Updated weights for policy 1, policy_version 991959 (0.0010) [2023-12-26 22:35:37,519][105620] Updated weights for policy 1, policy_version 991969 (0.0010) [2023-12-26 22:35:38,130][105692] Updated weights for policy 0, policy_version 991629 (0.0008) [2023-12-26 22:35:38,197][105620] Updated weights for policy 1, policy_version 991979 (0.0010) [2023-12-26 22:35:38,198][105692] Updated weights for policy 0, policy_version 991639 (0.0009) [2023-12-26 22:35:38,257][105620] Updated weights for policy 1, policy_version 991989 (0.0008) [2023-12-26 22:35:38,269][105692] Updated weights for policy 0, policy_version 991649 (0.0011) [2023-12-26 22:35:38,317][105620] Updated weights for policy 1, policy_version 991999 (0.0011) [2023-12-26 22:35:38,970][105692] Updated weights for policy 0, policy_version 991659 (0.0011) [2023-12-26 22:35:39,026][105692] Updated weights for policy 0, policy_version 991669 (0.0011) [2023-12-26 22:35:39,026][105620] Updated weights for policy 1, policy_version 992009 (0.0010) [2023-12-26 22:35:39,078][105692] Updated weights for policy 0, policy_version 991679 (0.0010) [2023-12-26 22:35:39,082][105620] Updated weights for policy 1, policy_version 992019 (0.0008) [2023-12-26 22:35:39,134][105620] Updated weights for policy 1, policy_version 992029 (0.0010) [2023-12-26 22:35:39,200][105620] Updated weights for policy 1, policy_version 992039 (0.0010) [2023-12-26 22:35:39,847][105692] Updated weights for policy 0, policy_version 991689 (0.0010) [2023-12-26 22:35:39,906][105692] Updated weights for policy 0, policy_version 991699 (0.0010) [2023-12-26 22:35:39,972][105692] Updated weights for policy 0, policy_version 991709 (0.0011) [2023-12-26 22:35:39,977][105620] Updated weights for policy 1, policy_version 992049 (0.0010) [2023-12-26 22:35:40,032][105692] Updated weights for policy 0, policy_version 991719 (0.0011) [2023-12-26 22:35:40,037][105620] Updated weights for policy 1, policy_version 992059 (0.0011) [2023-12-26 22:35:40,096][105620] Updated weights for policy 1, policy_version 992069 (0.0010) [2023-12-26 22:35:40,697][105692] Updated weights for policy 0, policy_version 991729 (0.0008) [2023-12-26 22:35:40,764][105692] Updated weights for policy 0, policy_version 991739 (0.0006) [2023-12-26 22:35:40,833][105692] Updated weights for policy 0, policy_version 991749 (0.0008) [2023-12-26 22:35:40,850][105620] Updated weights for policy 1, policy_version 992079 (0.0010) [2023-12-26 22:35:40,916][105620] Updated weights for policy 1, policy_version 992089 (0.0010) [2023-12-26 22:35:40,967][105620] Updated weights for policy 1, policy_version 992099 (0.0010) [2023-12-26 22:35:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 507936768. Throughput: 0: 9731.9, 1: 9584.3. Samples: 507939800. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-26 22:35:41,063][104569] Avg episode reward: [(0, '8375.479'), (1, '9171.363')] [2023-12-26 22:35:41,493][105692] Updated weights for policy 0, policy_version 991759 (0.0008) [2023-12-26 22:35:41,557][105692] Updated weights for policy 0, policy_version 991769 (0.0007) [2023-12-26 22:35:41,627][105692] Updated weights for policy 0, policy_version 991779 (0.0006) [2023-12-26 22:35:41,770][105620] Updated weights for policy 1, policy_version 992109 (0.0010) [2023-12-26 22:35:41,828][105620] Updated weights for policy 1, policy_version 992119 (0.0009) [2023-12-26 22:35:41,888][105620] Updated weights for policy 1, policy_version 992129 (0.0008) [2023-12-26 22:35:42,296][105692] Updated weights for policy 0, policy_version 991789 (0.0009) [2023-12-26 22:35:42,360][105692] Updated weights for policy 0, policy_version 991799 (0.0008) [2023-12-26 22:35:42,427][105692] Updated weights for policy 0, policy_version 991809 (0.0008) [2023-12-26 22:35:42,610][105620] Updated weights for policy 1, policy_version 992139 (0.0009) [2023-12-26 22:35:42,655][105620] Updated weights for policy 1, policy_version 992149 (0.0008) [2023-12-26 22:35:42,700][105620] Updated weights for policy 1, policy_version 992159 (0.0008) [2023-12-26 22:35:43,154][105692] Updated weights for policy 0, policy_version 991819 (0.0007) [2023-12-26 22:35:43,207][105692] Updated weights for policy 0, policy_version 991829 (0.0006) [2023-12-26 22:35:43,262][105692] Updated weights for policy 0, policy_version 991839 (0.0011) [2023-12-26 22:35:43,435][105620] Updated weights for policy 1, policy_version 992169 (0.0007) [2023-12-26 22:35:43,490][105620] Updated weights for policy 1, policy_version 992179 (0.0008) [2023-12-26 22:35:43,534][105620] Updated weights for policy 1, policy_version 992189 (0.0008) [2023-12-26 22:35:43,588][105620] Updated weights for policy 1, policy_version 992199 (0.0008) [2023-12-26 22:35:43,949][105692] Updated weights for policy 0, policy_version 991849 (0.0009) [2023-12-26 22:35:43,996][105692] Updated weights for policy 0, policy_version 991859 (0.0005) [2023-12-26 22:35:44,048][105692] Updated weights for policy 0, policy_version 991869 (0.0007) [2023-12-26 22:35:44,115][105692] Updated weights for policy 0, policy_version 991879 (0.0010) [2023-12-26 22:35:44,382][105620] Updated weights for policy 1, policy_version 992209 (0.0008) [2023-12-26 22:35:44,443][105620] Updated weights for policy 1, policy_version 992219 (0.0008) [2023-12-26 22:35:44,495][105620] Updated weights for policy 1, policy_version 992229 (0.0008) [2023-12-26 22:35:44,837][105692] Updated weights for policy 0, policy_version 991889 (0.0011) [2023-12-26 22:35:44,904][105692] Updated weights for policy 0, policy_version 991899 (0.0009) [2023-12-26 22:35:44,969][105692] Updated weights for policy 0, policy_version 991909 (0.0008) [2023-12-26 22:35:45,298][105620] Updated weights for policy 1, policy_version 992239 (0.0008) [2023-12-26 22:35:45,347][105620] Updated weights for policy 1, policy_version 992249 (0.0008) [2023-12-26 22:35:45,403][105620] Updated weights for policy 1, policy_version 992259 (0.0007) [2023-12-26 22:35:45,670][105692] Updated weights for policy 0, policy_version 991919 (0.0011) [2023-12-26 22:35:45,735][105692] Updated weights for policy 0, policy_version 991929 (0.0009) [2023-12-26 22:35:45,796][105692] Updated weights for policy 0, policy_version 991939 (0.0009) [2023-12-26 22:35:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 508026880. Throughput: 0: 9696.2, 1: 9536.6. Samples: 507997524. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:35:46,063][104569] Avg episode reward: [(0, '8139.689'), (1, '8186.571')] [2023-12-26 22:35:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000991944_253976576.pth... [2023-12-26 22:35:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000992264_254050304.pth... [2023-12-26 22:35:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000991144_253763584.pth [2023-12-26 22:35:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000990792_253681664.pth [2023-12-26 22:35:46,120][105620] Updated weights for policy 1, policy_version 992269 (0.0009) [2023-12-26 22:35:46,177][105620] Updated weights for policy 1, policy_version 992280 (0.0010) [2023-12-26 22:35:46,230][105620] Updated weights for policy 1, policy_version 992290 (0.0009) [2023-12-26 22:35:46,400][105692] Updated weights for policy 0, policy_version 991949 (0.0009) [2023-12-26 22:35:46,447][105692] Updated weights for policy 0, policy_version 991959 (0.0009) [2023-12-26 22:35:46,504][105692] Updated weights for policy 0, policy_version 991969 (0.0009) [2023-12-26 22:35:47,021][105620] Updated weights for policy 1, policy_version 992300 (0.0009) [2023-12-26 22:35:47,070][105620] Updated weights for policy 1, policy_version 992310 (0.0010) [2023-12-26 22:35:47,094][105692] Updated weights for policy 0, policy_version 991979 (0.0008) [2023-12-26 22:35:47,119][105620] Updated weights for policy 1, policy_version 992320 (0.0010) [2023-12-26 22:35:47,141][105692] Updated weights for policy 0, policy_version 991989 (0.0005) [2023-12-26 22:35:47,189][105692] Updated weights for policy 0, policy_version 991999 (0.0005) [2023-12-26 22:35:47,735][105620] Updated weights for policy 1, policy_version 992330 (0.0010) [2023-12-26 22:35:47,757][105692] Updated weights for policy 0, policy_version 992009 (0.0005) [2023-12-26 22:35:47,786][105620] Updated weights for policy 1, policy_version 992340 (0.0010) [2023-12-26 22:35:47,812][105692] Updated weights for policy 0, policy_version 992019 (0.0005) [2023-12-26 22:35:47,839][105620] Updated weights for policy 1, policy_version 992350 (0.0011) [2023-12-26 22:35:47,873][105692] Updated weights for policy 0, policy_version 992029 (0.0006) [2023-12-26 22:35:47,894][105620] Updated weights for policy 1, policy_version 992360 (0.0010) [2023-12-26 22:35:47,929][105692] Updated weights for policy 0, policy_version 992039 (0.0007) [2023-12-26 22:35:48,592][105620] Updated weights for policy 1, policy_version 992370 (0.0011) [2023-12-26 22:35:48,643][105692] Updated weights for policy 0, policy_version 992049 (0.0006) [2023-12-26 22:35:48,648][105620] Updated weights for policy 1, policy_version 992380 (0.0010) [2023-12-26 22:35:48,705][105692] Updated weights for policy 0, policy_version 992059 (0.0006) [2023-12-26 22:35:48,711][105620] Updated weights for policy 1, policy_version 992390 (0.0011) [2023-12-26 22:35:48,766][105692] Updated weights for policy 0, policy_version 992069 (0.0008) [2023-12-26 22:35:49,478][105620] Updated weights for policy 1, policy_version 992400 (0.0008) [2023-12-26 22:35:49,527][105692] Updated weights for policy 0, policy_version 992079 (0.0008) [2023-12-26 22:35:49,536][105620] Updated weights for policy 1, policy_version 992410 (0.0009) [2023-12-26 22:35:49,574][105692] Updated weights for policy 0, policy_version 992089 (0.0008) [2023-12-26 22:35:49,602][105620] Updated weights for policy 1, policy_version 992420 (0.0011) [2023-12-26 22:35:49,616][105692] Updated weights for policy 0, policy_version 992099 (0.0006) [2023-12-26 22:35:50,306][105620] Updated weights for policy 1, policy_version 992430 (0.0009) [2023-12-26 22:35:50,338][105692] Updated weights for policy 0, policy_version 992109 (0.0008) [2023-12-26 22:35:50,378][105620] Updated weights for policy 1, policy_version 992440 (0.0006) [2023-12-26 22:35:50,403][105692] Updated weights for policy 0, policy_version 992119 (0.0007) [2023-12-26 22:35:50,449][105620] Updated weights for policy 1, policy_version 992450 (0.0007) [2023-12-26 22:35:50,465][105692] Updated weights for policy 0, policy_version 992129 (0.0006) [2023-12-26 22:35:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 508125184. Throughput: 0: 9773.3, 1: 9600.4. Samples: 508116732. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:35:51,063][104569] Avg episode reward: [(0, '8319.477'), (1, '7574.172')] [2023-12-26 22:35:51,083][105620] Updated weights for policy 1, policy_version 992460 (0.0008) [2023-12-26 22:35:51,110][105692] Updated weights for policy 0, policy_version 992139 (0.0005) [2023-12-26 22:35:51,150][105620] Updated weights for policy 1, policy_version 992470 (0.0009) [2023-12-26 22:35:51,180][105692] Updated weights for policy 0, policy_version 992149 (0.0006) [2023-12-26 22:35:51,211][105620] Updated weights for policy 1, policy_version 992480 (0.0009) [2023-12-26 22:35:51,243][105692] Updated weights for policy 0, policy_version 992159 (0.0006) [2023-12-26 22:35:51,931][105620] Updated weights for policy 1, policy_version 992490 (0.0007) [2023-12-26 22:35:51,989][105620] Updated weights for policy 1, policy_version 992500 (0.0009) [2023-12-26 22:35:52,001][105692] Updated weights for policy 0, policy_version 992169 (0.0008) [2023-12-26 22:35:52,036][105620] Updated weights for policy 1, policy_version 992510 (0.0007) [2023-12-26 22:35:52,051][105692] Updated weights for policy 0, policy_version 992179 (0.0008) [2023-12-26 22:35:52,090][105620] Updated weights for policy 1, policy_version 992520 (0.0006) [2023-12-26 22:35:52,112][105692] Updated weights for policy 0, policy_version 992189 (0.0008) [2023-12-26 22:35:52,179][105692] Updated weights for policy 0, policy_version 992200 (0.0010) [2023-12-26 22:35:52,790][105620] Updated weights for policy 1, policy_version 992530 (0.0006) [2023-12-26 22:35:52,845][105620] Updated weights for policy 1, policy_version 992540 (0.0007) [2023-12-26 22:35:52,893][105620] Updated weights for policy 1, policy_version 992550 (0.0009) [2023-12-26 22:35:52,999][105692] Updated weights for policy 0, policy_version 992210 (0.0010) [2023-12-26 22:35:53,056][105692] Updated weights for policy 0, policy_version 992221 (0.0009) [2023-12-26 22:35:53,106][105692] Updated weights for policy 0, policy_version 992231 (0.0007) [2023-12-26 22:35:53,555][105620] Updated weights for policy 1, policy_version 992560 (0.0008) [2023-12-26 22:35:53,614][105620] Updated weights for policy 1, policy_version 992570 (0.0009) [2023-12-26 22:35:53,676][105620] Updated weights for policy 1, policy_version 992580 (0.0009) [2023-12-26 22:35:53,869][105692] Updated weights for policy 0, policy_version 992241 (0.0009) [2023-12-26 22:35:53,928][105692] Updated weights for policy 0, policy_version 992251 (0.0009) [2023-12-26 22:35:53,987][105692] Updated weights for policy 0, policy_version 992261 (0.0009) [2023-12-26 22:35:54,445][105620] Updated weights for policy 1, policy_version 992590 (0.0009) [2023-12-26 22:35:54,491][105620] Updated weights for policy 1, policy_version 992600 (0.0008) [2023-12-26 22:35:54,547][105620] Updated weights for policy 1, policy_version 992610 (0.0009) [2023-12-26 22:35:54,712][105692] Updated weights for policy 0, policy_version 992271 (0.0009) [2023-12-26 22:35:54,770][105692] Updated weights for policy 0, policy_version 992281 (0.0009) [2023-12-26 22:35:54,827][105692] Updated weights for policy 0, policy_version 992291 (0.0008) [2023-12-26 22:35:55,243][105620] Updated weights for policy 1, policy_version 992620 (0.0010) [2023-12-26 22:35:55,297][105620] Updated weights for policy 1, policy_version 992630 (0.0009) [2023-12-26 22:35:55,347][105620] Updated weights for policy 1, policy_version 992640 (0.0009) [2023-12-26 22:35:55,636][105692] Updated weights for policy 0, policy_version 992301 (0.0010) [2023-12-26 22:35:55,690][105692] Updated weights for policy 0, policy_version 992312 (0.0011) [2023-12-26 22:35:55,744][105692] Updated weights for policy 0, policy_version 992322 (0.0010) [2023-12-26 22:35:56,018][105620] Updated weights for policy 1, policy_version 992650 (0.0009) [2023-12-26 22:35:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.9, 300 sec: 19605.3). Total num frames: 508223488. Throughput: 0: 9784.1, 1: 9684.7. Samples: 508233616. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:35:56,062][104569] Avg episode reward: [(0, '8995.348'), (1, '8558.890')] [2023-12-26 22:35:56,083][105620] Updated weights for policy 1, policy_version 992660 (0.0009) [2023-12-26 22:35:56,150][105620] Updated weights for policy 1, policy_version 992670 (0.0005) [2023-12-26 22:35:56,201][105620] Updated weights for policy 1, policy_version 992680 (0.0006) [2023-12-26 22:35:56,393][105692] Updated weights for policy 0, policy_version 992333 (0.0007) [2023-12-26 22:35:56,441][105692] Updated weights for policy 0, policy_version 992343 (0.0005) [2023-12-26 22:35:56,488][105692] Updated weights for policy 0, policy_version 992353 (0.0005) [2023-12-26 22:35:56,814][105620] Updated weights for policy 1, policy_version 992690 (0.0010) [2023-12-26 22:35:56,877][105620] Updated weights for policy 1, policy_version 992700 (0.0010) [2023-12-26 22:35:56,928][105620] Updated weights for policy 1, policy_version 992710 (0.0010) [2023-12-26 22:35:57,118][105692] Updated weights for policy 0, policy_version 992363 (0.0005) [2023-12-26 22:35:57,172][105692] Updated weights for policy 0, policy_version 992373 (0.0005) [2023-12-26 22:35:57,228][105692] Updated weights for policy 0, policy_version 992383 (0.0007) [2023-12-26 22:35:57,648][105620] Updated weights for policy 1, policy_version 992720 (0.0007) [2023-12-26 22:35:57,701][105620] Updated weights for policy 1, policy_version 992730 (0.0009) [2023-12-26 22:35:57,759][105620] Updated weights for policy 1, policy_version 992740 (0.0009) [2023-12-26 22:35:57,903][105692] Updated weights for policy 0, policy_version 992393 (0.0006) [2023-12-26 22:35:57,958][105692] Updated weights for policy 0, policy_version 992403 (0.0009) [2023-12-26 22:35:58,014][105692] Updated weights for policy 0, policy_version 992413 (0.0010) [2023-12-26 22:35:58,060][105692] Updated weights for policy 0, policy_version 992423 (0.0009) [2023-12-26 22:35:58,478][105620] Updated weights for policy 1, policy_version 992750 (0.0009) [2023-12-26 22:35:58,541][105620] Updated weights for policy 1, policy_version 992760 (0.0008) [2023-12-26 22:35:58,603][105620] Updated weights for policy 1, policy_version 992770 (0.0008) [2023-12-26 22:35:58,931][105692] Updated weights for policy 0, policy_version 992433 (0.0009) [2023-12-26 22:35:58,995][105692] Updated weights for policy 0, policy_version 992443 (0.0009) [2023-12-26 22:35:59,051][105692] Updated weights for policy 0, policy_version 992453 (0.0008) [2023-12-26 22:35:59,510][105620] Updated weights for policy 1, policy_version 992780 (0.0008) [2023-12-26 22:35:59,561][105620] Updated weights for policy 1, policy_version 992790 (0.0009) [2023-12-26 22:35:59,619][105620] Updated weights for policy 1, policy_version 992800 (0.0008) [2023-12-26 22:35:59,805][105692] Updated weights for policy 0, policy_version 992463 (0.0009) [2023-12-26 22:35:59,871][105692] Updated weights for policy 0, policy_version 992473 (0.0009) [2023-12-26 22:35:59,938][105692] Updated weights for policy 0, policy_version 992483 (0.0009) [2023-12-26 22:36:00,325][105620] Updated weights for policy 1, policy_version 992810 (0.0008) [2023-12-26 22:36:00,383][105620] Updated weights for policy 1, policy_version 992820 (0.0008) [2023-12-26 22:36:00,432][105620] Updated weights for policy 1, policy_version 992830 (0.0008) [2023-12-26 22:36:00,482][105620] Updated weights for policy 1, policy_version 992840 (0.0008) [2023-12-26 22:36:00,696][105692] Updated weights for policy 0, policy_version 992493 (0.0010) [2023-12-26 22:36:00,762][105692] Updated weights for policy 0, policy_version 992503 (0.0010) [2023-12-26 22:36:00,815][105692] Updated weights for policy 0, policy_version 992513 (0.0010) [2023-12-26 22:36:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 508321792. Throughput: 0: 9782.8, 1: 9702.1. Samples: 508293140. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:36:01,063][104569] Avg episode reward: [(0, '8725.831'), (1, '9171.194')] [2023-12-26 22:36:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000992520_254124032.pth... [2023-12-26 22:36:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000991368_253829120.pth [2023-12-26 22:36:01,096][105620] Updated weights for policy 1, policy_version 992850 (0.0007) [2023-12-26 22:36:01,168][105620] Updated weights for policy 1, policy_version 992860 (0.0009) [2023-12-26 22:36:01,223][105620] Updated weights for policy 1, policy_version 992870 (0.0010) [2023-12-26 22:36:01,235][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000992872_254205952.pth... [2023-12-26 22:36:01,240][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000991720_253911040.pth [2023-12-26 22:36:01,553][105692] Updated weights for policy 0, policy_version 992524 (0.0009) [2023-12-26 22:36:01,612][105692] Updated weights for policy 0, policy_version 992534 (0.0008) [2023-12-26 22:36:01,674][105692] Updated weights for policy 0, policy_version 992544 (0.0007) [2023-12-26 22:36:01,972][105620] Updated weights for policy 1, policy_version 992880 (0.0010) [2023-12-26 22:36:02,031][105620] Updated weights for policy 1, policy_version 992890 (0.0010) [2023-12-26 22:36:02,082][105620] Updated weights for policy 1, policy_version 992900 (0.0010) [2023-12-26 22:36:02,322][105692] Updated weights for policy 0, policy_version 992554 (0.0008) [2023-12-26 22:36:02,387][105692] Updated weights for policy 0, policy_version 992564 (0.0007) [2023-12-26 22:36:02,444][105692] Updated weights for policy 0, policy_version 992574 (0.0009) [2023-12-26 22:36:02,500][105692] Updated weights for policy 0, policy_version 992584 (0.0009) [2023-12-26 22:36:02,860][105620] Updated weights for policy 1, policy_version 992910 (0.0009) [2023-12-26 22:36:02,910][105620] Updated weights for policy 1, policy_version 992920 (0.0008) [2023-12-26 22:36:02,960][105620] Updated weights for policy 1, policy_version 992930 (0.0007) [2023-12-26 22:36:03,257][105692] Updated weights for policy 0, policy_version 992594 (0.0008) [2023-12-26 22:36:03,312][105692] Updated weights for policy 0, policy_version 992604 (0.0009) [2023-12-26 22:36:03,363][105692] Updated weights for policy 0, policy_version 992614 (0.0009) [2023-12-26 22:36:03,699][105620] Updated weights for policy 1, policy_version 992940 (0.0007) [2023-12-26 22:36:03,755][105620] Updated weights for policy 1, policy_version 992950 (0.0008) [2023-12-26 22:36:03,806][105620] Updated weights for policy 1, policy_version 992960 (0.0008) [2023-12-26 22:36:04,200][105692] Updated weights for policy 0, policy_version 992624 (0.0010) [2023-12-26 22:36:04,267][105692] Updated weights for policy 0, policy_version 992634 (0.0008) [2023-12-26 22:36:04,336][105692] Updated weights for policy 0, policy_version 992644 (0.0008) [2023-12-26 22:36:04,499][105620] Updated weights for policy 1, policy_version 992970 (0.0007) [2023-12-26 22:36:04,563][105620] Updated weights for policy 1, policy_version 992980 (0.0010) [2023-12-26 22:36:04,627][105620] Updated weights for policy 1, policy_version 992990 (0.0010) [2023-12-26 22:36:04,680][105620] Updated weights for policy 1, policy_version 993000 (0.0010) [2023-12-26 22:36:05,147][105692] Updated weights for policy 0, policy_version 992654 (0.0008) [2023-12-26 22:36:05,209][105692] Updated weights for policy 0, policy_version 992664 (0.0008) [2023-12-26 22:36:05,268][105692] Updated weights for policy 0, policy_version 992674 (0.0007) [2023-12-26 22:36:05,282][105620] Updated weights for policy 1, policy_version 993010 (0.0010) [2023-12-26 22:36:05,326][105620] Updated weights for policy 1, policy_version 993020 (0.0010) [2023-12-26 22:36:05,371][105620] Updated weights for policy 1, policy_version 993030 (0.0010) [2023-12-26 22:36:05,952][105692] Updated weights for policy 0, policy_version 992684 (0.0007) [2023-12-26 22:36:06,014][105692] Updated weights for policy 0, policy_version 992694 (0.0007) [2023-12-26 22:36:06,062][105692] Updated weights for policy 0, policy_version 992704 (0.0008) [2023-12-26 22:36:06,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 508411904. Throughput: 0: 9656.9, 1: 9798.5. Samples: 508406884. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:36:06,062][104569] Avg episode reward: [(0, '8548.658'), (1, '9263.208')] [2023-12-26 22:36:06,152][105620] Updated weights for policy 1, policy_version 993040 (0.0009) [2023-12-26 22:36:06,211][105620] Updated weights for policy 1, policy_version 993050 (0.0010) [2023-12-26 22:36:06,275][105620] Updated weights for policy 1, policy_version 993060 (0.0010) [2023-12-26 22:36:06,893][105692] Updated weights for policy 0, policy_version 992714 (0.0010) [2023-12-26 22:36:06,913][105620] Updated weights for policy 1, policy_version 993070 (0.0009) [2023-12-26 22:36:06,951][105692] Updated weights for policy 0, policy_version 992724 (0.0007) [2023-12-26 22:36:06,972][105620] Updated weights for policy 1, policy_version 993080 (0.0010) [2023-12-26 22:36:07,005][105692] Updated weights for policy 0, policy_version 992734 (0.0007) [2023-12-26 22:36:07,027][105620] Updated weights for policy 1, policy_version 993090 (0.0007) [2023-12-26 22:36:07,050][105692] Updated weights for policy 0, policy_version 992744 (0.0006) [2023-12-26 22:36:07,711][105692] Updated weights for policy 0, policy_version 992754 (0.0006) [2023-12-26 22:36:07,777][105692] Updated weights for policy 0, policy_version 992764 (0.0006) [2023-12-26 22:36:07,777][105620] Updated weights for policy 1, policy_version 993100 (0.0010) [2023-12-26 22:36:07,833][105620] Updated weights for policy 1, policy_version 993110 (0.0010) [2023-12-26 22:36:07,839][105692] Updated weights for policy 0, policy_version 992774 (0.0007) [2023-12-26 22:36:07,888][105620] Updated weights for policy 1, policy_version 993120 (0.0010) [2023-12-26 22:36:08,583][105692] Updated weights for policy 0, policy_version 992784 (0.0008) [2023-12-26 22:36:08,642][105692] Updated weights for policy 0, policy_version 992794 (0.0009) [2023-12-26 22:36:08,649][105620] Updated weights for policy 1, policy_version 993130 (0.0009) [2023-12-26 22:36:08,696][105692] Updated weights for policy 0, policy_version 992804 (0.0006) [2023-12-26 22:36:08,706][105620] Updated weights for policy 1, policy_version 993140 (0.0007) [2023-12-26 22:36:08,755][105620] Updated weights for policy 1, policy_version 993150 (0.0008) [2023-12-26 22:36:08,808][105620] Updated weights for policy 1, policy_version 993160 (0.0009) [2023-12-26 22:36:09,369][105692] Updated weights for policy 0, policy_version 992814 (0.0008) [2023-12-26 22:36:09,430][105692] Updated weights for policy 0, policy_version 992824 (0.0009) [2023-12-26 22:36:09,497][105692] Updated weights for policy 0, policy_version 992834 (0.0009) [2023-12-26 22:36:09,628][105620] Updated weights for policy 1, policy_version 993170 (0.0008) [2023-12-26 22:36:09,702][105620] Updated weights for policy 1, policy_version 993180 (0.0009) [2023-12-26 22:36:09,760][105620] Updated weights for policy 1, policy_version 993190 (0.0008) [2023-12-26 22:36:10,225][105692] Updated weights for policy 0, policy_version 992844 (0.0009) [2023-12-26 22:36:10,288][105692] Updated weights for policy 0, policy_version 992854 (0.0011) [2023-12-26 22:36:10,353][105692] Updated weights for policy 0, policy_version 992864 (0.0011) [2023-12-26 22:36:10,561][105620] Updated weights for policy 1, policy_version 993200 (0.0009) [2023-12-26 22:36:10,613][105620] Updated weights for policy 1, policy_version 993210 (0.0010) [2023-12-26 22:36:10,671][105620] Updated weights for policy 1, policy_version 993220 (0.0010) [2023-12-26 22:36:10,983][105692] Updated weights for policy 0, policy_version 992874 (0.0010) [2023-12-26 22:36:11,041][105692] Updated weights for policy 0, policy_version 992884 (0.0009) [2023-12-26 22:36:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 508510208. Throughput: 0: 9633.9, 1: 9744.8. Samples: 508520748. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:36:11,063][104569] Avg episode reward: [(0, '9086.949'), (1, '9355.363')] [2023-12-26 22:36:11,113][105692] Updated weights for policy 0, policy_version 992894 (0.0007) [2023-12-26 22:36:11,178][105692] Updated weights for policy 0, policy_version 992904 (0.0006) [2023-12-26 22:36:11,483][105620] Updated weights for policy 1, policy_version 993230 (0.0008) [2023-12-26 22:36:11,536][105620] Updated weights for policy 1, policy_version 993240 (0.0008) [2023-12-26 22:36:11,585][105620] Updated weights for policy 1, policy_version 993250 (0.0008) [2023-12-26 22:36:11,918][105692] Updated weights for policy 0, policy_version 992914 (0.0011) [2023-12-26 22:36:11,975][105692] Updated weights for policy 0, policy_version 992924 (0.0011) [2023-12-26 22:36:12,025][105692] Updated weights for policy 0, policy_version 992934 (0.0010) [2023-12-26 22:36:12,411][105620] Updated weights for policy 1, policy_version 993260 (0.0008) [2023-12-26 22:36:12,477][105620] Updated weights for policy 1, policy_version 993270 (0.0008) [2023-12-26 22:36:12,536][105620] Updated weights for policy 1, policy_version 993280 (0.0008) [2023-12-26 22:36:12,830][105692] Updated weights for policy 0, policy_version 992944 (0.0010) [2023-12-26 22:36:12,897][105692] Updated weights for policy 0, policy_version 992954 (0.0009) [2023-12-26 22:36:12,959][105692] Updated weights for policy 0, policy_version 992964 (0.0008) [2023-12-26 22:36:13,351][105620] Updated weights for policy 1, policy_version 993290 (0.0009) [2023-12-26 22:36:13,408][105620] Updated weights for policy 1, policy_version 993300 (0.0009) [2023-12-26 22:36:13,462][105620] Updated weights for policy 1, policy_version 993310 (0.0009) [2023-12-26 22:36:13,508][105620] Updated weights for policy 1, policy_version 993320 (0.0008) [2023-12-26 22:36:13,531][105692] Updated weights for policy 0, policy_version 992974 (0.0007) [2023-12-26 22:36:13,587][105692] Updated weights for policy 0, policy_version 992984 (0.0008) [2023-12-26 22:36:13,641][105692] Updated weights for policy 0, policy_version 992994 (0.0005) [2023-12-26 22:36:14,304][105620] Updated weights for policy 1, policy_version 993330 (0.0008) [2023-12-26 22:36:14,323][105692] Updated weights for policy 0, policy_version 993004 (0.0006) [2023-12-26 22:36:14,357][105620] Updated weights for policy 1, policy_version 993340 (0.0006) [2023-12-26 22:36:14,375][105692] Updated weights for policy 0, policy_version 993014 (0.0010) [2023-12-26 22:36:14,405][105620] Updated weights for policy 1, policy_version 993350 (0.0006) [2023-12-26 22:36:14,436][105692] Updated weights for policy 0, policy_version 993024 (0.0010) [2023-12-26 22:36:15,178][105692] Updated weights for policy 0, policy_version 993034 (0.0011) [2023-12-26 22:36:15,185][105620] Updated weights for policy 1, policy_version 993360 (0.0008) [2023-12-26 22:36:15,230][105692] Updated weights for policy 0, policy_version 993044 (0.0006) [2023-12-26 22:36:15,235][105620] Updated weights for policy 1, policy_version 993370 (0.0009) [2023-12-26 22:36:15,283][105620] Updated weights for policy 1, policy_version 993380 (0.0007) [2023-12-26 22:36:15,285][105692] Updated weights for policy 0, policy_version 993054 (0.0009) [2023-12-26 22:36:15,341][105692] Updated weights for policy 0, policy_version 993064 (0.0010) [2023-12-26 22:36:16,051][105620] Updated weights for policy 1, policy_version 993390 (0.0008) [2023-12-26 22:36:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 508600320. Throughput: 0: 9651.5, 1: 9608.1. Samples: 508576656. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:36:16,062][104569] Avg episode reward: [(0, '8901.537'), (1, '9184.881')] [2023-12-26 22:36:16,083][105692] Updated weights for policy 0, policy_version 993074 (0.0008) [2023-12-26 22:36:16,103][105620] Updated weights for policy 1, policy_version 993400 (0.0008) [2023-12-26 22:36:16,145][105692] Updated weights for policy 0, policy_version 993084 (0.0005) [2023-12-26 22:36:16,155][105620] Updated weights for policy 1, policy_version 993411 (0.0008) [2023-12-26 22:36:16,175][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000993416_254345216.pth... [2023-12-26 22:36:16,179][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000992264_254050304.pth [2023-12-26 22:36:16,203][105692] Updated weights for policy 0, policy_version 993094 (0.0005) [2023-12-26 22:36:16,213][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000993096_254271488.pth... [2023-12-26 22:36:16,217][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000991944_253976576.pth [2023-12-26 22:36:16,772][105620] Updated weights for policy 1, policy_version 993421 (0.0008) [2023-12-26 22:36:16,823][105620] Updated weights for policy 1, policy_version 993431 (0.0009) [2023-12-26 22:36:16,873][105620] Updated weights for policy 1, policy_version 993441 (0.0009) [2023-12-26 22:36:16,962][105692] Updated weights for policy 0, policy_version 993104 (0.0008) [2023-12-26 22:36:17,010][105692] Updated weights for policy 0, policy_version 993114 (0.0007) [2023-12-26 22:36:17,070][105692] Updated weights for policy 0, policy_version 993124 (0.0005) [2023-12-26 22:36:17,680][105620] Updated weights for policy 1, policy_version 993451 (0.0009) [2023-12-26 22:36:17,733][105620] Updated weights for policy 1, policy_version 993461 (0.0007) [2023-12-26 22:36:17,734][105692] Updated weights for policy 0, policy_version 993134 (0.0006) [2023-12-26 22:36:17,784][105692] Updated weights for policy 0, policy_version 993144 (0.0006) [2023-12-26 22:36:17,793][105620] Updated weights for policy 1, policy_version 993471 (0.0008) [2023-12-26 22:36:17,840][105692] Updated weights for policy 0, policy_version 993154 (0.0005) [2023-12-26 22:36:18,560][105620] Updated weights for policy 1, policy_version 993481 (0.0009) [2023-12-26 22:36:18,613][105692] Updated weights for policy 0, policy_version 993164 (0.0008) [2023-12-26 22:36:18,615][105620] Updated weights for policy 1, policy_version 993491 (0.0008) [2023-12-26 22:36:18,663][105692] Updated weights for policy 0, policy_version 993174 (0.0006) [2023-12-26 22:36:18,669][105620] Updated weights for policy 1, policy_version 993501 (0.0007) [2023-12-26 22:36:18,712][105692] Updated weights for policy 0, policy_version 993184 (0.0008) [2023-12-26 22:36:18,724][105620] Updated weights for policy 1, policy_version 993511 (0.0009) [2023-12-26 22:36:19,420][105692] Updated weights for policy 0, policy_version 993194 (0.0009) [2023-12-26 22:36:19,482][105692] Updated weights for policy 0, policy_version 993204 (0.0009) [2023-12-26 22:36:19,537][105620] Updated weights for policy 1, policy_version 993521 (0.0009) [2023-12-26 22:36:19,539][105692] Updated weights for policy 0, policy_version 993214 (0.0006) [2023-12-26 22:36:19,597][105620] Updated weights for policy 1, policy_version 993531 (0.0008) [2023-12-26 22:36:19,598][105692] Updated weights for policy 0, policy_version 993224 (0.0009) [2023-12-26 22:36:19,659][105620] Updated weights for policy 1, policy_version 993541 (0.0009) [2023-12-26 22:36:20,388][105692] Updated weights for policy 0, policy_version 993234 (0.0008) [2023-12-26 22:36:20,416][105620] Updated weights for policy 1, policy_version 993551 (0.0007) [2023-12-26 22:36:20,445][105692] Updated weights for policy 0, policy_version 993244 (0.0008) [2023-12-26 22:36:20,466][105620] Updated weights for policy 1, policy_version 993561 (0.0009) [2023-12-26 22:36:20,509][105692] Updated weights for policy 0, policy_version 993254 (0.0008) [2023-12-26 22:36:20,520][105620] Updated weights for policy 1, policy_version 993571 (0.0006) [2023-12-26 22:36:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 508698624. Throughput: 0: 9703.8, 1: 9561.7. Samples: 508690328. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:36:21,062][104569] Avg episode reward: [(0, '8628.600'), (1, '9092.894')] [2023-12-26 22:36:21,293][105620] Updated weights for policy 1, policy_version 993581 (0.0007) [2023-12-26 22:36:21,351][105692] Updated weights for policy 0, policy_version 993264 (0.0008) [2023-12-26 22:36:21,355][105620] Updated weights for policy 1, policy_version 993591 (0.0007) [2023-12-26 22:36:21,415][105692] Updated weights for policy 0, policy_version 993274 (0.0009) [2023-12-26 22:36:21,422][105620] Updated weights for policy 1, policy_version 993601 (0.0008) [2023-12-26 22:36:21,475][105692] Updated weights for policy 0, policy_version 993284 (0.0008) [2023-12-26 22:36:22,133][105692] Updated weights for policy 0, policy_version 993294 (0.0008) [2023-12-26 22:36:22,182][105692] Updated weights for policy 0, policy_version 993304 (0.0008) [2023-12-26 22:36:22,219][105620] Updated weights for policy 1, policy_version 993611 (0.0010) [2023-12-26 22:36:22,226][105692] Updated weights for policy 0, policy_version 993314 (0.0008) [2023-12-26 22:36:22,285][105620] Updated weights for policy 1, policy_version 993621 (0.0011) [2023-12-26 22:36:22,342][105620] Updated weights for policy 1, policy_version 993631 (0.0011) [2023-12-26 22:36:23,034][105692] Updated weights for policy 0, policy_version 993324 (0.0008) [2023-12-26 22:36:23,093][105692] Updated weights for policy 0, policy_version 993334 (0.0008) [2023-12-26 22:36:23,100][105620] Updated weights for policy 1, policy_version 993641 (0.0010) [2023-12-26 22:36:23,147][105692] Updated weights for policy 0, policy_version 993344 (0.0007) [2023-12-26 22:36:23,153][105620] Updated weights for policy 1, policy_version 993651 (0.0010) [2023-12-26 22:36:23,206][105620] Updated weights for policy 1, policy_version 993661 (0.0011) [2023-12-26 22:36:23,258][105620] Updated weights for policy 1, policy_version 993671 (0.0011) [2023-12-26 22:36:23,909][105692] Updated weights for policy 0, policy_version 993354 (0.0006) [2023-12-26 22:36:23,965][105692] Updated weights for policy 0, policy_version 993364 (0.0011) [2023-12-26 22:36:23,994][105620] Updated weights for policy 1, policy_version 993681 (0.0010) [2023-12-26 22:36:24,018][105692] Updated weights for policy 0, policy_version 993374 (0.0010) [2023-12-26 22:36:24,039][105620] Updated weights for policy 1, policy_version 993691 (0.0010) [2023-12-26 22:36:24,066][105692] Updated weights for policy 0, policy_version 993384 (0.0010) [2023-12-26 22:36:24,087][105620] Updated weights for policy 1, policy_version 993701 (0.0010) [2023-12-26 22:36:24,823][105692] Updated weights for policy 0, policy_version 993394 (0.0006) [2023-12-26 22:36:24,862][105620] Updated weights for policy 1, policy_version 993711 (0.0010) [2023-12-26 22:36:24,880][105692] Updated weights for policy 0, policy_version 993404 (0.0005) [2023-12-26 22:36:24,927][105620] Updated weights for policy 1, policy_version 993721 (0.0010) [2023-12-26 22:36:24,934][105692] Updated weights for policy 0, policy_version 993414 (0.0005) [2023-12-26 22:36:24,983][105620] Updated weights for policy 1, policy_version 993731 (0.0011) [2023-12-26 22:36:25,616][105692] Updated weights for policy 0, policy_version 993424 (0.0009) [2023-12-26 22:36:25,670][105692] Updated weights for policy 0, policy_version 993434 (0.0010) [2023-12-26 22:36:25,717][105620] Updated weights for policy 1, policy_version 993741 (0.0011) [2023-12-26 22:36:25,728][105692] Updated weights for policy 0, policy_version 993444 (0.0010) [2023-12-26 22:36:25,734][105585] KL-divergence is very high: 134.5619 [2023-12-26 22:36:25,775][105620] Updated weights for policy 1, policy_version 993751 (0.0010) [2023-12-26 22:36:25,819][105620] Updated weights for policy 1, policy_version 993761 (0.0010) [2023-12-26 22:36:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 508796928. Throughput: 0: 9672.1, 1: 9492.2. Samples: 508802192. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:36:26,063][104569] Avg episode reward: [(0, '8900.618'), (1, '9092.924')] [2023-12-26 22:36:26,431][105692] Updated weights for policy 0, policy_version 993454 (0.0007) [2023-12-26 22:36:26,496][105692] Updated weights for policy 0, policy_version 993464 (0.0005) [2023-12-26 22:36:26,569][105692] Updated weights for policy 0, policy_version 993474 (0.0005) [2023-12-26 22:36:26,575][105620] Updated weights for policy 1, policy_version 993771 (0.0010) [2023-12-26 22:36:26,626][105620] Updated weights for policy 1, policy_version 993781 (0.0010) [2023-12-26 22:36:26,685][105620] Updated weights for policy 1, policy_version 993791 (0.0010) [2023-12-26 22:36:27,192][105692] Updated weights for policy 0, policy_version 993484 (0.0006) [2023-12-26 22:36:27,237][105692] Updated weights for policy 0, policy_version 993494 (0.0005) [2023-12-26 22:36:27,282][105692] Updated weights for policy 0, policy_version 993504 (0.0005) [2023-12-26 22:36:27,459][105620] Updated weights for policy 1, policy_version 993801 (0.0010) [2023-12-26 22:36:27,512][105620] Updated weights for policy 1, policy_version 993811 (0.0005) [2023-12-26 22:36:27,561][105620] Updated weights for policy 1, policy_version 993821 (0.0007) [2023-12-26 22:36:27,604][105620] Updated weights for policy 1, policy_version 993831 (0.0007) [2023-12-26 22:36:27,991][105692] Updated weights for policy 0, policy_version 993514 (0.0008) [2023-12-26 22:36:28,038][105692] Updated weights for policy 0, policy_version 993524 (0.0010) [2023-12-26 22:36:28,086][105692] Updated weights for policy 0, policy_version 993534 (0.0010) [2023-12-26 22:36:28,139][105692] Updated weights for policy 0, policy_version 993544 (0.0010) [2023-12-26 22:36:28,320][105620] Updated weights for policy 1, policy_version 993841 (0.0008) [2023-12-26 22:36:28,382][105620] Updated weights for policy 1, policy_version 993851 (0.0007) [2023-12-26 22:36:28,437][105620] Updated weights for policy 1, policy_version 993861 (0.0006) [2023-12-26 22:36:28,908][105692] Updated weights for policy 0, policy_version 993554 (0.0011) [2023-12-26 22:36:28,974][105692] Updated weights for policy 0, policy_version 993564 (0.0010) [2023-12-26 22:36:29,022][105692] Updated weights for policy 0, policy_version 993574 (0.0010) [2023-12-26 22:36:29,121][105620] Updated weights for policy 1, policy_version 993871 (0.0010) [2023-12-26 22:36:29,176][105620] Updated weights for policy 1, policy_version 993881 (0.0010) [2023-12-26 22:36:29,237][105620] Updated weights for policy 1, policy_version 993891 (0.0011) [2023-12-26 22:36:29,715][105692] Updated weights for policy 0, policy_version 993584 (0.0007) [2023-12-26 22:36:29,782][105692] Updated weights for policy 0, policy_version 993594 (0.0005) [2023-12-26 22:36:29,841][105692] Updated weights for policy 0, policy_version 993604 (0.0007) [2023-12-26 22:36:29,950][105620] Updated weights for policy 1, policy_version 993901 (0.0011) [2023-12-26 22:36:30,002][105620] Updated weights for policy 1, policy_version 993911 (0.0010) [2023-12-26 22:36:30,057][105620] Updated weights for policy 1, policy_version 993921 (0.0005) [2023-12-26 22:36:30,583][105692] Updated weights for policy 0, policy_version 993614 (0.0009) [2023-12-26 22:36:30,634][105692] Updated weights for policy 0, policy_version 993625 (0.0009) [2023-12-26 22:36:30,644][105620] Updated weights for policy 1, policy_version 993931 (0.0005) [2023-12-26 22:36:30,689][105692] Updated weights for policy 0, policy_version 993635 (0.0010) [2023-12-26 22:36:30,695][105620] Updated weights for policy 1, policy_version 993941 (0.0005) [2023-12-26 22:36:30,744][105620] Updated weights for policy 1, policy_version 993951 (0.0009) [2023-12-26 22:36:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 508895232. Throughput: 0: 9682.4, 1: 9517.3. Samples: 508861512. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:36:31,063][104569] Avg episode reward: [(0, '9081.332'), (1, '8748.678')] [2023-12-26 22:36:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000993960_254484480.pth... [2023-12-26 22:36:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000993640_254410752.pth... [2023-12-26 22:36:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000992520_254124032.pth [2023-12-26 22:36:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000992872_254205952.pth [2023-12-26 22:36:31,429][105620] Updated weights for policy 1, policy_version 993961 (0.0010) [2023-12-26 22:36:31,453][105692] Updated weights for policy 0, policy_version 993645 (0.0010) [2023-12-26 22:36:31,486][105620] Updated weights for policy 1, policy_version 993971 (0.0010) [2023-12-26 22:36:31,501][105692] Updated weights for policy 0, policy_version 993655 (0.0010) [2023-12-26 22:36:31,542][105620] Updated weights for policy 1, policy_version 993981 (0.0008) [2023-12-26 22:36:31,551][105692] Updated weights for policy 0, policy_version 993665 (0.0010) [2023-12-26 22:36:31,602][105620] Updated weights for policy 1, policy_version 993991 (0.0006) [2023-12-26 22:36:32,302][105620] Updated weights for policy 1, policy_version 994001 (0.0008) [2023-12-26 22:36:32,321][105692] Updated weights for policy 0, policy_version 993675 (0.0010) [2023-12-26 22:36:32,360][105620] Updated weights for policy 1, policy_version 994011 (0.0006) [2023-12-26 22:36:32,382][105692] Updated weights for policy 0, policy_version 993685 (0.0010) [2023-12-26 22:36:32,425][105620] Updated weights for policy 1, policy_version 994021 (0.0007) [2023-12-26 22:36:32,437][105692] Updated weights for policy 0, policy_version 993695 (0.0010) [2023-12-26 22:36:33,120][105620] Updated weights for policy 1, policy_version 994031 (0.0008) [2023-12-26 22:36:33,172][105692] Updated weights for policy 0, policy_version 993705 (0.0010) [2023-12-26 22:36:33,174][105620] Updated weights for policy 1, policy_version 994041 (0.0007) [2023-12-26 22:36:33,228][105620] Updated weights for policy 1, policy_version 994051 (0.0005) [2023-12-26 22:36:33,230][105692] Updated weights for policy 0, policy_version 993715 (0.0010) [2023-12-26 22:36:33,284][105692] Updated weights for policy 0, policy_version 993725 (0.0010) [2023-12-26 22:36:33,335][105692] Updated weights for policy 0, policy_version 993735 (0.0010) [2023-12-26 22:36:33,935][105692] Updated weights for policy 0, policy_version 993745 (0.0006) [2023-12-26 22:36:33,984][105692] Updated weights for policy 0, policy_version 993755 (0.0006) [2023-12-26 22:36:34,030][105692] Updated weights for policy 0, policy_version 993765 (0.0006) [2023-12-26 22:36:34,060][105620] Updated weights for policy 1, policy_version 994061 (0.0008) [2023-12-26 22:36:34,116][105620] Updated weights for policy 1, policy_version 994071 (0.0008) [2023-12-26 22:36:34,185][105620] Updated weights for policy 1, policy_version 994081 (0.0008) [2023-12-26 22:36:34,783][105692] Updated weights for policy 0, policy_version 993775 (0.0009) [2023-12-26 22:36:34,841][105692] Updated weights for policy 0, policy_version 993785 (0.0007) [2023-12-26 22:36:34,886][105620] Updated weights for policy 1, policy_version 994091 (0.0006) [2023-12-26 22:36:34,899][105692] Updated weights for policy 0, policy_version 993795 (0.0009) [2023-12-26 22:36:34,944][105620] Updated weights for policy 1, policy_version 994101 (0.0007) [2023-12-26 22:36:34,992][105620] Updated weights for policy 1, policy_version 994111 (0.0008) [2023-12-26 22:36:35,664][105692] Updated weights for policy 0, policy_version 993805 (0.0008) [2023-12-26 22:36:35,719][105692] Updated weights for policy 0, policy_version 993815 (0.0009) [2023-12-26 22:36:35,766][105692] Updated weights for policy 0, policy_version 993825 (0.0008) [2023-12-26 22:36:35,768][105620] Updated weights for policy 1, policy_version 994121 (0.0009) [2023-12-26 22:36:35,818][105620] Updated weights for policy 1, policy_version 994131 (0.0008) [2023-12-26 22:36:35,869][105620] Updated weights for policy 1, policy_version 994141 (0.0009) [2023-12-26 22:36:35,916][105620] Updated weights for policy 1, policy_version 994151 (0.0008) [2023-12-26 22:36:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 508993536. Throughput: 0: 9602.6, 1: 9554.3. Samples: 508978792. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:36:36,062][104569] Avg episode reward: [(0, '8811.900'), (1, '8664.706')] [2023-12-26 22:36:36,565][105692] Updated weights for policy 0, policy_version 993835 (0.0007) [2023-12-26 22:36:36,615][105692] Updated weights for policy 0, policy_version 993845 (0.0010) [2023-12-26 22:36:36,663][105692] Updated weights for policy 0, policy_version 993855 (0.0009) [2023-12-26 22:36:36,708][105620] Updated weights for policy 1, policy_version 994161 (0.0008) [2023-12-26 22:36:36,768][105620] Updated weights for policy 1, policy_version 994171 (0.0010) [2023-12-26 22:36:36,841][105620] Updated weights for policy 1, policy_version 994181 (0.0010) [2023-12-26 22:36:37,380][105692] Updated weights for policy 0, policy_version 993865 (0.0007) [2023-12-26 22:36:37,437][105692] Updated weights for policy 0, policy_version 993875 (0.0008) [2023-12-26 22:36:37,494][105692] Updated weights for policy 0, policy_version 993885 (0.0010) [2023-12-26 22:36:37,516][105620] Updated weights for policy 1, policy_version 994191 (0.0007) [2023-12-26 22:36:37,552][105692] Updated weights for policy 0, policy_version 993895 (0.0009) [2023-12-26 22:36:37,573][105620] Updated weights for policy 1, policy_version 994201 (0.0006) [2023-12-26 22:36:37,636][105620] Updated weights for policy 1, policy_version 994211 (0.0005) [2023-12-26 22:36:38,178][105620] Updated weights for policy 1, policy_version 994221 (0.0008) [2023-12-26 22:36:38,237][105620] Updated weights for policy 1, policy_version 994231 (0.0010) [2023-12-26 22:36:38,292][105620] Updated weights for policy 1, policy_version 994241 (0.0009) [2023-12-26 22:36:38,443][105692] Updated weights for policy 0, policy_version 993905 (0.0009) [2023-12-26 22:36:38,500][105692] Updated weights for policy 0, policy_version 993915 (0.0008) [2023-12-26 22:36:38,553][105692] Updated weights for policy 0, policy_version 993925 (0.0009) [2023-12-26 22:36:39,000][105620] Updated weights for policy 1, policy_version 994251 (0.0008) [2023-12-26 22:36:39,063][105620] Updated weights for policy 1, policy_version 994261 (0.0010) [2023-12-26 22:36:39,123][105620] Updated weights for policy 1, policy_version 994271 (0.0010) [2023-12-26 22:36:39,403][105692] Updated weights for policy 0, policy_version 993935 (0.0008) [2023-12-26 22:36:39,473][105692] Updated weights for policy 0, policy_version 993945 (0.0007) [2023-12-26 22:36:39,534][105692] Updated weights for policy 0, policy_version 993955 (0.0009) [2023-12-26 22:36:39,864][105620] Updated weights for policy 1, policy_version 994281 (0.0010) [2023-12-26 22:36:39,925][105620] Updated weights for policy 1, policy_version 994291 (0.0011) [2023-12-26 22:36:39,985][105620] Updated weights for policy 1, policy_version 994301 (0.0011) [2023-12-26 22:36:40,050][105620] Updated weights for policy 1, policy_version 994311 (0.0010) [2023-12-26 22:36:40,220][105692] Updated weights for policy 0, policy_version 993965 (0.0007) [2023-12-26 22:36:40,278][105692] Updated weights for policy 0, policy_version 993975 (0.0006) [2023-12-26 22:36:40,344][105692] Updated weights for policy 0, policy_version 993985 (0.0009) [2023-12-26 22:36:40,822][105620] Updated weights for policy 1, policy_version 994321 (0.0006) [2023-12-26 22:36:40,882][105620] Updated weights for policy 1, policy_version 994331 (0.0007) [2023-12-26 22:36:40,936][105620] Updated weights for policy 1, policy_version 994341 (0.0006) [2023-12-26 22:36:41,011][105692] Updated weights for policy 0, policy_version 993995 (0.0010) [2023-12-26 22:36:41,065][104569] Fps is (10 sec: 18836.5, 60 sec: 19113.8, 300 sec: 19521.8). Total num frames: 509083648. Throughput: 0: 9563.8, 1: 9514.5. Samples: 509092196. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:36:41,065][104569] Avg episode reward: [(0, '8812.293'), (1, '8585.921')] [2023-12-26 22:36:41,079][105692] Updated weights for policy 0, policy_version 994005 (0.0008) [2023-12-26 22:36:41,140][105692] Updated weights for policy 0, policy_version 994015 (0.0008) [2023-12-26 22:36:41,649][105620] Updated weights for policy 1, policy_version 994351 (0.0007) [2023-12-26 22:36:41,716][105620] Updated weights for policy 1, policy_version 994361 (0.0010) [2023-12-26 22:36:41,782][105620] Updated weights for policy 1, policy_version 994371 (0.0006) [2023-12-26 22:36:41,861][105692] Updated weights for policy 0, policy_version 994025 (0.0007) [2023-12-26 22:36:41,931][105692] Updated weights for policy 0, policy_version 994035 (0.0009) [2023-12-26 22:36:41,995][105692] Updated weights for policy 0, policy_version 994045 (0.0010) [2023-12-26 22:36:42,059][105692] Updated weights for policy 0, policy_version 994055 (0.0008) [2023-12-26 22:36:42,434][105620] Updated weights for policy 1, policy_version 994381 (0.0008) [2023-12-26 22:36:42,486][105620] Updated weights for policy 1, policy_version 994391 (0.0011) [2023-12-26 22:36:42,549][105620] Updated weights for policy 1, policy_version 994401 (0.0011) [2023-12-26 22:36:42,855][105692] Updated weights for policy 0, policy_version 994065 (0.0008) [2023-12-26 22:36:42,917][105692] Updated weights for policy 0, policy_version 994075 (0.0008) [2023-12-26 22:36:42,972][105692] Updated weights for policy 0, policy_version 994085 (0.0008) [2023-12-26 22:36:43,280][105620] Updated weights for policy 1, policy_version 994411 (0.0007) [2023-12-26 22:36:43,353][105620] Updated weights for policy 1, policy_version 994421 (0.0010) [2023-12-26 22:36:43,415][105620] Updated weights for policy 1, policy_version 994431 (0.0010) [2023-12-26 22:36:43,700][105692] Updated weights for policy 0, policy_version 994095 (0.0010) [2023-12-26 22:36:43,748][105692] Updated weights for policy 0, policy_version 994105 (0.0010) [2023-12-26 22:36:43,793][105692] Updated weights for policy 0, policy_version 994115 (0.0010) [2023-12-26 22:36:44,119][105620] Updated weights for policy 1, policy_version 994441 (0.0010) [2023-12-26 22:36:44,165][105620] Updated weights for policy 1, policy_version 994451 (0.0008) [2023-12-26 22:36:44,215][105620] Updated weights for policy 1, policy_version 994461 (0.0008) [2023-12-26 22:36:44,276][105620] Updated weights for policy 1, policy_version 994471 (0.0007) [2023-12-26 22:36:44,426][105692] Updated weights for policy 0, policy_version 994125 (0.0010) [2023-12-26 22:36:44,471][105692] Updated weights for policy 0, policy_version 994135 (0.0010) [2023-12-26 22:36:44,524][105692] Updated weights for policy 0, policy_version 994145 (0.0010) [2023-12-26 22:36:44,904][105620] Updated weights for policy 1, policy_version 994481 (0.0011) [2023-12-26 22:36:44,972][105620] Updated weights for policy 1, policy_version 994491 (0.0008) [2023-12-26 22:36:45,035][105620] Updated weights for policy 1, policy_version 994501 (0.0010) [2023-12-26 22:36:45,348][105692] Updated weights for policy 0, policy_version 994156 (0.0009) [2023-12-26 22:36:45,407][105692] Updated weights for policy 0, policy_version 994166 (0.0008) [2023-12-26 22:36:45,455][105692] Updated weights for policy 0, policy_version 994176 (0.0008) [2023-12-26 22:36:45,781][105620] Updated weights for policy 1, policy_version 994511 (0.0008) [2023-12-26 22:36:45,839][105620] Updated weights for policy 1, policy_version 994521 (0.0010) [2023-12-26 22:36:45,902][105620] Updated weights for policy 1, policy_version 994531 (0.0011) [2023-12-26 22:36:46,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 509181952. Throughput: 0: 9513.9, 1: 9526.4. Samples: 509149956. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:36:46,063][104569] Avg episode reward: [(0, '8991.958'), (1, '8055.868')] [2023-12-26 22:36:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000994536_254631936.pth... [2023-12-26 22:36:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000994184_254550016.pth... [2023-12-26 22:36:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000993416_254345216.pth [2023-12-26 22:36:46,081][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000993096_254271488.pth [2023-12-26 22:36:46,236][105692] Updated weights for policy 0, policy_version 994186 (0.0007) [2023-12-26 22:36:46,284][105692] Updated weights for policy 0, policy_version 994196 (0.0008) [2023-12-26 22:36:46,332][105692] Updated weights for policy 0, policy_version 994206 (0.0008) [2023-12-26 22:36:46,386][105692] Updated weights for policy 0, policy_version 994216 (0.0008) [2023-12-26 22:36:46,543][105620] Updated weights for policy 1, policy_version 994541 (0.0010) [2023-12-26 22:36:46,601][105620] Updated weights for policy 1, policy_version 994551 (0.0010) [2023-12-26 22:36:46,659][105620] Updated weights for policy 1, policy_version 994561 (0.0010) [2023-12-26 22:36:47,177][105692] Updated weights for policy 0, policy_version 994226 (0.0008) [2023-12-26 22:36:47,239][105692] Updated weights for policy 0, policy_version 994236 (0.0008) [2023-12-26 22:36:47,305][105692] Updated weights for policy 0, policy_version 994246 (0.0008) [2023-12-26 22:36:47,416][105620] Updated weights for policy 1, policy_version 994571 (0.0010) [2023-12-26 22:36:47,467][105620] Updated weights for policy 1, policy_version 994581 (0.0010) [2023-12-26 22:36:47,515][105620] Updated weights for policy 1, policy_version 994591 (0.0010) [2023-12-26 22:36:47,910][105692] Updated weights for policy 0, policy_version 994256 (0.0009) [2023-12-26 22:36:47,965][105692] Updated weights for policy 0, policy_version 994266 (0.0005) [2023-12-26 22:36:48,030][105692] Updated weights for policy 0, policy_version 994276 (0.0008) [2023-12-26 22:36:48,165][105620] Updated weights for policy 1, policy_version 994601 (0.0010) [2023-12-26 22:36:48,220][105620] Updated weights for policy 1, policy_version 994611 (0.0010) [2023-12-26 22:36:48,278][105620] Updated weights for policy 1, policy_version 994621 (0.0010) [2023-12-26 22:36:48,343][105620] Updated weights for policy 1, policy_version 994631 (0.0010) [2023-12-26 22:36:48,693][105692] Updated weights for policy 0, policy_version 994286 (0.0009) [2023-12-26 22:36:48,756][105692] Updated weights for policy 0, policy_version 994296 (0.0008) [2023-12-26 22:36:48,819][105692] Updated weights for policy 0, policy_version 994306 (0.0008) [2023-12-26 22:36:49,109][105620] Updated weights for policy 1, policy_version 994641 (0.0010) [2023-12-26 22:36:49,162][105620] Updated weights for policy 1, policy_version 994651 (0.0011) [2023-12-26 22:36:49,224][105620] Updated weights for policy 1, policy_version 994661 (0.0011) [2023-12-26 22:36:49,445][105692] Updated weights for policy 0, policy_version 994316 (0.0007) [2023-12-26 22:36:49,494][105692] Updated weights for policy 0, policy_version 994326 (0.0008) [2023-12-26 22:36:49,542][105692] Updated weights for policy 0, policy_version 994336 (0.0008) [2023-12-26 22:36:49,967][105620] Updated weights for policy 1, policy_version 994671 (0.0010) [2023-12-26 22:36:50,032][105620] Updated weights for policy 1, policy_version 994681 (0.0009) [2023-12-26 22:36:50,098][105620] Updated weights for policy 1, policy_version 994691 (0.0011) [2023-12-26 22:36:50,274][105692] Updated weights for policy 0, policy_version 994346 (0.0007) [2023-12-26 22:36:50,325][105692] Updated weights for policy 0, policy_version 994356 (0.0005) [2023-12-26 22:36:50,381][105692] Updated weights for policy 0, policy_version 994366 (0.0009) [2023-12-26 22:36:50,435][105692] Updated weights for policy 0, policy_version 994376 (0.0010) [2023-12-26 22:36:50,721][105620] Updated weights for policy 1, policy_version 994701 (0.0010) [2023-12-26 22:36:50,783][105620] Updated weights for policy 1, policy_version 994711 (0.0008) [2023-12-26 22:36:50,849][105620] Updated weights for policy 1, policy_version 994721 (0.0008) [2023-12-26 22:36:51,062][104569] Fps is (10 sec: 19666.3, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 509280256. Throughput: 0: 9620.9, 1: 9529.5. Samples: 509268652. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:36:51,063][104569] Avg episode reward: [(0, '8838.737'), (1, '8334.354')] [2023-12-26 22:36:51,271][105692] Updated weights for policy 0, policy_version 994386 (0.0007) [2023-12-26 22:36:51,338][105692] Updated weights for policy 0, policy_version 994396 (0.0007) [2023-12-26 22:36:51,392][105692] Updated weights for policy 0, policy_version 994406 (0.0008) [2023-12-26 22:36:51,513][105620] Updated weights for policy 1, policy_version 994731 (0.0009) [2023-12-26 22:36:51,575][105620] Updated weights for policy 1, policy_version 994741 (0.0009) [2023-12-26 22:36:51,638][105620] Updated weights for policy 1, policy_version 994751 (0.0008) [2023-12-26 22:36:52,142][105692] Updated weights for policy 0, policy_version 994416 (0.0009) [2023-12-26 22:36:52,198][105692] Updated weights for policy 0, policy_version 994426 (0.0009) [2023-12-26 22:36:52,267][105692] Updated weights for policy 0, policy_version 994436 (0.0009) [2023-12-26 22:36:52,274][105620] Updated weights for policy 1, policy_version 994761 (0.0009) [2023-12-26 22:36:52,335][105620] Updated weights for policy 1, policy_version 994771 (0.0008) [2023-12-26 22:36:52,396][105620] Updated weights for policy 1, policy_version 994781 (0.0009) [2023-12-26 22:36:52,457][105620] Updated weights for policy 1, policy_version 994791 (0.0008) [2023-12-26 22:36:53,005][105692] Updated weights for policy 0, policy_version 994446 (0.0006) [2023-12-26 22:36:53,068][105692] Updated weights for policy 0, policy_version 994456 (0.0008) [2023-12-26 22:36:53,123][105692] Updated weights for policy 0, policy_version 994466 (0.0009) [2023-12-26 22:36:53,223][105620] Updated weights for policy 1, policy_version 994801 (0.0009) [2023-12-26 22:36:53,285][105620] Updated weights for policy 1, policy_version 994811 (0.0009) [2023-12-26 22:36:53,332][105620] Updated weights for policy 1, policy_version 994821 (0.0008) [2023-12-26 22:36:53,740][105692] Updated weights for policy 0, policy_version 994476 (0.0007) [2023-12-26 22:36:53,794][105692] Updated weights for policy 0, policy_version 994486 (0.0007) [2023-12-26 22:36:53,853][105692] Updated weights for policy 0, policy_version 994496 (0.0011) [2023-12-26 22:36:54,072][105620] Updated weights for policy 1, policy_version 994831 (0.0008) [2023-12-26 22:36:54,120][105620] Updated weights for policy 1, policy_version 994841 (0.0008) [2023-12-26 22:36:54,177][105620] Updated weights for policy 1, policy_version 994851 (0.0008) [2023-12-26 22:36:54,571][105692] Updated weights for policy 0, policy_version 994506 (0.0011) [2023-12-26 22:36:54,626][105692] Updated weights for policy 0, policy_version 994516 (0.0010) [2023-12-26 22:36:54,682][105692] Updated weights for policy 0, policy_version 994526 (0.0010) [2023-12-26 22:36:54,734][105692] Updated weights for policy 0, policy_version 994536 (0.0011) [2023-12-26 22:36:54,943][105620] Updated weights for policy 1, policy_version 994861 (0.0008) [2023-12-26 22:36:54,997][105620] Updated weights for policy 1, policy_version 994871 (0.0007) [2023-12-26 22:36:55,044][105620] Updated weights for policy 1, policy_version 994881 (0.0006) [2023-12-26 22:36:55,483][105692] Updated weights for policy 0, policy_version 994546 (0.0011) [2023-12-26 22:36:55,537][105692] Updated weights for policy 0, policy_version 994556 (0.0010) [2023-12-26 22:36:55,589][105692] Updated weights for policy 0, policy_version 994566 (0.0009) [2023-12-26 22:36:55,769][105620] Updated weights for policy 1, policy_version 994891 (0.0006) [2023-12-26 22:36:55,824][105620] Updated weights for policy 1, policy_version 994901 (0.0008) [2023-12-26 22:36:55,880][105620] Updated weights for policy 1, policy_version 994911 (0.0008) [2023-12-26 22:36:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 509378560. Throughput: 0: 9598.8, 1: 9603.3. Samples: 509384840. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:36:56,062][104569] Avg episode reward: [(0, '8475.747'), (1, '3231.775')] [2023-12-26 22:36:56,341][105692] Updated weights for policy 0, policy_version 994576 (0.0007) [2023-12-26 22:36:56,387][105692] Updated weights for policy 0, policy_version 994586 (0.0005) [2023-12-26 22:36:56,439][105692] Updated weights for policy 0, policy_version 994596 (0.0010) [2023-12-26 22:36:56,650][105620] Updated weights for policy 1, policy_version 994921 (0.0007) [2023-12-26 22:36:56,713][105620] Updated weights for policy 1, policy_version 994931 (0.0005) [2023-12-26 22:36:56,759][105620] Updated weights for policy 1, policy_version 994941 (0.0005) [2023-12-26 22:36:56,801][105620] Updated weights for policy 1, policy_version 994951 (0.0005) [2023-12-26 22:36:57,115][105692] Updated weights for policy 0, policy_version 994606 (0.0010) [2023-12-26 22:36:57,173][105692] Updated weights for policy 0, policy_version 994616 (0.0010) [2023-12-26 22:36:57,225][105692] Updated weights for policy 0, policy_version 994626 (0.0010) [2023-12-26 22:36:57,346][105620] Updated weights for policy 1, policy_version 994961 (0.0009) [2023-12-26 22:36:57,394][105620] Updated weights for policy 1, policy_version 994971 (0.0008) [2023-12-26 22:36:57,435][105620] Updated weights for policy 1, policy_version 994981 (0.0007) [2023-12-26 22:36:57,918][105692] Updated weights for policy 0, policy_version 994636 (0.0009) [2023-12-26 22:36:57,979][105692] Updated weights for policy 0, policy_version 994646 (0.0009) [2023-12-26 22:36:58,040][105692] Updated weights for policy 0, policy_version 994656 (0.0010) [2023-12-26 22:36:58,141][105620] Updated weights for policy 1, policy_version 994991 (0.0010) [2023-12-26 22:36:58,214][105620] Updated weights for policy 1, policy_version 995001 (0.0011) [2023-12-26 22:36:58,274][105620] Updated weights for policy 1, policy_version 995011 (0.0011) [2023-12-26 22:36:58,781][105692] Updated weights for policy 0, policy_version 994666 (0.0010) [2023-12-26 22:36:58,845][105692] Updated weights for policy 0, policy_version 994676 (0.0011) [2023-12-26 22:36:58,911][105692] Updated weights for policy 0, policy_version 994686 (0.0011) [2023-12-26 22:36:58,970][105692] Updated weights for policy 0, policy_version 994696 (0.0010) [2023-12-26 22:36:58,992][105620] Updated weights for policy 1, policy_version 995021 (0.0008) [2023-12-26 22:36:59,053][105620] Updated weights for policy 1, policy_version 995031 (0.0006) [2023-12-26 22:36:59,110][105620] Updated weights for policy 1, policy_version 995041 (0.0005) [2023-12-26 22:36:59,714][105692] Updated weights for policy 0, policy_version 994706 (0.0010) [2023-12-26 22:36:59,721][105620] Updated weights for policy 1, policy_version 995051 (0.0007) [2023-12-26 22:36:59,768][105692] Updated weights for policy 0, policy_version 994716 (0.0010) [2023-12-26 22:36:59,779][105620] Updated weights for policy 1, policy_version 995061 (0.0006) [2023-12-26 22:36:59,827][105692] Updated weights for policy 0, policy_version 994726 (0.0010) [2023-12-26 22:36:59,846][105620] Updated weights for policy 1, policy_version 995071 (0.0006) [2023-12-26 22:37:00,496][105692] Updated weights for policy 0, policy_version 994736 (0.0009) [2023-12-26 22:37:00,497][105620] Updated weights for policy 1, policy_version 995081 (0.0007) [2023-12-26 22:37:00,560][105620] Updated weights for policy 1, policy_version 995091 (0.0007) [2023-12-26 22:37:00,561][105692] Updated weights for policy 0, policy_version 994746 (0.0007) [2023-12-26 22:37:00,615][105620] Updated weights for policy 1, policy_version 995101 (0.0006) [2023-12-26 22:37:00,623][105692] Updated weights for policy 0, policy_version 994756 (0.0007) [2023-12-26 22:37:00,661][105620] Updated weights for policy 1, policy_version 995111 (0.0005) [2023-12-26 22:37:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.3, 300 sec: 19522.0). Total num frames: 509476864. Throughput: 0: 9593.5, 1: 9711.5. Samples: 509445380. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:37:01,062][104569] Avg episode reward: [(0, '8812.326'), (1, '3131.113')] [2023-12-26 22:37:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000994760_254697472.pth... [2023-12-26 22:37:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000995112_254779392.pth... [2023-12-26 22:37:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000993960_254484480.pth [2023-12-26 22:37:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000993640_254410752.pth [2023-12-26 22:37:01,229][105620] Updated weights for policy 1, policy_version 995121 (0.0006) [2023-12-26 22:37:01,295][105620] Updated weights for policy 1, policy_version 995131 (0.0008) [2023-12-26 22:37:01,302][105692] Updated weights for policy 0, policy_version 994766 (0.0007) [2023-12-26 22:37:01,366][105620] Updated weights for policy 1, policy_version 995141 (0.0008) [2023-12-26 22:37:01,366][105692] Updated weights for policy 0, policy_version 994776 (0.0009) [2023-12-26 22:37:01,419][105692] Updated weights for policy 0, policy_version 994786 (0.0008) [2023-12-26 22:37:02,051][105692] Updated weights for policy 0, policy_version 994796 (0.0008) [2023-12-26 22:37:02,062][105620] Updated weights for policy 1, policy_version 995151 (0.0010) [2023-12-26 22:37:02,111][105692] Updated weights for policy 0, policy_version 994806 (0.0005) [2023-12-26 22:37:02,113][105620] Updated weights for policy 1, policy_version 995161 (0.0009) [2023-12-26 22:37:02,166][105692] Updated weights for policy 0, policy_version 994816 (0.0005) [2023-12-26 22:37:02,174][105620] Updated weights for policy 1, policy_version 995171 (0.0006) [2023-12-26 22:37:02,839][105692] Updated weights for policy 0, policy_version 994826 (0.0006) [2023-12-26 22:37:02,886][105692] Updated weights for policy 0, policy_version 994836 (0.0009) [2023-12-26 22:37:02,934][105692] Updated weights for policy 0, policy_version 994846 (0.0007) [2023-12-26 22:37:02,938][105620] Updated weights for policy 1, policy_version 995181 (0.0007) [2023-12-26 22:37:02,987][105692] Updated weights for policy 0, policy_version 994856 (0.0007) [2023-12-26 22:37:02,989][105620] Updated weights for policy 1, policy_version 995191 (0.0008) [2023-12-26 22:37:03,041][105620] Updated weights for policy 1, policy_version 995201 (0.0008) [2023-12-26 22:37:03,676][105620] Updated weights for policy 1, policy_version 995211 (0.0007) [2023-12-26 22:37:03,731][105620] Updated weights for policy 1, policy_version 995221 (0.0005) [2023-12-26 22:37:03,789][105620] Updated weights for policy 1, policy_version 995231 (0.0005) [2023-12-26 22:37:03,819][105692] Updated weights for policy 0, policy_version 994866 (0.0008) [2023-12-26 22:37:03,879][105692] Updated weights for policy 0, policy_version 994876 (0.0009) [2023-12-26 22:37:03,931][105692] Updated weights for policy 0, policy_version 994886 (0.0009) [2023-12-26 22:37:04,359][105620] Updated weights for policy 1, policy_version 995241 (0.0007) [2023-12-26 22:37:04,422][105620] Updated weights for policy 1, policy_version 995251 (0.0007) [2023-12-26 22:37:04,473][105620] Updated weights for policy 1, policy_version 995261 (0.0009) [2023-12-26 22:37:04,521][105620] Updated weights for policy 1, policy_version 995271 (0.0009) [2023-12-26 22:37:04,772][105692] Updated weights for policy 0, policy_version 994896 (0.0009) [2023-12-26 22:37:04,818][105692] Updated weights for policy 0, policy_version 994906 (0.0009) [2023-12-26 22:37:04,865][105692] Updated weights for policy 0, policy_version 994916 (0.0009) [2023-12-26 22:37:05,256][105620] Updated weights for policy 1, policy_version 995281 (0.0006) [2023-12-26 22:37:05,327][105620] Updated weights for policy 1, policy_version 995291 (0.0006) [2023-12-26 22:37:05,396][105620] Updated weights for policy 1, policy_version 995301 (0.0005) [2023-12-26 22:37:05,694][105692] Updated weights for policy 0, policy_version 994926 (0.0008) [2023-12-26 22:37:05,755][105692] Updated weights for policy 0, policy_version 994936 (0.0009) [2023-12-26 22:37:05,817][105692] Updated weights for policy 0, policy_version 994946 (0.0009) [2023-12-26 22:37:06,025][105620] Updated weights for policy 1, policy_version 995311 (0.0009) [2023-12-26 22:37:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 509575168. Throughput: 0: 9599.9, 1: 9871.4. Samples: 509566536. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:37:06,062][104569] Avg episode reward: [(0, '8995.597'), (1, '6053.066')] [2023-12-26 22:37:06,077][105620] Updated weights for policy 1, policy_version 995321 (0.0009) [2023-12-26 22:37:06,136][105620] Updated weights for policy 1, policy_version 995331 (0.0008) [2023-12-26 22:37:06,580][105692] Updated weights for policy 0, policy_version 994956 (0.0010) [2023-12-26 22:37:06,635][105692] Updated weights for policy 0, policy_version 994966 (0.0009) [2023-12-26 22:37:06,690][105692] Updated weights for policy 0, policy_version 994976 (0.0009) [2023-12-26 22:37:06,897][105620] Updated weights for policy 1, policy_version 995341 (0.0008) [2023-12-26 22:37:06,960][105620] Updated weights for policy 1, policy_version 995351 (0.0007) [2023-12-26 22:37:07,013][105620] Updated weights for policy 1, policy_version 995361 (0.0005) [2023-12-26 22:37:07,524][105692] Updated weights for policy 0, policy_version 994986 (0.0009) [2023-12-26 22:37:07,573][105692] Updated weights for policy 0, policy_version 994996 (0.0008) [2023-12-26 22:37:07,642][105692] Updated weights for policy 0, policy_version 995006 (0.0009) [2023-12-26 22:37:07,646][105620] Updated weights for policy 1, policy_version 995371 (0.0007) [2023-12-26 22:37:07,692][105620] Updated weights for policy 1, policy_version 995381 (0.0006) [2023-12-26 22:37:07,701][105692] Updated weights for policy 0, policy_version 995016 (0.0008) [2023-12-26 22:37:07,740][105620] Updated weights for policy 1, policy_version 995391 (0.0008) [2023-12-26 22:37:08,470][105692] Updated weights for policy 0, policy_version 995026 (0.0009) [2023-12-26 22:37:08,476][105620] Updated weights for policy 1, policy_version 995401 (0.0008) [2023-12-26 22:37:08,531][105692] Updated weights for policy 0, policy_version 995036 (0.0008) [2023-12-26 22:37:08,533][105620] Updated weights for policy 1, policy_version 995411 (0.0006) [2023-12-26 22:37:08,592][105692] Updated weights for policy 0, policy_version 995046 (0.0007) [2023-12-26 22:37:08,599][105620] Updated weights for policy 1, policy_version 995421 (0.0008) [2023-12-26 22:37:08,648][105620] Updated weights for policy 1, policy_version 995431 (0.0008) [2023-12-26 22:37:09,351][105692] Updated weights for policy 0, policy_version 995056 (0.0008) [2023-12-26 22:37:09,417][105692] Updated weights for policy 0, policy_version 995066 (0.0008) [2023-12-26 22:37:09,440][105620] Updated weights for policy 1, policy_version 995441 (0.0007) [2023-12-26 22:37:09,477][105692] Updated weights for policy 0, policy_version 995076 (0.0008) [2023-12-26 22:37:09,501][105620] Updated weights for policy 1, policy_version 995451 (0.0005) [2023-12-26 22:37:09,554][105620] Updated weights for policy 1, policy_version 995461 (0.0006) [2023-12-26 22:37:10,205][105692] Updated weights for policy 0, policy_version 995086 (0.0009) [2023-12-26 22:37:10,270][105692] Updated weights for policy 0, policy_version 995096 (0.0007) [2023-12-26 22:37:10,292][105620] Updated weights for policy 1, policy_version 995471 (0.0009) [2023-12-26 22:37:10,331][105692] Updated weights for policy 0, policy_version 995106 (0.0006) [2023-12-26 22:37:10,357][105620] Updated weights for policy 1, policy_version 995481 (0.0006) [2023-12-26 22:37:10,412][105620] Updated weights for policy 1, policy_version 995491 (0.0008) [2023-12-26 22:37:11,027][105692] Updated weights for policy 0, policy_version 995116 (0.0009) [2023-12-26 22:37:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 509665280. Throughput: 0: 9560.0, 1: 9918.6. Samples: 509678728. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:37:11,062][104569] Avg episode reward: [(0, '8902.503'), (1, '7597.616')] [2023-12-26 22:37:11,092][105692] Updated weights for policy 0, policy_version 995126 (0.0009) [2023-12-26 22:37:11,163][105692] Updated weights for policy 0, policy_version 995136 (0.0009) [2023-12-26 22:37:11,206][105620] Updated weights for policy 1, policy_version 995501 (0.0008) [2023-12-26 22:37:11,270][105620] Updated weights for policy 1, policy_version 995511 (0.0009) [2023-12-26 22:37:11,336][105620] Updated weights for policy 1, policy_version 995521 (0.0008) [2023-12-26 22:37:12,017][105692] Updated weights for policy 0, policy_version 995146 (0.0010) [2023-12-26 22:37:12,076][105692] Updated weights for policy 0, policy_version 995156 (0.0011) [2023-12-26 22:37:12,117][105620] Updated weights for policy 1, policy_version 995531 (0.0009) [2023-12-26 22:37:12,138][105692] Updated weights for policy 0, policy_version 995166 (0.0011) [2023-12-26 22:37:12,177][105620] Updated weights for policy 1, policy_version 995541 (0.0005) [2023-12-26 22:37:12,195][105692] Updated weights for policy 0, policy_version 995176 (0.0011) [2023-12-26 22:37:12,231][105620] Updated weights for policy 1, policy_version 995551 (0.0007) [2023-12-26 22:37:12,907][105692] Updated weights for policy 0, policy_version 995186 (0.0006) [2023-12-26 22:37:12,966][105692] Updated weights for policy 0, policy_version 995196 (0.0008) [2023-12-26 22:37:12,991][105620] Updated weights for policy 1, policy_version 995561 (0.0008) [2023-12-26 22:37:13,017][105692] Updated weights for policy 0, policy_version 995206 (0.0010) [2023-12-26 22:37:13,045][105620] Updated weights for policy 1, policy_version 995571 (0.0005) [2023-12-26 22:37:13,098][105620] Updated weights for policy 1, policy_version 995581 (0.0005) [2023-12-26 22:37:13,162][105620] Updated weights for policy 1, policy_version 995591 (0.0006) [2023-12-26 22:37:13,643][105692] Updated weights for policy 0, policy_version 995216 (0.0008) [2023-12-26 22:37:13,702][105692] Updated weights for policy 0, policy_version 995226 (0.0007) [2023-12-26 22:37:13,708][105620] Updated weights for policy 1, policy_version 995601 (0.0007) [2023-12-26 22:37:13,755][105620] Updated weights for policy 1, policy_version 995611 (0.0010) [2023-12-26 22:37:13,761][105692] Updated weights for policy 0, policy_version 995236 (0.0005) [2023-12-26 22:37:13,813][105620] Updated weights for policy 1, policy_version 995621 (0.0010) [2023-12-26 22:37:14,396][105692] Updated weights for policy 0, policy_version 995246 (0.0005) [2023-12-26 22:37:14,447][105692] Updated weights for policy 0, policy_version 995256 (0.0005) [2023-12-26 22:37:14,500][105692] Updated weights for policy 0, policy_version 995266 (0.0005) [2023-12-26 22:37:14,548][105620] Updated weights for policy 1, policy_version 995631 (0.0010) [2023-12-26 22:37:14,612][105620] Updated weights for policy 1, policy_version 995641 (0.0010) [2023-12-26 22:37:14,663][105620] Updated weights for policy 1, policy_version 995651 (0.0010) [2023-12-26 22:37:15,178][105692] Updated weights for policy 0, policy_version 995276 (0.0007) [2023-12-26 22:37:15,251][105692] Updated weights for policy 0, policy_version 995286 (0.0011) [2023-12-26 22:37:15,314][105692] Updated weights for policy 0, policy_version 995296 (0.0011) [2023-12-26 22:37:15,370][105620] Updated weights for policy 1, policy_version 995661 (0.0008) [2023-12-26 22:37:15,432][105620] Updated weights for policy 1, policy_version 995671 (0.0005) [2023-12-26 22:37:15,482][105620] Updated weights for policy 1, policy_version 995681 (0.0005) [2023-12-26 22:37:16,030][105620] Updated weights for policy 1, policy_version 995691 (0.0007) [2023-12-26 22:37:16,056][105692] Updated weights for policy 0, policy_version 995306 (0.0011) [2023-12-26 22:37:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 509763584. Throughput: 0: 9527.1, 1: 9927.0. Samples: 509736944. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:37:16,062][104569] Avg episode reward: [(0, '8992.329'), (1, '6784.968')] [2023-12-26 22:37:16,090][105620] Updated weights for policy 1, policy_version 995701 (0.0010) [2023-12-26 22:37:16,119][105692] Updated weights for policy 0, policy_version 995316 (0.0011) [2023-12-26 22:37:16,149][105620] Updated weights for policy 1, policy_version 995711 (0.0011) [2023-12-26 22:37:16,180][105692] Updated weights for policy 0, policy_version 995326 (0.0005) [2023-12-26 22:37:16,205][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000995720_254935040.pth... [2023-12-26 22:37:16,210][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000994536_254631936.pth [2023-12-26 22:37:16,237][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000995336_254844928.pth... [2023-12-26 22:37:16,238][105692] Updated weights for policy 0, policy_version 995336 (0.0007) [2023-12-26 22:37:16,240][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000994184_254550016.pth [2023-12-26 22:37:16,783][105692] Updated weights for policy 0, policy_version 995346 (0.0009) [2023-12-26 22:37:16,842][105692] Updated weights for policy 0, policy_version 995356 (0.0010) [2023-12-26 22:37:16,878][105620] Updated weights for policy 1, policy_version 995721 (0.0010) [2023-12-26 22:37:16,900][105692] Updated weights for policy 0, policy_version 995366 (0.0010) [2023-12-26 22:37:16,936][105620] Updated weights for policy 1, policy_version 995731 (0.0010) [2023-12-26 22:37:16,993][105620] Updated weights for policy 1, policy_version 995741 (0.0010) [2023-12-26 22:37:17,050][105620] Updated weights for policy 1, policy_version 995751 (0.0010) [2023-12-26 22:37:17,644][105692] Updated weights for policy 0, policy_version 995376 (0.0008) [2023-12-26 22:37:17,709][105692] Updated weights for policy 0, policy_version 995386 (0.0006) [2023-12-26 22:37:17,733][105620] Updated weights for policy 1, policy_version 995761 (0.0006) [2023-12-26 22:37:17,767][105692] Updated weights for policy 0, policy_version 995396 (0.0007) [2023-12-26 22:37:17,817][105620] Updated weights for policy 1, policy_version 995771 (0.0007) [2023-12-26 22:37:17,868][105620] Updated weights for policy 1, policy_version 995781 (0.0010) [2023-12-26 22:37:18,397][105692] Updated weights for policy 0, policy_version 995406 (0.0009) [2023-12-26 22:37:18,452][105692] Updated weights for policy 0, policy_version 995416 (0.0009) [2023-12-26 22:37:18,508][105692] Updated weights for policy 0, policy_version 995426 (0.0008) [2023-12-26 22:37:18,527][105620] Updated weights for policy 1, policy_version 995791 (0.0008) [2023-12-26 22:37:18,584][105620] Updated weights for policy 1, policy_version 995801 (0.0009) [2023-12-26 22:37:18,646][105620] Updated weights for policy 1, policy_version 995811 (0.0009) [2023-12-26 22:37:19,210][105692] Updated weights for policy 0, policy_version 995436 (0.0006) [2023-12-26 22:37:19,275][105692] Updated weights for policy 0, policy_version 995446 (0.0009) [2023-12-26 22:37:19,298][105620] Updated weights for policy 1, policy_version 995821 (0.0007) [2023-12-26 22:37:19,332][105692] Updated weights for policy 0, policy_version 995456 (0.0008) [2023-12-26 22:37:19,360][105620] Updated weights for policy 1, policy_version 995831 (0.0007) [2023-12-26 22:37:19,424][105620] Updated weights for policy 1, policy_version 995841 (0.0011) [2023-12-26 22:37:20,060][105692] Updated weights for policy 0, policy_version 995466 (0.0008) [2023-12-26 22:37:20,122][105692] Updated weights for policy 0, policy_version 995476 (0.0006) [2023-12-26 22:37:20,171][105620] Updated weights for policy 1, policy_version 995851 (0.0009) [2023-12-26 22:37:20,188][105692] Updated weights for policy 0, policy_version 995486 (0.0007) [2023-12-26 22:37:20,235][105620] Updated weights for policy 1, policy_version 995861 (0.0008) [2023-12-26 22:37:20,253][105692] Updated weights for policy 0, policy_version 995496 (0.0006) [2023-12-26 22:37:20,291][105620] Updated weights for policy 1, policy_version 995871 (0.0008) [2023-12-26 22:37:21,005][105692] Updated weights for policy 0, policy_version 995506 (0.0006) [2023-12-26 22:37:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 509861888. Throughput: 0: 9583.5, 1: 9961.6. Samples: 509858324. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:37:21,062][105620] Updated weights for policy 1, policy_version 995881 (0.0009) [2023-12-26 22:37:21,062][104569] Avg episode reward: [(0, '9081.961'), (1, '6595.791')] [2023-12-26 22:37:21,078][105692] Updated weights for policy 0, policy_version 995516 (0.0010) [2023-12-26 22:37:21,124][105620] Updated weights for policy 1, policy_version 995891 (0.0011) [2023-12-26 22:37:21,141][105692] Updated weights for policy 0, policy_version 995526 (0.0009) [2023-12-26 22:37:21,188][105620] Updated weights for policy 1, policy_version 995901 (0.0011) [2023-12-26 22:37:21,249][105620] Updated weights for policy 1, policy_version 995911 (0.0011) [2023-12-26 22:37:21,837][105692] Updated weights for policy 0, policy_version 995536 (0.0010) [2023-12-26 22:37:21,899][105692] Updated weights for policy 0, policy_version 995546 (0.0009) [2023-12-26 22:37:21,968][105692] Updated weights for policy 0, policy_version 995556 (0.0006) [2023-12-26 22:37:22,074][105620] Updated weights for policy 1, policy_version 995921 (0.0010) [2023-12-26 22:37:22,134][105620] Updated weights for policy 1, policy_version 995932 (0.0010) [2023-12-26 22:37:22,186][105620] Updated weights for policy 1, policy_version 995942 (0.0009) [2023-12-26 22:37:22,570][105692] Updated weights for policy 0, policy_version 995566 (0.0007) [2023-12-26 22:37:22,634][105692] Updated weights for policy 0, policy_version 995576 (0.0009) [2023-12-26 22:37:22,701][105692] Updated weights for policy 0, policy_version 995586 (0.0011) [2023-12-26 22:37:23,005][105620] Updated weights for policy 1, policy_version 995952 (0.0009) [2023-12-26 22:37:23,068][105620] Updated weights for policy 1, policy_version 995962 (0.0009) [2023-12-26 22:37:23,130][105620] Updated weights for policy 1, policy_version 995972 (0.0008) [2023-12-26 22:37:23,363][105692] Updated weights for policy 0, policy_version 995596 (0.0011) [2023-12-26 22:37:23,410][105692] Updated weights for policy 0, policy_version 995606 (0.0010) [2023-12-26 22:37:23,458][105692] Updated weights for policy 0, policy_version 995616 (0.0010) [2023-12-26 22:37:23,754][105620] Updated weights for policy 1, policy_version 995982 (0.0009) [2023-12-26 22:37:23,819][105620] Updated weights for policy 1, policy_version 995992 (0.0009) [2023-12-26 22:37:23,871][105620] Updated weights for policy 1, policy_version 996002 (0.0009) [2023-12-26 22:37:24,043][105692] Updated weights for policy 0, policy_version 995626 (0.0009) [2023-12-26 22:37:24,109][105692] Updated weights for policy 0, policy_version 995636 (0.0006) [2023-12-26 22:37:24,173][105692] Updated weights for policy 0, policy_version 995646 (0.0008) [2023-12-26 22:37:24,239][105692] Updated weights for policy 0, policy_version 995656 (0.0011) [2023-12-26 22:37:24,634][105620] Updated weights for policy 1, policy_version 996012 (0.0009) [2023-12-26 22:37:24,690][105620] Updated weights for policy 1, policy_version 996022 (0.0010) [2023-12-26 22:37:24,749][105620] Updated weights for policy 1, policy_version 996032 (0.0011) [2023-12-26 22:37:24,926][105692] Updated weights for policy 0, policy_version 995666 (0.0005) [2023-12-26 22:37:24,990][105692] Updated weights for policy 0, policy_version 995676 (0.0009) [2023-12-26 22:37:25,044][105692] Updated weights for policy 0, policy_version 995686 (0.0008) [2023-12-26 22:37:25,405][105620] Updated weights for policy 1, policy_version 996042 (0.0011) [2023-12-26 22:37:25,454][105620] Updated weights for policy 1, policy_version 996052 (0.0010) [2023-12-26 22:37:25,498][105620] Updated weights for policy 1, policy_version 996062 (0.0010) [2023-12-26 22:37:25,546][105620] Updated weights for policy 1, policy_version 996072 (0.0010) [2023-12-26 22:37:25,770][105692] Updated weights for policy 0, policy_version 995696 (0.0008) [2023-12-26 22:37:25,822][105692] Updated weights for policy 0, policy_version 995706 (0.0008) [2023-12-26 22:37:25,877][105692] Updated weights for policy 0, policy_version 995716 (0.0008) [2023-12-26 22:37:26,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 509968384. Throughput: 0: 9718.7, 1: 9912.5. Samples: 509975548. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:37:26,063][104569] Avg episode reward: [(0, '9172.673'), (1, '7384.681')] [2023-12-26 22:37:26,294][105620] Updated weights for policy 1, policy_version 996082 (0.0009) [2023-12-26 22:37:26,365][105620] Updated weights for policy 1, policy_version 996092 (0.0005) [2023-12-26 22:37:26,421][105620] Updated weights for policy 1, policy_version 996102 (0.0005) [2023-12-26 22:37:26,523][105692] Updated weights for policy 0, policy_version 995726 (0.0007) [2023-12-26 22:37:26,590][105692] Updated weights for policy 0, policy_version 995736 (0.0010) [2023-12-26 22:37:26,657][105692] Updated weights for policy 0, policy_version 995746 (0.0010) [2023-12-26 22:37:26,981][105620] Updated weights for policy 1, policy_version 996112 (0.0009) [2023-12-26 22:37:27,043][105620] Updated weights for policy 1, policy_version 996122 (0.0010) [2023-12-26 22:37:27,103][105620] Updated weights for policy 1, policy_version 996132 (0.0011) [2023-12-26 22:37:27,438][105692] Updated weights for policy 0, policy_version 995756 (0.0010) [2023-12-26 22:37:27,496][105692] Updated weights for policy 0, policy_version 995767 (0.0010) [2023-12-26 22:37:27,546][105692] Updated weights for policy 0, policy_version 995777 (0.0009) [2023-12-26 22:37:27,740][105620] Updated weights for policy 1, policy_version 996142 (0.0010) [2023-12-26 22:37:27,791][105620] Updated weights for policy 1, policy_version 996152 (0.0010) [2023-12-26 22:37:27,856][105620] Updated weights for policy 1, policy_version 996162 (0.0010) [2023-12-26 22:37:28,162][105692] Updated weights for policy 0, policy_version 995787 (0.0008) [2023-12-26 22:37:28,214][105692] Updated weights for policy 0, policy_version 995797 (0.0010) [2023-12-26 22:37:28,267][105692] Updated weights for policy 0, policy_version 995807 (0.0010) [2023-12-26 22:37:28,507][105620] Updated weights for policy 1, policy_version 996172 (0.0008) [2023-12-26 22:37:28,562][105620] Updated weights for policy 1, policy_version 996182 (0.0011) [2023-12-26 22:37:28,617][105620] Updated weights for policy 1, policy_version 996192 (0.0010) [2023-12-26 22:37:29,058][105692] Updated weights for policy 0, policy_version 995817 (0.0009) [2023-12-26 22:37:29,117][105692] Updated weights for policy 0, policy_version 995827 (0.0008) [2023-12-26 22:37:29,168][105692] Updated weights for policy 0, policy_version 995837 (0.0008) [2023-12-26 22:37:29,220][105692] Updated weights for policy 0, policy_version 995847 (0.0008) [2023-12-26 22:37:29,353][105620] Updated weights for policy 1, policy_version 996202 (0.0010) [2023-12-26 22:37:29,421][105620] Updated weights for policy 1, policy_version 996212 (0.0009) [2023-12-26 22:37:29,483][105620] Updated weights for policy 1, policy_version 996222 (0.0010) [2023-12-26 22:37:29,545][105620] Updated weights for policy 1, policy_version 996232 (0.0011) [2023-12-26 22:37:29,999][105692] Updated weights for policy 0, policy_version 995857 (0.0008) [2023-12-26 22:37:30,054][105692] Updated weights for policy 0, policy_version 995867 (0.0009) [2023-12-26 22:37:30,106][105692] Updated weights for policy 0, policy_version 995877 (0.0007) [2023-12-26 22:37:30,219][105620] Updated weights for policy 1, policy_version 996242 (0.0006) [2023-12-26 22:37:30,265][105620] Updated weights for policy 1, policy_version 996252 (0.0005) [2023-12-26 22:37:30,324][105620] Updated weights for policy 1, policy_version 996262 (0.0005) [2023-12-26 22:37:30,752][105692] Updated weights for policy 0, policy_version 995887 (0.0007) [2023-12-26 22:37:30,803][105692] Updated weights for policy 0, policy_version 995897 (0.0008) [2023-12-26 22:37:30,849][105692] Updated weights for policy 0, policy_version 995907 (0.0007) [2023-12-26 22:37:30,914][105620] Updated weights for policy 1, policy_version 996272 (0.0010) [2023-12-26 22:37:30,992][105620] Updated weights for policy 1, policy_version 996282 (0.0010) [2023-12-26 22:37:31,057][105620] Updated weights for policy 1, policy_version 996292 (0.0010) [2023-12-26 22:37:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 510066688. Throughput: 0: 9741.1, 1: 9974.5. Samples: 510037152. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:37:31,062][104569] Avg episode reward: [(0, '9263.310'), (1, '8489.461')] [2023-12-26 22:37:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000995912_254992384.pth... [2023-12-26 22:37:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000994760_254697472.pth [2023-12-26 22:37:31,082][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000996296_255082496.pth... [2023-12-26 22:37:31,092][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000995112_254779392.pth [2023-12-26 22:37:31,457][105692] Updated weights for policy 0, policy_version 995917 (0.0007) [2023-12-26 22:37:31,510][105692] Updated weights for policy 0, policy_version 995927 (0.0008) [2023-12-26 22:37:31,573][105692] Updated weights for policy 0, policy_version 995937 (0.0009) [2023-12-26 22:37:31,781][105620] Updated weights for policy 1, policy_version 996302 (0.0010) [2023-12-26 22:37:31,848][105620] Updated weights for policy 1, policy_version 996312 (0.0009) [2023-12-26 22:37:31,915][105620] Updated weights for policy 1, policy_version 996322 (0.0010) [2023-12-26 22:37:32,271][105692] Updated weights for policy 0, policy_version 995947 (0.0009) [2023-12-26 22:37:32,321][105692] Updated weights for policy 0, policy_version 995957 (0.0008) [2023-12-26 22:37:32,383][105692] Updated weights for policy 0, policy_version 995967 (0.0008) [2023-12-26 22:37:32,722][105620] Updated weights for policy 1, policy_version 996332 (0.0009) [2023-12-26 22:37:32,780][105620] Updated weights for policy 1, policy_version 996342 (0.0010) [2023-12-26 22:37:32,833][105620] Updated weights for policy 1, policy_version 996352 (0.0010) [2023-12-26 22:37:33,041][105692] Updated weights for policy 0, policy_version 995977 (0.0008) [2023-12-26 22:37:33,095][105692] Updated weights for policy 0, policy_version 995987 (0.0009) [2023-12-26 22:37:33,149][105692] Updated weights for policy 0, policy_version 995997 (0.0011) [2023-12-26 22:37:33,202][105692] Updated weights for policy 0, policy_version 996008 (0.0008) [2023-12-26 22:37:33,451][105620] Updated weights for policy 1, policy_version 996362 (0.0009) [2023-12-26 22:37:33,515][105620] Updated weights for policy 1, policy_version 996372 (0.0005) [2023-12-26 22:37:33,560][105620] Updated weights for policy 1, policy_version 996382 (0.0005) [2023-12-26 22:37:33,613][105620] Updated weights for policy 1, policy_version 996392 (0.0005) [2023-12-26 22:37:33,891][105692] Updated weights for policy 0, policy_version 996018 (0.0009) [2023-12-26 22:37:33,934][105692] Updated weights for policy 0, policy_version 996028 (0.0007) [2023-12-26 22:37:33,986][105692] Updated weights for policy 0, policy_version 996038 (0.0006) [2023-12-26 22:37:34,150][105620] Updated weights for policy 1, policy_version 996402 (0.0010) [2023-12-26 22:37:34,213][105620] Updated weights for policy 1, policy_version 996412 (0.0011) [2023-12-26 22:37:34,272][105620] Updated weights for policy 1, policy_version 996422 (0.0010) [2023-12-26 22:37:34,752][105692] Updated weights for policy 0, policy_version 996048 (0.0010) [2023-12-26 22:37:34,821][105692] Updated weights for policy 0, policy_version 996058 (0.0011) [2023-12-26 22:37:34,888][105692] Updated weights for policy 0, policy_version 996068 (0.0011) [2023-12-26 22:37:34,904][105620] Updated weights for policy 1, policy_version 996432 (0.0010) [2023-12-26 22:37:34,949][105620] Updated weights for policy 1, policy_version 996442 (0.0008) [2023-12-26 22:37:34,994][105620] Updated weights for policy 1, policy_version 996452 (0.0005) [2023-12-26 22:37:35,553][105620] Updated weights for policy 1, policy_version 996462 (0.0005) [2023-12-26 22:37:35,602][105620] Updated weights for policy 1, policy_version 996472 (0.0005) [2023-12-26 22:37:35,620][105692] Updated weights for policy 0, policy_version 996078 (0.0010) [2023-12-26 22:37:35,649][105620] Updated weights for policy 1, policy_version 996482 (0.0005) [2023-12-26 22:37:35,669][105692] Updated weights for policy 0, policy_version 996088 (0.0010) [2023-12-26 22:37:35,717][105692] Updated weights for policy 0, policy_version 996098 (0.0010) [2023-12-26 22:37:36,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 510173184. Throughput: 0: 9780.2, 1: 10027.2. Samples: 510159984. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:37:36,063][104569] Avg episode reward: [(0, '9263.539'), (1, '8833.851')] [2023-12-26 22:37:36,207][105620] Updated weights for policy 1, policy_version 996492 (0.0007) [2023-12-26 22:37:36,268][105620] Updated weights for policy 1, policy_version 996502 (0.0006) [2023-12-26 22:37:36,329][105620] Updated weights for policy 1, policy_version 996512 (0.0008) [2023-12-26 22:37:36,417][105692] Updated weights for policy 0, policy_version 996108 (0.0010) [2023-12-26 22:37:36,483][105692] Updated weights for policy 0, policy_version 996118 (0.0008) [2023-12-26 22:37:36,554][105692] Updated weights for policy 0, policy_version 996128 (0.0009) [2023-12-26 22:37:36,988][105620] Updated weights for policy 1, policy_version 996522 (0.0010) [2023-12-26 22:37:37,058][105620] Updated weights for policy 1, policy_version 996532 (0.0006) [2023-12-26 22:37:37,113][105620] Updated weights for policy 1, policy_version 996542 (0.0006) [2023-12-26 22:37:37,118][105692] Updated weights for policy 0, policy_version 996138 (0.0010) [2023-12-26 22:37:37,173][105692] Updated weights for policy 0, policy_version 996148 (0.0005) [2023-12-26 22:37:37,175][105620] Updated weights for policy 1, policy_version 996552 (0.0007) [2023-12-26 22:37:37,233][105692] Updated weights for policy 0, policy_version 996158 (0.0007) [2023-12-26 22:37:37,288][105692] Updated weights for policy 0, policy_version 996168 (0.0010) [2023-12-26 22:37:37,811][105620] Updated weights for policy 1, policy_version 996562 (0.0008) [2023-12-26 22:37:37,872][105620] Updated weights for policy 1, policy_version 996572 (0.0006) [2023-12-26 22:37:37,882][105692] Updated weights for policy 0, policy_version 996178 (0.0010) [2023-12-26 22:37:37,922][105620] Updated weights for policy 1, policy_version 996582 (0.0006) [2023-12-26 22:37:37,937][105692] Updated weights for policy 0, policy_version 996188 (0.0010) [2023-12-26 22:37:38,021][105692] Updated weights for policy 0, policy_version 996198 (0.0009) [2023-12-26 22:37:38,714][105620] Updated weights for policy 1, policy_version 996592 (0.0008) [2023-12-26 22:37:38,744][105692] Updated weights for policy 0, policy_version 996208 (0.0006) [2023-12-26 22:37:38,766][105620] Updated weights for policy 1, policy_version 996602 (0.0007) [2023-12-26 22:37:38,809][105692] Updated weights for policy 0, policy_version 996218 (0.0006) [2023-12-26 22:37:38,823][105620] Updated weights for policy 1, policy_version 996612 (0.0007) [2023-12-26 22:37:38,859][105692] Updated weights for policy 0, policy_version 996228 (0.0007) [2023-12-26 22:37:39,564][105620] Updated weights for policy 1, policy_version 996622 (0.0008) [2023-12-26 22:37:39,618][105620] Updated weights for policy 1, policy_version 996632 (0.0008) [2023-12-26 22:37:39,653][105692] Updated weights for policy 0, policy_version 996238 (0.0007) [2023-12-26 22:37:39,683][105620] Updated weights for policy 1, policy_version 996642 (0.0007) [2023-12-26 22:37:39,706][105692] Updated weights for policy 0, policy_version 996248 (0.0008) [2023-12-26 22:37:39,760][105692] Updated weights for policy 0, policy_version 996258 (0.0009) [2023-12-26 22:37:40,340][105620] Updated weights for policy 1, policy_version 996652 (0.0006) [2023-12-26 22:37:40,407][105620] Updated weights for policy 1, policy_version 996662 (0.0005) [2023-12-26 22:37:40,466][105620] Updated weights for policy 1, policy_version 996672 (0.0008) [2023-12-26 22:37:40,611][105692] Updated weights for policy 0, policy_version 996268 (0.0009) [2023-12-26 22:37:40,674][105692] Updated weights for policy 0, policy_version 996278 (0.0009) [2023-12-26 22:37:40,738][105692] Updated weights for policy 0, policy_version 996288 (0.0010) [2023-12-26 22:37:41,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19798.3, 300 sec: 19522.0). Total num frames: 510271488. Throughput: 0: 9808.6, 1: 10122.6. Samples: 510281740. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-12-26 22:37:41,062][104569] Avg episode reward: [(0, '8908.775'), (1, '9267.146')] [2023-12-26 22:37:41,092][105620] Updated weights for policy 1, policy_version 996682 (0.0010) [2023-12-26 22:37:41,158][105620] Updated weights for policy 1, policy_version 996692 (0.0009) [2023-12-26 22:37:41,221][105620] Updated weights for policy 1, policy_version 996702 (0.0009) [2023-12-26 22:37:41,280][105620] Updated weights for policy 1, policy_version 996712 (0.0009) [2023-12-26 22:37:41,524][105692] Updated weights for policy 0, policy_version 996298 (0.0010) [2023-12-26 22:37:41,586][105692] Updated weights for policy 0, policy_version 996308 (0.0008) [2023-12-26 22:37:41,655][105692] Updated weights for policy 0, policy_version 996318 (0.0009) [2023-12-26 22:37:41,720][105692] Updated weights for policy 0, policy_version 996328 (0.0006) [2023-12-26 22:37:42,123][105620] Updated weights for policy 1, policy_version 996722 (0.0010) [2023-12-26 22:37:42,180][105620] Updated weights for policy 1, policy_version 996732 (0.0010) [2023-12-26 22:37:42,249][105620] Updated weights for policy 1, policy_version 996742 (0.0011) [2023-12-26 22:37:42,477][105692] Updated weights for policy 0, policy_version 996338 (0.0006) [2023-12-26 22:37:42,544][105692] Updated weights for policy 0, policy_version 996348 (0.0006) [2023-12-26 22:37:42,607][105692] Updated weights for policy 0, policy_version 996358 (0.0006) [2023-12-26 22:37:43,044][105620] Updated weights for policy 1, policy_version 996752 (0.0010) [2023-12-26 22:37:43,109][105620] Updated weights for policy 1, policy_version 996762 (0.0007) [2023-12-26 22:37:43,174][105620] Updated weights for policy 1, policy_version 996772 (0.0006) [2023-12-26 22:37:43,287][105692] Updated weights for policy 0, policy_version 996368 (0.0010) [2023-12-26 22:37:43,346][105692] Updated weights for policy 0, policy_version 996379 (0.0010) [2023-12-26 22:37:43,402][105692] Updated weights for policy 0, policy_version 996389 (0.0012) [2023-12-26 22:37:43,721][105620] Updated weights for policy 1, policy_version 996782 (0.0007) [2023-12-26 22:37:43,784][105620] Updated weights for policy 1, policy_version 996792 (0.0007) [2023-12-26 22:37:43,841][105620] Updated weights for policy 1, policy_version 996802 (0.0009) [2023-12-26 22:37:44,257][105692] Updated weights for policy 0, policy_version 996399 (0.0009) [2023-12-26 22:37:44,318][105692] Updated weights for policy 0, policy_version 996409 (0.0010) [2023-12-26 22:37:44,378][105692] Updated weights for policy 0, policy_version 996419 (0.0008) [2023-12-26 22:37:44,458][105620] Updated weights for policy 1, policy_version 996812 (0.0009) [2023-12-26 22:37:44,515][105620] Updated weights for policy 1, policy_version 996822 (0.0009) [2023-12-26 22:37:44,562][105620] Updated weights for policy 1, policy_version 996832 (0.0008) [2023-12-26 22:37:45,184][105692] Updated weights for policy 0, policy_version 996429 (0.0008) [2023-12-26 22:37:45,241][105692] Updated weights for policy 0, policy_version 996439 (0.0008) [2023-12-26 22:37:45,303][105692] Updated weights for policy 0, policy_version 996449 (0.0008) [2023-12-26 22:37:45,332][105620] Updated weights for policy 1, policy_version 996842 (0.0007) [2023-12-26 22:37:45,389][105620] Updated weights for policy 1, policy_version 996852 (0.0010) [2023-12-26 22:37:45,437][105620] Updated weights for policy 1, policy_version 996862 (0.0010) [2023-12-26 22:37:45,487][105620] Updated weights for policy 1, policy_version 996872 (0.0009) [2023-12-26 22:37:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.9, 300 sec: 19494.2). Total num frames: 510361600. Throughput: 0: 9742.1, 1: 10068.2. Samples: 510336840. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:37:46,062][104569] Avg episode reward: [(0, '8457.307'), (1, '9264.772')] [2023-12-26 22:37:46,085][105692] Updated weights for policy 0, policy_version 996459 (0.0007) [2023-12-26 22:37:46,107][105620] Updated weights for policy 1, policy_version 996882 (0.0008) [2023-12-26 22:37:46,137][105692] Updated weights for policy 0, policy_version 996469 (0.0006) [2023-12-26 22:37:46,165][105620] Updated weights for policy 1, policy_version 996892 (0.0006) [2023-12-26 22:37:46,187][105692] Updated weights for policy 0, policy_version 996479 (0.0007) [2023-12-26 22:37:46,224][105620] Updated weights for policy 1, policy_version 996902 (0.0007) [2023-12-26 22:37:46,227][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000996488_255139840.pth... [2023-12-26 22:37:46,230][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000995336_254844928.pth [2023-12-26 22:37:46,235][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000996904_255238144.pth... [2023-12-26 22:37:46,238][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000995720_254935040.pth [2023-12-26 22:37:46,754][105692] Updated weights for policy 0, policy_version 996489 (0.0007) [2023-12-26 22:37:46,796][105620] Updated weights for policy 1, policy_version 996912 (0.0007) [2023-12-26 22:37:46,803][105692] Updated weights for policy 0, policy_version 996499 (0.0005) [2023-12-26 22:37:46,846][105620] Updated weights for policy 1, policy_version 996923 (0.0009) [2023-12-26 22:37:46,865][105692] Updated weights for policy 0, policy_version 996509 (0.0005) [2023-12-26 22:37:46,895][105620] Updated weights for policy 1, policy_version 996933 (0.0009) [2023-12-26 22:37:46,926][105692] Updated weights for policy 0, policy_version 996519 (0.0005) [2023-12-26 22:37:47,537][105692] Updated weights for policy 0, policy_version 996529 (0.0005) [2023-12-26 22:37:47,594][105692] Updated weights for policy 0, policy_version 996539 (0.0007) [2023-12-26 22:37:47,644][105692] Updated weights for policy 0, policy_version 996549 (0.0008) [2023-12-26 22:37:47,721][105620] Updated weights for policy 1, policy_version 996943 (0.0009) [2023-12-26 22:37:47,771][105620] Updated weights for policy 1, policy_version 996953 (0.0009) [2023-12-26 22:37:47,825][105620] Updated weights for policy 1, policy_version 996963 (0.0009) [2023-12-26 22:37:48,284][105692] Updated weights for policy 0, policy_version 996559 (0.0009) [2023-12-26 22:37:48,341][105692] Updated weights for policy 0, policy_version 996569 (0.0008) [2023-12-26 22:37:48,398][105692] Updated weights for policy 0, policy_version 996579 (0.0009) [2023-12-26 22:37:48,550][105620] Updated weights for policy 1, policy_version 996973 (0.0009) [2023-12-26 22:37:48,612][105620] Updated weights for policy 1, policy_version 996983 (0.0010) [2023-12-26 22:37:48,664][105620] Updated weights for policy 1, policy_version 996993 (0.0010) [2023-12-26 22:37:49,192][105692] Updated weights for policy 0, policy_version 996589 (0.0009) [2023-12-26 22:37:49,260][105692] Updated weights for policy 0, policy_version 996599 (0.0008) [2023-12-26 22:37:49,319][105692] Updated weights for policy 0, policy_version 996609 (0.0008) [2023-12-26 22:37:49,432][105620] Updated weights for policy 1, policy_version 997003 (0.0010) [2023-12-26 22:37:49,493][105620] Updated weights for policy 1, policy_version 997013 (0.0009) [2023-12-26 22:37:49,556][105620] Updated weights for policy 1, policy_version 997023 (0.0007) [2023-12-26 22:37:50,074][105692] Updated weights for policy 0, policy_version 996619 (0.0009) [2023-12-26 22:37:50,131][105692] Updated weights for policy 0, policy_version 996629 (0.0006) [2023-12-26 22:37:50,188][105692] Updated weights for policy 0, policy_version 996639 (0.0006) [2023-12-26 22:37:50,283][105620] Updated weights for policy 1, policy_version 997033 (0.0010) [2023-12-26 22:37:50,328][105620] Updated weights for policy 1, policy_version 997043 (0.0010) [2023-12-26 22:37:50,374][105620] Updated weights for policy 1, policy_version 997053 (0.0007) [2023-12-26 22:37:50,431][105620] Updated weights for policy 1, policy_version 997063 (0.0006) [2023-12-26 22:37:50,857][105692] Updated weights for policy 0, policy_version 996649 (0.0006) [2023-12-26 22:37:50,918][105692] Updated weights for policy 0, policy_version 996659 (0.0008) [2023-12-26 22:37:50,978][105692] Updated weights for policy 0, policy_version 996669 (0.0008) [2023-12-26 22:37:51,042][105692] Updated weights for policy 0, policy_version 996679 (0.0008) [2023-12-26 22:37:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19521.9). Total num frames: 510468096. Throughput: 0: 9753.7, 1: 10002.9. Samples: 510455584. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:37:51,063][104569] Avg episode reward: [(0, '8727.775'), (1, '9264.698')] [2023-12-26 22:37:51,208][105620] Updated weights for policy 1, policy_version 997073 (0.0010) [2023-12-26 22:37:51,274][105620] Updated weights for policy 1, policy_version 997083 (0.0011) [2023-12-26 22:37:51,352][105620] Updated weights for policy 1, policy_version 997093 (0.0010) [2023-12-26 22:37:51,863][105692] Updated weights for policy 0, policy_version 996689 (0.0008) [2023-12-26 22:37:51,918][105692] Updated weights for policy 0, policy_version 996699 (0.0006) [2023-12-26 22:37:51,982][105692] Updated weights for policy 0, policy_version 996709 (0.0008) [2023-12-26 22:37:52,069][105620] Updated weights for policy 1, policy_version 997103 (0.0006) [2023-12-26 22:37:52,129][105620] Updated weights for policy 1, policy_version 997113 (0.0006) [2023-12-26 22:37:52,189][105620] Updated weights for policy 1, policy_version 997123 (0.0007) [2023-12-26 22:37:52,696][105692] Updated weights for policy 0, policy_version 996719 (0.0009) [2023-12-26 22:37:52,746][105692] Updated weights for policy 0, policy_version 996729 (0.0009) [2023-12-26 22:37:52,797][105692] Updated weights for policy 0, policy_version 996739 (0.0009) [2023-12-26 22:37:52,876][105620] Updated weights for policy 1, policy_version 997133 (0.0010) [2023-12-26 22:37:52,935][105620] Updated weights for policy 1, policy_version 997143 (0.0009) [2023-12-26 22:37:53,001][105620] Updated weights for policy 1, policy_version 997153 (0.0009) [2023-12-26 22:37:53,515][105692] Updated weights for policy 0, policy_version 996749 (0.0006) [2023-12-26 22:37:53,564][105692] Updated weights for policy 0, policy_version 996759 (0.0005) [2023-12-26 22:37:53,618][105692] Updated weights for policy 0, policy_version 996769 (0.0005) [2023-12-26 22:37:53,674][105620] Updated weights for policy 1, policy_version 997163 (0.0008) [2023-12-26 22:37:53,729][105620] Updated weights for policy 1, policy_version 997173 (0.0006) [2023-12-26 22:37:53,783][105620] Updated weights for policy 1, policy_version 997183 (0.0005) [2023-12-26 22:37:54,196][105692] Updated weights for policy 0, policy_version 996779 (0.0007) [2023-12-26 22:37:54,251][105692] Updated weights for policy 0, policy_version 996789 (0.0009) [2023-12-26 22:37:54,306][105692] Updated weights for policy 0, policy_version 996799 (0.0009) [2023-12-26 22:37:54,495][105620] Updated weights for policy 1, policy_version 997193 (0.0006) [2023-12-26 22:37:54,553][105620] Updated weights for policy 1, policy_version 997203 (0.0008) [2023-12-26 22:37:54,615][105620] Updated weights for policy 1, policy_version 997213 (0.0008) [2023-12-26 22:37:54,680][105620] Updated weights for policy 1, policy_version 997223 (0.0008) [2023-12-26 22:37:55,015][105692] Updated weights for policy 0, policy_version 996809 (0.0009) [2023-12-26 22:37:55,080][105692] Updated weights for policy 0, policy_version 996819 (0.0006) [2023-12-26 22:37:55,142][105692] Updated weights for policy 0, policy_version 996829 (0.0005) [2023-12-26 22:37:55,202][105692] Updated weights for policy 0, policy_version 996839 (0.0007) [2023-12-26 22:37:55,403][105620] Updated weights for policy 1, policy_version 997233 (0.0006) [2023-12-26 22:37:55,461][105620] Updated weights for policy 1, policy_version 997243 (0.0006) [2023-12-26 22:37:55,510][105620] Updated weights for policy 1, policy_version 997253 (0.0009) [2023-12-26 22:37:55,800][105692] Updated weights for policy 0, policy_version 996849 (0.0006) [2023-12-26 22:37:55,854][105692] Updated weights for policy 0, policy_version 996859 (0.0007) [2023-12-26 22:37:55,898][105692] Updated weights for policy 0, policy_version 996869 (0.0008) [2023-12-26 22:37:56,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.4, 300 sec: 19522.0). Total num frames: 510566400. Throughput: 0: 9888.1, 1: 10022.8. Samples: 510574716. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:37:56,063][104569] Avg episode reward: [(0, '8637.557'), (1, '9356.199')] [2023-12-26 22:37:56,112][105620] Updated weights for policy 1, policy_version 997263 (0.0005) [2023-12-26 22:37:56,170][105620] Updated weights for policy 1, policy_version 997273 (0.0007) [2023-12-26 22:37:56,233][105620] Updated weights for policy 1, policy_version 997283 (0.0011) [2023-12-26 22:37:56,635][105692] Updated weights for policy 0, policy_version 996879 (0.0006) [2023-12-26 22:37:56,696][105692] Updated weights for policy 0, policy_version 996889 (0.0005) [2023-12-26 22:37:56,763][105692] Updated weights for policy 0, policy_version 996899 (0.0005) [2023-12-26 22:37:56,807][105620] Updated weights for policy 1, policy_version 997293 (0.0008) [2023-12-26 22:37:56,873][105620] Updated weights for policy 1, policy_version 997303 (0.0006) [2023-12-26 22:37:56,930][105620] Updated weights for policy 1, policy_version 997313 (0.0005) [2023-12-26 22:37:57,272][105692] Updated weights for policy 0, policy_version 996909 (0.0006) [2023-12-26 22:37:57,335][105692] Updated weights for policy 0, policy_version 996919 (0.0006) [2023-12-26 22:37:57,395][105692] Updated weights for policy 0, policy_version 996929 (0.0005) [2023-12-26 22:37:57,446][105620] Updated weights for policy 1, policy_version 997323 (0.0007) [2023-12-26 22:37:57,514][105620] Updated weights for policy 1, policy_version 997333 (0.0010) [2023-12-26 22:37:57,571][105620] Updated weights for policy 1, policy_version 997343 (0.0010) [2023-12-26 22:37:58,009][105692] Updated weights for policy 0, policy_version 996939 (0.0006) [2023-12-26 22:37:58,070][105692] Updated weights for policy 0, policy_version 996949 (0.0009) [2023-12-26 22:37:58,123][105692] Updated weights for policy 0, policy_version 996959 (0.0010) [2023-12-26 22:37:58,237][105620] Updated weights for policy 1, policy_version 997353 (0.0010) [2023-12-26 22:37:58,301][105620] Updated weights for policy 1, policy_version 997363 (0.0009) [2023-12-26 22:37:58,366][105620] Updated weights for policy 1, policy_version 997373 (0.0010) [2023-12-26 22:37:58,432][105620] Updated weights for policy 1, policy_version 997383 (0.0008) [2023-12-26 22:37:58,960][105692] Updated weights for policy 0, policy_version 996969 (0.0009) [2023-12-26 22:37:59,021][105692] Updated weights for policy 0, policy_version 996979 (0.0005) [2023-12-26 22:37:59,084][105692] Updated weights for policy 0, policy_version 996989 (0.0008) [2023-12-26 22:37:59,145][105692] Updated weights for policy 0, policy_version 996999 (0.0008) [2023-12-26 22:37:59,231][105620] Updated weights for policy 1, policy_version 997393 (0.0007) [2023-12-26 22:37:59,300][105620] Updated weights for policy 1, policy_version 997403 (0.0007) [2023-12-26 22:37:59,373][105620] Updated weights for policy 1, policy_version 997413 (0.0008) [2023-12-26 22:37:59,899][105692] Updated weights for policy 0, policy_version 997009 (0.0006) [2023-12-26 22:37:59,959][105692] Updated weights for policy 0, policy_version 997019 (0.0007) [2023-12-26 22:38:00,014][105692] Updated weights for policy 0, policy_version 997029 (0.0006) [2023-12-26 22:38:00,117][105620] Updated weights for policy 1, policy_version 997423 (0.0008) [2023-12-26 22:38:00,171][105620] Updated weights for policy 1, policy_version 997433 (0.0009) [2023-12-26 22:38:00,217][105620] Updated weights for policy 1, policy_version 997443 (0.0008) [2023-12-26 22:38:00,698][105692] Updated weights for policy 0, policy_version 997039 (0.0008) [2023-12-26 22:38:00,752][105692] Updated weights for policy 0, policy_version 997049 (0.0008) [2023-12-26 22:38:00,796][105692] Updated weights for policy 0, policy_version 997059 (0.0010) [2023-12-26 22:38:00,891][105620] Updated weights for policy 1, policy_version 997453 (0.0007) [2023-12-26 22:38:00,944][105620] Updated weights for policy 1, policy_version 997463 (0.0005) [2023-12-26 22:38:01,004][105620] Updated weights for policy 1, policy_version 997473 (0.0005) [2023-12-26 22:38:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19933.8, 300 sec: 19521.9). Total num frames: 510672896. Throughput: 0: 9939.0, 1: 10082.5. Samples: 510637912. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:38:01,063][104569] Avg episode reward: [(0, '8632.318'), (1, '9263.796')] [2023-12-26 22:38:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000997064_255287296.pth... [2023-12-26 22:38:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000997480_255385600.pth... [2023-12-26 22:38:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000995912_254992384.pth [2023-12-26 22:38:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000996296_255082496.pth [2023-12-26 22:38:01,523][105692] Updated weights for policy 0, policy_version 997069 (0.0010) [2023-12-26 22:38:01,586][105692] Updated weights for policy 0, policy_version 997079 (0.0010) [2023-12-26 22:38:01,649][105692] Updated weights for policy 0, policy_version 997089 (0.0011) [2023-12-26 22:38:01,658][105620] Updated weights for policy 1, policy_version 997483 (0.0007) [2023-12-26 22:38:01,707][105620] Updated weights for policy 1, policy_version 997493 (0.0010) [2023-12-26 22:38:01,768][105620] Updated weights for policy 1, policy_version 997503 (0.0009) [2023-12-26 22:38:02,316][105692] Updated weights for policy 0, policy_version 997099 (0.0009) [2023-12-26 22:38:02,386][105692] Updated weights for policy 0, policy_version 997109 (0.0006) [2023-12-26 22:38:02,452][105692] Updated weights for policy 0, policy_version 997119 (0.0008) [2023-12-26 22:38:02,524][105620] Updated weights for policy 1, policy_version 997513 (0.0009) [2023-12-26 22:38:02,578][105620] Updated weights for policy 1, policy_version 997523 (0.0010) [2023-12-26 22:38:02,626][105620] Updated weights for policy 1, policy_version 997533 (0.0010) [2023-12-26 22:38:02,677][105620] Updated weights for policy 1, policy_version 997543 (0.0010) [2023-12-26 22:38:03,044][105692] Updated weights for policy 0, policy_version 997129 (0.0010) [2023-12-26 22:38:03,092][105692] Updated weights for policy 0, policy_version 997139 (0.0008) [2023-12-26 22:38:03,139][105692] Updated weights for policy 0, policy_version 997149 (0.0008) [2023-12-26 22:38:03,200][105692] Updated weights for policy 0, policy_version 997159 (0.0008) [2023-12-26 22:38:03,428][105620] Updated weights for policy 1, policy_version 997553 (0.0010) [2023-12-26 22:38:03,476][105620] Updated weights for policy 1, policy_version 997563 (0.0010) [2023-12-26 22:38:03,533][105620] Updated weights for policy 1, policy_version 997573 (0.0010) [2023-12-26 22:38:03,832][105692] Updated weights for policy 0, policy_version 997169 (0.0006) [2023-12-26 22:38:03,896][105692] Updated weights for policy 0, policy_version 997179 (0.0007) [2023-12-26 22:38:03,955][105692] Updated weights for policy 0, policy_version 997189 (0.0006) [2023-12-26 22:38:04,287][105620] Updated weights for policy 1, policy_version 997583 (0.0010) [2023-12-26 22:38:04,353][105620] Updated weights for policy 1, policy_version 997593 (0.0010) [2023-12-26 22:38:04,415][105620] Updated weights for policy 1, policy_version 997603 (0.0010) [2023-12-26 22:38:04,614][105692] Updated weights for policy 0, policy_version 997199 (0.0005) [2023-12-26 22:38:04,667][105692] Updated weights for policy 0, policy_version 997209 (0.0005) [2023-12-26 22:38:04,717][105692] Updated weights for policy 0, policy_version 997219 (0.0006) [2023-12-26 22:38:05,147][105620] Updated weights for policy 1, policy_version 997613 (0.0010) [2023-12-26 22:38:05,203][105620] Updated weights for policy 1, policy_version 997623 (0.0010) [2023-12-26 22:38:05,264][105620] Updated weights for policy 1, policy_version 997633 (0.0010) [2023-12-26 22:38:05,287][105692] Updated weights for policy 0, policy_version 997229 (0.0009) [2023-12-26 22:38:05,348][105692] Updated weights for policy 0, policy_version 997239 (0.0010) [2023-12-26 22:38:05,403][105692] Updated weights for policy 0, policy_version 997249 (0.0010) [2023-12-26 22:38:05,917][105620] Updated weights for policy 1, policy_version 997643 (0.0009) [2023-12-26 22:38:05,980][105620] Updated weights for policy 1, policy_version 997653 (0.0008) [2023-12-26 22:38:06,031][105620] Updated weights for policy 1, policy_version 997663 (0.0008) [2023-12-26 22:38:06,032][105692] Updated weights for policy 0, policy_version 997259 (0.0008) [2023-12-26 22:38:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 510763008. Throughput: 0: 9935.5, 1: 10013.2. Samples: 510756020. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:38:06,063][104569] Avg episode reward: [(0, '8635.077'), (1, '9264.769')] [2023-12-26 22:38:06,081][105692] Updated weights for policy 0, policy_version 997269 (0.0006) [2023-12-26 22:38:06,143][105692] Updated weights for policy 0, policy_version 997279 (0.0008) [2023-12-26 22:38:06,799][105692] Updated weights for policy 0, policy_version 997289 (0.0006) [2023-12-26 22:38:06,864][105692] Updated weights for policy 0, policy_version 997299 (0.0006) [2023-12-26 22:38:06,871][105620] Updated weights for policy 1, policy_version 997673 (0.0007) [2023-12-26 22:38:06,924][105692] Updated weights for policy 0, policy_version 997309 (0.0007) [2023-12-26 22:38:06,933][105620] Updated weights for policy 1, policy_version 997683 (0.0008) [2023-12-26 22:38:06,985][105692] Updated weights for policy 0, policy_version 997319 (0.0006) [2023-12-26 22:38:07,000][105620] Updated weights for policy 1, policy_version 997693 (0.0008) [2023-12-26 22:38:07,061][105620] Updated weights for policy 1, policy_version 997703 (0.0009) [2023-12-26 22:38:07,600][105692] Updated weights for policy 0, policy_version 997329 (0.0009) [2023-12-26 22:38:07,664][105692] Updated weights for policy 0, policy_version 997339 (0.0007) [2023-12-26 22:38:07,716][105692] Updated weights for policy 0, policy_version 997349 (0.0006) [2023-12-26 22:38:07,824][105620] Updated weights for policy 1, policy_version 997713 (0.0009) [2023-12-26 22:38:07,878][105620] Updated weights for policy 1, policy_version 997723 (0.0009) [2023-12-26 22:38:07,925][105620] Updated weights for policy 1, policy_version 997733 (0.0008) [2023-12-26 22:38:08,442][105692] Updated weights for policy 0, policy_version 997359 (0.0006) [2023-12-26 22:38:08,508][105692] Updated weights for policy 0, policy_version 997369 (0.0006) [2023-12-26 22:38:08,552][105692] Updated weights for policy 0, policy_version 997379 (0.0005) [2023-12-26 22:38:08,743][105620] Updated weights for policy 1, policy_version 997743 (0.0009) [2023-12-26 22:38:08,801][105620] Updated weights for policy 1, policy_version 997753 (0.0009) [2023-12-26 22:38:08,855][105620] Updated weights for policy 1, policy_version 997763 (0.0009) [2023-12-26 22:38:09,141][105692] Updated weights for policy 0, policy_version 997389 (0.0005) [2023-12-26 22:38:09,200][105692] Updated weights for policy 0, policy_version 997399 (0.0006) [2023-12-26 22:38:09,284][105692] Updated weights for policy 0, policy_version 997409 (0.0007) [2023-12-26 22:38:09,605][105620] Updated weights for policy 1, policy_version 997773 (0.0008) [2023-12-26 22:38:09,658][105620] Updated weights for policy 1, policy_version 997783 (0.0005) [2023-12-26 22:38:09,715][105620] Updated weights for policy 1, policy_version 997793 (0.0007) [2023-12-26 22:38:10,044][105692] Updated weights for policy 0, policy_version 997419 (0.0008) [2023-12-26 22:38:10,104][105692] Updated weights for policy 0, policy_version 997429 (0.0009) [2023-12-26 22:38:10,155][105692] Updated weights for policy 0, policy_version 997440 (0.0009) [2023-12-26 22:38:10,378][105620] Updated weights for policy 1, policy_version 997803 (0.0006) [2023-12-26 22:38:10,439][105620] Updated weights for policy 1, policy_version 997813 (0.0009) [2023-12-26 22:38:10,501][105620] Updated weights for policy 1, policy_version 997823 (0.0009) [2023-12-26 22:38:11,045][105692] Updated weights for policy 0, policy_version 997450 (0.0009) [2023-12-26 22:38:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19933.9, 300 sec: 19494.2). Total num frames: 510861312. Throughput: 0: 9951.1, 1: 10005.7. Samples: 510873604. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:38:11,062][104569] Avg episode reward: [(0, '8462.479'), (1, '9264.713')] [2023-12-26 22:38:11,105][105692] Updated weights for policy 0, policy_version 997460 (0.0010) [2023-12-26 22:38:11,179][105692] Updated weights for policy 0, policy_version 997470 (0.0007) [2023-12-26 22:38:11,235][105692] Updated weights for policy 0, policy_version 997480 (0.0006) [2023-12-26 22:38:11,285][105620] Updated weights for policy 1, policy_version 997833 (0.0008) [2023-12-26 22:38:11,356][105620] Updated weights for policy 1, policy_version 997843 (0.0006) [2023-12-26 22:38:11,419][105620] Updated weights for policy 1, policy_version 997853 (0.0010) [2023-12-26 22:38:11,488][105620] Updated weights for policy 1, policy_version 997863 (0.0006) [2023-12-26 22:38:11,927][105692] Updated weights for policy 0, policy_version 997490 (0.0009) [2023-12-26 22:38:11,989][105692] Updated weights for policy 0, policy_version 997500 (0.0009) [2023-12-26 22:38:12,051][105692] Updated weights for policy 0, policy_version 997510 (0.0010) [2023-12-26 22:38:12,171][105620] Updated weights for policy 1, policy_version 997873 (0.0009) [2023-12-26 22:38:12,225][105620] Updated weights for policy 1, policy_version 997883 (0.0008) [2023-12-26 22:38:12,288][105620] Updated weights for policy 1, policy_version 997893 (0.0009) [2023-12-26 22:38:12,881][105692] Updated weights for policy 0, policy_version 997520 (0.0009) [2023-12-26 22:38:12,938][105692] Updated weights for policy 0, policy_version 997530 (0.0008) [2023-12-26 22:38:12,977][105620] Updated weights for policy 1, policy_version 997903 (0.0007) [2023-12-26 22:38:12,996][105692] Updated weights for policy 0, policy_version 997540 (0.0010) [2023-12-26 22:38:13,037][105620] Updated weights for policy 1, policy_version 997913 (0.0006) [2023-12-26 22:38:13,102][105620] Updated weights for policy 1, policy_version 997923 (0.0005) [2023-12-26 22:38:13,744][105692] Updated weights for policy 0, policy_version 997550 (0.0008) [2023-12-26 22:38:13,799][105620] Updated weights for policy 1, policy_version 997933 (0.0006) [2023-12-26 22:38:13,804][105692] Updated weights for policy 0, policy_version 997560 (0.0009) [2023-12-26 22:38:13,858][105620] Updated weights for policy 1, policy_version 997943 (0.0006) [2023-12-26 22:38:13,860][105692] Updated weights for policy 0, policy_version 997570 (0.0008) [2023-12-26 22:38:13,922][105620] Updated weights for policy 1, policy_version 997953 (0.0005) [2023-12-26 22:38:14,518][105620] Updated weights for policy 1, policy_version 997963 (0.0006) [2023-12-26 22:38:14,572][105620] Updated weights for policy 1, policy_version 997973 (0.0005) [2023-12-26 22:38:14,615][105620] Updated weights for policy 1, policy_version 997983 (0.0005) [2023-12-26 22:38:14,714][105692] Updated weights for policy 0, policy_version 997580 (0.0009) [2023-12-26 22:38:14,781][105692] Updated weights for policy 0, policy_version 997590 (0.0009) [2023-12-26 22:38:14,843][105692] Updated weights for policy 0, policy_version 997600 (0.0009) [2023-12-26 22:38:15,243][105620] Updated weights for policy 1, policy_version 997993 (0.0006) [2023-12-26 22:38:15,316][105620] Updated weights for policy 1, policy_version 998003 (0.0006) [2023-12-26 22:38:15,369][105620] Updated weights for policy 1, policy_version 998013 (0.0006) [2023-12-26 22:38:15,423][105620] Updated weights for policy 1, policy_version 998023 (0.0007) [2023-12-26 22:38:15,738][105692] Updated weights for policy 0, policy_version 997610 (0.0009) [2023-12-26 22:38:15,806][105692] Updated weights for policy 0, policy_version 997620 (0.0010) [2023-12-26 22:38:15,876][105692] Updated weights for policy 0, policy_version 997630 (0.0009) [2023-12-26 22:38:15,936][105692] Updated weights for policy 0, policy_version 997640 (0.0009) [2023-12-26 22:38:15,992][105620] Updated weights for policy 1, policy_version 998033 (0.0005) [2023-12-26 22:38:16,037][105620] Updated weights for policy 1, policy_version 998043 (0.0005) [2023-12-26 22:38:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19933.8, 300 sec: 19494.2). Total num frames: 510959616. Throughput: 0: 9892.9, 1: 9948.2. Samples: 510930004. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:38:16,063][104569] Avg episode reward: [(0, '8638.696'), (1, '9173.087')] [2023-12-26 22:38:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000997640_255434752.pth... [2023-12-26 22:38:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000996488_255139840.pth [2023-12-26 22:38:16,093][105620] Updated weights for policy 1, policy_version 998053 (0.0007) [2023-12-26 22:38:16,113][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000998056_255533056.pth... [2023-12-26 22:38:16,127][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000996904_255238144.pth [2023-12-26 22:38:16,658][105692] Updated weights for policy 0, policy_version 997650 (0.0006) [2023-12-26 22:38:16,707][105692] Updated weights for policy 0, policy_version 997660 (0.0007) [2023-12-26 22:38:16,756][105692] Updated weights for policy 0, policy_version 997670 (0.0005) [2023-12-26 22:38:16,819][105620] Updated weights for policy 1, policy_version 998063 (0.0010) [2023-12-26 22:38:16,881][105620] Updated weights for policy 1, policy_version 998073 (0.0010) [2023-12-26 22:38:16,927][105620] Updated weights for policy 1, policy_version 998083 (0.0010) [2023-12-26 22:38:17,373][105692] Updated weights for policy 0, policy_version 997680 (0.0005) [2023-12-26 22:38:17,416][105692] Updated weights for policy 0, policy_version 997690 (0.0005) [2023-12-26 22:38:17,463][105692] Updated weights for policy 0, policy_version 997700 (0.0005) [2023-12-26 22:38:17,721][105620] Updated weights for policy 1, policy_version 998093 (0.0010) [2023-12-26 22:38:17,776][105620] Updated weights for policy 1, policy_version 998103 (0.0010) [2023-12-26 22:38:17,821][105620] Updated weights for policy 1, policy_version 998113 (0.0006) [2023-12-26 22:38:18,019][105692] Updated weights for policy 0, policy_version 997710 (0.0006) [2023-12-26 22:38:18,076][105692] Updated weights for policy 0, policy_version 997720 (0.0010) [2023-12-26 22:38:18,132][105692] Updated weights for policy 0, policy_version 997730 (0.0010) [2023-12-26 22:38:18,429][105620] Updated weights for policy 1, policy_version 998123 (0.0006) [2023-12-26 22:38:18,492][105620] Updated weights for policy 1, policy_version 998133 (0.0008) [2023-12-26 22:38:18,541][105620] Updated weights for policy 1, policy_version 998143 (0.0008) [2023-12-26 22:38:18,816][105692] Updated weights for policy 0, policy_version 997740 (0.0005) [2023-12-26 22:38:18,885][105692] Updated weights for policy 0, policy_version 997750 (0.0008) [2023-12-26 22:38:18,945][105692] Updated weights for policy 0, policy_version 997760 (0.0010) [2023-12-26 22:38:19,335][105620] Updated weights for policy 1, policy_version 998153 (0.0008) [2023-12-26 22:38:19,399][105620] Updated weights for policy 1, policy_version 998163 (0.0010) [2023-12-26 22:38:19,461][105620] Updated weights for policy 1, policy_version 998173 (0.0011) [2023-12-26 22:38:19,525][105620] Updated weights for policy 1, policy_version 998183 (0.0009) [2023-12-26 22:38:19,671][105692] Updated weights for policy 0, policy_version 997770 (0.0010) [2023-12-26 22:38:19,724][105692] Updated weights for policy 0, policy_version 997780 (0.0005) [2023-12-26 22:38:19,786][105692] Updated weights for policy 0, policy_version 997790 (0.0006) [2023-12-26 22:38:19,852][105692] Updated weights for policy 0, policy_version 997800 (0.0010) [2023-12-26 22:38:20,256][105620] Updated weights for policy 1, policy_version 998193 (0.0008) [2023-12-26 22:38:20,314][105620] Updated weights for policy 1, policy_version 998203 (0.0005) [2023-12-26 22:38:20,363][105620] Updated weights for policy 1, policy_version 998213 (0.0006) [2023-12-26 22:38:20,566][105692] Updated weights for policy 0, policy_version 997810 (0.0010) [2023-12-26 22:38:20,634][105692] Updated weights for policy 0, policy_version 997820 (0.0006) [2023-12-26 22:38:20,701][105692] Updated weights for policy 0, policy_version 997830 (0.0009) [2023-12-26 22:38:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19933.8, 300 sec: 19521.9). Total num frames: 511057920. Throughput: 0: 9833.2, 1: 9937.3. Samples: 511049660. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:38:21,063][104569] Avg episode reward: [(0, '8541.130'), (1, '9080.461')] [2023-12-26 22:38:21,136][105620] Updated weights for policy 1, policy_version 998223 (0.0008) [2023-12-26 22:38:21,196][105620] Updated weights for policy 1, policy_version 998233 (0.0008) [2023-12-26 22:38:21,256][105620] Updated weights for policy 1, policy_version 998243 (0.0006) [2023-12-26 22:38:21,414][105692] Updated weights for policy 0, policy_version 997840 (0.0012) [2023-12-26 22:38:21,481][105692] Updated weights for policy 0, policy_version 997850 (0.0009) [2023-12-26 22:38:21,543][105692] Updated weights for policy 0, policy_version 997860 (0.0009) [2023-12-26 22:38:22,051][105620] Updated weights for policy 1, policy_version 998253 (0.0008) [2023-12-26 22:38:22,117][105620] Updated weights for policy 1, policy_version 998263 (0.0010) [2023-12-26 22:38:22,172][105620] Updated weights for policy 1, policy_version 998273 (0.0009) [2023-12-26 22:38:22,309][105692] Updated weights for policy 0, policy_version 997870 (0.0009) [2023-12-26 22:38:22,375][105692] Updated weights for policy 0, policy_version 997880 (0.0008) [2023-12-26 22:38:22,437][105692] Updated weights for policy 0, policy_version 997890 (0.0008) [2023-12-26 22:38:22,905][105620] Updated weights for policy 1, policy_version 998283 (0.0008) [2023-12-26 22:38:22,962][105620] Updated weights for policy 1, policy_version 998293 (0.0006) [2023-12-26 22:38:23,023][105620] Updated weights for policy 1, policy_version 998303 (0.0009) [2023-12-26 22:38:23,198][105692] Updated weights for policy 0, policy_version 997900 (0.0010) [2023-12-26 22:38:23,259][105692] Updated weights for policy 0, policy_version 997910 (0.0010) [2023-12-26 22:38:23,316][105692] Updated weights for policy 0, policy_version 997920 (0.0009) [2023-12-26 22:38:23,744][105620] Updated weights for policy 1, policy_version 998313 (0.0011) [2023-12-26 22:38:23,791][105620] Updated weights for policy 1, policy_version 998323 (0.0010) [2023-12-26 22:38:23,850][105620] Updated weights for policy 1, policy_version 998333 (0.0010) [2023-12-26 22:38:23,915][105620] Updated weights for policy 1, policy_version 998343 (0.0010) [2023-12-26 22:38:23,988][105692] Updated weights for policy 0, policy_version 997930 (0.0006) [2023-12-26 22:38:24,050][105692] Updated weights for policy 0, policy_version 997940 (0.0006) [2023-12-26 22:38:24,111][105692] Updated weights for policy 0, policy_version 997950 (0.0010) [2023-12-26 22:38:24,165][105692] Updated weights for policy 0, policy_version 997960 (0.0010) [2023-12-26 22:38:24,625][105620] Updated weights for policy 1, policy_version 998353 (0.0008) [2023-12-26 22:38:24,687][105620] Updated weights for policy 1, policy_version 998363 (0.0007) [2023-12-26 22:38:24,757][105620] Updated weights for policy 1, policy_version 998373 (0.0006) [2023-12-26 22:38:24,912][105692] Updated weights for policy 0, policy_version 997970 (0.0010) [2023-12-26 22:38:24,958][105692] Updated weights for policy 0, policy_version 997980 (0.0005) [2023-12-26 22:38:25,011][105692] Updated weights for policy 0, policy_version 997990 (0.0006) [2023-12-26 22:38:25,505][105620] Updated weights for policy 1, policy_version 998383 (0.0006) [2023-12-26 22:38:25,568][105620] Updated weights for policy 1, policy_version 998393 (0.0005) [2023-12-26 22:38:25,617][105620] Updated weights for policy 1, policy_version 998403 (0.0005) [2023-12-26 22:38:25,735][105692] Updated weights for policy 0, policy_version 998000 (0.0010) [2023-12-26 22:38:25,793][105692] Updated weights for policy 0, policy_version 998010 (0.0010) [2023-12-26 22:38:25,860][105692] Updated weights for policy 0, policy_version 998020 (0.0010) [2023-12-26 22:38:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 511156224. Throughput: 0: 9804.3, 1: 9810.8. Samples: 511164424. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:38:26,063][104569] Avg episode reward: [(0, '8724.932'), (1, '8987.819')] [2023-12-26 22:38:26,131][105620] Updated weights for policy 1, policy_version 998413 (0.0007) [2023-12-26 22:38:26,186][105620] Updated weights for policy 1, policy_version 998423 (0.0009) [2023-12-26 22:38:26,240][105620] Updated weights for policy 1, policy_version 998433 (0.0009) [2023-12-26 22:38:26,580][105692] Updated weights for policy 0, policy_version 998030 (0.0009) [2023-12-26 22:38:26,636][105692] Updated weights for policy 0, policy_version 998040 (0.0009) [2023-12-26 22:38:26,706][105692] Updated weights for policy 0, policy_version 998050 (0.0010) [2023-12-26 22:38:26,908][105620] Updated weights for policy 1, policy_version 998443 (0.0008) [2023-12-26 22:38:26,956][105620] Updated weights for policy 1, policy_version 998453 (0.0006) [2023-12-26 22:38:27,004][105620] Updated weights for policy 1, policy_version 998463 (0.0006) [2023-12-26 22:38:27,493][105692] Updated weights for policy 0, policy_version 998060 (0.0009) [2023-12-26 22:38:27,548][105692] Updated weights for policy 0, policy_version 998070 (0.0009) [2023-12-26 22:38:27,595][105692] Updated weights for policy 0, policy_version 998081 (0.0009) [2023-12-26 22:38:27,673][105620] Updated weights for policy 1, policy_version 998473 (0.0010) [2023-12-26 22:38:27,736][105620] Updated weights for policy 1, policy_version 998484 (0.0011) [2023-12-26 22:38:27,788][105620] Updated weights for policy 1, policy_version 998494 (0.0009) [2023-12-26 22:38:27,838][105620] Updated weights for policy 1, policy_version 998504 (0.0010) [2023-12-26 22:38:28,234][105692] Updated weights for policy 0, policy_version 998091 (0.0009) [2023-12-26 22:38:28,291][105692] Updated weights for policy 0, policy_version 998101 (0.0009) [2023-12-26 22:38:28,341][105692] Updated weights for policy 0, policy_version 998111 (0.0009) [2023-12-26 22:38:28,665][105620] Updated weights for policy 1, policy_version 998514 (0.0008) [2023-12-26 22:38:28,717][105620] Updated weights for policy 1, policy_version 998524 (0.0010) [2023-12-26 22:38:28,769][105620] Updated weights for policy 1, policy_version 998534 (0.0010) [2023-12-26 22:38:28,968][105692] Updated weights for policy 0, policy_version 998121 (0.0008) [2023-12-26 22:38:29,026][105692] Updated weights for policy 0, policy_version 998131 (0.0009) [2023-12-26 22:38:29,085][105692] Updated weights for policy 0, policy_version 998141 (0.0009) [2023-12-26 22:38:29,137][105692] Updated weights for policy 0, policy_version 998151 (0.0009) [2023-12-26 22:38:29,556][105620] Updated weights for policy 1, policy_version 998544 (0.0008) [2023-12-26 22:38:29,616][105620] Updated weights for policy 1, policy_version 998554 (0.0005) [2023-12-26 22:38:29,670][105620] Updated weights for policy 1, policy_version 998564 (0.0005) [2023-12-26 22:38:29,908][105692] Updated weights for policy 0, policy_version 998161 (0.0010) [2023-12-26 22:38:29,970][105692] Updated weights for policy 0, policy_version 998171 (0.0010) [2023-12-26 22:38:30,031][105692] Updated weights for policy 0, policy_version 998181 (0.0010) [2023-12-26 22:38:30,307][105620] Updated weights for policy 1, policy_version 998574 (0.0006) [2023-12-26 22:38:30,363][105620] Updated weights for policy 1, policy_version 998584 (0.0005) [2023-12-26 22:38:30,428][105620] Updated weights for policy 1, policy_version 998594 (0.0005) [2023-12-26 22:38:30,756][105692] Updated weights for policy 0, policy_version 998191 (0.0010) [2023-12-26 22:38:30,804][105692] Updated weights for policy 0, policy_version 998201 (0.0010) [2023-12-26 22:38:30,858][105692] Updated weights for policy 0, policy_version 998211 (0.0010) [2023-12-26 22:38:30,971][105620] Updated weights for policy 1, policy_version 998604 (0.0007) [2023-12-26 22:38:31,028][105620] Updated weights for policy 1, policy_version 998614 (0.0010) [2023-12-26 22:38:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 511254528. Throughput: 0: 9857.4, 1: 9833.5. Samples: 511222932. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:38:31,062][104569] Avg episode reward: [(0, '8907.361'), (1, '9079.582')] [2023-12-26 22:38:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000998216_255582208.pth... [2023-12-26 22:38:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000997064_255287296.pth [2023-12-26 22:38:31,092][105620] Updated weights for policy 1, policy_version 998624 (0.0011) [2023-12-26 22:38:31,144][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000998632_255680512.pth... [2023-12-26 22:38:31,148][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000997480_255385600.pth [2023-12-26 22:38:31,623][105692] Updated weights for policy 0, policy_version 998221 (0.0008) [2023-12-26 22:38:31,678][105692] Updated weights for policy 0, policy_version 998231 (0.0007) [2023-12-26 22:38:31,743][105692] Updated weights for policy 0, policy_version 998241 (0.0007) [2023-12-26 22:38:31,844][105620] Updated weights for policy 1, policy_version 998634 (0.0010) [2023-12-26 22:38:31,891][105620] Updated weights for policy 1, policy_version 998644 (0.0005) [2023-12-26 22:38:31,938][105620] Updated weights for policy 1, policy_version 998654 (0.0005) [2023-12-26 22:38:32,000][105620] Updated weights for policy 1, policy_version 998664 (0.0005) [2023-12-26 22:38:32,370][105692] Updated weights for policy 0, policy_version 998251 (0.0007) [2023-12-26 22:38:32,436][105692] Updated weights for policy 0, policy_version 998261 (0.0011) [2023-12-26 22:38:32,490][105692] Updated weights for policy 0, policy_version 998271 (0.0010) [2023-12-26 22:38:32,681][105620] Updated weights for policy 1, policy_version 998674 (0.0005) [2023-12-26 22:38:32,736][105620] Updated weights for policy 1, policy_version 998684 (0.0005) [2023-12-26 22:38:32,799][105620] Updated weights for policy 1, policy_version 998694 (0.0005) [2023-12-26 22:38:33,096][105692] Updated weights for policy 0, policy_version 998281 (0.0010) [2023-12-26 22:38:33,169][105692] Updated weights for policy 0, policy_version 998291 (0.0011) [2023-12-26 22:38:33,236][105692] Updated weights for policy 0, policy_version 998301 (0.0008) [2023-12-26 22:38:33,260][105585] KL-divergence is very high: 137.9383 [2023-12-26 22:38:33,277][105585] KL-divergence is very high: 108.2129 [2023-12-26 22:38:33,301][105692] Updated weights for policy 0, policy_version 998311 (0.0008) [2023-12-26 22:38:33,358][105620] Updated weights for policy 1, policy_version 998704 (0.0005) [2023-12-26 22:38:33,414][105620] Updated weights for policy 1, policy_version 998714 (0.0005) [2023-12-26 22:38:33,463][105620] Updated weights for policy 1, policy_version 998724 (0.0005) [2023-12-26 22:38:33,924][105585] KL-divergence is very high: 106.0351 [2023-12-26 22:38:33,968][105585] KL-divergence is very high: 107.1412 [2023-12-26 22:38:33,969][105692] Updated weights for policy 0, policy_version 998321 (0.0010) [2023-12-26 22:38:34,016][105585] KL-divergence is very high: 116.9376 [2023-12-26 22:38:34,026][105692] Updated weights for policy 0, policy_version 998331 (0.0010) [2023-12-26 22:38:34,062][105585] KL-divergence is very high: 100.0054 [2023-12-26 22:38:34,088][105692] Updated weights for policy 0, policy_version 998341 (0.0010) [2023-12-26 22:38:34,121][105620] Updated weights for policy 1, policy_version 998734 (0.0007) [2023-12-26 22:38:34,183][105620] Updated weights for policy 1, policy_version 998744 (0.0008) [2023-12-26 22:38:34,249][105620] Updated weights for policy 1, policy_version 998754 (0.0007) [2023-12-26 22:38:34,726][105692] Updated weights for policy 0, policy_version 998351 (0.0011) [2023-12-26 22:38:34,782][105692] Updated weights for policy 0, policy_version 998361 (0.0011) [2023-12-26 22:38:34,839][105692] Updated weights for policy 0, policy_version 998371 (0.0008) [2023-12-26 22:38:35,089][105620] Updated weights for policy 1, policy_version 998764 (0.0007) [2023-12-26 22:38:35,148][105620] Updated weights for policy 1, policy_version 998774 (0.0007) [2023-12-26 22:38:35,206][105620] Updated weights for policy 1, policy_version 998785 (0.0009) [2023-12-26 22:38:35,422][105692] Updated weights for policy 0, policy_version 998381 (0.0006) [2023-12-26 22:38:35,488][105692] Updated weights for policy 0, policy_version 998391 (0.0005) [2023-12-26 22:38:35,556][105692] Updated weights for policy 0, policy_version 998401 (0.0005) [2023-12-26 22:38:36,041][105620] Updated weights for policy 1, policy_version 998795 (0.0010) [2023-12-26 22:38:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.7, 300 sec: 19494.2). Total num frames: 511352832. Throughput: 0: 9905.2, 1: 9870.7. Samples: 511345504. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:38:36,063][104569] Avg episode reward: [(0, '8542.506'), (1, '9173.873')] [2023-12-26 22:38:36,098][105692] Updated weights for policy 0, policy_version 998411 (0.0006) [2023-12-26 22:38:36,100][105620] Updated weights for policy 1, policy_version 998805 (0.0009) [2023-12-26 22:38:36,160][105692] Updated weights for policy 0, policy_version 998421 (0.0008) [2023-12-26 22:38:36,166][105620] Updated weights for policy 1, policy_version 998815 (0.0006) [2023-12-26 22:38:36,215][105692] Updated weights for policy 0, policy_version 998431 (0.0008) [2023-12-26 22:38:36,800][105620] Updated weights for policy 1, policy_version 998825 (0.0006) [2023-12-26 22:38:36,858][105620] Updated weights for policy 1, policy_version 998835 (0.0006) [2023-12-26 22:38:36,922][105620] Updated weights for policy 1, policy_version 998845 (0.0007) [2023-12-26 22:38:36,981][105620] Updated weights for policy 1, policy_version 998855 (0.0006) [2023-12-26 22:38:37,029][105692] Updated weights for policy 0, policy_version 998441 (0.0009) [2023-12-26 22:38:37,104][105692] Updated weights for policy 0, policy_version 998451 (0.0009) [2023-12-26 22:38:37,166][105692] Updated weights for policy 0, policy_version 998461 (0.0007) [2023-12-26 22:38:37,220][105692] Updated weights for policy 0, policy_version 998471 (0.0007) [2023-12-26 22:38:37,626][105620] Updated weights for policy 1, policy_version 998865 (0.0009) [2023-12-26 22:38:37,684][105620] Updated weights for policy 1, policy_version 998875 (0.0005) [2023-12-26 22:38:37,739][105620] Updated weights for policy 1, policy_version 998885 (0.0006) [2023-12-26 22:38:37,943][105692] Updated weights for policy 0, policy_version 998481 (0.0008) [2023-12-26 22:38:38,000][105692] Updated weights for policy 0, policy_version 998491 (0.0008) [2023-12-26 22:38:38,063][105692] Updated weights for policy 0, policy_version 998501 (0.0008) [2023-12-26 22:38:38,458][105620] Updated weights for policy 1, policy_version 998895 (0.0011) [2023-12-26 22:38:38,504][105620] Updated weights for policy 1, policy_version 998905 (0.0010) [2023-12-26 22:38:38,553][105620] Updated weights for policy 1, policy_version 998915 (0.0010) [2023-12-26 22:38:38,771][105692] Updated weights for policy 0, policy_version 998511 (0.0008) [2023-12-26 22:38:38,823][105692] Updated weights for policy 0, policy_version 998521 (0.0009) [2023-12-26 22:38:38,877][105692] Updated weights for policy 0, policy_version 998531 (0.0008) [2023-12-26 22:38:39,263][105620] Updated weights for policy 1, policy_version 998925 (0.0009) [2023-12-26 22:38:39,323][105620] Updated weights for policy 1, policy_version 998935 (0.0008) [2023-12-26 22:38:39,391][105620] Updated weights for policy 1, policy_version 998945 (0.0010) [2023-12-26 22:38:39,725][105692] Updated weights for policy 0, policy_version 998541 (0.0008) [2023-12-26 22:38:39,798][105692] Updated weights for policy 0, policy_version 998551 (0.0008) [2023-12-26 22:38:39,865][105692] Updated weights for policy 0, policy_version 998561 (0.0008) [2023-12-26 22:38:40,154][105620] Updated weights for policy 1, policy_version 998955 (0.0010) [2023-12-26 22:38:40,213][105620] Updated weights for policy 1, policy_version 998965 (0.0009) [2023-12-26 22:38:40,266][105620] Updated weights for policy 1, policy_version 998975 (0.0009) [2023-12-26 22:38:40,596][105692] Updated weights for policy 0, policy_version 998571 (0.0009) [2023-12-26 22:38:40,643][105692] Updated weights for policy 0, policy_version 998581 (0.0008) [2023-12-26 22:38:40,694][105692] Updated weights for policy 0, policy_version 998591 (0.0009) [2023-12-26 22:38:40,952][105620] Updated weights for policy 1, policy_version 998985 (0.0009) [2023-12-26 22:38:41,014][105620] Updated weights for policy 1, policy_version 998995 (0.0005) [2023-12-26 22:38:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 511451136. Throughput: 0: 9866.5, 1: 9871.6. Samples: 511462928. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:38:41,062][104569] Avg episode reward: [(0, '8360.222'), (1, '8990.602')] [2023-12-26 22:38:41,074][105620] Updated weights for policy 1, policy_version 999005 (0.0008) [2023-12-26 22:38:41,148][105620] Updated weights for policy 1, policy_version 999015 (0.0011) [2023-12-26 22:38:41,582][105692] Updated weights for policy 0, policy_version 998601 (0.0009) [2023-12-26 22:38:41,645][105692] Updated weights for policy 0, policy_version 998611 (0.0009) [2023-12-26 22:38:41,712][105692] Updated weights for policy 0, policy_version 998621 (0.0008) [2023-12-26 22:38:41,775][105692] Updated weights for policy 0, policy_version 998631 (0.0009) [2023-12-26 22:38:41,859][105620] Updated weights for policy 1, policy_version 999025 (0.0010) [2023-12-26 22:38:41,916][105620] Updated weights for policy 1, policy_version 999035 (0.0010) [2023-12-26 22:38:41,986][105620] Updated weights for policy 1, policy_version 999045 (0.0010) [2023-12-26 22:38:42,564][105692] Updated weights for policy 0, policy_version 998641 (0.0008) [2023-12-26 22:38:42,630][105692] Updated weights for policy 0, policy_version 998651 (0.0009) [2023-12-26 22:38:42,687][105692] Updated weights for policy 0, policy_version 998661 (0.0008) [2023-12-26 22:38:42,756][105620] Updated weights for policy 1, policy_version 999055 (0.0010) [2023-12-26 22:38:42,822][105620] Updated weights for policy 1, policy_version 999065 (0.0011) [2023-12-26 22:38:42,887][105620] Updated weights for policy 1, policy_version 999075 (0.0010) [2023-12-26 22:38:43,464][105692] Updated weights for policy 0, policy_version 998671 (0.0008) [2023-12-26 22:38:43,508][105692] Updated weights for policy 0, policy_version 998681 (0.0008) [2023-12-26 22:38:43,554][105692] Updated weights for policy 0, policy_version 998691 (0.0008) [2023-12-26 22:38:43,622][105620] Updated weights for policy 1, policy_version 999085 (0.0010) [2023-12-26 22:38:43,679][105620] Updated weights for policy 1, policy_version 999095 (0.0010) [2023-12-26 22:38:43,737][105620] Updated weights for policy 1, policy_version 999105 (0.0010) [2023-12-26 22:38:44,355][105692] Updated weights for policy 0, policy_version 998701 (0.0008) [2023-12-26 22:38:44,403][105692] Updated weights for policy 0, policy_version 998711 (0.0008) [2023-12-26 22:38:44,458][105692] Updated weights for policy 0, policy_version 998721 (0.0008) [2023-12-26 22:38:44,478][105620] Updated weights for policy 1, policy_version 999115 (0.0010) [2023-12-26 22:38:44,532][105620] Updated weights for policy 1, policy_version 999125 (0.0010) [2023-12-26 22:38:44,583][105620] Updated weights for policy 1, policy_version 999135 (0.0010) [2023-12-26 22:38:45,251][105692] Updated weights for policy 0, policy_version 998731 (0.0007) [2023-12-26 22:38:45,313][105692] Updated weights for policy 0, policy_version 998741 (0.0008) [2023-12-26 22:38:45,340][105620] Updated weights for policy 1, policy_version 999145 (0.0010) [2023-12-26 22:38:45,374][105692] Updated weights for policy 0, policy_version 998751 (0.0007) [2023-12-26 22:38:45,400][105620] Updated weights for policy 1, policy_version 999155 (0.0010) [2023-12-26 22:38:45,463][105620] Updated weights for policy 1, policy_version 999165 (0.0010) [2023-12-26 22:38:45,521][105620] Updated weights for policy 1, policy_version 999175 (0.0010) [2023-12-26 22:38:46,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 511541248. Throughput: 0: 9749.4, 1: 9772.2. Samples: 511516380. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:38:46,062][104569] Avg episode reward: [(0, '8723.583'), (1, '9080.432')] [2023-12-26 22:38:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000999176_255819776.pth... [2023-12-26 22:38:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000998760_255721472.pth... [2023-12-26 22:38:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000997640_255434752.pth [2023-12-26 22:38:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000998056_255533056.pth [2023-12-26 22:38:46,175][105692] Updated weights for policy 0, policy_version 998761 (0.0007) [2023-12-26 22:38:46,197][105620] Updated weights for policy 1, policy_version 999185 (0.0007) [2023-12-26 22:38:46,233][105692] Updated weights for policy 0, policy_version 998771 (0.0009) [2023-12-26 22:38:46,252][105620] Updated weights for policy 1, policy_version 999195 (0.0006) [2023-12-26 22:38:46,283][105692] Updated weights for policy 0, policy_version 998781 (0.0007) [2023-12-26 22:38:46,310][105620] Updated weights for policy 1, policy_version 999205 (0.0007) [2023-12-26 22:38:46,334][105692] Updated weights for policy 0, policy_version 998791 (0.0007) [2023-12-26 22:38:46,904][105620] Updated weights for policy 1, policy_version 999215 (0.0006) [2023-12-26 22:38:46,959][105620] Updated weights for policy 1, policy_version 999225 (0.0005) [2023-12-26 22:38:47,022][105620] Updated weights for policy 1, policy_version 999235 (0.0010) [2023-12-26 22:38:47,073][105692] Updated weights for policy 0, policy_version 998801 (0.0007) [2023-12-26 22:38:47,120][105692] Updated weights for policy 0, policy_version 998811 (0.0008) [2023-12-26 22:38:47,168][105692] Updated weights for policy 0, policy_version 998822 (0.0009) [2023-12-26 22:38:47,696][105620] Updated weights for policy 1, policy_version 999245 (0.0009) [2023-12-26 22:38:47,744][105620] Updated weights for policy 1, policy_version 999255 (0.0009) [2023-12-26 22:38:47,802][105620] Updated weights for policy 1, policy_version 999265 (0.0009) [2023-12-26 22:38:47,954][105692] Updated weights for policy 0, policy_version 998832 (0.0006) [2023-12-26 22:38:48,013][105692] Updated weights for policy 0, policy_version 998842 (0.0009) [2023-12-26 22:38:48,077][105692] Updated weights for policy 0, policy_version 998852 (0.0008) [2023-12-26 22:38:48,502][105620] Updated weights for policy 1, policy_version 999275 (0.0007) [2023-12-26 22:38:48,556][105620] Updated weights for policy 1, policy_version 999285 (0.0007) [2023-12-26 22:38:48,627][105620] Updated weights for policy 1, policy_version 999295 (0.0005) [2023-12-26 22:38:48,827][105692] Updated weights for policy 0, policy_version 998862 (0.0008) [2023-12-26 22:38:48,877][105585] KL-divergence is very high: 354.0771 [2023-12-26 22:38:48,901][105692] Updated weights for policy 0, policy_version 998872 (0.0009) [2023-12-26 22:38:48,933][105585] KL-divergence is very high: 664.2942 [2023-12-26 22:38:48,963][105692] Updated weights for policy 0, policy_version 998882 (0.0010) [2023-12-26 22:38:48,983][105585] KL-divergence is very high: 767.7211 [2023-12-26 22:38:49,303][105620] Updated weights for policy 1, policy_version 999305 (0.0006) [2023-12-26 22:38:49,367][105620] Updated weights for policy 1, policy_version 999315 (0.0007) [2023-12-26 22:38:49,429][105620] Updated weights for policy 1, policy_version 999325 (0.0005) [2023-12-26 22:38:49,491][105620] Updated weights for policy 1, policy_version 999335 (0.0010) [2023-12-26 22:38:49,730][105692] Updated weights for policy 0, policy_version 998892 (0.0009) [2023-12-26 22:38:49,790][105692] Updated weights for policy 0, policy_version 998902 (0.0007) [2023-12-26 22:38:49,852][105692] Updated weights for policy 0, policy_version 998912 (0.0007) [2023-12-26 22:38:50,184][105620] Updated weights for policy 1, policy_version 999345 (0.0011) [2023-12-26 22:38:50,235][105620] Updated weights for policy 1, policy_version 999355 (0.0008) [2023-12-26 22:38:50,303][105620] Updated weights for policy 1, policy_version 999365 (0.0011) [2023-12-26 22:38:50,493][105692] Updated weights for policy 0, policy_version 998922 (0.0007) [2023-12-26 22:38:50,552][105692] Updated weights for policy 0, policy_version 998932 (0.0007) [2023-12-26 22:38:50,616][105692] Updated weights for policy 0, policy_version 998942 (0.0006) [2023-12-26 22:38:50,674][105692] Updated weights for policy 0, policy_version 998952 (0.0005) [2023-12-26 22:38:50,978][105620] Updated weights for policy 1, policy_version 999375 (0.0007) [2023-12-26 22:38:51,039][105620] Updated weights for policy 1, policy_version 999385 (0.0010) [2023-12-26 22:38:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 511639552. Throughput: 0: 9624.6, 1: 9837.7. Samples: 511631820. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:38:51,062][104569] Avg episode reward: [(0, '8902.273'), (1, '9265.985')] [2023-12-26 22:38:51,100][105620] Updated weights for policy 1, policy_version 999395 (0.0011) [2023-12-26 22:38:51,378][105692] Updated weights for policy 0, policy_version 998962 (0.0008) [2023-12-26 22:38:51,441][105692] Updated weights for policy 0, policy_version 998972 (0.0009) [2023-12-26 22:38:51,492][105692] Updated weights for policy 0, policy_version 998982 (0.0009) [2023-12-26 22:38:51,753][105620] Updated weights for policy 1, policy_version 999405 (0.0009) [2023-12-26 22:38:51,803][105620] Updated weights for policy 1, policy_version 999415 (0.0009) [2023-12-26 22:38:51,851][105620] Updated weights for policy 1, policy_version 999425 (0.0009) [2023-12-26 22:38:52,266][105692] Updated weights for policy 0, policy_version 998992 (0.0010) [2023-12-26 22:38:52,316][105692] Updated weights for policy 0, policy_version 999002 (0.0008) [2023-12-26 22:38:52,381][105692] Updated weights for policy 0, policy_version 999012 (0.0010) [2023-12-26 22:38:52,695][105620] Updated weights for policy 1, policy_version 999436 (0.0010) [2023-12-26 22:38:52,754][105620] Updated weights for policy 1, policy_version 999446 (0.0009) [2023-12-26 22:38:52,801][105620] Updated weights for policy 1, policy_version 999456 (0.0009) [2023-12-26 22:38:53,137][105692] Updated weights for policy 0, policy_version 999022 (0.0007) [2023-12-26 22:38:53,197][105692] Updated weights for policy 0, policy_version 999032 (0.0010) [2023-12-26 22:38:53,250][105692] Updated weights for policy 0, policy_version 999042 (0.0009) [2023-12-26 22:38:53,518][105620] Updated weights for policy 1, policy_version 999466 (0.0008) [2023-12-26 22:38:53,572][105620] Updated weights for policy 1, policy_version 999476 (0.0009) [2023-12-26 22:38:53,626][105620] Updated weights for policy 1, policy_version 999486 (0.0009) [2023-12-26 22:38:53,672][105620] Updated weights for policy 1, policy_version 999496 (0.0009) [2023-12-26 22:38:53,933][105692] Updated weights for policy 0, policy_version 999052 (0.0008) [2023-12-26 22:38:53,979][105692] Updated weights for policy 0, policy_version 999062 (0.0009) [2023-12-26 22:38:54,027][105692] Updated weights for policy 0, policy_version 999072 (0.0009) [2023-12-26 22:38:54,500][105620] Updated weights for policy 1, policy_version 999506 (0.0009) [2023-12-26 22:38:54,555][105620] Updated weights for policy 1, policy_version 999516 (0.0009) [2023-12-26 22:38:54,616][105620] Updated weights for policy 1, policy_version 999526 (0.0009) [2023-12-26 22:38:54,661][105692] Updated weights for policy 0, policy_version 999082 (0.0009) [2023-12-26 22:38:54,722][105692] Updated weights for policy 0, policy_version 999092 (0.0009) [2023-12-26 22:38:54,773][105692] Updated weights for policy 0, policy_version 999102 (0.0009) [2023-12-26 22:38:54,831][105692] Updated weights for policy 0, policy_version 999112 (0.0009) [2023-12-26 22:38:55,385][105620] Updated weights for policy 1, policy_version 999536 (0.0009) [2023-12-26 22:38:55,430][105620] Updated weights for policy 1, policy_version 999546 (0.0008) [2023-12-26 22:38:55,481][105620] Updated weights for policy 1, policy_version 999556 (0.0008) [2023-12-26 22:38:55,601][105692] Updated weights for policy 0, policy_version 999122 (0.0009) [2023-12-26 22:38:55,656][105692] Updated weights for policy 0, policy_version 999132 (0.0009) [2023-12-26 22:38:55,709][105692] Updated weights for policy 0, policy_version 999142 (0.0008) [2023-12-26 22:38:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 511737856. Throughput: 0: 9578.0, 1: 9855.5. Samples: 511748116. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:38:56,063][104569] Avg episode reward: [(0, '8813.327'), (1, '9266.048')] [2023-12-26 22:38:56,288][105620] Updated weights for policy 1, policy_version 999566 (0.0009) [2023-12-26 22:38:56,333][105620] Updated weights for policy 1, policy_version 999576 (0.0006) [2023-12-26 22:38:56,335][105692] Updated weights for policy 0, policy_version 999152 (0.0007) [2023-12-26 22:38:56,341][105585] KL-divergence is very high: 100.2329 [2023-12-26 22:38:56,379][105620] Updated weights for policy 1, policy_version 999586 (0.0005) [2023-12-26 22:38:56,387][105585] KL-divergence is very high: 192.9904 [2023-12-26 22:38:56,391][105692] Updated weights for policy 0, policy_version 999162 (0.0008) [2023-12-26 22:38:56,434][105585] KL-divergence is very high: 219.8083 [2023-12-26 22:38:56,453][105692] Updated weights for policy 0, policy_version 999172 (0.0009) [2023-12-26 22:38:57,156][105620] Updated weights for policy 1, policy_version 999596 (0.0007) [2023-12-26 22:38:57,189][105692] Updated weights for policy 0, policy_version 999182 (0.0010) [2023-12-26 22:38:57,204][105620] Updated weights for policy 1, policy_version 999606 (0.0008) [2023-12-26 22:38:57,246][105692] Updated weights for policy 0, policy_version 999192 (0.0007) [2023-12-26 22:38:57,254][105620] Updated weights for policy 1, policy_version 999616 (0.0005) [2023-12-26 22:38:57,299][105692] Updated weights for policy 0, policy_version 999202 (0.0006) [2023-12-26 22:38:57,964][105692] Updated weights for policy 0, policy_version 999212 (0.0008) [2023-12-26 22:38:58,014][105692] Updated weights for policy 0, policy_version 999222 (0.0008) [2023-12-26 22:38:58,032][105620] Updated weights for policy 1, policy_version 999626 (0.0008) [2023-12-26 22:38:58,073][105692] Updated weights for policy 0, policy_version 999232 (0.0008) [2023-12-26 22:38:58,083][105620] Updated weights for policy 1, policy_version 999636 (0.0007) [2023-12-26 22:38:58,133][105620] Updated weights for policy 1, policy_version 999646 (0.0007) [2023-12-26 22:38:58,198][105620] Updated weights for policy 1, policy_version 999656 (0.0008) [2023-12-26 22:38:58,839][105692] Updated weights for policy 0, policy_version 999242 (0.0007) [2023-12-26 22:38:58,901][105692] Updated weights for policy 0, policy_version 999252 (0.0008) [2023-12-26 22:38:58,960][105692] Updated weights for policy 0, policy_version 999262 (0.0008) [2023-12-26 22:38:58,992][105620] Updated weights for policy 1, policy_version 999666 (0.0008) [2023-12-26 22:38:59,014][105692] Updated weights for policy 0, policy_version 999272 (0.0008) [2023-12-26 22:38:59,045][105620] Updated weights for policy 1, policy_version 999676 (0.0007) [2023-12-26 22:38:59,106][105620] Updated weights for policy 1, policy_version 999686 (0.0007) [2023-12-26 22:38:59,736][105692] Updated weights for policy 0, policy_version 999282 (0.0007) [2023-12-26 22:38:59,798][105692] Updated weights for policy 0, policy_version 999292 (0.0009) [2023-12-26 22:38:59,861][105692] Updated weights for policy 0, policy_version 999302 (0.0009) [2023-12-26 22:38:59,868][105620] Updated weights for policy 1, policy_version 999696 (0.0006) [2023-12-26 22:38:59,927][105620] Updated weights for policy 1, policy_version 999706 (0.0008) [2023-12-26 22:38:59,990][105620] Updated weights for policy 1, policy_version 999716 (0.0010) [2023-12-26 22:39:00,534][105692] Updated weights for policy 0, policy_version 999312 (0.0010) [2023-12-26 22:39:00,593][105692] Updated weights for policy 0, policy_version 999322 (0.0011) [2023-12-26 22:39:00,611][105620] Updated weights for policy 1, policy_version 999726 (0.0007) [2023-12-26 22:39:00,652][105692] Updated weights for policy 0, policy_version 999332 (0.0010) [2023-12-26 22:39:00,662][105620] Updated weights for policy 1, policy_version 999736 (0.0007) [2023-12-26 22:39:00,715][105620] Updated weights for policy 1, policy_version 999746 (0.0007) [2023-12-26 22:39:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 511836160. Throughput: 0: 9649.1, 1: 9795.5. Samples: 511805008. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:39:01,062][104569] Avg episode reward: [(0, '8995.763'), (1, '9100.705')] [2023-12-26 22:39:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000999336_255868928.pth... [2023-12-26 22:39:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000999752_255967232.pth... [2023-12-26 22:39:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000998632_255680512.pth [2023-12-26 22:39:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000998216_255582208.pth [2023-12-26 22:39:01,352][105620] Updated weights for policy 1, policy_version 999756 (0.0010) [2023-12-26 22:39:01,372][105692] Updated weights for policy 0, policy_version 999342 (0.0011) [2023-12-26 22:39:01,424][105620] Updated weights for policy 1, policy_version 999766 (0.0009) [2023-12-26 22:39:01,438][105692] Updated weights for policy 0, policy_version 999352 (0.0007) [2023-12-26 22:39:01,486][105620] Updated weights for policy 1, policy_version 999776 (0.0010) [2023-12-26 22:39:01,498][105692] Updated weights for policy 0, policy_version 999362 (0.0005) [2023-12-26 22:39:02,100][105620] Updated weights for policy 1, policy_version 999786 (0.0009) [2023-12-26 22:39:02,160][105620] Updated weights for policy 1, policy_version 999796 (0.0006) [2023-12-26 22:39:02,231][105620] Updated weights for policy 1, policy_version 999806 (0.0006) [2023-12-26 22:39:02,289][105692] Updated weights for policy 0, policy_version 999372 (0.0006) [2023-12-26 22:39:02,296][105620] Updated weights for policy 1, policy_version 999816 (0.0010) [2023-12-26 22:39:02,347][105692] Updated weights for policy 0, policy_version 999382 (0.0006) [2023-12-26 22:39:02,403][105692] Updated weights for policy 0, policy_version 999392 (0.0009) [2023-12-26 22:39:02,997][105620] Updated weights for policy 1, policy_version 999826 (0.0010) [2023-12-26 22:39:03,042][105620] Updated weights for policy 1, policy_version 999836 (0.0008) [2023-12-26 22:39:03,094][105620] Updated weights for policy 1, policy_version 999846 (0.0010) [2023-12-26 22:39:03,171][105692] Updated weights for policy 0, policy_version 999402 (0.0009) [2023-12-26 22:39:03,229][105692] Updated weights for policy 0, policy_version 999412 (0.0011) [2023-12-26 22:39:03,290][105692] Updated weights for policy 0, policy_version 999422 (0.0009) [2023-12-26 22:39:03,341][105692] Updated weights for policy 0, policy_version 999432 (0.0005) [2023-12-26 22:39:03,698][105620] Updated weights for policy 1, policy_version 999856 (0.0010) [2023-12-26 22:39:03,749][105620] Updated weights for policy 1, policy_version 999866 (0.0010) [2023-12-26 22:39:03,801][105620] Updated weights for policy 1, policy_version 999876 (0.0010) [2023-12-26 22:39:03,955][105692] Updated weights for policy 0, policy_version 999442 (0.0005) [2023-12-26 22:39:04,015][105692] Updated weights for policy 0, policy_version 999452 (0.0005) [2023-12-26 22:39:04,075][105692] Updated weights for policy 0, policy_version 999462 (0.0006) [2023-12-26 22:39:04,574][105620] Updated weights for policy 1, policy_version 999886 (0.0009) [2023-12-26 22:39:04,635][105692] Updated weights for policy 0, policy_version 999472 (0.0007) [2023-12-26 22:39:04,644][105620] Updated weights for policy 1, policy_version 999896 (0.0010) [2023-12-26 22:39:04,694][105692] Updated weights for policy 0, policy_version 999482 (0.0006) [2023-12-26 22:39:04,699][105620] Updated weights for policy 1, policy_version 999906 (0.0010) [2023-12-26 22:39:04,754][105692] Updated weights for policy 0, policy_version 999492 (0.0006) [2023-12-26 22:39:05,288][105620] Updated weights for policy 1, policy_version 999916 (0.0010) [2023-12-26 22:39:05,339][105620] Updated weights for policy 1, policy_version 999926 (0.0010) [2023-12-26 22:39:05,391][105620] Updated weights for policy 1, policy_version 999936 (0.0010) [2023-12-26 22:39:05,538][105692] Updated weights for policy 0, policy_version 999502 (0.0007) [2023-12-26 22:39:05,581][105692] Updated weights for policy 0, policy_version 999512 (0.0007) [2023-12-26 22:39:05,625][105692] Updated weights for policy 0, policy_version 999522 (0.0008) [2023-12-26 22:39:06,046][105620] Updated weights for policy 1, policy_version 999946 (0.0010) [2023-12-26 22:39:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 511934464. Throughput: 0: 9673.2, 1: 9790.8. Samples: 511925540. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:39:06,062][104569] Avg episode reward: [(0, '9264.298'), (1, '8915.501')] [2023-12-26 22:39:06,105][105620] Updated weights for policy 1, policy_version 999956 (0.0011) [2023-12-26 22:39:06,161][105620] Updated weights for policy 1, policy_version 999966 (0.0010) [2023-12-26 22:39:06,212][105620] Updated weights for policy 1, policy_version 999976 (0.0009) [2023-12-26 22:39:06,455][105692] Updated weights for policy 0, policy_version 999532 (0.0007) [2023-12-26 22:39:06,518][105692] Updated weights for policy 0, policy_version 999542 (0.0006) [2023-12-26 22:39:06,575][105692] Updated weights for policy 0, policy_version 999552 (0.0005) [2023-12-26 22:39:06,925][105620] Updated weights for policy 1, policy_version 999986 (0.0007) [2023-12-26 22:39:06,988][105620] Updated weights for policy 1, policy_version 999996 (0.0009) [2023-12-26 22:39:07,048][105620] Updated weights for policy 1, policy_version 1000006 (0.0008) [2023-12-26 22:39:07,209][105692] Updated weights for policy 0, policy_version 999562 (0.0006) [2023-12-26 22:39:07,265][105692] Updated weights for policy 0, policy_version 999572 (0.0009) [2023-12-26 22:39:07,321][105692] Updated weights for policy 0, policy_version 999582 (0.0007) [2023-12-26 22:39:07,373][105692] Updated weights for policy 0, policy_version 999592 (0.0005) [2023-12-26 22:39:07,837][105620] Updated weights for policy 1, policy_version 1000016 (0.0008) [2023-12-26 22:39:07,894][105620] Updated weights for policy 1, policy_version 1000026 (0.0010) [2023-12-26 22:39:07,954][105620] Updated weights for policy 1, policy_version 1000036 (0.0010) [2023-12-26 22:39:08,036][105692] Updated weights for policy 0, policy_version 999602 (0.0008) [2023-12-26 22:39:08,085][105692] Updated weights for policy 0, policy_version 999612 (0.0009) [2023-12-26 22:39:08,143][105692] Updated weights for policy 0, policy_version 999622 (0.0010) [2023-12-26 22:39:08,577][105620] Updated weights for policy 1, policy_version 1000046 (0.0009) [2023-12-26 22:39:08,642][105620] Updated weights for policy 1, policy_version 1000056 (0.0009) [2023-12-26 22:39:08,707][105620] Updated weights for policy 1, policy_version 1000066 (0.0009) [2023-12-26 22:39:08,980][105692] Updated weights for policy 0, policy_version 999632 (0.0008) [2023-12-26 22:39:09,046][105692] Updated weights for policy 0, policy_version 999642 (0.0010) [2023-12-26 22:39:09,110][105692] Updated weights for policy 0, policy_version 999652 (0.0009) [2023-12-26 22:39:09,420][105620] Updated weights for policy 1, policy_version 1000076 (0.0010) [2023-12-26 22:39:09,478][105620] Updated weights for policy 1, policy_version 1000086 (0.0009) [2023-12-26 22:39:09,541][105620] Updated weights for policy 1, policy_version 1000096 (0.0010) [2023-12-26 22:39:09,911][105692] Updated weights for policy 0, policy_version 999662 (0.0008) [2023-12-26 22:39:09,969][105692] Updated weights for policy 0, policy_version 999672 (0.0009) [2023-12-26 22:39:10,034][105692] Updated weights for policy 0, policy_version 999682 (0.0009) [2023-12-26 22:39:10,288][105620] Updated weights for policy 1, policy_version 1000106 (0.0011) [2023-12-26 22:39:10,341][105620] Updated weights for policy 1, policy_version 1000116 (0.0011) [2023-12-26 22:39:10,397][105620] Updated weights for policy 1, policy_version 1000126 (0.0011) [2023-12-26 22:39:10,453][105620] Updated weights for policy 1, policy_version 1000136 (0.0011) [2023-12-26 22:39:10,802][105692] Updated weights for policy 0, policy_version 999692 (0.0008) [2023-12-26 22:39:10,853][105692] Updated weights for policy 0, policy_version 999702 (0.0008) [2023-12-26 22:39:10,905][105692] Updated weights for policy 0, policy_version 999712 (0.0008) [2023-12-26 22:39:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 512032768. Throughput: 0: 9639.6, 1: 9836.6. Samples: 512040848. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:39:11,062][104569] Avg episode reward: [(0, '9264.443'), (1, '9170.624')] [2023-12-26 22:39:11,208][105620] Updated weights for policy 1, policy_version 1000146 (0.0010) [2023-12-26 22:39:11,275][105620] Updated weights for policy 1, policy_version 1000156 (0.0010) [2023-12-26 22:39:11,331][105620] Updated weights for policy 1, policy_version 1000166 (0.0010) [2023-12-26 22:39:11,702][105692] Updated weights for policy 0, policy_version 999722 (0.0009) [2023-12-26 22:39:11,762][105692] Updated weights for policy 0, policy_version 999732 (0.0009) [2023-12-26 22:39:11,822][105692] Updated weights for policy 0, policy_version 999742 (0.0008) [2023-12-26 22:39:11,882][105692] Updated weights for policy 0, policy_version 999752 (0.0008) [2023-12-26 22:39:12,095][105620] Updated weights for policy 1, policy_version 1000176 (0.0010) [2023-12-26 22:39:12,153][105620] Updated weights for policy 1, policy_version 1000186 (0.0011) [2023-12-26 22:39:12,215][105620] Updated weights for policy 1, policy_version 1000196 (0.0010) [2023-12-26 22:39:12,657][105692] Updated weights for policy 0, policy_version 999762 (0.0008) [2023-12-26 22:39:12,716][105692] Updated weights for policy 0, policy_version 999772 (0.0008) [2023-12-26 22:39:12,776][105692] Updated weights for policy 0, policy_version 999782 (0.0008) [2023-12-26 22:39:12,989][105620] Updated weights for policy 1, policy_version 1000206 (0.0010) [2023-12-26 22:39:13,044][105620] Updated weights for policy 1, policy_version 1000216 (0.0010) [2023-12-26 22:39:13,103][105620] Updated weights for policy 1, policy_version 1000226 (0.0010) [2023-12-26 22:39:13,551][105692] Updated weights for policy 0, policy_version 999792 (0.0008) [2023-12-26 22:39:13,602][105692] Updated weights for policy 0, policy_version 999802 (0.0008) [2023-12-26 22:39:13,661][105692] Updated weights for policy 0, policy_version 999812 (0.0009) [2023-12-26 22:39:13,805][105620] Updated weights for policy 1, policy_version 1000236 (0.0008) [2023-12-26 22:39:13,858][105620] Updated weights for policy 1, policy_version 1000246 (0.0006) [2023-12-26 22:39:13,910][105620] Updated weights for policy 1, policy_version 1000256 (0.0009) [2023-12-26 22:39:14,485][105692] Updated weights for policy 0, policy_version 999822 (0.0009) [2023-12-26 22:39:14,545][105692] Updated weights for policy 0, policy_version 999832 (0.0005) [2023-12-26 22:39:14,556][105620] Updated weights for policy 1, policy_version 1000267 (0.0007) [2023-12-26 22:39:14,597][105692] Updated weights for policy 0, policy_version 999842 (0.0008) [2023-12-26 22:39:14,613][105620] Updated weights for policy 1, policy_version 1000277 (0.0006) [2023-12-26 22:39:14,666][105620] Updated weights for policy 1, policy_version 1000287 (0.0005) [2023-12-26 22:39:15,286][105692] Updated weights for policy 0, policy_version 999852 (0.0009) [2023-12-26 22:39:15,310][105620] Updated weights for policy 1, policy_version 1000297 (0.0006) [2023-12-26 22:39:15,336][105692] Updated weights for policy 0, policy_version 999862 (0.0009) [2023-12-26 22:39:15,374][105620] Updated weights for policy 1, policy_version 1000307 (0.0009) [2023-12-26 22:39:15,389][105692] Updated weights for policy 0, policy_version 999872 (0.0007) [2023-12-26 22:39:15,430][105620] Updated weights for policy 1, policy_version 1000317 (0.0008) [2023-12-26 22:39:15,488][105620] Updated weights for policy 1, policy_version 1000327 (0.0005) [2023-12-26 22:39:15,991][105692] Updated weights for policy 0, policy_version 999882 (0.0008) [2023-12-26 22:39:16,053][105692] Updated weights for policy 0, policy_version 999892 (0.0008) [2023-12-26 22:39:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 512122880. Throughput: 0: 9591.0, 1: 9803.7. Samples: 512095696. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:39:16,062][104569] Avg episode reward: [(0, '8904.544'), (1, '9263.213')] [2023-12-26 22:39:16,093][105620] Updated weights for policy 1, policy_version 1000337 (0.0006) [2023-12-26 22:39:16,114][105692] Updated weights for policy 0, policy_version 999902 (0.0008) [2023-12-26 22:39:16,141][105620] Updated weights for policy 1, policy_version 1000347 (0.0006) [2023-12-26 22:39:16,171][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000999912_256016384.pth... [2023-12-26 22:39:16,173][105692] Updated weights for policy 0, policy_version 999912 (0.0009) [2023-12-26 22:39:16,175][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000998760_255721472.pth [2023-12-26 22:39:16,190][105620] Updated weights for policy 1, policy_version 1000357 (0.0006) [2023-12-26 22:39:16,204][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001000360_256122880.pth... [2023-12-26 22:39:16,207][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000999176_255819776.pth [2023-12-26 22:39:16,813][105620] Updated weights for policy 1, policy_version 1000367 (0.0005) [2023-12-26 22:39:16,869][105620] Updated weights for policy 1, policy_version 1000377 (0.0006) [2023-12-26 22:39:16,933][105620] Updated weights for policy 1, policy_version 1000387 (0.0008) [2023-12-26 22:39:16,984][105692] Updated weights for policy 0, policy_version 999922 (0.0010) [2023-12-26 22:39:17,031][105692] Updated weights for policy 0, policy_version 999932 (0.0009) [2023-12-26 22:39:17,079][105692] Updated weights for policy 0, policy_version 999942 (0.0008) [2023-12-26 22:39:17,640][105620] Updated weights for policy 1, policy_version 1000397 (0.0008) [2023-12-26 22:39:17,700][105620] Updated weights for policy 1, policy_version 1000407 (0.0008) [2023-12-26 22:39:17,763][105620] Updated weights for policy 1, policy_version 1000417 (0.0007) [2023-12-26 22:39:17,817][105692] Updated weights for policy 0, policy_version 999952 (0.0009) [2023-12-26 22:39:17,875][105692] Updated weights for policy 0, policy_version 999962 (0.0009) [2023-12-26 22:39:17,933][105692] Updated weights for policy 0, policy_version 999972 (0.0010) [2023-12-26 22:39:18,363][105620] Updated weights for policy 1, policy_version 1000427 (0.0006) [2023-12-26 22:39:18,427][105620] Updated weights for policy 1, policy_version 1000437 (0.0010) [2023-12-26 22:39:18,484][105620] Updated weights for policy 1, policy_version 1000447 (0.0009) [2023-12-26 22:39:18,753][105692] Updated weights for policy 0, policy_version 999982 (0.0008) [2023-12-26 22:39:18,818][105692] Updated weights for policy 0, policy_version 999992 (0.0008) [2023-12-26 22:39:18,879][105692] Updated weights for policy 0, policy_version 1000002 (0.0008) [2023-12-26 22:39:19,229][105620] Updated weights for policy 1, policy_version 1000457 (0.0010) [2023-12-26 22:39:19,290][105620] Updated weights for policy 1, policy_version 1000467 (0.0010) [2023-12-26 22:39:19,356][105620] Updated weights for policy 1, policy_version 1000477 (0.0008) [2023-12-26 22:39:19,416][105620] Updated weights for policy 1, policy_version 1000487 (0.0008) [2023-12-26 22:39:19,676][105692] Updated weights for policy 0, policy_version 1000012 (0.0008) [2023-12-26 22:39:19,728][105692] Updated weights for policy 0, policy_version 1000022 (0.0009) [2023-12-26 22:39:19,781][105692] Updated weights for policy 0, policy_version 1000032 (0.0009) [2023-12-26 22:39:20,228][105620] Updated weights for policy 1, policy_version 1000497 (0.0008) [2023-12-26 22:39:20,281][105620] Updated weights for policy 1, policy_version 1000507 (0.0009) [2023-12-26 22:39:20,342][105620] Updated weights for policy 1, policy_version 1000517 (0.0007) [2023-12-26 22:39:20,605][105692] Updated weights for policy 0, policy_version 1000042 (0.0009) [2023-12-26 22:39:20,669][105692] Updated weights for policy 0, policy_version 1000052 (0.0010) [2023-12-26 22:39:20,729][105692] Updated weights for policy 0, policy_version 1000062 (0.0011) [2023-12-26 22:39:20,794][105692] Updated weights for policy 0, policy_version 1000072 (0.0011) [2023-12-26 22:39:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 512221184. Throughput: 0: 9493.0, 1: 9818.6. Samples: 512214524. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:39:21,063][104569] Avg episode reward: [(0, '8723.700'), (1, '9355.635')] [2023-12-26 22:39:21,072][105620] Updated weights for policy 1, policy_version 1000527 (0.0010) [2023-12-26 22:39:21,134][105620] Updated weights for policy 1, policy_version 1000537 (0.0010) [2023-12-26 22:39:21,199][105620] Updated weights for policy 1, policy_version 1000547 (0.0011) [2023-12-26 22:39:21,518][105692] Updated weights for policy 0, policy_version 1000082 (0.0010) [2023-12-26 22:39:21,574][105692] Updated weights for policy 0, policy_version 1000092 (0.0011) [2023-12-26 22:39:21,638][105692] Updated weights for policy 0, policy_version 1000102 (0.0012) [2023-12-26 22:39:21,989][105620] Updated weights for policy 1, policy_version 1000557 (0.0011) [2023-12-26 22:39:22,052][105620] Updated weights for policy 1, policy_version 1000567 (0.0011) [2023-12-26 22:39:22,112][105620] Updated weights for policy 1, policy_version 1000577 (0.0011) [2023-12-26 22:39:22,274][105692] Updated weights for policy 0, policy_version 1000112 (0.0007) [2023-12-26 22:39:22,342][105692] Updated weights for policy 0, policy_version 1000122 (0.0008) [2023-12-26 22:39:22,408][105692] Updated weights for policy 0, policy_version 1000132 (0.0009) [2023-12-26 22:39:22,781][105620] Updated weights for policy 1, policy_version 1000587 (0.0011) [2023-12-26 22:39:22,844][105620] Updated weights for policy 1, policy_version 1000597 (0.0011) [2023-12-26 22:39:22,893][105620] Updated weights for policy 1, policy_version 1000607 (0.0010) [2023-12-26 22:39:23,158][105692] Updated weights for policy 0, policy_version 1000142 (0.0009) [2023-12-26 22:39:23,210][105692] Updated weights for policy 0, policy_version 1000152 (0.0008) [2023-12-26 22:39:23,258][105692] Updated weights for policy 0, policy_version 1000162 (0.0008) [2023-12-26 22:39:23,586][105620] Updated weights for policy 1, policy_version 1000617 (0.0010) [2023-12-26 22:39:23,640][105620] Updated weights for policy 1, policy_version 1000627 (0.0005) [2023-12-26 22:39:23,692][105620] Updated weights for policy 1, policy_version 1000637 (0.0007) [2023-12-26 22:39:23,740][105620] Updated weights for policy 1, policy_version 1000647 (0.0010) [2023-12-26 22:39:24,078][105692] Updated weights for policy 0, policy_version 1000172 (0.0008) [2023-12-26 22:39:24,138][105692] Updated weights for policy 0, policy_version 1000182 (0.0008) [2023-12-26 22:39:24,201][105692] Updated weights for policy 0, policy_version 1000192 (0.0008) [2023-12-26 22:39:24,441][105620] Updated weights for policy 1, policy_version 1000657 (0.0006) [2023-12-26 22:39:24,499][105620] Updated weights for policy 1, policy_version 1000667 (0.0006) [2023-12-26 22:39:24,561][105620] Updated weights for policy 1, policy_version 1000677 (0.0006) [2023-12-26 22:39:24,967][105692] Updated weights for policy 0, policy_version 1000202 (0.0008) [2023-12-26 22:39:25,023][105692] Updated weights for policy 0, policy_version 1000212 (0.0010) [2023-12-26 22:39:25,079][105692] Updated weights for policy 0, policy_version 1000222 (0.0009) [2023-12-26 22:39:25,148][105620] Updated weights for policy 1, policy_version 1000687 (0.0007) [2023-12-26 22:39:25,159][105692] Updated weights for policy 0, policy_version 1000232 (0.0009) [2023-12-26 22:39:25,208][105620] Updated weights for policy 1, policy_version 1000697 (0.0009) [2023-12-26 22:39:25,259][105620] Updated weights for policy 1, policy_version 1000707 (0.0009) [2023-12-26 22:39:25,903][105692] Updated weights for policy 0, policy_version 1000242 (0.0009) [2023-12-26 22:39:25,949][105692] Updated weights for policy 0, policy_version 1000252 (0.0009) [2023-12-26 22:39:25,994][105692] Updated weights for policy 0, policy_version 1000262 (0.0009) [2023-12-26 22:39:26,026][105620] Updated weights for policy 1, policy_version 1000717 (0.0009) [2023-12-26 22:39:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 512319488. Throughput: 0: 9414.8, 1: 9814.5. Samples: 512328244. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:39:26,062][104569] Avg episode reward: [(0, '8905.110'), (1, '9355.546')] [2023-12-26 22:39:26,073][105620] Updated weights for policy 1, policy_version 1000727 (0.0007) [2023-12-26 22:39:26,119][105620] Updated weights for policy 1, policy_version 1000737 (0.0005) [2023-12-26 22:39:26,793][105692] Updated weights for policy 0, policy_version 1000272 (0.0009) [2023-12-26 22:39:26,813][105620] Updated weights for policy 1, policy_version 1000747 (0.0006) [2023-12-26 22:39:26,840][105692] Updated weights for policy 0, policy_version 1000282 (0.0008) [2023-12-26 22:39:26,858][105620] Updated weights for policy 1, policy_version 1000757 (0.0006) [2023-12-26 22:39:26,892][105692] Updated weights for policy 0, policy_version 1000292 (0.0008) [2023-12-26 22:39:26,902][105620] Updated weights for policy 1, policy_version 1000767 (0.0005) [2023-12-26 22:39:27,580][105620] Updated weights for policy 1, policy_version 1000777 (0.0008) [2023-12-26 22:39:27,628][105620] Updated weights for policy 1, policy_version 1000787 (0.0005) [2023-12-26 22:39:27,673][105620] Updated weights for policy 1, policy_version 1000797 (0.0005) [2023-12-26 22:39:27,722][105620] Updated weights for policy 1, policy_version 1000807 (0.0006) [2023-12-26 22:39:27,732][105692] Updated weights for policy 0, policy_version 1000302 (0.0008) [2023-12-26 22:39:27,789][105692] Updated weights for policy 0, policy_version 1000312 (0.0009) [2023-12-26 22:39:27,845][105692] Updated weights for policy 0, policy_version 1000322 (0.0009) [2023-12-26 22:39:28,411][105620] Updated weights for policy 1, policy_version 1000817 (0.0010) [2023-12-26 22:39:28,463][105620] Updated weights for policy 1, policy_version 1000827 (0.0010) [2023-12-26 22:39:28,522][105620] Updated weights for policy 1, policy_version 1000837 (0.0011) [2023-12-26 22:39:28,610][105692] Updated weights for policy 0, policy_version 1000332 (0.0008) [2023-12-26 22:39:28,666][105692] Updated weights for policy 0, policy_version 1000342 (0.0007) [2023-12-26 22:39:28,722][105692] Updated weights for policy 0, policy_version 1000352 (0.0008) [2023-12-26 22:39:29,263][105620] Updated weights for policy 1, policy_version 1000847 (0.0010) [2023-12-26 22:39:29,319][105620] Updated weights for policy 1, policy_version 1000857 (0.0010) [2023-12-26 22:39:29,385][105620] Updated weights for policy 1, policy_version 1000867 (0.0008) [2023-12-26 22:39:29,495][105692] Updated weights for policy 0, policy_version 1000362 (0.0008) [2023-12-26 22:39:29,551][105692] Updated weights for policy 0, policy_version 1000372 (0.0008) [2023-12-26 22:39:29,602][105692] Updated weights for policy 0, policy_version 1000382 (0.0008) [2023-12-26 22:39:29,658][105692] Updated weights for policy 0, policy_version 1000392 (0.0008) [2023-12-26 22:39:30,098][105620] Updated weights for policy 1, policy_version 1000877 (0.0009) [2023-12-26 22:39:30,145][105620] Updated weights for policy 1, policy_version 1000887 (0.0008) [2023-12-26 22:39:30,192][105620] Updated weights for policy 1, policy_version 1000897 (0.0009) [2023-12-26 22:39:30,414][105692] Updated weights for policy 0, policy_version 1000402 (0.0008) [2023-12-26 22:39:30,462][105692] Updated weights for policy 0, policy_version 1000412 (0.0009) [2023-12-26 22:39:30,510][105692] Updated weights for policy 0, policy_version 1000422 (0.0009) [2023-12-26 22:39:30,932][105620] Updated weights for policy 1, policy_version 1000907 (0.0009) [2023-12-26 22:39:30,977][105620] Updated weights for policy 1, policy_version 1000917 (0.0008) [2023-12-26 22:39:31,033][105620] Updated weights for policy 1, policy_version 1000927 (0.0008) [2023-12-26 22:39:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 512409600. Throughput: 0: 9443.3, 1: 9879.6. Samples: 512385912. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:39:31,062][104569] Avg episode reward: [(0, '8365.751'), (1, '9263.843')] [2023-12-26 22:39:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001000424_256147456.pth... [2023-12-26 22:39:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000999336_255868928.pth [2023-12-26 22:39:31,097][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001000936_256270336.pth... [2023-12-26 22:39:31,102][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_000999752_255967232.pth [2023-12-26 22:39:31,338][105692] Updated weights for policy 0, policy_version 1000432 (0.0009) [2023-12-26 22:39:31,401][105692] Updated weights for policy 0, policy_version 1000442 (0.0008) [2023-12-26 22:39:31,467][105692] Updated weights for policy 0, policy_version 1000452 (0.0008) [2023-12-26 22:39:31,767][105620] Updated weights for policy 1, policy_version 1000937 (0.0006) [2023-12-26 22:39:31,818][105620] Updated weights for policy 1, policy_version 1000947 (0.0008) [2023-12-26 22:39:31,873][105620] Updated weights for policy 1, policy_version 1000957 (0.0009) [2023-12-26 22:39:31,923][105620] Updated weights for policy 1, policy_version 1000967 (0.0008) [2023-12-26 22:39:32,233][105692] Updated weights for policy 0, policy_version 1000462 (0.0010) [2023-12-26 22:39:32,299][105692] Updated weights for policy 0, policy_version 1000472 (0.0009) [2023-12-26 22:39:32,362][105692] Updated weights for policy 0, policy_version 1000482 (0.0008) [2023-12-26 22:39:32,558][105620] Updated weights for policy 1, policy_version 1000977 (0.0009) [2023-12-26 22:39:32,615][105620] Updated weights for policy 1, policy_version 1000987 (0.0008) [2023-12-26 22:39:32,673][105620] Updated weights for policy 1, policy_version 1000997 (0.0009) [2023-12-26 22:39:33,014][105692] Updated weights for policy 0, policy_version 1000492 (0.0007) [2023-12-26 22:39:33,067][105692] Updated weights for policy 0, policy_version 1000502 (0.0005) [2023-12-26 22:39:33,125][105692] Updated weights for policy 0, policy_version 1000512 (0.0006) [2023-12-26 22:39:33,400][105620] Updated weights for policy 1, policy_version 1001007 (0.0010) [2023-12-26 22:39:33,458][105620] Updated weights for policy 1, policy_version 1001017 (0.0010) [2023-12-26 22:39:33,512][105620] Updated weights for policy 1, policy_version 1001027 (0.0010) [2023-12-26 22:39:33,691][105692] Updated weights for policy 0, policy_version 1000522 (0.0005) [2023-12-26 22:39:33,749][105692] Updated weights for policy 0, policy_version 1000532 (0.0006) [2023-12-26 22:39:33,811][105692] Updated weights for policy 0, policy_version 1000542 (0.0010) [2023-12-26 22:39:33,865][105692] Updated weights for policy 0, policy_version 1000552 (0.0010) [2023-12-26 22:39:34,206][105620] Updated weights for policy 1, policy_version 1001037 (0.0011) [2023-12-26 22:39:34,272][105620] Updated weights for policy 1, policy_version 1001047 (0.0010) [2023-12-26 22:39:34,335][105620] Updated weights for policy 1, policy_version 1001057 (0.0010) [2023-12-26 22:39:34,467][105692] Updated weights for policy 0, policy_version 1000562 (0.0011) [2023-12-26 22:39:34,530][105692] Updated weights for policy 0, policy_version 1000572 (0.0011) [2023-12-26 22:39:34,593][105692] Updated weights for policy 0, policy_version 1000582 (0.0010) [2023-12-26 22:39:35,065][105620] Updated weights for policy 1, policy_version 1001067 (0.0011) [2023-12-26 22:39:35,125][105620] Updated weights for policy 1, policy_version 1001077 (0.0010) [2023-12-26 22:39:35,166][105692] Updated weights for policy 0, policy_version 1000592 (0.0008) [2023-12-26 22:39:35,188][105620] Updated weights for policy 1, policy_version 1001087 (0.0010) [2023-12-26 22:39:35,221][105692] Updated weights for policy 0, policy_version 1000602 (0.0005) [2023-12-26 22:39:35,286][105692] Updated weights for policy 0, policy_version 1000612 (0.0007) [2023-12-26 22:39:35,780][105620] Updated weights for policy 1, policy_version 1001097 (0.0010) [2023-12-26 22:39:35,846][105620] Updated weights for policy 1, policy_version 1001107 (0.0006) [2023-12-26 22:39:35,880][105692] Updated weights for policy 0, policy_version 1000622 (0.0006) [2023-12-26 22:39:35,903][105620] Updated weights for policy 1, policy_version 1001117 (0.0010) [2023-12-26 22:39:35,925][105692] Updated weights for policy 0, policy_version 1000632 (0.0005) [2023-12-26 22:39:35,951][105620] Updated weights for policy 1, policy_version 1001127 (0.0009) [2023-12-26 22:39:35,970][105692] Updated weights for policy 0, policy_version 1000642 (0.0005) [2023-12-26 22:39:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.4, 300 sec: 19522.0). Total num frames: 512524288. Throughput: 0: 9528.2, 1: 9849.6. Samples: 512503820. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:39:36,063][104569] Avg episode reward: [(0, '8457.731'), (1, '9173.832')] [2023-12-26 22:39:36,546][105620] Updated weights for policy 1, policy_version 1001137 (0.0007) [2023-12-26 22:39:36,608][105620] Updated weights for policy 1, policy_version 1001147 (0.0010) [2023-12-26 22:39:36,671][105620] Updated weights for policy 1, policy_version 1001157 (0.0010) [2023-12-26 22:39:36,718][105692] Updated weights for policy 0, policy_version 1000652 (0.0007) [2023-12-26 22:39:36,781][105692] Updated weights for policy 0, policy_version 1000662 (0.0008) [2023-12-26 22:39:36,835][105692] Updated weights for policy 0, policy_version 1000672 (0.0008) [2023-12-26 22:39:37,400][105620] Updated weights for policy 1, policy_version 1001167 (0.0010) [2023-12-26 22:39:37,462][105620] Updated weights for policy 1, policy_version 1001177 (0.0010) [2023-12-26 22:39:37,520][105620] Updated weights for policy 1, policy_version 1001187 (0.0010) [2023-12-26 22:39:37,604][105692] Updated weights for policy 0, policy_version 1000682 (0.0008) [2023-12-26 22:39:37,661][105692] Updated weights for policy 0, policy_version 1000692 (0.0008) [2023-12-26 22:39:37,722][105692] Updated weights for policy 0, policy_version 1000702 (0.0009) [2023-12-26 22:39:37,783][105692] Updated weights for policy 0, policy_version 1000712 (0.0008) [2023-12-26 22:39:38,264][105620] Updated weights for policy 1, policy_version 1001197 (0.0008) [2023-12-26 22:39:38,330][105620] Updated weights for policy 1, policy_version 1001207 (0.0011) [2023-12-26 22:39:38,393][105620] Updated weights for policy 1, policy_version 1001217 (0.0010) [2023-12-26 22:39:38,580][105692] Updated weights for policy 0, policy_version 1000722 (0.0006) [2023-12-26 22:39:38,642][105692] Updated weights for policy 0, policy_version 1000732 (0.0008) [2023-12-26 22:39:38,708][105692] Updated weights for policy 0, policy_version 1000742 (0.0009) [2023-12-26 22:39:39,094][105620] Updated weights for policy 1, policy_version 1001227 (0.0007) [2023-12-26 22:39:39,148][105620] Updated weights for policy 1, policy_version 1001237 (0.0007) [2023-12-26 22:39:39,209][105620] Updated weights for policy 1, policy_version 1001247 (0.0006) [2023-12-26 22:39:39,477][105692] Updated weights for policy 0, policy_version 1000752 (0.0008) [2023-12-26 22:39:39,543][105692] Updated weights for policy 0, policy_version 1000762 (0.0008) [2023-12-26 22:39:39,604][105692] Updated weights for policy 0, policy_version 1000772 (0.0008) [2023-12-26 22:39:39,917][105620] Updated weights for policy 1, policy_version 1001257 (0.0008) [2023-12-26 22:39:39,986][105620] Updated weights for policy 1, policy_version 1001267 (0.0008) [2023-12-26 22:39:40,045][105620] Updated weights for policy 1, policy_version 1001277 (0.0008) [2023-12-26 22:39:40,111][105620] Updated weights for policy 1, policy_version 1001287 (0.0008) [2023-12-26 22:39:40,344][105692] Updated weights for policy 0, policy_version 1000782 (0.0008) [2023-12-26 22:39:40,410][105692] Updated weights for policy 0, policy_version 1000792 (0.0006) [2023-12-26 22:39:40,474][105692] Updated weights for policy 0, policy_version 1000802 (0.0008) [2023-12-26 22:39:40,927][105620] Updated weights for policy 1, policy_version 1001297 (0.0009) [2023-12-26 22:39:40,980][105620] Updated weights for policy 1, policy_version 1001307 (0.0009) [2023-12-26 22:39:41,046][105620] Updated weights for policy 1, policy_version 1001317 (0.0009) [2023-12-26 22:39:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 512606208. Throughput: 0: 9505.2, 1: 9890.6. Samples: 512620924. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-12-26 22:39:41,062][104569] Avg episode reward: [(0, '8906.312'), (1, '9173.780')] [2023-12-26 22:39:41,106][105692] Updated weights for policy 0, policy_version 1000812 (0.0007) [2023-12-26 22:39:41,176][105692] Updated weights for policy 0, policy_version 1000822 (0.0009) [2023-12-26 22:39:41,238][105692] Updated weights for policy 0, policy_version 1000832 (0.0009) [2023-12-26 22:39:41,858][105620] Updated weights for policy 1, policy_version 1001327 (0.0008) [2023-12-26 22:39:41,915][105620] Updated weights for policy 1, policy_version 1001337 (0.0009) [2023-12-26 22:39:41,977][105620] Updated weights for policy 1, policy_version 1001347 (0.0009) [2023-12-26 22:39:41,990][105692] Updated weights for policy 0, policy_version 1000842 (0.0009) [2023-12-26 22:39:42,051][105692] Updated weights for policy 0, policy_version 1000852 (0.0007) [2023-12-26 22:39:42,117][105692] Updated weights for policy 0, policy_version 1000862 (0.0007) [2023-12-26 22:39:42,190][105692] Updated weights for policy 0, policy_version 1000872 (0.0005) [2023-12-26 22:39:42,729][105620] Updated weights for policy 1, policy_version 1001357 (0.0007) [2023-12-26 22:39:42,790][105620] Updated weights for policy 1, policy_version 1001367 (0.0005) [2023-12-26 22:39:42,849][105620] Updated weights for policy 1, policy_version 1001377 (0.0008) [2023-12-26 22:39:42,933][105692] Updated weights for policy 0, policy_version 1000882 (0.0008) [2023-12-26 22:39:42,986][105692] Updated weights for policy 0, policy_version 1000892 (0.0009) [2023-12-26 22:39:43,040][105692] Updated weights for policy 0, policy_version 1000902 (0.0010) [2023-12-26 22:39:43,405][105620] Updated weights for policy 1, policy_version 1001387 (0.0007) [2023-12-26 22:39:43,469][105620] Updated weights for policy 1, policy_version 1001397 (0.0005) [2023-12-26 22:39:43,530][105620] Updated weights for policy 1, policy_version 1001407 (0.0006) [2023-12-26 22:39:43,734][105692] Updated weights for policy 0, policy_version 1000912 (0.0011) [2023-12-26 22:39:43,793][105692] Updated weights for policy 0, policy_version 1000922 (0.0011) [2023-12-26 22:39:43,855][105692] Updated weights for policy 0, policy_version 1000932 (0.0011) [2023-12-26 22:39:44,241][105620] Updated weights for policy 1, policy_version 1001417 (0.0008) [2023-12-26 22:39:44,293][105620] Updated weights for policy 1, policy_version 1001427 (0.0010) [2023-12-26 22:39:44,344][105620] Updated weights for policy 1, policy_version 1001437 (0.0008) [2023-12-26 22:39:44,397][105620] Updated weights for policy 1, policy_version 1001447 (0.0008) [2023-12-26 22:39:44,520][105692] Updated weights for policy 0, policy_version 1000942 (0.0010) [2023-12-26 22:39:44,578][105692] Updated weights for policy 0, policy_version 1000952 (0.0010) [2023-12-26 22:39:44,637][105692] Updated weights for policy 0, policy_version 1000962 (0.0010) [2023-12-26 22:39:45,129][105620] Updated weights for policy 1, policy_version 1001457 (0.0009) [2023-12-26 22:39:45,187][105620] Updated weights for policy 1, policy_version 1001467 (0.0009) [2023-12-26 22:39:45,245][105620] Updated weights for policy 1, policy_version 1001477 (0.0009) [2023-12-26 22:39:45,408][105692] Updated weights for policy 0, policy_version 1000973 (0.0010) [2023-12-26 22:39:45,471][105692] Updated weights for policy 0, policy_version 1000983 (0.0010) [2023-12-26 22:39:45,529][105692] Updated weights for policy 0, policy_version 1000993 (0.0009) [2023-12-26 22:39:45,941][105620] Updated weights for policy 1, policy_version 1001487 (0.0009) [2023-12-26 22:39:45,999][105620] Updated weights for policy 1, policy_version 1001497 (0.0009) [2023-12-26 22:39:46,056][105620] Updated weights for policy 1, policy_version 1001507 (0.0009) [2023-12-26 22:39:46,062][104569] Fps is (10 sec: 18021.8, 60 sec: 19387.6, 300 sec: 19438.6). Total num frames: 512704512. Throughput: 0: 9486.7, 1: 9953.7. Samples: 512679832. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:39:46,063][104569] Avg episode reward: [(0, '8631.365'), (1, '9265.423')] [2023-12-26 22:39:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001001000_256294912.pth... [2023-12-26 22:39:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_000999912_256016384.pth [2023-12-26 22:39:46,080][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001001512_256417792.pth... [2023-12-26 22:39:46,084][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001000360_256122880.pth [2023-12-26 22:39:46,295][105692] Updated weights for policy 0, policy_version 1001003 (0.0009) [2023-12-26 22:39:46,358][105692] Updated weights for policy 0, policy_version 1001013 (0.0009) [2023-12-26 22:39:46,411][105692] Updated weights for policy 0, policy_version 1001023 (0.0009) [2023-12-26 22:39:46,803][105620] Updated weights for policy 1, policy_version 1001517 (0.0008) [2023-12-26 22:39:46,849][105620] Updated weights for policy 1, policy_version 1001527 (0.0008) [2023-12-26 22:39:46,901][105620] Updated weights for policy 1, policy_version 1001537 (0.0009) [2023-12-26 22:39:47,166][105692] Updated weights for policy 0, policy_version 1001033 (0.0009) [2023-12-26 22:39:47,221][105692] Updated weights for policy 0, policy_version 1001043 (0.0009) [2023-12-26 22:39:47,280][105692] Updated weights for policy 0, policy_version 1001053 (0.0009) [2023-12-26 22:39:47,342][105692] Updated weights for policy 0, policy_version 1001063 (0.0007) [2023-12-26 22:39:47,747][105620] Updated weights for policy 1, policy_version 1001548 (0.0009) [2023-12-26 22:39:47,798][105620] Updated weights for policy 1, policy_version 1001558 (0.0009) [2023-12-26 22:39:47,859][105620] Updated weights for policy 1, policy_version 1001568 (0.0009) [2023-12-26 22:39:47,913][105692] Updated weights for policy 0, policy_version 1001073 (0.0007) [2023-12-26 22:39:47,957][105692] Updated weights for policy 0, policy_version 1001083 (0.0005) [2023-12-26 22:39:48,008][105692] Updated weights for policy 0, policy_version 1001093 (0.0005) [2023-12-26 22:39:48,697][105692] Updated weights for policy 0, policy_version 1001103 (0.0008) [2023-12-26 22:39:48,721][105620] Updated weights for policy 1, policy_version 1001578 (0.0007) [2023-12-26 22:39:48,752][105692] Updated weights for policy 0, policy_version 1001113 (0.0006) [2023-12-26 22:39:48,778][105620] Updated weights for policy 1, policy_version 1001588 (0.0008) [2023-12-26 22:39:48,809][105692] Updated weights for policy 0, policy_version 1001123 (0.0007) [2023-12-26 22:39:48,832][105620] Updated weights for policy 1, policy_version 1001598 (0.0007) [2023-12-26 22:39:48,878][105620] Updated weights for policy 1, policy_version 1001608 (0.0008) [2023-12-26 22:39:49,463][105692] Updated weights for policy 0, policy_version 1001133 (0.0006) [2023-12-26 22:39:49,526][105692] Updated weights for policy 0, policy_version 1001143 (0.0009) [2023-12-26 22:39:49,586][105692] Updated weights for policy 0, policy_version 1001153 (0.0009) [2023-12-26 22:39:49,715][105620] Updated weights for policy 1, policy_version 1001618 (0.0009) [2023-12-26 22:39:49,777][105620] Updated weights for policy 1, policy_version 1001628 (0.0008) [2023-12-26 22:39:49,840][105620] Updated weights for policy 1, policy_version 1001638 (0.0009) [2023-12-26 22:39:50,264][105692] Updated weights for policy 0, policy_version 1001163 (0.0008) [2023-12-26 22:39:50,322][105692] Updated weights for policy 0, policy_version 1001173 (0.0009) [2023-12-26 22:39:50,385][105692] Updated weights for policy 0, policy_version 1001183 (0.0008) [2023-12-26 22:39:50,620][105620] Updated weights for policy 1, policy_version 1001648 (0.0009) [2023-12-26 22:39:50,669][105620] Updated weights for policy 1, policy_version 1001658 (0.0008) [2023-12-26 22:39:50,731][105620] Updated weights for policy 1, policy_version 1001668 (0.0009) [2023-12-26 22:39:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 512802816. Throughput: 0: 9500.6, 1: 9795.0. Samples: 512793840. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:39:51,063][104569] Avg episode reward: [(0, '8360.102'), (1, '9264.165')] [2023-12-26 22:39:51,091][105692] Updated weights for policy 0, policy_version 1001193 (0.0008) [2023-12-26 22:39:51,162][105692] Updated weights for policy 0, policy_version 1001203 (0.0008) [2023-12-26 22:39:51,226][105692] Updated weights for policy 0, policy_version 1001213 (0.0008) [2023-12-26 22:39:51,289][105692] Updated weights for policy 0, policy_version 1001223 (0.0008) [2023-12-26 22:39:51,583][105620] Updated weights for policy 1, policy_version 1001678 (0.0010) [2023-12-26 22:39:51,660][105620] Updated weights for policy 1, policy_version 1001688 (0.0009) [2023-12-26 22:39:51,721][105620] Updated weights for policy 1, policy_version 1001698 (0.0010) [2023-12-26 22:39:51,995][105692] Updated weights for policy 0, policy_version 1001233 (0.0008) [2023-12-26 22:39:52,053][105692] Updated weights for policy 0, policy_version 1001243 (0.0006) [2023-12-26 22:39:52,115][105692] Updated weights for policy 0, policy_version 1001253 (0.0007) [2023-12-26 22:39:52,475][105620] Updated weights for policy 1, policy_version 1001708 (0.0011) [2023-12-26 22:39:52,544][105620] Updated weights for policy 1, policy_version 1001718 (0.0011) [2023-12-26 22:39:52,608][105620] Updated weights for policy 1, policy_version 1001728 (0.0008) [2023-12-26 22:39:52,804][105692] Updated weights for policy 0, policy_version 1001263 (0.0008) [2023-12-26 22:39:52,862][105692] Updated weights for policy 0, policy_version 1001273 (0.0005) [2023-12-26 22:39:52,917][105692] Updated weights for policy 0, policy_version 1001283 (0.0005) [2023-12-26 22:39:53,326][105620] Updated weights for policy 1, policy_version 1001738 (0.0010) [2023-12-26 22:39:53,371][105620] Updated weights for policy 1, policy_version 1001748 (0.0010) [2023-12-26 22:39:53,426][105620] Updated weights for policy 1, policy_version 1001758 (0.0010) [2023-12-26 22:39:53,457][105692] Updated weights for policy 0, policy_version 1001293 (0.0005) [2023-12-26 22:39:53,471][105620] Updated weights for policy 1, policy_version 1001768 (0.0010) [2023-12-26 22:39:53,520][105692] Updated weights for policy 0, policy_version 1001303 (0.0005) [2023-12-26 22:39:53,585][105692] Updated weights for policy 0, policy_version 1001313 (0.0005) [2023-12-26 22:39:54,131][105620] Updated weights for policy 1, policy_version 1001778 (0.0008) [2023-12-26 22:39:54,180][105620] Updated weights for policy 1, policy_version 1001788 (0.0009) [2023-12-26 22:39:54,229][105692] Updated weights for policy 0, policy_version 1001323 (0.0006) [2023-12-26 22:39:54,232][105620] Updated weights for policy 1, policy_version 1001798 (0.0008) [2023-12-26 22:39:54,281][105692] Updated weights for policy 0, policy_version 1001333 (0.0005) [2023-12-26 22:39:54,338][105692] Updated weights for policy 0, policy_version 1001343 (0.0005) [2023-12-26 22:39:55,002][105692] Updated weights for policy 0, policy_version 1001353 (0.0006) [2023-12-26 22:39:55,022][105620] Updated weights for policy 1, policy_version 1001808 (0.0010) [2023-12-26 22:39:55,061][105692] Updated weights for policy 0, policy_version 1001363 (0.0006) [2023-12-26 22:39:55,085][105620] Updated weights for policy 1, policy_version 1001818 (0.0010) [2023-12-26 22:39:55,119][105692] Updated weights for policy 0, policy_version 1001373 (0.0008) [2023-12-26 22:39:55,148][105620] Updated weights for policy 1, policy_version 1001828 (0.0010) [2023-12-26 22:39:55,175][105692] Updated weights for policy 0, policy_version 1001383 (0.0008) [2023-12-26 22:39:55,825][105620] Updated weights for policy 1, policy_version 1001838 (0.0011) [2023-12-26 22:39:55,893][105620] Updated weights for policy 1, policy_version 1001848 (0.0011) [2023-12-26 22:39:55,952][105620] Updated weights for policy 1, policy_version 1001858 (0.0010) [2023-12-26 22:39:55,987][105692] Updated weights for policy 0, policy_version 1001393 (0.0007) [2023-12-26 22:39:56,054][105692] Updated weights for policy 0, policy_version 1001403 (0.0009) [2023-12-26 22:39:56,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 512901120. Throughput: 0: 9607.4, 1: 9722.3. Samples: 512910684. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:39:56,063][104569] Avg episode reward: [(0, '8811.535'), (1, '5563.093')] [2023-12-26 22:39:56,116][105692] Updated weights for policy 0, policy_version 1001413 (0.0008) [2023-12-26 22:39:56,594][105620] Updated weights for policy 1, policy_version 1001868 (0.0006) [2023-12-26 22:39:56,645][105620] Updated weights for policy 1, policy_version 1001878 (0.0005) [2023-12-26 22:39:56,695][105620] Updated weights for policy 1, policy_version 1001888 (0.0008) [2023-12-26 22:39:56,898][105692] Updated weights for policy 0, policy_version 1001423 (0.0006) [2023-12-26 22:39:56,947][105692] Updated weights for policy 0, policy_version 1001433 (0.0005) [2023-12-26 22:39:56,991][105692] Updated weights for policy 0, policy_version 1001443 (0.0005) [2023-12-26 22:39:57,336][105620] Updated weights for policy 1, policy_version 1001898 (0.0010) [2023-12-26 22:39:57,394][105620] Updated weights for policy 1, policy_version 1001908 (0.0008) [2023-12-26 22:39:57,451][105586] KL-divergence is very high: 106.5347 [2023-12-26 22:39:57,456][105620] Updated weights for policy 1, policy_version 1001918 (0.0008) [2023-12-26 22:39:57,457][105586] KL-divergence is very high: 112.1276 [2023-12-26 22:39:57,478][105586] KL-divergence is very high: 115.9670 [2023-12-26 22:39:57,483][105586] KL-divergence is very high: 110.5669 [2023-12-26 22:39:57,488][105586] KL-divergence is very high: 102.1616 [2023-12-26 22:39:57,494][105586] KL-divergence is very high: 103.6231 [2023-12-26 22:39:57,511][105620] Updated weights for policy 1, policy_version 1001928 (0.0007) [2023-12-26 22:39:57,615][105692] Updated weights for policy 0, policy_version 1001453 (0.0008) [2023-12-26 22:39:57,659][105692] Updated weights for policy 0, policy_version 1001463 (0.0010) [2023-12-26 22:39:57,710][105692] Updated weights for policy 0, policy_version 1001473 (0.0010) [2023-12-26 22:39:58,222][105620] Updated weights for policy 1, policy_version 1001938 (0.0008) [2023-12-26 22:39:58,257][105586] KL-divergence is very high: 102.1068 [2023-12-26 22:39:58,285][105620] Updated weights for policy 1, policy_version 1001948 (0.0008) [2023-12-26 22:39:58,342][105620] Updated weights for policy 1, policy_version 1001958 (0.0007) [2023-12-26 22:39:58,444][105692] Updated weights for policy 0, policy_version 1001483 (0.0010) [2023-12-26 22:39:58,494][105692] Updated weights for policy 0, policy_version 1001493 (0.0011) [2023-12-26 22:39:58,552][105692] Updated weights for policy 0, policy_version 1001503 (0.0010) [2023-12-26 22:39:59,105][105620] Updated weights for policy 1, policy_version 1001968 (0.0010) [2023-12-26 22:39:59,155][105620] Updated weights for policy 1, policy_version 1001978 (0.0010) [2023-12-26 22:39:59,219][105620] Updated weights for policy 1, policy_version 1001988 (0.0010) [2023-12-26 22:39:59,306][105692] Updated weights for policy 0, policy_version 1001513 (0.0010) [2023-12-26 22:39:59,372][105692] Updated weights for policy 0, policy_version 1001523 (0.0009) [2023-12-26 22:39:59,431][105692] Updated weights for policy 0, policy_version 1001533 (0.0008) [2023-12-26 22:39:59,495][105692] Updated weights for policy 0, policy_version 1001543 (0.0008) [2023-12-26 22:39:59,987][105620] Updated weights for policy 1, policy_version 1001998 (0.0007) [2023-12-26 22:40:00,052][105620] Updated weights for policy 1, policy_version 1002008 (0.0006) [2023-12-26 22:40:00,117][105620] Updated weights for policy 1, policy_version 1002018 (0.0006) [2023-12-26 22:40:00,214][105692] Updated weights for policy 0, policy_version 1001553 (0.0008) [2023-12-26 22:40:00,265][105692] Updated weights for policy 0, policy_version 1001563 (0.0009) [2023-12-26 22:40:00,319][105692] Updated weights for policy 0, policy_version 1001573 (0.0010) [2023-12-26 22:40:00,723][105620] Updated weights for policy 1, policy_version 1002028 (0.0006) [2023-12-26 22:40:00,767][105620] Updated weights for policy 1, policy_version 1002038 (0.0005) [2023-12-26 22:40:00,823][105620] Updated weights for policy 1, policy_version 1002048 (0.0008) [2023-12-26 22:40:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 512999424. Throughput: 0: 9655.2, 1: 9769.2. Samples: 512969792. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:40:01,062][104569] Avg episode reward: [(0, '8812.921'), (1, '2101.897')] [2023-12-26 22:40:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001001576_256442368.pth... [2023-12-26 22:40:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001002056_256557056.pth... [2023-12-26 22:40:01,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001000936_256270336.pth [2023-12-26 22:40:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001000424_256147456.pth [2023-12-26 22:40:01,168][105692] Updated weights for policy 0, policy_version 1001583 (0.0008) [2023-12-26 22:40:01,228][105692] Updated weights for policy 0, policy_version 1001593 (0.0008) [2023-12-26 22:40:01,291][105692] Updated weights for policy 0, policy_version 1001603 (0.0009) [2023-12-26 22:40:01,505][105620] Updated weights for policy 1, policy_version 1002058 (0.0009) [2023-12-26 22:40:01,565][105620] Updated weights for policy 1, policy_version 1002068 (0.0011) [2023-12-26 22:40:01,630][105620] Updated weights for policy 1, policy_version 1002078 (0.0011) [2023-12-26 22:40:01,690][105620] Updated weights for policy 1, policy_version 1002088 (0.0010) [2023-12-26 22:40:02,005][105692] Updated weights for policy 0, policy_version 1001613 (0.0009) [2023-12-26 22:40:02,050][105692] Updated weights for policy 0, policy_version 1001623 (0.0008) [2023-12-26 22:40:02,103][105692] Updated weights for policy 0, policy_version 1001634 (0.0010) [2023-12-26 22:40:02,412][105620] Updated weights for policy 1, policy_version 1002098 (0.0011) [2023-12-26 22:40:02,479][105620] Updated weights for policy 1, policy_version 1002108 (0.0011) [2023-12-26 22:40:02,544][105620] Updated weights for policy 1, policy_version 1002118 (0.0011) [2023-12-26 22:40:02,869][105692] Updated weights for policy 0, policy_version 1001644 (0.0008) [2023-12-26 22:40:02,915][105692] Updated weights for policy 0, policy_version 1001654 (0.0007) [2023-12-26 22:40:02,966][105692] Updated weights for policy 0, policy_version 1001664 (0.0007) [2023-12-26 22:40:03,244][105620] Updated weights for policy 1, policy_version 1002128 (0.0010) [2023-12-26 22:40:03,291][105620] Updated weights for policy 1, policy_version 1002138 (0.0010) [2023-12-26 22:40:03,349][105620] Updated weights for policy 1, policy_version 1002148 (0.0010) [2023-12-26 22:40:03,637][105692] Updated weights for policy 0, policy_version 1001674 (0.0007) [2023-12-26 22:40:03,691][105692] Updated weights for policy 0, policy_version 1001684 (0.0005) [2023-12-26 22:40:03,746][105692] Updated weights for policy 0, policy_version 1001694 (0.0005) [2023-12-26 22:40:03,802][105692] Updated weights for policy 0, policy_version 1001704 (0.0005) [2023-12-26 22:40:04,129][105620] Updated weights for policy 1, policy_version 1002158 (0.0011) [2023-12-26 22:40:04,193][105620] Updated weights for policy 1, policy_version 1002168 (0.0011) [2023-12-26 22:40:04,258][105620] Updated weights for policy 1, policy_version 1002178 (0.0010) [2023-12-26 22:40:04,415][105692] Updated weights for policy 0, policy_version 1001714 (0.0008) [2023-12-26 22:40:04,482][105692] Updated weights for policy 0, policy_version 1001724 (0.0011) [2023-12-26 22:40:04,537][105692] Updated weights for policy 0, policy_version 1001734 (0.0010) [2023-12-26 22:40:04,994][105620] Updated weights for policy 1, policy_version 1002188 (0.0011) [2023-12-26 22:40:05,050][105620] Updated weights for policy 1, policy_version 1002198 (0.0011) [2023-12-26 22:40:05,114][105620] Updated weights for policy 1, policy_version 1002208 (0.0010) [2023-12-26 22:40:05,185][105692] Updated weights for policy 0, policy_version 1001744 (0.0008) [2023-12-26 22:40:05,230][105692] Updated weights for policy 0, policy_version 1001754 (0.0008) [2023-12-26 22:40:05,249][105585] KL-divergence is very high: 121.2487 [2023-12-26 22:40:05,276][105585] KL-divergence is very high: 101.7640 [2023-12-26 22:40:05,277][105692] Updated weights for policy 0, policy_version 1001764 (0.0008) [2023-12-26 22:40:05,286][105585] KL-divergence is very high: 129.8817 [2023-12-26 22:40:05,854][105620] Updated weights for policy 1, policy_version 1002218 (0.0010) [2023-12-26 22:40:05,917][105620] Updated weights for policy 1, policy_version 1002228 (0.0006) [2023-12-26 22:40:05,977][105620] Updated weights for policy 1, policy_version 1002238 (0.0005) [2023-12-26 22:40:06,014][105692] Updated weights for policy 0, policy_version 1001774 (0.0007) [2023-12-26 22:40:06,032][105620] Updated weights for policy 1, policy_version 1002248 (0.0005) [2023-12-26 22:40:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 513097728. Throughput: 0: 9701.6, 1: 9671.5. Samples: 513086312. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:40:06,062][104569] Avg episode reward: [(0, '8451.407'), (1, '2447.663')] [2023-12-26 22:40:06,085][105692] Updated weights for policy 0, policy_version 1001784 (0.0005) [2023-12-26 22:40:06,146][105692] Updated weights for policy 0, policy_version 1001794 (0.0008) [2023-12-26 22:40:06,708][105620] Updated weights for policy 1, policy_version 1002258 (0.0010) [2023-12-26 22:40:06,766][105620] Updated weights for policy 1, policy_version 1002268 (0.0010) [2023-12-26 22:40:06,822][105692] Updated weights for policy 0, policy_version 1001804 (0.0007) [2023-12-26 22:40:06,824][105620] Updated weights for policy 1, policy_version 1002278 (0.0010) [2023-12-26 22:40:06,877][105692] Updated weights for policy 0, policy_version 1001814 (0.0007) [2023-12-26 22:40:06,938][105692] Updated weights for policy 0, policy_version 1001824 (0.0008) [2023-12-26 22:40:07,567][105620] Updated weights for policy 1, policy_version 1002288 (0.0010) [2023-12-26 22:40:07,635][105620] Updated weights for policy 1, policy_version 1002298 (0.0010) [2023-12-26 22:40:07,700][105620] Updated weights for policy 1, policy_version 1002308 (0.0008) [2023-12-26 22:40:07,715][105692] Updated weights for policy 0, policy_version 1001834 (0.0008) [2023-12-26 22:40:07,776][105692] Updated weights for policy 0, policy_version 1001844 (0.0008) [2023-12-26 22:40:07,839][105692] Updated weights for policy 0, policy_version 1001854 (0.0009) [2023-12-26 22:40:07,897][105692] Updated weights for policy 0, policy_version 1001864 (0.0009) [2023-12-26 22:40:08,442][105620] Updated weights for policy 1, policy_version 1002318 (0.0007) [2023-12-26 22:40:08,502][105620] Updated weights for policy 1, policy_version 1002328 (0.0006) [2023-12-26 22:40:08,562][105620] Updated weights for policy 1, policy_version 1002338 (0.0005) [2023-12-26 22:40:08,720][105692] Updated weights for policy 0, policy_version 1001874 (0.0009) [2023-12-26 22:40:08,785][105692] Updated weights for policy 0, policy_version 1001884 (0.0009) [2023-12-26 22:40:08,852][105692] Updated weights for policy 0, policy_version 1001894 (0.0010) [2023-12-26 22:40:09,185][105620] Updated weights for policy 1, policy_version 1002348 (0.0007) [2023-12-26 22:40:09,252][105620] Updated weights for policy 1, policy_version 1002358 (0.0009) [2023-12-26 22:40:09,314][105620] Updated weights for policy 1, policy_version 1002368 (0.0009) [2023-12-26 22:40:09,654][105692] Updated weights for policy 0, policy_version 1001904 (0.0009) [2023-12-26 22:40:09,717][105692] Updated weights for policy 0, policy_version 1001914 (0.0009) [2023-12-26 22:40:09,781][105692] Updated weights for policy 0, policy_version 1001924 (0.0009) [2023-12-26 22:40:10,082][105620] Updated weights for policy 1, policy_version 1002378 (0.0008) [2023-12-26 22:40:10,136][105620] Updated weights for policy 1, policy_version 1002388 (0.0008) [2023-12-26 22:40:10,201][105620] Updated weights for policy 1, policy_version 1002398 (0.0009) [2023-12-26 22:40:10,261][105620] Updated weights for policy 1, policy_version 1002408 (0.0008) [2023-12-26 22:40:10,578][105692] Updated weights for policy 0, policy_version 1001934 (0.0009) [2023-12-26 22:40:10,630][105692] Updated weights for policy 0, policy_version 1001944 (0.0010) [2023-12-26 22:40:10,683][105692] Updated weights for policy 0, policy_version 1001954 (0.0010) [2023-12-26 22:40:11,044][105620] Updated weights for policy 1, policy_version 1002418 (0.0009) [2023-12-26 22:40:11,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 513187840. Throughput: 0: 9705.0, 1: 9655.1. Samples: 513199452. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:40:11,063][104569] Avg episode reward: [(0, '8992.589'), (1, '6980.022')] [2023-12-26 22:40:11,109][105620] Updated weights for policy 1, policy_version 1002428 (0.0008) [2023-12-26 22:40:11,176][105620] Updated weights for policy 1, policy_version 1002438 (0.0008) [2023-12-26 22:40:11,450][105692] Updated weights for policy 0, policy_version 1001964 (0.0010) [2023-12-26 22:40:11,496][105692] Updated weights for policy 0, policy_version 1001974 (0.0010) [2023-12-26 22:40:11,545][105692] Updated weights for policy 0, policy_version 1001984 (0.0010) [2023-12-26 22:40:11,948][105620] Updated weights for policy 1, policy_version 1002448 (0.0008) [2023-12-26 22:40:12,001][105620] Updated weights for policy 1, policy_version 1002458 (0.0008) [2023-12-26 22:40:12,046][105620] Updated weights for policy 1, policy_version 1002468 (0.0008) [2023-12-26 22:40:12,316][105692] Updated weights for policy 0, policy_version 1001994 (0.0010) [2023-12-26 22:40:12,386][105692] Updated weights for policy 0, policy_version 1002004 (0.0009) [2023-12-26 22:40:12,448][105692] Updated weights for policy 0, policy_version 1002014 (0.0008) [2023-12-26 22:40:12,499][105692] Updated weights for policy 0, policy_version 1002024 (0.0008) [2023-12-26 22:40:12,892][105620] Updated weights for policy 1, policy_version 1002478 (0.0009) [2023-12-26 22:40:12,952][105620] Updated weights for policy 1, policy_version 1002488 (0.0006) [2023-12-26 22:40:13,007][105620] Updated weights for policy 1, policy_version 1002498 (0.0005) [2023-12-26 22:40:13,251][105692] Updated weights for policy 0, policy_version 1002034 (0.0010) [2023-12-26 22:40:13,307][105692] Updated weights for policy 0, policy_version 1002044 (0.0011) [2023-12-26 22:40:13,352][105692] Updated weights for policy 0, policy_version 1002054 (0.0010) [2023-12-26 22:40:13,609][105620] Updated weights for policy 1, policy_version 1002508 (0.0007) [2023-12-26 22:40:13,675][105620] Updated weights for policy 1, policy_version 1002518 (0.0007) [2023-12-26 22:40:13,722][105620] Updated weights for policy 1, policy_version 1002528 (0.0009) [2023-12-26 22:40:14,051][105692] Updated weights for policy 0, policy_version 1002064 (0.0010) [2023-12-26 22:40:14,120][105692] Updated weights for policy 0, policy_version 1002074 (0.0006) [2023-12-26 22:40:14,190][105692] Updated weights for policy 0, policy_version 1002084 (0.0005) [2023-12-26 22:40:14,421][105620] Updated weights for policy 1, policy_version 1002538 (0.0007) [2023-12-26 22:40:14,480][105620] Updated weights for policy 1, policy_version 1002548 (0.0010) [2023-12-26 22:40:14,539][105620] Updated weights for policy 1, policy_version 1002558 (0.0010) [2023-12-26 22:40:14,594][105620] Updated weights for policy 1, policy_version 1002568 (0.0010) [2023-12-26 22:40:14,791][105692] Updated weights for policy 0, policy_version 1002094 (0.0009) [2023-12-26 22:40:14,850][105692] Updated weights for policy 0, policy_version 1002104 (0.0010) [2023-12-26 22:40:14,916][105692] Updated weights for policy 0, policy_version 1002114 (0.0009) [2023-12-26 22:40:15,360][105620] Updated weights for policy 1, policy_version 1002578 (0.0011) [2023-12-26 22:40:15,419][105620] Updated weights for policy 1, policy_version 1002588 (0.0011) [2023-12-26 22:40:15,479][105620] Updated weights for policy 1, policy_version 1002598 (0.0011) [2023-12-26 22:40:15,515][105692] Updated weights for policy 0, policy_version 1002124 (0.0007) [2023-12-26 22:40:15,582][105692] Updated weights for policy 0, policy_version 1002134 (0.0011) [2023-12-26 22:40:15,670][105692] Updated weights for policy 0, policy_version 1002144 (0.0010) [2023-12-26 22:40:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 513286144. Throughput: 0: 9730.8, 1: 9605.3. Samples: 513256040. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:40:16,062][104569] Avg episode reward: [(0, '9086.587'), (1, '9355.278')] [2023-12-26 22:40:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001002152_256589824.pth... [2023-12-26 22:40:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001002600_256696320.pth... [2023-12-26 22:40:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001001512_256417792.pth [2023-12-26 22:40:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001001000_256294912.pth [2023-12-26 22:40:16,210][105620] Updated weights for policy 1, policy_version 1002608 (0.0010) [2023-12-26 22:40:16,264][105620] Updated weights for policy 1, policy_version 1002618 (0.0010) [2023-12-26 22:40:16,326][105620] Updated weights for policy 1, policy_version 1002628 (0.0010) [2023-12-26 22:40:16,343][105692] Updated weights for policy 0, policy_version 1002154 (0.0009) [2023-12-26 22:40:16,398][105692] Updated weights for policy 0, policy_version 1002164 (0.0006) [2023-12-26 22:40:16,456][105692] Updated weights for policy 0, policy_version 1002174 (0.0006) [2023-12-26 22:40:16,512][105692] Updated weights for policy 0, policy_version 1002184 (0.0006) [2023-12-26 22:40:16,949][105620] Updated weights for policy 1, policy_version 1002638 (0.0010) [2023-12-26 22:40:17,000][105620] Updated weights for policy 1, policy_version 1002648 (0.0010) [2023-12-26 22:40:17,063][105620] Updated weights for policy 1, policy_version 1002658 (0.0010) [2023-12-26 22:40:17,067][105692] Updated weights for policy 0, policy_version 1002194 (0.0008) [2023-12-26 22:40:17,133][105692] Updated weights for policy 0, policy_version 1002204 (0.0005) [2023-12-26 22:40:17,195][105692] Updated weights for policy 0, policy_version 1002214 (0.0005) [2023-12-26 22:40:17,742][105692] Updated weights for policy 0, policy_version 1002224 (0.0005) [2023-12-26 22:40:17,798][105692] Updated weights for policy 0, policy_version 1002234 (0.0005) [2023-12-26 22:40:17,800][105620] Updated weights for policy 1, policy_version 1002668 (0.0008) [2023-12-26 22:40:17,851][105620] Updated weights for policy 1, policy_version 1002678 (0.0006) [2023-12-26 22:40:17,860][105692] Updated weights for policy 0, policy_version 1002244 (0.0005) [2023-12-26 22:40:17,904][105620] Updated weights for policy 1, policy_version 1002688 (0.0010) [2023-12-26 22:40:18,492][105692] Updated weights for policy 0, policy_version 1002254 (0.0008) [2023-12-26 22:40:18,545][105692] Updated weights for policy 0, policy_version 1002264 (0.0009) [2023-12-26 22:40:18,593][105620] Updated weights for policy 1, policy_version 1002698 (0.0010) [2023-12-26 22:40:18,615][105692] Updated weights for policy 0, policy_version 1002274 (0.0008) [2023-12-26 22:40:18,655][105620] Updated weights for policy 1, policy_version 1002708 (0.0008) [2023-12-26 22:40:18,717][105620] Updated weights for policy 1, policy_version 1002718 (0.0010) [2023-12-26 22:40:18,777][105620] Updated weights for policy 1, policy_version 1002728 (0.0009) [2023-12-26 22:40:19,356][105692] Updated weights for policy 0, policy_version 1002284 (0.0009) [2023-12-26 22:40:19,416][105692] Updated weights for policy 0, policy_version 1002294 (0.0010) [2023-12-26 22:40:19,483][105692] Updated weights for policy 0, policy_version 1002304 (0.0011) [2023-12-26 22:40:19,587][105620] Updated weights for policy 1, policy_version 1002738 (0.0008) [2023-12-26 22:40:19,643][105620] Updated weights for policy 1, policy_version 1002748 (0.0008) [2023-12-26 22:40:19,704][105620] Updated weights for policy 1, policy_version 1002758 (0.0009) [2023-12-26 22:40:20,258][105692] Updated weights for policy 0, policy_version 1002314 (0.0010) [2023-12-26 22:40:20,310][105692] Updated weights for policy 0, policy_version 1002324 (0.0011) [2023-12-26 22:40:20,376][105692] Updated weights for policy 0, policy_version 1002334 (0.0010) [2023-12-26 22:40:20,439][105692] Updated weights for policy 0, policy_version 1002344 (0.0011) [2023-12-26 22:40:20,468][105620] Updated weights for policy 1, policy_version 1002768 (0.0008) [2023-12-26 22:40:20,524][105620] Updated weights for policy 1, policy_version 1002778 (0.0005) [2023-12-26 22:40:20,589][105620] Updated weights for policy 1, policy_version 1002788 (0.0009) [2023-12-26 22:40:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 513384448. Throughput: 0: 9841.2, 1: 9583.5. Samples: 513377932. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:40:21,062][104569] Avg episode reward: [(0, '8816.716'), (1, '9170.083')] [2023-12-26 22:40:21,196][105692] Updated weights for policy 0, policy_version 1002354 (0.0011) [2023-12-26 22:40:21,267][105692] Updated weights for policy 0, policy_version 1002364 (0.0010) [2023-12-26 22:40:21,307][105620] Updated weights for policy 1, policy_version 1002798 (0.0010) [2023-12-26 22:40:21,330][105692] Updated weights for policy 0, policy_version 1002374 (0.0008) [2023-12-26 22:40:21,379][105620] Updated weights for policy 1, policy_version 1002808 (0.0009) [2023-12-26 22:40:21,447][105620] Updated weights for policy 1, policy_version 1002818 (0.0008) [2023-12-26 22:40:22,095][105692] Updated weights for policy 0, policy_version 1002384 (0.0010) [2023-12-26 22:40:22,147][105692] Updated weights for policy 0, policy_version 1002394 (0.0010) [2023-12-26 22:40:22,195][105620] Updated weights for policy 1, policy_version 1002828 (0.0007) [2023-12-26 22:40:22,205][105692] Updated weights for policy 0, policy_version 1002404 (0.0008) [2023-12-26 22:40:22,255][105620] Updated weights for policy 1, policy_version 1002838 (0.0009) [2023-12-26 22:40:22,318][105620] Updated weights for policy 1, policy_version 1002848 (0.0009) [2023-12-26 22:40:22,945][105692] Updated weights for policy 0, policy_version 1002414 (0.0008) [2023-12-26 22:40:23,007][105692] Updated weights for policy 0, policy_version 1002424 (0.0008) [2023-12-26 22:40:23,075][105692] Updated weights for policy 0, policy_version 1002434 (0.0005) [2023-12-26 22:40:23,149][105620] Updated weights for policy 1, policy_version 1002858 (0.0009) [2023-12-26 22:40:23,207][105620] Updated weights for policy 1, policy_version 1002868 (0.0009) [2023-12-26 22:40:23,263][105620] Updated weights for policy 1, policy_version 1002878 (0.0009) [2023-12-26 22:40:23,320][105620] Updated weights for policy 1, policy_version 1002888 (0.0008) [2023-12-26 22:40:23,626][105692] Updated weights for policy 0, policy_version 1002444 (0.0005) [2023-12-26 22:40:23,683][105692] Updated weights for policy 0, policy_version 1002454 (0.0005) [2023-12-26 22:40:23,738][105692] Updated weights for policy 0, policy_version 1002464 (0.0005) [2023-12-26 22:40:24,206][105620] Updated weights for policy 1, policy_version 1002898 (0.0009) [2023-12-26 22:40:24,254][105620] Updated weights for policy 1, policy_version 1002908 (0.0007) [2023-12-26 22:40:24,310][105620] Updated weights for policy 1, policy_version 1002918 (0.0005) [2023-12-26 22:40:24,369][105692] Updated weights for policy 0, policy_version 1002474 (0.0006) [2023-12-26 22:40:24,422][105692] Updated weights for policy 0, policy_version 1002484 (0.0009) [2023-12-26 22:40:24,476][105692] Updated weights for policy 0, policy_version 1002494 (0.0009) [2023-12-26 22:40:24,523][105692] Updated weights for policy 0, policy_version 1002504 (0.0009) [2023-12-26 22:40:25,121][105620] Updated weights for policy 1, policy_version 1002928 (0.0005) [2023-12-26 22:40:25,125][105692] Updated weights for policy 0, policy_version 1002514 (0.0009) [2023-12-26 22:40:25,176][105692] Updated weights for policy 0, policy_version 1002524 (0.0007) [2023-12-26 22:40:25,181][105620] Updated weights for policy 1, policy_version 1002938 (0.0006) [2023-12-26 22:40:25,224][105692] Updated weights for policy 0, policy_version 1002534 (0.0006) [2023-12-26 22:40:25,231][105620] Updated weights for policy 1, policy_version 1002948 (0.0006) [2023-12-26 22:40:25,963][105692] Updated weights for policy 0, policy_version 1002544 (0.0007) [2023-12-26 22:40:25,970][105620] Updated weights for policy 1, policy_version 1002958 (0.0008) [2023-12-26 22:40:26,011][105692] Updated weights for policy 0, policy_version 1002554 (0.0007) [2023-12-26 22:40:26,033][105620] Updated weights for policy 1, policy_version 1002968 (0.0006) [2023-12-26 22:40:26,056][105692] Updated weights for policy 0, policy_version 1002564 (0.0006) [2023-12-26 22:40:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19438.7). Total num frames: 513474560. Throughput: 0: 9881.7, 1: 9476.9. Samples: 513492060. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:40:26,062][104569] Avg episode reward: [(0, '8904.488'), (1, '9077.906')] [2023-12-26 22:40:26,093][105620] Updated weights for policy 1, policy_version 1002978 (0.0008) [2023-12-26 22:40:26,821][105692] Updated weights for policy 0, policy_version 1002574 (0.0009) [2023-12-26 22:40:26,831][105620] Updated weights for policy 1, policy_version 1002988 (0.0008) [2023-12-26 22:40:26,880][105692] Updated weights for policy 0, policy_version 1002584 (0.0007) [2023-12-26 22:40:26,890][105620] Updated weights for policy 1, policy_version 1002998 (0.0006) [2023-12-26 22:40:26,932][105692] Updated weights for policy 0, policy_version 1002594 (0.0006) [2023-12-26 22:40:26,948][105620] Updated weights for policy 1, policy_version 1003008 (0.0007) [2023-12-26 22:40:27,590][105620] Updated weights for policy 1, policy_version 1003018 (0.0008) [2023-12-26 22:40:27,651][105620] Updated weights for policy 1, policy_version 1003028 (0.0009) [2023-12-26 22:40:27,685][105692] Updated weights for policy 0, policy_version 1002604 (0.0008) [2023-12-26 22:40:27,711][105620] Updated weights for policy 1, policy_version 1003038 (0.0006) [2023-12-26 22:40:27,744][105692] Updated weights for policy 0, policy_version 1002614 (0.0008) [2023-12-26 22:40:27,762][105620] Updated weights for policy 1, policy_version 1003048 (0.0007) [2023-12-26 22:40:27,800][105692] Updated weights for policy 0, policy_version 1002624 (0.0007) [2023-12-26 22:40:28,405][105620] Updated weights for policy 1, policy_version 1003058 (0.0009) [2023-12-26 22:40:28,470][105620] Updated weights for policy 1, policy_version 1003068 (0.0006) [2023-12-26 22:40:28,540][105620] Updated weights for policy 1, policy_version 1003078 (0.0006) [2023-12-26 22:40:28,599][105692] Updated weights for policy 0, policy_version 1002634 (0.0010) [2023-12-26 22:40:28,658][105692] Updated weights for policy 0, policy_version 1002644 (0.0006) [2023-12-26 22:40:28,716][105692] Updated weights for policy 0, policy_version 1002654 (0.0005) [2023-12-26 22:40:28,781][105692] Updated weights for policy 0, policy_version 1002664 (0.0005) [2023-12-26 22:40:29,226][105620] Updated weights for policy 1, policy_version 1003088 (0.0007) [2023-12-26 22:40:29,289][105620] Updated weights for policy 1, policy_version 1003098 (0.0007) [2023-12-26 22:40:29,353][105620] Updated weights for policy 1, policy_version 1003108 (0.0009) [2023-12-26 22:40:29,429][105692] Updated weights for policy 0, policy_version 1002674 (0.0009) [2023-12-26 22:40:29,487][105692] Updated weights for policy 0, policy_version 1002684 (0.0009) [2023-12-26 22:40:29,542][105692] Updated weights for policy 0, policy_version 1002694 (0.0009) [2023-12-26 22:40:30,092][105620] Updated weights for policy 1, policy_version 1003118 (0.0006) [2023-12-26 22:40:30,148][105620] Updated weights for policy 1, policy_version 1003128 (0.0005) [2023-12-26 22:40:30,208][105620] Updated weights for policy 1, policy_version 1003138 (0.0005) [2023-12-26 22:40:30,252][105692] Updated weights for policy 0, policy_version 1002704 (0.0007) [2023-12-26 22:40:30,309][105692] Updated weights for policy 0, policy_version 1002714 (0.0005) [2023-12-26 22:40:30,374][105692] Updated weights for policy 0, policy_version 1002724 (0.0005) [2023-12-26 22:40:30,845][105620] Updated weights for policy 1, policy_version 1003148 (0.0008) [2023-12-26 22:40:30,891][105620] Updated weights for policy 1, policy_version 1003158 (0.0008) [2023-12-26 22:40:30,944][105620] Updated weights for policy 1, policy_version 1003168 (0.0007) [2023-12-26 22:40:31,061][105692] Updated weights for policy 0, policy_version 1002734 (0.0007) [2023-12-26 22:40:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 513581056. Throughput: 0: 9854.1, 1: 9477.9. Samples: 513549768. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:40:31,062][104569] Avg episode reward: [(0, '8817.809'), (1, '8895.309')] [2023-12-26 22:40:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001003176_256843776.pth... [2023-12-26 22:40:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001002056_256557056.pth [2023-12-26 22:40:31,107][105692] Updated weights for policy 0, policy_version 1002744 (0.0008) [2023-12-26 22:40:31,167][105692] Updated weights for policy 0, policy_version 1002754 (0.0009) [2023-12-26 22:40:31,193][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001002760_256745472.pth... [2023-12-26 22:40:31,196][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001001576_256442368.pth [2023-12-26 22:40:31,713][105620] Updated weights for policy 1, policy_version 1003178 (0.0006) [2023-12-26 22:40:31,775][105620] Updated weights for policy 1, policy_version 1003188 (0.0009) [2023-12-26 22:40:31,829][105620] Updated weights for policy 1, policy_version 1003198 (0.0009) [2023-12-26 22:40:31,883][105620] Updated weights for policy 1, policy_version 1003208 (0.0009) [2023-12-26 22:40:31,936][105692] Updated weights for policy 0, policy_version 1002764 (0.0009) [2023-12-26 22:40:31,997][105692] Updated weights for policy 0, policy_version 1002774 (0.0008) [2023-12-26 22:40:32,058][105692] Updated weights for policy 0, policy_version 1002784 (0.0009) [2023-12-26 22:40:32,626][105620] Updated weights for policy 1, policy_version 1003218 (0.0008) [2023-12-26 22:40:32,687][105620] Updated weights for policy 1, policy_version 1003228 (0.0008) [2023-12-26 22:40:32,746][105620] Updated weights for policy 1, policy_version 1003238 (0.0008) [2023-12-26 22:40:32,848][105692] Updated weights for policy 0, policy_version 1002794 (0.0009) [2023-12-26 22:40:32,895][105692] Updated weights for policy 0, policy_version 1002804 (0.0010) [2023-12-26 22:40:32,939][105692] Updated weights for policy 0, policy_version 1002814 (0.0010) [2023-12-26 22:40:32,987][105692] Updated weights for policy 0, policy_version 1002824 (0.0010) [2023-12-26 22:40:33,506][105620] Updated weights for policy 1, policy_version 1003248 (0.0008) [2023-12-26 22:40:33,561][105620] Updated weights for policy 1, policy_version 1003258 (0.0008) [2023-12-26 22:40:33,620][105620] Updated weights for policy 1, policy_version 1003268 (0.0008) [2023-12-26 22:40:33,752][105692] Updated weights for policy 0, policy_version 1002834 (0.0010) [2023-12-26 22:40:33,797][105692] Updated weights for policy 0, policy_version 1002844 (0.0010) [2023-12-26 22:40:33,855][105692] Updated weights for policy 0, policy_version 1002854 (0.0010) [2023-12-26 22:40:34,413][105620] Updated weights for policy 1, policy_version 1003278 (0.0008) [2023-12-26 22:40:34,477][105620] Updated weights for policy 1, policy_version 1003288 (0.0008) [2023-12-26 22:40:34,533][105620] Updated weights for policy 1, policy_version 1003298 (0.0008) [2023-12-26 22:40:34,609][105692] Updated weights for policy 0, policy_version 1002864 (0.0010) [2023-12-26 22:40:34,671][105692] Updated weights for policy 0, policy_version 1002874 (0.0010) [2023-12-26 22:40:34,736][105692] Updated weights for policy 0, policy_version 1002884 (0.0010) [2023-12-26 22:40:35,349][105620] Updated weights for policy 1, policy_version 1003308 (0.0008) [2023-12-26 22:40:35,370][105692] Updated weights for policy 0, policy_version 1002894 (0.0007) [2023-12-26 22:40:35,397][105620] Updated weights for policy 1, policy_version 1003318 (0.0008) [2023-12-26 22:40:35,433][105692] Updated weights for policy 0, policy_version 1002904 (0.0006) [2023-12-26 22:40:35,450][105620] Updated weights for policy 1, policy_version 1003328 (0.0009) [2023-12-26 22:40:35,486][105692] Updated weights for policy 0, policy_version 1002914 (0.0008) [2023-12-26 22:40:36,017][105692] Updated weights for policy 0, policy_version 1002924 (0.0006) [2023-12-26 22:40:36,061][105692] Updated weights for policy 0, policy_version 1002934 (0.0005) [2023-12-26 22:40:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19114.6, 300 sec: 19438.6). Total num frames: 513671168. Throughput: 0: 9800.2, 1: 9555.3. Samples: 513664840. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:40:36,063][104569] Avg episode reward: [(0, '8724.701'), (1, '8894.206')] [2023-12-26 22:40:36,128][105692] Updated weights for policy 0, policy_version 1002944 (0.0009) [2023-12-26 22:40:36,321][105620] Updated weights for policy 1, policy_version 1003338 (0.0009) [2023-12-26 22:40:36,383][105620] Updated weights for policy 1, policy_version 1003348 (0.0010) [2023-12-26 22:40:36,444][105620] Updated weights for policy 1, policy_version 1003358 (0.0009) [2023-12-26 22:40:36,507][105620] Updated weights for policy 1, policy_version 1003368 (0.0009) [2023-12-26 22:40:36,755][105692] Updated weights for policy 0, policy_version 1002954 (0.0010) [2023-12-26 22:40:36,815][105692] Updated weights for policy 0, policy_version 1002964 (0.0007) [2023-12-26 22:40:36,872][105692] Updated weights for policy 0, policy_version 1002974 (0.0006) [2023-12-26 22:40:36,931][105692] Updated weights for policy 0, policy_version 1002984 (0.0006) [2023-12-26 22:40:37,350][105620] Updated weights for policy 1, policy_version 1003378 (0.0009) [2023-12-26 22:40:37,412][105620] Updated weights for policy 1, policy_version 1003388 (0.0009) [2023-12-26 22:40:37,477][105620] Updated weights for policy 1, policy_version 1003398 (0.0009) [2023-12-26 22:40:37,622][105692] Updated weights for policy 0, policy_version 1002994 (0.0008) [2023-12-26 22:40:37,676][105692] Updated weights for policy 0, policy_version 1003004 (0.0006) [2023-12-26 22:40:37,723][105692] Updated weights for policy 0, policy_version 1003014 (0.0005) [2023-12-26 22:40:38,270][105692] Updated weights for policy 0, policy_version 1003024 (0.0005) [2023-12-26 22:40:38,331][105692] Updated weights for policy 0, policy_version 1003034 (0.0009) [2023-12-26 22:40:38,353][105620] Updated weights for policy 1, policy_version 1003408 (0.0008) [2023-12-26 22:40:38,391][105692] Updated weights for policy 0, policy_version 1003044 (0.0011) [2023-12-26 22:40:38,406][105620] Updated weights for policy 1, policy_version 1003418 (0.0006) [2023-12-26 22:40:38,462][105620] Updated weights for policy 1, policy_version 1003428 (0.0009) [2023-12-26 22:40:39,115][105692] Updated weights for policy 0, policy_version 1003054 (0.0010) [2023-12-26 22:40:39,165][105620] Updated weights for policy 1, policy_version 1003438 (0.0007) [2023-12-26 22:40:39,170][105692] Updated weights for policy 0, policy_version 1003064 (0.0010) [2023-12-26 22:40:39,226][105620] Updated weights for policy 1, policy_version 1003448 (0.0007) [2023-12-26 22:40:39,226][105692] Updated weights for policy 0, policy_version 1003074 (0.0010) [2023-12-26 22:40:39,285][105620] Updated weights for policy 1, policy_version 1003458 (0.0009) [2023-12-26 22:40:40,003][105692] Updated weights for policy 0, policy_version 1003084 (0.0009) [2023-12-26 22:40:40,063][105692] Updated weights for policy 0, policy_version 1003094 (0.0006) [2023-12-26 22:40:40,102][105620] Updated weights for policy 1, policy_version 1003468 (0.0007) [2023-12-26 22:40:40,122][105692] Updated weights for policy 0, policy_version 1003104 (0.0009) [2023-12-26 22:40:40,161][105620] Updated weights for policy 1, policy_version 1003478 (0.0006) [2023-12-26 22:40:40,215][105620] Updated weights for policy 1, policy_version 1003488 (0.0008) [2023-12-26 22:40:40,798][105692] Updated weights for policy 0, policy_version 1003114 (0.0008) [2023-12-26 22:40:40,869][105692] Updated weights for policy 0, policy_version 1003124 (0.0008) [2023-12-26 22:40:40,937][105692] Updated weights for policy 0, policy_version 1003134 (0.0008) [2023-12-26 22:40:40,969][105620] Updated weights for policy 1, policy_version 1003498 (0.0006) [2023-12-26 22:40:41,005][105692] Updated weights for policy 0, policy_version 1003144 (0.0008) [2023-12-26 22:40:41,031][105620] Updated weights for policy 1, policy_version 1003508 (0.0008) [2023-12-26 22:40:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 513769472. Throughput: 0: 9850.6, 1: 9455.9. Samples: 513779476. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:40:41,062][104569] Avg episode reward: [(0, '8721.826'), (1, '8985.414')] [2023-12-26 22:40:41,091][105620] Updated weights for policy 1, policy_version 1003518 (0.0009) [2023-12-26 22:40:41,154][105620] Updated weights for policy 1, policy_version 1003528 (0.0009) [2023-12-26 22:40:41,686][105692] Updated weights for policy 0, policy_version 1003154 (0.0009) [2023-12-26 22:40:41,754][105692] Updated weights for policy 0, policy_version 1003164 (0.0009) [2023-12-26 22:40:41,817][105692] Updated weights for policy 0, policy_version 1003174 (0.0009) [2023-12-26 22:40:41,939][105620] Updated weights for policy 1, policy_version 1003538 (0.0010) [2023-12-26 22:40:42,009][105620] Updated weights for policy 1, policy_version 1003548 (0.0008) [2023-12-26 22:40:42,062][105620] Updated weights for policy 1, policy_version 1003558 (0.0010) [2023-12-26 22:40:42,555][105692] Updated weights for policy 0, policy_version 1003184 (0.0009) [2023-12-26 22:40:42,615][105692] Updated weights for policy 0, policy_version 1003194 (0.0009) [2023-12-26 22:40:42,674][105692] Updated weights for policy 0, policy_version 1003204 (0.0009) [2023-12-26 22:40:42,834][105620] Updated weights for policy 1, policy_version 1003568 (0.0009) [2023-12-26 22:40:42,881][105620] Updated weights for policy 1, policy_version 1003578 (0.0009) [2023-12-26 22:40:42,939][105620] Updated weights for policy 1, policy_version 1003588 (0.0008) [2023-12-26 22:40:43,343][105692] Updated weights for policy 0, policy_version 1003214 (0.0009) [2023-12-26 22:40:43,392][105692] Updated weights for policy 0, policy_version 1003225 (0.0009) [2023-12-26 22:40:43,440][105692] Updated weights for policy 0, policy_version 1003236 (0.0010) [2023-12-26 22:40:43,712][105620] Updated weights for policy 1, policy_version 1003598 (0.0007) [2023-12-26 22:40:43,771][105620] Updated weights for policy 1, policy_version 1003608 (0.0006) [2023-12-26 22:40:43,835][105620] Updated weights for policy 1, policy_version 1003618 (0.0010) [2023-12-26 22:40:44,150][105692] Updated weights for policy 0, policy_version 1003246 (0.0008) [2023-12-26 22:40:44,201][105692] Updated weights for policy 0, policy_version 1003256 (0.0009) [2023-12-26 22:40:44,253][105692] Updated weights for policy 0, policy_version 1003266 (0.0009) [2023-12-26 22:40:44,395][105620] Updated weights for policy 1, policy_version 1003628 (0.0009) [2023-12-26 22:40:44,461][105620] Updated weights for policy 1, policy_version 1003638 (0.0008) [2023-12-26 22:40:44,523][105620] Updated weights for policy 1, policy_version 1003648 (0.0010) [2023-12-26 22:40:45,072][105692] Updated weights for policy 0, policy_version 1003276 (0.0010) [2023-12-26 22:40:45,139][105692] Updated weights for policy 0, policy_version 1003286 (0.0006) [2023-12-26 22:40:45,211][105692] Updated weights for policy 0, policy_version 1003296 (0.0010) [2023-12-26 22:40:45,212][105620] Updated weights for policy 1, policy_version 1003658 (0.0010) [2023-12-26 22:40:45,277][105620] Updated weights for policy 1, policy_version 1003668 (0.0007) [2023-12-26 22:40:45,342][105620] Updated weights for policy 1, policy_version 1003678 (0.0010) [2023-12-26 22:40:45,403][105620] Updated weights for policy 1, policy_version 1003688 (0.0010) [2023-12-26 22:40:45,932][105692] Updated weights for policy 0, policy_version 1003306 (0.0009) [2023-12-26 22:40:45,982][105692] Updated weights for policy 0, policy_version 1003316 (0.0007) [2023-12-26 22:40:46,033][105692] Updated weights for policy 0, policy_version 1003326 (0.0008) [2023-12-26 22:40:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.3, 300 sec: 19438.6). Total num frames: 513859584. Throughput: 0: 9859.2, 1: 9398.6. Samples: 513836396. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:40:46,063][104569] Avg episode reward: [(0, '8629.753'), (1, '8893.343')] [2023-12-26 22:40:46,086][105620] Updated weights for policy 1, policy_version 1003698 (0.0006) [2023-12-26 22:40:46,095][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001003336_256892928.pth... [2023-12-26 22:40:46,095][105692] Updated weights for policy 0, policy_version 1003336 (0.0009) [2023-12-26 22:40:46,098][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001002152_256589824.pth [2023-12-26 22:40:46,132][105620] Updated weights for policy 1, policy_version 1003708 (0.0007) [2023-12-26 22:40:46,184][105620] Updated weights for policy 1, policy_version 1003718 (0.0005) [2023-12-26 22:40:46,193][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001003720_256983040.pth... [2023-12-26 22:40:46,196][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001002600_256696320.pth [2023-12-26 22:40:46,843][105692] Updated weights for policy 0, policy_version 1003346 (0.0008) [2023-12-26 22:40:46,883][105620] Updated weights for policy 1, policy_version 1003728 (0.0009) [2023-12-26 22:40:46,899][105692] Updated weights for policy 0, policy_version 1003356 (0.0008) [2023-12-26 22:40:46,929][105620] Updated weights for policy 1, policy_version 1003738 (0.0010) [2023-12-26 22:40:46,959][105692] Updated weights for policy 0, policy_version 1003366 (0.0007) [2023-12-26 22:40:46,978][105620] Updated weights for policy 1, policy_version 1003748 (0.0010) [2023-12-26 22:40:47,615][105620] Updated weights for policy 1, policy_version 1003758 (0.0007) [2023-12-26 22:40:47,675][105620] Updated weights for policy 1, policy_version 1003768 (0.0009) [2023-12-26 22:40:47,676][105692] Updated weights for policy 0, policy_version 1003376 (0.0009) [2023-12-26 22:40:47,736][105692] Updated weights for policy 0, policy_version 1003386 (0.0008) [2023-12-26 22:40:47,739][105620] Updated weights for policy 1, policy_version 1003778 (0.0011) [2023-12-26 22:40:47,788][105692] Updated weights for policy 0, policy_version 1003396 (0.0005) [2023-12-26 22:40:48,325][105620] Updated weights for policy 1, policy_version 1003788 (0.0009) [2023-12-26 22:40:48,380][105692] Updated weights for policy 0, policy_version 1003406 (0.0006) [2023-12-26 22:40:48,386][105620] Updated weights for policy 1, policy_version 1003798 (0.0007) [2023-12-26 22:40:48,440][105692] Updated weights for policy 0, policy_version 1003416 (0.0009) [2023-12-26 22:40:48,442][105620] Updated weights for policy 1, policy_version 1003808 (0.0005) [2023-12-26 22:40:48,497][105692] Updated weights for policy 0, policy_version 1003426 (0.0008) [2023-12-26 22:40:49,151][105620] Updated weights for policy 1, policy_version 1003818 (0.0007) [2023-12-26 22:40:49,218][105620] Updated weights for policy 1, policy_version 1003828 (0.0008) [2023-12-26 22:40:49,275][105692] Updated weights for policy 0, policy_version 1003436 (0.0008) [2023-12-26 22:40:49,281][105620] Updated weights for policy 1, policy_version 1003838 (0.0007) [2023-12-26 22:40:49,338][105692] Updated weights for policy 0, policy_version 1003446 (0.0007) [2023-12-26 22:40:49,348][105620] Updated weights for policy 1, policy_version 1003848 (0.0007) [2023-12-26 22:40:49,414][105692] Updated weights for policy 0, policy_version 1003456 (0.0008) [2023-12-26 22:40:50,092][105620] Updated weights for policy 1, policy_version 1003858 (0.0010) [2023-12-26 22:40:50,133][105692] Updated weights for policy 0, policy_version 1003466 (0.0009) [2023-12-26 22:40:50,149][105620] Updated weights for policy 1, policy_version 1003868 (0.0010) [2023-12-26 22:40:50,197][105692] Updated weights for policy 0, policy_version 1003476 (0.0006) [2023-12-26 22:40:50,212][105620] Updated weights for policy 1, policy_version 1003878 (0.0010) [2023-12-26 22:40:50,258][105692] Updated weights for policy 0, policy_version 1003486 (0.0006) [2023-12-26 22:40:50,311][105692] Updated weights for policy 0, policy_version 1003496 (0.0009) [2023-12-26 22:40:50,949][105692] Updated weights for policy 0, policy_version 1003506 (0.0008) [2023-12-26 22:40:51,000][105692] Updated weights for policy 0, policy_version 1003516 (0.0007) [2023-12-26 22:40:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 513957888. Throughput: 0: 9839.8, 1: 9488.0. Samples: 513956064. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:40:51,062][104569] Avg episode reward: [(0, '4004.328'), (1, '9263.125')] [2023-12-26 22:40:51,064][105692] Updated weights for policy 0, policy_version 1003526 (0.0009) [2023-12-26 22:40:51,077][105620] Updated weights for policy 1, policy_version 1003888 (0.0008) [2023-12-26 22:40:51,139][105620] Updated weights for policy 1, policy_version 1003898 (0.0008) [2023-12-26 22:40:51,195][105620] Updated weights for policy 1, policy_version 1003908 (0.0010) [2023-12-26 22:40:51,887][105692] Updated weights for policy 0, policy_version 1003536 (0.0007) [2023-12-26 22:40:51,955][105692] Updated weights for policy 0, policy_version 1003546 (0.0009) [2023-12-26 22:40:52,011][105620] Updated weights for policy 1, policy_version 1003918 (0.0007) [2023-12-26 22:40:52,013][105692] Updated weights for policy 0, policy_version 1003556 (0.0012) [2023-12-26 22:40:52,062][105620] Updated weights for policy 1, policy_version 1003928 (0.0007) [2023-12-26 22:40:52,114][105620] Updated weights for policy 1, policy_version 1003938 (0.0008) [2023-12-26 22:40:52,721][105692] Updated weights for policy 0, policy_version 1003566 (0.0011) [2023-12-26 22:40:52,775][105692] Updated weights for policy 0, policy_version 1003576 (0.0009) [2023-12-26 22:40:52,840][105692] Updated weights for policy 0, policy_version 1003586 (0.0007) [2023-12-26 22:40:52,913][105620] Updated weights for policy 1, policy_version 1003949 (0.0009) [2023-12-26 22:40:52,968][105620] Updated weights for policy 1, policy_version 1003959 (0.0010) [2023-12-26 22:40:53,017][105620] Updated weights for policy 1, policy_version 1003969 (0.0010) [2023-12-26 22:40:53,460][105692] Updated weights for policy 0, policy_version 1003596 (0.0008) [2023-12-26 22:40:53,527][105692] Updated weights for policy 0, policy_version 1003606 (0.0007) [2023-12-26 22:40:53,588][105692] Updated weights for policy 0, policy_version 1003616 (0.0005) [2023-12-26 22:40:53,770][105620] Updated weights for policy 1, policy_version 1003979 (0.0010) [2023-12-26 22:40:53,821][105620] Updated weights for policy 1, policy_version 1003989 (0.0010) [2023-12-26 22:40:53,892][105620] Updated weights for policy 1, policy_version 1003999 (0.0010) [2023-12-26 22:40:54,233][105692] Updated weights for policy 0, policy_version 1003626 (0.0006) [2023-12-26 22:40:54,296][105692] Updated weights for policy 0, policy_version 1003636 (0.0010) [2023-12-26 22:40:54,365][105692] Updated weights for policy 0, policy_version 1003646 (0.0010) [2023-12-26 22:40:54,432][105692] Updated weights for policy 0, policy_version 1003656 (0.0010) [2023-12-26 22:40:54,550][105620] Updated weights for policy 1, policy_version 1004009 (0.0009) [2023-12-26 22:40:54,609][105620] Updated weights for policy 1, policy_version 1004019 (0.0006) [2023-12-26 22:40:54,661][105620] Updated weights for policy 1, policy_version 1004029 (0.0006) [2023-12-26 22:40:54,726][105620] Updated weights for policy 1, policy_version 1004039 (0.0006) [2023-12-26 22:40:55,199][105692] Updated weights for policy 0, policy_version 1003666 (0.0005) [2023-12-26 22:40:55,257][105692] Updated weights for policy 0, policy_version 1003676 (0.0005) [2023-12-26 22:40:55,270][105620] Updated weights for policy 1, policy_version 1004049 (0.0006) [2023-12-26 22:40:55,310][105692] Updated weights for policy 0, policy_version 1003686 (0.0005) [2023-12-26 22:40:55,332][105620] Updated weights for policy 1, policy_version 1004059 (0.0005) [2023-12-26 22:40:55,380][105620] Updated weights for policy 1, policy_version 1004069 (0.0005) [2023-12-26 22:40:55,919][105620] Updated weights for policy 1, policy_version 1004079 (0.0005) [2023-12-26 22:40:55,982][105620] Updated weights for policy 1, policy_version 1004089 (0.0008) [2023-12-26 22:40:56,040][105620] Updated weights for policy 1, policy_version 1004099 (0.0008) [2023-12-26 22:40:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.1, 300 sec: 19438.6). Total num frames: 514056192. Throughput: 0: 9926.1, 1: 9518.1. Samples: 514074444. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:40:56,063][104569] Avg episode reward: [(0, '6880.687'), (1, '9263.004')] [2023-12-26 22:40:56,121][105692] Updated weights for policy 0, policy_version 1003696 (0.0008) [2023-12-26 22:40:56,173][105692] Updated weights for policy 0, policy_version 1003706 (0.0008) [2023-12-26 22:40:56,225][105692] Updated weights for policy 0, policy_version 1003716 (0.0008) [2023-12-26 22:40:56,761][105620] Updated weights for policy 1, policy_version 1004109 (0.0008) [2023-12-26 22:40:56,812][105620] Updated weights for policy 1, policy_version 1004119 (0.0009) [2023-12-26 22:40:56,867][105620] Updated weights for policy 1, policy_version 1004129 (0.0007) [2023-12-26 22:40:56,995][105692] Updated weights for policy 0, policy_version 1003726 (0.0009) [2023-12-26 22:40:57,053][105692] Updated weights for policy 0, policy_version 1003737 (0.0011) [2023-12-26 22:40:57,103][105692] Updated weights for policy 0, policy_version 1003747 (0.0010) [2023-12-26 22:40:57,536][105620] Updated weights for policy 1, policy_version 1004139 (0.0008) [2023-12-26 22:40:57,594][105620] Updated weights for policy 1, policy_version 1004149 (0.0009) [2023-12-26 22:40:57,649][105620] Updated weights for policy 1, policy_version 1004159 (0.0010) [2023-12-26 22:40:57,877][105692] Updated weights for policy 0, policy_version 1003757 (0.0009) [2023-12-26 22:40:57,935][105692] Updated weights for policy 0, policy_version 1003767 (0.0009) [2023-12-26 22:40:57,993][105692] Updated weights for policy 0, policy_version 1003777 (0.0007) [2023-12-26 22:40:58,438][105620] Updated weights for policy 1, policy_version 1004169 (0.0009) [2023-12-26 22:40:58,501][105620] Updated weights for policy 1, policy_version 1004179 (0.0008) [2023-12-26 22:40:58,568][105620] Updated weights for policy 1, policy_version 1004189 (0.0008) [2023-12-26 22:40:58,631][105620] Updated weights for policy 1, policy_version 1004199 (0.0008) [2023-12-26 22:40:58,694][105692] Updated weights for policy 0, policy_version 1003787 (0.0006) [2023-12-26 22:40:58,754][105692] Updated weights for policy 0, policy_version 1003797 (0.0008) [2023-12-26 22:40:58,823][105692] Updated weights for policy 0, policy_version 1003807 (0.0007) [2023-12-26 22:40:59,387][105620] Updated weights for policy 1, policy_version 1004209 (0.0008) [2023-12-26 22:40:59,449][105620] Updated weights for policy 1, policy_version 1004219 (0.0009) [2023-12-26 22:40:59,499][105620] Updated weights for policy 1, policy_version 1004229 (0.0009) [2023-12-26 22:40:59,598][105692] Updated weights for policy 0, policy_version 1003817 (0.0008) [2023-12-26 22:40:59,659][105692] Updated weights for policy 0, policy_version 1003827 (0.0009) [2023-12-26 22:40:59,717][105692] Updated weights for policy 0, policy_version 1003837 (0.0009) [2023-12-26 22:40:59,778][105692] Updated weights for policy 0, policy_version 1003847 (0.0007) [2023-12-26 22:41:00,307][105620] Updated weights for policy 1, policy_version 1004239 (0.0007) [2023-12-26 22:41:00,354][105620] Updated weights for policy 1, policy_version 1004249 (0.0006) [2023-12-26 22:41:00,409][105620] Updated weights for policy 1, policy_version 1004259 (0.0006) [2023-12-26 22:41:00,499][105692] Updated weights for policy 0, policy_version 1003857 (0.0009) [2023-12-26 22:41:00,559][105692] Updated weights for policy 0, policy_version 1003867 (0.0008) [2023-12-26 22:41:00,606][105692] Updated weights for policy 0, policy_version 1003877 (0.0009) [2023-12-26 22:41:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 514154496. Throughput: 0: 9918.3, 1: 9506.4. Samples: 514130152. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:41:01,062][104569] Avg episode reward: [(0, '8392.093'), (1, '9262.979')] [2023-12-26 22:41:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001003880_257032192.pth... [2023-12-26 22:41:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001004264_257122304.pth... [2023-12-26 22:41:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001003176_256843776.pth [2023-12-26 22:41:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001002760_256745472.pth [2023-12-26 22:41:01,150][105620] Updated weights for policy 1, policy_version 1004269 (0.0007) [2023-12-26 22:41:01,206][105620] Updated weights for policy 1, policy_version 1004279 (0.0008) [2023-12-26 22:41:01,263][105620] Updated weights for policy 1, policy_version 1004289 (0.0008) [2023-12-26 22:41:01,303][105692] Updated weights for policy 0, policy_version 1003887 (0.0009) [2023-12-26 22:41:01,360][105692] Updated weights for policy 0, policy_version 1003897 (0.0008) [2023-12-26 22:41:01,420][105692] Updated weights for policy 0, policy_version 1003907 (0.0009) [2023-12-26 22:41:02,010][105620] Updated weights for policy 1, policy_version 1004299 (0.0009) [2023-12-26 22:41:02,069][105620] Updated weights for policy 1, policy_version 1004309 (0.0009) [2023-12-26 22:41:02,133][105620] Updated weights for policy 1, policy_version 1004319 (0.0008) [2023-12-26 22:41:02,142][105692] Updated weights for policy 0, policy_version 1003917 (0.0009) [2023-12-26 22:41:02,196][105692] Updated weights for policy 0, policy_version 1003927 (0.0006) [2023-12-26 22:41:02,245][105692] Updated weights for policy 0, policy_version 1003937 (0.0008) [2023-12-26 22:41:02,908][105692] Updated weights for policy 0, policy_version 1003947 (0.0010) [2023-12-26 22:41:02,943][105620] Updated weights for policy 1, policy_version 1004329 (0.0007) [2023-12-26 22:41:02,957][105692] Updated weights for policy 0, policy_version 1003957 (0.0010) [2023-12-26 22:41:03,003][105620] Updated weights for policy 1, policy_version 1004339 (0.0007) [2023-12-26 22:41:03,006][105692] Updated weights for policy 0, policy_version 1003967 (0.0010) [2023-12-26 22:41:03,063][105620] Updated weights for policy 1, policy_version 1004349 (0.0008) [2023-12-26 22:41:03,125][105620] Updated weights for policy 1, policy_version 1004359 (0.0008) [2023-12-26 22:41:03,773][105692] Updated weights for policy 0, policy_version 1003977 (0.0010) [2023-12-26 22:41:03,820][105692] Updated weights for policy 0, policy_version 1003987 (0.0008) [2023-12-26 22:41:03,823][105620] Updated weights for policy 1, policy_version 1004369 (0.0006) [2023-12-26 22:41:03,889][105620] Updated weights for policy 1, policy_version 1004379 (0.0008) [2023-12-26 22:41:03,889][105692] Updated weights for policy 0, policy_version 1003997 (0.0008) [2023-12-26 22:41:03,943][105692] Updated weights for policy 0, policy_version 1004007 (0.0010) [2023-12-26 22:41:03,949][105620] Updated weights for policy 1, policy_version 1004389 (0.0006) [2023-12-26 22:41:04,627][105692] Updated weights for policy 0, policy_version 1004017 (0.0009) [2023-12-26 22:41:04,691][105692] Updated weights for policy 0, policy_version 1004027 (0.0010) [2023-12-26 22:41:04,745][105620] Updated weights for policy 1, policy_version 1004399 (0.0008) [2023-12-26 22:41:04,753][105692] Updated weights for policy 0, policy_version 1004037 (0.0010) [2023-12-26 22:41:04,792][105620] Updated weights for policy 1, policy_version 1004409 (0.0008) [2023-12-26 22:41:04,841][105620] Updated weights for policy 1, policy_version 1004419 (0.0008) [2023-12-26 22:41:05,373][105692] Updated weights for policy 0, policy_version 1004047 (0.0010) [2023-12-26 22:41:05,432][105692] Updated weights for policy 0, policy_version 1004057 (0.0009) [2023-12-26 22:41:05,485][105692] Updated weights for policy 0, policy_version 1004067 (0.0010) [2023-12-26 22:41:05,640][105620] Updated weights for policy 1, policy_version 1004429 (0.0010) [2023-12-26 22:41:05,693][105620] Updated weights for policy 1, policy_version 1004439 (0.0010) [2023-12-26 22:41:05,752][105620] Updated weights for policy 1, policy_version 1004449 (0.0008) [2023-12-26 22:41:06,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 514252800. Throughput: 0: 9805.5, 1: 9430.1. Samples: 514243536. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:41:06,062][104569] Avg episode reward: [(0, '6331.081'), (1, '9079.188')] [2023-12-26 22:41:06,108][105692] Updated weights for policy 0, policy_version 1004077 (0.0009) [2023-12-26 22:41:06,171][105692] Updated weights for policy 0, policy_version 1004087 (0.0009) [2023-12-26 22:41:06,225][105692] Updated weights for policy 0, policy_version 1004097 (0.0010) [2023-12-26 22:41:06,487][105620] Updated weights for policy 1, policy_version 1004459 (0.0010) [2023-12-26 22:41:06,542][105620] Updated weights for policy 1, policy_version 1004469 (0.0009) [2023-12-26 22:41:06,607][105620] Updated weights for policy 1, policy_version 1004479 (0.0008) [2023-12-26 22:41:07,020][105692] Updated weights for policy 0, policy_version 1004108 (0.0010) [2023-12-26 22:41:07,085][105692] Updated weights for policy 0, policy_version 1004118 (0.0006) [2023-12-26 22:41:07,144][105692] Updated weights for policy 0, policy_version 1004128 (0.0008) [2023-12-26 22:41:07,381][105620] Updated weights for policy 1, policy_version 1004489 (0.0009) [2023-12-26 22:41:07,432][105620] Updated weights for policy 1, policy_version 1004499 (0.0009) [2023-12-26 22:41:07,479][105620] Updated weights for policy 1, policy_version 1004509 (0.0009) [2023-12-26 22:41:07,530][105620] Updated weights for policy 1, policy_version 1004519 (0.0009) [2023-12-26 22:41:07,843][105692] Updated weights for policy 0, policy_version 1004138 (0.0008) [2023-12-26 22:41:07,890][105692] Updated weights for policy 0, policy_version 1004148 (0.0007) [2023-12-26 22:41:07,948][105692] Updated weights for policy 0, policy_version 1004158 (0.0009) [2023-12-26 22:41:08,007][105692] Updated weights for policy 0, policy_version 1004168 (0.0008) [2023-12-26 22:41:08,323][105620] Updated weights for policy 1, policy_version 1004529 (0.0009) [2023-12-26 22:41:08,386][105620] Updated weights for policy 1, policy_version 1004539 (0.0009) [2023-12-26 22:41:08,444][105620] Updated weights for policy 1, policy_version 1004549 (0.0009) [2023-12-26 22:41:08,738][105692] Updated weights for policy 0, policy_version 1004178 (0.0009) [2023-12-26 22:41:08,796][105692] Updated weights for policy 0, policy_version 1004188 (0.0010) [2023-12-26 22:41:08,852][105692] Updated weights for policy 0, policy_version 1004198 (0.0009) [2023-12-26 22:41:09,176][105620] Updated weights for policy 1, policy_version 1004559 (0.0009) [2023-12-26 22:41:09,231][105620] Updated weights for policy 1, policy_version 1004569 (0.0009) [2023-12-26 22:41:09,292][105620] Updated weights for policy 1, policy_version 1004579 (0.0009) [2023-12-26 22:41:09,643][105692] Updated weights for policy 0, policy_version 1004208 (0.0009) [2023-12-26 22:41:09,707][105692] Updated weights for policy 0, policy_version 1004218 (0.0009) [2023-12-26 22:41:09,778][105692] Updated weights for policy 0, policy_version 1004228 (0.0010) [2023-12-26 22:41:10,011][105620] Updated weights for policy 1, policy_version 1004589 (0.0009) [2023-12-26 22:41:10,079][105620] Updated weights for policy 1, policy_version 1004599 (0.0007) [2023-12-26 22:41:10,137][105620] Updated weights for policy 1, policy_version 1004609 (0.0008) [2023-12-26 22:41:10,550][105692] Updated weights for policy 0, policy_version 1004238 (0.0007) [2023-12-26 22:41:10,603][105692] Updated weights for policy 0, policy_version 1004248 (0.0005) [2023-12-26 22:41:10,657][105692] Updated weights for policy 0, policy_version 1004258 (0.0005) [2023-12-26 22:41:10,910][105620] Updated weights for policy 1, policy_version 1004619 (0.0009) [2023-12-26 22:41:10,973][105620] Updated weights for policy 1, policy_version 1004630 (0.0010) [2023-12-26 22:41:11,037][105620] Updated weights for policy 1, policy_version 1004640 (0.0009) [2023-12-26 22:41:11,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 514342912. Throughput: 0: 9752.9, 1: 9471.8. Samples: 514357172. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:41:11,063][104569] Avg episode reward: [(0, '7357.826'), (1, '9170.670')] [2023-12-26 22:41:11,411][105692] Updated weights for policy 0, policy_version 1004268 (0.0009) [2023-12-26 22:41:11,475][105692] Updated weights for policy 0, policy_version 1004278 (0.0008) [2023-12-26 22:41:11,527][105692] Updated weights for policy 0, policy_version 1004288 (0.0008) [2023-12-26 22:41:11,842][105620] Updated weights for policy 1, policy_version 1004650 (0.0010) [2023-12-26 22:41:11,902][105620] Updated weights for policy 1, policy_version 1004660 (0.0011) [2023-12-26 22:41:11,961][105620] Updated weights for policy 1, policy_version 1004670 (0.0010) [2023-12-26 22:41:12,018][105620] Updated weights for policy 1, policy_version 1004680 (0.0011) [2023-12-26 22:41:12,317][105692] Updated weights for policy 0, policy_version 1004298 (0.0007) [2023-12-26 22:41:12,383][105692] Updated weights for policy 0, policy_version 1004308 (0.0008) [2023-12-26 22:41:12,450][105692] Updated weights for policy 0, policy_version 1004318 (0.0008) [2023-12-26 22:41:12,517][105692] Updated weights for policy 0, policy_version 1004328 (0.0008) [2023-12-26 22:41:12,715][105620] Updated weights for policy 1, policy_version 1004690 (0.0010) [2023-12-26 22:41:12,783][105620] Updated weights for policy 1, policy_version 1004700 (0.0010) [2023-12-26 22:41:12,842][105620] Updated weights for policy 1, policy_version 1004710 (0.0008) [2023-12-26 22:41:13,173][105692] Updated weights for policy 0, policy_version 1004338 (0.0008) [2023-12-26 22:41:13,236][105692] Updated weights for policy 0, policy_version 1004348 (0.0008) [2023-12-26 22:41:13,296][105692] Updated weights for policy 0, policy_version 1004358 (0.0008) [2023-12-26 22:41:13,584][105620] Updated weights for policy 1, policy_version 1004720 (0.0005) [2023-12-26 22:41:13,636][105620] Updated weights for policy 1, policy_version 1004730 (0.0007) [2023-12-26 22:41:13,683][105620] Updated weights for policy 1, policy_version 1004740 (0.0008) [2023-12-26 22:41:14,046][105692] Updated weights for policy 0, policy_version 1004368 (0.0009) [2023-12-26 22:41:14,107][105692] Updated weights for policy 0, policy_version 1004378 (0.0006) [2023-12-26 22:41:14,174][105692] Updated weights for policy 0, policy_version 1004388 (0.0008) [2023-12-26 22:41:14,328][105620] Updated weights for policy 1, policy_version 1004750 (0.0008) [2023-12-26 22:41:14,398][105620] Updated weights for policy 1, policy_version 1004760 (0.0009) [2023-12-26 22:41:14,458][105620] Updated weights for policy 1, policy_version 1004770 (0.0006) [2023-12-26 22:41:14,834][105692] Updated weights for policy 0, policy_version 1004398 (0.0008) [2023-12-26 22:41:14,895][105692] Updated weights for policy 0, policy_version 1004408 (0.0007) [2023-12-26 22:41:14,951][105692] Updated weights for policy 0, policy_version 1004418 (0.0005) [2023-12-26 22:41:15,130][105620] Updated weights for policy 1, policy_version 1004780 (0.0008) [2023-12-26 22:41:15,191][105620] Updated weights for policy 1, policy_version 1004790 (0.0011) [2023-12-26 22:41:15,247][105620] Updated weights for policy 1, policy_version 1004800 (0.0011) [2023-12-26 22:41:15,566][105692] Updated weights for policy 0, policy_version 1004428 (0.0005) [2023-12-26 22:41:15,625][105692] Updated weights for policy 0, policy_version 1004438 (0.0005) [2023-12-26 22:41:15,680][105692] Updated weights for policy 0, policy_version 1004448 (0.0007) [2023-12-26 22:41:15,975][105620] Updated weights for policy 1, policy_version 1004810 (0.0008) [2023-12-26 22:41:16,032][105620] Updated weights for policy 1, policy_version 1004820 (0.0005) [2023-12-26 22:41:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 514441216. Throughput: 0: 9747.6, 1: 9447.3. Samples: 514413536. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:41:16,062][104569] Avg episode reward: [(0, '8389.115'), (1, '9171.629')] [2023-12-26 22:41:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001004456_257179648.pth... [2023-12-26 22:41:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001003336_256892928.pth [2023-12-26 22:41:16,078][105620] Updated weights for policy 1, policy_version 1004830 (0.0005) [2023-12-26 22:41:16,142][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001004840_257269760.pth... [2023-12-26 22:41:16,143][105620] Updated weights for policy 1, policy_version 1004840 (0.0006) [2023-12-26 22:41:16,147][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001003720_256983040.pth [2023-12-26 22:41:16,213][105692] Updated weights for policy 0, policy_version 1004458 (0.0007) [2023-12-26 22:41:16,267][105692] Updated weights for policy 0, policy_version 1004468 (0.0006) [2023-12-26 22:41:16,328][105692] Updated weights for policy 0, policy_version 1004478 (0.0005) [2023-12-26 22:41:16,396][105692] Updated weights for policy 0, policy_version 1004488 (0.0005) [2023-12-26 22:41:16,684][105620] Updated weights for policy 1, policy_version 1004850 (0.0005) [2023-12-26 22:41:16,736][105620] Updated weights for policy 1, policy_version 1004860 (0.0006) [2023-12-26 22:41:16,784][105620] Updated weights for policy 1, policy_version 1004870 (0.0010) [2023-12-26 22:41:16,938][105692] Updated weights for policy 0, policy_version 1004498 (0.0005) [2023-12-26 22:41:16,988][105692] Updated weights for policy 0, policy_version 1004508 (0.0005) [2023-12-26 22:41:17,043][105692] Updated weights for policy 0, policy_version 1004518 (0.0005) [2023-12-26 22:41:17,444][105620] Updated weights for policy 1, policy_version 1004880 (0.0010) [2023-12-26 22:41:17,502][105620] Updated weights for policy 1, policy_version 1004890 (0.0010) [2023-12-26 22:41:17,559][105620] Updated weights for policy 1, policy_version 1004900 (0.0010) [2023-12-26 22:41:17,696][105692] Updated weights for policy 0, policy_version 1004528 (0.0008) [2023-12-26 22:41:17,757][105692] Updated weights for policy 0, policy_version 1004538 (0.0007) [2023-12-26 22:41:17,822][105692] Updated weights for policy 0, policy_version 1004548 (0.0008) [2023-12-26 22:41:18,221][105620] Updated weights for policy 1, policy_version 1004910 (0.0007) [2023-12-26 22:41:18,271][105620] Updated weights for policy 1, policy_version 1004920 (0.0005) [2023-12-26 22:41:18,337][105620] Updated weights for policy 1, policy_version 1004930 (0.0009) [2023-12-26 22:41:18,554][105692] Updated weights for policy 0, policy_version 1004558 (0.0009) [2023-12-26 22:41:18,609][105692] Updated weights for policy 0, policy_version 1004568 (0.0009) [2023-12-26 22:41:18,678][105692] Updated weights for policy 0, policy_version 1004578 (0.0009) [2023-12-26 22:41:18,951][105620] Updated weights for policy 1, policy_version 1004940 (0.0007) [2023-12-26 22:41:19,006][105620] Updated weights for policy 1, policy_version 1004950 (0.0005) [2023-12-26 22:41:19,066][105620] Updated weights for policy 1, policy_version 1004960 (0.0006) [2023-12-26 22:41:19,508][105692] Updated weights for policy 0, policy_version 1004588 (0.0009) [2023-12-26 22:41:19,569][105692] Updated weights for policy 0, policy_version 1004598 (0.0009) [2023-12-26 22:41:19,639][105692] Updated weights for policy 0, policy_version 1004608 (0.0008) [2023-12-26 22:41:19,728][105620] Updated weights for policy 1, policy_version 1004970 (0.0007) [2023-12-26 22:41:19,784][105620] Updated weights for policy 1, policy_version 1004980 (0.0011) [2023-12-26 22:41:19,851][105620] Updated weights for policy 1, policy_version 1004990 (0.0007) [2023-12-26 22:41:19,913][105620] Updated weights for policy 1, policy_version 1005000 (0.0008) [2023-12-26 22:41:20,414][105692] Updated weights for policy 0, policy_version 1004618 (0.0008) [2023-12-26 22:41:20,471][105692] Updated weights for policy 0, policy_version 1004628 (0.0009) [2023-12-26 22:41:20,531][105692] Updated weights for policy 0, policy_version 1004638 (0.0009) [2023-12-26 22:41:20,597][105692] Updated weights for policy 0, policy_version 1004648 (0.0008) [2023-12-26 22:41:20,724][105620] Updated weights for policy 1, policy_version 1005010 (0.0009) [2023-12-26 22:41:20,787][105620] Updated weights for policy 1, policy_version 1005020 (0.0009) [2023-12-26 22:41:20,848][105620] Updated weights for policy 1, policy_version 1005030 (0.0009) [2023-12-26 22:41:21,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 514547712. Throughput: 0: 9845.8, 1: 9563.1. Samples: 514538240. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:41:21,062][104569] Avg episode reward: [(0, '8470.190'), (1, '8987.723')] [2023-12-26 22:41:21,384][105692] Updated weights for policy 0, policy_version 1004658 (0.0009) [2023-12-26 22:41:21,437][105692] Updated weights for policy 0, policy_version 1004668 (0.0006) [2023-12-26 22:41:21,497][105692] Updated weights for policy 0, policy_version 1004678 (0.0008) [2023-12-26 22:41:21,630][105620] Updated weights for policy 1, policy_version 1005040 (0.0009) [2023-12-26 22:41:21,688][105620] Updated weights for policy 1, policy_version 1005050 (0.0010) [2023-12-26 22:41:21,749][105620] Updated weights for policy 1, policy_version 1005060 (0.0009) [2023-12-26 22:41:22,248][105692] Updated weights for policy 0, policy_version 1004688 (0.0009) [2023-12-26 22:41:22,315][105692] Updated weights for policy 0, policy_version 1004698 (0.0009) [2023-12-26 22:41:22,386][105692] Updated weights for policy 0, policy_version 1004708 (0.0008) [2023-12-26 22:41:22,485][105620] Updated weights for policy 1, policy_version 1005070 (0.0008) [2023-12-26 22:41:22,548][105620] Updated weights for policy 1, policy_version 1005080 (0.0009) [2023-12-26 22:41:22,612][105620] Updated weights for policy 1, policy_version 1005090 (0.0009) [2023-12-26 22:41:23,078][105692] Updated weights for policy 0, policy_version 1004718 (0.0009) [2023-12-26 22:41:23,137][105692] Updated weights for policy 0, policy_version 1004728 (0.0008) [2023-12-26 22:41:23,200][105692] Updated weights for policy 0, policy_version 1004738 (0.0008) [2023-12-26 22:41:23,401][105620] Updated weights for policy 1, policy_version 1005100 (0.0009) [2023-12-26 22:41:23,451][105620] Updated weights for policy 1, policy_version 1005110 (0.0008) [2023-12-26 22:41:23,511][105620] Updated weights for policy 1, policy_version 1005120 (0.0010) [2023-12-26 22:41:23,930][105692] Updated weights for policy 0, policy_version 1004748 (0.0008) [2023-12-26 22:41:23,991][105692] Updated weights for policy 0, policy_version 1004758 (0.0006) [2023-12-26 22:41:24,042][105692] Updated weights for policy 0, policy_version 1004768 (0.0005) [2023-12-26 22:41:24,239][105620] Updated weights for policy 1, policy_version 1005130 (0.0010) [2023-12-26 22:41:24,294][105620] Updated weights for policy 1, policy_version 1005140 (0.0010) [2023-12-26 22:41:24,352][105620] Updated weights for policy 1, policy_version 1005150 (0.0010) [2023-12-26 22:41:24,397][105620] Updated weights for policy 1, policy_version 1005160 (0.0010) [2023-12-26 22:41:24,647][105692] Updated weights for policy 0, policy_version 1004778 (0.0005) [2023-12-26 22:41:24,711][105692] Updated weights for policy 0, policy_version 1004788 (0.0009) [2023-12-26 22:41:24,770][105692] Updated weights for policy 0, policy_version 1004798 (0.0009) [2023-12-26 22:41:24,826][105692] Updated weights for policy 0, policy_version 1004808 (0.0009) [2023-12-26 22:41:25,057][105620] Updated weights for policy 1, policy_version 1005170 (0.0009) [2023-12-26 22:41:25,120][105620] Updated weights for policy 1, policy_version 1005180 (0.0009) [2023-12-26 22:41:25,170][105620] Updated weights for policy 1, policy_version 1005190 (0.0009) [2023-12-26 22:41:25,601][105692] Updated weights for policy 0, policy_version 1004818 (0.0009) [2023-12-26 22:41:25,650][105692] Updated weights for policy 0, policy_version 1004828 (0.0008) [2023-12-26 22:41:25,697][105692] Updated weights for policy 0, policy_version 1004838 (0.0009) [2023-12-26 22:41:25,912][105620] Updated weights for policy 1, policy_version 1005200 (0.0008) [2023-12-26 22:41:25,966][105620] Updated weights for policy 1, policy_version 1005210 (0.0007) [2023-12-26 22:41:26,028][105620] Updated weights for policy 1, policy_version 1005220 (0.0009) [2023-12-26 22:41:26,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 514646016. Throughput: 0: 9703.3, 1: 9661.6. Samples: 514650896. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:41:26,063][104569] Avg episode reward: [(0, '8382.169'), (1, '9170.980')] [2023-12-26 22:41:26,356][105692] Updated weights for policy 0, policy_version 1004848 (0.0010) [2023-12-26 22:41:26,409][105692] Updated weights for policy 0, policy_version 1004858 (0.0009) [2023-12-26 22:41:26,460][105692] Updated weights for policy 0, policy_version 1004868 (0.0010) [2023-12-26 22:41:26,794][105620] Updated weights for policy 1, policy_version 1005230 (0.0007) [2023-12-26 22:41:26,851][105620] Updated weights for policy 1, policy_version 1005240 (0.0005) [2023-12-26 22:41:26,902][105620] Updated weights for policy 1, policy_version 1005250 (0.0006) [2023-12-26 22:41:27,182][105692] Updated weights for policy 0, policy_version 1004878 (0.0007) [2023-12-26 22:41:27,246][105692] Updated weights for policy 0, policy_version 1004888 (0.0005) [2023-12-26 22:41:27,300][105692] Updated weights for policy 0, policy_version 1004898 (0.0005) [2023-12-26 22:41:27,505][105620] Updated weights for policy 1, policy_version 1005260 (0.0006) [2023-12-26 22:41:27,564][105620] Updated weights for policy 1, policy_version 1005270 (0.0005) [2023-12-26 22:41:27,617][105620] Updated weights for policy 1, policy_version 1005280 (0.0005) [2023-12-26 22:41:27,837][105692] Updated weights for policy 0, policy_version 1004908 (0.0006) [2023-12-26 22:41:27,891][105692] Updated weights for policy 0, policy_version 1004918 (0.0005) [2023-12-26 22:41:27,937][105692] Updated weights for policy 0, policy_version 1004928 (0.0005) [2023-12-26 22:41:28,223][105620] Updated weights for policy 1, policy_version 1005290 (0.0006) [2023-12-26 22:41:28,287][105620] Updated weights for policy 1, policy_version 1005300 (0.0010) [2023-12-26 22:41:28,351][105620] Updated weights for policy 1, policy_version 1005310 (0.0010) [2023-12-26 22:41:28,407][105620] Updated weights for policy 1, policy_version 1005320 (0.0010) [2023-12-26 22:41:28,620][105692] Updated weights for policy 0, policy_version 1004938 (0.0005) [2023-12-26 22:41:28,669][105692] Updated weights for policy 0, policy_version 1004948 (0.0008) [2023-12-26 22:41:28,718][105692] Updated weights for policy 0, policy_version 1004958 (0.0008) [2023-12-26 22:41:28,766][105692] Updated weights for policy 0, policy_version 1004968 (0.0008) [2023-12-26 22:41:29,137][105620] Updated weights for policy 1, policy_version 1005330 (0.0009) [2023-12-26 22:41:29,188][105620] Updated weights for policy 1, policy_version 1005341 (0.0007) [2023-12-26 22:41:29,246][105620] Updated weights for policy 1, policy_version 1005351 (0.0007) [2023-12-26 22:41:29,524][105692] Updated weights for policy 0, policy_version 1004978 (0.0008) [2023-12-26 22:41:29,568][105692] Updated weights for policy 0, policy_version 1004988 (0.0008) [2023-12-26 22:41:29,620][105692] Updated weights for policy 0, policy_version 1004998 (0.0008) [2023-12-26 22:41:30,015][105620] Updated weights for policy 1, policy_version 1005361 (0.0008) [2023-12-26 22:41:30,079][105620] Updated weights for policy 1, policy_version 1005371 (0.0005) [2023-12-26 22:41:30,134][105620] Updated weights for policy 1, policy_version 1005381 (0.0009) [2023-12-26 22:41:30,439][105692] Updated weights for policy 0, policy_version 1005008 (0.0006) [2023-12-26 22:41:30,498][105692] Updated weights for policy 0, policy_version 1005018 (0.0008) [2023-12-26 22:41:30,531][105585] KL-divergence is very high: 105.8698 [2023-12-26 22:41:30,556][105692] Updated weights for policy 0, policy_version 1005028 (0.0008) [2023-12-26 22:41:30,736][105620] Updated weights for policy 1, policy_version 1005391 (0.0010) [2023-12-26 22:41:30,790][105620] Updated weights for policy 1, policy_version 1005401 (0.0010) [2023-12-26 22:41:30,838][105620] Updated weights for policy 1, policy_version 1005411 (0.0010) [2023-12-26 22:41:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 514744320. Throughput: 0: 9764.0, 1: 9738.4. Samples: 514714004. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:41:31,063][104569] Avg episode reward: [(0, '8384.964'), (1, '9354.801')] [2023-12-26 22:41:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001005032_257327104.pth... [2023-12-26 22:41:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001005416_257417216.pth... [2023-12-26 22:41:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001003880_257032192.pth [2023-12-26 22:41:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001004264_257122304.pth [2023-12-26 22:41:31,183][105692] Updated weights for policy 0, policy_version 1005038 (0.0007) [2023-12-26 22:41:31,249][105692] Updated weights for policy 0, policy_version 1005048 (0.0006) [2023-12-26 22:41:31,313][105692] Updated weights for policy 0, policy_version 1005058 (0.0006) [2023-12-26 22:41:31,494][105620] Updated weights for policy 1, policy_version 1005421 (0.0008) [2023-12-26 22:41:31,546][105620] Updated weights for policy 1, policy_version 1005431 (0.0007) [2023-12-26 22:41:31,608][105620] Updated weights for policy 1, policy_version 1005441 (0.0010) [2023-12-26 22:41:32,035][105692] Updated weights for policy 0, policy_version 1005068 (0.0007) [2023-12-26 22:41:32,099][105692] Updated weights for policy 0, policy_version 1005078 (0.0005) [2023-12-26 22:41:32,154][105692] Updated weights for policy 0, policy_version 1005088 (0.0005) [2023-12-26 22:41:32,386][105620] Updated weights for policy 1, policy_version 1005451 (0.0008) [2023-12-26 22:41:32,437][105620] Updated weights for policy 1, policy_version 1005461 (0.0009) [2023-12-26 22:41:32,490][105620] Updated weights for policy 1, policy_version 1005471 (0.0009) [2023-12-26 22:41:32,784][105692] Updated weights for policy 0, policy_version 1005098 (0.0008) [2023-12-26 22:41:32,830][105692] Updated weights for policy 0, policy_version 1005108 (0.0008) [2023-12-26 22:41:32,881][105692] Updated weights for policy 0, policy_version 1005118 (0.0009) [2023-12-26 22:41:32,927][105692] Updated weights for policy 0, policy_version 1005128 (0.0008) [2023-12-26 22:41:33,173][105620] Updated weights for policy 1, policy_version 1005481 (0.0008) [2023-12-26 22:41:33,243][105620] Updated weights for policy 1, policy_version 1005491 (0.0005) [2023-12-26 22:41:33,303][105620] Updated weights for policy 1, policy_version 1005501 (0.0007) [2023-12-26 22:41:33,353][105620] Updated weights for policy 1, policy_version 1005511 (0.0009) [2023-12-26 22:41:33,745][105692] Updated weights for policy 0, policy_version 1005138 (0.0010) [2023-12-26 22:41:33,811][105692] Updated weights for policy 0, policy_version 1005148 (0.0007) [2023-12-26 22:41:33,879][105692] Updated weights for policy 0, policy_version 1005158 (0.0005) [2023-12-26 22:41:33,926][105620] Updated weights for policy 1, policy_version 1005521 (0.0006) [2023-12-26 22:41:33,977][105620] Updated weights for policy 1, policy_version 1005531 (0.0005) [2023-12-26 22:41:34,022][105620] Updated weights for policy 1, policy_version 1005541 (0.0005) [2023-12-26 22:41:34,454][105692] Updated weights for policy 0, policy_version 1005168 (0.0009) [2023-12-26 22:41:34,521][105692] Updated weights for policy 0, policy_version 1005178 (0.0011) [2023-12-26 22:41:34,577][105692] Updated weights for policy 0, policy_version 1005188 (0.0010) [2023-12-26 22:41:34,721][105620] Updated weights for policy 1, policy_version 1005551 (0.0009) [2023-12-26 22:41:34,780][105620] Updated weights for policy 1, policy_version 1005561 (0.0010) [2023-12-26 22:41:34,831][105620] Updated weights for policy 1, policy_version 1005571 (0.0010) [2023-12-26 22:41:35,336][105692] Updated weights for policy 0, policy_version 1005198 (0.0011) [2023-12-26 22:41:35,398][105692] Updated weights for policy 0, policy_version 1005208 (0.0007) [2023-12-26 22:41:35,448][105692] Updated weights for policy 0, policy_version 1005218 (0.0005) [2023-12-26 22:41:35,516][105620] Updated weights for policy 1, policy_version 1005581 (0.0010) [2023-12-26 22:41:35,566][105620] Updated weights for policy 1, policy_version 1005591 (0.0011) [2023-12-26 22:41:35,615][105620] Updated weights for policy 1, policy_version 1005601 (0.0010) [2023-12-26 22:41:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.1). Total num frames: 514842624. Throughput: 0: 9799.9, 1: 9703.8. Samples: 514833732. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:41:36,063][104569] Avg episode reward: [(0, '8096.518'), (1, '9171.543')] [2023-12-26 22:41:36,119][105692] Updated weights for policy 0, policy_version 1005228 (0.0008) [2023-12-26 22:41:36,185][105692] Updated weights for policy 0, policy_version 1005238 (0.0011) [2023-12-26 22:41:36,233][105620] Updated weights for policy 1, policy_version 1005611 (0.0007) [2023-12-26 22:41:36,245][105692] Updated weights for policy 0, policy_version 1005248 (0.0011) [2023-12-26 22:41:36,283][105620] Updated weights for policy 1, policy_version 1005621 (0.0008) [2023-12-26 22:41:36,327][105620] Updated weights for policy 1, policy_version 1005631 (0.0008) [2023-12-26 22:41:36,993][105692] Updated weights for policy 0, policy_version 1005258 (0.0011) [2023-12-26 22:41:37,055][105692] Updated weights for policy 0, policy_version 1005268 (0.0011) [2023-12-26 22:41:37,070][105620] Updated weights for policy 1, policy_version 1005641 (0.0008) [2023-12-26 22:41:37,108][105692] Updated weights for policy 0, policy_version 1005278 (0.0011) [2023-12-26 22:41:37,130][105620] Updated weights for policy 1, policy_version 1005651 (0.0006) [2023-12-26 22:41:37,167][105692] Updated weights for policy 0, policy_version 1005288 (0.0011) [2023-12-26 22:41:37,193][105620] Updated weights for policy 1, policy_version 1005661 (0.0006) [2023-12-26 22:41:37,259][105620] Updated weights for policy 1, policy_version 1005671 (0.0008) [2023-12-26 22:41:37,856][105692] Updated weights for policy 0, policy_version 1005298 (0.0010) [2023-12-26 22:41:37,916][105692] Updated weights for policy 0, policy_version 1005308 (0.0008) [2023-12-26 22:41:37,965][105620] Updated weights for policy 1, policy_version 1005681 (0.0008) [2023-12-26 22:41:37,974][105692] Updated weights for policy 0, policy_version 1005318 (0.0007) [2023-12-26 22:41:38,030][105620] Updated weights for policy 1, policy_version 1005691 (0.0009) [2023-12-26 22:41:38,085][105620] Updated weights for policy 1, policy_version 1005701 (0.0010) [2023-12-26 22:41:38,629][105692] Updated weights for policy 0, policy_version 1005328 (0.0009) [2023-12-26 22:41:38,685][105692] Updated weights for policy 0, policy_version 1005338 (0.0006) [2023-12-26 22:41:38,722][105620] Updated weights for policy 1, policy_version 1005711 (0.0011) [2023-12-26 22:41:38,740][105692] Updated weights for policy 0, policy_version 1005348 (0.0008) [2023-12-26 22:41:38,788][105620] Updated weights for policy 1, policy_version 1005721 (0.0010) [2023-12-26 22:41:38,857][105620] Updated weights for policy 1, policy_version 1005731 (0.0009) [2023-12-26 22:41:39,493][105620] Updated weights for policy 1, policy_version 1005741 (0.0006) [2023-12-26 22:41:39,494][105692] Updated weights for policy 0, policy_version 1005358 (0.0009) [2023-12-26 22:41:39,549][105620] Updated weights for policy 1, policy_version 1005751 (0.0006) [2023-12-26 22:41:39,554][105692] Updated weights for policy 0, policy_version 1005368 (0.0011) [2023-12-26 22:41:39,613][105620] Updated weights for policy 1, policy_version 1005761 (0.0009) [2023-12-26 22:41:39,621][105692] Updated weights for policy 0, policy_version 1005378 (0.0011) [2023-12-26 22:41:40,336][105620] Updated weights for policy 1, policy_version 1005771 (0.0007) [2023-12-26 22:41:40,374][105692] Updated weights for policy 0, policy_version 1005388 (0.0011) [2023-12-26 22:41:40,393][105620] Updated weights for policy 1, policy_version 1005781 (0.0007) [2023-12-26 22:41:40,430][105692] Updated weights for policy 0, policy_version 1005398 (0.0011) [2023-12-26 22:41:40,443][105620] Updated weights for policy 1, policy_version 1005791 (0.0007) [2023-12-26 22:41:40,490][105692] Updated weights for policy 0, policy_version 1005408 (0.0009) [2023-12-26 22:41:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 514940928. Throughput: 0: 9764.8, 1: 9743.2. Samples: 514952304. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-12-26 22:41:41,062][104569] Avg episode reward: [(0, '8185.315'), (1, '8987.472')] [2023-12-26 22:41:41,211][105620] Updated weights for policy 1, policy_version 1005801 (0.0007) [2023-12-26 22:41:41,254][105692] Updated weights for policy 0, policy_version 1005418 (0.0010) [2023-12-26 22:41:41,277][105620] Updated weights for policy 1, policy_version 1005811 (0.0008) [2023-12-26 22:41:41,321][105692] Updated weights for policy 0, policy_version 1005428 (0.0008) [2023-12-26 22:41:41,331][105620] Updated weights for policy 1, policy_version 1005821 (0.0007) [2023-12-26 22:41:41,395][105692] Updated weights for policy 0, policy_version 1005438 (0.0009) [2023-12-26 22:41:41,414][105620] Updated weights for policy 1, policy_version 1005831 (0.0009) [2023-12-26 22:41:41,457][105692] Updated weights for policy 0, policy_version 1005448 (0.0009) [2023-12-26 22:41:42,179][105620] Updated weights for policy 1, policy_version 1005841 (0.0006) [2023-12-26 22:41:42,241][105620] Updated weights for policy 1, policy_version 1005851 (0.0006) [2023-12-26 22:41:42,285][105692] Updated weights for policy 0, policy_version 1005458 (0.0011) [2023-12-26 22:41:42,304][105620] Updated weights for policy 1, policy_version 1005861 (0.0006) [2023-12-26 22:41:42,349][105692] Updated weights for policy 0, policy_version 1005468 (0.0011) [2023-12-26 22:41:42,412][105692] Updated weights for policy 0, policy_version 1005478 (0.0011) [2023-12-26 22:41:43,026][105620] Updated weights for policy 1, policy_version 1005871 (0.0009) [2023-12-26 22:41:43,070][105692] Updated weights for policy 0, policy_version 1005488 (0.0007) [2023-12-26 22:41:43,084][105620] Updated weights for policy 1, policy_version 1005881 (0.0008) [2023-12-26 22:41:43,126][105692] Updated weights for policy 0, policy_version 1005498 (0.0006) [2023-12-26 22:41:43,143][105620] Updated weights for policy 1, policy_version 1005891 (0.0007) [2023-12-26 22:41:43,179][105692] Updated weights for policy 0, policy_version 1005508 (0.0005) [2023-12-26 22:41:43,876][105620] Updated weights for policy 1, policy_version 1005901 (0.0009) [2023-12-26 22:41:43,890][105692] Updated weights for policy 0, policy_version 1005518 (0.0006) [2023-12-26 22:41:43,919][105620] Updated weights for policy 1, policy_version 1005911 (0.0009) [2023-12-26 22:41:43,938][105692] Updated weights for policy 0, policy_version 1005528 (0.0007) [2023-12-26 22:41:43,966][105620] Updated weights for policy 1, policy_version 1005921 (0.0005) [2023-12-26 22:41:43,994][105692] Updated weights for policy 0, policy_version 1005538 (0.0009) [2023-12-26 22:41:44,706][105620] Updated weights for policy 1, policy_version 1005931 (0.0010) [2023-12-26 22:41:44,751][105620] Updated weights for policy 1, policy_version 1005941 (0.0010) [2023-12-26 22:41:44,784][105692] Updated weights for policy 0, policy_version 1005548 (0.0009) [2023-12-26 22:41:44,819][105620] Updated weights for policy 1, policy_version 1005951 (0.0011) [2023-12-26 22:41:44,849][105692] Updated weights for policy 0, policy_version 1005558 (0.0006) [2023-12-26 22:41:44,902][105692] Updated weights for policy 0, policy_version 1005568 (0.0007) [2023-12-26 22:41:45,573][105620] Updated weights for policy 1, policy_version 1005961 (0.0011) [2023-12-26 22:41:45,636][105620] Updated weights for policy 1, policy_version 1005971 (0.0010) [2023-12-26 22:41:45,640][105692] Updated weights for policy 0, policy_version 1005578 (0.0008) [2023-12-26 22:41:45,700][105692] Updated weights for policy 0, policy_version 1005588 (0.0007) [2023-12-26 22:41:45,701][105620] Updated weights for policy 1, policy_version 1005981 (0.0011) [2023-12-26 22:41:45,755][105692] Updated weights for policy 0, policy_version 1005598 (0.0008) [2023-12-26 22:41:45,765][105620] Updated weights for policy 1, policy_version 1005991 (0.0011) [2023-12-26 22:41:45,812][105692] Updated weights for policy 0, policy_version 1005608 (0.0007) [2023-12-26 22:41:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 515039232. Throughput: 0: 9787.5, 1: 9742.2. Samples: 515008988. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:41:46,062][104569] Avg episode reward: [(0, '8638.794'), (1, '8986.263')] [2023-12-26 22:41:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001005608_257474560.pth... [2023-12-26 22:41:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001005992_257564672.pth... [2023-12-26 22:41:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001004456_257179648.pth [2023-12-26 22:41:46,074][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_001005608_257474560.pth [2023-12-26 22:41:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001004840_257269760.pth [2023-12-26 22:41:46,075][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_001005992_257564672.pth [2023-12-26 22:41:46,450][105692] Updated weights for policy 0, policy_version 1005618 (0.0010) [2023-12-26 22:41:46,455][105620] Updated weights for policy 1, policy_version 1006001 (0.0011) [2023-12-26 22:41:46,502][105692] Updated weights for policy 0, policy_version 1005628 (0.0011) [2023-12-26 22:41:46,511][105620] Updated weights for policy 1, policy_version 1006011 (0.0010) [2023-12-26 22:41:46,562][105692] Updated weights for policy 0, policy_version 1005638 (0.0010) [2023-12-26 22:41:46,569][105620] Updated weights for policy 1, policy_version 1006021 (0.0010) [2023-12-26 22:41:47,274][105692] Updated weights for policy 0, policy_version 1005648 (0.0006) [2023-12-26 22:41:47,313][105620] Updated weights for policy 1, policy_version 1006031 (0.0010) [2023-12-26 22:41:47,340][105692] Updated weights for policy 0, policy_version 1005658 (0.0009) [2023-12-26 22:41:47,362][105620] Updated weights for policy 1, policy_version 1006041 (0.0010) [2023-12-26 22:41:47,398][105692] Updated weights for policy 0, policy_version 1005668 (0.0010) [2023-12-26 22:41:47,419][105620] Updated weights for policy 1, policy_version 1006051 (0.0010) [2023-12-26 22:41:48,086][105692] Updated weights for policy 0, policy_version 1005678 (0.0010) [2023-12-26 22:41:48,133][105692] Updated weights for policy 0, policy_version 1005688 (0.0010) [2023-12-26 22:41:48,177][105620] Updated weights for policy 1, policy_version 1006061 (0.0010) [2023-12-26 22:41:48,188][105692] Updated weights for policy 0, policy_version 1005698 (0.0010) [2023-12-26 22:41:48,226][105620] Updated weights for policy 1, policy_version 1006071 (0.0010) [2023-12-26 22:41:48,274][105620] Updated weights for policy 1, policy_version 1006081 (0.0010) [2023-12-26 22:41:48,857][105620] Updated weights for policy 1, policy_version 1006091 (0.0009) [2023-12-26 22:41:48,906][105620] Updated weights for policy 1, policy_version 1006101 (0.0005) [2023-12-26 22:41:48,959][105620] Updated weights for policy 1, policy_version 1006111 (0.0008) [2023-12-26 22:41:48,961][105692] Updated weights for policy 0, policy_version 1005708 (0.0007) [2023-12-26 22:41:49,022][105692] Updated weights for policy 0, policy_version 1005718 (0.0006) [2023-12-26 22:41:49,080][105692] Updated weights for policy 0, policy_version 1005728 (0.0005) [2023-12-26 22:41:49,631][105620] Updated weights for policy 1, policy_version 1006121 (0.0011) [2023-12-26 22:41:49,653][105692] Updated weights for policy 0, policy_version 1005738 (0.0006) [2023-12-26 22:41:49,691][105620] Updated weights for policy 1, policy_version 1006131 (0.0011) [2023-12-26 22:41:49,701][105692] Updated weights for policy 0, policy_version 1005748 (0.0005) [2023-12-26 22:41:49,755][105620] Updated weights for policy 1, policy_version 1006141 (0.0010) [2023-12-26 22:41:49,761][105692] Updated weights for policy 0, policy_version 1005758 (0.0008) [2023-12-26 22:41:49,815][105692] Updated weights for policy 0, policy_version 1005768 (0.0008) [2023-12-26 22:41:49,815][105620] Updated weights for policy 1, policy_version 1006151 (0.0011) [2023-12-26 22:41:50,511][105692] Updated weights for policy 0, policy_version 1005778 (0.0010) [2023-12-26 22:41:50,514][105620] Updated weights for policy 1, policy_version 1006161 (0.0010) [2023-12-26 22:41:50,568][105692] Updated weights for policy 0, policy_version 1005788 (0.0012) [2023-12-26 22:41:50,576][105620] Updated weights for policy 1, policy_version 1006171 (0.0011) [2023-12-26 22:41:50,631][105692] Updated weights for policy 0, policy_version 1005798 (0.0008) [2023-12-26 22:41:50,639][105620] Updated weights for policy 1, policy_version 1006181 (0.0011) [2023-12-26 22:41:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 515137536. Throughput: 0: 9804.7, 1: 9834.1. Samples: 515127284. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:41:51,062][104569] Avg episode reward: [(0, '8728.729'), (1, '8985.487')] [2023-12-26 22:41:51,297][105692] Updated weights for policy 0, policy_version 1005808 (0.0007) [2023-12-26 22:41:51,335][105620] Updated weights for policy 1, policy_version 1006191 (0.0011) [2023-12-26 22:41:51,359][105692] Updated weights for policy 0, policy_version 1005818 (0.0007) [2023-12-26 22:41:51,400][105620] Updated weights for policy 1, policy_version 1006201 (0.0008) [2023-12-26 22:41:51,421][105692] Updated weights for policy 0, policy_version 1005828 (0.0011) [2023-12-26 22:41:51,457][105620] Updated weights for policy 1, policy_version 1006211 (0.0009) [2023-12-26 22:41:52,153][105692] Updated weights for policy 0, policy_version 1005838 (0.0008) [2023-12-26 22:41:52,193][105620] Updated weights for policy 1, policy_version 1006221 (0.0010) [2023-12-26 22:41:52,214][105692] Updated weights for policy 0, policy_version 1005848 (0.0007) [2023-12-26 22:41:52,244][105620] Updated weights for policy 1, policy_version 1006231 (0.0010) [2023-12-26 22:41:52,276][105692] Updated weights for policy 0, policy_version 1005858 (0.0008) [2023-12-26 22:41:52,311][105620] Updated weights for policy 1, policy_version 1006241 (0.0011) [2023-12-26 22:41:53,020][105692] Updated weights for policy 0, policy_version 1005868 (0.0008) [2023-12-26 22:41:53,029][105620] Updated weights for policy 1, policy_version 1006251 (0.0011) [2023-12-26 22:41:53,067][105692] Updated weights for policy 0, policy_version 1005878 (0.0007) [2023-12-26 22:41:53,080][105620] Updated weights for policy 1, policy_version 1006261 (0.0010) [2023-12-26 22:41:53,111][105692] Updated weights for policy 0, policy_version 1005888 (0.0007) [2023-12-26 22:41:53,128][105620] Updated weights for policy 1, policy_version 1006271 (0.0010) [2023-12-26 22:41:53,688][105620] Updated weights for policy 1, policy_version 1006281 (0.0007) [2023-12-26 22:41:53,756][105620] Updated weights for policy 1, policy_version 1006291 (0.0005) [2023-12-26 22:41:53,814][105620] Updated weights for policy 1, policy_version 1006301 (0.0006) [2023-12-26 22:41:53,867][105620] Updated weights for policy 1, policy_version 1006311 (0.0005) [2023-12-26 22:41:54,028][105692] Updated weights for policy 0, policy_version 1005898 (0.0010) [2023-12-26 22:41:54,082][105692] Updated weights for policy 0, policy_version 1005909 (0.0010) [2023-12-26 22:41:54,134][105692] Updated weights for policy 0, policy_version 1005920 (0.0009) [2023-12-26 22:41:54,393][105620] Updated weights for policy 1, policy_version 1006321 (0.0010) [2023-12-26 22:41:54,449][105620] Updated weights for policy 1, policy_version 1006331 (0.0010) [2023-12-26 22:41:54,497][105620] Updated weights for policy 1, policy_version 1006341 (0.0009) [2023-12-26 22:41:54,957][105692] Updated weights for policy 0, policy_version 1005931 (0.0009) [2023-12-26 22:41:55,010][105692] Updated weights for policy 0, policy_version 1005941 (0.0008) [2023-12-26 22:41:55,065][105692] Updated weights for policy 0, policy_version 1005951 (0.0008) [2023-12-26 22:41:55,255][105620] Updated weights for policy 1, policy_version 1006351 (0.0010) [2023-12-26 22:41:55,303][105620] Updated weights for policy 1, policy_version 1006361 (0.0010) [2023-12-26 22:41:55,351][105620] Updated weights for policy 1, policy_version 1006371 (0.0010) [2023-12-26 22:41:55,854][105692] Updated weights for policy 0, policy_version 1005961 (0.0009) [2023-12-26 22:41:55,923][105692] Updated weights for policy 0, policy_version 1005971 (0.0010) [2023-12-26 22:41:55,990][105692] Updated weights for policy 0, policy_version 1005981 (0.0005) [2023-12-26 22:41:56,001][105620] Updated weights for policy 1, policy_version 1006381 (0.0008) [2023-12-26 22:41:56,050][105692] Updated weights for policy 0, policy_version 1005991 (0.0006) [2023-12-26 22:41:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.9, 300 sec: 19522.0). Total num frames: 515235840. Throughput: 0: 9754.9, 1: 9970.2. Samples: 515244796. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:41:56,062][104569] Avg episode reward: [(0, '8726.551'), (1, '9078.382')] [2023-12-26 22:41:56,065][105620] Updated weights for policy 1, policy_version 1006391 (0.0006) [2023-12-26 22:41:56,120][105620] Updated weights for policy 1, policy_version 1006401 (0.0005) [2023-12-26 22:41:56,703][105620] Updated weights for policy 1, policy_version 1006411 (0.0006) [2023-12-26 22:41:56,750][105692] Updated weights for policy 0, policy_version 1006001 (0.0011) [2023-12-26 22:41:56,759][105620] Updated weights for policy 1, policy_version 1006421 (0.0006) [2023-12-26 22:41:56,813][105692] Updated weights for policy 0, policy_version 1006011 (0.0011) [2023-12-26 22:41:56,820][105620] Updated weights for policy 1, policy_version 1006431 (0.0006) [2023-12-26 22:41:56,877][105692] Updated weights for policy 0, policy_version 1006021 (0.0011) [2023-12-26 22:41:57,353][105620] Updated weights for policy 1, policy_version 1006441 (0.0006) [2023-12-26 22:41:57,402][105620] Updated weights for policy 1, policy_version 1006451 (0.0005) [2023-12-26 22:41:57,449][105620] Updated weights for policy 1, policy_version 1006461 (0.0007) [2023-12-26 22:41:57,493][105620] Updated weights for policy 1, policy_version 1006471 (0.0010) [2023-12-26 22:41:57,582][105692] Updated weights for policy 0, policy_version 1006031 (0.0007) [2023-12-26 22:41:57,635][105692] Updated weights for policy 0, policy_version 1006041 (0.0005) [2023-12-26 22:41:57,693][105692] Updated weights for policy 0, policy_version 1006051 (0.0005) [2023-12-26 22:41:58,280][105692] Updated weights for policy 0, policy_version 1006061 (0.0008) [2023-12-26 22:41:58,295][105620] Updated weights for policy 1, policy_version 1006481 (0.0008) [2023-12-26 22:41:58,343][105692] Updated weights for policy 0, policy_version 1006071 (0.0011) [2023-12-26 22:41:58,359][105620] Updated weights for policy 1, policy_version 1006491 (0.0007) [2023-12-26 22:41:58,410][105692] Updated weights for policy 0, policy_version 1006081 (0.0011) [2023-12-26 22:41:58,424][105620] Updated weights for policy 1, policy_version 1006501 (0.0007) [2023-12-26 22:41:59,238][105620] Updated weights for policy 1, policy_version 1006511 (0.0009) [2023-12-26 22:41:59,241][105692] Updated weights for policy 0, policy_version 1006091 (0.0010) [2023-12-26 22:41:59,307][105620] Updated weights for policy 1, policy_version 1006521 (0.0008) [2023-12-26 22:41:59,308][105692] Updated weights for policy 0, policy_version 1006101 (0.0007) [2023-12-26 22:41:59,370][105692] Updated weights for policy 0, policy_version 1006111 (0.0007) [2023-12-26 22:41:59,374][105620] Updated weights for policy 1, policy_version 1006531 (0.0008) [2023-12-26 22:42:00,084][105692] Updated weights for policy 0, policy_version 1006121 (0.0007) [2023-12-26 22:42:00,103][105620] Updated weights for policy 1, policy_version 1006541 (0.0008) [2023-12-26 22:42:00,143][105692] Updated weights for policy 0, policy_version 1006131 (0.0007) [2023-12-26 22:42:00,157][105620] Updated weights for policy 1, policy_version 1006551 (0.0008) [2023-12-26 22:42:00,196][105692] Updated weights for policy 0, policy_version 1006141 (0.0008) [2023-12-26 22:42:00,215][105620] Updated weights for policy 1, policy_version 1006561 (0.0008) [2023-12-26 22:42:00,254][105692] Updated weights for policy 0, policy_version 1006151 (0.0007) [2023-12-26 22:42:00,890][105620] Updated weights for policy 1, policy_version 1006571 (0.0007) [2023-12-26 22:42:00,923][105692] Updated weights for policy 0, policy_version 1006161 (0.0009) [2023-12-26 22:42:00,950][105620] Updated weights for policy 1, policy_version 1006581 (0.0006) [2023-12-26 22:42:00,975][105692] Updated weights for policy 0, policy_version 1006171 (0.0010) [2023-12-26 22:42:00,998][105620] Updated weights for policy 1, policy_version 1006591 (0.0006) [2023-12-26 22:42:01,019][105692] Updated weights for policy 0, policy_version 1006181 (0.0008) [2023-12-26 22:42:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 515342336. Throughput: 0: 9798.0, 1: 10022.4. Samples: 515305456. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:42:01,062][104569] Avg episode reward: [(0, '8638.890'), (1, '9170.803')] [2023-12-26 22:42:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001006184_257622016.pth... [2023-12-26 22:42:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001006600_257720320.pth... [2023-12-26 22:42:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001005416_257417216.pth [2023-12-26 22:42:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001005032_257327104.pth [2023-12-26 22:42:01,707][105692] Updated weights for policy 0, policy_version 1006191 (0.0009) [2023-12-26 22:42:01,745][105620] Updated weights for policy 1, policy_version 1006601 (0.0008) [2023-12-26 22:42:01,767][105692] Updated weights for policy 0, policy_version 1006201 (0.0011) [2023-12-26 22:42:01,798][105620] Updated weights for policy 1, policy_version 1006611 (0.0011) [2023-12-26 22:42:01,823][105692] Updated weights for policy 0, policy_version 1006211 (0.0011) [2023-12-26 22:42:01,858][105620] Updated weights for policy 1, policy_version 1006621 (0.0010) [2023-12-26 22:42:01,920][105620] Updated weights for policy 1, policy_version 1006631 (0.0010) [2023-12-26 22:42:02,496][105692] Updated weights for policy 0, policy_version 1006221 (0.0008) [2023-12-26 22:42:02,548][105692] Updated weights for policy 0, policy_version 1006231 (0.0005) [2023-12-26 22:42:02,618][105692] Updated weights for policy 0, policy_version 1006241 (0.0006) [2023-12-26 22:42:02,667][105620] Updated weights for policy 1, policy_version 1006641 (0.0006) [2023-12-26 22:42:02,723][105620] Updated weights for policy 1, policy_version 1006651 (0.0009) [2023-12-26 22:42:02,790][105620] Updated weights for policy 1, policy_version 1006661 (0.0009) [2023-12-26 22:42:03,271][105692] Updated weights for policy 0, policy_version 1006251 (0.0007) [2023-12-26 22:42:03,319][105692] Updated weights for policy 0, policy_version 1006261 (0.0005) [2023-12-26 22:42:03,367][105692] Updated weights for policy 0, policy_version 1006271 (0.0005) [2023-12-26 22:42:03,429][105620] Updated weights for policy 1, policy_version 1006671 (0.0005) [2023-12-26 22:42:03,481][105620] Updated weights for policy 1, policy_version 1006681 (0.0005) [2023-12-26 22:42:03,533][105620] Updated weights for policy 1, policy_version 1006691 (0.0006) [2023-12-26 22:42:04,048][105692] Updated weights for policy 0, policy_version 1006281 (0.0006) [2023-12-26 22:42:04,110][105692] Updated weights for policy 0, policy_version 1006291 (0.0011) [2023-12-26 22:42:04,173][105692] Updated weights for policy 0, policy_version 1006301 (0.0011) [2023-12-26 22:42:04,233][105692] Updated weights for policy 0, policy_version 1006311 (0.0011) [2023-12-26 22:42:04,260][105620] Updated weights for policy 1, policy_version 1006701 (0.0007) [2023-12-26 22:42:04,315][105620] Updated weights for policy 1, policy_version 1006711 (0.0008) [2023-12-26 22:42:04,374][105620] Updated weights for policy 1, policy_version 1006721 (0.0008) [2023-12-26 22:42:04,979][105692] Updated weights for policy 0, policy_version 1006321 (0.0011) [2023-12-26 22:42:05,028][105692] Updated weights for policy 0, policy_version 1006331 (0.0011) [2023-12-26 22:42:05,079][105692] Updated weights for policy 0, policy_version 1006341 (0.0010) [2023-12-26 22:42:05,132][105620] Updated weights for policy 1, policy_version 1006731 (0.0008) [2023-12-26 22:42:05,184][105620] Updated weights for policy 1, policy_version 1006741 (0.0008) [2023-12-26 22:42:05,228][105620] Updated weights for policy 1, policy_version 1006751 (0.0008) [2023-12-26 22:42:05,833][105692] Updated weights for policy 0, policy_version 1006351 (0.0010) [2023-12-26 22:42:05,881][105692] Updated weights for policy 0, policy_version 1006361 (0.0010) [2023-12-26 22:42:05,930][105692] Updated weights for policy 0, policy_version 1006371 (0.0010) [2023-12-26 22:42:05,998][105620] Updated weights for policy 1, policy_version 1006761 (0.0008) [2023-12-26 22:42:06,053][105620] Updated weights for policy 1, policy_version 1006771 (0.0008) [2023-12-26 22:42:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 515432448. Throughput: 0: 9742.2, 1: 9922.0. Samples: 515423128. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:42:06,062][104569] Avg episode reward: [(0, '8640.248'), (1, '9078.539')] [2023-12-26 22:42:06,113][105620] Updated weights for policy 1, policy_version 1006781 (0.0008) [2023-12-26 22:42:06,170][105620] Updated weights for policy 1, policy_version 1006791 (0.0008) [2023-12-26 22:42:06,700][105692] Updated weights for policy 0, policy_version 1006381 (0.0011) [2023-12-26 22:42:06,759][105692] Updated weights for policy 0, policy_version 1006391 (0.0011) [2023-12-26 22:42:06,814][105692] Updated weights for policy 0, policy_version 1006401 (0.0011) [2023-12-26 22:42:06,942][105620] Updated weights for policy 1, policy_version 1006801 (0.0008) [2023-12-26 22:42:06,992][105620] Updated weights for policy 1, policy_version 1006811 (0.0008) [2023-12-26 22:42:07,052][105620] Updated weights for policy 1, policy_version 1006821 (0.0008) [2023-12-26 22:42:07,576][105692] Updated weights for policy 0, policy_version 1006411 (0.0011) [2023-12-26 22:42:07,624][105692] Updated weights for policy 0, policy_version 1006421 (0.0010) [2023-12-26 22:42:07,672][105692] Updated weights for policy 0, policy_version 1006431 (0.0010) [2023-12-26 22:42:07,824][105620] Updated weights for policy 1, policy_version 1006831 (0.0008) [2023-12-26 22:42:07,889][105620] Updated weights for policy 1, policy_version 1006841 (0.0008) [2023-12-26 22:42:07,948][105620] Updated weights for policy 1, policy_version 1006851 (0.0008) [2023-12-26 22:42:08,438][105692] Updated weights for policy 0, policy_version 1006441 (0.0010) [2023-12-26 22:42:08,496][105692] Updated weights for policy 0, policy_version 1006451 (0.0010) [2023-12-26 22:42:08,558][105692] Updated weights for policy 0, policy_version 1006461 (0.0011) [2023-12-26 22:42:08,609][105692] Updated weights for policy 0, policy_version 1006471 (0.0010) [2023-12-26 22:42:08,679][105620] Updated weights for policy 1, policy_version 1006861 (0.0007) [2023-12-26 22:42:08,734][105620] Updated weights for policy 1, policy_version 1006871 (0.0008) [2023-12-26 22:42:08,793][105620] Updated weights for policy 1, policy_version 1006881 (0.0009) [2023-12-26 22:42:09,349][105692] Updated weights for policy 0, policy_version 1006481 (0.0010) [2023-12-26 22:42:09,414][105692] Updated weights for policy 0, policy_version 1006491 (0.0011) [2023-12-26 22:42:09,426][105620] Updated weights for policy 1, policy_version 1006891 (0.0008) [2023-12-26 22:42:09,474][105692] Updated weights for policy 0, policy_version 1006501 (0.0011) [2023-12-26 22:42:09,494][105620] Updated weights for policy 1, policy_version 1006901 (0.0006) [2023-12-26 22:42:09,557][105620] Updated weights for policy 1, policy_version 1006911 (0.0005) [2023-12-26 22:42:10,199][105620] Updated weights for policy 1, policy_version 1006921 (0.0006) [2023-12-26 22:42:10,254][105692] Updated weights for policy 0, policy_version 1006511 (0.0009) [2023-12-26 22:42:10,256][105620] Updated weights for policy 1, policy_version 1006931 (0.0007) [2023-12-26 22:42:10,308][105692] Updated weights for policy 0, policy_version 1006521 (0.0007) [2023-12-26 22:42:10,310][105620] Updated weights for policy 1, policy_version 1006941 (0.0006) [2023-12-26 22:42:10,364][105692] Updated weights for policy 0, policy_version 1006531 (0.0007) [2023-12-26 22:42:10,366][105620] Updated weights for policy 1, policy_version 1006951 (0.0007) [2023-12-26 22:42:11,031][105620] Updated weights for policy 1, policy_version 1006961 (0.0008) [2023-12-26 22:42:11,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 515522560. Throughput: 0: 9725.3, 1: 9974.1. Samples: 515537372. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:42:11,062][104569] Avg episode reward: [(0, '8362.471'), (1, '9078.818')] [2023-12-26 22:42:11,097][105620] Updated weights for policy 1, policy_version 1006971 (0.0009) [2023-12-26 22:42:11,108][105692] Updated weights for policy 0, policy_version 1006541 (0.0008) [2023-12-26 22:42:11,165][105620] Updated weights for policy 1, policy_version 1006981 (0.0007) [2023-12-26 22:42:11,178][105692] Updated weights for policy 0, policy_version 1006551 (0.0007) [2023-12-26 22:42:11,242][105692] Updated weights for policy 0, policy_version 1006561 (0.0009) [2023-12-26 22:42:11,916][105620] Updated weights for policy 1, policy_version 1006991 (0.0008) [2023-12-26 22:42:11,977][105620] Updated weights for policy 1, policy_version 1007001 (0.0010) [2023-12-26 22:42:12,031][105692] Updated weights for policy 0, policy_version 1006571 (0.0009) [2023-12-26 22:42:12,046][105620] Updated weights for policy 1, policy_version 1007011 (0.0009) [2023-12-26 22:42:12,095][105692] Updated weights for policy 0, policy_version 1006581 (0.0007) [2023-12-26 22:42:12,157][105692] Updated weights for policy 0, policy_version 1006591 (0.0009) [2023-12-26 22:42:12,810][105620] Updated weights for policy 1, policy_version 1007021 (0.0008) [2023-12-26 22:42:12,820][105692] Updated weights for policy 0, policy_version 1006601 (0.0009) [2023-12-26 22:42:12,856][105620] Updated weights for policy 1, policy_version 1007031 (0.0008) [2023-12-26 22:42:12,869][105692] Updated weights for policy 0, policy_version 1006611 (0.0008) [2023-12-26 22:42:12,909][105620] Updated weights for policy 1, policy_version 1007041 (0.0008) [2023-12-26 22:42:12,927][105692] Updated weights for policy 0, policy_version 1006621 (0.0008) [2023-12-26 22:42:12,991][105692] Updated weights for policy 0, policy_version 1006631 (0.0006) [2023-12-26 22:42:13,613][105692] Updated weights for policy 0, policy_version 1006641 (0.0009) [2023-12-26 22:42:13,665][105692] Updated weights for policy 0, policy_version 1006651 (0.0006) [2023-12-26 22:42:13,718][105692] Updated weights for policy 0, policy_version 1006661 (0.0005) [2023-12-26 22:42:13,742][105620] Updated weights for policy 1, policy_version 1007051 (0.0008) [2023-12-26 22:42:13,795][105620] Updated weights for policy 1, policy_version 1007061 (0.0005) [2023-12-26 22:42:13,845][105620] Updated weights for policy 1, policy_version 1007071 (0.0005) [2023-12-26 22:42:14,313][105692] Updated weights for policy 0, policy_version 1006671 (0.0008) [2023-12-26 22:42:14,373][105692] Updated weights for policy 0, policy_version 1006681 (0.0009) [2023-12-26 22:42:14,439][105692] Updated weights for policy 0, policy_version 1006691 (0.0009) [2023-12-26 22:42:14,506][105620] Updated weights for policy 1, policy_version 1007081 (0.0006) [2023-12-26 22:42:14,556][105620] Updated weights for policy 1, policy_version 1007091 (0.0005) [2023-12-26 22:42:14,604][105620] Updated weights for policy 1, policy_version 1007101 (0.0007) [2023-12-26 22:42:14,657][105620] Updated weights for policy 1, policy_version 1007111 (0.0009) [2023-12-26 22:42:15,119][105692] Updated weights for policy 0, policy_version 1006701 (0.0008) [2023-12-26 22:42:15,181][105692] Updated weights for policy 0, policy_version 1006711 (0.0009) [2023-12-26 22:42:15,244][105692] Updated weights for policy 0, policy_version 1006721 (0.0009) [2023-12-26 22:42:15,465][105620] Updated weights for policy 1, policy_version 1007121 (0.0009) [2023-12-26 22:42:15,525][105620] Updated weights for policy 1, policy_version 1007131 (0.0009) [2023-12-26 22:42:15,586][105620] Updated weights for policy 1, policy_version 1007141 (0.0009) [2023-12-26 22:42:15,900][105692] Updated weights for policy 0, policy_version 1006731 (0.0009) [2023-12-26 22:42:15,971][105692] Updated weights for policy 0, policy_version 1006741 (0.0008) [2023-12-26 22:42:16,033][105692] Updated weights for policy 0, policy_version 1006751 (0.0008) [2023-12-26 22:42:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 515620864. Throughput: 0: 9666.8, 1: 9898.0. Samples: 515594420. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:42:16,063][104569] Avg episode reward: [(0, '8723.516'), (1, '9262.652')] [2023-12-26 22:42:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001007144_257859584.pth... [2023-12-26 22:42:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001005992_257564672.pth [2023-12-26 22:42:16,086][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001006760_257769472.pth... [2023-12-26 22:42:16,089][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001005608_257474560.pth [2023-12-26 22:42:16,336][105620] Updated weights for policy 1, policy_version 1007151 (0.0007) [2023-12-26 22:42:16,394][105620] Updated weights for policy 1, policy_version 1007161 (0.0005) [2023-12-26 22:42:16,439][105620] Updated weights for policy 1, policy_version 1007171 (0.0005) [2023-12-26 22:42:16,785][105692] Updated weights for policy 0, policy_version 1006761 (0.0008) [2023-12-26 22:42:16,839][105692] Updated weights for policy 0, policy_version 1006771 (0.0010) [2023-12-26 22:42:16,859][105585] KL-divergence is very high: 180.3086 [2023-12-26 22:42:16,903][105692] Updated weights for policy 0, policy_version 1006782 (0.0010) [2023-12-26 22:42:16,916][105585] KL-divergence is very high: 243.1443 [2023-12-26 22:42:16,970][105692] Updated weights for policy 0, policy_version 1006792 (0.0010) [2023-12-26 22:42:17,055][105620] Updated weights for policy 1, policy_version 1007181 (0.0006) [2023-12-26 22:42:17,117][105620] Updated weights for policy 1, policy_version 1007191 (0.0006) [2023-12-26 22:42:17,170][105620] Updated weights for policy 1, policy_version 1007201 (0.0008) [2023-12-26 22:42:17,799][105620] Updated weights for policy 1, policy_version 1007211 (0.0008) [2023-12-26 22:42:17,821][105692] Updated weights for policy 0, policy_version 1006802 (0.0010) [2023-12-26 22:42:17,848][105620] Updated weights for policy 1, policy_version 1007221 (0.0006) [2023-12-26 22:42:17,883][105692] Updated weights for policy 0, policy_version 1006812 (0.0010) [2023-12-26 22:42:17,897][105620] Updated weights for policy 1, policy_version 1007231 (0.0007) [2023-12-26 22:42:17,931][105692] Updated weights for policy 0, policy_version 1006822 (0.0010) [2023-12-26 22:42:18,669][105620] Updated weights for policy 1, policy_version 1007241 (0.0006) [2023-12-26 22:42:18,681][105692] Updated weights for policy 0, policy_version 1006832 (0.0009) [2023-12-26 22:42:18,726][105620] Updated weights for policy 1, policy_version 1007251 (0.0011) [2023-12-26 22:42:18,736][105692] Updated weights for policy 0, policy_version 1006842 (0.0006) [2023-12-26 22:42:18,788][105620] Updated weights for policy 1, policy_version 1007261 (0.0011) [2023-12-26 22:42:18,794][105692] Updated weights for policy 0, policy_version 1006852 (0.0006) [2023-12-26 22:42:18,850][105620] Updated weights for policy 1, policy_version 1007271 (0.0009) [2023-12-26 22:42:19,405][105692] Updated weights for policy 0, policy_version 1006862 (0.0008) [2023-12-26 22:42:19,463][105692] Updated weights for policy 0, policy_version 1006872 (0.0010) [2023-12-26 22:42:19,533][105692] Updated weights for policy 0, policy_version 1006882 (0.0007) [2023-12-26 22:42:19,558][105620] Updated weights for policy 1, policy_version 1007281 (0.0007) [2023-12-26 22:42:19,620][105620] Updated weights for policy 1, policy_version 1007291 (0.0009) [2023-12-26 22:42:19,682][105620] Updated weights for policy 1, policy_version 1007301 (0.0011) [2023-12-26 22:42:20,293][105692] Updated weights for policy 0, policy_version 1006892 (0.0009) [2023-12-26 22:42:20,349][105692] Updated weights for policy 0, policy_version 1006902 (0.0010) [2023-12-26 22:42:20,395][105620] Updated weights for policy 1, policy_version 1007311 (0.0008) [2023-12-26 22:42:20,416][105692] Updated weights for policy 0, policy_version 1006912 (0.0009) [2023-12-26 22:42:20,456][105620] Updated weights for policy 1, policy_version 1007321 (0.0009) [2023-12-26 22:42:20,515][105620] Updated weights for policy 1, policy_version 1007331 (0.0007) [2023-12-26 22:42:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 515719168. Throughput: 0: 9662.4, 1: 9858.7. Samples: 515712184. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:42:21,063][104569] Avg episode reward: [(0, '8994.006'), (1, '9262.925')] [2023-12-26 22:42:21,137][105692] Updated weights for policy 0, policy_version 1006922 (0.0006) [2023-12-26 22:42:21,197][105692] Updated weights for policy 0, policy_version 1006932 (0.0008) [2023-12-26 22:42:21,264][105692] Updated weights for policy 0, policy_version 1006942 (0.0008) [2023-12-26 22:42:21,283][105620] Updated weights for policy 1, policy_version 1007341 (0.0008) [2023-12-26 22:42:21,317][105692] Updated weights for policy 0, policy_version 1006952 (0.0008) [2023-12-26 22:42:21,345][105620] Updated weights for policy 1, policy_version 1007351 (0.0008) [2023-12-26 22:42:21,415][105620] Updated weights for policy 1, policy_version 1007361 (0.0009) [2023-12-26 22:42:22,052][105692] Updated weights for policy 0, policy_version 1006962 (0.0009) [2023-12-26 22:42:22,115][105692] Updated weights for policy 0, policy_version 1006972 (0.0009) [2023-12-26 22:42:22,169][105692] Updated weights for policy 0, policy_version 1006982 (0.0009) [2023-12-26 22:42:22,192][105620] Updated weights for policy 1, policy_version 1007371 (0.0009) [2023-12-26 22:42:22,257][105620] Updated weights for policy 1, policy_version 1007381 (0.0008) [2023-12-26 22:42:22,324][105620] Updated weights for policy 1, policy_version 1007391 (0.0010) [2023-12-26 22:42:23,002][105692] Updated weights for policy 0, policy_version 1006992 (0.0007) [2023-12-26 22:42:23,006][105620] Updated weights for policy 1, policy_version 1007401 (0.0010) [2023-12-26 22:42:23,060][105692] Updated weights for policy 0, policy_version 1007002 (0.0006) [2023-12-26 22:42:23,066][105620] Updated weights for policy 1, policy_version 1007411 (0.0011) [2023-12-26 22:42:23,119][105692] Updated weights for policy 0, policy_version 1007012 (0.0005) [2023-12-26 22:42:23,125][105620] Updated weights for policy 1, policy_version 1007421 (0.0010) [2023-12-26 22:42:23,186][105620] Updated weights for policy 1, policy_version 1007431 (0.0010) [2023-12-26 22:42:23,741][105692] Updated weights for policy 0, policy_version 1007022 (0.0006) [2023-12-26 22:42:23,793][105692] Updated weights for policy 0, policy_version 1007032 (0.0007) [2023-12-26 22:42:23,851][105692] Updated weights for policy 0, policy_version 1007042 (0.0009) [2023-12-26 22:42:23,895][105620] Updated weights for policy 1, policy_version 1007441 (0.0008) [2023-12-26 22:42:23,945][105620] Updated weights for policy 1, policy_version 1007452 (0.0010) [2023-12-26 22:42:23,993][105620] Updated weights for policy 1, policy_version 1007462 (0.0006) [2023-12-26 22:42:24,542][105692] Updated weights for policy 0, policy_version 1007052 (0.0005) [2023-12-26 22:42:24,600][105692] Updated weights for policy 0, policy_version 1007062 (0.0008) [2023-12-26 22:42:24,644][105620] Updated weights for policy 1, policy_version 1007472 (0.0006) [2023-12-26 22:42:24,652][105692] Updated weights for policy 0, policy_version 1007072 (0.0010) [2023-12-26 22:42:24,700][105620] Updated weights for policy 1, policy_version 1007482 (0.0005) [2023-12-26 22:42:24,758][105620] Updated weights for policy 1, policy_version 1007492 (0.0005) [2023-12-26 22:42:25,344][105620] Updated weights for policy 1, policy_version 1007502 (0.0008) [2023-12-26 22:42:25,358][105692] Updated weights for policy 0, policy_version 1007082 (0.0009) [2023-12-26 22:42:25,397][105620] Updated weights for policy 1, policy_version 1007512 (0.0008) [2023-12-26 22:42:25,409][105692] Updated weights for policy 0, policy_version 1007092 (0.0006) [2023-12-26 22:42:25,455][105620] Updated weights for policy 1, policy_version 1007522 (0.0009) [2023-12-26 22:42:25,465][105692] Updated weights for policy 0, policy_version 1007102 (0.0006) [2023-12-26 22:42:25,518][105692] Updated weights for policy 0, policy_version 1007112 (0.0005) [2023-12-26 22:42:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 515817472. Throughput: 0: 9660.3, 1: 9832.7. Samples: 515829488. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:42:26,062][104569] Avg episode reward: [(0, '9174.292'), (1, '9170.364')] [2023-12-26 22:42:26,080][105620] Updated weights for policy 1, policy_version 1007532 (0.0007) [2023-12-26 22:42:26,129][105620] Updated weights for policy 1, policy_version 1007542 (0.0005) [2023-12-26 22:42:26,174][105692] Updated weights for policy 0, policy_version 1007122 (0.0010) [2023-12-26 22:42:26,186][105620] Updated weights for policy 1, policy_version 1007552 (0.0008) [2023-12-26 22:42:26,222][105692] Updated weights for policy 0, policy_version 1007132 (0.0010) [2023-12-26 22:42:26,270][105692] Updated weights for policy 0, policy_version 1007142 (0.0010) [2023-12-26 22:42:26,923][105620] Updated weights for policy 1, policy_version 1007562 (0.0010) [2023-12-26 22:42:26,954][105692] Updated weights for policy 0, policy_version 1007152 (0.0008) [2023-12-26 22:42:26,982][105620] Updated weights for policy 1, policy_version 1007572 (0.0011) [2023-12-26 22:42:27,014][105692] Updated weights for policy 0, policy_version 1007162 (0.0008) [2023-12-26 22:42:27,040][105620] Updated weights for policy 1, policy_version 1007582 (0.0010) [2023-12-26 22:42:27,069][105692] Updated weights for policy 0, policy_version 1007172 (0.0006) [2023-12-26 22:42:27,096][105620] Updated weights for policy 1, policy_version 1007592 (0.0010) [2023-12-26 22:42:27,658][105692] Updated weights for policy 0, policy_version 1007182 (0.0006) [2023-12-26 22:42:27,711][105692] Updated weights for policy 0, policy_version 1007192 (0.0005) [2023-12-26 22:42:27,757][105692] Updated weights for policy 0, policy_version 1007202 (0.0005) [2023-12-26 22:42:27,816][105620] Updated weights for policy 1, policy_version 1007602 (0.0010) [2023-12-26 22:42:27,870][105620] Updated weights for policy 1, policy_version 1007612 (0.0010) [2023-12-26 22:42:27,914][105620] Updated weights for policy 1, policy_version 1007622 (0.0010) [2023-12-26 22:42:28,449][105692] Updated weights for policy 0, policy_version 1007212 (0.0005) [2023-12-26 22:42:28,507][105692] Updated weights for policy 0, policy_version 1007222 (0.0005) [2023-12-26 22:42:28,569][105692] Updated weights for policy 0, policy_version 1007232 (0.0007) [2023-12-26 22:42:28,586][105620] Updated weights for policy 1, policy_version 1007632 (0.0010) [2023-12-26 22:42:28,641][105620] Updated weights for policy 1, policy_version 1007642 (0.0010) [2023-12-26 22:42:28,710][105620] Updated weights for policy 1, policy_version 1007652 (0.0010) [2023-12-26 22:42:29,079][105692] Updated weights for policy 0, policy_version 1007242 (0.0005) [2023-12-26 22:42:29,129][105692] Updated weights for policy 0, policy_version 1007252 (0.0005) [2023-12-26 22:42:29,180][105692] Updated weights for policy 0, policy_version 1007262 (0.0005) [2023-12-26 22:42:29,236][105692] Updated weights for policy 0, policy_version 1007272 (0.0007) [2023-12-26 22:42:29,459][105620] Updated weights for policy 1, policy_version 1007662 (0.0010) [2023-12-26 22:42:29,530][105620] Updated weights for policy 1, policy_version 1007672 (0.0010) [2023-12-26 22:42:29,578][105620] Updated weights for policy 1, policy_version 1007682 (0.0010) [2023-12-26 22:42:29,871][105692] Updated weights for policy 0, policy_version 1007282 (0.0008) [2023-12-26 22:42:29,926][105692] Updated weights for policy 0, policy_version 1007292 (0.0008) [2023-12-26 22:42:29,980][105692] Updated weights for policy 0, policy_version 1007302 (0.0008) [2023-12-26 22:42:30,302][105620] Updated weights for policy 1, policy_version 1007692 (0.0010) [2023-12-26 22:42:30,360][105620] Updated weights for policy 1, policy_version 1007702 (0.0010) [2023-12-26 22:42:30,422][105620] Updated weights for policy 1, policy_version 1007712 (0.0010) [2023-12-26 22:42:30,778][105692] Updated weights for policy 0, policy_version 1007312 (0.0008) [2023-12-26 22:42:30,833][105692] Updated weights for policy 0, policy_version 1007322 (0.0008) [2023-12-26 22:42:30,883][105692] Updated weights for policy 0, policy_version 1007332 (0.0005) [2023-12-26 22:42:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 515923968. Throughput: 0: 9739.5, 1: 9879.9. Samples: 515891860. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:42:31,062][104569] Avg episode reward: [(0, '9088.234'), (1, '9262.084')] [2023-12-26 22:42:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001007336_257916928.pth... [2023-12-26 22:42:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001007720_258007040.pth... [2023-12-26 22:42:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001006184_257622016.pth [2023-12-26 22:42:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001006600_257720320.pth [2023-12-26 22:42:31,174][105620] Updated weights for policy 1, policy_version 1007722 (0.0010) [2023-12-26 22:42:31,235][105620] Updated weights for policy 1, policy_version 1007732 (0.0006) [2023-12-26 22:42:31,290][105620] Updated weights for policy 1, policy_version 1007742 (0.0009) [2023-12-26 22:42:31,351][105620] Updated weights for policy 1, policy_version 1007752 (0.0010) [2023-12-26 22:42:31,538][105692] Updated weights for policy 0, policy_version 1007342 (0.0005) [2023-12-26 22:42:31,592][105692] Updated weights for policy 0, policy_version 1007352 (0.0008) [2023-12-26 22:42:31,660][105692] Updated weights for policy 0, policy_version 1007362 (0.0008) [2023-12-26 22:42:32,095][105620] Updated weights for policy 1, policy_version 1007762 (0.0006) [2023-12-26 22:42:32,158][105620] Updated weights for policy 1, policy_version 1007772 (0.0006) [2023-12-26 22:42:32,225][105620] Updated weights for policy 1, policy_version 1007782 (0.0005) [2023-12-26 22:42:32,393][105692] Updated weights for policy 0, policy_version 1007372 (0.0007) [2023-12-26 22:42:32,453][105692] Updated weights for policy 0, policy_version 1007382 (0.0006) [2023-12-26 22:42:32,512][105692] Updated weights for policy 0, policy_version 1007392 (0.0009) [2023-12-26 22:42:32,878][105620] Updated weights for policy 1, policy_version 1007792 (0.0009) [2023-12-26 22:42:32,926][105620] Updated weights for policy 1, policy_version 1007802 (0.0009) [2023-12-26 22:42:32,973][105620] Updated weights for policy 1, policy_version 1007812 (0.0010) [2023-12-26 22:42:33,224][105692] Updated weights for policy 0, policy_version 1007402 (0.0008) [2023-12-26 22:42:33,282][105692] Updated weights for policy 0, policy_version 1007412 (0.0010) [2023-12-26 22:42:33,339][105692] Updated weights for policy 0, policy_version 1007422 (0.0010) [2023-12-26 22:42:33,398][105692] Updated weights for policy 0, policy_version 1007432 (0.0010) [2023-12-26 22:42:33,594][105620] Updated weights for policy 1, policy_version 1007822 (0.0009) [2023-12-26 22:42:33,658][105620] Updated weights for policy 1, policy_version 1007832 (0.0009) [2023-12-26 22:42:33,726][105620] Updated weights for policy 1, policy_version 1007842 (0.0009) [2023-12-26 22:42:34,047][105692] Updated weights for policy 0, policy_version 1007442 (0.0009) [2023-12-26 22:42:34,096][105692] Updated weights for policy 0, policy_version 1007452 (0.0008) [2023-12-26 22:42:34,147][105692] Updated weights for policy 0, policy_version 1007462 (0.0008) [2023-12-26 22:42:34,470][105620] Updated weights for policy 1, policy_version 1007852 (0.0009) [2023-12-26 22:42:34,533][105620] Updated weights for policy 1, policy_version 1007862 (0.0009) [2023-12-26 22:42:34,591][105620] Updated weights for policy 1, policy_version 1007872 (0.0010) [2023-12-26 22:42:34,836][105692] Updated weights for policy 0, policy_version 1007472 (0.0009) [2023-12-26 22:42:34,895][105692] Updated weights for policy 0, policy_version 1007482 (0.0009) [2023-12-26 22:42:34,950][105692] Updated weights for policy 0, policy_version 1007492 (0.0009) [2023-12-26 22:42:35,319][105620] Updated weights for policy 1, policy_version 1007882 (0.0009) [2023-12-26 22:42:35,365][105620] Updated weights for policy 1, policy_version 1007892 (0.0006) [2023-12-26 22:42:35,418][105620] Updated weights for policy 1, policy_version 1007902 (0.0005) [2023-12-26 22:42:35,474][105620] Updated weights for policy 1, policy_version 1007912 (0.0006) [2023-12-26 22:42:35,769][105692] Updated weights for policy 0, policy_version 1007502 (0.0009) [2023-12-26 22:42:35,826][105692] Updated weights for policy 0, policy_version 1007512 (0.0009) [2023-12-26 22:42:35,883][105692] Updated weights for policy 0, policy_version 1007522 (0.0010) [2023-12-26 22:42:36,014][105620] Updated weights for policy 1, policy_version 1007922 (0.0005) [2023-12-26 22:42:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 516022272. Throughput: 0: 9777.6, 1: 9861.0. Samples: 516011020. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:42:36,063][104569] Avg episode reward: [(0, '8817.969'), (1, '9080.136')] [2023-12-26 22:42:36,072][105620] Updated weights for policy 1, policy_version 1007932 (0.0007) [2023-12-26 22:42:36,135][105620] Updated weights for policy 1, policy_version 1007942 (0.0007) [2023-12-26 22:42:36,723][105692] Updated weights for policy 0, policy_version 1007532 (0.0009) [2023-12-26 22:42:36,725][105620] Updated weights for policy 1, policy_version 1007952 (0.0007) [2023-12-26 22:42:36,782][105620] Updated weights for policy 1, policy_version 1007962 (0.0005) [2023-12-26 22:42:36,785][105692] Updated weights for policy 0, policy_version 1007542 (0.0008) [2023-12-26 22:42:36,842][105692] Updated weights for policy 0, policy_version 1007552 (0.0007) [2023-12-26 22:42:36,844][105620] Updated weights for policy 1, policy_version 1007972 (0.0007) [2023-12-26 22:42:37,431][105620] Updated weights for policy 1, policy_version 1007982 (0.0008) [2023-12-26 22:42:37,483][105620] Updated weights for policy 1, policy_version 1007992 (0.0007) [2023-12-26 22:42:37,530][105620] Updated weights for policy 1, policy_version 1008002 (0.0005) [2023-12-26 22:42:37,703][105692] Updated weights for policy 0, policy_version 1007562 (0.0008) [2023-12-26 22:42:37,761][105692] Updated weights for policy 0, policy_version 1007572 (0.0009) [2023-12-26 22:42:37,822][105692] Updated weights for policy 0, policy_version 1007582 (0.0009) [2023-12-26 22:42:37,888][105692] Updated weights for policy 0, policy_version 1007592 (0.0009) [2023-12-26 22:42:38,198][105620] Updated weights for policy 1, policy_version 1008012 (0.0007) [2023-12-26 22:42:38,260][105620] Updated weights for policy 1, policy_version 1008022 (0.0010) [2023-12-26 22:42:38,318][105620] Updated weights for policy 1, policy_version 1008032 (0.0009) [2023-12-26 22:42:38,591][105692] Updated weights for policy 0, policy_version 1007602 (0.0010) [2023-12-26 22:42:38,652][105692] Updated weights for policy 0, policy_version 1007612 (0.0011) [2023-12-26 22:42:38,713][105692] Updated weights for policy 0, policy_version 1007622 (0.0011) [2023-12-26 22:42:39,071][105620] Updated weights for policy 1, policy_version 1008042 (0.0009) [2023-12-26 22:42:39,124][105620] Updated weights for policy 1, policy_version 1008052 (0.0006) [2023-12-26 22:42:39,177][105620] Updated weights for policy 1, policy_version 1008062 (0.0007) [2023-12-26 22:42:39,236][105620] Updated weights for policy 1, policy_version 1008072 (0.0007) [2023-12-26 22:42:39,458][105692] Updated weights for policy 0, policy_version 1007632 (0.0009) [2023-12-26 22:42:39,525][105692] Updated weights for policy 0, policy_version 1007642 (0.0008) [2023-12-26 22:42:39,582][105692] Updated weights for policy 0, policy_version 1007652 (0.0008) [2023-12-26 22:42:39,995][105620] Updated weights for policy 1, policy_version 1008082 (0.0011) [2023-12-26 22:42:40,058][105620] Updated weights for policy 1, policy_version 1008092 (0.0009) [2023-12-26 22:42:40,118][105620] Updated weights for policy 1, policy_version 1008102 (0.0011) [2023-12-26 22:42:40,336][105692] Updated weights for policy 0, policy_version 1007662 (0.0009) [2023-12-26 22:42:40,392][105692] Updated weights for policy 0, policy_version 1007672 (0.0008) [2023-12-26 22:42:40,448][105692] Updated weights for policy 0, policy_version 1007682 (0.0008) [2023-12-26 22:42:40,852][105620] Updated weights for policy 1, policy_version 1008112 (0.0009) [2023-12-26 22:42:40,902][105620] Updated weights for policy 1, policy_version 1008122 (0.0005) [2023-12-26 22:42:40,948][105620] Updated weights for policy 1, policy_version 1008132 (0.0009) [2023-12-26 22:42:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 516120576. Throughput: 0: 9747.5, 1: 9888.6. Samples: 516128420. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:42:41,062][104569] Avg episode reward: [(0, '9085.412'), (1, '8987.727')] [2023-12-26 22:42:41,168][105692] Updated weights for policy 0, policy_version 1007692 (0.0009) [2023-12-26 22:42:41,235][105692] Updated weights for policy 0, policy_version 1007702 (0.0011) [2023-12-26 22:42:41,301][105692] Updated weights for policy 0, policy_version 1007712 (0.0009) [2023-12-26 22:42:41,734][105620] Updated weights for policy 1, policy_version 1008142 (0.0010) [2023-12-26 22:42:41,795][105620] Updated weights for policy 1, policy_version 1008152 (0.0007) [2023-12-26 22:42:41,862][105620] Updated weights for policy 1, policy_version 1008162 (0.0010) [2023-12-26 22:42:42,045][105692] Updated weights for policy 0, policy_version 1007722 (0.0009) [2023-12-26 22:42:42,108][105692] Updated weights for policy 0, policy_version 1007732 (0.0009) [2023-12-26 22:42:42,166][105692] Updated weights for policy 0, policy_version 1007742 (0.0009) [2023-12-26 22:42:42,229][105692] Updated weights for policy 0, policy_version 1007752 (0.0009) [2023-12-26 22:42:42,662][105620] Updated weights for policy 1, policy_version 1008172 (0.0010) [2023-12-26 22:42:42,726][105620] Updated weights for policy 1, policy_version 1008182 (0.0011) [2023-12-26 22:42:42,800][105620] Updated weights for policy 1, policy_version 1008192 (0.0011) [2023-12-26 22:42:42,964][105692] Updated weights for policy 0, policy_version 1007762 (0.0008) [2023-12-26 22:42:43,011][105692] Updated weights for policy 0, policy_version 1007772 (0.0010) [2023-12-26 22:42:43,063][105692] Updated weights for policy 0, policy_version 1007782 (0.0010) [2023-12-26 22:42:43,516][105620] Updated weights for policy 1, policy_version 1008202 (0.0010) [2023-12-26 22:42:43,574][105620] Updated weights for policy 1, policy_version 1008212 (0.0010) [2023-12-26 22:42:43,626][105620] Updated weights for policy 1, policy_version 1008222 (0.0010) [2023-12-26 22:42:43,636][105692] Updated weights for policy 0, policy_version 1007792 (0.0007) [2023-12-26 22:42:43,674][105620] Updated weights for policy 1, policy_version 1008232 (0.0010) [2023-12-26 22:42:43,693][105692] Updated weights for policy 0, policy_version 1007802 (0.0006) [2023-12-26 22:42:43,752][105692] Updated weights for policy 0, policy_version 1007812 (0.0006) [2023-12-26 22:42:44,305][105620] Updated weights for policy 1, policy_version 1008242 (0.0010) [2023-12-26 22:42:44,356][105620] Updated weights for policy 1, policy_version 1008252 (0.0007) [2023-12-26 22:42:44,405][105620] Updated weights for policy 1, policy_version 1008262 (0.0008) [2023-12-26 22:42:44,551][105692] Updated weights for policy 0, policy_version 1007822 (0.0008) [2023-12-26 22:42:44,596][105692] Updated weights for policy 0, policy_version 1007832 (0.0008) [2023-12-26 22:42:44,642][105692] Updated weights for policy 0, policy_version 1007842 (0.0009) [2023-12-26 22:42:45,137][105620] Updated weights for policy 1, policy_version 1008272 (0.0009) [2023-12-26 22:42:45,196][105620] Updated weights for policy 1, policy_version 1008282 (0.0009) [2023-12-26 22:42:45,262][105620] Updated weights for policy 1, policy_version 1008292 (0.0011) [2023-12-26 22:42:45,450][105692] Updated weights for policy 0, policy_version 1007852 (0.0008) [2023-12-26 22:42:45,508][105692] Updated weights for policy 0, policy_version 1007862 (0.0008) [2023-12-26 22:42:45,564][105692] Updated weights for policy 0, policy_version 1007872 (0.0008) [2023-12-26 22:42:46,011][105620] Updated weights for policy 1, policy_version 1008302 (0.0010) [2023-12-26 22:42:46,055][105620] Updated weights for policy 1, policy_version 1008312 (0.0010) [2023-12-26 22:42:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 516210688. Throughput: 0: 9758.0, 1: 9811.4. Samples: 516186084. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:42:46,063][104569] Avg episode reward: [(0, '8998.693'), (1, '9170.813')] [2023-12-26 22:42:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001007880_258056192.pth... [2023-12-26 22:42:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001006760_257769472.pth [2023-12-26 22:42:46,106][105620] Updated weights for policy 1, policy_version 1008322 (0.0010) [2023-12-26 22:42:46,138][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001008328_258162688.pth... [2023-12-26 22:42:46,143][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001007144_257859584.pth [2023-12-26 22:42:46,331][105692] Updated weights for policy 0, policy_version 1007882 (0.0008) [2023-12-26 22:42:46,385][105692] Updated weights for policy 0, policy_version 1007892 (0.0010) [2023-12-26 22:42:46,439][105692] Updated weights for policy 0, policy_version 1007904 (0.0011) [2023-12-26 22:42:46,788][105620] Updated weights for policy 1, policy_version 1008332 (0.0010) [2023-12-26 22:42:46,832][105620] Updated weights for policy 1, policy_version 1008342 (0.0010) [2023-12-26 22:42:46,882][105620] Updated weights for policy 1, policy_version 1008352 (0.0010) [2023-12-26 22:42:47,189][105692] Updated weights for policy 0, policy_version 1007914 (0.0009) [2023-12-26 22:42:47,258][105692] Updated weights for policy 0, policy_version 1007924 (0.0005) [2023-12-26 22:42:47,326][105692] Updated weights for policy 0, policy_version 1007934 (0.0005) [2023-12-26 22:42:47,394][105692] Updated weights for policy 0, policy_version 1007944 (0.0005) [2023-12-26 22:42:47,536][105620] Updated weights for policy 1, policy_version 1008362 (0.0010) [2023-12-26 22:42:47,589][105620] Updated weights for policy 1, policy_version 1008372 (0.0008) [2023-12-26 22:42:47,659][105620] Updated weights for policy 1, policy_version 1008382 (0.0011) [2023-12-26 22:42:47,718][105620] Updated weights for policy 1, policy_version 1008392 (0.0011) [2023-12-26 22:42:47,979][105692] Updated weights for policy 0, policy_version 1007954 (0.0008) [2023-12-26 22:42:48,043][105692] Updated weights for policy 0, policy_version 1007964 (0.0009) [2023-12-26 22:42:48,094][105692] Updated weights for policy 0, policy_version 1007974 (0.0005) [2023-12-26 22:42:48,487][105620] Updated weights for policy 1, policy_version 1008402 (0.0006) [2023-12-26 22:42:48,535][105620] Updated weights for policy 1, policy_version 1008412 (0.0008) [2023-12-26 22:42:48,591][105620] Updated weights for policy 1, policy_version 1008422 (0.0009) [2023-12-26 22:42:48,760][105692] Updated weights for policy 0, policy_version 1007984 (0.0009) [2023-12-26 22:42:48,817][105692] Updated weights for policy 0, policy_version 1007994 (0.0010) [2023-12-26 22:42:48,873][105692] Updated weights for policy 0, policy_version 1008004 (0.0010) [2023-12-26 22:42:49,291][105620] Updated weights for policy 1, policy_version 1008432 (0.0009) [2023-12-26 22:42:49,355][105620] Updated weights for policy 1, policy_version 1008442 (0.0007) [2023-12-26 22:42:49,426][105620] Updated weights for policy 1, policy_version 1008452 (0.0006) [2023-12-26 22:42:49,551][105692] Updated weights for policy 0, policy_version 1008014 (0.0007) [2023-12-26 22:42:49,608][105692] Updated weights for policy 0, policy_version 1008024 (0.0007) [2023-12-26 22:42:49,667][105692] Updated weights for policy 0, policy_version 1008034 (0.0008) [2023-12-26 22:42:50,150][105620] Updated weights for policy 1, policy_version 1008462 (0.0008) [2023-12-26 22:42:50,212][105620] Updated weights for policy 1, policy_version 1008472 (0.0006) [2023-12-26 22:42:50,279][105620] Updated weights for policy 1, policy_version 1008482 (0.0006) [2023-12-26 22:42:50,476][105692] Updated weights for policy 0, policy_version 1008044 (0.0009) [2023-12-26 22:42:50,525][105692] Updated weights for policy 0, policy_version 1008054 (0.0008) [2023-12-26 22:42:50,579][105692] Updated weights for policy 0, policy_version 1008064 (0.0007) [2023-12-26 22:42:51,005][105620] Updated weights for policy 1, policy_version 1008492 (0.0010) [2023-12-26 22:42:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 516308992. Throughput: 0: 9734.7, 1: 9832.5. Samples: 516303652. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:42:51,062][104569] Avg episode reward: [(0, '8105.682'), (1, '9263.347')] [2023-12-26 22:42:51,072][105620] Updated weights for policy 1, policy_version 1008502 (0.0011) [2023-12-26 22:42:51,130][105620] Updated weights for policy 1, policy_version 1008512 (0.0008) [2023-12-26 22:42:51,317][105692] Updated weights for policy 0, policy_version 1008074 (0.0009) [2023-12-26 22:42:51,384][105692] Updated weights for policy 0, policy_version 1008084 (0.0011) [2023-12-26 22:42:51,447][105692] Updated weights for policy 0, policy_version 1008094 (0.0009) [2023-12-26 22:42:51,509][105692] Updated weights for policy 0, policy_version 1008104 (0.0009) [2023-12-26 22:42:51,859][105620] Updated weights for policy 1, policy_version 1008522 (0.0007) [2023-12-26 22:42:51,913][105620] Updated weights for policy 1, policy_version 1008532 (0.0010) [2023-12-26 22:42:51,967][105620] Updated weights for policy 1, policy_version 1008542 (0.0008) [2023-12-26 22:42:52,028][105620] Updated weights for policy 1, policy_version 1008552 (0.0006) [2023-12-26 22:42:52,231][105692] Updated weights for policy 0, policy_version 1008114 (0.0010) [2023-12-26 22:42:52,294][105692] Updated weights for policy 0, policy_version 1008124 (0.0011) [2023-12-26 22:42:52,356][105692] Updated weights for policy 0, policy_version 1008134 (0.0011) [2023-12-26 22:42:52,655][105620] Updated weights for policy 1, policy_version 1008562 (0.0008) [2023-12-26 22:42:52,711][105620] Updated weights for policy 1, policy_version 1008572 (0.0009) [2023-12-26 22:42:52,775][105620] Updated weights for policy 1, policy_version 1008582 (0.0009) [2023-12-26 22:42:53,173][105692] Updated weights for policy 0, policy_version 1008144 (0.0009) [2023-12-26 22:42:53,239][105692] Updated weights for policy 0, policy_version 1008154 (0.0007) [2023-12-26 22:42:53,298][105692] Updated weights for policy 0, policy_version 1008164 (0.0008) [2023-12-26 22:42:53,491][105620] Updated weights for policy 1, policy_version 1008592 (0.0010) [2023-12-26 22:42:53,543][105620] Updated weights for policy 1, policy_version 1008602 (0.0010) [2023-12-26 22:42:53,614][105620] Updated weights for policy 1, policy_version 1008612 (0.0006) [2023-12-26 22:42:54,065][105692] Updated weights for policy 0, policy_version 1008174 (0.0007) [2023-12-26 22:42:54,109][105692] Updated weights for policy 0, policy_version 1008184 (0.0008) [2023-12-26 22:42:54,165][105692] Updated weights for policy 0, policy_version 1008194 (0.0008) [2023-12-26 22:42:54,282][105620] Updated weights for policy 1, policy_version 1008622 (0.0005) [2023-12-26 22:42:54,348][105620] Updated weights for policy 1, policy_version 1008632 (0.0008) [2023-12-26 22:42:54,407][105620] Updated weights for policy 1, policy_version 1008642 (0.0010) [2023-12-26 22:42:54,993][105620] Updated weights for policy 1, policy_version 1008652 (0.0008) [2023-12-26 22:42:55,001][105692] Updated weights for policy 0, policy_version 1008204 (0.0007) [2023-12-26 22:42:55,049][105620] Updated weights for policy 1, policy_version 1008662 (0.0010) [2023-12-26 22:42:55,064][105692] Updated weights for policy 0, policy_version 1008214 (0.0006) [2023-12-26 22:42:55,110][105620] Updated weights for policy 1, policy_version 1008672 (0.0005) [2023-12-26 22:42:55,122][105692] Updated weights for policy 0, policy_version 1008224 (0.0009) [2023-12-26 22:42:55,715][105620] Updated weights for policy 1, policy_version 1008682 (0.0005) [2023-12-26 22:42:55,773][105620] Updated weights for policy 1, policy_version 1008692 (0.0005) [2023-12-26 22:42:55,793][105692] Updated weights for policy 0, policy_version 1008234 (0.0007) [2023-12-26 22:42:55,823][105620] Updated weights for policy 1, policy_version 1008702 (0.0005) [2023-12-26 22:42:55,860][105692] Updated weights for policy 0, policy_version 1008244 (0.0008) [2023-12-26 22:42:55,881][105620] Updated weights for policy 1, policy_version 1008712 (0.0006) [2023-12-26 22:42:55,922][105692] Updated weights for policy 0, policy_version 1008254 (0.0009) [2023-12-26 22:42:55,977][105692] Updated weights for policy 0, policy_version 1008264 (0.0007) [2023-12-26 22:42:56,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 516415488. Throughput: 0: 9729.2, 1: 9905.5. Samples: 516420936. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:42:56,063][104569] Avg episode reward: [(0, '8010.081'), (1, '9354.447')] [2023-12-26 22:42:56,467][105620] Updated weights for policy 1, policy_version 1008722 (0.0005) [2023-12-26 22:42:56,526][105620] Updated weights for policy 1, policy_version 1008732 (0.0007) [2023-12-26 22:42:56,575][105620] Updated weights for policy 1, policy_version 1008742 (0.0011) [2023-12-26 22:42:56,777][105692] Updated weights for policy 0, policy_version 1008274 (0.0008) [2023-12-26 22:42:56,825][105692] Updated weights for policy 0, policy_version 1008284 (0.0008) [2023-12-26 22:42:56,872][105692] Updated weights for policy 0, policy_version 1008294 (0.0008) [2023-12-26 22:42:57,248][105620] Updated weights for policy 1, policy_version 1008752 (0.0006) [2023-12-26 22:42:57,305][105620] Updated weights for policy 1, policy_version 1008762 (0.0007) [2023-12-26 22:42:57,370][105620] Updated weights for policy 1, policy_version 1008772 (0.0009) [2023-12-26 22:42:57,732][105692] Updated weights for policy 0, policy_version 1008304 (0.0009) [2023-12-26 22:42:57,785][105692] Updated weights for policy 0, policy_version 1008314 (0.0009) [2023-12-26 22:42:57,838][105692] Updated weights for policy 0, policy_version 1008325 (0.0010) [2023-12-26 22:42:57,958][105620] Updated weights for policy 1, policy_version 1008782 (0.0007) [2023-12-26 22:42:58,008][105620] Updated weights for policy 1, policy_version 1008792 (0.0005) [2023-12-26 22:42:58,052][105620] Updated weights for policy 1, policy_version 1008802 (0.0007) [2023-12-26 22:42:58,705][105692] Updated weights for policy 0, policy_version 1008335 (0.0009) [2023-12-26 22:42:58,767][105620] Updated weights for policy 1, policy_version 1008812 (0.0006) [2023-12-26 22:42:58,770][105692] Updated weights for policy 0, policy_version 1008345 (0.0008) [2023-12-26 22:42:58,829][105620] Updated weights for policy 1, policy_version 1008822 (0.0008) [2023-12-26 22:42:58,838][105692] Updated weights for policy 0, policy_version 1008355 (0.0008) [2023-12-26 22:42:58,888][105620] Updated weights for policy 1, policy_version 1008832 (0.0008) [2023-12-26 22:42:59,596][105620] Updated weights for policy 1, policy_version 1008842 (0.0008) [2023-12-26 22:42:59,613][105692] Updated weights for policy 0, policy_version 1008365 (0.0008) [2023-12-26 22:42:59,644][105620] Updated weights for policy 1, policy_version 1008852 (0.0007) [2023-12-26 22:42:59,663][105692] Updated weights for policy 0, policy_version 1008375 (0.0008) [2023-12-26 22:42:59,697][105620] Updated weights for policy 1, policy_version 1008862 (0.0008) [2023-12-26 22:42:59,713][105692] Updated weights for policy 0, policy_version 1008385 (0.0006) [2023-12-26 22:42:59,753][105620] Updated weights for policy 1, policy_version 1008872 (0.0007) [2023-12-26 22:43:00,484][105620] Updated weights for policy 1, policy_version 1008882 (0.0010) [2023-12-26 22:43:00,515][105692] Updated weights for policy 0, policy_version 1008395 (0.0007) [2023-12-26 22:43:00,536][105620] Updated weights for policy 1, policy_version 1008892 (0.0010) [2023-12-26 22:43:00,568][105692] Updated weights for policy 0, policy_version 1008405 (0.0005) [2023-12-26 22:43:00,588][105620] Updated weights for policy 1, policy_version 1008902 (0.0010) [2023-12-26 22:43:00,621][105692] Updated weights for policy 0, policy_version 1008415 (0.0005) [2023-12-26 22:43:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 516505600. Throughput: 0: 9636.1, 1: 9993.9. Samples: 516477772. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:43:01,063][104569] Avg episode reward: [(0, '8274.831'), (1, '9263.044')] [2023-12-26 22:43:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001008424_258195456.pth... [2023-12-26 22:43:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001008904_258310144.pth... [2023-12-26 22:43:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001007336_257916928.pth [2023-12-26 22:43:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001007720_258007040.pth [2023-12-26 22:43:01,305][105620] Updated weights for policy 1, policy_version 1008912 (0.0010) [2023-12-26 22:43:01,319][105692] Updated weights for policy 0, policy_version 1008425 (0.0006) [2023-12-26 22:43:01,373][105620] Updated weights for policy 1, policy_version 1008922 (0.0010) [2023-12-26 22:43:01,378][105692] Updated weights for policy 0, policy_version 1008435 (0.0007) [2023-12-26 22:43:01,432][105620] Updated weights for policy 1, policy_version 1008932 (0.0010) [2023-12-26 22:43:01,438][105692] Updated weights for policy 0, policy_version 1008445 (0.0006) [2023-12-26 22:43:01,497][105692] Updated weights for policy 0, policy_version 1008455 (0.0005) [2023-12-26 22:43:02,182][105692] Updated weights for policy 0, policy_version 1008465 (0.0008) [2023-12-26 22:43:02,198][105620] Updated weights for policy 1, policy_version 1008942 (0.0011) [2023-12-26 22:43:02,240][105692] Updated weights for policy 0, policy_version 1008475 (0.0006) [2023-12-26 22:43:02,260][105620] Updated weights for policy 1, policy_version 1008952 (0.0011) [2023-12-26 22:43:02,302][105692] Updated weights for policy 0, policy_version 1008485 (0.0008) [2023-12-26 22:43:02,313][105620] Updated weights for policy 1, policy_version 1008962 (0.0010) [2023-12-26 22:43:03,018][105692] Updated weights for policy 0, policy_version 1008495 (0.0008) [2023-12-26 22:43:03,069][105620] Updated weights for policy 1, policy_version 1008972 (0.0010) [2023-12-26 22:43:03,075][105692] Updated weights for policy 0, policy_version 1008505 (0.0007) [2023-12-26 22:43:03,121][105620] Updated weights for policy 1, policy_version 1008982 (0.0010) [2023-12-26 22:43:03,135][105692] Updated weights for policy 0, policy_version 1008515 (0.0005) [2023-12-26 22:43:03,176][105620] Updated weights for policy 1, policy_version 1008992 (0.0010) [2023-12-26 22:43:03,846][105692] Updated weights for policy 0, policy_version 1008525 (0.0007) [2023-12-26 22:43:03,910][105692] Updated weights for policy 0, policy_version 1008535 (0.0006) [2023-12-26 22:43:03,925][105620] Updated weights for policy 1, policy_version 1009002 (0.0010) [2023-12-26 22:43:03,971][105692] Updated weights for policy 0, policy_version 1008545 (0.0005) [2023-12-26 22:43:03,988][105620] Updated weights for policy 1, policy_version 1009012 (0.0010) [2023-12-26 22:43:04,047][105620] Updated weights for policy 1, policy_version 1009022 (0.0010) [2023-12-26 22:43:04,106][105620] Updated weights for policy 1, policy_version 1009032 (0.0010) [2023-12-26 22:43:04,651][105692] Updated weights for policy 0, policy_version 1008555 (0.0006) [2023-12-26 22:43:04,696][105692] Updated weights for policy 0, policy_version 1008565 (0.0005) [2023-12-26 22:43:04,740][105692] Updated weights for policy 0, policy_version 1008575 (0.0005) [2023-12-26 22:43:04,846][105620] Updated weights for policy 1, policy_version 1009042 (0.0010) [2023-12-26 22:43:04,894][105620] Updated weights for policy 1, policy_version 1009052 (0.0010) [2023-12-26 22:43:04,940][105620] Updated weights for policy 1, policy_version 1009062 (0.0010) [2023-12-26 22:43:05,305][105692] Updated weights for policy 0, policy_version 1008585 (0.0005) [2023-12-26 22:43:05,361][105692] Updated weights for policy 0, policy_version 1008595 (0.0005) [2023-12-26 22:43:05,415][105692] Updated weights for policy 0, policy_version 1008605 (0.0007) [2023-12-26 22:43:05,473][105692] Updated weights for policy 0, policy_version 1008615 (0.0008) [2023-12-26 22:43:05,725][105620] Updated weights for policy 1, policy_version 1009072 (0.0010) [2023-12-26 22:43:05,780][105620] Updated weights for policy 1, policy_version 1009082 (0.0010) [2023-12-26 22:43:05,834][105620] Updated weights for policy 1, policy_version 1009092 (0.0010) [2023-12-26 22:43:06,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 516603904. Throughput: 0: 9602.3, 1: 9950.0. Samples: 516592040. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:43:06,063][105692] Updated weights for policy 0, policy_version 1008625 (0.0005) [2023-12-26 22:43:06,063][104569] Avg episode reward: [(0, '8271.403'), (1, '8897.575')] [2023-12-26 22:43:06,127][105692] Updated weights for policy 0, policy_version 1008635 (0.0007) [2023-12-26 22:43:06,142][105585] KL-divergence is very high: 133.6508 [2023-12-26 22:43:06,194][105692] Updated weights for policy 0, policy_version 1008645 (0.0007) [2023-12-26 22:43:06,195][105585] KL-divergence is very high: 150.3034 [2023-12-26 22:43:06,500][105620] Updated weights for policy 1, policy_version 1009102 (0.0007) [2023-12-26 22:43:06,560][105620] Updated weights for policy 1, policy_version 1009112 (0.0005) [2023-12-26 22:43:06,614][105620] Updated weights for policy 1, policy_version 1009122 (0.0007) [2023-12-26 22:43:06,969][105692] Updated weights for policy 0, policy_version 1008655 (0.0010) [2023-12-26 22:43:07,020][105692] Updated weights for policy 0, policy_version 1008665 (0.0010) [2023-12-26 22:43:07,073][105692] Updated weights for policy 0, policy_version 1008675 (0.0010) [2023-12-26 22:43:07,236][105620] Updated weights for policy 1, policy_version 1009132 (0.0011) [2023-12-26 22:43:07,288][105620] Updated weights for policy 1, policy_version 1009142 (0.0010) [2023-12-26 22:43:07,333][105620] Updated weights for policy 1, policy_version 1009152 (0.0010) [2023-12-26 22:43:07,731][105692] Updated weights for policy 0, policy_version 1008685 (0.0008) [2023-12-26 22:43:07,790][105692] Updated weights for policy 0, policy_version 1008695 (0.0006) [2023-12-26 22:43:07,843][105692] Updated weights for policy 0, policy_version 1008705 (0.0005) [2023-12-26 22:43:08,095][105620] Updated weights for policy 1, policy_version 1009162 (0.0010) [2023-12-26 22:43:08,150][105620] Updated weights for policy 1, policy_version 1009172 (0.0010) [2023-12-26 22:43:08,208][105620] Updated weights for policy 1, policy_version 1009182 (0.0010) [2023-12-26 22:43:08,263][105620] Updated weights for policy 1, policy_version 1009192 (0.0010) [2023-12-26 22:43:08,425][105692] Updated weights for policy 0, policy_version 1008715 (0.0006) [2023-12-26 22:43:08,494][105692] Updated weights for policy 0, policy_version 1008725 (0.0011) [2023-12-26 22:43:08,549][105692] Updated weights for policy 0, policy_version 1008735 (0.0011) [2023-12-26 22:43:08,996][105620] Updated weights for policy 1, policy_version 1009202 (0.0010) [2023-12-26 22:43:09,044][105620] Updated weights for policy 1, policy_version 1009212 (0.0010) [2023-12-26 22:43:09,099][105620] Updated weights for policy 1, policy_version 1009222 (0.0010) [2023-12-26 22:43:09,251][105692] Updated weights for policy 0, policy_version 1008745 (0.0010) [2023-12-26 22:43:09,300][105692] Updated weights for policy 0, policy_version 1008755 (0.0010) [2023-12-26 22:43:09,363][105692] Updated weights for policy 0, policy_version 1008765 (0.0011) [2023-12-26 22:43:09,427][105692] Updated weights for policy 0, policy_version 1008775 (0.0008) [2023-12-26 22:43:09,904][105620] Updated weights for policy 1, policy_version 1009232 (0.0011) [2023-12-26 22:43:09,968][105620] Updated weights for policy 1, policy_version 1009242 (0.0011) [2023-12-26 22:43:10,024][105620] Updated weights for policy 1, policy_version 1009252 (0.0008) [2023-12-26 22:43:10,179][105692] Updated weights for policy 0, policy_version 1008785 (0.0008) [2023-12-26 22:43:10,229][105692] Updated weights for policy 0, policy_version 1008795 (0.0008) [2023-12-26 22:43:10,279][105692] Updated weights for policy 0, policy_version 1008805 (0.0008) [2023-12-26 22:43:10,738][105620] Updated weights for policy 1, policy_version 1009262 (0.0009) [2023-12-26 22:43:10,800][105620] Updated weights for policy 1, policy_version 1009272 (0.0010) [2023-12-26 22:43:10,858][105620] Updated weights for policy 1, policy_version 1009282 (0.0010) [2023-12-26 22:43:11,055][105692] Updated weights for policy 0, policy_version 1008815 (0.0008) [2023-12-26 22:43:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 516702208. Throughput: 0: 9697.6, 1: 9915.8. Samples: 516712092. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:43:11,062][104569] Avg episode reward: [(0, '8812.842'), (1, '8805.104')] [2023-12-26 22:43:11,118][105692] Updated weights for policy 0, policy_version 1008825 (0.0008) [2023-12-26 22:43:11,182][105692] Updated weights for policy 0, policy_version 1008835 (0.0009) [2023-12-26 22:43:11,552][105620] Updated weights for policy 1, policy_version 1009292 (0.0010) [2023-12-26 22:43:11,600][105620] Updated weights for policy 1, policy_version 1009302 (0.0009) [2023-12-26 22:43:11,670][105620] Updated weights for policy 1, policy_version 1009312 (0.0009) [2023-12-26 22:43:11,932][105692] Updated weights for policy 0, policy_version 1008845 (0.0008) [2023-12-26 22:43:11,999][105692] Updated weights for policy 0, policy_version 1008855 (0.0009) [2023-12-26 22:43:12,064][105692] Updated weights for policy 0, policy_version 1008865 (0.0007) [2023-12-26 22:43:12,465][105620] Updated weights for policy 1, policy_version 1009322 (0.0008) [2023-12-26 22:43:12,517][105620] Updated weights for policy 1, policy_version 1009332 (0.0010) [2023-12-26 22:43:12,573][105620] Updated weights for policy 1, policy_version 1009342 (0.0010) [2023-12-26 22:43:12,634][105620] Updated weights for policy 1, policy_version 1009352 (0.0010) [2023-12-26 22:43:12,699][105692] Updated weights for policy 0, policy_version 1008875 (0.0007) [2023-12-26 22:43:12,759][105692] Updated weights for policy 0, policy_version 1008885 (0.0008) [2023-12-26 22:43:12,812][105692] Updated weights for policy 0, policy_version 1008895 (0.0008) [2023-12-26 22:43:13,355][105620] Updated weights for policy 1, policy_version 1009362 (0.0005) [2023-12-26 22:43:13,418][105620] Updated weights for policy 1, policy_version 1009372 (0.0005) [2023-12-26 22:43:13,479][105620] Updated weights for policy 1, policy_version 1009382 (0.0007) [2023-12-26 22:43:13,556][105692] Updated weights for policy 0, policy_version 1008905 (0.0008) [2023-12-26 22:43:13,602][105585] KL-divergence is very high: 196.4906 [2023-12-26 22:43:13,611][105692] Updated weights for policy 0, policy_version 1008915 (0.0011) [2023-12-26 22:43:13,645][105585] KL-divergence is very high: 360.8924 [2023-12-26 22:43:13,668][105692] Updated weights for policy 0, policy_version 1008926 (0.0010) [2023-12-26 22:43:13,685][105585] KL-divergence is very high: 428.4821 [2023-12-26 22:43:13,971][105620] Updated weights for policy 1, policy_version 1009392 (0.0010) [2023-12-26 22:43:14,029][105620] Updated weights for policy 1, policy_version 1009402 (0.0010) [2023-12-26 22:43:14,088][105620] Updated weights for policy 1, policy_version 1009412 (0.0010) [2023-12-26 22:43:14,409][105692] Updated weights for policy 0, policy_version 1008937 (0.0009) [2023-12-26 22:43:14,458][105692] Updated weights for policy 0, policy_version 1008947 (0.0010) [2023-12-26 22:43:14,504][105692] Updated weights for policy 0, policy_version 1008957 (0.0006) [2023-12-26 22:43:14,554][105692] Updated weights for policy 0, policy_version 1008967 (0.0005) [2023-12-26 22:43:14,772][105620] Updated weights for policy 1, policy_version 1009422 (0.0008) [2023-12-26 22:43:14,842][105620] Updated weights for policy 1, policy_version 1009432 (0.0006) [2023-12-26 22:43:14,913][105620] Updated weights for policy 1, policy_version 1009442 (0.0006) [2023-12-26 22:43:15,193][105692] Updated weights for policy 0, policy_version 1008977 (0.0010) [2023-12-26 22:43:15,242][105692] Updated weights for policy 0, policy_version 1008987 (0.0011) [2023-12-26 22:43:15,291][105692] Updated weights for policy 0, policy_version 1008997 (0.0010) [2023-12-26 22:43:15,487][105620] Updated weights for policy 1, policy_version 1009452 (0.0007) [2023-12-26 22:43:15,556][105620] Updated weights for policy 1, policy_version 1009462 (0.0006) [2023-12-26 22:43:15,622][105620] Updated weights for policy 1, policy_version 1009472 (0.0005) [2023-12-26 22:43:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 516800512. Throughput: 0: 9621.3, 1: 9929.4. Samples: 516771644. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:43:16,063][104569] Avg episode reward: [(0, '8817.848'), (1, '8986.428')] [2023-12-26 22:43:16,067][105692] Updated weights for policy 0, policy_version 1009007 (0.0009) [2023-12-26 22:43:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001009480_258457600.pth... [2023-12-26 22:43:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001008328_258162688.pth [2023-12-26 22:43:16,125][105692] Updated weights for policy 0, policy_version 1009017 (0.0007) [2023-12-26 22:43:16,187][105692] Updated weights for policy 0, policy_version 1009027 (0.0005) [2023-12-26 22:43:16,189][105620] Updated weights for policy 1, policy_version 1009482 (0.0006) [2023-12-26 22:43:16,216][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001009032_258351104.pth... [2023-12-26 22:43:16,220][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001007880_258056192.pth [2023-12-26 22:43:16,240][105620] Updated weights for policy 1, policy_version 1009492 (0.0006) [2023-12-26 22:43:16,284][105620] Updated weights for policy 1, policy_version 1009502 (0.0005) [2023-12-26 22:43:16,332][105620] Updated weights for policy 1, policy_version 1009512 (0.0005) [2023-12-26 22:43:16,730][105692] Updated weights for policy 0, policy_version 1009037 (0.0005) [2023-12-26 22:43:16,787][105692] Updated weights for policy 0, policy_version 1009047 (0.0006) [2023-12-26 22:43:16,849][105692] Updated weights for policy 0, policy_version 1009057 (0.0009) [2023-12-26 22:43:16,978][105620] Updated weights for policy 1, policy_version 1009522 (0.0005) [2023-12-26 22:43:17,031][105620] Updated weights for policy 1, policy_version 1009532 (0.0006) [2023-12-26 22:43:17,086][105620] Updated weights for policy 1, policy_version 1009542 (0.0010) [2023-12-26 22:43:17,578][105692] Updated weights for policy 0, policy_version 1009067 (0.0007) [2023-12-26 22:43:17,627][105692] Updated weights for policy 0, policy_version 1009077 (0.0005) [2023-12-26 22:43:17,687][105692] Updated weights for policy 0, policy_version 1009087 (0.0006) [2023-12-26 22:43:17,783][105620] Updated weights for policy 1, policy_version 1009552 (0.0009) [2023-12-26 22:43:17,853][105620] Updated weights for policy 1, policy_version 1009562 (0.0007) [2023-12-26 22:43:17,915][105620] Updated weights for policy 1, policy_version 1009572 (0.0009) [2023-12-26 22:43:18,428][105692] Updated weights for policy 0, policy_version 1009097 (0.0006) [2023-12-26 22:43:18,481][105692] Updated weights for policy 0, policy_version 1009107 (0.0008) [2023-12-26 22:43:18,512][105620] Updated weights for policy 1, policy_version 1009582 (0.0009) [2023-12-26 22:43:18,538][105692] Updated weights for policy 0, policy_version 1009117 (0.0009) [2023-12-26 22:43:18,569][105620] Updated weights for policy 1, policy_version 1009592 (0.0011) [2023-12-26 22:43:18,591][105692] Updated weights for policy 0, policy_version 1009127 (0.0006) [2023-12-26 22:43:18,625][105620] Updated weights for policy 1, policy_version 1009602 (0.0010) [2023-12-26 22:43:19,312][105692] Updated weights for policy 0, policy_version 1009137 (0.0010) [2023-12-26 22:43:19,377][105692] Updated weights for policy 0, policy_version 1009147 (0.0011) [2023-12-26 22:43:19,390][105620] Updated weights for policy 1, policy_version 1009612 (0.0011) [2023-12-26 22:43:19,436][105692] Updated weights for policy 0, policy_version 1009157 (0.0010) [2023-12-26 22:43:19,449][105620] Updated weights for policy 1, policy_version 1009622 (0.0010) [2023-12-26 22:43:19,513][105620] Updated weights for policy 1, policy_version 1009632 (0.0011) [2023-12-26 22:43:20,152][105692] Updated weights for policy 0, policy_version 1009167 (0.0011) [2023-12-26 22:43:20,220][105692] Updated weights for policy 0, policy_version 1009177 (0.0011) [2023-12-26 22:43:20,286][105692] Updated weights for policy 0, policy_version 1009187 (0.0011) [2023-12-26 22:43:20,289][105620] Updated weights for policy 1, policy_version 1009642 (0.0010) [2023-12-26 22:43:20,349][105620] Updated weights for policy 1, policy_version 1009652 (0.0010) [2023-12-26 22:43:20,414][105620] Updated weights for policy 1, policy_version 1009662 (0.0011) [2023-12-26 22:43:20,481][105620] Updated weights for policy 1, policy_version 1009672 (0.0011) [2023-12-26 22:43:20,953][105692] Updated weights for policy 0, policy_version 1009197 (0.0011) [2023-12-26 22:43:21,020][105692] Updated weights for policy 0, policy_version 1009207 (0.0011) [2023-12-26 22:43:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 516898816. Throughput: 0: 9588.3, 1: 10029.4. Samples: 516893812. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:43:21,062][104569] Avg episode reward: [(0, '8728.649'), (1, '9261.927')] [2023-12-26 22:43:21,087][105692] Updated weights for policy 0, policy_version 1009217 (0.0011) [2023-12-26 22:43:21,189][105620] Updated weights for policy 1, policy_version 1009682 (0.0009) [2023-12-26 22:43:21,255][105620] Updated weights for policy 1, policy_version 1009692 (0.0009) [2023-12-26 22:43:21,324][105620] Updated weights for policy 1, policy_version 1009702 (0.0007) [2023-12-26 22:43:21,880][105692] Updated weights for policy 0, policy_version 1009227 (0.0010) [2023-12-26 22:43:21,942][105692] Updated weights for policy 0, policy_version 1009237 (0.0005) [2023-12-26 22:43:22,006][105620] Updated weights for policy 1, policy_version 1009712 (0.0007) [2023-12-26 22:43:22,006][105692] Updated weights for policy 0, policy_version 1009247 (0.0006) [2023-12-26 22:43:22,072][105620] Updated weights for policy 1, policy_version 1009722 (0.0008) [2023-12-26 22:43:22,137][105620] Updated weights for policy 1, policy_version 1009732 (0.0008) [2023-12-26 22:43:22,639][105692] Updated weights for policy 0, policy_version 1009257 (0.0006) [2023-12-26 22:43:22,702][105692] Updated weights for policy 0, policy_version 1009267 (0.0008) [2023-12-26 22:43:22,750][105585] KL-divergence is very high: 108.1716 [2023-12-26 22:43:22,770][105692] Updated weights for policy 0, policy_version 1009277 (0.0011) [2023-12-26 22:43:22,801][105585] KL-divergence is very high: 120.4763 [2023-12-26 22:43:22,829][105692] Updated weights for policy 0, policy_version 1009287 (0.0011) [2023-12-26 22:43:22,875][105620] Updated weights for policy 1, policy_version 1009742 (0.0008) [2023-12-26 22:43:22,932][105620] Updated weights for policy 1, policy_version 1009752 (0.0009) [2023-12-26 22:43:22,991][105620] Updated weights for policy 1, policy_version 1009762 (0.0008) [2023-12-26 22:43:23,454][105692] Updated weights for policy 0, policy_version 1009297 (0.0010) [2023-12-26 22:43:23,502][105692] Updated weights for policy 0, policy_version 1009307 (0.0009) [2023-12-26 22:43:23,546][105692] Updated weights for policy 0, policy_version 1009317 (0.0005) [2023-12-26 22:43:23,844][105620] Updated weights for policy 1, policy_version 1009772 (0.0008) [2023-12-26 22:43:23,899][105620] Updated weights for policy 1, policy_version 1009782 (0.0009) [2023-12-26 22:43:23,950][105620] Updated weights for policy 1, policy_version 1009792 (0.0008) [2023-12-26 22:43:24,139][105692] Updated weights for policy 0, policy_version 1009327 (0.0007) [2023-12-26 22:43:24,183][105692] Updated weights for policy 0, policy_version 1009337 (0.0008) [2023-12-26 22:43:24,232][105692] Updated weights for policy 0, policy_version 1009347 (0.0005) [2023-12-26 22:43:24,765][105620] Updated weights for policy 1, policy_version 1009802 (0.0009) [2023-12-26 22:43:24,824][105620] Updated weights for policy 1, policy_version 1009812 (0.0008) [2023-12-26 22:43:24,866][105620] Updated weights for policy 1, policy_version 1009822 (0.0007) [2023-12-26 22:43:24,922][105620] Updated weights for policy 1, policy_version 1009832 (0.0007) [2023-12-26 22:43:24,942][105692] Updated weights for policy 0, policy_version 1009357 (0.0006) [2023-12-26 22:43:25,005][105692] Updated weights for policy 0, policy_version 1009367 (0.0010) [2023-12-26 22:43:25,067][105692] Updated weights for policy 0, policy_version 1009377 (0.0010) [2023-12-26 22:43:25,614][105620] Updated weights for policy 1, policy_version 1009842 (0.0008) [2023-12-26 22:43:25,661][105620] Updated weights for policy 1, policy_version 1009852 (0.0008) [2023-12-26 22:43:25,710][105620] Updated weights for policy 1, policy_version 1009862 (0.0008) [2023-12-26 22:43:25,789][105692] Updated weights for policy 0, policy_version 1009387 (0.0009) [2023-12-26 22:43:25,842][105692] Updated weights for policy 0, policy_version 1009397 (0.0010) [2023-12-26 22:43:25,905][105692] Updated weights for policy 0, policy_version 1009407 (0.0011) [2023-12-26 22:43:26,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 517005312. Throughput: 0: 9737.5, 1: 9861.2. Samples: 517010360. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:43:26,062][104569] Avg episode reward: [(0, '8816.613'), (1, '9265.499')] [2023-12-26 22:43:26,482][105620] Updated weights for policy 1, policy_version 1009872 (0.0008) [2023-12-26 22:43:26,541][105620] Updated weights for policy 1, policy_version 1009882 (0.0008) [2023-12-26 22:43:26,603][105620] Updated weights for policy 1, policy_version 1009892 (0.0006) [2023-12-26 22:43:26,629][105692] Updated weights for policy 0, policy_version 1009417 (0.0010) [2023-12-26 22:43:26,680][105692] Updated weights for policy 0, policy_version 1009427 (0.0009) [2023-12-26 22:43:26,734][105692] Updated weights for policy 0, policy_version 1009437 (0.0010) [2023-12-26 22:43:26,778][105692] Updated weights for policy 0, policy_version 1009447 (0.0010) [2023-12-26 22:43:27,344][105620] Updated weights for policy 1, policy_version 1009902 (0.0007) [2023-12-26 22:43:27,403][105620] Updated weights for policy 1, policy_version 1009912 (0.0009) [2023-12-26 22:43:27,463][105620] Updated weights for policy 1, policy_version 1009922 (0.0009) [2023-12-26 22:43:27,470][105692] Updated weights for policy 0, policy_version 1009457 (0.0006) [2023-12-26 22:43:27,532][105692] Updated weights for policy 0, policy_version 1009467 (0.0006) [2023-12-26 22:43:27,596][105692] Updated weights for policy 0, policy_version 1009477 (0.0005) [2023-12-26 22:43:28,110][105692] Updated weights for policy 0, policy_version 1009487 (0.0005) [2023-12-26 22:43:28,161][105692] Updated weights for policy 0, policy_version 1009497 (0.0005) [2023-12-26 22:43:28,215][105692] Updated weights for policy 0, policy_version 1009507 (0.0007) [2023-12-26 22:43:28,288][105620] Updated weights for policy 1, policy_version 1009932 (0.0009) [2023-12-26 22:43:28,341][105620] Updated weights for policy 1, policy_version 1009942 (0.0008) [2023-12-26 22:43:28,400][105620] Updated weights for policy 1, policy_version 1009952 (0.0008) [2023-12-26 22:43:28,953][105692] Updated weights for policy 0, policy_version 1009517 (0.0009) [2023-12-26 22:43:29,012][105692] Updated weights for policy 0, policy_version 1009527 (0.0009) [2023-12-26 22:43:29,079][105692] Updated weights for policy 0, policy_version 1009537 (0.0009) [2023-12-26 22:43:29,101][105620] Updated weights for policy 1, policy_version 1009962 (0.0008) [2023-12-26 22:43:29,161][105620] Updated weights for policy 1, policy_version 1009972 (0.0006) [2023-12-26 22:43:29,224][105620] Updated weights for policy 1, policy_version 1009982 (0.0006) [2023-12-26 22:43:29,291][105620] Updated weights for policy 1, policy_version 1009992 (0.0007) [2023-12-26 22:43:29,834][105692] Updated weights for policy 0, policy_version 1009547 (0.0010) [2023-12-26 22:43:29,884][105620] Updated weights for policy 1, policy_version 1010002 (0.0007) [2023-12-26 22:43:29,886][105692] Updated weights for policy 0, policy_version 1009557 (0.0008) [2023-12-26 22:43:29,949][105692] Updated weights for policy 0, policy_version 1009567 (0.0009) [2023-12-26 22:43:29,951][105620] Updated weights for policy 1, policy_version 1010012 (0.0007) [2023-12-26 22:43:30,006][105620] Updated weights for policy 1, policy_version 1010022 (0.0008) [2023-12-26 22:43:30,580][105692] Updated weights for policy 0, policy_version 1009577 (0.0006) [2023-12-26 22:43:30,655][105692] Updated weights for policy 0, policy_version 1009587 (0.0005) [2023-12-26 22:43:30,725][105692] Updated weights for policy 0, policy_version 1009597 (0.0005) [2023-12-26 22:43:30,772][105692] Updated weights for policy 0, policy_version 1009607 (0.0005) [2023-12-26 22:43:30,850][105620] Updated weights for policy 1, policy_version 1010033 (0.0009) [2023-12-26 22:43:30,902][105620] Updated weights for policy 1, policy_version 1010044 (0.0009) [2023-12-26 22:43:30,963][105620] Updated weights for policy 1, policy_version 1010054 (0.0009) [2023-12-26 22:43:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 517103616. Throughput: 0: 9768.8, 1: 9854.1. Samples: 517069108. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:43:31,062][104569] Avg episode reward: [(0, '8723.771'), (1, '8982.188')] [2023-12-26 22:43:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001009608_258498560.pth... [2023-12-26 22:43:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001010056_258605056.pth... [2023-12-26 22:43:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001008424_258195456.pth [2023-12-26 22:43:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001008904_258310144.pth [2023-12-26 22:43:31,380][105692] Updated weights for policy 0, policy_version 1009617 (0.0008) [2023-12-26 22:43:31,442][105692] Updated weights for policy 0, policy_version 1009627 (0.0009) [2023-12-26 22:43:31,492][105692] Updated weights for policy 0, policy_version 1009637 (0.0006) [2023-12-26 22:43:31,858][105620] Updated weights for policy 1, policy_version 1010064 (0.0009) [2023-12-26 22:43:31,913][105620] Updated weights for policy 1, policy_version 1010074 (0.0009) [2023-12-26 22:43:31,967][105620] Updated weights for policy 1, policy_version 1010086 (0.0010) [2023-12-26 22:43:32,114][105692] Updated weights for policy 0, policy_version 1009647 (0.0005) [2023-12-26 22:43:32,172][105692] Updated weights for policy 0, policy_version 1009657 (0.0005) [2023-12-26 22:43:32,230][105692] Updated weights for policy 0, policy_version 1009667 (0.0005) [2023-12-26 22:43:32,777][105620] Updated weights for policy 1, policy_version 1010096 (0.0010) [2023-12-26 22:43:32,808][105692] Updated weights for policy 0, policy_version 1009677 (0.0008) [2023-12-26 22:43:32,831][105620] Updated weights for policy 1, policy_version 1010106 (0.0007) [2023-12-26 22:43:32,857][105692] Updated weights for policy 0, policy_version 1009687 (0.0010) [2023-12-26 22:43:32,889][105620] Updated weights for policy 1, policy_version 1010116 (0.0005) [2023-12-26 22:43:32,909][105692] Updated weights for policy 0, policy_version 1009697 (0.0010) [2023-12-26 22:43:33,623][105692] Updated weights for policy 0, policy_version 1009707 (0.0009) [2023-12-26 22:43:33,670][105620] Updated weights for policy 1, policy_version 1010126 (0.0010) [2023-12-26 22:43:33,683][105692] Updated weights for policy 0, policy_version 1009717 (0.0007) [2023-12-26 22:43:33,720][105620] Updated weights for policy 1, policy_version 1010136 (0.0008) [2023-12-26 22:43:33,732][105692] Updated weights for policy 0, policy_version 1009727 (0.0006) [2023-12-26 22:43:33,774][105620] Updated weights for policy 1, policy_version 1010146 (0.0008) [2023-12-26 22:43:34,337][105692] Updated weights for policy 0, policy_version 1009737 (0.0007) [2023-12-26 22:43:34,402][105692] Updated weights for policy 0, policy_version 1009747 (0.0008) [2023-12-26 22:43:34,468][105692] Updated weights for policy 0, policy_version 1009757 (0.0008) [2023-12-26 22:43:34,526][105692] Updated weights for policy 0, policy_version 1009767 (0.0010) [2023-12-26 22:43:34,565][105620] Updated weights for policy 1, policy_version 1010156 (0.0008) [2023-12-26 22:43:34,629][105620] Updated weights for policy 1, policy_version 1010166 (0.0009) [2023-12-26 22:43:34,683][105620] Updated weights for policy 1, policy_version 1010176 (0.0010) [2023-12-26 22:43:35,220][105692] Updated weights for policy 0, policy_version 1009777 (0.0006) [2023-12-26 22:43:35,285][105692] Updated weights for policy 0, policy_version 1009787 (0.0006) [2023-12-26 22:43:35,349][105692] Updated weights for policy 0, policy_version 1009797 (0.0006) [2023-12-26 22:43:35,487][105620] Updated weights for policy 1, policy_version 1010186 (0.0009) [2023-12-26 22:43:35,549][105620] Updated weights for policy 1, policy_version 1010196 (0.0009) [2023-12-26 22:43:35,596][105620] Updated weights for policy 1, policy_version 1010206 (0.0009) [2023-12-26 22:43:35,641][105620] Updated weights for policy 1, policy_version 1010216 (0.0008) [2023-12-26 22:43:36,000][105692] Updated weights for policy 0, policy_version 1009807 (0.0009) [2023-12-26 22:43:36,057][105692] Updated weights for policy 0, policy_version 1009817 (0.0009) [2023-12-26 22:43:36,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 517193728. Throughput: 0: 9862.6, 1: 9756.3. Samples: 517186504. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:43:36,063][104569] Avg episode reward: [(0, '8366.384'), (1, '8608.125')] [2023-12-26 22:43:36,109][105692] Updated weights for policy 0, policy_version 1009827 (0.0009) [2023-12-26 22:43:36,356][105620] Updated weights for policy 1, policy_version 1010226 (0.0009) [2023-12-26 22:43:36,422][105620] Updated weights for policy 1, policy_version 1010236 (0.0009) [2023-12-26 22:43:36,484][105620] Updated weights for policy 1, policy_version 1010246 (0.0009) [2023-12-26 22:43:36,898][105692] Updated weights for policy 0, policy_version 1009837 (0.0008) [2023-12-26 22:43:36,947][105692] Updated weights for policy 0, policy_version 1009847 (0.0009) [2023-12-26 22:43:37,006][105692] Updated weights for policy 0, policy_version 1009857 (0.0009) [2023-12-26 22:43:37,235][105620] Updated weights for policy 1, policy_version 1010256 (0.0009) [2023-12-26 22:43:37,292][105620] Updated weights for policy 1, policy_version 1010266 (0.0008) [2023-12-26 22:43:37,348][105620] Updated weights for policy 1, policy_version 1010276 (0.0005) [2023-12-26 22:43:37,776][105692] Updated weights for policy 0, policy_version 1009867 (0.0009) [2023-12-26 22:43:37,831][105692] Updated weights for policy 0, policy_version 1009877 (0.0010) [2023-12-26 22:43:37,886][105692] Updated weights for policy 0, policy_version 1009887 (0.0007) [2023-12-26 22:43:38,052][105620] Updated weights for policy 1, policy_version 1010286 (0.0007) [2023-12-26 22:43:38,111][105620] Updated weights for policy 1, policy_version 1010296 (0.0009) [2023-12-26 22:43:38,157][105620] Updated weights for policy 1, policy_version 1010306 (0.0008) [2023-12-26 22:43:38,611][105692] Updated weights for policy 0, policy_version 1009897 (0.0006) [2023-12-26 22:43:38,658][105692] Updated weights for policy 0, policy_version 1009907 (0.0009) [2023-12-26 22:43:38,712][105692] Updated weights for policy 0, policy_version 1009917 (0.0009) [2023-12-26 22:43:38,763][105692] Updated weights for policy 0, policy_version 1009927 (0.0009) [2023-12-26 22:43:38,875][105620] Updated weights for policy 1, policy_version 1010316 (0.0008) [2023-12-26 22:43:38,938][105620] Updated weights for policy 1, policy_version 1010326 (0.0008) [2023-12-26 22:43:38,996][105620] Updated weights for policy 1, policy_version 1010336 (0.0009) [2023-12-26 22:43:39,586][105692] Updated weights for policy 0, policy_version 1009937 (0.0006) [2023-12-26 22:43:39,658][105692] Updated weights for policy 0, policy_version 1009947 (0.0009) [2023-12-26 22:43:39,682][105620] Updated weights for policy 1, policy_version 1010346 (0.0008) [2023-12-26 22:43:39,724][105692] Updated weights for policy 0, policy_version 1009957 (0.0010) [2023-12-26 22:43:39,744][105620] Updated weights for policy 1, policy_version 1010356 (0.0006) [2023-12-26 22:43:39,798][105620] Updated weights for policy 1, policy_version 1010366 (0.0007) [2023-12-26 22:43:39,867][105620] Updated weights for policy 1, policy_version 1010376 (0.0007) [2023-12-26 22:43:40,444][105692] Updated weights for policy 0, policy_version 1009967 (0.0008) [2023-12-26 22:43:40,504][105692] Updated weights for policy 0, policy_version 1009977 (0.0010) [2023-12-26 22:43:40,539][105620] Updated weights for policy 1, policy_version 1010386 (0.0006) [2023-12-26 22:43:40,554][105692] Updated weights for policy 0, policy_version 1009987 (0.0010) [2023-12-26 22:43:40,595][105620] Updated weights for policy 1, policy_version 1010396 (0.0008) [2023-12-26 22:43:40,664][105620] Updated weights for policy 1, policy_version 1010406 (0.0009) [2023-12-26 22:43:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 517292032. Throughput: 0: 9905.3, 1: 9669.9. Samples: 517301816. Policy #0 lag: (min: 3.0, avg: 13.2, max: 35.0) [2023-12-26 22:43:41,062][104569] Avg episode reward: [(0, '8638.280'), (1, '8619.874')] [2023-12-26 22:43:41,177][105692] Updated weights for policy 0, policy_version 1009997 (0.0009) [2023-12-26 22:43:41,240][105692] Updated weights for policy 0, policy_version 1010007 (0.0008) [2023-12-26 22:43:41,297][105692] Updated weights for policy 0, policy_version 1010017 (0.0008) [2023-12-26 22:43:41,514][105620] Updated weights for policy 1, policy_version 1010416 (0.0010) [2023-12-26 22:43:41,567][105620] Updated weights for policy 1, policy_version 1010426 (0.0011) [2023-12-26 22:43:41,633][105620] Updated weights for policy 1, policy_version 1010436 (0.0011) [2023-12-26 22:43:42,061][105692] Updated weights for policy 0, policy_version 1010027 (0.0008) [2023-12-26 22:43:42,129][105692] Updated weights for policy 0, policy_version 1010037 (0.0008) [2023-12-26 22:43:42,190][105692] Updated weights for policy 0, policy_version 1010047 (0.0008) [2023-12-26 22:43:42,388][105620] Updated weights for policy 1, policy_version 1010446 (0.0011) [2023-12-26 22:43:42,444][105620] Updated weights for policy 1, policy_version 1010456 (0.0010) [2023-12-26 22:43:42,498][105620] Updated weights for policy 1, policy_version 1010466 (0.0010) [2023-12-26 22:43:42,953][105692] Updated weights for policy 0, policy_version 1010057 (0.0008) [2023-12-26 22:43:43,005][105692] Updated weights for policy 0, policy_version 1010067 (0.0008) [2023-12-26 22:43:43,048][105692] Updated weights for policy 0, policy_version 1010077 (0.0007) [2023-12-26 22:43:43,100][105692] Updated weights for policy 0, policy_version 1010087 (0.0008) [2023-12-26 22:43:43,239][105620] Updated weights for policy 1, policy_version 1010476 (0.0010) [2023-12-26 22:43:43,288][105620] Updated weights for policy 1, policy_version 1010486 (0.0009) [2023-12-26 22:43:43,338][105620] Updated weights for policy 1, policy_version 1010496 (0.0009) [2023-12-26 22:43:43,880][105692] Updated weights for policy 0, policy_version 1010097 (0.0010) [2023-12-26 22:43:43,928][105692] Updated weights for policy 0, policy_version 1010107 (0.0010) [2023-12-26 22:43:43,976][105692] Updated weights for policy 0, policy_version 1010117 (0.0010) [2023-12-26 22:43:44,118][105620] Updated weights for policy 1, policy_version 1010506 (0.0009) [2023-12-26 22:43:44,183][105620] Updated weights for policy 1, policy_version 1010516 (0.0010) [2023-12-26 22:43:44,239][105620] Updated weights for policy 1, policy_version 1010526 (0.0010) [2023-12-26 22:43:44,297][105620] Updated weights for policy 1, policy_version 1010536 (0.0010) [2023-12-26 22:43:44,747][105692] Updated weights for policy 0, policy_version 1010127 (0.0009) [2023-12-26 22:43:44,811][105692] Updated weights for policy 0, policy_version 1010137 (0.0009) [2023-12-26 22:43:44,868][105692] Updated weights for policy 0, policy_version 1010147 (0.0008) [2023-12-26 22:43:45,041][105620] Updated weights for policy 1, policy_version 1010546 (0.0010) [2023-12-26 22:43:45,103][105620] Updated weights for policy 1, policy_version 1010556 (0.0010) [2023-12-26 22:43:45,171][105620] Updated weights for policy 1, policy_version 1010566 (0.0011) [2023-12-26 22:43:45,656][105692] Updated weights for policy 0, policy_version 1010157 (0.0007) [2023-12-26 22:43:45,720][105692] Updated weights for policy 0, policy_version 1010167 (0.0006) [2023-12-26 22:43:45,775][105692] Updated weights for policy 0, policy_version 1010177 (0.0011) [2023-12-26 22:43:45,912][105620] Updated weights for policy 1, policy_version 1010576 (0.0010) [2023-12-26 22:43:45,970][105620] Updated weights for policy 1, policy_version 1010586 (0.0010) [2023-12-26 22:43:46,034][105620] Updated weights for policy 1, policy_version 1010596 (0.0010) [2023-12-26 22:43:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 517390336. Throughput: 0: 9971.6, 1: 9578.1. Samples: 517357504. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:43:46,062][104569] Avg episode reward: [(0, '8603.675'), (1, '9077.103')] [2023-12-26 22:43:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001010600_258744320.pth... [2023-12-26 22:43:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001010184_258646016.pth... [2023-12-26 22:43:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001009480_258457600.pth [2023-12-26 22:43:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001009032_258351104.pth [2023-12-26 22:43:46,452][105692] Updated weights for policy 0, policy_version 1010187 (0.0010) [2023-12-26 22:43:46,504][105692] Updated weights for policy 0, policy_version 1010197 (0.0010) [2023-12-26 22:43:46,562][105692] Updated weights for policy 0, policy_version 1010207 (0.0007) [2023-12-26 22:43:46,754][105620] Updated weights for policy 1, policy_version 1010606 (0.0010) [2023-12-26 22:43:46,813][105620] Updated weights for policy 1, policy_version 1010616 (0.0010) [2023-12-26 22:43:46,867][105620] Updated weights for policy 1, policy_version 1010626 (0.0010) [2023-12-26 22:43:47,219][105692] Updated weights for policy 0, policy_version 1010217 (0.0006) [2023-12-26 22:43:47,272][105692] Updated weights for policy 0, policy_version 1010227 (0.0008) [2023-12-26 22:43:47,320][105692] Updated weights for policy 0, policy_version 1010237 (0.0008) [2023-12-26 22:43:47,377][105692] Updated weights for policy 0, policy_version 1010247 (0.0008) [2023-12-26 22:43:47,631][105620] Updated weights for policy 1, policy_version 1010636 (0.0010) [2023-12-26 22:43:47,692][105620] Updated weights for policy 1, policy_version 1010646 (0.0010) [2023-12-26 22:43:47,739][105620] Updated weights for policy 1, policy_version 1010656 (0.0010) [2023-12-26 22:43:48,117][105692] Updated weights for policy 0, policy_version 1010257 (0.0006) [2023-12-26 22:43:48,173][105692] Updated weights for policy 0, policy_version 1010267 (0.0005) [2023-12-26 22:43:48,229][105692] Updated weights for policy 0, policy_version 1010277 (0.0007) [2023-12-26 22:43:48,463][105620] Updated weights for policy 1, policy_version 1010666 (0.0010) [2023-12-26 22:43:48,525][105620] Updated weights for policy 1, policy_version 1010676 (0.0010) [2023-12-26 22:43:48,585][105620] Updated weights for policy 1, policy_version 1010686 (0.0010) [2023-12-26 22:43:48,640][105620] Updated weights for policy 1, policy_version 1010696 (0.0010) [2023-12-26 22:43:48,913][105692] Updated weights for policy 0, policy_version 1010287 (0.0008) [2023-12-26 22:43:48,966][105692] Updated weights for policy 0, policy_version 1010297 (0.0008) [2023-12-26 22:43:49,031][105692] Updated weights for policy 0, policy_version 1010307 (0.0008) [2023-12-26 22:43:49,317][105620] Updated weights for policy 1, policy_version 1010706 (0.0010) [2023-12-26 22:43:49,384][105620] Updated weights for policy 1, policy_version 1010716 (0.0011) [2023-12-26 22:43:49,442][105620] Updated weights for policy 1, policy_version 1010726 (0.0010) [2023-12-26 22:43:49,794][105692] Updated weights for policy 0, policy_version 1010317 (0.0009) [2023-12-26 22:43:49,854][105692] Updated weights for policy 0, policy_version 1010327 (0.0009) [2023-12-26 22:43:49,911][105692] Updated weights for policy 0, policy_version 1010337 (0.0007) [2023-12-26 22:43:50,187][105620] Updated weights for policy 1, policy_version 1010736 (0.0010) [2023-12-26 22:43:50,245][105620] Updated weights for policy 1, policy_version 1010746 (0.0010) [2023-12-26 22:43:50,311][105620] Updated weights for policy 1, policy_version 1010756 (0.0010) [2023-12-26 22:43:50,714][105692] Updated weights for policy 0, policy_version 1010347 (0.0006) [2023-12-26 22:43:50,779][105692] Updated weights for policy 0, policy_version 1010357 (0.0006) [2023-12-26 22:43:50,832][105692] Updated weights for policy 0, policy_version 1010367 (0.0007) [2023-12-26 22:43:51,008][105620] Updated weights for policy 1, policy_version 1010766 (0.0008) [2023-12-26 22:43:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 517480448. Throughput: 0: 9975.8, 1: 9581.9. Samples: 517472132. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:43:51,062][104569] Avg episode reward: [(0, '7988.221'), (1, '8987.061')] [2023-12-26 22:43:51,079][105620] Updated weights for policy 1, policy_version 1010776 (0.0007) [2023-12-26 22:43:51,141][105620] Updated weights for policy 1, policy_version 1010786 (0.0009) [2023-12-26 22:43:51,524][105692] Updated weights for policy 0, policy_version 1010377 (0.0006) [2023-12-26 22:43:51,588][105692] Updated weights for policy 0, policy_version 1010387 (0.0008) [2023-12-26 22:43:51,649][105692] Updated weights for policy 0, policy_version 1010397 (0.0008) [2023-12-26 22:43:51,708][105692] Updated weights for policy 0, policy_version 1010407 (0.0008) [2023-12-26 22:43:51,849][105620] Updated weights for policy 1, policy_version 1010796 (0.0009) [2023-12-26 22:43:51,904][105620] Updated weights for policy 1, policy_version 1010806 (0.0008) [2023-12-26 22:43:51,963][105620] Updated weights for policy 1, policy_version 1010816 (0.0008) [2023-12-26 22:43:52,436][105692] Updated weights for policy 0, policy_version 1010417 (0.0008) [2023-12-26 22:43:52,497][105692] Updated weights for policy 0, policy_version 1010427 (0.0008) [2023-12-26 22:43:52,553][105692] Updated weights for policy 0, policy_version 1010437 (0.0005) [2023-12-26 22:43:52,698][105620] Updated weights for policy 1, policy_version 1010826 (0.0008) [2023-12-26 22:43:52,763][105620] Updated weights for policy 1, policy_version 1010836 (0.0010) [2023-12-26 22:43:52,853][105620] Updated weights for policy 1, policy_version 1010846 (0.0009) [2023-12-26 22:43:52,912][105620] Updated weights for policy 1, policy_version 1010856 (0.0007) [2023-12-26 22:43:53,359][105692] Updated weights for policy 0, policy_version 1010447 (0.0007) [2023-12-26 22:43:53,414][105692] Updated weights for policy 0, policy_version 1010457 (0.0009) [2023-12-26 22:43:53,465][105692] Updated weights for policy 0, policy_version 1010467 (0.0008) [2023-12-26 22:43:53,483][105620] Updated weights for policy 1, policy_version 1010866 (0.0008) [2023-12-26 22:43:53,528][105620] Updated weights for policy 1, policy_version 1010876 (0.0007) [2023-12-26 22:43:53,574][105620] Updated weights for policy 1, policy_version 1010886 (0.0008) [2023-12-26 22:43:54,167][105692] Updated weights for policy 0, policy_version 1010477 (0.0007) [2023-12-26 22:43:54,200][105620] Updated weights for policy 1, policy_version 1010896 (0.0008) [2023-12-26 22:43:54,229][105692] Updated weights for policy 0, policy_version 1010487 (0.0009) [2023-12-26 22:43:54,265][105620] Updated weights for policy 1, policy_version 1010906 (0.0006) [2023-12-26 22:43:54,288][105692] Updated weights for policy 0, policy_version 1010497 (0.0010) [2023-12-26 22:43:54,324][105620] Updated weights for policy 1, policy_version 1010916 (0.0010) [2023-12-26 22:43:54,896][105620] Updated weights for policy 1, policy_version 1010926 (0.0008) [2023-12-26 22:43:54,943][105620] Updated weights for policy 1, policy_version 1010936 (0.0010) [2023-12-26 22:43:54,999][105692] Updated weights for policy 0, policy_version 1010507 (0.0009) [2023-12-26 22:43:55,004][105620] Updated weights for policy 1, policy_version 1010946 (0.0010) [2023-12-26 22:43:55,064][105692] Updated weights for policy 0, policy_version 1010517 (0.0011) [2023-12-26 22:43:55,122][105692] Updated weights for policy 0, policy_version 1010527 (0.0009) [2023-12-26 22:43:55,672][105620] Updated weights for policy 1, policy_version 1010956 (0.0007) [2023-12-26 22:43:55,684][105692] Updated weights for policy 0, policy_version 1010537 (0.0005) [2023-12-26 22:43:55,734][105692] Updated weights for policy 0, policy_version 1010547 (0.0005) [2023-12-26 22:43:55,738][105620] Updated weights for policy 1, policy_version 1010966 (0.0010) [2023-12-26 22:43:55,786][105692] Updated weights for policy 0, policy_version 1010557 (0.0007) [2023-12-26 22:43:55,797][105620] Updated weights for policy 1, policy_version 1010976 (0.0010) [2023-12-26 22:43:55,841][105692] Updated weights for policy 0, policy_version 1010567 (0.0010) [2023-12-26 22:43:56,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 517586944. Throughput: 0: 9897.7, 1: 9669.4. Samples: 517592616. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:43:56,063][104569] Avg episode reward: [(0, '8560.361'), (1, '9172.866')] [2023-12-26 22:43:56,464][105620] Updated weights for policy 1, policy_version 1010986 (0.0010) [2023-12-26 22:43:56,522][105620] Updated weights for policy 1, policy_version 1010996 (0.0008) [2023-12-26 22:43:56,566][105692] Updated weights for policy 0, policy_version 1010577 (0.0010) [2023-12-26 22:43:56,575][105620] Updated weights for policy 1, policy_version 1011006 (0.0008) [2023-12-26 22:43:56,610][105692] Updated weights for policy 0, policy_version 1010587 (0.0010) [2023-12-26 22:43:56,627][105620] Updated weights for policy 1, policy_version 1011016 (0.0007) [2023-12-26 22:43:56,662][105692] Updated weights for policy 0, policy_version 1010597 (0.0010) [2023-12-26 22:43:57,275][105620] Updated weights for policy 1, policy_version 1011026 (0.0005) [2023-12-26 22:43:57,331][105692] Updated weights for policy 0, policy_version 1010607 (0.0010) [2023-12-26 22:43:57,333][105620] Updated weights for policy 1, policy_version 1011036 (0.0007) [2023-12-26 22:43:57,385][105620] Updated weights for policy 1, policy_version 1011046 (0.0009) [2023-12-26 22:43:57,390][105692] Updated weights for policy 0, policy_version 1010617 (0.0010) [2023-12-26 22:43:57,449][105692] Updated weights for policy 0, policy_version 1010627 (0.0010) [2023-12-26 22:43:58,067][105620] Updated weights for policy 1, policy_version 1011056 (0.0006) [2023-12-26 22:43:58,127][105620] Updated weights for policy 1, policy_version 1011066 (0.0006) [2023-12-26 22:43:58,200][105692] Updated weights for policy 0, policy_version 1010637 (0.0010) [2023-12-26 22:43:58,202][105620] Updated weights for policy 1, policy_version 1011076 (0.0006) [2023-12-26 22:43:58,272][105692] Updated weights for policy 0, policy_version 1010647 (0.0011) [2023-12-26 22:43:58,336][105692] Updated weights for policy 0, policy_version 1010657 (0.0010) [2023-12-26 22:43:58,912][105620] Updated weights for policy 1, policy_version 1011086 (0.0008) [2023-12-26 22:43:58,981][105620] Updated weights for policy 1, policy_version 1011096 (0.0009) [2023-12-26 22:43:59,041][105620] Updated weights for policy 1, policy_version 1011107 (0.0008) [2023-12-26 22:43:59,070][105692] Updated weights for policy 0, policy_version 1010667 (0.0009) [2023-12-26 22:43:59,140][105692] Updated weights for policy 0, policy_version 1010677 (0.0005) [2023-12-26 22:43:59,210][105692] Updated weights for policy 0, policy_version 1010687 (0.0008) [2023-12-26 22:43:59,854][105620] Updated weights for policy 1, policy_version 1011117 (0.0008) [2023-12-26 22:43:59,902][105692] Updated weights for policy 0, policy_version 1010697 (0.0008) [2023-12-26 22:43:59,912][105620] Updated weights for policy 1, policy_version 1011127 (0.0008) [2023-12-26 22:43:59,965][105692] Updated weights for policy 0, policy_version 1010707 (0.0010) [2023-12-26 22:43:59,979][105620] Updated weights for policy 1, policy_version 1011137 (0.0008) [2023-12-26 22:44:00,022][105692] Updated weights for policy 0, policy_version 1010717 (0.0010) [2023-12-26 22:44:00,084][105692] Updated weights for policy 0, policy_version 1010727 (0.0010) [2023-12-26 22:44:00,758][105620] Updated weights for policy 1, policy_version 1011147 (0.0006) [2023-12-26 22:44:00,811][105620] Updated weights for policy 1, policy_version 1011157 (0.0005) [2023-12-26 22:44:00,837][105692] Updated weights for policy 0, policy_version 1010737 (0.0009) [2023-12-26 22:44:00,863][105620] Updated weights for policy 1, policy_version 1011167 (0.0005) [2023-12-26 22:44:00,884][105692] Updated weights for policy 0, policy_version 1010747 (0.0010) [2023-12-26 22:44:00,940][105692] Updated weights for policy 0, policy_version 1010757 (0.0010) [2023-12-26 22:44:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.9, 300 sec: 19494.2). Total num frames: 517685248. Throughput: 0: 9910.0, 1: 9664.0. Samples: 517652472. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:44:01,062][104569] Avg episode reward: [(0, '9083.634'), (1, '9174.440')] [2023-12-26 22:44:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001010760_258793472.pth... [2023-12-26 22:44:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001011176_258891776.pth... [2023-12-26 22:44:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001009608_258498560.pth [2023-12-26 22:44:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001010056_258605056.pth [2023-12-26 22:44:01,525][105620] Updated weights for policy 1, policy_version 1011177 (0.0005) [2023-12-26 22:44:01,591][105620] Updated weights for policy 1, policy_version 1011187 (0.0008) [2023-12-26 22:44:01,655][105620] Updated weights for policy 1, policy_version 1011197 (0.0008) [2023-12-26 22:44:01,670][105692] Updated weights for policy 0, policy_version 1010767 (0.0010) [2023-12-26 22:44:01,708][105620] Updated weights for policy 1, policy_version 1011207 (0.0006) [2023-12-26 22:44:01,736][105692] Updated weights for policy 0, policy_version 1010777 (0.0011) [2023-12-26 22:44:01,796][105692] Updated weights for policy 0, policy_version 1010787 (0.0010) [2023-12-26 22:44:02,365][105620] Updated weights for policy 1, policy_version 1011217 (0.0008) [2023-12-26 22:44:02,416][105620] Updated weights for policy 1, policy_version 1011227 (0.0007) [2023-12-26 22:44:02,461][105620] Updated weights for policy 1, policy_version 1011237 (0.0008) [2023-12-26 22:44:02,548][105692] Updated weights for policy 0, policy_version 1010797 (0.0010) [2023-12-26 22:44:02,606][105692] Updated weights for policy 0, policy_version 1010807 (0.0010) [2023-12-26 22:44:02,650][105692] Updated weights for policy 0, policy_version 1010817 (0.0010) [2023-12-26 22:44:03,144][105620] Updated weights for policy 1, policy_version 1011247 (0.0008) [2023-12-26 22:44:03,206][105620] Updated weights for policy 1, policy_version 1011257 (0.0008) [2023-12-26 22:44:03,265][105620] Updated weights for policy 1, policy_version 1011267 (0.0008) [2023-12-26 22:44:03,408][105692] Updated weights for policy 0, policy_version 1010827 (0.0010) [2023-12-26 22:44:03,468][105692] Updated weights for policy 0, policy_version 1010837 (0.0009) [2023-12-26 22:44:03,535][105692] Updated weights for policy 0, policy_version 1010847 (0.0005) [2023-12-26 22:44:03,868][105620] Updated weights for policy 1, policy_version 1011277 (0.0008) [2023-12-26 22:44:03,921][105620] Updated weights for policy 1, policy_version 1011287 (0.0006) [2023-12-26 22:44:03,988][105620] Updated weights for policy 1, policy_version 1011297 (0.0007) [2023-12-26 22:44:04,100][105692] Updated weights for policy 0, policy_version 1010857 (0.0005) [2023-12-26 22:44:04,167][105692] Updated weights for policy 0, policy_version 1010867 (0.0010) [2023-12-26 22:44:04,238][105692] Updated weights for policy 0, policy_version 1010877 (0.0011) [2023-12-26 22:44:04,302][105692] Updated weights for policy 0, policy_version 1010887 (0.0011) [2023-12-26 22:44:04,724][105620] Updated weights for policy 1, policy_version 1011307 (0.0009) [2023-12-26 22:44:04,774][105620] Updated weights for policy 1, policy_version 1011317 (0.0009) [2023-12-26 22:44:04,817][105620] Updated weights for policy 1, policy_version 1011327 (0.0007) [2023-12-26 22:44:05,021][105692] Updated weights for policy 0, policy_version 1010897 (0.0011) [2023-12-26 22:44:05,087][105692] Updated weights for policy 0, policy_version 1010907 (0.0010) [2023-12-26 22:44:05,151][105692] Updated weights for policy 0, policy_version 1010917 (0.0010) [2023-12-26 22:44:05,496][105620] Updated weights for policy 1, policy_version 1011337 (0.0009) [2023-12-26 22:44:05,556][105620] Updated weights for policy 1, policy_version 1011347 (0.0008) [2023-12-26 22:44:05,617][105620] Updated weights for policy 1, policy_version 1011357 (0.0009) [2023-12-26 22:44:05,676][105620] Updated weights for policy 1, policy_version 1011367 (0.0009) [2023-12-26 22:44:05,814][105692] Updated weights for policy 0, policy_version 1010927 (0.0009) [2023-12-26 22:44:05,863][105692] Updated weights for policy 0, policy_version 1010937 (0.0009) [2023-12-26 22:44:05,911][105692] Updated weights for policy 0, policy_version 1010947 (0.0009) [2023-12-26 22:44:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 517783552. Throughput: 0: 9872.4, 1: 9595.5. Samples: 517769872. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:44:06,063][104569] Avg episode reward: [(0, '8816.587'), (1, '8900.622')] [2023-12-26 22:44:06,425][105620] Updated weights for policy 1, policy_version 1011377 (0.0009) [2023-12-26 22:44:06,478][105620] Updated weights for policy 1, policy_version 1011387 (0.0009) [2023-12-26 22:44:06,536][105620] Updated weights for policy 1, policy_version 1011397 (0.0009) [2023-12-26 22:44:06,623][105692] Updated weights for policy 0, policy_version 1010957 (0.0008) [2023-12-26 22:44:06,678][105692] Updated weights for policy 0, policy_version 1010967 (0.0008) [2023-12-26 22:44:06,738][105692] Updated weights for policy 0, policy_version 1010977 (0.0007) [2023-12-26 22:44:07,350][105620] Updated weights for policy 1, policy_version 1011407 (0.0009) [2023-12-26 22:44:07,402][105620] Updated weights for policy 1, policy_version 1011417 (0.0008) [2023-12-26 22:44:07,457][105692] Updated weights for policy 0, policy_version 1010987 (0.0009) [2023-12-26 22:44:07,463][105620] Updated weights for policy 1, policy_version 1011427 (0.0007) [2023-12-26 22:44:07,511][105692] Updated weights for policy 0, policy_version 1010997 (0.0010) [2023-12-26 22:44:07,574][105692] Updated weights for policy 0, policy_version 1011007 (0.0010) [2023-12-26 22:44:08,172][105692] Updated weights for policy 0, policy_version 1011017 (0.0010) [2023-12-26 22:44:08,205][105620] Updated weights for policy 1, policy_version 1011437 (0.0005) [2023-12-26 22:44:08,228][105692] Updated weights for policy 0, policy_version 1011027 (0.0007) [2023-12-26 22:44:08,265][105620] Updated weights for policy 1, policy_version 1011447 (0.0005) [2023-12-26 22:44:08,284][105692] Updated weights for policy 0, policy_version 1011037 (0.0010) [2023-12-26 22:44:08,323][105620] Updated weights for policy 1, policy_version 1011457 (0.0006) [2023-12-26 22:44:08,342][105692] Updated weights for policy 0, policy_version 1011047 (0.0010) [2023-12-26 22:44:08,976][105692] Updated weights for policy 0, policy_version 1011057 (0.0009) [2023-12-26 22:44:09,034][105692] Updated weights for policy 0, policy_version 1011067 (0.0009) [2023-12-26 22:44:09,087][105692] Updated weights for policy 0, policy_version 1011077 (0.0008) [2023-12-26 22:44:09,092][105620] Updated weights for policy 1, policy_version 1011467 (0.0009) [2023-12-26 22:44:09,151][105620] Updated weights for policy 1, policy_version 1011477 (0.0009) [2023-12-26 22:44:09,216][105620] Updated weights for policy 1, policy_version 1011487 (0.0009) [2023-12-26 22:44:09,890][105620] Updated weights for policy 1, policy_version 1011497 (0.0009) [2023-12-26 22:44:09,906][105692] Updated weights for policy 0, policy_version 1011087 (0.0009) [2023-12-26 22:44:09,949][105620] Updated weights for policy 1, policy_version 1011507 (0.0009) [2023-12-26 22:44:09,968][105692] Updated weights for policy 0, policy_version 1011097 (0.0008) [2023-12-26 22:44:10,006][105620] Updated weights for policy 1, policy_version 1011517 (0.0006) [2023-12-26 22:44:10,033][105692] Updated weights for policy 0, policy_version 1011107 (0.0009) [2023-12-26 22:44:10,069][105620] Updated weights for policy 1, policy_version 1011527 (0.0006) [2023-12-26 22:44:10,711][105692] Updated weights for policy 0, policy_version 1011117 (0.0010) [2023-12-26 22:44:10,772][105692] Updated weights for policy 0, policy_version 1011127 (0.0007) [2023-12-26 22:44:10,827][105692] Updated weights for policy 0, policy_version 1011137 (0.0008) [2023-12-26 22:44:10,834][105620] Updated weights for policy 1, policy_version 1011537 (0.0006) [2023-12-26 22:44:10,888][105620] Updated weights for policy 1, policy_version 1011547 (0.0007) [2023-12-26 22:44:10,938][105620] Updated weights for policy 1, policy_version 1011557 (0.0008) [2023-12-26 22:44:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 517881856. Throughput: 0: 9844.3, 1: 9614.5. Samples: 517886008. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:44:11,063][104569] Avg episode reward: [(0, '8907.344'), (1, '8898.644')] [2023-12-26 22:44:11,604][105692] Updated weights for policy 0, policy_version 1011147 (0.0006) [2023-12-26 22:44:11,627][105620] Updated weights for policy 1, policy_version 1011567 (0.0007) [2023-12-26 22:44:11,673][105692] Updated weights for policy 0, policy_version 1011157 (0.0009) [2023-12-26 22:44:11,692][105620] Updated weights for policy 1, policy_version 1011577 (0.0008) [2023-12-26 22:44:11,732][105692] Updated weights for policy 0, policy_version 1011167 (0.0008) [2023-12-26 22:44:11,764][105620] Updated weights for policy 1, policy_version 1011587 (0.0008) [2023-12-26 22:44:12,454][105692] Updated weights for policy 0, policy_version 1011177 (0.0009) [2023-12-26 22:44:12,501][105692] Updated weights for policy 0, policy_version 1011187 (0.0005) [2023-12-26 22:44:12,560][105692] Updated weights for policy 0, policy_version 1011197 (0.0007) [2023-12-26 22:44:12,587][105620] Updated weights for policy 1, policy_version 1011597 (0.0007) [2023-12-26 22:44:12,627][105692] Updated weights for policy 0, policy_version 1011207 (0.0008) [2023-12-26 22:44:12,654][105620] Updated weights for policy 1, policy_version 1011607 (0.0010) [2023-12-26 22:44:12,718][105620] Updated weights for policy 1, policy_version 1011617 (0.0009) [2023-12-26 22:44:13,333][105692] Updated weights for policy 0, policy_version 1011217 (0.0009) [2023-12-26 22:44:13,398][105692] Updated weights for policy 0, policy_version 1011227 (0.0009) [2023-12-26 22:44:13,453][105620] Updated weights for policy 1, policy_version 1011627 (0.0008) [2023-12-26 22:44:13,460][105692] Updated weights for policy 0, policy_version 1011237 (0.0006) [2023-12-26 22:44:13,512][105620] Updated weights for policy 1, policy_version 1011637 (0.0005) [2023-12-26 22:44:13,572][105620] Updated weights for policy 1, policy_version 1011647 (0.0009) [2023-12-26 22:44:14,269][105620] Updated weights for policy 1, policy_version 1011657 (0.0009) [2023-12-26 22:44:14,272][105692] Updated weights for policy 0, policy_version 1011247 (0.0008) [2023-12-26 22:44:14,324][105692] Updated weights for policy 0, policy_version 1011257 (0.0008) [2023-12-26 22:44:14,335][105620] Updated weights for policy 1, policy_version 1011667 (0.0008) [2023-12-26 22:44:14,387][105692] Updated weights for policy 0, policy_version 1011267 (0.0009) [2023-12-26 22:44:14,397][105620] Updated weights for policy 1, policy_version 1011677 (0.0008) [2023-12-26 22:44:14,456][105620] Updated weights for policy 1, policy_version 1011687 (0.0009) [2023-12-26 22:44:15,120][105692] Updated weights for policy 0, policy_version 1011277 (0.0007) [2023-12-26 22:44:15,182][105692] Updated weights for policy 0, policy_version 1011287 (0.0009) [2023-12-26 22:44:15,218][105620] Updated weights for policy 1, policy_version 1011697 (0.0007) [2023-12-26 22:44:15,244][105692] Updated weights for policy 0, policy_version 1011297 (0.0008) [2023-12-26 22:44:15,268][105620] Updated weights for policy 1, policy_version 1011707 (0.0006) [2023-12-26 22:44:15,319][105620] Updated weights for policy 1, policy_version 1011717 (0.0007) [2023-12-26 22:44:16,032][105692] Updated weights for policy 0, policy_version 1011307 (0.0009) [2023-12-26 22:44:16,043][105620] Updated weights for policy 1, policy_version 1011727 (0.0008) [2023-12-26 22:44:16,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 517963776. Throughput: 0: 9771.5, 1: 9634.9. Samples: 517942400. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:44:16,062][104569] Avg episode reward: [(0, '9082.885'), (1, '9081.289')] [2023-12-26 22:44:16,082][105692] Updated weights for policy 0, policy_version 1011317 (0.0007) [2023-12-26 22:44:16,096][105620] Updated weights for policy 1, policy_version 1011737 (0.0007) [2023-12-26 22:44:16,136][105692] Updated weights for policy 0, policy_version 1011327 (0.0007) [2023-12-26 22:44:16,145][105620] Updated weights for policy 1, policy_version 1011747 (0.0007) [2023-12-26 22:44:16,167][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001011752_259039232.pth... [2023-12-26 22:44:16,171][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001010600_258744320.pth [2023-12-26 22:44:16,188][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001011336_258940928.pth... [2023-12-26 22:44:16,192][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001010184_258646016.pth [2023-12-26 22:44:16,780][105620] Updated weights for policy 1, policy_version 1011757 (0.0005) [2023-12-26 22:44:16,806][105692] Updated weights for policy 0, policy_version 1011337 (0.0008) [2023-12-26 22:44:16,833][105620] Updated weights for policy 1, policy_version 1011767 (0.0005) [2023-12-26 22:44:16,867][105692] Updated weights for policy 0, policy_version 1011347 (0.0005) [2023-12-26 22:44:16,878][105620] Updated weights for policy 1, policy_version 1011777 (0.0005) [2023-12-26 22:44:16,917][105692] Updated weights for policy 0, policy_version 1011357 (0.0006) [2023-12-26 22:44:16,983][105692] Updated weights for policy 0, policy_version 1011367 (0.0005) [2023-12-26 22:44:17,577][105620] Updated weights for policy 1, policy_version 1011787 (0.0007) [2023-12-26 22:44:17,639][105620] Updated weights for policy 1, policy_version 1011797 (0.0009) [2023-12-26 22:44:17,656][105692] Updated weights for policy 0, policy_version 1011377 (0.0005) [2023-12-26 22:44:17,694][105620] Updated weights for policy 1, policy_version 1011807 (0.0008) [2023-12-26 22:44:17,715][105692] Updated weights for policy 0, policy_version 1011387 (0.0008) [2023-12-26 22:44:17,769][105692] Updated weights for policy 0, policy_version 1011397 (0.0007) [2023-12-26 22:44:18,486][105692] Updated weights for policy 0, policy_version 1011407 (0.0010) [2023-12-26 22:44:18,489][105620] Updated weights for policy 1, policy_version 1011817 (0.0008) [2023-12-26 22:44:18,540][105692] Updated weights for policy 0, policy_version 1011417 (0.0010) [2023-12-26 22:44:18,543][105620] Updated weights for policy 1, policy_version 1011827 (0.0007) [2023-12-26 22:44:18,597][105620] Updated weights for policy 1, policy_version 1011837 (0.0006) [2023-12-26 22:44:18,598][105692] Updated weights for policy 0, policy_version 1011427 (0.0007) [2023-12-26 22:44:18,650][105620] Updated weights for policy 1, policy_version 1011847 (0.0009) [2023-12-26 22:44:19,285][105692] Updated weights for policy 0, policy_version 1011437 (0.0007) [2023-12-26 22:44:19,370][105692] Updated weights for policy 0, policy_version 1011447 (0.0008) [2023-12-26 22:44:19,436][105692] Updated weights for policy 0, policy_version 1011457 (0.0007) [2023-12-26 22:44:19,438][105620] Updated weights for policy 1, policy_version 1011857 (0.0007) [2023-12-26 22:44:19,500][105620] Updated weights for policy 1, policy_version 1011867 (0.0009) [2023-12-26 22:44:19,559][105620] Updated weights for policy 1, policy_version 1011877 (0.0010) [2023-12-26 22:44:20,140][105692] Updated weights for policy 0, policy_version 1011467 (0.0006) [2023-12-26 22:44:20,208][105692] Updated weights for policy 0, policy_version 1011477 (0.0008) [2023-12-26 22:44:20,271][105692] Updated weights for policy 0, policy_version 1011487 (0.0009) [2023-12-26 22:44:20,340][105620] Updated weights for policy 1, policy_version 1011887 (0.0009) [2023-12-26 22:44:20,395][105620] Updated weights for policy 1, policy_version 1011897 (0.0008) [2023-12-26 22:44:20,447][105620] Updated weights for policy 1, policy_version 1011907 (0.0010) [2023-12-26 22:44:20,994][105692] Updated weights for policy 0, policy_version 1011497 (0.0008) [2023-12-26 22:44:21,062][105692] Updated weights for policy 0, policy_version 1011507 (0.0009) [2023-12-26 22:44:21,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 518062080. Throughput: 0: 9658.5, 1: 9688.9. Samples: 518057136. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:44:21,063][104569] Avg episode reward: [(0, '8993.254'), (1, '9262.355')] [2023-12-26 22:44:21,126][105692] Updated weights for policy 0, policy_version 1011517 (0.0010) [2023-12-26 22:44:21,183][105692] Updated weights for policy 0, policy_version 1011527 (0.0009) [2023-12-26 22:44:21,238][105620] Updated weights for policy 1, policy_version 1011917 (0.0009) [2023-12-26 22:44:21,300][105620] Updated weights for policy 1, policy_version 1011927 (0.0008) [2023-12-26 22:44:21,366][105620] Updated weights for policy 1, policy_version 1011937 (0.0010) [2023-12-26 22:44:21,960][105692] Updated weights for policy 0, policy_version 1011537 (0.0009) [2023-12-26 22:44:22,016][105692] Updated weights for policy 0, policy_version 1011547 (0.0009) [2023-12-26 22:44:22,072][105692] Updated weights for policy 0, policy_version 1011557 (0.0009) [2023-12-26 22:44:22,123][105620] Updated weights for policy 1, policy_version 1011947 (0.0009) [2023-12-26 22:44:22,174][105620] Updated weights for policy 1, policy_version 1011957 (0.0008) [2023-12-26 22:44:22,240][105620] Updated weights for policy 1, policy_version 1011967 (0.0009) [2023-12-26 22:44:22,868][105692] Updated weights for policy 0, policy_version 1011567 (0.0009) [2023-12-26 22:44:22,923][105692] Updated weights for policy 0, policy_version 1011577 (0.0009) [2023-12-26 22:44:22,971][105692] Updated weights for policy 0, policy_version 1011587 (0.0009) [2023-12-26 22:44:22,975][105620] Updated weights for policy 1, policy_version 1011977 (0.0009) [2023-12-26 22:44:23,037][105620] Updated weights for policy 1, policy_version 1011987 (0.0007) [2023-12-26 22:44:23,084][105620] Updated weights for policy 1, policy_version 1011997 (0.0009) [2023-12-26 22:44:23,131][105620] Updated weights for policy 1, policy_version 1012007 (0.0009) [2023-12-26 22:44:23,748][105692] Updated weights for policy 0, policy_version 1011597 (0.0008) [2023-12-26 22:44:23,806][105692] Updated weights for policy 0, policy_version 1011607 (0.0009) [2023-12-26 22:44:23,856][105692] Updated weights for policy 0, policy_version 1011617 (0.0006) [2023-12-26 22:44:23,872][105620] Updated weights for policy 1, policy_version 1012017 (0.0008) [2023-12-26 22:44:23,927][105620] Updated weights for policy 1, policy_version 1012027 (0.0008) [2023-12-26 22:44:23,988][105620] Updated weights for policy 1, policy_version 1012037 (0.0009) [2023-12-26 22:44:24,608][105692] Updated weights for policy 0, policy_version 1011627 (0.0008) [2023-12-26 22:44:24,670][105692] Updated weights for policy 0, policy_version 1011637 (0.0009) [2023-12-26 22:44:24,725][105620] Updated weights for policy 1, policy_version 1012047 (0.0008) [2023-12-26 22:44:24,727][105692] Updated weights for policy 0, policy_version 1011647 (0.0008) [2023-12-26 22:44:24,779][105620] Updated weights for policy 1, policy_version 1012057 (0.0009) [2023-12-26 22:44:24,832][105620] Updated weights for policy 1, policy_version 1012067 (0.0009) [2023-12-26 22:44:25,480][105692] Updated weights for policy 0, policy_version 1011657 (0.0008) [2023-12-26 22:44:25,549][105692] Updated weights for policy 0, policy_version 1011667 (0.0009) [2023-12-26 22:44:25,566][105620] Updated weights for policy 1, policy_version 1012077 (0.0008) [2023-12-26 22:44:25,608][105692] Updated weights for policy 0, policy_version 1011677 (0.0007) [2023-12-26 22:44:25,625][105620] Updated weights for policy 1, policy_version 1012087 (0.0007) [2023-12-26 22:44:25,663][105692] Updated weights for policy 0, policy_version 1011687 (0.0007) [2023-12-26 22:44:25,686][105620] Updated weights for policy 1, policy_version 1012097 (0.0009) [2023-12-26 22:44:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 518160384. Throughput: 0: 9615.0, 1: 9649.8. Samples: 518168732. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:44:26,062][104569] Avg episode reward: [(0, '9174.248'), (1, '9265.473')] [2023-12-26 22:44:26,378][105620] Updated weights for policy 1, policy_version 1012107 (0.0009) [2023-12-26 22:44:26,418][105692] Updated weights for policy 0, policy_version 1011697 (0.0008) [2023-12-26 22:44:26,427][105620] Updated weights for policy 1, policy_version 1012117 (0.0005) [2023-12-26 22:44:26,468][105692] Updated weights for policy 0, policy_version 1011707 (0.0009) [2023-12-26 22:44:26,480][105620] Updated weights for policy 1, policy_version 1012127 (0.0006) [2023-12-26 22:44:26,522][105692] Updated weights for policy 0, policy_version 1011717 (0.0008) [2023-12-26 22:44:27,158][105620] Updated weights for policy 1, policy_version 1012137 (0.0006) [2023-12-26 22:44:27,225][105620] Updated weights for policy 1, policy_version 1012147 (0.0005) [2023-12-26 22:44:27,290][105620] Updated weights for policy 1, policy_version 1012157 (0.0005) [2023-12-26 22:44:27,312][105692] Updated weights for policy 0, policy_version 1011727 (0.0009) [2023-12-26 22:44:27,336][105620] Updated weights for policy 1, policy_version 1012167 (0.0006) [2023-12-26 22:44:27,360][105692] Updated weights for policy 0, policy_version 1011737 (0.0007) [2023-12-26 22:44:27,417][105692] Updated weights for policy 0, policy_version 1011747 (0.0009) [2023-12-26 22:44:28,023][105620] Updated weights for policy 1, policy_version 1012177 (0.0009) [2023-12-26 22:44:28,072][105620] Updated weights for policy 1, policy_version 1012187 (0.0008) [2023-12-26 22:44:28,129][105620] Updated weights for policy 1, policy_version 1012197 (0.0009) [2023-12-26 22:44:28,229][105692] Updated weights for policy 0, policy_version 1011757 (0.0009) [2023-12-26 22:44:28,284][105692] Updated weights for policy 0, policy_version 1011767 (0.0009) [2023-12-26 22:44:28,353][105692] Updated weights for policy 0, policy_version 1011777 (0.0009) [2023-12-26 22:44:28,886][105620] Updated weights for policy 1, policy_version 1012207 (0.0008) [2023-12-26 22:44:28,945][105620] Updated weights for policy 1, policy_version 1012217 (0.0009) [2023-12-26 22:44:29,003][105620] Updated weights for policy 1, policy_version 1012227 (0.0009) [2023-12-26 22:44:29,091][105692] Updated weights for policy 0, policy_version 1011787 (0.0008) [2023-12-26 22:44:29,148][105692] Updated weights for policy 0, policy_version 1011797 (0.0009) [2023-12-26 22:44:29,200][105692] Updated weights for policy 0, policy_version 1011807 (0.0009) [2023-12-26 22:44:29,757][105620] Updated weights for policy 1, policy_version 1012237 (0.0009) [2023-12-26 22:44:29,816][105620] Updated weights for policy 1, policy_version 1012247 (0.0008) [2023-12-26 22:44:29,878][105620] Updated weights for policy 1, policy_version 1012257 (0.0008) [2023-12-26 22:44:29,984][105692] Updated weights for policy 0, policy_version 1011817 (0.0008) [2023-12-26 22:44:30,042][105692] Updated weights for policy 0, policy_version 1011827 (0.0007) [2023-12-26 22:44:30,093][105692] Updated weights for policy 0, policy_version 1011837 (0.0009) [2023-12-26 22:44:30,144][105692] Updated weights for policy 0, policy_version 1011847 (0.0009) [2023-12-26 22:44:30,711][105620] Updated weights for policy 1, policy_version 1012267 (0.0008) [2023-12-26 22:44:30,769][105620] Updated weights for policy 1, policy_version 1012277 (0.0008) [2023-12-26 22:44:30,798][105692] Updated weights for policy 0, policy_version 1011857 (0.0008) [2023-12-26 22:44:30,826][105620] Updated weights for policy 1, policy_version 1012287 (0.0007) [2023-12-26 22:44:30,852][105692] Updated weights for policy 0, policy_version 1011867 (0.0009) [2023-12-26 22:44:30,912][105692] Updated weights for policy 0, policy_version 1011877 (0.0008) [2023-12-26 22:44:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 518258688. Throughput: 0: 9591.1, 1: 9710.2. Samples: 518226064. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:44:31,063][104569] Avg episode reward: [(0, '9264.091'), (1, '9173.045')] [2023-12-26 22:44:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001011880_259080192.pth... [2023-12-26 22:44:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001012296_259178496.pth... [2023-12-26 22:44:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001010760_258793472.pth [2023-12-26 22:44:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001011176_258891776.pth [2023-12-26 22:44:31,596][105692] Updated weights for policy 0, policy_version 1011887 (0.0009) [2023-12-26 22:44:31,650][105620] Updated weights for policy 1, policy_version 1012297 (0.0006) [2023-12-26 22:44:31,656][105692] Updated weights for policy 0, policy_version 1011897 (0.0007) [2023-12-26 22:44:31,712][105620] Updated weights for policy 1, policy_version 1012307 (0.0008) [2023-12-26 22:44:31,715][105692] Updated weights for policy 0, policy_version 1011907 (0.0007) [2023-12-26 22:44:31,767][105620] Updated weights for policy 1, policy_version 1012317 (0.0009) [2023-12-26 22:44:31,823][105620] Updated weights for policy 1, policy_version 1012327 (0.0009) [2023-12-26 22:44:32,410][105692] Updated weights for policy 0, policy_version 1011917 (0.0007) [2023-12-26 22:44:32,465][105692] Updated weights for policy 0, policy_version 1011927 (0.0009) [2023-12-26 22:44:32,517][105692] Updated weights for policy 0, policy_version 1011937 (0.0008) [2023-12-26 22:44:32,595][105620] Updated weights for policy 1, policy_version 1012337 (0.0009) [2023-12-26 22:44:32,655][105620] Updated weights for policy 1, policy_version 1012347 (0.0008) [2023-12-26 22:44:32,716][105620] Updated weights for policy 1, policy_version 1012357 (0.0009) [2023-12-26 22:44:33,191][105692] Updated weights for policy 0, policy_version 1011947 (0.0008) [2023-12-26 22:44:33,258][105692] Updated weights for policy 0, policy_version 1011957 (0.0009) [2023-12-26 22:44:33,319][105692] Updated weights for policy 0, policy_version 1011967 (0.0009) [2023-12-26 22:44:33,387][105620] Updated weights for policy 1, policy_version 1012367 (0.0007) [2023-12-26 22:44:33,433][105620] Updated weights for policy 1, policy_version 1012377 (0.0006) [2023-12-26 22:44:33,476][105620] Updated weights for policy 1, policy_version 1012387 (0.0005) [2023-12-26 22:44:34,038][105692] Updated weights for policy 0, policy_version 1011977 (0.0010) [2023-12-26 22:44:34,095][105620] Updated weights for policy 1, policy_version 1012397 (0.0005) [2023-12-26 22:44:34,096][105692] Updated weights for policy 0, policy_version 1011987 (0.0008) [2023-12-26 22:44:34,160][105692] Updated weights for policy 0, policy_version 1011997 (0.0008) [2023-12-26 22:44:34,161][105620] Updated weights for policy 1, policy_version 1012407 (0.0007) [2023-12-26 22:44:34,215][105692] Updated weights for policy 0, policy_version 1012007 (0.0008) [2023-12-26 22:44:34,228][105620] Updated weights for policy 1, policy_version 1012417 (0.0006) [2023-12-26 22:44:34,924][105692] Updated weights for policy 0, policy_version 1012017 (0.0006) [2023-12-26 22:44:34,964][105620] Updated weights for policy 1, policy_version 1012427 (0.0007) [2023-12-26 22:44:34,983][105692] Updated weights for policy 0, policy_version 1012027 (0.0006) [2023-12-26 22:44:35,027][105620] Updated weights for policy 1, policy_version 1012437 (0.0008) [2023-12-26 22:44:35,042][105692] Updated weights for policy 0, policy_version 1012037 (0.0008) [2023-12-26 22:44:35,090][105620] Updated weights for policy 1, policy_version 1012447 (0.0005) [2023-12-26 22:44:35,745][105692] Updated weights for policy 0, policy_version 1012047 (0.0008) [2023-12-26 22:44:35,762][105620] Updated weights for policy 1, policy_version 1012457 (0.0005) [2023-12-26 22:44:35,809][105692] Updated weights for policy 0, policy_version 1012057 (0.0008) [2023-12-26 22:44:35,819][105620] Updated weights for policy 1, policy_version 1012467 (0.0005) [2023-12-26 22:44:35,874][105692] Updated weights for policy 0, policy_version 1012067 (0.0009) [2023-12-26 22:44:35,880][105620] Updated weights for policy 1, policy_version 1012477 (0.0005) [2023-12-26 22:44:35,940][105620] Updated weights for policy 1, policy_version 1012487 (0.0008) [2023-12-26 22:44:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 518356992. Throughput: 0: 9619.6, 1: 9699.4. Samples: 518341484. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:44:36,062][104569] Avg episode reward: [(0, '8994.303'), (1, '9081.171')] [2023-12-26 22:44:36,602][105620] Updated weights for policy 1, policy_version 1012497 (0.0008) [2023-12-26 22:44:36,612][105692] Updated weights for policy 0, policy_version 1012077 (0.0007) [2023-12-26 22:44:36,671][105692] Updated weights for policy 0, policy_version 1012087 (0.0007) [2023-12-26 22:44:36,673][105620] Updated weights for policy 1, policy_version 1012507 (0.0006) [2023-12-26 22:44:36,726][105692] Updated weights for policy 0, policy_version 1012097 (0.0007) [2023-12-26 22:44:36,737][105620] Updated weights for policy 1, policy_version 1012517 (0.0006) [2023-12-26 22:44:37,345][105620] Updated weights for policy 1, policy_version 1012527 (0.0006) [2023-12-26 22:44:37,407][105620] Updated weights for policy 1, policy_version 1012537 (0.0005) [2023-12-26 22:44:37,473][105692] Updated weights for policy 0, policy_version 1012107 (0.0007) [2023-12-26 22:44:37,476][105620] Updated weights for policy 1, policy_version 1012547 (0.0006) [2023-12-26 22:44:37,537][105692] Updated weights for policy 0, policy_version 1012117 (0.0008) [2023-12-26 22:44:37,601][105692] Updated weights for policy 0, policy_version 1012127 (0.0009) [2023-12-26 22:44:38,103][105620] Updated weights for policy 1, policy_version 1012557 (0.0006) [2023-12-26 22:44:38,171][105620] Updated weights for policy 1, policy_version 1012567 (0.0006) [2023-12-26 22:44:38,222][105620] Updated weights for policy 1, policy_version 1012577 (0.0010) [2023-12-26 22:44:38,295][105692] Updated weights for policy 0, policy_version 1012137 (0.0010) [2023-12-26 22:44:38,359][105692] Updated weights for policy 0, policy_version 1012147 (0.0008) [2023-12-26 22:44:38,420][105692] Updated weights for policy 0, policy_version 1012157 (0.0008) [2023-12-26 22:44:38,483][105692] Updated weights for policy 0, policy_version 1012167 (0.0008) [2023-12-26 22:44:38,823][105620] Updated weights for policy 1, policy_version 1012587 (0.0009) [2023-12-26 22:44:38,884][105620] Updated weights for policy 1, policy_version 1012597 (0.0009) [2023-12-26 22:44:38,943][105620] Updated weights for policy 1, policy_version 1012607 (0.0009) [2023-12-26 22:44:39,273][105692] Updated weights for policy 0, policy_version 1012177 (0.0007) [2023-12-26 22:44:39,334][105692] Updated weights for policy 0, policy_version 1012187 (0.0008) [2023-12-26 22:44:39,408][105692] Updated weights for policy 0, policy_version 1012197 (0.0009) [2023-12-26 22:44:39,700][105620] Updated weights for policy 1, policy_version 1012617 (0.0010) [2023-12-26 22:44:39,763][105620] Updated weights for policy 1, policy_version 1012627 (0.0011) [2023-12-26 22:44:39,830][105620] Updated weights for policy 1, policy_version 1012637 (0.0011) [2023-12-26 22:44:39,891][105620] Updated weights for policy 1, policy_version 1012647 (0.0011) [2023-12-26 22:44:40,136][105692] Updated weights for policy 0, policy_version 1012207 (0.0007) [2023-12-26 22:44:40,208][105692] Updated weights for policy 0, policy_version 1012217 (0.0007) [2023-12-26 22:44:40,281][105692] Updated weights for policy 0, policy_version 1012227 (0.0006) [2023-12-26 22:44:40,605][105620] Updated weights for policy 1, policy_version 1012657 (0.0011) [2023-12-26 22:44:40,667][105620] Updated weights for policy 1, policy_version 1012667 (0.0010) [2023-12-26 22:44:40,722][105620] Updated weights for policy 1, policy_version 1012677 (0.0010) [2023-12-26 22:44:40,897][105692] Updated weights for policy 0, policy_version 1012237 (0.0006) [2023-12-26 22:44:40,952][105692] Updated weights for policy 0, policy_version 1012247 (0.0008) [2023-12-26 22:44:41,009][105692] Updated weights for policy 0, policy_version 1012257 (0.0010) [2023-12-26 22:44:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 518455296. Throughput: 0: 9604.5, 1: 9677.0. Samples: 518460280. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:44:41,063][104569] Avg episode reward: [(0, '8722.797'), (1, '9079.968')] [2023-12-26 22:44:41,430][105620] Updated weights for policy 1, policy_version 1012687 (0.0011) [2023-12-26 22:44:41,486][105620] Updated weights for policy 1, policy_version 1012697 (0.0009) [2023-12-26 22:44:41,548][105620] Updated weights for policy 1, policy_version 1012707 (0.0007) [2023-12-26 22:44:41,864][105692] Updated weights for policy 0, policy_version 1012267 (0.0008) [2023-12-26 22:44:41,924][105692] Updated weights for policy 0, policy_version 1012277 (0.0008) [2023-12-26 22:44:41,979][105692] Updated weights for policy 0, policy_version 1012287 (0.0010) [2023-12-26 22:44:42,191][105620] Updated weights for policy 1, policy_version 1012717 (0.0010) [2023-12-26 22:44:42,250][105620] Updated weights for policy 1, policy_version 1012727 (0.0009) [2023-12-26 22:44:42,314][105620] Updated weights for policy 1, policy_version 1012737 (0.0007) [2023-12-26 22:44:42,829][105692] Updated weights for policy 0, policy_version 1012297 (0.0010) [2023-12-26 22:44:42,894][105692] Updated weights for policy 0, policy_version 1012307 (0.0008) [2023-12-26 22:44:42,959][105692] Updated weights for policy 0, policy_version 1012317 (0.0009) [2023-12-26 22:44:42,970][105620] Updated weights for policy 1, policy_version 1012747 (0.0009) [2023-12-26 22:44:43,017][105692] Updated weights for policy 0, policy_version 1012327 (0.0009) [2023-12-26 22:44:43,035][105620] Updated weights for policy 1, policy_version 1012757 (0.0008) [2023-12-26 22:44:43,093][105620] Updated weights for policy 1, policy_version 1012767 (0.0006) [2023-12-26 22:44:43,729][105620] Updated weights for policy 1, policy_version 1012777 (0.0006) [2023-12-26 22:44:43,785][105620] Updated weights for policy 1, policy_version 1012787 (0.0010) [2023-12-26 22:44:43,798][105692] Updated weights for policy 0, policy_version 1012337 (0.0010) [2023-12-26 22:44:43,840][105620] Updated weights for policy 1, policy_version 1012797 (0.0010) [2023-12-26 22:44:43,847][105692] Updated weights for policy 0, policy_version 1012347 (0.0010) [2023-12-26 22:44:43,888][105620] Updated weights for policy 1, policy_version 1012807 (0.0006) [2023-12-26 22:44:43,892][105692] Updated weights for policy 0, policy_version 1012357 (0.0010) [2023-12-26 22:44:44,492][105620] Updated weights for policy 1, policy_version 1012817 (0.0010) [2023-12-26 22:44:44,548][105620] Updated weights for policy 1, policy_version 1012827 (0.0010) [2023-12-26 22:44:44,606][105692] Updated weights for policy 0, policy_version 1012367 (0.0009) [2023-12-26 22:44:44,614][105620] Updated weights for policy 1, policy_version 1012837 (0.0011) [2023-12-26 22:44:44,664][105692] Updated weights for policy 0, policy_version 1012377 (0.0007) [2023-12-26 22:44:44,727][105692] Updated weights for policy 0, policy_version 1012387 (0.0007) [2023-12-26 22:44:45,371][105620] Updated weights for policy 1, policy_version 1012847 (0.0009) [2023-12-26 22:44:45,428][105620] Updated weights for policy 1, policy_version 1012857 (0.0009) [2023-12-26 22:44:45,435][105692] Updated weights for policy 0, policy_version 1012397 (0.0010) [2023-12-26 22:44:45,479][105620] Updated weights for policy 1, policy_version 1012867 (0.0007) [2023-12-26 22:44:45,485][105692] Updated weights for policy 0, policy_version 1012407 (0.0007) [2023-12-26 22:44:45,543][105692] Updated weights for policy 0, policy_version 1012417 (0.0008) [2023-12-26 22:44:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 518545408. Throughput: 0: 9509.7, 1: 9695.0. Samples: 518516688. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:44:46,063][104569] Avg episode reward: [(0, '8811.919'), (1, '9077.454')] [2023-12-26 22:44:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001012424_259219456.pth... [2023-12-26 22:44:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001012872_259325952.pth... [2023-12-26 22:44:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001011336_258940928.pth [2023-12-26 22:44:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001011752_259039232.pth [2023-12-26 22:44:46,225][105620] Updated weights for policy 1, policy_version 1012877 (0.0006) [2023-12-26 22:44:46,283][105620] Updated weights for policy 1, policy_version 1012887 (0.0005) [2023-12-26 22:44:46,337][105620] Updated weights for policy 1, policy_version 1012897 (0.0007) [2023-12-26 22:44:46,348][105692] Updated weights for policy 0, policy_version 1012427 (0.0008) [2023-12-26 22:44:46,411][105692] Updated weights for policy 0, policy_version 1012437 (0.0009) [2023-12-26 22:44:46,477][105692] Updated weights for policy 0, policy_version 1012447 (0.0010) [2023-12-26 22:44:46,888][105620] Updated weights for policy 1, policy_version 1012907 (0.0006) [2023-12-26 22:44:46,940][105620] Updated weights for policy 1, policy_version 1012917 (0.0005) [2023-12-26 22:44:46,996][105620] Updated weights for policy 1, policy_version 1012927 (0.0005) [2023-12-26 22:44:47,177][105692] Updated weights for policy 0, policy_version 1012457 (0.0008) [2023-12-26 22:44:47,231][105692] Updated weights for policy 0, policy_version 1012467 (0.0010) [2023-12-26 22:44:47,283][105692] Updated weights for policy 0, policy_version 1012479 (0.0010) [2023-12-26 22:44:47,525][105620] Updated weights for policy 1, policy_version 1012937 (0.0005) [2023-12-26 22:44:47,571][105620] Updated weights for policy 1, policy_version 1012947 (0.0005) [2023-12-26 22:44:47,624][105620] Updated weights for policy 1, policy_version 1012957 (0.0005) [2023-12-26 22:44:47,683][105620] Updated weights for policy 1, policy_version 1012967 (0.0005) [2023-12-26 22:44:48,149][105692] Updated weights for policy 0, policy_version 1012489 (0.0010) [2023-12-26 22:44:48,205][105692] Updated weights for policy 0, policy_version 1012499 (0.0005) [2023-12-26 22:44:48,239][105620] Updated weights for policy 1, policy_version 1012977 (0.0007) [2023-12-26 22:44:48,265][105692] Updated weights for policy 0, policy_version 1012509 (0.0006) [2023-12-26 22:44:48,304][105620] Updated weights for policy 1, policy_version 1012987 (0.0007) [2023-12-26 22:44:48,324][105692] Updated weights for policy 0, policy_version 1012519 (0.0006) [2023-12-26 22:44:48,372][105620] Updated weights for policy 1, policy_version 1012997 (0.0006) [2023-12-26 22:44:48,923][105692] Updated weights for policy 0, policy_version 1012529 (0.0008) [2023-12-26 22:44:48,992][105692] Updated weights for policy 0, policy_version 1012539 (0.0009) [2023-12-26 22:44:49,050][105692] Updated weights for policy 0, policy_version 1012549 (0.0009) [2023-12-26 22:44:49,143][105620] Updated weights for policy 1, policy_version 1013007 (0.0009) [2023-12-26 22:44:49,196][105620] Updated weights for policy 1, policy_version 1013017 (0.0009) [2023-12-26 22:44:49,259][105620] Updated weights for policy 1, policy_version 1013027 (0.0009) [2023-12-26 22:44:49,658][105692] Updated weights for policy 0, policy_version 1012559 (0.0009) [2023-12-26 22:44:49,718][105692] Updated weights for policy 0, policy_version 1012569 (0.0009) [2023-12-26 22:44:49,777][105692] Updated weights for policy 0, policy_version 1012579 (0.0009) [2023-12-26 22:44:50,091][105620] Updated weights for policy 1, policy_version 1013037 (0.0009) [2023-12-26 22:44:50,160][105620] Updated weights for policy 1, policy_version 1013047 (0.0009) [2023-12-26 22:44:50,223][105620] Updated weights for policy 1, policy_version 1013057 (0.0009) [2023-12-26 22:44:50,488][105692] Updated weights for policy 0, policy_version 1012589 (0.0009) [2023-12-26 22:44:50,552][105692] Updated weights for policy 0, policy_version 1012599 (0.0009) [2023-12-26 22:44:50,615][105692] Updated weights for policy 0, policy_version 1012609 (0.0008) [2023-12-26 22:44:51,053][105620] Updated weights for policy 1, policy_version 1013067 (0.0008) [2023-12-26 22:44:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 518643712. Throughput: 0: 9526.9, 1: 9758.5. Samples: 518637712. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:44:51,063][104569] Avg episode reward: [(0, '8992.313'), (1, '9169.963')] [2023-12-26 22:44:51,117][105620] Updated weights for policy 1, policy_version 1013077 (0.0010) [2023-12-26 22:44:51,188][105620] Updated weights for policy 1, policy_version 1013087 (0.0008) [2023-12-26 22:44:51,302][105692] Updated weights for policy 0, policy_version 1012619 (0.0007) [2023-12-26 22:44:51,362][105692] Updated weights for policy 0, policy_version 1012629 (0.0009) [2023-12-26 22:44:51,423][105692] Updated weights for policy 0, policy_version 1012639 (0.0009) [2023-12-26 22:44:51,896][105620] Updated weights for policy 1, policy_version 1013097 (0.0008) [2023-12-26 22:44:51,947][105620] Updated weights for policy 1, policy_version 1013107 (0.0008) [2023-12-26 22:44:51,994][105620] Updated weights for policy 1, policy_version 1013117 (0.0009) [2023-12-26 22:44:52,042][105620] Updated weights for policy 1, policy_version 1013127 (0.0009) [2023-12-26 22:44:52,131][105692] Updated weights for policy 0, policy_version 1012649 (0.0007) [2023-12-26 22:44:52,183][105692] Updated weights for policy 0, policy_version 1012659 (0.0009) [2023-12-26 22:44:52,232][105692] Updated weights for policy 0, policy_version 1012670 (0.0009) [2023-12-26 22:44:52,287][105692] Updated weights for policy 0, policy_version 1012680 (0.0008) [2023-12-26 22:44:52,796][105620] Updated weights for policy 1, policy_version 1013137 (0.0009) [2023-12-26 22:44:52,850][105620] Updated weights for policy 1, policy_version 1013147 (0.0009) [2023-12-26 22:44:52,901][105620] Updated weights for policy 1, policy_version 1013157 (0.0008) [2023-12-26 22:44:53,082][105692] Updated weights for policy 0, policy_version 1012690 (0.0009) [2023-12-26 22:44:53,133][105692] Updated weights for policy 0, policy_version 1012700 (0.0009) [2023-12-26 22:44:53,183][105692] Updated weights for policy 0, policy_version 1012710 (0.0008) [2023-12-26 22:44:53,673][105620] Updated weights for policy 1, policy_version 1013167 (0.0010) [2023-12-26 22:44:53,728][105620] Updated weights for policy 1, policy_version 1013177 (0.0010) [2023-12-26 22:44:53,786][105620] Updated weights for policy 1, policy_version 1013187 (0.0010) [2023-12-26 22:44:53,884][105692] Updated weights for policy 0, policy_version 1012720 (0.0009) [2023-12-26 22:44:53,931][105692] Updated weights for policy 0, policy_version 1012730 (0.0008) [2023-12-26 22:44:53,978][105692] Updated weights for policy 0, policy_version 1012740 (0.0009) [2023-12-26 22:44:54,592][105620] Updated weights for policy 1, policy_version 1013198 (0.0010) [2023-12-26 22:44:54,648][105620] Updated weights for policy 1, policy_version 1013208 (0.0009) [2023-12-26 22:44:54,698][105692] Updated weights for policy 0, policy_version 1012750 (0.0007) [2023-12-26 22:44:54,701][105620] Updated weights for policy 1, policy_version 1013218 (0.0009) [2023-12-26 22:44:54,747][105692] Updated weights for policy 0, policy_version 1012760 (0.0005) [2023-12-26 22:44:54,809][105692] Updated weights for policy 0, policy_version 1012770 (0.0005) [2023-12-26 22:44:55,337][105692] Updated weights for policy 0, policy_version 1012780 (0.0005) [2023-12-26 22:44:55,385][105692] Updated weights for policy 0, policy_version 1012790 (0.0005) [2023-12-26 22:44:55,441][105692] Updated weights for policy 0, policy_version 1012800 (0.0005) [2023-12-26 22:44:55,500][105620] Updated weights for policy 1, policy_version 1013228 (0.0009) [2023-12-26 22:44:55,559][105620] Updated weights for policy 1, policy_version 1013238 (0.0010) [2023-12-26 22:44:55,619][105620] Updated weights for policy 1, policy_version 1013248 (0.0011) [2023-12-26 22:44:56,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19251.1, 300 sec: 19466.4). Total num frames: 518742016. Throughput: 0: 9546.3, 1: 9716.6. Samples: 518752848. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:44:56,064][104569] Avg episode reward: [(0, '9086.218'), (1, '9262.510')] [2023-12-26 22:44:56,074][105692] Updated weights for policy 0, policy_version 1012810 (0.0009) [2023-12-26 22:44:56,129][105692] Updated weights for policy 0, policy_version 1012820 (0.0010) [2023-12-26 22:44:56,187][105692] Updated weights for policy 0, policy_version 1012830 (0.0010) [2023-12-26 22:44:56,242][105692] Updated weights for policy 0, policy_version 1012840 (0.0010) [2023-12-26 22:44:56,300][105620] Updated weights for policy 1, policy_version 1013258 (0.0010) [2023-12-26 22:44:56,355][105620] Updated weights for policy 1, policy_version 1013268 (0.0007) [2023-12-26 22:44:56,406][105620] Updated weights for policy 1, policy_version 1013278 (0.0010) [2023-12-26 22:44:56,458][105620] Updated weights for policy 1, policy_version 1013288 (0.0011) [2023-12-26 22:44:56,971][105692] Updated weights for policy 0, policy_version 1012850 (0.0010) [2023-12-26 22:44:57,028][105692] Updated weights for policy 0, policy_version 1012860 (0.0010) [2023-12-26 22:44:57,076][105620] Updated weights for policy 1, policy_version 1013298 (0.0005) [2023-12-26 22:44:57,089][105692] Updated weights for policy 0, policy_version 1012870 (0.0010) [2023-12-26 22:44:57,124][105620] Updated weights for policy 1, policy_version 1013308 (0.0005) [2023-12-26 22:44:57,179][105620] Updated weights for policy 1, policy_version 1013318 (0.0005) [2023-12-26 22:44:57,751][105620] Updated weights for policy 1, policy_version 1013328 (0.0009) [2023-12-26 22:44:57,800][105620] Updated weights for policy 1, policy_version 1013338 (0.0006) [2023-12-26 22:44:57,805][105692] Updated weights for policy 0, policy_version 1012880 (0.0010) [2023-12-26 22:44:57,849][105692] Updated weights for policy 0, policy_version 1012890 (0.0010) [2023-12-26 22:44:57,852][105620] Updated weights for policy 1, policy_version 1013348 (0.0005) [2023-12-26 22:44:57,896][105692] Updated weights for policy 0, policy_version 1012900 (0.0010) [2023-12-26 22:44:58,511][105620] Updated weights for policy 1, policy_version 1013358 (0.0007) [2023-12-26 22:44:58,584][105620] Updated weights for policy 1, policy_version 1013368 (0.0007) [2023-12-26 22:44:58,649][105620] Updated weights for policy 1, policy_version 1013378 (0.0008) [2023-12-26 22:44:58,699][105692] Updated weights for policy 0, policy_version 1012910 (0.0009) [2023-12-26 22:44:58,767][105692] Updated weights for policy 0, policy_version 1012920 (0.0008) [2023-12-26 22:44:58,837][105692] Updated weights for policy 0, policy_version 1012930 (0.0009) [2023-12-26 22:44:59,433][105620] Updated weights for policy 1, policy_version 1013388 (0.0007) [2023-12-26 22:44:59,488][105620] Updated weights for policy 1, policy_version 1013398 (0.0005) [2023-12-26 22:44:59,551][105620] Updated weights for policy 1, policy_version 1013408 (0.0007) [2023-12-26 22:44:59,588][105692] Updated weights for policy 0, policy_version 1012940 (0.0010) [2023-12-26 22:44:59,640][105692] Updated weights for policy 0, policy_version 1012950 (0.0009) [2023-12-26 22:44:59,690][105692] Updated weights for policy 0, policy_version 1012960 (0.0006) [2023-12-26 22:45:00,207][105620] Updated weights for policy 1, policy_version 1013418 (0.0006) [2023-12-26 22:45:00,269][105620] Updated weights for policy 1, policy_version 1013428 (0.0006) [2023-12-26 22:45:00,330][105620] Updated weights for policy 1, policy_version 1013438 (0.0008) [2023-12-26 22:45:00,387][105620] Updated weights for policy 1, policy_version 1013448 (0.0005) [2023-12-26 22:45:00,465][105692] Updated weights for policy 0, policy_version 1012970 (0.0008) [2023-12-26 22:45:00,516][105692] Updated weights for policy 0, policy_version 1012980 (0.0010) [2023-12-26 22:45:00,575][105692] Updated weights for policy 0, policy_version 1012991 (0.0010) [2023-12-26 22:45:00,931][105620] Updated weights for policy 1, policy_version 1013458 (0.0006) [2023-12-26 22:45:00,980][105620] Updated weights for policy 1, policy_version 1013468 (0.0005) [2023-12-26 22:45:01,037][105620] Updated weights for policy 1, policy_version 1013478 (0.0007) [2023-12-26 22:45:01,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 518848512. Throughput: 0: 9566.9, 1: 9809.2. Samples: 518814320. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:45:01,062][104569] Avg episode reward: [(0, '9175.322'), (1, '9264.938')] [2023-12-26 22:45:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001013000_259366912.pth... [2023-12-26 22:45:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001013480_259481600.pth... [2023-12-26 22:45:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001011880_259080192.pth [2023-12-26 22:45:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001012296_259178496.pth [2023-12-26 22:45:01,276][105692] Updated weights for policy 0, policy_version 1013001 (0.0008) [2023-12-26 22:45:01,330][105692] Updated weights for policy 0, policy_version 1013011 (0.0008) [2023-12-26 22:45:01,394][105692] Updated weights for policy 0, policy_version 1013021 (0.0009) [2023-12-26 22:45:01,459][105692] Updated weights for policy 0, policy_version 1013031 (0.0007) [2023-12-26 22:45:01,788][105620] Updated weights for policy 1, policy_version 1013488 (0.0008) [2023-12-26 22:45:01,845][105620] Updated weights for policy 1, policy_version 1013498 (0.0006) [2023-12-26 22:45:01,899][105620] Updated weights for policy 1, policy_version 1013508 (0.0006) [2023-12-26 22:45:02,184][105692] Updated weights for policy 0, policy_version 1013041 (0.0008) [2023-12-26 22:45:02,237][105692] Updated weights for policy 0, policy_version 1013051 (0.0010) [2023-12-26 22:45:02,296][105692] Updated weights for policy 0, policy_version 1013061 (0.0008) [2023-12-26 22:45:02,555][105620] Updated weights for policy 1, policy_version 1013518 (0.0007) [2023-12-26 22:45:02,606][105620] Updated weights for policy 1, policy_version 1013528 (0.0005) [2023-12-26 22:45:02,669][105620] Updated weights for policy 1, policy_version 1013538 (0.0008) [2023-12-26 22:45:03,074][105692] Updated weights for policy 0, policy_version 1013071 (0.0010) [2023-12-26 22:45:03,127][105692] Updated weights for policy 0, policy_version 1013081 (0.0007) [2023-12-26 22:45:03,174][105692] Updated weights for policy 0, policy_version 1013091 (0.0005) [2023-12-26 22:45:03,282][105620] Updated weights for policy 1, policy_version 1013548 (0.0008) [2023-12-26 22:45:03,345][105620] Updated weights for policy 1, policy_version 1013558 (0.0010) [2023-12-26 22:45:03,396][105620] Updated weights for policy 1, policy_version 1013569 (0.0009) [2023-12-26 22:45:03,793][105692] Updated weights for policy 0, policy_version 1013101 (0.0008) [2023-12-26 22:45:03,850][105692] Updated weights for policy 0, policy_version 1013111 (0.0006) [2023-12-26 22:45:03,908][105692] Updated weights for policy 0, policy_version 1013121 (0.0007) [2023-12-26 22:45:04,224][105620] Updated weights for policy 1, policy_version 1013579 (0.0008) [2023-12-26 22:45:04,284][105620] Updated weights for policy 1, policy_version 1013589 (0.0007) [2023-12-26 22:45:04,347][105620] Updated weights for policy 1, policy_version 1013599 (0.0006) [2023-12-26 22:45:04,696][105692] Updated weights for policy 0, policy_version 1013131 (0.0009) [2023-12-26 22:45:04,747][105692] Updated weights for policy 0, policy_version 1013141 (0.0009) [2023-12-26 22:45:04,803][105692] Updated weights for policy 0, policy_version 1013151 (0.0009) [2023-12-26 22:45:04,911][105620] Updated weights for policy 1, policy_version 1013609 (0.0006) [2023-12-26 22:45:04,968][105620] Updated weights for policy 1, policy_version 1013619 (0.0005) [2023-12-26 22:45:05,022][105620] Updated weights for policy 1, policy_version 1013629 (0.0006) [2023-12-26 22:45:05,080][105620] Updated weights for policy 1, policy_version 1013639 (0.0009) [2023-12-26 22:45:05,638][105620] Updated weights for policy 1, policy_version 1013649 (0.0007) [2023-12-26 22:45:05,681][105692] Updated weights for policy 0, policy_version 1013161 (0.0009) [2023-12-26 22:45:05,697][105620] Updated weights for policy 1, policy_version 1013659 (0.0005) [2023-12-26 22:45:05,730][105692] Updated weights for policy 0, policy_version 1013171 (0.0009) [2023-12-26 22:45:05,751][105620] Updated weights for policy 1, policy_version 1013669 (0.0005) [2023-12-26 22:45:05,779][105692] Updated weights for policy 0, policy_version 1013181 (0.0007) [2023-12-26 22:45:05,835][105692] Updated weights for policy 0, policy_version 1013191 (0.0005) [2023-12-26 22:45:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 518946816. Throughput: 0: 9563.0, 1: 9910.6. Samples: 518933448. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:45:06,063][104569] Avg episode reward: [(0, '9172.521'), (1, '9354.957')] [2023-12-26 22:45:06,338][105620] Updated weights for policy 1, policy_version 1013679 (0.0007) [2023-12-26 22:45:06,406][105620] Updated weights for policy 1, policy_version 1013689 (0.0006) [2023-12-26 22:45:06,475][105620] Updated weights for policy 1, policy_version 1013699 (0.0007) [2023-12-26 22:45:06,658][105692] Updated weights for policy 0, policy_version 1013201 (0.0010) [2023-12-26 22:45:06,726][105692] Updated weights for policy 0, policy_version 1013211 (0.0011) [2023-12-26 22:45:06,782][105692] Updated weights for policy 0, policy_version 1013221 (0.0011) [2023-12-26 22:45:07,062][105620] Updated weights for policy 1, policy_version 1013709 (0.0007) [2023-12-26 22:45:07,115][105620] Updated weights for policy 1, policy_version 1013719 (0.0007) [2023-12-26 22:45:07,166][105620] Updated weights for policy 1, policy_version 1013729 (0.0010) [2023-12-26 22:45:07,420][105692] Updated weights for policy 0, policy_version 1013231 (0.0007) [2023-12-26 22:45:07,465][105692] Updated weights for policy 0, policy_version 1013241 (0.0005) [2023-12-26 22:45:07,516][105692] Updated weights for policy 0, policy_version 1013251 (0.0005) [2023-12-26 22:45:07,839][105620] Updated weights for policy 1, policy_version 1013739 (0.0008) [2023-12-26 22:45:07,884][105620] Updated weights for policy 1, policy_version 1013749 (0.0005) [2023-12-26 22:45:07,929][105620] Updated weights for policy 1, policy_version 1013759 (0.0005) [2023-12-26 22:45:08,077][105692] Updated weights for policy 0, policy_version 1013261 (0.0008) [2023-12-26 22:45:08,131][105692] Updated weights for policy 0, policy_version 1013271 (0.0010) [2023-12-26 22:45:08,183][105692] Updated weights for policy 0, policy_version 1013281 (0.0010) [2023-12-26 22:45:08,526][105620] Updated weights for policy 1, policy_version 1013769 (0.0006) [2023-12-26 22:45:08,589][105620] Updated weights for policy 1, policy_version 1013779 (0.0011) [2023-12-26 22:45:08,647][105620] Updated weights for policy 1, policy_version 1013789 (0.0010) [2023-12-26 22:45:08,720][105620] Updated weights for policy 1, policy_version 1013799 (0.0011) [2023-12-26 22:45:08,957][105692] Updated weights for policy 0, policy_version 1013291 (0.0009) [2023-12-26 22:45:09,008][105692] Updated weights for policy 0, policy_version 1013301 (0.0010) [2023-12-26 22:45:09,062][105692] Updated weights for policy 0, policy_version 1013311 (0.0010) [2023-12-26 22:45:09,377][105620] Updated weights for policy 1, policy_version 1013809 (0.0009) [2023-12-26 22:45:09,440][105620] Updated weights for policy 1, policy_version 1013819 (0.0009) [2023-12-26 22:45:09,502][105620] Updated weights for policy 1, policy_version 1013829 (0.0008) [2023-12-26 22:45:09,859][105692] Updated weights for policy 0, policy_version 1013321 (0.0010) [2023-12-26 22:45:09,917][105692] Updated weights for policy 0, policy_version 1013331 (0.0009) [2023-12-26 22:45:09,982][105692] Updated weights for policy 0, policy_version 1013341 (0.0009) [2023-12-26 22:45:10,040][105692] Updated weights for policy 0, policy_version 1013351 (0.0009) [2023-12-26 22:45:10,170][105620] Updated weights for policy 1, policy_version 1013839 (0.0007) [2023-12-26 22:45:10,232][105620] Updated weights for policy 1, policy_version 1013849 (0.0005) [2023-12-26 22:45:10,299][105620] Updated weights for policy 1, policy_version 1013859 (0.0006) [2023-12-26 22:45:10,848][105692] Updated weights for policy 0, policy_version 1013361 (0.0007) [2023-12-26 22:45:10,907][105692] Updated weights for policy 0, policy_version 1013371 (0.0009) [2023-12-26 22:45:10,963][105692] Updated weights for policy 0, policy_version 1013381 (0.0008) [2023-12-26 22:45:10,985][105620] Updated weights for policy 1, policy_version 1013869 (0.0007) [2023-12-26 22:45:11,040][105620] Updated weights for policy 1, policy_version 1013879 (0.0009) [2023-12-26 22:45:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 519045120. Throughput: 0: 9570.4, 1: 10107.5. Samples: 519054236. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:45:11,062][104569] Avg episode reward: [(0, '8991.756'), (1, '9174.205')] [2023-12-26 22:45:11,104][105620] Updated weights for policy 1, policy_version 1013889 (0.0010) [2023-12-26 22:45:11,708][105692] Updated weights for policy 0, policy_version 1013391 (0.0009) [2023-12-26 22:45:11,776][105692] Updated weights for policy 0, policy_version 1013401 (0.0009) [2023-12-26 22:45:11,838][105692] Updated weights for policy 0, policy_version 1013411 (0.0009) [2023-12-26 22:45:11,861][105620] Updated weights for policy 1, policy_version 1013899 (0.0009) [2023-12-26 22:45:11,927][105620] Updated weights for policy 1, policy_version 1013909 (0.0008) [2023-12-26 22:45:11,991][105620] Updated weights for policy 1, policy_version 1013919 (0.0010) [2023-12-26 22:45:12,584][105692] Updated weights for policy 0, policy_version 1013421 (0.0008) [2023-12-26 22:45:12,652][105692] Updated weights for policy 0, policy_version 1013431 (0.0008) [2023-12-26 22:45:12,712][105692] Updated weights for policy 0, policy_version 1013441 (0.0008) [2023-12-26 22:45:12,733][105620] Updated weights for policy 1, policy_version 1013929 (0.0010) [2023-12-26 22:45:12,796][105620] Updated weights for policy 1, policy_version 1013939 (0.0011) [2023-12-26 22:45:12,852][105620] Updated weights for policy 1, policy_version 1013949 (0.0010) [2023-12-26 22:45:12,917][105620] Updated weights for policy 1, policy_version 1013959 (0.0011) [2023-12-26 22:45:13,409][105692] Updated weights for policy 0, policy_version 1013451 (0.0006) [2023-12-26 22:45:13,456][105692] Updated weights for policy 0, policy_version 1013461 (0.0005) [2023-12-26 22:45:13,516][105692] Updated weights for policy 0, policy_version 1013471 (0.0008) [2023-12-26 22:45:13,576][105620] Updated weights for policy 1, policy_version 1013969 (0.0009) [2023-12-26 22:45:13,632][105620] Updated weights for policy 1, policy_version 1013979 (0.0009) [2023-12-26 22:45:13,685][105620] Updated weights for policy 1, policy_version 1013989 (0.0009) [2023-12-26 22:45:14,146][105692] Updated weights for policy 0, policy_version 1013481 (0.0007) [2023-12-26 22:45:14,209][105692] Updated weights for policy 0, policy_version 1013491 (0.0007) [2023-12-26 22:45:14,271][105692] Updated weights for policy 0, policy_version 1013501 (0.0005) [2023-12-26 22:45:14,320][105692] Updated weights for policy 0, policy_version 1013511 (0.0005) [2023-12-26 22:45:14,507][105620] Updated weights for policy 1, policy_version 1013999 (0.0008) [2023-12-26 22:45:14,552][105620] Updated weights for policy 1, policy_version 1014009 (0.0008) [2023-12-26 22:45:14,605][105620] Updated weights for policy 1, policy_version 1014019 (0.0005) [2023-12-26 22:45:14,983][105692] Updated weights for policy 0, policy_version 1013521 (0.0008) [2023-12-26 22:45:15,052][105692] Updated weights for policy 0, policy_version 1013531 (0.0008) [2023-12-26 22:45:15,121][105692] Updated weights for policy 0, policy_version 1013541 (0.0008) [2023-12-26 22:45:15,363][105620] Updated weights for policy 1, policy_version 1014029 (0.0008) [2023-12-26 22:45:15,428][105620] Updated weights for policy 1, policy_version 1014039 (0.0007) [2023-12-26 22:45:15,496][105620] Updated weights for policy 1, policy_version 1014049 (0.0009) [2023-12-26 22:45:15,903][105692] Updated weights for policy 0, policy_version 1013551 (0.0009) [2023-12-26 22:45:15,966][105692] Updated weights for policy 0, policy_version 1013561 (0.0010) [2023-12-26 22:45:16,029][105692] Updated weights for policy 0, policy_version 1013571 (0.0010) [2023-12-26 22:45:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 519143424. Throughput: 0: 9584.3, 1: 10060.1. Samples: 519110064. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:45:16,063][104569] Avg episode reward: [(0, '8991.345'), (1, '9174.177')] [2023-12-26 22:45:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001014056_259629056.pth... [2023-12-26 22:45:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001013576_259514368.pth... [2023-12-26 22:45:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001012872_259325952.pth [2023-12-26 22:45:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001012424_259219456.pth [2023-12-26 22:45:16,141][105620] Updated weights for policy 1, policy_version 1014059 (0.0009) [2023-12-26 22:45:16,190][105620] Updated weights for policy 1, policy_version 1014069 (0.0006) [2023-12-26 22:45:16,244][105620] Updated weights for policy 1, policy_version 1014079 (0.0005) [2023-12-26 22:45:16,842][105692] Updated weights for policy 0, policy_version 1013581 (0.0008) [2023-12-26 22:45:16,844][105620] Updated weights for policy 1, policy_version 1014089 (0.0005) [2023-12-26 22:45:16,900][105692] Updated weights for policy 0, policy_version 1013591 (0.0008) [2023-12-26 22:45:16,905][105620] Updated weights for policy 1, policy_version 1014099 (0.0006) [2023-12-26 22:45:16,954][105692] Updated weights for policy 0, policy_version 1013601 (0.0007) [2023-12-26 22:45:16,965][105620] Updated weights for policy 1, policy_version 1014109 (0.0007) [2023-12-26 22:45:17,021][105620] Updated weights for policy 1, policy_version 1014119 (0.0008) [2023-12-26 22:45:17,576][105692] Updated weights for policy 0, policy_version 1013611 (0.0008) [2023-12-26 22:45:17,624][105692] Updated weights for policy 0, policy_version 1013621 (0.0009) [2023-12-26 22:45:17,671][105692] Updated weights for policy 0, policy_version 1013631 (0.0009) [2023-12-26 22:45:17,718][105620] Updated weights for policy 1, policy_version 1014129 (0.0008) [2023-12-26 22:45:17,781][105620] Updated weights for policy 1, policy_version 1014139 (0.0008) [2023-12-26 22:45:17,840][105620] Updated weights for policy 1, policy_version 1014149 (0.0008) [2023-12-26 22:45:18,447][105692] Updated weights for policy 0, policy_version 1013641 (0.0008) [2023-12-26 22:45:18,505][105692] Updated weights for policy 0, policy_version 1013651 (0.0010) [2023-12-26 22:45:18,567][105692] Updated weights for policy 0, policy_version 1013661 (0.0010) [2023-12-26 22:45:18,589][105620] Updated weights for policy 1, policy_version 1014159 (0.0008) [2023-12-26 22:45:18,626][105692] Updated weights for policy 0, policy_version 1013671 (0.0010) [2023-12-26 22:45:18,641][105620] Updated weights for policy 1, policy_version 1014169 (0.0006) [2023-12-26 22:45:18,698][105620] Updated weights for policy 1, policy_version 1014179 (0.0009) [2023-12-26 22:45:19,335][105692] Updated weights for policy 0, policy_version 1013681 (0.0010) [2023-12-26 22:45:19,401][105692] Updated weights for policy 0, policy_version 1013691 (0.0008) [2023-12-26 22:45:19,454][105692] Updated weights for policy 0, policy_version 1013701 (0.0008) [2023-12-26 22:45:19,536][105620] Updated weights for policy 1, policy_version 1014189 (0.0008) [2023-12-26 22:45:19,594][105620] Updated weights for policy 1, policy_version 1014199 (0.0005) [2023-12-26 22:45:19,650][105620] Updated weights for policy 1, policy_version 1014209 (0.0006) [2023-12-26 22:45:20,180][105692] Updated weights for policy 0, policy_version 1013711 (0.0010) [2023-12-26 22:45:20,239][105692] Updated weights for policy 0, policy_version 1013721 (0.0010) [2023-12-26 22:45:20,302][105692] Updated weights for policy 0, policy_version 1013731 (0.0011) [2023-12-26 22:45:20,407][105620] Updated weights for policy 1, policy_version 1014219 (0.0007) [2023-12-26 22:45:20,463][105620] Updated weights for policy 1, policy_version 1014229 (0.0008) [2023-12-26 22:45:20,529][105620] Updated weights for policy 1, policy_version 1014239 (0.0009) [2023-12-26 22:45:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 519233536. Throughput: 0: 9567.8, 1: 10096.6. Samples: 519226384. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:45:21,062][104569] Avg episode reward: [(0, '8993.836'), (1, '9081.871')] [2023-12-26 22:45:21,111][105692] Updated weights for policy 0, policy_version 1013741 (0.0011) [2023-12-26 22:45:21,172][105692] Updated weights for policy 0, policy_version 1013751 (0.0010) [2023-12-26 22:45:21,227][105692] Updated weights for policy 0, policy_version 1013761 (0.0011) [2023-12-26 22:45:21,326][105620] Updated weights for policy 1, policy_version 1014249 (0.0008) [2023-12-26 22:45:21,391][105620] Updated weights for policy 1, policy_version 1014259 (0.0010) [2023-12-26 22:45:21,448][105620] Updated weights for policy 1, policy_version 1014269 (0.0010) [2023-12-26 22:45:21,514][105620] Updated weights for policy 1, policy_version 1014279 (0.0007) [2023-12-26 22:45:22,058][105692] Updated weights for policy 0, policy_version 1013771 (0.0010) [2023-12-26 22:45:22,118][105692] Updated weights for policy 0, policy_version 1013781 (0.0011) [2023-12-26 22:45:22,181][105692] Updated weights for policy 0, policy_version 1013791 (0.0011) [2023-12-26 22:45:22,297][105620] Updated weights for policy 1, policy_version 1014289 (0.0008) [2023-12-26 22:45:22,358][105620] Updated weights for policy 1, policy_version 1014299 (0.0008) [2023-12-26 22:45:22,428][105620] Updated weights for policy 1, policy_version 1014309 (0.0009) [2023-12-26 22:45:22,852][105692] Updated weights for policy 0, policy_version 1013801 (0.0009) [2023-12-26 22:45:22,913][105692] Updated weights for policy 0, policy_version 1013811 (0.0009) [2023-12-26 22:45:22,970][105692] Updated weights for policy 0, policy_version 1013821 (0.0008) [2023-12-26 22:45:23,030][105692] Updated weights for policy 0, policy_version 1013831 (0.0008) [2023-12-26 22:45:23,220][105620] Updated weights for policy 1, policy_version 1014319 (0.0010) [2023-12-26 22:45:23,269][105620] Updated weights for policy 1, policy_version 1014329 (0.0010) [2023-12-26 22:45:23,331][105620] Updated weights for policy 1, policy_version 1014339 (0.0010) [2023-12-26 22:45:23,793][105692] Updated weights for policy 0, policy_version 1013841 (0.0009) [2023-12-26 22:45:23,854][105692] Updated weights for policy 0, policy_version 1013851 (0.0008) [2023-12-26 22:45:23,911][105692] Updated weights for policy 0, policy_version 1013861 (0.0005) [2023-12-26 22:45:23,998][105620] Updated weights for policy 1, policy_version 1014349 (0.0008) [2023-12-26 22:45:24,055][105620] Updated weights for policy 1, policy_version 1014359 (0.0005) [2023-12-26 22:45:24,102][105620] Updated weights for policy 1, policy_version 1014369 (0.0005) [2023-12-26 22:45:24,565][105692] Updated weights for policy 0, policy_version 1013871 (0.0005) [2023-12-26 22:45:24,620][105692] Updated weights for policy 0, policy_version 1013881 (0.0005) [2023-12-26 22:45:24,677][105692] Updated weights for policy 0, policy_version 1013891 (0.0005) [2023-12-26 22:45:24,827][105620] Updated weights for policy 1, policy_version 1014379 (0.0007) [2023-12-26 22:45:24,904][105620] Updated weights for policy 1, policy_version 1014389 (0.0005) [2023-12-26 22:45:24,977][105620] Updated weights for policy 1, policy_version 1014399 (0.0005) [2023-12-26 22:45:25,238][105692] Updated weights for policy 0, policy_version 1013901 (0.0005) [2023-12-26 22:45:25,289][105692] Updated weights for policy 0, policy_version 1013911 (0.0005) [2023-12-26 22:45:25,345][105692] Updated weights for policy 0, policy_version 1013921 (0.0005) [2023-12-26 22:45:25,606][105620] Updated weights for policy 1, policy_version 1014409 (0.0006) [2023-12-26 22:45:25,666][105620] Updated weights for policy 1, policy_version 1014419 (0.0009) [2023-12-26 22:45:25,722][105620] Updated weights for policy 1, policy_version 1014429 (0.0006) [2023-12-26 22:45:25,779][105620] Updated weights for policy 1, policy_version 1014439 (0.0008) [2023-12-26 22:45:25,990][105692] Updated weights for policy 0, policy_version 1013931 (0.0007) [2023-12-26 22:45:26,045][105692] Updated weights for policy 0, policy_version 1013941 (0.0009) [2023-12-26 22:45:26,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 519331840. Throughput: 0: 9591.0, 1: 9996.3. Samples: 519341708. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:45:26,062][104569] Avg episode reward: [(0, '9174.950'), (1, '9262.649')] [2023-12-26 22:45:26,100][105692] Updated weights for policy 0, policy_version 1013951 (0.0009) [2023-12-26 22:45:26,497][105620] Updated weights for policy 1, policy_version 1014449 (0.0009) [2023-12-26 22:45:26,556][105620] Updated weights for policy 1, policy_version 1014459 (0.0009) [2023-12-26 22:45:26,603][105620] Updated weights for policy 1, policy_version 1014469 (0.0009) [2023-12-26 22:45:26,835][105692] Updated weights for policy 0, policy_version 1013961 (0.0009) [2023-12-26 22:45:26,891][105692] Updated weights for policy 0, policy_version 1013971 (0.0009) [2023-12-26 22:45:26,943][105692] Updated weights for policy 0, policy_version 1013981 (0.0008) [2023-12-26 22:45:26,998][105692] Updated weights for policy 0, policy_version 1013991 (0.0006) [2023-12-26 22:45:27,390][105620] Updated weights for policy 1, policy_version 1014479 (0.0009) [2023-12-26 22:45:27,448][105620] Updated weights for policy 1, policy_version 1014489 (0.0010) [2023-12-26 22:45:27,504][105620] Updated weights for policy 1, policy_version 1014499 (0.0009) [2023-12-26 22:45:27,622][105692] Updated weights for policy 0, policy_version 1014001 (0.0009) [2023-12-26 22:45:27,676][105692] Updated weights for policy 0, policy_version 1014011 (0.0008) [2023-12-26 22:45:27,733][105692] Updated weights for policy 0, policy_version 1014021 (0.0009) [2023-12-26 22:45:28,285][105620] Updated weights for policy 1, policy_version 1014510 (0.0009) [2023-12-26 22:45:28,352][105620] Updated weights for policy 1, policy_version 1014520 (0.0010) [2023-12-26 22:45:28,406][105692] Updated weights for policy 0, policy_version 1014031 (0.0007) [2023-12-26 22:45:28,412][105620] Updated weights for policy 1, policy_version 1014530 (0.0010) [2023-12-26 22:45:28,468][105692] Updated weights for policy 0, policy_version 1014041 (0.0007) [2023-12-26 22:45:28,527][105692] Updated weights for policy 0, policy_version 1014051 (0.0007) [2023-12-26 22:45:29,086][105620] Updated weights for policy 1, policy_version 1014540 (0.0006) [2023-12-26 22:45:29,142][105620] Updated weights for policy 1, policy_version 1014550 (0.0005) [2023-12-26 22:45:29,179][105692] Updated weights for policy 0, policy_version 1014061 (0.0006) [2023-12-26 22:45:29,200][105620] Updated weights for policy 1, policy_version 1014560 (0.0006) [2023-12-26 22:45:29,234][105692] Updated weights for policy 0, policy_version 1014071 (0.0008) [2023-12-26 22:45:29,294][105692] Updated weights for policy 0, policy_version 1014081 (0.0009) [2023-12-26 22:45:29,844][105620] Updated weights for policy 1, policy_version 1014570 (0.0008) [2023-12-26 22:45:29,918][105620] Updated weights for policy 1, policy_version 1014580 (0.0008) [2023-12-26 22:45:29,972][105620] Updated weights for policy 1, policy_version 1014590 (0.0008) [2023-12-26 22:45:30,027][105620] Updated weights for policy 1, policy_version 1014600 (0.0008) [2023-12-26 22:45:30,125][105692] Updated weights for policy 0, policy_version 1014091 (0.0008) [2023-12-26 22:45:30,195][105692] Updated weights for policy 0, policy_version 1014101 (0.0009) [2023-12-26 22:45:30,257][105692] Updated weights for policy 0, policy_version 1014111 (0.0008) [2023-12-26 22:45:30,718][105620] Updated weights for policy 1, policy_version 1014610 (0.0009) [2023-12-26 22:45:30,773][105620] Updated weights for policy 1, policy_version 1014620 (0.0008) [2023-12-26 22:45:30,834][105620] Updated weights for policy 1, policy_version 1014630 (0.0008) [2023-12-26 22:45:30,997][105692] Updated weights for policy 0, policy_version 1014121 (0.0009) [2023-12-26 22:45:31,059][105692] Updated weights for policy 0, policy_version 1014131 (0.0008) [2023-12-26 22:45:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 519430144. Throughput: 0: 9713.4, 1: 9924.6. Samples: 519400396. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:45:31,063][104569] Avg episode reward: [(0, '9172.317'), (1, '9262.542')] [2023-12-26 22:45:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001014632_259776512.pth... [2023-12-26 22:45:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001013480_259481600.pth [2023-12-26 22:45:31,117][105692] Updated weights for policy 0, policy_version 1014141 (0.0010) [2023-12-26 22:45:31,184][105692] Updated weights for policy 0, policy_version 1014151 (0.0009) [2023-12-26 22:45:31,188][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001014152_259661824.pth... [2023-12-26 22:45:31,192][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001013000_259366912.pth [2023-12-26 22:45:31,508][105620] Updated weights for policy 1, policy_version 1014640 (0.0006) [2023-12-26 22:45:31,556][105620] Updated weights for policy 1, policy_version 1014650 (0.0005) [2023-12-26 22:45:31,607][105620] Updated weights for policy 1, policy_version 1014660 (0.0006) [2023-12-26 22:45:31,903][105692] Updated weights for policy 0, policy_version 1014161 (0.0008) [2023-12-26 22:45:31,963][105692] Updated weights for policy 0, policy_version 1014171 (0.0009) [2023-12-26 22:45:32,020][105692] Updated weights for policy 0, policy_version 1014181 (0.0006) [2023-12-26 22:45:32,360][105620] Updated weights for policy 1, policy_version 1014670 (0.0008) [2023-12-26 22:45:32,419][105620] Updated weights for policy 1, policy_version 1014680 (0.0008) [2023-12-26 22:45:32,473][105620] Updated weights for policy 1, policy_version 1014690 (0.0008) [2023-12-26 22:45:32,713][105692] Updated weights for policy 0, policy_version 1014191 (0.0009) [2023-12-26 22:45:32,769][105692] Updated weights for policy 0, policy_version 1014201 (0.0010) [2023-12-26 22:45:32,831][105692] Updated weights for policy 0, policy_version 1014211 (0.0010) [2023-12-26 22:45:33,141][105620] Updated weights for policy 1, policy_version 1014700 (0.0009) [2023-12-26 22:45:33,186][105620] Updated weights for policy 1, policy_version 1014710 (0.0006) [2023-12-26 22:45:33,238][105620] Updated weights for policy 1, policy_version 1014720 (0.0008) [2023-12-26 22:45:33,488][105692] Updated weights for policy 0, policy_version 1014221 (0.0010) [2023-12-26 22:45:33,537][105692] Updated weights for policy 0, policy_version 1014231 (0.0010) [2023-12-26 22:45:33,582][105692] Updated weights for policy 0, policy_version 1014241 (0.0010) [2023-12-26 22:45:33,925][105620] Updated weights for policy 1, policy_version 1014730 (0.0009) [2023-12-26 22:45:33,990][105620] Updated weights for policy 1, policy_version 1014740 (0.0006) [2023-12-26 22:45:34,049][105620] Updated weights for policy 1, policy_version 1014750 (0.0006) [2023-12-26 22:45:34,097][105620] Updated weights for policy 1, policy_version 1014760 (0.0010) [2023-12-26 22:45:34,228][105692] Updated weights for policy 0, policy_version 1014251 (0.0010) [2023-12-26 22:45:34,289][105692] Updated weights for policy 0, policy_version 1014261 (0.0011) [2023-12-26 22:45:34,349][105692] Updated weights for policy 0, policy_version 1014271 (0.0011) [2023-12-26 22:45:34,703][105620] Updated weights for policy 1, policy_version 1014770 (0.0006) [2023-12-26 22:45:34,776][105620] Updated weights for policy 1, policy_version 1014780 (0.0008) [2023-12-26 22:45:34,842][105620] Updated weights for policy 1, policy_version 1014790 (0.0011) [2023-12-26 22:45:35,080][105692] Updated weights for policy 0, policy_version 1014281 (0.0011) [2023-12-26 22:45:35,132][105692] Updated weights for policy 0, policy_version 1014291 (0.0010) [2023-12-26 22:45:35,184][105692] Updated weights for policy 0, policy_version 1014301 (0.0010) [2023-12-26 22:45:35,242][105692] Updated weights for policy 0, policy_version 1014311 (0.0010) [2023-12-26 22:45:35,394][105620] Updated weights for policy 1, policy_version 1014800 (0.0009) [2023-12-26 22:45:35,446][105620] Updated weights for policy 1, policy_version 1014810 (0.0010) [2023-12-26 22:45:35,502][105620] Updated weights for policy 1, policy_version 1014820 (0.0008) [2023-12-26 22:45:35,941][105692] Updated weights for policy 0, policy_version 1014321 (0.0006) [2023-12-26 22:45:35,994][105692] Updated weights for policy 0, policy_version 1014331 (0.0010) [2023-12-26 22:45:36,037][105692] Updated weights for policy 0, policy_version 1014341 (0.0010) [2023-12-26 22:45:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 519536640. Throughput: 0: 9716.0, 1: 9927.3. Samples: 519521660. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:45:36,062][104569] Avg episode reward: [(0, '9082.100'), (1, '9354.983')] [2023-12-26 22:45:36,228][105620] Updated weights for policy 1, policy_version 1014830 (0.0008) [2023-12-26 22:45:36,288][105620] Updated weights for policy 1, policy_version 1014840 (0.0010) [2023-12-26 22:45:36,347][105620] Updated weights for policy 1, policy_version 1014850 (0.0009) [2023-12-26 22:45:36,757][105692] Updated weights for policy 0, policy_version 1014351 (0.0009) [2023-12-26 22:45:36,817][105692] Updated weights for policy 0, policy_version 1014361 (0.0008) [2023-12-26 22:45:36,869][105692] Updated weights for policy 0, policy_version 1014371 (0.0006) [2023-12-26 22:45:37,071][105620] Updated weights for policy 1, policy_version 1014860 (0.0011) [2023-12-26 22:45:37,129][105620] Updated weights for policy 1, policy_version 1014870 (0.0010) [2023-12-26 22:45:37,199][105620] Updated weights for policy 1, policy_version 1014880 (0.0005) [2023-12-26 22:45:37,674][105692] Updated weights for policy 0, policy_version 1014381 (0.0007) [2023-12-26 22:45:37,733][105692] Updated weights for policy 0, policy_version 1014391 (0.0006) [2023-12-26 22:45:37,768][105620] Updated weights for policy 1, policy_version 1014890 (0.0009) [2023-12-26 22:45:37,797][105692] Updated weights for policy 0, policy_version 1014401 (0.0007) [2023-12-26 22:45:37,828][105620] Updated weights for policy 1, policy_version 1014900 (0.0008) [2023-12-26 22:45:37,895][105620] Updated weights for policy 1, policy_version 1014910 (0.0007) [2023-12-26 22:45:37,955][105620] Updated weights for policy 1, policy_version 1014920 (0.0009) [2023-12-26 22:45:38,525][105692] Updated weights for policy 0, policy_version 1014411 (0.0008) [2023-12-26 22:45:38,569][105620] Updated weights for policy 1, policy_version 1014930 (0.0006) [2023-12-26 22:45:38,583][105692] Updated weights for policy 0, policy_version 1014421 (0.0008) [2023-12-26 22:45:38,632][105620] Updated weights for policy 1, policy_version 1014940 (0.0006) [2023-12-26 22:45:38,633][105692] Updated weights for policy 0, policy_version 1014431 (0.0008) [2023-12-26 22:45:38,701][105620] Updated weights for policy 1, policy_version 1014950 (0.0005) [2023-12-26 22:45:39,397][105692] Updated weights for policy 0, policy_version 1014441 (0.0009) [2023-12-26 22:45:39,407][105620] Updated weights for policy 1, policy_version 1014960 (0.0007) [2023-12-26 22:45:39,467][105692] Updated weights for policy 0, policy_version 1014451 (0.0007) [2023-12-26 22:45:39,472][105620] Updated weights for policy 1, policy_version 1014970 (0.0008) [2023-12-26 22:45:39,528][105692] Updated weights for policy 0, policy_version 1014461 (0.0006) [2023-12-26 22:45:39,534][105620] Updated weights for policy 1, policy_version 1014980 (0.0008) [2023-12-26 22:45:39,587][105692] Updated weights for policy 0, policy_version 1014471 (0.0006) [2023-12-26 22:45:40,220][105620] Updated weights for policy 1, policy_version 1014990 (0.0007) [2023-12-26 22:45:40,279][105620] Updated weights for policy 1, policy_version 1015000 (0.0008) [2023-12-26 22:45:40,332][105692] Updated weights for policy 0, policy_version 1014481 (0.0008) [2023-12-26 22:45:40,334][105620] Updated weights for policy 1, policy_version 1015010 (0.0008) [2023-12-26 22:45:40,391][105692] Updated weights for policy 0, policy_version 1014491 (0.0007) [2023-12-26 22:45:40,451][105692] Updated weights for policy 0, policy_version 1014501 (0.0010) [2023-12-26 22:45:41,014][105620] Updated weights for policy 1, policy_version 1015020 (0.0008) [2023-12-26 22:45:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 519626752. Throughput: 0: 9627.2, 1: 10092.4. Samples: 519640224. Policy #0 lag: (min: 23.0, avg: 35.9, max: 55.0) [2023-12-26 22:45:41,062][104569] Avg episode reward: [(0, '8994.362'), (1, '9263.593')] [2023-12-26 22:45:41,079][105620] Updated weights for policy 1, policy_version 1015030 (0.0008) [2023-12-26 22:45:41,139][105620] Updated weights for policy 1, policy_version 1015040 (0.0008) [2023-12-26 22:45:41,254][105692] Updated weights for policy 0, policy_version 1014511 (0.0009) [2023-12-26 22:45:41,305][105692] Updated weights for policy 0, policy_version 1014521 (0.0010) [2023-12-26 22:45:41,367][105692] Updated weights for policy 0, policy_version 1014531 (0.0011) [2023-12-26 22:45:41,912][105620] Updated weights for policy 1, policy_version 1015050 (0.0008) [2023-12-26 22:45:41,973][105620] Updated weights for policy 1, policy_version 1015060 (0.0007) [2023-12-26 22:45:42,042][105620] Updated weights for policy 1, policy_version 1015070 (0.0007) [2023-12-26 22:45:42,111][105620] Updated weights for policy 1, policy_version 1015080 (0.0008) [2023-12-26 22:45:42,164][105692] Updated weights for policy 0, policy_version 1014541 (0.0010) [2023-12-26 22:45:42,223][105692] Updated weights for policy 0, policy_version 1014551 (0.0010) [2023-12-26 22:45:42,291][105692] Updated weights for policy 0, policy_version 1014561 (0.0011) [2023-12-26 22:45:42,796][105620] Updated weights for policy 1, policy_version 1015090 (0.0006) [2023-12-26 22:45:42,858][105620] Updated weights for policy 1, policy_version 1015100 (0.0011) [2023-12-26 22:45:42,917][105620] Updated weights for policy 1, policy_version 1015110 (0.0011) [2023-12-26 22:45:43,034][105692] Updated weights for policy 0, policy_version 1014571 (0.0011) [2023-12-26 22:45:43,102][105692] Updated weights for policy 0, policy_version 1014581 (0.0010) [2023-12-26 22:45:43,162][105692] Updated weights for policy 0, policy_version 1014591 (0.0010) [2023-12-26 22:45:43,478][105620] Updated weights for policy 1, policy_version 1015120 (0.0007) [2023-12-26 22:45:43,535][105620] Updated weights for policy 1, policy_version 1015130 (0.0007) [2023-12-26 22:45:43,605][105620] Updated weights for policy 1, policy_version 1015140 (0.0010) [2023-12-26 22:45:43,826][105692] Updated weights for policy 0, policy_version 1014601 (0.0010) [2023-12-26 22:45:43,885][105692] Updated weights for policy 0, policy_version 1014611 (0.0011) [2023-12-26 22:45:43,945][105692] Updated weights for policy 0, policy_version 1014621 (0.0011) [2023-12-26 22:45:44,000][105692] Updated weights for policy 0, policy_version 1014631 (0.0011) [2023-12-26 22:45:44,325][105620] Updated weights for policy 1, policy_version 1015150 (0.0010) [2023-12-26 22:45:44,382][105620] Updated weights for policy 1, policy_version 1015160 (0.0010) [2023-12-26 22:45:44,430][105620] Updated weights for policy 1, policy_version 1015170 (0.0010) [2023-12-26 22:45:44,666][105692] Updated weights for policy 0, policy_version 1014641 (0.0010) [2023-12-26 22:45:44,718][105692] Updated weights for policy 0, policy_version 1014651 (0.0010) [2023-12-26 22:45:44,793][105692] Updated weights for policy 0, policy_version 1014661 (0.0011) [2023-12-26 22:45:45,121][105620] Updated weights for policy 1, policy_version 1015180 (0.0008) [2023-12-26 22:45:45,180][105620] Updated weights for policy 1, policy_version 1015190 (0.0007) [2023-12-26 22:45:45,233][105620] Updated weights for policy 1, policy_version 1015200 (0.0010) [2023-12-26 22:45:45,549][105692] Updated weights for policy 0, policy_version 1014671 (0.0011) [2023-12-26 22:45:45,613][105692] Updated weights for policy 0, policy_version 1014681 (0.0011) [2023-12-26 22:45:45,673][105692] Updated weights for policy 0, policy_version 1014691 (0.0011) [2023-12-26 22:45:45,933][105620] Updated weights for policy 1, policy_version 1015210 (0.0009) [2023-12-26 22:45:46,008][105620] Updated weights for policy 1, policy_version 1015220 (0.0006) [2023-12-26 22:45:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 519725056. Throughput: 0: 9593.7, 1: 10031.1. Samples: 519697436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:45:46,062][104569] Avg episode reward: [(0, '8813.793'), (1, '9263.729')] [2023-12-26 22:45:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001014696_259801088.pth... [2023-12-26 22:45:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001013576_259514368.pth [2023-12-26 22:45:46,074][105620] Updated weights for policy 1, policy_version 1015230 (0.0010) [2023-12-26 22:45:46,136][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001015240_259932160.pth... [2023-12-26 22:45:46,136][105620] Updated weights for policy 1, policy_version 1015240 (0.0010) [2023-12-26 22:45:46,140][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001014056_259629056.pth [2023-12-26 22:45:46,429][105692] Updated weights for policy 0, policy_version 1014701 (0.0011) [2023-12-26 22:45:46,480][105692] Updated weights for policy 0, policy_version 1014711 (0.0008) [2023-12-26 22:45:46,531][105692] Updated weights for policy 0, policy_version 1014721 (0.0008) [2023-12-26 22:45:46,769][105620] Updated weights for policy 1, policy_version 1015250 (0.0007) [2023-12-26 22:45:46,832][105620] Updated weights for policy 1, policy_version 1015260 (0.0009) [2023-12-26 22:45:46,893][105620] Updated weights for policy 1, policy_version 1015270 (0.0010) [2023-12-26 22:45:47,321][105692] Updated weights for policy 0, policy_version 1014731 (0.0010) [2023-12-26 22:45:47,387][105692] Updated weights for policy 0, policy_version 1014741 (0.0008) [2023-12-26 22:45:47,437][105692] Updated weights for policy 0, policy_version 1014751 (0.0008) [2023-12-26 22:45:47,594][105620] Updated weights for policy 1, policy_version 1015280 (0.0010) [2023-12-26 22:45:47,643][105620] Updated weights for policy 1, policy_version 1015290 (0.0010) [2023-12-26 22:45:47,700][105620] Updated weights for policy 1, policy_version 1015300 (0.0010) [2023-12-26 22:45:48,112][105692] Updated weights for policy 0, policy_version 1014761 (0.0008) [2023-12-26 22:45:48,159][105692] Updated weights for policy 0, policy_version 1014771 (0.0005) [2023-12-26 22:45:48,208][105692] Updated weights for policy 0, policy_version 1014781 (0.0009) [2023-12-26 22:45:48,263][105692] Updated weights for policy 0, policy_version 1014791 (0.0010) [2023-12-26 22:45:48,460][105620] Updated weights for policy 1, policy_version 1015310 (0.0010) [2023-12-26 22:45:48,523][105620] Updated weights for policy 1, policy_version 1015320 (0.0010) [2023-12-26 22:45:48,588][105620] Updated weights for policy 1, policy_version 1015330 (0.0010) [2023-12-26 22:45:49,039][105692] Updated weights for policy 0, policy_version 1014801 (0.0008) [2023-12-26 22:45:49,100][105692] Updated weights for policy 0, policy_version 1014811 (0.0008) [2023-12-26 22:45:49,163][105692] Updated weights for policy 0, policy_version 1014821 (0.0009) [2023-12-26 22:45:49,324][105620] Updated weights for policy 1, policy_version 1015340 (0.0010) [2023-12-26 22:45:49,394][105620] Updated weights for policy 1, policy_version 1015350 (0.0010) [2023-12-26 22:45:49,454][105620] Updated weights for policy 1, policy_version 1015360 (0.0011) [2023-12-26 22:45:49,975][105692] Updated weights for policy 0, policy_version 1014831 (0.0008) [2023-12-26 22:45:50,041][105692] Updated weights for policy 0, policy_version 1014841 (0.0007) [2023-12-26 22:45:50,110][105692] Updated weights for policy 0, policy_version 1014851 (0.0006) [2023-12-26 22:45:50,215][105620] Updated weights for policy 1, policy_version 1015370 (0.0010) [2023-12-26 22:45:50,276][105620] Updated weights for policy 1, policy_version 1015380 (0.0010) [2023-12-26 22:45:50,338][105620] Updated weights for policy 1, policy_version 1015390 (0.0011) [2023-12-26 22:45:50,402][105620] Updated weights for policy 1, policy_version 1015400 (0.0011) [2023-12-26 22:45:50,841][105692] Updated weights for policy 0, policy_version 1014861 (0.0007) [2023-12-26 22:45:50,901][105692] Updated weights for policy 0, policy_version 1014871 (0.0007) [2023-12-26 22:45:50,961][105692] Updated weights for policy 0, policy_version 1014881 (0.0010) [2023-12-26 22:45:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 519823360. Throughput: 0: 9596.1, 1: 9955.8. Samples: 519813280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:45:51,062][104569] Avg episode reward: [(0, '8902.401'), (1, '9077.595')] [2023-12-26 22:45:51,110][105620] Updated weights for policy 1, policy_version 1015410 (0.0007) [2023-12-26 22:45:51,179][105620] Updated weights for policy 1, policy_version 1015420 (0.0007) [2023-12-26 22:45:51,236][105620] Updated weights for policy 1, policy_version 1015430 (0.0008) [2023-12-26 22:45:51,805][105692] Updated weights for policy 0, policy_version 1014891 (0.0009) [2023-12-26 22:45:51,861][105692] Updated weights for policy 0, policy_version 1014901 (0.0008) [2023-12-26 22:45:51,921][105692] Updated weights for policy 0, policy_version 1014911 (0.0008) [2023-12-26 22:45:51,975][105620] Updated weights for policy 1, policy_version 1015440 (0.0009) [2023-12-26 22:45:52,036][105620] Updated weights for policy 1, policy_version 1015450 (0.0010) [2023-12-26 22:45:52,103][105620] Updated weights for policy 1, policy_version 1015460 (0.0009) [2023-12-26 22:45:52,621][105692] Updated weights for policy 0, policy_version 1014921 (0.0006) [2023-12-26 22:45:52,680][105692] Updated weights for policy 0, policy_version 1014931 (0.0009) [2023-12-26 22:45:52,738][105692] Updated weights for policy 0, policy_version 1014941 (0.0009) [2023-12-26 22:45:52,799][105692] Updated weights for policy 0, policy_version 1014951 (0.0007) [2023-12-26 22:45:52,952][105620] Updated weights for policy 1, policy_version 1015470 (0.0010) [2023-12-26 22:45:53,016][105620] Updated weights for policy 1, policy_version 1015480 (0.0010) [2023-12-26 22:45:53,068][105620] Updated weights for policy 1, policy_version 1015490 (0.0010) [2023-12-26 22:45:53,559][105692] Updated weights for policy 0, policy_version 1014961 (0.0008) [2023-12-26 22:45:53,611][105692] Updated weights for policy 0, policy_version 1014971 (0.0008) [2023-12-26 22:45:53,673][105692] Updated weights for policy 0, policy_version 1014981 (0.0008) [2023-12-26 22:45:53,815][105620] Updated weights for policy 1, policy_version 1015500 (0.0010) [2023-12-26 22:45:53,870][105620] Updated weights for policy 1, policy_version 1015510 (0.0010) [2023-12-26 22:45:53,929][105620] Updated weights for policy 1, policy_version 1015520 (0.0010) [2023-12-26 22:45:54,449][105692] Updated weights for policy 0, policy_version 1014991 (0.0010) [2023-12-26 22:45:54,500][105692] Updated weights for policy 0, policy_version 1015001 (0.0010) [2023-12-26 22:45:54,559][105692] Updated weights for policy 0, policy_version 1015011 (0.0011) [2023-12-26 22:45:54,631][105620] Updated weights for policy 1, policy_version 1015530 (0.0010) [2023-12-26 22:45:54,679][105620] Updated weights for policy 1, policy_version 1015540 (0.0010) [2023-12-26 22:45:54,726][105620] Updated weights for policy 1, policy_version 1015550 (0.0010) [2023-12-26 22:45:54,773][105620] Updated weights for policy 1, policy_version 1015560 (0.0010) [2023-12-26 22:45:55,328][105692] Updated weights for policy 0, policy_version 1015021 (0.0011) [2023-12-26 22:45:55,387][105692] Updated weights for policy 0, policy_version 1015031 (0.0011) [2023-12-26 22:45:55,447][105692] Updated weights for policy 0, policy_version 1015041 (0.0011) [2023-12-26 22:45:55,573][105620] Updated weights for policy 1, policy_version 1015570 (0.0011) [2023-12-26 22:45:55,638][105620] Updated weights for policy 1, policy_version 1015580 (0.0010) [2023-12-26 22:45:55,697][105620] Updated weights for policy 1, policy_version 1015590 (0.0010) [2023-12-26 22:45:56,010][105692] Updated weights for policy 0, policy_version 1015051 (0.0009) [2023-12-26 22:45:56,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 519913472. Throughput: 0: 9575.3, 1: 9739.6. Samples: 519923412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:45:56,063][104569] Avg episode reward: [(0, '8993.579'), (1, '8894.789')] [2023-12-26 22:45:56,072][105692] Updated weights for policy 0, policy_version 1015061 (0.0005) [2023-12-26 22:45:56,127][105692] Updated weights for policy 0, policy_version 1015071 (0.0007) [2023-12-26 22:45:56,416][105620] Updated weights for policy 1, policy_version 1015600 (0.0009) [2023-12-26 22:45:56,483][105620] Updated weights for policy 1, policy_version 1015610 (0.0011) [2023-12-26 22:45:56,543][105620] Updated weights for policy 1, policy_version 1015620 (0.0011) [2023-12-26 22:45:56,706][105692] Updated weights for policy 0, policy_version 1015081 (0.0006) [2023-12-26 22:45:56,759][105692] Updated weights for policy 0, policy_version 1015091 (0.0005) [2023-12-26 22:45:56,816][105692] Updated weights for policy 0, policy_version 1015101 (0.0008) [2023-12-26 22:45:56,878][105692] Updated weights for policy 0, policy_version 1015111 (0.0009) [2023-12-26 22:45:57,236][105620] Updated weights for policy 1, policy_version 1015630 (0.0011) [2023-12-26 22:45:57,288][105620] Updated weights for policy 1, policy_version 1015640 (0.0010) [2023-12-26 22:45:57,345][105620] Updated weights for policy 1, policy_version 1015650 (0.0010) [2023-12-26 22:45:57,446][105692] Updated weights for policy 0, policy_version 1015121 (0.0006) [2023-12-26 22:45:57,499][105692] Updated weights for policy 0, policy_version 1015131 (0.0005) [2023-12-26 22:45:57,554][105692] Updated weights for policy 0, policy_version 1015141 (0.0005) [2023-12-26 22:45:58,104][105620] Updated weights for policy 1, policy_version 1015660 (0.0010) [2023-12-26 22:45:58,153][105620] Updated weights for policy 1, policy_version 1015670 (0.0010) [2023-12-26 22:45:58,219][105620] Updated weights for policy 1, policy_version 1015680 (0.0010) [2023-12-26 22:45:58,226][105692] Updated weights for policy 0, policy_version 1015151 (0.0009) [2023-12-26 22:45:58,281][105692] Updated weights for policy 0, policy_version 1015161 (0.0011) [2023-12-26 22:45:58,353][105692] Updated weights for policy 0, policy_version 1015172 (0.0011) [2023-12-26 22:45:59,031][105620] Updated weights for policy 1, policy_version 1015690 (0.0010) [2023-12-26 22:45:59,100][105620] Updated weights for policy 1, policy_version 1015700 (0.0009) [2023-12-26 22:45:59,170][105620] Updated weights for policy 1, policy_version 1015710 (0.0009) [2023-12-26 22:45:59,223][105692] Updated weights for policy 0, policy_version 1015182 (0.0010) [2023-12-26 22:45:59,232][105620] Updated weights for policy 1, policy_version 1015720 (0.0008) [2023-12-26 22:45:59,290][105692] Updated weights for policy 0, policy_version 1015192 (0.0008) [2023-12-26 22:45:59,350][105692] Updated weights for policy 0, policy_version 1015202 (0.0008) [2023-12-26 22:46:00,033][105692] Updated weights for policy 0, policy_version 1015212 (0.0008) [2023-12-26 22:46:00,076][105620] Updated weights for policy 1, policy_version 1015730 (0.0008) [2023-12-26 22:46:00,091][105692] Updated weights for policy 0, policy_version 1015222 (0.0008) [2023-12-26 22:46:00,126][105620] Updated weights for policy 1, policy_version 1015740 (0.0007) [2023-12-26 22:46:00,149][105692] Updated weights for policy 0, policy_version 1015232 (0.0008) [2023-12-26 22:46:00,174][105620] Updated weights for policy 1, policy_version 1015750 (0.0006) [2023-12-26 22:46:00,752][105692] Updated weights for policy 0, policy_version 1015242 (0.0008) [2023-12-26 22:46:00,796][105692] Updated weights for policy 0, policy_version 1015252 (0.0010) [2023-12-26 22:46:00,843][105692] Updated weights for policy 0, policy_version 1015262 (0.0010) [2023-12-26 22:46:00,888][105692] Updated weights for policy 0, policy_version 1015272 (0.0010) [2023-12-26 22:46:00,958][105620] Updated weights for policy 1, policy_version 1015760 (0.0006) [2023-12-26 22:46:01,012][105620] Updated weights for policy 1, policy_version 1015770 (0.0006) [2023-12-26 22:46:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 520011776. Throughput: 0: 9693.1, 1: 9738.3. Samples: 519984472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:46:01,062][104569] Avg episode reward: [(0, '9263.519'), (1, '8986.943')] [2023-12-26 22:46:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001015272_259948544.pth... [2023-12-26 22:46:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001014152_259661824.pth [2023-12-26 22:46:01,074][105620] Updated weights for policy 1, policy_version 1015780 (0.0007) [2023-12-26 22:46:01,097][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001015784_260071424.pth... [2023-12-26 22:46:01,102][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001014632_259776512.pth [2023-12-26 22:46:01,619][105692] Updated weights for policy 0, policy_version 1015282 (0.0007) [2023-12-26 22:46:01,685][105692] Updated weights for policy 0, policy_version 1015292 (0.0008) [2023-12-26 22:46:01,752][105692] Updated weights for policy 0, policy_version 1015302 (0.0008) [2023-12-26 22:46:01,799][105620] Updated weights for policy 1, policy_version 1015790 (0.0008) [2023-12-26 22:46:01,854][105620] Updated weights for policy 1, policy_version 1015800 (0.0008) [2023-12-26 22:46:01,912][105620] Updated weights for policy 1, policy_version 1015810 (0.0007) [2023-12-26 22:46:02,335][105692] Updated weights for policy 0, policy_version 1015312 (0.0008) [2023-12-26 22:46:02,387][105692] Updated weights for policy 0, policy_version 1015322 (0.0007) [2023-12-26 22:46:02,433][105692] Updated weights for policy 0, policy_version 1015332 (0.0005) [2023-12-26 22:46:02,651][105620] Updated weights for policy 1, policy_version 1015820 (0.0009) [2023-12-26 22:46:02,702][105620] Updated weights for policy 1, policy_version 1015830 (0.0009) [2023-12-26 22:46:02,760][105620] Updated weights for policy 1, policy_version 1015840 (0.0009) [2023-12-26 22:46:03,111][105692] Updated weights for policy 0, policy_version 1015342 (0.0009) [2023-12-26 22:46:03,159][105692] Updated weights for policy 0, policy_version 1015352 (0.0010) [2023-12-26 22:46:03,204][105692] Updated weights for policy 0, policy_version 1015362 (0.0010) [2023-12-26 22:46:03,459][105620] Updated weights for policy 1, policy_version 1015850 (0.0008) [2023-12-26 22:46:03,511][105620] Updated weights for policy 1, policy_version 1015860 (0.0008) [2023-12-26 22:46:03,559][105620] Updated weights for policy 1, policy_version 1015870 (0.0008) [2023-12-26 22:46:03,607][105620] Updated weights for policy 1, policy_version 1015880 (0.0008) [2023-12-26 22:46:03,932][105692] Updated weights for policy 0, policy_version 1015372 (0.0009) [2023-12-26 22:46:03,998][105692] Updated weights for policy 0, policy_version 1015382 (0.0007) [2023-12-26 22:46:04,064][105692] Updated weights for policy 0, policy_version 1015392 (0.0006) [2023-12-26 22:46:04,330][105620] Updated weights for policy 1, policy_version 1015890 (0.0006) [2023-12-26 22:46:04,402][105620] Updated weights for policy 1, policy_version 1015900 (0.0006) [2023-12-26 22:46:04,474][105620] Updated weights for policy 1, policy_version 1015910 (0.0006) [2023-12-26 22:46:04,787][105692] Updated weights for policy 0, policy_version 1015402 (0.0008) [2023-12-26 22:46:04,855][105692] Updated weights for policy 0, policy_version 1015412 (0.0009) [2023-12-26 22:46:04,906][105692] Updated weights for policy 0, policy_version 1015422 (0.0009) [2023-12-26 22:46:04,955][105692] Updated weights for policy 0, policy_version 1015432 (0.0009) [2023-12-26 22:46:05,089][105620] Updated weights for policy 1, policy_version 1015920 (0.0006) [2023-12-26 22:46:05,150][105620] Updated weights for policy 1, policy_version 1015930 (0.0005) [2023-12-26 22:46:05,208][105620] Updated weights for policy 1, policy_version 1015940 (0.0006) [2023-12-26 22:46:05,747][105692] Updated weights for policy 0, policy_version 1015442 (0.0009) [2023-12-26 22:46:05,798][105620] Updated weights for policy 1, policy_version 1015950 (0.0007) [2023-12-26 22:46:05,808][105692] Updated weights for policy 0, policy_version 1015452 (0.0010) [2023-12-26 22:46:05,860][105620] Updated weights for policy 1, policy_version 1015960 (0.0006) [2023-12-26 22:46:05,862][105692] Updated weights for policy 0, policy_version 1015462 (0.0008) [2023-12-26 22:46:05,918][105620] Updated weights for policy 1, policy_version 1015970 (0.0006) [2023-12-26 22:46:06,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19524.4, 300 sec: 19577.5). Total num frames: 520118272. Throughput: 0: 9763.4, 1: 9711.4. Samples: 520102748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:46:06,062][104569] Avg episode reward: [(0, '9084.352'), (1, '9263.056')] [2023-12-26 22:46:06,578][105692] Updated weights for policy 0, policy_version 1015472 (0.0006) [2023-12-26 22:46:06,637][105692] Updated weights for policy 0, policy_version 1015482 (0.0006) [2023-12-26 22:46:06,681][105620] Updated weights for policy 1, policy_version 1015980 (0.0010) [2023-12-26 22:46:06,697][105692] Updated weights for policy 0, policy_version 1015492 (0.0008) [2023-12-26 22:46:06,744][105620] Updated weights for policy 1, policy_version 1015990 (0.0011) [2023-12-26 22:46:06,810][105620] Updated weights for policy 1, policy_version 1016000 (0.0011) [2023-12-26 22:46:07,431][105692] Updated weights for policy 0, policy_version 1015502 (0.0008) [2023-12-26 22:46:07,492][105692] Updated weights for policy 0, policy_version 1015512 (0.0010) [2023-12-26 22:46:07,528][105620] Updated weights for policy 1, policy_version 1016010 (0.0010) [2023-12-26 22:46:07,546][105692] Updated weights for policy 0, policy_version 1015522 (0.0009) [2023-12-26 22:46:07,585][105620] Updated weights for policy 1, policy_version 1016020 (0.0005) [2023-12-26 22:46:07,636][105620] Updated weights for policy 1, policy_version 1016030 (0.0005) [2023-12-26 22:46:07,684][105620] Updated weights for policy 1, policy_version 1016040 (0.0005) [2023-12-26 22:46:08,293][105620] Updated weights for policy 1, policy_version 1016050 (0.0010) [2023-12-26 22:46:08,352][105620] Updated weights for policy 1, policy_version 1016060 (0.0010) [2023-12-26 22:46:08,370][105692] Updated weights for policy 0, policy_version 1015532 (0.0009) [2023-12-26 22:46:08,412][105620] Updated weights for policy 1, policy_version 1016070 (0.0010) [2023-12-26 22:46:08,434][105692] Updated weights for policy 0, policy_version 1015542 (0.0006) [2023-12-26 22:46:08,525][105692] Updated weights for policy 0, policy_version 1015552 (0.0006) [2023-12-26 22:46:09,112][105620] Updated weights for policy 1, policy_version 1016080 (0.0006) [2023-12-26 22:46:09,176][105620] Updated weights for policy 1, policy_version 1016090 (0.0009) [2023-12-26 22:46:09,237][105620] Updated weights for policy 1, policy_version 1016100 (0.0008) [2023-12-26 22:46:09,260][105692] Updated weights for policy 0, policy_version 1015562 (0.0008) [2023-12-26 22:46:09,314][105692] Updated weights for policy 0, policy_version 1015572 (0.0009) [2023-12-26 22:46:09,384][105692] Updated weights for policy 0, policy_version 1015582 (0.0010) [2023-12-26 22:46:09,453][105692] Updated weights for policy 0, policy_version 1015592 (0.0009) [2023-12-26 22:46:09,988][105620] Updated weights for policy 1, policy_version 1016110 (0.0008) [2023-12-26 22:46:10,051][105620] Updated weights for policy 1, policy_version 1016120 (0.0008) [2023-12-26 22:46:10,112][105620] Updated weights for policy 1, policy_version 1016130 (0.0009) [2023-12-26 22:46:10,202][105692] Updated weights for policy 0, policy_version 1015602 (0.0008) [2023-12-26 22:46:10,269][105692] Updated weights for policy 0, policy_version 1015612 (0.0009) [2023-12-26 22:46:10,335][105692] Updated weights for policy 0, policy_version 1015622 (0.0008) [2023-12-26 22:46:10,850][105620] Updated weights for policy 1, policy_version 1016140 (0.0008) [2023-12-26 22:46:10,915][105620] Updated weights for policy 1, policy_version 1016150 (0.0008) [2023-12-26 22:46:10,966][105620] Updated weights for policy 1, policy_version 1016160 (0.0009) [2023-12-26 22:46:11,036][105692] Updated weights for policy 0, policy_version 1015632 (0.0007) [2023-12-26 22:46:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 520208384. Throughput: 0: 9674.0, 1: 9780.9. Samples: 520217180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:46:11,063][104569] Avg episode reward: [(0, '9084.638'), (1, '9261.762')] [2023-12-26 22:46:11,099][105692] Updated weights for policy 0, policy_version 1015642 (0.0009) [2023-12-26 22:46:11,167][105692] Updated weights for policy 0, policy_version 1015652 (0.0008) [2023-12-26 22:46:11,686][105620] Updated weights for policy 1, policy_version 1016170 (0.0007) [2023-12-26 22:46:11,745][105620] Updated weights for policy 1, policy_version 1016180 (0.0008) [2023-12-26 22:46:11,813][105620] Updated weights for policy 1, policy_version 1016190 (0.0006) [2023-12-26 22:46:11,864][105620] Updated weights for policy 1, policy_version 1016200 (0.0006) [2023-12-26 22:46:11,963][105692] Updated weights for policy 0, policy_version 1015662 (0.0009) [2023-12-26 22:46:12,025][105692] Updated weights for policy 0, policy_version 1015672 (0.0010) [2023-12-26 22:46:12,090][105692] Updated weights for policy 0, policy_version 1015682 (0.0009) [2023-12-26 22:46:12,568][105620] Updated weights for policy 1, policy_version 1016210 (0.0009) [2023-12-26 22:46:12,630][105620] Updated weights for policy 1, policy_version 1016220 (0.0009) [2023-12-26 22:46:12,689][105620] Updated weights for policy 1, policy_version 1016230 (0.0009) [2023-12-26 22:46:12,787][105692] Updated weights for policy 0, policy_version 1015692 (0.0008) [2023-12-26 22:46:12,847][105692] Updated weights for policy 0, policy_version 1015702 (0.0009) [2023-12-26 22:46:12,912][105692] Updated weights for policy 0, policy_version 1015712 (0.0009) [2023-12-26 22:46:13,425][105620] Updated weights for policy 1, policy_version 1016240 (0.0006) [2023-12-26 22:46:13,487][105620] Updated weights for policy 1, policy_version 1016250 (0.0005) [2023-12-26 22:46:13,554][105620] Updated weights for policy 1, policy_version 1016260 (0.0008) [2023-12-26 22:46:13,617][105692] Updated weights for policy 0, policy_version 1015722 (0.0008) [2023-12-26 22:46:13,671][105692] Updated weights for policy 0, policy_version 1015732 (0.0007) [2023-12-26 22:46:13,728][105692] Updated weights for policy 0, policy_version 1015742 (0.0008) [2023-12-26 22:46:13,780][105692] Updated weights for policy 0, policy_version 1015752 (0.0008) [2023-12-26 22:46:14,199][105620] Updated weights for policy 1, policy_version 1016270 (0.0010) [2023-12-26 22:46:14,254][105620] Updated weights for policy 1, policy_version 1016280 (0.0010) [2023-12-26 22:46:14,307][105620] Updated weights for policy 1, policy_version 1016290 (0.0010) [2023-12-26 22:46:14,558][105692] Updated weights for policy 0, policy_version 1015762 (0.0006) [2023-12-26 22:46:14,620][105692] Updated weights for policy 0, policy_version 1015772 (0.0009) [2023-12-26 22:46:14,632][105585] KL-divergence is very high: 121.8780 [2023-12-26 22:46:14,673][105585] KL-divergence is very high: 124.8749 [2023-12-26 22:46:14,681][105692] Updated weights for policy 0, policy_version 1015783 (0.0010) [2023-12-26 22:46:14,979][105620] Updated weights for policy 1, policy_version 1016300 (0.0010) [2023-12-26 22:46:15,034][105620] Updated weights for policy 1, policy_version 1016310 (0.0008) [2023-12-26 22:46:15,100][105620] Updated weights for policy 1, policy_version 1016320 (0.0009) [2023-12-26 22:46:15,406][105692] Updated weights for policy 0, policy_version 1015793 (0.0006) [2023-12-26 22:46:15,464][105692] Updated weights for policy 0, policy_version 1015803 (0.0007) [2023-12-26 22:46:15,526][105692] Updated weights for policy 0, policy_version 1015813 (0.0010) [2023-12-26 22:46:15,885][105620] Updated weights for policy 1, policy_version 1016330 (0.0010) [2023-12-26 22:46:15,942][105620] Updated weights for policy 1, policy_version 1016340 (0.0008) [2023-12-26 22:46:16,004][105620] Updated weights for policy 1, policy_version 1016350 (0.0005) [2023-12-26 22:46:16,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19251.3, 300 sec: 19494.2). Total num frames: 520298496. Throughput: 0: 9610.0, 1: 9804.2. Samples: 520274032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:46:16,063][104569] Avg episode reward: [(0, '9084.666'), (1, '9172.653')] [2023-12-26 22:46:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001015816_260087808.pth... [2023-12-26 22:46:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001016360_260218880.pth... [2023-12-26 22:46:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001014696_259801088.pth [2023-12-26 22:46:16,073][105620] Updated weights for policy 1, policy_version 1016360 (0.0006) [2023-12-26 22:46:16,104][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001015240_259932160.pth [2023-12-26 22:46:16,155][105692] Updated weights for policy 0, policy_version 1015823 (0.0008) [2023-12-26 22:46:16,219][105692] Updated weights for policy 0, policy_version 1015833 (0.0008) [2023-12-26 22:46:16,284][105692] Updated weights for policy 0, policy_version 1015843 (0.0008) [2023-12-26 22:46:16,687][105620] Updated weights for policy 1, policy_version 1016370 (0.0005) [2023-12-26 22:46:16,745][105620] Updated weights for policy 1, policy_version 1016380 (0.0008) [2023-12-26 22:46:16,794][105620] Updated weights for policy 1, policy_version 1016390 (0.0008) [2023-12-26 22:46:16,997][105692] Updated weights for policy 0, policy_version 1015853 (0.0008) [2023-12-26 22:46:17,057][105692] Updated weights for policy 0, policy_version 1015863 (0.0011) [2023-12-26 22:46:17,109][105692] Updated weights for policy 0, policy_version 1015873 (0.0010) [2023-12-26 22:46:17,546][105620] Updated weights for policy 1, policy_version 1016400 (0.0008) [2023-12-26 22:46:17,591][105620] Updated weights for policy 1, policy_version 1016410 (0.0008) [2023-12-26 22:46:17,643][105620] Updated weights for policy 1, policy_version 1016421 (0.0010) [2023-12-26 22:46:17,798][105692] Updated weights for policy 0, policy_version 1015883 (0.0010) [2023-12-26 22:46:17,855][105692] Updated weights for policy 0, policy_version 1015893 (0.0009) [2023-12-26 22:46:17,912][105692] Updated weights for policy 0, policy_version 1015903 (0.0008) [2023-12-26 22:46:18,434][105620] Updated weights for policy 1, policy_version 1016432 (0.0008) [2023-12-26 22:46:18,502][105620] Updated weights for policy 1, policy_version 1016442 (0.0010) [2023-12-26 22:46:18,567][105620] Updated weights for policy 1, policy_version 1016452 (0.0009) [2023-12-26 22:46:18,641][105692] Updated weights for policy 0, policy_version 1015913 (0.0009) [2023-12-26 22:46:18,688][105692] Updated weights for policy 0, policy_version 1015923 (0.0009) [2023-12-26 22:46:18,744][105692] Updated weights for policy 0, policy_version 1015933 (0.0007) [2023-12-26 22:46:18,814][105692] Updated weights for policy 0, policy_version 1015943 (0.0009) [2023-12-26 22:46:19,366][105620] Updated weights for policy 1, policy_version 1016462 (0.0008) [2023-12-26 22:46:19,435][105620] Updated weights for policy 1, policy_version 1016472 (0.0007) [2023-12-26 22:46:19,487][105620] Updated weights for policy 1, policy_version 1016482 (0.0008) [2023-12-26 22:46:19,594][105692] Updated weights for policy 0, policy_version 1015953 (0.0008) [2023-12-26 22:46:19,652][105692] Updated weights for policy 0, policy_version 1015963 (0.0008) [2023-12-26 22:46:19,711][105692] Updated weights for policy 0, policy_version 1015973 (0.0009) [2023-12-26 22:46:20,205][105620] Updated weights for policy 1, policy_version 1016492 (0.0009) [2023-12-26 22:46:20,269][105620] Updated weights for policy 1, policy_version 1016502 (0.0009) [2023-12-26 22:46:20,331][105620] Updated weights for policy 1, policy_version 1016512 (0.0009) [2023-12-26 22:46:20,519][105692] Updated weights for policy 0, policy_version 1015983 (0.0009) [2023-12-26 22:46:20,576][105692] Updated weights for policy 0, policy_version 1015993 (0.0009) [2023-12-26 22:46:20,645][105692] Updated weights for policy 0, policy_version 1016003 (0.0009) [2023-12-26 22:46:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 520396800. Throughput: 0: 9598.3, 1: 9694.9. Samples: 520389856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:46:21,062][104569] Avg episode reward: [(0, '8819.070'), (1, '9081.438')] [2023-12-26 22:46:21,064][105620] Updated weights for policy 1, policy_version 1016522 (0.0009) [2023-12-26 22:46:21,127][105620] Updated weights for policy 1, policy_version 1016532 (0.0008) [2023-12-26 22:46:21,189][105620] Updated weights for policy 1, policy_version 1016542 (0.0006) [2023-12-26 22:46:21,250][105620] Updated weights for policy 1, policy_version 1016552 (0.0006) [2023-12-26 22:46:21,415][105692] Updated weights for policy 0, policy_version 1016013 (0.0008) [2023-12-26 22:46:21,467][105692] Updated weights for policy 0, policy_version 1016023 (0.0009) [2023-12-26 22:46:21,525][105692] Updated weights for policy 0, policy_version 1016033 (0.0009) [2023-12-26 22:46:21,969][105620] Updated weights for policy 1, policy_version 1016562 (0.0009) [2023-12-26 22:46:22,031][105620] Updated weights for policy 1, policy_version 1016572 (0.0009) [2023-12-26 22:46:22,088][105620] Updated weights for policy 1, policy_version 1016582 (0.0009) [2023-12-26 22:46:22,284][105692] Updated weights for policy 0, policy_version 1016043 (0.0008) [2023-12-26 22:46:22,346][105692] Updated weights for policy 0, policy_version 1016053 (0.0009) [2023-12-26 22:46:22,410][105692] Updated weights for policy 0, policy_version 1016063 (0.0009) [2023-12-26 22:46:22,880][105620] Updated weights for policy 1, policy_version 1016592 (0.0009) [2023-12-26 22:46:22,930][105620] Updated weights for policy 1, policy_version 1016602 (0.0009) [2023-12-26 22:46:22,978][105620] Updated weights for policy 1, policy_version 1016612 (0.0009) [2023-12-26 22:46:23,161][105692] Updated weights for policy 0, policy_version 1016073 (0.0009) [2023-12-26 22:46:23,221][105692] Updated weights for policy 0, policy_version 1016083 (0.0007) [2023-12-26 22:46:23,281][105692] Updated weights for policy 0, policy_version 1016093 (0.0009) [2023-12-26 22:46:23,338][105692] Updated weights for policy 0, policy_version 1016103 (0.0009) [2023-12-26 22:46:23,767][105620] Updated weights for policy 1, policy_version 1016622 (0.0009) [2023-12-26 22:46:23,816][105620] Updated weights for policy 1, policy_version 1016632 (0.0008) [2023-12-26 22:46:23,866][105620] Updated weights for policy 1, policy_version 1016642 (0.0009) [2023-12-26 22:46:24,045][105692] Updated weights for policy 0, policy_version 1016113 (0.0006) [2023-12-26 22:46:24,092][105692] Updated weights for policy 0, policy_version 1016123 (0.0008) [2023-12-26 22:46:24,152][105692] Updated weights for policy 0, policy_version 1016133 (0.0009) [2023-12-26 22:46:24,655][105620] Updated weights for policy 1, policy_version 1016652 (0.0009) [2023-12-26 22:46:24,702][105620] Updated weights for policy 1, policy_version 1016662 (0.0008) [2023-12-26 22:46:24,755][105620] Updated weights for policy 1, policy_version 1016672 (0.0007) [2023-12-26 22:46:24,886][105692] Updated weights for policy 0, policy_version 1016143 (0.0009) [2023-12-26 22:46:24,932][105692] Updated weights for policy 0, policy_version 1016153 (0.0008) [2023-12-26 22:46:24,979][105692] Updated weights for policy 0, policy_version 1016163 (0.0009) [2023-12-26 22:46:25,511][105620] Updated weights for policy 1, policy_version 1016682 (0.0008) [2023-12-26 22:46:25,557][105620] Updated weights for policy 1, policy_version 1016692 (0.0008) [2023-12-26 22:46:25,603][105620] Updated weights for policy 1, policy_version 1016702 (0.0008) [2023-12-26 22:46:25,650][105620] Updated weights for policy 1, policy_version 1016712 (0.0008) [2023-12-26 22:46:25,767][105692] Updated weights for policy 0, policy_version 1016173 (0.0009) [2023-12-26 22:46:25,826][105692] Updated weights for policy 0, policy_version 1016183 (0.0009) [2023-12-26 22:46:25,885][105692] Updated weights for policy 0, policy_version 1016193 (0.0009) [2023-12-26 22:46:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 520495104. Throughput: 0: 9567.5, 1: 9559.1. Samples: 520500924. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:46:26,063][104569] Avg episode reward: [(0, '8821.926'), (1, '9081.414')] [2023-12-26 22:46:26,475][105620] Updated weights for policy 1, policy_version 1016722 (0.0010) [2023-12-26 22:46:26,504][105692] Updated weights for policy 0, policy_version 1016203 (0.0008) [2023-12-26 22:46:26,533][105620] Updated weights for policy 1, policy_version 1016732 (0.0009) [2023-12-26 22:46:26,563][105692] Updated weights for policy 0, policy_version 1016213 (0.0005) [2023-12-26 22:46:26,590][105620] Updated weights for policy 1, policy_version 1016742 (0.0009) [2023-12-26 22:46:26,614][105692] Updated weights for policy 0, policy_version 1016223 (0.0005) [2023-12-26 22:46:27,134][105692] Updated weights for policy 0, policy_version 1016233 (0.0005) [2023-12-26 22:46:27,200][105692] Updated weights for policy 0, policy_version 1016243 (0.0006) [2023-12-26 22:46:27,254][105692] Updated weights for policy 0, policy_version 1016253 (0.0005) [2023-12-26 22:46:27,307][105692] Updated weights for policy 0, policy_version 1016263 (0.0005) [2023-12-26 22:46:27,495][105620] Updated weights for policy 1, policy_version 1016752 (0.0009) [2023-12-26 22:46:27,552][105620] Updated weights for policy 1, policy_version 1016762 (0.0008) [2023-12-26 22:46:27,606][105620] Updated weights for policy 1, policy_version 1016772 (0.0008) [2023-12-26 22:46:27,959][105692] Updated weights for policy 0, policy_version 1016273 (0.0010) [2023-12-26 22:46:28,010][105692] Updated weights for policy 0, policy_version 1016283 (0.0010) [2023-12-26 22:46:28,060][105692] Updated weights for policy 0, policy_version 1016293 (0.0010) [2023-12-26 22:46:28,367][105620] Updated weights for policy 1, policy_version 1016782 (0.0009) [2023-12-26 22:46:28,431][105620] Updated weights for policy 1, policy_version 1016792 (0.0008) [2023-12-26 22:46:28,493][105620] Updated weights for policy 1, policy_version 1016802 (0.0008) [2023-12-26 22:46:28,824][105692] Updated weights for policy 0, policy_version 1016303 (0.0010) [2023-12-26 22:46:28,882][105692] Updated weights for policy 0, policy_version 1016313 (0.0011) [2023-12-26 22:46:28,942][105692] Updated weights for policy 0, policy_version 1016323 (0.0007) [2023-12-26 22:46:29,260][105620] Updated weights for policy 1, policy_version 1016812 (0.0008) [2023-12-26 22:46:29,328][105620] Updated weights for policy 1, policy_version 1016822 (0.0008) [2023-12-26 22:46:29,385][105620] Updated weights for policy 1, policy_version 1016832 (0.0008) [2023-12-26 22:46:29,637][105692] Updated weights for policy 0, policy_version 1016333 (0.0007) [2023-12-26 22:46:29,698][105692] Updated weights for policy 0, policy_version 1016343 (0.0009) [2023-12-26 22:46:29,749][105692] Updated weights for policy 0, policy_version 1016353 (0.0009) [2023-12-26 22:46:30,157][105620] Updated weights for policy 1, policy_version 1016842 (0.0009) [2023-12-26 22:46:30,212][105620] Updated weights for policy 1, policy_version 1016852 (0.0010) [2023-12-26 22:46:30,269][105620] Updated weights for policy 1, policy_version 1016862 (0.0009) [2023-12-26 22:46:30,325][105620] Updated weights for policy 1, policy_version 1016872 (0.0006) [2023-12-26 22:46:30,379][105692] Updated weights for policy 0, policy_version 1016363 (0.0009) [2023-12-26 22:46:30,428][105692] Updated weights for policy 0, policy_version 1016373 (0.0009) [2023-12-26 22:46:30,477][105692] Updated weights for policy 0, policy_version 1016383 (0.0007) [2023-12-26 22:46:31,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 520585216. Throughput: 0: 9674.2, 1: 9481.5. Samples: 520559444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:46:31,063][104569] Avg episode reward: [(0, '8733.285'), (1, '9353.916')] [2023-12-26 22:46:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001016392_260235264.pth... [2023-12-26 22:46:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001015272_259948544.pth [2023-12-26 22:46:31,127][105620] Updated weights for policy 1, policy_version 1016882 (0.0008) [2023-12-26 22:46:31,127][105692] Updated weights for policy 0, policy_version 1016393 (0.0006) [2023-12-26 22:46:31,187][105692] Updated weights for policy 0, policy_version 1016403 (0.0008) [2023-12-26 22:46:31,197][105620] Updated weights for policy 1, policy_version 1016892 (0.0007) [2023-12-26 22:46:31,247][105692] Updated weights for policy 0, policy_version 1016413 (0.0008) [2023-12-26 22:46:31,264][105620] Updated weights for policy 1, policy_version 1016902 (0.0007) [2023-12-26 22:46:31,277][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001016904_260358144.pth... [2023-12-26 22:46:31,281][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001015784_260071424.pth [2023-12-26 22:46:31,308][105692] Updated weights for policy 0, policy_version 1016423 (0.0006) [2023-12-26 22:46:32,027][105692] Updated weights for policy 0, policy_version 1016433 (0.0008) [2023-12-26 22:46:32,090][105692] Updated weights for policy 0, policy_version 1016443 (0.0009) [2023-12-26 22:46:32,100][105620] Updated weights for policy 1, policy_version 1016912 (0.0009) [2023-12-26 22:46:32,147][105692] Updated weights for policy 0, policy_version 1016453 (0.0006) [2023-12-26 22:46:32,154][105620] Updated weights for policy 1, policy_version 1016922 (0.0008) [2023-12-26 22:46:32,211][105620] Updated weights for policy 1, policy_version 1016932 (0.0009) [2023-12-26 22:46:32,842][105620] Updated weights for policy 1, policy_version 1016942 (0.0007) [2023-12-26 22:46:32,894][105692] Updated weights for policy 0, policy_version 1016463 (0.0006) [2023-12-26 22:46:32,903][105620] Updated weights for policy 1, policy_version 1016952 (0.0008) [2023-12-26 22:46:32,953][105692] Updated weights for policy 0, policy_version 1016473 (0.0005) [2023-12-26 22:46:32,969][105620] Updated weights for policy 1, policy_version 1016962 (0.0010) [2023-12-26 22:46:33,008][105692] Updated weights for policy 0, policy_version 1016483 (0.0007) [2023-12-26 22:46:33,579][105620] Updated weights for policy 1, policy_version 1016972 (0.0008) [2023-12-26 22:46:33,623][105620] Updated weights for policy 1, policy_version 1016982 (0.0005) [2023-12-26 22:46:33,673][105620] Updated weights for policy 1, policy_version 1016992 (0.0005) [2023-12-26 22:46:33,830][105692] Updated weights for policy 0, policy_version 1016493 (0.0009) [2023-12-26 22:46:33,888][105692] Updated weights for policy 0, policy_version 1016503 (0.0009) [2023-12-26 22:46:33,946][105692] Updated weights for policy 0, policy_version 1016513 (0.0009) [2023-12-26 22:46:34,307][105620] Updated weights for policy 1, policy_version 1017002 (0.0006) [2023-12-26 22:46:34,361][105620] Updated weights for policy 1, policy_version 1017012 (0.0009) [2023-12-26 22:46:34,419][105620] Updated weights for policy 1, policy_version 1017022 (0.0009) [2023-12-26 22:46:34,485][105620] Updated weights for policy 1, policy_version 1017032 (0.0009) [2023-12-26 22:46:34,691][105692] Updated weights for policy 0, policy_version 1016523 (0.0009) [2023-12-26 22:46:34,739][105692] Updated weights for policy 0, policy_version 1016533 (0.0009) [2023-12-26 22:46:34,788][105692] Updated weights for policy 0, policy_version 1016543 (0.0009) [2023-12-26 22:46:35,206][105620] Updated weights for policy 1, policy_version 1017042 (0.0009) [2023-12-26 22:46:35,259][105620] Updated weights for policy 1, policy_version 1017052 (0.0009) [2023-12-26 22:46:35,309][105620] Updated weights for policy 1, policy_version 1017062 (0.0009) [2023-12-26 22:46:35,572][105692] Updated weights for policy 0, policy_version 1016553 (0.0009) [2023-12-26 22:46:35,623][105692] Updated weights for policy 0, policy_version 1016563 (0.0009) [2023-12-26 22:46:35,685][105692] Updated weights for policy 0, policy_version 1016573 (0.0007) [2023-12-26 22:46:35,747][105692] Updated weights for policy 0, policy_version 1016583 (0.0005) [2023-12-26 22:46:36,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.6, 300 sec: 19466.4). Total num frames: 520683520. Throughput: 0: 9698.8, 1: 9475.5. Samples: 520676120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:46:36,062][104569] Avg episode reward: [(0, '8912.032'), (1, '9275.773')] [2023-12-26 22:46:36,114][105620] Updated weights for policy 1, policy_version 1017072 (0.0008) [2023-12-26 22:46:36,176][105620] Updated weights for policy 1, policy_version 1017082 (0.0009) [2023-12-26 22:46:36,238][105620] Updated weights for policy 1, policy_version 1017092 (0.0009) [2023-12-26 22:46:36,429][105692] Updated weights for policy 0, policy_version 1016593 (0.0008) [2023-12-26 22:46:36,496][105692] Updated weights for policy 0, policy_version 1016603 (0.0008) [2023-12-26 22:46:36,563][105692] Updated weights for policy 0, policy_version 1016613 (0.0009) [2023-12-26 22:46:36,978][105620] Updated weights for policy 1, policy_version 1017102 (0.0009) [2023-12-26 22:46:37,029][105620] Updated weights for policy 1, policy_version 1017112 (0.0009) [2023-12-26 22:46:37,084][105620] Updated weights for policy 1, policy_version 1017122 (0.0008) [2023-12-26 22:46:37,345][105692] Updated weights for policy 0, policy_version 1016623 (0.0009) [2023-12-26 22:46:37,395][105692] Updated weights for policy 0, policy_version 1016633 (0.0009) [2023-12-26 22:46:37,450][105692] Updated weights for policy 0, policy_version 1016643 (0.0009) [2023-12-26 22:46:37,832][105620] Updated weights for policy 1, policy_version 1017132 (0.0009) [2023-12-26 22:46:37,887][105620] Updated weights for policy 1, policy_version 1017142 (0.0009) [2023-12-26 22:46:37,942][105620] Updated weights for policy 1, policy_version 1017152 (0.0009) [2023-12-26 22:46:38,228][105692] Updated weights for policy 0, policy_version 1016653 (0.0008) [2023-12-26 22:46:38,276][105692] Updated weights for policy 0, policy_version 1016663 (0.0009) [2023-12-26 22:46:38,322][105692] Updated weights for policy 0, policy_version 1016673 (0.0009) [2023-12-26 22:46:38,711][105620] Updated weights for policy 1, policy_version 1017162 (0.0009) [2023-12-26 22:46:38,769][105620] Updated weights for policy 1, policy_version 1017172 (0.0009) [2023-12-26 22:46:38,823][105620] Updated weights for policy 1, policy_version 1017182 (0.0009) [2023-12-26 22:46:38,884][105620] Updated weights for policy 1, policy_version 1017192 (0.0009) [2023-12-26 22:46:39,089][105692] Updated weights for policy 0, policy_version 1016683 (0.0008) [2023-12-26 22:46:39,149][105692] Updated weights for policy 0, policy_version 1016693 (0.0009) [2023-12-26 22:46:39,203][105692] Updated weights for policy 0, policy_version 1016703 (0.0009) [2023-12-26 22:46:39,699][105620] Updated weights for policy 1, policy_version 1017202 (0.0010) [2023-12-26 22:46:39,774][105620] Updated weights for policy 1, policy_version 1017212 (0.0009) [2023-12-26 22:46:39,831][105620] Updated weights for policy 1, policy_version 1017222 (0.0008) [2023-12-26 22:46:39,864][105692] Updated weights for policy 0, policy_version 1016713 (0.0008) [2023-12-26 22:46:39,923][105692] Updated weights for policy 0, policy_version 1016723 (0.0009) [2023-12-26 22:46:39,991][105692] Updated weights for policy 0, policy_version 1016733 (0.0009) [2023-12-26 22:46:40,054][105692] Updated weights for policy 0, policy_version 1016743 (0.0009) [2023-12-26 22:46:40,675][105620] Updated weights for policy 1, policy_version 1017232 (0.0008) [2023-12-26 22:46:40,682][105692] Updated weights for policy 0, policy_version 1016753 (0.0007) [2023-12-26 22:46:40,732][105620] Updated weights for policy 1, policy_version 1017242 (0.0008) [2023-12-26 22:46:40,734][105692] Updated weights for policy 0, policy_version 1016763 (0.0006) [2023-12-26 22:46:40,781][105620] Updated weights for policy 1, policy_version 1017252 (0.0006) [2023-12-26 22:46:40,795][105692] Updated weights for policy 0, policy_version 1016773 (0.0007) [2023-12-26 22:46:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 520781824. Throughput: 0: 9766.8, 1: 9441.0. Samples: 520787760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:46:41,062][104569] Avg episode reward: [(0, '8384.713'), (1, '9196.831')] [2023-12-26 22:46:41,525][105692] Updated weights for policy 0, policy_version 1016783 (0.0009) [2023-12-26 22:46:41,557][105620] Updated weights for policy 1, policy_version 1017262 (0.0007) [2023-12-26 22:46:41,581][105692] Updated weights for policy 0, policy_version 1016793 (0.0007) [2023-12-26 22:46:41,604][105585] KL-divergence is very high: 103.7480 [2023-12-26 22:46:41,611][105585] KL-divergence is very high: 108.0404 [2023-12-26 22:46:41,625][105620] Updated weights for policy 1, policy_version 1017272 (0.0008) [2023-12-26 22:46:41,643][105692] Updated weights for policy 0, policy_version 1016803 (0.0008) [2023-12-26 22:46:41,651][105585] KL-divergence is very high: 123.4995 [2023-12-26 22:46:41,658][105585] KL-divergence is very high: 117.7279 [2023-12-26 22:46:41,665][105585] KL-divergence is very high: 106.9781 [2023-12-26 22:46:41,690][105620] Updated weights for policy 1, policy_version 1017282 (0.0008) [2023-12-26 22:46:42,421][105620] Updated weights for policy 1, policy_version 1017292 (0.0008) [2023-12-26 22:46:42,468][105692] Updated weights for policy 0, policy_version 1016813 (0.0007) [2023-12-26 22:46:42,488][105620] Updated weights for policy 1, policy_version 1017302 (0.0006) [2023-12-26 22:46:42,528][105692] Updated weights for policy 0, policy_version 1016823 (0.0009) [2023-12-26 22:46:42,548][105620] Updated weights for policy 1, policy_version 1017312 (0.0008) [2023-12-26 22:46:42,588][105692] Updated weights for policy 0, policy_version 1016833 (0.0007) [2023-12-26 22:46:43,119][105620] Updated weights for policy 1, policy_version 1017322 (0.0008) [2023-12-26 22:46:43,178][105620] Updated weights for policy 1, policy_version 1017332 (0.0009) [2023-12-26 22:46:43,227][105620] Updated weights for policy 1, policy_version 1017342 (0.0008) [2023-12-26 22:46:43,284][105620] Updated weights for policy 1, policy_version 1017352 (0.0008) [2023-12-26 22:46:43,410][105692] Updated weights for policy 0, policy_version 1016843 (0.0009) [2023-12-26 22:46:43,473][105692] Updated weights for policy 0, policy_version 1016853 (0.0009) [2023-12-26 22:46:43,532][105692] Updated weights for policy 0, policy_version 1016863 (0.0008) [2023-12-26 22:46:44,023][105620] Updated weights for policy 1, policy_version 1017362 (0.0008) [2023-12-26 22:46:44,080][105620] Updated weights for policy 1, policy_version 1017372 (0.0008) [2023-12-26 22:46:44,137][105620] Updated weights for policy 1, policy_version 1017382 (0.0008) [2023-12-26 22:46:44,304][105692] Updated weights for policy 0, policy_version 1016873 (0.0010) [2023-12-26 22:46:44,363][105692] Updated weights for policy 0, policy_version 1016883 (0.0009) [2023-12-26 22:46:44,427][105692] Updated weights for policy 0, policy_version 1016893 (0.0009) [2023-12-26 22:46:44,486][105692] Updated weights for policy 0, policy_version 1016903 (0.0009) [2023-12-26 22:46:44,927][105620] Updated weights for policy 1, policy_version 1017392 (0.0009) [2023-12-26 22:46:44,985][105620] Updated weights for policy 1, policy_version 1017402 (0.0009) [2023-12-26 22:46:45,045][105620] Updated weights for policy 1, policy_version 1017412 (0.0012) [2023-12-26 22:46:45,174][105692] Updated weights for policy 0, policy_version 1016913 (0.0009) [2023-12-26 22:46:45,230][105692] Updated weights for policy 0, policy_version 1016923 (0.0009) [2023-12-26 22:46:45,282][105692] Updated weights for policy 0, policy_version 1016933 (0.0009) [2023-12-26 22:46:45,823][105620] Updated weights for policy 1, policy_version 1017422 (0.0008) [2023-12-26 22:46:45,885][105620] Updated weights for policy 1, policy_version 1017432 (0.0008) [2023-12-26 22:46:45,947][105620] Updated weights for policy 1, policy_version 1017442 (0.0008) [2023-12-26 22:46:45,984][105692] Updated weights for policy 0, policy_version 1016943 (0.0006) [2023-12-26 22:46:46,043][105692] Updated weights for policy 0, policy_version 1016953 (0.0006) [2023-12-26 22:46:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19438.6). Total num frames: 520871936. Throughput: 0: 9620.3, 1: 9481.8. Samples: 520844064. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:46:46,062][104569] Avg episode reward: [(0, '8066.591'), (1, '9197.486')] [2023-12-26 22:46:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001017448_260497408.pth... [2023-12-26 22:46:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001016360_260218880.pth [2023-12-26 22:46:46,110][105692] Updated weights for policy 0, policy_version 1016963 (0.0006) [2023-12-26 22:46:46,134][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001016968_260382720.pth... [2023-12-26 22:46:46,137][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001015816_260087808.pth [2023-12-26 22:46:46,597][105620] Updated weights for policy 1, policy_version 1017452 (0.0007) [2023-12-26 22:46:46,642][105620] Updated weights for policy 1, policy_version 1017462 (0.0005) [2023-12-26 22:46:46,697][105620] Updated weights for policy 1, policy_version 1017472 (0.0006) [2023-12-26 22:46:46,829][105692] Updated weights for policy 0, policy_version 1016973 (0.0008) [2023-12-26 22:46:46,887][105692] Updated weights for policy 0, policy_version 1016983 (0.0010) [2023-12-26 22:46:46,954][105692] Updated weights for policy 0, policy_version 1016993 (0.0009) [2023-12-26 22:46:47,402][105620] Updated weights for policy 1, policy_version 1017482 (0.0009) [2023-12-26 22:46:47,453][105620] Updated weights for policy 1, policy_version 1017492 (0.0008) [2023-12-26 22:46:47,507][105620] Updated weights for policy 1, policy_version 1017502 (0.0008) [2023-12-26 22:46:47,557][105620] Updated weights for policy 1, policy_version 1017512 (0.0009) [2023-12-26 22:46:47,698][105692] Updated weights for policy 0, policy_version 1017003 (0.0009) [2023-12-26 22:46:47,754][105692] Updated weights for policy 0, policy_version 1017013 (0.0009) [2023-12-26 22:46:47,813][105692] Updated weights for policy 0, policy_version 1017023 (0.0009) [2023-12-26 22:46:48,246][105620] Updated weights for policy 1, policy_version 1017522 (0.0008) [2023-12-26 22:46:48,310][105620] Updated weights for policy 1, policy_version 1017532 (0.0008) [2023-12-26 22:46:48,377][105620] Updated weights for policy 1, policy_version 1017542 (0.0006) [2023-12-26 22:46:48,632][105692] Updated weights for policy 0, policy_version 1017033 (0.0009) [2023-12-26 22:46:48,701][105692] Updated weights for policy 0, policy_version 1017043 (0.0007) [2023-12-26 22:46:48,763][105692] Updated weights for policy 0, policy_version 1017053 (0.0008) [2023-12-26 22:46:48,823][105692] Updated weights for policy 0, policy_version 1017063 (0.0008) [2023-12-26 22:46:49,053][105620] Updated weights for policy 1, policy_version 1017552 (0.0010) [2023-12-26 22:46:49,125][105620] Updated weights for policy 1, policy_version 1017562 (0.0010) [2023-12-26 22:46:49,179][105620] Updated weights for policy 1, policy_version 1017572 (0.0010) [2023-12-26 22:46:49,540][105692] Updated weights for policy 0, policy_version 1017073 (0.0008) [2023-12-26 22:46:49,599][105692] Updated weights for policy 0, policy_version 1017083 (0.0008) [2023-12-26 22:46:49,658][105692] Updated weights for policy 0, policy_version 1017093 (0.0008) [2023-12-26 22:46:49,898][105620] Updated weights for policy 1, policy_version 1017582 (0.0009) [2023-12-26 22:46:49,960][105620] Updated weights for policy 1, policy_version 1017592 (0.0008) [2023-12-26 22:46:50,028][105620] Updated weights for policy 1, policy_version 1017602 (0.0010) [2023-12-26 22:46:50,310][105692] Updated weights for policy 0, policy_version 1017103 (0.0009) [2023-12-26 22:46:50,368][105692] Updated weights for policy 0, policy_version 1017113 (0.0007) [2023-12-26 22:46:50,423][105692] Updated weights for policy 0, policy_version 1017123 (0.0005) [2023-12-26 22:46:50,843][105620] Updated weights for policy 1, policy_version 1017612 (0.0009) [2023-12-26 22:46:50,899][105620] Updated weights for policy 1, policy_version 1017622 (0.0009) [2023-12-26 22:46:50,957][105620] Updated weights for policy 1, policy_version 1017632 (0.0010) [2023-12-26 22:46:51,032][105692] Updated weights for policy 0, policy_version 1017133 (0.0006) [2023-12-26 22:46:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19438.6). Total num frames: 520970240. Throughput: 0: 9509.2, 1: 9511.6. Samples: 520958684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:46:51,062][104569] Avg episode reward: [(0, '7999.438'), (1, '8985.149')] [2023-12-26 22:46:51,093][105692] Updated weights for policy 0, policy_version 1017143 (0.0007) [2023-12-26 22:46:51,159][105692] Updated weights for policy 0, policy_version 1017153 (0.0007) [2023-12-26 22:46:51,747][105620] Updated weights for policy 1, policy_version 1017642 (0.0009) [2023-12-26 22:46:51,805][105620] Updated weights for policy 1, policy_version 1017652 (0.0005) [2023-12-26 22:46:51,854][105620] Updated weights for policy 1, policy_version 1017662 (0.0008) [2023-12-26 22:46:51,908][105692] Updated weights for policy 0, policy_version 1017163 (0.0007) [2023-12-26 22:46:51,910][105620] Updated weights for policy 1, policy_version 1017672 (0.0008) [2023-12-26 22:46:51,970][105692] Updated weights for policy 0, policy_version 1017173 (0.0010) [2023-12-26 22:46:52,029][105692] Updated weights for policy 0, policy_version 1017183 (0.0006) [2023-12-26 22:46:52,696][105620] Updated weights for policy 1, policy_version 1017682 (0.0007) [2023-12-26 22:46:52,748][105692] Updated weights for policy 0, policy_version 1017193 (0.0008) [2023-12-26 22:46:52,759][105620] Updated weights for policy 1, policy_version 1017692 (0.0007) [2023-12-26 22:46:52,809][105692] Updated weights for policy 0, policy_version 1017203 (0.0009) [2023-12-26 22:46:52,821][105620] Updated weights for policy 1, policy_version 1017702 (0.0007) [2023-12-26 22:46:52,872][105692] Updated weights for policy 0, policy_version 1017213 (0.0008) [2023-12-26 22:46:52,937][105692] Updated weights for policy 0, policy_version 1017223 (0.0006) [2023-12-26 22:46:53,484][105692] Updated weights for policy 0, policy_version 1017233 (0.0005) [2023-12-26 22:46:53,531][105692] Updated weights for policy 0, policy_version 1017243 (0.0009) [2023-12-26 22:46:53,548][105620] Updated weights for policy 1, policy_version 1017713 (0.0007) [2023-12-26 22:46:53,576][105692] Updated weights for policy 0, policy_version 1017253 (0.0008) [2023-12-26 22:46:53,612][105620] Updated weights for policy 1, policy_version 1017723 (0.0007) [2023-12-26 22:46:53,672][105620] Updated weights for policy 1, policy_version 1017733 (0.0010) [2023-12-26 22:46:54,251][105692] Updated weights for policy 0, policy_version 1017263 (0.0009) [2023-12-26 22:46:54,288][105620] Updated weights for policy 1, policy_version 1017743 (0.0007) [2023-12-26 22:46:54,314][105692] Updated weights for policy 0, policy_version 1017273 (0.0011) [2023-12-26 22:46:54,347][105620] Updated weights for policy 1, policy_version 1017753 (0.0005) [2023-12-26 22:46:54,372][105692] Updated weights for policy 0, policy_version 1017283 (0.0010) [2023-12-26 22:46:54,410][105620] Updated weights for policy 1, policy_version 1017763 (0.0007) [2023-12-26 22:46:55,066][105692] Updated weights for policy 0, policy_version 1017293 (0.0008) [2023-12-26 22:46:55,098][105620] Updated weights for policy 1, policy_version 1017773 (0.0008) [2023-12-26 22:46:55,124][105692] Updated weights for policy 0, policy_version 1017303 (0.0010) [2023-12-26 22:46:55,160][105620] Updated weights for policy 1, policy_version 1017783 (0.0008) [2023-12-26 22:46:55,186][105692] Updated weights for policy 0, policy_version 1017313 (0.0011) [2023-12-26 22:46:55,216][105620] Updated weights for policy 1, policy_version 1017793 (0.0005) [2023-12-26 22:46:55,820][105620] Updated weights for policy 1, policy_version 1017803 (0.0005) [2023-12-26 22:46:55,877][105620] Updated weights for policy 1, policy_version 1017813 (0.0005) [2023-12-26 22:46:55,877][105692] Updated weights for policy 0, policy_version 1017323 (0.0009) [2023-12-26 22:46:55,922][105692] Updated weights for policy 0, policy_version 1017333 (0.0005) [2023-12-26 22:46:55,935][105620] Updated weights for policy 1, policy_version 1017823 (0.0005) [2023-12-26 22:46:55,977][105692] Updated weights for policy 0, policy_version 1017343 (0.0005) [2023-12-26 22:46:56,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 521076736. Throughput: 0: 9663.4, 1: 9478.1. Samples: 521078544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:46:56,062][104569] Avg episode reward: [(0, '8400.694'), (1, '8811.300')] [2023-12-26 22:46:56,533][105692] Updated weights for policy 0, policy_version 1017353 (0.0005) [2023-12-26 22:46:56,583][105692] Updated weights for policy 0, policy_version 1017363 (0.0005) [2023-12-26 22:46:56,631][105692] Updated weights for policy 0, policy_version 1017373 (0.0005) [2023-12-26 22:46:56,654][105620] Updated weights for policy 1, policy_version 1017833 (0.0005) [2023-12-26 22:46:56,698][105692] Updated weights for policy 0, policy_version 1017383 (0.0009) [2023-12-26 22:46:56,713][105620] Updated weights for policy 1, policy_version 1017843 (0.0006) [2023-12-26 22:46:56,777][105620] Updated weights for policy 1, policy_version 1017853 (0.0007) [2023-12-26 22:46:56,842][105620] Updated weights for policy 1, policy_version 1017863 (0.0005) [2023-12-26 22:46:57,359][105620] Updated weights for policy 1, policy_version 1017873 (0.0006) [2023-12-26 22:46:57,365][105692] Updated weights for policy 0, policy_version 1017393 (0.0008) [2023-12-26 22:46:57,417][105620] Updated weights for policy 1, policy_version 1017883 (0.0007) [2023-12-26 22:46:57,419][105692] Updated weights for policy 0, policy_version 1017403 (0.0008) [2023-12-26 22:46:57,467][105692] Updated weights for policy 0, policy_version 1017413 (0.0007) [2023-12-26 22:46:57,473][105620] Updated weights for policy 1, policy_version 1017893 (0.0007) [2023-12-26 22:46:58,115][105692] Updated weights for policy 0, policy_version 1017423 (0.0005) [2023-12-26 22:46:58,164][105620] Updated weights for policy 1, policy_version 1017903 (0.0007) [2023-12-26 22:46:58,173][105692] Updated weights for policy 0, policy_version 1017433 (0.0006) [2023-12-26 22:46:58,230][105620] Updated weights for policy 1, policy_version 1017913 (0.0008) [2023-12-26 22:46:58,234][105692] Updated weights for policy 0, policy_version 1017443 (0.0009) [2023-12-26 22:46:58,288][105620] Updated weights for policy 1, policy_version 1017923 (0.0007) [2023-12-26 22:46:59,047][105620] Updated weights for policy 1, policy_version 1017933 (0.0007) [2023-12-26 22:46:59,066][105692] Updated weights for policy 0, policy_version 1017453 (0.0008) [2023-12-26 22:46:59,101][105620] Updated weights for policy 1, policy_version 1017943 (0.0007) [2023-12-26 22:46:59,127][105692] Updated weights for policy 0, policy_version 1017463 (0.0009) [2023-12-26 22:46:59,161][105620] Updated weights for policy 1, policy_version 1017953 (0.0006) [2023-12-26 22:46:59,189][105692] Updated weights for policy 0, policy_version 1017473 (0.0009) [2023-12-26 22:46:59,877][105692] Updated weights for policy 0, policy_version 1017483 (0.0010) [2023-12-26 22:46:59,933][105692] Updated weights for policy 0, policy_version 1017493 (0.0007) [2023-12-26 22:46:59,993][105620] Updated weights for policy 1, policy_version 1017963 (0.0008) [2023-12-26 22:46:59,999][105692] Updated weights for policy 0, policy_version 1017503 (0.0006) [2023-12-26 22:47:00,049][105620] Updated weights for policy 1, policy_version 1017973 (0.0008) [2023-12-26 22:47:00,109][105620] Updated weights for policy 1, policy_version 1017983 (0.0008) [2023-12-26 22:47:00,622][105692] Updated weights for policy 0, policy_version 1017513 (0.0005) [2023-12-26 22:47:00,684][105692] Updated weights for policy 0, policy_version 1017523 (0.0005) [2023-12-26 22:47:00,752][105692] Updated weights for policy 0, policy_version 1017533 (0.0005) [2023-12-26 22:47:00,779][105620] Updated weights for policy 1, policy_version 1017993 (0.0009) [2023-12-26 22:47:00,805][105692] Updated weights for policy 0, policy_version 1017543 (0.0005) [2023-12-26 22:47:00,828][105620] Updated weights for policy 1, policy_version 1018003 (0.0008) [2023-12-26 22:47:00,877][105620] Updated weights for policy 1, policy_version 1018013 (0.0009) [2023-12-26 22:47:00,928][105620] Updated weights for policy 1, policy_version 1018023 (0.0009) [2023-12-26 22:47:01,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 521175040. Throughput: 0: 9762.5, 1: 9509.7. Samples: 521141284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:47:01,063][104569] Avg episode reward: [(0, '8817.374'), (1, '8811.513')] [2023-12-26 22:47:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001017544_260530176.pth... [2023-12-26 22:47:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001018024_260644864.pth... [2023-12-26 22:47:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001016392_260235264.pth [2023-12-26 22:47:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001016904_260358144.pth [2023-12-26 22:47:01,475][105692] Updated weights for policy 0, policy_version 1017553 (0.0008) [2023-12-26 22:47:01,531][105692] Updated weights for policy 0, policy_version 1017563 (0.0008) [2023-12-26 22:47:01,587][105692] Updated weights for policy 0, policy_version 1017573 (0.0006) [2023-12-26 22:47:01,706][105620] Updated weights for policy 1, policy_version 1018033 (0.0010) [2023-12-26 22:47:01,774][105620] Updated weights for policy 1, policy_version 1018043 (0.0011) [2023-12-26 22:47:01,832][105620] Updated weights for policy 1, policy_version 1018053 (0.0010) [2023-12-26 22:47:02,381][105692] Updated weights for policy 0, policy_version 1017583 (0.0008) [2023-12-26 22:47:02,436][105692] Updated weights for policy 0, policy_version 1017593 (0.0010) [2023-12-26 22:47:02,483][105692] Updated weights for policy 0, policy_version 1017603 (0.0006) [2023-12-26 22:47:02,582][105620] Updated weights for policy 1, policy_version 1018063 (0.0011) [2023-12-26 22:47:02,641][105620] Updated weights for policy 1, policy_version 1018073 (0.0011) [2023-12-26 22:47:02,699][105620] Updated weights for policy 1, policy_version 1018083 (0.0010) [2023-12-26 22:47:03,199][105692] Updated weights for policy 0, policy_version 1017613 (0.0007) [2023-12-26 22:47:03,249][105692] Updated weights for policy 0, policy_version 1017623 (0.0009) [2023-12-26 22:47:03,303][105692] Updated weights for policy 0, policy_version 1017633 (0.0009) [2023-12-26 22:47:03,390][105620] Updated weights for policy 1, policy_version 1018093 (0.0008) [2023-12-26 22:47:03,448][105620] Updated weights for policy 1, policy_version 1018103 (0.0005) [2023-12-26 22:47:03,518][105620] Updated weights for policy 1, policy_version 1018113 (0.0005) [2023-12-26 22:47:03,994][105692] Updated weights for policy 0, policy_version 1017643 (0.0008) [2023-12-26 22:47:04,024][105620] Updated weights for policy 1, policy_version 1018123 (0.0007) [2023-12-26 22:47:04,050][105692] Updated weights for policy 0, policy_version 1017653 (0.0007) [2023-12-26 22:47:04,080][105620] Updated weights for policy 1, policy_version 1018133 (0.0008) [2023-12-26 22:47:04,103][105692] Updated weights for policy 0, policy_version 1017663 (0.0007) [2023-12-26 22:47:04,136][105620] Updated weights for policy 1, policy_version 1018143 (0.0006) [2023-12-26 22:47:04,787][105692] Updated weights for policy 0, policy_version 1017673 (0.0009) [2023-12-26 22:47:04,830][105620] Updated weights for policy 1, policy_version 1018153 (0.0006) [2023-12-26 22:47:04,843][105692] Updated weights for policy 0, policy_version 1017684 (0.0010) [2023-12-26 22:47:04,890][105620] Updated weights for policy 1, policy_version 1018163 (0.0005) [2023-12-26 22:47:04,897][105692] Updated weights for policy 0, policy_version 1017694 (0.0008) [2023-12-26 22:47:04,945][105620] Updated weights for policy 1, policy_version 1018173 (0.0008) [2023-12-26 22:47:04,948][105692] Updated weights for policy 0, policy_version 1017704 (0.0007) [2023-12-26 22:47:05,001][105620] Updated weights for policy 1, policy_version 1018183 (0.0010) [2023-12-26 22:47:05,701][105620] Updated weights for policy 1, policy_version 1018193 (0.0011) [2023-12-26 22:47:05,734][105692] Updated weights for policy 0, policy_version 1017714 (0.0006) [2023-12-26 22:47:05,755][105620] Updated weights for policy 1, policy_version 1018203 (0.0010) [2023-12-26 22:47:05,779][105692] Updated weights for policy 0, policy_version 1017724 (0.0008) [2023-12-26 22:47:05,803][105620] Updated weights for policy 1, policy_version 1018213 (0.0010) [2023-12-26 22:47:05,831][105692] Updated weights for policy 0, policy_version 1017734 (0.0006) [2023-12-26 22:47:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 521273344. Throughput: 0: 9772.9, 1: 9552.2. Samples: 521259488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:47:06,063][104569] Avg episode reward: [(0, '8817.842'), (1, '9353.320')] [2023-12-26 22:47:06,533][105620] Updated weights for policy 1, policy_version 1018223 (0.0010) [2023-12-26 22:47:06,593][105620] Updated weights for policy 1, policy_version 1018233 (0.0011) [2023-12-26 22:47:06,625][105692] Updated weights for policy 0, policy_version 1017744 (0.0006) [2023-12-26 22:47:06,650][105620] Updated weights for policy 1, policy_version 1018243 (0.0011) [2023-12-26 22:47:06,688][105692] Updated weights for policy 0, policy_version 1017754 (0.0006) [2023-12-26 22:47:06,744][105692] Updated weights for policy 0, policy_version 1017764 (0.0008) [2023-12-26 22:47:07,350][105692] Updated weights for policy 0, policy_version 1017774 (0.0008) [2023-12-26 22:47:07,409][105692] Updated weights for policy 0, policy_version 1017784 (0.0007) [2023-12-26 22:47:07,411][105620] Updated weights for policy 1, policy_version 1018253 (0.0011) [2023-12-26 22:47:07,470][105620] Updated weights for policy 1, policy_version 1018263 (0.0011) [2023-12-26 22:47:07,471][105692] Updated weights for policy 0, policy_version 1017794 (0.0007) [2023-12-26 22:47:07,531][105620] Updated weights for policy 1, policy_version 1018273 (0.0010) [2023-12-26 22:47:08,094][105692] Updated weights for policy 0, policy_version 1017804 (0.0007) [2023-12-26 22:47:08,152][105692] Updated weights for policy 0, policy_version 1017814 (0.0007) [2023-12-26 22:47:08,203][105692] Updated weights for policy 0, policy_version 1017824 (0.0006) [2023-12-26 22:47:08,243][105620] Updated weights for policy 1, policy_version 1018283 (0.0007) [2023-12-26 22:47:08,296][105620] Updated weights for policy 1, policy_version 1018293 (0.0010) [2023-12-26 22:47:08,359][105620] Updated weights for policy 1, policy_version 1018303 (0.0011) [2023-12-26 22:47:08,913][105692] Updated weights for policy 0, policy_version 1017834 (0.0008) [2023-12-26 22:47:08,979][105692] Updated weights for policy 0, policy_version 1017844 (0.0005) [2023-12-26 22:47:09,046][105692] Updated weights for policy 0, policy_version 1017854 (0.0006) [2023-12-26 22:47:09,091][105620] Updated weights for policy 1, policy_version 1018313 (0.0009) [2023-12-26 22:47:09,118][105692] Updated weights for policy 0, policy_version 1017864 (0.0006) [2023-12-26 22:47:09,149][105620] Updated weights for policy 1, policy_version 1018323 (0.0006) [2023-12-26 22:47:09,213][105620] Updated weights for policy 1, policy_version 1018333 (0.0010) [2023-12-26 22:47:09,272][105620] Updated weights for policy 1, policy_version 1018343 (0.0008) [2023-12-26 22:47:09,682][105692] Updated weights for policy 0, policy_version 1017874 (0.0006) [2023-12-26 22:47:09,736][105692] Updated weights for policy 0, policy_version 1017884 (0.0010) [2023-12-26 22:47:09,810][105692] Updated weights for policy 0, policy_version 1017894 (0.0009) [2023-12-26 22:47:09,916][105620] Updated weights for policy 1, policy_version 1018353 (0.0006) [2023-12-26 22:47:09,974][105620] Updated weights for policy 1, policy_version 1018363 (0.0009) [2023-12-26 22:47:10,035][105620] Updated weights for policy 1, policy_version 1018373 (0.0008) [2023-12-26 22:47:10,533][105692] Updated weights for policy 0, policy_version 1017904 (0.0010) [2023-12-26 22:47:10,593][105692] Updated weights for policy 0, policy_version 1017914 (0.0011) [2023-12-26 22:47:10,659][105692] Updated weights for policy 0, policy_version 1017924 (0.0011) [2023-12-26 22:47:10,763][105620] Updated weights for policy 1, policy_version 1018383 (0.0006) [2023-12-26 22:47:10,820][105620] Updated weights for policy 1, policy_version 1018393 (0.0005) [2023-12-26 22:47:10,886][105620] Updated weights for policy 1, policy_version 1018403 (0.0008) [2023-12-26 22:47:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 521371648. Throughput: 0: 9887.0, 1: 9597.4. Samples: 521377724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:47:11,063][104569] Avg episode reward: [(0, '9175.996'), (1, '9259.211')] [2023-12-26 22:47:11,415][105692] Updated weights for policy 0, policy_version 1017934 (0.0009) [2023-12-26 22:47:11,467][105692] Updated weights for policy 0, policy_version 1017944 (0.0009) [2023-12-26 22:47:11,518][105692] Updated weights for policy 0, policy_version 1017954 (0.0009) [2023-12-26 22:47:11,541][105620] Updated weights for policy 1, policy_version 1018413 (0.0007) [2023-12-26 22:47:11,606][105620] Updated weights for policy 1, policy_version 1018423 (0.0008) [2023-12-26 22:47:11,676][105620] Updated weights for policy 1, policy_version 1018433 (0.0009) [2023-12-26 22:47:12,274][105692] Updated weights for policy 0, policy_version 1017964 (0.0008) [2023-12-26 22:47:12,332][105692] Updated weights for policy 0, policy_version 1017974 (0.0009) [2023-12-26 22:47:12,390][105692] Updated weights for policy 0, policy_version 1017984 (0.0009) [2023-12-26 22:47:12,460][105620] Updated weights for policy 1, policy_version 1018443 (0.0009) [2023-12-26 22:47:12,523][105620] Updated weights for policy 1, policy_version 1018453 (0.0010) [2023-12-26 22:47:12,581][105620] Updated weights for policy 1, policy_version 1018463 (0.0010) [2023-12-26 22:47:13,163][105692] Updated weights for policy 0, policy_version 1017994 (0.0007) [2023-12-26 22:47:13,217][105692] Updated weights for policy 0, policy_version 1018004 (0.0006) [2023-12-26 22:47:13,232][105620] Updated weights for policy 1, policy_version 1018473 (0.0010) [2023-12-26 22:47:13,266][105692] Updated weights for policy 0, policy_version 1018014 (0.0005) [2023-12-26 22:47:13,294][105620] Updated weights for policy 1, policy_version 1018483 (0.0008) [2023-12-26 22:47:13,331][105692] Updated weights for policy 0, policy_version 1018024 (0.0005) [2023-12-26 22:47:13,358][105620] Updated weights for policy 1, policy_version 1018493 (0.0007) [2023-12-26 22:47:13,430][105620] Updated weights for policy 1, policy_version 1018503 (0.0007) [2023-12-26 22:47:14,042][105620] Updated weights for policy 1, policy_version 1018513 (0.0008) [2023-12-26 22:47:14,047][105692] Updated weights for policy 0, policy_version 1018034 (0.0007) [2023-12-26 22:47:14,090][105620] Updated weights for policy 1, policy_version 1018523 (0.0006) [2023-12-26 22:47:14,107][105692] Updated weights for policy 0, policy_version 1018044 (0.0009) [2023-12-26 22:47:14,143][105620] Updated weights for policy 1, policy_version 1018533 (0.0007) [2023-12-26 22:47:14,170][105692] Updated weights for policy 0, policy_version 1018054 (0.0008) [2023-12-26 22:47:14,874][105692] Updated weights for policy 0, policy_version 1018064 (0.0009) [2023-12-26 22:47:14,925][105620] Updated weights for policy 1, policy_version 1018543 (0.0008) [2023-12-26 22:47:14,931][105692] Updated weights for policy 0, policy_version 1018074 (0.0007) [2023-12-26 22:47:14,986][105620] Updated weights for policy 1, policy_version 1018553 (0.0007) [2023-12-26 22:47:14,993][105692] Updated weights for policy 0, policy_version 1018084 (0.0007) [2023-12-26 22:47:15,051][105620] Updated weights for policy 1, policy_version 1018563 (0.0009) [2023-12-26 22:47:15,683][105620] Updated weights for policy 1, policy_version 1018573 (0.0008) [2023-12-26 22:47:15,737][105620] Updated weights for policy 1, policy_version 1018583 (0.0005) [2023-12-26 22:47:15,776][105692] Updated weights for policy 0, policy_version 1018094 (0.0007) [2023-12-26 22:47:15,788][105620] Updated weights for policy 1, policy_version 1018593 (0.0005) [2023-12-26 22:47:15,832][105692] Updated weights for policy 0, policy_version 1018104 (0.0008) [2023-12-26 22:47:15,887][105692] Updated weights for policy 0, policy_version 1018114 (0.0010) [2023-12-26 22:47:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 521469952. Throughput: 0: 9787.9, 1: 9684.2. Samples: 521435688. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:47:16,062][104569] Avg episode reward: [(0, '9174.219'), (1, '9349.917')] [2023-12-26 22:47:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001018600_260792320.pth... [2023-12-26 22:47:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001018120_260677632.pth... [2023-12-26 22:47:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001017448_260497408.pth [2023-12-26 22:47:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001016968_260382720.pth [2023-12-26 22:47:16,334][105620] Updated weights for policy 1, policy_version 1018603 (0.0005) [2023-12-26 22:47:16,380][105620] Updated weights for policy 1, policy_version 1018613 (0.0010) [2023-12-26 22:47:16,431][105620] Updated weights for policy 1, policy_version 1018623 (0.0010) [2023-12-26 22:47:16,481][105692] Updated weights for policy 0, policy_version 1018124 (0.0006) [2023-12-26 22:47:16,545][105692] Updated weights for policy 0, policy_version 1018134 (0.0008) [2023-12-26 22:47:16,600][105692] Updated weights for policy 0, policy_version 1018144 (0.0007) [2023-12-26 22:47:17,129][105620] Updated weights for policy 1, policy_version 1018633 (0.0010) [2023-12-26 22:47:17,132][105692] Updated weights for policy 0, policy_version 1018154 (0.0006) [2023-12-26 22:47:17,189][105620] Updated weights for policy 1, policy_version 1018643 (0.0005) [2023-12-26 22:47:17,196][105692] Updated weights for policy 0, policy_version 1018164 (0.0008) [2023-12-26 22:47:17,255][105620] Updated weights for policy 1, policy_version 1018653 (0.0006) [2023-12-26 22:47:17,262][105692] Updated weights for policy 0, policy_version 1018174 (0.0008) [2023-12-26 22:47:17,307][105692] Updated weights for policy 0, policy_version 1018184 (0.0008) [2023-12-26 22:47:17,313][105620] Updated weights for policy 1, policy_version 1018663 (0.0010) [2023-12-26 22:47:17,879][105620] Updated weights for policy 1, policy_version 1018673 (0.0005) [2023-12-26 22:47:17,934][105620] Updated weights for policy 1, policy_version 1018683 (0.0005) [2023-12-26 22:47:17,998][105620] Updated weights for policy 1, policy_version 1018693 (0.0008) [2023-12-26 22:47:18,114][105692] Updated weights for policy 0, policy_version 1018194 (0.0010) [2023-12-26 22:47:18,166][105692] Updated weights for policy 0, policy_version 1018204 (0.0010) [2023-12-26 22:47:18,224][105692] Updated weights for policy 0, policy_version 1018214 (0.0010) [2023-12-26 22:47:18,607][105620] Updated weights for policy 1, policy_version 1018703 (0.0009) [2023-12-26 22:47:18,653][105620] Updated weights for policy 1, policy_version 1018713 (0.0010) [2023-12-26 22:47:18,711][105620] Updated weights for policy 1, policy_version 1018723 (0.0009) [2023-12-26 22:47:19,007][105692] Updated weights for policy 0, policy_version 1018224 (0.0008) [2023-12-26 22:47:19,071][105692] Updated weights for policy 0, policy_version 1018234 (0.0007) [2023-12-26 22:47:19,127][105692] Updated weights for policy 0, policy_version 1018244 (0.0006) [2023-12-26 22:47:19,454][105620] Updated weights for policy 1, policy_version 1018733 (0.0008) [2023-12-26 22:47:19,522][105620] Updated weights for policy 1, policy_version 1018743 (0.0006) [2023-12-26 22:47:19,587][105620] Updated weights for policy 1, policy_version 1018753 (0.0007) [2023-12-26 22:47:19,936][105692] Updated weights for policy 0, policy_version 1018254 (0.0009) [2023-12-26 22:47:20,000][105692] Updated weights for policy 0, policy_version 1018264 (0.0009) [2023-12-26 22:47:20,053][105692] Updated weights for policy 0, policy_version 1018274 (0.0010) [2023-12-26 22:47:20,175][105620] Updated weights for policy 1, policy_version 1018763 (0.0007) [2023-12-26 22:47:20,241][105620] Updated weights for policy 1, policy_version 1018773 (0.0009) [2023-12-26 22:47:20,303][105620] Updated weights for policy 1, policy_version 1018783 (0.0009) [2023-12-26 22:47:20,844][105692] Updated weights for policy 0, policy_version 1018284 (0.0009) [2023-12-26 22:47:20,920][105692] Updated weights for policy 0, policy_version 1018294 (0.0009) [2023-12-26 22:47:20,984][105620] Updated weights for policy 1, policy_version 1018793 (0.0007) [2023-12-26 22:47:20,994][105692] Updated weights for policy 0, policy_version 1018304 (0.0010) [2023-12-26 22:47:21,046][105620] Updated weights for policy 1, policy_version 1018803 (0.0008) [2023-12-26 22:47:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 521568256. Throughput: 0: 9780.1, 1: 9799.6. Samples: 521557204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:47:21,062][104569] Avg episode reward: [(0, '8725.240'), (1, '9350.367')] [2023-12-26 22:47:21,109][105620] Updated weights for policy 1, policy_version 1018813 (0.0008) [2023-12-26 22:47:21,176][105620] Updated weights for policy 1, policy_version 1018823 (0.0010) [2023-12-26 22:47:21,801][105692] Updated weights for policy 0, policy_version 1018314 (0.0009) [2023-12-26 22:47:21,835][105620] Updated weights for policy 1, policy_version 1018833 (0.0011) [2023-12-26 22:47:21,851][105692] Updated weights for policy 0, policy_version 1018324 (0.0010) [2023-12-26 22:47:21,888][105620] Updated weights for policy 1, policy_version 1018843 (0.0011) [2023-12-26 22:47:21,904][105692] Updated weights for policy 0, policy_version 1018334 (0.0007) [2023-12-26 22:47:21,954][105620] Updated weights for policy 1, policy_version 1018853 (0.0009) [2023-12-26 22:47:21,960][105692] Updated weights for policy 0, policy_version 1018344 (0.0008) [2023-12-26 22:47:22,662][105620] Updated weights for policy 1, policy_version 1018863 (0.0011) [2023-12-26 22:47:22,669][105692] Updated weights for policy 0, policy_version 1018354 (0.0011) [2023-12-26 22:47:22,719][105620] Updated weights for policy 1, policy_version 1018873 (0.0011) [2023-12-26 22:47:22,734][105692] Updated weights for policy 0, policy_version 1018364 (0.0011) [2023-12-26 22:47:22,777][105620] Updated weights for policy 1, policy_version 1018883 (0.0006) [2023-12-26 22:47:22,790][105692] Updated weights for policy 0, policy_version 1018374 (0.0011) [2023-12-26 22:47:23,385][105692] Updated weights for policy 0, policy_version 1018384 (0.0006) [2023-12-26 22:47:23,440][105692] Updated weights for policy 0, policy_version 1018394 (0.0010) [2023-12-26 22:47:23,494][105692] Updated weights for policy 0, policy_version 1018404 (0.0011) [2023-12-26 22:47:23,554][105620] Updated weights for policy 1, policy_version 1018893 (0.0007) [2023-12-26 22:47:23,603][105620] Updated weights for policy 1, policy_version 1018903 (0.0008) [2023-12-26 22:47:23,657][105620] Updated weights for policy 1, policy_version 1018913 (0.0009) [2023-12-26 22:47:24,190][105692] Updated weights for policy 0, policy_version 1018414 (0.0007) [2023-12-26 22:47:24,240][105692] Updated weights for policy 0, policy_version 1018424 (0.0005) [2023-12-26 22:47:24,293][105692] Updated weights for policy 0, policy_version 1018434 (0.0007) [2023-12-26 22:47:24,461][105620] Updated weights for policy 1, policy_version 1018923 (0.0008) [2023-12-26 22:47:24,518][105620] Updated weights for policy 1, policy_version 1018933 (0.0009) [2023-12-26 22:47:24,580][105620] Updated weights for policy 1, policy_version 1018943 (0.0008) [2023-12-26 22:47:24,923][105692] Updated weights for policy 0, policy_version 1018444 (0.0009) [2023-12-26 22:47:24,970][105692] Updated weights for policy 0, policy_version 1018454 (0.0010) [2023-12-26 22:47:25,022][105692] Updated weights for policy 0, policy_version 1018464 (0.0008) [2023-12-26 22:47:25,397][105620] Updated weights for policy 1, policy_version 1018953 (0.0008) [2023-12-26 22:47:25,462][105620] Updated weights for policy 1, policy_version 1018963 (0.0008) [2023-12-26 22:47:25,528][105620] Updated weights for policy 1, policy_version 1018973 (0.0008) [2023-12-26 22:47:25,593][105620] Updated weights for policy 1, policy_version 1018983 (0.0008) [2023-12-26 22:47:25,749][105692] Updated weights for policy 0, policy_version 1018474 (0.0009) [2023-12-26 22:47:25,810][105692] Updated weights for policy 0, policy_version 1018484 (0.0011) [2023-12-26 22:47:25,869][105692] Updated weights for policy 0, policy_version 1018494 (0.0010) [2023-12-26 22:47:25,920][105692] Updated weights for policy 0, policy_version 1018504 (0.0010) [2023-12-26 22:47:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 521666560. Throughput: 0: 9803.9, 1: 9871.6. Samples: 521673156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:47:26,062][104569] Avg episode reward: [(0, '8817.007'), (1, '9166.444')] [2023-12-26 22:47:26,195][105620] Updated weights for policy 1, policy_version 1018993 (0.0005) [2023-12-26 22:47:26,244][105620] Updated weights for policy 1, policy_version 1019003 (0.0005) [2023-12-26 22:47:26,311][105620] Updated weights for policy 1, policy_version 1019013 (0.0005) [2023-12-26 22:47:26,662][105692] Updated weights for policy 0, policy_version 1018514 (0.0010) [2023-12-26 22:47:26,713][105692] Updated weights for policy 0, policy_version 1018524 (0.0010) [2023-12-26 22:47:26,767][105692] Updated weights for policy 0, policy_version 1018534 (0.0010) [2023-12-26 22:47:26,856][105620] Updated weights for policy 1, policy_version 1019023 (0.0007) [2023-12-26 22:47:26,914][105620] Updated weights for policy 1, policy_version 1019033 (0.0008) [2023-12-26 22:47:26,966][105620] Updated weights for policy 1, policy_version 1019043 (0.0008) [2023-12-26 22:47:27,515][105692] Updated weights for policy 0, policy_version 1018544 (0.0010) [2023-12-26 22:47:27,572][105692] Updated weights for policy 0, policy_version 1018554 (0.0010) [2023-12-26 22:47:27,628][105692] Updated weights for policy 0, policy_version 1018564 (0.0010) [2023-12-26 22:47:27,712][105620] Updated weights for policy 1, policy_version 1019053 (0.0008) [2023-12-26 22:47:27,760][105620] Updated weights for policy 1, policy_version 1019063 (0.0007) [2023-12-26 22:47:27,820][105620] Updated weights for policy 1, policy_version 1019073 (0.0008) [2023-12-26 22:47:28,371][105692] Updated weights for policy 0, policy_version 1018574 (0.0011) [2023-12-26 22:47:28,424][105692] Updated weights for policy 0, policy_version 1018584 (0.0006) [2023-12-26 22:47:28,485][105692] Updated weights for policy 0, policy_version 1018594 (0.0006) [2023-12-26 22:47:28,531][105620] Updated weights for policy 1, policy_version 1019083 (0.0007) [2023-12-26 22:47:28,602][105620] Updated weights for policy 1, policy_version 1019093 (0.0008) [2023-12-26 22:47:28,669][105620] Updated weights for policy 1, policy_version 1019103 (0.0007) [2023-12-26 22:47:29,082][105692] Updated weights for policy 0, policy_version 1018604 (0.0005) [2023-12-26 22:47:29,139][105692] Updated weights for policy 0, policy_version 1018614 (0.0005) [2023-12-26 22:47:29,192][105692] Updated weights for policy 0, policy_version 1018624 (0.0005) [2023-12-26 22:47:29,265][105620] Updated weights for policy 1, policy_version 1019113 (0.0005) [2023-12-26 22:47:29,331][105620] Updated weights for policy 1, policy_version 1019123 (0.0007) [2023-12-26 22:47:29,391][105620] Updated weights for policy 1, policy_version 1019133 (0.0008) [2023-12-26 22:47:29,453][105620] Updated weights for policy 1, policy_version 1019143 (0.0008) [2023-12-26 22:47:29,870][105692] Updated weights for policy 0, policy_version 1018634 (0.0009) [2023-12-26 22:47:29,937][105692] Updated weights for policy 0, policy_version 1018644 (0.0011) [2023-12-26 22:47:29,992][105692] Updated weights for policy 0, policy_version 1018654 (0.0010) [2023-12-26 22:47:30,057][105692] Updated weights for policy 0, policy_version 1018664 (0.0010) [2023-12-26 22:47:30,189][105620] Updated weights for policy 1, policy_version 1019153 (0.0008) [2023-12-26 22:47:30,240][105620] Updated weights for policy 1, policy_version 1019163 (0.0008) [2023-12-26 22:47:30,295][105620] Updated weights for policy 1, policy_version 1019173 (0.0008) [2023-12-26 22:47:30,757][105692] Updated weights for policy 0, policy_version 1018674 (0.0010) [2023-12-26 22:47:30,824][105692] Updated weights for policy 0, policy_version 1018684 (0.0010) [2023-12-26 22:47:30,890][105692] Updated weights for policy 0, policy_version 1018694 (0.0008) [2023-12-26 22:47:30,949][105620] Updated weights for policy 1, policy_version 1019183 (0.0006) [2023-12-26 22:47:31,003][105620] Updated weights for policy 1, policy_version 1019193 (0.0005) [2023-12-26 22:47:31,062][105620] Updated weights for policy 1, policy_version 1019203 (0.0007) [2023-12-26 22:47:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 521764864. Throughput: 0: 9847.0, 1: 9928.6. Samples: 521733968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:47:31,062][104569] Avg episode reward: [(0, '9176.470'), (1, '9075.314')] [2023-12-26 22:47:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001018696_260825088.pth... [2023-12-26 22:47:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001017544_260530176.pth [2023-12-26 22:47:31,085][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001019208_260947968.pth... [2023-12-26 22:47:31,090][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001018024_260644864.pth [2023-12-26 22:47:31,562][105692] Updated weights for policy 0, policy_version 1018704 (0.0009) [2023-12-26 22:47:31,607][105692] Updated weights for policy 0, policy_version 1018714 (0.0010) [2023-12-26 22:47:31,673][105692] Updated weights for policy 0, policy_version 1018724 (0.0011) [2023-12-26 22:47:31,796][105620] Updated weights for policy 1, policy_version 1019213 (0.0008) [2023-12-26 22:47:31,858][105620] Updated weights for policy 1, policy_version 1019223 (0.0010) [2023-12-26 22:47:31,920][105620] Updated weights for policy 1, policy_version 1019233 (0.0010) [2023-12-26 22:47:32,382][105692] Updated weights for policy 0, policy_version 1018734 (0.0009) [2023-12-26 22:47:32,433][105692] Updated weights for policy 0, policy_version 1018744 (0.0008) [2023-12-26 22:47:32,493][105692] Updated weights for policy 0, policy_version 1018754 (0.0008) [2023-12-26 22:47:32,632][105620] Updated weights for policy 1, policy_version 1019243 (0.0009) [2023-12-26 22:47:32,684][105620] Updated weights for policy 1, policy_version 1019253 (0.0009) [2023-12-26 22:47:32,739][105620] Updated weights for policy 1, policy_version 1019263 (0.0007) [2023-12-26 22:47:33,199][105692] Updated weights for policy 0, policy_version 1018764 (0.0010) [2023-12-26 22:47:33,252][105692] Updated weights for policy 0, policy_version 1018774 (0.0009) [2023-12-26 22:47:33,306][105692] Updated weights for policy 0, policy_version 1018784 (0.0008) [2023-12-26 22:47:33,354][105620] Updated weights for policy 1, policy_version 1019273 (0.0008) [2023-12-26 22:47:33,403][105620] Updated weights for policy 1, policy_version 1019283 (0.0005) [2023-12-26 22:47:33,448][105620] Updated weights for policy 1, policy_version 1019293 (0.0005) [2023-12-26 22:47:33,495][105620] Updated weights for policy 1, policy_version 1019303 (0.0007) [2023-12-26 22:47:34,066][105692] Updated weights for policy 0, policy_version 1018794 (0.0009) [2023-12-26 22:47:34,120][105692] Updated weights for policy 0, policy_version 1018804 (0.0005) [2023-12-26 22:47:34,140][105620] Updated weights for policy 1, policy_version 1019313 (0.0008) [2023-12-26 22:47:34,182][105692] Updated weights for policy 0, policy_version 1018814 (0.0008) [2023-12-26 22:47:34,207][105620] Updated weights for policy 1, policy_version 1019323 (0.0006) [2023-12-26 22:47:34,245][105692] Updated weights for policy 0, policy_version 1018824 (0.0008) [2023-12-26 22:47:34,267][105620] Updated weights for policy 1, policy_version 1019333 (0.0006) [2023-12-26 22:47:34,903][105692] Updated weights for policy 0, policy_version 1018834 (0.0005) [2023-12-26 22:47:34,957][105692] Updated weights for policy 0, policy_version 1018844 (0.0005) [2023-12-26 22:47:35,011][105692] Updated weights for policy 0, policy_version 1018854 (0.0006) [2023-12-26 22:47:35,041][105620] Updated weights for policy 1, policy_version 1019343 (0.0009) [2023-12-26 22:47:35,099][105620] Updated weights for policy 1, policy_version 1019354 (0.0010) [2023-12-26 22:47:35,147][105620] Updated weights for policy 1, policy_version 1019364 (0.0008) [2023-12-26 22:47:35,561][105692] Updated weights for policy 0, policy_version 1018864 (0.0005) [2023-12-26 22:47:35,623][105692] Updated weights for policy 0, policy_version 1018874 (0.0008) [2023-12-26 22:47:35,682][105692] Updated weights for policy 0, policy_version 1018884 (0.0007) [2023-12-26 22:47:36,019][105620] Updated weights for policy 1, policy_version 1019374 (0.0008) [2023-12-26 22:47:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 521863168. Throughput: 0: 9935.5, 1: 9966.5. Samples: 521854280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:47:36,063][104569] Avg episode reward: [(0, '9087.293'), (1, '9179.916')] [2023-12-26 22:47:36,075][105620] Updated weights for policy 1, policy_version 1019384 (0.0008) [2023-12-26 22:47:36,136][105620] Updated weights for policy 1, policy_version 1019394 (0.0007) [2023-12-26 22:47:36,306][105692] Updated weights for policy 0, policy_version 1018894 (0.0009) [2023-12-26 22:47:36,366][105692] Updated weights for policy 0, policy_version 1018904 (0.0011) [2023-12-26 22:47:36,428][105692] Updated weights for policy 0, policy_version 1018914 (0.0011) [2023-12-26 22:47:36,874][105620] Updated weights for policy 1, policy_version 1019404 (0.0009) [2023-12-26 22:47:36,929][105620] Updated weights for policy 1, policy_version 1019414 (0.0007) [2023-12-26 22:47:36,981][105620] Updated weights for policy 1, policy_version 1019424 (0.0008) [2023-12-26 22:47:37,163][105692] Updated weights for policy 0, policy_version 1018924 (0.0009) [2023-12-26 22:47:37,212][105692] Updated weights for policy 0, policy_version 1018934 (0.0009) [2023-12-26 22:47:37,267][105692] Updated weights for policy 0, policy_version 1018944 (0.0010) [2023-12-26 22:47:37,767][105620] Updated weights for policy 1, policy_version 1019434 (0.0009) [2023-12-26 22:47:37,822][105620] Updated weights for policy 1, policy_version 1019444 (0.0008) [2023-12-26 22:47:37,881][105620] Updated weights for policy 1, policy_version 1019454 (0.0008) [2023-12-26 22:47:37,940][105620] Updated weights for policy 1, policy_version 1019464 (0.0008) [2023-12-26 22:47:38,000][105692] Updated weights for policy 0, policy_version 1018954 (0.0010) [2023-12-26 22:47:38,055][105692] Updated weights for policy 0, policy_version 1018964 (0.0010) [2023-12-26 22:47:38,110][105692] Updated weights for policy 0, policy_version 1018974 (0.0011) [2023-12-26 22:47:38,167][105692] Updated weights for policy 0, policy_version 1018984 (0.0010) [2023-12-26 22:47:38,763][105620] Updated weights for policy 1, policy_version 1019474 (0.0009) [2023-12-26 22:47:38,794][105692] Updated weights for policy 0, policy_version 1018994 (0.0008) [2023-12-26 22:47:38,819][105620] Updated weights for policy 1, policy_version 1019484 (0.0008) [2023-12-26 22:47:38,858][105692] Updated weights for policy 0, policy_version 1019004 (0.0008) [2023-12-26 22:47:38,876][105620] Updated weights for policy 1, policy_version 1019494 (0.0007) [2023-12-26 22:47:38,917][105692] Updated weights for policy 0, policy_version 1019014 (0.0011) [2023-12-26 22:47:39,524][105692] Updated weights for policy 0, policy_version 1019024 (0.0009) [2023-12-26 22:47:39,587][105692] Updated weights for policy 0, policy_version 1019034 (0.0010) [2023-12-26 22:47:39,655][105692] Updated weights for policy 0, policy_version 1019044 (0.0008) [2023-12-26 22:47:39,712][105620] Updated weights for policy 1, policy_version 1019504 (0.0008) [2023-12-26 22:47:39,780][105620] Updated weights for policy 1, policy_version 1019514 (0.0008) [2023-12-26 22:47:39,849][105620] Updated weights for policy 1, policy_version 1019524 (0.0009) [2023-12-26 22:47:40,500][105620] Updated weights for policy 1, policy_version 1019534 (0.0009) [2023-12-26 22:47:40,510][105692] Updated weights for policy 0, policy_version 1019054 (0.0009) [2023-12-26 22:47:40,554][105620] Updated weights for policy 1, policy_version 1019544 (0.0007) [2023-12-26 22:47:40,569][105692] Updated weights for policy 0, policy_version 1019064 (0.0008) [2023-12-26 22:47:40,609][105620] Updated weights for policy 1, policy_version 1019554 (0.0006) [2023-12-26 22:47:40,610][105585] KL-divergence is very high: 111.8813 [2023-12-26 22:47:40,624][105692] Updated weights for policy 0, policy_version 1019074 (0.0006) [2023-12-26 22:47:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 521961472. Throughput: 0: 9920.8, 1: 9888.2. Samples: 521969948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:47:41,063][104569] Avg episode reward: [(0, '9177.607'), (1, '9179.577')] [2023-12-26 22:47:41,331][105620] Updated weights for policy 1, policy_version 1019564 (0.0007) [2023-12-26 22:47:41,402][105620] Updated weights for policy 1, policy_version 1019574 (0.0008) [2023-12-26 22:47:41,441][105692] Updated weights for policy 0, policy_version 1019084 (0.0009) [2023-12-26 22:47:41,463][105620] Updated weights for policy 1, policy_version 1019584 (0.0007) [2023-12-26 22:47:41,502][105692] Updated weights for policy 0, policy_version 1019094 (0.0011) [2023-12-26 22:47:41,565][105692] Updated weights for policy 0, policy_version 1019104 (0.0011) [2023-12-26 22:47:42,269][105620] Updated weights for policy 1, policy_version 1019594 (0.0006) [2023-12-26 22:47:42,290][105692] Updated weights for policy 0, policy_version 1019114 (0.0010) [2023-12-26 22:47:42,335][105620] Updated weights for policy 1, policy_version 1019604 (0.0009) [2023-12-26 22:47:42,357][105692] Updated weights for policy 0, policy_version 1019124 (0.0007) [2023-12-26 22:47:42,399][105620] Updated weights for policy 1, policy_version 1019614 (0.0010) [2023-12-26 22:47:42,410][105692] Updated weights for policy 0, policy_version 1019134 (0.0007) [2023-12-26 22:47:42,456][105620] Updated weights for policy 1, policy_version 1019624 (0.0008) [2023-12-26 22:47:42,467][105692] Updated weights for policy 0, policy_version 1019144 (0.0007) [2023-12-26 22:47:43,142][105620] Updated weights for policy 1, policy_version 1019634 (0.0009) [2023-12-26 22:47:43,201][105620] Updated weights for policy 1, policy_version 1019644 (0.0007) [2023-12-26 22:47:43,224][105692] Updated weights for policy 0, policy_version 1019154 (0.0007) [2023-12-26 22:47:43,256][105620] Updated weights for policy 1, policy_version 1019654 (0.0007) [2023-12-26 22:47:43,266][105692] Updated weights for policy 0, policy_version 1019164 (0.0007) [2023-12-26 22:47:43,314][105692] Updated weights for policy 0, policy_version 1019174 (0.0008) [2023-12-26 22:47:43,924][105692] Updated weights for policy 0, policy_version 1019184 (0.0006) [2023-12-26 22:47:43,983][105620] Updated weights for policy 1, policy_version 1019664 (0.0006) [2023-12-26 22:47:43,987][105692] Updated weights for policy 0, policy_version 1019194 (0.0006) [2023-12-26 22:47:44,042][105692] Updated weights for policy 0, policy_version 1019204 (0.0006) [2023-12-26 22:47:44,049][105620] Updated weights for policy 1, policy_version 1019674 (0.0006) [2023-12-26 22:47:44,107][105620] Updated weights for policy 1, policy_version 1019684 (0.0007) [2023-12-26 22:47:44,678][105692] Updated weights for policy 0, policy_version 1019214 (0.0007) [2023-12-26 22:47:44,741][105692] Updated weights for policy 0, policy_version 1019224 (0.0007) [2023-12-26 22:47:44,804][105692] Updated weights for policy 0, policy_version 1019234 (0.0009) [2023-12-26 22:47:44,822][105620] Updated weights for policy 1, policy_version 1019694 (0.0008) [2023-12-26 22:47:44,883][105620] Updated weights for policy 1, policy_version 1019704 (0.0006) [2023-12-26 22:47:44,947][105620] Updated weights for policy 1, policy_version 1019714 (0.0005) [2023-12-26 22:47:45,508][105692] Updated weights for policy 0, policy_version 1019244 (0.0008) [2023-12-26 22:47:45,567][105692] Updated weights for policy 0, policy_version 1019254 (0.0009) [2023-12-26 22:47:45,627][105692] Updated weights for policy 0, policy_version 1019264 (0.0009) [2023-12-26 22:47:45,677][105620] Updated weights for policy 1, policy_version 1019724 (0.0008) [2023-12-26 22:47:45,740][105620] Updated weights for policy 1, policy_version 1019734 (0.0009) [2023-12-26 22:47:45,798][105620] Updated weights for policy 1, policy_version 1019744 (0.0009) [2023-12-26 22:47:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 522059776. Throughput: 0: 9827.6, 1: 9832.1. Samples: 522025968. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:47:46,063][104569] Avg episode reward: [(0, '9090.128'), (1, '9179.576')] [2023-12-26 22:47:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001019752_261087232.pth... [2023-12-26 22:47:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001019272_260972544.pth... [2023-12-26 22:47:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001018600_260792320.pth [2023-12-26 22:47:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001018120_260677632.pth [2023-12-26 22:47:46,236][105692] Updated weights for policy 0, policy_version 1019274 (0.0008) [2023-12-26 22:47:46,290][105692] Updated weights for policy 0, policy_version 1019284 (0.0008) [2023-12-26 22:47:46,339][105692] Updated weights for policy 0, policy_version 1019294 (0.0007) [2023-12-26 22:47:46,389][105692] Updated weights for policy 0, policy_version 1019304 (0.0005) [2023-12-26 22:47:46,652][105620] Updated weights for policy 1, policy_version 1019754 (0.0009) [2023-12-26 22:47:46,707][105620] Updated weights for policy 1, policy_version 1019764 (0.0006) [2023-12-26 22:47:46,760][105620] Updated weights for policy 1, policy_version 1019774 (0.0005) [2023-12-26 22:47:46,816][105620] Updated weights for policy 1, policy_version 1019784 (0.0008) [2023-12-26 22:47:46,965][105692] Updated weights for policy 0, policy_version 1019314 (0.0007) [2023-12-26 22:47:47,018][105692] Updated weights for policy 0, policy_version 1019324 (0.0006) [2023-12-26 22:47:47,083][105692] Updated weights for policy 0, policy_version 1019334 (0.0005) [2023-12-26 22:47:47,489][105620] Updated weights for policy 1, policy_version 1019794 (0.0005) [2023-12-26 22:47:47,535][105620] Updated weights for policy 1, policy_version 1019804 (0.0005) [2023-12-26 22:47:47,594][105620] Updated weights for policy 1, policy_version 1019814 (0.0005) [2023-12-26 22:47:47,674][105692] Updated weights for policy 0, policy_version 1019344 (0.0007) [2023-12-26 22:47:47,722][105692] Updated weights for policy 0, policy_version 1019354 (0.0007) [2023-12-26 22:47:47,788][105692] Updated weights for policy 0, policy_version 1019364 (0.0008) [2023-12-26 22:47:48,296][105620] Updated weights for policy 1, policy_version 1019824 (0.0009) [2023-12-26 22:47:48,380][105620] Updated weights for policy 1, policy_version 1019834 (0.0010) [2023-12-26 22:47:48,433][105692] Updated weights for policy 0, policy_version 1019374 (0.0006) [2023-12-26 22:47:48,447][105620] Updated weights for policy 1, policy_version 1019844 (0.0011) [2023-12-26 22:47:48,491][105692] Updated weights for policy 0, policy_version 1019384 (0.0008) [2023-12-26 22:47:48,537][105692] Updated weights for policy 0, policy_version 1019394 (0.0009) [2023-12-26 22:47:49,050][105620] Updated weights for policy 1, policy_version 1019854 (0.0008) [2023-12-26 22:47:49,105][105620] Updated weights for policy 1, policy_version 1019864 (0.0005) [2023-12-26 22:47:49,157][105620] Updated weights for policy 1, policy_version 1019874 (0.0005) [2023-12-26 22:47:49,424][105692] Updated weights for policy 0, policy_version 1019404 (0.0008) [2023-12-26 22:47:49,489][105692] Updated weights for policy 0, policy_version 1019414 (0.0008) [2023-12-26 22:47:49,554][105692] Updated weights for policy 0, policy_version 1019424 (0.0008) [2023-12-26 22:47:49,780][105620] Updated weights for policy 1, policy_version 1019884 (0.0005) [2023-12-26 22:47:49,842][105620] Updated weights for policy 1, policy_version 1019894 (0.0007) [2023-12-26 22:47:49,906][105620] Updated weights for policy 1, policy_version 1019904 (0.0008) [2023-12-26 22:47:50,306][105692] Updated weights for policy 0, policy_version 1019434 (0.0007) [2023-12-26 22:47:50,364][105692] Updated weights for policy 0, policy_version 1019444 (0.0005) [2023-12-26 22:47:50,430][105692] Updated weights for policy 0, policy_version 1019454 (0.0008) [2023-12-26 22:47:50,485][105692] Updated weights for policy 0, policy_version 1019464 (0.0011) [2023-12-26 22:47:50,684][105620] Updated weights for policy 1, policy_version 1019914 (0.0009) [2023-12-26 22:47:50,746][105620] Updated weights for policy 1, policy_version 1019924 (0.0008) [2023-12-26 22:47:50,805][105620] Updated weights for policy 1, policy_version 1019934 (0.0008) [2023-12-26 22:47:50,863][105620] Updated weights for policy 1, policy_version 1019944 (0.0008) [2023-12-26 22:47:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 522158080. Throughput: 0: 9912.0, 1: 9823.7. Samples: 522147592. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:47:51,062][104569] Avg episode reward: [(0, '8818.652'), (1, '9350.679')] [2023-12-26 22:47:51,208][105692] Updated weights for policy 0, policy_version 1019474 (0.0010) [2023-12-26 22:47:51,272][105692] Updated weights for policy 0, policy_version 1019484 (0.0007) [2023-12-26 22:47:51,344][105692] Updated weights for policy 0, policy_version 1019494 (0.0008) [2023-12-26 22:47:51,598][105620] Updated weights for policy 1, policy_version 1019954 (0.0011) [2023-12-26 22:47:51,662][105620] Updated weights for policy 1, policy_version 1019964 (0.0011) [2023-12-26 22:47:51,726][105620] Updated weights for policy 1, policy_version 1019974 (0.0010) [2023-12-26 22:47:51,934][105692] Updated weights for policy 0, policy_version 1019504 (0.0006) [2023-12-26 22:47:51,979][105692] Updated weights for policy 0, policy_version 1019514 (0.0005) [2023-12-26 22:47:52,047][105692] Updated weights for policy 0, policy_version 1019524 (0.0005) [2023-12-26 22:47:52,546][105620] Updated weights for policy 1, policy_version 1019984 (0.0008) [2023-12-26 22:47:52,594][105620] Updated weights for policy 1, policy_version 1019994 (0.0009) [2023-12-26 22:47:52,649][105620] Updated weights for policy 1, policy_version 1020004 (0.0009) [2023-12-26 22:47:52,681][105692] Updated weights for policy 0, policy_version 1019534 (0.0006) [2023-12-26 22:47:52,736][105692] Updated weights for policy 0, policy_version 1019544 (0.0009) [2023-12-26 22:47:52,789][105692] Updated weights for policy 0, policy_version 1019554 (0.0009) [2023-12-26 22:47:53,460][105620] Updated weights for policy 1, policy_version 1020014 (0.0009) [2023-12-26 22:47:53,481][105692] Updated weights for policy 0, policy_version 1019564 (0.0007) [2023-12-26 22:47:53,512][105620] Updated weights for policy 1, policy_version 1020024 (0.0009) [2023-12-26 22:47:53,532][105692] Updated weights for policy 0, policy_version 1019574 (0.0005) [2023-12-26 22:47:53,568][105620] Updated weights for policy 1, policy_version 1020034 (0.0009) [2023-12-26 22:47:53,578][105692] Updated weights for policy 0, policy_version 1019584 (0.0005) [2023-12-26 22:47:54,152][105692] Updated weights for policy 0, policy_version 1019594 (0.0006) [2023-12-26 22:47:54,204][105692] Updated weights for policy 0, policy_version 1019604 (0.0010) [2023-12-26 22:47:54,268][105692] Updated weights for policy 0, policy_version 1019614 (0.0010) [2023-12-26 22:47:54,327][105692] Updated weights for policy 0, policy_version 1019624 (0.0010) [2023-12-26 22:47:54,374][105620] Updated weights for policy 1, policy_version 1020044 (0.0010) [2023-12-26 22:47:54,429][105620] Updated weights for policy 1, policy_version 1020054 (0.0010) [2023-12-26 22:47:54,483][105620] Updated weights for policy 1, policy_version 1020064 (0.0010) [2023-12-26 22:47:55,028][105585] KL-divergence is very high: 123.4281 [2023-12-26 22:47:55,033][105692] Updated weights for policy 0, policy_version 1019634 (0.0010) [2023-12-26 22:47:55,070][105585] KL-divergence is very high: 137.1324 [2023-12-26 22:47:55,085][105692] Updated weights for policy 0, policy_version 1019644 (0.0010) [2023-12-26 22:47:55,109][105585] KL-divergence is very high: 129.8199 [2023-12-26 22:47:55,133][105692] Updated weights for policy 0, policy_version 1019654 (0.0010) [2023-12-26 22:47:55,265][105620] Updated weights for policy 1, policy_version 1020074 (0.0008) [2023-12-26 22:47:55,310][105620] Updated weights for policy 1, policy_version 1020084 (0.0006) [2023-12-26 22:47:55,359][105620] Updated weights for policy 1, policy_version 1020094 (0.0005) [2023-12-26 22:47:55,405][105620] Updated weights for policy 1, policy_version 1020104 (0.0007) [2023-12-26 22:47:55,880][105692] Updated weights for policy 0, policy_version 1019664 (0.0010) [2023-12-26 22:47:55,941][105692] Updated weights for policy 0, policy_version 1019674 (0.0010) [2023-12-26 22:47:55,989][105692] Updated weights for policy 0, policy_version 1019684 (0.0010) [2023-12-26 22:47:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 522256384. Throughput: 0: 9938.4, 1: 9744.6. Samples: 522263460. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:47:56,062][104569] Avg episode reward: [(0, '8816.919'), (1, '9351.060')] [2023-12-26 22:47:56,151][105620] Updated weights for policy 1, policy_version 1020114 (0.0010) [2023-12-26 22:47:56,205][105620] Updated weights for policy 1, policy_version 1020124 (0.0010) [2023-12-26 22:47:56,263][105620] Updated weights for policy 1, policy_version 1020134 (0.0008) [2023-12-26 22:47:56,778][105692] Updated weights for policy 0, policy_version 1019694 (0.0009) [2023-12-26 22:47:56,829][105692] Updated weights for policy 0, policy_version 1019704 (0.0008) [2023-12-26 22:47:56,877][105692] Updated weights for policy 0, policy_version 1019714 (0.0008) [2023-12-26 22:47:56,957][105620] Updated weights for policy 1, policy_version 1020144 (0.0010) [2023-12-26 22:47:56,998][105620] Updated weights for policy 1, policy_version 1020154 (0.0010) [2023-12-26 22:47:57,042][105620] Updated weights for policy 1, policy_version 1020164 (0.0010) [2023-12-26 22:47:57,627][105692] Updated weights for policy 0, policy_version 1019724 (0.0008) [2023-12-26 22:47:57,671][105692] Updated weights for policy 0, policy_version 1019734 (0.0008) [2023-12-26 22:47:57,716][105692] Updated weights for policy 0, policy_version 1019744 (0.0008) [2023-12-26 22:47:57,802][105620] Updated weights for policy 1, policy_version 1020174 (0.0007) [2023-12-26 22:47:57,856][105620] Updated weights for policy 1, policy_version 1020184 (0.0005) [2023-12-26 22:47:57,924][105620] Updated weights for policy 1, policy_version 1020194 (0.0005) [2023-12-26 22:47:58,462][105692] Updated weights for policy 0, policy_version 1019755 (0.0009) [2023-12-26 22:47:58,524][105692] Updated weights for policy 0, policy_version 1019765 (0.0009) [2023-12-26 22:47:58,592][105692] Updated weights for policy 0, policy_version 1019775 (0.0009) [2023-12-26 22:47:58,611][105620] Updated weights for policy 1, policy_version 1020204 (0.0006) [2023-12-26 22:47:58,681][105620] Updated weights for policy 1, policy_version 1020214 (0.0008) [2023-12-26 22:47:58,757][105620] Updated weights for policy 1, policy_version 1020224 (0.0008) [2023-12-26 22:47:59,437][105692] Updated weights for policy 0, policy_version 1019785 (0.0007) [2023-12-26 22:47:59,495][105692] Updated weights for policy 0, policy_version 1019795 (0.0009) [2023-12-26 22:47:59,560][105692] Updated weights for policy 0, policy_version 1019805 (0.0009) [2023-12-26 22:47:59,591][105620] Updated weights for policy 1, policy_version 1020234 (0.0009) [2023-12-26 22:47:59,624][105692] Updated weights for policy 0, policy_version 1019815 (0.0007) [2023-12-26 22:47:59,639][105620] Updated weights for policy 1, policy_version 1020244 (0.0008) [2023-12-26 22:47:59,696][105620] Updated weights for policy 1, policy_version 1020254 (0.0009) [2023-12-26 22:47:59,745][105620] Updated weights for policy 1, policy_version 1020264 (0.0008) [2023-12-26 22:48:00,357][105692] Updated weights for policy 0, policy_version 1019825 (0.0009) [2023-12-26 22:48:00,416][105692] Updated weights for policy 0, policy_version 1019835 (0.0008) [2023-12-26 22:48:00,479][105692] Updated weights for policy 0, policy_version 1019845 (0.0010) [2023-12-26 22:48:00,513][105620] Updated weights for policy 1, policy_version 1020274 (0.0008) [2023-12-26 22:48:00,562][105620] Updated weights for policy 1, policy_version 1020284 (0.0008) [2023-12-26 22:48:00,614][105620] Updated weights for policy 1, policy_version 1020294 (0.0005) [2023-12-26 22:48:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 522346496. Throughput: 0: 9942.6, 1: 9709.8. Samples: 522320044. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:48:01,063][104569] Avg episode reward: [(0, '8830.235'), (1, '9351.864')] [2023-12-26 22:48:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001019848_261120000.pth... [2023-12-26 22:48:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001020296_261226496.pth... [2023-12-26 22:48:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001018696_260825088.pth [2023-12-26 22:48:01,088][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001019208_260947968.pth [2023-12-26 22:48:01,164][105692] Updated weights for policy 0, policy_version 1019855 (0.0008) [2023-12-26 22:48:01,224][105692] Updated weights for policy 0, policy_version 1019865 (0.0005) [2023-12-26 22:48:01,286][105692] Updated weights for policy 0, policy_version 1019875 (0.0009) [2023-12-26 22:48:01,362][105620] Updated weights for policy 1, policy_version 1020304 (0.0008) [2023-12-26 22:48:01,424][105620] Updated weights for policy 1, policy_version 1020314 (0.0007) [2023-12-26 22:48:01,483][105620] Updated weights for policy 1, policy_version 1020324 (0.0006) [2023-12-26 22:48:02,010][105692] Updated weights for policy 0, policy_version 1019885 (0.0009) [2023-12-26 22:48:02,080][105692] Updated weights for policy 0, policy_version 1019895 (0.0008) [2023-12-26 22:48:02,101][105585] KL-divergence is very high: 161.1457 [2023-12-26 22:48:02,140][105692] Updated weights for policy 0, policy_version 1019905 (0.0006) [2023-12-26 22:48:02,147][105585] KL-divergence is very high: 178.4011 [2023-12-26 22:48:02,180][105620] Updated weights for policy 1, policy_version 1020334 (0.0007) [2023-12-26 22:48:02,233][105620] Updated weights for policy 1, policy_version 1020345 (0.0010) [2023-12-26 22:48:02,296][105620] Updated weights for policy 1, policy_version 1020356 (0.0010) [2023-12-26 22:48:02,792][105692] Updated weights for policy 0, policy_version 1019915 (0.0007) [2023-12-26 22:48:02,836][105692] Updated weights for policy 0, policy_version 1019925 (0.0007) [2023-12-26 22:48:02,884][105692] Updated weights for policy 0, policy_version 1019935 (0.0006) [2023-12-26 22:48:03,054][105620] Updated weights for policy 1, policy_version 1020366 (0.0008) [2023-12-26 22:48:03,106][105620] Updated weights for policy 1, policy_version 1020376 (0.0006) [2023-12-26 22:48:03,175][105620] Updated weights for policy 1, policy_version 1020386 (0.0008) [2023-12-26 22:48:03,620][105692] Updated weights for policy 0, policy_version 1019945 (0.0009) [2023-12-26 22:48:03,674][105692] Updated weights for policy 0, policy_version 1019955 (0.0009) [2023-12-26 22:48:03,724][105692] Updated weights for policy 0, policy_version 1019965 (0.0009) [2023-12-26 22:48:03,753][105620] Updated weights for policy 1, policy_version 1020396 (0.0008) [2023-12-26 22:48:03,771][105692] Updated weights for policy 0, policy_version 1019975 (0.0008) [2023-12-26 22:48:03,799][105620] Updated weights for policy 1, policy_version 1020406 (0.0007) [2023-12-26 22:48:03,855][105620] Updated weights for policy 1, policy_version 1020416 (0.0008) [2023-12-26 22:48:04,460][105692] Updated weights for policy 0, policy_version 1019985 (0.0009) [2023-12-26 22:48:04,521][105692] Updated weights for policy 0, policy_version 1019995 (0.0009) [2023-12-26 22:48:04,579][105692] Updated weights for policy 0, policy_version 1020005 (0.0005) [2023-12-26 22:48:04,682][105620] Updated weights for policy 1, policy_version 1020426 (0.0009) [2023-12-26 22:48:04,735][105620] Updated weights for policy 1, policy_version 1020437 (0.0010) [2023-12-26 22:48:04,794][105620] Updated weights for policy 1, policy_version 1020448 (0.0010) [2023-12-26 22:48:05,125][105692] Updated weights for policy 0, policy_version 1020015 (0.0007) [2023-12-26 22:48:05,180][105692] Updated weights for policy 0, policy_version 1020025 (0.0011) [2023-12-26 22:48:05,244][105692] Updated weights for policy 0, policy_version 1020035 (0.0010) [2023-12-26 22:48:05,730][105620] Updated weights for policy 1, policy_version 1020459 (0.0010) [2023-12-26 22:48:05,784][105692] Updated weights for policy 0, policy_version 1020045 (0.0006) [2023-12-26 22:48:05,785][105620] Updated weights for policy 1, policy_version 1020469 (0.0010) [2023-12-26 22:48:05,832][105692] Updated weights for policy 0, policy_version 1020055 (0.0005) [2023-12-26 22:48:05,834][105620] Updated weights for policy 1, policy_version 1020479 (0.0009) [2023-12-26 22:48:05,884][105692] Updated weights for policy 0, policy_version 1020065 (0.0005) [2023-12-26 22:48:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 522452992. Throughput: 0: 9935.5, 1: 9559.5. Samples: 522434484. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:48:06,062][104569] Avg episode reward: [(0, '8571.662'), (1, '9259.832')] [2023-12-26 22:48:06,459][105692] Updated weights for policy 0, policy_version 1020075 (0.0005) [2023-12-26 22:48:06,525][105692] Updated weights for policy 0, policy_version 1020085 (0.0006) [2023-12-26 22:48:06,585][105692] Updated weights for policy 0, policy_version 1020095 (0.0006) [2023-12-26 22:48:06,638][105620] Updated weights for policy 1, policy_version 1020489 (0.0009) [2023-12-26 22:48:06,701][105620] Updated weights for policy 1, policy_version 1020499 (0.0010) [2023-12-26 22:48:06,760][105620] Updated weights for policy 1, policy_version 1020509 (0.0010) [2023-12-26 22:48:06,826][105620] Updated weights for policy 1, policy_version 1020519 (0.0010) [2023-12-26 22:48:07,228][105692] Updated weights for policy 0, policy_version 1020105 (0.0006) [2023-12-26 22:48:07,286][105692] Updated weights for policy 0, policy_version 1020115 (0.0011) [2023-12-26 22:48:07,347][105692] Updated weights for policy 0, policy_version 1020125 (0.0007) [2023-12-26 22:48:07,402][105692] Updated weights for policy 0, policy_version 1020135 (0.0011) [2023-12-26 22:48:07,530][105620] Updated weights for policy 1, policy_version 1020529 (0.0006) [2023-12-26 22:48:07,587][105620] Updated weights for policy 1, policy_version 1020539 (0.0005) [2023-12-26 22:48:07,632][105620] Updated weights for policy 1, policy_version 1020549 (0.0005) [2023-12-26 22:48:08,059][105692] Updated weights for policy 0, policy_version 1020145 (0.0007) [2023-12-26 22:48:08,117][105692] Updated weights for policy 0, policy_version 1020155 (0.0007) [2023-12-26 22:48:08,169][105692] Updated weights for policy 0, policy_version 1020165 (0.0005) [2023-12-26 22:48:08,197][105620] Updated weights for policy 1, policy_version 1020559 (0.0009) [2023-12-26 22:48:08,242][105620] Updated weights for policy 1, policy_version 1020569 (0.0010) [2023-12-26 22:48:08,294][105620] Updated weights for policy 1, policy_version 1020579 (0.0010) [2023-12-26 22:48:08,809][105692] Updated weights for policy 0, policy_version 1020175 (0.0008) [2023-12-26 22:48:08,863][105692] Updated weights for policy 0, policy_version 1020185 (0.0008) [2023-12-26 22:48:08,915][105692] Updated weights for policy 0, policy_version 1020195 (0.0007) [2023-12-26 22:48:09,063][105620] Updated weights for policy 1, policy_version 1020589 (0.0011) [2023-12-26 22:48:09,128][105620] Updated weights for policy 1, policy_version 1020599 (0.0010) [2023-12-26 22:48:09,186][105620] Updated weights for policy 1, policy_version 1020609 (0.0010) [2023-12-26 22:48:09,634][105692] Updated weights for policy 0, policy_version 1020205 (0.0006) [2023-12-26 22:48:09,701][105692] Updated weights for policy 0, policy_version 1020215 (0.0006) [2023-12-26 22:48:09,762][105692] Updated weights for policy 0, policy_version 1020225 (0.0006) [2023-12-26 22:48:10,017][105620] Updated weights for policy 1, policy_version 1020619 (0.0010) [2023-12-26 22:48:10,078][105620] Updated weights for policy 1, policy_version 1020629 (0.0011) [2023-12-26 22:48:10,141][105620] Updated weights for policy 1, policy_version 1020639 (0.0011) [2023-12-26 22:48:10,472][105692] Updated weights for policy 0, policy_version 1020235 (0.0008) [2023-12-26 22:48:10,532][105692] Updated weights for policy 0, policy_version 1020245 (0.0008) [2023-12-26 22:48:10,590][105692] Updated weights for policy 0, policy_version 1020255 (0.0008) [2023-12-26 22:48:10,883][105620] Updated weights for policy 1, policy_version 1020649 (0.0011) [2023-12-26 22:48:10,945][105620] Updated weights for policy 1, policy_version 1020659 (0.0007) [2023-12-26 22:48:11,008][105620] Updated weights for policy 1, policy_version 1020669 (0.0006) [2023-12-26 22:48:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 522543104. Throughput: 0: 10069.1, 1: 9550.8. Samples: 522556056. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:48:11,063][104569] Avg episode reward: [(0, '8389.628'), (1, '9260.335')] [2023-12-26 22:48:11,075][105620] Updated weights for policy 1, policy_version 1020679 (0.0009) [2023-12-26 22:48:11,392][105692] Updated weights for policy 0, policy_version 1020265 (0.0008) [2023-12-26 22:48:11,460][105692] Updated weights for policy 0, policy_version 1020275 (0.0008) [2023-12-26 22:48:11,525][105692] Updated weights for policy 0, policy_version 1020285 (0.0010) [2023-12-26 22:48:11,591][105692] Updated weights for policy 0, policy_version 1020295 (0.0008) [2023-12-26 22:48:11,785][105620] Updated weights for policy 1, policy_version 1020689 (0.0008) [2023-12-26 22:48:11,846][105620] Updated weights for policy 1, policy_version 1020699 (0.0007) [2023-12-26 22:48:11,899][105620] Updated weights for policy 1, policy_version 1020709 (0.0009) [2023-12-26 22:48:12,335][105692] Updated weights for policy 0, policy_version 1020305 (0.0008) [2023-12-26 22:48:12,380][105585] KL-divergence is very high: 121.3230 [2023-12-26 22:48:12,397][105692] Updated weights for policy 0, policy_version 1020315 (0.0008) [2023-12-26 22:48:12,421][105585] KL-divergence is very high: 120.1930 [2023-12-26 22:48:12,449][105692] Updated weights for policy 0, policy_version 1020325 (0.0008) [2023-12-26 22:48:12,696][105620] Updated weights for policy 1, policy_version 1020719 (0.0009) [2023-12-26 22:48:12,755][105620] Updated weights for policy 1, policy_version 1020729 (0.0009) [2023-12-26 22:48:12,813][105620] Updated weights for policy 1, policy_version 1020739 (0.0008) [2023-12-26 22:48:13,266][105692] Updated weights for policy 0, policy_version 1020335 (0.0009) [2023-12-26 22:48:13,324][105692] Updated weights for policy 0, policy_version 1020345 (0.0010) [2023-12-26 22:48:13,377][105692] Updated weights for policy 0, policy_version 1020355 (0.0010) [2023-12-26 22:48:13,501][105620] Updated weights for policy 1, policy_version 1020749 (0.0009) [2023-12-26 22:48:13,568][105620] Updated weights for policy 1, policy_version 1020759 (0.0006) [2023-12-26 22:48:13,633][105620] Updated weights for policy 1, policy_version 1020769 (0.0007) [2023-12-26 22:48:14,119][105692] Updated weights for policy 0, policy_version 1020365 (0.0009) [2023-12-26 22:48:14,178][105692] Updated weights for policy 0, policy_version 1020375 (0.0009) [2023-12-26 22:48:14,233][105692] Updated weights for policy 0, policy_version 1020385 (0.0008) [2023-12-26 22:48:14,333][105620] Updated weights for policy 1, policy_version 1020779 (0.0010) [2023-12-26 22:48:14,393][105620] Updated weights for policy 1, policy_version 1020789 (0.0010) [2023-12-26 22:48:14,458][105620] Updated weights for policy 1, policy_version 1020799 (0.0010) [2023-12-26 22:48:15,002][105692] Updated weights for policy 0, policy_version 1020395 (0.0008) [2023-12-26 22:48:15,071][105692] Updated weights for policy 0, policy_version 1020405 (0.0008) [2023-12-26 22:48:15,136][105692] Updated weights for policy 0, policy_version 1020415 (0.0009) [2023-12-26 22:48:15,177][105620] Updated weights for policy 1, policy_version 1020809 (0.0010) [2023-12-26 22:48:15,234][105620] Updated weights for policy 1, policy_version 1020819 (0.0007) [2023-12-26 22:48:15,281][105620] Updated weights for policy 1, policy_version 1020829 (0.0008) [2023-12-26 22:48:15,330][105620] Updated weights for policy 1, policy_version 1020839 (0.0005) [2023-12-26 22:48:15,922][105692] Updated weights for policy 0, policy_version 1020425 (0.0009) [2023-12-26 22:48:15,970][105692] Updated weights for policy 0, policy_version 1020435 (0.0010) [2023-12-26 22:48:15,981][105620] Updated weights for policy 1, policy_version 1020849 (0.0010) [2023-12-26 22:48:16,014][105692] Updated weights for policy 0, policy_version 1020445 (0.0010) [2023-12-26 22:48:16,040][105620] Updated weights for policy 1, policy_version 1020859 (0.0006) [2023-12-26 22:48:16,061][105692] Updated weights for policy 0, policy_version 1020455 (0.0008) [2023-12-26 22:48:16,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 522633216. Throughput: 0: 10035.6, 1: 9467.5. Samples: 522611612. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:48:16,062][104569] Avg episode reward: [(0, '7973.131'), (1, '9351.803')] [2023-12-26 22:48:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001020456_261275648.pth... [2023-12-26 22:48:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001019272_260972544.pth [2023-12-26 22:48:16,104][105620] Updated weights for policy 1, policy_version 1020869 (0.0006) [2023-12-26 22:48:16,120][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001020872_261373952.pth... [2023-12-26 22:48:16,123][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001019752_261087232.pth [2023-12-26 22:48:16,692][105692] Updated weights for policy 0, policy_version 1020465 (0.0005) [2023-12-26 22:48:16,694][105620] Updated weights for policy 1, policy_version 1020879 (0.0009) [2023-12-26 22:48:16,742][105620] Updated weights for policy 1, policy_version 1020889 (0.0010) [2023-12-26 22:48:16,745][105692] Updated weights for policy 0, policy_version 1020475 (0.0005) [2023-12-26 22:48:16,794][105620] Updated weights for policy 1, policy_version 1020899 (0.0010) [2023-12-26 22:48:16,807][105692] Updated weights for policy 0, policy_version 1020485 (0.0007) [2023-12-26 22:48:17,483][105692] Updated weights for policy 0, policy_version 1020495 (0.0009) [2023-12-26 22:48:17,543][105692] Updated weights for policy 0, policy_version 1020505 (0.0005) [2023-12-26 22:48:17,557][105620] Updated weights for policy 1, policy_version 1020909 (0.0008) [2023-12-26 22:48:17,604][105692] Updated weights for policy 0, policy_version 1020515 (0.0005) [2023-12-26 22:48:17,605][105620] Updated weights for policy 1, policy_version 1020919 (0.0010) [2023-12-26 22:48:17,653][105620] Updated weights for policy 1, policy_version 1020929 (0.0010) [2023-12-26 22:48:18,178][105692] Updated weights for policy 0, policy_version 1020525 (0.0008) [2023-12-26 22:48:18,237][105692] Updated weights for policy 0, policy_version 1020535 (0.0011) [2023-12-26 22:48:18,298][105692] Updated weights for policy 0, policy_version 1020545 (0.0010) [2023-12-26 22:48:18,383][105620] Updated weights for policy 1, policy_version 1020939 (0.0009) [2023-12-26 22:48:18,431][105620] Updated weights for policy 1, policy_version 1020949 (0.0010) [2023-12-26 22:48:18,480][105620] Updated weights for policy 1, policy_version 1020959 (0.0010) [2023-12-26 22:48:18,999][105692] Updated weights for policy 0, policy_version 1020555 (0.0009) [2023-12-26 22:48:19,063][105692] Updated weights for policy 0, policy_version 1020565 (0.0011) [2023-12-26 22:48:19,126][105692] Updated weights for policy 0, policy_version 1020575 (0.0010) [2023-12-26 22:48:19,245][105620] Updated weights for policy 1, policy_version 1020969 (0.0010) [2023-12-26 22:48:19,301][105620] Updated weights for policy 1, policy_version 1020979 (0.0006) [2023-12-26 22:48:19,361][105620] Updated weights for policy 1, policy_version 1020989 (0.0010) [2023-12-26 22:48:19,430][105620] Updated weights for policy 1, policy_version 1020999 (0.0006) [2023-12-26 22:48:19,764][105692] Updated weights for policy 0, policy_version 1020585 (0.0010) [2023-12-26 22:48:19,831][105692] Updated weights for policy 0, policy_version 1020595 (0.0006) [2023-12-26 22:48:19,889][105692] Updated weights for policy 0, policy_version 1020605 (0.0006) [2023-12-26 22:48:19,953][105692] Updated weights for policy 0, policy_version 1020615 (0.0007) [2023-12-26 22:48:20,232][105620] Updated weights for policy 1, policy_version 1021009 (0.0009) [2023-12-26 22:48:20,293][105620] Updated weights for policy 1, policy_version 1021019 (0.0008) [2023-12-26 22:48:20,350][105620] Updated weights for policy 1, policy_version 1021029 (0.0008) [2023-12-26 22:48:20,689][105692] Updated weights for policy 0, policy_version 1020625 (0.0011) [2023-12-26 22:48:20,750][105692] Updated weights for policy 0, policy_version 1020635 (0.0011) [2023-12-26 22:48:20,808][105692] Updated weights for policy 0, policy_version 1020645 (0.0007) [2023-12-26 22:48:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 522739712. Throughput: 0: 10017.1, 1: 9437.0. Samples: 522729712. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:48:21,062][104569] Avg episode reward: [(0, '8015.488'), (1, '9350.752')] [2023-12-26 22:48:21,179][105620] Updated weights for policy 1, policy_version 1021039 (0.0010) [2023-12-26 22:48:21,228][105620] Updated weights for policy 1, policy_version 1021049 (0.0010) [2023-12-26 22:48:21,285][105620] Updated weights for policy 1, policy_version 1021059 (0.0011) [2023-12-26 22:48:21,496][105692] Updated weights for policy 0, policy_version 1020655 (0.0006) [2023-12-26 22:48:21,557][105692] Updated weights for policy 0, policy_version 1020665 (0.0005) [2023-12-26 22:48:21,625][105692] Updated weights for policy 0, policy_version 1020675 (0.0007) [2023-12-26 22:48:22,069][105620] Updated weights for policy 1, policy_version 1021069 (0.0011) [2023-12-26 22:48:22,128][105620] Updated weights for policy 1, policy_version 1021079 (0.0011) [2023-12-26 22:48:22,188][105620] Updated weights for policy 1, policy_version 1021089 (0.0010) [2023-12-26 22:48:22,270][105692] Updated weights for policy 0, policy_version 1020685 (0.0011) [2023-12-26 22:48:22,334][105692] Updated weights for policy 0, policy_version 1020695 (0.0011) [2023-12-26 22:48:22,408][105692] Updated weights for policy 0, policy_version 1020705 (0.0008) [2023-12-26 22:48:22,847][105620] Updated weights for policy 1, policy_version 1021099 (0.0009) [2023-12-26 22:48:22,902][105620] Updated weights for policy 1, policy_version 1021109 (0.0005) [2023-12-26 22:48:22,962][105620] Updated weights for policy 1, policy_version 1021119 (0.0007) [2023-12-26 22:48:23,175][105692] Updated weights for policy 0, policy_version 1020715 (0.0008) [2023-12-26 22:48:23,234][105692] Updated weights for policy 0, policy_version 1020725 (0.0006) [2023-12-26 22:48:23,291][105692] Updated weights for policy 0, policy_version 1020735 (0.0005) [2023-12-26 22:48:23,587][105620] Updated weights for policy 1, policy_version 1021129 (0.0010) [2023-12-26 22:48:23,642][105620] Updated weights for policy 1, policy_version 1021139 (0.0009) [2023-12-26 22:48:23,700][105620] Updated weights for policy 1, policy_version 1021149 (0.0010) [2023-12-26 22:48:23,745][105620] Updated weights for policy 1, policy_version 1021159 (0.0010) [2023-12-26 22:48:24,038][105692] Updated weights for policy 0, policy_version 1020745 (0.0008) [2023-12-26 22:48:24,094][105692] Updated weights for policy 0, policy_version 1020755 (0.0009) [2023-12-26 22:48:24,148][105692] Updated weights for policy 0, policy_version 1020765 (0.0010) [2023-12-26 22:48:24,207][105692] Updated weights for policy 0, policy_version 1020775 (0.0008) [2023-12-26 22:48:24,450][105620] Updated weights for policy 1, policy_version 1021169 (0.0008) [2023-12-26 22:48:24,502][105620] Updated weights for policy 1, policy_version 1021179 (0.0008) [2023-12-26 22:48:24,569][105620] Updated weights for policy 1, policy_version 1021189 (0.0006) [2023-12-26 22:48:24,925][105692] Updated weights for policy 0, policy_version 1020785 (0.0010) [2023-12-26 22:48:24,976][105692] Updated weights for policy 0, policy_version 1020795 (0.0009) [2023-12-26 22:48:25,034][105692] Updated weights for policy 0, policy_version 1020805 (0.0009) [2023-12-26 22:48:25,133][105620] Updated weights for policy 1, policy_version 1021199 (0.0005) [2023-12-26 22:48:25,183][105620] Updated weights for policy 1, policy_version 1021209 (0.0005) [2023-12-26 22:48:25,236][105620] Updated weights for policy 1, policy_version 1021219 (0.0005) [2023-12-26 22:48:25,728][105692] Updated weights for policy 0, policy_version 1020815 (0.0010) [2023-12-26 22:48:25,762][105620] Updated weights for policy 1, policy_version 1021229 (0.0008) [2023-12-26 22:48:25,781][105692] Updated weights for policy 0, policy_version 1020825 (0.0011) [2023-12-26 22:48:25,819][105620] Updated weights for policy 1, policy_version 1021239 (0.0010) [2023-12-26 22:48:25,831][105692] Updated weights for policy 0, policy_version 1020835 (0.0011) [2023-12-26 22:48:25,874][105620] Updated weights for policy 1, policy_version 1021249 (0.0006) [2023-12-26 22:48:26,062][104569] Fps is (10 sec: 21299.3, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 522846208. Throughput: 0: 9970.8, 1: 9591.9. Samples: 522850272. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:48:26,063][104569] Avg episode reward: [(0, '8509.823'), (1, '9350.163')] [2023-12-26 22:48:26,467][105620] Updated weights for policy 1, policy_version 1021259 (0.0005) [2023-12-26 22:48:26,525][105620] Updated weights for policy 1, policy_version 1021269 (0.0005) [2023-12-26 22:48:26,537][105692] Updated weights for policy 0, policy_version 1020845 (0.0010) [2023-12-26 22:48:26,584][105620] Updated weights for policy 1, policy_version 1021279 (0.0009) [2023-12-26 22:48:26,593][105692] Updated weights for policy 0, policy_version 1020855 (0.0010) [2023-12-26 22:48:26,641][105692] Updated weights for policy 0, policy_version 1020865 (0.0010) [2023-12-26 22:48:27,163][105620] Updated weights for policy 1, policy_version 1021289 (0.0010) [2023-12-26 22:48:27,224][105620] Updated weights for policy 1, policy_version 1021299 (0.0005) [2023-12-26 22:48:27,280][105620] Updated weights for policy 1, policy_version 1021309 (0.0006) [2023-12-26 22:48:27,345][105620] Updated weights for policy 1, policy_version 1021319 (0.0006) [2023-12-26 22:48:27,365][105692] Updated weights for policy 0, policy_version 1020875 (0.0009) [2023-12-26 22:48:27,414][105692] Updated weights for policy 0, policy_version 1020885 (0.0010) [2023-12-26 22:48:27,468][105692] Updated weights for policy 0, policy_version 1020895 (0.0010) [2023-12-26 22:48:27,859][105620] Updated weights for policy 1, policy_version 1021329 (0.0005) [2023-12-26 22:48:27,919][105620] Updated weights for policy 1, policy_version 1021339 (0.0005) [2023-12-26 22:48:27,962][105620] Updated weights for policy 1, policy_version 1021349 (0.0005) [2023-12-26 22:48:28,141][105692] Updated weights for policy 0, policy_version 1020905 (0.0010) [2023-12-26 22:48:28,203][105692] Updated weights for policy 0, policy_version 1020915 (0.0011) [2023-12-26 22:48:28,255][105692] Updated weights for policy 0, policy_version 1020925 (0.0011) [2023-12-26 22:48:28,318][105692] Updated weights for policy 0, policy_version 1020935 (0.0011) [2023-12-26 22:48:28,537][105620] Updated weights for policy 1, policy_version 1021359 (0.0007) [2023-12-26 22:48:28,603][105620] Updated weights for policy 1, policy_version 1021369 (0.0008) [2023-12-26 22:48:28,655][105620] Updated weights for policy 1, policy_version 1021379 (0.0008) [2023-12-26 22:48:28,993][105692] Updated weights for policy 0, policy_version 1020945 (0.0010) [2023-12-26 22:48:29,040][105692] Updated weights for policy 0, policy_version 1020955 (0.0010) [2023-12-26 22:48:29,104][105692] Updated weights for policy 0, policy_version 1020965 (0.0010) [2023-12-26 22:48:29,446][105620] Updated weights for policy 1, policy_version 1021389 (0.0008) [2023-12-26 22:48:29,505][105620] Updated weights for policy 1, policy_version 1021399 (0.0008) [2023-12-26 22:48:29,576][105620] Updated weights for policy 1, policy_version 1021409 (0.0010) [2023-12-26 22:48:29,796][105692] Updated weights for policy 0, policy_version 1020975 (0.0011) [2023-12-26 22:48:29,864][105692] Updated weights for policy 0, policy_version 1020985 (0.0011) [2023-12-26 22:48:29,927][105692] Updated weights for policy 0, policy_version 1020995 (0.0008) [2023-12-26 22:48:30,307][105620] Updated weights for policy 1, policy_version 1021419 (0.0008) [2023-12-26 22:48:30,373][105620] Updated weights for policy 1, policy_version 1021429 (0.0005) [2023-12-26 22:48:30,427][105620] Updated weights for policy 1, policy_version 1021439 (0.0007) [2023-12-26 22:48:30,658][105692] Updated weights for policy 0, policy_version 1021005 (0.0007) [2023-12-26 22:48:30,709][105692] Updated weights for policy 0, policy_version 1021015 (0.0010) [2023-12-26 22:48:30,771][105692] Updated weights for policy 0, policy_version 1021025 (0.0010) [2023-12-26 22:48:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 522944512. Throughput: 0: 10003.1, 1: 9739.5. Samples: 522914384. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:48:31,062][104569] Avg episode reward: [(0, '9004.210'), (1, '9258.195')] [2023-12-26 22:48:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001021032_261423104.pth... [2023-12-26 22:48:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001019848_261120000.pth [2023-12-26 22:48:31,076][105620] Updated weights for policy 1, policy_version 1021449 (0.0007) [2023-12-26 22:48:31,127][105620] Updated weights for policy 1, policy_version 1021459 (0.0010) [2023-12-26 22:48:31,194][105620] Updated weights for policy 1, policy_version 1021469 (0.0008) [2023-12-26 22:48:31,264][105620] Updated weights for policy 1, policy_version 1021479 (0.0010) [2023-12-26 22:48:31,267][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001021480_261529600.pth... [2023-12-26 22:48:31,272][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001020296_261226496.pth [2023-12-26 22:48:31,548][105692] Updated weights for policy 0, policy_version 1021035 (0.0010) [2023-12-26 22:48:31,604][105692] Updated weights for policy 0, policy_version 1021046 (0.0009) [2023-12-26 22:48:31,663][105692] Updated weights for policy 0, policy_version 1021056 (0.0008) [2023-12-26 22:48:31,955][105620] Updated weights for policy 1, policy_version 1021489 (0.0006) [2023-12-26 22:48:32,006][105620] Updated weights for policy 1, policy_version 1021499 (0.0005) [2023-12-26 22:48:32,072][105620] Updated weights for policy 1, policy_version 1021509 (0.0005) [2023-12-26 22:48:32,411][105692] Updated weights for policy 0, policy_version 1021067 (0.0010) [2023-12-26 22:48:32,463][105692] Updated weights for policy 0, policy_version 1021077 (0.0009) [2023-12-26 22:48:32,516][105692] Updated weights for policy 0, policy_version 1021087 (0.0009) [2023-12-26 22:48:32,662][105620] Updated weights for policy 1, policy_version 1021519 (0.0008) [2023-12-26 22:48:32,716][105620] Updated weights for policy 1, policy_version 1021529 (0.0009) [2023-12-26 22:48:32,773][105620] Updated weights for policy 1, policy_version 1021539 (0.0009) [2023-12-26 22:48:33,222][105692] Updated weights for policy 0, policy_version 1021099 (0.0009) [2023-12-26 22:48:33,266][105692] Updated weights for policy 0, policy_version 1021109 (0.0005) [2023-12-26 22:48:33,317][105692] Updated weights for policy 0, policy_version 1021119 (0.0005) [2023-12-26 22:48:33,582][105620] Updated weights for policy 1, policy_version 1021549 (0.0010) [2023-12-26 22:48:33,636][105620] Updated weights for policy 1, policy_version 1021559 (0.0010) [2023-12-26 22:48:33,692][105620] Updated weights for policy 1, policy_version 1021569 (0.0010) [2023-12-26 22:48:33,928][105692] Updated weights for policy 0, policy_version 1021129 (0.0006) [2023-12-26 22:48:33,995][105692] Updated weights for policy 0, policy_version 1021139 (0.0006) [2023-12-26 22:48:34,060][105692] Updated weights for policy 0, policy_version 1021149 (0.0010) [2023-12-26 22:48:34,121][105692] Updated weights for policy 0, policy_version 1021159 (0.0009) [2023-12-26 22:48:34,368][105620] Updated weights for policy 1, policy_version 1021579 (0.0010) [2023-12-26 22:48:34,426][105620] Updated weights for policy 1, policy_version 1021589 (0.0009) [2023-12-26 22:48:34,489][105620] Updated weights for policy 1, policy_version 1021599 (0.0009) [2023-12-26 22:48:34,833][105692] Updated weights for policy 0, policy_version 1021169 (0.0009) [2023-12-26 22:48:34,881][105692] Updated weights for policy 0, policy_version 1021179 (0.0009) [2023-12-26 22:48:34,935][105692] Updated weights for policy 0, policy_version 1021189 (0.0009) [2023-12-26 22:48:35,229][105620] Updated weights for policy 1, policy_version 1021609 (0.0010) [2023-12-26 22:48:35,291][105620] Updated weights for policy 1, policy_version 1021620 (0.0010) [2023-12-26 22:48:35,342][105620] Updated weights for policy 1, policy_version 1021631 (0.0008) [2023-12-26 22:48:35,567][105692] Updated weights for policy 0, policy_version 1021199 (0.0009) [2023-12-26 22:48:35,624][105692] Updated weights for policy 0, policy_version 1021209 (0.0009) [2023-12-26 22:48:35,678][105692] Updated weights for policy 0, policy_version 1021219 (0.0008) [2023-12-26 22:48:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 523042816. Throughput: 0: 9935.6, 1: 9744.4. Samples: 523033188. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:48:36,062][104569] Avg episode reward: [(0, '9265.801'), (1, '9258.957')] [2023-12-26 22:48:36,097][105620] Updated weights for policy 1, policy_version 1021641 (0.0008) [2023-12-26 22:48:36,157][105620] Updated weights for policy 1, policy_version 1021651 (0.0009) [2023-12-26 22:48:36,209][105620] Updated weights for policy 1, policy_version 1021661 (0.0008) [2023-12-26 22:48:36,273][105620] Updated weights for policy 1, policy_version 1021671 (0.0009) [2023-12-26 22:48:36,413][105692] Updated weights for policy 0, policy_version 1021229 (0.0007) [2023-12-26 22:48:36,477][105692] Updated weights for policy 0, policy_version 1021239 (0.0008) [2023-12-26 22:48:36,542][105692] Updated weights for policy 0, policy_version 1021249 (0.0009) [2023-12-26 22:48:37,110][105620] Updated weights for policy 1, policy_version 1021681 (0.0006) [2023-12-26 22:48:37,176][105620] Updated weights for policy 1, policy_version 1021691 (0.0007) [2023-12-26 22:48:37,203][105692] Updated weights for policy 0, policy_version 1021259 (0.0008) [2023-12-26 22:48:37,244][105620] Updated weights for policy 1, policy_version 1021701 (0.0006) [2023-12-26 22:48:37,254][105692] Updated weights for policy 0, policy_version 1021269 (0.0008) [2023-12-26 22:48:37,306][105692] Updated weights for policy 0, policy_version 1021279 (0.0011) [2023-12-26 22:48:37,872][105620] Updated weights for policy 1, policy_version 1021711 (0.0007) [2023-12-26 22:48:37,928][105620] Updated weights for policy 1, policy_version 1021721 (0.0008) [2023-12-26 22:48:37,974][105620] Updated weights for policy 1, policy_version 1021731 (0.0005) [2023-12-26 22:48:38,038][105692] Updated weights for policy 0, policy_version 1021289 (0.0010) [2023-12-26 22:48:38,091][105692] Updated weights for policy 0, policy_version 1021299 (0.0009) [2023-12-26 22:48:38,155][105692] Updated weights for policy 0, policy_version 1021309 (0.0009) [2023-12-26 22:48:38,205][105692] Updated weights for policy 0, policy_version 1021319 (0.0009) [2023-12-26 22:48:38,711][105620] Updated weights for policy 1, policy_version 1021741 (0.0007) [2023-12-26 22:48:38,762][105620] Updated weights for policy 1, policy_version 1021751 (0.0009) [2023-12-26 22:48:38,809][105620] Updated weights for policy 1, policy_version 1021761 (0.0008) [2023-12-26 22:48:38,911][105692] Updated weights for policy 0, policy_version 1021329 (0.0008) [2023-12-26 22:48:38,958][105692] Updated weights for policy 0, policy_version 1021339 (0.0008) [2023-12-26 22:48:39,007][105692] Updated weights for policy 0, policy_version 1021349 (0.0008) [2023-12-26 22:48:39,594][105620] Updated weights for policy 1, policy_version 1021771 (0.0009) [2023-12-26 22:48:39,646][105620] Updated weights for policy 1, policy_version 1021781 (0.0008) [2023-12-26 22:48:39,712][105620] Updated weights for policy 1, policy_version 1021791 (0.0010) [2023-12-26 22:48:39,768][105692] Updated weights for policy 0, policy_version 1021359 (0.0007) [2023-12-26 22:48:39,832][105692] Updated weights for policy 0, policy_version 1021369 (0.0008) [2023-12-26 22:48:39,890][105692] Updated weights for policy 0, policy_version 1021379 (0.0008) [2023-12-26 22:48:40,551][105620] Updated weights for policy 1, policy_version 1021801 (0.0008) [2023-12-26 22:48:40,586][105692] Updated weights for policy 0, policy_version 1021389 (0.0008) [2023-12-26 22:48:40,608][105620] Updated weights for policy 1, policy_version 1021811 (0.0007) [2023-12-26 22:48:40,650][105692] Updated weights for policy 0, policy_version 1021399 (0.0008) [2023-12-26 22:48:40,662][105620] Updated weights for policy 1, policy_version 1021821 (0.0007) [2023-12-26 22:48:40,713][105692] Updated weights for policy 0, policy_version 1021409 (0.0005) [2023-12-26 22:48:40,715][105620] Updated weights for policy 1, policy_version 1021831 (0.0009) [2023-12-26 22:48:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 523141120. Throughput: 0: 9888.6, 1: 9770.4. Samples: 523148116. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:48:41,062][104569] Avg episode reward: [(0, '9175.947'), (1, '9258.436')] [2023-12-26 22:48:41,352][105692] Updated weights for policy 0, policy_version 1021419 (0.0006) [2023-12-26 22:48:41,423][105692] Updated weights for policy 0, policy_version 1021429 (0.0008) [2023-12-26 22:48:41,485][105692] Updated weights for policy 0, policy_version 1021439 (0.0008) [2023-12-26 22:48:41,503][105620] Updated weights for policy 1, policy_version 1021841 (0.0007) [2023-12-26 22:48:41,569][105620] Updated weights for policy 1, policy_version 1021851 (0.0008) [2023-12-26 22:48:41,638][105620] Updated weights for policy 1, policy_version 1021861 (0.0009) [2023-12-26 22:48:42,225][105692] Updated weights for policy 0, policy_version 1021449 (0.0008) [2023-12-26 22:48:42,291][105692] Updated weights for policy 0, policy_version 1021459 (0.0011) [2023-12-26 22:48:42,351][105692] Updated weights for policy 0, policy_version 1021469 (0.0011) [2023-12-26 22:48:42,362][105620] Updated weights for policy 1, policy_version 1021871 (0.0007) [2023-12-26 22:48:42,403][105692] Updated weights for policy 0, policy_version 1021479 (0.0011) [2023-12-26 22:48:42,422][105620] Updated weights for policy 1, policy_version 1021881 (0.0007) [2023-12-26 22:48:42,482][105620] Updated weights for policy 1, policy_version 1021891 (0.0009) [2023-12-26 22:48:43,136][105692] Updated weights for policy 0, policy_version 1021489 (0.0006) [2023-12-26 22:48:43,187][105692] Updated weights for policy 0, policy_version 1021499 (0.0005) [2023-12-26 22:48:43,270][105692] Updated weights for policy 0, policy_version 1021509 (0.0005) [2023-12-26 22:48:43,275][105620] Updated weights for policy 1, policy_version 1021901 (0.0009) [2023-12-26 22:48:43,331][105620] Updated weights for policy 1, policy_version 1021911 (0.0009) [2023-12-26 22:48:43,392][105620] Updated weights for policy 1, policy_version 1021921 (0.0009) [2023-12-26 22:48:43,904][105692] Updated weights for policy 0, policy_version 1021519 (0.0009) [2023-12-26 22:48:43,957][105692] Updated weights for policy 0, policy_version 1021529 (0.0006) [2023-12-26 22:48:44,009][105692] Updated weights for policy 0, policy_version 1021539 (0.0005) [2023-12-26 22:48:44,215][105620] Updated weights for policy 1, policy_version 1021931 (0.0007) [2023-12-26 22:48:44,267][105620] Updated weights for policy 1, policy_version 1021941 (0.0005) [2023-12-26 22:48:44,321][105620] Updated weights for policy 1, policy_version 1021951 (0.0008) [2023-12-26 22:48:44,654][105692] Updated weights for policy 0, policy_version 1021549 (0.0008) [2023-12-26 22:48:44,713][105692] Updated weights for policy 0, policy_version 1021559 (0.0011) [2023-12-26 22:48:44,766][105692] Updated weights for policy 0, policy_version 1021569 (0.0011) [2023-12-26 22:48:45,010][105620] Updated weights for policy 1, policy_version 1021961 (0.0008) [2023-12-26 22:48:45,066][105620] Updated weights for policy 1, policy_version 1021971 (0.0008) [2023-12-26 22:48:45,123][105620] Updated weights for policy 1, policy_version 1021981 (0.0009) [2023-12-26 22:48:45,183][105620] Updated weights for policy 1, policy_version 1021991 (0.0008) [2023-12-26 22:48:45,522][105692] Updated weights for policy 0, policy_version 1021579 (0.0009) [2023-12-26 22:48:45,574][105692] Updated weights for policy 0, policy_version 1021589 (0.0005) [2023-12-26 22:48:45,629][105692] Updated weights for policy 0, policy_version 1021599 (0.0005) [2023-12-26 22:48:46,039][105620] Updated weights for policy 1, policy_version 1022001 (0.0010) [2023-12-26 22:48:46,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 523231232. Throughput: 0: 9922.0, 1: 9748.9. Samples: 523205236. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:48:46,063][104569] Avg episode reward: [(0, '8995.858'), (1, '9258.346')] [2023-12-26 22:48:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001021608_261570560.pth... [2023-12-26 22:48:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001020456_261275648.pth [2023-12-26 22:48:46,093][105620] Updated weights for policy 1, policy_version 1022012 (0.0010) [2023-12-26 22:48:46,140][105692] Updated weights for policy 0, policy_version 1021609 (0.0005) [2023-12-26 22:48:46,149][105620] Updated weights for policy 1, policy_version 1022022 (0.0008) [2023-12-26 22:48:46,159][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001022024_261668864.pth... [2023-12-26 22:48:46,162][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001020872_261373952.pth [2023-12-26 22:48:46,188][105692] Updated weights for policy 0, policy_version 1021619 (0.0005) [2023-12-26 22:48:46,245][105692] Updated weights for policy 0, policy_version 1021629 (0.0005) [2023-12-26 22:48:46,294][105692] Updated weights for policy 0, policy_version 1021639 (0.0009) [2023-12-26 22:48:46,825][105692] Updated weights for policy 0, policy_version 1021649 (0.0006) [2023-12-26 22:48:46,879][105692] Updated weights for policy 0, policy_version 1021659 (0.0008) [2023-12-26 22:48:46,923][105692] Updated weights for policy 0, policy_version 1021669 (0.0010) [2023-12-26 22:48:47,051][105620] Updated weights for policy 1, policy_version 1022032 (0.0008) [2023-12-26 22:48:47,099][105620] Updated weights for policy 1, policy_version 1022042 (0.0007) [2023-12-26 22:48:47,143][105620] Updated weights for policy 1, policy_version 1022052 (0.0008) [2023-12-26 22:48:47,659][105692] Updated weights for policy 0, policy_version 1021679 (0.0010) [2023-12-26 22:48:47,707][105692] Updated weights for policy 0, policy_version 1021689 (0.0010) [2023-12-26 22:48:47,759][105692] Updated weights for policy 0, policy_version 1021699 (0.0010) [2023-12-26 22:48:47,951][105620] Updated weights for policy 1, policy_version 1022062 (0.0009) [2023-12-26 22:48:48,004][105620] Updated weights for policy 1, policy_version 1022072 (0.0009) [2023-12-26 22:48:48,057][105620] Updated weights for policy 1, policy_version 1022082 (0.0010) [2023-12-26 22:48:48,328][105692] Updated weights for policy 0, policy_version 1021709 (0.0008) [2023-12-26 22:48:48,378][105692] Updated weights for policy 0, policy_version 1021719 (0.0010) [2023-12-26 22:48:48,428][105692] Updated weights for policy 0, policy_version 1021729 (0.0011) [2023-12-26 22:48:48,845][105620] Updated weights for policy 1, policy_version 1022093 (0.0008) [2023-12-26 22:48:48,894][105620] Updated weights for policy 1, policy_version 1022103 (0.0005) [2023-12-26 22:48:48,946][105620] Updated weights for policy 1, policy_version 1022113 (0.0005) [2023-12-26 22:48:49,198][105692] Updated weights for policy 0, policy_version 1021739 (0.0011) [2023-12-26 22:48:49,259][105692] Updated weights for policy 0, policy_version 1021749 (0.0011) [2023-12-26 22:48:49,326][105692] Updated weights for policy 0, policy_version 1021759 (0.0011) [2023-12-26 22:48:49,685][105620] Updated weights for policy 1, policy_version 1022123 (0.0008) [2023-12-26 22:48:49,741][105620] Updated weights for policy 1, policy_version 1022133 (0.0010) [2023-12-26 22:48:49,797][105620] Updated weights for policy 1, policy_version 1022143 (0.0010) [2023-12-26 22:48:50,061][105692] Updated weights for policy 0, policy_version 1021769 (0.0011) [2023-12-26 22:48:50,112][105692] Updated weights for policy 0, policy_version 1021779 (0.0008) [2023-12-26 22:48:50,175][105692] Updated weights for policy 0, policy_version 1021789 (0.0011) [2023-12-26 22:48:50,235][105692] Updated weights for policy 0, policy_version 1021799 (0.0011) [2023-12-26 22:48:50,536][105620] Updated weights for policy 1, policy_version 1022153 (0.0009) [2023-12-26 22:48:50,599][105620] Updated weights for policy 1, policy_version 1022163 (0.0008) [2023-12-26 22:48:50,672][105620] Updated weights for policy 1, policy_version 1022173 (0.0009) [2023-12-26 22:48:50,738][105620] Updated weights for policy 1, policy_version 1022183 (0.0009) [2023-12-26 22:48:50,980][105692] Updated weights for policy 0, policy_version 1021809 (0.0006) [2023-12-26 22:48:51,047][105692] Updated weights for policy 0, policy_version 1021819 (0.0008) [2023-12-26 22:48:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 523329536. Throughput: 0: 10047.9, 1: 9684.4. Samples: 523322440. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:48:51,063][104569] Avg episode reward: [(0, '8727.917'), (1, '9168.712')] [2023-12-26 22:48:51,108][105692] Updated weights for policy 0, policy_version 1021829 (0.0007) [2023-12-26 22:48:51,462][105620] Updated weights for policy 1, policy_version 1022193 (0.0008) [2023-12-26 22:48:51,519][105620] Updated weights for policy 1, policy_version 1022203 (0.0005) [2023-12-26 22:48:51,574][105620] Updated weights for policy 1, policy_version 1022213 (0.0005) [2023-12-26 22:48:51,869][105692] Updated weights for policy 0, policy_version 1021839 (0.0009) [2023-12-26 22:48:51,933][105692] Updated weights for policy 0, policy_version 1021849 (0.0008) [2023-12-26 22:48:51,991][105692] Updated weights for policy 0, policy_version 1021859 (0.0009) [2023-12-26 22:48:52,361][105620] Updated weights for policy 1, policy_version 1022223 (0.0008) [2023-12-26 22:48:52,420][105620] Updated weights for policy 1, policy_version 1022233 (0.0009) [2023-12-26 22:48:52,479][105620] Updated weights for policy 1, policy_version 1022243 (0.0009) [2023-12-26 22:48:52,682][105692] Updated weights for policy 0, policy_version 1021869 (0.0007) [2023-12-26 22:48:52,746][105692] Updated weights for policy 0, policy_version 1021879 (0.0005) [2023-12-26 22:48:52,818][105692] Updated weights for policy 0, policy_version 1021889 (0.0005) [2023-12-26 22:48:53,302][105692] Updated weights for policy 0, policy_version 1021899 (0.0007) [2023-12-26 22:48:53,329][105620] Updated weights for policy 1, policy_version 1022253 (0.0007) [2023-12-26 22:48:53,359][105692] Updated weights for policy 0, policy_version 1021909 (0.0009) [2023-12-26 22:48:53,385][105620] Updated weights for policy 1, policy_version 1022263 (0.0005) [2023-12-26 22:48:53,424][105692] Updated weights for policy 0, policy_version 1021919 (0.0009) [2023-12-26 22:48:53,446][105620] Updated weights for policy 1, policy_version 1022273 (0.0006) [2023-12-26 22:48:54,085][105620] Updated weights for policy 1, policy_version 1022283 (0.0010) [2023-12-26 22:48:54,140][105620] Updated weights for policy 1, policy_version 1022293 (0.0010) [2023-12-26 22:48:54,192][105620] Updated weights for policy 1, policy_version 1022303 (0.0007) [2023-12-26 22:48:54,218][105692] Updated weights for policy 0, policy_version 1021929 (0.0008) [2023-12-26 22:48:54,272][105692] Updated weights for policy 0, policy_version 1021939 (0.0007) [2023-12-26 22:48:54,341][105692] Updated weights for policy 0, policy_version 1021949 (0.0009) [2023-12-26 22:48:54,396][105692] Updated weights for policy 0, policy_version 1021959 (0.0009) [2023-12-26 22:48:54,808][105620] Updated weights for policy 1, policy_version 1022313 (0.0010) [2023-12-26 22:48:54,863][105620] Updated weights for policy 1, policy_version 1022323 (0.0010) [2023-12-26 22:48:54,918][105620] Updated weights for policy 1, policy_version 1022333 (0.0010) [2023-12-26 22:48:54,983][105620] Updated weights for policy 1, policy_version 1022343 (0.0009) [2023-12-26 22:48:55,177][105692] Updated weights for policy 0, policy_version 1021969 (0.0007) [2023-12-26 22:48:55,230][105692] Updated weights for policy 0, policy_version 1021979 (0.0005) [2023-12-26 22:48:55,284][105692] Updated weights for policy 0, policy_version 1021989 (0.0007) [2023-12-26 22:48:55,698][105620] Updated weights for policy 1, policy_version 1022353 (0.0010) [2023-12-26 22:48:55,762][105620] Updated weights for policy 1, policy_version 1022363 (0.0010) [2023-12-26 22:48:55,820][105620] Updated weights for policy 1, policy_version 1022373 (0.0010) [2023-12-26 22:48:56,005][105692] Updated weights for policy 0, policy_version 1021999 (0.0008) [2023-12-26 22:48:56,060][105692] Updated weights for policy 0, policy_version 1022009 (0.0008) [2023-12-26 22:48:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 523427840. Throughput: 0: 9901.4, 1: 9713.5. Samples: 523438728. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:48:56,063][104569] Avg episode reward: [(0, '8731.219'), (1, '8708.487')] [2023-12-26 22:48:56,104][105692] Updated weights for policy 0, policy_version 1022019 (0.0008) [2023-12-26 22:48:56,547][105620] Updated weights for policy 1, policy_version 1022383 (0.0010) [2023-12-26 22:48:56,594][105620] Updated weights for policy 1, policy_version 1022393 (0.0010) [2023-12-26 22:48:56,649][105620] Updated weights for policy 1, policy_version 1022403 (0.0010) [2023-12-26 22:48:56,882][105692] Updated weights for policy 0, policy_version 1022029 (0.0009) [2023-12-26 22:48:56,935][105692] Updated weights for policy 0, policy_version 1022039 (0.0009) [2023-12-26 22:48:56,987][105692] Updated weights for policy 0, policy_version 1022049 (0.0010) [2023-12-26 22:48:57,242][105620] Updated weights for policy 1, policy_version 1022413 (0.0008) [2023-12-26 22:48:57,291][105620] Updated weights for policy 1, policy_version 1022423 (0.0006) [2023-12-26 22:48:57,357][105620] Updated weights for policy 1, policy_version 1022433 (0.0009) [2023-12-26 22:48:57,799][105692] Updated weights for policy 0, policy_version 1022060 (0.0010) [2023-12-26 22:48:57,849][105692] Updated weights for policy 0, policy_version 1022071 (0.0009) [2023-12-26 22:48:57,895][105692] Updated weights for policy 0, policy_version 1022081 (0.0008) [2023-12-26 22:48:57,992][105620] Updated weights for policy 1, policy_version 1022443 (0.0008) [2023-12-26 22:48:58,041][105620] Updated weights for policy 1, policy_version 1022453 (0.0005) [2023-12-26 22:48:58,099][105620] Updated weights for policy 1, policy_version 1022463 (0.0005) [2023-12-26 22:48:58,766][105692] Updated weights for policy 0, policy_version 1022091 (0.0008) [2023-12-26 22:48:58,840][105620] Updated weights for policy 1, policy_version 1022473 (0.0007) [2023-12-26 22:48:58,849][105692] Updated weights for policy 0, policy_version 1022101 (0.0009) [2023-12-26 22:48:58,907][105620] Updated weights for policy 1, policy_version 1022483 (0.0009) [2023-12-26 22:48:58,917][105692] Updated weights for policy 0, policy_version 1022111 (0.0007) [2023-12-26 22:48:58,972][105620] Updated weights for policy 1, policy_version 1022493 (0.0007) [2023-12-26 22:48:59,036][105620] Updated weights for policy 1, policy_version 1022503 (0.0009) [2023-12-26 22:48:59,597][105692] Updated weights for policy 0, policy_version 1022121 (0.0008) [2023-12-26 22:48:59,661][105692] Updated weights for policy 0, policy_version 1022131 (0.0005) [2023-12-26 22:48:59,722][105692] Updated weights for policy 0, policy_version 1022141 (0.0006) [2023-12-26 22:48:59,773][105692] Updated weights for policy 0, policy_version 1022151 (0.0006) [2023-12-26 22:48:59,783][105620] Updated weights for policy 1, policy_version 1022513 (0.0006) [2023-12-26 22:48:59,850][105620] Updated weights for policy 1, policy_version 1022523 (0.0006) [2023-12-26 22:48:59,918][105620] Updated weights for policy 1, policy_version 1022533 (0.0007) [2023-12-26 22:49:00,530][105692] Updated weights for policy 0, policy_version 1022161 (0.0010) [2023-12-26 22:49:00,581][105620] Updated weights for policy 1, policy_version 1022543 (0.0008) [2023-12-26 22:49:00,583][105692] Updated weights for policy 0, policy_version 1022171 (0.0008) [2023-12-26 22:49:00,637][105692] Updated weights for policy 0, policy_version 1022181 (0.0007) [2023-12-26 22:49:00,643][105620] Updated weights for policy 1, policy_version 1022553 (0.0008) [2023-12-26 22:49:00,703][105620] Updated weights for policy 1, policy_version 1022563 (0.0008) [2023-12-26 22:49:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 523526144. Throughput: 0: 9897.4, 1: 9758.8. Samples: 523496136. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:49:01,063][104569] Avg episode reward: [(0, '8618.883'), (1, '8708.091')] [2023-12-26 22:49:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001022184_261718016.pth... [2023-12-26 22:49:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001022568_261808128.pth... [2023-12-26 22:49:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001021480_261529600.pth [2023-12-26 22:49:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001021032_261423104.pth [2023-12-26 22:49:01,358][105692] Updated weights for policy 0, policy_version 1022191 (0.0008) [2023-12-26 22:49:01,426][105692] Updated weights for policy 0, policy_version 1022201 (0.0007) [2023-12-26 22:49:01,430][105620] Updated weights for policy 1, policy_version 1022573 (0.0007) [2023-12-26 22:49:01,486][105692] Updated weights for policy 0, policy_version 1022211 (0.0008) [2023-12-26 22:49:01,488][105620] Updated weights for policy 1, policy_version 1022583 (0.0006) [2023-12-26 22:49:01,550][105620] Updated weights for policy 1, policy_version 1022593 (0.0005) [2023-12-26 22:49:02,096][105692] Updated weights for policy 0, policy_version 1022221 (0.0005) [2023-12-26 22:49:02,154][105692] Updated weights for policy 0, policy_version 1022231 (0.0008) [2023-12-26 22:49:02,189][105620] Updated weights for policy 1, policy_version 1022603 (0.0007) [2023-12-26 22:49:02,220][105692] Updated weights for policy 0, policy_version 1022241 (0.0006) [2023-12-26 22:49:02,252][105620] Updated weights for policy 1, policy_version 1022613 (0.0009) [2023-12-26 22:49:02,310][105620] Updated weights for policy 1, policy_version 1022623 (0.0009) [2023-12-26 22:49:02,916][105620] Updated weights for policy 1, policy_version 1022633 (0.0007) [2023-12-26 22:49:02,955][105692] Updated weights for policy 0, policy_version 1022251 (0.0008) [2023-12-26 22:49:02,973][105620] Updated weights for policy 1, policy_version 1022643 (0.0006) [2023-12-26 22:49:03,006][105692] Updated weights for policy 0, policy_version 1022261 (0.0008) [2023-12-26 22:49:03,029][105620] Updated weights for policy 1, policy_version 1022653 (0.0005) [2023-12-26 22:49:03,058][105692] Updated weights for policy 0, policy_version 1022271 (0.0007) [2023-12-26 22:49:03,079][105620] Updated weights for policy 1, policy_version 1022663 (0.0008) [2023-12-26 22:49:03,689][105620] Updated weights for policy 1, policy_version 1022673 (0.0009) [2023-12-26 22:49:03,733][105620] Updated weights for policy 1, policy_version 1022683 (0.0010) [2023-12-26 22:49:03,753][105692] Updated weights for policy 0, policy_version 1022281 (0.0007) [2023-12-26 22:49:03,781][105620] Updated weights for policy 1, policy_version 1022693 (0.0010) [2023-12-26 22:49:03,808][105692] Updated weights for policy 0, policy_version 1022291 (0.0005) [2023-12-26 22:49:03,871][105692] Updated weights for policy 0, policy_version 1022301 (0.0007) [2023-12-26 22:49:03,924][105692] Updated weights for policy 0, policy_version 1022311 (0.0006) [2023-12-26 22:49:04,480][105620] Updated weights for policy 1, policy_version 1022703 (0.0008) [2023-12-26 22:49:04,480][105692] Updated weights for policy 0, policy_version 1022321 (0.0007) [2023-12-26 22:49:04,537][105692] Updated weights for policy 0, policy_version 1022331 (0.0008) [2023-12-26 22:49:04,545][105620] Updated weights for policy 1, policy_version 1022713 (0.0008) [2023-12-26 22:49:04,594][105692] Updated weights for policy 0, policy_version 1022341 (0.0010) [2023-12-26 22:49:04,605][105620] Updated weights for policy 1, policy_version 1022723 (0.0009) [2023-12-26 22:49:05,298][105620] Updated weights for policy 1, policy_version 1022733 (0.0007) [2023-12-26 22:49:05,300][105692] Updated weights for policy 0, policy_version 1022351 (0.0010) [2023-12-26 22:49:05,345][105620] Updated weights for policy 1, policy_version 1022743 (0.0005) [2023-12-26 22:49:05,351][105692] Updated weights for policy 0, policy_version 1022361 (0.0010) [2023-12-26 22:49:05,400][105620] Updated weights for policy 1, policy_version 1022753 (0.0005) [2023-12-26 22:49:05,402][105692] Updated weights for policy 0, policy_version 1022371 (0.0010) [2023-12-26 22:49:06,027][105692] Updated weights for policy 0, policy_version 1022381 (0.0010) [2023-12-26 22:49:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 523624448. Throughput: 0: 9914.6, 1: 9802.0. Samples: 523616960. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:49:06,063][104569] Avg episode reward: [(0, '8791.894'), (1, '8856.754')] [2023-12-26 22:49:06,089][105692] Updated weights for policy 0, policy_version 1022391 (0.0010) [2023-12-26 22:49:06,101][105620] Updated weights for policy 1, policy_version 1022763 (0.0006) [2023-12-26 22:49:06,152][105692] Updated weights for policy 0, policy_version 1022401 (0.0011) [2023-12-26 22:49:06,157][105620] Updated weights for policy 1, policy_version 1022773 (0.0008) [2023-12-26 22:49:06,216][105620] Updated weights for policy 1, policy_version 1022783 (0.0006) [2023-12-26 22:49:06,829][105692] Updated weights for policy 0, policy_version 1022411 (0.0011) [2023-12-26 22:49:06,890][105692] Updated weights for policy 0, policy_version 1022421 (0.0010) [2023-12-26 22:49:06,953][105692] Updated weights for policy 0, policy_version 1022431 (0.0011) [2023-12-26 22:49:06,993][105620] Updated weights for policy 1, policy_version 1022793 (0.0008) [2023-12-26 22:49:07,064][105620] Updated weights for policy 1, policy_version 1022803 (0.0010) [2023-12-26 22:49:07,130][105620] Updated weights for policy 1, policy_version 1022813 (0.0009) [2023-12-26 22:49:07,195][105620] Updated weights for policy 1, policy_version 1022823 (0.0009) [2023-12-26 22:49:07,597][105692] Updated weights for policy 0, policy_version 1022441 (0.0008) [2023-12-26 22:49:07,657][105692] Updated weights for policy 0, policy_version 1022451 (0.0005) [2023-12-26 22:49:07,714][105692] Updated weights for policy 0, policy_version 1022461 (0.0006) [2023-12-26 22:49:07,760][105692] Updated weights for policy 0, policy_version 1022471 (0.0005) [2023-12-26 22:49:08,050][105620] Updated weights for policy 1, policy_version 1022833 (0.0010) [2023-12-26 22:49:08,112][105620] Updated weights for policy 1, policy_version 1022843 (0.0009) [2023-12-26 22:49:08,168][105620] Updated weights for policy 1, policy_version 1022853 (0.0009) [2023-12-26 22:49:08,285][105692] Updated weights for policy 0, policy_version 1022481 (0.0010) [2023-12-26 22:49:08,345][105692] Updated weights for policy 0, policy_version 1022491 (0.0010) [2023-12-26 22:49:08,401][105692] Updated weights for policy 0, policy_version 1022501 (0.0010) [2023-12-26 22:49:08,899][105620] Updated weights for policy 1, policy_version 1022863 (0.0007) [2023-12-26 22:49:08,963][105620] Updated weights for policy 1, policy_version 1022873 (0.0005) [2023-12-26 22:49:09,028][105620] Updated weights for policy 1, policy_version 1022883 (0.0009) [2023-12-26 22:49:09,114][105692] Updated weights for policy 0, policy_version 1022511 (0.0006) [2023-12-26 22:49:09,169][105692] Updated weights for policy 0, policy_version 1022521 (0.0009) [2023-12-26 22:49:09,233][105692] Updated weights for policy 0, policy_version 1022531 (0.0009) [2023-12-26 22:49:09,669][105620] Updated weights for policy 1, policy_version 1022893 (0.0010) [2023-12-26 22:49:09,730][105620] Updated weights for policy 1, policy_version 1022903 (0.0011) [2023-12-26 22:49:09,794][105620] Updated weights for policy 1, policy_version 1022913 (0.0009) [2023-12-26 22:49:10,030][105692] Updated weights for policy 0, policy_version 1022541 (0.0007) [2023-12-26 22:49:10,097][105692] Updated weights for policy 0, policy_version 1022551 (0.0008) [2023-12-26 22:49:10,163][105692] Updated weights for policy 0, policy_version 1022561 (0.0009) [2023-12-26 22:49:10,489][105620] Updated weights for policy 1, policy_version 1022923 (0.0009) [2023-12-26 22:49:10,551][105620] Updated weights for policy 1, policy_version 1022933 (0.0008) [2023-12-26 22:49:10,610][105620] Updated weights for policy 1, policy_version 1022943 (0.0009) [2023-12-26 22:49:10,846][105692] Updated weights for policy 0, policy_version 1022571 (0.0009) [2023-12-26 22:49:10,901][105692] Updated weights for policy 0, policy_version 1022581 (0.0009) [2023-12-26 22:49:10,949][105692] Updated weights for policy 0, policy_version 1022591 (0.0009) [2023-12-26 22:49:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 523730944. Throughput: 0: 9974.0, 1: 9704.7. Samples: 523735816. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:49:11,063][104569] Avg episode reward: [(0, '8724.218'), (1, '9259.303')] [2023-12-26 22:49:11,320][105620] Updated weights for policy 1, policy_version 1022953 (0.0009) [2023-12-26 22:49:11,388][105620] Updated weights for policy 1, policy_version 1022963 (0.0009) [2023-12-26 22:49:11,451][105620] Updated weights for policy 1, policy_version 1022973 (0.0009) [2023-12-26 22:49:11,508][105620] Updated weights for policy 1, policy_version 1022983 (0.0010) [2023-12-26 22:49:11,730][105692] Updated weights for policy 0, policy_version 1022601 (0.0009) [2023-12-26 22:49:11,789][105692] Updated weights for policy 0, policy_version 1022611 (0.0008) [2023-12-26 22:49:11,844][105692] Updated weights for policy 0, policy_version 1022621 (0.0008) [2023-12-26 22:49:11,901][105692] Updated weights for policy 0, policy_version 1022631 (0.0009) [2023-12-26 22:49:12,300][105620] Updated weights for policy 1, policy_version 1022993 (0.0008) [2023-12-26 22:49:12,368][105620] Updated weights for policy 1, policy_version 1023003 (0.0008) [2023-12-26 22:49:12,426][105620] Updated weights for policy 1, policy_version 1023013 (0.0008) [2023-12-26 22:49:12,695][105692] Updated weights for policy 0, policy_version 1022641 (0.0009) [2023-12-26 22:49:12,753][105692] Updated weights for policy 0, policy_version 1022651 (0.0009) [2023-12-26 22:49:12,818][105692] Updated weights for policy 0, policy_version 1022661 (0.0009) [2023-12-26 22:49:13,169][105620] Updated weights for policy 1, policy_version 1023023 (0.0009) [2023-12-26 22:49:13,227][105620] Updated weights for policy 1, policy_version 1023033 (0.0009) [2023-12-26 22:49:13,274][105620] Updated weights for policy 1, policy_version 1023043 (0.0005) [2023-12-26 22:49:13,594][105692] Updated weights for policy 0, policy_version 1022671 (0.0009) [2023-12-26 22:49:13,652][105692] Updated weights for policy 0, policy_version 1022681 (0.0009) [2023-12-26 22:49:13,710][105692] Updated weights for policy 0, policy_version 1022691 (0.0009) [2023-12-26 22:49:13,987][105620] Updated weights for policy 1, policy_version 1023053 (0.0005) [2023-12-26 22:49:14,047][105620] Updated weights for policy 1, policy_version 1023063 (0.0007) [2023-12-26 22:49:14,099][105620] Updated weights for policy 1, policy_version 1023073 (0.0010) [2023-12-26 22:49:14,561][105692] Updated weights for policy 0, policy_version 1022701 (0.0009) [2023-12-26 22:49:14,618][105692] Updated weights for policy 0, policy_version 1022711 (0.0009) [2023-12-26 22:49:14,631][105620] Updated weights for policy 1, policy_version 1023083 (0.0008) [2023-12-26 22:49:14,666][105692] Updated weights for policy 0, policy_version 1022721 (0.0010) [2023-12-26 22:49:14,682][105620] Updated weights for policy 1, policy_version 1023093 (0.0009) [2023-12-26 22:49:14,735][105620] Updated weights for policy 1, policy_version 1023103 (0.0010) [2023-12-26 22:49:15,493][105620] Updated weights for policy 1, policy_version 1023113 (0.0010) [2023-12-26 22:49:15,514][105692] Updated weights for policy 0, policy_version 1022731 (0.0009) [2023-12-26 22:49:15,545][105620] Updated weights for policy 1, policy_version 1023123 (0.0005) [2023-12-26 22:49:15,562][105692] Updated weights for policy 0, policy_version 1022741 (0.0008) [2023-12-26 22:49:15,594][105620] Updated weights for policy 1, policy_version 1023133 (0.0007) [2023-12-26 22:49:15,620][105692] Updated weights for policy 0, policy_version 1022751 (0.0009) [2023-12-26 22:49:15,651][105620] Updated weights for policy 1, policy_version 1023143 (0.0008) [2023-12-26 22:49:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19522.0). Total num frames: 523821056. Throughput: 0: 9932.4, 1: 9569.3. Samples: 523791964. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:49:16,062][104569] Avg episode reward: [(0, '8728.519'), (1, '9083.832')] [2023-12-26 22:49:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001022760_261865472.pth... [2023-12-26 22:49:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001023144_261955584.pth... [2023-12-26 22:49:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001021608_261570560.pth [2023-12-26 22:49:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001022024_261668864.pth [2023-12-26 22:49:16,328][105620] Updated weights for policy 1, policy_version 1023153 (0.0010) [2023-12-26 22:49:16,344][105692] Updated weights for policy 0, policy_version 1022761 (0.0008) [2023-12-26 22:49:16,383][105620] Updated weights for policy 1, policy_version 1023163 (0.0010) [2023-12-26 22:49:16,408][105692] Updated weights for policy 0, policy_version 1022771 (0.0006) [2023-12-26 22:49:16,446][105620] Updated weights for policy 1, policy_version 1023173 (0.0011) [2023-12-26 22:49:16,473][105692] Updated weights for policy 0, policy_version 1022781 (0.0006) [2023-12-26 22:49:16,535][105692] Updated weights for policy 0, policy_version 1022791 (0.0005) [2023-12-26 22:49:17,081][105620] Updated weights for policy 1, policy_version 1023183 (0.0010) [2023-12-26 22:49:17,142][105620] Updated weights for policy 1, policy_version 1023193 (0.0010) [2023-12-26 22:49:17,194][105620] Updated weights for policy 1, policy_version 1023203 (0.0010) [2023-12-26 22:49:17,222][105692] Updated weights for policy 0, policy_version 1022801 (0.0008) [2023-12-26 22:49:17,285][105692] Updated weights for policy 0, policy_version 1022811 (0.0008) [2023-12-26 22:49:17,342][105692] Updated weights for policy 0, policy_version 1022821 (0.0009) [2023-12-26 22:49:17,868][105620] Updated weights for policy 1, policy_version 1023213 (0.0009) [2023-12-26 22:49:17,926][105620] Updated weights for policy 1, policy_version 1023223 (0.0008) [2023-12-26 22:49:17,990][105620] Updated weights for policy 1, policy_version 1023233 (0.0008) [2023-12-26 22:49:18,057][105692] Updated weights for policy 0, policy_version 1022831 (0.0010) [2023-12-26 22:49:18,121][105692] Updated weights for policy 0, policy_version 1022841 (0.0010) [2023-12-26 22:49:18,175][105692] Updated weights for policy 0, policy_version 1022851 (0.0010) [2023-12-26 22:49:18,734][105620] Updated weights for policy 1, policy_version 1023243 (0.0007) [2023-12-26 22:49:18,798][105620] Updated weights for policy 1, policy_version 1023253 (0.0008) [2023-12-26 22:49:18,830][105692] Updated weights for policy 0, policy_version 1022861 (0.0010) [2023-12-26 22:49:18,856][105620] Updated weights for policy 1, policy_version 1023263 (0.0006) [2023-12-26 22:49:18,882][105692] Updated weights for policy 0, policy_version 1022871 (0.0010) [2023-12-26 22:49:18,940][105692] Updated weights for policy 0, policy_version 1022881 (0.0010) [2023-12-26 22:49:19,588][105620] Updated weights for policy 1, policy_version 1023273 (0.0006) [2023-12-26 22:49:19,638][105620] Updated weights for policy 1, policy_version 1023283 (0.0009) [2023-12-26 22:49:19,693][105620] Updated weights for policy 1, policy_version 1023293 (0.0010) [2023-12-26 22:49:19,739][105692] Updated weights for policy 0, policy_version 1022891 (0.0010) [2023-12-26 22:49:19,749][105620] Updated weights for policy 1, policy_version 1023303 (0.0011) [2023-12-26 22:49:19,801][105692] Updated weights for policy 0, policy_version 1022901 (0.0011) [2023-12-26 22:49:19,878][105692] Updated weights for policy 0, policy_version 1022912 (0.0009) [2023-12-26 22:49:20,582][105692] Updated weights for policy 0, policy_version 1022922 (0.0010) [2023-12-26 22:49:20,584][105620] Updated weights for policy 1, policy_version 1023313 (0.0009) [2023-12-26 22:49:20,645][105620] Updated weights for policy 1, policy_version 1023323 (0.0007) [2023-12-26 22:49:20,651][105692] Updated weights for policy 0, policy_version 1022932 (0.0011) [2023-12-26 22:49:20,702][105620] Updated weights for policy 1, policy_version 1023333 (0.0007) [2023-12-26 22:49:20,706][105692] Updated weights for policy 0, policy_version 1022942 (0.0010) [2023-12-26 22:49:20,759][105692] Updated weights for policy 0, policy_version 1022952 (0.0009) [2023-12-26 22:49:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 523919360. Throughput: 0: 9844.4, 1: 9595.4. Samples: 523907976. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:49:21,062][104569] Avg episode reward: [(0, '9002.832'), (1, '8992.930')] [2023-12-26 22:49:21,454][105620] Updated weights for policy 1, policy_version 1023343 (0.0008) [2023-12-26 22:49:21,518][105620] Updated weights for policy 1, policy_version 1023353 (0.0011) [2023-12-26 22:49:21,581][105692] Updated weights for policy 0, policy_version 1022962 (0.0009) [2023-12-26 22:49:21,582][105620] Updated weights for policy 1, policy_version 1023363 (0.0011) [2023-12-26 22:49:21,642][105692] Updated weights for policy 0, policy_version 1022972 (0.0010) [2023-12-26 22:49:21,705][105692] Updated weights for policy 0, policy_version 1022982 (0.0011) [2023-12-26 22:49:22,278][105620] Updated weights for policy 1, policy_version 1023373 (0.0010) [2023-12-26 22:49:22,325][105692] Updated weights for policy 0, policy_version 1022992 (0.0008) [2023-12-26 22:49:22,346][105620] Updated weights for policy 1, policy_version 1023383 (0.0009) [2023-12-26 22:49:22,390][105692] Updated weights for policy 0, policy_version 1023002 (0.0009) [2023-12-26 22:49:22,412][105620] Updated weights for policy 1, policy_version 1023393 (0.0011) [2023-12-26 22:49:22,446][105692] Updated weights for policy 0, policy_version 1023012 (0.0009) [2023-12-26 22:49:23,148][105620] Updated weights for policy 1, policy_version 1023403 (0.0009) [2023-12-26 22:49:23,211][105620] Updated weights for policy 1, policy_version 1023413 (0.0007) [2023-12-26 22:49:23,232][105692] Updated weights for policy 0, policy_version 1023022 (0.0009) [2023-12-26 22:49:23,273][105620] Updated weights for policy 1, policy_version 1023423 (0.0007) [2023-12-26 22:49:23,297][105692] Updated weights for policy 0, policy_version 1023032 (0.0006) [2023-12-26 22:49:23,351][105692] Updated weights for policy 0, policy_version 1023042 (0.0010) [2023-12-26 22:49:23,927][105620] Updated weights for policy 1, policy_version 1023433 (0.0006) [2023-12-26 22:49:23,976][105692] Updated weights for policy 0, policy_version 1023052 (0.0010) [2023-12-26 22:49:23,986][105620] Updated weights for policy 1, policy_version 1023443 (0.0007) [2023-12-26 22:49:24,038][105692] Updated weights for policy 0, policy_version 1023062 (0.0010) [2023-12-26 22:49:24,044][105620] Updated weights for policy 1, policy_version 1023453 (0.0006) [2023-12-26 22:49:24,095][105620] Updated weights for policy 1, policy_version 1023463 (0.0006) [2023-12-26 22:49:24,096][105692] Updated weights for policy 0, policy_version 1023072 (0.0010) [2023-12-26 22:49:24,667][105620] Updated weights for policy 1, policy_version 1023473 (0.0008) [2023-12-26 22:49:24,711][105620] Updated weights for policy 1, policy_version 1023483 (0.0008) [2023-12-26 22:49:24,755][105620] Updated weights for policy 1, policy_version 1023493 (0.0008) [2023-12-26 22:49:24,842][105692] Updated weights for policy 0, policy_version 1023082 (0.0010) [2023-12-26 22:49:24,899][105692] Updated weights for policy 0, policy_version 1023092 (0.0010) [2023-12-26 22:49:24,957][105692] Updated weights for policy 0, policy_version 1023102 (0.0010) [2023-12-26 22:49:25,019][105692] Updated weights for policy 0, policy_version 1023112 (0.0010) [2023-12-26 22:49:25,527][105620] Updated weights for policy 1, policy_version 1023503 (0.0009) [2023-12-26 22:49:25,590][105620] Updated weights for policy 1, policy_version 1023513 (0.0007) [2023-12-26 22:49:25,660][105620] Updated weights for policy 1, policy_version 1023523 (0.0008) [2023-12-26 22:49:25,730][105692] Updated weights for policy 0, policy_version 1023122 (0.0009) [2023-12-26 22:49:25,792][105692] Updated weights for policy 0, policy_version 1023132 (0.0008) [2023-12-26 22:49:25,851][105692] Updated weights for policy 0, policy_version 1023142 (0.0009) [2023-12-26 22:49:26,063][104569] Fps is (10 sec: 19658.8, 60 sec: 19524.0, 300 sec: 19521.9). Total num frames: 524017664. Throughput: 0: 9810.7, 1: 9664.3. Samples: 524024512. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:49:26,064][104569] Avg episode reward: [(0, '9001.690'), (1, '9170.358')] [2023-12-26 22:49:26,345][105620] Updated weights for policy 1, policy_version 1023533 (0.0008) [2023-12-26 22:49:26,391][105620] Updated weights for policy 1, policy_version 1023543 (0.0005) [2023-12-26 22:49:26,443][105620] Updated weights for policy 1, policy_version 1023553 (0.0007) [2023-12-26 22:49:26,602][105692] Updated weights for policy 0, policy_version 1023152 (0.0009) [2023-12-26 22:49:26,661][105692] Updated weights for policy 0, policy_version 1023162 (0.0009) [2023-12-26 22:49:26,713][105692] Updated weights for policy 0, policy_version 1023172 (0.0009) [2023-12-26 22:49:27,130][105620] Updated weights for policy 1, policy_version 1023563 (0.0008) [2023-12-26 22:49:27,183][105620] Updated weights for policy 1, policy_version 1023573 (0.0005) [2023-12-26 22:49:27,229][105620] Updated weights for policy 1, policy_version 1023583 (0.0005) [2023-12-26 22:49:27,348][105692] Updated weights for policy 0, policy_version 1023182 (0.0009) [2023-12-26 22:49:27,399][105692] Updated weights for policy 0, policy_version 1023192 (0.0007) [2023-12-26 22:49:27,455][105692] Updated weights for policy 0, policy_version 1023202 (0.0005) [2023-12-26 22:49:27,800][105620] Updated weights for policy 1, policy_version 1023593 (0.0006) [2023-12-26 22:49:27,854][105620] Updated weights for policy 1, policy_version 1023603 (0.0009) [2023-12-26 22:49:27,902][105620] Updated weights for policy 1, policy_version 1023613 (0.0008) [2023-12-26 22:49:27,948][105620] Updated weights for policy 1, policy_version 1023623 (0.0009) [2023-12-26 22:49:28,097][105692] Updated weights for policy 0, policy_version 1023212 (0.0005) [2023-12-26 22:49:28,147][105692] Updated weights for policy 0, policy_version 1023222 (0.0005) [2023-12-26 22:49:28,201][105692] Updated weights for policy 0, policy_version 1023232 (0.0006) [2023-12-26 22:49:28,743][105620] Updated weights for policy 1, policy_version 1023633 (0.0009) [2023-12-26 22:49:28,796][105692] Updated weights for policy 0, policy_version 1023242 (0.0009) [2023-12-26 22:49:28,803][105620] Updated weights for policy 1, policy_version 1023643 (0.0011) [2023-12-26 22:49:28,859][105692] Updated weights for policy 0, policy_version 1023252 (0.0006) [2023-12-26 22:49:28,861][105620] Updated weights for policy 1, policy_version 1023653 (0.0010) [2023-12-26 22:49:28,928][105692] Updated weights for policy 0, policy_version 1023262 (0.0009) [2023-12-26 22:49:29,007][105692] Updated weights for policy 0, policy_version 1023272 (0.0006) [2023-12-26 22:49:29,506][105620] Updated weights for policy 1, policy_version 1023663 (0.0006) [2023-12-26 22:49:29,552][105692] Updated weights for policy 0, policy_version 1023282 (0.0010) [2023-12-26 22:49:29,564][105620] Updated weights for policy 1, policy_version 1023673 (0.0006) [2023-12-26 22:49:29,614][105692] Updated weights for policy 0, policy_version 1023292 (0.0010) [2023-12-26 22:49:29,616][105620] Updated weights for policy 1, policy_version 1023683 (0.0006) [2023-12-26 22:49:29,676][105692] Updated weights for policy 0, policy_version 1023302 (0.0010) [2023-12-26 22:49:30,266][105620] Updated weights for policy 1, policy_version 1023693 (0.0008) [2023-12-26 22:49:30,328][105692] Updated weights for policy 0, policy_version 1023312 (0.0010) [2023-12-26 22:49:30,330][105620] Updated weights for policy 1, policy_version 1023703 (0.0011) [2023-12-26 22:49:30,382][105692] Updated weights for policy 0, policy_version 1023322 (0.0010) [2023-12-26 22:49:30,389][105620] Updated weights for policy 1, policy_version 1023713 (0.0011) [2023-12-26 22:49:30,445][105692] Updated weights for policy 0, policy_version 1023332 (0.0010) [2023-12-26 22:49:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 524115968. Throughput: 0: 9840.1, 1: 9737.6. Samples: 524086228. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:49:31,062][104569] Avg episode reward: [(0, '8908.451'), (1, '9167.546')] [2023-12-26 22:49:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001023336_262012928.pth... [2023-12-26 22:49:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001022184_261718016.pth [2023-12-26 22:49:31,077][105620] Updated weights for policy 1, policy_version 1023723 (0.0010) [2023-12-26 22:49:31,148][105620] Updated weights for policy 1, policy_version 1023733 (0.0008) [2023-12-26 22:49:31,170][105692] Updated weights for policy 0, policy_version 1023342 (0.0010) [2023-12-26 22:49:31,209][105620] Updated weights for policy 1, policy_version 1023743 (0.0007) [2023-12-26 22:49:31,228][105692] Updated weights for policy 0, policy_version 1023352 (0.0007) [2023-12-26 22:49:31,254][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001023752_262111232.pth... [2023-12-26 22:49:31,259][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001022568_261808128.pth [2023-12-26 22:49:31,285][105692] Updated weights for policy 0, policy_version 1023362 (0.0006) [2023-12-26 22:49:31,994][105692] Updated weights for policy 0, policy_version 1023372 (0.0007) [2023-12-26 22:49:31,997][105620] Updated weights for policy 1, policy_version 1023753 (0.0006) [2023-12-26 22:49:32,046][105620] Updated weights for policy 1, policy_version 1023763 (0.0007) [2023-12-26 22:49:32,047][105692] Updated weights for policy 0, policy_version 1023382 (0.0006) [2023-12-26 22:49:32,102][105692] Updated weights for policy 0, policy_version 1023392 (0.0006) [2023-12-26 22:49:32,104][105620] Updated weights for policy 1, policy_version 1023773 (0.0007) [2023-12-26 22:49:32,156][105620] Updated weights for policy 1, policy_version 1023783 (0.0008) [2023-12-26 22:49:32,754][105692] Updated weights for policy 0, policy_version 1023402 (0.0007) [2023-12-26 22:49:32,812][105692] Updated weights for policy 0, policy_version 1023412 (0.0009) [2023-12-26 22:49:32,859][105692] Updated weights for policy 0, policy_version 1023422 (0.0009) [2023-12-26 22:49:32,913][105692] Updated weights for policy 0, policy_version 1023432 (0.0008) [2023-12-26 22:49:32,953][105620] Updated weights for policy 1, policy_version 1023793 (0.0009) [2023-12-26 22:49:33,002][105620] Updated weights for policy 1, policy_version 1023803 (0.0008) [2023-12-26 22:49:33,054][105620] Updated weights for policy 1, policy_version 1023813 (0.0008) [2023-12-26 22:49:33,615][105692] Updated weights for policy 0, policy_version 1023442 (0.0005) [2023-12-26 22:49:33,658][105692] Updated weights for policy 0, policy_version 1023452 (0.0005) [2023-12-26 22:49:33,703][105692] Updated weights for policy 0, policy_version 1023462 (0.0005) [2023-12-26 22:49:33,885][105620] Updated weights for policy 1, policy_version 1023823 (0.0009) [2023-12-26 22:49:33,932][105620] Updated weights for policy 1, policy_version 1023833 (0.0008) [2023-12-26 22:49:33,983][105620] Updated weights for policy 1, policy_version 1023844 (0.0010) [2023-12-26 22:49:34,323][105692] Updated weights for policy 0, policy_version 1023472 (0.0008) [2023-12-26 22:49:34,377][105692] Updated weights for policy 0, policy_version 1023482 (0.0008) [2023-12-26 22:49:34,426][105692] Updated weights for policy 0, policy_version 1023492 (0.0008) [2023-12-26 22:49:34,775][105620] Updated weights for policy 1, policy_version 1023854 (0.0010) [2023-12-26 22:49:34,837][105620] Updated weights for policy 1, policy_version 1023864 (0.0010) [2023-12-26 22:49:34,895][105620] Updated weights for policy 1, policy_version 1023874 (0.0010) [2023-12-26 22:49:35,220][105692] Updated weights for policy 0, policy_version 1023502 (0.0009) [2023-12-26 22:49:35,266][105692] Updated weights for policy 0, policy_version 1023512 (0.0009) [2023-12-26 22:49:35,314][105692] Updated weights for policy 0, policy_version 1023522 (0.0008) [2023-12-26 22:49:35,515][105620] Updated weights for policy 1, policy_version 1023884 (0.0010) [2023-12-26 22:49:35,570][105620] Updated weights for policy 1, policy_version 1023894 (0.0010) [2023-12-26 22:49:35,622][105620] Updated weights for policy 1, policy_version 1023904 (0.0010) [2023-12-26 22:49:36,062][104569] Fps is (10 sec: 19662.7, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 524214272. Throughput: 0: 9815.4, 1: 9805.8. Samples: 524205392. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:49:36,063][104569] Avg episode reward: [(0, '8636.001'), (1, '8628.013')] [2023-12-26 22:49:36,197][105692] Updated weights for policy 0, policy_version 1023532 (0.0008) [2023-12-26 22:49:36,205][105620] Updated weights for policy 1, policy_version 1023914 (0.0010) [2023-12-26 22:49:36,260][105692] Updated weights for policy 0, policy_version 1023542 (0.0009) [2023-12-26 22:49:36,267][105620] Updated weights for policy 1, policy_version 1023924 (0.0011) [2023-12-26 22:49:36,324][105692] Updated weights for policy 0, policy_version 1023552 (0.0006) [2023-12-26 22:49:36,331][105620] Updated weights for policy 1, policy_version 1023934 (0.0011) [2023-12-26 22:49:36,388][105620] Updated weights for policy 1, policy_version 1023944 (0.0008) [2023-12-26 22:49:37,042][105620] Updated weights for policy 1, policy_version 1023954 (0.0009) [2023-12-26 22:49:37,105][105620] Updated weights for policy 1, policy_version 1023964 (0.0006) [2023-12-26 22:49:37,131][105692] Updated weights for policy 0, policy_version 1023562 (0.0006) [2023-12-26 22:49:37,164][105620] Updated weights for policy 1, policy_version 1023974 (0.0006) [2023-12-26 22:49:37,193][105692] Updated weights for policy 0, policy_version 1023572 (0.0009) [2023-12-26 22:49:37,253][105692] Updated weights for policy 0, policy_version 1023582 (0.0009) [2023-12-26 22:49:37,305][105692] Updated weights for policy 0, policy_version 1023592 (0.0009) [2023-12-26 22:49:37,879][105620] Updated weights for policy 1, policy_version 1023984 (0.0008) [2023-12-26 22:49:37,931][105620] Updated weights for policy 1, policy_version 1023994 (0.0009) [2023-12-26 22:49:37,978][105620] Updated weights for policy 1, policy_version 1024004 (0.0009) [2023-12-26 22:49:38,063][105692] Updated weights for policy 0, policy_version 1023602 (0.0009) [2023-12-26 22:49:38,111][105692] Updated weights for policy 0, policy_version 1023612 (0.0009) [2023-12-26 22:49:38,159][105692] Updated weights for policy 0, policy_version 1023622 (0.0009) [2023-12-26 22:49:38,765][105620] Updated weights for policy 1, policy_version 1024014 (0.0008) [2023-12-26 22:49:38,824][105620] Updated weights for policy 1, policy_version 1024024 (0.0009) [2023-12-26 22:49:38,871][105620] Updated weights for policy 1, policy_version 1024034 (0.0008) [2023-12-26 22:49:38,978][105692] Updated weights for policy 0, policy_version 1023632 (0.0010) [2023-12-26 22:49:39,032][105692] Updated weights for policy 0, policy_version 1023642 (0.0009) [2023-12-26 22:49:39,097][105692] Updated weights for policy 0, policy_version 1023652 (0.0010) [2023-12-26 22:49:39,575][105620] Updated weights for policy 1, policy_version 1024044 (0.0010) [2023-12-26 22:49:39,625][105620] Updated weights for policy 1, policy_version 1024054 (0.0009) [2023-12-26 22:49:39,680][105620] Updated weights for policy 1, policy_version 1024064 (0.0009) [2023-12-26 22:49:39,906][105692] Updated weights for policy 0, policy_version 1023662 (0.0010) [2023-12-26 22:49:39,977][105692] Updated weights for policy 0, policy_version 1023672 (0.0009) [2023-12-26 22:49:40,037][105692] Updated weights for policy 0, policy_version 1023682 (0.0008) [2023-12-26 22:49:40,478][105620] Updated weights for policy 1, policy_version 1024074 (0.0009) [2023-12-26 22:49:40,538][105620] Updated weights for policy 1, policy_version 1024084 (0.0011) [2023-12-26 22:49:40,590][105620] Updated weights for policy 1, policy_version 1024094 (0.0010) [2023-12-26 22:49:40,643][105620] Updated weights for policy 1, policy_version 1024104 (0.0010) [2023-12-26 22:49:40,795][105692] Updated weights for policy 0, policy_version 1023692 (0.0008) [2023-12-26 22:49:40,858][105692] Updated weights for policy 0, policy_version 1023702 (0.0008) [2023-12-26 22:49:40,917][105692] Updated weights for policy 0, policy_version 1023712 (0.0008) [2023-12-26 22:49:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 524312576. Throughput: 0: 9697.2, 1: 9848.9. Samples: 524318300. Policy #0 lag: (min: 26.0, avg: 38.8, max: 58.0) [2023-12-26 22:49:41,063][104569] Avg episode reward: [(0, '8905.378'), (1, '8814.528')] [2023-12-26 22:49:41,373][105620] Updated weights for policy 1, policy_version 1024114 (0.0007) [2023-12-26 22:49:41,440][105620] Updated weights for policy 1, policy_version 1024124 (0.0009) [2023-12-26 22:49:41,494][105620] Updated weights for policy 1, policy_version 1024134 (0.0010) [2023-12-26 22:49:41,690][105692] Updated weights for policy 0, policy_version 1023722 (0.0008) [2023-12-26 22:49:41,764][105692] Updated weights for policy 0, policy_version 1023732 (0.0010) [2023-12-26 22:49:41,827][105692] Updated weights for policy 0, policy_version 1023742 (0.0009) [2023-12-26 22:49:41,879][105692] Updated weights for policy 0, policy_version 1023752 (0.0009) [2023-12-26 22:49:42,221][105620] Updated weights for policy 1, policy_version 1024144 (0.0006) [2023-12-26 22:49:42,276][105620] Updated weights for policy 1, policy_version 1024154 (0.0006) [2023-12-26 22:49:42,338][105620] Updated weights for policy 1, policy_version 1024164 (0.0007) [2023-12-26 22:49:42,735][105692] Updated weights for policy 0, policy_version 1023762 (0.0010) [2023-12-26 22:49:42,789][105692] Updated weights for policy 0, policy_version 1023772 (0.0010) [2023-12-26 22:49:42,843][105692] Updated weights for policy 0, policy_version 1023784 (0.0011) [2023-12-26 22:49:42,871][105620] Updated weights for policy 1, policy_version 1024174 (0.0007) [2023-12-26 22:49:42,926][105620] Updated weights for policy 1, policy_version 1024184 (0.0007) [2023-12-26 22:49:42,984][105620] Updated weights for policy 1, policy_version 1024194 (0.0009) [2023-12-26 22:49:43,656][105620] Updated weights for policy 1, policy_version 1024204 (0.0009) [2023-12-26 22:49:43,698][105692] Updated weights for policy 0, policy_version 1023794 (0.0007) [2023-12-26 22:49:43,712][105620] Updated weights for policy 1, policy_version 1024214 (0.0006) [2023-12-26 22:49:43,757][105692] Updated weights for policy 0, policy_version 1023804 (0.0006) [2023-12-26 22:49:43,769][105620] Updated weights for policy 1, policy_version 1024224 (0.0007) [2023-12-26 22:49:43,817][105692] Updated weights for policy 0, policy_version 1023814 (0.0005) [2023-12-26 22:49:44,488][105692] Updated weights for policy 0, policy_version 1023824 (0.0008) [2023-12-26 22:49:44,494][105620] Updated weights for policy 1, policy_version 1024234 (0.0009) [2023-12-26 22:49:44,536][105692] Updated weights for policy 0, policy_version 1023834 (0.0006) [2023-12-26 22:49:44,549][105620] Updated weights for policy 1, policy_version 1024244 (0.0008) [2023-12-26 22:49:44,603][105620] Updated weights for policy 1, policy_version 1024254 (0.0006) [2023-12-26 22:49:44,604][105692] Updated weights for policy 0, policy_version 1023844 (0.0009) [2023-12-26 22:49:44,656][105620] Updated weights for policy 1, policy_version 1024264 (0.0005) [2023-12-26 22:49:45,338][105620] Updated weights for policy 1, policy_version 1024274 (0.0005) [2023-12-26 22:49:45,403][105620] Updated weights for policy 1, policy_version 1024284 (0.0006) [2023-12-26 22:49:45,444][105692] Updated weights for policy 0, policy_version 1023854 (0.0008) [2023-12-26 22:49:45,468][105620] Updated weights for policy 1, policy_version 1024294 (0.0008) [2023-12-26 22:49:45,509][105692] Updated weights for policy 0, policy_version 1023864 (0.0009) [2023-12-26 22:49:45,571][105692] Updated weights for policy 0, policy_version 1023874 (0.0011) [2023-12-26 22:49:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 524402688. Throughput: 0: 9674.1, 1: 9866.3. Samples: 524375456. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:49:46,063][104569] Avg episode reward: [(0, '9082.441'), (1, '9169.310')] [2023-12-26 22:49:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001023880_262152192.pth... [2023-12-26 22:49:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001022760_261865472.pth [2023-12-26 22:49:46,112][105620] Updated weights for policy 1, policy_version 1024304 (0.0008) [2023-12-26 22:49:46,179][105620] Updated weights for policy 1, policy_version 1024314 (0.0010) [2023-12-26 22:49:46,231][105620] Updated weights for policy 1, policy_version 1024324 (0.0009) [2023-12-26 22:49:46,250][105692] Updated weights for policy 0, policy_version 1023884 (0.0009) [2023-12-26 22:49:46,251][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001024328_262258688.pth... [2023-12-26 22:49:46,255][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001023144_261955584.pth [2023-12-26 22:49:46,320][105692] Updated weights for policy 0, policy_version 1023894 (0.0006) [2023-12-26 22:49:46,384][105692] Updated weights for policy 0, policy_version 1023904 (0.0007) [2023-12-26 22:49:46,937][105692] Updated weights for policy 0, policy_version 1023914 (0.0006) [2023-12-26 22:49:46,995][105692] Updated weights for policy 0, policy_version 1023924 (0.0006) [2023-12-26 22:49:47,057][105692] Updated weights for policy 0, policy_version 1023934 (0.0009) [2023-12-26 22:49:47,096][105620] Updated weights for policy 1, policy_version 1024334 (0.0007) [2023-12-26 22:49:47,114][105692] Updated weights for policy 0, policy_version 1023944 (0.0008) [2023-12-26 22:49:47,151][105620] Updated weights for policy 1, policy_version 1024344 (0.0009) [2023-12-26 22:49:47,199][105620] Updated weights for policy 1, policy_version 1024354 (0.0006) [2023-12-26 22:49:47,750][105692] Updated weights for policy 0, policy_version 1023954 (0.0005) [2023-12-26 22:49:47,752][105620] Updated weights for policy 1, policy_version 1024364 (0.0007) [2023-12-26 22:49:47,809][105692] Updated weights for policy 0, policy_version 1023964 (0.0007) [2023-12-26 22:49:47,811][105620] Updated weights for policy 1, policy_version 1024374 (0.0008) [2023-12-26 22:49:47,860][105620] Updated weights for policy 1, policy_version 1024384 (0.0010) [2023-12-26 22:49:47,860][105692] Updated weights for policy 0, policy_version 1023974 (0.0007) [2023-12-26 22:49:48,606][105620] Updated weights for policy 1, policy_version 1024394 (0.0010) [2023-12-26 22:49:48,629][105692] Updated weights for policy 0, policy_version 1023984 (0.0007) [2023-12-26 22:49:48,662][105620] Updated weights for policy 1, policy_version 1024404 (0.0011) [2023-12-26 22:49:48,691][105692] Updated weights for policy 0, policy_version 1023994 (0.0005) [2023-12-26 22:49:48,721][105620] Updated weights for policy 1, policy_version 1024414 (0.0011) [2023-12-26 22:49:48,758][105692] Updated weights for policy 0, policy_version 1024004 (0.0008) [2023-12-26 22:49:48,767][105620] Updated weights for policy 1, policy_version 1024424 (0.0011) [2023-12-26 22:49:49,511][105692] Updated weights for policy 0, policy_version 1024014 (0.0008) [2023-12-26 22:49:49,548][105620] Updated weights for policy 1, policy_version 1024434 (0.0011) [2023-12-26 22:49:49,571][105692] Updated weights for policy 0, policy_version 1024024 (0.0006) [2023-12-26 22:49:49,608][105620] Updated weights for policy 1, policy_version 1024444 (0.0011) [2023-12-26 22:49:49,630][105692] Updated weights for policy 0, policy_version 1024034 (0.0005) [2023-12-26 22:49:49,663][105620] Updated weights for policy 1, policy_version 1024454 (0.0011) [2023-12-26 22:49:50,404][105692] Updated weights for policy 0, policy_version 1024044 (0.0006) [2023-12-26 22:49:50,431][105620] Updated weights for policy 1, policy_version 1024464 (0.0011) [2023-12-26 22:49:50,465][105692] Updated weights for policy 0, policy_version 1024054 (0.0005) [2023-12-26 22:49:50,491][105620] Updated weights for policy 1, policy_version 1024474 (0.0011) [2023-12-26 22:49:50,520][105692] Updated weights for policy 0, policy_version 1024064 (0.0006) [2023-12-26 22:49:50,550][105620] Updated weights for policy 1, policy_version 1024484 (0.0011) [2023-12-26 22:49:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 524500992. Throughput: 0: 9646.1, 1: 9814.1. Samples: 524492672. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:49:51,062][104569] Avg episode reward: [(0, '8992.276'), (1, '2675.750')] [2023-12-26 22:49:51,296][105692] Updated weights for policy 0, policy_version 1024074 (0.0007) [2023-12-26 22:49:51,327][105620] Updated weights for policy 1, policy_version 1024494 (0.0011) [2023-12-26 22:49:51,370][105692] Updated weights for policy 0, policy_version 1024084 (0.0007) [2023-12-26 22:49:51,409][105620] Updated weights for policy 1, policy_version 1024504 (0.0011) [2023-12-26 22:49:51,437][105692] Updated weights for policy 0, policy_version 1024094 (0.0006) [2023-12-26 22:49:51,467][105620] Updated weights for policy 1, policy_version 1024514 (0.0011) [2023-12-26 22:49:51,498][105692] Updated weights for policy 0, policy_version 1024104 (0.0007) [2023-12-26 22:49:52,236][105620] Updated weights for policy 1, policy_version 1024524 (0.0011) [2023-12-26 22:49:52,297][105620] Updated weights for policy 1, policy_version 1024534 (0.0011) [2023-12-26 22:49:52,298][105692] Updated weights for policy 0, policy_version 1024114 (0.0009) [2023-12-26 22:49:52,355][105620] Updated weights for policy 1, policy_version 1024544 (0.0010) [2023-12-26 22:49:52,367][105692] Updated weights for policy 0, policy_version 1024124 (0.0007) [2023-12-26 22:49:52,427][105692] Updated weights for policy 0, policy_version 1024134 (0.0006) [2023-12-26 22:49:53,120][105620] Updated weights for policy 1, policy_version 1024554 (0.0010) [2023-12-26 22:49:53,171][105692] Updated weights for policy 0, policy_version 1024144 (0.0008) [2023-12-26 22:49:53,181][105620] Updated weights for policy 1, policy_version 1024564 (0.0011) [2023-12-26 22:49:53,216][105692] Updated weights for policy 0, policy_version 1024154 (0.0007) [2023-12-26 22:49:53,240][105620] Updated weights for policy 1, policy_version 1024574 (0.0010) [2023-12-26 22:49:53,262][105692] Updated weights for policy 0, policy_version 1024164 (0.0006) [2023-12-26 22:49:53,299][105620] Updated weights for policy 1, policy_version 1024584 (0.0010) [2023-12-26 22:49:53,942][105620] Updated weights for policy 1, policy_version 1024594 (0.0006) [2023-12-26 22:49:54,004][105620] Updated weights for policy 1, policy_version 1024604 (0.0011) [2023-12-26 22:49:54,066][105620] Updated weights for policy 1, policy_version 1024614 (0.0010) [2023-12-26 22:49:54,111][105692] Updated weights for policy 0, policy_version 1024174 (0.0008) [2023-12-26 22:49:54,176][105692] Updated weights for policy 0, policy_version 1024184 (0.0007) [2023-12-26 22:49:54,243][105692] Updated weights for policy 0, policy_version 1024194 (0.0005) [2023-12-26 22:49:54,695][105620] Updated weights for policy 1, policy_version 1024624 (0.0009) [2023-12-26 22:49:54,746][105620] Updated weights for policy 1, policy_version 1024634 (0.0009) [2023-12-26 22:49:54,797][105620] Updated weights for policy 1, policy_version 1024644 (0.0009) [2023-12-26 22:49:54,940][105692] Updated weights for policy 0, policy_version 1024204 (0.0007) [2023-12-26 22:49:55,013][105692] Updated weights for policy 0, policy_version 1024214 (0.0010) [2023-12-26 22:49:55,074][105692] Updated weights for policy 0, policy_version 1024224 (0.0005) [2023-12-26 22:49:55,528][105620] Updated weights for policy 1, policy_version 1024654 (0.0010) [2023-12-26 22:49:55,574][105620] Updated weights for policy 1, policy_version 1024664 (0.0010) [2023-12-26 22:49:55,622][105620] Updated weights for policy 1, policy_version 1024674 (0.0010) [2023-12-26 22:49:55,645][105692] Updated weights for policy 0, policy_version 1024234 (0.0006) [2023-12-26 22:49:55,702][105692] Updated weights for policy 0, policy_version 1024244 (0.0007) [2023-12-26 22:49:55,757][105692] Updated weights for policy 0, policy_version 1024254 (0.0008) [2023-12-26 22:49:55,814][105692] Updated weights for policy 0, policy_version 1024264 (0.0007) [2023-12-26 22:49:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 524599296. Throughput: 0: 9517.2, 1: 9824.5. Samples: 524606188. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:49:56,062][104569] Avg episode reward: [(0, '8997.408'), (1, '2287.362')] [2023-12-26 22:49:56,258][105620] Updated weights for policy 1, policy_version 1024684 (0.0008) [2023-12-26 22:49:56,304][105620] Updated weights for policy 1, policy_version 1024694 (0.0005) [2023-12-26 22:49:56,351][105620] Updated weights for policy 1, policy_version 1024704 (0.0010) [2023-12-26 22:49:56,644][105585] KL-divergence is very high: 206.4201 [2023-12-26 22:49:56,649][105692] Updated weights for policy 0, policy_version 1024274 (0.0006) [2023-12-26 22:49:56,688][105585] KL-divergence is very high: 362.0206 [2023-12-26 22:49:56,703][105692] Updated weights for policy 0, policy_version 1024284 (0.0006) [2023-12-26 22:49:56,732][105585] KL-divergence is very high: 346.3075 [2023-12-26 22:49:56,760][105692] Updated weights for policy 0, policy_version 1024294 (0.0005) [2023-12-26 22:49:57,079][105620] Updated weights for policy 1, policy_version 1024714 (0.0010) [2023-12-26 22:49:57,143][105620] Updated weights for policy 1, policy_version 1024724 (0.0010) [2023-12-26 22:49:57,200][105620] Updated weights for policy 1, policy_version 1024734 (0.0010) [2023-12-26 22:49:57,265][105620] Updated weights for policy 1, policy_version 1024744 (0.0010) [2023-12-26 22:49:57,317][105692] Updated weights for policy 0, policy_version 1024304 (0.0010) [2023-12-26 22:49:57,375][105692] Updated weights for policy 0, policy_version 1024314 (0.0009) [2023-12-26 22:49:57,424][105692] Updated weights for policy 0, policy_version 1024324 (0.0005) [2023-12-26 22:49:57,936][105620] Updated weights for policy 1, policy_version 1024754 (0.0005) [2023-12-26 22:49:57,989][105620] Updated weights for policy 1, policy_version 1024764 (0.0005) [2023-12-26 22:49:58,016][105692] Updated weights for policy 0, policy_version 1024334 (0.0008) [2023-12-26 22:49:58,050][105620] Updated weights for policy 1, policy_version 1024774 (0.0006) [2023-12-26 22:49:58,059][105692] Updated weights for policy 0, policy_version 1024344 (0.0010) [2023-12-26 22:49:58,113][105692] Updated weights for policy 0, policy_version 1024354 (0.0010) [2023-12-26 22:49:58,793][105620] Updated weights for policy 1, policy_version 1024784 (0.0009) [2023-12-26 22:49:58,861][105620] Updated weights for policy 1, policy_version 1024794 (0.0008) [2023-12-26 22:49:58,896][105692] Updated weights for policy 0, policy_version 1024364 (0.0010) [2023-12-26 22:49:58,924][105620] Updated weights for policy 1, policy_version 1024804 (0.0007) [2023-12-26 22:49:58,964][105692] Updated weights for policy 0, policy_version 1024374 (0.0011) [2023-12-26 22:49:59,031][105692] Updated weights for policy 0, policy_version 1024384 (0.0010) [2023-12-26 22:49:59,647][105620] Updated weights for policy 1, policy_version 1024814 (0.0007) [2023-12-26 22:49:59,704][105620] Updated weights for policy 1, policy_version 1024824 (0.0006) [2023-12-26 22:49:59,766][105620] Updated weights for policy 1, policy_version 1024834 (0.0007) [2023-12-26 22:49:59,815][105692] Updated weights for policy 0, policy_version 1024394 (0.0010) [2023-12-26 22:49:59,877][105692] Updated weights for policy 0, policy_version 1024404 (0.0010) [2023-12-26 22:49:59,937][105692] Updated weights for policy 0, policy_version 1024414 (0.0010) [2023-12-26 22:49:59,985][105692] Updated weights for policy 0, policy_version 1024424 (0.0010) [2023-12-26 22:50:00,434][105620] Updated weights for policy 1, policy_version 1024844 (0.0008) [2023-12-26 22:50:00,493][105620] Updated weights for policy 1, policy_version 1024854 (0.0010) [2023-12-26 22:50:00,543][105620] Updated weights for policy 1, policy_version 1024864 (0.0010) [2023-12-26 22:50:00,634][105692] Updated weights for policy 0, policy_version 1024434 (0.0010) [2023-12-26 22:50:00,693][105692] Updated weights for policy 0, policy_version 1024444 (0.0011) [2023-12-26 22:50:00,745][105692] Updated weights for policy 0, policy_version 1024454 (0.0008) [2023-12-26 22:50:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 524697600. Throughput: 0: 9578.8, 1: 9856.7. Samples: 524666564. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:50:01,062][104569] Avg episode reward: [(0, '8997.256'), (1, '6925.294')] [2023-12-26 22:50:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001024456_262299648.pth... [2023-12-26 22:50:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001024872_262397952.pth... [2023-12-26 22:50:01,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001023752_262111232.pth [2023-12-26 22:50:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001023336_262012928.pth [2023-12-26 22:50:01,242][105620] Updated weights for policy 1, policy_version 1024874 (0.0010) [2023-12-26 22:50:01,304][105620] Updated weights for policy 1, policy_version 1024884 (0.0011) [2023-12-26 22:50:01,367][105692] Updated weights for policy 0, policy_version 1024464 (0.0009) [2023-12-26 22:50:01,375][105620] Updated weights for policy 1, policy_version 1024894 (0.0008) [2023-12-26 22:50:01,423][105692] Updated weights for policy 0, policy_version 1024474 (0.0011) [2023-12-26 22:50:01,437][105620] Updated weights for policy 1, policy_version 1024904 (0.0008) [2023-12-26 22:50:01,486][105692] Updated weights for policy 0, policy_version 1024484 (0.0011) [2023-12-26 22:50:02,182][105692] Updated weights for policy 0, policy_version 1024494 (0.0011) [2023-12-26 22:50:02,185][105620] Updated weights for policy 1, policy_version 1024914 (0.0007) [2023-12-26 22:50:02,240][105620] Updated weights for policy 1, policy_version 1024924 (0.0006) [2023-12-26 22:50:02,248][105692] Updated weights for policy 0, policy_version 1024504 (0.0011) [2023-12-26 22:50:02,297][105620] Updated weights for policy 1, policy_version 1024934 (0.0008) [2023-12-26 22:50:02,307][105692] Updated weights for policy 0, policy_version 1024514 (0.0007) [2023-12-26 22:50:02,878][105620] Updated weights for policy 1, policy_version 1024944 (0.0006) [2023-12-26 22:50:02,923][105620] Updated weights for policy 1, policy_version 1024954 (0.0005) [2023-12-26 22:50:02,988][105620] Updated weights for policy 1, policy_version 1024964 (0.0006) [2023-12-26 22:50:03,039][105692] Updated weights for policy 0, policy_version 1024524 (0.0009) [2023-12-26 22:50:03,094][105692] Updated weights for policy 0, policy_version 1024534 (0.0010) [2023-12-26 22:50:03,155][105692] Updated weights for policy 0, policy_version 1024544 (0.0010) [2023-12-26 22:50:03,561][105620] Updated weights for policy 1, policy_version 1024974 (0.0006) [2023-12-26 22:50:03,613][105620] Updated weights for policy 1, policy_version 1024984 (0.0005) [2023-12-26 22:50:03,660][105620] Updated weights for policy 1, policy_version 1024994 (0.0006) [2023-12-26 22:50:03,868][105692] Updated weights for policy 0, policy_version 1024554 (0.0010) [2023-12-26 22:50:03,930][105692] Updated weights for policy 0, policy_version 1024564 (0.0010) [2023-12-26 22:50:03,988][105692] Updated weights for policy 0, policy_version 1024574 (0.0010) [2023-12-26 22:50:04,049][105692] Updated weights for policy 0, policy_version 1024584 (0.0010) [2023-12-26 22:50:04,396][105620] Updated weights for policy 1, policy_version 1025004 (0.0008) [2023-12-26 22:50:04,450][105620] Updated weights for policy 1, policy_version 1025014 (0.0010) [2023-12-26 22:50:04,507][105620] Updated weights for policy 1, policy_version 1025024 (0.0010) [2023-12-26 22:50:04,763][105692] Updated weights for policy 0, policy_version 1024594 (0.0010) [2023-12-26 22:50:04,823][105692] Updated weights for policy 0, policy_version 1024604 (0.0010) [2023-12-26 22:50:04,882][105692] Updated weights for policy 0, policy_version 1024615 (0.0010) [2023-12-26 22:50:05,158][105620] Updated weights for policy 1, policy_version 1025034 (0.0007) [2023-12-26 22:50:05,211][105620] Updated weights for policy 1, policy_version 1025044 (0.0005) [2023-12-26 22:50:05,267][105620] Updated weights for policy 1, policy_version 1025054 (0.0005) [2023-12-26 22:50:05,329][105620] Updated weights for policy 1, policy_version 1025064 (0.0005) [2023-12-26 22:50:05,631][105692] Updated weights for policy 0, policy_version 1024625 (0.0010) [2023-12-26 22:50:05,679][105692] Updated weights for policy 0, policy_version 1024635 (0.0010) [2023-12-26 22:50:05,732][105692] Updated weights for policy 0, policy_version 1024645 (0.0011) [2023-12-26 22:50:05,840][105620] Updated weights for policy 1, policy_version 1025074 (0.0005) [2023-12-26 22:50:05,887][105620] Updated weights for policy 1, policy_version 1025084 (0.0007) [2023-12-26 22:50:05,936][105620] Updated weights for policy 1, policy_version 1025094 (0.0007) [2023-12-26 22:50:06,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 524804096. Throughput: 0: 9653.2, 1: 9888.8. Samples: 524787364. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:50:06,062][104569] Avg episode reward: [(0, '9087.197'), (1, '8821.348')] [2023-12-26 22:50:06,443][105692] Updated weights for policy 0, policy_version 1024655 (0.0010) [2023-12-26 22:50:06,500][105692] Updated weights for policy 0, policy_version 1024665 (0.0009) [2023-12-26 22:50:06,549][105692] Updated weights for policy 0, policy_version 1024675 (0.0009) [2023-12-26 22:50:06,671][105620] Updated weights for policy 1, policy_version 1025104 (0.0008) [2023-12-26 22:50:06,723][105620] Updated weights for policy 1, policy_version 1025114 (0.0009) [2023-12-26 22:50:06,774][105620] Updated weights for policy 1, policy_version 1025124 (0.0009) [2023-12-26 22:50:07,330][105692] Updated weights for policy 0, policy_version 1024685 (0.0009) [2023-12-26 22:50:07,378][105692] Updated weights for policy 0, policy_version 1024695 (0.0009) [2023-12-26 22:50:07,442][105692] Updated weights for policy 0, policy_version 1024705 (0.0009) [2023-12-26 22:50:07,549][105620] Updated weights for policy 1, policy_version 1025134 (0.0009) [2023-12-26 22:50:07,596][105620] Updated weights for policy 1, policy_version 1025144 (0.0008) [2023-12-26 22:50:07,643][105620] Updated weights for policy 1, policy_version 1025154 (0.0009) [2023-12-26 22:50:08,153][105692] Updated weights for policy 0, policy_version 1024715 (0.0009) [2023-12-26 22:50:08,201][105692] Updated weights for policy 0, policy_version 1024725 (0.0010) [2023-12-26 22:50:08,267][105692] Updated weights for policy 0, policy_version 1024735 (0.0010) [2023-12-26 22:50:08,339][105620] Updated weights for policy 1, policy_version 1025164 (0.0008) [2023-12-26 22:50:08,402][105620] Updated weights for policy 1, policy_version 1025174 (0.0008) [2023-12-26 22:50:08,460][105620] Updated weights for policy 1, policy_version 1025184 (0.0010) [2023-12-26 22:50:08,947][105692] Updated weights for policy 0, policy_version 1024745 (0.0009) [2023-12-26 22:50:09,003][105692] Updated weights for policy 0, policy_version 1024755 (0.0011) [2023-12-26 22:50:09,063][105692] Updated weights for policy 0, policy_version 1024765 (0.0011) [2023-12-26 22:50:09,118][105692] Updated weights for policy 0, policy_version 1024775 (0.0011) [2023-12-26 22:50:09,191][105620] Updated weights for policy 1, policy_version 1025194 (0.0010) [2023-12-26 22:50:09,256][105620] Updated weights for policy 1, policy_version 1025204 (0.0008) [2023-12-26 22:50:09,306][105620] Updated weights for policy 1, policy_version 1025214 (0.0009) [2023-12-26 22:50:09,371][105620] Updated weights for policy 1, policy_version 1025224 (0.0007) [2023-12-26 22:50:09,986][105692] Updated weights for policy 0, policy_version 1024785 (0.0007) [2023-12-26 22:50:10,046][105692] Updated weights for policy 0, policy_version 1024795 (0.0007) [2023-12-26 22:50:10,094][105620] Updated weights for policy 1, policy_version 1025234 (0.0007) [2023-12-26 22:50:10,103][105692] Updated weights for policy 0, policy_version 1024805 (0.0009) [2023-12-26 22:50:10,149][105620] Updated weights for policy 1, policy_version 1025244 (0.0007) [2023-12-26 22:50:10,209][105620] Updated weights for policy 1, policy_version 1025254 (0.0011) [2023-12-26 22:50:10,790][105692] Updated weights for policy 0, policy_version 1024815 (0.0009) [2023-12-26 22:50:10,839][105692] Updated weights for policy 0, policy_version 1024825 (0.0008) [2023-12-26 22:50:10,888][105620] Updated weights for policy 1, policy_version 1025264 (0.0010) [2023-12-26 22:50:10,897][105692] Updated weights for policy 0, policy_version 1024835 (0.0008) [2023-12-26 22:50:10,947][105620] Updated weights for policy 1, policy_version 1025274 (0.0010) [2023-12-26 22:50:11,002][105620] Updated weights for policy 1, policy_version 1025284 (0.0010) [2023-12-26 22:50:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 524902400. Throughput: 0: 9628.6, 1: 9932.8. Samples: 524904756. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:50:11,063][104569] Avg episode reward: [(0, '6867.366'), (1, '8999.900')] [2023-12-26 22:50:11,648][105692] Updated weights for policy 0, policy_version 1024845 (0.0007) [2023-12-26 22:50:11,713][105692] Updated weights for policy 0, policy_version 1024855 (0.0011) [2023-12-26 22:50:11,745][105585] KL-divergence is very high: 123.0266 [2023-12-26 22:50:11,781][105620] Updated weights for policy 1, policy_version 1025294 (0.0010) [2023-12-26 22:50:11,786][105692] Updated weights for policy 0, policy_version 1024865 (0.0009) [2023-12-26 22:50:11,795][105585] KL-divergence is very high: 124.1396 [2023-12-26 22:50:11,839][105620] Updated weights for policy 1, policy_version 1025304 (0.0009) [2023-12-26 22:50:11,899][105620] Updated weights for policy 1, policy_version 1025314 (0.0008) [2023-12-26 22:50:12,460][105692] Updated weights for policy 0, policy_version 1024875 (0.0008) [2023-12-26 22:50:12,506][105692] Updated weights for policy 0, policy_version 1024885 (0.0008) [2023-12-26 22:50:12,554][105692] Updated weights for policy 0, policy_version 1024895 (0.0008) [2023-12-26 22:50:12,663][105620] Updated weights for policy 1, policy_version 1025324 (0.0011) [2023-12-26 22:50:12,723][105620] Updated weights for policy 1, policy_version 1025334 (0.0011) [2023-12-26 22:50:12,783][105620] Updated weights for policy 1, policy_version 1025344 (0.0011) [2023-12-26 22:50:13,217][105692] Updated weights for policy 0, policy_version 1024905 (0.0009) [2023-12-26 22:50:13,270][105692] Updated weights for policy 0, policy_version 1024915 (0.0008) [2023-12-26 22:50:13,326][105692] Updated weights for policy 0, policy_version 1024925 (0.0008) [2023-12-26 22:50:13,387][105692] Updated weights for policy 0, policy_version 1024935 (0.0005) [2023-12-26 22:50:13,545][105620] Updated weights for policy 1, policy_version 1025354 (0.0011) [2023-12-26 22:50:13,597][105620] Updated weights for policy 1, policy_version 1025364 (0.0010) [2023-12-26 22:50:13,663][105620] Updated weights for policy 1, policy_version 1025374 (0.0010) [2023-12-26 22:50:13,726][105620] Updated weights for policy 1, policy_version 1025384 (0.0011) [2023-12-26 22:50:13,994][105692] Updated weights for policy 0, policy_version 1024945 (0.0008) [2023-12-26 22:50:14,049][105692] Updated weights for policy 0, policy_version 1024955 (0.0007) [2023-12-26 22:50:14,115][105692] Updated weights for policy 0, policy_version 1024965 (0.0008) [2023-12-26 22:50:14,481][105620] Updated weights for policy 1, policy_version 1025394 (0.0011) [2023-12-26 22:50:14,537][105620] Updated weights for policy 1, policy_version 1025404 (0.0010) [2023-12-26 22:50:14,596][105620] Updated weights for policy 1, policy_version 1025414 (0.0010) [2023-12-26 22:50:14,869][105692] Updated weights for policy 0, policy_version 1024975 (0.0006) [2023-12-26 22:50:14,915][105692] Updated weights for policy 0, policy_version 1024985 (0.0007) [2023-12-26 22:50:14,964][105692] Updated weights for policy 0, policy_version 1024995 (0.0008) [2023-12-26 22:50:15,286][105620] Updated weights for policy 1, policy_version 1025424 (0.0007) [2023-12-26 22:50:15,356][105620] Updated weights for policy 1, policy_version 1025434 (0.0006) [2023-12-26 22:50:15,421][105620] Updated weights for policy 1, policy_version 1025444 (0.0006) [2023-12-26 22:50:15,674][105692] Updated weights for policy 0, policy_version 1025005 (0.0010) [2023-12-26 22:50:15,743][105692] Updated weights for policy 0, policy_version 1025015 (0.0008) [2023-12-26 22:50:15,806][105692] Updated weights for policy 0, policy_version 1025025 (0.0011) [2023-12-26 22:50:16,061][105620] Updated weights for policy 1, policy_version 1025454 (0.0005) [2023-12-26 22:50:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 524992512. Throughput: 0: 9611.1, 1: 9858.5. Samples: 524962364. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:50:16,063][104569] Avg episode reward: [(0, '6996.103'), (1, '9174.856')] [2023-12-26 22:50:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001025032_262447104.pth... [2023-12-26 22:50:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001023880_262152192.pth [2023-12-26 22:50:16,114][105620] Updated weights for policy 1, policy_version 1025464 (0.0006) [2023-12-26 22:50:16,170][105620] Updated weights for policy 1, policy_version 1025474 (0.0006) [2023-12-26 22:50:16,199][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001025480_262553600.pth... [2023-12-26 22:50:16,202][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001024328_262258688.pth [2023-12-26 22:50:16,563][105692] Updated weights for policy 0, policy_version 1025035 (0.0010) [2023-12-26 22:50:16,610][105692] Updated weights for policy 0, policy_version 1025045 (0.0008) [2023-12-26 22:50:16,655][105692] Updated weights for policy 0, policy_version 1025055 (0.0008) [2023-12-26 22:50:16,877][105620] Updated weights for policy 1, policy_version 1025484 (0.0011) [2023-12-26 22:50:16,933][105620] Updated weights for policy 1, policy_version 1025494 (0.0010) [2023-12-26 22:50:16,996][105620] Updated weights for policy 1, policy_version 1025504 (0.0010) [2023-12-26 22:50:17,340][105692] Updated weights for policy 0, policy_version 1025065 (0.0010) [2023-12-26 22:50:17,397][105692] Updated weights for policy 0, policy_version 1025075 (0.0006) [2023-12-26 22:50:17,450][105692] Updated weights for policy 0, policy_version 1025085 (0.0005) [2023-12-26 22:50:17,503][105692] Updated weights for policy 0, policy_version 1025095 (0.0007) [2023-12-26 22:50:17,763][105620] Updated weights for policy 1, policy_version 1025514 (0.0010) [2023-12-26 22:50:17,826][105620] Updated weights for policy 1, policy_version 1025524 (0.0005) [2023-12-26 22:50:17,883][105620] Updated weights for policy 1, policy_version 1025534 (0.0006) [2023-12-26 22:50:17,940][105620] Updated weights for policy 1, policy_version 1025544 (0.0007) [2023-12-26 22:50:18,126][105692] Updated weights for policy 0, policy_version 1025105 (0.0006) [2023-12-26 22:50:18,179][105692] Updated weights for policy 0, policy_version 1025115 (0.0006) [2023-12-26 22:50:18,232][105692] Updated weights for policy 0, policy_version 1025125 (0.0005) [2023-12-26 22:50:18,573][105620] Updated weights for policy 1, policy_version 1025554 (0.0011) [2023-12-26 22:50:18,638][105620] Updated weights for policy 1, policy_version 1025564 (0.0008) [2023-12-26 22:50:18,695][105620] Updated weights for policy 1, policy_version 1025574 (0.0005) [2023-12-26 22:50:18,904][105692] Updated weights for policy 0, policy_version 1025135 (0.0009) [2023-12-26 22:50:18,962][105692] Updated weights for policy 0, policy_version 1025145 (0.0011) [2023-12-26 22:50:19,008][105692] Updated weights for policy 0, policy_version 1025155 (0.0010) [2023-12-26 22:50:19,428][105620] Updated weights for policy 1, policy_version 1025584 (0.0010) [2023-12-26 22:50:19,481][105620] Updated weights for policy 1, policy_version 1025594 (0.0010) [2023-12-26 22:50:19,547][105620] Updated weights for policy 1, policy_version 1025604 (0.0010) [2023-12-26 22:50:19,688][105692] Updated weights for policy 0, policy_version 1025165 (0.0008) [2023-12-26 22:50:19,741][105692] Updated weights for policy 0, policy_version 1025175 (0.0011) [2023-12-26 22:50:19,790][105692] Updated weights for policy 0, policy_version 1025185 (0.0011) [2023-12-26 22:50:20,312][105620] Updated weights for policy 1, policy_version 1025614 (0.0011) [2023-12-26 22:50:20,371][105620] Updated weights for policy 1, policy_version 1025624 (0.0010) [2023-12-26 22:50:20,433][105620] Updated weights for policy 1, policy_version 1025634 (0.0010) [2023-12-26 22:50:20,530][105692] Updated weights for policy 0, policy_version 1025195 (0.0009) [2023-12-26 22:50:20,598][105692] Updated weights for policy 0, policy_version 1025205 (0.0007) [2023-12-26 22:50:20,660][105692] Updated weights for policy 0, policy_version 1025215 (0.0006) [2023-12-26 22:50:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 525090816. Throughput: 0: 9575.2, 1: 9903.0. Samples: 525081908. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:50:21,062][104569] Avg episode reward: [(0, '8426.806'), (1, '9273.406')] [2023-12-26 22:50:21,125][105620] Updated weights for policy 1, policy_version 1025644 (0.0006) [2023-12-26 22:50:21,192][105620] Updated weights for policy 1, policy_version 1025654 (0.0007) [2023-12-26 22:50:21,260][105620] Updated weights for policy 1, policy_version 1025664 (0.0007) [2023-12-26 22:50:21,311][105692] Updated weights for policy 0, policy_version 1025225 (0.0008) [2023-12-26 22:50:21,377][105692] Updated weights for policy 0, policy_version 1025235 (0.0008) [2023-12-26 22:50:21,445][105692] Updated weights for policy 0, policy_version 1025245 (0.0008) [2023-12-26 22:50:21,499][105692] Updated weights for policy 0, policy_version 1025255 (0.0005) [2023-12-26 22:50:21,929][105620] Updated weights for policy 1, policy_version 1025674 (0.0007) [2023-12-26 22:50:21,996][105620] Updated weights for policy 1, policy_version 1025684 (0.0011) [2023-12-26 22:50:22,059][105620] Updated weights for policy 1, policy_version 1025694 (0.0011) [2023-12-26 22:50:22,124][105620] Updated weights for policy 1, policy_version 1025704 (0.0011) [2023-12-26 22:50:22,233][105692] Updated weights for policy 0, policy_version 1025265 (0.0009) [2023-12-26 22:50:22,298][105692] Updated weights for policy 0, policy_version 1025275 (0.0008) [2023-12-26 22:50:22,364][105692] Updated weights for policy 0, policy_version 1025285 (0.0008) [2023-12-26 22:50:22,892][105620] Updated weights for policy 1, policy_version 1025714 (0.0011) [2023-12-26 22:50:22,952][105620] Updated weights for policy 1, policy_version 1025724 (0.0010) [2023-12-26 22:50:23,012][105620] Updated weights for policy 1, policy_version 1025734 (0.0010) [2023-12-26 22:50:23,151][105692] Updated weights for policy 0, policy_version 1025295 (0.0008) [2023-12-26 22:50:23,208][105692] Updated weights for policy 0, policy_version 1025305 (0.0008) [2023-12-26 22:50:23,268][105692] Updated weights for policy 0, policy_version 1025315 (0.0008) [2023-12-26 22:50:23,756][105620] Updated weights for policy 1, policy_version 1025744 (0.0007) [2023-12-26 22:50:23,818][105620] Updated weights for policy 1, policy_version 1025754 (0.0010) [2023-12-26 22:50:23,882][105620] Updated weights for policy 1, policy_version 1025764 (0.0008) [2023-12-26 22:50:24,026][105692] Updated weights for policy 0, policy_version 1025325 (0.0008) [2023-12-26 22:50:24,082][105692] Updated weights for policy 0, policy_version 1025335 (0.0008) [2023-12-26 22:50:24,141][105692] Updated weights for policy 0, policy_version 1025345 (0.0008) [2023-12-26 22:50:24,614][105620] Updated weights for policy 1, policy_version 1025774 (0.0011) [2023-12-26 22:50:24,670][105620] Updated weights for policy 1, policy_version 1025784 (0.0011) [2023-12-26 22:50:24,729][105620] Updated weights for policy 1, policy_version 1025794 (0.0011) [2023-12-26 22:50:24,907][105692] Updated weights for policy 0, policy_version 1025355 (0.0008) [2023-12-26 22:50:24,952][105692] Updated weights for policy 0, policy_version 1025365 (0.0009) [2023-12-26 22:50:25,007][105692] Updated weights for policy 0, policy_version 1025375 (0.0010) [2023-12-26 22:50:25,556][105620] Updated weights for policy 1, policy_version 1025804 (0.0010) [2023-12-26 22:50:25,568][105692] Updated weights for policy 0, policy_version 1025385 (0.0008) [2023-12-26 22:50:25,617][105620] Updated weights for policy 1, policy_version 1025814 (0.0008) [2023-12-26 22:50:25,626][105692] Updated weights for policy 0, policy_version 1025395 (0.0010) [2023-12-26 22:50:25,677][105692] Updated weights for policy 0, policy_version 1025405 (0.0010) [2023-12-26 22:50:25,682][105620] Updated weights for policy 1, policy_version 1025824 (0.0006) [2023-12-26 22:50:25,726][105692] Updated weights for policy 0, policy_version 1025415 (0.0010) [2023-12-26 22:50:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.5, 300 sec: 19521.9). Total num frames: 525189120. Throughput: 0: 9695.9, 1: 9818.4. Samples: 525196452. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:50:26,064][104569] Avg episode reward: [(0, '9082.961'), (1, '9273.530')] [2023-12-26 22:50:26,261][105620] Updated weights for policy 1, policy_version 1025834 (0.0008) [2023-12-26 22:50:26,311][105620] Updated weights for policy 1, policy_version 1025844 (0.0008) [2023-12-26 22:50:26,359][105620] Updated weights for policy 1, policy_version 1025854 (0.0008) [2023-12-26 22:50:26,411][105620] Updated weights for policy 1, policy_version 1025864 (0.0008) [2023-12-26 22:50:26,472][105692] Updated weights for policy 0, policy_version 1025425 (0.0007) [2023-12-26 22:50:26,521][105692] Updated weights for policy 0, policy_version 1025435 (0.0010) [2023-12-26 22:50:26,580][105692] Updated weights for policy 0, policy_version 1025445 (0.0010) [2023-12-26 22:50:27,131][105620] Updated weights for policy 1, policy_version 1025874 (0.0010) [2023-12-26 22:50:27,178][105620] Updated weights for policy 1, policy_version 1025884 (0.0010) [2023-12-26 22:50:27,225][105620] Updated weights for policy 1, policy_version 1025894 (0.0010) [2023-12-26 22:50:27,277][105692] Updated weights for policy 0, policy_version 1025455 (0.0010) [2023-12-26 22:50:27,332][105692] Updated weights for policy 0, policy_version 1025465 (0.0008) [2023-12-26 22:50:27,386][105692] Updated weights for policy 0, policy_version 1025475 (0.0006) [2023-12-26 22:50:27,972][105620] Updated weights for policy 1, policy_version 1025904 (0.0010) [2023-12-26 22:50:28,012][105692] Updated weights for policy 0, policy_version 1025485 (0.0005) [2023-12-26 22:50:28,027][105620] Updated weights for policy 1, policy_version 1025914 (0.0010) [2023-12-26 22:50:28,063][105692] Updated weights for policy 0, policy_version 1025495 (0.0006) [2023-12-26 22:50:28,085][105620] Updated weights for policy 1, policy_version 1025924 (0.0010) [2023-12-26 22:50:28,107][105692] Updated weights for policy 0, policy_version 1025505 (0.0010) [2023-12-26 22:50:28,752][105620] Updated weights for policy 1, policy_version 1025934 (0.0008) [2023-12-26 22:50:28,818][105620] Updated weights for policy 1, policy_version 1025944 (0.0006) [2023-12-26 22:50:28,838][105692] Updated weights for policy 0, policy_version 1025515 (0.0010) [2023-12-26 22:50:28,867][105620] Updated weights for policy 1, policy_version 1025954 (0.0007) [2023-12-26 22:50:28,899][105692] Updated weights for policy 0, policy_version 1025525 (0.0010) [2023-12-26 22:50:28,948][105692] Updated weights for policy 0, policy_version 1025535 (0.0010) [2023-12-26 22:50:29,584][105620] Updated weights for policy 1, policy_version 1025964 (0.0009) [2023-12-26 22:50:29,646][105620] Updated weights for policy 1, policy_version 1025974 (0.0010) [2023-12-26 22:50:29,677][105692] Updated weights for policy 0, policy_version 1025545 (0.0010) [2023-12-26 22:50:29,702][105620] Updated weights for policy 1, policy_version 1025984 (0.0011) [2023-12-26 22:50:29,734][105692] Updated weights for policy 0, policy_version 1025555 (0.0010) [2023-12-26 22:50:29,789][105692] Updated weights for policy 0, policy_version 1025565 (0.0010) [2023-12-26 22:50:29,848][105692] Updated weights for policy 0, policy_version 1025575 (0.0012) [2023-12-26 22:50:30,381][105620] Updated weights for policy 1, policy_version 1025994 (0.0010) [2023-12-26 22:50:30,450][105620] Updated weights for policy 1, policy_version 1026004 (0.0008) [2023-12-26 22:50:30,510][105620] Updated weights for policy 1, policy_version 1026014 (0.0009) [2023-12-26 22:50:30,528][105692] Updated weights for policy 0, policy_version 1025585 (0.0005) [2023-12-26 22:50:30,564][105620] Updated weights for policy 1, policy_version 1026024 (0.0010) [2023-12-26 22:50:30,581][105692] Updated weights for policy 0, policy_version 1025595 (0.0006) [2023-12-26 22:50:30,625][105692] Updated weights for policy 0, policy_version 1025605 (0.0006) [2023-12-26 22:50:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 525287424. Throughput: 0: 9793.0, 1: 9808.4. Samples: 525257516. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:50:31,062][104569] Avg episode reward: [(0, '8998.526'), (1, '9092.196')] [2023-12-26 22:50:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001025608_262594560.pth... [2023-12-26 22:50:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001026024_262692864.pth... [2023-12-26 22:50:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001024456_262299648.pth [2023-12-26 22:50:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001024872_262397952.pth [2023-12-26 22:50:31,226][105620] Updated weights for policy 1, policy_version 1026034 (0.0005) [2023-12-26 22:50:31,273][105692] Updated weights for policy 0, policy_version 1025615 (0.0008) [2023-12-26 22:50:31,291][105620] Updated weights for policy 1, policy_version 1026044 (0.0008) [2023-12-26 22:50:31,333][105692] Updated weights for policy 0, policy_version 1025625 (0.0008) [2023-12-26 22:50:31,359][105620] Updated weights for policy 1, policy_version 1026054 (0.0010) [2023-12-26 22:50:31,394][105692] Updated weights for policy 0, policy_version 1025635 (0.0008) [2023-12-26 22:50:32,038][105620] Updated weights for policy 1, policy_version 1026064 (0.0007) [2023-12-26 22:50:32,092][105620] Updated weights for policy 1, policy_version 1026074 (0.0006) [2023-12-26 22:50:32,121][105692] Updated weights for policy 0, policy_version 1025645 (0.0011) [2023-12-26 22:50:32,152][105620] Updated weights for policy 1, policy_version 1026084 (0.0006) [2023-12-26 22:50:32,181][105692] Updated weights for policy 0, policy_version 1025655 (0.0011) [2023-12-26 22:50:32,240][105692] Updated weights for policy 0, policy_version 1025665 (0.0011) [2023-12-26 22:50:32,862][105692] Updated weights for policy 0, policy_version 1025675 (0.0011) [2023-12-26 22:50:32,879][105620] Updated weights for policy 1, policy_version 1026094 (0.0010) [2023-12-26 22:50:32,914][105692] Updated weights for policy 0, policy_version 1025685 (0.0010) [2023-12-26 22:50:32,940][105620] Updated weights for policy 1, policy_version 1026104 (0.0008) [2023-12-26 22:50:32,966][105692] Updated weights for policy 0, policy_version 1025695 (0.0010) [2023-12-26 22:50:32,995][105620] Updated weights for policy 1, policy_version 1026114 (0.0010) [2023-12-26 22:50:33,613][105692] Updated weights for policy 0, policy_version 1025705 (0.0010) [2023-12-26 22:50:33,663][105692] Updated weights for policy 0, policy_version 1025715 (0.0005) [2023-12-26 22:50:33,716][105692] Updated weights for policy 0, policy_version 1025725 (0.0007) [2023-12-26 22:50:33,719][105620] Updated weights for policy 1, policy_version 1026124 (0.0010) [2023-12-26 22:50:33,772][105620] Updated weights for policy 1, policy_version 1026134 (0.0007) [2023-12-26 22:50:33,774][105692] Updated weights for policy 0, policy_version 1025735 (0.0010) [2023-12-26 22:50:33,822][105620] Updated weights for policy 1, policy_version 1026144 (0.0005) [2023-12-26 22:50:34,486][105692] Updated weights for policy 0, policy_version 1025745 (0.0011) [2023-12-26 22:50:34,536][105620] Updated weights for policy 1, policy_version 1026154 (0.0006) [2023-12-26 22:50:34,549][105692] Updated weights for policy 0, policy_version 1025755 (0.0010) [2023-12-26 22:50:34,604][105620] Updated weights for policy 1, policy_version 1026164 (0.0007) [2023-12-26 22:50:34,605][105692] Updated weights for policy 0, policy_version 1025765 (0.0010) [2023-12-26 22:50:34,670][105620] Updated weights for policy 1, policy_version 1026174 (0.0006) [2023-12-26 22:50:34,721][105620] Updated weights for policy 1, policy_version 1026184 (0.0008) [2023-12-26 22:50:35,346][105692] Updated weights for policy 0, policy_version 1025775 (0.0011) [2023-12-26 22:50:35,394][105692] Updated weights for policy 0, policy_version 1025785 (0.0010) [2023-12-26 22:50:35,411][105620] Updated weights for policy 1, policy_version 1026194 (0.0010) [2023-12-26 22:50:35,449][105692] Updated weights for policy 0, policy_version 1025795 (0.0010) [2023-12-26 22:50:35,471][105620] Updated weights for policy 1, policy_version 1026204 (0.0011) [2023-12-26 22:50:35,530][105620] Updated weights for policy 1, policy_version 1026214 (0.0011) [2023-12-26 22:50:36,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 525385728. Throughput: 0: 9837.5, 1: 9823.4. Samples: 525377412. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:50:36,063][104569] Avg episode reward: [(0, '8913.057'), (1, '8835.615')] [2023-12-26 22:50:36,211][105692] Updated weights for policy 0, policy_version 1025805 (0.0011) [2023-12-26 22:50:36,275][105692] Updated weights for policy 0, policy_version 1025815 (0.0011) [2023-12-26 22:50:36,278][105620] Updated weights for policy 1, policy_version 1026224 (0.0009) [2023-12-26 22:50:36,334][105692] Updated weights for policy 0, policy_version 1025825 (0.0006) [2023-12-26 22:50:36,338][105620] Updated weights for policy 1, policy_version 1026234 (0.0008) [2023-12-26 22:50:36,434][105620] Updated weights for policy 1, policy_version 1026244 (0.0010) [2023-12-26 22:50:37,054][105692] Updated weights for policy 0, policy_version 1025835 (0.0008) [2023-12-26 22:50:37,119][105692] Updated weights for policy 0, policy_version 1025845 (0.0006) [2023-12-26 22:50:37,167][105692] Updated weights for policy 0, policy_version 1025855 (0.0008) [2023-12-26 22:50:37,176][105620] Updated weights for policy 1, policy_version 1026254 (0.0011) [2023-12-26 22:50:37,232][105620] Updated weights for policy 1, policy_version 1026264 (0.0010) [2023-12-26 22:50:37,286][105620] Updated weights for policy 1, policy_version 1026274 (0.0011) [2023-12-26 22:50:37,730][105692] Updated weights for policy 0, policy_version 1025865 (0.0006) [2023-12-26 22:50:37,789][105692] Updated weights for policy 0, policy_version 1025875 (0.0005) [2023-12-26 22:50:37,837][105692] Updated weights for policy 0, policy_version 1025885 (0.0005) [2023-12-26 22:50:37,892][105692] Updated weights for policy 0, policy_version 1025895 (0.0005) [2023-12-26 22:50:37,918][105620] Updated weights for policy 1, policy_version 1026284 (0.0009) [2023-12-26 22:50:37,978][105620] Updated weights for policy 1, policy_version 1026294 (0.0005) [2023-12-26 22:50:38,048][105620] Updated weights for policy 1, policy_version 1026304 (0.0006) [2023-12-26 22:50:38,429][105692] Updated weights for policy 0, policy_version 1025905 (0.0005) [2023-12-26 22:50:38,497][105692] Updated weights for policy 0, policy_version 1025915 (0.0009) [2023-12-26 22:50:38,552][105692] Updated weights for policy 0, policy_version 1025925 (0.0010) [2023-12-26 22:50:38,731][105620] Updated weights for policy 1, policy_version 1026314 (0.0009) [2023-12-26 22:50:38,792][105620] Updated weights for policy 1, policy_version 1026324 (0.0010) [2023-12-26 22:50:38,851][105620] Updated weights for policy 1, policy_version 1026334 (0.0010) [2023-12-26 22:50:38,917][105620] Updated weights for policy 1, policy_version 1026344 (0.0010) [2023-12-26 22:50:39,229][105692] Updated weights for policy 0, policy_version 1025935 (0.0010) [2023-12-26 22:50:39,291][105692] Updated weights for policy 0, policy_version 1025945 (0.0010) [2023-12-26 22:50:39,358][105692] Updated weights for policy 0, policy_version 1025955 (0.0011) [2023-12-26 22:50:39,685][105620] Updated weights for policy 1, policy_version 1026354 (0.0011) [2023-12-26 22:50:39,749][105620] Updated weights for policy 1, policy_version 1026364 (0.0011) [2023-12-26 22:50:39,809][105620] Updated weights for policy 1, policy_version 1026374 (0.0011) [2023-12-26 22:50:40,114][105692] Updated weights for policy 0, policy_version 1025965 (0.0011) [2023-12-26 22:50:40,185][105692] Updated weights for policy 0, policy_version 1025975 (0.0011) [2023-12-26 22:50:40,253][105692] Updated weights for policy 0, policy_version 1025985 (0.0011) [2023-12-26 22:50:40,522][105620] Updated weights for policy 1, policy_version 1026384 (0.0011) [2023-12-26 22:50:40,588][105620] Updated weights for policy 1, policy_version 1026394 (0.0010) [2023-12-26 22:50:40,654][105620] Updated weights for policy 1, policy_version 1026404 (0.0010) [2023-12-26 22:50:40,939][105692] Updated weights for policy 0, policy_version 1025995 (0.0011) [2023-12-26 22:50:40,988][105692] Updated weights for policy 0, policy_version 1026005 (0.0005) [2023-12-26 22:50:41,051][105692] Updated weights for policy 0, policy_version 1026015 (0.0007) [2023-12-26 22:50:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 525484032. Throughput: 0: 9932.4, 1: 9826.4. Samples: 525495332. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:50:41,062][104569] Avg episode reward: [(0, '8911.075'), (1, '8927.285')] [2023-12-26 22:50:41,434][105620] Updated weights for policy 1, policy_version 1026414 (0.0009) [2023-12-26 22:50:41,495][105620] Updated weights for policy 1, policy_version 1026424 (0.0010) [2023-12-26 22:50:41,559][105620] Updated weights for policy 1, policy_version 1026434 (0.0011) [2023-12-26 22:50:41,759][105692] Updated weights for policy 0, policy_version 1026025 (0.0008) [2023-12-26 22:50:41,825][105692] Updated weights for policy 0, policy_version 1026035 (0.0010) [2023-12-26 22:50:41,887][105692] Updated weights for policy 0, policy_version 1026045 (0.0011) [2023-12-26 22:50:41,955][105692] Updated weights for policy 0, policy_version 1026055 (0.0011) [2023-12-26 22:50:42,236][105620] Updated weights for policy 1, policy_version 1026444 (0.0009) [2023-12-26 22:50:42,301][105620] Updated weights for policy 1, policy_version 1026454 (0.0008) [2023-12-26 22:50:42,373][105620] Updated weights for policy 1, policy_version 1026464 (0.0008) [2023-12-26 22:50:42,620][105692] Updated weights for policy 0, policy_version 1026065 (0.0007) [2023-12-26 22:50:42,677][105692] Updated weights for policy 0, policy_version 1026075 (0.0009) [2023-12-26 22:50:42,731][105692] Updated weights for policy 0, policy_version 1026085 (0.0010) [2023-12-26 22:50:43,087][105620] Updated weights for policy 1, policy_version 1026474 (0.0007) [2023-12-26 22:50:43,141][105620] Updated weights for policy 1, policy_version 1026484 (0.0005) [2023-12-26 22:50:43,201][105620] Updated weights for policy 1, policy_version 1026494 (0.0005) [2023-12-26 22:50:43,269][105620] Updated weights for policy 1, policy_version 1026504 (0.0005) [2023-12-26 22:50:43,350][105692] Updated weights for policy 0, policy_version 1026095 (0.0007) [2023-12-26 22:50:43,417][105692] Updated weights for policy 0, policy_version 1026105 (0.0007) [2023-12-26 22:50:43,478][105692] Updated weights for policy 0, policy_version 1026115 (0.0008) [2023-12-26 22:50:43,954][105620] Updated weights for policy 1, policy_version 1026515 (0.0009) [2023-12-26 22:50:44,008][105620] Updated weights for policy 1, policy_version 1026525 (0.0009) [2023-12-26 22:50:44,066][105620] Updated weights for policy 1, policy_version 1026535 (0.0011) [2023-12-26 22:50:44,111][105692] Updated weights for policy 0, policy_version 1026125 (0.0008) [2023-12-26 22:50:44,171][105692] Updated weights for policy 0, policy_version 1026135 (0.0005) [2023-12-26 22:50:44,219][105692] Updated weights for policy 0, policy_version 1026145 (0.0006) [2023-12-26 22:50:44,769][105620] Updated weights for policy 1, policy_version 1026545 (0.0008) [2023-12-26 22:50:44,827][105692] Updated weights for policy 0, policy_version 1026155 (0.0007) [2023-12-26 22:50:44,829][105620] Updated weights for policy 1, policy_version 1026555 (0.0008) [2023-12-26 22:50:44,889][105620] Updated weights for policy 1, policy_version 1026565 (0.0006) [2023-12-26 22:50:44,891][105692] Updated weights for policy 0, policy_version 1026165 (0.0011) [2023-12-26 22:50:44,944][105692] Updated weights for policy 0, policy_version 1026175 (0.0010) [2023-12-26 22:50:45,662][105620] Updated weights for policy 1, policy_version 1026575 (0.0007) [2023-12-26 22:50:45,713][105620] Updated weights for policy 1, policy_version 1026585 (0.0008) [2023-12-26 22:50:45,714][105692] Updated weights for policy 0, policy_version 1026185 (0.0010) [2023-12-26 22:50:45,760][105620] Updated weights for policy 1, policy_version 1026595 (0.0008) [2023-12-26 22:50:45,764][105692] Updated weights for policy 0, policy_version 1026195 (0.0006) [2023-12-26 22:50:45,819][105692] Updated weights for policy 0, policy_version 1026205 (0.0006) [2023-12-26 22:50:45,872][105692] Updated weights for policy 0, policy_version 1026215 (0.0006) [2023-12-26 22:50:46,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 525590528. Throughput: 0: 9948.8, 1: 9808.4. Samples: 525555640. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:50:46,062][104569] Avg episode reward: [(0, '9176.746'), (1, '9056.244')] [2023-12-26 22:50:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001026216_262750208.pth... [2023-12-26 22:50:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001026600_262840320.pth... [2023-12-26 22:50:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001025032_262447104.pth [2023-12-26 22:50:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001025480_262553600.pth [2023-12-26 22:50:46,502][105620] Updated weights for policy 1, policy_version 1026605 (0.0007) [2023-12-26 22:50:46,514][105692] Updated weights for policy 0, policy_version 1026225 (0.0008) [2023-12-26 22:50:46,553][105620] Updated weights for policy 1, policy_version 1026615 (0.0006) [2023-12-26 22:50:46,571][105692] Updated weights for policy 0, policy_version 1026235 (0.0007) [2023-12-26 22:50:46,602][105620] Updated weights for policy 1, policy_version 1026625 (0.0006) [2023-12-26 22:50:46,624][105692] Updated weights for policy 0, policy_version 1026245 (0.0007) [2023-12-26 22:50:47,254][105620] Updated weights for policy 1, policy_version 1026635 (0.0007) [2023-12-26 22:50:47,303][105692] Updated weights for policy 0, policy_version 1026255 (0.0009) [2023-12-26 22:50:47,305][105620] Updated weights for policy 1, policy_version 1026645 (0.0006) [2023-12-26 22:50:47,358][105620] Updated weights for policy 1, policy_version 1026655 (0.0006) [2023-12-26 22:50:47,370][105692] Updated weights for policy 0, policy_version 1026265 (0.0010) [2023-12-26 22:50:47,439][105692] Updated weights for policy 0, policy_version 1026275 (0.0009) [2023-12-26 22:50:47,926][105620] Updated weights for policy 1, policy_version 1026665 (0.0007) [2023-12-26 22:50:47,997][105620] Updated weights for policy 1, policy_version 1026675 (0.0010) [2023-12-26 22:50:48,063][105620] Updated weights for policy 1, policy_version 1026685 (0.0006) [2023-12-26 22:50:48,076][105692] Updated weights for policy 0, policy_version 1026285 (0.0008) [2023-12-26 22:50:48,126][105620] Updated weights for policy 1, policy_version 1026695 (0.0011) [2023-12-26 22:50:48,129][105692] Updated weights for policy 0, policy_version 1026295 (0.0006) [2023-12-26 22:50:48,185][105692] Updated weights for policy 0, policy_version 1026305 (0.0007) [2023-12-26 22:50:48,785][105620] Updated weights for policy 1, policy_version 1026705 (0.0008) [2023-12-26 22:50:48,851][105620] Updated weights for policy 1, policy_version 1026715 (0.0009) [2023-12-26 22:50:48,906][105620] Updated weights for policy 1, policy_version 1026725 (0.0009) [2023-12-26 22:50:48,935][105692] Updated weights for policy 0, policy_version 1026315 (0.0008) [2023-12-26 22:50:48,990][105692] Updated weights for policy 0, policy_version 1026325 (0.0009) [2023-12-26 22:50:49,049][105692] Updated weights for policy 0, policy_version 1026335 (0.0010) [2023-12-26 22:50:49,622][105620] Updated weights for policy 1, policy_version 1026735 (0.0006) [2023-12-26 22:50:49,683][105620] Updated weights for policy 1, policy_version 1026745 (0.0007) [2023-12-26 22:50:49,749][105620] Updated weights for policy 1, policy_version 1026755 (0.0009) [2023-12-26 22:50:49,880][105692] Updated weights for policy 0, policy_version 1026345 (0.0009) [2023-12-26 22:50:49,942][105692] Updated weights for policy 0, policy_version 1026355 (0.0009) [2023-12-26 22:50:49,994][105692] Updated weights for policy 0, policy_version 1026365 (0.0009) [2023-12-26 22:50:50,051][105692] Updated weights for policy 0, policy_version 1026375 (0.0008) [2023-12-26 22:50:50,476][105620] Updated weights for policy 1, policy_version 1026765 (0.0010) [2023-12-26 22:50:50,534][105620] Updated weights for policy 1, policy_version 1026775 (0.0010) [2023-12-26 22:50:50,603][105620] Updated weights for policy 1, policy_version 1026785 (0.0010) [2023-12-26 22:50:50,857][105692] Updated weights for policy 0, policy_version 1026385 (0.0008) [2023-12-26 22:50:50,921][105692] Updated weights for policy 0, policy_version 1026395 (0.0010) [2023-12-26 22:50:50,974][105692] Updated weights for policy 0, policy_version 1026405 (0.0010) [2023-12-26 22:50:51,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 525688832. Throughput: 0: 9976.2, 1: 9771.1. Samples: 525675996. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:50:51,063][104569] Avg episode reward: [(0, '9266.851'), (1, '9056.798')] [2023-12-26 22:50:51,241][105620] Updated weights for policy 1, policy_version 1026795 (0.0011) [2023-12-26 22:50:51,302][105620] Updated weights for policy 1, policy_version 1026805 (0.0010) [2023-12-26 22:50:51,369][105620] Updated weights for policy 1, policy_version 1026815 (0.0009) [2023-12-26 22:50:51,808][105692] Updated weights for policy 0, policy_version 1026415 (0.0009) [2023-12-26 22:50:51,860][105692] Updated weights for policy 0, policy_version 1026425 (0.0009) [2023-12-26 22:50:51,923][105692] Updated weights for policy 0, policy_version 1026435 (0.0008) [2023-12-26 22:50:52,105][105620] Updated weights for policy 1, policy_version 1026825 (0.0007) [2023-12-26 22:50:52,173][105620] Updated weights for policy 1, policy_version 1026835 (0.0010) [2023-12-26 22:50:52,231][105620] Updated weights for policy 1, policy_version 1026845 (0.0010) [2023-12-26 22:50:52,287][105620] Updated weights for policy 1, policy_version 1026855 (0.0008) [2023-12-26 22:50:52,577][105692] Updated weights for policy 0, policy_version 1026445 (0.0008) [2023-12-26 22:50:52,638][105692] Updated weights for policy 0, policy_version 1026455 (0.0008) [2023-12-26 22:50:52,702][105692] Updated weights for policy 0, policy_version 1026465 (0.0007) [2023-12-26 22:50:53,015][105620] Updated weights for policy 1, policy_version 1026865 (0.0010) [2023-12-26 22:50:53,067][105620] Updated weights for policy 1, policy_version 1026875 (0.0010) [2023-12-26 22:50:53,129][105620] Updated weights for policy 1, policy_version 1026885 (0.0010) [2023-12-26 22:50:53,343][105692] Updated weights for policy 0, policy_version 1026475 (0.0007) [2023-12-26 22:50:53,405][105692] Updated weights for policy 0, policy_version 1026485 (0.0005) [2023-12-26 22:50:53,452][105692] Updated weights for policy 0, policy_version 1026495 (0.0005) [2023-12-26 22:50:53,830][105620] Updated weights for policy 1, policy_version 1026895 (0.0007) [2023-12-26 22:50:53,881][105620] Updated weights for policy 1, policy_version 1026905 (0.0005) [2023-12-26 22:50:53,930][105620] Updated weights for policy 1, policy_version 1026915 (0.0008) [2023-12-26 22:50:54,152][105692] Updated weights for policy 0, policy_version 1026505 (0.0006) [2023-12-26 22:50:54,206][105692] Updated weights for policy 0, policy_version 1026515 (0.0009) [2023-12-26 22:50:54,264][105692] Updated weights for policy 0, policy_version 1026525 (0.0008) [2023-12-26 22:50:54,322][105692] Updated weights for policy 0, policy_version 1026536 (0.0010) [2023-12-26 22:50:54,536][105620] Updated weights for policy 1, policy_version 1026925 (0.0005) [2023-12-26 22:50:54,588][105620] Updated weights for policy 1, policy_version 1026935 (0.0005) [2023-12-26 22:50:54,639][105620] Updated weights for policy 1, policy_version 1026945 (0.0006) [2023-12-26 22:50:55,093][105692] Updated weights for policy 0, policy_version 1026546 (0.0007) [2023-12-26 22:50:55,148][105692] Updated weights for policy 0, policy_version 1026556 (0.0009) [2023-12-26 22:50:55,200][105692] Updated weights for policy 0, policy_version 1026567 (0.0007) [2023-12-26 22:50:55,300][105620] Updated weights for policy 1, policy_version 1026955 (0.0006) [2023-12-26 22:50:55,366][105620] Updated weights for policy 1, policy_version 1026965 (0.0009) [2023-12-26 22:50:55,421][105620] Updated weights for policy 1, policy_version 1026975 (0.0010) [2023-12-26 22:50:55,881][105692] Updated weights for policy 0, policy_version 1026578 (0.0010) [2023-12-26 22:50:55,937][105692] Updated weights for policy 0, policy_version 1026589 (0.0010) [2023-12-26 22:50:55,995][105692] Updated weights for policy 0, policy_version 1026599 (0.0010) [2023-12-26 22:50:56,060][105620] Updated weights for policy 1, policy_version 1026985 (0.0010) [2023-12-26 22:50:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 525787136. Throughput: 0: 9991.0, 1: 9769.7. Samples: 525793988. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:50:56,063][104569] Avg episode reward: [(0, '9086.581'), (1, '9146.108')] [2023-12-26 22:50:56,116][105620] Updated weights for policy 1, policy_version 1026995 (0.0007) [2023-12-26 22:50:56,167][105620] Updated weights for policy 1, policy_version 1027005 (0.0010) [2023-12-26 22:50:56,221][105620] Updated weights for policy 1, policy_version 1027015 (0.0010) [2023-12-26 22:50:56,767][105692] Updated weights for policy 0, policy_version 1026609 (0.0006) [2023-12-26 22:50:56,822][105692] Updated weights for policy 0, policy_version 1026619 (0.0007) [2023-12-26 22:50:56,845][105620] Updated weights for policy 1, policy_version 1027025 (0.0006) [2023-12-26 22:50:56,866][105692] Updated weights for policy 0, policy_version 1026629 (0.0007) [2023-12-26 22:50:56,903][105620] Updated weights for policy 1, policy_version 1027035 (0.0009) [2023-12-26 22:50:56,951][105620] Updated weights for policy 1, policy_version 1027045 (0.0010) [2023-12-26 22:50:57,583][105620] Updated weights for policy 1, policy_version 1027055 (0.0010) [2023-12-26 22:50:57,630][105620] Updated weights for policy 1, policy_version 1027065 (0.0009) [2023-12-26 22:50:57,635][105692] Updated weights for policy 0, policy_version 1026639 (0.0008) [2023-12-26 22:50:57,673][105620] Updated weights for policy 1, policy_version 1027075 (0.0005) [2023-12-26 22:50:57,697][105692] Updated weights for policy 0, policy_version 1026649 (0.0008) [2023-12-26 22:50:57,759][105692] Updated weights for policy 0, policy_version 1026659 (0.0010) [2023-12-26 22:50:58,245][105620] Updated weights for policy 1, policy_version 1027085 (0.0006) [2023-12-26 22:50:58,303][105620] Updated weights for policy 1, policy_version 1027095 (0.0007) [2023-12-26 22:50:58,367][105620] Updated weights for policy 1, policy_version 1027105 (0.0007) [2023-12-26 22:50:58,504][105692] Updated weights for policy 0, policy_version 1026669 (0.0009) [2023-12-26 22:50:58,575][105692] Updated weights for policy 0, policy_version 1026679 (0.0010) [2023-12-26 22:50:58,637][105692] Updated weights for policy 0, policy_version 1026689 (0.0009) [2023-12-26 22:50:59,154][105620] Updated weights for policy 1, policy_version 1027115 (0.0009) [2023-12-26 22:50:59,222][105620] Updated weights for policy 1, policy_version 1027125 (0.0009) [2023-12-26 22:50:59,283][105620] Updated weights for policy 1, policy_version 1027135 (0.0010) [2023-12-26 22:50:59,437][105692] Updated weights for policy 0, policy_version 1026699 (0.0008) [2023-12-26 22:50:59,495][105692] Updated weights for policy 0, policy_version 1026709 (0.0005) [2023-12-26 22:50:59,554][105692] Updated weights for policy 0, policy_version 1026719 (0.0005) [2023-12-26 22:51:00,027][105620] Updated weights for policy 1, policy_version 1027145 (0.0011) [2023-12-26 22:51:00,072][105620] Updated weights for policy 1, policy_version 1027155 (0.0010) [2023-12-26 22:51:00,130][105620] Updated weights for policy 1, policy_version 1027165 (0.0010) [2023-12-26 22:51:00,186][105620] Updated weights for policy 1, policy_version 1027175 (0.0011) [2023-12-26 22:51:00,218][105692] Updated weights for policy 0, policy_version 1026729 (0.0009) [2023-12-26 22:51:00,285][105692] Updated weights for policy 0, policy_version 1026739 (0.0006) [2023-12-26 22:51:00,347][105692] Updated weights for policy 0, policy_version 1026749 (0.0005) [2023-12-26 22:51:00,405][105692] Updated weights for policy 0, policy_version 1026759 (0.0008) [2023-12-26 22:51:00,953][105620] Updated weights for policy 1, policy_version 1027185 (0.0010) [2023-12-26 22:51:01,001][105620] Updated weights for policy 1, policy_version 1027195 (0.0010) [2023-12-26 22:51:01,021][105692] Updated weights for policy 0, policy_version 1026769 (0.0006) [2023-12-26 22:51:01,060][105620] Updated weights for policy 1, policy_version 1027205 (0.0009) [2023-12-26 22:51:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 525877248. Throughput: 0: 9939.8, 1: 9878.3. Samples: 525854180. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:51:01,062][104569] Avg episode reward: [(0, '8734.507'), (1, '9353.554')] [2023-12-26 22:51:01,079][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001027208_262995968.pth... [2023-12-26 22:51:01,079][105692] Updated weights for policy 0, policy_version 1026779 (0.0008) [2023-12-26 22:51:01,084][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001026024_262692864.pth [2023-12-26 22:51:01,146][105692] Updated weights for policy 0, policy_version 1026789 (0.0008) [2023-12-26 22:51:01,161][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001026792_262897664.pth... [2023-12-26 22:51:01,165][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001025608_262594560.pth [2023-12-26 22:51:01,747][105620] Updated weights for policy 1, policy_version 1027215 (0.0008) [2023-12-26 22:51:01,802][105620] Updated weights for policy 1, policy_version 1027225 (0.0011) [2023-12-26 22:51:01,857][105620] Updated weights for policy 1, policy_version 1027235 (0.0011) [2023-12-26 22:51:01,886][105692] Updated weights for policy 0, policy_version 1026799 (0.0008) [2023-12-26 22:51:01,943][105692] Updated weights for policy 0, policy_version 1026809 (0.0007) [2023-12-26 22:51:02,004][105692] Updated weights for policy 0, policy_version 1026819 (0.0008) [2023-12-26 22:51:02,573][105620] Updated weights for policy 1, policy_version 1027245 (0.0008) [2023-12-26 22:51:02,633][105620] Updated weights for policy 1, policy_version 1027255 (0.0005) [2023-12-26 22:51:02,681][105620] Updated weights for policy 1, policy_version 1027265 (0.0008) [2023-12-26 22:51:02,782][105692] Updated weights for policy 0, policy_version 1026829 (0.0009) [2023-12-26 22:51:02,843][105692] Updated weights for policy 0, policy_version 1026839 (0.0008) [2023-12-26 22:51:02,892][105692] Updated weights for policy 0, policy_version 1026849 (0.0008) [2023-12-26 22:51:03,348][105620] Updated weights for policy 1, policy_version 1027275 (0.0009) [2023-12-26 22:51:03,399][105620] Updated weights for policy 1, policy_version 1027285 (0.0009) [2023-12-26 22:51:03,450][105620] Updated weights for policy 1, policy_version 1027295 (0.0009) [2023-12-26 22:51:03,706][105692] Updated weights for policy 0, policy_version 1026859 (0.0009) [2023-12-26 22:51:03,768][105692] Updated weights for policy 0, policy_version 1026869 (0.0010) [2023-12-26 22:51:03,833][105692] Updated weights for policy 0, policy_version 1026879 (0.0010) [2023-12-26 22:51:04,063][105620] Updated weights for policy 1, policy_version 1027305 (0.0006) [2023-12-26 22:51:04,133][105620] Updated weights for policy 1, policy_version 1027315 (0.0006) [2023-12-26 22:51:04,192][105620] Updated weights for policy 1, policy_version 1027325 (0.0006) [2023-12-26 22:51:04,256][105620] Updated weights for policy 1, policy_version 1027335 (0.0007) [2023-12-26 22:51:04,703][105692] Updated weights for policy 0, policy_version 1026889 (0.0009) [2023-12-26 22:51:04,772][105692] Updated weights for policy 0, policy_version 1026899 (0.0009) [2023-12-26 22:51:04,813][105620] Updated weights for policy 1, policy_version 1027345 (0.0005) [2023-12-26 22:51:04,835][105692] Updated weights for policy 0, policy_version 1026909 (0.0008) [2023-12-26 22:51:04,874][105620] Updated weights for policy 1, policy_version 1027355 (0.0005) [2023-12-26 22:51:04,888][105692] Updated weights for policy 0, policy_version 1026919 (0.0009) [2023-12-26 22:51:04,932][105620] Updated weights for policy 1, policy_version 1027365 (0.0007) [2023-12-26 22:51:05,492][105620] Updated weights for policy 1, policy_version 1027375 (0.0007) [2023-12-26 22:51:05,549][105620] Updated weights for policy 1, policy_version 1027386 (0.0010) [2023-12-26 22:51:05,583][105692] Updated weights for policy 0, policy_version 1026929 (0.0006) [2023-12-26 22:51:05,600][105620] Updated weights for policy 1, policy_version 1027396 (0.0006) [2023-12-26 22:51:05,626][105692] Updated weights for policy 0, policy_version 1026939 (0.0005) [2023-12-26 22:51:05,670][105692] Updated weights for policy 0, policy_version 1026949 (0.0005) [2023-12-26 22:51:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 525983744. Throughput: 0: 9829.2, 1: 9931.4. Samples: 525971136. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:51:06,062][104569] Avg episode reward: [(0, '8588.090'), (1, '9353.331')] [2023-12-26 22:51:06,260][105620] Updated weights for policy 1, policy_version 1027406 (0.0009) [2023-12-26 22:51:06,285][105692] Updated weights for policy 0, policy_version 1026959 (0.0006) [2023-12-26 22:51:06,326][105620] Updated weights for policy 1, policy_version 1027416 (0.0007) [2023-12-26 22:51:06,353][105692] Updated weights for policy 0, policy_version 1026969 (0.0006) [2023-12-26 22:51:06,390][105620] Updated weights for policy 1, policy_version 1027426 (0.0006) [2023-12-26 22:51:06,419][105692] Updated weights for policy 0, policy_version 1026979 (0.0006) [2023-12-26 22:51:07,037][105692] Updated weights for policy 0, policy_version 1026989 (0.0008) [2023-12-26 22:51:07,069][105620] Updated weights for policy 1, policy_version 1027436 (0.0007) [2023-12-26 22:51:07,089][105692] Updated weights for policy 0, policy_version 1026999 (0.0007) [2023-12-26 22:51:07,128][105620] Updated weights for policy 1, policy_version 1027446 (0.0006) [2023-12-26 22:51:07,149][105692] Updated weights for policy 0, policy_version 1027009 (0.0011) [2023-12-26 22:51:07,184][105620] Updated weights for policy 1, policy_version 1027456 (0.0005) [2023-12-26 22:51:07,762][105620] Updated weights for policy 1, policy_version 1027466 (0.0006) [2023-12-26 22:51:07,819][105620] Updated weights for policy 1, policy_version 1027477 (0.0010) [2023-12-26 22:51:07,841][105692] Updated weights for policy 0, policy_version 1027019 (0.0008) [2023-12-26 22:51:07,875][105620] Updated weights for policy 1, policy_version 1027487 (0.0008) [2023-12-26 22:51:07,892][105692] Updated weights for policy 0, policy_version 1027029 (0.0005) [2023-12-26 22:51:07,946][105692] Updated weights for policy 0, policy_version 1027039 (0.0005) [2023-12-26 22:51:08,535][105692] Updated weights for policy 0, policy_version 1027049 (0.0006) [2023-12-26 22:51:08,596][105692] Updated weights for policy 0, policy_version 1027059 (0.0008) [2023-12-26 22:51:08,664][105692] Updated weights for policy 0, policy_version 1027069 (0.0006) [2023-12-26 22:51:08,675][105620] Updated weights for policy 1, policy_version 1027497 (0.0010) [2023-12-26 22:51:08,722][105692] Updated weights for policy 0, policy_version 1027079 (0.0006) [2023-12-26 22:51:08,740][105620] Updated weights for policy 1, policy_version 1027507 (0.0005) [2023-12-26 22:51:08,794][105620] Updated weights for policy 1, policy_version 1027517 (0.0005) [2023-12-26 22:51:08,849][105620] Updated weights for policy 1, policy_version 1027527 (0.0005) [2023-12-26 22:51:09,295][105692] Updated weights for policy 0, policy_version 1027089 (0.0008) [2023-12-26 22:51:09,369][105692] Updated weights for policy 0, policy_version 1027099 (0.0009) [2023-12-26 22:51:09,435][105692] Updated weights for policy 0, policy_version 1027109 (0.0011) [2023-12-26 22:51:09,534][105620] Updated weights for policy 1, policy_version 1027537 (0.0008) [2023-12-26 22:51:09,588][105620] Updated weights for policy 1, policy_version 1027547 (0.0008) [2023-12-26 22:51:09,644][105620] Updated weights for policy 1, policy_version 1027557 (0.0010) [2023-12-26 22:51:10,093][105692] Updated weights for policy 0, policy_version 1027119 (0.0009) [2023-12-26 22:51:10,153][105692] Updated weights for policy 0, policy_version 1027129 (0.0009) [2023-12-26 22:51:10,203][105692] Updated weights for policy 0, policy_version 1027139 (0.0009) [2023-12-26 22:51:10,504][105620] Updated weights for policy 1, policy_version 1027567 (0.0008) [2023-12-26 22:51:10,569][105620] Updated weights for policy 1, policy_version 1027577 (0.0009) [2023-12-26 22:51:10,622][105620] Updated weights for policy 1, policy_version 1027587 (0.0010) [2023-12-26 22:51:10,866][105692] Updated weights for policy 0, policy_version 1027149 (0.0009) [2023-12-26 22:51:10,920][105692] Updated weights for policy 0, policy_version 1027159 (0.0007) [2023-12-26 22:51:10,973][105692] Updated weights for policy 0, policy_version 1027169 (0.0006) [2023-12-26 22:51:11,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 526090240. Throughput: 0: 9939.1, 1: 10014.7. Samples: 526094368. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:51:11,062][104569] Avg episode reward: [(0, '9027.416'), (1, '9263.544')] [2023-12-26 22:51:11,465][105620] Updated weights for policy 1, policy_version 1027597 (0.0007) [2023-12-26 22:51:11,528][105620] Updated weights for policy 1, policy_version 1027607 (0.0007) [2023-12-26 22:51:11,591][105620] Updated weights for policy 1, policy_version 1027617 (0.0009) [2023-12-26 22:51:11,818][105692] Updated weights for policy 0, policy_version 1027179 (0.0007) [2023-12-26 22:51:11,871][105692] Updated weights for policy 0, policy_version 1027189 (0.0008) [2023-12-26 22:51:11,919][105692] Updated weights for policy 0, policy_version 1027199 (0.0005) [2023-12-26 22:51:12,363][105620] Updated weights for policy 1, policy_version 1027627 (0.0009) [2023-12-26 22:51:12,431][105620] Updated weights for policy 1, policy_version 1027637 (0.0010) [2023-12-26 22:51:12,486][105620] Updated weights for policy 1, policy_version 1027647 (0.0009) [2023-12-26 22:51:12,626][105692] Updated weights for policy 0, policy_version 1027209 (0.0006) [2023-12-26 22:51:12,686][105692] Updated weights for policy 0, policy_version 1027219 (0.0008) [2023-12-26 22:51:12,746][105692] Updated weights for policy 0, policy_version 1027229 (0.0008) [2023-12-26 22:51:12,799][105692] Updated weights for policy 0, policy_version 1027239 (0.0008) [2023-12-26 22:51:13,261][105620] Updated weights for policy 1, policy_version 1027657 (0.0009) [2023-12-26 22:51:13,322][105620] Updated weights for policy 1, policy_version 1027667 (0.0010) [2023-12-26 22:51:13,386][105620] Updated weights for policy 1, policy_version 1027677 (0.0010) [2023-12-26 22:51:13,451][105620] Updated weights for policy 1, policy_version 1027687 (0.0010) [2023-12-26 22:51:13,543][105692] Updated weights for policy 0, policy_version 1027249 (0.0006) [2023-12-26 22:51:13,593][105692] Updated weights for policy 0, policy_version 1027259 (0.0008) [2023-12-26 22:51:13,651][105692] Updated weights for policy 0, policy_version 1027269 (0.0005) [2023-12-26 22:51:14,115][105620] Updated weights for policy 1, policy_version 1027697 (0.0006) [2023-12-26 22:51:14,176][105620] Updated weights for policy 1, policy_version 1027707 (0.0005) [2023-12-26 22:51:14,228][105620] Updated weights for policy 1, policy_version 1027717 (0.0006) [2023-12-26 22:51:14,424][105692] Updated weights for policy 0, policy_version 1027279 (0.0008) [2023-12-26 22:51:14,497][105692] Updated weights for policy 0, policy_version 1027289 (0.0009) [2023-12-26 22:51:14,557][105692] Updated weights for policy 0, policy_version 1027299 (0.0009) [2023-12-26 22:51:14,825][105620] Updated weights for policy 1, policy_version 1027727 (0.0007) [2023-12-26 22:51:14,888][105620] Updated weights for policy 1, policy_version 1027737 (0.0008) [2023-12-26 22:51:14,940][105620] Updated weights for policy 1, policy_version 1027747 (0.0008) [2023-12-26 22:51:15,265][105692] Updated weights for policy 0, policy_version 1027309 (0.0008) [2023-12-26 22:51:15,312][105692] Updated weights for policy 0, policy_version 1027319 (0.0009) [2023-12-26 22:51:15,361][105692] Updated weights for policy 0, policy_version 1027329 (0.0009) [2023-12-26 22:51:15,612][105620] Updated weights for policy 1, policy_version 1027757 (0.0008) [2023-12-26 22:51:15,673][105620] Updated weights for policy 1, policy_version 1027767 (0.0009) [2023-12-26 22:51:15,734][105620] Updated weights for policy 1, policy_version 1027777 (0.0007) [2023-12-26 22:51:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 526180352. Throughput: 0: 9904.9, 1: 9939.1. Samples: 526150496. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:51:16,062][104569] Avg episode reward: [(0, '9172.282'), (1, '9082.311')] [2023-12-26 22:51:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001027784_263143424.pth... [2023-12-26 22:51:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001027336_263036928.pth... [2023-12-26 22:51:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001026600_262840320.pth [2023-12-26 22:51:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001026216_262750208.pth [2023-12-26 22:51:16,151][105692] Updated weights for policy 0, policy_version 1027339 (0.0009) [2023-12-26 22:51:16,216][105692] Updated weights for policy 0, policy_version 1027349 (0.0009) [2023-12-26 22:51:16,280][105692] Updated weights for policy 0, policy_version 1027359 (0.0008) [2023-12-26 22:51:16,445][105620] Updated weights for policy 1, policy_version 1027787 (0.0010) [2023-12-26 22:51:16,502][105620] Updated weights for policy 1, policy_version 1027797 (0.0010) [2023-12-26 22:51:16,560][105620] Updated weights for policy 1, policy_version 1027807 (0.0009) [2023-12-26 22:51:17,048][105692] Updated weights for policy 0, policy_version 1027369 (0.0007) [2023-12-26 22:51:17,100][105692] Updated weights for policy 0, policy_version 1027379 (0.0008) [2023-12-26 22:51:17,144][105692] Updated weights for policy 0, policy_version 1027389 (0.0008) [2023-12-26 22:51:17,193][105692] Updated weights for policy 0, policy_version 1027399 (0.0008) [2023-12-26 22:51:17,333][105620] Updated weights for policy 1, policy_version 1027817 (0.0009) [2023-12-26 22:51:17,387][105620] Updated weights for policy 1, policy_version 1027827 (0.0010) [2023-12-26 22:51:17,452][105620] Updated weights for policy 1, policy_version 1027837 (0.0010) [2023-12-26 22:51:17,508][105620] Updated weights for policy 1, policy_version 1027847 (0.0006) [2023-12-26 22:51:18,025][105692] Updated weights for policy 0, policy_version 1027409 (0.0008) [2023-12-26 22:51:18,073][105692] Updated weights for policy 0, policy_version 1027419 (0.0008) [2023-12-26 22:51:18,120][105620] Updated weights for policy 1, policy_version 1027857 (0.0010) [2023-12-26 22:51:18,127][105692] Updated weights for policy 0, policy_version 1027429 (0.0009) [2023-12-26 22:51:18,178][105620] Updated weights for policy 1, policy_version 1027867 (0.0007) [2023-12-26 22:51:18,241][105620] Updated weights for policy 1, policy_version 1027877 (0.0005) [2023-12-26 22:51:18,822][105620] Updated weights for policy 1, policy_version 1027887 (0.0007) [2023-12-26 22:51:18,885][105620] Updated weights for policy 1, policy_version 1027897 (0.0007) [2023-12-26 22:51:18,898][105692] Updated weights for policy 0, policy_version 1027439 (0.0011) [2023-12-26 22:51:18,946][105620] Updated weights for policy 1, policy_version 1027907 (0.0008) [2023-12-26 22:51:18,947][105692] Updated weights for policy 0, policy_version 1027449 (0.0006) [2023-12-26 22:51:18,995][105692] Updated weights for policy 0, policy_version 1027459 (0.0010) [2023-12-26 22:51:19,665][105620] Updated weights for policy 1, policy_version 1027917 (0.0007) [2023-12-26 22:51:19,696][105692] Updated weights for policy 0, policy_version 1027469 (0.0009) [2023-12-26 22:51:19,729][105620] Updated weights for policy 1, policy_version 1027927 (0.0008) [2023-12-26 22:51:19,756][105692] Updated weights for policy 0, policy_version 1027479 (0.0007) [2023-12-26 22:51:19,791][105620] Updated weights for policy 1, policy_version 1027937 (0.0010) [2023-12-26 22:51:19,818][105692] Updated weights for policy 0, policy_version 1027489 (0.0007) [2023-12-26 22:51:20,483][105692] Updated weights for policy 0, policy_version 1027499 (0.0008) [2023-12-26 22:51:20,538][105620] Updated weights for policy 1, policy_version 1027947 (0.0008) [2023-12-26 22:51:20,544][105692] Updated weights for policy 0, policy_version 1027509 (0.0006) [2023-12-26 22:51:20,603][105620] Updated weights for policy 1, policy_version 1027957 (0.0008) [2023-12-26 22:51:20,613][105692] Updated weights for policy 0, policy_version 1027519 (0.0009) [2023-12-26 22:51:20,668][105620] Updated weights for policy 1, policy_version 1027967 (0.0006) [2023-12-26 22:51:21,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 526278656. Throughput: 0: 9776.7, 1: 9988.5. Samples: 526266848. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:51:21,063][104569] Avg episode reward: [(0, '8993.218'), (1, '9171.796')] [2023-12-26 22:51:21,304][105692] Updated weights for policy 0, policy_version 1027529 (0.0010) [2023-12-26 22:51:21,378][105692] Updated weights for policy 0, policy_version 1027539 (0.0007) [2023-12-26 22:51:21,430][105692] Updated weights for policy 0, policy_version 1027549 (0.0009) [2023-12-26 22:51:21,450][105620] Updated weights for policy 1, policy_version 1027977 (0.0007) [2023-12-26 22:51:21,501][105692] Updated weights for policy 0, policy_version 1027559 (0.0008) [2023-12-26 22:51:21,507][105620] Updated weights for policy 1, policy_version 1027987 (0.0007) [2023-12-26 22:51:21,571][105620] Updated weights for policy 1, policy_version 1027997 (0.0009) [2023-12-26 22:51:21,635][105620] Updated weights for policy 1, policy_version 1028007 (0.0008) [2023-12-26 22:51:22,220][105692] Updated weights for policy 0, policy_version 1027569 (0.0008) [2023-12-26 22:51:22,284][105692] Updated weights for policy 0, policy_version 1027579 (0.0007) [2023-12-26 22:51:22,357][105692] Updated weights for policy 0, policy_version 1027589 (0.0007) [2023-12-26 22:51:22,461][105620] Updated weights for policy 1, policy_version 1028017 (0.0008) [2023-12-26 22:51:22,517][105620] Updated weights for policy 1, policy_version 1028027 (0.0009) [2023-12-26 22:51:22,564][105620] Updated weights for policy 1, policy_version 1028037 (0.0009) [2023-12-26 22:51:22,985][105692] Updated weights for policy 0, policy_version 1027599 (0.0009) [2023-12-26 22:51:23,047][105692] Updated weights for policy 0, policy_version 1027609 (0.0009) [2023-12-26 22:51:23,113][105692] Updated weights for policy 0, policy_version 1027619 (0.0009) [2023-12-26 22:51:23,406][105620] Updated weights for policy 1, policy_version 1028047 (0.0009) [2023-12-26 22:51:23,455][105620] Updated weights for policy 1, policy_version 1028057 (0.0009) [2023-12-26 22:51:23,510][105620] Updated weights for policy 1, policy_version 1028067 (0.0009) [2023-12-26 22:51:23,819][105692] Updated weights for policy 0, policy_version 1027630 (0.0010) [2023-12-26 22:51:23,867][105692] Updated weights for policy 0, policy_version 1027640 (0.0008) [2023-12-26 22:51:23,912][105692] Updated weights for policy 0, policy_version 1027650 (0.0008) [2023-12-26 22:51:24,281][105620] Updated weights for policy 1, policy_version 1028077 (0.0010) [2023-12-26 22:51:24,330][105620] Updated weights for policy 1, policy_version 1028087 (0.0010) [2023-12-26 22:51:24,379][105620] Updated weights for policy 1, policy_version 1028097 (0.0011) [2023-12-26 22:51:24,612][105692] Updated weights for policy 0, policy_version 1027660 (0.0007) [2023-12-26 22:51:24,666][105692] Updated weights for policy 0, policy_version 1027670 (0.0009) [2023-12-26 22:51:24,720][105692] Updated weights for policy 0, policy_version 1027680 (0.0010) [2023-12-26 22:51:25,048][105620] Updated weights for policy 1, policy_version 1028107 (0.0010) [2023-12-26 22:51:25,099][105620] Updated weights for policy 1, policy_version 1028117 (0.0010) [2023-12-26 22:51:25,156][105620] Updated weights for policy 1, policy_version 1028127 (0.0010) [2023-12-26 22:51:25,516][105692] Updated weights for policy 0, policy_version 1027691 (0.0009) [2023-12-26 22:51:25,573][105692] Updated weights for policy 0, policy_version 1027701 (0.0008) [2023-12-26 22:51:25,624][105692] Updated weights for policy 0, policy_version 1027711 (0.0008) [2023-12-26 22:51:25,917][105620] Updated weights for policy 1, policy_version 1028137 (0.0010) [2023-12-26 22:51:25,974][105620] Updated weights for policy 1, policy_version 1028147 (0.0009) [2023-12-26 22:51:26,015][105586] KL-divergence is very high: 106.2921 [2023-12-26 22:51:26,035][105620] Updated weights for policy 1, policy_version 1028157 (0.0009) [2023-12-26 22:51:26,049][105586] KL-divergence is very high: 121.0826 [2023-12-26 22:51:26,060][105586] KL-divergence is very high: 111.3043 [2023-12-26 22:51:26,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 526368768. Throughput: 0: 9751.3, 1: 9927.6. Samples: 526380888. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:51:26,063][104569] Avg episode reward: [(0, '9173.149'), (1, '5065.777')] [2023-12-26 22:51:26,086][105620] Updated weights for policy 1, policy_version 1028167 (0.0009) [2023-12-26 22:51:26,326][105692] Updated weights for policy 0, policy_version 1027721 (0.0008) [2023-12-26 22:51:26,372][105692] Updated weights for policy 0, policy_version 1027731 (0.0005) [2023-12-26 22:51:26,415][105692] Updated weights for policy 0, policy_version 1027741 (0.0005) [2023-12-26 22:51:26,460][105692] Updated weights for policy 0, policy_version 1027751 (0.0005) [2023-12-26 22:51:26,753][105620] Updated weights for policy 1, policy_version 1028177 (0.0009) [2023-12-26 22:51:26,811][105620] Updated weights for policy 1, policy_version 1028187 (0.0009) [2023-12-26 22:51:26,876][105620] Updated weights for policy 1, policy_version 1028197 (0.0011) [2023-12-26 22:51:27,141][105692] Updated weights for policy 0, policy_version 1027761 (0.0009) [2023-12-26 22:51:27,186][105692] Updated weights for policy 0, policy_version 1027771 (0.0008) [2023-12-26 22:51:27,240][105692] Updated weights for policy 0, policy_version 1027781 (0.0009) [2023-12-26 22:51:27,544][105620] Updated weights for policy 1, policy_version 1028207 (0.0007) [2023-12-26 22:51:27,601][105620] Updated weights for policy 1, policy_version 1028217 (0.0005) [2023-12-26 22:51:27,667][105620] Updated weights for policy 1, policy_version 1028227 (0.0009) [2023-12-26 22:51:28,088][105692] Updated weights for policy 0, policy_version 1027791 (0.0009) [2023-12-26 22:51:28,133][105585] KL-divergence is very high: 163.1657 [2023-12-26 22:51:28,138][105692] Updated weights for policy 0, policy_version 1027801 (0.0008) [2023-12-26 22:51:28,173][105585] KL-divergence is very high: 172.2556 [2023-12-26 22:51:28,190][105692] Updated weights for policy 0, policy_version 1027812 (0.0010) [2023-12-26 22:51:28,279][105620] Updated weights for policy 1, policy_version 1028238 (0.0009) [2023-12-26 22:51:28,344][105620] Updated weights for policy 1, policy_version 1028248 (0.0009) [2023-12-26 22:51:28,406][105620] Updated weights for policy 1, policy_version 1028258 (0.0009) [2023-12-26 22:51:28,939][105692] Updated weights for policy 0, policy_version 1027822 (0.0007) [2023-12-26 22:51:28,993][105692] Updated weights for policy 0, policy_version 1027832 (0.0009) [2023-12-26 22:51:29,048][105692] Updated weights for policy 0, policy_version 1027842 (0.0009) [2023-12-26 22:51:29,123][105620] Updated weights for policy 1, policy_version 1028268 (0.0010) [2023-12-26 22:51:29,169][105620] Updated weights for policy 1, policy_version 1028278 (0.0009) [2023-12-26 22:51:29,216][105620] Updated weights for policy 1, policy_version 1028288 (0.0009) [2023-12-26 22:51:29,806][105692] Updated weights for policy 0, policy_version 1027852 (0.0009) [2023-12-26 22:51:29,876][105692] Updated weights for policy 0, policy_version 1027862 (0.0009) [2023-12-26 22:51:29,936][105620] Updated weights for policy 1, policy_version 1028298 (0.0007) [2023-12-26 22:51:29,946][105692] Updated weights for policy 0, policy_version 1027872 (0.0009) [2023-12-26 22:51:29,995][105620] Updated weights for policy 1, policy_version 1028308 (0.0008) [2023-12-26 22:51:30,044][105620] Updated weights for policy 1, policy_version 1028318 (0.0008) [2023-12-26 22:51:30,098][105620] Updated weights for policy 1, policy_version 1028328 (0.0009) [2023-12-26 22:51:30,664][105692] Updated weights for policy 0, policy_version 1027882 (0.0009) [2023-12-26 22:51:30,730][105692] Updated weights for policy 0, policy_version 1027892 (0.0009) [2023-12-26 22:51:30,778][105692] Updated weights for policy 0, policy_version 1027902 (0.0010) [2023-12-26 22:51:30,833][105692] Updated weights for policy 0, policy_version 1027912 (0.0010) [2023-12-26 22:51:30,856][105620] Updated weights for policy 1, policy_version 1028338 (0.0006) [2023-12-26 22:51:30,906][105620] Updated weights for policy 1, policy_version 1028348 (0.0009) [2023-12-26 22:51:30,959][105620] Updated weights for policy 1, policy_version 1028358 (0.0010) [2023-12-26 22:51:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 526475264. Throughput: 0: 9701.1, 1: 9963.2. Samples: 526440536. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:51:31,063][104569] Avg episode reward: [(0, '9093.447'), (1, '4636.933')] [2023-12-26 22:51:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001028360_263290880.pth... [2023-12-26 22:51:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001027912_263184384.pth... [2023-12-26 22:51:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001027208_262995968.pth [2023-12-26 22:51:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001026792_262897664.pth [2023-12-26 22:51:31,450][105692] Updated weights for policy 0, policy_version 1027922 (0.0010) [2023-12-26 22:51:31,502][105692] Updated weights for policy 0, policy_version 1027932 (0.0011) [2023-12-26 22:51:31,558][105692] Updated weights for policy 0, policy_version 1027942 (0.0011) [2023-12-26 22:51:31,809][105620] Updated weights for policy 1, policy_version 1028368 (0.0007) [2023-12-26 22:51:31,859][105620] Updated weights for policy 1, policy_version 1028378 (0.0006) [2023-12-26 22:51:31,924][105620] Updated weights for policy 1, policy_version 1028388 (0.0008) [2023-12-26 22:51:32,291][105692] Updated weights for policy 0, policy_version 1027952 (0.0010) [2023-12-26 22:51:32,346][105692] Updated weights for policy 0, policy_version 1027962 (0.0010) [2023-12-26 22:51:32,401][105692] Updated weights for policy 0, policy_version 1027972 (0.0008) [2023-12-26 22:51:32,721][105620] Updated weights for policy 1, policy_version 1028398 (0.0007) [2023-12-26 22:51:32,787][105620] Updated weights for policy 1, policy_version 1028408 (0.0005) [2023-12-26 22:51:32,853][105620] Updated weights for policy 1, policy_version 1028418 (0.0008) [2023-12-26 22:51:33,140][105692] Updated weights for policy 0, policy_version 1027982 (0.0011) [2023-12-26 22:51:33,200][105692] Updated weights for policy 0, policy_version 1027992 (0.0010) [2023-12-26 22:51:33,263][105692] Updated weights for policy 0, policy_version 1028002 (0.0005) [2023-12-26 22:51:33,599][105620] Updated weights for policy 1, policy_version 1028428 (0.0009) [2023-12-26 22:51:33,651][105620] Updated weights for policy 1, policy_version 1028439 (0.0010) [2023-12-26 22:51:33,699][105620] Updated weights for policy 1, policy_version 1028449 (0.0010) [2023-12-26 22:51:33,821][105692] Updated weights for policy 0, policy_version 1028012 (0.0007) [2023-12-26 22:51:33,878][105692] Updated weights for policy 0, policy_version 1028022 (0.0010) [2023-12-26 22:51:33,935][105692] Updated weights for policy 0, policy_version 1028032 (0.0010) [2023-12-26 22:51:34,387][105620] Updated weights for policy 1, policy_version 1028459 (0.0009) [2023-12-26 22:51:34,442][105620] Updated weights for policy 1, policy_version 1028469 (0.0006) [2023-12-26 22:51:34,506][105620] Updated weights for policy 1, policy_version 1028479 (0.0006) [2023-12-26 22:51:34,596][105692] Updated weights for policy 0, policy_version 1028042 (0.0009) [2023-12-26 22:51:34,648][105692] Updated weights for policy 0, policy_version 1028052 (0.0009) [2023-12-26 22:51:34,707][105692] Updated weights for policy 0, policy_version 1028062 (0.0007) [2023-12-26 22:51:34,766][105692] Updated weights for policy 0, policy_version 1028072 (0.0005) [2023-12-26 22:51:35,312][105620] Updated weights for policy 1, policy_version 1028489 (0.0008) [2023-12-26 22:51:35,361][105620] Updated weights for policy 1, policy_version 1028499 (0.0006) [2023-12-26 22:51:35,373][105692] Updated weights for policy 0, policy_version 1028082 (0.0010) [2023-12-26 22:51:35,417][105620] Updated weights for policy 1, policy_version 1028509 (0.0009) [2023-12-26 22:51:35,427][105692] Updated weights for policy 0, policy_version 1028092 (0.0010) [2023-12-26 22:51:35,474][105620] Updated weights for policy 1, policy_version 1028519 (0.0006) [2023-12-26 22:51:35,486][105692] Updated weights for policy 0, policy_version 1028102 (0.0009) [2023-12-26 22:51:36,027][105692] Updated weights for policy 0, policy_version 1028112 (0.0006) [2023-12-26 22:51:36,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 526565376. Throughput: 0: 9699.8, 1: 9868.5. Samples: 526556564. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:51:36,062][104569] Avg episode reward: [(0, '9119.110'), (1, '7235.251')] [2023-12-26 22:51:36,072][105692] Updated weights for policy 0, policy_version 1028122 (0.0005) [2023-12-26 22:51:36,136][105692] Updated weights for policy 0, policy_version 1028132 (0.0007) [2023-12-26 22:51:36,351][105620] Updated weights for policy 1, policy_version 1028529 (0.0008) [2023-12-26 22:51:36,411][105620] Updated weights for policy 1, policy_version 1028539 (0.0008) [2023-12-26 22:51:36,471][105620] Updated weights for policy 1, policy_version 1028549 (0.0008) [2023-12-26 22:51:36,825][105692] Updated weights for policy 0, policy_version 1028142 (0.0010) [2023-12-26 22:51:36,877][105692] Updated weights for policy 0, policy_version 1028152 (0.0010) [2023-12-26 22:51:36,925][105692] Updated weights for policy 0, policy_version 1028162 (0.0010) [2023-12-26 22:51:37,206][105620] Updated weights for policy 1, policy_version 1028559 (0.0010) [2023-12-26 22:51:37,265][105620] Updated weights for policy 1, policy_version 1028569 (0.0010) [2023-12-26 22:51:37,320][105620] Updated weights for policy 1, policy_version 1028579 (0.0010) [2023-12-26 22:51:37,642][105692] Updated weights for policy 0, policy_version 1028172 (0.0010) [2023-12-26 22:51:37,703][105692] Updated weights for policy 0, policy_version 1028182 (0.0011) [2023-12-26 22:51:37,763][105692] Updated weights for policy 0, policy_version 1028192 (0.0011) [2023-12-26 22:51:38,027][105620] Updated weights for policy 1, policy_version 1028589 (0.0008) [2023-12-26 22:51:38,091][105620] Updated weights for policy 1, policy_version 1028599 (0.0009) [2023-12-26 22:51:38,153][105620] Updated weights for policy 1, policy_version 1028609 (0.0008) [2023-12-26 22:51:38,467][105692] Updated weights for policy 0, policy_version 1028202 (0.0010) [2023-12-26 22:51:38,533][105692] Updated weights for policy 0, policy_version 1028212 (0.0009) [2023-12-26 22:51:38,592][105692] Updated weights for policy 0, policy_version 1028222 (0.0009) [2023-12-26 22:51:38,644][105692] Updated weights for policy 0, policy_version 1028232 (0.0009) [2023-12-26 22:51:38,851][105620] Updated weights for policy 1, policy_version 1028619 (0.0009) [2023-12-26 22:51:38,903][105620] Updated weights for policy 1, policy_version 1028629 (0.0011) [2023-12-26 22:51:38,962][105620] Updated weights for policy 1, policy_version 1028639 (0.0011) [2023-12-26 22:51:39,469][105692] Updated weights for policy 0, policy_version 1028242 (0.0008) [2023-12-26 22:51:39,528][105692] Updated weights for policy 0, policy_version 1028252 (0.0008) [2023-12-26 22:51:39,595][105692] Updated weights for policy 0, policy_version 1028262 (0.0008) [2023-12-26 22:51:39,642][105620] Updated weights for policy 1, policy_version 1028649 (0.0007) [2023-12-26 22:51:39,704][105620] Updated weights for policy 1, policy_version 1028659 (0.0006) [2023-12-26 22:51:39,769][105620] Updated weights for policy 1, policy_version 1028669 (0.0008) [2023-12-26 22:51:39,837][105620] Updated weights for policy 1, policy_version 1028679 (0.0007) [2023-12-26 22:51:40,385][105692] Updated weights for policy 0, policy_version 1028272 (0.0011) [2023-12-26 22:51:40,441][105692] Updated weights for policy 0, policy_version 1028282 (0.0010) [2023-12-26 22:51:40,489][105620] Updated weights for policy 1, policy_version 1028689 (0.0009) [2023-12-26 22:51:40,494][105692] Updated weights for policy 0, policy_version 1028292 (0.0010) [2023-12-26 22:51:40,560][105620] Updated weights for policy 1, policy_version 1028699 (0.0006) [2023-12-26 22:51:40,618][105620] Updated weights for policy 1, policy_version 1028709 (0.0007) [2023-12-26 22:51:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 526663680. Throughput: 0: 9751.1, 1: 9793.7. Samples: 526673504. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:51:41,063][104569] Avg episode reward: [(0, '9081.032'), (1, '8988.327')] [2023-12-26 22:51:41,199][105692] Updated weights for policy 0, policy_version 1028302 (0.0006) [2023-12-26 22:51:41,269][105692] Updated weights for policy 0, policy_version 1028312 (0.0007) [2023-12-26 22:51:41,323][105620] Updated weights for policy 1, policy_version 1028719 (0.0007) [2023-12-26 22:51:41,335][105692] Updated weights for policy 0, policy_version 1028322 (0.0011) [2023-12-26 22:51:41,396][105620] Updated weights for policy 1, policy_version 1028729 (0.0008) [2023-12-26 22:51:41,462][105620] Updated weights for policy 1, policy_version 1028739 (0.0008) [2023-12-26 22:51:42,017][105692] Updated weights for policy 0, policy_version 1028332 (0.0009) [2023-12-26 22:51:42,078][105692] Updated weights for policy 0, policy_version 1028342 (0.0006) [2023-12-26 22:51:42,135][105692] Updated weights for policy 0, policy_version 1028352 (0.0006) [2023-12-26 22:51:42,228][105620] Updated weights for policy 1, policy_version 1028749 (0.0009) [2023-12-26 22:51:42,288][105620] Updated weights for policy 1, policy_version 1028759 (0.0008) [2023-12-26 22:51:42,348][105620] Updated weights for policy 1, policy_version 1028769 (0.0008) [2023-12-26 22:51:42,824][105692] Updated weights for policy 0, policy_version 1028362 (0.0006) [2023-12-26 22:51:42,880][105692] Updated weights for policy 0, policy_version 1028372 (0.0011) [2023-12-26 22:51:42,934][105692] Updated weights for policy 0, policy_version 1028382 (0.0011) [2023-12-26 22:51:42,990][105692] Updated weights for policy 0, policy_version 1028392 (0.0011) [2023-12-26 22:51:43,132][105620] Updated weights for policy 1, policy_version 1028779 (0.0008) [2023-12-26 22:51:43,184][105620] Updated weights for policy 1, policy_version 1028789 (0.0007) [2023-12-26 22:51:43,232][105620] Updated weights for policy 1, policy_version 1028799 (0.0008) [2023-12-26 22:51:43,713][105692] Updated weights for policy 0, policy_version 1028402 (0.0006) [2023-12-26 22:51:43,775][105692] Updated weights for policy 0, policy_version 1028412 (0.0010) [2023-12-26 22:51:43,842][105692] Updated weights for policy 0, policy_version 1028422 (0.0010) [2023-12-26 22:51:43,976][105620] Updated weights for policy 1, policy_version 1028809 (0.0008) [2023-12-26 22:51:44,037][105620] Updated weights for policy 1, policy_version 1028819 (0.0010) [2023-12-26 22:51:44,095][105620] Updated weights for policy 1, policy_version 1028829 (0.0010) [2023-12-26 22:51:44,159][105620] Updated weights for policy 1, policy_version 1028839 (0.0005) [2023-12-26 22:51:44,476][105692] Updated weights for policy 0, policy_version 1028432 (0.0006) [2023-12-26 22:51:44,539][105692] Updated weights for policy 0, policy_version 1028442 (0.0008) [2023-12-26 22:51:44,587][105692] Updated weights for policy 0, policy_version 1028452 (0.0009) [2023-12-26 22:51:44,762][105620] Updated weights for policy 1, policy_version 1028849 (0.0008) [2023-12-26 22:51:44,824][105620] Updated weights for policy 1, policy_version 1028859 (0.0006) [2023-12-26 22:51:44,894][105620] Updated weights for policy 1, policy_version 1028869 (0.0007) [2023-12-26 22:51:45,286][105692] Updated weights for policy 0, policy_version 1028462 (0.0010) [2023-12-26 22:51:45,351][105692] Updated weights for policy 0, policy_version 1028472 (0.0009) [2023-12-26 22:51:45,412][105692] Updated weights for policy 0, policy_version 1028482 (0.0009) [2023-12-26 22:51:45,583][105620] Updated weights for policy 1, policy_version 1028879 (0.0006) [2023-12-26 22:51:45,634][105620] Updated weights for policy 1, policy_version 1028889 (0.0005) [2023-12-26 22:51:45,686][105620] Updated weights for policy 1, policy_version 1028899 (0.0005) [2023-12-26 22:51:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 526761984. Throughput: 0: 9797.5, 1: 9697.9. Samples: 526731476. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 22:51:46,063][104569] Avg episode reward: [(0, '9082.392'), (1, '9170.391')] [2023-12-26 22:51:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001028904_263430144.pth... [2023-12-26 22:51:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001028488_263331840.pth... [2023-12-26 22:51:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001027784_263143424.pth [2023-12-26 22:51:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001027336_263036928.pth [2023-12-26 22:51:46,226][105620] Updated weights for policy 1, policy_version 1028909 (0.0008) [2023-12-26 22:51:46,271][105692] Updated weights for policy 0, policy_version 1028492 (0.0007) [2023-12-26 22:51:46,284][105620] Updated weights for policy 1, policy_version 1028919 (0.0009) [2023-12-26 22:51:46,326][105692] Updated weights for policy 0, policy_version 1028502 (0.0005) [2023-12-26 22:51:46,339][105620] Updated weights for policy 1, policy_version 1028929 (0.0010) [2023-12-26 22:51:46,381][105692] Updated weights for policy 0, policy_version 1028512 (0.0006) [2023-12-26 22:51:47,039][105692] Updated weights for policy 0, policy_version 1028522 (0.0008) [2023-12-26 22:51:47,102][105692] Updated weights for policy 0, policy_version 1028532 (0.0008) [2023-12-26 22:51:47,109][105620] Updated weights for policy 1, policy_version 1028939 (0.0010) [2023-12-26 22:51:47,157][105620] Updated weights for policy 1, policy_version 1028949 (0.0010) [2023-12-26 22:51:47,163][105692] Updated weights for policy 0, policy_version 1028542 (0.0009) [2023-12-26 22:51:47,211][105620] Updated weights for policy 1, policy_version 1028959 (0.0008) [2023-12-26 22:51:47,219][105692] Updated weights for policy 0, policy_version 1028552 (0.0008) [2023-12-26 22:51:47,899][105620] Updated weights for policy 1, policy_version 1028969 (0.0006) [2023-12-26 22:51:47,906][105692] Updated weights for policy 0, policy_version 1028562 (0.0005) [2023-12-26 22:51:47,951][105620] Updated weights for policy 1, policy_version 1028979 (0.0010) [2023-12-26 22:51:47,958][105692] Updated weights for policy 0, policy_version 1028572 (0.0005) [2023-12-26 22:51:48,002][105692] Updated weights for policy 0, policy_version 1028582 (0.0006) [2023-12-26 22:51:48,010][105620] Updated weights for policy 1, policy_version 1028989 (0.0010) [2023-12-26 22:51:48,065][105620] Updated weights for policy 1, policy_version 1028999 (0.0010) [2023-12-26 22:51:48,693][105692] Updated weights for policy 0, policy_version 1028592 (0.0009) [2023-12-26 22:51:48,746][105692] Updated weights for policy 0, policy_version 1028602 (0.0007) [2023-12-26 22:51:48,813][105692] Updated weights for policy 0, policy_version 1028612 (0.0005) [2023-12-26 22:51:48,858][105620] Updated weights for policy 1, policy_version 1029009 (0.0007) [2023-12-26 22:51:48,905][105620] Updated weights for policy 1, policy_version 1029019 (0.0005) [2023-12-26 22:51:48,960][105620] Updated weights for policy 1, policy_version 1029029 (0.0005) [2023-12-26 22:51:49,479][105692] Updated weights for policy 0, policy_version 1028622 (0.0009) [2023-12-26 22:51:49,531][105692] Updated weights for policy 0, policy_version 1028632 (0.0010) [2023-12-26 22:51:49,569][105620] Updated weights for policy 1, policy_version 1029039 (0.0006) [2023-12-26 22:51:49,590][105692] Updated weights for policy 0, policy_version 1028642 (0.0010) [2023-12-26 22:51:49,620][105620] Updated weights for policy 1, policy_version 1029049 (0.0006) [2023-12-26 22:51:49,679][105620] Updated weights for policy 1, policy_version 1029059 (0.0008) [2023-12-26 22:51:50,379][105620] Updated weights for policy 1, policy_version 1029069 (0.0008) [2023-12-26 22:51:50,386][105692] Updated weights for policy 0, policy_version 1028652 (0.0008) [2023-12-26 22:51:50,430][105620] Updated weights for policy 1, policy_version 1029079 (0.0007) [2023-12-26 22:51:50,432][105692] Updated weights for policy 0, policy_version 1028662 (0.0008) [2023-12-26 22:51:50,485][105692] Updated weights for policy 0, policy_version 1028672 (0.0006) [2023-12-26 22:51:50,487][105620] Updated weights for policy 1, policy_version 1029089 (0.0007) [2023-12-26 22:51:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 526860288. Throughput: 0: 9865.2, 1: 9708.3. Samples: 526851944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:51:51,062][104569] Avg episode reward: [(0, '9180.727'), (1, '9081.240')] [2023-12-26 22:51:51,253][105692] Updated weights for policy 0, policy_version 1028682 (0.0007) [2023-12-26 22:51:51,294][105620] Updated weights for policy 1, policy_version 1029099 (0.0007) [2023-12-26 22:51:51,309][105692] Updated weights for policy 0, policy_version 1028692 (0.0005) [2023-12-26 22:51:51,354][105620] Updated weights for policy 1, policy_version 1029109 (0.0009) [2023-12-26 22:51:51,380][105692] Updated weights for policy 0, policy_version 1028702 (0.0008) [2023-12-26 22:51:51,420][105620] Updated weights for policy 1, policy_version 1029119 (0.0007) [2023-12-26 22:51:51,438][105692] Updated weights for policy 0, policy_version 1028712 (0.0008) [2023-12-26 22:51:52,128][105620] Updated weights for policy 1, policy_version 1029129 (0.0007) [2023-12-26 22:51:52,155][105692] Updated weights for policy 0, policy_version 1028722 (0.0007) [2023-12-26 22:51:52,189][105620] Updated weights for policy 1, policy_version 1029139 (0.0008) [2023-12-26 22:51:52,208][105692] Updated weights for policy 0, policy_version 1028732 (0.0009) [2023-12-26 22:51:52,245][105620] Updated weights for policy 1, policy_version 1029149 (0.0007) [2023-12-26 22:51:52,259][105692] Updated weights for policy 0, policy_version 1028742 (0.0009) [2023-12-26 22:51:52,306][105620] Updated weights for policy 1, policy_version 1029159 (0.0007) [2023-12-26 22:51:53,037][105620] Updated weights for policy 1, policy_version 1029169 (0.0009) [2023-12-26 22:51:53,064][105692] Updated weights for policy 0, policy_version 1028752 (0.0007) [2023-12-26 22:51:53,093][105620] Updated weights for policy 1, policy_version 1029179 (0.0008) [2023-12-26 22:51:53,112][105692] Updated weights for policy 0, policy_version 1028762 (0.0006) [2023-12-26 22:51:53,151][105620] Updated weights for policy 1, policy_version 1029189 (0.0008) [2023-12-26 22:51:53,158][105692] Updated weights for policy 0, policy_version 1028772 (0.0008) [2023-12-26 22:51:53,850][105620] Updated weights for policy 1, policy_version 1029199 (0.0006) [2023-12-26 22:51:53,912][105620] Updated weights for policy 1, policy_version 1029209 (0.0006) [2023-12-26 22:51:53,968][105620] Updated weights for policy 1, policy_version 1029219 (0.0006) [2023-12-26 22:51:53,984][105692] Updated weights for policy 0, policy_version 1028782 (0.0008) [2023-12-26 22:51:54,042][105692] Updated weights for policy 0, policy_version 1028792 (0.0008) [2023-12-26 22:51:54,096][105692] Updated weights for policy 0, policy_version 1028802 (0.0009) [2023-12-26 22:51:54,639][105620] Updated weights for policy 1, policy_version 1029229 (0.0007) [2023-12-26 22:51:54,700][105620] Updated weights for policy 1, policy_version 1029239 (0.0007) [2023-12-26 22:51:54,759][105620] Updated weights for policy 1, policy_version 1029249 (0.0005) [2023-12-26 22:51:54,916][105692] Updated weights for policy 0, policy_version 1028812 (0.0009) [2023-12-26 22:51:54,979][105692] Updated weights for policy 0, policy_version 1028822 (0.0009) [2023-12-26 22:51:55,027][105692] Updated weights for policy 0, policy_version 1028832 (0.0009) [2023-12-26 22:51:55,443][105620] Updated weights for policy 1, policy_version 1029259 (0.0007) [2023-12-26 22:51:55,492][105620] Updated weights for policy 1, policy_version 1029269 (0.0007) [2023-12-26 22:51:55,543][105620] Updated weights for policy 1, policy_version 1029279 (0.0005) [2023-12-26 22:51:55,822][105692] Updated weights for policy 0, policy_version 1028842 (0.0009) [2023-12-26 22:51:55,890][105692] Updated weights for policy 0, policy_version 1028852 (0.0009) [2023-12-26 22:51:55,946][105692] Updated weights for policy 0, policy_version 1028862 (0.0010) [2023-12-26 22:51:56,007][105692] Updated weights for policy 0, policy_version 1028872 (0.0009) [2023-12-26 22:51:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 526958592. Throughput: 0: 9661.0, 1: 9688.8. Samples: 526965108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:51:56,062][104569] Avg episode reward: [(0, '8822.695'), (1, '9081.771')] [2023-12-26 22:51:56,138][105620] Updated weights for policy 1, policy_version 1029289 (0.0005) [2023-12-26 22:51:56,198][105620] Updated weights for policy 1, policy_version 1029299 (0.0005) [2023-12-26 22:51:56,253][105620] Updated weights for policy 1, policy_version 1029309 (0.0005) [2023-12-26 22:51:56,305][105620] Updated weights for policy 1, policy_version 1029319 (0.0005) [2023-12-26 22:51:56,852][105692] Updated weights for policy 0, policy_version 1028882 (0.0008) [2023-12-26 22:51:56,897][105692] Updated weights for policy 0, policy_version 1028892 (0.0006) [2023-12-26 22:51:56,910][105620] Updated weights for policy 1, policy_version 1029329 (0.0010) [2023-12-26 22:51:56,945][105692] Updated weights for policy 0, policy_version 1028902 (0.0007) [2023-12-26 22:51:56,970][105620] Updated weights for policy 1, policy_version 1029339 (0.0009) [2023-12-26 22:51:57,030][105620] Updated weights for policy 1, policy_version 1029349 (0.0010) [2023-12-26 22:51:57,677][105620] Updated weights for policy 1, policy_version 1029359 (0.0010) [2023-12-26 22:51:57,732][105620] Updated weights for policy 1, policy_version 1029369 (0.0008) [2023-12-26 22:51:57,750][105692] Updated weights for policy 0, policy_version 1028912 (0.0006) [2023-12-26 22:51:57,787][105620] Updated weights for policy 1, policy_version 1029379 (0.0009) [2023-12-26 22:51:57,798][105692] Updated weights for policy 0, policy_version 1028922 (0.0006) [2023-12-26 22:51:57,861][105692] Updated weights for policy 0, policy_version 1028932 (0.0005) [2023-12-26 22:51:58,571][105692] Updated weights for policy 0, policy_version 1028942 (0.0006) [2023-12-26 22:51:58,603][105620] Updated weights for policy 1, policy_version 1029389 (0.0008) [2023-12-26 22:51:58,630][105692] Updated weights for policy 0, policy_version 1028952 (0.0008) [2023-12-26 22:51:58,669][105620] Updated weights for policy 1, policy_version 1029399 (0.0008) [2023-12-26 22:51:58,693][105692] Updated weights for policy 0, policy_version 1028962 (0.0008) [2023-12-26 22:51:58,734][105620] Updated weights for policy 1, policy_version 1029409 (0.0008) [2023-12-26 22:51:59,449][105692] Updated weights for policy 0, policy_version 1028972 (0.0008) [2023-12-26 22:51:59,465][105620] Updated weights for policy 1, policy_version 1029419 (0.0008) [2023-12-26 22:51:59,511][105692] Updated weights for policy 0, policy_version 1028982 (0.0010) [2023-12-26 22:51:59,527][105620] Updated weights for policy 1, policy_version 1029429 (0.0009) [2023-12-26 22:51:59,573][105692] Updated weights for policy 0, policy_version 1028992 (0.0009) [2023-12-26 22:51:59,589][105620] Updated weights for policy 1, policy_version 1029439 (0.0006) [2023-12-26 22:52:00,318][105620] Updated weights for policy 1, policy_version 1029449 (0.0007) [2023-12-26 22:52:00,325][105692] Updated weights for policy 0, policy_version 1029002 (0.0008) [2023-12-26 22:52:00,381][105620] Updated weights for policy 1, policy_version 1029459 (0.0007) [2023-12-26 22:52:00,383][105692] Updated weights for policy 0, policy_version 1029012 (0.0009) [2023-12-26 22:52:00,439][105692] Updated weights for policy 0, policy_version 1029022 (0.0006) [2023-12-26 22:52:00,441][105620] Updated weights for policy 1, policy_version 1029469 (0.0008) [2023-12-26 22:52:00,492][105692] Updated weights for policy 0, policy_version 1029032 (0.0006) [2023-12-26 22:52:00,501][105620] Updated weights for policy 1, policy_version 1029479 (0.0007) [2023-12-26 22:52:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 527048704. Throughput: 0: 9623.1, 1: 9749.3. Samples: 527022256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:52:01,062][104569] Avg episode reward: [(0, '8818.180'), (1, '8989.676')] [2023-12-26 22:52:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001029032_263471104.pth... [2023-12-26 22:52:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001029480_263577600.pth... [2023-12-26 22:52:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001028360_263290880.pth [2023-12-26 22:52:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001027912_263184384.pth [2023-12-26 22:52:01,220][105620] Updated weights for policy 1, policy_version 1029489 (0.0008) [2023-12-26 22:52:01,273][105620] Updated weights for policy 1, policy_version 1029499 (0.0008) [2023-12-26 22:52:01,282][105692] Updated weights for policy 0, policy_version 1029042 (0.0007) [2023-12-26 22:52:01,330][105620] Updated weights for policy 1, policy_version 1029509 (0.0008) [2023-12-26 22:52:01,349][105692] Updated weights for policy 0, policy_version 1029052 (0.0006) [2023-12-26 22:52:01,405][105692] Updated weights for policy 0, policy_version 1029062 (0.0009) [2023-12-26 22:52:02,035][105620] Updated weights for policy 1, policy_version 1029519 (0.0006) [2023-12-26 22:52:02,061][105692] Updated weights for policy 0, policy_version 1029072 (0.0009) [2023-12-26 22:52:02,091][105620] Updated weights for policy 1, policy_version 1029529 (0.0005) [2023-12-26 22:52:02,110][105692] Updated weights for policy 0, policy_version 1029082 (0.0009) [2023-12-26 22:52:02,141][105620] Updated weights for policy 1, policy_version 1029539 (0.0007) [2023-12-26 22:52:02,154][105692] Updated weights for policy 0, policy_version 1029092 (0.0009) [2023-12-26 22:52:02,718][105620] Updated weights for policy 1, policy_version 1029549 (0.0008) [2023-12-26 22:52:02,767][105620] Updated weights for policy 1, policy_version 1029559 (0.0008) [2023-12-26 22:52:02,822][105620] Updated weights for policy 1, policy_version 1029569 (0.0010) [2023-12-26 22:52:02,873][105692] Updated weights for policy 0, policy_version 1029102 (0.0006) [2023-12-26 22:52:02,919][105692] Updated weights for policy 0, policy_version 1029112 (0.0008) [2023-12-26 22:52:02,967][105692] Updated weights for policy 0, policy_version 1029122 (0.0008) [2023-12-26 22:52:03,550][105620] Updated weights for policy 1, policy_version 1029579 (0.0010) [2023-12-26 22:52:03,598][105620] Updated weights for policy 1, policy_version 1029589 (0.0010) [2023-12-26 22:52:03,655][105620] Updated weights for policy 1, policy_version 1029599 (0.0010) [2023-12-26 22:52:03,682][105692] Updated weights for policy 0, policy_version 1029132 (0.0009) [2023-12-26 22:52:03,737][105692] Updated weights for policy 0, policy_version 1029142 (0.0007) [2023-12-26 22:52:03,785][105692] Updated weights for policy 0, policy_version 1029152 (0.0008) [2023-12-26 22:52:04,425][105620] Updated weights for policy 1, policy_version 1029609 (0.0010) [2023-12-26 22:52:04,488][105620] Updated weights for policy 1, policy_version 1029619 (0.0011) [2023-12-26 22:52:04,537][105620] Updated weights for policy 1, policy_version 1029629 (0.0010) [2023-12-26 22:52:04,583][105692] Updated weights for policy 0, policy_version 1029162 (0.0007) [2023-12-26 22:52:04,593][105620] Updated weights for policy 1, policy_version 1029639 (0.0011) [2023-12-26 22:52:04,639][105692] Updated weights for policy 0, policy_version 1029172 (0.0008) [2023-12-26 22:52:04,698][105692] Updated weights for policy 0, policy_version 1029182 (0.0010) [2023-12-26 22:52:04,753][105692] Updated weights for policy 0, policy_version 1029192 (0.0010) [2023-12-26 22:52:05,271][105620] Updated weights for policy 1, policy_version 1029649 (0.0010) [2023-12-26 22:52:05,315][105620] Updated weights for policy 1, policy_version 1029659 (0.0010) [2023-12-26 22:52:05,369][105620] Updated weights for policy 1, policy_version 1029669 (0.0008) [2023-12-26 22:52:05,503][105692] Updated weights for policy 0, policy_version 1029202 (0.0006) [2023-12-26 22:52:05,561][105692] Updated weights for policy 0, policy_version 1029212 (0.0006) [2023-12-26 22:52:05,614][105692] Updated weights for policy 0, policy_version 1029222 (0.0009) [2023-12-26 22:52:06,048][105620] Updated weights for policy 1, policy_version 1029679 (0.0006) [2023-12-26 22:52:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 527147008. Throughput: 0: 9664.1, 1: 9711.2. Samples: 527138732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:52:06,062][104569] Avg episode reward: [(0, '9087.277'), (1, '9080.777')] [2023-12-26 22:52:06,105][105620] Updated weights for policy 1, policy_version 1029689 (0.0006) [2023-12-26 22:52:06,171][105620] Updated weights for policy 1, policy_version 1029699 (0.0007) [2023-12-26 22:52:06,436][105692] Updated weights for policy 0, policy_version 1029232 (0.0008) [2023-12-26 22:52:06,486][105692] Updated weights for policy 0, policy_version 1029242 (0.0008) [2023-12-26 22:52:06,536][105692] Updated weights for policy 0, policy_version 1029252 (0.0008) [2023-12-26 22:52:06,838][105620] Updated weights for policy 1, policy_version 1029709 (0.0008) [2023-12-26 22:52:06,888][105620] Updated weights for policy 1, policy_version 1029719 (0.0009) [2023-12-26 22:52:06,940][105620] Updated weights for policy 1, policy_version 1029729 (0.0005) [2023-12-26 22:52:07,352][105692] Updated weights for policy 0, policy_version 1029262 (0.0008) [2023-12-26 22:52:07,412][105692] Updated weights for policy 0, policy_version 1029273 (0.0010) [2023-12-26 22:52:07,468][105692] Updated weights for policy 0, policy_version 1029283 (0.0007) [2023-12-26 22:52:07,592][105620] Updated weights for policy 1, policy_version 1029739 (0.0008) [2023-12-26 22:52:07,640][105620] Updated weights for policy 1, policy_version 1029749 (0.0010) [2023-12-26 22:52:07,698][105620] Updated weights for policy 1, policy_version 1029759 (0.0010) [2023-12-26 22:52:08,068][105692] Updated weights for policy 0, policy_version 1029293 (0.0006) [2023-12-26 22:52:08,120][105692] Updated weights for policy 0, policy_version 1029303 (0.0008) [2023-12-26 22:52:08,169][105692] Updated weights for policy 0, policy_version 1029313 (0.0009) [2023-12-26 22:52:08,421][105620] Updated weights for policy 1, policy_version 1029769 (0.0010) [2023-12-26 22:52:08,484][105620] Updated weights for policy 1, policy_version 1029779 (0.0011) [2023-12-26 22:52:08,542][105620] Updated weights for policy 1, policy_version 1029789 (0.0010) [2023-12-26 22:52:08,604][105620] Updated weights for policy 1, policy_version 1029799 (0.0010) [2023-12-26 22:52:08,915][105692] Updated weights for policy 0, policy_version 1029323 (0.0009) [2023-12-26 22:52:08,980][105692] Updated weights for policy 0, policy_version 1029333 (0.0008) [2023-12-26 22:52:09,040][105692] Updated weights for policy 0, policy_version 1029343 (0.0009) [2023-12-26 22:52:09,278][105620] Updated weights for policy 1, policy_version 1029809 (0.0008) [2023-12-26 22:52:09,338][105620] Updated weights for policy 1, policy_version 1029819 (0.0009) [2023-12-26 22:52:09,408][105620] Updated weights for policy 1, policy_version 1029829 (0.0009) [2023-12-26 22:52:09,748][105692] Updated weights for policy 0, policy_version 1029353 (0.0009) [2023-12-26 22:52:09,820][105692] Updated weights for policy 0, policy_version 1029363 (0.0006) [2023-12-26 22:52:09,880][105692] Updated weights for policy 0, policy_version 1029373 (0.0008) [2023-12-26 22:52:09,950][105692] Updated weights for policy 0, policy_version 1029383 (0.0008) [2023-12-26 22:52:10,236][105620] Updated weights for policy 1, policy_version 1029839 (0.0008) [2023-12-26 22:52:10,286][105620] Updated weights for policy 1, policy_version 1029849 (0.0007) [2023-12-26 22:52:10,340][105620] Updated weights for policy 1, policy_version 1029860 (0.0010) [2023-12-26 22:52:10,595][105692] Updated weights for policy 0, policy_version 1029393 (0.0005) [2023-12-26 22:52:10,649][105692] Updated weights for policy 0, policy_version 1029403 (0.0008) [2023-12-26 22:52:10,710][105692] Updated weights for policy 0, policy_version 1029413 (0.0009) [2023-12-26 22:52:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 527245312. Throughput: 0: 9630.0, 1: 9795.1. Samples: 527255016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:52:11,063][104569] Avg episode reward: [(0, '9178.698'), (1, '9010.510')] [2023-12-26 22:52:11,070][105620] Updated weights for policy 1, policy_version 1029870 (0.0007) [2023-12-26 22:52:11,136][105620] Updated weights for policy 1, policy_version 1029880 (0.0006) [2023-12-26 22:52:11,202][105620] Updated weights for policy 1, policy_version 1029890 (0.0007) [2023-12-26 22:52:11,526][105692] Updated weights for policy 0, policy_version 1029423 (0.0010) [2023-12-26 22:52:11,581][105692] Updated weights for policy 0, policy_version 1029433 (0.0010) [2023-12-26 22:52:11,645][105692] Updated weights for policy 0, policy_version 1029443 (0.0008) [2023-12-26 22:52:11,913][105620] Updated weights for policy 1, policy_version 1029900 (0.0010) [2023-12-26 22:52:11,974][105620] Updated weights for policy 1, policy_version 1029910 (0.0009) [2023-12-26 22:52:12,042][105620] Updated weights for policy 1, policy_version 1029920 (0.0010) [2023-12-26 22:52:12,330][105692] Updated weights for policy 0, policy_version 1029453 (0.0009) [2023-12-26 22:52:12,394][105692] Updated weights for policy 0, policy_version 1029463 (0.0010) [2023-12-26 22:52:12,452][105692] Updated weights for policy 0, policy_version 1029473 (0.0010) [2023-12-26 22:52:12,850][105620] Updated weights for policy 1, policy_version 1029930 (0.0010) [2023-12-26 22:52:12,932][105620] Updated weights for policy 1, policy_version 1029940 (0.0009) [2023-12-26 22:52:12,995][105620] Updated weights for policy 1, policy_version 1029950 (0.0009) [2023-12-26 22:52:13,048][105620] Updated weights for policy 1, policy_version 1029960 (0.0010) [2023-12-26 22:52:13,098][105692] Updated weights for policy 0, policy_version 1029483 (0.0007) [2023-12-26 22:52:13,153][105692] Updated weights for policy 0, policy_version 1029493 (0.0006) [2023-12-26 22:52:13,205][105692] Updated weights for policy 0, policy_version 1029503 (0.0006) [2023-12-26 22:52:13,800][105620] Updated weights for policy 1, policy_version 1029970 (0.0010) [2023-12-26 22:52:13,828][105692] Updated weights for policy 0, policy_version 1029513 (0.0007) [2023-12-26 22:52:13,862][105620] Updated weights for policy 1, policy_version 1029980 (0.0010) [2023-12-26 22:52:13,886][105692] Updated weights for policy 0, policy_version 1029523 (0.0010) [2023-12-26 22:52:13,919][105620] Updated weights for policy 1, policy_version 1029990 (0.0010) [2023-12-26 22:52:13,941][105692] Updated weights for policy 0, policy_version 1029533 (0.0010) [2023-12-26 22:52:14,008][105692] Updated weights for policy 0, policy_version 1029543 (0.0010) [2023-12-26 22:52:14,580][105620] Updated weights for policy 1, policy_version 1030000 (0.0010) [2023-12-26 22:52:14,601][105692] Updated weights for policy 0, policy_version 1029553 (0.0009) [2023-12-26 22:52:14,627][105620] Updated weights for policy 1, policy_version 1030010 (0.0007) [2023-12-26 22:52:14,653][105692] Updated weights for policy 0, policy_version 1029563 (0.0010) [2023-12-26 22:52:14,687][105620] Updated weights for policy 1, policy_version 1030020 (0.0007) [2023-12-26 22:52:14,713][105692] Updated weights for policy 0, policy_version 1029573 (0.0007) [2023-12-26 22:52:15,421][105692] Updated weights for policy 0, policy_version 1029583 (0.0007) [2023-12-26 22:52:15,461][105620] Updated weights for policy 1, policy_version 1030030 (0.0010) [2023-12-26 22:52:15,485][105692] Updated weights for policy 0, policy_version 1029593 (0.0006) [2023-12-26 22:52:15,524][105620] Updated weights for policy 1, policy_version 1030040 (0.0009) [2023-12-26 22:52:15,546][105692] Updated weights for policy 0, policy_version 1029603 (0.0007) [2023-12-26 22:52:15,591][105620] Updated weights for policy 1, policy_version 1030050 (0.0009) [2023-12-26 22:52:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19387.6, 300 sec: 19577.5). Total num frames: 527343616. Throughput: 0: 9645.9, 1: 9733.1. Samples: 527312596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:52:16,063][104569] Avg episode reward: [(0, '8996.294'), (1, '7974.955')] [2023-12-26 22:52:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001029608_263618560.pth... [2023-12-26 22:52:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001030056_263725056.pth... [2023-12-26 22:52:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001028904_263430144.pth [2023-12-26 22:52:16,080][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001028488_263331840.pth [2023-12-26 22:52:16,238][105692] Updated weights for policy 0, policy_version 1029613 (0.0007) [2023-12-26 22:52:16,255][105620] Updated weights for policy 1, policy_version 1030060 (0.0009) [2023-12-26 22:52:16,286][105692] Updated weights for policy 0, policy_version 1029623 (0.0010) [2023-12-26 22:52:16,304][105620] Updated weights for policy 1, policy_version 1030070 (0.0005) [2023-12-26 22:52:16,331][105692] Updated weights for policy 0, policy_version 1029633 (0.0010) [2023-12-26 22:52:16,350][105620] Updated weights for policy 1, policy_version 1030080 (0.0005) [2023-12-26 22:52:16,945][105692] Updated weights for policy 0, policy_version 1029643 (0.0010) [2023-12-26 22:52:17,011][105692] Updated weights for policy 0, policy_version 1029653 (0.0009) [2023-12-26 22:52:17,056][105620] Updated weights for policy 1, policy_version 1030090 (0.0006) [2023-12-26 22:52:17,072][105692] Updated weights for policy 0, policy_version 1029663 (0.0008) [2023-12-26 22:52:17,107][105620] Updated weights for policy 1, policy_version 1030100 (0.0006) [2023-12-26 22:52:17,170][105620] Updated weights for policy 1, policy_version 1030110 (0.0009) [2023-12-26 22:52:17,219][105620] Updated weights for policy 1, policy_version 1030120 (0.0009) [2023-12-26 22:52:17,673][105692] Updated weights for policy 0, policy_version 1029673 (0.0007) [2023-12-26 22:52:17,730][105692] Updated weights for policy 0, policy_version 1029683 (0.0005) [2023-12-26 22:52:17,787][105692] Updated weights for policy 0, policy_version 1029693 (0.0010) [2023-12-26 22:52:17,900][105620] Updated weights for policy 1, policy_version 1030130 (0.0005) [2023-12-26 22:52:17,956][105620] Updated weights for policy 1, policy_version 1030140 (0.0006) [2023-12-26 22:52:18,017][105620] Updated weights for policy 1, policy_version 1030150 (0.0007) [2023-12-26 22:52:18,424][105692] Updated weights for policy 0, policy_version 1029705 (0.0010) [2023-12-26 22:52:18,479][105692] Updated weights for policy 0, policy_version 1029715 (0.0009) [2023-12-26 22:52:18,531][105692] Updated weights for policy 0, policy_version 1029725 (0.0009) [2023-12-26 22:52:18,593][105692] Updated weights for policy 0, policy_version 1029735 (0.0009) [2023-12-26 22:52:18,791][105620] Updated weights for policy 1, policy_version 1030160 (0.0009) [2023-12-26 22:52:18,855][105620] Updated weights for policy 1, policy_version 1030170 (0.0009) [2023-12-26 22:52:18,913][105620] Updated weights for policy 1, policy_version 1030180 (0.0009) [2023-12-26 22:52:19,346][105692] Updated weights for policy 0, policy_version 1029745 (0.0009) [2023-12-26 22:52:19,408][105692] Updated weights for policy 0, policy_version 1029755 (0.0009) [2023-12-26 22:52:19,464][105692] Updated weights for policy 0, policy_version 1029765 (0.0008) [2023-12-26 22:52:19,710][105620] Updated weights for policy 1, policy_version 1030190 (0.0010) [2023-12-26 22:52:19,771][105620] Updated weights for policy 1, policy_version 1030200 (0.0009) [2023-12-26 22:52:19,824][105620] Updated weights for policy 1, policy_version 1030210 (0.0009) [2023-12-26 22:52:20,223][105692] Updated weights for policy 0, policy_version 1029775 (0.0009) [2023-12-26 22:52:20,270][105692] Updated weights for policy 0, policy_version 1029785 (0.0008) [2023-12-26 22:52:20,326][105692] Updated weights for policy 0, policy_version 1029797 (0.0010) [2023-12-26 22:52:20,592][105620] Updated weights for policy 1, policy_version 1030220 (0.0006) [2023-12-26 22:52:20,661][105620] Updated weights for policy 1, policy_version 1030230 (0.0007) [2023-12-26 22:52:20,725][105620] Updated weights for policy 1, policy_version 1030240 (0.0006) [2023-12-26 22:52:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 527441920. Throughput: 0: 9698.3, 1: 9777.5. Samples: 527432976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:52:21,062][104569] Avg episode reward: [(0, '8911.871'), (1, '8047.737')] [2023-12-26 22:52:21,161][105692] Updated weights for policy 0, policy_version 1029807 (0.0008) [2023-12-26 22:52:21,234][105692] Updated weights for policy 0, policy_version 1029817 (0.0008) [2023-12-26 22:52:21,308][105692] Updated weights for policy 0, policy_version 1029827 (0.0009) [2023-12-26 22:52:21,355][105620] Updated weights for policy 1, policy_version 1030250 (0.0006) [2023-12-26 22:52:21,417][105620] Updated weights for policy 1, policy_version 1030260 (0.0008) [2023-12-26 22:52:21,480][105620] Updated weights for policy 1, policy_version 1030270 (0.0006) [2023-12-26 22:52:21,532][105620] Updated weights for policy 1, policy_version 1030280 (0.0007) [2023-12-26 22:52:22,055][105692] Updated weights for policy 0, policy_version 1029837 (0.0010) [2023-12-26 22:52:22,124][105692] Updated weights for policy 0, policy_version 1029847 (0.0009) [2023-12-26 22:52:22,193][105692] Updated weights for policy 0, policy_version 1029857 (0.0009) [2023-12-26 22:52:22,288][105620] Updated weights for policy 1, policy_version 1030290 (0.0008) [2023-12-26 22:52:22,353][105620] Updated weights for policy 1, policy_version 1030300 (0.0009) [2023-12-26 22:52:22,419][105620] Updated weights for policy 1, policy_version 1030310 (0.0008) [2023-12-26 22:52:22,955][105692] Updated weights for policy 0, policy_version 1029867 (0.0008) [2023-12-26 22:52:23,019][105692] Updated weights for policy 0, policy_version 1029877 (0.0010) [2023-12-26 22:52:23,084][105692] Updated weights for policy 0, policy_version 1029887 (0.0010) [2023-12-26 22:52:23,206][105620] Updated weights for policy 1, policy_version 1030320 (0.0008) [2023-12-26 22:52:23,270][105620] Updated weights for policy 1, policy_version 1030330 (0.0008) [2023-12-26 22:52:23,330][105620] Updated weights for policy 1, policy_version 1030340 (0.0006) [2023-12-26 22:52:23,741][105692] Updated weights for policy 0, policy_version 1029897 (0.0010) [2023-12-26 22:52:23,800][105692] Updated weights for policy 0, policy_version 1029907 (0.0005) [2023-12-26 22:52:23,858][105692] Updated weights for policy 0, policy_version 1029917 (0.0005) [2023-12-26 22:52:23,907][105692] Updated weights for policy 0, policy_version 1029927 (0.0005) [2023-12-26 22:52:24,056][105620] Updated weights for policy 1, policy_version 1030350 (0.0008) [2023-12-26 22:52:24,115][105620] Updated weights for policy 1, policy_version 1030360 (0.0010) [2023-12-26 22:52:24,173][105620] Updated weights for policy 1, policy_version 1030370 (0.0010) [2023-12-26 22:52:24,591][105692] Updated weights for policy 0, policy_version 1029937 (0.0010) [2023-12-26 22:52:24,644][105692] Updated weights for policy 0, policy_version 1029947 (0.0010) [2023-12-26 22:52:24,705][105692] Updated weights for policy 0, policy_version 1029957 (0.0011) [2023-12-26 22:52:24,927][105620] Updated weights for policy 1, policy_version 1030380 (0.0011) [2023-12-26 22:52:24,979][105620] Updated weights for policy 1, policy_version 1030390 (0.0010) [2023-12-26 22:52:25,041][105620] Updated weights for policy 1, policy_version 1030400 (0.0011) [2023-12-26 22:52:25,353][105692] Updated weights for policy 0, policy_version 1029967 (0.0007) [2023-12-26 22:52:25,399][105692] Updated weights for policy 0, policy_version 1029977 (0.0005) [2023-12-26 22:52:25,445][105692] Updated weights for policy 0, policy_version 1029987 (0.0005) [2023-12-26 22:52:25,786][105620] Updated weights for policy 1, policy_version 1030410 (0.0010) [2023-12-26 22:52:25,844][105620] Updated weights for policy 1, policy_version 1030420 (0.0010) [2023-12-26 22:52:25,905][105620] Updated weights for policy 1, policy_version 1030430 (0.0010) [2023-12-26 22:52:25,960][105620] Updated weights for policy 1, policy_version 1030440 (0.0010) [2023-12-26 22:52:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 527540224. Throughput: 0: 9660.5, 1: 9763.9. Samples: 527547604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:52:26,063][104569] Avg episode reward: [(0, '8824.850'), (1, '9173.132')] [2023-12-26 22:52:26,151][105692] Updated weights for policy 0, policy_version 1029997 (0.0009) [2023-12-26 22:52:26,208][105692] Updated weights for policy 0, policy_version 1030007 (0.0010) [2023-12-26 22:52:26,270][105692] Updated weights for policy 0, policy_version 1030017 (0.0010) [2023-12-26 22:52:26,710][105620] Updated weights for policy 1, policy_version 1030450 (0.0010) [2023-12-26 22:52:26,761][105620] Updated weights for policy 1, policy_version 1030460 (0.0010) [2023-12-26 22:52:26,808][105620] Updated weights for policy 1, policy_version 1030470 (0.0010) [2023-12-26 22:52:26,959][105692] Updated weights for policy 0, policy_version 1030027 (0.0009) [2023-12-26 22:52:27,016][105692] Updated weights for policy 0, policy_version 1030037 (0.0005) [2023-12-26 22:52:27,071][105692] Updated weights for policy 0, policy_version 1030047 (0.0005) [2023-12-26 22:52:27,532][105620] Updated weights for policy 1, policy_version 1030480 (0.0010) [2023-12-26 22:52:27,590][105620] Updated weights for policy 1, policy_version 1030490 (0.0010) [2023-12-26 22:52:27,629][105692] Updated weights for policy 0, policy_version 1030057 (0.0008) [2023-12-26 22:52:27,644][105620] Updated weights for policy 1, policy_version 1030500 (0.0010) [2023-12-26 22:52:27,677][105692] Updated weights for policy 0, policy_version 1030067 (0.0010) [2023-12-26 22:52:27,728][105692] Updated weights for policy 0, policy_version 1030077 (0.0010) [2023-12-26 22:52:27,772][105692] Updated weights for policy 0, policy_version 1030087 (0.0010) [2023-12-26 22:52:28,335][105620] Updated weights for policy 1, policy_version 1030510 (0.0008) [2023-12-26 22:52:28,360][105692] Updated weights for policy 0, policy_version 1030097 (0.0008) [2023-12-26 22:52:28,386][105620] Updated weights for policy 1, policy_version 1030520 (0.0008) [2023-12-26 22:52:28,409][105692] Updated weights for policy 0, policy_version 1030107 (0.0006) [2023-12-26 22:52:28,441][105620] Updated weights for policy 1, policy_version 1030530 (0.0008) [2023-12-26 22:52:28,464][105692] Updated weights for policy 0, policy_version 1030117 (0.0006) [2023-12-26 22:52:29,118][105692] Updated weights for policy 0, policy_version 1030127 (0.0009) [2023-12-26 22:52:29,171][105692] Updated weights for policy 0, policy_version 1030137 (0.0011) [2023-12-26 22:52:29,181][105620] Updated weights for policy 1, policy_version 1030540 (0.0007) [2023-12-26 22:52:29,224][105692] Updated weights for policy 0, policy_version 1030147 (0.0010) [2023-12-26 22:52:29,248][105620] Updated weights for policy 1, policy_version 1030550 (0.0007) [2023-12-26 22:52:29,306][105620] Updated weights for policy 1, policy_version 1030560 (0.0007) [2023-12-26 22:52:29,971][105692] Updated weights for policy 0, policy_version 1030157 (0.0009) [2023-12-26 22:52:30,000][105620] Updated weights for policy 1, policy_version 1030570 (0.0010) [2023-12-26 22:52:30,028][105692] Updated weights for policy 0, policy_version 1030167 (0.0008) [2023-12-26 22:52:30,049][105620] Updated weights for policy 1, policy_version 1030580 (0.0010) [2023-12-26 22:52:30,087][105692] Updated weights for policy 0, policy_version 1030177 (0.0010) [2023-12-26 22:52:30,105][105620] Updated weights for policy 1, policy_version 1030590 (0.0010) [2023-12-26 22:52:30,160][105620] Updated weights for policy 1, policy_version 1030600 (0.0010) [2023-12-26 22:52:30,735][105692] Updated weights for policy 0, policy_version 1030187 (0.0011) [2023-12-26 22:52:30,789][105692] Updated weights for policy 0, policy_version 1030197 (0.0010) [2023-12-26 22:52:30,838][105692] Updated weights for policy 0, policy_version 1030207 (0.0010) [2023-12-26 22:52:30,879][105620] Updated weights for policy 1, policy_version 1030610 (0.0010) [2023-12-26 22:52:30,930][105620] Updated weights for policy 1, policy_version 1030620 (0.0010) [2023-12-26 22:52:30,987][105620] Updated weights for policy 1, policy_version 1030630 (0.0008) [2023-12-26 22:52:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 527646720. Throughput: 0: 9721.5, 1: 9787.8. Samples: 527609392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:52:31,063][104569] Avg episode reward: [(0, '9088.546'), (1, '8852.185')] [2023-12-26 22:52:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001030216_263774208.pth... [2023-12-26 22:52:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001030632_263872512.pth... [2023-12-26 22:52:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001029032_263471104.pth [2023-12-26 22:52:31,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001029480_263577600.pth [2023-12-26 22:52:31,614][105692] Updated weights for policy 0, policy_version 1030217 (0.0010) [2023-12-26 22:52:31,678][105692] Updated weights for policy 0, policy_version 1030227 (0.0009) [2023-12-26 22:52:31,680][105620] Updated weights for policy 1, policy_version 1030640 (0.0007) [2023-12-26 22:52:31,741][105692] Updated weights for policy 0, policy_version 1030237 (0.0008) [2023-12-26 22:52:31,741][105620] Updated weights for policy 1, policy_version 1030650 (0.0008) [2023-12-26 22:52:31,802][105620] Updated weights for policy 1, policy_version 1030660 (0.0007) [2023-12-26 22:52:31,803][105692] Updated weights for policy 0, policy_version 1030247 (0.0007) [2023-12-26 22:52:32,511][105692] Updated weights for policy 0, policy_version 1030257 (0.0006) [2023-12-26 22:52:32,571][105692] Updated weights for policy 0, policy_version 1030267 (0.0007) [2023-12-26 22:52:32,585][105620] Updated weights for policy 1, policy_version 1030670 (0.0009) [2023-12-26 22:52:32,628][105692] Updated weights for policy 0, policy_version 1030277 (0.0008) [2023-12-26 22:52:32,639][105620] Updated weights for policy 1, policy_version 1030680 (0.0006) [2023-12-26 22:52:32,687][105620] Updated weights for policy 1, policy_version 1030690 (0.0008) [2023-12-26 22:52:33,287][105692] Updated weights for policy 0, policy_version 1030287 (0.0006) [2023-12-26 22:52:33,343][105692] Updated weights for policy 0, policy_version 1030297 (0.0007) [2023-12-26 22:52:33,393][105692] Updated weights for policy 0, policy_version 1030307 (0.0008) [2023-12-26 22:52:33,477][105620] Updated weights for policy 1, policy_version 1030700 (0.0008) [2023-12-26 22:52:33,531][105620] Updated weights for policy 1, policy_version 1030710 (0.0009) [2023-12-26 22:52:33,582][105620] Updated weights for policy 1, policy_version 1030720 (0.0010) [2023-12-26 22:52:33,947][105692] Updated weights for policy 0, policy_version 1030317 (0.0006) [2023-12-26 22:52:34,002][105692] Updated weights for policy 0, policy_version 1030327 (0.0005) [2023-12-26 22:52:34,046][105692] Updated weights for policy 0, policy_version 1030337 (0.0005) [2023-12-26 22:52:34,131][105620] Updated weights for policy 1, policy_version 1030730 (0.0006) [2023-12-26 22:52:34,193][105620] Updated weights for policy 1, policy_version 1030740 (0.0009) [2023-12-26 22:52:34,254][105620] Updated weights for policy 1, policy_version 1030750 (0.0007) [2023-12-26 22:52:34,322][105620] Updated weights for policy 1, policy_version 1030760 (0.0009) [2023-12-26 22:52:34,797][105692] Updated weights for policy 0, policy_version 1030347 (0.0006) [2023-12-26 22:52:34,852][105692] Updated weights for policy 0, policy_version 1030357 (0.0010) [2023-12-26 22:52:34,905][105692] Updated weights for policy 0, policy_version 1030367 (0.0009) [2023-12-26 22:52:34,957][105620] Updated weights for policy 1, policy_version 1030770 (0.0008) [2023-12-26 22:52:35,019][105620] Updated weights for policy 1, policy_version 1030780 (0.0008) [2023-12-26 22:52:35,077][105620] Updated weights for policy 1, policy_version 1030790 (0.0006) [2023-12-26 22:52:35,647][105692] Updated weights for policy 0, policy_version 1030377 (0.0005) [2023-12-26 22:52:35,709][105692] Updated weights for policy 0, policy_version 1030387 (0.0008) [2023-12-26 22:52:35,723][105620] Updated weights for policy 1, policy_version 1030800 (0.0009) [2023-12-26 22:52:35,764][105692] Updated weights for policy 0, policy_version 1030397 (0.0005) [2023-12-26 22:52:35,774][105620] Updated weights for policy 1, policy_version 1030810 (0.0010) [2023-12-26 22:52:35,824][105692] Updated weights for policy 0, policy_version 1030407 (0.0006) [2023-12-26 22:52:35,837][105620] Updated weights for policy 1, policy_version 1030820 (0.0011) [2023-12-26 22:52:36,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 527745024. Throughput: 0: 9763.3, 1: 9752.4. Samples: 527730148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:52:36,062][104569] Avg episode reward: [(0, '8816.564'), (1, '8761.201')] [2023-12-26 22:52:36,564][105692] Updated weights for policy 0, policy_version 1030417 (0.0006) [2023-12-26 22:52:36,569][105620] Updated weights for policy 1, policy_version 1030830 (0.0010) [2023-12-26 22:52:36,630][105692] Updated weights for policy 0, policy_version 1030427 (0.0006) [2023-12-26 22:52:36,633][105620] Updated weights for policy 1, policy_version 1030840 (0.0009) [2023-12-26 22:52:36,692][105620] Updated weights for policy 1, policy_version 1030850 (0.0010) [2023-12-26 22:52:36,701][105692] Updated weights for policy 0, policy_version 1030437 (0.0006) [2023-12-26 22:52:37,229][105692] Updated weights for policy 0, policy_version 1030447 (0.0009) [2023-12-26 22:52:37,288][105692] Updated weights for policy 0, policy_version 1030457 (0.0008) [2023-12-26 22:52:37,292][105620] Updated weights for policy 1, policy_version 1030860 (0.0011) [2023-12-26 22:52:37,345][105620] Updated weights for policy 1, policy_version 1030870 (0.0011) [2023-12-26 22:52:37,346][105692] Updated weights for policy 0, policy_version 1030467 (0.0006) [2023-12-26 22:52:37,398][105620] Updated weights for policy 1, policy_version 1030880 (0.0011) [2023-12-26 22:52:38,014][105620] Updated weights for policy 1, policy_version 1030890 (0.0008) [2023-12-26 22:52:38,062][105620] Updated weights for policy 1, policy_version 1030900 (0.0010) [2023-12-26 22:52:38,114][105620] Updated weights for policy 1, policy_version 1030910 (0.0010) [2023-12-26 22:52:38,170][105620] Updated weights for policy 1, policy_version 1030920 (0.0011) [2023-12-26 22:52:38,177][105692] Updated weights for policy 0, policy_version 1030477 (0.0007) [2023-12-26 22:52:38,241][105692] Updated weights for policy 0, policy_version 1030487 (0.0009) [2023-12-26 22:52:38,295][105692] Updated weights for policy 0, policy_version 1030497 (0.0009) [2023-12-26 22:52:38,885][105620] Updated weights for policy 1, policy_version 1030930 (0.0005) [2023-12-26 22:52:38,943][105620] Updated weights for policy 1, policy_version 1030940 (0.0008) [2023-12-26 22:52:39,005][105620] Updated weights for policy 1, policy_version 1030950 (0.0009) [2023-12-26 22:52:39,079][105692] Updated weights for policy 0, policy_version 1030507 (0.0009) [2023-12-26 22:52:39,140][105692] Updated weights for policy 0, policy_version 1030517 (0.0010) [2023-12-26 22:52:39,186][105692] Updated weights for policy 0, policy_version 1030527 (0.0008) [2023-12-26 22:52:39,661][105620] Updated weights for policy 1, policy_version 1030960 (0.0009) [2023-12-26 22:52:39,723][105620] Updated weights for policy 1, policy_version 1030970 (0.0008) [2023-12-26 22:52:39,787][105620] Updated weights for policy 1, policy_version 1030980 (0.0008) [2023-12-26 22:52:39,963][105692] Updated weights for policy 0, policy_version 1030537 (0.0009) [2023-12-26 22:52:40,016][105692] Updated weights for policy 0, policy_version 1030547 (0.0008) [2023-12-26 22:52:40,064][105692] Updated weights for policy 0, policy_version 1030557 (0.0008) [2023-12-26 22:52:40,117][105692] Updated weights for policy 0, policy_version 1030567 (0.0010) [2023-12-26 22:52:40,502][105620] Updated weights for policy 1, policy_version 1030990 (0.0009) [2023-12-26 22:52:40,565][105620] Updated weights for policy 1, policy_version 1031000 (0.0009) [2023-12-26 22:52:40,631][105620] Updated weights for policy 1, policy_version 1031010 (0.0009) [2023-12-26 22:52:40,899][105692] Updated weights for policy 0, policy_version 1030577 (0.0009) [2023-12-26 22:52:40,955][105692] Updated weights for policy 0, policy_version 1030587 (0.0009) [2023-12-26 22:52:41,002][105692] Updated weights for policy 0, policy_version 1030597 (0.0009) [2023-12-26 22:52:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 527843328. Throughput: 0: 9811.9, 1: 9807.5. Samples: 527847984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:52:41,063][104569] Avg episode reward: [(0, '8751.723'), (1, '9171.507')] [2023-12-26 22:52:41,394][105620] Updated weights for policy 1, policy_version 1031020 (0.0009) [2023-12-26 22:52:41,457][105620] Updated weights for policy 1, policy_version 1031030 (0.0008) [2023-12-26 22:52:41,517][105620] Updated weights for policy 1, policy_version 1031040 (0.0009) [2023-12-26 22:52:41,809][105692] Updated weights for policy 0, policy_version 1030607 (0.0006) [2023-12-26 22:52:41,877][105692] Updated weights for policy 0, policy_version 1030617 (0.0009) [2023-12-26 22:52:41,948][105692] Updated weights for policy 0, policy_version 1030627 (0.0010) [2023-12-26 22:52:42,244][105620] Updated weights for policy 1, policy_version 1031050 (0.0008) [2023-12-26 22:52:42,308][105620] Updated weights for policy 1, policy_version 1031060 (0.0009) [2023-12-26 22:52:42,382][105620] Updated weights for policy 1, policy_version 1031070 (0.0008) [2023-12-26 22:52:42,440][105620] Updated weights for policy 1, policy_version 1031080 (0.0007) [2023-12-26 22:52:42,728][105692] Updated weights for policy 0, policy_version 1030637 (0.0009) [2023-12-26 22:52:42,783][105692] Updated weights for policy 0, policy_version 1030647 (0.0009) [2023-12-26 22:52:42,847][105692] Updated weights for policy 0, policy_version 1030657 (0.0006) [2023-12-26 22:52:43,031][105620] Updated weights for policy 1, policy_version 1031090 (0.0009) [2023-12-26 22:52:43,097][105620] Updated weights for policy 1, policy_version 1031100 (0.0009) [2023-12-26 22:52:43,143][105620] Updated weights for policy 1, policy_version 1031110 (0.0009) [2023-12-26 22:52:43,485][105692] Updated weights for policy 0, policy_version 1030667 (0.0007) [2023-12-26 22:52:43,538][105692] Updated weights for policy 0, policy_version 1030677 (0.0007) [2023-12-26 22:52:43,602][105692] Updated weights for policy 0, policy_version 1030687 (0.0005) [2023-12-26 22:52:43,857][105620] Updated weights for policy 1, policy_version 1031120 (0.0007) [2023-12-26 22:52:43,914][105620] Updated weights for policy 1, policy_version 1031130 (0.0008) [2023-12-26 22:52:43,974][105620] Updated weights for policy 1, policy_version 1031140 (0.0009) [2023-12-26 22:52:44,239][105692] Updated weights for policy 0, policy_version 1030697 (0.0008) [2023-12-26 22:52:44,291][105692] Updated weights for policy 0, policy_version 1030707 (0.0005) [2023-12-26 22:52:44,339][105692] Updated weights for policy 0, policy_version 1030717 (0.0005) [2023-12-26 22:52:44,408][105692] Updated weights for policy 0, policy_version 1030727 (0.0005) [2023-12-26 22:52:44,575][105620] Updated weights for policy 1, policy_version 1031150 (0.0011) [2023-12-26 22:52:44,634][105620] Updated weights for policy 1, policy_version 1031160 (0.0010) [2023-12-26 22:52:44,687][105620] Updated weights for policy 1, policy_version 1031170 (0.0008) [2023-12-26 22:52:44,999][105692] Updated weights for policy 0, policy_version 1030737 (0.0008) [2023-12-26 22:52:45,056][105692] Updated weights for policy 0, policy_version 1030747 (0.0009) [2023-12-26 22:52:45,108][105692] Updated weights for policy 0, policy_version 1030757 (0.0009) [2023-12-26 22:52:45,342][105620] Updated weights for policy 1, policy_version 1031180 (0.0009) [2023-12-26 22:52:45,403][105620] Updated weights for policy 1, policy_version 1031190 (0.0006) [2023-12-26 22:52:45,463][105620] Updated weights for policy 1, policy_version 1031200 (0.0007) [2023-12-26 22:52:45,843][105692] Updated weights for policy 0, policy_version 1030767 (0.0008) [2023-12-26 22:52:45,896][105692] Updated weights for policy 0, policy_version 1030777 (0.0010) [2023-12-26 22:52:45,931][105585] KL-divergence is very high: 115.6552 [2023-12-26 22:52:45,949][105692] Updated weights for policy 0, policy_version 1030787 (0.0008) [2023-12-26 22:52:45,970][105585] KL-divergence is very high: 116.2262 [2023-12-26 22:52:46,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19660.7, 300 sec: 19605.2). Total num frames: 527941632. Throughput: 0: 9839.2, 1: 9792.9. Samples: 527905708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:52:46,063][104569] Avg episode reward: [(0, '8839.483'), (1, '9354.173')] [2023-12-26 22:52:46,067][105620] Updated weights for policy 1, policy_version 1031210 (0.0006) [2023-12-26 22:52:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001030792_263921664.pth... [2023-12-26 22:52:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001029608_263618560.pth [2023-12-26 22:52:46,133][105620] Updated weights for policy 1, policy_version 1031220 (0.0005) [2023-12-26 22:52:46,199][105620] Updated weights for policy 1, policy_version 1031230 (0.0005) [2023-12-26 22:52:46,256][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001031240_264028160.pth... [2023-12-26 22:52:46,258][105620] Updated weights for policy 1, policy_version 1031240 (0.0007) [2023-12-26 22:52:46,259][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001030056_263725056.pth [2023-12-26 22:52:46,799][105692] Updated weights for policy 0, policy_version 1030797 (0.0008) [2023-12-26 22:52:46,852][105692] Updated weights for policy 0, policy_version 1030807 (0.0005) [2023-12-26 22:52:46,854][105620] Updated weights for policy 1, policy_version 1031250 (0.0011) [2023-12-26 22:52:46,900][105692] Updated weights for policy 0, policy_version 1030817 (0.0005) [2023-12-26 22:52:46,902][105620] Updated weights for policy 1, policy_version 1031260 (0.0011) [2023-12-26 22:52:46,963][105620] Updated weights for policy 1, policy_version 1031270 (0.0010) [2023-12-26 22:52:47,695][105692] Updated weights for policy 0, policy_version 1030827 (0.0008) [2023-12-26 22:52:47,699][105620] Updated weights for policy 1, policy_version 1031280 (0.0006) [2023-12-26 22:52:47,755][105692] Updated weights for policy 0, policy_version 1030837 (0.0008) [2023-12-26 22:52:47,764][105620] Updated weights for policy 1, policy_version 1031290 (0.0005) [2023-12-26 22:52:47,809][105692] Updated weights for policy 0, policy_version 1030847 (0.0009) [2023-12-26 22:52:47,819][105620] Updated weights for policy 1, policy_version 1031300 (0.0006) [2023-12-26 22:52:48,311][105620] Updated weights for policy 1, policy_version 1031310 (0.0005) [2023-12-26 22:52:48,365][105620] Updated weights for policy 1, policy_version 1031320 (0.0008) [2023-12-26 22:52:48,415][105620] Updated weights for policy 1, policy_version 1031330 (0.0008) [2023-12-26 22:52:48,671][105692] Updated weights for policy 0, policy_version 1030857 (0.0009) [2023-12-26 22:52:48,732][105692] Updated weights for policy 0, policy_version 1030867 (0.0009) [2023-12-26 22:52:48,793][105692] Updated weights for policy 0, policy_version 1030877 (0.0009) [2023-12-26 22:52:48,863][105692] Updated weights for policy 0, policy_version 1030887 (0.0009) [2023-12-26 22:52:49,195][105620] Updated weights for policy 1, policy_version 1031340 (0.0009) [2023-12-26 22:52:49,261][105620] Updated weights for policy 1, policy_version 1031350 (0.0009) [2023-12-26 22:52:49,317][105620] Updated weights for policy 1, policy_version 1031360 (0.0009) [2023-12-26 22:52:49,594][105692] Updated weights for policy 0, policy_version 1030897 (0.0009) [2023-12-26 22:52:49,651][105692] Updated weights for policy 0, policy_version 1030907 (0.0009) [2023-12-26 22:52:49,702][105692] Updated weights for policy 0, policy_version 1030917 (0.0008) [2023-12-26 22:52:50,078][105620] Updated weights for policy 1, policy_version 1031370 (0.0008) [2023-12-26 22:52:50,140][105620] Updated weights for policy 1, policy_version 1031380 (0.0007) [2023-12-26 22:52:50,198][105620] Updated weights for policy 1, policy_version 1031390 (0.0009) [2023-12-26 22:52:50,256][105620] Updated weights for policy 1, policy_version 1031400 (0.0009) [2023-12-26 22:52:50,495][105692] Updated weights for policy 0, policy_version 1030927 (0.0009) [2023-12-26 22:52:50,545][105692] Updated weights for policy 0, policy_version 1030937 (0.0009) [2023-12-26 22:52:50,609][105692] Updated weights for policy 0, policy_version 1030947 (0.0008) [2023-12-26 22:52:51,029][105620] Updated weights for policy 1, policy_version 1031410 (0.0008) [2023-12-26 22:52:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 528031744. Throughput: 0: 9826.5, 1: 9873.8. Samples: 528025248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:52:51,063][104569] Avg episode reward: [(0, '8926.698'), (1, '9354.110')] [2023-12-26 22:52:51,092][105620] Updated weights for policy 1, policy_version 1031420 (0.0008) [2023-12-26 22:52:51,153][105620] Updated weights for policy 1, policy_version 1031430 (0.0008) [2023-12-26 22:52:51,380][105692] Updated weights for policy 0, policy_version 1030957 (0.0009) [2023-12-26 22:52:51,437][105692] Updated weights for policy 0, policy_version 1030967 (0.0009) [2023-12-26 22:52:51,493][105692] Updated weights for policy 0, policy_version 1030977 (0.0005) [2023-12-26 22:52:51,922][105620] Updated weights for policy 1, policy_version 1031440 (0.0008) [2023-12-26 22:52:51,970][105620] Updated weights for policy 1, policy_version 1031450 (0.0008) [2023-12-26 22:52:52,028][105620] Updated weights for policy 1, policy_version 1031460 (0.0008) [2023-12-26 22:52:52,248][105692] Updated weights for policy 0, policy_version 1030987 (0.0008) [2023-12-26 22:52:52,307][105692] Updated weights for policy 0, policy_version 1030997 (0.0009) [2023-12-26 22:52:52,367][105692] Updated weights for policy 0, policy_version 1031007 (0.0009) [2023-12-26 22:52:52,733][105620] Updated weights for policy 1, policy_version 1031470 (0.0008) [2023-12-26 22:52:52,792][105620] Updated weights for policy 1, policy_version 1031480 (0.0008) [2023-12-26 22:52:52,852][105620] Updated weights for policy 1, policy_version 1031490 (0.0008) [2023-12-26 22:52:53,156][105692] Updated weights for policy 0, policy_version 1031017 (0.0009) [2023-12-26 22:52:53,220][105692] Updated weights for policy 0, policy_version 1031027 (0.0005) [2023-12-26 22:52:53,278][105692] Updated weights for policy 0, policy_version 1031037 (0.0008) [2023-12-26 22:52:53,342][105692] Updated weights for policy 0, policy_version 1031047 (0.0005) [2023-12-26 22:52:53,695][105620] Updated weights for policy 1, policy_version 1031500 (0.0008) [2023-12-26 22:52:53,747][105620] Updated weights for policy 1, policy_version 1031511 (0.0010) [2023-12-26 22:52:53,799][105620] Updated weights for policy 1, policy_version 1031522 (0.0010) [2023-12-26 22:52:53,871][105692] Updated weights for policy 0, policy_version 1031057 (0.0005) [2023-12-26 22:52:53,934][105692] Updated weights for policy 0, policy_version 1031067 (0.0008) [2023-12-26 22:52:53,994][105692] Updated weights for policy 0, policy_version 1031077 (0.0007) [2023-12-26 22:52:54,554][105692] Updated weights for policy 0, policy_version 1031087 (0.0006) [2023-12-26 22:52:54,599][105620] Updated weights for policy 1, policy_version 1031532 (0.0008) [2023-12-26 22:52:54,613][105692] Updated weights for policy 0, policy_version 1031097 (0.0005) [2023-12-26 22:52:54,654][105620] Updated weights for policy 1, policy_version 1031542 (0.0005) [2023-12-26 22:52:54,673][105692] Updated weights for policy 0, policy_version 1031107 (0.0008) [2023-12-26 22:52:54,708][105620] Updated weights for policy 1, policy_version 1031552 (0.0005) [2023-12-26 22:52:55,270][105692] Updated weights for policy 0, policy_version 1031117 (0.0008) [2023-12-26 22:52:55,326][105692] Updated weights for policy 0, policy_version 1031127 (0.0008) [2023-12-26 22:52:55,387][105620] Updated weights for policy 1, policy_version 1031562 (0.0006) [2023-12-26 22:52:55,390][105692] Updated weights for policy 0, policy_version 1031137 (0.0008) [2023-12-26 22:52:55,446][105620] Updated weights for policy 1, policy_version 1031572 (0.0010) [2023-12-26 22:52:55,510][105620] Updated weights for policy 1, policy_version 1031582 (0.0007) [2023-12-26 22:52:55,580][105620] Updated weights for policy 1, policy_version 1031592 (0.0006) [2023-12-26 22:52:56,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 528130048. Throughput: 0: 9889.6, 1: 9822.8. Samples: 528142076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:52:56,063][104569] Avg episode reward: [(0, '9171.340'), (1, '9173.065')] [2023-12-26 22:52:56,146][105692] Updated weights for policy 0, policy_version 1031147 (0.0006) [2023-12-26 22:52:56,179][105620] Updated weights for policy 1, policy_version 1031602 (0.0010) [2023-12-26 22:52:56,202][105692] Updated weights for policy 0, policy_version 1031157 (0.0005) [2023-12-26 22:52:56,239][105620] Updated weights for policy 1, policy_version 1031612 (0.0009) [2023-12-26 22:52:56,268][105692] Updated weights for policy 0, policy_version 1031167 (0.0006) [2023-12-26 22:52:56,303][105620] Updated weights for policy 1, policy_version 1031622 (0.0008) [2023-12-26 22:52:56,854][105620] Updated weights for policy 1, policy_version 1031632 (0.0006) [2023-12-26 22:52:56,904][105620] Updated weights for policy 1, policy_version 1031642 (0.0005) [2023-12-26 22:52:56,961][105620] Updated weights for policy 1, policy_version 1031652 (0.0005) [2023-12-26 22:52:57,058][105692] Updated weights for policy 0, policy_version 1031177 (0.0006) [2023-12-26 22:52:57,112][105692] Updated weights for policy 0, policy_version 1031188 (0.0010) [2023-12-26 22:52:57,166][105692] Updated weights for policy 0, policy_version 1031198 (0.0010) [2023-12-26 22:52:57,220][105692] Updated weights for policy 0, policy_version 1031208 (0.0010) [2023-12-26 22:52:57,523][105620] Updated weights for policy 1, policy_version 1031662 (0.0008) [2023-12-26 22:52:57,570][105620] Updated weights for policy 1, policy_version 1031672 (0.0010) [2023-12-26 22:52:57,623][105620] Updated weights for policy 1, policy_version 1031682 (0.0008) [2023-12-26 22:52:57,969][105692] Updated weights for policy 0, policy_version 1031218 (0.0010) [2023-12-26 22:52:58,020][105692] Updated weights for policy 0, policy_version 1031228 (0.0010) [2023-12-26 22:52:58,078][105692] Updated weights for policy 0, policy_version 1031238 (0.0010) [2023-12-26 22:52:58,391][105620] Updated weights for policy 1, policy_version 1031692 (0.0007) [2023-12-26 22:52:58,454][105620] Updated weights for policy 1, policy_version 1031702 (0.0010) [2023-12-26 22:52:58,512][105620] Updated weights for policy 1, policy_version 1031712 (0.0009) [2023-12-26 22:52:58,830][105692] Updated weights for policy 0, policy_version 1031248 (0.0011) [2023-12-26 22:52:58,900][105692] Updated weights for policy 0, policy_version 1031258 (0.0010) [2023-12-26 22:52:58,967][105692] Updated weights for policy 0, policy_version 1031268 (0.0007) [2023-12-26 22:52:59,357][105620] Updated weights for policy 1, policy_version 1031722 (0.0009) [2023-12-26 22:52:59,413][105620] Updated weights for policy 1, policy_version 1031732 (0.0010) [2023-12-26 22:52:59,470][105620] Updated weights for policy 1, policy_version 1031743 (0.0010) [2023-12-26 22:52:59,684][105692] Updated weights for policy 0, policy_version 1031278 (0.0009) [2023-12-26 22:52:59,746][105692] Updated weights for policy 0, policy_version 1031288 (0.0009) [2023-12-26 22:52:59,806][105692] Updated weights for policy 0, policy_version 1031298 (0.0010) [2023-12-26 22:53:00,182][105620] Updated weights for policy 1, policy_version 1031754 (0.0007) [2023-12-26 22:53:00,243][105620] Updated weights for policy 1, policy_version 1031764 (0.0005) [2023-12-26 22:53:00,309][105620] Updated weights for policy 1, policy_version 1031774 (0.0008) [2023-12-26 22:53:00,367][105620] Updated weights for policy 1, policy_version 1031784 (0.0009) [2023-12-26 22:53:00,604][105692] Updated weights for policy 0, policy_version 1031308 (0.0009) [2023-12-26 22:53:00,664][105692] Updated weights for policy 0, policy_version 1031318 (0.0009) [2023-12-26 22:53:00,725][105692] Updated weights for policy 0, policy_version 1031328 (0.0009) [2023-12-26 22:53:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 528228352. Throughput: 0: 9849.6, 1: 9887.7. Samples: 528200768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:53:01,062][104569] Avg episode reward: [(0, '8727.645'), (1, '8902.816')] [2023-12-26 22:53:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001031336_264060928.pth... [2023-12-26 22:53:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001030216_263774208.pth [2023-12-26 22:53:01,082][105620] Updated weights for policy 1, policy_version 1031794 (0.0009) [2023-12-26 22:53:01,148][105620] Updated weights for policy 1, policy_version 1031804 (0.0009) [2023-12-26 22:53:01,208][105620] Updated weights for policy 1, policy_version 1031814 (0.0010) [2023-12-26 22:53:01,219][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001031816_264175616.pth... [2023-12-26 22:53:01,222][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001030632_263872512.pth [2023-12-26 22:53:01,467][105692] Updated weights for policy 0, policy_version 1031338 (0.0008) [2023-12-26 22:53:01,515][105692] Updated weights for policy 0, policy_version 1031348 (0.0009) [2023-12-26 22:53:01,572][105692] Updated weights for policy 0, policy_version 1031358 (0.0009) [2023-12-26 22:53:01,633][105692] Updated weights for policy 0, policy_version 1031368 (0.0008) [2023-12-26 22:53:02,006][105620] Updated weights for policy 1, policy_version 1031824 (0.0009) [2023-12-26 22:53:02,060][105620] Updated weights for policy 1, policy_version 1031834 (0.0009) [2023-12-26 22:53:02,119][105620] Updated weights for policy 1, policy_version 1031844 (0.0009) [2023-12-26 22:53:02,315][105692] Updated weights for policy 0, policy_version 1031378 (0.0007) [2023-12-26 22:53:02,379][105692] Updated weights for policy 0, policy_version 1031388 (0.0011) [2023-12-26 22:53:02,439][105692] Updated weights for policy 0, policy_version 1031398 (0.0010) [2023-12-26 22:53:02,901][105620] Updated weights for policy 1, policy_version 1031854 (0.0009) [2023-12-26 22:53:02,957][105620] Updated weights for policy 1, policy_version 1031864 (0.0008) [2023-12-26 22:53:03,017][105620] Updated weights for policy 1, policy_version 1031874 (0.0008) [2023-12-26 22:53:03,089][105692] Updated weights for policy 0, policy_version 1031408 (0.0009) [2023-12-26 22:53:03,143][105692] Updated weights for policy 0, policy_version 1031418 (0.0005) [2023-12-26 22:53:03,191][105692] Updated weights for policy 0, policy_version 1031428 (0.0005) [2023-12-26 22:53:03,610][105620] Updated weights for policy 1, policy_version 1031884 (0.0006) [2023-12-26 22:53:03,663][105620] Updated weights for policy 1, policy_version 1031894 (0.0005) [2023-12-26 22:53:03,721][105620] Updated weights for policy 1, policy_version 1031904 (0.0005) [2023-12-26 22:53:03,748][105692] Updated weights for policy 0, policy_version 1031438 (0.0005) [2023-12-26 22:53:03,800][105692] Updated weights for policy 0, policy_version 1031448 (0.0005) [2023-12-26 22:53:03,863][105692] Updated weights for policy 0, policy_version 1031458 (0.0008) [2023-12-26 22:53:04,346][105620] Updated weights for policy 1, policy_version 1031914 (0.0006) [2023-12-26 22:53:04,413][105620] Updated weights for policy 1, policy_version 1031924 (0.0009) [2023-12-26 22:53:04,471][105620] Updated weights for policy 1, policy_version 1031934 (0.0009) [2023-12-26 22:53:04,539][105620] Updated weights for policy 1, policy_version 1031944 (0.0009) [2023-12-26 22:53:04,547][105692] Updated weights for policy 0, policy_version 1031468 (0.0007) [2023-12-26 22:53:04,600][105692] Updated weights for policy 0, policy_version 1031478 (0.0009) [2023-12-26 22:53:04,651][105692] Updated weights for policy 0, policy_version 1031488 (0.0009) [2023-12-26 22:53:05,228][105620] Updated weights for policy 1, policy_version 1031954 (0.0010) [2023-12-26 22:53:05,277][105620] Updated weights for policy 1, policy_version 1031964 (0.0008) [2023-12-26 22:53:05,313][105692] Updated weights for policy 0, policy_version 1031498 (0.0006) [2023-12-26 22:53:05,337][105620] Updated weights for policy 1, policy_version 1031974 (0.0009) [2023-12-26 22:53:05,365][105692] Updated weights for policy 0, policy_version 1031508 (0.0005) [2023-12-26 22:53:05,421][105692] Updated weights for policy 0, policy_version 1031518 (0.0005) [2023-12-26 22:53:05,469][105692] Updated weights for policy 0, policy_version 1031528 (0.0005) [2023-12-26 22:53:06,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19660.7, 300 sec: 19605.2). Total num frames: 528326656. Throughput: 0: 9785.5, 1: 9906.9. Samples: 528319144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:53:06,063][104569] Avg episode reward: [(0, '8546.737'), (1, '8993.072')] [2023-12-26 22:53:06,118][105692] Updated weights for policy 0, policy_version 1031538 (0.0007) [2023-12-26 22:53:06,127][105620] Updated weights for policy 1, policy_version 1031984 (0.0008) [2023-12-26 22:53:06,182][105692] Updated weights for policy 0, policy_version 1031548 (0.0008) [2023-12-26 22:53:06,189][105620] Updated weights for policy 1, policy_version 1031994 (0.0007) [2023-12-26 22:53:06,231][105692] Updated weights for policy 0, policy_version 1031558 (0.0008) [2023-12-26 22:53:06,234][105620] Updated weights for policy 1, policy_version 1032004 (0.0007) [2023-12-26 22:53:06,949][105692] Updated weights for policy 0, policy_version 1031568 (0.0008) [2023-12-26 22:53:07,001][105692] Updated weights for policy 0, policy_version 1031578 (0.0005) [2023-12-26 22:53:07,014][105620] Updated weights for policy 1, policy_version 1032014 (0.0009) [2023-12-26 22:53:07,049][105692] Updated weights for policy 0, policy_version 1031588 (0.0005) [2023-12-26 22:53:07,075][105620] Updated weights for policy 1, policy_version 1032025 (0.0010) [2023-12-26 22:53:07,126][105620] Updated weights for policy 1, policy_version 1032035 (0.0008) [2023-12-26 22:53:07,587][105692] Updated weights for policy 0, policy_version 1031598 (0.0005) [2023-12-26 22:53:07,645][105692] Updated weights for policy 0, policy_version 1031608 (0.0010) [2023-12-26 22:53:07,693][105692] Updated weights for policy 0, policy_version 1031618 (0.0008) [2023-12-26 22:53:07,800][105620] Updated weights for policy 1, policy_version 1032045 (0.0005) [2023-12-26 22:53:07,847][105620] Updated weights for policy 1, policy_version 1032055 (0.0008) [2023-12-26 22:53:07,894][105620] Updated weights for policy 1, policy_version 1032065 (0.0009) [2023-12-26 22:53:08,460][105692] Updated weights for policy 0, policy_version 1031628 (0.0009) [2023-12-26 22:53:08,525][105692] Updated weights for policy 0, policy_version 1031638 (0.0006) [2023-12-26 22:53:08,588][105692] Updated weights for policy 0, policy_version 1031648 (0.0006) [2023-12-26 22:53:08,653][105620] Updated weights for policy 1, policy_version 1032075 (0.0009) [2023-12-26 22:53:08,716][105620] Updated weights for policy 1, policy_version 1032085 (0.0011) [2023-12-26 22:53:08,762][105620] Updated weights for policy 1, policy_version 1032095 (0.0011) [2023-12-26 22:53:09,346][105692] Updated weights for policy 0, policy_version 1031658 (0.0006) [2023-12-26 22:53:09,416][105692] Updated weights for policy 0, policy_version 1031668 (0.0009) [2023-12-26 22:53:09,484][105692] Updated weights for policy 0, policy_version 1031678 (0.0009) [2023-12-26 22:53:09,541][105620] Updated weights for policy 1, policy_version 1032105 (0.0010) [2023-12-26 22:53:09,549][105692] Updated weights for policy 0, policy_version 1031688 (0.0009) [2023-12-26 22:53:09,609][105620] Updated weights for policy 1, policy_version 1032115 (0.0006) [2023-12-26 22:53:09,678][105620] Updated weights for policy 1, policy_version 1032125 (0.0007) [2023-12-26 22:53:09,745][105620] Updated weights for policy 1, policy_version 1032135 (0.0008) [2023-12-26 22:53:10,273][105692] Updated weights for policy 0, policy_version 1031698 (0.0010) [2023-12-26 22:53:10,325][105692] Updated weights for policy 0, policy_version 1031708 (0.0009) [2023-12-26 22:53:10,384][105692] Updated weights for policy 0, policy_version 1031718 (0.0008) [2023-12-26 22:53:10,427][105620] Updated weights for policy 1, policy_version 1032145 (0.0009) [2023-12-26 22:53:10,477][105620] Updated weights for policy 1, policy_version 1032155 (0.0009) [2023-12-26 22:53:10,529][105620] Updated weights for policy 1, policy_version 1032165 (0.0006) [2023-12-26 22:53:11,062][104569] Fps is (10 sec: 19659.9, 60 sec: 19660.7, 300 sec: 19633.0). Total num frames: 528424960. Throughput: 0: 9832.0, 1: 9914.5. Samples: 528436204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:53:11,063][104569] Avg episode reward: [(0, '8818.263'), (1, '7592.348')] [2023-12-26 22:53:11,199][105692] Updated weights for policy 0, policy_version 1031728 (0.0009) [2023-12-26 22:53:11,231][105620] Updated weights for policy 1, policy_version 1032175 (0.0007) [2023-12-26 22:53:11,261][105692] Updated weights for policy 0, policy_version 1031738 (0.0008) [2023-12-26 22:53:11,294][105620] Updated weights for policy 1, policy_version 1032185 (0.0008) [2023-12-26 22:53:11,326][105692] Updated weights for policy 0, policy_version 1031748 (0.0008) [2023-12-26 22:53:11,359][105620] Updated weights for policy 1, policy_version 1032195 (0.0008) [2023-12-26 22:53:12,112][105620] Updated weights for policy 1, policy_version 1032205 (0.0009) [2023-12-26 22:53:12,119][105692] Updated weights for policy 0, policy_version 1031758 (0.0007) [2023-12-26 22:53:12,158][105620] Updated weights for policy 1, policy_version 1032215 (0.0006) [2023-12-26 22:53:12,175][105692] Updated weights for policy 0, policy_version 1031768 (0.0007) [2023-12-26 22:53:12,204][105620] Updated weights for policy 1, policy_version 1032225 (0.0007) [2023-12-26 22:53:12,232][105692] Updated weights for policy 0, policy_version 1031778 (0.0007) [2023-12-26 22:53:12,900][105692] Updated weights for policy 0, policy_version 1031788 (0.0008) [2023-12-26 22:53:12,904][105620] Updated weights for policy 1, policy_version 1032235 (0.0009) [2023-12-26 22:53:12,950][105692] Updated weights for policy 0, policy_version 1031798 (0.0008) [2023-12-26 22:53:12,967][105620] Updated weights for policy 1, policy_version 1032245 (0.0008) [2023-12-26 22:53:13,006][105692] Updated weights for policy 0, policy_version 1031808 (0.0007) [2023-12-26 22:53:13,024][105620] Updated weights for policy 1, policy_version 1032255 (0.0008) [2023-12-26 22:53:13,760][105692] Updated weights for policy 0, policy_version 1031818 (0.0007) [2023-12-26 22:53:13,781][105620] Updated weights for policy 1, policy_version 1032265 (0.0007) [2023-12-26 22:53:13,814][105692] Updated weights for policy 0, policy_version 1031828 (0.0008) [2023-12-26 22:53:13,840][105620] Updated weights for policy 1, policy_version 1032275 (0.0008) [2023-12-26 22:53:13,866][105692] Updated weights for policy 0, policy_version 1031838 (0.0006) [2023-12-26 22:53:13,902][105620] Updated weights for policy 1, policy_version 1032285 (0.0007) [2023-12-26 22:53:13,919][105692] Updated weights for policy 0, policy_version 1031848 (0.0008) [2023-12-26 22:53:13,959][105620] Updated weights for policy 1, policy_version 1032295 (0.0008) [2023-12-26 22:53:14,680][105620] Updated weights for policy 1, policy_version 1032305 (0.0008) [2023-12-26 22:53:14,686][105692] Updated weights for policy 0, policy_version 1031858 (0.0007) [2023-12-26 22:53:14,737][105620] Updated weights for policy 1, policy_version 1032315 (0.0008) [2023-12-26 22:53:14,738][105692] Updated weights for policy 0, policy_version 1031868 (0.0006) [2023-12-26 22:53:14,794][105692] Updated weights for policy 0, policy_version 1031878 (0.0008) [2023-12-26 22:53:14,799][105620] Updated weights for policy 1, policy_version 1032325 (0.0007) [2023-12-26 22:53:15,486][105692] Updated weights for policy 0, policy_version 1031888 (0.0009) [2023-12-26 22:53:15,530][105620] Updated weights for policy 1, policy_version 1032335 (0.0007) [2023-12-26 22:53:15,544][105692] Updated weights for policy 0, policy_version 1031898 (0.0008) [2023-12-26 22:53:15,594][105620] Updated weights for policy 1, policy_version 1032345 (0.0007) [2023-12-26 22:53:15,596][105692] Updated weights for policy 0, policy_version 1031908 (0.0006) [2023-12-26 22:53:15,645][105620] Updated weights for policy 1, policy_version 1032355 (0.0007) [2023-12-26 22:53:16,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 528523264. Throughput: 0: 9731.6, 1: 9900.0. Samples: 528492812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:53:16,062][104569] Avg episode reward: [(0, '8512.809'), (1, '5385.178')] [2023-12-26 22:53:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001031912_264208384.pth... [2023-12-26 22:53:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001032360_264314880.pth... [2023-12-26 22:53:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001031240_264028160.pth [2023-12-26 22:53:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001030792_263921664.pth [2023-12-26 22:53:16,277][105692] Updated weights for policy 0, policy_version 1031918 (0.0006) [2023-12-26 22:53:16,332][105692] Updated weights for policy 0, policy_version 1031928 (0.0008) [2023-12-26 22:53:16,391][105692] Updated weights for policy 0, policy_version 1031938 (0.0006) [2023-12-26 22:53:16,395][105620] Updated weights for policy 1, policy_version 1032365 (0.0009) [2023-12-26 22:53:16,448][105620] Updated weights for policy 1, policy_version 1032375 (0.0009) [2023-12-26 22:53:16,501][105620] Updated weights for policy 1, policy_version 1032385 (0.0009) [2023-12-26 22:53:17,108][105692] Updated weights for policy 0, policy_version 1031948 (0.0007) [2023-12-26 22:53:17,154][105692] Updated weights for policy 0, policy_version 1031958 (0.0009) [2023-12-26 22:53:17,202][105692] Updated weights for policy 0, policy_version 1031968 (0.0009) [2023-12-26 22:53:17,238][105620] Updated weights for policy 1, policy_version 1032395 (0.0009) [2023-12-26 22:53:17,305][105620] Updated weights for policy 1, policy_version 1032405 (0.0006) [2023-12-26 22:53:17,356][105620] Updated weights for policy 1, policy_version 1032415 (0.0009) [2023-12-26 22:53:17,937][105620] Updated weights for policy 1, policy_version 1032425 (0.0008) [2023-12-26 22:53:17,990][105620] Updated weights for policy 1, policy_version 1032435 (0.0010) [2023-12-26 22:53:18,042][105620] Updated weights for policy 1, policy_version 1032445 (0.0010) [2023-12-26 22:53:18,099][105620] Updated weights for policy 1, policy_version 1032455 (0.0011) [2023-12-26 22:53:18,105][105692] Updated weights for policy 0, policy_version 1031978 (0.0007) [2023-12-26 22:53:18,160][105692] Updated weights for policy 0, policy_version 1031988 (0.0008) [2023-12-26 22:53:18,217][105692] Updated weights for policy 0, policy_version 1031998 (0.0008) [2023-12-26 22:53:18,273][105692] Updated weights for policy 0, policy_version 1032008 (0.0008) [2023-12-26 22:53:18,788][105620] Updated weights for policy 1, policy_version 1032465 (0.0009) [2023-12-26 22:53:18,846][105620] Updated weights for policy 1, policy_version 1032475 (0.0009) [2023-12-26 22:53:18,896][105620] Updated weights for policy 1, policy_version 1032485 (0.0010) [2023-12-26 22:53:19,083][105692] Updated weights for policy 0, policy_version 1032018 (0.0008) [2023-12-26 22:53:19,143][105692] Updated weights for policy 0, policy_version 1032028 (0.0008) [2023-12-26 22:53:19,198][105692] Updated weights for policy 0, policy_version 1032038 (0.0009) [2023-12-26 22:53:19,695][105620] Updated weights for policy 1, policy_version 1032495 (0.0010) [2023-12-26 22:53:19,762][105620] Updated weights for policy 1, policy_version 1032505 (0.0011) [2023-12-26 22:53:19,830][105620] Updated weights for policy 1, policy_version 1032515 (0.0011) [2023-12-26 22:53:19,994][105692] Updated weights for policy 0, policy_version 1032048 (0.0008) [2023-12-26 22:53:20,051][105692] Updated weights for policy 0, policy_version 1032058 (0.0008) [2023-12-26 22:53:20,104][105692] Updated weights for policy 0, policy_version 1032068 (0.0008) [2023-12-26 22:53:20,593][105620] Updated weights for policy 1, policy_version 1032525 (0.0011) [2023-12-26 22:53:20,653][105620] Updated weights for policy 1, policy_version 1032535 (0.0010) [2023-12-26 22:53:20,713][105620] Updated weights for policy 1, policy_version 1032545 (0.0010) [2023-12-26 22:53:20,895][105692] Updated weights for policy 0, policy_version 1032078 (0.0009) [2023-12-26 22:53:20,953][105692] Updated weights for policy 0, policy_version 1032088 (0.0008) [2023-12-26 22:53:21,016][105692] Updated weights for policy 0, policy_version 1032098 (0.0008) [2023-12-26 22:53:21,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 528621568. Throughput: 0: 9617.7, 1: 9866.0. Samples: 528606920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:53:21,063][104569] Avg episode reward: [(0, '8754.193'), (1, '3671.048')] [2023-12-26 22:53:21,476][105620] Updated weights for policy 1, policy_version 1032555 (0.0010) [2023-12-26 22:53:21,538][105620] Updated weights for policy 1, policy_version 1032565 (0.0010) [2023-12-26 22:53:21,596][105620] Updated weights for policy 1, policy_version 1032575 (0.0010) [2023-12-26 22:53:21,809][105692] Updated weights for policy 0, policy_version 1032108 (0.0007) [2023-12-26 22:53:21,875][105692] Updated weights for policy 0, policy_version 1032118 (0.0006) [2023-12-26 22:53:21,935][105692] Updated weights for policy 0, policy_version 1032128 (0.0009) [2023-12-26 22:53:22,476][105620] Updated weights for policy 1, policy_version 1032585 (0.0010) [2023-12-26 22:53:22,530][105620] Updated weights for policy 1, policy_version 1032595 (0.0009) [2023-12-26 22:53:22,537][105692] Updated weights for policy 0, policy_version 1032138 (0.0007) [2023-12-26 22:53:22,589][105620] Updated weights for policy 1, policy_version 1032605 (0.0007) [2023-12-26 22:53:22,599][105692] Updated weights for policy 0, policy_version 1032148 (0.0008) [2023-12-26 22:53:22,643][105620] Updated weights for policy 1, policy_version 1032615 (0.0006) [2023-12-26 22:53:22,658][105692] Updated weights for policy 0, policy_version 1032158 (0.0007) [2023-12-26 22:53:22,721][105692] Updated weights for policy 0, policy_version 1032168 (0.0009) [2023-12-26 22:53:23,409][105620] Updated weights for policy 1, policy_version 1032625 (0.0009) [2023-12-26 22:53:23,459][105620] Updated weights for policy 1, policy_version 1032635 (0.0008) [2023-12-26 22:53:23,469][105692] Updated weights for policy 0, policy_version 1032178 (0.0005) [2023-12-26 22:53:23,518][105620] Updated weights for policy 1, policy_version 1032645 (0.0009) [2023-12-26 22:53:23,522][105692] Updated weights for policy 0, policy_version 1032188 (0.0006) [2023-12-26 22:53:23,571][105692] Updated weights for policy 0, policy_version 1032198 (0.0005) [2023-12-26 22:53:24,284][105620] Updated weights for policy 1, policy_version 1032655 (0.0007) [2023-12-26 22:53:24,302][105692] Updated weights for policy 0, policy_version 1032208 (0.0008) [2023-12-26 22:53:24,343][105620] Updated weights for policy 1, policy_version 1032665 (0.0006) [2023-12-26 22:53:24,365][105692] Updated weights for policy 0, policy_version 1032218 (0.0008) [2023-12-26 22:53:24,400][105620] Updated weights for policy 1, policy_version 1032675 (0.0006) [2023-12-26 22:53:24,422][105692] Updated weights for policy 0, policy_version 1032228 (0.0008) [2023-12-26 22:53:25,083][105692] Updated weights for policy 0, policy_version 1032238 (0.0008) [2023-12-26 22:53:25,145][105692] Updated weights for policy 0, policy_version 1032248 (0.0010) [2023-12-26 22:53:25,148][105620] Updated weights for policy 1, policy_version 1032685 (0.0006) [2023-12-26 22:53:25,203][105620] Updated weights for policy 1, policy_version 1032695 (0.0006) [2023-12-26 22:53:25,207][105692] Updated weights for policy 0, policy_version 1032258 (0.0010) [2023-12-26 22:53:25,259][105620] Updated weights for policy 1, policy_version 1032705 (0.0007) [2023-12-26 22:53:25,872][105692] Updated weights for policy 0, policy_version 1032268 (0.0010) [2023-12-26 22:53:25,919][105692] Updated weights for policy 0, policy_version 1032278 (0.0009) [2023-12-26 22:53:25,953][105620] Updated weights for policy 1, policy_version 1032715 (0.0009) [2023-12-26 22:53:25,967][105692] Updated weights for policy 0, policy_version 1032288 (0.0008) [2023-12-26 22:53:26,010][105620] Updated weights for policy 1, policy_version 1032725 (0.0006) [2023-12-26 22:53:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 528711680. Throughput: 0: 9655.8, 1: 9726.1. Samples: 528720164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:53:26,062][104569] Avg episode reward: [(0, '8910.673'), (1, '4373.756')] [2023-12-26 22:53:26,069][105620] Updated weights for policy 1, policy_version 1032735 (0.0008) [2023-12-26 22:53:26,683][105692] Updated weights for policy 0, policy_version 1032298 (0.0009) [2023-12-26 22:53:26,747][105692] Updated weights for policy 0, policy_version 1032308 (0.0009) [2023-12-26 22:53:26,776][105620] Updated weights for policy 1, policy_version 1032745 (0.0008) [2023-12-26 22:53:26,802][105692] Updated weights for policy 0, policy_version 1032318 (0.0011) [2023-12-26 22:53:26,829][105620] Updated weights for policy 1, policy_version 1032755 (0.0006) [2023-12-26 22:53:26,866][105692] Updated weights for policy 0, policy_version 1032328 (0.0011) [2023-12-26 22:53:26,888][105620] Updated weights for policy 1, policy_version 1032765 (0.0007) [2023-12-26 22:53:26,940][105620] Updated weights for policy 1, policy_version 1032775 (0.0009) [2023-12-26 22:53:27,488][105692] Updated weights for policy 0, policy_version 1032338 (0.0008) [2023-12-26 22:53:27,535][105692] Updated weights for policy 0, policy_version 1032348 (0.0008) [2023-12-26 22:53:27,594][105692] Updated weights for policy 0, policy_version 1032358 (0.0009) [2023-12-26 22:53:27,736][105620] Updated weights for policy 1, policy_version 1032785 (0.0006) [2023-12-26 22:53:27,792][105620] Updated weights for policy 1, policy_version 1032795 (0.0005) [2023-12-26 22:53:27,847][105620] Updated weights for policy 1, policy_version 1032805 (0.0005) [2023-12-26 22:53:28,273][105692] Updated weights for policy 0, policy_version 1032368 (0.0008) [2023-12-26 22:53:28,323][105692] Updated weights for policy 0, policy_version 1032378 (0.0008) [2023-12-26 22:53:28,385][105692] Updated weights for policy 0, policy_version 1032388 (0.0008) [2023-12-26 22:53:28,519][105620] Updated weights for policy 1, policy_version 1032815 (0.0009) [2023-12-26 22:53:28,578][105620] Updated weights for policy 1, policy_version 1032825 (0.0009) [2023-12-26 22:53:28,637][105620] Updated weights for policy 1, policy_version 1032835 (0.0009) [2023-12-26 22:53:29,029][105692] Updated weights for policy 0, policy_version 1032398 (0.0007) [2023-12-26 22:53:29,085][105692] Updated weights for policy 0, policy_version 1032408 (0.0006) [2023-12-26 22:53:29,132][105692] Updated weights for policy 0, policy_version 1032418 (0.0005) [2023-12-26 22:53:29,293][105620] Updated weights for policy 1, policy_version 1032845 (0.0008) [2023-12-26 22:53:29,354][105620] Updated weights for policy 1, policy_version 1032855 (0.0009) [2023-12-26 22:53:29,420][105620] Updated weights for policy 1, policy_version 1032865 (0.0010) [2023-12-26 22:53:29,715][105692] Updated weights for policy 0, policy_version 1032428 (0.0008) [2023-12-26 22:53:29,770][105692] Updated weights for policy 0, policy_version 1032438 (0.0011) [2023-12-26 22:53:29,832][105692] Updated weights for policy 0, policy_version 1032448 (0.0007) [2023-12-26 22:53:30,105][105620] Updated weights for policy 1, policy_version 1032875 (0.0011) [2023-12-26 22:53:30,164][105620] Updated weights for policy 1, policy_version 1032885 (0.0010) [2023-12-26 22:53:30,227][105620] Updated weights for policy 1, policy_version 1032895 (0.0008) [2023-12-26 22:53:30,425][105692] Updated weights for policy 0, policy_version 1032458 (0.0007) [2023-12-26 22:53:30,471][105692] Updated weights for policy 0, policy_version 1032468 (0.0006) [2023-12-26 22:53:30,527][105692] Updated weights for policy 0, policy_version 1032478 (0.0010) [2023-12-26 22:53:30,581][105692] Updated weights for policy 0, policy_version 1032488 (0.0010) [2023-12-26 22:53:30,881][105620] Updated weights for policy 1, policy_version 1032905 (0.0010) [2023-12-26 22:53:30,936][105620] Updated weights for policy 1, policy_version 1032915 (0.0005) [2023-12-26 22:53:30,987][105620] Updated weights for policy 1, policy_version 1032925 (0.0005) [2023-12-26 22:53:31,054][105620] Updated weights for policy 1, policy_version 1032935 (0.0007) [2023-12-26 22:53:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 528818176. Throughput: 0: 9703.1, 1: 9702.4. Samples: 528778948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:53:31,063][104569] Avg episode reward: [(0, '9176.535'), (1, '6487.358')] [2023-12-26 22:53:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001032488_264355840.pth... [2023-12-26 22:53:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001032936_264462336.pth... [2023-12-26 22:53:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001031816_264175616.pth [2023-12-26 22:53:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001031336_264060928.pth [2023-12-26 22:53:31,276][105692] Updated weights for policy 0, policy_version 1032498 (0.0010) [2023-12-26 22:53:31,334][105692] Updated weights for policy 0, policy_version 1032508 (0.0010) [2023-12-26 22:53:31,407][105692] Updated weights for policy 0, policy_version 1032518 (0.0009) [2023-12-26 22:53:31,709][105620] Updated weights for policy 1, policy_version 1032946 (0.0010) [2023-12-26 22:53:31,773][105620] Updated weights for policy 1, policy_version 1032956 (0.0008) [2023-12-26 22:53:31,829][105620] Updated weights for policy 1, policy_version 1032966 (0.0009) [2023-12-26 22:53:32,078][105692] Updated weights for policy 0, policy_version 1032528 (0.0008) [2023-12-26 22:53:32,133][105692] Updated weights for policy 0, policy_version 1032538 (0.0008) [2023-12-26 22:53:32,196][105692] Updated weights for policy 0, policy_version 1032548 (0.0008) [2023-12-26 22:53:32,597][105620] Updated weights for policy 1, policy_version 1032976 (0.0006) [2023-12-26 22:53:32,652][105620] Updated weights for policy 1, policy_version 1032986 (0.0006) [2023-12-26 22:53:32,720][105620] Updated weights for policy 1, policy_version 1032996 (0.0005) [2023-12-26 22:53:32,929][105692] Updated weights for policy 0, policy_version 1032558 (0.0010) [2023-12-26 22:53:32,983][105692] Updated weights for policy 0, policy_version 1032568 (0.0009) [2023-12-26 22:53:33,044][105692] Updated weights for policy 0, policy_version 1032578 (0.0010) [2023-12-26 22:53:33,252][105620] Updated weights for policy 1, policy_version 1033006 (0.0008) [2023-12-26 22:53:33,303][105620] Updated weights for policy 1, policy_version 1033016 (0.0010) [2023-12-26 22:53:33,350][105620] Updated weights for policy 1, policy_version 1033026 (0.0010) [2023-12-26 22:53:33,722][105692] Updated weights for policy 0, policy_version 1032588 (0.0010) [2023-12-26 22:53:33,769][105692] Updated weights for policy 0, policy_version 1032598 (0.0010) [2023-12-26 22:53:33,811][105692] Updated weights for policy 0, policy_version 1032608 (0.0005) [2023-12-26 22:53:34,090][105620] Updated weights for policy 1, policy_version 1033036 (0.0010) [2023-12-26 22:53:34,164][105620] Updated weights for policy 1, policy_version 1033046 (0.0010) [2023-12-26 22:53:34,225][105620] Updated weights for policy 1, policy_version 1033056 (0.0007) [2023-12-26 22:53:34,465][105692] Updated weights for policy 0, policy_version 1032618 (0.0005) [2023-12-26 22:53:34,525][105692] Updated weights for policy 0, policy_version 1032628 (0.0007) [2023-12-26 22:53:34,585][105692] Updated weights for policy 0, policy_version 1032638 (0.0007) [2023-12-26 22:53:34,649][105692] Updated weights for policy 0, policy_version 1032648 (0.0009) [2023-12-26 22:53:34,899][105620] Updated weights for policy 1, policy_version 1033066 (0.0008) [2023-12-26 22:53:34,960][105620] Updated weights for policy 1, policy_version 1033076 (0.0005) [2023-12-26 22:53:35,023][105620] Updated weights for policy 1, policy_version 1033086 (0.0009) [2023-12-26 22:53:35,084][105620] Updated weights for policy 1, policy_version 1033096 (0.0005) [2023-12-26 22:53:35,361][105692] Updated weights for policy 0, policy_version 1032658 (0.0006) [2023-12-26 22:53:35,407][105692] Updated weights for policy 0, policy_version 1032668 (0.0005) [2023-12-26 22:53:35,460][105692] Updated weights for policy 0, policy_version 1032678 (0.0007) [2023-12-26 22:53:35,697][105620] Updated weights for policy 1, policy_version 1033106 (0.0011) [2023-12-26 22:53:35,763][105620] Updated weights for policy 1, policy_version 1033116 (0.0011) [2023-12-26 22:53:35,822][105620] Updated weights for policy 1, policy_version 1033126 (0.0011) [2023-12-26 22:53:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 528916480. Throughput: 0: 9857.4, 1: 9682.4. Samples: 528904536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:53:36,062][104569] Avg episode reward: [(0, '9086.199'), (1, '8901.172')] [2023-12-26 22:53:36,088][105692] Updated weights for policy 0, policy_version 1032688 (0.0007) [2023-12-26 22:53:36,154][105692] Updated weights for policy 0, policy_version 1032698 (0.0008) [2023-12-26 22:53:36,214][105692] Updated weights for policy 0, policy_version 1032708 (0.0008) [2023-12-26 22:53:36,556][105620] Updated weights for policy 1, policy_version 1033136 (0.0011) [2023-12-26 22:53:36,616][105620] Updated weights for policy 1, policy_version 1033146 (0.0011) [2023-12-26 22:53:36,676][105620] Updated weights for policy 1, policy_version 1033156 (0.0009) [2023-12-26 22:53:36,920][105692] Updated weights for policy 0, policy_version 1032718 (0.0006) [2023-12-26 22:53:36,994][105692] Updated weights for policy 0, policy_version 1032728 (0.0005) [2023-12-26 22:53:37,054][105692] Updated weights for policy 0, policy_version 1032738 (0.0007) [2023-12-26 22:53:37,368][105620] Updated weights for policy 1, policy_version 1033166 (0.0009) [2023-12-26 22:53:37,425][105620] Updated weights for policy 1, policy_version 1033176 (0.0008) [2023-12-26 22:53:37,478][105620] Updated weights for policy 1, policy_version 1033186 (0.0006) [2023-12-26 22:53:37,680][105692] Updated weights for policy 0, policy_version 1032748 (0.0009) [2023-12-26 22:53:37,744][105692] Updated weights for policy 0, policy_version 1032758 (0.0009) [2023-12-26 22:53:37,803][105692] Updated weights for policy 0, policy_version 1032768 (0.0009) [2023-12-26 22:53:38,182][105620] Updated weights for policy 1, policy_version 1033196 (0.0007) [2023-12-26 22:53:38,232][105620] Updated weights for policy 1, policy_version 1033206 (0.0008) [2023-12-26 22:53:38,293][105620] Updated weights for policy 1, policy_version 1033216 (0.0009) [2023-12-26 22:53:38,548][105692] Updated weights for policy 0, policy_version 1032778 (0.0008) [2023-12-26 22:53:38,619][105692] Updated weights for policy 0, policy_version 1032788 (0.0006) [2023-12-26 22:53:38,680][105692] Updated weights for policy 0, policy_version 1032798 (0.0008) [2023-12-26 22:53:38,753][105692] Updated weights for policy 0, policy_version 1032808 (0.0009) [2023-12-26 22:53:39,085][105620] Updated weights for policy 1, policy_version 1033226 (0.0008) [2023-12-26 22:53:39,153][105620] Updated weights for policy 1, policy_version 1033236 (0.0005) [2023-12-26 22:53:39,223][105620] Updated weights for policy 1, policy_version 1033246 (0.0009) [2023-12-26 22:53:39,286][105620] Updated weights for policy 1, policy_version 1033256 (0.0009) [2023-12-26 22:53:39,484][105692] Updated weights for policy 0, policy_version 1032818 (0.0009) [2023-12-26 22:53:39,541][105692] Updated weights for policy 0, policy_version 1032828 (0.0009) [2023-12-26 22:53:39,603][105692] Updated weights for policy 0, policy_version 1032838 (0.0009) [2023-12-26 22:53:40,019][105620] Updated weights for policy 1, policy_version 1033266 (0.0008) [2023-12-26 22:53:40,082][105620] Updated weights for policy 1, policy_version 1033276 (0.0009) [2023-12-26 22:53:40,148][105620] Updated weights for policy 1, policy_version 1033286 (0.0010) [2023-12-26 22:53:40,364][105692] Updated weights for policy 0, policy_version 1032848 (0.0010) [2023-12-26 22:53:40,428][105692] Updated weights for policy 0, policy_version 1032858 (0.0008) [2023-12-26 22:53:40,484][105692] Updated weights for policy 0, policy_version 1032868 (0.0005) [2023-12-26 22:53:40,863][105620] Updated weights for policy 1, policy_version 1033296 (0.0009) [2023-12-26 22:53:40,920][105620] Updated weights for policy 1, policy_version 1033306 (0.0008) [2023-12-26 22:53:40,965][105620] Updated weights for policy 1, policy_version 1033316 (0.0008) [2023-12-26 22:53:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 529014784. Throughput: 0: 9828.2, 1: 9703.7. Samples: 529021012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:53:41,062][104569] Avg episode reward: [(0, '8997.525'), (1, '9083.142')] [2023-12-26 22:53:41,143][105692] Updated weights for policy 0, policy_version 1032878 (0.0008) [2023-12-26 22:53:41,213][105692] Updated weights for policy 0, policy_version 1032888 (0.0007) [2023-12-26 22:53:41,277][105692] Updated weights for policy 0, policy_version 1032898 (0.0008) [2023-12-26 22:53:41,798][105620] Updated weights for policy 1, policy_version 1033326 (0.0009) [2023-12-26 22:53:41,842][105620] Updated weights for policy 1, policy_version 1033336 (0.0008) [2023-12-26 22:53:41,893][105620] Updated weights for policy 1, policy_version 1033346 (0.0008) [2023-12-26 22:53:42,065][105692] Updated weights for policy 0, policy_version 1032908 (0.0010) [2023-12-26 22:53:42,120][105692] Updated weights for policy 0, policy_version 1032918 (0.0009) [2023-12-26 22:53:42,169][105692] Updated weights for policy 0, policy_version 1032928 (0.0009) [2023-12-26 22:53:42,674][105620] Updated weights for policy 1, policy_version 1033356 (0.0008) [2023-12-26 22:53:42,726][105620] Updated weights for policy 1, policy_version 1033366 (0.0008) [2023-12-26 22:53:42,781][105620] Updated weights for policy 1, policy_version 1033376 (0.0007) [2023-12-26 22:53:42,956][105692] Updated weights for policy 0, policy_version 1032938 (0.0009) [2023-12-26 22:53:43,009][105692] Updated weights for policy 0, policy_version 1032948 (0.0010) [2023-12-26 22:53:43,064][105692] Updated weights for policy 0, policy_version 1032958 (0.0010) [2023-12-26 22:53:43,118][105692] Updated weights for policy 0, policy_version 1032968 (0.0010) [2023-12-26 22:53:43,546][105620] Updated weights for policy 1, policy_version 1033386 (0.0008) [2023-12-26 22:53:43,597][105620] Updated weights for policy 1, policy_version 1033396 (0.0009) [2023-12-26 22:53:43,648][105620] Updated weights for policy 1, policy_version 1033406 (0.0008) [2023-12-26 22:53:43,706][105620] Updated weights for policy 1, policy_version 1033416 (0.0008) [2023-12-26 22:53:43,878][105692] Updated weights for policy 0, policy_version 1032978 (0.0010) [2023-12-26 22:53:43,929][105692] Updated weights for policy 0, policy_version 1032988 (0.0010) [2023-12-26 22:53:43,974][105692] Updated weights for policy 0, policy_version 1032998 (0.0010) [2023-12-26 22:53:44,468][105620] Updated weights for policy 1, policy_version 1033426 (0.0008) [2023-12-26 22:53:44,530][105620] Updated weights for policy 1, policy_version 1033436 (0.0008) [2023-12-26 22:53:44,581][105620] Updated weights for policy 1, policy_version 1033446 (0.0008) [2023-12-26 22:53:44,742][105692] Updated weights for policy 0, policy_version 1033008 (0.0010) [2023-12-26 22:53:44,827][105692] Updated weights for policy 0, policy_version 1033018 (0.0011) [2023-12-26 22:53:44,883][105692] Updated weights for policy 0, policy_version 1033028 (0.0011) [2023-12-26 22:53:45,349][105620] Updated weights for policy 1, policy_version 1033456 (0.0010) [2023-12-26 22:53:45,399][105620] Updated weights for policy 1, policy_version 1033466 (0.0010) [2023-12-26 22:53:45,455][105620] Updated weights for policy 1, policy_version 1033476 (0.0007) [2023-12-26 22:53:45,624][105692] Updated weights for policy 0, policy_version 1033038 (0.0010) [2023-12-26 22:53:45,682][105692] Updated weights for policy 0, policy_version 1033048 (0.0010) [2023-12-26 22:53:45,747][105692] Updated weights for policy 0, policy_version 1033058 (0.0010) [2023-12-26 22:53:46,006][105620] Updated weights for policy 1, policy_version 1033486 (0.0007) [2023-12-26 22:53:46,058][105620] Updated weights for policy 1, policy_version 1033496 (0.0005) [2023-12-26 22:53:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 529104896. Throughput: 0: 9827.8, 1: 9635.5. Samples: 529076620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 22:53:46,063][104569] Avg episode reward: [(0, '8995.071'), (1, '9089.759')] [2023-12-26 22:53:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001033064_264503296.pth... [2023-12-26 22:53:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001031912_264208384.pth [2023-12-26 22:53:46,123][105620] Updated weights for policy 1, policy_version 1033506 (0.0006) [2023-12-26 22:53:46,151][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001033512_264609792.pth... [2023-12-26 22:53:46,153][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001032360_264314880.pth [2023-12-26 22:53:46,481][105692] Updated weights for policy 0, policy_version 1033068 (0.0010) [2023-12-26 22:53:46,529][105692] Updated weights for policy 0, policy_version 1033078 (0.0010) [2023-12-26 22:53:46,577][105692] Updated weights for policy 0, policy_version 1033088 (0.0010) [2023-12-26 22:53:46,703][105620] Updated weights for policy 1, policy_version 1033516 (0.0006) [2023-12-26 22:53:46,755][105620] Updated weights for policy 1, policy_version 1033526 (0.0007) [2023-12-26 22:53:46,809][105620] Updated weights for policy 1, policy_version 1033536 (0.0006) [2023-12-26 22:53:47,339][105692] Updated weights for policy 0, policy_version 1033098 (0.0010) [2023-12-26 22:53:47,403][105692] Updated weights for policy 0, policy_version 1033108 (0.0010) [2023-12-26 22:53:47,406][105620] Updated weights for policy 1, policy_version 1033546 (0.0006) [2023-12-26 22:53:47,463][105692] Updated weights for policy 0, policy_version 1033118 (0.0010) [2023-12-26 22:53:47,468][105620] Updated weights for policy 1, policy_version 1033556 (0.0008) [2023-12-26 22:53:47,519][105692] Updated weights for policy 0, policy_version 1033128 (0.0011) [2023-12-26 22:53:47,528][105620] Updated weights for policy 1, policy_version 1033566 (0.0008) [2023-12-26 22:53:47,583][105620] Updated weights for policy 1, policy_version 1033576 (0.0008) [2023-12-26 22:53:48,266][105692] Updated weights for policy 0, policy_version 1033138 (0.0007) [2023-12-26 22:53:48,300][105620] Updated weights for policy 1, policy_version 1033586 (0.0006) [2023-12-26 22:53:48,316][105692] Updated weights for policy 0, policy_version 1033148 (0.0007) [2023-12-26 22:53:48,365][105620] Updated weights for policy 1, policy_version 1033596 (0.0007) [2023-12-26 22:53:48,381][105692] Updated weights for policy 0, policy_version 1033158 (0.0007) [2023-12-26 22:53:48,428][105620] Updated weights for policy 1, policy_version 1033606 (0.0008) [2023-12-26 22:53:48,946][105692] Updated weights for policy 0, policy_version 1033168 (0.0006) [2023-12-26 22:53:49,007][105692] Updated weights for policy 0, policy_version 1033178 (0.0005) [2023-12-26 22:53:49,072][105692] Updated weights for policy 0, policy_version 1033188 (0.0009) [2023-12-26 22:53:49,302][105620] Updated weights for policy 1, policy_version 1033616 (0.0009) [2023-12-26 22:53:49,372][105620] Updated weights for policy 1, policy_version 1033626 (0.0009) [2023-12-26 22:53:49,426][105620] Updated weights for policy 1, policy_version 1033636 (0.0008) [2023-12-26 22:53:49,738][105692] Updated weights for policy 0, policy_version 1033198 (0.0010) [2023-12-26 22:53:49,807][105692] Updated weights for policy 0, policy_version 1033208 (0.0010) [2023-12-26 22:53:49,870][105692] Updated weights for policy 0, policy_version 1033218 (0.0009) [2023-12-26 22:53:50,136][105620] Updated weights for policy 1, policy_version 1033646 (0.0007) [2023-12-26 22:53:50,191][105620] Updated weights for policy 1, policy_version 1033656 (0.0007) [2023-12-26 22:53:50,251][105620] Updated weights for policy 1, policy_version 1033666 (0.0007) [2023-12-26 22:53:50,566][105692] Updated weights for policy 0, policy_version 1033228 (0.0010) [2023-12-26 22:53:50,624][105692] Updated weights for policy 0, policy_version 1033238 (0.0009) [2023-12-26 22:53:50,680][105692] Updated weights for policy 0, policy_version 1033248 (0.0009) [2023-12-26 22:53:50,937][105620] Updated weights for policy 1, policy_version 1033676 (0.0006) [2023-12-26 22:53:50,990][105620] Updated weights for policy 1, policy_version 1033686 (0.0005) [2023-12-26 22:53:51,052][105620] Updated weights for policy 1, policy_version 1033696 (0.0008) [2023-12-26 22:53:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 529203200. Throughput: 0: 9799.6, 1: 9664.1. Samples: 529195004. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:53:51,062][104569] Avg episode reward: [(0, '8997.663'), (1, '9008.691')] [2023-12-26 22:53:51,472][105692] Updated weights for policy 0, policy_version 1033258 (0.0008) [2023-12-26 22:53:51,525][105692] Updated weights for policy 0, policy_version 1033268 (0.0006) [2023-12-26 22:53:51,588][105692] Updated weights for policy 0, policy_version 1033278 (0.0008) [2023-12-26 22:53:51,653][105692] Updated weights for policy 0, policy_version 1033288 (0.0008) [2023-12-26 22:53:51,750][105620] Updated weights for policy 1, policy_version 1033706 (0.0008) [2023-12-26 22:53:51,822][105620] Updated weights for policy 1, policy_version 1033716 (0.0011) [2023-12-26 22:53:51,879][105620] Updated weights for policy 1, policy_version 1033726 (0.0010) [2023-12-26 22:53:51,938][105620] Updated weights for policy 1, policy_version 1033736 (0.0010) [2023-12-26 22:53:52,353][105692] Updated weights for policy 0, policy_version 1033298 (0.0009) [2023-12-26 22:53:52,414][105692] Updated weights for policy 0, policy_version 1033308 (0.0008) [2023-12-26 22:53:52,469][105692] Updated weights for policy 0, policy_version 1033318 (0.0008) [2023-12-26 22:53:52,713][105620] Updated weights for policy 1, policy_version 1033746 (0.0006) [2023-12-26 22:53:52,772][105620] Updated weights for policy 1, policy_version 1033756 (0.0005) [2023-12-26 22:53:52,830][105620] Updated weights for policy 1, policy_version 1033766 (0.0008) [2023-12-26 22:53:53,157][105692] Updated weights for policy 0, policy_version 1033328 (0.0010) [2023-12-26 22:53:53,202][105692] Updated weights for policy 0, policy_version 1033338 (0.0010) [2023-12-26 22:53:53,254][105692] Updated weights for policy 0, policy_version 1033348 (0.0010) [2023-12-26 22:53:53,492][105620] Updated weights for policy 1, policy_version 1033776 (0.0010) [2023-12-26 22:53:53,543][105620] Updated weights for policy 1, policy_version 1033786 (0.0010) [2023-12-26 22:53:53,598][105620] Updated weights for policy 1, policy_version 1033796 (0.0010) [2023-12-26 22:53:53,939][105692] Updated weights for policy 0, policy_version 1033358 (0.0007) [2023-12-26 22:53:53,990][105692] Updated weights for policy 0, policy_version 1033368 (0.0005) [2023-12-26 22:53:54,046][105692] Updated weights for policy 0, policy_version 1033378 (0.0005) [2023-12-26 22:53:54,313][105620] Updated weights for policy 1, policy_version 1033806 (0.0010) [2023-12-26 22:53:54,364][105620] Updated weights for policy 1, policy_version 1033816 (0.0009) [2023-12-26 22:53:54,415][105620] Updated weights for policy 1, policy_version 1033826 (0.0008) [2023-12-26 22:53:54,627][105692] Updated weights for policy 0, policy_version 1033388 (0.0007) [2023-12-26 22:53:54,694][105692] Updated weights for policy 0, policy_version 1033398 (0.0008) [2023-12-26 22:53:54,766][105692] Updated weights for policy 0, policy_version 1033408 (0.0010) [2023-12-26 22:53:55,070][105620] Updated weights for policy 1, policy_version 1033836 (0.0009) [2023-12-26 22:53:55,126][105620] Updated weights for policy 1, policy_version 1033846 (0.0008) [2023-12-26 22:53:55,184][105620] Updated weights for policy 1, policy_version 1033856 (0.0008) [2023-12-26 22:53:55,489][105692] Updated weights for policy 0, policy_version 1033418 (0.0010) [2023-12-26 22:53:55,542][105692] Updated weights for policy 0, policy_version 1033428 (0.0010) [2023-12-26 22:53:55,600][105692] Updated weights for policy 0, policy_version 1033438 (0.0010) [2023-12-26 22:53:55,651][105692] Updated weights for policy 0, policy_version 1033448 (0.0010) [2023-12-26 22:53:55,918][105620] Updated weights for policy 1, policy_version 1033866 (0.0009) [2023-12-26 22:53:55,973][105620] Updated weights for policy 1, policy_version 1033876 (0.0010) [2023-12-26 22:53:56,024][105620] Updated weights for policy 1, policy_version 1033886 (0.0010) [2023-12-26 22:53:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 529301504. Throughput: 0: 9786.8, 1: 9721.3. Samples: 529314056. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:53:56,062][104569] Avg episode reward: [(0, '8641.059'), (1, '8915.355')] [2023-12-26 22:53:56,082][105620] Updated weights for policy 1, policy_version 1033896 (0.0005) [2023-12-26 22:53:56,382][105692] Updated weights for policy 0, policy_version 1033458 (0.0011) [2023-12-26 22:53:56,439][105692] Updated weights for policy 0, policy_version 1033468 (0.0010) [2023-12-26 22:53:56,498][105692] Updated weights for policy 0, policy_version 1033478 (0.0010) [2023-12-26 22:53:56,702][105620] Updated weights for policy 1, policy_version 1033906 (0.0010) [2023-12-26 22:53:56,757][105620] Updated weights for policy 1, policy_version 1033916 (0.0007) [2023-12-26 22:53:56,815][105620] Updated weights for policy 1, policy_version 1033926 (0.0009) [2023-12-26 22:53:57,248][105692] Updated weights for policy 0, policy_version 1033488 (0.0010) [2023-12-26 22:53:57,313][105692] Updated weights for policy 0, policy_version 1033498 (0.0010) [2023-12-26 22:53:57,360][105620] Updated weights for policy 1, policy_version 1033936 (0.0007) [2023-12-26 22:53:57,368][105692] Updated weights for policy 0, policy_version 1033508 (0.0010) [2023-12-26 22:53:57,411][105620] Updated weights for policy 1, policy_version 1033946 (0.0010) [2023-12-26 22:53:57,462][105620] Updated weights for policy 1, policy_version 1033956 (0.0010) [2023-12-26 22:53:58,075][105692] Updated weights for policy 0, policy_version 1033518 (0.0010) [2023-12-26 22:53:58,085][105620] Updated weights for policy 1, policy_version 1033966 (0.0007) [2023-12-26 22:53:58,123][105692] Updated weights for policy 0, policy_version 1033528 (0.0010) [2023-12-26 22:53:58,146][105620] Updated weights for policy 1, policy_version 1033976 (0.0006) [2023-12-26 22:53:58,188][105692] Updated weights for policy 0, policy_version 1033538 (0.0008) [2023-12-26 22:53:58,216][105620] Updated weights for policy 1, policy_version 1033986 (0.0009) [2023-12-26 22:53:58,987][105620] Updated weights for policy 1, policy_version 1033996 (0.0009) [2023-12-26 22:53:59,043][105620] Updated weights for policy 1, policy_version 1034006 (0.0006) [2023-12-26 22:53:59,050][105692] Updated weights for policy 0, policy_version 1033548 (0.0010) [2023-12-26 22:53:59,099][105620] Updated weights for policy 1, policy_version 1034016 (0.0010) [2023-12-26 22:53:59,106][105692] Updated weights for policy 0, policy_version 1033558 (0.0008) [2023-12-26 22:53:59,159][105692] Updated weights for policy 0, policy_version 1033568 (0.0008) [2023-12-26 22:53:59,820][105620] Updated weights for policy 1, policy_version 1034026 (0.0010) [2023-12-26 22:53:59,854][105692] Updated weights for policy 0, policy_version 1033578 (0.0008) [2023-12-26 22:53:59,882][105620] Updated weights for policy 1, policy_version 1034036 (0.0009) [2023-12-26 22:53:59,910][105692] Updated weights for policy 0, policy_version 1033588 (0.0009) [2023-12-26 22:53:59,942][105620] Updated weights for policy 1, policy_version 1034046 (0.0009) [2023-12-26 22:53:59,970][105692] Updated weights for policy 0, policy_version 1033598 (0.0009) [2023-12-26 22:53:59,998][105620] Updated weights for policy 1, policy_version 1034056 (0.0009) [2023-12-26 22:54:00,027][105692] Updated weights for policy 0, policy_version 1033608 (0.0009) [2023-12-26 22:54:00,696][105692] Updated weights for policy 0, policy_version 1033618 (0.0009) [2023-12-26 22:54:00,716][105620] Updated weights for policy 1, policy_version 1034066 (0.0005) [2023-12-26 22:54:00,746][105692] Updated weights for policy 0, policy_version 1033628 (0.0010) [2023-12-26 22:54:00,780][105620] Updated weights for policy 1, policy_version 1034076 (0.0005) [2023-12-26 22:54:00,792][105692] Updated weights for policy 0, policy_version 1033638 (0.0008) [2023-12-26 22:54:00,837][105620] Updated weights for policy 1, policy_version 1034086 (0.0005) [2023-12-26 22:54:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 529408000. Throughput: 0: 9787.0, 1: 9808.3. Samples: 529374600. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:54:01,062][104569] Avg episode reward: [(0, '8548.386'), (1, '9172.960')] [2023-12-26 22:54:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001033640_264650752.pth... [2023-12-26 22:54:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001034088_264757248.pth... [2023-12-26 22:54:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001032936_264462336.pth [2023-12-26 22:54:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001032488_264355840.pth [2023-12-26 22:54:01,414][105620] Updated weights for policy 1, policy_version 1034096 (0.0007) [2023-12-26 22:54:01,481][105620] Updated weights for policy 1, policy_version 1034106 (0.0005) [2023-12-26 22:54:01,544][105620] Updated weights for policy 1, policy_version 1034116 (0.0005) [2023-12-26 22:54:01,557][105692] Updated weights for policy 0, policy_version 1033648 (0.0010) [2023-12-26 22:54:01,616][105692] Updated weights for policy 0, policy_version 1033658 (0.0010) [2023-12-26 22:54:01,682][105692] Updated weights for policy 0, policy_version 1033668 (0.0010) [2023-12-26 22:54:02,160][105620] Updated weights for policy 1, policy_version 1034126 (0.0006) [2023-12-26 22:54:02,218][105620] Updated weights for policy 1, policy_version 1034136 (0.0010) [2023-12-26 22:54:02,282][105620] Updated weights for policy 1, policy_version 1034146 (0.0011) [2023-12-26 22:54:02,437][105692] Updated weights for policy 0, policy_version 1033678 (0.0009) [2023-12-26 22:54:02,490][105692] Updated weights for policy 0, policy_version 1033688 (0.0008) [2023-12-26 22:54:02,551][105692] Updated weights for policy 0, policy_version 1033698 (0.0008) [2023-12-26 22:54:02,950][105620] Updated weights for policy 1, policy_version 1034156 (0.0007) [2023-12-26 22:54:03,001][105620] Updated weights for policy 1, policy_version 1034166 (0.0005) [2023-12-26 22:54:03,063][105620] Updated weights for policy 1, policy_version 1034176 (0.0005) [2023-12-26 22:54:03,357][105692] Updated weights for policy 0, policy_version 1033708 (0.0008) [2023-12-26 22:54:03,415][105692] Updated weights for policy 0, policy_version 1033718 (0.0007) [2023-12-26 22:54:03,477][105692] Updated weights for policy 0, policy_version 1033728 (0.0007) [2023-12-26 22:54:03,632][105620] Updated weights for policy 1, policy_version 1034186 (0.0005) [2023-12-26 22:54:03,687][105620] Updated weights for policy 1, policy_version 1034196 (0.0005) [2023-12-26 22:54:03,731][105620] Updated weights for policy 1, policy_version 1034206 (0.0005) [2023-12-26 22:54:03,780][105620] Updated weights for policy 1, policy_version 1034216 (0.0008) [2023-12-26 22:54:04,269][105692] Updated weights for policy 0, policy_version 1033738 (0.0008) [2023-12-26 22:54:04,329][105692] Updated weights for policy 0, policy_version 1033748 (0.0011) [2023-12-26 22:54:04,389][105692] Updated weights for policy 0, policy_version 1033758 (0.0010) [2023-12-26 22:54:04,450][105692] Updated weights for policy 0, policy_version 1033768 (0.0011) [2023-12-26 22:54:04,474][105620] Updated weights for policy 1, policy_version 1034226 (0.0011) [2023-12-26 22:54:04,544][105620] Updated weights for policy 1, policy_version 1034236 (0.0010) [2023-12-26 22:54:04,596][105620] Updated weights for policy 1, policy_version 1034246 (0.0006) [2023-12-26 22:54:05,125][105692] Updated weights for policy 0, policy_version 1033779 (0.0010) [2023-12-26 22:54:05,183][105692] Updated weights for policy 0, policy_version 1033791 (0.0010) [2023-12-26 22:54:05,234][105620] Updated weights for policy 1, policy_version 1034256 (0.0009) [2023-12-26 22:54:05,296][105620] Updated weights for policy 1, policy_version 1034266 (0.0010) [2023-12-26 22:54:05,354][105620] Updated weights for policy 1, policy_version 1034276 (0.0010) [2023-12-26 22:54:05,975][105692] Updated weights for policy 0, policy_version 1033801 (0.0007) [2023-12-26 22:54:06,024][105692] Updated weights for policy 0, policy_version 1033811 (0.0008) [2023-12-26 22:54:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.4, 300 sec: 19549.7). Total num frames: 529498112. Throughput: 0: 9776.6, 1: 9894.4. Samples: 529492116. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:54:06,062][104569] Avg episode reward: [(0, '8653.623'), (1, '9187.251')] [2023-12-26 22:54:06,069][105692] Updated weights for policy 0, policy_version 1033821 (0.0008) [2023-12-26 22:54:06,092][105620] Updated weights for policy 1, policy_version 1034286 (0.0010) [2023-12-26 22:54:06,135][105692] Updated weights for policy 0, policy_version 1033831 (0.0008) [2023-12-26 22:54:06,147][105620] Updated weights for policy 1, policy_version 1034296 (0.0009) [2023-12-26 22:54:06,200][105620] Updated weights for policy 1, policy_version 1034306 (0.0011) [2023-12-26 22:54:06,924][105692] Updated weights for policy 0, policy_version 1033841 (0.0006) [2023-12-26 22:54:06,989][105692] Updated weights for policy 0, policy_version 1033851 (0.0007) [2023-12-26 22:54:07,011][105620] Updated weights for policy 1, policy_version 1034316 (0.0009) [2023-12-26 22:54:07,042][105692] Updated weights for policy 0, policy_version 1033861 (0.0008) [2023-12-26 22:54:07,058][105620] Updated weights for policy 1, policy_version 1034326 (0.0005) [2023-12-26 22:54:07,107][105620] Updated weights for policy 1, policy_version 1034336 (0.0005) [2023-12-26 22:54:07,653][105620] Updated weights for policy 1, policy_version 1034346 (0.0005) [2023-12-26 22:54:07,700][105620] Updated weights for policy 1, policy_version 1034356 (0.0006) [2023-12-26 22:54:07,759][105620] Updated weights for policy 1, policy_version 1034366 (0.0006) [2023-12-26 22:54:07,811][105620] Updated weights for policy 1, policy_version 1034376 (0.0005) [2023-12-26 22:54:07,859][105692] Updated weights for policy 0, policy_version 1033871 (0.0009) [2023-12-26 22:54:07,908][105692] Updated weights for policy 0, policy_version 1033881 (0.0009) [2023-12-26 22:54:07,958][105692] Updated weights for policy 0, policy_version 1033891 (0.0009) [2023-12-26 22:54:08,479][105620] Updated weights for policy 1, policy_version 1034386 (0.0009) [2023-12-26 22:54:08,530][105620] Updated weights for policy 1, policy_version 1034396 (0.0009) [2023-12-26 22:54:08,584][105620] Updated weights for policy 1, policy_version 1034406 (0.0009) [2023-12-26 22:54:08,751][105692] Updated weights for policy 0, policy_version 1033901 (0.0009) [2023-12-26 22:54:08,801][105692] Updated weights for policy 0, policy_version 1033911 (0.0008) [2023-12-26 22:54:08,849][105692] Updated weights for policy 0, policy_version 1033921 (0.0009) [2023-12-26 22:54:09,350][105620] Updated weights for policy 1, policy_version 1034416 (0.0009) [2023-12-26 22:54:09,423][105620] Updated weights for policy 1, policy_version 1034426 (0.0009) [2023-12-26 22:54:09,485][105620] Updated weights for policy 1, policy_version 1034436 (0.0009) [2023-12-26 22:54:09,662][105692] Updated weights for policy 0, policy_version 1033931 (0.0009) [2023-12-26 22:54:09,724][105692] Updated weights for policy 0, policy_version 1033941 (0.0008) [2023-12-26 22:54:09,783][105692] Updated weights for policy 0, policy_version 1033951 (0.0008) [2023-12-26 22:54:10,256][105620] Updated weights for policy 1, policy_version 1034446 (0.0009) [2023-12-26 22:54:10,322][105620] Updated weights for policy 1, policy_version 1034456 (0.0009) [2023-12-26 22:54:10,388][105620] Updated weights for policy 1, policy_version 1034466 (0.0009) [2023-12-26 22:54:10,574][105692] Updated weights for policy 0, policy_version 1033961 (0.0009) [2023-12-26 22:54:10,634][105692] Updated weights for policy 0, policy_version 1033971 (0.0009) [2023-12-26 22:54:10,694][105692] Updated weights for policy 0, policy_version 1033981 (0.0009) [2023-12-26 22:54:10,760][105692] Updated weights for policy 0, policy_version 1033991 (0.0009) [2023-12-26 22:54:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.4, 300 sec: 19577.5). Total num frames: 529596416. Throughput: 0: 9705.0, 1: 9979.1. Samples: 529605948. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:54:11,062][104569] Avg episode reward: [(0, '8167.736'), (1, '9187.773')] [2023-12-26 22:54:11,140][105620] Updated weights for policy 1, policy_version 1034476 (0.0009) [2023-12-26 22:54:11,200][105620] Updated weights for policy 1, policy_version 1034486 (0.0008) [2023-12-26 22:54:11,263][105620] Updated weights for policy 1, policy_version 1034496 (0.0009) [2023-12-26 22:54:11,573][105692] Updated weights for policy 0, policy_version 1034001 (0.0009) [2023-12-26 22:54:11,631][105692] Updated weights for policy 0, policy_version 1034011 (0.0007) [2023-12-26 22:54:11,698][105692] Updated weights for policy 0, policy_version 1034021 (0.0010) [2023-12-26 22:54:11,987][105620] Updated weights for policy 1, policy_version 1034506 (0.0008) [2023-12-26 22:54:12,055][105620] Updated weights for policy 1, policy_version 1034516 (0.0008) [2023-12-26 22:54:12,113][105620] Updated weights for policy 1, policy_version 1034526 (0.0009) [2023-12-26 22:54:12,173][105620] Updated weights for policy 1, policy_version 1034536 (0.0006) [2023-12-26 22:54:12,490][105692] Updated weights for policy 0, policy_version 1034031 (0.0010) [2023-12-26 22:54:12,538][105692] Updated weights for policy 0, policy_version 1034041 (0.0009) [2023-12-26 22:54:12,591][105692] Updated weights for policy 0, policy_version 1034051 (0.0010) [2023-12-26 22:54:12,812][105620] Updated weights for policy 1, policy_version 1034546 (0.0005) [2023-12-26 22:54:12,872][105620] Updated weights for policy 1, policy_version 1034556 (0.0005) [2023-12-26 22:54:12,936][105620] Updated weights for policy 1, policy_version 1034566 (0.0008) [2023-12-26 22:54:13,410][105692] Updated weights for policy 0, policy_version 1034061 (0.0009) [2023-12-26 22:54:13,468][105692] Updated weights for policy 0, policy_version 1034071 (0.0009) [2023-12-26 22:54:13,522][105692] Updated weights for policy 0, policy_version 1034081 (0.0008) [2023-12-26 22:54:13,597][105620] Updated weights for policy 1, policy_version 1034576 (0.0006) [2023-12-26 22:54:13,655][105620] Updated weights for policy 1, policy_version 1034586 (0.0009) [2023-12-26 22:54:13,702][105620] Updated weights for policy 1, policy_version 1034596 (0.0008) [2023-12-26 22:54:14,290][105692] Updated weights for policy 0, policy_version 1034091 (0.0009) [2023-12-26 22:54:14,351][105692] Updated weights for policy 0, policy_version 1034101 (0.0009) [2023-12-26 22:54:14,408][105692] Updated weights for policy 0, policy_version 1034111 (0.0009) [2023-12-26 22:54:14,457][105620] Updated weights for policy 1, policy_version 1034606 (0.0008) [2023-12-26 22:54:14,520][105620] Updated weights for policy 1, policy_version 1034616 (0.0008) [2023-12-26 22:54:14,578][105620] Updated weights for policy 1, policy_version 1034626 (0.0009) [2023-12-26 22:54:15,182][105692] Updated weights for policy 0, policy_version 1034121 (0.0008) [2023-12-26 22:54:15,192][105620] Updated weights for policy 1, policy_version 1034636 (0.0009) [2023-12-26 22:54:15,243][105692] Updated weights for policy 0, policy_version 1034131 (0.0008) [2023-12-26 22:54:15,254][105620] Updated weights for policy 1, policy_version 1034646 (0.0006) [2023-12-26 22:54:15,303][105692] Updated weights for policy 0, policy_version 1034141 (0.0009) [2023-12-26 22:54:15,319][105620] Updated weights for policy 1, policy_version 1034656 (0.0006) [2023-12-26 22:54:15,357][105692] Updated weights for policy 0, policy_version 1034151 (0.0008) [2023-12-26 22:54:16,014][105620] Updated weights for policy 1, policy_version 1034666 (0.0009) [2023-12-26 22:54:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 529686528. Throughput: 0: 9607.8, 1: 9999.5. Samples: 529661280. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:54:16,063][104569] Avg episode reward: [(0, '8329.836'), (1, '8204.624')] [2023-12-26 22:54:16,065][105620] Updated weights for policy 1, policy_version 1034676 (0.0009) [2023-12-26 22:54:16,115][105620] Updated weights for policy 1, policy_version 1034686 (0.0007) [2023-12-26 22:54:16,115][105692] Updated weights for policy 0, policy_version 1034161 (0.0009) [2023-12-26 22:54:16,157][105620] Updated weights for policy 1, policy_version 1034696 (0.0008) [2023-12-26 22:54:16,157][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001034696_264912896.pth... [2023-12-26 22:54:16,161][105692] Updated weights for policy 0, policy_version 1034171 (0.0009) [2023-12-26 22:54:16,175][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001033512_264609792.pth [2023-12-26 22:54:16,208][105692] Updated weights for policy 0, policy_version 1034181 (0.0007) [2023-12-26 22:54:16,221][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001034184_264790016.pth... [2023-12-26 22:54:16,225][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001033064_264503296.pth [2023-12-26 22:54:16,818][105692] Updated weights for policy 0, policy_version 1034191 (0.0005) [2023-12-26 22:54:16,877][105692] Updated weights for policy 0, policy_version 1034201 (0.0005) [2023-12-26 22:54:16,928][105692] Updated weights for policy 0, policy_version 1034211 (0.0007) [2023-12-26 22:54:16,934][105620] Updated weights for policy 1, policy_version 1034706 (0.0007) [2023-12-26 22:54:16,989][105620] Updated weights for policy 1, policy_version 1034716 (0.0007) [2023-12-26 22:54:17,040][105620] Updated weights for policy 1, policy_version 1034726 (0.0009) [2023-12-26 22:54:17,486][105692] Updated weights for policy 0, policy_version 1034221 (0.0009) [2023-12-26 22:54:17,536][105692] Updated weights for policy 0, policy_version 1034231 (0.0009) [2023-12-26 22:54:17,583][105692] Updated weights for policy 0, policy_version 1034241 (0.0005) [2023-12-26 22:54:17,870][105620] Updated weights for policy 1, policy_version 1034736 (0.0009) [2023-12-26 22:54:17,939][105620] Updated weights for policy 1, policy_version 1034746 (0.0010) [2023-12-26 22:54:18,008][105620] Updated weights for policy 1, policy_version 1034756 (0.0008) [2023-12-26 22:54:18,289][105692] Updated weights for policy 0, policy_version 1034251 (0.0005) [2023-12-26 22:54:18,353][105692] Updated weights for policy 0, policy_version 1034261 (0.0007) [2023-12-26 22:54:18,414][105692] Updated weights for policy 0, policy_version 1034271 (0.0008) [2023-12-26 22:54:18,771][105620] Updated weights for policy 1, policy_version 1034766 (0.0010) [2023-12-26 22:54:18,833][105620] Updated weights for policy 1, policy_version 1034776 (0.0011) [2023-12-26 22:54:18,900][105620] Updated weights for policy 1, policy_version 1034786 (0.0011) [2023-12-26 22:54:19,176][105692] Updated weights for policy 0, policy_version 1034281 (0.0008) [2023-12-26 22:54:19,234][105692] Updated weights for policy 0, policy_version 1034291 (0.0008) [2023-12-26 22:54:19,301][105692] Updated weights for policy 0, policy_version 1034301 (0.0008) [2023-12-26 22:54:19,365][105692] Updated weights for policy 0, policy_version 1034311 (0.0009) [2023-12-26 22:54:19,660][105620] Updated weights for policy 1, policy_version 1034796 (0.0011) [2023-12-26 22:54:19,716][105620] Updated weights for policy 1, policy_version 1034806 (0.0010) [2023-12-26 22:54:19,769][105620] Updated weights for policy 1, policy_version 1034816 (0.0010) [2023-12-26 22:54:20,141][105692] Updated weights for policy 0, policy_version 1034321 (0.0007) [2023-12-26 22:54:20,206][105692] Updated weights for policy 0, policy_version 1034331 (0.0009) [2023-12-26 22:54:20,269][105692] Updated weights for policy 0, policy_version 1034341 (0.0008) [2023-12-26 22:54:20,502][105620] Updated weights for policy 1, policy_version 1034826 (0.0010) [2023-12-26 22:54:20,576][105620] Updated weights for policy 1, policy_version 1034836 (0.0008) [2023-12-26 22:54:20,637][105620] Updated weights for policy 1, policy_version 1034846 (0.0008) [2023-12-26 22:54:20,709][105620] Updated weights for policy 1, policy_version 1034856 (0.0007) [2023-12-26 22:54:20,964][105692] Updated weights for policy 0, policy_version 1034351 (0.0008) [2023-12-26 22:54:21,021][105692] Updated weights for policy 0, policy_version 1034361 (0.0009) [2023-12-26 22:54:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19549.8). Total num frames: 529784832. Throughput: 0: 9511.6, 1: 9867.2. Samples: 529776584. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:54:21,063][104569] Avg episode reward: [(0, '8996.709'), (1, '7869.322')] [2023-12-26 22:54:21,087][105692] Updated weights for policy 0, policy_version 1034371 (0.0008) [2023-12-26 22:54:21,474][105620] Updated weights for policy 1, policy_version 1034866 (0.0006) [2023-12-26 22:54:21,538][105620] Updated weights for policy 1, policy_version 1034876 (0.0009) [2023-12-26 22:54:21,603][105620] Updated weights for policy 1, policy_version 1034886 (0.0009) [2023-12-26 22:54:21,880][105692] Updated weights for policy 0, policy_version 1034381 (0.0008) [2023-12-26 22:54:21,937][105692] Updated weights for policy 0, policy_version 1034391 (0.0009) [2023-12-26 22:54:21,994][105692] Updated weights for policy 0, policy_version 1034401 (0.0009) [2023-12-26 22:54:22,351][105620] Updated weights for policy 1, policy_version 1034896 (0.0009) [2023-12-26 22:54:22,420][105620] Updated weights for policy 1, policy_version 1034906 (0.0010) [2023-12-26 22:54:22,448][105586] KL-divergence is very high: 138.9847 [2023-12-26 22:54:22,489][105620] Updated weights for policy 1, policy_version 1034916 (0.0009) [2023-12-26 22:54:22,501][105586] KL-divergence is very high: 251.7414 [2023-12-26 22:54:22,820][105692] Updated weights for policy 0, policy_version 1034411 (0.0010) [2023-12-26 22:54:22,879][105692] Updated weights for policy 0, policy_version 1034421 (0.0008) [2023-12-26 22:54:22,935][105692] Updated weights for policy 0, policy_version 1034431 (0.0007) [2023-12-26 22:54:23,150][105620] Updated weights for policy 1, policy_version 1034926 (0.0007) [2023-12-26 22:54:23,215][105620] Updated weights for policy 1, policy_version 1034936 (0.0008) [2023-12-26 22:54:23,285][105620] Updated weights for policy 1, policy_version 1034946 (0.0008) [2023-12-26 22:54:23,730][105692] Updated weights for policy 0, policy_version 1034441 (0.0007) [2023-12-26 22:54:23,782][105692] Updated weights for policy 0, policy_version 1034451 (0.0008) [2023-12-26 22:54:23,828][105692] Updated weights for policy 0, policy_version 1034461 (0.0008) [2023-12-26 22:54:23,873][105692] Updated weights for policy 0, policy_version 1034471 (0.0008) [2023-12-26 22:54:23,971][105620] Updated weights for policy 1, policy_version 1034956 (0.0009) [2023-12-26 22:54:24,018][105620] Updated weights for policy 1, policy_version 1034966 (0.0010) [2023-12-26 22:54:24,076][105620] Updated weights for policy 1, policy_version 1034976 (0.0010) [2023-12-26 22:54:24,576][105692] Updated weights for policy 0, policy_version 1034481 (0.0006) [2023-12-26 22:54:24,631][105692] Updated weights for policy 0, policy_version 1034491 (0.0005) [2023-12-26 22:54:24,688][105692] Updated weights for policy 0, policy_version 1034501 (0.0007) [2023-12-26 22:54:24,765][105620] Updated weights for policy 1, policy_version 1034986 (0.0010) [2023-12-26 22:54:24,819][105620] Updated weights for policy 1, policy_version 1034996 (0.0010) [2023-12-26 22:54:24,870][105620] Updated weights for policy 1, policy_version 1035006 (0.0010) [2023-12-26 22:54:24,917][105620] Updated weights for policy 1, policy_version 1035016 (0.0010) [2023-12-26 22:54:25,408][105692] Updated weights for policy 0, policy_version 1034511 (0.0009) [2023-12-26 22:54:25,454][105692] Updated weights for policy 0, policy_version 1034521 (0.0008) [2023-12-26 22:54:25,503][105692] Updated weights for policy 0, policy_version 1034531 (0.0008) [2023-12-26 22:54:25,649][105620] Updated weights for policy 1, policy_version 1035026 (0.0010) [2023-12-26 22:54:25,697][105620] Updated weights for policy 1, policy_version 1035036 (0.0010) [2023-12-26 22:54:25,744][105620] Updated weights for policy 1, policy_version 1035046 (0.0010) [2023-12-26 22:54:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 529883136. Throughput: 0: 9438.6, 1: 9888.2. Samples: 529890724. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:54:26,063][104569] Avg episode reward: [(0, '9175.475'), (1, '8679.695')] [2023-12-26 22:54:26,194][105692] Updated weights for policy 0, policy_version 1034541 (0.0008) [2023-12-26 22:54:26,244][105692] Updated weights for policy 0, policy_version 1034551 (0.0008) [2023-12-26 22:54:26,298][105692] Updated weights for policy 0, policy_version 1034561 (0.0007) [2023-12-26 22:54:26,514][105620] Updated weights for policy 1, policy_version 1035056 (0.0010) [2023-12-26 22:54:26,568][105620] Updated weights for policy 1, policy_version 1035066 (0.0010) [2023-12-26 22:54:26,625][105620] Updated weights for policy 1, policy_version 1035076 (0.0010) [2023-12-26 22:54:27,062][105692] Updated weights for policy 0, policy_version 1034571 (0.0008) [2023-12-26 22:54:27,117][105692] Updated weights for policy 0, policy_version 1034581 (0.0009) [2023-12-26 22:54:27,179][105692] Updated weights for policy 0, policy_version 1034591 (0.0008) [2023-12-26 22:54:27,368][105620] Updated weights for policy 1, policy_version 1035086 (0.0010) [2023-12-26 22:54:27,430][105620] Updated weights for policy 1, policy_version 1035096 (0.0010) [2023-12-26 22:54:27,490][105620] Updated weights for policy 1, policy_version 1035106 (0.0010) [2023-12-26 22:54:27,922][105692] Updated weights for policy 0, policy_version 1034601 (0.0009) [2023-12-26 22:54:27,971][105692] Updated weights for policy 0, policy_version 1034611 (0.0008) [2023-12-26 22:54:28,027][105692] Updated weights for policy 0, policy_version 1034621 (0.0008) [2023-12-26 22:54:28,083][105692] Updated weights for policy 0, policy_version 1034631 (0.0008) [2023-12-26 22:54:28,226][105620] Updated weights for policy 1, policy_version 1035116 (0.0008) [2023-12-26 22:54:28,276][105620] Updated weights for policy 1, policy_version 1035126 (0.0009) [2023-12-26 22:54:28,324][105620] Updated weights for policy 1, policy_version 1035136 (0.0010) [2023-12-26 22:54:28,964][105620] Updated weights for policy 1, policy_version 1035146 (0.0007) [2023-12-26 22:54:28,969][105692] Updated weights for policy 0, policy_version 1034641 (0.0009) [2023-12-26 22:54:29,023][105692] Updated weights for policy 0, policy_version 1034651 (0.0006) [2023-12-26 22:54:29,023][105620] Updated weights for policy 1, policy_version 1035156 (0.0010) [2023-12-26 22:54:29,079][105692] Updated weights for policy 0, policy_version 1034661 (0.0008) [2023-12-26 22:54:29,083][105620] Updated weights for policy 1, policy_version 1035166 (0.0010) [2023-12-26 22:54:29,140][105620] Updated weights for policy 1, policy_version 1035176 (0.0010) [2023-12-26 22:54:29,804][105620] Updated weights for policy 1, policy_version 1035186 (0.0008) [2023-12-26 22:54:29,870][105620] Updated weights for policy 1, policy_version 1035196 (0.0010) [2023-12-26 22:54:29,926][105692] Updated weights for policy 0, policy_version 1034671 (0.0008) [2023-12-26 22:54:29,939][105620] Updated weights for policy 1, policy_version 1035206 (0.0010) [2023-12-26 22:54:29,993][105692] Updated weights for policy 0, policy_version 1034681 (0.0008) [2023-12-26 22:54:30,061][105692] Updated weights for policy 0, policy_version 1034691 (0.0009) [2023-12-26 22:54:30,499][105620] Updated weights for policy 1, policy_version 1035216 (0.0010) [2023-12-26 22:54:30,557][105620] Updated weights for policy 1, policy_version 1035226 (0.0006) [2023-12-26 22:54:30,608][105620] Updated weights for policy 1, policy_version 1035236 (0.0009) [2023-12-26 22:54:30,893][105692] Updated weights for policy 0, policy_version 1034701 (0.0009) [2023-12-26 22:54:30,947][105692] Updated weights for policy 0, policy_version 1034711 (0.0009) [2023-12-26 22:54:30,994][105692] Updated weights for policy 0, policy_version 1034721 (0.0009) [2023-12-26 22:54:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 529981440. Throughput: 0: 9450.9, 1: 9932.4. Samples: 529948864. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:54:31,062][104569] Avg episode reward: [(0, '9085.396'), (1, '8667.298')] [2023-12-26 22:54:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001034728_264929280.pth... [2023-12-26 22:54:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001035240_265052160.pth... [2023-12-26 22:54:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001034088_264757248.pth [2023-12-26 22:54:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001033640_264650752.pth [2023-12-26 22:54:31,309][105620] Updated weights for policy 1, policy_version 1035246 (0.0007) [2023-12-26 22:54:31,371][105620] Updated weights for policy 1, policy_version 1035256 (0.0007) [2023-12-26 22:54:31,431][105620] Updated weights for policy 1, policy_version 1035266 (0.0009) [2023-12-26 22:54:31,849][105692] Updated weights for policy 0, policy_version 1034731 (0.0009) [2023-12-26 22:54:31,905][105692] Updated weights for policy 0, policy_version 1034741 (0.0009) [2023-12-26 22:54:31,970][105692] Updated weights for policy 0, policy_version 1034751 (0.0008) [2023-12-26 22:54:32,136][105620] Updated weights for policy 1, policy_version 1035276 (0.0006) [2023-12-26 22:54:32,190][105620] Updated weights for policy 1, policy_version 1035286 (0.0005) [2023-12-26 22:54:32,242][105620] Updated weights for policy 1, policy_version 1035296 (0.0006) [2023-12-26 22:54:32,827][105692] Updated weights for policy 0, policy_version 1034761 (0.0009) [2023-12-26 22:54:32,867][105620] Updated weights for policy 1, policy_version 1035306 (0.0009) [2023-12-26 22:54:32,884][105692] Updated weights for policy 0, policy_version 1034771 (0.0009) [2023-12-26 22:54:32,929][105620] Updated weights for policy 1, policy_version 1035316 (0.0009) [2023-12-26 22:54:32,942][105692] Updated weights for policy 0, policy_version 1034781 (0.0008) [2023-12-26 22:54:32,987][105620] Updated weights for policy 1, policy_version 1035326 (0.0009) [2023-12-26 22:54:32,994][105692] Updated weights for policy 0, policy_version 1034791 (0.0006) [2023-12-26 22:54:33,046][105620] Updated weights for policy 1, policy_version 1035336 (0.0008) [2023-12-26 22:54:33,636][105620] Updated weights for policy 1, policy_version 1035346 (0.0009) [2023-12-26 22:54:33,694][105620] Updated weights for policy 1, policy_version 1035356 (0.0009) [2023-12-26 22:54:33,744][105620] Updated weights for policy 1, policy_version 1035366 (0.0009) [2023-12-26 22:54:33,837][105692] Updated weights for policy 0, policy_version 1034801 (0.0008) [2023-12-26 22:54:33,884][105692] Updated weights for policy 0, policy_version 1034811 (0.0009) [2023-12-26 22:54:33,934][105692] Updated weights for policy 0, policy_version 1034821 (0.0009) [2023-12-26 22:54:34,409][105620] Updated weights for policy 1, policy_version 1035376 (0.0010) [2023-12-26 22:54:34,462][105620] Updated weights for policy 1, policy_version 1035386 (0.0009) [2023-12-26 22:54:34,525][105620] Updated weights for policy 1, policy_version 1035396 (0.0008) [2023-12-26 22:54:34,697][105692] Updated weights for policy 0, policy_version 1034831 (0.0007) [2023-12-26 22:54:34,761][105692] Updated weights for policy 0, policy_version 1034841 (0.0006) [2023-12-26 22:54:34,824][105692] Updated weights for policy 0, policy_version 1034851 (0.0007) [2023-12-26 22:54:35,169][105620] Updated weights for policy 1, policy_version 1035406 (0.0006) [2023-12-26 22:54:35,216][105620] Updated weights for policy 1, policy_version 1035416 (0.0005) [2023-12-26 22:54:35,264][105620] Updated weights for policy 1, policy_version 1035426 (0.0005) [2023-12-26 22:54:35,376][105692] Updated weights for policy 0, policy_version 1034861 (0.0008) [2023-12-26 22:54:35,428][105692] Updated weights for policy 0, policy_version 1034871 (0.0005) [2023-12-26 22:54:35,478][105692] Updated weights for policy 0, policy_version 1034881 (0.0005) [2023-12-26 22:54:35,803][105620] Updated weights for policy 1, policy_version 1035436 (0.0005) [2023-12-26 22:54:35,850][105620] Updated weights for policy 1, policy_version 1035446 (0.0005) [2023-12-26 22:54:35,900][105620] Updated weights for policy 1, policy_version 1035456 (0.0005) [2023-12-26 22:54:36,003][105692] Updated weights for policy 0, policy_version 1034891 (0.0007) [2023-12-26 22:54:36,054][105692] Updated weights for policy 0, policy_version 1034901 (0.0009) [2023-12-26 22:54:36,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 530079744. Throughput: 0: 9302.9, 1: 9992.4. Samples: 530063292. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:54:36,062][104569] Avg episode reward: [(0, '8995.388'), (1, '8901.322')] [2023-12-26 22:54:36,099][105692] Updated weights for policy 0, policy_version 1034911 (0.0008) [2023-12-26 22:54:36,592][105620] Updated weights for policy 1, policy_version 1035466 (0.0007) [2023-12-26 22:54:36,646][105620] Updated weights for policy 1, policy_version 1035476 (0.0011) [2023-12-26 22:54:36,702][105620] Updated weights for policy 1, policy_version 1035486 (0.0011) [2023-12-26 22:54:36,758][105620] Updated weights for policy 1, policy_version 1035496 (0.0008) [2023-12-26 22:54:36,759][105692] Updated weights for policy 0, policy_version 1034921 (0.0008) [2023-12-26 22:54:36,808][105692] Updated weights for policy 0, policy_version 1034931 (0.0009) [2023-12-26 22:54:36,860][105692] Updated weights for policy 0, policy_version 1034941 (0.0008) [2023-12-26 22:54:36,905][105692] Updated weights for policy 0, policy_version 1034951 (0.0005) [2023-12-26 22:54:37,456][105620] Updated weights for policy 1, policy_version 1035506 (0.0006) [2023-12-26 22:54:37,483][105692] Updated weights for policy 0, policy_version 1034961 (0.0009) [2023-12-26 22:54:37,505][105620] Updated weights for policy 1, policy_version 1035516 (0.0005) [2023-12-26 22:54:37,547][105692] Updated weights for policy 0, policy_version 1034971 (0.0008) [2023-12-26 22:54:37,566][105620] Updated weights for policy 1, policy_version 1035526 (0.0010) [2023-12-26 22:54:37,610][105692] Updated weights for policy 0, policy_version 1034981 (0.0010) [2023-12-26 22:54:38,274][105620] Updated weights for policy 1, policy_version 1035536 (0.0011) [2023-12-26 22:54:38,284][105692] Updated weights for policy 0, policy_version 1034991 (0.0011) [2023-12-26 22:54:38,325][105620] Updated weights for policy 1, policy_version 1035546 (0.0010) [2023-12-26 22:54:38,346][105692] Updated weights for policy 0, policy_version 1035001 (0.0010) [2023-12-26 22:54:38,395][105620] Updated weights for policy 1, policy_version 1035556 (0.0009) [2023-12-26 22:54:38,406][105692] Updated weights for policy 0, policy_version 1035011 (0.0011) [2023-12-26 22:54:39,117][105620] Updated weights for policy 1, policy_version 1035566 (0.0007) [2023-12-26 22:54:39,137][105692] Updated weights for policy 0, policy_version 1035021 (0.0009) [2023-12-26 22:54:39,186][105620] Updated weights for policy 1, policy_version 1035576 (0.0006) [2023-12-26 22:54:39,209][105692] Updated weights for policy 0, policy_version 1035031 (0.0007) [2023-12-26 22:54:39,258][105620] Updated weights for policy 1, policy_version 1035586 (0.0007) [2023-12-26 22:54:39,278][105692] Updated weights for policy 0, policy_version 1035041 (0.0009) [2023-12-26 22:54:40,020][105620] Updated weights for policy 1, policy_version 1035596 (0.0007) [2023-12-26 22:54:40,035][105692] Updated weights for policy 0, policy_version 1035051 (0.0010) [2023-12-26 22:54:40,087][105620] Updated weights for policy 1, policy_version 1035606 (0.0008) [2023-12-26 22:54:40,097][105692] Updated weights for policy 0, policy_version 1035061 (0.0007) [2023-12-26 22:54:40,151][105620] Updated weights for policy 1, policy_version 1035616 (0.0008) [2023-12-26 22:54:40,157][105692] Updated weights for policy 0, policy_version 1035071 (0.0007) [2023-12-26 22:54:40,772][105692] Updated weights for policy 0, policy_version 1035081 (0.0007) [2023-12-26 22:54:40,826][105692] Updated weights for policy 0, policy_version 1035091 (0.0005) [2023-12-26 22:54:40,884][105692] Updated weights for policy 0, policy_version 1035101 (0.0007) [2023-12-26 22:54:40,936][105692] Updated weights for policy 0, policy_version 1035111 (0.0010) [2023-12-26 22:54:40,970][105620] Updated weights for policy 1, policy_version 1035626 (0.0009) [2023-12-26 22:54:41,026][105620] Updated weights for policy 1, policy_version 1035636 (0.0010) [2023-12-26 22:54:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 530178048. Throughput: 0: 9405.2, 1: 10004.5. Samples: 530187492. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:54:41,062][104569] Avg episode reward: [(0, '8995.449'), (1, '8274.118')] [2023-12-26 22:54:41,088][105620] Updated weights for policy 1, policy_version 1035646 (0.0008) [2023-12-26 22:54:41,156][105620] Updated weights for policy 1, policy_version 1035656 (0.0008) [2023-12-26 22:54:41,688][105692] Updated weights for policy 0, policy_version 1035121 (0.0008) [2023-12-26 22:54:41,761][105692] Updated weights for policy 0, policy_version 1035131 (0.0008) [2023-12-26 22:54:41,820][105692] Updated weights for policy 0, policy_version 1035141 (0.0008) [2023-12-26 22:54:41,963][105620] Updated weights for policy 1, policy_version 1035666 (0.0009) [2023-12-26 22:54:42,012][105620] Updated weights for policy 1, policy_version 1035676 (0.0010) [2023-12-26 22:54:42,075][105620] Updated weights for policy 1, policy_version 1035686 (0.0010) [2023-12-26 22:54:42,511][105692] Updated weights for policy 0, policy_version 1035151 (0.0009) [2023-12-26 22:54:42,573][105692] Updated weights for policy 0, policy_version 1035161 (0.0007) [2023-12-26 22:54:42,636][105692] Updated weights for policy 0, policy_version 1035171 (0.0007) [2023-12-26 22:54:42,922][105620] Updated weights for policy 1, policy_version 1035696 (0.0008) [2023-12-26 22:54:42,994][105620] Updated weights for policy 1, policy_version 1035706 (0.0008) [2023-12-26 22:54:43,061][105620] Updated weights for policy 1, policy_version 1035716 (0.0008) [2023-12-26 22:54:43,432][105692] Updated weights for policy 0, policy_version 1035181 (0.0009) [2023-12-26 22:54:43,484][105692] Updated weights for policy 0, policy_version 1035191 (0.0009) [2023-12-26 22:54:43,538][105692] Updated weights for policy 0, policy_version 1035201 (0.0009) [2023-12-26 22:54:43,609][105620] Updated weights for policy 1, policy_version 1035726 (0.0005) [2023-12-26 22:54:43,670][105620] Updated weights for policy 1, policy_version 1035736 (0.0005) [2023-12-26 22:54:43,724][105620] Updated weights for policy 1, policy_version 1035746 (0.0005) [2023-12-26 22:54:44,280][105620] Updated weights for policy 1, policy_version 1035756 (0.0007) [2023-12-26 22:54:44,332][105620] Updated weights for policy 1, policy_version 1035766 (0.0008) [2023-12-26 22:54:44,380][105620] Updated weights for policy 1, policy_version 1035776 (0.0006) [2023-12-26 22:54:44,405][105692] Updated weights for policy 0, policy_version 1035211 (0.0009) [2023-12-26 22:54:44,455][105692] Updated weights for policy 0, policy_version 1035221 (0.0007) [2023-12-26 22:54:44,504][105692] Updated weights for policy 0, policy_version 1035231 (0.0005) [2023-12-26 22:54:45,094][105620] Updated weights for policy 1, policy_version 1035786 (0.0006) [2023-12-26 22:54:45,149][105620] Updated weights for policy 1, policy_version 1035796 (0.0010) [2023-12-26 22:54:45,198][105620] Updated weights for policy 1, policy_version 1035806 (0.0010) [2023-12-26 22:54:45,237][105692] Updated weights for policy 0, policy_version 1035241 (0.0005) [2023-12-26 22:54:45,255][105620] Updated weights for policy 1, policy_version 1035816 (0.0011) [2023-12-26 22:54:45,288][105692] Updated weights for policy 0, policy_version 1035251 (0.0007) [2023-12-26 22:54:45,349][105692] Updated weights for policy 0, policy_version 1035261 (0.0008) [2023-12-26 22:54:45,405][105692] Updated weights for policy 0, policy_version 1035271 (0.0008) [2023-12-26 22:54:45,997][105620] Updated weights for policy 1, policy_version 1035826 (0.0005) [2023-12-26 22:54:46,043][105620] Updated weights for policy 1, policy_version 1035836 (0.0005) [2023-12-26 22:54:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 530268160. Throughput: 0: 9399.1, 1: 9935.9. Samples: 530244676. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:54:46,063][104569] Avg episode reward: [(0, '9086.845'), (1, '7712.213')] [2023-12-26 22:54:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001035272_265068544.pth... [2023-12-26 22:54:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001034184_264790016.pth [2023-12-26 22:54:46,099][105620] Updated weights for policy 1, policy_version 1035846 (0.0005) [2023-12-26 22:54:46,108][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001035848_265207808.pth... [2023-12-26 22:54:46,111][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001034696_264912896.pth [2023-12-26 22:54:46,220][105692] Updated weights for policy 0, policy_version 1035281 (0.0009) [2023-12-26 22:54:46,274][105692] Updated weights for policy 0, policy_version 1035292 (0.0009) [2023-12-26 22:54:46,322][105692] Updated weights for policy 0, policy_version 1035302 (0.0008) [2023-12-26 22:54:46,739][105620] Updated weights for policy 1, policy_version 1035856 (0.0009) [2023-12-26 22:54:46,797][105620] Updated weights for policy 1, policy_version 1035866 (0.0010) [2023-12-26 22:54:46,859][105620] Updated weights for policy 1, policy_version 1035876 (0.0010) [2023-12-26 22:54:47,029][105692] Updated weights for policy 0, policy_version 1035312 (0.0008) [2023-12-26 22:54:47,083][105692] Updated weights for policy 0, policy_version 1035322 (0.0008) [2023-12-26 22:54:47,128][105692] Updated weights for policy 0, policy_version 1035332 (0.0008) [2023-12-26 22:54:47,599][105620] Updated weights for policy 1, policy_version 1035886 (0.0010) [2023-12-26 22:54:47,657][105620] Updated weights for policy 1, policy_version 1035896 (0.0007) [2023-12-26 22:54:47,715][105620] Updated weights for policy 1, policy_version 1035906 (0.0006) [2023-12-26 22:54:47,919][105692] Updated weights for policy 0, policy_version 1035342 (0.0009) [2023-12-26 22:54:47,985][105692] Updated weights for policy 0, policy_version 1035352 (0.0009) [2023-12-26 22:54:48,052][105692] Updated weights for policy 0, policy_version 1035362 (0.0008) [2023-12-26 22:54:48,338][105620] Updated weights for policy 1, policy_version 1035916 (0.0008) [2023-12-26 22:54:48,397][105620] Updated weights for policy 1, policy_version 1035926 (0.0009) [2023-12-26 22:54:48,457][105620] Updated weights for policy 1, policy_version 1035936 (0.0006) [2023-12-26 22:54:48,901][105692] Updated weights for policy 0, policy_version 1035372 (0.0009) [2023-12-26 22:54:48,959][105692] Updated weights for policy 0, policy_version 1035382 (0.0008) [2023-12-26 22:54:49,019][105692] Updated weights for policy 0, policy_version 1035393 (0.0013) [2023-12-26 22:54:49,056][105620] Updated weights for policy 1, policy_version 1035946 (0.0006) [2023-12-26 22:54:49,124][105620] Updated weights for policy 1, policy_version 1035956 (0.0008) [2023-12-26 22:54:49,175][105620] Updated weights for policy 1, policy_version 1035966 (0.0009) [2023-12-26 22:54:49,234][105620] Updated weights for policy 1, policy_version 1035976 (0.0009) [2023-12-26 22:54:49,836][105692] Updated weights for policy 0, policy_version 1035403 (0.0009) [2023-12-26 22:54:49,891][105692] Updated weights for policy 0, policy_version 1035413 (0.0009) [2023-12-26 22:54:49,953][105692] Updated weights for policy 0, policy_version 1035423 (0.0008) [2023-12-26 22:54:49,976][105620] Updated weights for policy 1, policy_version 1035986 (0.0007) [2023-12-26 22:54:50,039][105620] Updated weights for policy 1, policy_version 1035996 (0.0006) [2023-12-26 22:54:50,100][105620] Updated weights for policy 1, policy_version 1036006 (0.0007) [2023-12-26 22:54:50,710][105692] Updated weights for policy 0, policy_version 1035433 (0.0008) [2023-12-26 22:54:50,774][105692] Updated weights for policy 0, policy_version 1035443 (0.0009) [2023-12-26 22:54:50,792][105620] Updated weights for policy 1, policy_version 1036016 (0.0007) [2023-12-26 22:54:50,835][105692] Updated weights for policy 0, policy_version 1035453 (0.0007) [2023-12-26 22:54:50,843][105620] Updated weights for policy 1, policy_version 1036026 (0.0007) [2023-12-26 22:54:50,891][105692] Updated weights for policy 0, policy_version 1035463 (0.0010) [2023-12-26 22:54:50,906][105620] Updated weights for policy 1, policy_version 1036036 (0.0009) [2023-12-26 22:54:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 530374656. Throughput: 0: 9369.2, 1: 9913.0. Samples: 530359812. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:54:51,062][104569] Avg episode reward: [(0, '9087.863'), (1, '8495.126')] [2023-12-26 22:54:51,627][105620] Updated weights for policy 1, policy_version 1036046 (0.0008) [2023-12-26 22:54:51,649][105692] Updated weights for policy 0, policy_version 1035473 (0.0008) [2023-12-26 22:54:51,684][105620] Updated weights for policy 1, policy_version 1036056 (0.0007) [2023-12-26 22:54:51,703][105692] Updated weights for policy 0, policy_version 1035483 (0.0006) [2023-12-26 22:54:51,751][105620] Updated weights for policy 1, policy_version 1036066 (0.0006) [2023-12-26 22:54:51,773][105692] Updated weights for policy 0, policy_version 1035493 (0.0008) [2023-12-26 22:54:52,478][105620] Updated weights for policy 1, policy_version 1036076 (0.0007) [2023-12-26 22:54:52,544][105620] Updated weights for policy 1, policy_version 1036086 (0.0006) [2023-12-26 22:54:52,608][105620] Updated weights for policy 1, policy_version 1036096 (0.0006) [2023-12-26 22:54:52,609][105692] Updated weights for policy 0, policy_version 1035503 (0.0008) [2023-12-26 22:54:52,665][105692] Updated weights for policy 0, policy_version 1035513 (0.0009) [2023-12-26 22:54:52,722][105692] Updated weights for policy 0, policy_version 1035523 (0.0009) [2023-12-26 22:54:53,187][105620] Updated weights for policy 1, policy_version 1036106 (0.0006) [2023-12-26 22:54:53,236][105620] Updated weights for policy 1, policy_version 1036116 (0.0005) [2023-12-26 22:54:53,290][105620] Updated weights for policy 1, policy_version 1036126 (0.0007) [2023-12-26 22:54:53,348][105620] Updated weights for policy 1, policy_version 1036136 (0.0009) [2023-12-26 22:54:53,548][105692] Updated weights for policy 0, policy_version 1035533 (0.0009) [2023-12-26 22:54:53,598][105692] Updated weights for policy 0, policy_version 1035543 (0.0009) [2023-12-26 22:54:53,644][105692] Updated weights for policy 0, policy_version 1035553 (0.0008) [2023-12-26 22:54:54,050][105620] Updated weights for policy 1, policy_version 1036146 (0.0009) [2023-12-26 22:54:54,108][105620] Updated weights for policy 1, policy_version 1036156 (0.0010) [2023-12-26 22:54:54,161][105620] Updated weights for policy 1, policy_version 1036166 (0.0009) [2023-12-26 22:54:54,329][105692] Updated weights for policy 0, policy_version 1035563 (0.0009) [2023-12-26 22:54:54,397][105692] Updated weights for policy 0, policy_version 1035573 (0.0007) [2023-12-26 22:54:54,455][105692] Updated weights for policy 0, policy_version 1035583 (0.0006) [2023-12-26 22:54:54,967][105620] Updated weights for policy 1, policy_version 1036176 (0.0009) [2023-12-26 22:54:55,023][105620] Updated weights for policy 1, policy_version 1036186 (0.0010) [2023-12-26 22:54:55,078][105620] Updated weights for policy 1, policy_version 1036196 (0.0009) [2023-12-26 22:54:55,134][105692] Updated weights for policy 0, policy_version 1035593 (0.0009) [2023-12-26 22:54:55,195][105692] Updated weights for policy 0, policy_version 1035603 (0.0009) [2023-12-26 22:54:55,249][105692] Updated weights for policy 0, policy_version 1035613 (0.0009) [2023-12-26 22:54:55,300][105692] Updated weights for policy 0, policy_version 1035623 (0.0009) [2023-12-26 22:54:55,846][105620] Updated weights for policy 1, policy_version 1036206 (0.0008) [2023-12-26 22:54:55,916][105620] Updated weights for policy 1, policy_version 1036216 (0.0005) [2023-12-26 22:54:55,974][105620] Updated weights for policy 1, policy_version 1036226 (0.0005) [2023-12-26 22:54:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 530464768. Throughput: 0: 9381.3, 1: 9897.5. Samples: 530473496. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:54:56,063][104569] Avg episode reward: [(0, '9087.740'), (1, '9085.164')] [2023-12-26 22:54:56,102][105692] Updated weights for policy 0, policy_version 1035633 (0.0009) [2023-12-26 22:54:56,165][105692] Updated weights for policy 0, policy_version 1035643 (0.0009) [2023-12-26 22:54:56,220][105692] Updated weights for policy 0, policy_version 1035653 (0.0009) [2023-12-26 22:54:56,584][105620] Updated weights for policy 1, policy_version 1036236 (0.0007) [2023-12-26 22:54:56,627][105620] Updated weights for policy 1, policy_version 1036246 (0.0007) [2023-12-26 22:54:56,674][105620] Updated weights for policy 1, policy_version 1036256 (0.0008) [2023-12-26 22:54:56,923][105692] Updated weights for policy 0, policy_version 1035663 (0.0010) [2023-12-26 22:54:56,987][105692] Updated weights for policy 0, policy_version 1035673 (0.0009) [2023-12-26 22:54:57,048][105692] Updated weights for policy 0, policy_version 1035683 (0.0008) [2023-12-26 22:54:57,415][105620] Updated weights for policy 1, policy_version 1036266 (0.0008) [2023-12-26 22:54:57,467][105620] Updated weights for policy 1, policy_version 1036276 (0.0008) [2023-12-26 22:54:57,519][105620] Updated weights for policy 1, policy_version 1036286 (0.0008) [2023-12-26 22:54:57,574][105620] Updated weights for policy 1, policy_version 1036296 (0.0009) [2023-12-26 22:54:57,704][105692] Updated weights for policy 0, policy_version 1035693 (0.0006) [2023-12-26 22:54:57,766][105692] Updated weights for policy 0, policy_version 1035703 (0.0006) [2023-12-26 22:54:57,832][105692] Updated weights for policy 0, policy_version 1035713 (0.0005) [2023-12-26 22:54:58,318][105620] Updated weights for policy 1, policy_version 1036306 (0.0008) [2023-12-26 22:54:58,381][105620] Updated weights for policy 1, policy_version 1036316 (0.0009) [2023-12-26 22:54:58,441][105620] Updated weights for policy 1, policy_version 1036326 (0.0008) [2023-12-26 22:54:58,525][105692] Updated weights for policy 0, policy_version 1035723 (0.0006) [2023-12-26 22:54:58,586][105692] Updated weights for policy 0, policy_version 1035733 (0.0009) [2023-12-26 22:54:58,647][105692] Updated weights for policy 0, policy_version 1035743 (0.0009) [2023-12-26 22:54:59,277][105620] Updated weights for policy 1, policy_version 1036336 (0.0010) [2023-12-26 22:54:59,342][105620] Updated weights for policy 1, policy_version 1036346 (0.0011) [2023-12-26 22:54:59,411][105620] Updated weights for policy 1, policy_version 1036356 (0.0011) [2023-12-26 22:54:59,496][105692] Updated weights for policy 0, policy_version 1035753 (0.0009) [2023-12-26 22:54:59,560][105692] Updated weights for policy 0, policy_version 1035763 (0.0010) [2023-12-26 22:54:59,626][105692] Updated weights for policy 0, policy_version 1035773 (0.0010) [2023-12-26 22:54:59,677][105692] Updated weights for policy 0, policy_version 1035783 (0.0010) [2023-12-26 22:55:00,120][105620] Updated weights for policy 1, policy_version 1036366 (0.0007) [2023-12-26 22:55:00,179][105620] Updated weights for policy 1, policy_version 1036376 (0.0005) [2023-12-26 22:55:00,244][105620] Updated weights for policy 1, policy_version 1036386 (0.0005) [2023-12-26 22:55:00,458][105692] Updated weights for policy 0, policy_version 1035793 (0.0010) [2023-12-26 22:55:00,514][105692] Updated weights for policy 0, policy_version 1035803 (0.0008) [2023-12-26 22:55:00,561][105692] Updated weights for policy 0, policy_version 1035813 (0.0010) [2023-12-26 22:55:00,826][105620] Updated weights for policy 1, policy_version 1036396 (0.0007) [2023-12-26 22:55:00,873][105620] Updated weights for policy 1, policy_version 1036406 (0.0010) [2023-12-26 22:55:00,927][105620] Updated weights for policy 1, policy_version 1036416 (0.0010) [2023-12-26 22:55:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 530563072. Throughput: 0: 9458.3, 1: 9889.7. Samples: 530531940. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:55:01,062][104569] Avg episode reward: [(0, '8910.855'), (1, '9264.725')] [2023-12-26 22:55:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001035816_265207808.pth... [2023-12-26 22:55:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001036424_265355264.pth... [2023-12-26 22:55:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001034728_264929280.pth [2023-12-26 22:55:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001035240_265052160.pth [2023-12-26 22:55:01,283][105692] Updated weights for policy 0, policy_version 1035823 (0.0009) [2023-12-26 22:55:01,349][105692] Updated weights for policy 0, policy_version 1035833 (0.0007) [2023-12-26 22:55:01,417][105692] Updated weights for policy 0, policy_version 1035843 (0.0008) [2023-12-26 22:55:01,636][105620] Updated weights for policy 1, policy_version 1036426 (0.0010) [2023-12-26 22:55:01,698][105620] Updated weights for policy 1, policy_version 1036436 (0.0009) [2023-12-26 22:55:01,768][105620] Updated weights for policy 1, policy_version 1036446 (0.0008) [2023-12-26 22:55:01,824][105620] Updated weights for policy 1, policy_version 1036456 (0.0008) [2023-12-26 22:55:02,200][105692] Updated weights for policy 0, policy_version 1035853 (0.0010) [2023-12-26 22:55:02,266][105692] Updated weights for policy 0, policy_version 1035863 (0.0010) [2023-12-26 22:55:02,326][105692] Updated weights for policy 0, policy_version 1035873 (0.0009) [2023-12-26 22:55:02,521][105620] Updated weights for policy 1, policy_version 1036466 (0.0005) [2023-12-26 22:55:02,580][105620] Updated weights for policy 1, policy_version 1036476 (0.0005) [2023-12-26 22:55:02,640][105620] Updated weights for policy 1, policy_version 1036486 (0.0009) [2023-12-26 22:55:03,067][105692] Updated weights for policy 0, policy_version 1035883 (0.0009) [2023-12-26 22:55:03,128][105692] Updated weights for policy 0, policy_version 1035893 (0.0010) [2023-12-26 22:55:03,190][105620] Updated weights for policy 1, policy_version 1036496 (0.0010) [2023-12-26 22:55:03,190][105692] Updated weights for policy 0, policy_version 1035903 (0.0010) [2023-12-26 22:55:03,235][105620] Updated weights for policy 1, policy_version 1036506 (0.0010) [2023-12-26 22:55:03,293][105620] Updated weights for policy 1, policy_version 1036516 (0.0010) [2023-12-26 22:55:03,902][105692] Updated weights for policy 0, policy_version 1035913 (0.0010) [2023-12-26 22:55:03,963][105692] Updated weights for policy 0, policy_version 1035923 (0.0008) [2023-12-26 22:55:04,030][105692] Updated weights for policy 0, policy_version 1035933 (0.0008) [2023-12-26 22:55:04,063][105620] Updated weights for policy 1, policy_version 1036526 (0.0011) [2023-12-26 22:55:04,094][105692] Updated weights for policy 0, policy_version 1035943 (0.0006) [2023-12-26 22:55:04,127][105620] Updated weights for policy 1, policy_version 1036536 (0.0009) [2023-12-26 22:55:04,182][105620] Updated weights for policy 1, policy_version 1036546 (0.0010) [2023-12-26 22:55:04,791][105692] Updated weights for policy 0, policy_version 1035953 (0.0008) [2023-12-26 22:55:04,834][105692] Updated weights for policy 0, policy_version 1035963 (0.0007) [2023-12-26 22:55:04,887][105692] Updated weights for policy 0, policy_version 1035973 (0.0009) [2023-12-26 22:55:04,921][105620] Updated weights for policy 1, policy_version 1036556 (0.0009) [2023-12-26 22:55:04,976][105620] Updated weights for policy 1, policy_version 1036566 (0.0011) [2023-12-26 22:55:05,035][105620] Updated weights for policy 1, policy_version 1036576 (0.0010) [2023-12-26 22:55:05,636][105692] Updated weights for policy 0, policy_version 1035983 (0.0008) [2023-12-26 22:55:05,701][105692] Updated weights for policy 0, policy_version 1035993 (0.0006) [2023-12-26 22:55:05,758][105692] Updated weights for policy 0, policy_version 1036003 (0.0008) [2023-12-26 22:55:05,766][105620] Updated weights for policy 1, policy_version 1036586 (0.0010) [2023-12-26 22:55:05,818][105620] Updated weights for policy 1, policy_version 1036596 (0.0010) [2023-12-26 22:55:05,880][105620] Updated weights for policy 1, policy_version 1036606 (0.0010) [2023-12-26 22:55:05,931][105620] Updated weights for policy 1, policy_version 1036616 (0.0010) [2023-12-26 22:55:06,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 530661376. Throughput: 0: 9356.5, 1: 9981.0. Samples: 530646768. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:55:06,062][104569] Avg episode reward: [(0, '8823.948'), (1, '8990.240')] [2023-12-26 22:55:06,457][105692] Updated weights for policy 0, policy_version 1036013 (0.0009) [2023-12-26 22:55:06,515][105692] Updated weights for policy 0, policy_version 1036023 (0.0011) [2023-12-26 22:55:06,574][105692] Updated weights for policy 0, policy_version 1036033 (0.0011) [2023-12-26 22:55:06,648][105620] Updated weights for policy 1, policy_version 1036626 (0.0011) [2023-12-26 22:55:06,711][105620] Updated weights for policy 1, policy_version 1036636 (0.0011) [2023-12-26 22:55:06,770][105620] Updated weights for policy 1, policy_version 1036646 (0.0011) [2023-12-26 22:55:07,339][105692] Updated weights for policy 0, policy_version 1036043 (0.0011) [2023-12-26 22:55:07,390][105692] Updated weights for policy 0, policy_version 1036053 (0.0010) [2023-12-26 22:55:07,456][105692] Updated weights for policy 0, policy_version 1036063 (0.0007) [2023-12-26 22:55:07,525][105620] Updated weights for policy 1, policy_version 1036656 (0.0011) [2023-12-26 22:55:07,574][105620] Updated weights for policy 1, policy_version 1036666 (0.0011) [2023-12-26 22:55:07,623][105620] Updated weights for policy 1, policy_version 1036676 (0.0010) [2023-12-26 22:55:08,068][105692] Updated weights for policy 0, policy_version 1036073 (0.0005) [2023-12-26 22:55:08,121][105692] Updated weights for policy 0, policy_version 1036083 (0.0007) [2023-12-26 22:55:08,176][105692] Updated weights for policy 0, policy_version 1036093 (0.0006) [2023-12-26 22:55:08,238][105692] Updated weights for policy 0, policy_version 1036103 (0.0010) [2023-12-26 22:55:08,399][105620] Updated weights for policy 1, policy_version 1036686 (0.0010) [2023-12-26 22:55:08,456][105620] Updated weights for policy 1, policy_version 1036696 (0.0011) [2023-12-26 22:55:08,514][105620] Updated weights for policy 1, policy_version 1036706 (0.0011) [2023-12-26 22:55:08,941][105692] Updated weights for policy 0, policy_version 1036113 (0.0011) [2023-12-26 22:55:08,993][105692] Updated weights for policy 0, policy_version 1036123 (0.0010) [2023-12-26 22:55:09,042][105692] Updated weights for policy 0, policy_version 1036133 (0.0011) [2023-12-26 22:55:09,120][105620] Updated weights for policy 1, policy_version 1036716 (0.0008) [2023-12-26 22:55:09,183][105620] Updated weights for policy 1, policy_version 1036726 (0.0007) [2023-12-26 22:55:09,252][105620] Updated weights for policy 1, policy_version 1036736 (0.0010) [2023-12-26 22:55:09,784][105692] Updated weights for policy 0, policy_version 1036143 (0.0008) [2023-12-26 22:55:09,853][105692] Updated weights for policy 0, policy_version 1036153 (0.0009) [2023-12-26 22:55:09,915][105692] Updated weights for policy 0, policy_version 1036163 (0.0007) [2023-12-26 22:55:10,013][105620] Updated weights for policy 1, policy_version 1036746 (0.0010) [2023-12-26 22:55:10,068][105620] Updated weights for policy 1, policy_version 1036756 (0.0010) [2023-12-26 22:55:10,121][105620] Updated weights for policy 1, policy_version 1036766 (0.0010) [2023-12-26 22:55:10,174][105620] Updated weights for policy 1, policy_version 1036776 (0.0010) [2023-12-26 22:55:10,535][105692] Updated weights for policy 0, policy_version 1036173 (0.0007) [2023-12-26 22:55:10,588][105692] Updated weights for policy 0, policy_version 1036183 (0.0005) [2023-12-26 22:55:10,641][105692] Updated weights for policy 0, policy_version 1036193 (0.0007) [2023-12-26 22:55:10,932][105620] Updated weights for policy 1, policy_version 1036786 (0.0009) [2023-12-26 22:55:10,990][105620] Updated weights for policy 1, policy_version 1036796 (0.0010) [2023-12-26 22:55:11,050][105620] Updated weights for policy 1, policy_version 1036806 (0.0008) [2023-12-26 22:55:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 530751488. Throughput: 0: 9457.9, 1: 9957.3. Samples: 530764404. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:55:11,063][104569] Avg episode reward: [(0, '9086.784'), (1, '8717.706')] [2023-12-26 22:55:11,351][105692] Updated weights for policy 0, policy_version 1036203 (0.0009) [2023-12-26 22:55:11,419][105692] Updated weights for policy 0, policy_version 1036213 (0.0009) [2023-12-26 22:55:11,483][105692] Updated weights for policy 0, policy_version 1036223 (0.0006) [2023-12-26 22:55:11,824][105620] Updated weights for policy 1, policy_version 1036816 (0.0010) [2023-12-26 22:55:11,880][105620] Updated weights for policy 1, policy_version 1036826 (0.0010) [2023-12-26 22:55:11,930][105620] Updated weights for policy 1, policy_version 1036836 (0.0008) [2023-12-26 22:55:12,275][105692] Updated weights for policy 0, policy_version 1036233 (0.0009) [2023-12-26 22:55:12,333][105692] Updated weights for policy 0, policy_version 1036243 (0.0008) [2023-12-26 22:55:12,398][105692] Updated weights for policy 0, policy_version 1036253 (0.0009) [2023-12-26 22:55:12,447][105692] Updated weights for policy 0, policy_version 1036263 (0.0009) [2023-12-26 22:55:12,734][105620] Updated weights for policy 1, policy_version 1036846 (0.0009) [2023-12-26 22:55:12,792][105620] Updated weights for policy 1, policy_version 1036856 (0.0008) [2023-12-26 22:55:12,843][105620] Updated weights for policy 1, policy_version 1036866 (0.0009) [2023-12-26 22:55:13,160][105692] Updated weights for policy 0, policy_version 1036273 (0.0009) [2023-12-26 22:55:13,211][105692] Updated weights for policy 0, policy_version 1036283 (0.0009) [2023-12-26 22:55:13,273][105692] Updated weights for policy 0, policy_version 1036293 (0.0009) [2023-12-26 22:55:13,588][105620] Updated weights for policy 1, policy_version 1036876 (0.0008) [2023-12-26 22:55:13,634][105620] Updated weights for policy 1, policy_version 1036886 (0.0005) [2023-12-26 22:55:13,681][105620] Updated weights for policy 1, policy_version 1036896 (0.0005) [2023-12-26 22:55:13,937][105692] Updated weights for policy 0, policy_version 1036303 (0.0009) [2023-12-26 22:55:13,997][105692] Updated weights for policy 0, policy_version 1036313 (0.0009) [2023-12-26 22:55:14,062][105692] Updated weights for policy 0, policy_version 1036323 (0.0009) [2023-12-26 22:55:14,234][105620] Updated weights for policy 1, policy_version 1036906 (0.0005) [2023-12-26 22:55:14,295][105620] Updated weights for policy 1, policy_version 1036916 (0.0005) [2023-12-26 22:55:14,360][105620] Updated weights for policy 1, policy_version 1036926 (0.0009) [2023-12-26 22:55:14,406][105620] Updated weights for policy 1, policy_version 1036936 (0.0008) [2023-12-26 22:55:14,800][105692] Updated weights for policy 0, policy_version 1036333 (0.0010) [2023-12-26 22:55:14,857][105692] Updated weights for policy 0, policy_version 1036343 (0.0009) [2023-12-26 22:55:14,917][105692] Updated weights for policy 0, policy_version 1036353 (0.0009) [2023-12-26 22:55:15,123][105620] Updated weights for policy 1, policy_version 1036946 (0.0009) [2023-12-26 22:55:15,181][105620] Updated weights for policy 1, policy_version 1036956 (0.0009) [2023-12-26 22:55:15,240][105620] Updated weights for policy 1, policy_version 1036966 (0.0009) [2023-12-26 22:55:15,719][105692] Updated weights for policy 0, policy_version 1036363 (0.0010) [2023-12-26 22:55:15,771][105692] Updated weights for policy 0, policy_version 1036373 (0.0009) [2023-12-26 22:55:15,824][105692] Updated weights for policy 0, policy_version 1036383 (0.0009) [2023-12-26 22:55:15,941][105620] Updated weights for policy 1, policy_version 1036976 (0.0009) [2023-12-26 22:55:15,993][105620] Updated weights for policy 1, policy_version 1036986 (0.0008) [2023-12-26 22:55:16,044][105620] Updated weights for policy 1, policy_version 1036996 (0.0009) [2023-12-26 22:55:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 530849792. Throughput: 0: 9456.5, 1: 9928.6. Samples: 530821196. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:55:16,063][104569] Avg episode reward: [(0, '9173.468'), (1, '8902.877')] [2023-12-26 22:55:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001037000_265502720.pth... [2023-12-26 22:55:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001036392_265355264.pth... [2023-12-26 22:55:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001035272_265068544.pth [2023-12-26 22:55:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001035848_265207808.pth [2023-12-26 22:55:16,606][105692] Updated weights for policy 0, policy_version 1036393 (0.0009) [2023-12-26 22:55:16,668][105692] Updated weights for policy 0, policy_version 1036403 (0.0009) [2023-12-26 22:55:16,727][105692] Updated weights for policy 0, policy_version 1036413 (0.0009) [2023-12-26 22:55:16,787][105692] Updated weights for policy 0, policy_version 1036423 (0.0008) [2023-12-26 22:55:16,801][105620] Updated weights for policy 1, policy_version 1037006 (0.0009) [2023-12-26 22:55:16,865][105620] Updated weights for policy 1, policy_version 1037016 (0.0008) [2023-12-26 22:55:16,917][105620] Updated weights for policy 1, policy_version 1037026 (0.0009) [2023-12-26 22:55:17,542][105692] Updated weights for policy 0, policy_version 1036433 (0.0009) [2023-12-26 22:55:17,589][105692] Updated weights for policy 0, policy_version 1036443 (0.0009) [2023-12-26 22:55:17,640][105692] Updated weights for policy 0, policy_version 1036453 (0.0009) [2023-12-26 22:55:17,671][105620] Updated weights for policy 1, policy_version 1037036 (0.0008) [2023-12-26 22:55:17,722][105620] Updated weights for policy 1, policy_version 1037046 (0.0008) [2023-12-26 22:55:17,772][105620] Updated weights for policy 1, policy_version 1037056 (0.0009) [2023-12-26 22:55:18,464][105692] Updated weights for policy 0, policy_version 1036463 (0.0010) [2023-12-26 22:55:18,501][105620] Updated weights for policy 1, policy_version 1037066 (0.0009) [2023-12-26 22:55:18,522][105692] Updated weights for policy 0, policy_version 1036473 (0.0010) [2023-12-26 22:55:18,556][105620] Updated weights for policy 1, policy_version 1037076 (0.0005) [2023-12-26 22:55:18,577][105692] Updated weights for policy 0, policy_version 1036483 (0.0010) [2023-12-26 22:55:18,602][105620] Updated weights for policy 1, policy_version 1037086 (0.0010) [2023-12-26 22:55:18,650][105620] Updated weights for policy 1, policy_version 1037096 (0.0008) [2023-12-26 22:55:19,308][105692] Updated weights for policy 0, policy_version 1036493 (0.0011) [2023-12-26 22:55:19,376][105692] Updated weights for policy 0, policy_version 1036503 (0.0010) [2023-12-26 22:55:19,433][105692] Updated weights for policy 0, policy_version 1036513 (0.0007) [2023-12-26 22:55:19,461][105620] Updated weights for policy 1, policy_version 1037106 (0.0007) [2023-12-26 22:55:19,529][105620] Updated weights for policy 1, policy_version 1037116 (0.0008) [2023-12-26 22:55:19,584][105620] Updated weights for policy 1, policy_version 1037126 (0.0005) [2023-12-26 22:55:20,167][105692] Updated weights for policy 0, policy_version 1036523 (0.0006) [2023-12-26 22:55:20,227][105692] Updated weights for policy 0, policy_version 1036533 (0.0009) [2023-12-26 22:55:20,291][105692] Updated weights for policy 0, policy_version 1036543 (0.0008) [2023-12-26 22:55:20,317][105620] Updated weights for policy 1, policy_version 1037136 (0.0010) [2023-12-26 22:55:20,377][105620] Updated weights for policy 1, policy_version 1037146 (0.0011) [2023-12-26 22:55:20,436][105620] Updated weights for policy 1, policy_version 1037156 (0.0011) [2023-12-26 22:55:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 530939904. Throughput: 0: 9553.3, 1: 9819.3. Samples: 530935060. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:55:21,063][104569] Avg episode reward: [(0, '9086.275'), (1, '9085.151')] [2023-12-26 22:55:21,067][105692] Updated weights for policy 0, policy_version 1036553 (0.0006) [2023-12-26 22:55:21,131][105692] Updated weights for policy 0, policy_version 1036563 (0.0011) [2023-12-26 22:55:21,198][105692] Updated weights for policy 0, policy_version 1036573 (0.0009) [2023-12-26 22:55:21,204][105620] Updated weights for policy 1, policy_version 1037166 (0.0009) [2023-12-26 22:55:21,263][105692] Updated weights for policy 0, policy_version 1036583 (0.0008) [2023-12-26 22:55:21,275][105620] Updated weights for policy 1, policy_version 1037176 (0.0007) [2023-12-26 22:55:21,329][105620] Updated weights for policy 1, policy_version 1037186 (0.0008) [2023-12-26 22:55:21,964][105692] Updated weights for policy 0, policy_version 1036593 (0.0008) [2023-12-26 22:55:22,027][105692] Updated weights for policy 0, policy_version 1036603 (0.0009) [2023-12-26 22:55:22,089][105692] Updated weights for policy 0, policy_version 1036613 (0.0006) [2023-12-26 22:55:22,156][105620] Updated weights for policy 1, policy_version 1037196 (0.0009) [2023-12-26 22:55:22,225][105620] Updated weights for policy 1, policy_version 1037206 (0.0010) [2023-12-26 22:55:22,295][105620] Updated weights for policy 1, policy_version 1037216 (0.0009) [2023-12-26 22:55:22,711][105692] Updated weights for policy 0, policy_version 1036623 (0.0008) [2023-12-26 22:55:22,775][105692] Updated weights for policy 0, policy_version 1036633 (0.0010) [2023-12-26 22:55:22,830][105692] Updated weights for policy 0, policy_version 1036643 (0.0006) [2023-12-26 22:55:23,104][105620] Updated weights for policy 1, policy_version 1037226 (0.0008) [2023-12-26 22:55:23,156][105620] Updated weights for policy 1, policy_version 1037236 (0.0009) [2023-12-26 22:55:23,214][105620] Updated weights for policy 1, policy_version 1037246 (0.0010) [2023-12-26 22:55:23,272][105620] Updated weights for policy 1, policy_version 1037256 (0.0010) [2023-12-26 22:55:23,430][105692] Updated weights for policy 0, policy_version 1036653 (0.0008) [2023-12-26 22:55:23,485][105692] Updated weights for policy 0, policy_version 1036664 (0.0009) [2023-12-26 22:55:23,538][105692] Updated weights for policy 0, policy_version 1036674 (0.0010) [2023-12-26 22:55:23,894][105620] Updated weights for policy 1, policy_version 1037266 (0.0006) [2023-12-26 22:55:23,945][105620] Updated weights for policy 1, policy_version 1037276 (0.0007) [2023-12-26 22:55:24,004][105620] Updated weights for policy 1, policy_version 1037286 (0.0008) [2023-12-26 22:55:24,397][105692] Updated weights for policy 0, policy_version 1036684 (0.0009) [2023-12-26 22:55:24,448][105692] Updated weights for policy 0, policy_version 1036694 (0.0008) [2023-12-26 22:55:24,496][105692] Updated weights for policy 0, policy_version 1036704 (0.0008) [2023-12-26 22:55:24,747][105620] Updated weights for policy 1, policy_version 1037296 (0.0009) [2023-12-26 22:55:24,796][105620] Updated weights for policy 1, policy_version 1037306 (0.0008) [2023-12-26 22:55:24,841][105620] Updated weights for policy 1, policy_version 1037316 (0.0009) [2023-12-26 22:55:25,303][105692] Updated weights for policy 0, policy_version 1036714 (0.0009) [2023-12-26 22:55:25,358][105692] Updated weights for policy 0, policy_version 1036724 (0.0009) [2023-12-26 22:55:25,413][105692] Updated weights for policy 0, policy_version 1036734 (0.0009) [2023-12-26 22:55:25,464][105692] Updated weights for policy 0, policy_version 1036744 (0.0009) [2023-12-26 22:55:25,552][105620] Updated weights for policy 1, policy_version 1037326 (0.0009) [2023-12-26 22:55:25,598][105620] Updated weights for policy 1, policy_version 1037336 (0.0009) [2023-12-26 22:55:25,648][105620] Updated weights for policy 1, policy_version 1037346 (0.0009) [2023-12-26 22:55:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 531038208. Throughput: 0: 9387.3, 1: 9736.5. Samples: 531048064. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:55:26,063][104569] Avg episode reward: [(0, '8906.938'), (1, '9265.387')] [2023-12-26 22:55:26,234][105692] Updated weights for policy 0, policy_version 1036754 (0.0009) [2023-12-26 22:55:26,285][105692] Updated weights for policy 0, policy_version 1036764 (0.0009) [2023-12-26 22:55:26,334][105692] Updated weights for policy 0, policy_version 1036774 (0.0009) [2023-12-26 22:55:26,415][105620] Updated weights for policy 1, policy_version 1037356 (0.0008) [2023-12-26 22:55:26,462][105620] Updated weights for policy 1, policy_version 1037366 (0.0009) [2023-12-26 22:55:26,509][105620] Updated weights for policy 1, policy_version 1037376 (0.0008) [2023-12-26 22:55:27,105][105692] Updated weights for policy 0, policy_version 1036784 (0.0009) [2023-12-26 22:55:27,160][105692] Updated weights for policy 0, policy_version 1036794 (0.0009) [2023-12-26 22:55:27,218][105692] Updated weights for policy 0, policy_version 1036804 (0.0009) [2023-12-26 22:55:27,272][105620] Updated weights for policy 1, policy_version 1037386 (0.0008) [2023-12-26 22:55:27,329][105620] Updated weights for policy 1, policy_version 1037396 (0.0009) [2023-12-26 22:55:27,379][105620] Updated weights for policy 1, policy_version 1037406 (0.0009) [2023-12-26 22:55:27,424][105620] Updated weights for policy 1, policy_version 1037416 (0.0008) [2023-12-26 22:55:27,937][105692] Updated weights for policy 0, policy_version 1036814 (0.0009) [2023-12-26 22:55:27,992][105692] Updated weights for policy 0, policy_version 1036824 (0.0009) [2023-12-26 22:55:28,049][105692] Updated weights for policy 0, policy_version 1036834 (0.0009) [2023-12-26 22:55:28,227][105620] Updated weights for policy 1, policy_version 1037426 (0.0009) [2023-12-26 22:55:28,275][105620] Updated weights for policy 1, policy_version 1037436 (0.0009) [2023-12-26 22:55:28,330][105620] Updated weights for policy 1, policy_version 1037446 (0.0009) [2023-12-26 22:55:28,810][105692] Updated weights for policy 0, policy_version 1036844 (0.0009) [2023-12-26 22:55:28,869][105692] Updated weights for policy 0, policy_version 1036854 (0.0009) [2023-12-26 22:55:28,915][105692] Updated weights for policy 0, policy_version 1036864 (0.0007) [2023-12-26 22:55:29,112][105620] Updated weights for policy 1, policy_version 1037456 (0.0010) [2023-12-26 22:55:29,159][105620] Updated weights for policy 1, policy_version 1037466 (0.0010) [2023-12-26 22:55:29,216][105620] Updated weights for policy 1, policy_version 1037476 (0.0010) [2023-12-26 22:55:29,769][105692] Updated weights for policy 0, policy_version 1036874 (0.0008) [2023-12-26 22:55:29,792][105620] Updated weights for policy 1, policy_version 1037486 (0.0006) [2023-12-26 22:55:29,818][105692] Updated weights for policy 0, policy_version 1036884 (0.0008) [2023-12-26 22:55:29,856][105620] Updated weights for policy 1, policy_version 1037496 (0.0007) [2023-12-26 22:55:29,875][105692] Updated weights for policy 0, policy_version 1036894 (0.0008) [2023-12-26 22:55:29,915][105620] Updated weights for policy 1, policy_version 1037506 (0.0007) [2023-12-26 22:55:29,922][105692] Updated weights for policy 0, policy_version 1036904 (0.0007) [2023-12-26 22:55:30,620][105620] Updated weights for policy 1, policy_version 1037516 (0.0008) [2023-12-26 22:55:30,672][105692] Updated weights for policy 0, policy_version 1036914 (0.0008) [2023-12-26 22:55:30,677][105620] Updated weights for policy 1, policy_version 1037526 (0.0010) [2023-12-26 22:55:30,721][105692] Updated weights for policy 0, policy_version 1036924 (0.0010) [2023-12-26 22:55:30,734][105620] Updated weights for policy 1, policy_version 1037536 (0.0010) [2023-12-26 22:55:30,768][105692] Updated weights for policy 0, policy_version 1036934 (0.0008) [2023-12-26 22:55:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 531136512. Throughput: 0: 9398.3, 1: 9707.9. Samples: 531104456. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:55:31,063][104569] Avg episode reward: [(0, '9087.525'), (1, '9265.635')] [2023-12-26 22:55:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001036936_265494528.pth... [2023-12-26 22:55:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001037544_265641984.pth... [2023-12-26 22:55:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001035816_265207808.pth [2023-12-26 22:55:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001036424_265355264.pth [2023-12-26 22:55:31,363][105620] Updated weights for policy 1, policy_version 1037546 (0.0010) [2023-12-26 22:55:31,427][105620] Updated weights for policy 1, policy_version 1037556 (0.0010) [2023-12-26 22:55:31,482][105620] Updated weights for policy 1, policy_version 1037566 (0.0007) [2023-12-26 22:55:31,530][105620] Updated weights for policy 1, policy_version 1037576 (0.0010) [2023-12-26 22:55:31,656][105692] Updated weights for policy 0, policy_version 1036944 (0.0008) [2023-12-26 22:55:31,713][105692] Updated weights for policy 0, policy_version 1036954 (0.0009) [2023-12-26 22:55:31,781][105692] Updated weights for policy 0, policy_version 1036964 (0.0009) [2023-12-26 22:55:32,237][105620] Updated weights for policy 1, policy_version 1037586 (0.0008) [2023-12-26 22:55:32,301][105620] Updated weights for policy 1, policy_version 1037596 (0.0007) [2023-12-26 22:55:32,367][105620] Updated weights for policy 1, policy_version 1037606 (0.0007) [2023-12-26 22:55:32,572][105692] Updated weights for policy 0, policy_version 1036974 (0.0007) [2023-12-26 22:55:32,634][105692] Updated weights for policy 0, policy_version 1036984 (0.0006) [2023-12-26 22:55:32,696][105692] Updated weights for policy 0, policy_version 1036994 (0.0006) [2023-12-26 22:55:33,133][105620] Updated weights for policy 1, policy_version 1037616 (0.0008) [2023-12-26 22:55:33,185][105620] Updated weights for policy 1, policy_version 1037626 (0.0007) [2023-12-26 22:55:33,242][105620] Updated weights for policy 1, policy_version 1037636 (0.0009) [2023-12-26 22:55:33,326][105692] Updated weights for policy 0, policy_version 1037004 (0.0008) [2023-12-26 22:55:33,390][105692] Updated weights for policy 0, policy_version 1037014 (0.0008) [2023-12-26 22:55:33,450][105692] Updated weights for policy 0, policy_version 1037024 (0.0010) [2023-12-26 22:55:34,001][105620] Updated weights for policy 1, policy_version 1037646 (0.0009) [2023-12-26 22:55:34,059][105620] Updated weights for policy 1, policy_version 1037656 (0.0009) [2023-12-26 22:55:34,120][105620] Updated weights for policy 1, policy_version 1037666 (0.0009) [2023-12-26 22:55:34,173][105692] Updated weights for policy 0, policy_version 1037034 (0.0009) [2023-12-26 22:55:34,235][105692] Updated weights for policy 0, policy_version 1037044 (0.0006) [2023-12-26 22:55:34,297][105692] Updated weights for policy 0, policy_version 1037054 (0.0005) [2023-12-26 22:55:34,362][105692] Updated weights for policy 0, policy_version 1037064 (0.0008) [2023-12-26 22:55:34,916][105620] Updated weights for policy 1, policy_version 1037676 (0.0008) [2023-12-26 22:55:34,948][105692] Updated weights for policy 0, policy_version 1037074 (0.0006) [2023-12-26 22:55:34,968][105620] Updated weights for policy 1, policy_version 1037686 (0.0006) [2023-12-26 22:55:35,003][105692] Updated weights for policy 0, policy_version 1037084 (0.0006) [2023-12-26 22:55:35,018][105620] Updated weights for policy 1, policy_version 1037696 (0.0006) [2023-12-26 22:55:35,060][105692] Updated weights for policy 0, policy_version 1037094 (0.0008) [2023-12-26 22:55:35,616][105620] Updated weights for policy 1, policy_version 1037706 (0.0006) [2023-12-26 22:55:35,671][105620] Updated weights for policy 1, policy_version 1037716 (0.0006) [2023-12-26 22:55:35,731][105620] Updated weights for policy 1, policy_version 1037726 (0.0006) [2023-12-26 22:55:35,766][105692] Updated weights for policy 0, policy_version 1037104 (0.0007) [2023-12-26 22:55:35,794][105620] Updated weights for policy 1, policy_version 1037736 (0.0006) [2023-12-26 22:55:35,819][105692] Updated weights for policy 0, policy_version 1037114 (0.0006) [2023-12-26 22:55:35,874][105692] Updated weights for policy 0, policy_version 1037124 (0.0005) [2023-12-26 22:55:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.1, 300 sec: 19494.2). Total num frames: 531234816. Throughput: 0: 9428.8, 1: 9660.0. Samples: 531218812. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:55:36,063][104569] Avg episode reward: [(0, '9265.336'), (1, '9265.515')] [2023-12-26 22:55:36,486][105620] Updated weights for policy 1, policy_version 1037746 (0.0008) [2023-12-26 22:55:36,544][105620] Updated weights for policy 1, policy_version 1037756 (0.0009) [2023-12-26 22:55:36,580][105692] Updated weights for policy 0, policy_version 1037134 (0.0007) [2023-12-26 22:55:36,606][105620] Updated weights for policy 1, policy_version 1037766 (0.0007) [2023-12-26 22:55:36,643][105692] Updated weights for policy 0, policy_version 1037144 (0.0008) [2023-12-26 22:55:36,712][105692] Updated weights for policy 0, policy_version 1037154 (0.0008) [2023-12-26 22:55:37,230][105620] Updated weights for policy 1, policy_version 1037776 (0.0006) [2023-12-26 22:55:37,280][105620] Updated weights for policy 1, policy_version 1037786 (0.0008) [2023-12-26 22:55:37,331][105620] Updated weights for policy 1, policy_version 1037796 (0.0007) [2023-12-26 22:55:37,429][105692] Updated weights for policy 0, policy_version 1037164 (0.0007) [2023-12-26 22:55:37,487][105692] Updated weights for policy 0, policy_version 1037174 (0.0008) [2023-12-26 22:55:37,537][105692] Updated weights for policy 0, policy_version 1037184 (0.0009) [2023-12-26 22:55:37,983][105620] Updated weights for policy 1, policy_version 1037806 (0.0006) [2023-12-26 22:55:38,041][105620] Updated weights for policy 1, policy_version 1037816 (0.0006) [2023-12-26 22:55:38,097][105620] Updated weights for policy 1, policy_version 1037826 (0.0006) [2023-12-26 22:55:38,264][105692] Updated weights for policy 0, policy_version 1037195 (0.0008) [2023-12-26 22:55:38,311][105692] Updated weights for policy 0, policy_version 1037205 (0.0005) [2023-12-26 22:55:38,364][105692] Updated weights for policy 0, policy_version 1037215 (0.0008) [2023-12-26 22:55:38,754][105620] Updated weights for policy 1, policy_version 1037836 (0.0011) [2023-12-26 22:55:38,810][105620] Updated weights for policy 1, policy_version 1037846 (0.0011) [2023-12-26 22:55:38,865][105620] Updated weights for policy 1, policy_version 1037856 (0.0011) [2023-12-26 22:55:39,010][105692] Updated weights for policy 0, policy_version 1037225 (0.0008) [2023-12-26 22:55:39,064][105692] Updated weights for policy 0, policy_version 1037235 (0.0008) [2023-12-26 22:55:39,121][105692] Updated weights for policy 0, policy_version 1037245 (0.0009) [2023-12-26 22:55:39,171][105692] Updated weights for policy 0, policy_version 1037255 (0.0009) [2023-12-26 22:55:39,541][105620] Updated weights for policy 1, policy_version 1037866 (0.0010) [2023-12-26 22:55:39,609][105620] Updated weights for policy 1, policy_version 1037876 (0.0009) [2023-12-26 22:55:39,666][105620] Updated weights for policy 1, policy_version 1037886 (0.0008) [2023-12-26 22:55:39,728][105620] Updated weights for policy 1, policy_version 1037896 (0.0008) [2023-12-26 22:55:39,980][105692] Updated weights for policy 0, policy_version 1037265 (0.0010) [2023-12-26 22:55:40,036][105692] Updated weights for policy 0, policy_version 1037275 (0.0010) [2023-12-26 22:55:40,094][105692] Updated weights for policy 0, policy_version 1037285 (0.0011) [2023-12-26 22:55:40,524][105620] Updated weights for policy 1, policy_version 1037906 (0.0005) [2023-12-26 22:55:40,583][105620] Updated weights for policy 1, policy_version 1037916 (0.0008) [2023-12-26 22:55:40,646][105620] Updated weights for policy 1, policy_version 1037926 (0.0010) [2023-12-26 22:55:40,783][105692] Updated weights for policy 0, policy_version 1037295 (0.0010) [2023-12-26 22:55:40,843][105692] Updated weights for policy 0, policy_version 1037305 (0.0007) [2023-12-26 22:55:40,903][105692] Updated weights for policy 0, policy_version 1037315 (0.0009) [2023-12-26 22:55:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 531333120. Throughput: 0: 9522.6, 1: 9714.6. Samples: 531339168. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:55:41,062][104569] Avg episode reward: [(0, '9174.078'), (1, '9356.578')] [2023-12-26 22:55:41,468][105620] Updated weights for policy 1, policy_version 1037936 (0.0009) [2023-12-26 22:55:41,523][105620] Updated weights for policy 1, policy_version 1037946 (0.0009) [2023-12-26 22:55:41,574][105620] Updated weights for policy 1, policy_version 1037957 (0.0008) [2023-12-26 22:55:41,633][105692] Updated weights for policy 0, policy_version 1037325 (0.0009) [2023-12-26 22:55:41,691][105692] Updated weights for policy 0, policy_version 1037335 (0.0009) [2023-12-26 22:55:41,752][105692] Updated weights for policy 0, policy_version 1037345 (0.0008) [2023-12-26 22:55:42,344][105620] Updated weights for policy 1, policy_version 1037967 (0.0008) [2023-12-26 22:55:42,405][105620] Updated weights for policy 1, policy_version 1037977 (0.0009) [2023-12-26 22:55:42,470][105620] Updated weights for policy 1, policy_version 1037987 (0.0009) [2023-12-26 22:55:42,527][105692] Updated weights for policy 0, policy_version 1037355 (0.0010) [2023-12-26 22:55:42,579][105692] Updated weights for policy 0, policy_version 1037365 (0.0009) [2023-12-26 22:55:42,630][105692] Updated weights for policy 0, policy_version 1037375 (0.0009) [2023-12-26 22:55:43,214][105620] Updated weights for policy 1, policy_version 1037997 (0.0009) [2023-12-26 22:55:43,266][105620] Updated weights for policy 1, policy_version 1038007 (0.0009) [2023-12-26 22:55:43,325][105620] Updated weights for policy 1, policy_version 1038017 (0.0007) [2023-12-26 22:55:43,392][105692] Updated weights for policy 0, policy_version 1037385 (0.0008) [2023-12-26 22:55:43,448][105692] Updated weights for policy 0, policy_version 1037395 (0.0005) [2023-12-26 22:55:43,501][105692] Updated weights for policy 0, policy_version 1037405 (0.0005) [2023-12-26 22:55:43,548][105692] Updated weights for policy 0, policy_version 1037415 (0.0005) [2023-12-26 22:55:43,993][105620] Updated weights for policy 1, policy_version 1038027 (0.0008) [2023-12-26 22:55:44,040][105620] Updated weights for policy 1, policy_version 1038037 (0.0005) [2023-12-26 22:55:44,085][105620] Updated weights for policy 1, policy_version 1038047 (0.0005) [2023-12-26 22:55:44,159][105692] Updated weights for policy 0, policy_version 1037425 (0.0008) [2023-12-26 22:55:44,212][105692] Updated weights for policy 0, policy_version 1037435 (0.0009) [2023-12-26 22:55:44,272][105692] Updated weights for policy 0, policy_version 1037445 (0.0008) [2023-12-26 22:55:44,705][105620] Updated weights for policy 1, policy_version 1038057 (0.0005) [2023-12-26 22:55:44,757][105620] Updated weights for policy 1, policy_version 1038067 (0.0006) [2023-12-26 22:55:44,815][105620] Updated weights for policy 1, policy_version 1038077 (0.0009) [2023-12-26 22:55:44,878][105620] Updated weights for policy 1, policy_version 1038087 (0.0010) [2023-12-26 22:55:45,042][105692] Updated weights for policy 0, policy_version 1037455 (0.0008) [2023-12-26 22:55:45,104][105692] Updated weights for policy 0, policy_version 1037465 (0.0008) [2023-12-26 22:55:45,168][105692] Updated weights for policy 0, policy_version 1037475 (0.0009) [2023-12-26 22:55:45,615][105620] Updated weights for policy 1, policy_version 1038097 (0.0008) [2023-12-26 22:55:45,660][105620] Updated weights for policy 1, policy_version 1038107 (0.0008) [2023-12-26 22:55:45,715][105620] Updated weights for policy 1, policy_version 1038117 (0.0007) [2023-12-26 22:55:45,937][105692] Updated weights for policy 0, policy_version 1037485 (0.0010) [2023-12-26 22:55:45,994][105692] Updated weights for policy 0, policy_version 1037495 (0.0010) [2023-12-26 22:55:46,057][105692] Updated weights for policy 0, policy_version 1037505 (0.0011) [2023-12-26 22:55:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.1, 300 sec: 19438.6). Total num frames: 531423232. Throughput: 0: 9500.2, 1: 9699.1. Samples: 531395916. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-26 22:55:46,063][104569] Avg episode reward: [(0, '8903.606'), (1, '9356.604')] [2023-12-26 22:55:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001038120_265789440.pth... [2023-12-26 22:55:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001037000_265502720.pth [2023-12-26 22:55:46,098][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001037512_265641984.pth... [2023-12-26 22:55:46,102][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001036392_265355264.pth [2023-12-26 22:55:46,481][105620] Updated weights for policy 1, policy_version 1038127 (0.0008) [2023-12-26 22:55:46,536][105620] Updated weights for policy 1, policy_version 1038137 (0.0008) [2023-12-26 22:55:46,580][105620] Updated weights for policy 1, policy_version 1038147 (0.0008) [2023-12-26 22:55:46,796][105692] Updated weights for policy 0, policy_version 1037515 (0.0010) [2023-12-26 22:55:46,857][105692] Updated weights for policy 0, policy_version 1037525 (0.0011) [2023-12-26 22:55:46,909][105692] Updated weights for policy 0, policy_version 1037535 (0.0010) [2023-12-26 22:55:47,355][105620] Updated weights for policy 1, policy_version 1038157 (0.0008) [2023-12-26 22:55:47,404][105620] Updated weights for policy 1, policy_version 1038167 (0.0008) [2023-12-26 22:55:47,455][105620] Updated weights for policy 1, policy_version 1038177 (0.0008) [2023-12-26 22:55:47,659][105692] Updated weights for policy 0, policy_version 1037545 (0.0011) [2023-12-26 22:55:47,724][105692] Updated weights for policy 0, policy_version 1037555 (0.0011) [2023-12-26 22:55:47,782][105692] Updated weights for policy 0, policy_version 1037565 (0.0010) [2023-12-26 22:55:47,808][105585] KL-divergence is very high: 115.8710 [2023-12-26 22:55:47,840][105692] Updated weights for policy 0, policy_version 1037575 (0.0010) [2023-12-26 22:55:48,272][105620] Updated weights for policy 1, policy_version 1038187 (0.0009) [2023-12-26 22:55:48,357][105620] Updated weights for policy 1, policy_version 1038197 (0.0009) [2023-12-26 22:55:48,420][105620] Updated weights for policy 1, policy_version 1038207 (0.0009) [2023-12-26 22:55:48,423][105692] Updated weights for policy 0, policy_version 1037585 (0.0006) [2023-12-26 22:55:48,481][105692] Updated weights for policy 0, policy_version 1037595 (0.0006) [2023-12-26 22:55:48,541][105692] Updated weights for policy 0, policy_version 1037605 (0.0007) [2023-12-26 22:55:49,147][105692] Updated weights for policy 0, policy_version 1037615 (0.0008) [2023-12-26 22:55:49,198][105692] Updated weights for policy 0, policy_version 1037625 (0.0011) [2023-12-26 22:55:49,261][105620] Updated weights for policy 1, policy_version 1038217 (0.0008) [2023-12-26 22:55:49,262][105692] Updated weights for policy 0, policy_version 1037635 (0.0011) [2023-12-26 22:55:49,323][105620] Updated weights for policy 1, policy_version 1038227 (0.0008) [2023-12-26 22:55:49,384][105620] Updated weights for policy 1, policy_version 1038237 (0.0008) [2023-12-26 22:55:49,436][105620] Updated weights for policy 1, policy_version 1038247 (0.0008) [2023-12-26 22:55:49,988][105692] Updated weights for policy 0, policy_version 1037645 (0.0011) [2023-12-26 22:55:50,053][105692] Updated weights for policy 0, policy_version 1037655 (0.0010) [2023-12-26 22:55:50,115][105692] Updated weights for policy 0, policy_version 1037665 (0.0009) [2023-12-26 22:55:50,260][105620] Updated weights for policy 1, policy_version 1038257 (0.0009) [2023-12-26 22:55:50,324][105620] Updated weights for policy 1, policy_version 1038267 (0.0008) [2023-12-26 22:55:50,392][105620] Updated weights for policy 1, policy_version 1038277 (0.0009) [2023-12-26 22:55:50,851][105692] Updated weights for policy 0, policy_version 1037675 (0.0007) [2023-12-26 22:55:50,910][105692] Updated weights for policy 0, policy_version 1037685 (0.0011) [2023-12-26 22:55:50,966][105692] Updated weights for policy 0, policy_version 1037695 (0.0011) [2023-12-26 22:55:51,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19114.6, 300 sec: 19438.6). Total num frames: 531521536. Throughput: 0: 9615.3, 1: 9586.9. Samples: 531510872. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:55:51,063][104569] Avg episode reward: [(0, '8904.789'), (1, '9264.222')] [2023-12-26 22:55:51,106][105620] Updated weights for policy 1, policy_version 1038287 (0.0007) [2023-12-26 22:55:51,172][105620] Updated weights for policy 1, policy_version 1038297 (0.0008) [2023-12-26 22:55:51,233][105620] Updated weights for policy 1, policy_version 1038307 (0.0011) [2023-12-26 22:55:51,769][105692] Updated weights for policy 0, policy_version 1037705 (0.0011) [2023-12-26 22:55:51,829][105692] Updated weights for policy 0, policy_version 1037715 (0.0011) [2023-12-26 22:55:51,885][105692] Updated weights for policy 0, policy_version 1037725 (0.0011) [2023-12-26 22:55:51,913][105620] Updated weights for policy 1, policy_version 1038317 (0.0008) [2023-12-26 22:55:51,934][105692] Updated weights for policy 0, policy_version 1037735 (0.0010) [2023-12-26 22:55:51,972][105620] Updated weights for policy 1, policy_version 1038327 (0.0006) [2023-12-26 22:55:52,038][105620] Updated weights for policy 1, policy_version 1038337 (0.0005) [2023-12-26 22:55:52,564][105692] Updated weights for policy 0, policy_version 1037745 (0.0008) [2023-12-26 22:55:52,623][105692] Updated weights for policy 0, policy_version 1037755 (0.0010) [2023-12-26 22:55:52,675][105692] Updated weights for policy 0, policy_version 1037765 (0.0010) [2023-12-26 22:55:52,710][105620] Updated weights for policy 1, policy_version 1038347 (0.0006) [2023-12-26 22:55:52,770][105620] Updated weights for policy 1, policy_version 1038357 (0.0008) [2023-12-26 22:55:52,832][105620] Updated weights for policy 1, policy_version 1038367 (0.0007) [2023-12-26 22:55:53,375][105692] Updated weights for policy 0, policy_version 1037775 (0.0009) [2023-12-26 22:55:53,443][105692] Updated weights for policy 0, policy_version 1037785 (0.0008) [2023-12-26 22:55:53,501][105692] Updated weights for policy 0, policy_version 1037795 (0.0008) [2023-12-26 22:55:53,550][105620] Updated weights for policy 1, policy_version 1038377 (0.0009) [2023-12-26 22:55:53,608][105620] Updated weights for policy 1, policy_version 1038387 (0.0006) [2023-12-26 22:55:53,666][105620] Updated weights for policy 1, policy_version 1038397 (0.0010) [2023-12-26 22:55:53,717][105620] Updated weights for policy 1, policy_version 1038407 (0.0010) [2023-12-26 22:55:54,140][105692] Updated weights for policy 0, policy_version 1037805 (0.0009) [2023-12-26 22:55:54,203][105692] Updated weights for policy 0, policy_version 1037815 (0.0009) [2023-12-26 22:55:54,255][105692] Updated weights for policy 0, policy_version 1037825 (0.0009) [2023-12-26 22:55:54,381][105620] Updated weights for policy 1, policy_version 1038417 (0.0006) [2023-12-26 22:55:54,428][105620] Updated weights for policy 1, policy_version 1038427 (0.0005) [2023-12-26 22:55:54,474][105620] Updated weights for policy 1, policy_version 1038437 (0.0005) [2023-12-26 22:55:55,034][105620] Updated weights for policy 1, policy_version 1038447 (0.0008) [2023-12-26 22:55:55,086][105620] Updated weights for policy 1, policy_version 1038457 (0.0010) [2023-12-26 22:55:55,122][105692] Updated weights for policy 0, policy_version 1037835 (0.0010) [2023-12-26 22:55:55,145][105620] Updated weights for policy 1, policy_version 1038467 (0.0010) [2023-12-26 22:55:55,179][105692] Updated weights for policy 0, policy_version 1037845 (0.0006) [2023-12-26 22:55:55,232][105692] Updated weights for policy 0, policy_version 1037855 (0.0010) [2023-12-26 22:55:55,780][105620] Updated weights for policy 1, policy_version 1038477 (0.0010) [2023-12-26 22:55:55,833][105620] Updated weights for policy 1, policy_version 1038487 (0.0006) [2023-12-26 22:55:55,879][105620] Updated weights for policy 1, policy_version 1038497 (0.0005) [2023-12-26 22:55:55,956][105692] Updated weights for policy 0, policy_version 1037865 (0.0009) [2023-12-26 22:55:56,011][105692] Updated weights for policy 0, policy_version 1037875 (0.0007) [2023-12-26 22:55:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 531619840. Throughput: 0: 9540.4, 1: 9680.7. Samples: 531629356. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:55:56,064][104569] Avg episode reward: [(0, '9174.142'), (1, '9264.087')] [2023-12-26 22:55:56,066][105692] Updated weights for policy 0, policy_version 1037885 (0.0006) [2023-12-26 22:55:56,124][105692] Updated weights for policy 0, policy_version 1037895 (0.0006) [2023-12-26 22:55:56,526][105620] Updated weights for policy 1, policy_version 1038507 (0.0005) [2023-12-26 22:55:56,581][105620] Updated weights for policy 1, policy_version 1038517 (0.0006) [2023-12-26 22:55:56,636][105620] Updated weights for policy 1, policy_version 1038527 (0.0008) [2023-12-26 22:55:56,747][105692] Updated weights for policy 0, policy_version 1037905 (0.0008) [2023-12-26 22:55:56,804][105692] Updated weights for policy 0, policy_version 1037915 (0.0008) [2023-12-26 22:55:56,862][105692] Updated weights for policy 0, policy_version 1037925 (0.0009) [2023-12-26 22:55:57,242][105620] Updated weights for policy 1, policy_version 1038537 (0.0008) [2023-12-26 22:55:57,289][105620] Updated weights for policy 1, policy_version 1038547 (0.0005) [2023-12-26 22:55:57,349][105620] Updated weights for policy 1, policy_version 1038557 (0.0005) [2023-12-26 22:55:57,402][105620] Updated weights for policy 1, policy_version 1038567 (0.0005) [2023-12-26 22:55:57,537][105692] Updated weights for policy 0, policy_version 1037935 (0.0008) [2023-12-26 22:55:57,601][105692] Updated weights for policy 0, policy_version 1037945 (0.0010) [2023-12-26 22:55:57,664][105692] Updated weights for policy 0, policy_version 1037955 (0.0010) [2023-12-26 22:55:57,940][105620] Updated weights for policy 1, policy_version 1038577 (0.0005) [2023-12-26 22:55:57,996][105620] Updated weights for policy 1, policy_version 1038587 (0.0005) [2023-12-26 22:55:58,064][105620] Updated weights for policy 1, policy_version 1038597 (0.0005) [2023-12-26 22:55:58,360][105692] Updated weights for policy 0, policy_version 1037965 (0.0010) [2023-12-26 22:55:58,425][105692] Updated weights for policy 0, policy_version 1037975 (0.0007) [2023-12-26 22:55:58,494][105692] Updated weights for policy 0, policy_version 1037985 (0.0008) [2023-12-26 22:55:58,721][105620] Updated weights for policy 1, policy_version 1038607 (0.0009) [2023-12-26 22:55:58,784][105620] Updated weights for policy 1, policy_version 1038617 (0.0009) [2023-12-26 22:55:58,857][105620] Updated weights for policy 1, policy_version 1038627 (0.0015) [2023-12-26 22:55:59,378][105692] Updated weights for policy 0, policy_version 1037995 (0.0008) [2023-12-26 22:55:59,448][105692] Updated weights for policy 0, policy_version 1038005 (0.0008) [2023-12-26 22:55:59,511][105692] Updated weights for policy 0, policy_version 1038015 (0.0007) [2023-12-26 22:55:59,689][105620] Updated weights for policy 1, policy_version 1038637 (0.0009) [2023-12-26 22:55:59,736][105620] Updated weights for policy 1, policy_version 1038647 (0.0008) [2023-12-26 22:55:59,783][105620] Updated weights for policy 1, policy_version 1038657 (0.0009) [2023-12-26 22:56:00,282][105692] Updated weights for policy 0, policy_version 1038025 (0.0008) [2023-12-26 22:56:00,343][105692] Updated weights for policy 0, policy_version 1038035 (0.0009) [2023-12-26 22:56:00,403][105692] Updated weights for policy 0, policy_version 1038045 (0.0009) [2023-12-26 22:56:00,464][105692] Updated weights for policy 0, policy_version 1038055 (0.0008) [2023-12-26 22:56:00,500][105620] Updated weights for policy 1, policy_version 1038667 (0.0009) [2023-12-26 22:56:00,557][105620] Updated weights for policy 1, policy_version 1038677 (0.0009) [2023-12-26 22:56:00,611][105620] Updated weights for policy 1, policy_version 1038687 (0.0009) [2023-12-26 22:56:01,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 531718144. Throughput: 0: 9591.9, 1: 9769.7. Samples: 531692468. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:56:01,062][104569] Avg episode reward: [(0, '9082.577'), (1, '9265.332')] [2023-12-26 22:56:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001038696_265936896.pth... [2023-12-26 22:56:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001038056_265781248.pth... [2023-12-26 22:56:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001037544_265641984.pth [2023-12-26 22:56:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001036936_265494528.pth [2023-12-26 22:56:01,154][105692] Updated weights for policy 0, policy_version 1038065 (0.0008) [2023-12-26 22:56:01,208][105692] Updated weights for policy 0, policy_version 1038075 (0.0009) [2023-12-26 22:56:01,272][105692] Updated weights for policy 0, policy_version 1038085 (0.0008) [2023-12-26 22:56:01,341][105620] Updated weights for policy 1, policy_version 1038697 (0.0009) [2023-12-26 22:56:01,410][105620] Updated weights for policy 1, policy_version 1038707 (0.0008) [2023-12-26 22:56:01,476][105620] Updated weights for policy 1, policy_version 1038717 (0.0009) [2023-12-26 22:56:01,534][105620] Updated weights for policy 1, policy_version 1038727 (0.0008) [2023-12-26 22:56:01,973][105692] Updated weights for policy 0, policy_version 1038095 (0.0006) [2023-12-26 22:56:02,030][105692] Updated weights for policy 0, policy_version 1038105 (0.0007) [2023-12-26 22:56:02,079][105692] Updated weights for policy 0, policy_version 1038115 (0.0006) [2023-12-26 22:56:02,359][105620] Updated weights for policy 1, policy_version 1038737 (0.0009) [2023-12-26 22:56:02,416][105620] Updated weights for policy 1, policy_version 1038747 (0.0009) [2023-12-26 22:56:02,466][105620] Updated weights for policy 1, policy_version 1038757 (0.0008) [2023-12-26 22:56:02,754][105692] Updated weights for policy 0, policy_version 1038125 (0.0008) [2023-12-26 22:56:02,808][105692] Updated weights for policy 0, policy_version 1038135 (0.0008) [2023-12-26 22:56:02,871][105692] Updated weights for policy 0, policy_version 1038145 (0.0008) [2023-12-26 22:56:03,150][105620] Updated weights for policy 1, policy_version 1038767 (0.0008) [2023-12-26 22:56:03,203][105620] Updated weights for policy 1, policy_version 1038777 (0.0005) [2023-12-26 22:56:03,254][105620] Updated weights for policy 1, policy_version 1038787 (0.0005) [2023-12-26 22:56:03,654][105692] Updated weights for policy 0, policy_version 1038155 (0.0009) [2023-12-26 22:56:03,712][105692] Updated weights for policy 0, policy_version 1038165 (0.0008) [2023-12-26 22:56:03,763][105692] Updated weights for policy 0, policy_version 1038175 (0.0009) [2023-12-26 22:56:03,964][105620] Updated weights for policy 1, policy_version 1038797 (0.0007) [2023-12-26 22:56:04,021][105620] Updated weights for policy 1, policy_version 1038807 (0.0009) [2023-12-26 22:56:04,086][105620] Updated weights for policy 1, policy_version 1038817 (0.0008) [2023-12-26 22:56:04,511][105692] Updated weights for policy 0, policy_version 1038185 (0.0009) [2023-12-26 22:56:04,565][105692] Updated weights for policy 0, policy_version 1038195 (0.0008) [2023-12-26 22:56:04,622][105692] Updated weights for policy 0, policy_version 1038205 (0.0008) [2023-12-26 22:56:04,674][105692] Updated weights for policy 0, policy_version 1038215 (0.0007) [2023-12-26 22:56:04,881][105620] Updated weights for policy 1, policy_version 1038827 (0.0010) [2023-12-26 22:56:04,934][105620] Updated weights for policy 1, policy_version 1038837 (0.0010) [2023-12-26 22:56:04,985][105620] Updated weights for policy 1, policy_version 1038847 (0.0010) [2023-12-26 22:56:05,333][105692] Updated weights for policy 0, policy_version 1038225 (0.0008) [2023-12-26 22:56:05,394][105692] Updated weights for policy 0, policy_version 1038235 (0.0008) [2023-12-26 22:56:05,452][105692] Updated weights for policy 0, policy_version 1038245 (0.0005) [2023-12-26 22:56:05,756][105620] Updated weights for policy 1, policy_version 1038857 (0.0010) [2023-12-26 22:56:05,818][105620] Updated weights for policy 1, policy_version 1038867 (0.0009) [2023-12-26 22:56:05,872][105620] Updated weights for policy 1, policy_version 1038877 (0.0009) [2023-12-26 22:56:05,927][105620] Updated weights for policy 1, policy_version 1038887 (0.0010) [2023-12-26 22:56:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 531816448. Throughput: 0: 9596.5, 1: 9740.0. Samples: 531805204. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:56:06,063][104569] Avg episode reward: [(0, '8916.909'), (1, '9173.083')] [2023-12-26 22:56:06,155][105692] Updated weights for policy 0, policy_version 1038255 (0.0006) [2023-12-26 22:56:06,212][105692] Updated weights for policy 0, policy_version 1038265 (0.0008) [2023-12-26 22:56:06,279][105692] Updated weights for policy 0, policy_version 1038275 (0.0010) [2023-12-26 22:56:06,574][105620] Updated weights for policy 1, policy_version 1038897 (0.0010) [2023-12-26 22:56:06,632][105620] Updated weights for policy 1, policy_version 1038907 (0.0009) [2023-12-26 22:56:06,692][105620] Updated weights for policy 1, policy_version 1038917 (0.0009) [2023-12-26 22:56:07,089][105692] Updated weights for policy 0, policy_version 1038285 (0.0008) [2023-12-26 22:56:07,136][105692] Updated weights for policy 0, policy_version 1038295 (0.0008) [2023-12-26 22:56:07,185][105692] Updated weights for policy 0, policy_version 1038305 (0.0008) [2023-12-26 22:56:07,413][105620] Updated weights for policy 1, policy_version 1038927 (0.0009) [2023-12-26 22:56:07,477][105620] Updated weights for policy 1, policy_version 1038937 (0.0010) [2023-12-26 22:56:07,533][105620] Updated weights for policy 1, policy_version 1038947 (0.0010) [2023-12-26 22:56:07,972][105692] Updated weights for policy 0, policy_version 1038315 (0.0008) [2023-12-26 22:56:08,030][105692] Updated weights for policy 0, policy_version 1038325 (0.0005) [2023-12-26 22:56:08,087][105692] Updated weights for policy 0, policy_version 1038335 (0.0005) [2023-12-26 22:56:08,290][105620] Updated weights for policy 1, policy_version 1038957 (0.0010) [2023-12-26 22:56:08,352][105620] Updated weights for policy 1, policy_version 1038967 (0.0010) [2023-12-26 22:56:08,412][105620] Updated weights for policy 1, policy_version 1038977 (0.0010) [2023-12-26 22:56:08,683][105692] Updated weights for policy 0, policy_version 1038345 (0.0005) [2023-12-26 22:56:08,746][105692] Updated weights for policy 0, policy_version 1038355 (0.0007) [2023-12-26 22:56:08,802][105692] Updated weights for policy 0, policy_version 1038365 (0.0008) [2023-12-26 22:56:08,857][105692] Updated weights for policy 0, policy_version 1038375 (0.0008) [2023-12-26 22:56:09,173][105620] Updated weights for policy 1, policy_version 1038987 (0.0010) [2023-12-26 22:56:09,237][105620] Updated weights for policy 1, policy_version 1038997 (0.0011) [2023-12-26 22:56:09,302][105620] Updated weights for policy 1, policy_version 1039007 (0.0011) [2023-12-26 22:56:09,596][105692] Updated weights for policy 0, policy_version 1038385 (0.0008) [2023-12-26 22:56:09,656][105692] Updated weights for policy 0, policy_version 1038395 (0.0008) [2023-12-26 22:56:09,723][105692] Updated weights for policy 0, policy_version 1038405 (0.0008) [2023-12-26 22:56:10,082][105620] Updated weights for policy 1, policy_version 1039017 (0.0010) [2023-12-26 22:56:10,136][105620] Updated weights for policy 1, policy_version 1039027 (0.0005) [2023-12-26 22:56:10,188][105620] Updated weights for policy 1, policy_version 1039037 (0.0005) [2023-12-26 22:56:10,241][105620] Updated weights for policy 1, policy_version 1039047 (0.0006) [2023-12-26 22:56:10,534][105692] Updated weights for policy 0, policy_version 1038415 (0.0009) [2023-12-26 22:56:10,583][105692] Updated weights for policy 0, policy_version 1038425 (0.0008) [2023-12-26 22:56:10,643][105692] Updated weights for policy 0, policy_version 1038435 (0.0010) [2023-12-26 22:56:10,864][105620] Updated weights for policy 1, policy_version 1039057 (0.0006) [2023-12-26 22:56:10,917][105620] Updated weights for policy 1, policy_version 1039067 (0.0005) [2023-12-26 22:56:10,964][105620] Updated weights for policy 1, policy_version 1039077 (0.0005) [2023-12-26 22:56:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 531914752. Throughput: 0: 9610.2, 1: 9786.4. Samples: 531920912. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:56:11,062][104569] Avg episode reward: [(0, '8740.107'), (1, '9264.100')] [2023-12-26 22:56:11,418][105692] Updated weights for policy 0, policy_version 1038445 (0.0008) [2023-12-26 22:56:11,482][105692] Updated weights for policy 0, policy_version 1038455 (0.0011) [2023-12-26 22:56:11,551][105692] Updated weights for policy 0, policy_version 1038465 (0.0010) [2023-12-26 22:56:11,631][105620] Updated weights for policy 1, policy_version 1039087 (0.0007) [2023-12-26 22:56:11,700][105620] Updated weights for policy 1, policy_version 1039097 (0.0008) [2023-12-26 22:56:11,772][105620] Updated weights for policy 1, policy_version 1039107 (0.0008) [2023-12-26 22:56:12,296][105692] Updated weights for policy 0, policy_version 1038475 (0.0009) [2023-12-26 22:56:12,355][105692] Updated weights for policy 0, policy_version 1038485 (0.0008) [2023-12-26 22:56:12,413][105692] Updated weights for policy 0, policy_version 1038495 (0.0009) [2023-12-26 22:56:12,535][105620] Updated weights for policy 1, policy_version 1039117 (0.0009) [2023-12-26 22:56:12,588][105620] Updated weights for policy 1, policy_version 1039127 (0.0011) [2023-12-26 22:56:12,649][105620] Updated weights for policy 1, policy_version 1039137 (0.0011) [2023-12-26 22:56:13,139][105692] Updated weights for policy 0, policy_version 1038505 (0.0009) [2023-12-26 22:56:13,202][105692] Updated weights for policy 0, policy_version 1038515 (0.0008) [2023-12-26 22:56:13,260][105692] Updated weights for policy 0, policy_version 1038525 (0.0008) [2023-12-26 22:56:13,315][105692] Updated weights for policy 0, policy_version 1038535 (0.0008) [2023-12-26 22:56:13,355][105620] Updated weights for policy 1, policy_version 1039147 (0.0011) [2023-12-26 22:56:13,419][105620] Updated weights for policy 1, policy_version 1039157 (0.0010) [2023-12-26 22:56:13,484][105620] Updated weights for policy 1, policy_version 1039167 (0.0010) [2023-12-26 22:56:14,097][105620] Updated weights for policy 1, policy_version 1039177 (0.0007) [2023-12-26 22:56:14,116][105692] Updated weights for policy 0, policy_version 1038545 (0.0008) [2023-12-26 22:56:14,148][105620] Updated weights for policy 1, policy_version 1039187 (0.0010) [2023-12-26 22:56:14,162][105692] Updated weights for policy 0, policy_version 1038555 (0.0009) [2023-12-26 22:56:14,202][105620] Updated weights for policy 1, policy_version 1039197 (0.0009) [2023-12-26 22:56:14,221][105692] Updated weights for policy 0, policy_version 1038565 (0.0008) [2023-12-26 22:56:14,262][105620] Updated weights for policy 1, policy_version 1039207 (0.0005) [2023-12-26 22:56:14,821][105620] Updated weights for policy 1, policy_version 1039217 (0.0010) [2023-12-26 22:56:14,885][105620] Updated weights for policy 1, policy_version 1039227 (0.0011) [2023-12-26 22:56:14,954][105620] Updated weights for policy 1, policy_version 1039237 (0.0006) [2023-12-26 22:56:15,091][105692] Updated weights for policy 0, policy_version 1038575 (0.0009) [2023-12-26 22:56:15,149][105692] Updated weights for policy 0, policy_version 1038585 (0.0009) [2023-12-26 22:56:15,207][105692] Updated weights for policy 0, policy_version 1038595 (0.0009) [2023-12-26 22:56:15,573][105620] Updated weights for policy 1, policy_version 1039247 (0.0006) [2023-12-26 22:56:15,631][105620] Updated weights for policy 1, policy_version 1039257 (0.0005) [2023-12-26 22:56:15,692][105620] Updated weights for policy 1, policy_version 1039267 (0.0005) [2023-12-26 22:56:16,049][105692] Updated weights for policy 0, policy_version 1038605 (0.0008) [2023-12-26 22:56:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 532004864. Throughput: 0: 9589.7, 1: 9825.4. Samples: 531978136. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:56:16,063][104569] Avg episode reward: [(0, '8727.532'), (1, '9263.693')] [2023-12-26 22:56:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001039272_266084352.pth... [2023-12-26 22:56:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001038120_265789440.pth [2023-12-26 22:56:16,105][105692] Updated weights for policy 0, policy_version 1038615 (0.0009) [2023-12-26 22:56:16,156][105692] Updated weights for policy 0, policy_version 1038625 (0.0008) [2023-12-26 22:56:16,189][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001038632_265928704.pth... [2023-12-26 22:56:16,192][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001037512_265641984.pth [2023-12-26 22:56:16,288][105620] Updated weights for policy 1, policy_version 1039277 (0.0005) [2023-12-26 22:56:16,337][105620] Updated weights for policy 1, policy_version 1039287 (0.0008) [2023-12-26 22:56:16,395][105620] Updated weights for policy 1, policy_version 1039297 (0.0010) [2023-12-26 22:56:16,968][105620] Updated weights for policy 1, policy_version 1039307 (0.0009) [2023-12-26 22:56:17,016][105692] Updated weights for policy 0, policy_version 1038635 (0.0010) [2023-12-26 22:56:17,019][105620] Updated weights for policy 1, policy_version 1039317 (0.0005) [2023-12-26 22:56:17,073][105620] Updated weights for policy 1, policy_version 1039327 (0.0005) [2023-12-26 22:56:17,080][105692] Updated weights for policy 0, policy_version 1038645 (0.0008) [2023-12-26 22:56:17,141][105692] Updated weights for policy 0, policy_version 1038655 (0.0008) [2023-12-26 22:56:17,765][105620] Updated weights for policy 1, policy_version 1039337 (0.0008) [2023-12-26 22:56:17,820][105620] Updated weights for policy 1, policy_version 1039347 (0.0009) [2023-12-26 22:56:17,847][105692] Updated weights for policy 0, policy_version 1038665 (0.0008) [2023-12-26 22:56:17,876][105620] Updated weights for policy 1, policy_version 1039357 (0.0010) [2023-12-26 22:56:17,908][105692] Updated weights for policy 0, policy_version 1038675 (0.0007) [2023-12-26 22:56:17,937][105620] Updated weights for policy 1, policy_version 1039367 (0.0009) [2023-12-26 22:56:17,966][105692] Updated weights for policy 0, policy_version 1038685 (0.0007) [2023-12-26 22:56:18,021][105692] Updated weights for policy 0, policy_version 1038695 (0.0009) [2023-12-26 22:56:18,664][105620] Updated weights for policy 1, policy_version 1039377 (0.0009) [2023-12-26 22:56:18,713][105620] Updated weights for policy 1, policy_version 1039387 (0.0008) [2023-12-26 22:56:18,769][105620] Updated weights for policy 1, policy_version 1039397 (0.0008) [2023-12-26 22:56:18,799][105692] Updated weights for policy 0, policy_version 1038705 (0.0009) [2023-12-26 22:56:18,850][105692] Updated weights for policy 0, policy_version 1038715 (0.0008) [2023-12-26 22:56:18,912][105692] Updated weights for policy 0, policy_version 1038725 (0.0009) [2023-12-26 22:56:19,524][105620] Updated weights for policy 1, policy_version 1039407 (0.0008) [2023-12-26 22:56:19,578][105620] Updated weights for policy 1, policy_version 1039417 (0.0009) [2023-12-26 22:56:19,636][105620] Updated weights for policy 1, policy_version 1039427 (0.0009) [2023-12-26 22:56:19,696][105692] Updated weights for policy 0, policy_version 1038735 (0.0009) [2023-12-26 22:56:19,751][105692] Updated weights for policy 0, policy_version 1038745 (0.0010) [2023-12-26 22:56:19,804][105692] Updated weights for policy 0, policy_version 1038755 (0.0010) [2023-12-26 22:56:20,388][105620] Updated weights for policy 1, policy_version 1039437 (0.0009) [2023-12-26 22:56:20,444][105620] Updated weights for policy 1, policy_version 1039447 (0.0010) [2023-12-26 22:56:20,507][105620] Updated weights for policy 1, policy_version 1039457 (0.0011) [2023-12-26 22:56:20,600][105692] Updated weights for policy 0, policy_version 1038765 (0.0008) [2023-12-26 22:56:20,664][105692] Updated weights for policy 0, policy_version 1038775 (0.0008) [2023-12-26 22:56:20,728][105692] Updated weights for policy 0, policy_version 1038785 (0.0009) [2023-12-26 22:56:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19438.7). Total num frames: 532103168. Throughput: 0: 9527.8, 1: 9912.0. Samples: 532093596. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:56:21,062][104569] Avg episode reward: [(0, '8818.001'), (1, '9263.651')] [2023-12-26 22:56:21,260][105620] Updated weights for policy 1, policy_version 1039467 (0.0011) [2023-12-26 22:56:21,324][105620] Updated weights for policy 1, policy_version 1039477 (0.0010) [2023-12-26 22:56:21,396][105620] Updated weights for policy 1, policy_version 1039487 (0.0009) [2023-12-26 22:56:21,431][105692] Updated weights for policy 0, policy_version 1038795 (0.0009) [2023-12-26 22:56:21,496][105692] Updated weights for policy 0, policy_version 1038805 (0.0009) [2023-12-26 22:56:21,559][105692] Updated weights for policy 0, policy_version 1038815 (0.0008) [2023-12-26 22:56:22,159][105620] Updated weights for policy 1, policy_version 1039497 (0.0008) [2023-12-26 22:56:22,223][105620] Updated weights for policy 1, policy_version 1039507 (0.0011) [2023-12-26 22:56:22,284][105620] Updated weights for policy 1, policy_version 1039517 (0.0009) [2023-12-26 22:56:22,338][105692] Updated weights for policy 0, policy_version 1038825 (0.0008) [2023-12-26 22:56:22,354][105620] Updated weights for policy 1, policy_version 1039527 (0.0007) [2023-12-26 22:56:22,405][105692] Updated weights for policy 0, policy_version 1038835 (0.0009) [2023-12-26 22:56:22,466][105692] Updated weights for policy 0, policy_version 1038845 (0.0009) [2023-12-26 22:56:22,527][105692] Updated weights for policy 0, policy_version 1038855 (0.0009) [2023-12-26 22:56:23,037][105620] Updated weights for policy 1, policy_version 1039537 (0.0007) [2023-12-26 22:56:23,103][105620] Updated weights for policy 1, policy_version 1039547 (0.0008) [2023-12-26 22:56:23,170][105620] Updated weights for policy 1, policy_version 1039557 (0.0008) [2023-12-26 22:56:23,274][105692] Updated weights for policy 0, policy_version 1038865 (0.0008) [2023-12-26 22:56:23,333][105692] Updated weights for policy 0, policy_version 1038875 (0.0009) [2023-12-26 22:56:23,392][105692] Updated weights for policy 0, policy_version 1038885 (0.0008) [2023-12-26 22:56:23,791][105620] Updated weights for policy 1, policy_version 1039567 (0.0008) [2023-12-26 22:56:23,850][105620] Updated weights for policy 1, policy_version 1039577 (0.0009) [2023-12-26 22:56:23,908][105620] Updated weights for policy 1, policy_version 1039587 (0.0009) [2023-12-26 22:56:24,130][105692] Updated weights for policy 0, policy_version 1038895 (0.0008) [2023-12-26 22:56:24,198][105692] Updated weights for policy 0, policy_version 1038905 (0.0009) [2023-12-26 22:56:24,248][105692] Updated weights for policy 0, policy_version 1038915 (0.0009) [2023-12-26 22:56:24,643][105620] Updated weights for policy 1, policy_version 1039597 (0.0009) [2023-12-26 22:56:24,690][105620] Updated weights for policy 1, policy_version 1039607 (0.0009) [2023-12-26 22:56:24,738][105620] Updated weights for policy 1, policy_version 1039617 (0.0009) [2023-12-26 22:56:24,999][105692] Updated weights for policy 0, policy_version 1038925 (0.0009) [2023-12-26 22:56:25,064][105692] Updated weights for policy 0, policy_version 1038935 (0.0010) [2023-12-26 22:56:25,121][105692] Updated weights for policy 0, policy_version 1038945 (0.0013) [2023-12-26 22:56:25,400][105620] Updated weights for policy 1, policy_version 1039627 (0.0008) [2023-12-26 22:56:25,456][105620] Updated weights for policy 1, policy_version 1039637 (0.0005) [2023-12-26 22:56:25,520][105620] Updated weights for policy 1, policy_version 1039647 (0.0009) [2023-12-26 22:56:26,000][105692] Updated weights for policy 0, policy_version 1038956 (0.0010) [2023-12-26 22:56:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 532193280. Throughput: 0: 9422.5, 1: 9876.8. Samples: 532207636. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:56:26,062][104569] Avg episode reward: [(0, '9086.872'), (1, '9356.187')] [2023-12-26 22:56:26,068][105692] Updated weights for policy 0, policy_version 1038966 (0.0009) [2023-12-26 22:56:26,082][105620] Updated weights for policy 1, policy_version 1039657 (0.0007) [2023-12-26 22:56:26,121][105692] Updated weights for policy 0, policy_version 1038976 (0.0009) [2023-12-26 22:56:26,145][105620] Updated weights for policy 1, policy_version 1039667 (0.0005) [2023-12-26 22:56:26,208][105620] Updated weights for policy 1, policy_version 1039677 (0.0008) [2023-12-26 22:56:26,259][105620] Updated weights for policy 1, policy_version 1039687 (0.0006) [2023-12-26 22:56:26,765][105620] Updated weights for policy 1, policy_version 1039697 (0.0005) [2023-12-26 22:56:26,817][105620] Updated weights for policy 1, policy_version 1039707 (0.0006) [2023-12-26 22:56:26,873][105620] Updated weights for policy 1, policy_version 1039717 (0.0005) [2023-12-26 22:56:27,028][105692] Updated weights for policy 0, policy_version 1038986 (0.0009) [2023-12-26 22:56:27,086][105692] Updated weights for policy 0, policy_version 1038997 (0.0010) [2023-12-26 22:56:27,139][105692] Updated weights for policy 0, policy_version 1039007 (0.0009) [2023-12-26 22:56:27,435][105620] Updated weights for policy 1, policy_version 1039727 (0.0009) [2023-12-26 22:56:27,479][105620] Updated weights for policy 1, policy_version 1039737 (0.0010) [2023-12-26 22:56:27,526][105620] Updated weights for policy 1, policy_version 1039747 (0.0010) [2023-12-26 22:56:27,922][105692] Updated weights for policy 0, policy_version 1039017 (0.0009) [2023-12-26 22:56:27,977][105692] Updated weights for policy 0, policy_version 1039027 (0.0008) [2023-12-26 22:56:28,032][105692] Updated weights for policy 0, policy_version 1039037 (0.0008) [2023-12-26 22:56:28,086][105692] Updated weights for policy 0, policy_version 1039047 (0.0007) [2023-12-26 22:56:28,284][105620] Updated weights for policy 1, policy_version 1039757 (0.0010) [2023-12-26 22:56:28,338][105620] Updated weights for policy 1, policy_version 1039767 (0.0010) [2023-12-26 22:56:28,398][105620] Updated weights for policy 1, policy_version 1039777 (0.0011) [2023-12-26 22:56:28,831][105692] Updated weights for policy 0, policy_version 1039057 (0.0008) [2023-12-26 22:56:28,888][105692] Updated weights for policy 0, policy_version 1039067 (0.0008) [2023-12-26 22:56:28,948][105692] Updated weights for policy 0, policy_version 1039077 (0.0008) [2023-12-26 22:56:29,153][105620] Updated weights for policy 1, policy_version 1039787 (0.0011) [2023-12-26 22:56:29,224][105620] Updated weights for policy 1, policy_version 1039797 (0.0010) [2023-12-26 22:56:29,292][105620] Updated weights for policy 1, policy_version 1039807 (0.0008) [2023-12-26 22:56:29,700][105692] Updated weights for policy 0, policy_version 1039087 (0.0008) [2023-12-26 22:56:29,753][105692] Updated weights for policy 0, policy_version 1039097 (0.0010) [2023-12-26 22:56:29,797][105692] Updated weights for policy 0, policy_version 1039107 (0.0008) [2023-12-26 22:56:30,093][105620] Updated weights for policy 1, policy_version 1039817 (0.0009) [2023-12-26 22:56:30,139][105620] Updated weights for policy 1, policy_version 1039827 (0.0008) [2023-12-26 22:56:30,185][105620] Updated weights for policy 1, policy_version 1039837 (0.0009) [2023-12-26 22:56:30,232][105620] Updated weights for policy 1, policy_version 1039847 (0.0009) [2023-12-26 22:56:30,574][105692] Updated weights for policy 0, policy_version 1039117 (0.0008) [2023-12-26 22:56:30,621][105692] Updated weights for policy 0, policy_version 1039127 (0.0007) [2023-12-26 22:56:30,666][105692] Updated weights for policy 0, policy_version 1039137 (0.0008) [2023-12-26 22:56:31,020][105620] Updated weights for policy 1, policy_version 1039857 (0.0010) [2023-12-26 22:56:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 532291584. Throughput: 0: 9362.1, 1: 9974.2. Samples: 532266044. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:56:31,062][104569] Avg episode reward: [(0, '9176.876'), (1, '9264.629')] [2023-12-26 22:56:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001039144_266059776.pth... [2023-12-26 22:56:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001038056_265781248.pth [2023-12-26 22:56:31,079][105620] Updated weights for policy 1, policy_version 1039867 (0.0011) [2023-12-26 22:56:31,128][105620] Updated weights for policy 1, policy_version 1039877 (0.0010) [2023-12-26 22:56:31,147][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001039880_266240000.pth... [2023-12-26 22:56:31,151][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001038696_265936896.pth [2023-12-26 22:56:31,401][105692] Updated weights for policy 0, policy_version 1039147 (0.0008) [2023-12-26 22:56:31,456][105692] Updated weights for policy 0, policy_version 1039157 (0.0006) [2023-12-26 22:56:31,514][105692] Updated weights for policy 0, policy_version 1039167 (0.0008) [2023-12-26 22:56:31,902][105620] Updated weights for policy 1, policy_version 1039887 (0.0007) [2023-12-26 22:56:31,953][105620] Updated weights for policy 1, policy_version 1039897 (0.0005) [2023-12-26 22:56:32,006][105620] Updated weights for policy 1, policy_version 1039907 (0.0010) [2023-12-26 22:56:32,209][105692] Updated weights for policy 0, policy_version 1039177 (0.0008) [2023-12-26 22:56:32,271][105692] Updated weights for policy 0, policy_version 1039187 (0.0008) [2023-12-26 22:56:32,333][105692] Updated weights for policy 0, policy_version 1039197 (0.0010) [2023-12-26 22:56:32,396][105692] Updated weights for policy 0, policy_version 1039207 (0.0010) [2023-12-26 22:56:32,705][105620] Updated weights for policy 1, policy_version 1039917 (0.0008) [2023-12-26 22:56:32,771][105620] Updated weights for policy 1, policy_version 1039927 (0.0005) [2023-12-26 22:56:32,836][105620] Updated weights for policy 1, policy_version 1039937 (0.0010) [2023-12-26 22:56:33,118][105692] Updated weights for policy 0, policy_version 1039217 (0.0010) [2023-12-26 22:56:33,165][105692] Updated weights for policy 0, policy_version 1039227 (0.0010) [2023-12-26 22:56:33,215][105692] Updated weights for policy 0, policy_version 1039237 (0.0010) [2023-12-26 22:56:33,523][105620] Updated weights for policy 1, policy_version 1039947 (0.0010) [2023-12-26 22:56:33,584][105620] Updated weights for policy 1, policy_version 1039957 (0.0010) [2023-12-26 22:56:33,651][105620] Updated weights for policy 1, policy_version 1039967 (0.0010) [2023-12-26 22:56:33,985][105692] Updated weights for policy 0, policy_version 1039247 (0.0010) [2023-12-26 22:56:34,050][105692] Updated weights for policy 0, policy_version 1039257 (0.0010) [2023-12-26 22:56:34,113][105692] Updated weights for policy 0, policy_version 1039267 (0.0010) [2023-12-26 22:56:34,383][105620] Updated weights for policy 1, policy_version 1039977 (0.0011) [2023-12-26 22:56:34,439][105620] Updated weights for policy 1, policy_version 1039987 (0.0011) [2023-12-26 22:56:34,494][105620] Updated weights for policy 1, policy_version 1039997 (0.0010) [2023-12-26 22:56:34,546][105620] Updated weights for policy 1, policy_version 1040007 (0.0010) [2023-12-26 22:56:34,852][105692] Updated weights for policy 0, policy_version 1039277 (0.0010) [2023-12-26 22:56:34,909][105692] Updated weights for policy 0, policy_version 1039287 (0.0009) [2023-12-26 22:56:34,971][105692] Updated weights for policy 0, policy_version 1039297 (0.0009) [2023-12-26 22:56:35,209][105620] Updated weights for policy 1, policy_version 1040017 (0.0006) [2023-12-26 22:56:35,255][105620] Updated weights for policy 1, policy_version 1040027 (0.0005) [2023-12-26 22:56:35,303][105620] Updated weights for policy 1, policy_version 1040037 (0.0005) [2023-12-26 22:56:35,661][105692] Updated weights for policy 0, policy_version 1039307 (0.0009) [2023-12-26 22:56:35,717][105692] Updated weights for policy 0, policy_version 1039317 (0.0005) [2023-12-26 22:56:35,774][105692] Updated weights for policy 0, policy_version 1039327 (0.0005) [2023-12-26 22:56:35,984][105620] Updated weights for policy 1, policy_version 1040047 (0.0009) [2023-12-26 22:56:36,042][105620] Updated weights for policy 1, policy_version 1040057 (0.0010) [2023-12-26 22:56:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.3, 300 sec: 19410.9). Total num frames: 532389888. Throughput: 0: 9304.8, 1: 10004.4. Samples: 532379784. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:56:36,062][104569] Avg episode reward: [(0, '9356.347'), (1, '9081.671')] [2023-12-26 22:56:36,091][105620] Updated weights for policy 1, policy_version 1040067 (0.0010) [2023-12-26 22:56:36,575][105692] Updated weights for policy 0, policy_version 1039337 (0.0009) [2023-12-26 22:56:36,627][105692] Updated weights for policy 0, policy_version 1039347 (0.0009) [2023-12-26 22:56:36,675][105692] Updated weights for policy 0, policy_version 1039357 (0.0010) [2023-12-26 22:56:36,725][105692] Updated weights for policy 0, policy_version 1039367 (0.0011) [2023-12-26 22:56:36,804][105620] Updated weights for policy 1, policy_version 1040077 (0.0008) [2023-12-26 22:56:36,861][105620] Updated weights for policy 1, policy_version 1040087 (0.0006) [2023-12-26 22:56:36,911][105620] Updated weights for policy 1, policy_version 1040097 (0.0010) [2023-12-26 22:56:37,486][105692] Updated weights for policy 0, policy_version 1039377 (0.0007) [2023-12-26 22:56:37,541][105692] Updated weights for policy 0, policy_version 1039387 (0.0008) [2023-12-26 22:56:37,587][105692] Updated weights for policy 0, policy_version 1039397 (0.0010) [2023-12-26 22:56:37,622][105620] Updated weights for policy 1, policy_version 1040107 (0.0010) [2023-12-26 22:56:37,686][105620] Updated weights for policy 1, policy_version 1040117 (0.0011) [2023-12-26 22:56:37,746][105620] Updated weights for policy 1, policy_version 1040127 (0.0011) [2023-12-26 22:56:38,195][105692] Updated weights for policy 0, policy_version 1039407 (0.0008) [2023-12-26 22:56:38,253][105692] Updated weights for policy 0, policy_version 1039417 (0.0006) [2023-12-26 22:56:38,319][105692] Updated weights for policy 0, policy_version 1039427 (0.0005) [2023-12-26 22:56:38,540][105620] Updated weights for policy 1, policy_version 1040137 (0.0010) [2023-12-26 22:56:38,587][105620] Updated weights for policy 1, policy_version 1040147 (0.0010) [2023-12-26 22:56:38,639][105620] Updated weights for policy 1, policy_version 1040157 (0.0007) [2023-12-26 22:56:38,690][105620] Updated weights for policy 1, policy_version 1040167 (0.0006) [2023-12-26 22:56:39,020][105692] Updated weights for policy 0, policy_version 1039437 (0.0009) [2023-12-26 22:56:39,072][105692] Updated weights for policy 0, policy_version 1039447 (0.0010) [2023-12-26 22:56:39,135][105692] Updated weights for policy 0, policy_version 1039457 (0.0011) [2023-12-26 22:56:39,425][105620] Updated weights for policy 1, policy_version 1040177 (0.0010) [2023-12-26 22:56:39,487][105620] Updated weights for policy 1, policy_version 1040187 (0.0006) [2023-12-26 22:56:39,533][105620] Updated weights for policy 1, policy_version 1040197 (0.0010) [2023-12-26 22:56:39,933][105692] Updated weights for policy 0, policy_version 1039467 (0.0010) [2023-12-26 22:56:39,996][105692] Updated weights for policy 0, policy_version 1039477 (0.0008) [2023-12-26 22:56:40,060][105692] Updated weights for policy 0, policy_version 1039487 (0.0006) [2023-12-26 22:56:40,278][105620] Updated weights for policy 1, policy_version 1040207 (0.0008) [2023-12-26 22:56:40,346][105620] Updated weights for policy 1, policy_version 1040217 (0.0010) [2023-12-26 22:56:40,410][105620] Updated weights for policy 1, policy_version 1040227 (0.0009) [2023-12-26 22:56:40,768][105692] Updated weights for policy 0, policy_version 1039497 (0.0007) [2023-12-26 22:56:40,841][105692] Updated weights for policy 0, policy_version 1039507 (0.0006) [2023-12-26 22:56:40,899][105692] Updated weights for policy 0, policy_version 1039517 (0.0006) [2023-12-26 22:56:40,963][105692] Updated weights for policy 0, policy_version 1039527 (0.0006) [2023-12-26 22:56:41,035][105620] Updated weights for policy 1, policy_version 1040237 (0.0008) [2023-12-26 22:56:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 532488192. Throughput: 0: 9321.1, 1: 9964.9. Samples: 532497224. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:56:41,062][104569] Avg episode reward: [(0, '9266.184'), (1, '9081.135')] [2023-12-26 22:56:41,091][105620] Updated weights for policy 1, policy_version 1040247 (0.0010) [2023-12-26 22:56:41,159][105620] Updated weights for policy 1, policy_version 1040257 (0.0010) [2023-12-26 22:56:41,623][105692] Updated weights for policy 0, policy_version 1039537 (0.0008) [2023-12-26 22:56:41,680][105692] Updated weights for policy 0, policy_version 1039547 (0.0008) [2023-12-26 22:56:41,747][105692] Updated weights for policy 0, policy_version 1039557 (0.0009) [2023-12-26 22:56:41,909][105620] Updated weights for policy 1, policy_version 1040267 (0.0010) [2023-12-26 22:56:41,963][105620] Updated weights for policy 1, policy_version 1040277 (0.0006) [2023-12-26 22:56:42,020][105620] Updated weights for policy 1, policy_version 1040287 (0.0005) [2023-12-26 22:56:42,566][105692] Updated weights for policy 0, policy_version 1039567 (0.0008) [2023-12-26 22:56:42,630][105692] Updated weights for policy 0, policy_version 1039577 (0.0008) [2023-12-26 22:56:42,690][105692] Updated weights for policy 0, policy_version 1039587 (0.0006) [2023-12-26 22:56:42,698][105620] Updated weights for policy 1, policy_version 1040297 (0.0006) [2023-12-26 22:56:42,756][105620] Updated weights for policy 1, policy_version 1040307 (0.0007) [2023-12-26 22:56:42,820][105620] Updated weights for policy 1, policy_version 1040317 (0.0007) [2023-12-26 22:56:42,888][105620] Updated weights for policy 1, policy_version 1040327 (0.0010) [2023-12-26 22:56:43,437][105692] Updated weights for policy 0, policy_version 1039597 (0.0006) [2023-12-26 22:56:43,492][105620] Updated weights for policy 1, policy_version 1040337 (0.0010) [2023-12-26 22:56:43,499][105692] Updated weights for policy 0, policy_version 1039607 (0.0006) [2023-12-26 22:56:43,551][105620] Updated weights for policy 1, policy_version 1040347 (0.0007) [2023-12-26 22:56:43,565][105692] Updated weights for policy 0, policy_version 1039617 (0.0008) [2023-12-26 22:56:43,608][105620] Updated weights for policy 1, policy_version 1040357 (0.0005) [2023-12-26 22:56:44,191][105620] Updated weights for policy 1, policy_version 1040367 (0.0007) [2023-12-26 22:56:44,257][105620] Updated weights for policy 1, policy_version 1040377 (0.0007) [2023-12-26 22:56:44,321][105620] Updated weights for policy 1, policy_version 1040387 (0.0009) [2023-12-26 22:56:44,400][105692] Updated weights for policy 0, policy_version 1039627 (0.0008) [2023-12-26 22:56:44,449][105692] Updated weights for policy 0, policy_version 1039637 (0.0008) [2023-12-26 22:56:44,483][105585] KL-divergence is very high: 121.2964 [2023-12-26 22:56:44,488][105585] KL-divergence is very high: 116.6140 [2023-12-26 22:56:44,497][105692] Updated weights for policy 0, policy_version 1039647 (0.0007) [2023-12-26 22:56:44,526][105585] KL-divergence is very high: 219.5909 [2023-12-26 22:56:44,531][105585] KL-divergence is very high: 192.0288 [2023-12-26 22:56:44,979][105620] Updated weights for policy 1, policy_version 1040397 (0.0010) [2023-12-26 22:56:45,039][105620] Updated weights for policy 1, policy_version 1040407 (0.0010) [2023-12-26 22:56:45,099][105620] Updated weights for policy 1, policy_version 1040417 (0.0010) [2023-12-26 22:56:45,312][105692] Updated weights for policy 0, policy_version 1039657 (0.0008) [2023-12-26 22:56:45,366][105692] Updated weights for policy 0, policy_version 1039667 (0.0008) [2023-12-26 22:56:45,423][105692] Updated weights for policy 0, policy_version 1039677 (0.0008) [2023-12-26 22:56:45,475][105692] Updated weights for policy 0, policy_version 1039687 (0.0008) [2023-12-26 22:56:45,852][105620] Updated weights for policy 1, policy_version 1040427 (0.0009) [2023-12-26 22:56:45,901][105620] Updated weights for policy 1, policy_version 1040437 (0.0008) [2023-12-26 22:56:45,958][105620] Updated weights for policy 1, policy_version 1040447 (0.0006) [2023-12-26 22:56:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 532586496. Throughput: 0: 9261.6, 1: 9922.0. Samples: 532555732. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:56:46,062][104569] Avg episode reward: [(0, '8904.530'), (1, '9264.134')] [2023-12-26 22:56:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001040456_266387456.pth... [2023-12-26 22:56:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001039688_266199040.pth... [2023-12-26 22:56:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001038632_265928704.pth [2023-12-26 22:56:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001039272_266084352.pth [2023-12-26 22:56:46,269][105692] Updated weights for policy 0, policy_version 1039697 (0.0009) [2023-12-26 22:56:46,331][105692] Updated weights for policy 0, policy_version 1039707 (0.0009) [2023-12-26 22:56:46,389][105692] Updated weights for policy 0, policy_version 1039717 (0.0009) [2023-12-26 22:56:46,652][105620] Updated weights for policy 1, policy_version 1040457 (0.0006) [2023-12-26 22:56:46,723][105620] Updated weights for policy 1, policy_version 1040467 (0.0009) [2023-12-26 22:56:46,790][105620] Updated weights for policy 1, policy_version 1040477 (0.0009) [2023-12-26 22:56:46,844][105620] Updated weights for policy 1, policy_version 1040487 (0.0009) [2023-12-26 22:56:47,150][105692] Updated weights for policy 0, policy_version 1039727 (0.0008) [2023-12-26 22:56:47,210][105692] Updated weights for policy 0, policy_version 1039737 (0.0009) [2023-12-26 22:56:47,267][105692] Updated weights for policy 0, policy_version 1039747 (0.0009) [2023-12-26 22:56:47,488][105620] Updated weights for policy 1, policy_version 1040497 (0.0005) [2023-12-26 22:56:47,534][105620] Updated weights for policy 1, policy_version 1040507 (0.0005) [2023-12-26 22:56:47,580][105620] Updated weights for policy 1, policy_version 1040517 (0.0005) [2023-12-26 22:56:48,145][105620] Updated weights for policy 1, policy_version 1040527 (0.0005) [2023-12-26 22:56:48,173][105692] Updated weights for policy 0, policy_version 1039757 (0.0009) [2023-12-26 22:56:48,200][105620] Updated weights for policy 1, policy_version 1040537 (0.0008) [2023-12-26 22:56:48,226][105692] Updated weights for policy 0, policy_version 1039767 (0.0006) [2023-12-26 22:56:48,258][105620] Updated weights for policy 1, policy_version 1040547 (0.0010) [2023-12-26 22:56:48,281][105692] Updated weights for policy 0, policy_version 1039777 (0.0006) [2023-12-26 22:56:48,924][105620] Updated weights for policy 1, policy_version 1040557 (0.0008) [2023-12-26 22:56:48,976][105620] Updated weights for policy 1, policy_version 1040567 (0.0005) [2023-12-26 22:56:49,037][105620] Updated weights for policy 1, policy_version 1040577 (0.0007) [2023-12-26 22:56:49,131][105692] Updated weights for policy 0, policy_version 1039787 (0.0008) [2023-12-26 22:56:49,196][105692] Updated weights for policy 0, policy_version 1039797 (0.0009) [2023-12-26 22:56:49,266][105692] Updated weights for policy 0, policy_version 1039807 (0.0008) [2023-12-26 22:56:49,689][105620] Updated weights for policy 1, policy_version 1040587 (0.0009) [2023-12-26 22:56:49,755][105620] Updated weights for policy 1, policy_version 1040597 (0.0009) [2023-12-26 22:56:49,826][105620] Updated weights for policy 1, policy_version 1040607 (0.0007) [2023-12-26 22:56:50,035][105692] Updated weights for policy 0, policy_version 1039817 (0.0007) [2023-12-26 22:56:50,102][105692] Updated weights for policy 0, policy_version 1039827 (0.0008) [2023-12-26 22:56:50,163][105692] Updated weights for policy 0, policy_version 1039837 (0.0009) [2023-12-26 22:56:50,227][105692] Updated weights for policy 0, policy_version 1039847 (0.0009) [2023-12-26 22:56:50,426][105620] Updated weights for policy 1, policy_version 1040617 (0.0008) [2023-12-26 22:56:50,482][105620] Updated weights for policy 1, policy_version 1040627 (0.0005) [2023-12-26 22:56:50,537][105620] Updated weights for policy 1, policy_version 1040637 (0.0005) [2023-12-26 22:56:50,627][105620] Updated weights for policy 1, policy_version 1040647 (0.0009) [2023-12-26 22:56:51,055][105692] Updated weights for policy 0, policy_version 1039857 (0.0009) [2023-12-26 22:56:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.3, 300 sec: 19383.1). Total num frames: 532676608. Throughput: 0: 9171.5, 1: 10059.6. Samples: 532670600. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:56:51,062][104569] Avg episode reward: [(0, '8729.188'), (1, '9263.920')] [2023-12-26 22:56:51,114][105692] Updated weights for policy 0, policy_version 1039867 (0.0009) [2023-12-26 22:56:51,179][105692] Updated weights for policy 0, policy_version 1039877 (0.0009) [2023-12-26 22:56:51,230][105620] Updated weights for policy 1, policy_version 1040657 (0.0006) [2023-12-26 22:56:51,292][105620] Updated weights for policy 1, policy_version 1040667 (0.0008) [2023-12-26 22:56:51,357][105620] Updated weights for policy 1, policy_version 1040677 (0.0010) [2023-12-26 22:56:51,975][105692] Updated weights for policy 0, policy_version 1039887 (0.0009) [2023-12-26 22:56:52,027][105692] Updated weights for policy 0, policy_version 1039897 (0.0009) [2023-12-26 22:56:52,080][105620] Updated weights for policy 1, policy_version 1040687 (0.0006) [2023-12-26 22:56:52,082][105692] Updated weights for policy 0, policy_version 1039907 (0.0008) [2023-12-26 22:56:52,128][105620] Updated weights for policy 1, policy_version 1040697 (0.0007) [2023-12-26 22:56:52,175][105620] Updated weights for policy 1, policy_version 1040707 (0.0007) [2023-12-26 22:56:52,813][105692] Updated weights for policy 0, policy_version 1039917 (0.0008) [2023-12-26 22:56:52,871][105692] Updated weights for policy 0, policy_version 1039927 (0.0009) [2023-12-26 22:56:52,925][105620] Updated weights for policy 1, policy_version 1040717 (0.0006) [2023-12-26 22:56:52,927][105692] Updated weights for policy 0, policy_version 1039937 (0.0007) [2023-12-26 22:56:52,982][105620] Updated weights for policy 1, policy_version 1040727 (0.0008) [2023-12-26 22:56:53,042][105620] Updated weights for policy 1, policy_version 1040737 (0.0009) [2023-12-26 22:56:53,658][105620] Updated weights for policy 1, policy_version 1040747 (0.0007) [2023-12-26 22:56:53,710][105620] Updated weights for policy 1, policy_version 1040757 (0.0006) [2023-12-26 22:56:53,765][105620] Updated weights for policy 1, policy_version 1040767 (0.0007) [2023-12-26 22:56:53,766][105692] Updated weights for policy 0, policy_version 1039947 (0.0006) [2023-12-26 22:56:53,825][105692] Updated weights for policy 0, policy_version 1039957 (0.0008) [2023-12-26 22:56:53,896][105692] Updated weights for policy 0, policy_version 1039967 (0.0009) [2023-12-26 22:56:54,374][105620] Updated weights for policy 1, policy_version 1040777 (0.0008) [2023-12-26 22:56:54,423][105620] Updated weights for policy 1, policy_version 1040787 (0.0008) [2023-12-26 22:56:54,473][105620] Updated weights for policy 1, policy_version 1040797 (0.0006) [2023-12-26 22:56:54,532][105620] Updated weights for policy 1, policy_version 1040807 (0.0006) [2023-12-26 22:56:54,728][105692] Updated weights for policy 0, policy_version 1039977 (0.0009) [2023-12-26 22:56:54,786][105692] Updated weights for policy 0, policy_version 1039987 (0.0009) [2023-12-26 22:56:54,834][105692] Updated weights for policy 0, policy_version 1039997 (0.0009) [2023-12-26 22:56:54,886][105692] Updated weights for policy 0, policy_version 1040007 (0.0009) [2023-12-26 22:56:55,215][105620] Updated weights for policy 1, policy_version 1040817 (0.0007) [2023-12-26 22:56:55,277][105620] Updated weights for policy 1, policy_version 1040827 (0.0006) [2023-12-26 22:56:55,333][105620] Updated weights for policy 1, policy_version 1040837 (0.0005) [2023-12-26 22:56:55,717][105692] Updated weights for policy 0, policy_version 1040017 (0.0009) [2023-12-26 22:56:55,775][105692] Updated weights for policy 0, policy_version 1040027 (0.0007) [2023-12-26 22:56:55,833][105692] Updated weights for policy 0, policy_version 1040037 (0.0005) [2023-12-26 22:56:56,024][105620] Updated weights for policy 1, policy_version 1040847 (0.0008) [2023-12-26 22:56:56,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 532774912. Throughput: 0: 9064.6, 1: 10146.3. Samples: 532785404. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:56:56,063][104569] Avg episode reward: [(0, '8910.044'), (1, '9263.520')] [2023-12-26 22:56:56,079][105620] Updated weights for policy 1, policy_version 1040857 (0.0008) [2023-12-26 22:56:56,143][105620] Updated weights for policy 1, policy_version 1040867 (0.0006) [2023-12-26 22:56:56,408][105692] Updated weights for policy 0, policy_version 1040047 (0.0007) [2023-12-26 22:56:56,457][105692] Updated weights for policy 0, policy_version 1040057 (0.0005) [2023-12-26 22:56:56,505][105692] Updated weights for policy 0, policy_version 1040067 (0.0005) [2023-12-26 22:56:56,919][105620] Updated weights for policy 1, policy_version 1040877 (0.0007) [2023-12-26 22:56:56,969][105620] Updated weights for policy 1, policy_version 1040887 (0.0008) [2023-12-26 22:56:57,013][105620] Updated weights for policy 1, policy_version 1040897 (0.0008) [2023-12-26 22:56:57,134][105692] Updated weights for policy 0, policy_version 1040077 (0.0008) [2023-12-26 22:56:57,188][105692] Updated weights for policy 0, policy_version 1040087 (0.0010) [2023-12-26 22:56:57,239][105692] Updated weights for policy 0, policy_version 1040097 (0.0010) [2023-12-26 22:56:57,847][105620] Updated weights for policy 1, policy_version 1040907 (0.0008) [2023-12-26 22:56:57,851][105692] Updated weights for policy 0, policy_version 1040107 (0.0009) [2023-12-26 22:56:57,902][105692] Updated weights for policy 0, policy_version 1040117 (0.0005) [2023-12-26 22:56:57,906][105620] Updated weights for policy 1, policy_version 1040917 (0.0009) [2023-12-26 22:56:57,952][105692] Updated weights for policy 0, policy_version 1040127 (0.0006) [2023-12-26 22:56:57,966][105620] Updated weights for policy 1, policy_version 1040927 (0.0009) [2023-12-26 22:56:58,646][105692] Updated weights for policy 0, policy_version 1040137 (0.0007) [2023-12-26 22:56:58,709][105692] Updated weights for policy 0, policy_version 1040147 (0.0009) [2023-12-26 22:56:58,782][105692] Updated weights for policy 0, policy_version 1040157 (0.0010) [2023-12-26 22:56:58,815][105620] Updated weights for policy 1, policy_version 1040937 (0.0007) [2023-12-26 22:56:58,853][105692] Updated weights for policy 0, policy_version 1040167 (0.0008) [2023-12-26 22:56:58,880][105620] Updated weights for policy 1, policy_version 1040947 (0.0007) [2023-12-26 22:56:58,948][105620] Updated weights for policy 1, policy_version 1040957 (0.0008) [2023-12-26 22:56:59,012][105620] Updated weights for policy 1, policy_version 1040967 (0.0007) [2023-12-26 22:56:59,659][105620] Updated weights for policy 1, policy_version 1040977 (0.0008) [2023-12-26 22:56:59,722][105620] Updated weights for policy 1, policy_version 1040987 (0.0006) [2023-12-26 22:56:59,723][105692] Updated weights for policy 0, policy_version 1040177 (0.0008) [2023-12-26 22:56:59,782][105620] Updated weights for policy 1, policy_version 1040997 (0.0006) [2023-12-26 22:56:59,784][105692] Updated weights for policy 0, policy_version 1040187 (0.0008) [2023-12-26 22:56:59,847][105692] Updated weights for policy 0, policy_version 1040197 (0.0008) [2023-12-26 22:57:00,562][105692] Updated weights for policy 0, policy_version 1040207 (0.0009) [2023-12-26 22:57:00,592][105620] Updated weights for policy 1, policy_version 1041007 (0.0009) [2023-12-26 22:57:00,614][105692] Updated weights for policy 0, policy_version 1040217 (0.0006) [2023-12-26 22:57:00,646][105620] Updated weights for policy 1, policy_version 1041017 (0.0007) [2023-12-26 22:57:00,675][105692] Updated weights for policy 0, policy_version 1040227 (0.0008) [2023-12-26 22:57:00,702][105620] Updated weights for policy 1, policy_version 1041027 (0.0006) [2023-12-26 22:57:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 532873216. Throughput: 0: 9185.9, 1: 10060.5. Samples: 532844224. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:57:01,063][104569] Avg episode reward: [(0, '8910.064'), (1, '9355.363')] [2023-12-26 22:57:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001040232_266338304.pth... [2023-12-26 22:57:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001041032_266534912.pth... [2023-12-26 22:57:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001039144_266059776.pth [2023-12-26 22:57:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001039880_266240000.pth [2023-12-26 22:57:01,357][105692] Updated weights for policy 0, policy_version 1040237 (0.0008) [2023-12-26 22:57:01,404][105620] Updated weights for policy 1, policy_version 1041037 (0.0007) [2023-12-26 22:57:01,427][105692] Updated weights for policy 0, policy_version 1040247 (0.0008) [2023-12-26 22:57:01,472][105620] Updated weights for policy 1, policy_version 1041047 (0.0006) [2023-12-26 22:57:01,479][105692] Updated weights for policy 0, policy_version 1040257 (0.0008) [2023-12-26 22:57:01,535][105620] Updated weights for policy 1, policy_version 1041057 (0.0007) [2023-12-26 22:57:02,191][105620] Updated weights for policy 1, policy_version 1041067 (0.0008) [2023-12-26 22:57:02,260][105620] Updated weights for policy 1, policy_version 1041077 (0.0011) [2023-12-26 22:57:02,262][105692] Updated weights for policy 0, policy_version 1040267 (0.0007) [2023-12-26 22:57:02,313][105692] Updated weights for policy 0, policy_version 1040277 (0.0009) [2023-12-26 22:57:02,315][105620] Updated weights for policy 1, policy_version 1041087 (0.0010) [2023-12-26 22:57:02,372][105692] Updated weights for policy 0, policy_version 1040287 (0.0008) [2023-12-26 22:57:03,037][105692] Updated weights for policy 0, policy_version 1040297 (0.0010) [2023-12-26 22:57:03,074][105620] Updated weights for policy 1, policy_version 1041097 (0.0010) [2023-12-26 22:57:03,080][105692] Updated weights for policy 0, policy_version 1040307 (0.0007) [2023-12-26 22:57:03,128][105620] Updated weights for policy 1, policy_version 1041107 (0.0010) [2023-12-26 22:57:03,134][105692] Updated weights for policy 0, policy_version 1040317 (0.0005) [2023-12-26 22:57:03,179][105620] Updated weights for policy 1, policy_version 1041117 (0.0010) [2023-12-26 22:57:03,196][105692] Updated weights for policy 0, policy_version 1040327 (0.0007) [2023-12-26 22:57:03,230][105620] Updated weights for policy 1, policy_version 1041127 (0.0010) [2023-12-26 22:57:03,821][105620] Updated weights for policy 1, policy_version 1041137 (0.0006) [2023-12-26 22:57:03,878][105620] Updated weights for policy 1, policy_version 1041147 (0.0008) [2023-12-26 22:57:03,932][105620] Updated weights for policy 1, policy_version 1041157 (0.0008) [2023-12-26 22:57:04,049][105692] Updated weights for policy 0, policy_version 1040337 (0.0009) [2023-12-26 22:57:04,109][105692] Updated weights for policy 0, policy_version 1040347 (0.0009) [2023-12-26 22:57:04,200][105692] Updated weights for policy 0, policy_version 1040357 (0.0008) [2023-12-26 22:57:04,615][105620] Updated weights for policy 1, policy_version 1041167 (0.0009) [2023-12-26 22:57:04,675][105620] Updated weights for policy 1, policy_version 1041177 (0.0008) [2023-12-26 22:57:04,726][105620] Updated weights for policy 1, policy_version 1041187 (0.0007) [2023-12-26 22:57:04,996][105692] Updated weights for policy 0, policy_version 1040367 (0.0008) [2023-12-26 22:57:05,057][105692] Updated weights for policy 0, policy_version 1040377 (0.0009) [2023-12-26 22:57:05,118][105692] Updated weights for policy 0, policy_version 1040387 (0.0009) [2023-12-26 22:57:05,471][105620] Updated weights for policy 1, policy_version 1041197 (0.0008) [2023-12-26 22:57:05,525][105620] Updated weights for policy 1, policy_version 1041207 (0.0009) [2023-12-26 22:57:05,589][105620] Updated weights for policy 1, policy_version 1041217 (0.0008) [2023-12-26 22:57:05,746][105692] Updated weights for policy 0, policy_version 1040397 (0.0010) [2023-12-26 22:57:05,812][105692] Updated weights for policy 0, policy_version 1040407 (0.0010) [2023-12-26 22:57:05,871][105692] Updated weights for policy 0, policy_version 1040417 (0.0008) [2023-12-26 22:57:06,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 532971520. Throughput: 0: 9223.8, 1: 10011.5. Samples: 532959184. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:57:06,062][104569] Avg episode reward: [(0, '9086.997'), (1, '9263.841')] [2023-12-26 22:57:06,295][105620] Updated weights for policy 1, policy_version 1041227 (0.0009) [2023-12-26 22:57:06,350][105620] Updated weights for policy 1, policy_version 1041237 (0.0009) [2023-12-26 22:57:06,409][105620] Updated weights for policy 1, policy_version 1041247 (0.0009) [2023-12-26 22:57:06,611][105692] Updated weights for policy 0, policy_version 1040427 (0.0007) [2023-12-26 22:57:06,671][105692] Updated weights for policy 0, policy_version 1040437 (0.0007) [2023-12-26 22:57:06,734][105692] Updated weights for policy 0, policy_version 1040447 (0.0011) [2023-12-26 22:57:07,235][105620] Updated weights for policy 1, policy_version 1041257 (0.0009) [2023-12-26 22:57:07,303][105620] Updated weights for policy 1, policy_version 1041267 (0.0009) [2023-12-26 22:57:07,364][105620] Updated weights for policy 1, policy_version 1041277 (0.0009) [2023-12-26 22:57:07,373][105692] Updated weights for policy 0, policy_version 1040457 (0.0010) [2023-12-26 22:57:07,423][105620] Updated weights for policy 1, policy_version 1041287 (0.0009) [2023-12-26 22:57:07,433][105692] Updated weights for policy 0, policy_version 1040467 (0.0006) [2023-12-26 22:57:07,495][105692] Updated weights for policy 0, policy_version 1040477 (0.0008) [2023-12-26 22:57:07,551][105692] Updated weights for policy 0, policy_version 1040487 (0.0010) [2023-12-26 22:57:08,214][105692] Updated weights for policy 0, policy_version 1040497 (0.0006) [2023-12-26 22:57:08,215][105620] Updated weights for policy 1, policy_version 1041297 (0.0007) [2023-12-26 22:57:08,274][105620] Updated weights for policy 1, policy_version 1041307 (0.0008) [2023-12-26 22:57:08,280][105692] Updated weights for policy 0, policy_version 1040507 (0.0006) [2023-12-26 22:57:08,337][105620] Updated weights for policy 1, policy_version 1041317 (0.0008) [2023-12-26 22:57:08,341][105692] Updated weights for policy 0, policy_version 1040517 (0.0008) [2023-12-26 22:57:08,999][105620] Updated weights for policy 1, policy_version 1041327 (0.0013) [2023-12-26 22:57:09,058][105620] Updated weights for policy 1, policy_version 1041337 (0.0011) [2023-12-26 22:57:09,061][105692] Updated weights for policy 0, policy_version 1040527 (0.0010) [2023-12-26 22:57:09,114][105620] Updated weights for policy 1, policy_version 1041347 (0.0010) [2023-12-26 22:57:09,125][105692] Updated weights for policy 0, policy_version 1040537 (0.0010) [2023-12-26 22:57:09,173][105692] Updated weights for policy 0, policy_version 1040547 (0.0010) [2023-12-26 22:57:09,882][105620] Updated weights for policy 1, policy_version 1041357 (0.0010) [2023-12-26 22:57:09,914][105692] Updated weights for policy 0, policy_version 1040557 (0.0010) [2023-12-26 22:57:09,948][105620] Updated weights for policy 1, policy_version 1041367 (0.0010) [2023-12-26 22:57:09,979][105692] Updated weights for policy 0, policy_version 1040567 (0.0011) [2023-12-26 22:57:10,012][105620] Updated weights for policy 1, policy_version 1041377 (0.0011) [2023-12-26 22:57:10,040][105692] Updated weights for policy 0, policy_version 1040577 (0.0011) [2023-12-26 22:57:10,650][105620] Updated weights for policy 1, policy_version 1041387 (0.0009) [2023-12-26 22:57:10,699][105692] Updated weights for policy 0, policy_version 1040587 (0.0011) [2023-12-26 22:57:10,714][105620] Updated weights for policy 1, policy_version 1041397 (0.0011) [2023-12-26 22:57:10,755][105692] Updated weights for policy 0, policy_version 1040597 (0.0010) [2023-12-26 22:57:10,773][105620] Updated weights for policy 1, policy_version 1041407 (0.0010) [2023-12-26 22:57:10,815][105692] Updated weights for policy 0, policy_version 1040607 (0.0011) [2023-12-26 22:57:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 533069824. Throughput: 0: 9333.0, 1: 9942.5. Samples: 533075032. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:57:11,063][104569] Avg episode reward: [(0, '9264.848'), (1, '9263.715')] [2023-12-26 22:57:11,519][105692] Updated weights for policy 0, policy_version 1040617 (0.0006) [2023-12-26 22:57:11,531][105620] Updated weights for policy 1, policy_version 1041417 (0.0010) [2023-12-26 22:57:11,583][105692] Updated weights for policy 0, policy_version 1040627 (0.0007) [2023-12-26 22:57:11,594][105620] Updated weights for policy 1, policy_version 1041427 (0.0010) [2023-12-26 22:57:11,655][105692] Updated weights for policy 0, policy_version 1040637 (0.0007) [2023-12-26 22:57:11,660][105620] Updated weights for policy 1, policy_version 1041437 (0.0011) [2023-12-26 22:57:11,720][105692] Updated weights for policy 0, policy_version 1040647 (0.0007) [2023-12-26 22:57:11,730][105620] Updated weights for policy 1, policy_version 1041447 (0.0011) [2023-12-26 22:57:12,373][105620] Updated weights for policy 1, policy_version 1041457 (0.0012) [2023-12-26 22:57:12,432][105620] Updated weights for policy 1, policy_version 1041467 (0.0010) [2023-12-26 22:57:12,476][105692] Updated weights for policy 0, policy_version 1040657 (0.0006) [2023-12-26 22:57:12,491][105620] Updated weights for policy 1, policy_version 1041477 (0.0010) [2023-12-26 22:57:12,525][105692] Updated weights for policy 0, policy_version 1040667 (0.0005) [2023-12-26 22:57:12,591][105692] Updated weights for policy 0, policy_version 1040677 (0.0008) [2023-12-26 22:57:13,230][105620] Updated weights for policy 1, policy_version 1041487 (0.0010) [2023-12-26 22:57:13,285][105692] Updated weights for policy 0, policy_version 1040687 (0.0006) [2023-12-26 22:57:13,289][105620] Updated weights for policy 1, policy_version 1041497 (0.0010) [2023-12-26 22:57:13,335][105692] Updated weights for policy 0, policy_version 1040697 (0.0005) [2023-12-26 22:57:13,340][105620] Updated weights for policy 1, policy_version 1041507 (0.0010) [2023-12-26 22:57:13,387][105692] Updated weights for policy 0, policy_version 1040707 (0.0007) [2023-12-26 22:57:14,037][105620] Updated weights for policy 1, policy_version 1041517 (0.0010) [2023-12-26 22:57:14,095][105620] Updated weights for policy 1, policy_version 1041527 (0.0010) [2023-12-26 22:57:14,141][105692] Updated weights for policy 0, policy_version 1040717 (0.0007) [2023-12-26 22:57:14,154][105620] Updated weights for policy 1, policy_version 1041537 (0.0010) [2023-12-26 22:57:14,188][105692] Updated weights for policy 0, policy_version 1040727 (0.0006) [2023-12-26 22:57:14,242][105692] Updated weights for policy 0, policy_version 1040737 (0.0008) [2023-12-26 22:57:14,886][105620] Updated weights for policy 1, policy_version 1041547 (0.0010) [2023-12-26 22:57:14,949][105620] Updated weights for policy 1, policy_version 1041557 (0.0011) [2023-12-26 22:57:14,990][105692] Updated weights for policy 0, policy_version 1040747 (0.0008) [2023-12-26 22:57:15,016][105620] Updated weights for policy 1, policy_version 1041567 (0.0007) [2023-12-26 22:57:15,056][105692] Updated weights for policy 0, policy_version 1040757 (0.0009) [2023-12-26 22:57:15,110][105692] Updated weights for policy 0, policy_version 1040767 (0.0009) [2023-12-26 22:57:15,725][105620] Updated weights for policy 1, policy_version 1041577 (0.0006) [2023-12-26 22:57:15,777][105620] Updated weights for policy 1, policy_version 1041587 (0.0010) [2023-12-26 22:57:15,825][105692] Updated weights for policy 0, policy_version 1040777 (0.0007) [2023-12-26 22:57:15,834][105620] Updated weights for policy 1, policy_version 1041597 (0.0012) [2023-12-26 22:57:15,874][105692] Updated weights for policy 0, policy_version 1040787 (0.0007) [2023-12-26 22:57:15,890][105620] Updated weights for policy 1, policy_version 1041607 (0.0010) [2023-12-26 22:57:15,926][105692] Updated weights for policy 0, policy_version 1040797 (0.0009) [2023-12-26 22:57:15,985][105692] Updated weights for policy 0, policy_version 1040807 (0.0005) [2023-12-26 22:57:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 533168128. Throughput: 0: 9408.1, 1: 9861.8. Samples: 533133192. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:57:16,063][104569] Avg episode reward: [(0, '9265.012'), (1, '9354.615')] [2023-12-26 22:57:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001040808_266485760.pth... [2023-12-26 22:57:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001041608_266682368.pth... [2023-12-26 22:57:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001039688_266199040.pth [2023-12-26 22:57:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001040456_266387456.pth [2023-12-26 22:57:16,684][105620] Updated weights for policy 1, policy_version 1041617 (0.0010) [2023-12-26 22:57:16,732][105620] Updated weights for policy 1, policy_version 1041627 (0.0006) [2023-12-26 22:57:16,752][105692] Updated weights for policy 0, policy_version 1040817 (0.0008) [2023-12-26 22:57:16,791][105620] Updated weights for policy 1, policy_version 1041637 (0.0005) [2023-12-26 22:57:16,810][105692] Updated weights for policy 0, policy_version 1040827 (0.0008) [2023-12-26 22:57:16,877][105692] Updated weights for policy 0, policy_version 1040837 (0.0010) [2023-12-26 22:57:17,351][105620] Updated weights for policy 1, policy_version 1041647 (0.0006) [2023-12-26 22:57:17,402][105620] Updated weights for policy 1, policy_version 1041657 (0.0005) [2023-12-26 22:57:17,457][105620] Updated weights for policy 1, policy_version 1041667 (0.0005) [2023-12-26 22:57:17,771][105692] Updated weights for policy 0, policy_version 1040847 (0.0008) [2023-12-26 22:57:17,828][105692] Updated weights for policy 0, policy_version 1040857 (0.0005) [2023-12-26 22:57:17,884][105692] Updated weights for policy 0, policy_version 1040867 (0.0005) [2023-12-26 22:57:17,986][105620] Updated weights for policy 1, policy_version 1041677 (0.0005) [2023-12-26 22:57:18,055][105620] Updated weights for policy 1, policy_version 1041687 (0.0006) [2023-12-26 22:57:18,113][105620] Updated weights for policy 1, policy_version 1041697 (0.0010) [2023-12-26 22:57:18,448][105692] Updated weights for policy 0, policy_version 1040877 (0.0006) [2023-12-26 22:57:18,509][105692] Updated weights for policy 0, policy_version 1040887 (0.0007) [2023-12-26 22:57:18,570][105692] Updated weights for policy 0, policy_version 1040897 (0.0006) [2023-12-26 22:57:18,757][105620] Updated weights for policy 1, policy_version 1041707 (0.0009) [2023-12-26 22:57:18,807][105620] Updated weights for policy 1, policy_version 1041717 (0.0005) [2023-12-26 22:57:18,851][105620] Updated weights for policy 1, policy_version 1041727 (0.0005) [2023-12-26 22:57:19,232][105692] Updated weights for policy 0, policy_version 1040907 (0.0007) [2023-12-26 22:57:19,292][105692] Updated weights for policy 0, policy_version 1040917 (0.0009) [2023-12-26 22:57:19,355][105692] Updated weights for policy 0, policy_version 1040927 (0.0008) [2023-12-26 22:57:19,621][105620] Updated weights for policy 1, policy_version 1041737 (0.0008) [2023-12-26 22:57:19,682][105620] Updated weights for policy 1, policy_version 1041747 (0.0006) [2023-12-26 22:57:19,743][105620] Updated weights for policy 1, policy_version 1041757 (0.0007) [2023-12-26 22:57:19,802][105620] Updated weights for policy 1, policy_version 1041767 (0.0008) [2023-12-26 22:57:20,101][105692] Updated weights for policy 0, policy_version 1040937 (0.0008) [2023-12-26 22:57:20,166][105692] Updated weights for policy 0, policy_version 1040947 (0.0006) [2023-12-26 22:57:20,220][105692] Updated weights for policy 0, policy_version 1040957 (0.0006) [2023-12-26 22:57:20,288][105692] Updated weights for policy 0, policy_version 1040967 (0.0010) [2023-12-26 22:57:20,538][105620] Updated weights for policy 1, policy_version 1041777 (0.0006) [2023-12-26 22:57:20,609][105620] Updated weights for policy 1, policy_version 1041787 (0.0008) [2023-12-26 22:57:20,668][105620] Updated weights for policy 1, policy_version 1041797 (0.0008) [2023-12-26 22:57:21,016][105692] Updated weights for policy 0, policy_version 1040977 (0.0009) [2023-12-26 22:57:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 533258240. Throughput: 0: 9413.9, 1: 9966.3. Samples: 533251896. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:57:21,063][104569] Avg episode reward: [(0, '9086.108'), (1, '9353.869')] [2023-12-26 22:57:21,083][105692] Updated weights for policy 0, policy_version 1040987 (0.0010) [2023-12-26 22:57:21,149][105692] Updated weights for policy 0, policy_version 1040997 (0.0008) [2023-12-26 22:57:21,381][105620] Updated weights for policy 1, policy_version 1041807 (0.0008) [2023-12-26 22:57:21,449][105620] Updated weights for policy 1, policy_version 1041817 (0.0008) [2023-12-26 22:57:21,508][105620] Updated weights for policy 1, policy_version 1041827 (0.0008) [2023-12-26 22:57:21,859][105692] Updated weights for policy 0, policy_version 1041007 (0.0008) [2023-12-26 22:57:21,924][105692] Updated weights for policy 0, policy_version 1041017 (0.0008) [2023-12-26 22:57:21,988][105692] Updated weights for policy 0, policy_version 1041027 (0.0007) [2023-12-26 22:57:22,279][105620] Updated weights for policy 1, policy_version 1041837 (0.0010) [2023-12-26 22:57:22,342][105620] Updated weights for policy 1, policy_version 1041847 (0.0010) [2023-12-26 22:57:22,410][105620] Updated weights for policy 1, policy_version 1041857 (0.0009) [2023-12-26 22:57:22,784][105692] Updated weights for policy 0, policy_version 1041037 (0.0008) [2023-12-26 22:57:22,844][105692] Updated weights for policy 0, policy_version 1041047 (0.0008) [2023-12-26 22:57:22,911][105692] Updated weights for policy 0, policy_version 1041057 (0.0009) [2023-12-26 22:57:23,091][105620] Updated weights for policy 1, policy_version 1041867 (0.0009) [2023-12-26 22:57:23,149][105620] Updated weights for policy 1, policy_version 1041877 (0.0006) [2023-12-26 22:57:23,218][105620] Updated weights for policy 1, policy_version 1041887 (0.0006) [2023-12-26 22:57:23,764][105620] Updated weights for policy 1, policy_version 1041897 (0.0005) [2023-12-26 22:57:23,785][105692] Updated weights for policy 0, policy_version 1041067 (0.0009) [2023-12-26 22:57:23,822][105620] Updated weights for policy 1, policy_version 1041907 (0.0005) [2023-12-26 22:57:23,838][105692] Updated weights for policy 0, policy_version 1041077 (0.0007) [2023-12-26 22:57:23,890][105620] Updated weights for policy 1, policy_version 1041917 (0.0005) [2023-12-26 22:57:23,900][105692] Updated weights for policy 0, policy_version 1041087 (0.0005) [2023-12-26 22:57:23,953][105620] Updated weights for policy 1, policy_version 1041927 (0.0005) [2023-12-26 22:57:24,470][105620] Updated weights for policy 1, policy_version 1041937 (0.0010) [2023-12-26 22:57:24,524][105620] Updated weights for policy 1, policy_version 1041947 (0.0010) [2023-12-26 22:57:24,584][105620] Updated weights for policy 1, policy_version 1041957 (0.0009) [2023-12-26 22:57:24,671][105692] Updated weights for policy 0, policy_version 1041097 (0.0009) [2023-12-26 22:57:24,723][105692] Updated weights for policy 0, policy_version 1041107 (0.0010) [2023-12-26 22:57:24,779][105692] Updated weights for policy 0, policy_version 1041117 (0.0010) [2023-12-26 22:57:24,824][105692] Updated weights for policy 0, policy_version 1041127 (0.0010) [2023-12-26 22:57:25,254][105620] Updated weights for policy 1, policy_version 1041967 (0.0007) [2023-12-26 22:57:25,312][105620] Updated weights for policy 1, policy_version 1041977 (0.0010) [2023-12-26 22:57:25,357][105620] Updated weights for policy 1, policy_version 1041987 (0.0010) [2023-12-26 22:57:25,527][105692] Updated weights for policy 0, policy_version 1041137 (0.0006) [2023-12-26 22:57:25,588][105692] Updated weights for policy 0, policy_version 1041147 (0.0010) [2023-12-26 22:57:25,649][105692] Updated weights for policy 0, policy_version 1041157 (0.0010) [2023-12-26 22:57:25,986][105620] Updated weights for policy 1, policy_version 1041997 (0.0008) [2023-12-26 22:57:26,036][105620] Updated weights for policy 1, policy_version 1042007 (0.0009) [2023-12-26 22:57:26,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.8, 300 sec: 19355.3). Total num frames: 533356544. Throughput: 0: 9359.7, 1: 10018.0. Samples: 533369216. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:57:26,062][104569] Avg episode reward: [(0, '9086.828'), (1, '9353.784')] [2023-12-26 22:57:26,091][105620] Updated weights for policy 1, policy_version 1042017 (0.0006) [2023-12-26 22:57:26,361][105692] Updated weights for policy 0, policy_version 1041167 (0.0010) [2023-12-26 22:57:26,409][105692] Updated weights for policy 0, policy_version 1041177 (0.0010) [2023-12-26 22:57:26,465][105692] Updated weights for policy 0, policy_version 1041187 (0.0010) [2023-12-26 22:57:26,733][105620] Updated weights for policy 1, policy_version 1042027 (0.0006) [2023-12-26 22:57:26,786][105620] Updated weights for policy 1, policy_version 1042037 (0.0005) [2023-12-26 22:57:26,835][105620] Updated weights for policy 1, policy_version 1042047 (0.0005) [2023-12-26 22:57:27,133][105692] Updated weights for policy 0, policy_version 1041197 (0.0009) [2023-12-26 22:57:27,191][105692] Updated weights for policy 0, policy_version 1041207 (0.0010) [2023-12-26 22:57:27,247][105692] Updated weights for policy 0, policy_version 1041217 (0.0010) [2023-12-26 22:57:27,350][105620] Updated weights for policy 1, policy_version 1042057 (0.0005) [2023-12-26 22:57:27,401][105620] Updated weights for policy 1, policy_version 1042067 (0.0005) [2023-12-26 22:57:27,453][105620] Updated weights for policy 1, policy_version 1042077 (0.0006) [2023-12-26 22:57:27,505][105620] Updated weights for policy 1, policy_version 1042087 (0.0005) [2023-12-26 22:57:27,930][105692] Updated weights for policy 0, policy_version 1041227 (0.0009) [2023-12-26 22:57:27,988][105692] Updated weights for policy 0, policy_version 1041237 (0.0006) [2023-12-26 22:57:28,023][105620] Updated weights for policy 1, policy_version 1042097 (0.0008) [2023-12-26 22:57:28,042][105692] Updated weights for policy 0, policy_version 1041247 (0.0005) [2023-12-26 22:57:28,071][105620] Updated weights for policy 1, policy_version 1042107 (0.0008) [2023-12-26 22:57:28,121][105620] Updated weights for policy 1, policy_version 1042117 (0.0009) [2023-12-26 22:57:28,661][105692] Updated weights for policy 0, policy_version 1041257 (0.0005) [2023-12-26 22:57:28,714][105692] Updated weights for policy 0, policy_version 1041267 (0.0006) [2023-12-26 22:57:28,759][105692] Updated weights for policy 0, policy_version 1041277 (0.0005) [2023-12-26 22:57:28,815][105692] Updated weights for policy 0, policy_version 1041287 (0.0007) [2023-12-26 22:57:28,850][105620] Updated weights for policy 1, policy_version 1042127 (0.0009) [2023-12-26 22:57:28,903][105620] Updated weights for policy 1, policy_version 1042137 (0.0009) [2023-12-26 22:57:28,964][105620] Updated weights for policy 1, policy_version 1042147 (0.0008) [2023-12-26 22:57:29,402][105692] Updated weights for policy 0, policy_version 1041297 (0.0010) [2023-12-26 22:57:29,460][105692] Updated weights for policy 0, policy_version 1041307 (0.0010) [2023-12-26 22:57:29,518][105692] Updated weights for policy 0, policy_version 1041317 (0.0011) [2023-12-26 22:57:29,660][105620] Updated weights for policy 1, policy_version 1042158 (0.0009) [2023-12-26 22:57:29,715][105620] Updated weights for policy 1, policy_version 1042168 (0.0008) [2023-12-26 22:57:29,776][105620] Updated weights for policy 1, policy_version 1042178 (0.0008) [2023-12-26 22:57:30,268][105692] Updated weights for policy 0, policy_version 1041327 (0.0010) [2023-12-26 22:57:30,330][105692] Updated weights for policy 0, policy_version 1041337 (0.0011) [2023-12-26 22:57:30,385][105692] Updated weights for policy 0, policy_version 1041347 (0.0010) [2023-12-26 22:57:30,522][105620] Updated weights for policy 1, policy_version 1042188 (0.0008) [2023-12-26 22:57:30,571][105620] Updated weights for policy 1, policy_version 1042198 (0.0008) [2023-12-26 22:57:30,629][105620] Updated weights for policy 1, policy_version 1042208 (0.0008) [2023-12-26 22:57:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 533463040. Throughput: 0: 9420.2, 1: 10090.5. Samples: 533433712. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:57:31,062][104569] Avg episode reward: [(0, '9266.593'), (1, '9353.621')] [2023-12-26 22:57:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001041352_266625024.pth... [2023-12-26 22:57:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001042216_266838016.pth... [2023-12-26 22:57:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001041032_266534912.pth [2023-12-26 22:57:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001040232_266338304.pth [2023-12-26 22:57:31,153][105692] Updated weights for policy 0, policy_version 1041357 (0.0011) [2023-12-26 22:57:31,213][105692] Updated weights for policy 0, policy_version 1041367 (0.0011) [2023-12-26 22:57:31,278][105692] Updated weights for policy 0, policy_version 1041377 (0.0011) [2023-12-26 22:57:31,317][105620] Updated weights for policy 1, policy_version 1042218 (0.0006) [2023-12-26 22:57:31,381][105620] Updated weights for policy 1, policy_version 1042228 (0.0009) [2023-12-26 22:57:31,434][105620] Updated weights for policy 1, policy_version 1042238 (0.0008) [2023-12-26 22:57:31,483][105620] Updated weights for policy 1, policy_version 1042248 (0.0008) [2023-12-26 22:57:32,051][105692] Updated weights for policy 0, policy_version 1041387 (0.0010) [2023-12-26 22:57:32,105][105692] Updated weights for policy 0, policy_version 1041397 (0.0008) [2023-12-26 22:57:32,156][105692] Updated weights for policy 0, policy_version 1041407 (0.0008) [2023-12-26 22:57:32,185][105620] Updated weights for policy 1, policy_version 1042258 (0.0010) [2023-12-26 22:57:32,244][105620] Updated weights for policy 1, policy_version 1042268 (0.0008) [2023-12-26 22:57:32,296][105620] Updated weights for policy 1, policy_version 1042278 (0.0009) [2023-12-26 22:57:32,895][105692] Updated weights for policy 0, policy_version 1041417 (0.0006) [2023-12-26 22:57:32,946][105692] Updated weights for policy 0, policy_version 1041427 (0.0008) [2023-12-26 22:57:33,005][105692] Updated weights for policy 0, policy_version 1041437 (0.0009) [2023-12-26 22:57:33,055][105620] Updated weights for policy 1, policy_version 1042288 (0.0007) [2023-12-26 22:57:33,064][105692] Updated weights for policy 0, policy_version 1041447 (0.0009) [2023-12-26 22:57:33,108][105620] Updated weights for policy 1, policy_version 1042299 (0.0009) [2023-12-26 22:57:33,165][105620] Updated weights for policy 1, policy_version 1042309 (0.0009) [2023-12-26 22:57:33,642][105692] Updated weights for policy 0, policy_version 1041457 (0.0008) [2023-12-26 22:57:33,699][105692] Updated weights for policy 0, policy_version 1041467 (0.0008) [2023-12-26 22:57:33,752][105692] Updated weights for policy 0, policy_version 1041477 (0.0008) [2023-12-26 22:57:33,981][105620] Updated weights for policy 1, policy_version 1042319 (0.0009) [2023-12-26 22:57:34,027][105620] Updated weights for policy 1, policy_version 1042329 (0.0009) [2023-12-26 22:57:34,079][105620] Updated weights for policy 1, policy_version 1042339 (0.0008) [2023-12-26 22:57:34,505][105692] Updated weights for policy 0, policy_version 1041487 (0.0009) [2023-12-26 22:57:34,555][105692] Updated weights for policy 0, policy_version 1041497 (0.0008) [2023-12-26 22:57:34,611][105692] Updated weights for policy 0, policy_version 1041507 (0.0008) [2023-12-26 22:57:34,870][105620] Updated weights for policy 1, policy_version 1042349 (0.0010) [2023-12-26 22:57:34,930][105620] Updated weights for policy 1, policy_version 1042359 (0.0011) [2023-12-26 22:57:34,998][105620] Updated weights for policy 1, policy_version 1042369 (0.0011) [2023-12-26 22:57:35,388][105692] Updated weights for policy 0, policy_version 1041517 (0.0008) [2023-12-26 22:57:35,439][105692] Updated weights for policy 0, policy_version 1041527 (0.0008) [2023-12-26 22:57:35,503][105692] Updated weights for policy 0, policy_version 1041537 (0.0008) [2023-12-26 22:57:35,733][105620] Updated weights for policy 1, policy_version 1042379 (0.0011) [2023-12-26 22:57:35,781][105620] Updated weights for policy 1, policy_version 1042389 (0.0010) [2023-12-26 22:57:35,833][105620] Updated weights for policy 1, policy_version 1042399 (0.0010) [2023-12-26 22:57:36,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19524.2, 300 sec: 19383.1). Total num frames: 533561344. Throughput: 0: 9575.3, 1: 9965.3. Samples: 533549932. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:57:36,063][104569] Avg episode reward: [(0, '9355.452'), (1, '9353.434')] [2023-12-26 22:57:36,148][105692] Updated weights for policy 0, policy_version 1041547 (0.0007) [2023-12-26 22:57:36,220][105692] Updated weights for policy 0, policy_version 1041557 (0.0006) [2023-12-26 22:57:36,285][105692] Updated weights for policy 0, policy_version 1041567 (0.0008) [2023-12-26 22:57:36,475][105620] Updated weights for policy 1, policy_version 1042409 (0.0010) [2023-12-26 22:57:36,529][105620] Updated weights for policy 1, policy_version 1042419 (0.0006) [2023-12-26 22:57:36,585][105620] Updated weights for policy 1, policy_version 1042429 (0.0008) [2023-12-26 22:57:36,645][105620] Updated weights for policy 1, policy_version 1042439 (0.0009) [2023-12-26 22:57:37,002][105692] Updated weights for policy 0, policy_version 1041577 (0.0010) [2023-12-26 22:57:37,068][105692] Updated weights for policy 0, policy_version 1041587 (0.0009) [2023-12-26 22:57:37,123][105692] Updated weights for policy 0, policy_version 1041597 (0.0006) [2023-12-26 22:57:37,177][105692] Updated weights for policy 0, policy_version 1041607 (0.0009) [2023-12-26 22:57:37,378][105620] Updated weights for policy 1, policy_version 1042449 (0.0009) [2023-12-26 22:57:37,428][105620] Updated weights for policy 1, policy_version 1042459 (0.0009) [2023-12-26 22:57:37,482][105620] Updated weights for policy 1, policy_version 1042469 (0.0009) [2023-12-26 22:57:37,916][105692] Updated weights for policy 0, policy_version 1041617 (0.0008) [2023-12-26 22:57:37,965][105692] Updated weights for policy 0, policy_version 1041627 (0.0008) [2023-12-26 22:57:38,026][105692] Updated weights for policy 0, policy_version 1041637 (0.0009) [2023-12-26 22:57:38,232][105620] Updated weights for policy 1, policy_version 1042479 (0.0009) [2023-12-26 22:57:38,286][105620] Updated weights for policy 1, policy_version 1042489 (0.0009) [2023-12-26 22:57:38,347][105620] Updated weights for policy 1, policy_version 1042499 (0.0009) [2023-12-26 22:57:38,791][105692] Updated weights for policy 0, policy_version 1041647 (0.0009) [2023-12-26 22:57:38,857][105692] Updated weights for policy 0, policy_version 1041657 (0.0007) [2023-12-26 22:57:38,916][105692] Updated weights for policy 0, policy_version 1041667 (0.0005) [2023-12-26 22:57:39,184][105620] Updated weights for policy 1, policy_version 1042509 (0.0010) [2023-12-26 22:57:39,244][105620] Updated weights for policy 1, policy_version 1042519 (0.0008) [2023-12-26 22:57:39,303][105620] Updated weights for policy 1, policy_version 1042529 (0.0009) [2023-12-26 22:57:39,523][105692] Updated weights for policy 0, policy_version 1041677 (0.0006) [2023-12-26 22:57:39,574][105692] Updated weights for policy 0, policy_version 1041687 (0.0005) [2023-12-26 22:57:39,629][105692] Updated weights for policy 0, policy_version 1041697 (0.0005) [2023-12-26 22:57:40,173][105620] Updated weights for policy 1, policy_version 1042539 (0.0009) [2023-12-26 22:57:40,233][105620] Updated weights for policy 1, policy_version 1042549 (0.0009) [2023-12-26 22:57:40,276][105692] Updated weights for policy 0, policy_version 1041707 (0.0007) [2023-12-26 22:57:40,293][105620] Updated weights for policy 1, policy_version 1042559 (0.0009) [2023-12-26 22:57:40,329][105692] Updated weights for policy 0, policy_version 1041717 (0.0007) [2023-12-26 22:57:40,387][105692] Updated weights for policy 0, policy_version 1041727 (0.0008) [2023-12-26 22:57:41,007][105620] Updated weights for policy 1, policy_version 1042569 (0.0008) [2023-12-26 22:57:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19355.4). Total num frames: 533651456. Throughput: 0: 9716.8, 1: 9821.5. Samples: 533664620. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:57:41,062][104569] Avg episode reward: [(0, '9264.044'), (1, '9353.231')] [2023-12-26 22:57:41,090][105620] Updated weights for policy 1, policy_version 1042579 (0.0009) [2023-12-26 22:57:41,150][105620] Updated weights for policy 1, policy_version 1042589 (0.0009) [2023-12-26 22:57:41,208][105692] Updated weights for policy 0, policy_version 1041737 (0.0009) [2023-12-26 22:57:41,222][105620] Updated weights for policy 1, policy_version 1042599 (0.0006) [2023-12-26 22:57:41,269][105692] Updated weights for policy 0, policy_version 1041747 (0.0008) [2023-12-26 22:57:41,325][105692] Updated weights for policy 0, policy_version 1041757 (0.0009) [2023-12-26 22:57:41,390][105692] Updated weights for policy 0, policy_version 1041767 (0.0008) [2023-12-26 22:57:41,934][105620] Updated weights for policy 1, policy_version 1042609 (0.0009) [2023-12-26 22:57:42,002][105620] Updated weights for policy 1, policy_version 1042619 (0.0009) [2023-12-26 22:57:42,058][105620] Updated weights for policy 1, policy_version 1042629 (0.0009) [2023-12-26 22:57:42,173][105692] Updated weights for policy 0, policy_version 1041777 (0.0009) [2023-12-26 22:57:42,227][105692] Updated weights for policy 0, policy_version 1041787 (0.0010) [2023-12-26 22:57:42,292][105692] Updated weights for policy 0, policy_version 1041797 (0.0009) [2023-12-26 22:57:42,746][105620] Updated weights for policy 1, policy_version 1042639 (0.0009) [2023-12-26 22:57:42,798][105620] Updated weights for policy 1, policy_version 1042649 (0.0009) [2023-12-26 22:57:42,852][105620] Updated weights for policy 1, policy_version 1042660 (0.0009) [2023-12-26 22:57:42,949][105692] Updated weights for policy 0, policy_version 1041807 (0.0006) [2023-12-26 22:57:43,007][105692] Updated weights for policy 0, policy_version 1041817 (0.0005) [2023-12-26 22:57:43,040][105585] KL-divergence is very high: 110.4301 [2023-12-26 22:57:43,046][105585] KL-divergence is very high: 106.5158 [2023-12-26 22:57:43,061][105692] Updated weights for policy 0, policy_version 1041827 (0.0005) [2023-12-26 22:57:43,611][105620] Updated weights for policy 1, policy_version 1042670 (0.0007) [2023-12-26 22:57:43,668][105620] Updated weights for policy 1, policy_version 1042680 (0.0006) [2023-12-26 22:57:43,679][105692] Updated weights for policy 0, policy_version 1041837 (0.0005) [2023-12-26 22:57:43,724][105620] Updated weights for policy 1, policy_version 1042690 (0.0008) [2023-12-26 22:57:43,732][105692] Updated weights for policy 0, policy_version 1041847 (0.0005) [2023-12-26 22:57:43,785][105692] Updated weights for policy 0, policy_version 1041857 (0.0006) [2023-12-26 22:57:44,427][105620] Updated weights for policy 1, policy_version 1042700 (0.0009) [2023-12-26 22:57:44,447][105692] Updated weights for policy 0, policy_version 1041867 (0.0007) [2023-12-26 22:57:44,491][105620] Updated weights for policy 1, policy_version 1042710 (0.0006) [2023-12-26 22:57:44,501][105692] Updated weights for policy 0, policy_version 1041877 (0.0010) [2023-12-26 22:57:44,557][105620] Updated weights for policy 1, policy_version 1042720 (0.0005) [2023-12-26 22:57:44,564][105692] Updated weights for policy 0, policy_version 1041887 (0.0006) [2023-12-26 22:57:45,241][105620] Updated weights for policy 1, policy_version 1042730 (0.0006) [2023-12-26 22:57:45,247][105692] Updated weights for policy 0, policy_version 1041897 (0.0007) [2023-12-26 22:57:45,295][105620] Updated weights for policy 1, policy_version 1042740 (0.0009) [2023-12-26 22:57:45,310][105692] Updated weights for policy 0, policy_version 1041907 (0.0011) [2023-12-26 22:57:45,349][105620] Updated weights for policy 1, policy_version 1042750 (0.0006) [2023-12-26 22:57:45,360][105692] Updated weights for policy 0, policy_version 1041917 (0.0011) [2023-12-26 22:57:45,403][105620] Updated weights for policy 1, policy_version 1042760 (0.0005) [2023-12-26 22:57:45,409][105692] Updated weights for policy 0, policy_version 1041927 (0.0011) [2023-12-26 22:57:46,007][105692] Updated weights for policy 0, policy_version 1041937 (0.0006) [2023-12-26 22:57:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 533749760. Throughput: 0: 9637.3, 1: 9879.5. Samples: 533722480. Policy #0 lag: (min: 11.0, avg: 14.9, max: 43.0) [2023-12-26 22:57:46,063][104569] Avg episode reward: [(0, '8644.173'), (1, '9352.900')] [2023-12-26 22:57:46,064][105692] Updated weights for policy 0, policy_version 1041947 (0.0007) [2023-12-26 22:57:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001042760_266977280.pth... [2023-12-26 22:57:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001041608_266682368.pth [2023-12-26 22:57:46,122][105692] Updated weights for policy 0, policy_version 1041957 (0.0008) [2023-12-26 22:57:46,143][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001041960_266780672.pth... [2023-12-26 22:57:46,147][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001040808_266485760.pth [2023-12-26 22:57:46,263][105620] Updated weights for policy 1, policy_version 1042770 (0.0007) [2023-12-26 22:57:46,311][105620] Updated weights for policy 1, policy_version 1042780 (0.0008) [2023-12-26 22:57:46,366][105620] Updated weights for policy 1, policy_version 1042790 (0.0008) [2023-12-26 22:57:46,814][105692] Updated weights for policy 0, policy_version 1041967 (0.0008) [2023-12-26 22:57:46,877][105692] Updated weights for policy 0, policy_version 1041977 (0.0008) [2023-12-26 22:57:46,931][105692] Updated weights for policy 0, policy_version 1041987 (0.0009) [2023-12-26 22:57:47,136][105620] Updated weights for policy 1, policy_version 1042800 (0.0010) [2023-12-26 22:57:47,194][105620] Updated weights for policy 1, policy_version 1042810 (0.0010) [2023-12-26 22:57:47,252][105620] Updated weights for policy 1, policy_version 1042820 (0.0010) [2023-12-26 22:57:47,622][105692] Updated weights for policy 0, policy_version 1041997 (0.0008) [2023-12-26 22:57:47,673][105692] Updated weights for policy 0, policy_version 1042007 (0.0007) [2023-12-26 22:57:47,743][105692] Updated weights for policy 0, policy_version 1042017 (0.0008) [2023-12-26 22:57:47,967][105620] Updated weights for policy 1, policy_version 1042830 (0.0010) [2023-12-26 22:57:48,016][105620] Updated weights for policy 1, policy_version 1042840 (0.0009) [2023-12-26 22:57:48,073][105620] Updated weights for policy 1, policy_version 1042850 (0.0009) [2023-12-26 22:57:48,519][105692] Updated weights for policy 0, policy_version 1042027 (0.0008) [2023-12-26 22:57:48,574][105692] Updated weights for policy 0, policy_version 1042037 (0.0009) [2023-12-26 22:57:48,630][105692] Updated weights for policy 0, policy_version 1042047 (0.0009) [2023-12-26 22:57:48,803][105620] Updated weights for policy 1, policy_version 1042860 (0.0007) [2023-12-26 22:57:48,858][105620] Updated weights for policy 1, policy_version 1042870 (0.0010) [2023-12-26 22:57:48,916][105620] Updated weights for policy 1, policy_version 1042881 (0.0010) [2023-12-26 22:57:49,291][105692] Updated weights for policy 0, policy_version 1042057 (0.0009) [2023-12-26 22:57:49,355][105692] Updated weights for policy 0, policy_version 1042067 (0.0008) [2023-12-26 22:57:49,420][105692] Updated weights for policy 0, policy_version 1042077 (0.0006) [2023-12-26 22:57:49,467][105585] KL-divergence is very high: 125.9653 [2023-12-26 22:57:49,487][105692] Updated weights for policy 0, policy_version 1042087 (0.0006) [2023-12-26 22:57:49,638][105620] Updated weights for policy 1, policy_version 1042891 (0.0006) [2023-12-26 22:57:49,686][105620] Updated weights for policy 1, policy_version 1042901 (0.0005) [2023-12-26 22:57:49,746][105620] Updated weights for policy 1, policy_version 1042911 (0.0008) [2023-12-26 22:57:50,111][105692] Updated weights for policy 0, policy_version 1042097 (0.0006) [2023-12-26 22:57:50,178][105692] Updated weights for policy 0, policy_version 1042107 (0.0006) [2023-12-26 22:57:50,246][105692] Updated weights for policy 0, policy_version 1042117 (0.0007) [2023-12-26 22:57:50,342][105620] Updated weights for policy 1, policy_version 1042921 (0.0007) [2023-12-26 22:57:50,412][105620] Updated weights for policy 1, policy_version 1042931 (0.0005) [2023-12-26 22:57:50,476][105620] Updated weights for policy 1, policy_version 1042941 (0.0006) [2023-12-26 22:57:50,536][105620] Updated weights for policy 1, policy_version 1042951 (0.0008) [2023-12-26 22:57:50,911][105692] Updated weights for policy 0, policy_version 1042127 (0.0006) [2023-12-26 22:57:50,978][105692] Updated weights for policy 0, policy_version 1042137 (0.0006) [2023-12-26 22:57:51,044][105692] Updated weights for policy 0, policy_version 1042147 (0.0009) [2023-12-26 22:57:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 533848064. Throughput: 0: 9793.9, 1: 9802.0. Samples: 533840996. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:57:51,062][104569] Avg episode reward: [(0, '8815.875'), (1, '9352.871')] [2023-12-26 22:57:51,239][105620] Updated weights for policy 1, policy_version 1042961 (0.0007) [2023-12-26 22:57:51,302][105620] Updated weights for policy 1, policy_version 1042971 (0.0009) [2023-12-26 22:57:51,371][105620] Updated weights for policy 1, policy_version 1042981 (0.0009) [2023-12-26 22:57:51,764][105692] Updated weights for policy 0, policy_version 1042157 (0.0008) [2023-12-26 22:57:51,823][105692] Updated weights for policy 0, policy_version 1042167 (0.0005) [2023-12-26 22:57:51,883][105692] Updated weights for policy 0, policy_version 1042177 (0.0005) [2023-12-26 22:57:52,126][105620] Updated weights for policy 1, policy_version 1042991 (0.0007) [2023-12-26 22:57:52,179][105620] Updated weights for policy 1, policy_version 1043001 (0.0008) [2023-12-26 22:57:52,226][105620] Updated weights for policy 1, policy_version 1043011 (0.0008) [2023-12-26 22:57:52,576][105692] Updated weights for policy 0, policy_version 1042187 (0.0007) [2023-12-26 22:57:52,636][105692] Updated weights for policy 0, policy_version 1042197 (0.0008) [2023-12-26 22:57:52,691][105692] Updated weights for policy 0, policy_version 1042207 (0.0007) [2023-12-26 22:57:52,994][105620] Updated weights for policy 1, policy_version 1043021 (0.0011) [2023-12-26 22:57:53,056][105620] Updated weights for policy 1, policy_version 1043031 (0.0011) [2023-12-26 22:57:53,111][105620] Updated weights for policy 1, policy_version 1043041 (0.0010) [2023-12-26 22:57:53,372][105692] Updated weights for policy 0, policy_version 1042217 (0.0008) [2023-12-26 22:57:53,423][105692] Updated weights for policy 0, policy_version 1042227 (0.0008) [2023-12-26 22:57:53,470][105692] Updated weights for policy 0, policy_version 1042237 (0.0007) [2023-12-26 22:57:53,522][105692] Updated weights for policy 0, policy_version 1042247 (0.0008) [2023-12-26 22:57:53,840][105620] Updated weights for policy 1, policy_version 1043051 (0.0010) [2023-12-26 22:57:53,891][105620] Updated weights for policy 1, policy_version 1043061 (0.0009) [2023-12-26 22:57:53,939][105620] Updated weights for policy 1, policy_version 1043071 (0.0007) [2023-12-26 22:57:54,319][105692] Updated weights for policy 0, policy_version 1042257 (0.0009) [2023-12-26 22:57:54,371][105692] Updated weights for policy 0, policy_version 1042268 (0.0009) [2023-12-26 22:57:54,425][105692] Updated weights for policy 0, policy_version 1042278 (0.0009) [2023-12-26 22:57:54,647][105620] Updated weights for policy 1, policy_version 1043081 (0.0006) [2023-12-26 22:57:54,706][105620] Updated weights for policy 1, policy_version 1043092 (0.0010) [2023-12-26 22:57:54,763][105620] Updated weights for policy 1, policy_version 1043102 (0.0010) [2023-12-26 22:57:54,824][105620] Updated weights for policy 1, policy_version 1043112 (0.0009) [2023-12-26 22:57:55,077][105692] Updated weights for policy 0, policy_version 1042288 (0.0006) [2023-12-26 22:57:55,138][105692] Updated weights for policy 0, policy_version 1042298 (0.0005) [2023-12-26 22:57:55,199][105692] Updated weights for policy 0, policy_version 1042308 (0.0005) [2023-12-26 22:57:55,626][105620] Updated weights for policy 1, policy_version 1043122 (0.0009) [2023-12-26 22:57:55,684][105620] Updated weights for policy 1, policy_version 1043132 (0.0009) [2023-12-26 22:57:55,732][105620] Updated weights for policy 1, policy_version 1043142 (0.0009) [2023-12-26 22:57:55,818][105692] Updated weights for policy 0, policy_version 1042318 (0.0008) [2023-12-26 22:57:55,869][105692] Updated weights for policy 0, policy_version 1042328 (0.0009) [2023-12-26 22:57:55,916][105692] Updated weights for policy 0, policy_version 1042338 (0.0009) [2023-12-26 22:57:56,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 533954560. Throughput: 0: 9809.0, 1: 9835.6. Samples: 533959040. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:57:56,063][104569] Avg episode reward: [(0, '8991.675'), (1, '9077.344')] [2023-12-26 22:57:56,517][105620] Updated weights for policy 1, policy_version 1043152 (0.0009) [2023-12-26 22:57:56,563][105620] Updated weights for policy 1, policy_version 1043162 (0.0008) [2023-12-26 22:57:56,615][105620] Updated weights for policy 1, policy_version 1043172 (0.0008) [2023-12-26 22:57:56,625][105692] Updated weights for policy 0, policy_version 1042348 (0.0008) [2023-12-26 22:57:56,670][105692] Updated weights for policy 0, policy_version 1042358 (0.0008) [2023-12-26 22:57:56,716][105692] Updated weights for policy 0, policy_version 1042368 (0.0008) [2023-12-26 22:57:57,237][105620] Updated weights for policy 1, policy_version 1043182 (0.0006) [2023-12-26 22:57:57,282][105620] Updated weights for policy 1, policy_version 1043192 (0.0005) [2023-12-26 22:57:57,337][105620] Updated weights for policy 1, policy_version 1043202 (0.0007) [2023-12-26 22:57:57,495][105692] Updated weights for policy 0, policy_version 1042378 (0.0008) [2023-12-26 22:57:57,546][105692] Updated weights for policy 0, policy_version 1042388 (0.0010) [2023-12-26 22:57:57,593][105692] Updated weights for policy 0, policy_version 1042398 (0.0010) [2023-12-26 22:57:57,640][105692] Updated weights for policy 0, policy_version 1042408 (0.0010) [2023-12-26 22:57:57,932][105620] Updated weights for policy 1, policy_version 1043212 (0.0008) [2023-12-26 22:57:57,979][105620] Updated weights for policy 1, policy_version 1043222 (0.0005) [2023-12-26 22:57:58,026][105620] Updated weights for policy 1, policy_version 1043232 (0.0005) [2023-12-26 22:57:58,407][105692] Updated weights for policy 0, policy_version 1042418 (0.0007) [2023-12-26 22:57:58,448][105585] KL-divergence is very high: 117.8632 [2023-12-26 22:57:58,474][105692] Updated weights for policy 0, policy_version 1042428 (0.0008) [2023-12-26 22:57:58,500][105585] KL-divergence is very high: 118.9486 [2023-12-26 22:57:58,537][105692] Updated weights for policy 0, policy_version 1042438 (0.0008) [2023-12-26 22:57:58,786][105620] Updated weights for policy 1, policy_version 1043242 (0.0006) [2023-12-26 22:57:58,856][105620] Updated weights for policy 1, policy_version 1043252 (0.0010) [2023-12-26 22:57:58,934][105620] Updated weights for policy 1, policy_version 1043262 (0.0009) [2023-12-26 22:57:58,999][105620] Updated weights for policy 1, policy_version 1043272 (0.0008) [2023-12-26 22:57:59,357][105692] Updated weights for policy 0, policy_version 1042448 (0.0009) [2023-12-26 22:57:59,413][105692] Updated weights for policy 0, policy_version 1042458 (0.0008) [2023-12-26 22:57:59,460][105692] Updated weights for policy 0, policy_version 1042468 (0.0008) [2023-12-26 22:57:59,716][105620] Updated weights for policy 1, policy_version 1043282 (0.0005) [2023-12-26 22:57:59,762][105620] Updated weights for policy 1, policy_version 1043292 (0.0005) [2023-12-26 22:57:59,811][105620] Updated weights for policy 1, policy_version 1043302 (0.0006) [2023-12-26 22:58:00,158][105692] Updated weights for policy 0, policy_version 1042478 (0.0008) [2023-12-26 22:58:00,222][105692] Updated weights for policy 0, policy_version 1042488 (0.0008) [2023-12-26 22:58:00,281][105692] Updated weights for policy 0, policy_version 1042498 (0.0005) [2023-12-26 22:58:00,424][105620] Updated weights for policy 1, policy_version 1043312 (0.0009) [2023-12-26 22:58:00,474][105620] Updated weights for policy 1, policy_version 1043322 (0.0006) [2023-12-26 22:58:00,534][105620] Updated weights for policy 1, policy_version 1043332 (0.0005) [2023-12-26 22:58:01,062][105692] Updated weights for policy 0, policy_version 1042508 (0.0008) [2023-12-26 22:58:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 534044672. Throughput: 0: 9797.0, 1: 9867.2. Samples: 534018080. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:58:01,063][104569] Avg episode reward: [(0, '9082.369'), (1, '8984.867')] [2023-12-26 22:58:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001043336_267124736.pth... [2023-12-26 22:58:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001042216_266838016.pth [2023-12-26 22:58:01,126][105692] Updated weights for policy 0, policy_version 1042518 (0.0011) [2023-12-26 22:58:01,187][105692] Updated weights for policy 0, policy_version 1042528 (0.0011) [2023-12-26 22:58:01,205][105620] Updated weights for policy 1, policy_version 1043342 (0.0006) [2023-12-26 22:58:01,229][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001042536_266928128.pth... [2023-12-26 22:58:01,233][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001041352_266625024.pth [2023-12-26 22:58:01,264][105620] Updated weights for policy 1, policy_version 1043352 (0.0007) [2023-12-26 22:58:01,333][105620] Updated weights for policy 1, policy_version 1043362 (0.0007) [2023-12-26 22:58:01,957][105692] Updated weights for policy 0, policy_version 1042538 (0.0011) [2023-12-26 22:58:02,024][105692] Updated weights for policy 0, policy_version 1042548 (0.0007) [2023-12-26 22:58:02,075][105692] Updated weights for policy 0, policy_version 1042558 (0.0009) [2023-12-26 22:58:02,136][105620] Updated weights for policy 1, policy_version 1043372 (0.0009) [2023-12-26 22:58:02,138][105692] Updated weights for policy 0, policy_version 1042568 (0.0008) [2023-12-26 22:58:02,188][105620] Updated weights for policy 1, policy_version 1043382 (0.0006) [2023-12-26 22:58:02,240][105620] Updated weights for policy 1, policy_version 1043392 (0.0009) [2023-12-26 22:58:02,876][105692] Updated weights for policy 0, policy_version 1042578 (0.0008) [2023-12-26 22:58:02,902][105620] Updated weights for policy 1, policy_version 1043402 (0.0008) [2023-12-26 22:58:02,939][105692] Updated weights for policy 0, policy_version 1042588 (0.0005) [2023-12-26 22:58:02,960][105620] Updated weights for policy 1, policy_version 1043412 (0.0006) [2023-12-26 22:58:02,990][105692] Updated weights for policy 0, policy_version 1042598 (0.0005) [2023-12-26 22:58:03,008][105620] Updated weights for policy 1, policy_version 1043422 (0.0008) [2023-12-26 22:58:03,054][105620] Updated weights for policy 1, policy_version 1043432 (0.0009) [2023-12-26 22:58:03,559][105692] Updated weights for policy 0, policy_version 1042608 (0.0005) [2023-12-26 22:58:03,610][105692] Updated weights for policy 0, policy_version 1042618 (0.0005) [2023-12-26 22:58:03,662][105692] Updated weights for policy 0, policy_version 1042628 (0.0005) [2023-12-26 22:58:03,915][105620] Updated weights for policy 1, policy_version 1043442 (0.0008) [2023-12-26 22:58:03,972][105620] Updated weights for policy 1, policy_version 1043452 (0.0008) [2023-12-26 22:58:04,037][105620] Updated weights for policy 1, policy_version 1043462 (0.0010) [2023-12-26 22:58:04,274][105692] Updated weights for policy 0, policy_version 1042638 (0.0007) [2023-12-26 22:58:04,329][105692] Updated weights for policy 0, policy_version 1042648 (0.0009) [2023-12-26 22:58:04,385][105692] Updated weights for policy 0, policy_version 1042658 (0.0009) [2023-12-26 22:58:04,774][105620] Updated weights for policy 1, policy_version 1043472 (0.0006) [2023-12-26 22:58:04,843][105620] Updated weights for policy 1, policy_version 1043482 (0.0005) [2023-12-26 22:58:04,907][105620] Updated weights for policy 1, policy_version 1043492 (0.0006) [2023-12-26 22:58:05,153][105692] Updated weights for policy 0, policy_version 1042668 (0.0010) [2023-12-26 22:58:05,207][105692] Updated weights for policy 0, policy_version 1042678 (0.0009) [2023-12-26 22:58:05,265][105692] Updated weights for policy 0, policy_version 1042688 (0.0010) [2023-12-26 22:58:05,497][105620] Updated weights for policy 1, policy_version 1043502 (0.0007) [2023-12-26 22:58:05,546][105620] Updated weights for policy 1, policy_version 1043512 (0.0008) [2023-12-26 22:58:05,597][105620] Updated weights for policy 1, policy_version 1043522 (0.0007) [2023-12-26 22:58:05,969][105692] Updated weights for policy 0, policy_version 1042698 (0.0010) [2023-12-26 22:58:06,041][105692] Updated weights for policy 0, policy_version 1042708 (0.0011) [2023-12-26 22:58:06,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 534142976. Throughput: 0: 9817.6, 1: 9807.4. Samples: 534135020. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:58:06,062][104569] Avg episode reward: [(0, '9084.635'), (1, '9075.644')] [2023-12-26 22:58:06,100][105692] Updated weights for policy 0, policy_version 1042718 (0.0010) [2023-12-26 22:58:06,164][105692] Updated weights for policy 0, policy_version 1042728 (0.0010) [2023-12-26 22:58:06,353][105620] Updated weights for policy 1, policy_version 1043532 (0.0008) [2023-12-26 22:58:06,414][105620] Updated weights for policy 1, policy_version 1043542 (0.0005) [2023-12-26 22:58:06,478][105620] Updated weights for policy 1, policy_version 1043552 (0.0006) [2023-12-26 22:58:06,849][105692] Updated weights for policy 0, policy_version 1042738 (0.0010) [2023-12-26 22:58:06,904][105692] Updated weights for policy 0, policy_version 1042748 (0.0009) [2023-12-26 22:58:06,966][105692] Updated weights for policy 0, policy_version 1042758 (0.0009) [2023-12-26 22:58:07,204][105620] Updated weights for policy 1, policy_version 1043562 (0.0008) [2023-12-26 22:58:07,252][105620] Updated weights for policy 1, policy_version 1043572 (0.0009) [2023-12-26 22:58:07,303][105620] Updated weights for policy 1, policy_version 1043582 (0.0009) [2023-12-26 22:58:07,361][105620] Updated weights for policy 1, policy_version 1043592 (0.0009) [2023-12-26 22:58:07,652][105692] Updated weights for policy 0, policy_version 1042768 (0.0006) [2023-12-26 22:58:07,697][105692] Updated weights for policy 0, policy_version 1042778 (0.0005) [2023-12-26 22:58:07,747][105692] Updated weights for policy 0, policy_version 1042788 (0.0007) [2023-12-26 22:58:08,226][105620] Updated weights for policy 1, policy_version 1043602 (0.0005) [2023-12-26 22:58:08,278][105620] Updated weights for policy 1, policy_version 1043612 (0.0009) [2023-12-26 22:58:08,310][105692] Updated weights for policy 0, policy_version 1042798 (0.0007) [2023-12-26 22:58:08,339][105620] Updated weights for policy 1, policy_version 1043622 (0.0009) [2023-12-26 22:58:08,370][105692] Updated weights for policy 0, policy_version 1042808 (0.0007) [2023-12-26 22:58:08,428][105692] Updated weights for policy 0, policy_version 1042818 (0.0007) [2023-12-26 22:58:09,049][105692] Updated weights for policy 0, policy_version 1042828 (0.0008) [2023-12-26 22:58:09,111][105692] Updated weights for policy 0, policy_version 1042838 (0.0007) [2023-12-26 22:58:09,148][105620] Updated weights for policy 1, policy_version 1043632 (0.0006) [2023-12-26 22:58:09,165][105692] Updated weights for policy 0, policy_version 1042848 (0.0007) [2023-12-26 22:58:09,216][105620] Updated weights for policy 1, policy_version 1043642 (0.0007) [2023-12-26 22:58:09,285][105620] Updated weights for policy 1, policy_version 1043652 (0.0009) [2023-12-26 22:58:09,908][105692] Updated weights for policy 0, policy_version 1042858 (0.0007) [2023-12-26 22:58:09,978][105692] Updated weights for policy 0, policy_version 1042868 (0.0009) [2023-12-26 22:58:10,046][105692] Updated weights for policy 0, policy_version 1042878 (0.0007) [2023-12-26 22:58:10,056][105620] Updated weights for policy 1, policy_version 1043662 (0.0007) [2023-12-26 22:58:10,111][105692] Updated weights for policy 0, policy_version 1042888 (0.0007) [2023-12-26 22:58:10,124][105620] Updated weights for policy 1, policy_version 1043672 (0.0010) [2023-12-26 22:58:10,192][105620] Updated weights for policy 1, policy_version 1043682 (0.0010) [2023-12-26 22:58:10,814][105620] Updated weights for policy 1, policy_version 1043692 (0.0008) [2023-12-26 22:58:10,876][105620] Updated weights for policy 1, policy_version 1043702 (0.0006) [2023-12-26 22:58:10,939][105620] Updated weights for policy 1, policy_version 1043712 (0.0006) [2023-12-26 22:58:10,953][105692] Updated weights for policy 0, policy_version 1042898 (0.0008) [2023-12-26 22:58:11,020][105692] Updated weights for policy 0, policy_version 1042908 (0.0009) [2023-12-26 22:58:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 534241280. Throughput: 0: 9937.1, 1: 9676.3. Samples: 534251824. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:58:11,063][104569] Avg episode reward: [(0, '8906.110'), (1, '9075.509')] [2023-12-26 22:58:11,088][105692] Updated weights for policy 0, policy_version 1042918 (0.0009) [2023-12-26 22:58:11,600][105620] Updated weights for policy 1, policy_version 1043722 (0.0007) [2023-12-26 22:58:11,672][105620] Updated weights for policy 1, policy_version 1043732 (0.0009) [2023-12-26 22:58:11,738][105620] Updated weights for policy 1, policy_version 1043742 (0.0011) [2023-12-26 22:58:11,795][105620] Updated weights for policy 1, policy_version 1043752 (0.0009) [2023-12-26 22:58:11,851][105692] Updated weights for policy 0, policy_version 1042928 (0.0008) [2023-12-26 22:58:11,913][105692] Updated weights for policy 0, policy_version 1042938 (0.0007) [2023-12-26 22:58:11,969][105692] Updated weights for policy 0, policy_version 1042948 (0.0009) [2023-12-26 22:58:12,528][105620] Updated weights for policy 1, policy_version 1043762 (0.0005) [2023-12-26 22:58:12,584][105620] Updated weights for policy 1, policy_version 1043772 (0.0009) [2023-12-26 22:58:12,648][105620] Updated weights for policy 1, policy_version 1043782 (0.0007) [2023-12-26 22:58:12,732][105692] Updated weights for policy 0, policy_version 1042958 (0.0009) [2023-12-26 22:58:12,787][105692] Updated weights for policy 0, policy_version 1042968 (0.0008) [2023-12-26 22:58:12,846][105692] Updated weights for policy 0, policy_version 1042978 (0.0008) [2023-12-26 22:58:13,398][105620] Updated weights for policy 1, policy_version 1043792 (0.0009) [2023-12-26 22:58:13,453][105620] Updated weights for policy 1, policy_version 1043802 (0.0009) [2023-12-26 22:58:13,454][105692] Updated weights for policy 0, policy_version 1042988 (0.0005) [2023-12-26 22:58:13,510][105620] Updated weights for policy 1, policy_version 1043812 (0.0009) [2023-12-26 22:58:13,515][105692] Updated weights for policy 0, policy_version 1042998 (0.0006) [2023-12-26 22:58:13,574][105692] Updated weights for policy 0, policy_version 1043008 (0.0005) [2023-12-26 22:58:14,114][105692] Updated weights for policy 0, policy_version 1043018 (0.0005) [2023-12-26 22:58:14,179][105692] Updated weights for policy 0, policy_version 1043028 (0.0007) [2023-12-26 22:58:14,239][105692] Updated weights for policy 0, policy_version 1043038 (0.0009) [2023-12-26 22:58:14,288][105692] Updated weights for policy 0, policy_version 1043048 (0.0009) [2023-12-26 22:58:14,360][105620] Updated weights for policy 1, policy_version 1043822 (0.0007) [2023-12-26 22:58:14,417][105620] Updated weights for policy 1, policy_version 1043832 (0.0009) [2023-12-26 22:58:14,472][105620] Updated weights for policy 1, policy_version 1043842 (0.0009) [2023-12-26 22:58:14,949][105692] Updated weights for policy 0, policy_version 1043058 (0.0010) [2023-12-26 22:58:15,003][105692] Updated weights for policy 0, policy_version 1043068 (0.0011) [2023-12-26 22:58:15,060][105692] Updated weights for policy 0, policy_version 1043078 (0.0011) [2023-12-26 22:58:15,292][105620] Updated weights for policy 1, policy_version 1043852 (0.0010) [2023-12-26 22:58:15,356][105620] Updated weights for policy 1, policy_version 1043862 (0.0011) [2023-12-26 22:58:15,420][105620] Updated weights for policy 1, policy_version 1043872 (0.0011) [2023-12-26 22:58:15,712][105692] Updated weights for policy 0, policy_version 1043088 (0.0005) [2023-12-26 22:58:15,762][105692] Updated weights for policy 0, policy_version 1043098 (0.0005) [2023-12-26 22:58:15,786][105585] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000008 [2023-12-26 22:58:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 534339584. Throughput: 0: 9890.4, 1: 9540.2. Samples: 534308092. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:58:16,063][104569] Avg episode reward: [(0, '8994.410'), (1, '9259.313')] [2023-12-26 22:58:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001043104_267075584.pth... [2023-12-26 22:58:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001043880_267264000.pth... [2023-12-26 22:58:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001041960_266780672.pth [2023-12-26 22:58:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001042760_266977280.pth [2023-12-26 22:58:16,099][105620] Updated weights for policy 1, policy_version 1043882 (0.0009) [2023-12-26 22:58:16,151][105620] Updated weights for policy 1, policy_version 1043892 (0.0010) [2023-12-26 22:58:16,206][105620] Updated weights for policy 1, policy_version 1043902 (0.0007) [2023-12-26 22:58:16,259][105620] Updated weights for policy 1, policy_version 1043912 (0.0005) [2023-12-26 22:58:16,533][105692] Updated weights for policy 0, policy_version 1043108 (0.0007) [2023-12-26 22:58:16,591][105692] Updated weights for policy 0, policy_version 1043118 (0.0010) [2023-12-26 22:58:16,650][105692] Updated weights for policy 0, policy_version 1043128 (0.0010) [2023-12-26 22:58:16,791][105620] Updated weights for policy 1, policy_version 1043922 (0.0010) [2023-12-26 22:58:16,839][105620] Updated weights for policy 1, policy_version 1043932 (0.0010) [2023-12-26 22:58:16,886][105620] Updated weights for policy 1, policy_version 1043942 (0.0010) [2023-12-26 22:58:17,380][105692] Updated weights for policy 0, policy_version 1043138 (0.0010) [2023-12-26 22:58:17,433][105692] Updated weights for policy 0, policy_version 1043148 (0.0010) [2023-12-26 22:58:17,484][105692] Updated weights for policy 0, policy_version 1043159 (0.0009) [2023-12-26 22:58:17,538][105620] Updated weights for policy 1, policy_version 1043952 (0.0008) [2023-12-26 22:58:17,604][105620] Updated weights for policy 1, policy_version 1043962 (0.0008) [2023-12-26 22:58:17,667][105620] Updated weights for policy 1, policy_version 1043972 (0.0008) [2023-12-26 22:58:18,258][105692] Updated weights for policy 0, policy_version 1043169 (0.0007) [2023-12-26 22:58:18,306][105692] Updated weights for policy 0, policy_version 1043179 (0.0008) [2023-12-26 22:58:18,360][105620] Updated weights for policy 1, policy_version 1043982 (0.0008) [2023-12-26 22:58:18,365][105692] Updated weights for policy 0, policy_version 1043189 (0.0008) [2023-12-26 22:58:18,415][105620] Updated weights for policy 1, policy_version 1043992 (0.0007) [2023-12-26 22:58:18,429][105692] Updated weights for policy 0, policy_version 1043199 (0.0009) [2023-12-26 22:58:18,471][105620] Updated weights for policy 1, policy_version 1044002 (0.0010) [2023-12-26 22:58:19,197][105692] Updated weights for policy 0, policy_version 1043209 (0.0008) [2023-12-26 22:58:19,244][105620] Updated weights for policy 1, policy_version 1044012 (0.0010) [2023-12-26 22:58:19,258][105692] Updated weights for policy 0, policy_version 1043219 (0.0009) [2023-12-26 22:58:19,311][105620] Updated weights for policy 1, policy_version 1044022 (0.0008) [2023-12-26 22:58:19,327][105692] Updated weights for policy 0, policy_version 1043229 (0.0009) [2023-12-26 22:58:19,383][105620] Updated weights for policy 1, policy_version 1044032 (0.0009) [2023-12-26 22:58:20,091][105692] Updated weights for policy 0, policy_version 1043239 (0.0009) [2023-12-26 22:58:20,147][105692] Updated weights for policy 0, policy_version 1043249 (0.0008) [2023-12-26 22:58:20,159][105620] Updated weights for policy 1, policy_version 1044042 (0.0010) [2023-12-26 22:58:20,209][105692] Updated weights for policy 0, policy_version 1043259 (0.0010) [2023-12-26 22:58:20,219][105620] Updated weights for policy 1, policy_version 1044052 (0.0008) [2023-12-26 22:58:20,272][105620] Updated weights for policy 1, policy_version 1044062 (0.0008) [2023-12-26 22:58:20,320][105620] Updated weights for policy 1, policy_version 1044072 (0.0009) [2023-12-26 22:58:20,976][105692] Updated weights for policy 0, policy_version 1043269 (0.0008) [2023-12-26 22:58:21,036][105692] Updated weights for policy 0, policy_version 1043279 (0.0009) [2023-12-26 22:58:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 534429696. Throughput: 0: 9885.6, 1: 9581.9. Samples: 534425968. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:58:21,063][104569] Avg episode reward: [(0, '9084.345'), (1, '9259.527')] [2023-12-26 22:58:21,103][105692] Updated weights for policy 0, policy_version 1043289 (0.0008) [2023-12-26 22:58:21,125][105620] Updated weights for policy 1, policy_version 1044082 (0.0008) [2023-12-26 22:58:21,191][105620] Updated weights for policy 1, policy_version 1044092 (0.0008) [2023-12-26 22:58:21,263][105620] Updated weights for policy 1, policy_version 1044102 (0.0009) [2023-12-26 22:58:21,922][105692] Updated weights for policy 0, policy_version 1043299 (0.0007) [2023-12-26 22:58:21,985][105692] Updated weights for policy 0, policy_version 1043309 (0.0008) [2023-12-26 22:58:22,022][105620] Updated weights for policy 1, policy_version 1044112 (0.0008) [2023-12-26 22:58:22,052][105692] Updated weights for policy 0, policy_version 1043319 (0.0008) [2023-12-26 22:58:22,083][105620] Updated weights for policy 1, policy_version 1044122 (0.0007) [2023-12-26 22:58:22,153][105620] Updated weights for policy 1, policy_version 1044132 (0.0006) [2023-12-26 22:58:22,844][105620] Updated weights for policy 1, policy_version 1044142 (0.0007) [2023-12-26 22:58:22,866][105692] Updated weights for policy 0, policy_version 1043329 (0.0008) [2023-12-26 22:58:22,900][105620] Updated weights for policy 1, policy_version 1044152 (0.0010) [2023-12-26 22:58:22,918][105692] Updated weights for policy 0, policy_version 1043339 (0.0010) [2023-12-26 22:58:22,953][105620] Updated weights for policy 1, policy_version 1044162 (0.0010) [2023-12-26 22:58:22,971][105692] Updated weights for policy 0, policy_version 1043349 (0.0011) [2023-12-26 22:58:23,023][105692] Updated weights for policy 0, policy_version 1043359 (0.0011) [2023-12-26 22:58:23,704][105620] Updated weights for policy 1, policy_version 1044172 (0.0011) [2023-12-26 22:58:23,762][105620] Updated weights for policy 1, policy_version 1044182 (0.0010) [2023-12-26 22:58:23,765][105692] Updated weights for policy 0, policy_version 1043369 (0.0010) [2023-12-26 22:58:23,823][105692] Updated weights for policy 0, policy_version 1043379 (0.0007) [2023-12-26 22:58:23,824][105620] Updated weights for policy 1, policy_version 1044192 (0.0010) [2023-12-26 22:58:23,870][105692] Updated weights for policy 0, policy_version 1043389 (0.0005) [2023-12-26 22:58:24,427][105692] Updated weights for policy 0, policy_version 1043399 (0.0007) [2023-12-26 22:58:24,488][105692] Updated weights for policy 0, policy_version 1043409 (0.0010) [2023-12-26 22:58:24,545][105692] Updated weights for policy 0, policy_version 1043419 (0.0010) [2023-12-26 22:58:24,562][105620] Updated weights for policy 1, policy_version 1044202 (0.0010) [2023-12-26 22:58:24,622][105620] Updated weights for policy 1, policy_version 1044212 (0.0008) [2023-12-26 22:58:24,686][105620] Updated weights for policy 1, policy_version 1044222 (0.0008) [2023-12-26 22:58:24,740][105620] Updated weights for policy 1, policy_version 1044232 (0.0007) [2023-12-26 22:58:25,270][105692] Updated weights for policy 0, policy_version 1043429 (0.0011) [2023-12-26 22:58:25,318][105692] Updated weights for policy 0, policy_version 1043439 (0.0010) [2023-12-26 22:58:25,372][105692] Updated weights for policy 0, policy_version 1043449 (0.0010) [2023-12-26 22:58:25,387][105620] Updated weights for policy 1, policy_version 1044242 (0.0006) [2023-12-26 22:58:25,433][105620] Updated weights for policy 1, policy_version 1044252 (0.0007) [2023-12-26 22:58:25,481][105620] Updated weights for policy 1, policy_version 1044262 (0.0008) [2023-12-26 22:58:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.2, 300 sec: 19355.3). Total num frames: 534528000. Throughput: 0: 9862.3, 1: 9621.3. Samples: 534541384. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:58:26,062][104569] Avg episode reward: [(0, '8994.892'), (1, '8718.938')] [2023-12-26 22:58:26,072][105692] Updated weights for policy 0, policy_version 1043459 (0.0010) [2023-12-26 22:58:26,093][105620] Updated weights for policy 1, policy_version 1044272 (0.0006) [2023-12-26 22:58:26,134][105692] Updated weights for policy 0, policy_version 1043469 (0.0011) [2023-12-26 22:58:26,149][105620] Updated weights for policy 1, policy_version 1044282 (0.0006) [2023-12-26 22:58:26,189][105692] Updated weights for policy 0, policy_version 1043479 (0.0010) [2023-12-26 22:58:26,195][105620] Updated weights for policy 1, policy_version 1044292 (0.0005) [2023-12-26 22:58:26,859][105692] Updated weights for policy 0, policy_version 1043489 (0.0010) [2023-12-26 22:58:26,890][105620] Updated weights for policy 1, policy_version 1044302 (0.0009) [2023-12-26 22:58:26,917][105692] Updated weights for policy 0, policy_version 1043499 (0.0007) [2023-12-26 22:58:26,939][105620] Updated weights for policy 1, policy_version 1044312 (0.0007) [2023-12-26 22:58:26,976][105692] Updated weights for policy 0, policy_version 1043509 (0.0006) [2023-12-26 22:58:26,998][105620] Updated weights for policy 1, policy_version 1044322 (0.0008) [2023-12-26 22:58:27,032][105692] Updated weights for policy 0, policy_version 1043519 (0.0005) [2023-12-26 22:58:27,652][105692] Updated weights for policy 0, policy_version 1043529 (0.0010) [2023-12-26 22:58:27,704][105692] Updated weights for policy 0, policy_version 1043539 (0.0010) [2023-12-26 22:58:27,752][105692] Updated weights for policy 0, policy_version 1043549 (0.0010) [2023-12-26 22:58:27,753][105620] Updated weights for policy 1, policy_version 1044332 (0.0009) [2023-12-26 22:58:27,802][105620] Updated weights for policy 1, policy_version 1044342 (0.0008) [2023-12-26 22:58:27,854][105620] Updated weights for policy 1, policy_version 1044352 (0.0006) [2023-12-26 22:58:28,503][105692] Updated weights for policy 0, policy_version 1043559 (0.0011) [2023-12-26 22:58:28,514][105620] Updated weights for policy 1, policy_version 1044362 (0.0008) [2023-12-26 22:58:28,564][105620] Updated weights for policy 1, policy_version 1044372 (0.0006) [2023-12-26 22:58:28,566][105692] Updated weights for policy 0, policy_version 1043569 (0.0010) [2023-12-26 22:58:28,621][105692] Updated weights for policy 0, policy_version 1043579 (0.0010) [2023-12-26 22:58:28,621][105620] Updated weights for policy 1, policy_version 1044382 (0.0007) [2023-12-26 22:58:28,682][105620] Updated weights for policy 1, policy_version 1044392 (0.0006) [2023-12-26 22:58:29,224][105620] Updated weights for policy 1, policy_version 1044402 (0.0006) [2023-12-26 22:58:29,258][105692] Updated weights for policy 0, policy_version 1043589 (0.0008) [2023-12-26 22:58:29,285][105620] Updated weights for policy 1, policy_version 1044412 (0.0007) [2023-12-26 22:58:29,314][105692] Updated weights for policy 0, policy_version 1043599 (0.0006) [2023-12-26 22:58:29,351][105620] Updated weights for policy 1, policy_version 1044422 (0.0008) [2023-12-26 22:58:29,381][105692] Updated weights for policy 0, policy_version 1043609 (0.0008) [2023-12-26 22:58:30,010][105692] Updated weights for policy 0, policy_version 1043619 (0.0008) [2023-12-26 22:58:30,068][105692] Updated weights for policy 0, policy_version 1043629 (0.0007) [2023-12-26 22:58:30,073][105620] Updated weights for policy 1, policy_version 1044432 (0.0010) [2023-12-26 22:58:30,129][105692] Updated weights for policy 0, policy_version 1043639 (0.0006) [2023-12-26 22:58:30,136][105620] Updated weights for policy 1, policy_version 1044442 (0.0011) [2023-12-26 22:58:30,191][105620] Updated weights for policy 1, policy_version 1044452 (0.0010) [2023-12-26 22:58:30,727][105692] Updated weights for policy 0, policy_version 1043649 (0.0006) [2023-12-26 22:58:30,772][105692] Updated weights for policy 0, policy_version 1043659 (0.0010) [2023-12-26 22:58:30,816][105692] Updated weights for policy 0, policy_version 1043669 (0.0010) [2023-12-26 22:58:30,817][105620] Updated weights for policy 1, policy_version 1044462 (0.0007) [2023-12-26 22:58:30,861][105692] Updated weights for policy 0, policy_version 1043679 (0.0010) [2023-12-26 22:58:30,876][105620] Updated weights for policy 1, policy_version 1044472 (0.0005) [2023-12-26 22:58:30,939][105620] Updated weights for policy 1, policy_version 1044482 (0.0007) [2023-12-26 22:58:31,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 534642688. Throughput: 0: 9890.6, 1: 9678.3. Samples: 534603080. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:58:31,063][104569] Avg episode reward: [(0, '8993.294'), (1, '8798.379')] [2023-12-26 22:58:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001043680_267223040.pth... [2023-12-26 22:58:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001044488_267419648.pth... [2023-12-26 22:58:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001042536_266928128.pth [2023-12-26 22:58:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001043336_267124736.pth [2023-12-26 22:58:31,594][105692] Updated weights for policy 0, policy_version 1043689 (0.0011) [2023-12-26 22:58:31,662][105692] Updated weights for policy 0, policy_version 1043699 (0.0010) [2023-12-26 22:58:31,665][105620] Updated weights for policy 1, policy_version 1044492 (0.0009) [2023-12-26 22:58:31,728][105692] Updated weights for policy 0, policy_version 1043709 (0.0011) [2023-12-26 22:58:31,733][105620] Updated weights for policy 1, policy_version 1044502 (0.0007) [2023-12-26 22:58:31,795][105620] Updated weights for policy 1, policy_version 1044512 (0.0008) [2023-12-26 22:58:32,417][105692] Updated weights for policy 0, policy_version 1043719 (0.0008) [2023-12-26 22:58:32,468][105692] Updated weights for policy 0, policy_version 1043729 (0.0008) [2023-12-26 22:58:32,515][105692] Updated weights for policy 0, policy_version 1043739 (0.0007) [2023-12-26 22:58:32,581][105620] Updated weights for policy 1, policy_version 1044522 (0.0010) [2023-12-26 22:58:32,632][105620] Updated weights for policy 1, policy_version 1044532 (0.0010) [2023-12-26 22:58:32,698][105620] Updated weights for policy 1, policy_version 1044542 (0.0011) [2023-12-26 22:58:32,758][105620] Updated weights for policy 1, policy_version 1044552 (0.0011) [2023-12-26 22:58:33,207][105692] Updated weights for policy 0, policy_version 1043749 (0.0008) [2023-12-26 22:58:33,261][105692] Updated weights for policy 0, policy_version 1043759 (0.0008) [2023-12-26 22:58:33,315][105692] Updated weights for policy 0, policy_version 1043769 (0.0008) [2023-12-26 22:58:33,489][105620] Updated weights for policy 1, policy_version 1044562 (0.0010) [2023-12-26 22:58:33,543][105620] Updated weights for policy 1, policy_version 1044572 (0.0010) [2023-12-26 22:58:33,587][105620] Updated weights for policy 1, policy_version 1044582 (0.0010) [2023-12-26 22:58:33,902][105692] Updated weights for policy 0, policy_version 1043779 (0.0007) [2023-12-26 22:58:33,960][105692] Updated weights for policy 0, policy_version 1043789 (0.0005) [2023-12-26 22:58:34,015][105692] Updated weights for policy 0, policy_version 1043799 (0.0008) [2023-12-26 22:58:34,351][105620] Updated weights for policy 1, policy_version 1044592 (0.0008) [2023-12-26 22:58:34,420][105620] Updated weights for policy 1, policy_version 1044602 (0.0008) [2023-12-26 22:58:34,486][105620] Updated weights for policy 1, policy_version 1044612 (0.0008) [2023-12-26 22:58:34,608][105692] Updated weights for policy 0, policy_version 1043809 (0.0007) [2023-12-26 22:58:34,662][105692] Updated weights for policy 0, policy_version 1043819 (0.0010) [2023-12-26 22:58:34,716][105692] Updated weights for policy 0, policy_version 1043829 (0.0009) [2023-12-26 22:58:34,780][105692] Updated weights for policy 0, policy_version 1043839 (0.0008) [2023-12-26 22:58:35,182][105620] Updated weights for policy 1, policy_version 1044622 (0.0010) [2023-12-26 22:58:35,235][105620] Updated weights for policy 1, policy_version 1044632 (0.0011) [2023-12-26 22:58:35,295][105620] Updated weights for policy 1, policy_version 1044642 (0.0011) [2023-12-26 22:58:35,592][105585] KL-divergence is very high: 162.7112 [2023-12-26 22:58:35,597][105692] Updated weights for policy 0, policy_version 1043849 (0.0007) [2023-12-26 22:58:35,632][105585] KL-divergence is very high: 281.2975 [2023-12-26 22:58:35,645][105692] Updated weights for policy 0, policy_version 1043859 (0.0008) [2023-12-26 22:58:35,668][105585] KL-divergence is very high: 289.6139 [2023-12-26 22:58:35,693][105692] Updated weights for policy 0, policy_version 1043869 (0.0008) [2023-12-26 22:58:35,993][105620] Updated weights for policy 1, policy_version 1044652 (0.0009) [2023-12-26 22:58:36,046][105620] Updated weights for policy 1, policy_version 1044662 (0.0005) [2023-12-26 22:58:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 534732800. Throughput: 0: 9944.3, 1: 9697.0. Samples: 534724856. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:58:36,063][104569] Avg episode reward: [(0, '8815.662'), (1, '9260.673')] [2023-12-26 22:58:36,118][105620] Updated weights for policy 1, policy_version 1044672 (0.0007) [2023-12-26 22:58:36,494][105692] Updated weights for policy 0, policy_version 1043879 (0.0009) [2023-12-26 22:58:36,542][105692] Updated weights for policy 0, policy_version 1043889 (0.0009) [2023-12-26 22:58:36,597][105692] Updated weights for policy 0, policy_version 1043899 (0.0009) [2023-12-26 22:58:36,816][105620] Updated weights for policy 1, policy_version 1044682 (0.0010) [2023-12-26 22:58:36,874][105620] Updated weights for policy 1, policy_version 1044692 (0.0009) [2023-12-26 22:58:36,935][105620] Updated weights for policy 1, policy_version 1044702 (0.0009) [2023-12-26 22:58:36,993][105620] Updated weights for policy 1, policy_version 1044712 (0.0009) [2023-12-26 22:58:37,360][105692] Updated weights for policy 0, policy_version 1043909 (0.0009) [2023-12-26 22:58:37,405][105692] Updated weights for policy 0, policy_version 1043919 (0.0008) [2023-12-26 22:58:37,460][105692] Updated weights for policy 0, policy_version 1043929 (0.0009) [2023-12-26 22:58:37,731][105620] Updated weights for policy 1, policy_version 1044722 (0.0011) [2023-12-26 22:58:37,780][105620] Updated weights for policy 1, policy_version 1044732 (0.0010) [2023-12-26 22:58:37,833][105620] Updated weights for policy 1, policy_version 1044742 (0.0010) [2023-12-26 22:58:38,269][105692] Updated weights for policy 0, policy_version 1043939 (0.0008) [2023-12-26 22:58:38,334][105692] Updated weights for policy 0, policy_version 1043949 (0.0008) [2023-12-26 22:58:38,392][105692] Updated weights for policy 0, policy_version 1043959 (0.0008) [2023-12-26 22:58:38,603][105620] Updated weights for policy 1, policy_version 1044752 (0.0010) [2023-12-26 22:58:38,665][105620] Updated weights for policy 1, policy_version 1044762 (0.0010) [2023-12-26 22:58:38,727][105620] Updated weights for policy 1, policy_version 1044772 (0.0010) [2023-12-26 22:58:39,162][105692] Updated weights for policy 0, policy_version 1043969 (0.0008) [2023-12-26 22:58:39,228][105692] Updated weights for policy 0, policy_version 1043979 (0.0008) [2023-12-26 22:58:39,290][105692] Updated weights for policy 0, policy_version 1043989 (0.0007) [2023-12-26 22:58:39,355][105692] Updated weights for policy 0, policy_version 1043999 (0.0007) [2023-12-26 22:58:39,477][105620] Updated weights for policy 1, policy_version 1044782 (0.0010) [2023-12-26 22:58:39,535][105620] Updated weights for policy 1, policy_version 1044792 (0.0009) [2023-12-26 22:58:39,594][105620] Updated weights for policy 1, policy_version 1044802 (0.0009) [2023-12-26 22:58:40,144][105692] Updated weights for policy 0, policy_version 1044009 (0.0009) [2023-12-26 22:58:40,201][105692] Updated weights for policy 0, policy_version 1044019 (0.0010) [2023-12-26 22:58:40,260][105692] Updated weights for policy 0, policy_version 1044029 (0.0009) [2023-12-26 22:58:40,269][105620] Updated weights for policy 1, policy_version 1044812 (0.0009) [2023-12-26 22:58:40,325][105620] Updated weights for policy 1, policy_version 1044822 (0.0008) [2023-12-26 22:58:40,380][105620] Updated weights for policy 1, policy_version 1044832 (0.0008) [2023-12-26 22:58:41,034][105692] Updated weights for policy 0, policy_version 1044039 (0.0009) [2023-12-26 22:58:41,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19524.2, 300 sec: 19383.1). Total num frames: 534822912. Throughput: 0: 9799.4, 1: 9696.6. Samples: 534836356. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:58:41,062][104569] Avg episode reward: [(0, '8904.400'), (1, '9261.654')] [2023-12-26 22:58:41,096][105692] Updated weights for policy 0, policy_version 1044049 (0.0008) [2023-12-26 22:58:41,147][105620] Updated weights for policy 1, policy_version 1044842 (0.0010) [2023-12-26 22:58:41,162][105692] Updated weights for policy 0, policy_version 1044059 (0.0008) [2023-12-26 22:58:41,212][105620] Updated weights for policy 1, policy_version 1044852 (0.0010) [2023-12-26 22:58:41,283][105620] Updated weights for policy 1, policy_version 1044862 (0.0011) [2023-12-26 22:58:41,343][105620] Updated weights for policy 1, policy_version 1044872 (0.0011) [2023-12-26 22:58:41,955][105692] Updated weights for policy 0, policy_version 1044069 (0.0008) [2023-12-26 22:58:42,024][105692] Updated weights for policy 0, policy_version 1044079 (0.0009) [2023-12-26 22:58:42,092][105692] Updated weights for policy 0, policy_version 1044089 (0.0008) [2023-12-26 22:58:42,163][105620] Updated weights for policy 1, policy_version 1044882 (0.0010) [2023-12-26 22:58:42,216][105620] Updated weights for policy 1, policy_version 1044892 (0.0011) [2023-12-26 22:58:42,274][105620] Updated weights for policy 1, policy_version 1044902 (0.0011) [2023-12-26 22:58:42,867][105692] Updated weights for policy 0, policy_version 1044099 (0.0008) [2023-12-26 22:58:42,915][105692] Updated weights for policy 0, policy_version 1044109 (0.0008) [2023-12-26 22:58:42,960][105692] Updated weights for policy 0, policy_version 1044119 (0.0008) [2023-12-26 22:58:43,040][105620] Updated weights for policy 1, policy_version 1044912 (0.0010) [2023-12-26 22:58:43,091][105620] Updated weights for policy 1, policy_version 1044922 (0.0010) [2023-12-26 22:58:43,138][105620] Updated weights for policy 1, policy_version 1044932 (0.0010) [2023-12-26 22:58:43,754][105692] Updated weights for policy 0, policy_version 1044129 (0.0007) [2023-12-26 22:58:43,805][105692] Updated weights for policy 0, policy_version 1044139 (0.0008) [2023-12-26 22:58:43,857][105692] Updated weights for policy 0, policy_version 1044149 (0.0008) [2023-12-26 22:58:43,898][105620] Updated weights for policy 1, policy_version 1044942 (0.0010) [2023-12-26 22:58:43,903][105692] Updated weights for policy 0, policy_version 1044159 (0.0008) [2023-12-26 22:58:43,944][105620] Updated weights for policy 1, policy_version 1044952 (0.0009) [2023-12-26 22:58:43,997][105620] Updated weights for policy 1, policy_version 1044962 (0.0007) [2023-12-26 22:58:44,574][105620] Updated weights for policy 1, policy_version 1044972 (0.0005) [2023-12-26 22:58:44,633][105620] Updated weights for policy 1, policy_version 1044982 (0.0005) [2023-12-26 22:58:44,648][105692] Updated weights for policy 0, policy_version 1044169 (0.0005) [2023-12-26 22:58:44,681][105620] Updated weights for policy 1, policy_version 1044992 (0.0005) [2023-12-26 22:58:44,698][105692] Updated weights for policy 0, policy_version 1044179 (0.0008) [2023-12-26 22:58:44,753][105692] Updated weights for policy 0, policy_version 1044189 (0.0009) [2023-12-26 22:58:45,384][105620] Updated weights for policy 1, policy_version 1045002 (0.0006) [2023-12-26 22:58:45,445][105620] Updated weights for policy 1, policy_version 1045012 (0.0009) [2023-12-26 22:58:45,488][105692] Updated weights for policy 0, policy_version 1044199 (0.0008) [2023-12-26 22:58:45,496][105585] KL-divergence is very high: 102.8704 [2023-12-26 22:58:45,509][105620] Updated weights for policy 1, policy_version 1045022 (0.0008) [2023-12-26 22:58:45,539][105585] KL-divergence is very high: 195.2250 [2023-12-26 22:58:45,541][105692] Updated weights for policy 0, policy_version 1044209 (0.0006) [2023-12-26 22:58:45,567][105620] Updated weights for policy 1, policy_version 1045032 (0.0008) [2023-12-26 22:58:45,583][105585] KL-divergence is very high: 218.3734 [2023-12-26 22:58:45,592][105692] Updated weights for policy 0, policy_version 1044219 (0.0007) [2023-12-26 22:58:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 534921216. Throughput: 0: 9756.3, 1: 9625.1. Samples: 534890240. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:58:46,062][104569] Avg episode reward: [(0, '9080.507'), (1, '9262.263')] [2023-12-26 22:58:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001044224_267362304.pth... [2023-12-26 22:58:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001045032_267558912.pth... [2023-12-26 22:58:46,069][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001043880_267264000.pth [2023-12-26 22:58:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001043104_267075584.pth [2023-12-26 22:58:46,291][105620] Updated weights for policy 1, policy_version 1045042 (0.0009) [2023-12-26 22:58:46,343][105692] Updated weights for policy 0, policy_version 1044229 (0.0007) [2023-12-26 22:58:46,353][105620] Updated weights for policy 1, policy_version 1045052 (0.0008) [2023-12-26 22:58:46,399][105692] Updated weights for policy 0, policy_version 1044239 (0.0006) [2023-12-26 22:58:46,402][105620] Updated weights for policy 1, policy_version 1045062 (0.0007) [2023-12-26 22:58:46,444][105692] Updated weights for policy 0, policy_version 1044249 (0.0007) [2023-12-26 22:58:46,976][105620] Updated weights for policy 1, policy_version 1045072 (0.0008) [2023-12-26 22:58:47,023][105620] Updated weights for policy 1, policy_version 1045082 (0.0005) [2023-12-26 22:58:47,077][105620] Updated weights for policy 1, policy_version 1045092 (0.0005) [2023-12-26 22:58:47,087][105692] Updated weights for policy 0, policy_version 1044259 (0.0008) [2023-12-26 22:58:47,136][105692] Updated weights for policy 0, policy_version 1044269 (0.0006) [2023-12-26 22:58:47,185][105692] Updated weights for policy 0, policy_version 1044279 (0.0006) [2023-12-26 22:58:47,652][105620] Updated weights for policy 1, policy_version 1045102 (0.0008) [2023-12-26 22:58:47,699][105620] Updated weights for policy 1, policy_version 1045112 (0.0008) [2023-12-26 22:58:47,750][105620] Updated weights for policy 1, policy_version 1045122 (0.0008) [2023-12-26 22:58:47,816][105692] Updated weights for policy 0, policy_version 1044289 (0.0006) [2023-12-26 22:58:47,881][105692] Updated weights for policy 0, policy_version 1044299 (0.0010) [2023-12-26 22:58:47,940][105692] Updated weights for policy 0, policy_version 1044309 (0.0010) [2023-12-26 22:58:48,002][105692] Updated weights for policy 0, policy_version 1044319 (0.0011) [2023-12-26 22:58:48,498][105620] Updated weights for policy 1, policy_version 1045132 (0.0007) [2023-12-26 22:58:48,547][105620] Updated weights for policy 1, policy_version 1045142 (0.0005) [2023-12-26 22:58:48,594][105620] Updated weights for policy 1, policy_version 1045152 (0.0005) [2023-12-26 22:58:48,642][105692] Updated weights for policy 0, policy_version 1044329 (0.0010) [2023-12-26 22:58:48,693][105692] Updated weights for policy 0, policy_version 1044339 (0.0011) [2023-12-26 22:58:48,746][105692] Updated weights for policy 0, policy_version 1044349 (0.0010) [2023-12-26 22:58:49,332][105620] Updated weights for policy 1, policy_version 1045162 (0.0006) [2023-12-26 22:58:49,397][105620] Updated weights for policy 1, policy_version 1045172 (0.0008) [2023-12-26 22:58:49,452][105620] Updated weights for policy 1, policy_version 1045182 (0.0009) [2023-12-26 22:58:49,463][105692] Updated weights for policy 0, policy_version 1044359 (0.0006) [2023-12-26 22:58:49,511][105692] Updated weights for policy 0, policy_version 1044369 (0.0007) [2023-12-26 22:58:49,513][105620] Updated weights for policy 1, policy_version 1045192 (0.0009) [2023-12-26 22:58:49,557][105692] Updated weights for policy 0, policy_version 1044379 (0.0010) [2023-12-26 22:58:50,209][105620] Updated weights for policy 1, policy_version 1045202 (0.0006) [2023-12-26 22:58:50,258][105620] Updated weights for policy 1, policy_version 1045212 (0.0006) [2023-12-26 22:58:50,318][105620] Updated weights for policy 1, policy_version 1045222 (0.0006) [2023-12-26 22:58:50,319][105692] Updated weights for policy 0, policy_version 1044389 (0.0011) [2023-12-26 22:58:50,374][105692] Updated weights for policy 0, policy_version 1044399 (0.0010) [2023-12-26 22:58:50,430][105692] Updated weights for policy 0, policy_version 1044409 (0.0010) [2023-12-26 22:58:51,001][105620] Updated weights for policy 1, policy_version 1045232 (0.0009) [2023-12-26 22:58:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19383.1). Total num frames: 535019520. Throughput: 0: 9799.5, 1: 9703.0. Samples: 535012632. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:58:51,063][104569] Avg episode reward: [(0, '9079.800'), (1, '9132.577')] [2023-12-26 22:58:51,068][105620] Updated weights for policy 1, policy_version 1045242 (0.0008) [2023-12-26 22:58:51,124][105620] Updated weights for policy 1, policy_version 1045252 (0.0010) [2023-12-26 22:58:51,206][105692] Updated weights for policy 0, policy_version 1044419 (0.0011) [2023-12-26 22:58:51,270][105692] Updated weights for policy 0, policy_version 1044429 (0.0011) [2023-12-26 22:58:51,329][105692] Updated weights for policy 0, policy_version 1044439 (0.0011) [2023-12-26 22:58:51,924][105620] Updated weights for policy 1, policy_version 1045262 (0.0009) [2023-12-26 22:58:51,989][105620] Updated weights for policy 1, policy_version 1045272 (0.0007) [2023-12-26 22:58:51,996][105692] Updated weights for policy 0, policy_version 1044449 (0.0009) [2023-12-26 22:58:52,055][105620] Updated weights for policy 1, policy_version 1045282 (0.0009) [2023-12-26 22:58:52,057][105692] Updated weights for policy 0, policy_version 1044459 (0.0007) [2023-12-26 22:58:52,114][105692] Updated weights for policy 0, policy_version 1044469 (0.0009) [2023-12-26 22:58:52,175][105692] Updated weights for policy 0, policy_version 1044479 (0.0009) [2023-12-26 22:58:52,672][105620] Updated weights for policy 1, policy_version 1045292 (0.0008) [2023-12-26 22:58:52,727][105620] Updated weights for policy 1, policy_version 1045302 (0.0010) [2023-12-26 22:58:52,789][105620] Updated weights for policy 1, policy_version 1045312 (0.0010) [2023-12-26 22:58:52,848][105692] Updated weights for policy 0, policy_version 1044489 (0.0010) [2023-12-26 22:58:52,899][105692] Updated weights for policy 0, policy_version 1044499 (0.0006) [2023-12-26 22:58:52,957][105692] Updated weights for policy 0, policy_version 1044509 (0.0005) [2023-12-26 22:58:53,542][105620] Updated weights for policy 1, policy_version 1045322 (0.0010) [2023-12-26 22:58:53,590][105620] Updated weights for policy 1, policy_version 1045332 (0.0010) [2023-12-26 22:58:53,651][105620] Updated weights for policy 1, policy_version 1045342 (0.0010) [2023-12-26 22:58:53,654][105692] Updated weights for policy 0, policy_version 1044519 (0.0009) [2023-12-26 22:58:53,712][105692] Updated weights for policy 0, policy_version 1044529 (0.0010) [2023-12-26 22:58:53,716][105620] Updated weights for policy 1, policy_version 1045352 (0.0010) [2023-12-26 22:58:53,778][105692] Updated weights for policy 0, policy_version 1044539 (0.0011) [2023-12-26 22:58:54,383][105620] Updated weights for policy 1, policy_version 1045362 (0.0010) [2023-12-26 22:58:54,414][105692] Updated weights for policy 0, policy_version 1044549 (0.0008) [2023-12-26 22:58:54,441][105620] Updated weights for policy 1, policy_version 1045372 (0.0010) [2023-12-26 22:58:54,460][105692] Updated weights for policy 0, policy_version 1044559 (0.0005) [2023-12-26 22:58:54,500][105620] Updated weights for policy 1, policy_version 1045382 (0.0010) [2023-12-26 22:58:54,509][105692] Updated weights for policy 0, policy_version 1044569 (0.0005) [2023-12-26 22:58:55,050][105692] Updated weights for policy 0, policy_version 1044579 (0.0006) [2023-12-26 22:58:55,099][105692] Updated weights for policy 0, policy_version 1044589 (0.0006) [2023-12-26 22:58:55,160][105692] Updated weights for policy 0, policy_version 1044599 (0.0005) [2023-12-26 22:58:55,225][105620] Updated weights for policy 1, policy_version 1045392 (0.0010) [2023-12-26 22:58:55,284][105620] Updated weights for policy 1, policy_version 1045402 (0.0006) [2023-12-26 22:58:55,341][105620] Updated weights for policy 1, policy_version 1045412 (0.0008) [2023-12-26 22:58:55,670][105692] Updated weights for policy 0, policy_version 1044609 (0.0006) [2023-12-26 22:58:55,726][105692] Updated weights for policy 0, policy_version 1044619 (0.0005) [2023-12-26 22:58:55,775][105692] Updated weights for policy 0, policy_version 1044629 (0.0005) [2023-12-26 22:58:55,822][105692] Updated weights for policy 0, policy_version 1044639 (0.0008) [2023-12-26 22:58:56,049][105620] Updated weights for policy 1, policy_version 1045423 (0.0010) [2023-12-26 22:58:56,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 535126016. Throughput: 0: 9850.0, 1: 9772.8. Samples: 535134852. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:58:56,063][104569] Avg episode reward: [(0, '4445.820'), (1, '8302.758')] [2023-12-26 22:58:56,110][105620] Updated weights for policy 1, policy_version 1045433 (0.0008) [2023-12-26 22:58:56,123][105586] KL-divergence is very high: 101.8428 [2023-12-26 22:58:56,168][105620] Updated weights for policy 1, policy_version 1045443 (0.0005) [2023-12-26 22:58:56,561][105692] Updated weights for policy 0, policy_version 1044649 (0.0011) [2023-12-26 22:58:56,608][105692] Updated weights for policy 0, policy_version 1044659 (0.0010) [2023-12-26 22:58:56,669][105692] Updated weights for policy 0, policy_version 1044669 (0.0008) [2023-12-26 22:58:56,838][105620] Updated weights for policy 1, policy_version 1045453 (0.0007) [2023-12-26 22:58:56,894][105620] Updated weights for policy 1, policy_version 1045463 (0.0008) [2023-12-26 22:58:56,942][105620] Updated weights for policy 1, policy_version 1045473 (0.0008) [2023-12-26 22:58:57,413][105692] Updated weights for policy 0, policy_version 1044679 (0.0007) [2023-12-26 22:58:57,463][105692] Updated weights for policy 0, policy_version 1044689 (0.0009) [2023-12-26 22:58:57,510][105620] Updated weights for policy 1, policy_version 1045483 (0.0006) [2023-12-26 22:58:57,523][105692] Updated weights for policy 0, policy_version 1044699 (0.0010) [2023-12-26 22:58:57,562][105620] Updated weights for policy 1, policy_version 1045493 (0.0008) [2023-12-26 22:58:57,614][105620] Updated weights for policy 1, policy_version 1045503 (0.0007) [2023-12-26 22:58:58,156][105692] Updated weights for policy 0, policy_version 1044709 (0.0006) [2023-12-26 22:58:58,225][105692] Updated weights for policy 0, policy_version 1044719 (0.0006) [2023-12-26 22:58:58,289][105692] Updated weights for policy 0, policy_version 1044729 (0.0006) [2023-12-26 22:58:58,454][105620] Updated weights for policy 1, policy_version 1045513 (0.0008) [2023-12-26 22:58:58,520][105620] Updated weights for policy 1, policy_version 1045523 (0.0011) [2023-12-26 22:58:58,588][105620] Updated weights for policy 1, policy_version 1045533 (0.0010) [2023-12-26 22:58:58,655][105620] Updated weights for policy 1, policy_version 1045543 (0.0011) [2023-12-26 22:58:59,101][105692] Updated weights for policy 0, policy_version 1044739 (0.0007) [2023-12-26 22:58:59,161][105692] Updated weights for policy 0, policy_version 1044749 (0.0008) [2023-12-26 22:58:59,228][105692] Updated weights for policy 0, policy_version 1044759 (0.0008) [2023-12-26 22:58:59,460][105620] Updated weights for policy 1, policy_version 1045553 (0.0010) [2023-12-26 22:58:59,525][105620] Updated weights for policy 1, policy_version 1045563 (0.0010) [2023-12-26 22:58:59,588][105620] Updated weights for policy 1, policy_version 1045573 (0.0010) [2023-12-26 22:58:59,913][105692] Updated weights for policy 0, policy_version 1044769 (0.0008) [2023-12-26 22:58:59,978][105692] Updated weights for policy 0, policy_version 1044779 (0.0007) [2023-12-26 22:59:00,044][105692] Updated weights for policy 0, policy_version 1044789 (0.0006) [2023-12-26 22:59:00,100][105692] Updated weights for policy 0, policy_version 1044799 (0.0005) [2023-12-26 22:59:00,301][105620] Updated weights for policy 1, policy_version 1045583 (0.0010) [2023-12-26 22:59:00,349][105620] Updated weights for policy 1, policy_version 1045593 (0.0010) [2023-12-26 22:59:00,400][105620] Updated weights for policy 1, policy_version 1045603 (0.0010) [2023-12-26 22:59:00,797][105692] Updated weights for policy 0, policy_version 1044809 (0.0010) [2023-12-26 22:59:00,857][105692] Updated weights for policy 0, policy_version 1044819 (0.0009) [2023-12-26 22:59:00,913][105692] Updated weights for policy 0, policy_version 1044829 (0.0007) [2023-12-26 22:59:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 535224320. Throughput: 0: 9880.6, 1: 9809.0. Samples: 535194124. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:59:01,063][104569] Avg episode reward: [(0, '4726.962'), (1, '5014.896')] [2023-12-26 22:59:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001044832_267517952.pth... [2023-12-26 22:59:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001045608_267706368.pth... [2023-12-26 22:59:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001044488_267419648.pth [2023-12-26 22:59:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001043680_267223040.pth [2023-12-26 22:59:01,166][105620] Updated weights for policy 1, policy_version 1045613 (0.0010) [2023-12-26 22:59:01,222][105620] Updated weights for policy 1, policy_version 1045623 (0.0010) [2023-12-26 22:59:01,275][105620] Updated weights for policy 1, policy_version 1045633 (0.0007) [2023-12-26 22:59:01,646][105692] Updated weights for policy 0, policy_version 1044839 (0.0007) [2023-12-26 22:59:01,709][105692] Updated weights for policy 0, policy_version 1044849 (0.0006) [2023-12-26 22:59:01,776][105692] Updated weights for policy 0, policy_version 1044859 (0.0008) [2023-12-26 22:59:01,969][105620] Updated weights for policy 1, policy_version 1045643 (0.0007) [2023-12-26 22:59:02,038][105620] Updated weights for policy 1, policy_version 1045653 (0.0011) [2023-12-26 22:59:02,091][105620] Updated weights for policy 1, policy_version 1045663 (0.0011) [2023-12-26 22:59:02,435][105692] Updated weights for policy 0, policy_version 1044869 (0.0008) [2023-12-26 22:59:02,501][105692] Updated weights for policy 0, policy_version 1044879 (0.0008) [2023-12-26 22:59:02,571][105692] Updated weights for policy 0, policy_version 1044889 (0.0008) [2023-12-26 22:59:02,828][105620] Updated weights for policy 1, policy_version 1045673 (0.0010) [2023-12-26 22:59:02,885][105620] Updated weights for policy 1, policy_version 1045683 (0.0008) [2023-12-26 22:59:02,938][105620] Updated weights for policy 1, policy_version 1045693 (0.0008) [2023-12-26 22:59:02,988][105620] Updated weights for policy 1, policy_version 1045703 (0.0008) [2023-12-26 22:59:03,286][105692] Updated weights for policy 0, policy_version 1044899 (0.0007) [2023-12-26 22:59:03,335][105692] Updated weights for policy 0, policy_version 1044909 (0.0005) [2023-12-26 22:59:03,381][105692] Updated weights for policy 0, policy_version 1044919 (0.0005) [2023-12-26 22:59:03,573][105620] Updated weights for policy 1, policy_version 1045713 (0.0009) [2023-12-26 22:59:03,624][105620] Updated weights for policy 1, policy_version 1045724 (0.0009) [2023-12-26 22:59:03,674][105620] Updated weights for policy 1, policy_version 1045734 (0.0009) [2023-12-26 22:59:04,001][105692] Updated weights for policy 0, policy_version 1044929 (0.0005) [2023-12-26 22:59:04,059][105692] Updated weights for policy 0, policy_version 1044939 (0.0009) [2023-12-26 22:59:04,125][105692] Updated weights for policy 0, policy_version 1044949 (0.0009) [2023-12-26 22:59:04,188][105692] Updated weights for policy 0, policy_version 1044959 (0.0008) [2023-12-26 22:59:04,480][105620] Updated weights for policy 1, policy_version 1045744 (0.0008) [2023-12-26 22:59:04,540][105620] Updated weights for policy 1, policy_version 1045754 (0.0008) [2023-12-26 22:59:04,594][105620] Updated weights for policy 1, policy_version 1045764 (0.0009) [2023-12-26 22:59:04,943][105692] Updated weights for policy 0, policy_version 1044969 (0.0009) [2023-12-26 22:59:05,001][105692] Updated weights for policy 0, policy_version 1044979 (0.0009) [2023-12-26 22:59:05,064][105692] Updated weights for policy 0, policy_version 1044989 (0.0008) [2023-12-26 22:59:05,280][105620] Updated weights for policy 1, policy_version 1045774 (0.0007) [2023-12-26 22:59:05,332][105620] Updated weights for policy 1, policy_version 1045784 (0.0005) [2023-12-26 22:59:05,379][105620] Updated weights for policy 1, policy_version 1045794 (0.0005) [2023-12-26 22:59:05,935][105692] Updated weights for policy 0, policy_version 1044999 (0.0009) [2023-12-26 22:59:05,996][105692] Updated weights for policy 0, policy_version 1045009 (0.0008) [2023-12-26 22:59:06,007][105620] Updated weights for policy 1, policy_version 1045804 (0.0006) [2023-12-26 22:59:06,050][105692] Updated weights for policy 0, policy_version 1045019 (0.0007) [2023-12-26 22:59:06,062][105620] Updated weights for policy 1, policy_version 1045814 (0.0007) [2023-12-26 22:59:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 535314432. Throughput: 0: 9863.1, 1: 9792.4. Samples: 535310460. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:59:06,062][104569] Avg episode reward: [(0, '7269.591'), (1, '5321.412')] [2023-12-26 22:59:06,117][105620] Updated weights for policy 1, policy_version 1045824 (0.0007) [2023-12-26 22:59:06,823][105620] Updated weights for policy 1, policy_version 1045834 (0.0006) [2023-12-26 22:59:06,879][105692] Updated weights for policy 0, policy_version 1045029 (0.0010) [2023-12-26 22:59:06,885][105620] Updated weights for policy 1, policy_version 1045844 (0.0007) [2023-12-26 22:59:06,929][105692] Updated weights for policy 0, policy_version 1045039 (0.0006) [2023-12-26 22:59:06,937][105620] Updated weights for policy 1, policy_version 1045854 (0.0008) [2023-12-26 22:59:06,984][105692] Updated weights for policy 0, policy_version 1045049 (0.0008) [2023-12-26 22:59:06,996][105620] Updated weights for policy 1, policy_version 1045864 (0.0006) [2023-12-26 22:59:07,702][105620] Updated weights for policy 1, policy_version 1045874 (0.0005) [2023-12-26 22:59:07,709][105692] Updated weights for policy 0, policy_version 1045059 (0.0010) [2023-12-26 22:59:07,751][105620] Updated weights for policy 1, policy_version 1045884 (0.0006) [2023-12-26 22:59:07,757][105692] Updated weights for policy 0, policy_version 1045069 (0.0010) [2023-12-26 22:59:07,805][105692] Updated weights for policy 0, policy_version 1045079 (0.0010) [2023-12-26 22:59:07,807][105620] Updated weights for policy 1, policy_version 1045894 (0.0006) [2023-12-26 22:59:08,524][105620] Updated weights for policy 1, policy_version 1045904 (0.0006) [2023-12-26 22:59:08,581][105692] Updated weights for policy 0, policy_version 1045089 (0.0010) [2023-12-26 22:59:08,591][105620] Updated weights for policy 1, policy_version 1045914 (0.0005) [2023-12-26 22:59:08,644][105692] Updated weights for policy 0, policy_version 1045099 (0.0009) [2023-12-26 22:59:08,646][105620] Updated weights for policy 1, policy_version 1045924 (0.0006) [2023-12-26 22:59:08,702][105692] Updated weights for policy 0, policy_version 1045109 (0.0010) [2023-12-26 22:59:08,765][105692] Updated weights for policy 0, policy_version 1045119 (0.0010) [2023-12-26 22:59:09,264][105620] Updated weights for policy 1, policy_version 1045934 (0.0007) [2023-12-26 22:59:09,325][105620] Updated weights for policy 1, policy_version 1045944 (0.0007) [2023-12-26 22:59:09,395][105620] Updated weights for policy 1, policy_version 1045954 (0.0009) [2023-12-26 22:59:09,504][105692] Updated weights for policy 0, policy_version 1045129 (0.0006) [2023-12-26 22:59:09,559][105692] Updated weights for policy 0, policy_version 1045139 (0.0006) [2023-12-26 22:59:09,624][105692] Updated weights for policy 0, policy_version 1045149 (0.0007) [2023-12-26 22:59:10,143][105620] Updated weights for policy 1, policy_version 1045964 (0.0009) [2023-12-26 22:59:10,216][105620] Updated weights for policy 1, policy_version 1045974 (0.0009) [2023-12-26 22:59:10,219][105692] Updated weights for policy 0, policy_version 1045159 (0.0006) [2023-12-26 22:59:10,271][105692] Updated weights for policy 0, policy_version 1045169 (0.0008) [2023-12-26 22:59:10,281][105620] Updated weights for policy 1, policy_version 1045984 (0.0008) [2023-12-26 22:59:10,328][105692] Updated weights for policy 0, policy_version 1045179 (0.0007) [2023-12-26 22:59:10,956][105692] Updated weights for policy 0, policy_version 1045189 (0.0006) [2023-12-26 22:59:11,012][105692] Updated weights for policy 0, policy_version 1045199 (0.0006) [2023-12-26 22:59:11,032][105620] Updated weights for policy 1, policy_version 1045994 (0.0010) [2023-12-26 22:59:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 535412736. Throughput: 0: 9830.0, 1: 9838.1. Samples: 535426452. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:59:11,063][104569] Avg episode reward: [(0, '9257.147'), (1, '7532.079')] [2023-12-26 22:59:11,077][105692] Updated weights for policy 0, policy_version 1045209 (0.0007) [2023-12-26 22:59:11,103][105620] Updated weights for policy 1, policy_version 1046004 (0.0008) [2023-12-26 22:59:11,172][105620] Updated weights for policy 1, policy_version 1046014 (0.0008) [2023-12-26 22:59:11,224][105620] Updated weights for policy 1, policy_version 1046024 (0.0008) [2023-12-26 22:59:11,785][105692] Updated weights for policy 0, policy_version 1045219 (0.0007) [2023-12-26 22:59:11,853][105692] Updated weights for policy 0, policy_version 1045229 (0.0005) [2023-12-26 22:59:11,919][105692] Updated weights for policy 0, policy_version 1045239 (0.0007) [2023-12-26 22:59:12,009][105620] Updated weights for policy 1, policy_version 1046034 (0.0008) [2023-12-26 22:59:12,075][105620] Updated weights for policy 1, policy_version 1046044 (0.0008) [2023-12-26 22:59:12,130][105620] Updated weights for policy 1, policy_version 1046054 (0.0009) [2023-12-26 22:59:12,670][105692] Updated weights for policy 0, policy_version 1045249 (0.0007) [2023-12-26 22:59:12,732][105692] Updated weights for policy 0, policy_version 1045259 (0.0006) [2023-12-26 22:59:12,795][105692] Updated weights for policy 0, policy_version 1045269 (0.0008) [2023-12-26 22:59:12,848][105692] Updated weights for policy 0, policy_version 1045279 (0.0009) [2023-12-26 22:59:12,963][105620] Updated weights for policy 1, policy_version 1046064 (0.0010) [2023-12-26 22:59:13,026][105620] Updated weights for policy 1, policy_version 1046074 (0.0010) [2023-12-26 22:59:13,090][105620] Updated weights for policy 1, policy_version 1046084 (0.0011) [2023-12-26 22:59:13,531][105692] Updated weights for policy 0, policy_version 1045289 (0.0009) [2023-12-26 22:59:13,592][105692] Updated weights for policy 0, policy_version 1045299 (0.0009) [2023-12-26 22:59:13,649][105692] Updated weights for policy 0, policy_version 1045309 (0.0009) [2023-12-26 22:59:13,757][105620] Updated weights for policy 1, policy_version 1046094 (0.0010) [2023-12-26 22:59:13,821][105620] Updated weights for policy 1, policy_version 1046104 (0.0010) [2023-12-26 22:59:13,879][105620] Updated weights for policy 1, policy_version 1046114 (0.0009) [2023-12-26 22:59:14,336][105692] Updated weights for policy 0, policy_version 1045319 (0.0007) [2023-12-26 22:59:14,396][105692] Updated weights for policy 0, policy_version 1045329 (0.0008) [2023-12-26 22:59:14,462][105692] Updated weights for policy 0, policy_version 1045339 (0.0008) [2023-12-26 22:59:14,669][105620] Updated weights for policy 1, policy_version 1046125 (0.0008) [2023-12-26 22:59:14,721][105620] Updated weights for policy 1, policy_version 1046135 (0.0005) [2023-12-26 22:59:14,784][105620] Updated weights for policy 1, policy_version 1046145 (0.0010) [2023-12-26 22:59:15,143][105692] Updated weights for policy 0, policy_version 1045349 (0.0007) [2023-12-26 22:59:15,202][105692] Updated weights for policy 0, policy_version 1045359 (0.0008) [2023-12-26 22:59:15,274][105692] Updated weights for policy 0, policy_version 1045369 (0.0008) [2023-12-26 22:59:15,413][105620] Updated weights for policy 1, policy_version 1046155 (0.0009) [2023-12-26 22:59:15,467][105620] Updated weights for policy 1, policy_version 1046165 (0.0007) [2023-12-26 22:59:15,536][105620] Updated weights for policy 1, policy_version 1046175 (0.0008) [2023-12-26 22:59:15,853][105692] Updated weights for policy 0, policy_version 1045379 (0.0005) [2023-12-26 22:59:15,902][105692] Updated weights for policy 0, policy_version 1045389 (0.0005) [2023-12-26 22:59:15,954][105692] Updated weights for policy 0, policy_version 1045399 (0.0008) [2023-12-26 22:59:16,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 535519232. Throughput: 0: 9819.7, 1: 9749.4. Samples: 535483684. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:59:16,062][104569] Avg episode reward: [(0, '9263.809'), (1, '9168.470')] [2023-12-26 22:59:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001045408_267665408.pth... [2023-12-26 22:59:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001046184_267853824.pth... [2023-12-26 22:59:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001045032_267558912.pth [2023-12-26 22:59:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001044224_267362304.pth [2023-12-26 22:59:16,173][105620] Updated weights for policy 1, policy_version 1046185 (0.0006) [2023-12-26 22:59:16,235][105620] Updated weights for policy 1, policy_version 1046195 (0.0010) [2023-12-26 22:59:16,283][105620] Updated weights for policy 1, policy_version 1046205 (0.0010) [2023-12-26 22:59:16,352][105620] Updated weights for policy 1, policy_version 1046215 (0.0010) [2023-12-26 22:59:16,651][105692] Updated weights for policy 0, policy_version 1045409 (0.0008) [2023-12-26 22:59:16,711][105692] Updated weights for policy 0, policy_version 1045419 (0.0006) [2023-12-26 22:59:16,762][105692] Updated weights for policy 0, policy_version 1045429 (0.0005) [2023-12-26 22:59:16,810][105692] Updated weights for policy 0, policy_version 1045439 (0.0005) [2023-12-26 22:59:17,095][105620] Updated weights for policy 1, policy_version 1046225 (0.0010) [2023-12-26 22:59:17,160][105620] Updated weights for policy 1, policy_version 1046235 (0.0010) [2023-12-26 22:59:17,220][105620] Updated weights for policy 1, policy_version 1046245 (0.0010) [2023-12-26 22:59:17,405][105692] Updated weights for policy 0, policy_version 1045449 (0.0005) [2023-12-26 22:59:17,467][105692] Updated weights for policy 0, policy_version 1045459 (0.0010) [2023-12-26 22:59:17,529][105692] Updated weights for policy 0, policy_version 1045469 (0.0010) [2023-12-26 22:59:17,954][105620] Updated weights for policy 1, policy_version 1046255 (0.0010) [2023-12-26 22:59:18,007][105620] Updated weights for policy 1, policy_version 1046265 (0.0009) [2023-12-26 22:59:18,062][105620] Updated weights for policy 1, policy_version 1046275 (0.0010) [2023-12-26 22:59:18,246][105692] Updated weights for policy 0, policy_version 1045479 (0.0007) [2023-12-26 22:59:18,303][105692] Updated weights for policy 0, policy_version 1045489 (0.0009) [2023-12-26 22:59:18,365][105692] Updated weights for policy 0, policy_version 1045499 (0.0009) [2023-12-26 22:59:18,733][105620] Updated weights for policy 1, policy_version 1046285 (0.0010) [2023-12-26 22:59:18,781][105620] Updated weights for policy 1, policy_version 1046295 (0.0010) [2023-12-26 22:59:18,835][105620] Updated weights for policy 1, policy_version 1046305 (0.0009) [2023-12-26 22:59:19,038][105692] Updated weights for policy 0, policy_version 1045509 (0.0010) [2023-12-26 22:59:19,090][105692] Updated weights for policy 0, policy_version 1045519 (0.0008) [2023-12-26 22:59:19,152][105692] Updated weights for policy 0, policy_version 1045529 (0.0008) [2023-12-26 22:59:19,595][105620] Updated weights for policy 1, policy_version 1046315 (0.0006) [2023-12-26 22:59:19,655][105620] Updated weights for policy 1, policy_version 1046325 (0.0009) [2023-12-26 22:59:19,717][105620] Updated weights for policy 1, policy_version 1046335 (0.0007) [2023-12-26 22:59:19,939][105692] Updated weights for policy 0, policy_version 1045539 (0.0006) [2023-12-26 22:59:20,000][105692] Updated weights for policy 0, policy_version 1045549 (0.0008) [2023-12-26 22:59:20,054][105692] Updated weights for policy 0, policy_version 1045559 (0.0008) [2023-12-26 22:59:20,430][105620] Updated weights for policy 1, policy_version 1046345 (0.0007) [2023-12-26 22:59:20,493][105620] Updated weights for policy 1, policy_version 1046355 (0.0009) [2023-12-26 22:59:20,553][105620] Updated weights for policy 1, policy_version 1046365 (0.0009) [2023-12-26 22:59:20,613][105620] Updated weights for policy 1, policy_version 1046375 (0.0008) [2023-12-26 22:59:20,818][105692] Updated weights for policy 0, policy_version 1045569 (0.0008) [2023-12-26 22:59:20,868][105692] Updated weights for policy 0, policy_version 1045579 (0.0010) [2023-12-26 22:59:20,914][105692] Updated weights for policy 0, policy_version 1045589 (0.0009) [2023-12-26 22:59:20,974][105692] Updated weights for policy 0, policy_version 1045599 (0.0011) [2023-12-26 22:59:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19438.7). Total num frames: 535617536. Throughput: 0: 9775.6, 1: 9777.2. Samples: 535604736. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:59:21,063][104569] Avg episode reward: [(0, '9265.360'), (1, '9262.013')] [2023-12-26 22:59:21,371][105620] Updated weights for policy 1, policy_version 1046385 (0.0009) [2023-12-26 22:59:21,434][105620] Updated weights for policy 1, policy_version 1046395 (0.0007) [2023-12-26 22:59:21,493][105620] Updated weights for policy 1, policy_version 1046405 (0.0008) [2023-12-26 22:59:21,741][105692] Updated weights for policy 0, policy_version 1045609 (0.0008) [2023-12-26 22:59:21,806][105692] Updated weights for policy 0, policy_version 1045619 (0.0006) [2023-12-26 22:59:21,877][105692] Updated weights for policy 0, policy_version 1045629 (0.0006) [2023-12-26 22:59:22,334][105620] Updated weights for policy 1, policy_version 1046415 (0.0010) [2023-12-26 22:59:22,397][105620] Updated weights for policy 1, policy_version 1046425 (0.0009) [2023-12-26 22:59:22,455][105620] Updated weights for policy 1, policy_version 1046435 (0.0009) [2023-12-26 22:59:22,509][105692] Updated weights for policy 0, policy_version 1045639 (0.0006) [2023-12-26 22:59:22,563][105692] Updated weights for policy 0, policy_version 1045649 (0.0007) [2023-12-26 22:59:22,625][105692] Updated weights for policy 0, policy_version 1045659 (0.0007) [2023-12-26 22:59:23,224][105620] Updated weights for policy 1, policy_version 1046445 (0.0009) [2023-12-26 22:59:23,276][105620] Updated weights for policy 1, policy_version 1046455 (0.0006) [2023-12-26 22:59:23,348][105620] Updated weights for policy 1, policy_version 1046465 (0.0005) [2023-12-26 22:59:23,386][105692] Updated weights for policy 0, policy_version 1045669 (0.0009) [2023-12-26 22:59:23,444][105692] Updated weights for policy 0, policy_version 1045679 (0.0010) [2023-12-26 22:59:23,506][105692] Updated weights for policy 0, policy_version 1045689 (0.0009) [2023-12-26 22:59:24,018][105620] Updated weights for policy 1, policy_version 1046475 (0.0005) [2023-12-26 22:59:24,082][105620] Updated weights for policy 1, policy_version 1046485 (0.0005) [2023-12-26 22:59:24,119][105692] Updated weights for policy 0, policy_version 1045699 (0.0006) [2023-12-26 22:59:24,148][105620] Updated weights for policy 1, policy_version 1046495 (0.0009) [2023-12-26 22:59:24,167][105692] Updated weights for policy 0, policy_version 1045709 (0.0006) [2023-12-26 22:59:24,223][105692] Updated weights for policy 0, policy_version 1045719 (0.0006) [2023-12-26 22:59:24,854][105692] Updated weights for policy 0, policy_version 1045729 (0.0008) [2023-12-26 22:59:24,858][105620] Updated weights for policy 1, policy_version 1046505 (0.0010) [2023-12-26 22:59:24,903][105692] Updated weights for policy 0, policy_version 1045739 (0.0006) [2023-12-26 22:59:24,916][105620] Updated weights for policy 1, policy_version 1046515 (0.0010) [2023-12-26 22:59:24,950][105692] Updated weights for policy 0, policy_version 1045749 (0.0005) [2023-12-26 22:59:24,968][105620] Updated weights for policy 1, policy_version 1046525 (0.0010) [2023-12-26 22:59:24,998][105692] Updated weights for policy 0, policy_version 1045759 (0.0005) [2023-12-26 22:59:25,026][105620] Updated weights for policy 1, policy_version 1046535 (0.0010) [2023-12-26 22:59:25,606][105692] Updated weights for policy 0, policy_version 1045769 (0.0005) [2023-12-26 22:59:25,662][105692] Updated weights for policy 0, policy_version 1045779 (0.0005) [2023-12-26 22:59:25,665][105620] Updated weights for policy 1, policy_version 1046545 (0.0006) [2023-12-26 22:59:25,715][105692] Updated weights for policy 0, policy_version 1045789 (0.0005) [2023-12-26 22:59:25,727][105620] Updated weights for policy 1, policy_version 1046555 (0.0005) [2023-12-26 22:59:25,788][105620] Updated weights for policy 1, policy_version 1046565 (0.0005) [2023-12-26 22:59:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19438.6). Total num frames: 535715840. Throughput: 0: 9926.2, 1: 9769.2. Samples: 535722644. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:59:26,062][104569] Avg episode reward: [(0, '8997.237'), (1, '9260.929')] [2023-12-26 22:59:26,222][105692] Updated weights for policy 0, policy_version 1045799 (0.0005) [2023-12-26 22:59:26,288][105692] Updated weights for policy 0, policy_version 1045809 (0.0006) [2023-12-26 22:59:26,344][105692] Updated weights for policy 0, policy_version 1045819 (0.0005) [2023-12-26 22:59:26,445][105620] Updated weights for policy 1, policy_version 1046575 (0.0011) [2023-12-26 22:59:26,537][105620] Updated weights for policy 1, policy_version 1046585 (0.0008) [2023-12-26 22:59:26,605][105620] Updated weights for policy 1, policy_version 1046595 (0.0008) [2023-12-26 22:59:26,946][105692] Updated weights for policy 0, policy_version 1045829 (0.0006) [2023-12-26 22:59:26,992][105692] Updated weights for policy 0, policy_version 1045839 (0.0005) [2023-12-26 22:59:27,047][105692] Updated weights for policy 0, policy_version 1045849 (0.0005) [2023-12-26 22:59:27,150][105620] Updated weights for policy 1, policy_version 1046605 (0.0008) [2023-12-26 22:59:27,195][105620] Updated weights for policy 1, policy_version 1046615 (0.0005) [2023-12-26 22:59:27,251][105620] Updated weights for policy 1, policy_version 1046625 (0.0005) [2023-12-26 22:59:27,613][105692] Updated weights for policy 0, policy_version 1045859 (0.0005) [2023-12-26 22:59:27,672][105692] Updated weights for policy 0, policy_version 1045869 (0.0005) [2023-12-26 22:59:27,734][105692] Updated weights for policy 0, policy_version 1045879 (0.0007) [2023-12-26 22:59:27,836][105620] Updated weights for policy 1, policy_version 1046635 (0.0007) [2023-12-26 22:59:27,887][105620] Updated weights for policy 1, policy_version 1046645 (0.0005) [2023-12-26 22:59:27,939][105620] Updated weights for policy 1, policy_version 1046655 (0.0005) [2023-12-26 22:59:28,313][105692] Updated weights for policy 0, policy_version 1045889 (0.0009) [2023-12-26 22:59:28,378][105692] Updated weights for policy 0, policy_version 1045899 (0.0007) [2023-12-26 22:59:28,446][105692] Updated weights for policy 0, policy_version 1045909 (0.0008) [2023-12-26 22:59:28,513][105692] Updated weights for policy 0, policy_version 1045919 (0.0008) [2023-12-26 22:59:28,581][105620] Updated weights for policy 1, policy_version 1046665 (0.0006) [2023-12-26 22:59:28,646][105620] Updated weights for policy 1, policy_version 1046675 (0.0010) [2023-12-26 22:59:28,705][105620] Updated weights for policy 1, policy_version 1046685 (0.0009) [2023-12-26 22:59:28,770][105620] Updated weights for policy 1, policy_version 1046695 (0.0010) [2023-12-26 22:59:29,073][105692] Updated weights for policy 0, policy_version 1045929 (0.0006) [2023-12-26 22:59:29,136][105692] Updated weights for policy 0, policy_version 1045939 (0.0006) [2023-12-26 22:59:29,195][105692] Updated weights for policy 0, policy_version 1045949 (0.0005) [2023-12-26 22:59:29,576][105620] Updated weights for policy 1, policy_version 1046705 (0.0009) [2023-12-26 22:59:29,626][105620] Updated weights for policy 1, policy_version 1046715 (0.0008) [2023-12-26 22:59:29,676][105620] Updated weights for policy 1, policy_version 1046725 (0.0009) [2023-12-26 22:59:29,871][105692] Updated weights for policy 0, policy_version 1045959 (0.0007) [2023-12-26 22:59:29,934][105692] Updated weights for policy 0, policy_version 1045969 (0.0009) [2023-12-26 22:59:29,982][105692] Updated weights for policy 0, policy_version 1045979 (0.0008) [2023-12-26 22:59:30,455][105620] Updated weights for policy 1, policy_version 1046735 (0.0010) [2023-12-26 22:59:30,506][105620] Updated weights for policy 1, policy_version 1046745 (0.0006) [2023-12-26 22:59:30,551][105620] Updated weights for policy 1, policy_version 1046755 (0.0005) [2023-12-26 22:59:30,652][105692] Updated weights for policy 0, policy_version 1045989 (0.0009) [2023-12-26 22:59:30,701][105692] Updated weights for policy 0, policy_version 1045999 (0.0007) [2023-12-26 22:59:30,759][105692] Updated weights for policy 0, policy_version 1046009 (0.0008) [2023-12-26 22:59:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 535822336. Throughput: 0: 10108.2, 1: 9875.7. Samples: 535789516. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:59:31,063][104569] Avg episode reward: [(0, '8817.994'), (1, '9261.477')] [2023-12-26 22:59:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001046016_267821056.pth... [2023-12-26 22:59:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001044832_267517952.pth [2023-12-26 22:59:31,098][105620] Updated weights for policy 1, policy_version 1046765 (0.0007) [2023-12-26 22:59:31,167][105620] Updated weights for policy 1, policy_version 1046775 (0.0009) [2023-12-26 22:59:31,229][105620] Updated weights for policy 1, policy_version 1046785 (0.0010) [2023-12-26 22:59:31,270][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001046792_268009472.pth... [2023-12-26 22:59:31,273][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001045608_267706368.pth [2023-12-26 22:59:31,528][105692] Updated weights for policy 0, policy_version 1046019 (0.0010) [2023-12-26 22:59:31,582][105692] Updated weights for policy 0, policy_version 1046029 (0.0006) [2023-12-26 22:59:31,644][105692] Updated weights for policy 0, policy_version 1046039 (0.0010) [2023-12-26 22:59:31,882][105620] Updated weights for policy 1, policy_version 1046795 (0.0010) [2023-12-26 22:59:31,944][105620] Updated weights for policy 1, policy_version 1046805 (0.0007) [2023-12-26 22:59:31,997][105620] Updated weights for policy 1, policy_version 1046815 (0.0005) [2023-12-26 22:59:32,367][105692] Updated weights for policy 0, policy_version 1046049 (0.0011) [2023-12-26 22:59:32,432][105692] Updated weights for policy 0, policy_version 1046059 (0.0010) [2023-12-26 22:59:32,500][105692] Updated weights for policy 0, policy_version 1046069 (0.0010) [2023-12-26 22:59:32,572][105692] Updated weights for policy 0, policy_version 1046079 (0.0010) [2023-12-26 22:59:32,622][105620] Updated weights for policy 1, policy_version 1046825 (0.0005) [2023-12-26 22:59:32,682][105620] Updated weights for policy 1, policy_version 1046835 (0.0005) [2023-12-26 22:59:32,748][105620] Updated weights for policy 1, policy_version 1046845 (0.0009) [2023-12-26 22:59:32,802][105620] Updated weights for policy 1, policy_version 1046855 (0.0008) [2023-12-26 22:59:33,237][105692] Updated weights for policy 0, policy_version 1046089 (0.0007) [2023-12-26 22:59:33,300][105692] Updated weights for policy 0, policy_version 1046099 (0.0009) [2023-12-26 22:59:33,355][105692] Updated weights for policy 0, policy_version 1046109 (0.0005) [2023-12-26 22:59:33,603][105620] Updated weights for policy 1, policy_version 1046865 (0.0008) [2023-12-26 22:59:33,656][105620] Updated weights for policy 1, policy_version 1046875 (0.0010) [2023-12-26 22:59:33,708][105620] Updated weights for policy 1, policy_version 1046885 (0.0009) [2023-12-26 22:59:33,913][105692] Updated weights for policy 0, policy_version 1046119 (0.0009) [2023-12-26 22:59:33,974][105692] Updated weights for policy 0, policy_version 1046129 (0.0010) [2023-12-26 22:59:34,036][105692] Updated weights for policy 0, policy_version 1046139 (0.0009) [2023-12-26 22:59:34,488][105620] Updated weights for policy 1, policy_version 1046895 (0.0009) [2023-12-26 22:59:34,541][105620] Updated weights for policy 1, policy_version 1046905 (0.0009) [2023-12-26 22:59:34,593][105620] Updated weights for policy 1, policy_version 1046915 (0.0010) [2023-12-26 22:59:34,636][105692] Updated weights for policy 0, policy_version 1046149 (0.0005) [2023-12-26 22:59:34,688][105692] Updated weights for policy 0, policy_version 1046159 (0.0005) [2023-12-26 22:59:34,735][105692] Updated weights for policy 0, policy_version 1046169 (0.0005) [2023-12-26 22:59:35,332][105692] Updated weights for policy 0, policy_version 1046179 (0.0006) [2023-12-26 22:59:35,339][105620] Updated weights for policy 1, policy_version 1046925 (0.0007) [2023-12-26 22:59:35,379][105692] Updated weights for policy 0, policy_version 1046189 (0.0006) [2023-12-26 22:59:35,400][105620] Updated weights for policy 1, policy_version 1046935 (0.0005) [2023-12-26 22:59:35,439][105692] Updated weights for policy 0, policy_version 1046199 (0.0011) [2023-12-26 22:59:35,465][105620] Updated weights for policy 1, policy_version 1046945 (0.0008) [2023-12-26 22:59:36,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 535920640. Throughput: 0: 10155.6, 1: 9784.0. Samples: 535909920. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:59:36,063][104569] Avg episode reward: [(0, '8725.893'), (1, '9078.924')] [2023-12-26 22:59:36,085][105692] Updated weights for policy 0, policy_version 1046209 (0.0010) [2023-12-26 22:59:36,107][105620] Updated weights for policy 1, policy_version 1046955 (0.0008) [2023-12-26 22:59:36,149][105692] Updated weights for policy 0, policy_version 1046219 (0.0008) [2023-12-26 22:59:36,167][105620] Updated weights for policy 1, policy_version 1046965 (0.0007) [2023-12-26 22:59:36,207][105692] Updated weights for policy 0, policy_version 1046229 (0.0008) [2023-12-26 22:59:36,232][105620] Updated weights for policy 1, policy_version 1046975 (0.0006) [2023-12-26 22:59:36,277][105692] Updated weights for policy 0, policy_version 1046239 (0.0009) [2023-12-26 22:59:36,871][105692] Updated weights for policy 0, policy_version 1046249 (0.0007) [2023-12-26 22:59:36,921][105692] Updated weights for policy 0, policy_version 1046259 (0.0008) [2023-12-26 22:59:36,936][105620] Updated weights for policy 1, policy_version 1046985 (0.0006) [2023-12-26 22:59:36,982][105692] Updated weights for policy 0, policy_version 1046269 (0.0007) [2023-12-26 22:59:36,995][105620] Updated weights for policy 1, policy_version 1046995 (0.0010) [2023-12-26 22:59:37,060][105620] Updated weights for policy 1, policy_version 1047005 (0.0010) [2023-12-26 22:59:37,126][105620] Updated weights for policy 1, policy_version 1047015 (0.0010) [2023-12-26 22:59:37,639][105692] Updated weights for policy 0, policy_version 1046279 (0.0009) [2023-12-26 22:59:37,693][105692] Updated weights for policy 0, policy_version 1046289 (0.0008) [2023-12-26 22:59:37,749][105692] Updated weights for policy 0, policy_version 1046299 (0.0008) [2023-12-26 22:59:37,850][105620] Updated weights for policy 1, policy_version 1047025 (0.0006) [2023-12-26 22:59:37,914][105620] Updated weights for policy 1, policy_version 1047035 (0.0005) [2023-12-26 22:59:37,980][105620] Updated weights for policy 1, policy_version 1047045 (0.0005) [2023-12-26 22:59:38,532][105620] Updated weights for policy 1, policy_version 1047055 (0.0008) [2023-12-26 22:59:38,592][105620] Updated weights for policy 1, policy_version 1047065 (0.0008) [2023-12-26 22:59:38,603][105692] Updated weights for policy 0, policy_version 1046309 (0.0007) [2023-12-26 22:59:38,651][105620] Updated weights for policy 1, policy_version 1047075 (0.0008) [2023-12-26 22:59:38,661][105692] Updated weights for policy 0, policy_version 1046319 (0.0008) [2023-12-26 22:59:38,715][105692] Updated weights for policy 0, policy_version 1046329 (0.0009) [2023-12-26 22:59:39,367][105620] Updated weights for policy 1, policy_version 1047085 (0.0008) [2023-12-26 22:59:39,433][105620] Updated weights for policy 1, policy_version 1047095 (0.0008) [2023-12-26 22:59:39,487][105620] Updated weights for policy 1, policy_version 1047105 (0.0009) [2023-12-26 22:59:39,531][105692] Updated weights for policy 0, policy_version 1046339 (0.0009) [2023-12-26 22:59:39,594][105692] Updated weights for policy 0, policy_version 1046349 (0.0009) [2023-12-26 22:59:39,653][105692] Updated weights for policy 0, policy_version 1046359 (0.0009) [2023-12-26 22:59:40,225][105620] Updated weights for policy 1, policy_version 1047115 (0.0008) [2023-12-26 22:59:40,291][105620] Updated weights for policy 1, policy_version 1047125 (0.0006) [2023-12-26 22:59:40,348][105620] Updated weights for policy 1, policy_version 1047135 (0.0010) [2023-12-26 22:59:40,488][105692] Updated weights for policy 0, policy_version 1046369 (0.0010) [2023-12-26 22:59:40,550][105692] Updated weights for policy 0, policy_version 1046379 (0.0008) [2023-12-26 22:59:40,602][105692] Updated weights for policy 0, policy_version 1046389 (0.0008) [2023-12-26 22:59:40,658][105692] Updated weights for policy 0, policy_version 1046399 (0.0008) [2023-12-26 22:59:41,051][105620] Updated weights for policy 1, policy_version 1047145 (0.0006) [2023-12-26 22:59:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19933.9, 300 sec: 19494.2). Total num frames: 536018944. Throughput: 0: 10071.5, 1: 9816.3. Samples: 536029800. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:59:41,062][104569] Avg episode reward: [(0, '8905.119'), (1, '8895.522')] [2023-12-26 22:59:41,127][105620] Updated weights for policy 1, policy_version 1047155 (0.0006) [2023-12-26 22:59:41,191][105620] Updated weights for policy 1, policy_version 1047165 (0.0008) [2023-12-26 22:59:41,259][105620] Updated weights for policy 1, policy_version 1047175 (0.0008) [2023-12-26 22:59:41,434][105692] Updated weights for policy 0, policy_version 1046409 (0.0009) [2023-12-26 22:59:41,496][105692] Updated weights for policy 0, policy_version 1046419 (0.0009) [2023-12-26 22:59:41,554][105692] Updated weights for policy 0, policy_version 1046429 (0.0008) [2023-12-26 22:59:42,063][105620] Updated weights for policy 1, policy_version 1047185 (0.0006) [2023-12-26 22:59:42,122][105620] Updated weights for policy 1, policy_version 1047195 (0.0007) [2023-12-26 22:59:42,184][105620] Updated weights for policy 1, policy_version 1047205 (0.0008) [2023-12-26 22:59:42,293][105692] Updated weights for policy 0, policy_version 1046439 (0.0009) [2023-12-26 22:59:42,362][105692] Updated weights for policy 0, policy_version 1046449 (0.0010) [2023-12-26 22:59:42,421][105692] Updated weights for policy 0, policy_version 1046459 (0.0009) [2023-12-26 22:59:42,829][105620] Updated weights for policy 1, policy_version 1047215 (0.0007) [2023-12-26 22:59:42,887][105620] Updated weights for policy 1, policy_version 1047225 (0.0007) [2023-12-26 22:59:42,938][105620] Updated weights for policy 1, policy_version 1047236 (0.0008) [2023-12-26 22:59:43,151][105692] Updated weights for policy 0, policy_version 1046469 (0.0008) [2023-12-26 22:59:43,205][105692] Updated weights for policy 0, policy_version 1046479 (0.0009) [2023-12-26 22:59:43,273][105692] Updated weights for policy 0, policy_version 1046489 (0.0010) [2023-12-26 22:59:43,520][105620] Updated weights for policy 1, policy_version 1047246 (0.0005) [2023-12-26 22:59:43,572][105620] Updated weights for policy 1, policy_version 1047256 (0.0008) [2023-12-26 22:59:43,629][105620] Updated weights for policy 1, policy_version 1047266 (0.0007) [2023-12-26 22:59:43,832][105692] Updated weights for policy 0, policy_version 1046499 (0.0009) [2023-12-26 22:59:43,900][105692] Updated weights for policy 0, policy_version 1046509 (0.0005) [2023-12-26 22:59:43,950][105692] Updated weights for policy 0, policy_version 1046519 (0.0005) [2023-12-26 22:59:44,219][105620] Updated weights for policy 1, policy_version 1047276 (0.0008) [2023-12-26 22:59:44,285][105620] Updated weights for policy 1, policy_version 1047286 (0.0006) [2023-12-26 22:59:44,350][105620] Updated weights for policy 1, policy_version 1047296 (0.0006) [2023-12-26 22:59:44,573][105692] Updated weights for policy 0, policy_version 1046529 (0.0006) [2023-12-26 22:59:44,636][105692] Updated weights for policy 0, policy_version 1046539 (0.0010) [2023-12-26 22:59:44,701][105692] Updated weights for policy 0, policy_version 1046549 (0.0010) [2023-12-26 22:59:44,764][105692] Updated weights for policy 0, policy_version 1046559 (0.0010) [2023-12-26 22:59:44,880][105620] Updated weights for policy 1, policy_version 1047306 (0.0006) [2023-12-26 22:59:44,942][105620] Updated weights for policy 1, policy_version 1047316 (0.0006) [2023-12-26 22:59:45,004][105620] Updated weights for policy 1, policy_version 1047326 (0.0008) [2023-12-26 22:59:45,071][105620] Updated weights for policy 1, policy_version 1047336 (0.0009) [2023-12-26 22:59:45,510][105692] Updated weights for policy 0, policy_version 1046569 (0.0011) [2023-12-26 22:59:45,562][105692] Updated weights for policy 0, policy_version 1046579 (0.0010) [2023-12-26 22:59:45,610][105692] Updated weights for policy 0, policy_version 1046589 (0.0010) [2023-12-26 22:59:45,736][105620] Updated weights for policy 1, policy_version 1047346 (0.0008) [2023-12-26 22:59:45,787][105620] Updated weights for policy 1, policy_version 1047356 (0.0009) [2023-12-26 22:59:45,844][105620] Updated weights for policy 1, policy_version 1047366 (0.0009) [2023-12-26 22:59:46,062][104569] Fps is (10 sec: 20480.0, 60 sec: 20070.3, 300 sec: 19494.2). Total num frames: 536125440. Throughput: 0: 10050.8, 1: 9863.1. Samples: 536090252. Policy #0 lag: (min: 26.0, avg: 38.6, max: 58.0) [2023-12-26 22:59:46,063][104569] Avg episode reward: [(0, '9172.871'), (1, '9077.066')] [2023-12-26 22:59:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001047368_268156928.pth... [2023-12-26 22:59:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001046592_267968512.pth... [2023-12-26 22:59:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001045408_267665408.pth [2023-12-26 22:59:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001046184_267853824.pth [2023-12-26 22:59:46,212][105692] Updated weights for policy 0, policy_version 1046599 (0.0007) [2023-12-26 22:59:46,262][105692] Updated weights for policy 0, policy_version 1046609 (0.0005) [2023-12-26 22:59:46,311][105692] Updated weights for policy 0, policy_version 1046619 (0.0005) [2023-12-26 22:59:46,758][105620] Updated weights for policy 1, policy_version 1047376 (0.0009) [2023-12-26 22:59:46,816][105620] Updated weights for policy 1, policy_version 1047386 (0.0010) [2023-12-26 22:59:46,832][105692] Updated weights for policy 0, policy_version 1046629 (0.0008) [2023-12-26 22:59:46,867][105620] Updated weights for policy 1, policy_version 1047396 (0.0008) [2023-12-26 22:59:46,881][105692] Updated weights for policy 0, policy_version 1046639 (0.0010) [2023-12-26 22:59:46,925][105692] Updated weights for policy 0, policy_version 1046649 (0.0006) [2023-12-26 22:59:47,653][105692] Updated weights for policy 0, policy_version 1046659 (0.0008) [2023-12-26 22:59:47,663][105620] Updated weights for policy 1, policy_version 1047406 (0.0009) [2023-12-26 22:59:47,710][105692] Updated weights for policy 0, policy_version 1046669 (0.0006) [2023-12-26 22:59:47,723][105620] Updated weights for policy 1, policy_version 1047416 (0.0007) [2023-12-26 22:59:47,768][105692] Updated weights for policy 0, policy_version 1046679 (0.0007) [2023-12-26 22:59:47,781][105620] Updated weights for policy 1, policy_version 1047426 (0.0010) [2023-12-26 22:59:48,493][105620] Updated weights for policy 1, policy_version 1047436 (0.0008) [2023-12-26 22:59:48,540][105692] Updated weights for policy 0, policy_version 1046689 (0.0008) [2023-12-26 22:59:48,547][105620] Updated weights for policy 1, policy_version 1047446 (0.0005) [2023-12-26 22:59:48,596][105692] Updated weights for policy 0, policy_version 1046699 (0.0007) [2023-12-26 22:59:48,602][105620] Updated weights for policy 1, policy_version 1047456 (0.0006) [2023-12-26 22:59:48,656][105692] Updated weights for policy 0, policy_version 1046709 (0.0007) [2023-12-26 22:59:48,710][105692] Updated weights for policy 0, policy_version 1046719 (0.0009) [2023-12-26 22:59:49,320][105620] Updated weights for policy 1, policy_version 1047466 (0.0007) [2023-12-26 22:59:49,392][105620] Updated weights for policy 1, policy_version 1047476 (0.0008) [2023-12-26 22:59:49,446][105620] Updated weights for policy 1, policy_version 1047486 (0.0009) [2023-12-26 22:59:49,501][105692] Updated weights for policy 0, policy_version 1046729 (0.0007) [2023-12-26 22:59:49,508][105620] Updated weights for policy 1, policy_version 1047496 (0.0008) [2023-12-26 22:59:49,556][105692] Updated weights for policy 0, policy_version 1046739 (0.0008) [2023-12-26 22:59:49,611][105692] Updated weights for policy 0, policy_version 1046749 (0.0009) [2023-12-26 22:59:50,259][105620] Updated weights for policy 1, policy_version 1047506 (0.0009) [2023-12-26 22:59:50,318][105620] Updated weights for policy 1, policy_version 1047516 (0.0009) [2023-12-26 22:59:50,377][105620] Updated weights for policy 1, policy_version 1047526 (0.0009) [2023-12-26 22:59:50,392][105692] Updated weights for policy 0, policy_version 1046759 (0.0007) [2023-12-26 22:59:50,442][105692] Updated weights for policy 0, policy_version 1046769 (0.0009) [2023-12-26 22:59:50,498][105692] Updated weights for policy 0, policy_version 1046779 (0.0009) [2023-12-26 22:59:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19933.8, 300 sec: 19494.2). Total num frames: 536215552. Throughput: 0: 10122.2, 1: 9860.7. Samples: 536209692. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 22:59:51,063][104569] Avg episode reward: [(0, '8814.468'), (1, '9168.207')] [2023-12-26 22:59:51,163][105620] Updated weights for policy 1, policy_version 1047536 (0.0008) [2023-12-26 22:59:51,221][105620] Updated weights for policy 1, policy_version 1047546 (0.0009) [2023-12-26 22:59:51,274][105692] Updated weights for policy 0, policy_version 1046789 (0.0009) [2023-12-26 22:59:51,284][105620] Updated weights for policy 1, policy_version 1047556 (0.0008) [2023-12-26 22:59:51,332][105692] Updated weights for policy 0, policy_version 1046799 (0.0007) [2023-12-26 22:59:51,405][105692] Updated weights for policy 0, policy_version 1046809 (0.0008) [2023-12-26 22:59:52,003][105620] Updated weights for policy 1, policy_version 1047566 (0.0009) [2023-12-26 22:59:52,068][105620] Updated weights for policy 1, policy_version 1047576 (0.0009) [2023-12-26 22:59:52,088][105692] Updated weights for policy 0, policy_version 1046819 (0.0007) [2023-12-26 22:59:52,125][105620] Updated weights for policy 1, policy_version 1047586 (0.0009) [2023-12-26 22:59:52,143][105692] Updated weights for policy 0, policy_version 1046829 (0.0006) [2023-12-26 22:59:52,197][105692] Updated weights for policy 0, policy_version 1046839 (0.0008) [2023-12-26 22:59:52,889][105620] Updated weights for policy 1, policy_version 1047596 (0.0008) [2023-12-26 22:59:52,949][105620] Updated weights for policy 1, policy_version 1047606 (0.0009) [2023-12-26 22:59:52,963][105692] Updated weights for policy 0, policy_version 1046849 (0.0009) [2023-12-26 22:59:52,997][105620] Updated weights for policy 1, policy_version 1047616 (0.0009) [2023-12-26 22:59:53,016][105692] Updated weights for policy 0, policy_version 1046859 (0.0007) [2023-12-26 22:59:53,078][105692] Updated weights for policy 0, policy_version 1046869 (0.0008) [2023-12-26 22:59:53,132][105692] Updated weights for policy 0, policy_version 1046879 (0.0009) [2023-12-26 22:59:53,757][105620] Updated weights for policy 1, policy_version 1047626 (0.0007) [2023-12-26 22:59:53,818][105620] Updated weights for policy 1, policy_version 1047636 (0.0009) [2023-12-26 22:59:53,876][105620] Updated weights for policy 1, policy_version 1047646 (0.0007) [2023-12-26 22:59:53,898][105692] Updated weights for policy 0, policy_version 1046889 (0.0008) [2023-12-26 22:59:53,925][105620] Updated weights for policy 1, policy_version 1047656 (0.0007) [2023-12-26 22:59:53,955][105692] Updated weights for policy 0, policy_version 1046899 (0.0008) [2023-12-26 22:59:54,007][105692] Updated weights for policy 0, policy_version 1046909 (0.0009) [2023-12-26 22:59:54,698][105620] Updated weights for policy 1, policy_version 1047666 (0.0009) [2023-12-26 22:59:54,713][105692] Updated weights for policy 0, policy_version 1046919 (0.0009) [2023-12-26 22:59:54,747][105620] Updated weights for policy 1, policy_version 1047676 (0.0006) [2023-12-26 22:59:54,765][105692] Updated weights for policy 0, policy_version 1046929 (0.0007) [2023-12-26 22:59:54,792][105620] Updated weights for policy 1, policy_version 1047686 (0.0006) [2023-12-26 22:59:54,819][105692] Updated weights for policy 0, policy_version 1046939 (0.0008) [2023-12-26 22:59:55,528][105620] Updated weights for policy 1, policy_version 1047696 (0.0008) [2023-12-26 22:59:55,581][105620] Updated weights for policy 1, policy_version 1047706 (0.0008) [2023-12-26 22:59:55,602][105692] Updated weights for policy 0, policy_version 1046949 (0.0009) [2023-12-26 22:59:55,636][105620] Updated weights for policy 1, policy_version 1047716 (0.0009) [2023-12-26 22:59:55,647][105692] Updated weights for policy 0, policy_version 1046959 (0.0006) [2023-12-26 22:59:55,699][105692] Updated weights for policy 0, policy_version 1046969 (0.0009) [2023-12-26 22:59:56,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19797.4, 300 sec: 19494.2). Total num frames: 536313856. Throughput: 0: 10117.5, 1: 9769.0. Samples: 536321344. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 22:59:56,062][104569] Avg episode reward: [(0, '8727.947'), (1, '9168.555')] [2023-12-26 22:59:56,410][105620] Updated weights for policy 1, policy_version 1047726 (0.0009) [2023-12-26 22:59:56,461][105620] Updated weights for policy 1, policy_version 1047736 (0.0006) [2023-12-26 22:59:56,480][105692] Updated weights for policy 0, policy_version 1046979 (0.0009) [2023-12-26 22:59:56,511][105620] Updated weights for policy 1, policy_version 1047746 (0.0006) [2023-12-26 22:59:56,529][105692] Updated weights for policy 0, policy_version 1046989 (0.0006) [2023-12-26 22:59:56,574][105692] Updated weights for policy 0, policy_version 1046999 (0.0009) [2023-12-26 22:59:57,238][105620] Updated weights for policy 1, policy_version 1047756 (0.0008) [2023-12-26 22:59:57,288][105620] Updated weights for policy 1, policy_version 1047766 (0.0009) [2023-12-26 22:59:57,322][105692] Updated weights for policy 0, policy_version 1047009 (0.0009) [2023-12-26 22:59:57,340][105620] Updated weights for policy 1, policy_version 1047776 (0.0008) [2023-12-26 22:59:57,378][105692] Updated weights for policy 0, policy_version 1047019 (0.0008) [2023-12-26 22:59:57,424][105692] Updated weights for policy 0, policy_version 1047029 (0.0008) [2023-12-26 22:59:57,473][105692] Updated weights for policy 0, policy_version 1047039 (0.0008) [2023-12-26 22:59:58,095][105620] Updated weights for policy 1, policy_version 1047786 (0.0007) [2023-12-26 22:59:58,153][105620] Updated weights for policy 1, policy_version 1047796 (0.0008) [2023-12-26 22:59:58,215][105620] Updated weights for policy 1, policy_version 1047806 (0.0007) [2023-12-26 22:59:58,244][105692] Updated weights for policy 0, policy_version 1047049 (0.0008) [2023-12-26 22:59:58,277][105620] Updated weights for policy 1, policy_version 1047816 (0.0006) [2023-12-26 22:59:58,313][105692] Updated weights for policy 0, policy_version 1047059 (0.0008) [2023-12-26 22:59:58,381][105692] Updated weights for policy 0, policy_version 1047069 (0.0007) [2023-12-26 22:59:59,032][105620] Updated weights for policy 1, policy_version 1047826 (0.0008) [2023-12-26 22:59:59,096][105620] Updated weights for policy 1, policy_version 1047836 (0.0008) [2023-12-26 22:59:59,114][105692] Updated weights for policy 0, policy_version 1047079 (0.0008) [2023-12-26 22:59:59,150][105620] Updated weights for policy 1, policy_version 1047846 (0.0008) [2023-12-26 22:59:59,168][105692] Updated weights for policy 0, policy_version 1047089 (0.0006) [2023-12-26 22:59:59,220][105692] Updated weights for policy 0, policy_version 1047099 (0.0006) [2023-12-26 22:59:59,898][105620] Updated weights for policy 1, policy_version 1047856 (0.0008) [2023-12-26 22:59:59,964][105620] Updated weights for policy 1, policy_version 1047866 (0.0009) [2023-12-26 22:59:59,967][105692] Updated weights for policy 0, policy_version 1047109 (0.0011) [2023-12-26 23:00:00,015][105620] Updated weights for policy 1, policy_version 1047876 (0.0006) [2023-12-26 23:00:00,020][105692] Updated weights for policy 0, policy_version 1047119 (0.0011) [2023-12-26 23:00:00,077][105692] Updated weights for policy 0, policy_version 1047129 (0.0011) [2023-12-26 23:00:00,694][105620] Updated weights for policy 1, policy_version 1047886 (0.0007) [2023-12-26 23:00:00,751][105620] Updated weights for policy 1, policy_version 1047896 (0.0008) [2023-12-26 23:00:00,812][105620] Updated weights for policy 1, policy_version 1047906 (0.0007) [2023-12-26 23:00:00,828][105692] Updated weights for policy 0, policy_version 1047139 (0.0011) [2023-12-26 23:00:00,881][105692] Updated weights for policy 0, policy_version 1047150 (0.0008) [2023-12-26 23:00:00,927][105692] Updated weights for policy 0, policy_version 1047160 (0.0005) [2023-12-26 23:00:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 536412160. Throughput: 0: 10063.0, 1: 9801.2. Samples: 536377576. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 23:00:01,063][104569] Avg episode reward: [(0, '9084.800'), (1, '9168.862')] [2023-12-26 23:00:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001047168_268115968.pth... [2023-12-26 23:00:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001047912_268296192.pth... [2023-12-26 23:00:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001046016_267821056.pth [2023-12-26 23:00:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001046792_268009472.pth [2023-12-26 23:00:01,532][105620] Updated weights for policy 1, policy_version 1047916 (0.0006) [2023-12-26 23:00:01,587][105620] Updated weights for policy 1, policy_version 1047926 (0.0008) [2023-12-26 23:00:01,626][105692] Updated weights for policy 0, policy_version 1047170 (0.0006) [2023-12-26 23:00:01,653][105620] Updated weights for policy 1, policy_version 1047936 (0.0008) [2023-12-26 23:00:01,687][105692] Updated weights for policy 0, policy_version 1047180 (0.0007) [2023-12-26 23:00:01,748][105692] Updated weights for policy 0, policy_version 1047190 (0.0008) [2023-12-26 23:00:01,812][105692] Updated weights for policy 0, policy_version 1047200 (0.0005) [2023-12-26 23:00:02,399][105620] Updated weights for policy 1, policy_version 1047946 (0.0009) [2023-12-26 23:00:02,403][105692] Updated weights for policy 0, policy_version 1047210 (0.0006) [2023-12-26 23:00:02,453][105692] Updated weights for policy 0, policy_version 1047220 (0.0006) [2023-12-26 23:00:02,457][105620] Updated weights for policy 1, policy_version 1047956 (0.0010) [2023-12-26 23:00:02,503][105692] Updated weights for policy 0, policy_version 1047230 (0.0008) [2023-12-26 23:00:02,512][105620] Updated weights for policy 1, policy_version 1047966 (0.0010) [2023-12-26 23:00:02,567][105620] Updated weights for policy 1, policy_version 1047976 (0.0006) [2023-12-26 23:00:03,142][105620] Updated weights for policy 1, policy_version 1047986 (0.0005) [2023-12-26 23:00:03,212][105620] Updated weights for policy 1, policy_version 1047996 (0.0007) [2023-12-26 23:00:03,275][105692] Updated weights for policy 0, policy_version 1047240 (0.0006) [2023-12-26 23:00:03,277][105620] Updated weights for policy 1, policy_version 1048006 (0.0009) [2023-12-26 23:00:03,335][105692] Updated weights for policy 0, policy_version 1047250 (0.0007) [2023-12-26 23:00:03,393][105692] Updated weights for policy 0, policy_version 1047260 (0.0008) [2023-12-26 23:00:03,877][105620] Updated weights for policy 1, policy_version 1048016 (0.0006) [2023-12-26 23:00:03,929][105620] Updated weights for policy 1, policy_version 1048026 (0.0006) [2023-12-26 23:00:03,976][105620] Updated weights for policy 1, policy_version 1048036 (0.0005) [2023-12-26 23:00:04,074][105692] Updated weights for policy 0, policy_version 1047270 (0.0009) [2023-12-26 23:00:04,130][105692] Updated weights for policy 0, policy_version 1047280 (0.0010) [2023-12-26 23:00:04,189][105692] Updated weights for policy 0, policy_version 1047290 (0.0009) [2023-12-26 23:00:04,586][105620] Updated weights for policy 1, policy_version 1048046 (0.0006) [2023-12-26 23:00:04,650][105620] Updated weights for policy 1, policy_version 1048056 (0.0007) [2023-12-26 23:00:04,697][105620] Updated weights for policy 1, policy_version 1048066 (0.0005) [2023-12-26 23:00:04,967][105692] Updated weights for policy 0, policy_version 1047300 (0.0008) [2023-12-26 23:00:05,024][105692] Updated weights for policy 0, policy_version 1047310 (0.0008) [2023-12-26 23:00:05,076][105692] Updated weights for policy 0, policy_version 1047320 (0.0008) [2023-12-26 23:00:05,380][105620] Updated weights for policy 1, policy_version 1048076 (0.0007) [2023-12-26 23:00:05,441][105620] Updated weights for policy 1, policy_version 1048086 (0.0010) [2023-12-26 23:00:05,499][105620] Updated weights for policy 1, policy_version 1048096 (0.0010) [2023-12-26 23:00:05,828][105692] Updated weights for policy 0, policy_version 1047330 (0.0007) [2023-12-26 23:00:05,876][105692] Updated weights for policy 0, policy_version 1047340 (0.0008) [2023-12-26 23:00:05,935][105692] Updated weights for policy 0, policy_version 1047350 (0.0008) [2023-12-26 23:00:05,988][105692] Updated weights for policy 0, policy_version 1047360 (0.0008) [2023-12-26 23:00:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19522.0). Total num frames: 536510464. Throughput: 0: 9991.9, 1: 9853.6. Samples: 536497784. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 23:00:06,062][104569] Avg episode reward: [(0, '9262.190'), (1, '8997.682')] [2023-12-26 23:00:06,240][105620] Updated weights for policy 1, policy_version 1048106 (0.0010) [2023-12-26 23:00:06,310][105620] Updated weights for policy 1, policy_version 1048116 (0.0010) [2023-12-26 23:00:06,378][105620] Updated weights for policy 1, policy_version 1048126 (0.0007) [2023-12-26 23:00:06,448][105620] Updated weights for policy 1, policy_version 1048136 (0.0007) [2023-12-26 23:00:06,758][105692] Updated weights for policy 0, policy_version 1047370 (0.0008) [2023-12-26 23:00:06,817][105692] Updated weights for policy 0, policy_version 1047380 (0.0008) [2023-12-26 23:00:06,877][105692] Updated weights for policy 0, policy_version 1047390 (0.0008) [2023-12-26 23:00:07,060][105620] Updated weights for policy 1, policy_version 1048146 (0.0010) [2023-12-26 23:00:07,132][105620] Updated weights for policy 1, policy_version 1048156 (0.0007) [2023-12-26 23:00:07,191][105620] Updated weights for policy 1, policy_version 1048166 (0.0006) [2023-12-26 23:00:07,621][105692] Updated weights for policy 0, policy_version 1047400 (0.0010) [2023-12-26 23:00:07,674][105692] Updated weights for policy 0, policy_version 1047410 (0.0010) [2023-12-26 23:00:07,729][105692] Updated weights for policy 0, policy_version 1047420 (0.0010) [2023-12-26 23:00:07,886][105620] Updated weights for policy 1, policy_version 1048176 (0.0006) [2023-12-26 23:00:07,937][105620] Updated weights for policy 1, policy_version 1048186 (0.0009) [2023-12-26 23:00:07,985][105620] Updated weights for policy 1, policy_version 1048196 (0.0010) [2023-12-26 23:00:08,507][105692] Updated weights for policy 0, policy_version 1047430 (0.0010) [2023-12-26 23:00:08,559][105692] Updated weights for policy 0, policy_version 1047440 (0.0010) [2023-12-26 23:00:08,609][105692] Updated weights for policy 0, policy_version 1047450 (0.0008) [2023-12-26 23:00:08,718][105620] Updated weights for policy 1, policy_version 1048206 (0.0010) [2023-12-26 23:00:08,765][105620] Updated weights for policy 1, policy_version 1048216 (0.0010) [2023-12-26 23:00:08,813][105620] Updated weights for policy 1, policy_version 1048226 (0.0010) [2023-12-26 23:00:09,279][105692] Updated weights for policy 0, policy_version 1047460 (0.0008) [2023-12-26 23:00:09,340][105692] Updated weights for policy 0, policy_version 1047470 (0.0011) [2023-12-26 23:00:09,410][105692] Updated weights for policy 0, policy_version 1047480 (0.0011) [2023-12-26 23:00:09,471][105620] Updated weights for policy 1, policy_version 1048236 (0.0009) [2023-12-26 23:00:09,537][105620] Updated weights for policy 1, policy_version 1048246 (0.0006) [2023-12-26 23:00:09,606][105620] Updated weights for policy 1, policy_version 1048256 (0.0009) [2023-12-26 23:00:10,086][105692] Updated weights for policy 0, policy_version 1047490 (0.0008) [2023-12-26 23:00:10,143][105692] Updated weights for policy 0, policy_version 1047500 (0.0008) [2023-12-26 23:00:10,198][105692] Updated weights for policy 0, policy_version 1047510 (0.0008) [2023-12-26 23:00:10,248][105692] Updated weights for policy 0, policy_version 1047520 (0.0008) [2023-12-26 23:00:10,349][105620] Updated weights for policy 1, policy_version 1048266 (0.0011) [2023-12-26 23:00:10,415][105620] Updated weights for policy 1, policy_version 1048276 (0.0010) [2023-12-26 23:00:10,484][105620] Updated weights for policy 1, policy_version 1048286 (0.0010) [2023-12-26 23:00:10,533][105620] Updated weights for policy 1, policy_version 1048296 (0.0008) [2023-12-26 23:00:10,874][105692] Updated weights for policy 0, policy_version 1047530 (0.0007) [2023-12-26 23:00:10,920][105692] Updated weights for policy 0, policy_version 1047540 (0.0009) [2023-12-26 23:00:10,976][105692] Updated weights for policy 0, policy_version 1047550 (0.0008) [2023-12-26 23:00:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19933.9, 300 sec: 19522.0). Total num frames: 536608768. Throughput: 0: 9941.6, 1: 9878.2. Samples: 536614536. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 23:00:11,063][104569] Avg episode reward: [(0, '9264.092'), (1, '8907.459')] [2023-12-26 23:00:11,339][105620] Updated weights for policy 1, policy_version 1048306 (0.0009) [2023-12-26 23:00:11,409][105620] Updated weights for policy 1, policy_version 1048316 (0.0009) [2023-12-26 23:00:11,474][105620] Updated weights for policy 1, policy_version 1048326 (0.0008) [2023-12-26 23:00:11,746][105692] Updated weights for policy 0, policy_version 1047560 (0.0009) [2023-12-26 23:00:11,805][105692] Updated weights for policy 0, policy_version 1047570 (0.0009) [2023-12-26 23:00:11,853][105692] Updated weights for policy 0, policy_version 1047580 (0.0008) [2023-12-26 23:00:12,225][105620] Updated weights for policy 1, policy_version 1048336 (0.0008) [2023-12-26 23:00:12,287][105620] Updated weights for policy 1, policy_version 1048346 (0.0006) [2023-12-26 23:00:12,350][105620] Updated weights for policy 1, policy_version 1048356 (0.0009) [2023-12-26 23:00:12,669][105692] Updated weights for policy 0, policy_version 1047590 (0.0010) [2023-12-26 23:00:12,728][105692] Updated weights for policy 0, policy_version 1047600 (0.0008) [2023-12-26 23:00:12,788][105692] Updated weights for policy 0, policy_version 1047610 (0.0009) [2023-12-26 23:00:13,054][105620] Updated weights for policy 1, policy_version 1048366 (0.0009) [2023-12-26 23:00:13,120][105620] Updated weights for policy 1, policy_version 1048376 (0.0010) [2023-12-26 23:00:13,177][105620] Updated weights for policy 1, policy_version 1048386 (0.0009) [2023-12-26 23:00:13,451][105692] Updated weights for policy 0, policy_version 1047620 (0.0010) [2023-12-26 23:00:13,506][105692] Updated weights for policy 0, policy_version 1047630 (0.0010) [2023-12-26 23:00:13,566][105692] Updated weights for policy 0, policy_version 1047640 (0.0010) [2023-12-26 23:00:13,916][105620] Updated weights for policy 1, policy_version 1048396 (0.0007) [2023-12-26 23:00:13,981][105620] Updated weights for policy 1, policy_version 1048406 (0.0005) [2023-12-26 23:00:14,044][105620] Updated weights for policy 1, policy_version 1048416 (0.0008) [2023-12-26 23:00:14,215][105692] Updated weights for policy 0, policy_version 1047650 (0.0009) [2023-12-26 23:00:14,274][105692] Updated weights for policy 0, policy_version 1047660 (0.0008) [2023-12-26 23:00:14,332][105692] Updated weights for policy 0, policy_version 1047670 (0.0009) [2023-12-26 23:00:14,400][105692] Updated weights for policy 0, policy_version 1047680 (0.0010) [2023-12-26 23:00:14,721][105620] Updated weights for policy 1, policy_version 1048426 (0.0007) [2023-12-26 23:00:14,787][105620] Updated weights for policy 1, policy_version 1048436 (0.0007) [2023-12-26 23:00:14,845][105620] Updated weights for policy 1, policy_version 1048446 (0.0009) [2023-12-26 23:00:14,897][105620] Updated weights for policy 1, policy_version 1048456 (0.0008) [2023-12-26 23:00:14,961][105692] Updated weights for policy 0, policy_version 1047690 (0.0011) [2023-12-26 23:00:15,010][105692] Updated weights for policy 0, policy_version 1047700 (0.0011) [2023-12-26 23:00:15,070][105692] Updated weights for policy 0, policy_version 1047710 (0.0011) [2023-12-26 23:00:15,598][105620] Updated weights for policy 1, policy_version 1048466 (0.0008) [2023-12-26 23:00:15,654][105620] Updated weights for policy 1, policy_version 1048476 (0.0008) [2023-12-26 23:00:15,708][105620] Updated weights for policy 1, policy_version 1048486 (0.0008) [2023-12-26 23:00:15,836][105692] Updated weights for policy 0, policy_version 1047720 (0.0010) [2023-12-26 23:00:15,884][105692] Updated weights for policy 0, policy_version 1047730 (0.0010) [2023-12-26 23:00:15,932][105692] Updated weights for policy 0, policy_version 1047740 (0.0010) [2023-12-26 23:00:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 536707072. Throughput: 0: 9806.2, 1: 9792.5. Samples: 536671452. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 23:00:16,062][104569] Avg episode reward: [(0, '9000.742'), (1, '8903.735')] [2023-12-26 23:00:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001047744_268263424.pth... [2023-12-26 23:00:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001048488_268443648.pth... [2023-12-26 23:00:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001047368_268156928.pth [2023-12-26 23:00:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001046592_267968512.pth [2023-12-26 23:00:16,504][105620] Updated weights for policy 1, policy_version 1048496 (0.0008) [2023-12-26 23:00:16,555][105620] Updated weights for policy 1, policy_version 1048506 (0.0008) [2023-12-26 23:00:16,606][105620] Updated weights for policy 1, policy_version 1048516 (0.0007) [2023-12-26 23:00:16,669][105692] Updated weights for policy 0, policy_version 1047750 (0.0010) [2023-12-26 23:00:16,718][105692] Updated weights for policy 0, policy_version 1047760 (0.0010) [2023-12-26 23:00:16,767][105692] Updated weights for policy 0, policy_version 1047770 (0.0010) [2023-12-26 23:00:17,267][105620] Updated weights for policy 1, policy_version 1048526 (0.0008) [2023-12-26 23:00:17,322][105620] Updated weights for policy 1, policy_version 1048536 (0.0008) [2023-12-26 23:00:17,377][105620] Updated weights for policy 1, policy_version 1048546 (0.0007) [2023-12-26 23:00:17,535][105692] Updated weights for policy 0, policy_version 1047780 (0.0010) [2023-12-26 23:00:17,587][105692] Updated weights for policy 0, policy_version 1047790 (0.0010) [2023-12-26 23:00:17,642][105692] Updated weights for policy 0, policy_version 1047800 (0.0010) [2023-12-26 23:00:17,980][105620] Updated weights for policy 1, policy_version 1048556 (0.0007) [2023-12-26 23:00:18,037][105620] Updated weights for policy 1, policy_version 1048566 (0.0006) [2023-12-26 23:00:18,102][105620] Updated weights for policy 1, policy_version 1048576 (0.0010) [2023-12-26 23:00:18,374][105692] Updated weights for policy 0, policy_version 1047810 (0.0010) [2023-12-26 23:00:18,433][105692] Updated weights for policy 0, policy_version 1047820 (0.0010) [2023-12-26 23:00:18,498][105692] Updated weights for policy 0, policy_version 1047830 (0.0010) [2023-12-26 23:00:18,557][105692] Updated weights for policy 0, policy_version 1047840 (0.0010) [2023-12-26 23:00:18,732][105620] Updated weights for policy 1, policy_version 1048586 (0.0007) [2023-12-26 23:00:18,787][105620] Updated weights for policy 1, policy_version 1048596 (0.0006) [2023-12-26 23:00:18,850][105620] Updated weights for policy 1, policy_version 1048606 (0.0005) [2023-12-26 23:00:18,913][105620] Updated weights for policy 1, policy_version 1048616 (0.0008) [2023-12-26 23:00:19,292][105692] Updated weights for policy 0, policy_version 1047850 (0.0011) [2023-12-26 23:00:19,360][105692] Updated weights for policy 0, policy_version 1047860 (0.0011) [2023-12-26 23:00:19,426][105692] Updated weights for policy 0, policy_version 1047870 (0.0009) [2023-12-26 23:00:19,650][105620] Updated weights for policy 1, policy_version 1048626 (0.0008) [2023-12-26 23:00:19,712][105620] Updated weights for policy 1, policy_version 1048636 (0.0008) [2023-12-26 23:00:19,779][105620] Updated weights for policy 1, policy_version 1048646 (0.0008) [2023-12-26 23:00:20,140][105692] Updated weights for policy 0, policy_version 1047880 (0.0008) [2023-12-26 23:00:20,217][105692] Updated weights for policy 0, policy_version 1047890 (0.0006) [2023-12-26 23:00:20,287][105692] Updated weights for policy 0, policy_version 1047900 (0.0005) [2023-12-26 23:00:20,492][105620] Updated weights for policy 1, policy_version 1048656 (0.0006) [2023-12-26 23:00:20,555][105620] Updated weights for policy 1, policy_version 1048666 (0.0006) [2023-12-26 23:00:20,612][105620] Updated weights for policy 1, policy_version 1048676 (0.0008) [2023-12-26 23:00:20,960][105692] Updated weights for policy 0, policy_version 1047910 (0.0006) [2023-12-26 23:00:21,019][105692] Updated weights for policy 0, policy_version 1047920 (0.0006) [2023-12-26 23:00:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 536797184. Throughput: 0: 9739.6, 1: 9835.9. Samples: 536790812. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 23:00:21,062][104569] Avg episode reward: [(0, '8730.034'), (1, '8989.753')] [2023-12-26 23:00:21,086][105692] Updated weights for policy 0, policy_version 1047930 (0.0008) [2023-12-26 23:00:21,350][105620] Updated weights for policy 1, policy_version 1048686 (0.0008) [2023-12-26 23:00:21,425][105620] Updated weights for policy 1, policy_version 1048696 (0.0009) [2023-12-26 23:00:21,476][105620] Updated weights for policy 1, policy_version 1048706 (0.0005) [2023-12-26 23:00:21,804][105692] Updated weights for policy 0, policy_version 1047940 (0.0008) [2023-12-26 23:00:21,856][105692] Updated weights for policy 0, policy_version 1047950 (0.0008) [2023-12-26 23:00:21,913][105692] Updated weights for policy 0, policy_version 1047960 (0.0008) [2023-12-26 23:00:22,098][105620] Updated weights for policy 1, policy_version 1048716 (0.0005) [2023-12-26 23:00:22,154][105620] Updated weights for policy 1, policy_version 1048726 (0.0006) [2023-12-26 23:00:22,220][105620] Updated weights for policy 1, policy_version 1048736 (0.0007) [2023-12-26 23:00:22,737][105692] Updated weights for policy 0, policy_version 1047970 (0.0008) [2023-12-26 23:00:22,799][105692] Updated weights for policy 0, policy_version 1047980 (0.0009) [2023-12-26 23:00:22,850][105692] Updated weights for policy 0, policy_version 1047990 (0.0009) [2023-12-26 23:00:22,898][105692] Updated weights for policy 0, policy_version 1048000 (0.0009) [2023-12-26 23:00:22,935][105620] Updated weights for policy 1, policy_version 1048746 (0.0009) [2023-12-26 23:00:22,994][105620] Updated weights for policy 1, policy_version 1048756 (0.0009) [2023-12-26 23:00:23,056][105620] Updated weights for policy 1, policy_version 1048766 (0.0009) [2023-12-26 23:00:23,120][105620] Updated weights for policy 1, policy_version 1048776 (0.0009) [2023-12-26 23:00:23,639][105692] Updated weights for policy 0, policy_version 1048010 (0.0008) [2023-12-26 23:00:23,697][105692] Updated weights for policy 0, policy_version 1048020 (0.0009) [2023-12-26 23:00:23,756][105692] Updated weights for policy 0, policy_version 1048030 (0.0009) [2023-12-26 23:00:23,866][105620] Updated weights for policy 1, policy_version 1048786 (0.0009) [2023-12-26 23:00:23,941][105620] Updated weights for policy 1, policy_version 1048796 (0.0009) [2023-12-26 23:00:24,009][105620] Updated weights for policy 1, policy_version 1048806 (0.0009) [2023-12-26 23:00:24,484][105692] Updated weights for policy 0, policy_version 1048040 (0.0006) [2023-12-26 23:00:24,528][105692] Updated weights for policy 0, policy_version 1048050 (0.0005) [2023-12-26 23:00:24,580][105692] Updated weights for policy 0, policy_version 1048060 (0.0005) [2023-12-26 23:00:24,780][105620] Updated weights for policy 1, policy_version 1048816 (0.0008) [2023-12-26 23:00:24,838][105620] Updated weights for policy 1, policy_version 1048826 (0.0007) [2023-12-26 23:00:24,892][105620] Updated weights for policy 1, policy_version 1048836 (0.0007) [2023-12-26 23:00:25,218][105692] Updated weights for policy 0, policy_version 1048070 (0.0006) [2023-12-26 23:00:25,282][105692] Updated weights for policy 0, policy_version 1048080 (0.0005) [2023-12-26 23:00:25,341][105692] Updated weights for policy 0, policy_version 1048090 (0.0005) [2023-12-26 23:00:25,477][105620] Updated weights for policy 1, policy_version 1048846 (0.0009) [2023-12-26 23:00:25,526][105620] Updated weights for policy 1, policy_version 1048856 (0.0008) [2023-12-26 23:00:25,571][105620] Updated weights for policy 1, policy_version 1048866 (0.0005) [2023-12-26 23:00:25,857][105692] Updated weights for policy 0, policy_version 1048100 (0.0007) [2023-12-26 23:00:25,906][105692] Updated weights for policy 0, policy_version 1048110 (0.0011) [2023-12-26 23:00:25,955][105692] Updated weights for policy 0, policy_version 1048120 (0.0011) [2023-12-26 23:00:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 536903680. Throughput: 0: 9742.8, 1: 9814.4. Samples: 536909872. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 23:00:26,062][104569] Avg episode reward: [(0, '8816.922'), (1, '9169.307')] [2023-12-26 23:00:26,211][105620] Updated weights for policy 1, policy_version 1048876 (0.0007) [2023-12-26 23:00:26,265][105620] Updated weights for policy 1, policy_version 1048886 (0.0009) [2023-12-26 23:00:26,319][105620] Updated weights for policy 1, policy_version 1048896 (0.0009) [2023-12-26 23:00:26,543][105692] Updated weights for policy 0, policy_version 1048130 (0.0009) [2023-12-26 23:00:26,597][105692] Updated weights for policy 0, policy_version 1048140 (0.0005) [2023-12-26 23:00:26,649][105692] Updated weights for policy 0, policy_version 1048150 (0.0009) [2023-12-26 23:00:26,714][105692] Updated weights for policy 0, policy_version 1048160 (0.0010) [2023-12-26 23:00:27,058][105620] Updated weights for policy 1, policy_version 1048906 (0.0009) [2023-12-26 23:00:27,119][105620] Updated weights for policy 1, policy_version 1048916 (0.0009) [2023-12-26 23:00:27,176][105620] Updated weights for policy 1, policy_version 1048926 (0.0009) [2023-12-26 23:00:27,228][105620] Updated weights for policy 1, policy_version 1048936 (0.0009) [2023-12-26 23:00:27,335][105692] Updated weights for policy 0, policy_version 1048170 (0.0007) [2023-12-26 23:00:27,393][105692] Updated weights for policy 0, policy_version 1048180 (0.0009) [2023-12-26 23:00:27,454][105692] Updated weights for policy 0, policy_version 1048190 (0.0009) [2023-12-26 23:00:27,947][105620] Updated weights for policy 1, policy_version 1048946 (0.0009) [2023-12-26 23:00:27,999][105620] Updated weights for policy 1, policy_version 1048956 (0.0009) [2023-12-26 23:00:28,048][105620] Updated weights for policy 1, policy_version 1048966 (0.0008) [2023-12-26 23:00:28,188][105692] Updated weights for policy 0, policy_version 1048200 (0.0006) [2023-12-26 23:00:28,245][105692] Updated weights for policy 0, policy_version 1048210 (0.0005) [2023-12-26 23:00:28,306][105692] Updated weights for policy 0, policy_version 1048220 (0.0010) [2023-12-26 23:00:28,716][105620] Updated weights for policy 1, policy_version 1048976 (0.0008) [2023-12-26 23:00:28,764][105620] Updated weights for policy 1, policy_version 1048986 (0.0008) [2023-12-26 23:00:28,816][105620] Updated weights for policy 1, policy_version 1048996 (0.0007) [2023-12-26 23:00:29,001][105692] Updated weights for policy 0, policy_version 1048230 (0.0010) [2023-12-26 23:00:29,055][105692] Updated weights for policy 0, policy_version 1048240 (0.0010) [2023-12-26 23:00:29,110][105692] Updated weights for policy 0, policy_version 1048250 (0.0011) [2023-12-26 23:00:29,604][105620] Updated weights for policy 1, policy_version 1049006 (0.0008) [2023-12-26 23:00:29,655][105620] Updated weights for policy 1, policy_version 1049016 (0.0008) [2023-12-26 23:00:29,717][105620] Updated weights for policy 1, policy_version 1049026 (0.0008) [2023-12-26 23:00:29,862][105692] Updated weights for policy 0, policy_version 1048260 (0.0011) [2023-12-26 23:00:29,916][105692] Updated weights for policy 0, policy_version 1048270 (0.0010) [2023-12-26 23:00:29,978][105692] Updated weights for policy 0, policy_version 1048280 (0.0010) [2023-12-26 23:00:30,488][105620] Updated weights for policy 1, policy_version 1049036 (0.0008) [2023-12-26 23:00:30,536][105620] Updated weights for policy 1, policy_version 1049046 (0.0007) [2023-12-26 23:00:30,590][105620] Updated weights for policy 1, policy_version 1049056 (0.0008) [2023-12-26 23:00:30,725][105692] Updated weights for policy 0, policy_version 1048290 (0.0011) [2023-12-26 23:00:30,786][105692] Updated weights for policy 0, policy_version 1048300 (0.0010) [2023-12-26 23:00:30,848][105692] Updated weights for policy 0, policy_version 1048310 (0.0011) [2023-12-26 23:00:30,913][105692] Updated weights for policy 0, policy_version 1048320 (0.0010) [2023-12-26 23:00:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 537001984. Throughput: 0: 9814.9, 1: 9762.8. Samples: 536971244. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 23:00:31,062][104569] Avg episode reward: [(0, '9079.431'), (1, '9257.765')] [2023-12-26 23:00:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001049064_268591104.pth... [2023-12-26 23:00:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001048320_268410880.pth... [2023-12-26 23:00:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001047168_268115968.pth [2023-12-26 23:00:31,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001047912_268296192.pth [2023-12-26 23:00:31,349][105620] Updated weights for policy 1, policy_version 1049066 (0.0008) [2023-12-26 23:00:31,416][105620] Updated weights for policy 1, policy_version 1049076 (0.0007) [2023-12-26 23:00:31,468][105620] Updated weights for policy 1, policy_version 1049086 (0.0007) [2023-12-26 23:00:31,520][105620] Updated weights for policy 1, policy_version 1049096 (0.0008) [2023-12-26 23:00:31,677][105692] Updated weights for policy 0, policy_version 1048330 (0.0008) [2023-12-26 23:00:31,736][105692] Updated weights for policy 0, policy_version 1048340 (0.0009) [2023-12-26 23:00:31,798][105692] Updated weights for policy 0, policy_version 1048350 (0.0009) [2023-12-26 23:00:32,146][105620] Updated weights for policy 1, policy_version 1049106 (0.0005) [2023-12-26 23:00:32,206][105620] Updated weights for policy 1, policy_version 1049116 (0.0005) [2023-12-26 23:00:32,256][105620] Updated weights for policy 1, policy_version 1049126 (0.0007) [2023-12-26 23:00:32,672][105692] Updated weights for policy 0, policy_version 1048360 (0.0010) [2023-12-26 23:00:32,727][105692] Updated weights for policy 0, policy_version 1048370 (0.0009) [2023-12-26 23:00:32,777][105692] Updated weights for policy 0, policy_version 1048380 (0.0009) [2023-12-26 23:00:32,860][105620] Updated weights for policy 1, policy_version 1049136 (0.0006) [2023-12-26 23:00:32,910][105620] Updated weights for policy 1, policy_version 1049146 (0.0005) [2023-12-26 23:00:32,960][105620] Updated weights for policy 1, policy_version 1049156 (0.0005) [2023-12-26 23:00:33,564][105620] Updated weights for policy 1, policy_version 1049166 (0.0005) [2023-12-26 23:00:33,615][105620] Updated weights for policy 1, policy_version 1049176 (0.0006) [2023-12-26 23:00:33,630][105692] Updated weights for policy 0, policy_version 1048390 (0.0009) [2023-12-26 23:00:33,666][105620] Updated weights for policy 1, policy_version 1049186 (0.0005) [2023-12-26 23:00:33,692][105692] Updated weights for policy 0, policy_version 1048400 (0.0008) [2023-12-26 23:00:33,755][105692] Updated weights for policy 0, policy_version 1048410 (0.0009) [2023-12-26 23:00:34,304][105620] Updated weights for policy 1, policy_version 1049196 (0.0007) [2023-12-26 23:00:34,359][105620] Updated weights for policy 1, policy_version 1049206 (0.0008) [2023-12-26 23:00:34,418][105620] Updated weights for policy 1, policy_version 1049216 (0.0009) [2023-12-26 23:00:34,508][105692] Updated weights for policy 0, policy_version 1048420 (0.0009) [2023-12-26 23:00:34,575][105692] Updated weights for policy 0, policy_version 1048430 (0.0009) [2023-12-26 23:00:34,641][105692] Updated weights for policy 0, policy_version 1048440 (0.0009) [2023-12-26 23:00:35,183][105620] Updated weights for policy 1, policy_version 1049226 (0.0009) [2023-12-26 23:00:35,242][105620] Updated weights for policy 1, policy_version 1049236 (0.0009) [2023-12-26 23:00:35,294][105620] Updated weights for policy 1, policy_version 1049246 (0.0008) [2023-12-26 23:00:35,348][105620] Updated weights for policy 1, policy_version 1049256 (0.0005) [2023-12-26 23:00:35,411][105692] Updated weights for policy 0, policy_version 1048450 (0.0009) [2023-12-26 23:00:35,472][105692] Updated weights for policy 0, policy_version 1048460 (0.0008) [2023-12-26 23:00:35,530][105692] Updated weights for policy 0, policy_version 1048470 (0.0005) [2023-12-26 23:00:35,583][105692] Updated weights for policy 0, policy_version 1048480 (0.0005) [2023-12-26 23:00:36,015][105620] Updated weights for policy 1, policy_version 1049266 (0.0009) [2023-12-26 23:00:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 537092096. Throughput: 0: 9640.2, 1: 9834.1. Samples: 537086032. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 23:00:36,062][104569] Avg episode reward: [(0, '8631.409'), (1, '9350.216')] [2023-12-26 23:00:36,072][105620] Updated weights for policy 1, policy_version 1049276 (0.0010) [2023-12-26 23:00:36,140][105620] Updated weights for policy 1, policy_version 1049286 (0.0011) [2023-12-26 23:00:36,364][105692] Updated weights for policy 0, policy_version 1048490 (0.0009) [2023-12-26 23:00:36,423][105692] Updated weights for policy 0, policy_version 1048500 (0.0008) [2023-12-26 23:00:36,484][105692] Updated weights for policy 0, policy_version 1048510 (0.0008) [2023-12-26 23:00:36,837][105620] Updated weights for policy 1, policy_version 1049296 (0.0008) [2023-12-26 23:00:36,888][105620] Updated weights for policy 1, policy_version 1049306 (0.0009) [2023-12-26 23:00:36,943][105620] Updated weights for policy 1, policy_version 1049316 (0.0008) [2023-12-26 23:00:37,309][105692] Updated weights for policy 0, policy_version 1048520 (0.0009) [2023-12-26 23:00:37,370][105692] Updated weights for policy 0, policy_version 1048530 (0.0009) [2023-12-26 23:00:37,432][105692] Updated weights for policy 0, policy_version 1048540 (0.0009) [2023-12-26 23:00:37,603][105620] Updated weights for policy 1, policy_version 1049326 (0.0008) [2023-12-26 23:00:37,665][105620] Updated weights for policy 1, policy_version 1049336 (0.0010) [2023-12-26 23:00:37,720][105620] Updated weights for policy 1, policy_version 1049346 (0.0010) [2023-12-26 23:00:38,222][105692] Updated weights for policy 0, policy_version 1048550 (0.0009) [2023-12-26 23:00:38,271][105692] Updated weights for policy 0, policy_version 1048560 (0.0009) [2023-12-26 23:00:38,328][105692] Updated weights for policy 0, policy_version 1048570 (0.0009) [2023-12-26 23:00:38,375][105620] Updated weights for policy 1, policy_version 1049356 (0.0008) [2023-12-26 23:00:38,442][105620] Updated weights for policy 1, policy_version 1049366 (0.0008) [2023-12-26 23:00:38,508][105620] Updated weights for policy 1, policy_version 1049376 (0.0008) [2023-12-26 23:00:39,142][105692] Updated weights for policy 0, policy_version 1048580 (0.0006) [2023-12-26 23:00:39,198][105692] Updated weights for policy 0, policy_version 1048590 (0.0006) [2023-12-26 23:00:39,237][105620] Updated weights for policy 1, policy_version 1049386 (0.0009) [2023-12-26 23:00:39,263][105692] Updated weights for policy 0, policy_version 1048600 (0.0007) [2023-12-26 23:00:39,306][105620] Updated weights for policy 1, policy_version 1049396 (0.0007) [2023-12-26 23:00:39,377][105620] Updated weights for policy 1, policy_version 1049406 (0.0007) [2023-12-26 23:00:39,449][105620] Updated weights for policy 1, policy_version 1049416 (0.0010) [2023-12-26 23:00:39,946][105692] Updated weights for policy 0, policy_version 1048610 (0.0008) [2023-12-26 23:00:40,020][105692] Updated weights for policy 0, policy_version 1048620 (0.0009) [2023-12-26 23:00:40,084][105692] Updated weights for policy 0, policy_version 1048630 (0.0006) [2023-12-26 23:00:40,146][105692] Updated weights for policy 0, policy_version 1048640 (0.0009) [2023-12-26 23:00:40,155][105620] Updated weights for policy 1, policy_version 1049426 (0.0007) [2023-12-26 23:00:40,209][105620] Updated weights for policy 1, policy_version 1049436 (0.0008) [2023-12-26 23:00:40,256][105620] Updated weights for policy 1, policy_version 1049446 (0.0009) [2023-12-26 23:00:40,771][105692] Updated weights for policy 0, policy_version 1048650 (0.0006) [2023-12-26 23:00:40,827][105692] Updated weights for policy 0, policy_version 1048660 (0.0008) [2023-12-26 23:00:40,871][105692] Updated weights for policy 0, policy_version 1048670 (0.0008) [2023-12-26 23:00:41,056][105620] Updated weights for policy 1, policy_version 1049456 (0.0010) [2023-12-26 23:00:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 537190400. Throughput: 0: 9628.9, 1: 9911.7. Samples: 537200672. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 23:00:41,063][104569] Avg episode reward: [(0, '9083.652'), (1, '8883.133')] [2023-12-26 23:00:41,118][105620] Updated weights for policy 1, policy_version 1049466 (0.0010) [2023-12-26 23:00:41,188][105620] Updated weights for policy 1, policy_version 1049476 (0.0010) [2023-12-26 23:00:41,571][105692] Updated weights for policy 0, policy_version 1048680 (0.0006) [2023-12-26 23:00:41,628][105692] Updated weights for policy 0, policy_version 1048690 (0.0008) [2023-12-26 23:00:41,692][105692] Updated weights for policy 0, policy_version 1048700 (0.0007) [2023-12-26 23:00:41,960][105620] Updated weights for policy 1, policy_version 1049486 (0.0008) [2023-12-26 23:00:42,025][105620] Updated weights for policy 1, policy_version 1049496 (0.0006) [2023-12-26 23:00:42,096][105620] Updated weights for policy 1, policy_version 1049506 (0.0008) [2023-12-26 23:00:42,448][105692] Updated weights for policy 0, policy_version 1048710 (0.0008) [2023-12-26 23:00:42,493][105692] Updated weights for policy 0, policy_version 1048720 (0.0008) [2023-12-26 23:00:42,546][105692] Updated weights for policy 0, policy_version 1048730 (0.0008) [2023-12-26 23:00:42,810][105620] Updated weights for policy 1, policy_version 1049516 (0.0010) [2023-12-26 23:00:42,861][105620] Updated weights for policy 1, policy_version 1049526 (0.0010) [2023-12-26 23:00:42,912][105620] Updated weights for policy 1, policy_version 1049536 (0.0010) [2023-12-26 23:00:43,238][105692] Updated weights for policy 0, policy_version 1048740 (0.0008) [2023-12-26 23:00:43,298][105692] Updated weights for policy 0, policy_version 1048750 (0.0007) [2023-12-26 23:00:43,347][105692] Updated weights for policy 0, policy_version 1048760 (0.0007) [2023-12-26 23:00:43,652][105620] Updated weights for policy 1, policy_version 1049546 (0.0010) [2023-12-26 23:00:43,724][105620] Updated weights for policy 1, policy_version 1049556 (0.0010) [2023-12-26 23:00:43,774][105620] Updated weights for policy 1, policy_version 1049566 (0.0010) [2023-12-26 23:00:43,822][105620] Updated weights for policy 1, policy_version 1049576 (0.0010) [2023-12-26 23:00:44,037][105692] Updated weights for policy 0, policy_version 1048770 (0.0008) [2023-12-26 23:00:44,095][105692] Updated weights for policy 0, policy_version 1048780 (0.0010) [2023-12-26 23:00:44,158][105692] Updated weights for policy 0, policy_version 1048790 (0.0010) [2023-12-26 23:00:44,216][105692] Updated weights for policy 0, policy_version 1048800 (0.0010) [2023-12-26 23:00:44,468][105620] Updated weights for policy 1, policy_version 1049586 (0.0010) [2023-12-26 23:00:44,535][105620] Updated weights for policy 1, policy_version 1049596 (0.0010) [2023-12-26 23:00:44,588][105620] Updated weights for policy 1, policy_version 1049606 (0.0008) [2023-12-26 23:00:44,885][105692] Updated weights for policy 0, policy_version 1048810 (0.0008) [2023-12-26 23:00:44,936][105692] Updated weights for policy 0, policy_version 1048820 (0.0008) [2023-12-26 23:00:45,004][105692] Updated weights for policy 0, policy_version 1048830 (0.0008) [2023-12-26 23:00:45,307][105620] Updated weights for policy 1, policy_version 1049616 (0.0010) [2023-12-26 23:00:45,366][105620] Updated weights for policy 1, policy_version 1049626 (0.0011) [2023-12-26 23:00:45,435][105620] Updated weights for policy 1, policy_version 1049636 (0.0010) [2023-12-26 23:00:45,706][105692] Updated weights for policy 0, policy_version 1048840 (0.0007) [2023-12-26 23:00:45,767][105692] Updated weights for policy 0, policy_version 1048850 (0.0005) [2023-12-26 23:00:45,813][105692] Updated weights for policy 0, policy_version 1048860 (0.0005) [2023-12-26 23:00:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 537288704. Throughput: 0: 9666.3, 1: 9896.9. Samples: 537257920. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 23:00:46,062][104569] Avg episode reward: [(0, '9353.296'), (1, '8990.758')] [2023-12-26 23:00:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001049640_268738560.pth... [2023-12-26 23:00:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001048864_268550144.pth... [2023-12-26 23:00:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001048488_268443648.pth [2023-12-26 23:00:46,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001047744_268263424.pth [2023-12-26 23:00:46,144][105620] Updated weights for policy 1, policy_version 1049646 (0.0010) [2023-12-26 23:00:46,192][105620] Updated weights for policy 1, policy_version 1049656 (0.0010) [2023-12-26 23:00:46,245][105620] Updated weights for policy 1, policy_version 1049666 (0.0010) [2023-12-26 23:00:46,366][105692] Updated weights for policy 0, policy_version 1048870 (0.0008) [2023-12-26 23:00:46,421][105692] Updated weights for policy 0, policy_version 1048881 (0.0010) [2023-12-26 23:00:46,474][105692] Updated weights for policy 0, policy_version 1048891 (0.0009) [2023-12-26 23:00:46,819][105620] Updated weights for policy 1, policy_version 1049676 (0.0007) [2023-12-26 23:00:46,883][105620] Updated weights for policy 1, policy_version 1049686 (0.0006) [2023-12-26 23:00:46,938][105620] Updated weights for policy 1, policy_version 1049696 (0.0005) [2023-12-26 23:00:47,257][105692] Updated weights for policy 0, policy_version 1048901 (0.0009) [2023-12-26 23:00:47,312][105692] Updated weights for policy 0, policy_version 1048911 (0.0009) [2023-12-26 23:00:47,366][105692] Updated weights for policy 0, policy_version 1048921 (0.0010) [2023-12-26 23:00:47,509][105620] Updated weights for policy 1, policy_version 1049706 (0.0005) [2023-12-26 23:00:47,559][105620] Updated weights for policy 1, policy_version 1049716 (0.0005) [2023-12-26 23:00:47,612][105620] Updated weights for policy 1, policy_version 1049726 (0.0005) [2023-12-26 23:00:47,673][105620] Updated weights for policy 1, policy_version 1049736 (0.0007) [2023-12-26 23:00:48,181][105692] Updated weights for policy 0, policy_version 1048931 (0.0009) [2023-12-26 23:00:48,238][105692] Updated weights for policy 0, policy_version 1048941 (0.0008) [2023-12-26 23:00:48,273][105620] Updated weights for policy 1, policy_version 1049746 (0.0011) [2023-12-26 23:00:48,304][105692] Updated weights for policy 0, policy_version 1048951 (0.0006) [2023-12-26 23:00:48,330][105620] Updated weights for policy 1, policy_version 1049756 (0.0010) [2023-12-26 23:00:48,399][105620] Updated weights for policy 1, policy_version 1049766 (0.0011) [2023-12-26 23:00:49,122][105692] Updated weights for policy 0, policy_version 1048961 (0.0008) [2023-12-26 23:00:49,176][105620] Updated weights for policy 1, policy_version 1049776 (0.0010) [2023-12-26 23:00:49,180][105692] Updated weights for policy 0, policy_version 1048971 (0.0010) [2023-12-26 23:00:49,243][105692] Updated weights for policy 0, policy_version 1048981 (0.0009) [2023-12-26 23:00:49,245][105620] Updated weights for policy 1, policy_version 1049786 (0.0012) [2023-12-26 23:00:49,306][105692] Updated weights for policy 0, policy_version 1048991 (0.0008) [2023-12-26 23:00:49,307][105620] Updated weights for policy 1, policy_version 1049796 (0.0008) [2023-12-26 23:00:50,082][105620] Updated weights for policy 1, policy_version 1049806 (0.0009) [2023-12-26 23:00:50,128][105692] Updated weights for policy 0, policy_version 1049001 (0.0007) [2023-12-26 23:00:50,148][105620] Updated weights for policy 1, policy_version 1049816 (0.0007) [2023-12-26 23:00:50,194][105692] Updated weights for policy 0, policy_version 1049011 (0.0008) [2023-12-26 23:00:50,216][105620] Updated weights for policy 1, policy_version 1049826 (0.0007) [2023-12-26 23:00:50,266][105692] Updated weights for policy 0, policy_version 1049021 (0.0009) [2023-12-26 23:00:50,943][105620] Updated weights for policy 1, policy_version 1049836 (0.0007) [2023-12-26 23:00:50,993][105620] Updated weights for policy 1, policy_version 1049846 (0.0006) [2023-12-26 23:00:51,038][105692] Updated weights for policy 0, policy_version 1049031 (0.0008) [2023-12-26 23:00:51,062][105620] Updated weights for policy 1, policy_version 1049856 (0.0007) [2023-12-26 23:00:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 537378816. Throughput: 0: 9644.8, 1: 9906.6. Samples: 537377596. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 23:00:51,062][104569] Avg episode reward: [(0, '9263.829'), (1, '9169.093')] [2023-12-26 23:00:51,105][105692] Updated weights for policy 0, policy_version 1049041 (0.0007) [2023-12-26 23:00:51,167][105692] Updated weights for policy 0, policy_version 1049051 (0.0009) [2023-12-26 23:00:51,868][105620] Updated weights for policy 1, policy_version 1049866 (0.0007) [2023-12-26 23:00:51,879][105692] Updated weights for policy 0, policy_version 1049061 (0.0008) [2023-12-26 23:00:51,929][105620] Updated weights for policy 1, policy_version 1049876 (0.0008) [2023-12-26 23:00:51,943][105692] Updated weights for policy 0, policy_version 1049071 (0.0006) [2023-12-26 23:00:51,988][105620] Updated weights for policy 1, policy_version 1049886 (0.0007) [2023-12-26 23:00:52,003][105692] Updated weights for policy 0, policy_version 1049081 (0.0008) [2023-12-26 23:00:52,046][105620] Updated weights for policy 1, policy_version 1049896 (0.0007) [2023-12-26 23:00:52,778][105692] Updated weights for policy 0, policy_version 1049091 (0.0006) [2023-12-26 23:00:52,784][105620] Updated weights for policy 1, policy_version 1049906 (0.0011) [2023-12-26 23:00:52,835][105692] Updated weights for policy 0, policy_version 1049101 (0.0006) [2023-12-26 23:00:52,845][105620] Updated weights for policy 1, policy_version 1049916 (0.0011) [2023-12-26 23:00:52,895][105692] Updated weights for policy 0, policy_version 1049111 (0.0005) [2023-12-26 23:00:52,905][105620] Updated weights for policy 1, policy_version 1049926 (0.0011) [2023-12-26 23:00:53,663][105692] Updated weights for policy 0, policy_version 1049121 (0.0007) [2023-12-26 23:00:53,663][105620] Updated weights for policy 1, policy_version 1049936 (0.0011) [2023-12-26 23:00:53,722][105692] Updated weights for policy 0, policy_version 1049131 (0.0007) [2023-12-26 23:00:53,724][105620] Updated weights for policy 1, policy_version 1049946 (0.0011) [2023-12-26 23:00:53,773][105620] Updated weights for policy 1, policy_version 1049956 (0.0010) [2023-12-26 23:00:53,778][105692] Updated weights for policy 0, policy_version 1049141 (0.0006) [2023-12-26 23:00:53,836][105692] Updated weights for policy 0, policy_version 1049151 (0.0007) [2023-12-26 23:00:54,535][105620] Updated weights for policy 1, policy_version 1049966 (0.0010) [2023-12-26 23:00:54,598][105692] Updated weights for policy 0, policy_version 1049161 (0.0007) [2023-12-26 23:00:54,599][105620] Updated weights for policy 1, policy_version 1049976 (0.0010) [2023-12-26 23:00:54,645][105692] Updated weights for policy 0, policy_version 1049171 (0.0008) [2023-12-26 23:00:54,659][105620] Updated weights for policy 1, policy_version 1049986 (0.0010) [2023-12-26 23:00:54,706][105692] Updated weights for policy 0, policy_version 1049181 (0.0006) [2023-12-26 23:00:55,426][105620] Updated weights for policy 1, policy_version 1049996 (0.0010) [2023-12-26 23:00:55,472][105692] Updated weights for policy 0, policy_version 1049191 (0.0007) [2023-12-26 23:00:55,479][105620] Updated weights for policy 1, policy_version 1050006 (0.0010) [2023-12-26 23:00:55,533][105692] Updated weights for policy 0, policy_version 1049201 (0.0006) [2023-12-26 23:00:55,535][105620] Updated weights for policy 1, policy_version 1050016 (0.0010) [2023-12-26 23:00:55,584][105692] Updated weights for policy 0, policy_version 1049211 (0.0007) [2023-12-26 23:00:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 537477120. Throughput: 0: 9562.6, 1: 9833.7. Samples: 537487368. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 23:00:56,062][104569] Avg episode reward: [(0, '8170.564'), (1, '9168.758')] [2023-12-26 23:00:56,297][105620] Updated weights for policy 1, policy_version 1050026 (0.0011) [2023-12-26 23:00:56,350][105620] Updated weights for policy 1, policy_version 1050036 (0.0011) [2023-12-26 23:00:56,360][105692] Updated weights for policy 0, policy_version 1049221 (0.0007) [2023-12-26 23:00:56,407][105620] Updated weights for policy 1, policy_version 1050046 (0.0011) [2023-12-26 23:00:56,413][105692] Updated weights for policy 0, policy_version 1049231 (0.0006) [2023-12-26 23:00:56,453][105620] Updated weights for policy 1, policy_version 1050056 (0.0010) [2023-12-26 23:00:56,467][105692] Updated weights for policy 0, policy_version 1049241 (0.0006) [2023-12-26 23:00:57,205][105620] Updated weights for policy 1, policy_version 1050066 (0.0010) [2023-12-26 23:00:57,238][105692] Updated weights for policy 0, policy_version 1049251 (0.0007) [2023-12-26 23:00:57,256][105620] Updated weights for policy 1, policy_version 1050076 (0.0010) [2023-12-26 23:00:57,290][105692] Updated weights for policy 0, policy_version 1049261 (0.0006) [2023-12-26 23:00:57,305][105620] Updated weights for policy 1, policy_version 1050086 (0.0010) [2023-12-26 23:00:57,350][105692] Updated weights for policy 0, policy_version 1049271 (0.0006) [2023-12-26 23:00:58,038][105620] Updated weights for policy 1, policy_version 1050096 (0.0011) [2023-12-26 23:00:58,100][105620] Updated weights for policy 1, policy_version 1050106 (0.0010) [2023-12-26 23:00:58,115][105692] Updated weights for policy 0, policy_version 1049281 (0.0008) [2023-12-26 23:00:58,166][105620] Updated weights for policy 1, policy_version 1050116 (0.0010) [2023-12-26 23:00:58,176][105692] Updated weights for policy 0, policy_version 1049291 (0.0007) [2023-12-26 23:00:58,238][105692] Updated weights for policy 0, policy_version 1049301 (0.0008) [2023-12-26 23:00:58,299][105692] Updated weights for policy 0, policy_version 1049311 (0.0008) [2023-12-26 23:00:58,959][105620] Updated weights for policy 1, policy_version 1050126 (0.0010) [2023-12-26 23:00:59,024][105620] Updated weights for policy 1, policy_version 1050136 (0.0010) [2023-12-26 23:00:59,083][105620] Updated weights for policy 1, policy_version 1050146 (0.0010) [2023-12-26 23:00:59,209][105692] Updated weights for policy 0, policy_version 1049321 (0.0009) [2023-12-26 23:00:59,281][105692] Updated weights for policy 0, policy_version 1049331 (0.0009) [2023-12-26 23:00:59,348][105692] Updated weights for policy 0, policy_version 1049341 (0.0009) [2023-12-26 23:00:59,836][105620] Updated weights for policy 1, policy_version 1050156 (0.0010) [2023-12-26 23:00:59,903][105620] Updated weights for policy 1, policy_version 1050166 (0.0011) [2023-12-26 23:00:59,964][105620] Updated weights for policy 1, policy_version 1050176 (0.0009) [2023-12-26 23:01:00,051][105692] Updated weights for policy 0, policy_version 1049351 (0.0008) [2023-12-26 23:01:00,113][105692] Updated weights for policy 0, policy_version 1049361 (0.0008) [2023-12-26 23:01:00,171][105692] Updated weights for policy 0, policy_version 1049371 (0.0010) [2023-12-26 23:01:00,584][105620] Updated weights for policy 1, policy_version 1050186 (0.0010) [2023-12-26 23:01:00,651][105620] Updated weights for policy 1, policy_version 1050196 (0.0005) [2023-12-26 23:01:00,717][105620] Updated weights for policy 1, policy_version 1050206 (0.0007) [2023-12-26 23:01:00,776][105620] Updated weights for policy 1, policy_version 1050216 (0.0010) [2023-12-26 23:01:01,038][105692] Updated weights for policy 0, policy_version 1049381 (0.0009) [2023-12-26 23:01:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 537567232. Throughput: 0: 9524.5, 1: 9827.4. Samples: 537542288. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 23:01:01,063][104569] Avg episode reward: [(0, '8375.668'), (1, '9259.953')] [2023-12-26 23:01:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001050216_268886016.pth... [2023-12-26 23:01:01,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001049064_268591104.pth [2023-12-26 23:01:01,109][105692] Updated weights for policy 0, policy_version 1049391 (0.0008) [2023-12-26 23:01:01,175][105692] Updated weights for policy 0, policy_version 1049401 (0.0008) [2023-12-26 23:01:01,215][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001049408_268689408.pth... [2023-12-26 23:01:01,220][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001048320_268410880.pth [2023-12-26 23:01:01,388][105620] Updated weights for policy 1, policy_version 1050226 (0.0009) [2023-12-26 23:01:01,447][105620] Updated weights for policy 1, policy_version 1050236 (0.0009) [2023-12-26 23:01:01,503][105620] Updated weights for policy 1, policy_version 1050246 (0.0010) [2023-12-26 23:01:01,821][105692] Updated weights for policy 0, policy_version 1049411 (0.0008) [2023-12-26 23:01:01,870][105692] Updated weights for policy 0, policy_version 1049421 (0.0005) [2023-12-26 23:01:01,937][105692] Updated weights for policy 0, policy_version 1049431 (0.0009) [2023-12-26 23:01:02,334][105620] Updated weights for policy 1, policy_version 1050256 (0.0009) [2023-12-26 23:01:02,390][105620] Updated weights for policy 1, policy_version 1050266 (0.0009) [2023-12-26 23:01:02,440][105620] Updated weights for policy 1, policy_version 1050276 (0.0008) [2023-12-26 23:01:02,688][105692] Updated weights for policy 0, policy_version 1049441 (0.0009) [2023-12-26 23:01:02,755][105692] Updated weights for policy 0, policy_version 1049451 (0.0010) [2023-12-26 23:01:02,820][105692] Updated weights for policy 0, policy_version 1049461 (0.0009) [2023-12-26 23:01:02,889][105692] Updated weights for policy 0, policy_version 1049471 (0.0009) [2023-12-26 23:01:03,123][105620] Updated weights for policy 1, policy_version 1050286 (0.0008) [2023-12-26 23:01:03,171][105620] Updated weights for policy 1, policy_version 1050296 (0.0009) [2023-12-26 23:01:03,227][105620] Updated weights for policy 1, policy_version 1050306 (0.0009) [2023-12-26 23:01:03,590][105692] Updated weights for policy 0, policy_version 1049481 (0.0009) [2023-12-26 23:01:03,646][105692] Updated weights for policy 0, policy_version 1049491 (0.0010) [2023-12-26 23:01:03,697][105692] Updated weights for policy 0, policy_version 1049501 (0.0008) [2023-12-26 23:01:03,991][105620] Updated weights for policy 1, policy_version 1050316 (0.0008) [2023-12-26 23:01:04,052][105620] Updated weights for policy 1, policy_version 1050326 (0.0009) [2023-12-26 23:01:04,111][105620] Updated weights for policy 1, policy_version 1050336 (0.0011) [2023-12-26 23:01:04,152][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000006 [2023-12-26 23:01:04,519][105692] Updated weights for policy 0, policy_version 1049511 (0.0008) [2023-12-26 23:01:04,583][105692] Updated weights for policy 0, policy_version 1049521 (0.0007) [2023-12-26 23:01:04,639][105692] Updated weights for policy 0, policy_version 1049531 (0.0007) [2023-12-26 23:01:04,840][105620] Updated weights for policy 1, policy_version 1050346 (0.0009) [2023-12-26 23:01:04,894][105620] Updated weights for policy 1, policy_version 1050356 (0.0005) [2023-12-26 23:01:04,948][105620] Updated weights for policy 1, policy_version 1050366 (0.0005) [2023-12-26 23:01:04,993][105620] Updated weights for policy 1, policy_version 1050376 (0.0005) [2023-12-26 23:01:05,393][105692] Updated weights for policy 0, policy_version 1049541 (0.0006) [2023-12-26 23:01:05,448][105692] Updated weights for policy 0, policy_version 1049551 (0.0008) [2023-12-26 23:01:05,504][105692] Updated weights for policy 0, policy_version 1049561 (0.0008) [2023-12-26 23:01:05,709][105620] Updated weights for policy 1, policy_version 1050386 (0.0011) [2023-12-26 23:01:05,774][105620] Updated weights for policy 1, policy_version 1050396 (0.0010) [2023-12-26 23:01:05,833][105620] Updated weights for policy 1, policy_version 1050406 (0.0010) [2023-12-26 23:01:06,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19251.1, 300 sec: 19494.2). Total num frames: 537665536. Throughput: 0: 9417.9, 1: 9790.4. Samples: 537655188. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 23:01:06,063][104569] Avg episode reward: [(0, '8552.529'), (1, '9080.066')] [2023-12-26 23:01:06,278][105692] Updated weights for policy 0, policy_version 1049571 (0.0008) [2023-12-26 23:01:06,342][105692] Updated weights for policy 0, policy_version 1049581 (0.0008) [2023-12-26 23:01:06,401][105692] Updated weights for policy 0, policy_version 1049591 (0.0008) [2023-12-26 23:01:06,510][105620] Updated weights for policy 1, policy_version 1050416 (0.0010) [2023-12-26 23:01:06,570][105620] Updated weights for policy 1, policy_version 1050426 (0.0011) [2023-12-26 23:01:06,638][105620] Updated weights for policy 1, policy_version 1050436 (0.0011) [2023-12-26 23:01:07,166][105692] Updated weights for policy 0, policy_version 1049601 (0.0008) [2023-12-26 23:01:07,235][105692] Updated weights for policy 0, policy_version 1049611 (0.0005) [2023-12-26 23:01:07,296][105692] Updated weights for policy 0, policy_version 1049621 (0.0005) [2023-12-26 23:01:07,351][105692] Updated weights for policy 0, policy_version 1049631 (0.0008) [2023-12-26 23:01:07,383][105620] Updated weights for policy 1, policy_version 1050446 (0.0011) [2023-12-26 23:01:07,442][105620] Updated weights for policy 1, policy_version 1050456 (0.0006) [2023-12-26 23:01:07,496][105620] Updated weights for policy 1, policy_version 1050466 (0.0008) [2023-12-26 23:01:08,036][105692] Updated weights for policy 0, policy_version 1049641 (0.0009) [2023-12-26 23:01:08,083][105620] Updated weights for policy 1, policy_version 1050476 (0.0008) [2023-12-26 23:01:08,095][105692] Updated weights for policy 0, policy_version 1049651 (0.0011) [2023-12-26 23:01:08,142][105620] Updated weights for policy 1, policy_version 1050486 (0.0009) [2023-12-26 23:01:08,152][105692] Updated weights for policy 0, policy_version 1049661 (0.0011) [2023-12-26 23:01:08,199][105620] Updated weights for policy 1, policy_version 1050496 (0.0010) [2023-12-26 23:01:08,858][105692] Updated weights for policy 0, policy_version 1049671 (0.0010) [2023-12-26 23:01:08,884][105620] Updated weights for policy 1, policy_version 1050506 (0.0011) [2023-12-26 23:01:08,915][105692] Updated weights for policy 0, policy_version 1049681 (0.0007) [2023-12-26 23:01:08,937][105620] Updated weights for policy 1, policy_version 1050516 (0.0010) [2023-12-26 23:01:08,944][105585] KL-divergence is very high: 131.0686 [2023-12-26 23:01:08,972][105692] Updated weights for policy 0, policy_version 1049691 (0.0005) [2023-12-26 23:01:08,989][105620] Updated weights for policy 1, policy_version 1050526 (0.0010) [2023-12-26 23:01:08,990][105585] KL-divergence is very high: 124.7294 [2023-12-26 23:01:09,041][105620] Updated weights for policy 1, policy_version 1050536 (0.0010) [2023-12-26 23:01:09,637][105692] Updated weights for policy 0, policy_version 1049701 (0.0006) [2023-12-26 23:01:09,699][105692] Updated weights for policy 0, policy_version 1049711 (0.0006) [2023-12-26 23:01:09,764][105692] Updated weights for policy 0, policy_version 1049721 (0.0008) [2023-12-26 23:01:09,808][105620] Updated weights for policy 1, policy_version 1050546 (0.0006) [2023-12-26 23:01:09,874][105620] Updated weights for policy 1, policy_version 1050556 (0.0008) [2023-12-26 23:01:09,935][105620] Updated weights for policy 1, policy_version 1050566 (0.0008) [2023-12-26 23:01:10,554][105692] Updated weights for policy 0, policy_version 1049731 (0.0008) [2023-12-26 23:01:10,612][105620] Updated weights for policy 1, policy_version 1050576 (0.0007) [2023-12-26 23:01:10,615][105692] Updated weights for policy 0, policy_version 1049741 (0.0009) [2023-12-26 23:01:10,665][105692] Updated weights for policy 0, policy_version 1049751 (0.0009) [2023-12-26 23:01:10,683][105620] Updated weights for policy 1, policy_version 1050586 (0.0006) [2023-12-26 23:01:10,747][105620] Updated weights for policy 1, policy_version 1050596 (0.0006) [2023-12-26 23:01:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 537763840. Throughput: 0: 9369.5, 1: 9769.7. Samples: 537771136. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 23:01:11,062][104569] Avg episode reward: [(0, '8906.880'), (1, '8809.350')] [2023-12-26 23:01:11,411][105692] Updated weights for policy 0, policy_version 1049761 (0.0009) [2023-12-26 23:01:11,464][105692] Updated weights for policy 0, policy_version 1049771 (0.0009) [2023-12-26 23:01:11,518][105620] Updated weights for policy 1, policy_version 1050606 (0.0009) [2023-12-26 23:01:11,520][105692] Updated weights for policy 0, policy_version 1049781 (0.0006) [2023-12-26 23:01:11,575][105620] Updated weights for policy 1, policy_version 1050616 (0.0009) [2023-12-26 23:01:11,581][105692] Updated weights for policy 0, policy_version 1049791 (0.0007) [2023-12-26 23:01:11,648][105620] Updated weights for policy 1, policy_version 1050628 (0.0007) [2023-12-26 23:01:12,280][105620] Updated weights for policy 1, policy_version 1050638 (0.0008) [2023-12-26 23:01:12,346][105620] Updated weights for policy 1, policy_version 1050648 (0.0009) [2023-12-26 23:01:12,380][105692] Updated weights for policy 0, policy_version 1049801 (0.0009) [2023-12-26 23:01:12,405][105620] Updated weights for policy 1, policy_version 1050658 (0.0008) [2023-12-26 23:01:12,432][105692] Updated weights for policy 0, policy_version 1049811 (0.0008) [2023-12-26 23:01:12,487][105692] Updated weights for policy 0, policy_version 1049821 (0.0009) [2023-12-26 23:01:13,156][105620] Updated weights for policy 1, policy_version 1050668 (0.0007) [2023-12-26 23:01:13,208][105692] Updated weights for policy 0, policy_version 1049831 (0.0007) [2023-12-26 23:01:13,213][105620] Updated weights for policy 1, policy_version 1050678 (0.0008) [2023-12-26 23:01:13,253][105692] Updated weights for policy 0, policy_version 1049841 (0.0006) [2023-12-26 23:01:13,267][105620] Updated weights for policy 1, policy_version 1050688 (0.0008) [2023-12-26 23:01:13,314][105692] Updated weights for policy 0, policy_version 1049851 (0.0006) [2023-12-26 23:01:13,941][105620] Updated weights for policy 1, policy_version 1050698 (0.0007) [2023-12-26 23:01:14,002][105620] Updated weights for policy 1, policy_version 1050708 (0.0005) [2023-12-26 23:01:14,055][105620] Updated weights for policy 1, policy_version 1050718 (0.0005) [2023-12-26 23:01:14,110][105620] Updated weights for policy 1, policy_version 1050728 (0.0006) [2023-12-26 23:01:14,129][105692] Updated weights for policy 0, policy_version 1049861 (0.0009) [2023-12-26 23:01:14,193][105692] Updated weights for policy 0, policy_version 1049871 (0.0008) [2023-12-26 23:01:14,259][105692] Updated weights for policy 0, policy_version 1049881 (0.0009) [2023-12-26 23:01:14,731][105620] Updated weights for policy 1, policy_version 1050738 (0.0009) [2023-12-26 23:01:14,799][105620] Updated weights for policy 1, policy_version 1050748 (0.0008) [2023-12-26 23:01:14,855][105620] Updated weights for policy 1, policy_version 1050758 (0.0007) [2023-12-26 23:01:15,058][105692] Updated weights for policy 0, policy_version 1049891 (0.0009) [2023-12-26 23:01:15,113][105692] Updated weights for policy 0, policy_version 1049901 (0.0009) [2023-12-26 23:01:15,175][105692] Updated weights for policy 0, policy_version 1049911 (0.0008) [2023-12-26 23:01:15,497][105620] Updated weights for policy 1, policy_version 1050768 (0.0008) [2023-12-26 23:01:15,563][105620] Updated weights for policy 1, policy_version 1050778 (0.0009) [2023-12-26 23:01:15,617][105620] Updated weights for policy 1, policy_version 1050788 (0.0009) [2023-12-26 23:01:15,975][105692] Updated weights for policy 0, policy_version 1049921 (0.0010) [2023-12-26 23:01:16,029][105692] Updated weights for policy 0, policy_version 1049931 (0.0009) [2023-12-26 23:01:16,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19114.7, 300 sec: 19494.2). Total num frames: 537853952. Throughput: 0: 9266.0, 1: 9782.4. Samples: 537828420. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 23:01:16,062][104569] Avg episode reward: [(0, '9176.342'), (1, '8989.767')] [2023-12-26 23:01:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001050792_269033472.pth... [2023-12-26 23:01:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001049640_268738560.pth [2023-12-26 23:01:16,080][105692] Updated weights for policy 0, policy_version 1049941 (0.0007) [2023-12-26 23:01:16,143][105692] Updated weights for policy 0, policy_version 1049951 (0.0005) [2023-12-26 23:01:16,149][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001049952_268828672.pth... [2023-12-26 23:01:16,154][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001048864_268550144.pth [2023-12-26 23:01:16,302][105620] Updated weights for policy 1, policy_version 1050798 (0.0010) [2023-12-26 23:01:16,364][105620] Updated weights for policy 1, policy_version 1050808 (0.0010) [2023-12-26 23:01:16,418][105620] Updated weights for policy 1, policy_version 1050818 (0.0010) [2023-12-26 23:01:16,781][105692] Updated weights for policy 0, policy_version 1049961 (0.0005) [2023-12-26 23:01:16,850][105692] Updated weights for policy 0, policy_version 1049971 (0.0009) [2023-12-26 23:01:16,903][105692] Updated weights for policy 0, policy_version 1049981 (0.0009) [2023-12-26 23:01:17,069][105620] Updated weights for policy 1, policy_version 1050828 (0.0008) [2023-12-26 23:01:17,115][105620] Updated weights for policy 1, policy_version 1050838 (0.0005) [2023-12-26 23:01:17,161][105620] Updated weights for policy 1, policy_version 1050848 (0.0005) [2023-12-26 23:01:17,712][105692] Updated weights for policy 0, policy_version 1049991 (0.0007) [2023-12-26 23:01:17,737][105620] Updated weights for policy 1, policy_version 1050858 (0.0006) [2023-12-26 23:01:17,755][105692] Updated weights for policy 0, policy_version 1050001 (0.0008) [2023-12-26 23:01:17,793][105620] Updated weights for policy 1, policy_version 1050868 (0.0010) [2023-12-26 23:01:17,815][105692] Updated weights for policy 0, policy_version 1050011 (0.0005) [2023-12-26 23:01:17,852][105620] Updated weights for policy 1, policy_version 1050878 (0.0011) [2023-12-26 23:01:17,898][105620] Updated weights for policy 1, policy_version 1050888 (0.0011) [2023-12-26 23:01:18,538][105692] Updated weights for policy 0, policy_version 1050021 (0.0008) [2023-12-26 23:01:18,595][105692] Updated weights for policy 0, policy_version 1050031 (0.0008) [2023-12-26 23:01:18,604][105620] Updated weights for policy 1, policy_version 1050898 (0.0006) [2023-12-26 23:01:18,654][105692] Updated weights for policy 0, policy_version 1050041 (0.0009) [2023-12-26 23:01:18,664][105620] Updated weights for policy 1, policy_version 1050908 (0.0005) [2023-12-26 23:01:18,727][105620] Updated weights for policy 1, policy_version 1050918 (0.0006) [2023-12-26 23:01:19,402][105620] Updated weights for policy 1, policy_version 1050928 (0.0010) [2023-12-26 23:01:19,457][105620] Updated weights for policy 1, policy_version 1050938 (0.0010) [2023-12-26 23:01:19,497][105692] Updated weights for policy 0, policy_version 1050051 (0.0010) [2023-12-26 23:01:19,525][105620] Updated weights for policy 1, policy_version 1050948 (0.0010) [2023-12-26 23:01:19,558][105692] Updated weights for policy 0, policy_version 1050061 (0.0009) [2023-12-26 23:01:19,614][105692] Updated weights for policy 0, policy_version 1050071 (0.0009) [2023-12-26 23:01:20,315][105620] Updated weights for policy 1, policy_version 1050958 (0.0010) [2023-12-26 23:01:20,367][105620] Updated weights for policy 1, policy_version 1050968 (0.0010) [2023-12-26 23:01:20,426][105620] Updated weights for policy 1, policy_version 1050978 (0.0011) [2023-12-26 23:01:20,443][105692] Updated weights for policy 0, policy_version 1050081 (0.0009) [2023-12-26 23:01:20,502][105692] Updated weights for policy 0, policy_version 1050091 (0.0007) [2023-12-26 23:01:20,569][105692] Updated weights for policy 0, policy_version 1050101 (0.0009) [2023-12-26 23:01:20,631][105692] Updated weights for policy 0, policy_version 1050111 (0.0008) [2023-12-26 23:01:21,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 537952256. Throughput: 0: 9286.3, 1: 9804.0. Samples: 537945096. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 23:01:21,063][104569] Avg episode reward: [(0, '9181.863'), (1, '9183.900')] [2023-12-26 23:01:21,161][105620] Updated weights for policy 1, policy_version 1050988 (0.0010) [2023-12-26 23:01:21,213][105620] Updated weights for policy 1, policy_version 1050998 (0.0010) [2023-12-26 23:01:21,275][105620] Updated weights for policy 1, policy_version 1051008 (0.0011) [2023-12-26 23:01:21,365][105692] Updated weights for policy 0, policy_version 1050121 (0.0009) [2023-12-26 23:01:21,426][105692] Updated weights for policy 0, policy_version 1050131 (0.0008) [2023-12-26 23:01:21,485][105692] Updated weights for policy 0, policy_version 1050141 (0.0008) [2023-12-26 23:01:21,949][105620] Updated weights for policy 1, policy_version 1051018 (0.0010) [2023-12-26 23:01:22,013][105620] Updated weights for policy 1, policy_version 1051028 (0.0006) [2023-12-26 23:01:22,068][105620] Updated weights for policy 1, policy_version 1051038 (0.0005) [2023-12-26 23:01:22,125][105620] Updated weights for policy 1, policy_version 1051048 (0.0006) [2023-12-26 23:01:22,289][105692] Updated weights for policy 0, policy_version 1050151 (0.0008) [2023-12-26 23:01:22,344][105692] Updated weights for policy 0, policy_version 1050161 (0.0008) [2023-12-26 23:01:22,402][105692] Updated weights for policy 0, policy_version 1050171 (0.0008) [2023-12-26 23:01:22,784][105620] Updated weights for policy 1, policy_version 1051058 (0.0006) [2023-12-26 23:01:22,852][105620] Updated weights for policy 1, policy_version 1051068 (0.0005) [2023-12-26 23:01:22,922][105620] Updated weights for policy 1, policy_version 1051078 (0.0006) [2023-12-26 23:01:23,192][105692] Updated weights for policy 0, policy_version 1050181 (0.0007) [2023-12-26 23:01:23,251][105692] Updated weights for policy 0, policy_version 1050191 (0.0006) [2023-12-26 23:01:23,320][105692] Updated weights for policy 0, policy_version 1050201 (0.0005) [2023-12-26 23:01:23,513][105620] Updated weights for policy 1, policy_version 1051088 (0.0010) [2023-12-26 23:01:23,578][105620] Updated weights for policy 1, policy_version 1051098 (0.0006) [2023-12-26 23:01:23,637][105620] Updated weights for policy 1, policy_version 1051108 (0.0005) [2023-12-26 23:01:23,918][105692] Updated weights for policy 0, policy_version 1050211 (0.0007) [2023-12-26 23:01:23,967][105692] Updated weights for policy 0, policy_version 1050221 (0.0006) [2023-12-26 23:01:24,020][105692] Updated weights for policy 0, policy_version 1050231 (0.0005) [2023-12-26 23:01:24,309][105620] Updated weights for policy 1, policy_version 1051118 (0.0008) [2023-12-26 23:01:24,365][105620] Updated weights for policy 1, policy_version 1051128 (0.0010) [2023-12-26 23:01:24,421][105620] Updated weights for policy 1, policy_version 1051138 (0.0010) [2023-12-26 23:01:24,770][105692] Updated weights for policy 0, policy_version 1050241 (0.0006) [2023-12-26 23:01:24,828][105692] Updated weights for policy 0, policy_version 1050251 (0.0009) [2023-12-26 23:01:24,879][105692] Updated weights for policy 0, policy_version 1050261 (0.0009) [2023-12-26 23:01:24,932][105692] Updated weights for policy 0, policy_version 1050271 (0.0009) [2023-12-26 23:01:24,992][105620] Updated weights for policy 1, policy_version 1051148 (0.0010) [2023-12-26 23:01:25,037][105620] Updated weights for policy 1, policy_version 1051158 (0.0010) [2023-12-26 23:01:25,091][105620] Updated weights for policy 1, policy_version 1051168 (0.0010) [2023-12-26 23:01:25,662][105620] Updated weights for policy 1, policy_version 1051178 (0.0009) [2023-12-26 23:01:25,707][105620] Updated weights for policy 1, policy_version 1051188 (0.0005) [2023-12-26 23:01:25,769][105620] Updated weights for policy 1, policy_version 1051198 (0.0009) [2023-12-26 23:01:25,779][105692] Updated weights for policy 0, policy_version 1050281 (0.0006) [2023-12-26 23:01:25,828][105620] Updated weights for policy 1, policy_version 1051208 (0.0010) [2023-12-26 23:01:25,839][105692] Updated weights for policy 0, policy_version 1050291 (0.0006) [2023-12-26 23:01:25,903][105692] Updated weights for policy 0, policy_version 1050301 (0.0009) [2023-12-26 23:01:26,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 538058752. Throughput: 0: 9273.0, 1: 9901.5. Samples: 538063524. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 23:01:26,062][104569] Avg episode reward: [(0, '9183.048'), (1, '9182.463')] [2023-12-26 23:01:26,428][105620] Updated weights for policy 1, policy_version 1051218 (0.0005) [2023-12-26 23:01:26,478][105620] Updated weights for policy 1, policy_version 1051228 (0.0005) [2023-12-26 23:01:26,525][105620] Updated weights for policy 1, policy_version 1051238 (0.0005) [2023-12-26 23:01:26,756][105692] Updated weights for policy 0, policy_version 1050311 (0.0010) [2023-12-26 23:01:26,814][105692] Updated weights for policy 0, policy_version 1050322 (0.0010) [2023-12-26 23:01:26,858][105692] Updated weights for policy 0, policy_version 1050332 (0.0007) [2023-12-26 23:01:27,043][105620] Updated weights for policy 1, policy_version 1051248 (0.0005) [2023-12-26 23:01:27,086][105620] Updated weights for policy 1, policy_version 1051258 (0.0005) [2023-12-26 23:01:27,141][105620] Updated weights for policy 1, policy_version 1051268 (0.0006) [2023-12-26 23:01:27,713][105692] Updated weights for policy 0, policy_version 1050342 (0.0009) [2023-12-26 23:01:27,763][105692] Updated weights for policy 0, policy_version 1050352 (0.0009) [2023-12-26 23:01:27,809][105692] Updated weights for policy 0, policy_version 1050362 (0.0007) [2023-12-26 23:01:27,810][105620] Updated weights for policy 1, policy_version 1051278 (0.0009) [2023-12-26 23:01:27,864][105620] Updated weights for policy 1, policy_version 1051288 (0.0007) [2023-12-26 23:01:27,918][105620] Updated weights for policy 1, policy_version 1051298 (0.0007) [2023-12-26 23:01:28,497][105620] Updated weights for policy 1, policy_version 1051308 (0.0005) [2023-12-26 23:01:28,549][105620] Updated weights for policy 1, policy_version 1051318 (0.0005) [2023-12-26 23:01:28,602][105620] Updated weights for policy 1, policy_version 1051328 (0.0005) [2023-12-26 23:01:28,677][105692] Updated weights for policy 0, policy_version 1050372 (0.0008) [2023-12-26 23:01:28,747][105692] Updated weights for policy 0, policy_version 1050382 (0.0009) [2023-12-26 23:01:28,814][105692] Updated weights for policy 0, policy_version 1050392 (0.0010) [2023-12-26 23:01:29,125][105620] Updated weights for policy 1, policy_version 1051338 (0.0005) [2023-12-26 23:01:29,169][105620] Updated weights for policy 1, policy_version 1051348 (0.0005) [2023-12-26 23:01:29,220][105620] Updated weights for policy 1, policy_version 1051358 (0.0006) [2023-12-26 23:01:29,276][105620] Updated weights for policy 1, policy_version 1051368 (0.0007) [2023-12-26 23:01:29,606][105692] Updated weights for policy 0, policy_version 1050402 (0.0011) [2023-12-26 23:01:29,660][105692] Updated weights for policy 0, policy_version 1050412 (0.0008) [2023-12-26 23:01:29,717][105692] Updated weights for policy 0, policy_version 1050422 (0.0009) [2023-12-26 23:01:29,770][105692] Updated weights for policy 0, policy_version 1050432 (0.0010) [2023-12-26 23:01:29,934][105620] Updated weights for policy 1, policy_version 1051378 (0.0010) [2023-12-26 23:01:30,002][105620] Updated weights for policy 1, policy_version 1051388 (0.0009) [2023-12-26 23:01:30,052][105620] Updated weights for policy 1, policy_version 1051398 (0.0005) [2023-12-26 23:01:30,598][105692] Updated weights for policy 0, policy_version 1050442 (0.0007) [2023-12-26 23:01:30,619][105620] Updated weights for policy 1, policy_version 1051408 (0.0009) [2023-12-26 23:01:30,650][105692] Updated weights for policy 0, policy_version 1050452 (0.0008) [2023-12-26 23:01:30,675][105620] Updated weights for policy 1, policy_version 1051418 (0.0006) [2023-12-26 23:01:30,702][105692] Updated weights for policy 0, policy_version 1050462 (0.0008) [2023-12-26 23:01:30,725][105620] Updated weights for policy 1, policy_version 1051428 (0.0005) [2023-12-26 23:01:31,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 538157056. Throughput: 0: 9178.2, 1: 10077.9. Samples: 538124448. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 23:01:31,062][104569] Avg episode reward: [(0, '8828.881'), (1, '9260.408')] [2023-12-26 23:01:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001050464_268959744.pth... [2023-12-26 23:01:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001051432_269197312.pth... [2023-12-26 23:01:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001049408_268689408.pth [2023-12-26 23:01:31,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001050216_268886016.pth [2023-12-26 23:01:31,417][105620] Updated weights for policy 1, policy_version 1051438 (0.0005) [2023-12-26 23:01:31,468][105620] Updated weights for policy 1, policy_version 1051448 (0.0009) [2023-12-26 23:01:31,529][105620] Updated weights for policy 1, policy_version 1051458 (0.0010) [2023-12-26 23:01:31,532][105692] Updated weights for policy 0, policy_version 1050472 (0.0006) [2023-12-26 23:01:31,587][105692] Updated weights for policy 0, policy_version 1050482 (0.0007) [2023-12-26 23:01:31,649][105692] Updated weights for policy 0, policy_version 1050492 (0.0007) [2023-12-26 23:01:32,257][105692] Updated weights for policy 0, policy_version 1050502 (0.0007) [2023-12-26 23:01:32,293][105620] Updated weights for policy 1, policy_version 1051468 (0.0010) [2023-12-26 23:01:32,318][105692] Updated weights for policy 0, policy_version 1050512 (0.0005) [2023-12-26 23:01:32,356][105620] Updated weights for policy 1, policy_version 1051478 (0.0010) [2023-12-26 23:01:32,384][105692] Updated weights for policy 0, policy_version 1050522 (0.0008) [2023-12-26 23:01:32,420][105620] Updated weights for policy 1, policy_version 1051488 (0.0008) [2023-12-26 23:01:33,065][105692] Updated weights for policy 0, policy_version 1050532 (0.0008) [2023-12-26 23:01:33,096][105620] Updated weights for policy 1, policy_version 1051498 (0.0007) [2023-12-26 23:01:33,127][105692] Updated weights for policy 0, policy_version 1050542 (0.0010) [2023-12-26 23:01:33,148][105620] Updated weights for policy 1, policy_version 1051508 (0.0005) [2023-12-26 23:01:33,181][105692] Updated weights for policy 0, policy_version 1050552 (0.0010) [2023-12-26 23:01:33,192][105620] Updated weights for policy 1, policy_version 1051518 (0.0006) [2023-12-26 23:01:33,245][105620] Updated weights for policy 1, policy_version 1051528 (0.0007) [2023-12-26 23:01:33,764][105692] Updated weights for policy 0, policy_version 1050562 (0.0010) [2023-12-26 23:01:33,823][105692] Updated weights for policy 0, policy_version 1050572 (0.0010) [2023-12-26 23:01:33,884][105692] Updated weights for policy 0, policy_version 1050582 (0.0010) [2023-12-26 23:01:33,944][105692] Updated weights for policy 0, policy_version 1050592 (0.0010) [2023-12-26 23:01:33,996][105620] Updated weights for policy 1, policy_version 1051538 (0.0010) [2023-12-26 23:01:34,050][105620] Updated weights for policy 1, policy_version 1051548 (0.0010) [2023-12-26 23:01:34,094][105620] Updated weights for policy 1, policy_version 1051558 (0.0010) [2023-12-26 23:01:34,712][105692] Updated weights for policy 0, policy_version 1050602 (0.0008) [2023-12-26 23:01:34,767][105692] Updated weights for policy 0, policy_version 1050612 (0.0009) [2023-12-26 23:01:34,773][105620] Updated weights for policy 1, policy_version 1051568 (0.0007) [2023-12-26 23:01:34,824][105692] Updated weights for policy 0, policy_version 1050622 (0.0006) [2023-12-26 23:01:34,834][105620] Updated weights for policy 1, policy_version 1051578 (0.0007) [2023-12-26 23:01:34,893][105620] Updated weights for policy 1, policy_version 1051588 (0.0009) [2023-12-26 23:01:35,598][105620] Updated weights for policy 1, policy_version 1051598 (0.0007) [2023-12-26 23:01:35,611][105692] Updated weights for policy 0, policy_version 1050632 (0.0008) [2023-12-26 23:01:35,655][105620] Updated weights for policy 1, policy_version 1051608 (0.0005) [2023-12-26 23:01:35,680][105692] Updated weights for policy 0, policy_version 1050642 (0.0007) [2023-12-26 23:01:35,722][105620] Updated weights for policy 1, policy_version 1051618 (0.0005) [2023-12-26 23:01:35,738][105692] Updated weights for policy 0, policy_version 1050652 (0.0007) [2023-12-26 23:01:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 538255360. Throughput: 0: 9161.2, 1: 10060.4. Samples: 538242572. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 23:01:36,063][104569] Avg episode reward: [(0, '8221.590'), (1, '8898.341')] [2023-12-26 23:01:36,377][105620] Updated weights for policy 1, policy_version 1051628 (0.0007) [2023-12-26 23:01:36,436][105620] Updated weights for policy 1, policy_version 1051638 (0.0006) [2023-12-26 23:01:36,496][105620] Updated weights for policy 1, policy_version 1051648 (0.0010) [2023-12-26 23:01:36,531][105692] Updated weights for policy 0, policy_version 1050662 (0.0007) [2023-12-26 23:01:36,589][105692] Updated weights for policy 0, policy_version 1050672 (0.0008) [2023-12-26 23:01:36,640][105692] Updated weights for policy 0, policy_version 1050682 (0.0008) [2023-12-26 23:01:37,153][105620] Updated weights for policy 1, policy_version 1051658 (0.0011) [2023-12-26 23:01:37,204][105620] Updated weights for policy 1, policy_version 1051668 (0.0007) [2023-12-26 23:01:37,253][105620] Updated weights for policy 1, policy_version 1051678 (0.0005) [2023-12-26 23:01:37,308][105620] Updated weights for policy 1, policy_version 1051688 (0.0005) [2023-12-26 23:01:37,489][105692] Updated weights for policy 0, policy_version 1050692 (0.0009) [2023-12-26 23:01:37,536][105692] Updated weights for policy 0, policy_version 1050702 (0.0009) [2023-12-26 23:01:37,596][105692] Updated weights for policy 0, policy_version 1050713 (0.0010) [2023-12-26 23:01:37,912][105620] Updated weights for policy 1, policy_version 1051698 (0.0006) [2023-12-26 23:01:37,978][105620] Updated weights for policy 1, policy_version 1051708 (0.0005) [2023-12-26 23:01:38,033][105620] Updated weights for policy 1, policy_version 1051718 (0.0007) [2023-12-26 23:01:38,406][105692] Updated weights for policy 0, policy_version 1050723 (0.0009) [2023-12-26 23:01:38,456][105692] Updated weights for policy 0, policy_version 1050733 (0.0009) [2023-12-26 23:01:38,510][105692] Updated weights for policy 0, policy_version 1050743 (0.0009) [2023-12-26 23:01:38,729][105620] Updated weights for policy 1, policy_version 1051728 (0.0009) [2023-12-26 23:01:38,799][105620] Updated weights for policy 1, policy_version 1051738 (0.0009) [2023-12-26 23:01:38,856][105620] Updated weights for policy 1, policy_version 1051748 (0.0007) [2023-12-26 23:01:39,326][105692] Updated weights for policy 0, policy_version 1050753 (0.0008) [2023-12-26 23:01:39,415][105692] Updated weights for policy 0, policy_version 1050763 (0.0007) [2023-12-26 23:01:39,470][105692] Updated weights for policy 0, policy_version 1050773 (0.0009) [2023-12-26 23:01:39,525][105692] Updated weights for policy 0, policy_version 1050783 (0.0009) [2023-12-26 23:01:39,540][105620] Updated weights for policy 1, policy_version 1051758 (0.0006) [2023-12-26 23:01:39,607][105620] Updated weights for policy 1, policy_version 1051768 (0.0006) [2023-12-26 23:01:39,673][105620] Updated weights for policy 1, policy_version 1051778 (0.0006) [2023-12-26 23:01:40,341][105620] Updated weights for policy 1, policy_version 1051788 (0.0007) [2023-12-26 23:01:40,356][105692] Updated weights for policy 0, policy_version 1050793 (0.0008) [2023-12-26 23:01:40,398][105620] Updated weights for policy 1, policy_version 1051798 (0.0008) [2023-12-26 23:01:40,416][105692] Updated weights for policy 0, policy_version 1050803 (0.0007) [2023-12-26 23:01:40,456][105620] Updated weights for policy 1, policy_version 1051808 (0.0008) [2023-12-26 23:01:40,473][105692] Updated weights for policy 0, policy_version 1050813 (0.0009) [2023-12-26 23:01:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 538345472. Throughput: 0: 9108.7, 1: 10200.7. Samples: 538356292. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 23:01:41,062][104569] Avg episode reward: [(0, '8221.003'), (1, '8716.088')] [2023-12-26 23:01:41,202][105620] Updated weights for policy 1, policy_version 1051818 (0.0008) [2023-12-26 23:01:41,252][105692] Updated weights for policy 0, policy_version 1050823 (0.0008) [2023-12-26 23:01:41,261][105620] Updated weights for policy 1, policy_version 1051828 (0.0008) [2023-12-26 23:01:41,315][105692] Updated weights for policy 0, policy_version 1050833 (0.0008) [2023-12-26 23:01:41,324][105620] Updated weights for policy 1, policy_version 1051838 (0.0008) [2023-12-26 23:01:41,392][105692] Updated weights for policy 0, policy_version 1050843 (0.0010) [2023-12-26 23:01:41,395][105620] Updated weights for policy 1, policy_version 1051848 (0.0008) [2023-12-26 23:01:42,108][105620] Updated weights for policy 1, policy_version 1051858 (0.0006) [2023-12-26 23:01:42,135][105692] Updated weights for policy 0, policy_version 1050853 (0.0009) [2023-12-26 23:01:42,173][105620] Updated weights for policy 1, policy_version 1051868 (0.0008) [2023-12-26 23:01:42,192][105692] Updated weights for policy 0, policy_version 1050863 (0.0006) [2023-12-26 23:01:42,240][105620] Updated weights for policy 1, policy_version 1051878 (0.0008) [2023-12-26 23:01:42,241][105692] Updated weights for policy 0, policy_version 1050873 (0.0007) [2023-12-26 23:01:42,883][105620] Updated weights for policy 1, policy_version 1051888 (0.0008) [2023-12-26 23:01:42,911][105692] Updated weights for policy 0, policy_version 1050883 (0.0009) [2023-12-26 23:01:42,939][105620] Updated weights for policy 1, policy_version 1051898 (0.0009) [2023-12-26 23:01:42,960][105692] Updated weights for policy 0, policy_version 1050893 (0.0005) [2023-12-26 23:01:42,988][105620] Updated weights for policy 1, policy_version 1051908 (0.0009) [2023-12-26 23:01:43,014][105692] Updated weights for policy 0, policy_version 1050903 (0.0005) [2023-12-26 23:01:43,595][105692] Updated weights for policy 0, policy_version 1050913 (0.0005) [2023-12-26 23:01:43,650][105692] Updated weights for policy 0, policy_version 1050923 (0.0005) [2023-12-26 23:01:43,709][105692] Updated weights for policy 0, policy_version 1050933 (0.0005) [2023-12-26 23:01:43,776][105692] Updated weights for policy 0, policy_version 1050943 (0.0005) [2023-12-26 23:01:43,808][105620] Updated weights for policy 1, policy_version 1051918 (0.0009) [2023-12-26 23:01:43,870][105620] Updated weights for policy 1, policy_version 1051928 (0.0008) [2023-12-26 23:01:43,922][105620] Updated weights for policy 1, policy_version 1051938 (0.0008) [2023-12-26 23:01:44,423][105692] Updated weights for policy 0, policy_version 1050953 (0.0007) [2023-12-26 23:01:44,481][105692] Updated weights for policy 0, policy_version 1050963 (0.0009) [2023-12-26 23:01:44,528][105692] Updated weights for policy 0, policy_version 1050973 (0.0009) [2023-12-26 23:01:44,660][105620] Updated weights for policy 1, policy_version 1051948 (0.0008) [2023-12-26 23:01:44,710][105620] Updated weights for policy 1, policy_version 1051958 (0.0009) [2023-12-26 23:01:44,757][105620] Updated weights for policy 1, policy_version 1051968 (0.0008) [2023-12-26 23:01:45,329][105692] Updated weights for policy 0, policy_version 1050983 (0.0009) [2023-12-26 23:01:45,391][105692] Updated weights for policy 0, policy_version 1050993 (0.0009) [2023-12-26 23:01:45,457][105692] Updated weights for policy 0, policy_version 1051003 (0.0009) [2023-12-26 23:01:45,481][105620] Updated weights for policy 1, policy_version 1051978 (0.0008) [2023-12-26 23:01:45,535][105620] Updated weights for policy 1, policy_version 1051988 (0.0008) [2023-12-26 23:01:45,601][105620] Updated weights for policy 1, policy_version 1051998 (0.0009) [2023-12-26 23:01:45,659][105620] Updated weights for policy 1, policy_version 1052008 (0.0008) [2023-12-26 23:01:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.1, 300 sec: 19549.7). Total num frames: 538443776. Throughput: 0: 9181.4, 1: 10207.2. Samples: 538414776. Policy #0 lag: (min: 20.0, avg: 20.6, max: 32.0) [2023-12-26 23:01:46,063][104569] Avg episode reward: [(0, '8828.372'), (1, '8862.537')] [2023-12-26 23:01:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001052008_269344768.pth... [2023-12-26 23:01:46,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001050792_269033472.pth [2023-12-26 23:01:46,081][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_001052008_269344768.pth [2023-12-26 23:01:46,109][105692] Updated weights for policy 0, policy_version 1051013 (0.0010) [2023-12-26 23:01:46,165][105692] Updated weights for policy 0, policy_version 1051023 (0.0011) [2023-12-26 23:01:46,213][105692] Updated weights for policy 0, policy_version 1051033 (0.0005) [2023-12-26 23:01:46,251][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001051040_269107200.pth... [2023-12-26 23:01:46,255][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001049952_268828672.pth [2023-12-26 23:01:46,255][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_001051040_269107200.pth [2023-12-26 23:01:46,388][105620] Updated weights for policy 1, policy_version 1052018 (0.0010) [2023-12-26 23:01:46,446][105620] Updated weights for policy 1, policy_version 1052028 (0.0007) [2023-12-26 23:01:46,492][105620] Updated weights for policy 1, policy_version 1052038 (0.0005) [2023-12-26 23:01:46,799][105692] Updated weights for policy 0, policy_version 1051043 (0.0007) [2023-12-26 23:01:46,844][105692] Updated weights for policy 0, policy_version 1051053 (0.0008) [2023-12-26 23:01:46,902][105692] Updated weights for policy 0, policy_version 1051063 (0.0005) [2023-12-26 23:01:47,283][105620] Updated weights for policy 1, policy_version 1052048 (0.0008) [2023-12-26 23:01:47,340][105620] Updated weights for policy 1, policy_version 1052058 (0.0009) [2023-12-26 23:01:47,403][105620] Updated weights for policy 1, policy_version 1052068 (0.0008) [2023-12-26 23:01:47,433][105692] Updated weights for policy 0, policy_version 1051073 (0.0005) [2023-12-26 23:01:47,491][105692] Updated weights for policy 0, policy_version 1051083 (0.0005) [2023-12-26 23:01:47,552][105692] Updated weights for policy 0, policy_version 1051093 (0.0005) [2023-12-26 23:01:47,618][105692] Updated weights for policy 0, policy_version 1051103 (0.0005) [2023-12-26 23:01:48,104][105620] Updated weights for policy 1, policy_version 1052078 (0.0006) [2023-12-26 23:01:48,118][105692] Updated weights for policy 0, policy_version 1051113 (0.0010) [2023-12-26 23:01:48,164][105620] Updated weights for policy 1, policy_version 1052088 (0.0005) [2023-12-26 23:01:48,170][105692] Updated weights for policy 0, policy_version 1051123 (0.0010) [2023-12-26 23:01:48,220][105620] Updated weights for policy 1, policy_version 1052098 (0.0010) [2023-12-26 23:01:48,226][105692] Updated weights for policy 0, policy_version 1051133 (0.0011) [2023-12-26 23:01:48,909][105620] Updated weights for policy 1, policy_version 1052108 (0.0009) [2023-12-26 23:01:48,972][105692] Updated weights for policy 0, policy_version 1051143 (0.0008) [2023-12-26 23:01:48,977][105620] Updated weights for policy 1, policy_version 1052118 (0.0007) [2023-12-26 23:01:49,037][105692] Updated weights for policy 0, policy_version 1051153 (0.0005) [2023-12-26 23:01:49,041][105620] Updated weights for policy 1, policy_version 1052128 (0.0008) [2023-12-26 23:01:49,097][105692] Updated weights for policy 0, policy_version 1051163 (0.0008) [2023-12-26 23:01:49,757][105692] Updated weights for policy 0, policy_version 1051173 (0.0008) [2023-12-26 23:01:49,782][105620] Updated weights for policy 1, policy_version 1052138 (0.0009) [2023-12-26 23:01:49,820][105692] Updated weights for policy 0, policy_version 1051183 (0.0008) [2023-12-26 23:01:49,846][105620] Updated weights for policy 1, policy_version 1052148 (0.0010) [2023-12-26 23:01:49,881][105692] Updated weights for policy 0, policy_version 1051193 (0.0008) [2023-12-26 23:01:49,902][105620] Updated weights for policy 1, policy_version 1052158 (0.0010) [2023-12-26 23:01:49,979][105620] Updated weights for policy 1, policy_version 1052168 (0.0010) [2023-12-26 23:01:50,531][105692] Updated weights for policy 0, policy_version 1051203 (0.0008) [2023-12-26 23:01:50,600][105692] Updated weights for policy 0, policy_version 1051213 (0.0010) [2023-12-26 23:01:50,660][105692] Updated weights for policy 0, policy_version 1051223 (0.0006) [2023-12-26 23:01:50,673][105620] Updated weights for policy 1, policy_version 1052178 (0.0009) [2023-12-26 23:01:50,746][105620] Updated weights for policy 1, policy_version 1052188 (0.0010) [2023-12-26 23:01:50,807][105620] Updated weights for policy 1, policy_version 1052198 (0.0009) [2023-12-26 23:01:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 538550272. Throughput: 0: 9388.5, 1: 10187.5. Samples: 538536104. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:01:51,062][104569] Avg episode reward: [(0, '9171.708'), (1, '8972.502')] [2023-12-26 23:01:51,430][105692] Updated weights for policy 0, policy_version 1051233 (0.0007) [2023-12-26 23:01:51,492][105692] Updated weights for policy 0, policy_version 1051243 (0.0006) [2023-12-26 23:01:51,532][105620] Updated weights for policy 1, policy_version 1052208 (0.0007) [2023-12-26 23:01:51,558][105692] Updated weights for policy 0, policy_version 1051253 (0.0008) [2023-12-26 23:01:51,598][105620] Updated weights for policy 1, policy_version 1052218 (0.0006) [2023-12-26 23:01:51,619][105692] Updated weights for policy 0, policy_version 1051263 (0.0007) [2023-12-26 23:01:51,665][105620] Updated weights for policy 1, policy_version 1052228 (0.0008) [2023-12-26 23:01:52,382][105620] Updated weights for policy 1, policy_version 1052238 (0.0009) [2023-12-26 23:01:52,394][105692] Updated weights for policy 0, policy_version 1051273 (0.0008) [2023-12-26 23:01:52,441][105620] Updated weights for policy 1, policy_version 1052248 (0.0010) [2023-12-26 23:01:52,451][105692] Updated weights for policy 0, policy_version 1051283 (0.0007) [2023-12-26 23:01:52,502][105620] Updated weights for policy 1, policy_version 1052258 (0.0010) [2023-12-26 23:01:52,509][105692] Updated weights for policy 0, policy_version 1051293 (0.0007) [2023-12-26 23:01:53,140][105692] Updated weights for policy 0, policy_version 1051303 (0.0009) [2023-12-26 23:01:53,195][105692] Updated weights for policy 0, policy_version 1051313 (0.0010) [2023-12-26 23:01:53,225][105620] Updated weights for policy 1, policy_version 1052268 (0.0010) [2023-12-26 23:01:53,249][105692] Updated weights for policy 0, policy_version 1051323 (0.0010) [2023-12-26 23:01:53,287][105620] Updated weights for policy 1, policy_version 1052278 (0.0010) [2023-12-26 23:01:53,345][105620] Updated weights for policy 1, policy_version 1052288 (0.0011) [2023-12-26 23:01:53,919][105620] Updated weights for policy 1, policy_version 1052298 (0.0008) [2023-12-26 23:01:53,966][105692] Updated weights for policy 0, policy_version 1051333 (0.0010) [2023-12-26 23:01:53,974][105620] Updated weights for policy 1, policy_version 1052308 (0.0010) [2023-12-26 23:01:54,011][105692] Updated weights for policy 0, policy_version 1051343 (0.0010) [2023-12-26 23:01:54,032][105620] Updated weights for policy 1, policy_version 1052318 (0.0010) [2023-12-26 23:01:54,059][105692] Updated weights for policy 0, policy_version 1051353 (0.0010) [2023-12-26 23:01:54,090][105620] Updated weights for policy 1, policy_version 1052328 (0.0010) [2023-12-26 23:01:54,806][105692] Updated weights for policy 0, policy_version 1051363 (0.0011) [2023-12-26 23:01:54,836][105620] Updated weights for policy 1, policy_version 1052338 (0.0010) [2023-12-26 23:01:54,861][105692] Updated weights for policy 0, policy_version 1051373 (0.0010) [2023-12-26 23:01:54,885][105620] Updated weights for policy 1, policy_version 1052348 (0.0010) [2023-12-26 23:01:54,923][105692] Updated weights for policy 0, policy_version 1051383 (0.0010) [2023-12-26 23:01:54,940][105620] Updated weights for policy 1, policy_version 1052358 (0.0010) [2023-12-26 23:01:55,597][105692] Updated weights for policy 0, policy_version 1051393 (0.0009) [2023-12-26 23:01:55,653][105692] Updated weights for policy 0, policy_version 1051403 (0.0006) [2023-12-26 23:01:55,659][105620] Updated weights for policy 1, policy_version 1052368 (0.0010) [2023-12-26 23:01:55,707][105620] Updated weights for policy 1, policy_version 1052378 (0.0010) [2023-12-26 23:01:55,709][105692] Updated weights for policy 0, policy_version 1051413 (0.0005) [2023-12-26 23:01:55,759][105620] Updated weights for policy 1, policy_version 1052388 (0.0010) [2023-12-26 23:01:55,763][105692] Updated weights for policy 0, policy_version 1051423 (0.0005) [2023-12-26 23:01:56,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 538648576. Throughput: 0: 9436.0, 1: 10182.6. Samples: 538653976. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:01:56,063][104569] Avg episode reward: [(0, '9175.262'), (1, '8621.811')] [2023-12-26 23:01:56,333][105692] Updated weights for policy 0, policy_version 1051433 (0.0007) [2023-12-26 23:01:56,392][105692] Updated weights for policy 0, policy_version 1051443 (0.0008) [2023-12-26 23:01:56,448][105692] Updated weights for policy 0, policy_version 1051453 (0.0008) [2023-12-26 23:01:56,489][105620] Updated weights for policy 1, policy_version 1052398 (0.0010) [2023-12-26 23:01:56,548][105620] Updated weights for policy 1, policy_version 1052408 (0.0010) [2023-12-26 23:01:56,599][105620] Updated weights for policy 1, policy_version 1052418 (0.0010) [2023-12-26 23:01:57,029][105692] Updated weights for policy 0, policy_version 1051463 (0.0006) [2023-12-26 23:01:57,089][105692] Updated weights for policy 0, policy_version 1051473 (0.0005) [2023-12-26 23:01:57,143][105692] Updated weights for policy 0, policy_version 1051483 (0.0005) [2023-12-26 23:01:57,307][105620] Updated weights for policy 1, policy_version 1052428 (0.0009) [2023-12-26 23:01:57,355][105620] Updated weights for policy 1, policy_version 1052438 (0.0007) [2023-12-26 23:01:57,408][105620] Updated weights for policy 1, policy_version 1052448 (0.0005) [2023-12-26 23:01:57,800][105692] Updated weights for policy 0, policy_version 1051493 (0.0005) [2023-12-26 23:01:57,859][105692] Updated weights for policy 0, policy_version 1051503 (0.0005) [2023-12-26 23:01:57,910][105692] Updated weights for policy 0, policy_version 1051513 (0.0005) [2023-12-26 23:01:57,948][105620] Updated weights for policy 1, policy_version 1052458 (0.0005) [2023-12-26 23:01:58,008][105620] Updated weights for policy 1, policy_version 1052468 (0.0006) [2023-12-26 23:01:58,076][105620] Updated weights for policy 1, policy_version 1052478 (0.0010) [2023-12-26 23:01:58,120][105620] Updated weights for policy 1, policy_version 1052488 (0.0010) [2023-12-26 23:01:58,494][105692] Updated weights for policy 0, policy_version 1051523 (0.0006) [2023-12-26 23:01:58,556][105692] Updated weights for policy 0, policy_version 1051533 (0.0007) [2023-12-26 23:01:58,619][105692] Updated weights for policy 0, policy_version 1051543 (0.0006) [2023-12-26 23:01:58,948][105620] Updated weights for policy 1, policy_version 1052498 (0.0009) [2023-12-26 23:01:59,015][105620] Updated weights for policy 1, policy_version 1052508 (0.0010) [2023-12-26 23:01:59,084][105620] Updated weights for policy 1, policy_version 1052518 (0.0011) [2023-12-26 23:01:59,299][105692] Updated weights for policy 0, policy_version 1051553 (0.0007) [2023-12-26 23:01:59,366][105692] Updated weights for policy 0, policy_version 1051563 (0.0008) [2023-12-26 23:01:59,431][105692] Updated weights for policy 0, policy_version 1051573 (0.0008) [2023-12-26 23:01:59,495][105692] Updated weights for policy 0, policy_version 1051583 (0.0007) [2023-12-26 23:01:59,872][105620] Updated weights for policy 1, policy_version 1052528 (0.0009) [2023-12-26 23:01:59,932][105620] Updated weights for policy 1, policy_version 1052538 (0.0009) [2023-12-26 23:01:59,993][105620] Updated weights for policy 1, policy_version 1052548 (0.0009) [2023-12-26 23:02:00,156][105692] Updated weights for policy 0, policy_version 1051593 (0.0006) [2023-12-26 23:02:00,219][105692] Updated weights for policy 0, policy_version 1051603 (0.0005) [2023-12-26 23:02:00,274][105692] Updated weights for policy 0, policy_version 1051613 (0.0005) [2023-12-26 23:02:00,798][105620] Updated weights for policy 1, policy_version 1052559 (0.0010) [2023-12-26 23:02:00,849][105620] Updated weights for policy 1, policy_version 1052569 (0.0009) [2023-12-26 23:02:00,890][105692] Updated weights for policy 0, policy_version 1051623 (0.0008) [2023-12-26 23:02:00,908][105620] Updated weights for policy 1, policy_version 1052579 (0.0008) [2023-12-26 23:02:00,945][105692] Updated weights for policy 0, policy_version 1051633 (0.0006) [2023-12-26 23:02:00,991][105692] Updated weights for policy 0, policy_version 1051643 (0.0009) [2023-12-26 23:02:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 538755072. Throughput: 0: 9573.0, 1: 10204.9. Samples: 538718424. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:02:01,062][104569] Avg episode reward: [(0, '8729.284'), (1, '8703.064')] [2023-12-26 23:02:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001051648_269262848.pth... [2023-12-26 23:02:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001052584_269492224.pth... [2023-12-26 23:02:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001050464_268959744.pth [2023-12-26 23:02:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001051432_269197312.pth [2023-12-26 23:02:01,598][105620] Updated weights for policy 1, policy_version 1052589 (0.0006) [2023-12-26 23:02:01,666][105620] Updated weights for policy 1, policy_version 1052599 (0.0007) [2023-12-26 23:02:01,728][105620] Updated weights for policy 1, policy_version 1052609 (0.0008) [2023-12-26 23:02:01,729][105692] Updated weights for policy 0, policy_version 1051653 (0.0009) [2023-12-26 23:02:01,780][105692] Updated weights for policy 0, policy_version 1051663 (0.0008) [2023-12-26 23:02:01,828][105692] Updated weights for policy 0, policy_version 1051673 (0.0008) [2023-12-26 23:02:02,329][105620] Updated weights for policy 1, policy_version 1052619 (0.0008) [2023-12-26 23:02:02,398][105620] Updated weights for policy 1, policy_version 1052629 (0.0008) [2023-12-26 23:02:02,443][105620] Updated weights for policy 1, policy_version 1052639 (0.0008) [2023-12-26 23:02:02,629][105692] Updated weights for policy 0, policy_version 1051683 (0.0008) [2023-12-26 23:02:02,683][105692] Updated weights for policy 0, policy_version 1051693 (0.0009) [2023-12-26 23:02:02,735][105692] Updated weights for policy 0, policy_version 1051703 (0.0009) [2023-12-26 23:02:03,050][105620] Updated weights for policy 1, policy_version 1052649 (0.0005) [2023-12-26 23:02:03,100][105620] Updated weights for policy 1, policy_version 1052659 (0.0005) [2023-12-26 23:02:03,144][105620] Updated weights for policy 1, policy_version 1052669 (0.0005) [2023-12-26 23:02:03,201][105620] Updated weights for policy 1, policy_version 1052679 (0.0006) [2023-12-26 23:02:03,555][105692] Updated weights for policy 0, policy_version 1051713 (0.0009) [2023-12-26 23:02:03,607][105692] Updated weights for policy 0, policy_version 1051723 (0.0008) [2023-12-26 23:02:03,661][105692] Updated weights for policy 0, policy_version 1051734 (0.0008) [2023-12-26 23:02:03,707][105692] Updated weights for policy 0, policy_version 1051744 (0.0008) [2023-12-26 23:02:03,882][105620] Updated weights for policy 1, policy_version 1052689 (0.0008) [2023-12-26 23:02:03,951][105620] Updated weights for policy 1, policy_version 1052699 (0.0009) [2023-12-26 23:02:04,011][105620] Updated weights for policy 1, policy_version 1052709 (0.0009) [2023-12-26 23:02:04,520][105692] Updated weights for policy 0, policy_version 1051754 (0.0006) [2023-12-26 23:02:04,585][105692] Updated weights for policy 0, policy_version 1051764 (0.0007) [2023-12-26 23:02:04,642][105692] Updated weights for policy 0, policy_version 1051774 (0.0005) [2023-12-26 23:02:04,700][105620] Updated weights for policy 1, policy_version 1052719 (0.0007) [2023-12-26 23:02:04,761][105620] Updated weights for policy 1, policy_version 1052729 (0.0008) [2023-12-26 23:02:04,817][105620] Updated weights for policy 1, policy_version 1052739 (0.0009) [2023-12-26 23:02:05,284][105692] Updated weights for policy 0, policy_version 1051784 (0.0008) [2023-12-26 23:02:05,350][105692] Updated weights for policy 0, policy_version 1051794 (0.0009) [2023-12-26 23:02:05,403][105692] Updated weights for policy 0, policy_version 1051804 (0.0008) [2023-12-26 23:02:05,541][105620] Updated weights for policy 1, policy_version 1052749 (0.0009) [2023-12-26 23:02:05,598][105620] Updated weights for policy 1, policy_version 1052759 (0.0009) [2023-12-26 23:02:05,669][105620] Updated weights for policy 1, policy_version 1052769 (0.0010) [2023-12-26 23:02:06,027][105692] Updated weights for policy 0, policy_version 1051814 (0.0009) [2023-12-26 23:02:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 538845184. Throughput: 0: 9649.3, 1: 10133.6. Samples: 538835320. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:02:06,062][104569] Avg episode reward: [(0, '8458.513'), (1, '8991.628')] [2023-12-26 23:02:06,075][105692] Updated weights for policy 0, policy_version 1051824 (0.0009) [2023-12-26 23:02:06,135][105692] Updated weights for policy 0, policy_version 1051834 (0.0008) [2023-12-26 23:02:06,421][105620] Updated weights for policy 1, policy_version 1052779 (0.0010) [2023-12-26 23:02:06,490][105620] Updated weights for policy 1, policy_version 1052789 (0.0006) [2023-12-26 23:02:06,554][105620] Updated weights for policy 1, policy_version 1052799 (0.0006) [2023-12-26 23:02:06,920][105692] Updated weights for policy 0, policy_version 1051844 (0.0008) [2023-12-26 23:02:06,985][105692] Updated weights for policy 0, policy_version 1051854 (0.0009) [2023-12-26 23:02:07,052][105692] Updated weights for policy 0, policy_version 1051864 (0.0007) [2023-12-26 23:02:07,206][105620] Updated weights for policy 1, policy_version 1052809 (0.0006) [2023-12-26 23:02:07,273][105620] Updated weights for policy 1, policy_version 1052819 (0.0005) [2023-12-26 23:02:07,336][105620] Updated weights for policy 1, policy_version 1052829 (0.0008) [2023-12-26 23:02:07,394][105620] Updated weights for policy 1, policy_version 1052839 (0.0008) [2023-12-26 23:02:07,785][105692] Updated weights for policy 0, policy_version 1051874 (0.0009) [2023-12-26 23:02:07,833][105692] Updated weights for policy 0, policy_version 1051884 (0.0009) [2023-12-26 23:02:07,884][105692] Updated weights for policy 0, policy_version 1051894 (0.0006) [2023-12-26 23:02:07,935][105692] Updated weights for policy 0, policy_version 1051904 (0.0007) [2023-12-26 23:02:08,131][105620] Updated weights for policy 1, policy_version 1052849 (0.0008) [2023-12-26 23:02:08,180][105620] Updated weights for policy 1, policy_version 1052859 (0.0009) [2023-12-26 23:02:08,237][105620] Updated weights for policy 1, policy_version 1052870 (0.0010) [2023-12-26 23:02:08,565][105692] Updated weights for policy 0, policy_version 1051914 (0.0009) [2023-12-26 23:02:08,617][105692] Updated weights for policy 0, policy_version 1051924 (0.0009) [2023-12-26 23:02:08,672][105692] Updated weights for policy 0, policy_version 1051934 (0.0009) [2023-12-26 23:02:09,025][105620] Updated weights for policy 1, policy_version 1052880 (0.0006) [2023-12-26 23:02:09,091][105620] Updated weights for policy 1, policy_version 1052890 (0.0005) [2023-12-26 23:02:09,138][105620] Updated weights for policy 1, policy_version 1052900 (0.0005) [2023-12-26 23:02:09,308][105692] Updated weights for policy 0, policy_version 1051944 (0.0009) [2023-12-26 23:02:09,382][105692] Updated weights for policy 0, policy_version 1051954 (0.0009) [2023-12-26 23:02:09,449][105692] Updated weights for policy 0, policy_version 1051964 (0.0009) [2023-12-26 23:02:09,812][105620] Updated weights for policy 1, policy_version 1052910 (0.0008) [2023-12-26 23:02:09,877][105620] Updated weights for policy 1, policy_version 1052920 (0.0008) [2023-12-26 23:02:09,938][105620] Updated weights for policy 1, policy_version 1052930 (0.0007) [2023-12-26 23:02:10,241][105692] Updated weights for policy 0, policy_version 1051974 (0.0007) [2023-12-26 23:02:10,303][105692] Updated weights for policy 0, policy_version 1051984 (0.0006) [2023-12-26 23:02:10,366][105692] Updated weights for policy 0, policy_version 1051994 (0.0007) [2023-12-26 23:02:10,717][105620] Updated weights for policy 1, policy_version 1052940 (0.0007) [2023-12-26 23:02:10,774][105620] Updated weights for policy 1, policy_version 1052950 (0.0008) [2023-12-26 23:02:10,824][105620] Updated weights for policy 1, policy_version 1052960 (0.0009) [2023-12-26 23:02:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 538943488. Throughput: 0: 9757.5, 1: 10001.0. Samples: 538952656. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:02:11,062][104569] Avg episode reward: [(0, '8457.093'), (1, '9260.754')] [2023-12-26 23:02:11,111][105692] Updated weights for policy 0, policy_version 1052004 (0.0007) [2023-12-26 23:02:11,175][105692] Updated weights for policy 0, policy_version 1052014 (0.0007) [2023-12-26 23:02:11,234][105692] Updated weights for policy 0, policy_version 1052024 (0.0006) [2023-12-26 23:02:11,555][105620] Updated weights for policy 1, policy_version 1052970 (0.0006) [2023-12-26 23:02:11,607][105620] Updated weights for policy 1, policy_version 1052980 (0.0006) [2023-12-26 23:02:11,676][105620] Updated weights for policy 1, policy_version 1052990 (0.0007) [2023-12-26 23:02:11,745][105620] Updated weights for policy 1, policy_version 1053000 (0.0008) [2023-12-26 23:02:11,988][105692] Updated weights for policy 0, policy_version 1052034 (0.0007) [2023-12-26 23:02:12,058][105692] Updated weights for policy 0, policy_version 1052044 (0.0007) [2023-12-26 23:02:12,127][105692] Updated weights for policy 0, policy_version 1052054 (0.0006) [2023-12-26 23:02:12,195][105692] Updated weights for policy 0, policy_version 1052064 (0.0006) [2023-12-26 23:02:12,459][105620] Updated weights for policy 1, policy_version 1053010 (0.0008) [2023-12-26 23:02:12,511][105620] Updated weights for policy 1, policy_version 1053020 (0.0008) [2023-12-26 23:02:12,570][105620] Updated weights for policy 1, policy_version 1053030 (0.0008) [2023-12-26 23:02:12,840][105692] Updated weights for policy 0, policy_version 1052074 (0.0005) [2023-12-26 23:02:12,896][105692] Updated weights for policy 0, policy_version 1052084 (0.0005) [2023-12-26 23:02:12,949][105692] Updated weights for policy 0, policy_version 1052094 (0.0009) [2023-12-26 23:02:13,364][105620] Updated weights for policy 1, policy_version 1053040 (0.0008) [2023-12-26 23:02:13,419][105620] Updated weights for policy 1, policy_version 1053050 (0.0007) [2023-12-26 23:02:13,481][105620] Updated weights for policy 1, policy_version 1053060 (0.0009) [2023-12-26 23:02:13,651][105692] Updated weights for policy 0, policy_version 1052104 (0.0007) [2023-12-26 23:02:13,709][105692] Updated weights for policy 0, policy_version 1052114 (0.0005) [2023-12-26 23:02:13,766][105692] Updated weights for policy 0, policy_version 1052124 (0.0008) [2023-12-26 23:02:14,138][105620] Updated weights for policy 1, policy_version 1053070 (0.0010) [2023-12-26 23:02:14,203][105620] Updated weights for policy 1, policy_version 1053080 (0.0010) [2023-12-26 23:02:14,267][105620] Updated weights for policy 1, policy_version 1053090 (0.0010) [2023-12-26 23:02:14,545][105692] Updated weights for policy 0, policy_version 1052135 (0.0010) [2023-12-26 23:02:14,597][105692] Updated weights for policy 0, policy_version 1052146 (0.0009) [2023-12-26 23:02:14,661][105692] Updated weights for policy 0, policy_version 1052156 (0.0009) [2023-12-26 23:02:14,846][105620] Updated weights for policy 1, policy_version 1053100 (0.0009) [2023-12-26 23:02:14,899][105620] Updated weights for policy 1, policy_version 1053110 (0.0008) [2023-12-26 23:02:14,948][105620] Updated weights for policy 1, policy_version 1053120 (0.0008) [2023-12-26 23:02:15,392][105692] Updated weights for policy 0, policy_version 1052166 (0.0007) [2023-12-26 23:02:15,451][105692] Updated weights for policy 0, policy_version 1052176 (0.0009) [2023-12-26 23:02:15,504][105692] Updated weights for policy 0, policy_version 1052186 (0.0009) [2023-12-26 23:02:15,736][105620] Updated weights for policy 1, policy_version 1053130 (0.0008) [2023-12-26 23:02:15,787][105620] Updated weights for policy 1, policy_version 1053140 (0.0005) [2023-12-26 23:02:15,833][105620] Updated weights for policy 1, policy_version 1053150 (0.0005) [2023-12-26 23:02:15,909][105620] Updated weights for policy 1, policy_version 1053160 (0.0005) [2023-12-26 23:02:16,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19797.2, 300 sec: 19605.2). Total num frames: 539041792. Throughput: 0: 9841.2, 1: 9845.3. Samples: 539010344. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:02:16,063][104569] Avg episode reward: [(0, '8549.752'), (1, '9169.624')] [2023-12-26 23:02:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001052192_269402112.pth... [2023-12-26 23:02:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001053160_269639680.pth... [2023-12-26 23:02:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001051040_269107200.pth [2023-12-26 23:02:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001052008_269344768.pth [2023-12-26 23:02:16,341][105692] Updated weights for policy 0, policy_version 1052196 (0.0009) [2023-12-26 23:02:16,397][105692] Updated weights for policy 0, policy_version 1052206 (0.0009) [2023-12-26 23:02:16,448][105692] Updated weights for policy 0, policy_version 1052216 (0.0009) [2023-12-26 23:02:16,473][105620] Updated weights for policy 1, policy_version 1053170 (0.0005) [2023-12-26 23:02:16,519][105620] Updated weights for policy 1, policy_version 1053180 (0.0009) [2023-12-26 23:02:16,568][105620] Updated weights for policy 1, policy_version 1053190 (0.0008) [2023-12-26 23:02:17,243][105692] Updated weights for policy 0, policy_version 1052226 (0.0009) [2023-12-26 23:02:17,291][105692] Updated weights for policy 0, policy_version 1052236 (0.0009) [2023-12-26 23:02:17,320][105620] Updated weights for policy 1, policy_version 1053200 (0.0007) [2023-12-26 23:02:17,339][105692] Updated weights for policy 0, policy_version 1052246 (0.0007) [2023-12-26 23:02:17,374][105620] Updated weights for policy 1, policy_version 1053210 (0.0007) [2023-12-26 23:02:17,397][105692] Updated weights for policy 0, policy_version 1052256 (0.0008) [2023-12-26 23:02:17,428][105620] Updated weights for policy 1, policy_version 1053220 (0.0007) [2023-12-26 23:02:18,189][105692] Updated weights for policy 0, policy_version 1052266 (0.0008) [2023-12-26 23:02:18,200][105620] Updated weights for policy 1, policy_version 1053230 (0.0009) [2023-12-26 23:02:18,235][105692] Updated weights for policy 0, policy_version 1052276 (0.0008) [2023-12-26 23:02:18,260][105620] Updated weights for policy 1, policy_version 1053240 (0.0008) [2023-12-26 23:02:18,286][105692] Updated weights for policy 0, policy_version 1052286 (0.0007) [2023-12-26 23:02:18,320][105620] Updated weights for policy 1, policy_version 1053250 (0.0008) [2023-12-26 23:02:18,911][105692] Updated weights for policy 0, policy_version 1052296 (0.0006) [2023-12-26 23:02:18,970][105692] Updated weights for policy 0, policy_version 1052306 (0.0009) [2023-12-26 23:02:19,023][105692] Updated weights for policy 0, policy_version 1052316 (0.0009) [2023-12-26 23:02:19,138][105620] Updated weights for policy 1, policy_version 1053260 (0.0008) [2023-12-26 23:02:19,191][105620] Updated weights for policy 1, policy_version 1053270 (0.0009) [2023-12-26 23:02:19,253][105620] Updated weights for policy 1, policy_version 1053280 (0.0009) [2023-12-26 23:02:19,796][105692] Updated weights for policy 0, policy_version 1052326 (0.0010) [2023-12-26 23:02:19,868][105692] Updated weights for policy 0, policy_version 1052336 (0.0009) [2023-12-26 23:02:19,932][105692] Updated weights for policy 0, policy_version 1052346 (0.0007) [2023-12-26 23:02:20,037][105620] Updated weights for policy 1, policy_version 1053290 (0.0009) [2023-12-26 23:02:20,093][105620] Updated weights for policy 1, policy_version 1053300 (0.0008) [2023-12-26 23:02:20,146][105620] Updated weights for policy 1, policy_version 1053310 (0.0011) [2023-12-26 23:02:20,200][105620] Updated weights for policy 1, policy_version 1053320 (0.0011) [2023-12-26 23:02:20,662][105692] Updated weights for policy 0, policy_version 1052356 (0.0008) [2023-12-26 23:02:20,720][105692] Updated weights for policy 0, policy_version 1052366 (0.0008) [2023-12-26 23:02:20,789][105692] Updated weights for policy 0, policy_version 1052376 (0.0009) [2023-12-26 23:02:20,973][105620] Updated weights for policy 1, policy_version 1053330 (0.0011) [2023-12-26 23:02:21,029][105620] Updated weights for policy 1, policy_version 1053340 (0.0010) [2023-12-26 23:02:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 539131904. Throughput: 0: 9830.6, 1: 9778.6. Samples: 539124984. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:02:21,063][104569] Avg episode reward: [(0, '8994.375'), (1, '9169.636')] [2023-12-26 23:02:21,095][105620] Updated weights for policy 1, policy_version 1053350 (0.0009) [2023-12-26 23:02:21,590][105692] Updated weights for policy 0, policy_version 1052386 (0.0009) [2023-12-26 23:02:21,660][105692] Updated weights for policy 0, policy_version 1052396 (0.0009) [2023-12-26 23:02:21,725][105692] Updated weights for policy 0, policy_version 1052406 (0.0009) [2023-12-26 23:02:21,779][105692] Updated weights for policy 0, policy_version 1052416 (0.0008) [2023-12-26 23:02:21,819][105620] Updated weights for policy 1, policy_version 1053360 (0.0010) [2023-12-26 23:02:21,887][105620] Updated weights for policy 1, policy_version 1053370 (0.0011) [2023-12-26 23:02:21,954][105620] Updated weights for policy 1, policy_version 1053380 (0.0011) [2023-12-26 23:02:22,589][105692] Updated weights for policy 0, policy_version 1052426 (0.0008) [2023-12-26 23:02:22,650][105692] Updated weights for policy 0, policy_version 1052436 (0.0009) [2023-12-26 23:02:22,707][105692] Updated weights for policy 0, policy_version 1052446 (0.0009) [2023-12-26 23:02:22,713][105620] Updated weights for policy 1, policy_version 1053390 (0.0008) [2023-12-26 23:02:22,767][105620] Updated weights for policy 1, policy_version 1053400 (0.0008) [2023-12-26 23:02:22,814][105620] Updated weights for policy 1, policy_version 1053410 (0.0008) [2023-12-26 23:02:23,402][105692] Updated weights for policy 0, policy_version 1052456 (0.0009) [2023-12-26 23:02:23,451][105692] Updated weights for policy 0, policy_version 1052466 (0.0009) [2023-12-26 23:02:23,503][105692] Updated weights for policy 0, policy_version 1052476 (0.0009) [2023-12-26 23:02:23,607][105620] Updated weights for policy 1, policy_version 1053420 (0.0008) [2023-12-26 23:02:23,659][105620] Updated weights for policy 1, policy_version 1053430 (0.0005) [2023-12-26 23:02:23,721][105620] Updated weights for policy 1, policy_version 1053440 (0.0008) [2023-12-26 23:02:24,340][105692] Updated weights for policy 0, policy_version 1052486 (0.0009) [2023-12-26 23:02:24,374][105620] Updated weights for policy 1, policy_version 1053450 (0.0006) [2023-12-26 23:02:24,399][105692] Updated weights for policy 0, policy_version 1052496 (0.0008) [2023-12-26 23:02:24,434][105620] Updated weights for policy 1, policy_version 1053460 (0.0006) [2023-12-26 23:02:24,463][105692] Updated weights for policy 0, policy_version 1052506 (0.0009) [2023-12-26 23:02:24,494][105620] Updated weights for policy 1, policy_version 1053470 (0.0005) [2023-12-26 23:02:24,566][105620] Updated weights for policy 1, policy_version 1053480 (0.0006) [2023-12-26 23:02:25,073][105620] Updated weights for policy 1, policy_version 1053490 (0.0006) [2023-12-26 23:02:25,128][105620] Updated weights for policy 1, policy_version 1053500 (0.0007) [2023-12-26 23:02:25,179][105620] Updated weights for policy 1, policy_version 1053510 (0.0007) [2023-12-26 23:02:25,365][105692] Updated weights for policy 0, policy_version 1052516 (0.0010) [2023-12-26 23:02:25,428][105692] Updated weights for policy 0, policy_version 1052526 (0.0010) [2023-12-26 23:02:25,488][105692] Updated weights for policy 0, policy_version 1052536 (0.0010) [2023-12-26 23:02:25,791][105620] Updated weights for policy 1, policy_version 1053520 (0.0009) [2023-12-26 23:02:25,851][105620] Updated weights for policy 1, policy_version 1053530 (0.0005) [2023-12-26 23:02:25,914][105620] Updated weights for policy 1, policy_version 1053540 (0.0010) [2023-12-26 23:02:26,062][104569] Fps is (10 sec: 18842.3, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 539230208. Throughput: 0: 9835.6, 1: 9740.6. Samples: 539237220. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:02:26,062][104569] Avg episode reward: [(0, '9084.710'), (1, '9351.977')] [2023-12-26 23:02:26,188][105692] Updated weights for policy 0, policy_version 1052546 (0.0010) [2023-12-26 23:02:26,243][105692] Updated weights for policy 0, policy_version 1052556 (0.0010) [2023-12-26 23:02:26,298][105692] Updated weights for policy 0, policy_version 1052566 (0.0010) [2023-12-26 23:02:26,356][105692] Updated weights for policy 0, policy_version 1052576 (0.0010) [2023-12-26 23:02:26,544][105620] Updated weights for policy 1, policy_version 1053550 (0.0007) [2023-12-26 23:02:26,604][105620] Updated weights for policy 1, policy_version 1053560 (0.0005) [2023-12-26 23:02:26,666][105620] Updated weights for policy 1, policy_version 1053570 (0.0005) [2023-12-26 23:02:27,084][105692] Updated weights for policy 0, policy_version 1052586 (0.0010) [2023-12-26 23:02:27,141][105692] Updated weights for policy 0, policy_version 1052596 (0.0010) [2023-12-26 23:02:27,196][105692] Updated weights for policy 0, policy_version 1052606 (0.0010) [2023-12-26 23:02:27,261][105620] Updated weights for policy 1, policy_version 1053580 (0.0007) [2023-12-26 23:02:27,313][105620] Updated weights for policy 1, policy_version 1053590 (0.0010) [2023-12-26 23:02:27,367][105620] Updated weights for policy 1, policy_version 1053600 (0.0008) [2023-12-26 23:02:27,833][105692] Updated weights for policy 0, policy_version 1052616 (0.0006) [2023-12-26 23:02:27,891][105692] Updated weights for policy 0, policy_version 1052626 (0.0006) [2023-12-26 23:02:27,949][105692] Updated weights for policy 0, policy_version 1052636 (0.0005) [2023-12-26 23:02:27,961][105620] Updated weights for policy 1, policy_version 1053610 (0.0006) [2023-12-26 23:02:28,019][105620] Updated weights for policy 1, policy_version 1053620 (0.0009) [2023-12-26 23:02:28,078][105620] Updated weights for policy 1, policy_version 1053631 (0.0009) [2023-12-26 23:02:28,093][105586] KL-divergence is very high: 106.9924 [2023-12-26 23:02:28,561][105692] Updated weights for policy 0, policy_version 1052646 (0.0005) [2023-12-26 23:02:28,615][105692] Updated weights for policy 0, policy_version 1052656 (0.0005) [2023-12-26 23:02:28,686][105692] Updated weights for policy 0, policy_version 1052666 (0.0005) [2023-12-26 23:02:28,757][105586] KL-divergence is very high: 104.6580 [2023-12-26 23:02:28,762][105620] Updated weights for policy 1, policy_version 1053641 (0.0007) [2023-12-26 23:02:28,783][105586] KL-divergence is very high: 154.1117 [2023-12-26 23:02:28,828][105620] Updated weights for policy 1, policy_version 1053651 (0.0005) [2023-12-26 23:02:28,855][105586] KL-divergence is very high: 102.0647 [2023-12-26 23:02:28,861][105586] KL-divergence is very high: 117.7778 [2023-12-26 23:02:28,866][105586] KL-divergence is very high: 159.7165 [2023-12-26 23:02:28,882][105586] KL-divergence is very high: 174.3590 [2023-12-26 23:02:28,887][105586] KL-divergence is very high: 150.2697 [2023-12-26 23:02:28,889][105620] Updated weights for policy 1, policy_version 1053661 (0.0009) [2023-12-26 23:02:28,892][105586] KL-divergence is very high: 149.4108 [2023-12-26 23:02:28,897][105586] KL-divergence is very high: 126.1179 [2023-12-26 23:02:28,902][105586] KL-divergence is very high: 135.3187 [2023-12-26 23:02:28,908][105586] KL-divergence is very high: 154.9858 [2023-12-26 23:02:28,925][105586] KL-divergence is very high: 144.1329 [2023-12-26 23:02:28,930][105586] KL-divergence is very high: 102.7421 [2023-12-26 23:02:28,936][105586] KL-divergence is very high: 100.7882 [2023-12-26 23:02:28,943][105620] Updated weights for policy 1, policy_version 1053671 (0.0010) [2023-12-26 23:02:29,205][105692] Updated weights for policy 0, policy_version 1052676 (0.0007) [2023-12-26 23:02:29,263][105692] Updated weights for policy 0, policy_version 1052686 (0.0010) [2023-12-26 23:02:29,332][105692] Updated weights for policy 0, policy_version 1052696 (0.0007) [2023-12-26 23:02:29,631][105586] KL-divergence is very high: 111.2953 [2023-12-26 23:02:29,679][105620] Updated weights for policy 1, policy_version 1053681 (0.0008) [2023-12-26 23:02:29,728][105620] Updated weights for policy 1, policy_version 1053691 (0.0008) [2023-12-26 23:02:29,773][105620] Updated weights for policy 1, policy_version 1053701 (0.0008) [2023-12-26 23:02:30,027][105692] Updated weights for policy 0, policy_version 1052706 (0.0009) [2023-12-26 23:02:30,087][105692] Updated weights for policy 0, policy_version 1052717 (0.0010) [2023-12-26 23:02:30,145][105692] Updated weights for policy 0, policy_version 1052727 (0.0006) [2023-12-26 23:02:30,531][105620] Updated weights for policy 1, policy_version 1053712 (0.0010) [2023-12-26 23:02:30,590][105620] Updated weights for policy 1, policy_version 1053722 (0.0006) [2023-12-26 23:02:30,649][105620] Updated weights for policy 1, policy_version 1053732 (0.0008) [2023-12-26 23:02:30,844][105692] Updated weights for policy 0, policy_version 1052737 (0.0005) [2023-12-26 23:02:30,905][105692] Updated weights for policy 0, policy_version 1052747 (0.0010) [2023-12-26 23:02:30,971][105692] Updated weights for policy 0, policy_version 1052757 (0.0005) [2023-12-26 23:02:31,038][105692] Updated weights for policy 0, policy_version 1052767 (0.0009) [2023-12-26 23:02:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 539336704. Throughput: 0: 9863.2, 1: 9841.3. Samples: 539301476. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:02:31,062][104569] Avg episode reward: [(0, '8991.831'), (1, '6511.187')] [2023-12-26 23:02:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001052768_269549568.pth... [2023-12-26 23:02:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001053736_269787136.pth... [2023-12-26 23:02:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001051648_269262848.pth [2023-12-26 23:02:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001052584_269492224.pth [2023-12-26 23:02:31,336][105620] Updated weights for policy 1, policy_version 1053742 (0.0009) [2023-12-26 23:02:31,406][105620] Updated weights for policy 1, policy_version 1053752 (0.0009) [2023-12-26 23:02:31,459][105620] Updated weights for policy 1, policy_version 1053762 (0.0010) [2023-12-26 23:02:31,603][105692] Updated weights for policy 0, policy_version 1052777 (0.0007) [2023-12-26 23:02:31,667][105692] Updated weights for policy 0, policy_version 1052787 (0.0011) [2023-12-26 23:02:31,734][105692] Updated weights for policy 0, policy_version 1052797 (0.0010) [2023-12-26 23:02:32,200][105620] Updated weights for policy 1, policy_version 1053772 (0.0009) [2023-12-26 23:02:32,247][105620] Updated weights for policy 1, policy_version 1053782 (0.0007) [2023-12-26 23:02:32,310][105620] Updated weights for policy 1, policy_version 1053792 (0.0008) [2023-12-26 23:02:32,423][105692] Updated weights for policy 0, policy_version 1052807 (0.0010) [2023-12-26 23:02:32,470][105692] Updated weights for policy 0, policy_version 1052817 (0.0010) [2023-12-26 23:02:32,539][105692] Updated weights for policy 0, policy_version 1052827 (0.0005) [2023-12-26 23:02:33,028][105620] Updated weights for policy 1, policy_version 1053802 (0.0007) [2023-12-26 23:02:33,084][105620] Updated weights for policy 1, policy_version 1053812 (0.0005) [2023-12-26 23:02:33,140][105620] Updated weights for policy 1, policy_version 1053822 (0.0005) [2023-12-26 23:02:33,160][105692] Updated weights for policy 0, policy_version 1052837 (0.0006) [2023-12-26 23:02:33,208][105620] Updated weights for policy 1, policy_version 1053832 (0.0008) [2023-12-26 23:02:33,220][105692] Updated weights for policy 0, policy_version 1052847 (0.0005) [2023-12-26 23:02:33,265][105692] Updated weights for policy 0, policy_version 1052857 (0.0005) [2023-12-26 23:02:33,788][105692] Updated weights for policy 0, policy_version 1052867 (0.0005) [2023-12-26 23:02:33,850][105692] Updated weights for policy 0, policy_version 1052877 (0.0005) [2023-12-26 23:02:33,884][105620] Updated weights for policy 1, policy_version 1053842 (0.0006) [2023-12-26 23:02:33,895][105692] Updated weights for policy 0, policy_version 1052887 (0.0005) [2023-12-26 23:02:33,942][105620] Updated weights for policy 1, policy_version 1053852 (0.0006) [2023-12-26 23:02:34,015][105620] Updated weights for policy 1, policy_version 1053862 (0.0006) [2023-12-26 23:02:34,489][105692] Updated weights for policy 0, policy_version 1052897 (0.0005) [2023-12-26 23:02:34,548][105692] Updated weights for policy 0, policy_version 1052907 (0.0005) [2023-12-26 23:02:34,615][105692] Updated weights for policy 0, policy_version 1052917 (0.0008) [2023-12-26 23:02:34,666][105692] Updated weights for policy 0, policy_version 1052927 (0.0007) [2023-12-26 23:02:34,667][105620] Updated weights for policy 1, policy_version 1053872 (0.0009) [2023-12-26 23:02:34,724][105620] Updated weights for policy 1, policy_version 1053882 (0.0009) [2023-12-26 23:02:34,783][105620] Updated weights for policy 1, policy_version 1053892 (0.0010) [2023-12-26 23:02:35,361][105692] Updated weights for policy 0, policy_version 1052937 (0.0007) [2023-12-26 23:02:35,396][105620] Updated weights for policy 1, policy_version 1053902 (0.0008) [2023-12-26 23:02:35,415][105692] Updated weights for policy 0, policy_version 1052947 (0.0008) [2023-12-26 23:02:35,459][105620] Updated weights for policy 1, policy_version 1053912 (0.0007) [2023-12-26 23:02:35,465][105692] Updated weights for policy 0, policy_version 1052957 (0.0009) [2023-12-26 23:02:35,521][105620] Updated weights for policy 1, policy_version 1053922 (0.0008) [2023-12-26 23:02:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 539435008. Throughput: 0: 9904.0, 1: 9858.3. Samples: 539425408. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:02:36,063][104569] Avg episode reward: [(0, '8810.779'), (1, '6807.263')] [2023-12-26 23:02:36,070][105692] Updated weights for policy 0, policy_version 1052967 (0.0008) [2023-12-26 23:02:36,128][105692] Updated weights for policy 0, policy_version 1052977 (0.0007) [2023-12-26 23:02:36,189][105692] Updated weights for policy 0, policy_version 1052987 (0.0009) [2023-12-26 23:02:36,199][105620] Updated weights for policy 1, policy_version 1053932 (0.0009) [2023-12-26 23:02:36,251][105620] Updated weights for policy 1, policy_version 1053942 (0.0007) [2023-12-26 23:02:36,303][105620] Updated weights for policy 1, policy_version 1053952 (0.0009) [2023-12-26 23:02:36,967][105692] Updated weights for policy 0, policy_version 1052997 (0.0009) [2023-12-26 23:02:37,019][105692] Updated weights for policy 0, policy_version 1053007 (0.0009) [2023-12-26 23:02:37,040][105620] Updated weights for policy 1, policy_version 1053962 (0.0009) [2023-12-26 23:02:37,079][105692] Updated weights for policy 0, policy_version 1053017 (0.0007) [2023-12-26 23:02:37,106][105620] Updated weights for policy 1, policy_version 1053972 (0.0008) [2023-12-26 23:02:37,167][105620] Updated weights for policy 1, policy_version 1053982 (0.0009) [2023-12-26 23:02:37,230][105620] Updated weights for policy 1, policy_version 1053992 (0.0009) [2023-12-26 23:02:37,861][105692] Updated weights for policy 0, policy_version 1053027 (0.0007) [2023-12-26 23:02:37,918][105692] Updated weights for policy 0, policy_version 1053037 (0.0010) [2023-12-26 23:02:37,973][105692] Updated weights for policy 0, policy_version 1053047 (0.0011) [2023-12-26 23:02:37,990][105620] Updated weights for policy 1, policy_version 1054002 (0.0011) [2023-12-26 23:02:38,039][105620] Updated weights for policy 1, policy_version 1054012 (0.0010) [2023-12-26 23:02:38,094][105620] Updated weights for policy 1, policy_version 1054022 (0.0010) [2023-12-26 23:02:38,622][105692] Updated weights for policy 0, policy_version 1053057 (0.0007) [2023-12-26 23:02:38,674][105692] Updated weights for policy 0, policy_version 1053067 (0.0008) [2023-12-26 23:02:38,723][105692] Updated weights for policy 0, policy_version 1053077 (0.0008) [2023-12-26 23:02:38,767][105692] Updated weights for policy 0, policy_version 1053087 (0.0008) [2023-12-26 23:02:38,870][105620] Updated weights for policy 1, policy_version 1054032 (0.0011) [2023-12-26 23:02:38,926][105620] Updated weights for policy 1, policy_version 1054042 (0.0010) [2023-12-26 23:02:38,985][105620] Updated weights for policy 1, policy_version 1054052 (0.0010) [2023-12-26 23:02:39,459][105692] Updated weights for policy 0, policy_version 1053097 (0.0008) [2023-12-26 23:02:39,532][105692] Updated weights for policy 0, policy_version 1053107 (0.0009) [2023-12-26 23:02:39,596][105692] Updated weights for policy 0, policy_version 1053117 (0.0009) [2023-12-26 23:02:39,680][105620] Updated weights for policy 1, policy_version 1054062 (0.0010) [2023-12-26 23:02:39,726][105620] Updated weights for policy 1, policy_version 1054072 (0.0010) [2023-12-26 23:02:39,775][105620] Updated weights for policy 1, policy_version 1054082 (0.0010) [2023-12-26 23:02:40,321][105692] Updated weights for policy 0, policy_version 1053127 (0.0009) [2023-12-26 23:02:40,373][105692] Updated weights for policy 0, policy_version 1053137 (0.0009) [2023-12-26 23:02:40,434][105692] Updated weights for policy 0, policy_version 1053147 (0.0010) [2023-12-26 23:02:40,475][105620] Updated weights for policy 1, policy_version 1054092 (0.0008) [2023-12-26 23:02:40,532][105620] Updated weights for policy 1, policy_version 1054102 (0.0008) [2023-12-26 23:02:40,597][105620] Updated weights for policy 1, policy_version 1054112 (0.0009) [2023-12-26 23:02:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 539533312. Throughput: 0: 9885.1, 1: 9868.3. Samples: 539542876. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:02:41,063][104569] Avg episode reward: [(0, '9174.240'), (1, '8781.659')] [2023-12-26 23:02:41,288][105692] Updated weights for policy 0, policy_version 1053157 (0.0010) [2023-12-26 23:02:41,321][105620] Updated weights for policy 1, policy_version 1054122 (0.0009) [2023-12-26 23:02:41,349][105692] Updated weights for policy 0, policy_version 1053167 (0.0009) [2023-12-26 23:02:41,386][105620] Updated weights for policy 1, policy_version 1054132 (0.0009) [2023-12-26 23:02:41,416][105692] Updated weights for policy 0, policy_version 1053177 (0.0009) [2023-12-26 23:02:41,444][105620] Updated weights for policy 1, policy_version 1054142 (0.0007) [2023-12-26 23:02:41,499][105620] Updated weights for policy 1, policy_version 1054152 (0.0008) [2023-12-26 23:02:42,090][105692] Updated weights for policy 0, policy_version 1053187 (0.0007) [2023-12-26 23:02:42,154][105692] Updated weights for policy 0, policy_version 1053197 (0.0007) [2023-12-26 23:02:42,217][105692] Updated weights for policy 0, policy_version 1053207 (0.0009) [2023-12-26 23:02:42,321][105620] Updated weights for policy 1, policy_version 1054162 (0.0008) [2023-12-26 23:02:42,383][105620] Updated weights for policy 1, policy_version 1054172 (0.0010) [2023-12-26 23:02:42,442][105620] Updated weights for policy 1, policy_version 1054182 (0.0009) [2023-12-26 23:02:42,948][105692] Updated weights for policy 0, policy_version 1053217 (0.0008) [2023-12-26 23:02:42,996][105692] Updated weights for policy 0, policy_version 1053227 (0.0008) [2023-12-26 23:02:43,049][105692] Updated weights for policy 0, policy_version 1053237 (0.0009) [2023-12-26 23:02:43,106][105692] Updated weights for policy 0, policy_version 1053248 (0.0010) [2023-12-26 23:02:43,151][105620] Updated weights for policy 1, policy_version 1054192 (0.0010) [2023-12-26 23:02:43,206][105620] Updated weights for policy 1, policy_version 1054202 (0.0006) [2023-12-26 23:02:43,252][105620] Updated weights for policy 1, policy_version 1054212 (0.0005) [2023-12-26 23:02:43,910][105620] Updated weights for policy 1, policy_version 1054222 (0.0006) [2023-12-26 23:02:43,939][105692] Updated weights for policy 0, policy_version 1053258 (0.0007) [2023-12-26 23:02:43,959][105620] Updated weights for policy 1, policy_version 1054232 (0.0005) [2023-12-26 23:02:43,987][105692] Updated weights for policy 0, policy_version 1053268 (0.0009) [2023-12-26 23:02:44,015][105620] Updated weights for policy 1, policy_version 1054242 (0.0005) [2023-12-26 23:02:44,045][105692] Updated weights for policy 0, policy_version 1053278 (0.0008) [2023-12-26 23:02:44,588][105620] Updated weights for policy 1, policy_version 1054252 (0.0007) [2023-12-26 23:02:44,649][105620] Updated weights for policy 1, policy_version 1054262 (0.0008) [2023-12-26 23:02:44,710][105620] Updated weights for policy 1, policy_version 1054272 (0.0009) [2023-12-26 23:02:44,889][105692] Updated weights for policy 0, policy_version 1053288 (0.0009) [2023-12-26 23:02:44,946][105692] Updated weights for policy 0, policy_version 1053298 (0.0009) [2023-12-26 23:02:45,005][105692] Updated weights for policy 0, policy_version 1053308 (0.0009) [2023-12-26 23:02:45,406][105620] Updated weights for policy 1, policy_version 1054282 (0.0009) [2023-12-26 23:02:45,457][105620] Updated weights for policy 1, policy_version 1054292 (0.0009) [2023-12-26 23:02:45,508][105620] Updated weights for policy 1, policy_version 1054302 (0.0009) [2023-12-26 23:02:45,554][105620] Updated weights for policy 1, policy_version 1054312 (0.0008) [2023-12-26 23:02:45,799][105692] Updated weights for policy 0, policy_version 1053318 (0.0008) [2023-12-26 23:02:45,848][105692] Updated weights for policy 0, policy_version 1053328 (0.0009) [2023-12-26 23:02:45,917][105692] Updated weights for policy 0, policy_version 1053338 (0.0009) [2023-12-26 23:02:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19797.3, 300 sec: 19605.2). Total num frames: 539631616. Throughput: 0: 9755.7, 1: 9825.3. Samples: 539599572. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:02:46,063][104569] Avg episode reward: [(0, '9264.338'), (1, '9272.803')] [2023-12-26 23:02:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001053344_269697024.pth... [2023-12-26 23:02:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001054312_269934592.pth... [2023-12-26 23:02:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001053160_269639680.pth [2023-12-26 23:02:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001052192_269402112.pth [2023-12-26 23:02:46,311][105620] Updated weights for policy 1, policy_version 1054322 (0.0006) [2023-12-26 23:02:46,357][105620] Updated weights for policy 1, policy_version 1054332 (0.0008) [2023-12-26 23:02:46,404][105620] Updated weights for policy 1, policy_version 1054342 (0.0008) [2023-12-26 23:02:46,670][105692] Updated weights for policy 0, policy_version 1053348 (0.0009) [2023-12-26 23:02:46,732][105692] Updated weights for policy 0, policy_version 1053358 (0.0009) [2023-12-26 23:02:46,786][105692] Updated weights for policy 0, policy_version 1053368 (0.0009) [2023-12-26 23:02:47,163][105620] Updated weights for policy 1, policy_version 1054352 (0.0009) [2023-12-26 23:02:47,228][105620] Updated weights for policy 1, policy_version 1054362 (0.0009) [2023-12-26 23:02:47,278][105620] Updated weights for policy 1, policy_version 1054372 (0.0008) [2023-12-26 23:02:47,545][105692] Updated weights for policy 0, policy_version 1053378 (0.0009) [2023-12-26 23:02:47,603][105692] Updated weights for policy 0, policy_version 1053388 (0.0005) [2023-12-26 23:02:47,660][105692] Updated weights for policy 0, policy_version 1053398 (0.0005) [2023-12-26 23:02:47,711][105692] Updated weights for policy 0, policy_version 1053408 (0.0010) [2023-12-26 23:02:47,899][105620] Updated weights for policy 1, policy_version 1054382 (0.0009) [2023-12-26 23:02:47,947][105620] Updated weights for policy 1, policy_version 1054392 (0.0010) [2023-12-26 23:02:47,992][105620] Updated weights for policy 1, policy_version 1054402 (0.0010) [2023-12-26 23:02:48,304][105692] Updated weights for policy 0, policy_version 1053418 (0.0011) [2023-12-26 23:02:48,370][105692] Updated weights for policy 0, policy_version 1053428 (0.0008) [2023-12-26 23:02:48,433][105692] Updated weights for policy 0, policy_version 1053438 (0.0011) [2023-12-26 23:02:48,740][105620] Updated weights for policy 1, policy_version 1054412 (0.0009) [2023-12-26 23:02:48,803][105620] Updated weights for policy 1, policy_version 1054422 (0.0008) [2023-12-26 23:02:48,860][105620] Updated weights for policy 1, policy_version 1054432 (0.0008) [2023-12-26 23:02:49,177][105692] Updated weights for policy 0, policy_version 1053448 (0.0010) [2023-12-26 23:02:49,237][105692] Updated weights for policy 0, policy_version 1053458 (0.0009) [2023-12-26 23:02:49,296][105692] Updated weights for policy 0, policy_version 1053468 (0.0009) [2023-12-26 23:02:49,602][105620] Updated weights for policy 1, policy_version 1054442 (0.0008) [2023-12-26 23:02:49,665][105620] Updated weights for policy 1, policy_version 1054452 (0.0009) [2023-12-26 23:02:49,719][105620] Updated weights for policy 1, policy_version 1054462 (0.0009) [2023-12-26 23:02:49,774][105620] Updated weights for policy 1, policy_version 1054472 (0.0009) [2023-12-26 23:02:50,098][105692] Updated weights for policy 0, policy_version 1053478 (0.0008) [2023-12-26 23:02:50,163][105692] Updated weights for policy 0, policy_version 1053488 (0.0008) [2023-12-26 23:02:50,225][105692] Updated weights for policy 0, policy_version 1053498 (0.0008) [2023-12-26 23:02:50,488][105620] Updated weights for policy 1, policy_version 1054482 (0.0011) [2023-12-26 23:02:50,537][105620] Updated weights for policy 1, policy_version 1054492 (0.0010) [2023-12-26 23:02:50,595][105620] Updated weights for policy 1, policy_version 1054502 (0.0011) [2023-12-26 23:02:51,002][105692] Updated weights for policy 0, policy_version 1053508 (0.0008) [2023-12-26 23:02:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 539721728. Throughput: 0: 9706.8, 1: 9840.7. Samples: 539714956. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:02:51,062][104569] Avg episode reward: [(0, '9266.615'), (1, '9271.093')] [2023-12-26 23:02:51,069][105692] Updated weights for policy 0, policy_version 1053518 (0.0008) [2023-12-26 23:02:51,126][105692] Updated weights for policy 0, policy_version 1053528 (0.0008) [2023-12-26 23:02:51,376][105620] Updated weights for policy 1, policy_version 1054512 (0.0011) [2023-12-26 23:02:51,435][105620] Updated weights for policy 1, policy_version 1054522 (0.0010) [2023-12-26 23:02:51,497][105620] Updated weights for policy 1, policy_version 1054532 (0.0009) [2023-12-26 23:02:51,835][105692] Updated weights for policy 0, policy_version 1053538 (0.0009) [2023-12-26 23:02:51,883][105692] Updated weights for policy 0, policy_version 1053548 (0.0010) [2023-12-26 23:02:51,949][105692] Updated weights for policy 0, policy_version 1053558 (0.0011) [2023-12-26 23:02:52,011][105692] Updated weights for policy 0, policy_version 1053568 (0.0010) [2023-12-26 23:02:52,233][105620] Updated weights for policy 1, policy_version 1054542 (0.0006) [2023-12-26 23:02:52,295][105620] Updated weights for policy 1, policy_version 1054552 (0.0006) [2023-12-26 23:02:52,355][105620] Updated weights for policy 1, policy_version 1054562 (0.0007) [2023-12-26 23:02:52,764][105692] Updated weights for policy 0, policy_version 1053578 (0.0010) [2023-12-26 23:02:52,826][105692] Updated weights for policy 0, policy_version 1053588 (0.0007) [2023-12-26 23:02:52,883][105692] Updated weights for policy 0, policy_version 1053598 (0.0005) [2023-12-26 23:02:52,934][105620] Updated weights for policy 1, policy_version 1054572 (0.0006) [2023-12-26 23:02:52,990][105620] Updated weights for policy 1, policy_version 1054582 (0.0006) [2023-12-26 23:02:53,053][105620] Updated weights for policy 1, policy_version 1054592 (0.0008) [2023-12-26 23:02:53,471][105692] Updated weights for policy 0, policy_version 1053608 (0.0005) [2023-12-26 23:02:53,528][105692] Updated weights for policy 0, policy_version 1053618 (0.0007) [2023-12-26 23:02:53,576][105692] Updated weights for policy 0, policy_version 1053628 (0.0010) [2023-12-26 23:02:53,632][105620] Updated weights for policy 1, policy_version 1054602 (0.0006) [2023-12-26 23:02:53,680][105620] Updated weights for policy 1, policy_version 1054612 (0.0010) [2023-12-26 23:02:53,734][105620] Updated weights for policy 1, policy_version 1054622 (0.0011) [2023-12-26 23:02:53,782][105620] Updated weights for policy 1, policy_version 1054632 (0.0009) [2023-12-26 23:02:54,306][105692] Updated weights for policy 0, policy_version 1053638 (0.0007) [2023-12-26 23:02:54,372][105692] Updated weights for policy 0, policy_version 1053648 (0.0006) [2023-12-26 23:02:54,439][105692] Updated weights for policy 0, policy_version 1053658 (0.0005) [2023-12-26 23:02:54,493][105620] Updated weights for policy 1, policy_version 1054642 (0.0011) [2023-12-26 23:02:54,555][105620] Updated weights for policy 1, policy_version 1054652 (0.0010) [2023-12-26 23:02:54,615][105620] Updated weights for policy 1, policy_version 1054662 (0.0010) [2023-12-26 23:02:54,939][105692] Updated weights for policy 0, policy_version 1053668 (0.0006) [2023-12-26 23:02:55,001][105692] Updated weights for policy 0, policy_version 1053678 (0.0010) [2023-12-26 23:02:55,062][105692] Updated weights for policy 0, policy_version 1053688 (0.0009) [2023-12-26 23:02:55,327][105620] Updated weights for policy 1, policy_version 1054672 (0.0010) [2023-12-26 23:02:55,378][105620] Updated weights for policy 1, policy_version 1054682 (0.0010) [2023-12-26 23:02:55,437][105620] Updated weights for policy 1, policy_version 1054692 (0.0010) [2023-12-26 23:02:55,626][105692] Updated weights for policy 0, policy_version 1053698 (0.0006) [2023-12-26 23:02:55,680][105692] Updated weights for policy 0, policy_version 1053708 (0.0010) [2023-12-26 23:02:55,739][105692] Updated weights for policy 0, policy_version 1053718 (0.0010) [2023-12-26 23:02:55,797][105692] Updated weights for policy 0, policy_version 1053728 (0.0010) [2023-12-26 23:02:56,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 539828224. Throughput: 0: 9741.9, 1: 9923.5. Samples: 539837600. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:02:56,062][104569] Avg episode reward: [(0, '9264.903'), (1, '9348.324')] [2023-12-26 23:02:56,086][105620] Updated weights for policy 1, policy_version 1054702 (0.0007) [2023-12-26 23:02:56,140][105620] Updated weights for policy 1, policy_version 1054712 (0.0010) [2023-12-26 23:02:56,203][105620] Updated weights for policy 1, policy_version 1054722 (0.0011) [2023-12-26 23:02:56,393][105692] Updated weights for policy 0, policy_version 1053738 (0.0005) [2023-12-26 23:02:56,459][105692] Updated weights for policy 0, policy_version 1053748 (0.0008) [2023-12-26 23:02:56,516][105692] Updated weights for policy 0, policy_version 1053758 (0.0010) [2023-12-26 23:02:56,875][105620] Updated weights for policy 1, policy_version 1054732 (0.0011) [2023-12-26 23:02:56,918][105620] Updated weights for policy 1, policy_version 1054742 (0.0010) [2023-12-26 23:02:56,969][105620] Updated weights for policy 1, policy_version 1054752 (0.0010) [2023-12-26 23:02:57,205][105692] Updated weights for policy 0, policy_version 1053768 (0.0010) [2023-12-26 23:02:57,260][105692] Updated weights for policy 0, policy_version 1053778 (0.0009) [2023-12-26 23:02:57,318][105692] Updated weights for policy 0, policy_version 1053788 (0.0008) [2023-12-26 23:02:57,732][105620] Updated weights for policy 1, policy_version 1054762 (0.0010) [2023-12-26 23:02:57,780][105620] Updated weights for policy 1, policy_version 1054772 (0.0010) [2023-12-26 23:02:57,823][105620] Updated weights for policy 1, policy_version 1054782 (0.0010) [2023-12-26 23:02:57,864][105620] Updated weights for policy 1, policy_version 1054792 (0.0010) [2023-12-26 23:02:58,022][105692] Updated weights for policy 0, policy_version 1053798 (0.0007) [2023-12-26 23:02:58,067][105692] Updated weights for policy 0, policy_version 1053808 (0.0005) [2023-12-26 23:02:58,131][105692] Updated weights for policy 0, policy_version 1053818 (0.0007) [2023-12-26 23:02:58,658][105620] Updated weights for policy 1, policy_version 1054802 (0.0008) [2023-12-26 23:02:58,722][105620] Updated weights for policy 1, policy_version 1054812 (0.0007) [2023-12-26 23:02:58,804][105620] Updated weights for policy 1, policy_version 1054822 (0.0008) [2023-12-26 23:02:58,841][105692] Updated weights for policy 0, policy_version 1053828 (0.0010) [2023-12-26 23:02:58,907][105692] Updated weights for policy 0, policy_version 1053838 (0.0008) [2023-12-26 23:02:58,982][105692] Updated weights for policy 0, policy_version 1053848 (0.0008) [2023-12-26 23:02:59,597][105620] Updated weights for policy 1, policy_version 1054832 (0.0006) [2023-12-26 23:02:59,669][105620] Updated weights for policy 1, policy_version 1054842 (0.0006) [2023-12-26 23:02:59,721][105620] Updated weights for policy 1, policy_version 1054852 (0.0010) [2023-12-26 23:02:59,754][105692] Updated weights for policy 0, policy_version 1053858 (0.0007) [2023-12-26 23:02:59,816][105692] Updated weights for policy 0, policy_version 1053868 (0.0008) [2023-12-26 23:02:59,880][105692] Updated weights for policy 0, policy_version 1053878 (0.0009) [2023-12-26 23:02:59,946][105692] Updated weights for policy 0, policy_version 1053888 (0.0006) [2023-12-26 23:03:00,388][105620] Updated weights for policy 1, policy_version 1054862 (0.0008) [2023-12-26 23:03:00,446][105620] Updated weights for policy 1, policy_version 1054872 (0.0010) [2023-12-26 23:03:00,506][105692] Updated weights for policy 0, policy_version 1053898 (0.0007) [2023-12-26 23:03:00,508][105620] Updated weights for policy 1, policy_version 1054882 (0.0010) [2023-12-26 23:03:00,566][105692] Updated weights for policy 0, policy_version 1053908 (0.0006) [2023-12-26 23:03:00,628][105692] Updated weights for policy 0, policy_version 1053918 (0.0008) [2023-12-26 23:03:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 539926528. Throughput: 0: 9785.1, 1: 9916.7. Samples: 539896920. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:03:01,062][104569] Avg episode reward: [(0, '9085.287'), (1, '9256.550')] [2023-12-26 23:03:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001053920_269844480.pth... [2023-12-26 23:03:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001054888_270082048.pth... [2023-12-26 23:03:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001052768_269549568.pth [2023-12-26 23:03:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001053736_269787136.pth [2023-12-26 23:03:01,112][105620] Updated weights for policy 1, policy_version 1054892 (0.0008) [2023-12-26 23:03:01,182][105620] Updated weights for policy 1, policy_version 1054902 (0.0007) [2023-12-26 23:03:01,236][105620] Updated weights for policy 1, policy_version 1054912 (0.0008) [2023-12-26 23:03:01,271][105692] Updated weights for policy 0, policy_version 1053928 (0.0008) [2023-12-26 23:03:01,327][105692] Updated weights for policy 0, policy_version 1053938 (0.0008) [2023-12-26 23:03:01,396][105692] Updated weights for policy 0, policy_version 1053948 (0.0008) [2023-12-26 23:03:02,006][105620] Updated weights for policy 1, policy_version 1054922 (0.0008) [2023-12-26 23:03:02,064][105620] Updated weights for policy 1, policy_version 1054932 (0.0009) [2023-12-26 23:03:02,121][105620] Updated weights for policy 1, policy_version 1054942 (0.0008) [2023-12-26 23:03:02,127][105692] Updated weights for policy 0, policy_version 1053958 (0.0008) [2023-12-26 23:03:02,179][105620] Updated weights for policy 1, policy_version 1054952 (0.0007) [2023-12-26 23:03:02,181][105692] Updated weights for policy 0, policy_version 1053968 (0.0006) [2023-12-26 23:03:02,243][105692] Updated weights for policy 0, policy_version 1053978 (0.0010) [2023-12-26 23:03:02,770][105620] Updated weights for policy 1, policy_version 1054962 (0.0005) [2023-12-26 23:03:02,818][105620] Updated weights for policy 1, policy_version 1054972 (0.0005) [2023-12-26 23:03:02,877][105620] Updated weights for policy 1, policy_version 1054982 (0.0005) [2023-12-26 23:03:03,134][105692] Updated weights for policy 0, policy_version 1053988 (0.0008) [2023-12-26 23:03:03,206][105692] Updated weights for policy 0, policy_version 1053998 (0.0008) [2023-12-26 23:03:03,253][105692] Updated weights for policy 0, policy_version 1054008 (0.0008) [2023-12-26 23:03:03,400][105620] Updated weights for policy 1, policy_version 1054992 (0.0008) [2023-12-26 23:03:03,449][105620] Updated weights for policy 1, policy_version 1055002 (0.0008) [2023-12-26 23:03:03,495][105620] Updated weights for policy 1, policy_version 1055012 (0.0009) [2023-12-26 23:03:04,041][105692] Updated weights for policy 0, policy_version 1054018 (0.0009) [2023-12-26 23:03:04,087][105692] Updated weights for policy 0, policy_version 1054028 (0.0008) [2023-12-26 23:03:04,131][105620] Updated weights for policy 1, policy_version 1055022 (0.0006) [2023-12-26 23:03:04,142][105692] Updated weights for policy 0, policy_version 1054038 (0.0007) [2023-12-26 23:03:04,193][105620] Updated weights for policy 1, policy_version 1055032 (0.0008) [2023-12-26 23:03:04,197][105692] Updated weights for policy 0, policy_version 1054048 (0.0006) [2023-12-26 23:03:04,253][105620] Updated weights for policy 1, policy_version 1055042 (0.0009) [2023-12-26 23:03:04,905][105620] Updated weights for policy 1, policy_version 1055052 (0.0009) [2023-12-26 23:03:04,961][105620] Updated weights for policy 1, policy_version 1055062 (0.0009) [2023-12-26 23:03:05,018][105620] Updated weights for policy 1, policy_version 1055072 (0.0007) [2023-12-26 23:03:05,036][105692] Updated weights for policy 0, policy_version 1054058 (0.0007) [2023-12-26 23:03:05,094][105692] Updated weights for policy 0, policy_version 1054068 (0.0007) [2023-12-26 23:03:05,151][105692] Updated weights for policy 0, policy_version 1054078 (0.0009) [2023-12-26 23:03:05,658][105620] Updated weights for policy 1, policy_version 1055082 (0.0007) [2023-12-26 23:03:05,714][105620] Updated weights for policy 1, policy_version 1055092 (0.0008) [2023-12-26 23:03:05,768][105620] Updated weights for policy 1, policy_version 1055102 (0.0009) [2023-12-26 23:03:05,814][105620] Updated weights for policy 1, policy_version 1055112 (0.0008) [2023-12-26 23:03:05,932][105692] Updated weights for policy 0, policy_version 1054088 (0.0009) [2023-12-26 23:03:05,984][105692] Updated weights for policy 0, policy_version 1054098 (0.0009) [2023-12-26 23:03:06,032][105692] Updated weights for policy 0, policy_version 1054108 (0.0009) [2023-12-26 23:03:06,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 540033024. Throughput: 0: 9780.8, 1: 10041.9. Samples: 540017004. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:03:06,062][104569] Avg episode reward: [(0, '9174.516'), (1, '9257.919')] [2023-12-26 23:03:06,609][105620] Updated weights for policy 1, policy_version 1055122 (0.0008) [2023-12-26 23:03:06,665][105620] Updated weights for policy 1, policy_version 1055132 (0.0008) [2023-12-26 23:03:06,727][105620] Updated weights for policy 1, policy_version 1055142 (0.0008) [2023-12-26 23:03:06,815][105692] Updated weights for policy 0, policy_version 1054119 (0.0010) [2023-12-26 23:03:06,872][105692] Updated weights for policy 0, policy_version 1054129 (0.0010) [2023-12-26 23:03:06,936][105692] Updated weights for policy 0, policy_version 1054139 (0.0008) [2023-12-26 23:03:07,459][105620] Updated weights for policy 1, policy_version 1055152 (0.0009) [2023-12-26 23:03:07,520][105620] Updated weights for policy 1, policy_version 1055162 (0.0009) [2023-12-26 23:03:07,577][105620] Updated weights for policy 1, policy_version 1055172 (0.0007) [2023-12-26 23:03:07,658][105692] Updated weights for policy 0, policy_version 1054149 (0.0009) [2023-12-26 23:03:07,720][105692] Updated weights for policy 0, policy_version 1054159 (0.0009) [2023-12-26 23:03:07,787][105692] Updated weights for policy 0, policy_version 1054169 (0.0009) [2023-12-26 23:03:08,358][105620] Updated weights for policy 1, policy_version 1055182 (0.0009) [2023-12-26 23:03:08,412][105620] Updated weights for policy 1, policy_version 1055192 (0.0009) [2023-12-26 23:03:08,462][105620] Updated weights for policy 1, policy_version 1055202 (0.0009) [2023-12-26 23:03:08,472][105692] Updated weights for policy 0, policy_version 1054179 (0.0009) [2023-12-26 23:03:08,532][105692] Updated weights for policy 0, policy_version 1054189 (0.0007) [2023-12-26 23:03:08,590][105692] Updated weights for policy 0, policy_version 1054199 (0.0009) [2023-12-26 23:03:09,212][105692] Updated weights for policy 0, policy_version 1054209 (0.0009) [2023-12-26 23:03:09,277][105692] Updated weights for policy 0, policy_version 1054219 (0.0008) [2023-12-26 23:03:09,340][105692] Updated weights for policy 0, policy_version 1054229 (0.0009) [2023-12-26 23:03:09,349][105620] Updated weights for policy 1, policy_version 1055212 (0.0008) [2023-12-26 23:03:09,406][105692] Updated weights for policy 0, policy_version 1054239 (0.0009) [2023-12-26 23:03:09,419][105620] Updated weights for policy 1, policy_version 1055222 (0.0007) [2023-12-26 23:03:09,479][105620] Updated weights for policy 1, policy_version 1055232 (0.0006) [2023-12-26 23:03:10,124][105692] Updated weights for policy 0, policy_version 1054249 (0.0009) [2023-12-26 23:03:10,196][105692] Updated weights for policy 0, policy_version 1054259 (0.0010) [2023-12-26 23:03:10,225][105620] Updated weights for policy 1, policy_version 1055242 (0.0007) [2023-12-26 23:03:10,256][105692] Updated weights for policy 0, policy_version 1054269 (0.0009) [2023-12-26 23:03:10,290][105620] Updated weights for policy 1, policy_version 1055252 (0.0008) [2023-12-26 23:03:10,348][105620] Updated weights for policy 1, policy_version 1055262 (0.0007) [2023-12-26 23:03:10,400][105620] Updated weights for policy 1, policy_version 1055272 (0.0005) [2023-12-26 23:03:10,990][105692] Updated weights for policy 0, policy_version 1054279 (0.0008) [2023-12-26 23:03:11,050][105620] Updated weights for policy 1, policy_version 1055282 (0.0008) [2023-12-26 23:03:11,056][105692] Updated weights for policy 0, policy_version 1054289 (0.0008) [2023-12-26 23:03:11,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 540114944. Throughput: 0: 9870.8, 1: 9961.4. Samples: 540129672. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:03:11,063][104569] Avg episode reward: [(0, '8904.197'), (1, '9259.652')] [2023-12-26 23:03:11,116][105692] Updated weights for policy 0, policy_version 1054299 (0.0009) [2023-12-26 23:03:11,122][105620] Updated weights for policy 1, policy_version 1055292 (0.0006) [2023-12-26 23:03:11,188][105620] Updated weights for policy 1, policy_version 1055302 (0.0009) [2023-12-26 23:03:11,892][105692] Updated weights for policy 0, policy_version 1054309 (0.0009) [2023-12-26 23:03:11,951][105692] Updated weights for policy 0, policy_version 1054319 (0.0007) [2023-12-26 23:03:11,977][105620] Updated weights for policy 1, policy_version 1055312 (0.0010) [2023-12-26 23:03:12,012][105692] Updated weights for policy 0, policy_version 1054329 (0.0009) [2023-12-26 23:03:12,035][105620] Updated weights for policy 1, policy_version 1055322 (0.0007) [2023-12-26 23:03:12,099][105620] Updated weights for policy 1, policy_version 1055332 (0.0008) [2023-12-26 23:03:12,796][105620] Updated weights for policy 1, policy_version 1055342 (0.0008) [2023-12-26 23:03:12,806][105692] Updated weights for policy 0, policy_version 1054339 (0.0008) [2023-12-26 23:03:12,855][105620] Updated weights for policy 1, policy_version 1055352 (0.0009) [2023-12-26 23:03:12,862][105692] Updated weights for policy 0, policy_version 1054349 (0.0007) [2023-12-26 23:03:12,912][105620] Updated weights for policy 1, policy_version 1055362 (0.0005) [2023-12-26 23:03:12,926][105692] Updated weights for policy 0, policy_version 1054359 (0.0009) [2023-12-26 23:03:13,603][105620] Updated weights for policy 1, policy_version 1055372 (0.0007) [2023-12-26 23:03:13,611][105692] Updated weights for policy 0, policy_version 1054370 (0.0010) [2023-12-26 23:03:13,657][105620] Updated weights for policy 1, policy_version 1055382 (0.0008) [2023-12-26 23:03:13,666][105692] Updated weights for policy 0, policy_version 1054380 (0.0010) [2023-12-26 23:03:13,716][105620] Updated weights for policy 1, policy_version 1055392 (0.0005) [2023-12-26 23:03:13,730][105692] Updated weights for policy 0, policy_version 1054390 (0.0007) [2023-12-26 23:03:13,797][105692] Updated weights for policy 0, policy_version 1054400 (0.0008) [2023-12-26 23:03:14,363][105620] Updated weights for policy 1, policy_version 1055402 (0.0006) [2023-12-26 23:03:14,427][105620] Updated weights for policy 1, policy_version 1055412 (0.0010) [2023-12-26 23:03:14,455][105692] Updated weights for policy 0, policy_version 1054410 (0.0005) [2023-12-26 23:03:14,482][105620] Updated weights for policy 1, policy_version 1055422 (0.0011) [2023-12-26 23:03:14,501][105692] Updated weights for policy 0, policy_version 1054420 (0.0005) [2023-12-26 23:03:14,548][105620] Updated weights for policy 1, policy_version 1055432 (0.0008) [2023-12-26 23:03:14,557][105692] Updated weights for policy 0, policy_version 1054430 (0.0006) [2023-12-26 23:03:15,237][105620] Updated weights for policy 1, policy_version 1055442 (0.0011) [2023-12-26 23:03:15,264][105692] Updated weights for policy 0, policy_version 1054440 (0.0010) [2023-12-26 23:03:15,301][105620] Updated weights for policy 1, policy_version 1055452 (0.0011) [2023-12-26 23:03:15,321][105692] Updated weights for policy 0, policy_version 1054450 (0.0011) [2023-12-26 23:03:15,358][105620] Updated weights for policy 1, policy_version 1055462 (0.0011) [2023-12-26 23:03:15,381][105692] Updated weights for policy 0, policy_version 1054460 (0.0008) [2023-12-26 23:03:16,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19524.4, 300 sec: 19605.3). Total num frames: 540213248. Throughput: 0: 9798.4, 1: 9880.8. Samples: 540187040. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:03:16,062][104569] Avg episode reward: [(0, '9080.914'), (1, '9352.525')] [2023-12-26 23:03:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001054464_269983744.pth... [2023-12-26 23:03:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001053344_269697024.pth [2023-12-26 23:03:16,108][105620] Updated weights for policy 1, policy_version 1055472 (0.0006) [2023-12-26 23:03:16,121][105692] Updated weights for policy 0, policy_version 1054470 (0.0010) [2023-12-26 23:03:16,164][105620] Updated weights for policy 1, policy_version 1055482 (0.0005) [2023-12-26 23:03:16,180][105692] Updated weights for policy 0, policy_version 1054480 (0.0010) [2023-12-26 23:03:16,214][105620] Updated weights for policy 1, policy_version 1055492 (0.0010) [2023-12-26 23:03:16,237][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001055496_270237696.pth... [2023-12-26 23:03:16,241][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001054312_269934592.pth [2023-12-26 23:03:16,245][105692] Updated weights for policy 0, policy_version 1054490 (0.0010) [2023-12-26 23:03:16,754][105620] Updated weights for policy 1, policy_version 1055502 (0.0010) [2023-12-26 23:03:16,808][105620] Updated weights for policy 1, policy_version 1055512 (0.0010) [2023-12-26 23:03:16,860][105620] Updated weights for policy 1, policy_version 1055522 (0.0010) [2023-12-26 23:03:16,985][105692] Updated weights for policy 0, policy_version 1054500 (0.0010) [2023-12-26 23:03:17,054][105692] Updated weights for policy 0, policy_version 1054510 (0.0007) [2023-12-26 23:03:17,113][105692] Updated weights for policy 0, policy_version 1054520 (0.0009) [2023-12-26 23:03:17,410][105620] Updated weights for policy 1, policy_version 1055532 (0.0006) [2023-12-26 23:03:17,471][105620] Updated weights for policy 1, policy_version 1055542 (0.0005) [2023-12-26 23:03:17,527][105620] Updated weights for policy 1, policy_version 1055552 (0.0005) [2023-12-26 23:03:17,744][105692] Updated weights for policy 0, policy_version 1054530 (0.0006) [2023-12-26 23:03:17,806][105692] Updated weights for policy 0, policy_version 1054540 (0.0011) [2023-12-26 23:03:17,867][105692] Updated weights for policy 0, policy_version 1054550 (0.0009) [2023-12-26 23:03:17,926][105692] Updated weights for policy 0, policy_version 1054560 (0.0005) [2023-12-26 23:03:18,134][105620] Updated weights for policy 1, policy_version 1055562 (0.0005) [2023-12-26 23:03:18,182][105620] Updated weights for policy 1, policy_version 1055572 (0.0005) [2023-12-26 23:03:18,229][105620] Updated weights for policy 1, policy_version 1055582 (0.0005) [2023-12-26 23:03:18,279][105620] Updated weights for policy 1, policy_version 1055592 (0.0006) [2023-12-26 23:03:18,587][105692] Updated weights for policy 0, policy_version 1054570 (0.0011) [2023-12-26 23:03:18,645][105692] Updated weights for policy 0, policy_version 1054580 (0.0010) [2023-12-26 23:03:18,704][105692] Updated weights for policy 0, policy_version 1054590 (0.0011) [2023-12-26 23:03:18,972][105620] Updated weights for policy 1, policy_version 1055602 (0.0010) [2023-12-26 23:03:19,020][105620] Updated weights for policy 1, policy_version 1055612 (0.0010) [2023-12-26 23:03:19,072][105620] Updated weights for policy 1, policy_version 1055622 (0.0010) [2023-12-26 23:03:19,463][105692] Updated weights for policy 0, policy_version 1054600 (0.0011) [2023-12-26 23:03:19,526][105692] Updated weights for policy 0, policy_version 1054610 (0.0009) [2023-12-26 23:03:19,578][105692] Updated weights for policy 0, policy_version 1054620 (0.0010) [2023-12-26 23:03:19,867][105620] Updated weights for policy 1, policy_version 1055632 (0.0011) [2023-12-26 23:03:19,934][105620] Updated weights for policy 1, policy_version 1055642 (0.0010) [2023-12-26 23:03:20,002][105620] Updated weights for policy 1, policy_version 1055652 (0.0006) [2023-12-26 23:03:20,350][105692] Updated weights for policy 0, policy_version 1054630 (0.0011) [2023-12-26 23:03:20,413][105692] Updated weights for policy 0, policy_version 1054640 (0.0011) [2023-12-26 23:03:20,477][105692] Updated weights for policy 0, policy_version 1054650 (0.0011) [2023-12-26 23:03:20,734][105620] Updated weights for policy 1, policy_version 1055662 (0.0009) [2023-12-26 23:03:20,802][105620] Updated weights for policy 1, policy_version 1055672 (0.0011) [2023-12-26 23:03:20,865][105620] Updated weights for policy 1, policy_version 1055682 (0.0011) [2023-12-26 23:03:21,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 540319744. Throughput: 0: 9652.4, 1: 9978.1. Samples: 540308780. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:03:21,062][104569] Avg episode reward: [(0, '9169.763'), (1, '9259.594')] [2023-12-26 23:03:21,244][105692] Updated weights for policy 0, policy_version 1054660 (0.0011) [2023-12-26 23:03:21,304][105692] Updated weights for policy 0, policy_version 1054670 (0.0010) [2023-12-26 23:03:21,372][105692] Updated weights for policy 0, policy_version 1054680 (0.0007) [2023-12-26 23:03:21,670][105620] Updated weights for policy 1, policy_version 1055692 (0.0010) [2023-12-26 23:03:21,746][105620] Updated weights for policy 1, policy_version 1055702 (0.0009) [2023-12-26 23:03:21,802][105620] Updated weights for policy 1, policy_version 1055712 (0.0010) [2023-12-26 23:03:22,069][105692] Updated weights for policy 0, policy_version 1054690 (0.0010) [2023-12-26 23:03:22,129][105692] Updated weights for policy 0, policy_version 1054700 (0.0010) [2023-12-26 23:03:22,188][105692] Updated weights for policy 0, policy_version 1054710 (0.0010) [2023-12-26 23:03:22,254][105692] Updated weights for policy 0, policy_version 1054720 (0.0011) [2023-12-26 23:03:22,541][105620] Updated weights for policy 1, policy_version 1055722 (0.0009) [2023-12-26 23:03:22,600][105620] Updated weights for policy 1, policy_version 1055732 (0.0008) [2023-12-26 23:03:22,660][105620] Updated weights for policy 1, policy_version 1055742 (0.0008) [2023-12-26 23:03:22,724][105620] Updated weights for policy 1, policy_version 1055752 (0.0009) [2023-12-26 23:03:22,995][105692] Updated weights for policy 0, policy_version 1054730 (0.0011) [2023-12-26 23:03:23,047][105692] Updated weights for policy 0, policy_version 1054740 (0.0011) [2023-12-26 23:03:23,103][105692] Updated weights for policy 0, policy_version 1054750 (0.0011) [2023-12-26 23:03:23,356][105620] Updated weights for policy 1, policy_version 1055762 (0.0005) [2023-12-26 23:03:23,412][105620] Updated weights for policy 1, policy_version 1055772 (0.0005) [2023-12-26 23:03:23,470][105620] Updated weights for policy 1, policy_version 1055782 (0.0005) [2023-12-26 23:03:23,809][105692] Updated weights for policy 0, policy_version 1054760 (0.0010) [2023-12-26 23:03:23,867][105692] Updated weights for policy 0, policy_version 1054770 (0.0010) [2023-12-26 23:03:23,917][105692] Updated weights for policy 0, policy_version 1054780 (0.0010) [2023-12-26 23:03:23,969][105620] Updated weights for policy 1, policy_version 1055792 (0.0008) [2023-12-26 23:03:24,022][105620] Updated weights for policy 1, policy_version 1055802 (0.0005) [2023-12-26 23:03:24,074][105620] Updated weights for policy 1, policy_version 1055812 (0.0005) [2023-12-26 23:03:24,678][105620] Updated weights for policy 1, policy_version 1055822 (0.0008) [2023-12-26 23:03:24,688][105692] Updated weights for policy 0, policy_version 1054790 (0.0007) [2023-12-26 23:03:24,741][105620] Updated weights for policy 1, policy_version 1055832 (0.0011) [2023-12-26 23:03:24,753][105692] Updated weights for policy 0, policy_version 1054800 (0.0005) [2023-12-26 23:03:24,797][105620] Updated weights for policy 1, policy_version 1055842 (0.0011) [2023-12-26 23:03:24,819][105692] Updated weights for policy 0, policy_version 1054810 (0.0006) [2023-12-26 23:03:25,449][105692] Updated weights for policy 0, policy_version 1054820 (0.0009) [2023-12-26 23:03:25,499][105620] Updated weights for policy 1, policy_version 1055852 (0.0010) [2023-12-26 23:03:25,509][105692] Updated weights for policy 0, policy_version 1054830 (0.0007) [2023-12-26 23:03:25,552][105620] Updated weights for policy 1, policy_version 1055862 (0.0010) [2023-12-26 23:03:25,563][105692] Updated weights for policy 0, policy_version 1054840 (0.0006) [2023-12-26 23:03:25,615][105620] Updated weights for policy 1, policy_version 1055872 (0.0010) [2023-12-26 23:03:26,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 540418048. Throughput: 0: 9626.2, 1: 10020.2. Samples: 540426960. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:03:26,062][104569] Avg episode reward: [(0, '8989.057'), (1, '9168.396')] [2023-12-26 23:03:26,216][105620] Updated weights for policy 1, policy_version 1055882 (0.0010) [2023-12-26 23:03:26,269][105620] Updated weights for policy 1, policy_version 1055892 (0.0009) [2023-12-26 23:03:26,296][105692] Updated weights for policy 0, policy_version 1054850 (0.0011) [2023-12-26 23:03:26,329][105620] Updated weights for policy 1, policy_version 1055902 (0.0007) [2023-12-26 23:03:26,355][105692] Updated weights for policy 0, policy_version 1054860 (0.0011) [2023-12-26 23:03:26,385][105620] Updated weights for policy 1, policy_version 1055912 (0.0005) [2023-12-26 23:03:26,407][105692] Updated weights for policy 0, policy_version 1054870 (0.0010) [2023-12-26 23:03:26,472][105692] Updated weights for policy 0, policy_version 1054880 (0.0011) [2023-12-26 23:03:27,075][105620] Updated weights for policy 1, policy_version 1055922 (0.0008) [2023-12-26 23:03:27,121][105620] Updated weights for policy 1, policy_version 1055932 (0.0009) [2023-12-26 23:03:27,175][105692] Updated weights for policy 0, policy_version 1054890 (0.0007) [2023-12-26 23:03:27,176][105620] Updated weights for policy 1, policy_version 1055942 (0.0006) [2023-12-26 23:03:27,235][105692] Updated weights for policy 0, policy_version 1054900 (0.0009) [2023-12-26 23:03:27,300][105692] Updated weights for policy 0, policy_version 1054910 (0.0009) [2023-12-26 23:03:27,847][105620] Updated weights for policy 1, policy_version 1055952 (0.0005) [2023-12-26 23:03:27,873][105692] Updated weights for policy 0, policy_version 1054920 (0.0006) [2023-12-26 23:03:27,906][105620] Updated weights for policy 1, policy_version 1055962 (0.0008) [2023-12-26 23:03:27,923][105692] Updated weights for policy 0, policy_version 1054930 (0.0005) [2023-12-26 23:03:27,962][105620] Updated weights for policy 1, policy_version 1055972 (0.0009) [2023-12-26 23:03:27,973][105692] Updated weights for policy 0, policy_version 1054940 (0.0005) [2023-12-26 23:03:28,586][105692] Updated weights for policy 0, policy_version 1054950 (0.0008) [2023-12-26 23:03:28,644][105692] Updated weights for policy 0, policy_version 1054960 (0.0010) [2023-12-26 23:03:28,670][105620] Updated weights for policy 1, policy_version 1055982 (0.0006) [2023-12-26 23:03:28,696][105585] KL-divergence is very high: 106.2957 [2023-12-26 23:03:28,703][105692] Updated weights for policy 0, policy_version 1054970 (0.0010) [2023-12-26 23:03:28,729][105620] Updated weights for policy 1, policy_version 1055992 (0.0006) [2023-12-26 23:03:28,786][105620] Updated weights for policy 1, policy_version 1056002 (0.0007) [2023-12-26 23:03:29,442][105692] Updated weights for policy 0, policy_version 1054980 (0.0011) [2023-12-26 23:03:29,490][105692] Updated weights for policy 0, policy_version 1054990 (0.0011) [2023-12-26 23:03:29,491][105620] Updated weights for policy 1, policy_version 1056012 (0.0009) [2023-12-26 23:03:29,539][105620] Updated weights for policy 1, policy_version 1056022 (0.0008) [2023-12-26 23:03:29,545][105692] Updated weights for policy 0, policy_version 1055000 (0.0011) [2023-12-26 23:03:29,584][105620] Updated weights for policy 1, policy_version 1056032 (0.0005) [2023-12-26 23:03:30,251][105692] Updated weights for policy 0, policy_version 1055010 (0.0010) [2023-12-26 23:03:30,298][105620] Updated weights for policy 1, policy_version 1056042 (0.0008) [2023-12-26 23:03:30,311][105692] Updated weights for policy 0, policy_version 1055020 (0.0007) [2023-12-26 23:03:30,359][105620] Updated weights for policy 1, policy_version 1056052 (0.0011) [2023-12-26 23:03:30,368][105692] Updated weights for policy 0, policy_version 1055030 (0.0011) [2023-12-26 23:03:30,415][105620] Updated weights for policy 1, policy_version 1056062 (0.0010) [2023-12-26 23:03:30,416][105692] Updated weights for policy 0, policy_version 1055040 (0.0011) [2023-12-26 23:03:30,474][105620] Updated weights for policy 1, policy_version 1056072 (0.0010) [2023-12-26 23:03:31,035][105692] Updated weights for policy 0, policy_version 1055050 (0.0008) [2023-12-26 23:03:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 540516352. Throughput: 0: 9707.6, 1: 10067.5. Samples: 540489448. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:03:31,063][104569] Avg episode reward: [(0, '8899.850'), (1, '9169.600')] [2023-12-26 23:03:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001056072_270385152.pth... [2023-12-26 23:03:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001054888_270082048.pth [2023-12-26 23:03:31,102][105692] Updated weights for policy 0, policy_version 1055060 (0.0011) [2023-12-26 23:03:31,168][105692] Updated weights for policy 0, policy_version 1055070 (0.0014) [2023-12-26 23:03:31,179][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001055072_270139392.pth... [2023-12-26 23:03:31,184][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001053920_269844480.pth [2023-12-26 23:03:31,291][105620] Updated weights for policy 1, policy_version 1056082 (0.0008) [2023-12-26 23:03:31,347][105620] Updated weights for policy 1, policy_version 1056092 (0.0009) [2023-12-26 23:03:31,410][105620] Updated weights for policy 1, policy_version 1056102 (0.0009) [2023-12-26 23:03:31,920][105692] Updated weights for policy 0, policy_version 1055080 (0.0007) [2023-12-26 23:03:31,980][105692] Updated weights for policy 0, policy_version 1055090 (0.0006) [2023-12-26 23:03:32,031][105692] Updated weights for policy 0, policy_version 1055100 (0.0005) [2023-12-26 23:03:32,105][105620] Updated weights for policy 1, policy_version 1056112 (0.0008) [2023-12-26 23:03:32,157][105620] Updated weights for policy 1, policy_version 1056122 (0.0010) [2023-12-26 23:03:32,212][105620] Updated weights for policy 1, policy_version 1056132 (0.0006) [2023-12-26 23:03:32,644][105692] Updated weights for policy 0, policy_version 1055110 (0.0005) [2023-12-26 23:03:32,699][105692] Updated weights for policy 0, policy_version 1055120 (0.0005) [2023-12-26 23:03:32,769][105692] Updated weights for policy 0, policy_version 1055130 (0.0005) [2023-12-26 23:03:32,913][105620] Updated weights for policy 1, policy_version 1056142 (0.0009) [2023-12-26 23:03:32,974][105620] Updated weights for policy 1, policy_version 1056152 (0.0010) [2023-12-26 23:03:33,028][105620] Updated weights for policy 1, policy_version 1056162 (0.0010) [2023-12-26 23:03:33,323][105692] Updated weights for policy 0, policy_version 1055140 (0.0009) [2023-12-26 23:03:33,377][105692] Updated weights for policy 0, policy_version 1055150 (0.0010) [2023-12-26 23:03:33,433][105692] Updated weights for policy 0, policy_version 1055160 (0.0005) [2023-12-26 23:03:33,760][105620] Updated weights for policy 1, policy_version 1056172 (0.0008) [2023-12-26 23:03:33,818][105620] Updated weights for policy 1, policy_version 1056182 (0.0010) [2023-12-26 23:03:33,868][105620] Updated weights for policy 1, policy_version 1056192 (0.0010) [2023-12-26 23:03:34,045][105692] Updated weights for policy 0, policy_version 1055170 (0.0007) [2023-12-26 23:03:34,096][105692] Updated weights for policy 0, policy_version 1055180 (0.0008) [2023-12-26 23:03:34,147][105692] Updated weights for policy 0, policy_version 1055190 (0.0008) [2023-12-26 23:03:34,206][105692] Updated weights for policy 0, policy_version 1055200 (0.0008) [2023-12-26 23:03:34,617][105620] Updated weights for policy 1, policy_version 1056202 (0.0009) [2023-12-26 23:03:34,680][105620] Updated weights for policy 1, policy_version 1056212 (0.0010) [2023-12-26 23:03:34,738][105620] Updated weights for policy 1, policy_version 1056222 (0.0010) [2023-12-26 23:03:34,796][105620] Updated weights for policy 1, policy_version 1056232 (0.0010) [2023-12-26 23:03:34,926][105692] Updated weights for policy 0, policy_version 1055210 (0.0009) [2023-12-26 23:03:34,979][105692] Updated weights for policy 0, policy_version 1055220 (0.0006) [2023-12-26 23:03:35,040][105692] Updated weights for policy 0, policy_version 1055230 (0.0005) [2023-12-26 23:03:35,354][105620] Updated weights for policy 1, policy_version 1056242 (0.0006) [2023-12-26 23:03:35,415][105620] Updated weights for policy 1, policy_version 1056252 (0.0005) [2023-12-26 23:03:35,467][105620] Updated weights for policy 1, policy_version 1056262 (0.0005) [2023-12-26 23:03:35,708][105692] Updated weights for policy 0, policy_version 1055240 (0.0005) [2023-12-26 23:03:35,768][105692] Updated weights for policy 0, policy_version 1055250 (0.0005) [2023-12-26 23:03:35,816][105692] Updated weights for policy 0, policy_version 1055260 (0.0005) [2023-12-26 23:03:36,036][105620] Updated weights for policy 1, policy_version 1056272 (0.0005) [2023-12-26 23:03:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 540622848. Throughput: 0: 9852.0, 1: 10032.1. Samples: 540609740. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:03:36,063][104569] Avg episode reward: [(0, '9170.899'), (1, '9259.539')] [2023-12-26 23:03:36,091][105620] Updated weights for policy 1, policy_version 1056282 (0.0010) [2023-12-26 23:03:36,162][105620] Updated weights for policy 1, policy_version 1056292 (0.0006) [2023-12-26 23:03:36,481][105692] Updated weights for policy 0, policy_version 1055270 (0.0009) [2023-12-26 23:03:36,545][105692] Updated weights for policy 0, policy_version 1055280 (0.0011) [2023-12-26 23:03:36,608][105692] Updated weights for policy 0, policy_version 1055290 (0.0011) [2023-12-26 23:03:36,839][105620] Updated weights for policy 1, policy_version 1056302 (0.0008) [2023-12-26 23:03:36,908][105620] Updated weights for policy 1, policy_version 1056312 (0.0011) [2023-12-26 23:03:36,977][105620] Updated weights for policy 1, policy_version 1056322 (0.0010) [2023-12-26 23:03:37,296][105692] Updated weights for policy 0, policy_version 1055300 (0.0008) [2023-12-26 23:03:37,356][105692] Updated weights for policy 0, policy_version 1055310 (0.0010) [2023-12-26 23:03:37,425][105692] Updated weights for policy 0, policy_version 1055320 (0.0008) [2023-12-26 23:03:37,579][105620] Updated weights for policy 1, policy_version 1056332 (0.0008) [2023-12-26 23:03:37,639][105620] Updated weights for policy 1, policy_version 1056342 (0.0010) [2023-12-26 23:03:37,692][105620] Updated weights for policy 1, policy_version 1056353 (0.0010) [2023-12-26 23:03:38,076][105692] Updated weights for policy 0, policy_version 1055330 (0.0009) [2023-12-26 23:03:38,148][105692] Updated weights for policy 0, policy_version 1055340 (0.0009) [2023-12-26 23:03:38,215][105692] Updated weights for policy 0, policy_version 1055350 (0.0008) [2023-12-26 23:03:38,274][105692] Updated weights for policy 0, policy_version 1055360 (0.0006) [2023-12-26 23:03:38,501][105620] Updated weights for policy 1, policy_version 1056363 (0.0009) [2023-12-26 23:03:38,564][105620] Updated weights for policy 1, policy_version 1056373 (0.0009) [2023-12-26 23:03:38,623][105620] Updated weights for policy 1, policy_version 1056383 (0.0009) [2023-12-26 23:03:38,998][105692] Updated weights for policy 0, policy_version 1055370 (0.0009) [2023-12-26 23:03:39,057][105692] Updated weights for policy 0, policy_version 1055380 (0.0009) [2023-12-26 23:03:39,121][105692] Updated weights for policy 0, policy_version 1055390 (0.0010) [2023-12-26 23:03:39,319][105620] Updated weights for policy 1, policy_version 1056393 (0.0009) [2023-12-26 23:03:39,386][105620] Updated weights for policy 1, policy_version 1056403 (0.0009) [2023-12-26 23:03:39,453][105620] Updated weights for policy 1, policy_version 1056413 (0.0009) [2023-12-26 23:03:39,519][105620] Updated weights for policy 1, policy_version 1056423 (0.0009) [2023-12-26 23:03:39,874][105692] Updated weights for policy 0, policy_version 1055400 (0.0008) [2023-12-26 23:03:39,938][105692] Updated weights for policy 0, policy_version 1055410 (0.0008) [2023-12-26 23:03:39,995][105692] Updated weights for policy 0, policy_version 1055420 (0.0010) [2023-12-26 23:03:40,241][105620] Updated weights for policy 1, policy_version 1056433 (0.0009) [2023-12-26 23:03:40,303][105620] Updated weights for policy 1, policy_version 1056443 (0.0009) [2023-12-26 23:03:40,367][105620] Updated weights for policy 1, policy_version 1056453 (0.0007) [2023-12-26 23:03:40,717][105692] Updated weights for policy 0, policy_version 1055430 (0.0010) [2023-12-26 23:03:40,768][105692] Updated weights for policy 0, policy_version 1055440 (0.0009) [2023-12-26 23:03:40,823][105692] Updated weights for policy 0, policy_version 1055450 (0.0009) [2023-12-26 23:03:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 540721152. Throughput: 0: 9795.0, 1: 10029.4. Samples: 540729700. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:03:41,063][104569] Avg episode reward: [(0, '9260.996'), (1, '9349.507')] [2023-12-26 23:03:41,103][105620] Updated weights for policy 1, policy_version 1056463 (0.0009) [2023-12-26 23:03:41,172][105620] Updated weights for policy 1, policy_version 1056473 (0.0009) [2023-12-26 23:03:41,237][105620] Updated weights for policy 1, policy_version 1056483 (0.0008) [2023-12-26 23:03:41,576][105692] Updated weights for policy 0, policy_version 1055460 (0.0007) [2023-12-26 23:03:41,647][105692] Updated weights for policy 0, policy_version 1055470 (0.0007) [2023-12-26 23:03:41,706][105692] Updated weights for policy 0, policy_version 1055480 (0.0009) [2023-12-26 23:03:42,004][105620] Updated weights for policy 1, policy_version 1056493 (0.0010) [2023-12-26 23:03:42,061][105620] Updated weights for policy 1, policy_version 1056503 (0.0008) [2023-12-26 23:03:42,118][105620] Updated weights for policy 1, policy_version 1056513 (0.0007) [2023-12-26 23:03:42,532][105692] Updated weights for policy 0, policy_version 1055490 (0.0010) [2023-12-26 23:03:42,593][105692] Updated weights for policy 0, policy_version 1055500 (0.0010) [2023-12-26 23:03:42,649][105692] Updated weights for policy 0, policy_version 1055510 (0.0008) [2023-12-26 23:03:42,712][105692] Updated weights for policy 0, policy_version 1055520 (0.0008) [2023-12-26 23:03:42,866][105620] Updated weights for policy 1, policy_version 1056523 (0.0010) [2023-12-26 23:03:42,924][105620] Updated weights for policy 1, policy_version 1056533 (0.0010) [2023-12-26 23:03:42,977][105620] Updated weights for policy 1, policy_version 1056543 (0.0008) [2023-12-26 23:03:43,524][105692] Updated weights for policy 0, policy_version 1055530 (0.0008) [2023-12-26 23:03:43,539][105620] Updated weights for policy 1, policy_version 1056553 (0.0005) [2023-12-26 23:03:43,573][105692] Updated weights for policy 0, policy_version 1055540 (0.0005) [2023-12-26 23:03:43,593][105620] Updated weights for policy 1, policy_version 1056563 (0.0005) [2023-12-26 23:03:43,627][105692] Updated weights for policy 0, policy_version 1055550 (0.0005) [2023-12-26 23:03:43,643][105620] Updated weights for policy 1, policy_version 1056573 (0.0006) [2023-12-26 23:03:43,692][105620] Updated weights for policy 1, policy_version 1056583 (0.0005) [2023-12-26 23:03:44,174][105692] Updated weights for policy 0, policy_version 1055560 (0.0005) [2023-12-26 23:03:44,223][105692] Updated weights for policy 0, policy_version 1055570 (0.0006) [2023-12-26 23:03:44,262][105620] Updated weights for policy 1, policy_version 1056593 (0.0007) [2023-12-26 23:03:44,269][105692] Updated weights for policy 0, policy_version 1055580 (0.0007) [2023-12-26 23:03:44,321][105620] Updated weights for policy 1, policy_version 1056603 (0.0009) [2023-12-26 23:03:44,385][105620] Updated weights for policy 1, policy_version 1056613 (0.0010) [2023-12-26 23:03:44,989][105692] Updated weights for policy 0, policy_version 1055590 (0.0009) [2023-12-26 23:03:45,046][105692] Updated weights for policy 0, policy_version 1055600 (0.0011) [2023-12-26 23:03:45,109][105692] Updated weights for policy 0, policy_version 1055610 (0.0011) [2023-12-26 23:03:45,158][105620] Updated weights for policy 1, policy_version 1056623 (0.0008) [2023-12-26 23:03:45,207][105620] Updated weights for policy 1, policy_version 1056633 (0.0008) [2023-12-26 23:03:45,255][105620] Updated weights for policy 1, policy_version 1056643 (0.0007) [2023-12-26 23:03:45,863][105692] Updated weights for policy 0, policy_version 1055620 (0.0009) [2023-12-26 23:03:45,932][105692] Updated weights for policy 0, policy_version 1055630 (0.0005) [2023-12-26 23:03:45,951][105620] Updated weights for policy 1, policy_version 1056653 (0.0009) [2023-12-26 23:03:45,992][105692] Updated weights for policy 0, policy_version 1055640 (0.0009) [2023-12-26 23:03:46,011][105620] Updated weights for policy 1, policy_version 1056663 (0.0006) [2023-12-26 23:03:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 540819456. Throughput: 0: 9709.0, 1: 10072.7. Samples: 540787096. Policy #0 lag: (min: 8.0, avg: 35.4, max: 40.0) [2023-12-26 23:03:46,062][104569] Avg episode reward: [(0, '9354.737'), (1, '9349.548')] [2023-12-26 23:03:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001055648_270286848.pth... [2023-12-26 23:03:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001054464_269983744.pth [2023-12-26 23:03:46,076][105620] Updated weights for policy 1, policy_version 1056673 (0.0008) [2023-12-26 23:03:46,112][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001056680_270540800.pth... [2023-12-26 23:03:46,115][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001055496_270237696.pth [2023-12-26 23:03:46,563][105692] Updated weights for policy 0, policy_version 1055650 (0.0010) [2023-12-26 23:03:46,619][105692] Updated weights for policy 0, policy_version 1055660 (0.0008) [2023-12-26 23:03:46,666][105692] Updated weights for policy 0, policy_version 1055670 (0.0007) [2023-12-26 23:03:46,720][105692] Updated weights for policy 0, policy_version 1055680 (0.0005) [2023-12-26 23:03:46,763][105620] Updated weights for policy 1, policy_version 1056683 (0.0010) [2023-12-26 23:03:46,835][105620] Updated weights for policy 1, policy_version 1056693 (0.0010) [2023-12-26 23:03:46,901][105620] Updated weights for policy 1, policy_version 1056703 (0.0010) [2023-12-26 23:03:47,316][105692] Updated weights for policy 0, policy_version 1055690 (0.0006) [2023-12-26 23:03:47,360][105692] Updated weights for policy 0, policy_version 1055700 (0.0007) [2023-12-26 23:03:47,409][105692] Updated weights for policy 0, policy_version 1055710 (0.0008) [2023-12-26 23:03:47,627][105620] Updated weights for policy 1, policy_version 1056713 (0.0011) [2023-12-26 23:03:47,678][105620] Updated weights for policy 1, policy_version 1056723 (0.0010) [2023-12-26 23:03:47,733][105620] Updated weights for policy 1, policy_version 1056733 (0.0010) [2023-12-26 23:03:47,791][105620] Updated weights for policy 1, policy_version 1056743 (0.0010) [2023-12-26 23:03:47,990][105692] Updated weights for policy 0, policy_version 1055720 (0.0006) [2023-12-26 23:03:48,044][105692] Updated weights for policy 0, policy_version 1055730 (0.0005) [2023-12-26 23:03:48,093][105692] Updated weights for policy 0, policy_version 1055740 (0.0005) [2023-12-26 23:03:48,495][105620] Updated weights for policy 1, policy_version 1056753 (0.0010) [2023-12-26 23:03:48,566][105620] Updated weights for policy 1, policy_version 1056763 (0.0009) [2023-12-26 23:03:48,623][105620] Updated weights for policy 1, policy_version 1056773 (0.0009) [2023-12-26 23:03:48,696][105692] Updated weights for policy 0, policy_version 1055750 (0.0005) [2023-12-26 23:03:48,749][105692] Updated weights for policy 0, policy_version 1055760 (0.0010) [2023-12-26 23:03:48,805][105692] Updated weights for policy 0, policy_version 1055770 (0.0010) [2023-12-26 23:03:49,312][105620] Updated weights for policy 1, policy_version 1056783 (0.0007) [2023-12-26 23:03:49,379][105620] Updated weights for policy 1, policy_version 1056793 (0.0008) [2023-12-26 23:03:49,432][105620] Updated weights for policy 1, policy_version 1056804 (0.0010) [2023-12-26 23:03:49,545][105692] Updated weights for policy 0, policy_version 1055780 (0.0010) [2023-12-26 23:03:49,609][105692] Updated weights for policy 0, policy_version 1055790 (0.0010) [2023-12-26 23:03:49,677][105692] Updated weights for policy 0, policy_version 1055800 (0.0010) [2023-12-26 23:03:50,109][105620] Updated weights for policy 1, policy_version 1056814 (0.0009) [2023-12-26 23:03:50,159][105620] Updated weights for policy 1, policy_version 1056824 (0.0007) [2023-12-26 23:03:50,210][105620] Updated weights for policy 1, policy_version 1056834 (0.0009) [2023-12-26 23:03:50,473][105692] Updated weights for policy 0, policy_version 1055810 (0.0009) [2023-12-26 23:03:50,527][105692] Updated weights for policy 0, policy_version 1055820 (0.0009) [2023-12-26 23:03:50,586][105692] Updated weights for policy 0, policy_version 1055830 (0.0009) [2023-12-26 23:03:50,647][105692] Updated weights for policy 0, policy_version 1055840 (0.0009) [2023-12-26 23:03:50,900][105620] Updated weights for policy 1, policy_version 1056844 (0.0008) [2023-12-26 23:03:50,966][105620] Updated weights for policy 1, policy_version 1056854 (0.0007) [2023-12-26 23:03:51,031][105620] Updated weights for policy 1, policy_version 1056864 (0.0009) [2023-12-26 23:03:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 540917760. Throughput: 0: 9912.6, 1: 9966.1. Samples: 540911548. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:03:51,062][104569] Avg episode reward: [(0, '9267.084'), (1, '9261.463')] [2023-12-26 23:03:51,490][105692] Updated weights for policy 0, policy_version 1055850 (0.0009) [2023-12-26 23:03:51,548][105692] Updated weights for policy 0, policy_version 1055860 (0.0009) [2023-12-26 23:03:51,608][105692] Updated weights for policy 0, policy_version 1055870 (0.0009) [2023-12-26 23:03:51,736][105620] Updated weights for policy 1, policy_version 1056874 (0.0009) [2023-12-26 23:03:51,801][105620] Updated weights for policy 1, policy_version 1056884 (0.0007) [2023-12-26 23:03:51,868][105620] Updated weights for policy 1, policy_version 1056894 (0.0008) [2023-12-26 23:03:51,925][105620] Updated weights for policy 1, policy_version 1056904 (0.0008) [2023-12-26 23:03:52,416][105692] Updated weights for policy 0, policy_version 1055880 (0.0011) [2023-12-26 23:03:52,472][105692] Updated weights for policy 0, policy_version 1055890 (0.0011) [2023-12-26 23:03:52,530][105692] Updated weights for policy 0, policy_version 1055900 (0.0010) [2023-12-26 23:03:52,673][105620] Updated weights for policy 1, policy_version 1056914 (0.0009) [2023-12-26 23:03:52,727][105620] Updated weights for policy 1, policy_version 1056924 (0.0008) [2023-12-26 23:03:52,774][105620] Updated weights for policy 1, policy_version 1056934 (0.0006) [2023-12-26 23:03:53,260][105692] Updated weights for policy 0, policy_version 1055910 (0.0007) [2023-12-26 23:03:53,327][105692] Updated weights for policy 0, policy_version 1055920 (0.0006) [2023-12-26 23:03:53,393][105692] Updated weights for policy 0, policy_version 1055930 (0.0007) [2023-12-26 23:03:53,467][105620] Updated weights for policy 1, policy_version 1056944 (0.0006) [2023-12-26 23:03:53,521][105620] Updated weights for policy 1, policy_version 1056954 (0.0005) [2023-12-26 23:03:53,576][105620] Updated weights for policy 1, policy_version 1056964 (0.0005) [2023-12-26 23:03:53,962][105692] Updated weights for policy 0, policy_version 1055940 (0.0010) [2023-12-26 23:03:54,034][105692] Updated weights for policy 0, policy_version 1055950 (0.0011) [2023-12-26 23:03:54,095][105692] Updated weights for policy 0, policy_version 1055960 (0.0010) [2023-12-26 23:03:54,112][105585] KL-divergence is very high: 176.6139 [2023-12-26 23:03:54,234][105620] Updated weights for policy 1, policy_version 1056974 (0.0006) [2023-12-26 23:03:54,286][105620] Updated weights for policy 1, policy_version 1056984 (0.0005) [2023-12-26 23:03:54,339][105620] Updated weights for policy 1, policy_version 1056994 (0.0005) [2023-12-26 23:03:54,809][105692] Updated weights for policy 0, policy_version 1055970 (0.0011) [2023-12-26 23:03:54,866][105692] Updated weights for policy 0, policy_version 1055980 (0.0010) [2023-12-26 23:03:54,910][105692] Updated weights for policy 0, policy_version 1055990 (0.0010) [2023-12-26 23:03:54,970][105692] Updated weights for policy 0, policy_version 1056000 (0.0010) [2023-12-26 23:03:55,015][105620] Updated weights for policy 1, policy_version 1057004 (0.0009) [2023-12-26 23:03:55,074][105620] Updated weights for policy 1, policy_version 1057014 (0.0007) [2023-12-26 23:03:55,131][105620] Updated weights for policy 1, policy_version 1057024 (0.0007) [2023-12-26 23:03:55,680][105692] Updated weights for policy 0, policy_version 1056010 (0.0010) [2023-12-26 23:03:55,734][105692] Updated weights for policy 0, policy_version 1056020 (0.0010) [2023-12-26 23:03:55,788][105692] Updated weights for policy 0, policy_version 1056030 (0.0010) [2023-12-26 23:03:55,815][105620] Updated weights for policy 1, policy_version 1057034 (0.0009) [2023-12-26 23:03:55,882][105620] Updated weights for policy 1, policy_version 1057044 (0.0008) [2023-12-26 23:03:55,934][105620] Updated weights for policy 1, policy_version 1057054 (0.0005) [2023-12-26 23:03:55,982][105620] Updated weights for policy 1, policy_version 1057064 (0.0005) [2023-12-26 23:03:56,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19933.9, 300 sec: 19660.8). Total num frames: 541024256. Throughput: 0: 9911.9, 1: 10053.2. Samples: 541028100. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:03:56,062][104569] Avg episode reward: [(0, '8997.066'), (1, '9262.128')] [2023-12-26 23:03:56,498][105692] Updated weights for policy 0, policy_version 1056040 (0.0005) [2023-12-26 23:03:56,544][105692] Updated weights for policy 0, policy_version 1056050 (0.0005) [2023-12-26 23:03:56,590][105692] Updated weights for policy 0, policy_version 1056060 (0.0005) [2023-12-26 23:03:56,752][105620] Updated weights for policy 1, policy_version 1057074 (0.0010) [2023-12-26 23:03:56,805][105620] Updated weights for policy 1, policy_version 1057084 (0.0010) [2023-12-26 23:03:56,865][105620] Updated weights for policy 1, policy_version 1057094 (0.0009) [2023-12-26 23:03:57,191][105692] Updated weights for policy 0, policy_version 1056070 (0.0008) [2023-12-26 23:03:57,241][105692] Updated weights for policy 0, policy_version 1056080 (0.0006) [2023-12-26 23:03:57,286][105692] Updated weights for policy 0, policy_version 1056090 (0.0006) [2023-12-26 23:03:57,701][105620] Updated weights for policy 1, policy_version 1057104 (0.0009) [2023-12-26 23:03:57,748][105620] Updated weights for policy 1, policy_version 1057114 (0.0009) [2023-12-26 23:03:57,806][105620] Updated weights for policy 1, policy_version 1057124 (0.0009) [2023-12-26 23:03:57,911][105692] Updated weights for policy 0, policy_version 1056100 (0.0008) [2023-12-26 23:03:57,958][105692] Updated weights for policy 0, policy_version 1056110 (0.0010) [2023-12-26 23:03:58,009][105692] Updated weights for policy 0, policy_version 1056120 (0.0010) [2023-12-26 23:03:58,588][105620] Updated weights for policy 1, policy_version 1057134 (0.0009) [2023-12-26 23:03:58,651][105620] Updated weights for policy 1, policy_version 1057144 (0.0007) [2023-12-26 23:03:58,712][105620] Updated weights for policy 1, policy_version 1057154 (0.0009) [2023-12-26 23:03:58,748][105692] Updated weights for policy 0, policy_version 1056130 (0.0008) [2023-12-26 23:03:58,803][105692] Updated weights for policy 0, policy_version 1056140 (0.0009) [2023-12-26 23:03:58,871][105692] Updated weights for policy 0, policy_version 1056150 (0.0014) [2023-12-26 23:03:58,936][105692] Updated weights for policy 0, policy_version 1056160 (0.0009) [2023-12-26 23:03:59,505][105620] Updated weights for policy 1, policy_version 1057164 (0.0008) [2023-12-26 23:03:59,558][105620] Updated weights for policy 1, policy_version 1057174 (0.0010) [2023-12-26 23:03:59,611][105620] Updated weights for policy 1, policy_version 1057184 (0.0009) [2023-12-26 23:03:59,711][105692] Updated weights for policy 0, policy_version 1056170 (0.0007) [2023-12-26 23:03:59,765][105692] Updated weights for policy 0, policy_version 1056180 (0.0006) [2023-12-26 23:03:59,818][105692] Updated weights for policy 0, policy_version 1056190 (0.0006) [2023-12-26 23:04:00,416][105620] Updated weights for policy 1, policy_version 1057194 (0.0009) [2023-12-26 23:04:00,477][105692] Updated weights for policy 0, policy_version 1056200 (0.0009) [2023-12-26 23:04:00,478][105620] Updated weights for policy 1, policy_version 1057204 (0.0006) [2023-12-26 23:04:00,526][105692] Updated weights for policy 0, policy_version 1056210 (0.0009) [2023-12-26 23:04:00,544][105620] Updated weights for policy 1, policy_version 1057214 (0.0006) [2023-12-26 23:04:00,582][105692] Updated weights for policy 0, policy_version 1056220 (0.0008) [2023-12-26 23:04:00,599][105620] Updated weights for policy 1, policy_version 1057224 (0.0006) [2023-12-26 23:04:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 541114368. Throughput: 0: 9988.3, 1: 9994.0. Samples: 541086248. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:04:01,063][104569] Avg episode reward: [(0, '8996.185'), (1, '9086.046')] [2023-12-26 23:04:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001056224_270434304.pth... [2023-12-26 23:04:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001057224_270680064.pth... [2023-12-26 23:04:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001056072_270385152.pth [2023-12-26 23:04:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001055072_270139392.pth [2023-12-26 23:04:01,329][105620] Updated weights for policy 1, policy_version 1057234 (0.0009) [2023-12-26 23:04:01,385][105692] Updated weights for policy 0, policy_version 1056230 (0.0007) [2023-12-26 23:04:01,394][105620] Updated weights for policy 1, policy_version 1057244 (0.0012) [2023-12-26 23:04:01,451][105692] Updated weights for policy 0, policy_version 1056240 (0.0008) [2023-12-26 23:04:01,454][105620] Updated weights for policy 1, policy_version 1057254 (0.0008) [2023-12-26 23:04:01,510][105692] Updated weights for policy 0, policy_version 1056250 (0.0008) [2023-12-26 23:04:02,160][105620] Updated weights for policy 1, policy_version 1057264 (0.0009) [2023-12-26 23:04:02,209][105620] Updated weights for policy 1, policy_version 1057274 (0.0008) [2023-12-26 23:04:02,260][105620] Updated weights for policy 1, policy_version 1057284 (0.0008) [2023-12-26 23:04:02,278][105692] Updated weights for policy 0, policy_version 1056260 (0.0008) [2023-12-26 23:04:02,340][105692] Updated weights for policy 0, policy_version 1056270 (0.0008) [2023-12-26 23:04:02,400][105692] Updated weights for policy 0, policy_version 1056280 (0.0009) [2023-12-26 23:04:03,028][105620] Updated weights for policy 1, policy_version 1057294 (0.0009) [2023-12-26 23:04:03,089][105620] Updated weights for policy 1, policy_version 1057304 (0.0009) [2023-12-26 23:04:03,153][105620] Updated weights for policy 1, policy_version 1057314 (0.0009) [2023-12-26 23:04:03,168][105692] Updated weights for policy 0, policy_version 1056290 (0.0008) [2023-12-26 23:04:03,227][105692] Updated weights for policy 0, policy_version 1056300 (0.0007) [2023-12-26 23:04:03,286][105692] Updated weights for policy 0, policy_version 1056310 (0.0009) [2023-12-26 23:04:03,347][105692] Updated weights for policy 0, policy_version 1056320 (0.0010) [2023-12-26 23:04:03,813][105620] Updated weights for policy 1, policy_version 1057324 (0.0007) [2023-12-26 23:04:03,878][105620] Updated weights for policy 1, policy_version 1057334 (0.0007) [2023-12-26 23:04:03,940][105620] Updated weights for policy 1, policy_version 1057344 (0.0007) [2023-12-26 23:04:04,171][105692] Updated weights for policy 0, policy_version 1056330 (0.0009) [2023-12-26 23:04:04,233][105692] Updated weights for policy 0, policy_version 1056340 (0.0008) [2023-12-26 23:04:04,298][105692] Updated weights for policy 0, policy_version 1056350 (0.0006) [2023-12-26 23:04:04,593][105620] Updated weights for policy 1, policy_version 1057354 (0.0005) [2023-12-26 23:04:04,661][105620] Updated weights for policy 1, policy_version 1057364 (0.0005) [2023-12-26 23:04:04,712][105620] Updated weights for policy 1, policy_version 1057374 (0.0005) [2023-12-26 23:04:04,766][105620] Updated weights for policy 1, policy_version 1057384 (0.0005) [2023-12-26 23:04:05,044][105692] Updated weights for policy 0, policy_version 1056360 (0.0009) [2023-12-26 23:04:05,098][105692] Updated weights for policy 0, policy_version 1056370 (0.0009) [2023-12-26 23:04:05,152][105692] Updated weights for policy 0, policy_version 1056381 (0.0011) [2023-12-26 23:04:05,260][105620] Updated weights for policy 1, policy_version 1057394 (0.0005) [2023-12-26 23:04:05,321][105620] Updated weights for policy 1, policy_version 1057404 (0.0005) [2023-12-26 23:04:05,388][105620] Updated weights for policy 1, policy_version 1057414 (0.0007) [2023-12-26 23:04:05,930][105692] Updated weights for policy 0, policy_version 1056391 (0.0008) [2023-12-26 23:04:05,977][105692] Updated weights for policy 0, policy_version 1056401 (0.0008) [2023-12-26 23:04:06,009][105620] Updated weights for policy 1, policy_version 1057424 (0.0008) [2023-12-26 23:04:06,023][105692] Updated weights for policy 0, policy_version 1056411 (0.0007) [2023-12-26 23:04:06,053][105586] KL-divergence is very high: 107.3232 [2023-12-26 23:04:06,059][105620] Updated weights for policy 1, policy_version 1057434 (0.0007) [2023-12-26 23:04:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 541212672. Throughput: 0: 9891.0, 1: 9905.1. Samples: 541199604. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:04:06,062][104569] Avg episode reward: [(0, '9176.272'), (1, '8472.360')] [2023-12-26 23:04:06,121][105620] Updated weights for policy 1, policy_version 1057444 (0.0009) [2023-12-26 23:04:06,830][105692] Updated weights for policy 0, policy_version 1056421 (0.0007) [2023-12-26 23:04:06,894][105692] Updated weights for policy 0, policy_version 1056431 (0.0010) [2023-12-26 23:04:06,901][105620] Updated weights for policy 1, policy_version 1057454 (0.0007) [2023-12-26 23:04:06,951][105692] Updated weights for policy 0, policy_version 1056441 (0.0009) [2023-12-26 23:04:06,971][105620] Updated weights for policy 1, policy_version 1057464 (0.0007) [2023-12-26 23:04:07,032][105620] Updated weights for policy 1, policy_version 1057474 (0.0008) [2023-12-26 23:04:07,731][105620] Updated weights for policy 1, policy_version 1057484 (0.0008) [2023-12-26 23:04:07,733][105692] Updated weights for policy 0, policy_version 1056451 (0.0009) [2023-12-26 23:04:07,783][105692] Updated weights for policy 0, policy_version 1056461 (0.0007) [2023-12-26 23:04:07,788][105620] Updated weights for policy 1, policy_version 1057494 (0.0008) [2023-12-26 23:04:07,842][105692] Updated weights for policy 0, policy_version 1056471 (0.0009) [2023-12-26 23:04:07,843][105620] Updated weights for policy 1, policy_version 1057504 (0.0006) [2023-12-26 23:04:08,580][105692] Updated weights for policy 0, policy_version 1056481 (0.0006) [2023-12-26 23:04:08,646][105692] Updated weights for policy 0, policy_version 1056491 (0.0009) [2023-12-26 23:04:08,649][105620] Updated weights for policy 1, policy_version 1057514 (0.0007) [2023-12-26 23:04:08,698][105620] Updated weights for policy 1, policy_version 1057524 (0.0007) [2023-12-26 23:04:08,706][105692] Updated weights for policy 0, policy_version 1056501 (0.0009) [2023-12-26 23:04:08,759][105620] Updated weights for policy 1, policy_version 1057534 (0.0010) [2023-12-26 23:04:08,763][105692] Updated weights for policy 0, policy_version 1056511 (0.0008) [2023-12-26 23:04:08,815][105620] Updated weights for policy 1, policy_version 1057544 (0.0009) [2023-12-26 23:04:09,560][105692] Updated weights for policy 0, policy_version 1056521 (0.0008) [2023-12-26 23:04:09,603][105620] Updated weights for policy 1, policy_version 1057554 (0.0008) [2023-12-26 23:04:09,621][105692] Updated weights for policy 0, policy_version 1056531 (0.0009) [2023-12-26 23:04:09,668][105620] Updated weights for policy 1, policy_version 1057564 (0.0007) [2023-12-26 23:04:09,683][105692] Updated weights for policy 0, policy_version 1056541 (0.0006) [2023-12-26 23:04:09,724][105620] Updated weights for policy 1, policy_version 1057574 (0.0007) [2023-12-26 23:04:10,476][105692] Updated weights for policy 0, policy_version 1056551 (0.0007) [2023-12-26 23:04:10,503][105620] Updated weights for policy 1, policy_version 1057584 (0.0007) [2023-12-26 23:04:10,542][105692] Updated weights for policy 0, policy_version 1056561 (0.0009) [2023-12-26 23:04:10,567][105620] Updated weights for policy 1, policy_version 1057594 (0.0007) [2023-12-26 23:04:10,602][105692] Updated weights for policy 0, policy_version 1056571 (0.0008) [2023-12-26 23:04:10,624][105620] Updated weights for policy 1, policy_version 1057604 (0.0007) [2023-12-26 23:04:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 541302784. Throughput: 0: 9832.1, 1: 9815.0. Samples: 541311084. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:04:11,063][104569] Avg episode reward: [(0, '9000.410'), (1, '8418.574')] [2023-12-26 23:04:11,337][105692] Updated weights for policy 0, policy_version 1056581 (0.0009) [2023-12-26 23:04:11,406][105692] Updated weights for policy 0, policy_version 1056591 (0.0011) [2023-12-26 23:04:11,435][105620] Updated weights for policy 1, policy_version 1057614 (0.0009) [2023-12-26 23:04:11,466][105692] Updated weights for policy 0, policy_version 1056601 (0.0011) [2023-12-26 23:04:11,488][105620] Updated weights for policy 1, policy_version 1057624 (0.0009) [2023-12-26 23:04:11,547][105620] Updated weights for policy 1, policy_version 1057634 (0.0007) [2023-12-26 23:04:12,086][105692] Updated weights for policy 0, policy_version 1056611 (0.0010) [2023-12-26 23:04:12,150][105692] Updated weights for policy 0, policy_version 1056621 (0.0008) [2023-12-26 23:04:12,213][105692] Updated weights for policy 0, policy_version 1056631 (0.0008) [2023-12-26 23:04:12,397][105620] Updated weights for policy 1, policy_version 1057644 (0.0008) [2023-12-26 23:04:12,465][105620] Updated weights for policy 1, policy_version 1057654 (0.0006) [2023-12-26 23:04:12,526][105620] Updated weights for policy 1, policy_version 1057664 (0.0006) [2023-12-26 23:04:12,925][105692] Updated weights for policy 0, policy_version 1056641 (0.0008) [2023-12-26 23:04:12,982][105692] Updated weights for policy 0, policy_version 1056651 (0.0008) [2023-12-26 23:04:13,030][105692] Updated weights for policy 0, policy_version 1056661 (0.0010) [2023-12-26 23:04:13,081][105692] Updated weights for policy 0, policy_version 1056671 (0.0010) [2023-12-26 23:04:13,203][105620] Updated weights for policy 1, policy_version 1057674 (0.0006) [2023-12-26 23:04:13,252][105620] Updated weights for policy 1, policy_version 1057684 (0.0008) [2023-12-26 23:04:13,310][105620] Updated weights for policy 1, policy_version 1057694 (0.0008) [2023-12-26 23:04:13,365][105620] Updated weights for policy 1, policy_version 1057704 (0.0008) [2023-12-26 23:04:13,849][105692] Updated weights for policy 0, policy_version 1056681 (0.0010) [2023-12-26 23:04:13,907][105692] Updated weights for policy 0, policy_version 1056691 (0.0010) [2023-12-26 23:04:13,960][105692] Updated weights for policy 0, policy_version 1056701 (0.0010) [2023-12-26 23:04:14,136][105620] Updated weights for policy 1, policy_version 1057714 (0.0008) [2023-12-26 23:04:14,184][105620] Updated weights for policy 1, policy_version 1057724 (0.0007) [2023-12-26 23:04:14,228][105620] Updated weights for policy 1, policy_version 1057734 (0.0008) [2023-12-26 23:04:14,697][105692] Updated weights for policy 0, policy_version 1056711 (0.0010) [2023-12-26 23:04:14,761][105692] Updated weights for policy 0, policy_version 1056721 (0.0010) [2023-12-26 23:04:14,830][105692] Updated weights for policy 0, policy_version 1056731 (0.0010) [2023-12-26 23:04:15,013][105620] Updated weights for policy 1, policy_version 1057744 (0.0008) [2023-12-26 23:04:15,066][105620] Updated weights for policy 1, policy_version 1057754 (0.0008) [2023-12-26 23:04:15,116][105620] Updated weights for policy 1, policy_version 1057764 (0.0008) [2023-12-26 23:04:15,573][105692] Updated weights for policy 0, policy_version 1056741 (0.0011) [2023-12-26 23:04:15,626][105692] Updated weights for policy 0, policy_version 1056751 (0.0007) [2023-12-26 23:04:15,687][105692] Updated weights for policy 0, policy_version 1056761 (0.0006) [2023-12-26 23:04:15,949][105620] Updated weights for policy 1, policy_version 1057774 (0.0008) [2023-12-26 23:04:16,005][105620] Updated weights for policy 1, policy_version 1057784 (0.0008) [2023-12-26 23:04:16,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 541392896. Throughput: 0: 9775.8, 1: 9730.8. Samples: 541367244. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:04:16,062][105620] Updated weights for policy 1, policy_version 1057794 (0.0009) [2023-12-26 23:04:16,063][104569] Avg episode reward: [(0, '8462.900'), (1, '8849.016')] [2023-12-26 23:04:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001056768_270573568.pth... [2023-12-26 23:04:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001055648_270286848.pth [2023-12-26 23:04:16,113][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001057800_270827520.pth... [2023-12-26 23:04:16,117][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001056680_270540800.pth [2023-12-26 23:04:16,299][105692] Updated weights for policy 0, policy_version 1056771 (0.0007) [2023-12-26 23:04:16,361][105692] Updated weights for policy 0, policy_version 1056781 (0.0009) [2023-12-26 23:04:16,412][105692] Updated weights for policy 0, policy_version 1056791 (0.0009) [2023-12-26 23:04:16,880][105620] Updated weights for policy 1, policy_version 1057805 (0.0009) [2023-12-26 23:04:16,938][105620] Updated weights for policy 1, policy_version 1057815 (0.0009) [2023-12-26 23:04:17,004][105620] Updated weights for policy 1, policy_version 1057825 (0.0010) [2023-12-26 23:04:17,021][105692] Updated weights for policy 0, policy_version 1056801 (0.0006) [2023-12-26 23:04:17,071][105692] Updated weights for policy 0, policy_version 1056811 (0.0005) [2023-12-26 23:04:17,120][105692] Updated weights for policy 0, policy_version 1056821 (0.0005) [2023-12-26 23:04:17,165][105692] Updated weights for policy 0, policy_version 1056831 (0.0006) [2023-12-26 23:04:17,824][105692] Updated weights for policy 0, policy_version 1056841 (0.0008) [2023-12-26 23:04:17,825][105620] Updated weights for policy 1, policy_version 1057835 (0.0009) [2023-12-26 23:04:17,876][105620] Updated weights for policy 1, policy_version 1057845 (0.0008) [2023-12-26 23:04:17,881][105692] Updated weights for policy 0, policy_version 1056851 (0.0008) [2023-12-26 23:04:17,929][105692] Updated weights for policy 0, policy_version 1056861 (0.0010) [2023-12-26 23:04:17,931][105620] Updated weights for policy 1, policy_version 1057855 (0.0005) [2023-12-26 23:04:18,583][105692] Updated weights for policy 0, policy_version 1056871 (0.0011) [2023-12-26 23:04:18,645][105692] Updated weights for policy 0, policy_version 1056881 (0.0011) [2023-12-26 23:04:18,706][105692] Updated weights for policy 0, policy_version 1056891 (0.0011) [2023-12-26 23:04:18,745][105620] Updated weights for policy 1, policy_version 1057865 (0.0008) [2023-12-26 23:04:18,802][105620] Updated weights for policy 1, policy_version 1057875 (0.0009) [2023-12-26 23:04:18,867][105620] Updated weights for policy 1, policy_version 1057885 (0.0008) [2023-12-26 23:04:18,927][105620] Updated weights for policy 1, policy_version 1057895 (0.0008) [2023-12-26 23:04:19,420][105692] Updated weights for policy 0, policy_version 1056901 (0.0008) [2023-12-26 23:04:19,479][105692] Updated weights for policy 0, policy_version 1056911 (0.0006) [2023-12-26 23:04:19,539][105692] Updated weights for policy 0, policy_version 1056921 (0.0008) [2023-12-26 23:04:19,759][105620] Updated weights for policy 1, policy_version 1057905 (0.0007) [2023-12-26 23:04:19,823][105620] Updated weights for policy 1, policy_version 1057915 (0.0008) [2023-12-26 23:04:19,874][105620] Updated weights for policy 1, policy_version 1057925 (0.0006) [2023-12-26 23:04:20,237][105692] Updated weights for policy 0, policy_version 1056931 (0.0007) [2023-12-26 23:04:20,299][105692] Updated weights for policy 0, policy_version 1056941 (0.0006) [2023-12-26 23:04:20,360][105692] Updated weights for policy 0, policy_version 1056951 (0.0006) [2023-12-26 23:04:20,620][105620] Updated weights for policy 1, policy_version 1057935 (0.0008) [2023-12-26 23:04:20,692][105620] Updated weights for policy 1, policy_version 1057945 (0.0009) [2023-12-26 23:04:20,759][105620] Updated weights for policy 1, policy_version 1057955 (0.0009) [2023-12-26 23:04:21,039][105692] Updated weights for policy 0, policy_version 1056961 (0.0006) [2023-12-26 23:04:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 541491200. Throughput: 0: 9752.0, 1: 9613.6. Samples: 541481192. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:04:21,063][104569] Avg episode reward: [(0, '8472.342'), (1, '9351.552')] [2023-12-26 23:04:21,107][105692] Updated weights for policy 0, policy_version 1056971 (0.0009) [2023-12-26 23:04:21,174][105692] Updated weights for policy 0, policy_version 1056981 (0.0009) [2023-12-26 23:04:21,237][105692] Updated weights for policy 0, policy_version 1056991 (0.0009) [2023-12-26 23:04:21,502][105620] Updated weights for policy 1, policy_version 1057965 (0.0007) [2023-12-26 23:04:21,562][105620] Updated weights for policy 1, policy_version 1057975 (0.0006) [2023-12-26 23:04:21,624][105620] Updated weights for policy 1, policy_version 1057985 (0.0007) [2023-12-26 23:04:21,966][105692] Updated weights for policy 0, policy_version 1057001 (0.0006) [2023-12-26 23:04:22,036][105692] Updated weights for policy 0, policy_version 1057011 (0.0006) [2023-12-26 23:04:22,090][105692] Updated weights for policy 0, policy_version 1057021 (0.0006) [2023-12-26 23:04:22,443][105620] Updated weights for policy 1, policy_version 1057995 (0.0010) [2023-12-26 23:04:22,508][105620] Updated weights for policy 1, policy_version 1058006 (0.0009) [2023-12-26 23:04:22,584][105620] Updated weights for policy 1, policy_version 1058016 (0.0011) [2023-12-26 23:04:22,692][105692] Updated weights for policy 0, policy_version 1057031 (0.0006) [2023-12-26 23:04:22,748][105692] Updated weights for policy 0, policy_version 1057041 (0.0009) [2023-12-26 23:04:22,818][105692] Updated weights for policy 0, policy_version 1057051 (0.0009) [2023-12-26 23:04:23,324][105620] Updated weights for policy 1, policy_version 1058026 (0.0009) [2023-12-26 23:04:23,375][105620] Updated weights for policy 1, policy_version 1058036 (0.0009) [2023-12-26 23:04:23,433][105620] Updated weights for policy 1, policy_version 1058046 (0.0010) [2023-12-26 23:04:23,491][105620] Updated weights for policy 1, policy_version 1058056 (0.0010) [2023-12-26 23:04:23,583][105692] Updated weights for policy 0, policy_version 1057061 (0.0008) [2023-12-26 23:04:23,648][105692] Updated weights for policy 0, policy_version 1057071 (0.0010) [2023-12-26 23:04:23,703][105692] Updated weights for policy 0, policy_version 1057081 (0.0010) [2023-12-26 23:04:24,111][105620] Updated weights for policy 1, policy_version 1058066 (0.0011) [2023-12-26 23:04:24,177][105620] Updated weights for policy 1, policy_version 1058076 (0.0010) [2023-12-26 23:04:24,226][105620] Updated weights for policy 1, policy_version 1058086 (0.0010) [2023-12-26 23:04:24,379][105692] Updated weights for policy 0, policy_version 1057091 (0.0009) [2023-12-26 23:04:24,446][105692] Updated weights for policy 0, policy_version 1057101 (0.0006) [2023-12-26 23:04:24,506][105692] Updated weights for policy 0, policy_version 1057111 (0.0010) [2023-12-26 23:04:24,904][105620] Updated weights for policy 1, policy_version 1058096 (0.0010) [2023-12-26 23:04:24,961][105620] Updated weights for policy 1, policy_version 1058106 (0.0010) [2023-12-26 23:04:25,016][105620] Updated weights for policy 1, policy_version 1058116 (0.0010) [2023-12-26 23:04:25,099][105692] Updated weights for policy 0, policy_version 1057121 (0.0010) [2023-12-26 23:04:25,143][105692] Updated weights for policy 0, policy_version 1057131 (0.0010) [2023-12-26 23:04:25,195][105692] Updated weights for policy 0, policy_version 1057141 (0.0010) [2023-12-26 23:04:25,250][105692] Updated weights for policy 0, policy_version 1057151 (0.0010) [2023-12-26 23:04:25,716][105620] Updated weights for policy 1, policy_version 1058126 (0.0010) [2023-12-26 23:04:25,768][105620] Updated weights for policy 1, policy_version 1058136 (0.0010) [2023-12-26 23:04:25,832][105620] Updated weights for policy 1, policy_version 1058146 (0.0010) [2023-12-26 23:04:25,875][105692] Updated weights for policy 0, policy_version 1057161 (0.0006) [2023-12-26 23:04:25,926][105692] Updated weights for policy 0, policy_version 1057171 (0.0005) [2023-12-26 23:04:25,972][105692] Updated weights for policy 0, policy_version 1057181 (0.0007) [2023-12-26 23:04:26,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 541597696. Throughput: 0: 9792.4, 1: 9533.2. Samples: 541599356. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:04:26,063][104569] Avg episode reward: [(0, '8730.916'), (1, '9258.616')] [2023-12-26 23:04:26,531][105620] Updated weights for policy 1, policy_version 1058156 (0.0008) [2023-12-26 23:04:26,598][105620] Updated weights for policy 1, policy_version 1058166 (0.0006) [2023-12-26 23:04:26,664][105620] Updated weights for policy 1, policy_version 1058176 (0.0005) [2023-12-26 23:04:26,690][105692] Updated weights for policy 0, policy_version 1057191 (0.0010) [2023-12-26 23:04:26,743][105692] Updated weights for policy 0, policy_version 1057201 (0.0008) [2023-12-26 23:04:26,798][105692] Updated weights for policy 0, policy_version 1057211 (0.0010) [2023-12-26 23:04:27,259][105620] Updated weights for policy 1, policy_version 1058186 (0.0006) [2023-12-26 23:04:27,321][105620] Updated weights for policy 1, policy_version 1058196 (0.0010) [2023-12-26 23:04:27,372][105620] Updated weights for policy 1, policy_version 1058206 (0.0010) [2023-12-26 23:04:27,415][105620] Updated weights for policy 1, policy_version 1058216 (0.0010) [2023-12-26 23:04:27,485][105692] Updated weights for policy 0, policy_version 1057221 (0.0010) [2023-12-26 23:04:27,533][105692] Updated weights for policy 0, policy_version 1057231 (0.0010) [2023-12-26 23:04:27,587][105692] Updated weights for policy 0, policy_version 1057241 (0.0010) [2023-12-26 23:04:28,074][105620] Updated weights for policy 1, policy_version 1058226 (0.0010) [2023-12-26 23:04:28,135][105620] Updated weights for policy 1, policy_version 1058236 (0.0010) [2023-12-26 23:04:28,193][105620] Updated weights for policy 1, policy_version 1058246 (0.0010) [2023-12-26 23:04:28,326][105692] Updated weights for policy 0, policy_version 1057251 (0.0011) [2023-12-26 23:04:28,390][105692] Updated weights for policy 0, policy_version 1057261 (0.0011) [2023-12-26 23:04:28,449][105692] Updated weights for policy 0, policy_version 1057271 (0.0011) [2023-12-26 23:04:28,934][105620] Updated weights for policy 1, policy_version 1058256 (0.0010) [2023-12-26 23:04:28,991][105620] Updated weights for policy 1, policy_version 1058266 (0.0010) [2023-12-26 23:04:29,045][105620] Updated weights for policy 1, policy_version 1058276 (0.0010) [2023-12-26 23:04:29,115][105692] Updated weights for policy 0, policy_version 1057281 (0.0011) [2023-12-26 23:04:29,182][105692] Updated weights for policy 0, policy_version 1057291 (0.0009) [2023-12-26 23:04:29,242][105692] Updated weights for policy 0, policy_version 1057301 (0.0008) [2023-12-26 23:04:29,304][105692] Updated weights for policy 0, policy_version 1057311 (0.0006) [2023-12-26 23:04:29,818][105620] Updated weights for policy 1, policy_version 1058286 (0.0010) [2023-12-26 23:04:29,883][105620] Updated weights for policy 1, policy_version 1058296 (0.0010) [2023-12-26 23:04:29,935][105620] Updated weights for policy 1, policy_version 1058306 (0.0009) [2023-12-26 23:04:30,025][105692] Updated weights for policy 0, policy_version 1057322 (0.0010) [2023-12-26 23:04:30,083][105692] Updated weights for policy 0, policy_version 1057332 (0.0009) [2023-12-26 23:04:30,146][105692] Updated weights for policy 0, policy_version 1057342 (0.0006) [2023-12-26 23:04:30,610][105620] Updated weights for policy 1, policy_version 1058316 (0.0011) [2023-12-26 23:04:30,661][105620] Updated weights for policy 1, policy_version 1058326 (0.0010) [2023-12-26 23:04:30,711][105620] Updated weights for policy 1, policy_version 1058336 (0.0009) [2023-12-26 23:04:30,797][105692] Updated weights for policy 0, policy_version 1057352 (0.0008) [2023-12-26 23:04:30,851][105692] Updated weights for policy 0, policy_version 1057362 (0.0009) [2023-12-26 23:04:30,908][105692] Updated weights for policy 0, policy_version 1057373 (0.0011) [2023-12-26 23:04:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 541696000. Throughput: 0: 9867.7, 1: 9550.0. Samples: 541660892. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:04:31,062][104569] Avg episode reward: [(0, '8653.968'), (1, '9257.858')] [2023-12-26 23:04:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001057376_270729216.pth... [2023-12-26 23:04:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001058344_270966784.pth... [2023-12-26 23:04:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001056224_270434304.pth [2023-12-26 23:04:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001057224_270680064.pth [2023-12-26 23:04:31,303][105620] Updated weights for policy 1, policy_version 1058346 (0.0007) [2023-12-26 23:04:31,365][105620] Updated weights for policy 1, policy_version 1058356 (0.0008) [2023-12-26 23:04:31,428][105620] Updated weights for policy 1, policy_version 1058366 (0.0011) [2023-12-26 23:04:31,493][105620] Updated weights for policy 1, policy_version 1058376 (0.0010) [2023-12-26 23:04:31,776][105692] Updated weights for policy 0, policy_version 1057383 (0.0009) [2023-12-26 23:04:31,837][105692] Updated weights for policy 0, policy_version 1057393 (0.0007) [2023-12-26 23:04:31,893][105692] Updated weights for policy 0, policy_version 1057403 (0.0009) [2023-12-26 23:04:32,243][105620] Updated weights for policy 1, policy_version 1058386 (0.0009) [2023-12-26 23:04:32,306][105620] Updated weights for policy 1, policy_version 1058396 (0.0010) [2023-12-26 23:04:32,363][105620] Updated weights for policy 1, policy_version 1058406 (0.0008) [2023-12-26 23:04:32,649][105692] Updated weights for policy 0, policy_version 1057414 (0.0009) [2023-12-26 23:04:32,710][105692] Updated weights for policy 0, policy_version 1057424 (0.0006) [2023-12-26 23:04:32,766][105692] Updated weights for policy 0, policy_version 1057434 (0.0005) [2023-12-26 23:04:33,138][105620] Updated weights for policy 1, policy_version 1058416 (0.0009) [2023-12-26 23:04:33,186][105620] Updated weights for policy 1, policy_version 1058426 (0.0008) [2023-12-26 23:04:33,233][105620] Updated weights for policy 1, policy_version 1058436 (0.0009) [2023-12-26 23:04:33,438][105692] Updated weights for policy 0, policy_version 1057444 (0.0007) [2023-12-26 23:04:33,486][105692] Updated weights for policy 0, policy_version 1057454 (0.0009) [2023-12-26 23:04:33,540][105692] Updated weights for policy 0, policy_version 1057464 (0.0008) [2023-12-26 23:04:34,022][105620] Updated weights for policy 1, policy_version 1058446 (0.0009) [2023-12-26 23:04:34,077][105620] Updated weights for policy 1, policy_version 1058457 (0.0011) [2023-12-26 23:04:34,130][105620] Updated weights for policy 1, policy_version 1058467 (0.0006) [2023-12-26 23:04:34,183][105692] Updated weights for policy 0, policy_version 1057474 (0.0008) [2023-12-26 23:04:34,244][105692] Updated weights for policy 0, policy_version 1057484 (0.0007) [2023-12-26 23:04:34,310][105692] Updated weights for policy 0, policy_version 1057494 (0.0009) [2023-12-26 23:04:34,366][105692] Updated weights for policy 0, policy_version 1057504 (0.0009) [2023-12-26 23:04:34,914][105620] Updated weights for policy 1, policy_version 1058477 (0.0008) [2023-12-26 23:04:34,965][105620] Updated weights for policy 1, policy_version 1058487 (0.0008) [2023-12-26 23:04:35,030][105620] Updated weights for policy 1, policy_version 1058497 (0.0008) [2023-12-26 23:04:35,120][105692] Updated weights for policy 0, policy_version 1057514 (0.0010) [2023-12-26 23:04:35,171][105692] Updated weights for policy 0, policy_version 1057524 (0.0010) [2023-12-26 23:04:35,218][105692] Updated weights for policy 0, policy_version 1057534 (0.0010) [2023-12-26 23:04:35,715][105620] Updated weights for policy 1, policy_version 1058507 (0.0008) [2023-12-26 23:04:35,771][105620] Updated weights for policy 1, policy_version 1058517 (0.0010) [2023-12-26 23:04:35,830][105620] Updated weights for policy 1, policy_version 1058527 (0.0011) [2023-12-26 23:04:35,954][105692] Updated weights for policy 0, policy_version 1057544 (0.0010) [2023-12-26 23:04:36,002][105692] Updated weights for policy 0, policy_version 1057554 (0.0010) [2023-12-26 23:04:36,052][105692] Updated weights for policy 0, policy_version 1057564 (0.0010) [2023-12-26 23:04:36,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 541786112. Throughput: 0: 9723.0, 1: 9506.0. Samples: 541776852. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:04:36,062][104569] Avg episode reward: [(0, '8740.093'), (1, '9349.541')] [2023-12-26 23:04:36,527][105620] Updated weights for policy 1, policy_version 1058537 (0.0010) [2023-12-26 23:04:36,591][105620] Updated weights for policy 1, policy_version 1058547 (0.0011) [2023-12-26 23:04:36,654][105620] Updated weights for policy 1, policy_version 1058557 (0.0011) [2023-12-26 23:04:36,713][105620] Updated weights for policy 1, policy_version 1058567 (0.0010) [2023-12-26 23:04:36,839][105692] Updated weights for policy 0, policy_version 1057574 (0.0009) [2023-12-26 23:04:36,893][105692] Updated weights for policy 0, policy_version 1057584 (0.0007) [2023-12-26 23:04:36,942][105692] Updated weights for policy 0, policy_version 1057594 (0.0008) [2023-12-26 23:04:37,453][105620] Updated weights for policy 1, policy_version 1058577 (0.0011) [2023-12-26 23:04:37,503][105620] Updated weights for policy 1, policy_version 1058587 (0.0011) [2023-12-26 23:04:37,558][105620] Updated weights for policy 1, policy_version 1058597 (0.0010) [2023-12-26 23:04:37,648][105692] Updated weights for policy 0, policy_version 1057604 (0.0009) [2023-12-26 23:04:37,707][105692] Updated weights for policy 0, policy_version 1057614 (0.0010) [2023-12-26 23:04:37,772][105692] Updated weights for policy 0, policy_version 1057624 (0.0010) [2023-12-26 23:04:38,231][105620] Updated weights for policy 1, policy_version 1058607 (0.0005) [2023-12-26 23:04:38,288][105620] Updated weights for policy 1, policy_version 1058617 (0.0006) [2023-12-26 23:04:38,354][105620] Updated weights for policy 1, policy_version 1058627 (0.0006) [2023-12-26 23:04:38,471][105692] Updated weights for policy 0, policy_version 1057634 (0.0009) [2023-12-26 23:04:38,537][105692] Updated weights for policy 0, policy_version 1057644 (0.0010) [2023-12-26 23:04:38,592][105692] Updated weights for policy 0, policy_version 1057654 (0.0010) [2023-12-26 23:04:38,638][105692] Updated weights for policy 0, policy_version 1057664 (0.0009) [2023-12-26 23:04:38,962][105620] Updated weights for policy 1, policy_version 1058637 (0.0006) [2023-12-26 23:04:39,023][105620] Updated weights for policy 1, policy_version 1058647 (0.0005) [2023-12-26 23:04:39,081][105620] Updated weights for policy 1, policy_version 1058657 (0.0008) [2023-12-26 23:04:39,406][105692] Updated weights for policy 0, policy_version 1057674 (0.0009) [2023-12-26 23:04:39,468][105692] Updated weights for policy 0, policy_version 1057684 (0.0009) [2023-12-26 23:04:39,524][105692] Updated weights for policy 0, policy_version 1057694 (0.0009) [2023-12-26 23:04:39,857][105620] Updated weights for policy 1, policy_version 1058667 (0.0009) [2023-12-26 23:04:39,931][105620] Updated weights for policy 1, policy_version 1058677 (0.0009) [2023-12-26 23:04:40,000][105620] Updated weights for policy 1, policy_version 1058687 (0.0007) [2023-12-26 23:04:40,275][105692] Updated weights for policy 0, policy_version 1057704 (0.0007) [2023-12-26 23:04:40,333][105692] Updated weights for policy 0, policy_version 1057714 (0.0009) [2023-12-26 23:04:40,390][105692] Updated weights for policy 0, policy_version 1057724 (0.0009) [2023-12-26 23:04:40,687][105620] Updated weights for policy 1, policy_version 1058697 (0.0007) [2023-12-26 23:04:40,732][105620] Updated weights for policy 1, policy_version 1058707 (0.0005) [2023-12-26 23:04:40,790][105620] Updated weights for policy 1, policy_version 1058717 (0.0009) [2023-12-26 23:04:40,857][105620] Updated weights for policy 1, policy_version 1058727 (0.0006) [2023-12-26 23:04:41,030][105692] Updated weights for policy 0, policy_version 1057734 (0.0009) [2023-12-26 23:04:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 541884416. Throughput: 0: 9726.5, 1: 9497.4. Samples: 541893176. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:04:41,063][104569] Avg episode reward: [(0, '8908.834'), (1, '9259.856')] [2023-12-26 23:04:41,102][105692] Updated weights for policy 0, policy_version 1057744 (0.0007) [2023-12-26 23:04:41,173][105692] Updated weights for policy 0, policy_version 1057754 (0.0008) [2023-12-26 23:04:41,635][105620] Updated weights for policy 1, policy_version 1058737 (0.0010) [2023-12-26 23:04:41,694][105620] Updated weights for policy 1, policy_version 1058747 (0.0008) [2023-12-26 23:04:41,753][105620] Updated weights for policy 1, policy_version 1058757 (0.0009) [2023-12-26 23:04:41,931][105692] Updated weights for policy 0, policy_version 1057764 (0.0009) [2023-12-26 23:04:41,985][105692] Updated weights for policy 0, policy_version 1057774 (0.0009) [2023-12-26 23:04:42,039][105692] Updated weights for policy 0, policy_version 1057784 (0.0009) [2023-12-26 23:04:42,514][105620] Updated weights for policy 1, policy_version 1058767 (0.0006) [2023-12-26 23:04:42,561][105620] Updated weights for policy 1, policy_version 1058777 (0.0005) [2023-12-26 23:04:42,617][105620] Updated weights for policy 1, policy_version 1058787 (0.0006) [2023-12-26 23:04:42,856][105692] Updated weights for policy 0, policy_version 1057794 (0.0009) [2023-12-26 23:04:42,911][105692] Updated weights for policy 0, policy_version 1057804 (0.0006) [2023-12-26 23:04:42,969][105692] Updated weights for policy 0, policy_version 1057814 (0.0006) [2023-12-26 23:04:42,971][105585] KL-divergence is very high: 137.8331 [2023-12-26 23:04:43,015][105585] KL-divergence is very high: 116.5119 [2023-12-26 23:04:43,027][105692] Updated weights for policy 0, policy_version 1057824 (0.0006) [2023-12-26 23:04:43,240][105620] Updated weights for policy 1, policy_version 1058797 (0.0006) [2023-12-26 23:04:43,290][105620] Updated weights for policy 1, policy_version 1058807 (0.0008) [2023-12-26 23:04:43,355][105620] Updated weights for policy 1, policy_version 1058817 (0.0005) [2023-12-26 23:04:43,567][105692] Updated weights for policy 0, policy_version 1057834 (0.0006) [2023-12-26 23:04:43,631][105692] Updated weights for policy 0, policy_version 1057844 (0.0007) [2023-12-26 23:04:43,691][105692] Updated weights for policy 0, policy_version 1057854 (0.0006) [2023-12-26 23:04:43,957][105620] Updated weights for policy 1, policy_version 1058827 (0.0006) [2023-12-26 23:04:44,013][105620] Updated weights for policy 1, policy_version 1058837 (0.0005) [2023-12-26 23:04:44,071][105620] Updated weights for policy 1, policy_version 1058847 (0.0005) [2023-12-26 23:04:44,425][105692] Updated weights for policy 0, policy_version 1057864 (0.0007) [2023-12-26 23:04:44,489][105692] Updated weights for policy 0, policy_version 1057874 (0.0009) [2023-12-26 23:04:44,546][105692] Updated weights for policy 0, policy_version 1057884 (0.0010) [2023-12-26 23:04:44,602][105620] Updated weights for policy 1, policy_version 1058857 (0.0006) [2023-12-26 23:04:44,654][105620] Updated weights for policy 1, policy_version 1058867 (0.0009) [2023-12-26 23:04:44,708][105620] Updated weights for policy 1, policy_version 1058877 (0.0010) [2023-12-26 23:04:44,767][105620] Updated weights for policy 1, policy_version 1058887 (0.0009) [2023-12-26 23:04:45,219][105692] Updated weights for policy 0, policy_version 1057895 (0.0009) [2023-12-26 23:04:45,284][105692] Updated weights for policy 0, policy_version 1057905 (0.0007) [2023-12-26 23:04:45,352][105692] Updated weights for policy 0, policy_version 1057915 (0.0006) [2023-12-26 23:04:45,583][105620] Updated weights for policy 1, policy_version 1058897 (0.0008) [2023-12-26 23:04:45,644][105620] Updated weights for policy 1, policy_version 1058907 (0.0009) [2023-12-26 23:04:45,705][105620] Updated weights for policy 1, policy_version 1058917 (0.0009) [2023-12-26 23:04:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 541982720. Throughput: 0: 9694.1, 1: 9595.4. Samples: 541954272. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:04:46,063][104569] Avg episode reward: [(0, '9084.956'), (1, '9260.100')] [2023-12-26 23:04:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001058920_271114240.pth... [2023-12-26 23:04:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001057800_270827520.pth [2023-12-26 23:04:46,070][105692] Updated weights for policy 0, policy_version 1057925 (0.0009) [2023-12-26 23:04:46,128][105692] Updated weights for policy 0, policy_version 1057935 (0.0009) [2023-12-26 23:04:46,176][105692] Updated weights for policy 0, policy_version 1057945 (0.0009) [2023-12-26 23:04:46,209][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001057952_270876672.pth... [2023-12-26 23:04:46,214][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001056768_270573568.pth [2023-12-26 23:04:46,419][105620] Updated weights for policy 1, policy_version 1058927 (0.0009) [2023-12-26 23:04:46,466][105620] Updated weights for policy 1, policy_version 1058937 (0.0009) [2023-12-26 23:04:46,514][105620] Updated weights for policy 1, policy_version 1058947 (0.0009) [2023-12-26 23:04:46,898][105692] Updated weights for policy 0, policy_version 1057955 (0.0008) [2023-12-26 23:04:46,960][105692] Updated weights for policy 0, policy_version 1057965 (0.0009) [2023-12-26 23:04:47,009][105692] Updated weights for policy 0, policy_version 1057975 (0.0010) [2023-12-26 23:04:47,186][105620] Updated weights for policy 1, policy_version 1058957 (0.0007) [2023-12-26 23:04:47,238][105620] Updated weights for policy 1, policy_version 1058967 (0.0008) [2023-12-26 23:04:47,282][105620] Updated weights for policy 1, policy_version 1058977 (0.0008) [2023-12-26 23:04:47,659][105692] Updated weights for policy 0, policy_version 1057985 (0.0010) [2023-12-26 23:04:47,718][105692] Updated weights for policy 0, policy_version 1057995 (0.0006) [2023-12-26 23:04:47,763][105692] Updated weights for policy 0, policy_version 1058005 (0.0010) [2023-12-26 23:04:47,811][105692] Updated weights for policy 0, policy_version 1058015 (0.0010) [2023-12-26 23:04:48,051][105620] Updated weights for policy 1, policy_version 1058987 (0.0008) [2023-12-26 23:04:48,108][105620] Updated weights for policy 1, policy_version 1058997 (0.0009) [2023-12-26 23:04:48,170][105620] Updated weights for policy 1, policy_version 1059007 (0.0009) [2023-12-26 23:04:48,459][105692] Updated weights for policy 0, policy_version 1058025 (0.0010) [2023-12-26 23:04:48,508][105692] Updated weights for policy 0, policy_version 1058035 (0.0010) [2023-12-26 23:04:48,555][105692] Updated weights for policy 0, policy_version 1058045 (0.0007) [2023-12-26 23:04:48,991][105620] Updated weights for policy 1, policy_version 1059017 (0.0009) [2023-12-26 23:04:49,043][105620] Updated weights for policy 1, policy_version 1059027 (0.0008) [2023-12-26 23:04:49,106][105620] Updated weights for policy 1, policy_version 1059037 (0.0007) [2023-12-26 23:04:49,154][105620] Updated weights for policy 1, policy_version 1059047 (0.0008) [2023-12-26 23:04:49,339][105692] Updated weights for policy 0, policy_version 1058055 (0.0011) [2023-12-26 23:04:49,409][105692] Updated weights for policy 0, policy_version 1058065 (0.0008) [2023-12-26 23:04:49,461][105692] Updated weights for policy 0, policy_version 1058075 (0.0011) [2023-12-26 23:04:49,879][105620] Updated weights for policy 1, policy_version 1059057 (0.0008) [2023-12-26 23:04:49,952][105620] Updated weights for policy 1, policy_version 1059067 (0.0007) [2023-12-26 23:04:50,014][105620] Updated weights for policy 1, policy_version 1059077 (0.0006) [2023-12-26 23:04:50,198][105692] Updated weights for policy 0, policy_version 1058085 (0.0010) [2023-12-26 23:04:50,263][105692] Updated weights for policy 0, policy_version 1058095 (0.0010) [2023-12-26 23:04:50,315][105692] Updated weights for policy 0, policy_version 1058105 (0.0010) [2023-12-26 23:04:50,700][105620] Updated weights for policy 1, policy_version 1059087 (0.0009) [2023-12-26 23:04:50,758][105620] Updated weights for policy 1, policy_version 1059097 (0.0010) [2023-12-26 23:04:50,811][105620] Updated weights for policy 1, policy_version 1059107 (0.0009) [2023-12-26 23:04:50,956][105692] Updated weights for policy 0, policy_version 1058115 (0.0009) [2023-12-26 23:04:51,017][105692] Updated weights for policy 0, policy_version 1058125 (0.0005) [2023-12-26 23:04:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 542081024. Throughput: 0: 9791.4, 1: 9595.8. Samples: 542072028. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:04:51,063][104569] Avg episode reward: [(0, '9085.497'), (1, '9167.692')] [2023-12-26 23:04:51,083][105692] Updated weights for policy 0, policy_version 1058135 (0.0008) [2023-12-26 23:04:51,571][105620] Updated weights for policy 1, policy_version 1059117 (0.0008) [2023-12-26 23:04:51,630][105620] Updated weights for policy 1, policy_version 1059127 (0.0007) [2023-12-26 23:04:51,699][105620] Updated weights for policy 1, policy_version 1059137 (0.0007) [2023-12-26 23:04:51,762][105692] Updated weights for policy 0, policy_version 1058145 (0.0009) [2023-12-26 23:04:51,827][105692] Updated weights for policy 0, policy_version 1058155 (0.0009) [2023-12-26 23:04:51,891][105692] Updated weights for policy 0, policy_version 1058165 (0.0008) [2023-12-26 23:04:51,946][105692] Updated weights for policy 0, policy_version 1058175 (0.0005) [2023-12-26 23:04:52,473][105620] Updated weights for policy 1, policy_version 1059147 (0.0008) [2023-12-26 23:04:52,538][105620] Updated weights for policy 1, policy_version 1059157 (0.0007) [2023-12-26 23:04:52,602][105620] Updated weights for policy 1, policy_version 1059167 (0.0006) [2023-12-26 23:04:52,614][105692] Updated weights for policy 0, policy_version 1058185 (0.0008) [2023-12-26 23:04:52,678][105692] Updated weights for policy 0, policy_version 1058195 (0.0009) [2023-12-26 23:04:52,739][105692] Updated weights for policy 0, policy_version 1058205 (0.0008) [2023-12-26 23:04:53,229][105620] Updated weights for policy 1, policy_version 1059177 (0.0006) [2023-12-26 23:04:53,288][105620] Updated weights for policy 1, policy_version 1059187 (0.0008) [2023-12-26 23:04:53,353][105620] Updated weights for policy 1, policy_version 1059197 (0.0009) [2023-12-26 23:04:53,409][105620] Updated weights for policy 1, policy_version 1059207 (0.0009) [2023-12-26 23:04:53,452][105692] Updated weights for policy 0, policy_version 1058215 (0.0005) [2023-12-26 23:04:53,510][105692] Updated weights for policy 0, policy_version 1058225 (0.0005) [2023-12-26 23:04:53,563][105692] Updated weights for policy 0, policy_version 1058235 (0.0006) [2023-12-26 23:04:54,138][105692] Updated weights for policy 0, policy_version 1058245 (0.0009) [2023-12-26 23:04:54,199][105692] Updated weights for policy 0, policy_version 1058255 (0.0008) [2023-12-26 23:04:54,230][105620] Updated weights for policy 1, policy_version 1059217 (0.0006) [2023-12-26 23:04:54,263][105692] Updated weights for policy 0, policy_version 1058265 (0.0008) [2023-12-26 23:04:54,286][105620] Updated weights for policy 1, policy_version 1059227 (0.0006) [2023-12-26 23:04:54,341][105620] Updated weights for policy 1, policy_version 1059237 (0.0008) [2023-12-26 23:04:54,923][105692] Updated weights for policy 0, policy_version 1058275 (0.0009) [2023-12-26 23:04:54,976][105692] Updated weights for policy 0, policy_version 1058285 (0.0009) [2023-12-26 23:04:55,028][105692] Updated weights for policy 0, policy_version 1058295 (0.0009) [2023-12-26 23:04:55,141][105620] Updated weights for policy 1, policy_version 1059247 (0.0008) [2023-12-26 23:04:55,204][105620] Updated weights for policy 1, policy_version 1059257 (0.0009) [2023-12-26 23:04:55,252][105620] Updated weights for policy 1, policy_version 1059267 (0.0009) [2023-12-26 23:04:55,798][105692] Updated weights for policy 0, policy_version 1058305 (0.0009) [2023-12-26 23:04:55,850][105692] Updated weights for policy 0, policy_version 1058315 (0.0010) [2023-12-26 23:04:55,900][105692] Updated weights for policy 0, policy_version 1058325 (0.0010) [2023-12-26 23:04:55,952][105692] Updated weights for policy 0, policy_version 1058335 (0.0010) [2023-12-26 23:04:55,983][105620] Updated weights for policy 1, policy_version 1059277 (0.0009) [2023-12-26 23:04:56,048][105620] Updated weights for policy 1, policy_version 1059287 (0.0007) [2023-12-26 23:04:56,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19251.1, 300 sec: 19549.7). Total num frames: 542179328. Throughput: 0: 9938.5, 1: 9571.9. Samples: 542189056. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:04:56,063][104569] Avg episode reward: [(0, '8995.994'), (1, '9076.967')] [2023-12-26 23:04:56,117][105620] Updated weights for policy 1, policy_version 1059297 (0.0005) [2023-12-26 23:04:56,679][105692] Updated weights for policy 0, policy_version 1058345 (0.0009) [2023-12-26 23:04:56,727][105692] Updated weights for policy 0, policy_version 1058355 (0.0009) [2023-12-26 23:04:56,773][105692] Updated weights for policy 0, policy_version 1058365 (0.0007) [2023-12-26 23:04:56,796][105620] Updated weights for policy 1, policy_version 1059307 (0.0008) [2023-12-26 23:04:56,845][105620] Updated weights for policy 1, policy_version 1059317 (0.0005) [2023-12-26 23:04:56,891][105620] Updated weights for policy 1, policy_version 1059327 (0.0005) [2023-12-26 23:04:57,544][105620] Updated weights for policy 1, policy_version 1059337 (0.0005) [2023-12-26 23:04:57,561][105692] Updated weights for policy 0, policy_version 1058375 (0.0009) [2023-12-26 23:04:57,600][105620] Updated weights for policy 1, policy_version 1059347 (0.0006) [2023-12-26 23:04:57,622][105692] Updated weights for policy 0, policy_version 1058385 (0.0008) [2023-12-26 23:04:57,659][105620] Updated weights for policy 1, policy_version 1059357 (0.0007) [2023-12-26 23:04:57,677][105692] Updated weights for policy 0, policy_version 1058395 (0.0006) [2023-12-26 23:04:57,715][105620] Updated weights for policy 1, policy_version 1059367 (0.0008) [2023-12-26 23:04:58,383][105692] Updated weights for policy 0, policy_version 1058405 (0.0008) [2023-12-26 23:04:58,443][105692] Updated weights for policy 0, policy_version 1058415 (0.0010) [2023-12-26 23:04:58,502][105620] Updated weights for policy 1, policy_version 1059377 (0.0007) [2023-12-26 23:04:58,509][105692] Updated weights for policy 0, policy_version 1058425 (0.0009) [2023-12-26 23:04:58,561][105620] Updated weights for policy 1, policy_version 1059387 (0.0006) [2023-12-26 23:04:58,619][105620] Updated weights for policy 1, policy_version 1059397 (0.0009) [2023-12-26 23:04:59,346][105692] Updated weights for policy 0, policy_version 1058435 (0.0009) [2023-12-26 23:04:59,392][105620] Updated weights for policy 1, policy_version 1059407 (0.0008) [2023-12-26 23:04:59,406][105692] Updated weights for policy 0, policy_version 1058445 (0.0007) [2023-12-26 23:04:59,449][105620] Updated weights for policy 1, policy_version 1059417 (0.0008) [2023-12-26 23:04:59,460][105692] Updated weights for policy 0, policy_version 1058455 (0.0005) [2023-12-26 23:04:59,502][105620] Updated weights for policy 1, policy_version 1059427 (0.0010) [2023-12-26 23:05:00,202][105692] Updated weights for policy 0, policy_version 1058465 (0.0006) [2023-12-26 23:05:00,261][105692] Updated weights for policy 0, policy_version 1058475 (0.0008) [2023-12-26 23:05:00,267][105620] Updated weights for policy 1, policy_version 1059437 (0.0011) [2023-12-26 23:05:00,323][105692] Updated weights for policy 0, policy_version 1058485 (0.0005) [2023-12-26 23:05:00,325][105620] Updated weights for policy 1, policy_version 1059447 (0.0010) [2023-12-26 23:05:00,378][105692] Updated weights for policy 0, policy_version 1058495 (0.0006) [2023-12-26 23:05:00,387][105620] Updated weights for policy 1, policy_version 1059457 (0.0010) [2023-12-26 23:05:00,961][105620] Updated weights for policy 1, policy_version 1059467 (0.0009) [2023-12-26 23:05:01,021][105620] Updated weights for policy 1, policy_version 1059477 (0.0008) [2023-12-26 23:05:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 542269440. Throughput: 0: 9921.4, 1: 9617.2. Samples: 542246484. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:05:01,063][104569] Avg episode reward: [(0, '9086.713'), (1, '9260.146')] [2023-12-26 23:05:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001058496_271015936.pth... [2023-12-26 23:05:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001057376_270729216.pth [2023-12-26 23:05:01,087][105620] Updated weights for policy 1, policy_version 1059487 (0.0008) [2023-12-26 23:05:01,139][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001059496_271261696.pth... [2023-12-26 23:05:01,144][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001058344_270966784.pth [2023-12-26 23:05:01,202][105692] Updated weights for policy 0, policy_version 1058505 (0.0009) [2023-12-26 23:05:01,265][105692] Updated weights for policy 0, policy_version 1058515 (0.0009) [2023-12-26 23:05:01,321][105692] Updated weights for policy 0, policy_version 1058525 (0.0009) [2023-12-26 23:05:01,814][105620] Updated weights for policy 1, policy_version 1059497 (0.0009) [2023-12-26 23:05:01,876][105620] Updated weights for policy 1, policy_version 1059507 (0.0005) [2023-12-26 23:05:01,945][105620] Updated weights for policy 1, policy_version 1059517 (0.0008) [2023-12-26 23:05:02,005][105620] Updated weights for policy 1, policy_version 1059527 (0.0009) [2023-12-26 23:05:02,085][105692] Updated weights for policy 0, policy_version 1058535 (0.0009) [2023-12-26 23:05:02,149][105692] Updated weights for policy 0, policy_version 1058545 (0.0009) [2023-12-26 23:05:02,207][105692] Updated weights for policy 0, policy_version 1058555 (0.0009) [2023-12-26 23:05:02,731][105620] Updated weights for policy 1, policy_version 1059537 (0.0009) [2023-12-26 23:05:02,789][105620] Updated weights for policy 1, policy_version 1059547 (0.0010) [2023-12-26 23:05:02,853][105620] Updated weights for policy 1, policy_version 1059557 (0.0010) [2023-12-26 23:05:02,865][105692] Updated weights for policy 0, policy_version 1058565 (0.0007) [2023-12-26 23:05:02,917][105692] Updated weights for policy 0, policy_version 1058575 (0.0008) [2023-12-26 23:05:02,973][105692] Updated weights for policy 0, policy_version 1058585 (0.0008) [2023-12-26 23:05:03,640][105692] Updated weights for policy 0, policy_version 1058595 (0.0008) [2023-12-26 23:05:03,649][105620] Updated weights for policy 1, policy_version 1059567 (0.0009) [2023-12-26 23:05:03,697][105692] Updated weights for policy 0, policy_version 1058605 (0.0006) [2023-12-26 23:05:03,706][105620] Updated weights for policy 1, policy_version 1059577 (0.0008) [2023-12-26 23:05:03,756][105692] Updated weights for policy 0, policy_version 1058615 (0.0006) [2023-12-26 23:05:03,762][105620] Updated weights for policy 1, policy_version 1059587 (0.0007) [2023-12-26 23:05:04,370][105692] Updated weights for policy 0, policy_version 1058625 (0.0008) [2023-12-26 23:05:04,435][105692] Updated weights for policy 0, policy_version 1058635 (0.0008) [2023-12-26 23:05:04,493][105692] Updated weights for policy 0, policy_version 1058645 (0.0009) [2023-12-26 23:05:04,554][105692] Updated weights for policy 0, policy_version 1058655 (0.0008) [2023-12-26 23:05:04,584][105620] Updated weights for policy 1, policy_version 1059597 (0.0006) [2023-12-26 23:05:04,658][105620] Updated weights for policy 1, policy_version 1059607 (0.0006) [2023-12-26 23:05:04,711][105620] Updated weights for policy 1, policy_version 1059617 (0.0008) [2023-12-26 23:05:05,167][105692] Updated weights for policy 0, policy_version 1058665 (0.0005) [2023-12-26 23:05:05,222][105692] Updated weights for policy 0, policy_version 1058675 (0.0005) [2023-12-26 23:05:05,289][105692] Updated weights for policy 0, policy_version 1058685 (0.0005) [2023-12-26 23:05:05,344][105620] Updated weights for policy 1, policy_version 1059627 (0.0007) [2023-12-26 23:05:05,389][105620] Updated weights for policy 1, policy_version 1059637 (0.0010) [2023-12-26 23:05:05,445][105620] Updated weights for policy 1, policy_version 1059647 (0.0010) [2023-12-26 23:05:05,893][105692] Updated weights for policy 0, policy_version 1058695 (0.0007) [2023-12-26 23:05:05,959][105692] Updated weights for policy 0, policy_version 1058705 (0.0010) [2023-12-26 23:05:06,017][105692] Updated weights for policy 0, policy_version 1058715 (0.0010) [2023-12-26 23:05:06,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 542375936. Throughput: 0: 9828.7, 1: 9712.4. Samples: 542360540. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:05:06,062][104569] Avg episode reward: [(0, '8819.757'), (1, '9260.542')] [2023-12-26 23:05:06,184][105620] Updated weights for policy 1, policy_version 1059657 (0.0010) [2023-12-26 23:05:06,251][105620] Updated weights for policy 1, policy_version 1059667 (0.0011) [2023-12-26 23:05:06,312][105620] Updated weights for policy 1, policy_version 1059677 (0.0010) [2023-12-26 23:05:06,377][105620] Updated weights for policy 1, policy_version 1059687 (0.0010) [2023-12-26 23:05:06,714][105692] Updated weights for policy 0, policy_version 1058725 (0.0010) [2023-12-26 23:05:06,777][105692] Updated weights for policy 0, policy_version 1058735 (0.0011) [2023-12-26 23:05:06,844][105692] Updated weights for policy 0, policy_version 1058745 (0.0011) [2023-12-26 23:05:07,107][105620] Updated weights for policy 1, policy_version 1059697 (0.0009) [2023-12-26 23:05:07,175][105620] Updated weights for policy 1, policy_version 1059707 (0.0008) [2023-12-26 23:05:07,237][105620] Updated weights for policy 1, policy_version 1059717 (0.0009) [2023-12-26 23:05:07,556][105692] Updated weights for policy 0, policy_version 1058755 (0.0009) [2023-12-26 23:05:07,622][105692] Updated weights for policy 0, policy_version 1058765 (0.0005) [2023-12-26 23:05:07,670][105692] Updated weights for policy 0, policy_version 1058775 (0.0005) [2023-12-26 23:05:08,006][105620] Updated weights for policy 1, policy_version 1059727 (0.0010) [2023-12-26 23:05:08,063][105620] Updated weights for policy 1, policy_version 1059737 (0.0009) [2023-12-26 23:05:08,114][105620] Updated weights for policy 1, policy_version 1059747 (0.0009) [2023-12-26 23:05:08,262][105692] Updated weights for policy 0, policy_version 1058785 (0.0006) [2023-12-26 23:05:08,328][105692] Updated weights for policy 0, policy_version 1058795 (0.0009) [2023-12-26 23:05:08,390][105692] Updated weights for policy 0, policy_version 1058805 (0.0010) [2023-12-26 23:05:08,450][105692] Updated weights for policy 0, policy_version 1058815 (0.0009) [2023-12-26 23:05:08,836][105620] Updated weights for policy 1, policy_version 1059757 (0.0007) [2023-12-26 23:05:08,895][105620] Updated weights for policy 1, policy_version 1059767 (0.0005) [2023-12-26 23:05:08,946][105620] Updated weights for policy 1, policy_version 1059777 (0.0005) [2023-12-26 23:05:09,195][105692] Updated weights for policy 0, policy_version 1058825 (0.0009) [2023-12-26 23:05:09,253][105692] Updated weights for policy 0, policy_version 1058835 (0.0008) [2023-12-26 23:05:09,302][105692] Updated weights for policy 0, policy_version 1058845 (0.0007) [2023-12-26 23:05:09,605][105620] Updated weights for policy 1, policy_version 1059787 (0.0005) [2023-12-26 23:05:09,662][105620] Updated weights for policy 1, policy_version 1059797 (0.0006) [2023-12-26 23:05:09,716][105620] Updated weights for policy 1, policy_version 1059807 (0.0006) [2023-12-26 23:05:10,157][105692] Updated weights for policy 0, policy_version 1058855 (0.0008) [2023-12-26 23:05:10,225][105692] Updated weights for policy 0, policy_version 1058865 (0.0009) [2023-12-26 23:05:10,289][105692] Updated weights for policy 0, policy_version 1058875 (0.0010) [2023-12-26 23:05:10,403][105620] Updated weights for policy 1, policy_version 1059817 (0.0006) [2023-12-26 23:05:10,456][105620] Updated weights for policy 1, policy_version 1059827 (0.0010) [2023-12-26 23:05:10,519][105620] Updated weights for policy 1, policy_version 1059837 (0.0011) [2023-12-26 23:05:10,582][105620] Updated weights for policy 1, policy_version 1059847 (0.0009) [2023-12-26 23:05:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 542466048. Throughput: 0: 9804.7, 1: 9723.9. Samples: 542478144. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:05:11,063][104569] Avg episode reward: [(0, '8727.749'), (1, '9259.740')] [2023-12-26 23:05:11,092][105692] Updated weights for policy 0, policy_version 1058885 (0.0009) [2023-12-26 23:05:11,156][105692] Updated weights for policy 0, policy_version 1058895 (0.0008) [2023-12-26 23:05:11,217][105692] Updated weights for policy 0, policy_version 1058905 (0.0008) [2023-12-26 23:05:11,326][105620] Updated weights for policy 1, policy_version 1059857 (0.0009) [2023-12-26 23:05:11,401][105620] Updated weights for policy 1, policy_version 1059867 (0.0009) [2023-12-26 23:05:11,466][105620] Updated weights for policy 1, policy_version 1059877 (0.0009) [2023-12-26 23:05:11,937][105692] Updated weights for policy 0, policy_version 1058915 (0.0009) [2023-12-26 23:05:11,994][105692] Updated weights for policy 0, policy_version 1058925 (0.0011) [2023-12-26 23:05:12,050][105692] Updated weights for policy 0, policy_version 1058935 (0.0011) [2023-12-26 23:05:12,261][105620] Updated weights for policy 1, policy_version 1059887 (0.0009) [2023-12-26 23:05:12,321][105620] Updated weights for policy 1, policy_version 1059897 (0.0009) [2023-12-26 23:05:12,390][105620] Updated weights for policy 1, policy_version 1059907 (0.0009) [2023-12-26 23:05:12,805][105692] Updated weights for policy 0, policy_version 1058945 (0.0010) [2023-12-26 23:05:12,862][105692] Updated weights for policy 0, policy_version 1058955 (0.0007) [2023-12-26 23:05:12,927][105692] Updated weights for policy 0, policy_version 1058965 (0.0010) [2023-12-26 23:05:12,982][105692] Updated weights for policy 0, policy_version 1058975 (0.0010) [2023-12-26 23:05:13,116][105620] Updated weights for policy 1, policy_version 1059917 (0.0010) [2023-12-26 23:05:13,160][105620] Updated weights for policy 1, policy_version 1059927 (0.0010) [2023-12-26 23:05:13,209][105620] Updated weights for policy 1, policy_version 1059937 (0.0008) [2023-12-26 23:05:13,664][105692] Updated weights for policy 0, policy_version 1058985 (0.0008) [2023-12-26 23:05:13,729][105692] Updated weights for policy 0, policy_version 1058995 (0.0008) [2023-12-26 23:05:13,796][105692] Updated weights for policy 0, policy_version 1059005 (0.0008) [2023-12-26 23:05:13,940][105620] Updated weights for policy 1, policy_version 1059947 (0.0006) [2023-12-26 23:05:13,998][105620] Updated weights for policy 1, policy_version 1059957 (0.0010) [2023-12-26 23:05:14,056][105620] Updated weights for policy 1, policy_version 1059967 (0.0010) [2023-12-26 23:05:14,546][105692] Updated weights for policy 0, policy_version 1059015 (0.0008) [2023-12-26 23:05:14,599][105692] Updated weights for policy 0, policy_version 1059025 (0.0008) [2023-12-26 23:05:14,667][105692] Updated weights for policy 0, policy_version 1059035 (0.0009) [2023-12-26 23:05:14,761][105620] Updated weights for policy 1, policy_version 1059977 (0.0010) [2023-12-26 23:05:14,825][105620] Updated weights for policy 1, policy_version 1059987 (0.0008) [2023-12-26 23:05:14,881][105620] Updated weights for policy 1, policy_version 1059997 (0.0006) [2023-12-26 23:05:14,951][105620] Updated weights for policy 1, policy_version 1060007 (0.0006) [2023-12-26 23:05:15,439][105692] Updated weights for policy 0, policy_version 1059045 (0.0008) [2023-12-26 23:05:15,503][105692] Updated weights for policy 0, policy_version 1059055 (0.0006) [2023-12-26 23:05:15,558][105620] Updated weights for policy 1, policy_version 1060017 (0.0005) [2023-12-26 23:05:15,560][105692] Updated weights for policy 0, policy_version 1059065 (0.0008) [2023-12-26 23:05:15,614][105620] Updated weights for policy 1, policy_version 1060027 (0.0005) [2023-12-26 23:05:15,663][105620] Updated weights for policy 1, policy_version 1060037 (0.0005) [2023-12-26 23:05:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 542564352. Throughput: 0: 9762.2, 1: 9643.8. Samples: 542534164. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:05:16,063][104569] Avg episode reward: [(0, '8991.896'), (1, '9256.269')] [2023-12-26 23:05:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001059072_271163392.pth... [2023-12-26 23:05:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001060040_271400960.pth... [2023-12-26 23:05:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001058920_271114240.pth [2023-12-26 23:05:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001057952_270876672.pth [2023-12-26 23:05:16,229][105620] Updated weights for policy 1, policy_version 1060047 (0.0008) [2023-12-26 23:05:16,283][105620] Updated weights for policy 1, policy_version 1060057 (0.0009) [2023-12-26 23:05:16,341][105692] Updated weights for policy 0, policy_version 1059075 (0.0008) [2023-12-26 23:05:16,346][105620] Updated weights for policy 1, policy_version 1060067 (0.0008) [2023-12-26 23:05:16,386][105692] Updated weights for policy 0, policy_version 1059085 (0.0006) [2023-12-26 23:05:16,429][105692] Updated weights for policy 0, policy_version 1059095 (0.0007) [2023-12-26 23:05:17,056][105620] Updated weights for policy 1, policy_version 1060077 (0.0009) [2023-12-26 23:05:17,110][105620] Updated weights for policy 1, policy_version 1060087 (0.0009) [2023-12-26 23:05:17,172][105620] Updated weights for policy 1, policy_version 1060097 (0.0009) [2023-12-26 23:05:17,222][105692] Updated weights for policy 0, policy_version 1059105 (0.0009) [2023-12-26 23:05:17,275][105692] Updated weights for policy 0, policy_version 1059115 (0.0009) [2023-12-26 23:05:17,325][105692] Updated weights for policy 0, policy_version 1059125 (0.0009) [2023-12-26 23:05:17,386][105692] Updated weights for policy 0, policy_version 1059135 (0.0009) [2023-12-26 23:05:17,951][105620] Updated weights for policy 1, policy_version 1060107 (0.0008) [2023-12-26 23:05:18,014][105620] Updated weights for policy 1, policy_version 1060117 (0.0009) [2023-12-26 23:05:18,076][105620] Updated weights for policy 1, policy_version 1060127 (0.0008) [2023-12-26 23:05:18,098][105692] Updated weights for policy 0, policy_version 1059145 (0.0008) [2023-12-26 23:05:18,157][105692] Updated weights for policy 0, policy_version 1059155 (0.0008) [2023-12-26 23:05:18,203][105692] Updated weights for policy 0, policy_version 1059165 (0.0008) [2023-12-26 23:05:18,836][105620] Updated weights for policy 1, policy_version 1060137 (0.0007) [2023-12-26 23:05:18,895][105620] Updated weights for policy 1, policy_version 1060147 (0.0009) [2023-12-26 23:05:18,947][105692] Updated weights for policy 0, policy_version 1059175 (0.0007) [2023-12-26 23:05:18,953][105620] Updated weights for policy 1, policy_version 1060157 (0.0009) [2023-12-26 23:05:19,008][105692] Updated weights for policy 0, policy_version 1059185 (0.0006) [2023-12-26 23:05:19,010][105620] Updated weights for policy 1, policy_version 1060167 (0.0008) [2023-12-26 23:05:19,064][105692] Updated weights for policy 0, policy_version 1059195 (0.0007) [2023-12-26 23:05:19,766][105620] Updated weights for policy 1, policy_version 1060177 (0.0006) [2023-12-26 23:05:19,823][105620] Updated weights for policy 1, policy_version 1060187 (0.0007) [2023-12-26 23:05:19,849][105692] Updated weights for policy 0, policy_version 1059205 (0.0009) [2023-12-26 23:05:19,880][105620] Updated weights for policy 1, policy_version 1060197 (0.0009) [2023-12-26 23:05:19,921][105692] Updated weights for policy 0, policy_version 1059215 (0.0009) [2023-12-26 23:05:19,978][105692] Updated weights for policy 0, policy_version 1059225 (0.0006) [2023-12-26 23:05:20,628][105692] Updated weights for policy 0, policy_version 1059235 (0.0008) [2023-12-26 23:05:20,660][105620] Updated weights for policy 1, policy_version 1060207 (0.0008) [2023-12-26 23:05:20,690][105692] Updated weights for policy 0, policy_version 1059245 (0.0010) [2023-12-26 23:05:20,721][105620] Updated weights for policy 1, policy_version 1060217 (0.0007) [2023-12-26 23:05:20,748][105692] Updated weights for policy 0, policy_version 1059255 (0.0009) [2023-12-26 23:05:20,783][105620] Updated weights for policy 1, policy_version 1060227 (0.0007) [2023-12-26 23:05:21,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 542662656. Throughput: 0: 9693.4, 1: 9689.6. Samples: 542649088. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:05:21,063][104569] Avg episode reward: [(0, '8904.311'), (1, '9161.912')] [2023-12-26 23:05:21,475][105620] Updated weights for policy 1, policy_version 1060237 (0.0010) [2023-12-26 23:05:21,504][105692] Updated weights for policy 0, policy_version 1059265 (0.0007) [2023-12-26 23:05:21,544][105620] Updated weights for policy 1, policy_version 1060247 (0.0009) [2023-12-26 23:05:21,567][105692] Updated weights for policy 0, policy_version 1059275 (0.0006) [2023-12-26 23:05:21,605][105620] Updated weights for policy 1, policy_version 1060257 (0.0008) [2023-12-26 23:05:21,642][105692] Updated weights for policy 0, policy_version 1059285 (0.0008) [2023-12-26 23:05:21,710][105692] Updated weights for policy 0, policy_version 1059295 (0.0008) [2023-12-26 23:05:22,327][105692] Updated weights for policy 0, policy_version 1059305 (0.0008) [2023-12-26 23:05:22,393][105692] Updated weights for policy 0, policy_version 1059315 (0.0008) [2023-12-26 23:05:22,427][105620] Updated weights for policy 1, policy_version 1060267 (0.0006) [2023-12-26 23:05:22,453][105692] Updated weights for policy 0, policy_version 1059325 (0.0008) [2023-12-26 23:05:22,490][105620] Updated weights for policy 1, policy_version 1060277 (0.0007) [2023-12-26 23:05:22,549][105620] Updated weights for policy 1, policy_version 1060287 (0.0009) [2023-12-26 23:05:23,218][105620] Updated weights for policy 1, policy_version 1060297 (0.0009) [2023-12-26 23:05:23,268][105692] Updated weights for policy 0, policy_version 1059335 (0.0008) [2023-12-26 23:05:23,274][105620] Updated weights for policy 1, policy_version 1060307 (0.0006) [2023-12-26 23:05:23,332][105692] Updated weights for policy 0, policy_version 1059345 (0.0008) [2023-12-26 23:05:23,335][105620] Updated weights for policy 1, policy_version 1060317 (0.0008) [2023-12-26 23:05:23,389][105692] Updated weights for policy 0, policy_version 1059355 (0.0007) [2023-12-26 23:05:23,396][105620] Updated weights for policy 1, policy_version 1060327 (0.0008) [2023-12-26 23:05:24,140][105692] Updated weights for policy 0, policy_version 1059365 (0.0009) [2023-12-26 23:05:24,152][105620] Updated weights for policy 1, policy_version 1060337 (0.0007) [2023-12-26 23:05:24,193][105692] Updated weights for policy 0, policy_version 1059375 (0.0010) [2023-12-26 23:05:24,198][105620] Updated weights for policy 1, policy_version 1060347 (0.0008) [2023-12-26 23:05:24,248][105692] Updated weights for policy 0, policy_version 1059385 (0.0010) [2023-12-26 23:05:24,253][105620] Updated weights for policy 1, policy_version 1060357 (0.0008) [2023-12-26 23:05:24,963][105692] Updated weights for policy 0, policy_version 1059395 (0.0009) [2023-12-26 23:05:25,011][105692] Updated weights for policy 0, policy_version 1059405 (0.0005) [2023-12-26 23:05:25,065][105692] Updated weights for policy 0, policy_version 1059415 (0.0005) [2023-12-26 23:05:25,065][105620] Updated weights for policy 1, policy_version 1060367 (0.0007) [2023-12-26 23:05:25,125][105620] Updated weights for policy 1, policy_version 1060377 (0.0006) [2023-12-26 23:05:25,180][105620] Updated weights for policy 1, policy_version 1060387 (0.0006) [2023-12-26 23:05:25,708][105692] Updated weights for policy 0, policy_version 1059425 (0.0005) [2023-12-26 23:05:25,765][105692] Updated weights for policy 0, policy_version 1059435 (0.0009) [2023-12-26 23:05:25,812][105692] Updated weights for policy 0, policy_version 1059445 (0.0009) [2023-12-26 23:05:25,861][105692] Updated weights for policy 0, policy_version 1059455 (0.0009) [2023-12-26 23:05:25,904][105620] Updated weights for policy 1, policy_version 1060397 (0.0007) [2023-12-26 23:05:25,955][105620] Updated weights for policy 1, policy_version 1060407 (0.0009) [2023-12-26 23:05:26,012][105620] Updated weights for policy 1, policy_version 1060418 (0.0010) [2023-12-26 23:05:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19521.9). Total num frames: 542760960. Throughput: 0: 9718.4, 1: 9600.6. Samples: 542762532. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:05:26,062][104569] Avg episode reward: [(0, '8728.960'), (1, '9162.876')] [2023-12-26 23:05:26,542][105692] Updated weights for policy 0, policy_version 1059465 (0.0010) [2023-12-26 23:05:26,601][105692] Updated weights for policy 0, policy_version 1059475 (0.0009) [2023-12-26 23:05:26,660][105692] Updated weights for policy 0, policy_version 1059485 (0.0008) [2023-12-26 23:05:26,824][105620] Updated weights for policy 1, policy_version 1060428 (0.0009) [2023-12-26 23:05:26,891][105620] Updated weights for policy 1, policy_version 1060438 (0.0009) [2023-12-26 23:05:26,943][105620] Updated weights for policy 1, policy_version 1060448 (0.0009) [2023-12-26 23:05:27,275][105692] Updated weights for policy 0, policy_version 1059495 (0.0006) [2023-12-26 23:05:27,340][105692] Updated weights for policy 0, policy_version 1059505 (0.0005) [2023-12-26 23:05:27,402][105692] Updated weights for policy 0, policy_version 1059515 (0.0006) [2023-12-26 23:05:27,646][105620] Updated weights for policy 1, policy_version 1060458 (0.0009) [2023-12-26 23:05:27,699][105620] Updated weights for policy 1, policy_version 1060469 (0.0010) [2023-12-26 23:05:27,752][105620] Updated weights for policy 1, policy_version 1060479 (0.0010) [2023-12-26 23:05:27,947][105692] Updated weights for policy 0, policy_version 1059525 (0.0009) [2023-12-26 23:05:28,008][105692] Updated weights for policy 0, policy_version 1059535 (0.0007) [2023-12-26 23:05:28,058][105692] Updated weights for policy 0, policy_version 1059545 (0.0006) [2023-12-26 23:05:28,609][105620] Updated weights for policy 1, policy_version 1060489 (0.0010) [2023-12-26 23:05:28,671][105620] Updated weights for policy 1, policy_version 1060499 (0.0008) [2023-12-26 23:05:28,716][105692] Updated weights for policy 0, policy_version 1059555 (0.0010) [2023-12-26 23:05:28,726][105620] Updated weights for policy 1, policy_version 1060509 (0.0006) [2023-12-26 23:05:28,778][105692] Updated weights for policy 0, policy_version 1059565 (0.0008) [2023-12-26 23:05:28,785][105620] Updated weights for policy 1, policy_version 1060519 (0.0006) [2023-12-26 23:05:28,825][105692] Updated weights for policy 0, policy_version 1059575 (0.0005) [2023-12-26 23:05:29,441][105620] Updated weights for policy 1, policy_version 1060529 (0.0006) [2023-12-26 23:05:29,495][105620] Updated weights for policy 1, policy_version 1060539 (0.0005) [2023-12-26 23:05:29,549][105692] Updated weights for policy 0, policy_version 1059585 (0.0007) [2023-12-26 23:05:29,554][105620] Updated weights for policy 1, policy_version 1060549 (0.0005) [2023-12-26 23:05:29,605][105692] Updated weights for policy 0, policy_version 1059595 (0.0011) [2023-12-26 23:05:29,677][105692] Updated weights for policy 0, policy_version 1059605 (0.0011) [2023-12-26 23:05:29,745][105692] Updated weights for policy 0, policy_version 1059615 (0.0011) [2023-12-26 23:05:30,157][105620] Updated weights for policy 1, policy_version 1060559 (0.0007) [2023-12-26 23:05:30,208][105620] Updated weights for policy 1, policy_version 1060569 (0.0010) [2023-12-26 23:05:30,262][105620] Updated weights for policy 1, policy_version 1060579 (0.0010) [2023-12-26 23:05:30,430][105692] Updated weights for policy 0, policy_version 1059625 (0.0006) [2023-12-26 23:05:30,496][105692] Updated weights for policy 0, policy_version 1059635 (0.0007) [2023-12-26 23:05:30,586][105692] Updated weights for policy 0, policy_version 1059645 (0.0009) [2023-12-26 23:05:30,973][105620] Updated weights for policy 1, policy_version 1060589 (0.0010) [2023-12-26 23:05:31,035][105620] Updated weights for policy 1, policy_version 1060599 (0.0009) [2023-12-26 23:05:31,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.1, 300 sec: 19521.9). Total num frames: 542851072. Throughput: 0: 9778.4, 1: 9516.1. Samples: 542822524. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:05:31,063][104569] Avg episode reward: [(0, '8998.371'), (1, '9164.737')] [2023-12-26 23:05:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001059648_271310848.pth... [2023-12-26 23:05:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001058496_271015936.pth [2023-12-26 23:05:31,105][105620] Updated weights for policy 1, policy_version 1060609 (0.0011) [2023-12-26 23:05:31,147][105692] Updated weights for policy 0, policy_version 1059655 (0.0008) [2023-12-26 23:05:31,159][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001060616_271548416.pth... [2023-12-26 23:05:31,164][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001059496_271261696.pth [2023-12-26 23:05:31,211][105692] Updated weights for policy 0, policy_version 1059665 (0.0006) [2023-12-26 23:05:31,272][105692] Updated weights for policy 0, policy_version 1059675 (0.0007) [2023-12-26 23:05:31,879][105692] Updated weights for policy 0, policy_version 1059685 (0.0009) [2023-12-26 23:05:31,895][105620] Updated weights for policy 1, policy_version 1060619 (0.0010) [2023-12-26 23:05:31,931][105692] Updated weights for policy 0, policy_version 1059695 (0.0007) [2023-12-26 23:05:31,952][105620] Updated weights for policy 1, policy_version 1060629 (0.0010) [2023-12-26 23:05:31,985][105692] Updated weights for policy 0, policy_version 1059705 (0.0005) [2023-12-26 23:05:32,011][105620] Updated weights for policy 1, policy_version 1060639 (0.0011) [2023-12-26 23:05:32,701][105620] Updated weights for policy 1, policy_version 1060649 (0.0011) [2023-12-26 23:05:32,708][105692] Updated weights for policy 0, policy_version 1059715 (0.0006) [2023-12-26 23:05:32,749][105620] Updated weights for policy 1, policy_version 1060659 (0.0010) [2023-12-26 23:05:32,760][105692] Updated weights for policy 0, policy_version 1059725 (0.0005) [2023-12-26 23:05:32,798][105620] Updated weights for policy 1, policy_version 1060669 (0.0010) [2023-12-26 23:05:32,809][105692] Updated weights for policy 0, policy_version 1059735 (0.0005) [2023-12-26 23:05:32,851][105620] Updated weights for policy 1, policy_version 1060679 (0.0011) [2023-12-26 23:05:33,578][105620] Updated weights for policy 1, policy_version 1060689 (0.0010) [2023-12-26 23:05:33,580][105692] Updated weights for policy 0, policy_version 1059745 (0.0005) [2023-12-26 23:05:33,625][105692] Updated weights for policy 0, policy_version 1059755 (0.0008) [2023-12-26 23:05:33,633][105620] Updated weights for policy 1, policy_version 1060699 (0.0010) [2023-12-26 23:05:33,671][105692] Updated weights for policy 0, policy_version 1059765 (0.0005) [2023-12-26 23:05:33,687][105620] Updated weights for policy 1, policy_version 1060709 (0.0010) [2023-12-26 23:05:33,730][105692] Updated weights for policy 0, policy_version 1059775 (0.0008) [2023-12-26 23:05:34,339][105620] Updated weights for policy 1, policy_version 1060719 (0.0007) [2023-12-26 23:05:34,401][105620] Updated weights for policy 1, policy_version 1060729 (0.0008) [2023-12-26 23:05:34,453][105692] Updated weights for policy 0, policy_version 1059785 (0.0006) [2023-12-26 23:05:34,457][105620] Updated weights for policy 1, policy_version 1060739 (0.0009) [2023-12-26 23:05:34,501][105692] Updated weights for policy 0, policy_version 1059795 (0.0009) [2023-12-26 23:05:34,566][105692] Updated weights for policy 0, policy_version 1059805 (0.0009) [2023-12-26 23:05:35,260][105620] Updated weights for policy 1, policy_version 1060749 (0.0009) [2023-12-26 23:05:35,285][105692] Updated weights for policy 0, policy_version 1059815 (0.0006) [2023-12-26 23:05:35,323][105620] Updated weights for policy 1, policy_version 1060759 (0.0009) [2023-12-26 23:05:35,336][105692] Updated weights for policy 0, policy_version 1059825 (0.0005) [2023-12-26 23:05:35,385][105620] Updated weights for policy 1, policy_version 1060769 (0.0006) [2023-12-26 23:05:35,391][105692] Updated weights for policy 0, policy_version 1059835 (0.0010) [2023-12-26 23:05:35,992][105692] Updated weights for policy 0, policy_version 1059845 (0.0007) [2023-12-26 23:05:36,049][105692] Updated weights for policy 0, policy_version 1059855 (0.0008) [2023-12-26 23:05:36,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 542949376. Throughput: 0: 9804.3, 1: 9543.4. Samples: 542942672. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:05:36,062][104569] Avg episode reward: [(0, '9265.534'), (1, '9256.496')] [2023-12-26 23:05:36,115][105692] Updated weights for policy 0, policy_version 1059865 (0.0010) [2023-12-26 23:05:36,149][105620] Updated weights for policy 1, policy_version 1060779 (0.0006) [2023-12-26 23:05:36,204][105620] Updated weights for policy 1, policy_version 1060789 (0.0009) [2023-12-26 23:05:36,260][105620] Updated weights for policy 1, policy_version 1060799 (0.0008) [2023-12-26 23:05:36,884][105692] Updated weights for policy 0, policy_version 1059875 (0.0010) [2023-12-26 23:05:36,908][105620] Updated weights for policy 1, policy_version 1060809 (0.0008) [2023-12-26 23:05:36,932][105692] Updated weights for policy 0, policy_version 1059885 (0.0010) [2023-12-26 23:05:36,959][105620] Updated weights for policy 1, policy_version 1060819 (0.0005) [2023-12-26 23:05:36,977][105692] Updated weights for policy 0, policy_version 1059895 (0.0010) [2023-12-26 23:05:37,018][105620] Updated weights for policy 1, policy_version 1060829 (0.0005) [2023-12-26 23:05:37,079][105620] Updated weights for policy 1, policy_version 1060839 (0.0005) [2023-12-26 23:05:37,604][105620] Updated weights for policy 1, policy_version 1060849 (0.0005) [2023-12-26 23:05:37,669][105620] Updated weights for policy 1, policy_version 1060859 (0.0005) [2023-12-26 23:05:37,737][105620] Updated weights for policy 1, policy_version 1060869 (0.0006) [2023-12-26 23:05:37,746][105692] Updated weights for policy 0, policy_version 1059905 (0.0010) [2023-12-26 23:05:37,752][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000010 [2023-12-26 23:05:37,796][105692] Updated weights for policy 0, policy_version 1059915 (0.0010) [2023-12-26 23:05:37,862][105692] Updated weights for policy 0, policy_version 1059925 (0.0010) [2023-12-26 23:05:37,924][105692] Updated weights for policy 0, policy_version 1059935 (0.0010) [2023-12-26 23:05:38,296][105620] Updated weights for policy 1, policy_version 1060879 (0.0009) [2023-12-26 23:05:38,362][105620] Updated weights for policy 1, policy_version 1060889 (0.0011) [2023-12-26 23:05:38,421][105620] Updated weights for policy 1, policy_version 1060899 (0.0010) [2023-12-26 23:05:38,614][105692] Updated weights for policy 0, policy_version 1059945 (0.0011) [2023-12-26 23:05:38,683][105692] Updated weights for policy 0, policy_version 1059955 (0.0011) [2023-12-26 23:05:38,745][105692] Updated weights for policy 0, policy_version 1059965 (0.0010) [2023-12-26 23:05:39,101][105620] Updated weights for policy 1, policy_version 1060909 (0.0008) [2023-12-26 23:05:39,159][105620] Updated weights for policy 1, policy_version 1060919 (0.0005) [2023-12-26 23:05:39,214][105620] Updated weights for policy 1, policy_version 1060929 (0.0006) [2023-12-26 23:05:39,486][105692] Updated weights for policy 0, policy_version 1059975 (0.0009) [2023-12-26 23:05:39,543][105692] Updated weights for policy 0, policy_version 1059985 (0.0008) [2023-12-26 23:05:39,604][105692] Updated weights for policy 0, policy_version 1059995 (0.0008) [2023-12-26 23:05:39,989][105620] Updated weights for policy 1, policy_version 1060939 (0.0008) [2023-12-26 23:05:40,050][105620] Updated weights for policy 1, policy_version 1060949 (0.0008) [2023-12-26 23:05:40,101][105620] Updated weights for policy 1, policy_version 1060959 (0.0008) [2023-12-26 23:05:40,385][105692] Updated weights for policy 0, policy_version 1060005 (0.0009) [2023-12-26 23:05:40,443][105692] Updated weights for policy 0, policy_version 1060015 (0.0008) [2023-12-26 23:05:40,502][105692] Updated weights for policy 0, policy_version 1060025 (0.0006) [2023-12-26 23:05:40,872][105620] Updated weights for policy 1, policy_version 1060969 (0.0009) [2023-12-26 23:05:40,925][105620] Updated weights for policy 1, policy_version 1060979 (0.0010) [2023-12-26 23:05:40,992][105620] Updated weights for policy 1, policy_version 1060989 (0.0010) [2023-12-26 23:05:41,055][105620] Updated weights for policy 1, policy_version 1060999 (0.0007) [2023-12-26 23:05:41,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 543055872. Throughput: 0: 9740.4, 1: 9654.9. Samples: 543061840. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:05:41,062][104569] Avg episode reward: [(0, '9354.709'), (1, '9256.532')] [2023-12-26 23:05:41,205][105692] Updated weights for policy 0, policy_version 1060035 (0.0006) [2023-12-26 23:05:41,266][105692] Updated weights for policy 0, policy_version 1060045 (0.0008) [2023-12-26 23:05:41,320][105692] Updated weights for policy 0, policy_version 1060055 (0.0008) [2023-12-26 23:05:41,864][105620] Updated weights for policy 1, policy_version 1061009 (0.0010) [2023-12-26 23:05:41,927][105620] Updated weights for policy 1, policy_version 1061019 (0.0010) [2023-12-26 23:05:41,991][105620] Updated weights for policy 1, policy_version 1061029 (0.0011) [2023-12-26 23:05:42,125][105692] Updated weights for policy 0, policy_version 1060065 (0.0008) [2023-12-26 23:05:42,178][105692] Updated weights for policy 0, policy_version 1060075 (0.0008) [2023-12-26 23:05:42,227][105692] Updated weights for policy 0, policy_version 1060085 (0.0008) [2023-12-26 23:05:42,284][105692] Updated weights for policy 0, policy_version 1060095 (0.0008) [2023-12-26 23:05:42,743][105620] Updated weights for policy 1, policy_version 1061039 (0.0010) [2023-12-26 23:05:42,795][105620] Updated weights for policy 1, policy_version 1061049 (0.0010) [2023-12-26 23:05:42,850][105620] Updated weights for policy 1, policy_version 1061059 (0.0010) [2023-12-26 23:05:43,080][105692] Updated weights for policy 0, policy_version 1060105 (0.0008) [2023-12-26 23:05:43,144][105692] Updated weights for policy 0, policy_version 1060115 (0.0008) [2023-12-26 23:05:43,208][105692] Updated weights for policy 0, policy_version 1060125 (0.0009) [2023-12-26 23:05:43,523][105620] Updated weights for policy 1, policy_version 1061069 (0.0010) [2023-12-26 23:05:43,580][105620] Updated weights for policy 1, policy_version 1061079 (0.0010) [2023-12-26 23:05:43,635][105620] Updated weights for policy 1, policy_version 1061089 (0.0010) [2023-12-26 23:05:43,859][105692] Updated weights for policy 0, policy_version 1060135 (0.0007) [2023-12-26 23:05:43,913][105692] Updated weights for policy 0, policy_version 1060145 (0.0005) [2023-12-26 23:05:43,967][105692] Updated weights for policy 0, policy_version 1060155 (0.0005) [2023-12-26 23:05:44,434][105620] Updated weights for policy 1, policy_version 1061099 (0.0011) [2023-12-26 23:05:44,497][105620] Updated weights for policy 1, policy_version 1061109 (0.0008) [2023-12-26 23:05:44,518][105692] Updated weights for policy 0, policy_version 1060165 (0.0007) [2023-12-26 23:05:44,552][105620] Updated weights for policy 1, policy_version 1061119 (0.0005) [2023-12-26 23:05:44,580][105692] Updated weights for policy 0, policy_version 1060175 (0.0009) [2023-12-26 23:05:44,626][105692] Updated weights for policy 0, policy_version 1060185 (0.0007) [2023-12-26 23:05:45,191][105620] Updated weights for policy 1, policy_version 1061129 (0.0007) [2023-12-26 23:05:45,241][105620] Updated weights for policy 1, policy_version 1061139 (0.0005) [2023-12-26 23:05:45,297][105620] Updated weights for policy 1, policy_version 1061149 (0.0006) [2023-12-26 23:05:45,352][105620] Updated weights for policy 1, policy_version 1061159 (0.0005) [2023-12-26 23:05:45,396][105692] Updated weights for policy 0, policy_version 1060195 (0.0007) [2023-12-26 23:05:45,457][105692] Updated weights for policy 0, policy_version 1060205 (0.0010) [2023-12-26 23:05:45,524][105692] Updated weights for policy 0, policy_version 1060215 (0.0010) [2023-12-26 23:05:45,924][105620] Updated weights for policy 1, policy_version 1061169 (0.0009) [2023-12-26 23:05:45,988][105620] Updated weights for policy 1, policy_version 1061179 (0.0006) [2023-12-26 23:05:46,035][105620] Updated weights for policy 1, policy_version 1061189 (0.0009) [2023-12-26 23:05:46,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 543154176. Throughput: 0: 9721.5, 1: 9630.7. Samples: 543117332. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-26 23:05:46,063][104569] Avg episode reward: [(0, '9355.121'), (1, '9073.191')] [2023-12-26 23:05:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001060224_271458304.pth... [2023-12-26 23:05:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001061192_271695872.pth... [2023-12-26 23:05:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001059072_271163392.pth [2023-12-26 23:05:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001060040_271400960.pth [2023-12-26 23:05:46,300][105692] Updated weights for policy 0, policy_version 1060225 (0.0010) [2023-12-26 23:05:46,357][105692] Updated weights for policy 0, policy_version 1060235 (0.0009) [2023-12-26 23:05:46,412][105692] Updated weights for policy 0, policy_version 1060245 (0.0007) [2023-12-26 23:05:46,467][105692] Updated weights for policy 0, policy_version 1060255 (0.0005) [2023-12-26 23:05:46,621][105620] Updated weights for policy 1, policy_version 1061199 (0.0009) [2023-12-26 23:05:46,674][105620] Updated weights for policy 1, policy_version 1061209 (0.0007) [2023-12-26 23:05:46,724][105620] Updated weights for policy 1, policy_version 1061219 (0.0005) [2023-12-26 23:05:47,062][105692] Updated weights for policy 0, policy_version 1060265 (0.0006) [2023-12-26 23:05:47,122][105692] Updated weights for policy 0, policy_version 1060275 (0.0006) [2023-12-26 23:05:47,174][105692] Updated weights for policy 0, policy_version 1060285 (0.0009) [2023-12-26 23:05:47,283][105620] Updated weights for policy 1, policy_version 1061229 (0.0005) [2023-12-26 23:05:47,330][105620] Updated weights for policy 1, policy_version 1061239 (0.0005) [2023-12-26 23:05:47,381][105620] Updated weights for policy 1, policy_version 1061249 (0.0006) [2023-12-26 23:05:47,828][105692] Updated weights for policy 0, policy_version 1060295 (0.0009) [2023-12-26 23:05:47,882][105692] Updated weights for policy 0, policy_version 1060305 (0.0007) [2023-12-26 23:05:47,948][105692] Updated weights for policy 0, policy_version 1060315 (0.0006) [2023-12-26 23:05:47,951][105620] Updated weights for policy 1, policy_version 1061259 (0.0006) [2023-12-26 23:05:48,009][105620] Updated weights for policy 1, policy_version 1061269 (0.0007) [2023-12-26 23:05:48,058][105620] Updated weights for policy 1, policy_version 1061279 (0.0005) [2023-12-26 23:05:48,661][105620] Updated weights for policy 1, policy_version 1061289 (0.0006) [2023-12-26 23:05:48,689][105692] Updated weights for policy 0, policy_version 1060325 (0.0006) [2023-12-26 23:05:48,719][105620] Updated weights for policy 1, policy_version 1061299 (0.0007) [2023-12-26 23:05:48,743][105692] Updated weights for policy 0, policy_version 1060335 (0.0006) [2023-12-26 23:05:48,774][105620] Updated weights for policy 1, policy_version 1061309 (0.0007) [2023-12-26 23:05:48,797][105692] Updated weights for policy 0, policy_version 1060345 (0.0006) [2023-12-26 23:05:48,835][105620] Updated weights for policy 1, policy_version 1061319 (0.0007) [2023-12-26 23:05:49,455][105620] Updated weights for policy 1, policy_version 1061329 (0.0009) [2023-12-26 23:05:49,518][105620] Updated weights for policy 1, policy_version 1061339 (0.0009) [2023-12-26 23:05:49,579][105620] Updated weights for policy 1, policy_version 1061349 (0.0008) [2023-12-26 23:05:49,623][105692] Updated weights for policy 0, policy_version 1060355 (0.0007) [2023-12-26 23:05:49,681][105692] Updated weights for policy 0, policy_version 1060365 (0.0009) [2023-12-26 23:05:49,739][105692] Updated weights for policy 0, policy_version 1060375 (0.0009) [2023-12-26 23:05:50,307][105620] Updated weights for policy 1, policy_version 1061359 (0.0009) [2023-12-26 23:05:50,362][105620] Updated weights for policy 1, policy_version 1061369 (0.0008) [2023-12-26 23:05:50,426][105620] Updated weights for policy 1, policy_version 1061379 (0.0008) [2023-12-26 23:05:50,533][105692] Updated weights for policy 0, policy_version 1060385 (0.0010) [2023-12-26 23:05:50,592][105692] Updated weights for policy 0, policy_version 1060395 (0.0008) [2023-12-26 23:05:50,647][105692] Updated weights for policy 0, policy_version 1060405 (0.0009) [2023-12-26 23:05:50,698][105692] Updated weights for policy 0, policy_version 1060415 (0.0008) [2023-12-26 23:05:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 543252480. Throughput: 0: 9790.8, 1: 9835.3. Samples: 543243716. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:05:51,062][104569] Avg episode reward: [(0, '9267.749'), (1, '9165.097')] [2023-12-26 23:05:51,113][105620] Updated weights for policy 1, policy_version 1061389 (0.0007) [2023-12-26 23:05:51,174][105620] Updated weights for policy 1, policy_version 1061399 (0.0008) [2023-12-26 23:05:51,240][105620] Updated weights for policy 1, policy_version 1061409 (0.0009) [2023-12-26 23:05:51,522][105692] Updated weights for policy 0, policy_version 1060425 (0.0006) [2023-12-26 23:05:51,571][105692] Updated weights for policy 0, policy_version 1060435 (0.0009) [2023-12-26 23:05:51,624][105692] Updated weights for policy 0, policy_version 1060445 (0.0008) [2023-12-26 23:05:52,073][105620] Updated weights for policy 1, policy_version 1061419 (0.0010) [2023-12-26 23:05:52,130][105620] Updated weights for policy 1, policy_version 1061429 (0.0009) [2023-12-26 23:05:52,182][105620] Updated weights for policy 1, policy_version 1061439 (0.0010) [2023-12-26 23:05:52,236][105692] Updated weights for policy 0, policy_version 1060455 (0.0007) [2023-12-26 23:05:52,296][105692] Updated weights for policy 0, policy_version 1060465 (0.0007) [2023-12-26 23:05:52,356][105692] Updated weights for policy 0, policy_version 1060475 (0.0010) [2023-12-26 23:05:52,978][105620] Updated weights for policy 1, policy_version 1061449 (0.0008) [2023-12-26 23:05:53,022][105620] Updated weights for policy 1, policy_version 1061459 (0.0008) [2023-12-26 23:05:53,077][105620] Updated weights for policy 1, policy_version 1061469 (0.0008) [2023-12-26 23:05:53,087][105692] Updated weights for policy 0, policy_version 1060485 (0.0010) [2023-12-26 23:05:53,139][105620] Updated weights for policy 1, policy_version 1061479 (0.0005) [2023-12-26 23:05:53,145][105692] Updated weights for policy 0, policy_version 1060495 (0.0010) [2023-12-26 23:05:53,200][105692] Updated weights for policy 0, policy_version 1060505 (0.0010) [2023-12-26 23:05:53,874][105620] Updated weights for policy 1, policy_version 1061489 (0.0010) [2023-12-26 23:05:53,940][105620] Updated weights for policy 1, policy_version 1061499 (0.0007) [2023-12-26 23:05:53,958][105692] Updated weights for policy 0, policy_version 1060515 (0.0010) [2023-12-26 23:05:53,994][105620] Updated weights for policy 1, policy_version 1061509 (0.0008) [2023-12-26 23:05:54,025][105692] Updated weights for policy 0, policy_version 1060525 (0.0011) [2023-12-26 23:05:54,091][105692] Updated weights for policy 0, policy_version 1060535 (0.0010) [2023-12-26 23:05:54,632][105620] Updated weights for policy 1, policy_version 1061519 (0.0007) [2023-12-26 23:05:54,701][105620] Updated weights for policy 1, policy_version 1061529 (0.0008) [2023-12-26 23:05:54,767][105620] Updated weights for policy 1, policy_version 1061539 (0.0007) [2023-12-26 23:05:54,815][105692] Updated weights for policy 0, policy_version 1060545 (0.0010) [2023-12-26 23:05:54,882][105692] Updated weights for policy 0, policy_version 1060555 (0.0005) [2023-12-26 23:05:54,939][105692] Updated weights for policy 0, policy_version 1060565 (0.0010) [2023-12-26 23:05:54,996][105692] Updated weights for policy 0, policy_version 1060575 (0.0010) [2023-12-26 23:05:55,349][105620] Updated weights for policy 1, policy_version 1061549 (0.0006) [2023-12-26 23:05:55,411][105620] Updated weights for policy 1, policy_version 1061559 (0.0006) [2023-12-26 23:05:55,473][105620] Updated weights for policy 1, policy_version 1061569 (0.0008) [2023-12-26 23:05:55,717][105692] Updated weights for policy 0, policy_version 1060585 (0.0010) [2023-12-26 23:05:55,774][105692] Updated weights for policy 0, policy_version 1060595 (0.0010) [2023-12-26 23:05:55,821][105692] Updated weights for policy 0, policy_version 1060605 (0.0010) [2023-12-26 23:05:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 543350784. Throughput: 0: 9726.7, 1: 9842.4. Samples: 543358752. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:05:56,062][104569] Avg episode reward: [(0, '9178.271'), (1, '9349.034')] [2023-12-26 23:05:56,160][105620] Updated weights for policy 1, policy_version 1061579 (0.0008) [2023-12-26 23:05:56,218][105620] Updated weights for policy 1, policy_version 1061589 (0.0008) [2023-12-26 23:05:56,276][105620] Updated weights for policy 1, policy_version 1061599 (0.0010) [2023-12-26 23:05:56,533][105692] Updated weights for policy 0, policy_version 1060615 (0.0010) [2023-12-26 23:05:56,581][105692] Updated weights for policy 0, policy_version 1060625 (0.0010) [2023-12-26 23:05:56,629][105692] Updated weights for policy 0, policy_version 1060635 (0.0010) [2023-12-26 23:05:57,008][105620] Updated weights for policy 1, policy_version 1061609 (0.0010) [2023-12-26 23:05:57,062][105620] Updated weights for policy 1, policy_version 1061620 (0.0010) [2023-12-26 23:05:57,119][105620] Updated weights for policy 1, policy_version 1061631 (0.0010) [2023-12-26 23:05:57,272][105692] Updated weights for policy 0, policy_version 1060645 (0.0008) [2023-12-26 23:05:57,324][105692] Updated weights for policy 0, policy_version 1060655 (0.0008) [2023-12-26 23:05:57,376][105692] Updated weights for policy 0, policy_version 1060665 (0.0010) [2023-12-26 23:05:57,800][105620] Updated weights for policy 1, policy_version 1061642 (0.0009) [2023-12-26 23:05:57,855][105620] Updated weights for policy 1, policy_version 1061652 (0.0005) [2023-12-26 23:05:57,909][105620] Updated weights for policy 1, policy_version 1061662 (0.0008) [2023-12-26 23:05:57,953][105620] Updated weights for policy 1, policy_version 1061672 (0.0007) [2023-12-26 23:05:58,099][105692] Updated weights for policy 0, policy_version 1060675 (0.0010) [2023-12-26 23:05:58,163][105692] Updated weights for policy 0, policy_version 1060685 (0.0010) [2023-12-26 23:05:58,230][105692] Updated weights for policy 0, policy_version 1060695 (0.0008) [2023-12-26 23:05:58,741][105620] Updated weights for policy 1, policy_version 1061682 (0.0007) [2023-12-26 23:05:58,807][105620] Updated weights for policy 1, policy_version 1061692 (0.0008) [2023-12-26 23:05:58,879][105620] Updated weights for policy 1, policy_version 1061702 (0.0008) [2023-12-26 23:05:59,055][105692] Updated weights for policy 0, policy_version 1060705 (0.0008) [2023-12-26 23:05:59,083][105585] KL-divergence is very high: 138.9566 [2023-12-26 23:05:59,099][105585] KL-divergence is very high: 277.1169 [2023-12-26 23:05:59,106][105585] KL-divergence is very high: 286.3467 [2023-12-26 23:05:59,121][105692] Updated weights for policy 0, policy_version 1060715 (0.0008) [2023-12-26 23:05:59,133][105585] KL-divergence is very high: 301.3091 [2023-12-26 23:05:59,139][105585] KL-divergence is very high: 132.1344 [2023-12-26 23:05:59,150][105585] KL-divergence is very high: 404.6263 [2023-12-26 23:05:59,157][105585] KL-divergence is very high: 313.1273 [2023-12-26 23:05:59,183][105585] KL-divergence is very high: 226.4482 [2023-12-26 23:05:59,183][105692] Updated weights for policy 0, policy_version 1060725 (0.0009) [2023-12-26 23:05:59,190][105585] KL-divergence is very high: 107.4262 [2023-12-26 23:05:59,203][105585] KL-divergence is very high: 313.1887 [2023-12-26 23:05:59,212][105585] KL-divergence is very high: 170.9207 [2023-12-26 23:05:59,254][105692] Updated weights for policy 0, policy_version 1060735 (0.0008) [2023-12-26 23:05:59,611][105620] Updated weights for policy 1, policy_version 1061712 (0.0009) [2023-12-26 23:05:59,675][105620] Updated weights for policy 1, policy_version 1061722 (0.0010) [2023-12-26 23:05:59,735][105620] Updated weights for policy 1, policy_version 1061732 (0.0009) [2023-12-26 23:05:59,962][105692] Updated weights for policy 0, policy_version 1060745 (0.0007) [2023-12-26 23:06:00,021][105692] Updated weights for policy 0, policy_version 1060755 (0.0007) [2023-12-26 23:06:00,082][105692] Updated weights for policy 0, policy_version 1060765 (0.0005) [2023-12-26 23:06:00,532][105620] Updated weights for policy 1, policy_version 1061742 (0.0007) [2023-12-26 23:06:00,594][105620] Updated weights for policy 1, policy_version 1061752 (0.0005) [2023-12-26 23:06:00,657][105620] Updated weights for policy 1, policy_version 1061762 (0.0006) [2023-12-26 23:06:00,843][105692] Updated weights for policy 0, policy_version 1060775 (0.0008) [2023-12-26 23:06:00,891][105692] Updated weights for policy 0, policy_version 1060785 (0.0010) [2023-12-26 23:06:00,938][105692] Updated weights for policy 0, policy_version 1060795 (0.0010) [2023-12-26 23:06:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 543449088. Throughput: 0: 9757.9, 1: 9868.1. Samples: 543417332. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:06:01,062][104569] Avg episode reward: [(0, '9089.881'), (1, '9350.682')] [2023-12-26 23:06:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001061768_271843328.pth... [2023-12-26 23:06:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001060800_271605760.pth... [2023-12-26 23:06:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001059648_271310848.pth [2023-12-26 23:06:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001060616_271548416.pth [2023-12-26 23:06:01,183][105620] Updated weights for policy 1, policy_version 1061772 (0.0007) [2023-12-26 23:06:01,236][105620] Updated weights for policy 1, policy_version 1061782 (0.0005) [2023-12-26 23:06:01,299][105620] Updated weights for policy 1, policy_version 1061792 (0.0008) [2023-12-26 23:06:01,695][105692] Updated weights for policy 0, policy_version 1060805 (0.0009) [2023-12-26 23:06:01,767][105692] Updated weights for policy 0, policy_version 1060815 (0.0010) [2023-12-26 23:06:01,831][105692] Updated weights for policy 0, policy_version 1060825 (0.0010) [2023-12-26 23:06:02,017][105620] Updated weights for policy 1, policy_version 1061802 (0.0008) [2023-12-26 23:06:02,070][105620] Updated weights for policy 1, policy_version 1061812 (0.0008) [2023-12-26 23:06:02,131][105620] Updated weights for policy 1, policy_version 1061822 (0.0008) [2023-12-26 23:06:02,184][105620] Updated weights for policy 1, policy_version 1061832 (0.0010) [2023-12-26 23:06:02,541][105692] Updated weights for policy 0, policy_version 1060835 (0.0009) [2023-12-26 23:06:02,602][105692] Updated weights for policy 0, policy_version 1060845 (0.0008) [2023-12-26 23:06:02,661][105692] Updated weights for policy 0, policy_version 1060855 (0.0011) [2023-12-26 23:06:02,914][105620] Updated weights for policy 1, policy_version 1061842 (0.0006) [2023-12-26 23:06:02,968][105620] Updated weights for policy 1, policy_version 1061852 (0.0008) [2023-12-26 23:06:03,018][105620] Updated weights for policy 1, policy_version 1061862 (0.0008) [2023-12-26 23:06:03,301][105692] Updated weights for policy 0, policy_version 1060865 (0.0010) [2023-12-26 23:06:03,360][105692] Updated weights for policy 0, policy_version 1060875 (0.0010) [2023-12-26 23:06:03,422][105692] Updated weights for policy 0, policy_version 1060885 (0.0010) [2023-12-26 23:06:03,480][105692] Updated weights for policy 0, policy_version 1060895 (0.0010) [2023-12-26 23:06:03,611][105620] Updated weights for policy 1, policy_version 1061872 (0.0008) [2023-12-26 23:06:03,671][105620] Updated weights for policy 1, policy_version 1061882 (0.0008) [2023-12-26 23:06:03,728][105620] Updated weights for policy 1, policy_version 1061892 (0.0009) [2023-12-26 23:06:04,208][105692] Updated weights for policy 0, policy_version 1060905 (0.0009) [2023-12-26 23:06:04,272][105692] Updated weights for policy 0, policy_version 1060915 (0.0009) [2023-12-26 23:06:04,325][105692] Updated weights for policy 0, policy_version 1060925 (0.0009) [2023-12-26 23:06:04,392][105620] Updated weights for policy 1, policy_version 1061902 (0.0009) [2023-12-26 23:06:04,455][105620] Updated weights for policy 1, policy_version 1061912 (0.0009) [2023-12-26 23:06:04,511][105620] Updated weights for policy 1, policy_version 1061922 (0.0009) [2023-12-26 23:06:05,016][105692] Updated weights for policy 0, policy_version 1060935 (0.0010) [2023-12-26 23:06:05,071][105692] Updated weights for policy 0, policy_version 1060947 (0.0011) [2023-12-26 23:06:05,112][105620] Updated weights for policy 1, policy_version 1061932 (0.0008) [2023-12-26 23:06:05,134][105692] Updated weights for policy 0, policy_version 1060958 (0.0008) [2023-12-26 23:06:05,169][105620] Updated weights for policy 1, policy_version 1061942 (0.0008) [2023-12-26 23:06:05,228][105620] Updated weights for policy 1, policy_version 1061952 (0.0009) [2023-12-26 23:06:05,886][105692] Updated weights for policy 0, policy_version 1060968 (0.0009) [2023-12-26 23:06:05,940][105692] Updated weights for policy 0, policy_version 1060978 (0.0008) [2023-12-26 23:06:05,949][105620] Updated weights for policy 1, policy_version 1061962 (0.0009) [2023-12-26 23:06:05,990][105692] Updated weights for policy 0, policy_version 1060988 (0.0009) [2023-12-26 23:06:06,004][105620] Updated weights for policy 1, policy_version 1061972 (0.0008) [2023-12-26 23:06:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 543547392. Throughput: 0: 9778.7, 1: 9924.3. Samples: 543535728. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:06:06,062][104569] Avg episode reward: [(0, '8827.610'), (1, '9352.176')] [2023-12-26 23:06:06,067][105620] Updated weights for policy 1, policy_version 1061982 (0.0010) [2023-12-26 23:06:06,130][105620] Updated weights for policy 1, policy_version 1061992 (0.0010) [2023-12-26 23:06:06,784][105692] Updated weights for policy 0, policy_version 1060998 (0.0008) [2023-12-26 23:06:06,835][105692] Updated weights for policy 0, policy_version 1061008 (0.0007) [2023-12-26 23:06:06,854][105620] Updated weights for policy 1, policy_version 1062002 (0.0007) [2023-12-26 23:06:06,886][105692] Updated weights for policy 0, policy_version 1061018 (0.0008) [2023-12-26 23:06:06,922][105620] Updated weights for policy 1, policy_version 1062012 (0.0006) [2023-12-26 23:06:06,977][105620] Updated weights for policy 1, policy_version 1062022 (0.0006) [2023-12-26 23:06:07,543][105620] Updated weights for policy 1, policy_version 1062032 (0.0005) [2023-12-26 23:06:07,597][105620] Updated weights for policy 1, policy_version 1062042 (0.0005) [2023-12-26 23:06:07,659][105620] Updated weights for policy 1, policy_version 1062052 (0.0006) [2023-12-26 23:06:07,702][105692] Updated weights for policy 0, policy_version 1061028 (0.0007) [2023-12-26 23:06:07,752][105692] Updated weights for policy 0, policy_version 1061038 (0.0005) [2023-12-26 23:06:07,814][105692] Updated weights for policy 0, policy_version 1061048 (0.0007) [2023-12-26 23:06:08,313][105620] Updated weights for policy 1, policy_version 1062062 (0.0008) [2023-12-26 23:06:08,374][105620] Updated weights for policy 1, policy_version 1062072 (0.0010) [2023-12-26 23:06:08,433][105620] Updated weights for policy 1, policy_version 1062082 (0.0010) [2023-12-26 23:06:08,532][105692] Updated weights for policy 0, policy_version 1061058 (0.0008) [2023-12-26 23:06:08,592][105692] Updated weights for policy 0, policy_version 1061068 (0.0009) [2023-12-26 23:06:08,661][105692] Updated weights for policy 0, policy_version 1061078 (0.0010) [2023-12-26 23:06:08,727][105692] Updated weights for policy 0, policy_version 1061088 (0.0009) [2023-12-26 23:06:09,039][105620] Updated weights for policy 1, policy_version 1062092 (0.0010) [2023-12-26 23:06:09,099][105620] Updated weights for policy 1, policy_version 1062102 (0.0010) [2023-12-26 23:06:09,164][105620] Updated weights for policy 1, policy_version 1062112 (0.0010) [2023-12-26 23:06:09,469][105692] Updated weights for policy 0, policy_version 1061098 (0.0006) [2023-12-26 23:06:09,530][105692] Updated weights for policy 0, policy_version 1061108 (0.0009) [2023-12-26 23:06:09,597][105692] Updated weights for policy 0, policy_version 1061118 (0.0008) [2023-12-26 23:06:09,868][105620] Updated weights for policy 1, policy_version 1062122 (0.0010) [2023-12-26 23:06:09,932][105620] Updated weights for policy 1, policy_version 1062132 (0.0009) [2023-12-26 23:06:09,994][105620] Updated weights for policy 1, policy_version 1062142 (0.0007) [2023-12-26 23:06:10,050][105620] Updated weights for policy 1, policy_version 1062152 (0.0005) [2023-12-26 23:06:10,381][105692] Updated weights for policy 0, policy_version 1061128 (0.0009) [2023-12-26 23:06:10,429][105692] Updated weights for policy 0, policy_version 1061138 (0.0008) [2023-12-26 23:06:10,493][105692] Updated weights for policy 0, policy_version 1061148 (0.0008) [2023-12-26 23:06:10,769][105620] Updated weights for policy 1, policy_version 1062162 (0.0010) [2023-12-26 23:06:10,817][105620] Updated weights for policy 1, policy_version 1062172 (0.0010) [2023-12-26 23:06:10,872][105620] Updated weights for policy 1, policy_version 1062182 (0.0010) [2023-12-26 23:06:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 543645696. Throughput: 0: 9728.6, 1: 10054.5. Samples: 543652772. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:06:11,062][104569] Avg episode reward: [(0, '8738.371'), (1, '9261.322')] [2023-12-26 23:06:11,210][105692] Updated weights for policy 0, policy_version 1061158 (0.0007) [2023-12-26 23:06:11,280][105692] Updated weights for policy 0, policy_version 1061168 (0.0007) [2023-12-26 23:06:11,352][105692] Updated weights for policy 0, policy_version 1061178 (0.0008) [2023-12-26 23:06:11,684][105620] Updated weights for policy 1, policy_version 1062192 (0.0009) [2023-12-26 23:06:11,757][105620] Updated weights for policy 1, policy_version 1062202 (0.0007) [2023-12-26 23:06:11,824][105620] Updated weights for policy 1, policy_version 1062212 (0.0009) [2023-12-26 23:06:12,038][105692] Updated weights for policy 0, policy_version 1061188 (0.0010) [2023-12-26 23:06:12,087][105692] Updated weights for policy 0, policy_version 1061198 (0.0007) [2023-12-26 23:06:12,152][105692] Updated weights for policy 0, policy_version 1061208 (0.0010) [2023-12-26 23:06:12,582][105620] Updated weights for policy 1, policy_version 1062222 (0.0010) [2023-12-26 23:06:12,644][105620] Updated weights for policy 1, policy_version 1062232 (0.0011) [2023-12-26 23:06:12,692][105586] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000007 [2023-12-26 23:06:12,879][105692] Updated weights for policy 0, policy_version 1061218 (0.0011) [2023-12-26 23:06:12,943][105692] Updated weights for policy 0, policy_version 1061228 (0.0011) [2023-12-26 23:06:13,002][105692] Updated weights for policy 0, policy_version 1061238 (0.0011) [2023-12-26 23:06:13,061][105692] Updated weights for policy 0, policy_version 1061248 (0.0011) [2023-12-26 23:06:13,386][105620] Updated weights for policy 1, policy_version 1062242 (0.0011) [2023-12-26 23:06:13,431][105620] Updated weights for policy 1, policy_version 1062252 (0.0010) [2023-12-26 23:06:13,479][105620] Updated weights for policy 1, policy_version 1062262 (0.0010) [2023-12-26 23:06:13,527][105620] Updated weights for policy 1, policy_version 1062272 (0.0010) [2023-12-26 23:06:13,719][105692] Updated weights for policy 0, policy_version 1061258 (0.0008) [2023-12-26 23:06:13,763][105692] Updated weights for policy 0, policy_version 1061268 (0.0008) [2023-12-26 23:06:13,808][105692] Updated weights for policy 0, policy_version 1061278 (0.0008) [2023-12-26 23:06:14,314][105620] Updated weights for policy 1, policy_version 1062282 (0.0010) [2023-12-26 23:06:14,365][105620] Updated weights for policy 1, policy_version 1062292 (0.0010) [2023-12-26 23:06:14,420][105620] Updated weights for policy 1, policy_version 1062302 (0.0010) [2023-12-26 23:06:14,600][105692] Updated weights for policy 0, policy_version 1061288 (0.0009) [2023-12-26 23:06:14,658][105692] Updated weights for policy 0, policy_version 1061298 (0.0010) [2023-12-26 23:06:14,707][105692] Updated weights for policy 0, policy_version 1061308 (0.0010) [2023-12-26 23:06:15,057][105620] Updated weights for policy 1, policy_version 1062312 (0.0011) [2023-12-26 23:06:15,128][105620] Updated weights for policy 1, policy_version 1062322 (0.0011) [2023-12-26 23:06:15,189][105620] Updated weights for policy 1, policy_version 1062332 (0.0011) [2023-12-26 23:06:15,486][105692] Updated weights for policy 0, policy_version 1061318 (0.0008) [2023-12-26 23:06:15,553][105692] Updated weights for policy 0, policy_version 1061328 (0.0006) [2023-12-26 23:06:15,618][105692] Updated weights for policy 0, policy_version 1061338 (0.0005) [2023-12-26 23:06:15,941][105620] Updated weights for policy 1, policy_version 1062342 (0.0011) [2023-12-26 23:06:15,993][105620] Updated weights for policy 1, policy_version 1062352 (0.0010) [2023-12-26 23:06:16,047][105620] Updated weights for policy 1, policy_version 1062362 (0.0010) [2023-12-26 23:06:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 543735808. Throughput: 0: 9663.4, 1: 10065.0. Samples: 543710300. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:06:16,062][104569] Avg episode reward: [(0, '8645.039'), (1, '9260.988')] [2023-12-26 23:06:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001061344_271745024.pth... [2023-12-26 23:06:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001060224_271458304.pth [2023-12-26 23:06:16,082][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001062368_271998976.pth... [2023-12-26 23:06:16,085][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001061192_271695872.pth [2023-12-26 23:06:16,284][105692] Updated weights for policy 0, policy_version 1061348 (0.0010) [2023-12-26 23:06:16,352][105692] Updated weights for policy 0, policy_version 1061358 (0.0010) [2023-12-26 23:06:16,400][105692] Updated weights for policy 0, policy_version 1061368 (0.0010) [2023-12-26 23:06:16,702][105620] Updated weights for policy 1, policy_version 1062372 (0.0011) [2023-12-26 23:06:16,767][105620] Updated weights for policy 1, policy_version 1062382 (0.0011) [2023-12-26 23:06:16,828][105620] Updated weights for policy 1, policy_version 1062392 (0.0010) [2023-12-26 23:06:17,123][105692] Updated weights for policy 0, policy_version 1061378 (0.0010) [2023-12-26 23:06:17,191][105692] Updated weights for policy 0, policy_version 1061388 (0.0010) [2023-12-26 23:06:17,262][105692] Updated weights for policy 0, policy_version 1061398 (0.0010) [2023-12-26 23:06:17,317][105692] Updated weights for policy 0, policy_version 1061408 (0.0010) [2023-12-26 23:06:17,488][105620] Updated weights for policy 1, policy_version 1062402 (0.0009) [2023-12-26 23:06:17,539][105620] Updated weights for policy 1, policy_version 1062412 (0.0005) [2023-12-26 23:06:17,598][105620] Updated weights for policy 1, policy_version 1062422 (0.0005) [2023-12-26 23:06:17,659][105620] Updated weights for policy 1, policy_version 1062432 (0.0005) [2023-12-26 23:06:18,044][105692] Updated weights for policy 0, policy_version 1061418 (0.0006) [2023-12-26 23:06:18,099][105692] Updated weights for policy 0, policy_version 1061428 (0.0006) [2023-12-26 23:06:18,107][105585] KL-divergence is very high: 169.1479 [2023-12-26 23:06:18,147][105585] KL-divergence is very high: 185.9187 [2023-12-26 23:06:18,148][105692] Updated weights for policy 0, policy_version 1061438 (0.0007) [2023-12-26 23:06:18,191][105620] Updated weights for policy 1, policy_version 1062442 (0.0005) [2023-12-26 23:06:18,253][105620] Updated weights for policy 1, policy_version 1062452 (0.0005) [2023-12-26 23:06:18,311][105620] Updated weights for policy 1, policy_version 1062462 (0.0007) [2023-12-26 23:06:18,865][105692] Updated weights for policy 0, policy_version 1061448 (0.0010) [2023-12-26 23:06:18,920][105692] Updated weights for policy 0, policy_version 1061458 (0.0010) [2023-12-26 23:06:18,969][105692] Updated weights for policy 0, policy_version 1061468 (0.0010) [2023-12-26 23:06:19,014][105620] Updated weights for policy 1, policy_version 1062472 (0.0009) [2023-12-26 23:06:19,071][105620] Updated weights for policy 1, policy_version 1062482 (0.0010) [2023-12-26 23:06:19,123][105620] Updated weights for policy 1, policy_version 1062492 (0.0008) [2023-12-26 23:06:19,696][105692] Updated weights for policy 0, policy_version 1061478 (0.0011) [2023-12-26 23:06:19,763][105692] Updated weights for policy 0, policy_version 1061488 (0.0011) [2023-12-26 23:06:19,834][105692] Updated weights for policy 0, policy_version 1061498 (0.0013) [2023-12-26 23:06:19,967][105620] Updated weights for policy 1, policy_version 1062502 (0.0008) [2023-12-26 23:06:20,026][105620] Updated weights for policy 1, policy_version 1062512 (0.0007) [2023-12-26 23:06:20,083][105620] Updated weights for policy 1, policy_version 1062522 (0.0008) [2023-12-26 23:06:20,581][105692] Updated weights for policy 0, policy_version 1061508 (0.0009) [2023-12-26 23:06:20,643][105692] Updated weights for policy 0, policy_version 1061518 (0.0010) [2023-12-26 23:06:20,703][105692] Updated weights for policy 0, policy_version 1061528 (0.0011) [2023-12-26 23:06:20,859][105620] Updated weights for policy 1, policy_version 1062532 (0.0008) [2023-12-26 23:06:20,924][105620] Updated weights for policy 1, policy_version 1062542 (0.0006) [2023-12-26 23:06:20,987][105620] Updated weights for policy 1, policy_version 1062552 (0.0005) [2023-12-26 23:06:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 543842304. Throughput: 0: 9602.8, 1: 10080.4. Samples: 543828416. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:06:21,062][104569] Avg episode reward: [(0, '8818.295'), (1, '9351.407')] [2023-12-26 23:06:21,515][105692] Updated weights for policy 0, policy_version 1061538 (0.0011) [2023-12-26 23:06:21,570][105692] Updated weights for policy 0, policy_version 1061548 (0.0011) [2023-12-26 23:06:21,623][105692] Updated weights for policy 0, policy_version 1061558 (0.0010) [2023-12-26 23:06:21,689][105692] Updated weights for policy 0, policy_version 1061568 (0.0010) [2023-12-26 23:06:21,705][105620] Updated weights for policy 1, policy_version 1062562 (0.0007) [2023-12-26 23:06:21,777][105620] Updated weights for policy 1, policy_version 1062572 (0.0008) [2023-12-26 23:06:21,835][105620] Updated weights for policy 1, policy_version 1062582 (0.0008) [2023-12-26 23:06:21,894][105620] Updated weights for policy 1, policy_version 1062592 (0.0006) [2023-12-26 23:06:22,432][105692] Updated weights for policy 0, policy_version 1061578 (0.0008) [2023-12-26 23:06:22,498][105692] Updated weights for policy 0, policy_version 1061588 (0.0008) [2023-12-26 23:06:22,570][105692] Updated weights for policy 0, policy_version 1061598 (0.0009) [2023-12-26 23:06:22,638][105620] Updated weights for policy 1, policy_version 1062602 (0.0007) [2023-12-26 23:06:22,707][105620] Updated weights for policy 1, policy_version 1062612 (0.0007) [2023-12-26 23:06:22,764][105620] Updated weights for policy 1, policy_version 1062622 (0.0007) [2023-12-26 23:06:23,292][105692] Updated weights for policy 0, policy_version 1061608 (0.0009) [2023-12-26 23:06:23,340][105692] Updated weights for policy 0, policy_version 1061618 (0.0008) [2023-12-26 23:06:23,406][105692] Updated weights for policy 0, policy_version 1061628 (0.0008) [2023-12-26 23:06:23,456][105620] Updated weights for policy 1, policy_version 1062632 (0.0008) [2023-12-26 23:06:23,506][105620] Updated weights for policy 1, policy_version 1062642 (0.0008) [2023-12-26 23:06:23,559][105620] Updated weights for policy 1, policy_version 1062652 (0.0008) [2023-12-26 23:06:24,170][105692] Updated weights for policy 0, policy_version 1061638 (0.0007) [2023-12-26 23:06:24,220][105692] Updated weights for policy 0, policy_version 1061648 (0.0008) [2023-12-26 23:06:24,280][105692] Updated weights for policy 0, policy_version 1061658 (0.0008) [2023-12-26 23:06:24,316][105620] Updated weights for policy 1, policy_version 1062662 (0.0007) [2023-12-26 23:06:24,385][105620] Updated weights for policy 1, policy_version 1062672 (0.0006) [2023-12-26 23:06:24,441][105620] Updated weights for policy 1, policy_version 1062682 (0.0010) [2023-12-26 23:06:24,957][105692] Updated weights for policy 0, policy_version 1061668 (0.0008) [2023-12-26 23:06:25,020][105692] Updated weights for policy 0, policy_version 1061678 (0.0006) [2023-12-26 23:06:25,080][105620] Updated weights for policy 1, policy_version 1062692 (0.0010) [2023-12-26 23:06:25,081][105692] Updated weights for policy 0, policy_version 1061688 (0.0006) [2023-12-26 23:06:25,138][105620] Updated weights for policy 1, policy_version 1062702 (0.0010) [2023-12-26 23:06:25,199][105620] Updated weights for policy 1, policy_version 1062712 (0.0010) [2023-12-26 23:06:25,664][105692] Updated weights for policy 0, policy_version 1061698 (0.0006) [2023-12-26 23:06:25,716][105692] Updated weights for policy 0, policy_version 1061708 (0.0010) [2023-12-26 23:06:25,775][105692] Updated weights for policy 0, policy_version 1061718 (0.0010) [2023-12-26 23:06:25,833][105692] Updated weights for policy 0, policy_version 1061728 (0.0010) [2023-12-26 23:06:25,923][105620] Updated weights for policy 1, policy_version 1062722 (0.0010) [2023-12-26 23:06:25,984][105620] Updated weights for policy 1, policy_version 1062732 (0.0010) [2023-12-26 23:06:26,041][105620] Updated weights for policy 1, policy_version 1062742 (0.0010) [2023-12-26 23:06:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 543932416. Throughput: 0: 9589.7, 1: 10018.2. Samples: 543944196. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:06:26,062][104569] Avg episode reward: [(0, '8364.755'), (1, '9258.424')] [2023-12-26 23:06:26,102][105620] Updated weights for policy 1, policy_version 1062752 (0.0010) [2023-12-26 23:06:26,516][105692] Updated weights for policy 0, policy_version 1061738 (0.0009) [2023-12-26 23:06:26,566][105692] Updated weights for policy 0, policy_version 1061749 (0.0009) [2023-12-26 23:06:26,620][105692] Updated weights for policy 0, policy_version 1061760 (0.0007) [2023-12-26 23:06:26,771][105620] Updated weights for policy 1, policy_version 1062762 (0.0010) [2023-12-26 23:06:26,838][105620] Updated weights for policy 1, policy_version 1062772 (0.0008) [2023-12-26 23:06:26,907][105620] Updated weights for policy 1, policy_version 1062782 (0.0009) [2023-12-26 23:06:27,270][105692] Updated weights for policy 0, policy_version 1061770 (0.0005) [2023-12-26 23:06:27,327][105692] Updated weights for policy 0, policy_version 1061780 (0.0005) [2023-12-26 23:06:27,374][105692] Updated weights for policy 0, policy_version 1061790 (0.0005) [2023-12-26 23:06:27,544][105620] Updated weights for policy 1, policy_version 1062792 (0.0010) [2023-12-26 23:06:27,588][105620] Updated weights for policy 1, policy_version 1062802 (0.0010) [2023-12-26 23:06:27,632][105620] Updated weights for policy 1, policy_version 1062812 (0.0010) [2023-12-26 23:06:27,927][105692] Updated weights for policy 0, policy_version 1061800 (0.0009) [2023-12-26 23:06:27,984][105692] Updated weights for policy 0, policy_version 1061810 (0.0010) [2023-12-26 23:06:28,038][105692] Updated weights for policy 0, policy_version 1061820 (0.0010) [2023-12-26 23:06:28,376][105620] Updated weights for policy 1, policy_version 1062822 (0.0010) [2023-12-26 23:06:28,428][105620] Updated weights for policy 1, policy_version 1062832 (0.0010) [2023-12-26 23:06:28,486][105620] Updated weights for policy 1, policy_version 1062842 (0.0010) [2023-12-26 23:06:28,772][105692] Updated weights for policy 0, policy_version 1061830 (0.0010) [2023-12-26 23:06:28,823][105692] Updated weights for policy 0, policy_version 1061840 (0.0010) [2023-12-26 23:06:28,871][105692] Updated weights for policy 0, policy_version 1061850 (0.0010) [2023-12-26 23:06:29,233][105620] Updated weights for policy 1, policy_version 1062852 (0.0009) [2023-12-26 23:06:29,288][105620] Updated weights for policy 1, policy_version 1062862 (0.0010) [2023-12-26 23:06:29,346][105620] Updated weights for policy 1, policy_version 1062872 (0.0010) [2023-12-26 23:06:29,568][105692] Updated weights for policy 0, policy_version 1061860 (0.0010) [2023-12-26 23:06:29,617][105692] Updated weights for policy 0, policy_version 1061870 (0.0010) [2023-12-26 23:06:29,668][105692] Updated weights for policy 0, policy_version 1061880 (0.0010) [2023-12-26 23:06:30,065][105620] Updated weights for policy 1, policy_version 1062882 (0.0010) [2023-12-26 23:06:30,120][105620] Updated weights for policy 1, policy_version 1062892 (0.0010) [2023-12-26 23:06:30,172][105620] Updated weights for policy 1, policy_version 1062902 (0.0010) [2023-12-26 23:06:30,220][105620] Updated weights for policy 1, policy_version 1062912 (0.0010) [2023-12-26 23:06:30,336][105692] Updated weights for policy 0, policy_version 1061890 (0.0007) [2023-12-26 23:06:30,392][105692] Updated weights for policy 0, policy_version 1061900 (0.0006) [2023-12-26 23:06:30,444][105692] Updated weights for policy 0, policy_version 1061910 (0.0005) [2023-12-26 23:06:30,495][105692] Updated weights for policy 0, policy_version 1061920 (0.0006) [2023-12-26 23:06:30,963][105620] Updated weights for policy 1, policy_version 1062922 (0.0009) [2023-12-26 23:06:31,026][105620] Updated weights for policy 1, policy_version 1062932 (0.0008) [2023-12-26 23:06:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 544030720. Throughput: 0: 9689.2, 1: 10060.6. Samples: 544006068. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:06:31,062][104569] Avg episode reward: [(0, '7744.387'), (1, '9350.207')] [2023-12-26 23:06:31,086][105620] Updated weights for policy 1, policy_version 1062942 (0.0010) [2023-12-26 23:06:31,086][105692] Updated weights for policy 0, policy_version 1061930 (0.0009) [2023-12-26 23:06:31,098][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001062944_272146432.pth... [2023-12-26 23:06:31,103][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001061768_271843328.pth [2023-12-26 23:06:31,146][105692] Updated weights for policy 0, policy_version 1061940 (0.0010) [2023-12-26 23:06:31,205][105692] Updated weights for policy 0, policy_version 1061950 (0.0011) [2023-12-26 23:06:31,214][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001061952_271900672.pth... [2023-12-26 23:06:31,219][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001060800_271605760.pth [2023-12-26 23:06:31,793][105620] Updated weights for policy 1, policy_version 1062952 (0.0010) [2023-12-26 23:06:31,852][105620] Updated weights for policy 1, policy_version 1062962 (0.0010) [2023-12-26 23:06:31,914][105620] Updated weights for policy 1, policy_version 1062972 (0.0009) [2023-12-26 23:06:31,932][105692] Updated weights for policy 0, policy_version 1061960 (0.0010) [2023-12-26 23:06:31,988][105692] Updated weights for policy 0, policy_version 1061970 (0.0011) [2023-12-26 23:06:32,036][105692] Updated weights for policy 0, policy_version 1061980 (0.0010) [2023-12-26 23:06:32,638][105620] Updated weights for policy 1, policy_version 1062982 (0.0009) [2023-12-26 23:06:32,645][105692] Updated weights for policy 0, policy_version 1061990 (0.0011) [2023-12-26 23:06:32,700][105692] Updated weights for policy 0, policy_version 1062000 (0.0010) [2023-12-26 23:06:32,702][105620] Updated weights for policy 1, policy_version 1062992 (0.0010) [2023-12-26 23:06:32,759][105692] Updated weights for policy 0, policy_version 1062010 (0.0010) [2023-12-26 23:06:32,761][105620] Updated weights for policy 1, policy_version 1063002 (0.0011) [2023-12-26 23:06:33,360][105692] Updated weights for policy 0, policy_version 1062020 (0.0008) [2023-12-26 23:06:33,412][105692] Updated weights for policy 0, policy_version 1062030 (0.0005) [2023-12-26 23:06:33,467][105692] Updated weights for policy 0, policy_version 1062040 (0.0005) [2023-12-26 23:06:33,503][105620] Updated weights for policy 1, policy_version 1063012 (0.0010) [2023-12-26 23:06:33,557][105620] Updated weights for policy 1, policy_version 1063022 (0.0010) [2023-12-26 23:06:33,622][105620] Updated weights for policy 1, policy_version 1063032 (0.0010) [2023-12-26 23:06:34,027][105692] Updated weights for policy 0, policy_version 1062050 (0.0006) [2023-12-26 23:06:34,094][105692] Updated weights for policy 0, policy_version 1062060 (0.0010) [2023-12-26 23:06:34,160][105692] Updated weights for policy 0, policy_version 1062070 (0.0010) [2023-12-26 23:06:34,225][105692] Updated weights for policy 0, policy_version 1062080 (0.0010) [2023-12-26 23:06:34,370][105620] Updated weights for policy 1, policy_version 1063042 (0.0010) [2023-12-26 23:06:34,437][105620] Updated weights for policy 1, policy_version 1063052 (0.0010) [2023-12-26 23:06:34,503][105620] Updated weights for policy 1, policy_version 1063062 (0.0011) [2023-12-26 23:06:34,571][105620] Updated weights for policy 1, policy_version 1063072 (0.0011) [2023-12-26 23:06:34,955][105692] Updated weights for policy 0, policy_version 1062090 (0.0007) [2023-12-26 23:06:35,022][105692] Updated weights for policy 0, policy_version 1062100 (0.0009) [2023-12-26 23:06:35,080][105692] Updated weights for policy 0, policy_version 1062110 (0.0010) [2023-12-26 23:06:35,282][105620] Updated weights for policy 1, policy_version 1063082 (0.0010) [2023-12-26 23:06:35,331][105620] Updated weights for policy 1, policy_version 1063092 (0.0010) [2023-12-26 23:06:35,391][105620] Updated weights for policy 1, policy_version 1063102 (0.0010) [2023-12-26 23:06:35,790][105692] Updated weights for policy 0, policy_version 1062120 (0.0010) [2023-12-26 23:06:35,847][105692] Updated weights for policy 0, policy_version 1062130 (0.0006) [2023-12-26 23:06:35,899][105692] Updated weights for policy 0, policy_version 1062140 (0.0007) [2023-12-26 23:06:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 544137216. Throughput: 0: 9774.1, 1: 9850.1. Samples: 544126800. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:06:36,062][104569] Avg episode reward: [(0, '7663.085'), (1, '9350.493')] [2023-12-26 23:06:36,156][105620] Updated weights for policy 1, policy_version 1063112 (0.0010) [2023-12-26 23:06:36,224][105620] Updated weights for policy 1, policy_version 1063122 (0.0009) [2023-12-26 23:06:36,284][105620] Updated weights for policy 1, policy_version 1063132 (0.0011) [2023-12-26 23:06:36,578][105692] Updated weights for policy 0, policy_version 1062150 (0.0008) [2023-12-26 23:06:36,641][105692] Updated weights for policy 0, policy_version 1062160 (0.0010) [2023-12-26 23:06:36,704][105692] Updated weights for policy 0, policy_version 1062170 (0.0010) [2023-12-26 23:06:37,022][105620] Updated weights for policy 1, policy_version 1063142 (0.0011) [2023-12-26 23:06:37,093][105620] Updated weights for policy 1, policy_version 1063152 (0.0011) [2023-12-26 23:06:37,162][105620] Updated weights for policy 1, policy_version 1063162 (0.0010) [2023-12-26 23:06:37,383][105692] Updated weights for policy 0, policy_version 1062180 (0.0008) [2023-12-26 23:06:37,442][105692] Updated weights for policy 0, policy_version 1062190 (0.0005) [2023-12-26 23:06:37,499][105692] Updated weights for policy 0, policy_version 1062200 (0.0005) [2023-12-26 23:06:37,894][105620] Updated weights for policy 1, policy_version 1063172 (0.0009) [2023-12-26 23:06:37,952][105620] Updated weights for policy 1, policy_version 1063182 (0.0005) [2023-12-26 23:06:38,003][105620] Updated weights for policy 1, policy_version 1063192 (0.0005) [2023-12-26 23:06:38,139][105692] Updated weights for policy 0, policy_version 1062210 (0.0006) [2023-12-26 23:06:38,188][105692] Updated weights for policy 0, policy_version 1062220 (0.0008) [2023-12-26 23:06:38,237][105692] Updated weights for policy 0, policy_version 1062230 (0.0008) [2023-12-26 23:06:38,286][105692] Updated weights for policy 0, policy_version 1062240 (0.0008) [2023-12-26 23:06:38,691][105620] Updated weights for policy 1, policy_version 1063202 (0.0006) [2023-12-26 23:06:38,749][105620] Updated weights for policy 1, policy_version 1063212 (0.0009) [2023-12-26 23:06:38,808][105620] Updated weights for policy 1, policy_version 1063222 (0.0009) [2023-12-26 23:06:38,863][105620] Updated weights for policy 1, policy_version 1063232 (0.0007) [2023-12-26 23:06:39,079][105692] Updated weights for policy 0, policy_version 1062250 (0.0009) [2023-12-26 23:06:39,135][105692] Updated weights for policy 0, policy_version 1062260 (0.0009) [2023-12-26 23:06:39,197][105692] Updated weights for policy 0, policy_version 1062270 (0.0009) [2023-12-26 23:06:39,591][105620] Updated weights for policy 1, policy_version 1063242 (0.0008) [2023-12-26 23:06:39,656][105620] Updated weights for policy 1, policy_version 1063252 (0.0008) [2023-12-26 23:06:39,723][105620] Updated weights for policy 1, policy_version 1063262 (0.0010) [2023-12-26 23:06:40,011][105692] Updated weights for policy 0, policy_version 1062280 (0.0008) [2023-12-26 23:06:40,078][105692] Updated weights for policy 0, policy_version 1062290 (0.0005) [2023-12-26 23:06:40,142][105692] Updated weights for policy 0, policy_version 1062300 (0.0008) [2023-12-26 23:06:40,511][105620] Updated weights for policy 1, policy_version 1063272 (0.0006) [2023-12-26 23:06:40,571][105620] Updated weights for policy 1, policy_version 1063282 (0.0005) [2023-12-26 23:06:40,627][105620] Updated weights for policy 1, policy_version 1063292 (0.0008) [2023-12-26 23:06:40,865][105692] Updated weights for policy 0, policy_version 1062310 (0.0009) [2023-12-26 23:06:40,888][105585] KL-divergence is very high: 142.8220 [2023-12-26 23:06:40,937][105692] Updated weights for policy 0, policy_version 1062320 (0.0007) [2023-12-26 23:06:40,946][105585] KL-divergence is very high: 235.2571 [2023-12-26 23:06:40,996][105585] KL-divergence is very high: 240.2291 [2023-12-26 23:06:41,000][105692] Updated weights for policy 0, policy_version 1062330 (0.0005) [2023-12-26 23:06:41,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 544235520. Throughput: 0: 9806.8, 1: 9835.9. Samples: 544242676. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:06:41,062][104569] Avg episode reward: [(0, '6843.158'), (1, '9350.996')] [2023-12-26 23:06:41,200][105620] Updated weights for policy 1, policy_version 1063302 (0.0009) [2023-12-26 23:06:41,256][105620] Updated weights for policy 1, policy_version 1063312 (0.0008) [2023-12-26 23:06:41,319][105620] Updated weights for policy 1, policy_version 1063322 (0.0008) [2023-12-26 23:06:41,766][105692] Updated weights for policy 0, policy_version 1062340 (0.0008) [2023-12-26 23:06:41,838][105692] Updated weights for policy 0, policy_version 1062350 (0.0011) [2023-12-26 23:06:41,906][105692] Updated weights for policy 0, policy_version 1062360 (0.0009) [2023-12-26 23:06:42,080][105620] Updated weights for policy 1, policy_version 1063332 (0.0009) [2023-12-26 23:06:42,138][105620] Updated weights for policy 1, policy_version 1063342 (0.0007) [2023-12-26 23:06:42,196][105620] Updated weights for policy 1, policy_version 1063352 (0.0009) [2023-12-26 23:06:42,702][105692] Updated weights for policy 0, policy_version 1062370 (0.0010) [2023-12-26 23:06:42,751][105692] Updated weights for policy 0, policy_version 1062380 (0.0009) [2023-12-26 23:06:42,808][105692] Updated weights for policy 0, policy_version 1062390 (0.0009) [2023-12-26 23:06:42,862][105692] Updated weights for policy 0, policy_version 1062400 (0.0009) [2023-12-26 23:06:42,954][105620] Updated weights for policy 1, policy_version 1063362 (0.0009) [2023-12-26 23:06:43,003][105620] Updated weights for policy 1, policy_version 1063372 (0.0005) [2023-12-26 23:06:43,057][105620] Updated weights for policy 1, policy_version 1063382 (0.0005) [2023-12-26 23:06:43,111][105620] Updated weights for policy 1, policy_version 1063392 (0.0006) [2023-12-26 23:06:43,585][105692] Updated weights for policy 0, policy_version 1062410 (0.0010) [2023-12-26 23:06:43,641][105692] Updated weights for policy 0, policy_version 1062420 (0.0010) [2023-12-26 23:06:43,697][105692] Updated weights for policy 0, policy_version 1062430 (0.0011) [2023-12-26 23:06:43,722][105620] Updated weights for policy 1, policy_version 1063402 (0.0006) [2023-12-26 23:06:43,787][105620] Updated weights for policy 1, policy_version 1063412 (0.0009) [2023-12-26 23:06:43,851][105620] Updated weights for policy 1, policy_version 1063422 (0.0010) [2023-12-26 23:06:44,334][105692] Updated weights for policy 0, policy_version 1062440 (0.0006) [2023-12-26 23:06:44,380][105692] Updated weights for policy 0, policy_version 1062450 (0.0005) [2023-12-26 23:06:44,430][105692] Updated weights for policy 0, policy_version 1062460 (0.0006) [2023-12-26 23:06:44,592][105620] Updated weights for policy 1, policy_version 1063432 (0.0010) [2023-12-26 23:06:44,651][105620] Updated weights for policy 1, policy_version 1063442 (0.0010) [2023-12-26 23:06:44,705][105620] Updated weights for policy 1, policy_version 1063452 (0.0010) [2023-12-26 23:06:45,161][105692] Updated weights for policy 0, policy_version 1062470 (0.0011) [2023-12-26 23:06:45,232][105692] Updated weights for policy 0, policy_version 1062480 (0.0011) [2023-12-26 23:06:45,298][105692] Updated weights for policy 0, policy_version 1062490 (0.0009) [2023-12-26 23:06:45,303][105585] KL-divergence is very high: 143.3396 [2023-12-26 23:06:45,456][105620] Updated weights for policy 1, policy_version 1063462 (0.0010) [2023-12-26 23:06:45,518][105620] Updated weights for policy 1, policy_version 1063472 (0.0011) [2023-12-26 23:06:45,572][105620] Updated weights for policy 1, policy_version 1063482 (0.0011) [2023-12-26 23:06:46,028][105692] Updated weights for policy 0, policy_version 1062500 (0.0010) [2023-12-26 23:06:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 544325632. Throughput: 0: 9754.6, 1: 9855.1. Samples: 544299772. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:06:46,063][104569] Avg episode reward: [(0, '6514.328'), (1, '9258.831')] [2023-12-26 23:06:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001063488_272285696.pth... [2023-12-26 23:06:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001062368_271998976.pth [2023-12-26 23:06:46,076][105692] Updated weights for policy 0, policy_version 1062510 (0.0010) [2023-12-26 23:06:46,123][105692] Updated weights for policy 0, policy_version 1062520 (0.0010) [2023-12-26 23:06:46,162][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001062528_272048128.pth... [2023-12-26 23:06:46,166][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001061344_271745024.pth [2023-12-26 23:06:46,256][105620] Updated weights for policy 1, policy_version 1063492 (0.0008) [2023-12-26 23:06:46,311][105620] Updated weights for policy 1, policy_version 1063502 (0.0005) [2023-12-26 23:06:46,370][105620] Updated weights for policy 1, policy_version 1063512 (0.0005) [2023-12-26 23:06:46,704][105692] Updated weights for policy 0, policy_version 1062530 (0.0009) [2023-12-26 23:06:46,768][105692] Updated weights for policy 0, policy_version 1062540 (0.0009) [2023-12-26 23:06:46,822][105692] Updated weights for policy 0, policy_version 1062550 (0.0006) [2023-12-26 23:06:46,871][105692] Updated weights for policy 0, policy_version 1062560 (0.0005) [2023-12-26 23:06:46,888][105620] Updated weights for policy 1, policy_version 1063522 (0.0005) [2023-12-26 23:06:46,956][105620] Updated weights for policy 1, policy_version 1063532 (0.0005) [2023-12-26 23:06:47,018][105620] Updated weights for policy 1, policy_version 1063542 (0.0007) [2023-12-26 23:06:47,080][105620] Updated weights for policy 1, policy_version 1063552 (0.0010) [2023-12-26 23:06:47,478][105692] Updated weights for policy 0, policy_version 1062570 (0.0005) [2023-12-26 23:06:47,531][105692] Updated weights for policy 0, policy_version 1062580 (0.0005) [2023-12-26 23:06:47,590][105692] Updated weights for policy 0, policy_version 1062590 (0.0005) [2023-12-26 23:06:47,715][105620] Updated weights for policy 1, policy_version 1063562 (0.0010) [2023-12-26 23:06:47,763][105620] Updated weights for policy 1, policy_version 1063572 (0.0010) [2023-12-26 23:06:47,808][105620] Updated weights for policy 1, policy_version 1063582 (0.0010) [2023-12-26 23:06:48,251][105692] Updated weights for policy 0, policy_version 1062600 (0.0009) [2023-12-26 23:06:48,304][105692] Updated weights for policy 0, policy_version 1062610 (0.0008) [2023-12-26 23:06:48,366][105692] Updated weights for policy 0, policy_version 1062620 (0.0008) [2023-12-26 23:06:48,606][105620] Updated weights for policy 1, policy_version 1063592 (0.0009) [2023-12-26 23:06:48,663][105620] Updated weights for policy 1, policy_version 1063602 (0.0009) [2023-12-26 23:06:48,724][105620] Updated weights for policy 1, policy_version 1063612 (0.0009) [2023-12-26 23:06:49,109][105692] Updated weights for policy 0, policy_version 1062630 (0.0009) [2023-12-26 23:06:49,164][105692] Updated weights for policy 0, policy_version 1062640 (0.0009) [2023-12-26 23:06:49,217][105692] Updated weights for policy 0, policy_version 1062650 (0.0008) [2023-12-26 23:06:49,486][105620] Updated weights for policy 1, policy_version 1063622 (0.0009) [2023-12-26 23:06:49,532][105620] Updated weights for policy 1, policy_version 1063632 (0.0008) [2023-12-26 23:06:49,579][105620] Updated weights for policy 1, policy_version 1063642 (0.0008) [2023-12-26 23:06:50,002][105692] Updated weights for policy 0, policy_version 1062660 (0.0008) [2023-12-26 23:06:50,061][105692] Updated weights for policy 0, policy_version 1062670 (0.0008) [2023-12-26 23:06:50,125][105692] Updated weights for policy 0, policy_version 1062680 (0.0009) [2023-12-26 23:06:50,330][105620] Updated weights for policy 1, policy_version 1063652 (0.0007) [2023-12-26 23:06:50,397][105620] Updated weights for policy 1, policy_version 1063662 (0.0006) [2023-12-26 23:06:50,467][105620] Updated weights for policy 1, policy_version 1063672 (0.0005) [2023-12-26 23:06:50,882][105692] Updated weights for policy 0, policy_version 1062690 (0.0010) [2023-12-26 23:06:50,942][105692] Updated weights for policy 0, policy_version 1062700 (0.0005) [2023-12-26 23:06:51,004][105692] Updated weights for policy 0, policy_version 1062710 (0.0005) [2023-12-26 23:06:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 544423936. Throughput: 0: 9859.6, 1: 9793.1. Samples: 544420096. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:06:51,062][104569] Avg episode reward: [(0, '7150.066'), (1, '9350.304')] [2023-12-26 23:06:51,068][105692] Updated weights for policy 0, policy_version 1062720 (0.0007) [2023-12-26 23:06:51,177][105620] Updated weights for policy 1, policy_version 1063682 (0.0007) [2023-12-26 23:06:51,247][105620] Updated weights for policy 1, policy_version 1063692 (0.0011) [2023-12-26 23:06:51,303][105620] Updated weights for policy 1, policy_version 1063702 (0.0011) [2023-12-26 23:06:51,367][105620] Updated weights for policy 1, policy_version 1063712 (0.0010) [2023-12-26 23:06:51,763][105692] Updated weights for policy 0, policy_version 1062730 (0.0010) [2023-12-26 23:06:51,823][105692] Updated weights for policy 0, policy_version 1062740 (0.0010) [2023-12-26 23:06:51,882][105692] Updated weights for policy 0, policy_version 1062750 (0.0011) [2023-12-26 23:06:52,058][105620] Updated weights for policy 1, policy_version 1063722 (0.0011) [2023-12-26 23:06:52,119][105620] Updated weights for policy 1, policy_version 1063732 (0.0010) [2023-12-26 23:06:52,206][105620] Updated weights for policy 1, policy_version 1063742 (0.0010) [2023-12-26 23:06:52,618][105692] Updated weights for policy 0, policy_version 1062760 (0.0007) [2023-12-26 23:06:52,685][105692] Updated weights for policy 0, policy_version 1062770 (0.0005) [2023-12-26 23:06:52,748][105692] Updated weights for policy 0, policy_version 1062780 (0.0005) [2023-12-26 23:06:52,948][105620] Updated weights for policy 1, policy_version 1063752 (0.0010) [2023-12-26 23:06:53,000][105620] Updated weights for policy 1, policy_version 1063762 (0.0010) [2023-12-26 23:06:53,059][105620] Updated weights for policy 1, policy_version 1063772 (0.0010) [2023-12-26 23:06:53,351][105692] Updated weights for policy 0, policy_version 1062790 (0.0006) [2023-12-26 23:06:53,412][105692] Updated weights for policy 0, policy_version 1062800 (0.0005) [2023-12-26 23:06:53,469][105692] Updated weights for policy 0, policy_version 1062810 (0.0010) [2023-12-26 23:06:53,805][105620] Updated weights for policy 1, policy_version 1063782 (0.0008) [2023-12-26 23:06:53,871][105620] Updated weights for policy 1, policy_version 1063792 (0.0007) [2023-12-26 23:06:53,939][105620] Updated weights for policy 1, policy_version 1063802 (0.0010) [2023-12-26 23:06:54,171][105692] Updated weights for policy 0, policy_version 1062820 (0.0008) [2023-12-26 23:06:54,224][105692] Updated weights for policy 0, policy_version 1062830 (0.0006) [2023-12-26 23:06:54,277][105692] Updated weights for policy 0, policy_version 1062840 (0.0007) [2023-12-26 23:06:54,636][105620] Updated weights for policy 1, policy_version 1063812 (0.0011) [2023-12-26 23:06:54,698][105620] Updated weights for policy 1, policy_version 1063822 (0.0010) [2023-12-26 23:06:54,753][105620] Updated weights for policy 1, policy_version 1063832 (0.0010) [2023-12-26 23:06:54,945][105692] Updated weights for policy 0, policy_version 1062850 (0.0009) [2023-12-26 23:06:55,004][105692] Updated weights for policy 0, policy_version 1062860 (0.0006) [2023-12-26 23:06:55,055][105692] Updated weights for policy 0, policy_version 1062870 (0.0006) [2023-12-26 23:06:55,107][105692] Updated weights for policy 0, policy_version 1062880 (0.0005) [2023-12-26 23:06:55,489][105620] Updated weights for policy 1, policy_version 1063842 (0.0010) [2023-12-26 23:06:55,548][105620] Updated weights for policy 1, policy_version 1063852 (0.0010) [2023-12-26 23:06:55,606][105620] Updated weights for policy 1, policy_version 1063862 (0.0010) [2023-12-26 23:06:55,660][105620] Updated weights for policy 1, policy_version 1063872 (0.0010) [2023-12-26 23:06:55,767][105692] Updated weights for policy 0, policy_version 1062890 (0.0010) [2023-12-26 23:06:55,818][105692] Updated weights for policy 0, policy_version 1062900 (0.0010) [2023-12-26 23:06:55,865][105692] Updated weights for policy 0, policy_version 1062910 (0.0010) [2023-12-26 23:06:56,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 544530432. Throughput: 0: 9954.0, 1: 9707.9. Samples: 544537560. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:06:56,063][104569] Avg episode reward: [(0, '7147.092'), (1, '9349.848')] [2023-12-26 23:06:56,314][105620] Updated weights for policy 1, policy_version 1063882 (0.0011) [2023-12-26 23:06:56,371][105620] Updated weights for policy 1, policy_version 1063892 (0.0008) [2023-12-26 23:06:56,424][105620] Updated weights for policy 1, policy_version 1063902 (0.0008) [2023-12-26 23:06:56,580][105692] Updated weights for policy 0, policy_version 1062920 (0.0009) [2023-12-26 23:06:56,634][105692] Updated weights for policy 0, policy_version 1062930 (0.0010) [2023-12-26 23:06:56,655][105585] KL-divergence is very high: 156.0566 [2023-12-26 23:06:56,685][105692] Updated weights for policy 0, policy_version 1062940 (0.0010) [2023-12-26 23:06:56,692][105585] KL-divergence is very high: 165.1735 [2023-12-26 23:06:57,033][105620] Updated weights for policy 1, policy_version 1063912 (0.0006) [2023-12-26 23:06:57,076][105620] Updated weights for policy 1, policy_version 1063922 (0.0005) [2023-12-26 23:06:57,122][105620] Updated weights for policy 1, policy_version 1063932 (0.0005) [2023-12-26 23:06:57,335][105692] Updated weights for policy 0, policy_version 1062950 (0.0009) [2023-12-26 23:06:57,397][105692] Updated weights for policy 0, policy_version 1062960 (0.0006) [2023-12-26 23:06:57,458][105692] Updated weights for policy 0, policy_version 1062970 (0.0008) [2023-12-26 23:06:57,692][105620] Updated weights for policy 1, policy_version 1063942 (0.0008) [2023-12-26 23:06:57,750][105620] Updated weights for policy 1, policy_version 1063952 (0.0010) [2023-12-26 23:06:57,807][105620] Updated weights for policy 1, policy_version 1063962 (0.0010) [2023-12-26 23:06:58,233][105692] Updated weights for policy 0, policy_version 1062981 (0.0010) [2023-12-26 23:06:58,266][105585] KL-divergence is very high: 137.3110 [2023-12-26 23:06:58,298][105692] Updated weights for policy 0, policy_version 1062991 (0.0008) [2023-12-26 23:06:58,299][105585] KL-divergence is very high: 152.4982 [2023-12-26 23:06:58,306][105585] KL-divergence is very high: 112.5337 [2023-12-26 23:06:58,319][105585] KL-divergence is very high: 206.0752 [2023-12-26 23:06:58,354][105585] KL-divergence is very high: 144.7556 [2023-12-26 23:06:58,368][105692] Updated weights for policy 0, policy_version 1063001 (0.0009) [2023-12-26 23:06:58,375][105585] KL-divergence is very high: 162.2969 [2023-12-26 23:06:58,559][105620] Updated weights for policy 1, policy_version 1063972 (0.0010) [2023-12-26 23:06:58,622][105620] Updated weights for policy 1, policy_version 1063982 (0.0011) [2023-12-26 23:06:58,682][105620] Updated weights for policy 1, policy_version 1063992 (0.0011) [2023-12-26 23:06:59,184][105692] Updated weights for policy 0, policy_version 1063011 (0.0008) [2023-12-26 23:06:59,256][105692] Updated weights for policy 0, policy_version 1063021 (0.0008) [2023-12-26 23:06:59,314][105692] Updated weights for policy 0, policy_version 1063031 (0.0007) [2023-12-26 23:06:59,456][105620] Updated weights for policy 1, policy_version 1064002 (0.0008) [2023-12-26 23:06:59,508][105620] Updated weights for policy 1, policy_version 1064012 (0.0006) [2023-12-26 23:06:59,571][105620] Updated weights for policy 1, policy_version 1064022 (0.0011) [2023-12-26 23:06:59,638][105620] Updated weights for policy 1, policy_version 1064032 (0.0011) [2023-12-26 23:06:59,991][105692] Updated weights for policy 0, policy_version 1063041 (0.0007) [2023-12-26 23:07:00,058][105692] Updated weights for policy 0, policy_version 1063051 (0.0008) [2023-12-26 23:07:00,117][105692] Updated weights for policy 0, policy_version 1063061 (0.0008) [2023-12-26 23:07:00,175][105692] Updated weights for policy 0, policy_version 1063071 (0.0008) [2023-12-26 23:07:00,326][105620] Updated weights for policy 1, policy_version 1064042 (0.0011) [2023-12-26 23:07:00,384][105620] Updated weights for policy 1, policy_version 1064052 (0.0010) [2023-12-26 23:07:00,437][105620] Updated weights for policy 1, policy_version 1064062 (0.0010) [2023-12-26 23:07:00,964][105692] Updated weights for policy 0, policy_version 1063081 (0.0008) [2023-12-26 23:07:01,028][105692] Updated weights for policy 0, policy_version 1063091 (0.0008) [2023-12-26 23:07:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 544620544. Throughput: 0: 9932.0, 1: 9799.6. Samples: 544598224. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:07:01,062][104569] Avg episode reward: [(0, '7220.547'), (1, '9258.164')] [2023-12-26 23:07:01,082][105620] Updated weights for policy 1, policy_version 1064072 (0.0007) [2023-12-26 23:07:01,087][105692] Updated weights for policy 0, policy_version 1063101 (0.0008) [2023-12-26 23:07:01,101][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001063104_272195584.pth... [2023-12-26 23:07:01,104][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001061952_271900672.pth [2023-12-26 23:07:01,146][105620] Updated weights for policy 1, policy_version 1064082 (0.0008) [2023-12-26 23:07:01,206][105620] Updated weights for policy 1, policy_version 1064092 (0.0009) [2023-12-26 23:07:01,222][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001064096_272441344.pth... [2023-12-26 23:07:01,225][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001062944_272146432.pth [2023-12-26 23:07:01,848][105692] Updated weights for policy 0, policy_version 1063111 (0.0005) [2023-12-26 23:07:01,871][105585] KL-divergence is very high: 508.1581 [2023-12-26 23:07:01,894][105692] Updated weights for policy 0, policy_version 1063121 (0.0005) [2023-12-26 23:07:01,911][105585] KL-divergence is very high: 825.7039 [2023-12-26 23:07:01,948][105692] Updated weights for policy 0, policy_version 1063131 (0.0005) [2023-12-26 23:07:01,956][105585] KL-divergence is very high: 876.7904 [2023-12-26 23:07:01,962][105620] Updated weights for policy 1, policy_version 1064102 (0.0007) [2023-12-26 23:07:02,030][105620] Updated weights for policy 1, policy_version 1064112 (0.0006) [2023-12-26 23:07:02,091][105620] Updated weights for policy 1, policy_version 1064122 (0.0005) [2023-12-26 23:07:02,629][105692] Updated weights for policy 0, policy_version 1063141 (0.0008) [2023-12-26 23:07:02,677][105692] Updated weights for policy 0, policy_version 1063151 (0.0010) [2023-12-26 23:07:02,714][105620] Updated weights for policy 1, policy_version 1064132 (0.0007) [2023-12-26 23:07:02,729][105692] Updated weights for policy 0, policy_version 1063161 (0.0010) [2023-12-26 23:07:02,776][105620] Updated weights for policy 1, policy_version 1064142 (0.0007) [2023-12-26 23:07:02,824][105620] Updated weights for policy 1, policy_version 1064152 (0.0008) [2023-12-26 23:07:03,484][105620] Updated weights for policy 1, policy_version 1064162 (0.0008) [2023-12-26 23:07:03,489][105692] Updated weights for policy 0, policy_version 1063171 (0.0010) [2023-12-26 23:07:03,529][105620] Updated weights for policy 1, policy_version 1064172 (0.0009) [2023-12-26 23:07:03,543][105692] Updated weights for policy 0, policy_version 1063181 (0.0010) [2023-12-26 23:07:03,580][105620] Updated weights for policy 1, policy_version 1064182 (0.0005) [2023-12-26 23:07:03,593][105692] Updated weights for policy 0, policy_version 1063191 (0.0010) [2023-12-26 23:07:03,624][105620] Updated weights for policy 1, policy_version 1064192 (0.0008) [2023-12-26 23:07:04,284][105692] Updated weights for policy 0, policy_version 1063201 (0.0010) [2023-12-26 23:07:04,313][105620] Updated weights for policy 1, policy_version 1064202 (0.0008) [2023-12-26 23:07:04,337][105692] Updated weights for policy 0, policy_version 1063211 (0.0010) [2023-12-26 23:07:04,382][105620] Updated weights for policy 1, policy_version 1064212 (0.0008) [2023-12-26 23:07:04,394][105692] Updated weights for policy 0, policy_version 1063221 (0.0011) [2023-12-26 23:07:04,448][105620] Updated weights for policy 1, policy_version 1064222 (0.0007) [2023-12-26 23:07:04,450][105692] Updated weights for policy 0, policy_version 1063231 (0.0011) [2023-12-26 23:07:05,112][105585] KL-divergence is very high: 119.3922 [2023-12-26 23:07:05,118][105692] Updated weights for policy 0, policy_version 1063241 (0.0007) [2023-12-26 23:07:05,150][105585] KL-divergence is very high: 105.2270 [2023-12-26 23:07:05,164][105692] Updated weights for policy 0, policy_version 1063251 (0.0007) [2023-12-26 23:07:05,191][105620] Updated weights for policy 1, policy_version 1064232 (0.0005) [2023-12-26 23:07:05,213][105692] Updated weights for policy 0, policy_version 1063261 (0.0010) [2023-12-26 23:07:05,235][105620] Updated weights for policy 1, policy_version 1064242 (0.0006) [2023-12-26 23:07:05,284][105620] Updated weights for policy 1, policy_version 1064252 (0.0008) [2023-12-26 23:07:05,957][105692] Updated weights for policy 0, policy_version 1063271 (0.0010) [2023-12-26 23:07:06,009][105692] Updated weights for policy 0, policy_version 1063281 (0.0010) [2023-12-26 23:07:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 544718848. Throughput: 0: 9916.7, 1: 9786.1. Samples: 544715044. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:07:06,062][105620] Updated weights for policy 1, policy_version 1064262 (0.0008) [2023-12-26 23:07:06,062][104569] Avg episode reward: [(0, '7519.132'), (1, '9165.534')] [2023-12-26 23:07:06,068][105692] Updated weights for policy 0, policy_version 1063291 (0.0011) [2023-12-26 23:07:06,117][105620] Updated weights for policy 1, policy_version 1064272 (0.0006) [2023-12-26 23:07:06,184][105620] Updated weights for policy 1, policy_version 1064282 (0.0008) [2023-12-26 23:07:06,797][105692] Updated weights for policy 0, policy_version 1063301 (0.0010) [2023-12-26 23:07:06,859][105692] Updated weights for policy 0, policy_version 1063311 (0.0008) [2023-12-26 23:07:06,871][105620] Updated weights for policy 1, policy_version 1064292 (0.0008) [2023-12-26 23:07:06,923][105692] Updated weights for policy 0, policy_version 1063321 (0.0010) [2023-12-26 23:07:06,925][105620] Updated weights for policy 1, policy_version 1064302 (0.0006) [2023-12-26 23:07:06,972][105620] Updated weights for policy 1, policy_version 1064312 (0.0008) [2023-12-26 23:07:07,541][105620] Updated weights for policy 1, policy_version 1064322 (0.0007) [2023-12-26 23:07:07,586][105620] Updated weights for policy 1, policy_version 1064332 (0.0005) [2023-12-26 23:07:07,640][105620] Updated weights for policy 1, policy_version 1064342 (0.0005) [2023-12-26 23:07:07,658][105692] Updated weights for policy 0, policy_version 1063331 (0.0011) [2023-12-26 23:07:07,686][105620] Updated weights for policy 1, policy_version 1064352 (0.0005) [2023-12-26 23:07:07,723][105692] Updated weights for policy 0, policy_version 1063341 (0.0010) [2023-12-26 23:07:07,774][105692] Updated weights for policy 0, policy_version 1063351 (0.0010) [2023-12-26 23:07:08,353][105620] Updated weights for policy 1, policy_version 1064362 (0.0011) [2023-12-26 23:07:08,415][105620] Updated weights for policy 1, policy_version 1064372 (0.0010) [2023-12-26 23:07:08,481][105620] Updated weights for policy 1, policy_version 1064382 (0.0011) [2023-12-26 23:07:08,528][105692] Updated weights for policy 0, policy_version 1063361 (0.0010) [2023-12-26 23:07:08,589][105692] Updated weights for policy 0, policy_version 1063371 (0.0011) [2023-12-26 23:07:08,655][105692] Updated weights for policy 0, policy_version 1063381 (0.0011) [2023-12-26 23:07:08,713][105692] Updated weights for policy 0, policy_version 1063391 (0.0011) [2023-12-26 23:07:09,274][105620] Updated weights for policy 1, policy_version 1064392 (0.0009) [2023-12-26 23:07:09,341][105620] Updated weights for policy 1, policy_version 1064402 (0.0008) [2023-12-26 23:07:09,389][105620] Updated weights for policy 1, policy_version 1064412 (0.0008) [2023-12-26 23:07:09,469][105692] Updated weights for policy 0, policy_version 1063401 (0.0010) [2023-12-26 23:07:09,522][105692] Updated weights for policy 0, policy_version 1063411 (0.0011) [2023-12-26 23:07:09,584][105692] Updated weights for policy 0, policy_version 1063421 (0.0010) [2023-12-26 23:07:10,115][105620] Updated weights for policy 1, policy_version 1064422 (0.0008) [2023-12-26 23:07:10,164][105620] Updated weights for policy 1, policy_version 1064432 (0.0008) [2023-12-26 23:07:10,225][105620] Updated weights for policy 1, policy_version 1064442 (0.0008) [2023-12-26 23:07:10,372][105692] Updated weights for policy 0, policy_version 1063431 (0.0010) [2023-12-26 23:07:10,439][105692] Updated weights for policy 0, policy_version 1063441 (0.0009) [2023-12-26 23:07:10,505][105692] Updated weights for policy 0, policy_version 1063451 (0.0009) [2023-12-26 23:07:11,013][105620] Updated weights for policy 1, policy_version 1064452 (0.0008) [2023-12-26 23:07:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 544817152. Throughput: 0: 9898.7, 1: 9804.2. Samples: 544830832. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:07:11,063][104569] Avg episode reward: [(0, '7992.035'), (1, '9166.527')] [2023-12-26 23:07:11,084][105620] Updated weights for policy 1, policy_version 1064462 (0.0008) [2023-12-26 23:07:11,151][105620] Updated weights for policy 1, policy_version 1064472 (0.0008) [2023-12-26 23:07:11,236][105692] Updated weights for policy 0, policy_version 1063461 (0.0008) [2023-12-26 23:07:11,299][105692] Updated weights for policy 0, policy_version 1063471 (0.0008) [2023-12-26 23:07:11,369][105692] Updated weights for policy 0, policy_version 1063481 (0.0009) [2023-12-26 23:07:11,380][105585] KL-divergence is very high: 117.0746 [2023-12-26 23:07:11,400][105585] KL-divergence is very high: 152.5139 [2023-12-26 23:07:11,908][105620] Updated weights for policy 1, policy_version 1064482 (0.0009) [2023-12-26 23:07:11,964][105620] Updated weights for policy 1, policy_version 1064492 (0.0006) [2023-12-26 23:07:12,019][105620] Updated weights for policy 1, policy_version 1064502 (0.0009) [2023-12-26 23:07:12,074][105692] Updated weights for policy 0, policy_version 1063491 (0.0009) [2023-12-26 23:07:12,078][105620] Updated weights for policy 1, policy_version 1064512 (0.0006) [2023-12-26 23:07:12,135][105692] Updated weights for policy 0, policy_version 1063501 (0.0009) [2023-12-26 23:07:12,200][105692] Updated weights for policy 0, policy_version 1063511 (0.0009) [2023-12-26 23:07:12,781][105620] Updated weights for policy 1, policy_version 1064522 (0.0008) [2023-12-26 23:07:12,842][105620] Updated weights for policy 1, policy_version 1064532 (0.0009) [2023-12-26 23:07:12,906][105620] Updated weights for policy 1, policy_version 1064542 (0.0008) [2023-12-26 23:07:12,965][105692] Updated weights for policy 0, policy_version 1063521 (0.0007) [2023-12-26 23:07:13,024][105692] Updated weights for policy 0, policy_version 1063531 (0.0010) [2023-12-26 23:07:13,077][105692] Updated weights for policy 0, policy_version 1063541 (0.0009) [2023-12-26 23:07:13,137][105692] Updated weights for policy 0, policy_version 1063551 (0.0009) [2023-12-26 23:07:13,601][105620] Updated weights for policy 1, policy_version 1064552 (0.0009) [2023-12-26 23:07:13,649][105620] Updated weights for policy 1, policy_version 1064562 (0.0005) [2023-12-26 23:07:13,702][105620] Updated weights for policy 1, policy_version 1064572 (0.0005) [2023-12-26 23:07:13,861][105692] Updated weights for policy 0, policy_version 1063561 (0.0009) [2023-12-26 23:07:13,914][105692] Updated weights for policy 0, policy_version 1063571 (0.0010) [2023-12-26 23:07:13,968][105692] Updated weights for policy 0, policy_version 1063581 (0.0010) [2023-12-26 23:07:14,389][105620] Updated weights for policy 1, policy_version 1064582 (0.0008) [2023-12-26 23:07:14,450][105620] Updated weights for policy 1, policy_version 1064592 (0.0010) [2023-12-26 23:07:14,510][105620] Updated weights for policy 1, policy_version 1064602 (0.0006) [2023-12-26 23:07:14,662][105692] Updated weights for policy 0, policy_version 1063591 (0.0007) [2023-12-26 23:07:14,714][105692] Updated weights for policy 0, policy_version 1063601 (0.0006) [2023-12-26 23:07:14,762][105692] Updated weights for policy 0, policy_version 1063611 (0.0008) [2023-12-26 23:07:15,253][105620] Updated weights for policy 1, policy_version 1064612 (0.0007) [2023-12-26 23:07:15,321][105620] Updated weights for policy 1, policy_version 1064622 (0.0010) [2023-12-26 23:07:15,369][105620] Updated weights for policy 1, policy_version 1064632 (0.0009) [2023-12-26 23:07:15,486][105692] Updated weights for policy 0, policy_version 1063621 (0.0008) [2023-12-26 23:07:15,550][105692] Updated weights for policy 0, policy_version 1063631 (0.0008) [2023-12-26 23:07:15,611][105692] Updated weights for policy 0, policy_version 1063641 (0.0009) [2023-12-26 23:07:15,998][105620] Updated weights for policy 1, policy_version 1064642 (0.0008) [2023-12-26 23:07:16,059][105620] Updated weights for policy 1, policy_version 1064652 (0.0005) [2023-12-26 23:07:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 544915456. Throughput: 0: 9802.5, 1: 9784.0. Samples: 544887460. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:07:16,062][104569] Avg episode reward: [(0, '8160.945'), (1, '9259.277')] [2023-12-26 23:07:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001063648_272334848.pth... [2023-12-26 23:07:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001062528_272048128.pth [2023-12-26 23:07:16,109][105620] Updated weights for policy 1, policy_version 1064662 (0.0005) [2023-12-26 23:07:16,166][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001064672_272588800.pth... [2023-12-26 23:07:16,167][105620] Updated weights for policy 1, policy_version 1064672 (0.0006) [2023-12-26 23:07:16,169][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001063488_272285696.pth [2023-12-26 23:07:16,307][105692] Updated weights for policy 0, policy_version 1063651 (0.0009) [2023-12-26 23:07:16,361][105692] Updated weights for policy 0, policy_version 1063661 (0.0010) [2023-12-26 23:07:16,423][105692] Updated weights for policy 0, policy_version 1063671 (0.0010) [2023-12-26 23:07:16,694][105620] Updated weights for policy 1, policy_version 1064682 (0.0008) [2023-12-26 23:07:16,766][105620] Updated weights for policy 1, policy_version 1064692 (0.0005) [2023-12-26 23:07:16,826][105620] Updated weights for policy 1, policy_version 1064702 (0.0008) [2023-12-26 23:07:17,170][105692] Updated weights for policy 0, policy_version 1063681 (0.0010) [2023-12-26 23:07:17,224][105692] Updated weights for policy 0, policy_version 1063691 (0.0005) [2023-12-26 23:07:17,273][105692] Updated weights for policy 0, policy_version 1063701 (0.0005) [2023-12-26 23:07:17,318][105692] Updated weights for policy 0, policy_version 1063711 (0.0005) [2023-12-26 23:07:17,496][105620] Updated weights for policy 1, policy_version 1064712 (0.0009) [2023-12-26 23:07:17,554][105620] Updated weights for policy 1, policy_version 1064722 (0.0010) [2023-12-26 23:07:17,607][105620] Updated weights for policy 1, policy_version 1064733 (0.0010) [2023-12-26 23:07:17,851][105692] Updated weights for policy 0, policy_version 1063721 (0.0005) [2023-12-26 23:07:17,904][105692] Updated weights for policy 0, policy_version 1063731 (0.0005) [2023-12-26 23:07:17,962][105692] Updated weights for policy 0, policy_version 1063741 (0.0005) [2023-12-26 23:07:18,254][105620] Updated weights for policy 1, policy_version 1064744 (0.0009) [2023-12-26 23:07:18,322][105620] Updated weights for policy 1, policy_version 1064754 (0.0008) [2023-12-26 23:07:18,387][105620] Updated weights for policy 1, policy_version 1064764 (0.0007) [2023-12-26 23:07:18,586][105692] Updated weights for policy 0, policy_version 1063751 (0.0008) [2023-12-26 23:07:18,638][105692] Updated weights for policy 0, policy_version 1063761 (0.0009) [2023-12-26 23:07:18,700][105692] Updated weights for policy 0, policy_version 1063771 (0.0009) [2023-12-26 23:07:19,107][105620] Updated weights for policy 1, policy_version 1064774 (0.0008) [2023-12-26 23:07:19,153][105620] Updated weights for policy 1, policy_version 1064784 (0.0008) [2023-12-26 23:07:19,210][105620] Updated weights for policy 1, policy_version 1064794 (0.0009) [2023-12-26 23:07:19,469][105692] Updated weights for policy 0, policy_version 1063781 (0.0008) [2023-12-26 23:07:19,530][105692] Updated weights for policy 0, policy_version 1063791 (0.0008) [2023-12-26 23:07:19,584][105692] Updated weights for policy 0, policy_version 1063801 (0.0008) [2023-12-26 23:07:19,992][105620] Updated weights for policy 1, policy_version 1064804 (0.0008) [2023-12-26 23:07:20,046][105620] Updated weights for policy 1, policy_version 1064814 (0.0008) [2023-12-26 23:07:20,105][105620] Updated weights for policy 1, policy_version 1064824 (0.0009) [2023-12-26 23:07:20,325][105692] Updated weights for policy 0, policy_version 1063811 (0.0008) [2023-12-26 23:07:20,381][105692] Updated weights for policy 0, policy_version 1063821 (0.0007) [2023-12-26 23:07:20,431][105692] Updated weights for policy 0, policy_version 1063831 (0.0005) [2023-12-26 23:07:20,880][105620] Updated weights for policy 1, policy_version 1064834 (0.0009) [2023-12-26 23:07:20,935][105620] Updated weights for policy 1, policy_version 1064844 (0.0009) [2023-12-26 23:07:20,990][105620] Updated weights for policy 1, policy_version 1064854 (0.0009) [2023-12-26 23:07:21,050][105620] Updated weights for policy 1, policy_version 1064864 (0.0007) [2023-12-26 23:07:21,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 545021952. Throughput: 0: 9741.2, 1: 9875.5. Samples: 545009556. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:07:21,063][104569] Avg episode reward: [(0, '7969.826'), (1, '9176.354')] [2023-12-26 23:07:21,100][105692] Updated weights for policy 0, policy_version 1063841 (0.0006) [2023-12-26 23:07:21,169][105692] Updated weights for policy 0, policy_version 1063851 (0.0008) [2023-12-26 23:07:21,234][105692] Updated weights for policy 0, policy_version 1063861 (0.0009) [2023-12-26 23:07:21,303][105692] Updated weights for policy 0, policy_version 1063871 (0.0008) [2023-12-26 23:07:21,876][105620] Updated weights for policy 1, policy_version 1064874 (0.0006) [2023-12-26 23:07:21,940][105620] Updated weights for policy 1, policy_version 1064884 (0.0006) [2023-12-26 23:07:21,995][105692] Updated weights for policy 0, policy_version 1063881 (0.0006) [2023-12-26 23:07:21,998][105620] Updated weights for policy 1, policy_version 1064894 (0.0006) [2023-12-26 23:07:22,060][105692] Updated weights for policy 0, policy_version 1063891 (0.0006) [2023-12-26 23:07:22,124][105692] Updated weights for policy 0, policy_version 1063901 (0.0009) [2023-12-26 23:07:22,674][105620] Updated weights for policy 1, policy_version 1064904 (0.0006) [2023-12-26 23:07:22,740][105620] Updated weights for policy 1, policy_version 1064914 (0.0008) [2023-12-26 23:07:22,801][105620] Updated weights for policy 1, policy_version 1064924 (0.0009) [2023-12-26 23:07:22,861][105692] Updated weights for policy 0, policy_version 1063911 (0.0006) [2023-12-26 23:07:22,913][105692] Updated weights for policy 0, policy_version 1063921 (0.0007) [2023-12-26 23:07:22,965][105692] Updated weights for policy 0, policy_version 1063931 (0.0009) [2023-12-26 23:07:23,448][105620] Updated weights for policy 1, policy_version 1064934 (0.0006) [2023-12-26 23:07:23,498][105620] Updated weights for policy 1, policy_version 1064944 (0.0005) [2023-12-26 23:07:23,558][105620] Updated weights for policy 1, policy_version 1064954 (0.0005) [2023-12-26 23:07:23,655][105692] Updated weights for policy 0, policy_version 1063941 (0.0009) [2023-12-26 23:07:23,706][105692] Updated weights for policy 0, policy_version 1063951 (0.0009) [2023-12-26 23:07:23,756][105692] Updated weights for policy 0, policy_version 1063961 (0.0009) [2023-12-26 23:07:24,151][105620] Updated weights for policy 1, policy_version 1064964 (0.0005) [2023-12-26 23:07:24,198][105620] Updated weights for policy 1, policy_version 1064974 (0.0005) [2023-12-26 23:07:24,245][105620] Updated weights for policy 1, policy_version 1064984 (0.0006) [2023-12-26 23:07:24,451][105692] Updated weights for policy 0, policy_version 1063971 (0.0009) [2023-12-26 23:07:24,511][105692] Updated weights for policy 0, policy_version 1063981 (0.0008) [2023-12-26 23:07:24,568][105692] Updated weights for policy 0, policy_version 1063991 (0.0008) [2023-12-26 23:07:24,922][105620] Updated weights for policy 1, policy_version 1064994 (0.0006) [2023-12-26 23:07:24,978][105620] Updated weights for policy 1, policy_version 1065004 (0.0010) [2023-12-26 23:07:25,033][105620] Updated weights for policy 1, policy_version 1065014 (0.0010) [2023-12-26 23:07:25,088][105620] Updated weights for policy 1, policy_version 1065024 (0.0010) [2023-12-26 23:07:25,367][105692] Updated weights for policy 0, policy_version 1064001 (0.0008) [2023-12-26 23:07:25,413][105692] Updated weights for policy 0, policy_version 1064011 (0.0008) [2023-12-26 23:07:25,461][105692] Updated weights for policy 0, policy_version 1064021 (0.0008) [2023-12-26 23:07:25,513][105692] Updated weights for policy 0, policy_version 1064031 (0.0008) [2023-12-26 23:07:25,752][105620] Updated weights for policy 1, policy_version 1065034 (0.0006) [2023-12-26 23:07:25,809][105620] Updated weights for policy 1, policy_version 1065044 (0.0005) [2023-12-26 23:07:25,863][105620] Updated weights for policy 1, policy_version 1065054 (0.0010) [2023-12-26 23:07:26,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 545120256. Throughput: 0: 9736.5, 1: 9931.7. Samples: 545127748. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:07:26,062][104569] Avg episode reward: [(0, '8596.044'), (1, '9174.433')] [2023-12-26 23:07:26,401][105692] Updated weights for policy 0, policy_version 1064041 (0.0008) [2023-12-26 23:07:26,439][105620] Updated weights for policy 1, policy_version 1065064 (0.0010) [2023-12-26 23:07:26,446][105692] Updated weights for policy 0, policy_version 1064051 (0.0006) [2023-12-26 23:07:26,493][105620] Updated weights for policy 1, policy_version 1065074 (0.0007) [2023-12-26 23:07:26,506][105692] Updated weights for policy 0, policy_version 1064061 (0.0008) [2023-12-26 23:07:26,552][105620] Updated weights for policy 1, policy_version 1065084 (0.0005) [2023-12-26 23:07:27,124][105620] Updated weights for policy 1, policy_version 1065094 (0.0005) [2023-12-26 23:07:27,181][105620] Updated weights for policy 1, policy_version 1065104 (0.0005) [2023-12-26 23:07:27,241][105620] Updated weights for policy 1, policy_version 1065114 (0.0005) [2023-12-26 23:07:27,330][105692] Updated weights for policy 0, policy_version 1064071 (0.0009) [2023-12-26 23:07:27,379][105692] Updated weights for policy 0, policy_version 1064081 (0.0005) [2023-12-26 23:07:27,429][105692] Updated weights for policy 0, policy_version 1064091 (0.0007) [2023-12-26 23:07:27,793][105620] Updated weights for policy 1, policy_version 1065124 (0.0007) [2023-12-26 23:07:27,847][105620] Updated weights for policy 1, policy_version 1065134 (0.0009) [2023-12-26 23:07:27,893][105620] Updated weights for policy 1, policy_version 1065144 (0.0009) [2023-12-26 23:07:28,204][105692] Updated weights for policy 0, policy_version 1064101 (0.0009) [2023-12-26 23:07:28,260][105692] Updated weights for policy 0, policy_version 1064111 (0.0009) [2023-12-26 23:07:28,319][105692] Updated weights for policy 0, policy_version 1064121 (0.0009) [2023-12-26 23:07:28,642][105620] Updated weights for policy 1, policy_version 1065154 (0.0009) [2023-12-26 23:07:28,691][105620] Updated weights for policy 1, policy_version 1065164 (0.0009) [2023-12-26 23:07:28,738][105620] Updated weights for policy 1, policy_version 1065174 (0.0010) [2023-12-26 23:07:28,784][105620] Updated weights for policy 1, policy_version 1065184 (0.0009) [2023-12-26 23:07:29,146][105692] Updated weights for policy 0, policy_version 1064131 (0.0009) [2023-12-26 23:07:29,200][105692] Updated weights for policy 0, policy_version 1064141 (0.0008) [2023-12-26 23:07:29,250][105692] Updated weights for policy 0, policy_version 1064151 (0.0008) [2023-12-26 23:07:29,414][105620] Updated weights for policy 1, policy_version 1065194 (0.0008) [2023-12-26 23:07:29,461][105620] Updated weights for policy 1, policy_version 1065204 (0.0010) [2023-12-26 23:07:29,515][105620] Updated weights for policy 1, policy_version 1065214 (0.0009) [2023-12-26 23:07:30,031][105692] Updated weights for policy 0, policy_version 1064161 (0.0009) [2023-12-26 23:07:30,090][105692] Updated weights for policy 0, policy_version 1064171 (0.0010) [2023-12-26 23:07:30,156][105692] Updated weights for policy 0, policy_version 1064181 (0.0010) [2023-12-26 23:07:30,208][105620] Updated weights for policy 1, policy_version 1065224 (0.0006) [2023-12-26 23:07:30,214][105692] Updated weights for policy 0, policy_version 1064191 (0.0008) [2023-12-26 23:07:30,261][105620] Updated weights for policy 1, policy_version 1065234 (0.0008) [2023-12-26 23:07:30,311][105620] Updated weights for policy 1, policy_version 1065244 (0.0009) [2023-12-26 23:07:30,948][105692] Updated weights for policy 0, policy_version 1064201 (0.0009) [2023-12-26 23:07:31,001][105692] Updated weights for policy 0, policy_version 1064211 (0.0010) [2023-12-26 23:07:31,047][105620] Updated weights for policy 1, policy_version 1065254 (0.0008) [2023-12-26 23:07:31,056][105692] Updated weights for policy 0, policy_version 1064221 (0.0008) [2023-12-26 23:07:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 545210368. Throughput: 0: 9707.7, 1: 10017.6. Samples: 545187408. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:07:31,062][104569] Avg episode reward: [(0, '8762.909'), (1, '9346.309')] [2023-12-26 23:07:31,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001064224_272482304.pth... [2023-12-26 23:07:31,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001063104_272195584.pth [2023-12-26 23:07:31,103][105620] Updated weights for policy 1, policy_version 1065264 (0.0007) [2023-12-26 23:07:31,171][105620] Updated weights for policy 1, policy_version 1065274 (0.0008) [2023-12-26 23:07:31,196][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001065280_272744448.pth... [2023-12-26 23:07:31,199][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001064096_272441344.pth [2023-12-26 23:07:31,843][105692] Updated weights for policy 0, policy_version 1064231 (0.0006) [2023-12-26 23:07:31,912][105692] Updated weights for policy 0, policy_version 1064241 (0.0006) [2023-12-26 23:07:31,965][105620] Updated weights for policy 1, policy_version 1065284 (0.0008) [2023-12-26 23:07:31,969][105692] Updated weights for policy 0, policy_version 1064251 (0.0008) [2023-12-26 23:07:32,026][105620] Updated weights for policy 1, policy_version 1065294 (0.0010) [2023-12-26 23:07:32,095][105620] Updated weights for policy 1, policy_version 1065304 (0.0009) [2023-12-26 23:07:32,532][105692] Updated weights for policy 0, policy_version 1064261 (0.0007) [2023-12-26 23:07:32,594][105692] Updated weights for policy 0, policy_version 1064271 (0.0008) [2023-12-26 23:07:32,655][105692] Updated weights for policy 0, policy_version 1064281 (0.0008) [2023-12-26 23:07:32,852][105620] Updated weights for policy 1, policy_version 1065314 (0.0009) [2023-12-26 23:07:32,909][105620] Updated weights for policy 1, policy_version 1065324 (0.0009) [2023-12-26 23:07:32,968][105620] Updated weights for policy 1, policy_version 1065334 (0.0009) [2023-12-26 23:07:33,026][105620] Updated weights for policy 1, policy_version 1065344 (0.0009) [2023-12-26 23:07:33,369][105692] Updated weights for policy 0, policy_version 1064291 (0.0008) [2023-12-26 23:07:33,426][105692] Updated weights for policy 0, policy_version 1064301 (0.0009) [2023-12-26 23:07:33,476][105692] Updated weights for policy 0, policy_version 1064311 (0.0006) [2023-12-26 23:07:33,818][105620] Updated weights for policy 1, policy_version 1065354 (0.0007) [2023-12-26 23:07:33,875][105620] Updated weights for policy 1, policy_version 1065364 (0.0005) [2023-12-26 23:07:33,933][105620] Updated weights for policy 1, policy_version 1065374 (0.0005) [2023-12-26 23:07:34,200][105692] Updated weights for policy 0, policy_version 1064321 (0.0007) [2023-12-26 23:07:34,260][105692] Updated weights for policy 0, policy_version 1064331 (0.0009) [2023-12-26 23:07:34,315][105692] Updated weights for policy 0, policy_version 1064341 (0.0008) [2023-12-26 23:07:34,377][105692] Updated weights for policy 0, policy_version 1064351 (0.0009) [2023-12-26 23:07:34,574][105620] Updated weights for policy 1, policy_version 1065384 (0.0008) [2023-12-26 23:07:34,636][105620] Updated weights for policy 1, policy_version 1065394 (0.0009) [2023-12-26 23:07:34,696][105620] Updated weights for policy 1, policy_version 1065404 (0.0009) [2023-12-26 23:07:35,159][105692] Updated weights for policy 0, policy_version 1064361 (0.0009) [2023-12-26 23:07:35,214][105692] Updated weights for policy 0, policy_version 1064371 (0.0009) [2023-12-26 23:07:35,268][105692] Updated weights for policy 0, policy_version 1064381 (0.0009) [2023-12-26 23:07:35,413][105620] Updated weights for policy 1, policy_version 1065414 (0.0009) [2023-12-26 23:07:35,474][105620] Updated weights for policy 1, policy_version 1065424 (0.0009) [2023-12-26 23:07:35,527][105620] Updated weights for policy 1, policy_version 1065434 (0.0010) [2023-12-26 23:07:35,890][105692] Updated weights for policy 0, policy_version 1064391 (0.0006) [2023-12-26 23:07:35,949][105692] Updated weights for policy 0, policy_version 1064401 (0.0006) [2023-12-26 23:07:36,004][105692] Updated weights for policy 0, policy_version 1064411 (0.0005) [2023-12-26 23:07:36,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.7, 300 sec: 19605.3). Total num frames: 545316864. Throughput: 0: 9613.5, 1: 9992.1. Samples: 545302352. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:07:36,063][104569] Avg episode reward: [(0, '8744.233'), (1, '9256.548')] [2023-12-26 23:07:36,220][105620] Updated weights for policy 1, policy_version 1065444 (0.0010) [2023-12-26 23:07:36,289][105620] Updated weights for policy 1, policy_version 1065454 (0.0009) [2023-12-26 23:07:36,360][105620] Updated weights for policy 1, policy_version 1065464 (0.0010) [2023-12-26 23:07:36,615][105692] Updated weights for policy 0, policy_version 1064421 (0.0007) [2023-12-26 23:07:36,682][105692] Updated weights for policy 0, policy_version 1064431 (0.0008) [2023-12-26 23:07:36,743][105692] Updated weights for policy 0, policy_version 1064441 (0.0008) [2023-12-26 23:07:37,099][105620] Updated weights for policy 1, policy_version 1065474 (0.0010) [2023-12-26 23:07:37,168][105620] Updated weights for policy 1, policy_version 1065484 (0.0011) [2023-12-26 23:07:37,223][105620] Updated weights for policy 1, policy_version 1065494 (0.0006) [2023-12-26 23:07:37,286][105620] Updated weights for policy 1, policy_version 1065504 (0.0007) [2023-12-26 23:07:37,493][105692] Updated weights for policy 0, policy_version 1064451 (0.0007) [2023-12-26 23:07:37,552][105692] Updated weights for policy 0, policy_version 1064461 (0.0005) [2023-12-26 23:07:37,619][105692] Updated weights for policy 0, policy_version 1064471 (0.0007) [2023-12-26 23:07:37,995][105620] Updated weights for policy 1, policy_version 1065514 (0.0011) [2023-12-26 23:07:38,051][105620] Updated weights for policy 1, policy_version 1065524 (0.0011) [2023-12-26 23:07:38,103][105620] Updated weights for policy 1, policy_version 1065534 (0.0011) [2023-12-26 23:07:38,323][105692] Updated weights for policy 0, policy_version 1064481 (0.0008) [2023-12-26 23:07:38,388][105692] Updated weights for policy 0, policy_version 1064491 (0.0010) [2023-12-26 23:07:38,445][105692] Updated weights for policy 0, policy_version 1064501 (0.0009) [2023-12-26 23:07:38,509][105692] Updated weights for policy 0, policy_version 1064511 (0.0006) [2023-12-26 23:07:38,753][105620] Updated weights for policy 1, policy_version 1065544 (0.0011) [2023-12-26 23:07:38,812][105620] Updated weights for policy 1, policy_version 1065554 (0.0010) [2023-12-26 23:07:38,864][105620] Updated weights for policy 1, policy_version 1065564 (0.0010) [2023-12-26 23:07:39,187][105692] Updated weights for policy 0, policy_version 1064521 (0.0009) [2023-12-26 23:07:39,251][105692] Updated weights for policy 0, policy_version 1064531 (0.0009) [2023-12-26 23:07:39,308][105692] Updated weights for policy 0, policy_version 1064541 (0.0009) [2023-12-26 23:07:39,494][105620] Updated weights for policy 1, policy_version 1065574 (0.0007) [2023-12-26 23:07:39,557][105620] Updated weights for policy 1, policy_version 1065584 (0.0008) [2023-12-26 23:07:39,621][105620] Updated weights for policy 1, policy_version 1065594 (0.0009) [2023-12-26 23:07:40,087][105692] Updated weights for policy 0, policy_version 1064551 (0.0010) [2023-12-26 23:07:40,148][105692] Updated weights for policy 0, policy_version 1064561 (0.0010) [2023-12-26 23:07:40,215][105692] Updated weights for policy 0, policy_version 1064571 (0.0011) [2023-12-26 23:07:40,395][105620] Updated weights for policy 1, policy_version 1065604 (0.0009) [2023-12-26 23:07:40,451][105620] Updated weights for policy 1, policy_version 1065614 (0.0009) [2023-12-26 23:07:40,502][105620] Updated weights for policy 1, policy_version 1065624 (0.0008) [2023-12-26 23:07:40,899][105692] Updated weights for policy 0, policy_version 1064581 (0.0008) [2023-12-26 23:07:40,958][105692] Updated weights for policy 0, policy_version 1064591 (0.0007) [2023-12-26 23:07:41,010][105692] Updated weights for policy 0, policy_version 1064601 (0.0008) [2023-12-26 23:07:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 545415168. Throughput: 0: 9595.0, 1: 10012.6. Samples: 545419904. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:07:41,062][104569] Avg episode reward: [(0, '8728.705'), (1, '9257.912')] [2023-12-26 23:07:41,279][105620] Updated weights for policy 1, policy_version 1065634 (0.0010) [2023-12-26 23:07:41,341][105620] Updated weights for policy 1, policy_version 1065644 (0.0009) [2023-12-26 23:07:41,402][105620] Updated weights for policy 1, policy_version 1065654 (0.0009) [2023-12-26 23:07:41,457][105620] Updated weights for policy 1, policy_version 1065664 (0.0008) [2023-12-26 23:07:41,779][105692] Updated weights for policy 0, policy_version 1064611 (0.0008) [2023-12-26 23:07:41,842][105692] Updated weights for policy 0, policy_version 1064621 (0.0010) [2023-12-26 23:07:41,908][105692] Updated weights for policy 0, policy_version 1064631 (0.0011) [2023-12-26 23:07:42,220][105620] Updated weights for policy 1, policy_version 1065674 (0.0005) [2023-12-26 23:07:42,281][105620] Updated weights for policy 1, policy_version 1065684 (0.0007) [2023-12-26 23:07:42,344][105620] Updated weights for policy 1, policy_version 1065694 (0.0007) [2023-12-26 23:07:42,602][105692] Updated weights for policy 0, policy_version 1064641 (0.0010) [2023-12-26 23:07:42,661][105692] Updated weights for policy 0, policy_version 1064651 (0.0011) [2023-12-26 23:07:42,725][105692] Updated weights for policy 0, policy_version 1064661 (0.0011) [2023-12-26 23:07:42,789][105692] Updated weights for policy 0, policy_version 1064671 (0.0011) [2023-12-26 23:07:43,048][105620] Updated weights for policy 1, policy_version 1065704 (0.0008) [2023-12-26 23:07:43,109][105620] Updated weights for policy 1, policy_version 1065714 (0.0009) [2023-12-26 23:07:43,173][105620] Updated weights for policy 1, policy_version 1065724 (0.0009) [2023-12-26 23:07:43,533][105692] Updated weights for policy 0, policy_version 1064681 (0.0006) [2023-12-26 23:07:43,589][105692] Updated weights for policy 0, policy_version 1064691 (0.0006) [2023-12-26 23:07:43,644][105692] Updated weights for policy 0, policy_version 1064701 (0.0006) [2023-12-26 23:07:43,934][105620] Updated weights for policy 1, policy_version 1065734 (0.0006) [2023-12-26 23:07:43,990][105620] Updated weights for policy 1, policy_version 1065744 (0.0005) [2023-12-26 23:07:44,056][105620] Updated weights for policy 1, policy_version 1065754 (0.0009) [2023-12-26 23:07:44,308][105692] Updated weights for policy 0, policy_version 1064711 (0.0006) [2023-12-26 23:07:44,368][105692] Updated weights for policy 0, policy_version 1064721 (0.0007) [2023-12-26 23:07:44,427][105692] Updated weights for policy 0, policy_version 1064731 (0.0007) [2023-12-26 23:07:44,772][105620] Updated weights for policy 1, policy_version 1065764 (0.0009) [2023-12-26 23:07:44,827][105620] Updated weights for policy 1, policy_version 1065774 (0.0008) [2023-12-26 23:07:44,883][105620] Updated weights for policy 1, policy_version 1065784 (0.0008) [2023-12-26 23:07:45,059][105692] Updated weights for policy 0, policy_version 1064741 (0.0008) [2023-12-26 23:07:45,121][105692] Updated weights for policy 0, policy_version 1064751 (0.0011) [2023-12-26 23:07:45,187][105692] Updated weights for policy 0, policy_version 1064761 (0.0010) [2023-12-26 23:07:45,639][105620] Updated weights for policy 1, policy_version 1065794 (0.0008) [2023-12-26 23:07:45,700][105620] Updated weights for policy 1, policy_version 1065804 (0.0010) [2023-12-26 23:07:45,759][105620] Updated weights for policy 1, policy_version 1065814 (0.0010) [2023-12-26 23:07:45,824][105620] Updated weights for policy 1, policy_version 1065824 (0.0010) [2023-12-26 23:07:45,936][105692] Updated weights for policy 0, policy_version 1064771 (0.0010) [2023-12-26 23:07:46,004][105692] Updated weights for policy 0, policy_version 1064781 (0.0010) [2023-12-26 23:07:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19605.2). Total num frames: 545505280. Throughput: 0: 9597.7, 1: 9934.4. Samples: 545477168. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:07:46,063][104569] Avg episode reward: [(0, '8997.260'), (1, '9258.468')] [2023-12-26 23:07:46,068][105692] Updated weights for policy 0, policy_version 1064791 (0.0010) [2023-12-26 23:07:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001065824_272883712.pth... [2023-12-26 23:07:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001064672_272588800.pth [2023-12-26 23:07:46,125][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001064800_272629760.pth... [2023-12-26 23:07:46,155][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001063648_272334848.pth [2023-12-26 23:07:46,507][105620] Updated weights for policy 1, policy_version 1065834 (0.0011) [2023-12-26 23:07:46,577][105620] Updated weights for policy 1, policy_version 1065844 (0.0006) [2023-12-26 23:07:46,645][105620] Updated weights for policy 1, policy_version 1065854 (0.0006) [2023-12-26 23:07:46,768][105692] Updated weights for policy 0, policy_version 1064802 (0.0010) [2023-12-26 23:07:46,826][105692] Updated weights for policy 0, policy_version 1064812 (0.0011) [2023-12-26 23:07:46,888][105692] Updated weights for policy 0, policy_version 1064822 (0.0011) [2023-12-26 23:07:46,939][105692] Updated weights for policy 0, policy_version 1064832 (0.0010) [2023-12-26 23:07:47,344][105620] Updated weights for policy 1, policy_version 1065864 (0.0008) [2023-12-26 23:07:47,403][105620] Updated weights for policy 1, policy_version 1065874 (0.0008) [2023-12-26 23:07:47,470][105620] Updated weights for policy 1, policy_version 1065884 (0.0008) [2023-12-26 23:07:47,681][105692] Updated weights for policy 0, policy_version 1064842 (0.0010) [2023-12-26 23:07:47,732][105692] Updated weights for policy 0, policy_version 1064852 (0.0010) [2023-12-26 23:07:47,780][105692] Updated weights for policy 0, policy_version 1064862 (0.0010) [2023-12-26 23:07:48,112][105620] Updated weights for policy 1, policy_version 1065894 (0.0007) [2023-12-26 23:07:48,160][105620] Updated weights for policy 1, policy_version 1065904 (0.0010) [2023-12-26 23:07:48,217][105620] Updated weights for policy 1, policy_version 1065914 (0.0007) [2023-12-26 23:07:48,538][105692] Updated weights for policy 0, policy_version 1064872 (0.0011) [2023-12-26 23:07:48,587][105692] Updated weights for policy 0, policy_version 1064882 (0.0011) [2023-12-26 23:07:48,642][105692] Updated weights for policy 0, policy_version 1064892 (0.0011) [2023-12-26 23:07:48,835][105620] Updated weights for policy 1, policy_version 1065924 (0.0010) [2023-12-26 23:07:48,895][105620] Updated weights for policy 1, policy_version 1065934 (0.0011) [2023-12-26 23:07:48,954][105620] Updated weights for policy 1, policy_version 1065944 (0.0011) [2023-12-26 23:07:49,361][105692] Updated weights for policy 0, policy_version 1064902 (0.0009) [2023-12-26 23:07:49,433][105692] Updated weights for policy 0, policy_version 1064912 (0.0007) [2023-12-26 23:07:49,497][105692] Updated weights for policy 0, policy_version 1064922 (0.0007) [2023-12-26 23:07:49,690][105620] Updated weights for policy 1, policy_version 1065954 (0.0010) [2023-12-26 23:07:49,756][105620] Updated weights for policy 1, policy_version 1065964 (0.0011) [2023-12-26 23:07:49,815][105620] Updated weights for policy 1, policy_version 1065974 (0.0010) [2023-12-26 23:07:49,881][105620] Updated weights for policy 1, policy_version 1065984 (0.0008) [2023-12-26 23:07:50,219][105692] Updated weights for policy 0, policy_version 1064932 (0.0010) [2023-12-26 23:07:50,274][105692] Updated weights for policy 0, policy_version 1064942 (0.0010) [2023-12-26 23:07:50,329][105692] Updated weights for policy 0, policy_version 1064952 (0.0010) [2023-12-26 23:07:50,464][105620] Updated weights for policy 1, policy_version 1065994 (0.0010) [2023-12-26 23:07:50,519][105620] Updated weights for policy 1, policy_version 1066004 (0.0010) [2023-12-26 23:07:50,570][105620] Updated weights for policy 1, policy_version 1066014 (0.0009) [2023-12-26 23:07:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 545603584. Throughput: 0: 9635.1, 1: 9911.7. Samples: 545594652. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-12-26 23:07:51,062][104569] Avg episode reward: [(0, '9177.191'), (1, '9166.287')] [2023-12-26 23:07:51,094][105692] Updated weights for policy 0, policy_version 1064962 (0.0010) [2023-12-26 23:07:51,163][105692] Updated weights for policy 0, policy_version 1064972 (0.0008) [2023-12-26 23:07:51,225][105692] Updated weights for policy 0, policy_version 1064982 (0.0008) [2023-12-26 23:07:51,234][105620] Updated weights for policy 1, policy_version 1066024 (0.0007) [2023-12-26 23:07:51,289][105692] Updated weights for policy 0, policy_version 1064992 (0.0006) [2023-12-26 23:07:51,293][105620] Updated weights for policy 1, policy_version 1066034 (0.0008) [2023-12-26 23:07:51,361][105620] Updated weights for policy 1, policy_version 1066044 (0.0010) [2023-12-26 23:07:51,984][105692] Updated weights for policy 0, policy_version 1065002 (0.0008) [2023-12-26 23:07:52,040][105692] Updated weights for policy 0, policy_version 1065012 (0.0007) [2023-12-26 23:07:52,087][105692] Updated weights for policy 0, policy_version 1065022 (0.0008) [2023-12-26 23:07:52,093][105620] Updated weights for policy 1, policy_version 1066054 (0.0010) [2023-12-26 23:07:52,142][105620] Updated weights for policy 1, policy_version 1066064 (0.0011) [2023-12-26 23:07:52,194][105620] Updated weights for policy 1, policy_version 1066074 (0.0011) [2023-12-26 23:07:52,867][105692] Updated weights for policy 0, policy_version 1065032 (0.0009) [2023-12-26 23:07:52,903][105620] Updated weights for policy 1, policy_version 1066084 (0.0008) [2023-12-26 23:07:52,924][105692] Updated weights for policy 0, policy_version 1065042 (0.0008) [2023-12-26 23:07:52,956][105620] Updated weights for policy 1, policy_version 1066094 (0.0005) [2023-12-26 23:07:52,979][105692] Updated weights for policy 0, policy_version 1065052 (0.0008) [2023-12-26 23:07:53,016][105620] Updated weights for policy 1, policy_version 1066104 (0.0006) [2023-12-26 23:07:53,649][105692] Updated weights for policy 0, policy_version 1065062 (0.0008) [2023-12-26 23:07:53,701][105620] Updated weights for policy 1, policy_version 1066114 (0.0011) [2023-12-26 23:07:53,707][105692] Updated weights for policy 0, policy_version 1065072 (0.0007) [2023-12-26 23:07:53,749][105620] Updated weights for policy 1, policy_version 1066124 (0.0010) [2023-12-26 23:07:53,759][105692] Updated weights for policy 0, policy_version 1065082 (0.0005) [2023-12-26 23:07:53,798][105620] Updated weights for policy 1, policy_version 1066134 (0.0010) [2023-12-26 23:07:53,853][105620] Updated weights for policy 1, policy_version 1066144 (0.0010) [2023-12-26 23:07:54,363][105692] Updated weights for policy 0, policy_version 1065092 (0.0005) [2023-12-26 23:07:54,417][105692] Updated weights for policy 0, policy_version 1065102 (0.0005) [2023-12-26 23:07:54,474][105692] Updated weights for policy 0, policy_version 1065112 (0.0007) [2023-12-26 23:07:54,644][105620] Updated weights for policy 1, policy_version 1066154 (0.0011) [2023-12-26 23:07:54,712][105620] Updated weights for policy 1, policy_version 1066164 (0.0011) [2023-12-26 23:07:54,775][105620] Updated weights for policy 1, policy_version 1066174 (0.0011) [2023-12-26 23:07:55,184][105692] Updated weights for policy 0, policy_version 1065122 (0.0009) [2023-12-26 23:07:55,240][105692] Updated weights for policy 0, policy_version 1065132 (0.0010) [2023-12-26 23:07:55,290][105692] Updated weights for policy 0, policy_version 1065142 (0.0009) [2023-12-26 23:07:55,340][105692] Updated weights for policy 0, policy_version 1065152 (0.0006) [2023-12-26 23:07:55,415][105620] Updated weights for policy 1, policy_version 1066184 (0.0007) [2023-12-26 23:07:55,468][105620] Updated weights for policy 1, policy_version 1066194 (0.0009) [2023-12-26 23:07:55,520][105620] Updated weights for policy 1, policy_version 1066204 (0.0009) [2023-12-26 23:07:55,982][105692] Updated weights for policy 0, policy_version 1065162 (0.0005) [2023-12-26 23:07:56,042][105692] Updated weights for policy 0, policy_version 1065172 (0.0005) [2023-12-26 23:07:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 545701888. Throughput: 0: 9680.5, 1: 9974.7. Samples: 545715316. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:07:56,063][104569] Avg episode reward: [(0, '9176.692'), (1, '9350.678')] [2023-12-26 23:07:56,063][105620] Updated weights for policy 1, policy_version 1066214 (0.0006) [2023-12-26 23:07:56,098][105692] Updated weights for policy 0, policy_version 1065182 (0.0006) [2023-12-26 23:07:56,126][105620] Updated weights for policy 1, policy_version 1066224 (0.0007) [2023-12-26 23:07:56,180][105620] Updated weights for policy 1, policy_version 1066234 (0.0010) [2023-12-26 23:07:56,790][105692] Updated weights for policy 0, policy_version 1065192 (0.0009) [2023-12-26 23:07:56,849][105692] Updated weights for policy 0, policy_version 1065202 (0.0010) [2023-12-26 23:07:56,901][105620] Updated weights for policy 1, policy_version 1066244 (0.0009) [2023-12-26 23:07:56,907][105692] Updated weights for policy 0, policy_version 1065212 (0.0010) [2023-12-26 23:07:56,953][105620] Updated weights for policy 1, policy_version 1066254 (0.0007) [2023-12-26 23:07:57,002][105620] Updated weights for policy 1, policy_version 1066264 (0.0006) [2023-12-26 23:07:57,600][105692] Updated weights for policy 0, policy_version 1065222 (0.0009) [2023-12-26 23:07:57,619][105620] Updated weights for policy 1, policy_version 1066274 (0.0006) [2023-12-26 23:07:57,655][105692] Updated weights for policy 0, policy_version 1065232 (0.0010) [2023-12-26 23:07:57,674][105620] Updated weights for policy 1, policy_version 1066284 (0.0006) [2023-12-26 23:07:57,703][105692] Updated weights for policy 0, policy_version 1065242 (0.0010) [2023-12-26 23:07:57,720][105620] Updated weights for policy 1, policy_version 1066294 (0.0005) [2023-12-26 23:07:57,766][105620] Updated weights for policy 1, policy_version 1066304 (0.0005) [2023-12-26 23:07:58,343][105620] Updated weights for policy 1, policy_version 1066314 (0.0007) [2023-12-26 23:07:58,371][105692] Updated weights for policy 0, policy_version 1065252 (0.0010) [2023-12-26 23:07:58,394][105620] Updated weights for policy 1, policy_version 1066324 (0.0008) [2023-12-26 23:07:58,435][105692] Updated weights for policy 0, policy_version 1065262 (0.0009) [2023-12-26 23:07:58,460][105620] Updated weights for policy 1, policy_version 1066334 (0.0008) [2023-12-26 23:07:58,496][105692] Updated weights for policy 0, policy_version 1065272 (0.0010) [2023-12-26 23:07:59,368][105620] Updated weights for policy 1, policy_version 1066344 (0.0008) [2023-12-26 23:07:59,376][105692] Updated weights for policy 0, policy_version 1065282 (0.0009) [2023-12-26 23:07:59,427][105620] Updated weights for policy 1, policy_version 1066354 (0.0008) [2023-12-26 23:07:59,437][105692] Updated weights for policy 0, policy_version 1065292 (0.0007) [2023-12-26 23:07:59,479][105620] Updated weights for policy 1, policy_version 1066364 (0.0008) [2023-12-26 23:07:59,493][105692] Updated weights for policy 0, policy_version 1065302 (0.0010) [2023-12-26 23:07:59,551][105692] Updated weights for policy 0, policy_version 1065312 (0.0010) [2023-12-26 23:08:00,219][105692] Updated weights for policy 0, policy_version 1065322 (0.0008) [2023-12-26 23:08:00,220][105620] Updated weights for policy 1, policy_version 1066374 (0.0008) [2023-12-26 23:08:00,276][105692] Updated weights for policy 0, policy_version 1065332 (0.0005) [2023-12-26 23:08:00,281][105620] Updated weights for policy 1, policy_version 1066384 (0.0008) [2023-12-26 23:08:00,334][105692] Updated weights for policy 0, policy_version 1065342 (0.0009) [2023-12-26 23:08:00,339][105620] Updated weights for policy 1, policy_version 1066394 (0.0006) [2023-12-26 23:08:01,000][105692] Updated weights for policy 0, policy_version 1065352 (0.0009) [2023-12-26 23:08:01,059][105692] Updated weights for policy 0, policy_version 1065362 (0.0008) [2023-12-26 23:08:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 545800192. Throughput: 0: 9753.0, 1: 10022.1. Samples: 545777336. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:08:01,062][104569] Avg episode reward: [(0, '9007.412'), (1, '9324.570')] [2023-12-26 23:08:01,086][105620] Updated weights for policy 1, policy_version 1066404 (0.0008) [2023-12-26 23:08:01,123][105692] Updated weights for policy 0, policy_version 1065372 (0.0009) [2023-12-26 23:08:01,146][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001065376_272777216.pth... [2023-12-26 23:08:01,150][105620] Updated weights for policy 1, policy_version 1066414 (0.0006) [2023-12-26 23:08:01,152][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001064224_272482304.pth [2023-12-26 23:08:01,216][105620] Updated weights for policy 1, policy_version 1066424 (0.0008) [2023-12-26 23:08:01,265][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001066432_273039360.pth... [2023-12-26 23:08:01,269][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001065280_272744448.pth [2023-12-26 23:08:01,839][105692] Updated weights for policy 0, policy_version 1065382 (0.0008) [2023-12-26 23:08:01,899][105692] Updated weights for policy 0, policy_version 1065392 (0.0008) [2023-12-26 23:08:01,957][105692] Updated weights for policy 0, policy_version 1065402 (0.0008) [2023-12-26 23:08:01,971][105620] Updated weights for policy 1, policy_version 1066434 (0.0011) [2023-12-26 23:08:02,030][105620] Updated weights for policy 1, policy_version 1066444 (0.0010) [2023-12-26 23:08:02,091][105620] Updated weights for policy 1, policy_version 1066454 (0.0010) [2023-12-26 23:08:02,143][105620] Updated weights for policy 1, policy_version 1066464 (0.0010) [2023-12-26 23:08:02,584][105692] Updated weights for policy 0, policy_version 1065412 (0.0008) [2023-12-26 23:08:02,634][105692] Updated weights for policy 0, policy_version 1065422 (0.0009) [2023-12-26 23:08:02,685][105692] Updated weights for policy 0, policy_version 1065432 (0.0007) [2023-12-26 23:08:02,920][105620] Updated weights for policy 1, policy_version 1066474 (0.0010) [2023-12-26 23:08:02,980][105620] Updated weights for policy 1, policy_version 1066484 (0.0010) [2023-12-26 23:08:03,029][105620] Updated weights for policy 1, policy_version 1066494 (0.0009) [2023-12-26 23:08:03,390][105692] Updated weights for policy 0, policy_version 1065442 (0.0008) [2023-12-26 23:08:03,446][105692] Updated weights for policy 0, policy_version 1065452 (0.0010) [2023-12-26 23:08:03,500][105692] Updated weights for policy 0, policy_version 1065462 (0.0010) [2023-12-26 23:08:03,554][105692] Updated weights for policy 0, policy_version 1065472 (0.0009) [2023-12-26 23:08:03,743][105620] Updated weights for policy 1, policy_version 1066504 (0.0009) [2023-12-26 23:08:03,790][105620] Updated weights for policy 1, policy_version 1066514 (0.0010) [2023-12-26 23:08:03,842][105620] Updated weights for policy 1, policy_version 1066524 (0.0010) [2023-12-26 23:08:04,145][105692] Updated weights for policy 0, policy_version 1065482 (0.0005) [2023-12-26 23:08:04,216][105692] Updated weights for policy 0, policy_version 1065492 (0.0009) [2023-12-26 23:08:04,285][105692] Updated weights for policy 0, policy_version 1065502 (0.0010) [2023-12-26 23:08:04,561][105620] Updated weights for policy 1, policy_version 1066534 (0.0007) [2023-12-26 23:08:04,620][105620] Updated weights for policy 1, policy_version 1066544 (0.0009) [2023-12-26 23:08:04,682][105620] Updated weights for policy 1, policy_version 1066554 (0.0010) [2023-12-26 23:08:04,871][105692] Updated weights for policy 0, policy_version 1065512 (0.0006) [2023-12-26 23:08:04,919][105692] Updated weights for policy 0, policy_version 1065522 (0.0006) [2023-12-26 23:08:04,981][105692] Updated weights for policy 0, policy_version 1065532 (0.0010) [2023-12-26 23:08:05,398][105620] Updated weights for policy 1, policy_version 1066564 (0.0010) [2023-12-26 23:08:05,462][105620] Updated weights for policy 1, policy_version 1066574 (0.0010) [2023-12-26 23:08:05,520][105620] Updated weights for policy 1, policy_version 1066584 (0.0010) [2023-12-26 23:08:05,535][105692] Updated weights for policy 0, policy_version 1065542 (0.0007) [2023-12-26 23:08:05,591][105692] Updated weights for policy 0, policy_version 1065552 (0.0005) [2023-12-26 23:08:05,648][105692] Updated weights for policy 0, policy_version 1065562 (0.0005) [2023-12-26 23:08:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 545906688. Throughput: 0: 9741.4, 1: 9916.7. Samples: 545894168. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:08:06,062][104569] Avg episode reward: [(0, '9005.601'), (1, '9233.461')] [2023-12-26 23:08:06,222][105692] Updated weights for policy 0, policy_version 1065572 (0.0007) [2023-12-26 23:08:06,238][105620] Updated weights for policy 1, policy_version 1066594 (0.0010) [2023-12-26 23:08:06,274][105692] Updated weights for policy 0, policy_version 1065582 (0.0011) [2023-12-26 23:08:06,297][105620] Updated weights for policy 1, policy_version 1066604 (0.0011) [2023-12-26 23:08:06,334][105692] Updated weights for policy 0, policy_version 1065592 (0.0011) [2023-12-26 23:08:06,357][105620] Updated weights for policy 1, policy_version 1066614 (0.0011) [2023-12-26 23:08:06,414][105620] Updated weights for policy 1, policy_version 1066624 (0.0010) [2023-12-26 23:08:07,087][105692] Updated weights for policy 0, policy_version 1065602 (0.0011) [2023-12-26 23:08:07,146][105692] Updated weights for policy 0, policy_version 1065612 (0.0011) [2023-12-26 23:08:07,156][105620] Updated weights for policy 1, policy_version 1066634 (0.0008) [2023-12-26 23:08:07,205][105692] Updated weights for policy 0, policy_version 1065622 (0.0011) [2023-12-26 23:08:07,214][105620] Updated weights for policy 1, policy_version 1066644 (0.0006) [2023-12-26 23:08:07,262][105692] Updated weights for policy 0, policy_version 1065632 (0.0011) [2023-12-26 23:08:07,271][105620] Updated weights for policy 1, policy_version 1066654 (0.0006) [2023-12-26 23:08:07,889][105620] Updated weights for policy 1, policy_version 1066664 (0.0006) [2023-12-26 23:08:07,926][105692] Updated weights for policy 0, policy_version 1065642 (0.0006) [2023-12-26 23:08:07,948][105620] Updated weights for policy 1, policy_version 1066674 (0.0005) [2023-12-26 23:08:07,979][105692] Updated weights for policy 0, policy_version 1065652 (0.0006) [2023-12-26 23:08:08,008][105620] Updated weights for policy 1, policy_version 1066684 (0.0006) [2023-12-26 23:08:08,033][105692] Updated weights for policy 0, policy_version 1065662 (0.0009) [2023-12-26 23:08:08,583][105620] Updated weights for policy 1, policy_version 1066694 (0.0008) [2023-12-26 23:08:08,642][105620] Updated weights for policy 1, policy_version 1066704 (0.0010) [2023-12-26 23:08:08,705][105620] Updated weights for policy 1, policy_version 1066714 (0.0010) [2023-12-26 23:08:08,763][105692] Updated weights for policy 0, policy_version 1065672 (0.0011) [2023-12-26 23:08:08,822][105692] Updated weights for policy 0, policy_version 1065682 (0.0011) [2023-12-26 23:08:08,870][105692] Updated weights for policy 0, policy_version 1065692 (0.0011) [2023-12-26 23:08:09,379][105620] Updated weights for policy 1, policy_version 1066724 (0.0009) [2023-12-26 23:08:09,442][105620] Updated weights for policy 1, policy_version 1066734 (0.0008) [2023-12-26 23:08:09,491][105620] Updated weights for policy 1, policy_version 1066744 (0.0010) [2023-12-26 23:08:09,687][105692] Updated weights for policy 0, policy_version 1065702 (0.0011) [2023-12-26 23:08:09,749][105692] Updated weights for policy 0, policy_version 1065712 (0.0009) [2023-12-26 23:08:09,813][105692] Updated weights for policy 0, policy_version 1065722 (0.0008) [2023-12-26 23:08:10,265][105620] Updated weights for policy 1, policy_version 1066754 (0.0010) [2023-12-26 23:08:10,332][105620] Updated weights for policy 1, policy_version 1066764 (0.0011) [2023-12-26 23:08:10,398][105620] Updated weights for policy 1, policy_version 1066774 (0.0011) [2023-12-26 23:08:10,459][105620] Updated weights for policy 1, policy_version 1066784 (0.0009) [2023-12-26 23:08:10,575][105692] Updated weights for policy 0, policy_version 1065732 (0.0009) [2023-12-26 23:08:10,623][105692] Updated weights for policy 0, policy_version 1065742 (0.0007) [2023-12-26 23:08:10,676][105692] Updated weights for policy 0, policy_version 1065752 (0.0010) [2023-12-26 23:08:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 546004992. Throughput: 0: 9806.0, 1: 9925.2. Samples: 546015656. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:08:11,062][104569] Avg episode reward: [(0, '9176.017'), (1, '9168.146')] [2023-12-26 23:08:11,080][105620] Updated weights for policy 1, policy_version 1066794 (0.0008) [2023-12-26 23:08:11,147][105620] Updated weights for policy 1, policy_version 1066805 (0.0010) [2023-12-26 23:08:11,209][105620] Updated weights for policy 1, policy_version 1066815 (0.0008) [2023-12-26 23:08:11,595][105692] Updated weights for policy 0, policy_version 1065762 (0.0009) [2023-12-26 23:08:11,661][105692] Updated weights for policy 0, policy_version 1065772 (0.0009) [2023-12-26 23:08:11,734][105692] Updated weights for policy 0, policy_version 1065782 (0.0009) [2023-12-26 23:08:11,798][105692] Updated weights for policy 0, policy_version 1065792 (0.0007) [2023-12-26 23:08:11,925][105620] Updated weights for policy 1, policy_version 1066825 (0.0008) [2023-12-26 23:08:11,992][105620] Updated weights for policy 1, policy_version 1066835 (0.0010) [2023-12-26 23:08:12,060][105620] Updated weights for policy 1, policy_version 1066845 (0.0010) [2023-12-26 23:08:12,488][105692] Updated weights for policy 0, policy_version 1065802 (0.0008) [2023-12-26 23:08:12,547][105692] Updated weights for policy 0, policy_version 1065812 (0.0008) [2023-12-26 23:08:12,596][105692] Updated weights for policy 0, policy_version 1065822 (0.0008) [2023-12-26 23:08:12,788][105620] Updated weights for policy 1, policy_version 1066855 (0.0010) [2023-12-26 23:08:12,854][105620] Updated weights for policy 1, policy_version 1066865 (0.0009) [2023-12-26 23:08:12,905][105620] Updated weights for policy 1, policy_version 1066875 (0.0005) [2023-12-26 23:08:13,406][105692] Updated weights for policy 0, policy_version 1065832 (0.0008) [2023-12-26 23:08:13,472][105692] Updated weights for policy 0, policy_version 1065842 (0.0008) [2023-12-26 23:08:13,515][105620] Updated weights for policy 1, policy_version 1066885 (0.0008) [2023-12-26 23:08:13,526][105692] Updated weights for policy 0, policy_version 1065852 (0.0006) [2023-12-26 23:08:13,568][105620] Updated weights for policy 1, policy_version 1066895 (0.0011) [2023-12-26 23:08:13,621][105620] Updated weights for policy 1, policy_version 1066905 (0.0011) [2023-12-26 23:08:14,119][105692] Updated weights for policy 0, policy_version 1065862 (0.0006) [2023-12-26 23:08:14,166][105692] Updated weights for policy 0, policy_version 1065872 (0.0005) [2023-12-26 23:08:14,212][105692] Updated weights for policy 0, policy_version 1065882 (0.0005) [2023-12-26 23:08:14,358][105620] Updated weights for policy 1, policy_version 1066915 (0.0009) [2023-12-26 23:08:14,415][105620] Updated weights for policy 1, policy_version 1066925 (0.0007) [2023-12-26 23:08:14,471][105620] Updated weights for policy 1, policy_version 1066935 (0.0008) [2023-12-26 23:08:14,882][105692] Updated weights for policy 0, policy_version 1065892 (0.0008) [2023-12-26 23:08:14,945][105692] Updated weights for policy 0, policy_version 1065902 (0.0009) [2023-12-26 23:08:15,008][105692] Updated weights for policy 0, policy_version 1065912 (0.0006) [2023-12-26 23:08:15,107][105620] Updated weights for policy 1, policy_version 1066946 (0.0009) [2023-12-26 23:08:15,170][105620] Updated weights for policy 1, policy_version 1066956 (0.0009) [2023-12-26 23:08:15,236][105620] Updated weights for policy 1, policy_version 1066966 (0.0009) [2023-12-26 23:08:15,300][105620] Updated weights for policy 1, policy_version 1066976 (0.0010) [2023-12-26 23:08:15,673][105692] Updated weights for policy 0, policy_version 1065922 (0.0008) [2023-12-26 23:08:15,720][105692] Updated weights for policy 0, policy_version 1065932 (0.0005) [2023-12-26 23:08:15,778][105692] Updated weights for policy 0, policy_version 1065942 (0.0005) [2023-12-26 23:08:15,841][105692] Updated weights for policy 0, policy_version 1065952 (0.0008) [2023-12-26 23:08:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 546103296. Throughput: 0: 9805.9, 1: 9839.2. Samples: 546071440. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:08:16,063][104569] Avg episode reward: [(0, '9357.134'), (1, '9168.282')] [2023-12-26 23:08:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001065952_272924672.pth... [2023-12-26 23:08:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001064800_272629760.pth [2023-12-26 23:08:16,081][105620] Updated weights for policy 1, policy_version 1066986 (0.0009) [2023-12-26 23:08:16,143][105620] Updated weights for policy 1, policy_version 1066996 (0.0007) [2023-12-26 23:08:16,197][105620] Updated weights for policy 1, policy_version 1067006 (0.0009) [2023-12-26 23:08:16,208][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001067008_273186816.pth... [2023-12-26 23:08:16,212][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001065824_272883712.pth [2023-12-26 23:08:16,492][105692] Updated weights for policy 0, policy_version 1065962 (0.0009) [2023-12-26 23:08:16,552][105692] Updated weights for policy 0, policy_version 1065972 (0.0009) [2023-12-26 23:08:16,615][105692] Updated weights for policy 0, policy_version 1065982 (0.0009) [2023-12-26 23:08:16,994][105620] Updated weights for policy 1, policy_version 1067016 (0.0010) [2023-12-26 23:08:17,052][105620] Updated weights for policy 1, policy_version 1067026 (0.0009) [2023-12-26 23:08:17,106][105620] Updated weights for policy 1, policy_version 1067036 (0.0010) [2023-12-26 23:08:17,227][105692] Updated weights for policy 0, policy_version 1065992 (0.0007) [2023-12-26 23:08:17,295][105692] Updated weights for policy 0, policy_version 1066002 (0.0008) [2023-12-26 23:08:17,357][105692] Updated weights for policy 0, policy_version 1066012 (0.0007) [2023-12-26 23:08:17,893][105620] Updated weights for policy 1, policy_version 1067046 (0.0009) [2023-12-26 23:08:17,944][105620] Updated weights for policy 1, policy_version 1067056 (0.0009) [2023-12-26 23:08:17,991][105620] Updated weights for policy 1, policy_version 1067066 (0.0009) [2023-12-26 23:08:18,045][105692] Updated weights for policy 0, policy_version 1066022 (0.0009) [2023-12-26 23:08:18,093][105692] Updated weights for policy 0, policy_version 1066032 (0.0009) [2023-12-26 23:08:18,140][105692] Updated weights for policy 0, policy_version 1066042 (0.0008) [2023-12-26 23:08:18,720][105620] Updated weights for policy 1, policy_version 1067076 (0.0008) [2023-12-26 23:08:18,777][105620] Updated weights for policy 1, policy_version 1067086 (0.0005) [2023-12-26 23:08:18,836][105620] Updated weights for policy 1, policy_version 1067096 (0.0005) [2023-12-26 23:08:18,983][105692] Updated weights for policy 0, policy_version 1066052 (0.0009) [2023-12-26 23:08:19,030][105692] Updated weights for policy 0, policy_version 1066062 (0.0009) [2023-12-26 23:08:19,086][105692] Updated weights for policy 0, policy_version 1066072 (0.0009) [2023-12-26 23:08:19,563][105620] Updated weights for policy 1, policy_version 1067106 (0.0009) [2023-12-26 23:08:19,623][105620] Updated weights for policy 1, policy_version 1067116 (0.0009) [2023-12-26 23:08:19,677][105620] Updated weights for policy 1, policy_version 1067126 (0.0010) [2023-12-26 23:08:19,733][105620] Updated weights for policy 1, policy_version 1067136 (0.0008) [2023-12-26 23:08:19,822][105692] Updated weights for policy 0, policy_version 1066082 (0.0009) [2023-12-26 23:08:19,888][105692] Updated weights for policy 0, policy_version 1066092 (0.0009) [2023-12-26 23:08:19,954][105692] Updated weights for policy 0, policy_version 1066102 (0.0009) [2023-12-26 23:08:20,008][105692] Updated weights for policy 0, policy_version 1066112 (0.0008) [2023-12-26 23:08:20,544][105620] Updated weights for policy 1, policy_version 1067146 (0.0008) [2023-12-26 23:08:20,612][105620] Updated weights for policy 1, policy_version 1067156 (0.0008) [2023-12-26 23:08:20,674][105620] Updated weights for policy 1, policy_version 1067166 (0.0008) [2023-12-26 23:08:20,788][105692] Updated weights for policy 0, policy_version 1066122 (0.0009) [2023-12-26 23:08:20,844][105692] Updated weights for policy 0, policy_version 1066132 (0.0008) [2023-12-26 23:08:20,901][105692] Updated weights for policy 0, policy_version 1066142 (0.0008) [2023-12-26 23:08:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 546201600. Throughput: 0: 9890.7, 1: 9812.5. Samples: 546188992. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:08:21,062][104569] Avg episode reward: [(0, '9267.479'), (1, '9259.913')] [2023-12-26 23:08:21,398][105620] Updated weights for policy 1, policy_version 1067176 (0.0009) [2023-12-26 23:08:21,462][105620] Updated weights for policy 1, policy_version 1067186 (0.0007) [2023-12-26 23:08:21,525][105620] Updated weights for policy 1, policy_version 1067196 (0.0008) [2023-12-26 23:08:21,726][105692] Updated weights for policy 0, policy_version 1066152 (0.0010) [2023-12-26 23:08:21,784][105692] Updated weights for policy 0, policy_version 1066162 (0.0008) [2023-12-26 23:08:21,835][105692] Updated weights for policy 0, policy_version 1066172 (0.0009) [2023-12-26 23:08:22,258][105620] Updated weights for policy 1, policy_version 1067206 (0.0009) [2023-12-26 23:08:22,312][105620] Updated weights for policy 1, policy_version 1067216 (0.0009) [2023-12-26 23:08:22,368][105620] Updated weights for policy 1, policy_version 1067226 (0.0009) [2023-12-26 23:08:22,637][105692] Updated weights for policy 0, policy_version 1066182 (0.0009) [2023-12-26 23:08:22,699][105692] Updated weights for policy 0, policy_version 1066192 (0.0011) [2023-12-26 23:08:22,755][105692] Updated weights for policy 0, policy_version 1066202 (0.0010) [2023-12-26 23:08:23,160][105620] Updated weights for policy 1, policy_version 1067236 (0.0010) [2023-12-26 23:08:23,225][105620] Updated weights for policy 1, policy_version 1067246 (0.0008) [2023-12-26 23:08:23,283][105620] Updated weights for policy 1, policy_version 1067256 (0.0008) [2023-12-26 23:08:23,416][105692] Updated weights for policy 0, policy_version 1066212 (0.0010) [2023-12-26 23:08:23,473][105692] Updated weights for policy 0, policy_version 1066222 (0.0010) [2023-12-26 23:08:23,519][105692] Updated weights for policy 0, policy_version 1066232 (0.0010) [2023-12-26 23:08:24,059][105620] Updated weights for policy 1, policy_version 1067266 (0.0010) [2023-12-26 23:08:24,111][105620] Updated weights for policy 1, policy_version 1067276 (0.0009) [2023-12-26 23:08:24,127][105692] Updated weights for policy 0, policy_version 1066242 (0.0009) [2023-12-26 23:08:24,169][105620] Updated weights for policy 1, policy_version 1067286 (0.0007) [2023-12-26 23:08:24,176][105692] Updated weights for policy 0, policy_version 1066252 (0.0007) [2023-12-26 23:08:24,224][105620] Updated weights for policy 1, policy_version 1067296 (0.0006) [2023-12-26 23:08:24,232][105692] Updated weights for policy 0, policy_version 1066262 (0.0008) [2023-12-26 23:08:24,274][105692] Updated weights for policy 0, policy_version 1066272 (0.0007) [2023-12-26 23:08:24,910][105692] Updated weights for policy 0, policy_version 1066282 (0.0008) [2023-12-26 23:08:24,976][105692] Updated weights for policy 0, policy_version 1066292 (0.0006) [2023-12-26 23:08:25,043][105692] Updated weights for policy 0, policy_version 1066302 (0.0005) [2023-12-26 23:08:25,055][105620] Updated weights for policy 1, policy_version 1067306 (0.0006) [2023-12-26 23:08:25,111][105620] Updated weights for policy 1, policy_version 1067316 (0.0007) [2023-12-26 23:08:25,167][105620] Updated weights for policy 1, policy_version 1067326 (0.0011) [2023-12-26 23:08:25,594][105692] Updated weights for policy 0, policy_version 1066312 (0.0006) [2023-12-26 23:08:25,656][105692] Updated weights for policy 0, policy_version 1066322 (0.0005) [2023-12-26 23:08:25,718][105692] Updated weights for policy 0, policy_version 1066332 (0.0005) [2023-12-26 23:08:25,780][105620] Updated weights for policy 1, policy_version 1067336 (0.0006) [2023-12-26 23:08:25,832][105620] Updated weights for policy 1, policy_version 1067346 (0.0005) [2023-12-26 23:08:25,889][105620] Updated weights for policy 1, policy_version 1067356 (0.0006) [2023-12-26 23:08:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 546299904. Throughput: 0: 9925.8, 1: 9761.1. Samples: 546305816. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:08:26,063][104569] Avg episode reward: [(0, '8995.126'), (1, '9258.519')] [2023-12-26 23:08:26,418][105692] Updated weights for policy 0, policy_version 1066342 (0.0008) [2023-12-26 23:08:26,418][105620] Updated weights for policy 1, policy_version 1067366 (0.0005) [2023-12-26 23:08:26,470][105620] Updated weights for policy 1, policy_version 1067376 (0.0005) [2023-12-26 23:08:26,479][105692] Updated weights for policy 0, policy_version 1066352 (0.0010) [2023-12-26 23:08:26,520][105620] Updated weights for policy 1, policy_version 1067386 (0.0005) [2023-12-26 23:08:26,541][105692] Updated weights for policy 0, policy_version 1066362 (0.0010) [2023-12-26 23:08:27,100][105620] Updated weights for policy 1, policy_version 1067396 (0.0007) [2023-12-26 23:08:27,163][105620] Updated weights for policy 1, policy_version 1067406 (0.0009) [2023-12-26 23:08:27,220][105620] Updated weights for policy 1, policy_version 1067416 (0.0008) [2023-12-26 23:08:27,254][105692] Updated weights for policy 0, policy_version 1066372 (0.0010) [2023-12-26 23:08:27,313][105692] Updated weights for policy 0, policy_version 1066382 (0.0010) [2023-12-26 23:08:27,369][105692] Updated weights for policy 0, policy_version 1066392 (0.0009) [2023-12-26 23:08:27,836][105620] Updated weights for policy 1, policy_version 1067426 (0.0006) [2023-12-26 23:08:27,892][105620] Updated weights for policy 1, policy_version 1067436 (0.0005) [2023-12-26 23:08:27,946][105620] Updated weights for policy 1, policy_version 1067446 (0.0005) [2023-12-26 23:08:28,006][105620] Updated weights for policy 1, policy_version 1067456 (0.0008) [2023-12-26 23:08:28,118][105692] Updated weights for policy 0, policy_version 1066402 (0.0010) [2023-12-26 23:08:28,175][105692] Updated weights for policy 0, policy_version 1066412 (0.0010) [2023-12-26 23:08:28,222][105692] Updated weights for policy 0, policy_version 1066422 (0.0010) [2023-12-26 23:08:28,277][105692] Updated weights for policy 0, policy_version 1066432 (0.0010) [2023-12-26 23:08:28,583][105620] Updated weights for policy 1, policy_version 1067466 (0.0007) [2023-12-26 23:08:28,639][105620] Updated weights for policy 1, policy_version 1067476 (0.0005) [2023-12-26 23:08:28,698][105620] Updated weights for policy 1, policy_version 1067486 (0.0005) [2023-12-26 23:08:29,029][105692] Updated weights for policy 0, policy_version 1066442 (0.0010) [2023-12-26 23:08:29,090][105692] Updated weights for policy 0, policy_version 1066452 (0.0010) [2023-12-26 23:08:29,148][105692] Updated weights for policy 0, policy_version 1066462 (0.0010) [2023-12-26 23:08:29,364][105620] Updated weights for policy 1, policy_version 1067496 (0.0010) [2023-12-26 23:08:29,422][105620] Updated weights for policy 1, policy_version 1067506 (0.0007) [2023-12-26 23:08:29,471][105620] Updated weights for policy 1, policy_version 1067516 (0.0009) [2023-12-26 23:08:29,783][105692] Updated weights for policy 0, policy_version 1066472 (0.0006) [2023-12-26 23:08:29,840][105692] Updated weights for policy 0, policy_version 1066482 (0.0006) [2023-12-26 23:08:29,908][105692] Updated weights for policy 0, policy_version 1066492 (0.0006) [2023-12-26 23:08:30,228][105620] Updated weights for policy 1, policy_version 1067526 (0.0008) [2023-12-26 23:08:30,294][105620] Updated weights for policy 1, policy_version 1067536 (0.0008) [2023-12-26 23:08:30,360][105620] Updated weights for policy 1, policy_version 1067546 (0.0008) [2023-12-26 23:08:30,523][105692] Updated weights for policy 0, policy_version 1066502 (0.0008) [2023-12-26 23:08:30,567][105692] Updated weights for policy 0, policy_version 1066512 (0.0010) [2023-12-26 23:08:30,610][105692] Updated weights for policy 0, policy_version 1066522 (0.0005) [2023-12-26 23:08:30,930][105620] Updated weights for policy 1, policy_version 1067556 (0.0007) [2023-12-26 23:08:30,976][105620] Updated weights for policy 1, policy_version 1067566 (0.0005) [2023-12-26 23:08:31,021][105620] Updated weights for policy 1, policy_version 1067576 (0.0005) [2023-12-26 23:08:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 546398208. Throughput: 0: 9914.5, 1: 9908.1. Samples: 546369184. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:08:31,063][104569] Avg episode reward: [(0, '9088.129'), (1, '9076.358')] [2023-12-26 23:08:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001066528_273072128.pth... [2023-12-26 23:08:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001065376_272777216.pth [2023-12-26 23:08:31,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001067584_273334272.pth... [2023-12-26 23:08:31,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001066432_273039360.pth [2023-12-26 23:08:31,318][105692] Updated weights for policy 0, policy_version 1066532 (0.0006) [2023-12-26 23:08:31,389][105692] Updated weights for policy 0, policy_version 1066542 (0.0008) [2023-12-26 23:08:31,449][105692] Updated weights for policy 0, policy_version 1066552 (0.0008) [2023-12-26 23:08:31,779][105620] Updated weights for policy 1, policy_version 1067586 (0.0008) [2023-12-26 23:08:31,840][105620] Updated weights for policy 1, policy_version 1067596 (0.0010) [2023-12-26 23:08:31,899][105620] Updated weights for policy 1, policy_version 1067606 (0.0010) [2023-12-26 23:08:31,953][105620] Updated weights for policy 1, policy_version 1067616 (0.0008) [2023-12-26 23:08:32,185][105692] Updated weights for policy 0, policy_version 1066562 (0.0009) [2023-12-26 23:08:32,254][105692] Updated weights for policy 0, policy_version 1066572 (0.0009) [2023-12-26 23:08:32,319][105692] Updated weights for policy 0, policy_version 1066582 (0.0008) [2023-12-26 23:08:32,383][105692] Updated weights for policy 0, policy_version 1066592 (0.0008) [2023-12-26 23:08:32,623][105620] Updated weights for policy 1, policy_version 1067626 (0.0006) [2023-12-26 23:08:32,688][105620] Updated weights for policy 1, policy_version 1067636 (0.0006) [2023-12-26 23:08:32,741][105620] Updated weights for policy 1, policy_version 1067646 (0.0010) [2023-12-26 23:08:33,028][105692] Updated weights for policy 0, policy_version 1066602 (0.0009) [2023-12-26 23:08:33,081][105692] Updated weights for policy 0, policy_version 1066612 (0.0010) [2023-12-26 23:08:33,137][105692] Updated weights for policy 0, policy_version 1066622 (0.0008) [2023-12-26 23:08:33,353][105620] Updated weights for policy 1, policy_version 1067656 (0.0006) [2023-12-26 23:08:33,412][105620] Updated weights for policy 1, policy_version 1067666 (0.0005) [2023-12-26 23:08:33,469][105620] Updated weights for policy 1, policy_version 1067676 (0.0009) [2023-12-26 23:08:33,775][105692] Updated weights for policy 0, policy_version 1066632 (0.0008) [2023-12-26 23:08:33,829][105692] Updated weights for policy 0, policy_version 1066642 (0.0008) [2023-12-26 23:08:33,882][105692] Updated weights for policy 0, policy_version 1066652 (0.0009) [2023-12-26 23:08:34,236][105620] Updated weights for policy 1, policy_version 1067686 (0.0007) [2023-12-26 23:08:34,301][105620] Updated weights for policy 1, policy_version 1067696 (0.0006) [2023-12-26 23:08:34,369][105620] Updated weights for policy 1, policy_version 1067706 (0.0006) [2023-12-26 23:08:34,494][105692] Updated weights for policy 0, policy_version 1066662 (0.0009) [2023-12-26 23:08:34,546][105692] Updated weights for policy 0, policy_version 1066672 (0.0010) [2023-12-26 23:08:34,603][105692] Updated weights for policy 0, policy_version 1066682 (0.0010) [2023-12-26 23:08:34,959][105620] Updated weights for policy 1, policy_version 1067716 (0.0009) [2023-12-26 23:08:35,028][105620] Updated weights for policy 1, policy_version 1067726 (0.0010) [2023-12-26 23:08:35,095][105620] Updated weights for policy 1, policy_version 1067736 (0.0009) [2023-12-26 23:08:35,224][105692] Updated weights for policy 0, policy_version 1066692 (0.0008) [2023-12-26 23:08:35,275][105692] Updated weights for policy 0, policy_version 1066702 (0.0005) [2023-12-26 23:08:35,324][105692] Updated weights for policy 0, policy_version 1066712 (0.0005) [2023-12-26 23:08:35,835][105620] Updated weights for policy 1, policy_version 1067746 (0.0010) [2023-12-26 23:08:35,883][105620] Updated weights for policy 1, policy_version 1067756 (0.0010) [2023-12-26 23:08:35,901][105692] Updated weights for policy 0, policy_version 1066722 (0.0008) [2023-12-26 23:08:35,929][105620] Updated weights for policy 1, policy_version 1067766 (0.0010) [2023-12-26 23:08:35,953][105692] Updated weights for policy 0, policy_version 1066732 (0.0005) [2023-12-26 23:08:35,983][105620] Updated weights for policy 1, policy_version 1067776 (0.0009) [2023-12-26 23:08:36,008][105692] Updated weights for policy 0, policy_version 1066742 (0.0006) [2023-12-26 23:08:36,060][105692] Updated weights for policy 0, policy_version 1066752 (0.0005) [2023-12-26 23:08:36,062][104569] Fps is (10 sec: 21299.6, 60 sec: 19934.0, 300 sec: 19633.0). Total num frames: 546512896. Throughput: 0: 9994.8, 1: 9971.3. Samples: 546493124. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:08:36,062][104569] Avg episode reward: [(0, '9177.410'), (1, '8984.859')] [2023-12-26 23:08:36,719][105620] Updated weights for policy 1, policy_version 1067786 (0.0011) [2023-12-26 23:08:36,766][105692] Updated weights for policy 0, policy_version 1066762 (0.0011) [2023-12-26 23:08:36,768][105620] Updated weights for policy 1, policy_version 1067796 (0.0010) [2023-12-26 23:08:36,821][105620] Updated weights for policy 1, policy_version 1067806 (0.0010) [2023-12-26 23:08:36,822][105692] Updated weights for policy 0, policy_version 1066772 (0.0011) [2023-12-26 23:08:36,872][105692] Updated weights for policy 0, policy_version 1066782 (0.0010) [2023-12-26 23:08:37,597][105620] Updated weights for policy 1, policy_version 1067816 (0.0010) [2023-12-26 23:08:37,631][105692] Updated weights for policy 0, policy_version 1066792 (0.0011) [2023-12-26 23:08:37,652][105620] Updated weights for policy 1, policy_version 1067826 (0.0010) [2023-12-26 23:08:37,684][105692] Updated weights for policy 0, policy_version 1066802 (0.0011) [2023-12-26 23:08:37,705][105620] Updated weights for policy 1, policy_version 1067836 (0.0010) [2023-12-26 23:08:37,745][105692] Updated weights for policy 0, policy_version 1066812 (0.0011) [2023-12-26 23:08:38,451][105620] Updated weights for policy 1, policy_version 1067846 (0.0009) [2023-12-26 23:08:38,471][105692] Updated weights for policy 0, policy_version 1066822 (0.0008) [2023-12-26 23:08:38,520][105620] Updated weights for policy 1, policy_version 1067856 (0.0010) [2023-12-26 23:08:38,535][105692] Updated weights for policy 0, policy_version 1066832 (0.0006) [2023-12-26 23:08:38,588][105620] Updated weights for policy 1, policy_version 1067866 (0.0007) [2023-12-26 23:08:38,596][105692] Updated weights for policy 0, policy_version 1066842 (0.0008) [2023-12-26 23:08:39,186][105620] Updated weights for policy 1, policy_version 1067876 (0.0008) [2023-12-26 23:08:39,247][105620] Updated weights for policy 1, policy_version 1067886 (0.0008) [2023-12-26 23:08:39,303][105692] Updated weights for policy 0, policy_version 1066852 (0.0007) [2023-12-26 23:08:39,312][105620] Updated weights for policy 1, policy_version 1067896 (0.0008) [2023-12-26 23:08:39,378][105692] Updated weights for policy 0, policy_version 1066862 (0.0006) [2023-12-26 23:08:39,445][105692] Updated weights for policy 0, policy_version 1066872 (0.0009) [2023-12-26 23:08:40,062][105620] Updated weights for policy 1, policy_version 1067906 (0.0009) [2023-12-26 23:08:40,128][105620] Updated weights for policy 1, policy_version 1067916 (0.0009) [2023-12-26 23:08:40,187][105620] Updated weights for policy 1, policy_version 1067926 (0.0009) [2023-12-26 23:08:40,237][105692] Updated weights for policy 0, policy_version 1066882 (0.0009) [2023-12-26 23:08:40,251][105620] Updated weights for policy 1, policy_version 1067936 (0.0009) [2023-12-26 23:08:40,296][105692] Updated weights for policy 0, policy_version 1066892 (0.0008) [2023-12-26 23:08:40,358][105692] Updated weights for policy 0, policy_version 1066902 (0.0010) [2023-12-26 23:08:40,421][105692] Updated weights for policy 0, policy_version 1066912 (0.0009) [2023-12-26 23:08:41,004][105620] Updated weights for policy 1, policy_version 1067946 (0.0008) [2023-12-26 23:08:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 546594816. Throughput: 0: 10005.6, 1: 9867.1. Samples: 546609588. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:08:41,062][104569] Avg episode reward: [(0, '9264.606'), (1, '8984.243')] [2023-12-26 23:08:41,070][105620] Updated weights for policy 1, policy_version 1067956 (0.0009) [2023-12-26 23:08:41,136][105620] Updated weights for policy 1, policy_version 1067966 (0.0009) [2023-12-26 23:08:41,205][105692] Updated weights for policy 0, policy_version 1066922 (0.0009) [2023-12-26 23:08:41,272][105692] Updated weights for policy 0, policy_version 1066932 (0.0009) [2023-12-26 23:08:41,336][105692] Updated weights for policy 0, policy_version 1066942 (0.0009) [2023-12-26 23:08:41,972][105620] Updated weights for policy 1, policy_version 1067976 (0.0009) [2023-12-26 23:08:42,029][105620] Updated weights for policy 1, policy_version 1067986 (0.0009) [2023-12-26 23:08:42,059][105692] Updated weights for policy 0, policy_version 1066952 (0.0007) [2023-12-26 23:08:42,092][105620] Updated weights for policy 1, policy_version 1067996 (0.0008) [2023-12-26 23:08:42,118][105692] Updated weights for policy 0, policy_version 1066962 (0.0006) [2023-12-26 23:08:42,173][105692] Updated weights for policy 0, policy_version 1066972 (0.0009) [2023-12-26 23:08:42,770][105620] Updated weights for policy 1, policy_version 1068006 (0.0009) [2023-12-26 23:08:42,816][105620] Updated weights for policy 1, policy_version 1068016 (0.0008) [2023-12-26 23:08:42,866][105620] Updated weights for policy 1, policy_version 1068026 (0.0008) [2023-12-26 23:08:42,969][105692] Updated weights for policy 0, policy_version 1066982 (0.0009) [2023-12-26 23:08:43,033][105692] Updated weights for policy 0, policy_version 1066992 (0.0009) [2023-12-26 23:08:43,097][105692] Updated weights for policy 0, policy_version 1067002 (0.0008) [2023-12-26 23:08:43,604][105620] Updated weights for policy 1, policy_version 1068036 (0.0010) [2023-12-26 23:08:43,662][105620] Updated weights for policy 1, policy_version 1068046 (0.0010) [2023-12-26 23:08:43,720][105620] Updated weights for policy 1, policy_version 1068056 (0.0005) [2023-12-26 23:08:43,805][105692] Updated weights for policy 0, policy_version 1067012 (0.0007) [2023-12-26 23:08:43,856][105692] Updated weights for policy 0, policy_version 1067022 (0.0007) [2023-12-26 23:08:43,915][105692] Updated weights for policy 0, policy_version 1067032 (0.0008) [2023-12-26 23:08:44,411][105620] Updated weights for policy 1, policy_version 1068066 (0.0007) [2023-12-26 23:08:44,478][105620] Updated weights for policy 1, policy_version 1068076 (0.0006) [2023-12-26 23:08:44,524][105692] Updated weights for policy 0, policy_version 1067042 (0.0006) [2023-12-26 23:08:44,527][105620] Updated weights for policy 1, policy_version 1068086 (0.0007) [2023-12-26 23:08:44,583][105692] Updated weights for policy 0, policy_version 1067052 (0.0011) [2023-12-26 23:08:44,588][105620] Updated weights for policy 1, policy_version 1068096 (0.0006) [2023-12-26 23:08:44,652][105692] Updated weights for policy 0, policy_version 1067062 (0.0010) [2023-12-26 23:08:44,713][105692] Updated weights for policy 0, policy_version 1067072 (0.0010) [2023-12-26 23:08:45,268][105620] Updated weights for policy 1, policy_version 1068106 (0.0008) [2023-12-26 23:08:45,317][105620] Updated weights for policy 1, policy_version 1068116 (0.0009) [2023-12-26 23:08:45,376][105620] Updated weights for policy 1, policy_version 1068126 (0.0009) [2023-12-26 23:08:45,437][105692] Updated weights for policy 0, policy_version 1067082 (0.0006) [2023-12-26 23:08:45,485][105692] Updated weights for policy 0, policy_version 1067092 (0.0007) [2023-12-26 23:08:45,542][105692] Updated weights for policy 0, policy_version 1067102 (0.0009) [2023-12-26 23:08:46,062][104569] Fps is (10 sec: 18022.1, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 546693120. Throughput: 0: 9923.5, 1: 9797.0. Samples: 546664760. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:08:46,062][104569] Avg episode reward: [(0, '9260.457'), (1, '9076.478')] [2023-12-26 23:08:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001067104_273219584.pth... [2023-12-26 23:08:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001068128_273473536.pth... [2023-12-26 23:08:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001065952_272924672.pth [2023-12-26 23:08:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001067008_273186816.pth [2023-12-26 23:08:46,204][105692] Updated weights for policy 0, policy_version 1067112 (0.0007) [2023-12-26 23:08:46,220][105620] Updated weights for policy 1, policy_version 1068136 (0.0009) [2023-12-26 23:08:46,257][105692] Updated weights for policy 0, policy_version 1067122 (0.0007) [2023-12-26 23:08:46,277][105620] Updated weights for policy 1, policy_version 1068146 (0.0008) [2023-12-26 23:08:46,321][105692] Updated weights for policy 0, policy_version 1067132 (0.0005) [2023-12-26 23:08:46,322][105586] KL-divergence is very high: 100.2244 [2023-12-26 23:08:46,335][105620] Updated weights for policy 1, policy_version 1068156 (0.0010) [2023-12-26 23:08:46,959][105620] Updated weights for policy 1, policy_version 1068166 (0.0007) [2023-12-26 23:08:47,013][105620] Updated weights for policy 1, policy_version 1068176 (0.0006) [2023-12-26 23:08:47,036][105692] Updated weights for policy 0, policy_version 1067142 (0.0005) [2023-12-26 23:08:47,075][105620] Updated weights for policy 1, policy_version 1068186 (0.0005) [2023-12-26 23:08:47,094][105692] Updated weights for policy 0, policy_version 1067152 (0.0006) [2023-12-26 23:08:47,155][105692] Updated weights for policy 0, policy_version 1067162 (0.0005) [2023-12-26 23:08:47,726][105620] Updated weights for policy 1, policy_version 1068196 (0.0007) [2023-12-26 23:08:47,776][105692] Updated weights for policy 0, policy_version 1067172 (0.0005) [2023-12-26 23:08:47,779][105620] Updated weights for policy 1, policy_version 1068206 (0.0009) [2023-12-26 23:08:47,824][105692] Updated weights for policy 0, policy_version 1067182 (0.0005) [2023-12-26 23:08:47,834][105620] Updated weights for policy 1, policy_version 1068216 (0.0006) [2023-12-26 23:08:47,877][105692] Updated weights for policy 0, policy_version 1067192 (0.0006) [2023-12-26 23:08:48,531][105692] Updated weights for policy 0, policy_version 1067202 (0.0007) [2023-12-26 23:08:48,590][105692] Updated weights for policy 0, policy_version 1067212 (0.0011) [2023-12-26 23:08:48,630][105620] Updated weights for policy 1, policy_version 1068226 (0.0007) [2023-12-26 23:08:48,642][105692] Updated weights for policy 0, policy_version 1067222 (0.0011) [2023-12-26 23:08:48,684][105620] Updated weights for policy 1, policy_version 1068236 (0.0006) [2023-12-26 23:08:48,704][105692] Updated weights for policy 0, policy_version 1067232 (0.0011) [2023-12-26 23:08:48,745][105620] Updated weights for policy 1, policy_version 1068246 (0.0008) [2023-12-26 23:08:48,805][105620] Updated weights for policy 1, policy_version 1068256 (0.0008) [2023-12-26 23:08:49,486][105692] Updated weights for policy 0, policy_version 1067242 (0.0011) [2023-12-26 23:08:49,548][105692] Updated weights for policy 0, policy_version 1067252 (0.0010) [2023-12-26 23:08:49,574][105620] Updated weights for policy 1, policy_version 1068266 (0.0011) [2023-12-26 23:08:49,603][105692] Updated weights for policy 0, policy_version 1067262 (0.0010) [2023-12-26 23:08:49,634][105620] Updated weights for policy 1, policy_version 1068276 (0.0006) [2023-12-26 23:08:49,700][105620] Updated weights for policy 1, policy_version 1068286 (0.0007) [2023-12-26 23:08:50,352][105692] Updated weights for policy 0, policy_version 1067272 (0.0010) [2023-12-26 23:08:50,403][105692] Updated weights for policy 0, policy_version 1067282 (0.0010) [2023-12-26 23:08:50,406][105620] Updated weights for policy 1, policy_version 1068296 (0.0007) [2023-12-26 23:08:50,462][105692] Updated weights for policy 0, policy_version 1067292 (0.0011) [2023-12-26 23:08:50,465][105620] Updated weights for policy 1, policy_version 1068306 (0.0005) [2023-12-26 23:08:50,536][105620] Updated weights for policy 1, policy_version 1068316 (0.0005) [2023-12-26 23:08:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 546791424. Throughput: 0: 9942.8, 1: 9834.8. Samples: 546784164. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:08:51,063][104569] Avg episode reward: [(0, '9347.966'), (1, '9076.129')] [2023-12-26 23:08:51,211][105620] Updated weights for policy 1, policy_version 1068326 (0.0008) [2023-12-26 23:08:51,223][105692] Updated weights for policy 0, policy_version 1067302 (0.0008) [2023-12-26 23:08:51,276][105620] Updated weights for policy 1, policy_version 1068336 (0.0009) [2023-12-26 23:08:51,294][105692] Updated weights for policy 0, policy_version 1067312 (0.0006) [2023-12-26 23:08:51,331][105620] Updated weights for policy 1, policy_version 1068346 (0.0008) [2023-12-26 23:08:51,356][105692] Updated weights for policy 0, policy_version 1067322 (0.0009) [2023-12-26 23:08:52,051][105692] Updated weights for policy 0, policy_version 1067332 (0.0009) [2023-12-26 23:08:52,120][105692] Updated weights for policy 0, policy_version 1067342 (0.0008) [2023-12-26 23:08:52,127][105620] Updated weights for policy 1, policy_version 1068356 (0.0007) [2023-12-26 23:08:52,176][105620] Updated weights for policy 1, policy_version 1068366 (0.0006) [2023-12-26 23:08:52,181][105692] Updated weights for policy 0, policy_version 1067352 (0.0009) [2023-12-26 23:08:52,232][105620] Updated weights for policy 1, policy_version 1068376 (0.0008) [2023-12-26 23:08:52,836][105692] Updated weights for policy 0, policy_version 1067362 (0.0007) [2023-12-26 23:08:52,898][105692] Updated weights for policy 0, policy_version 1067372 (0.0009) [2023-12-26 23:08:52,956][105692] Updated weights for policy 0, policy_version 1067382 (0.0009) [2023-12-26 23:08:53,018][105692] Updated weights for policy 0, policy_version 1067392 (0.0009) [2023-12-26 23:08:53,036][105620] Updated weights for policy 1, policy_version 1068386 (0.0009) [2023-12-26 23:08:53,096][105620] Updated weights for policy 1, policy_version 1068396 (0.0009) [2023-12-26 23:08:53,158][105620] Updated weights for policy 1, policy_version 1068406 (0.0009) [2023-12-26 23:08:53,214][105620] Updated weights for policy 1, policy_version 1068416 (0.0009) [2023-12-26 23:08:53,758][105692] Updated weights for policy 0, policy_version 1067402 (0.0009) [2023-12-26 23:08:53,823][105692] Updated weights for policy 0, policy_version 1067412 (0.0009) [2023-12-26 23:08:53,875][105692] Updated weights for policy 0, policy_version 1067422 (0.0010) [2023-12-26 23:08:53,961][105620] Updated weights for policy 1, policy_version 1068426 (0.0008) [2023-12-26 23:08:54,022][105620] Updated weights for policy 1, policy_version 1068436 (0.0009) [2023-12-26 23:08:54,083][105620] Updated weights for policy 1, policy_version 1068446 (0.0009) [2023-12-26 23:08:54,660][105692] Updated weights for policy 0, policy_version 1067432 (0.0009) [2023-12-26 23:08:54,710][105692] Updated weights for policy 0, policy_version 1067442 (0.0009) [2023-12-26 23:08:54,758][105620] Updated weights for policy 1, policy_version 1068456 (0.0008) [2023-12-26 23:08:54,760][105692] Updated weights for policy 0, policy_version 1067452 (0.0007) [2023-12-26 23:08:54,806][105620] Updated weights for policy 1, policy_version 1068466 (0.0007) [2023-12-26 23:08:54,864][105620] Updated weights for policy 1, policy_version 1068476 (0.0009) [2023-12-26 23:08:55,562][105692] Updated weights for policy 0, policy_version 1067462 (0.0009) [2023-12-26 23:08:55,624][105692] Updated weights for policy 0, policy_version 1067472 (0.0009) [2023-12-26 23:08:55,651][105620] Updated weights for policy 1, policy_version 1068486 (0.0008) [2023-12-26 23:08:55,686][105692] Updated weights for policy 0, policy_version 1067482 (0.0008) [2023-12-26 23:08:55,712][105620] Updated weights for policy 1, policy_version 1068496 (0.0008) [2023-12-26 23:08:55,771][105620] Updated weights for policy 1, policy_version 1068506 (0.0008) [2023-12-26 23:08:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 546889728. Throughput: 0: 9845.2, 1: 9726.4. Samples: 546896380. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:08:56,062][104569] Avg episode reward: [(0, '9259.418'), (1, '9349.784')] [2023-12-26 23:08:56,281][105692] Updated weights for policy 0, policy_version 1067492 (0.0007) [2023-12-26 23:08:56,335][105692] Updated weights for policy 0, policy_version 1067502 (0.0009) [2023-12-26 23:08:56,392][105692] Updated weights for policy 0, policy_version 1067512 (0.0009) [2023-12-26 23:08:56,607][105620] Updated weights for policy 1, policy_version 1068516 (0.0011) [2023-12-26 23:08:56,670][105620] Updated weights for policy 1, policy_version 1068526 (0.0009) [2023-12-26 23:08:56,724][105620] Updated weights for policy 1, policy_version 1068536 (0.0007) [2023-12-26 23:08:57,087][105692] Updated weights for policy 0, policy_version 1067522 (0.0008) [2023-12-26 23:08:57,140][105692] Updated weights for policy 0, policy_version 1067532 (0.0009) [2023-12-26 23:08:57,188][105692] Updated weights for policy 0, policy_version 1067542 (0.0010) [2023-12-26 23:08:57,245][105692] Updated weights for policy 0, policy_version 1067552 (0.0006) [2023-12-26 23:08:57,454][105620] Updated weights for policy 1, policy_version 1068546 (0.0008) [2023-12-26 23:08:57,505][105620] Updated weights for policy 1, policy_version 1068556 (0.0010) [2023-12-26 23:08:57,554][105620] Updated weights for policy 1, policy_version 1068566 (0.0005) [2023-12-26 23:08:57,605][105620] Updated weights for policy 1, policy_version 1068576 (0.0007) [2023-12-26 23:08:57,940][105692] Updated weights for policy 0, policy_version 1067562 (0.0010) [2023-12-26 23:08:57,994][105692] Updated weights for policy 0, policy_version 1067572 (0.0010) [2023-12-26 23:08:58,048][105692] Updated weights for policy 0, policy_version 1067582 (0.0010) [2023-12-26 23:08:58,220][105620] Updated weights for policy 1, policy_version 1068586 (0.0007) [2023-12-26 23:08:58,284][105620] Updated weights for policy 1, policy_version 1068596 (0.0007) [2023-12-26 23:08:58,358][105620] Updated weights for policy 1, policy_version 1068606 (0.0009) [2023-12-26 23:08:58,819][105692] Updated weights for policy 0, policy_version 1067592 (0.0008) [2023-12-26 23:08:58,892][105692] Updated weights for policy 0, policy_version 1067602 (0.0007) [2023-12-26 23:08:58,963][105692] Updated weights for policy 0, policy_version 1067612 (0.0007) [2023-12-26 23:08:59,166][105620] Updated weights for policy 1, policy_version 1068616 (0.0008) [2023-12-26 23:08:59,229][105620] Updated weights for policy 1, policy_version 1068626 (0.0008) [2023-12-26 23:08:59,295][105620] Updated weights for policy 1, policy_version 1068636 (0.0008) [2023-12-26 23:08:59,640][105692] Updated weights for policy 0, policy_version 1067622 (0.0007) [2023-12-26 23:08:59,703][105692] Updated weights for policy 0, policy_version 1067632 (0.0006) [2023-12-26 23:08:59,763][105692] Updated weights for policy 0, policy_version 1067642 (0.0008) [2023-12-26 23:09:00,029][105620] Updated weights for policy 1, policy_version 1068646 (0.0007) [2023-12-26 23:09:00,090][105620] Updated weights for policy 1, policy_version 1068656 (0.0005) [2023-12-26 23:09:00,152][105620] Updated weights for policy 1, policy_version 1068666 (0.0009) [2023-12-26 23:09:00,409][105692] Updated weights for policy 0, policy_version 1067652 (0.0007) [2023-12-26 23:09:00,465][105692] Updated weights for policy 0, policy_version 1067662 (0.0008) [2023-12-26 23:09:00,520][105692] Updated weights for policy 0, policy_version 1067672 (0.0009) [2023-12-26 23:09:00,835][105620] Updated weights for policy 1, policy_version 1068676 (0.0007) [2023-12-26 23:09:00,882][105620] Updated weights for policy 1, policy_version 1068686 (0.0005) [2023-12-26 23:09:00,928][105620] Updated weights for policy 1, policy_version 1068696 (0.0005) [2023-12-26 23:09:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 546988032. Throughput: 0: 9939.9, 1: 9708.8. Samples: 546955632. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:09:01,063][104569] Avg episode reward: [(0, '9259.805'), (1, '9349.593')] [2023-12-26 23:09:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001068704_273620992.pth... [2023-12-26 23:09:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001067680_273367040.pth... [2023-12-26 23:09:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001067584_273334272.pth [2023-12-26 23:09:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001066528_273072128.pth [2023-12-26 23:09:01,283][105692] Updated weights for policy 0, policy_version 1067682 (0.0008) [2023-12-26 23:09:01,348][105692] Updated weights for policy 0, policy_version 1067692 (0.0008) [2023-12-26 23:09:01,412][105692] Updated weights for policy 0, policy_version 1067702 (0.0008) [2023-12-26 23:09:01,473][105692] Updated weights for policy 0, policy_version 1067712 (0.0009) [2023-12-26 23:09:01,588][105620] Updated weights for policy 1, policy_version 1068706 (0.0006) [2023-12-26 23:09:01,658][105620] Updated weights for policy 1, policy_version 1068716 (0.0009) [2023-12-26 23:09:01,723][105620] Updated weights for policy 1, policy_version 1068726 (0.0008) [2023-12-26 23:09:01,790][105620] Updated weights for policy 1, policy_version 1068736 (0.0009) [2023-12-26 23:09:02,169][105692] Updated weights for policy 0, policy_version 1067722 (0.0009) [2023-12-26 23:09:02,226][105692] Updated weights for policy 0, policy_version 1067732 (0.0009) [2023-12-26 23:09:02,288][105692] Updated weights for policy 0, policy_version 1067742 (0.0010) [2023-12-26 23:09:02,501][105620] Updated weights for policy 1, policy_version 1068746 (0.0009) [2023-12-26 23:09:02,559][105620] Updated weights for policy 1, policy_version 1068756 (0.0009) [2023-12-26 23:09:02,617][105620] Updated weights for policy 1, policy_version 1068766 (0.0009) [2023-12-26 23:09:03,097][105692] Updated weights for policy 0, policy_version 1067752 (0.0009) [2023-12-26 23:09:03,147][105692] Updated weights for policy 0, policy_version 1067763 (0.0007) [2023-12-26 23:09:03,203][105692] Updated weights for policy 0, policy_version 1067773 (0.0006) [2023-12-26 23:09:03,241][105620] Updated weights for policy 1, policy_version 1068776 (0.0006) [2023-12-26 23:09:03,299][105620] Updated weights for policy 1, policy_version 1068786 (0.0005) [2023-12-26 23:09:03,362][105620] Updated weights for policy 1, policy_version 1068796 (0.0005) [2023-12-26 23:09:03,951][105692] Updated weights for policy 0, policy_version 1067783 (0.0008) [2023-12-26 23:09:03,980][105620] Updated weights for policy 1, policy_version 1068806 (0.0006) [2023-12-26 23:09:03,998][105692] Updated weights for policy 0, policy_version 1067793 (0.0008) [2023-12-26 23:09:04,039][105620] Updated weights for policy 1, policy_version 1068816 (0.0008) [2023-12-26 23:09:04,048][105692] Updated weights for policy 0, policy_version 1067803 (0.0007) [2023-12-26 23:09:04,105][105620] Updated weights for policy 1, policy_version 1068826 (0.0009) [2023-12-26 23:09:04,829][105692] Updated weights for policy 0, policy_version 1067813 (0.0007) [2023-12-26 23:09:04,842][105620] Updated weights for policy 1, policy_version 1068836 (0.0009) [2023-12-26 23:09:04,880][105692] Updated weights for policy 0, policy_version 1067823 (0.0005) [2023-12-26 23:09:04,901][105620] Updated weights for policy 1, policy_version 1068846 (0.0010) [2023-12-26 23:09:04,928][105692] Updated weights for policy 0, policy_version 1067833 (0.0006) [2023-12-26 23:09:04,962][105620] Updated weights for policy 1, policy_version 1068856 (0.0010) [2023-12-26 23:09:05,560][105692] Updated weights for policy 0, policy_version 1067843 (0.0007) [2023-12-26 23:09:05,614][105692] Updated weights for policy 0, policy_version 1067853 (0.0010) [2023-12-26 23:09:05,672][105692] Updated weights for policy 0, policy_version 1067863 (0.0010) [2023-12-26 23:09:05,720][105620] Updated weights for policy 1, policy_version 1068866 (0.0010) [2023-12-26 23:09:05,778][105620] Updated weights for policy 1, policy_version 1068876 (0.0010) [2023-12-26 23:09:05,837][105620] Updated weights for policy 1, policy_version 1068886 (0.0010) [2023-12-26 23:09:05,898][105620] Updated weights for policy 1, policy_version 1068896 (0.0007) [2023-12-26 23:09:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 547086336. Throughput: 0: 9849.3, 1: 9783.2. Samples: 547072452. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:09:06,062][104569] Avg episode reward: [(0, '9350.331'), (1, '9258.771')] [2023-12-26 23:09:06,296][105692] Updated weights for policy 0, policy_version 1067873 (0.0010) [2023-12-26 23:09:06,361][105692] Updated weights for policy 0, policy_version 1067883 (0.0006) [2023-12-26 23:09:06,429][105692] Updated weights for policy 0, policy_version 1067893 (0.0006) [2023-12-26 23:09:06,505][105692] Updated weights for policy 0, policy_version 1067903 (0.0006) [2023-12-26 23:09:06,589][105620] Updated weights for policy 1, policy_version 1068906 (0.0006) [2023-12-26 23:09:06,655][105620] Updated weights for policy 1, policy_version 1068916 (0.0005) [2023-12-26 23:09:06,721][105620] Updated weights for policy 1, policy_version 1068926 (0.0005) [2023-12-26 23:09:07,141][105692] Updated weights for policy 0, policy_version 1067913 (0.0005) [2023-12-26 23:09:07,189][105692] Updated weights for policy 0, policy_version 1067923 (0.0006) [2023-12-26 23:09:07,244][105692] Updated weights for policy 0, policy_version 1067933 (0.0010) [2023-12-26 23:09:07,291][105620] Updated weights for policy 1, policy_version 1068936 (0.0008) [2023-12-26 23:09:07,360][105620] Updated weights for policy 1, policy_version 1068946 (0.0010) [2023-12-26 23:09:07,428][105620] Updated weights for policy 1, policy_version 1068956 (0.0008) [2023-12-26 23:09:07,967][105692] Updated weights for policy 0, policy_version 1067943 (0.0010) [2023-12-26 23:09:08,015][105692] Updated weights for policy 0, policy_version 1067953 (0.0010) [2023-12-26 23:09:08,061][105692] Updated weights for policy 0, policy_version 1067963 (0.0010) [2023-12-26 23:09:08,098][105620] Updated weights for policy 1, policy_version 1068966 (0.0008) [2023-12-26 23:09:08,161][105620] Updated weights for policy 1, policy_version 1068976 (0.0006) [2023-12-26 23:09:08,227][105620] Updated weights for policy 1, policy_version 1068986 (0.0007) [2023-12-26 23:09:08,849][105692] Updated weights for policy 0, policy_version 1067973 (0.0011) [2023-12-26 23:09:08,898][105620] Updated weights for policy 1, policy_version 1068996 (0.0006) [2023-12-26 23:09:08,911][105692] Updated weights for policy 0, policy_version 1067983 (0.0010) [2023-12-26 23:09:08,951][105620] Updated weights for policy 1, policy_version 1069006 (0.0006) [2023-12-26 23:09:08,971][105692] Updated weights for policy 0, policy_version 1067993 (0.0010) [2023-12-26 23:09:09,006][105620] Updated weights for policy 1, policy_version 1069016 (0.0007) [2023-12-26 23:09:09,717][105620] Updated weights for policy 1, policy_version 1069026 (0.0006) [2023-12-26 23:09:09,743][105692] Updated weights for policy 0, policy_version 1068003 (0.0009) [2023-12-26 23:09:09,780][105620] Updated weights for policy 1, policy_version 1069036 (0.0009) [2023-12-26 23:09:09,801][105692] Updated weights for policy 0, policy_version 1068013 (0.0008) [2023-12-26 23:09:09,847][105620] Updated weights for policy 1, policy_version 1069046 (0.0007) [2023-12-26 23:09:09,869][105692] Updated weights for policy 0, policy_version 1068023 (0.0008) [2023-12-26 23:09:09,909][105620] Updated weights for policy 1, policy_version 1069056 (0.0008) [2023-12-26 23:09:10,535][105692] Updated weights for policy 0, policy_version 1068033 (0.0007) [2023-12-26 23:09:10,590][105692] Updated weights for policy 0, policy_version 1068043 (0.0008) [2023-12-26 23:09:10,646][105692] Updated weights for policy 0, policy_version 1068053 (0.0009) [2023-12-26 23:09:10,667][105620] Updated weights for policy 1, policy_version 1069066 (0.0008) [2023-12-26 23:09:10,711][105692] Updated weights for policy 0, policy_version 1068063 (0.0009) [2023-12-26 23:09:10,727][105620] Updated weights for policy 1, policy_version 1069076 (0.0008) [2023-12-26 23:09:10,793][105620] Updated weights for policy 1, policy_version 1069086 (0.0009) [2023-12-26 23:09:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 547184640. Throughput: 0: 9834.9, 1: 9849.1. Samples: 547191596. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:09:11,063][104569] Avg episode reward: [(0, '9349.135'), (1, '9258.772')] [2023-12-26 23:09:11,371][105692] Updated weights for policy 0, policy_version 1068073 (0.0014) [2023-12-26 23:09:11,427][105692] Updated weights for policy 0, policy_version 1068083 (0.0009) [2023-12-26 23:09:11,496][105692] Updated weights for policy 0, policy_version 1068093 (0.0009) [2023-12-26 23:09:11,555][105620] Updated weights for policy 1, policy_version 1069096 (0.0006) [2023-12-26 23:09:11,621][105620] Updated weights for policy 1, policy_version 1069106 (0.0006) [2023-12-26 23:09:11,690][105620] Updated weights for policy 1, policy_version 1069116 (0.0008) [2023-12-26 23:09:12,299][105692] Updated weights for policy 0, policy_version 1068103 (0.0009) [2023-12-26 23:09:12,365][105692] Updated weights for policy 0, policy_version 1068113 (0.0008) [2023-12-26 23:09:12,430][105692] Updated weights for policy 0, policy_version 1068123 (0.0008) [2023-12-26 23:09:12,457][105620] Updated weights for policy 1, policy_version 1069126 (0.0010) [2023-12-26 23:09:12,507][105620] Updated weights for policy 1, policy_version 1069136 (0.0011) [2023-12-26 23:09:12,571][105620] Updated weights for policy 1, policy_version 1069146 (0.0011) [2023-12-26 23:09:13,065][105692] Updated weights for policy 0, policy_version 1068133 (0.0007) [2023-12-26 23:09:13,122][105692] Updated weights for policy 0, policy_version 1068143 (0.0006) [2023-12-26 23:09:13,177][105692] Updated weights for policy 0, policy_version 1068153 (0.0005) [2023-12-26 23:09:13,308][105620] Updated weights for policy 1, policy_version 1069156 (0.0011) [2023-12-26 23:09:13,356][105620] Updated weights for policy 1, policy_version 1069166 (0.0010) [2023-12-26 23:09:13,408][105620] Updated weights for policy 1, policy_version 1069176 (0.0010) [2023-12-26 23:09:13,747][105692] Updated weights for policy 0, policy_version 1068163 (0.0007) [2023-12-26 23:09:13,806][105692] Updated weights for policy 0, policy_version 1068173 (0.0008) [2023-12-26 23:09:13,864][105692] Updated weights for policy 0, policy_version 1068183 (0.0006) [2023-12-26 23:09:14,178][105620] Updated weights for policy 1, policy_version 1069186 (0.0010) [2023-12-26 23:09:14,227][105620] Updated weights for policy 1, policy_version 1069196 (0.0010) [2023-12-26 23:09:14,282][105620] Updated weights for policy 1, policy_version 1069206 (0.0010) [2023-12-26 23:09:14,338][105620] Updated weights for policy 1, policy_version 1069216 (0.0010) [2023-12-26 23:09:14,583][105692] Updated weights for policy 0, policy_version 1068193 (0.0009) [2023-12-26 23:09:14,648][105692] Updated weights for policy 0, policy_version 1068203 (0.0005) [2023-12-26 23:09:14,698][105692] Updated weights for policy 0, policy_version 1068213 (0.0010) [2023-12-26 23:09:14,750][105692] Updated weights for policy 0, policy_version 1068223 (0.0010) [2023-12-26 23:09:15,099][105620] Updated weights for policy 1, policy_version 1069226 (0.0010) [2023-12-26 23:09:15,155][105620] Updated weights for policy 1, policy_version 1069236 (0.0010) [2023-12-26 23:09:15,216][105620] Updated weights for policy 1, policy_version 1069246 (0.0008) [2023-12-26 23:09:15,524][105692] Updated weights for policy 0, policy_version 1068233 (0.0008) [2023-12-26 23:09:15,584][105692] Updated weights for policy 0, policy_version 1068243 (0.0007) [2023-12-26 23:09:15,650][105692] Updated weights for policy 0, policy_version 1068253 (0.0007) [2023-12-26 23:09:15,875][105620] Updated weights for policy 1, policy_version 1069256 (0.0006) [2023-12-26 23:09:15,930][105620] Updated weights for policy 1, policy_version 1069266 (0.0005) [2023-12-26 23:09:15,979][105620] Updated weights for policy 1, policy_version 1069276 (0.0005) [2023-12-26 23:09:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 547282944. Throughput: 0: 9883.9, 1: 9701.4. Samples: 547250520. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:09:16,062][104569] Avg episode reward: [(0, '9261.543'), (1, '9260.958')] [2023-12-26 23:09:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001068256_273514496.pth... [2023-12-26 23:09:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001069280_273768448.pth... [2023-12-26 23:09:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001068128_273473536.pth [2023-12-26 23:09:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001067104_273219584.pth [2023-12-26 23:09:16,239][105692] Updated weights for policy 0, policy_version 1068263 (0.0006) [2023-12-26 23:09:16,294][105692] Updated weights for policy 0, policy_version 1068273 (0.0006) [2023-12-26 23:09:16,350][105692] Updated weights for policy 0, policy_version 1068283 (0.0009) [2023-12-26 23:09:16,608][105620] Updated weights for policy 1, policy_version 1069286 (0.0008) [2023-12-26 23:09:16,653][105620] Updated weights for policy 1, policy_version 1069296 (0.0010) [2023-12-26 23:09:16,704][105620] Updated weights for policy 1, policy_version 1069306 (0.0010) [2023-12-26 23:09:16,994][105692] Updated weights for policy 0, policy_version 1068293 (0.0010) [2023-12-26 23:09:17,042][105692] Updated weights for policy 0, policy_version 1068303 (0.0010) [2023-12-26 23:09:17,095][105692] Updated weights for policy 0, policy_version 1068313 (0.0010) [2023-12-26 23:09:17,345][105620] Updated weights for policy 1, policy_version 1069316 (0.0008) [2023-12-26 23:09:17,404][105620] Updated weights for policy 1, policy_version 1069326 (0.0005) [2023-12-26 23:09:17,460][105620] Updated weights for policy 1, policy_version 1069336 (0.0007) [2023-12-26 23:09:17,759][105692] Updated weights for policy 0, policy_version 1068323 (0.0010) [2023-12-26 23:09:17,810][105692] Updated weights for policy 0, policy_version 1068333 (0.0010) [2023-12-26 23:09:17,865][105692] Updated weights for policy 0, policy_version 1068343 (0.0010) [2023-12-26 23:09:18,090][105620] Updated weights for policy 1, policy_version 1069346 (0.0010) [2023-12-26 23:09:18,157][105620] Updated weights for policy 1, policy_version 1069356 (0.0011) [2023-12-26 23:09:18,223][105620] Updated weights for policy 1, policy_version 1069366 (0.0011) [2023-12-26 23:09:18,284][105620] Updated weights for policy 1, policy_version 1069376 (0.0010) [2023-12-26 23:09:18,591][105692] Updated weights for policy 0, policy_version 1068353 (0.0010) [2023-12-26 23:09:18,649][105692] Updated weights for policy 0, policy_version 1068363 (0.0010) [2023-12-26 23:09:18,702][105692] Updated weights for policy 0, policy_version 1068374 (0.0010) [2023-12-26 23:09:18,759][105692] Updated weights for policy 0, policy_version 1068384 (0.0010) [2023-12-26 23:09:18,903][105620] Updated weights for policy 1, policy_version 1069386 (0.0006) [2023-12-26 23:09:18,954][105620] Updated weights for policy 1, policy_version 1069396 (0.0006) [2023-12-26 23:09:19,005][105620] Updated weights for policy 1, policy_version 1069406 (0.0005) [2023-12-26 23:09:19,649][105692] Updated weights for policy 0, policy_version 1068394 (0.0010) [2023-12-26 23:09:19,705][105692] Updated weights for policy 0, policy_version 1068404 (0.0010) [2023-12-26 23:09:19,715][105620] Updated weights for policy 1, policy_version 1069416 (0.0007) [2023-12-26 23:09:19,771][105692] Updated weights for policy 0, policy_version 1068414 (0.0008) [2023-12-26 23:09:19,777][105620] Updated weights for policy 1, policy_version 1069426 (0.0006) [2023-12-26 23:09:19,838][105620] Updated weights for policy 1, policy_version 1069436 (0.0008) [2023-12-26 23:09:20,533][105692] Updated weights for policy 0, policy_version 1068424 (0.0006) [2023-12-26 23:09:20,599][105692] Updated weights for policy 0, policy_version 1068434 (0.0008) [2023-12-26 23:09:20,617][105620] Updated weights for policy 1, policy_version 1069446 (0.0009) [2023-12-26 23:09:20,656][105692] Updated weights for policy 0, policy_version 1068444 (0.0009) [2023-12-26 23:09:20,670][105620] Updated weights for policy 1, policy_version 1069456 (0.0009) [2023-12-26 23:09:20,734][105620] Updated weights for policy 1, policy_version 1069466 (0.0009) [2023-12-26 23:09:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 547381248. Throughput: 0: 9812.3, 1: 9698.9. Samples: 547371132. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:09:21,062][104569] Avg episode reward: [(0, '9174.364'), (1, '9260.677')] [2023-12-26 23:09:21,293][105692] Updated weights for policy 0, policy_version 1068454 (0.0007) [2023-12-26 23:09:21,357][105692] Updated weights for policy 0, policy_version 1068464 (0.0008) [2023-12-26 23:09:21,427][105692] Updated weights for policy 0, policy_version 1068474 (0.0010) [2023-12-26 23:09:21,501][105620] Updated weights for policy 1, policy_version 1069476 (0.0010) [2023-12-26 23:09:21,565][105620] Updated weights for policy 1, policy_version 1069486 (0.0008) [2023-12-26 23:09:21,635][105620] Updated weights for policy 1, policy_version 1069496 (0.0008) [2023-12-26 23:09:22,184][105692] Updated weights for policy 0, policy_version 1068484 (0.0011) [2023-12-26 23:09:22,255][105692] Updated weights for policy 0, policy_version 1068494 (0.0008) [2023-12-26 23:09:22,334][105692] Updated weights for policy 0, policy_version 1068504 (0.0012) [2023-12-26 23:09:22,350][105620] Updated weights for policy 1, policy_version 1069506 (0.0007) [2023-12-26 23:09:22,419][105620] Updated weights for policy 1, policy_version 1069516 (0.0008) [2023-12-26 23:09:22,480][105620] Updated weights for policy 1, policy_version 1069526 (0.0008) [2023-12-26 23:09:22,550][105620] Updated weights for policy 1, policy_version 1069536 (0.0008) [2023-12-26 23:09:22,988][105692] Updated weights for policy 0, policy_version 1068514 (0.0010) [2023-12-26 23:09:23,049][105692] Updated weights for policy 0, policy_version 1068524 (0.0006) [2023-12-26 23:09:23,115][105692] Updated weights for policy 0, policy_version 1068534 (0.0009) [2023-12-26 23:09:23,127][105620] Updated weights for policy 1, policy_version 1069546 (0.0005) [2023-12-26 23:09:23,176][105692] Updated weights for policy 0, policy_version 1068544 (0.0009) [2023-12-26 23:09:23,183][105620] Updated weights for policy 1, policy_version 1069556 (0.0006) [2023-12-26 23:09:23,245][105620] Updated weights for policy 1, policy_version 1069566 (0.0005) [2023-12-26 23:09:23,780][105692] Updated weights for policy 0, policy_version 1068554 (0.0009) [2023-12-26 23:09:23,838][105692] Updated weights for policy 0, policy_version 1068564 (0.0008) [2023-12-26 23:09:23,890][105692] Updated weights for policy 0, policy_version 1068574 (0.0007) [2023-12-26 23:09:24,001][105620] Updated weights for policy 1, policy_version 1069576 (0.0009) [2023-12-26 23:09:24,055][105620] Updated weights for policy 1, policy_version 1069586 (0.0009) [2023-12-26 23:09:24,114][105620] Updated weights for policy 1, policy_version 1069596 (0.0008) [2023-12-26 23:09:24,530][105692] Updated weights for policy 0, policy_version 1068584 (0.0009) [2023-12-26 23:09:24,592][105692] Updated weights for policy 0, policy_version 1068594 (0.0009) [2023-12-26 23:09:24,646][105692] Updated weights for policy 0, policy_version 1068604 (0.0009) [2023-12-26 23:09:24,943][105620] Updated weights for policy 1, policy_version 1069606 (0.0008) [2023-12-26 23:09:24,998][105620] Updated weights for policy 1, policy_version 1069616 (0.0008) [2023-12-26 23:09:25,058][105620] Updated weights for policy 1, policy_version 1069626 (0.0008) [2023-12-26 23:09:25,308][105692] Updated weights for policy 0, policy_version 1068614 (0.0010) [2023-12-26 23:09:25,367][105692] Updated weights for policy 0, policy_version 1068624 (0.0009) [2023-12-26 23:09:25,427][105692] Updated weights for policy 0, policy_version 1068634 (0.0009) [2023-12-26 23:09:25,832][105620] Updated weights for policy 1, policy_version 1069636 (0.0007) [2023-12-26 23:09:25,897][105620] Updated weights for policy 1, policy_version 1069646 (0.0005) [2023-12-26 23:09:25,952][105620] Updated weights for policy 1, policy_version 1069656 (0.0010) [2023-12-26 23:09:26,038][105692] Updated weights for policy 0, policy_version 1068644 (0.0009) [2023-12-26 23:09:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 547479552. Throughput: 0: 9823.2, 1: 9679.8. Samples: 547487224. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:09:26,062][104569] Avg episode reward: [(0, '9266.896'), (1, '9257.368')] [2023-12-26 23:09:26,101][105692] Updated weights for policy 0, policy_version 1068654 (0.0011) [2023-12-26 23:09:26,161][105692] Updated weights for policy 0, policy_version 1068664 (0.0010) [2023-12-26 23:09:26,620][105620] Updated weights for policy 1, policy_version 1069667 (0.0009) [2023-12-26 23:09:26,680][105620] Updated weights for policy 1, policy_version 1069677 (0.0008) [2023-12-26 23:09:26,732][105620] Updated weights for policy 1, policy_version 1069687 (0.0008) [2023-12-26 23:09:26,772][105692] Updated weights for policy 0, policy_version 1068674 (0.0006) [2023-12-26 23:09:26,818][105692] Updated weights for policy 0, policy_version 1068684 (0.0008) [2023-12-26 23:09:26,866][105692] Updated weights for policy 0, policy_version 1068694 (0.0005) [2023-12-26 23:09:26,912][105692] Updated weights for policy 0, policy_version 1068704 (0.0005) [2023-12-26 23:09:27,417][105620] Updated weights for policy 1, policy_version 1069697 (0.0006) [2023-12-26 23:09:27,478][105620] Updated weights for policy 1, policy_version 1069707 (0.0007) [2023-12-26 23:09:27,488][105692] Updated weights for policy 0, policy_version 1068714 (0.0008) [2023-12-26 23:09:27,531][105620] Updated weights for policy 1, policy_version 1069717 (0.0006) [2023-12-26 23:09:27,549][105692] Updated weights for policy 0, policy_version 1068724 (0.0008) [2023-12-26 23:09:27,586][105620] Updated weights for policy 1, policy_version 1069727 (0.0005) [2023-12-26 23:09:27,607][105692] Updated weights for policy 0, policy_version 1068734 (0.0008) [2023-12-26 23:09:28,245][105620] Updated weights for policy 1, policy_version 1069737 (0.0010) [2023-12-26 23:09:28,295][105620] Updated weights for policy 1, policy_version 1069747 (0.0009) [2023-12-26 23:09:28,358][105620] Updated weights for policy 1, policy_version 1069757 (0.0007) [2023-12-26 23:09:28,378][105692] Updated weights for policy 0, policy_version 1068744 (0.0008) [2023-12-26 23:09:28,436][105692] Updated weights for policy 0, policy_version 1068754 (0.0010) [2023-12-26 23:09:28,483][105692] Updated weights for policy 0, policy_version 1068764 (0.0009) [2023-12-26 23:09:28,930][105620] Updated weights for policy 1, policy_version 1069767 (0.0009) [2023-12-26 23:09:28,989][105620] Updated weights for policy 1, policy_version 1069777 (0.0010) [2023-12-26 23:09:29,043][105620] Updated weights for policy 1, policy_version 1069787 (0.0010) [2023-12-26 23:09:29,332][105692] Updated weights for policy 0, policy_version 1068774 (0.0007) [2023-12-26 23:09:29,395][105692] Updated weights for policy 0, policy_version 1068784 (0.0008) [2023-12-26 23:09:29,444][105692] Updated weights for policy 0, policy_version 1068794 (0.0008) [2023-12-26 23:09:29,759][105620] Updated weights for policy 1, policy_version 1069797 (0.0008) [2023-12-26 23:09:29,826][105620] Updated weights for policy 1, policy_version 1069807 (0.0009) [2023-12-26 23:09:29,883][105620] Updated weights for policy 1, policy_version 1069817 (0.0010) [2023-12-26 23:09:30,142][105692] Updated weights for policy 0, policy_version 1068804 (0.0008) [2023-12-26 23:09:30,193][105692] Updated weights for policy 0, policy_version 1068814 (0.0007) [2023-12-26 23:09:30,245][105692] Updated weights for policy 0, policy_version 1068824 (0.0009) [2023-12-26 23:09:30,494][105620] Updated weights for policy 1, policy_version 1069827 (0.0009) [2023-12-26 23:09:30,546][105620] Updated weights for policy 1, policy_version 1069837 (0.0005) [2023-12-26 23:09:30,600][105620] Updated weights for policy 1, policy_version 1069847 (0.0005) [2023-12-26 23:09:31,025][105692] Updated weights for policy 0, policy_version 1068834 (0.0010) [2023-12-26 23:09:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 547577856. Throughput: 0: 9926.4, 1: 9764.2. Samples: 547550836. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:09:31,062][104569] Avg episode reward: [(0, '9266.843'), (1, '9166.079')] [2023-12-26 23:09:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001069856_273915904.pth... [2023-12-26 23:09:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001068704_273620992.pth [2023-12-26 23:09:31,090][105692] Updated weights for policy 0, policy_version 1068844 (0.0010) [2023-12-26 23:09:31,155][105692] Updated weights for policy 0, policy_version 1068854 (0.0008) [2023-12-26 23:09:31,210][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001068864_273670144.pth... [2023-12-26 23:09:31,211][105692] Updated weights for policy 0, policy_version 1068864 (0.0006) [2023-12-26 23:09:31,214][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001067680_273367040.pth [2023-12-26 23:09:31,307][105620] Updated weights for policy 1, policy_version 1069857 (0.0008) [2023-12-26 23:09:31,369][105620] Updated weights for policy 1, policy_version 1069867 (0.0009) [2023-12-26 23:09:31,429][105620] Updated weights for policy 1, policy_version 1069877 (0.0009) [2023-12-26 23:09:31,482][105620] Updated weights for policy 1, policy_version 1069887 (0.0010) [2023-12-26 23:09:31,855][105692] Updated weights for policy 0, policy_version 1068874 (0.0008) [2023-12-26 23:09:31,916][105692] Updated weights for policy 0, policy_version 1068884 (0.0009) [2023-12-26 23:09:31,973][105692] Updated weights for policy 0, policy_version 1068894 (0.0007) [2023-12-26 23:09:32,244][105620] Updated weights for policy 1, policy_version 1069897 (0.0006) [2023-12-26 23:09:32,301][105620] Updated weights for policy 1, policy_version 1069907 (0.0008) [2023-12-26 23:09:32,356][105620] Updated weights for policy 1, policy_version 1069917 (0.0009) [2023-12-26 23:09:32,753][105692] Updated weights for policy 0, policy_version 1068904 (0.0010) [2023-12-26 23:09:32,814][105692] Updated weights for policy 0, policy_version 1068914 (0.0009) [2023-12-26 23:09:32,876][105692] Updated weights for policy 0, policy_version 1068924 (0.0009) [2023-12-26 23:09:33,014][105620] Updated weights for policy 1, policy_version 1069927 (0.0008) [2023-12-26 23:09:33,060][105620] Updated weights for policy 1, policy_version 1069937 (0.0009) [2023-12-26 23:09:33,107][105620] Updated weights for policy 1, policy_version 1069947 (0.0009) [2023-12-26 23:09:33,538][105692] Updated weights for policy 0, policy_version 1068934 (0.0009) [2023-12-26 23:09:33,584][105692] Updated weights for policy 0, policy_version 1068944 (0.0009) [2023-12-26 23:09:33,631][105692] Updated weights for policy 0, policy_version 1068955 (0.0008) [2023-12-26 23:09:33,936][105620] Updated weights for policy 1, policy_version 1069957 (0.0008) [2023-12-26 23:09:33,983][105620] Updated weights for policy 1, policy_version 1069967 (0.0008) [2023-12-26 23:09:34,029][105620] Updated weights for policy 1, policy_version 1069977 (0.0008) [2023-12-26 23:09:34,363][105692] Updated weights for policy 0, policy_version 1068965 (0.0007) [2023-12-26 23:09:34,425][105692] Updated weights for policy 0, policy_version 1068975 (0.0009) [2023-12-26 23:09:34,487][105692] Updated weights for policy 0, policy_version 1068985 (0.0010) [2023-12-26 23:09:34,701][105620] Updated weights for policy 1, policy_version 1069987 (0.0008) [2023-12-26 23:09:34,763][105620] Updated weights for policy 1, policy_version 1069997 (0.0008) [2023-12-26 23:09:34,831][105620] Updated weights for policy 1, policy_version 1070007 (0.0006) [2023-12-26 23:09:35,355][105620] Updated weights for policy 1, policy_version 1070017 (0.0006) [2023-12-26 23:09:35,374][105692] Updated weights for policy 0, policy_version 1068995 (0.0008) [2023-12-26 23:09:35,404][105620] Updated weights for policy 1, policy_version 1070027 (0.0007) [2023-12-26 23:09:35,419][105692] Updated weights for policy 0, policy_version 1069005 (0.0006) [2023-12-26 23:09:35,451][105620] Updated weights for policy 1, policy_version 1070037 (0.0007) [2023-12-26 23:09:35,474][105692] Updated weights for policy 0, policy_version 1069015 (0.0006) [2023-12-26 23:09:35,511][105620] Updated weights for policy 1, policy_version 1070047 (0.0008) [2023-12-26 23:09:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 547676160. Throughput: 0: 9835.7, 1: 9798.2. Samples: 547667692. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:09:36,063][104569] Avg episode reward: [(0, '9270.197'), (1, '9258.016')] [2023-12-26 23:09:36,239][105692] Updated weights for policy 0, policy_version 1069025 (0.0006) [2023-12-26 23:09:36,297][105692] Updated weights for policy 0, policy_version 1069035 (0.0007) [2023-12-26 23:09:36,300][105620] Updated weights for policy 1, policy_version 1070057 (0.0007) [2023-12-26 23:09:36,348][105692] Updated weights for policy 0, policy_version 1069045 (0.0007) [2023-12-26 23:09:36,360][105620] Updated weights for policy 1, policy_version 1070067 (0.0007) [2023-12-26 23:09:36,399][105692] Updated weights for policy 0, policy_version 1069055 (0.0007) [2023-12-26 23:09:36,416][105620] Updated weights for policy 1, policy_version 1070077 (0.0008) [2023-12-26 23:09:37,153][105692] Updated weights for policy 0, policy_version 1069065 (0.0006) [2023-12-26 23:09:37,188][105620] Updated weights for policy 1, policy_version 1070087 (0.0007) [2023-12-26 23:09:37,215][105692] Updated weights for policy 0, policy_version 1069075 (0.0005) [2023-12-26 23:09:37,237][105620] Updated weights for policy 1, policy_version 1070097 (0.0005) [2023-12-26 23:09:37,274][105692] Updated weights for policy 0, policy_version 1069085 (0.0005) [2023-12-26 23:09:37,284][105620] Updated weights for policy 1, policy_version 1070107 (0.0005) [2023-12-26 23:09:37,918][105692] Updated weights for policy 0, policy_version 1069095 (0.0007) [2023-12-26 23:09:37,973][105620] Updated weights for policy 1, policy_version 1070117 (0.0008) [2023-12-26 23:09:37,975][105692] Updated weights for policy 0, policy_version 1069105 (0.0006) [2023-12-26 23:09:38,026][105620] Updated weights for policy 1, policy_version 1070127 (0.0010) [2023-12-26 23:09:38,032][105692] Updated weights for policy 0, policy_version 1069115 (0.0006) [2023-12-26 23:09:38,085][105620] Updated weights for policy 1, policy_version 1070137 (0.0011) [2023-12-26 23:09:38,805][105692] Updated weights for policy 0, policy_version 1069125 (0.0007) [2023-12-26 23:09:38,855][105692] Updated weights for policy 0, policy_version 1069135 (0.0009) [2023-12-26 23:09:38,865][105620] Updated weights for policy 1, policy_version 1070147 (0.0010) [2023-12-26 23:09:38,909][105692] Updated weights for policy 0, policy_version 1069145 (0.0007) [2023-12-26 23:09:38,911][105620] Updated weights for policy 1, policy_version 1070157 (0.0006) [2023-12-26 23:09:38,965][105620] Updated weights for policy 1, policy_version 1070167 (0.0007) [2023-12-26 23:09:39,691][105692] Updated weights for policy 0, policy_version 1069155 (0.0008) [2023-12-26 23:09:39,738][105620] Updated weights for policy 1, policy_version 1070177 (0.0009) [2023-12-26 23:09:39,748][105692] Updated weights for policy 0, policy_version 1069165 (0.0008) [2023-12-26 23:09:39,802][105692] Updated weights for policy 0, policy_version 1069175 (0.0006) [2023-12-26 23:09:39,804][105620] Updated weights for policy 1, policy_version 1070187 (0.0008) [2023-12-26 23:09:39,873][105620] Updated weights for policy 1, policy_version 1070197 (0.0009) [2023-12-26 23:09:39,941][105620] Updated weights for policy 1, policy_version 1070207 (0.0008) [2023-12-26 23:09:40,565][105692] Updated weights for policy 0, policy_version 1069185 (0.0008) [2023-12-26 23:09:40,626][105692] Updated weights for policy 0, policy_version 1069195 (0.0008) [2023-12-26 23:09:40,654][105620] Updated weights for policy 1, policy_version 1070217 (0.0009) [2023-12-26 23:09:40,685][105692] Updated weights for policy 0, policy_version 1069205 (0.0008) [2023-12-26 23:09:40,714][105620] Updated weights for policy 1, policy_version 1070227 (0.0009) [2023-12-26 23:09:40,745][105692] Updated weights for policy 0, policy_version 1069215 (0.0006) [2023-12-26 23:09:40,763][105620] Updated weights for policy 1, policy_version 1070237 (0.0010) [2023-12-26 23:09:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 547774464. Throughput: 0: 9811.0, 1: 9850.6. Samples: 547781156. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:09:41,063][104569] Avg episode reward: [(0, '9350.716'), (1, '8690.990')] [2023-12-26 23:09:41,447][105620] Updated weights for policy 1, policy_version 1070247 (0.0010) [2023-12-26 23:09:41,507][105620] Updated weights for policy 1, policy_version 1070257 (0.0009) [2023-12-26 23:09:41,564][105620] Updated weights for policy 1, policy_version 1070267 (0.0008) [2023-12-26 23:09:41,567][105692] Updated weights for policy 0, policy_version 1069225 (0.0006) [2023-12-26 23:09:41,626][105692] Updated weights for policy 0, policy_version 1069235 (0.0007) [2023-12-26 23:09:41,692][105692] Updated weights for policy 0, policy_version 1069245 (0.0009) [2023-12-26 23:09:42,325][105620] Updated weights for policy 1, policy_version 1070277 (0.0008) [2023-12-26 23:09:42,392][105620] Updated weights for policy 1, policy_version 1070287 (0.0010) [2023-12-26 23:09:42,450][105620] Updated weights for policy 1, policy_version 1070297 (0.0009) [2023-12-26 23:09:42,482][105692] Updated weights for policy 0, policy_version 1069255 (0.0008) [2023-12-26 23:09:42,541][105692] Updated weights for policy 0, policy_version 1069265 (0.0008) [2023-12-26 23:09:42,591][105692] Updated weights for policy 0, policy_version 1069275 (0.0009) [2023-12-26 23:09:43,121][105620] Updated weights for policy 1, policy_version 1070307 (0.0008) [2023-12-26 23:09:43,168][105620] Updated weights for policy 1, policy_version 1070317 (0.0009) [2023-12-26 23:09:43,215][105620] Updated weights for policy 1, policy_version 1070327 (0.0008) [2023-12-26 23:09:43,393][105692] Updated weights for policy 0, policy_version 1069285 (0.0009) [2023-12-26 23:09:43,460][105692] Updated weights for policy 0, policy_version 1069295 (0.0009) [2023-12-26 23:09:43,519][105692] Updated weights for policy 0, policy_version 1069305 (0.0009) [2023-12-26 23:09:43,941][105620] Updated weights for policy 1, policy_version 1070337 (0.0008) [2023-12-26 23:09:43,996][105620] Updated weights for policy 1, policy_version 1070347 (0.0009) [2023-12-26 23:09:44,044][105620] Updated weights for policy 1, policy_version 1070357 (0.0009) [2023-12-26 23:09:44,097][105620] Updated weights for policy 1, policy_version 1070367 (0.0009) [2023-12-26 23:09:44,241][105692] Updated weights for policy 0, policy_version 1069315 (0.0009) [2023-12-26 23:09:44,303][105692] Updated weights for policy 0, policy_version 1069325 (0.0009) [2023-12-26 23:09:44,362][105692] Updated weights for policy 0, policy_version 1069335 (0.0010) [2023-12-26 23:09:44,827][105620] Updated weights for policy 1, policy_version 1070377 (0.0009) [2023-12-26 23:09:44,893][105620] Updated weights for policy 1, policy_version 1070387 (0.0009) [2023-12-26 23:09:44,953][105620] Updated weights for policy 1, policy_version 1070397 (0.0009) [2023-12-26 23:09:45,145][105692] Updated weights for policy 0, policy_version 1069345 (0.0009) [2023-12-26 23:09:45,207][105692] Updated weights for policy 0, policy_version 1069355 (0.0010) [2023-12-26 23:09:45,261][105692] Updated weights for policy 0, policy_version 1069365 (0.0010) [2023-12-26 23:09:45,318][105692] Updated weights for policy 0, policy_version 1069375 (0.0009) [2023-12-26 23:09:45,652][105620] Updated weights for policy 1, policy_version 1070407 (0.0009) [2023-12-26 23:09:45,712][105620] Updated weights for policy 1, policy_version 1070417 (0.0009) [2023-12-26 23:09:45,777][105620] Updated weights for policy 1, policy_version 1070427 (0.0008) [2023-12-26 23:09:46,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 547864576. Throughput: 0: 9712.3, 1: 9863.7. Samples: 547836548. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:09:46,062][104569] Avg episode reward: [(0, '9350.188'), (1, '8599.883')] [2023-12-26 23:09:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001070432_274063360.pth... [2023-12-26 23:09:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001069280_273768448.pth [2023-12-26 23:09:46,119][105692] Updated weights for policy 0, policy_version 1069385 (0.0008) [2023-12-26 23:09:46,173][105692] Updated weights for policy 0, policy_version 1069395 (0.0008) [2023-12-26 23:09:46,221][105692] Updated weights for policy 0, policy_version 1069405 (0.0008) [2023-12-26 23:09:46,236][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001069408_273809408.pth... [2023-12-26 23:09:46,239][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001068256_273514496.pth [2023-12-26 23:09:46,506][105620] Updated weights for policy 1, policy_version 1070437 (0.0009) [2023-12-26 23:09:46,555][105620] Updated weights for policy 1, policy_version 1070447 (0.0011) [2023-12-26 23:09:46,600][105620] Updated weights for policy 1, policy_version 1070457 (0.0010) [2023-12-26 23:09:47,012][105692] Updated weights for policy 0, policy_version 1069415 (0.0008) [2023-12-26 23:09:47,073][105692] Updated weights for policy 0, policy_version 1069425 (0.0008) [2023-12-26 23:09:47,137][105692] Updated weights for policy 0, policy_version 1069435 (0.0008) [2023-12-26 23:09:47,303][105620] Updated weights for policy 1, policy_version 1070467 (0.0010) [2023-12-26 23:09:47,353][105620] Updated weights for policy 1, policy_version 1070477 (0.0008) [2023-12-26 23:09:47,400][105620] Updated weights for policy 1, policy_version 1070487 (0.0009) [2023-12-26 23:09:47,854][105692] Updated weights for policy 0, policy_version 1069445 (0.0007) [2023-12-26 23:09:47,908][105692] Updated weights for policy 0, policy_version 1069455 (0.0006) [2023-12-26 23:09:47,964][105692] Updated weights for policy 0, policy_version 1069465 (0.0010) [2023-12-26 23:09:48,158][105620] Updated weights for policy 1, policy_version 1070497 (0.0009) [2023-12-26 23:09:48,212][105620] Updated weights for policy 1, policy_version 1070507 (0.0006) [2023-12-26 23:09:48,267][105620] Updated weights for policy 1, policy_version 1070517 (0.0010) [2023-12-26 23:09:48,324][105620] Updated weights for policy 1, policy_version 1070527 (0.0007) [2023-12-26 23:09:48,716][105692] Updated weights for policy 0, policy_version 1069475 (0.0008) [2023-12-26 23:09:48,767][105692] Updated weights for policy 0, policy_version 1069485 (0.0008) [2023-12-26 23:09:48,825][105692] Updated weights for policy 0, policy_version 1069495 (0.0009) [2023-12-26 23:09:49,046][105620] Updated weights for policy 1, policy_version 1070537 (0.0006) [2023-12-26 23:09:49,094][105620] Updated weights for policy 1, policy_version 1070547 (0.0005) [2023-12-26 23:09:49,144][105620] Updated weights for policy 1, policy_version 1070557 (0.0006) [2023-12-26 23:09:49,636][105692] Updated weights for policy 0, policy_version 1069505 (0.0009) [2023-12-26 23:09:49,694][105692] Updated weights for policy 0, policy_version 1069515 (0.0010) [2023-12-26 23:09:49,753][105692] Updated weights for policy 0, policy_version 1069525 (0.0008) [2023-12-26 23:09:49,807][105692] Updated weights for policy 0, policy_version 1069535 (0.0009) [2023-12-26 23:09:49,857][105620] Updated weights for policy 1, policy_version 1070567 (0.0010) [2023-12-26 23:09:49,914][105620] Updated weights for policy 1, policy_version 1070577 (0.0009) [2023-12-26 23:09:49,981][105620] Updated weights for policy 1, policy_version 1070587 (0.0008) [2023-12-26 23:09:50,527][105692] Updated weights for policy 0, policy_version 1069545 (0.0008) [2023-12-26 23:09:50,586][105692] Updated weights for policy 0, policy_version 1069555 (0.0009) [2023-12-26 23:09:50,643][105692] Updated weights for policy 0, policy_version 1069565 (0.0007) [2023-12-26 23:09:50,736][105620] Updated weights for policy 1, policy_version 1070597 (0.0009) [2023-12-26 23:09:50,789][105620] Updated weights for policy 1, policy_version 1070607 (0.0009) [2023-12-26 23:09:50,839][105620] Updated weights for policy 1, policy_version 1070617 (0.0009) [2023-12-26 23:09:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 547962880. Throughput: 0: 9678.1, 1: 9827.5. Samples: 547950208. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-26 23:09:51,062][104569] Avg episode reward: [(0, '9262.436'), (1, '8834.412')] [2023-12-26 23:09:51,380][105692] Updated weights for policy 0, policy_version 1069575 (0.0009) [2023-12-26 23:09:51,442][105692] Updated weights for policy 0, policy_version 1069585 (0.0009) [2023-12-26 23:09:51,512][105692] Updated weights for policy 0, policy_version 1069595 (0.0010) [2023-12-26 23:09:51,589][105620] Updated weights for policy 1, policy_version 1070627 (0.0009) [2023-12-26 23:09:51,660][105620] Updated weights for policy 1, policy_version 1070637 (0.0008) [2023-12-26 23:09:51,716][105620] Updated weights for policy 1, policy_version 1070647 (0.0009) [2023-12-26 23:09:52,341][105692] Updated weights for policy 0, policy_version 1069605 (0.0008) [2023-12-26 23:09:52,410][105692] Updated weights for policy 0, policy_version 1069615 (0.0006) [2023-12-26 23:09:52,439][105620] Updated weights for policy 1, policy_version 1070657 (0.0010) [2023-12-26 23:09:52,475][105692] Updated weights for policy 0, policy_version 1069625 (0.0007) [2023-12-26 23:09:52,502][105620] Updated weights for policy 1, policy_version 1070667 (0.0007) [2023-12-26 23:09:52,560][105620] Updated weights for policy 1, policy_version 1070677 (0.0007) [2023-12-26 23:09:52,626][105620] Updated weights for policy 1, policy_version 1070687 (0.0009) [2023-12-26 23:09:53,143][105692] Updated weights for policy 0, policy_version 1069635 (0.0008) [2023-12-26 23:09:53,192][105692] Updated weights for policy 0, policy_version 1069645 (0.0006) [2023-12-26 23:09:53,237][105692] Updated weights for policy 0, policy_version 1069655 (0.0005) [2023-12-26 23:09:53,407][105620] Updated weights for policy 1, policy_version 1070697 (0.0009) [2023-12-26 23:09:53,454][105620] Updated weights for policy 1, policy_version 1070707 (0.0009) [2023-12-26 23:09:53,505][105620] Updated weights for policy 1, policy_version 1070717 (0.0009) [2023-12-26 23:09:53,936][105692] Updated weights for policy 0, policy_version 1069665 (0.0006) [2023-12-26 23:09:53,997][105692] Updated weights for policy 0, policy_version 1069675 (0.0009) [2023-12-26 23:09:54,054][105692] Updated weights for policy 0, policy_version 1069685 (0.0009) [2023-12-26 23:09:54,110][105692] Updated weights for policy 0, policy_version 1069695 (0.0010) [2023-12-26 23:09:54,215][105620] Updated weights for policy 1, policy_version 1070727 (0.0006) [2023-12-26 23:09:54,272][105620] Updated weights for policy 1, policy_version 1070737 (0.0005) [2023-12-26 23:09:54,325][105620] Updated weights for policy 1, policy_version 1070747 (0.0005) [2023-12-26 23:09:54,918][105692] Updated weights for policy 0, policy_version 1069705 (0.0009) [2023-12-26 23:09:54,978][105692] Updated weights for policy 0, policy_version 1069715 (0.0008) [2023-12-26 23:09:54,997][105620] Updated weights for policy 1, policy_version 1070757 (0.0008) [2023-12-26 23:09:55,031][105692] Updated weights for policy 0, policy_version 1069725 (0.0007) [2023-12-26 23:09:55,052][105620] Updated weights for policy 1, policy_version 1070767 (0.0006) [2023-12-26 23:09:55,104][105620] Updated weights for policy 1, policy_version 1070777 (0.0009) [2023-12-26 23:09:55,793][105692] Updated weights for policy 0, policy_version 1069735 (0.0009) [2023-12-26 23:09:55,857][105692] Updated weights for policy 0, policy_version 1069745 (0.0009) [2023-12-26 23:09:55,869][105620] Updated weights for policy 1, policy_version 1070787 (0.0010) [2023-12-26 23:09:55,908][105692] Updated weights for policy 0, policy_version 1069755 (0.0007) [2023-12-26 23:09:55,920][105620] Updated weights for policy 1, policy_version 1070797 (0.0008) [2023-12-26 23:09:55,979][105620] Updated weights for policy 1, policy_version 1070807 (0.0009) [2023-12-26 23:09:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 548061184. Throughput: 0: 9592.3, 1: 9783.6. Samples: 548063512. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:09:56,063][104569] Avg episode reward: [(0, '9264.604'), (1, '8893.747')] [2023-12-26 23:09:56,682][105692] Updated weights for policy 0, policy_version 1069765 (0.0010) [2023-12-26 23:09:56,704][105620] Updated weights for policy 1, policy_version 1070817 (0.0009) [2023-12-26 23:09:56,741][105692] Updated weights for policy 0, policy_version 1069775 (0.0009) [2023-12-26 23:09:56,753][105620] Updated weights for policy 1, policy_version 1070827 (0.0008) [2023-12-26 23:09:56,803][105692] Updated weights for policy 0, policy_version 1069785 (0.0008) [2023-12-26 23:09:56,813][105620] Updated weights for policy 1, policy_version 1070837 (0.0006) [2023-12-26 23:09:56,870][105620] Updated weights for policy 1, policy_version 1070847 (0.0007) [2023-12-26 23:09:57,534][105692] Updated weights for policy 0, policy_version 1069795 (0.0007) [2023-12-26 23:09:57,589][105692] Updated weights for policy 0, policy_version 1069805 (0.0008) [2023-12-26 23:09:57,626][105620] Updated weights for policy 1, policy_version 1070857 (0.0008) [2023-12-26 23:09:57,646][105692] Updated weights for policy 0, policy_version 1069815 (0.0005) [2023-12-26 23:09:57,677][105620] Updated weights for policy 1, policy_version 1070867 (0.0010) [2023-12-26 23:09:57,725][105620] Updated weights for policy 1, policy_version 1070877 (0.0010) [2023-12-26 23:09:58,330][105620] Updated weights for policy 1, policy_version 1070887 (0.0010) [2023-12-26 23:09:58,398][105692] Updated weights for policy 0, policy_version 1069825 (0.0006) [2023-12-26 23:09:58,400][105620] Updated weights for policy 1, policy_version 1070897 (0.0010) [2023-12-26 23:09:58,462][105692] Updated weights for policy 0, policy_version 1069835 (0.0007) [2023-12-26 23:09:58,467][105620] Updated weights for policy 1, policy_version 1070907 (0.0011) [2023-12-26 23:09:58,528][105692] Updated weights for policy 0, policy_version 1069845 (0.0008) [2023-12-26 23:09:58,591][105692] Updated weights for policy 0, policy_version 1069855 (0.0008) [2023-12-26 23:09:59,265][105620] Updated weights for policy 1, policy_version 1070917 (0.0009) [2023-12-26 23:09:59,330][105620] Updated weights for policy 1, policy_version 1070927 (0.0008) [2023-12-26 23:09:59,399][105620] Updated weights for policy 1, policy_version 1070937 (0.0008) [2023-12-26 23:09:59,492][105692] Updated weights for policy 0, policy_version 1069865 (0.0010) [2023-12-26 23:09:59,554][105692] Updated weights for policy 0, policy_version 1069875 (0.0010) [2023-12-26 23:09:59,619][105692] Updated weights for policy 0, policy_version 1069885 (0.0010) [2023-12-26 23:10:00,086][105620] Updated weights for policy 1, policy_version 1070947 (0.0007) [2023-12-26 23:10:00,137][105620] Updated weights for policy 1, policy_version 1070957 (0.0010) [2023-12-26 23:10:00,189][105620] Updated weights for policy 1, policy_version 1070967 (0.0010) [2023-12-26 23:10:00,273][105692] Updated weights for policy 0, policy_version 1069895 (0.0009) [2023-12-26 23:10:00,332][105692] Updated weights for policy 0, policy_version 1069905 (0.0005) [2023-12-26 23:10:00,390][105692] Updated weights for policy 0, policy_version 1069915 (0.0005) [2023-12-26 23:10:00,870][105620] Updated weights for policy 1, policy_version 1070977 (0.0010) [2023-12-26 23:10:00,921][105620] Updated weights for policy 1, policy_version 1070987 (0.0006) [2023-12-26 23:10:00,976][105620] Updated weights for policy 1, policy_version 1070997 (0.0006) [2023-12-26 23:10:01,026][105692] Updated weights for policy 0, policy_version 1069925 (0.0006) [2023-12-26 23:10:01,040][105620] Updated weights for policy 1, policy_version 1071007 (0.0007) [2023-12-26 23:10:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 548151296. Throughput: 0: 9520.2, 1: 9801.7. Samples: 548120004. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:10:01,062][104569] Avg episode reward: [(0, '9356.273'), (1, '9258.706')] [2023-12-26 23:10:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001071008_274210816.pth... [2023-12-26 23:10:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001069856_273915904.pth [2023-12-26 23:10:01,090][105692] Updated weights for policy 0, policy_version 1069935 (0.0007) [2023-12-26 23:10:01,161][105692] Updated weights for policy 0, policy_version 1069945 (0.0007) [2023-12-26 23:10:01,201][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001069952_273948672.pth... [2023-12-26 23:10:01,205][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001068864_273670144.pth [2023-12-26 23:10:01,750][105620] Updated weights for policy 1, policy_version 1071017 (0.0011) [2023-12-26 23:10:01,799][105620] Updated weights for policy 1, policy_version 1071027 (0.0010) [2023-12-26 23:10:01,841][105692] Updated weights for policy 0, policy_version 1069955 (0.0006) [2023-12-26 23:10:01,847][105620] Updated weights for policy 1, policy_version 1071037 (0.0010) [2023-12-26 23:10:01,891][105692] Updated weights for policy 0, policy_version 1069965 (0.0007) [2023-12-26 23:10:01,939][105692] Updated weights for policy 0, policy_version 1069975 (0.0008) [2023-12-26 23:10:02,618][105620] Updated weights for policy 1, policy_version 1071047 (0.0010) [2023-12-26 23:10:02,672][105620] Updated weights for policy 1, policy_version 1071057 (0.0010) [2023-12-26 23:10:02,694][105692] Updated weights for policy 0, policy_version 1069985 (0.0008) [2023-12-26 23:10:02,730][105620] Updated weights for policy 1, policy_version 1071067 (0.0009) [2023-12-26 23:10:02,754][105692] Updated weights for policy 0, policy_version 1069995 (0.0005) [2023-12-26 23:10:02,813][105692] Updated weights for policy 0, policy_version 1070005 (0.0005) [2023-12-26 23:10:02,874][105692] Updated weights for policy 0, policy_version 1070015 (0.0005) [2023-12-26 23:10:03,327][105620] Updated weights for policy 1, policy_version 1071077 (0.0008) [2023-12-26 23:10:03,382][105620] Updated weights for policy 1, policy_version 1071087 (0.0005) [2023-12-26 23:10:03,431][105620] Updated weights for policy 1, policy_version 1071097 (0.0005) [2023-12-26 23:10:03,459][105692] Updated weights for policy 0, policy_version 1070025 (0.0007) [2023-12-26 23:10:03,523][105692] Updated weights for policy 0, policy_version 1070035 (0.0005) [2023-12-26 23:10:03,580][105692] Updated weights for policy 0, policy_version 1070045 (0.0006) [2023-12-26 23:10:04,091][105620] Updated weights for policy 1, policy_version 1071107 (0.0006) [2023-12-26 23:10:04,148][105620] Updated weights for policy 1, policy_version 1071117 (0.0008) [2023-12-26 23:10:04,209][105620] Updated weights for policy 1, policy_version 1071127 (0.0006) [2023-12-26 23:10:04,394][105692] Updated weights for policy 0, policy_version 1070055 (0.0008) [2023-12-26 23:10:04,456][105692] Updated weights for policy 0, policy_version 1070065 (0.0009) [2023-12-26 23:10:04,510][105692] Updated weights for policy 0, policy_version 1070075 (0.0010) [2023-12-26 23:10:04,800][105620] Updated weights for policy 1, policy_version 1071137 (0.0007) [2023-12-26 23:10:04,854][105620] Updated weights for policy 1, policy_version 1071147 (0.0010) [2023-12-26 23:10:04,919][105620] Updated weights for policy 1, policy_version 1071157 (0.0010) [2023-12-26 23:10:04,978][105620] Updated weights for policy 1, policy_version 1071167 (0.0010) [2023-12-26 23:10:05,378][105692] Updated weights for policy 0, policy_version 1070085 (0.0009) [2023-12-26 23:10:05,436][105692] Updated weights for policy 0, policy_version 1070095 (0.0006) [2023-12-26 23:10:05,503][105692] Updated weights for policy 0, policy_version 1070105 (0.0006) [2023-12-26 23:10:05,555][105620] Updated weights for policy 1, policy_version 1071177 (0.0006) [2023-12-26 23:10:05,608][105620] Updated weights for policy 1, policy_version 1071187 (0.0005) [2023-12-26 23:10:05,672][105620] Updated weights for policy 1, policy_version 1071197 (0.0005) [2023-12-26 23:10:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 548249600. Throughput: 0: 9496.8, 1: 9803.6. Samples: 548239648. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:10:06,062][104569] Avg episode reward: [(0, '9358.038'), (1, '9255.903')] [2023-12-26 23:10:06,227][105620] Updated weights for policy 1, policy_version 1071207 (0.0008) [2023-12-26 23:10:06,292][105620] Updated weights for policy 1, policy_version 1071217 (0.0007) [2023-12-26 23:10:06,335][105692] Updated weights for policy 0, policy_version 1070115 (0.0008) [2023-12-26 23:10:06,356][105620] Updated weights for policy 1, policy_version 1071227 (0.0008) [2023-12-26 23:10:06,398][105692] Updated weights for policy 0, policy_version 1070125 (0.0009) [2023-12-26 23:10:06,460][105692] Updated weights for policy 0, policy_version 1070135 (0.0009) [2023-12-26 23:10:06,990][105620] Updated weights for policy 1, policy_version 1071237 (0.0009) [2023-12-26 23:10:07,052][105620] Updated weights for policy 1, policy_version 1071247 (0.0010) [2023-12-26 23:10:07,118][105620] Updated weights for policy 1, policy_version 1071257 (0.0011) [2023-12-26 23:10:07,150][105692] Updated weights for policy 0, policy_version 1070145 (0.0009) [2023-12-26 23:10:07,197][105692] Updated weights for policy 0, policy_version 1070155 (0.0006) [2023-12-26 23:10:07,246][105692] Updated weights for policy 0, policy_version 1070165 (0.0007) [2023-12-26 23:10:07,295][105692] Updated weights for policy 0, policy_version 1070175 (0.0008) [2023-12-26 23:10:07,730][105620] Updated weights for policy 1, policy_version 1071267 (0.0006) [2023-12-26 23:10:07,790][105620] Updated weights for policy 1, policy_version 1071277 (0.0006) [2023-12-26 23:10:07,849][105620] Updated weights for policy 1, policy_version 1071287 (0.0005) [2023-12-26 23:10:08,032][105692] Updated weights for policy 0, policy_version 1070185 (0.0009) [2023-12-26 23:10:08,083][105692] Updated weights for policy 0, policy_version 1070195 (0.0009) [2023-12-26 23:10:08,130][105692] Updated weights for policy 0, policy_version 1070205 (0.0009) [2023-12-26 23:10:08,528][105620] Updated weights for policy 1, policy_version 1071297 (0.0008) [2023-12-26 23:10:08,582][105620] Updated weights for policy 1, policy_version 1071307 (0.0008) [2023-12-26 23:10:08,646][105620] Updated weights for policy 1, policy_version 1071317 (0.0008) [2023-12-26 23:10:08,714][105620] Updated weights for policy 1, policy_version 1071327 (0.0008) [2023-12-26 23:10:08,890][105692] Updated weights for policy 0, policy_version 1070215 (0.0010) [2023-12-26 23:10:08,935][105692] Updated weights for policy 0, policy_version 1070225 (0.0010) [2023-12-26 23:10:08,984][105692] Updated weights for policy 0, policy_version 1070235 (0.0009) [2023-12-26 23:10:09,335][105620] Updated weights for policy 1, policy_version 1071337 (0.0010) [2023-12-26 23:10:09,404][105620] Updated weights for policy 1, policy_version 1071347 (0.0010) [2023-12-26 23:10:09,471][105620] Updated weights for policy 1, policy_version 1071357 (0.0011) [2023-12-26 23:10:09,697][105692] Updated weights for policy 0, policy_version 1070245 (0.0010) [2023-12-26 23:10:09,761][105692] Updated weights for policy 0, policy_version 1070255 (0.0011) [2023-12-26 23:10:09,829][105692] Updated weights for policy 0, policy_version 1070265 (0.0010) [2023-12-26 23:10:10,218][105620] Updated weights for policy 1, policy_version 1071367 (0.0011) [2023-12-26 23:10:10,274][105620] Updated weights for policy 1, policy_version 1071377 (0.0010) [2023-12-26 23:10:10,326][105620] Updated weights for policy 1, policy_version 1071387 (0.0010) [2023-12-26 23:10:10,528][105692] Updated weights for policy 0, policy_version 1070275 (0.0010) [2023-12-26 23:10:10,589][105692] Updated weights for policy 0, policy_version 1070285 (0.0010) [2023-12-26 23:10:10,650][105692] Updated weights for policy 0, policy_version 1070295 (0.0009) [2023-12-26 23:10:11,059][105620] Updated weights for policy 1, policy_version 1071397 (0.0010) [2023-12-26 23:10:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 548347904. Throughput: 0: 9418.5, 1: 9960.8. Samples: 548359292. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:10:11,063][104569] Avg episode reward: [(0, '9266.715'), (1, '9255.932')] [2023-12-26 23:10:11,126][105620] Updated weights for policy 1, policy_version 1071407 (0.0008) [2023-12-26 23:10:11,190][105620] Updated weights for policy 1, policy_version 1071417 (0.0008) [2023-12-26 23:10:11,361][105692] Updated weights for policy 0, policy_version 1070305 (0.0010) [2023-12-26 23:10:11,424][105692] Updated weights for policy 0, policy_version 1070315 (0.0008) [2023-12-26 23:10:11,480][105692] Updated weights for policy 0, policy_version 1070325 (0.0008) [2023-12-26 23:10:11,535][105692] Updated weights for policy 0, policy_version 1070335 (0.0008) [2023-12-26 23:10:11,932][105620] Updated weights for policy 1, policy_version 1071427 (0.0009) [2023-12-26 23:10:11,989][105620] Updated weights for policy 1, policy_version 1071437 (0.0010) [2023-12-26 23:10:12,049][105620] Updated weights for policy 1, policy_version 1071447 (0.0010) [2023-12-26 23:10:12,335][105692] Updated weights for policy 0, policy_version 1070345 (0.0009) [2023-12-26 23:10:12,396][105692] Updated weights for policy 0, policy_version 1070355 (0.0011) [2023-12-26 23:10:12,448][105692] Updated weights for policy 0, policy_version 1070365 (0.0010) [2023-12-26 23:10:12,811][105620] Updated weights for policy 1, policy_version 1071457 (0.0011) [2023-12-26 23:10:12,862][105620] Updated weights for policy 1, policy_version 1071467 (0.0010) [2023-12-26 23:10:12,915][105620] Updated weights for policy 1, policy_version 1071477 (0.0011) [2023-12-26 23:10:12,975][105620] Updated weights for policy 1, policy_version 1071487 (0.0007) [2023-12-26 23:10:13,136][105692] Updated weights for policy 0, policy_version 1070375 (0.0011) [2023-12-26 23:10:13,181][105692] Updated weights for policy 0, policy_version 1070385 (0.0010) [2023-12-26 23:10:13,223][105692] Updated weights for policy 0, policy_version 1070395 (0.0005) [2023-12-26 23:10:13,675][105620] Updated weights for policy 1, policy_version 1071497 (0.0008) [2023-12-26 23:10:13,723][105620] Updated weights for policy 1, policy_version 1071507 (0.0007) [2023-12-26 23:10:13,771][105620] Updated weights for policy 1, policy_version 1071517 (0.0008) [2023-12-26 23:10:13,916][105692] Updated weights for policy 0, policy_version 1070405 (0.0008) [2023-12-26 23:10:13,975][105692] Updated weights for policy 0, policy_version 1070415 (0.0007) [2023-12-26 23:10:14,025][105692] Updated weights for policy 0, policy_version 1070425 (0.0005) [2023-12-26 23:10:14,565][105620] Updated weights for policy 1, policy_version 1071527 (0.0006) [2023-12-26 23:10:14,631][105620] Updated weights for policy 1, policy_version 1071537 (0.0008) [2023-12-26 23:10:14,691][105620] Updated weights for policy 1, policy_version 1071547 (0.0011) [2023-12-26 23:10:14,756][105692] Updated weights for policy 0, policy_version 1070435 (0.0010) [2023-12-26 23:10:14,819][105692] Updated weights for policy 0, policy_version 1070445 (0.0010) [2023-12-26 23:10:14,889][105692] Updated weights for policy 0, policy_version 1070455 (0.0009) [2023-12-26 23:10:15,444][105620] Updated weights for policy 1, policy_version 1071557 (0.0008) [2023-12-26 23:10:15,499][105620] Updated weights for policy 1, policy_version 1071567 (0.0007) [2023-12-26 23:10:15,551][105620] Updated weights for policy 1, policy_version 1071577 (0.0011) [2023-12-26 23:10:15,581][105692] Updated weights for policy 0, policy_version 1070465 (0.0010) [2023-12-26 23:10:15,638][105692] Updated weights for policy 0, policy_version 1070475 (0.0005) [2023-12-26 23:10:15,697][105692] Updated weights for policy 0, policy_version 1070485 (0.0010) [2023-12-26 23:10:15,767][105692] Updated weights for policy 0, policy_version 1070495 (0.0005) [2023-12-26 23:10:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 548446208. Throughput: 0: 9360.3, 1: 9876.7. Samples: 548416500. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:10:16,062][104569] Avg episode reward: [(0, '9355.914'), (1, '9164.847')] [2023-12-26 23:10:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001071584_274358272.pth... [2023-12-26 23:10:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001070496_274087936.pth... [2023-12-26 23:10:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001070432_274063360.pth [2023-12-26 23:10:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001069408_273809408.pth [2023-12-26 23:10:16,134][105620] Updated weights for policy 1, policy_version 1071587 (0.0010) [2023-12-26 23:10:16,188][105620] Updated weights for policy 1, policy_version 1071597 (0.0008) [2023-12-26 23:10:16,252][105620] Updated weights for policy 1, policy_version 1071607 (0.0006) [2023-12-26 23:10:16,394][105692] Updated weights for policy 0, policy_version 1070505 (0.0010) [2023-12-26 23:10:16,449][105692] Updated weights for policy 0, policy_version 1070515 (0.0010) [2023-12-26 23:10:16,504][105692] Updated weights for policy 0, policy_version 1070525 (0.0010) [2023-12-26 23:10:16,825][105620] Updated weights for policy 1, policy_version 1071617 (0.0006) [2023-12-26 23:10:16,884][105620] Updated weights for policy 1, policy_version 1071627 (0.0007) [2023-12-26 23:10:16,934][105620] Updated weights for policy 1, policy_version 1071637 (0.0008) [2023-12-26 23:10:16,987][105620] Updated weights for policy 1, policy_version 1071647 (0.0008) [2023-12-26 23:10:17,248][105692] Updated weights for policy 0, policy_version 1070535 (0.0010) [2023-12-26 23:10:17,309][105692] Updated weights for policy 0, policy_version 1070545 (0.0010) [2023-12-26 23:10:17,373][105692] Updated weights for policy 0, policy_version 1070555 (0.0010) [2023-12-26 23:10:17,735][105620] Updated weights for policy 1, policy_version 1071657 (0.0010) [2023-12-26 23:10:17,800][105620] Updated weights for policy 1, policy_version 1071667 (0.0010) [2023-12-26 23:10:17,855][105620] Updated weights for policy 1, policy_version 1071677 (0.0010) [2023-12-26 23:10:18,104][105692] Updated weights for policy 0, policy_version 1070565 (0.0010) [2023-12-26 23:10:18,164][105692] Updated weights for policy 0, policy_version 1070575 (0.0010) [2023-12-26 23:10:18,217][105692] Updated weights for policy 0, policy_version 1070585 (0.0010) [2023-12-26 23:10:18,592][105620] Updated weights for policy 1, policy_version 1071687 (0.0010) [2023-12-26 23:10:18,647][105620] Updated weights for policy 1, policy_version 1071697 (0.0010) [2023-12-26 23:10:18,705][105620] Updated weights for policy 1, policy_version 1071707 (0.0010) [2023-12-26 23:10:18,970][105692] Updated weights for policy 0, policy_version 1070595 (0.0009) [2023-12-26 23:10:19,019][105692] Updated weights for policy 0, policy_version 1070605 (0.0006) [2023-12-26 23:10:19,073][105692] Updated weights for policy 0, policy_version 1070615 (0.0008) [2023-12-26 23:10:19,387][105620] Updated weights for policy 1, policy_version 1071717 (0.0008) [2023-12-26 23:10:19,446][105620] Updated weights for policy 1, policy_version 1071727 (0.0011) [2023-12-26 23:10:19,508][105620] Updated weights for policy 1, policy_version 1071737 (0.0011) [2023-12-26 23:10:19,851][105692] Updated weights for policy 0, policy_version 1070625 (0.0009) [2023-12-26 23:10:19,920][105692] Updated weights for policy 0, policy_version 1070635 (0.0011) [2023-12-26 23:10:19,980][105692] Updated weights for policy 0, policy_version 1070645 (0.0011) [2023-12-26 23:10:20,046][105692] Updated weights for policy 0, policy_version 1070655 (0.0010) [2023-12-26 23:10:20,277][105620] Updated weights for policy 1, policy_version 1071747 (0.0010) [2023-12-26 23:10:20,349][105620] Updated weights for policy 1, policy_version 1071757 (0.0009) [2023-12-26 23:10:20,407][105620] Updated weights for policy 1, policy_version 1071767 (0.0007) [2023-12-26 23:10:20,660][105692] Updated weights for policy 0, policy_version 1070665 (0.0007) [2023-12-26 23:10:20,728][105692] Updated weights for policy 0, policy_version 1070675 (0.0006) [2023-12-26 23:10:20,791][105692] Updated weights for policy 0, policy_version 1070685 (0.0010) [2023-12-26 23:10:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 548544512. Throughput: 0: 9374.4, 1: 9886.8. Samples: 548534448. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:10:21,063][104569] Avg episode reward: [(0, '9354.878'), (1, '8984.620')] [2023-12-26 23:10:21,078][105620] Updated weights for policy 1, policy_version 1071777 (0.0008) [2023-12-26 23:10:21,148][105620] Updated weights for policy 1, policy_version 1071787 (0.0008) [2023-12-26 23:10:21,209][105620] Updated weights for policy 1, policy_version 1071797 (0.0008) [2023-12-26 23:10:21,274][105620] Updated weights for policy 1, policy_version 1071807 (0.0008) [2023-12-26 23:10:21,480][105692] Updated weights for policy 0, policy_version 1070695 (0.0008) [2023-12-26 23:10:21,541][105692] Updated weights for policy 0, policy_version 1070705 (0.0008) [2023-12-26 23:10:21,601][105692] Updated weights for policy 0, policy_version 1070715 (0.0006) [2023-12-26 23:10:22,027][105620] Updated weights for policy 1, policy_version 1071817 (0.0008) [2023-12-26 23:10:22,083][105620] Updated weights for policy 1, policy_version 1071827 (0.0009) [2023-12-26 23:10:22,145][105620] Updated weights for policy 1, policy_version 1071837 (0.0010) [2023-12-26 23:10:22,265][105692] Updated weights for policy 0, policy_version 1070725 (0.0008) [2023-12-26 23:10:22,322][105692] Updated weights for policy 0, policy_version 1070735 (0.0008) [2023-12-26 23:10:22,394][105692] Updated weights for policy 0, policy_version 1070745 (0.0009) [2023-12-26 23:10:22,864][105620] Updated weights for policy 1, policy_version 1071847 (0.0010) [2023-12-26 23:10:22,912][105620] Updated weights for policy 1, policy_version 1071857 (0.0008) [2023-12-26 23:10:22,974][105620] Updated weights for policy 1, policy_version 1071867 (0.0010) [2023-12-26 23:10:23,115][105692] Updated weights for policy 0, policy_version 1070755 (0.0009) [2023-12-26 23:10:23,176][105692] Updated weights for policy 0, policy_version 1070765 (0.0008) [2023-12-26 23:10:23,262][105692] Updated weights for policy 0, policy_version 1070775 (0.0005) [2023-12-26 23:10:23,787][105692] Updated weights for policy 0, policy_version 1070785 (0.0006) [2023-12-26 23:10:23,825][105620] Updated weights for policy 1, policy_version 1071877 (0.0009) [2023-12-26 23:10:23,831][105692] Updated weights for policy 0, policy_version 1070795 (0.0008) [2023-12-26 23:10:23,870][105620] Updated weights for policy 1, policy_version 1071887 (0.0006) [2023-12-26 23:10:23,888][105692] Updated weights for policy 0, policy_version 1070805 (0.0008) [2023-12-26 23:10:23,915][105620] Updated weights for policy 1, policy_version 1071897 (0.0006) [2023-12-26 23:10:23,947][105692] Updated weights for policy 0, policy_version 1070815 (0.0009) [2023-12-26 23:10:24,659][105620] Updated weights for policy 1, policy_version 1071907 (0.0007) [2023-12-26 23:10:24,715][105620] Updated weights for policy 1, policy_version 1071917 (0.0005) [2023-12-26 23:10:24,717][105692] Updated weights for policy 0, policy_version 1070825 (0.0009) [2023-12-26 23:10:24,762][105692] Updated weights for policy 0, policy_version 1070835 (0.0007) [2023-12-26 23:10:24,764][105620] Updated weights for policy 1, policy_version 1071927 (0.0005) [2023-12-26 23:10:24,819][105692] Updated weights for policy 0, policy_version 1070845 (0.0008) [2023-12-26 23:10:25,502][105620] Updated weights for policy 1, policy_version 1071937 (0.0007) [2023-12-26 23:10:25,567][105620] Updated weights for policy 1, policy_version 1071947 (0.0011) [2023-12-26 23:10:25,609][105692] Updated weights for policy 0, policy_version 1070855 (0.0008) [2023-12-26 23:10:25,625][105620] Updated weights for policy 1, policy_version 1071957 (0.0010) [2023-12-26 23:10:25,659][105692] Updated weights for policy 0, policy_version 1070865 (0.0006) [2023-12-26 23:10:25,680][105620] Updated weights for policy 1, policy_version 1071967 (0.0010) [2023-12-26 23:10:25,719][105692] Updated weights for policy 0, policy_version 1070875 (0.0007) [2023-12-26 23:10:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 548642816. Throughput: 0: 9465.6, 1: 9847.7. Samples: 548650256. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:10:26,062][104569] Avg episode reward: [(0, '9355.039'), (1, '9075.356')] [2023-12-26 23:10:26,431][105620] Updated weights for policy 1, policy_version 1071977 (0.0011) [2023-12-26 23:10:26,491][105692] Updated weights for policy 0, policy_version 1070885 (0.0009) [2023-12-26 23:10:26,495][105620] Updated weights for policy 1, policy_version 1071987 (0.0011) [2023-12-26 23:10:26,552][105692] Updated weights for policy 0, policy_version 1070895 (0.0010) [2023-12-26 23:10:26,559][105620] Updated weights for policy 1, policy_version 1071997 (0.0011) [2023-12-26 23:10:26,612][105692] Updated weights for policy 0, policy_version 1070905 (0.0008) [2023-12-26 23:10:27,288][105692] Updated weights for policy 0, policy_version 1070915 (0.0009) [2023-12-26 23:10:27,300][105620] Updated weights for policy 1, policy_version 1072007 (0.0010) [2023-12-26 23:10:27,337][105692] Updated weights for policy 0, policy_version 1070925 (0.0007) [2023-12-26 23:10:27,350][105620] Updated weights for policy 1, policy_version 1072017 (0.0011) [2023-12-26 23:10:27,391][105692] Updated weights for policy 0, policy_version 1070935 (0.0010) [2023-12-26 23:10:27,405][105620] Updated weights for policy 1, policy_version 1072027 (0.0010) [2023-12-26 23:10:28,070][105692] Updated weights for policy 0, policy_version 1070945 (0.0010) [2023-12-26 23:10:28,125][105692] Updated weights for policy 0, policy_version 1070955 (0.0005) [2023-12-26 23:10:28,151][105620] Updated weights for policy 1, policy_version 1072037 (0.0010) [2023-12-26 23:10:28,179][105692] Updated weights for policy 0, policy_version 1070965 (0.0010) [2023-12-26 23:10:28,199][105620] Updated weights for policy 1, policy_version 1072047 (0.0010) [2023-12-26 23:10:28,233][105692] Updated weights for policy 0, policy_version 1070975 (0.0010) [2023-12-26 23:10:28,247][105620] Updated weights for policy 1, policy_version 1072057 (0.0010) [2023-12-26 23:10:28,908][105692] Updated weights for policy 0, policy_version 1070985 (0.0006) [2023-12-26 23:10:28,967][105692] Updated weights for policy 0, policy_version 1070995 (0.0006) [2023-12-26 23:10:29,027][105692] Updated weights for policy 0, policy_version 1071005 (0.0006) [2023-12-26 23:10:29,027][105620] Updated weights for policy 1, policy_version 1072067 (0.0010) [2023-12-26 23:10:29,081][105620] Updated weights for policy 1, policy_version 1072077 (0.0009) [2023-12-26 23:10:29,135][105620] Updated weights for policy 1, policy_version 1072087 (0.0010) [2023-12-26 23:10:29,636][105692] Updated weights for policy 0, policy_version 1071015 (0.0006) [2023-12-26 23:10:29,697][105692] Updated weights for policy 0, policy_version 1071025 (0.0005) [2023-12-26 23:10:29,755][105692] Updated weights for policy 0, policy_version 1071035 (0.0005) [2023-12-26 23:10:29,995][105620] Updated weights for policy 1, policy_version 1072097 (0.0008) [2023-12-26 23:10:30,049][105620] Updated weights for policy 1, policy_version 1072107 (0.0010) [2023-12-26 23:10:30,101][105620] Updated weights for policy 1, policy_version 1072117 (0.0010) [2023-12-26 23:10:30,156][105620] Updated weights for policy 1, policy_version 1072127 (0.0010) [2023-12-26 23:10:30,424][105692] Updated weights for policy 0, policy_version 1071045 (0.0008) [2023-12-26 23:10:30,480][105692] Updated weights for policy 0, policy_version 1071055 (0.0010) [2023-12-26 23:10:30,541][105692] Updated weights for policy 0, policy_version 1071065 (0.0010) [2023-12-26 23:10:30,831][105620] Updated weights for policy 1, policy_version 1072137 (0.0010) [2023-12-26 23:10:30,882][105620] Updated weights for policy 1, policy_version 1072147 (0.0010) [2023-12-26 23:10:30,927][105620] Updated weights for policy 1, policy_version 1072157 (0.0010) [2023-12-26 23:10:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 548741120. Throughput: 0: 9551.9, 1: 9816.8. Samples: 548708140. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:10:31,062][104569] Avg episode reward: [(0, '9172.693'), (1, '9255.710')] [2023-12-26 23:10:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001071072_274235392.pth... [2023-12-26 23:10:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001072160_274505728.pth... [2023-12-26 23:10:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001071008_274210816.pth [2023-12-26 23:10:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001069952_273948672.pth [2023-12-26 23:10:31,112][105692] Updated weights for policy 0, policy_version 1071075 (0.0009) [2023-12-26 23:10:31,175][105692] Updated weights for policy 0, policy_version 1071085 (0.0011) [2023-12-26 23:10:31,238][105692] Updated weights for policy 0, policy_version 1071095 (0.0010) [2023-12-26 23:10:31,744][105620] Updated weights for policy 1, policy_version 1072167 (0.0010) [2023-12-26 23:10:31,804][105620] Updated weights for policy 1, policy_version 1072177 (0.0007) [2023-12-26 23:10:31,865][105620] Updated weights for policy 1, policy_version 1072187 (0.0007) [2023-12-26 23:10:32,007][105692] Updated weights for policy 0, policy_version 1071105 (0.0011) [2023-12-26 23:10:32,067][105692] Updated weights for policy 0, policy_version 1071115 (0.0009) [2023-12-26 23:10:32,118][105692] Updated weights for policy 0, policy_version 1071125 (0.0009) [2023-12-26 23:10:32,180][105692] Updated weights for policy 0, policy_version 1071135 (0.0008) [2023-12-26 23:10:32,505][105620] Updated weights for policy 1, policy_version 1072197 (0.0009) [2023-12-26 23:10:32,574][105620] Updated weights for policy 1, policy_version 1072207 (0.0011) [2023-12-26 23:10:32,630][105620] Updated weights for policy 1, policy_version 1072217 (0.0011) [2023-12-26 23:10:32,980][105692] Updated weights for policy 0, policy_version 1071145 (0.0010) [2023-12-26 23:10:33,031][105692] Updated weights for policy 0, policy_version 1071155 (0.0010) [2023-12-26 23:10:33,092][105692] Updated weights for policy 0, policy_version 1071165 (0.0010) [2023-12-26 23:10:33,367][105620] Updated weights for policy 1, policy_version 1072227 (0.0009) [2023-12-26 23:10:33,415][105620] Updated weights for policy 1, policy_version 1072237 (0.0005) [2023-12-26 23:10:33,477][105620] Updated weights for policy 1, policy_version 1072247 (0.0005) [2023-12-26 23:10:33,806][105692] Updated weights for policy 0, policy_version 1071175 (0.0010) [2023-12-26 23:10:33,853][105692] Updated weights for policy 0, policy_version 1071185 (0.0010) [2023-12-26 23:10:33,901][105692] Updated weights for policy 0, policy_version 1071195 (0.0010) [2023-12-26 23:10:34,152][105620] Updated weights for policy 1, policy_version 1072257 (0.0006) [2023-12-26 23:10:34,211][105620] Updated weights for policy 1, policy_version 1072267 (0.0009) [2023-12-26 23:10:34,266][105620] Updated weights for policy 1, policy_version 1072277 (0.0010) [2023-12-26 23:10:34,322][105620] Updated weights for policy 1, policy_version 1072287 (0.0007) [2023-12-26 23:10:34,607][105692] Updated weights for policy 0, policy_version 1071205 (0.0010) [2023-12-26 23:10:34,670][105692] Updated weights for policy 0, policy_version 1071215 (0.0011) [2023-12-26 23:10:34,728][105692] Updated weights for policy 0, policy_version 1071225 (0.0010) [2023-12-26 23:10:35,014][105620] Updated weights for policy 1, policy_version 1072297 (0.0008) [2023-12-26 23:10:35,064][105620] Updated weights for policy 1, policy_version 1072307 (0.0008) [2023-12-26 23:10:35,113][105620] Updated weights for policy 1, policy_version 1072317 (0.0005) [2023-12-26 23:10:35,473][105692] Updated weights for policy 0, policy_version 1071235 (0.0011) [2023-12-26 23:10:35,532][105692] Updated weights for policy 0, policy_version 1071245 (0.0010) [2023-12-26 23:10:35,586][105692] Updated weights for policy 0, policy_version 1071255 (0.0010) [2023-12-26 23:10:35,741][105620] Updated weights for policy 1, policy_version 1072327 (0.0005) [2023-12-26 23:10:35,795][105620] Updated weights for policy 1, policy_version 1072337 (0.0005) [2023-12-26 23:10:35,853][105620] Updated weights for policy 1, policy_version 1072347 (0.0005) [2023-12-26 23:10:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19605.2). Total num frames: 548839424. Throughput: 0: 9662.0, 1: 9805.9. Samples: 548826268. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:10:36,063][104569] Avg episode reward: [(0, '9083.004'), (1, '9257.103')] [2023-12-26 23:10:36,291][105692] Updated weights for policy 0, policy_version 1071265 (0.0010) [2023-12-26 23:10:36,352][105692] Updated weights for policy 0, policy_version 1071275 (0.0008) [2023-12-26 23:10:36,414][105692] Updated weights for policy 0, policy_version 1071285 (0.0008) [2023-12-26 23:10:36,436][105620] Updated weights for policy 1, policy_version 1072357 (0.0007) [2023-12-26 23:10:36,471][105692] Updated weights for policy 0, policy_version 1071295 (0.0007) [2023-12-26 23:10:36,499][105620] Updated weights for policy 1, policy_version 1072367 (0.0007) [2023-12-26 23:10:36,555][105620] Updated weights for policy 1, policy_version 1072377 (0.0009) [2023-12-26 23:10:37,109][105692] Updated weights for policy 0, policy_version 1071305 (0.0005) [2023-12-26 23:10:37,170][105692] Updated weights for policy 0, policy_version 1071315 (0.0007) [2023-12-26 23:10:37,229][105692] Updated weights for policy 0, policy_version 1071325 (0.0008) [2023-12-26 23:10:37,291][105620] Updated weights for policy 1, policy_version 1072387 (0.0008) [2023-12-26 23:10:37,341][105620] Updated weights for policy 1, policy_version 1072397 (0.0005) [2023-12-26 23:10:37,393][105620] Updated weights for policy 1, policy_version 1072407 (0.0005) [2023-12-26 23:10:37,906][105692] Updated weights for policy 0, policy_version 1071335 (0.0010) [2023-12-26 23:10:37,956][105692] Updated weights for policy 0, policy_version 1071345 (0.0010) [2023-12-26 23:10:38,011][105692] Updated weights for policy 0, policy_version 1071355 (0.0010) [2023-12-26 23:10:38,028][105620] Updated weights for policy 1, policy_version 1072417 (0.0007) [2023-12-26 23:10:38,084][105620] Updated weights for policy 1, policy_version 1072427 (0.0005) [2023-12-26 23:10:38,140][105620] Updated weights for policy 1, policy_version 1072437 (0.0005) [2023-12-26 23:10:38,197][105620] Updated weights for policy 1, policy_version 1072447 (0.0008) [2023-12-26 23:10:38,780][105692] Updated weights for policy 0, policy_version 1071365 (0.0010) [2023-12-26 23:10:38,831][105692] Updated weights for policy 0, policy_version 1071375 (0.0010) [2023-12-26 23:10:38,887][105692] Updated weights for policy 0, policy_version 1071385 (0.0010) [2023-12-26 23:10:38,907][105620] Updated weights for policy 1, policy_version 1072457 (0.0010) [2023-12-26 23:10:38,972][105620] Updated weights for policy 1, policy_version 1072467 (0.0010) [2023-12-26 23:10:39,037][105620] Updated weights for policy 1, policy_version 1072477 (0.0010) [2023-12-26 23:10:39,652][105692] Updated weights for policy 0, policy_version 1071395 (0.0008) [2023-12-26 23:10:39,718][105692] Updated weights for policy 0, policy_version 1071405 (0.0009) [2023-12-26 23:10:39,741][105620] Updated weights for policy 1, policy_version 1072487 (0.0008) [2023-12-26 23:10:39,778][105692] Updated weights for policy 0, policy_version 1071415 (0.0007) [2023-12-26 23:10:39,802][105620] Updated weights for policy 1, policy_version 1072497 (0.0007) [2023-12-26 23:10:39,871][105620] Updated weights for policy 1, policy_version 1072507 (0.0008) [2023-12-26 23:10:40,542][105692] Updated weights for policy 0, policy_version 1071425 (0.0009) [2023-12-26 23:10:40,583][105620] Updated weights for policy 1, policy_version 1072517 (0.0007) [2023-12-26 23:10:40,606][105692] Updated weights for policy 0, policy_version 1071435 (0.0010) [2023-12-26 23:10:40,635][105620] Updated weights for policy 1, policy_version 1072527 (0.0005) [2023-12-26 23:10:40,668][105692] Updated weights for policy 0, policy_version 1071445 (0.0008) [2023-12-26 23:10:40,692][105620] Updated weights for policy 1, policy_version 1072537 (0.0006) [2023-12-26 23:10:40,727][105692] Updated weights for policy 0, policy_version 1071455 (0.0006) [2023-12-26 23:10:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 548937728. Throughput: 0: 9698.7, 1: 9908.6. Samples: 548945836. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:10:41,062][104569] Avg episode reward: [(0, '9174.510'), (1, '9086.184')] [2023-12-26 23:10:41,392][105620] Updated weights for policy 1, policy_version 1072547 (0.0006) [2023-12-26 23:10:41,455][105620] Updated weights for policy 1, policy_version 1072557 (0.0010) [2023-12-26 23:10:41,494][105692] Updated weights for policy 0, policy_version 1071465 (0.0008) [2023-12-26 23:10:41,504][105620] Updated weights for policy 1, policy_version 1072567 (0.0007) [2023-12-26 23:10:41,553][105692] Updated weights for policy 0, policy_version 1071475 (0.0006) [2023-12-26 23:10:41,600][105692] Updated weights for policy 0, policy_version 1071485 (0.0010) [2023-12-26 23:10:42,245][105620] Updated weights for policy 1, policy_version 1072577 (0.0008) [2023-12-26 23:10:42,307][105620] Updated weights for policy 1, policy_version 1072587 (0.0007) [2023-12-26 23:10:42,372][105620] Updated weights for policy 1, policy_version 1072597 (0.0007) [2023-12-26 23:10:42,431][105620] Updated weights for policy 1, policy_version 1072607 (0.0009) [2023-12-26 23:10:42,438][105692] Updated weights for policy 0, policy_version 1071495 (0.0007) [2023-12-26 23:10:42,497][105692] Updated weights for policy 0, policy_version 1071505 (0.0009) [2023-12-26 23:10:42,546][105692] Updated weights for policy 0, policy_version 1071515 (0.0009) [2023-12-26 23:10:43,089][105620] Updated weights for policy 1, policy_version 1072617 (0.0007) [2023-12-26 23:10:43,155][105620] Updated weights for policy 1, policy_version 1072627 (0.0005) [2023-12-26 23:10:43,211][105620] Updated weights for policy 1, policy_version 1072637 (0.0009) [2023-12-26 23:10:43,438][105692] Updated weights for policy 0, policy_version 1071525 (0.0008) [2023-12-26 23:10:43,495][105692] Updated weights for policy 0, policy_version 1071535 (0.0005) [2023-12-26 23:10:43,550][105692] Updated weights for policy 0, policy_version 1071545 (0.0005) [2023-12-26 23:10:43,736][105620] Updated weights for policy 1, policy_version 1072647 (0.0006) [2023-12-26 23:10:43,801][105620] Updated weights for policy 1, policy_version 1072657 (0.0007) [2023-12-26 23:10:43,868][105620] Updated weights for policy 1, policy_version 1072667 (0.0008) [2023-12-26 23:10:44,194][105692] Updated weights for policy 0, policy_version 1071555 (0.0007) [2023-12-26 23:10:44,242][105692] Updated weights for policy 0, policy_version 1071565 (0.0010) [2023-12-26 23:10:44,300][105692] Updated weights for policy 0, policy_version 1071575 (0.0010) [2023-12-26 23:10:44,521][105620] Updated weights for policy 1, policy_version 1072677 (0.0009) [2023-12-26 23:10:44,575][105620] Updated weights for policy 1, policy_version 1072687 (0.0009) [2023-12-26 23:10:44,642][105620] Updated weights for policy 1, policy_version 1072697 (0.0006) [2023-12-26 23:10:45,053][105692] Updated weights for policy 0, policy_version 1071585 (0.0010) [2023-12-26 23:10:45,104][105692] Updated weights for policy 0, policy_version 1071595 (0.0009) [2023-12-26 23:10:45,164][105692] Updated weights for policy 0, policy_version 1071605 (0.0010) [2023-12-26 23:10:45,216][105692] Updated weights for policy 0, policy_version 1071615 (0.0009) [2023-12-26 23:10:45,288][105620] Updated weights for policy 1, policy_version 1072707 (0.0006) [2023-12-26 23:10:45,344][105620] Updated weights for policy 1, policy_version 1072717 (0.0008) [2023-12-26 23:10:45,404][105620] Updated weights for policy 1, policy_version 1072727 (0.0008) [2023-12-26 23:10:45,860][105692] Updated weights for policy 0, policy_version 1071625 (0.0006) [2023-12-26 23:10:45,909][105692] Updated weights for policy 0, policy_version 1071635 (0.0005) [2023-12-26 23:10:45,971][105692] Updated weights for policy 0, policy_version 1071645 (0.0005) [2023-12-26 23:10:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 549036032. Throughput: 0: 9666.0, 1: 9953.0. Samples: 549002860. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:10:46,063][104569] Avg episode reward: [(0, '9265.098'), (1, '9176.789')] [2023-12-26 23:10:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001071648_274382848.pth... [2023-12-26 23:10:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001072736_274653184.pth... [2023-12-26 23:10:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001070496_274087936.pth [2023-12-26 23:10:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001071584_274358272.pth [2023-12-26 23:10:46,113][105620] Updated weights for policy 1, policy_version 1072737 (0.0009) [2023-12-26 23:10:46,164][105620] Updated weights for policy 1, policy_version 1072747 (0.0005) [2023-12-26 23:10:46,224][105620] Updated weights for policy 1, policy_version 1072757 (0.0009) [2023-12-26 23:10:46,302][105620] Updated weights for policy 1, policy_version 1072767 (0.0010) [2023-12-26 23:10:46,542][105692] Updated weights for policy 0, policy_version 1071655 (0.0005) [2023-12-26 23:10:46,603][105692] Updated weights for policy 0, policy_version 1071665 (0.0006) [2023-12-26 23:10:46,655][105692] Updated weights for policy 0, policy_version 1071675 (0.0005) [2023-12-26 23:10:47,141][105620] Updated weights for policy 1, policy_version 1072777 (0.0009) [2023-12-26 23:10:47,201][105620] Updated weights for policy 1, policy_version 1072787 (0.0005) [2023-12-26 23:10:47,203][105692] Updated weights for policy 0, policy_version 1071685 (0.0005) [2023-12-26 23:10:47,261][105692] Updated weights for policy 0, policy_version 1071695 (0.0005) [2023-12-26 23:10:47,265][105620] Updated weights for policy 1, policy_version 1072797 (0.0008) [2023-12-26 23:10:47,319][105692] Updated weights for policy 0, policy_version 1071705 (0.0005) [2023-12-26 23:10:47,816][105692] Updated weights for policy 0, policy_version 1071715 (0.0005) [2023-12-26 23:10:47,877][105692] Updated weights for policy 0, policy_version 1071725 (0.0010) [2023-12-26 23:10:47,923][105692] Updated weights for policy 0, policy_version 1071735 (0.0006) [2023-12-26 23:10:48,085][105620] Updated weights for policy 1, policy_version 1072807 (0.0009) [2023-12-26 23:10:48,146][105620] Updated weights for policy 1, policy_version 1072817 (0.0008) [2023-12-26 23:10:48,202][105620] Updated weights for policy 1, policy_version 1072827 (0.0007) [2023-12-26 23:10:48,522][105692] Updated weights for policy 0, policy_version 1071745 (0.0005) [2023-12-26 23:10:48,589][105692] Updated weights for policy 0, policy_version 1071755 (0.0010) [2023-12-26 23:10:48,649][105692] Updated weights for policy 0, policy_version 1071765 (0.0011) [2023-12-26 23:10:48,701][105692] Updated weights for policy 0, policy_version 1071775 (0.0011) [2023-12-26 23:10:48,918][105620] Updated weights for policy 1, policy_version 1072837 (0.0008) [2023-12-26 23:10:48,970][105620] Updated weights for policy 1, policy_version 1072847 (0.0010) [2023-12-26 23:10:49,027][105620] Updated weights for policy 1, policy_version 1072857 (0.0009) [2023-12-26 23:10:49,473][105692] Updated weights for policy 0, policy_version 1071785 (0.0006) [2023-12-26 23:10:49,531][105692] Updated weights for policy 0, policy_version 1071795 (0.0009) [2023-12-26 23:10:49,581][105692] Updated weights for policy 0, policy_version 1071805 (0.0010) [2023-12-26 23:10:49,713][105620] Updated weights for policy 1, policy_version 1072867 (0.0007) [2023-12-26 23:10:49,761][105620] Updated weights for policy 1, policy_version 1072877 (0.0010) [2023-12-26 23:10:49,809][105620] Updated weights for policy 1, policy_version 1072887 (0.0010) [2023-12-26 23:10:50,282][105692] Updated weights for policy 0, policy_version 1071815 (0.0007) [2023-12-26 23:10:50,342][105692] Updated weights for policy 0, policy_version 1071825 (0.0009) [2023-12-26 23:10:50,398][105692] Updated weights for policy 0, policy_version 1071835 (0.0011) [2023-12-26 23:10:50,535][105620] Updated weights for policy 1, policy_version 1072897 (0.0008) [2023-12-26 23:10:50,596][105620] Updated weights for policy 1, policy_version 1072907 (0.0008) [2023-12-26 23:10:50,662][105620] Updated weights for policy 1, policy_version 1072917 (0.0011) [2023-12-26 23:10:50,735][105620] Updated weights for policy 1, policy_version 1072927 (0.0011) [2023-12-26 23:10:51,020][105692] Updated weights for policy 0, policy_version 1071845 (0.0010) [2023-12-26 23:10:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 549134336. Throughput: 0: 9833.0, 1: 9845.8. Samples: 549125192. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:10:51,062][104569] Avg episode reward: [(0, '9263.850'), (1, '9072.672')] [2023-12-26 23:10:51,086][105692] Updated weights for policy 0, policy_version 1071855 (0.0011) [2023-12-26 23:10:51,143][105692] Updated weights for policy 0, policy_version 1071865 (0.0011) [2023-12-26 23:10:51,430][105620] Updated weights for policy 1, policy_version 1072937 (0.0007) [2023-12-26 23:10:51,479][105620] Updated weights for policy 1, policy_version 1072947 (0.0008) [2023-12-26 23:10:51,529][105620] Updated weights for policy 1, policy_version 1072957 (0.0008) [2023-12-26 23:10:51,859][105692] Updated weights for policy 0, policy_version 1071875 (0.0010) [2023-12-26 23:10:51,912][105692] Updated weights for policy 0, policy_version 1071885 (0.0010) [2023-12-26 23:10:51,962][105692] Updated weights for policy 0, policy_version 1071895 (0.0010) [2023-12-26 23:10:52,326][105620] Updated weights for policy 1, policy_version 1072967 (0.0007) [2023-12-26 23:10:52,393][105620] Updated weights for policy 1, policy_version 1072977 (0.0008) [2023-12-26 23:10:52,450][105620] Updated weights for policy 1, policy_version 1072987 (0.0008) [2023-12-26 23:10:52,707][105692] Updated weights for policy 0, policy_version 1071905 (0.0011) [2023-12-26 23:10:52,769][105692] Updated weights for policy 0, policy_version 1071915 (0.0011) [2023-12-26 23:10:52,834][105692] Updated weights for policy 0, policy_version 1071925 (0.0011) [2023-12-26 23:10:52,904][105692] Updated weights for policy 0, policy_version 1071935 (0.0010) [2023-12-26 23:10:53,242][105620] Updated weights for policy 1, policy_version 1072998 (0.0010) [2023-12-26 23:10:53,299][105620] Updated weights for policy 1, policy_version 1073008 (0.0009) [2023-12-26 23:10:53,349][105620] Updated weights for policy 1, policy_version 1073018 (0.0008) [2023-12-26 23:10:53,567][105692] Updated weights for policy 0, policy_version 1071945 (0.0010) [2023-12-26 23:10:53,622][105585] KL-divergence is very high: 155.3628 [2023-12-26 23:10:53,622][105692] Updated weights for policy 0, policy_version 1071955 (0.0010) [2023-12-26 23:10:53,658][105585] KL-divergence is very high: 165.9921 [2023-12-26 23:10:53,674][105692] Updated weights for policy 0, policy_version 1071965 (0.0010) [2023-12-26 23:10:54,128][105620] Updated weights for policy 1, policy_version 1073028 (0.0007) [2023-12-26 23:10:54,175][105620] Updated weights for policy 1, policy_version 1073038 (0.0005) [2023-12-26 23:10:54,230][105620] Updated weights for policy 1, policy_version 1073048 (0.0005) [2023-12-26 23:10:54,417][105692] Updated weights for policy 0, policy_version 1071975 (0.0010) [2023-12-26 23:10:54,465][105692] Updated weights for policy 0, policy_version 1071985 (0.0010) [2023-12-26 23:10:54,517][105692] Updated weights for policy 0, policy_version 1071995 (0.0010) [2023-12-26 23:10:54,868][105620] Updated weights for policy 1, policy_version 1073058 (0.0006) [2023-12-26 23:10:54,919][105620] Updated weights for policy 1, policy_version 1073068 (0.0005) [2023-12-26 23:10:54,981][105620] Updated weights for policy 1, policy_version 1073078 (0.0005) [2023-12-26 23:10:55,041][105620] Updated weights for policy 1, policy_version 1073088 (0.0006) [2023-12-26 23:10:55,266][105692] Updated weights for policy 0, policy_version 1072005 (0.0011) [2023-12-26 23:10:55,318][105692] Updated weights for policy 0, policy_version 1072015 (0.0011) [2023-12-26 23:10:55,377][105692] Updated weights for policy 0, policy_version 1072025 (0.0011) [2023-12-26 23:10:55,653][105620] Updated weights for policy 1, policy_version 1073098 (0.0010) [2023-12-26 23:10:55,712][105620] Updated weights for policy 1, policy_version 1073108 (0.0010) [2023-12-26 23:10:55,771][105620] Updated weights for policy 1, policy_version 1073118 (0.0010) [2023-12-26 23:10:56,003][105692] Updated weights for policy 0, policy_version 1072035 (0.0009) [2023-12-26 23:10:56,057][105692] Updated weights for policy 0, policy_version 1072045 (0.0011) [2023-12-26 23:10:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 549232640. Throughput: 0: 9888.7, 1: 9746.8. Samples: 549242888. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:10:56,062][104569] Avg episode reward: [(0, '9087.364'), (1, '9164.206')] [2023-12-26 23:10:56,108][105692] Updated weights for policy 0, policy_version 1072055 (0.0010) [2023-12-26 23:10:56,377][105620] Updated weights for policy 1, policy_version 1073128 (0.0006) [2023-12-26 23:10:56,433][105620] Updated weights for policy 1, policy_version 1073138 (0.0005) [2023-12-26 23:10:56,495][105620] Updated weights for policy 1, policy_version 1073148 (0.0007) [2023-12-26 23:10:56,913][105692] Updated weights for policy 0, policy_version 1072065 (0.0011) [2023-12-26 23:10:56,957][105692] Updated weights for policy 0, policy_version 1072075 (0.0008) [2023-12-26 23:10:57,014][105692] Updated weights for policy 0, policy_version 1072085 (0.0008) [2023-12-26 23:10:57,067][105692] Updated weights for policy 0, policy_version 1072095 (0.0007) [2023-12-26 23:10:57,194][105620] Updated weights for policy 1, policy_version 1073158 (0.0010) [2023-12-26 23:10:57,265][105620] Updated weights for policy 1, policy_version 1073168 (0.0010) [2023-12-26 23:10:57,320][105620] Updated weights for policy 1, policy_version 1073178 (0.0010) [2023-12-26 23:10:57,753][105692] Updated weights for policy 0, policy_version 1072105 (0.0006) [2023-12-26 23:10:57,805][105692] Updated weights for policy 0, policy_version 1072115 (0.0005) [2023-12-26 23:10:57,851][105692] Updated weights for policy 0, policy_version 1072125 (0.0005) [2023-12-26 23:10:57,983][105620] Updated weights for policy 1, policy_version 1073188 (0.0008) [2023-12-26 23:10:58,043][105620] Updated weights for policy 1, policy_version 1073198 (0.0006) [2023-12-26 23:10:58,101][105620] Updated weights for policy 1, policy_version 1073208 (0.0005) [2023-12-26 23:10:58,475][105692] Updated weights for policy 0, policy_version 1072135 (0.0007) [2023-12-26 23:10:58,534][105692] Updated weights for policy 0, policy_version 1072145 (0.0008) [2023-12-26 23:10:58,592][105692] Updated weights for policy 0, policy_version 1072155 (0.0008) [2023-12-26 23:10:58,792][105620] Updated weights for policy 1, policy_version 1073218 (0.0007) [2023-12-26 23:10:58,863][105620] Updated weights for policy 1, policy_version 1073228 (0.0008) [2023-12-26 23:10:58,931][105620] Updated weights for policy 1, policy_version 1073238 (0.0009) [2023-12-26 23:10:58,994][105620] Updated weights for policy 1, policy_version 1073248 (0.0008) [2023-12-26 23:10:59,453][105692] Updated weights for policy 0, policy_version 1072165 (0.0010) [2023-12-26 23:10:59,518][105692] Updated weights for policy 0, policy_version 1072175 (0.0010) [2023-12-26 23:10:59,583][105692] Updated weights for policy 0, policy_version 1072185 (0.0009) [2023-12-26 23:10:59,737][105620] Updated weights for policy 1, policy_version 1073258 (0.0006) [2023-12-26 23:10:59,802][105620] Updated weights for policy 1, policy_version 1073268 (0.0006) [2023-12-26 23:10:59,863][105620] Updated weights for policy 1, policy_version 1073278 (0.0010) [2023-12-26 23:11:00,327][105692] Updated weights for policy 0, policy_version 1072195 (0.0009) [2023-12-26 23:11:00,391][105692] Updated weights for policy 0, policy_version 1072205 (0.0010) [2023-12-26 23:11:00,447][105692] Updated weights for policy 0, policy_version 1072215 (0.0010) [2023-12-26 23:11:00,487][105620] Updated weights for policy 1, policy_version 1073288 (0.0011) [2023-12-26 23:11:00,548][105620] Updated weights for policy 1, policy_version 1073298 (0.0009) [2023-12-26 23:11:00,605][105620] Updated weights for policy 1, policy_version 1073308 (0.0009) [2023-12-26 23:11:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 549330944. Throughput: 0: 9914.5, 1: 9807.8. Samples: 549304004. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:11:01,063][104569] Avg episode reward: [(0, '8916.811'), (1, '9255.379')] [2023-12-26 23:11:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001072224_274530304.pth... [2023-12-26 23:11:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001073312_274800640.pth... [2023-12-26 23:11:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001072160_274505728.pth [2023-12-26 23:11:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001071072_274235392.pth [2023-12-26 23:11:01,153][105692] Updated weights for policy 0, policy_version 1072225 (0.0009) [2023-12-26 23:11:01,221][105692] Updated weights for policy 0, policy_version 1072235 (0.0008) [2023-12-26 23:11:01,287][105692] Updated weights for policy 0, policy_version 1072245 (0.0008) [2023-12-26 23:11:01,337][105620] Updated weights for policy 1, policy_version 1073318 (0.0008) [2023-12-26 23:11:01,345][105692] Updated weights for policy 0, policy_version 1072255 (0.0008) [2023-12-26 23:11:01,404][105620] Updated weights for policy 1, policy_version 1073328 (0.0009) [2023-12-26 23:11:01,466][105620] Updated weights for policy 1, policy_version 1073338 (0.0009) [2023-12-26 23:11:01,974][105692] Updated weights for policy 0, policy_version 1072265 (0.0006) [2023-12-26 23:11:02,029][105692] Updated weights for policy 0, policy_version 1072275 (0.0005) [2023-12-26 23:11:02,081][105692] Updated weights for policy 0, policy_version 1072285 (0.0005) [2023-12-26 23:11:02,309][105620] Updated weights for policy 1, policy_version 1073348 (0.0010) [2023-12-26 23:11:02,376][105620] Updated weights for policy 1, policy_version 1073358 (0.0010) [2023-12-26 23:11:02,431][105620] Updated weights for policy 1, policy_version 1073368 (0.0010) [2023-12-26 23:11:02,762][105692] Updated weights for policy 0, policy_version 1072295 (0.0007) [2023-12-26 23:11:02,822][105692] Updated weights for policy 0, policy_version 1072305 (0.0008) [2023-12-26 23:11:02,881][105692] Updated weights for policy 0, policy_version 1072315 (0.0008) [2023-12-26 23:11:03,155][105620] Updated weights for policy 1, policy_version 1073378 (0.0009) [2023-12-26 23:11:03,200][105620] Updated weights for policy 1, policy_version 1073388 (0.0008) [2023-12-26 23:11:03,250][105620] Updated weights for policy 1, policy_version 1073398 (0.0008) [2023-12-26 23:11:03,304][105620] Updated weights for policy 1, policy_version 1073408 (0.0005) [2023-12-26 23:11:03,682][105692] Updated weights for policy 0, policy_version 1072326 (0.0009) [2023-12-26 23:11:03,726][105692] Updated weights for policy 0, policy_version 1072336 (0.0007) [2023-12-26 23:11:03,771][105692] Updated weights for policy 0, policy_version 1072346 (0.0008) [2023-12-26 23:11:03,974][105620] Updated weights for policy 1, policy_version 1073418 (0.0007) [2023-12-26 23:11:04,028][105620] Updated weights for policy 1, policy_version 1073428 (0.0007) [2023-12-26 23:11:04,095][105620] Updated weights for policy 1, policy_version 1073438 (0.0011) [2023-12-26 23:11:04,555][105692] Updated weights for policy 0, policy_version 1072356 (0.0007) [2023-12-26 23:11:04,624][105692] Updated weights for policy 0, policy_version 1072366 (0.0007) [2023-12-26 23:11:04,690][105692] Updated weights for policy 0, policy_version 1072376 (0.0008) [2023-12-26 23:11:04,812][105620] Updated weights for policy 1, policy_version 1073448 (0.0010) [2023-12-26 23:11:04,860][105620] Updated weights for policy 1, policy_version 1073458 (0.0010) [2023-12-26 23:11:04,915][105620] Updated weights for policy 1, policy_version 1073468 (0.0010) [2023-12-26 23:11:05,367][105692] Updated weights for policy 0, policy_version 1072386 (0.0008) [2023-12-26 23:11:05,422][105692] Updated weights for policy 0, policy_version 1072396 (0.0008) [2023-12-26 23:11:05,481][105692] Updated weights for policy 0, policy_version 1072406 (0.0009) [2023-12-26 23:11:05,543][105692] Updated weights for policy 0, policy_version 1072416 (0.0008) [2023-12-26 23:11:05,658][105620] Updated weights for policy 1, policy_version 1073478 (0.0010) [2023-12-26 23:11:05,722][105620] Updated weights for policy 1, policy_version 1073488 (0.0010) [2023-12-26 23:11:05,778][105620] Updated weights for policy 1, policy_version 1073498 (0.0011) [2023-12-26 23:11:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 549429248. Throughput: 0: 9891.2, 1: 9739.4. Samples: 549417824. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:11:06,062][104569] Avg episode reward: [(0, '8568.035'), (1, '8649.774')] [2023-12-26 23:11:06,307][105692] Updated weights for policy 0, policy_version 1072426 (0.0008) [2023-12-26 23:11:06,367][105692] Updated weights for policy 0, policy_version 1072436 (0.0008) [2023-12-26 23:11:06,427][105692] Updated weights for policy 0, policy_version 1072446 (0.0008) [2023-12-26 23:11:06,521][105620] Updated weights for policy 1, policy_version 1073508 (0.0008) [2023-12-26 23:11:06,573][105620] Updated weights for policy 1, policy_version 1073518 (0.0005) [2023-12-26 23:11:06,639][105620] Updated weights for policy 1, policy_version 1073528 (0.0005) [2023-12-26 23:11:07,185][105692] Updated weights for policy 0, policy_version 1072456 (0.0008) [2023-12-26 23:11:07,237][105692] Updated weights for policy 0, policy_version 1072466 (0.0008) [2023-12-26 23:11:07,285][105692] Updated weights for policy 0, policy_version 1072476 (0.0008) [2023-12-26 23:11:07,317][105620] Updated weights for policy 1, policy_version 1073538 (0.0007) [2023-12-26 23:11:07,371][105620] Updated weights for policy 1, policy_version 1073548 (0.0010) [2023-12-26 23:11:07,422][105620] Updated weights for policy 1, policy_version 1073558 (0.0010) [2023-12-26 23:11:07,486][105620] Updated weights for policy 1, policy_version 1073568 (0.0009) [2023-12-26 23:11:08,074][105692] Updated weights for policy 0, policy_version 1072486 (0.0008) [2023-12-26 23:11:08,131][105692] Updated weights for policy 0, policy_version 1072496 (0.0005) [2023-12-26 23:11:08,197][105692] Updated weights for policy 0, policy_version 1072506 (0.0010) [2023-12-26 23:11:08,226][105620] Updated weights for policy 1, policy_version 1073578 (0.0005) [2023-12-26 23:11:08,283][105620] Updated weights for policy 1, policy_version 1073588 (0.0005) [2023-12-26 23:11:08,350][105620] Updated weights for policy 1, policy_version 1073598 (0.0008) [2023-12-26 23:11:08,960][105692] Updated weights for policy 0, policy_version 1072516 (0.0007) [2023-12-26 23:11:09,012][105692] Updated weights for policy 0, policy_version 1072526 (0.0008) [2023-12-26 23:11:09,045][105620] Updated weights for policy 1, policy_version 1073608 (0.0011) [2023-12-26 23:11:09,067][105692] Updated weights for policy 0, policy_version 1072536 (0.0005) [2023-12-26 23:11:09,103][105620] Updated weights for policy 1, policy_version 1073618 (0.0010) [2023-12-26 23:11:09,166][105620] Updated weights for policy 1, policy_version 1073628 (0.0010) [2023-12-26 23:11:09,875][105692] Updated weights for policy 0, policy_version 1072546 (0.0006) [2023-12-26 23:11:09,892][105620] Updated weights for policy 1, policy_version 1073638 (0.0010) [2023-12-26 23:11:09,936][105692] Updated weights for policy 0, policy_version 1072556 (0.0006) [2023-12-26 23:11:09,954][105620] Updated weights for policy 1, policy_version 1073648 (0.0011) [2023-12-26 23:11:09,997][105692] Updated weights for policy 0, policy_version 1072566 (0.0009) [2023-12-26 23:11:10,013][105620] Updated weights for policy 1, policy_version 1073658 (0.0011) [2023-12-26 23:11:10,063][105692] Updated weights for policy 0, policy_version 1072576 (0.0009) [2023-12-26 23:11:10,772][105620] Updated weights for policy 1, policy_version 1073668 (0.0010) [2023-12-26 23:11:10,821][105692] Updated weights for policy 0, policy_version 1072586 (0.0006) [2023-12-26 23:11:10,833][105620] Updated weights for policy 1, policy_version 1073678 (0.0008) [2023-12-26 23:11:10,877][105692] Updated weights for policy 0, policy_version 1072596 (0.0005) [2023-12-26 23:11:10,886][105620] Updated weights for policy 1, policy_version 1073688 (0.0009) [2023-12-26 23:11:10,935][105692] Updated weights for policy 0, policy_version 1072606 (0.0007) [2023-12-26 23:11:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 549527552. Throughput: 0: 9786.4, 1: 9769.9. Samples: 549530292. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:11:11,063][104569] Avg episode reward: [(0, '8569.296'), (1, '8571.235')] [2023-12-26 23:11:11,636][105692] Updated weights for policy 0, policy_version 1072616 (0.0007) [2023-12-26 23:11:11,697][105692] Updated weights for policy 0, policy_version 1072626 (0.0009) [2023-12-26 23:11:11,704][105620] Updated weights for policy 1, policy_version 1073698 (0.0005) [2023-12-26 23:11:11,762][105692] Updated weights for policy 0, policy_version 1072636 (0.0007) [2023-12-26 23:11:11,769][105620] Updated weights for policy 1, policy_version 1073708 (0.0007) [2023-12-26 23:11:11,827][105620] Updated weights for policy 1, policy_version 1073718 (0.0006) [2023-12-26 23:11:11,885][105620] Updated weights for policy 1, policy_version 1073728 (0.0006) [2023-12-26 23:11:12,470][105692] Updated weights for policy 0, policy_version 1072646 (0.0009) [2023-12-26 23:11:12,517][105692] Updated weights for policy 0, policy_version 1072656 (0.0008) [2023-12-26 23:11:12,573][105692] Updated weights for policy 0, policy_version 1072666 (0.0006) [2023-12-26 23:11:12,584][105620] Updated weights for policy 1, policy_version 1073738 (0.0011) [2023-12-26 23:11:12,644][105620] Updated weights for policy 1, policy_version 1073748 (0.0011) [2023-12-26 23:11:12,687][105620] Updated weights for policy 1, policy_version 1073758 (0.0010) [2023-12-26 23:11:13,261][105692] Updated weights for policy 0, policy_version 1072676 (0.0006) [2023-12-26 23:11:13,312][105692] Updated weights for policy 0, policy_version 1072686 (0.0008) [2023-12-26 23:11:13,375][105692] Updated weights for policy 0, policy_version 1072696 (0.0007) [2023-12-26 23:11:13,378][105620] Updated weights for policy 1, policy_version 1073768 (0.0008) [2023-12-26 23:11:13,435][105620] Updated weights for policy 1, policy_version 1073778 (0.0005) [2023-12-26 23:11:13,504][105620] Updated weights for policy 1, policy_version 1073788 (0.0008) [2023-12-26 23:11:14,023][105692] Updated weights for policy 0, policy_version 1072706 (0.0008) [2023-12-26 23:11:14,055][105620] Updated weights for policy 1, policy_version 1073798 (0.0007) [2023-12-26 23:11:14,071][105692] Updated weights for policy 0, policy_version 1072716 (0.0008) [2023-12-26 23:11:14,106][105620] Updated weights for policy 1, policy_version 1073808 (0.0006) [2023-12-26 23:11:14,118][105692] Updated weights for policy 0, policy_version 1072726 (0.0008) [2023-12-26 23:11:14,156][105620] Updated weights for policy 1, policy_version 1073818 (0.0008) [2023-12-26 23:11:14,178][105692] Updated weights for policy 0, policy_version 1072736 (0.0007) [2023-12-26 23:11:14,800][105620] Updated weights for policy 1, policy_version 1073828 (0.0008) [2023-12-26 23:11:14,831][105692] Updated weights for policy 0, policy_version 1072746 (0.0007) [2023-12-26 23:11:14,863][105620] Updated weights for policy 1, policy_version 1073838 (0.0010) [2023-12-26 23:11:14,894][105692] Updated weights for policy 0, policy_version 1072756 (0.0007) [2023-12-26 23:11:14,923][105620] Updated weights for policy 1, policy_version 1073848 (0.0010) [2023-12-26 23:11:14,954][105692] Updated weights for policy 0, policy_version 1072766 (0.0006) [2023-12-26 23:11:15,563][105692] Updated weights for policy 0, policy_version 1072776 (0.0007) [2023-12-26 23:11:15,629][105692] Updated weights for policy 0, policy_version 1072786 (0.0005) [2023-12-26 23:11:15,632][105620] Updated weights for policy 1, policy_version 1073858 (0.0009) [2023-12-26 23:11:15,689][105692] Updated weights for policy 0, policy_version 1072796 (0.0006) [2023-12-26 23:11:15,694][105620] Updated weights for policy 1, policy_version 1073868 (0.0005) [2023-12-26 23:11:15,762][105620] Updated weights for policy 1, policy_version 1073878 (0.0010) [2023-12-26 23:11:15,817][105620] Updated weights for policy 1, policy_version 1073888 (0.0010) [2023-12-26 23:11:16,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19660.7, 300 sec: 19605.2). Total num frames: 549625856. Throughput: 0: 9781.4, 1: 9821.0. Samples: 549590252. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:11:16,063][104569] Avg episode reward: [(0, '8999.653'), (1, '8834.918')] [2023-12-26 23:11:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001073888_274948096.pth... [2023-12-26 23:11:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001072800_274677760.pth... [2023-12-26 23:11:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001072736_274653184.pth [2023-12-26 23:11:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001071648_274382848.pth [2023-12-26 23:11:16,212][105692] Updated weights for policy 0, policy_version 1072806 (0.0008) [2023-12-26 23:11:16,258][105692] Updated weights for policy 0, policy_version 1072817 (0.0008) [2023-12-26 23:11:16,306][105692] Updated weights for policy 0, policy_version 1072827 (0.0005) [2023-12-26 23:11:16,397][105620] Updated weights for policy 1, policy_version 1073898 (0.0011) [2023-12-26 23:11:16,458][105620] Updated weights for policy 1, policy_version 1073909 (0.0008) [2023-12-26 23:11:16,518][105620] Updated weights for policy 1, policy_version 1073919 (0.0010) [2023-12-26 23:11:16,947][105692] Updated weights for policy 0, policy_version 1072837 (0.0006) [2023-12-26 23:11:17,012][105692] Updated weights for policy 0, policy_version 1072847 (0.0005) [2023-12-26 23:11:17,071][105692] Updated weights for policy 0, policy_version 1072857 (0.0008) [2023-12-26 23:11:17,202][105620] Updated weights for policy 1, policy_version 1073929 (0.0011) [2023-12-26 23:11:17,270][105620] Updated weights for policy 1, policy_version 1073939 (0.0007) [2023-12-26 23:11:17,336][105620] Updated weights for policy 1, policy_version 1073949 (0.0007) [2023-12-26 23:11:17,693][105692] Updated weights for policy 0, policy_version 1072867 (0.0008) [2023-12-26 23:11:17,747][105692] Updated weights for policy 0, policy_version 1072877 (0.0005) [2023-12-26 23:11:17,800][105692] Updated weights for policy 0, policy_version 1072887 (0.0007) [2023-12-26 23:11:18,001][105620] Updated weights for policy 1, policy_version 1073959 (0.0010) [2023-12-26 23:11:18,067][105620] Updated weights for policy 1, policy_version 1073969 (0.0008) [2023-12-26 23:11:18,132][105620] Updated weights for policy 1, policy_version 1073979 (0.0011) [2023-12-26 23:11:18,458][105692] Updated weights for policy 0, policy_version 1072897 (0.0009) [2023-12-26 23:11:18,524][105692] Updated weights for policy 0, policy_version 1072907 (0.0011) [2023-12-26 23:11:18,584][105692] Updated weights for policy 0, policy_version 1072917 (0.0011) [2023-12-26 23:11:18,650][105692] Updated weights for policy 0, policy_version 1072927 (0.0011) [2023-12-26 23:11:18,862][105620] Updated weights for policy 1, policy_version 1073989 (0.0011) [2023-12-26 23:11:18,911][105620] Updated weights for policy 1, policy_version 1073999 (0.0010) [2023-12-26 23:11:18,964][105620] Updated weights for policy 1, policy_version 1074009 (0.0010) [2023-12-26 23:11:19,382][105692] Updated weights for policy 0, policy_version 1072937 (0.0008) [2023-12-26 23:11:19,436][105692] Updated weights for policy 0, policy_version 1072947 (0.0009) [2023-12-26 23:11:19,502][105692] Updated weights for policy 0, policy_version 1072957 (0.0011) [2023-12-26 23:11:19,653][105620] Updated weights for policy 1, policy_version 1074019 (0.0009) [2023-12-26 23:11:19,717][105620] Updated weights for policy 1, policy_version 1074029 (0.0010) [2023-12-26 23:11:19,770][105620] Updated weights for policy 1, policy_version 1074039 (0.0010) [2023-12-26 23:11:20,280][105692] Updated weights for policy 0, policy_version 1072967 (0.0011) [2023-12-26 23:11:20,336][105692] Updated weights for policy 0, policy_version 1072977 (0.0011) [2023-12-26 23:11:20,403][105692] Updated weights for policy 0, policy_version 1072987 (0.0011) [2023-12-26 23:11:20,482][105620] Updated weights for policy 1, policy_version 1074049 (0.0008) [2023-12-26 23:11:20,543][105620] Updated weights for policy 1, policy_version 1074059 (0.0006) [2023-12-26 23:11:20,616][105620] Updated weights for policy 1, policy_version 1074069 (0.0008) [2023-12-26 23:11:20,684][105620] Updated weights for policy 1, policy_version 1074079 (0.0010) [2023-12-26 23:11:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 549724160. Throughput: 0: 9872.6, 1: 9897.6. Samples: 549715920. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:11:21,063][104569] Avg episode reward: [(0, '8912.551'), (1, '9089.226')] [2023-12-26 23:11:21,160][105692] Updated weights for policy 0, policy_version 1072997 (0.0009) [2023-12-26 23:11:21,233][105692] Updated weights for policy 0, policy_version 1073007 (0.0006) [2023-12-26 23:11:21,294][105692] Updated weights for policy 0, policy_version 1073017 (0.0010) [2023-12-26 23:11:21,352][105620] Updated weights for policy 1, policy_version 1074089 (0.0008) [2023-12-26 23:11:21,424][105620] Updated weights for policy 1, policy_version 1074099 (0.0009) [2023-12-26 23:11:21,474][105620] Updated weights for policy 1, policy_version 1074109 (0.0005) [2023-12-26 23:11:21,996][105692] Updated weights for policy 0, policy_version 1073027 (0.0009) [2023-12-26 23:11:22,045][105692] Updated weights for policy 0, policy_version 1073037 (0.0006) [2023-12-26 23:11:22,106][105692] Updated weights for policy 0, policy_version 1073047 (0.0005) [2023-12-26 23:11:22,184][105620] Updated weights for policy 1, policy_version 1074119 (0.0006) [2023-12-26 23:11:22,246][105620] Updated weights for policy 1, policy_version 1074129 (0.0009) [2023-12-26 23:11:22,310][105620] Updated weights for policy 1, policy_version 1074139 (0.0010) [2023-12-26 23:11:22,789][105692] Updated weights for policy 0, policy_version 1073057 (0.0007) [2023-12-26 23:11:22,850][105692] Updated weights for policy 0, policy_version 1073067 (0.0007) [2023-12-26 23:11:22,906][105692] Updated weights for policy 0, policy_version 1073077 (0.0009) [2023-12-26 23:11:22,953][105692] Updated weights for policy 0, policy_version 1073087 (0.0008) [2023-12-26 23:11:23,166][105620] Updated weights for policy 1, policy_version 1074149 (0.0010) [2023-12-26 23:11:23,225][105620] Updated weights for policy 1, policy_version 1074159 (0.0009) [2023-12-26 23:11:23,281][105620] Updated weights for policy 1, policy_version 1074169 (0.0009) [2023-12-26 23:11:23,637][105692] Updated weights for policy 0, policy_version 1073097 (0.0008) [2023-12-26 23:11:23,684][105692] Updated weights for policy 0, policy_version 1073107 (0.0009) [2023-12-26 23:11:23,735][105692] Updated weights for policy 0, policy_version 1073117 (0.0010) [2023-12-26 23:11:23,938][105620] Updated weights for policy 1, policy_version 1074179 (0.0009) [2023-12-26 23:11:23,982][105620] Updated weights for policy 1, policy_version 1074189 (0.0008) [2023-12-26 23:11:24,038][105620] Updated weights for policy 1, policy_version 1074199 (0.0005) [2023-12-26 23:11:24,579][105620] Updated weights for policy 1, policy_version 1074209 (0.0005) [2023-12-26 23:11:24,636][105620] Updated weights for policy 1, policy_version 1074219 (0.0008) [2023-12-26 23:11:24,651][105692] Updated weights for policy 0, policy_version 1073128 (0.0008) [2023-12-26 23:11:24,689][105620] Updated weights for policy 1, policy_version 1074229 (0.0005) [2023-12-26 23:11:24,703][105692] Updated weights for policy 0, policy_version 1073138 (0.0009) [2023-12-26 23:11:24,740][105620] Updated weights for policy 1, policy_version 1074239 (0.0005) [2023-12-26 23:11:24,755][105692] Updated weights for policy 0, policy_version 1073148 (0.0009) [2023-12-26 23:11:25,270][105620] Updated weights for policy 1, policy_version 1074249 (0.0009) [2023-12-26 23:11:25,329][105620] Updated weights for policy 1, policy_version 1074259 (0.0006) [2023-12-26 23:11:25,386][105620] Updated weights for policy 1, policy_version 1074269 (0.0005) [2023-12-26 23:11:25,664][105692] Updated weights for policy 0, policy_version 1073158 (0.0010) [2023-12-26 23:11:25,716][105692] Updated weights for policy 0, policy_version 1073168 (0.0009) [2023-12-26 23:11:25,780][105692] Updated weights for policy 0, policy_version 1073178 (0.0009) [2023-12-26 23:11:25,933][105620] Updated weights for policy 1, policy_version 1074279 (0.0006) [2023-12-26 23:11:25,984][105620] Updated weights for policy 1, policy_version 1074289 (0.0008) [2023-12-26 23:11:26,035][105620] Updated weights for policy 1, policy_version 1074299 (0.0005) [2023-12-26 23:11:26,062][104569] Fps is (10 sec: 20480.7, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 549830656. Throughput: 0: 9810.6, 1: 9927.3. Samples: 549834040. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:11:26,062][104569] Avg episode reward: [(0, '8563.949'), (1, '9176.708')] [2023-12-26 23:11:26,619][105692] Updated weights for policy 0, policy_version 1073188 (0.0009) [2023-12-26 23:11:26,685][105692] Updated weights for policy 0, policy_version 1073198 (0.0008) [2023-12-26 23:11:26,695][105620] Updated weights for policy 1, policy_version 1074309 (0.0008) [2023-12-26 23:11:26,742][105692] Updated weights for policy 0, policy_version 1073208 (0.0006) [2023-12-26 23:11:26,756][105620] Updated weights for policy 1, policy_version 1074319 (0.0008) [2023-12-26 23:11:26,824][105620] Updated weights for policy 1, policy_version 1074329 (0.0008) [2023-12-26 23:11:27,474][105692] Updated weights for policy 0, policy_version 1073218 (0.0008) [2023-12-26 23:11:27,534][105692] Updated weights for policy 0, policy_version 1073228 (0.0008) [2023-12-26 23:11:27,566][105620] Updated weights for policy 1, policy_version 1074339 (0.0008) [2023-12-26 23:11:27,584][105692] Updated weights for policy 0, policy_version 1073238 (0.0008) [2023-12-26 23:11:27,614][105620] Updated weights for policy 1, policy_version 1074349 (0.0006) [2023-12-26 23:11:27,629][105692] Updated weights for policy 0, policy_version 1073248 (0.0006) [2023-12-26 23:11:27,665][105620] Updated weights for policy 1, policy_version 1074359 (0.0007) [2023-12-26 23:11:28,364][105692] Updated weights for policy 0, policy_version 1073258 (0.0009) [2023-12-26 23:11:28,414][105692] Updated weights for policy 0, policy_version 1073268 (0.0008) [2023-12-26 23:11:28,447][105620] Updated weights for policy 1, policy_version 1074369 (0.0009) [2023-12-26 23:11:28,469][105692] Updated weights for policy 0, policy_version 1073278 (0.0008) [2023-12-26 23:11:28,498][105620] Updated weights for policy 1, policy_version 1074379 (0.0007) [2023-12-26 23:11:28,544][105620] Updated weights for policy 1, policy_version 1074389 (0.0008) [2023-12-26 23:11:28,591][105620] Updated weights for policy 1, policy_version 1074399 (0.0009) [2023-12-26 23:11:29,256][105692] Updated weights for policy 0, policy_version 1073288 (0.0009) [2023-12-26 23:11:29,307][105692] Updated weights for policy 0, policy_version 1073298 (0.0008) [2023-12-26 23:11:29,322][105620] Updated weights for policy 1, policy_version 1074409 (0.0006) [2023-12-26 23:11:29,371][105692] Updated weights for policy 0, policy_version 1073308 (0.0008) [2023-12-26 23:11:29,388][105620] Updated weights for policy 1, policy_version 1074419 (0.0007) [2023-12-26 23:11:29,444][105620] Updated weights for policy 1, policy_version 1074429 (0.0006) [2023-12-26 23:11:30,130][105692] Updated weights for policy 0, policy_version 1073318 (0.0008) [2023-12-26 23:11:30,187][105620] Updated weights for policy 1, policy_version 1074439 (0.0007) [2023-12-26 23:11:30,194][105692] Updated weights for policy 0, policy_version 1073328 (0.0009) [2023-12-26 23:11:30,243][105620] Updated weights for policy 1, policy_version 1074449 (0.0006) [2023-12-26 23:11:30,249][105692] Updated weights for policy 0, policy_version 1073338 (0.0008) [2023-12-26 23:11:30,301][105620] Updated weights for policy 1, policy_version 1074459 (0.0007) [2023-12-26 23:11:30,985][105620] Updated weights for policy 1, policy_version 1074469 (0.0007) [2023-12-26 23:11:31,004][105692] Updated weights for policy 0, policy_version 1073348 (0.0007) [2023-12-26 23:11:31,044][105620] Updated weights for policy 1, policy_version 1074479 (0.0006) [2023-12-26 23:11:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 549912576. Throughput: 0: 9836.3, 1: 9885.3. Samples: 549890332. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:11:31,063][105692] Updated weights for policy 0, policy_version 1073358 (0.0006) [2023-12-26 23:11:31,063][104569] Avg episode reward: [(0, '8557.134'), (1, '8669.442')] [2023-12-26 23:11:31,103][105620] Updated weights for policy 1, policy_version 1074489 (0.0008) [2023-12-26 23:11:31,130][105692] Updated weights for policy 0, policy_version 1073368 (0.0007) [2023-12-26 23:11:31,149][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001074496_275103744.pth... [2023-12-26 23:11:31,152][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001073312_274800640.pth [2023-12-26 23:11:31,175][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001073376_274825216.pth... [2023-12-26 23:11:31,179][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001072224_274530304.pth [2023-12-26 23:11:31,803][105620] Updated weights for policy 1, policy_version 1074499 (0.0007) [2023-12-26 23:11:31,857][105620] Updated weights for policy 1, policy_version 1074509 (0.0010) [2023-12-26 23:11:31,890][105692] Updated weights for policy 0, policy_version 1073378 (0.0007) [2023-12-26 23:11:31,906][105620] Updated weights for policy 1, policy_version 1074519 (0.0010) [2023-12-26 23:11:31,946][105692] Updated weights for policy 0, policy_version 1073388 (0.0010) [2023-12-26 23:11:32,009][105692] Updated weights for policy 0, policy_version 1073398 (0.0011) [2023-12-26 23:11:32,065][105692] Updated weights for policy 0, policy_version 1073408 (0.0010) [2023-12-26 23:11:32,586][105620] Updated weights for policy 1, policy_version 1074529 (0.0010) [2023-12-26 23:11:32,636][105620] Updated weights for policy 1, policy_version 1074539 (0.0008) [2023-12-26 23:11:32,687][105620] Updated weights for policy 1, policy_version 1074549 (0.0005) [2023-12-26 23:11:32,739][105620] Updated weights for policy 1, policy_version 1074559 (0.0005) [2023-12-26 23:11:32,785][105692] Updated weights for policy 0, policy_version 1073418 (0.0010) [2023-12-26 23:11:32,836][105692] Updated weights for policy 0, policy_version 1073428 (0.0010) [2023-12-26 23:11:32,886][105692] Updated weights for policy 0, policy_version 1073438 (0.0009) [2023-12-26 23:11:33,287][105620] Updated weights for policy 1, policy_version 1074569 (0.0005) [2023-12-26 23:11:33,339][105620] Updated weights for policy 1, policy_version 1074579 (0.0005) [2023-12-26 23:11:33,397][105620] Updated weights for policy 1, policy_version 1074589 (0.0005) [2023-12-26 23:11:33,637][105692] Updated weights for policy 0, policy_version 1073448 (0.0010) [2023-12-26 23:11:33,682][105692] Updated weights for policy 0, policy_version 1073458 (0.0010) [2023-12-26 23:11:33,730][105692] Updated weights for policy 0, policy_version 1073468 (0.0010) [2023-12-26 23:11:33,913][105620] Updated weights for policy 1, policy_version 1074599 (0.0007) [2023-12-26 23:11:33,958][105620] Updated weights for policy 1, policy_version 1074609 (0.0007) [2023-12-26 23:11:34,013][105620] Updated weights for policy 1, policy_version 1074619 (0.0006) [2023-12-26 23:11:34,556][105692] Updated weights for policy 0, policy_version 1073478 (0.0010) [2023-12-26 23:11:34,618][105692] Updated weights for policy 0, policy_version 1073488 (0.0007) [2023-12-26 23:11:34,650][105620] Updated weights for policy 1, policy_version 1074629 (0.0007) [2023-12-26 23:11:34,678][105692] Updated weights for policy 0, policy_version 1073498 (0.0009) [2023-12-26 23:11:34,717][105620] Updated weights for policy 1, policy_version 1074639 (0.0007) [2023-12-26 23:11:34,791][105620] Updated weights for policy 1, policy_version 1074649 (0.0005) [2023-12-26 23:11:35,273][105692] Updated weights for policy 0, policy_version 1073508 (0.0007) [2023-12-26 23:11:35,316][105620] Updated weights for policy 1, policy_version 1074659 (0.0006) [2023-12-26 23:11:35,327][105692] Updated weights for policy 0, policy_version 1073518 (0.0007) [2023-12-26 23:11:35,380][105692] Updated weights for policy 0, policy_version 1073528 (0.0007) [2023-12-26 23:11:35,387][105620] Updated weights for policy 1, policy_version 1074669 (0.0005) [2023-12-26 23:11:35,444][105620] Updated weights for policy 1, policy_version 1074679 (0.0005) [2023-12-26 23:11:35,917][105692] Updated weights for policy 0, policy_version 1073538 (0.0007) [2023-12-26 23:11:35,977][105692] Updated weights for policy 0, policy_version 1073548 (0.0005) [2023-12-26 23:11:36,025][105692] Updated weights for policy 0, policy_version 1073558 (0.0005) [2023-12-26 23:11:36,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 550019072. Throughput: 0: 9617.1, 1: 10047.4. Samples: 550010096. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:11:36,062][104569] Avg episode reward: [(0, '8644.843'), (1, '4657.770')] [2023-12-26 23:11:36,081][105692] Updated weights for policy 0, policy_version 1073568 (0.0005) [2023-12-26 23:11:36,136][105620] Updated weights for policy 1, policy_version 1074689 (0.0010) [2023-12-26 23:11:36,200][105620] Updated weights for policy 1, policy_version 1074699 (0.0010) [2023-12-26 23:11:36,253][105620] Updated weights for policy 1, policy_version 1074709 (0.0010) [2023-12-26 23:11:36,308][105620] Updated weights for policy 1, policy_version 1074719 (0.0010) [2023-12-26 23:11:36,803][105692] Updated weights for policy 0, policy_version 1073578 (0.0010) [2023-12-26 23:11:36,857][105692] Updated weights for policy 0, policy_version 1073588 (0.0010) [2023-12-26 23:11:36,919][105692] Updated weights for policy 0, policy_version 1073598 (0.0010) [2023-12-26 23:11:36,987][105620] Updated weights for policy 1, policy_version 1074729 (0.0010) [2023-12-26 23:11:37,052][105620] Updated weights for policy 1, policy_version 1074739 (0.0010) [2023-12-26 23:11:37,118][105620] Updated weights for policy 1, policy_version 1074749 (0.0011) [2023-12-26 23:11:37,680][105692] Updated weights for policy 0, policy_version 1073608 (0.0008) [2023-12-26 23:11:37,734][105692] Updated weights for policy 0, policy_version 1073618 (0.0007) [2023-12-26 23:11:37,792][105692] Updated weights for policy 0, policy_version 1073628 (0.0009) [2023-12-26 23:11:37,858][105620] Updated weights for policy 1, policy_version 1074759 (0.0007) [2023-12-26 23:11:37,906][105620] Updated weights for policy 1, policy_version 1074769 (0.0005) [2023-12-26 23:11:37,960][105620] Updated weights for policy 1, policy_version 1074779 (0.0005) [2023-12-26 23:11:38,476][105692] Updated weights for policy 0, policy_version 1073638 (0.0008) [2023-12-26 23:11:38,532][105692] Updated weights for policy 0, policy_version 1073648 (0.0008) [2023-12-26 23:11:38,581][105692] Updated weights for policy 0, policy_version 1073658 (0.0008) [2023-12-26 23:11:38,660][105620] Updated weights for policy 1, policy_version 1074789 (0.0008) [2023-12-26 23:11:38,709][105620] Updated weights for policy 1, policy_version 1074799 (0.0010) [2023-12-26 23:11:38,758][105620] Updated weights for policy 1, policy_version 1074809 (0.0010) [2023-12-26 23:11:39,338][105692] Updated weights for policy 0, policy_version 1073668 (0.0009) [2023-12-26 23:11:39,411][105692] Updated weights for policy 0, policy_version 1073679 (0.0009) [2023-12-26 23:11:39,472][105692] Updated weights for policy 0, policy_version 1073689 (0.0006) [2023-12-26 23:11:39,534][105620] Updated weights for policy 1, policy_version 1074819 (0.0011) [2023-12-26 23:11:39,580][105620] Updated weights for policy 1, policy_version 1074829 (0.0010) [2023-12-26 23:11:39,635][105620] Updated weights for policy 1, policy_version 1074839 (0.0011) [2023-12-26 23:11:40,129][105692] Updated weights for policy 0, policy_version 1073699 (0.0007) [2023-12-26 23:11:40,181][105692] Updated weights for policy 0, policy_version 1073709 (0.0009) [2023-12-26 23:11:40,229][105692] Updated weights for policy 0, policy_version 1073719 (0.0008) [2023-12-26 23:11:40,386][105620] Updated weights for policy 1, policy_version 1074849 (0.0011) [2023-12-26 23:11:40,434][105586] KL-divergence is very high: 100.7272 [2023-12-26 23:11:40,449][105620] Updated weights for policy 1, policy_version 1074859 (0.0010) [2023-12-26 23:11:40,487][105586] KL-divergence is very high: 123.1049 [2023-12-26 23:11:40,509][105620] Updated weights for policy 1, policy_version 1074869 (0.0009) [2023-12-26 23:11:40,535][105586] KL-divergence is very high: 106.4758 [2023-12-26 23:11:40,570][105620] Updated weights for policy 1, policy_version 1074879 (0.0006) [2023-12-26 23:11:40,959][105692] Updated weights for policy 0, policy_version 1073729 (0.0009) [2023-12-26 23:11:41,010][105692] Updated weights for policy 0, policy_version 1073739 (0.0008) [2023-12-26 23:11:41,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 550117376. Throughput: 0: 9661.9, 1: 10071.9. Samples: 550130908. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:11:41,062][104569] Avg episode reward: [(0, '8470.162'), (1, '4152.616')] [2023-12-26 23:11:41,074][105692] Updated weights for policy 0, policy_version 1073749 (0.0007) [2023-12-26 23:11:41,140][105692] Updated weights for policy 0, policy_version 1073759 (0.0006) [2023-12-26 23:11:41,254][105620] Updated weights for policy 1, policy_version 1074889 (0.0009) [2023-12-26 23:11:41,318][105620] Updated weights for policy 1, policy_version 1074899 (0.0009) [2023-12-26 23:11:41,384][105620] Updated weights for policy 1, policy_version 1074909 (0.0008) [2023-12-26 23:11:41,847][105692] Updated weights for policy 0, policy_version 1073769 (0.0009) [2023-12-26 23:11:41,906][105692] Updated weights for policy 0, policy_version 1073779 (0.0008) [2023-12-26 23:11:41,966][105692] Updated weights for policy 0, policy_version 1073789 (0.0008) [2023-12-26 23:11:42,205][105620] Updated weights for policy 1, policy_version 1074919 (0.0009) [2023-12-26 23:11:42,264][105620] Updated weights for policy 1, policy_version 1074929 (0.0009) [2023-12-26 23:11:42,323][105620] Updated weights for policy 1, policy_version 1074939 (0.0008) [2023-12-26 23:11:42,606][105692] Updated weights for policy 0, policy_version 1073799 (0.0006) [2023-12-26 23:11:42,673][105692] Updated weights for policy 0, policy_version 1073809 (0.0008) [2023-12-26 23:11:42,728][105692] Updated weights for policy 0, policy_version 1073819 (0.0008) [2023-12-26 23:11:43,226][105620] Updated weights for policy 1, policy_version 1074949 (0.0009) [2023-12-26 23:11:43,261][105692] Updated weights for policy 0, policy_version 1073829 (0.0005) [2023-12-26 23:11:43,283][105620] Updated weights for policy 1, policy_version 1074959 (0.0009) [2023-12-26 23:11:43,324][105692] Updated weights for policy 0, policy_version 1073839 (0.0005) [2023-12-26 23:11:43,343][105620] Updated weights for policy 1, policy_version 1074969 (0.0007) [2023-12-26 23:11:43,372][105692] Updated weights for policy 0, policy_version 1073849 (0.0005) [2023-12-26 23:11:44,018][105692] Updated weights for policy 0, policy_version 1073859 (0.0007) [2023-12-26 23:11:44,078][105692] Updated weights for policy 0, policy_version 1073869 (0.0006) [2023-12-26 23:11:44,118][105620] Updated weights for policy 1, policy_version 1074979 (0.0009) [2023-12-26 23:11:44,143][105692] Updated weights for policy 0, policy_version 1073879 (0.0008) [2023-12-26 23:11:44,181][105620] Updated weights for policy 1, policy_version 1074989 (0.0008) [2023-12-26 23:11:44,236][105620] Updated weights for policy 1, policy_version 1074999 (0.0009) [2023-12-26 23:11:44,860][105692] Updated weights for policy 0, policy_version 1073889 (0.0006) [2023-12-26 23:11:44,896][105620] Updated weights for policy 1, policy_version 1075009 (0.0008) [2023-12-26 23:11:44,929][105692] Updated weights for policy 0, policy_version 1073899 (0.0008) [2023-12-26 23:11:44,961][105620] Updated weights for policy 1, policy_version 1075019 (0.0006) [2023-12-26 23:11:44,991][105692] Updated weights for policy 0, policy_version 1073909 (0.0008) [2023-12-26 23:11:45,022][105620] Updated weights for policy 1, policy_version 1075029 (0.0006) [2023-12-26 23:11:45,050][105692] Updated weights for policy 0, policy_version 1073919 (0.0008) [2023-12-26 23:11:45,081][105620] Updated weights for policy 1, policy_version 1075039 (0.0009) [2023-12-26 23:11:45,781][105692] Updated weights for policy 0, policy_version 1073929 (0.0008) [2023-12-26 23:11:45,802][105620] Updated weights for policy 1, policy_version 1075049 (0.0009) [2023-12-26 23:11:45,825][105692] Updated weights for policy 0, policy_version 1073939 (0.0007) [2023-12-26 23:11:45,861][105620] Updated weights for policy 1, policy_version 1075059 (0.0010) [2023-12-26 23:11:45,883][105692] Updated weights for policy 0, policy_version 1073949 (0.0008) [2023-12-26 23:11:45,923][105620] Updated weights for policy 1, policy_version 1075069 (0.0007) [2023-12-26 23:11:46,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 550223872. Throughput: 0: 9702.5, 1: 9947.4. Samples: 550188248. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:11:46,062][104569] Avg episode reward: [(0, '8637.010'), (1, '6703.980')] [2023-12-26 23:11:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001073952_274972672.pth... [2023-12-26 23:11:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001075072_275251200.pth... [2023-12-26 23:11:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001072800_274677760.pth [2023-12-26 23:11:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001073888_274948096.pth [2023-12-26 23:11:46,618][105620] Updated weights for policy 1, policy_version 1075079 (0.0008) [2023-12-26 23:11:46,633][105692] Updated weights for policy 0, policy_version 1073959 (0.0009) [2023-12-26 23:11:46,667][105620] Updated weights for policy 1, policy_version 1075089 (0.0006) [2023-12-26 23:11:46,686][105692] Updated weights for policy 0, policy_version 1073969 (0.0007) [2023-12-26 23:11:46,728][105620] Updated weights for policy 1, policy_version 1075099 (0.0006) [2023-12-26 23:11:46,742][105692] Updated weights for policy 0, policy_version 1073979 (0.0008) [2023-12-26 23:11:47,424][105620] Updated weights for policy 1, policy_version 1075109 (0.0008) [2023-12-26 23:11:47,441][105692] Updated weights for policy 0, policy_version 1073989 (0.0008) [2023-12-26 23:11:47,483][105620] Updated weights for policy 1, policy_version 1075119 (0.0006) [2023-12-26 23:11:47,496][105692] Updated weights for policy 0, policy_version 1073999 (0.0008) [2023-12-26 23:11:47,546][105620] Updated weights for policy 1, policy_version 1075129 (0.0007) [2023-12-26 23:11:47,561][105692] Updated weights for policy 0, policy_version 1074009 (0.0006) [2023-12-26 23:11:48,138][105620] Updated weights for policy 1, policy_version 1075139 (0.0007) [2023-12-26 23:11:48,192][105620] Updated weights for policy 1, policy_version 1075149 (0.0007) [2023-12-26 23:11:48,255][105620] Updated weights for policy 1, policy_version 1075159 (0.0005) [2023-12-26 23:11:48,341][105692] Updated weights for policy 0, policy_version 1074019 (0.0009) [2023-12-26 23:11:48,406][105692] Updated weights for policy 0, policy_version 1074029 (0.0008) [2023-12-26 23:11:48,475][105692] Updated weights for policy 0, policy_version 1074039 (0.0009) [2023-12-26 23:11:48,942][105620] Updated weights for policy 1, policy_version 1075169 (0.0006) [2023-12-26 23:11:49,001][105620] Updated weights for policy 1, policy_version 1075179 (0.0011) [2023-12-26 23:11:49,065][105620] Updated weights for policy 1, policy_version 1075189 (0.0011) [2023-12-26 23:11:49,128][105620] Updated weights for policy 1, policy_version 1075199 (0.0010) [2023-12-26 23:11:49,233][105692] Updated weights for policy 0, policy_version 1074049 (0.0010) [2023-12-26 23:11:49,296][105692] Updated weights for policy 0, policy_version 1074059 (0.0007) [2023-12-26 23:11:49,363][105692] Updated weights for policy 0, policy_version 1074069 (0.0012) [2023-12-26 23:11:49,430][105692] Updated weights for policy 0, policy_version 1074079 (0.0008) [2023-12-26 23:11:49,793][105620] Updated weights for policy 1, policy_version 1075209 (0.0006) [2023-12-26 23:11:49,861][105620] Updated weights for policy 1, policy_version 1075219 (0.0008) [2023-12-26 23:11:49,926][105620] Updated weights for policy 1, policy_version 1075229 (0.0008) [2023-12-26 23:11:50,165][105692] Updated weights for policy 0, policy_version 1074089 (0.0006) [2023-12-26 23:11:50,221][105692] Updated weights for policy 0, policy_version 1074099 (0.0008) [2023-12-26 23:11:50,271][105692] Updated weights for policy 0, policy_version 1074109 (0.0007) [2023-12-26 23:11:50,642][105620] Updated weights for policy 1, policy_version 1075239 (0.0009) [2023-12-26 23:11:50,708][105620] Updated weights for policy 1, policy_version 1075249 (0.0009) [2023-12-26 23:11:50,772][105620] Updated weights for policy 1, policy_version 1075259 (0.0008) [2023-12-26 23:11:50,926][105692] Updated weights for policy 0, policy_version 1074119 (0.0007) [2023-12-26 23:11:50,996][105692] Updated weights for policy 0, policy_version 1074129 (0.0006) [2023-12-26 23:11:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 550313984. Throughput: 0: 9691.9, 1: 10027.2. Samples: 550305184. Policy #0 lag: (min: 4.0, avg: 4.2, max: 13.0) [2023-12-26 23:11:51,062][104569] Avg episode reward: [(0, '8460.562'), (1, '9002.981')] [2023-12-26 23:11:51,063][105692] Updated weights for policy 0, policy_version 1074139 (0.0009) [2023-12-26 23:11:51,535][105620] Updated weights for policy 1, policy_version 1075269 (0.0006) [2023-12-26 23:11:51,592][105620] Updated weights for policy 1, policy_version 1075279 (0.0009) [2023-12-26 23:11:51,652][105620] Updated weights for policy 1, policy_version 1075289 (0.0011) [2023-12-26 23:11:51,800][105692] Updated weights for policy 0, policy_version 1074149 (0.0010) [2023-12-26 23:11:51,860][105692] Updated weights for policy 0, policy_version 1074159 (0.0011) [2023-12-26 23:11:51,916][105692] Updated weights for policy 0, policy_version 1074169 (0.0011) [2023-12-26 23:11:52,395][105620] Updated weights for policy 1, policy_version 1075299 (0.0010) [2023-12-26 23:11:52,451][105620] Updated weights for policy 1, policy_version 1075309 (0.0008) [2023-12-26 23:11:52,518][105620] Updated weights for policy 1, policy_version 1075319 (0.0005) [2023-12-26 23:11:52,671][105692] Updated weights for policy 0, policy_version 1074179 (0.0009) [2023-12-26 23:11:52,731][105692] Updated weights for policy 0, policy_version 1074189 (0.0005) [2023-12-26 23:11:52,783][105692] Updated weights for policy 0, policy_version 1074199 (0.0005) [2023-12-26 23:11:53,278][105620] Updated weights for policy 1, policy_version 1075329 (0.0006) [2023-12-26 23:11:53,297][105692] Updated weights for policy 0, policy_version 1074209 (0.0005) [2023-12-26 23:11:53,326][105620] Updated weights for policy 1, policy_version 1075339 (0.0005) [2023-12-26 23:11:53,365][105692] Updated weights for policy 0, policy_version 1074219 (0.0005) [2023-12-26 23:11:53,380][105620] Updated weights for policy 1, policy_version 1075349 (0.0005) [2023-12-26 23:11:53,430][105620] Updated weights for policy 1, policy_version 1075359 (0.0005) [2023-12-26 23:11:53,433][105692] Updated weights for policy 0, policy_version 1074229 (0.0006) [2023-12-26 23:11:53,494][105692] Updated weights for policy 0, policy_version 1074239 (0.0009) [2023-12-26 23:11:54,019][105692] Updated weights for policy 0, policy_version 1074249 (0.0006) [2023-12-26 23:11:54,065][105692] Updated weights for policy 0, policy_version 1074259 (0.0005) [2023-12-26 23:11:54,082][105585] KL-divergence is very high: 105.3677 [2023-12-26 23:11:54,084][105620] Updated weights for policy 1, policy_version 1075369 (0.0007) [2023-12-26 23:11:54,135][105692] Updated weights for policy 0, policy_version 1074269 (0.0009) [2023-12-26 23:11:54,140][105620] Updated weights for policy 1, policy_version 1075379 (0.0005) [2023-12-26 23:11:54,196][105620] Updated weights for policy 1, policy_version 1075389 (0.0005) [2023-12-26 23:11:54,677][105692] Updated weights for policy 0, policy_version 1074279 (0.0006) [2023-12-26 23:11:54,733][105692] Updated weights for policy 0, policy_version 1074289 (0.0005) [2023-12-26 23:11:54,777][105620] Updated weights for policy 1, policy_version 1075399 (0.0007) [2023-12-26 23:11:54,782][105692] Updated weights for policy 0, policy_version 1074299 (0.0007) [2023-12-26 23:11:54,840][105620] Updated weights for policy 1, policy_version 1075409 (0.0008) [2023-12-26 23:11:54,905][105620] Updated weights for policy 1, policy_version 1075419 (0.0009) [2023-12-26 23:11:55,451][105692] Updated weights for policy 0, policy_version 1074309 (0.0008) [2023-12-26 23:11:55,511][105692] Updated weights for policy 0, policy_version 1074319 (0.0009) [2023-12-26 23:11:55,568][105692] Updated weights for policy 0, policy_version 1074329 (0.0010) [2023-12-26 23:11:55,569][105620] Updated weights for policy 1, policy_version 1075429 (0.0007) [2023-12-26 23:11:55,625][105620] Updated weights for policy 1, policy_version 1075439 (0.0005) [2023-12-26 23:11:55,675][105620] Updated weights for policy 1, policy_version 1075449 (0.0007) [2023-12-26 23:11:56,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 550420480. Throughput: 0: 9898.0, 1: 10071.9. Samples: 550428936. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:11:56,063][104569] Avg episode reward: [(0, '8728.755'), (1, '9258.148')] [2023-12-26 23:11:56,243][105620] Updated weights for policy 1, policy_version 1075459 (0.0009) [2023-12-26 23:11:56,291][105620] Updated weights for policy 1, policy_version 1075469 (0.0010) [2023-12-26 23:11:56,304][105692] Updated weights for policy 0, policy_version 1074339 (0.0008) [2023-12-26 23:11:56,337][105620] Updated weights for policy 1, policy_version 1075479 (0.0010) [2023-12-26 23:11:56,356][105692] Updated weights for policy 0, policy_version 1074349 (0.0005) [2023-12-26 23:11:56,409][105692] Updated weights for policy 0, policy_version 1074359 (0.0007) [2023-12-26 23:11:57,083][105620] Updated weights for policy 1, policy_version 1075489 (0.0010) [2023-12-26 23:11:57,107][105692] Updated weights for policy 0, policy_version 1074369 (0.0008) [2023-12-26 23:11:57,151][105620] Updated weights for policy 1, policy_version 1075499 (0.0009) [2023-12-26 23:11:57,168][105692] Updated weights for policy 0, policy_version 1074379 (0.0006) [2023-12-26 23:11:57,217][105692] Updated weights for policy 0, policy_version 1074389 (0.0005) [2023-12-26 23:11:57,219][105620] Updated weights for policy 1, policy_version 1075509 (0.0008) [2023-12-26 23:11:57,280][105692] Updated weights for policy 0, policy_version 1074399 (0.0005) [2023-12-26 23:11:57,284][105620] Updated weights for policy 1, policy_version 1075519 (0.0008) [2023-12-26 23:11:57,970][105620] Updated weights for policy 1, policy_version 1075529 (0.0008) [2023-12-26 23:11:57,980][105692] Updated weights for policy 0, policy_version 1074409 (0.0006) [2023-12-26 23:11:58,023][105620] Updated weights for policy 1, policy_version 1075539 (0.0006) [2023-12-26 23:11:58,037][105692] Updated weights for policy 0, policy_version 1074419 (0.0006) [2023-12-26 23:11:58,071][105620] Updated weights for policy 1, policy_version 1075549 (0.0007) [2023-12-26 23:11:58,089][105692] Updated weights for policy 0, policy_version 1074429 (0.0006) [2023-12-26 23:11:58,866][105620] Updated weights for policy 1, policy_version 1075559 (0.0009) [2023-12-26 23:11:58,913][105692] Updated weights for policy 0, policy_version 1074439 (0.0008) [2023-12-26 23:11:58,936][105620] Updated weights for policy 1, policy_version 1075569 (0.0008) [2023-12-26 23:11:58,979][105692] Updated weights for policy 0, policy_version 1074449 (0.0007) [2023-12-26 23:11:59,002][105620] Updated weights for policy 1, policy_version 1075579 (0.0010) [2023-12-26 23:11:59,045][105692] Updated weights for policy 0, policy_version 1074459 (0.0006) [2023-12-26 23:11:59,790][105620] Updated weights for policy 1, policy_version 1075589 (0.0008) [2023-12-26 23:11:59,797][105692] Updated weights for policy 0, policy_version 1074469 (0.0008) [2023-12-26 23:11:59,856][105620] Updated weights for policy 1, policy_version 1075599 (0.0009) [2023-12-26 23:11:59,863][105692] Updated weights for policy 0, policy_version 1074479 (0.0006) [2023-12-26 23:11:59,904][105620] Updated weights for policy 1, policy_version 1075609 (0.0008) [2023-12-26 23:11:59,924][105692] Updated weights for policy 0, policy_version 1074489 (0.0006) [2023-12-26 23:12:00,628][105620] Updated weights for policy 1, policy_version 1075619 (0.0008) [2023-12-26 23:12:00,649][105692] Updated weights for policy 0, policy_version 1074499 (0.0007) [2023-12-26 23:12:00,686][105620] Updated weights for policy 1, policy_version 1075629 (0.0010) [2023-12-26 23:12:00,701][105692] Updated weights for policy 0, policy_version 1074509 (0.0006) [2023-12-26 23:12:00,738][105620] Updated weights for policy 1, policy_version 1075639 (0.0010) [2023-12-26 23:12:00,760][105692] Updated weights for policy 0, policy_version 1074519 (0.0007) [2023-12-26 23:12:00,769][105585] KL-divergence is very high: 135.1878 [2023-12-26 23:12:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 550518784. Throughput: 0: 9880.2, 1: 10053.0. Samples: 550487244. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:12:01,063][104569] Avg episode reward: [(0, '8296.340'), (1, '9073.834')] [2023-12-26 23:12:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001074528_275120128.pth... [2023-12-26 23:12:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001075648_275398656.pth... [2023-12-26 23:12:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001074496_275103744.pth [2023-12-26 23:12:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001073376_274825216.pth [2023-12-26 23:12:01,474][105620] Updated weights for policy 1, policy_version 1075649 (0.0011) [2023-12-26 23:12:01,528][105620] Updated weights for policy 1, policy_version 1075659 (0.0009) [2023-12-26 23:12:01,567][105692] Updated weights for policy 0, policy_version 1074529 (0.0007) [2023-12-26 23:12:01,574][105585] KL-divergence is very high: 107.6176 [2023-12-26 23:12:01,589][105620] Updated weights for policy 1, policy_version 1075669 (0.0007) [2023-12-26 23:12:01,622][105585] KL-divergence is very high: 116.3742 [2023-12-26 23:12:01,627][105692] Updated weights for policy 0, policy_version 1074539 (0.0008) [2023-12-26 23:12:01,649][105620] Updated weights for policy 1, policy_version 1075679 (0.0007) [2023-12-26 23:12:01,672][105585] KL-divergence is very high: 114.4325 [2023-12-26 23:12:01,693][105692] Updated weights for policy 0, policy_version 1074549 (0.0008) [2023-12-26 23:12:01,750][105692] Updated weights for policy 0, policy_version 1074559 (0.0009) [2023-12-26 23:12:02,311][105620] Updated weights for policy 1, policy_version 1075689 (0.0007) [2023-12-26 23:12:02,378][105620] Updated weights for policy 1, policy_version 1075699 (0.0007) [2023-12-26 23:12:02,435][105620] Updated weights for policy 1, policy_version 1075709 (0.0006) [2023-12-26 23:12:02,568][105692] Updated weights for policy 0, policy_version 1074569 (0.0009) [2023-12-26 23:12:02,626][105692] Updated weights for policy 0, policy_version 1074579 (0.0009) [2023-12-26 23:12:02,677][105692] Updated weights for policy 0, policy_version 1074589 (0.0009) [2023-12-26 23:12:03,048][105620] Updated weights for policy 1, policy_version 1075719 (0.0007) [2023-12-26 23:12:03,112][105620] Updated weights for policy 1, policy_version 1075729 (0.0007) [2023-12-26 23:12:03,166][105620] Updated weights for policy 1, policy_version 1075739 (0.0009) [2023-12-26 23:12:03,438][105692] Updated weights for policy 0, policy_version 1074599 (0.0006) [2023-12-26 23:12:03,484][105692] Updated weights for policy 0, policy_version 1074609 (0.0005) [2023-12-26 23:12:03,535][105692] Updated weights for policy 0, policy_version 1074619 (0.0005) [2023-12-26 23:12:03,802][105620] Updated weights for policy 1, policy_version 1075749 (0.0009) [2023-12-26 23:12:03,862][105620] Updated weights for policy 1, policy_version 1075759 (0.0008) [2023-12-26 23:12:03,931][105620] Updated weights for policy 1, policy_version 1075769 (0.0009) [2023-12-26 23:12:04,072][105692] Updated weights for policy 0, policy_version 1074629 (0.0007) [2023-12-26 23:12:04,136][105692] Updated weights for policy 0, policy_version 1074639 (0.0009) [2023-12-26 23:12:04,196][105692] Updated weights for policy 0, policy_version 1074649 (0.0011) [2023-12-26 23:12:04,680][105620] Updated weights for policy 1, policy_version 1075779 (0.0009) [2023-12-26 23:12:04,731][105620] Updated weights for policy 1, policy_version 1075789 (0.0006) [2023-12-26 23:12:04,775][105620] Updated weights for policy 1, policy_version 1075799 (0.0008) [2023-12-26 23:12:04,846][105692] Updated weights for policy 0, policy_version 1074659 (0.0011) [2023-12-26 23:12:04,890][105692] Updated weights for policy 0, policy_version 1074669 (0.0010) [2023-12-26 23:12:04,948][105692] Updated weights for policy 0, policy_version 1074679 (0.0010) [2023-12-26 23:12:05,513][105620] Updated weights for policy 1, policy_version 1075809 (0.0008) [2023-12-26 23:12:05,569][105620] Updated weights for policy 1, policy_version 1075819 (0.0010) [2023-12-26 23:12:05,574][105692] Updated weights for policy 0, policy_version 1074689 (0.0007) [2023-12-26 23:12:05,628][105620] Updated weights for policy 1, policy_version 1075829 (0.0006) [2023-12-26 23:12:05,629][105692] Updated weights for policy 0, policy_version 1074699 (0.0007) [2023-12-26 23:12:05,677][105692] Updated weights for policy 0, policy_version 1074709 (0.0005) [2023-12-26 23:12:05,687][105620] Updated weights for policy 1, policy_version 1075839 (0.0007) [2023-12-26 23:12:05,723][105692] Updated weights for policy 0, policy_version 1074719 (0.0005) [2023-12-26 23:12:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 550617088. Throughput: 0: 9722.7, 1: 10000.4. Samples: 550603464. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:12:06,062][104569] Avg episode reward: [(0, '8301.240'), (1, '8114.007')] [2023-12-26 23:12:06,228][105620] Updated weights for policy 1, policy_version 1075849 (0.0009) [2023-12-26 23:12:06,296][105620] Updated weights for policy 1, policy_version 1075859 (0.0006) [2023-12-26 23:12:06,340][105692] Updated weights for policy 0, policy_version 1074729 (0.0010) [2023-12-26 23:12:06,367][105620] Updated weights for policy 1, policy_version 1075869 (0.0008) [2023-12-26 23:12:06,392][105692] Updated weights for policy 0, policy_version 1074739 (0.0010) [2023-12-26 23:12:06,455][105692] Updated weights for policy 0, policy_version 1074749 (0.0011) [2023-12-26 23:12:06,989][105620] Updated weights for policy 1, policy_version 1075879 (0.0006) [2023-12-26 23:12:07,043][105620] Updated weights for policy 1, policy_version 1075889 (0.0005) [2023-12-26 23:12:07,095][105620] Updated weights for policy 1, policy_version 1075899 (0.0005) [2023-12-26 23:12:07,200][105692] Updated weights for policy 0, policy_version 1074759 (0.0009) [2023-12-26 23:12:07,249][105692] Updated weights for policy 0, policy_version 1074769 (0.0008) [2023-12-26 23:12:07,299][105692] Updated weights for policy 0, policy_version 1074779 (0.0008) [2023-12-26 23:12:07,769][105620] Updated weights for policy 1, policy_version 1075909 (0.0007) [2023-12-26 23:12:07,820][105620] Updated weights for policy 1, policy_version 1075919 (0.0005) [2023-12-26 23:12:07,869][105620] Updated weights for policy 1, policy_version 1075929 (0.0005) [2023-12-26 23:12:07,902][105692] Updated weights for policy 0, policy_version 1074789 (0.0005) [2023-12-26 23:12:07,946][105692] Updated weights for policy 0, policy_version 1074799 (0.0005) [2023-12-26 23:12:07,994][105692] Updated weights for policy 0, policy_version 1074809 (0.0005) [2023-12-26 23:12:08,426][105620] Updated weights for policy 1, policy_version 1075939 (0.0007) [2023-12-26 23:12:08,485][105620] Updated weights for policy 1, policy_version 1075949 (0.0011) [2023-12-26 23:12:08,538][105620] Updated weights for policy 1, policy_version 1075959 (0.0011) [2023-12-26 23:12:08,594][105692] Updated weights for policy 0, policy_version 1074819 (0.0009) [2023-12-26 23:12:08,659][105692] Updated weights for policy 0, policy_version 1074829 (0.0009) [2023-12-26 23:12:08,721][105692] Updated weights for policy 0, policy_version 1074839 (0.0011) [2023-12-26 23:12:09,153][105620] Updated weights for policy 1, policy_version 1075969 (0.0011) [2023-12-26 23:12:09,214][105620] Updated weights for policy 1, policy_version 1075979 (0.0006) [2023-12-26 23:12:09,279][105620] Updated weights for policy 1, policy_version 1075989 (0.0006) [2023-12-26 23:12:09,347][105620] Updated weights for policy 1, policy_version 1075999 (0.0007) [2023-12-26 23:12:09,450][105692] Updated weights for policy 0, policy_version 1074849 (0.0010) [2023-12-26 23:12:09,512][105692] Updated weights for policy 0, policy_version 1074859 (0.0012) [2023-12-26 23:12:09,574][105692] Updated weights for policy 0, policy_version 1074869 (0.0011) [2023-12-26 23:12:09,641][105692] Updated weights for policy 0, policy_version 1074879 (0.0011) [2023-12-26 23:12:10,157][105620] Updated weights for policy 1, policy_version 1076009 (0.0009) [2023-12-26 23:12:10,222][105620] Updated weights for policy 1, policy_version 1076019 (0.0009) [2023-12-26 23:12:10,280][105620] Updated weights for policy 1, policy_version 1076029 (0.0009) [2023-12-26 23:12:10,360][105692] Updated weights for policy 0, policy_version 1074889 (0.0009) [2023-12-26 23:12:10,423][105692] Updated weights for policy 0, policy_version 1074899 (0.0006) [2023-12-26 23:12:10,485][105692] Updated weights for policy 0, policy_version 1074909 (0.0006) [2023-12-26 23:12:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 550715392. Throughput: 0: 9900.5, 1: 9987.0. Samples: 550728980. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:12:11,062][104569] Avg episode reward: [(0, '8827.299'), (1, '6372.149')] [2023-12-26 23:12:11,085][105620] Updated weights for policy 1, policy_version 1076039 (0.0009) [2023-12-26 23:12:11,152][105620] Updated weights for policy 1, policy_version 1076049 (0.0007) [2023-12-26 23:12:11,205][105692] Updated weights for policy 0, policy_version 1074919 (0.0007) [2023-12-26 23:12:11,215][105620] Updated weights for policy 1, policy_version 1076059 (0.0007) [2023-12-26 23:12:11,265][105692] Updated weights for policy 0, policy_version 1074929 (0.0008) [2023-12-26 23:12:11,330][105692] Updated weights for policy 0, policy_version 1074939 (0.0008) [2023-12-26 23:12:12,027][105620] Updated weights for policy 1, policy_version 1076069 (0.0009) [2023-12-26 23:12:12,085][105620] Updated weights for policy 1, policy_version 1076079 (0.0008) [2023-12-26 23:12:12,087][105692] Updated weights for policy 0, policy_version 1074949 (0.0007) [2023-12-26 23:12:12,143][105620] Updated weights for policy 1, policy_version 1076089 (0.0006) [2023-12-26 23:12:12,148][105692] Updated weights for policy 0, policy_version 1074959 (0.0008) [2023-12-26 23:12:12,203][105692] Updated weights for policy 0, policy_version 1074969 (0.0009) [2023-12-26 23:12:12,836][105692] Updated weights for policy 0, policy_version 1074979 (0.0008) [2023-12-26 23:12:12,890][105692] Updated weights for policy 0, policy_version 1074989 (0.0005) [2023-12-26 23:12:12,941][105692] Updated weights for policy 0, policy_version 1074999 (0.0007) [2023-12-26 23:12:12,972][105620] Updated weights for policy 1, policy_version 1076099 (0.0010) [2023-12-26 23:12:13,020][105620] Updated weights for policy 1, policy_version 1076109 (0.0007) [2023-12-26 23:12:13,072][105620] Updated weights for policy 1, policy_version 1076119 (0.0008) [2023-12-26 23:12:13,541][105692] Updated weights for policy 0, policy_version 1075009 (0.0007) [2023-12-26 23:12:13,593][105692] Updated weights for policy 0, policy_version 1075019 (0.0005) [2023-12-26 23:12:13,648][105692] Updated weights for policy 0, policy_version 1075029 (0.0005) [2023-12-26 23:12:13,697][105692] Updated weights for policy 0, policy_version 1075039 (0.0005) [2023-12-26 23:12:13,766][105620] Updated weights for policy 1, policy_version 1076129 (0.0006) [2023-12-26 23:12:13,838][105620] Updated weights for policy 1, policy_version 1076139 (0.0005) [2023-12-26 23:12:13,904][105620] Updated weights for policy 1, policy_version 1076149 (0.0005) [2023-12-26 23:12:13,971][105620] Updated weights for policy 1, policy_version 1076159 (0.0005) [2023-12-26 23:12:14,270][105692] Updated weights for policy 0, policy_version 1075049 (0.0006) [2023-12-26 23:12:14,327][105692] Updated weights for policy 0, policy_version 1075059 (0.0005) [2023-12-26 23:12:14,378][105692] Updated weights for policy 0, policy_version 1075069 (0.0006) [2023-12-26 23:12:14,454][105620] Updated weights for policy 1, policy_version 1076169 (0.0009) [2023-12-26 23:12:14,519][105620] Updated weights for policy 1, policy_version 1076179 (0.0009) [2023-12-26 23:12:14,578][105620] Updated weights for policy 1, policy_version 1076189 (0.0011) [2023-12-26 23:12:14,935][105692] Updated weights for policy 0, policy_version 1075079 (0.0005) [2023-12-26 23:12:14,989][105692] Updated weights for policy 0, policy_version 1075089 (0.0007) [2023-12-26 23:12:15,047][105692] Updated weights for policy 0, policy_version 1075099 (0.0009) [2023-12-26 23:12:15,375][105620] Updated weights for policy 1, policy_version 1076199 (0.0009) [2023-12-26 23:12:15,431][105620] Updated weights for policy 1, policy_version 1076209 (0.0010) [2023-12-26 23:12:15,491][105620] Updated weights for policy 1, policy_version 1076219 (0.0011) [2023-12-26 23:12:15,771][105692] Updated weights for policy 0, policy_version 1075109 (0.0007) [2023-12-26 23:12:15,828][105692] Updated weights for policy 0, policy_version 1075119 (0.0010) [2023-12-26 23:12:15,886][105692] Updated weights for policy 0, policy_version 1075130 (0.0010) [2023-12-26 23:12:16,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19934.0, 300 sec: 19660.8). Total num frames: 550821888. Throughput: 0: 9970.2, 1: 9963.4. Samples: 550787340. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:12:16,062][104569] Avg episode reward: [(0, '9087.300'), (1, '7880.456')] [2023-12-26 23:12:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001075136_275275776.pth... [2023-12-26 23:12:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001076224_275546112.pth... [2023-12-26 23:12:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001075072_275251200.pth [2023-12-26 23:12:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001073952_274972672.pth [2023-12-26 23:12:16,200][105620] Updated weights for policy 1, policy_version 1076229 (0.0010) [2023-12-26 23:12:16,259][105620] Updated weights for policy 1, policy_version 1076239 (0.0011) [2023-12-26 23:12:16,306][105620] Updated weights for policy 1, policy_version 1076249 (0.0006) [2023-12-26 23:12:16,501][105692] Updated weights for policy 0, policy_version 1075140 (0.0007) [2023-12-26 23:12:16,550][105692] Updated weights for policy 0, policy_version 1075150 (0.0005) [2023-12-26 23:12:16,603][105692] Updated weights for policy 0, policy_version 1075160 (0.0005) [2023-12-26 23:12:17,055][105620] Updated weights for policy 1, policy_version 1076259 (0.0007) [2023-12-26 23:12:17,121][105620] Updated weights for policy 1, policy_version 1076269 (0.0010) [2023-12-26 23:12:17,185][105620] Updated weights for policy 1, policy_version 1076279 (0.0009) [2023-12-26 23:12:17,200][105692] Updated weights for policy 0, policy_version 1075170 (0.0007) [2023-12-26 23:12:17,255][105692] Updated weights for policy 0, policy_version 1075180 (0.0006) [2023-12-26 23:12:17,308][105692] Updated weights for policy 0, policy_version 1075190 (0.0008) [2023-12-26 23:12:17,362][105692] Updated weights for policy 0, policy_version 1075200 (0.0005) [2023-12-26 23:12:17,981][105692] Updated weights for policy 0, policy_version 1075210 (0.0005) [2023-12-26 23:12:18,002][105620] Updated weights for policy 1, policy_version 1076289 (0.0009) [2023-12-26 23:12:18,043][105692] Updated weights for policy 0, policy_version 1075220 (0.0006) [2023-12-26 23:12:18,066][105620] Updated weights for policy 1, policy_version 1076299 (0.0009) [2023-12-26 23:12:18,108][105692] Updated weights for policy 0, policy_version 1075230 (0.0008) [2023-12-26 23:12:18,123][105620] Updated weights for policy 1, policy_version 1076309 (0.0005) [2023-12-26 23:12:18,176][105620] Updated weights for policy 1, policy_version 1076319 (0.0009) [2023-12-26 23:12:18,809][105692] Updated weights for policy 0, policy_version 1075240 (0.0008) [2023-12-26 23:12:18,878][105692] Updated weights for policy 0, policy_version 1075250 (0.0009) [2023-12-26 23:12:18,933][105620] Updated weights for policy 1, policy_version 1076329 (0.0006) [2023-12-26 23:12:18,935][105692] Updated weights for policy 0, policy_version 1075260 (0.0007) [2023-12-26 23:12:18,993][105620] Updated weights for policy 1, policy_version 1076339 (0.0008) [2023-12-26 23:12:19,060][105620] Updated weights for policy 1, policy_version 1076349 (0.0009) [2023-12-26 23:12:19,700][105692] Updated weights for policy 0, policy_version 1075270 (0.0008) [2023-12-26 23:12:19,762][105692] Updated weights for policy 0, policy_version 1075280 (0.0009) [2023-12-26 23:12:19,811][105620] Updated weights for policy 1, policy_version 1076359 (0.0009) [2023-12-26 23:12:19,827][105692] Updated weights for policy 0, policy_version 1075290 (0.0009) [2023-12-26 23:12:19,876][105620] Updated weights for policy 1, policy_version 1076369 (0.0009) [2023-12-26 23:12:19,945][105620] Updated weights for policy 1, policy_version 1076379 (0.0009) [2023-12-26 23:12:20,576][105692] Updated weights for policy 0, policy_version 1075300 (0.0008) [2023-12-26 23:12:20,636][105692] Updated weights for policy 0, policy_version 1075310 (0.0008) [2023-12-26 23:12:20,696][105620] Updated weights for policy 1, policy_version 1076389 (0.0011) [2023-12-26 23:12:20,696][105692] Updated weights for policy 0, policy_version 1075320 (0.0009) [2023-12-26 23:12:20,763][105620] Updated weights for policy 1, policy_version 1076399 (0.0010) [2023-12-26 23:12:20,817][105620] Updated weights for policy 1, policy_version 1076409 (0.0010) [2023-12-26 23:12:21,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19933.8, 300 sec: 19660.8). Total num frames: 550920192. Throughput: 0: 10162.5, 1: 9790.2. Samples: 550907968. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:12:21,063][104569] Avg episode reward: [(0, '9174.852'), (1, '9071.536')] [2023-12-26 23:12:21,481][105692] Updated weights for policy 0, policy_version 1075330 (0.0009) [2023-12-26 23:12:21,533][105692] Updated weights for policy 0, policy_version 1075340 (0.0008) [2023-12-26 23:12:21,584][105620] Updated weights for policy 1, policy_version 1076419 (0.0009) [2023-12-26 23:12:21,595][105692] Updated weights for policy 0, policy_version 1075350 (0.0009) [2023-12-26 23:12:21,652][105620] Updated weights for policy 1, policy_version 1076429 (0.0010) [2023-12-26 23:12:21,666][105692] Updated weights for policy 0, policy_version 1075360 (0.0008) [2023-12-26 23:12:21,717][105620] Updated weights for policy 1, policy_version 1076439 (0.0011) [2023-12-26 23:12:22,444][105692] Updated weights for policy 0, policy_version 1075370 (0.0008) [2023-12-26 23:12:22,499][105692] Updated weights for policy 0, policy_version 1075380 (0.0006) [2023-12-26 23:12:22,525][105620] Updated weights for policy 1, policy_version 1076449 (0.0009) [2023-12-26 23:12:22,554][105692] Updated weights for policy 0, policy_version 1075390 (0.0006) [2023-12-26 23:12:22,583][105620] Updated weights for policy 1, policy_version 1076459 (0.0008) [2023-12-26 23:12:22,643][105620] Updated weights for policy 1, policy_version 1076469 (0.0008) [2023-12-26 23:12:22,708][105620] Updated weights for policy 1, policy_version 1076479 (0.0008) [2023-12-26 23:12:23,258][105692] Updated weights for policy 0, policy_version 1075400 (0.0009) [2023-12-26 23:12:23,319][105692] Updated weights for policy 0, policy_version 1075410 (0.0006) [2023-12-26 23:12:23,384][105692] Updated weights for policy 0, policy_version 1075420 (0.0009) [2023-12-26 23:12:23,503][105620] Updated weights for policy 1, policy_version 1076489 (0.0006) [2023-12-26 23:12:23,554][105620] Updated weights for policy 1, policy_version 1076499 (0.0005) [2023-12-26 23:12:23,613][105620] Updated weights for policy 1, policy_version 1076509 (0.0007) [2023-12-26 23:12:24,100][105692] Updated weights for policy 0, policy_version 1075430 (0.0011) [2023-12-26 23:12:24,156][105692] Updated weights for policy 0, policy_version 1075440 (0.0011) [2023-12-26 23:12:24,203][105692] Updated weights for policy 0, policy_version 1075450 (0.0010) [2023-12-26 23:12:24,254][105620] Updated weights for policy 1, policy_version 1076519 (0.0010) [2023-12-26 23:12:24,303][105620] Updated weights for policy 1, policy_version 1076529 (0.0010) [2023-12-26 23:12:24,352][105620] Updated weights for policy 1, policy_version 1076539 (0.0010) [2023-12-26 23:12:24,958][105692] Updated weights for policy 0, policy_version 1075460 (0.0008) [2023-12-26 23:12:25,011][105692] Updated weights for policy 0, policy_version 1075470 (0.0008) [2023-12-26 23:12:25,011][105620] Updated weights for policy 1, policy_version 1076549 (0.0010) [2023-12-26 23:12:25,059][105692] Updated weights for policy 0, policy_version 1075480 (0.0010) [2023-12-26 23:12:25,060][105620] Updated weights for policy 1, policy_version 1076559 (0.0010) [2023-12-26 23:12:25,112][105620] Updated weights for policy 1, policy_version 1076569 (0.0009) [2023-12-26 23:12:25,687][105620] Updated weights for policy 1, policy_version 1076579 (0.0005) [2023-12-26 23:12:25,753][105620] Updated weights for policy 1, policy_version 1076589 (0.0005) [2023-12-26 23:12:25,763][105692] Updated weights for policy 0, policy_version 1075490 (0.0009) [2023-12-26 23:12:25,808][105620] Updated weights for policy 1, policy_version 1076599 (0.0005) [2023-12-26 23:12:25,817][105692] Updated weights for policy 0, policy_version 1075500 (0.0005) [2023-12-26 23:12:25,871][105692] Updated weights for policy 0, policy_version 1075510 (0.0007) [2023-12-26 23:12:25,929][105692] Updated weights for policy 0, policy_version 1075520 (0.0010) [2023-12-26 23:12:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 551018496. Throughput: 0: 10055.6, 1: 9762.0. Samples: 551022704. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:12:26,063][104569] Avg episode reward: [(0, '8396.305'), (1, '9255.074')] [2023-12-26 23:12:26,316][105620] Updated weights for policy 1, policy_version 1076609 (0.0005) [2023-12-26 23:12:26,367][105620] Updated weights for policy 1, policy_version 1076619 (0.0006) [2023-12-26 23:12:26,414][105620] Updated weights for policy 1, policy_version 1076629 (0.0005) [2023-12-26 23:12:26,466][105620] Updated weights for policy 1, policy_version 1076639 (0.0005) [2023-12-26 23:12:26,632][105692] Updated weights for policy 0, policy_version 1075530 (0.0011) [2023-12-26 23:12:26,693][105692] Updated weights for policy 0, policy_version 1075540 (0.0009) [2023-12-26 23:12:26,754][105692] Updated weights for policy 0, policy_version 1075550 (0.0010) [2023-12-26 23:12:27,149][105620] Updated weights for policy 1, policy_version 1076649 (0.0008) [2023-12-26 23:12:27,210][105620] Updated weights for policy 1, policy_version 1076659 (0.0008) [2023-12-26 23:12:27,272][105620] Updated weights for policy 1, policy_version 1076669 (0.0008) [2023-12-26 23:12:27,527][105692] Updated weights for policy 0, policy_version 1075560 (0.0010) [2023-12-26 23:12:27,589][105692] Updated weights for policy 0, policy_version 1075570 (0.0010) [2023-12-26 23:12:27,640][105692] Updated weights for policy 0, policy_version 1075580 (0.0010) [2023-12-26 23:12:27,935][105620] Updated weights for policy 1, policy_version 1076679 (0.0006) [2023-12-26 23:12:27,987][105620] Updated weights for policy 1, policy_version 1076689 (0.0005) [2023-12-26 23:12:28,045][105620] Updated weights for policy 1, policy_version 1076699 (0.0008) [2023-12-26 23:12:28,380][105692] Updated weights for policy 0, policy_version 1075590 (0.0010) [2023-12-26 23:12:28,434][105692] Updated weights for policy 0, policy_version 1075600 (0.0009) [2023-12-26 23:12:28,486][105692] Updated weights for policy 0, policy_version 1075610 (0.0006) [2023-12-26 23:12:28,650][105620] Updated weights for policy 1, policy_version 1076709 (0.0005) [2023-12-26 23:12:28,700][105620] Updated weights for policy 1, policy_version 1076719 (0.0009) [2023-12-26 23:12:28,758][105620] Updated weights for policy 1, policy_version 1076729 (0.0010) [2023-12-26 23:12:29,223][105692] Updated weights for policy 0, policy_version 1075620 (0.0008) [2023-12-26 23:12:29,282][105692] Updated weights for policy 0, policy_version 1075630 (0.0011) [2023-12-26 23:12:29,341][105692] Updated weights for policy 0, policy_version 1075640 (0.0009) [2023-12-26 23:12:29,505][105620] Updated weights for policy 1, policy_version 1076739 (0.0009) [2023-12-26 23:12:29,553][105620] Updated weights for policy 1, policy_version 1076749 (0.0010) [2023-12-26 23:12:29,600][105620] Updated weights for policy 1, policy_version 1076759 (0.0010) [2023-12-26 23:12:30,021][105692] Updated weights for policy 0, policy_version 1075650 (0.0011) [2023-12-26 23:12:30,077][105692] Updated weights for policy 0, policy_version 1075660 (0.0010) [2023-12-26 23:12:30,143][105692] Updated weights for policy 0, policy_version 1075670 (0.0010) [2023-12-26 23:12:30,204][105692] Updated weights for policy 0, policy_version 1075680 (0.0010) [2023-12-26 23:12:30,293][105620] Updated weights for policy 1, policy_version 1076769 (0.0010) [2023-12-26 23:12:30,341][105620] Updated weights for policy 1, policy_version 1076779 (0.0008) [2023-12-26 23:12:30,396][105620] Updated weights for policy 1, policy_version 1076789 (0.0006) [2023-12-26 23:12:30,461][105620] Updated weights for policy 1, policy_version 1076799 (0.0006) [2023-12-26 23:12:30,876][105692] Updated weights for policy 0, policy_version 1075690 (0.0005) [2023-12-26 23:12:30,933][105692] Updated weights for policy 0, policy_version 1075700 (0.0006) [2023-12-26 23:12:30,988][105692] Updated weights for policy 0, policy_version 1075710 (0.0006) [2023-12-26 23:12:31,031][105620] Updated weights for policy 1, policy_version 1076809 (0.0007) [2023-12-26 23:12:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 20070.4, 300 sec: 19660.8). Total num frames: 551116800. Throughput: 0: 9982.9, 1: 9921.3. Samples: 551083940. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:12:31,063][104569] Avg episode reward: [(0, '8305.604'), (1, '9255.109')] [2023-12-26 23:12:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001075712_275423232.pth... [2023-12-26 23:12:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001074528_275120128.pth [2023-12-26 23:12:31,096][105620] Updated weights for policy 1, policy_version 1076819 (0.0007) [2023-12-26 23:12:31,159][105620] Updated weights for policy 1, policy_version 1076829 (0.0009) [2023-12-26 23:12:31,178][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001076832_275701760.pth... [2023-12-26 23:12:31,184][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001075648_275398656.pth [2023-12-26 23:12:31,711][105692] Updated weights for policy 0, policy_version 1075720 (0.0008) [2023-12-26 23:12:31,766][105692] Updated weights for policy 0, policy_version 1075730 (0.0009) [2023-12-26 23:12:31,817][105692] Updated weights for policy 0, policy_version 1075740 (0.0009) [2023-12-26 23:12:31,906][105620] Updated weights for policy 1, policy_version 1076839 (0.0009) [2023-12-26 23:12:31,961][105620] Updated weights for policy 1, policy_version 1076849 (0.0010) [2023-12-26 23:12:32,013][105620] Updated weights for policy 1, policy_version 1076859 (0.0010) [2023-12-26 23:12:32,456][105692] Updated weights for policy 0, policy_version 1075750 (0.0010) [2023-12-26 23:12:32,513][105692] Updated weights for policy 0, policy_version 1075760 (0.0009) [2023-12-26 23:12:32,569][105692] Updated weights for policy 0, policy_version 1075770 (0.0005) [2023-12-26 23:12:32,738][105620] Updated weights for policy 1, policy_version 1076869 (0.0009) [2023-12-26 23:12:32,803][105620] Updated weights for policy 1, policy_version 1076879 (0.0006) [2023-12-26 23:12:32,854][105620] Updated weights for policy 1, policy_version 1076889 (0.0006) [2023-12-26 23:12:33,134][105692] Updated weights for policy 0, policy_version 1075780 (0.0007) [2023-12-26 23:12:33,185][105692] Updated weights for policy 0, policy_version 1075790 (0.0010) [2023-12-26 23:12:33,232][105692] Updated weights for policy 0, policy_version 1075800 (0.0010) [2023-12-26 23:12:33,438][105620] Updated weights for policy 1, policy_version 1076899 (0.0008) [2023-12-26 23:12:33,504][105620] Updated weights for policy 1, policy_version 1076909 (0.0010) [2023-12-26 23:12:33,548][105620] Updated weights for policy 1, policy_version 1076919 (0.0010) [2023-12-26 23:12:33,827][105692] Updated weights for policy 0, policy_version 1075810 (0.0010) [2023-12-26 23:12:33,877][105692] Updated weights for policy 0, policy_version 1075820 (0.0010) [2023-12-26 23:12:33,931][105692] Updated weights for policy 0, policy_version 1075830 (0.0009) [2023-12-26 23:12:34,001][105692] Updated weights for policy 0, policy_version 1075840 (0.0010) [2023-12-26 23:12:34,203][105620] Updated weights for policy 1, policy_version 1076929 (0.0009) [2023-12-26 23:12:34,269][105620] Updated weights for policy 1, policy_version 1076939 (0.0007) [2023-12-26 23:12:34,325][105620] Updated weights for policy 1, policy_version 1076949 (0.0008) [2023-12-26 23:12:34,390][105620] Updated weights for policy 1, policy_version 1076959 (0.0008) [2023-12-26 23:12:34,715][105692] Updated weights for policy 0, policy_version 1075850 (0.0011) [2023-12-26 23:12:34,770][105692] Updated weights for policy 0, policy_version 1075860 (0.0010) [2023-12-26 23:12:34,822][105692] Updated weights for policy 0, policy_version 1075870 (0.0010) [2023-12-26 23:12:35,127][105620] Updated weights for policy 1, policy_version 1076969 (0.0008) [2023-12-26 23:12:35,179][105620] Updated weights for policy 1, policy_version 1076979 (0.0008) [2023-12-26 23:12:35,234][105620] Updated weights for policy 1, policy_version 1076989 (0.0008) [2023-12-26 23:12:35,562][105692] Updated weights for policy 0, policy_version 1075880 (0.0010) [2023-12-26 23:12:35,622][105692] Updated weights for policy 0, policy_version 1075890 (0.0007) [2023-12-26 23:12:35,672][105692] Updated weights for policy 0, policy_version 1075900 (0.0007) [2023-12-26 23:12:35,974][105620] Updated weights for policy 1, policy_version 1076999 (0.0009) [2023-12-26 23:12:36,027][105620] Updated weights for policy 1, policy_version 1077009 (0.0010) [2023-12-26 23:12:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19933.9, 300 sec: 19660.8). Total num frames: 551215104. Throughput: 0: 10121.2, 1: 9949.8. Samples: 551208376. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:12:36,062][104569] Avg episode reward: [(0, '8753.862'), (1, '9071.428')] [2023-12-26 23:12:36,084][105620] Updated weights for policy 1, policy_version 1077020 (0.0008) [2023-12-26 23:12:36,327][105692] Updated weights for policy 0, policy_version 1075910 (0.0011) [2023-12-26 23:12:36,386][105692] Updated weights for policy 0, policy_version 1075920 (0.0011) [2023-12-26 23:12:36,446][105692] Updated weights for policy 0, policy_version 1075930 (0.0011) [2023-12-26 23:12:36,870][105620] Updated weights for policy 1, policy_version 1077030 (0.0009) [2023-12-26 23:12:36,922][105620] Updated weights for policy 1, policy_version 1077040 (0.0010) [2023-12-26 23:12:36,974][105620] Updated weights for policy 1, policy_version 1077050 (0.0010) [2023-12-26 23:12:37,193][105692] Updated weights for policy 0, policy_version 1075940 (0.0011) [2023-12-26 23:12:37,255][105692] Updated weights for policy 0, policy_version 1075950 (0.0011) [2023-12-26 23:12:37,306][105692] Updated weights for policy 0, policy_version 1075960 (0.0005) [2023-12-26 23:12:37,745][105620] Updated weights for policy 1, policy_version 1077060 (0.0010) [2023-12-26 23:12:37,801][105620] Updated weights for policy 1, policy_version 1077070 (0.0011) [2023-12-26 23:12:37,861][105692] Updated weights for policy 0, policy_version 1075970 (0.0007) [2023-12-26 23:12:37,867][105620] Updated weights for policy 1, policy_version 1077080 (0.0011) [2023-12-26 23:12:37,913][105692] Updated weights for policy 0, policy_version 1075980 (0.0011) [2023-12-26 23:12:37,972][105692] Updated weights for policy 0, policy_version 1075990 (0.0010) [2023-12-26 23:12:38,035][105692] Updated weights for policy 0, policy_version 1076000 (0.0010) [2023-12-26 23:12:38,555][105620] Updated weights for policy 1, policy_version 1077090 (0.0010) [2023-12-26 23:12:38,620][105620] Updated weights for policy 1, policy_version 1077100 (0.0007) [2023-12-26 23:12:38,682][105620] Updated weights for policy 1, policy_version 1077110 (0.0007) [2023-12-26 23:12:38,744][105620] Updated weights for policy 1, policy_version 1077120 (0.0008) [2023-12-26 23:12:38,790][105692] Updated weights for policy 0, policy_version 1076010 (0.0011) [2023-12-26 23:12:38,842][105692] Updated weights for policy 0, policy_version 1076020 (0.0010) [2023-12-26 23:12:38,890][105692] Updated weights for policy 0, policy_version 1076030 (0.0011) [2023-12-26 23:12:39,392][105620] Updated weights for policy 1, policy_version 1077130 (0.0007) [2023-12-26 23:12:39,460][105620] Updated weights for policy 1, policy_version 1077140 (0.0008) [2023-12-26 23:12:39,528][105620] Updated weights for policy 1, policy_version 1077150 (0.0008) [2023-12-26 23:12:39,625][105692] Updated weights for policy 0, policy_version 1076040 (0.0008) [2023-12-26 23:12:39,684][105692] Updated weights for policy 0, policy_version 1076050 (0.0011) [2023-12-26 23:12:39,743][105692] Updated weights for policy 0, policy_version 1076060 (0.0010) [2023-12-26 23:12:40,277][105620] Updated weights for policy 1, policy_version 1077160 (0.0009) [2023-12-26 23:12:40,339][105620] Updated weights for policy 1, policy_version 1077170 (0.0010) [2023-12-26 23:12:40,404][105620] Updated weights for policy 1, policy_version 1077180 (0.0008) [2023-12-26 23:12:40,438][105692] Updated weights for policy 0, policy_version 1076070 (0.0011) [2023-12-26 23:12:40,497][105692] Updated weights for policy 0, policy_version 1076080 (0.0011) [2023-12-26 23:12:40,553][105692] Updated weights for policy 0, policy_version 1076090 (0.0010) [2023-12-26 23:12:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19688.6). Total num frames: 551313408. Throughput: 0: 10012.9, 1: 9881.9. Samples: 551324200. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:12:41,062][104569] Avg episode reward: [(0, '9169.714'), (1, '9005.759')] [2023-12-26 23:12:41,224][105620] Updated weights for policy 1, policy_version 1077190 (0.0007) [2023-12-26 23:12:41,287][105620] Updated weights for policy 1, policy_version 1077200 (0.0008) [2023-12-26 23:12:41,324][105692] Updated weights for policy 0, policy_version 1076100 (0.0009) [2023-12-26 23:12:41,354][105620] Updated weights for policy 1, policy_version 1077210 (0.0008) [2023-12-26 23:12:41,395][105692] Updated weights for policy 0, policy_version 1076110 (0.0010) [2023-12-26 23:12:41,458][105692] Updated weights for policy 0, policy_version 1076120 (0.0009) [2023-12-26 23:12:42,090][105620] Updated weights for policy 1, policy_version 1077220 (0.0008) [2023-12-26 23:12:42,152][105620] Updated weights for policy 1, policy_version 1077230 (0.0009) [2023-12-26 23:12:42,220][105620] Updated weights for policy 1, policy_version 1077240 (0.0009) [2023-12-26 23:12:42,256][105692] Updated weights for policy 0, policy_version 1076130 (0.0010) [2023-12-26 23:12:42,323][105692] Updated weights for policy 0, policy_version 1076140 (0.0009) [2023-12-26 23:12:42,395][105692] Updated weights for policy 0, policy_version 1076150 (0.0009) [2023-12-26 23:12:42,455][105692] Updated weights for policy 0, policy_version 1076160 (0.0011) [2023-12-26 23:12:42,827][105620] Updated weights for policy 1, policy_version 1077250 (0.0009) [2023-12-26 23:12:42,881][105620] Updated weights for policy 1, policy_version 1077260 (0.0008) [2023-12-26 23:12:42,940][105620] Updated weights for policy 1, policy_version 1077270 (0.0009) [2023-12-26 23:12:42,994][105620] Updated weights for policy 1, policy_version 1077280 (0.0009) [2023-12-26 23:12:43,241][105692] Updated weights for policy 0, policy_version 1076170 (0.0010) [2023-12-26 23:12:43,297][105692] Updated weights for policy 0, policy_version 1076180 (0.0009) [2023-12-26 23:12:43,352][105692] Updated weights for policy 0, policy_version 1076190 (0.0006) [2023-12-26 23:12:43,769][105620] Updated weights for policy 1, policy_version 1077290 (0.0009) [2023-12-26 23:12:43,816][105620] Updated weights for policy 1, policy_version 1077300 (0.0009) [2023-12-26 23:12:43,878][105620] Updated weights for policy 1, policy_version 1077310 (0.0009) [2023-12-26 23:12:44,016][105692] Updated weights for policy 0, policy_version 1076200 (0.0008) [2023-12-26 23:12:44,067][105692] Updated weights for policy 0, policy_version 1076210 (0.0007) [2023-12-26 23:12:44,122][105692] Updated weights for policy 0, policy_version 1076220 (0.0006) [2023-12-26 23:12:44,679][105620] Updated weights for policy 1, policy_version 1077320 (0.0008) [2023-12-26 23:12:44,733][105620] Updated weights for policy 1, policy_version 1077330 (0.0008) [2023-12-26 23:12:44,796][105620] Updated weights for policy 1, policy_version 1077340 (0.0008) [2023-12-26 23:12:44,900][105692] Updated weights for policy 0, policy_version 1076230 (0.0010) [2023-12-26 23:12:44,954][105692] Updated weights for policy 0, policy_version 1076240 (0.0008) [2023-12-26 23:12:45,007][105692] Updated weights for policy 0, policy_version 1076250 (0.0006) [2023-12-26 23:12:45,549][105620] Updated weights for policy 1, policy_version 1077350 (0.0008) [2023-12-26 23:12:45,592][105692] Updated weights for policy 0, policy_version 1076260 (0.0005) [2023-12-26 23:12:45,615][105620] Updated weights for policy 1, policy_version 1077360 (0.0010) [2023-12-26 23:12:45,652][105692] Updated weights for policy 0, policy_version 1076270 (0.0006) [2023-12-26 23:12:45,680][105620] Updated weights for policy 1, policy_version 1077370 (0.0009) [2023-12-26 23:12:45,724][105692] Updated weights for policy 0, policy_version 1076280 (0.0011) [2023-12-26 23:12:46,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19797.2, 300 sec: 19688.5). Total num frames: 551411712. Throughput: 0: 9965.6, 1: 9863.0. Samples: 551379536. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:12:46,063][104569] Avg episode reward: [(0, '9347.638'), (1, '9097.735')] [2023-12-26 23:12:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001076288_275570688.pth... [2023-12-26 23:12:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001077376_275841024.pth... [2023-12-26 23:12:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001076224_275546112.pth [2023-12-26 23:12:46,089][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001075136_275275776.pth [2023-12-26 23:12:46,306][105620] Updated weights for policy 1, policy_version 1077380 (0.0007) [2023-12-26 23:12:46,360][105692] Updated weights for policy 0, policy_version 1076290 (0.0010) [2023-12-26 23:12:46,366][105620] Updated weights for policy 1, policy_version 1077390 (0.0011) [2023-12-26 23:12:46,415][105692] Updated weights for policy 0, policy_version 1076300 (0.0005) [2023-12-26 23:12:46,419][105620] Updated weights for policy 1, policy_version 1077400 (0.0011) [2023-12-26 23:12:46,476][105692] Updated weights for policy 0, policy_version 1076310 (0.0007) [2023-12-26 23:12:46,535][105692] Updated weights for policy 0, policy_version 1076320 (0.0010) [2023-12-26 23:12:47,155][105692] Updated weights for policy 0, policy_version 1076330 (0.0006) [2023-12-26 23:12:47,155][105620] Updated weights for policy 1, policy_version 1077410 (0.0010) [2023-12-26 23:12:47,201][105692] Updated weights for policy 0, policy_version 1076340 (0.0007) [2023-12-26 23:12:47,203][105620] Updated weights for policy 1, policy_version 1077420 (0.0010) [2023-12-26 23:12:47,246][105692] Updated weights for policy 0, policy_version 1076350 (0.0006) [2023-12-26 23:12:47,250][105620] Updated weights for policy 1, policy_version 1077430 (0.0010) [2023-12-26 23:12:47,309][105620] Updated weights for policy 1, policy_version 1077440 (0.0010) [2023-12-26 23:12:47,971][105692] Updated weights for policy 0, policy_version 1076360 (0.0008) [2023-12-26 23:12:47,983][105620] Updated weights for policy 1, policy_version 1077450 (0.0005) [2023-12-26 23:12:48,034][105692] Updated weights for policy 0, policy_version 1076370 (0.0006) [2023-12-26 23:12:48,040][105620] Updated weights for policy 1, policy_version 1077460 (0.0009) [2023-12-26 23:12:48,091][105620] Updated weights for policy 1, policy_version 1077470 (0.0012) [2023-12-26 23:12:48,095][105692] Updated weights for policy 0, policy_version 1076380 (0.0006) [2023-12-26 23:12:48,759][105620] Updated weights for policy 1, policy_version 1077480 (0.0010) [2023-12-26 23:12:48,801][105692] Updated weights for policy 0, policy_version 1076390 (0.0009) [2023-12-26 23:12:48,819][105620] Updated weights for policy 1, policy_version 1077490 (0.0010) [2023-12-26 23:12:48,859][105692] Updated weights for policy 0, policy_version 1076400 (0.0007) [2023-12-26 23:12:48,881][105620] Updated weights for policy 1, policy_version 1077500 (0.0010) [2023-12-26 23:12:48,925][105692] Updated weights for policy 0, policy_version 1076410 (0.0006) [2023-12-26 23:12:49,583][105620] Updated weights for policy 1, policy_version 1077510 (0.0009) [2023-12-26 23:12:49,643][105620] Updated weights for policy 1, policy_version 1077520 (0.0008) [2023-12-26 23:12:49,706][105620] Updated weights for policy 1, policy_version 1077530 (0.0006) [2023-12-26 23:12:49,733][105692] Updated weights for policy 0, policy_version 1076420 (0.0010) [2023-12-26 23:12:49,789][105692] Updated weights for policy 0, policy_version 1076430 (0.0010) [2023-12-26 23:12:49,852][105692] Updated weights for policy 0, policy_version 1076440 (0.0010) [2023-12-26 23:12:50,339][105620] Updated weights for policy 1, policy_version 1077540 (0.0007) [2023-12-26 23:12:50,406][105620] Updated weights for policy 1, policy_version 1077550 (0.0008) [2023-12-26 23:12:50,462][105620] Updated weights for policy 1, policy_version 1077560 (0.0008) [2023-12-26 23:12:50,627][105692] Updated weights for policy 0, policy_version 1076450 (0.0009) [2023-12-26 23:12:50,688][105692] Updated weights for policy 0, policy_version 1076460 (0.0009) [2023-12-26 23:12:50,745][105692] Updated weights for policy 0, policy_version 1076470 (0.0011) [2023-12-26 23:12:50,803][105692] Updated weights for policy 0, policy_version 1076480 (0.0011) [2023-12-26 23:12:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19933.9, 300 sec: 19688.6). Total num frames: 551510016. Throughput: 0: 10056.9, 1: 9872.5. Samples: 551500284. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:12:51,062][104569] Avg episode reward: [(0, '9256.596'), (1, '9111.097')] [2023-12-26 23:12:51,162][105620] Updated weights for policy 1, policy_version 1077570 (0.0008) [2023-12-26 23:12:51,211][105620] Updated weights for policy 1, policy_version 1077580 (0.0006) [2023-12-26 23:12:51,273][105620] Updated weights for policy 1, policy_version 1077590 (0.0007) [2023-12-26 23:12:51,346][105620] Updated weights for policy 1, policy_version 1077600 (0.0006) [2023-12-26 23:12:51,563][105692] Updated weights for policy 0, policy_version 1076490 (0.0009) [2023-12-26 23:12:51,639][105692] Updated weights for policy 0, policy_version 1076500 (0.0007) [2023-12-26 23:12:51,695][105692] Updated weights for policy 0, policy_version 1076510 (0.0008) [2023-12-26 23:12:51,998][105620] Updated weights for policy 1, policy_version 1077610 (0.0008) [2023-12-26 23:12:52,058][105620] Updated weights for policy 1, policy_version 1077620 (0.0005) [2023-12-26 23:12:52,120][105620] Updated weights for policy 1, policy_version 1077630 (0.0007) [2023-12-26 23:12:52,392][105692] Updated weights for policy 0, policy_version 1076520 (0.0010) [2023-12-26 23:12:52,452][105692] Updated weights for policy 0, policy_version 1076530 (0.0011) [2023-12-26 23:12:52,512][105692] Updated weights for policy 0, policy_version 1076540 (0.0009) [2023-12-26 23:12:52,774][105620] Updated weights for policy 1, policy_version 1077640 (0.0007) [2023-12-26 23:12:52,828][105620] Updated weights for policy 1, policy_version 1077650 (0.0007) [2023-12-26 23:12:52,878][105620] Updated weights for policy 1, policy_version 1077660 (0.0008) [2023-12-26 23:12:53,229][105692] Updated weights for policy 0, policy_version 1076550 (0.0007) [2023-12-26 23:12:53,285][105692] Updated weights for policy 0, policy_version 1076560 (0.0011) [2023-12-26 23:12:53,341][105692] Updated weights for policy 0, policy_version 1076570 (0.0011) [2023-12-26 23:12:53,510][105620] Updated weights for policy 1, policy_version 1077670 (0.0006) [2023-12-26 23:12:53,563][105620] Updated weights for policy 1, policy_version 1077680 (0.0008) [2023-12-26 23:12:53,623][105620] Updated weights for policy 1, policy_version 1077690 (0.0009) [2023-12-26 23:12:53,919][105692] Updated weights for policy 0, policy_version 1076580 (0.0008) [2023-12-26 23:12:53,969][105692] Updated weights for policy 0, policy_version 1076590 (0.0009) [2023-12-26 23:12:54,022][105692] Updated weights for policy 0, policy_version 1076600 (0.0009) [2023-12-26 23:12:54,335][105620] Updated weights for policy 1, policy_version 1077700 (0.0009) [2023-12-26 23:12:54,386][105620] Updated weights for policy 1, policy_version 1077710 (0.0008) [2023-12-26 23:12:54,447][105620] Updated weights for policy 1, policy_version 1077720 (0.0008) [2023-12-26 23:12:54,748][105692] Updated weights for policy 0, policy_version 1076610 (0.0008) [2023-12-26 23:12:54,802][105692] Updated weights for policy 0, policy_version 1076620 (0.0005) [2023-12-26 23:12:54,849][105692] Updated weights for policy 0, policy_version 1076630 (0.0005) [2023-12-26 23:12:54,892][105692] Updated weights for policy 0, policy_version 1076640 (0.0005) [2023-12-26 23:12:55,155][105620] Updated weights for policy 1, policy_version 1077730 (0.0007) [2023-12-26 23:12:55,217][105620] Updated weights for policy 1, policy_version 1077740 (0.0010) [2023-12-26 23:12:55,271][105620] Updated weights for policy 1, policy_version 1077750 (0.0010) [2023-12-26 23:12:55,327][105620] Updated weights for policy 1, policy_version 1077760 (0.0005) [2023-12-26 23:12:55,520][105692] Updated weights for policy 0, policy_version 1076650 (0.0010) [2023-12-26 23:12:55,571][105692] Updated weights for policy 0, policy_version 1076660 (0.0010) [2023-12-26 23:12:55,622][105692] Updated weights for policy 0, policy_version 1076670 (0.0010) [2023-12-26 23:12:56,014][105620] Updated weights for policy 1, policy_version 1077770 (0.0010) [2023-12-26 23:12:56,059][105620] Updated weights for policy 1, policy_version 1077780 (0.0010) [2023-12-26 23:12:56,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19797.4, 300 sec: 19688.6). Total num frames: 551608320. Throughput: 0: 9975.1, 1: 9838.8. Samples: 551620604. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:12:56,062][104569] Avg episode reward: [(0, '9256.769'), (1, '8641.303')] [2023-12-26 23:12:56,115][105620] Updated weights for policy 1, policy_version 1077790 (0.0011) [2023-12-26 23:12:56,394][105692] Updated weights for policy 0, policy_version 1076680 (0.0010) [2023-12-26 23:12:56,455][105692] Updated weights for policy 0, policy_version 1076690 (0.0010) [2023-12-26 23:12:56,508][105692] Updated weights for policy 0, policy_version 1076700 (0.0010) [2023-12-26 23:12:56,763][105620] Updated weights for policy 1, policy_version 1077800 (0.0006) [2023-12-26 23:12:56,819][105620] Updated weights for policy 1, policy_version 1077810 (0.0005) [2023-12-26 23:12:56,874][105620] Updated weights for policy 1, policy_version 1077820 (0.0005) [2023-12-26 23:12:57,235][105692] Updated weights for policy 0, policy_version 1076710 (0.0010) [2023-12-26 23:12:57,296][105692] Updated weights for policy 0, policy_version 1076720 (0.0010) [2023-12-26 23:12:57,361][105692] Updated weights for policy 0, policy_version 1076730 (0.0010) [2023-12-26 23:12:57,439][105620] Updated weights for policy 1, policy_version 1077830 (0.0008) [2023-12-26 23:12:57,505][105620] Updated weights for policy 1, policy_version 1077840 (0.0010) [2023-12-26 23:12:57,571][105620] Updated weights for policy 1, policy_version 1077850 (0.0010) [2023-12-26 23:12:58,154][105692] Updated weights for policy 0, policy_version 1076740 (0.0008) [2023-12-26 23:12:58,214][105692] Updated weights for policy 0, policy_version 1076750 (0.0008) [2023-12-26 23:12:58,268][105692] Updated weights for policy 0, policy_version 1076760 (0.0006) [2023-12-26 23:12:58,272][105620] Updated weights for policy 1, policy_version 1077860 (0.0011) [2023-12-26 23:12:58,336][105620] Updated weights for policy 1, policy_version 1077870 (0.0010) [2023-12-26 23:12:58,398][105620] Updated weights for policy 1, policy_version 1077880 (0.0012) [2023-12-26 23:12:59,057][105692] Updated weights for policy 0, policy_version 1076770 (0.0006) [2023-12-26 23:12:59,118][105692] Updated weights for policy 0, policy_version 1076780 (0.0008) [2023-12-26 23:12:59,172][105692] Updated weights for policy 0, policy_version 1076790 (0.0008) [2023-12-26 23:12:59,226][105692] Updated weights for policy 0, policy_version 1076800 (0.0008) [2023-12-26 23:12:59,238][105620] Updated weights for policy 1, policy_version 1077890 (0.0009) [2023-12-26 23:12:59,302][105620] Updated weights for policy 1, policy_version 1077900 (0.0008) [2023-12-26 23:12:59,374][105620] Updated weights for policy 1, policy_version 1077910 (0.0008) [2023-12-26 23:12:59,427][105620] Updated weights for policy 1, policy_version 1077920 (0.0007) [2023-12-26 23:12:59,952][105692] Updated weights for policy 0, policy_version 1076810 (0.0008) [2023-12-26 23:13:00,014][105692] Updated weights for policy 0, policy_version 1076820 (0.0006) [2023-12-26 23:13:00,081][105692] Updated weights for policy 0, policy_version 1076830 (0.0005) [2023-12-26 23:13:00,205][105620] Updated weights for policy 1, policy_version 1077930 (0.0011) [2023-12-26 23:13:00,270][105620] Updated weights for policy 1, policy_version 1077940 (0.0010) [2023-12-26 23:13:00,337][105620] Updated weights for policy 1, policy_version 1077950 (0.0011) [2023-12-26 23:13:00,723][105692] Updated weights for policy 0, policy_version 1076840 (0.0008) [2023-12-26 23:13:00,783][105692] Updated weights for policy 0, policy_version 1076850 (0.0009) [2023-12-26 23:13:00,846][105692] Updated weights for policy 0, policy_version 1076860 (0.0009) [2023-12-26 23:13:00,964][105620] Updated weights for policy 1, policy_version 1077960 (0.0006) [2023-12-26 23:13:01,024][105620] Updated weights for policy 1, policy_version 1077970 (0.0006) [2023-12-26 23:13:01,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 551706624. Throughput: 0: 9918.4, 1: 9891.7. Samples: 551678796. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:13:01,063][104569] Avg episode reward: [(0, '9259.068'), (1, '8961.724')] [2023-12-26 23:13:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001076864_275718144.pth... [2023-12-26 23:13:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001075712_275423232.pth [2023-12-26 23:13:01,084][105620] Updated weights for policy 1, policy_version 1077980 (0.0008) [2023-12-26 23:13:01,107][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001077984_275996672.pth... [2023-12-26 23:13:01,111][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001076832_275701760.pth [2023-12-26 23:13:01,615][105692] Updated weights for policy 0, policy_version 1076870 (0.0009) [2023-12-26 23:13:01,679][105692] Updated weights for policy 0, policy_version 1076880 (0.0010) [2023-12-26 23:13:01,746][105692] Updated weights for policy 0, policy_version 1076891 (0.0009) [2023-12-26 23:13:01,795][105620] Updated weights for policy 1, policy_version 1077990 (0.0009) [2023-12-26 23:13:01,842][105620] Updated weights for policy 1, policy_version 1078000 (0.0008) [2023-12-26 23:13:01,892][105620] Updated weights for policy 1, policy_version 1078010 (0.0008) [2023-12-26 23:13:02,440][105692] Updated weights for policy 0, policy_version 1076901 (0.0005) [2023-12-26 23:13:02,496][105692] Updated weights for policy 0, policy_version 1076911 (0.0007) [2023-12-26 23:13:02,551][105692] Updated weights for policy 0, policy_version 1076921 (0.0006) [2023-12-26 23:13:02,553][105620] Updated weights for policy 1, policy_version 1078020 (0.0007) [2023-12-26 23:13:02,600][105620] Updated weights for policy 1, policy_version 1078030 (0.0008) [2023-12-26 23:13:02,650][105620] Updated weights for policy 1, policy_version 1078040 (0.0009) [2023-12-26 23:13:03,279][105692] Updated weights for policy 0, policy_version 1076931 (0.0007) [2023-12-26 23:13:03,326][105692] Updated weights for policy 0, policy_version 1076941 (0.0009) [2023-12-26 23:13:03,364][105620] Updated weights for policy 1, policy_version 1078050 (0.0008) [2023-12-26 23:13:03,378][105692] Updated weights for policy 0, policy_version 1076951 (0.0007) [2023-12-26 23:13:03,421][105620] Updated weights for policy 1, policy_version 1078060 (0.0007) [2023-12-26 23:13:03,470][105620] Updated weights for policy 1, policy_version 1078070 (0.0009) [2023-12-26 23:13:03,529][105620] Updated weights for policy 1, policy_version 1078080 (0.0009) [2023-12-26 23:13:04,123][105692] Updated weights for policy 0, policy_version 1076961 (0.0007) [2023-12-26 23:13:04,192][105692] Updated weights for policy 0, policy_version 1076971 (0.0006) [2023-12-26 23:13:04,251][105692] Updated weights for policy 0, policy_version 1076981 (0.0005) [2023-12-26 23:13:04,320][105692] Updated weights for policy 0, policy_version 1076991 (0.0006) [2023-12-26 23:13:04,339][105620] Updated weights for policy 1, policy_version 1078090 (0.0007) [2023-12-26 23:13:04,401][105620] Updated weights for policy 1, policy_version 1078100 (0.0008) [2023-12-26 23:13:04,448][105620] Updated weights for policy 1, policy_version 1078110 (0.0008) [2023-12-26 23:13:04,912][105692] Updated weights for policy 0, policy_version 1077001 (0.0006) [2023-12-26 23:13:04,968][105692] Updated weights for policy 0, policy_version 1077011 (0.0005) [2023-12-26 23:13:05,017][105692] Updated weights for policy 0, policy_version 1077021 (0.0006) [2023-12-26 23:13:05,261][105620] Updated weights for policy 1, policy_version 1078120 (0.0008) [2023-12-26 23:13:05,315][105620] Updated weights for policy 1, policy_version 1078130 (0.0007) [2023-12-26 23:13:05,367][105620] Updated weights for policy 1, policy_version 1078140 (0.0009) [2023-12-26 23:13:05,706][105692] Updated weights for policy 0, policy_version 1077031 (0.0009) [2023-12-26 23:13:05,765][105692] Updated weights for policy 0, policy_version 1077041 (0.0009) [2023-12-26 23:13:05,827][105692] Updated weights for policy 0, policy_version 1077051 (0.0009) [2023-12-26 23:13:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 551804928. Throughput: 0: 9768.5, 1: 9918.2. Samples: 551793872. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:13:06,063][104569] Avg episode reward: [(0, '9167.492'), (1, '9255.134')] [2023-12-26 23:13:06,097][105620] Updated weights for policy 1, policy_version 1078150 (0.0007) [2023-12-26 23:13:06,168][105620] Updated weights for policy 1, policy_version 1078160 (0.0007) [2023-12-26 23:13:06,214][105620] Updated weights for policy 1, policy_version 1078170 (0.0009) [2023-12-26 23:13:06,643][105692] Updated weights for policy 0, policy_version 1077062 (0.0009) [2023-12-26 23:13:06,705][105692] Updated weights for policy 0, policy_version 1077072 (0.0008) [2023-12-26 23:13:06,768][105692] Updated weights for policy 0, policy_version 1077082 (0.0009) [2023-12-26 23:13:06,833][105620] Updated weights for policy 1, policy_version 1078180 (0.0008) [2023-12-26 23:13:06,890][105620] Updated weights for policy 1, policy_version 1078190 (0.0005) [2023-12-26 23:13:06,946][105620] Updated weights for policy 1, policy_version 1078200 (0.0005) [2023-12-26 23:13:07,478][105620] Updated weights for policy 1, policy_version 1078210 (0.0005) [2023-12-26 23:13:07,537][105620] Updated weights for policy 1, policy_version 1078220 (0.0008) [2023-12-26 23:13:07,599][105620] Updated weights for policy 1, policy_version 1078230 (0.0009) [2023-12-26 23:13:07,637][105692] Updated weights for policy 0, policy_version 1077092 (0.0010) [2023-12-26 23:13:07,648][105620] Updated weights for policy 1, policy_version 1078240 (0.0007) [2023-12-26 23:13:07,704][105692] Updated weights for policy 0, policy_version 1077102 (0.0009) [2023-12-26 23:13:07,769][105692] Updated weights for policy 0, policy_version 1077112 (0.0010) [2023-12-26 23:13:08,255][105620] Updated weights for policy 1, policy_version 1078250 (0.0005) [2023-12-26 23:13:08,323][105620] Updated weights for policy 1, policy_version 1078260 (0.0006) [2023-12-26 23:13:08,382][105620] Updated weights for policy 1, policy_version 1078270 (0.0010) [2023-12-26 23:13:08,647][105692] Updated weights for policy 0, policy_version 1077122 (0.0008) [2023-12-26 23:13:08,709][105692] Updated weights for policy 0, policy_version 1077132 (0.0009) [2023-12-26 23:13:08,766][105692] Updated weights for policy 0, policy_version 1077142 (0.0008) [2023-12-26 23:13:08,824][105692] Updated weights for policy 0, policy_version 1077152 (0.0008) [2023-12-26 23:13:09,030][105620] Updated weights for policy 1, policy_version 1078280 (0.0010) [2023-12-26 23:13:09,082][105620] Updated weights for policy 1, policy_version 1078290 (0.0010) [2023-12-26 23:13:09,137][105620] Updated weights for policy 1, policy_version 1078300 (0.0010) [2023-12-26 23:13:09,623][105692] Updated weights for policy 0, policy_version 1077162 (0.0008) [2023-12-26 23:13:09,677][105692] Updated weights for policy 0, policy_version 1077172 (0.0010) [2023-12-26 23:13:09,733][105692] Updated weights for policy 0, policy_version 1077182 (0.0010) [2023-12-26 23:13:09,875][105620] Updated weights for policy 1, policy_version 1078310 (0.0010) [2023-12-26 23:13:09,938][105620] Updated weights for policy 1, policy_version 1078320 (0.0006) [2023-12-26 23:13:09,992][105620] Updated weights for policy 1, policy_version 1078330 (0.0006) [2023-12-26 23:13:10,528][105692] Updated weights for policy 0, policy_version 1077192 (0.0010) [2023-12-26 23:13:10,591][105692] Updated weights for policy 0, policy_version 1077202 (0.0010) [2023-12-26 23:13:10,601][105620] Updated weights for policy 1, policy_version 1078340 (0.0006) [2023-12-26 23:13:10,640][105692] Updated weights for policy 0, policy_version 1077212 (0.0009) [2023-12-26 23:13:10,655][105620] Updated weights for policy 1, policy_version 1078350 (0.0005) [2023-12-26 23:13:10,704][105620] Updated weights for policy 1, policy_version 1078360 (0.0005) [2023-12-26 23:13:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 551903232. Throughput: 0: 9717.6, 1: 10032.5. Samples: 551911456. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:13:11,062][104569] Avg episode reward: [(0, '9256.986'), (1, '9254.947')] [2023-12-26 23:13:11,408][105620] Updated weights for policy 1, policy_version 1078370 (0.0006) [2023-12-26 23:13:11,471][105620] Updated weights for policy 1, policy_version 1078380 (0.0011) [2023-12-26 23:13:11,482][105692] Updated weights for policy 0, policy_version 1077222 (0.0008) [2023-12-26 23:13:11,528][105620] Updated weights for policy 1, policy_version 1078390 (0.0011) [2023-12-26 23:13:11,529][105692] Updated weights for policy 0, policy_version 1077232 (0.0008) [2023-12-26 23:13:11,584][105692] Updated weights for policy 0, policy_version 1077242 (0.0007) [2023-12-26 23:13:11,585][105620] Updated weights for policy 1, policy_version 1078400 (0.0008) [2023-12-26 23:13:12,345][105620] Updated weights for policy 1, policy_version 1078410 (0.0008) [2023-12-26 23:13:12,358][105692] Updated weights for policy 0, policy_version 1077252 (0.0009) [2023-12-26 23:13:12,410][105620] Updated weights for policy 1, policy_version 1078420 (0.0011) [2023-12-26 23:13:12,424][105692] Updated weights for policy 0, policy_version 1077262 (0.0006) [2023-12-26 23:13:12,476][105620] Updated weights for policy 1, policy_version 1078430 (0.0011) [2023-12-26 23:13:12,490][105692] Updated weights for policy 0, policy_version 1077272 (0.0006) [2023-12-26 23:13:13,138][105692] Updated weights for policy 0, policy_version 1077282 (0.0007) [2023-12-26 23:13:13,206][105692] Updated weights for policy 0, policy_version 1077292 (0.0010) [2023-12-26 23:13:13,259][105620] Updated weights for policy 1, policy_version 1078440 (0.0011) [2023-12-26 23:13:13,262][105692] Updated weights for policy 0, policy_version 1077302 (0.0010) [2023-12-26 23:13:13,312][105620] Updated weights for policy 1, policy_version 1078450 (0.0010) [2023-12-26 23:13:13,318][105692] Updated weights for policy 0, policy_version 1077312 (0.0010) [2023-12-26 23:13:13,366][105620] Updated weights for policy 1, policy_version 1078460 (0.0010) [2023-12-26 23:13:13,935][105692] Updated weights for policy 0, policy_version 1077322 (0.0005) [2023-12-26 23:13:13,982][105692] Updated weights for policy 0, policy_version 1077332 (0.0005) [2023-12-26 23:13:14,033][105692] Updated weights for policy 0, policy_version 1077342 (0.0005) [2023-12-26 23:13:14,074][105620] Updated weights for policy 1, policy_version 1078470 (0.0011) [2023-12-26 23:13:14,137][105620] Updated weights for policy 1, policy_version 1078480 (0.0011) [2023-12-26 23:13:14,195][105620] Updated weights for policy 1, policy_version 1078490 (0.0010) [2023-12-26 23:13:14,661][105692] Updated weights for policy 0, policy_version 1077352 (0.0008) [2023-12-26 23:13:14,717][105692] Updated weights for policy 0, policy_version 1077362 (0.0008) [2023-12-26 23:13:14,776][105692] Updated weights for policy 0, policy_version 1077372 (0.0008) [2023-12-26 23:13:14,936][105620] Updated weights for policy 1, policy_version 1078500 (0.0010) [2023-12-26 23:13:14,998][105620] Updated weights for policy 1, policy_version 1078510 (0.0009) [2023-12-26 23:13:15,053][105620] Updated weights for policy 1, policy_version 1078520 (0.0010) [2023-12-26 23:13:15,561][105692] Updated weights for policy 0, policy_version 1077382 (0.0007) [2023-12-26 23:13:15,626][105692] Updated weights for policy 0, policy_version 1077392 (0.0008) [2023-12-26 23:13:15,674][105692] Updated weights for policy 0, policy_version 1077402 (0.0008) [2023-12-26 23:13:15,799][105620] Updated weights for policy 1, policy_version 1078530 (0.0011) [2023-12-26 23:13:15,847][105620] Updated weights for policy 1, policy_version 1078540 (0.0010) [2023-12-26 23:13:15,905][105620] Updated weights for policy 1, policy_version 1078550 (0.0010) [2023-12-26 23:13:15,966][105620] Updated weights for policy 1, policy_version 1078560 (0.0010) [2023-12-26 23:13:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 552001536. Throughput: 0: 9699.6, 1: 9928.9. Samples: 551967220. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:13:16,062][104569] Avg episode reward: [(0, '9076.254'), (1, '9163.365')] [2023-12-26 23:13:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001077408_275857408.pth... [2023-12-26 23:13:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001078560_276144128.pth... [2023-12-26 23:13:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001076288_275570688.pth [2023-12-26 23:13:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001077376_275841024.pth [2023-12-26 23:13:16,391][105692] Updated weights for policy 0, policy_version 1077412 (0.0007) [2023-12-26 23:13:16,442][105692] Updated weights for policy 0, policy_version 1077422 (0.0005) [2023-12-26 23:13:16,502][105692] Updated weights for policy 0, policy_version 1077432 (0.0007) [2023-12-26 23:13:16,703][105620] Updated weights for policy 1, policy_version 1078570 (0.0010) [2023-12-26 23:13:16,762][105620] Updated weights for policy 1, policy_version 1078580 (0.0010) [2023-12-26 23:13:16,830][105620] Updated weights for policy 1, policy_version 1078590 (0.0010) [2023-12-26 23:13:17,078][105692] Updated weights for policy 0, policy_version 1077442 (0.0006) [2023-12-26 23:13:17,143][105692] Updated weights for policy 0, policy_version 1077452 (0.0006) [2023-12-26 23:13:17,202][105692] Updated weights for policy 0, policy_version 1077462 (0.0005) [2023-12-26 23:13:17,250][105692] Updated weights for policy 0, policy_version 1077472 (0.0005) [2023-12-26 23:13:17,583][105620] Updated weights for policy 1, policy_version 1078600 (0.0010) [2023-12-26 23:13:17,631][105620] Updated weights for policy 1, policy_version 1078610 (0.0010) [2023-12-26 23:13:17,681][105620] Updated weights for policy 1, policy_version 1078620 (0.0010) [2023-12-26 23:13:17,832][105692] Updated weights for policy 0, policy_version 1077482 (0.0005) [2023-12-26 23:13:17,881][105692] Updated weights for policy 0, policy_version 1077492 (0.0005) [2023-12-26 23:13:17,934][105692] Updated weights for policy 0, policy_version 1077502 (0.0005) [2023-12-26 23:13:18,452][105620] Updated weights for policy 1, policy_version 1078630 (0.0007) [2023-12-26 23:13:18,510][105620] Updated weights for policy 1, policy_version 1078640 (0.0005) [2023-12-26 23:13:18,578][105620] Updated weights for policy 1, policy_version 1078650 (0.0005) [2023-12-26 23:13:18,638][105692] Updated weights for policy 0, policy_version 1077512 (0.0009) [2023-12-26 23:13:18,700][105692] Updated weights for policy 0, policy_version 1077522 (0.0009) [2023-12-26 23:13:18,766][105692] Updated weights for policy 0, policy_version 1077532 (0.0008) [2023-12-26 23:13:19,257][105620] Updated weights for policy 1, policy_version 1078660 (0.0008) [2023-12-26 23:13:19,327][105620] Updated weights for policy 1, policy_version 1078670 (0.0009) [2023-12-26 23:13:19,393][105620] Updated weights for policy 1, policy_version 1078680 (0.0013) [2023-12-26 23:13:19,503][105692] Updated weights for policy 0, policy_version 1077542 (0.0009) [2023-12-26 23:13:19,566][105692] Updated weights for policy 0, policy_version 1077552 (0.0008) [2023-12-26 23:13:19,623][105692] Updated weights for policy 0, policy_version 1077562 (0.0006) [2023-12-26 23:13:20,122][105620] Updated weights for policy 1, policy_version 1078690 (0.0009) [2023-12-26 23:13:20,190][105620] Updated weights for policy 1, policy_version 1078700 (0.0006) [2023-12-26 23:13:20,283][105620] Updated weights for policy 1, policy_version 1078710 (0.0006) [2023-12-26 23:13:20,312][105692] Updated weights for policy 0, policy_version 1077572 (0.0006) [2023-12-26 23:13:20,338][105620] Updated weights for policy 1, policy_version 1078720 (0.0009) [2023-12-26 23:13:20,372][105692] Updated weights for policy 0, policy_version 1077582 (0.0007) [2023-12-26 23:13:20,432][105692] Updated weights for policy 0, policy_version 1077592 (0.0006) [2023-12-26 23:13:20,954][105620] Updated weights for policy 1, policy_version 1078730 (0.0009) [2023-12-26 23:13:21,025][105620] Updated weights for policy 1, policy_version 1078740 (0.0008) [2023-12-26 23:13:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 552091648. Throughput: 0: 9689.2, 1: 9813.8. Samples: 552086012. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:13:21,063][104569] Avg episode reward: [(0, '9168.970'), (1, '9094.983')] [2023-12-26 23:13:21,097][105620] Updated weights for policy 1, policy_version 1078750 (0.0007) [2023-12-26 23:13:21,109][105692] Updated weights for policy 0, policy_version 1077602 (0.0006) [2023-12-26 23:13:21,180][105692] Updated weights for policy 0, policy_version 1077612 (0.0007) [2023-12-26 23:13:21,241][105692] Updated weights for policy 0, policy_version 1077622 (0.0006) [2023-12-26 23:13:21,303][105692] Updated weights for policy 0, policy_version 1077632 (0.0010) [2023-12-26 23:13:21,811][105620] Updated weights for policy 1, policy_version 1078760 (0.0007) [2023-12-26 23:13:21,869][105620] Updated weights for policy 1, policy_version 1078770 (0.0005) [2023-12-26 23:13:21,919][105620] Updated weights for policy 1, policy_version 1078780 (0.0005) [2023-12-26 23:13:22,055][105692] Updated weights for policy 0, policy_version 1077642 (0.0010) [2023-12-26 23:13:22,122][105692] Updated weights for policy 0, policy_version 1077652 (0.0009) [2023-12-26 23:13:22,198][105692] Updated weights for policy 0, policy_version 1077662 (0.0010) [2023-12-26 23:13:22,557][105620] Updated weights for policy 1, policy_version 1078790 (0.0009) [2023-12-26 23:13:22,617][105620] Updated weights for policy 1, policy_version 1078800 (0.0011) [2023-12-26 23:13:22,680][105620] Updated weights for policy 1, policy_version 1078810 (0.0011) [2023-12-26 23:13:22,894][105692] Updated weights for policy 0, policy_version 1077672 (0.0007) [2023-12-26 23:13:22,942][105692] Updated weights for policy 0, policy_version 1077682 (0.0008) [2023-12-26 23:13:22,997][105692] Updated weights for policy 0, policy_version 1077692 (0.0006) [2023-12-26 23:13:23,450][105620] Updated weights for policy 1, policy_version 1078820 (0.0010) [2023-12-26 23:13:23,517][105620] Updated weights for policy 1, policy_version 1078830 (0.0009) [2023-12-26 23:13:23,575][105620] Updated weights for policy 1, policy_version 1078840 (0.0010) [2023-12-26 23:13:23,618][105692] Updated weights for policy 0, policy_version 1077702 (0.0005) [2023-12-26 23:13:23,674][105692] Updated weights for policy 0, policy_version 1077712 (0.0005) [2023-12-26 23:13:23,742][105692] Updated weights for policy 0, policy_version 1077722 (0.0005) [2023-12-26 23:13:24,248][105692] Updated weights for policy 0, policy_version 1077732 (0.0007) [2023-12-26 23:13:24,302][105692] Updated weights for policy 0, policy_version 1077742 (0.0008) [2023-12-26 23:13:24,317][105620] Updated weights for policy 1, policy_version 1078850 (0.0009) [2023-12-26 23:13:24,350][105585] KL-divergence is very high: 122.1445 [2023-12-26 23:13:24,362][105692] Updated weights for policy 0, policy_version 1077752 (0.0009) [2023-12-26 23:13:24,388][105620] Updated weights for policy 1, policy_version 1078860 (0.0009) [2023-12-26 23:13:24,395][105585] KL-divergence is very high: 134.3490 [2023-12-26 23:13:24,451][105620] Updated weights for policy 1, policy_version 1078870 (0.0008) [2023-12-26 23:13:24,518][105620] Updated weights for policy 1, policy_version 1078880 (0.0008) [2023-12-26 23:13:24,999][105692] Updated weights for policy 0, policy_version 1077762 (0.0010) [2023-12-26 23:13:25,047][105692] Updated weights for policy 0, policy_version 1077772 (0.0010) [2023-12-26 23:13:25,102][105692] Updated weights for policy 0, policy_version 1077782 (0.0010) [2023-12-26 23:13:25,113][105620] Updated weights for policy 1, policy_version 1078890 (0.0005) [2023-12-26 23:13:25,157][105692] Updated weights for policy 0, policy_version 1077792 (0.0010) [2023-12-26 23:13:25,168][105620] Updated weights for policy 1, policy_version 1078900 (0.0006) [2023-12-26 23:13:25,221][105620] Updated weights for policy 1, policy_version 1078910 (0.0007) [2023-12-26 23:13:25,799][105620] Updated weights for policy 1, policy_version 1078920 (0.0008) [2023-12-26 23:13:25,852][105620] Updated weights for policy 1, policy_version 1078930 (0.0006) [2023-12-26 23:13:25,865][105692] Updated weights for policy 0, policy_version 1077802 (0.0010) [2023-12-26 23:13:25,903][105620] Updated weights for policy 1, policy_version 1078940 (0.0005) [2023-12-26 23:13:25,920][105692] Updated weights for policy 0, policy_version 1077812 (0.0010) [2023-12-26 23:13:25,968][105692] Updated weights for policy 0, policy_version 1077822 (0.0010) [2023-12-26 23:13:26,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.4, 300 sec: 19688.6). Total num frames: 552206336. Throughput: 0: 9759.3, 1: 9918.8. Samples: 552209716. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:13:26,062][104569] Avg episode reward: [(0, '9351.706'), (1, '9186.098')] [2023-12-26 23:13:26,651][105620] Updated weights for policy 1, policy_version 1078950 (0.0008) [2023-12-26 23:13:26,690][105692] Updated weights for policy 0, policy_version 1077832 (0.0007) [2023-12-26 23:13:26,702][105620] Updated weights for policy 1, policy_version 1078960 (0.0008) [2023-12-26 23:13:26,741][105692] Updated weights for policy 0, policy_version 1077842 (0.0005) [2023-12-26 23:13:26,758][105620] Updated weights for policy 1, policy_version 1078970 (0.0009) [2023-12-26 23:13:26,794][105692] Updated weights for policy 0, policy_version 1077852 (0.0006) [2023-12-26 23:13:27,457][105692] Updated weights for policy 0, policy_version 1077862 (0.0011) [2023-12-26 23:13:27,518][105692] Updated weights for policy 0, policy_version 1077872 (0.0009) [2023-12-26 23:13:27,543][105620] Updated weights for policy 1, policy_version 1078980 (0.0007) [2023-12-26 23:13:27,586][105692] Updated weights for policy 0, policy_version 1077882 (0.0008) [2023-12-26 23:13:27,605][105620] Updated weights for policy 1, policy_version 1078990 (0.0007) [2023-12-26 23:13:27,672][105620] Updated weights for policy 1, policy_version 1079000 (0.0006) [2023-12-26 23:13:28,214][105692] Updated weights for policy 0, policy_version 1077892 (0.0007) [2023-12-26 23:13:28,276][105692] Updated weights for policy 0, policy_version 1077902 (0.0009) [2023-12-26 23:13:28,336][105692] Updated weights for policy 0, policy_version 1077912 (0.0009) [2023-12-26 23:13:28,371][105620] Updated weights for policy 1, policy_version 1079010 (0.0006) [2023-12-26 23:13:28,416][105620] Updated weights for policy 1, policy_version 1079020 (0.0006) [2023-12-26 23:13:28,481][105620] Updated weights for policy 1, policy_version 1079030 (0.0010) [2023-12-26 23:13:28,542][105620] Updated weights for policy 1, policy_version 1079040 (0.0010) [2023-12-26 23:13:29,038][105692] Updated weights for policy 0, policy_version 1077922 (0.0008) [2023-12-26 23:13:29,093][105692] Updated weights for policy 0, policy_version 1077932 (0.0007) [2023-12-26 23:13:29,142][105692] Updated weights for policy 0, policy_version 1077942 (0.0007) [2023-12-26 23:13:29,240][105620] Updated weights for policy 1, policy_version 1079050 (0.0009) [2023-12-26 23:13:29,293][105620] Updated weights for policy 1, policy_version 1079060 (0.0008) [2023-12-26 23:13:29,351][105620] Updated weights for policy 1, policy_version 1079070 (0.0009) [2023-12-26 23:13:29,802][105692] Updated weights for policy 0, policy_version 1077953 (0.0009) [2023-12-26 23:13:29,860][105692] Updated weights for policy 0, policy_version 1077963 (0.0008) [2023-12-26 23:13:29,924][105692] Updated weights for policy 0, policy_version 1077973 (0.0007) [2023-12-26 23:13:29,973][105692] Updated weights for policy 0, policy_version 1077983 (0.0009) [2023-12-26 23:13:30,145][105620] Updated weights for policy 1, policy_version 1079080 (0.0007) [2023-12-26 23:13:30,211][105620] Updated weights for policy 1, policy_version 1079090 (0.0005) [2023-12-26 23:13:30,272][105620] Updated weights for policy 1, policy_version 1079100 (0.0006) [2023-12-26 23:13:30,664][105692] Updated weights for policy 0, policy_version 1077993 (0.0010) [2023-12-26 23:13:30,732][105692] Updated weights for policy 0, policy_version 1078003 (0.0010) [2023-12-26 23:13:30,796][105692] Updated weights for policy 0, policy_version 1078013 (0.0010) [2023-12-26 23:13:30,909][105620] Updated weights for policy 1, policy_version 1079110 (0.0007) [2023-12-26 23:13:30,955][105620] Updated weights for policy 1, policy_version 1079120 (0.0007) [2023-12-26 23:13:31,003][105620] Updated weights for policy 1, policy_version 1079130 (0.0006) [2023-12-26 23:13:31,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 552304640. Throughput: 0: 9837.8, 1: 9917.9. Samples: 552268532. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:13:31,062][104569] Avg episode reward: [(0, '9259.520'), (1, '9136.973')] [2023-12-26 23:13:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001078016_276013056.pth... [2023-12-26 23:13:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001079136_276291584.pth... [2023-12-26 23:13:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001076864_275718144.pth [2023-12-26 23:13:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001077984_275996672.pth [2023-12-26 23:13:31,519][105692] Updated weights for policy 0, policy_version 1078023 (0.0010) [2023-12-26 23:13:31,578][105692] Updated weights for policy 0, policy_version 1078033 (0.0010) [2023-12-26 23:13:31,645][105692] Updated weights for policy 0, policy_version 1078043 (0.0009) [2023-12-26 23:13:31,782][105620] Updated weights for policy 1, policy_version 1079140 (0.0009) [2023-12-26 23:13:31,842][105620] Updated weights for policy 1, policy_version 1079150 (0.0008) [2023-12-26 23:13:31,900][105620] Updated weights for policy 1, policy_version 1079160 (0.0008) [2023-12-26 23:13:32,382][105692] Updated weights for policy 0, policy_version 1078053 (0.0009) [2023-12-26 23:13:32,445][105692] Updated weights for policy 0, policy_version 1078063 (0.0011) [2023-12-26 23:13:32,510][105692] Updated weights for policy 0, policy_version 1078073 (0.0010) [2023-12-26 23:13:32,644][105620] Updated weights for policy 1, policy_version 1079170 (0.0008) [2023-12-26 23:13:32,704][105620] Updated weights for policy 1, policy_version 1079180 (0.0008) [2023-12-26 23:13:32,758][105620] Updated weights for policy 1, policy_version 1079190 (0.0008) [2023-12-26 23:13:32,816][105620] Updated weights for policy 1, policy_version 1079200 (0.0008) [2023-12-26 23:13:33,159][105692] Updated weights for policy 0, policy_version 1078083 (0.0007) [2023-12-26 23:13:33,227][105692] Updated weights for policy 0, policy_version 1078093 (0.0006) [2023-12-26 23:13:33,290][105692] Updated weights for policy 0, policy_version 1078103 (0.0008) [2023-12-26 23:13:33,549][105620] Updated weights for policy 1, policy_version 1079210 (0.0007) [2023-12-26 23:13:33,599][105620] Updated weights for policy 1, policy_version 1079220 (0.0008) [2023-12-26 23:13:33,647][105620] Updated weights for policy 1, policy_version 1079230 (0.0008) [2023-12-26 23:13:33,870][105692] Updated weights for policy 0, policy_version 1078113 (0.0011) [2023-12-26 23:13:33,925][105692] Updated weights for policy 0, policy_version 1078123 (0.0009) [2023-12-26 23:13:33,984][105692] Updated weights for policy 0, policy_version 1078133 (0.0009) [2023-12-26 23:13:34,042][105692] Updated weights for policy 0, policy_version 1078143 (0.0009) [2023-12-26 23:13:34,343][105620] Updated weights for policy 1, policy_version 1079240 (0.0007) [2023-12-26 23:13:34,402][105620] Updated weights for policy 1, policy_version 1079250 (0.0009) [2023-12-26 23:13:34,457][105620] Updated weights for policy 1, policy_version 1079260 (0.0009) [2023-12-26 23:13:34,856][105692] Updated weights for policy 0, policy_version 1078153 (0.0010) [2023-12-26 23:13:34,909][105692] Updated weights for policy 0, policy_version 1078163 (0.0010) [2023-12-26 23:13:34,962][105692] Updated weights for policy 0, policy_version 1078173 (0.0010) [2023-12-26 23:13:35,100][105620] Updated weights for policy 1, policy_version 1079270 (0.0007) [2023-12-26 23:13:35,160][105620] Updated weights for policy 1, policy_version 1079280 (0.0005) [2023-12-26 23:13:35,209][105620] Updated weights for policy 1, policy_version 1079290 (0.0008) [2023-12-26 23:13:35,785][105692] Updated weights for policy 0, policy_version 1078183 (0.0010) [2023-12-26 23:13:35,840][105692] Updated weights for policy 0, policy_version 1078193 (0.0012) [2023-12-26 23:13:35,884][105620] Updated weights for policy 1, policy_version 1079300 (0.0010) [2023-12-26 23:13:35,898][105692] Updated weights for policy 0, policy_version 1078203 (0.0008) [2023-12-26 23:13:35,938][105620] Updated weights for policy 1, policy_version 1079310 (0.0010) [2023-12-26 23:13:35,995][105620] Updated weights for policy 1, policy_version 1079320 (0.0006) [2023-12-26 23:13:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 552402944. Throughput: 0: 9798.0, 1: 9895.5. Samples: 552386496. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:13:36,062][104569] Avg episode reward: [(0, '9258.497'), (1, '5869.640')] [2023-12-26 23:13:36,663][105620] Updated weights for policy 1, policy_version 1079330 (0.0007) [2023-12-26 23:13:36,670][105692] Updated weights for policy 0, policy_version 1078213 (0.0007) [2023-12-26 23:13:36,727][105620] Updated weights for policy 1, policy_version 1079340 (0.0009) [2023-12-26 23:13:36,729][105692] Updated weights for policy 0, policy_version 1078223 (0.0009) [2023-12-26 23:13:36,785][105692] Updated weights for policy 0, policy_version 1078233 (0.0008) [2023-12-26 23:13:36,790][105620] Updated weights for policy 1, policy_version 1079350 (0.0009) [2023-12-26 23:13:36,846][105620] Updated weights for policy 1, policy_version 1079360 (0.0010) [2023-12-26 23:13:37,552][105692] Updated weights for policy 0, policy_version 1078243 (0.0006) [2023-12-26 23:13:37,584][105620] Updated weights for policy 1, policy_version 1079370 (0.0010) [2023-12-26 23:13:37,602][105692] Updated weights for policy 0, policy_version 1078253 (0.0008) [2023-12-26 23:13:37,637][105620] Updated weights for policy 1, policy_version 1079380 (0.0007) [2023-12-26 23:13:37,649][105692] Updated weights for policy 0, policy_version 1078263 (0.0008) [2023-12-26 23:13:37,694][105620] Updated weights for policy 1, policy_version 1079390 (0.0005) [2023-12-26 23:13:38,326][105692] Updated weights for policy 0, policy_version 1078273 (0.0009) [2023-12-26 23:13:38,353][105620] Updated weights for policy 1, policy_version 1079400 (0.0009) [2023-12-26 23:13:38,391][105692] Updated weights for policy 0, policy_version 1078283 (0.0007) [2023-12-26 23:13:38,412][105620] Updated weights for policy 1, policy_version 1079410 (0.0010) [2023-12-26 23:13:38,447][105692] Updated weights for policy 0, policy_version 1078293 (0.0006) [2023-12-26 23:13:38,476][105620] Updated weights for policy 1, policy_version 1079420 (0.0011) [2023-12-26 23:13:38,506][105692] Updated weights for policy 0, policy_version 1078303 (0.0006) [2023-12-26 23:13:39,220][105620] Updated weights for policy 1, policy_version 1079430 (0.0011) [2023-12-26 23:13:39,289][105620] Updated weights for policy 1, policy_version 1079440 (0.0011) [2023-12-26 23:13:39,295][105692] Updated weights for policy 0, policy_version 1078313 (0.0007) [2023-12-26 23:13:39,355][105620] Updated weights for policy 1, policy_version 1079450 (0.0011) [2023-12-26 23:13:39,366][105692] Updated weights for policy 0, policy_version 1078323 (0.0007) [2023-12-26 23:13:39,432][105692] Updated weights for policy 0, policy_version 1078333 (0.0008) [2023-12-26 23:13:39,977][105620] Updated weights for policy 1, policy_version 1079460 (0.0009) [2023-12-26 23:13:40,043][105620] Updated weights for policy 1, policy_version 1079470 (0.0008) [2023-12-26 23:13:40,105][105620] Updated weights for policy 1, policy_version 1079480 (0.0008) [2023-12-26 23:13:40,135][105692] Updated weights for policy 0, policy_version 1078343 (0.0007) [2023-12-26 23:13:40,186][105692] Updated weights for policy 0, policy_version 1078353 (0.0008) [2023-12-26 23:13:40,236][105692] Updated weights for policy 0, policy_version 1078363 (0.0009) [2023-12-26 23:13:40,848][105620] Updated weights for policy 1, policy_version 1079490 (0.0009) [2023-12-26 23:13:40,911][105620] Updated weights for policy 1, policy_version 1079500 (0.0009) [2023-12-26 23:13:40,970][105620] Updated weights for policy 1, policy_version 1079510 (0.0010) [2023-12-26 23:13:40,995][105692] Updated weights for policy 0, policy_version 1078373 (0.0007) [2023-12-26 23:13:41,022][105620] Updated weights for policy 1, policy_version 1079520 (0.0009) [2023-12-26 23:13:41,057][105692] Updated weights for policy 0, policy_version 1078383 (0.0008) [2023-12-26 23:13:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 552493056. Throughput: 0: 9722.0, 1: 9867.4. Samples: 552502124. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:13:41,062][104569] Avg episode reward: [(0, '9350.004'), (1, '7141.804')] [2023-12-26 23:13:41,124][105692] Updated weights for policy 0, policy_version 1078393 (0.0008) [2023-12-26 23:13:41,763][105620] Updated weights for policy 1, policy_version 1079530 (0.0009) [2023-12-26 23:13:41,824][105620] Updated weights for policy 1, policy_version 1079540 (0.0010) [2023-12-26 23:13:41,843][105692] Updated weights for policy 0, policy_version 1078403 (0.0008) [2023-12-26 23:13:41,886][105620] Updated weights for policy 1, policy_version 1079550 (0.0009) [2023-12-26 23:13:41,896][105692] Updated weights for policy 0, policy_version 1078413 (0.0005) [2023-12-26 23:13:41,961][105692] Updated weights for policy 0, policy_version 1078423 (0.0006) [2023-12-26 23:13:42,665][105620] Updated weights for policy 1, policy_version 1079560 (0.0010) [2023-12-26 23:13:42,666][105692] Updated weights for policy 0, policy_version 1078433 (0.0006) [2023-12-26 23:13:42,720][105692] Updated weights for policy 0, policy_version 1078443 (0.0006) [2023-12-26 23:13:42,721][105620] Updated weights for policy 1, policy_version 1079570 (0.0011) [2023-12-26 23:13:42,787][105620] Updated weights for policy 1, policy_version 1079580 (0.0011) [2023-12-26 23:13:42,787][105692] Updated weights for policy 0, policy_version 1078453 (0.0009) [2023-12-26 23:13:42,847][105692] Updated weights for policy 0, policy_version 1078463 (0.0008) [2023-12-26 23:13:43,541][105620] Updated weights for policy 1, policy_version 1079590 (0.0011) [2023-12-26 23:13:43,597][105620] Updated weights for policy 1, policy_version 1079600 (0.0011) [2023-12-26 23:13:43,607][105692] Updated weights for policy 0, policy_version 1078473 (0.0006) [2023-12-26 23:13:43,655][105620] Updated weights for policy 1, policy_version 1079610 (0.0010) [2023-12-26 23:13:43,666][105692] Updated weights for policy 0, policy_version 1078483 (0.0006) [2023-12-26 23:13:43,721][105692] Updated weights for policy 0, policy_version 1078493 (0.0009) [2023-12-26 23:13:44,376][105620] Updated weights for policy 1, policy_version 1079620 (0.0008) [2023-12-26 23:13:44,438][105620] Updated weights for policy 1, policy_version 1079630 (0.0010) [2023-12-26 23:13:44,459][105692] Updated weights for policy 0, policy_version 1078503 (0.0008) [2023-12-26 23:13:44,500][105620] Updated weights for policy 1, policy_version 1079640 (0.0010) [2023-12-26 23:13:44,509][105692] Updated weights for policy 0, policy_version 1078513 (0.0007) [2023-12-26 23:13:44,565][105692] Updated weights for policy 0, policy_version 1078523 (0.0007) [2023-12-26 23:13:45,228][105620] Updated weights for policy 1, policy_version 1079650 (0.0011) [2023-12-26 23:13:45,244][105692] Updated weights for policy 0, policy_version 1078533 (0.0006) [2023-12-26 23:13:45,291][105620] Updated weights for policy 1, policy_version 1079660 (0.0011) [2023-12-26 23:13:45,302][105692] Updated weights for policy 0, policy_version 1078543 (0.0006) [2023-12-26 23:13:45,354][105620] Updated weights for policy 1, policy_version 1079670 (0.0011) [2023-12-26 23:13:45,364][105692] Updated weights for policy 0, policy_version 1078553 (0.0006) [2023-12-26 23:13:45,417][105620] Updated weights for policy 1, policy_version 1079680 (0.0011) [2023-12-26 23:13:46,055][105692] Updated weights for policy 0, policy_version 1078563 (0.0006) [2023-12-26 23:13:46,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19524.4, 300 sec: 19633.0). Total num frames: 552583168. Throughput: 0: 9735.9, 1: 9820.7. Samples: 552558840. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:13:46,062][104569] Avg episode reward: [(0, '9350.996'), (1, '8930.077')] [2023-12-26 23:13:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001079680_276430848.pth... [2023-12-26 23:13:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001078560_276144128.pth [2023-12-26 23:13:46,102][105692] Updated weights for policy 0, policy_version 1078573 (0.0008) [2023-12-26 23:13:46,148][105692] Updated weights for policy 0, policy_version 1078583 (0.0007) [2023-12-26 23:13:46,160][105620] Updated weights for policy 1, policy_version 1079690 (0.0010) [2023-12-26 23:13:46,199][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001078592_276160512.pth... [2023-12-26 23:13:46,203][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001077408_275857408.pth [2023-12-26 23:13:46,218][105620] Updated weights for policy 1, policy_version 1079700 (0.0010) [2023-12-26 23:13:46,283][105620] Updated weights for policy 1, policy_version 1079710 (0.0010) [2023-12-26 23:13:46,917][105692] Updated weights for policy 0, policy_version 1078593 (0.0007) [2023-12-26 23:13:46,978][105692] Updated weights for policy 0, policy_version 1078603 (0.0008) [2023-12-26 23:13:47,021][105620] Updated weights for policy 1, policy_version 1079720 (0.0010) [2023-12-26 23:13:47,035][105692] Updated weights for policy 0, policy_version 1078613 (0.0006) [2023-12-26 23:13:47,080][105620] Updated weights for policy 1, policy_version 1079730 (0.0010) [2023-12-26 23:13:47,089][105692] Updated weights for policy 0, policy_version 1078623 (0.0006) [2023-12-26 23:13:47,138][105620] Updated weights for policy 1, policy_version 1079740 (0.0010) [2023-12-26 23:13:47,820][105692] Updated weights for policy 0, policy_version 1078633 (0.0008) [2023-12-26 23:13:47,868][105692] Updated weights for policy 0, policy_version 1078643 (0.0005) [2023-12-26 23:13:47,878][105620] Updated weights for policy 1, policy_version 1079750 (0.0010) [2023-12-26 23:13:47,920][105692] Updated weights for policy 0, policy_version 1078653 (0.0005) [2023-12-26 23:13:47,929][105620] Updated weights for policy 1, policy_version 1079760 (0.0010) [2023-12-26 23:13:47,984][105620] Updated weights for policy 1, policy_version 1079770 (0.0010) [2023-12-26 23:13:48,677][105620] Updated weights for policy 1, policy_version 1079780 (0.0010) [2023-12-26 23:13:48,706][105692] Updated weights for policy 0, policy_version 1078663 (0.0006) [2023-12-26 23:13:48,740][105620] Updated weights for policy 1, policy_version 1079790 (0.0010) [2023-12-26 23:13:48,763][105692] Updated weights for policy 0, policy_version 1078673 (0.0005) [2023-12-26 23:13:48,807][105620] Updated weights for policy 1, policy_version 1079800 (0.0011) [2023-12-26 23:13:48,818][105692] Updated weights for policy 0, policy_version 1078683 (0.0005) [2023-12-26 23:13:49,499][105692] Updated weights for policy 0, policy_version 1078693 (0.0005) [2023-12-26 23:13:49,510][105620] Updated weights for policy 1, policy_version 1079810 (0.0010) [2023-12-26 23:13:49,547][105692] Updated weights for policy 0, policy_version 1078703 (0.0006) [2023-12-26 23:13:49,575][105620] Updated weights for policy 1, policy_version 1079820 (0.0008) [2023-12-26 23:13:49,598][105692] Updated weights for policy 0, policy_version 1078713 (0.0005) [2023-12-26 23:13:49,632][105620] Updated weights for policy 1, policy_version 1079830 (0.0011) [2023-12-26 23:13:49,692][105620] Updated weights for policy 1, policy_version 1079840 (0.0010) [2023-12-26 23:13:50,303][105692] Updated weights for policy 0, policy_version 1078723 (0.0007) [2023-12-26 23:13:50,363][105692] Updated weights for policy 0, policy_version 1078733 (0.0008) [2023-12-26 23:13:50,419][105692] Updated weights for policy 0, policy_version 1078743 (0.0008) [2023-12-26 23:13:50,450][105620] Updated weights for policy 1, policy_version 1079850 (0.0008) [2023-12-26 23:13:50,507][105620] Updated weights for policy 1, policy_version 1079860 (0.0008) [2023-12-26 23:13:50,569][105620] Updated weights for policy 1, policy_version 1079870 (0.0009) [2023-12-26 23:13:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 552681472. Throughput: 0: 9764.0, 1: 9809.0. Samples: 552674656. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-12-26 23:13:51,063][104569] Avg episode reward: [(0, '9351.504'), (1, '9073.160')] [2023-12-26 23:13:51,066][105692] Updated weights for policy 0, policy_version 1078753 (0.0007) [2023-12-26 23:13:51,130][105692] Updated weights for policy 0, policy_version 1078763 (0.0008) [2023-12-26 23:13:51,200][105692] Updated weights for policy 0, policy_version 1078773 (0.0009) [2023-12-26 23:13:51,266][105692] Updated weights for policy 0, policy_version 1078783 (0.0008) [2023-12-26 23:13:51,394][105620] Updated weights for policy 1, policy_version 1079880 (0.0008) [2023-12-26 23:13:51,462][105620] Updated weights for policy 1, policy_version 1079890 (0.0009) [2023-12-26 23:13:51,524][105620] Updated weights for policy 1, policy_version 1079900 (0.0010) [2023-12-26 23:13:51,950][105692] Updated weights for policy 0, policy_version 1078793 (0.0009) [2023-12-26 23:13:52,015][105692] Updated weights for policy 0, policy_version 1078803 (0.0008) [2023-12-26 23:13:52,069][105692] Updated weights for policy 0, policy_version 1078813 (0.0009) [2023-12-26 23:13:52,305][105620] Updated weights for policy 1, policy_version 1079910 (0.0009) [2023-12-26 23:13:52,373][105620] Updated weights for policy 1, policy_version 1079920 (0.0009) [2023-12-26 23:13:52,434][105620] Updated weights for policy 1, policy_version 1079930 (0.0009) [2023-12-26 23:13:52,817][105692] Updated weights for policy 0, policy_version 1078823 (0.0009) [2023-12-26 23:13:52,877][105692] Updated weights for policy 0, policy_version 1078833 (0.0008) [2023-12-26 23:13:52,939][105692] Updated weights for policy 0, policy_version 1078843 (0.0009) [2023-12-26 23:13:53,183][105620] Updated weights for policy 1, policy_version 1079940 (0.0009) [2023-12-26 23:13:53,241][105620] Updated weights for policy 1, policy_version 1079950 (0.0009) [2023-12-26 23:13:53,302][105620] Updated weights for policy 1, policy_version 1079960 (0.0009) [2023-12-26 23:13:53,699][105692] Updated weights for policy 0, policy_version 1078853 (0.0009) [2023-12-26 23:13:53,746][105692] Updated weights for policy 0, policy_version 1078863 (0.0009) [2023-12-26 23:13:53,807][105692] Updated weights for policy 0, policy_version 1078873 (0.0009) [2023-12-26 23:13:54,039][105620] Updated weights for policy 1, policy_version 1079970 (0.0006) [2023-12-26 23:13:54,099][105620] Updated weights for policy 1, policy_version 1079980 (0.0006) [2023-12-26 23:13:54,160][105620] Updated weights for policy 1, policy_version 1079990 (0.0009) [2023-12-26 23:13:54,224][105620] Updated weights for policy 1, policy_version 1080000 (0.0007) [2023-12-26 23:13:54,607][105692] Updated weights for policy 0, policy_version 1078883 (0.0008) [2023-12-26 23:13:54,661][105692] Updated weights for policy 0, policy_version 1078893 (0.0009) [2023-12-26 23:13:54,719][105692] Updated weights for policy 0, policy_version 1078903 (0.0009) [2023-12-26 23:13:54,875][105620] Updated weights for policy 1, policy_version 1080010 (0.0009) [2023-12-26 23:13:54,922][105620] Updated weights for policy 1, policy_version 1080020 (0.0008) [2023-12-26 23:13:54,984][105620] Updated weights for policy 1, policy_version 1080031 (0.0010) [2023-12-26 23:13:55,490][105692] Updated weights for policy 0, policy_version 1078913 (0.0009) [2023-12-26 23:13:55,538][105692] Updated weights for policy 0, policy_version 1078923 (0.0009) [2023-12-26 23:13:55,598][105692] Updated weights for policy 0, policy_version 1078934 (0.0010) [2023-12-26 23:13:55,651][105692] Updated weights for policy 0, policy_version 1078944 (0.0010) [2023-12-26 23:13:55,719][105620] Updated weights for policy 1, policy_version 1080041 (0.0006) [2023-12-26 23:13:55,785][105620] Updated weights for policy 1, policy_version 1080051 (0.0006) [2023-12-26 23:13:55,853][105620] Updated weights for policy 1, policy_version 1080061 (0.0006) [2023-12-26 23:13:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 552779776. Throughput: 0: 9830.9, 1: 9645.7. Samples: 552787904. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:13:56,063][104569] Avg episode reward: [(0, '9263.937'), (1, '9347.425')] [2023-12-26 23:13:56,432][105692] Updated weights for policy 0, policy_version 1078954 (0.0010) [2023-12-26 23:13:56,480][105692] Updated weights for policy 0, policy_version 1078964 (0.0010) [2023-12-26 23:13:56,526][105620] Updated weights for policy 1, policy_version 1080071 (0.0006) [2023-12-26 23:13:56,528][105692] Updated weights for policy 0, policy_version 1078974 (0.0010) [2023-12-26 23:13:56,572][105620] Updated weights for policy 1, policy_version 1080081 (0.0007) [2023-12-26 23:13:56,619][105620] Updated weights for policy 1, policy_version 1080091 (0.0007) [2023-12-26 23:13:57,223][105692] Updated weights for policy 0, policy_version 1078985 (0.0010) [2023-12-26 23:13:57,269][105692] Updated weights for policy 0, policy_version 1078995 (0.0007) [2023-12-26 23:13:57,315][105620] Updated weights for policy 1, policy_version 1080101 (0.0008) [2023-12-26 23:13:57,325][105692] Updated weights for policy 0, policy_version 1079005 (0.0006) [2023-12-26 23:13:57,370][105620] Updated weights for policy 1, policy_version 1080111 (0.0008) [2023-12-26 23:13:57,421][105620] Updated weights for policy 1, policy_version 1080121 (0.0008) [2023-12-26 23:13:57,916][105692] Updated weights for policy 0, policy_version 1079015 (0.0006) [2023-12-26 23:13:57,986][105692] Updated weights for policy 0, policy_version 1079025 (0.0006) [2023-12-26 23:13:58,046][105692] Updated weights for policy 0, policy_version 1079035 (0.0006) [2023-12-26 23:13:58,089][105620] Updated weights for policy 1, policy_version 1080131 (0.0007) [2023-12-26 23:13:58,151][105620] Updated weights for policy 1, policy_version 1080141 (0.0010) [2023-12-26 23:13:58,214][105620] Updated weights for policy 1, policy_version 1080151 (0.0008) [2023-12-26 23:13:58,753][105692] Updated weights for policy 0, policy_version 1079045 (0.0006) [2023-12-26 23:13:58,829][105692] Updated weights for policy 0, policy_version 1079055 (0.0008) [2023-12-26 23:13:58,874][105585] KL-divergence is very high: 149.3551 [2023-12-26 23:13:58,901][105692] Updated weights for policy 0, policy_version 1079065 (0.0008) [2023-12-26 23:13:58,926][105585] KL-divergence is very high: 160.3468 [2023-12-26 23:13:59,019][105620] Updated weights for policy 1, policy_version 1080161 (0.0008) [2023-12-26 23:13:59,074][105620] Updated weights for policy 1, policy_version 1080171 (0.0006) [2023-12-26 23:13:59,138][105620] Updated weights for policy 1, policy_version 1080181 (0.0008) [2023-12-26 23:13:59,189][105620] Updated weights for policy 1, policy_version 1080191 (0.0008) [2023-12-26 23:13:59,658][105692] Updated weights for policy 0, policy_version 1079075 (0.0008) [2023-12-26 23:13:59,710][105692] Updated weights for policy 0, policy_version 1079086 (0.0009) [2023-12-26 23:13:59,764][105692] Updated weights for policy 0, policy_version 1079096 (0.0008) [2023-12-26 23:13:59,930][105620] Updated weights for policy 1, policy_version 1080201 (0.0009) [2023-12-26 23:13:59,999][105620] Updated weights for policy 1, policy_version 1080211 (0.0006) [2023-12-26 23:14:00,062][105620] Updated weights for policy 1, policy_version 1080221 (0.0006) [2023-12-26 23:14:00,382][105692] Updated weights for policy 0, policy_version 1079106 (0.0006) [2023-12-26 23:14:00,448][105692] Updated weights for policy 0, policy_version 1079116 (0.0009) [2023-12-26 23:14:00,512][105692] Updated weights for policy 0, policy_version 1079126 (0.0007) [2023-12-26 23:14:00,565][105692] Updated weights for policy 0, policy_version 1079136 (0.0005) [2023-12-26 23:14:00,778][105620] Updated weights for policy 1, policy_version 1080231 (0.0010) [2023-12-26 23:14:00,828][105620] Updated weights for policy 1, policy_version 1080241 (0.0009) [2023-12-26 23:14:00,886][105620] Updated weights for policy 1, policy_version 1080251 (0.0005) [2023-12-26 23:14:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 552878080. Throughput: 0: 9880.8, 1: 9673.6. Samples: 552847176. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:14:01,063][104569] Avg episode reward: [(0, '9174.730'), (1, '9002.631')] [2023-12-26 23:14:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001080256_276578304.pth... [2023-12-26 23:14:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001079136_276291584.pth [2023-12-26 23:14:01,109][105692] Updated weights for policy 0, policy_version 1079146 (0.0006) [2023-12-26 23:14:01,173][105692] Updated weights for policy 0, policy_version 1079156 (0.0010) [2023-12-26 23:14:01,232][105692] Updated weights for policy 0, policy_version 1079166 (0.0006) [2023-12-26 23:14:01,243][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001079168_276307968.pth... [2023-12-26 23:14:01,247][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001078016_276013056.pth [2023-12-26 23:14:01,576][105620] Updated weights for policy 1, policy_version 1080261 (0.0006) [2023-12-26 23:14:01,636][105620] Updated weights for policy 1, policy_version 1080271 (0.0007) [2023-12-26 23:14:01,693][105620] Updated weights for policy 1, policy_version 1080281 (0.0008) [2023-12-26 23:14:01,869][105692] Updated weights for policy 0, policy_version 1079176 (0.0010) [2023-12-26 23:14:01,923][105692] Updated weights for policy 0, policy_version 1079186 (0.0010) [2023-12-26 23:14:01,983][105692] Updated weights for policy 0, policy_version 1079196 (0.0010) [2023-12-26 23:14:02,523][105620] Updated weights for policy 1, policy_version 1080291 (0.0009) [2023-12-26 23:14:02,581][105620] Updated weights for policy 1, policy_version 1080301 (0.0009) [2023-12-26 23:14:02,620][105692] Updated weights for policy 0, policy_version 1079206 (0.0010) [2023-12-26 23:14:02,641][105620] Updated weights for policy 1, policy_version 1080311 (0.0007) [2023-12-26 23:14:02,675][105692] Updated weights for policy 0, policy_version 1079216 (0.0008) [2023-12-26 23:14:02,741][105692] Updated weights for policy 0, policy_version 1079226 (0.0005) [2023-12-26 23:14:03,285][105692] Updated weights for policy 0, policy_version 1079236 (0.0006) [2023-12-26 23:14:03,340][105692] Updated weights for policy 0, policy_version 1079246 (0.0005) [2023-12-26 23:14:03,398][105692] Updated weights for policy 0, policy_version 1079256 (0.0005) [2023-12-26 23:14:03,499][105620] Updated weights for policy 1, policy_version 1080321 (0.0008) [2023-12-26 23:14:03,562][105620] Updated weights for policy 1, policy_version 1080331 (0.0009) [2023-12-26 23:14:03,611][105620] Updated weights for policy 1, policy_version 1080341 (0.0009) [2023-12-26 23:14:03,657][105620] Updated weights for policy 1, policy_version 1080351 (0.0009) [2023-12-26 23:14:03,984][105692] Updated weights for policy 0, policy_version 1079266 (0.0006) [2023-12-26 23:14:04,046][105692] Updated weights for policy 0, policy_version 1079276 (0.0006) [2023-12-26 23:14:04,107][105692] Updated weights for policy 0, policy_version 1079286 (0.0008) [2023-12-26 23:14:04,166][105692] Updated weights for policy 0, policy_version 1079296 (0.0009) [2023-12-26 23:14:04,456][105620] Updated weights for policy 1, policy_version 1080361 (0.0009) [2023-12-26 23:14:04,519][105620] Updated weights for policy 1, policy_version 1080371 (0.0007) [2023-12-26 23:14:04,580][105620] Updated weights for policy 1, policy_version 1080381 (0.0008) [2023-12-26 23:14:04,915][105692] Updated weights for policy 0, policy_version 1079306 (0.0006) [2023-12-26 23:14:04,969][105692] Updated weights for policy 0, policy_version 1079316 (0.0005) [2023-12-26 23:14:05,033][105692] Updated weights for policy 0, policy_version 1079326 (0.0006) [2023-12-26 23:14:05,209][105620] Updated weights for policy 1, policy_version 1080391 (0.0006) [2023-12-26 23:14:05,258][105620] Updated weights for policy 1, policy_version 1080401 (0.0005) [2023-12-26 23:14:05,304][105620] Updated weights for policy 1, policy_version 1080411 (0.0005) [2023-12-26 23:14:05,575][105692] Updated weights for policy 0, policy_version 1079336 (0.0010) [2023-12-26 23:14:05,642][105692] Updated weights for policy 0, policy_version 1079346 (0.0007) [2023-12-26 23:14:05,701][105692] Updated weights for policy 0, policy_version 1079356 (0.0005) [2023-12-26 23:14:05,986][105620] Updated weights for policy 1, policy_version 1080421 (0.0008) [2023-12-26 23:14:06,038][105620] Updated weights for policy 1, policy_version 1080431 (0.0010) [2023-12-26 23:14:06,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 552976384. Throughput: 0: 9934.4, 1: 9626.8. Samples: 552966268. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:14:06,062][104569] Avg episode reward: [(0, '9260.986'), (1, '8910.823')] [2023-12-26 23:14:06,086][105620] Updated weights for policy 1, policy_version 1080441 (0.0010) [2023-12-26 23:14:06,285][105692] Updated weights for policy 0, policy_version 1079366 (0.0005) [2023-12-26 23:14:06,345][105692] Updated weights for policy 0, policy_version 1079376 (0.0005) [2023-12-26 23:14:06,404][105692] Updated weights for policy 0, policy_version 1079386 (0.0005) [2023-12-26 23:14:06,864][105620] Updated weights for policy 1, policy_version 1080451 (0.0013) [2023-12-26 23:14:06,922][105620] Updated weights for policy 1, policy_version 1080461 (0.0009) [2023-12-26 23:14:06,986][105620] Updated weights for policy 1, policy_version 1080471 (0.0005) [2023-12-26 23:14:07,025][105692] Updated weights for policy 0, policy_version 1079396 (0.0007) [2023-12-26 23:14:07,081][105692] Updated weights for policy 0, policy_version 1079406 (0.0007) [2023-12-26 23:14:07,141][105692] Updated weights for policy 0, policy_version 1079416 (0.0006) [2023-12-26 23:14:07,717][105620] Updated weights for policy 1, policy_version 1080481 (0.0007) [2023-12-26 23:14:07,768][105620] Updated weights for policy 1, policy_version 1080491 (0.0010) [2023-12-26 23:14:07,830][105620] Updated weights for policy 1, policy_version 1080501 (0.0010) [2023-12-26 23:14:07,848][105692] Updated weights for policy 0, policy_version 1079426 (0.0008) [2023-12-26 23:14:07,892][105620] Updated weights for policy 1, policy_version 1080511 (0.0007) [2023-12-26 23:14:07,904][105692] Updated weights for policy 0, policy_version 1079436 (0.0007) [2023-12-26 23:14:07,962][105692] Updated weights for policy 0, policy_version 1079446 (0.0009) [2023-12-26 23:14:08,028][105692] Updated weights for policy 0, policy_version 1079456 (0.0009) [2023-12-26 23:14:08,546][105620] Updated weights for policy 1, policy_version 1080521 (0.0007) [2023-12-26 23:14:08,608][105620] Updated weights for policy 1, policy_version 1080531 (0.0010) [2023-12-26 23:14:08,669][105620] Updated weights for policy 1, policy_version 1080541 (0.0010) [2023-12-26 23:14:08,745][105692] Updated weights for policy 0, policy_version 1079466 (0.0008) [2023-12-26 23:14:08,797][105692] Updated weights for policy 0, policy_version 1079476 (0.0008) [2023-12-26 23:14:08,860][105692] Updated weights for policy 0, policy_version 1079486 (0.0009) [2023-12-26 23:14:09,434][105620] Updated weights for policy 1, policy_version 1080551 (0.0009) [2023-12-26 23:14:09,499][105620] Updated weights for policy 1, policy_version 1080561 (0.0005) [2023-12-26 23:14:09,556][105620] Updated weights for policy 1, policy_version 1080571 (0.0007) [2023-12-26 23:14:09,616][105692] Updated weights for policy 0, policy_version 1079496 (0.0009) [2023-12-26 23:14:09,668][105692] Updated weights for policy 0, policy_version 1079506 (0.0009) [2023-12-26 23:14:09,733][105692] Updated weights for policy 0, policy_version 1079516 (0.0007) [2023-12-26 23:14:10,209][105620] Updated weights for policy 1, policy_version 1080581 (0.0007) [2023-12-26 23:14:10,264][105620] Updated weights for policy 1, policy_version 1080591 (0.0007) [2023-12-26 23:14:10,321][105620] Updated weights for policy 1, policy_version 1080601 (0.0009) [2023-12-26 23:14:10,515][105692] Updated weights for policy 0, policy_version 1079526 (0.0008) [2023-12-26 23:14:10,571][105692] Updated weights for policy 0, policy_version 1079536 (0.0009) [2023-12-26 23:14:10,627][105692] Updated weights for policy 0, policy_version 1079546 (0.0009) [2023-12-26 23:14:10,986][105620] Updated weights for policy 1, policy_version 1080611 (0.0008) [2023-12-26 23:14:11,046][105620] Updated weights for policy 1, policy_version 1080621 (0.0006) [2023-12-26 23:14:11,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 553074688. Throughput: 0: 9895.5, 1: 9606.1. Samples: 553087288. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:14:11,062][104569] Avg episode reward: [(0, '9170.117'), (1, '8985.287')] [2023-12-26 23:14:11,108][105620] Updated weights for policy 1, policy_version 1080631 (0.0009) [2023-12-26 23:14:11,468][105692] Updated weights for policy 0, policy_version 1079556 (0.0009) [2023-12-26 23:14:11,522][105692] Updated weights for policy 0, policy_version 1079566 (0.0008) [2023-12-26 23:14:11,574][105692] Updated weights for policy 0, policy_version 1079576 (0.0008) [2023-12-26 23:14:11,874][105620] Updated weights for policy 1, policy_version 1080641 (0.0010) [2023-12-26 23:14:11,935][105620] Updated weights for policy 1, policy_version 1080651 (0.0009) [2023-12-26 23:14:11,994][105620] Updated weights for policy 1, policy_version 1080661 (0.0009) [2023-12-26 23:14:12,056][105620] Updated weights for policy 1, policy_version 1080671 (0.0009) [2023-12-26 23:14:12,345][105692] Updated weights for policy 0, policy_version 1079586 (0.0009) [2023-12-26 23:14:12,412][105692] Updated weights for policy 0, policy_version 1079596 (0.0009) [2023-12-26 23:14:12,475][105692] Updated weights for policy 0, policy_version 1079606 (0.0008) [2023-12-26 23:14:12,521][105692] Updated weights for policy 0, policy_version 1079616 (0.0009) [2023-12-26 23:14:12,823][105620] Updated weights for policy 1, policy_version 1080681 (0.0009) [2023-12-26 23:14:12,880][105620] Updated weights for policy 1, policy_version 1080691 (0.0010) [2023-12-26 23:14:12,930][105620] Updated weights for policy 1, policy_version 1080701 (0.0009) [2023-12-26 23:14:13,183][105692] Updated weights for policy 0, policy_version 1079626 (0.0006) [2023-12-26 23:14:13,242][105692] Updated weights for policy 0, policy_version 1079636 (0.0009) [2023-12-26 23:14:13,301][105692] Updated weights for policy 0, policy_version 1079646 (0.0009) [2023-12-26 23:14:13,825][105620] Updated weights for policy 1, policy_version 1080711 (0.0009) [2023-12-26 23:14:13,890][105620] Updated weights for policy 1, policy_version 1080721 (0.0008) [2023-12-26 23:14:13,906][105692] Updated weights for policy 0, policy_version 1079656 (0.0007) [2023-12-26 23:14:13,944][105620] Updated weights for policy 1, policy_version 1080731 (0.0007) [2023-12-26 23:14:13,954][105692] Updated weights for policy 0, policy_version 1079666 (0.0006) [2023-12-26 23:14:14,010][105692] Updated weights for policy 0, policy_version 1079676 (0.0007) [2023-12-26 23:14:14,687][105620] Updated weights for policy 1, policy_version 1080741 (0.0009) [2023-12-26 23:14:14,749][105620] Updated weights for policy 1, policy_version 1080751 (0.0008) [2023-12-26 23:14:14,771][105692] Updated weights for policy 0, policy_version 1079686 (0.0009) [2023-12-26 23:14:14,812][105620] Updated weights for policy 1, policy_version 1080761 (0.0007) [2023-12-26 23:14:14,829][105692] Updated weights for policy 0, policy_version 1079696 (0.0010) [2023-12-26 23:14:14,889][105692] Updated weights for policy 0, policy_version 1079706 (0.0007) [2023-12-26 23:14:15,543][105692] Updated weights for policy 0, policy_version 1079716 (0.0008) [2023-12-26 23:14:15,593][105620] Updated weights for policy 1, policy_version 1080771 (0.0006) [2023-12-26 23:14:15,603][105692] Updated weights for policy 0, policy_version 1079726 (0.0008) [2023-12-26 23:14:15,651][105620] Updated weights for policy 1, policy_version 1080781 (0.0008) [2023-12-26 23:14:15,662][105692] Updated weights for policy 0, policy_version 1079736 (0.0005) [2023-12-26 23:14:15,704][105620] Updated weights for policy 1, policy_version 1080791 (0.0009) [2023-12-26 23:14:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 553172992. Throughput: 0: 9848.0, 1: 9560.9. Samples: 553141932. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:14:16,062][104569] Avg episode reward: [(0, '9082.391'), (1, '8908.557')] [2023-12-26 23:14:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001080800_276717568.pth... [2023-12-26 23:14:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001079744_276455424.pth... [2023-12-26 23:14:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001079680_276430848.pth [2023-12-26 23:14:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001078592_276160512.pth [2023-12-26 23:14:16,330][105692] Updated weights for policy 0, policy_version 1079746 (0.0006) [2023-12-26 23:14:16,343][105585] KL-divergence is very high: 109.7406 [2023-12-26 23:14:16,379][105692] Updated weights for policy 0, policy_version 1079756 (0.0010) [2023-12-26 23:14:16,400][105620] Updated weights for policy 1, policy_version 1080801 (0.0009) [2023-12-26 23:14:16,433][105692] Updated weights for policy 0, policy_version 1079766 (0.0006) [2023-12-26 23:14:16,461][105620] Updated weights for policy 1, policy_version 1080811 (0.0005) [2023-12-26 23:14:16,478][105692] Updated weights for policy 0, policy_version 1079776 (0.0005) [2023-12-26 23:14:16,507][105620] Updated weights for policy 1, policy_version 1080821 (0.0005) [2023-12-26 23:14:16,555][105620] Updated weights for policy 1, policy_version 1080831 (0.0005) [2023-12-26 23:14:17,064][105692] Updated weights for policy 0, policy_version 1079786 (0.0005) [2023-12-26 23:14:17,121][105692] Updated weights for policy 0, policy_version 1079796 (0.0005) [2023-12-26 23:14:17,173][105692] Updated weights for policy 0, policy_version 1079806 (0.0005) [2023-12-26 23:14:17,187][105620] Updated weights for policy 1, policy_version 1080841 (0.0008) [2023-12-26 23:14:17,253][105620] Updated weights for policy 1, policy_version 1080851 (0.0009) [2023-12-26 23:14:17,310][105620] Updated weights for policy 1, policy_version 1080861 (0.0008) [2023-12-26 23:14:17,743][105692] Updated weights for policy 0, policy_version 1079816 (0.0005) [2023-12-26 23:14:17,789][105692] Updated weights for policy 0, policy_version 1079826 (0.0005) [2023-12-26 23:14:17,835][105692] Updated weights for policy 0, policy_version 1079836 (0.0006) [2023-12-26 23:14:18,140][105620] Updated weights for policy 1, policy_version 1080871 (0.0009) [2023-12-26 23:14:18,197][105620] Updated weights for policy 1, policy_version 1080881 (0.0008) [2023-12-26 23:14:18,261][105620] Updated weights for policy 1, policy_version 1080891 (0.0009) [2023-12-26 23:14:18,507][105692] Updated weights for policy 0, policy_version 1079846 (0.0009) [2023-12-26 23:14:18,562][105692] Updated weights for policy 0, policy_version 1079857 (0.0010) [2023-12-26 23:14:18,623][105692] Updated weights for policy 0, policy_version 1079867 (0.0005) [2023-12-26 23:14:19,046][105620] Updated weights for policy 1, policy_version 1080901 (0.0009) [2023-12-26 23:14:19,096][105620] Updated weights for policy 1, policy_version 1080911 (0.0009) [2023-12-26 23:14:19,146][105620] Updated weights for policy 1, policy_version 1080921 (0.0008) [2023-12-26 23:14:19,271][105692] Updated weights for policy 0, policy_version 1079877 (0.0007) [2023-12-26 23:14:19,339][105692] Updated weights for policy 0, policy_version 1079887 (0.0008) [2023-12-26 23:14:19,408][105692] Updated weights for policy 0, policy_version 1079897 (0.0009) [2023-12-26 23:14:19,904][105620] Updated weights for policy 1, policy_version 1080931 (0.0008) [2023-12-26 23:14:19,979][105620] Updated weights for policy 1, policy_version 1080941 (0.0006) [2023-12-26 23:14:20,045][105620] Updated weights for policy 1, policy_version 1080951 (0.0008) [2023-12-26 23:14:20,110][105692] Updated weights for policy 0, policy_version 1079907 (0.0008) [2023-12-26 23:14:20,172][105692] Updated weights for policy 0, policy_version 1079917 (0.0005) [2023-12-26 23:14:20,229][105692] Updated weights for policy 0, policy_version 1079927 (0.0005) [2023-12-26 23:14:20,612][105620] Updated weights for policy 1, policy_version 1080961 (0.0006) [2023-12-26 23:14:20,679][105620] Updated weights for policy 1, policy_version 1080971 (0.0009) [2023-12-26 23:14:20,740][105620] Updated weights for policy 1, policy_version 1080981 (0.0010) [2023-12-26 23:14:20,801][105620] Updated weights for policy 1, policy_version 1080991 (0.0009) [2023-12-26 23:14:20,900][105692] Updated weights for policy 0, policy_version 1079937 (0.0006) [2023-12-26 23:14:20,968][105692] Updated weights for policy 0, policy_version 1079947 (0.0010) [2023-12-26 23:14:21,038][105692] Updated weights for policy 0, policy_version 1079957 (0.0007) [2023-12-26 23:14:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 553271296. Throughput: 0: 9937.3, 1: 9514.5. Samples: 553261828. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:14:21,063][104569] Avg episode reward: [(0, '9168.771'), (1, '9177.552')] [2023-12-26 23:14:21,108][105692] Updated weights for policy 0, policy_version 1079967 (0.0009) [2023-12-26 23:14:21,627][105620] Updated weights for policy 1, policy_version 1081001 (0.0009) [2023-12-26 23:14:21,690][105620] Updated weights for policy 1, policy_version 1081011 (0.0009) [2023-12-26 23:14:21,760][105620] Updated weights for policy 1, policy_version 1081021 (0.0009) [2023-12-26 23:14:21,852][105692] Updated weights for policy 0, policy_version 1079977 (0.0010) [2023-12-26 23:14:21,925][105692] Updated weights for policy 0, policy_version 1079987 (0.0010) [2023-12-26 23:14:21,984][105692] Updated weights for policy 0, policy_version 1079997 (0.0010) [2023-12-26 23:14:22,460][105620] Updated weights for policy 1, policy_version 1081031 (0.0006) [2023-12-26 23:14:22,521][105620] Updated weights for policy 1, policy_version 1081041 (0.0006) [2023-12-26 23:14:22,585][105620] Updated weights for policy 1, policy_version 1081051 (0.0006) [2023-12-26 23:14:22,678][105692] Updated weights for policy 0, policy_version 1080007 (0.0009) [2023-12-26 23:14:22,742][105692] Updated weights for policy 0, policy_version 1080017 (0.0007) [2023-12-26 23:14:22,803][105692] Updated weights for policy 0, policy_version 1080027 (0.0008) [2023-12-26 23:14:23,226][105620] Updated weights for policy 1, policy_version 1081061 (0.0008) [2023-12-26 23:14:23,279][105620] Updated weights for policy 1, policy_version 1081071 (0.0009) [2023-12-26 23:14:23,339][105620] Updated weights for policy 1, policy_version 1081081 (0.0010) [2023-12-26 23:14:23,478][105692] Updated weights for policy 0, policy_version 1080037 (0.0009) [2023-12-26 23:14:23,532][105692] Updated weights for policy 0, policy_version 1080047 (0.0008) [2023-12-26 23:14:23,587][105692] Updated weights for policy 0, policy_version 1080057 (0.0009) [2023-12-26 23:14:24,161][105620] Updated weights for policy 1, policy_version 1081091 (0.0008) [2023-12-26 23:14:24,213][105620] Updated weights for policy 1, policy_version 1081101 (0.0005) [2023-12-26 23:14:24,269][105692] Updated weights for policy 0, policy_version 1080067 (0.0007) [2023-12-26 23:14:24,272][105620] Updated weights for policy 1, policy_version 1081111 (0.0010) [2023-12-26 23:14:24,331][105692] Updated weights for policy 0, policy_version 1080077 (0.0007) [2023-12-26 23:14:24,384][105692] Updated weights for policy 0, policy_version 1080087 (0.0008) [2023-12-26 23:14:24,864][105620] Updated weights for policy 1, policy_version 1081121 (0.0010) [2023-12-26 23:14:24,935][105620] Updated weights for policy 1, policy_version 1081131 (0.0009) [2023-12-26 23:14:24,989][105620] Updated weights for policy 1, policy_version 1081141 (0.0005) [2023-12-26 23:14:25,046][105620] Updated weights for policy 1, policy_version 1081151 (0.0006) [2023-12-26 23:14:25,069][105692] Updated weights for policy 0, policy_version 1080097 (0.0007) [2023-12-26 23:14:25,115][105692] Updated weights for policy 0, policy_version 1080107 (0.0009) [2023-12-26 23:14:25,162][105692] Updated weights for policy 0, policy_version 1080117 (0.0009) [2023-12-26 23:14:25,209][105692] Updated weights for policy 0, policy_version 1080127 (0.0009) [2023-12-26 23:14:25,697][105620] Updated weights for policy 1, policy_version 1081161 (0.0008) [2023-12-26 23:14:25,755][105620] Updated weights for policy 1, policy_version 1081171 (0.0009) [2023-12-26 23:14:25,814][105620] Updated weights for policy 1, policy_version 1081181 (0.0008) [2023-12-26 23:14:26,012][105692] Updated weights for policy 0, policy_version 1080137 (0.0008) [2023-12-26 23:14:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 553369600. Throughput: 0: 9999.0, 1: 9508.4. Samples: 553379960. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:14:26,062][104569] Avg episode reward: [(0, '9256.533'), (1, '9256.152')] [2023-12-26 23:14:26,064][105692] Updated weights for policy 0, policy_version 1080147 (0.0008) [2023-12-26 23:14:26,116][105692] Updated weights for policy 0, policy_version 1080157 (0.0008) [2023-12-26 23:14:26,474][105620] Updated weights for policy 1, policy_version 1081191 (0.0007) [2023-12-26 23:14:26,522][105620] Updated weights for policy 1, policy_version 1081201 (0.0005) [2023-12-26 23:14:26,570][105620] Updated weights for policy 1, policy_version 1081211 (0.0006) [2023-12-26 23:14:26,977][105692] Updated weights for policy 0, policy_version 1080167 (0.0009) [2023-12-26 23:14:27,030][105692] Updated weights for policy 0, policy_version 1080177 (0.0008) [2023-12-26 23:14:27,079][105692] Updated weights for policy 0, policy_version 1080187 (0.0008) [2023-12-26 23:14:27,195][105620] Updated weights for policy 1, policy_version 1081221 (0.0005) [2023-12-26 23:14:27,246][105620] Updated weights for policy 1, policy_version 1081231 (0.0005) [2023-12-26 23:14:27,302][105620] Updated weights for policy 1, policy_version 1081241 (0.0005) [2023-12-26 23:14:27,880][105620] Updated weights for policy 1, policy_version 1081251 (0.0007) [2023-12-26 23:14:27,939][105620] Updated weights for policy 1, policy_version 1081261 (0.0008) [2023-12-26 23:14:27,942][105692] Updated weights for policy 0, policy_version 1080197 (0.0007) [2023-12-26 23:14:27,987][105692] Updated weights for policy 0, policy_version 1080207 (0.0006) [2023-12-26 23:14:27,993][105620] Updated weights for policy 1, policy_version 1081271 (0.0007) [2023-12-26 23:14:28,040][105692] Updated weights for policy 0, policy_version 1080217 (0.0007) [2023-12-26 23:14:28,062][105585] KL-divergence is very high: 185.0486 [2023-12-26 23:14:28,746][105692] Updated weights for policy 0, policy_version 1080227 (0.0010) [2023-12-26 23:14:28,746][105620] Updated weights for policy 1, policy_version 1081281 (0.0006) [2023-12-26 23:14:28,788][105692] Updated weights for policy 0, policy_version 1080237 (0.0008) [2023-12-26 23:14:28,790][105620] Updated weights for policy 1, policy_version 1081291 (0.0007) [2023-12-26 23:14:28,832][105692] Updated weights for policy 0, policy_version 1080247 (0.0005) [2023-12-26 23:14:28,841][105620] Updated weights for policy 1, policy_version 1081301 (0.0008) [2023-12-26 23:14:28,894][105620] Updated weights for policy 1, policy_version 1081311 (0.0009) [2023-12-26 23:14:29,626][105692] Updated weights for policy 0, policy_version 1080257 (0.0007) [2023-12-26 23:14:29,636][105620] Updated weights for policy 1, policy_version 1081321 (0.0010) [2023-12-26 23:14:29,672][105692] Updated weights for policy 0, policy_version 1080267 (0.0007) [2023-12-26 23:14:29,695][105620] Updated weights for policy 1, policy_version 1081331 (0.0010) [2023-12-26 23:14:29,725][105692] Updated weights for policy 0, policy_version 1080277 (0.0007) [2023-12-26 23:14:29,755][105620] Updated weights for policy 1, policy_version 1081341 (0.0011) [2023-12-26 23:14:29,785][105692] Updated weights for policy 0, policy_version 1080287 (0.0007) [2023-12-26 23:14:30,505][105620] Updated weights for policy 1, policy_version 1081351 (0.0010) [2023-12-26 23:14:30,559][105692] Updated weights for policy 0, policy_version 1080297 (0.0006) [2023-12-26 23:14:30,561][105620] Updated weights for policy 1, policy_version 1081361 (0.0010) [2023-12-26 23:14:30,607][105692] Updated weights for policy 0, policy_version 1080307 (0.0005) [2023-12-26 23:14:30,613][105620] Updated weights for policy 1, policy_version 1081371 (0.0010) [2023-12-26 23:14:30,652][105692] Updated weights for policy 0, policy_version 1080317 (0.0006) [2023-12-26 23:14:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 553467904. Throughput: 0: 9961.0, 1: 9590.8. Samples: 553438668. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:14:31,062][104569] Avg episode reward: [(0, '9256.937'), (1, '9164.316')] [2023-12-26 23:14:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001080320_276602880.pth... [2023-12-26 23:14:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001081376_276865024.pth... [2023-12-26 23:14:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001080256_276578304.pth [2023-12-26 23:14:31,088][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001079168_276307968.pth [2023-12-26 23:14:31,384][105620] Updated weights for policy 1, policy_version 1081381 (0.0009) [2023-12-26 23:14:31,396][105692] Updated weights for policy 0, policy_version 1080327 (0.0007) [2023-12-26 23:14:31,445][105620] Updated weights for policy 1, policy_version 1081391 (0.0009) [2023-12-26 23:14:31,457][105692] Updated weights for policy 0, policy_version 1080337 (0.0007) [2023-12-26 23:14:31,507][105620] Updated weights for policy 1, policy_version 1081401 (0.0006) [2023-12-26 23:14:31,518][105692] Updated weights for policy 0, policy_version 1080347 (0.0007) [2023-12-26 23:14:32,216][105692] Updated weights for policy 0, policy_version 1080357 (0.0008) [2023-12-26 23:14:32,262][105620] Updated weights for policy 1, policy_version 1081411 (0.0006) [2023-12-26 23:14:32,271][105692] Updated weights for policy 0, policy_version 1080367 (0.0008) [2023-12-26 23:14:32,325][105620] Updated weights for policy 1, policy_version 1081421 (0.0007) [2023-12-26 23:14:32,337][105692] Updated weights for policy 0, policy_version 1080377 (0.0006) [2023-12-26 23:14:32,391][105620] Updated weights for policy 1, policy_version 1081431 (0.0008) [2023-12-26 23:14:33,073][105692] Updated weights for policy 0, policy_version 1080387 (0.0008) [2023-12-26 23:14:33,125][105692] Updated weights for policy 0, policy_version 1080397 (0.0007) [2023-12-26 23:14:33,145][105620] Updated weights for policy 1, policy_version 1081441 (0.0009) [2023-12-26 23:14:33,193][105692] Updated weights for policy 0, policy_version 1080407 (0.0005) [2023-12-26 23:14:33,201][105620] Updated weights for policy 1, policy_version 1081451 (0.0005) [2023-12-26 23:14:33,257][105620] Updated weights for policy 1, policy_version 1081461 (0.0007) [2023-12-26 23:14:33,314][105620] Updated weights for policy 1, policy_version 1081471 (0.0009) [2023-12-26 23:14:33,862][105692] Updated weights for policy 0, policy_version 1080417 (0.0007) [2023-12-26 23:14:33,929][105692] Updated weights for policy 0, policy_version 1080427 (0.0007) [2023-12-26 23:14:33,988][105692] Updated weights for policy 0, policy_version 1080437 (0.0008) [2023-12-26 23:14:34,054][105692] Updated weights for policy 0, policy_version 1080447 (0.0008) [2023-12-26 23:14:34,077][105620] Updated weights for policy 1, policy_version 1081481 (0.0007) [2023-12-26 23:14:34,138][105620] Updated weights for policy 1, policy_version 1081491 (0.0006) [2023-12-26 23:14:34,205][105620] Updated weights for policy 1, policy_version 1081501 (0.0007) [2023-12-26 23:14:34,770][105620] Updated weights for policy 1, policy_version 1081511 (0.0009) [2023-12-26 23:14:34,782][105692] Updated weights for policy 0, policy_version 1080457 (0.0011) [2023-12-26 23:14:34,823][105620] Updated weights for policy 1, policy_version 1081521 (0.0006) [2023-12-26 23:14:34,841][105692] Updated weights for policy 0, policy_version 1080467 (0.0009) [2023-12-26 23:14:34,883][105620] Updated weights for policy 1, policy_version 1081531 (0.0006) [2023-12-26 23:14:34,901][105692] Updated weights for policy 0, policy_version 1080477 (0.0009) [2023-12-26 23:14:35,498][105692] Updated weights for policy 0, policy_version 1080487 (0.0006) [2023-12-26 23:14:35,564][105692] Updated weights for policy 0, policy_version 1080497 (0.0005) [2023-12-26 23:14:35,632][105692] Updated weights for policy 0, policy_version 1080507 (0.0007) [2023-12-26 23:14:35,651][105620] Updated weights for policy 1, policy_version 1081541 (0.0008) [2023-12-26 23:14:35,706][105620] Updated weights for policy 1, policy_version 1081551 (0.0008) [2023-12-26 23:14:35,750][105620] Updated weights for policy 1, policy_version 1081561 (0.0007) [2023-12-26 23:14:36,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 553566208. Throughput: 0: 9933.8, 1: 9603.7. Samples: 553553848. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:14:36,063][104569] Avg episode reward: [(0, '9259.703'), (1, '8980.002')] [2023-12-26 23:14:36,280][105692] Updated weights for policy 0, policy_version 1080517 (0.0008) [2023-12-26 23:14:36,330][105692] Updated weights for policy 0, policy_version 1080527 (0.0010) [2023-12-26 23:14:36,379][105692] Updated weights for policy 0, policy_version 1080537 (0.0011) [2023-12-26 23:14:36,504][105620] Updated weights for policy 1, policy_version 1081571 (0.0008) [2023-12-26 23:14:36,566][105620] Updated weights for policy 1, policy_version 1081581 (0.0008) [2023-12-26 23:14:36,592][105586] KL-divergence is very high: 106.5371 [2023-12-26 23:14:36,626][105620] Updated weights for policy 1, policy_version 1081591 (0.0008) [2023-12-26 23:14:36,638][105586] KL-divergence is very high: 115.8465 [2023-12-26 23:14:37,158][105692] Updated weights for policy 0, policy_version 1080547 (0.0011) [2023-12-26 23:14:37,220][105692] Updated weights for policy 0, policy_version 1080557 (0.0011) [2023-12-26 23:14:37,285][105692] Updated weights for policy 0, policy_version 1080567 (0.0011) [2023-12-26 23:14:37,364][105620] Updated weights for policy 1, policy_version 1081601 (0.0008) [2023-12-26 23:14:37,421][105620] Updated weights for policy 1, policy_version 1081611 (0.0008) [2023-12-26 23:14:37,477][105620] Updated weights for policy 1, policy_version 1081621 (0.0008) [2023-12-26 23:14:37,535][105620] Updated weights for policy 1, policy_version 1081631 (0.0005) [2023-12-26 23:14:37,972][105692] Updated weights for policy 0, policy_version 1080577 (0.0010) [2023-12-26 23:14:38,024][105692] Updated weights for policy 0, policy_version 1080587 (0.0005) [2023-12-26 23:14:38,070][105692] Updated weights for policy 0, policy_version 1080597 (0.0005) [2023-12-26 23:14:38,123][105692] Updated weights for policy 0, policy_version 1080607 (0.0005) [2023-12-26 23:14:38,316][105620] Updated weights for policy 1, policy_version 1081641 (0.0009) [2023-12-26 23:14:38,379][105620] Updated weights for policy 1, policy_version 1081652 (0.0010) [2023-12-26 23:14:38,432][105620] Updated weights for policy 1, policy_version 1081662 (0.0008) [2023-12-26 23:14:38,730][105692] Updated weights for policy 0, policy_version 1080617 (0.0006) [2023-12-26 23:14:38,782][105692] Updated weights for policy 0, policy_version 1080627 (0.0005) [2023-12-26 23:14:38,835][105692] Updated weights for policy 0, policy_version 1080637 (0.0005) [2023-12-26 23:14:39,312][105620] Updated weights for policy 1, policy_version 1081672 (0.0009) [2023-12-26 23:14:39,382][105620] Updated weights for policy 1, policy_version 1081682 (0.0007) [2023-12-26 23:14:39,404][105692] Updated weights for policy 0, policy_version 1080647 (0.0007) [2023-12-26 23:14:39,457][105620] Updated weights for policy 1, policy_version 1081692 (0.0009) [2023-12-26 23:14:39,469][105692] Updated weights for policy 0, policy_version 1080657 (0.0008) [2023-12-26 23:14:39,535][105692] Updated weights for policy 0, policy_version 1080667 (0.0009) [2023-12-26 23:14:40,201][105620] Updated weights for policy 1, policy_version 1081702 (0.0009) [2023-12-26 23:14:40,268][105620] Updated weights for policy 1, policy_version 1081712 (0.0011) [2023-12-26 23:14:40,275][105692] Updated weights for policy 0, policy_version 1080677 (0.0007) [2023-12-26 23:14:40,324][105620] Updated weights for policy 1, policy_version 1081722 (0.0010) [2023-12-26 23:14:40,325][105692] Updated weights for policy 0, policy_version 1080687 (0.0009) [2023-12-26 23:14:40,380][105692] Updated weights for policy 0, policy_version 1080697 (0.0008) [2023-12-26 23:14:40,970][105620] Updated weights for policy 1, policy_version 1081732 (0.0010) [2023-12-26 23:14:41,027][105620] Updated weights for policy 1, policy_version 1081742 (0.0009) [2023-12-26 23:14:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 553656320. Throughput: 0: 10033.8, 1: 9585.1. Samples: 553670752. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:14:41,062][104569] Avg episode reward: [(0, '9260.640'), (1, '9071.639')] [2023-12-26 23:14:41,090][105620] Updated weights for policy 1, policy_version 1081752 (0.0008) [2023-12-26 23:14:41,133][105692] Updated weights for policy 0, policy_version 1080707 (0.0009) [2023-12-26 23:14:41,202][105692] Updated weights for policy 0, policy_version 1080717 (0.0011) [2023-12-26 23:14:41,266][105692] Updated weights for policy 0, policy_version 1080727 (0.0009) [2023-12-26 23:14:41,837][105620] Updated weights for policy 1, policy_version 1081762 (0.0010) [2023-12-26 23:14:41,885][105620] Updated weights for policy 1, policy_version 1081772 (0.0010) [2023-12-26 23:14:41,942][105620] Updated weights for policy 1, policy_version 1081782 (0.0011) [2023-12-26 23:14:41,990][105620] Updated weights for policy 1, policy_version 1081792 (0.0010) [2023-12-26 23:14:42,010][105692] Updated weights for policy 0, policy_version 1080737 (0.0008) [2023-12-26 23:14:42,073][105692] Updated weights for policy 0, policy_version 1080747 (0.0008) [2023-12-26 23:14:42,128][105692] Updated weights for policy 0, policy_version 1080757 (0.0008) [2023-12-26 23:14:42,180][105692] Updated weights for policy 0, policy_version 1080767 (0.0008) [2023-12-26 23:14:42,773][105620] Updated weights for policy 1, policy_version 1081802 (0.0011) [2023-12-26 23:14:42,836][105620] Updated weights for policy 1, policy_version 1081812 (0.0010) [2023-12-26 23:14:42,894][105620] Updated weights for policy 1, policy_version 1081822 (0.0011) [2023-12-26 23:14:42,969][105692] Updated weights for policy 0, policy_version 1080777 (0.0008) [2023-12-26 23:14:43,028][105692] Updated weights for policy 0, policy_version 1080787 (0.0008) [2023-12-26 23:14:43,093][105692] Updated weights for policy 0, policy_version 1080797 (0.0008) [2023-12-26 23:14:43,629][105620] Updated weights for policy 1, policy_version 1081832 (0.0010) [2023-12-26 23:14:43,681][105620] Updated weights for policy 1, policy_version 1081842 (0.0010) [2023-12-26 23:14:43,746][105620] Updated weights for policy 1, policy_version 1081852 (0.0010) [2023-12-26 23:14:43,848][105692] Updated weights for policy 0, policy_version 1080807 (0.0008) [2023-12-26 23:14:43,907][105692] Updated weights for policy 0, policy_version 1080817 (0.0008) [2023-12-26 23:14:43,959][105692] Updated weights for policy 0, policy_version 1080827 (0.0008) [2023-12-26 23:14:44,464][105620] Updated weights for policy 1, policy_version 1081862 (0.0008) [2023-12-26 23:14:44,529][105620] Updated weights for policy 1, policy_version 1081872 (0.0011) [2023-12-26 23:14:44,574][105620] Updated weights for policy 1, policy_version 1081882 (0.0010) [2023-12-26 23:14:44,751][105692] Updated weights for policy 0, policy_version 1080837 (0.0009) [2023-12-26 23:14:44,819][105692] Updated weights for policy 0, policy_version 1080847 (0.0009) [2023-12-26 23:14:44,870][105692] Updated weights for policy 0, policy_version 1080857 (0.0008) [2023-12-26 23:14:45,290][105620] Updated weights for policy 1, policy_version 1081892 (0.0011) [2023-12-26 23:14:45,353][105620] Updated weights for policy 1, policy_version 1081902 (0.0010) [2023-12-26 23:14:45,419][105620] Updated weights for policy 1, policy_version 1081912 (0.0007) [2023-12-26 23:14:45,582][105692] Updated weights for policy 0, policy_version 1080867 (0.0008) [2023-12-26 23:14:45,642][105692] Updated weights for policy 0, policy_version 1080877 (0.0006) [2023-12-26 23:14:45,698][105692] Updated weights for policy 0, policy_version 1080887 (0.0008) [2023-12-26 23:14:46,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 553754624. Throughput: 0: 9980.6, 1: 9558.8. Samples: 553726444. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:14:46,062][104569] Avg episode reward: [(0, '9353.219'), (1, '9347.409')] [2023-12-26 23:14:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001081920_277004288.pth... [2023-12-26 23:14:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001080896_276750336.pth... [2023-12-26 23:14:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001080800_276717568.pth [2023-12-26 23:14:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001079744_276455424.pth [2023-12-26 23:14:46,140][105620] Updated weights for policy 1, policy_version 1081922 (0.0010) [2023-12-26 23:14:46,187][105620] Updated weights for policy 1, policy_version 1081932 (0.0006) [2023-12-26 23:14:46,239][105620] Updated weights for policy 1, policy_version 1081942 (0.0009) [2023-12-26 23:14:46,287][105620] Updated weights for policy 1, policy_version 1081952 (0.0006) [2023-12-26 23:14:46,322][105692] Updated weights for policy 0, policy_version 1080897 (0.0008) [2023-12-26 23:14:46,373][105692] Updated weights for policy 0, policy_version 1080907 (0.0005) [2023-12-26 23:14:46,432][105692] Updated weights for policy 0, policy_version 1080917 (0.0005) [2023-12-26 23:14:46,487][105692] Updated weights for policy 0, policy_version 1080927 (0.0006) [2023-12-26 23:14:46,999][105620] Updated weights for policy 1, policy_version 1081962 (0.0006) [2023-12-26 23:14:47,054][105620] Updated weights for policy 1, policy_version 1081972 (0.0010) [2023-12-26 23:14:47,102][105620] Updated weights for policy 1, policy_version 1081982 (0.0010) [2023-12-26 23:14:47,111][105692] Updated weights for policy 0, policy_version 1080937 (0.0006) [2023-12-26 23:14:47,175][105692] Updated weights for policy 0, policy_version 1080947 (0.0007) [2023-12-26 23:14:47,233][105692] Updated weights for policy 0, policy_version 1080957 (0.0008) [2023-12-26 23:14:47,825][105620] Updated weights for policy 1, policy_version 1081992 (0.0010) [2023-12-26 23:14:47,850][105692] Updated weights for policy 0, policy_version 1080967 (0.0006) [2023-12-26 23:14:47,881][105620] Updated weights for policy 1, policy_version 1082002 (0.0008) [2023-12-26 23:14:47,901][105692] Updated weights for policy 0, policy_version 1080977 (0.0005) [2023-12-26 23:14:47,946][105620] Updated weights for policy 1, policy_version 1082012 (0.0006) [2023-12-26 23:14:47,972][105692] Updated weights for policy 0, policy_version 1080987 (0.0005) [2023-12-26 23:14:48,549][105620] Updated weights for policy 1, policy_version 1082022 (0.0009) [2023-12-26 23:14:48,605][105620] Updated weights for policy 1, policy_version 1082032 (0.0009) [2023-12-26 23:14:48,664][105620] Updated weights for policy 1, policy_version 1082042 (0.0009) [2023-12-26 23:14:48,702][105692] Updated weights for policy 0, policy_version 1080997 (0.0010) [2023-12-26 23:14:48,749][105692] Updated weights for policy 0, policy_version 1081007 (0.0009) [2023-12-26 23:14:48,806][105692] Updated weights for policy 0, policy_version 1081017 (0.0008) [2023-12-26 23:14:49,469][105620] Updated weights for policy 1, policy_version 1082052 (0.0009) [2023-12-26 23:14:49,531][105620] Updated weights for policy 1, policy_version 1082062 (0.0009) [2023-12-26 23:14:49,554][105692] Updated weights for policy 0, policy_version 1081027 (0.0008) [2023-12-26 23:14:49,588][105620] Updated weights for policy 1, policy_version 1082072 (0.0010) [2023-12-26 23:14:49,606][105692] Updated weights for policy 0, policy_version 1081037 (0.0006) [2023-12-26 23:14:49,664][105692] Updated weights for policy 0, policy_version 1081047 (0.0008) [2023-12-26 23:14:50,323][105620] Updated weights for policy 1, policy_version 1082082 (0.0010) [2023-12-26 23:14:50,389][105620] Updated weights for policy 1, policy_version 1082092 (0.0011) [2023-12-26 23:14:50,448][105620] Updated weights for policy 1, policy_version 1082102 (0.0011) [2023-12-26 23:14:50,450][105692] Updated weights for policy 0, policy_version 1081057 (0.0008) [2023-12-26 23:14:50,506][105692] Updated weights for policy 0, policy_version 1081067 (0.0006) [2023-12-26 23:14:50,508][105620] Updated weights for policy 1, policy_version 1082112 (0.0011) [2023-12-26 23:14:50,561][105692] Updated weights for policy 0, policy_version 1081077 (0.0008) [2023-12-26 23:14:50,626][105692] Updated weights for policy 0, policy_version 1081087 (0.0009) [2023-12-26 23:14:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 553852928. Throughput: 0: 9870.1, 1: 9653.7. Samples: 553844840. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:14:51,063][104569] Avg episode reward: [(0, '9268.118'), (1, '9255.684')] [2023-12-26 23:14:51,129][105620] Updated weights for policy 1, policy_version 1082122 (0.0011) [2023-12-26 23:14:51,199][105620] Updated weights for policy 1, policy_version 1082132 (0.0008) [2023-12-26 23:14:51,267][105620] Updated weights for policy 1, policy_version 1082142 (0.0011) [2023-12-26 23:14:51,404][105692] Updated weights for policy 0, policy_version 1081097 (0.0008) [2023-12-26 23:14:51,454][105692] Updated weights for policy 0, policy_version 1081107 (0.0005) [2023-12-26 23:14:51,523][105692] Updated weights for policy 0, policy_version 1081117 (0.0006) [2023-12-26 23:14:52,040][105620] Updated weights for policy 1, policy_version 1082152 (0.0008) [2023-12-26 23:14:52,098][105620] Updated weights for policy 1, policy_version 1082162 (0.0006) [2023-12-26 23:14:52,158][105692] Updated weights for policy 0, policy_version 1081127 (0.0008) [2023-12-26 23:14:52,160][105620] Updated weights for policy 1, policy_version 1082172 (0.0006) [2023-12-26 23:14:52,227][105692] Updated weights for policy 0, policy_version 1081137 (0.0009) [2023-12-26 23:14:52,293][105692] Updated weights for policy 0, policy_version 1081147 (0.0009) [2023-12-26 23:14:52,914][105620] Updated weights for policy 1, policy_version 1082182 (0.0008) [2023-12-26 23:14:52,919][105692] Updated weights for policy 0, policy_version 1081157 (0.0008) [2023-12-26 23:14:52,962][105620] Updated weights for policy 1, policy_version 1082192 (0.0010) [2023-12-26 23:14:52,979][105692] Updated weights for policy 0, policy_version 1081167 (0.0006) [2023-12-26 23:14:53,017][105620] Updated weights for policy 1, policy_version 1082202 (0.0006) [2023-12-26 23:14:53,040][105692] Updated weights for policy 0, policy_version 1081177 (0.0007) [2023-12-26 23:14:53,712][105692] Updated weights for policy 0, policy_version 1081187 (0.0006) [2023-12-26 23:14:53,763][105620] Updated weights for policy 1, policy_version 1082212 (0.0007) [2023-12-26 23:14:53,765][105692] Updated weights for policy 0, policy_version 1081197 (0.0008) [2023-12-26 23:14:53,811][105620] Updated weights for policy 1, policy_version 1082222 (0.0006) [2023-12-26 23:14:53,822][105692] Updated weights for policy 0, policy_version 1081207 (0.0006) [2023-12-26 23:14:53,856][105620] Updated weights for policy 1, policy_version 1082232 (0.0007) [2023-12-26 23:14:54,582][105692] Updated weights for policy 0, policy_version 1081217 (0.0008) [2023-12-26 23:14:54,626][105620] Updated weights for policy 1, policy_version 1082242 (0.0007) [2023-12-26 23:14:54,640][105692] Updated weights for policy 0, policy_version 1081227 (0.0007) [2023-12-26 23:14:54,681][105620] Updated weights for policy 1, policy_version 1082252 (0.0008) [2023-12-26 23:14:54,699][105692] Updated weights for policy 0, policy_version 1081237 (0.0005) [2023-12-26 23:14:54,736][105620] Updated weights for policy 1, policy_version 1082262 (0.0007) [2023-12-26 23:14:54,760][105692] Updated weights for policy 0, policy_version 1081247 (0.0010) [2023-12-26 23:14:54,792][105620] Updated weights for policy 1, policy_version 1082272 (0.0008) [2023-12-26 23:14:55,404][105692] Updated weights for policy 0, policy_version 1081257 (0.0009) [2023-12-26 23:14:55,462][105692] Updated weights for policy 0, policy_version 1081267 (0.0010) [2023-12-26 23:14:55,533][105692] Updated weights for policy 0, policy_version 1081277 (0.0010) [2023-12-26 23:14:55,596][105620] Updated weights for policy 1, policy_version 1082282 (0.0010) [2023-12-26 23:14:55,645][105620] Updated weights for policy 1, policy_version 1082292 (0.0010) [2023-12-26 23:14:55,692][105620] Updated weights for policy 1, policy_version 1082302 (0.0010) [2023-12-26 23:14:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 553951232. Throughput: 0: 9829.7, 1: 9576.8. Samples: 553960580. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:14:56,062][104569] Avg episode reward: [(0, '9265.701'), (1, '9163.644')] [2023-12-26 23:14:56,250][105692] Updated weights for policy 0, policy_version 1081287 (0.0010) [2023-12-26 23:14:56,296][105620] Updated weights for policy 1, policy_version 1082312 (0.0006) [2023-12-26 23:14:56,298][105692] Updated weights for policy 0, policy_version 1081297 (0.0010) [2023-12-26 23:14:56,356][105692] Updated weights for policy 0, policy_version 1081307 (0.0010) [2023-12-26 23:14:56,366][105620] Updated weights for policy 1, policy_version 1082322 (0.0005) [2023-12-26 23:14:56,436][105620] Updated weights for policy 1, policy_version 1082332 (0.0006) [2023-12-26 23:14:56,949][105692] Updated weights for policy 0, policy_version 1081317 (0.0010) [2023-12-26 23:14:56,993][105692] Updated weights for policy 0, policy_version 1081327 (0.0010) [2023-12-26 23:14:57,025][105620] Updated weights for policy 1, policy_version 1082342 (0.0010) [2023-12-26 23:14:57,044][105692] Updated weights for policy 0, policy_version 1081337 (0.0010) [2023-12-26 23:14:57,085][105620] Updated weights for policy 1, policy_version 1082352 (0.0011) [2023-12-26 23:14:57,149][105620] Updated weights for policy 1, policy_version 1082362 (0.0010) [2023-12-26 23:14:57,734][105620] Updated weights for policy 1, policy_version 1082372 (0.0008) [2023-12-26 23:14:57,736][105692] Updated weights for policy 0, policy_version 1081347 (0.0009) [2023-12-26 23:14:57,782][105692] Updated weights for policy 0, policy_version 1081357 (0.0007) [2023-12-26 23:14:57,793][105620] Updated weights for policy 1, policy_version 1082382 (0.0005) [2023-12-26 23:14:57,836][105692] Updated weights for policy 0, policy_version 1081367 (0.0010) [2023-12-26 23:14:57,848][105620] Updated weights for policy 1, policy_version 1082392 (0.0005) [2023-12-26 23:14:58,592][105620] Updated weights for policy 1, policy_version 1082402 (0.0007) [2023-12-26 23:14:58,624][105692] Updated weights for policy 0, policy_version 1081377 (0.0010) [2023-12-26 23:14:58,657][105620] Updated weights for policy 1, policy_version 1082412 (0.0008) [2023-12-26 23:14:58,686][105692] Updated weights for policy 0, policy_version 1081387 (0.0010) [2023-12-26 23:14:58,722][105620] Updated weights for policy 1, policy_version 1082422 (0.0008) [2023-12-26 23:14:58,742][105585] KL-divergence is very high: 101.4235 [2023-12-26 23:14:58,748][105692] Updated weights for policy 0, policy_version 1081397 (0.0012) [2023-12-26 23:14:58,790][105620] Updated weights for policy 1, policy_version 1082432 (0.0008) [2023-12-26 23:14:58,796][105585] KL-divergence is very high: 108.4637 [2023-12-26 23:14:58,822][105692] Updated weights for policy 0, policy_version 1081408 (0.0008) [2023-12-26 23:14:59,573][105620] Updated weights for policy 1, policy_version 1082442 (0.0008) [2023-12-26 23:14:59,626][105620] Updated weights for policy 1, policy_version 1082452 (0.0008) [2023-12-26 23:14:59,677][105692] Updated weights for policy 0, policy_version 1081418 (0.0005) [2023-12-26 23:14:59,682][105620] Updated weights for policy 1, policy_version 1082462 (0.0009) [2023-12-26 23:14:59,734][105692] Updated weights for policy 0, policy_version 1081428 (0.0008) [2023-12-26 23:14:59,789][105692] Updated weights for policy 0, policy_version 1081438 (0.0009) [2023-12-26 23:15:00,470][105620] Updated weights for policy 1, policy_version 1082472 (0.0007) [2023-12-26 23:15:00,520][105692] Updated weights for policy 0, policy_version 1081448 (0.0009) [2023-12-26 23:15:00,522][105620] Updated weights for policy 1, policy_version 1082482 (0.0006) [2023-12-26 23:15:00,580][105692] Updated weights for policy 0, policy_version 1081458 (0.0008) [2023-12-26 23:15:00,586][105620] Updated weights for policy 1, policy_version 1082492 (0.0006) [2023-12-26 23:15:00,639][105692] Updated weights for policy 0, policy_version 1081468 (0.0008) [2023-12-26 23:15:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 554049536. Throughput: 0: 9882.0, 1: 9673.0. Samples: 554021912. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:15:01,063][104569] Avg episode reward: [(0, '9169.937'), (1, '9255.327')] [2023-12-26 23:15:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001081472_276897792.pth... [2023-12-26 23:15:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001082496_277151744.pth... [2023-12-26 23:15:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001080320_276602880.pth [2023-12-26 23:15:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001081376_276865024.pth [2023-12-26 23:15:01,300][105620] Updated weights for policy 1, policy_version 1082502 (0.0007) [2023-12-26 23:15:01,365][105620] Updated weights for policy 1, policy_version 1082512 (0.0008) [2023-12-26 23:15:01,391][105692] Updated weights for policy 0, policy_version 1081478 (0.0008) [2023-12-26 23:15:01,430][105620] Updated weights for policy 1, policy_version 1082522 (0.0007) [2023-12-26 23:15:01,439][105692] Updated weights for policy 0, policy_version 1081488 (0.0008) [2023-12-26 23:15:01,491][105692] Updated weights for policy 0, policy_version 1081498 (0.0007) [2023-12-26 23:15:02,113][105620] Updated weights for policy 1, policy_version 1082532 (0.0006) [2023-12-26 23:15:02,169][105620] Updated weights for policy 1, policy_version 1082542 (0.0005) [2023-12-26 23:15:02,232][105620] Updated weights for policy 1, policy_version 1082552 (0.0008) [2023-12-26 23:15:02,333][105692] Updated weights for policy 0, policy_version 1081508 (0.0009) [2023-12-26 23:15:02,398][105692] Updated weights for policy 0, policy_version 1081518 (0.0009) [2023-12-26 23:15:02,457][105692] Updated weights for policy 0, policy_version 1081528 (0.0008) [2023-12-26 23:15:02,864][105620] Updated weights for policy 1, policy_version 1082562 (0.0010) [2023-12-26 23:15:02,921][105620] Updated weights for policy 1, policy_version 1082572 (0.0010) [2023-12-26 23:15:02,978][105620] Updated weights for policy 1, policy_version 1082582 (0.0010) [2023-12-26 23:15:03,032][105620] Updated weights for policy 1, policy_version 1082592 (0.0005) [2023-12-26 23:15:03,144][105692] Updated weights for policy 0, policy_version 1081538 (0.0009) [2023-12-26 23:15:03,198][105692] Updated weights for policy 0, policy_version 1081548 (0.0008) [2023-12-26 23:15:03,244][105692] Updated weights for policy 0, policy_version 1081558 (0.0008) [2023-12-26 23:15:03,290][105692] Updated weights for policy 0, policy_version 1081568 (0.0009) [2023-12-26 23:15:03,695][105620] Updated weights for policy 1, policy_version 1082602 (0.0009) [2023-12-26 23:15:03,753][105620] Updated weights for policy 1, policy_version 1082612 (0.0010) [2023-12-26 23:15:03,814][105620] Updated weights for policy 1, policy_version 1082622 (0.0008) [2023-12-26 23:15:04,076][105692] Updated weights for policy 0, policy_version 1081578 (0.0009) [2023-12-26 23:15:04,136][105692] Updated weights for policy 0, policy_version 1081588 (0.0009) [2023-12-26 23:15:04,191][105692] Updated weights for policy 0, policy_version 1081598 (0.0009) [2023-12-26 23:15:04,568][105620] Updated weights for policy 1, policy_version 1082632 (0.0010) [2023-12-26 23:15:04,617][105620] Updated weights for policy 1, policy_version 1082642 (0.0010) [2023-12-26 23:15:04,665][105620] Updated weights for policy 1, policy_version 1082652 (0.0008) [2023-12-26 23:15:04,914][105692] Updated weights for policy 0, policy_version 1081608 (0.0006) [2023-12-26 23:15:04,962][105692] Updated weights for policy 0, policy_version 1081618 (0.0006) [2023-12-26 23:15:05,024][105692] Updated weights for policy 0, policy_version 1081628 (0.0010) [2023-12-26 23:15:05,239][105620] Updated weights for policy 1, policy_version 1082662 (0.0005) [2023-12-26 23:15:05,282][105620] Updated weights for policy 1, policy_version 1082672 (0.0005) [2023-12-26 23:15:05,331][105620] Updated weights for policy 1, policy_version 1082682 (0.0005) [2023-12-26 23:15:05,774][105692] Updated weights for policy 0, policy_version 1081638 (0.0010) [2023-12-26 23:15:05,834][105692] Updated weights for policy 0, policy_version 1081648 (0.0010) [2023-12-26 23:15:05,842][105620] Updated weights for policy 1, policy_version 1082692 (0.0005) [2023-12-26 23:15:05,888][105692] Updated weights for policy 0, policy_version 1081658 (0.0006) [2023-12-26 23:15:05,898][105620] Updated weights for policy 1, policy_version 1082702 (0.0005) [2023-12-26 23:15:05,956][105620] Updated weights for policy 1, policy_version 1082712 (0.0007) [2023-12-26 23:15:06,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19660.7, 300 sec: 19688.5). Total num frames: 554156032. Throughput: 0: 9686.0, 1: 9733.3. Samples: 554135700. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:15:06,063][104569] Avg episode reward: [(0, '9172.749'), (1, '9347.399')] [2023-12-26 23:15:06,623][105692] Updated weights for policy 0, policy_version 1081668 (0.0007) [2023-12-26 23:15:06,643][105620] Updated weights for policy 1, policy_version 1082722 (0.0010) [2023-12-26 23:15:06,677][105692] Updated weights for policy 0, policy_version 1081678 (0.0006) [2023-12-26 23:15:06,702][105620] Updated weights for policy 1, policy_version 1082732 (0.0011) [2023-12-26 23:15:06,738][105692] Updated weights for policy 0, policy_version 1081688 (0.0010) [2023-12-26 23:15:06,760][105620] Updated weights for policy 1, policy_version 1082742 (0.0010) [2023-12-26 23:15:06,825][105620] Updated weights for policy 1, policy_version 1082752 (0.0010) [2023-12-26 23:15:07,479][105692] Updated weights for policy 0, policy_version 1081698 (0.0006) [2023-12-26 23:15:07,529][105692] Updated weights for policy 0, policy_version 1081708 (0.0008) [2023-12-26 23:15:07,559][105620] Updated weights for policy 1, policy_version 1082762 (0.0010) [2023-12-26 23:15:07,581][105692] Updated weights for policy 0, policy_version 1081718 (0.0007) [2023-12-26 23:15:07,617][105620] Updated weights for policy 1, policy_version 1082772 (0.0010) [2023-12-26 23:15:07,642][105692] Updated weights for policy 0, policy_version 1081728 (0.0008) [2023-12-26 23:15:07,678][105620] Updated weights for policy 1, policy_version 1082782 (0.0010) [2023-12-26 23:15:08,342][105620] Updated weights for policy 1, policy_version 1082792 (0.0009) [2023-12-26 23:15:08,408][105620] Updated weights for policy 1, policy_version 1082802 (0.0009) [2023-12-26 23:15:08,458][105692] Updated weights for policy 0, policy_version 1081738 (0.0006) [2023-12-26 23:15:08,467][105620] Updated weights for policy 1, policy_version 1082812 (0.0010) [2023-12-26 23:15:08,510][105692] Updated weights for policy 0, policy_version 1081748 (0.0009) [2023-12-26 23:15:08,570][105692] Updated weights for policy 0, policy_version 1081758 (0.0008) [2023-12-26 23:15:09,210][105620] Updated weights for policy 1, policy_version 1082822 (0.0008) [2023-12-26 23:15:09,259][105692] Updated weights for policy 0, policy_version 1081768 (0.0008) [2023-12-26 23:15:09,275][105620] Updated weights for policy 1, policy_version 1082832 (0.0010) [2023-12-26 23:15:09,315][105692] Updated weights for policy 0, policy_version 1081778 (0.0008) [2023-12-26 23:15:09,336][105620] Updated weights for policy 1, policy_version 1082842 (0.0011) [2023-12-26 23:15:09,387][105692] Updated weights for policy 0, policy_version 1081788 (0.0008) [2023-12-26 23:15:10,086][105620] Updated weights for policy 1, policy_version 1082852 (0.0008) [2023-12-26 23:15:10,143][105620] Updated weights for policy 1, policy_version 1082862 (0.0008) [2023-12-26 23:15:10,175][105692] Updated weights for policy 0, policy_version 1081798 (0.0009) [2023-12-26 23:15:10,205][105620] Updated weights for policy 1, policy_version 1082872 (0.0009) [2023-12-26 23:15:10,236][105692] Updated weights for policy 0, policy_version 1081808 (0.0007) [2023-12-26 23:15:10,286][105692] Updated weights for policy 0, policy_version 1081818 (0.0008) [2023-12-26 23:15:10,963][105692] Updated weights for policy 0, policy_version 1081828 (0.0008) [2023-12-26 23:15:11,000][105620] Updated weights for policy 1, policy_version 1082882 (0.0007) [2023-12-26 23:15:11,023][105692] Updated weights for policy 0, policy_version 1081838 (0.0007) [2023-12-26 23:15:11,058][105620] Updated weights for policy 1, policy_version 1082892 (0.0009) [2023-12-26 23:15:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 554237952. Throughput: 0: 9622.4, 1: 9747.8. Samples: 554251620. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:15:11,063][104569] Avg episode reward: [(0, '9267.684'), (1, '9255.604')] [2023-12-26 23:15:11,092][105692] Updated weights for policy 0, policy_version 1081848 (0.0011) [2023-12-26 23:15:11,123][105620] Updated weights for policy 1, policy_version 1082902 (0.0008) [2023-12-26 23:15:11,192][105620] Updated weights for policy 1, policy_version 1082912 (0.0010) [2023-12-26 23:15:11,896][105692] Updated weights for policy 0, policy_version 1081858 (0.0010) [2023-12-26 23:15:11,949][105620] Updated weights for policy 1, policy_version 1082922 (0.0011) [2023-12-26 23:15:11,953][105692] Updated weights for policy 0, policy_version 1081868 (0.0007) [2023-12-26 23:15:12,006][105620] Updated weights for policy 1, policy_version 1082932 (0.0011) [2023-12-26 23:15:12,013][105692] Updated weights for policy 0, policy_version 1081878 (0.0009) [2023-12-26 23:15:12,070][105620] Updated weights for policy 1, policy_version 1082942 (0.0011) [2023-12-26 23:15:12,071][105692] Updated weights for policy 0, policy_version 1081888 (0.0007) [2023-12-26 23:15:12,721][105620] Updated weights for policy 1, policy_version 1082952 (0.0008) [2023-12-26 23:15:12,758][105692] Updated weights for policy 0, policy_version 1081898 (0.0008) [2023-12-26 23:15:12,780][105620] Updated weights for policy 1, policy_version 1082962 (0.0008) [2023-12-26 23:15:12,807][105692] Updated weights for policy 0, policy_version 1081908 (0.0006) [2023-12-26 23:15:12,841][105620] Updated weights for policy 1, policy_version 1082972 (0.0009) [2023-12-26 23:15:12,859][105692] Updated weights for policy 0, policy_version 1081918 (0.0006) [2023-12-26 23:15:13,558][105620] Updated weights for policy 1, policy_version 1082982 (0.0008) [2023-12-26 23:15:13,572][105692] Updated weights for policy 0, policy_version 1081928 (0.0009) [2023-12-26 23:15:13,617][105620] Updated weights for policy 1, policy_version 1082992 (0.0006) [2023-12-26 23:15:13,628][105692] Updated weights for policy 0, policy_version 1081938 (0.0010) [2023-12-26 23:15:13,675][105620] Updated weights for policy 1, policy_version 1083002 (0.0006) [2023-12-26 23:15:13,683][105692] Updated weights for policy 0, policy_version 1081948 (0.0010) [2023-12-26 23:15:14,346][105620] Updated weights for policy 1, policy_version 1083012 (0.0007) [2023-12-26 23:15:14,357][105692] Updated weights for policy 0, policy_version 1081958 (0.0011) [2023-12-26 23:15:14,408][105620] Updated weights for policy 1, policy_version 1083022 (0.0006) [2023-12-26 23:15:14,410][105692] Updated weights for policy 0, policy_version 1081968 (0.0011) [2023-12-26 23:15:14,463][105620] Updated weights for policy 1, policy_version 1083032 (0.0007) [2023-12-26 23:15:14,469][105692] Updated weights for policy 0, policy_version 1081978 (0.0011) [2023-12-26 23:15:15,187][105620] Updated weights for policy 1, policy_version 1083042 (0.0008) [2023-12-26 23:15:15,230][105692] Updated weights for policy 0, policy_version 1081988 (0.0009) [2023-12-26 23:15:15,252][105620] Updated weights for policy 1, policy_version 1083052 (0.0006) [2023-12-26 23:15:15,297][105692] Updated weights for policy 0, policy_version 1081998 (0.0010) [2023-12-26 23:15:15,316][105620] Updated weights for policy 1, policy_version 1083062 (0.0006) [2023-12-26 23:15:15,357][105692] Updated weights for policy 0, policy_version 1082008 (0.0011) [2023-12-26 23:15:15,376][105620] Updated weights for policy 1, policy_version 1083072 (0.0007) [2023-12-26 23:15:15,984][105692] Updated weights for policy 0, policy_version 1082018 (0.0010) [2023-12-26 23:15:16,042][105692] Updated weights for policy 0, policy_version 1082028 (0.0009) [2023-12-26 23:15:16,062][104569] Fps is (10 sec: 18022.9, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 554336256. Throughput: 0: 9684.9, 1: 9701.3. Samples: 554311048. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:15:16,062][104569] Avg episode reward: [(0, '9356.164'), (1, '9163.313')] [2023-12-26 23:15:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001083072_277299200.pth... [2023-12-26 23:15:16,090][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001081920_277004288.pth [2023-12-26 23:15:16,108][105692] Updated weights for policy 0, policy_version 1082038 (0.0009) [2023-12-26 23:15:16,152][105620] Updated weights for policy 1, policy_version 1083082 (0.0006) [2023-12-26 23:15:16,163][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001082048_277045248.pth... [2023-12-26 23:15:16,166][105692] Updated weights for policy 0, policy_version 1082048 (0.0007) [2023-12-26 23:15:16,168][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001080896_276750336.pth [2023-12-26 23:15:16,201][105620] Updated weights for policy 1, policy_version 1083092 (0.0008) [2023-12-26 23:15:16,256][105620] Updated weights for policy 1, policy_version 1083102 (0.0006) [2023-12-26 23:15:16,868][105692] Updated weights for policy 0, policy_version 1082058 (0.0008) [2023-12-26 23:15:16,925][105692] Updated weights for policy 0, policy_version 1082068 (0.0010) [2023-12-26 23:15:16,987][105692] Updated weights for policy 0, policy_version 1082078 (0.0010) [2023-12-26 23:15:17,026][105620] Updated weights for policy 1, policy_version 1083112 (0.0006) [2023-12-26 23:15:17,079][105620] Updated weights for policy 1, policy_version 1083122 (0.0005) [2023-12-26 23:15:17,131][105620] Updated weights for policy 1, policy_version 1083132 (0.0005) [2023-12-26 23:15:17,630][105692] Updated weights for policy 0, policy_version 1082088 (0.0007) [2023-12-26 23:15:17,693][105692] Updated weights for policy 0, policy_version 1082098 (0.0006) [2023-12-26 23:15:17,741][105692] Updated weights for policy 0, policy_version 1082108 (0.0007) [2023-12-26 23:15:17,838][105620] Updated weights for policy 1, policy_version 1083142 (0.0007) [2023-12-26 23:15:17,892][105620] Updated weights for policy 1, policy_version 1083152 (0.0009) [2023-12-26 23:15:17,939][105620] Updated weights for policy 1, policy_version 1083162 (0.0008) [2023-12-26 23:15:18,410][105692] Updated weights for policy 0, policy_version 1082118 (0.0006) [2023-12-26 23:15:18,473][105692] Updated weights for policy 0, policy_version 1082128 (0.0009) [2023-12-26 23:15:18,535][105692] Updated weights for policy 0, policy_version 1082138 (0.0009) [2023-12-26 23:15:18,700][105620] Updated weights for policy 1, policy_version 1083172 (0.0009) [2023-12-26 23:15:18,751][105620] Updated weights for policy 1, policy_version 1083182 (0.0009) [2023-12-26 23:15:18,803][105620] Updated weights for policy 1, policy_version 1083192 (0.0009) [2023-12-26 23:15:19,179][105692] Updated weights for policy 0, policy_version 1082148 (0.0009) [2023-12-26 23:15:19,239][105692] Updated weights for policy 0, policy_version 1082158 (0.0007) [2023-12-26 23:15:19,300][105692] Updated weights for policy 0, policy_version 1082168 (0.0008) [2023-12-26 23:15:19,646][105620] Updated weights for policy 1, policy_version 1083202 (0.0009) [2023-12-26 23:15:19,708][105620] Updated weights for policy 1, policy_version 1083212 (0.0008) [2023-12-26 23:15:19,767][105620] Updated weights for policy 1, policy_version 1083222 (0.0009) [2023-12-26 23:15:19,831][105620] Updated weights for policy 1, policy_version 1083232 (0.0010) [2023-12-26 23:15:19,961][105692] Updated weights for policy 0, policy_version 1082178 (0.0007) [2023-12-26 23:15:20,025][105692] Updated weights for policy 0, policy_version 1082188 (0.0007) [2023-12-26 23:15:20,086][105692] Updated weights for policy 0, policy_version 1082198 (0.0007) [2023-12-26 23:15:20,140][105692] Updated weights for policy 0, policy_version 1082208 (0.0006) [2023-12-26 23:15:20,579][105620] Updated weights for policy 1, policy_version 1083242 (0.0010) [2023-12-26 23:15:20,645][105620] Updated weights for policy 1, policy_version 1083252 (0.0008) [2023-12-26 23:15:20,706][105620] Updated weights for policy 1, policy_version 1083262 (0.0008) [2023-12-26 23:15:20,803][105692] Updated weights for policy 0, policy_version 1082218 (0.0008) [2023-12-26 23:15:20,863][105692] Updated weights for policy 0, policy_version 1082228 (0.0008) [2023-12-26 23:15:20,924][105692] Updated weights for policy 0, policy_version 1082238 (0.0009) [2023-12-26 23:15:21,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 554442752. Throughput: 0: 9767.8, 1: 9635.6. Samples: 554426996. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:15:21,062][104569] Avg episode reward: [(0, '9357.254'), (1, '9163.258')] [2023-12-26 23:15:21,492][105620] Updated weights for policy 1, policy_version 1083272 (0.0008) [2023-12-26 23:15:21,548][105620] Updated weights for policy 1, policy_version 1083282 (0.0008) [2023-12-26 23:15:21,600][105620] Updated weights for policy 1, policy_version 1083292 (0.0008) [2023-12-26 23:15:21,745][105692] Updated weights for policy 0, policy_version 1082248 (0.0010) [2023-12-26 23:15:21,798][105692] Updated weights for policy 0, policy_version 1082258 (0.0011) [2023-12-26 23:15:21,855][105692] Updated weights for policy 0, policy_version 1082268 (0.0011) [2023-12-26 23:15:22,263][105620] Updated weights for policy 1, policy_version 1083302 (0.0008) [2023-12-26 23:15:22,332][105620] Updated weights for policy 1, policy_version 1083312 (0.0009) [2023-12-26 23:15:22,406][105620] Updated weights for policy 1, policy_version 1083322 (0.0007) [2023-12-26 23:15:22,634][105692] Updated weights for policy 0, policy_version 1082278 (0.0011) [2023-12-26 23:15:22,695][105692] Updated weights for policy 0, policy_version 1082288 (0.0008) [2023-12-26 23:15:22,756][105692] Updated weights for policy 0, policy_version 1082298 (0.0006) [2023-12-26 23:15:23,059][105620] Updated weights for policy 1, policy_version 1083332 (0.0007) [2023-12-26 23:15:23,110][105620] Updated weights for policy 1, policy_version 1083342 (0.0008) [2023-12-26 23:15:23,168][105620] Updated weights for policy 1, policy_version 1083352 (0.0008) [2023-12-26 23:15:23,498][105692] Updated weights for policy 0, policy_version 1082308 (0.0008) [2023-12-26 23:15:23,556][105692] Updated weights for policy 0, policy_version 1082318 (0.0010) [2023-12-26 23:15:23,614][105692] Updated weights for policy 0, policy_version 1082328 (0.0009) [2023-12-26 23:15:23,865][105620] Updated weights for policy 1, policy_version 1083362 (0.0007) [2023-12-26 23:15:23,918][105620] Updated weights for policy 1, policy_version 1083372 (0.0009) [2023-12-26 23:15:23,973][105620] Updated weights for policy 1, policy_version 1083382 (0.0010) [2023-12-26 23:15:24,039][105620] Updated weights for policy 1, policy_version 1083392 (0.0009) [2023-12-26 23:15:24,272][105692] Updated weights for policy 0, policy_version 1082338 (0.0009) [2023-12-26 23:15:24,332][105692] Updated weights for policy 0, policy_version 1082348 (0.0008) [2023-12-26 23:15:24,390][105692] Updated weights for policy 0, policy_version 1082358 (0.0009) [2023-12-26 23:15:24,449][105692] Updated weights for policy 0, policy_version 1082368 (0.0009) [2023-12-26 23:15:24,710][105620] Updated weights for policy 1, policy_version 1083402 (0.0006) [2023-12-26 23:15:24,772][105620] Updated weights for policy 1, policy_version 1083412 (0.0005) [2023-12-26 23:15:24,829][105620] Updated weights for policy 1, policy_version 1083422 (0.0005) [2023-12-26 23:15:25,182][105692] Updated weights for policy 0, policy_version 1082378 (0.0009) [2023-12-26 23:15:25,234][105692] Updated weights for policy 0, policy_version 1082388 (0.0009) [2023-12-26 23:15:25,281][105692] Updated weights for policy 0, policy_version 1082398 (0.0009) [2023-12-26 23:15:25,444][105620] Updated weights for policy 1, policy_version 1083432 (0.0008) [2023-12-26 23:15:25,499][105620] Updated weights for policy 1, policy_version 1083442 (0.0009) [2023-12-26 23:15:25,554][105620] Updated weights for policy 1, policy_version 1083452 (0.0009) [2023-12-26 23:15:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 554532864. Throughput: 0: 9669.7, 1: 9729.7. Samples: 554543728. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:15:26,063][104569] Avg episode reward: [(0, '9356.830'), (1, '9255.291')] [2023-12-26 23:15:26,071][105692] Updated weights for policy 0, policy_version 1082408 (0.0009) [2023-12-26 23:15:26,133][105692] Updated weights for policy 0, policy_version 1082418 (0.0009) [2023-12-26 23:15:26,193][105692] Updated weights for policy 0, policy_version 1082428 (0.0008) [2023-12-26 23:15:26,309][105620] Updated weights for policy 1, policy_version 1083462 (0.0009) [2023-12-26 23:15:26,367][105620] Updated weights for policy 1, policy_version 1083472 (0.0009) [2023-12-26 23:15:26,418][105620] Updated weights for policy 1, policy_version 1083482 (0.0006) [2023-12-26 23:15:26,933][105692] Updated weights for policy 0, policy_version 1082438 (0.0008) [2023-12-26 23:15:26,990][105692] Updated weights for policy 0, policy_version 1082448 (0.0009) [2023-12-26 23:15:27,050][105692] Updated weights for policy 0, policy_version 1082458 (0.0009) [2023-12-26 23:15:27,139][105620] Updated weights for policy 1, policy_version 1083492 (0.0007) [2023-12-26 23:15:27,199][105620] Updated weights for policy 1, policy_version 1083502 (0.0008) [2023-12-26 23:15:27,245][105620] Updated weights for policy 1, policy_version 1083512 (0.0008) [2023-12-26 23:15:27,777][105692] Updated weights for policy 0, policy_version 1082468 (0.0009) [2023-12-26 23:15:27,837][105692] Updated weights for policy 0, policy_version 1082478 (0.0009) [2023-12-26 23:15:27,898][105692] Updated weights for policy 0, policy_version 1082488 (0.0009) [2023-12-26 23:15:28,004][105620] Updated weights for policy 1, policy_version 1083522 (0.0009) [2023-12-26 23:15:28,065][105620] Updated weights for policy 1, policy_version 1083532 (0.0009) [2023-12-26 23:15:28,125][105620] Updated weights for policy 1, policy_version 1083542 (0.0009) [2023-12-26 23:15:28,189][105620] Updated weights for policy 1, policy_version 1083552 (0.0008) [2023-12-26 23:15:28,615][105692] Updated weights for policy 0, policy_version 1082498 (0.0009) [2023-12-26 23:15:28,682][105692] Updated weights for policy 0, policy_version 1082508 (0.0010) [2023-12-26 23:15:28,734][105692] Updated weights for policy 0, policy_version 1082518 (0.0010) [2023-12-26 23:15:28,789][105692] Updated weights for policy 0, policy_version 1082528 (0.0009) [2023-12-26 23:15:28,928][105620] Updated weights for policy 1, policy_version 1083562 (0.0008) [2023-12-26 23:15:28,992][105620] Updated weights for policy 1, policy_version 1083572 (0.0010) [2023-12-26 23:15:29,063][105620] Updated weights for policy 1, policy_version 1083582 (0.0006) [2023-12-26 23:15:29,455][105692] Updated weights for policy 0, policy_version 1082538 (0.0009) [2023-12-26 23:15:29,508][105692] Updated weights for policy 0, policy_version 1082548 (0.0008) [2023-12-26 23:15:29,572][105692] Updated weights for policy 0, policy_version 1082558 (0.0009) [2023-12-26 23:15:29,799][105620] Updated weights for policy 1, policy_version 1083592 (0.0008) [2023-12-26 23:15:29,864][105620] Updated weights for policy 1, policy_version 1083602 (0.0008) [2023-12-26 23:15:29,919][105620] Updated weights for policy 1, policy_version 1083612 (0.0008) [2023-12-26 23:15:30,359][105692] Updated weights for policy 0, policy_version 1082568 (0.0008) [2023-12-26 23:15:30,421][105692] Updated weights for policy 0, policy_version 1082578 (0.0009) [2023-12-26 23:15:30,485][105692] Updated weights for policy 0, policy_version 1082588 (0.0009) [2023-12-26 23:15:30,628][105620] Updated weights for policy 1, policy_version 1083622 (0.0006) [2023-12-26 23:15:30,686][105620] Updated weights for policy 1, policy_version 1083632 (0.0005) [2023-12-26 23:15:30,742][105620] Updated weights for policy 1, policy_version 1083642 (0.0008) [2023-12-26 23:15:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 554631168. Throughput: 0: 9685.1, 1: 9747.3. Samples: 554600904. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:15:31,063][104569] Avg episode reward: [(0, '9184.322'), (1, '9255.338')] [2023-12-26 23:15:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001083648_277446656.pth... [2023-12-26 23:15:31,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001082592_277184512.pth... [2023-12-26 23:15:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001082496_277151744.pth [2023-12-26 23:15:31,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001081472_276897792.pth [2023-12-26 23:15:31,319][105692] Updated weights for policy 0, policy_version 1082598 (0.0010) [2023-12-26 23:15:31,380][105692] Updated weights for policy 0, policy_version 1082608 (0.0008) [2023-12-26 23:15:31,429][105692] Updated weights for policy 0, policy_version 1082618 (0.0008) [2023-12-26 23:15:31,494][105620] Updated weights for policy 1, policy_version 1083652 (0.0009) [2023-12-26 23:15:31,552][105620] Updated weights for policy 1, policy_version 1083662 (0.0009) [2023-12-26 23:15:31,614][105620] Updated weights for policy 1, policy_version 1083672 (0.0009) [2023-12-26 23:15:32,205][105692] Updated weights for policy 0, policy_version 1082628 (0.0009) [2023-12-26 23:15:32,272][105692] Updated weights for policy 0, policy_version 1082638 (0.0009) [2023-12-26 23:15:32,325][105692] Updated weights for policy 0, policy_version 1082648 (0.0009) [2023-12-26 23:15:32,409][105620] Updated weights for policy 1, policy_version 1083682 (0.0009) [2023-12-26 23:15:32,456][105620] Updated weights for policy 1, policy_version 1083692 (0.0006) [2023-12-26 23:15:32,502][105620] Updated weights for policy 1, policy_version 1083702 (0.0005) [2023-12-26 23:15:32,553][105620] Updated weights for policy 1, policy_version 1083712 (0.0005) [2023-12-26 23:15:33,104][105692] Updated weights for policy 0, policy_version 1082658 (0.0009) [2023-12-26 23:15:33,153][105692] Updated weights for policy 0, policy_version 1082668 (0.0008) [2023-12-26 23:15:33,211][105692] Updated weights for policy 0, policy_version 1082678 (0.0009) [2023-12-26 23:15:33,268][105692] Updated weights for policy 0, policy_version 1082688 (0.0008) [2023-12-26 23:15:33,273][105620] Updated weights for policy 1, policy_version 1083722 (0.0007) [2023-12-26 23:15:33,328][105620] Updated weights for policy 1, policy_version 1083732 (0.0009) [2023-12-26 23:15:33,386][105620] Updated weights for policy 1, policy_version 1083742 (0.0010) [2023-12-26 23:15:33,990][105692] Updated weights for policy 0, policy_version 1082698 (0.0009) [2023-12-26 23:15:34,049][105692] Updated weights for policy 0, policy_version 1082708 (0.0008) [2023-12-26 23:15:34,107][105692] Updated weights for policy 0, policy_version 1082718 (0.0009) [2023-12-26 23:15:34,139][105620] Updated weights for policy 1, policy_version 1083752 (0.0008) [2023-12-26 23:15:34,199][105620] Updated weights for policy 1, policy_version 1083762 (0.0008) [2023-12-26 23:15:34,250][105620] Updated weights for policy 1, policy_version 1083772 (0.0007) [2023-12-26 23:15:34,801][105692] Updated weights for policy 0, policy_version 1082728 (0.0006) [2023-12-26 23:15:34,861][105692] Updated weights for policy 0, policy_version 1082738 (0.0005) [2023-12-26 23:15:34,876][105620] Updated weights for policy 1, policy_version 1083782 (0.0008) [2023-12-26 23:15:34,926][105692] Updated weights for policy 0, policy_version 1082748 (0.0005) [2023-12-26 23:15:34,929][105620] Updated weights for policy 1, policy_version 1083792 (0.0007) [2023-12-26 23:15:34,990][105620] Updated weights for policy 1, policy_version 1083802 (0.0008) [2023-12-26 23:15:35,472][105692] Updated weights for policy 0, policy_version 1082758 (0.0008) [2023-12-26 23:15:35,539][105692] Updated weights for policy 0, policy_version 1082768 (0.0008) [2023-12-26 23:15:35,603][105692] Updated weights for policy 0, policy_version 1082778 (0.0008) [2023-12-26 23:15:35,695][105620] Updated weights for policy 1, policy_version 1083812 (0.0009) [2023-12-26 23:15:35,750][105620] Updated weights for policy 1, policy_version 1083822 (0.0010) [2023-12-26 23:15:35,811][105620] Updated weights for policy 1, policy_version 1083832 (0.0010) [2023-12-26 23:15:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 554729472. Throughput: 0: 9594.2, 1: 9721.8. Samples: 554714060. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:15:36,063][104569] Avg episode reward: [(0, '9002.532'), (1, '9347.221')] [2023-12-26 23:15:36,357][105692] Updated weights for policy 0, policy_version 1082788 (0.0009) [2023-12-26 23:15:36,423][105692] Updated weights for policy 0, policy_version 1082798 (0.0011) [2023-12-26 23:15:36,484][105692] Updated weights for policy 0, policy_version 1082808 (0.0008) [2023-12-26 23:15:36,564][105620] Updated weights for policy 1, policy_version 1083842 (0.0010) [2023-12-26 23:15:36,609][105620] Updated weights for policy 1, policy_version 1083852 (0.0010) [2023-12-26 23:15:36,664][105620] Updated weights for policy 1, policy_version 1083862 (0.0010) [2023-12-26 23:15:36,713][105620] Updated weights for policy 1, policy_version 1083872 (0.0010) [2023-12-26 23:15:37,153][105692] Updated weights for policy 0, policy_version 1082818 (0.0007) [2023-12-26 23:15:37,211][105692] Updated weights for policy 0, policy_version 1082828 (0.0008) [2023-12-26 23:15:37,269][105692] Updated weights for policy 0, policy_version 1082838 (0.0010) [2023-12-26 23:15:37,327][105692] Updated weights for policy 0, policy_version 1082848 (0.0009) [2023-12-26 23:15:37,390][105620] Updated weights for policy 1, policy_version 1083882 (0.0005) [2023-12-26 23:15:37,457][105620] Updated weights for policy 1, policy_version 1083892 (0.0010) [2023-12-26 23:15:37,525][105620] Updated weights for policy 1, policy_version 1083902 (0.0010) [2023-12-26 23:15:38,062][105692] Updated weights for policy 0, policy_version 1082858 (0.0009) [2023-12-26 23:15:38,115][105692] Updated weights for policy 0, policy_version 1082868 (0.0009) [2023-12-26 23:15:38,160][105692] Updated weights for policy 0, policy_version 1082878 (0.0009) [2023-12-26 23:15:38,184][105620] Updated weights for policy 1, policy_version 1083912 (0.0009) [2023-12-26 23:15:38,235][105620] Updated weights for policy 1, policy_version 1083922 (0.0009) [2023-12-26 23:15:38,284][105620] Updated weights for policy 1, policy_version 1083932 (0.0008) [2023-12-26 23:15:38,949][105692] Updated weights for policy 0, policy_version 1082888 (0.0008) [2023-12-26 23:15:39,002][105692] Updated weights for policy 0, policy_version 1082898 (0.0008) [2023-12-26 23:15:39,056][105692] Updated weights for policy 0, policy_version 1082908 (0.0007) [2023-12-26 23:15:39,061][105620] Updated weights for policy 1, policy_version 1083942 (0.0008) [2023-12-26 23:15:39,114][105620] Updated weights for policy 1, policy_version 1083952 (0.0010) [2023-12-26 23:15:39,178][105620] Updated weights for policy 1, policy_version 1083962 (0.0009) [2023-12-26 23:15:39,880][105692] Updated weights for policy 0, policy_version 1082918 (0.0007) [2023-12-26 23:15:39,912][105620] Updated weights for policy 1, policy_version 1083972 (0.0010) [2023-12-26 23:15:39,945][105692] Updated weights for policy 0, policy_version 1082928 (0.0009) [2023-12-26 23:15:39,978][105620] Updated weights for policy 1, policy_version 1083982 (0.0010) [2023-12-26 23:15:40,005][105692] Updated weights for policy 0, policy_version 1082938 (0.0006) [2023-12-26 23:15:40,046][105620] Updated weights for policy 1, policy_version 1083992 (0.0010) [2023-12-26 23:15:40,637][105692] Updated weights for policy 0, policy_version 1082948 (0.0006) [2023-12-26 23:15:40,706][105692] Updated weights for policy 0, policy_version 1082958 (0.0006) [2023-12-26 23:15:40,772][105692] Updated weights for policy 0, policy_version 1082968 (0.0009) [2023-12-26 23:15:40,773][105620] Updated weights for policy 1, policy_version 1084002 (0.0010) [2023-12-26 23:15:40,834][105620] Updated weights for policy 1, policy_version 1084012 (0.0010) [2023-12-26 23:15:40,894][105620] Updated weights for policy 1, policy_version 1084022 (0.0011) [2023-12-26 23:15:40,951][105620] Updated weights for policy 1, policy_version 1084032 (0.0009) [2023-12-26 23:15:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 554827776. Throughput: 0: 9611.2, 1: 9741.3. Samples: 554831444. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:15:41,063][104569] Avg episode reward: [(0, '9082.199'), (1, '9347.104')] [2023-12-26 23:15:41,636][105692] Updated weights for policy 0, policy_version 1082978 (0.0009) [2023-12-26 23:15:41,676][105620] Updated weights for policy 1, policy_version 1084042 (0.0008) [2023-12-26 23:15:41,705][105692] Updated weights for policy 0, policy_version 1082988 (0.0009) [2023-12-26 23:15:41,740][105620] Updated weights for policy 1, policy_version 1084052 (0.0008) [2023-12-26 23:15:41,771][105692] Updated weights for policy 0, policy_version 1082998 (0.0008) [2023-12-26 23:15:41,801][105620] Updated weights for policy 1, policy_version 1084062 (0.0010) [2023-12-26 23:15:41,830][105692] Updated weights for policy 0, policy_version 1083008 (0.0009) [2023-12-26 23:15:42,544][105620] Updated weights for policy 1, policy_version 1084072 (0.0009) [2023-12-26 23:15:42,602][105620] Updated weights for policy 1, policy_version 1084082 (0.0007) [2023-12-26 23:15:42,604][105692] Updated weights for policy 0, policy_version 1083018 (0.0006) [2023-12-26 23:15:42,661][105692] Updated weights for policy 0, policy_version 1083028 (0.0005) [2023-12-26 23:15:42,667][105620] Updated weights for policy 1, policy_version 1084092 (0.0009) [2023-12-26 23:15:42,708][105692] Updated weights for policy 0, policy_version 1083038 (0.0007) [2023-12-26 23:15:43,340][105620] Updated weights for policy 1, policy_version 1084102 (0.0009) [2023-12-26 23:15:43,400][105620] Updated weights for policy 1, policy_version 1084112 (0.0006) [2023-12-26 23:15:43,460][105620] Updated weights for policy 1, policy_version 1084122 (0.0005) [2023-12-26 23:15:43,521][105692] Updated weights for policy 0, policy_version 1083048 (0.0008) [2023-12-26 23:15:43,579][105692] Updated weights for policy 0, policy_version 1083058 (0.0008) [2023-12-26 23:15:43,640][105692] Updated weights for policy 0, policy_version 1083068 (0.0009) [2023-12-26 23:15:44,184][105620] Updated weights for policy 1, policy_version 1084132 (0.0008) [2023-12-26 23:15:44,239][105620] Updated weights for policy 1, policy_version 1084142 (0.0008) [2023-12-26 23:15:44,292][105620] Updated weights for policy 1, policy_version 1084152 (0.0008) [2023-12-26 23:15:44,353][105692] Updated weights for policy 0, policy_version 1083078 (0.0010) [2023-12-26 23:15:44,415][105692] Updated weights for policy 0, policy_version 1083088 (0.0010) [2023-12-26 23:15:44,464][105692] Updated weights for policy 0, policy_version 1083098 (0.0006) [2023-12-26 23:15:45,064][105620] Updated weights for policy 1, policy_version 1084162 (0.0008) [2023-12-26 23:15:45,127][105620] Updated weights for policy 1, policy_version 1084172 (0.0008) [2023-12-26 23:15:45,187][105620] Updated weights for policy 1, policy_version 1084182 (0.0008) [2023-12-26 23:15:45,216][105692] Updated weights for policy 0, policy_version 1083108 (0.0010) [2023-12-26 23:15:45,243][105620] Updated weights for policy 1, policy_version 1084192 (0.0007) [2023-12-26 23:15:45,286][105692] Updated weights for policy 0, policy_version 1083118 (0.0010) [2023-12-26 23:15:45,349][105692] Updated weights for policy 0, policy_version 1083128 (0.0011) [2023-12-26 23:15:45,939][105620] Updated weights for policy 1, policy_version 1084202 (0.0010) [2023-12-26 23:15:45,988][105620] Updated weights for policy 1, policy_version 1084212 (0.0010) [2023-12-26 23:15:46,035][105692] Updated weights for policy 0, policy_version 1083138 (0.0009) [2023-12-26 23:15:46,045][105620] Updated weights for policy 1, policy_version 1084222 (0.0010) [2023-12-26 23:15:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19605.2). Total num frames: 554917888. Throughput: 0: 9505.5, 1: 9714.8. Samples: 554886828. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:15:46,063][104569] Avg episode reward: [(0, '9081.008'), (1, '9346.874')] [2023-12-26 23:15:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001084224_277594112.pth... [2023-12-26 23:15:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001083072_277299200.pth [2023-12-26 23:15:46,094][105692] Updated weights for policy 0, policy_version 1083148 (0.0006) [2023-12-26 23:15:46,156][105692] Updated weights for policy 0, policy_version 1083158 (0.0005) [2023-12-26 23:15:46,215][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001083168_277331968.pth... [2023-12-26 23:15:46,215][105692] Updated weights for policy 0, policy_version 1083168 (0.0005) [2023-12-26 23:15:46,219][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001082048_277045248.pth [2023-12-26 23:15:46,772][105692] Updated weights for policy 0, policy_version 1083178 (0.0005) [2023-12-26 23:15:46,817][105620] Updated weights for policy 1, policy_version 1084232 (0.0009) [2023-12-26 23:15:46,827][105692] Updated weights for policy 0, policy_version 1083188 (0.0005) [2023-12-26 23:15:46,877][105620] Updated weights for policy 1, policy_version 1084242 (0.0009) [2023-12-26 23:15:46,885][105692] Updated weights for policy 0, policy_version 1083198 (0.0005) [2023-12-26 23:15:46,939][105620] Updated weights for policy 1, policy_version 1084252 (0.0009) [2023-12-26 23:15:47,414][105692] Updated weights for policy 0, policy_version 1083208 (0.0006) [2023-12-26 23:15:47,482][105692] Updated weights for policy 0, policy_version 1083218 (0.0010) [2023-12-26 23:15:47,548][105692] Updated weights for policy 0, policy_version 1083228 (0.0007) [2023-12-26 23:15:47,749][105620] Updated weights for policy 1, policy_version 1084262 (0.0010) [2023-12-26 23:15:47,808][105620] Updated weights for policy 1, policy_version 1084272 (0.0010) [2023-12-26 23:15:47,870][105620] Updated weights for policy 1, policy_version 1084282 (0.0011) [2023-12-26 23:15:48,104][105692] Updated weights for policy 0, policy_version 1083238 (0.0008) [2023-12-26 23:15:48,150][105692] Updated weights for policy 0, policy_version 1083248 (0.0006) [2023-12-26 23:15:48,200][105692] Updated weights for policy 0, policy_version 1083258 (0.0005) [2023-12-26 23:15:48,554][105620] Updated weights for policy 1, policy_version 1084292 (0.0009) [2023-12-26 23:15:48,617][105620] Updated weights for policy 1, policy_version 1084302 (0.0008) [2023-12-26 23:15:48,678][105620] Updated weights for policy 1, policy_version 1084312 (0.0006) [2023-12-26 23:15:48,943][105692] Updated weights for policy 0, policy_version 1083268 (0.0006) [2023-12-26 23:15:49,003][105692] Updated weights for policy 0, policy_version 1083278 (0.0008) [2023-12-26 23:15:49,067][105692] Updated weights for policy 0, policy_version 1083288 (0.0009) [2023-12-26 23:15:49,353][105620] Updated weights for policy 1, policy_version 1084322 (0.0007) [2023-12-26 23:15:49,415][105620] Updated weights for policy 1, policy_version 1084332 (0.0011) [2023-12-26 23:15:49,465][105620] Updated weights for policy 1, policy_version 1084342 (0.0010) [2023-12-26 23:15:49,510][105620] Updated weights for policy 1, policy_version 1084352 (0.0010) [2023-12-26 23:15:49,717][105692] Updated weights for policy 0, policy_version 1083298 (0.0009) [2023-12-26 23:15:49,775][105692] Updated weights for policy 0, policy_version 1083308 (0.0006) [2023-12-26 23:15:49,836][105692] Updated weights for policy 0, policy_version 1083318 (0.0006) [2023-12-26 23:15:49,905][105692] Updated weights for policy 0, policy_version 1083328 (0.0008) [2023-12-26 23:15:50,327][105620] Updated weights for policy 1, policy_version 1084362 (0.0006) [2023-12-26 23:15:50,390][105620] Updated weights for policy 1, policy_version 1084372 (0.0009) [2023-12-26 23:15:50,446][105620] Updated weights for policy 1, policy_version 1084382 (0.0009) [2023-12-26 23:15:50,543][105692] Updated weights for policy 0, policy_version 1083338 (0.0009) [2023-12-26 23:15:50,602][105692] Updated weights for policy 0, policy_version 1083348 (0.0008) [2023-12-26 23:15:50,665][105692] Updated weights for policy 0, policy_version 1083358 (0.0010) [2023-12-26 23:15:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 555016192. Throughput: 0: 9687.5, 1: 9653.1. Samples: 555006020. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-26 23:15:51,062][104569] Avg episode reward: [(0, '9260.854'), (1, '9254.852')] [2023-12-26 23:15:51,196][105620] Updated weights for policy 1, policy_version 1084392 (0.0009) [2023-12-26 23:15:51,252][105620] Updated weights for policy 1, policy_version 1084402 (0.0008) [2023-12-26 23:15:51,309][105620] Updated weights for policy 1, policy_version 1084412 (0.0008) [2023-12-26 23:15:51,497][105692] Updated weights for policy 0, policy_version 1083368 (0.0008) [2023-12-26 23:15:51,560][105692] Updated weights for policy 0, policy_version 1083378 (0.0008) [2023-12-26 23:15:51,626][105692] Updated weights for policy 0, policy_version 1083388 (0.0007) [2023-12-26 23:15:52,101][105620] Updated weights for policy 1, policy_version 1084422 (0.0009) [2023-12-26 23:15:52,151][105620] Updated weights for policy 1, policy_version 1084432 (0.0006) [2023-12-26 23:15:52,200][105620] Updated weights for policy 1, policy_version 1084442 (0.0005) [2023-12-26 23:15:52,397][105692] Updated weights for policy 0, policy_version 1083398 (0.0009) [2023-12-26 23:15:52,458][105692] Updated weights for policy 0, policy_version 1083408 (0.0013) [2023-12-26 23:15:52,525][105692] Updated weights for policy 0, policy_version 1083419 (0.0008) [2023-12-26 23:15:52,848][105620] Updated weights for policy 1, policy_version 1084452 (0.0008) [2023-12-26 23:15:52,907][105620] Updated weights for policy 1, policy_version 1084462 (0.0009) [2023-12-26 23:15:52,971][105620] Updated weights for policy 1, policy_version 1084472 (0.0006) [2023-12-26 23:15:53,349][105692] Updated weights for policy 0, policy_version 1083429 (0.0007) [2023-12-26 23:15:53,412][105692] Updated weights for policy 0, policy_version 1083439 (0.0009) [2023-12-26 23:15:53,478][105692] Updated weights for policy 0, policy_version 1083449 (0.0009) [2023-12-26 23:15:53,673][105620] Updated weights for policy 1, policy_version 1084482 (0.0008) [2023-12-26 23:15:53,734][105620] Updated weights for policy 1, policy_version 1084492 (0.0008) [2023-12-26 23:15:53,801][105620] Updated weights for policy 1, policy_version 1084502 (0.0010) [2023-12-26 23:15:53,865][105620] Updated weights for policy 1, policy_version 1084512 (0.0009) [2023-12-26 23:15:54,178][105692] Updated weights for policy 0, policy_version 1083459 (0.0008) [2023-12-26 23:15:54,237][105692] Updated weights for policy 0, policy_version 1083469 (0.0007) [2023-12-26 23:15:54,295][105692] Updated weights for policy 0, policy_version 1083479 (0.0008) [2023-12-26 23:15:54,653][105620] Updated weights for policy 1, policy_version 1084522 (0.0008) [2023-12-26 23:15:54,708][105620] Updated weights for policy 1, policy_version 1084532 (0.0009) [2023-12-26 23:15:54,770][105620] Updated weights for policy 1, policy_version 1084542 (0.0009) [2023-12-26 23:15:55,047][105692] Updated weights for policy 0, policy_version 1083489 (0.0009) [2023-12-26 23:15:55,098][105692] Updated weights for policy 0, policy_version 1083499 (0.0008) [2023-12-26 23:15:55,154][105692] Updated weights for policy 0, policy_version 1083509 (0.0009) [2023-12-26 23:15:55,202][105692] Updated weights for policy 0, policy_version 1083519 (0.0009) [2023-12-26 23:15:55,553][105620] Updated weights for policy 1, policy_version 1084552 (0.0009) [2023-12-26 23:15:55,610][105620] Updated weights for policy 1, policy_version 1084562 (0.0008) [2023-12-26 23:15:55,661][105620] Updated weights for policy 1, policy_version 1084572 (0.0009) [2023-12-26 23:15:55,984][105692] Updated weights for policy 0, policy_version 1083529 (0.0009) [2023-12-26 23:15:55,999][105585] KL-divergence is very high: 158.0752 [2023-12-26 23:15:56,043][105692] Updated weights for policy 0, policy_version 1083539 (0.0009) [2023-12-26 23:15:56,051][105585] KL-divergence is very high: 226.7264 [2023-12-26 23:15:56,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 555106304. Throughput: 0: 9689.3, 1: 9558.2. Samples: 555117760. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:15:56,062][104569] Avg episode reward: [(0, '9262.963'), (1, '9071.174')] [2023-12-26 23:15:56,097][105585] KL-divergence is very high: 184.1120 [2023-12-26 23:15:56,103][105692] Updated weights for policy 0, policy_version 1083549 (0.0009) [2023-12-26 23:15:56,421][105620] Updated weights for policy 1, policy_version 1084582 (0.0008) [2023-12-26 23:15:56,477][105620] Updated weights for policy 1, policy_version 1084592 (0.0009) [2023-12-26 23:15:56,528][105620] Updated weights for policy 1, policy_version 1084602 (0.0009) [2023-12-26 23:15:56,875][105692] Updated weights for policy 0, policy_version 1083559 (0.0009) [2023-12-26 23:15:56,942][105692] Updated weights for policy 0, policy_version 1083569 (0.0008) [2023-12-26 23:15:57,001][105692] Updated weights for policy 0, policy_version 1083579 (0.0010) [2023-12-26 23:15:57,224][105620] Updated weights for policy 1, policy_version 1084612 (0.0009) [2023-12-26 23:15:57,278][105620] Updated weights for policy 1, policy_version 1084622 (0.0009) [2023-12-26 23:15:57,341][105620] Updated weights for policy 1, policy_version 1084632 (0.0008) [2023-12-26 23:15:57,778][105692] Updated weights for policy 0, policy_version 1083589 (0.0009) [2023-12-26 23:15:57,827][105692] Updated weights for policy 0, policy_version 1083599 (0.0009) [2023-12-26 23:15:57,875][105692] Updated weights for policy 0, policy_version 1083609 (0.0009) [2023-12-26 23:15:58,085][105620] Updated weights for policy 1, policy_version 1084642 (0.0009) [2023-12-26 23:15:58,134][105620] Updated weights for policy 1, policy_version 1084652 (0.0008) [2023-12-26 23:15:58,202][105620] Updated weights for policy 1, policy_version 1084662 (0.0009) [2023-12-26 23:15:58,264][105620] Updated weights for policy 1, policy_version 1084672 (0.0009) [2023-12-26 23:15:58,652][105692] Updated weights for policy 0, policy_version 1083619 (0.0007) [2023-12-26 23:15:58,720][105692] Updated weights for policy 0, policy_version 1083629 (0.0007) [2023-12-26 23:15:58,786][105692] Updated weights for policy 0, policy_version 1083639 (0.0008) [2023-12-26 23:15:59,065][105620] Updated weights for policy 1, policy_version 1084682 (0.0008) [2023-12-26 23:15:59,125][105620] Updated weights for policy 1, policy_version 1084692 (0.0008) [2023-12-26 23:15:59,174][105620] Updated weights for policy 1, policy_version 1084702 (0.0008) [2023-12-26 23:15:59,529][105692] Updated weights for policy 0, policy_version 1083649 (0.0009) [2023-12-26 23:15:59,596][105692] Updated weights for policy 0, policy_version 1083659 (0.0010) [2023-12-26 23:15:59,656][105692] Updated weights for policy 0, policy_version 1083669 (0.0010) [2023-12-26 23:15:59,709][105692] Updated weights for policy 0, policy_version 1083679 (0.0010) [2023-12-26 23:15:59,995][105620] Updated weights for policy 1, policy_version 1084712 (0.0008) [2023-12-26 23:16:00,064][105620] Updated weights for policy 1, policy_version 1084722 (0.0009) [2023-12-26 23:16:00,121][105620] Updated weights for policy 1, policy_version 1084732 (0.0010) [2023-12-26 23:16:00,373][105692] Updated weights for policy 0, policy_version 1083689 (0.0008) [2023-12-26 23:16:00,425][105692] Updated weights for policy 0, policy_version 1083699 (0.0006) [2023-12-26 23:16:00,471][105692] Updated weights for policy 0, policy_version 1083709 (0.0010) [2023-12-26 23:16:00,912][105620] Updated weights for policy 1, policy_version 1084742 (0.0007) [2023-12-26 23:16:00,963][105620] Updated weights for policy 1, policy_version 1084752 (0.0005) [2023-12-26 23:16:01,015][105620] Updated weights for policy 1, policy_version 1084762 (0.0007) [2023-12-26 23:16:01,030][105692] Updated weights for policy 0, policy_version 1083719 (0.0006) [2023-12-26 23:16:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 555204608. Throughput: 0: 9636.2, 1: 9525.1. Samples: 555173308. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:16:01,062][104569] Avg episode reward: [(0, '9179.483'), (1, '9070.975')] [2023-12-26 23:16:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001084768_277733376.pth... [2023-12-26 23:16:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001083648_277446656.pth [2023-12-26 23:16:01,086][105692] Updated weights for policy 0, policy_version 1083729 (0.0008) [2023-12-26 23:16:01,134][105692] Updated weights for policy 0, policy_version 1083739 (0.0010) [2023-12-26 23:16:01,164][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001083744_277479424.pth... [2023-12-26 23:16:01,169][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001082592_277184512.pth [2023-12-26 23:16:01,792][105620] Updated weights for policy 1, policy_version 1084772 (0.0008) [2023-12-26 23:16:01,842][105620] Updated weights for policy 1, policy_version 1084782 (0.0008) [2023-12-26 23:16:01,899][105620] Updated weights for policy 1, policy_version 1084792 (0.0008) [2023-12-26 23:16:01,938][105692] Updated weights for policy 0, policy_version 1083749 (0.0009) [2023-12-26 23:16:01,991][105692] Updated weights for policy 0, policy_version 1083759 (0.0010) [2023-12-26 23:16:02,043][105692] Updated weights for policy 0, policy_version 1083769 (0.0010) [2023-12-26 23:16:02,698][105620] Updated weights for policy 1, policy_version 1084802 (0.0007) [2023-12-26 23:16:02,762][105620] Updated weights for policy 1, policy_version 1084812 (0.0008) [2023-12-26 23:16:02,791][105692] Updated weights for policy 0, policy_version 1083779 (0.0010) [2023-12-26 23:16:02,825][105620] Updated weights for policy 1, policy_version 1084822 (0.0008) [2023-12-26 23:16:02,851][105692] Updated weights for policy 0, policy_version 1083789 (0.0009) [2023-12-26 23:16:02,883][105620] Updated weights for policy 1, policy_version 1084832 (0.0008) [2023-12-26 23:16:02,911][105692] Updated weights for policy 0, policy_version 1083799 (0.0008) [2023-12-26 23:16:03,548][105692] Updated weights for policy 0, policy_version 1083809 (0.0009) [2023-12-26 23:16:03,610][105692] Updated weights for policy 0, policy_version 1083819 (0.0007) [2023-12-26 23:16:03,665][105692] Updated weights for policy 0, policy_version 1083829 (0.0007) [2023-12-26 23:16:03,703][105620] Updated weights for policy 1, policy_version 1084842 (0.0010) [2023-12-26 23:16:03,709][105692] Updated weights for policy 0, policy_version 1083839 (0.0005) [2023-12-26 23:16:03,765][105620] Updated weights for policy 1, policy_version 1084852 (0.0010) [2023-12-26 23:16:03,816][105620] Updated weights for policy 1, policy_version 1084862 (0.0010) [2023-12-26 23:16:04,400][105692] Updated weights for policy 0, policy_version 1083849 (0.0009) [2023-12-26 23:16:04,454][105692] Updated weights for policy 0, policy_version 1083859 (0.0010) [2023-12-26 23:16:04,499][105620] Updated weights for policy 1, policy_version 1084872 (0.0011) [2023-12-26 23:16:04,508][105692] Updated weights for policy 0, policy_version 1083869 (0.0006) [2023-12-26 23:16:04,558][105620] Updated weights for policy 1, policy_version 1084882 (0.0010) [2023-12-26 23:16:04,620][105620] Updated weights for policy 1, policy_version 1084892 (0.0010) [2023-12-26 23:16:05,248][105692] Updated weights for policy 0, policy_version 1083879 (0.0008) [2023-12-26 23:16:05,300][105692] Updated weights for policy 0, policy_version 1083889 (0.0010) [2023-12-26 23:16:05,328][105620] Updated weights for policy 1, policy_version 1084902 (0.0009) [2023-12-26 23:16:05,345][105692] Updated weights for policy 0, policy_version 1083899 (0.0008) [2023-12-26 23:16:05,389][105620] Updated weights for policy 1, policy_version 1084912 (0.0010) [2023-12-26 23:16:05,450][105620] Updated weights for policy 1, policy_version 1084922 (0.0010) [2023-12-26 23:16:06,062][104569] Fps is (10 sec: 18841.4, 60 sec: 18978.2, 300 sec: 19549.7). Total num frames: 555294720. Throughput: 0: 9612.0, 1: 9525.7. Samples: 555288192. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:16:06,063][104569] Avg episode reward: [(0, '9177.313'), (1, '9254.683')] [2023-12-26 23:16:06,084][105692] Updated weights for policy 0, policy_version 1083909 (0.0008) [2023-12-26 23:16:06,145][105692] Updated weights for policy 0, policy_version 1083919 (0.0009) [2023-12-26 23:16:06,154][105620] Updated weights for policy 1, policy_version 1084932 (0.0011) [2023-12-26 23:16:06,209][105692] Updated weights for policy 0, policy_version 1083929 (0.0008) [2023-12-26 23:16:06,213][105620] Updated weights for policy 1, policy_version 1084942 (0.0007) [2023-12-26 23:16:06,278][105620] Updated weights for policy 1, policy_version 1084952 (0.0011) [2023-12-26 23:16:06,854][105692] Updated weights for policy 0, policy_version 1083939 (0.0009) [2023-12-26 23:16:06,901][105692] Updated weights for policy 0, policy_version 1083949 (0.0005) [2023-12-26 23:16:06,961][105692] Updated weights for policy 0, policy_version 1083959 (0.0005) [2023-12-26 23:16:07,030][105620] Updated weights for policy 1, policy_version 1084962 (0.0011) [2023-12-26 23:16:07,099][105620] Updated weights for policy 1, policy_version 1084972 (0.0011) [2023-12-26 23:16:07,166][105620] Updated weights for policy 1, policy_version 1084982 (0.0010) [2023-12-26 23:16:07,226][105620] Updated weights for policy 1, policy_version 1084992 (0.0011) [2023-12-26 23:16:07,552][105692] Updated weights for policy 0, policy_version 1083969 (0.0007) [2023-12-26 23:16:07,606][105692] Updated weights for policy 0, policy_version 1083979 (0.0010) [2023-12-26 23:16:07,668][105692] Updated weights for policy 0, policy_version 1083989 (0.0010) [2023-12-26 23:16:07,734][105692] Updated weights for policy 0, policy_version 1083999 (0.0009) [2023-12-26 23:16:07,853][105620] Updated weights for policy 1, policy_version 1085002 (0.0010) [2023-12-26 23:16:07,904][105620] Updated weights for policy 1, policy_version 1085012 (0.0010) [2023-12-26 23:16:07,952][105620] Updated weights for policy 1, policy_version 1085022 (0.0009) [2023-12-26 23:16:08,541][105692] Updated weights for policy 0, policy_version 1084009 (0.0010) [2023-12-26 23:16:08,546][105620] Updated weights for policy 1, policy_version 1085032 (0.0007) [2023-12-26 23:16:08,603][105692] Updated weights for policy 0, policy_version 1084019 (0.0010) [2023-12-26 23:16:08,604][105620] Updated weights for policy 1, policy_version 1085042 (0.0010) [2023-12-26 23:16:08,656][105620] Updated weights for policy 1, policy_version 1085052 (0.0008) [2023-12-26 23:16:08,670][105692] Updated weights for policy 0, policy_version 1084029 (0.0011) [2023-12-26 23:16:09,234][105620] Updated weights for policy 1, policy_version 1085062 (0.0008) [2023-12-26 23:16:09,294][105620] Updated weights for policy 1, policy_version 1085072 (0.0009) [2023-12-26 23:16:09,359][105620] Updated weights for policy 1, policy_version 1085082 (0.0012) [2023-12-26 23:16:09,395][105692] Updated weights for policy 0, policy_version 1084039 (0.0010) [2023-12-26 23:16:09,459][105692] Updated weights for policy 0, policy_version 1084049 (0.0011) [2023-12-26 23:16:09,526][105692] Updated weights for policy 0, policy_version 1084059 (0.0011) [2023-12-26 23:16:09,969][105620] Updated weights for policy 1, policy_version 1085092 (0.0009) [2023-12-26 23:16:10,026][105620] Updated weights for policy 1, policy_version 1085102 (0.0005) [2023-12-26 23:16:10,078][105620] Updated weights for policy 1, policy_version 1085112 (0.0006) [2023-12-26 23:16:10,321][105692] Updated weights for policy 0, policy_version 1084069 (0.0010) [2023-12-26 23:16:10,381][105692] Updated weights for policy 0, policy_version 1084079 (0.0006) [2023-12-26 23:16:10,444][105692] Updated weights for policy 0, policy_version 1084089 (0.0007) [2023-12-26 23:16:10,702][105620] Updated weights for policy 1, policy_version 1085122 (0.0005) [2023-12-26 23:16:10,755][105620] Updated weights for policy 1, policy_version 1085132 (0.0006) [2023-12-26 23:16:10,809][105620] Updated weights for policy 1, policy_version 1085142 (0.0008) [2023-12-26 23:16:10,868][105620] Updated weights for policy 1, policy_version 1085152 (0.0006) [2023-12-26 23:16:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 555401216. Throughput: 0: 9629.2, 1: 9602.8. Samples: 555409164. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:16:11,063][104569] Avg episode reward: [(0, '9265.323'), (1, '9163.164')] [2023-12-26 23:16:11,253][105692] Updated weights for policy 0, policy_version 1084099 (0.0009) [2023-12-26 23:16:11,316][105692] Updated weights for policy 0, policy_version 1084109 (0.0009) [2023-12-26 23:16:11,381][105692] Updated weights for policy 0, policy_version 1084119 (0.0009) [2023-12-26 23:16:11,554][105620] Updated weights for policy 1, policy_version 1085162 (0.0008) [2023-12-26 23:16:11,617][105620] Updated weights for policy 1, policy_version 1085172 (0.0009) [2023-12-26 23:16:11,682][105620] Updated weights for policy 1, policy_version 1085182 (0.0008) [2023-12-26 23:16:12,131][105692] Updated weights for policy 0, policy_version 1084129 (0.0008) [2023-12-26 23:16:12,197][105692] Updated weights for policy 0, policy_version 1084139 (0.0009) [2023-12-26 23:16:12,259][105692] Updated weights for policy 0, policy_version 1084149 (0.0009) [2023-12-26 23:16:12,322][105692] Updated weights for policy 0, policy_version 1084159 (0.0009) [2023-12-26 23:16:12,487][105620] Updated weights for policy 1, policy_version 1085192 (0.0009) [2023-12-26 23:16:12,546][105620] Updated weights for policy 1, policy_version 1085202 (0.0009) [2023-12-26 23:16:12,597][105620] Updated weights for policy 1, policy_version 1085212 (0.0009) [2023-12-26 23:16:13,088][105692] Updated weights for policy 0, policy_version 1084169 (0.0009) [2023-12-26 23:16:13,136][105692] Updated weights for policy 0, policy_version 1084179 (0.0009) [2023-12-26 23:16:13,192][105692] Updated weights for policy 0, policy_version 1084189 (0.0009) [2023-12-26 23:16:13,350][105620] Updated weights for policy 1, policy_version 1085222 (0.0010) [2023-12-26 23:16:13,396][105620] Updated weights for policy 1, policy_version 1085232 (0.0008) [2023-12-26 23:16:13,442][105620] Updated weights for policy 1, policy_version 1085242 (0.0009) [2023-12-26 23:16:13,960][105692] Updated weights for policy 0, policy_version 1084199 (0.0008) [2023-12-26 23:16:14,004][105692] Updated weights for policy 0, policy_version 1084209 (0.0007) [2023-12-26 23:16:14,067][105692] Updated weights for policy 0, policy_version 1084219 (0.0007) [2023-12-26 23:16:14,184][105620] Updated weights for policy 1, policy_version 1085252 (0.0008) [2023-12-26 23:16:14,230][105620] Updated weights for policy 1, policy_version 1085262 (0.0005) [2023-12-26 23:16:14,282][105620] Updated weights for policy 1, policy_version 1085272 (0.0008) [2023-12-26 23:16:14,894][105692] Updated weights for policy 0, policy_version 1084229 (0.0007) [2023-12-26 23:16:14,958][105692] Updated weights for policy 0, policy_version 1084239 (0.0006) [2023-12-26 23:16:15,004][105620] Updated weights for policy 1, policy_version 1085282 (0.0010) [2023-12-26 23:16:15,023][105692] Updated weights for policy 0, policy_version 1084249 (0.0007) [2023-12-26 23:16:15,066][105620] Updated weights for policy 1, policy_version 1085292 (0.0006) [2023-12-26 23:16:15,128][105620] Updated weights for policy 1, policy_version 1085302 (0.0009) [2023-12-26 23:16:15,185][105620] Updated weights for policy 1, policy_version 1085312 (0.0009) [2023-12-26 23:16:15,626][105692] Updated weights for policy 0, policy_version 1084259 (0.0007) [2023-12-26 23:16:15,677][105692] Updated weights for policy 0, policy_version 1084269 (0.0006) [2023-12-26 23:16:15,739][105692] Updated weights for policy 0, policy_version 1084279 (0.0008) [2023-12-26 23:16:15,967][105620] Updated weights for policy 1, policy_version 1085322 (0.0009) [2023-12-26 23:16:16,026][105620] Updated weights for policy 1, policy_version 1085332 (0.0010) [2023-12-26 23:16:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 555491328. Throughput: 0: 9591.2, 1: 9577.3. Samples: 555463488. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:16:16,063][104569] Avg episode reward: [(0, '9349.254'), (1, '9163.450')] [2023-12-26 23:16:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001084288_277618688.pth... [2023-12-26 23:16:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001083168_277331968.pth [2023-12-26 23:16:16,085][105620] Updated weights for policy 1, policy_version 1085342 (0.0010) [2023-12-26 23:16:16,091][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001085344_277880832.pth... [2023-12-26 23:16:16,096][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001084224_277594112.pth [2023-12-26 23:16:16,468][105692] Updated weights for policy 0, policy_version 1084289 (0.0010) [2023-12-26 23:16:16,518][105692] Updated weights for policy 0, policy_version 1084299 (0.0010) [2023-12-26 23:16:16,569][105692] Updated weights for policy 0, policy_version 1084309 (0.0006) [2023-12-26 23:16:16,625][105692] Updated weights for policy 0, policy_version 1084319 (0.0005) [2023-12-26 23:16:16,821][105620] Updated weights for policy 1, policy_version 1085352 (0.0011) [2023-12-26 23:16:16,886][105620] Updated weights for policy 1, policy_version 1085362 (0.0010) [2023-12-26 23:16:16,951][105620] Updated weights for policy 1, policy_version 1085372 (0.0010) [2023-12-26 23:16:17,194][105692] Updated weights for policy 0, policy_version 1084329 (0.0009) [2023-12-26 23:16:17,252][105692] Updated weights for policy 0, policy_version 1084339 (0.0010) [2023-12-26 23:16:17,313][105692] Updated weights for policy 0, policy_version 1084349 (0.0010) [2023-12-26 23:16:17,680][105620] Updated weights for policy 1, policy_version 1085382 (0.0011) [2023-12-26 23:16:17,735][105620] Updated weights for policy 1, policy_version 1085392 (0.0011) [2023-12-26 23:16:17,788][105620] Updated weights for policy 1, policy_version 1085402 (0.0010) [2023-12-26 23:16:18,026][105692] Updated weights for policy 0, policy_version 1084359 (0.0010) [2023-12-26 23:16:18,090][105692] Updated weights for policy 0, policy_version 1084369 (0.0010) [2023-12-26 23:16:18,144][105692] Updated weights for policy 0, policy_version 1084379 (0.0010) [2023-12-26 23:16:18,524][105620] Updated weights for policy 1, policy_version 1085412 (0.0010) [2023-12-26 23:16:18,583][105620] Updated weights for policy 1, policy_version 1085422 (0.0011) [2023-12-26 23:16:18,643][105620] Updated weights for policy 1, policy_version 1085432 (0.0011) [2023-12-26 23:16:18,900][105692] Updated weights for policy 0, policy_version 1084389 (0.0010) [2023-12-26 23:16:18,959][105692] Updated weights for policy 0, policy_version 1084399 (0.0010) [2023-12-26 23:16:19,010][105692] Updated weights for policy 0, policy_version 1084409 (0.0010) [2023-12-26 23:16:19,404][105620] Updated weights for policy 1, policy_version 1085442 (0.0010) [2023-12-26 23:16:19,466][105620] Updated weights for policy 1, policy_version 1085452 (0.0010) [2023-12-26 23:16:19,530][105620] Updated weights for policy 1, policy_version 1085462 (0.0009) [2023-12-26 23:16:19,589][105620] Updated weights for policy 1, policy_version 1085472 (0.0009) [2023-12-26 23:16:19,742][105692] Updated weights for policy 0, policy_version 1084419 (0.0010) [2023-12-26 23:16:19,799][105692] Updated weights for policy 0, policy_version 1084429 (0.0007) [2023-12-26 23:16:19,862][105692] Updated weights for policy 0, policy_version 1084439 (0.0007) [2023-12-26 23:16:20,407][105620] Updated weights for policy 1, policy_version 1085482 (0.0010) [2023-12-26 23:16:20,456][105620] Updated weights for policy 1, policy_version 1085492 (0.0010) [2023-12-26 23:16:20,515][105620] Updated weights for policy 1, policy_version 1085502 (0.0010) [2023-12-26 23:16:20,639][105692] Updated weights for policy 0, policy_version 1084449 (0.0007) [2023-12-26 23:16:20,697][105692] Updated weights for policy 0, policy_version 1084459 (0.0006) [2023-12-26 23:16:20,747][105692] Updated weights for policy 0, policy_version 1084469 (0.0007) [2023-12-26 23:16:20,799][105692] Updated weights for policy 0, policy_version 1084479 (0.0006) [2023-12-26 23:16:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19521.9). Total num frames: 555589632. Throughput: 0: 9679.1, 1: 9541.1. Samples: 555578968. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:16:21,062][104569] Avg episode reward: [(0, '9348.983'), (1, '9164.556')] [2023-12-26 23:16:21,286][105620] Updated weights for policy 1, policy_version 1085512 (0.0008) [2023-12-26 23:16:21,335][105620] Updated weights for policy 1, policy_version 1085522 (0.0008) [2023-12-26 23:16:21,407][105620] Updated weights for policy 1, policy_version 1085532 (0.0008) [2023-12-26 23:16:21,512][105692] Updated weights for policy 0, policy_version 1084489 (0.0010) [2023-12-26 23:16:21,574][105692] Updated weights for policy 0, policy_version 1084499 (0.0009) [2023-12-26 23:16:21,581][105585] KL-divergence is very high: 103.7199 [2023-12-26 23:16:21,632][105585] KL-divergence is very high: 149.3921 [2023-12-26 23:16:21,641][105692] Updated weights for policy 0, policy_version 1084509 (0.0011) [2023-12-26 23:16:22,146][105620] Updated weights for policy 1, policy_version 1085542 (0.0009) [2023-12-26 23:16:22,203][105620] Updated weights for policy 1, policy_version 1085552 (0.0006) [2023-12-26 23:16:22,257][105620] Updated weights for policy 1, policy_version 1085562 (0.0008) [2023-12-26 23:16:22,406][105692] Updated weights for policy 0, policy_version 1084519 (0.0010) [2023-12-26 23:16:22,468][105692] Updated weights for policy 0, policy_version 1084529 (0.0009) [2023-12-26 23:16:22,530][105692] Updated weights for policy 0, policy_version 1084539 (0.0009) [2023-12-26 23:16:22,896][105620] Updated weights for policy 1, policy_version 1085572 (0.0010) [2023-12-26 23:16:22,958][105620] Updated weights for policy 1, policy_version 1085582 (0.0010) [2023-12-26 23:16:23,017][105620] Updated weights for policy 1, policy_version 1085592 (0.0009) [2023-12-26 23:16:23,269][105692] Updated weights for policy 0, policy_version 1084549 (0.0009) [2023-12-26 23:16:23,323][105692] Updated weights for policy 0, policy_version 1084559 (0.0009) [2023-12-26 23:16:23,382][105692] Updated weights for policy 0, policy_version 1084569 (0.0009) [2023-12-26 23:16:23,767][105620] Updated weights for policy 1, policy_version 1085602 (0.0008) [2023-12-26 23:16:23,814][105620] Updated weights for policy 1, policy_version 1085612 (0.0009) [2023-12-26 23:16:23,868][105620] Updated weights for policy 1, policy_version 1085622 (0.0009) [2023-12-26 23:16:23,926][105620] Updated weights for policy 1, policy_version 1085632 (0.0009) [2023-12-26 23:16:24,155][105692] Updated weights for policy 0, policy_version 1084579 (0.0009) [2023-12-26 23:16:24,209][105692] Updated weights for policy 0, policy_version 1084589 (0.0009) [2023-12-26 23:16:24,262][105692] Updated weights for policy 0, policy_version 1084599 (0.0009) [2023-12-26 23:16:24,723][105620] Updated weights for policy 1, policy_version 1085642 (0.0009) [2023-12-26 23:16:24,786][105620] Updated weights for policy 1, policy_version 1085652 (0.0009) [2023-12-26 23:16:24,850][105620] Updated weights for policy 1, policy_version 1085662 (0.0008) [2023-12-26 23:16:24,960][105692] Updated weights for policy 0, policy_version 1084609 (0.0008) [2023-12-26 23:16:25,007][105692] Updated weights for policy 0, policy_version 1084619 (0.0008) [2023-12-26 23:16:25,074][105692] Updated weights for policy 0, policy_version 1084629 (0.0009) [2023-12-26 23:16:25,132][105692] Updated weights for policy 0, policy_version 1084639 (0.0006) [2023-12-26 23:16:25,533][105620] Updated weights for policy 1, policy_version 1085672 (0.0009) [2023-12-26 23:16:25,597][105620] Updated weights for policy 1, policy_version 1085682 (0.0009) [2023-12-26 23:16:25,658][105620] Updated weights for policy 1, policy_version 1085692 (0.0009) [2023-12-26 23:16:25,745][105692] Updated weights for policy 0, policy_version 1084649 (0.0005) [2023-12-26 23:16:25,793][105692] Updated weights for policy 0, policy_version 1084659 (0.0007) [2023-12-26 23:16:25,841][105692] Updated weights for policy 0, policy_version 1084669 (0.0009) [2023-12-26 23:16:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 555687936. Throughput: 0: 9638.2, 1: 9525.2. Samples: 555693796. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:16:26,062][104569] Avg episode reward: [(0, '9258.808'), (1, '9073.262')] [2023-12-26 23:16:26,461][105692] Updated weights for policy 0, policy_version 1084679 (0.0009) [2023-12-26 23:16:26,497][105620] Updated weights for policy 1, policy_version 1085702 (0.0008) [2023-12-26 23:16:26,519][105692] Updated weights for policy 0, policy_version 1084689 (0.0008) [2023-12-26 23:16:26,560][105620] Updated weights for policy 1, policy_version 1085712 (0.0009) [2023-12-26 23:16:26,575][105692] Updated weights for policy 0, policy_version 1084699 (0.0007) [2023-12-26 23:16:26,618][105620] Updated weights for policy 1, policy_version 1085722 (0.0009) [2023-12-26 23:16:27,253][105692] Updated weights for policy 0, policy_version 1084709 (0.0007) [2023-12-26 23:16:27,319][105692] Updated weights for policy 0, policy_version 1084719 (0.0008) [2023-12-26 23:16:27,377][105692] Updated weights for policy 0, policy_version 1084729 (0.0008) [2023-12-26 23:16:27,389][105620] Updated weights for policy 1, policy_version 1085732 (0.0010) [2023-12-26 23:16:27,447][105620] Updated weights for policy 1, policy_version 1085742 (0.0010) [2023-12-26 23:16:27,508][105620] Updated weights for policy 1, policy_version 1085752 (0.0010) [2023-12-26 23:16:28,070][105692] Updated weights for policy 0, policy_version 1084739 (0.0009) [2023-12-26 23:16:28,123][105692] Updated weights for policy 0, policy_version 1084749 (0.0009) [2023-12-26 23:16:28,180][105692] Updated weights for policy 0, policy_version 1084759 (0.0009) [2023-12-26 23:16:28,197][105620] Updated weights for policy 1, policy_version 1085762 (0.0009) [2023-12-26 23:16:28,252][105620] Updated weights for policy 1, policy_version 1085772 (0.0009) [2023-12-26 23:16:28,296][105620] Updated weights for policy 1, policy_version 1085782 (0.0010) [2023-12-26 23:16:28,358][105620] Updated weights for policy 1, policy_version 1085792 (0.0007) [2023-12-26 23:16:29,007][105692] Updated weights for policy 0, policy_version 1084769 (0.0007) [2023-12-26 23:16:29,046][105620] Updated weights for policy 1, policy_version 1085802 (0.0009) [2023-12-26 23:16:29,061][105692] Updated weights for policy 0, policy_version 1084779 (0.0006) [2023-12-26 23:16:29,111][105692] Updated weights for policy 0, policy_version 1084789 (0.0007) [2023-12-26 23:16:29,112][105620] Updated weights for policy 1, policy_version 1085812 (0.0008) [2023-12-26 23:16:29,164][105692] Updated weights for policy 0, policy_version 1084799 (0.0006) [2023-12-26 23:16:29,175][105620] Updated weights for policy 1, policy_version 1085822 (0.0008) [2023-12-26 23:16:29,901][105620] Updated weights for policy 1, policy_version 1085832 (0.0008) [2023-12-26 23:16:29,930][105692] Updated weights for policy 0, policy_version 1084809 (0.0009) [2023-12-26 23:16:29,968][105620] Updated weights for policy 1, policy_version 1085842 (0.0006) [2023-12-26 23:16:29,991][105692] Updated weights for policy 0, policy_version 1084819 (0.0008) [2023-12-26 23:16:30,025][105620] Updated weights for policy 1, policy_version 1085852 (0.0006) [2023-12-26 23:16:30,055][105692] Updated weights for policy 0, policy_version 1084829 (0.0009) [2023-12-26 23:16:30,739][105620] Updated weights for policy 1, policy_version 1085862 (0.0008) [2023-12-26 23:16:30,801][105620] Updated weights for policy 1, policy_version 1085872 (0.0008) [2023-12-26 23:16:30,804][105692] Updated weights for policy 0, policy_version 1084839 (0.0007) [2023-12-26 23:16:30,843][105620] Updated weights for policy 1, policy_version 1085882 (0.0006) [2023-12-26 23:16:30,848][105692] Updated weights for policy 0, policy_version 1084849 (0.0006) [2023-12-26 23:16:30,899][105692] Updated weights for policy 0, policy_version 1084859 (0.0008) [2023-12-26 23:16:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 555786240. Throughput: 0: 9748.1, 1: 9499.1. Samples: 555752952. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:16:31,063][104569] Avg episode reward: [(0, '9259.117'), (1, '9072.872')] [2023-12-26 23:16:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001084864_277766144.pth... [2023-12-26 23:16:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001085888_278020096.pth... [2023-12-26 23:16:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001084768_277733376.pth [2023-12-26 23:16:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001083744_277479424.pth [2023-12-26 23:16:31,538][105620] Updated weights for policy 1, policy_version 1085892 (0.0006) [2023-12-26 23:16:31,590][105620] Updated weights for policy 1, policy_version 1085902 (0.0008) [2023-12-26 23:16:31,647][105620] Updated weights for policy 1, policy_version 1085912 (0.0008) [2023-12-26 23:16:31,692][105692] Updated weights for policy 0, policy_version 1084869 (0.0007) [2023-12-26 23:16:31,752][105692] Updated weights for policy 0, policy_version 1084879 (0.0010) [2023-12-26 23:16:31,812][105692] Updated weights for policy 0, policy_version 1084889 (0.0007) [2023-12-26 23:16:32,436][105620] Updated weights for policy 1, policy_version 1085922 (0.0008) [2023-12-26 23:16:32,478][105692] Updated weights for policy 0, policy_version 1084899 (0.0007) [2023-12-26 23:16:32,485][105620] Updated weights for policy 1, policy_version 1085932 (0.0007) [2023-12-26 23:16:32,528][105620] Updated weights for policy 1, policy_version 1085942 (0.0007) [2023-12-26 23:16:32,541][105692] Updated weights for policy 0, policy_version 1084909 (0.0010) [2023-12-26 23:16:32,588][105620] Updated weights for policy 1, policy_version 1085952 (0.0006) [2023-12-26 23:16:32,604][105692] Updated weights for policy 0, policy_version 1084919 (0.0010) [2023-12-26 23:16:33,226][105692] Updated weights for policy 0, policy_version 1084929 (0.0007) [2023-12-26 23:16:33,268][105620] Updated weights for policy 1, policy_version 1085962 (0.0005) [2023-12-26 23:16:33,275][105692] Updated weights for policy 0, policy_version 1084939 (0.0010) [2023-12-26 23:16:33,317][105620] Updated weights for policy 1, policy_version 1085972 (0.0005) [2023-12-26 23:16:33,320][105692] Updated weights for policy 0, policy_version 1084949 (0.0010) [2023-12-26 23:16:33,371][105692] Updated weights for policy 0, policy_version 1084959 (0.0010) [2023-12-26 23:16:33,374][105620] Updated weights for policy 1, policy_version 1085982 (0.0005) [2023-12-26 23:16:34,004][105620] Updated weights for policy 1, policy_version 1085992 (0.0009) [2023-12-26 23:16:34,054][105620] Updated weights for policy 1, policy_version 1086002 (0.0008) [2023-12-26 23:16:34,114][105620] Updated weights for policy 1, policy_version 1086012 (0.0008) [2023-12-26 23:16:34,160][105692] Updated weights for policy 0, policy_version 1084969 (0.0009) [2023-12-26 23:16:34,224][105692] Updated weights for policy 0, policy_version 1084979 (0.0010) [2023-12-26 23:16:34,273][105692] Updated weights for policy 0, policy_version 1084989 (0.0009) [2023-12-26 23:16:34,879][105620] Updated weights for policy 1, policy_version 1086022 (0.0008) [2023-12-26 23:16:34,937][105620] Updated weights for policy 1, policy_version 1086032 (0.0009) [2023-12-26 23:16:34,992][105620] Updated weights for policy 1, policy_version 1086042 (0.0009) [2023-12-26 23:16:35,011][105692] Updated weights for policy 0, policy_version 1084999 (0.0008) [2023-12-26 23:16:35,065][105692] Updated weights for policy 0, policy_version 1085009 (0.0006) [2023-12-26 23:16:35,125][105692] Updated weights for policy 0, policy_version 1085019 (0.0005) [2023-12-26 23:16:35,772][105692] Updated weights for policy 0, policy_version 1085029 (0.0007) [2023-12-26 23:16:35,794][105620] Updated weights for policy 1, policy_version 1086052 (0.0009) [2023-12-26 23:16:35,831][105692] Updated weights for policy 0, policy_version 1085039 (0.0005) [2023-12-26 23:16:35,854][105620] Updated weights for policy 1, policy_version 1086062 (0.0009) [2023-12-26 23:16:35,889][105692] Updated weights for policy 0, policy_version 1085049 (0.0005) [2023-12-26 23:16:35,906][105620] Updated weights for policy 1, policy_version 1086072 (0.0008) [2023-12-26 23:16:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 555884544. Throughput: 0: 9607.7, 1: 9547.4. Samples: 555868004. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:16:36,063][104569] Avg episode reward: [(0, '9349.563'), (1, '9256.117')] [2023-12-26 23:16:36,574][105692] Updated weights for policy 0, policy_version 1085059 (0.0007) [2023-12-26 23:16:36,626][105692] Updated weights for policy 0, policy_version 1085069 (0.0009) [2023-12-26 23:16:36,674][105692] Updated weights for policy 0, policy_version 1085079 (0.0009) [2023-12-26 23:16:36,709][105620] Updated weights for policy 1, policy_version 1086082 (0.0009) [2023-12-26 23:16:36,758][105620] Updated weights for policy 1, policy_version 1086092 (0.0008) [2023-12-26 23:16:36,806][105620] Updated weights for policy 1, policy_version 1086102 (0.0009) [2023-12-26 23:16:36,853][105620] Updated weights for policy 1, policy_version 1086112 (0.0009) [2023-12-26 23:16:37,466][105692] Updated weights for policy 0, policy_version 1085089 (0.0008) [2023-12-26 23:16:37,519][105692] Updated weights for policy 0, policy_version 1085099 (0.0010) [2023-12-26 23:16:37,571][105692] Updated weights for policy 0, policy_version 1085109 (0.0008) [2023-12-26 23:16:37,590][105620] Updated weights for policy 1, policy_version 1086122 (0.0006) [2023-12-26 23:16:37,625][105692] Updated weights for policy 0, policy_version 1085119 (0.0006) [2023-12-26 23:16:37,644][105620] Updated weights for policy 1, policy_version 1086132 (0.0006) [2023-12-26 23:16:37,704][105620] Updated weights for policy 1, policy_version 1086142 (0.0008) [2023-12-26 23:16:38,397][105620] Updated weights for policy 1, policy_version 1086152 (0.0007) [2023-12-26 23:16:38,447][105692] Updated weights for policy 0, policy_version 1085129 (0.0008) [2023-12-26 23:16:38,461][105620] Updated weights for policy 1, policy_version 1086162 (0.0006) [2023-12-26 23:16:38,510][105692] Updated weights for policy 0, policy_version 1085139 (0.0008) [2023-12-26 23:16:38,520][105620] Updated weights for policy 1, policy_version 1086172 (0.0007) [2023-12-26 23:16:38,569][105692] Updated weights for policy 0, policy_version 1085149 (0.0008) [2023-12-26 23:16:39,189][105620] Updated weights for policy 1, policy_version 1086182 (0.0007) [2023-12-26 23:16:39,259][105620] Updated weights for policy 1, policy_version 1086192 (0.0008) [2023-12-26 23:16:39,322][105620] Updated weights for policy 1, policy_version 1086203 (0.0009) [2023-12-26 23:16:39,368][105692] Updated weights for policy 0, policy_version 1085159 (0.0008) [2023-12-26 23:16:39,436][105692] Updated weights for policy 0, policy_version 1085169 (0.0009) [2023-12-26 23:16:39,503][105692] Updated weights for policy 0, policy_version 1085179 (0.0009) [2023-12-26 23:16:40,016][105620] Updated weights for policy 1, policy_version 1086213 (0.0006) [2023-12-26 23:16:40,077][105620] Updated weights for policy 1, policy_version 1086223 (0.0008) [2023-12-26 23:16:40,136][105620] Updated weights for policy 1, policy_version 1086233 (0.0009) [2023-12-26 23:16:40,335][105692] Updated weights for policy 0, policy_version 1085189 (0.0007) [2023-12-26 23:16:40,394][105692] Updated weights for policy 0, policy_version 1085199 (0.0008) [2023-12-26 23:16:40,449][105692] Updated weights for policy 0, policy_version 1085209 (0.0005) [2023-12-26 23:16:40,868][105620] Updated weights for policy 1, policy_version 1086243 (0.0009) [2023-12-26 23:16:40,931][105620] Updated weights for policy 1, policy_version 1086253 (0.0009) [2023-12-26 23:16:40,996][105620] Updated weights for policy 1, policy_version 1086263 (0.0009) [2023-12-26 23:16:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19114.7, 300 sec: 19494.2). Total num frames: 555974656. Throughput: 0: 9616.3, 1: 9585.3. Samples: 555981832. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:16:41,063][104569] Avg episode reward: [(0, '9349.358'), (1, '9165.247')] [2023-12-26 23:16:41,173][105692] Updated weights for policy 0, policy_version 1085219 (0.0008) [2023-12-26 23:16:41,240][105692] Updated weights for policy 0, policy_version 1085229 (0.0007) [2023-12-26 23:16:41,308][105692] Updated weights for policy 0, policy_version 1085239 (0.0008) [2023-12-26 23:16:41,796][105620] Updated weights for policy 1, policy_version 1086273 (0.0010) [2023-12-26 23:16:41,861][105620] Updated weights for policy 1, policy_version 1086283 (0.0008) [2023-12-26 23:16:41,921][105620] Updated weights for policy 1, policy_version 1086293 (0.0009) [2023-12-26 23:16:41,983][105620] Updated weights for policy 1, policy_version 1086303 (0.0008) [2023-12-26 23:16:42,092][105692] Updated weights for policy 0, policy_version 1085249 (0.0008) [2023-12-26 23:16:42,154][105692] Updated weights for policy 0, policy_version 1085259 (0.0010) [2023-12-26 23:16:42,212][105692] Updated weights for policy 0, policy_version 1085269 (0.0010) [2023-12-26 23:16:42,276][105692] Updated weights for policy 0, policy_version 1085279 (0.0010) [2023-12-26 23:16:42,696][105620] Updated weights for policy 1, policy_version 1086313 (0.0010) [2023-12-26 23:16:42,751][105620] Updated weights for policy 1, policy_version 1086323 (0.0008) [2023-12-26 23:16:42,824][105620] Updated weights for policy 1, policy_version 1086333 (0.0010) [2023-12-26 23:16:43,014][105692] Updated weights for policy 0, policy_version 1085289 (0.0008) [2023-12-26 23:16:43,074][105692] Updated weights for policy 0, policy_version 1085299 (0.0008) [2023-12-26 23:16:43,134][105692] Updated weights for policy 0, policy_version 1085309 (0.0008) [2023-12-26 23:16:43,538][105620] Updated weights for policy 1, policy_version 1086343 (0.0009) [2023-12-26 23:16:43,597][105620] Updated weights for policy 1, policy_version 1086353 (0.0010) [2023-12-26 23:16:43,659][105620] Updated weights for policy 1, policy_version 1086363 (0.0010) [2023-12-26 23:16:43,875][105692] Updated weights for policy 0, policy_version 1085319 (0.0008) [2023-12-26 23:16:43,933][105692] Updated weights for policy 0, policy_version 1085329 (0.0008) [2023-12-26 23:16:43,987][105692] Updated weights for policy 0, policy_version 1085339 (0.0010) [2023-12-26 23:16:44,394][105620] Updated weights for policy 1, policy_version 1086373 (0.0011) [2023-12-26 23:16:44,455][105620] Updated weights for policy 1, policy_version 1086383 (0.0008) [2023-12-26 23:16:44,519][105620] Updated weights for policy 1, policy_version 1086393 (0.0005) [2023-12-26 23:16:44,730][105692] Updated weights for policy 0, policy_version 1085349 (0.0011) [2023-12-26 23:16:44,788][105692] Updated weights for policy 0, policy_version 1085359 (0.0011) [2023-12-26 23:16:44,846][105692] Updated weights for policy 0, policy_version 1085369 (0.0011) [2023-12-26 23:16:45,109][105620] Updated weights for policy 1, policy_version 1086403 (0.0006) [2023-12-26 23:16:45,169][105620] Updated weights for policy 1, policy_version 1086413 (0.0011) [2023-12-26 23:16:45,219][105620] Updated weights for policy 1, policy_version 1086423 (0.0011) [2023-12-26 23:16:45,611][105692] Updated weights for policy 0, policy_version 1085379 (0.0011) [2023-12-26 23:16:45,665][105692] Updated weights for policy 0, policy_version 1085389 (0.0010) [2023-12-26 23:16:45,727][105692] Updated weights for policy 0, policy_version 1085399 (0.0010) [2023-12-26 23:16:45,962][105620] Updated weights for policy 1, policy_version 1086433 (0.0011) [2023-12-26 23:16:46,009][105620] Updated weights for policy 1, policy_version 1086443 (0.0010) [2023-12-26 23:16:46,054][105620] Updated weights for policy 1, policy_version 1086453 (0.0010) [2023-12-26 23:16:46,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19114.7, 300 sec: 19494.2). Total num frames: 556064768. Throughput: 0: 9621.9, 1: 9579.9. Samples: 556037388. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:16:46,062][104569] Avg episode reward: [(0, '9349.147'), (1, '9073.396')] [2023-12-26 23:16:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001085408_277905408.pth... [2023-12-26 23:16:46,095][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001084288_277618688.pth [2023-12-26 23:16:46,106][105620] Updated weights for policy 1, policy_version 1086463 (0.0010) [2023-12-26 23:16:46,108][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001086464_278167552.pth... [2023-12-26 23:16:46,111][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001085344_277880832.pth [2023-12-26 23:16:46,428][105692] Updated weights for policy 0, policy_version 1085409 (0.0010) [2023-12-26 23:16:46,481][105692] Updated weights for policy 0, policy_version 1085419 (0.0010) [2023-12-26 23:16:46,543][105692] Updated weights for policy 0, policy_version 1085429 (0.0007) [2023-12-26 23:16:46,609][105692] Updated weights for policy 0, policy_version 1085439 (0.0005) [2023-12-26 23:16:46,864][105620] Updated weights for policy 1, policy_version 1086473 (0.0010) [2023-12-26 23:16:46,919][105620] Updated weights for policy 1, policy_version 1086483 (0.0010) [2023-12-26 23:16:46,977][105620] Updated weights for policy 1, policy_version 1086493 (0.0010) [2023-12-26 23:16:47,136][105692] Updated weights for policy 0, policy_version 1085449 (0.0005) [2023-12-26 23:16:47,192][105692] Updated weights for policy 0, policy_version 1085459 (0.0005) [2023-12-26 23:16:47,252][105692] Updated weights for policy 0, policy_version 1085469 (0.0009) [2023-12-26 23:16:47,663][105620] Updated weights for policy 1, policy_version 1086503 (0.0009) [2023-12-26 23:16:47,726][105620] Updated weights for policy 1, policy_version 1086514 (0.0011) [2023-12-26 23:16:47,780][105692] Updated weights for policy 0, policy_version 1085479 (0.0009) [2023-12-26 23:16:47,794][105620] Updated weights for policy 1, policy_version 1086524 (0.0006) [2023-12-26 23:16:47,825][105692] Updated weights for policy 0, policy_version 1085489 (0.0007) [2023-12-26 23:16:47,874][105692] Updated weights for policy 0, policy_version 1085499 (0.0008) [2023-12-26 23:16:48,400][105620] Updated weights for policy 1, policy_version 1086534 (0.0007) [2023-12-26 23:16:48,462][105620] Updated weights for policy 1, policy_version 1086544 (0.0008) [2023-12-26 23:16:48,519][105692] Updated weights for policy 0, policy_version 1085509 (0.0008) [2023-12-26 23:16:48,528][105620] Updated weights for policy 1, policy_version 1086554 (0.0007) [2023-12-26 23:16:48,578][105692] Updated weights for policy 0, policy_version 1085519 (0.0010) [2023-12-26 23:16:48,634][105692] Updated weights for policy 0, policy_version 1085529 (0.0009) [2023-12-26 23:16:49,243][105620] Updated weights for policy 1, policy_version 1086564 (0.0009) [2023-12-26 23:16:49,299][105620] Updated weights for policy 1, policy_version 1086574 (0.0009) [2023-12-26 23:16:49,331][105692] Updated weights for policy 0, policy_version 1085539 (0.0009) [2023-12-26 23:16:49,365][105620] Updated weights for policy 1, policy_version 1086584 (0.0009) [2023-12-26 23:16:49,403][105692] Updated weights for policy 0, policy_version 1085549 (0.0007) [2023-12-26 23:16:49,471][105692] Updated weights for policy 0, policy_version 1085559 (0.0007) [2023-12-26 23:16:50,051][105620] Updated weights for policy 1, policy_version 1086594 (0.0006) [2023-12-26 23:16:50,113][105620] Updated weights for policy 1, policy_version 1086604 (0.0008) [2023-12-26 23:16:50,120][105692] Updated weights for policy 0, policy_version 1085569 (0.0005) [2023-12-26 23:16:50,165][105620] Updated weights for policy 1, policy_version 1086614 (0.0007) [2023-12-26 23:16:50,171][105692] Updated weights for policy 0, policy_version 1085579 (0.0008) [2023-12-26 23:16:50,227][105620] Updated weights for policy 1, policy_version 1086624 (0.0006) [2023-12-26 23:16:50,229][105692] Updated weights for policy 0, policy_version 1085589 (0.0008) [2023-12-26 23:16:50,284][105692] Updated weights for policy 0, policy_version 1085599 (0.0009) [2023-12-26 23:16:50,968][105620] Updated weights for policy 1, policy_version 1086635 (0.0010) [2023-12-26 23:16:51,024][105620] Updated weights for policy 1, policy_version 1086645 (0.0008) [2023-12-26 23:16:51,038][105692] Updated weights for policy 0, policy_version 1085609 (0.0008) [2023-12-26 23:16:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 556163072. Throughput: 0: 9669.3, 1: 9707.9. Samples: 556160164. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:16:51,062][104569] Avg episode reward: [(0, '9258.929'), (1, '9163.812')] [2023-12-26 23:16:51,088][105620] Updated weights for policy 1, policy_version 1086655 (0.0006) [2023-12-26 23:16:51,098][105692] Updated weights for policy 0, policy_version 1085619 (0.0008) [2023-12-26 23:16:51,162][105692] Updated weights for policy 0, policy_version 1085629 (0.0009) [2023-12-26 23:16:51,856][105620] Updated weights for policy 1, policy_version 1086665 (0.0009) [2023-12-26 23:16:51,921][105620] Updated weights for policy 1, policy_version 1086675 (0.0009) [2023-12-26 23:16:51,956][105692] Updated weights for policy 0, policy_version 1085639 (0.0007) [2023-12-26 23:16:51,981][105620] Updated weights for policy 1, policy_version 1086685 (0.0009) [2023-12-26 23:16:52,004][105692] Updated weights for policy 0, policy_version 1085649 (0.0008) [2023-12-26 23:16:52,051][105692] Updated weights for policy 0, policy_version 1085659 (0.0008) [2023-12-26 23:16:52,754][105620] Updated weights for policy 1, policy_version 1086695 (0.0008) [2023-12-26 23:16:52,798][105692] Updated weights for policy 0, policy_version 1085669 (0.0007) [2023-12-26 23:16:52,805][105620] Updated weights for policy 1, policy_version 1086705 (0.0008) [2023-12-26 23:16:52,856][105692] Updated weights for policy 0, policy_version 1085679 (0.0007) [2023-12-26 23:16:52,866][105620] Updated weights for policy 1, policy_version 1086715 (0.0006) [2023-12-26 23:16:52,915][105692] Updated weights for policy 0, policy_version 1085689 (0.0007) [2023-12-26 23:16:53,539][105620] Updated weights for policy 1, policy_version 1086725 (0.0008) [2023-12-26 23:16:53,594][105620] Updated weights for policy 1, policy_version 1086735 (0.0005) [2023-12-26 23:16:53,645][105620] Updated weights for policy 1, policy_version 1086745 (0.0005) [2023-12-26 23:16:53,710][105692] Updated weights for policy 0, policy_version 1085699 (0.0009) [2023-12-26 23:16:53,776][105692] Updated weights for policy 0, policy_version 1085709 (0.0010) [2023-12-26 23:16:53,835][105692] Updated weights for policy 0, policy_version 1085719 (0.0010) [2023-12-26 23:16:54,253][105620] Updated weights for policy 1, policy_version 1086755 (0.0007) [2023-12-26 23:16:54,305][105620] Updated weights for policy 1, policy_version 1086765 (0.0007) [2023-12-26 23:16:54,356][105620] Updated weights for policy 1, policy_version 1086775 (0.0006) [2023-12-26 23:16:54,640][105692] Updated weights for policy 0, policy_version 1085729 (0.0009) [2023-12-26 23:16:54,698][105692] Updated weights for policy 0, policy_version 1085739 (0.0009) [2023-12-26 23:16:54,764][105692] Updated weights for policy 0, policy_version 1085749 (0.0009) [2023-12-26 23:16:54,826][105692] Updated weights for policy 0, policy_version 1085759 (0.0009) [2023-12-26 23:16:55,085][105620] Updated weights for policy 1, policy_version 1086785 (0.0007) [2023-12-26 23:16:55,143][105620] Updated weights for policy 1, policy_version 1086795 (0.0008) [2023-12-26 23:16:55,195][105620] Updated weights for policy 1, policy_version 1086805 (0.0006) [2023-12-26 23:16:55,250][105620] Updated weights for policy 1, policy_version 1086815 (0.0005) [2023-12-26 23:16:55,619][105692] Updated weights for policy 0, policy_version 1085769 (0.0010) [2023-12-26 23:16:55,679][105692] Updated weights for policy 0, policy_version 1085779 (0.0009) [2023-12-26 23:16:55,740][105692] Updated weights for policy 0, policy_version 1085789 (0.0009) [2023-12-26 23:16:55,929][105620] Updated weights for policy 1, policy_version 1086825 (0.0005) [2023-12-26 23:16:55,988][105620] Updated weights for policy 1, policy_version 1086835 (0.0006) [2023-12-26 23:16:56,044][105620] Updated weights for policy 1, policy_version 1086845 (0.0005) [2023-12-26 23:16:56,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 556269568. Throughput: 0: 9617.4, 1: 9600.9. Samples: 556273988. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:16:56,062][104569] Avg episode reward: [(0, '9078.441'), (1, '8980.609')] [2023-12-26 23:16:56,539][105692] Updated weights for policy 0, policy_version 1085799 (0.0010) [2023-12-26 23:16:56,591][105692] Updated weights for policy 0, policy_version 1085810 (0.0009) [2023-12-26 23:16:56,617][105620] Updated weights for policy 1, policy_version 1086855 (0.0007) [2023-12-26 23:16:56,639][105692] Updated weights for policy 0, policy_version 1085820 (0.0006) [2023-12-26 23:16:56,669][105620] Updated weights for policy 1, policy_version 1086865 (0.0007) [2023-12-26 23:16:56,726][105620] Updated weights for policy 1, policy_version 1086875 (0.0009) [2023-12-26 23:16:57,398][105692] Updated weights for policy 0, policy_version 1085830 (0.0006) [2023-12-26 23:16:57,448][105692] Updated weights for policy 0, policy_version 1085840 (0.0005) [2023-12-26 23:16:57,483][105620] Updated weights for policy 1, policy_version 1086885 (0.0007) [2023-12-26 23:16:57,515][105692] Updated weights for policy 0, policy_version 1085850 (0.0009) [2023-12-26 23:16:57,535][105620] Updated weights for policy 1, policy_version 1086895 (0.0007) [2023-12-26 23:16:57,584][105620] Updated weights for policy 1, policy_version 1086905 (0.0010) [2023-12-26 23:16:58,262][105620] Updated weights for policy 1, policy_version 1086915 (0.0009) [2023-12-26 23:16:58,269][105692] Updated weights for policy 0, policy_version 1085860 (0.0006) [2023-12-26 23:16:58,323][105620] Updated weights for policy 1, policy_version 1086925 (0.0009) [2023-12-26 23:16:58,331][105692] Updated weights for policy 0, policy_version 1085870 (0.0007) [2023-12-26 23:16:58,394][105620] Updated weights for policy 1, policy_version 1086935 (0.0011) [2023-12-26 23:16:58,404][105692] Updated weights for policy 0, policy_version 1085880 (0.0007) [2023-12-26 23:16:59,156][105692] Updated weights for policy 0, policy_version 1085890 (0.0006) [2023-12-26 23:16:59,199][105620] Updated weights for policy 1, policy_version 1086945 (0.0011) [2023-12-26 23:16:59,224][105692] Updated weights for policy 0, policy_version 1085900 (0.0008) [2023-12-26 23:16:59,261][105620] Updated weights for policy 1, policy_version 1086955 (0.0007) [2023-12-26 23:16:59,297][105692] Updated weights for policy 0, policy_version 1085910 (0.0008) [2023-12-26 23:16:59,322][105620] Updated weights for policy 1, policy_version 1086965 (0.0006) [2023-12-26 23:16:59,363][105692] Updated weights for policy 0, policy_version 1085920 (0.0008) [2023-12-26 23:16:59,383][105620] Updated weights for policy 1, policy_version 1086975 (0.0008) [2023-12-26 23:17:00,080][105692] Updated weights for policy 0, policy_version 1085930 (0.0009) [2023-12-26 23:17:00,110][105620] Updated weights for policy 1, policy_version 1086985 (0.0006) [2023-12-26 23:17:00,141][105692] Updated weights for policy 0, policy_version 1085940 (0.0009) [2023-12-26 23:17:00,161][105620] Updated weights for policy 1, policy_version 1086995 (0.0006) [2023-12-26 23:17:00,199][105692] Updated weights for policy 0, policy_version 1085950 (0.0007) [2023-12-26 23:17:00,217][105620] Updated weights for policy 1, policy_version 1087005 (0.0005) [2023-12-26 23:17:00,750][105620] Updated weights for policy 1, policy_version 1087015 (0.0005) [2023-12-26 23:17:00,800][105620] Updated weights for policy 1, policy_version 1087025 (0.0005) [2023-12-26 23:17:00,851][105620] Updated weights for policy 1, policy_version 1087035 (0.0005) [2023-12-26 23:17:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 556359680. Throughput: 0: 9615.8, 1: 9654.8. Samples: 556330660. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:17:01,062][104569] Avg episode reward: [(0, '9169.104'), (1, '9256.459')] [2023-12-26 23:17:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001087040_278315008.pth... [2023-12-26 23:17:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001085888_278020096.pth [2023-12-26 23:17:01,090][105692] Updated weights for policy 0, policy_version 1085960 (0.0008) [2023-12-26 23:17:01,150][105692] Updated weights for policy 0, policy_version 1085970 (0.0008) [2023-12-26 23:17:01,222][105692] Updated weights for policy 0, policy_version 1085980 (0.0008) [2023-12-26 23:17:01,246][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001085984_278052864.pth... [2023-12-26 23:17:01,251][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001084864_277766144.pth [2023-12-26 23:17:01,497][105620] Updated weights for policy 1, policy_version 1087045 (0.0008) [2023-12-26 23:17:01,551][105620] Updated weights for policy 1, policy_version 1087055 (0.0010) [2023-12-26 23:17:01,606][105620] Updated weights for policy 1, policy_version 1087065 (0.0010) [2023-12-26 23:17:01,962][105692] Updated weights for policy 0, policy_version 1085990 (0.0007) [2023-12-26 23:17:02,006][105692] Updated weights for policy 0, policy_version 1086000 (0.0008) [2023-12-26 23:17:02,051][105692] Updated weights for policy 0, policy_version 1086010 (0.0007) [2023-12-26 23:17:02,356][105620] Updated weights for policy 1, policy_version 1087075 (0.0010) [2023-12-26 23:17:02,408][105620] Updated weights for policy 1, policy_version 1087085 (0.0008) [2023-12-26 23:17:02,459][105620] Updated weights for policy 1, policy_version 1087095 (0.0009) [2023-12-26 23:17:02,749][105692] Updated weights for policy 0, policy_version 1086020 (0.0008) [2023-12-26 23:17:02,812][105692] Updated weights for policy 0, policy_version 1086030 (0.0009) [2023-12-26 23:17:02,865][105692] Updated weights for policy 0, policy_version 1086040 (0.0010) [2023-12-26 23:17:03,022][105620] Updated weights for policy 1, policy_version 1087105 (0.0005) [2023-12-26 23:17:03,075][105620] Updated weights for policy 1, policy_version 1087115 (0.0005) [2023-12-26 23:17:03,130][105620] Updated weights for policy 1, policy_version 1087125 (0.0005) [2023-12-26 23:17:03,180][105620] Updated weights for policy 1, policy_version 1087135 (0.0005) [2023-12-26 23:17:03,639][105692] Updated weights for policy 0, policy_version 1086050 (0.0007) [2023-12-26 23:17:03,698][105692] Updated weights for policy 0, policy_version 1086060 (0.0008) [2023-12-26 23:17:03,756][105692] Updated weights for policy 0, policy_version 1086070 (0.0007) [2023-12-26 23:17:03,790][105620] Updated weights for policy 1, policy_version 1087145 (0.0010) [2023-12-26 23:17:03,804][105692] Updated weights for policy 0, policy_version 1086080 (0.0007) [2023-12-26 23:17:03,841][105620] Updated weights for policy 1, policy_version 1087155 (0.0010) [2023-12-26 23:17:03,902][105620] Updated weights for policy 1, policy_version 1087165 (0.0010) [2023-12-26 23:17:04,575][105692] Updated weights for policy 0, policy_version 1086090 (0.0008) [2023-12-26 23:17:04,637][105692] Updated weights for policy 0, policy_version 1086100 (0.0009) [2023-12-26 23:17:04,640][105620] Updated weights for policy 1, policy_version 1087175 (0.0010) [2023-12-26 23:17:04,691][105620] Updated weights for policy 1, policy_version 1087185 (0.0008) [2023-12-26 23:17:04,697][105692] Updated weights for policy 0, policy_version 1086110 (0.0007) [2023-12-26 23:17:04,743][105620] Updated weights for policy 1, policy_version 1087195 (0.0010) [2023-12-26 23:17:05,490][105692] Updated weights for policy 0, policy_version 1086120 (0.0007) [2023-12-26 23:17:05,497][105620] Updated weights for policy 1, policy_version 1087205 (0.0009) [2023-12-26 23:17:05,506][105585] KL-divergence is very high: 112.5495 [2023-12-26 23:17:05,549][105620] Updated weights for policy 1, policy_version 1087215 (0.0007) [2023-12-26 23:17:05,557][105585] KL-divergence is very high: 180.1065 [2023-12-26 23:17:05,559][105692] Updated weights for policy 0, policy_version 1086130 (0.0009) [2023-12-26 23:17:05,608][105620] Updated weights for policy 1, policy_version 1087225 (0.0006) [2023-12-26 23:17:05,610][105585] KL-divergence is very high: 181.6540 [2023-12-26 23:17:05,625][105692] Updated weights for policy 0, policy_version 1086140 (0.0009) [2023-12-26 23:17:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 556457984. Throughput: 0: 9520.7, 1: 9796.2. Samples: 556448232. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:17:06,063][104569] Avg episode reward: [(0, '9170.216'), (1, '9164.925')] [2023-12-26 23:17:06,176][105620] Updated weights for policy 1, policy_version 1087235 (0.0005) [2023-12-26 23:17:06,238][105620] Updated weights for policy 1, policy_version 1087245 (0.0005) [2023-12-26 23:17:06,297][105620] Updated weights for policy 1, policy_version 1087255 (0.0005) [2023-12-26 23:17:06,420][105692] Updated weights for policy 0, policy_version 1086150 (0.0009) [2023-12-26 23:17:06,485][105692] Updated weights for policy 0, policy_version 1086160 (0.0009) [2023-12-26 23:17:06,550][105692] Updated weights for policy 0, policy_version 1086170 (0.0009) [2023-12-26 23:17:06,997][105620] Updated weights for policy 1, policy_version 1087265 (0.0011) [2023-12-26 23:17:07,052][105620] Updated weights for policy 1, policy_version 1087275 (0.0010) [2023-12-26 23:17:07,109][105620] Updated weights for policy 1, policy_version 1087285 (0.0011) [2023-12-26 23:17:07,165][105620] Updated weights for policy 1, policy_version 1087295 (0.0010) [2023-12-26 23:17:07,319][105692] Updated weights for policy 0, policy_version 1086180 (0.0007) [2023-12-26 23:17:07,381][105692] Updated weights for policy 0, policy_version 1086190 (0.0008) [2023-12-26 23:17:07,442][105692] Updated weights for policy 0, policy_version 1086200 (0.0010) [2023-12-26 23:17:07,783][105620] Updated weights for policy 1, policy_version 1087305 (0.0006) [2023-12-26 23:17:07,828][105620] Updated weights for policy 1, policy_version 1087315 (0.0005) [2023-12-26 23:17:07,879][105620] Updated weights for policy 1, policy_version 1087325 (0.0005) [2023-12-26 23:17:08,326][105692] Updated weights for policy 0, policy_version 1086210 (0.0010) [2023-12-26 23:17:08,392][105692] Updated weights for policy 0, policy_version 1086220 (0.0007) [2023-12-26 23:17:08,455][105692] Updated weights for policy 0, policy_version 1086230 (0.0005) [2023-12-26 23:17:08,484][105620] Updated weights for policy 1, policy_version 1087335 (0.0009) [2023-12-26 23:17:08,522][105692] Updated weights for policy 0, policy_version 1086240 (0.0006) [2023-12-26 23:17:08,536][105620] Updated weights for policy 1, policy_version 1087345 (0.0010) [2023-12-26 23:17:08,588][105620] Updated weights for policy 1, policy_version 1087355 (0.0010) [2023-12-26 23:17:09,092][105692] Updated weights for policy 0, policy_version 1086250 (0.0010) [2023-12-26 23:17:09,137][105692] Updated weights for policy 0, policy_version 1086260 (0.0010) [2023-12-26 23:17:09,181][105692] Updated weights for policy 0, policy_version 1086270 (0.0010) [2023-12-26 23:17:09,261][105620] Updated weights for policy 1, policy_version 1087365 (0.0010) [2023-12-26 23:17:09,321][105620] Updated weights for policy 1, policy_version 1087375 (0.0008) [2023-12-26 23:17:09,396][105620] Updated weights for policy 1, policy_version 1087385 (0.0013) [2023-12-26 23:17:09,999][105692] Updated weights for policy 0, policy_version 1086280 (0.0008) [2023-12-26 23:17:10,068][105692] Updated weights for policy 0, policy_version 1086290 (0.0008) [2023-12-26 23:17:10,132][105692] Updated weights for policy 0, policy_version 1086300 (0.0008) [2023-12-26 23:17:10,141][105620] Updated weights for policy 1, policy_version 1087395 (0.0008) [2023-12-26 23:17:10,208][105620] Updated weights for policy 1, policy_version 1087405 (0.0011) [2023-12-26 23:17:10,261][105620] Updated weights for policy 1, policy_version 1087415 (0.0011) [2023-12-26 23:17:10,896][105692] Updated weights for policy 0, policy_version 1086310 (0.0007) [2023-12-26 23:17:10,952][105692] Updated weights for policy 0, policy_version 1086320 (0.0008) [2023-12-26 23:17:11,012][105692] Updated weights for policy 0, policy_version 1086330 (0.0008) [2023-12-26 23:17:11,020][105620] Updated weights for policy 1, policy_version 1087425 (0.0010) [2023-12-26 23:17:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 556556288. Throughput: 0: 9440.6, 1: 9905.5. Samples: 556564372. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:17:11,063][104569] Avg episode reward: [(0, '9354.188'), (1, '9071.727')] [2023-12-26 23:17:11,083][105620] Updated weights for policy 1, policy_version 1087435 (0.0009) [2023-12-26 23:17:11,150][105620] Updated weights for policy 1, policy_version 1087445 (0.0014) [2023-12-26 23:17:11,206][105620] Updated weights for policy 1, policy_version 1087455 (0.0008) [2023-12-26 23:17:11,788][105692] Updated weights for policy 0, policy_version 1086340 (0.0008) [2023-12-26 23:17:11,843][105692] Updated weights for policy 0, policy_version 1086350 (0.0009) [2023-12-26 23:17:11,903][105692] Updated weights for policy 0, policy_version 1086360 (0.0009) [2023-12-26 23:17:11,982][105620] Updated weights for policy 1, policy_version 1087465 (0.0006) [2023-12-26 23:17:12,041][105620] Updated weights for policy 1, policy_version 1087475 (0.0008) [2023-12-26 23:17:12,096][105620] Updated weights for policy 1, policy_version 1087485 (0.0009) [2023-12-26 23:17:12,622][105692] Updated weights for policy 0, policy_version 1086370 (0.0007) [2023-12-26 23:17:12,674][105692] Updated weights for policy 0, policy_version 1086380 (0.0009) [2023-12-26 23:17:12,726][105692] Updated weights for policy 0, policy_version 1086390 (0.0010) [2023-12-26 23:17:12,782][105692] Updated weights for policy 0, policy_version 1086400 (0.0010) [2023-12-26 23:17:12,787][105620] Updated weights for policy 1, policy_version 1087495 (0.0007) [2023-12-26 23:17:12,843][105620] Updated weights for policy 1, policy_version 1087505 (0.0008) [2023-12-26 23:17:12,889][105620] Updated weights for policy 1, policy_version 1087515 (0.0008) [2023-12-26 23:17:13,451][105692] Updated weights for policy 0, policy_version 1086410 (0.0010) [2023-12-26 23:17:13,495][105692] Updated weights for policy 0, policy_version 1086420 (0.0010) [2023-12-26 23:17:13,535][105620] Updated weights for policy 1, policy_version 1087525 (0.0008) [2023-12-26 23:17:13,540][105692] Updated weights for policy 0, policy_version 1086430 (0.0010) [2023-12-26 23:17:13,585][105620] Updated weights for policy 1, policy_version 1087535 (0.0005) [2023-12-26 23:17:13,639][105620] Updated weights for policy 1, policy_version 1087545 (0.0006) [2023-12-26 23:17:14,208][105620] Updated weights for policy 1, policy_version 1087555 (0.0005) [2023-12-26 23:17:14,271][105620] Updated weights for policy 1, policy_version 1087565 (0.0005) [2023-12-26 23:17:14,331][105620] Updated weights for policy 1, policy_version 1087575 (0.0005) [2023-12-26 23:17:14,355][105692] Updated weights for policy 0, policy_version 1086440 (0.0010) [2023-12-26 23:17:14,415][105692] Updated weights for policy 0, policy_version 1086450 (0.0010) [2023-12-26 23:17:14,473][105692] Updated weights for policy 0, policy_version 1086460 (0.0010) [2023-12-26 23:17:14,948][105620] Updated weights for policy 1, policy_version 1087585 (0.0006) [2023-12-26 23:17:15,018][105620] Updated weights for policy 1, policy_version 1087595 (0.0008) [2023-12-26 23:17:15,083][105620] Updated weights for policy 1, policy_version 1087605 (0.0009) [2023-12-26 23:17:15,149][105620] Updated weights for policy 1, policy_version 1087615 (0.0008) [2023-12-26 23:17:15,232][105692] Updated weights for policy 0, policy_version 1086470 (0.0011) [2023-12-26 23:17:15,301][105692] Updated weights for policy 0, policy_version 1086480 (0.0010) [2023-12-26 23:17:15,366][105692] Updated weights for policy 0, policy_version 1086490 (0.0010) [2023-12-26 23:17:15,814][105620] Updated weights for policy 1, policy_version 1087625 (0.0006) [2023-12-26 23:17:15,877][105620] Updated weights for policy 1, policy_version 1087635 (0.0005) [2023-12-26 23:17:15,932][105620] Updated weights for policy 1, policy_version 1087645 (0.0006) [2023-12-26 23:17:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 556654592. Throughput: 0: 9399.8, 1: 9958.7. Samples: 556624088. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:17:16,063][104569] Avg episode reward: [(0, '9352.659'), (1, '9070.398')] [2023-12-26 23:17:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001087648_278470656.pth... [2023-12-26 23:17:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001086464_278167552.pth [2023-12-26 23:17:16,083][105692] Updated weights for policy 0, policy_version 1086500 (0.0011) [2023-12-26 23:17:16,141][105692] Updated weights for policy 0, policy_version 1086510 (0.0010) [2023-12-26 23:17:16,189][105692] Updated weights for policy 0, policy_version 1086520 (0.0008) [2023-12-26 23:17:16,223][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001086528_278192128.pth... [2023-12-26 23:17:16,226][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001085408_277905408.pth [2023-12-26 23:17:16,576][105620] Updated weights for policy 1, policy_version 1087655 (0.0007) [2023-12-26 23:17:16,635][105620] Updated weights for policy 1, policy_version 1087665 (0.0008) [2023-12-26 23:17:16,682][105620] Updated weights for policy 1, policy_version 1087675 (0.0008) [2023-12-26 23:17:16,910][105692] Updated weights for policy 0, policy_version 1086530 (0.0010) [2023-12-26 23:17:16,973][105692] Updated weights for policy 0, policy_version 1086540 (0.0009) [2023-12-26 23:17:17,032][105692] Updated weights for policy 0, policy_version 1086550 (0.0010) [2023-12-26 23:17:17,090][105692] Updated weights for policy 0, policy_version 1086560 (0.0010) [2023-12-26 23:17:17,294][105620] Updated weights for policy 1, policy_version 1087685 (0.0009) [2023-12-26 23:17:17,348][105620] Updated weights for policy 1, policy_version 1087696 (0.0010) [2023-12-26 23:17:17,403][105620] Updated weights for policy 1, policy_version 1087707 (0.0009) [2023-12-26 23:17:17,729][105692] Updated weights for policy 0, policy_version 1086570 (0.0011) [2023-12-26 23:17:17,788][105692] Updated weights for policy 0, policy_version 1086580 (0.0010) [2023-12-26 23:17:17,836][105692] Updated weights for policy 0, policy_version 1086590 (0.0010) [2023-12-26 23:17:18,110][105620] Updated weights for policy 1, policy_version 1087717 (0.0010) [2023-12-26 23:17:18,157][105620] Updated weights for policy 1, policy_version 1087728 (0.0008) [2023-12-26 23:17:18,211][105620] Updated weights for policy 1, policy_version 1087739 (0.0010) [2023-12-26 23:17:18,482][105692] Updated weights for policy 0, policy_version 1086600 (0.0011) [2023-12-26 23:17:18,538][105692] Updated weights for policy 0, policy_version 1086610 (0.0011) [2023-12-26 23:17:18,585][105692] Updated weights for policy 0, policy_version 1086620 (0.0006) [2023-12-26 23:17:18,954][105620] Updated weights for policy 1, policy_version 1087750 (0.0008) [2023-12-26 23:17:19,013][105620] Updated weights for policy 1, policy_version 1087760 (0.0006) [2023-12-26 23:17:19,061][105620] Updated weights for policy 1, policy_version 1087770 (0.0008) [2023-12-26 23:17:19,274][105692] Updated weights for policy 0, policy_version 1086630 (0.0009) [2023-12-26 23:17:19,327][105692] Updated weights for policy 0, policy_version 1086640 (0.0008) [2023-12-26 23:17:19,389][105692] Updated weights for policy 0, policy_version 1086650 (0.0009) [2023-12-26 23:17:19,735][105620] Updated weights for policy 1, policy_version 1087780 (0.0009) [2023-12-26 23:17:19,799][105620] Updated weights for policy 1, policy_version 1087790 (0.0011) [2023-12-26 23:17:19,870][105620] Updated weights for policy 1, policy_version 1087800 (0.0011) [2023-12-26 23:17:20,170][105692] Updated weights for policy 0, policy_version 1086660 (0.0010) [2023-12-26 23:17:20,229][105692] Updated weights for policy 0, policy_version 1086670 (0.0008) [2023-12-26 23:17:20,290][105692] Updated weights for policy 0, policy_version 1086680 (0.0010) [2023-12-26 23:17:20,522][105620] Updated weights for policy 1, policy_version 1087810 (0.0012) [2023-12-26 23:17:20,593][105620] Updated weights for policy 1, policy_version 1087820 (0.0008) [2023-12-26 23:17:20,662][105620] Updated weights for policy 1, policy_version 1087830 (0.0006) [2023-12-26 23:17:20,727][105620] Updated weights for policy 1, policy_version 1087840 (0.0007) [2023-12-26 23:17:21,060][105692] Updated weights for policy 0, policy_version 1086690 (0.0010) [2023-12-26 23:17:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 556752896. Throughput: 0: 9435.1, 1: 10041.6. Samples: 556744456. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:17:21,062][104569] Avg episode reward: [(0, '9350.896'), (1, '9254.523')] [2023-12-26 23:17:21,124][105692] Updated weights for policy 0, policy_version 1086700 (0.0011) [2023-12-26 23:17:21,194][105692] Updated weights for policy 0, policy_version 1086710 (0.0008) [2023-12-26 23:17:21,257][105692] Updated weights for policy 0, policy_version 1086720 (0.0009) [2023-12-26 23:17:21,422][105620] Updated weights for policy 1, policy_version 1087850 (0.0009) [2023-12-26 23:17:21,486][105620] Updated weights for policy 1, policy_version 1087860 (0.0009) [2023-12-26 23:17:21,549][105620] Updated weights for policy 1, policy_version 1087870 (0.0008) [2023-12-26 23:17:21,996][105692] Updated weights for policy 0, policy_version 1086730 (0.0006) [2023-12-26 23:17:22,066][105692] Updated weights for policy 0, policy_version 1086740 (0.0006) [2023-12-26 23:17:22,122][105692] Updated weights for policy 0, policy_version 1086750 (0.0008) [2023-12-26 23:17:22,319][105620] Updated weights for policy 1, policy_version 1087880 (0.0008) [2023-12-26 23:17:22,378][105620] Updated weights for policy 1, policy_version 1087890 (0.0008) [2023-12-26 23:17:22,431][105620] Updated weights for policy 1, policy_version 1087900 (0.0010) [2023-12-26 23:17:22,766][105692] Updated weights for policy 0, policy_version 1086760 (0.0009) [2023-12-26 23:17:22,815][105692] Updated weights for policy 0, policy_version 1086770 (0.0007) [2023-12-26 23:17:22,868][105692] Updated weights for policy 0, policy_version 1086780 (0.0009) [2023-12-26 23:17:23,264][105620] Updated weights for policy 1, policy_version 1087910 (0.0009) [2023-12-26 23:17:23,313][105620] Updated weights for policy 1, policy_version 1087920 (0.0008) [2023-12-26 23:17:23,367][105620] Updated weights for policy 1, policy_version 1087930 (0.0008) [2023-12-26 23:17:23,587][105692] Updated weights for policy 0, policy_version 1086790 (0.0007) [2023-12-26 23:17:23,645][105692] Updated weights for policy 0, policy_version 1086800 (0.0006) [2023-12-26 23:17:23,707][105692] Updated weights for policy 0, policy_version 1086810 (0.0005) [2023-12-26 23:17:24,202][105620] Updated weights for policy 1, policy_version 1087940 (0.0008) [2023-12-26 23:17:24,257][105620] Updated weights for policy 1, policy_version 1087950 (0.0010) [2023-12-26 23:17:24,295][105692] Updated weights for policy 0, policy_version 1086820 (0.0006) [2023-12-26 23:17:24,307][105620] Updated weights for policy 1, policy_version 1087960 (0.0008) [2023-12-26 23:17:24,354][105692] Updated weights for policy 0, policy_version 1086830 (0.0005) [2023-12-26 23:17:24,415][105692] Updated weights for policy 0, policy_version 1086840 (0.0008) [2023-12-26 23:17:25,108][105692] Updated weights for policy 0, policy_version 1086850 (0.0009) [2023-12-26 23:17:25,129][105620] Updated weights for policy 1, policy_version 1087970 (0.0009) [2023-12-26 23:17:25,173][105692] Updated weights for policy 0, policy_version 1086860 (0.0008) [2023-12-26 23:17:25,178][105620] Updated weights for policy 1, policy_version 1087980 (0.0005) [2023-12-26 23:17:25,233][105620] Updated weights for policy 1, policy_version 1087990 (0.0005) [2023-12-26 23:17:25,237][105692] Updated weights for policy 0, policy_version 1086870 (0.0008) [2023-12-26 23:17:25,286][105620] Updated weights for policy 1, policy_version 1088000 (0.0005) [2023-12-26 23:17:25,299][105692] Updated weights for policy 0, policy_version 1086880 (0.0008) [2023-12-26 23:17:25,948][105620] Updated weights for policy 1, policy_version 1088010 (0.0006) [2023-12-26 23:17:25,954][105692] Updated weights for policy 0, policy_version 1086890 (0.0011) [2023-12-26 23:17:26,003][105620] Updated weights for policy 1, policy_version 1088020 (0.0009) [2023-12-26 23:17:26,011][105692] Updated weights for policy 0, policy_version 1086900 (0.0008) [2023-12-26 23:17:26,061][105620] Updated weights for policy 1, policy_version 1088030 (0.0007) [2023-12-26 23:17:26,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 556843008. Throughput: 0: 9505.9, 1: 10001.1. Samples: 556859644. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:17:26,062][104569] Avg episode reward: [(0, '9260.621'), (1, '9347.477')] [2023-12-26 23:17:26,066][105692] Updated weights for policy 0, policy_version 1086910 (0.0011) [2023-12-26 23:17:26,679][105692] Updated weights for policy 0, policy_version 1086920 (0.0010) [2023-12-26 23:17:26,724][105692] Updated weights for policy 0, policy_version 1086930 (0.0010) [2023-12-26 23:17:26,772][105692] Updated weights for policy 0, policy_version 1086940 (0.0010) [2023-12-26 23:17:26,896][105620] Updated weights for policy 1, policy_version 1088040 (0.0009) [2023-12-26 23:17:26,966][105620] Updated weights for policy 1, policy_version 1088050 (0.0010) [2023-12-26 23:17:27,036][105620] Updated weights for policy 1, policy_version 1088060 (0.0010) [2023-12-26 23:17:27,359][105692] Updated weights for policy 0, policy_version 1086950 (0.0005) [2023-12-26 23:17:27,404][105692] Updated weights for policy 0, policy_version 1086960 (0.0005) [2023-12-26 23:17:27,450][105692] Updated weights for policy 0, policy_version 1086970 (0.0005) [2023-12-26 23:17:27,923][105620] Updated weights for policy 1, policy_version 1088071 (0.0010) [2023-12-26 23:17:27,973][105620] Updated weights for policy 1, policy_version 1088081 (0.0008) [2023-12-26 23:17:27,975][105692] Updated weights for policy 0, policy_version 1086980 (0.0007) [2023-12-26 23:17:28,026][105620] Updated weights for policy 1, policy_version 1088091 (0.0005) [2023-12-26 23:17:28,031][105692] Updated weights for policy 0, policy_version 1086990 (0.0011) [2023-12-26 23:17:28,079][105692] Updated weights for policy 0, policy_version 1087000 (0.0008) [2023-12-26 23:17:28,725][105620] Updated weights for policy 1, policy_version 1088101 (0.0009) [2023-12-26 23:17:28,762][105692] Updated weights for policy 0, policy_version 1087010 (0.0009) [2023-12-26 23:17:28,776][105620] Updated weights for policy 1, policy_version 1088111 (0.0009) [2023-12-26 23:17:28,820][105692] Updated weights for policy 0, policy_version 1087020 (0.0005) [2023-12-26 23:17:28,825][105620] Updated weights for policy 1, policy_version 1088121 (0.0008) [2023-12-26 23:17:28,881][105692] Updated weights for policy 0, policy_version 1087030 (0.0005) [2023-12-26 23:17:28,938][105692] Updated weights for policy 0, policy_version 1087040 (0.0006) [2023-12-26 23:17:29,615][105620] Updated weights for policy 1, policy_version 1088131 (0.0007) [2023-12-26 23:17:29,636][105692] Updated weights for policy 0, policy_version 1087050 (0.0011) [2023-12-26 23:17:29,671][105620] Updated weights for policy 1, policy_version 1088141 (0.0007) [2023-12-26 23:17:29,689][105692] Updated weights for policy 0, policy_version 1087060 (0.0005) [2023-12-26 23:17:29,729][105620] Updated weights for policy 1, policy_version 1088151 (0.0006) [2023-12-26 23:17:29,751][105692] Updated weights for policy 0, policy_version 1087070 (0.0008) [2023-12-26 23:17:30,325][105620] Updated weights for policy 1, policy_version 1088161 (0.0006) [2023-12-26 23:17:30,379][105620] Updated weights for policy 1, policy_version 1088171 (0.0007) [2023-12-26 23:17:30,434][105620] Updated weights for policy 1, policy_version 1088181 (0.0006) [2023-12-26 23:17:30,467][105692] Updated weights for policy 0, policy_version 1087080 (0.0010) [2023-12-26 23:17:30,487][105620] Updated weights for policy 1, policy_version 1088191 (0.0005) [2023-12-26 23:17:30,518][105692] Updated weights for policy 0, policy_version 1087090 (0.0010) [2023-12-26 23:17:30,566][105692] Updated weights for policy 0, policy_version 1087100 (0.0010) [2023-12-26 23:17:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 556949504. Throughput: 0: 9642.8, 1: 9950.9. Samples: 556919104. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:17:31,062][104569] Avg episode reward: [(0, '9174.465'), (1, '9347.141')] [2023-12-26 23:17:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001087104_278339584.pth... [2023-12-26 23:17:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001085984_278052864.pth [2023-12-26 23:17:31,094][105620] Updated weights for policy 1, policy_version 1088201 (0.0008) [2023-12-26 23:17:31,160][105620] Updated weights for policy 1, policy_version 1088211 (0.0008) [2023-12-26 23:17:31,221][105620] Updated weights for policy 1, policy_version 1088221 (0.0005) [2023-12-26 23:17:31,241][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001088224_278618112.pth... [2023-12-26 23:17:31,246][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001087040_278315008.pth [2023-12-26 23:17:31,324][105692] Updated weights for policy 0, policy_version 1087110 (0.0007) [2023-12-26 23:17:31,391][105692] Updated weights for policy 0, policy_version 1087120 (0.0009) [2023-12-26 23:17:31,452][105692] Updated weights for policy 0, policy_version 1087130 (0.0008) [2023-12-26 23:17:31,929][105620] Updated weights for policy 1, policy_version 1088231 (0.0008) [2023-12-26 23:17:31,978][105620] Updated weights for policy 1, policy_version 1088241 (0.0008) [2023-12-26 23:17:32,027][105620] Updated weights for policy 1, policy_version 1088251 (0.0008) [2023-12-26 23:17:32,165][105692] Updated weights for policy 0, policy_version 1087140 (0.0010) [2023-12-26 23:17:32,221][105692] Updated weights for policy 0, policy_version 1087150 (0.0007) [2023-12-26 23:17:32,285][105692] Updated weights for policy 0, policy_version 1087160 (0.0007) [2023-12-26 23:17:32,794][105620] Updated weights for policy 1, policy_version 1088261 (0.0008) [2023-12-26 23:17:32,859][105620] Updated weights for policy 1, policy_version 1088271 (0.0009) [2023-12-26 23:17:32,914][105620] Updated weights for policy 1, policy_version 1088281 (0.0008) [2023-12-26 23:17:33,004][105692] Updated weights for policy 0, policy_version 1087170 (0.0010) [2023-12-26 23:17:33,061][105692] Updated weights for policy 0, policy_version 1087180 (0.0010) [2023-12-26 23:17:33,109][105692] Updated weights for policy 0, policy_version 1087190 (0.0010) [2023-12-26 23:17:33,167][105692] Updated weights for policy 0, policy_version 1087200 (0.0010) [2023-12-26 23:17:33,606][105620] Updated weights for policy 1, policy_version 1088291 (0.0007) [2023-12-26 23:17:33,666][105620] Updated weights for policy 1, policy_version 1088301 (0.0009) [2023-12-26 23:17:33,718][105620] Updated weights for policy 1, policy_version 1088311 (0.0009) [2023-12-26 23:17:33,879][105692] Updated weights for policy 0, policy_version 1087210 (0.0009) [2023-12-26 23:17:33,946][105692] Updated weights for policy 0, policy_version 1087220 (0.0010) [2023-12-26 23:17:34,000][105692] Updated weights for policy 0, policy_version 1087230 (0.0010) [2023-12-26 23:17:34,557][105620] Updated weights for policy 1, policy_version 1088321 (0.0009) [2023-12-26 23:17:34,615][105620] Updated weights for policy 1, policy_version 1088331 (0.0009) [2023-12-26 23:17:34,621][105692] Updated weights for policy 0, policy_version 1087240 (0.0006) [2023-12-26 23:17:34,677][105692] Updated weights for policy 0, policy_version 1087250 (0.0006) [2023-12-26 23:17:34,678][105620] Updated weights for policy 1, policy_version 1088341 (0.0008) [2023-12-26 23:17:34,742][105620] Updated weights for policy 1, policy_version 1088351 (0.0008) [2023-12-26 23:17:34,745][105692] Updated weights for policy 0, policy_version 1087260 (0.0006) [2023-12-26 23:17:35,394][105692] Updated weights for policy 0, policy_version 1087270 (0.0007) [2023-12-26 23:17:35,445][105620] Updated weights for policy 1, policy_version 1088361 (0.0008) [2023-12-26 23:17:35,460][105692] Updated weights for policy 0, policy_version 1087280 (0.0005) [2023-12-26 23:17:35,509][105620] Updated weights for policy 1, policy_version 1088371 (0.0005) [2023-12-26 23:17:35,532][105692] Updated weights for policy 0, policy_version 1087290 (0.0006) [2023-12-26 23:17:35,556][105620] Updated weights for policy 1, policy_version 1088381 (0.0005) [2023-12-26 23:17:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 557047808. Throughput: 0: 9581.2, 1: 9913.5. Samples: 557037428. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:17:36,063][104569] Avg episode reward: [(0, '9258.605'), (1, '9346.977')] [2023-12-26 23:17:36,189][105692] Updated weights for policy 0, policy_version 1087300 (0.0009) [2023-12-26 23:17:36,249][105692] Updated weights for policy 0, policy_version 1087310 (0.0009) [2023-12-26 23:17:36,292][105620] Updated weights for policy 1, policy_version 1088391 (0.0006) [2023-12-26 23:17:36,311][105692] Updated weights for policy 0, policy_version 1087320 (0.0008) [2023-12-26 23:17:36,354][105620] Updated weights for policy 1, policy_version 1088401 (0.0006) [2023-12-26 23:17:36,414][105620] Updated weights for policy 1, policy_version 1088411 (0.0006) [2023-12-26 23:17:37,004][105692] Updated weights for policy 0, policy_version 1087330 (0.0009) [2023-12-26 23:17:37,078][105692] Updated weights for policy 0, policy_version 1087340 (0.0008) [2023-12-26 23:17:37,145][105692] Updated weights for policy 0, policy_version 1087350 (0.0008) [2023-12-26 23:17:37,174][105620] Updated weights for policy 1, policy_version 1088421 (0.0006) [2023-12-26 23:17:37,204][105692] Updated weights for policy 0, policy_version 1087360 (0.0006) [2023-12-26 23:17:37,238][105620] Updated weights for policy 1, policy_version 1088431 (0.0006) [2023-12-26 23:17:37,304][105620] Updated weights for policy 1, policy_version 1088441 (0.0006) [2023-12-26 23:17:37,760][105692] Updated weights for policy 0, policy_version 1087370 (0.0008) [2023-12-26 23:17:37,824][105692] Updated weights for policy 0, policy_version 1087380 (0.0009) [2023-12-26 23:17:37,866][105620] Updated weights for policy 1, policy_version 1088451 (0.0005) [2023-12-26 23:17:37,880][105692] Updated weights for policy 0, policy_version 1087390 (0.0009) [2023-12-26 23:17:37,933][105620] Updated weights for policy 1, policy_version 1088461 (0.0006) [2023-12-26 23:17:37,994][105620] Updated weights for policy 1, policy_version 1088471 (0.0008) [2023-12-26 23:17:38,556][105620] Updated weights for policy 1, policy_version 1088481 (0.0006) [2023-12-26 23:17:38,570][105692] Updated weights for policy 0, policy_version 1087400 (0.0010) [2023-12-26 23:17:38,613][105620] Updated weights for policy 1, policy_version 1088491 (0.0010) [2023-12-26 23:17:38,628][105692] Updated weights for policy 0, policy_version 1087410 (0.0005) [2023-12-26 23:17:38,669][105620] Updated weights for policy 1, policy_version 1088501 (0.0010) [2023-12-26 23:17:38,686][105692] Updated weights for policy 0, policy_version 1087420 (0.0008) [2023-12-26 23:17:38,719][105620] Updated weights for policy 1, policy_version 1088511 (0.0010) [2023-12-26 23:17:39,375][105620] Updated weights for policy 1, policy_version 1088521 (0.0009) [2023-12-26 23:17:39,412][105692] Updated weights for policy 0, policy_version 1087430 (0.0010) [2023-12-26 23:17:39,443][105620] Updated weights for policy 1, policy_version 1088531 (0.0010) [2023-12-26 23:17:39,471][105692] Updated weights for policy 0, policy_version 1087440 (0.0011) [2023-12-26 23:17:39,498][105620] Updated weights for policy 1, policy_version 1088541 (0.0007) [2023-12-26 23:17:39,531][105692] Updated weights for policy 0, policy_version 1087450 (0.0010) [2023-12-26 23:17:40,272][105692] Updated weights for policy 0, policy_version 1087460 (0.0010) [2023-12-26 23:17:40,282][105620] Updated weights for policy 1, policy_version 1088551 (0.0009) [2023-12-26 23:17:40,331][105692] Updated weights for policy 0, policy_version 1087470 (0.0010) [2023-12-26 23:17:40,342][105620] Updated weights for policy 1, policy_version 1088561 (0.0009) [2023-12-26 23:17:40,387][105692] Updated weights for policy 0, policy_version 1087480 (0.0010) [2023-12-26 23:17:40,400][105620] Updated weights for policy 1, policy_version 1088571 (0.0009) [2023-12-26 23:17:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19438.7). Total num frames: 557146112. Throughput: 0: 9694.6, 1: 9954.6. Samples: 557158200. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:17:41,063][104569] Avg episode reward: [(0, '9353.166'), (1, '9259.075')] [2023-12-26 23:17:41,126][105620] Updated weights for policy 1, policy_version 1088581 (0.0008) [2023-12-26 23:17:41,150][105692] Updated weights for policy 0, policy_version 1087490 (0.0010) [2023-12-26 23:17:41,195][105620] Updated weights for policy 1, policy_version 1088591 (0.0007) [2023-12-26 23:17:41,214][105692] Updated weights for policy 0, policy_version 1087500 (0.0010) [2023-12-26 23:17:41,263][105620] Updated weights for policy 1, policy_version 1088601 (0.0006) [2023-12-26 23:17:41,281][105692] Updated weights for policy 0, policy_version 1087510 (0.0010) [2023-12-26 23:17:41,344][105692] Updated weights for policy 0, policy_version 1087520 (0.0010) [2023-12-26 23:17:41,960][105620] Updated weights for policy 1, policy_version 1088611 (0.0007) [2023-12-26 23:17:42,022][105620] Updated weights for policy 1, policy_version 1088621 (0.0006) [2023-12-26 23:17:42,086][105620] Updated weights for policy 1, policy_version 1088631 (0.0009) [2023-12-26 23:17:42,127][105692] Updated weights for policy 0, policy_version 1087530 (0.0010) [2023-12-26 23:17:42,177][105692] Updated weights for policy 0, policy_version 1087540 (0.0008) [2023-12-26 23:17:42,232][105692] Updated weights for policy 0, policy_version 1087550 (0.0009) [2023-12-26 23:17:42,767][105620] Updated weights for policy 1, policy_version 1088641 (0.0009) [2023-12-26 23:17:42,833][105620] Updated weights for policy 1, policy_version 1088651 (0.0008) [2023-12-26 23:17:42,896][105620] Updated weights for policy 1, policy_version 1088661 (0.0009) [2023-12-26 23:17:42,955][105620] Updated weights for policy 1, policy_version 1088671 (0.0009) [2023-12-26 23:17:43,048][105692] Updated weights for policy 0, policy_version 1087560 (0.0009) [2023-12-26 23:17:43,108][105692] Updated weights for policy 0, policy_version 1087570 (0.0009) [2023-12-26 23:17:43,179][105692] Updated weights for policy 0, policy_version 1087580 (0.0005) [2023-12-26 23:17:43,730][105692] Updated weights for policy 0, policy_version 1087590 (0.0008) [2023-12-26 23:17:43,769][105620] Updated weights for policy 1, policy_version 1088681 (0.0008) [2023-12-26 23:17:43,791][105692] Updated weights for policy 0, policy_version 1087600 (0.0006) [2023-12-26 23:17:43,828][105620] Updated weights for policy 1, policy_version 1088691 (0.0008) [2023-12-26 23:17:43,846][105692] Updated weights for policy 0, policy_version 1087610 (0.0005) [2023-12-26 23:17:43,880][105620] Updated weights for policy 1, policy_version 1088701 (0.0008) [2023-12-26 23:17:44,536][105692] Updated weights for policy 0, policy_version 1087620 (0.0007) [2023-12-26 23:17:44,581][105692] Updated weights for policy 0, policy_version 1087630 (0.0005) [2023-12-26 23:17:44,624][105692] Updated weights for policy 0, policy_version 1087640 (0.0005) [2023-12-26 23:17:44,661][105620] Updated weights for policy 1, policy_version 1088711 (0.0008) [2023-12-26 23:17:44,706][105620] Updated weights for policy 1, policy_version 1088721 (0.0008) [2023-12-26 23:17:44,753][105620] Updated weights for policy 1, policy_version 1088731 (0.0008) [2023-12-26 23:17:45,313][105692] Updated weights for policy 0, policy_version 1087650 (0.0006) [2023-12-26 23:17:45,372][105692] Updated weights for policy 0, policy_version 1087660 (0.0005) [2023-12-26 23:17:45,424][105692] Updated weights for policy 0, policy_version 1087670 (0.0006) [2023-12-26 23:17:45,486][105692] Updated weights for policy 0, policy_version 1087680 (0.0005) [2023-12-26 23:17:45,629][105620] Updated weights for policy 1, policy_version 1088741 (0.0009) [2023-12-26 23:17:45,680][105620] Updated weights for policy 1, policy_version 1088751 (0.0009) [2023-12-26 23:17:45,734][105620] Updated weights for policy 1, policy_version 1088761 (0.0009) [2023-12-26 23:17:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 557244416. Throughput: 0: 9724.2, 1: 9913.9. Samples: 557214372. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:17:46,062][105692] Updated weights for policy 0, policy_version 1087690 (0.0005) [2023-12-26 23:17:46,062][104569] Avg episode reward: [(0, '9354.217'), (1, '9260.131')] [2023-12-26 23:17:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001088768_278757376.pth... [2023-12-26 23:17:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001087648_278470656.pth [2023-12-26 23:17:46,126][105692] Updated weights for policy 0, policy_version 1087700 (0.0005) [2023-12-26 23:17:46,195][105692] Updated weights for policy 0, policy_version 1087710 (0.0005) [2023-12-26 23:17:46,208][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001087712_278495232.pth... [2023-12-26 23:17:46,214][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001086528_278192128.pth [2023-12-26 23:17:46,321][105620] Updated weights for policy 1, policy_version 1088771 (0.0010) [2023-12-26 23:17:46,384][105620] Updated weights for policy 1, policy_version 1088781 (0.0008) [2023-12-26 23:17:46,442][105620] Updated weights for policy 1, policy_version 1088791 (0.0010) [2023-12-26 23:17:46,678][105692] Updated weights for policy 0, policy_version 1087720 (0.0006) [2023-12-26 23:17:46,726][105692] Updated weights for policy 0, policy_version 1087730 (0.0005) [2023-12-26 23:17:46,785][105692] Updated weights for policy 0, policy_version 1087740 (0.0005) [2023-12-26 23:17:47,178][105620] Updated weights for policy 1, policy_version 1088801 (0.0010) [2023-12-26 23:17:47,227][105620] Updated weights for policy 1, policy_version 1088811 (0.0010) [2023-12-26 23:17:47,279][105620] Updated weights for policy 1, policy_version 1088821 (0.0010) [2023-12-26 23:17:47,327][105692] Updated weights for policy 0, policy_version 1087750 (0.0008) [2023-12-26 23:17:47,340][105620] Updated weights for policy 1, policy_version 1088831 (0.0010) [2023-12-26 23:17:47,379][105692] Updated weights for policy 0, policy_version 1087760 (0.0010) [2023-12-26 23:17:47,423][105692] Updated weights for policy 0, policy_version 1087770 (0.0010) [2023-12-26 23:17:48,023][105620] Updated weights for policy 1, policy_version 1088841 (0.0007) [2023-12-26 23:17:48,079][105620] Updated weights for policy 1, policy_version 1088851 (0.0008) [2023-12-26 23:17:48,128][105620] Updated weights for policy 1, policy_version 1088861 (0.0006) [2023-12-26 23:17:48,192][105692] Updated weights for policy 0, policy_version 1087780 (0.0010) [2023-12-26 23:17:48,253][105692] Updated weights for policy 0, policy_version 1087790 (0.0010) [2023-12-26 23:17:48,308][105692] Updated weights for policy 0, policy_version 1087800 (0.0008) [2023-12-26 23:17:48,804][105620] Updated weights for policy 1, policy_version 1088871 (0.0009) [2023-12-26 23:17:48,861][105620] Updated weights for policy 1, policy_version 1088881 (0.0010) [2023-12-26 23:17:48,914][105620] Updated weights for policy 1, policy_version 1088891 (0.0011) [2023-12-26 23:17:48,953][105692] Updated weights for policy 0, policy_version 1087810 (0.0008) [2023-12-26 23:17:49,004][105692] Updated weights for policy 0, policy_version 1087820 (0.0010) [2023-12-26 23:17:49,073][105692] Updated weights for policy 0, policy_version 1087830 (0.0010) [2023-12-26 23:17:49,134][105692] Updated weights for policy 0, policy_version 1087840 (0.0010) [2023-12-26 23:17:49,725][105620] Updated weights for policy 1, policy_version 1088901 (0.0010) [2023-12-26 23:17:49,788][105620] Updated weights for policy 1, policy_version 1088911 (0.0010) [2023-12-26 23:17:49,828][105692] Updated weights for policy 0, policy_version 1087850 (0.0011) [2023-12-26 23:17:49,855][105620] Updated weights for policy 1, policy_version 1088921 (0.0010) [2023-12-26 23:17:49,892][105692] Updated weights for policy 0, policy_version 1087860 (0.0011) [2023-12-26 23:17:49,959][105692] Updated weights for policy 0, policy_version 1087870 (0.0008) [2023-12-26 23:17:50,568][105620] Updated weights for policy 1, policy_version 1088931 (0.0008) [2023-12-26 23:17:50,624][105620] Updated weights for policy 1, policy_version 1088941 (0.0006) [2023-12-26 23:17:50,682][105620] Updated weights for policy 1, policy_version 1088951 (0.0006) [2023-12-26 23:17:50,693][105692] Updated weights for policy 0, policy_version 1087880 (0.0009) [2023-12-26 23:17:50,754][105692] Updated weights for policy 0, policy_version 1087890 (0.0007) [2023-12-26 23:17:50,824][105692] Updated weights for policy 0, policy_version 1087900 (0.0010) [2023-12-26 23:17:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 557350912. Throughput: 0: 9948.5, 1: 9814.9. Samples: 557337584. Policy #0 lag: (min: 19.0, avg: 20.6, max: 51.0) [2023-12-26 23:17:51,063][104569] Avg episode reward: [(0, '9352.504'), (1, '9258.283')] [2023-12-26 23:17:51,277][105620] Updated weights for policy 1, policy_version 1088961 (0.0006) [2023-12-26 23:17:51,328][105620] Updated weights for policy 1, policy_version 1088971 (0.0007) [2023-12-26 23:17:51,401][105620] Updated weights for policy 1, policy_version 1088981 (0.0010) [2023-12-26 23:17:51,464][105620] Updated weights for policy 1, policy_version 1088991 (0.0011) [2023-12-26 23:17:51,561][105692] Updated weights for policy 0, policy_version 1087910 (0.0009) [2023-12-26 23:17:51,622][105692] Updated weights for policy 0, policy_version 1087920 (0.0008) [2023-12-26 23:17:51,683][105692] Updated weights for policy 0, policy_version 1087930 (0.0008) [2023-12-26 23:17:52,233][105620] Updated weights for policy 1, policy_version 1089001 (0.0008) [2023-12-26 23:17:52,297][105620] Updated weights for policy 1, policy_version 1089011 (0.0010) [2023-12-26 23:17:52,377][105692] Updated weights for policy 0, policy_version 1087940 (0.0008) [2023-12-26 23:17:52,390][105620] Updated weights for policy 1, policy_version 1089021 (0.0009) [2023-12-26 23:17:52,437][105692] Updated weights for policy 0, policy_version 1087950 (0.0006) [2023-12-26 23:17:52,500][105692] Updated weights for policy 0, policy_version 1087960 (0.0006) [2023-12-26 23:17:53,146][105692] Updated weights for policy 0, policy_version 1087970 (0.0007) [2023-12-26 23:17:53,180][105620] Updated weights for policy 1, policy_version 1089031 (0.0010) [2023-12-26 23:17:53,201][105692] Updated weights for policy 0, policy_version 1087980 (0.0006) [2023-12-26 23:17:53,235][105620] Updated weights for policy 1, policy_version 1089041 (0.0010) [2023-12-26 23:17:53,254][105692] Updated weights for policy 0, policy_version 1087990 (0.0006) [2023-12-26 23:17:53,284][105620] Updated weights for policy 1, policy_version 1089051 (0.0010) [2023-12-26 23:17:53,306][105692] Updated weights for policy 0, policy_version 1088000 (0.0005) [2023-12-26 23:17:53,936][105620] Updated weights for policy 1, policy_version 1089061 (0.0008) [2023-12-26 23:17:53,983][105620] Updated weights for policy 1, policy_version 1089071 (0.0009) [2023-12-26 23:17:54,026][105620] Updated weights for policy 1, policy_version 1089081 (0.0007) [2023-12-26 23:17:54,034][105692] Updated weights for policy 0, policy_version 1088010 (0.0007) [2023-12-26 23:17:54,079][105692] Updated weights for policy 0, policy_version 1088020 (0.0006) [2023-12-26 23:17:54,127][105692] Updated weights for policy 0, policy_version 1088030 (0.0007) [2023-12-26 23:17:54,780][105620] Updated weights for policy 1, policy_version 1089091 (0.0010) [2023-12-26 23:17:54,840][105620] Updated weights for policy 1, policy_version 1089102 (0.0006) [2023-12-26 23:17:54,891][105620] Updated weights for policy 1, policy_version 1089112 (0.0006) [2023-12-26 23:17:54,921][105692] Updated weights for policy 0, policy_version 1088040 (0.0006) [2023-12-26 23:17:54,990][105692] Updated weights for policy 0, policy_version 1088050 (0.0006) [2023-12-26 23:17:55,053][105692] Updated weights for policy 0, policy_version 1088060 (0.0006) [2023-12-26 23:17:55,466][105620] Updated weights for policy 1, policy_version 1089122 (0.0007) [2023-12-26 23:17:55,538][105620] Updated weights for policy 1, policy_version 1089132 (0.0005) [2023-12-26 23:17:55,599][105620] Updated weights for policy 1, policy_version 1089142 (0.0008) [2023-12-26 23:17:55,651][105620] Updated weights for policy 1, policy_version 1089152 (0.0006) [2023-12-26 23:17:55,671][105692] Updated weights for policy 0, policy_version 1088070 (0.0005) [2023-12-26 23:17:55,727][105692] Updated weights for policy 0, policy_version 1088080 (0.0005) [2023-12-26 23:17:55,790][105692] Updated weights for policy 0, policy_version 1088090 (0.0005) [2023-12-26 23:17:56,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 557449216. Throughput: 0: 10039.6, 1: 9796.6. Samples: 557457000. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:17:56,062][104569] Avg episode reward: [(0, '9350.892'), (1, '9258.763')] [2023-12-26 23:17:56,171][105620] Updated weights for policy 1, policy_version 1089162 (0.0005) [2023-12-26 23:17:56,235][105620] Updated weights for policy 1, policy_version 1089172 (0.0005) [2023-12-26 23:17:56,292][105620] Updated weights for policy 1, policy_version 1089182 (0.0006) [2023-12-26 23:17:56,479][105692] Updated weights for policy 0, policy_version 1088100 (0.0006) [2023-12-26 23:17:56,535][105692] Updated weights for policy 0, policy_version 1088110 (0.0006) [2023-12-26 23:17:56,591][105692] Updated weights for policy 0, policy_version 1088120 (0.0011) [2023-12-26 23:17:57,067][105620] Updated weights for policy 1, policy_version 1089192 (0.0010) [2023-12-26 23:17:57,129][105620] Updated weights for policy 1, policy_version 1089202 (0.0008) [2023-12-26 23:17:57,131][105692] Updated weights for policy 0, policy_version 1088130 (0.0007) [2023-12-26 23:17:57,182][105620] Updated weights for policy 1, policy_version 1089212 (0.0005) [2023-12-26 23:17:57,189][105692] Updated weights for policy 0, policy_version 1088140 (0.0010) [2023-12-26 23:17:57,240][105692] Updated weights for policy 0, policy_version 1088150 (0.0010) [2023-12-26 23:17:57,288][105692] Updated weights for policy 0, policy_version 1088160 (0.0010) [2023-12-26 23:17:57,731][105620] Updated weights for policy 1, policy_version 1089222 (0.0006) [2023-12-26 23:17:57,788][105620] Updated weights for policy 1, policy_version 1089232 (0.0009) [2023-12-26 23:17:57,840][105620] Updated weights for policy 1, policy_version 1089242 (0.0010) [2023-12-26 23:17:57,891][105692] Updated weights for policy 0, policy_version 1088170 (0.0005) [2023-12-26 23:17:57,937][105692] Updated weights for policy 0, policy_version 1088180 (0.0005) [2023-12-26 23:17:57,980][105692] Updated weights for policy 0, policy_version 1088190 (0.0005) [2023-12-26 23:17:58,677][105620] Updated weights for policy 1, policy_version 1089252 (0.0009) [2023-12-26 23:17:58,691][105692] Updated weights for policy 0, policy_version 1088200 (0.0008) [2023-12-26 23:17:58,740][105620] Updated weights for policy 1, policy_version 1089262 (0.0007) [2023-12-26 23:17:58,766][105692] Updated weights for policy 0, policy_version 1088210 (0.0011) [2023-12-26 23:17:58,809][105620] Updated weights for policy 1, policy_version 1089272 (0.0007) [2023-12-26 23:17:58,838][105692] Updated weights for policy 0, policy_version 1088220 (0.0014) [2023-12-26 23:17:59,618][105692] Updated weights for policy 0, policy_version 1088230 (0.0008) [2023-12-26 23:17:59,629][105620] Updated weights for policy 1, policy_version 1089282 (0.0008) [2023-12-26 23:17:59,668][105692] Updated weights for policy 0, policy_version 1088240 (0.0007) [2023-12-26 23:17:59,692][105620] Updated weights for policy 1, policy_version 1089292 (0.0008) [2023-12-26 23:17:59,724][105692] Updated weights for policy 0, policy_version 1088250 (0.0009) [2023-12-26 23:17:59,757][105620] Updated weights for policy 1, policy_version 1089302 (0.0008) [2023-12-26 23:17:59,820][105620] Updated weights for policy 1, policy_version 1089312 (0.0008) [2023-12-26 23:18:00,405][105692] Updated weights for policy 0, policy_version 1088260 (0.0007) [2023-12-26 23:18:00,452][105692] Updated weights for policy 0, policy_version 1088270 (0.0008) [2023-12-26 23:18:00,502][105692] Updated weights for policy 0, policy_version 1088280 (0.0009) [2023-12-26 23:18:00,604][105620] Updated weights for policy 1, policy_version 1089322 (0.0006) [2023-12-26 23:18:00,659][105620] Updated weights for policy 1, policy_version 1089332 (0.0005) [2023-12-26 23:18:00,717][105620] Updated weights for policy 1, policy_version 1089342 (0.0006) [2023-12-26 23:18:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 557547520. Throughput: 0: 10129.0, 1: 9759.9. Samples: 557519084. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:18:01,062][104569] Avg episode reward: [(0, '9355.315'), (1, '9350.286')] [2023-12-26 23:18:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001088288_278642688.pth... [2023-12-26 23:18:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001089344_278904832.pth... [2023-12-26 23:18:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001087104_278339584.pth [2023-12-26 23:18:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001088224_278618112.pth [2023-12-26 23:18:01,209][105692] Updated weights for policy 0, policy_version 1088290 (0.0009) [2023-12-26 23:18:01,269][105692] Updated weights for policy 0, policy_version 1088300 (0.0011) [2023-12-26 23:18:01,322][105692] Updated weights for policy 0, policy_version 1088310 (0.0007) [2023-12-26 23:18:01,373][105620] Updated weights for policy 1, policy_version 1089352 (0.0009) [2023-12-26 23:18:01,385][105692] Updated weights for policy 0, policy_version 1088320 (0.0010) [2023-12-26 23:18:01,427][105620] Updated weights for policy 1, policy_version 1089362 (0.0008) [2023-12-26 23:18:01,475][105620] Updated weights for policy 1, policy_version 1089372 (0.0008) [2023-12-26 23:18:02,110][105692] Updated weights for policy 0, policy_version 1088330 (0.0009) [2023-12-26 23:18:02,166][105692] Updated weights for policy 0, policy_version 1088340 (0.0007) [2023-12-26 23:18:02,217][105692] Updated weights for policy 0, policy_version 1088350 (0.0005) [2023-12-26 23:18:02,283][105620] Updated weights for policy 1, policy_version 1089382 (0.0009) [2023-12-26 23:18:02,340][105620] Updated weights for policy 1, policy_version 1089393 (0.0009) [2023-12-26 23:18:02,407][105620] Updated weights for policy 1, policy_version 1089403 (0.0008) [2023-12-26 23:18:02,838][105692] Updated weights for policy 0, policy_version 1088360 (0.0007) [2023-12-26 23:18:02,888][105692] Updated weights for policy 0, policy_version 1088370 (0.0008) [2023-12-26 23:18:02,940][105692] Updated weights for policy 0, policy_version 1088381 (0.0010) [2023-12-26 23:18:03,065][105620] Updated weights for policy 1, policy_version 1089413 (0.0006) [2023-12-26 23:18:03,123][105620] Updated weights for policy 1, policy_version 1089423 (0.0005) [2023-12-26 23:18:03,175][105620] Updated weights for policy 1, policy_version 1089433 (0.0005) [2023-12-26 23:18:03,727][105692] Updated weights for policy 0, policy_version 1088392 (0.0009) [2023-12-26 23:18:03,740][105620] Updated weights for policy 1, policy_version 1089443 (0.0006) [2023-12-26 23:18:03,785][105620] Updated weights for policy 1, policy_version 1089453 (0.0006) [2023-12-26 23:18:03,791][105585] KL-divergence is very high: 136.9193 [2023-12-26 23:18:03,792][105692] Updated weights for policy 0, policy_version 1088402 (0.0007) [2023-12-26 23:18:03,843][105585] KL-divergence is very high: 246.8716 [2023-12-26 23:18:03,843][105620] Updated weights for policy 1, policy_version 1089463 (0.0008) [2023-12-26 23:18:03,857][105692] Updated weights for policy 0, policy_version 1088412 (0.0008) [2023-12-26 23:18:04,606][105620] Updated weights for policy 1, policy_version 1089473 (0.0008) [2023-12-26 23:18:04,618][105692] Updated weights for policy 0, policy_version 1088422 (0.0009) [2023-12-26 23:18:04,656][105620] Updated weights for policy 1, policy_version 1089483 (0.0005) [2023-12-26 23:18:04,667][105692] Updated weights for policy 0, policy_version 1088432 (0.0009) [2023-12-26 23:18:04,714][105620] Updated weights for policy 1, policy_version 1089493 (0.0005) [2023-12-26 23:18:04,716][105692] Updated weights for policy 0, policy_version 1088442 (0.0009) [2023-12-26 23:18:04,767][105620] Updated weights for policy 1, policy_version 1089503 (0.0007) [2023-12-26 23:18:05,379][105620] Updated weights for policy 1, policy_version 1089513 (0.0006) [2023-12-26 23:18:05,427][105620] Updated weights for policy 1, policy_version 1089523 (0.0007) [2023-12-26 23:18:05,478][105620] Updated weights for policy 1, policy_version 1089533 (0.0010) [2023-12-26 23:18:05,502][105692] Updated weights for policy 0, policy_version 1088452 (0.0007) [2023-12-26 23:18:05,564][105692] Updated weights for policy 0, policy_version 1088462 (0.0009) [2023-12-26 23:18:05,622][105692] Updated weights for policy 0, policy_version 1088472 (0.0009) [2023-12-26 23:18:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 557645824. Throughput: 0: 10129.9, 1: 9673.4. Samples: 557635604. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:18:06,062][104569] Avg episode reward: [(0, '9357.563'), (1, '9258.350')] [2023-12-26 23:18:06,256][105620] Updated weights for policy 1, policy_version 1089543 (0.0009) [2023-12-26 23:18:06,292][105692] Updated weights for policy 0, policy_version 1088482 (0.0008) [2023-12-26 23:18:06,322][105620] Updated weights for policy 1, policy_version 1089553 (0.0009) [2023-12-26 23:18:06,353][105692] Updated weights for policy 0, policy_version 1088492 (0.0008) [2023-12-26 23:18:06,380][105620] Updated weights for policy 1, policy_version 1089563 (0.0008) [2023-12-26 23:18:06,402][105692] Updated weights for policy 0, policy_version 1088502 (0.0006) [2023-12-26 23:18:06,449][105692] Updated weights for policy 0, policy_version 1088512 (0.0005) [2023-12-26 23:18:07,037][105620] Updated weights for policy 1, policy_version 1089573 (0.0007) [2023-12-26 23:18:07,092][105620] Updated weights for policy 1, policy_version 1089583 (0.0006) [2023-12-26 23:18:07,143][105620] Updated weights for policy 1, policy_version 1089593 (0.0005) [2023-12-26 23:18:07,253][105692] Updated weights for policy 0, policy_version 1088522 (0.0010) [2023-12-26 23:18:07,316][105692] Updated weights for policy 0, policy_version 1088532 (0.0010) [2023-12-26 23:18:07,374][105692] Updated weights for policy 0, policy_version 1088542 (0.0010) [2023-12-26 23:18:07,709][105620] Updated weights for policy 1, policy_version 1089603 (0.0008) [2023-12-26 23:18:07,763][105620] Updated weights for policy 1, policy_version 1089614 (0.0010) [2023-12-26 23:18:07,810][105620] Updated weights for policy 1, policy_version 1089624 (0.0008) [2023-12-26 23:18:08,113][105692] Updated weights for policy 0, policy_version 1088552 (0.0009) [2023-12-26 23:18:08,164][105692] Updated weights for policy 0, policy_version 1088562 (0.0009) [2023-12-26 23:18:08,214][105692] Updated weights for policy 0, policy_version 1088572 (0.0008) [2023-12-26 23:18:08,666][105620] Updated weights for policy 1, policy_version 1089634 (0.0009) [2023-12-26 23:18:08,724][105620] Updated weights for policy 1, policy_version 1089644 (0.0009) [2023-12-26 23:18:08,783][105620] Updated weights for policy 1, policy_version 1089654 (0.0009) [2023-12-26 23:18:08,846][105620] Updated weights for policy 1, policy_version 1089664 (0.0009) [2023-12-26 23:18:08,914][105692] Updated weights for policy 0, policy_version 1088582 (0.0008) [2023-12-26 23:18:08,973][105692] Updated weights for policy 0, policy_version 1088592 (0.0008) [2023-12-26 23:18:09,026][105692] Updated weights for policy 0, policy_version 1088602 (0.0005) [2023-12-26 23:18:09,588][105620] Updated weights for policy 1, policy_version 1089674 (0.0008) [2023-12-26 23:18:09,652][105620] Updated weights for policy 1, policy_version 1089684 (0.0008) [2023-12-26 23:18:09,717][105620] Updated weights for policy 1, policy_version 1089694 (0.0006) [2023-12-26 23:18:09,782][105692] Updated weights for policy 0, policy_version 1088612 (0.0007) [2023-12-26 23:18:09,849][105692] Updated weights for policy 0, policy_version 1088622 (0.0011) [2023-12-26 23:18:09,909][105692] Updated weights for policy 0, policy_version 1088632 (0.0011) [2023-12-26 23:18:10,374][105620] Updated weights for policy 1, policy_version 1089704 (0.0008) [2023-12-26 23:18:10,425][105620] Updated weights for policy 1, policy_version 1089714 (0.0009) [2023-12-26 23:18:10,486][105620] Updated weights for policy 1, policy_version 1089724 (0.0009) [2023-12-26 23:18:10,604][105692] Updated weights for policy 0, policy_version 1088642 (0.0010) [2023-12-26 23:18:10,661][105692] Updated weights for policy 0, policy_version 1088652 (0.0005) [2023-12-26 23:18:10,715][105692] Updated weights for policy 0, policy_version 1088662 (0.0006) [2023-12-26 23:18:10,765][105692] Updated weights for policy 0, policy_version 1088672 (0.0005) [2023-12-26 23:18:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19466.4). Total num frames: 557744128. Throughput: 0: 10073.8, 1: 9785.7. Samples: 557753320. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:18:11,062][104569] Avg episode reward: [(0, '9357.099'), (1, '9165.901')] [2023-12-26 23:18:11,166][105620] Updated weights for policy 1, policy_version 1089734 (0.0010) [2023-12-26 23:18:11,220][105620] Updated weights for policy 1, policy_version 1089744 (0.0010) [2023-12-26 23:18:11,286][105620] Updated weights for policy 1, policy_version 1089754 (0.0009) [2023-12-26 23:18:11,420][105692] Updated weights for policy 0, policy_version 1088682 (0.0008) [2023-12-26 23:18:11,478][105692] Updated weights for policy 0, policy_version 1088692 (0.0008) [2023-12-26 23:18:11,535][105692] Updated weights for policy 0, policy_version 1088702 (0.0009) [2023-12-26 23:18:12,063][105620] Updated weights for policy 1, policy_version 1089764 (0.0009) [2023-12-26 23:18:12,114][105620] Updated weights for policy 1, policy_version 1089774 (0.0008) [2023-12-26 23:18:12,168][105620] Updated weights for policy 1, policy_version 1089784 (0.0009) [2023-12-26 23:18:12,301][105692] Updated weights for policy 0, policy_version 1088712 (0.0010) [2023-12-26 23:18:12,363][105692] Updated weights for policy 0, policy_version 1088722 (0.0010) [2023-12-26 23:18:12,429][105692] Updated weights for policy 0, policy_version 1088732 (0.0010) [2023-12-26 23:18:12,959][105620] Updated weights for policy 1, policy_version 1089794 (0.0008) [2023-12-26 23:18:13,018][105620] Updated weights for policy 1, policy_version 1089804 (0.0005) [2023-12-26 23:18:13,078][105620] Updated weights for policy 1, policy_version 1089814 (0.0005) [2023-12-26 23:18:13,124][105620] Updated weights for policy 1, policy_version 1089824 (0.0005) [2023-12-26 23:18:13,186][105692] Updated weights for policy 0, policy_version 1088742 (0.0010) [2023-12-26 23:18:13,244][105692] Updated weights for policy 0, policy_version 1088753 (0.0010) [2023-12-26 23:18:13,299][105692] Updated weights for policy 0, policy_version 1088763 (0.0010) [2023-12-26 23:18:13,694][105620] Updated weights for policy 1, policy_version 1089834 (0.0009) [2023-12-26 23:18:13,752][105620] Updated weights for policy 1, policy_version 1089844 (0.0009) [2023-12-26 23:18:13,808][105620] Updated weights for policy 1, policy_version 1089854 (0.0009) [2023-12-26 23:18:14,022][105692] Updated weights for policy 0, policy_version 1088773 (0.0009) [2023-12-26 23:18:14,075][105692] Updated weights for policy 0, policy_version 1088783 (0.0006) [2023-12-26 23:18:14,137][105692] Updated weights for policy 0, policy_version 1088793 (0.0007) [2023-12-26 23:18:14,565][105620] Updated weights for policy 1, policy_version 1089864 (0.0009) [2023-12-26 23:18:14,620][105620] Updated weights for policy 1, policy_version 1089874 (0.0008) [2023-12-26 23:18:14,682][105620] Updated weights for policy 1, policy_version 1089884 (0.0009) [2023-12-26 23:18:14,841][105692] Updated weights for policy 0, policy_version 1088803 (0.0009) [2023-12-26 23:18:14,897][105692] Updated weights for policy 0, policy_version 1088813 (0.0009) [2023-12-26 23:18:14,944][105692] Updated weights for policy 0, policy_version 1088823 (0.0009) [2023-12-26 23:18:15,467][105620] Updated weights for policy 1, policy_version 1089894 (0.0009) [2023-12-26 23:18:15,526][105620] Updated weights for policy 1, policy_version 1089904 (0.0009) [2023-12-26 23:18:15,580][105620] Updated weights for policy 1, policy_version 1089914 (0.0009) [2023-12-26 23:18:15,730][105692] Updated weights for policy 0, policy_version 1088833 (0.0009) [2023-12-26 23:18:15,784][105692] Updated weights for policy 0, policy_version 1088843 (0.0008) [2023-12-26 23:18:15,837][105692] Updated weights for policy 0, policy_version 1088853 (0.0008) [2023-12-26 23:18:15,902][105692] Updated weights for policy 0, policy_version 1088863 (0.0005) [2023-12-26 23:18:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.4, 300 sec: 19494.2). Total num frames: 557842432. Throughput: 0: 9964.9, 1: 9854.9. Samples: 557810996. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:18:16,062][104569] Avg episode reward: [(0, '9355.747'), (1, '9257.453')] [2023-12-26 23:18:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001088864_278790144.pth... [2023-12-26 23:18:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001089920_279052288.pth... [2023-12-26 23:18:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001088768_278757376.pth [2023-12-26 23:18:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001087712_278495232.pth [2023-12-26 23:18:16,418][105620] Updated weights for policy 1, policy_version 1089924 (0.0009) [2023-12-26 23:18:16,489][105620] Updated weights for policy 1, policy_version 1089934 (0.0009) [2023-12-26 23:18:16,493][105692] Updated weights for policy 0, policy_version 1088873 (0.0005) [2023-12-26 23:18:16,546][105620] Updated weights for policy 1, policy_version 1089944 (0.0007) [2023-12-26 23:18:16,550][105692] Updated weights for policy 0, policy_version 1088883 (0.0005) [2023-12-26 23:18:16,604][105692] Updated weights for policy 0, policy_version 1088893 (0.0006) [2023-12-26 23:18:17,204][105620] Updated weights for policy 1, policy_version 1089954 (0.0007) [2023-12-26 23:18:17,255][105620] Updated weights for policy 1, policy_version 1089964 (0.0005) [2023-12-26 23:18:17,305][105620] Updated weights for policy 1, policy_version 1089974 (0.0005) [2023-12-26 23:18:17,360][105692] Updated weights for policy 0, policy_version 1088903 (0.0007) [2023-12-26 23:18:17,368][105620] Updated weights for policy 1, policy_version 1089984 (0.0007) [2023-12-26 23:18:17,411][105692] Updated weights for policy 0, policy_version 1088913 (0.0005) [2023-12-26 23:18:17,469][105692] Updated weights for policy 0, policy_version 1088923 (0.0005) [2023-12-26 23:18:17,966][105620] Updated weights for policy 1, policy_version 1089994 (0.0005) [2023-12-26 23:18:18,020][105620] Updated weights for policy 1, policy_version 1090004 (0.0005) [2023-12-26 23:18:18,080][105620] Updated weights for policy 1, policy_version 1090014 (0.0010) [2023-12-26 23:18:18,208][105692] Updated weights for policy 0, policy_version 1088933 (0.0008) [2023-12-26 23:18:18,261][105692] Updated weights for policy 0, policy_version 1088943 (0.0009) [2023-12-26 23:18:18,314][105692] Updated weights for policy 0, policy_version 1088953 (0.0008) [2023-12-26 23:18:18,723][105620] Updated weights for policy 1, policy_version 1090024 (0.0010) [2023-12-26 23:18:18,777][105620] Updated weights for policy 1, policy_version 1090034 (0.0009) [2023-12-26 23:18:18,835][105620] Updated weights for policy 1, policy_version 1090044 (0.0008) [2023-12-26 23:18:19,107][105692] Updated weights for policy 0, policy_version 1088963 (0.0009) [2023-12-26 23:18:19,167][105692] Updated weights for policy 0, policy_version 1088973 (0.0009) [2023-12-26 23:18:19,226][105692] Updated weights for policy 0, policy_version 1088983 (0.0009) [2023-12-26 23:18:19,561][105620] Updated weights for policy 1, policy_version 1090054 (0.0010) [2023-12-26 23:18:19,626][105620] Updated weights for policy 1, policy_version 1090064 (0.0008) [2023-12-26 23:18:19,703][105620] Updated weights for policy 1, policy_version 1090074 (0.0011) [2023-12-26 23:18:20,113][105692] Updated weights for policy 0, policy_version 1088993 (0.0009) [2023-12-26 23:18:20,173][105692] Updated weights for policy 0, policy_version 1089003 (0.0008) [2023-12-26 23:18:20,233][105692] Updated weights for policy 0, policy_version 1089013 (0.0009) [2023-12-26 23:18:20,288][105692] Updated weights for policy 0, policy_version 1089023 (0.0010) [2023-12-26 23:18:20,345][105620] Updated weights for policy 1, policy_version 1090084 (0.0006) [2023-12-26 23:18:20,406][105620] Updated weights for policy 1, policy_version 1090094 (0.0005) [2023-12-26 23:18:20,457][105620] Updated weights for policy 1, policy_version 1090104 (0.0005) [2023-12-26 23:18:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 557932544. Throughput: 0: 9914.9, 1: 9854.1. Samples: 557927032. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:18:21,062][104569] Avg episode reward: [(0, '9352.286'), (1, '9257.183')] [2023-12-26 23:18:21,105][105692] Updated weights for policy 0, policy_version 1089033 (0.0010) [2023-12-26 23:18:21,130][105620] Updated weights for policy 1, policy_version 1090114 (0.0007) [2023-12-26 23:18:21,170][105692] Updated weights for policy 0, policy_version 1089043 (0.0010) [2023-12-26 23:18:21,191][105620] Updated weights for policy 1, policy_version 1090124 (0.0010) [2023-12-26 23:18:21,227][105692] Updated weights for policy 0, policy_version 1089053 (0.0011) [2023-12-26 23:18:21,254][105620] Updated weights for policy 1, policy_version 1090134 (0.0010) [2023-12-26 23:18:21,320][105620] Updated weights for policy 1, policy_version 1090144 (0.0011) [2023-12-26 23:18:22,028][105692] Updated weights for policy 0, policy_version 1089063 (0.0010) [2023-12-26 23:18:22,043][105620] Updated weights for policy 1, policy_version 1090154 (0.0006) [2023-12-26 23:18:22,085][105692] Updated weights for policy 0, policy_version 1089073 (0.0011) [2023-12-26 23:18:22,104][105620] Updated weights for policy 1, policy_version 1090164 (0.0005) [2023-12-26 23:18:22,143][105692] Updated weights for policy 0, policy_version 1089083 (0.0009) [2023-12-26 23:18:22,166][105620] Updated weights for policy 1, policy_version 1090174 (0.0007) [2023-12-26 23:18:22,912][105692] Updated weights for policy 0, policy_version 1089093 (0.0008) [2023-12-26 23:18:22,925][105620] Updated weights for policy 1, policy_version 1090184 (0.0008) [2023-12-26 23:18:22,972][105692] Updated weights for policy 0, policy_version 1089103 (0.0006) [2023-12-26 23:18:22,993][105620] Updated weights for policy 1, policy_version 1090194 (0.0008) [2023-12-26 23:18:23,025][105692] Updated weights for policy 0, policy_version 1089113 (0.0005) [2023-12-26 23:18:23,055][105620] Updated weights for policy 1, policy_version 1090204 (0.0007) [2023-12-26 23:18:23,646][105692] Updated weights for policy 0, policy_version 1089123 (0.0009) [2023-12-26 23:18:23,706][105692] Updated weights for policy 0, policy_version 1089133 (0.0009) [2023-12-26 23:18:23,762][105692] Updated weights for policy 0, policy_version 1089143 (0.0010) [2023-12-26 23:18:23,821][105620] Updated weights for policy 1, policy_version 1090214 (0.0007) [2023-12-26 23:18:23,875][105620] Updated weights for policy 1, policy_version 1090224 (0.0009) [2023-12-26 23:18:23,935][105620] Updated weights for policy 1, policy_version 1090234 (0.0008) [2023-12-26 23:18:24,543][105692] Updated weights for policy 0, policy_version 1089153 (0.0009) [2023-12-26 23:18:24,591][105692] Updated weights for policy 0, policy_version 1089163 (0.0008) [2023-12-26 23:18:24,594][105620] Updated weights for policy 1, policy_version 1090244 (0.0008) [2023-12-26 23:18:24,645][105692] Updated weights for policy 0, policy_version 1089173 (0.0006) [2023-12-26 23:18:24,655][105620] Updated weights for policy 1, policy_version 1090254 (0.0009) [2023-12-26 23:18:24,698][105692] Updated weights for policy 0, policy_version 1089183 (0.0006) [2023-12-26 23:18:24,708][105620] Updated weights for policy 1, policy_version 1090264 (0.0007) [2023-12-26 23:18:25,287][105620] Updated weights for policy 1, policy_version 1090274 (0.0006) [2023-12-26 23:18:25,353][105620] Updated weights for policy 1, policy_version 1090284 (0.0010) [2023-12-26 23:18:25,407][105620] Updated weights for policy 1, policy_version 1090294 (0.0009) [2023-12-26 23:18:25,436][105692] Updated weights for policy 0, policy_version 1089193 (0.0005) [2023-12-26 23:18:25,473][105620] Updated weights for policy 1, policy_version 1090304 (0.0007) [2023-12-26 23:18:25,485][105692] Updated weights for policy 0, policy_version 1089203 (0.0005) [2023-12-26 23:18:25,548][105692] Updated weights for policy 0, policy_version 1089213 (0.0008) [2023-12-26 23:18:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19797.3, 300 sec: 19410.9). Total num frames: 558030848. Throughput: 0: 9811.4, 1: 9832.3. Samples: 558042172. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:18:26,063][104569] Avg episode reward: [(0, '9350.005'), (1, '9165.102')] [2023-12-26 23:18:26,148][105692] Updated weights for policy 0, policy_version 1089223 (0.0010) [2023-12-26 23:18:26,208][105692] Updated weights for policy 0, policy_version 1089233 (0.0011) [2023-12-26 23:18:26,246][105620] Updated weights for policy 1, policy_version 1090314 (0.0006) [2023-12-26 23:18:26,268][105692] Updated weights for policy 0, policy_version 1089243 (0.0008) [2023-12-26 23:18:26,309][105620] Updated weights for policy 1, policy_version 1090324 (0.0008) [2023-12-26 23:18:26,369][105620] Updated weights for policy 1, policy_version 1090334 (0.0008) [2023-12-26 23:18:27,001][105692] Updated weights for policy 0, policy_version 1089253 (0.0009) [2023-12-26 23:18:27,054][105692] Updated weights for policy 0, policy_version 1089263 (0.0010) [2023-12-26 23:18:27,112][105692] Updated weights for policy 0, policy_version 1089273 (0.0010) [2023-12-26 23:18:27,114][105620] Updated weights for policy 1, policy_version 1090344 (0.0006) [2023-12-26 23:18:27,169][105620] Updated weights for policy 1, policy_version 1090354 (0.0007) [2023-12-26 23:18:27,220][105620] Updated weights for policy 1, policy_version 1090364 (0.0008) [2023-12-26 23:18:27,850][105692] Updated weights for policy 0, policy_version 1089283 (0.0010) [2023-12-26 23:18:27,897][105692] Updated weights for policy 0, policy_version 1089293 (0.0010) [2023-12-26 23:18:27,944][105692] Updated weights for policy 0, policy_version 1089303 (0.0010) [2023-12-26 23:18:27,981][105620] Updated weights for policy 1, policy_version 1090374 (0.0007) [2023-12-26 23:18:28,031][105620] Updated weights for policy 1, policy_version 1090384 (0.0007) [2023-12-26 23:18:28,078][105620] Updated weights for policy 1, policy_version 1090394 (0.0008) [2023-12-26 23:18:28,711][105692] Updated weights for policy 0, policy_version 1089313 (0.0010) [2023-12-26 23:18:28,763][105692] Updated weights for policy 0, policy_version 1089323 (0.0010) [2023-12-26 23:18:28,827][105692] Updated weights for policy 0, policy_version 1089333 (0.0010) [2023-12-26 23:18:28,845][105620] Updated weights for policy 1, policy_version 1090404 (0.0007) [2023-12-26 23:18:28,889][105692] Updated weights for policy 0, policy_version 1089343 (0.0010) [2023-12-26 23:18:28,903][105620] Updated weights for policy 1, policy_version 1090414 (0.0006) [2023-12-26 23:18:28,959][105620] Updated weights for policy 1, policy_version 1090424 (0.0008) [2023-12-26 23:18:29,622][105692] Updated weights for policy 0, policy_version 1089353 (0.0010) [2023-12-26 23:18:29,680][105692] Updated weights for policy 0, policy_version 1089363 (0.0010) [2023-12-26 23:18:29,729][105620] Updated weights for policy 1, policy_version 1090434 (0.0008) [2023-12-26 23:18:29,743][105692] Updated weights for policy 0, policy_version 1089373 (0.0011) [2023-12-26 23:18:29,781][105620] Updated weights for policy 1, policy_version 1090444 (0.0006) [2023-12-26 23:18:29,840][105620] Updated weights for policy 1, policy_version 1090454 (0.0008) [2023-12-26 23:18:29,903][105620] Updated weights for policy 1, policy_version 1090464 (0.0008) [2023-12-26 23:18:30,500][105692] Updated weights for policy 0, policy_version 1089383 (0.0010) [2023-12-26 23:18:30,544][105692] Updated weights for policy 0, policy_version 1089393 (0.0010) [2023-12-26 23:18:30,594][105692] Updated weights for policy 0, policy_version 1089403 (0.0010) [2023-12-26 23:18:30,671][105620] Updated weights for policy 1, policy_version 1090474 (0.0008) [2023-12-26 23:18:30,721][105620] Updated weights for policy 1, policy_version 1090484 (0.0007) [2023-12-26 23:18:30,769][105620] Updated weights for policy 1, policy_version 1090494 (0.0008) [2023-12-26 23:18:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 558129152. Throughput: 0: 9840.6, 1: 9825.2. Samples: 558099332. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:18:31,062][104569] Avg episode reward: [(0, '9350.503'), (1, '9257.092')] [2023-12-26 23:18:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001089408_278929408.pth... [2023-12-26 23:18:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001090496_279199744.pth... [2023-12-26 23:18:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001089344_278904832.pth [2023-12-26 23:18:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001088288_278642688.pth [2023-12-26 23:18:31,364][105692] Updated weights for policy 0, policy_version 1089413 (0.0010) [2023-12-26 23:18:31,428][105692] Updated weights for policy 0, policy_version 1089423 (0.0011) [2023-12-26 23:18:31,486][105692] Updated weights for policy 0, policy_version 1089433 (0.0011) [2023-12-26 23:18:31,565][105620] Updated weights for policy 1, policy_version 1090504 (0.0008) [2023-12-26 23:18:31,622][105620] Updated weights for policy 1, policy_version 1090514 (0.0009) [2023-12-26 23:18:31,680][105620] Updated weights for policy 1, policy_version 1090524 (0.0008) [2023-12-26 23:18:32,226][105692] Updated weights for policy 0, policy_version 1089443 (0.0011) [2023-12-26 23:18:32,288][105692] Updated weights for policy 0, policy_version 1089453 (0.0011) [2023-12-26 23:18:32,336][105692] Updated weights for policy 0, policy_version 1089463 (0.0010) [2023-12-26 23:18:32,460][105620] Updated weights for policy 1, policy_version 1090534 (0.0010) [2023-12-26 23:18:32,521][105620] Updated weights for policy 1, policy_version 1090544 (0.0010) [2023-12-26 23:18:32,579][105620] Updated weights for policy 1, policy_version 1090554 (0.0010) [2023-12-26 23:18:33,035][105692] Updated weights for policy 0, policy_version 1089473 (0.0010) [2023-12-26 23:18:33,092][105692] Updated weights for policy 0, policy_version 1089483 (0.0010) [2023-12-26 23:18:33,139][105692] Updated weights for policy 0, policy_version 1089493 (0.0010) [2023-12-26 23:18:33,186][105692] Updated weights for policy 0, policy_version 1089503 (0.0010) [2023-12-26 23:18:33,199][105620] Updated weights for policy 1, policy_version 1090564 (0.0008) [2023-12-26 23:18:33,246][105620] Updated weights for policy 1, policy_version 1090574 (0.0010) [2023-12-26 23:18:33,290][105620] Updated weights for policy 1, policy_version 1090584 (0.0010) [2023-12-26 23:18:33,944][105692] Updated weights for policy 0, policy_version 1089513 (0.0011) [2023-12-26 23:18:33,979][105620] Updated weights for policy 1, policy_version 1090594 (0.0010) [2023-12-26 23:18:34,002][105692] Updated weights for policy 0, policy_version 1089523 (0.0009) [2023-12-26 23:18:34,037][105620] Updated weights for policy 1, policy_version 1090604 (0.0010) [2023-12-26 23:18:34,059][105692] Updated weights for policy 0, policy_version 1089533 (0.0006) [2023-12-26 23:18:34,094][105620] Updated weights for policy 1, policy_version 1090614 (0.0010) [2023-12-26 23:18:34,150][105620] Updated weights for policy 1, policy_version 1090624 (0.0009) [2023-12-26 23:18:34,639][105692] Updated weights for policy 0, policy_version 1089543 (0.0009) [2023-12-26 23:18:34,705][105692] Updated weights for policy 0, policy_version 1089553 (0.0011) [2023-12-26 23:18:34,771][105692] Updated weights for policy 0, policy_version 1089563 (0.0010) [2023-12-26 23:18:34,902][105620] Updated weights for policy 1, policy_version 1090634 (0.0005) [2023-12-26 23:18:34,950][105620] Updated weights for policy 1, policy_version 1090644 (0.0008) [2023-12-26 23:18:35,009][105620] Updated weights for policy 1, policy_version 1090654 (0.0010) [2023-12-26 23:18:35,440][105692] Updated weights for policy 0, policy_version 1089573 (0.0010) [2023-12-26 23:18:35,492][105692] Updated weights for policy 0, policy_version 1089583 (0.0010) [2023-12-26 23:18:35,536][105692] Updated weights for policy 0, policy_version 1089593 (0.0010) [2023-12-26 23:18:35,712][105620] Updated weights for policy 1, policy_version 1090664 (0.0010) [2023-12-26 23:18:35,763][105620] Updated weights for policy 1, policy_version 1090674 (0.0010) [2023-12-26 23:18:35,818][105620] Updated weights for policy 1, policy_version 1090684 (0.0010) [2023-12-26 23:18:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 558227456. Throughput: 0: 9682.1, 1: 9813.9. Samples: 558214908. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:18:36,063][104569] Avg episode reward: [(0, '9352.444'), (1, '9177.347')] [2023-12-26 23:18:36,282][105692] Updated weights for policy 0, policy_version 1089603 (0.0010) [2023-12-26 23:18:36,345][105692] Updated weights for policy 0, policy_version 1089613 (0.0011) [2023-12-26 23:18:36,404][105692] Updated weights for policy 0, policy_version 1089623 (0.0010) [2023-12-26 23:18:36,573][105620] Updated weights for policy 1, policy_version 1090694 (0.0011) [2023-12-26 23:18:36,636][105620] Updated weights for policy 1, policy_version 1090704 (0.0010) [2023-12-26 23:18:36,703][105620] Updated weights for policy 1, policy_version 1090714 (0.0011) [2023-12-26 23:18:37,111][105692] Updated weights for policy 0, policy_version 1089633 (0.0010) [2023-12-26 23:18:37,175][105692] Updated weights for policy 0, policy_version 1089643 (0.0005) [2023-12-26 23:18:37,232][105692] Updated weights for policy 0, policy_version 1089653 (0.0007) [2023-12-26 23:18:37,294][105692] Updated weights for policy 0, policy_version 1089663 (0.0010) [2023-12-26 23:18:37,425][105620] Updated weights for policy 1, policy_version 1090724 (0.0010) [2023-12-26 23:18:37,481][105620] Updated weights for policy 1, policy_version 1090734 (0.0008) [2023-12-26 23:18:37,538][105620] Updated weights for policy 1, policy_version 1090744 (0.0005) [2023-12-26 23:18:37,956][105692] Updated weights for policy 0, policy_version 1089673 (0.0010) [2023-12-26 23:18:38,008][105692] Updated weights for policy 0, policy_version 1089683 (0.0010) [2023-12-26 23:18:38,059][105692] Updated weights for policy 0, policy_version 1089693 (0.0010) [2023-12-26 23:18:38,194][105620] Updated weights for policy 1, policy_version 1090754 (0.0006) [2023-12-26 23:18:38,260][105620] Updated weights for policy 1, policy_version 1090764 (0.0011) [2023-12-26 23:18:38,348][105620] Updated weights for policy 1, policy_version 1090774 (0.0011) [2023-12-26 23:18:38,410][105620] Updated weights for policy 1, policy_version 1090784 (0.0009) [2023-12-26 23:18:38,807][105692] Updated weights for policy 0, policy_version 1089703 (0.0010) [2023-12-26 23:18:38,872][105692] Updated weights for policy 0, policy_version 1089713 (0.0011) [2023-12-26 23:18:38,925][105692] Updated weights for policy 0, policy_version 1089723 (0.0010) [2023-12-26 23:18:38,997][105620] Updated weights for policy 1, policy_version 1090794 (0.0011) [2023-12-26 23:18:39,056][105620] Updated weights for policy 1, policy_version 1090804 (0.0010) [2023-12-26 23:18:39,114][105620] Updated weights for policy 1, policy_version 1090814 (0.0010) [2023-12-26 23:18:39,651][105692] Updated weights for policy 0, policy_version 1089733 (0.0008) [2023-12-26 23:18:39,719][105692] Updated weights for policy 0, policy_version 1089743 (0.0007) [2023-12-26 23:18:39,787][105692] Updated weights for policy 0, policy_version 1089753 (0.0008) [2023-12-26 23:18:39,883][105620] Updated weights for policy 1, policy_version 1090824 (0.0008) [2023-12-26 23:18:39,949][105620] Updated weights for policy 1, policy_version 1090834 (0.0007) [2023-12-26 23:18:40,017][105620] Updated weights for policy 1, policy_version 1090844 (0.0009) [2023-12-26 23:18:40,535][105692] Updated weights for policy 0, policy_version 1089763 (0.0009) [2023-12-26 23:18:40,598][105692] Updated weights for policy 0, policy_version 1089773 (0.0008) [2023-12-26 23:18:40,659][105692] Updated weights for policy 0, policy_version 1089783 (0.0009) [2023-12-26 23:18:40,754][105620] Updated weights for policy 1, policy_version 1090854 (0.0009) [2023-12-26 23:18:40,808][105620] Updated weights for policy 1, policy_version 1090864 (0.0008) [2023-12-26 23:18:40,863][105620] Updated weights for policy 1, policy_version 1090874 (0.0009) [2023-12-26 23:18:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 558325760. Throughput: 0: 9668.4, 1: 9741.9. Samples: 558330468. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:18:41,063][104569] Avg episode reward: [(0, '9265.349'), (1, '8478.090')] [2023-12-26 23:18:41,474][105692] Updated weights for policy 0, policy_version 1089793 (0.0009) [2023-12-26 23:18:41,541][105692] Updated weights for policy 0, policy_version 1089803 (0.0010) [2023-12-26 23:18:41,570][105620] Updated weights for policy 1, policy_version 1090884 (0.0008) [2023-12-26 23:18:41,599][105692] Updated weights for policy 0, policy_version 1089813 (0.0009) [2023-12-26 23:18:41,630][105620] Updated weights for policy 1, policy_version 1090894 (0.0007) [2023-12-26 23:18:41,663][105692] Updated weights for policy 0, policy_version 1089823 (0.0008) [2023-12-26 23:18:41,691][105620] Updated weights for policy 1, policy_version 1090904 (0.0007) [2023-12-26 23:18:42,400][105620] Updated weights for policy 1, policy_version 1090914 (0.0008) [2023-12-26 23:18:42,447][105692] Updated weights for policy 0, policy_version 1089833 (0.0007) [2023-12-26 23:18:42,469][105620] Updated weights for policy 1, policy_version 1090924 (0.0007) [2023-12-26 23:18:42,511][105692] Updated weights for policy 0, policy_version 1089843 (0.0008) [2023-12-26 23:18:42,526][105620] Updated weights for policy 1, policy_version 1090934 (0.0005) [2023-12-26 23:18:42,568][105692] Updated weights for policy 0, policy_version 1089853 (0.0007) [2023-12-26 23:18:42,582][105620] Updated weights for policy 1, policy_version 1090944 (0.0006) [2023-12-26 23:18:43,318][105692] Updated weights for policy 0, policy_version 1089863 (0.0006) [2023-12-26 23:18:43,329][105620] Updated weights for policy 1, policy_version 1090954 (0.0007) [2023-12-26 23:18:43,371][105620] Updated weights for policy 1, policy_version 1090964 (0.0007) [2023-12-26 23:18:43,382][105692] Updated weights for policy 0, policy_version 1089873 (0.0007) [2023-12-26 23:18:43,418][105620] Updated weights for policy 1, policy_version 1090974 (0.0005) [2023-12-26 23:18:43,439][105692] Updated weights for policy 0, policy_version 1089883 (0.0009) [2023-12-26 23:18:43,974][105620] Updated weights for policy 1, policy_version 1090984 (0.0006) [2023-12-26 23:18:44,028][105620] Updated weights for policy 1, policy_version 1090994 (0.0005) [2023-12-26 23:18:44,096][105620] Updated weights for policy 1, policy_version 1091004 (0.0006) [2023-12-26 23:18:44,320][105692] Updated weights for policy 0, policy_version 1089893 (0.0009) [2023-12-26 23:18:44,378][105692] Updated weights for policy 0, policy_version 1089903 (0.0009) [2023-12-26 23:18:44,444][105692] Updated weights for policy 0, policy_version 1089913 (0.0008) [2023-12-26 23:18:44,779][105620] Updated weights for policy 1, policy_version 1091014 (0.0008) [2023-12-26 23:18:44,842][105620] Updated weights for policy 1, policy_version 1091024 (0.0009) [2023-12-26 23:18:44,905][105620] Updated weights for policy 1, policy_version 1091034 (0.0009) [2023-12-26 23:18:45,196][105692] Updated weights for policy 0, policy_version 1089923 (0.0008) [2023-12-26 23:18:45,264][105692] Updated weights for policy 0, policy_version 1089933 (0.0009) [2023-12-26 23:18:45,319][105692] Updated weights for policy 0, policy_version 1089943 (0.0009) [2023-12-26 23:18:45,585][105620] Updated weights for policy 1, policy_version 1091044 (0.0009) [2023-12-26 23:18:45,636][105620] Updated weights for policy 1, policy_version 1091054 (0.0009) [2023-12-26 23:18:45,683][105620] Updated weights for policy 1, policy_version 1091064 (0.0008) [2023-12-26 23:18:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 558415872. Throughput: 0: 9526.8, 1: 9784.1. Samples: 558388080. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:18:46,063][104569] Avg episode reward: [(0, '9172.415'), (1, '8557.260')] [2023-12-26 23:18:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001089952_279068672.pth... [2023-12-26 23:18:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001091072_279347200.pth... [2023-12-26 23:18:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001088864_278790144.pth [2023-12-26 23:18:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001089920_279052288.pth [2023-12-26 23:18:46,104][105692] Updated weights for policy 0, policy_version 1089953 (0.0009) [2023-12-26 23:18:46,155][105692] Updated weights for policy 0, policy_version 1089963 (0.0010) [2023-12-26 23:18:46,205][105692] Updated weights for policy 0, policy_version 1089973 (0.0005) [2023-12-26 23:18:46,274][105692] Updated weights for policy 0, policy_version 1089983 (0.0006) [2023-12-26 23:18:46,508][105620] Updated weights for policy 1, policy_version 1091074 (0.0009) [2023-12-26 23:18:46,561][105620] Updated weights for policy 1, policy_version 1091084 (0.0009) [2023-12-26 23:18:46,618][105620] Updated weights for policy 1, policy_version 1091095 (0.0010) [2023-12-26 23:18:46,838][105692] Updated weights for policy 0, policy_version 1089993 (0.0006) [2023-12-26 23:18:46,892][105692] Updated weights for policy 0, policy_version 1090003 (0.0005) [2023-12-26 23:18:46,952][105692] Updated weights for policy 0, policy_version 1090013 (0.0008) [2023-12-26 23:18:47,490][105692] Updated weights for policy 0, policy_version 1090023 (0.0010) [2023-12-26 23:18:47,513][105620] Updated weights for policy 1, policy_version 1091105 (0.0010) [2023-12-26 23:18:47,538][105692] Updated weights for policy 0, policy_version 1090033 (0.0010) [2023-12-26 23:18:47,568][105620] Updated weights for policy 1, policy_version 1091115 (0.0006) [2023-12-26 23:18:47,593][105692] Updated weights for policy 0, policy_version 1090043 (0.0010) [2023-12-26 23:18:47,619][105620] Updated weights for policy 1, policy_version 1091125 (0.0005) [2023-12-26 23:18:47,667][105620] Updated weights for policy 1, policy_version 1091135 (0.0008) [2023-12-26 23:18:48,255][105692] Updated weights for policy 0, policy_version 1090053 (0.0010) [2023-12-26 23:18:48,310][105692] Updated weights for policy 0, policy_version 1090063 (0.0010) [2023-12-26 23:18:48,376][105692] Updated weights for policy 0, policy_version 1090073 (0.0011) [2023-12-26 23:18:48,493][105620] Updated weights for policy 1, policy_version 1091145 (0.0008) [2023-12-26 23:18:48,556][105620] Updated weights for policy 1, policy_version 1091155 (0.0008) [2023-12-26 23:18:48,622][105620] Updated weights for policy 1, policy_version 1091165 (0.0009) [2023-12-26 23:18:49,087][105692] Updated weights for policy 0, policy_version 1090083 (0.0009) [2023-12-26 23:18:49,133][105692] Updated weights for policy 0, policy_version 1090093 (0.0007) [2023-12-26 23:18:49,194][105692] Updated weights for policy 0, policy_version 1090103 (0.0009) [2023-12-26 23:18:49,386][105620] Updated weights for policy 1, policy_version 1091175 (0.0007) [2023-12-26 23:18:49,445][105620] Updated weights for policy 1, policy_version 1091185 (0.0010) [2023-12-26 23:18:49,513][105620] Updated weights for policy 1, policy_version 1091195 (0.0010) [2023-12-26 23:18:49,911][105692] Updated weights for policy 0, policy_version 1090113 (0.0009) [2023-12-26 23:18:49,977][105692] Updated weights for policy 0, policy_version 1090123 (0.0009) [2023-12-26 23:18:50,031][105692] Updated weights for policy 0, policy_version 1090133 (0.0009) [2023-12-26 23:18:50,088][105692] Updated weights for policy 0, policy_version 1090143 (0.0010) [2023-12-26 23:18:50,263][105620] Updated weights for policy 1, policy_version 1091205 (0.0009) [2023-12-26 23:18:50,326][105620] Updated weights for policy 1, policy_version 1091215 (0.0009) [2023-12-26 23:18:50,390][105620] Updated weights for policy 1, policy_version 1091225 (0.0008) [2023-12-26 23:18:50,814][105692] Updated weights for policy 0, policy_version 1090153 (0.0009) [2023-12-26 23:18:50,862][105692] Updated weights for policy 0, policy_version 1090163 (0.0009) [2023-12-26 23:18:50,914][105692] Updated weights for policy 0, policy_version 1090173 (0.0009) [2023-12-26 23:18:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 558514176. Throughput: 0: 9573.1, 1: 9688.6. Samples: 558502380. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:18:51,062][104569] Avg episode reward: [(0, '9002.363'), (1, '9073.433')] [2023-12-26 23:18:51,126][105620] Updated weights for policy 1, policy_version 1091235 (0.0009) [2023-12-26 23:18:51,187][105620] Updated weights for policy 1, policy_version 1091245 (0.0010) [2023-12-26 23:18:51,249][105620] Updated weights for policy 1, policy_version 1091255 (0.0008) [2023-12-26 23:18:51,634][105692] Updated weights for policy 0, policy_version 1090183 (0.0008) [2023-12-26 23:18:51,698][105692] Updated weights for policy 0, policy_version 1090193 (0.0009) [2023-12-26 23:18:51,764][105692] Updated weights for policy 0, policy_version 1090203 (0.0008) [2023-12-26 23:18:52,016][105620] Updated weights for policy 1, policy_version 1091265 (0.0010) [2023-12-26 23:18:52,067][105620] Updated weights for policy 1, policy_version 1091275 (0.0009) [2023-12-26 23:18:52,121][105620] Updated weights for policy 1, policy_version 1091285 (0.0009) [2023-12-26 23:18:52,179][105620] Updated weights for policy 1, policy_version 1091295 (0.0009) [2023-12-26 23:18:52,515][105692] Updated weights for policy 0, policy_version 1090213 (0.0010) [2023-12-26 23:18:52,575][105692] Updated weights for policy 0, policy_version 1090223 (0.0011) [2023-12-26 23:18:52,634][105692] Updated weights for policy 0, policy_version 1090233 (0.0011) [2023-12-26 23:18:52,978][105620] Updated weights for policy 1, policy_version 1091305 (0.0009) [2023-12-26 23:18:53,025][105620] Updated weights for policy 1, policy_version 1091315 (0.0008) [2023-12-26 23:18:53,084][105620] Updated weights for policy 1, policy_version 1091325 (0.0008) [2023-12-26 23:18:53,359][105692] Updated weights for policy 0, policy_version 1090243 (0.0009) [2023-12-26 23:18:53,414][105692] Updated weights for policy 0, policy_version 1090253 (0.0007) [2023-12-26 23:18:53,468][105692] Updated weights for policy 0, policy_version 1090263 (0.0005) [2023-12-26 23:18:53,846][105620] Updated weights for policy 1, policy_version 1091335 (0.0010) [2023-12-26 23:18:53,909][105620] Updated weights for policy 1, policy_version 1091345 (0.0011) [2023-12-26 23:18:53,967][105620] Updated weights for policy 1, policy_version 1091355 (0.0010) [2023-12-26 23:18:53,983][105692] Updated weights for policy 0, policy_version 1090273 (0.0005) [2023-12-26 23:18:54,047][105692] Updated weights for policy 0, policy_version 1090283 (0.0007) [2023-12-26 23:18:54,119][105692] Updated weights for policy 0, policy_version 1090293 (0.0007) [2023-12-26 23:18:54,183][105692] Updated weights for policy 0, policy_version 1090303 (0.0007) [2023-12-26 23:18:54,656][105620] Updated weights for policy 1, policy_version 1091365 (0.0010) [2023-12-26 23:18:54,714][105620] Updated weights for policy 1, policy_version 1091375 (0.0010) [2023-12-26 23:18:54,775][105620] Updated weights for policy 1, policy_version 1091385 (0.0010) [2023-12-26 23:18:54,787][105692] Updated weights for policy 0, policy_version 1090313 (0.0010) [2023-12-26 23:18:54,845][105692] Updated weights for policy 0, policy_version 1090323 (0.0010) [2023-12-26 23:18:54,903][105692] Updated weights for policy 0, policy_version 1090333 (0.0010) [2023-12-26 23:18:55,497][105620] Updated weights for policy 1, policy_version 1091395 (0.0010) [2023-12-26 23:18:55,508][105692] Updated weights for policy 0, policy_version 1090343 (0.0007) [2023-12-26 23:18:55,555][105620] Updated weights for policy 1, policy_version 1091405 (0.0010) [2023-12-26 23:18:55,563][105692] Updated weights for policy 0, policy_version 1090353 (0.0010) [2023-12-26 23:18:55,613][105620] Updated weights for policy 1, policy_version 1091415 (0.0010) [2023-12-26 23:18:55,618][105692] Updated weights for policy 0, policy_version 1090363 (0.0010) [2023-12-26 23:18:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.8, 300 sec: 19438.7). Total num frames: 558612480. Throughput: 0: 9679.6, 1: 9596.3. Samples: 558620732. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:18:56,062][104569] Avg episode reward: [(0, '8514.808'), (1, '9165.766')] [2023-12-26 23:18:56,312][105692] Updated weights for policy 0, policy_version 1090373 (0.0010) [2023-12-26 23:18:56,331][105620] Updated weights for policy 1, policy_version 1091425 (0.0010) [2023-12-26 23:18:56,372][105692] Updated weights for policy 0, policy_version 1090383 (0.0011) [2023-12-26 23:18:56,393][105620] Updated weights for policy 1, policy_version 1091435 (0.0010) [2023-12-26 23:18:56,429][105692] Updated weights for policy 0, policy_version 1090393 (0.0011) [2023-12-26 23:18:56,453][105620] Updated weights for policy 1, policy_version 1091445 (0.0011) [2023-12-26 23:18:56,502][105620] Updated weights for policy 1, policy_version 1091455 (0.0008) [2023-12-26 23:18:57,115][105620] Updated weights for policy 1, policy_version 1091465 (0.0008) [2023-12-26 23:18:57,172][105620] Updated weights for policy 1, policy_version 1091475 (0.0007) [2023-12-26 23:18:57,177][105692] Updated weights for policy 0, policy_version 1090403 (0.0010) [2023-12-26 23:18:57,226][105620] Updated weights for policy 1, policy_version 1091485 (0.0008) [2023-12-26 23:18:57,234][105692] Updated weights for policy 0, policy_version 1090413 (0.0010) [2023-12-26 23:18:57,282][105692] Updated weights for policy 0, policy_version 1090423 (0.0010) [2023-12-26 23:18:57,830][105692] Updated weights for policy 0, policy_version 1090433 (0.0010) [2023-12-26 23:18:57,884][105692] Updated weights for policy 0, policy_version 1090443 (0.0005) [2023-12-26 23:18:57,945][105692] Updated weights for policy 0, policy_version 1090453 (0.0005) [2023-12-26 23:18:58,002][105692] Updated weights for policy 0, policy_version 1090463 (0.0006) [2023-12-26 23:18:58,099][105620] Updated weights for policy 1, policy_version 1091495 (0.0007) [2023-12-26 23:18:58,157][105620] Updated weights for policy 1, policy_version 1091505 (0.0006) [2023-12-26 23:18:58,219][105620] Updated weights for policy 1, policy_version 1091515 (0.0008) [2023-12-26 23:18:58,726][105692] Updated weights for policy 0, policy_version 1090473 (0.0008) [2023-12-26 23:18:58,798][105692] Updated weights for policy 0, policy_version 1090483 (0.0007) [2023-12-26 23:18:58,870][105692] Updated weights for policy 0, policy_version 1090493 (0.0007) [2023-12-26 23:18:59,062][105620] Updated weights for policy 1, policy_version 1091525 (0.0008) [2023-12-26 23:18:59,126][105620] Updated weights for policy 1, policy_version 1091535 (0.0009) [2023-12-26 23:18:59,192][105620] Updated weights for policy 1, policy_version 1091545 (0.0009) [2023-12-26 23:18:59,601][105692] Updated weights for policy 0, policy_version 1090503 (0.0008) [2023-12-26 23:18:59,645][105692] Updated weights for policy 0, policy_version 1090513 (0.0006) [2023-12-26 23:18:59,693][105692] Updated weights for policy 0, policy_version 1090523 (0.0005) [2023-12-26 23:18:59,889][105620] Updated weights for policy 1, policy_version 1091555 (0.0008) [2023-12-26 23:18:59,956][105620] Updated weights for policy 1, policy_version 1091565 (0.0006) [2023-12-26 23:19:00,015][105620] Updated weights for policy 1, policy_version 1091575 (0.0006) [2023-12-26 23:19:00,436][105692] Updated weights for policy 0, policy_version 1090533 (0.0007) [2023-12-26 23:19:00,496][105692] Updated weights for policy 0, policy_version 1090543 (0.0009) [2023-12-26 23:19:00,551][105692] Updated weights for policy 0, policy_version 1090554 (0.0010) [2023-12-26 23:19:00,569][105620] Updated weights for policy 1, policy_version 1091585 (0.0007) [2023-12-26 23:19:00,624][105620] Updated weights for policy 1, policy_version 1091595 (0.0005) [2023-12-26 23:19:00,678][105620] Updated weights for policy 1, policy_version 1091605 (0.0005) [2023-12-26 23:19:00,736][105620] Updated weights for policy 1, policy_version 1091615 (0.0005) [2023-12-26 23:19:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 558710784. Throughput: 0: 9727.2, 1: 9574.3. Samples: 558679564. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:19:01,063][104569] Avg episode reward: [(0, '8760.076'), (1, '9258.081')] [2023-12-26 23:19:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001090560_279224320.pth... [2023-12-26 23:19:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001091616_279486464.pth... [2023-12-26 23:19:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001090496_279199744.pth [2023-12-26 23:19:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001089408_278929408.pth [2023-12-26 23:19:01,347][105692] Updated weights for policy 0, policy_version 1090564 (0.0008) [2023-12-26 23:19:01,399][105620] Updated weights for policy 1, policy_version 1091625 (0.0007) [2023-12-26 23:19:01,409][105692] Updated weights for policy 0, policy_version 1090574 (0.0009) [2023-12-26 23:19:01,460][105620] Updated weights for policy 1, policy_version 1091635 (0.0006) [2023-12-26 23:19:01,463][105692] Updated weights for policy 0, policy_version 1090584 (0.0008) [2023-12-26 23:19:01,520][105620] Updated weights for policy 1, policy_version 1091645 (0.0008) [2023-12-26 23:19:02,183][105620] Updated weights for policy 1, policy_version 1091655 (0.0010) [2023-12-26 23:19:02,207][105692] Updated weights for policy 0, policy_version 1090594 (0.0008) [2023-12-26 23:19:02,237][105620] Updated weights for policy 1, policy_version 1091665 (0.0008) [2023-12-26 23:19:02,266][105692] Updated weights for policy 0, policy_version 1090604 (0.0007) [2023-12-26 23:19:02,294][105620] Updated weights for policy 1, policy_version 1091675 (0.0011) [2023-12-26 23:19:02,329][105692] Updated weights for policy 0, policy_version 1090614 (0.0006) [2023-12-26 23:19:02,390][105692] Updated weights for policy 0, policy_version 1090624 (0.0009) [2023-12-26 23:19:03,002][105692] Updated weights for policy 0, policy_version 1090634 (0.0008) [2023-12-26 23:19:03,041][105620] Updated weights for policy 1, policy_version 1091685 (0.0011) [2023-12-26 23:19:03,053][105692] Updated weights for policy 0, policy_version 1090644 (0.0006) [2023-12-26 23:19:03,093][105620] Updated weights for policy 1, policy_version 1091695 (0.0010) [2023-12-26 23:19:03,100][105692] Updated weights for policy 0, policy_version 1090654 (0.0005) [2023-12-26 23:19:03,144][105620] Updated weights for policy 1, policy_version 1091705 (0.0010) [2023-12-26 23:19:03,744][105692] Updated weights for policy 0, policy_version 1090664 (0.0008) [2023-12-26 23:19:03,800][105692] Updated weights for policy 0, policy_version 1090674 (0.0007) [2023-12-26 23:19:03,818][105620] Updated weights for policy 1, policy_version 1091715 (0.0010) [2023-12-26 23:19:03,854][105692] Updated weights for policy 0, policy_version 1090684 (0.0007) [2023-12-26 23:19:03,879][105620] Updated weights for policy 1, policy_version 1091725 (0.0008) [2023-12-26 23:19:03,942][105620] Updated weights for policy 1, policy_version 1091735 (0.0008) [2023-12-26 23:19:04,543][105692] Updated weights for policy 0, policy_version 1090694 (0.0007) [2023-12-26 23:19:04,590][105692] Updated weights for policy 0, policy_version 1090704 (0.0005) [2023-12-26 23:19:04,646][105692] Updated weights for policy 0, policy_version 1090714 (0.0005) [2023-12-26 23:19:04,725][105620] Updated weights for policy 1, policy_version 1091745 (0.0008) [2023-12-26 23:19:04,781][105620] Updated weights for policy 1, policy_version 1091755 (0.0005) [2023-12-26 23:19:04,842][105620] Updated weights for policy 1, policy_version 1091765 (0.0006) [2023-12-26 23:19:04,898][105620] Updated weights for policy 1, policy_version 1091775 (0.0009) [2023-12-26 23:19:05,286][105692] Updated weights for policy 0, policy_version 1090724 (0.0007) [2023-12-26 23:19:05,336][105692] Updated weights for policy 0, policy_version 1090734 (0.0005) [2023-12-26 23:19:05,394][105692] Updated weights for policy 0, policy_version 1090744 (0.0005) [2023-12-26 23:19:05,720][105620] Updated weights for policy 1, policy_version 1091785 (0.0009) [2023-12-26 23:19:05,776][105620] Updated weights for policy 1, policy_version 1091795 (0.0012) [2023-12-26 23:19:05,834][105620] Updated weights for policy 1, policy_version 1091806 (0.0010) [2023-12-26 23:19:05,925][105692] Updated weights for policy 0, policy_version 1090754 (0.0005) [2023-12-26 23:19:05,970][105692] Updated weights for policy 0, policy_version 1090764 (0.0005) [2023-12-26 23:19:06,014][105692] Updated weights for policy 0, policy_version 1090774 (0.0005) [2023-12-26 23:19:06,060][105692] Updated weights for policy 0, policy_version 1090784 (0.0005) [2023-12-26 23:19:06,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 558817280. Throughput: 0: 9768.6, 1: 9603.2. Samples: 558798764. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:19:06,063][104569] Avg episode reward: [(0, '9347.675'), (1, '9349.686')] [2023-12-26 23:19:06,663][105620] Updated weights for policy 1, policy_version 1091816 (0.0006) [2023-12-26 23:19:06,723][105620] Updated weights for policy 1, policy_version 1091826 (0.0008) [2023-12-26 23:19:06,775][105692] Updated weights for policy 0, policy_version 1090794 (0.0010) [2023-12-26 23:19:06,783][105620] Updated weights for policy 1, policy_version 1091836 (0.0010) [2023-12-26 23:19:06,837][105692] Updated weights for policy 0, policy_version 1090804 (0.0010) [2023-12-26 23:19:06,903][105692] Updated weights for policy 0, policy_version 1090814 (0.0010) [2023-12-26 23:19:07,519][105620] Updated weights for policy 1, policy_version 1091846 (0.0007) [2023-12-26 23:19:07,572][105620] Updated weights for policy 1, policy_version 1091856 (0.0008) [2023-12-26 23:19:07,627][105620] Updated weights for policy 1, policy_version 1091866 (0.0007) [2023-12-26 23:19:07,644][105692] Updated weights for policy 0, policy_version 1090824 (0.0010) [2023-12-26 23:19:07,699][105692] Updated weights for policy 0, policy_version 1090834 (0.0010) [2023-12-26 23:19:07,747][105692] Updated weights for policy 0, policy_version 1090844 (0.0010) [2023-12-26 23:19:08,375][105620] Updated weights for policy 1, policy_version 1091876 (0.0006) [2023-12-26 23:19:08,434][105620] Updated weights for policy 1, policy_version 1091886 (0.0008) [2023-12-26 23:19:08,488][105620] Updated weights for policy 1, policy_version 1091896 (0.0006) [2023-12-26 23:19:08,490][105692] Updated weights for policy 0, policy_version 1090854 (0.0011) [2023-12-26 23:19:08,542][105692] Updated weights for policy 0, policy_version 1090864 (0.0011) [2023-12-26 23:19:08,604][105692] Updated weights for policy 0, policy_version 1090874 (0.0011) [2023-12-26 23:19:09,234][105620] Updated weights for policy 1, policy_version 1091906 (0.0006) [2023-12-26 23:19:09,300][105620] Updated weights for policy 1, policy_version 1091916 (0.0008) [2023-12-26 23:19:09,364][105620] Updated weights for policy 1, policy_version 1091926 (0.0008) [2023-12-26 23:19:09,375][105692] Updated weights for policy 0, policy_version 1090884 (0.0010) [2023-12-26 23:19:09,432][105620] Updated weights for policy 1, policy_version 1091936 (0.0008) [2023-12-26 23:19:09,435][105692] Updated weights for policy 0, policy_version 1090894 (0.0008) [2023-12-26 23:19:09,507][105692] Updated weights for policy 0, policy_version 1090904 (0.0010) [2023-12-26 23:19:10,153][105692] Updated weights for policy 0, policy_version 1090914 (0.0011) [2023-12-26 23:19:10,206][105692] Updated weights for policy 0, policy_version 1090924 (0.0011) [2023-12-26 23:19:10,224][105620] Updated weights for policy 1, policy_version 1091946 (0.0006) [2023-12-26 23:19:10,265][105692] Updated weights for policy 0, policy_version 1090934 (0.0011) [2023-12-26 23:19:10,283][105620] Updated weights for policy 1, policy_version 1091956 (0.0006) [2023-12-26 23:19:10,325][105692] Updated weights for policy 0, policy_version 1090944 (0.0010) [2023-12-26 23:19:10,348][105620] Updated weights for policy 1, policy_version 1091966 (0.0007) [2023-12-26 23:19:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.1, 300 sec: 19410.9). Total num frames: 558899200. Throughput: 0: 9880.0, 1: 9481.2. Samples: 558913424. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:19:11,063][104569] Avg episode reward: [(0, '9347.838'), (1, '9074.711')] [2023-12-26 23:19:11,099][105692] Updated weights for policy 0, policy_version 1090954 (0.0010) [2023-12-26 23:19:11,116][105620] Updated weights for policy 1, policy_version 1091976 (0.0007) [2023-12-26 23:19:11,164][105692] Updated weights for policy 0, policy_version 1090964 (0.0010) [2023-12-26 23:19:11,178][105620] Updated weights for policy 1, policy_version 1091986 (0.0008) [2023-12-26 23:19:11,224][105692] Updated weights for policy 0, policy_version 1090974 (0.0010) [2023-12-26 23:19:11,234][105620] Updated weights for policy 1, policy_version 1091996 (0.0006) [2023-12-26 23:19:12,042][105692] Updated weights for policy 0, policy_version 1090984 (0.0011) [2023-12-26 23:19:12,064][105620] Updated weights for policy 1, policy_version 1092006 (0.0006) [2023-12-26 23:19:12,094][105692] Updated weights for policy 0, policy_version 1090994 (0.0010) [2023-12-26 23:19:12,125][105620] Updated weights for policy 1, policy_version 1092016 (0.0006) [2023-12-26 23:19:12,147][105692] Updated weights for policy 0, policy_version 1091004 (0.0010) [2023-12-26 23:19:12,186][105620] Updated weights for policy 1, policy_version 1092026 (0.0006) [2023-12-26 23:19:12,918][105620] Updated weights for policy 1, policy_version 1092036 (0.0009) [2023-12-26 23:19:12,927][105692] Updated weights for policy 0, policy_version 1091014 (0.0010) [2023-12-26 23:19:12,967][105620] Updated weights for policy 1, policy_version 1092046 (0.0010) [2023-12-26 23:19:12,975][105692] Updated weights for policy 0, policy_version 1091024 (0.0010) [2023-12-26 23:19:13,026][105620] Updated weights for policy 1, policy_version 1092056 (0.0010) [2023-12-26 23:19:13,026][105692] Updated weights for policy 0, policy_version 1091034 (0.0010) [2023-12-26 23:19:13,661][105620] Updated weights for policy 1, policy_version 1092066 (0.0009) [2023-12-26 23:19:13,722][105620] Updated weights for policy 1, policy_version 1092076 (0.0010) [2023-12-26 23:19:13,779][105692] Updated weights for policy 0, policy_version 1091044 (0.0010) [2023-12-26 23:19:13,787][105620] Updated weights for policy 1, policy_version 1092086 (0.0010) [2023-12-26 23:19:13,827][105692] Updated weights for policy 0, policy_version 1091054 (0.0010) [2023-12-26 23:19:13,852][105620] Updated weights for policy 1, policy_version 1092096 (0.0009) [2023-12-26 23:19:13,878][105692] Updated weights for policy 0, policy_version 1091064 (0.0010) [2023-12-26 23:19:14,442][105620] Updated weights for policy 1, policy_version 1092106 (0.0005) [2023-12-26 23:19:14,490][105620] Updated weights for policy 1, policy_version 1092116 (0.0010) [2023-12-26 23:19:14,542][105620] Updated weights for policy 1, policy_version 1092126 (0.0010) [2023-12-26 23:19:14,639][105692] Updated weights for policy 0, policy_version 1091074 (0.0010) [2023-12-26 23:19:14,697][105692] Updated weights for policy 0, policy_version 1091084 (0.0010) [2023-12-26 23:19:14,745][105692] Updated weights for policy 0, policy_version 1091094 (0.0010) [2023-12-26 23:19:14,805][105692] Updated weights for policy 0, policy_version 1091104 (0.0010) [2023-12-26 23:19:15,321][105620] Updated weights for policy 1, policy_version 1092136 (0.0010) [2023-12-26 23:19:15,378][105620] Updated weights for policy 1, policy_version 1092146 (0.0011) [2023-12-26 23:19:15,441][105620] Updated weights for policy 1, policy_version 1092156 (0.0011) [2023-12-26 23:19:15,576][105692] Updated weights for policy 0, policy_version 1091114 (0.0010) [2023-12-26 23:19:15,628][105692] Updated weights for policy 0, policy_version 1091124 (0.0010) [2023-12-26 23:19:15,672][105692] Updated weights for policy 0, policy_version 1091134 (0.0010) [2023-12-26 23:19:16,062][104569] Fps is (10 sec: 18022.1, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 558997504. Throughput: 0: 9830.0, 1: 9501.5. Samples: 558969252. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:19:16,063][104569] Avg episode reward: [(0, '9175.648'), (1, '8892.947')] [2023-12-26 23:19:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001091136_279371776.pth... [2023-12-26 23:19:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001092160_279625728.pth... [2023-12-26 23:19:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001091072_279347200.pth [2023-12-26 23:19:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001089952_279068672.pth [2023-12-26 23:19:16,187][105620] Updated weights for policy 1, policy_version 1092166 (0.0011) [2023-12-26 23:19:16,248][105620] Updated weights for policy 1, policy_version 1092176 (0.0010) [2023-12-26 23:19:16,303][105620] Updated weights for policy 1, policy_version 1092186 (0.0010) [2023-12-26 23:19:16,402][105692] Updated weights for policy 0, policy_version 1091144 (0.0010) [2023-12-26 23:19:16,455][105692] Updated weights for policy 0, policy_version 1091154 (0.0010) [2023-12-26 23:19:16,515][105692] Updated weights for policy 0, policy_version 1091164 (0.0006) [2023-12-26 23:19:17,080][105692] Updated weights for policy 0, policy_version 1091174 (0.0009) [2023-12-26 23:19:17,094][105620] Updated weights for policy 1, policy_version 1092196 (0.0009) [2023-12-26 23:19:17,136][105692] Updated weights for policy 0, policy_version 1091184 (0.0011) [2023-12-26 23:19:17,146][105620] Updated weights for policy 1, policy_version 1092206 (0.0006) [2023-12-26 23:19:17,188][105692] Updated weights for policy 0, policy_version 1091194 (0.0010) [2023-12-26 23:19:17,198][105620] Updated weights for policy 1, policy_version 1092216 (0.0005) [2023-12-26 23:19:17,942][105692] Updated weights for policy 0, policy_version 1091204 (0.0010) [2023-12-26 23:19:17,963][105620] Updated weights for policy 1, policy_version 1092226 (0.0008) [2023-12-26 23:19:18,000][105692] Updated weights for policy 0, policy_version 1091214 (0.0010) [2023-12-26 23:19:18,011][105620] Updated weights for policy 1, policy_version 1092236 (0.0007) [2023-12-26 23:19:18,061][105692] Updated weights for policy 0, policy_version 1091224 (0.0010) [2023-12-26 23:19:18,064][105620] Updated weights for policy 1, policy_version 1092246 (0.0006) [2023-12-26 23:19:18,116][105620] Updated weights for policy 1, policy_version 1092256 (0.0007) [2023-12-26 23:19:18,778][105620] Updated weights for policy 1, policy_version 1092266 (0.0005) [2023-12-26 23:19:18,806][105692] Updated weights for policy 0, policy_version 1091234 (0.0010) [2023-12-26 23:19:18,832][105620] Updated weights for policy 1, policy_version 1092276 (0.0007) [2023-12-26 23:19:18,854][105692] Updated weights for policy 0, policy_version 1091244 (0.0008) [2023-12-26 23:19:18,899][105620] Updated weights for policy 1, policy_version 1092286 (0.0009) [2023-12-26 23:19:18,914][105692] Updated weights for policy 0, policy_version 1091254 (0.0009) [2023-12-26 23:19:18,967][105692] Updated weights for policy 0, policy_version 1091264 (0.0010) [2023-12-26 23:19:19,634][105620] Updated weights for policy 1, policy_version 1092296 (0.0010) [2023-12-26 23:19:19,701][105620] Updated weights for policy 1, policy_version 1092306 (0.0008) [2023-12-26 23:19:19,729][105692] Updated weights for policy 0, policy_version 1091274 (0.0007) [2023-12-26 23:19:19,761][105620] Updated weights for policy 1, policy_version 1092316 (0.0008) [2023-12-26 23:19:19,792][105692] Updated weights for policy 0, policy_version 1091284 (0.0009) [2023-12-26 23:19:19,853][105692] Updated weights for policy 0, policy_version 1091294 (0.0008) [2023-12-26 23:19:20,499][105620] Updated weights for policy 1, policy_version 1092326 (0.0008) [2023-12-26 23:19:20,530][105692] Updated weights for policy 0, policy_version 1091304 (0.0010) [2023-12-26 23:19:20,559][105620] Updated weights for policy 1, policy_version 1092336 (0.0008) [2023-12-26 23:19:20,584][105692] Updated weights for policy 0, policy_version 1091314 (0.0010) [2023-12-26 23:19:20,633][105620] Updated weights for policy 1, policy_version 1092346 (0.0007) [2023-12-26 23:19:20,647][105692] Updated weights for policy 0, policy_version 1091324 (0.0010) [2023-12-26 23:19:21,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 559095808. Throughput: 0: 9827.1, 1: 9508.9. Samples: 559085028. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:19:21,062][104569] Avg episode reward: [(0, '9176.017'), (1, '8983.956')] [2023-12-26 23:19:21,398][105620] Updated weights for policy 1, policy_version 1092356 (0.0007) [2023-12-26 23:19:21,418][105692] Updated weights for policy 0, policy_version 1091334 (0.0011) [2023-12-26 23:19:21,449][105620] Updated weights for policy 1, policy_version 1092366 (0.0007) [2023-12-26 23:19:21,471][105692] Updated weights for policy 0, policy_version 1091344 (0.0010) [2023-12-26 23:19:21,502][105620] Updated weights for policy 1, policy_version 1092376 (0.0007) [2023-12-26 23:19:21,537][105692] Updated weights for policy 0, policy_version 1091354 (0.0011) [2023-12-26 23:19:22,200][105620] Updated weights for policy 1, policy_version 1092386 (0.0008) [2023-12-26 23:19:22,260][105620] Updated weights for policy 1, policy_version 1092396 (0.0009) [2023-12-26 23:19:22,274][105692] Updated weights for policy 0, policy_version 1091364 (0.0010) [2023-12-26 23:19:22,330][105620] Updated weights for policy 1, policy_version 1092406 (0.0008) [2023-12-26 23:19:22,338][105692] Updated weights for policy 0, policy_version 1091374 (0.0007) [2023-12-26 23:19:22,396][105620] Updated weights for policy 1, policy_version 1092416 (0.0007) [2023-12-26 23:19:22,408][105692] Updated weights for policy 0, policy_version 1091384 (0.0009) [2023-12-26 23:19:23,107][105620] Updated weights for policy 1, policy_version 1092426 (0.0007) [2023-12-26 23:19:23,157][105620] Updated weights for policy 1, policy_version 1092436 (0.0008) [2023-12-26 23:19:23,195][105692] Updated weights for policy 0, policy_version 1091394 (0.0009) [2023-12-26 23:19:23,211][105620] Updated weights for policy 1, policy_version 1092446 (0.0008) [2023-12-26 23:19:23,249][105692] Updated weights for policy 0, policy_version 1091404 (0.0009) [2023-12-26 23:19:23,296][105692] Updated weights for policy 0, policy_version 1091414 (0.0008) [2023-12-26 23:19:23,355][105692] Updated weights for policy 0, policy_version 1091424 (0.0009) [2023-12-26 23:19:23,979][105620] Updated weights for policy 1, policy_version 1092456 (0.0007) [2023-12-26 23:19:24,029][105620] Updated weights for policy 1, policy_version 1092466 (0.0009) [2023-12-26 23:19:24,075][105620] Updated weights for policy 1, policy_version 1092476 (0.0006) [2023-12-26 23:19:24,088][105692] Updated weights for policy 0, policy_version 1091434 (0.0008) [2023-12-26 23:19:24,139][105692] Updated weights for policy 0, policy_version 1091444 (0.0008) [2023-12-26 23:19:24,198][105692] Updated weights for policy 0, policy_version 1091454 (0.0009) [2023-12-26 23:19:24,716][105620] Updated weights for policy 1, policy_version 1092486 (0.0008) [2023-12-26 23:19:24,786][105620] Updated weights for policy 1, policy_version 1092496 (0.0006) [2023-12-26 23:19:24,843][105620] Updated weights for policy 1, policy_version 1092506 (0.0008) [2023-12-26 23:19:24,948][105692] Updated weights for policy 0, policy_version 1091464 (0.0006) [2023-12-26 23:19:25,010][105692] Updated weights for policy 0, policy_version 1091474 (0.0005) [2023-12-26 23:19:25,059][105692] Updated weights for policy 0, policy_version 1091484 (0.0005) [2023-12-26 23:19:25,536][105620] Updated weights for policy 1, policy_version 1092516 (0.0009) [2023-12-26 23:19:25,591][105692] Updated weights for policy 0, policy_version 1091494 (0.0007) [2023-12-26 23:19:25,592][105620] Updated weights for policy 1, policy_version 1092526 (0.0008) [2023-12-26 23:19:25,645][105692] Updated weights for policy 0, policy_version 1091504 (0.0007) [2023-12-26 23:19:25,654][105620] Updated weights for policy 1, policy_version 1092536 (0.0007) [2023-12-26 23:19:25,692][105692] Updated weights for policy 0, policy_version 1091514 (0.0009) [2023-12-26 23:19:26,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 559194112. Throughput: 0: 9854.0, 1: 9525.2. Samples: 559202528. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:19:26,062][104569] Avg episode reward: [(0, '9259.142'), (1, '9270.671')] [2023-12-26 23:19:26,248][105620] Updated weights for policy 1, policy_version 1092546 (0.0009) [2023-12-26 23:19:26,297][105620] Updated weights for policy 1, policy_version 1092556 (0.0006) [2023-12-26 23:19:26,346][105620] Updated weights for policy 1, policy_version 1092566 (0.0005) [2023-12-26 23:19:26,395][105692] Updated weights for policy 0, policy_version 1091524 (0.0010) [2023-12-26 23:19:26,406][105620] Updated weights for policy 1, policy_version 1092576 (0.0005) [2023-12-26 23:19:26,455][105692] Updated weights for policy 0, policy_version 1091534 (0.0010) [2023-12-26 23:19:26,521][105692] Updated weights for policy 0, policy_version 1091544 (0.0005) [2023-12-26 23:19:26,971][105620] Updated weights for policy 1, policy_version 1092586 (0.0010) [2023-12-26 23:19:27,023][105620] Updated weights for policy 1, policy_version 1092596 (0.0010) [2023-12-26 23:19:27,074][105620] Updated weights for policy 1, policy_version 1092606 (0.0010) [2023-12-26 23:19:27,145][105692] Updated weights for policy 0, policy_version 1091554 (0.0005) [2023-12-26 23:19:27,207][105692] Updated weights for policy 0, policy_version 1091564 (0.0005) [2023-12-26 23:19:27,263][105692] Updated weights for policy 0, policy_version 1091574 (0.0005) [2023-12-26 23:19:27,318][105692] Updated weights for policy 0, policy_version 1091584 (0.0006) [2023-12-26 23:19:27,810][105620] Updated weights for policy 1, policy_version 1092616 (0.0010) [2023-12-26 23:19:27,862][105620] Updated weights for policy 1, policy_version 1092626 (0.0010) [2023-12-26 23:19:27,866][105692] Updated weights for policy 0, policy_version 1091594 (0.0007) [2023-12-26 23:19:27,918][105692] Updated weights for policy 0, policy_version 1091604 (0.0010) [2023-12-26 23:19:27,918][105620] Updated weights for policy 1, policy_version 1092636 (0.0007) [2023-12-26 23:19:27,972][105692] Updated weights for policy 0, policy_version 1091614 (0.0010) [2023-12-26 23:19:28,604][105692] Updated weights for policy 0, policy_version 1091624 (0.0006) [2023-12-26 23:19:28,619][105620] Updated weights for policy 1, policy_version 1092646 (0.0006) [2023-12-26 23:19:28,666][105692] Updated weights for policy 0, policy_version 1091634 (0.0008) [2023-12-26 23:19:28,671][105620] Updated weights for policy 1, policy_version 1092656 (0.0005) [2023-12-26 23:19:28,723][105692] Updated weights for policy 0, policy_version 1091644 (0.0009) [2023-12-26 23:19:28,724][105620] Updated weights for policy 1, policy_version 1092666 (0.0005) [2023-12-26 23:19:29,369][105620] Updated weights for policy 1, policy_version 1092676 (0.0007) [2023-12-26 23:19:29,371][105692] Updated weights for policy 0, policy_version 1091654 (0.0008) [2023-12-26 23:19:29,421][105620] Updated weights for policy 1, policy_version 1092686 (0.0008) [2023-12-26 23:19:29,423][105692] Updated weights for policy 0, policy_version 1091664 (0.0005) [2023-12-26 23:19:29,438][105585] KL-divergence is very high: 124.4592 [2023-12-26 23:19:29,472][105692] Updated weights for policy 0, policy_version 1091674 (0.0008) [2023-12-26 23:19:29,477][105585] KL-divergence is very high: 132.0053 [2023-12-26 23:19:29,478][105620] Updated weights for policy 1, policy_version 1092696 (0.0008) [2023-12-26 23:19:30,163][105620] Updated weights for policy 1, policy_version 1092706 (0.0008) [2023-12-26 23:19:30,228][105620] Updated weights for policy 1, policy_version 1092716 (0.0010) [2023-12-26 23:19:30,237][105692] Updated weights for policy 0, policy_version 1091684 (0.0006) [2023-12-26 23:19:30,280][105620] Updated weights for policy 1, policy_version 1092726 (0.0010) [2023-12-26 23:19:30,294][105692] Updated weights for policy 0, policy_version 1091694 (0.0006) [2023-12-26 23:19:30,332][105620] Updated weights for policy 1, policy_version 1092736 (0.0010) [2023-12-26 23:19:30,360][105692] Updated weights for policy 0, policy_version 1091704 (0.0007) [2023-12-26 23:19:30,936][105692] Updated weights for policy 0, policy_version 1091714 (0.0005) [2023-12-26 23:19:30,986][105620] Updated weights for policy 1, policy_version 1092746 (0.0005) [2023-12-26 23:19:30,999][105692] Updated weights for policy 0, policy_version 1091724 (0.0006) [2023-12-26 23:19:31,045][105620] Updated weights for policy 1, policy_version 1092756 (0.0008) [2023-12-26 23:19:31,062][105692] Updated weights for policy 0, policy_version 1091734 (0.0010) [2023-12-26 23:19:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 559292416. Throughput: 0: 9992.0, 1: 9559.4. Samples: 559267892. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:19:31,062][104569] Avg episode reward: [(0, '9082.677'), (1, '9180.287')] [2023-12-26 23:19:31,099][105620] Updated weights for policy 1, policy_version 1092766 (0.0007) [2023-12-26 23:19:31,111][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001092768_279781376.pth... [2023-12-26 23:19:31,117][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001091616_279486464.pth [2023-12-26 23:19:31,126][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001091744_279527424.pth... [2023-12-26 23:19:31,128][105692] Updated weights for policy 0, policy_version 1091744 (0.0012) [2023-12-26 23:19:31,130][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001090560_279224320.pth [2023-12-26 23:19:31,754][105692] Updated weights for policy 0, policy_version 1091754 (0.0008) [2023-12-26 23:19:31,798][105620] Updated weights for policy 1, policy_version 1092776 (0.0010) [2023-12-26 23:19:31,820][105692] Updated weights for policy 0, policy_version 1091764 (0.0006) [2023-12-26 23:19:31,857][105620] Updated weights for policy 1, policy_version 1092786 (0.0011) [2023-12-26 23:19:31,884][105692] Updated weights for policy 0, policy_version 1091774 (0.0006) [2023-12-26 23:19:31,923][105620] Updated weights for policy 1, policy_version 1092796 (0.0010) [2023-12-26 23:19:32,488][105692] Updated weights for policy 0, policy_version 1091784 (0.0009) [2023-12-26 23:19:32,551][105692] Updated weights for policy 0, policy_version 1091794 (0.0009) [2023-12-26 23:19:32,607][105692] Updated weights for policy 0, policy_version 1091804 (0.0008) [2023-12-26 23:19:32,638][105620] Updated weights for policy 1, policy_version 1092806 (0.0010) [2023-12-26 23:19:32,689][105620] Updated weights for policy 1, policy_version 1092816 (0.0010) [2023-12-26 23:19:32,740][105620] Updated weights for policy 1, policy_version 1092826 (0.0010) [2023-12-26 23:19:33,234][105692] Updated weights for policy 0, policy_version 1091814 (0.0006) [2023-12-26 23:19:33,285][105692] Updated weights for policy 0, policy_version 1091824 (0.0006) [2023-12-26 23:19:33,348][105692] Updated weights for policy 0, policy_version 1091834 (0.0006) [2023-12-26 23:19:33,357][105620] Updated weights for policy 1, policy_version 1092836 (0.0009) [2023-12-26 23:19:33,413][105620] Updated weights for policy 1, policy_version 1092846 (0.0007) [2023-12-26 23:19:33,478][105620] Updated weights for policy 1, policy_version 1092856 (0.0005) [2023-12-26 23:19:33,942][105692] Updated weights for policy 0, policy_version 1091844 (0.0007) [2023-12-26 23:19:33,996][105692] Updated weights for policy 0, policy_version 1091854 (0.0008) [2023-12-26 23:19:34,030][105620] Updated weights for policy 1, policy_version 1092866 (0.0006) [2023-12-26 23:19:34,050][105692] Updated weights for policy 0, policy_version 1091864 (0.0008) [2023-12-26 23:19:34,095][105620] Updated weights for policy 1, policy_version 1092876 (0.0008) [2023-12-26 23:19:34,158][105620] Updated weights for policy 1, policy_version 1092886 (0.0009) [2023-12-26 23:19:34,227][105620] Updated weights for policy 1, policy_version 1092896 (0.0009) [2023-12-26 23:19:34,747][105692] Updated weights for policy 0, policy_version 1091874 (0.0008) [2023-12-26 23:19:34,792][105692] Updated weights for policy 0, policy_version 1091884 (0.0008) [2023-12-26 23:19:34,844][105692] Updated weights for policy 0, policy_version 1091894 (0.0007) [2023-12-26 23:19:34,888][105692] Updated weights for policy 0, policy_version 1091904 (0.0007) [2023-12-26 23:19:34,957][105620] Updated weights for policy 1, policy_version 1092906 (0.0006) [2023-12-26 23:19:35,010][105620] Updated weights for policy 1, policy_version 1092916 (0.0010) [2023-12-26 23:19:35,056][105620] Updated weights for policy 1, policy_version 1092926 (0.0009) [2023-12-26 23:19:35,595][105692] Updated weights for policy 0, policy_version 1091914 (0.0005) [2023-12-26 23:19:35,657][105692] Updated weights for policy 0, policy_version 1091924 (0.0005) [2023-12-26 23:19:35,722][105692] Updated weights for policy 0, policy_version 1091934 (0.0005) [2023-12-26 23:19:35,743][105620] Updated weights for policy 1, policy_version 1092936 (0.0009) [2023-12-26 23:19:35,811][105620] Updated weights for policy 1, policy_version 1092946 (0.0011) [2023-12-26 23:19:35,871][105620] Updated weights for policy 1, policy_version 1092956 (0.0009) [2023-12-26 23:19:36,062][104569] Fps is (10 sec: 21299.1, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 559407104. Throughput: 0: 10060.4, 1: 9730.0. Samples: 559392948. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:19:36,063][104569] Avg episode reward: [(0, '9083.402'), (1, '9236.211')] [2023-12-26 23:19:36,381][105692] Updated weights for policy 0, policy_version 1091944 (0.0010) [2023-12-26 23:19:36,437][105692] Updated weights for policy 0, policy_version 1091954 (0.0011) [2023-12-26 23:19:36,449][105620] Updated weights for policy 1, policy_version 1092966 (0.0005) [2023-12-26 23:19:36,504][105692] Updated weights for policy 0, policy_version 1091964 (0.0011) [2023-12-26 23:19:36,513][105620] Updated weights for policy 1, policy_version 1092976 (0.0006) [2023-12-26 23:19:36,575][105620] Updated weights for policy 1, policy_version 1092986 (0.0009) [2023-12-26 23:19:37,239][105692] Updated weights for policy 0, policy_version 1091974 (0.0008) [2023-12-26 23:19:37,269][105620] Updated weights for policy 1, policy_version 1092996 (0.0009) [2023-12-26 23:19:37,290][105692] Updated weights for policy 0, policy_version 1091984 (0.0005) [2023-12-26 23:19:37,329][105620] Updated weights for policy 1, policy_version 1093006 (0.0011) [2023-12-26 23:19:37,347][105692] Updated weights for policy 0, policy_version 1091994 (0.0007) [2023-12-26 23:19:37,390][105620] Updated weights for policy 1, policy_version 1093016 (0.0006) [2023-12-26 23:19:37,989][105620] Updated weights for policy 1, policy_version 1093026 (0.0006) [2023-12-26 23:19:38,037][105692] Updated weights for policy 0, policy_version 1092004 (0.0007) [2023-12-26 23:19:38,047][105620] Updated weights for policy 1, policy_version 1093036 (0.0009) [2023-12-26 23:19:38,080][105692] Updated weights for policy 0, policy_version 1092014 (0.0005) [2023-12-26 23:19:38,094][105620] Updated weights for policy 1, policy_version 1093046 (0.0009) [2023-12-26 23:19:38,143][105692] Updated weights for policy 0, policy_version 1092024 (0.0006) [2023-12-26 23:19:38,146][105620] Updated weights for policy 1, policy_version 1093056 (0.0006) [2023-12-26 23:19:38,779][105692] Updated weights for policy 0, policy_version 1092034 (0.0009) [2023-12-26 23:19:38,854][105692] Updated weights for policy 0, policy_version 1092044 (0.0010) [2023-12-26 23:19:38,921][105692] Updated weights for policy 0, policy_version 1092054 (0.0008) [2023-12-26 23:19:38,945][105620] Updated weights for policy 1, policy_version 1093066 (0.0007) [2023-12-26 23:19:38,988][105692] Updated weights for policy 0, policy_version 1092064 (0.0006) [2023-12-26 23:19:39,005][105620] Updated weights for policy 1, policy_version 1093076 (0.0009) [2023-12-26 23:19:39,064][105620] Updated weights for policy 1, policy_version 1093086 (0.0009) [2023-12-26 23:19:39,739][105692] Updated weights for policy 0, policy_version 1092074 (0.0005) [2023-12-26 23:19:39,804][105692] Updated weights for policy 0, policy_version 1092084 (0.0007) [2023-12-26 23:19:39,809][105620] Updated weights for policy 1, policy_version 1093096 (0.0008) [2023-12-26 23:19:39,867][105692] Updated weights for policy 0, policy_version 1092094 (0.0008) [2023-12-26 23:19:39,868][105620] Updated weights for policy 1, policy_version 1093106 (0.0009) [2023-12-26 23:19:39,917][105620] Updated weights for policy 1, policy_version 1093116 (0.0009) [2023-12-26 23:19:40,504][105692] Updated weights for policy 0, policy_version 1092104 (0.0005) [2023-12-26 23:19:40,571][105692] Updated weights for policy 0, policy_version 1092114 (0.0005) [2023-12-26 23:19:40,631][105692] Updated weights for policy 0, policy_version 1092124 (0.0006) [2023-12-26 23:19:40,763][105620] Updated weights for policy 1, policy_version 1093126 (0.0009) [2023-12-26 23:19:40,808][105620] Updated weights for policy 1, policy_version 1093136 (0.0006) [2023-12-26 23:19:40,864][105620] Updated weights for policy 1, policy_version 1093146 (0.0005) [2023-12-26 23:19:41,062][104569] Fps is (10 sec: 21299.6, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 559505408. Throughput: 0: 10027.0, 1: 9780.5. Samples: 559512072. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:19:41,062][104569] Avg episode reward: [(0, '9090.559'), (1, '9328.126')] [2023-12-26 23:19:41,318][105692] Updated weights for policy 0, policy_version 1092134 (0.0007) [2023-12-26 23:19:41,386][105692] Updated weights for policy 0, policy_version 1092144 (0.0009) [2023-12-26 23:19:41,446][105692] Updated weights for policy 0, policy_version 1092154 (0.0009) [2023-12-26 23:19:41,598][105620] Updated weights for policy 1, policy_version 1093156 (0.0007) [2023-12-26 23:19:41,666][105620] Updated weights for policy 1, policy_version 1093166 (0.0007) [2023-12-26 23:19:41,727][105620] Updated weights for policy 1, policy_version 1093176 (0.0009) [2023-12-26 23:19:42,290][105692] Updated weights for policy 0, policy_version 1092164 (0.0009) [2023-12-26 23:19:42,353][105692] Updated weights for policy 0, policy_version 1092174 (0.0007) [2023-12-26 23:19:42,412][105692] Updated weights for policy 0, policy_version 1092184 (0.0009) [2023-12-26 23:19:42,445][105620] Updated weights for policy 1, policy_version 1093186 (0.0009) [2023-12-26 23:19:42,499][105620] Updated weights for policy 1, policy_version 1093196 (0.0009) [2023-12-26 23:19:42,554][105620] Updated weights for policy 1, policy_version 1093206 (0.0010) [2023-12-26 23:19:42,612][105620] Updated weights for policy 1, policy_version 1093216 (0.0010) [2023-12-26 23:19:43,103][105692] Updated weights for policy 0, policy_version 1092194 (0.0009) [2023-12-26 23:19:43,158][105692] Updated weights for policy 0, policy_version 1092204 (0.0010) [2023-12-26 23:19:43,222][105692] Updated weights for policy 0, policy_version 1092214 (0.0007) [2023-12-26 23:19:43,280][105692] Updated weights for policy 0, policy_version 1092224 (0.0009) [2023-12-26 23:19:43,296][105620] Updated weights for policy 1, policy_version 1093226 (0.0006) [2023-12-26 23:19:43,356][105620] Updated weights for policy 1, policy_version 1093236 (0.0007) [2023-12-26 23:19:43,419][105620] Updated weights for policy 1, policy_version 1093246 (0.0009) [2023-12-26 23:19:44,068][105620] Updated weights for policy 1, policy_version 1093256 (0.0008) [2023-12-26 23:19:44,071][105692] Updated weights for policy 0, policy_version 1092234 (0.0007) [2023-12-26 23:19:44,126][105620] Updated weights for policy 1, policy_version 1093266 (0.0007) [2023-12-26 23:19:44,128][105692] Updated weights for policy 0, policy_version 1092244 (0.0006) [2023-12-26 23:19:44,183][105692] Updated weights for policy 0, policy_version 1092254 (0.0006) [2023-12-26 23:19:44,189][105620] Updated weights for policy 1, policy_version 1093276 (0.0007) [2023-12-26 23:19:44,846][105692] Updated weights for policy 0, policy_version 1092264 (0.0008) [2023-12-26 23:19:44,903][105692] Updated weights for policy 0, policy_version 1092274 (0.0007) [2023-12-26 23:19:44,968][105692] Updated weights for policy 0, policy_version 1092284 (0.0006) [2023-12-26 23:19:44,985][105620] Updated weights for policy 1, policy_version 1093286 (0.0008) [2023-12-26 23:19:45,055][105620] Updated weights for policy 1, policy_version 1093296 (0.0009) [2023-12-26 23:19:45,114][105620] Updated weights for policy 1, policy_version 1093306 (0.0009) [2023-12-26 23:19:45,700][105692] Updated weights for policy 0, policy_version 1092294 (0.0007) [2023-12-26 23:19:45,759][105692] Updated weights for policy 0, policy_version 1092304 (0.0009) [2023-12-26 23:19:45,819][105692] Updated weights for policy 0, policy_version 1092314 (0.0008) [2023-12-26 23:19:45,836][105620] Updated weights for policy 1, policy_version 1093316 (0.0008) [2023-12-26 23:19:45,891][105620] Updated weights for policy 1, policy_version 1093326 (0.0007) [2023-12-26 23:19:45,940][105620] Updated weights for policy 1, policy_version 1093336 (0.0008) [2023-12-26 23:19:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19494.2). Total num frames: 559603712. Throughput: 0: 9950.9, 1: 9826.5. Samples: 559569548. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:19:46,062][104569] Avg episode reward: [(0, '9176.434'), (1, '9177.688')] [2023-12-26 23:19:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001092320_279674880.pth... [2023-12-26 23:19:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001093344_279928832.pth... [2023-12-26 23:19:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001092160_279625728.pth [2023-12-26 23:19:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001091136_279371776.pth [2023-12-26 23:19:46,624][105692] Updated weights for policy 0, policy_version 1092324 (0.0008) [2023-12-26 23:19:46,630][105620] Updated weights for policy 1, policy_version 1093346 (0.0008) [2023-12-26 23:19:46,672][105692] Updated weights for policy 0, policy_version 1092334 (0.0008) [2023-12-26 23:19:46,681][105620] Updated weights for policy 1, policy_version 1093356 (0.0005) [2023-12-26 23:19:46,731][105692] Updated weights for policy 0, policy_version 1092344 (0.0006) [2023-12-26 23:19:46,732][105620] Updated weights for policy 1, policy_version 1093366 (0.0005) [2023-12-26 23:19:46,790][105620] Updated weights for policy 1, policy_version 1093376 (0.0006) [2023-12-26 23:19:47,380][105620] Updated weights for policy 1, policy_version 1093386 (0.0010) [2023-12-26 23:19:47,428][105620] Updated weights for policy 1, policy_version 1093396 (0.0010) [2023-12-26 23:19:47,462][105692] Updated weights for policy 0, policy_version 1092354 (0.0006) [2023-12-26 23:19:47,474][105620] Updated weights for policy 1, policy_version 1093406 (0.0010) [2023-12-26 23:19:47,517][105692] Updated weights for policy 0, policy_version 1092364 (0.0006) [2023-12-26 23:19:47,576][105692] Updated weights for policy 0, policy_version 1092374 (0.0008) [2023-12-26 23:19:47,637][105692] Updated weights for policy 0, policy_version 1092384 (0.0009) [2023-12-26 23:19:48,169][105620] Updated weights for policy 1, policy_version 1093416 (0.0008) [2023-12-26 23:19:48,234][105620] Updated weights for policy 1, policy_version 1093426 (0.0005) [2023-12-26 23:19:48,297][105620] Updated weights for policy 1, policy_version 1093436 (0.0005) [2023-12-26 23:19:48,368][105692] Updated weights for policy 0, policy_version 1092394 (0.0007) [2023-12-26 23:19:48,430][105692] Updated weights for policy 0, policy_version 1092404 (0.0009) [2023-12-26 23:19:48,490][105692] Updated weights for policy 0, policy_version 1092415 (0.0011) [2023-12-26 23:19:48,850][105620] Updated weights for policy 1, policy_version 1093446 (0.0006) [2023-12-26 23:19:48,908][105620] Updated weights for policy 1, policy_version 1093456 (0.0005) [2023-12-26 23:19:48,966][105620] Updated weights for policy 1, policy_version 1093466 (0.0009) [2023-12-26 23:19:49,353][105692] Updated weights for policy 0, policy_version 1092425 (0.0010) [2023-12-26 23:19:49,420][105692] Updated weights for policy 0, policy_version 1092435 (0.0008) [2023-12-26 23:19:49,476][105692] Updated weights for policy 0, policy_version 1092445 (0.0009) [2023-12-26 23:19:49,636][105620] Updated weights for policy 1, policy_version 1093476 (0.0011) [2023-12-26 23:19:49,705][105620] Updated weights for policy 1, policy_version 1093486 (0.0011) [2023-12-26 23:19:49,776][105620] Updated weights for policy 1, policy_version 1093496 (0.0011) [2023-12-26 23:19:50,150][105692] Updated weights for policy 0, policy_version 1092455 (0.0007) [2023-12-26 23:19:50,212][105692] Updated weights for policy 0, policy_version 1092465 (0.0008) [2023-12-26 23:19:50,262][105692] Updated weights for policy 0, policy_version 1092475 (0.0006) [2023-12-26 23:19:50,520][105620] Updated weights for policy 1, policy_version 1093506 (0.0011) [2023-12-26 23:19:50,582][105620] Updated weights for policy 1, policy_version 1093516 (0.0010) [2023-12-26 23:19:50,646][105620] Updated weights for policy 1, policy_version 1093526 (0.0009) [2023-12-26 23:19:50,697][105620] Updated weights for policy 1, policy_version 1093536 (0.0008) [2023-12-26 23:19:50,899][105692] Updated weights for policy 0, policy_version 1092485 (0.0009) [2023-12-26 23:19:50,954][105692] Updated weights for policy 0, policy_version 1092495 (0.0009) [2023-12-26 23:19:51,011][105692] Updated weights for policy 0, policy_version 1092505 (0.0010) [2023-12-26 23:19:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19494.2). Total num frames: 559702016. Throughput: 0: 9870.9, 1: 9850.9. Samples: 559686240. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-26 23:19:51,062][104569] Avg episode reward: [(0, '9259.816'), (1, '9259.369')] [2023-12-26 23:19:51,439][105620] Updated weights for policy 1, policy_version 1093546 (0.0010) [2023-12-26 23:19:51,488][105620] Updated weights for policy 1, policy_version 1093556 (0.0009) [2023-12-26 23:19:51,537][105620] Updated weights for policy 1, policy_version 1093566 (0.0009) [2023-12-26 23:19:51,821][105692] Updated weights for policy 0, policy_version 1092515 (0.0009) [2023-12-26 23:19:51,881][105692] Updated weights for policy 0, policy_version 1092525 (0.0009) [2023-12-26 23:19:51,946][105692] Updated weights for policy 0, policy_version 1092535 (0.0008) [2023-12-26 23:19:52,242][105620] Updated weights for policy 1, policy_version 1093576 (0.0007) [2023-12-26 23:19:52,307][105620] Updated weights for policy 1, policy_version 1093586 (0.0007) [2023-12-26 23:19:52,380][105620] Updated weights for policy 1, policy_version 1093596 (0.0009) [2023-12-26 23:19:52,676][105692] Updated weights for policy 0, policy_version 1092545 (0.0010) [2023-12-26 23:19:52,739][105692] Updated weights for policy 0, policy_version 1092555 (0.0009) [2023-12-26 23:19:52,794][105692] Updated weights for policy 0, policy_version 1092565 (0.0009) [2023-12-26 23:19:52,854][105692] Updated weights for policy 0, policy_version 1092575 (0.0009) [2023-12-26 23:19:52,999][105620] Updated weights for policy 1, policy_version 1093606 (0.0008) [2023-12-26 23:19:53,058][105620] Updated weights for policy 1, policy_version 1093616 (0.0008) [2023-12-26 23:19:53,117][105620] Updated weights for policy 1, policy_version 1093626 (0.0005) [2023-12-26 23:19:53,624][105692] Updated weights for policy 0, policy_version 1092585 (0.0009) [2023-12-26 23:19:53,675][105692] Updated weights for policy 0, policy_version 1092595 (0.0007) [2023-12-26 23:19:53,696][105620] Updated weights for policy 1, policy_version 1093636 (0.0005) [2023-12-26 23:19:53,732][105692] Updated weights for policy 0, policy_version 1092605 (0.0009) [2023-12-26 23:19:53,749][105620] Updated weights for policy 1, policy_version 1093646 (0.0005) [2023-12-26 23:19:53,808][105620] Updated weights for policy 1, policy_version 1093656 (0.0005) [2023-12-26 23:19:54,349][105620] Updated weights for policy 1, policy_version 1093666 (0.0005) [2023-12-26 23:19:54,412][105620] Updated weights for policy 1, policy_version 1093676 (0.0005) [2023-12-26 23:19:54,453][105692] Updated weights for policy 0, policy_version 1092615 (0.0009) [2023-12-26 23:19:54,480][105620] Updated weights for policy 1, policy_version 1093686 (0.0007) [2023-12-26 23:19:54,518][105692] Updated weights for policy 0, policy_version 1092625 (0.0008) [2023-12-26 23:19:54,539][105620] Updated weights for policy 1, policy_version 1093696 (0.0007) [2023-12-26 23:19:54,579][105692] Updated weights for policy 0, policy_version 1092635 (0.0008) [2023-12-26 23:19:55,234][105620] Updated weights for policy 1, policy_version 1093706 (0.0007) [2023-12-26 23:19:55,284][105692] Updated weights for policy 0, policy_version 1092645 (0.0008) [2023-12-26 23:19:55,286][105620] Updated weights for policy 1, policy_version 1093716 (0.0008) [2023-12-26 23:19:55,334][105620] Updated weights for policy 1, policy_version 1093726 (0.0006) [2023-12-26 23:19:55,336][105692] Updated weights for policy 0, policy_version 1092655 (0.0006) [2023-12-26 23:19:55,397][105692] Updated weights for policy 0, policy_version 1092665 (0.0008) [2023-12-26 23:19:55,990][105620] Updated weights for policy 1, policy_version 1093736 (0.0005) [2023-12-26 23:19:56,051][105620] Updated weights for policy 1, policy_version 1093746 (0.0005) [2023-12-26 23:19:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 559792128. Throughput: 0: 9802.6, 1: 10039.0. Samples: 559806292. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:19:56,062][104569] Avg episode reward: [(0, '9177.008'), (1, '8985.070')] [2023-12-26 23:19:56,116][105692] Updated weights for policy 0, policy_version 1092675 (0.0009) [2023-12-26 23:19:56,116][105620] Updated weights for policy 1, policy_version 1093756 (0.0005) [2023-12-26 23:19:56,177][105692] Updated weights for policy 0, policy_version 1092685 (0.0010) [2023-12-26 23:19:56,227][105692] Updated weights for policy 0, policy_version 1092695 (0.0009) [2023-12-26 23:19:56,619][105620] Updated weights for policy 1, policy_version 1093766 (0.0005) [2023-12-26 23:19:56,670][105620] Updated weights for policy 1, policy_version 1093776 (0.0005) [2023-12-26 23:19:56,719][105620] Updated weights for policy 1, policy_version 1093786 (0.0005) [2023-12-26 23:19:56,944][105692] Updated weights for policy 0, policy_version 1092705 (0.0008) [2023-12-26 23:19:56,998][105692] Updated weights for policy 0, policy_version 1092715 (0.0006) [2023-12-26 23:19:57,048][105692] Updated weights for policy 0, policy_version 1092725 (0.0005) [2023-12-26 23:19:57,095][105692] Updated weights for policy 0, policy_version 1092735 (0.0005) [2023-12-26 23:19:57,306][105620] Updated weights for policy 1, policy_version 1093796 (0.0006) [2023-12-26 23:19:57,363][105620] Updated weights for policy 1, policy_version 1093806 (0.0006) [2023-12-26 23:19:57,408][105620] Updated weights for policy 1, policy_version 1093816 (0.0005) [2023-12-26 23:19:57,768][105692] Updated weights for policy 0, policy_version 1092745 (0.0010) [2023-12-26 23:19:57,843][105692] Updated weights for policy 0, policy_version 1092755 (0.0010) [2023-12-26 23:19:57,911][105692] Updated weights for policy 0, policy_version 1092765 (0.0010) [2023-12-26 23:19:57,935][105620] Updated weights for policy 1, policy_version 1093826 (0.0005) [2023-12-26 23:19:57,987][105620] Updated weights for policy 1, policy_version 1093836 (0.0010) [2023-12-26 23:19:58,052][105620] Updated weights for policy 1, policy_version 1093846 (0.0010) [2023-12-26 23:19:58,120][105620] Updated weights for policy 1, policy_version 1093856 (0.0010) [2023-12-26 23:19:58,621][105692] Updated weights for policy 0, policy_version 1092775 (0.0009) [2023-12-26 23:19:58,668][105585] KL-divergence is very high: 100.0569 [2023-12-26 23:19:58,689][105692] Updated weights for policy 0, policy_version 1092785 (0.0008) [2023-12-26 23:19:58,726][105585] KL-divergence is very high: 111.4067 [2023-12-26 23:19:58,758][105692] Updated weights for policy 0, policy_version 1092795 (0.0007) [2023-12-26 23:19:58,780][105585] KL-divergence is very high: 104.4999 [2023-12-26 23:19:58,877][105620] Updated weights for policy 1, policy_version 1093866 (0.0010) [2023-12-26 23:19:58,953][105620] Updated weights for policy 1, policy_version 1093877 (0.0009) [2023-12-26 23:19:59,016][105620] Updated weights for policy 1, policy_version 1093887 (0.0009) [2023-12-26 23:19:59,542][105692] Updated weights for policy 0, policy_version 1092805 (0.0011) [2023-12-26 23:19:59,607][105692] Updated weights for policy 0, policy_version 1092815 (0.0008) [2023-12-26 23:19:59,672][105692] Updated weights for policy 0, policy_version 1092825 (0.0010) [2023-12-26 23:19:59,794][105620] Updated weights for policy 1, policy_version 1093897 (0.0009) [2023-12-26 23:19:59,856][105620] Updated weights for policy 1, policy_version 1093907 (0.0008) [2023-12-26 23:19:59,923][105620] Updated weights for policy 1, policy_version 1093917 (0.0007) [2023-12-26 23:20:00,352][105692] Updated weights for policy 0, policy_version 1092835 (0.0010) [2023-12-26 23:20:00,408][105692] Updated weights for policy 0, policy_version 1092845 (0.0006) [2023-12-26 23:20:00,462][105692] Updated weights for policy 0, policy_version 1092855 (0.0006) [2023-12-26 23:20:00,716][105620] Updated weights for policy 1, policy_version 1093927 (0.0009) [2023-12-26 23:20:00,766][105620] Updated weights for policy 1, policy_version 1093937 (0.0009) [2023-12-26 23:20:00,816][105620] Updated weights for policy 1, policy_version 1093947 (0.0008) [2023-12-26 23:20:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 559898624. Throughput: 0: 9867.9, 1: 10159.0. Samples: 559870464. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:20:01,063][104569] Avg episode reward: [(0, '6459.810'), (1, '8986.551')] [2023-12-26 23:20:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001093952_280084480.pth... [2023-12-26 23:20:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001092768_279781376.pth [2023-12-26 23:20:01,072][105692] Updated weights for policy 0, policy_version 1092865 (0.0006) [2023-12-26 23:20:01,125][105692] Updated weights for policy 0, policy_version 1092875 (0.0005) [2023-12-26 23:20:01,187][105692] Updated weights for policy 0, policy_version 1092885 (0.0008) [2023-12-26 23:20:01,255][105692] Updated weights for policy 0, policy_version 1092895 (0.0008) [2023-12-26 23:20:01,262][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001092896_279822336.pth... [2023-12-26 23:20:01,268][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001091744_279527424.pth [2023-12-26 23:20:01,585][105620] Updated weights for policy 1, policy_version 1093957 (0.0010) [2023-12-26 23:20:01,645][105620] Updated weights for policy 1, policy_version 1093967 (0.0009) [2023-12-26 23:20:01,696][105620] Updated weights for policy 1, policy_version 1093977 (0.0008) [2023-12-26 23:20:01,878][105692] Updated weights for policy 0, policy_version 1092905 (0.0009) [2023-12-26 23:20:01,931][105692] Updated weights for policy 0, policy_version 1092915 (0.0009) [2023-12-26 23:20:01,983][105692] Updated weights for policy 0, policy_version 1092925 (0.0009) [2023-12-26 23:20:02,451][105620] Updated weights for policy 1, policy_version 1093987 (0.0007) [2023-12-26 23:20:02,504][105620] Updated weights for policy 1, policy_version 1093997 (0.0009) [2023-12-26 23:20:02,555][105620] Updated weights for policy 1, policy_version 1094007 (0.0009) [2023-12-26 23:20:02,727][105692] Updated weights for policy 0, policy_version 1092935 (0.0007) [2023-12-26 23:20:02,790][105692] Updated weights for policy 0, policy_version 1092945 (0.0006) [2023-12-26 23:20:02,851][105692] Updated weights for policy 0, policy_version 1092955 (0.0006) [2023-12-26 23:20:03,308][105620] Updated weights for policy 1, policy_version 1094017 (0.0012) [2023-12-26 23:20:03,356][105620] Updated weights for policy 1, policy_version 1094027 (0.0009) [2023-12-26 23:20:03,402][105620] Updated weights for policy 1, policy_version 1094037 (0.0008) [2023-12-26 23:20:03,449][105620] Updated weights for policy 1, policy_version 1094047 (0.0008) [2023-12-26 23:20:03,503][105692] Updated weights for policy 0, policy_version 1092965 (0.0010) [2023-12-26 23:20:03,555][105692] Updated weights for policy 0, policy_version 1092975 (0.0010) [2023-12-26 23:20:03,609][105692] Updated weights for policy 0, policy_version 1092985 (0.0010) [2023-12-26 23:20:04,198][105620] Updated weights for policy 1, policy_version 1094057 (0.0006) [2023-12-26 23:20:04,267][105620] Updated weights for policy 1, policy_version 1094067 (0.0006) [2023-12-26 23:20:04,315][105620] Updated weights for policy 1, policy_version 1094077 (0.0007) [2023-12-26 23:20:04,347][105692] Updated weights for policy 0, policy_version 1092995 (0.0009) [2023-12-26 23:20:04,408][105692] Updated weights for policy 0, policy_version 1093005 (0.0010) [2023-12-26 23:20:04,471][105692] Updated weights for policy 0, policy_version 1093015 (0.0011) [2023-12-26 23:20:05,020][105620] Updated weights for policy 1, policy_version 1094087 (0.0005) [2023-12-26 23:20:05,056][105692] Updated weights for policy 0, policy_version 1093025 (0.0010) [2023-12-26 23:20:05,084][105620] Updated weights for policy 1, policy_version 1094097 (0.0005) [2023-12-26 23:20:05,114][105692] Updated weights for policy 0, policy_version 1093035 (0.0009) [2023-12-26 23:20:05,141][105620] Updated weights for policy 1, policy_version 1094107 (0.0006) [2023-12-26 23:20:05,163][105692] Updated weights for policy 0, policy_version 1093045 (0.0010) [2023-12-26 23:20:05,223][105692] Updated weights for policy 0, policy_version 1093055 (0.0011) [2023-12-26 23:20:05,739][105620] Updated weights for policy 1, policy_version 1094117 (0.0007) [2023-12-26 23:20:05,796][105620] Updated weights for policy 1, policy_version 1094127 (0.0009) [2023-12-26 23:20:05,833][105692] Updated weights for policy 0, policy_version 1093065 (0.0006) [2023-12-26 23:20:05,859][105585] KL-divergence is very high: 123.4576 [2023-12-26 23:20:05,859][105620] Updated weights for policy 1, policy_version 1094137 (0.0008) [2023-12-26 23:20:05,897][105692] Updated weights for policy 0, policy_version 1093075 (0.0005) [2023-12-26 23:20:05,906][105585] KL-divergence is very high: 214.6184 [2023-12-26 23:20:05,947][105692] Updated weights for policy 0, policy_version 1093085 (0.0005) [2023-12-26 23:20:05,949][105585] KL-divergence is very high: 233.1063 [2023-12-26 23:20:06,062][104569] Fps is (10 sec: 21299.3, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 560005120. Throughput: 0: 9890.1, 1: 10124.1. Samples: 559985664. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:20:06,062][104569] Avg episode reward: [(0, '7644.127'), (1, '9168.773')] [2023-12-26 23:20:06,603][105692] Updated weights for policy 0, policy_version 1093095 (0.0011) [2023-12-26 23:20:06,636][105620] Updated weights for policy 1, policy_version 1094147 (0.0007) [2023-12-26 23:20:06,663][105692] Updated weights for policy 0, policy_version 1093105 (0.0010) [2023-12-26 23:20:06,689][105620] Updated weights for policy 1, policy_version 1094157 (0.0005) [2023-12-26 23:20:06,718][105692] Updated weights for policy 0, policy_version 1093115 (0.0010) [2023-12-26 23:20:06,752][105620] Updated weights for policy 1, policy_version 1094167 (0.0006) [2023-12-26 23:20:07,458][105692] Updated weights for policy 0, policy_version 1093125 (0.0010) [2023-12-26 23:20:07,513][105692] Updated weights for policy 0, policy_version 1093135 (0.0010) [2023-12-26 23:20:07,526][105620] Updated weights for policy 1, policy_version 1094177 (0.0007) [2023-12-26 23:20:07,569][105692] Updated weights for policy 0, policy_version 1093145 (0.0010) [2023-12-26 23:20:07,579][105620] Updated weights for policy 1, policy_version 1094187 (0.0006) [2023-12-26 23:20:07,637][105620] Updated weights for policy 1, policy_version 1094197 (0.0008) [2023-12-26 23:20:07,686][105620] Updated weights for policy 1, policy_version 1094207 (0.0008) [2023-12-26 23:20:08,177][105692] Updated weights for policy 0, policy_version 1093155 (0.0010) [2023-12-26 23:20:08,238][105692] Updated weights for policy 0, policy_version 1093165 (0.0010) [2023-12-26 23:20:08,286][105692] Updated weights for policy 0, policy_version 1093175 (0.0010) [2023-12-26 23:20:08,456][105620] Updated weights for policy 1, policy_version 1094217 (0.0008) [2023-12-26 23:20:08,509][105620] Updated weights for policy 1, policy_version 1094227 (0.0008) [2023-12-26 23:20:08,557][105620] Updated weights for policy 1, policy_version 1094237 (0.0008) [2023-12-26 23:20:09,050][105692] Updated weights for policy 0, policy_version 1093185 (0.0011) [2023-12-26 23:20:09,097][105692] Updated weights for policy 0, policy_version 1093195 (0.0009) [2023-12-26 23:20:09,144][105692] Updated weights for policy 0, policy_version 1093205 (0.0009) [2023-12-26 23:20:09,191][105692] Updated weights for policy 0, policy_version 1093215 (0.0010) [2023-12-26 23:20:09,333][105620] Updated weights for policy 1, policy_version 1094247 (0.0009) [2023-12-26 23:20:09,394][105620] Updated weights for policy 1, policy_version 1094257 (0.0009) [2023-12-26 23:20:09,454][105620] Updated weights for policy 1, policy_version 1094267 (0.0009) [2023-12-26 23:20:10,048][105692] Updated weights for policy 0, policy_version 1093225 (0.0008) [2023-12-26 23:20:10,112][105692] Updated weights for policy 0, policy_version 1093235 (0.0008) [2023-12-26 23:20:10,178][105692] Updated weights for policy 0, policy_version 1093245 (0.0009) [2023-12-26 23:20:10,219][105620] Updated weights for policy 1, policy_version 1094277 (0.0010) [2023-12-26 23:20:10,284][105620] Updated weights for policy 1, policy_version 1094287 (0.0010) [2023-12-26 23:20:10,340][105620] Updated weights for policy 1, policy_version 1094297 (0.0009) [2023-12-26 23:20:10,924][105692] Updated weights for policy 0, policy_version 1093255 (0.0007) [2023-12-26 23:20:10,968][105620] Updated weights for policy 1, policy_version 1094307 (0.0010) [2023-12-26 23:20:10,982][105692] Updated weights for policy 0, policy_version 1093265 (0.0008) [2023-12-26 23:20:11,029][105620] Updated weights for policy 1, policy_version 1094317 (0.0009) [2023-12-26 23:20:11,037][105692] Updated weights for policy 0, policy_version 1093275 (0.0009) [2023-12-26 23:20:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19797.4, 300 sec: 19494.2). Total num frames: 560087040. Throughput: 0: 9924.7, 1: 10095.0. Samples: 560103416. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:20:11,062][104569] Avg episode reward: [(0, '9173.348'), (1, '9350.866')] [2023-12-26 23:20:11,091][105620] Updated weights for policy 1, policy_version 1094327 (0.0009) [2023-12-26 23:20:11,743][105692] Updated weights for policy 0, policy_version 1093285 (0.0007) [2023-12-26 23:20:11,809][105692] Updated weights for policy 0, policy_version 1093295 (0.0006) [2023-12-26 23:20:11,880][105692] Updated weights for policy 0, policy_version 1093305 (0.0006) [2023-12-26 23:20:11,899][105620] Updated weights for policy 1, policy_version 1094337 (0.0010) [2023-12-26 23:20:11,954][105620] Updated weights for policy 1, policy_version 1094347 (0.0010) [2023-12-26 23:20:12,007][105620] Updated weights for policy 1, policy_version 1094357 (0.0010) [2023-12-26 23:20:12,056][105620] Updated weights for policy 1, policy_version 1094367 (0.0010) [2023-12-26 23:20:12,618][105692] Updated weights for policy 0, policy_version 1093315 (0.0008) [2023-12-26 23:20:12,676][105692] Updated weights for policy 0, policy_version 1093325 (0.0010) [2023-12-26 23:20:12,721][105692] Updated weights for policy 0, policy_version 1093335 (0.0007) [2023-12-26 23:20:12,723][105620] Updated weights for policy 1, policy_version 1094377 (0.0010) [2023-12-26 23:20:12,774][105620] Updated weights for policy 1, policy_version 1094387 (0.0010) [2023-12-26 23:20:12,832][105620] Updated weights for policy 1, policy_version 1094397 (0.0010) [2023-12-26 23:20:13,522][105692] Updated weights for policy 0, policy_version 1093345 (0.0007) [2023-12-26 23:20:13,523][105620] Updated weights for policy 1, policy_version 1094407 (0.0009) [2023-12-26 23:20:13,573][105620] Updated weights for policy 1, policy_version 1094417 (0.0007) [2023-12-26 23:20:13,584][105692] Updated weights for policy 0, policy_version 1093355 (0.0007) [2023-12-26 23:20:13,623][105620] Updated weights for policy 1, policy_version 1094427 (0.0006) [2023-12-26 23:20:13,640][105692] Updated weights for policy 0, policy_version 1093365 (0.0008) [2023-12-26 23:20:13,689][105692] Updated weights for policy 0, policy_version 1093375 (0.0008) [2023-12-26 23:20:14,232][105620] Updated weights for policy 1, policy_version 1094437 (0.0006) [2023-12-26 23:20:14,297][105620] Updated weights for policy 1, policy_version 1094447 (0.0009) [2023-12-26 23:20:14,364][105620] Updated weights for policy 1, policy_version 1094457 (0.0011) [2023-12-26 23:20:14,487][105692] Updated weights for policy 0, policy_version 1093385 (0.0008) [2023-12-26 23:20:14,542][105692] Updated weights for policy 0, policy_version 1093395 (0.0008) [2023-12-26 23:20:14,602][105692] Updated weights for policy 0, policy_version 1093405 (0.0008) [2023-12-26 23:20:15,037][105620] Updated weights for policy 1, policy_version 1094467 (0.0009) [2023-12-26 23:20:15,108][105620] Updated weights for policy 1, policy_version 1094477 (0.0006) [2023-12-26 23:20:15,182][105620] Updated weights for policy 1, policy_version 1094487 (0.0007) [2023-12-26 23:20:15,315][105692] Updated weights for policy 0, policy_version 1093415 (0.0008) [2023-12-26 23:20:15,385][105692] Updated weights for policy 0, policy_version 1093425 (0.0006) [2023-12-26 23:20:15,445][105692] Updated weights for policy 0, policy_version 1093435 (0.0006) [2023-12-26 23:20:15,814][105620] Updated weights for policy 1, policy_version 1094497 (0.0007) [2023-12-26 23:20:15,866][105620] Updated weights for policy 1, policy_version 1094507 (0.0008) [2023-12-26 23:20:15,923][105620] Updated weights for policy 1, policy_version 1094517 (0.0007) [2023-12-26 23:20:15,974][105620] Updated weights for policy 1, policy_version 1094527 (0.0005) [2023-12-26 23:20:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19933.9, 300 sec: 19494.2). Total num frames: 560193536. Throughput: 0: 9825.4, 1: 10031.7. Samples: 560161456. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:20:16,062][104569] Avg episode reward: [(0, '9172.303'), (1, '9351.153')] [2023-12-26 23:20:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001093440_279961600.pth... [2023-12-26 23:20:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001094528_280231936.pth... [2023-12-26 23:20:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001092320_279674880.pth [2023-12-26 23:20:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001093344_279928832.pth [2023-12-26 23:20:16,205][105692] Updated weights for policy 0, policy_version 1093445 (0.0007) [2023-12-26 23:20:16,254][105692] Updated weights for policy 0, policy_version 1093455 (0.0008) [2023-12-26 23:20:16,302][105692] Updated weights for policy 0, policy_version 1093465 (0.0008) [2023-12-26 23:20:16,683][105620] Updated weights for policy 1, policy_version 1094537 (0.0010) [2023-12-26 23:20:16,748][105620] Updated weights for policy 1, policy_version 1094547 (0.0006) [2023-12-26 23:20:16,816][105620] Updated weights for policy 1, policy_version 1094557 (0.0007) [2023-12-26 23:20:17,148][105692] Updated weights for policy 0, policy_version 1093475 (0.0008) [2023-12-26 23:20:17,207][105692] Updated weights for policy 0, policy_version 1093485 (0.0009) [2023-12-26 23:20:17,263][105692] Updated weights for policy 0, policy_version 1093495 (0.0009) [2023-12-26 23:20:17,374][105620] Updated weights for policy 1, policy_version 1094567 (0.0006) [2023-12-26 23:20:17,423][105620] Updated weights for policy 1, policy_version 1094577 (0.0005) [2023-12-26 23:20:17,477][105620] Updated weights for policy 1, policy_version 1094587 (0.0009) [2023-12-26 23:20:18,077][105620] Updated weights for policy 1, policy_version 1094597 (0.0008) [2023-12-26 23:20:18,120][105692] Updated weights for policy 0, policy_version 1093506 (0.0010) [2023-12-26 23:20:18,133][105620] Updated weights for policy 1, policy_version 1094607 (0.0005) [2023-12-26 23:20:18,174][105692] Updated weights for policy 0, policy_version 1093516 (0.0009) [2023-12-26 23:20:18,197][105620] Updated weights for policy 1, policy_version 1094617 (0.0010) [2023-12-26 23:20:18,219][105692] Updated weights for policy 0, policy_version 1093526 (0.0006) [2023-12-26 23:20:18,274][105692] Updated weights for policy 0, policy_version 1093536 (0.0009) [2023-12-26 23:20:18,880][105620] Updated weights for policy 1, policy_version 1094627 (0.0008) [2023-12-26 23:20:18,930][105620] Updated weights for policy 1, policy_version 1094637 (0.0005) [2023-12-26 23:20:18,981][105620] Updated weights for policy 1, policy_version 1094647 (0.0005) [2023-12-26 23:20:19,088][105692] Updated weights for policy 0, policy_version 1093548 (0.0010) [2023-12-26 23:20:19,142][105692] Updated weights for policy 0, policy_version 1093560 (0.0010) [2023-12-26 23:20:19,670][105620] Updated weights for policy 1, policy_version 1094657 (0.0006) [2023-12-26 23:20:19,732][105620] Updated weights for policy 1, policy_version 1094667 (0.0008) [2023-12-26 23:20:19,793][105620] Updated weights for policy 1, policy_version 1094677 (0.0006) [2023-12-26 23:20:19,868][105620] Updated weights for policy 1, policy_version 1094687 (0.0008) [2023-12-26 23:20:19,982][105692] Updated weights for policy 0, policy_version 1093571 (0.0009) [2023-12-26 23:20:20,035][105692] Updated weights for policy 0, policy_version 1093581 (0.0008) [2023-12-26 23:20:20,095][105692] Updated weights for policy 0, policy_version 1093591 (0.0008) [2023-12-26 23:20:20,611][105620] Updated weights for policy 1, policy_version 1094697 (0.0007) [2023-12-26 23:20:20,672][105620] Updated weights for policy 1, policy_version 1094707 (0.0008) [2023-12-26 23:20:20,745][105620] Updated weights for policy 1, policy_version 1094717 (0.0011) [2023-12-26 23:20:20,862][105692] Updated weights for policy 0, policy_version 1093601 (0.0008) [2023-12-26 23:20:20,926][105692] Updated weights for policy 0, policy_version 1093611 (0.0008) [2023-12-26 23:20:20,985][105692] Updated weights for policy 0, policy_version 1093621 (0.0008) [2023-12-26 23:20:21,043][105692] Updated weights for policy 0, policy_version 1093631 (0.0008) [2023-12-26 23:20:21,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19933.8, 300 sec: 19522.0). Total num frames: 560291840. Throughput: 0: 9587.6, 1: 10063.2. Samples: 560277236. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:20:21,063][104569] Avg episode reward: [(0, '9262.765'), (1, '9168.728')] [2023-12-26 23:20:21,477][105620] Updated weights for policy 1, policy_version 1094727 (0.0010) [2023-12-26 23:20:21,535][105620] Updated weights for policy 1, policy_version 1094737 (0.0009) [2023-12-26 23:20:21,591][105620] Updated weights for policy 1, policy_version 1094747 (0.0008) [2023-12-26 23:20:21,806][105692] Updated weights for policy 0, policy_version 1093641 (0.0007) [2023-12-26 23:20:21,866][105692] Updated weights for policy 0, policy_version 1093651 (0.0006) [2023-12-26 23:20:21,925][105692] Updated weights for policy 0, policy_version 1093661 (0.0006) [2023-12-26 23:20:22,411][105620] Updated weights for policy 1, policy_version 1094757 (0.0010) [2023-12-26 23:20:22,477][105620] Updated weights for policy 1, policy_version 1094767 (0.0011) [2023-12-26 23:20:22,541][105620] Updated weights for policy 1, policy_version 1094777 (0.0011) [2023-12-26 23:20:22,565][105692] Updated weights for policy 0, policy_version 1093671 (0.0007) [2023-12-26 23:20:22,625][105692] Updated weights for policy 0, policy_version 1093681 (0.0005) [2023-12-26 23:20:22,689][105692] Updated weights for policy 0, policy_version 1093691 (0.0005) [2023-12-26 23:20:23,189][105620] Updated weights for policy 1, policy_version 1094787 (0.0009) [2023-12-26 23:20:23,246][105620] Updated weights for policy 1, policy_version 1094797 (0.0007) [2023-12-26 23:20:23,308][105620] Updated weights for policy 1, policy_version 1094807 (0.0009) [2023-12-26 23:20:23,331][105692] Updated weights for policy 0, policy_version 1093701 (0.0007) [2023-12-26 23:20:23,386][105692] Updated weights for policy 0, policy_version 1093711 (0.0005) [2023-12-26 23:20:23,434][105692] Updated weights for policy 0, policy_version 1093721 (0.0005) [2023-12-26 23:20:24,049][105620] Updated weights for policy 1, policy_version 1094817 (0.0006) [2023-12-26 23:20:24,091][105692] Updated weights for policy 0, policy_version 1093731 (0.0005) [2023-12-26 23:20:24,097][105620] Updated weights for policy 1, policy_version 1094827 (0.0009) [2023-12-26 23:20:24,147][105692] Updated weights for policy 0, policy_version 1093741 (0.0006) [2023-12-26 23:20:24,167][105620] Updated weights for policy 1, policy_version 1094837 (0.0007) [2023-12-26 23:20:24,197][105692] Updated weights for policy 0, policy_version 1093751 (0.0007) [2023-12-26 23:20:24,216][105620] Updated weights for policy 1, policy_version 1094847 (0.0006) [2023-12-26 23:20:24,893][105692] Updated weights for policy 0, policy_version 1093761 (0.0007) [2023-12-26 23:20:24,950][105692] Updated weights for policy 0, policy_version 1093771 (0.0007) [2023-12-26 23:20:25,009][105692] Updated weights for policy 0, policy_version 1093781 (0.0011) [2023-12-26 23:20:25,016][105620] Updated weights for policy 1, policy_version 1094857 (0.0008) [2023-12-26 23:20:25,061][105692] Updated weights for policy 0, policy_version 1093791 (0.0010) [2023-12-26 23:20:25,063][105620] Updated weights for policy 1, policy_version 1094867 (0.0005) [2023-12-26 23:20:25,118][105620] Updated weights for policy 1, policy_version 1094877 (0.0008) [2023-12-26 23:20:25,727][105692] Updated weights for policy 0, policy_version 1093801 (0.0008) [2023-12-26 23:20:25,789][105692] Updated weights for policy 0, policy_version 1093811 (0.0006) [2023-12-26 23:20:25,852][105692] Updated weights for policy 0, policy_version 1093821 (0.0005) [2023-12-26 23:20:25,926][105620] Updated weights for policy 1, policy_version 1094888 (0.0009) [2023-12-26 23:20:25,985][105620] Updated weights for policy 1, policy_version 1094898 (0.0009) [2023-12-26 23:20:26,043][105620] Updated weights for policy 1, policy_version 1094908 (0.0010) [2023-12-26 23:20:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19933.9, 300 sec: 19522.0). Total num frames: 560390144. Throughput: 0: 9578.7, 1: 9983.5. Samples: 560392376. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:20:26,063][104569] Avg episode reward: [(0, '9352.740'), (1, '9168.963')] [2023-12-26 23:20:26,420][105692] Updated weights for policy 0, policy_version 1093831 (0.0008) [2023-12-26 23:20:26,479][105692] Updated weights for policy 0, policy_version 1093841 (0.0009) [2023-12-26 23:20:26,537][105692] Updated weights for policy 0, policy_version 1093851 (0.0008) [2023-12-26 23:20:26,859][105620] Updated weights for policy 1, policy_version 1094919 (0.0007) [2023-12-26 23:20:26,905][105620] Updated weights for policy 1, policy_version 1094929 (0.0005) [2023-12-26 23:20:26,953][105620] Updated weights for policy 1, policy_version 1094939 (0.0005) [2023-12-26 23:20:27,247][105692] Updated weights for policy 0, policy_version 1093861 (0.0007) [2023-12-26 23:20:27,308][105692] Updated weights for policy 0, policy_version 1093871 (0.0006) [2023-12-26 23:20:27,368][105692] Updated weights for policy 0, policy_version 1093881 (0.0010) [2023-12-26 23:20:27,579][105620] Updated weights for policy 1, policy_version 1094949 (0.0007) [2023-12-26 23:20:27,640][105620] Updated weights for policy 1, policy_version 1094959 (0.0009) [2023-12-26 23:20:27,693][105620] Updated weights for policy 1, policy_version 1094969 (0.0009) [2023-12-26 23:20:27,991][105692] Updated weights for policy 0, policy_version 1093892 (0.0009) [2023-12-26 23:20:28,039][105692] Updated weights for policy 0, policy_version 1093902 (0.0009) [2023-12-26 23:20:28,086][105692] Updated weights for policy 0, policy_version 1093912 (0.0008) [2023-12-26 23:20:28,484][105620] Updated weights for policy 1, policy_version 1094979 (0.0009) [2023-12-26 23:20:28,538][105620] Updated weights for policy 1, policy_version 1094989 (0.0010) [2023-12-26 23:20:28,595][105620] Updated weights for policy 1, policy_version 1094999 (0.0009) [2023-12-26 23:20:28,785][105692] Updated weights for policy 0, policy_version 1093922 (0.0009) [2023-12-26 23:20:28,833][105692] Updated weights for policy 0, policy_version 1093932 (0.0008) [2023-12-26 23:20:28,880][105692] Updated weights for policy 0, policy_version 1093942 (0.0009) [2023-12-26 23:20:28,934][105692] Updated weights for policy 0, policy_version 1093952 (0.0009) [2023-12-26 23:20:29,437][105620] Updated weights for policy 1, policy_version 1095009 (0.0008) [2023-12-26 23:20:29,489][105620] Updated weights for policy 1, policy_version 1095019 (0.0009) [2023-12-26 23:20:29,541][105620] Updated weights for policy 1, policy_version 1095029 (0.0009) [2023-12-26 23:20:29,602][105620] Updated weights for policy 1, policy_version 1095039 (0.0008) [2023-12-26 23:20:29,622][105692] Updated weights for policy 0, policy_version 1093962 (0.0007) [2023-12-26 23:20:29,684][105692] Updated weights for policy 0, policy_version 1093972 (0.0009) [2023-12-26 23:20:29,742][105692] Updated weights for policy 0, policy_version 1093982 (0.0008) [2023-12-26 23:20:30,287][105620] Updated weights for policy 1, policy_version 1095049 (0.0009) [2023-12-26 23:20:30,352][105620] Updated weights for policy 1, policy_version 1095059 (0.0007) [2023-12-26 23:20:30,422][105620] Updated weights for policy 1, policy_version 1095069 (0.0009) [2023-12-26 23:20:30,517][105692] Updated weights for policy 0, policy_version 1093992 (0.0006) [2023-12-26 23:20:30,585][105692] Updated weights for policy 0, policy_version 1094002 (0.0005) [2023-12-26 23:20:30,650][105692] Updated weights for policy 0, policy_version 1094012 (0.0009) [2023-12-26 23:20:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 560480256. Throughput: 0: 9674.5, 1: 9948.1. Samples: 560452564. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:20:31,063][104569] Avg episode reward: [(0, '9263.126'), (1, '9351.793')] [2023-12-26 23:20:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001094016_280109056.pth... [2023-12-26 23:20:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001095072_280371200.pth... [2023-12-26 23:20:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001093952_280084480.pth [2023-12-26 23:20:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001092896_279822336.pth [2023-12-26 23:20:31,175][105620] Updated weights for policy 1, policy_version 1095079 (0.0010) [2023-12-26 23:20:31,236][105620] Updated weights for policy 1, policy_version 1095090 (0.0007) [2023-12-26 23:20:31,288][105620] Updated weights for policy 1, policy_version 1095100 (0.0009) [2023-12-26 23:20:31,384][105692] Updated weights for policy 0, policy_version 1094022 (0.0007) [2023-12-26 23:20:31,443][105692] Updated weights for policy 0, policy_version 1094032 (0.0005) [2023-12-26 23:20:31,501][105692] Updated weights for policy 0, policy_version 1094042 (0.0005) [2023-12-26 23:20:32,120][105692] Updated weights for policy 0, policy_version 1094052 (0.0005) [2023-12-26 23:20:32,149][105620] Updated weights for policy 1, policy_version 1095110 (0.0009) [2023-12-26 23:20:32,180][105692] Updated weights for policy 0, policy_version 1094062 (0.0007) [2023-12-26 23:20:32,195][105620] Updated weights for policy 1, policy_version 1095120 (0.0007) [2023-12-26 23:20:32,243][105620] Updated weights for policy 1, policy_version 1095130 (0.0007) [2023-12-26 23:20:32,244][105692] Updated weights for policy 0, policy_version 1094072 (0.0006) [2023-12-26 23:20:32,988][105692] Updated weights for policy 0, policy_version 1094082 (0.0009) [2023-12-26 23:20:32,989][105620] Updated weights for policy 1, policy_version 1095140 (0.0008) [2023-12-26 23:20:33,032][105620] Updated weights for policy 1, policy_version 1095150 (0.0008) [2023-12-26 23:20:33,043][105692] Updated weights for policy 0, policy_version 1094092 (0.0007) [2023-12-26 23:20:33,081][105620] Updated weights for policy 1, policy_version 1095160 (0.0007) [2023-12-26 23:20:33,095][105692] Updated weights for policy 0, policy_version 1094102 (0.0007) [2023-12-26 23:20:33,157][105692] Updated weights for policy 0, policy_version 1094112 (0.0008) [2023-12-26 23:20:33,794][105620] Updated weights for policy 1, policy_version 1095170 (0.0006) [2023-12-26 23:20:33,841][105620] Updated weights for policy 1, policy_version 1095180 (0.0008) [2023-12-26 23:20:33,889][105620] Updated weights for policy 1, policy_version 1095190 (0.0007) [2023-12-26 23:20:33,920][105692] Updated weights for policy 0, policy_version 1094122 (0.0007) [2023-12-26 23:20:33,941][105620] Updated weights for policy 1, policy_version 1095200 (0.0007) [2023-12-26 23:20:33,970][105692] Updated weights for policy 0, policy_version 1094132 (0.0008) [2023-12-26 23:20:34,018][105692] Updated weights for policy 0, policy_version 1094142 (0.0009) [2023-12-26 23:20:34,686][105620] Updated weights for policy 1, policy_version 1095210 (0.0010) [2023-12-26 23:20:34,742][105620] Updated weights for policy 1, policy_version 1095220 (0.0009) [2023-12-26 23:20:34,758][105692] Updated weights for policy 0, policy_version 1094152 (0.0008) [2023-12-26 23:20:34,792][105620] Updated weights for policy 1, policy_version 1095230 (0.0007) [2023-12-26 23:20:34,817][105692] Updated weights for policy 0, policy_version 1094162 (0.0008) [2023-12-26 23:20:34,881][105692] Updated weights for policy 0, policy_version 1094172 (0.0009) [2023-12-26 23:20:35,550][105692] Updated weights for policy 0, policy_version 1094182 (0.0008) [2023-12-26 23:20:35,597][105620] Updated weights for policy 1, policy_version 1095240 (0.0008) [2023-12-26 23:20:35,601][105692] Updated weights for policy 0, policy_version 1094192 (0.0005) [2023-12-26 23:20:35,651][105620] Updated weights for policy 1, policy_version 1095250 (0.0009) [2023-12-26 23:20:35,654][105692] Updated weights for policy 0, policy_version 1094202 (0.0005) [2023-12-26 23:20:35,709][105620] Updated weights for policy 1, policy_version 1095260 (0.0008) [2023-12-26 23:20:36,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 560578560. Throughput: 0: 9722.6, 1: 9835.8. Samples: 560566368. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:20:36,062][104569] Avg episode reward: [(0, '9263.568'), (1, '9260.028')] [2023-12-26 23:20:36,387][105692] Updated weights for policy 0, policy_version 1094212 (0.0007) [2023-12-26 23:20:36,439][105692] Updated weights for policy 0, policy_version 1094222 (0.0010) [2023-12-26 23:20:36,485][105692] Updated weights for policy 0, policy_version 1094232 (0.0011) [2023-12-26 23:20:36,489][105620] Updated weights for policy 1, policy_version 1095270 (0.0008) [2023-12-26 23:20:36,553][105620] Updated weights for policy 1, policy_version 1095280 (0.0007) [2023-12-26 23:20:36,625][105620] Updated weights for policy 1, policy_version 1095290 (0.0008) [2023-12-26 23:20:37,247][105692] Updated weights for policy 0, policy_version 1094242 (0.0011) [2023-12-26 23:20:37,284][105620] Updated weights for policy 1, policy_version 1095300 (0.0008) [2023-12-26 23:20:37,309][105692] Updated weights for policy 0, policy_version 1094252 (0.0010) [2023-12-26 23:20:37,339][105620] Updated weights for policy 1, policy_version 1095310 (0.0008) [2023-12-26 23:20:37,372][105692] Updated weights for policy 0, policy_version 1094262 (0.0009) [2023-12-26 23:20:37,396][105620] Updated weights for policy 1, policy_version 1095320 (0.0008) [2023-12-26 23:20:37,431][105692] Updated weights for policy 0, policy_version 1094272 (0.0010) [2023-12-26 23:20:38,111][105620] Updated weights for policy 1, policy_version 1095330 (0.0006) [2023-12-26 23:20:38,139][105692] Updated weights for policy 0, policy_version 1094282 (0.0010) [2023-12-26 23:20:38,160][105620] Updated weights for policy 1, policy_version 1095340 (0.0005) [2023-12-26 23:20:38,197][105692] Updated weights for policy 0, policy_version 1094292 (0.0011) [2023-12-26 23:20:38,219][105620] Updated weights for policy 1, policy_version 1095350 (0.0005) [2023-12-26 23:20:38,259][105692] Updated weights for policy 0, policy_version 1094302 (0.0010) [2023-12-26 23:20:38,273][105620] Updated weights for policy 1, policy_version 1095360 (0.0005) [2023-12-26 23:20:38,920][105620] Updated weights for policy 1, policy_version 1095370 (0.0006) [2023-12-26 23:20:38,966][105692] Updated weights for policy 0, policy_version 1094312 (0.0007) [2023-12-26 23:20:38,979][105620] Updated weights for policy 1, policy_version 1095380 (0.0007) [2023-12-26 23:20:39,031][105692] Updated weights for policy 0, policy_version 1094322 (0.0010) [2023-12-26 23:20:39,032][105620] Updated weights for policy 1, policy_version 1095390 (0.0011) [2023-12-26 23:20:39,098][105692] Updated weights for policy 0, policy_version 1094332 (0.0011) [2023-12-26 23:20:39,726][105620] Updated weights for policy 1, policy_version 1095400 (0.0011) [2023-12-26 23:20:39,744][105692] Updated weights for policy 0, policy_version 1094342 (0.0008) [2023-12-26 23:20:39,784][105620] Updated weights for policy 1, policy_version 1095410 (0.0010) [2023-12-26 23:20:39,804][105692] Updated weights for policy 0, policy_version 1094352 (0.0007) [2023-12-26 23:20:39,850][105620] Updated weights for policy 1, policy_version 1095420 (0.0011) [2023-12-26 23:20:39,872][105692] Updated weights for policy 0, policy_version 1094362 (0.0011) [2023-12-26 23:20:40,451][105692] Updated weights for policy 0, policy_version 1094372 (0.0008) [2023-12-26 23:20:40,517][105692] Updated weights for policy 0, policy_version 1094382 (0.0011) [2023-12-26 23:20:40,580][105692] Updated weights for policy 0, policy_version 1094392 (0.0010) [2023-12-26 23:20:40,591][105620] Updated weights for policy 1, policy_version 1095430 (0.0008) [2023-12-26 23:20:40,647][105620] Updated weights for policy 1, policy_version 1095440 (0.0006) [2023-12-26 23:20:40,693][105620] Updated weights for policy 1, policy_version 1095450 (0.0009) [2023-12-26 23:20:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 560676864. Throughput: 0: 9783.0, 1: 9749.1. Samples: 560685236. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:20:41,063][104569] Avg episode reward: [(0, '9174.946'), (1, '9259.909')] [2023-12-26 23:20:41,359][105692] Updated weights for policy 0, policy_version 1094402 (0.0010) [2023-12-26 23:20:41,422][105692] Updated weights for policy 0, policy_version 1094412 (0.0009) [2023-12-26 23:20:41,479][105692] Updated weights for policy 0, policy_version 1094422 (0.0011) [2023-12-26 23:20:41,485][105620] Updated weights for policy 1, policy_version 1095460 (0.0007) [2023-12-26 23:20:41,542][105692] Updated weights for policy 0, policy_version 1094432 (0.0011) [2023-12-26 23:20:41,549][105620] Updated weights for policy 1, policy_version 1095470 (0.0006) [2023-12-26 23:20:41,601][105620] Updated weights for policy 1, policy_version 1095480 (0.0008) [2023-12-26 23:20:42,241][105692] Updated weights for policy 0, policy_version 1094442 (0.0007) [2023-12-26 23:20:42,310][105692] Updated weights for policy 0, policy_version 1094452 (0.0009) [2023-12-26 23:20:42,381][105692] Updated weights for policy 0, policy_version 1094462 (0.0008) [2023-12-26 23:20:42,418][105620] Updated weights for policy 1, policy_version 1095490 (0.0009) [2023-12-26 23:20:42,482][105620] Updated weights for policy 1, policy_version 1095500 (0.0009) [2023-12-26 23:20:42,542][105620] Updated weights for policy 1, policy_version 1095510 (0.0009) [2023-12-26 23:20:42,602][105620] Updated weights for policy 1, policy_version 1095520 (0.0010) [2023-12-26 23:20:43,023][105692] Updated weights for policy 0, policy_version 1094472 (0.0008) [2023-12-26 23:20:43,090][105692] Updated weights for policy 0, policy_version 1094482 (0.0009) [2023-12-26 23:20:43,153][105692] Updated weights for policy 0, policy_version 1094492 (0.0009) [2023-12-26 23:20:43,388][105620] Updated weights for policy 1, policy_version 1095530 (0.0008) [2023-12-26 23:20:43,437][105620] Updated weights for policy 1, policy_version 1095540 (0.0009) [2023-12-26 23:20:43,481][105620] Updated weights for policy 1, policy_version 1095550 (0.0008) [2023-12-26 23:20:43,885][105692] Updated weights for policy 0, policy_version 1094502 (0.0008) [2023-12-26 23:20:43,943][105692] Updated weights for policy 0, policy_version 1094512 (0.0005) [2023-12-26 23:20:43,990][105692] Updated weights for policy 0, policy_version 1094522 (0.0005) [2023-12-26 23:20:44,264][105620] Updated weights for policy 1, policy_version 1095560 (0.0009) [2023-12-26 23:20:44,314][105620] Updated weights for policy 1, policy_version 1095571 (0.0007) [2023-12-26 23:20:44,368][105620] Updated weights for policy 1, policy_version 1095581 (0.0005) [2023-12-26 23:20:44,536][105692] Updated weights for policy 0, policy_version 1094532 (0.0006) [2023-12-26 23:20:44,583][105692] Updated weights for policy 0, policy_version 1094542 (0.0005) [2023-12-26 23:20:44,643][105692] Updated weights for policy 0, policy_version 1094552 (0.0009) [2023-12-26 23:20:44,995][105620] Updated weights for policy 1, policy_version 1095591 (0.0008) [2023-12-26 23:20:45,053][105620] Updated weights for policy 1, policy_version 1095601 (0.0009) [2023-12-26 23:20:45,114][105620] Updated weights for policy 1, policy_version 1095611 (0.0008) [2023-12-26 23:20:45,367][105692] Updated weights for policy 0, policy_version 1094562 (0.0009) [2023-12-26 23:20:45,424][105692] Updated weights for policy 0, policy_version 1094572 (0.0009) [2023-12-26 23:20:45,487][105692] Updated weights for policy 0, policy_version 1094582 (0.0009) [2023-12-26 23:20:45,552][105692] Updated weights for policy 0, policy_version 1094592 (0.0010) [2023-12-26 23:20:45,921][105620] Updated weights for policy 1, policy_version 1095621 (0.0010) [2023-12-26 23:20:45,973][105620] Updated weights for policy 1, policy_version 1095631 (0.0010) [2023-12-26 23:20:46,040][105620] Updated weights for policy 1, policy_version 1095641 (0.0010) [2023-12-26 23:20:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 560766976. Throughput: 0: 9749.0, 1: 9595.7. Samples: 560740972. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:20:46,062][104569] Avg episode reward: [(0, '9264.093'), (1, '9351.459')] [2023-12-26 23:20:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001094592_280256512.pth... [2023-12-26 23:20:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001093440_279961600.pth [2023-12-26 23:20:46,084][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001095648_280518656.pth... [2023-12-26 23:20:46,087][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001094528_280231936.pth [2023-12-26 23:20:46,269][105692] Updated weights for policy 0, policy_version 1094602 (0.0009) [2023-12-26 23:20:46,344][105692] Updated weights for policy 0, policy_version 1094612 (0.0010) [2023-12-26 23:20:46,410][105692] Updated weights for policy 0, policy_version 1094622 (0.0009) [2023-12-26 23:20:46,704][105620] Updated weights for policy 1, policy_version 1095651 (0.0009) [2023-12-26 23:20:46,757][105620] Updated weights for policy 1, policy_version 1095661 (0.0008) [2023-12-26 23:20:46,813][105620] Updated weights for policy 1, policy_version 1095671 (0.0008) [2023-12-26 23:20:47,163][105692] Updated weights for policy 0, policy_version 1094632 (0.0007) [2023-12-26 23:20:47,211][105692] Updated weights for policy 0, policy_version 1094642 (0.0008) [2023-12-26 23:20:47,263][105692] Updated weights for policy 0, policy_version 1094652 (0.0006) [2023-12-26 23:20:47,592][105620] Updated weights for policy 1, policy_version 1095681 (0.0008) [2023-12-26 23:20:47,644][105620] Updated weights for policy 1, policy_version 1095691 (0.0010) [2023-12-26 23:20:47,698][105620] Updated weights for policy 1, policy_version 1095701 (0.0010) [2023-12-26 23:20:47,751][105620] Updated weights for policy 1, policy_version 1095711 (0.0009) [2023-12-26 23:20:47,971][105692] Updated weights for policy 0, policy_version 1094662 (0.0006) [2023-12-26 23:20:48,020][105692] Updated weights for policy 0, policy_version 1094672 (0.0005) [2023-12-26 23:20:48,077][105692] Updated weights for policy 0, policy_version 1094682 (0.0005) [2023-12-26 23:20:48,343][105620] Updated weights for policy 1, policy_version 1095721 (0.0007) [2023-12-26 23:20:48,397][105620] Updated weights for policy 1, policy_version 1095731 (0.0009) [2023-12-26 23:20:48,456][105620] Updated weights for policy 1, policy_version 1095741 (0.0011) [2023-12-26 23:20:48,721][105692] Updated weights for policy 0, policy_version 1094692 (0.0007) [2023-12-26 23:20:48,776][105692] Updated weights for policy 0, policy_version 1094702 (0.0010) [2023-12-26 23:20:48,825][105692] Updated weights for policy 0, policy_version 1094712 (0.0010) [2023-12-26 23:20:49,045][105620] Updated weights for policy 1, policy_version 1095751 (0.0011) [2023-12-26 23:20:49,097][105620] Updated weights for policy 1, policy_version 1095761 (0.0010) [2023-12-26 23:20:49,156][105620] Updated weights for policy 1, policy_version 1095771 (0.0010) [2023-12-26 23:20:49,600][105692] Updated weights for policy 0, policy_version 1094722 (0.0010) [2023-12-26 23:20:49,645][105692] Updated weights for policy 0, policy_version 1094732 (0.0008) [2023-12-26 23:20:49,692][105692] Updated weights for policy 0, policy_version 1094742 (0.0007) [2023-12-26 23:20:49,749][105692] Updated weights for policy 0, policy_version 1094752 (0.0008) [2023-12-26 23:20:49,879][105620] Updated weights for policy 1, policy_version 1095781 (0.0010) [2023-12-26 23:20:49,945][105620] Updated weights for policy 1, policy_version 1095791 (0.0010) [2023-12-26 23:20:50,005][105620] Updated weights for policy 1, policy_version 1095801 (0.0009) [2023-12-26 23:20:50,615][105692] Updated weights for policy 0, policy_version 1094762 (0.0008) [2023-12-26 23:20:50,652][105620] Updated weights for policy 1, policy_version 1095811 (0.0008) [2023-12-26 23:20:50,679][105692] Updated weights for policy 0, policy_version 1094772 (0.0008) [2023-12-26 23:20:50,716][105620] Updated weights for policy 1, policy_version 1095821 (0.0008) [2023-12-26 23:20:50,740][105692] Updated weights for policy 0, policy_version 1094782 (0.0009) [2023-12-26 23:20:50,773][105620] Updated weights for policy 1, policy_version 1095831 (0.0008) [2023-12-26 23:20:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 560873472. Throughput: 0: 9777.5, 1: 9688.3. Samples: 560861628. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:20:51,063][104569] Avg episode reward: [(0, '9353.852'), (1, '9259.908')] [2023-12-26 23:20:51,529][105692] Updated weights for policy 0, policy_version 1094792 (0.0009) [2023-12-26 23:20:51,544][105620] Updated weights for policy 1, policy_version 1095841 (0.0009) [2023-12-26 23:20:51,583][105692] Updated weights for policy 0, policy_version 1094802 (0.0007) [2023-12-26 23:20:51,593][105620] Updated weights for policy 1, policy_version 1095851 (0.0007) [2023-12-26 23:20:51,643][105692] Updated weights for policy 0, policy_version 1094812 (0.0007) [2023-12-26 23:20:51,654][105620] Updated weights for policy 1, policy_version 1095861 (0.0008) [2023-12-26 23:20:51,722][105620] Updated weights for policy 1, policy_version 1095871 (0.0008) [2023-12-26 23:20:52,422][105620] Updated weights for policy 1, policy_version 1095881 (0.0006) [2023-12-26 23:20:52,468][105692] Updated weights for policy 0, policy_version 1094822 (0.0008) [2023-12-26 23:20:52,480][105620] Updated weights for policy 1, policy_version 1095891 (0.0007) [2023-12-26 23:20:52,533][105692] Updated weights for policy 0, policy_version 1094832 (0.0007) [2023-12-26 23:20:52,541][105620] Updated weights for policy 1, policy_version 1095901 (0.0006) [2023-12-26 23:20:52,599][105692] Updated weights for policy 0, policy_version 1094842 (0.0010) [2023-12-26 23:20:53,208][105620] Updated weights for policy 1, policy_version 1095911 (0.0007) [2023-12-26 23:20:53,258][105620] Updated weights for policy 1, policy_version 1095921 (0.0008) [2023-12-26 23:20:53,313][105620] Updated weights for policy 1, policy_version 1095931 (0.0009) [2023-12-26 23:20:53,362][105692] Updated weights for policy 0, policy_version 1094852 (0.0009) [2023-12-26 23:20:53,427][105692] Updated weights for policy 0, policy_version 1094862 (0.0008) [2023-12-26 23:20:53,487][105692] Updated weights for policy 0, policy_version 1094872 (0.0009) [2023-12-26 23:20:54,064][105620] Updated weights for policy 1, policy_version 1095941 (0.0009) [2023-12-26 23:20:54,127][105620] Updated weights for policy 1, policy_version 1095951 (0.0009) [2023-12-26 23:20:54,189][105620] Updated weights for policy 1, policy_version 1095961 (0.0009) [2023-12-26 23:20:54,212][105692] Updated weights for policy 0, policy_version 1094882 (0.0009) [2023-12-26 23:20:54,271][105692] Updated weights for policy 0, policy_version 1094892 (0.0009) [2023-12-26 23:20:54,328][105692] Updated weights for policy 0, policy_version 1094902 (0.0009) [2023-12-26 23:20:54,385][105692] Updated weights for policy 0, policy_version 1094912 (0.0009) [2023-12-26 23:20:54,964][105620] Updated weights for policy 1, policy_version 1095971 (0.0007) [2023-12-26 23:20:55,021][105620] Updated weights for policy 1, policy_version 1095981 (0.0009) [2023-12-26 23:20:55,079][105620] Updated weights for policy 1, policy_version 1095991 (0.0008) [2023-12-26 23:20:55,135][105692] Updated weights for policy 0, policy_version 1094922 (0.0010) [2023-12-26 23:20:55,186][105692] Updated weights for policy 0, policy_version 1094932 (0.0010) [2023-12-26 23:20:55,245][105692] Updated weights for policy 0, policy_version 1094942 (0.0010) [2023-12-26 23:20:55,837][105620] Updated weights for policy 1, policy_version 1096001 (0.0007) [2023-12-26 23:20:55,903][105620] Updated weights for policy 1, policy_version 1096011 (0.0008) [2023-12-26 23:20:55,959][105620] Updated weights for policy 1, policy_version 1096021 (0.0008) [2023-12-26 23:20:56,002][105692] Updated weights for policy 0, policy_version 1094952 (0.0010) [2023-12-26 23:20:56,040][105620] Updated weights for policy 1, policy_version 1096031 (0.0008) [2023-12-26 23:20:56,055][105692] Updated weights for policy 0, policy_version 1094962 (0.0011) [2023-12-26 23:20:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 560963584. Throughput: 0: 9640.3, 1: 9692.5. Samples: 560973392. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:20:56,062][104569] Avg episode reward: [(0, '9353.193'), (1, '9076.555')] [2023-12-26 23:20:56,111][105692] Updated weights for policy 0, policy_version 1094972 (0.0011) [2023-12-26 23:20:56,767][105692] Updated weights for policy 0, policy_version 1094982 (0.0007) [2023-12-26 23:20:56,802][105620] Updated weights for policy 1, policy_version 1096041 (0.0007) [2023-12-26 23:20:56,816][105692] Updated weights for policy 0, policy_version 1094992 (0.0010) [2023-12-26 23:20:56,851][105620] Updated weights for policy 1, policy_version 1096051 (0.0005) [2023-12-26 23:20:56,864][105692] Updated weights for policy 0, policy_version 1095002 (0.0010) [2023-12-26 23:20:56,909][105620] Updated weights for policy 1, policy_version 1096061 (0.0007) [2023-12-26 23:20:57,498][105692] Updated weights for policy 0, policy_version 1095012 (0.0010) [2023-12-26 23:20:57,552][105692] Updated weights for policy 0, policy_version 1095022 (0.0010) [2023-12-26 23:20:57,602][105692] Updated weights for policy 0, policy_version 1095032 (0.0010) [2023-12-26 23:20:57,678][105620] Updated weights for policy 1, policy_version 1096071 (0.0009) [2023-12-26 23:20:57,735][105620] Updated weights for policy 1, policy_version 1096081 (0.0009) [2023-12-26 23:20:57,804][105620] Updated weights for policy 1, policy_version 1096091 (0.0010) [2023-12-26 23:20:58,221][105692] Updated weights for policy 0, policy_version 1095042 (0.0008) [2023-12-26 23:20:58,285][105692] Updated weights for policy 0, policy_version 1095052 (0.0011) [2023-12-26 23:20:58,347][105692] Updated weights for policy 0, policy_version 1095062 (0.0009) [2023-12-26 23:20:58,410][105692] Updated weights for policy 0, policy_version 1095072 (0.0009) [2023-12-26 23:20:58,668][105620] Updated weights for policy 1, policy_version 1096101 (0.0009) [2023-12-26 23:20:58,732][105620] Updated weights for policy 1, policy_version 1096111 (0.0008) [2023-12-26 23:20:58,795][105620] Updated weights for policy 1, policy_version 1096121 (0.0008) [2023-12-26 23:20:59,215][105692] Updated weights for policy 0, policy_version 1095082 (0.0009) [2023-12-26 23:20:59,286][105692] Updated weights for policy 0, policy_version 1095092 (0.0008) [2023-12-26 23:20:59,349][105692] Updated weights for policy 0, policy_version 1095102 (0.0007) [2023-12-26 23:20:59,669][105620] Updated weights for policy 1, policy_version 1096131 (0.0008) [2023-12-26 23:20:59,716][105620] Updated weights for policy 1, policy_version 1096141 (0.0009) [2023-12-26 23:20:59,761][105620] Updated weights for policy 1, policy_version 1096151 (0.0008) [2023-12-26 23:21:00,115][105692] Updated weights for policy 0, policy_version 1095112 (0.0007) [2023-12-26 23:21:00,174][105692] Updated weights for policy 0, policy_version 1095122 (0.0008) [2023-12-26 23:21:00,228][105692] Updated weights for policy 0, policy_version 1095132 (0.0005) [2023-12-26 23:21:00,639][105620] Updated weights for policy 1, policy_version 1096161 (0.0009) [2023-12-26 23:21:00,686][105620] Updated weights for policy 1, policy_version 1096171 (0.0009) [2023-12-26 23:21:00,739][105620] Updated weights for policy 1, policy_version 1096181 (0.0009) [2023-12-26 23:21:00,794][105620] Updated weights for policy 1, policy_version 1096191 (0.0007) [2023-12-26 23:21:00,799][105692] Updated weights for policy 0, policy_version 1095142 (0.0007) [2023-12-26 23:21:00,859][105692] Updated weights for policy 0, policy_version 1095152 (0.0008) [2023-12-26 23:21:00,918][105692] Updated weights for policy 0, policy_version 1095162 (0.0008) [2023-12-26 23:21:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 561061888. Throughput: 0: 9703.8, 1: 9606.9. Samples: 561030440. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:21:01,063][104569] Avg episode reward: [(0, '9353.471'), (1, '9076.687')] [2023-12-26 23:21:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001095168_280403968.pth... [2023-12-26 23:21:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001096192_280657920.pth... [2023-12-26 23:21:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001095072_280371200.pth [2023-12-26 23:21:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001094016_280109056.pth [2023-12-26 23:21:01,517][105620] Updated weights for policy 1, policy_version 1096201 (0.0008) [2023-12-26 23:21:01,564][105620] Updated weights for policy 1, policy_version 1096211 (0.0009) [2023-12-26 23:21:01,617][105620] Updated weights for policy 1, policy_version 1096221 (0.0009) [2023-12-26 23:21:01,739][105692] Updated weights for policy 0, policy_version 1095172 (0.0008) [2023-12-26 23:21:01,798][105692] Updated weights for policy 0, policy_version 1095182 (0.0009) [2023-12-26 23:21:01,855][105692] Updated weights for policy 0, policy_version 1095192 (0.0009) [2023-12-26 23:21:02,388][105620] Updated weights for policy 1, policy_version 1096231 (0.0009) [2023-12-26 23:21:02,438][105620] Updated weights for policy 1, policy_version 1096241 (0.0008) [2023-12-26 23:21:02,487][105620] Updated weights for policy 1, policy_version 1096251 (0.0007) [2023-12-26 23:21:02,629][105692] Updated weights for policy 0, policy_version 1095202 (0.0008) [2023-12-26 23:21:02,682][105692] Updated weights for policy 0, policy_version 1095212 (0.0009) [2023-12-26 23:21:02,734][105692] Updated weights for policy 0, policy_version 1095222 (0.0010) [2023-12-26 23:21:02,793][105692] Updated weights for policy 0, policy_version 1095232 (0.0010) [2023-12-26 23:21:03,098][105620] Updated weights for policy 1, policy_version 1096261 (0.0005) [2023-12-26 23:21:03,165][105620] Updated weights for policy 1, policy_version 1096271 (0.0006) [2023-12-26 23:21:03,221][105620] Updated weights for policy 1, policy_version 1096281 (0.0009) [2023-12-26 23:21:03,612][105692] Updated weights for policy 0, policy_version 1095242 (0.0009) [2023-12-26 23:21:03,667][105692] Updated weights for policy 0, policy_version 1095252 (0.0009) [2023-12-26 23:21:03,729][105692] Updated weights for policy 0, policy_version 1095262 (0.0009) [2023-12-26 23:21:03,912][105620] Updated weights for policy 1, policy_version 1096291 (0.0009) [2023-12-26 23:21:03,982][105620] Updated weights for policy 1, policy_version 1096301 (0.0008) [2023-12-26 23:21:04,048][105620] Updated weights for policy 1, policy_version 1096311 (0.0007) [2023-12-26 23:21:04,490][105692] Updated weights for policy 0, policy_version 1095272 (0.0009) [2023-12-26 23:21:04,552][105692] Updated weights for policy 0, policy_version 1095282 (0.0008) [2023-12-26 23:21:04,619][105692] Updated weights for policy 0, policy_version 1095292 (0.0008) [2023-12-26 23:21:04,708][105620] Updated weights for policy 1, policy_version 1096321 (0.0006) [2023-12-26 23:21:04,761][105620] Updated weights for policy 1, policy_version 1096331 (0.0005) [2023-12-26 23:21:04,819][105620] Updated weights for policy 1, policy_version 1096341 (0.0010) [2023-12-26 23:21:04,865][105620] Updated weights for policy 1, policy_version 1096351 (0.0009) [2023-12-26 23:21:05,208][105692] Updated weights for policy 0, policy_version 1095302 (0.0006) [2023-12-26 23:21:05,258][105692] Updated weights for policy 0, policy_version 1095312 (0.0006) [2023-12-26 23:21:05,321][105692] Updated weights for policy 0, policy_version 1095322 (0.0009) [2023-12-26 23:21:05,461][105620] Updated weights for policy 1, policy_version 1096361 (0.0005) [2023-12-26 23:21:05,508][105620] Updated weights for policy 1, policy_version 1096371 (0.0005) [2023-12-26 23:21:05,561][105620] Updated weights for policy 1, policy_version 1096381 (0.0005) [2023-12-26 23:21:05,939][105692] Updated weights for policy 0, policy_version 1095332 (0.0010) [2023-12-26 23:21:06,003][105692] Updated weights for policy 0, policy_version 1095342 (0.0010) [2023-12-26 23:21:06,059][105692] Updated weights for policy 0, policy_version 1095352 (0.0010) [2023-12-26 23:21:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.6, 300 sec: 19494.2). Total num frames: 561152000. Throughput: 0: 9778.1, 1: 9478.8. Samples: 561143792. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:21:06,062][104569] Avg episode reward: [(0, '9354.616'), (1, '9077.104')] [2023-12-26 23:21:06,122][105620] Updated weights for policy 1, policy_version 1096391 (0.0008) [2023-12-26 23:21:06,172][105620] Updated weights for policy 1, policy_version 1096401 (0.0006) [2023-12-26 23:21:06,229][105620] Updated weights for policy 1, policy_version 1096411 (0.0008) [2023-12-26 23:21:06,798][105692] Updated weights for policy 0, policy_version 1095362 (0.0009) [2023-12-26 23:21:06,819][105620] Updated weights for policy 1, policy_version 1096421 (0.0007) [2023-12-26 23:21:06,859][105692] Updated weights for policy 0, policy_version 1095372 (0.0008) [2023-12-26 23:21:06,874][105620] Updated weights for policy 1, policy_version 1096431 (0.0006) [2023-12-26 23:21:06,922][105692] Updated weights for policy 0, policy_version 1095382 (0.0011) [2023-12-26 23:21:06,929][105620] Updated weights for policy 1, policy_version 1096441 (0.0006) [2023-12-26 23:21:06,985][105692] Updated weights for policy 0, policy_version 1095392 (0.0011) [2023-12-26 23:21:07,664][105620] Updated weights for policy 1, policy_version 1096451 (0.0008) [2023-12-26 23:21:07,674][105692] Updated weights for policy 0, policy_version 1095402 (0.0005) [2023-12-26 23:21:07,721][105620] Updated weights for policy 1, policy_version 1096461 (0.0011) [2023-12-26 23:21:07,733][105692] Updated weights for policy 0, policy_version 1095412 (0.0006) [2023-12-26 23:21:07,776][105620] Updated weights for policy 1, policy_version 1096471 (0.0010) [2023-12-26 23:21:07,785][105692] Updated weights for policy 0, policy_version 1095422 (0.0006) [2023-12-26 23:21:08,417][105692] Updated weights for policy 0, policy_version 1095432 (0.0008) [2023-12-26 23:21:08,466][105620] Updated weights for policy 1, policy_version 1096481 (0.0010) [2023-12-26 23:21:08,477][105692] Updated weights for policy 0, policy_version 1095442 (0.0008) [2023-12-26 23:21:08,530][105620] Updated weights for policy 1, policy_version 1096491 (0.0011) [2023-12-26 23:21:08,541][105692] Updated weights for policy 0, policy_version 1095452 (0.0009) [2023-12-26 23:21:08,595][105620] Updated weights for policy 1, policy_version 1096501 (0.0011) [2023-12-26 23:21:08,658][105620] Updated weights for policy 1, policy_version 1096511 (0.0011) [2023-12-26 23:21:09,253][105692] Updated weights for policy 0, policy_version 1095462 (0.0008) [2023-12-26 23:21:09,309][105692] Updated weights for policy 0, policy_version 1095472 (0.0009) [2023-12-26 23:21:09,369][105692] Updated weights for policy 0, policy_version 1095482 (0.0009) [2023-12-26 23:21:09,406][105620] Updated weights for policy 1, policy_version 1096521 (0.0010) [2023-12-26 23:21:09,476][105620] Updated weights for policy 1, policy_version 1096531 (0.0009) [2023-12-26 23:21:09,530][105620] Updated weights for policy 1, policy_version 1096541 (0.0010) [2023-12-26 23:21:10,058][105692] Updated weights for policy 0, policy_version 1095492 (0.0008) [2023-12-26 23:21:10,122][105692] Updated weights for policy 0, policy_version 1095502 (0.0009) [2023-12-26 23:21:10,172][105692] Updated weights for policy 0, policy_version 1095512 (0.0008) [2023-12-26 23:21:10,351][105620] Updated weights for policy 1, policy_version 1096551 (0.0010) [2023-12-26 23:21:10,407][105620] Updated weights for policy 1, policy_version 1096561 (0.0011) [2023-12-26 23:21:10,472][105620] Updated weights for policy 1, policy_version 1096571 (0.0009) [2023-12-26 23:21:11,011][105692] Updated weights for policy 0, policy_version 1095522 (0.0008) [2023-12-26 23:21:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 561250304. Throughput: 0: 9803.2, 1: 9619.6. Samples: 561266404. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:21:11,062][104569] Avg episode reward: [(0, '9263.115'), (1, '9168.278')] [2023-12-26 23:21:11,083][105692] Updated weights for policy 0, policy_version 1095532 (0.0009) [2023-12-26 23:21:11,111][105620] Updated weights for policy 1, policy_version 1096581 (0.0008) [2023-12-26 23:21:11,144][105692] Updated weights for policy 0, policy_version 1095542 (0.0009) [2023-12-26 23:21:11,174][105620] Updated weights for policy 1, policy_version 1096591 (0.0007) [2023-12-26 23:21:11,200][105692] Updated weights for policy 0, policy_version 1095552 (0.0007) [2023-12-26 23:21:11,233][105620] Updated weights for policy 1, policy_version 1096601 (0.0007) [2023-12-26 23:21:11,979][105620] Updated weights for policy 1, policy_version 1096611 (0.0008) [2023-12-26 23:21:11,992][105692] Updated weights for policy 0, policy_version 1095562 (0.0006) [2023-12-26 23:21:12,043][105620] Updated weights for policy 1, policy_version 1096621 (0.0007) [2023-12-26 23:21:12,055][105692] Updated weights for policy 0, policy_version 1095572 (0.0006) [2023-12-26 23:21:12,106][105620] Updated weights for policy 1, policy_version 1096631 (0.0007) [2023-12-26 23:21:12,121][105692] Updated weights for policy 0, policy_version 1095582 (0.0009) [2023-12-26 23:21:12,850][105620] Updated weights for policy 1, policy_version 1096641 (0.0008) [2023-12-26 23:21:12,889][105692] Updated weights for policy 0, policy_version 1095592 (0.0006) [2023-12-26 23:21:12,907][105620] Updated weights for policy 1, policy_version 1096651 (0.0011) [2023-12-26 23:21:12,948][105692] Updated weights for policy 0, policy_version 1095602 (0.0006) [2023-12-26 23:21:12,969][105620] Updated weights for policy 1, policy_version 1096661 (0.0010) [2023-12-26 23:21:12,997][105692] Updated weights for policy 0, policy_version 1095612 (0.0009) [2023-12-26 23:21:13,031][105620] Updated weights for policy 1, policy_version 1096671 (0.0010) [2023-12-26 23:21:13,707][105620] Updated weights for policy 1, policy_version 1096681 (0.0011) [2023-12-26 23:21:13,722][105692] Updated weights for policy 0, policy_version 1095622 (0.0008) [2023-12-26 23:21:13,761][105620] Updated weights for policy 1, policy_version 1096691 (0.0011) [2023-12-26 23:21:13,782][105692] Updated weights for policy 0, policy_version 1095632 (0.0010) [2023-12-26 23:21:13,822][105620] Updated weights for policy 1, policy_version 1096701 (0.0011) [2023-12-26 23:21:13,841][105692] Updated weights for policy 0, policy_version 1095642 (0.0007) [2023-12-26 23:21:14,473][105692] Updated weights for policy 0, policy_version 1095652 (0.0005) [2023-12-26 23:21:14,522][105692] Updated weights for policy 0, policy_version 1095662 (0.0009) [2023-12-26 23:21:14,567][105620] Updated weights for policy 1, policy_version 1096711 (0.0007) [2023-12-26 23:21:14,577][105692] Updated weights for policy 0, policy_version 1095672 (0.0010) [2023-12-26 23:21:14,623][105620] Updated weights for policy 1, policy_version 1096721 (0.0005) [2023-12-26 23:21:14,691][105620] Updated weights for policy 1, policy_version 1096731 (0.0005) [2023-12-26 23:21:15,257][105620] Updated weights for policy 1, policy_version 1096741 (0.0008) [2023-12-26 23:21:15,307][105692] Updated weights for policy 0, policy_version 1095682 (0.0009) [2023-12-26 23:21:15,310][105620] Updated weights for policy 1, policy_version 1096751 (0.0010) [2023-12-26 23:21:15,363][105692] Updated weights for policy 0, policy_version 1095692 (0.0006) [2023-12-26 23:21:15,367][105620] Updated weights for policy 1, policy_version 1096761 (0.0009) [2023-12-26 23:21:15,427][105692] Updated weights for policy 0, policy_version 1095702 (0.0006) [2023-12-26 23:21:15,494][105692] Updated weights for policy 0, policy_version 1095712 (0.0006) [2023-12-26 23:21:16,058][105692] Updated weights for policy 0, policy_version 1095722 (0.0009) [2023-12-26 23:21:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 561348608. Throughput: 0: 9701.8, 1: 9622.8. Samples: 561322168. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:21:16,062][104569] Avg episode reward: [(0, '9263.198'), (1, '9350.914')] [2023-12-26 23:21:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001096768_280805376.pth... [2023-12-26 23:21:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001095648_280518656.pth [2023-12-26 23:21:16,110][105692] Updated weights for policy 0, policy_version 1095732 (0.0009) [2023-12-26 23:21:16,165][105692] Updated weights for policy 0, policy_version 1095742 (0.0009) [2023-12-26 23:21:16,175][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001095744_280551424.pth... [2023-12-26 23:21:16,179][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001094592_280256512.pth [2023-12-26 23:21:16,222][105620] Updated weights for policy 1, policy_version 1096771 (0.0009) [2023-12-26 23:21:16,277][105620] Updated weights for policy 1, policy_version 1096781 (0.0009) [2023-12-26 23:21:16,323][105620] Updated weights for policy 1, policy_version 1096791 (0.0008) [2023-12-26 23:21:16,951][105692] Updated weights for policy 0, policy_version 1095752 (0.0009) [2023-12-26 23:21:17,012][105692] Updated weights for policy 0, policy_version 1095762 (0.0007) [2023-12-26 23:21:17,071][105692] Updated weights for policy 0, policy_version 1095772 (0.0005) [2023-12-26 23:21:17,111][105620] Updated weights for policy 1, policy_version 1096801 (0.0009) [2023-12-26 23:21:17,175][105620] Updated weights for policy 1, policy_version 1096811 (0.0009) [2023-12-26 23:21:17,250][105620] Updated weights for policy 1, policy_version 1096821 (0.0010) [2023-12-26 23:21:17,325][105620] Updated weights for policy 1, policy_version 1096831 (0.0009) [2023-12-26 23:21:17,647][105692] Updated weights for policy 0, policy_version 1095782 (0.0007) [2023-12-26 23:21:17,702][105692] Updated weights for policy 0, policy_version 1095792 (0.0009) [2023-12-26 23:21:17,754][105692] Updated weights for policy 0, policy_version 1095802 (0.0009) [2023-12-26 23:21:18,115][105620] Updated weights for policy 1, policy_version 1096841 (0.0009) [2023-12-26 23:21:18,176][105620] Updated weights for policy 1, policy_version 1096851 (0.0008) [2023-12-26 23:21:18,234][105620] Updated weights for policy 1, policy_version 1096861 (0.0008) [2023-12-26 23:21:18,551][105692] Updated weights for policy 0, policy_version 1095812 (0.0007) [2023-12-26 23:21:18,621][105692] Updated weights for policy 0, policy_version 1095822 (0.0005) [2023-12-26 23:21:18,690][105692] Updated weights for policy 0, policy_version 1095832 (0.0006) [2023-12-26 23:21:19,014][105620] Updated weights for policy 1, policy_version 1096871 (0.0009) [2023-12-26 23:21:19,075][105620] Updated weights for policy 1, policy_version 1096881 (0.0010) [2023-12-26 23:21:19,126][105620] Updated weights for policy 1, policy_version 1096891 (0.0009) [2023-12-26 23:21:19,224][105692] Updated weights for policy 0, policy_version 1095842 (0.0006) [2023-12-26 23:21:19,294][105692] Updated weights for policy 0, policy_version 1095852 (0.0009) [2023-12-26 23:21:19,371][105692] Updated weights for policy 0, policy_version 1095863 (0.0010) [2023-12-26 23:21:19,926][105620] Updated weights for policy 1, policy_version 1096901 (0.0009) [2023-12-26 23:21:19,990][105620] Updated weights for policy 1, policy_version 1096911 (0.0008) [2023-12-26 23:21:20,054][105620] Updated weights for policy 1, policy_version 1096921 (0.0008) [2023-12-26 23:21:20,153][105692] Updated weights for policy 0, policy_version 1095873 (0.0009) [2023-12-26 23:21:20,205][105692] Updated weights for policy 0, policy_version 1095883 (0.0008) [2023-12-26 23:21:20,261][105692] Updated weights for policy 0, policy_version 1095893 (0.0008) [2023-12-26 23:21:20,315][105692] Updated weights for policy 0, policy_version 1095903 (0.0008) [2023-12-26 23:21:20,775][105620] Updated weights for policy 1, policy_version 1096931 (0.0007) [2023-12-26 23:21:20,831][105620] Updated weights for policy 1, policy_version 1096941 (0.0006) [2023-12-26 23:21:20,892][105620] Updated weights for policy 1, policy_version 1096951 (0.0006) [2023-12-26 23:21:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 561446912. Throughput: 0: 9775.0, 1: 9601.0. Samples: 561438292. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:21:21,063][104569] Avg episode reward: [(0, '9354.104'), (1, '9351.030')] [2023-12-26 23:21:21,156][105692] Updated weights for policy 0, policy_version 1095913 (0.0010) [2023-12-26 23:21:21,213][105692] Updated weights for policy 0, policy_version 1095923 (0.0009) [2023-12-26 23:21:21,275][105692] Updated weights for policy 0, policy_version 1095933 (0.0009) [2023-12-26 23:21:21,568][105620] Updated weights for policy 1, policy_version 1096961 (0.0010) [2023-12-26 23:21:21,625][105620] Updated weights for policy 1, policy_version 1096971 (0.0009) [2023-12-26 23:21:21,690][105620] Updated weights for policy 1, policy_version 1096981 (0.0010) [2023-12-26 23:21:21,755][105620] Updated weights for policy 1, policy_version 1096991 (0.0011) [2023-12-26 23:21:22,075][105692] Updated weights for policy 0, policy_version 1095943 (0.0010) [2023-12-26 23:21:22,131][105692] Updated weights for policy 0, policy_version 1095953 (0.0011) [2023-12-26 23:21:22,181][105692] Updated weights for policy 0, policy_version 1095963 (0.0011) [2023-12-26 23:21:22,512][105620] Updated weights for policy 1, policy_version 1097001 (0.0008) [2023-12-26 23:21:22,584][105620] Updated weights for policy 1, policy_version 1097011 (0.0007) [2023-12-26 23:21:22,646][105620] Updated weights for policy 1, policy_version 1097021 (0.0006) [2023-12-26 23:21:22,938][105692] Updated weights for policy 0, policy_version 1095973 (0.0009) [2023-12-26 23:21:23,001][105692] Updated weights for policy 0, policy_version 1095983 (0.0008) [2023-12-26 23:21:23,059][105692] Updated weights for policy 0, policy_version 1095993 (0.0008) [2023-12-26 23:21:23,282][105620] Updated weights for policy 1, policy_version 1097031 (0.0009) [2023-12-26 23:21:23,330][105620] Updated weights for policy 1, policy_version 1097041 (0.0010) [2023-12-26 23:21:23,388][105620] Updated weights for policy 1, policy_version 1097051 (0.0010) [2023-12-26 23:21:23,809][105692] Updated weights for policy 0, policy_version 1096003 (0.0008) [2023-12-26 23:21:23,863][105692] Updated weights for policy 0, policy_version 1096013 (0.0011) [2023-12-26 23:21:23,916][105692] Updated weights for policy 0, policy_version 1096023 (0.0010) [2023-12-26 23:21:23,957][105620] Updated weights for policy 1, policy_version 1097061 (0.0008) [2023-12-26 23:21:24,012][105620] Updated weights for policy 1, policy_version 1097071 (0.0005) [2023-12-26 23:21:24,067][105620] Updated weights for policy 1, policy_version 1097081 (0.0006) [2023-12-26 23:21:24,615][105620] Updated weights for policy 1, policy_version 1097091 (0.0006) [2023-12-26 23:21:24,680][105620] Updated weights for policy 1, policy_version 1097101 (0.0007) [2023-12-26 23:21:24,732][105620] Updated weights for policy 1, policy_version 1097111 (0.0009) [2023-12-26 23:21:24,790][105692] Updated weights for policy 0, policy_version 1096033 (0.0008) [2023-12-26 23:21:24,859][105692] Updated weights for policy 0, policy_version 1096043 (0.0009) [2023-12-26 23:21:24,913][105692] Updated weights for policy 0, policy_version 1096053 (0.0009) [2023-12-26 23:21:24,966][105692] Updated weights for policy 0, policy_version 1096063 (0.0010) [2023-12-26 23:21:25,378][105620] Updated weights for policy 1, policy_version 1097121 (0.0008) [2023-12-26 23:21:25,434][105620] Updated weights for policy 1, policy_version 1097131 (0.0005) [2023-12-26 23:21:25,485][105620] Updated weights for policy 1, policy_version 1097141 (0.0006) [2023-12-26 23:21:25,531][105620] Updated weights for policy 1, policy_version 1097151 (0.0005) [2023-12-26 23:21:25,689][105692] Updated weights for policy 0, policy_version 1096073 (0.0009) [2023-12-26 23:21:25,735][105692] Updated weights for policy 0, policy_version 1096083 (0.0008) [2023-12-26 23:21:25,788][105692] Updated weights for policy 0, policy_version 1096093 (0.0009) [2023-12-26 23:21:26,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 561545216. Throughput: 0: 9634.6, 1: 9704.2. Samples: 561555488. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:21:26,063][104569] Avg episode reward: [(0, '9353.566'), (1, '9351.056')] [2023-12-26 23:21:26,171][105620] Updated weights for policy 1, policy_version 1097161 (0.0008) [2023-12-26 23:21:26,230][105620] Updated weights for policy 1, policy_version 1097171 (0.0009) [2023-12-26 23:21:26,294][105620] Updated weights for policy 1, policy_version 1097181 (0.0010) [2023-12-26 23:21:26,568][105692] Updated weights for policy 0, policy_version 1096103 (0.0008) [2023-12-26 23:21:26,625][105692] Updated weights for policy 0, policy_version 1096114 (0.0010) [2023-12-26 23:21:26,682][105692] Updated weights for policy 0, policy_version 1096124 (0.0009) [2023-12-26 23:21:26,929][105620] Updated weights for policy 1, policy_version 1097191 (0.0007) [2023-12-26 23:21:26,983][105620] Updated weights for policy 1, policy_version 1097201 (0.0010) [2023-12-26 23:21:27,038][105620] Updated weights for policy 1, policy_version 1097211 (0.0010) [2023-12-26 23:21:27,431][105692] Updated weights for policy 0, policy_version 1096134 (0.0007) [2023-12-26 23:21:27,486][105692] Updated weights for policy 0, policy_version 1096144 (0.0006) [2023-12-26 23:21:27,554][105692] Updated weights for policy 0, policy_version 1096154 (0.0005) [2023-12-26 23:21:27,692][105620] Updated weights for policy 1, policy_version 1097221 (0.0008) [2023-12-26 23:21:27,754][105620] Updated weights for policy 1, policy_version 1097231 (0.0005) [2023-12-26 23:21:27,810][105620] Updated weights for policy 1, policy_version 1097241 (0.0005) [2023-12-26 23:21:28,129][105692] Updated weights for policy 0, policy_version 1096164 (0.0007) [2023-12-26 23:21:28,181][105692] Updated weights for policy 0, policy_version 1096174 (0.0010) [2023-12-26 23:21:28,232][105692] Updated weights for policy 0, policy_version 1096184 (0.0010) [2023-12-26 23:21:28,340][105620] Updated weights for policy 1, policy_version 1097251 (0.0006) [2023-12-26 23:21:28,400][105620] Updated weights for policy 1, policy_version 1097261 (0.0008) [2023-12-26 23:21:28,457][105620] Updated weights for policy 1, policy_version 1097271 (0.0010) [2023-12-26 23:21:28,870][105692] Updated weights for policy 0, policy_version 1096194 (0.0010) [2023-12-26 23:21:28,932][105692] Updated weights for policy 0, policy_version 1096204 (0.0010) [2023-12-26 23:21:28,992][105692] Updated weights for policy 0, policy_version 1096214 (0.0007) [2023-12-26 23:21:29,061][105692] Updated weights for policy 0, policy_version 1096224 (0.0005) [2023-12-26 23:21:29,142][105620] Updated weights for policy 1, policy_version 1097281 (0.0009) [2023-12-26 23:21:29,187][105620] Updated weights for policy 1, policy_version 1097291 (0.0010) [2023-12-26 23:21:29,254][105620] Updated weights for policy 1, policy_version 1097301 (0.0009) [2023-12-26 23:21:29,314][105620] Updated weights for policy 1, policy_version 1097311 (0.0006) [2023-12-26 23:21:29,632][105692] Updated weights for policy 0, policy_version 1096234 (0.0007) [2023-12-26 23:21:29,694][105692] Updated weights for policy 0, policy_version 1096244 (0.0005) [2023-12-26 23:21:29,751][105692] Updated weights for policy 0, policy_version 1096254 (0.0007) [2023-12-26 23:21:30,064][105620] Updated weights for policy 1, policy_version 1097321 (0.0008) [2023-12-26 23:21:30,116][105620] Updated weights for policy 1, policy_version 1097331 (0.0008) [2023-12-26 23:21:30,169][105620] Updated weights for policy 1, policy_version 1097341 (0.0007) [2023-12-26 23:21:30,424][105692] Updated weights for policy 0, policy_version 1096264 (0.0009) [2023-12-26 23:21:30,472][105692] Updated weights for policy 0, policy_version 1096274 (0.0010) [2023-12-26 23:21:30,525][105692] Updated weights for policy 0, policy_version 1096284 (0.0006) [2023-12-26 23:21:30,863][105620] Updated weights for policy 1, policy_version 1097351 (0.0006) [2023-12-26 23:21:30,908][105620] Updated weights for policy 1, policy_version 1097361 (0.0006) [2023-12-26 23:21:30,956][105620] Updated weights for policy 1, policy_version 1097371 (0.0005) [2023-12-26 23:21:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 561651712. Throughput: 0: 9659.3, 1: 9811.3. Samples: 561617148. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:21:31,062][104569] Avg episode reward: [(0, '9328.259'), (1, '9350.771')] [2023-12-26 23:21:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001096288_280690688.pth... [2023-12-26 23:21:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001097376_280961024.pth... [2023-12-26 23:21:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001095168_280403968.pth [2023-12-26 23:21:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001096192_280657920.pth [2023-12-26 23:21:31,121][105692] Updated weights for policy 0, policy_version 1096294 (0.0007) [2023-12-26 23:21:31,180][105692] Updated weights for policy 0, policy_version 1096304 (0.0008) [2023-12-26 23:21:31,244][105692] Updated weights for policy 0, policy_version 1096314 (0.0008) [2023-12-26 23:21:31,684][105620] Updated weights for policy 1, policy_version 1097381 (0.0006) [2023-12-26 23:21:31,753][105620] Updated weights for policy 1, policy_version 1097391 (0.0008) [2023-12-26 23:21:31,811][105620] Updated weights for policy 1, policy_version 1097401 (0.0009) [2023-12-26 23:21:31,952][105692] Updated weights for policy 0, policy_version 1096324 (0.0009) [2023-12-26 23:21:32,016][105692] Updated weights for policy 0, policy_version 1096334 (0.0006) [2023-12-26 23:21:32,066][105692] Updated weights for policy 0, policy_version 1096344 (0.0007) [2023-12-26 23:21:32,556][105620] Updated weights for policy 1, policy_version 1097412 (0.0012) [2023-12-26 23:21:32,606][105620] Updated weights for policy 1, policy_version 1097422 (0.0008) [2023-12-26 23:21:32,658][105620] Updated weights for policy 1, policy_version 1097432 (0.0006) [2023-12-26 23:21:32,718][105692] Updated weights for policy 0, policy_version 1096354 (0.0006) [2023-12-26 23:21:32,788][105692] Updated weights for policy 0, policy_version 1096365 (0.0010) [2023-12-26 23:21:32,849][105692] Updated weights for policy 0, policy_version 1096375 (0.0010) [2023-12-26 23:21:33,314][105620] Updated weights for policy 1, policy_version 1097442 (0.0006) [2023-12-26 23:21:33,375][105620] Updated weights for policy 1, policy_version 1097452 (0.0007) [2023-12-26 23:21:33,431][105620] Updated weights for policy 1, policy_version 1097462 (0.0005) [2023-12-26 23:21:33,489][105692] Updated weights for policy 0, policy_version 1096385 (0.0009) [2023-12-26 23:21:33,496][105620] Updated weights for policy 1, policy_version 1097472 (0.0005) [2023-12-26 23:21:33,545][105692] Updated weights for policy 0, policy_version 1096395 (0.0005) [2023-12-26 23:21:33,598][105692] Updated weights for policy 0, policy_version 1096405 (0.0005) [2023-12-26 23:21:33,647][105692] Updated weights for policy 0, policy_version 1096415 (0.0005) [2023-12-26 23:21:34,178][105620] Updated weights for policy 1, policy_version 1097482 (0.0009) [2023-12-26 23:21:34,214][105692] Updated weights for policy 0, policy_version 1096425 (0.0008) [2023-12-26 23:21:34,240][105620] Updated weights for policy 1, policy_version 1097492 (0.0007) [2023-12-26 23:21:34,278][105692] Updated weights for policy 0, policy_version 1096435 (0.0011) [2023-12-26 23:21:34,296][105620] Updated weights for policy 1, policy_version 1097502 (0.0005) [2023-12-26 23:21:34,334][105692] Updated weights for policy 0, policy_version 1096445 (0.0011) [2023-12-26 23:21:35,082][105620] Updated weights for policy 1, policy_version 1097512 (0.0006) [2023-12-26 23:21:35,084][105692] Updated weights for policy 0, policy_version 1096455 (0.0010) [2023-12-26 23:21:35,130][105620] Updated weights for policy 1, policy_version 1097522 (0.0006) [2023-12-26 23:21:35,139][105692] Updated weights for policy 0, policy_version 1096465 (0.0010) [2023-12-26 23:21:35,189][105620] Updated weights for policy 1, policy_version 1097532 (0.0005) [2023-12-26 23:21:35,202][105692] Updated weights for policy 0, policy_version 1096475 (0.0010) [2023-12-26 23:21:35,858][105692] Updated weights for policy 0, policy_version 1096485 (0.0009) [2023-12-26 23:21:35,915][105692] Updated weights for policy 0, policy_version 1096495 (0.0006) [2023-12-26 23:21:35,969][105692] Updated weights for policy 0, policy_version 1096505 (0.0008) [2023-12-26 23:21:36,011][105620] Updated weights for policy 1, policy_version 1097542 (0.0007) [2023-12-26 23:21:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 561750016. Throughput: 0: 9741.6, 1: 9783.6. Samples: 561740264. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:21:36,063][104569] Avg episode reward: [(0, '9237.206'), (1, '9349.943')] [2023-12-26 23:21:36,075][105620] Updated weights for policy 1, policy_version 1097552 (0.0009) [2023-12-26 23:21:36,141][105620] Updated weights for policy 1, policy_version 1097562 (0.0008) [2023-12-26 23:21:36,616][105692] Updated weights for policy 0, policy_version 1096515 (0.0006) [2023-12-26 23:21:36,684][105692] Updated weights for policy 0, policy_version 1096525 (0.0006) [2023-12-26 23:21:36,744][105692] Updated weights for policy 0, policy_version 1096535 (0.0009) [2023-12-26 23:21:36,960][105620] Updated weights for policy 1, policy_version 1097572 (0.0008) [2023-12-26 23:21:37,017][105620] Updated weights for policy 1, policy_version 1097582 (0.0009) [2023-12-26 23:21:37,075][105620] Updated weights for policy 1, policy_version 1097592 (0.0010) [2023-12-26 23:21:37,384][105692] Updated weights for policy 0, policy_version 1096545 (0.0009) [2023-12-26 23:21:37,455][105692] Updated weights for policy 0, policy_version 1096555 (0.0010) [2023-12-26 23:21:37,515][105585] KL-divergence is very high: 124.9776 [2023-12-26 23:21:37,520][105692] Updated weights for policy 0, policy_version 1096565 (0.0010) [2023-12-26 23:21:37,582][105692] Updated weights for policy 0, policy_version 1096575 (0.0010) [2023-12-26 23:21:37,828][105620] Updated weights for policy 1, policy_version 1097603 (0.0010) [2023-12-26 23:21:37,880][105620] Updated weights for policy 1, policy_version 1097613 (0.0009) [2023-12-26 23:21:37,932][105620] Updated weights for policy 1, policy_version 1097623 (0.0005) [2023-12-26 23:21:38,292][105692] Updated weights for policy 0, policy_version 1096585 (0.0010) [2023-12-26 23:21:38,351][105692] Updated weights for policy 0, policy_version 1096595 (0.0010) [2023-12-26 23:21:38,404][105692] Updated weights for policy 0, policy_version 1096605 (0.0010) [2023-12-26 23:21:38,539][105620] Updated weights for policy 1, policy_version 1097633 (0.0006) [2023-12-26 23:21:38,607][105620] Updated weights for policy 1, policy_version 1097643 (0.0011) [2023-12-26 23:21:38,663][105620] Updated weights for policy 1, policy_version 1097653 (0.0010) [2023-12-26 23:21:38,728][105620] Updated weights for policy 1, policy_version 1097663 (0.0009) [2023-12-26 23:21:39,115][105692] Updated weights for policy 0, policy_version 1096615 (0.0010) [2023-12-26 23:21:39,173][105692] Updated weights for policy 0, policy_version 1096625 (0.0010) [2023-12-26 23:21:39,231][105692] Updated weights for policy 0, policy_version 1096635 (0.0006) [2023-12-26 23:21:39,463][105620] Updated weights for policy 1, policy_version 1097673 (0.0008) [2023-12-26 23:21:39,519][105620] Updated weights for policy 1, policy_version 1097683 (0.0007) [2023-12-26 23:21:39,581][105620] Updated weights for policy 1, policy_version 1097693 (0.0008) [2023-12-26 23:21:39,979][105692] Updated weights for policy 0, policy_version 1096645 (0.0008) [2023-12-26 23:21:40,035][105692] Updated weights for policy 0, policy_version 1096655 (0.0008) [2023-12-26 23:21:40,100][105692] Updated weights for policy 0, policy_version 1096665 (0.0011) [2023-12-26 23:21:40,338][105620] Updated weights for policy 1, policy_version 1097703 (0.0009) [2023-12-26 23:21:40,407][105620] Updated weights for policy 1, policy_version 1097713 (0.0007) [2023-12-26 23:21:40,469][105620] Updated weights for policy 1, policy_version 1097723 (0.0007) [2023-12-26 23:21:40,821][105692] Updated weights for policy 0, policy_version 1096675 (0.0009) [2023-12-26 23:21:40,874][105692] Updated weights for policy 0, policy_version 1096685 (0.0007) [2023-12-26 23:21:40,924][105692] Updated weights for policy 0, policy_version 1096695 (0.0011) [2023-12-26 23:21:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 561848320. Throughput: 0: 9857.8, 1: 9737.1. Samples: 561855164. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:21:41,062][104569] Avg episode reward: [(0, '9171.499'), (1, '9258.404')] [2023-12-26 23:21:41,251][105620] Updated weights for policy 1, policy_version 1097733 (0.0010) [2023-12-26 23:21:41,303][105620] Updated weights for policy 1, policy_version 1097743 (0.0011) [2023-12-26 23:21:41,383][105620] Updated weights for policy 1, policy_version 1097753 (0.0010) [2023-12-26 23:21:41,668][105692] Updated weights for policy 0, policy_version 1096705 (0.0011) [2023-12-26 23:21:41,736][105692] Updated weights for policy 0, policy_version 1096715 (0.0010) [2023-12-26 23:21:41,792][105692] Updated weights for policy 0, policy_version 1096725 (0.0008) [2023-12-26 23:21:41,841][105692] Updated weights for policy 0, policy_version 1096735 (0.0008) [2023-12-26 23:21:42,114][105620] Updated weights for policy 1, policy_version 1097763 (0.0008) [2023-12-26 23:21:42,162][105620] Updated weights for policy 1, policy_version 1097773 (0.0010) [2023-12-26 23:21:42,210][105620] Updated weights for policy 1, policy_version 1097783 (0.0010) [2023-12-26 23:21:42,569][105692] Updated weights for policy 0, policy_version 1096745 (0.0010) [2023-12-26 23:21:42,637][105692] Updated weights for policy 0, policy_version 1096755 (0.0010) [2023-12-26 23:21:42,692][105692] Updated weights for policy 0, policy_version 1096765 (0.0010) [2023-12-26 23:21:42,981][105620] Updated weights for policy 1, policy_version 1097793 (0.0010) [2023-12-26 23:21:43,028][105620] Updated weights for policy 1, policy_version 1097803 (0.0005) [2023-12-26 23:21:43,072][105620] Updated weights for policy 1, policy_version 1097813 (0.0005) [2023-12-26 23:21:43,135][105620] Updated weights for policy 1, policy_version 1097823 (0.0005) [2023-12-26 23:21:43,409][105692] Updated weights for policy 0, policy_version 1096775 (0.0009) [2023-12-26 23:21:43,462][105692] Updated weights for policy 0, policy_version 1096785 (0.0005) [2023-12-26 23:21:43,516][105692] Updated weights for policy 0, policy_version 1096795 (0.0010) [2023-12-26 23:21:43,684][105620] Updated weights for policy 1, policy_version 1097833 (0.0005) [2023-12-26 23:21:43,734][105620] Updated weights for policy 1, policy_version 1097843 (0.0010) [2023-12-26 23:21:43,786][105620] Updated weights for policy 1, policy_version 1097853 (0.0008) [2023-12-26 23:21:44,167][105692] Updated weights for policy 0, policy_version 1096805 (0.0007) [2023-12-26 23:21:44,223][105692] Updated weights for policy 0, policy_version 1096815 (0.0008) [2023-12-26 23:21:44,283][105692] Updated weights for policy 0, policy_version 1096825 (0.0009) [2023-12-26 23:21:44,369][105620] Updated weights for policy 1, policy_version 1097863 (0.0005) [2023-12-26 23:21:44,423][105620] Updated weights for policy 1, policy_version 1097873 (0.0005) [2023-12-26 23:21:44,474][105620] Updated weights for policy 1, policy_version 1097883 (0.0005) [2023-12-26 23:21:45,067][105620] Updated weights for policy 1, policy_version 1097893 (0.0006) [2023-12-26 23:21:45,126][105620] Updated weights for policy 1, policy_version 1097903 (0.0007) [2023-12-26 23:21:45,155][105692] Updated weights for policy 0, policy_version 1096835 (0.0008) [2023-12-26 23:21:45,206][105620] Updated weights for policy 1, policy_version 1097913 (0.0010) [2023-12-26 23:21:45,228][105692] Updated weights for policy 0, policy_version 1096846 (0.0007) [2023-12-26 23:21:45,293][105692] Updated weights for policy 0, policy_version 1096856 (0.0007) [2023-12-26 23:21:45,937][105620] Updated weights for policy 1, policy_version 1097923 (0.0011) [2023-12-26 23:21:45,982][105620] Updated weights for policy 1, policy_version 1097933 (0.0010) [2023-12-26 23:21:46,030][105620] Updated weights for policy 1, policy_version 1097943 (0.0010) [2023-12-26 23:21:46,061][105692] Updated weights for policy 0, policy_version 1096866 (0.0008) [2023-12-26 23:21:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 561938432. Throughput: 0: 9802.9, 1: 9838.3. Samples: 561914296. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:21:46,063][104569] Avg episode reward: [(0, '9263.479'), (1, '9168.733')] [2023-12-26 23:21:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001097952_281108480.pth... [2023-12-26 23:21:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001096768_280805376.pth [2023-12-26 23:21:46,073][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_001097952_281108480.pth [2023-12-26 23:21:46,108][105692] Updated weights for policy 0, policy_version 1096876 (0.0007) [2023-12-26 23:21:46,156][105692] Updated weights for policy 0, policy_version 1096886 (0.0008) [2023-12-26 23:21:46,215][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001096896_280846336.pth... [2023-12-26 23:21:46,216][105692] Updated weights for policy 0, policy_version 1096896 (0.0008) [2023-12-26 23:21:46,220][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001095744_280551424.pth [2023-12-26 23:21:46,221][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_001096896_280846336.pth [2023-12-26 23:21:46,740][105620] Updated weights for policy 1, policy_version 1097953 (0.0010) [2023-12-26 23:21:46,792][105620] Updated weights for policy 1, policy_version 1097963 (0.0010) [2023-12-26 23:21:46,860][105620] Updated weights for policy 1, policy_version 1097973 (0.0009) [2023-12-26 23:21:46,924][105620] Updated weights for policy 1, policy_version 1097983 (0.0008) [2023-12-26 23:21:46,953][105692] Updated weights for policy 0, policy_version 1096906 (0.0006) [2023-12-26 23:21:47,018][105692] Updated weights for policy 0, policy_version 1096916 (0.0007) [2023-12-26 23:21:47,076][105692] Updated weights for policy 0, policy_version 1096926 (0.0008) [2023-12-26 23:21:47,653][105620] Updated weights for policy 1, policy_version 1097993 (0.0010) [2023-12-26 23:21:47,702][105620] Updated weights for policy 1, policy_version 1098003 (0.0010) [2023-12-26 23:21:47,747][105620] Updated weights for policy 1, policy_version 1098013 (0.0010) [2023-12-26 23:21:47,793][105692] Updated weights for policy 0, policy_version 1096936 (0.0008) [2023-12-26 23:21:47,851][105692] Updated weights for policy 0, policy_version 1096946 (0.0009) [2023-12-26 23:21:47,906][105692] Updated weights for policy 0, policy_version 1096956 (0.0008) [2023-12-26 23:21:48,450][105620] Updated weights for policy 1, policy_version 1098023 (0.0009) [2023-12-26 23:21:48,506][105620] Updated weights for policy 1, policy_version 1098033 (0.0008) [2023-12-26 23:21:48,555][105620] Updated weights for policy 1, policy_version 1098043 (0.0008) [2023-12-26 23:21:48,699][105692] Updated weights for policy 0, policy_version 1096966 (0.0010) [2023-12-26 23:21:48,755][105692] Updated weights for policy 0, policy_version 1096976 (0.0010) [2023-12-26 23:21:48,807][105692] Updated weights for policy 0, policy_version 1096986 (0.0010) [2023-12-26 23:21:49,405][105620] Updated weights for policy 1, policy_version 1098053 (0.0008) [2023-12-26 23:21:49,457][105692] Updated weights for policy 0, policy_version 1096996 (0.0010) [2023-12-26 23:21:49,459][105620] Updated weights for policy 1, policy_version 1098063 (0.0009) [2023-12-26 23:21:49,512][105620] Updated weights for policy 1, policy_version 1098073 (0.0006) [2023-12-26 23:21:49,514][105692] Updated weights for policy 0, policy_version 1097006 (0.0011) [2023-12-26 23:21:49,571][105692] Updated weights for policy 0, policy_version 1097016 (0.0010) [2023-12-26 23:21:50,142][105620] Updated weights for policy 1, policy_version 1098083 (0.0006) [2023-12-26 23:21:50,204][105620] Updated weights for policy 1, policy_version 1098093 (0.0008) [2023-12-26 23:21:50,262][105620] Updated weights for policy 1, policy_version 1098103 (0.0009) [2023-12-26 23:21:50,312][105692] Updated weights for policy 0, policy_version 1097026 (0.0010) [2023-12-26 23:21:50,375][105692] Updated weights for policy 0, policy_version 1097036 (0.0008) [2023-12-26 23:21:50,440][105692] Updated weights for policy 0, policy_version 1097046 (0.0010) [2023-12-26 23:21:50,503][105692] Updated weights for policy 0, policy_version 1097056 (0.0007) [2023-12-26 23:21:50,927][105620] Updated weights for policy 1, policy_version 1098113 (0.0008) [2023-12-26 23:21:50,990][105620] Updated weights for policy 1, policy_version 1098123 (0.0011) [2023-12-26 23:21:51,058][105620] Updated weights for policy 1, policy_version 1098133 (0.0009) [2023-12-26 23:21:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 562036736. Throughput: 0: 9820.4, 1: 9890.2. Samples: 562030772. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-26 23:21:51,062][104569] Avg episode reward: [(0, '9185.134'), (1, '9260.597')] [2023-12-26 23:21:51,120][105620] Updated weights for policy 1, policy_version 1098143 (0.0006) [2023-12-26 23:21:51,211][105692] Updated weights for policy 0, policy_version 1097066 (0.0006) [2023-12-26 23:21:51,279][105692] Updated weights for policy 0, policy_version 1097076 (0.0009) [2023-12-26 23:21:51,336][105692] Updated weights for policy 0, policy_version 1097086 (0.0009) [2023-12-26 23:21:51,813][105620] Updated weights for policy 1, policy_version 1098153 (0.0011) [2023-12-26 23:21:51,862][105620] Updated weights for policy 1, policy_version 1098163 (0.0010) [2023-12-26 23:21:51,912][105620] Updated weights for policy 1, policy_version 1098173 (0.0009) [2023-12-26 23:21:52,100][105692] Updated weights for policy 0, policy_version 1097096 (0.0010) [2023-12-26 23:21:52,159][105692] Updated weights for policy 0, policy_version 1097106 (0.0010) [2023-12-26 23:21:52,228][105692] Updated weights for policy 0, policy_version 1097116 (0.0011) [2023-12-26 23:21:52,564][105620] Updated weights for policy 1, policy_version 1098183 (0.0009) [2023-12-26 23:21:52,619][105620] Updated weights for policy 1, policy_version 1098193 (0.0010) [2023-12-26 23:21:52,681][105620] Updated weights for policy 1, policy_version 1098203 (0.0011) [2023-12-26 23:21:52,945][105692] Updated weights for policy 0, policy_version 1097126 (0.0011) [2023-12-26 23:21:53,006][105692] Updated weights for policy 0, policy_version 1097136 (0.0010) [2023-12-26 23:21:53,068][105692] Updated weights for policy 0, policy_version 1097146 (0.0010) [2023-12-26 23:21:53,346][105620] Updated weights for policy 1, policy_version 1098213 (0.0011) [2023-12-26 23:21:53,394][105620] Updated weights for policy 1, policy_version 1098223 (0.0010) [2023-12-26 23:21:53,442][105620] Updated weights for policy 1, policy_version 1098233 (0.0010) [2023-12-26 23:21:53,655][105692] Updated weights for policy 0, policy_version 1097156 (0.0008) [2023-12-26 23:21:53,708][105692] Updated weights for policy 0, policy_version 1097166 (0.0005) [2023-12-26 23:21:53,767][105692] Updated weights for policy 0, policy_version 1097176 (0.0005) [2023-12-26 23:21:54,174][105620] Updated weights for policy 1, policy_version 1098243 (0.0010) [2023-12-26 23:21:54,230][105620] Updated weights for policy 1, policy_version 1098253 (0.0011) [2023-12-26 23:21:54,282][105620] Updated weights for policy 1, policy_version 1098263 (0.0010) [2023-12-26 23:21:54,365][105692] Updated weights for policy 0, policy_version 1097186 (0.0006) [2023-12-26 23:21:54,420][105692] Updated weights for policy 0, policy_version 1097196 (0.0011) [2023-12-26 23:21:54,478][105692] Updated weights for policy 0, policy_version 1097206 (0.0010) [2023-12-26 23:21:54,547][105692] Updated weights for policy 0, policy_version 1097216 (0.0010) [2023-12-26 23:21:54,901][105620] Updated weights for policy 1, policy_version 1098273 (0.0010) [2023-12-26 23:21:54,968][105620] Updated weights for policy 1, policy_version 1098283 (0.0006) [2023-12-26 23:21:55,019][105620] Updated weights for policy 1, policy_version 1098293 (0.0006) [2023-12-26 23:21:55,074][105620] Updated weights for policy 1, policy_version 1098303 (0.0010) [2023-12-26 23:21:55,262][105692] Updated weights for policy 0, policy_version 1097226 (0.0010) [2023-12-26 23:21:55,313][105692] Updated weights for policy 0, policy_version 1097236 (0.0010) [2023-12-26 23:21:55,374][105692] Updated weights for policy 0, policy_version 1097246 (0.0010) [2023-12-26 23:21:55,623][105620] Updated weights for policy 1, policy_version 1098313 (0.0009) [2023-12-26 23:21:55,686][105620] Updated weights for policy 1, policy_version 1098323 (0.0008) [2023-12-26 23:21:55,748][105620] Updated weights for policy 1, policy_version 1098333 (0.0005) [2023-12-26 23:21:56,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 562143232. Throughput: 0: 9790.7, 1: 9936.4. Samples: 562154124. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:21:56,063][104569] Avg episode reward: [(0, '9185.161'), (1, '9260.004')] [2023-12-26 23:21:56,085][105692] Updated weights for policy 0, policy_version 1097256 (0.0011) [2023-12-26 23:21:56,144][105692] Updated weights for policy 0, policy_version 1097266 (0.0010) [2023-12-26 23:21:56,196][105692] Updated weights for policy 0, policy_version 1097276 (0.0010) [2023-12-26 23:21:56,383][105620] Updated weights for policy 1, policy_version 1098343 (0.0009) [2023-12-26 23:21:56,434][105620] Updated weights for policy 1, policy_version 1098353 (0.0010) [2023-12-26 23:21:56,488][105620] Updated weights for policy 1, policy_version 1098363 (0.0010) [2023-12-26 23:21:56,877][105692] Updated weights for policy 0, policy_version 1097286 (0.0008) [2023-12-26 23:21:56,928][105692] Updated weights for policy 0, policy_version 1097296 (0.0005) [2023-12-26 23:21:56,978][105692] Updated weights for policy 0, policy_version 1097306 (0.0005) [2023-12-26 23:21:57,254][105620] Updated weights for policy 1, policy_version 1098373 (0.0008) [2023-12-26 23:21:57,310][105620] Updated weights for policy 1, policy_version 1098383 (0.0008) [2023-12-26 23:21:57,362][105620] Updated weights for policy 1, policy_version 1098393 (0.0008) [2023-12-26 23:21:57,602][105692] Updated weights for policy 0, policy_version 1097316 (0.0005) [2023-12-26 23:21:57,669][105692] Updated weights for policy 0, policy_version 1097326 (0.0007) [2023-12-26 23:21:57,726][105692] Updated weights for policy 0, policy_version 1097336 (0.0010) [2023-12-26 23:21:58,008][105620] Updated weights for policy 1, policy_version 1098403 (0.0007) [2023-12-26 23:21:58,064][105620] Updated weights for policy 1, policy_version 1098413 (0.0005) [2023-12-26 23:21:58,124][105620] Updated weights for policy 1, policy_version 1098423 (0.0006) [2023-12-26 23:21:58,365][105692] Updated weights for policy 0, policy_version 1097346 (0.0010) [2023-12-26 23:21:58,431][105692] Updated weights for policy 0, policy_version 1097356 (0.0009) [2023-12-26 23:21:58,496][105692] Updated weights for policy 0, policy_version 1097366 (0.0006) [2023-12-26 23:21:58,563][105692] Updated weights for policy 0, policy_version 1097376 (0.0009) [2023-12-26 23:21:58,946][105620] Updated weights for policy 1, policy_version 1098433 (0.0009) [2023-12-26 23:21:59,010][105620] Updated weights for policy 1, policy_version 1098443 (0.0008) [2023-12-26 23:21:59,073][105620] Updated weights for policy 1, policy_version 1098453 (0.0006) [2023-12-26 23:21:59,135][105620] Updated weights for policy 1, policy_version 1098463 (0.0007) [2023-12-26 23:21:59,302][105692] Updated weights for policy 0, policy_version 1097386 (0.0010) [2023-12-26 23:21:59,359][105692] Updated weights for policy 0, policy_version 1097396 (0.0010) [2023-12-26 23:21:59,420][105692] Updated weights for policy 0, policy_version 1097406 (0.0007) [2023-12-26 23:21:59,842][105620] Updated weights for policy 1, policy_version 1098473 (0.0010) [2023-12-26 23:21:59,904][105620] Updated weights for policy 1, policy_version 1098483 (0.0010) [2023-12-26 23:21:59,956][105620] Updated weights for policy 1, policy_version 1098493 (0.0009) [2023-12-26 23:22:00,126][105692] Updated weights for policy 0, policy_version 1097416 (0.0008) [2023-12-26 23:22:00,185][105692] Updated weights for policy 0, policy_version 1097426 (0.0009) [2023-12-26 23:22:00,256][105692] Updated weights for policy 0, policy_version 1097436 (0.0005) [2023-12-26 23:22:00,752][105620] Updated weights for policy 1, policy_version 1098503 (0.0008) [2023-12-26 23:22:00,811][105620] Updated weights for policy 1, policy_version 1098513 (0.0005) [2023-12-26 23:22:00,865][105620] Updated weights for policy 1, policy_version 1098523 (0.0005) [2023-12-26 23:22:00,917][105692] Updated weights for policy 0, policy_version 1097446 (0.0008) [2023-12-26 23:22:00,971][105692] Updated weights for policy 0, policy_version 1097457 (0.0011) [2023-12-26 23:22:01,025][105692] Updated weights for policy 0, policy_version 1097467 (0.0010) [2023-12-26 23:22:01,062][104569] Fps is (10 sec: 21299.0, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 562249728. Throughput: 0: 9879.4, 1: 9944.1. Samples: 562214228. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:22:01,062][104569] Avg episode reward: [(0, '9354.820'), (1, '9259.770')] [2023-12-26 23:22:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001097472_280993792.pth... [2023-12-26 23:22:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001098528_281255936.pth... [2023-12-26 23:22:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001097376_280961024.pth [2023-12-26 23:22:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001096288_280690688.pth [2023-12-26 23:22:01,521][105620] Updated weights for policy 1, policy_version 1098533 (0.0007) [2023-12-26 23:22:01,585][105620] Updated weights for policy 1, policy_version 1098543 (0.0008) [2023-12-26 23:22:01,641][105620] Updated weights for policy 1, policy_version 1098553 (0.0008) [2023-12-26 23:22:01,862][105692] Updated weights for policy 0, policy_version 1097477 (0.0009) [2023-12-26 23:22:01,926][105692] Updated weights for policy 0, policy_version 1097487 (0.0009) [2023-12-26 23:22:01,981][105692] Updated weights for policy 0, policy_version 1097497 (0.0009) [2023-12-26 23:22:02,406][105620] Updated weights for policy 1, policy_version 1098563 (0.0008) [2023-12-26 23:22:02,459][105620] Updated weights for policy 1, policy_version 1098573 (0.0009) [2023-12-26 23:22:02,506][105620] Updated weights for policy 1, policy_version 1098583 (0.0009) [2023-12-26 23:22:02,711][105692] Updated weights for policy 0, policy_version 1097507 (0.0009) [2023-12-26 23:22:02,762][105692] Updated weights for policy 0, policy_version 1097517 (0.0009) [2023-12-26 23:22:02,817][105692] Updated weights for policy 0, policy_version 1097527 (0.0009) [2023-12-26 23:22:03,269][105620] Updated weights for policy 1, policy_version 1098593 (0.0009) [2023-12-26 23:22:03,326][105620] Updated weights for policy 1, policy_version 1098603 (0.0005) [2023-12-26 23:22:03,386][105620] Updated weights for policy 1, policy_version 1098613 (0.0005) [2023-12-26 23:22:03,439][105620] Updated weights for policy 1, policy_version 1098623 (0.0005) [2023-12-26 23:22:03,562][105692] Updated weights for policy 0, policy_version 1097537 (0.0006) [2023-12-26 23:22:03,612][105692] Updated weights for policy 0, policy_version 1097547 (0.0009) [2023-12-26 23:22:03,659][105692] Updated weights for policy 0, policy_version 1097557 (0.0009) [2023-12-26 23:22:03,706][105692] Updated weights for policy 0, policy_version 1097567 (0.0009) [2023-12-26 23:22:04,090][105620] Updated weights for policy 1, policy_version 1098633 (0.0008) [2023-12-26 23:22:04,150][105620] Updated weights for policy 1, policy_version 1098643 (0.0008) [2023-12-26 23:22:04,210][105620] Updated weights for policy 1, policy_version 1098653 (0.0008) [2023-12-26 23:22:04,438][105692] Updated weights for policy 0, policy_version 1097577 (0.0011) [2023-12-26 23:22:04,487][105692] Updated weights for policy 0, policy_version 1097587 (0.0011) [2023-12-26 23:22:04,540][105692] Updated weights for policy 0, policy_version 1097597 (0.0010) [2023-12-26 23:22:04,988][105620] Updated weights for policy 1, policy_version 1098663 (0.0006) [2023-12-26 23:22:05,050][105620] Updated weights for policy 1, policy_version 1098673 (0.0005) [2023-12-26 23:22:05,114][105620] Updated weights for policy 1, policy_version 1098683 (0.0005) [2023-12-26 23:22:05,263][105692] Updated weights for policy 0, policy_version 1097607 (0.0010) [2023-12-26 23:22:05,312][105692] Updated weights for policy 0, policy_version 1097617 (0.0010) [2023-12-26 23:22:05,368][105692] Updated weights for policy 0, policy_version 1097627 (0.0010) [2023-12-26 23:22:05,642][105620] Updated weights for policy 1, policy_version 1098693 (0.0006) [2023-12-26 23:22:05,701][105620] Updated weights for policy 1, policy_version 1098703 (0.0005) [2023-12-26 23:22:05,759][105620] Updated weights for policy 1, policy_version 1098713 (0.0005) [2023-12-26 23:22:06,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 562339840. Throughput: 0: 9795.8, 1: 9984.8. Samples: 562328416. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:22:06,062][104569] Avg episode reward: [(0, '9354.731'), (1, '9167.784')] [2023-12-26 23:22:06,125][105692] Updated weights for policy 0, policy_version 1097637 (0.0010) [2023-12-26 23:22:06,189][105692] Updated weights for policy 0, policy_version 1097647 (0.0010) [2023-12-26 23:22:06,247][105692] Updated weights for policy 0, policy_version 1097657 (0.0010) [2023-12-26 23:22:06,306][105620] Updated weights for policy 1, policy_version 1098723 (0.0006) [2023-12-26 23:22:06,354][105620] Updated weights for policy 1, policy_version 1098733 (0.0007) [2023-12-26 23:22:06,406][105620] Updated weights for policy 1, policy_version 1098743 (0.0008) [2023-12-26 23:22:06,981][105692] Updated weights for policy 0, policy_version 1097667 (0.0011) [2023-12-26 23:22:07,029][105692] Updated weights for policy 0, policy_version 1097677 (0.0010) [2023-12-26 23:22:07,091][105692] Updated weights for policy 0, policy_version 1097687 (0.0010) [2023-12-26 23:22:07,202][105620] Updated weights for policy 1, policy_version 1098753 (0.0008) [2023-12-26 23:22:07,267][105620] Updated weights for policy 1, policy_version 1098763 (0.0008) [2023-12-26 23:22:07,337][105620] Updated weights for policy 1, policy_version 1098773 (0.0008) [2023-12-26 23:22:07,396][105620] Updated weights for policy 1, policy_version 1098783 (0.0008) [2023-12-26 23:22:07,858][105692] Updated weights for policy 0, policy_version 1097697 (0.0011) [2023-12-26 23:22:07,916][105692] Updated weights for policy 0, policy_version 1097707 (0.0010) [2023-12-26 23:22:07,973][105692] Updated weights for policy 0, policy_version 1097717 (0.0010) [2023-12-26 23:22:08,024][105692] Updated weights for policy 0, policy_version 1097727 (0.0010) [2023-12-26 23:22:08,132][105620] Updated weights for policy 1, policy_version 1098793 (0.0008) [2023-12-26 23:22:08,188][105620] Updated weights for policy 1, policy_version 1098803 (0.0008) [2023-12-26 23:22:08,236][105620] Updated weights for policy 1, policy_version 1098813 (0.0008) [2023-12-26 23:22:08,766][105692] Updated weights for policy 0, policy_version 1097737 (0.0009) [2023-12-26 23:22:08,819][105692] Updated weights for policy 0, policy_version 1097747 (0.0006) [2023-12-26 23:22:08,871][105692] Updated weights for policy 0, policy_version 1097757 (0.0009) [2023-12-26 23:22:09,019][105620] Updated weights for policy 1, policy_version 1098823 (0.0008) [2023-12-26 23:22:09,078][105620] Updated weights for policy 1, policy_version 1098833 (0.0007) [2023-12-26 23:22:09,141][105620] Updated weights for policy 1, policy_version 1098843 (0.0010) [2023-12-26 23:22:09,636][105692] Updated weights for policy 0, policy_version 1097767 (0.0008) [2023-12-26 23:22:09,703][105692] Updated weights for policy 0, policy_version 1097777 (0.0006) [2023-12-26 23:22:09,763][105692] Updated weights for policy 0, policy_version 1097787 (0.0008) [2023-12-26 23:22:09,890][105620] Updated weights for policy 1, policy_version 1098853 (0.0007) [2023-12-26 23:22:09,950][105620] Updated weights for policy 1, policy_version 1098863 (0.0008) [2023-12-26 23:22:10,014][105620] Updated weights for policy 1, policy_version 1098873 (0.0006) [2023-12-26 23:22:10,391][105692] Updated weights for policy 0, policy_version 1097797 (0.0008) [2023-12-26 23:22:10,449][105692] Updated weights for policy 0, policy_version 1097807 (0.0006) [2023-12-26 23:22:10,514][105692] Updated weights for policy 0, policy_version 1097817 (0.0007) [2023-12-26 23:22:10,588][105620] Updated weights for policy 1, policy_version 1098883 (0.0008) [2023-12-26 23:22:10,649][105620] Updated weights for policy 1, policy_version 1098893 (0.0005) [2023-12-26 23:22:10,712][105620] Updated weights for policy 1, policy_version 1098903 (0.0005) [2023-12-26 23:22:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 562438144. Throughput: 0: 9876.5, 1: 9954.9. Samples: 562447896. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:22:11,063][104569] Avg episode reward: [(0, '9096.298'), (1, '9166.673')] [2023-12-26 23:22:11,090][105692] Updated weights for policy 0, policy_version 1097827 (0.0007) [2023-12-26 23:22:11,157][105692] Updated weights for policy 0, policy_version 1097837 (0.0010) [2023-12-26 23:22:11,228][105692] Updated weights for policy 0, policy_version 1097847 (0.0009) [2023-12-26 23:22:11,277][105620] Updated weights for policy 1, policy_version 1098913 (0.0006) [2023-12-26 23:22:11,330][105620] Updated weights for policy 1, policy_version 1098923 (0.0008) [2023-12-26 23:22:11,392][105620] Updated weights for policy 1, policy_version 1098933 (0.0009) [2023-12-26 23:22:11,465][105620] Updated weights for policy 1, policy_version 1098943 (0.0010) [2023-12-26 23:22:11,996][105692] Updated weights for policy 0, policy_version 1097857 (0.0010) [2023-12-26 23:22:12,067][105692] Updated weights for policy 0, policy_version 1097867 (0.0005) [2023-12-26 23:22:12,140][105692] Updated weights for policy 0, policy_version 1097877 (0.0006) [2023-12-26 23:22:12,211][105692] Updated weights for policy 0, policy_version 1097887 (0.0007) [2023-12-26 23:22:12,317][105620] Updated weights for policy 1, policy_version 1098953 (0.0011) [2023-12-26 23:22:12,384][105620] Updated weights for policy 1, policy_version 1098963 (0.0009) [2023-12-26 23:22:12,449][105620] Updated weights for policy 1, policy_version 1098973 (0.0009) [2023-12-26 23:22:12,792][105692] Updated weights for policy 0, policy_version 1097897 (0.0009) [2023-12-26 23:22:12,845][105692] Updated weights for policy 0, policy_version 1097907 (0.0009) [2023-12-26 23:22:12,892][105692] Updated weights for policy 0, policy_version 1097917 (0.0009) [2023-12-26 23:22:13,188][105620] Updated weights for policy 1, policy_version 1098983 (0.0009) [2023-12-26 23:22:13,242][105620] Updated weights for policy 1, policy_version 1098993 (0.0009) [2023-12-26 23:22:13,290][105620] Updated weights for policy 1, policy_version 1099003 (0.0009) [2023-12-26 23:22:13,577][105692] Updated weights for policy 0, policy_version 1097927 (0.0006) [2023-12-26 23:22:13,633][105692] Updated weights for policy 0, policy_version 1097937 (0.0005) [2023-12-26 23:22:13,686][105692] Updated weights for policy 0, policy_version 1097947 (0.0005) [2023-12-26 23:22:14,191][105692] Updated weights for policy 0, policy_version 1097957 (0.0006) [2023-12-26 23:22:14,200][105620] Updated weights for policy 1, policy_version 1099013 (0.0008) [2023-12-26 23:22:14,248][105692] Updated weights for policy 0, policy_version 1097967 (0.0008) [2023-12-26 23:22:14,266][105620] Updated weights for policy 1, policy_version 1099023 (0.0011) [2023-12-26 23:22:14,303][105692] Updated weights for policy 0, policy_version 1097977 (0.0008) [2023-12-26 23:22:14,321][105620] Updated weights for policy 1, policy_version 1099033 (0.0010) [2023-12-26 23:22:15,048][105620] Updated weights for policy 1, policy_version 1099043 (0.0008) [2023-12-26 23:22:15,081][105692] Updated weights for policy 0, policy_version 1097987 (0.0007) [2023-12-26 23:22:15,104][105620] Updated weights for policy 1, policy_version 1099053 (0.0006) [2023-12-26 23:22:15,137][105692] Updated weights for policy 0, policy_version 1097997 (0.0009) [2023-12-26 23:22:15,165][105620] Updated weights for policy 1, policy_version 1099063 (0.0006) [2023-12-26 23:22:15,196][105692] Updated weights for policy 0, policy_version 1098007 (0.0006) [2023-12-26 23:22:15,776][105620] Updated weights for policy 1, policy_version 1099073 (0.0008) [2023-12-26 23:22:15,835][105620] Updated weights for policy 1, policy_version 1099083 (0.0011) [2023-12-26 23:22:15,884][105620] Updated weights for policy 1, policy_version 1099093 (0.0010) [2023-12-26 23:22:15,930][105620] Updated weights for policy 1, policy_version 1099103 (0.0009) [2023-12-26 23:22:16,046][105692] Updated weights for policy 0, policy_version 1098017 (0.0007) [2023-12-26 23:22:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 562536448. Throughput: 0: 9905.6, 1: 9836.8. Samples: 562505556. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:22:16,063][104569] Avg episode reward: [(0, '9179.579'), (1, '9074.653')] [2023-12-26 23:22:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001099104_281403392.pth... [2023-12-26 23:22:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001097952_281108480.pth [2023-12-26 23:22:16,099][105692] Updated weights for policy 0, policy_version 1098027 (0.0009) [2023-12-26 23:22:16,151][105692] Updated weights for policy 0, policy_version 1098037 (0.0010) [2023-12-26 23:22:16,197][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001098048_281141248.pth... [2023-12-26 23:22:16,200][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001096896_280846336.pth [2023-12-26 23:22:16,647][105620] Updated weights for policy 1, policy_version 1099113 (0.0008) [2023-12-26 23:22:16,693][105620] Updated weights for policy 1, policy_version 1099123 (0.0005) [2023-12-26 23:22:16,741][105620] Updated weights for policy 1, policy_version 1099133 (0.0005) [2023-12-26 23:22:16,953][105692] Updated weights for policy 0, policy_version 1098049 (0.0010) [2023-12-26 23:22:17,006][105692] Updated weights for policy 0, policy_version 1098059 (0.0009) [2023-12-26 23:22:17,061][105692] Updated weights for policy 0, policy_version 1098069 (0.0008) [2023-12-26 23:22:17,120][105692] Updated weights for policy 0, policy_version 1098079 (0.0009) [2023-12-26 23:22:17,485][105620] Updated weights for policy 1, policy_version 1099143 (0.0008) [2023-12-26 23:22:17,531][105620] Updated weights for policy 1, policy_version 1099153 (0.0009) [2023-12-26 23:22:17,582][105620] Updated weights for policy 1, policy_version 1099163 (0.0009) [2023-12-26 23:22:17,927][105692] Updated weights for policy 0, policy_version 1098089 (0.0009) [2023-12-26 23:22:17,988][105692] Updated weights for policy 0, policy_version 1098099 (0.0010) [2023-12-26 23:22:18,050][105692] Updated weights for policy 0, policy_version 1098109 (0.0011) [2023-12-26 23:22:18,285][105620] Updated weights for policy 1, policy_version 1099173 (0.0009) [2023-12-26 23:22:18,348][105620] Updated weights for policy 1, policy_version 1099183 (0.0010) [2023-12-26 23:22:18,416][105620] Updated weights for policy 1, policy_version 1099193 (0.0006) [2023-12-26 23:22:18,824][105692] Updated weights for policy 0, policy_version 1098119 (0.0009) [2023-12-26 23:22:18,883][105692] Updated weights for policy 0, policy_version 1098129 (0.0008) [2023-12-26 23:22:18,941][105692] Updated weights for policy 0, policy_version 1098139 (0.0006) [2023-12-26 23:22:19,127][105620] Updated weights for policy 1, policy_version 1099203 (0.0008) [2023-12-26 23:22:19,186][105620] Updated weights for policy 1, policy_version 1099213 (0.0010) [2023-12-26 23:22:19,251][105620] Updated weights for policy 1, policy_version 1099223 (0.0010) [2023-12-26 23:22:19,582][105692] Updated weights for policy 0, policy_version 1098149 (0.0007) [2023-12-26 23:22:19,643][105692] Updated weights for policy 0, policy_version 1098159 (0.0009) [2023-12-26 23:22:19,696][105692] Updated weights for policy 0, policy_version 1098169 (0.0010) [2023-12-26 23:22:20,019][105620] Updated weights for policy 1, policy_version 1099233 (0.0010) [2023-12-26 23:22:20,075][105620] Updated weights for policy 1, policy_version 1099243 (0.0010) [2023-12-26 23:22:20,129][105620] Updated weights for policy 1, policy_version 1099253 (0.0010) [2023-12-26 23:22:20,189][105620] Updated weights for policy 1, policy_version 1099263 (0.0010) [2023-12-26 23:22:20,458][105692] Updated weights for policy 0, policy_version 1098179 (0.0010) [2023-12-26 23:22:20,512][105692] Updated weights for policy 0, policy_version 1098189 (0.0009) [2023-12-26 23:22:20,571][105692] Updated weights for policy 0, policy_version 1098199 (0.0010) [2023-12-26 23:22:20,970][105620] Updated weights for policy 1, policy_version 1099273 (0.0011) [2023-12-26 23:22:21,037][105620] Updated weights for policy 1, policy_version 1099283 (0.0008) [2023-12-26 23:22:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 562626560. Throughput: 0: 9753.3, 1: 9828.8. Samples: 562621456. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:22:21,063][104569] Avg episode reward: [(0, '9261.621'), (1, '8983.635')] [2023-12-26 23:22:21,106][105620] Updated weights for policy 1, policy_version 1099293 (0.0007) [2023-12-26 23:22:21,355][105692] Updated weights for policy 0, policy_version 1098209 (0.0009) [2023-12-26 23:22:21,418][105692] Updated weights for policy 0, policy_version 1098219 (0.0010) [2023-12-26 23:22:21,478][105692] Updated weights for policy 0, policy_version 1098229 (0.0008) [2023-12-26 23:22:21,539][105692] Updated weights for policy 0, policy_version 1098239 (0.0010) [2023-12-26 23:22:21,866][105620] Updated weights for policy 1, policy_version 1099303 (0.0007) [2023-12-26 23:22:21,915][105620] Updated weights for policy 1, policy_version 1099313 (0.0008) [2023-12-26 23:22:21,969][105620] Updated weights for policy 1, policy_version 1099323 (0.0007) [2023-12-26 23:22:22,297][105692] Updated weights for policy 0, policy_version 1098249 (0.0009) [2023-12-26 23:22:22,352][105692] Updated weights for policy 0, policy_version 1098259 (0.0008) [2023-12-26 23:22:22,421][105692] Updated weights for policy 0, policy_version 1098269 (0.0008) [2023-12-26 23:22:22,675][105620] Updated weights for policy 1, policy_version 1099333 (0.0009) [2023-12-26 23:22:22,732][105620] Updated weights for policy 1, policy_version 1099343 (0.0008) [2023-12-26 23:22:22,797][105620] Updated weights for policy 1, policy_version 1099353 (0.0009) [2023-12-26 23:22:23,165][105692] Updated weights for policy 0, policy_version 1098279 (0.0008) [2023-12-26 23:22:23,221][105692] Updated weights for policy 0, policy_version 1098289 (0.0009) [2023-12-26 23:22:23,276][105692] Updated weights for policy 0, policy_version 1098299 (0.0009) [2023-12-26 23:22:23,512][105620] Updated weights for policy 1, policy_version 1099363 (0.0008) [2023-12-26 23:22:23,558][105620] Updated weights for policy 1, policy_version 1099373 (0.0009) [2023-12-26 23:22:23,605][105620] Updated weights for policy 1, policy_version 1099383 (0.0009) [2023-12-26 23:22:23,970][105692] Updated weights for policy 0, policy_version 1098309 (0.0009) [2023-12-26 23:22:24,021][105692] Updated weights for policy 0, policy_version 1098319 (0.0008) [2023-12-26 23:22:24,065][105692] Updated weights for policy 0, policy_version 1098329 (0.0008) [2023-12-26 23:22:24,391][105620] Updated weights for policy 1, policy_version 1099393 (0.0009) [2023-12-26 23:22:24,447][105620] Updated weights for policy 1, policy_version 1099403 (0.0010) [2023-12-26 23:22:24,511][105620] Updated weights for policy 1, policy_version 1099413 (0.0010) [2023-12-26 23:22:24,574][105620] Updated weights for policy 1, policy_version 1099423 (0.0011) [2023-12-26 23:22:24,876][105692] Updated weights for policy 0, policy_version 1098339 (0.0008) [2023-12-26 23:22:24,938][105692] Updated weights for policy 0, policy_version 1098349 (0.0008) [2023-12-26 23:22:24,995][105692] Updated weights for policy 0, policy_version 1098359 (0.0008) [2023-12-26 23:22:25,229][105620] Updated weights for policy 1, policy_version 1099433 (0.0010) [2023-12-26 23:22:25,286][105620] Updated weights for policy 1, policy_version 1099443 (0.0010) [2023-12-26 23:22:25,347][105620] Updated weights for policy 1, policy_version 1099453 (0.0010) [2023-12-26 23:22:25,737][105692] Updated weights for policy 0, policy_version 1098369 (0.0008) [2023-12-26 23:22:25,798][105692] Updated weights for policy 0, policy_version 1098379 (0.0009) [2023-12-26 23:22:25,845][105692] Updated weights for policy 0, policy_version 1098389 (0.0009) [2023-12-26 23:22:25,896][105692] Updated weights for policy 0, policy_version 1098399 (0.0009) [2023-12-26 23:22:26,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 562724864. Throughput: 0: 9664.6, 1: 9865.0. Samples: 562733992. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:22:26,062][104569] Avg episode reward: [(0, '9352.991'), (1, '9167.389')] [2023-12-26 23:22:26,071][105620] Updated weights for policy 1, policy_version 1099463 (0.0008) [2023-12-26 23:22:26,144][105620] Updated weights for policy 1, policy_version 1099473 (0.0006) [2023-12-26 23:22:26,212][105620] Updated weights for policy 1, policy_version 1099483 (0.0007) [2023-12-26 23:22:26,696][105692] Updated weights for policy 0, policy_version 1098409 (0.0009) [2023-12-26 23:22:26,742][105692] Updated weights for policy 0, policy_version 1098419 (0.0008) [2023-12-26 23:22:26,802][105692] Updated weights for policy 0, policy_version 1098429 (0.0009) [2023-12-26 23:22:26,852][105620] Updated weights for policy 1, policy_version 1099493 (0.0007) [2023-12-26 23:22:26,897][105620] Updated weights for policy 1, policy_version 1099503 (0.0005) [2023-12-26 23:22:26,947][105620] Updated weights for policy 1, policy_version 1099513 (0.0005) [2023-12-26 23:22:27,526][105620] Updated weights for policy 1, policy_version 1099523 (0.0005) [2023-12-26 23:22:27,589][105620] Updated weights for policy 1, policy_version 1099533 (0.0006) [2023-12-26 23:22:27,614][105692] Updated weights for policy 0, policy_version 1098439 (0.0009) [2023-12-26 23:22:27,650][105620] Updated weights for policy 1, policy_version 1099543 (0.0006) [2023-12-26 23:22:27,674][105692] Updated weights for policy 0, policy_version 1098449 (0.0008) [2023-12-26 23:22:27,735][105692] Updated weights for policy 0, policy_version 1098459 (0.0007) [2023-12-26 23:22:28,349][105620] Updated weights for policy 1, policy_version 1099553 (0.0009) [2023-12-26 23:22:28,408][105620] Updated weights for policy 1, policy_version 1099563 (0.0010) [2023-12-26 23:22:28,466][105620] Updated weights for policy 1, policy_version 1099573 (0.0010) [2023-12-26 23:22:28,503][105692] Updated weights for policy 0, policy_version 1098469 (0.0009) [2023-12-26 23:22:28,524][105620] Updated weights for policy 1, policy_version 1099583 (0.0010) [2023-12-26 23:22:28,548][105692] Updated weights for policy 0, policy_version 1098479 (0.0007) [2023-12-26 23:22:28,600][105692] Updated weights for policy 0, policy_version 1098489 (0.0007) [2023-12-26 23:22:29,236][105620] Updated weights for policy 1, policy_version 1099593 (0.0009) [2023-12-26 23:22:29,298][105620] Updated weights for policy 1, policy_version 1099603 (0.0009) [2023-12-26 23:22:29,298][105692] Updated weights for policy 0, policy_version 1098499 (0.0007) [2023-12-26 23:22:29,364][105692] Updated weights for policy 0, policy_version 1098509 (0.0009) [2023-12-26 23:22:29,365][105620] Updated weights for policy 1, policy_version 1099613 (0.0009) [2023-12-26 23:22:29,427][105692] Updated weights for policy 0, policy_version 1098519 (0.0009) [2023-12-26 23:22:30,072][105620] Updated weights for policy 1, policy_version 1099623 (0.0007) [2023-12-26 23:22:30,139][105620] Updated weights for policy 1, policy_version 1099633 (0.0006) [2023-12-26 23:22:30,182][105692] Updated weights for policy 0, policy_version 1098529 (0.0008) [2023-12-26 23:22:30,205][105620] Updated weights for policy 1, policy_version 1099643 (0.0007) [2023-12-26 23:22:30,243][105692] Updated weights for policy 0, policy_version 1098539 (0.0007) [2023-12-26 23:22:30,302][105692] Updated weights for policy 0, policy_version 1098549 (0.0009) [2023-12-26 23:22:30,369][105692] Updated weights for policy 0, policy_version 1098559 (0.0009) [2023-12-26 23:22:30,858][105620] Updated weights for policy 1, policy_version 1099653 (0.0007) [2023-12-26 23:22:30,916][105620] Updated weights for policy 1, policy_version 1099663 (0.0005) [2023-12-26 23:22:30,977][105620] Updated weights for policy 1, policy_version 1099673 (0.0005) [2023-12-26 23:22:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 562823168. Throughput: 0: 9630.0, 1: 9882.2. Samples: 562792340. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:22:31,062][104569] Avg episode reward: [(0, '9174.832'), (1, '9259.829')] [2023-12-26 23:22:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001098560_281272320.pth... [2023-12-26 23:22:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001099680_281550848.pth... [2023-12-26 23:22:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001097472_280993792.pth [2023-12-26 23:22:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001098528_281255936.pth [2023-12-26 23:22:31,138][105692] Updated weights for policy 0, policy_version 1098569 (0.0006) [2023-12-26 23:22:31,205][105692] Updated weights for policy 0, policy_version 1098579 (0.0009) [2023-12-26 23:22:31,273][105692] Updated weights for policy 0, policy_version 1098589 (0.0008) [2023-12-26 23:22:31,669][105620] Updated weights for policy 1, policy_version 1099683 (0.0007) [2023-12-26 23:22:31,739][105620] Updated weights for policy 1, policy_version 1099693 (0.0008) [2023-12-26 23:22:31,800][105620] Updated weights for policy 1, policy_version 1099703 (0.0008) [2023-12-26 23:22:32,029][105692] Updated weights for policy 0, policy_version 1098599 (0.0007) [2023-12-26 23:22:32,091][105692] Updated weights for policy 0, policy_version 1098609 (0.0006) [2023-12-26 23:22:32,149][105692] Updated weights for policy 0, policy_version 1098619 (0.0006) [2023-12-26 23:22:32,541][105620] Updated weights for policy 1, policy_version 1099713 (0.0009) [2023-12-26 23:22:32,605][105620] Updated weights for policy 1, policy_version 1099723 (0.0008) [2023-12-26 23:22:32,667][105620] Updated weights for policy 1, policy_version 1099733 (0.0009) [2023-12-26 23:22:32,725][105620] Updated weights for policy 1, policy_version 1099743 (0.0009) [2023-12-26 23:22:32,859][105692] Updated weights for policy 0, policy_version 1098629 (0.0008) [2023-12-26 23:22:32,913][105692] Updated weights for policy 0, policy_version 1098639 (0.0008) [2023-12-26 23:22:32,963][105692] Updated weights for policy 0, policy_version 1098649 (0.0009) [2023-12-26 23:22:33,388][105620] Updated weights for policy 1, policy_version 1099753 (0.0008) [2023-12-26 23:22:33,447][105620] Updated weights for policy 1, policy_version 1099763 (0.0008) [2023-12-26 23:22:33,509][105620] Updated weights for policy 1, policy_version 1099773 (0.0009) [2023-12-26 23:22:33,703][105692] Updated weights for policy 0, policy_version 1098659 (0.0007) [2023-12-26 23:22:33,751][105692] Updated weights for policy 0, policy_version 1098669 (0.0005) [2023-12-26 23:22:33,807][105692] Updated weights for policy 0, policy_version 1098679 (0.0005) [2023-12-26 23:22:34,154][105620] Updated weights for policy 1, policy_version 1099783 (0.0007) [2023-12-26 23:22:34,222][105620] Updated weights for policy 1, policy_version 1099793 (0.0007) [2023-12-26 23:22:34,284][105620] Updated weights for policy 1, policy_version 1099803 (0.0007) [2023-12-26 23:22:34,405][105692] Updated weights for policy 0, policy_version 1098689 (0.0005) [2023-12-26 23:22:34,460][105692] Updated weights for policy 0, policy_version 1098699 (0.0006) [2023-12-26 23:22:34,510][105692] Updated weights for policy 0, policy_version 1098709 (0.0011) [2023-12-26 23:22:34,577][105692] Updated weights for policy 0, policy_version 1098719 (0.0011) [2023-12-26 23:22:34,966][105620] Updated weights for policy 1, policy_version 1099813 (0.0008) [2023-12-26 23:22:35,029][105620] Updated weights for policy 1, policy_version 1099823 (0.0008) [2023-12-26 23:22:35,081][105620] Updated weights for policy 1, policy_version 1099833 (0.0008) [2023-12-26 23:22:35,266][105692] Updated weights for policy 0, policy_version 1098729 (0.0010) [2023-12-26 23:22:35,321][105692] Updated weights for policy 0, policy_version 1098739 (0.0010) [2023-12-26 23:22:35,372][105692] Updated weights for policy 0, policy_version 1098749 (0.0010) [2023-12-26 23:22:35,712][105620] Updated weights for policy 1, policy_version 1099843 (0.0007) [2023-12-26 23:22:35,767][105620] Updated weights for policy 1, policy_version 1099853 (0.0005) [2023-12-26 23:22:35,826][105620] Updated weights for policy 1, policy_version 1099863 (0.0010) [2023-12-26 23:22:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 562921472. Throughput: 0: 9659.1, 1: 9888.1. Samples: 562910396. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:22:36,062][104569] Avg episode reward: [(0, '9086.001'), (1, '9258.985')] [2023-12-26 23:22:36,101][105692] Updated weights for policy 0, policy_version 1098759 (0.0011) [2023-12-26 23:22:36,164][105692] Updated weights for policy 0, policy_version 1098769 (0.0011) [2023-12-26 23:22:36,223][105692] Updated weights for policy 0, policy_version 1098779 (0.0011) [2023-12-26 23:22:36,554][105620] Updated weights for policy 1, policy_version 1099873 (0.0010) [2023-12-26 23:22:36,614][105620] Updated weights for policy 1, policy_version 1099883 (0.0011) [2023-12-26 23:22:36,676][105620] Updated weights for policy 1, policy_version 1099893 (0.0011) [2023-12-26 23:22:36,740][105620] Updated weights for policy 1, policy_version 1099903 (0.0010) [2023-12-26 23:22:36,943][105692] Updated weights for policy 0, policy_version 1098789 (0.0010) [2023-12-26 23:22:37,001][105692] Updated weights for policy 0, policy_version 1098799 (0.0010) [2023-12-26 23:22:37,057][105692] Updated weights for policy 0, policy_version 1098809 (0.0010) [2023-12-26 23:22:37,479][105620] Updated weights for policy 1, policy_version 1099913 (0.0010) [2023-12-26 23:22:37,534][105620] Updated weights for policy 1, policy_version 1099923 (0.0010) [2023-12-26 23:22:37,589][105620] Updated weights for policy 1, policy_version 1099933 (0.0010) [2023-12-26 23:22:37,768][105692] Updated weights for policy 0, policy_version 1098819 (0.0010) [2023-12-26 23:22:37,831][105692] Updated weights for policy 0, policy_version 1098829 (0.0010) [2023-12-26 23:22:37,897][105692] Updated weights for policy 0, policy_version 1098839 (0.0008) [2023-12-26 23:22:38,368][105620] Updated weights for policy 1, policy_version 1099943 (0.0011) [2023-12-26 23:22:38,428][105620] Updated weights for policy 1, policy_version 1099953 (0.0010) [2023-12-26 23:22:38,486][105620] Updated weights for policy 1, policy_version 1099963 (0.0010) [2023-12-26 23:22:38,568][105692] Updated weights for policy 0, policy_version 1098849 (0.0007) [2023-12-26 23:22:38,621][105692] Updated weights for policy 0, policy_version 1098859 (0.0008) [2023-12-26 23:22:38,670][105692] Updated weights for policy 0, policy_version 1098869 (0.0008) [2023-12-26 23:22:38,734][105692] Updated weights for policy 0, policy_version 1098879 (0.0008) [2023-12-26 23:22:39,224][105620] Updated weights for policy 1, policy_version 1099973 (0.0010) [2023-12-26 23:22:39,285][105620] Updated weights for policy 1, policy_version 1099983 (0.0010) [2023-12-26 23:22:39,352][105620] Updated weights for policy 1, policy_version 1099993 (0.0011) [2023-12-26 23:22:39,563][105692] Updated weights for policy 0, policy_version 1098889 (0.0008) [2023-12-26 23:22:39,623][105692] Updated weights for policy 0, policy_version 1098899 (0.0008) [2023-12-26 23:22:39,673][105692] Updated weights for policy 0, policy_version 1098909 (0.0008) [2023-12-26 23:22:40,126][105620] Updated weights for policy 1, policy_version 1100003 (0.0011) [2023-12-26 23:22:40,189][105620] Updated weights for policy 1, policy_version 1100013 (0.0011) [2023-12-26 23:22:40,249][105620] Updated weights for policy 1, policy_version 1100023 (0.0010) [2023-12-26 23:22:40,486][105692] Updated weights for policy 0, policy_version 1098919 (0.0010) [2023-12-26 23:22:40,545][105692] Updated weights for policy 0, policy_version 1098929 (0.0010) [2023-12-26 23:22:40,598][105692] Updated weights for policy 0, policy_version 1098939 (0.0011) [2023-12-26 23:22:41,012][105620] Updated weights for policy 1, policy_version 1100033 (0.0010) [2023-12-26 23:22:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 563011584. Throughput: 0: 9597.8, 1: 9729.5. Samples: 563023852. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:22:41,062][104569] Avg episode reward: [(0, '9176.582'), (1, '9166.546')] [2023-12-26 23:22:41,079][105620] Updated weights for policy 1, policy_version 1100043 (0.0011) [2023-12-26 23:22:41,130][105620] Updated weights for policy 1, policy_version 1100053 (0.0009) [2023-12-26 23:22:41,196][105620] Updated weights for policy 1, policy_version 1100063 (0.0010) [2023-12-26 23:22:41,356][105692] Updated weights for policy 0, policy_version 1098949 (0.0009) [2023-12-26 23:22:41,416][105692] Updated weights for policy 0, policy_version 1098959 (0.0008) [2023-12-26 23:22:41,476][105692] Updated weights for policy 0, policy_version 1098969 (0.0008) [2023-12-26 23:22:41,977][105620] Updated weights for policy 1, policy_version 1100073 (0.0011) [2023-12-26 23:22:42,034][105620] Updated weights for policy 1, policy_version 1100083 (0.0011) [2023-12-26 23:22:42,094][105620] Updated weights for policy 1, policy_version 1100093 (0.0010) [2023-12-26 23:22:42,234][105692] Updated weights for policy 0, policy_version 1098979 (0.0008) [2023-12-26 23:22:42,301][105692] Updated weights for policy 0, policy_version 1098989 (0.0009) [2023-12-26 23:22:42,371][105692] Updated weights for policy 0, policy_version 1098999 (0.0008) [2023-12-26 23:22:42,843][105620] Updated weights for policy 1, policy_version 1100103 (0.0010) [2023-12-26 23:22:42,899][105620] Updated weights for policy 1, policy_version 1100113 (0.0007) [2023-12-26 23:22:42,951][105620] Updated weights for policy 1, policy_version 1100123 (0.0005) [2023-12-26 23:22:43,091][105692] Updated weights for policy 0, policy_version 1099009 (0.0008) [2023-12-26 23:22:43,153][105692] Updated weights for policy 0, policy_version 1099019 (0.0008) [2023-12-26 23:22:43,221][105692] Updated weights for policy 0, policy_version 1099029 (0.0006) [2023-12-26 23:22:43,276][105692] Updated weights for policy 0, policy_version 1099039 (0.0005) [2023-12-26 23:22:43,674][105620] Updated weights for policy 1, policy_version 1100133 (0.0009) [2023-12-26 23:22:43,729][105620] Updated weights for policy 1, policy_version 1100143 (0.0010) [2023-12-26 23:22:43,780][105620] Updated weights for policy 1, policy_version 1100153 (0.0010) [2023-12-26 23:22:43,923][105692] Updated weights for policy 0, policy_version 1099049 (0.0005) [2023-12-26 23:22:43,980][105692] Updated weights for policy 0, policy_version 1099059 (0.0006) [2023-12-26 23:22:44,043][105692] Updated weights for policy 0, policy_version 1099069 (0.0009) [2023-12-26 23:22:44,455][105620] Updated weights for policy 1, policy_version 1100163 (0.0010) [2023-12-26 23:22:44,516][105620] Updated weights for policy 1, policy_version 1100173 (0.0010) [2023-12-26 23:22:44,570][105620] Updated weights for policy 1, policy_version 1100183 (0.0010) [2023-12-26 23:22:44,806][105692] Updated weights for policy 0, policy_version 1099079 (0.0009) [2023-12-26 23:22:44,863][105692] Updated weights for policy 0, policy_version 1099089 (0.0008) [2023-12-26 23:22:44,921][105692] Updated weights for policy 0, policy_version 1099099 (0.0008) [2023-12-26 23:22:45,333][105620] Updated weights for policy 1, policy_version 1100193 (0.0010) [2023-12-26 23:22:45,393][105620] Updated weights for policy 1, policy_version 1100203 (0.0010) [2023-12-26 23:22:45,454][105620] Updated weights for policy 1, policy_version 1100213 (0.0010) [2023-12-26 23:22:45,521][105620] Updated weights for policy 1, policy_version 1100223 (0.0009) [2023-12-26 23:22:45,703][105692] Updated weights for policy 0, policy_version 1099109 (0.0009) [2023-12-26 23:22:45,764][105692] Updated weights for policy 0, policy_version 1099119 (0.0008) [2023-12-26 23:22:45,828][105692] Updated weights for policy 0, policy_version 1099129 (0.0008) [2023-12-26 23:22:46,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 563109888. Throughput: 0: 9526.2, 1: 9699.8. Samples: 563079400. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:22:46,063][104569] Avg episode reward: [(0, '9265.773'), (1, '9074.913')] [2023-12-26 23:22:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001099136_281419776.pth... [2023-12-26 23:22:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001100224_281690112.pth... [2023-12-26 23:22:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001098048_281141248.pth [2023-12-26 23:22:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001099104_281403392.pth [2023-12-26 23:22:46,268][105620] Updated weights for policy 1, policy_version 1100233 (0.0010) [2023-12-26 23:22:46,326][105620] Updated weights for policy 1, policy_version 1100243 (0.0010) [2023-12-26 23:22:46,387][105620] Updated weights for policy 1, policy_version 1100253 (0.0010) [2023-12-26 23:22:46,600][105692] Updated weights for policy 0, policy_version 1099139 (0.0008) [2023-12-26 23:22:46,667][105692] Updated weights for policy 0, policy_version 1099149 (0.0009) [2023-12-26 23:22:46,727][105692] Updated weights for policy 0, policy_version 1099159 (0.0008) [2023-12-26 23:22:47,112][105620] Updated weights for policy 1, policy_version 1100263 (0.0010) [2023-12-26 23:22:47,164][105620] Updated weights for policy 1, policy_version 1100273 (0.0010) [2023-12-26 23:22:47,227][105620] Updated weights for policy 1, policy_version 1100283 (0.0011) [2023-12-26 23:22:47,342][105692] Updated weights for policy 0, policy_version 1099169 (0.0008) [2023-12-26 23:22:47,417][105692] Updated weights for policy 0, policy_version 1099179 (0.0008) [2023-12-26 23:22:47,480][105692] Updated weights for policy 0, policy_version 1099189 (0.0010) [2023-12-26 23:22:47,539][105692] Updated weights for policy 0, policy_version 1099199 (0.0011) [2023-12-26 23:22:48,030][105620] Updated weights for policy 1, policy_version 1100293 (0.0010) [2023-12-26 23:22:48,093][105620] Updated weights for policy 1, policy_version 1100303 (0.0008) [2023-12-26 23:22:48,156][105620] Updated weights for policy 1, policy_version 1100313 (0.0008) [2023-12-26 23:22:48,294][105692] Updated weights for policy 0, policy_version 1099209 (0.0011) [2023-12-26 23:22:48,357][105692] Updated weights for policy 0, policy_version 1099219 (0.0010) [2023-12-26 23:22:48,415][105692] Updated weights for policy 0, policy_version 1099229 (0.0010) [2023-12-26 23:22:48,919][105620] Updated weights for policy 1, policy_version 1100323 (0.0008) [2023-12-26 23:22:48,978][105620] Updated weights for policy 1, policy_version 1100333 (0.0008) [2023-12-26 23:22:49,023][105620] Updated weights for policy 1, policy_version 1100343 (0.0008) [2023-12-26 23:22:49,187][105692] Updated weights for policy 0, policy_version 1099239 (0.0010) [2023-12-26 23:22:49,257][105692] Updated weights for policy 0, policy_version 1099249 (0.0011) [2023-12-26 23:22:49,314][105692] Updated weights for policy 0, policy_version 1099259 (0.0011) [2023-12-26 23:22:49,809][105620] Updated weights for policy 1, policy_version 1100353 (0.0007) [2023-12-26 23:22:49,874][105620] Updated weights for policy 1, policy_version 1100363 (0.0008) [2023-12-26 23:22:49,930][105620] Updated weights for policy 1, policy_version 1100373 (0.0008) [2023-12-26 23:22:49,993][105620] Updated weights for policy 1, policy_version 1100383 (0.0008) [2023-12-26 23:22:50,089][105692] Updated weights for policy 0, policy_version 1099269 (0.0011) [2023-12-26 23:22:50,143][105692] Updated weights for policy 0, policy_version 1099279 (0.0010) [2023-12-26 23:22:50,201][105692] Updated weights for policy 0, policy_version 1099289 (0.0007) [2023-12-26 23:22:50,795][105620] Updated weights for policy 1, policy_version 1100393 (0.0009) [2023-12-26 23:22:50,849][105620] Updated weights for policy 1, policy_version 1100403 (0.0009) [2023-12-26 23:22:50,885][105692] Updated weights for policy 0, policy_version 1099299 (0.0007) [2023-12-26 23:22:50,910][105620] Updated weights for policy 1, policy_version 1100413 (0.0007) [2023-12-26 23:22:50,946][105692] Updated weights for policy 0, policy_version 1099309 (0.0009) [2023-12-26 23:22:51,015][105692] Updated weights for policy 0, policy_version 1099319 (0.0006) [2023-12-26 23:22:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 563200000. Throughput: 0: 9508.9, 1: 9672.2. Samples: 563191564. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:22:51,063][104569] Avg episode reward: [(0, '9351.568'), (1, '9167.028')] [2023-12-26 23:22:51,723][105620] Updated weights for policy 1, policy_version 1100423 (0.0008) [2023-12-26 23:22:51,745][105692] Updated weights for policy 0, policy_version 1099329 (0.0009) [2023-12-26 23:22:51,787][105620] Updated weights for policy 1, policy_version 1100433 (0.0009) [2023-12-26 23:22:51,808][105692] Updated weights for policy 0, policy_version 1099339 (0.0007) [2023-12-26 23:22:51,847][105620] Updated weights for policy 1, policy_version 1100443 (0.0008) [2023-12-26 23:22:51,870][105692] Updated weights for policy 0, policy_version 1099349 (0.0006) [2023-12-26 23:22:51,934][105692] Updated weights for policy 0, policy_version 1099359 (0.0009) [2023-12-26 23:22:52,660][105692] Updated weights for policy 0, policy_version 1099369 (0.0007) [2023-12-26 23:22:52,673][105620] Updated weights for policy 1, policy_version 1100453 (0.0009) [2023-12-26 23:22:52,721][105692] Updated weights for policy 0, policy_version 1099379 (0.0010) [2023-12-26 23:22:52,732][105620] Updated weights for policy 1, policy_version 1100463 (0.0009) [2023-12-26 23:22:52,780][105692] Updated weights for policy 0, policy_version 1099389 (0.0006) [2023-12-26 23:22:52,790][105620] Updated weights for policy 1, policy_version 1100473 (0.0008) [2023-12-26 23:22:53,419][105692] Updated weights for policy 0, policy_version 1099399 (0.0006) [2023-12-26 23:22:53,464][105692] Updated weights for policy 0, policy_version 1099409 (0.0005) [2023-12-26 23:22:53,487][105620] Updated weights for policy 1, policy_version 1100483 (0.0008) [2023-12-26 23:22:53,509][105692] Updated weights for policy 0, policy_version 1099419 (0.0009) [2023-12-26 23:22:53,551][105620] Updated weights for policy 1, policy_version 1100493 (0.0006) [2023-12-26 23:22:53,598][105620] Updated weights for policy 1, policy_version 1100503 (0.0008) [2023-12-26 23:22:54,204][105692] Updated weights for policy 0, policy_version 1099429 (0.0010) [2023-12-26 23:22:54,262][105692] Updated weights for policy 0, policy_version 1099439 (0.0010) [2023-12-26 23:22:54,316][105620] Updated weights for policy 1, policy_version 1100513 (0.0007) [2023-12-26 23:22:54,321][105692] Updated weights for policy 0, policy_version 1099449 (0.0009) [2023-12-26 23:22:54,375][105620] Updated weights for policy 1, policy_version 1100523 (0.0007) [2023-12-26 23:22:54,426][105620] Updated weights for policy 1, policy_version 1100533 (0.0009) [2023-12-26 23:22:54,474][105620] Updated weights for policy 1, policy_version 1100543 (0.0010) [2023-12-26 23:22:54,966][105692] Updated weights for policy 0, policy_version 1099459 (0.0009) [2023-12-26 23:22:55,014][105692] Updated weights for policy 0, policy_version 1099469 (0.0009) [2023-12-26 23:22:55,059][105692] Updated weights for policy 0, policy_version 1099479 (0.0010) [2023-12-26 23:22:55,084][105620] Updated weights for policy 1, policy_version 1100553 (0.0007) [2023-12-26 23:22:55,151][105620] Updated weights for policy 1, policy_version 1100563 (0.0009) [2023-12-26 23:22:55,215][105620] Updated weights for policy 1, policy_version 1100573 (0.0009) [2023-12-26 23:22:55,701][105692] Updated weights for policy 0, policy_version 1099489 (0.0005) [2023-12-26 23:22:55,759][105692] Updated weights for policy 0, policy_version 1099499 (0.0006) [2023-12-26 23:22:55,814][105692] Updated weights for policy 0, policy_version 1099509 (0.0011) [2023-12-26 23:22:55,873][105692] Updated weights for policy 0, policy_version 1099519 (0.0011) [2023-12-26 23:22:55,939][105620] Updated weights for policy 1, policy_version 1100583 (0.0008) [2023-12-26 23:22:56,004][105620] Updated weights for policy 1, policy_version 1100593 (0.0008) [2023-12-26 23:22:56,062][104569] Fps is (10 sec: 18842.3, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 563298304. Throughput: 0: 9559.6, 1: 9563.8. Samples: 563308444. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:22:56,062][104569] Avg episode reward: [(0, '9259.027'), (1, '9076.797')] [2023-12-26 23:22:56,066][105620] Updated weights for policy 1, policy_version 1100603 (0.0009) [2023-12-26 23:22:56,463][105692] Updated weights for policy 0, policy_version 1099529 (0.0008) [2023-12-26 23:22:56,528][105692] Updated weights for policy 0, policy_version 1099539 (0.0006) [2023-12-26 23:22:56,595][105692] Updated weights for policy 0, policy_version 1099549 (0.0008) [2023-12-26 23:22:56,910][105620] Updated weights for policy 1, policy_version 1100613 (0.0010) [2023-12-26 23:22:56,959][105620] Updated weights for policy 1, policy_version 1100623 (0.0006) [2023-12-26 23:22:57,007][105620] Updated weights for policy 1, policy_version 1100633 (0.0005) [2023-12-26 23:22:57,117][105692] Updated weights for policy 0, policy_version 1099559 (0.0006) [2023-12-26 23:22:57,168][105692] Updated weights for policy 0, policy_version 1099569 (0.0005) [2023-12-26 23:22:57,219][105692] Updated weights for policy 0, policy_version 1099579 (0.0005) [2023-12-26 23:22:57,782][105620] Updated weights for policy 1, policy_version 1100643 (0.0007) [2023-12-26 23:22:57,845][105620] Updated weights for policy 1, policy_version 1100653 (0.0010) [2023-12-26 23:22:57,898][105692] Updated weights for policy 0, policy_version 1099589 (0.0008) [2023-12-26 23:22:57,902][105620] Updated weights for policy 1, policy_version 1100663 (0.0010) [2023-12-26 23:22:57,946][105692] Updated weights for policy 0, policy_version 1099599 (0.0010) [2023-12-26 23:22:57,994][105692] Updated weights for policy 0, policy_version 1099609 (0.0010) [2023-12-26 23:22:58,576][105620] Updated weights for policy 1, policy_version 1100673 (0.0010) [2023-12-26 23:22:58,644][105620] Updated weights for policy 1, policy_version 1100683 (0.0008) [2023-12-26 23:22:58,714][105620] Updated weights for policy 1, policy_version 1100693 (0.0007) [2023-12-26 23:22:58,775][105692] Updated weights for policy 0, policy_version 1099619 (0.0010) [2023-12-26 23:22:58,778][105620] Updated weights for policy 1, policy_version 1100703 (0.0007) [2023-12-26 23:22:58,841][105692] Updated weights for policy 0, policy_version 1099629 (0.0009) [2023-12-26 23:22:58,906][105692] Updated weights for policy 0, policy_version 1099639 (0.0010) [2023-12-26 23:22:59,602][105692] Updated weights for policy 0, policy_version 1099649 (0.0011) [2023-12-26 23:22:59,613][105620] Updated weights for policy 1, policy_version 1100713 (0.0007) [2023-12-26 23:22:59,652][105692] Updated weights for policy 0, policy_version 1099659 (0.0007) [2023-12-26 23:22:59,659][105620] Updated weights for policy 1, policy_version 1100723 (0.0006) [2023-12-26 23:22:59,710][105692] Updated weights for policy 0, policy_version 1099669 (0.0010) [2023-12-26 23:22:59,712][105620] Updated weights for policy 1, policy_version 1100733 (0.0009) [2023-12-26 23:22:59,763][105692] Updated weights for policy 0, policy_version 1099679 (0.0008) [2023-12-26 23:23:00,451][105620] Updated weights for policy 1, policy_version 1100743 (0.0008) [2023-12-26 23:23:00,496][105620] Updated weights for policy 1, policy_version 1100753 (0.0010) [2023-12-26 23:23:00,524][105692] Updated weights for policy 0, policy_version 1099689 (0.0005) [2023-12-26 23:23:00,541][105620] Updated weights for policy 1, policy_version 1100763 (0.0010) [2023-12-26 23:23:00,577][105692] Updated weights for policy 0, policy_version 1099699 (0.0007) [2023-12-26 23:23:00,631][105692] Updated weights for policy 0, policy_version 1099709 (0.0008) [2023-12-26 23:23:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19114.7, 300 sec: 19494.2). Total num frames: 563396608. Throughput: 0: 9594.1, 1: 9573.4. Samples: 563368088. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:23:01,062][104569] Avg episode reward: [(0, '9258.858'), (1, '9166.166')] [2023-12-26 23:23:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001099712_281567232.pth... [2023-12-26 23:23:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001100768_281829376.pth... [2023-12-26 23:23:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001098560_281272320.pth [2023-12-26 23:23:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001099680_281550848.pth [2023-12-26 23:23:01,268][105692] Updated weights for policy 0, policy_version 1099719 (0.0008) [2023-12-26 23:23:01,325][105692] Updated weights for policy 0, policy_version 1099729 (0.0008) [2023-12-26 23:23:01,334][105620] Updated weights for policy 1, policy_version 1100773 (0.0009) [2023-12-26 23:23:01,383][105692] Updated weights for policy 0, policy_version 1099739 (0.0009) [2023-12-26 23:23:01,402][105620] Updated weights for policy 1, policy_version 1100783 (0.0009) [2023-12-26 23:23:01,464][105620] Updated weights for policy 1, policy_version 1100793 (0.0008) [2023-12-26 23:23:02,103][105692] Updated weights for policy 0, policy_version 1099749 (0.0008) [2023-12-26 23:23:02,167][105692] Updated weights for policy 0, policy_version 1099759 (0.0006) [2023-12-26 23:23:02,217][105692] Updated weights for policy 0, policy_version 1099769 (0.0007) [2023-12-26 23:23:02,248][105620] Updated weights for policy 1, policy_version 1100803 (0.0009) [2023-12-26 23:23:02,309][105620] Updated weights for policy 1, policy_version 1100813 (0.0009) [2023-12-26 23:23:02,366][105620] Updated weights for policy 1, policy_version 1100823 (0.0009) [2023-12-26 23:23:03,017][105692] Updated weights for policy 0, policy_version 1099779 (0.0007) [2023-12-26 23:23:03,072][105692] Updated weights for policy 0, policy_version 1099789 (0.0009) [2023-12-26 23:23:03,118][105620] Updated weights for policy 1, policy_version 1100833 (0.0008) [2023-12-26 23:23:03,125][105692] Updated weights for policy 0, policy_version 1099799 (0.0010) [2023-12-26 23:23:03,180][105620] Updated weights for policy 1, policy_version 1100843 (0.0006) [2023-12-26 23:23:03,247][105620] Updated weights for policy 1, policy_version 1100853 (0.0009) [2023-12-26 23:23:03,308][105620] Updated weights for policy 1, policy_version 1100863 (0.0009) [2023-12-26 23:23:03,934][105692] Updated weights for policy 0, policy_version 1099809 (0.0008) [2023-12-26 23:23:03,990][105692] Updated weights for policy 0, policy_version 1099819 (0.0007) [2023-12-26 23:23:04,005][105620] Updated weights for policy 1, policy_version 1100873 (0.0007) [2023-12-26 23:23:04,041][105692] Updated weights for policy 0, policy_version 1099829 (0.0006) [2023-12-26 23:23:04,062][105620] Updated weights for policy 1, policy_version 1100883 (0.0009) [2023-12-26 23:23:04,087][105692] Updated weights for policy 0, policy_version 1099839 (0.0009) [2023-12-26 23:23:04,119][105620] Updated weights for policy 1, policy_version 1100893 (0.0008) [2023-12-26 23:23:04,885][105692] Updated weights for policy 0, policy_version 1099849 (0.0007) [2023-12-26 23:23:04,897][105620] Updated weights for policy 1, policy_version 1100903 (0.0010) [2023-12-26 23:23:04,947][105620] Updated weights for policy 1, policy_version 1100913 (0.0010) [2023-12-26 23:23:04,948][105692] Updated weights for policy 0, policy_version 1099859 (0.0007) [2023-12-26 23:23:04,998][105620] Updated weights for policy 1, policy_version 1100923 (0.0010) [2023-12-26 23:23:05,009][105692] Updated weights for policy 0, policy_version 1099869 (0.0006) [2023-12-26 23:23:05,736][105620] Updated weights for policy 1, policy_version 1100933 (0.0010) [2023-12-26 23:23:05,783][105692] Updated weights for policy 0, policy_version 1099879 (0.0007) [2023-12-26 23:23:05,786][105620] Updated weights for policy 1, policy_version 1100943 (0.0008) [2023-12-26 23:23:05,830][105620] Updated weights for policy 1, policy_version 1100953 (0.0009) [2023-12-26 23:23:05,832][105692] Updated weights for policy 0, policy_version 1099889 (0.0006) [2023-12-26 23:23:05,884][105692] Updated weights for policy 0, policy_version 1099899 (0.0006) [2023-12-26 23:23:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 563494912. Throughput: 0: 9576.1, 1: 9508.0. Samples: 563480240. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:23:06,063][104569] Avg episode reward: [(0, '9350.267'), (1, '9164.917')] [2023-12-26 23:23:06,519][105620] Updated weights for policy 1, policy_version 1100963 (0.0010) [2023-12-26 23:23:06,585][105620] Updated weights for policy 1, policy_version 1100973 (0.0010) [2023-12-26 23:23:06,652][105620] Updated weights for policy 1, policy_version 1100983 (0.0009) [2023-12-26 23:23:06,693][105692] Updated weights for policy 0, policy_version 1099909 (0.0009) [2023-12-26 23:23:06,747][105692] Updated weights for policy 0, policy_version 1099919 (0.0009) [2023-12-26 23:23:06,806][105692] Updated weights for policy 0, policy_version 1099929 (0.0009) [2023-12-26 23:23:07,320][105620] Updated weights for policy 1, policy_version 1100993 (0.0008) [2023-12-26 23:23:07,382][105620] Updated weights for policy 1, policy_version 1101003 (0.0007) [2023-12-26 23:23:07,430][105620] Updated weights for policy 1, policy_version 1101013 (0.0005) [2023-12-26 23:23:07,480][105620] Updated weights for policy 1, policy_version 1101023 (0.0007) [2023-12-26 23:23:07,633][105692] Updated weights for policy 0, policy_version 1099939 (0.0009) [2023-12-26 23:23:07,684][105692] Updated weights for policy 0, policy_version 1099949 (0.0009) [2023-12-26 23:23:07,739][105692] Updated weights for policy 0, policy_version 1099959 (0.0010) [2023-12-26 23:23:08,177][105620] Updated weights for policy 1, policy_version 1101033 (0.0010) [2023-12-26 23:23:08,231][105620] Updated weights for policy 1, policy_version 1101043 (0.0009) [2023-12-26 23:23:08,288][105620] Updated weights for policy 1, policy_version 1101053 (0.0008) [2023-12-26 23:23:08,530][105692] Updated weights for policy 0, policy_version 1099970 (0.0010) [2023-12-26 23:23:08,585][105692] Updated weights for policy 0, policy_version 1099980 (0.0009) [2023-12-26 23:23:08,636][105692] Updated weights for policy 0, policy_version 1099990 (0.0009) [2023-12-26 23:23:08,698][105692] Updated weights for policy 0, policy_version 1100000 (0.0009) [2023-12-26 23:23:09,030][105620] Updated weights for policy 1, policy_version 1101063 (0.0007) [2023-12-26 23:23:09,086][105620] Updated weights for policy 1, policy_version 1101073 (0.0005) [2023-12-26 23:23:09,139][105620] Updated weights for policy 1, policy_version 1101083 (0.0005) [2023-12-26 23:23:09,512][105692] Updated weights for policy 0, policy_version 1100010 (0.0009) [2023-12-26 23:23:09,558][105692] Updated weights for policy 0, policy_version 1100020 (0.0008) [2023-12-26 23:23:09,611][105692] Updated weights for policy 0, policy_version 1100030 (0.0008) [2023-12-26 23:23:09,876][105620] Updated weights for policy 1, policy_version 1101093 (0.0008) [2023-12-26 23:23:09,936][105620] Updated weights for policy 1, policy_version 1101103 (0.0011) [2023-12-26 23:23:10,001][105620] Updated weights for policy 1, policy_version 1101113 (0.0010) [2023-12-26 23:23:10,448][105692] Updated weights for policy 0, policy_version 1100040 (0.0007) [2023-12-26 23:23:10,513][105692] Updated weights for policy 0, policy_version 1100050 (0.0007) [2023-12-26 23:23:10,561][105692] Updated weights for policy 0, policy_version 1100060 (0.0005) [2023-12-26 23:23:10,740][105620] Updated weights for policy 1, policy_version 1101123 (0.0009) [2023-12-26 23:23:10,796][105620] Updated weights for policy 1, policy_version 1101133 (0.0009) [2023-12-26 23:23:10,849][105620] Updated weights for policy 1, policy_version 1101143 (0.0009) [2023-12-26 23:23:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 563585024. Throughput: 0: 9529.1, 1: 9522.2. Samples: 563591300. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:23:11,062][104569] Avg episode reward: [(0, '9353.948'), (1, '9074.836')] [2023-12-26 23:23:11,248][105692] Updated weights for policy 0, policy_version 1100070 (0.0007) [2023-12-26 23:23:11,315][105692] Updated weights for policy 0, policy_version 1100080 (0.0009) [2023-12-26 23:23:11,377][105692] Updated weights for policy 0, policy_version 1100090 (0.0008) [2023-12-26 23:23:11,632][105620] Updated weights for policy 1, policy_version 1101153 (0.0010) [2023-12-26 23:23:11,698][105620] Updated weights for policy 1, policy_version 1101163 (0.0011) [2023-12-26 23:23:11,765][105620] Updated weights for policy 1, policy_version 1101173 (0.0010) [2023-12-26 23:23:11,818][105620] Updated weights for policy 1, policy_version 1101183 (0.0010) [2023-12-26 23:23:12,188][105692] Updated weights for policy 0, policy_version 1100100 (0.0008) [2023-12-26 23:23:12,246][105692] Updated weights for policy 0, policy_version 1100110 (0.0008) [2023-12-26 23:23:12,306][105692] Updated weights for policy 0, policy_version 1100120 (0.0010) [2023-12-26 23:23:12,565][105620] Updated weights for policy 1, policy_version 1101193 (0.0009) [2023-12-26 23:23:12,624][105620] Updated weights for policy 1, policy_version 1101203 (0.0009) [2023-12-26 23:23:12,674][105620] Updated weights for policy 1, policy_version 1101213 (0.0010) [2023-12-26 23:23:13,089][105692] Updated weights for policy 0, policy_version 1100130 (0.0010) [2023-12-26 23:23:13,143][105692] Updated weights for policy 0, policy_version 1100140 (0.0010) [2023-12-26 23:23:13,204][105692] Updated weights for policy 0, policy_version 1100151 (0.0008) [2023-12-26 23:23:13,311][105620] Updated weights for policy 1, policy_version 1101223 (0.0006) [2023-12-26 23:23:13,365][105620] Updated weights for policy 1, policy_version 1101233 (0.0009) [2023-12-26 23:23:13,419][105620] Updated weights for policy 1, policy_version 1101243 (0.0010) [2023-12-26 23:23:13,905][105692] Updated weights for policy 0, policy_version 1100161 (0.0008) [2023-12-26 23:23:13,951][105692] Updated weights for policy 0, policy_version 1100171 (0.0009) [2023-12-26 23:23:13,999][105692] Updated weights for policy 0, policy_version 1100181 (0.0009) [2023-12-26 23:23:14,049][105692] Updated weights for policy 0, policy_version 1100191 (0.0009) [2023-12-26 23:23:14,168][105620] Updated weights for policy 1, policy_version 1101253 (0.0010) [2023-12-26 23:23:14,221][105620] Updated weights for policy 1, policy_version 1101263 (0.0009) [2023-12-26 23:23:14,284][105620] Updated weights for policy 1, policy_version 1101273 (0.0009) [2023-12-26 23:23:14,663][105692] Updated weights for policy 0, policy_version 1100201 (0.0008) [2023-12-26 23:23:14,721][105692] Updated weights for policy 0, policy_version 1100211 (0.0009) [2023-12-26 23:23:14,774][105692] Updated weights for policy 0, policy_version 1100221 (0.0008) [2023-12-26 23:23:15,139][105620] Updated weights for policy 1, policy_version 1101283 (0.0009) [2023-12-26 23:23:15,200][105620] Updated weights for policy 1, policy_version 1101293 (0.0010) [2023-12-26 23:23:15,266][105620] Updated weights for policy 1, policy_version 1101303 (0.0008) [2023-12-26 23:23:15,542][105692] Updated weights for policy 0, policy_version 1100231 (0.0007) [2023-12-26 23:23:15,598][105692] Updated weights for policy 0, policy_version 1100241 (0.0005) [2023-12-26 23:23:15,654][105692] Updated weights for policy 0, policy_version 1100251 (0.0006) [2023-12-26 23:23:16,062][104569] Fps is (10 sec: 18022.0, 60 sec: 18978.1, 300 sec: 19466.4). Total num frames: 563675136. Throughput: 0: 9547.9, 1: 9476.1. Samples: 563648428. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:23:16,063][104569] Avg episode reward: [(0, '9355.771'), (1, '8894.188')] [2023-12-26 23:23:16,074][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001100256_281706496.pth... [2023-12-26 23:23:16,075][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001101312_281968640.pth... [2023-12-26 23:23:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001099136_281419776.pth [2023-12-26 23:23:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001100224_281690112.pth [2023-12-26 23:23:16,098][105620] Updated weights for policy 1, policy_version 1101313 (0.0009) [2023-12-26 23:23:16,155][105620] Updated weights for policy 1, policy_version 1101323 (0.0010) [2023-12-26 23:23:16,214][105620] Updated weights for policy 1, policy_version 1101333 (0.0008) [2023-12-26 23:23:16,227][105692] Updated weights for policy 0, policy_version 1100261 (0.0009) [2023-12-26 23:23:16,272][105620] Updated weights for policy 1, policy_version 1101343 (0.0007) [2023-12-26 23:23:16,288][105692] Updated weights for policy 0, policy_version 1100271 (0.0006) [2023-12-26 23:23:16,357][105692] Updated weights for policy 0, policy_version 1100281 (0.0006) [2023-12-26 23:23:16,904][105692] Updated weights for policy 0, policy_version 1100291 (0.0007) [2023-12-26 23:23:16,953][105620] Updated weights for policy 1, policy_version 1101353 (0.0008) [2023-12-26 23:23:16,966][105692] Updated weights for policy 0, policy_version 1100301 (0.0008) [2023-12-26 23:23:16,999][105620] Updated weights for policy 1, policy_version 1101363 (0.0009) [2023-12-26 23:23:17,024][105692] Updated weights for policy 0, policy_version 1100311 (0.0010) [2023-12-26 23:23:17,043][105620] Updated weights for policy 1, policy_version 1101373 (0.0005) [2023-12-26 23:23:17,613][105692] Updated weights for policy 0, policy_version 1100321 (0.0010) [2023-12-26 23:23:17,673][105692] Updated weights for policy 0, policy_version 1100331 (0.0006) [2023-12-26 23:23:17,731][105620] Updated weights for policy 1, policy_version 1101383 (0.0009) [2023-12-26 23:23:17,737][105692] Updated weights for policy 0, policy_version 1100341 (0.0006) [2023-12-26 23:23:17,792][105620] Updated weights for policy 1, policy_version 1101393 (0.0007) [2023-12-26 23:23:17,797][105692] Updated weights for policy 0, policy_version 1100351 (0.0008) [2023-12-26 23:23:17,853][105620] Updated weights for policy 1, policy_version 1101403 (0.0009) [2023-12-26 23:23:18,530][105692] Updated weights for policy 0, policy_version 1100361 (0.0009) [2023-12-26 23:23:18,592][105692] Updated weights for policy 0, policy_version 1100371 (0.0009) [2023-12-26 23:23:18,628][105620] Updated weights for policy 1, policy_version 1101413 (0.0008) [2023-12-26 23:23:18,647][105692] Updated weights for policy 0, policy_version 1100381 (0.0007) [2023-12-26 23:23:18,689][105620] Updated weights for policy 1, policy_version 1101423 (0.0008) [2023-12-26 23:23:18,741][105620] Updated weights for policy 1, policy_version 1101433 (0.0009) [2023-12-26 23:23:19,428][105692] Updated weights for policy 0, policy_version 1100391 (0.0007) [2023-12-26 23:23:19,437][105620] Updated weights for policy 1, policy_version 1101443 (0.0009) [2023-12-26 23:23:19,489][105692] Updated weights for policy 0, policy_version 1100401 (0.0006) [2023-12-26 23:23:19,501][105620] Updated weights for policy 1, policy_version 1101453 (0.0007) [2023-12-26 23:23:19,551][105692] Updated weights for policy 0, policy_version 1100411 (0.0007) [2023-12-26 23:23:19,555][105620] Updated weights for policy 1, policy_version 1101463 (0.0009) [2023-12-26 23:23:20,318][105692] Updated weights for policy 0, policy_version 1100421 (0.0009) [2023-12-26 23:23:20,324][105620] Updated weights for policy 1, policy_version 1101473 (0.0008) [2023-12-26 23:23:20,378][105692] Updated weights for policy 0, policy_version 1100431 (0.0009) [2023-12-26 23:23:20,378][105620] Updated weights for policy 1, policy_version 1101483 (0.0009) [2023-12-26 23:23:20,439][105692] Updated weights for policy 0, policy_version 1100441 (0.0007) [2023-12-26 23:23:20,440][105620] Updated weights for policy 1, policy_version 1101493 (0.0010) [2023-12-26 23:23:20,509][105620] Updated weights for policy 1, policy_version 1101503 (0.0009) [2023-12-26 23:23:21,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19114.6, 300 sec: 19466.4). Total num frames: 563773440. Throughput: 0: 9631.4, 1: 9382.2. Samples: 563766012. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:23:21,063][104569] Avg episode reward: [(0, '9355.713'), (1, '8802.965')] [2023-12-26 23:23:21,295][105620] Updated weights for policy 1, policy_version 1101513 (0.0008) [2023-12-26 23:23:21,304][105692] Updated weights for policy 0, policy_version 1100451 (0.0008) [2023-12-26 23:23:21,366][105620] Updated weights for policy 1, policy_version 1101523 (0.0008) [2023-12-26 23:23:21,370][105692] Updated weights for policy 0, policy_version 1100461 (0.0008) [2023-12-26 23:23:21,425][105692] Updated weights for policy 0, policy_version 1100471 (0.0006) [2023-12-26 23:23:21,431][105620] Updated weights for policy 1, policy_version 1101533 (0.0009) [2023-12-26 23:23:22,159][105692] Updated weights for policy 0, policy_version 1100481 (0.0008) [2023-12-26 23:23:22,186][105620] Updated weights for policy 1, policy_version 1101543 (0.0007) [2023-12-26 23:23:22,231][105692] Updated weights for policy 0, policy_version 1100491 (0.0008) [2023-12-26 23:23:22,245][105620] Updated weights for policy 1, policy_version 1101553 (0.0006) [2023-12-26 23:23:22,310][105692] Updated weights for policy 0, policy_version 1100501 (0.0008) [2023-12-26 23:23:22,312][105620] Updated weights for policy 1, policy_version 1101563 (0.0008) [2023-12-26 23:23:22,380][105692] Updated weights for policy 0, policy_version 1100511 (0.0009) [2023-12-26 23:23:22,889][105620] Updated weights for policy 1, policy_version 1101573 (0.0006) [2023-12-26 23:23:22,951][105620] Updated weights for policy 1, policy_version 1101583 (0.0006) [2023-12-26 23:23:23,025][105620] Updated weights for policy 1, policy_version 1101593 (0.0008) [2023-12-26 23:23:23,196][105692] Updated weights for policy 0, policy_version 1100521 (0.0006) [2023-12-26 23:23:23,267][105692] Updated weights for policy 0, policy_version 1100531 (0.0006) [2023-12-26 23:23:23,324][105692] Updated weights for policy 0, policy_version 1100541 (0.0009) [2023-12-26 23:23:23,601][105620] Updated weights for policy 1, policy_version 1101603 (0.0008) [2023-12-26 23:23:23,655][105620] Updated weights for policy 1, policy_version 1101613 (0.0009) [2023-12-26 23:23:23,715][105620] Updated weights for policy 1, policy_version 1101623 (0.0008) [2023-12-26 23:23:24,067][105692] Updated weights for policy 0, policy_version 1100551 (0.0008) [2023-12-26 23:23:24,115][105692] Updated weights for policy 0, policy_version 1100561 (0.0009) [2023-12-26 23:23:24,168][105692] Updated weights for policy 0, policy_version 1100571 (0.0009) [2023-12-26 23:23:24,388][105620] Updated weights for policy 1, policy_version 1101633 (0.0006) [2023-12-26 23:23:24,447][105620] Updated weights for policy 1, policy_version 1101643 (0.0010) [2023-12-26 23:23:24,507][105620] Updated weights for policy 1, policy_version 1101653 (0.0011) [2023-12-26 23:23:24,562][105620] Updated weights for policy 1, policy_version 1101663 (0.0011) [2023-12-26 23:23:24,960][105692] Updated weights for policy 0, policy_version 1100581 (0.0007) [2023-12-26 23:23:25,014][105692] Updated weights for policy 0, policy_version 1100591 (0.0007) [2023-12-26 23:23:25,067][105692] Updated weights for policy 0, policy_version 1100601 (0.0006) [2023-12-26 23:23:25,269][105620] Updated weights for policy 1, policy_version 1101673 (0.0009) [2023-12-26 23:23:25,331][105620] Updated weights for policy 1, policy_version 1101683 (0.0010) [2023-12-26 23:23:25,386][105620] Updated weights for policy 1, policy_version 1101693 (0.0006) [2023-12-26 23:23:25,761][105692] Updated weights for policy 0, policy_version 1100611 (0.0009) [2023-12-26 23:23:25,832][105692] Updated weights for policy 0, policy_version 1100621 (0.0007) [2023-12-26 23:23:25,895][105692] Updated weights for policy 0, policy_version 1100631 (0.0008) [2023-12-26 23:23:25,902][105620] Updated weights for policy 1, policy_version 1101703 (0.0009) [2023-12-26 23:23:25,955][105620] Updated weights for policy 1, policy_version 1101713 (0.0009) [2023-12-26 23:23:26,003][105620] Updated weights for policy 1, policy_version 1101723 (0.0010) [2023-12-26 23:23:26,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19251.1, 300 sec: 19494.2). Total num frames: 563879936. Throughput: 0: 9566.0, 1: 9494.5. Samples: 563881576. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:23:26,063][104569] Avg episode reward: [(0, '9264.850'), (1, '8892.491')] [2023-12-26 23:23:26,496][105692] Updated weights for policy 0, policy_version 1100641 (0.0008) [2023-12-26 23:23:26,553][105692] Updated weights for policy 0, policy_version 1100651 (0.0010) [2023-12-26 23:23:26,617][105692] Updated weights for policy 0, policy_version 1100661 (0.0010) [2023-12-26 23:23:26,678][105692] Updated weights for policy 0, policy_version 1100671 (0.0010) [2023-12-26 23:23:26,764][105620] Updated weights for policy 1, policy_version 1101733 (0.0010) [2023-12-26 23:23:26,819][105620] Updated weights for policy 1, policy_version 1101743 (0.0010) [2023-12-26 23:23:26,880][105620] Updated weights for policy 1, policy_version 1101753 (0.0010) [2023-12-26 23:23:27,394][105692] Updated weights for policy 0, policy_version 1100681 (0.0010) [2023-12-26 23:23:27,455][105692] Updated weights for policy 0, policy_version 1100691 (0.0010) [2023-12-26 23:23:27,512][105692] Updated weights for policy 0, policy_version 1100701 (0.0010) [2023-12-26 23:23:27,614][105620] Updated weights for policy 1, policy_version 1101763 (0.0010) [2023-12-26 23:23:27,662][105620] Updated weights for policy 1, policy_version 1101773 (0.0008) [2023-12-26 23:23:27,722][105620] Updated weights for policy 1, policy_version 1101783 (0.0009) [2023-12-26 23:23:28,177][105692] Updated weights for policy 0, policy_version 1100711 (0.0010) [2023-12-26 23:23:28,226][105692] Updated weights for policy 0, policy_version 1100721 (0.0008) [2023-12-26 23:23:28,281][105692] Updated weights for policy 0, policy_version 1100731 (0.0006) [2023-12-26 23:23:28,452][105620] Updated weights for policy 1, policy_version 1101793 (0.0010) [2023-12-26 23:23:28,500][105620] Updated weights for policy 1, policy_version 1101803 (0.0010) [2023-12-26 23:23:28,552][105620] Updated weights for policy 1, policy_version 1101813 (0.0010) [2023-12-26 23:23:28,600][105620] Updated weights for policy 1, policy_version 1101823 (0.0010) [2023-12-26 23:23:28,909][105692] Updated weights for policy 0, policy_version 1100741 (0.0005) [2023-12-26 23:23:28,963][105692] Updated weights for policy 0, policy_version 1100751 (0.0006) [2023-12-26 23:23:29,018][105692] Updated weights for policy 0, policy_version 1100761 (0.0006) [2023-12-26 23:23:29,357][105620] Updated weights for policy 1, policy_version 1101833 (0.0011) [2023-12-26 23:23:29,424][105620] Updated weights for policy 1, policy_version 1101843 (0.0007) [2023-12-26 23:23:29,494][105620] Updated weights for policy 1, policy_version 1101853 (0.0008) [2023-12-26 23:23:29,592][105692] Updated weights for policy 0, policy_version 1100771 (0.0006) [2023-12-26 23:23:29,636][105692] Updated weights for policy 0, policy_version 1100781 (0.0005) [2023-12-26 23:23:29,687][105692] Updated weights for policy 0, policy_version 1100791 (0.0005) [2023-12-26 23:23:30,171][105620] Updated weights for policy 1, policy_version 1101863 (0.0010) [2023-12-26 23:23:30,226][105620] Updated weights for policy 1, policy_version 1101873 (0.0010) [2023-12-26 23:23:30,274][105692] Updated weights for policy 0, policy_version 1100801 (0.0007) [2023-12-26 23:23:30,284][105620] Updated weights for policy 1, policy_version 1101883 (0.0010) [2023-12-26 23:23:30,319][105692] Updated weights for policy 0, policy_version 1100811 (0.0005) [2023-12-26 23:23:30,367][105692] Updated weights for policy 0, policy_version 1100821 (0.0007) [2023-12-26 23:23:30,419][105692] Updated weights for policy 0, policy_version 1100831 (0.0005) [2023-12-26 23:23:31,026][105620] Updated weights for policy 1, policy_version 1101893 (0.0010) [2023-12-26 23:23:31,042][105692] Updated weights for policy 0, policy_version 1100841 (0.0006) [2023-12-26 23:23:31,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 563970048. Throughput: 0: 9652.3, 1: 9513.8. Samples: 563941872. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:23:31,062][104569] Avg episode reward: [(0, '9264.169'), (1, '8983.732')] [2023-12-26 23:23:31,090][105620] Updated weights for policy 1, policy_version 1101903 (0.0010) [2023-12-26 23:23:31,101][105692] Updated weights for policy 0, policy_version 1100851 (0.0007) [2023-12-26 23:23:31,159][105620] Updated weights for policy 1, policy_version 1101913 (0.0010) [2023-12-26 23:23:31,167][105692] Updated weights for policy 0, policy_version 1100861 (0.0008) [2023-12-26 23:23:31,183][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001100864_281862144.pth... [2023-12-26 23:23:31,186][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001099712_281567232.pth [2023-12-26 23:23:31,203][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001101920_282124288.pth... [2023-12-26 23:23:31,208][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001100768_281829376.pth [2023-12-26 23:23:31,856][105620] Updated weights for policy 1, policy_version 1101923 (0.0009) [2023-12-26 23:23:31,859][105692] Updated weights for policy 0, policy_version 1100871 (0.0008) [2023-12-26 23:23:31,906][105620] Updated weights for policy 1, policy_version 1101933 (0.0008) [2023-12-26 23:23:31,916][105692] Updated weights for policy 0, policy_version 1100881 (0.0007) [2023-12-26 23:23:31,962][105620] Updated weights for policy 1, policy_version 1101943 (0.0010) [2023-12-26 23:23:31,978][105692] Updated weights for policy 0, policy_version 1100891 (0.0005) [2023-12-26 23:23:32,639][105692] Updated weights for policy 0, policy_version 1100901 (0.0005) [2023-12-26 23:23:32,686][105692] Updated weights for policy 0, policy_version 1100911 (0.0005) [2023-12-26 23:23:32,716][105620] Updated weights for policy 1, policy_version 1101953 (0.0010) [2023-12-26 23:23:32,737][105692] Updated weights for policy 0, policy_version 1100921 (0.0006) [2023-12-26 23:23:32,783][105620] Updated weights for policy 1, policy_version 1101963 (0.0009) [2023-12-26 23:23:32,842][105620] Updated weights for policy 1, policy_version 1101973 (0.0006) [2023-12-26 23:23:32,900][105620] Updated weights for policy 1, policy_version 1101983 (0.0007) [2023-12-26 23:23:33,458][105620] Updated weights for policy 1, policy_version 1101993 (0.0008) [2023-12-26 23:23:33,486][105692] Updated weights for policy 0, policy_version 1100931 (0.0007) [2023-12-26 23:23:33,523][105620] Updated weights for policy 1, policy_version 1102003 (0.0006) [2023-12-26 23:23:33,537][105692] Updated weights for policy 0, policy_version 1100941 (0.0005) [2023-12-26 23:23:33,585][105620] Updated weights for policy 1, policy_version 1102013 (0.0006) [2023-12-26 23:23:33,591][105692] Updated weights for policy 0, policy_version 1100951 (0.0010) [2023-12-26 23:23:34,172][105620] Updated weights for policy 1, policy_version 1102023 (0.0007) [2023-12-26 23:23:34,193][105692] Updated weights for policy 0, policy_version 1100961 (0.0010) [2023-12-26 23:23:34,223][105620] Updated weights for policy 1, policy_version 1102033 (0.0009) [2023-12-26 23:23:34,251][105692] Updated weights for policy 0, policy_version 1100971 (0.0009) [2023-12-26 23:23:34,283][105620] Updated weights for policy 1, policy_version 1102043 (0.0009) [2023-12-26 23:23:34,312][105692] Updated weights for policy 0, policy_version 1100981 (0.0008) [2023-12-26 23:23:34,364][105692] Updated weights for policy 0, policy_version 1100991 (0.0010) [2023-12-26 23:23:35,000][105620] Updated weights for policy 1, policy_version 1102053 (0.0007) [2023-12-26 23:23:35,021][105692] Updated weights for policy 0, policy_version 1101001 (0.0010) [2023-12-26 23:23:35,056][105620] Updated weights for policy 1, policy_version 1102063 (0.0005) [2023-12-26 23:23:35,066][105692] Updated weights for policy 0, policy_version 1101011 (0.0010) [2023-12-26 23:23:35,108][105620] Updated weights for policy 1, policy_version 1102073 (0.0006) [2023-12-26 23:23:35,115][105692] Updated weights for policy 0, policy_version 1101021 (0.0010) [2023-12-26 23:23:35,834][105620] Updated weights for policy 1, policy_version 1102083 (0.0006) [2023-12-26 23:23:35,895][105620] Updated weights for policy 1, policy_version 1102093 (0.0006) [2023-12-26 23:23:35,898][105692] Updated weights for policy 0, policy_version 1101031 (0.0010) [2023-12-26 23:23:35,953][105620] Updated weights for policy 1, policy_version 1102103 (0.0006) [2023-12-26 23:23:35,961][105692] Updated weights for policy 0, policy_version 1101041 (0.0010) [2023-12-26 23:23:36,022][105692] Updated weights for policy 0, policy_version 1101051 (0.0010) [2023-12-26 23:23:36,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 564084736. Throughput: 0: 9833.0, 1: 9619.2. Samples: 564066912. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:23:36,063][104569] Avg episode reward: [(0, '9263.715'), (1, '9167.788')] [2023-12-26 23:23:36,596][105620] Updated weights for policy 1, policy_version 1102113 (0.0006) [2023-12-26 23:23:36,656][105620] Updated weights for policy 1, policy_version 1102123 (0.0008) [2023-12-26 23:23:36,719][105620] Updated weights for policy 1, policy_version 1102133 (0.0008) [2023-12-26 23:23:36,772][105620] Updated weights for policy 1, policy_version 1102143 (0.0008) [2023-12-26 23:23:36,811][105692] Updated weights for policy 0, policy_version 1101061 (0.0010) [2023-12-26 23:23:36,874][105692] Updated weights for policy 0, policy_version 1101071 (0.0009) [2023-12-26 23:23:36,933][105692] Updated weights for policy 0, policy_version 1101081 (0.0009) [2023-12-26 23:23:37,466][105620] Updated weights for policy 1, policy_version 1102153 (0.0008) [2023-12-26 23:23:37,523][105620] Updated weights for policy 1, policy_version 1102163 (0.0009) [2023-12-26 23:23:37,574][105620] Updated weights for policy 1, policy_version 1102173 (0.0010) [2023-12-26 23:23:37,651][105692] Updated weights for policy 0, policy_version 1101091 (0.0007) [2023-12-26 23:23:37,717][105692] Updated weights for policy 0, policy_version 1101101 (0.0006) [2023-12-26 23:23:37,779][105692] Updated weights for policy 0, policy_version 1101111 (0.0007) [2023-12-26 23:23:38,334][105620] Updated weights for policy 1, policy_version 1102183 (0.0010) [2023-12-26 23:23:38,406][105620] Updated weights for policy 1, policy_version 1102193 (0.0007) [2023-12-26 23:23:38,438][105692] Updated weights for policy 0, policy_version 1101121 (0.0009) [2023-12-26 23:23:38,475][105620] Updated weights for policy 1, policy_version 1102203 (0.0006) [2023-12-26 23:23:38,488][105692] Updated weights for policy 0, policy_version 1101131 (0.0009) [2023-12-26 23:23:38,546][105692] Updated weights for policy 0, policy_version 1101141 (0.0009) [2023-12-26 23:23:38,602][105692] Updated weights for policy 0, policy_version 1101151 (0.0005) [2023-12-26 23:23:39,086][105620] Updated weights for policy 1, policy_version 1102213 (0.0007) [2023-12-26 23:23:39,144][105620] Updated weights for policy 1, policy_version 1102223 (0.0007) [2023-12-26 23:23:39,207][105620] Updated weights for policy 1, policy_version 1102233 (0.0011) [2023-12-26 23:23:39,377][105692] Updated weights for policy 0, policy_version 1101161 (0.0011) [2023-12-26 23:23:39,446][105692] Updated weights for policy 0, policy_version 1101171 (0.0008) [2023-12-26 23:23:39,504][105692] Updated weights for policy 0, policy_version 1101181 (0.0006) [2023-12-26 23:23:40,009][105620] Updated weights for policy 1, policy_version 1102243 (0.0009) [2023-12-26 23:23:40,068][105620] Updated weights for policy 1, policy_version 1102253 (0.0010) [2023-12-26 23:23:40,123][105620] Updated weights for policy 1, policy_version 1102263 (0.0010) [2023-12-26 23:23:40,143][105692] Updated weights for policy 0, policy_version 1101191 (0.0009) [2023-12-26 23:23:40,203][105692] Updated weights for policy 0, policy_version 1101201 (0.0011) [2023-12-26 23:23:40,252][105692] Updated weights for policy 0, policy_version 1101211 (0.0010) [2023-12-26 23:23:40,847][105620] Updated weights for policy 1, policy_version 1102273 (0.0010) [2023-12-26 23:23:40,903][105620] Updated weights for policy 1, policy_version 1102283 (0.0005) [2023-12-26 23:23:40,963][105620] Updated weights for policy 1, policy_version 1102293 (0.0005) [2023-12-26 23:23:41,014][105620] Updated weights for policy 1, policy_version 1102303 (0.0005) [2023-12-26 23:23:41,054][105692] Updated weights for policy 0, policy_version 1101221 (0.0009) [2023-12-26 23:23:41,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 564174848. Throughput: 0: 9789.1, 1: 9660.0. Samples: 564183656. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:23:41,063][104569] Avg episode reward: [(0, '9262.394'), (1, '9168.245')] [2023-12-26 23:23:41,115][105692] Updated weights for policy 0, policy_version 1101231 (0.0006) [2023-12-26 23:23:41,175][105692] Updated weights for policy 0, policy_version 1101241 (0.0008) [2023-12-26 23:23:41,762][105620] Updated weights for policy 1, policy_version 1102313 (0.0009) [2023-12-26 23:23:41,828][105620] Updated weights for policy 1, policy_version 1102323 (0.0007) [2023-12-26 23:23:41,888][105692] Updated weights for policy 0, policy_version 1101251 (0.0007) [2023-12-26 23:23:41,895][105620] Updated weights for policy 1, policy_version 1102333 (0.0009) [2023-12-26 23:23:41,951][105692] Updated weights for policy 0, policy_version 1101261 (0.0009) [2023-12-26 23:23:42,006][105692] Updated weights for policy 0, policy_version 1101271 (0.0009) [2023-12-26 23:23:42,630][105620] Updated weights for policy 1, policy_version 1102343 (0.0007) [2023-12-26 23:23:42,680][105620] Updated weights for policy 1, policy_version 1102353 (0.0005) [2023-12-26 23:23:42,747][105620] Updated weights for policy 1, policy_version 1102363 (0.0008) [2023-12-26 23:23:42,771][105692] Updated weights for policy 0, policy_version 1101281 (0.0008) [2023-12-26 23:23:42,833][105692] Updated weights for policy 0, policy_version 1101291 (0.0009) [2023-12-26 23:23:42,888][105692] Updated weights for policy 0, policy_version 1101301 (0.0009) [2023-12-26 23:23:42,947][105692] Updated weights for policy 0, policy_version 1101311 (0.0009) [2023-12-26 23:23:43,473][105620] Updated weights for policy 1, policy_version 1102373 (0.0009) [2023-12-26 23:23:43,543][105620] Updated weights for policy 1, policy_version 1102383 (0.0009) [2023-12-26 23:23:43,561][105692] Updated weights for policy 0, policy_version 1101321 (0.0006) [2023-12-26 23:23:43,598][105620] Updated weights for policy 1, policy_version 1102393 (0.0008) [2023-12-26 23:23:43,621][105692] Updated weights for policy 0, policy_version 1101331 (0.0006) [2023-12-26 23:23:43,686][105692] Updated weights for policy 0, policy_version 1101341 (0.0009) [2023-12-26 23:23:44,290][105692] Updated weights for policy 0, policy_version 1101351 (0.0009) [2023-12-26 23:23:44,347][105692] Updated weights for policy 0, policy_version 1101361 (0.0010) [2023-12-26 23:23:44,378][105620] Updated weights for policy 1, policy_version 1102403 (0.0006) [2023-12-26 23:23:44,401][105692] Updated weights for policy 0, policy_version 1101371 (0.0010) [2023-12-26 23:23:44,424][105620] Updated weights for policy 1, policy_version 1102413 (0.0005) [2023-12-26 23:23:44,472][105620] Updated weights for policy 1, policy_version 1102423 (0.0005) [2023-12-26 23:23:45,080][105620] Updated weights for policy 1, policy_version 1102433 (0.0006) [2023-12-26 23:23:45,145][105620] Updated weights for policy 1, policy_version 1102443 (0.0008) [2023-12-26 23:23:45,207][105620] Updated weights for policy 1, policy_version 1102453 (0.0009) [2023-12-26 23:23:45,260][105692] Updated weights for policy 0, policy_version 1101381 (0.0007) [2023-12-26 23:23:45,270][105620] Updated weights for policy 1, policy_version 1102463 (0.0009) [2023-12-26 23:23:45,321][105692] Updated weights for policy 0, policy_version 1101391 (0.0008) [2023-12-26 23:23:45,383][105692] Updated weights for policy 0, policy_version 1101401 (0.0009) [2023-12-26 23:23:45,966][105620] Updated weights for policy 1, policy_version 1102473 (0.0009) [2023-12-26 23:23:46,020][105620] Updated weights for policy 1, policy_version 1102484 (0.0009) [2023-12-26 23:23:46,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19251.3, 300 sec: 19494.2). Total num frames: 564264960. Throughput: 0: 9708.2, 1: 9672.3. Samples: 564240216. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:23:46,063][104569] Avg episode reward: [(0, '9351.586'), (1, '9076.685')] [2023-12-26 23:23:46,080][105620] Updated weights for policy 1, policy_version 1102494 (0.0009) [2023-12-26 23:23:46,091][105692] Updated weights for policy 0, policy_version 1101411 (0.0008) [2023-12-26 23:23:46,091][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001102496_282271744.pth... [2023-12-26 23:23:46,094][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001101312_281968640.pth [2023-12-26 23:23:46,148][105692] Updated weights for policy 0, policy_version 1101421 (0.0008) [2023-12-26 23:23:46,201][105692] Updated weights for policy 0, policy_version 1101431 (0.0005) [2023-12-26 23:23:46,252][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001101440_282009600.pth... [2023-12-26 23:23:46,270][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001100256_281706496.pth [2023-12-26 23:23:46,855][105620] Updated weights for policy 1, policy_version 1102504 (0.0009) [2023-12-26 23:23:46,871][105692] Updated weights for policy 0, policy_version 1101441 (0.0005) [2023-12-26 23:23:46,909][105620] Updated weights for policy 1, policy_version 1102514 (0.0007) [2023-12-26 23:23:46,920][105692] Updated weights for policy 0, policy_version 1101451 (0.0007) [2023-12-26 23:23:46,971][105620] Updated weights for policy 1, policy_version 1102524 (0.0008) [2023-12-26 23:23:46,973][105692] Updated weights for policy 0, policy_version 1101461 (0.0008) [2023-12-26 23:23:47,030][105692] Updated weights for policy 0, policy_version 1101471 (0.0008) [2023-12-26 23:23:47,615][105620] Updated weights for policy 1, policy_version 1102534 (0.0009) [2023-12-26 23:23:47,663][105620] Updated weights for policy 1, policy_version 1102544 (0.0010) [2023-12-26 23:23:47,725][105620] Updated weights for policy 1, policy_version 1102554 (0.0009) [2023-12-26 23:23:47,742][105692] Updated weights for policy 0, policy_version 1101481 (0.0008) [2023-12-26 23:23:47,806][105692] Updated weights for policy 0, policy_version 1101491 (0.0009) [2023-12-26 23:23:47,868][105692] Updated weights for policy 0, policy_version 1101501 (0.0009) [2023-12-26 23:23:48,458][105620] Updated weights for policy 1, policy_version 1102564 (0.0006) [2023-12-26 23:23:48,536][105620] Updated weights for policy 1, policy_version 1102574 (0.0007) [2023-12-26 23:23:48,598][105620] Updated weights for policy 1, policy_version 1102584 (0.0008) [2023-12-26 23:23:48,604][105692] Updated weights for policy 0, policy_version 1101511 (0.0008) [2023-12-26 23:23:48,665][105692] Updated weights for policy 0, policy_version 1101521 (0.0009) [2023-12-26 23:23:48,724][105692] Updated weights for policy 0, policy_version 1101531 (0.0011) [2023-12-26 23:23:49,126][105620] Updated weights for policy 1, policy_version 1102594 (0.0006) [2023-12-26 23:23:49,184][105620] Updated weights for policy 1, policy_version 1102604 (0.0005) [2023-12-26 23:23:49,248][105620] Updated weights for policy 1, policy_version 1102614 (0.0009) [2023-12-26 23:23:49,317][105620] Updated weights for policy 1, policy_version 1102624 (0.0007) [2023-12-26 23:23:49,376][105692] Updated weights for policy 0, policy_version 1101541 (0.0009) [2023-12-26 23:23:49,425][105692] Updated weights for policy 0, policy_version 1101551 (0.0008) [2023-12-26 23:23:49,475][105692] Updated weights for policy 0, policy_version 1101561 (0.0008) [2023-12-26 23:23:49,910][105620] Updated weights for policy 1, policy_version 1102634 (0.0008) [2023-12-26 23:23:49,974][105620] Updated weights for policy 1, policy_version 1102644 (0.0006) [2023-12-26 23:23:50,036][105620] Updated weights for policy 1, policy_version 1102654 (0.0010) [2023-12-26 23:23:50,271][105692] Updated weights for policy 0, policy_version 1101571 (0.0008) [2023-12-26 23:23:50,322][105692] Updated weights for policy 0, policy_version 1101581 (0.0007) [2023-12-26 23:23:50,380][105692] Updated weights for policy 0, policy_version 1101591 (0.0009) [2023-12-26 23:23:50,659][105620] Updated weights for policy 1, policy_version 1102664 (0.0010) [2023-12-26 23:23:50,726][105620] Updated weights for policy 1, policy_version 1102674 (0.0011) [2023-12-26 23:23:50,789][105620] Updated weights for policy 1, policy_version 1102684 (0.0011) [2023-12-26 23:23:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 564371456. Throughput: 0: 9758.0, 1: 9815.0. Samples: 564361024. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:23:51,062][104569] Avg episode reward: [(0, '9351.994'), (1, '9166.464')] [2023-12-26 23:23:51,204][105692] Updated weights for policy 0, policy_version 1101601 (0.0010) [2023-12-26 23:23:51,276][105692] Updated weights for policy 0, policy_version 1101612 (0.0010) [2023-12-26 23:23:51,335][105692] Updated weights for policy 0, policy_version 1101622 (0.0009) [2023-12-26 23:23:51,400][105692] Updated weights for policy 0, policy_version 1101632 (0.0006) [2023-12-26 23:23:51,533][105620] Updated weights for policy 1, policy_version 1102694 (0.0008) [2023-12-26 23:23:51,602][105620] Updated weights for policy 1, policy_version 1102704 (0.0005) [2023-12-26 23:23:51,666][105620] Updated weights for policy 1, policy_version 1102714 (0.0008) [2023-12-26 23:23:52,019][105692] Updated weights for policy 0, policy_version 1101642 (0.0006) [2023-12-26 23:23:52,088][105692] Updated weights for policy 0, policy_version 1101652 (0.0010) [2023-12-26 23:23:52,140][105692] Updated weights for policy 0, policy_version 1101662 (0.0011) [2023-12-26 23:23:52,386][105620] Updated weights for policy 1, policy_version 1102724 (0.0008) [2023-12-26 23:23:52,456][105620] Updated weights for policy 1, policy_version 1102734 (0.0007) [2023-12-26 23:23:52,524][105620] Updated weights for policy 1, policy_version 1102744 (0.0009) [2023-12-26 23:23:52,795][105692] Updated weights for policy 0, policy_version 1101672 (0.0010) [2023-12-26 23:23:52,853][105692] Updated weights for policy 0, policy_version 1101682 (0.0011) [2023-12-26 23:23:52,912][105692] Updated weights for policy 0, policy_version 1101692 (0.0010) [2023-12-26 23:23:53,249][105620] Updated weights for policy 1, policy_version 1102754 (0.0009) [2023-12-26 23:23:53,313][105620] Updated weights for policy 1, policy_version 1102764 (0.0008) [2023-12-26 23:23:53,366][105620] Updated weights for policy 1, policy_version 1102774 (0.0009) [2023-12-26 23:23:53,417][105620] Updated weights for policy 1, policy_version 1102784 (0.0008) [2023-12-26 23:23:53,596][105692] Updated weights for policy 0, policy_version 1101702 (0.0008) [2023-12-26 23:23:53,639][105692] Updated weights for policy 0, policy_version 1101712 (0.0005) [2023-12-26 23:23:53,684][105692] Updated weights for policy 0, policy_version 1101722 (0.0005) [2023-12-26 23:23:54,160][105620] Updated weights for policy 1, policy_version 1102794 (0.0005) [2023-12-26 23:23:54,211][105620] Updated weights for policy 1, policy_version 1102804 (0.0005) [2023-12-26 23:23:54,271][105620] Updated weights for policy 1, policy_version 1102814 (0.0005) [2023-12-26 23:23:54,359][105692] Updated weights for policy 0, policy_version 1101732 (0.0009) [2023-12-26 23:23:54,414][105692] Updated weights for policy 0, policy_version 1101742 (0.0010) [2023-12-26 23:23:54,465][105692] Updated weights for policy 0, policy_version 1101752 (0.0010) [2023-12-26 23:23:54,861][105620] Updated weights for policy 1, policy_version 1102824 (0.0005) [2023-12-26 23:23:54,907][105620] Updated weights for policy 1, policy_version 1102834 (0.0005) [2023-12-26 23:23:54,963][105620] Updated weights for policy 1, policy_version 1102844 (0.0006) [2023-12-26 23:23:55,167][105692] Updated weights for policy 0, policy_version 1101762 (0.0010) [2023-12-26 23:23:55,233][105692] Updated weights for policy 0, policy_version 1101772 (0.0011) [2023-12-26 23:23:55,295][105692] Updated weights for policy 0, policy_version 1101782 (0.0010) [2023-12-26 23:23:55,359][105692] Updated weights for policy 0, policy_version 1101792 (0.0011) [2023-12-26 23:23:55,599][105620] Updated weights for policy 1, policy_version 1102854 (0.0007) [2023-12-26 23:23:55,649][105620] Updated weights for policy 1, policy_version 1102864 (0.0009) [2023-12-26 23:23:55,698][105620] Updated weights for policy 1, policy_version 1102874 (0.0006) [2023-12-26 23:23:55,982][105692] Updated weights for policy 0, policy_version 1101802 (0.0005) [2023-12-26 23:23:56,039][105692] Updated weights for policy 0, policy_version 1101812 (0.0005) [2023-12-26 23:23:56,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 564469760. Throughput: 0: 9899.8, 1: 9862.8. Samples: 564480620. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-26 23:23:56,062][104569] Avg episode reward: [(0, '9352.862'), (1, '9166.996')] [2023-12-26 23:23:56,104][105692] Updated weights for policy 0, policy_version 1101822 (0.0006) [2023-12-26 23:23:56,432][105620] Updated weights for policy 1, policy_version 1102884 (0.0007) [2023-12-26 23:23:56,480][105620] Updated weights for policy 1, policy_version 1102894 (0.0009) [2023-12-26 23:23:56,528][105620] Updated weights for policy 1, policy_version 1102904 (0.0009) [2023-12-26 23:23:56,755][105692] Updated weights for policy 0, policy_version 1101832 (0.0007) [2023-12-26 23:23:56,810][105692] Updated weights for policy 0, policy_version 1101842 (0.0006) [2023-12-26 23:23:56,861][105692] Updated weights for policy 0, policy_version 1101852 (0.0008) [2023-12-26 23:23:57,161][105620] Updated weights for policy 1, policy_version 1102914 (0.0005) [2023-12-26 23:23:57,234][105620] Updated weights for policy 1, policy_version 1102924 (0.0005) [2023-12-26 23:23:57,299][105620] Updated weights for policy 1, policy_version 1102934 (0.0005) [2023-12-26 23:23:57,352][105620] Updated weights for policy 1, policy_version 1102944 (0.0005) [2023-12-26 23:23:57,511][105692] Updated weights for policy 0, policy_version 1101862 (0.0007) [2023-12-26 23:23:57,569][105692] Updated weights for policy 0, policy_version 1101872 (0.0006) [2023-12-26 23:23:57,623][105692] Updated weights for policy 0, policy_version 1101882 (0.0005) [2023-12-26 23:23:58,016][105620] Updated weights for policy 1, policy_version 1102954 (0.0006) [2023-12-26 23:23:58,062][105620] Updated weights for policy 1, policy_version 1102964 (0.0005) [2023-12-26 23:23:58,105][105620] Updated weights for policy 1, policy_version 1102974 (0.0005) [2023-12-26 23:23:58,177][105692] Updated weights for policy 0, policy_version 1101892 (0.0007) [2023-12-26 23:23:58,241][105692] Updated weights for policy 0, policy_version 1101902 (0.0009) [2023-12-26 23:23:58,300][105692] Updated weights for policy 0, policy_version 1101912 (0.0009) [2023-12-26 23:23:58,825][105620] Updated weights for policy 1, policy_version 1102984 (0.0009) [2023-12-26 23:23:58,894][105620] Updated weights for policy 1, policy_version 1102994 (0.0009) [2023-12-26 23:23:58,964][105620] Updated weights for policy 1, policy_version 1103004 (0.0010) [2023-12-26 23:23:59,129][105692] Updated weights for policy 0, policy_version 1101922 (0.0009) [2023-12-26 23:23:59,192][105692] Updated weights for policy 0, policy_version 1101932 (0.0008) [2023-12-26 23:23:59,249][105692] Updated weights for policy 0, policy_version 1101942 (0.0009) [2023-12-26 23:23:59,316][105692] Updated weights for policy 0, policy_version 1101952 (0.0008) [2023-12-26 23:23:59,753][105620] Updated weights for policy 1, policy_version 1103014 (0.0008) [2023-12-26 23:23:59,814][105620] Updated weights for policy 1, policy_version 1103024 (0.0008) [2023-12-26 23:23:59,879][105620] Updated weights for policy 1, policy_version 1103034 (0.0006) [2023-12-26 23:24:00,053][105692] Updated weights for policy 0, policy_version 1101962 (0.0006) [2023-12-26 23:24:00,110][105692] Updated weights for policy 0, policy_version 1101972 (0.0006) [2023-12-26 23:24:00,155][105692] Updated weights for policy 0, policy_version 1101982 (0.0008) [2023-12-26 23:24:00,566][105620] Updated weights for policy 1, policy_version 1103044 (0.0007) [2023-12-26 23:24:00,613][105620] Updated weights for policy 1, policy_version 1103054 (0.0009) [2023-12-26 23:24:00,659][105620] Updated weights for policy 1, policy_version 1103064 (0.0009) [2023-12-26 23:24:00,839][105692] Updated weights for policy 0, policy_version 1101992 (0.0006) [2023-12-26 23:24:00,895][105692] Updated weights for policy 0, policy_version 1102002 (0.0009) [2023-12-26 23:24:00,943][105692] Updated weights for policy 0, policy_version 1102012 (0.0010) [2023-12-26 23:24:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 564576256. Throughput: 0: 9988.2, 1: 9889.5. Samples: 564542920. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:24:01,063][104569] Avg episode reward: [(0, '9259.692'), (1, '8989.064')] [2023-12-26 23:24:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001102016_282157056.pth... [2023-12-26 23:24:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001103072_282419200.pth... [2023-12-26 23:24:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001100864_281862144.pth [2023-12-26 23:24:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001101920_282124288.pth [2023-12-26 23:24:01,360][105620] Updated weights for policy 1, policy_version 1103074 (0.0008) [2023-12-26 23:24:01,417][105620] Updated weights for policy 1, policy_version 1103084 (0.0008) [2023-12-26 23:24:01,469][105620] Updated weights for policy 1, policy_version 1103094 (0.0008) [2023-12-26 23:24:01,516][105620] Updated weights for policy 1, policy_version 1103104 (0.0007) [2023-12-26 23:24:01,693][105692] Updated weights for policy 0, policy_version 1102022 (0.0010) [2023-12-26 23:24:01,750][105692] Updated weights for policy 0, policy_version 1102032 (0.0010) [2023-12-26 23:24:01,812][105692] Updated weights for policy 0, policy_version 1102042 (0.0010) [2023-12-26 23:24:02,317][105620] Updated weights for policy 1, policy_version 1103114 (0.0008) [2023-12-26 23:24:02,383][105620] Updated weights for policy 1, policy_version 1103124 (0.0008) [2023-12-26 23:24:02,435][105620] Updated weights for policy 1, policy_version 1103134 (0.0008) [2023-12-26 23:24:02,568][105692] Updated weights for policy 0, policy_version 1102052 (0.0010) [2023-12-26 23:24:02,626][105692] Updated weights for policy 0, policy_version 1102062 (0.0010) [2023-12-26 23:24:02,691][105692] Updated weights for policy 0, policy_version 1102072 (0.0010) [2023-12-26 23:24:03,133][105620] Updated weights for policy 1, policy_version 1103144 (0.0006) [2023-12-26 23:24:03,182][105620] Updated weights for policy 1, policy_version 1103154 (0.0006) [2023-12-26 23:24:03,248][105620] Updated weights for policy 1, policy_version 1103164 (0.0008) [2023-12-26 23:24:03,422][105692] Updated weights for policy 0, policy_version 1102082 (0.0010) [2023-12-26 23:24:03,486][105692] Updated weights for policy 0, policy_version 1102092 (0.0010) [2023-12-26 23:24:03,551][105692] Updated weights for policy 0, policy_version 1102102 (0.0010) [2023-12-26 23:24:03,615][105692] Updated weights for policy 0, policy_version 1102112 (0.0010) [2023-12-26 23:24:03,932][105620] Updated weights for policy 1, policy_version 1103174 (0.0009) [2023-12-26 23:24:03,998][105620] Updated weights for policy 1, policy_version 1103184 (0.0010) [2023-12-26 23:24:04,050][105620] Updated weights for policy 1, policy_version 1103194 (0.0010) [2023-12-26 23:24:04,348][105692] Updated weights for policy 0, policy_version 1102122 (0.0008) [2023-12-26 23:24:04,412][105692] Updated weights for policy 0, policy_version 1102132 (0.0008) [2023-12-26 23:24:04,471][105692] Updated weights for policy 0, policy_version 1102142 (0.0008) [2023-12-26 23:24:04,804][105620] Updated weights for policy 1, policy_version 1103204 (0.0010) [2023-12-26 23:24:04,865][105620] Updated weights for policy 1, policy_version 1103214 (0.0010) [2023-12-26 23:24:04,935][105620] Updated weights for policy 1, policy_version 1103224 (0.0010) [2023-12-26 23:24:05,227][105692] Updated weights for policy 0, policy_version 1102152 (0.0006) [2023-12-26 23:24:05,273][105692] Updated weights for policy 0, policy_version 1102162 (0.0005) [2023-12-26 23:24:05,328][105692] Updated weights for policy 0, policy_version 1102172 (0.0006) [2023-12-26 23:24:05,522][105620] Updated weights for policy 1, policy_version 1103234 (0.0009) [2023-12-26 23:24:05,586][105620] Updated weights for policy 1, policy_version 1103244 (0.0009) [2023-12-26 23:24:05,648][105620] Updated weights for policy 1, policy_version 1103254 (0.0010) [2023-12-26 23:24:05,709][105620] Updated weights for policy 1, policy_version 1103264 (0.0010) [2023-12-26 23:24:05,851][105692] Updated weights for policy 0, policy_version 1102182 (0.0005) [2023-12-26 23:24:05,905][105692] Updated weights for policy 0, policy_version 1102192 (0.0005) [2023-12-26 23:24:05,957][105692] Updated weights for policy 0, policy_version 1102202 (0.0005) [2023-12-26 23:24:06,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 564674560. Throughput: 0: 9860.8, 1: 9930.9. Samples: 564656636. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:24:06,063][104569] Avg episode reward: [(0, '9075.996'), (1, '8804.481')] [2023-12-26 23:24:06,422][105620] Updated weights for policy 1, policy_version 1103274 (0.0005) [2023-12-26 23:24:06,490][105620] Updated weights for policy 1, policy_version 1103284 (0.0008) [2023-12-26 23:24:06,556][105620] Updated weights for policy 1, policy_version 1103294 (0.0007) [2023-12-26 23:24:06,673][105692] Updated weights for policy 0, policy_version 1102212 (0.0007) [2023-12-26 23:24:06,732][105692] Updated weights for policy 0, policy_version 1102222 (0.0010) [2023-12-26 23:24:06,790][105692] Updated weights for policy 0, policy_version 1102232 (0.0010) [2023-12-26 23:24:07,155][105620] Updated weights for policy 1, policy_version 1103304 (0.0011) [2023-12-26 23:24:07,230][105620] Updated weights for policy 1, policy_version 1103314 (0.0006) [2023-12-26 23:24:07,292][105620] Updated weights for policy 1, policy_version 1103324 (0.0009) [2023-12-26 23:24:07,551][105692] Updated weights for policy 0, policy_version 1102242 (0.0010) [2023-12-26 23:24:07,603][105692] Updated weights for policy 0, policy_version 1102252 (0.0010) [2023-12-26 23:24:07,660][105692] Updated weights for policy 0, policy_version 1102262 (0.0011) [2023-12-26 23:24:07,720][105692] Updated weights for policy 0, policy_version 1102272 (0.0011) [2023-12-26 23:24:07,895][105620] Updated weights for policy 1, policy_version 1103334 (0.0009) [2023-12-26 23:24:07,941][105620] Updated weights for policy 1, policy_version 1103344 (0.0008) [2023-12-26 23:24:07,999][105620] Updated weights for policy 1, policy_version 1103354 (0.0007) [2023-12-26 23:24:08,453][105692] Updated weights for policy 0, policy_version 1102282 (0.0010) [2023-12-26 23:24:08,503][105692] Updated weights for policy 0, policy_version 1102292 (0.0007) [2023-12-26 23:24:08,567][105692] Updated weights for policy 0, policy_version 1102302 (0.0006) [2023-12-26 23:24:08,725][105620] Updated weights for policy 1, policy_version 1103364 (0.0008) [2023-12-26 23:24:08,790][105620] Updated weights for policy 1, policy_version 1103374 (0.0010) [2023-12-26 23:24:08,860][105620] Updated weights for policy 1, policy_version 1103384 (0.0011) [2023-12-26 23:24:09,196][105692] Updated weights for policy 0, policy_version 1102312 (0.0007) [2023-12-26 23:24:09,258][105692] Updated weights for policy 0, policy_version 1102322 (0.0009) [2023-12-26 23:24:09,318][105692] Updated weights for policy 0, policy_version 1102332 (0.0008) [2023-12-26 23:24:09,535][105620] Updated weights for policy 1, policy_version 1103394 (0.0010) [2023-12-26 23:24:09,607][105620] Updated weights for policy 1, policy_version 1103404 (0.0011) [2023-12-26 23:24:09,670][105620] Updated weights for policy 1, policy_version 1103414 (0.0005) [2023-12-26 23:24:09,725][105620] Updated weights for policy 1, policy_version 1103424 (0.0005) [2023-12-26 23:24:10,051][105692] Updated weights for policy 0, policy_version 1102342 (0.0009) [2023-12-26 23:24:10,104][105692] Updated weights for policy 0, policy_version 1102352 (0.0008) [2023-12-26 23:24:10,156][105692] Updated weights for policy 0, policy_version 1102362 (0.0009) [2023-12-26 23:24:10,393][105620] Updated weights for policy 1, policy_version 1103434 (0.0010) [2023-12-26 23:24:10,456][105620] Updated weights for policy 1, policy_version 1103444 (0.0010) [2023-12-26 23:24:10,516][105620] Updated weights for policy 1, policy_version 1103454 (0.0010) [2023-12-26 23:24:10,927][105692] Updated weights for policy 0, policy_version 1102372 (0.0008) [2023-12-26 23:24:10,978][105692] Updated weights for policy 0, policy_version 1102382 (0.0009) [2023-12-26 23:24:11,041][105692] Updated weights for policy 0, policy_version 1102392 (0.0010) [2023-12-26 23:24:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 564764672. Throughput: 0: 10000.7, 1: 9920.8. Samples: 564778044. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:24:11,063][104569] Avg episode reward: [(0, '9075.952'), (1, '8985.200')] [2023-12-26 23:24:11,290][105620] Updated weights for policy 1, policy_version 1103464 (0.0009) [2023-12-26 23:24:11,357][105620] Updated weights for policy 1, policy_version 1103474 (0.0008) [2023-12-26 23:24:11,420][105620] Updated weights for policy 1, policy_version 1103484 (0.0007) [2023-12-26 23:24:11,785][105692] Updated weights for policy 0, policy_version 1102402 (0.0010) [2023-12-26 23:24:11,845][105692] Updated weights for policy 0, policy_version 1102412 (0.0011) [2023-12-26 23:24:11,905][105692] Updated weights for policy 0, policy_version 1102422 (0.0011) [2023-12-26 23:24:11,961][105692] Updated weights for policy 0, policy_version 1102432 (0.0010) [2023-12-26 23:24:12,101][105620] Updated weights for policy 1, policy_version 1103494 (0.0009) [2023-12-26 23:24:12,164][105620] Updated weights for policy 1, policy_version 1103504 (0.0010) [2023-12-26 23:24:12,223][105620] Updated weights for policy 1, policy_version 1103514 (0.0010) [2023-12-26 23:24:12,672][105692] Updated weights for policy 0, policy_version 1102442 (0.0010) [2023-12-26 23:24:12,727][105692] Updated weights for policy 0, policy_version 1102452 (0.0010) [2023-12-26 23:24:12,790][105692] Updated weights for policy 0, policy_version 1102462 (0.0011) [2023-12-26 23:24:12,965][105620] Updated weights for policy 1, policy_version 1103524 (0.0010) [2023-12-26 23:24:13,013][105620] Updated weights for policy 1, policy_version 1103534 (0.0010) [2023-12-26 23:24:13,065][105620] Updated weights for policy 1, policy_version 1103544 (0.0010) [2023-12-26 23:24:13,469][105692] Updated weights for policy 0, policy_version 1102472 (0.0009) [2023-12-26 23:24:13,530][105692] Updated weights for policy 0, policy_version 1102482 (0.0010) [2023-12-26 23:24:13,592][105692] Updated weights for policy 0, policy_version 1102492 (0.0007) [2023-12-26 23:24:13,821][105620] Updated weights for policy 1, policy_version 1103554 (0.0010) [2023-12-26 23:24:13,889][105620] Updated weights for policy 1, policy_version 1103564 (0.0010) [2023-12-26 23:24:13,952][105620] Updated weights for policy 1, policy_version 1103574 (0.0006) [2023-12-26 23:24:14,022][105620] Updated weights for policy 1, policy_version 1103584 (0.0010) [2023-12-26 23:24:14,175][105692] Updated weights for policy 0, policy_version 1102502 (0.0006) [2023-12-26 23:24:14,207][105585] KL-divergence is very high: 226.3519 [2023-12-26 23:24:14,232][105692] Updated weights for policy 0, policy_version 1102512 (0.0006) [2023-12-26 23:24:14,245][105585] KL-divergence is very high: 431.8041 [2023-12-26 23:24:14,290][105692] Updated weights for policy 0, policy_version 1102522 (0.0005) [2023-12-26 23:24:14,298][105585] KL-divergence is very high: 444.3092 [2023-12-26 23:24:14,649][105620] Updated weights for policy 1, policy_version 1103594 (0.0008) [2023-12-26 23:24:14,710][105620] Updated weights for policy 1, policy_version 1103604 (0.0005) [2023-12-26 23:24:14,774][105620] Updated weights for policy 1, policy_version 1103614 (0.0007) [2023-12-26 23:24:14,951][105692] Updated weights for policy 0, policy_version 1102532 (0.0008) [2023-12-26 23:24:15,019][105692] Updated weights for policy 0, policy_version 1102542 (0.0011) [2023-12-26 23:24:15,082][105692] Updated weights for policy 0, policy_version 1102552 (0.0010) [2023-12-26 23:24:15,517][105620] Updated weights for policy 1, policy_version 1103624 (0.0007) [2023-12-26 23:24:15,586][105620] Updated weights for policy 1, policy_version 1103634 (0.0005) [2023-12-26 23:24:15,651][105620] Updated weights for policy 1, policy_version 1103644 (0.0007) [2023-12-26 23:24:15,818][105692] Updated weights for policy 0, policy_version 1102562 (0.0011) [2023-12-26 23:24:15,869][105692] Updated weights for policy 0, policy_version 1102572 (0.0010) [2023-12-26 23:24:15,920][105692] Updated weights for policy 0, policy_version 1102582 (0.0010) [2023-12-26 23:24:15,968][105692] Updated weights for policy 0, policy_version 1102592 (0.0010) [2023-12-26 23:24:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 564871168. Throughput: 0: 9939.4, 1: 9919.4. Samples: 564835524. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:24:16,063][104569] Avg episode reward: [(0, '9258.094'), (1, '9169.323')] [2023-12-26 23:24:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001102592_282304512.pth... [2023-12-26 23:24:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001103648_282566656.pth... [2023-12-26 23:24:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001102496_282271744.pth [2023-12-26 23:24:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001101440_282009600.pth [2023-12-26 23:24:16,239][105620] Updated weights for policy 1, policy_version 1103654 (0.0008) [2023-12-26 23:24:16,287][105620] Updated weights for policy 1, policy_version 1103664 (0.0010) [2023-12-26 23:24:16,342][105620] Updated weights for policy 1, policy_version 1103674 (0.0010) [2023-12-26 23:24:16,727][105692] Updated weights for policy 0, policy_version 1102602 (0.0008) [2023-12-26 23:24:16,782][105692] Updated weights for policy 0, policy_version 1102612 (0.0008) [2023-12-26 23:24:16,829][105692] Updated weights for policy 0, policy_version 1102622 (0.0007) [2023-12-26 23:24:17,093][105620] Updated weights for policy 1, policy_version 1103684 (0.0010) [2023-12-26 23:24:17,159][105620] Updated weights for policy 1, policy_version 1103694 (0.0011) [2023-12-26 23:24:17,225][105620] Updated weights for policy 1, policy_version 1103704 (0.0005) [2023-12-26 23:24:17,587][105692] Updated weights for policy 0, policy_version 1102632 (0.0010) [2023-12-26 23:24:17,658][105692] Updated weights for policy 0, policy_version 1102642 (0.0010) [2023-12-26 23:24:17,727][105692] Updated weights for policy 0, policy_version 1102652 (0.0010) [2023-12-26 23:24:17,820][105620] Updated weights for policy 1, policy_version 1103714 (0.0008) [2023-12-26 23:24:17,873][105620] Updated weights for policy 1, policy_version 1103724 (0.0008) [2023-12-26 23:24:17,931][105620] Updated weights for policy 1, policy_version 1103734 (0.0009) [2023-12-26 23:24:17,980][105620] Updated weights for policy 1, policy_version 1103744 (0.0007) [2023-12-26 23:24:18,523][105692] Updated weights for policy 0, policy_version 1102662 (0.0007) [2023-12-26 23:24:18,584][105692] Updated weights for policy 0, policy_version 1102672 (0.0008) [2023-12-26 23:24:18,642][105692] Updated weights for policy 0, policy_version 1102682 (0.0007) [2023-12-26 23:24:18,669][105620] Updated weights for policy 1, policy_version 1103754 (0.0008) [2023-12-26 23:24:18,741][105620] Updated weights for policy 1, policy_version 1103764 (0.0008) [2023-12-26 23:24:18,805][105620] Updated weights for policy 1, policy_version 1103774 (0.0005) [2023-12-26 23:24:19,310][105692] Updated weights for policy 0, policy_version 1102692 (0.0006) [2023-12-26 23:24:19,373][105692] Updated weights for policy 0, policy_version 1102702 (0.0008) [2023-12-26 23:24:19,430][105692] Updated weights for policy 0, policy_version 1102712 (0.0005) [2023-12-26 23:24:19,437][105620] Updated weights for policy 1, policy_version 1103784 (0.0008) [2023-12-26 23:24:19,492][105620] Updated weights for policy 1, policy_version 1103794 (0.0009) [2023-12-26 23:24:19,557][105620] Updated weights for policy 1, policy_version 1103804 (0.0006) [2023-12-26 23:24:20,199][105692] Updated weights for policy 0, policy_version 1102722 (0.0008) [2023-12-26 23:24:20,265][105692] Updated weights for policy 0, policy_version 1102732 (0.0009) [2023-12-26 23:24:20,279][105620] Updated weights for policy 1, policy_version 1103814 (0.0008) [2023-12-26 23:24:20,330][105692] Updated weights for policy 0, policy_version 1102742 (0.0006) [2023-12-26 23:24:20,344][105620] Updated weights for policy 1, policy_version 1103824 (0.0009) [2023-12-26 23:24:20,385][105692] Updated weights for policy 0, policy_version 1102752 (0.0007) [2023-12-26 23:24:20,406][105620] Updated weights for policy 1, policy_version 1103834 (0.0009) [2023-12-26 23:24:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 564961280. Throughput: 0: 9806.8, 1: 9940.0. Samples: 564955516. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:24:21,062][104569] Avg episode reward: [(0, '9350.449'), (1, '9351.084')] [2023-12-26 23:24:21,088][105620] Updated weights for policy 1, policy_version 1103844 (0.0008) [2023-12-26 23:24:21,150][105620] Updated weights for policy 1, policy_version 1103854 (0.0009) [2023-12-26 23:24:21,216][105620] Updated weights for policy 1, policy_version 1103864 (0.0007) [2023-12-26 23:24:21,226][105692] Updated weights for policy 0, policy_version 1102762 (0.0007) [2023-12-26 23:24:21,298][105692] Updated weights for policy 0, policy_version 1102772 (0.0008) [2023-12-26 23:24:21,365][105692] Updated weights for policy 0, policy_version 1102782 (0.0009) [2023-12-26 23:24:22,049][105620] Updated weights for policy 1, policy_version 1103874 (0.0007) [2023-12-26 23:24:22,076][105692] Updated weights for policy 0, policy_version 1102792 (0.0010) [2023-12-26 23:24:22,105][105620] Updated weights for policy 1, policy_version 1103884 (0.0006) [2023-12-26 23:24:22,144][105692] Updated weights for policy 0, policy_version 1102802 (0.0007) [2023-12-26 23:24:22,166][105620] Updated weights for policy 1, policy_version 1103894 (0.0007) [2023-12-26 23:24:22,199][105692] Updated weights for policy 0, policy_version 1102812 (0.0006) [2023-12-26 23:24:22,235][105620] Updated weights for policy 1, policy_version 1103904 (0.0007) [2023-12-26 23:24:22,929][105620] Updated weights for policy 1, policy_version 1103914 (0.0006) [2023-12-26 23:24:22,971][105692] Updated weights for policy 0, policy_version 1102822 (0.0010) [2023-12-26 23:24:22,985][105620] Updated weights for policy 1, policy_version 1103924 (0.0005) [2023-12-26 23:24:23,031][105692] Updated weights for policy 0, policy_version 1102832 (0.0011) [2023-12-26 23:24:23,039][105620] Updated weights for policy 1, policy_version 1103934 (0.0008) [2023-12-26 23:24:23,085][105692] Updated weights for policy 0, policy_version 1102842 (0.0011) [2023-12-26 23:24:23,576][105620] Updated weights for policy 1, policy_version 1103944 (0.0010) [2023-12-26 23:24:23,631][105620] Updated weights for policy 1, policy_version 1103954 (0.0008) [2023-12-26 23:24:23,679][105620] Updated weights for policy 1, policy_version 1103964 (0.0008) [2023-12-26 23:24:23,803][105692] Updated weights for policy 0, policy_version 1102852 (0.0011) [2023-12-26 23:24:23,851][105692] Updated weights for policy 0, policy_version 1102862 (0.0010) [2023-12-26 23:24:23,895][105692] Updated weights for policy 0, policy_version 1102872 (0.0010) [2023-12-26 23:24:24,417][105620] Updated weights for policy 1, policy_version 1103974 (0.0007) [2023-12-26 23:24:24,473][105620] Updated weights for policy 1, policy_version 1103984 (0.0008) [2023-12-26 23:24:24,529][105620] Updated weights for policy 1, policy_version 1103994 (0.0009) [2023-12-26 23:24:24,595][105692] Updated weights for policy 0, policy_version 1102882 (0.0010) [2023-12-26 23:24:24,654][105692] Updated weights for policy 0, policy_version 1102892 (0.0009) [2023-12-26 23:24:24,714][105692] Updated weights for policy 0, policy_version 1102902 (0.0010) [2023-12-26 23:24:24,761][105692] Updated weights for policy 0, policy_version 1102912 (0.0010) [2023-12-26 23:24:25,231][105620] Updated weights for policy 1, policy_version 1104004 (0.0010) [2023-12-26 23:24:25,288][105620] Updated weights for policy 1, policy_version 1104014 (0.0010) [2023-12-26 23:24:25,343][105620] Updated weights for policy 1, policy_version 1104024 (0.0008) [2023-12-26 23:24:25,352][105692] Updated weights for policy 0, policy_version 1102922 (0.0005) [2023-12-26 23:24:25,411][105692] Updated weights for policy 0, policy_version 1102932 (0.0005) [2023-12-26 23:24:25,467][105692] Updated weights for policy 0, policy_version 1102942 (0.0010) [2023-12-26 23:24:25,996][105620] Updated weights for policy 1, policy_version 1104034 (0.0008) [2023-12-26 23:24:26,052][105620] Updated weights for policy 1, policy_version 1104044 (0.0007) [2023-12-26 23:24:26,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 565059584. Throughput: 0: 9799.6, 1: 9964.5. Samples: 565073040. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:24:26,063][104569] Avg episode reward: [(0, '9258.711'), (1, '9350.293')] [2023-12-26 23:24:26,101][105620] Updated weights for policy 1, policy_version 1104055 (0.0006) [2023-12-26 23:24:26,155][105692] Updated weights for policy 0, policy_version 1102952 (0.0006) [2023-12-26 23:24:26,209][105692] Updated weights for policy 0, policy_version 1102962 (0.0006) [2023-12-26 23:24:26,258][105692] Updated weights for policy 0, policy_version 1102972 (0.0006) [2023-12-26 23:24:26,798][105692] Updated weights for policy 0, policy_version 1102982 (0.0006) [2023-12-26 23:24:26,856][105692] Updated weights for policy 0, policy_version 1102992 (0.0010) [2023-12-26 23:24:26,906][105620] Updated weights for policy 1, policy_version 1104065 (0.0005) [2023-12-26 23:24:26,907][105692] Updated weights for policy 0, policy_version 1103002 (0.0005) [2023-12-26 23:24:26,968][105620] Updated weights for policy 1, policy_version 1104075 (0.0007) [2023-12-26 23:24:27,029][105620] Updated weights for policy 1, policy_version 1104085 (0.0008) [2023-12-26 23:24:27,079][105620] Updated weights for policy 1, policy_version 1104095 (0.0009) [2023-12-26 23:24:27,564][105692] Updated weights for policy 0, policy_version 1103012 (0.0007) [2023-12-26 23:24:27,633][105692] Updated weights for policy 0, policy_version 1103022 (0.0005) [2023-12-26 23:24:27,700][105692] Updated weights for policy 0, policy_version 1103032 (0.0007) [2023-12-26 23:24:27,856][105620] Updated weights for policy 1, policy_version 1104105 (0.0009) [2023-12-26 23:24:27,906][105620] Updated weights for policy 1, policy_version 1104116 (0.0009) [2023-12-26 23:24:27,957][105620] Updated weights for policy 1, policy_version 1104127 (0.0009) [2023-12-26 23:24:28,359][105692] Updated weights for policy 0, policy_version 1103042 (0.0006) [2023-12-26 23:24:28,410][105692] Updated weights for policy 0, policy_version 1103052 (0.0009) [2023-12-26 23:24:28,460][105692] Updated weights for policy 0, policy_version 1103062 (0.0008) [2023-12-26 23:24:28,507][105692] Updated weights for policy 0, policy_version 1103072 (0.0009) [2023-12-26 23:24:28,725][105620] Updated weights for policy 1, policy_version 1104137 (0.0009) [2023-12-26 23:24:28,772][105620] Updated weights for policy 1, policy_version 1104147 (0.0009) [2023-12-26 23:24:28,818][105620] Updated weights for policy 1, policy_version 1104157 (0.0008) [2023-12-26 23:24:29,279][105692] Updated weights for policy 0, policy_version 1103082 (0.0008) [2023-12-26 23:24:29,343][105692] Updated weights for policy 0, policy_version 1103092 (0.0007) [2023-12-26 23:24:29,404][105692] Updated weights for policy 0, policy_version 1103102 (0.0006) [2023-12-26 23:24:29,646][105620] Updated weights for policy 1, policy_version 1104167 (0.0009) [2023-12-26 23:24:29,708][105620] Updated weights for policy 1, policy_version 1104177 (0.0010) [2023-12-26 23:24:29,756][105620] Updated weights for policy 1, policy_version 1104187 (0.0009) [2023-12-26 23:24:30,089][105692] Updated weights for policy 0, policy_version 1103112 (0.0009) [2023-12-26 23:24:30,149][105692] Updated weights for policy 0, policy_version 1103122 (0.0009) [2023-12-26 23:24:30,206][105692] Updated weights for policy 0, policy_version 1103132 (0.0008) [2023-12-26 23:24:30,538][105620] Updated weights for policy 1, policy_version 1104197 (0.0009) [2023-12-26 23:24:30,585][105620] Updated weights for policy 1, policy_version 1104207 (0.0009) [2023-12-26 23:24:30,631][105620] Updated weights for policy 1, policy_version 1104217 (0.0008) [2023-12-26 23:24:30,904][105692] Updated weights for policy 0, policy_version 1103142 (0.0006) [2023-12-26 23:24:30,959][105692] Updated weights for policy 0, policy_version 1103152 (0.0005) [2023-12-26 23:24:31,013][105692] Updated weights for policy 0, policy_version 1103162 (0.0006) [2023-12-26 23:24:31,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19933.8, 300 sec: 19522.0). Total num frames: 565166080. Throughput: 0: 9856.7, 1: 9959.3. Samples: 565131936. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:24:31,063][104569] Avg episode reward: [(0, '9259.778'), (1, '9350.251')] [2023-12-26 23:24:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001103168_282451968.pth... [2023-12-26 23:24:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001104224_282714112.pth... [2023-12-26 23:24:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001102016_282157056.pth [2023-12-26 23:24:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001103072_282419200.pth [2023-12-26 23:24:31,501][105620] Updated weights for policy 1, policy_version 1104227 (0.0009) [2023-12-26 23:24:31,567][105620] Updated weights for policy 1, policy_version 1104237 (0.0009) [2023-12-26 23:24:31,620][105620] Updated weights for policy 1, policy_version 1104247 (0.0009) [2023-12-26 23:24:31,649][105692] Updated weights for policy 0, policy_version 1103172 (0.0011) [2023-12-26 23:24:31,713][105692] Updated weights for policy 0, policy_version 1103182 (0.0009) [2023-12-26 23:24:31,778][105692] Updated weights for policy 0, policy_version 1103192 (0.0012) [2023-12-26 23:24:32,318][105620] Updated weights for policy 1, policy_version 1104257 (0.0007) [2023-12-26 23:24:32,377][105620] Updated weights for policy 1, policy_version 1104267 (0.0009) [2023-12-26 23:24:32,438][105620] Updated weights for policy 1, policy_version 1104277 (0.0008) [2023-12-26 23:24:32,476][105692] Updated weights for policy 0, policy_version 1103202 (0.0010) [2023-12-26 23:24:32,499][105620] Updated weights for policy 1, policy_version 1104287 (0.0007) [2023-12-26 23:24:32,540][105692] Updated weights for policy 0, policy_version 1103212 (0.0011) [2023-12-26 23:24:32,609][105692] Updated weights for policy 0, policy_version 1103222 (0.0011) [2023-12-26 23:24:32,677][105692] Updated weights for policy 0, policy_version 1103232 (0.0010) [2023-12-26 23:24:33,206][105620] Updated weights for policy 1, policy_version 1104297 (0.0007) [2023-12-26 23:24:33,257][105620] Updated weights for policy 1, policy_version 1104307 (0.0008) [2023-12-26 23:24:33,317][105620] Updated weights for policy 1, policy_version 1104317 (0.0008) [2023-12-26 23:24:33,374][105692] Updated weights for policy 0, policy_version 1103242 (0.0010) [2023-12-26 23:24:33,433][105692] Updated weights for policy 0, policy_version 1103252 (0.0011) [2023-12-26 23:24:33,481][105692] Updated weights for policy 0, policy_version 1103262 (0.0010) [2023-12-26 23:24:34,071][105620] Updated weights for policy 1, policy_version 1104327 (0.0009) [2023-12-26 23:24:34,120][105620] Updated weights for policy 1, policy_version 1104337 (0.0009) [2023-12-26 23:24:34,150][105692] Updated weights for policy 0, policy_version 1103272 (0.0010) [2023-12-26 23:24:34,179][105620] Updated weights for policy 1, policy_version 1104347 (0.0007) [2023-12-26 23:24:34,216][105692] Updated weights for policy 0, policy_version 1103282 (0.0011) [2023-12-26 23:24:34,278][105692] Updated weights for policy 0, policy_version 1103292 (0.0011) [2023-12-26 23:24:35,002][105620] Updated weights for policy 1, policy_version 1104357 (0.0007) [2023-12-26 23:24:35,013][105692] Updated weights for policy 0, policy_version 1103302 (0.0008) [2023-12-26 23:24:35,048][105620] Updated weights for policy 1, policy_version 1104367 (0.0006) [2023-12-26 23:24:35,063][105692] Updated weights for policy 0, policy_version 1103312 (0.0006) [2023-12-26 23:24:35,099][105620] Updated weights for policy 1, policy_version 1104377 (0.0008) [2023-12-26 23:24:35,113][105692] Updated weights for policy 0, policy_version 1103322 (0.0006) [2023-12-26 23:24:35,737][105692] Updated weights for policy 0, policy_version 1103332 (0.0008) [2023-12-26 23:24:35,796][105692] Updated weights for policy 0, policy_version 1103342 (0.0009) [2023-12-26 23:24:35,855][105692] Updated weights for policy 0, policy_version 1103352 (0.0006) [2023-12-26 23:24:35,960][105620] Updated weights for policy 1, policy_version 1104387 (0.0008) [2023-12-26 23:24:36,025][105620] Updated weights for policy 1, policy_version 1104397 (0.0010) [2023-12-26 23:24:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 565256192. Throughput: 0: 9885.4, 1: 9803.5. Samples: 565247024. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:24:36,062][104569] Avg episode reward: [(0, '9260.985'), (1, '9168.893')] [2023-12-26 23:24:36,083][105620] Updated weights for policy 1, policy_version 1104407 (0.0009) [2023-12-26 23:24:36,529][105692] Updated weights for policy 0, policy_version 1103362 (0.0006) [2023-12-26 23:24:36,586][105692] Updated weights for policy 0, policy_version 1103372 (0.0007) [2023-12-26 23:24:36,643][105692] Updated weights for policy 0, policy_version 1103382 (0.0009) [2023-12-26 23:24:36,703][105692] Updated weights for policy 0, policy_version 1103392 (0.0009) [2023-12-26 23:24:36,857][105620] Updated weights for policy 1, policy_version 1104417 (0.0009) [2023-12-26 23:24:36,922][105620] Updated weights for policy 1, policy_version 1104427 (0.0010) [2023-12-26 23:24:36,974][105620] Updated weights for policy 1, policy_version 1104437 (0.0008) [2023-12-26 23:24:37,025][105620] Updated weights for policy 1, policy_version 1104447 (0.0010) [2023-12-26 23:24:37,379][105692] Updated weights for policy 0, policy_version 1103402 (0.0009) [2023-12-26 23:24:37,425][105692] Updated weights for policy 0, policy_version 1103412 (0.0008) [2023-12-26 23:24:37,480][105692] Updated weights for policy 0, policy_version 1103422 (0.0009) [2023-12-26 23:24:37,856][105620] Updated weights for policy 1, policy_version 1104457 (0.0010) [2023-12-26 23:24:37,911][105620] Updated weights for policy 1, policy_version 1104468 (0.0010) [2023-12-26 23:24:37,965][105620] Updated weights for policy 1, policy_version 1104478 (0.0010) [2023-12-26 23:24:38,123][105692] Updated weights for policy 0, policy_version 1103432 (0.0006) [2023-12-26 23:24:38,177][105692] Updated weights for policy 0, policy_version 1103442 (0.0005) [2023-12-26 23:24:38,233][105692] Updated weights for policy 0, policy_version 1103452 (0.0005) [2023-12-26 23:24:38,792][105620] Updated weights for policy 1, policy_version 1104488 (0.0009) [2023-12-26 23:24:38,852][105620] Updated weights for policy 1, policy_version 1104498 (0.0008) [2023-12-26 23:24:38,915][105620] Updated weights for policy 1, policy_version 1104508 (0.0007) [2023-12-26 23:24:38,927][105692] Updated weights for policy 0, policy_version 1103462 (0.0009) [2023-12-26 23:24:38,993][105692] Updated weights for policy 0, policy_version 1103472 (0.0011) [2023-12-26 23:24:39,058][105692] Updated weights for policy 0, policy_version 1103482 (0.0010) [2023-12-26 23:24:39,612][105620] Updated weights for policy 1, policy_version 1104518 (0.0009) [2023-12-26 23:24:39,671][105620] Updated weights for policy 1, policy_version 1104528 (0.0008) [2023-12-26 23:24:39,750][105620] Updated weights for policy 1, policy_version 1104538 (0.0009) [2023-12-26 23:24:39,823][105692] Updated weights for policy 0, policy_version 1103492 (0.0009) [2023-12-26 23:24:39,884][105692] Updated weights for policy 0, policy_version 1103502 (0.0008) [2023-12-26 23:24:39,951][105692] Updated weights for policy 0, policy_version 1103512 (0.0008) [2023-12-26 23:24:40,466][105620] Updated weights for policy 1, policy_version 1104548 (0.0008) [2023-12-26 23:24:40,527][105620] Updated weights for policy 1, policy_version 1104558 (0.0006) [2023-12-26 23:24:40,587][105620] Updated weights for policy 1, policy_version 1104568 (0.0006) [2023-12-26 23:24:40,771][105692] Updated weights for policy 0, policy_version 1103522 (0.0008) [2023-12-26 23:24:40,822][105692] Updated weights for policy 0, policy_version 1103532 (0.0008) [2023-12-26 23:24:40,879][105692] Updated weights for policy 0, policy_version 1103542 (0.0009) [2023-12-26 23:24:40,932][105692] Updated weights for policy 0, policy_version 1103552 (0.0006) [2023-12-26 23:24:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 565354496. Throughput: 0: 9888.4, 1: 9678.3. Samples: 565361124. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:24:41,062][104569] Avg episode reward: [(0, '9352.202'), (1, '9169.055')] [2023-12-26 23:24:41,234][105620] Updated weights for policy 1, policy_version 1104578 (0.0007) [2023-12-26 23:24:41,297][105620] Updated weights for policy 1, policy_version 1104588 (0.0009) [2023-12-26 23:24:41,367][105620] Updated weights for policy 1, policy_version 1104598 (0.0009) [2023-12-26 23:24:41,424][105620] Updated weights for policy 1, policy_version 1104608 (0.0008) [2023-12-26 23:24:41,693][105692] Updated weights for policy 0, policy_version 1103562 (0.0009) [2023-12-26 23:24:41,758][105692] Updated weights for policy 0, policy_version 1103572 (0.0010) [2023-12-26 23:24:41,826][105692] Updated weights for policy 0, policy_version 1103582 (0.0009) [2023-12-26 23:24:42,155][105620] Updated weights for policy 1, policy_version 1104618 (0.0009) [2023-12-26 23:24:42,220][105620] Updated weights for policy 1, policy_version 1104628 (0.0009) [2023-12-26 23:24:42,284][105620] Updated weights for policy 1, policy_version 1104638 (0.0008) [2023-12-26 23:24:42,556][105692] Updated weights for policy 0, policy_version 1103592 (0.0009) [2023-12-26 23:24:42,628][105692] Updated weights for policy 0, policy_version 1103602 (0.0011) [2023-12-26 23:24:42,690][105692] Updated weights for policy 0, policy_version 1103612 (0.0011) [2023-12-26 23:24:42,971][105620] Updated weights for policy 1, policy_version 1104648 (0.0008) [2023-12-26 23:24:43,029][105620] Updated weights for policy 1, policy_version 1104658 (0.0008) [2023-12-26 23:24:43,086][105620] Updated weights for policy 1, policy_version 1104668 (0.0010) [2023-12-26 23:24:43,332][105692] Updated weights for policy 0, policy_version 1103622 (0.0007) [2023-12-26 23:24:43,383][105692] Updated weights for policy 0, policy_version 1103632 (0.0005) [2023-12-26 23:24:43,448][105692] Updated weights for policy 0, policy_version 1103642 (0.0005) [2023-12-26 23:24:43,807][105620] Updated weights for policy 1, policy_version 1104678 (0.0010) [2023-12-26 23:24:43,865][105620] Updated weights for policy 1, policy_version 1104688 (0.0010) [2023-12-26 23:24:43,918][105620] Updated weights for policy 1, policy_version 1104698 (0.0010) [2023-12-26 23:24:44,001][105692] Updated weights for policy 0, policy_version 1103652 (0.0008) [2023-12-26 23:24:44,055][105692] Updated weights for policy 0, policy_version 1103662 (0.0005) [2023-12-26 23:24:44,109][105692] Updated weights for policy 0, policy_version 1103672 (0.0005) [2023-12-26 23:24:44,589][105620] Updated weights for policy 1, policy_version 1104708 (0.0010) [2023-12-26 23:24:44,638][105620] Updated weights for policy 1, policy_version 1104718 (0.0010) [2023-12-26 23:24:44,683][105620] Updated weights for policy 1, policy_version 1104728 (0.0010) [2023-12-26 23:24:44,826][105692] Updated weights for policy 0, policy_version 1103682 (0.0010) [2023-12-26 23:24:44,888][105692] Updated weights for policy 0, policy_version 1103692 (0.0008) [2023-12-26 23:24:44,953][105692] Updated weights for policy 0, policy_version 1103702 (0.0008) [2023-12-26 23:24:45,011][105692] Updated weights for policy 0, policy_version 1103712 (0.0009) [2023-12-26 23:24:45,512][105620] Updated weights for policy 1, policy_version 1104738 (0.0010) [2023-12-26 23:24:45,577][105620] Updated weights for policy 1, policy_version 1104748 (0.0009) [2023-12-26 23:24:45,637][105620] Updated weights for policy 1, policy_version 1104758 (0.0009) [2023-12-26 23:24:45,680][105692] Updated weights for policy 0, policy_version 1103722 (0.0006) [2023-12-26 23:24:45,690][105620] Updated weights for policy 1, policy_version 1104768 (0.0008) [2023-12-26 23:24:45,727][105692] Updated weights for policy 0, policy_version 1103732 (0.0006) [2023-12-26 23:24:45,773][105692] Updated weights for policy 0, policy_version 1103742 (0.0006) [2023-12-26 23:24:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19494.2). Total num frames: 565452800. Throughput: 0: 9824.2, 1: 9660.0. Samples: 565419708. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:24:46,062][104569] Avg episode reward: [(0, '9260.971'), (1, '9351.030')] [2023-12-26 23:24:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001103744_282599424.pth... [2023-12-26 23:24:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001104768_282853376.pth... [2023-12-26 23:24:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001103648_282566656.pth [2023-12-26 23:24:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001102592_282304512.pth [2023-12-26 23:24:46,342][105692] Updated weights for policy 0, policy_version 1103752 (0.0008) [2023-12-26 23:24:46,394][105692] Updated weights for policy 0, policy_version 1103762 (0.0009) [2023-12-26 23:24:46,394][105620] Updated weights for policy 1, policy_version 1104778 (0.0010) [2023-12-26 23:24:46,451][105692] Updated weights for policy 0, policy_version 1103772 (0.0007) [2023-12-26 23:24:46,453][105620] Updated weights for policy 1, policy_version 1104788 (0.0010) [2023-12-26 23:24:46,508][105620] Updated weights for policy 1, policy_version 1104798 (0.0010) [2023-12-26 23:24:47,110][105692] Updated weights for policy 0, policy_version 1103782 (0.0009) [2023-12-26 23:24:47,161][105692] Updated weights for policy 0, policy_version 1103792 (0.0010) [2023-12-26 23:24:47,209][105692] Updated weights for policy 0, policy_version 1103802 (0.0010) [2023-12-26 23:24:47,210][105620] Updated weights for policy 1, policy_version 1104808 (0.0008) [2023-12-26 23:24:47,262][105620] Updated weights for policy 1, policy_version 1104818 (0.0010) [2023-12-26 23:24:47,319][105620] Updated weights for policy 1, policy_version 1104828 (0.0010) [2023-12-26 23:24:47,914][105620] Updated weights for policy 1, policy_version 1104838 (0.0007) [2023-12-26 23:24:47,965][105620] Updated weights for policy 1, policy_version 1104848 (0.0007) [2023-12-26 23:24:47,975][105692] Updated weights for policy 0, policy_version 1103812 (0.0008) [2023-12-26 23:24:48,016][105620] Updated weights for policy 1, policy_version 1104858 (0.0008) [2023-12-26 23:24:48,030][105692] Updated weights for policy 0, policy_version 1103822 (0.0011) [2023-12-26 23:24:48,082][105692] Updated weights for policy 0, policy_version 1103832 (0.0010) [2023-12-26 23:24:48,620][105620] Updated weights for policy 1, policy_version 1104868 (0.0006) [2023-12-26 23:24:48,677][105620] Updated weights for policy 1, policy_version 1104878 (0.0007) [2023-12-26 23:24:48,726][105620] Updated weights for policy 1, policy_version 1104888 (0.0008) [2023-12-26 23:24:48,800][105692] Updated weights for policy 0, policy_version 1103842 (0.0010) [2023-12-26 23:24:48,845][105692] Updated weights for policy 0, policy_version 1103852 (0.0008) [2023-12-26 23:24:48,900][105692] Updated weights for policy 0, policy_version 1103862 (0.0006) [2023-12-26 23:24:48,954][105692] Updated weights for policy 0, policy_version 1103872 (0.0005) [2023-12-26 23:24:49,545][105620] Updated weights for policy 1, policy_version 1104898 (0.0008) [2023-12-26 23:24:49,590][105692] Updated weights for policy 0, policy_version 1103882 (0.0007) [2023-12-26 23:24:49,606][105620] Updated weights for policy 1, policy_version 1104908 (0.0007) [2023-12-26 23:24:49,648][105692] Updated weights for policy 0, policy_version 1103892 (0.0007) [2023-12-26 23:24:49,669][105620] Updated weights for policy 1, policy_version 1104918 (0.0009) [2023-12-26 23:24:49,704][105692] Updated weights for policy 0, policy_version 1103902 (0.0010) [2023-12-26 23:24:49,731][105620] Updated weights for policy 1, policy_version 1104928 (0.0009) [2023-12-26 23:24:50,419][105692] Updated weights for policy 0, policy_version 1103912 (0.0008) [2023-12-26 23:24:50,477][105692] Updated weights for policy 0, policy_version 1103922 (0.0007) [2023-12-26 23:24:50,493][105620] Updated weights for policy 1, policy_version 1104938 (0.0008) [2023-12-26 23:24:50,545][105692] Updated weights for policy 0, policy_version 1103932 (0.0006) [2023-12-26 23:24:50,551][105620] Updated weights for policy 1, policy_version 1104948 (0.0009) [2023-12-26 23:24:50,619][105620] Updated weights for policy 1, policy_version 1104958 (0.0008) [2023-12-26 23:24:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 565551104. Throughput: 0: 9974.6, 1: 9695.2. Samples: 565541776. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:24:51,063][104569] Avg episode reward: [(0, '9259.650'), (1, '9263.164')] [2023-12-26 23:24:51,261][105692] Updated weights for policy 0, policy_version 1103942 (0.0008) [2023-12-26 23:24:51,320][105692] Updated weights for policy 0, policy_version 1103952 (0.0009) [2023-12-26 23:24:51,381][105692] Updated weights for policy 0, policy_version 1103962 (0.0009) [2023-12-26 23:24:51,403][105620] Updated weights for policy 1, policy_version 1104968 (0.0008) [2023-12-26 23:24:51,458][105620] Updated weights for policy 1, policy_version 1104978 (0.0009) [2023-12-26 23:24:51,512][105620] Updated weights for policy 1, policy_version 1104988 (0.0010) [2023-12-26 23:24:52,101][105692] Updated weights for policy 0, policy_version 1103972 (0.0007) [2023-12-26 23:24:52,161][105692] Updated weights for policy 0, policy_version 1103982 (0.0008) [2023-12-26 23:24:52,228][105692] Updated weights for policy 0, policy_version 1103992 (0.0009) [2023-12-26 23:24:52,277][105620] Updated weights for policy 1, policy_version 1104999 (0.0007) [2023-12-26 23:24:52,349][105620] Updated weights for policy 1, policy_version 1105009 (0.0006) [2023-12-26 23:24:52,409][105620] Updated weights for policy 1, policy_version 1105019 (0.0006) [2023-12-26 23:24:52,952][105692] Updated weights for policy 0, policy_version 1104002 (0.0009) [2023-12-26 23:24:52,974][105620] Updated weights for policy 1, policy_version 1105029 (0.0006) [2023-12-26 23:24:53,009][105692] Updated weights for policy 0, policy_version 1104012 (0.0006) [2023-12-26 23:24:53,031][105620] Updated weights for policy 1, policy_version 1105039 (0.0007) [2023-12-26 23:24:53,071][105692] Updated weights for policy 0, policy_version 1104022 (0.0007) [2023-12-26 23:24:53,085][105620] Updated weights for policy 1, policy_version 1105049 (0.0005) [2023-12-26 23:24:53,123][105692] Updated weights for policy 0, policy_version 1104032 (0.0008) [2023-12-26 23:24:53,641][105620] Updated weights for policy 1, policy_version 1105059 (0.0006) [2023-12-26 23:24:53,691][105620] Updated weights for policy 1, policy_version 1105069 (0.0007) [2023-12-26 23:24:53,750][105620] Updated weights for policy 1, policy_version 1105079 (0.0005) [2023-12-26 23:24:54,015][105692] Updated weights for policy 0, policy_version 1104042 (0.0009) [2023-12-26 23:24:54,072][105692] Updated weights for policy 0, policy_version 1104052 (0.0007) [2023-12-26 23:24:54,132][105692] Updated weights for policy 0, policy_version 1104062 (0.0009) [2023-12-26 23:24:54,365][105620] Updated weights for policy 1, policy_version 1105089 (0.0005) [2023-12-26 23:24:54,436][105620] Updated weights for policy 1, policy_version 1105099 (0.0006) [2023-12-26 23:24:54,486][105620] Updated weights for policy 1, policy_version 1105109 (0.0008) [2023-12-26 23:24:54,533][105620] Updated weights for policy 1, policy_version 1105119 (0.0009) [2023-12-26 23:24:54,919][105692] Updated weights for policy 0, policy_version 1104072 (0.0009) [2023-12-26 23:24:54,986][105692] Updated weights for policy 0, policy_version 1104082 (0.0009) [2023-12-26 23:24:55,045][105692] Updated weights for policy 0, policy_version 1104092 (0.0009) [2023-12-26 23:24:55,259][105620] Updated weights for policy 1, policy_version 1105129 (0.0010) [2023-12-26 23:24:55,310][105620] Updated weights for policy 1, policy_version 1105139 (0.0010) [2023-12-26 23:24:55,354][105620] Updated weights for policy 1, policy_version 1105149 (0.0010) [2023-12-26 23:24:55,805][105692] Updated weights for policy 0, policy_version 1104102 (0.0009) [2023-12-26 23:24:55,864][105692] Updated weights for policy 0, policy_version 1104112 (0.0009) [2023-12-26 23:24:55,931][105692] Updated weights for policy 0, policy_version 1104122 (0.0008) [2023-12-26 23:24:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 565649408. Throughput: 0: 9849.9, 1: 9677.0. Samples: 565656752. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:24:56,062][104569] Avg episode reward: [(0, '9259.267'), (1, '9262.222')] [2023-12-26 23:24:56,084][105620] Updated weights for policy 1, policy_version 1105159 (0.0009) [2023-12-26 23:24:56,135][105620] Updated weights for policy 1, policy_version 1105169 (0.0008) [2023-12-26 23:24:56,187][105620] Updated weights for policy 1, policy_version 1105179 (0.0008) [2023-12-26 23:24:56,761][105692] Updated weights for policy 0, policy_version 1104132 (0.0009) [2023-12-26 23:24:56,820][105692] Updated weights for policy 0, policy_version 1104142 (0.0009) [2023-12-26 23:24:56,831][105620] Updated weights for policy 1, policy_version 1105189 (0.0006) [2023-12-26 23:24:56,871][105692] Updated weights for policy 0, policy_version 1104152 (0.0008) [2023-12-26 23:24:56,881][105620] Updated weights for policy 1, policy_version 1105199 (0.0009) [2023-12-26 23:24:56,938][105620] Updated weights for policy 1, policy_version 1105209 (0.0007) [2023-12-26 23:24:57,613][105692] Updated weights for policy 0, policy_version 1104162 (0.0008) [2023-12-26 23:24:57,641][105620] Updated weights for policy 1, policy_version 1105219 (0.0008) [2023-12-26 23:24:57,675][105692] Updated weights for policy 0, policy_version 1104172 (0.0010) [2023-12-26 23:24:57,694][105620] Updated weights for policy 1, policy_version 1105229 (0.0006) [2023-12-26 23:24:57,728][105692] Updated weights for policy 0, policy_version 1104182 (0.0011) [2023-12-26 23:24:57,748][105620] Updated weights for policy 1, policy_version 1105239 (0.0006) [2023-12-26 23:24:57,776][105692] Updated weights for policy 0, policy_version 1104192 (0.0010) [2023-12-26 23:24:58,419][105620] Updated weights for policy 1, policy_version 1105249 (0.0008) [2023-12-26 23:24:58,488][105620] Updated weights for policy 1, policy_version 1105259 (0.0008) [2023-12-26 23:24:58,506][105692] Updated weights for policy 0, policy_version 1104202 (0.0008) [2023-12-26 23:24:58,555][105620] Updated weights for policy 1, policy_version 1105269 (0.0008) [2023-12-26 23:24:58,567][105692] Updated weights for policy 0, policy_version 1104212 (0.0008) [2023-12-26 23:24:58,628][105620] Updated weights for policy 1, policy_version 1105279 (0.0009) [2023-12-26 23:24:58,637][105692] Updated weights for policy 0, policy_version 1104222 (0.0009) [2023-12-26 23:24:59,494][105620] Updated weights for policy 1, policy_version 1105289 (0.0010) [2023-12-26 23:24:59,495][105692] Updated weights for policy 0, policy_version 1104232 (0.0006) [2023-12-26 23:24:59,554][105620] Updated weights for policy 1, policy_version 1105299 (0.0011) [2023-12-26 23:24:59,556][105692] Updated weights for policy 0, policy_version 1104242 (0.0006) [2023-12-26 23:24:59,611][105692] Updated weights for policy 0, policy_version 1104252 (0.0005) [2023-12-26 23:24:59,614][105620] Updated weights for policy 1, policy_version 1105309 (0.0011) [2023-12-26 23:25:00,264][105692] Updated weights for policy 0, policy_version 1104262 (0.0005) [2023-12-26 23:25:00,319][105692] Updated weights for policy 0, policy_version 1104272 (0.0005) [2023-12-26 23:25:00,368][105620] Updated weights for policy 1, policy_version 1105319 (0.0008) [2023-12-26 23:25:00,380][105692] Updated weights for policy 0, policy_version 1104282 (0.0006) [2023-12-26 23:25:00,419][105620] Updated weights for policy 1, policy_version 1105329 (0.0005) [2023-12-26 23:25:00,484][105620] Updated weights for policy 1, policy_version 1105339 (0.0011) [2023-12-26 23:25:01,041][105620] Updated weights for policy 1, policy_version 1105349 (0.0009) [2023-12-26 23:25:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 565739520. Throughput: 0: 9819.0, 1: 9710.5. Samples: 565714348. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:25:01,063][104569] Avg episode reward: [(0, '9261.940'), (1, '9260.773')] [2023-12-26 23:25:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001104288_282738688.pth... [2023-12-26 23:25:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001103168_282451968.pth [2023-12-26 23:25:01,102][105620] Updated weights for policy 1, policy_version 1105359 (0.0008) [2023-12-26 23:25:01,148][105692] Updated weights for policy 0, policy_version 1104292 (0.0009) [2023-12-26 23:25:01,169][105620] Updated weights for policy 1, policy_version 1105369 (0.0008) [2023-12-26 23:25:01,209][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001105376_283009024.pth... [2023-12-26 23:25:01,210][105692] Updated weights for policy 0, policy_version 1104302 (0.0009) [2023-12-26 23:25:01,213][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001104224_282714112.pth [2023-12-26 23:25:01,273][105692] Updated weights for policy 0, policy_version 1104312 (0.0008) [2023-12-26 23:25:01,876][105620] Updated weights for policy 1, policy_version 1105379 (0.0007) [2023-12-26 23:25:01,935][105620] Updated weights for policy 1, policy_version 1105389 (0.0006) [2023-12-26 23:25:01,994][105620] Updated weights for policy 1, policy_version 1105399 (0.0009) [2023-12-26 23:25:02,064][105692] Updated weights for policy 0, policy_version 1104322 (0.0008) [2023-12-26 23:25:02,127][105692] Updated weights for policy 0, policy_version 1104332 (0.0007) [2023-12-26 23:25:02,191][105692] Updated weights for policy 0, policy_version 1104342 (0.0007) [2023-12-26 23:25:02,263][105692] Updated weights for policy 0, policy_version 1104352 (0.0008) [2023-12-26 23:25:02,661][105620] Updated weights for policy 1, policy_version 1105409 (0.0008) [2023-12-26 23:25:02,732][105620] Updated weights for policy 1, policy_version 1105419 (0.0005) [2023-12-26 23:25:02,797][105620] Updated weights for policy 1, policy_version 1105429 (0.0006) [2023-12-26 23:25:02,852][105620] Updated weights for policy 1, policy_version 1105439 (0.0009) [2023-12-26 23:25:03,072][105585] KL-divergence is very high: 140.7480 [2023-12-26 23:25:03,076][105692] Updated weights for policy 0, policy_version 1104362 (0.0010) [2023-12-26 23:25:03,109][105585] KL-divergence is very high: 110.0442 [2023-12-26 23:25:03,114][105585] KL-divergence is very high: 227.0566 [2023-12-26 23:25:03,130][105692] Updated weights for policy 0, policy_version 1104372 (0.0009) [2023-12-26 23:25:03,157][105585] KL-divergence is very high: 199.9531 [2023-12-26 23:25:03,190][105692] Updated weights for policy 0, policy_version 1104382 (0.0009) [2023-12-26 23:25:03,389][105620] Updated weights for policy 1, policy_version 1105449 (0.0006) [2023-12-26 23:25:03,451][105620] Updated weights for policy 1, policy_version 1105459 (0.0007) [2023-12-26 23:25:03,508][105620] Updated weights for policy 1, policy_version 1105469 (0.0009) [2023-12-26 23:25:04,079][105692] Updated weights for policy 0, policy_version 1104392 (0.0009) [2023-12-26 23:25:04,127][105692] Updated weights for policy 0, policy_version 1104402 (0.0008) [2023-12-26 23:25:04,181][105620] Updated weights for policy 1, policy_version 1105479 (0.0010) [2023-12-26 23:25:04,185][105692] Updated weights for policy 0, policy_version 1104412 (0.0009) [2023-12-26 23:25:04,237][105620] Updated weights for policy 1, policy_version 1105489 (0.0011) [2023-12-26 23:25:04,282][105620] Updated weights for policy 1, policy_version 1105499 (0.0010) [2023-12-26 23:25:04,857][105620] Updated weights for policy 1, policy_version 1105509 (0.0007) [2023-12-26 23:25:04,906][105620] Updated weights for policy 1, policy_version 1105519 (0.0006) [2023-12-26 23:25:04,968][105620] Updated weights for policy 1, policy_version 1105529 (0.0005) [2023-12-26 23:25:05,035][105692] Updated weights for policy 0, policy_version 1104422 (0.0009) [2023-12-26 23:25:05,083][105692] Updated weights for policy 0, policy_version 1104432 (0.0008) [2023-12-26 23:25:05,127][105692] Updated weights for policy 0, policy_version 1104442 (0.0007) [2023-12-26 23:25:05,659][105620] Updated weights for policy 1, policy_version 1105539 (0.0008) [2023-12-26 23:25:05,704][105620] Updated weights for policy 1, policy_version 1105549 (0.0010) [2023-12-26 23:25:05,749][105620] Updated weights for policy 1, policy_version 1105559 (0.0010) [2023-12-26 23:25:05,791][105692] Updated weights for policy 0, policy_version 1104452 (0.0008) [2023-12-26 23:25:05,839][105692] Updated weights for policy 0, policy_version 1104462 (0.0008) [2023-12-26 23:25:05,888][105692] Updated weights for policy 0, policy_version 1104472 (0.0010) [2023-12-26 23:25:06,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 565846016. Throughput: 0: 9693.0, 1: 9746.0. Samples: 565830276. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:25:06,063][104569] Avg episode reward: [(0, '9080.822'), (1, '9256.813')] [2023-12-26 23:25:06,530][105692] Updated weights for policy 0, policy_version 1104482 (0.0009) [2023-12-26 23:25:06,555][105620] Updated weights for policy 1, policy_version 1105569 (0.0010) [2023-12-26 23:25:06,594][105692] Updated weights for policy 0, policy_version 1104492 (0.0011) [2023-12-26 23:25:06,615][105620] Updated weights for policy 1, policy_version 1105579 (0.0011) [2023-12-26 23:25:06,653][105692] Updated weights for policy 0, policy_version 1104502 (0.0010) [2023-12-26 23:25:06,673][105620] Updated weights for policy 1, policy_version 1105589 (0.0011) [2023-12-26 23:25:06,713][105692] Updated weights for policy 0, policy_version 1104512 (0.0011) [2023-12-26 23:25:06,741][105620] Updated weights for policy 1, policy_version 1105599 (0.0011) [2023-12-26 23:25:07,464][105692] Updated weights for policy 0, policy_version 1104522 (0.0005) [2023-12-26 23:25:07,503][105620] Updated weights for policy 1, policy_version 1105609 (0.0011) [2023-12-26 23:25:07,523][105692] Updated weights for policy 0, policy_version 1104532 (0.0005) [2023-12-26 23:25:07,551][105620] Updated weights for policy 1, policy_version 1105619 (0.0010) [2023-12-26 23:25:07,580][105692] Updated weights for policy 0, policy_version 1104542 (0.0005) [2023-12-26 23:25:07,600][105620] Updated weights for policy 1, policy_version 1105629 (0.0010) [2023-12-26 23:25:08,097][105692] Updated weights for policy 0, policy_version 1104552 (0.0005) [2023-12-26 23:25:08,161][105692] Updated weights for policy 0, policy_version 1104562 (0.0005) [2023-12-26 23:25:08,226][105692] Updated weights for policy 0, policy_version 1104572 (0.0005) [2023-12-26 23:25:08,307][105620] Updated weights for policy 1, policy_version 1105639 (0.0010) [2023-12-26 23:25:08,372][105620] Updated weights for policy 1, policy_version 1105649 (0.0009) [2023-12-26 23:25:08,421][105620] Updated weights for policy 1, policy_version 1105659 (0.0010) [2023-12-26 23:25:08,846][105692] Updated weights for policy 0, policy_version 1104582 (0.0009) [2023-12-26 23:25:08,908][105692] Updated weights for policy 0, policy_version 1104592 (0.0009) [2023-12-26 23:25:08,964][105692] Updated weights for policy 0, policy_version 1104602 (0.0008) [2023-12-26 23:25:09,055][105620] Updated weights for policy 1, policy_version 1105669 (0.0008) [2023-12-26 23:25:09,109][105620] Updated weights for policy 1, policy_version 1105679 (0.0010) [2023-12-26 23:25:09,157][105620] Updated weights for policy 1, policy_version 1105689 (0.0010) [2023-12-26 23:25:09,683][105692] Updated weights for policy 0, policy_version 1104612 (0.0007) [2023-12-26 23:25:09,744][105692] Updated weights for policy 0, policy_version 1104622 (0.0008) [2023-12-26 23:25:09,802][105692] Updated weights for policy 0, policy_version 1104632 (0.0008) [2023-12-26 23:25:09,975][105620] Updated weights for policy 1, policy_version 1105699 (0.0010) [2023-12-26 23:25:10,031][105620] Updated weights for policy 1, policy_version 1105709 (0.0009) [2023-12-26 23:25:10,091][105620] Updated weights for policy 1, policy_version 1105719 (0.0009) [2023-12-26 23:25:10,499][105692] Updated weights for policy 0, policy_version 1104642 (0.0009) [2023-12-26 23:25:10,557][105692] Updated weights for policy 0, policy_version 1104652 (0.0010) [2023-12-26 23:25:10,610][105692] Updated weights for policy 0, policy_version 1104662 (0.0009) [2023-12-26 23:25:10,668][105692] Updated weights for policy 0, policy_version 1104672 (0.0009) [2023-12-26 23:25:10,838][105620] Updated weights for policy 1, policy_version 1105730 (0.0009) [2023-12-26 23:25:10,892][105620] Updated weights for policy 1, policy_version 1105740 (0.0008) [2023-12-26 23:25:10,944][105620] Updated weights for policy 1, policy_version 1105750 (0.0008) [2023-12-26 23:25:10,994][105620] Updated weights for policy 1, policy_version 1105760 (0.0006) [2023-12-26 23:25:11,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 565944320. Throughput: 0: 9776.5, 1: 9686.0. Samples: 565948856. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:25:11,063][104569] Avg episode reward: [(0, '9081.535'), (1, '9166.377')] [2023-12-26 23:25:11,463][105692] Updated weights for policy 0, policy_version 1104682 (0.0008) [2023-12-26 23:25:11,518][105692] Updated weights for policy 0, policy_version 1104692 (0.0009) [2023-12-26 23:25:11,569][105692] Updated weights for policy 0, policy_version 1104702 (0.0009) [2023-12-26 23:25:11,757][105620] Updated weights for policy 1, policy_version 1105770 (0.0010) [2023-12-26 23:25:11,816][105620] Updated weights for policy 1, policy_version 1105780 (0.0009) [2023-12-26 23:25:11,871][105620] Updated weights for policy 1, policy_version 1105790 (0.0008) [2023-12-26 23:25:12,386][105692] Updated weights for policy 0, policy_version 1104712 (0.0008) [2023-12-26 23:25:12,455][105692] Updated weights for policy 0, policy_version 1104722 (0.0005) [2023-12-26 23:25:12,515][105692] Updated weights for policy 0, policy_version 1104732 (0.0006) [2023-12-26 23:25:12,665][105620] Updated weights for policy 1, policy_version 1105800 (0.0009) [2023-12-26 23:25:12,720][105620] Updated weights for policy 1, policy_version 1105810 (0.0009) [2023-12-26 23:25:12,775][105620] Updated weights for policy 1, policy_version 1105820 (0.0009) [2023-12-26 23:25:13,119][105692] Updated weights for policy 0, policy_version 1104742 (0.0007) [2023-12-26 23:25:13,187][105692] Updated weights for policy 0, policy_version 1104752 (0.0007) [2023-12-26 23:25:13,252][105692] Updated weights for policy 0, policy_version 1104762 (0.0009) [2023-12-26 23:25:13,613][105620] Updated weights for policy 1, policy_version 1105830 (0.0010) [2023-12-26 23:25:13,666][105620] Updated weights for policy 1, policy_version 1105841 (0.0010) [2023-12-26 23:25:13,720][105620] Updated weights for policy 1, policy_version 1105851 (0.0009) [2023-12-26 23:25:13,830][105692] Updated weights for policy 0, policy_version 1104772 (0.0008) [2023-12-26 23:25:13,891][105692] Updated weights for policy 0, policy_version 1104782 (0.0005) [2023-12-26 23:25:13,953][105692] Updated weights for policy 0, policy_version 1104792 (0.0005) [2023-12-26 23:25:14,571][105620] Updated weights for policy 1, policy_version 1105862 (0.0010) [2023-12-26 23:25:14,574][105692] Updated weights for policy 0, policy_version 1104802 (0.0006) [2023-12-26 23:25:14,623][105692] Updated weights for policy 0, policy_version 1104812 (0.0006) [2023-12-26 23:25:14,624][105620] Updated weights for policy 1, policy_version 1105872 (0.0007) [2023-12-26 23:25:14,667][105692] Updated weights for policy 0, policy_version 1104822 (0.0006) [2023-12-26 23:25:14,677][105620] Updated weights for policy 1, policy_version 1105882 (0.0007) [2023-12-26 23:25:14,721][105692] Updated weights for policy 0, policy_version 1104832 (0.0007) [2023-12-26 23:25:15,429][105620] Updated weights for policy 1, policy_version 1105892 (0.0007) [2023-12-26 23:25:15,493][105620] Updated weights for policy 1, policy_version 1105902 (0.0008) [2023-12-26 23:25:15,508][105692] Updated weights for policy 0, policy_version 1104842 (0.0007) [2023-12-26 23:25:15,555][105620] Updated weights for policy 1, policy_version 1105912 (0.0006) [2023-12-26 23:25:15,568][105692] Updated weights for policy 0, policy_version 1104852 (0.0009) [2023-12-26 23:25:15,630][105692] Updated weights for policy 0, policy_version 1104862 (0.0009) [2023-12-26 23:25:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 566034432. Throughput: 0: 9719.9, 1: 9662.4. Samples: 566004140. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:25:16,063][104569] Avg episode reward: [(0, '9080.134'), (1, '9167.629')] [2023-12-26 23:25:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001104864_282886144.pth... [2023-12-26 23:25:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001105920_283148288.pth... [2023-12-26 23:25:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001104768_282853376.pth [2023-12-26 23:25:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001103744_282599424.pth [2023-12-26 23:25:16,322][105620] Updated weights for policy 1, policy_version 1105922 (0.0010) [2023-12-26 23:25:16,378][105620] Updated weights for policy 1, policy_version 1105932 (0.0008) [2023-12-26 23:25:16,393][105692] Updated weights for policy 0, policy_version 1104872 (0.0008) [2023-12-26 23:25:16,431][105620] Updated weights for policy 1, policy_version 1105942 (0.0006) [2023-12-26 23:25:16,445][105692] Updated weights for policy 0, policy_version 1104882 (0.0006) [2023-12-26 23:25:16,491][105620] Updated weights for policy 1, policy_version 1105952 (0.0007) [2023-12-26 23:25:16,503][105692] Updated weights for policy 0, policy_version 1104892 (0.0008) [2023-12-26 23:25:17,121][105692] Updated weights for policy 0, policy_version 1104902 (0.0010) [2023-12-26 23:25:17,173][105692] Updated weights for policy 0, policy_version 1104912 (0.0009) [2023-12-26 23:25:17,222][105692] Updated weights for policy 0, policy_version 1104922 (0.0009) [2023-12-26 23:25:17,288][105620] Updated weights for policy 1, policy_version 1105962 (0.0006) [2023-12-26 23:25:17,348][105620] Updated weights for policy 1, policy_version 1105972 (0.0006) [2023-12-26 23:25:17,403][105620] Updated weights for policy 1, policy_version 1105982 (0.0005) [2023-12-26 23:25:17,999][105692] Updated weights for policy 0, policy_version 1104932 (0.0007) [2023-12-26 23:25:18,057][105692] Updated weights for policy 0, policy_version 1104942 (0.0005) [2023-12-26 23:25:18,119][105692] Updated weights for policy 0, policy_version 1104952 (0.0006) [2023-12-26 23:25:18,134][105620] Updated weights for policy 1, policy_version 1105992 (0.0008) [2023-12-26 23:25:18,192][105620] Updated weights for policy 1, policy_version 1106002 (0.0008) [2023-12-26 23:25:18,246][105620] Updated weights for policy 1, policy_version 1106012 (0.0009) [2023-12-26 23:25:18,806][105692] Updated weights for policy 0, policy_version 1104962 (0.0006) [2023-12-26 23:25:18,870][105692] Updated weights for policy 0, policy_version 1104972 (0.0007) [2023-12-26 23:25:18,940][105692] Updated weights for policy 0, policy_version 1104982 (0.0006) [2023-12-26 23:25:19,001][105692] Updated weights for policy 0, policy_version 1104992 (0.0007) [2023-12-26 23:25:19,045][105620] Updated weights for policy 1, policy_version 1106022 (0.0009) [2023-12-26 23:25:19,110][105620] Updated weights for policy 1, policy_version 1106032 (0.0007) [2023-12-26 23:25:19,178][105620] Updated weights for policy 1, policy_version 1106042 (0.0005) [2023-12-26 23:25:19,747][105692] Updated weights for policy 0, policy_version 1105002 (0.0009) [2023-12-26 23:25:19,796][105692] Updated weights for policy 0, policy_version 1105012 (0.0009) [2023-12-26 23:25:19,857][105692] Updated weights for policy 0, policy_version 1105022 (0.0010) [2023-12-26 23:25:19,907][105620] Updated weights for policy 1, policy_version 1106052 (0.0008) [2023-12-26 23:25:19,971][105620] Updated weights for policy 1, policy_version 1106062 (0.0009) [2023-12-26 23:25:20,029][105620] Updated weights for policy 1, policy_version 1106072 (0.0009) [2023-12-26 23:25:20,674][105692] Updated weights for policy 0, policy_version 1105032 (0.0009) [2023-12-26 23:25:20,739][105692] Updated weights for policy 0, policy_version 1105042 (0.0009) [2023-12-26 23:25:20,796][105620] Updated weights for policy 1, policy_version 1106082 (0.0008) [2023-12-26 23:25:20,798][105692] Updated weights for policy 0, policy_version 1105052 (0.0008) [2023-12-26 23:25:20,853][105620] Updated weights for policy 1, policy_version 1106092 (0.0007) [2023-12-26 23:25:20,901][105620] Updated weights for policy 1, policy_version 1106102 (0.0009) [2023-12-26 23:25:20,951][105620] Updated weights for policy 1, policy_version 1106112 (0.0009) [2023-12-26 23:25:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 566132736. Throughput: 0: 9705.4, 1: 9672.5. Samples: 566119032. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:25:21,062][104569] Avg episode reward: [(0, '8716.709'), (1, '9076.642')] [2023-12-26 23:25:21,568][105692] Updated weights for policy 0, policy_version 1105062 (0.0008) [2023-12-26 23:25:21,628][105692] Updated weights for policy 0, policy_version 1105072 (0.0010) [2023-12-26 23:25:21,690][105692] Updated weights for policy 0, policy_version 1105082 (0.0009) [2023-12-26 23:25:21,751][105620] Updated weights for policy 1, policy_version 1106122 (0.0009) [2023-12-26 23:25:21,815][105620] Updated weights for policy 1, policy_version 1106132 (0.0009) [2023-12-26 23:25:21,866][105620] Updated weights for policy 1, policy_version 1106142 (0.0010) [2023-12-26 23:25:22,439][105692] Updated weights for policy 0, policy_version 1105092 (0.0007) [2023-12-26 23:25:22,499][105692] Updated weights for policy 0, policy_version 1105102 (0.0009) [2023-12-26 23:25:22,566][105692] Updated weights for policy 0, policy_version 1105112 (0.0009) [2023-12-26 23:25:22,666][105620] Updated weights for policy 1, policy_version 1106152 (0.0009) [2023-12-26 23:25:22,719][105620] Updated weights for policy 1, policy_version 1106162 (0.0010) [2023-12-26 23:25:22,791][105620] Updated weights for policy 1, policy_version 1106172 (0.0010) [2023-12-26 23:25:23,294][105692] Updated weights for policy 0, policy_version 1105122 (0.0009) [2023-12-26 23:25:23,345][105692] Updated weights for policy 0, policy_version 1105132 (0.0009) [2023-12-26 23:25:23,396][105692] Updated weights for policy 0, policy_version 1105142 (0.0009) [2023-12-26 23:25:23,444][105692] Updated weights for policy 0, policy_version 1105152 (0.0009) [2023-12-26 23:25:23,499][105620] Updated weights for policy 1, policy_version 1106182 (0.0008) [2023-12-26 23:25:23,566][105620] Updated weights for policy 1, policy_version 1106192 (0.0009) [2023-12-26 23:25:23,622][105620] Updated weights for policy 1, policy_version 1106202 (0.0008) [2023-12-26 23:25:24,206][105692] Updated weights for policy 0, policy_version 1105162 (0.0011) [2023-12-26 23:25:24,262][105692] Updated weights for policy 0, policy_version 1105172 (0.0010) [2023-12-26 23:25:24,324][105692] Updated weights for policy 0, policy_version 1105182 (0.0010) [2023-12-26 23:25:24,366][105620] Updated weights for policy 1, policy_version 1106212 (0.0008) [2023-12-26 23:25:24,414][105620] Updated weights for policy 1, policy_version 1106222 (0.0008) [2023-12-26 23:25:24,469][105620] Updated weights for policy 1, policy_version 1106232 (0.0008) [2023-12-26 23:25:25,001][105692] Updated weights for policy 0, policy_version 1105192 (0.0009) [2023-12-26 23:25:25,072][105692] Updated weights for policy 0, policy_version 1105202 (0.0010) [2023-12-26 23:25:25,124][105692] Updated weights for policy 0, policy_version 1105212 (0.0009) [2023-12-26 23:25:25,186][105620] Updated weights for policy 1, policy_version 1106242 (0.0008) [2023-12-26 23:25:25,246][105620] Updated weights for policy 1, policy_version 1106252 (0.0005) [2023-12-26 23:25:25,308][105620] Updated weights for policy 1, policy_version 1106262 (0.0005) [2023-12-26 23:25:25,365][105620] Updated weights for policy 1, policy_version 1106272 (0.0008) [2023-12-26 23:25:25,851][105692] Updated weights for policy 0, policy_version 1105222 (0.0009) [2023-12-26 23:25:25,909][105692] Updated weights for policy 0, policy_version 1105232 (0.0009) [2023-12-26 23:25:25,962][105692] Updated weights for policy 0, policy_version 1105242 (0.0008) [2023-12-26 23:25:26,034][105620] Updated weights for policy 1, policy_version 1106282 (0.0007) [2023-12-26 23:25:26,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 566222848. Throughput: 0: 9616.9, 1: 9716.8. Samples: 566231140. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:25:26,062][104569] Avg episode reward: [(0, '8991.980'), (1, '9167.438')] [2023-12-26 23:25:26,086][105620] Updated weights for policy 1, policy_version 1106292 (0.0005) [2023-12-26 23:25:26,132][105620] Updated weights for policy 1, policy_version 1106302 (0.0008) [2023-12-26 23:25:26,695][105692] Updated weights for policy 0, policy_version 1105252 (0.0008) [2023-12-26 23:25:26,741][105692] Updated weights for policy 0, policy_version 1105262 (0.0005) [2023-12-26 23:25:26,792][105692] Updated weights for policy 0, policy_version 1105272 (0.0006) [2023-12-26 23:25:26,875][105620] Updated weights for policy 1, policy_version 1106312 (0.0010) [2023-12-26 23:25:26,933][105620] Updated weights for policy 1, policy_version 1106323 (0.0010) [2023-12-26 23:25:26,986][105620] Updated weights for policy 1, policy_version 1106334 (0.0009) [2023-12-26 23:25:27,360][105692] Updated weights for policy 0, policy_version 1105282 (0.0006) [2023-12-26 23:25:27,411][105692] Updated weights for policy 0, policy_version 1105292 (0.0009) [2023-12-26 23:25:27,476][105692] Updated weights for policy 0, policy_version 1105302 (0.0009) [2023-12-26 23:25:27,542][105692] Updated weights for policy 0, policy_version 1105312 (0.0009) [2023-12-26 23:25:27,695][105620] Updated weights for policy 1, policy_version 1106344 (0.0008) [2023-12-26 23:25:27,747][105620] Updated weights for policy 1, policy_version 1106354 (0.0005) [2023-12-26 23:25:27,806][105620] Updated weights for policy 1, policy_version 1106364 (0.0006) [2023-12-26 23:25:28,192][105692] Updated weights for policy 0, policy_version 1105322 (0.0006) [2023-12-26 23:25:28,246][105692] Updated weights for policy 0, policy_version 1105332 (0.0008) [2023-12-26 23:25:28,300][105692] Updated weights for policy 0, policy_version 1105342 (0.0010) [2023-12-26 23:25:28,363][105620] Updated weights for policy 1, policy_version 1106374 (0.0007) [2023-12-26 23:25:28,411][105620] Updated weights for policy 1, policy_version 1106384 (0.0006) [2023-12-26 23:25:28,488][105620] Updated weights for policy 1, policy_version 1106394 (0.0005) [2023-12-26 23:25:29,008][105692] Updated weights for policy 0, policy_version 1105352 (0.0010) [2023-12-26 23:25:29,059][105692] Updated weights for policy 0, policy_version 1105362 (0.0010) [2023-12-26 23:25:29,092][105620] Updated weights for policy 1, policy_version 1106404 (0.0007) [2023-12-26 23:25:29,107][105692] Updated weights for policy 0, policy_version 1105372 (0.0010) [2023-12-26 23:25:29,149][105620] Updated weights for policy 1, policy_version 1106414 (0.0009) [2023-12-26 23:25:29,204][105620] Updated weights for policy 1, policy_version 1106424 (0.0008) [2023-12-26 23:25:29,918][105620] Updated weights for policy 1, policy_version 1106434 (0.0008) [2023-12-26 23:25:29,949][105692] Updated weights for policy 0, policy_version 1105382 (0.0009) [2023-12-26 23:25:29,980][105620] Updated weights for policy 1, policy_version 1106444 (0.0008) [2023-12-26 23:25:30,010][105692] Updated weights for policy 0, policy_version 1105392 (0.0008) [2023-12-26 23:25:30,037][105620] Updated weights for policy 1, policy_version 1106454 (0.0010) [2023-12-26 23:25:30,075][105692] Updated weights for policy 0, policy_version 1105402 (0.0007) [2023-12-26 23:25:30,101][105620] Updated weights for policy 1, policy_version 1106464 (0.0008) [2023-12-26 23:25:30,671][105692] Updated weights for policy 0, policy_version 1105412 (0.0005) [2023-12-26 23:25:30,740][105692] Updated weights for policy 0, policy_version 1105422 (0.0007) [2023-12-26 23:25:30,799][105692] Updated weights for policy 0, policy_version 1105432 (0.0010) [2023-12-26 23:25:30,902][105620] Updated weights for policy 1, policy_version 1106474 (0.0008) [2023-12-26 23:25:30,953][105620] Updated weights for policy 1, policy_version 1106484 (0.0008) [2023-12-26 23:25:31,000][105620] Updated weights for policy 1, policy_version 1106494 (0.0009) [2023-12-26 23:25:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 566329344. Throughput: 0: 9665.4, 1: 9770.4. Samples: 566294320. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:25:31,062][104569] Avg episode reward: [(0, '9171.801'), (1, '9080.079')] [2023-12-26 23:25:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001105440_283033600.pth... [2023-12-26 23:25:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001106496_283295744.pth... [2023-12-26 23:25:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001105376_283009024.pth [2023-12-26 23:25:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001104288_282738688.pth [2023-12-26 23:25:31,546][105692] Updated weights for policy 0, policy_version 1105442 (0.0010) [2023-12-26 23:25:31,614][105692] Updated weights for policy 0, policy_version 1105452 (0.0009) [2023-12-26 23:25:31,675][105692] Updated weights for policy 0, policy_version 1105462 (0.0008) [2023-12-26 23:25:31,694][105620] Updated weights for policy 1, policy_version 1106504 (0.0008) [2023-12-26 23:25:31,740][105692] Updated weights for policy 0, policy_version 1105472 (0.0007) [2023-12-26 23:25:31,753][105620] Updated weights for policy 1, policy_version 1106514 (0.0008) [2023-12-26 23:25:31,816][105620] Updated weights for policy 1, policy_version 1106524 (0.0009) [2023-12-26 23:25:32,460][105692] Updated weights for policy 0, policy_version 1105482 (0.0005) [2023-12-26 23:25:32,515][105692] Updated weights for policy 0, policy_version 1105492 (0.0006) [2023-12-26 23:25:32,571][105692] Updated weights for policy 0, policy_version 1105502 (0.0005) [2023-12-26 23:25:32,587][105620] Updated weights for policy 1, policy_version 1106534 (0.0009) [2023-12-26 23:25:32,653][105620] Updated weights for policy 1, policy_version 1106544 (0.0010) [2023-12-26 23:25:32,716][105620] Updated weights for policy 1, policy_version 1106554 (0.0010) [2023-12-26 23:25:33,184][105692] Updated weights for policy 0, policy_version 1105512 (0.0006) [2023-12-26 23:25:33,227][105692] Updated weights for policy 0, policy_version 1105522 (0.0005) [2023-12-26 23:25:33,274][105692] Updated weights for policy 0, policy_version 1105532 (0.0008) [2023-12-26 23:25:33,507][105620] Updated weights for policy 1, policy_version 1106564 (0.0008) [2023-12-26 23:25:33,552][105620] Updated weights for policy 1, policy_version 1106574 (0.0005) [2023-12-26 23:25:33,604][105620] Updated weights for policy 1, policy_version 1106584 (0.0005) [2023-12-26 23:25:34,084][105692] Updated weights for policy 0, policy_version 1105542 (0.0009) [2023-12-26 23:25:34,147][105692] Updated weights for policy 0, policy_version 1105552 (0.0009) [2023-12-26 23:25:34,206][105620] Updated weights for policy 1, policy_version 1106594 (0.0006) [2023-12-26 23:25:34,209][105692] Updated weights for policy 0, policy_version 1105562 (0.0008) [2023-12-26 23:25:34,276][105620] Updated weights for policy 1, policy_version 1106604 (0.0009) [2023-12-26 23:25:34,329][105620] Updated weights for policy 1, policy_version 1106614 (0.0010) [2023-12-26 23:25:34,383][105620] Updated weights for policy 1, policy_version 1106624 (0.0008) [2023-12-26 23:25:34,914][105692] Updated weights for policy 0, policy_version 1105572 (0.0006) [2023-12-26 23:25:34,981][105692] Updated weights for policy 0, policy_version 1105582 (0.0005) [2023-12-26 23:25:35,042][105692] Updated weights for policy 0, policy_version 1105592 (0.0006) [2023-12-26 23:25:35,151][105620] Updated weights for policy 1, policy_version 1106634 (0.0009) [2023-12-26 23:25:35,215][105620] Updated weights for policy 1, policy_version 1106644 (0.0006) [2023-12-26 23:25:35,279][105620] Updated weights for policy 1, policy_version 1106654 (0.0005) [2023-12-26 23:25:35,618][105692] Updated weights for policy 0, policy_version 1105602 (0.0005) [2023-12-26 23:25:35,687][105692] Updated weights for policy 0, policy_version 1105612 (0.0005) [2023-12-26 23:25:35,735][105692] Updated weights for policy 0, policy_version 1105622 (0.0005) [2023-12-26 23:25:35,784][105692] Updated weights for policy 0, policy_version 1105632 (0.0009) [2023-12-26 23:25:35,911][105620] Updated weights for policy 1, policy_version 1106664 (0.0008) [2023-12-26 23:25:35,962][105620] Updated weights for policy 1, policy_version 1106674 (0.0010) [2023-12-26 23:25:36,022][105620] Updated weights for policy 1, policy_version 1106684 (0.0009) [2023-12-26 23:25:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 566427648. Throughput: 0: 9554.4, 1: 9722.9. Samples: 566409256. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:25:36,062][104569] Avg episode reward: [(0, '8989.180'), (1, '8904.465')] [2023-12-26 23:25:36,450][105692] Updated weights for policy 0, policy_version 1105642 (0.0008) [2023-12-26 23:25:36,511][105692] Updated weights for policy 0, policy_version 1105652 (0.0008) [2023-12-26 23:25:36,572][105692] Updated weights for policy 0, policy_version 1105662 (0.0008) [2023-12-26 23:25:36,757][105620] Updated weights for policy 1, policy_version 1106694 (0.0010) [2023-12-26 23:25:36,810][105620] Updated weights for policy 1, policy_version 1106704 (0.0011) [2023-12-26 23:25:36,866][105620] Updated weights for policy 1, policy_version 1106714 (0.0011) [2023-12-26 23:25:37,308][105692] Updated weights for policy 0, policy_version 1105672 (0.0008) [2023-12-26 23:25:37,368][105692] Updated weights for policy 0, policy_version 1105682 (0.0006) [2023-12-26 23:25:37,415][105692] Updated weights for policy 0, policy_version 1105692 (0.0007) [2023-12-26 23:25:37,598][105620] Updated weights for policy 1, policy_version 1106724 (0.0011) [2023-12-26 23:25:37,649][105620] Updated weights for policy 1, policy_version 1106734 (0.0010) [2023-12-26 23:25:37,713][105620] Updated weights for policy 1, policy_version 1106744 (0.0011) [2023-12-26 23:25:38,076][105692] Updated weights for policy 0, policy_version 1105702 (0.0006) [2023-12-26 23:25:38,142][105692] Updated weights for policy 0, policy_version 1105712 (0.0006) [2023-12-26 23:25:38,206][105692] Updated weights for policy 0, policy_version 1105722 (0.0007) [2023-12-26 23:25:38,459][105620] Updated weights for policy 1, policy_version 1106754 (0.0011) [2023-12-26 23:25:38,511][105620] Updated weights for policy 1, policy_version 1106764 (0.0007) [2023-12-26 23:25:38,577][105620] Updated weights for policy 1, policy_version 1106774 (0.0009) [2023-12-26 23:25:38,644][105620] Updated weights for policy 1, policy_version 1106784 (0.0010) [2023-12-26 23:25:38,822][105692] Updated weights for policy 0, policy_version 1105732 (0.0010) [2023-12-26 23:25:38,883][105692] Updated weights for policy 0, policy_version 1105742 (0.0008) [2023-12-26 23:25:38,937][105692] Updated weights for policy 0, policy_version 1105752 (0.0008) [2023-12-26 23:25:39,276][105620] Updated weights for policy 1, policy_version 1106794 (0.0010) [2023-12-26 23:25:39,335][105620] Updated weights for policy 1, policy_version 1106804 (0.0010) [2023-12-26 23:25:39,408][105620] Updated weights for policy 1, policy_version 1106814 (0.0010) [2023-12-26 23:25:39,807][105692] Updated weights for policy 0, policy_version 1105762 (0.0008) [2023-12-26 23:25:39,881][105692] Updated weights for policy 0, policy_version 1105772 (0.0010) [2023-12-26 23:25:39,945][105692] Updated weights for policy 0, policy_version 1105782 (0.0010) [2023-12-26 23:25:39,999][105692] Updated weights for policy 0, policy_version 1105792 (0.0008) [2023-12-26 23:25:40,000][105620] Updated weights for policy 1, policy_version 1106824 (0.0007) [2023-12-26 23:25:40,057][105620] Updated weights for policy 1, policy_version 1106834 (0.0009) [2023-12-26 23:25:40,112][105620] Updated weights for policy 1, policy_version 1106844 (0.0009) [2023-12-26 23:25:40,788][105692] Updated weights for policy 0, policy_version 1105802 (0.0009) [2023-12-26 23:25:40,850][105692] Updated weights for policy 0, policy_version 1105812 (0.0009) [2023-12-26 23:25:40,898][105620] Updated weights for policy 1, policy_version 1106854 (0.0008) [2023-12-26 23:25:40,909][105692] Updated weights for policy 0, policy_version 1105822 (0.0007) [2023-12-26 23:25:40,953][105620] Updated weights for policy 1, policy_version 1106864 (0.0007) [2023-12-26 23:25:41,012][105620] Updated weights for policy 1, policy_version 1106874 (0.0007) [2023-12-26 23:25:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 566525952. Throughput: 0: 9654.6, 1: 9702.4. Samples: 566527816. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:25:41,063][104569] Avg episode reward: [(0, '8991.223'), (1, '9083.912')] [2023-12-26 23:25:41,695][105620] Updated weights for policy 1, policy_version 1106884 (0.0008) [2023-12-26 23:25:41,765][105620] Updated weights for policy 1, policy_version 1106894 (0.0010) [2023-12-26 23:25:41,774][105692] Updated weights for policy 0, policy_version 1105832 (0.0007) [2023-12-26 23:25:41,822][105620] Updated weights for policy 1, policy_version 1106904 (0.0008) [2023-12-26 23:25:41,842][105692] Updated weights for policy 0, policy_version 1105842 (0.0008) [2023-12-26 23:25:41,904][105692] Updated weights for policy 0, policy_version 1105852 (0.0009) [2023-12-26 23:25:42,538][105620] Updated weights for policy 1, policy_version 1106914 (0.0007) [2023-12-26 23:25:42,597][105620] Updated weights for policy 1, policy_version 1106924 (0.0009) [2023-12-26 23:25:42,606][105692] Updated weights for policy 0, policy_version 1105862 (0.0007) [2023-12-26 23:25:42,658][105692] Updated weights for policy 0, policy_version 1105872 (0.0006) [2023-12-26 23:25:42,663][105620] Updated weights for policy 1, policy_version 1106934 (0.0009) [2023-12-26 23:25:42,705][105692] Updated weights for policy 0, policy_version 1105882 (0.0008) [2023-12-26 23:25:42,717][105620] Updated weights for policy 1, policy_version 1106944 (0.0007) [2023-12-26 23:25:43,445][105692] Updated weights for policy 0, policy_version 1105892 (0.0008) [2023-12-26 23:25:43,460][105620] Updated weights for policy 1, policy_version 1106954 (0.0006) [2023-12-26 23:25:43,499][105692] Updated weights for policy 0, policy_version 1105902 (0.0010) [2023-12-26 23:25:43,514][105620] Updated weights for policy 1, policy_version 1106964 (0.0008) [2023-12-26 23:25:43,555][105692] Updated weights for policy 0, policy_version 1105912 (0.0008) [2023-12-26 23:25:43,575][105620] Updated weights for policy 1, policy_version 1106974 (0.0008) [2023-12-26 23:25:44,292][105692] Updated weights for policy 0, policy_version 1105922 (0.0009) [2023-12-26 23:25:44,310][105620] Updated weights for policy 1, policy_version 1106984 (0.0010) [2023-12-26 23:25:44,344][105692] Updated weights for policy 0, policy_version 1105932 (0.0008) [2023-12-26 23:25:44,368][105620] Updated weights for policy 1, policy_version 1106994 (0.0010) [2023-12-26 23:25:44,404][105692] Updated weights for policy 0, policy_version 1105942 (0.0008) [2023-12-26 23:25:44,428][105620] Updated weights for policy 1, policy_version 1107004 (0.0010) [2023-12-26 23:25:44,463][105692] Updated weights for policy 0, policy_version 1105952 (0.0006) [2023-12-26 23:25:45,168][105620] Updated weights for policy 1, policy_version 1107014 (0.0011) [2023-12-26 23:25:45,207][105692] Updated weights for policy 0, policy_version 1105962 (0.0006) [2023-12-26 23:25:45,228][105620] Updated weights for policy 1, policy_version 1107024 (0.0011) [2023-12-26 23:25:45,264][105692] Updated weights for policy 0, policy_version 1105972 (0.0006) [2023-12-26 23:25:45,296][105620] Updated weights for policy 1, policy_version 1107034 (0.0011) [2023-12-26 23:25:45,331][105692] Updated weights for policy 0, policy_version 1105982 (0.0007) [2023-12-26 23:25:45,991][105620] Updated weights for policy 1, policy_version 1107044 (0.0008) [2023-12-26 23:25:46,045][105620] Updated weights for policy 1, policy_version 1107054 (0.0005) [2023-12-26 23:25:46,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 566607872. Throughput: 0: 9640.4, 1: 9685.6. Samples: 566584016. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:25:46,062][104569] Avg episode reward: [(0, '8988.759'), (1, '9078.127')] [2023-12-26 23:25:46,097][105620] Updated weights for policy 1, policy_version 1107064 (0.0007) [2023-12-26 23:25:46,104][105692] Updated weights for policy 0, policy_version 1105992 (0.0007) [2023-12-26 23:25:46,144][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001107072_283443200.pth... [2023-12-26 23:25:46,147][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001105920_283148288.pth [2023-12-26 23:25:46,160][105692] Updated weights for policy 0, policy_version 1106002 (0.0009) [2023-12-26 23:25:46,215][105692] Updated weights for policy 0, policy_version 1106012 (0.0009) [2023-12-26 23:25:46,234][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001106016_283181056.pth... [2023-12-26 23:25:46,237][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001104864_282886144.pth [2023-12-26 23:25:46,763][105620] Updated weights for policy 1, policy_version 1107074 (0.0009) [2023-12-26 23:25:46,817][105620] Updated weights for policy 1, policy_version 1107084 (0.0010) [2023-12-26 23:25:46,865][105620] Updated weights for policy 1, policy_version 1107094 (0.0010) [2023-12-26 23:25:46,918][105620] Updated weights for policy 1, policy_version 1107104 (0.0010) [2023-12-26 23:25:46,951][105692] Updated weights for policy 0, policy_version 1106022 (0.0009) [2023-12-26 23:25:47,007][105692] Updated weights for policy 0, policy_version 1106032 (0.0008) [2023-12-26 23:25:47,055][105692] Updated weights for policy 0, policy_version 1106042 (0.0008) [2023-12-26 23:25:47,647][105620] Updated weights for policy 1, policy_version 1107114 (0.0010) [2023-12-26 23:25:47,708][105620] Updated weights for policy 1, policy_version 1107124 (0.0010) [2023-12-26 23:25:47,771][105620] Updated weights for policy 1, policy_version 1107134 (0.0011) [2023-12-26 23:25:47,844][105692] Updated weights for policy 0, policy_version 1106052 (0.0007) [2023-12-26 23:25:47,895][105692] Updated weights for policy 0, policy_version 1106062 (0.0005) [2023-12-26 23:25:47,964][105692] Updated weights for policy 0, policy_version 1106072 (0.0005) [2023-12-26 23:25:48,455][105620] Updated weights for policy 1, policy_version 1107144 (0.0010) [2023-12-26 23:25:48,518][105620] Updated weights for policy 1, policy_version 1107154 (0.0011) [2023-12-26 23:25:48,581][105620] Updated weights for policy 1, policy_version 1107164 (0.0010) [2023-12-26 23:25:48,716][105692] Updated weights for policy 0, policy_version 1106082 (0.0007) [2023-12-26 23:25:48,777][105692] Updated weights for policy 0, policy_version 1106092 (0.0008) [2023-12-26 23:25:48,840][105692] Updated weights for policy 0, policy_version 1106102 (0.0009) [2023-12-26 23:25:48,908][105692] Updated weights for policy 0, policy_version 1106112 (0.0008) [2023-12-26 23:25:49,333][105620] Updated weights for policy 1, policy_version 1107174 (0.0011) [2023-12-26 23:25:49,401][105620] Updated weights for policy 1, policy_version 1107184 (0.0011) [2023-12-26 23:25:49,463][105620] Updated weights for policy 1, policy_version 1107194 (0.0010) [2023-12-26 23:25:49,684][105692] Updated weights for policy 0, policy_version 1106122 (0.0009) [2023-12-26 23:25:49,737][105692] Updated weights for policy 0, policy_version 1106132 (0.0008) [2023-12-26 23:25:49,789][105692] Updated weights for policy 0, policy_version 1106142 (0.0007) [2023-12-26 23:25:50,255][105620] Updated weights for policy 1, policy_version 1107204 (0.0010) [2023-12-26 23:25:50,307][105620] Updated weights for policy 1, policy_version 1107214 (0.0010) [2023-12-26 23:25:50,362][105620] Updated weights for policy 1, policy_version 1107224 (0.0010) [2023-12-26 23:25:50,575][105692] Updated weights for policy 0, policy_version 1106152 (0.0008) [2023-12-26 23:25:50,635][105692] Updated weights for policy 0, policy_version 1106162 (0.0008) [2023-12-26 23:25:50,696][105692] Updated weights for policy 0, policy_version 1106172 (0.0008) [2023-12-26 23:25:51,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 566706176. Throughput: 0: 9704.9, 1: 9576.4. Samples: 566697928. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:25:51,062][104569] Avg episode reward: [(0, '9076.816'), (1, '9167.692')] [2023-12-26 23:25:51,125][105620] Updated weights for policy 1, policy_version 1107234 (0.0011) [2023-12-26 23:25:51,188][105620] Updated weights for policy 1, policy_version 1107244 (0.0010) [2023-12-26 23:25:51,250][105620] Updated weights for policy 1, policy_version 1107254 (0.0010) [2023-12-26 23:25:51,309][105620] Updated weights for policy 1, policy_version 1107264 (0.0008) [2023-12-26 23:25:51,462][105692] Updated weights for policy 0, policy_version 1106182 (0.0007) [2023-12-26 23:25:51,515][105692] Updated weights for policy 0, policy_version 1106192 (0.0006) [2023-12-26 23:25:51,575][105692] Updated weights for policy 0, policy_version 1106202 (0.0008) [2023-12-26 23:25:52,075][105620] Updated weights for policy 1, policy_version 1107274 (0.0010) [2023-12-26 23:25:52,137][105620] Updated weights for policy 1, policy_version 1107284 (0.0010) [2023-12-26 23:25:52,202][105620] Updated weights for policy 1, policy_version 1107294 (0.0010) [2023-12-26 23:25:52,322][105692] Updated weights for policy 0, policy_version 1106212 (0.0007) [2023-12-26 23:25:52,389][105692] Updated weights for policy 0, policy_version 1106222 (0.0008) [2023-12-26 23:25:52,452][105692] Updated weights for policy 0, policy_version 1106232 (0.0007) [2023-12-26 23:25:52,912][105620] Updated weights for policy 1, policy_version 1107304 (0.0009) [2023-12-26 23:25:52,969][105620] Updated weights for policy 1, policy_version 1107314 (0.0009) [2023-12-26 23:25:53,032][105620] Updated weights for policy 1, policy_version 1107324 (0.0007) [2023-12-26 23:25:53,221][105692] Updated weights for policy 0, policy_version 1106242 (0.0006) [2023-12-26 23:25:53,276][105692] Updated weights for policy 0, policy_version 1106253 (0.0010) [2023-12-26 23:25:53,330][105692] Updated weights for policy 0, policy_version 1106263 (0.0010) [2023-12-26 23:25:53,579][105620] Updated weights for policy 1, policy_version 1107334 (0.0005) [2023-12-26 23:25:53,628][105620] Updated weights for policy 1, policy_version 1107344 (0.0006) [2023-12-26 23:25:53,673][105620] Updated weights for policy 1, policy_version 1107354 (0.0006) [2023-12-26 23:25:54,222][105692] Updated weights for policy 0, policy_version 1106273 (0.0010) [2023-12-26 23:25:54,245][105620] Updated weights for policy 1, policy_version 1107364 (0.0006) [2023-12-26 23:25:54,280][105692] Updated weights for policy 0, policy_version 1106283 (0.0006) [2023-12-26 23:25:54,299][105620] Updated weights for policy 1, policy_version 1107374 (0.0006) [2023-12-26 23:25:54,330][105692] Updated weights for policy 0, policy_version 1106293 (0.0006) [2023-12-26 23:25:54,343][105585] KL-divergence is very high: 483.6601 [2023-12-26 23:25:54,349][105620] Updated weights for policy 1, policy_version 1107384 (0.0007) [2023-12-26 23:25:54,380][105692] Updated weights for policy 0, policy_version 1106303 (0.0007) [2023-12-26 23:25:55,053][105692] Updated weights for policy 0, policy_version 1106313 (0.0008) [2023-12-26 23:25:55,054][105620] Updated weights for policy 1, policy_version 1107394 (0.0006) [2023-12-26 23:25:55,099][105692] Updated weights for policy 0, policy_version 1106323 (0.0008) [2023-12-26 23:25:55,120][105620] Updated weights for policy 1, policy_version 1107404 (0.0005) [2023-12-26 23:25:55,153][105692] Updated weights for policy 0, policy_version 1106334 (0.0008) [2023-12-26 23:25:55,182][105620] Updated weights for policy 1, policy_version 1107414 (0.0007) [2023-12-26 23:25:55,251][105620] Updated weights for policy 1, policy_version 1107424 (0.0008) [2023-12-26 23:25:55,821][105620] Updated weights for policy 1, policy_version 1107434 (0.0010) [2023-12-26 23:25:55,839][105692] Updated weights for policy 0, policy_version 1106344 (0.0006) [2023-12-26 23:25:55,875][105620] Updated weights for policy 1, policy_version 1107444 (0.0006) [2023-12-26 23:25:55,896][105692] Updated weights for policy 0, policy_version 1106354 (0.0010) [2023-12-26 23:25:55,927][105620] Updated weights for policy 1, policy_version 1107454 (0.0005) [2023-12-26 23:25:55,945][105692] Updated weights for policy 0, policy_version 1106364 (0.0009) [2023-12-26 23:25:56,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 566812672. Throughput: 0: 9582.2, 1: 9675.0. Samples: 566815432. Policy #0 lag: (min: 30.0, avg: 39.4, max: 62.0) [2023-12-26 23:25:56,063][104569] Avg episode reward: [(0, '8985.421'), (1, '9259.574')] [2023-12-26 23:25:56,506][105620] Updated weights for policy 1, policy_version 1107464 (0.0006) [2023-12-26 23:25:56,555][105620] Updated weights for policy 1, policy_version 1107474 (0.0005) [2023-12-26 23:25:56,600][105620] Updated weights for policy 1, policy_version 1107484 (0.0005) [2023-12-26 23:25:56,811][105692] Updated weights for policy 0, policy_version 1106374 (0.0009) [2023-12-26 23:25:56,863][105692] Updated weights for policy 0, policy_version 1106384 (0.0009) [2023-12-26 23:25:56,918][105692] Updated weights for policy 0, policy_version 1106394 (0.0009) [2023-12-26 23:25:57,182][105620] Updated weights for policy 1, policy_version 1107494 (0.0006) [2023-12-26 23:25:57,225][105620] Updated weights for policy 1, policy_version 1107504 (0.0005) [2023-12-26 23:25:57,270][105620] Updated weights for policy 1, policy_version 1107514 (0.0005) [2023-12-26 23:25:57,805][105692] Updated weights for policy 0, policy_version 1106404 (0.0009) [2023-12-26 23:25:57,863][105620] Updated weights for policy 1, policy_version 1107524 (0.0006) [2023-12-26 23:25:57,865][105692] Updated weights for policy 0, policy_version 1106414 (0.0009) [2023-12-26 23:25:57,918][105692] Updated weights for policy 0, policy_version 1106424 (0.0006) [2023-12-26 23:25:57,923][105620] Updated weights for policy 1, policy_version 1107534 (0.0007) [2023-12-26 23:25:57,980][105620] Updated weights for policy 1, policy_version 1107544 (0.0007) [2023-12-26 23:25:58,698][105620] Updated weights for policy 1, policy_version 1107554 (0.0008) [2023-12-26 23:25:58,766][105620] Updated weights for policy 1, policy_version 1107564 (0.0008) [2023-12-26 23:25:58,792][105692] Updated weights for policy 0, policy_version 1106434 (0.0007) [2023-12-26 23:25:58,832][105620] Updated weights for policy 1, policy_version 1107574 (0.0008) [2023-12-26 23:25:58,855][105692] Updated weights for policy 0, policy_version 1106444 (0.0007) [2023-12-26 23:25:58,898][105620] Updated weights for policy 1, policy_version 1107584 (0.0008) [2023-12-26 23:25:58,925][105692] Updated weights for policy 0, policy_version 1106454 (0.0008) [2023-12-26 23:25:58,977][105692] Updated weights for policy 0, policy_version 1106464 (0.0008) [2023-12-26 23:25:59,710][105620] Updated weights for policy 1, policy_version 1107594 (0.0006) [2023-12-26 23:25:59,730][105692] Updated weights for policy 0, policy_version 1106474 (0.0008) [2023-12-26 23:25:59,774][105620] Updated weights for policy 1, policy_version 1107604 (0.0005) [2023-12-26 23:25:59,790][105692] Updated weights for policy 0, policy_version 1106484 (0.0008) [2023-12-26 23:25:59,838][105620] Updated weights for policy 1, policy_version 1107614 (0.0006) [2023-12-26 23:25:59,847][105692] Updated weights for policy 0, policy_version 1106494 (0.0008) [2023-12-26 23:26:00,481][105620] Updated weights for policy 1, policy_version 1107624 (0.0009) [2023-12-26 23:26:00,538][105620] Updated weights for policy 1, policy_version 1107634 (0.0009) [2023-12-26 23:26:00,593][105692] Updated weights for policy 0, policy_version 1106504 (0.0007) [2023-12-26 23:26:00,594][105620] Updated weights for policy 1, policy_version 1107644 (0.0007) [2023-12-26 23:26:00,640][105692] Updated weights for policy 0, policy_version 1106514 (0.0008) [2023-12-26 23:26:00,698][105692] Updated weights for policy 0, policy_version 1106524 (0.0009) [2023-12-26 23:26:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 566902784. Throughput: 0: 9493.7, 1: 9816.5. Samples: 566873096. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:26:01,062][104569] Avg episode reward: [(0, '9166.509'), (1, '9259.968')] [2023-12-26 23:26:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001106528_283312128.pth... [2023-12-26 23:26:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001107648_283590656.pth... [2023-12-26 23:26:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001105440_283033600.pth [2023-12-26 23:26:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001106496_283295744.pth [2023-12-26 23:26:01,326][105620] Updated weights for policy 1, policy_version 1107654 (0.0007) [2023-12-26 23:26:01,392][105620] Updated weights for policy 1, policy_version 1107664 (0.0007) [2023-12-26 23:26:01,455][105620] Updated weights for policy 1, policy_version 1107674 (0.0008) [2023-12-26 23:26:01,510][105692] Updated weights for policy 0, policy_version 1106534 (0.0008) [2023-12-26 23:26:01,565][105692] Updated weights for policy 0, policy_version 1106544 (0.0010) [2023-12-26 23:26:01,623][105692] Updated weights for policy 0, policy_version 1106555 (0.0010) [2023-12-26 23:26:02,103][105620] Updated weights for policy 1, policy_version 1107684 (0.0008) [2023-12-26 23:26:02,167][105620] Updated weights for policy 1, policy_version 1107694 (0.0006) [2023-12-26 23:26:02,217][105620] Updated weights for policy 1, policy_version 1107704 (0.0006) [2023-12-26 23:26:02,517][105692] Updated weights for policy 0, policy_version 1106565 (0.0009) [2023-12-26 23:26:02,569][105692] Updated weights for policy 0, policy_version 1106575 (0.0009) [2023-12-26 23:26:02,622][105692] Updated weights for policy 0, policy_version 1106585 (0.0009) [2023-12-26 23:26:02,857][105620] Updated weights for policy 1, policy_version 1107714 (0.0008) [2023-12-26 23:26:02,919][105620] Updated weights for policy 1, policy_version 1107724 (0.0009) [2023-12-26 23:26:02,981][105620] Updated weights for policy 1, policy_version 1107734 (0.0009) [2023-12-26 23:26:03,041][105620] Updated weights for policy 1, policy_version 1107744 (0.0009) [2023-12-26 23:26:03,361][105692] Updated weights for policy 0, policy_version 1106595 (0.0009) [2023-12-26 23:26:03,408][105692] Updated weights for policy 0, policy_version 1106605 (0.0009) [2023-12-26 23:26:03,456][105692] Updated weights for policy 0, policy_version 1106615 (0.0009) [2023-12-26 23:26:03,787][105620] Updated weights for policy 1, policy_version 1107754 (0.0009) [2023-12-26 23:26:03,842][105620] Updated weights for policy 1, policy_version 1107764 (0.0009) [2023-12-26 23:26:03,907][105620] Updated weights for policy 1, policy_version 1107774 (0.0009) [2023-12-26 23:26:04,265][105692] Updated weights for policy 0, policy_version 1106625 (0.0009) [2023-12-26 23:26:04,325][105692] Updated weights for policy 0, policy_version 1106635 (0.0009) [2023-12-26 23:26:04,384][105692] Updated weights for policy 0, policy_version 1106645 (0.0009) [2023-12-26 23:26:04,438][105692] Updated weights for policy 0, policy_version 1106655 (0.0010) [2023-12-26 23:26:04,636][105620] Updated weights for policy 1, policy_version 1107784 (0.0009) [2023-12-26 23:26:04,691][105620] Updated weights for policy 1, policy_version 1107794 (0.0006) [2023-12-26 23:26:04,731][105586] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000001 [2023-12-26 23:26:05,137][105692] Updated weights for policy 0, policy_version 1106665 (0.0006) [2023-12-26 23:26:05,187][105692] Updated weights for policy 0, policy_version 1106675 (0.0005) [2023-12-26 23:26:05,241][105692] Updated weights for policy 0, policy_version 1106685 (0.0005) [2023-12-26 23:26:05,509][105620] Updated weights for policy 1, policy_version 1107804 (0.0007) [2023-12-26 23:26:05,577][105620] Updated weights for policy 1, policy_version 1107814 (0.0008) [2023-12-26 23:26:05,626][105620] Updated weights for policy 1, policy_version 1107824 (0.0010) [2023-12-26 23:26:05,780][105692] Updated weights for policy 0, policy_version 1106695 (0.0006) [2023-12-26 23:26:05,848][105692] Updated weights for policy 0, policy_version 1106705 (0.0006) [2023-12-26 23:26:05,913][105692] Updated weights for policy 0, policy_version 1106715 (0.0010) [2023-12-26 23:26:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.3, 300 sec: 19494.2). Total num frames: 567001088. Throughput: 0: 9358.3, 1: 9905.7. Samples: 566985912. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:26:06,062][104569] Avg episode reward: [(0, '9078.552'), (1, '9168.803')] [2023-12-26 23:26:06,273][105620] Updated weights for policy 1, policy_version 1107834 (0.0009) [2023-12-26 23:26:06,331][105620] Updated weights for policy 1, policy_version 1107844 (0.0005) [2023-12-26 23:26:06,391][105620] Updated weights for policy 1, policy_version 1107854 (0.0005) [2023-12-26 23:26:06,448][105620] Updated weights for policy 1, policy_version 1107864 (0.0006) [2023-12-26 23:26:06,550][105692] Updated weights for policy 0, policy_version 1106725 (0.0008) [2023-12-26 23:26:06,614][105692] Updated weights for policy 0, policy_version 1106735 (0.0008) [2023-12-26 23:26:06,671][105692] Updated weights for policy 0, policy_version 1106745 (0.0006) [2023-12-26 23:26:07,047][105620] Updated weights for policy 1, policy_version 1107874 (0.0005) [2023-12-26 23:26:07,112][105620] Updated weights for policy 1, policy_version 1107884 (0.0008) [2023-12-26 23:26:07,170][105620] Updated weights for policy 1, policy_version 1107894 (0.0008) [2023-12-26 23:26:07,339][105692] Updated weights for policy 0, policy_version 1106755 (0.0007) [2023-12-26 23:26:07,392][105692] Updated weights for policy 0, policy_version 1106765 (0.0006) [2023-12-26 23:26:07,444][105692] Updated weights for policy 0, policy_version 1106775 (0.0008) [2023-12-26 23:26:07,757][105620] Updated weights for policy 1, policy_version 1107904 (0.0005) [2023-12-26 23:26:07,807][105620] Updated weights for policy 1, policy_version 1107914 (0.0005) [2023-12-26 23:26:07,854][105620] Updated weights for policy 1, policy_version 1107924 (0.0005) [2023-12-26 23:26:08,043][105692] Updated weights for policy 0, policy_version 1106785 (0.0010) [2023-12-26 23:26:08,098][105692] Updated weights for policy 0, policy_version 1106795 (0.0011) [2023-12-26 23:26:08,146][105692] Updated weights for policy 0, policy_version 1106805 (0.0010) [2023-12-26 23:26:08,199][105692] Updated weights for policy 0, policy_version 1106815 (0.0011) [2023-12-26 23:26:08,400][105620] Updated weights for policy 1, policy_version 1107934 (0.0005) [2023-12-26 23:26:08,458][105620] Updated weights for policy 1, policy_version 1107944 (0.0009) [2023-12-26 23:26:08,513][105620] Updated weights for policy 1, policy_version 1107954 (0.0011) [2023-12-26 23:26:08,866][105692] Updated weights for policy 0, policy_version 1106825 (0.0011) [2023-12-26 23:26:08,930][105692] Updated weights for policy 0, policy_version 1106835 (0.0011) [2023-12-26 23:26:08,989][105692] Updated weights for policy 0, policy_version 1106845 (0.0011) [2023-12-26 23:26:09,248][105620] Updated weights for policy 1, policy_version 1107964 (0.0010) [2023-12-26 23:26:09,314][105620] Updated weights for policy 1, policy_version 1107974 (0.0010) [2023-12-26 23:26:09,382][105620] Updated weights for policy 1, policy_version 1107984 (0.0010) [2023-12-26 23:26:09,699][105692] Updated weights for policy 0, policy_version 1106855 (0.0009) [2023-12-26 23:26:09,751][105692] Updated weights for policy 0, policy_version 1106865 (0.0008) [2023-12-26 23:26:09,809][105692] Updated weights for policy 0, policy_version 1106875 (0.0006) [2023-12-26 23:26:10,144][105620] Updated weights for policy 1, policy_version 1107994 (0.0011) [2023-12-26 23:26:10,205][105620] Updated weights for policy 1, policy_version 1108004 (0.0011) [2023-12-26 23:26:10,264][105620] Updated weights for policy 1, policy_version 1108014 (0.0011) [2023-12-26 23:26:10,320][105620] Updated weights for policy 1, policy_version 1108024 (0.0010) [2023-12-26 23:26:10,595][105692] Updated weights for policy 0, policy_version 1106885 (0.0008) [2023-12-26 23:26:10,658][105692] Updated weights for policy 0, policy_version 1106895 (0.0010) [2023-12-26 23:26:10,722][105692] Updated weights for policy 0, policy_version 1106905 (0.0011) [2023-12-26 23:26:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 567099392. Throughput: 0: 9525.0, 1: 10015.4. Samples: 567110460. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:26:11,063][104569] Avg episode reward: [(0, '9172.727'), (1, '9351.410')] [2023-12-26 23:26:11,108][105620] Updated weights for policy 1, policy_version 1108034 (0.0008) [2023-12-26 23:26:11,171][105620] Updated weights for policy 1, policy_version 1108044 (0.0008) [2023-12-26 23:26:11,239][105620] Updated weights for policy 1, policy_version 1108054 (0.0008) [2023-12-26 23:26:11,495][105692] Updated weights for policy 0, policy_version 1106915 (0.0011) [2023-12-26 23:26:11,558][105692] Updated weights for policy 0, policy_version 1106925 (0.0011) [2023-12-26 23:26:11,616][105692] Updated weights for policy 0, policy_version 1106935 (0.0010) [2023-12-26 23:26:12,015][105620] Updated weights for policy 1, policy_version 1108064 (0.0010) [2023-12-26 23:26:12,079][105620] Updated weights for policy 1, policy_version 1108074 (0.0010) [2023-12-26 23:26:12,133][105620] Updated weights for policy 1, policy_version 1108084 (0.0009) [2023-12-26 23:26:12,363][105692] Updated weights for policy 0, policy_version 1106945 (0.0008) [2023-12-26 23:26:12,420][105692] Updated weights for policy 0, policy_version 1106955 (0.0008) [2023-12-26 23:26:12,480][105692] Updated weights for policy 0, policy_version 1106965 (0.0008) [2023-12-26 23:26:12,543][105692] Updated weights for policy 0, policy_version 1106975 (0.0008) [2023-12-26 23:26:12,921][105620] Updated weights for policy 1, policy_version 1108094 (0.0010) [2023-12-26 23:26:12,985][105620] Updated weights for policy 1, policy_version 1108104 (0.0010) [2023-12-26 23:26:13,050][105620] Updated weights for policy 1, policy_version 1108114 (0.0010) [2023-12-26 23:26:13,305][105692] Updated weights for policy 0, policy_version 1106985 (0.0008) [2023-12-26 23:26:13,352][105692] Updated weights for policy 0, policy_version 1106995 (0.0008) [2023-12-26 23:26:13,405][105692] Updated weights for policy 0, policy_version 1107006 (0.0009) [2023-12-26 23:26:13,702][105620] Updated weights for policy 1, policy_version 1108124 (0.0008) [2023-12-26 23:26:13,754][105620] Updated weights for policy 1, policy_version 1108134 (0.0007) [2023-12-26 23:26:13,812][105620] Updated weights for policy 1, policy_version 1108145 (0.0008) [2023-12-26 23:26:14,076][105692] Updated weights for policy 0, policy_version 1107016 (0.0007) [2023-12-26 23:26:14,129][105692] Updated weights for policy 0, policy_version 1107026 (0.0008) [2023-12-26 23:26:14,190][105692] Updated weights for policy 0, policy_version 1107036 (0.0005) [2023-12-26 23:26:14,489][105620] Updated weights for policy 1, policy_version 1108155 (0.0007) [2023-12-26 23:26:14,546][105620] Updated weights for policy 1, policy_version 1108165 (0.0009) [2023-12-26 23:26:14,611][105620] Updated weights for policy 1, policy_version 1108175 (0.0010) [2023-12-26 23:26:14,832][105692] Updated weights for policy 0, policy_version 1107046 (0.0006) [2023-12-26 23:26:14,889][105692] Updated weights for policy 0, policy_version 1107056 (0.0008) [2023-12-26 23:26:14,955][105692] Updated weights for policy 0, policy_version 1107066 (0.0006) [2023-12-26 23:26:15,354][105620] Updated weights for policy 1, policy_version 1108185 (0.0010) [2023-12-26 23:26:15,415][105620] Updated weights for policy 1, policy_version 1108195 (0.0011) [2023-12-26 23:26:15,472][105620] Updated weights for policy 1, policy_version 1108205 (0.0010) [2023-12-26 23:26:15,527][105620] Updated weights for policy 1, policy_version 1108215 (0.0009) [2023-12-26 23:26:15,677][105692] Updated weights for policy 0, policy_version 1107076 (0.0006) [2023-12-26 23:26:15,732][105692] Updated weights for policy 0, policy_version 1107086 (0.0005) [2023-12-26 23:26:15,780][105692] Updated weights for policy 0, policy_version 1107096 (0.0005) [2023-12-26 23:26:16,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 567197696. Throughput: 0: 9440.6, 1: 9935.9. Samples: 567166268. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:26:16,063][104569] Avg episode reward: [(0, '9354.976'), (1, '9351.238')] [2023-12-26 23:26:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001107104_283459584.pth... [2023-12-26 23:26:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001108216_283738112.pth... [2023-12-26 23:26:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001106016_283181056.pth [2023-12-26 23:26:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001107072_283443200.pth [2023-12-26 23:26:16,286][105620] Updated weights for policy 1, policy_version 1108225 (0.0010) [2023-12-26 23:26:16,337][105620] Updated weights for policy 1, policy_version 1108235 (0.0010) [2023-12-26 23:26:16,395][105620] Updated weights for policy 1, policy_version 1108245 (0.0010) [2023-12-26 23:26:16,453][105692] Updated weights for policy 0, policy_version 1107106 (0.0006) [2023-12-26 23:26:16,511][105692] Updated weights for policy 0, policy_version 1107116 (0.0006) [2023-12-26 23:26:16,567][105692] Updated weights for policy 0, policy_version 1107126 (0.0008) [2023-12-26 23:26:16,619][105692] Updated weights for policy 0, policy_version 1107136 (0.0009) [2023-12-26 23:26:17,082][105620] Updated weights for policy 1, policy_version 1108255 (0.0009) [2023-12-26 23:26:17,134][105620] Updated weights for policy 1, policy_version 1108265 (0.0008) [2023-12-26 23:26:17,182][105620] Updated weights for policy 1, policy_version 1108275 (0.0006) [2023-12-26 23:26:17,417][105692] Updated weights for policy 0, policy_version 1107146 (0.0009) [2023-12-26 23:26:17,473][105692] Updated weights for policy 0, policy_version 1107156 (0.0009) [2023-12-26 23:26:17,528][105692] Updated weights for policy 0, policy_version 1107166 (0.0009) [2023-12-26 23:26:17,894][105620] Updated weights for policy 1, policy_version 1108285 (0.0010) [2023-12-26 23:26:17,953][105620] Updated weights for policy 1, policy_version 1108295 (0.0010) [2023-12-26 23:26:18,008][105620] Updated weights for policy 1, policy_version 1108305 (0.0010) [2023-12-26 23:26:18,308][105692] Updated weights for policy 0, policy_version 1107176 (0.0009) [2023-12-26 23:26:18,372][105692] Updated weights for policy 0, policy_version 1107186 (0.0007) [2023-12-26 23:26:18,438][105692] Updated weights for policy 0, policy_version 1107196 (0.0006) [2023-12-26 23:26:18,750][105620] Updated weights for policy 1, policy_version 1108315 (0.0010) [2023-12-26 23:26:18,821][105620] Updated weights for policy 1, policy_version 1108325 (0.0011) [2023-12-26 23:26:18,869][105620] Updated weights for policy 1, policy_version 1108335 (0.0010) [2023-12-26 23:26:19,030][105692] Updated weights for policy 0, policy_version 1107206 (0.0006) [2023-12-26 23:26:19,088][105692] Updated weights for policy 0, policy_version 1107216 (0.0006) [2023-12-26 23:26:19,146][105692] Updated weights for policy 0, policy_version 1107226 (0.0006) [2023-12-26 23:26:19,565][105620] Updated weights for policy 1, policy_version 1108345 (0.0010) [2023-12-26 23:26:19,627][105620] Updated weights for policy 1, policy_version 1108355 (0.0010) [2023-12-26 23:26:19,676][105620] Updated weights for policy 1, policy_version 1108365 (0.0010) [2023-12-26 23:26:19,735][105620] Updated weights for policy 1, policy_version 1108375 (0.0010) [2023-12-26 23:26:19,805][105692] Updated weights for policy 0, policy_version 1107236 (0.0007) [2023-12-26 23:26:19,887][105692] Updated weights for policy 0, policy_version 1107246 (0.0007) [2023-12-26 23:26:19,953][105692] Updated weights for policy 0, policy_version 1107256 (0.0007) [2023-12-26 23:26:20,520][105620] Updated weights for policy 1, policy_version 1108385 (0.0011) [2023-12-26 23:26:20,586][105620] Updated weights for policy 1, policy_version 1108395 (0.0011) [2023-12-26 23:26:20,652][105620] Updated weights for policy 1, policy_version 1108405 (0.0011) [2023-12-26 23:26:20,744][105692] Updated weights for policy 0, policy_version 1107266 (0.0008) [2023-12-26 23:26:20,811][105692] Updated weights for policy 0, policy_version 1107276 (0.0009) [2023-12-26 23:26:20,866][105692] Updated weights for policy 0, policy_version 1107286 (0.0010) [2023-12-26 23:26:20,923][105692] Updated weights for policy 0, policy_version 1107296 (0.0009) [2023-12-26 23:26:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 567296000. Throughput: 0: 9494.7, 1: 9952.6. Samples: 567284388. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:26:21,063][104569] Avg episode reward: [(0, '9354.295'), (1, '9351.269')] [2023-12-26 23:26:21,314][105620] Updated weights for policy 1, policy_version 1108415 (0.0011) [2023-12-26 23:26:21,376][105620] Updated weights for policy 1, policy_version 1108425 (0.0011) [2023-12-26 23:26:21,448][105620] Updated weights for policy 1, policy_version 1108435 (0.0010) [2023-12-26 23:26:21,709][105692] Updated weights for policy 0, policy_version 1107306 (0.0010) [2023-12-26 23:26:21,774][105692] Updated weights for policy 0, policy_version 1107316 (0.0008) [2023-12-26 23:26:21,827][105692] Updated weights for policy 0, policy_version 1107326 (0.0008) [2023-12-26 23:26:22,113][105620] Updated weights for policy 1, policy_version 1108445 (0.0010) [2023-12-26 23:26:22,173][105620] Updated weights for policy 1, policy_version 1108455 (0.0011) [2023-12-26 23:26:22,232][105620] Updated weights for policy 1, policy_version 1108465 (0.0010) [2023-12-26 23:26:22,603][105692] Updated weights for policy 0, policy_version 1107337 (0.0008) [2023-12-26 23:26:22,668][105692] Updated weights for policy 0, policy_version 1107347 (0.0009) [2023-12-26 23:26:22,729][105692] Updated weights for policy 0, policy_version 1107357 (0.0007) [2023-12-26 23:26:23,033][105620] Updated weights for policy 1, policy_version 1108475 (0.0011) [2023-12-26 23:26:23,079][105620] Updated weights for policy 1, policy_version 1108485 (0.0010) [2023-12-26 23:26:23,129][105620] Updated weights for policy 1, policy_version 1108495 (0.0010) [2023-12-26 23:26:23,321][105692] Updated weights for policy 0, policy_version 1107367 (0.0007) [2023-12-26 23:26:23,374][105692] Updated weights for policy 0, policy_version 1107377 (0.0005) [2023-12-26 23:26:23,451][105692] Updated weights for policy 0, policy_version 1107387 (0.0005) [2023-12-26 23:26:23,773][105620] Updated weights for policy 1, policy_version 1108505 (0.0010) [2023-12-26 23:26:23,829][105620] Updated weights for policy 1, policy_version 1108515 (0.0005) [2023-12-26 23:26:23,885][105620] Updated weights for policy 1, policy_version 1108525 (0.0010) [2023-12-26 23:26:23,939][105620] Updated weights for policy 1, policy_version 1108535 (0.0010) [2023-12-26 23:26:24,147][105692] Updated weights for policy 0, policy_version 1107397 (0.0007) [2023-12-26 23:26:24,192][105692] Updated weights for policy 0, policy_version 1107407 (0.0006) [2023-12-26 23:26:24,244][105692] Updated weights for policy 0, policy_version 1107417 (0.0005) [2023-12-26 23:26:24,656][105620] Updated weights for policy 1, policy_version 1108545 (0.0010) [2023-12-26 23:26:24,705][105620] Updated weights for policy 1, policy_version 1108555 (0.0010) [2023-12-26 23:26:24,750][105620] Updated weights for policy 1, policy_version 1108565 (0.0010) [2023-12-26 23:26:24,793][105692] Updated weights for policy 0, policy_version 1107427 (0.0005) [2023-12-26 23:26:24,851][105692] Updated weights for policy 0, policy_version 1107437 (0.0005) [2023-12-26 23:26:24,913][105692] Updated weights for policy 0, policy_version 1107447 (0.0005) [2023-12-26 23:26:25,455][105692] Updated weights for policy 0, policy_version 1107457 (0.0006) [2023-12-26 23:26:25,512][105692] Updated weights for policy 0, policy_version 1107467 (0.0008) [2023-12-26 23:26:25,520][105620] Updated weights for policy 1, policy_version 1108575 (0.0010) [2023-12-26 23:26:25,571][105692] Updated weights for policy 0, policy_version 1107477 (0.0005) [2023-12-26 23:26:25,572][105620] Updated weights for policy 1, policy_version 1108585 (0.0010) [2023-12-26 23:26:25,621][105620] Updated weights for policy 1, policy_version 1108595 (0.0010) [2023-12-26 23:26:25,622][105692] Updated weights for policy 0, policy_version 1107487 (0.0007) [2023-12-26 23:26:26,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 567394304. Throughput: 0: 9536.7, 1: 9919.0. Samples: 567403320. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:26:26,062][104569] Avg episode reward: [(0, '9353.173'), (1, '9168.601')] [2023-12-26 23:26:26,192][105692] Updated weights for policy 0, policy_version 1107497 (0.0007) [2023-12-26 23:26:26,243][105692] Updated weights for policy 0, policy_version 1107507 (0.0008) [2023-12-26 23:26:26,296][105692] Updated weights for policy 0, policy_version 1107517 (0.0005) [2023-12-26 23:26:26,397][105620] Updated weights for policy 1, policy_version 1108605 (0.0010) [2023-12-26 23:26:26,451][105620] Updated weights for policy 1, policy_version 1108615 (0.0010) [2023-12-26 23:26:26,512][105620] Updated weights for policy 1, policy_version 1108625 (0.0010) [2023-12-26 23:26:26,832][105692] Updated weights for policy 0, policy_version 1107527 (0.0005) [2023-12-26 23:26:26,884][105692] Updated weights for policy 0, policy_version 1107537 (0.0005) [2023-12-26 23:26:26,934][105692] Updated weights for policy 0, policy_version 1107547 (0.0005) [2023-12-26 23:26:27,257][105620] Updated weights for policy 1, policy_version 1108635 (0.0010) [2023-12-26 23:26:27,308][105620] Updated weights for policy 1, policy_version 1108645 (0.0010) [2023-12-26 23:26:27,369][105620] Updated weights for policy 1, policy_version 1108655 (0.0006) [2023-12-26 23:26:27,566][105692] Updated weights for policy 0, policy_version 1107557 (0.0008) [2023-12-26 23:26:27,614][105692] Updated weights for policy 0, policy_version 1107567 (0.0008) [2023-12-26 23:26:27,664][105692] Updated weights for policy 0, policy_version 1107578 (0.0008) [2023-12-26 23:26:27,970][105620] Updated weights for policy 1, policy_version 1108665 (0.0006) [2023-12-26 23:26:28,033][105620] Updated weights for policy 1, policy_version 1108675 (0.0009) [2023-12-26 23:26:28,101][105620] Updated weights for policy 1, policy_version 1108685 (0.0010) [2023-12-26 23:26:28,162][105620] Updated weights for policy 1, policy_version 1108695 (0.0010) [2023-12-26 23:26:28,275][105692] Updated weights for policy 0, policy_version 1107589 (0.0007) [2023-12-26 23:26:28,343][105692] Updated weights for policy 0, policy_version 1107599 (0.0007) [2023-12-26 23:26:28,398][105692] Updated weights for policy 0, policy_version 1107609 (0.0006) [2023-12-26 23:26:28,857][105620] Updated weights for policy 1, policy_version 1108705 (0.0010) [2023-12-26 23:26:28,909][105620] Updated weights for policy 1, policy_version 1108715 (0.0010) [2023-12-26 23:26:28,922][105692] Updated weights for policy 0, policy_version 1107619 (0.0005) [2023-12-26 23:26:28,962][105620] Updated weights for policy 1, policy_version 1108725 (0.0010) [2023-12-26 23:26:28,987][105692] Updated weights for policy 0, policy_version 1107629 (0.0005) [2023-12-26 23:26:29,050][105692] Updated weights for policy 0, policy_version 1107639 (0.0009) [2023-12-26 23:26:29,713][105620] Updated weights for policy 1, policy_version 1108735 (0.0007) [2023-12-26 23:26:29,716][105692] Updated weights for policy 0, policy_version 1107649 (0.0010) [2023-12-26 23:26:29,781][105692] Updated weights for policy 0, policy_version 1107659 (0.0011) [2023-12-26 23:26:29,782][105620] Updated weights for policy 1, policy_version 1108745 (0.0006) [2023-12-26 23:26:29,846][105620] Updated weights for policy 1, policy_version 1108755 (0.0008) [2023-12-26 23:26:29,852][105692] Updated weights for policy 0, policy_version 1107669 (0.0010) [2023-12-26 23:26:29,905][105692] Updated weights for policy 0, policy_version 1107679 (0.0011) [2023-12-26 23:26:30,531][105620] Updated weights for policy 1, policy_version 1108765 (0.0010) [2023-12-26 23:26:30,581][105620] Updated weights for policy 1, policy_version 1108775 (0.0007) [2023-12-26 23:26:30,645][105620] Updated weights for policy 1, policy_version 1108785 (0.0007) [2023-12-26 23:26:30,649][105692] Updated weights for policy 0, policy_version 1107689 (0.0006) [2023-12-26 23:26:30,704][105692] Updated weights for policy 0, policy_version 1107699 (0.0005) [2023-12-26 23:26:30,760][105692] Updated weights for policy 0, policy_version 1107709 (0.0008) [2023-12-26 23:26:31,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 567500800. Throughput: 0: 9721.4, 1: 9927.8. Samples: 567468232. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:26:31,062][104569] Avg episode reward: [(0, '9350.729'), (1, '9077.329')] [2023-12-26 23:26:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001107712_283615232.pth... [2023-12-26 23:26:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001108792_283885568.pth... [2023-12-26 23:26:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001106528_283312128.pth [2023-12-26 23:26:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001107648_283590656.pth [2023-12-26 23:26:31,372][105620] Updated weights for policy 1, policy_version 1108795 (0.0008) [2023-12-26 23:26:31,426][105620] Updated weights for policy 1, policy_version 1108805 (0.0006) [2023-12-26 23:26:31,466][105692] Updated weights for policy 0, policy_version 1107719 (0.0009) [2023-12-26 23:26:31,485][105620] Updated weights for policy 1, policy_version 1108815 (0.0006) [2023-12-26 23:26:31,528][105692] Updated weights for policy 0, policy_version 1107729 (0.0008) [2023-12-26 23:26:31,587][105692] Updated weights for policy 0, policy_version 1107739 (0.0008) [2023-12-26 23:26:32,220][105620] Updated weights for policy 1, policy_version 1108825 (0.0008) [2023-12-26 23:26:32,282][105620] Updated weights for policy 1, policy_version 1108835 (0.0008) [2023-12-26 23:26:32,348][105692] Updated weights for policy 0, policy_version 1107749 (0.0009) [2023-12-26 23:26:32,348][105620] Updated weights for policy 1, policy_version 1108845 (0.0009) [2023-12-26 23:26:32,413][105620] Updated weights for policy 1, policy_version 1108855 (0.0006) [2023-12-26 23:26:32,414][105692] Updated weights for policy 0, policy_version 1107759 (0.0008) [2023-12-26 23:26:32,475][105692] Updated weights for policy 0, policy_version 1107769 (0.0010) [2023-12-26 23:26:33,101][105620] Updated weights for policy 1, policy_version 1108865 (0.0008) [2023-12-26 23:26:33,147][105620] Updated weights for policy 1, policy_version 1108875 (0.0007) [2023-12-26 23:26:33,194][105620] Updated weights for policy 1, policy_version 1108885 (0.0007) [2023-12-26 23:26:33,227][105692] Updated weights for policy 0, policy_version 1107779 (0.0009) [2023-12-26 23:26:33,287][105692] Updated weights for policy 0, policy_version 1107789 (0.0005) [2023-12-26 23:26:33,355][105692] Updated weights for policy 0, policy_version 1107799 (0.0006) [2023-12-26 23:26:33,850][105620] Updated weights for policy 1, policy_version 1108895 (0.0009) [2023-12-26 23:26:33,905][105620] Updated weights for policy 1, policy_version 1108905 (0.0009) [2023-12-26 23:26:33,955][105620] Updated weights for policy 1, policy_version 1108915 (0.0009) [2023-12-26 23:26:33,991][105692] Updated weights for policy 0, policy_version 1107809 (0.0011) [2023-12-26 23:26:34,043][105692] Updated weights for policy 0, policy_version 1107819 (0.0009) [2023-12-26 23:26:34,094][105692] Updated weights for policy 0, policy_version 1107829 (0.0009) [2023-12-26 23:26:34,156][105692] Updated weights for policy 0, policy_version 1107839 (0.0009) [2023-12-26 23:26:34,626][105620] Updated weights for policy 1, policy_version 1108925 (0.0008) [2023-12-26 23:26:34,684][105620] Updated weights for policy 1, policy_version 1108935 (0.0006) [2023-12-26 23:26:34,740][105620] Updated weights for policy 1, policy_version 1108945 (0.0006) [2023-12-26 23:26:35,094][105692] Updated weights for policy 0, policy_version 1107849 (0.0008) [2023-12-26 23:26:35,143][105692] Updated weights for policy 0, policy_version 1107859 (0.0008) [2023-12-26 23:26:35,191][105692] Updated weights for policy 0, policy_version 1107869 (0.0008) [2023-12-26 23:26:35,365][105620] Updated weights for policy 1, policy_version 1108955 (0.0007) [2023-12-26 23:26:35,427][105620] Updated weights for policy 1, policy_version 1108965 (0.0010) [2023-12-26 23:26:35,486][105620] Updated weights for policy 1, policy_version 1108975 (0.0010) [2023-12-26 23:26:35,908][105692] Updated weights for policy 0, policy_version 1107879 (0.0008) [2023-12-26 23:26:35,969][105692] Updated weights for policy 0, policy_version 1107889 (0.0008) [2023-12-26 23:26:36,024][105692] Updated weights for policy 0, policy_version 1107899 (0.0009) [2023-12-26 23:26:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 567599104. Throughput: 0: 9771.0, 1: 9969.0. Samples: 567586228. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:26:36,063][104569] Avg episode reward: [(0, '9349.979'), (1, '9259.785')] [2023-12-26 23:26:36,186][105620] Updated weights for policy 1, policy_version 1108985 (0.0010) [2023-12-26 23:26:36,240][105620] Updated weights for policy 1, policy_version 1108995 (0.0006) [2023-12-26 23:26:36,293][105620] Updated weights for policy 1, policy_version 1109005 (0.0006) [2023-12-26 23:26:36,361][105620] Updated weights for policy 1, policy_version 1109015 (0.0007) [2023-12-26 23:26:36,915][105692] Updated weights for policy 0, policy_version 1107909 (0.0007) [2023-12-26 23:26:36,942][105620] Updated weights for policy 1, policy_version 1109025 (0.0006) [2023-12-26 23:26:36,981][105692] Updated weights for policy 0, policy_version 1107919 (0.0006) [2023-12-26 23:26:37,004][105620] Updated weights for policy 1, policy_version 1109035 (0.0006) [2023-12-26 23:26:37,026][105692] Updated weights for policy 0, policy_version 1107929 (0.0006) [2023-12-26 23:26:37,068][105620] Updated weights for policy 1, policy_version 1109045 (0.0009) [2023-12-26 23:26:37,670][105620] Updated weights for policy 1, policy_version 1109055 (0.0007) [2023-12-26 23:26:37,731][105620] Updated weights for policy 1, policy_version 1109065 (0.0006) [2023-12-26 23:26:37,785][105620] Updated weights for policy 1, policy_version 1109075 (0.0005) [2023-12-26 23:26:37,800][105692] Updated weights for policy 0, policy_version 1107939 (0.0007) [2023-12-26 23:26:37,854][105692] Updated weights for policy 0, policy_version 1107950 (0.0010) [2023-12-26 23:26:37,908][105692] Updated weights for policy 0, policy_version 1107962 (0.0010) [2023-12-26 23:26:38,429][105620] Updated weights for policy 1, policy_version 1109085 (0.0007) [2023-12-26 23:26:38,481][105620] Updated weights for policy 1, policy_version 1109095 (0.0009) [2023-12-26 23:26:38,546][105620] Updated weights for policy 1, policy_version 1109105 (0.0010) [2023-12-26 23:26:38,757][105692] Updated weights for policy 0, policy_version 1107973 (0.0009) [2023-12-26 23:26:38,816][105692] Updated weights for policy 0, policy_version 1107983 (0.0008) [2023-12-26 23:26:38,877][105692] Updated weights for policy 0, policy_version 1107993 (0.0008) [2023-12-26 23:26:39,280][105620] Updated weights for policy 1, policy_version 1109115 (0.0009) [2023-12-26 23:26:39,341][105620] Updated weights for policy 1, policy_version 1109125 (0.0008) [2023-12-26 23:26:39,420][105620] Updated weights for policy 1, policy_version 1109135 (0.0008) [2023-12-26 23:26:39,698][105692] Updated weights for policy 0, policy_version 1108003 (0.0008) [2023-12-26 23:26:39,765][105692] Updated weights for policy 0, policy_version 1108013 (0.0009) [2023-12-26 23:26:39,828][105692] Updated weights for policy 0, policy_version 1108023 (0.0009) [2023-12-26 23:26:40,205][105620] Updated weights for policy 1, policy_version 1109145 (0.0008) [2023-12-26 23:26:40,264][105620] Updated weights for policy 1, policy_version 1109155 (0.0009) [2023-12-26 23:26:40,317][105620] Updated weights for policy 1, policy_version 1109165 (0.0010) [2023-12-26 23:26:40,376][105620] Updated weights for policy 1, policy_version 1109175 (0.0010) [2023-12-26 23:26:40,556][105692] Updated weights for policy 0, policy_version 1108033 (0.0009) [2023-12-26 23:26:40,614][105692] Updated weights for policy 0, policy_version 1108043 (0.0009) [2023-12-26 23:26:40,677][105692] Updated weights for policy 0, policy_version 1108053 (0.0009) [2023-12-26 23:26:40,734][105692] Updated weights for policy 0, policy_version 1108063 (0.0009) [2023-12-26 23:26:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 567689216. Throughput: 0: 9708.5, 1: 9943.4. Samples: 567699764. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:26:41,062][104569] Avg episode reward: [(0, '9260.660'), (1, '9264.753')] [2023-12-26 23:26:41,173][105620] Updated weights for policy 1, policy_version 1109185 (0.0007) [2023-12-26 23:26:41,225][105620] Updated weights for policy 1, policy_version 1109195 (0.0009) [2023-12-26 23:26:41,287][105620] Updated weights for policy 1, policy_version 1109205 (0.0010) [2023-12-26 23:26:41,438][105692] Updated weights for policy 0, policy_version 1108073 (0.0009) [2023-12-26 23:26:41,486][105692] Updated weights for policy 0, policy_version 1108083 (0.0008) [2023-12-26 23:26:41,539][105692] Updated weights for policy 0, policy_version 1108093 (0.0010) [2023-12-26 23:26:42,083][105620] Updated weights for policy 1, policy_version 1109215 (0.0009) [2023-12-26 23:26:42,147][105620] Updated weights for policy 1, policy_version 1109225 (0.0008) [2023-12-26 23:26:42,212][105620] Updated weights for policy 1, policy_version 1109235 (0.0009) [2023-12-26 23:26:42,391][105692] Updated weights for policy 0, policy_version 1108103 (0.0010) [2023-12-26 23:26:42,454][105692] Updated weights for policy 0, policy_version 1108113 (0.0009) [2023-12-26 23:26:42,522][105692] Updated weights for policy 0, policy_version 1108123 (0.0009) [2023-12-26 23:26:42,969][105620] Updated weights for policy 1, policy_version 1109245 (0.0008) [2023-12-26 23:26:43,020][105620] Updated weights for policy 1, policy_version 1109255 (0.0008) [2023-12-26 23:26:43,082][105620] Updated weights for policy 1, policy_version 1109265 (0.0008) [2023-12-26 23:26:43,281][105692] Updated weights for policy 0, policy_version 1108133 (0.0010) [2023-12-26 23:26:43,333][105692] Updated weights for policy 0, policy_version 1108143 (0.0010) [2023-12-26 23:26:43,381][105692] Updated weights for policy 0, policy_version 1108153 (0.0010) [2023-12-26 23:26:43,703][105620] Updated weights for policy 1, policy_version 1109275 (0.0009) [2023-12-26 23:26:43,757][105620] Updated weights for policy 1, policy_version 1109285 (0.0010) [2023-12-26 23:26:43,801][105620] Updated weights for policy 1, policy_version 1109295 (0.0010) [2023-12-26 23:26:44,159][105692] Updated weights for policy 0, policy_version 1108163 (0.0010) [2023-12-26 23:26:44,209][105692] Updated weights for policy 0, policy_version 1108173 (0.0010) [2023-12-26 23:26:44,277][105692] Updated weights for policy 0, policy_version 1108183 (0.0010) [2023-12-26 23:26:44,486][105620] Updated weights for policy 1, policy_version 1109305 (0.0010) [2023-12-26 23:26:44,548][105620] Updated weights for policy 1, policy_version 1109315 (0.0008) [2023-12-26 23:26:44,605][105620] Updated weights for policy 1, policy_version 1109325 (0.0008) [2023-12-26 23:26:44,671][105620] Updated weights for policy 1, policy_version 1109335 (0.0006) [2023-12-26 23:26:45,014][105692] Updated weights for policy 0, policy_version 1108193 (0.0010) [2023-12-26 23:26:45,080][105692] Updated weights for policy 0, policy_version 1108203 (0.0010) [2023-12-26 23:26:45,143][105692] Updated weights for policy 0, policy_version 1108213 (0.0010) [2023-12-26 23:26:45,205][105692] Updated weights for policy 0, policy_version 1108223 (0.0010) [2023-12-26 23:26:45,410][105620] Updated weights for policy 1, policy_version 1109345 (0.0010) [2023-12-26 23:26:45,480][105620] Updated weights for policy 1, policy_version 1109355 (0.0011) [2023-12-26 23:26:45,542][105620] Updated weights for policy 1, policy_version 1109365 (0.0010) [2023-12-26 23:26:45,939][105692] Updated weights for policy 0, policy_version 1108233 (0.0010) [2023-12-26 23:26:45,990][105692] Updated weights for policy 0, policy_version 1108243 (0.0010) [2023-12-26 23:26:46,035][105692] Updated weights for policy 0, policy_version 1108253 (0.0010) [2023-12-26 23:26:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 567787520. Throughput: 0: 9754.5, 1: 9852.4. Samples: 567755408. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:26:46,063][104569] Avg episode reward: [(0, '9263.107'), (1, '9263.339')] [2023-12-26 23:26:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001108256_283754496.pth... [2023-12-26 23:26:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001109368_284033024.pth... [2023-12-26 23:26:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001107104_283459584.pth [2023-12-26 23:26:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001108216_283738112.pth [2023-12-26 23:26:46,206][105620] Updated weights for policy 1, policy_version 1109375 (0.0007) [2023-12-26 23:26:46,267][105620] Updated weights for policy 1, policy_version 1109385 (0.0005) [2023-12-26 23:26:46,322][105620] Updated weights for policy 1, policy_version 1109395 (0.0005) [2023-12-26 23:26:46,724][105692] Updated weights for policy 0, policy_version 1108263 (0.0007) [2023-12-26 23:26:46,786][105692] Updated weights for policy 0, policy_version 1108273 (0.0005) [2023-12-26 23:26:46,846][105692] Updated weights for policy 0, policy_version 1108283 (0.0008) [2023-12-26 23:26:47,006][105620] Updated weights for policy 1, policy_version 1109405 (0.0005) [2023-12-26 23:26:47,068][105620] Updated weights for policy 1, policy_version 1109415 (0.0005) [2023-12-26 23:26:47,135][105620] Updated weights for policy 1, policy_version 1109425 (0.0005) [2023-12-26 23:26:47,513][105692] Updated weights for policy 0, policy_version 1108293 (0.0007) [2023-12-26 23:26:47,572][105692] Updated weights for policy 0, policy_version 1108303 (0.0009) [2023-12-26 23:26:47,624][105692] Updated weights for policy 0, policy_version 1108313 (0.0009) [2023-12-26 23:26:47,712][105620] Updated weights for policy 1, policy_version 1109435 (0.0006) [2023-12-26 23:26:47,761][105620] Updated weights for policy 1, policy_version 1109445 (0.0005) [2023-12-26 23:26:47,814][105620] Updated weights for policy 1, policy_version 1109455 (0.0005) [2023-12-26 23:26:48,458][105692] Updated weights for policy 0, policy_version 1108323 (0.0008) [2023-12-26 23:26:48,459][105620] Updated weights for policy 1, policy_version 1109465 (0.0008) [2023-12-26 23:26:48,514][105692] Updated weights for policy 0, policy_version 1108333 (0.0006) [2023-12-26 23:26:48,524][105620] Updated weights for policy 1, policy_version 1109475 (0.0008) [2023-12-26 23:26:48,567][105692] Updated weights for policy 0, policy_version 1108343 (0.0005) [2023-12-26 23:26:48,578][105620] Updated weights for policy 1, policy_version 1109485 (0.0008) [2023-12-26 23:26:48,639][105620] Updated weights for policy 1, policy_version 1109495 (0.0008) [2023-12-26 23:26:49,280][105692] Updated weights for policy 0, policy_version 1108353 (0.0005) [2023-12-26 23:26:49,340][105692] Updated weights for policy 0, policy_version 1108363 (0.0007) [2023-12-26 23:26:49,347][105620] Updated weights for policy 1, policy_version 1109505 (0.0008) [2023-12-26 23:26:49,409][105692] Updated weights for policy 0, policy_version 1108373 (0.0009) [2023-12-26 23:26:49,415][105620] Updated weights for policy 1, policy_version 1109515 (0.0008) [2023-12-26 23:26:49,467][105692] Updated weights for policy 0, policy_version 1108383 (0.0008) [2023-12-26 23:26:49,479][105620] Updated weights for policy 1, policy_version 1109525 (0.0007) [2023-12-26 23:26:50,172][105620] Updated weights for policy 1, policy_version 1109535 (0.0007) [2023-12-26 23:26:50,203][105692] Updated weights for policy 0, policy_version 1108393 (0.0006) [2023-12-26 23:26:50,229][105620] Updated weights for policy 1, policy_version 1109545 (0.0005) [2023-12-26 23:26:50,263][105692] Updated weights for policy 0, policy_version 1108403 (0.0007) [2023-12-26 23:26:50,283][105620] Updated weights for policy 1, policy_version 1109555 (0.0008) [2023-12-26 23:26:50,318][105692] Updated weights for policy 0, policy_version 1108413 (0.0007) [2023-12-26 23:26:50,962][105620] Updated weights for policy 1, policy_version 1109565 (0.0005) [2023-12-26 23:26:51,023][105620] Updated weights for policy 1, policy_version 1109575 (0.0008) [2023-12-26 23:26:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 567877632. Throughput: 0: 9837.6, 1: 9880.3. Samples: 567873216. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:26:51,062][104569] Avg episode reward: [(0, '9266.754'), (1, '9257.829')] [2023-12-26 23:26:51,088][105620] Updated weights for policy 1, policy_version 1109585 (0.0008) [2023-12-26 23:26:51,145][105692] Updated weights for policy 0, policy_version 1108423 (0.0007) [2023-12-26 23:26:51,204][105692] Updated weights for policy 0, policy_version 1108433 (0.0008) [2023-12-26 23:26:51,264][105692] Updated weights for policy 0, policy_version 1108443 (0.0008) [2023-12-26 23:26:51,816][105620] Updated weights for policy 1, policy_version 1109595 (0.0008) [2023-12-26 23:26:51,881][105620] Updated weights for policy 1, policy_version 1109605 (0.0009) [2023-12-26 23:26:51,948][105620] Updated weights for policy 1, policy_version 1109615 (0.0010) [2023-12-26 23:26:52,029][105692] Updated weights for policy 0, policy_version 1108453 (0.0007) [2023-12-26 23:26:52,094][105692] Updated weights for policy 0, policy_version 1108463 (0.0006) [2023-12-26 23:26:52,162][105692] Updated weights for policy 0, policy_version 1108473 (0.0005) [2023-12-26 23:26:52,674][105620] Updated weights for policy 1, policy_version 1109625 (0.0009) [2023-12-26 23:26:52,734][105620] Updated weights for policy 1, policy_version 1109635 (0.0009) [2023-12-26 23:26:52,777][105692] Updated weights for policy 0, policy_version 1108483 (0.0009) [2023-12-26 23:26:52,789][105620] Updated weights for policy 1, policy_version 1109645 (0.0009) [2023-12-26 23:26:52,823][105692] Updated weights for policy 0, policy_version 1108493 (0.0005) [2023-12-26 23:26:52,840][105620] Updated weights for policy 1, policy_version 1109655 (0.0008) [2023-12-26 23:26:52,870][105692] Updated weights for policy 0, policy_version 1108503 (0.0005) [2023-12-26 23:26:52,912][105585] KL-divergence is very high: 153.1077 [2023-12-26 23:26:53,409][105692] Updated weights for policy 0, policy_version 1108513 (0.0006) [2023-12-26 23:26:53,465][105692] Updated weights for policy 0, policy_version 1108523 (0.0006) [2023-12-26 23:26:53,529][105692] Updated weights for policy 0, policy_version 1108533 (0.0005) [2023-12-26 23:26:53,580][105692] Updated weights for policy 0, policy_version 1108543 (0.0005) [2023-12-26 23:26:53,744][105620] Updated weights for policy 1, policy_version 1109665 (0.0009) [2023-12-26 23:26:53,796][105620] Updated weights for policy 1, policy_version 1109676 (0.0008) [2023-12-26 23:26:53,849][105620] Updated weights for policy 1, policy_version 1109686 (0.0005) [2023-12-26 23:26:54,085][105692] Updated weights for policy 0, policy_version 1108553 (0.0005) [2023-12-26 23:26:54,134][105692] Updated weights for policy 0, policy_version 1108563 (0.0006) [2023-12-26 23:26:54,193][105692] Updated weights for policy 0, policy_version 1108573 (0.0010) [2023-12-26 23:26:54,445][105620] Updated weights for policy 1, policy_version 1109696 (0.0007) [2023-12-26 23:26:54,511][105620] Updated weights for policy 1, policy_version 1109706 (0.0008) [2023-12-26 23:26:54,572][105620] Updated weights for policy 1, policy_version 1109716 (0.0009) [2023-12-26 23:26:54,852][105692] Updated weights for policy 0, policy_version 1108583 (0.0010) [2023-12-26 23:26:54,920][105692] Updated weights for policy 0, policy_version 1108593 (0.0005) [2023-12-26 23:26:54,979][105692] Updated weights for policy 0, policy_version 1108603 (0.0005) [2023-12-26 23:26:55,337][105620] Updated weights for policy 1, policy_version 1109726 (0.0009) [2023-12-26 23:26:55,393][105620] Updated weights for policy 1, policy_version 1109736 (0.0008) [2023-12-26 23:26:55,444][105620] Updated weights for policy 1, policy_version 1109746 (0.0009) [2023-12-26 23:26:55,551][105692] Updated weights for policy 0, policy_version 1108613 (0.0006) [2023-12-26 23:26:55,604][105692] Updated weights for policy 0, policy_version 1108623 (0.0006) [2023-12-26 23:26:55,666][105692] Updated weights for policy 0, policy_version 1108633 (0.0008) [2023-12-26 23:26:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 567984128. Throughput: 0: 9842.9, 1: 9811.2. Samples: 567994892. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:26:56,062][104569] Avg episode reward: [(0, '9267.534'), (1, '9076.410')] [2023-12-26 23:26:56,099][105620] Updated weights for policy 1, policy_version 1109757 (0.0008) [2023-12-26 23:26:56,151][105620] Updated weights for policy 1, policy_version 1109767 (0.0005) [2023-12-26 23:26:56,212][105620] Updated weights for policy 1, policy_version 1109777 (0.0005) [2023-12-26 23:26:56,273][105692] Updated weights for policy 0, policy_version 1108643 (0.0008) [2023-12-26 23:26:56,341][105692] Updated weights for policy 0, policy_version 1108653 (0.0010) [2023-12-26 23:26:56,404][105692] Updated weights for policy 0, policy_version 1108663 (0.0007) [2023-12-26 23:26:56,837][105620] Updated weights for policy 1, policy_version 1109787 (0.0007) [2023-12-26 23:26:56,884][105620] Updated weights for policy 1, policy_version 1109797 (0.0010) [2023-12-26 23:26:56,928][105620] Updated weights for policy 1, policy_version 1109807 (0.0010) [2023-12-26 23:26:56,954][105692] Updated weights for policy 0, policy_version 1108673 (0.0005) [2023-12-26 23:26:57,007][105692] Updated weights for policy 0, policy_version 1108683 (0.0009) [2023-12-26 23:26:57,051][105692] Updated weights for policy 0, policy_version 1108693 (0.0010) [2023-12-26 23:26:57,095][105692] Updated weights for policy 0, policy_version 1108703 (0.0010) [2023-12-26 23:26:57,553][105620] Updated weights for policy 1, policy_version 1109817 (0.0010) [2023-12-26 23:26:57,606][105620] Updated weights for policy 1, policy_version 1109827 (0.0005) [2023-12-26 23:26:57,662][105620] Updated weights for policy 1, policy_version 1109837 (0.0007) [2023-12-26 23:26:57,710][105620] Updated weights for policy 1, policy_version 1109847 (0.0010) [2023-12-26 23:26:57,834][105692] Updated weights for policy 0, policy_version 1108713 (0.0008) [2023-12-26 23:26:57,879][105692] Updated weights for policy 0, policy_version 1108723 (0.0008) [2023-12-26 23:26:57,931][105692] Updated weights for policy 0, policy_version 1108733 (0.0006) [2023-12-26 23:26:58,472][105620] Updated weights for policy 1, policy_version 1109857 (0.0010) [2023-12-26 23:26:58,529][105620] Updated weights for policy 1, policy_version 1109867 (0.0009) [2023-12-26 23:26:58,599][105620] Updated weights for policy 1, policy_version 1109877 (0.0010) [2023-12-26 23:26:58,760][105692] Updated weights for policy 0, policy_version 1108743 (0.0008) [2023-12-26 23:26:58,825][105692] Updated weights for policy 0, policy_version 1108753 (0.0011) [2023-12-26 23:26:58,897][105692] Updated weights for policy 0, policy_version 1108763 (0.0009) [2023-12-26 23:26:59,405][105620] Updated weights for policy 1, policy_version 1109887 (0.0009) [2023-12-26 23:26:59,467][105620] Updated weights for policy 1, policy_version 1109897 (0.0008) [2023-12-26 23:26:59,524][105620] Updated weights for policy 1, policy_version 1109907 (0.0007) [2023-12-26 23:26:59,623][105692] Updated weights for policy 0, policy_version 1108773 (0.0009) [2023-12-26 23:26:59,680][105692] Updated weights for policy 0, policy_version 1108783 (0.0009) [2023-12-26 23:26:59,736][105692] Updated weights for policy 0, policy_version 1108793 (0.0005) [2023-12-26 23:27:00,245][105620] Updated weights for policy 1, policy_version 1109917 (0.0009) [2023-12-26 23:27:00,294][105620] Updated weights for policy 1, policy_version 1109927 (0.0009) [2023-12-26 23:27:00,321][105692] Updated weights for policy 0, policy_version 1108803 (0.0007) [2023-12-26 23:27:00,352][105620] Updated weights for policy 1, policy_version 1109937 (0.0008) [2023-12-26 23:27:00,381][105692] Updated weights for policy 0, policy_version 1108813 (0.0005) [2023-12-26 23:27:00,436][105692] Updated weights for policy 0, policy_version 1108823 (0.0005) [2023-12-26 23:27:00,952][105692] Updated weights for policy 0, policy_version 1108833 (0.0006) [2023-12-26 23:27:01,012][105692] Updated weights for policy 0, policy_version 1108843 (0.0005) [2023-12-26 23:27:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 568082432. Throughput: 0: 9926.4, 1: 9856.7. Samples: 568056500. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:27:01,063][104569] Avg episode reward: [(0, '9264.797'), (1, '9076.333')] [2023-12-26 23:27:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001109944_284180480.pth... [2023-12-26 23:27:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001108792_283885568.pth [2023-12-26 23:27:01,079][105692] Updated weights for policy 0, policy_version 1108853 (0.0006) [2023-12-26 23:27:01,148][105692] Updated weights for policy 0, policy_version 1108863 (0.0006) [2023-12-26 23:27:01,152][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001108864_283910144.pth... [2023-12-26 23:27:01,157][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001107712_283615232.pth [2023-12-26 23:27:01,232][105620] Updated weights for policy 1, policy_version 1109947 (0.0009) [2023-12-26 23:27:01,288][105620] Updated weights for policy 1, policy_version 1109957 (0.0007) [2023-12-26 23:27:01,337][105620] Updated weights for policy 1, policy_version 1109967 (0.0008) [2023-12-26 23:27:01,895][105692] Updated weights for policy 0, policy_version 1108873 (0.0009) [2023-12-26 23:27:01,951][105692] Updated weights for policy 0, policy_version 1108883 (0.0009) [2023-12-26 23:27:01,965][105620] Updated weights for policy 1, policy_version 1109977 (0.0007) [2023-12-26 23:27:02,001][105692] Updated weights for policy 0, policy_version 1108893 (0.0008) [2023-12-26 23:27:02,022][105620] Updated weights for policy 1, policy_version 1109987 (0.0005) [2023-12-26 23:27:02,082][105620] Updated weights for policy 1, policy_version 1109997 (0.0005) [2023-12-26 23:27:02,146][105620] Updated weights for policy 1, policy_version 1110007 (0.0005) [2023-12-26 23:27:02,751][105620] Updated weights for policy 1, policy_version 1110017 (0.0005) [2023-12-26 23:27:02,773][105692] Updated weights for policy 0, policy_version 1108903 (0.0007) [2023-12-26 23:27:02,801][105620] Updated weights for policy 1, policy_version 1110027 (0.0006) [2023-12-26 23:27:02,832][105692] Updated weights for policy 0, policy_version 1108913 (0.0007) [2023-12-26 23:27:02,864][105620] Updated weights for policy 1, policy_version 1110037 (0.0011) [2023-12-26 23:27:02,890][105692] Updated weights for policy 0, policy_version 1108923 (0.0010) [2023-12-26 23:27:03,424][105620] Updated weights for policy 1, policy_version 1110047 (0.0007) [2023-12-26 23:27:03,463][105692] Updated weights for policy 0, policy_version 1108933 (0.0007) [2023-12-26 23:27:03,473][105620] Updated weights for policy 1, policy_version 1110057 (0.0005) [2023-12-26 23:27:03,519][105692] Updated weights for policy 0, policy_version 1108943 (0.0007) [2023-12-26 23:27:03,533][105620] Updated weights for policy 1, policy_version 1110067 (0.0007) [2023-12-26 23:27:03,585][105692] Updated weights for policy 0, policy_version 1108953 (0.0008) [2023-12-26 23:27:04,222][105620] Updated weights for policy 1, policy_version 1110077 (0.0007) [2023-12-26 23:27:04,269][105620] Updated weights for policy 1, policy_version 1110087 (0.0009) [2023-12-26 23:27:04,333][105620] Updated weights for policy 1, policy_version 1110097 (0.0008) [2023-12-26 23:27:04,335][105692] Updated weights for policy 0, policy_version 1108963 (0.0010) [2023-12-26 23:27:04,391][105692] Updated weights for policy 0, policy_version 1108973 (0.0007) [2023-12-26 23:27:04,445][105692] Updated weights for policy 0, policy_version 1108983 (0.0010) [2023-12-26 23:27:04,976][105620] Updated weights for policy 1, policy_version 1110107 (0.0007) [2023-12-26 23:27:05,034][105620] Updated weights for policy 1, policy_version 1110117 (0.0006) [2023-12-26 23:27:05,090][105620] Updated weights for policy 1, policy_version 1110127 (0.0006) [2023-12-26 23:27:05,345][105692] Updated weights for policy 0, policy_version 1108993 (0.0010) [2023-12-26 23:27:05,405][105692] Updated weights for policy 0, policy_version 1109003 (0.0009) [2023-12-26 23:27:05,459][105692] Updated weights for policy 0, policy_version 1109013 (0.0009) [2023-12-26 23:27:05,513][105692] Updated weights for policy 0, policy_version 1109023 (0.0005) [2023-12-26 23:27:05,706][105620] Updated weights for policy 1, policy_version 1110137 (0.0006) [2023-12-26 23:27:05,772][105620] Updated weights for policy 1, policy_version 1110147 (0.0005) [2023-12-26 23:27:05,844][105620] Updated weights for policy 1, policy_version 1110157 (0.0005) [2023-12-26 23:27:05,902][105620] Updated weights for policy 1, policy_version 1110167 (0.0005) [2023-12-26 23:27:06,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 568188928. Throughput: 0: 9922.9, 1: 9929.3. Samples: 568177732. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:27:06,062][104569] Avg episode reward: [(0, '9264.020'), (1, '9167.031')] [2023-12-26 23:27:06,161][105692] Updated weights for policy 0, policy_version 1109033 (0.0009) [2023-12-26 23:27:06,213][105692] Updated weights for policy 0, policy_version 1109043 (0.0009) [2023-12-26 23:27:06,265][105692] Updated weights for policy 0, policy_version 1109053 (0.0009) [2023-12-26 23:27:06,473][105620] Updated weights for policy 1, policy_version 1110177 (0.0008) [2023-12-26 23:27:06,529][105620] Updated weights for policy 1, policy_version 1110187 (0.0009) [2023-12-26 23:27:06,585][105620] Updated weights for policy 1, policy_version 1110197 (0.0009) [2023-12-26 23:27:07,044][105692] Updated weights for policy 0, policy_version 1109063 (0.0009) [2023-12-26 23:27:07,098][105692] Updated weights for policy 0, policy_version 1109073 (0.0008) [2023-12-26 23:27:07,153][105692] Updated weights for policy 0, policy_version 1109083 (0.0009) [2023-12-26 23:27:07,379][105620] Updated weights for policy 1, policy_version 1110207 (0.0007) [2023-12-26 23:27:07,440][105620] Updated weights for policy 1, policy_version 1110217 (0.0006) [2023-12-26 23:27:07,495][105620] Updated weights for policy 1, policy_version 1110227 (0.0008) [2023-12-26 23:27:07,922][105692] Updated weights for policy 0, policy_version 1109093 (0.0007) [2023-12-26 23:27:07,967][105692] Updated weights for policy 0, policy_version 1109103 (0.0005) [2023-12-26 23:27:08,025][105692] Updated weights for policy 0, policy_version 1109113 (0.0006) [2023-12-26 23:27:08,105][105620] Updated weights for policy 1, policy_version 1110237 (0.0007) [2023-12-26 23:27:08,165][105620] Updated weights for policy 1, policy_version 1110247 (0.0011) [2023-12-26 23:27:08,201][105586] KL-divergence is very high: 110.1808 [2023-12-26 23:27:08,233][105620] Updated weights for policy 1, policy_version 1110257 (0.0011) [2023-12-26 23:27:08,620][105692] Updated weights for policy 0, policy_version 1109123 (0.0006) [2023-12-26 23:27:08,686][105692] Updated weights for policy 0, policy_version 1109133 (0.0008) [2023-12-26 23:27:08,739][105692] Updated weights for policy 0, policy_version 1109143 (0.0010) [2023-12-26 23:27:08,842][105620] Updated weights for policy 1, policy_version 1110267 (0.0010) [2023-12-26 23:27:08,889][105620] Updated weights for policy 1, policy_version 1110277 (0.0007) [2023-12-26 23:27:08,948][105620] Updated weights for policy 1, policy_version 1110287 (0.0009) [2023-12-26 23:27:09,599][105692] Updated weights for policy 0, policy_version 1109153 (0.0008) [2023-12-26 23:27:09,613][105620] Updated weights for policy 1, policy_version 1110297 (0.0005) [2023-12-26 23:27:09,661][105692] Updated weights for policy 0, policy_version 1109163 (0.0009) [2023-12-26 23:27:09,672][105620] Updated weights for policy 1, policy_version 1110307 (0.0007) [2023-12-26 23:27:09,710][105692] Updated weights for policy 0, policy_version 1109173 (0.0009) [2023-12-26 23:27:09,734][105620] Updated weights for policy 1, policy_version 1110317 (0.0006) [2023-12-26 23:27:09,763][105692] Updated weights for policy 0, policy_version 1109183 (0.0009) [2023-12-26 23:27:09,790][105620] Updated weights for policy 1, policy_version 1110327 (0.0011) [2023-12-26 23:27:10,522][105620] Updated weights for policy 1, policy_version 1110337 (0.0011) [2023-12-26 23:27:10,577][105692] Updated weights for policy 0, policy_version 1109193 (0.0008) [2023-12-26 23:27:10,585][105620] Updated weights for policy 1, policy_version 1110347 (0.0010) [2023-12-26 23:27:10,636][105692] Updated weights for policy 0, policy_version 1109203 (0.0010) [2023-12-26 23:27:10,644][105620] Updated weights for policy 1, policy_version 1110357 (0.0010) [2023-12-26 23:27:10,697][105692] Updated weights for policy 0, policy_version 1109213 (0.0009) [2023-12-26 23:27:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19494.2). Total num frames: 568287232. Throughput: 0: 9816.6, 1: 10007.8. Samples: 568295420. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:27:11,062][104569] Avg episode reward: [(0, '9355.951'), (1, '8985.486')] [2023-12-26 23:27:11,420][105620] Updated weights for policy 1, policy_version 1110367 (0.0007) [2023-12-26 23:27:11,479][105620] Updated weights for policy 1, policy_version 1110377 (0.0005) [2023-12-26 23:27:11,495][105692] Updated weights for policy 0, policy_version 1109223 (0.0008) [2023-12-26 23:27:11,536][105620] Updated weights for policy 1, policy_version 1110387 (0.0007) [2023-12-26 23:27:11,563][105692] Updated weights for policy 0, policy_version 1109233 (0.0008) [2023-12-26 23:27:11,628][105692] Updated weights for policy 0, policy_version 1109243 (0.0008) [2023-12-26 23:27:12,179][105620] Updated weights for policy 1, policy_version 1110397 (0.0009) [2023-12-26 23:27:12,245][105620] Updated weights for policy 1, policy_version 1110407 (0.0009) [2023-12-26 23:27:12,311][105620] Updated weights for policy 1, policy_version 1110417 (0.0009) [2023-12-26 23:27:12,412][105692] Updated weights for policy 0, policy_version 1109253 (0.0010) [2023-12-26 23:27:12,474][105692] Updated weights for policy 0, policy_version 1109263 (0.0011) [2023-12-26 23:27:12,534][105692] Updated weights for policy 0, policy_version 1109273 (0.0011) [2023-12-26 23:27:12,945][105620] Updated weights for policy 1, policy_version 1110427 (0.0008) [2023-12-26 23:27:13,017][105620] Updated weights for policy 1, policy_version 1110437 (0.0007) [2023-12-26 23:27:13,082][105620] Updated weights for policy 1, policy_version 1110447 (0.0010) [2023-12-26 23:27:13,250][105692] Updated weights for policy 0, policy_version 1109283 (0.0009) [2023-12-26 23:27:13,311][105692] Updated weights for policy 0, policy_version 1109293 (0.0005) [2023-12-26 23:27:13,364][105692] Updated weights for policy 0, policy_version 1109303 (0.0005) [2023-12-26 23:27:13,693][105620] Updated weights for policy 1, policy_version 1110457 (0.0008) [2023-12-26 23:27:13,751][105620] Updated weights for policy 1, policy_version 1110467 (0.0010) [2023-12-26 23:27:13,809][105620] Updated weights for policy 1, policy_version 1110477 (0.0010) [2023-12-26 23:27:13,877][105620] Updated weights for policy 1, policy_version 1110487 (0.0010) [2023-12-26 23:27:14,053][105692] Updated weights for policy 0, policy_version 1109313 (0.0009) [2023-12-26 23:27:14,111][105692] Updated weights for policy 0, policy_version 1109323 (0.0010) [2023-12-26 23:27:14,163][105692] Updated weights for policy 0, policy_version 1109333 (0.0010) [2023-12-26 23:27:14,211][105692] Updated weights for policy 0, policy_version 1109343 (0.0010) [2023-12-26 23:27:14,556][105620] Updated weights for policy 1, policy_version 1110497 (0.0008) [2023-12-26 23:27:14,610][105620] Updated weights for policy 1, policy_version 1110507 (0.0005) [2023-12-26 23:27:14,666][105620] Updated weights for policy 1, policy_version 1110517 (0.0005) [2023-12-26 23:27:14,935][105692] Updated weights for policy 0, policy_version 1109353 (0.0008) [2023-12-26 23:27:14,994][105692] Updated weights for policy 0, policy_version 1109363 (0.0009) [2023-12-26 23:27:15,047][105692] Updated weights for policy 0, policy_version 1109373 (0.0010) [2023-12-26 23:27:15,339][105620] Updated weights for policy 1, policy_version 1110527 (0.0009) [2023-12-26 23:27:15,391][105620] Updated weights for policy 1, policy_version 1110537 (0.0006) [2023-12-26 23:27:15,438][105620] Updated weights for policy 1, policy_version 1110547 (0.0005) [2023-12-26 23:27:15,738][105692] Updated weights for policy 0, policy_version 1109383 (0.0008) [2023-12-26 23:27:15,790][105692] Updated weights for policy 0, policy_version 1109393 (0.0008) [2023-12-26 23:27:15,851][105692] Updated weights for policy 0, policy_version 1109403 (0.0010) [2023-12-26 23:27:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.5, 300 sec: 19522.0). Total num frames: 568385536. Throughput: 0: 9647.8, 1: 10033.3. Samples: 568353884. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:27:16,062][104569] Avg episode reward: [(0, '9356.685'), (1, '8985.366')] [2023-12-26 23:27:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001109408_284049408.pth... [2023-12-26 23:27:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001110552_284336128.pth... [2023-12-26 23:27:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001109368_284033024.pth [2023-12-26 23:27:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001108256_283754496.pth [2023-12-26 23:27:16,132][105620] Updated weights for policy 1, policy_version 1110557 (0.0005) [2023-12-26 23:27:16,180][105620] Updated weights for policy 1, policy_version 1110567 (0.0005) [2023-12-26 23:27:16,242][105620] Updated weights for policy 1, policy_version 1110577 (0.0005) [2023-12-26 23:27:16,538][105692] Updated weights for policy 0, policy_version 1109413 (0.0010) [2023-12-26 23:27:16,608][105692] Updated weights for policy 0, policy_version 1109423 (0.0010) [2023-12-26 23:27:16,668][105692] Updated weights for policy 0, policy_version 1109433 (0.0009) [2023-12-26 23:27:16,763][105620] Updated weights for policy 1, policy_version 1110587 (0.0006) [2023-12-26 23:27:16,817][105620] Updated weights for policy 1, policy_version 1110597 (0.0009) [2023-12-26 23:27:16,882][105620] Updated weights for policy 1, policy_version 1110607 (0.0005) [2023-12-26 23:27:17,434][105620] Updated weights for policy 1, policy_version 1110617 (0.0005) [2023-12-26 23:27:17,494][105620] Updated weights for policy 1, policy_version 1110627 (0.0009) [2023-12-26 23:27:17,556][105692] Updated weights for policy 0, policy_version 1109444 (0.0009) [2023-12-26 23:27:17,556][105620] Updated weights for policy 1, policy_version 1110637 (0.0008) [2023-12-26 23:27:17,613][105692] Updated weights for policy 0, policy_version 1109454 (0.0008) [2023-12-26 23:27:17,618][105620] Updated weights for policy 1, policy_version 1110647 (0.0005) [2023-12-26 23:27:17,665][105692] Updated weights for policy 0, policy_version 1109464 (0.0010) [2023-12-26 23:27:18,321][105620] Updated weights for policy 1, policy_version 1110657 (0.0009) [2023-12-26 23:27:18,382][105620] Updated weights for policy 1, policy_version 1110667 (0.0008) [2023-12-26 23:27:18,443][105692] Updated weights for policy 0, policy_version 1109474 (0.0009) [2023-12-26 23:27:18,444][105620] Updated weights for policy 1, policy_version 1110677 (0.0010) [2023-12-26 23:27:18,500][105692] Updated weights for policy 0, policy_version 1109484 (0.0006) [2023-12-26 23:27:18,559][105692] Updated weights for policy 0, policy_version 1109494 (0.0009) [2023-12-26 23:27:18,611][105692] Updated weights for policy 0, policy_version 1109504 (0.0009) [2023-12-26 23:27:19,206][105620] Updated weights for policy 1, policy_version 1110687 (0.0009) [2023-12-26 23:27:19,269][105620] Updated weights for policy 1, policy_version 1110697 (0.0008) [2023-12-26 23:27:19,325][105620] Updated weights for policy 1, policy_version 1110707 (0.0009) [2023-12-26 23:27:19,362][105692] Updated weights for policy 0, policy_version 1109514 (0.0008) [2023-12-26 23:27:19,423][105692] Updated weights for policy 0, policy_version 1109524 (0.0009) [2023-12-26 23:27:19,480][105692] Updated weights for policy 0, policy_version 1109534 (0.0009) [2023-12-26 23:27:20,111][105620] Updated weights for policy 1, policy_version 1110717 (0.0009) [2023-12-26 23:27:20,169][105620] Updated weights for policy 1, policy_version 1110727 (0.0008) [2023-12-26 23:27:20,231][105620] Updated weights for policy 1, policy_version 1110737 (0.0009) [2023-12-26 23:27:20,254][105692] Updated weights for policy 0, policy_version 1109544 (0.0006) [2023-12-26 23:27:20,300][105692] Updated weights for policy 0, policy_version 1109554 (0.0008) [2023-12-26 23:27:20,360][105692] Updated weights for policy 0, policy_version 1109564 (0.0009) [2023-12-26 23:27:21,024][105620] Updated weights for policy 1, policy_version 1110747 (0.0008) [2023-12-26 23:27:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.9, 300 sec: 19494.2). Total num frames: 568475648. Throughput: 0: 9594.5, 1: 10082.7. Samples: 568471700. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:27:21,062][104569] Avg episode reward: [(0, '9356.865'), (1, '9259.505')] [2023-12-26 23:27:21,084][105620] Updated weights for policy 1, policy_version 1110757 (0.0009) [2023-12-26 23:27:21,142][105620] Updated weights for policy 1, policy_version 1110767 (0.0010) [2023-12-26 23:27:21,209][105692] Updated weights for policy 0, policy_version 1109574 (0.0007) [2023-12-26 23:27:21,266][105692] Updated weights for policy 0, policy_version 1109584 (0.0009) [2023-12-26 23:27:21,330][105692] Updated weights for policy 0, policy_version 1109594 (0.0009) [2023-12-26 23:27:21,995][105692] Updated weights for policy 0, policy_version 1109604 (0.0008) [2023-12-26 23:27:22,010][105620] Updated weights for policy 1, policy_version 1110777 (0.0007) [2023-12-26 23:27:22,058][105692] Updated weights for policy 0, policy_version 1109614 (0.0007) [2023-12-26 23:27:22,072][105620] Updated weights for policy 1, policy_version 1110787 (0.0007) [2023-12-26 23:27:22,115][105692] Updated weights for policy 0, policy_version 1109624 (0.0006) [2023-12-26 23:27:22,133][105620] Updated weights for policy 1, policy_version 1110797 (0.0008) [2023-12-26 23:27:22,189][105620] Updated weights for policy 1, policy_version 1110807 (0.0007) [2023-12-26 23:27:22,764][105692] Updated weights for policy 0, policy_version 1109634 (0.0006) [2023-12-26 23:27:22,836][105692] Updated weights for policy 0, policy_version 1109644 (0.0006) [2023-12-26 23:27:22,902][105692] Updated weights for policy 0, policy_version 1109654 (0.0007) [2023-12-26 23:27:22,967][105692] Updated weights for policy 0, policy_version 1109664 (0.0007) [2023-12-26 23:27:23,023][105620] Updated weights for policy 1, policy_version 1110817 (0.0009) [2023-12-26 23:27:23,071][105620] Updated weights for policy 1, policy_version 1110827 (0.0009) [2023-12-26 23:27:23,127][105620] Updated weights for policy 1, policy_version 1110837 (0.0009) [2023-12-26 23:27:23,622][105692] Updated weights for policy 0, policy_version 1109674 (0.0009) [2023-12-26 23:27:23,687][105692] Updated weights for policy 0, policy_version 1109684 (0.0009) [2023-12-26 23:27:23,749][105692] Updated weights for policy 0, policy_version 1109694 (0.0009) [2023-12-26 23:27:23,902][105620] Updated weights for policy 1, policy_version 1110847 (0.0009) [2023-12-26 23:27:23,957][105620] Updated weights for policy 1, policy_version 1110857 (0.0009) [2023-12-26 23:27:24,008][105620] Updated weights for policy 1, policy_version 1110867 (0.0009) [2023-12-26 23:27:24,478][105692] Updated weights for policy 0, policy_version 1109704 (0.0009) [2023-12-26 23:27:24,525][105692] Updated weights for policy 0, policy_version 1109714 (0.0009) [2023-12-26 23:27:24,573][105692] Updated weights for policy 0, policy_version 1109724 (0.0009) [2023-12-26 23:27:24,760][105620] Updated weights for policy 1, policy_version 1110877 (0.0009) [2023-12-26 23:27:24,813][105620] Updated weights for policy 1, policy_version 1110887 (0.0010) [2023-12-26 23:27:24,870][105620] Updated weights for policy 1, policy_version 1110897 (0.0009) [2023-12-26 23:27:25,336][105692] Updated weights for policy 0, policy_version 1109735 (0.0009) [2023-12-26 23:27:25,388][105692] Updated weights for policy 0, policy_version 1109745 (0.0009) [2023-12-26 23:27:25,441][105692] Updated weights for policy 0, policy_version 1109757 (0.0010) [2023-12-26 23:27:25,571][105620] Updated weights for policy 1, policy_version 1110907 (0.0009) [2023-12-26 23:27:25,635][105620] Updated weights for policy 1, policy_version 1110917 (0.0010) [2023-12-26 23:27:25,710][105620] Updated weights for policy 1, policy_version 1110927 (0.0011) [2023-12-26 23:27:26,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19660.7, 300 sec: 19494.2). Total num frames: 568573952. Throughput: 0: 9685.3, 1: 9946.8. Samples: 568583212. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:27:26,063][104569] Avg episode reward: [(0, '9265.693'), (1, '9259.602')] [2023-12-26 23:27:26,242][105692] Updated weights for policy 0, policy_version 1109767 (0.0009) [2023-12-26 23:27:26,289][105692] Updated weights for policy 0, policy_version 1109777 (0.0009) [2023-12-26 23:27:26,338][105692] Updated weights for policy 0, policy_version 1109787 (0.0009) [2023-12-26 23:27:26,365][105620] Updated weights for policy 1, policy_version 1110937 (0.0010) [2023-12-26 23:27:26,418][105620] Updated weights for policy 1, policy_version 1110947 (0.0005) [2023-12-26 23:27:26,479][105620] Updated weights for policy 1, policy_version 1110957 (0.0005) [2023-12-26 23:27:26,526][105620] Updated weights for policy 1, policy_version 1110967 (0.0005) [2023-12-26 23:27:27,083][105620] Updated weights for policy 1, policy_version 1110977 (0.0005) [2023-12-26 23:27:27,139][105692] Updated weights for policy 0, policy_version 1109797 (0.0009) [2023-12-26 23:27:27,144][105620] Updated weights for policy 1, policy_version 1110987 (0.0006) [2023-12-26 23:27:27,197][105620] Updated weights for policy 1, policy_version 1110997 (0.0006) [2023-12-26 23:27:27,201][105692] Updated weights for policy 0, policy_version 1109807 (0.0008) [2023-12-26 23:27:27,251][105692] Updated weights for policy 0, policy_version 1109817 (0.0008) [2023-12-26 23:27:27,749][105620] Updated weights for policy 1, policy_version 1111007 (0.0006) [2023-12-26 23:27:27,805][105620] Updated weights for policy 1, policy_version 1111017 (0.0008) [2023-12-26 23:27:27,858][105620] Updated weights for policy 1, policy_version 1111027 (0.0010) [2023-12-26 23:27:28,002][105692] Updated weights for policy 0, policy_version 1109828 (0.0010) [2023-12-26 23:27:28,060][105692] Updated weights for policy 0, policy_version 1109839 (0.0010) [2023-12-26 23:27:28,118][105692] Updated weights for policy 0, policy_version 1109849 (0.0010) [2023-12-26 23:27:28,494][105620] Updated weights for policy 1, policy_version 1111037 (0.0010) [2023-12-26 23:27:28,555][105620] Updated weights for policy 1, policy_version 1111047 (0.0009) [2023-12-26 23:27:28,609][105620] Updated weights for policy 1, policy_version 1111057 (0.0009) [2023-12-26 23:27:28,843][105692] Updated weights for policy 0, policy_version 1109859 (0.0009) [2023-12-26 23:27:28,903][105692] Updated weights for policy 0, policy_version 1109869 (0.0008) [2023-12-26 23:27:28,953][105692] Updated weights for policy 0, policy_version 1109879 (0.0008) [2023-12-26 23:27:29,416][105620] Updated weights for policy 1, policy_version 1111067 (0.0009) [2023-12-26 23:27:29,471][105620] Updated weights for policy 1, policy_version 1111077 (0.0010) [2023-12-26 23:27:29,529][105620] Updated weights for policy 1, policy_version 1111087 (0.0010) [2023-12-26 23:27:29,718][105692] Updated weights for policy 0, policy_version 1109889 (0.0007) [2023-12-26 23:27:29,786][105692] Updated weights for policy 0, policy_version 1109899 (0.0006) [2023-12-26 23:27:29,849][105692] Updated weights for policy 0, policy_version 1109909 (0.0007) [2023-12-26 23:27:29,905][105692] Updated weights for policy 0, policy_version 1109919 (0.0008) [2023-12-26 23:27:30,184][105620] Updated weights for policy 1, policy_version 1111097 (0.0010) [2023-12-26 23:27:30,244][105620] Updated weights for policy 1, policy_version 1111107 (0.0006) [2023-12-26 23:27:30,301][105620] Updated weights for policy 1, policy_version 1111117 (0.0005) [2023-12-26 23:27:30,363][105620] Updated weights for policy 1, policy_version 1111127 (0.0007) [2023-12-26 23:27:30,604][105692] Updated weights for policy 0, policy_version 1109929 (0.0010) [2023-12-26 23:27:30,667][105692] Updated weights for policy 0, policy_version 1109939 (0.0007) [2023-12-26 23:27:30,717][105692] Updated weights for policy 0, policy_version 1109949 (0.0007) [2023-12-26 23:27:31,049][105620] Updated weights for policy 1, policy_version 1111137 (0.0010) [2023-12-26 23:27:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 568672256. Throughput: 0: 9694.4, 1: 10048.0. Samples: 568643816. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:27:31,063][104569] Avg episode reward: [(0, '9173.811'), (1, '9258.261')] [2023-12-26 23:27:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001109952_284188672.pth... [2023-12-26 23:27:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001108864_283910144.pth [2023-12-26 23:27:31,118][105620] Updated weights for policy 1, policy_version 1111147 (0.0007) [2023-12-26 23:27:31,187][105620] Updated weights for policy 1, policy_version 1111157 (0.0008) [2023-12-26 23:27:31,206][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001111160_284491776.pth... [2023-12-26 23:27:31,211][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001109944_284180480.pth [2023-12-26 23:27:31,460][105692] Updated weights for policy 0, policy_version 1109959 (0.0009) [2023-12-26 23:27:31,521][105692] Updated weights for policy 0, policy_version 1109969 (0.0009) [2023-12-26 23:27:31,576][105692] Updated weights for policy 0, policy_version 1109979 (0.0009) [2023-12-26 23:27:31,924][105620] Updated weights for policy 1, policy_version 1111167 (0.0006) [2023-12-26 23:27:31,972][105620] Updated weights for policy 1, policy_version 1111177 (0.0005) [2023-12-26 23:27:32,037][105620] Updated weights for policy 1, policy_version 1111187 (0.0006) [2023-12-26 23:27:32,288][105692] Updated weights for policy 0, policy_version 1109989 (0.0008) [2023-12-26 23:27:32,312][105585] KL-divergence is very high: 166.5469 [2023-12-26 23:27:32,351][105692] Updated weights for policy 0, policy_version 1109999 (0.0006) [2023-12-26 23:27:32,366][105585] KL-divergence is very high: 321.2848 [2023-12-26 23:27:32,411][105692] Updated weights for policy 0, policy_version 1110009 (0.0009) [2023-12-26 23:27:32,411][105585] KL-divergence is very high: 352.1274 [2023-12-26 23:27:32,669][105620] Updated weights for policy 1, policy_version 1111197 (0.0008) [2023-12-26 23:27:32,726][105620] Updated weights for policy 1, policy_version 1111207 (0.0009) [2023-12-26 23:27:32,771][105620] Updated weights for policy 1, policy_version 1111217 (0.0008) [2023-12-26 23:27:33,032][105692] Updated weights for policy 0, policy_version 1110019 (0.0007) [2023-12-26 23:27:33,087][105692] Updated weights for policy 0, policy_version 1110029 (0.0005) [2023-12-26 23:27:33,135][105692] Updated weights for policy 0, policy_version 1110039 (0.0005) [2023-12-26 23:27:33,570][105620] Updated weights for policy 1, policy_version 1111227 (0.0009) [2023-12-26 23:27:33,624][105620] Updated weights for policy 1, policy_version 1111237 (0.0009) [2023-12-26 23:27:33,678][105620] Updated weights for policy 1, policy_version 1111248 (0.0010) [2023-12-26 23:27:33,767][105692] Updated weights for policy 0, policy_version 1110049 (0.0006) [2023-12-26 23:27:33,820][105692] Updated weights for policy 0, policy_version 1110059 (0.0010) [2023-12-26 23:27:33,873][105692] Updated weights for policy 0, policy_version 1110069 (0.0008) [2023-12-26 23:27:33,926][105692] Updated weights for policy 0, policy_version 1110079 (0.0009) [2023-12-26 23:27:34,422][105620] Updated weights for policy 1, policy_version 1111258 (0.0008) [2023-12-26 23:27:34,491][105620] Updated weights for policy 1, policy_version 1111268 (0.0007) [2023-12-26 23:27:34,560][105620] Updated weights for policy 1, policy_version 1111278 (0.0006) [2023-12-26 23:27:34,619][105620] Updated weights for policy 1, policy_version 1111288 (0.0006) [2023-12-26 23:27:34,719][105692] Updated weights for policy 0, policy_version 1110089 (0.0010) [2023-12-26 23:27:34,788][105692] Updated weights for policy 0, policy_version 1110099 (0.0009) [2023-12-26 23:27:34,847][105692] Updated weights for policy 0, policy_version 1110109 (0.0010) [2023-12-26 23:27:35,304][105620] Updated weights for policy 1, policy_version 1111298 (0.0010) [2023-12-26 23:27:35,371][105620] Updated weights for policy 1, policy_version 1111308 (0.0010) [2023-12-26 23:27:35,429][105620] Updated weights for policy 1, policy_version 1111318 (0.0008) [2023-12-26 23:27:35,430][105692] Updated weights for policy 0, policy_version 1110119 (0.0007) [2023-12-26 23:27:35,486][105692] Updated weights for policy 0, policy_version 1110129 (0.0005) [2023-12-26 23:27:35,550][105692] Updated weights for policy 0, policy_version 1110139 (0.0005) [2023-12-26 23:27:36,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 568770560. Throughput: 0: 9751.1, 1: 9997.4. Samples: 568761900. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:27:36,062][104569] Avg episode reward: [(0, '9170.026'), (1, '9072.190')] [2023-12-26 23:27:36,126][105620] Updated weights for policy 1, policy_version 1111328 (0.0007) [2023-12-26 23:27:36,140][105692] Updated weights for policy 0, policy_version 1110149 (0.0006) [2023-12-26 23:27:36,191][105620] Updated weights for policy 1, policy_version 1111338 (0.0007) [2023-12-26 23:27:36,197][105692] Updated weights for policy 0, policy_version 1110159 (0.0007) [2023-12-26 23:27:36,253][105620] Updated weights for policy 1, policy_version 1111348 (0.0009) [2023-12-26 23:27:36,263][105692] Updated weights for policy 0, policy_version 1110169 (0.0007) [2023-12-26 23:27:36,973][105620] Updated weights for policy 1, policy_version 1111358 (0.0008) [2023-12-26 23:27:37,016][105692] Updated weights for policy 0, policy_version 1110179 (0.0008) [2023-12-26 23:27:37,035][105620] Updated weights for policy 1, policy_version 1111368 (0.0008) [2023-12-26 23:27:37,078][105692] Updated weights for policy 0, policy_version 1110189 (0.0008) [2023-12-26 23:27:37,096][105620] Updated weights for policy 1, policy_version 1111378 (0.0007) [2023-12-26 23:27:37,140][105692] Updated weights for policy 0, policy_version 1110199 (0.0008) [2023-12-26 23:27:37,773][105692] Updated weights for policy 0, policy_version 1110209 (0.0008) [2023-12-26 23:27:37,837][105692] Updated weights for policy 0, policy_version 1110219 (0.0008) [2023-12-26 23:27:37,884][105692] Updated weights for policy 0, policy_version 1110229 (0.0009) [2023-12-26 23:27:37,926][105620] Updated weights for policy 1, policy_version 1111388 (0.0006) [2023-12-26 23:27:37,935][105692] Updated weights for policy 0, policy_version 1110239 (0.0008) [2023-12-26 23:27:37,983][105620] Updated weights for policy 1, policy_version 1111398 (0.0007) [2023-12-26 23:27:38,037][105620] Updated weights for policy 1, policy_version 1111408 (0.0008) [2023-12-26 23:27:38,666][105692] Updated weights for policy 0, policy_version 1110249 (0.0009) [2023-12-26 23:27:38,730][105692] Updated weights for policy 0, policy_version 1110259 (0.0007) [2023-12-26 23:27:38,786][105692] Updated weights for policy 0, policy_version 1110269 (0.0005) [2023-12-26 23:27:38,818][105620] Updated weights for policy 1, policy_version 1111418 (0.0007) [2023-12-26 23:27:38,887][105620] Updated weights for policy 1, policy_version 1111428 (0.0010) [2023-12-26 23:27:38,947][105620] Updated weights for policy 1, policy_version 1111438 (0.0008) [2023-12-26 23:27:38,999][105620] Updated weights for policy 1, policy_version 1111448 (0.0008) [2023-12-26 23:27:39,467][105692] Updated weights for policy 0, policy_version 1110279 (0.0009) [2023-12-26 23:27:39,520][105692] Updated weights for policy 0, policy_version 1110289 (0.0011) [2023-12-26 23:27:39,573][105692] Updated weights for policy 0, policy_version 1110299 (0.0011) [2023-12-26 23:27:39,731][105620] Updated weights for policy 1, policy_version 1111458 (0.0007) [2023-12-26 23:27:39,791][105620] Updated weights for policy 1, policy_version 1111468 (0.0008) [2023-12-26 23:27:39,851][105620] Updated weights for policy 1, policy_version 1111478 (0.0009) [2023-12-26 23:27:40,326][105692] Updated weights for policy 0, policy_version 1110309 (0.0010) [2023-12-26 23:27:40,387][105692] Updated weights for policy 0, policy_version 1110319 (0.0009) [2023-12-26 23:27:40,447][105692] Updated weights for policy 0, policy_version 1110329 (0.0010) [2023-12-26 23:27:40,627][105620] Updated weights for policy 1, policy_version 1111488 (0.0006) [2023-12-26 23:27:40,690][105620] Updated weights for policy 1, policy_version 1111498 (0.0005) [2023-12-26 23:27:40,753][105620] Updated weights for policy 1, policy_version 1111508 (0.0005) [2023-12-26 23:27:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 568868864. Throughput: 0: 9680.8, 1: 9965.7. Samples: 568878984. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:27:41,063][104569] Avg episode reward: [(0, '9168.282'), (1, '8980.702')] [2023-12-26 23:27:41,237][105692] Updated weights for policy 0, policy_version 1110339 (0.0009) [2023-12-26 23:27:41,297][105692] Updated weights for policy 0, policy_version 1110349 (0.0009) [2023-12-26 23:27:41,332][105620] Updated weights for policy 1, policy_version 1111518 (0.0006) [2023-12-26 23:27:41,360][105692] Updated weights for policy 0, policy_version 1110359 (0.0007) [2023-12-26 23:27:41,397][105620] Updated weights for policy 1, policy_version 1111528 (0.0008) [2023-12-26 23:27:41,458][105620] Updated weights for policy 1, policy_version 1111538 (0.0008) [2023-12-26 23:27:42,119][105692] Updated weights for policy 0, policy_version 1110369 (0.0007) [2023-12-26 23:27:42,179][105692] Updated weights for policy 0, policy_version 1110379 (0.0008) [2023-12-26 23:27:42,193][105620] Updated weights for policy 1, policy_version 1111548 (0.0009) [2023-12-26 23:27:42,245][105692] Updated weights for policy 0, policy_version 1110389 (0.0007) [2023-12-26 23:27:42,258][105620] Updated weights for policy 1, policy_version 1111558 (0.0007) [2023-12-26 23:27:42,314][105692] Updated weights for policy 0, policy_version 1110399 (0.0009) [2023-12-26 23:27:42,321][105620] Updated weights for policy 1, policy_version 1111568 (0.0008) [2023-12-26 23:27:43,050][105620] Updated weights for policy 1, policy_version 1111578 (0.0009) [2023-12-26 23:27:43,095][105620] Updated weights for policy 1, policy_version 1111588 (0.0006) [2023-12-26 23:27:43,098][105692] Updated weights for policy 0, policy_version 1110409 (0.0008) [2023-12-26 23:27:43,140][105620] Updated weights for policy 1, policy_version 1111598 (0.0007) [2023-12-26 23:27:43,154][105692] Updated weights for policy 0, policy_version 1110419 (0.0008) [2023-12-26 23:27:43,192][105620] Updated weights for policy 1, policy_version 1111608 (0.0007) [2023-12-26 23:27:43,216][105692] Updated weights for policy 0, policy_version 1110429 (0.0007) [2023-12-26 23:27:43,862][105692] Updated weights for policy 0, policy_version 1110439 (0.0010) [2023-12-26 23:27:43,915][105692] Updated weights for policy 0, policy_version 1110449 (0.0010) [2023-12-26 23:27:43,970][105692] Updated weights for policy 0, policy_version 1110459 (0.0010) [2023-12-26 23:27:43,991][105620] Updated weights for policy 1, policy_version 1111618 (0.0007) [2023-12-26 23:27:44,056][105620] Updated weights for policy 1, policy_version 1111628 (0.0008) [2023-12-26 23:27:44,114][105620] Updated weights for policy 1, policy_version 1111638 (0.0008) [2023-12-26 23:27:44,692][105692] Updated weights for policy 0, policy_version 1110469 (0.0010) [2023-12-26 23:27:44,724][105620] Updated weights for policy 1, policy_version 1111648 (0.0010) [2023-12-26 23:27:44,744][105692] Updated weights for policy 0, policy_version 1110479 (0.0010) [2023-12-26 23:27:44,772][105620] Updated weights for policy 1, policy_version 1111658 (0.0010) [2023-12-26 23:27:44,801][105692] Updated weights for policy 0, policy_version 1110489 (0.0007) [2023-12-26 23:27:44,837][105620] Updated weights for policy 1, policy_version 1111668 (0.0007) [2023-12-26 23:27:45,477][105620] Updated weights for policy 1, policy_version 1111678 (0.0008) [2023-12-26 23:27:45,531][105692] Updated weights for policy 0, policy_version 1110499 (0.0007) [2023-12-26 23:27:45,533][105620] Updated weights for policy 1, policy_version 1111688 (0.0006) [2023-12-26 23:27:45,582][105692] Updated weights for policy 0, policy_version 1110509 (0.0010) [2023-12-26 23:27:45,584][105620] Updated weights for policy 1, policy_version 1111698 (0.0008) [2023-12-26 23:27:45,634][105692] Updated weights for policy 0, policy_version 1110519 (0.0010) [2023-12-26 23:27:46,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 568967168. Throughput: 0: 9607.9, 1: 9915.7. Samples: 568935064. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:27:46,063][104569] Avg episode reward: [(0, '9078.537'), (1, '8981.975')] [2023-12-26 23:27:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001110528_284336128.pth... [2023-12-26 23:27:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001111704_284631040.pth... [2023-12-26 23:27:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001109408_284049408.pth [2023-12-26 23:27:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001110552_284336128.pth [2023-12-26 23:27:46,136][105620] Updated weights for policy 1, policy_version 1111708 (0.0006) [2023-12-26 23:27:46,202][105620] Updated weights for policy 1, policy_version 1111718 (0.0006) [2023-12-26 23:27:46,256][105620] Updated weights for policy 1, policy_version 1111728 (0.0006) [2023-12-26 23:27:46,399][105692] Updated weights for policy 0, policy_version 1110529 (0.0010) [2023-12-26 23:27:46,458][105692] Updated weights for policy 0, policy_version 1110539 (0.0010) [2023-12-26 23:27:46,512][105692] Updated weights for policy 0, policy_version 1110549 (0.0010) [2023-12-26 23:27:46,572][105692] Updated weights for policy 0, policy_version 1110559 (0.0010) [2023-12-26 23:27:46,766][105620] Updated weights for policy 1, policy_version 1111738 (0.0007) [2023-12-26 23:27:46,817][105620] Updated weights for policy 1, policy_version 1111748 (0.0005) [2023-12-26 23:27:46,886][105620] Updated weights for policy 1, policy_version 1111758 (0.0005) [2023-12-26 23:27:46,950][105620] Updated weights for policy 1, policy_version 1111768 (0.0007) [2023-12-26 23:27:47,161][105692] Updated weights for policy 0, policy_version 1110569 (0.0008) [2023-12-26 23:27:47,223][105692] Updated weights for policy 0, policy_version 1110579 (0.0010) [2023-12-26 23:27:47,288][105692] Updated weights for policy 0, policy_version 1110589 (0.0011) [2023-12-26 23:27:47,531][105620] Updated weights for policy 1, policy_version 1111778 (0.0008) [2023-12-26 23:27:47,582][105620] Updated weights for policy 1, policy_version 1111788 (0.0008) [2023-12-26 23:27:47,636][105620] Updated weights for policy 1, policy_version 1111798 (0.0008) [2023-12-26 23:27:47,991][105692] Updated weights for policy 0, policy_version 1110599 (0.0010) [2023-12-26 23:27:48,053][105692] Updated weights for policy 0, policy_version 1110609 (0.0010) [2023-12-26 23:27:48,108][105692] Updated weights for policy 0, policy_version 1110619 (0.0010) [2023-12-26 23:27:48,260][105620] Updated weights for policy 1, policy_version 1111808 (0.0010) [2023-12-26 23:27:48,319][105620] Updated weights for policy 1, policy_version 1111818 (0.0010) [2023-12-26 23:27:48,389][105620] Updated weights for policy 1, policy_version 1111828 (0.0010) [2023-12-26 23:27:48,724][105692] Updated weights for policy 0, policy_version 1110629 (0.0010) [2023-12-26 23:27:48,770][105692] Updated weights for policy 0, policy_version 1110639 (0.0010) [2023-12-26 23:27:48,828][105692] Updated weights for policy 0, policy_version 1110649 (0.0010) [2023-12-26 23:27:49,145][105620] Updated weights for policy 1, policy_version 1111838 (0.0010) [2023-12-26 23:27:49,193][105620] Updated weights for policy 1, policy_version 1111848 (0.0010) [2023-12-26 23:27:49,255][105620] Updated weights for policy 1, policy_version 1111858 (0.0011) [2023-12-26 23:27:49,490][105692] Updated weights for policy 0, policy_version 1110659 (0.0010) [2023-12-26 23:27:49,554][105692] Updated weights for policy 0, policy_version 1110669 (0.0010) [2023-12-26 23:27:49,616][105692] Updated weights for policy 0, policy_version 1110679 (0.0010) [2023-12-26 23:27:50,037][105620] Updated weights for policy 1, policy_version 1111868 (0.0011) [2023-12-26 23:27:50,094][105620] Updated weights for policy 1, policy_version 1111878 (0.0009) [2023-12-26 23:27:50,143][105620] Updated weights for policy 1, policy_version 1111888 (0.0010) [2023-12-26 23:27:50,368][105692] Updated weights for policy 0, policy_version 1110689 (0.0010) [2023-12-26 23:27:50,439][105692] Updated weights for policy 0, policy_version 1110699 (0.0010) [2023-12-26 23:27:50,498][105692] Updated weights for policy 0, policy_version 1110709 (0.0010) [2023-12-26 23:27:50,556][105692] Updated weights for policy 0, policy_version 1110719 (0.0010) [2023-12-26 23:27:50,884][105620] Updated weights for policy 1, policy_version 1111898 (0.0011) [2023-12-26 23:27:50,938][105620] Updated weights for policy 1, policy_version 1111908 (0.0009) [2023-12-26 23:27:50,989][105620] Updated weights for policy 1, policy_version 1111918 (0.0005) [2023-12-26 23:27:51,045][105620] Updated weights for policy 1, policy_version 1111928 (0.0007) [2023-12-26 23:27:51,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 569073664. Throughput: 0: 9625.1, 1: 10002.5. Samples: 569060972. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:27:51,062][104569] Avg episode reward: [(0, '9261.810'), (1, '9164.871')] [2023-12-26 23:27:51,272][105692] Updated weights for policy 0, policy_version 1110729 (0.0009) [2023-12-26 23:27:51,329][105692] Updated weights for policy 0, policy_version 1110739 (0.0007) [2023-12-26 23:27:51,399][105692] Updated weights for policy 0, policy_version 1110749 (0.0008) [2023-12-26 23:27:51,836][105620] Updated weights for policy 1, policy_version 1111938 (0.0008) [2023-12-26 23:27:51,886][105620] Updated weights for policy 1, policy_version 1111948 (0.0007) [2023-12-26 23:27:51,942][105620] Updated weights for policy 1, policy_version 1111958 (0.0006) [2023-12-26 23:27:52,133][105692] Updated weights for policy 0, policy_version 1110759 (0.0009) [2023-12-26 23:27:52,192][105692] Updated weights for policy 0, policy_version 1110769 (0.0010) [2023-12-26 23:27:52,247][105692] Updated weights for policy 0, policy_version 1110779 (0.0010) [2023-12-26 23:27:52,625][105620] Updated weights for policy 1, policy_version 1111968 (0.0005) [2023-12-26 23:27:52,681][105620] Updated weights for policy 1, policy_version 1111978 (0.0007) [2023-12-26 23:27:52,740][105620] Updated weights for policy 1, policy_version 1111988 (0.0010) [2023-12-26 23:27:53,000][105692] Updated weights for policy 0, policy_version 1110789 (0.0007) [2023-12-26 23:27:53,059][105692] Updated weights for policy 0, policy_version 1110799 (0.0005) [2023-12-26 23:27:53,113][105692] Updated weights for policy 0, policy_version 1110809 (0.0005) [2023-12-26 23:27:53,397][105620] Updated weights for policy 1, policy_version 1111998 (0.0007) [2023-12-26 23:27:53,466][105620] Updated weights for policy 1, policy_version 1112008 (0.0005) [2023-12-26 23:27:53,532][105620] Updated weights for policy 1, policy_version 1112018 (0.0006) [2023-12-26 23:27:53,685][105692] Updated weights for policy 0, policy_version 1110819 (0.0005) [2023-12-26 23:27:53,740][105692] Updated weights for policy 0, policy_version 1110829 (0.0005) [2023-12-26 23:27:53,793][105692] Updated weights for policy 0, policy_version 1110839 (0.0005) [2023-12-26 23:27:54,010][105620] Updated weights for policy 1, policy_version 1112028 (0.0005) [2023-12-26 23:27:54,061][105620] Updated weights for policy 1, policy_version 1112038 (0.0005) [2023-12-26 23:27:54,118][105620] Updated weights for policy 1, policy_version 1112048 (0.0005) [2023-12-26 23:27:54,389][105692] Updated weights for policy 0, policy_version 1110849 (0.0006) [2023-12-26 23:27:54,443][105692] Updated weights for policy 0, policy_version 1110859 (0.0009) [2023-12-26 23:27:54,494][105692] Updated weights for policy 0, policy_version 1110869 (0.0009) [2023-12-26 23:27:54,545][105692] Updated weights for policy 0, policy_version 1110880 (0.0008) [2023-12-26 23:27:54,718][105620] Updated weights for policy 1, policy_version 1112058 (0.0006) [2023-12-26 23:27:54,776][105620] Updated weights for policy 1, policy_version 1112068 (0.0010) [2023-12-26 23:27:54,828][105620] Updated weights for policy 1, policy_version 1112078 (0.0005) [2023-12-26 23:27:54,874][105620] Updated weights for policy 1, policy_version 1112088 (0.0005) [2023-12-26 23:27:55,420][105692] Updated weights for policy 0, policy_version 1110890 (0.0008) [2023-12-26 23:27:55,423][105620] Updated weights for policy 1, policy_version 1112098 (0.0007) [2023-12-26 23:27:55,479][105692] Updated weights for policy 0, policy_version 1110900 (0.0008) [2023-12-26 23:27:55,482][105620] Updated weights for policy 1, policy_version 1112108 (0.0006) [2023-12-26 23:27:55,539][105692] Updated weights for policy 0, policy_version 1110910 (0.0008) [2023-12-26 23:27:55,541][105620] Updated weights for policy 1, policy_version 1112118 (0.0007) [2023-12-26 23:27:56,062][104569] Fps is (10 sec: 20480.7, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 569171968. Throughput: 0: 9669.2, 1: 10074.7. Samples: 569183892. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-12-26 23:27:56,062][104569] Avg episode reward: [(0, '9262.206'), (1, '9347.150')] [2023-12-26 23:27:56,134][105620] Updated weights for policy 1, policy_version 1112128 (0.0010) [2023-12-26 23:27:56,184][105620] Updated weights for policy 1, policy_version 1112138 (0.0009) [2023-12-26 23:27:56,235][105620] Updated weights for policy 1, policy_version 1112148 (0.0005) [2023-12-26 23:27:56,262][105692] Updated weights for policy 0, policy_version 1110920 (0.0009) [2023-12-26 23:27:56,313][105692] Updated weights for policy 0, policy_version 1110930 (0.0008) [2023-12-26 23:27:56,370][105692] Updated weights for policy 0, policy_version 1110940 (0.0008) [2023-12-26 23:27:56,870][105620] Updated weights for policy 1, policy_version 1112158 (0.0005) [2023-12-26 23:27:56,927][105620] Updated weights for policy 1, policy_version 1112168 (0.0005) [2023-12-26 23:27:56,980][105620] Updated weights for policy 1, policy_version 1112178 (0.0005) [2023-12-26 23:27:57,175][105692] Updated weights for policy 0, policy_version 1110950 (0.0008) [2023-12-26 23:27:57,225][105692] Updated weights for policy 0, policy_version 1110960 (0.0008) [2023-12-26 23:27:57,283][105692] Updated weights for policy 0, policy_version 1110970 (0.0008) [2023-12-26 23:27:57,619][105620] Updated weights for policy 1, policy_version 1112188 (0.0008) [2023-12-26 23:27:57,672][105620] Updated weights for policy 1, policy_version 1112198 (0.0005) [2023-12-26 23:27:57,731][105620] Updated weights for policy 1, policy_version 1112208 (0.0005) [2023-12-26 23:27:58,108][105692] Updated weights for policy 0, policy_version 1110980 (0.0007) [2023-12-26 23:27:58,162][105692] Updated weights for policy 0, policy_version 1110990 (0.0010) [2023-12-26 23:27:58,226][105692] Updated weights for policy 0, policy_version 1111000 (0.0008) [2023-12-26 23:27:58,337][105620] Updated weights for policy 1, policy_version 1112218 (0.0010) [2023-12-26 23:27:58,403][105620] Updated weights for policy 1, policy_version 1112228 (0.0007) [2023-12-26 23:27:58,469][105620] Updated weights for policy 1, policy_version 1112238 (0.0008) [2023-12-26 23:27:58,528][105620] Updated weights for policy 1, policy_version 1112248 (0.0008) [2023-12-26 23:27:59,038][105692] Updated weights for policy 0, policy_version 1111010 (0.0009) [2023-12-26 23:27:59,097][105692] Updated weights for policy 0, policy_version 1111020 (0.0011) [2023-12-26 23:27:59,156][105692] Updated weights for policy 0, policy_version 1111030 (0.0011) [2023-12-26 23:27:59,222][105692] Updated weights for policy 0, policy_version 1111040 (0.0009) [2023-12-26 23:27:59,413][105620] Updated weights for policy 1, policy_version 1112258 (0.0008) [2023-12-26 23:27:59,461][105620] Updated weights for policy 1, policy_version 1112268 (0.0008) [2023-12-26 23:27:59,509][105620] Updated weights for policy 1, policy_version 1112278 (0.0007) [2023-12-26 23:27:59,967][105692] Updated weights for policy 0, policy_version 1111050 (0.0007) [2023-12-26 23:28:00,035][105692] Updated weights for policy 0, policy_version 1111060 (0.0008) [2023-12-26 23:28:00,092][105692] Updated weights for policy 0, policy_version 1111070 (0.0009) [2023-12-26 23:28:00,291][105620] Updated weights for policy 1, policy_version 1112288 (0.0006) [2023-12-26 23:28:00,354][105620] Updated weights for policy 1, policy_version 1112298 (0.0007) [2023-12-26 23:28:00,414][105620] Updated weights for policy 1, policy_version 1112308 (0.0008) [2023-12-26 23:28:00,697][105692] Updated weights for policy 0, policy_version 1111080 (0.0008) [2023-12-26 23:28:00,752][105692] Updated weights for policy 0, policy_version 1111090 (0.0006) [2023-12-26 23:28:00,809][105692] Updated weights for policy 0, policy_version 1111100 (0.0006) [2023-12-26 23:28:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 569270272. Throughput: 0: 9657.3, 1: 10079.5. Samples: 569242044. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:28:01,062][104569] Avg episode reward: [(0, '8988.270'), (1, '9255.455')] [2023-12-26 23:28:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001111104_284483584.pth... [2023-12-26 23:28:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001112312_284786688.pth... [2023-12-26 23:28:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001109952_284188672.pth [2023-12-26 23:28:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001111160_284491776.pth [2023-12-26 23:28:01,140][105620] Updated weights for policy 1, policy_version 1112318 (0.0007) [2023-12-26 23:28:01,192][105620] Updated weights for policy 1, policy_version 1112328 (0.0007) [2023-12-26 23:28:01,285][105620] Updated weights for policy 1, policy_version 1112338 (0.0007) [2023-12-26 23:28:01,456][105692] Updated weights for policy 0, policy_version 1111110 (0.0009) [2023-12-26 23:28:01,504][105692] Updated weights for policy 0, policy_version 1111120 (0.0009) [2023-12-26 23:28:01,559][105692] Updated weights for policy 0, policy_version 1111130 (0.0010) [2023-12-26 23:28:01,913][105620] Updated weights for policy 1, policy_version 1112348 (0.0008) [2023-12-26 23:28:01,982][105620] Updated weights for policy 1, policy_version 1112358 (0.0006) [2023-12-26 23:28:02,043][105620] Updated weights for policy 1, policy_version 1112368 (0.0005) [2023-12-26 23:28:02,430][105692] Updated weights for policy 0, policy_version 1111140 (0.0009) [2023-12-26 23:28:02,488][105692] Updated weights for policy 0, policy_version 1111150 (0.0009) [2023-12-26 23:28:02,551][105692] Updated weights for policy 0, policy_version 1111160 (0.0008) [2023-12-26 23:28:02,655][105620] Updated weights for policy 1, policy_version 1112378 (0.0006) [2023-12-26 23:28:02,707][105620] Updated weights for policy 1, policy_version 1112388 (0.0006) [2023-12-26 23:28:02,770][105620] Updated weights for policy 1, policy_version 1112398 (0.0005) [2023-12-26 23:28:02,823][105620] Updated weights for policy 1, policy_version 1112408 (0.0005) [2023-12-26 23:28:03,362][105620] Updated weights for policy 1, policy_version 1112418 (0.0008) [2023-12-26 23:28:03,374][105692] Updated weights for policy 0, policy_version 1111170 (0.0008) [2023-12-26 23:28:03,408][105620] Updated weights for policy 1, policy_version 1112428 (0.0006) [2023-12-26 23:28:03,426][105692] Updated weights for policy 0, policy_version 1111180 (0.0008) [2023-12-26 23:28:03,461][105620] Updated weights for policy 1, policy_version 1112438 (0.0006) [2023-12-26 23:28:03,472][105692] Updated weights for policy 0, policy_version 1111190 (0.0006) [2023-12-26 23:28:03,518][105692] Updated weights for policy 0, policy_version 1111200 (0.0009) [2023-12-26 23:28:04,212][105620] Updated weights for policy 1, policy_version 1112448 (0.0009) [2023-12-26 23:28:04,271][105620] Updated weights for policy 1, policy_version 1112458 (0.0010) [2023-12-26 23:28:04,306][105692] Updated weights for policy 0, policy_version 1111210 (0.0008) [2023-12-26 23:28:04,335][105620] Updated weights for policy 1, policy_version 1112468 (0.0010) [2023-12-26 23:28:04,365][105692] Updated weights for policy 0, policy_version 1111220 (0.0006) [2023-12-26 23:28:04,429][105692] Updated weights for policy 0, policy_version 1111230 (0.0009) [2023-12-26 23:28:05,017][105620] Updated weights for policy 1, policy_version 1112478 (0.0010) [2023-12-26 23:28:05,072][105620] Updated weights for policy 1, policy_version 1112488 (0.0010) [2023-12-26 23:28:05,124][105620] Updated weights for policy 1, policy_version 1112498 (0.0010) [2023-12-26 23:28:05,211][105692] Updated weights for policy 0, policy_version 1111240 (0.0009) [2023-12-26 23:28:05,276][105692] Updated weights for policy 0, policy_version 1111250 (0.0008) [2023-12-26 23:28:05,343][105692] Updated weights for policy 0, policy_version 1111260 (0.0008) [2023-12-26 23:28:05,827][105620] Updated weights for policy 1, policy_version 1112508 (0.0008) [2023-12-26 23:28:05,886][105620] Updated weights for policy 1, policy_version 1112518 (0.0005) [2023-12-26 23:28:05,947][105620] Updated weights for policy 1, policy_version 1112528 (0.0005) [2023-12-26 23:28:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 569368576. Throughput: 0: 9655.1, 1: 10038.3. Samples: 569357904. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:28:06,062][104569] Avg episode reward: [(0, '9078.072'), (1, '9257.253')] [2023-12-26 23:28:06,092][105692] Updated weights for policy 0, policy_version 1111270 (0.0007) [2023-12-26 23:28:06,158][105692] Updated weights for policy 0, policy_version 1111280 (0.0009) [2023-12-26 23:28:06,225][105692] Updated weights for policy 0, policy_version 1111290 (0.0009) [2023-12-26 23:28:06,528][105620] Updated weights for policy 1, policy_version 1112538 (0.0007) [2023-12-26 23:28:06,581][105620] Updated weights for policy 1, policy_version 1112548 (0.0007) [2023-12-26 23:28:06,629][105620] Updated weights for policy 1, policy_version 1112558 (0.0009) [2023-12-26 23:28:06,679][105620] Updated weights for policy 1, policy_version 1112568 (0.0007) [2023-12-26 23:28:07,057][105692] Updated weights for policy 0, policy_version 1111300 (0.0009) [2023-12-26 23:28:07,115][105692] Updated weights for policy 0, policy_version 1111310 (0.0010) [2023-12-26 23:28:07,173][105692] Updated weights for policy 0, policy_version 1111321 (0.0009) [2023-12-26 23:28:07,324][105620] Updated weights for policy 1, policy_version 1112578 (0.0008) [2023-12-26 23:28:07,386][105620] Updated weights for policy 1, policy_version 1112588 (0.0008) [2023-12-26 23:28:07,445][105620] Updated weights for policy 1, policy_version 1112598 (0.0010) [2023-12-26 23:28:07,807][105692] Updated weights for policy 0, policy_version 1111331 (0.0008) [2023-12-26 23:28:07,856][105692] Updated weights for policy 0, policy_version 1111341 (0.0010) [2023-12-26 23:28:07,903][105692] Updated weights for policy 0, policy_version 1111351 (0.0007) [2023-12-26 23:28:08,251][105620] Updated weights for policy 1, policy_version 1112608 (0.0006) [2023-12-26 23:28:08,304][105620] Updated weights for policy 1, policy_version 1112618 (0.0005) [2023-12-26 23:28:08,358][105620] Updated weights for policy 1, policy_version 1112628 (0.0007) [2023-12-26 23:28:08,667][105692] Updated weights for policy 0, policy_version 1111361 (0.0006) [2023-12-26 23:28:08,732][105692] Updated weights for policy 0, policy_version 1111371 (0.0009) [2023-12-26 23:28:08,785][105692] Updated weights for policy 0, policy_version 1111381 (0.0009) [2023-12-26 23:28:08,848][105692] Updated weights for policy 0, policy_version 1111391 (0.0009) [2023-12-26 23:28:09,093][105620] Updated weights for policy 1, policy_version 1112638 (0.0009) [2023-12-26 23:28:09,148][105620] Updated weights for policy 1, policy_version 1112648 (0.0009) [2023-12-26 23:28:09,200][105620] Updated weights for policy 1, policy_version 1112658 (0.0009) [2023-12-26 23:28:09,630][105692] Updated weights for policy 0, policy_version 1111401 (0.0011) [2023-12-26 23:28:09,685][105692] Updated weights for policy 0, policy_version 1111411 (0.0010) [2023-12-26 23:28:09,749][105692] Updated weights for policy 0, policy_version 1111421 (0.0009) [2023-12-26 23:28:09,995][105620] Updated weights for policy 1, policy_version 1112668 (0.0009) [2023-12-26 23:28:10,051][105620] Updated weights for policy 1, policy_version 1112678 (0.0006) [2023-12-26 23:28:10,115][105620] Updated weights for policy 1, policy_version 1112688 (0.0008) [2023-12-26 23:28:10,532][105692] Updated weights for policy 0, policy_version 1111431 (0.0008) [2023-12-26 23:28:10,594][105692] Updated weights for policy 0, policy_version 1111441 (0.0009) [2023-12-26 23:28:10,653][105692] Updated weights for policy 0, policy_version 1111451 (0.0008) [2023-12-26 23:28:10,857][105620] Updated weights for policy 1, policy_version 1112698 (0.0009) [2023-12-26 23:28:10,908][105620] Updated weights for policy 1, policy_version 1112708 (0.0005) [2023-12-26 23:28:10,966][105620] Updated weights for policy 1, policy_version 1112718 (0.0009) [2023-12-26 23:28:11,025][105620] Updated weights for policy 1, policy_version 1112728 (0.0011) [2023-12-26 23:28:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 569466880. Throughput: 0: 9609.4, 1: 10138.0. Samples: 569471840. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:28:11,062][104569] Avg episode reward: [(0, '9168.317'), (1, '9259.194')] [2023-12-26 23:28:11,466][105692] Updated weights for policy 0, policy_version 1111461 (0.0008) [2023-12-26 23:28:11,521][105692] Updated weights for policy 0, policy_version 1111471 (0.0009) [2023-12-26 23:28:11,573][105692] Updated weights for policy 0, policy_version 1111481 (0.0009) [2023-12-26 23:28:11,785][105620] Updated weights for policy 1, policy_version 1112738 (0.0008) [2023-12-26 23:28:11,853][105620] Updated weights for policy 1, policy_version 1112748 (0.0008) [2023-12-26 23:28:11,920][105620] Updated weights for policy 1, policy_version 1112758 (0.0008) [2023-12-26 23:28:12,410][105692] Updated weights for policy 0, policy_version 1111491 (0.0009) [2023-12-26 23:28:12,470][105692] Updated weights for policy 0, policy_version 1111501 (0.0011) [2023-12-26 23:28:12,536][105692] Updated weights for policy 0, policy_version 1111511 (0.0011) [2023-12-26 23:28:12,656][105620] Updated weights for policy 1, policy_version 1112768 (0.0006) [2023-12-26 23:28:12,724][105620] Updated weights for policy 1, policy_version 1112778 (0.0008) [2023-12-26 23:28:12,776][105620] Updated weights for policy 1, policy_version 1112788 (0.0009) [2023-12-26 23:28:13,245][105692] Updated weights for policy 0, policy_version 1111521 (0.0011) [2023-12-26 23:28:13,300][105692] Updated weights for policy 0, policy_version 1111531 (0.0010) [2023-12-26 23:28:13,359][105692] Updated weights for policy 0, policy_version 1111541 (0.0010) [2023-12-26 23:28:13,408][105620] Updated weights for policy 1, policy_version 1112798 (0.0008) [2023-12-26 23:28:13,415][105692] Updated weights for policy 0, policy_version 1111551 (0.0010) [2023-12-26 23:28:13,465][105620] Updated weights for policy 1, policy_version 1112808 (0.0008) [2023-12-26 23:28:13,529][105620] Updated weights for policy 1, policy_version 1112818 (0.0007) [2023-12-26 23:28:14,088][105620] Updated weights for policy 1, policy_version 1112828 (0.0005) [2023-12-26 23:28:14,140][105620] Updated weights for policy 1, policy_version 1112838 (0.0005) [2023-12-26 23:28:14,147][105692] Updated weights for policy 0, policy_version 1111561 (0.0010) [2023-12-26 23:28:14,189][105620] Updated weights for policy 1, policy_version 1112848 (0.0006) [2023-12-26 23:28:14,199][105692] Updated weights for policy 0, policy_version 1111571 (0.0010) [2023-12-26 23:28:14,253][105692] Updated weights for policy 0, policy_version 1111581 (0.0010) [2023-12-26 23:28:14,871][105692] Updated weights for policy 0, policy_version 1111591 (0.0011) [2023-12-26 23:28:14,934][105692] Updated weights for policy 0, policy_version 1111601 (0.0011) [2023-12-26 23:28:14,981][105620] Updated weights for policy 1, policy_version 1112858 (0.0006) [2023-12-26 23:28:14,983][105692] Updated weights for policy 0, policy_version 1111611 (0.0010) [2023-12-26 23:28:15,040][105620] Updated weights for policy 1, policy_version 1112868 (0.0009) [2023-12-26 23:28:15,099][105620] Updated weights for policy 1, policy_version 1112878 (0.0009) [2023-12-26 23:28:15,158][105620] Updated weights for policy 1, policy_version 1112888 (0.0010) [2023-12-26 23:28:15,614][105692] Updated weights for policy 0, policy_version 1111621 (0.0008) [2023-12-26 23:28:15,662][105692] Updated weights for policy 0, policy_version 1111631 (0.0010) [2023-12-26 23:28:15,714][105692] Updated weights for policy 0, policy_version 1111641 (0.0009) [2023-12-26 23:28:15,942][105620] Updated weights for policy 1, policy_version 1112898 (0.0005) [2023-12-26 23:28:15,989][105620] Updated weights for policy 1, policy_version 1112908 (0.0005) [2023-12-26 23:28:16,033][105620] Updated weights for policy 1, policy_version 1112918 (0.0006) [2023-12-26 23:28:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 569565184. Throughput: 0: 9599.6, 1: 10070.6. Samples: 569528976. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:28:16,062][104569] Avg episode reward: [(0, '9076.106'), (1, '9258.215')] [2023-12-26 23:28:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001112920_284942336.pth... [2023-12-26 23:28:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001111648_284622848.pth... [2023-12-26 23:28:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001111704_284631040.pth [2023-12-26 23:28:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001110528_284336128.pth [2023-12-26 23:28:16,315][105692] Updated weights for policy 0, policy_version 1111651 (0.0008) [2023-12-26 23:28:16,367][105692] Updated weights for policy 0, policy_version 1111661 (0.0007) [2023-12-26 23:28:16,430][105692] Updated weights for policy 0, policy_version 1111671 (0.0005) [2023-12-26 23:28:16,681][105620] Updated weights for policy 1, policy_version 1112928 (0.0010) [2023-12-26 23:28:16,740][105620] Updated weights for policy 1, policy_version 1112938 (0.0011) [2023-12-26 23:28:16,796][105620] Updated weights for policy 1, policy_version 1112948 (0.0011) [2023-12-26 23:28:17,074][105692] Updated weights for policy 0, policy_version 1111681 (0.0006) [2023-12-26 23:28:17,144][105692] Updated weights for policy 0, policy_version 1111691 (0.0009) [2023-12-26 23:28:17,210][105692] Updated weights for policy 0, policy_version 1111702 (0.0006) [2023-12-26 23:28:17,271][105692] Updated weights for policy 0, policy_version 1111712 (0.0009) [2023-12-26 23:28:17,480][105620] Updated weights for policy 1, policy_version 1112958 (0.0011) [2023-12-26 23:28:17,532][105620] Updated weights for policy 1, policy_version 1112968 (0.0009) [2023-12-26 23:28:17,577][105620] Updated weights for policy 1, policy_version 1112978 (0.0010) [2023-12-26 23:28:17,905][105692] Updated weights for policy 0, policy_version 1111722 (0.0011) [2023-12-26 23:28:17,960][105692] Updated weights for policy 0, policy_version 1111732 (0.0008) [2023-12-26 23:28:18,016][105692] Updated weights for policy 0, policy_version 1111742 (0.0005) [2023-12-26 23:28:18,282][105620] Updated weights for policy 1, policy_version 1112988 (0.0010) [2023-12-26 23:28:18,345][105620] Updated weights for policy 1, policy_version 1112998 (0.0011) [2023-12-26 23:28:18,415][105620] Updated weights for policy 1, policy_version 1113008 (0.0011) [2023-12-26 23:28:18,592][105692] Updated weights for policy 0, policy_version 1111752 (0.0005) [2023-12-26 23:28:18,656][105692] Updated weights for policy 0, policy_version 1111762 (0.0006) [2023-12-26 23:28:18,718][105692] Updated weights for policy 0, policy_version 1111772 (0.0007) [2023-12-26 23:28:19,105][105620] Updated weights for policy 1, policy_version 1113018 (0.0011) [2023-12-26 23:28:19,160][105620] Updated weights for policy 1, policy_version 1113028 (0.0010) [2023-12-26 23:28:19,212][105620] Updated weights for policy 1, policy_version 1113038 (0.0010) [2023-12-26 23:28:19,276][105620] Updated weights for policy 1, policy_version 1113048 (0.0011) [2023-12-26 23:28:19,413][105692] Updated weights for policy 0, policy_version 1111782 (0.0011) [2023-12-26 23:28:19,478][105692] Updated weights for policy 0, policy_version 1111792 (0.0010) [2023-12-26 23:28:19,538][105692] Updated weights for policy 0, policy_version 1111802 (0.0007) [2023-12-26 23:28:20,042][105620] Updated weights for policy 1, policy_version 1113058 (0.0008) [2023-12-26 23:28:20,111][105620] Updated weights for policy 1, policy_version 1113068 (0.0010) [2023-12-26 23:28:20,178][105620] Updated weights for policy 1, policy_version 1113078 (0.0011) [2023-12-26 23:28:20,262][105692] Updated weights for policy 0, policy_version 1111812 (0.0009) [2023-12-26 23:28:20,320][105692] Updated weights for policy 0, policy_version 1111822 (0.0009) [2023-12-26 23:28:20,383][105692] Updated weights for policy 0, policy_version 1111832 (0.0011) [2023-12-26 23:28:20,933][105620] Updated weights for policy 1, policy_version 1113088 (0.0008) [2023-12-26 23:28:21,002][105620] Updated weights for policy 1, policy_version 1113098 (0.0006) [2023-12-26 23:28:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 569655296. Throughput: 0: 9704.2, 1: 10074.1. Samples: 569651924. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:28:21,062][104569] Avg episode reward: [(0, '8896.169'), (1, '9255.675')] [2023-12-26 23:28:21,070][105620] Updated weights for policy 1, policy_version 1113108 (0.0009) [2023-12-26 23:28:21,123][105692] Updated weights for policy 0, policy_version 1111842 (0.0010) [2023-12-26 23:28:21,178][105692] Updated weights for policy 0, policy_version 1111852 (0.0011) [2023-12-26 23:28:21,235][105692] Updated weights for policy 0, policy_version 1111862 (0.0011) [2023-12-26 23:28:21,293][105692] Updated weights for policy 0, policy_version 1111872 (0.0011) [2023-12-26 23:28:21,745][105620] Updated weights for policy 1, policy_version 1113118 (0.0009) [2023-12-26 23:28:21,814][105620] Updated weights for policy 1, policy_version 1113128 (0.0011) [2023-12-26 23:28:21,877][105620] Updated weights for policy 1, policy_version 1113138 (0.0012) [2023-12-26 23:28:22,112][105692] Updated weights for policy 0, policy_version 1111882 (0.0011) [2023-12-26 23:28:22,178][105692] Updated weights for policy 0, policy_version 1111892 (0.0011) [2023-12-26 23:28:22,245][105692] Updated weights for policy 0, policy_version 1111902 (0.0011) [2023-12-26 23:28:22,637][105620] Updated weights for policy 1, policy_version 1113148 (0.0010) [2023-12-26 23:28:22,699][105620] Updated weights for policy 1, policy_version 1113158 (0.0010) [2023-12-26 23:28:22,764][105620] Updated weights for policy 1, policy_version 1113168 (0.0007) [2023-12-26 23:28:22,956][105692] Updated weights for policy 0, policy_version 1111912 (0.0010) [2023-12-26 23:28:23,007][105692] Updated weights for policy 0, policy_version 1111922 (0.0008) [2023-12-26 23:28:23,069][105692] Updated weights for policy 0, policy_version 1111932 (0.0008) [2023-12-26 23:28:23,339][105620] Updated weights for policy 1, policy_version 1113178 (0.0007) [2023-12-26 23:28:23,400][105620] Updated weights for policy 1, policy_version 1113188 (0.0005) [2023-12-26 23:28:23,459][105620] Updated weights for policy 1, policy_version 1113198 (0.0005) [2023-12-26 23:28:23,507][105620] Updated weights for policy 1, policy_version 1113208 (0.0005) [2023-12-26 23:28:23,992][105692] Updated weights for policy 0, policy_version 1111942 (0.0008) [2023-12-26 23:28:24,012][105620] Updated weights for policy 1, policy_version 1113218 (0.0005) [2023-12-26 23:28:24,048][105692] Updated weights for policy 0, policy_version 1111952 (0.0010) [2023-12-26 23:28:24,078][105620] Updated weights for policy 1, policy_version 1113228 (0.0005) [2023-12-26 23:28:24,109][105692] Updated weights for policy 0, policy_version 1111962 (0.0009) [2023-12-26 23:28:24,142][105620] Updated weights for policy 1, policy_version 1113238 (0.0005) [2023-12-26 23:28:24,663][105620] Updated weights for policy 1, policy_version 1113248 (0.0005) [2023-12-26 23:28:24,713][105620] Updated weights for policy 1, policy_version 1113258 (0.0005) [2023-12-26 23:28:24,766][105620] Updated weights for policy 1, policy_version 1113268 (0.0005) [2023-12-26 23:28:24,803][105692] Updated weights for policy 0, policy_version 1111972 (0.0008) [2023-12-26 23:28:24,855][105692] Updated weights for policy 0, policy_version 1111982 (0.0009) [2023-12-26 23:28:24,910][105692] Updated weights for policy 0, policy_version 1111992 (0.0010) [2023-12-26 23:28:25,402][105620] Updated weights for policy 1, policy_version 1113278 (0.0009) [2023-12-26 23:28:25,450][105620] Updated weights for policy 1, policy_version 1113288 (0.0010) [2023-12-26 23:28:25,505][105620] Updated weights for policy 1, policy_version 1113298 (0.0010) [2023-12-26 23:28:25,660][105692] Updated weights for policy 0, policy_version 1112002 (0.0008) [2023-12-26 23:28:25,705][105692] Updated weights for policy 0, policy_version 1112012 (0.0005) [2023-12-26 23:28:25,755][105692] Updated weights for policy 0, policy_version 1112022 (0.0007) [2023-12-26 23:28:25,806][105692] Updated weights for policy 0, policy_version 1112032 (0.0008) [2023-12-26 23:28:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.5, 300 sec: 19633.0). Total num frames: 569761792. Throughput: 0: 9579.7, 1: 10213.9. Samples: 569769692. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:28:26,062][104569] Avg episode reward: [(0, '8989.668'), (1, '9162.829')] [2023-12-26 23:28:26,262][105620] Updated weights for policy 1, policy_version 1113308 (0.0010) [2023-12-26 23:28:26,313][105620] Updated weights for policy 1, policy_version 1113318 (0.0010) [2023-12-26 23:28:26,361][105620] Updated weights for policy 1, policy_version 1113328 (0.0010) [2023-12-26 23:28:26,455][105692] Updated weights for policy 0, policy_version 1112042 (0.0008) [2023-12-26 23:28:26,513][105692] Updated weights for policy 0, policy_version 1112052 (0.0008) [2023-12-26 23:28:26,572][105692] Updated weights for policy 0, policy_version 1112062 (0.0008) [2023-12-26 23:28:27,130][105620] Updated weights for policy 1, policy_version 1113338 (0.0010) [2023-12-26 23:28:27,186][105620] Updated weights for policy 1, policy_version 1113348 (0.0006) [2023-12-26 23:28:27,237][105620] Updated weights for policy 1, policy_version 1113358 (0.0005) [2023-12-26 23:28:27,283][105620] Updated weights for policy 1, policy_version 1113368 (0.0006) [2023-12-26 23:28:27,342][105692] Updated weights for policy 0, policy_version 1112072 (0.0008) [2023-12-26 23:28:27,403][105692] Updated weights for policy 0, policy_version 1112082 (0.0007) [2023-12-26 23:28:27,458][105692] Updated weights for policy 0, policy_version 1112092 (0.0008) [2023-12-26 23:28:27,990][105620] Updated weights for policy 1, policy_version 1113378 (0.0011) [2023-12-26 23:28:28,042][105620] Updated weights for policy 1, policy_version 1113388 (0.0011) [2023-12-26 23:28:28,092][105620] Updated weights for policy 1, policy_version 1113398 (0.0010) [2023-12-26 23:28:28,204][105692] Updated weights for policy 0, policy_version 1112102 (0.0009) [2023-12-26 23:28:28,251][105692] Updated weights for policy 0, policy_version 1112112 (0.0010) [2023-12-26 23:28:28,295][105692] Updated weights for policy 0, policy_version 1112122 (0.0010) [2023-12-26 23:28:28,844][105620] Updated weights for policy 1, policy_version 1113408 (0.0010) [2023-12-26 23:28:28,900][105620] Updated weights for policy 1, policy_version 1113418 (0.0011) [2023-12-26 23:28:28,952][105620] Updated weights for policy 1, policy_version 1113428 (0.0009) [2023-12-26 23:28:29,048][105692] Updated weights for policy 0, policy_version 1112132 (0.0008) [2023-12-26 23:28:29,113][105692] Updated weights for policy 0, policy_version 1112142 (0.0007) [2023-12-26 23:28:29,174][105692] Updated weights for policy 0, policy_version 1112152 (0.0008) [2023-12-26 23:28:29,727][105620] Updated weights for policy 1, policy_version 1113438 (0.0008) [2023-12-26 23:28:29,784][105620] Updated weights for policy 1, policy_version 1113448 (0.0008) [2023-12-26 23:28:29,841][105620] Updated weights for policy 1, policy_version 1113458 (0.0009) [2023-12-26 23:28:29,901][105692] Updated weights for policy 0, policy_version 1112162 (0.0008) [2023-12-26 23:28:29,964][105692] Updated weights for policy 0, policy_version 1112172 (0.0008) [2023-12-26 23:28:30,023][105692] Updated weights for policy 0, policy_version 1112182 (0.0008) [2023-12-26 23:28:30,081][105692] Updated weights for policy 0, policy_version 1112192 (0.0008) [2023-12-26 23:28:30,495][105620] Updated weights for policy 1, policy_version 1113468 (0.0009) [2023-12-26 23:28:30,545][105620] Updated weights for policy 1, policy_version 1113478 (0.0009) [2023-12-26 23:28:30,596][105620] Updated weights for policy 1, policy_version 1113488 (0.0009) [2023-12-26 23:28:30,834][105692] Updated weights for policy 0, policy_version 1112202 (0.0011) [2023-12-26 23:28:30,889][105692] Updated weights for policy 0, policy_version 1112212 (0.0010) [2023-12-26 23:28:30,944][105692] Updated weights for policy 0, policy_version 1112222 (0.0010) [2023-12-26 23:28:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 569860096. Throughput: 0: 9605.7, 1: 10224.3. Samples: 569827408. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:28:31,062][104569] Avg episode reward: [(0, '9170.686'), (1, '9070.407')] [2023-12-26 23:28:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001112224_284770304.pth... [2023-12-26 23:28:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001113496_285089792.pth... [2023-12-26 23:28:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001111104_284483584.pth [2023-12-26 23:28:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001112312_284786688.pth [2023-12-26 23:28:31,336][105620] Updated weights for policy 1, policy_version 1113498 (0.0009) [2023-12-26 23:28:31,400][105620] Updated weights for policy 1, policy_version 1113508 (0.0009) [2023-12-26 23:28:31,458][105620] Updated weights for policy 1, policy_version 1113518 (0.0008) [2023-12-26 23:28:31,514][105620] Updated weights for policy 1, policy_version 1113528 (0.0008) [2023-12-26 23:28:31,696][105692] Updated weights for policy 0, policy_version 1112232 (0.0006) [2023-12-26 23:28:31,758][105692] Updated weights for policy 0, policy_version 1112242 (0.0010) [2023-12-26 23:28:31,806][105692] Updated weights for policy 0, policy_version 1112252 (0.0010) [2023-12-26 23:28:32,319][105620] Updated weights for policy 1, policy_version 1113538 (0.0008) [2023-12-26 23:28:32,387][105620] Updated weights for policy 1, policy_version 1113548 (0.0008) [2023-12-26 23:28:32,450][105620] Updated weights for policy 1, policy_version 1113558 (0.0009) [2023-12-26 23:28:32,466][105692] Updated weights for policy 0, policy_version 1112262 (0.0011) [2023-12-26 23:28:32,525][105692] Updated weights for policy 0, policy_version 1112272 (0.0009) [2023-12-26 23:28:32,586][105692] Updated weights for policy 0, policy_version 1112282 (0.0008) [2023-12-26 23:28:33,155][105620] Updated weights for policy 1, policy_version 1113568 (0.0009) [2023-12-26 23:28:33,209][105620] Updated weights for policy 1, policy_version 1113578 (0.0006) [2023-12-26 23:28:33,274][105620] Updated weights for policy 1, policy_version 1113588 (0.0005) [2023-12-26 23:28:33,289][105692] Updated weights for policy 0, policy_version 1112292 (0.0008) [2023-12-26 23:28:33,343][105692] Updated weights for policy 0, policy_version 1112302 (0.0006) [2023-12-26 23:28:33,413][105692] Updated weights for policy 0, policy_version 1112312 (0.0005) [2023-12-26 23:28:33,808][105620] Updated weights for policy 1, policy_version 1113598 (0.0008) [2023-12-26 23:28:33,856][105620] Updated weights for policy 1, policy_version 1113608 (0.0010) [2023-12-26 23:28:33,903][105620] Updated weights for policy 1, policy_version 1113618 (0.0010) [2023-12-26 23:28:34,117][105692] Updated weights for policy 0, policy_version 1112322 (0.0009) [2023-12-26 23:28:34,187][105692] Updated weights for policy 0, policy_version 1112332 (0.0008) [2023-12-26 23:28:34,238][105692] Updated weights for policy 0, policy_version 1112342 (0.0006) [2023-12-26 23:28:34,295][105692] Updated weights for policy 0, policy_version 1112352 (0.0006) [2023-12-26 23:28:34,667][105620] Updated weights for policy 1, policy_version 1113628 (0.0006) [2023-12-26 23:28:34,726][105620] Updated weights for policy 1, policy_version 1113638 (0.0009) [2023-12-26 23:28:34,789][105620] Updated weights for policy 1, policy_version 1113648 (0.0008) [2023-12-26 23:28:34,891][105692] Updated weights for policy 0, policy_version 1112362 (0.0006) [2023-12-26 23:28:34,952][105692] Updated weights for policy 0, policy_version 1112372 (0.0008) [2023-12-26 23:28:35,003][105692] Updated weights for policy 0, policy_version 1112382 (0.0006) [2023-12-26 23:28:35,541][105692] Updated weights for policy 0, policy_version 1112392 (0.0005) [2023-12-26 23:28:35,551][105620] Updated weights for policy 1, policy_version 1113658 (0.0009) [2023-12-26 23:28:35,596][105692] Updated weights for policy 0, policy_version 1112402 (0.0005) [2023-12-26 23:28:35,615][105620] Updated weights for policy 1, policy_version 1113668 (0.0008) [2023-12-26 23:28:35,649][105692] Updated weights for policy 0, policy_version 1112412 (0.0005) [2023-12-26 23:28:35,687][105620] Updated weights for policy 1, policy_version 1113678 (0.0009) [2023-12-26 23:28:35,755][105620] Updated weights for policy 1, policy_version 1113688 (0.0007) [2023-12-26 23:28:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 569958400. Throughput: 0: 9562.5, 1: 10085.9. Samples: 569945148. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:28:36,062][104569] Avg episode reward: [(0, '9169.641'), (1, '9253.834')] [2023-12-26 23:28:36,365][105620] Updated weights for policy 1, policy_version 1113698 (0.0007) [2023-12-26 23:28:36,405][105692] Updated weights for policy 0, policy_version 1112422 (0.0007) [2023-12-26 23:28:36,430][105620] Updated weights for policy 1, policy_version 1113708 (0.0006) [2023-12-26 23:28:36,469][105692] Updated weights for policy 0, policy_version 1112432 (0.0008) [2023-12-26 23:28:36,493][105620] Updated weights for policy 1, policy_version 1113718 (0.0009) [2023-12-26 23:28:36,531][105692] Updated weights for policy 0, policy_version 1112442 (0.0009) [2023-12-26 23:28:37,099][105692] Updated weights for policy 0, policy_version 1112452 (0.0009) [2023-12-26 23:28:37,145][105692] Updated weights for policy 0, policy_version 1112462 (0.0008) [2023-12-26 23:28:37,176][105620] Updated weights for policy 1, policy_version 1113728 (0.0009) [2023-12-26 23:28:37,191][105692] Updated weights for policy 0, policy_version 1112472 (0.0006) [2023-12-26 23:28:37,237][105620] Updated weights for policy 1, policy_version 1113738 (0.0008) [2023-12-26 23:28:37,296][105620] Updated weights for policy 1, policy_version 1113748 (0.0008) [2023-12-26 23:28:37,841][105692] Updated weights for policy 0, policy_version 1112482 (0.0010) [2023-12-26 23:28:37,908][105692] Updated weights for policy 0, policy_version 1112492 (0.0006) [2023-12-26 23:28:37,972][105692] Updated weights for policy 0, policy_version 1112502 (0.0010) [2023-12-26 23:28:38,032][105692] Updated weights for policy 0, policy_version 1112512 (0.0011) [2023-12-26 23:28:38,106][105620] Updated weights for policy 1, policy_version 1113758 (0.0007) [2023-12-26 23:28:38,178][105620] Updated weights for policy 1, policy_version 1113768 (0.0008) [2023-12-26 23:28:38,251][105620] Updated weights for policy 1, policy_version 1113778 (0.0011) [2023-12-26 23:28:38,669][105692] Updated weights for policy 0, policy_version 1112522 (0.0010) [2023-12-26 23:28:38,714][105692] Updated weights for policy 0, policy_version 1112532 (0.0010) [2023-12-26 23:28:38,765][105692] Updated weights for policy 0, policy_version 1112542 (0.0010) [2023-12-26 23:28:38,974][105620] Updated weights for policy 1, policy_version 1113788 (0.0009) [2023-12-26 23:28:39,047][105620] Updated weights for policy 1, policy_version 1113798 (0.0006) [2023-12-26 23:28:39,112][105620] Updated weights for policy 1, policy_version 1113808 (0.0011) [2023-12-26 23:28:39,570][105692] Updated weights for policy 0, policy_version 1112552 (0.0011) [2023-12-26 23:28:39,636][105692] Updated weights for policy 0, policy_version 1112562 (0.0010) [2023-12-26 23:28:39,692][105692] Updated weights for policy 0, policy_version 1112572 (0.0011) [2023-12-26 23:28:39,852][105620] Updated weights for policy 1, policy_version 1113818 (0.0011) [2023-12-26 23:28:39,922][105620] Updated weights for policy 1, policy_version 1113828 (0.0011) [2023-12-26 23:28:39,993][105620] Updated weights for policy 1, policy_version 1113838 (0.0011) [2023-12-26 23:28:40,061][105620] Updated weights for policy 1, policy_version 1113848 (0.0009) [2023-12-26 23:28:40,422][105692] Updated weights for policy 0, policy_version 1112582 (0.0008) [2023-12-26 23:28:40,489][105692] Updated weights for policy 0, policy_version 1112592 (0.0006) [2023-12-26 23:28:40,549][105692] Updated weights for policy 0, policy_version 1112602 (0.0005) [2023-12-26 23:28:40,769][105620] Updated weights for policy 1, policy_version 1113858 (0.0011) [2023-12-26 23:28:40,829][105620] Updated weights for policy 1, policy_version 1113868 (0.0010) [2023-12-26 23:28:40,893][105620] Updated weights for policy 1, policy_version 1113878 (0.0010) [2023-12-26 23:28:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 570056704. Throughput: 0: 9655.6, 1: 9908.3. Samples: 570064268. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:28:41,062][104569] Avg episode reward: [(0, '9078.524'), (1, '9346.983')] [2023-12-26 23:28:41,131][105692] Updated weights for policy 0, policy_version 1112612 (0.0007) [2023-12-26 23:28:41,183][105692] Updated weights for policy 0, policy_version 1112622 (0.0010) [2023-12-26 23:28:41,243][105692] Updated weights for policy 0, policy_version 1112632 (0.0011) [2023-12-26 23:28:41,669][105620] Updated weights for policy 1, policy_version 1113888 (0.0008) [2023-12-26 23:28:41,732][105620] Updated weights for policy 1, policy_version 1113898 (0.0008) [2023-12-26 23:28:41,794][105620] Updated weights for policy 1, policy_version 1113908 (0.0007) [2023-12-26 23:28:42,066][105692] Updated weights for policy 0, policy_version 1112643 (0.0009) [2023-12-26 23:28:42,132][105692] Updated weights for policy 0, policy_version 1112653 (0.0008) [2023-12-26 23:28:42,200][105692] Updated weights for policy 0, policy_version 1112663 (0.0010) [2023-12-26 23:28:42,533][105620] Updated weights for policy 1, policy_version 1113918 (0.0010) [2023-12-26 23:28:42,589][105620] Updated weights for policy 1, policy_version 1113928 (0.0009) [2023-12-26 23:28:42,645][105620] Updated weights for policy 1, policy_version 1113938 (0.0009) [2023-12-26 23:28:42,828][105692] Updated weights for policy 0, policy_version 1112673 (0.0009) [2023-12-26 23:28:42,875][105692] Updated weights for policy 0, policy_version 1112683 (0.0006) [2023-12-26 23:28:42,930][105692] Updated weights for policy 0, policy_version 1112693 (0.0011) [2023-12-26 23:28:42,990][105692] Updated weights for policy 0, policy_version 1112703 (0.0011) [2023-12-26 23:28:43,305][105620] Updated weights for policy 1, policy_version 1113948 (0.0006) [2023-12-26 23:28:43,353][105620] Updated weights for policy 1, policy_version 1113958 (0.0005) [2023-12-26 23:28:43,397][105620] Updated weights for policy 1, policy_version 1113968 (0.0005) [2023-12-26 23:28:43,658][105692] Updated weights for policy 0, policy_version 1112713 (0.0005) [2023-12-26 23:28:43,714][105692] Updated weights for policy 0, policy_version 1112723 (0.0005) [2023-12-26 23:28:43,777][105692] Updated weights for policy 0, policy_version 1112733 (0.0005) [2023-12-26 23:28:43,969][105620] Updated weights for policy 1, policy_version 1113978 (0.0007) [2023-12-26 23:28:44,023][105620] Updated weights for policy 1, policy_version 1113990 (0.0010) [2023-12-26 23:28:44,076][105620] Updated weights for policy 1, policy_version 1114000 (0.0009) [2023-12-26 23:28:44,341][105692] Updated weights for policy 0, policy_version 1112743 (0.0007) [2023-12-26 23:28:44,399][105692] Updated weights for policy 0, policy_version 1112753 (0.0009) [2023-12-26 23:28:44,459][105692] Updated weights for policy 0, policy_version 1112763 (0.0008) [2023-12-26 23:28:44,854][105620] Updated weights for policy 1, policy_version 1114010 (0.0009) [2023-12-26 23:28:44,906][105620] Updated weights for policy 1, policy_version 1114020 (0.0005) [2023-12-26 23:28:44,955][105620] Updated weights for policy 1, policy_version 1114030 (0.0005) [2023-12-26 23:28:45,016][105620] Updated weights for policy 1, policy_version 1114040 (0.0006) [2023-12-26 23:28:45,214][105692] Updated weights for policy 0, policy_version 1112773 (0.0008) [2023-12-26 23:28:45,273][105692] Updated weights for policy 0, policy_version 1112783 (0.0009) [2023-12-26 23:28:45,335][105692] Updated weights for policy 0, policy_version 1112793 (0.0009) [2023-12-26 23:28:45,682][105620] Updated weights for policy 1, policy_version 1114050 (0.0009) [2023-12-26 23:28:45,739][105620] Updated weights for policy 1, policy_version 1114061 (0.0010) [2023-12-26 23:28:45,797][105620] Updated weights for policy 1, policy_version 1114071 (0.0010) [2023-12-26 23:28:45,980][105692] Updated weights for policy 0, policy_version 1112803 (0.0008) [2023-12-26 23:28:46,031][105692] Updated weights for policy 0, policy_version 1112813 (0.0005) [2023-12-26 23:28:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 570155008. Throughput: 0: 9722.0, 1: 9902.4. Samples: 570125140. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:28:46,062][104569] Avg episode reward: [(0, '8897.729'), (1, '8982.140')] [2023-12-26 23:28:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001114072_285237248.pth... [2023-12-26 23:28:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001112920_284942336.pth [2023-12-26 23:28:46,081][105692] Updated weights for policy 0, policy_version 1112823 (0.0005) [2023-12-26 23:28:46,126][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001112832_284925952.pth... [2023-12-26 23:28:46,130][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001111648_284622848.pth [2023-12-26 23:28:46,598][105620] Updated weights for policy 1, policy_version 1114081 (0.0008) [2023-12-26 23:28:46,660][105620] Updated weights for policy 1, policy_version 1114091 (0.0008) [2023-12-26 23:28:46,721][105620] Updated weights for policy 1, policy_version 1114101 (0.0009) [2023-12-26 23:28:46,807][105692] Updated weights for policy 0, policy_version 1112833 (0.0008) [2023-12-26 23:28:46,857][105692] Updated weights for policy 0, policy_version 1112843 (0.0009) [2023-12-26 23:28:46,911][105692] Updated weights for policy 0, policy_version 1112853 (0.0009) [2023-12-26 23:28:46,965][105692] Updated weights for policy 0, policy_version 1112863 (0.0008) [2023-12-26 23:28:47,443][105620] Updated weights for policy 1, policy_version 1114111 (0.0009) [2023-12-26 23:28:47,494][105620] Updated weights for policy 1, policy_version 1114121 (0.0005) [2023-12-26 23:28:47,547][105620] Updated weights for policy 1, policy_version 1114131 (0.0005) [2023-12-26 23:28:47,737][105692] Updated weights for policy 0, policy_version 1112873 (0.0009) [2023-12-26 23:28:47,788][105692] Updated weights for policy 0, policy_version 1112883 (0.0009) [2023-12-26 23:28:47,842][105692] Updated weights for policy 0, policy_version 1112893 (0.0009) [2023-12-26 23:28:48,280][105620] Updated weights for policy 1, policy_version 1114141 (0.0008) [2023-12-26 23:28:48,348][105620] Updated weights for policy 1, policy_version 1114151 (0.0009) [2023-12-26 23:28:48,411][105620] Updated weights for policy 1, policy_version 1114161 (0.0007) [2023-12-26 23:28:48,540][105692] Updated weights for policy 0, policy_version 1112903 (0.0008) [2023-12-26 23:28:48,591][105692] Updated weights for policy 0, policy_version 1112913 (0.0005) [2023-12-26 23:28:48,648][105692] Updated weights for policy 0, policy_version 1112923 (0.0008) [2023-12-26 23:28:49,192][105620] Updated weights for policy 1, policy_version 1114171 (0.0009) [2023-12-26 23:28:49,262][105620] Updated weights for policy 1, policy_version 1114181 (0.0008) [2023-12-26 23:28:49,323][105692] Updated weights for policy 0, policy_version 1112933 (0.0009) [2023-12-26 23:28:49,328][105620] Updated weights for policy 1, policy_version 1114191 (0.0007) [2023-12-26 23:28:49,390][105692] Updated weights for policy 0, policy_version 1112943 (0.0008) [2023-12-26 23:28:49,438][105692] Updated weights for policy 0, policy_version 1112953 (0.0008) [2023-12-26 23:28:50,082][105620] Updated weights for policy 1, policy_version 1114201 (0.0007) [2023-12-26 23:28:50,145][105620] Updated weights for policy 1, policy_version 1114211 (0.0008) [2023-12-26 23:28:50,193][105692] Updated weights for policy 0, policy_version 1112963 (0.0008) [2023-12-26 23:28:50,203][105620] Updated weights for policy 1, policy_version 1114221 (0.0007) [2023-12-26 23:28:50,251][105692] Updated weights for policy 0, policy_version 1112973 (0.0008) [2023-12-26 23:28:50,253][105620] Updated weights for policy 1, policy_version 1114231 (0.0007) [2023-12-26 23:28:50,313][105692] Updated weights for policy 0, policy_version 1112983 (0.0007) [2023-12-26 23:28:50,973][105692] Updated weights for policy 0, policy_version 1112993 (0.0010) [2023-12-26 23:28:51,025][105620] Updated weights for policy 1, policy_version 1114241 (0.0008) [2023-12-26 23:28:51,031][105692] Updated weights for policy 0, policy_version 1113003 (0.0007) [2023-12-26 23:28:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 570245120. Throughput: 0: 9822.7, 1: 9812.6. Samples: 570241496. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:28:51,062][104569] Avg episode reward: [(0, '9078.825'), (1, '8983.026')] [2023-12-26 23:28:51,085][105620] Updated weights for policy 1, policy_version 1114251 (0.0007) [2023-12-26 23:28:51,095][105692] Updated weights for policy 0, policy_version 1113013 (0.0008) [2023-12-26 23:28:51,148][105620] Updated weights for policy 1, policy_version 1114261 (0.0007) [2023-12-26 23:28:51,161][105692] Updated weights for policy 0, policy_version 1113023 (0.0009) [2023-12-26 23:28:51,893][105692] Updated weights for policy 0, policy_version 1113033 (0.0009) [2023-12-26 23:28:51,941][105620] Updated weights for policy 1, policy_version 1114271 (0.0008) [2023-12-26 23:28:51,948][105692] Updated weights for policy 0, policy_version 1113043 (0.0006) [2023-12-26 23:28:51,993][105692] Updated weights for policy 0, policy_version 1113053 (0.0008) [2023-12-26 23:28:52,002][105620] Updated weights for policy 1, policy_version 1114281 (0.0007) [2023-12-26 23:28:52,063][105620] Updated weights for policy 1, policy_version 1114291 (0.0005) [2023-12-26 23:28:52,675][105620] Updated weights for policy 1, policy_version 1114301 (0.0008) [2023-12-26 23:28:52,722][105692] Updated weights for policy 0, policy_version 1113063 (0.0007) [2023-12-26 23:28:52,728][105620] Updated weights for policy 1, policy_version 1114311 (0.0008) [2023-12-26 23:28:52,774][105692] Updated weights for policy 0, policy_version 1113073 (0.0005) [2023-12-26 23:28:52,786][105620] Updated weights for policy 1, policy_version 1114321 (0.0009) [2023-12-26 23:28:52,833][105692] Updated weights for policy 0, policy_version 1113083 (0.0006) [2023-12-26 23:28:53,425][105692] Updated weights for policy 0, policy_version 1113093 (0.0005) [2023-12-26 23:28:53,476][105692] Updated weights for policy 0, policy_version 1113103 (0.0005) [2023-12-26 23:28:53,528][105692] Updated weights for policy 0, policy_version 1113113 (0.0005) [2023-12-26 23:28:53,623][105620] Updated weights for policy 1, policy_version 1114331 (0.0007) [2023-12-26 23:28:53,679][105620] Updated weights for policy 1, policy_version 1114341 (0.0005) [2023-12-26 23:28:53,741][105620] Updated weights for policy 1, policy_version 1114351 (0.0005) [2023-12-26 23:28:54,239][105692] Updated weights for policy 0, policy_version 1113123 (0.0008) [2023-12-26 23:28:54,290][105692] Updated weights for policy 0, policy_version 1113133 (0.0010) [2023-12-26 23:28:54,324][105620] Updated weights for policy 1, policy_version 1114361 (0.0006) [2023-12-26 23:28:54,352][105692] Updated weights for policy 0, policy_version 1113143 (0.0010) [2023-12-26 23:28:54,384][105620] Updated weights for policy 1, policy_version 1114371 (0.0010) [2023-12-26 23:28:54,444][105620] Updated weights for policy 1, policy_version 1114381 (0.0008) [2023-12-26 23:28:54,489][105620] Updated weights for policy 1, policy_version 1114391 (0.0008) [2023-12-26 23:28:55,071][105692] Updated weights for policy 0, policy_version 1113153 (0.0007) [2023-12-26 23:28:55,093][105620] Updated weights for policy 1, policy_version 1114401 (0.0007) [2023-12-26 23:28:55,137][105692] Updated weights for policy 0, policy_version 1113163 (0.0008) [2023-12-26 23:28:55,145][105620] Updated weights for policy 1, policy_version 1114411 (0.0007) [2023-12-26 23:28:55,197][105620] Updated weights for policy 1, policy_version 1114421 (0.0005) [2023-12-26 23:28:55,206][105692] Updated weights for policy 0, policy_version 1113173 (0.0008) [2023-12-26 23:28:55,258][105692] Updated weights for policy 0, policy_version 1113183 (0.0009) [2023-12-26 23:28:55,845][105620] Updated weights for policy 1, policy_version 1114431 (0.0006) [2023-12-26 23:28:55,897][105620] Updated weights for policy 1, policy_version 1114441 (0.0008) [2023-12-26 23:28:55,952][105620] Updated weights for policy 1, policy_version 1114451 (0.0007) [2023-12-26 23:28:55,970][105692] Updated weights for policy 0, policy_version 1113193 (0.0008) [2023-12-26 23:28:56,034][105692] Updated weights for policy 0, policy_version 1113203 (0.0008) [2023-12-26 23:28:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 570351616. Throughput: 0: 9930.8, 1: 9838.1. Samples: 570361436. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:28:56,062][104569] Avg episode reward: [(0, '9170.918'), (1, '9258.706')] [2023-12-26 23:28:56,104][105692] Updated weights for policy 0, policy_version 1113213 (0.0008) [2023-12-26 23:28:56,571][105620] Updated weights for policy 1, policy_version 1114461 (0.0009) [2023-12-26 23:28:56,625][105620] Updated weights for policy 1, policy_version 1114471 (0.0010) [2023-12-26 23:28:56,677][105620] Updated weights for policy 1, policy_version 1114481 (0.0010) [2023-12-26 23:28:56,733][105692] Updated weights for policy 0, policy_version 1113223 (0.0007) [2023-12-26 23:28:56,785][105692] Updated weights for policy 0, policy_version 1113233 (0.0005) [2023-12-26 23:28:56,837][105692] Updated weights for policy 0, policy_version 1113243 (0.0005) [2023-12-26 23:28:57,373][105620] Updated weights for policy 1, policy_version 1114491 (0.0009) [2023-12-26 23:28:57,395][105692] Updated weights for policy 0, policy_version 1113253 (0.0007) [2023-12-26 23:28:57,434][105620] Updated weights for policy 1, policy_version 1114501 (0.0005) [2023-12-26 23:28:57,444][105692] Updated weights for policy 0, policy_version 1113263 (0.0005) [2023-12-26 23:28:57,490][105620] Updated weights for policy 1, policy_version 1114511 (0.0005) [2023-12-26 23:28:57,493][105692] Updated weights for policy 0, policy_version 1113273 (0.0005) [2023-12-26 23:28:58,024][105620] Updated weights for policy 1, policy_version 1114521 (0.0006) [2023-12-26 23:28:58,078][105620] Updated weights for policy 1, policy_version 1114531 (0.0005) [2023-12-26 23:28:58,087][105692] Updated weights for policy 0, policy_version 1113283 (0.0007) [2023-12-26 23:28:58,124][105620] Updated weights for policy 1, policy_version 1114541 (0.0007) [2023-12-26 23:28:58,149][105692] Updated weights for policy 0, policy_version 1113293 (0.0008) [2023-12-26 23:28:58,186][105620] Updated weights for policy 1, policy_version 1114551 (0.0009) [2023-12-26 23:28:58,210][105692] Updated weights for policy 0, policy_version 1113303 (0.0011) [2023-12-26 23:28:58,846][105620] Updated weights for policy 1, policy_version 1114561 (0.0007) [2023-12-26 23:28:58,916][105620] Updated weights for policy 1, policy_version 1114571 (0.0007) [2023-12-26 23:28:58,991][105620] Updated weights for policy 1, policy_version 1114581 (0.0008) [2023-12-26 23:28:59,001][105692] Updated weights for policy 0, policy_version 1113313 (0.0011) [2023-12-26 23:28:59,062][105692] Updated weights for policy 0, policy_version 1113323 (0.0010) [2023-12-26 23:28:59,120][105692] Updated weights for policy 0, policy_version 1113333 (0.0009) [2023-12-26 23:28:59,185][105692] Updated weights for policy 0, policy_version 1113343 (0.0009) [2023-12-26 23:28:59,714][105620] Updated weights for policy 1, policy_version 1114591 (0.0006) [2023-12-26 23:28:59,764][105620] Updated weights for policy 1, policy_version 1114601 (0.0007) [2023-12-26 23:28:59,824][105620] Updated weights for policy 1, policy_version 1114611 (0.0007) [2023-12-26 23:28:59,980][105692] Updated weights for policy 0, policy_version 1113353 (0.0011) [2023-12-26 23:29:00,033][105692] Updated weights for policy 0, policy_version 1113363 (0.0011) [2023-12-26 23:29:00,039][105585] KL-divergence is very high: 110.4634 [2023-12-26 23:29:00,080][105585] KL-divergence is very high: 123.8209 [2023-12-26 23:29:00,087][105692] Updated weights for policy 0, policy_version 1113373 (0.0009) [2023-12-26 23:29:00,540][105620] Updated weights for policy 1, policy_version 1114621 (0.0010) [2023-12-26 23:29:00,604][105620] Updated weights for policy 1, policy_version 1114631 (0.0010) [2023-12-26 23:29:00,664][105620] Updated weights for policy 1, policy_version 1114641 (0.0010) [2023-12-26 23:29:00,765][105692] Updated weights for policy 0, policy_version 1113383 (0.0007) [2023-12-26 23:29:00,810][105692] Updated weights for policy 0, policy_version 1113393 (0.0005) [2023-12-26 23:29:00,857][105692] Updated weights for policy 0, policy_version 1113403 (0.0006) [2023-12-26 23:29:01,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 570458112. Throughput: 0: 10047.2, 1: 9894.8. Samples: 570426364. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:29:01,062][104569] Avg episode reward: [(0, '9172.084'), (1, '9349.521')] [2023-12-26 23:29:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001113408_285073408.pth... [2023-12-26 23:29:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001114648_285384704.pth... [2023-12-26 23:29:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001113496_285089792.pth [2023-12-26 23:29:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001112224_284770304.pth [2023-12-26 23:29:01,358][105620] Updated weights for policy 1, policy_version 1114651 (0.0010) [2023-12-26 23:29:01,426][105620] Updated weights for policy 1, policy_version 1114661 (0.0007) [2023-12-26 23:29:01,497][105620] Updated weights for policy 1, policy_version 1114671 (0.0010) [2023-12-26 23:29:01,540][105692] Updated weights for policy 0, policy_version 1113413 (0.0008) [2023-12-26 23:29:01,604][105692] Updated weights for policy 0, policy_version 1113423 (0.0009) [2023-12-26 23:29:01,668][105692] Updated weights for policy 0, policy_version 1113433 (0.0008) [2023-12-26 23:29:02,224][105620] Updated weights for policy 1, policy_version 1114681 (0.0011) [2023-12-26 23:29:02,286][105620] Updated weights for policy 1, policy_version 1114691 (0.0010) [2023-12-26 23:29:02,347][105620] Updated weights for policy 1, policy_version 1114701 (0.0008) [2023-12-26 23:29:02,390][105692] Updated weights for policy 0, policy_version 1113443 (0.0008) [2023-12-26 23:29:02,405][105620] Updated weights for policy 1, policy_version 1114711 (0.0008) [2023-12-26 23:29:02,442][105692] Updated weights for policy 0, policy_version 1113453 (0.0009) [2023-12-26 23:29:02,491][105692] Updated weights for policy 0, policy_version 1113463 (0.0010) [2023-12-26 23:29:03,118][105620] Updated weights for policy 1, policy_version 1114721 (0.0007) [2023-12-26 23:29:03,128][105692] Updated weights for policy 0, policy_version 1113473 (0.0010) [2023-12-26 23:29:03,179][105692] Updated weights for policy 0, policy_version 1113483 (0.0006) [2023-12-26 23:29:03,180][105620] Updated weights for policy 1, policy_version 1114731 (0.0006) [2023-12-26 23:29:03,231][105692] Updated weights for policy 0, policy_version 1113493 (0.0006) [2023-12-26 23:29:03,241][105620] Updated weights for policy 1, policy_version 1114741 (0.0008) [2023-12-26 23:29:03,290][105692] Updated weights for policy 0, policy_version 1113503 (0.0008) [2023-12-26 23:29:03,822][105692] Updated weights for policy 0, policy_version 1113513 (0.0005) [2023-12-26 23:29:03,884][105692] Updated weights for policy 0, policy_version 1113523 (0.0009) [2023-12-26 23:29:03,904][105620] Updated weights for policy 1, policy_version 1114751 (0.0007) [2023-12-26 23:29:03,936][105692] Updated weights for policy 0, policy_version 1113533 (0.0010) [2023-12-26 23:29:03,968][105620] Updated weights for policy 1, policy_version 1114761 (0.0008) [2023-12-26 23:29:04,026][105620] Updated weights for policy 1, policy_version 1114771 (0.0010) [2023-12-26 23:29:04,665][105692] Updated weights for policy 0, policy_version 1113543 (0.0010) [2023-12-26 23:29:04,710][105620] Updated weights for policy 1, policy_version 1114781 (0.0010) [2023-12-26 23:29:04,720][105692] Updated weights for policy 0, policy_version 1113553 (0.0010) [2023-12-26 23:29:04,758][105620] Updated weights for policy 1, policy_version 1114791 (0.0010) [2023-12-26 23:29:04,778][105692] Updated weights for policy 0, policy_version 1113563 (0.0010) [2023-12-26 23:29:04,802][105620] Updated weights for policy 1, policy_version 1114801 (0.0010) [2023-12-26 23:29:05,483][105692] Updated weights for policy 0, policy_version 1113573 (0.0009) [2023-12-26 23:29:05,548][105692] Updated weights for policy 0, policy_version 1113583 (0.0005) [2023-12-26 23:29:05,551][105620] Updated weights for policy 1, policy_version 1114811 (0.0009) [2023-12-26 23:29:05,602][105692] Updated weights for policy 0, policy_version 1113593 (0.0006) [2023-12-26 23:29:05,619][105620] Updated weights for policy 1, policy_version 1114821 (0.0009) [2023-12-26 23:29:05,688][105620] Updated weights for policy 1, policy_version 1114831 (0.0011) [2023-12-26 23:29:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 570556416. Throughput: 0: 9958.7, 1: 9882.8. Samples: 570544792. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:29:06,062][104569] Avg episode reward: [(0, '9263.235'), (1, '9256.997')] [2023-12-26 23:29:06,311][105692] Updated weights for policy 0, policy_version 1113603 (0.0008) [2023-12-26 23:29:06,379][105692] Updated weights for policy 0, policy_version 1113613 (0.0009) [2023-12-26 23:29:06,392][105620] Updated weights for policy 1, policy_version 1114841 (0.0010) [2023-12-26 23:29:06,439][105692] Updated weights for policy 0, policy_version 1113623 (0.0006) [2023-12-26 23:29:06,455][105620] Updated weights for policy 1, policy_version 1114851 (0.0009) [2023-12-26 23:29:06,519][105620] Updated weights for policy 1, policy_version 1114861 (0.0009) [2023-12-26 23:29:06,585][105620] Updated weights for policy 1, policy_version 1114871 (0.0009) [2023-12-26 23:29:07,210][105692] Updated weights for policy 0, policy_version 1113633 (0.0007) [2023-12-26 23:29:07,235][105620] Updated weights for policy 1, policy_version 1114881 (0.0011) [2023-12-26 23:29:07,277][105692] Updated weights for policy 0, policy_version 1113643 (0.0008) [2023-12-26 23:29:07,301][105620] Updated weights for policy 1, policy_version 1114891 (0.0010) [2023-12-26 23:29:07,348][105692] Updated weights for policy 0, policy_version 1113653 (0.0008) [2023-12-26 23:29:07,357][105620] Updated weights for policy 1, policy_version 1114901 (0.0010) [2023-12-26 23:29:07,415][105692] Updated weights for policy 0, policy_version 1113663 (0.0008) [2023-12-26 23:29:07,959][105620] Updated weights for policy 1, policy_version 1114911 (0.0007) [2023-12-26 23:29:08,006][105692] Updated weights for policy 0, policy_version 1113673 (0.0010) [2023-12-26 23:29:08,016][105620] Updated weights for policy 1, policy_version 1114921 (0.0005) [2023-12-26 23:29:08,062][105692] Updated weights for policy 0, policy_version 1113683 (0.0010) [2023-12-26 23:29:08,067][105620] Updated weights for policy 1, policy_version 1114931 (0.0008) [2023-12-26 23:29:08,117][105692] Updated weights for policy 0, policy_version 1113693 (0.0010) [2023-12-26 23:29:08,728][105620] Updated weights for policy 1, policy_version 1114941 (0.0010) [2023-12-26 23:29:08,796][105620] Updated weights for policy 1, policy_version 1114951 (0.0009) [2023-12-26 23:29:08,805][105692] Updated weights for policy 0, policy_version 1113703 (0.0007) [2023-12-26 23:29:08,853][105620] Updated weights for policy 1, policy_version 1114961 (0.0009) [2023-12-26 23:29:08,864][105692] Updated weights for policy 0, policy_version 1113713 (0.0005) [2023-12-26 23:29:08,927][105692] Updated weights for policy 0, policy_version 1113723 (0.0005) [2023-12-26 23:29:09,567][105692] Updated weights for policy 0, policy_version 1113733 (0.0008) [2023-12-26 23:29:09,630][105692] Updated weights for policy 0, policy_version 1113743 (0.0011) [2023-12-26 23:29:09,689][105692] Updated weights for policy 0, policy_version 1113753 (0.0005) [2023-12-26 23:29:09,788][105620] Updated weights for policy 1, policy_version 1114971 (0.0008) [2023-12-26 23:29:09,853][105620] Updated weights for policy 1, policy_version 1114981 (0.0009) [2023-12-26 23:29:09,907][105620] Updated weights for policy 1, policy_version 1114991 (0.0009) [2023-12-26 23:29:10,356][105692] Updated weights for policy 0, policy_version 1113763 (0.0006) [2023-12-26 23:29:10,422][105692] Updated weights for policy 0, policy_version 1113773 (0.0007) [2023-12-26 23:29:10,488][105692] Updated weights for policy 0, policy_version 1113783 (0.0011) [2023-12-26 23:29:10,704][105620] Updated weights for policy 1, policy_version 1115001 (0.0009) [2023-12-26 23:29:10,764][105620] Updated weights for policy 1, policy_version 1115011 (0.0011) [2023-12-26 23:29:10,824][105620] Updated weights for policy 1, policy_version 1115021 (0.0011) [2023-12-26 23:29:10,883][105620] Updated weights for policy 1, policy_version 1115031 (0.0010) [2023-12-26 23:29:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 570654720. Throughput: 0: 10076.1, 1: 9762.9. Samples: 570662448. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:29:11,062][104569] Avg episode reward: [(0, '9260.202'), (1, '9258.349')] [2023-12-26 23:29:11,224][105692] Updated weights for policy 0, policy_version 1113793 (0.0011) [2023-12-26 23:29:11,286][105692] Updated weights for policy 0, policy_version 1113803 (0.0011) [2023-12-26 23:29:11,360][105692] Updated weights for policy 0, policy_version 1113813 (0.0014) [2023-12-26 23:29:11,423][105692] Updated weights for policy 0, policy_version 1113823 (0.0008) [2023-12-26 23:29:11,677][105620] Updated weights for policy 1, policy_version 1115041 (0.0009) [2023-12-26 23:29:11,760][105620] Updated weights for policy 1, policy_version 1115051 (0.0008) [2023-12-26 23:29:11,820][105620] Updated weights for policy 1, policy_version 1115061 (0.0008) [2023-12-26 23:29:12,177][105692] Updated weights for policy 0, policy_version 1113833 (0.0007) [2023-12-26 23:29:12,246][105692] Updated weights for policy 0, policy_version 1113843 (0.0006) [2023-12-26 23:29:12,319][105692] Updated weights for policy 0, policy_version 1113853 (0.0006) [2023-12-26 23:29:12,533][105620] Updated weights for policy 1, policy_version 1115071 (0.0006) [2023-12-26 23:29:12,586][105620] Updated weights for policy 1, policy_version 1115081 (0.0006) [2023-12-26 23:29:12,643][105620] Updated weights for policy 1, policy_version 1115091 (0.0006) [2023-12-26 23:29:12,985][105692] Updated weights for policy 0, policy_version 1113863 (0.0006) [2023-12-26 23:29:13,043][105692] Updated weights for policy 0, policy_version 1113873 (0.0007) [2023-12-26 23:29:13,097][105692] Updated weights for policy 0, policy_version 1113883 (0.0009) [2023-12-26 23:29:13,394][105620] Updated weights for policy 1, policy_version 1115101 (0.0010) [2023-12-26 23:29:13,439][105620] Updated weights for policy 1, policy_version 1115111 (0.0010) [2023-12-26 23:29:13,491][105620] Updated weights for policy 1, policy_version 1115121 (0.0010) [2023-12-26 23:29:13,710][105692] Updated weights for policy 0, policy_version 1113893 (0.0008) [2023-12-26 23:29:13,766][105692] Updated weights for policy 0, policy_version 1113903 (0.0008) [2023-12-26 23:29:13,824][105692] Updated weights for policy 0, policy_version 1113913 (0.0008) [2023-12-26 23:29:14,243][105620] Updated weights for policy 1, policy_version 1115131 (0.0010) [2023-12-26 23:29:14,293][105620] Updated weights for policy 1, policy_version 1115141 (0.0006) [2023-12-26 23:29:14,360][105620] Updated weights for policy 1, policy_version 1115151 (0.0005) [2023-12-26 23:29:14,580][105692] Updated weights for policy 0, policy_version 1113923 (0.0007) [2023-12-26 23:29:14,626][105692] Updated weights for policy 0, policy_version 1113933 (0.0005) [2023-12-26 23:29:14,683][105692] Updated weights for policy 0, policy_version 1113943 (0.0006) [2023-12-26 23:29:15,026][105620] Updated weights for policy 1, policy_version 1115161 (0.0006) [2023-12-26 23:29:15,075][105620] Updated weights for policy 1, policy_version 1115171 (0.0010) [2023-12-26 23:29:15,136][105620] Updated weights for policy 1, policy_version 1115181 (0.0009) [2023-12-26 23:29:15,205][105620] Updated weights for policy 1, policy_version 1115191 (0.0008) [2023-12-26 23:29:15,441][105692] Updated weights for policy 0, policy_version 1113953 (0.0008) [2023-12-26 23:29:15,497][105692] Updated weights for policy 0, policy_version 1113963 (0.0010) [2023-12-26 23:29:15,545][105692] Updated weights for policy 0, policy_version 1113973 (0.0006) [2023-12-26 23:29:15,606][105692] Updated weights for policy 0, policy_version 1113983 (0.0008) [2023-12-26 23:29:15,903][105620] Updated weights for policy 1, policy_version 1115201 (0.0006) [2023-12-26 23:29:15,957][105620] Updated weights for policy 1, policy_version 1115211 (0.0005) [2023-12-26 23:29:16,001][105620] Updated weights for policy 1, policy_version 1115221 (0.0005) [2023-12-26 23:29:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 570753024. Throughput: 0: 10089.9, 1: 9742.5. Samples: 570719872. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:29:16,063][104569] Avg episode reward: [(0, '9259.745'), (1, '9261.683')] [2023-12-26 23:29:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001113984_285220864.pth... [2023-12-26 23:29:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001115224_285532160.pth... [2023-12-26 23:29:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001112832_284925952.pth [2023-12-26 23:29:16,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001114072_285237248.pth [2023-12-26 23:29:16,411][105692] Updated weights for policy 0, policy_version 1113993 (0.0009) [2023-12-26 23:29:16,474][105692] Updated weights for policy 0, policy_version 1114003 (0.0010) [2023-12-26 23:29:16,507][105620] Updated weights for policy 1, policy_version 1115231 (0.0005) [2023-12-26 23:29:16,526][105692] Updated weights for policy 0, policy_version 1114013 (0.0009) [2023-12-26 23:29:16,556][105620] Updated weights for policy 1, policy_version 1115241 (0.0005) [2023-12-26 23:29:16,601][105620] Updated weights for policy 1, policy_version 1115251 (0.0007) [2023-12-26 23:29:17,186][105692] Updated weights for policy 0, policy_version 1114023 (0.0007) [2023-12-26 23:29:17,200][105620] Updated weights for policy 1, policy_version 1115261 (0.0010) [2023-12-26 23:29:17,240][105692] Updated weights for policy 0, policy_version 1114033 (0.0005) [2023-12-26 23:29:17,255][105620] Updated weights for policy 1, policy_version 1115271 (0.0010) [2023-12-26 23:29:17,295][105692] Updated weights for policy 0, policy_version 1114043 (0.0005) [2023-12-26 23:29:17,314][105620] Updated weights for policy 1, policy_version 1115281 (0.0010) [2023-12-26 23:29:17,819][105692] Updated weights for policy 0, policy_version 1114053 (0.0005) [2023-12-26 23:29:17,871][105692] Updated weights for policy 0, policy_version 1114063 (0.0005) [2023-12-26 23:29:17,925][105692] Updated weights for policy 0, policy_version 1114073 (0.0006) [2023-12-26 23:29:17,950][105620] Updated weights for policy 1, policy_version 1115291 (0.0010) [2023-12-26 23:29:18,008][105620] Updated weights for policy 1, policy_version 1115301 (0.0010) [2023-12-26 23:29:18,070][105620] Updated weights for policy 1, policy_version 1115311 (0.0010) [2023-12-26 23:29:18,507][105692] Updated weights for policy 0, policy_version 1114083 (0.0006) [2023-12-26 23:29:18,560][105692] Updated weights for policy 0, policy_version 1114093 (0.0008) [2023-12-26 23:29:18,626][105692] Updated weights for policy 0, policy_version 1114103 (0.0008) [2023-12-26 23:29:18,804][105620] Updated weights for policy 1, policy_version 1115321 (0.0009) [2023-12-26 23:29:18,874][105620] Updated weights for policy 1, policy_version 1115331 (0.0010) [2023-12-26 23:29:18,929][105620] Updated weights for policy 1, policy_version 1115341 (0.0011) [2023-12-26 23:29:18,981][105620] Updated weights for policy 1, policy_version 1115351 (0.0010) [2023-12-26 23:29:19,229][105692] Updated weights for policy 0, policy_version 1114113 (0.0009) [2023-12-26 23:29:19,289][105692] Updated weights for policy 0, policy_version 1114123 (0.0007) [2023-12-26 23:29:19,352][105692] Updated weights for policy 0, policy_version 1114133 (0.0009) [2023-12-26 23:29:19,416][105692] Updated weights for policy 0, policy_version 1114143 (0.0011) [2023-12-26 23:29:19,678][105620] Updated weights for policy 1, policy_version 1115361 (0.0011) [2023-12-26 23:29:19,737][105620] Updated weights for policy 1, policy_version 1115371 (0.0010) [2023-12-26 23:29:19,809][105620] Updated weights for policy 1, policy_version 1115381 (0.0006) [2023-12-26 23:29:20,108][105692] Updated weights for policy 0, policy_version 1114153 (0.0012) [2023-12-26 23:29:20,171][105692] Updated weights for policy 0, policy_version 1114163 (0.0009) [2023-12-26 23:29:20,232][105692] Updated weights for policy 0, policy_version 1114173 (0.0009) [2023-12-26 23:29:20,533][105620] Updated weights for policy 1, policy_version 1115391 (0.0007) [2023-12-26 23:29:20,602][105620] Updated weights for policy 1, policy_version 1115401 (0.0007) [2023-12-26 23:29:20,665][105620] Updated weights for policy 1, policy_version 1115411 (0.0008) [2023-12-26 23:29:20,985][105692] Updated weights for policy 0, policy_version 1114183 (0.0010) [2023-12-26 23:29:21,047][105692] Updated weights for policy 0, policy_version 1114193 (0.0011) [2023-12-26 23:29:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 570851328. Throughput: 0: 10153.7, 1: 9842.3. Samples: 570844968. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:29:21,062][104569] Avg episode reward: [(0, '9264.811'), (1, '8986.664')] [2023-12-26 23:29:21,108][105692] Updated weights for policy 0, policy_version 1114203 (0.0010) [2023-12-26 23:29:21,373][105620] Updated weights for policy 1, policy_version 1115421 (0.0008) [2023-12-26 23:29:21,438][105620] Updated weights for policy 1, policy_version 1115431 (0.0009) [2023-12-26 23:29:21,484][105620] Updated weights for policy 1, policy_version 1115441 (0.0006) [2023-12-26 23:29:21,864][105692] Updated weights for policy 0, policy_version 1114213 (0.0008) [2023-12-26 23:29:21,930][105692] Updated weights for policy 0, policy_version 1114223 (0.0009) [2023-12-26 23:29:21,985][105692] Updated weights for policy 0, policy_version 1114233 (0.0009) [2023-12-26 23:29:22,292][105620] Updated weights for policy 1, policy_version 1115451 (0.0006) [2023-12-26 23:29:22,353][105620] Updated weights for policy 1, policy_version 1115461 (0.0008) [2023-12-26 23:29:22,419][105620] Updated weights for policy 1, policy_version 1115471 (0.0008) [2023-12-26 23:29:22,726][105692] Updated weights for policy 0, policy_version 1114243 (0.0008) [2023-12-26 23:29:22,797][105692] Updated weights for policy 0, policy_version 1114253 (0.0008) [2023-12-26 23:29:22,870][105692] Updated weights for policy 0, policy_version 1114263 (0.0006) [2023-12-26 23:29:23,154][105620] Updated weights for policy 1, policy_version 1115481 (0.0008) [2023-12-26 23:29:23,209][105620] Updated weights for policy 1, policy_version 1115491 (0.0007) [2023-12-26 23:29:23,261][105620] Updated weights for policy 1, policy_version 1115501 (0.0009) [2023-12-26 23:29:23,308][105620] Updated weights for policy 1, policy_version 1115511 (0.0009) [2023-12-26 23:29:23,483][105692] Updated weights for policy 0, policy_version 1114273 (0.0009) [2023-12-26 23:29:23,545][105692] Updated weights for policy 0, policy_version 1114283 (0.0005) [2023-12-26 23:29:23,598][105692] Updated weights for policy 0, policy_version 1114293 (0.0006) [2023-12-26 23:29:23,651][105692] Updated weights for policy 0, policy_version 1114303 (0.0005) [2023-12-26 23:29:23,930][105620] Updated weights for policy 1, policy_version 1115521 (0.0006) [2023-12-26 23:29:23,978][105620] Updated weights for policy 1, policy_version 1115531 (0.0005) [2023-12-26 23:29:24,022][105620] Updated weights for policy 1, policy_version 1115541 (0.0005) [2023-12-26 23:29:24,178][105692] Updated weights for policy 0, policy_version 1114313 (0.0009) [2023-12-26 23:29:24,245][105692] Updated weights for policy 0, policy_version 1114323 (0.0008) [2023-12-26 23:29:24,299][105692] Updated weights for policy 0, policy_version 1114333 (0.0009) [2023-12-26 23:29:24,605][105620] Updated weights for policy 1, policy_version 1115551 (0.0006) [2023-12-26 23:29:24,669][105620] Updated weights for policy 1, policy_version 1115561 (0.0005) [2023-12-26 23:29:24,736][105620] Updated weights for policy 1, policy_version 1115571 (0.0005) [2023-12-26 23:29:24,988][105692] Updated weights for policy 0, policy_version 1114343 (0.0008) [2023-12-26 23:29:25,050][105692] Updated weights for policy 0, policy_version 1114353 (0.0010) [2023-12-26 23:29:25,104][105692] Updated weights for policy 0, policy_version 1114363 (0.0010) [2023-12-26 23:29:25,251][105620] Updated weights for policy 1, policy_version 1115581 (0.0006) [2023-12-26 23:29:25,302][105620] Updated weights for policy 1, policy_version 1115591 (0.0009) [2023-12-26 23:29:25,350][105620] Updated weights for policy 1, policy_version 1115601 (0.0009) [2023-12-26 23:29:25,807][105692] Updated weights for policy 0, policy_version 1114373 (0.0007) [2023-12-26 23:29:25,853][105692] Updated weights for policy 0, policy_version 1114383 (0.0005) [2023-12-26 23:29:25,909][105692] Updated weights for policy 0, policy_version 1114393 (0.0005) [2023-12-26 23:29:25,946][105620] Updated weights for policy 1, policy_version 1115611 (0.0008) [2023-12-26 23:29:25,998][105620] Updated weights for policy 1, policy_version 1115621 (0.0005) [2023-12-26 23:29:26,045][105620] Updated weights for policy 1, policy_version 1115631 (0.0005) [2023-12-26 23:29:26,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19933.8, 300 sec: 19633.0). Total num frames: 570957824. Throughput: 0: 10087.3, 1: 9980.6. Samples: 570967328. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:29:26,063][104569] Avg episode reward: [(0, '9086.063'), (1, '8819.238')] [2023-12-26 23:29:26,432][105692] Updated weights for policy 0, policy_version 1114403 (0.0008) [2023-12-26 23:29:26,493][105692] Updated weights for policy 0, policy_version 1114413 (0.0005) [2023-12-26 23:29:26,553][105692] Updated weights for policy 0, policy_version 1114423 (0.0006) [2023-12-26 23:29:26,627][105620] Updated weights for policy 1, policy_version 1115641 (0.0005) [2023-12-26 23:29:26,692][105620] Updated weights for policy 1, policy_version 1115651 (0.0007) [2023-12-26 23:29:26,753][105620] Updated weights for policy 1, policy_version 1115661 (0.0009) [2023-12-26 23:29:26,806][105620] Updated weights for policy 1, policy_version 1115671 (0.0009) [2023-12-26 23:29:27,142][105692] Updated weights for policy 0, policy_version 1114433 (0.0010) [2023-12-26 23:29:27,205][105692] Updated weights for policy 0, policy_version 1114443 (0.0007) [2023-12-26 23:29:27,272][105692] Updated weights for policy 0, policy_version 1114453 (0.0008) [2023-12-26 23:29:27,337][105692] Updated weights for policy 0, policy_version 1114463 (0.0007) [2023-12-26 23:29:27,568][105620] Updated weights for policy 1, policy_version 1115681 (0.0006) [2023-12-26 23:29:27,629][105620] Updated weights for policy 1, policy_version 1115691 (0.0005) [2023-12-26 23:29:27,682][105620] Updated weights for policy 1, policy_version 1115701 (0.0005) [2023-12-26 23:29:28,011][105692] Updated weights for policy 0, policy_version 1114473 (0.0009) [2023-12-26 23:29:28,067][105692] Updated weights for policy 0, policy_version 1114483 (0.0008) [2023-12-26 23:29:28,125][105692] Updated weights for policy 0, policy_version 1114493 (0.0009) [2023-12-26 23:29:28,304][105620] Updated weights for policy 1, policy_version 1115711 (0.0005) [2023-12-26 23:29:28,363][105620] Updated weights for policy 1, policy_version 1115721 (0.0007) [2023-12-26 23:29:28,428][105620] Updated weights for policy 1, policy_version 1115731 (0.0009) [2023-12-26 23:29:28,898][105692] Updated weights for policy 0, policy_version 1114503 (0.0010) [2023-12-26 23:29:28,957][105692] Updated weights for policy 0, policy_version 1114513 (0.0010) [2023-12-26 23:29:29,019][105692] Updated weights for policy 0, policy_version 1114523 (0.0010) [2023-12-26 23:29:29,156][105620] Updated weights for policy 1, policy_version 1115741 (0.0008) [2023-12-26 23:29:29,210][105620] Updated weights for policy 1, policy_version 1115751 (0.0008) [2023-12-26 23:29:29,273][105620] Updated weights for policy 1, policy_version 1115761 (0.0006) [2023-12-26 23:29:29,765][105692] Updated weights for policy 0, policy_version 1114533 (0.0010) [2023-12-26 23:29:29,830][105692] Updated weights for policy 0, policy_version 1114543 (0.0010) [2023-12-26 23:29:29,897][105692] Updated weights for policy 0, policy_version 1114553 (0.0011) [2023-12-26 23:29:30,018][105620] Updated weights for policy 1, policy_version 1115771 (0.0006) [2023-12-26 23:29:30,080][105620] Updated weights for policy 1, policy_version 1115781 (0.0008) [2023-12-26 23:29:30,144][105620] Updated weights for policy 1, policy_version 1115791 (0.0009) [2023-12-26 23:29:30,504][105692] Updated weights for policy 0, policy_version 1114563 (0.0010) [2023-12-26 23:29:30,567][105692] Updated weights for policy 0, policy_version 1114573 (0.0011) [2023-12-26 23:29:30,619][105692] Updated weights for policy 0, policy_version 1114583 (0.0010) [2023-12-26 23:29:30,959][105620] Updated weights for policy 1, policy_version 1115801 (0.0010) [2023-12-26 23:29:31,011][105620] Updated weights for policy 1, policy_version 1115811 (0.0008) [2023-12-26 23:29:31,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19933.8, 300 sec: 19660.8). Total num frames: 571056128. Throughput: 0: 10145.8, 1: 9983.6. Samples: 571030968. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:29:31,063][104569] Avg episode reward: [(0, '9086.149'), (1, '9001.443')] [2023-12-26 23:29:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001114592_285376512.pth... [2023-12-26 23:29:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001113408_285073408.pth [2023-12-26 23:29:31,078][105620] Updated weights for policy 1, policy_version 1115821 (0.0009) [2023-12-26 23:29:31,143][105620] Updated weights for policy 1, policy_version 1115831 (0.0011) [2023-12-26 23:29:31,147][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001115832_285687808.pth... [2023-12-26 23:29:31,152][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001114648_285384704.pth [2023-12-26 23:29:31,357][105692] Updated weights for policy 0, policy_version 1114593 (0.0006) [2023-12-26 23:29:31,419][105692] Updated weights for policy 0, policy_version 1114603 (0.0011) [2023-12-26 23:29:31,471][105692] Updated weights for policy 0, policy_version 1114613 (0.0010) [2023-12-26 23:29:31,519][105692] Updated weights for policy 0, policy_version 1114623 (0.0010) [2023-12-26 23:29:31,898][105620] Updated weights for policy 1, policy_version 1115841 (0.0008) [2023-12-26 23:29:31,955][105620] Updated weights for policy 1, policy_version 1115851 (0.0008) [2023-12-26 23:29:32,006][105620] Updated weights for policy 1, policy_version 1115861 (0.0007) [2023-12-26 23:29:32,237][105692] Updated weights for policy 0, policy_version 1114633 (0.0006) [2023-12-26 23:29:32,302][105692] Updated weights for policy 0, policy_version 1114643 (0.0008) [2023-12-26 23:29:32,355][105692] Updated weights for policy 0, policy_version 1114653 (0.0008) [2023-12-26 23:29:32,820][105620] Updated weights for policy 1, policy_version 1115871 (0.0006) [2023-12-26 23:29:32,877][105620] Updated weights for policy 1, policy_version 1115881 (0.0010) [2023-12-26 23:29:32,922][105620] Updated weights for policy 1, policy_version 1115891 (0.0010) [2023-12-26 23:29:33,099][105692] Updated weights for policy 0, policy_version 1114663 (0.0009) [2023-12-26 23:29:33,152][105692] Updated weights for policy 0, policy_version 1114674 (0.0010) [2023-12-26 23:29:33,208][105692] Updated weights for policy 0, policy_version 1114684 (0.0011) [2023-12-26 23:29:33,482][105620] Updated weights for policy 1, policy_version 1115901 (0.0010) [2023-12-26 23:29:33,532][105620] Updated weights for policy 1, policy_version 1115911 (0.0010) [2023-12-26 23:29:33,594][105620] Updated weights for policy 1, policy_version 1115921 (0.0010) [2023-12-26 23:29:33,909][105692] Updated weights for policy 0, policy_version 1114694 (0.0007) [2023-12-26 23:29:33,955][105692] Updated weights for policy 0, policy_version 1114704 (0.0005) [2023-12-26 23:29:34,004][105692] Updated weights for policy 0, policy_version 1114714 (0.0005) [2023-12-26 23:29:34,152][105620] Updated weights for policy 1, policy_version 1115931 (0.0010) [2023-12-26 23:29:34,212][105620] Updated weights for policy 1, policy_version 1115941 (0.0008) [2023-12-26 23:29:34,273][105620] Updated weights for policy 1, policy_version 1115951 (0.0008) [2023-12-26 23:29:34,668][105692] Updated weights for policy 0, policy_version 1114724 (0.0007) [2023-12-26 23:29:34,718][105692] Updated weights for policy 0, policy_version 1114734 (0.0009) [2023-12-26 23:29:34,768][105692] Updated weights for policy 0, policy_version 1114744 (0.0009) [2023-12-26 23:29:34,989][105620] Updated weights for policy 1, policy_version 1115961 (0.0008) [2023-12-26 23:29:35,051][105620] Updated weights for policy 1, policy_version 1115971 (0.0009) [2023-12-26 23:29:35,103][105620] Updated weights for policy 1, policy_version 1115982 (0.0009) [2023-12-26 23:29:35,150][105620] Updated weights for policy 1, policy_version 1115992 (0.0008) [2023-12-26 23:29:35,459][105692] Updated weights for policy 0, policy_version 1114754 (0.0009) [2023-12-26 23:29:35,531][105692] Updated weights for policy 0, policy_version 1114764 (0.0009) [2023-12-26 23:29:35,603][105692] Updated weights for policy 0, policy_version 1114774 (0.0010) [2023-12-26 23:29:35,675][105692] Updated weights for policy 0, policy_version 1114784 (0.0009) [2023-12-26 23:29:35,845][105620] Updated weights for policy 1, policy_version 1116002 (0.0006) [2023-12-26 23:29:35,912][105620] Updated weights for policy 1, policy_version 1116012 (0.0008) [2023-12-26 23:29:35,975][105620] Updated weights for policy 1, policy_version 1116022 (0.0010) [2023-12-26 23:29:36,062][104569] Fps is (10 sec: 20480.3, 60 sec: 20070.4, 300 sec: 19688.6). Total num frames: 571162624. Throughput: 0: 10124.9, 1: 10038.2. Samples: 571148836. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:29:36,062][104569] Avg episode reward: [(0, '9266.263'), (1, '9078.429')] [2023-12-26 23:29:36,343][105692] Updated weights for policy 0, policy_version 1114794 (0.0011) [2023-12-26 23:29:36,403][105692] Updated weights for policy 0, policy_version 1114804 (0.0011) [2023-12-26 23:29:36,458][105692] Updated weights for policy 0, policy_version 1114814 (0.0011) [2023-12-26 23:29:36,676][105620] Updated weights for policy 1, policy_version 1116032 (0.0010) [2023-12-26 23:29:36,744][105620] Updated weights for policy 1, policy_version 1116042 (0.0010) [2023-12-26 23:29:36,808][105620] Updated weights for policy 1, policy_version 1116052 (0.0008) [2023-12-26 23:29:37,057][105692] Updated weights for policy 0, policy_version 1114824 (0.0006) [2023-12-26 23:29:37,114][105692] Updated weights for policy 0, policy_version 1114834 (0.0005) [2023-12-26 23:29:37,171][105692] Updated weights for policy 0, policy_version 1114844 (0.0005) [2023-12-26 23:29:37,438][105620] Updated weights for policy 1, policy_version 1116062 (0.0008) [2023-12-26 23:29:37,490][105620] Updated weights for policy 1, policy_version 1116072 (0.0010) [2023-12-26 23:29:37,555][105620] Updated weights for policy 1, policy_version 1116082 (0.0010) [2023-12-26 23:29:37,816][105692] Updated weights for policy 0, policy_version 1114854 (0.0009) [2023-12-26 23:29:37,878][105692] Updated weights for policy 0, policy_version 1114864 (0.0010) [2023-12-26 23:29:37,936][105692] Updated weights for policy 0, policy_version 1114874 (0.0010) [2023-12-26 23:29:38,272][105620] Updated weights for policy 1, policy_version 1116092 (0.0010) [2023-12-26 23:29:38,325][105620] Updated weights for policy 1, policy_version 1116102 (0.0010) [2023-12-26 23:29:38,382][105620] Updated weights for policy 1, policy_version 1116112 (0.0010) [2023-12-26 23:29:38,633][105692] Updated weights for policy 0, policy_version 1114884 (0.0010) [2023-12-26 23:29:38,682][105692] Updated weights for policy 0, policy_version 1114894 (0.0010) [2023-12-26 23:29:38,727][105692] Updated weights for policy 0, policy_version 1114904 (0.0010) [2023-12-26 23:29:39,052][105620] Updated weights for policy 1, policy_version 1116122 (0.0011) [2023-12-26 23:29:39,103][105620] Updated weights for policy 1, policy_version 1116132 (0.0010) [2023-12-26 23:29:39,158][105620] Updated weights for policy 1, policy_version 1116142 (0.0010) [2023-12-26 23:29:39,213][105620] Updated weights for policy 1, policy_version 1116152 (0.0010) [2023-12-26 23:29:39,514][105692] Updated weights for policy 0, policy_version 1114914 (0.0010) [2023-12-26 23:29:39,564][105692] Updated weights for policy 0, policy_version 1114924 (0.0010) [2023-12-26 23:29:39,624][105692] Updated weights for policy 0, policy_version 1114934 (0.0011) [2023-12-26 23:29:39,680][105692] Updated weights for policy 0, policy_version 1114944 (0.0011) [2023-12-26 23:29:40,025][105620] Updated weights for policy 1, policy_version 1116162 (0.0009) [2023-12-26 23:29:40,087][105620] Updated weights for policy 1, policy_version 1116172 (0.0009) [2023-12-26 23:29:40,148][105620] Updated weights for policy 1, policy_version 1116182 (0.0009) [2023-12-26 23:29:40,454][105692] Updated weights for policy 0, policy_version 1114954 (0.0009) [2023-12-26 23:29:40,508][105692] Updated weights for policy 0, policy_version 1114964 (0.0008) [2023-12-26 23:29:40,577][105692] Updated weights for policy 0, policy_version 1114974 (0.0006) [2023-12-26 23:29:40,897][105620] Updated weights for policy 1, policy_version 1116192 (0.0010) [2023-12-26 23:29:40,958][105620] Updated weights for policy 1, policy_version 1116202 (0.0010) [2023-12-26 23:29:41,014][105620] Updated weights for policy 1, policy_version 1116212 (0.0011) [2023-12-26 23:29:41,062][104569] Fps is (10 sec: 20479.8, 60 sec: 20070.3, 300 sec: 19688.6). Total num frames: 571260928. Throughput: 0: 10110.8, 1: 10006.1. Samples: 571266700. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:29:41,063][104569] Avg episode reward: [(0, '9177.449'), (1, '9078.780')] [2023-12-26 23:29:41,357][105692] Updated weights for policy 0, policy_version 1114984 (0.0008) [2023-12-26 23:29:41,432][105692] Updated weights for policy 0, policy_version 1114994 (0.0008) [2023-12-26 23:29:41,485][105692] Updated weights for policy 0, policy_version 1115004 (0.0008) [2023-12-26 23:29:41,861][105620] Updated weights for policy 1, policy_version 1116222 (0.0006) [2023-12-26 23:29:41,924][105620] Updated weights for policy 1, policy_version 1116232 (0.0009) [2023-12-26 23:29:41,986][105620] Updated weights for policy 1, policy_version 1116242 (0.0011) [2023-12-26 23:29:42,308][105692] Updated weights for policy 0, policy_version 1115014 (0.0008) [2023-12-26 23:29:42,383][105692] Updated weights for policy 0, policy_version 1115024 (0.0008) [2023-12-26 23:29:42,449][105692] Updated weights for policy 0, policy_version 1115034 (0.0008) [2023-12-26 23:29:42,624][105620] Updated weights for policy 1, policy_version 1116252 (0.0008) [2023-12-26 23:29:42,691][105620] Updated weights for policy 1, policy_version 1116262 (0.0010) [2023-12-26 23:29:42,743][105620] Updated weights for policy 1, policy_version 1116272 (0.0010) [2023-12-26 23:29:43,134][105692] Updated weights for policy 0, policy_version 1115044 (0.0007) [2023-12-26 23:29:43,187][105692] Updated weights for policy 0, policy_version 1115054 (0.0006) [2023-12-26 23:29:43,258][105692] Updated weights for policy 0, policy_version 1115064 (0.0006) [2023-12-26 23:29:43,480][105620] Updated weights for policy 1, policy_version 1116282 (0.0009) [2023-12-26 23:29:43,532][105620] Updated weights for policy 1, policy_version 1116292 (0.0010) [2023-12-26 23:29:43,580][105620] Updated weights for policy 1, policy_version 1116302 (0.0008) [2023-12-26 23:29:43,630][105620] Updated weights for policy 1, policy_version 1116312 (0.0010) [2023-12-26 23:29:43,972][105692] Updated weights for policy 0, policy_version 1115074 (0.0008) [2023-12-26 23:29:44,028][105692] Updated weights for policy 0, policy_version 1115084 (0.0008) [2023-12-26 23:29:44,073][105692] Updated weights for policy 0, policy_version 1115094 (0.0008) [2023-12-26 23:29:44,127][105692] Updated weights for policy 0, policy_version 1115104 (0.0009) [2023-12-26 23:29:44,371][105620] Updated weights for policy 1, policy_version 1116322 (0.0011) [2023-12-26 23:29:44,429][105620] Updated weights for policy 1, policy_version 1116332 (0.0011) [2023-12-26 23:29:44,491][105620] Updated weights for policy 1, policy_version 1116342 (0.0010) [2023-12-26 23:29:44,833][105692] Updated weights for policy 0, policy_version 1115114 (0.0008) [2023-12-26 23:29:44,893][105692] Updated weights for policy 0, policy_version 1115124 (0.0008) [2023-12-26 23:29:44,950][105692] Updated weights for policy 0, policy_version 1115134 (0.0008) [2023-12-26 23:29:45,234][105620] Updated weights for policy 1, policy_version 1116352 (0.0011) [2023-12-26 23:29:45,296][105620] Updated weights for policy 1, policy_version 1116362 (0.0011) [2023-12-26 23:29:45,367][105620] Updated weights for policy 1, policy_version 1116372 (0.0011) [2023-12-26 23:29:45,671][105692] Updated weights for policy 0, policy_version 1115144 (0.0009) [2023-12-26 23:29:45,728][105692] Updated weights for policy 0, policy_version 1115154 (0.0008) [2023-12-26 23:29:45,781][105692] Updated weights for policy 0, policy_version 1115164 (0.0008) [2023-12-26 23:29:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19933.9, 300 sec: 19660.8). Total num frames: 571351040. Throughput: 0: 10003.6, 1: 9917.4. Samples: 571322812. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:29:46,063][104569] Avg episode reward: [(0, '9177.975'), (1, '9166.344')] [2023-12-26 23:29:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001115168_285523968.pth... [2023-12-26 23:29:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001113984_285220864.pth [2023-12-26 23:29:46,104][105620] Updated weights for policy 1, policy_version 1116382 (0.0011) [2023-12-26 23:29:46,149][105620] Updated weights for policy 1, policy_version 1116392 (0.0010) [2023-12-26 23:29:46,200][105620] Updated weights for policy 1, policy_version 1116402 (0.0010) [2023-12-26 23:29:46,228][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001116408_285835264.pth... [2023-12-26 23:29:46,233][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001115224_285532160.pth [2023-12-26 23:29:46,437][105692] Updated weights for policy 0, policy_version 1115174 (0.0006) [2023-12-26 23:29:46,494][105692] Updated weights for policy 0, policy_version 1115184 (0.0007) [2023-12-26 23:29:46,548][105692] Updated weights for policy 0, policy_version 1115194 (0.0008) [2023-12-26 23:29:46,949][105620] Updated weights for policy 1, policy_version 1116412 (0.0010) [2023-12-26 23:29:46,997][105620] Updated weights for policy 1, policy_version 1116422 (0.0010) [2023-12-26 23:29:47,045][105620] Updated weights for policy 1, policy_version 1116432 (0.0010) [2023-12-26 23:29:47,267][105692] Updated weights for policy 0, policy_version 1115204 (0.0008) [2023-12-26 23:29:47,319][105692] Updated weights for policy 0, policy_version 1115214 (0.0008) [2023-12-26 23:29:47,373][105692] Updated weights for policy 0, policy_version 1115224 (0.0007) [2023-12-26 23:29:47,756][105620] Updated weights for policy 1, policy_version 1116442 (0.0010) [2023-12-26 23:29:47,811][105620] Updated weights for policy 1, policy_version 1116452 (0.0006) [2023-12-26 23:29:47,869][105620] Updated weights for policy 1, policy_version 1116462 (0.0006) [2023-12-26 23:29:47,929][105620] Updated weights for policy 1, policy_version 1116472 (0.0009) [2023-12-26 23:29:48,188][105692] Updated weights for policy 0, policy_version 1115234 (0.0008) [2023-12-26 23:29:48,241][105692] Updated weights for policy 0, policy_version 1115244 (0.0010) [2023-12-26 23:29:48,291][105692] Updated weights for policy 0, policy_version 1115255 (0.0009) [2023-12-26 23:29:48,561][105620] Updated weights for policy 1, policy_version 1116482 (0.0009) [2023-12-26 23:29:48,621][105620] Updated weights for policy 1, policy_version 1116492 (0.0009) [2023-12-26 23:29:48,682][105620] Updated weights for policy 1, policy_version 1116502 (0.0009) [2023-12-26 23:29:49,025][105692] Updated weights for policy 0, policy_version 1115265 (0.0009) [2023-12-26 23:29:49,087][105692] Updated weights for policy 0, policy_version 1115275 (0.0009) [2023-12-26 23:29:49,140][105692] Updated weights for policy 0, policy_version 1115285 (0.0008) [2023-12-26 23:29:49,202][105692] Updated weights for policy 0, policy_version 1115295 (0.0009) [2023-12-26 23:29:49,475][105620] Updated weights for policy 1, policy_version 1116512 (0.0009) [2023-12-26 23:29:49,537][105620] Updated weights for policy 1, policy_version 1116522 (0.0009) [2023-12-26 23:29:49,591][105620] Updated weights for policy 1, policy_version 1116532 (0.0008) [2023-12-26 23:29:49,981][105692] Updated weights for policy 0, policy_version 1115305 (0.0009) [2023-12-26 23:29:50,038][105692] Updated weights for policy 0, policy_version 1115315 (0.0009) [2023-12-26 23:29:50,086][105692] Updated weights for policy 0, policy_version 1115325 (0.0009) [2023-12-26 23:29:50,259][105620] Updated weights for policy 1, policy_version 1116542 (0.0010) [2023-12-26 23:29:50,308][105620] Updated weights for policy 1, policy_version 1116552 (0.0010) [2023-12-26 23:29:50,365][105620] Updated weights for policy 1, policy_version 1116562 (0.0010) [2023-12-26 23:29:50,931][105692] Updated weights for policy 0, policy_version 1115335 (0.0009) [2023-12-26 23:29:50,995][105692] Updated weights for policy 0, policy_version 1115345 (0.0008) [2023-12-26 23:29:51,062][104569] Fps is (10 sec: 18022.8, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 571441152. Throughput: 0: 9951.1, 1: 9905.8. Samples: 571438352. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:29:51,062][104569] Avg episode reward: [(0, '9356.142'), (1, '9167.413')] [2023-12-26 23:29:51,065][105692] Updated weights for policy 0, policy_version 1115355 (0.0008) [2023-12-26 23:29:51,070][105620] Updated weights for policy 1, policy_version 1116572 (0.0010) [2023-12-26 23:29:51,132][105620] Updated weights for policy 1, policy_version 1116582 (0.0008) [2023-12-26 23:29:51,202][105620] Updated weights for policy 1, policy_version 1116592 (0.0009) [2023-12-26 23:29:51,886][105620] Updated weights for policy 1, policy_version 1116602 (0.0010) [2023-12-26 23:29:51,932][105692] Updated weights for policy 0, policy_version 1115365 (0.0007) [2023-12-26 23:29:51,946][105620] Updated weights for policy 1, policy_version 1116612 (0.0008) [2023-12-26 23:29:51,989][105692] Updated weights for policy 0, policy_version 1115375 (0.0007) [2023-12-26 23:29:52,003][105620] Updated weights for policy 1, policy_version 1116622 (0.0007) [2023-12-26 23:29:52,042][105692] Updated weights for policy 0, policy_version 1115385 (0.0006) [2023-12-26 23:29:52,056][105620] Updated weights for policy 1, policy_version 1116632 (0.0006) [2023-12-26 23:29:52,821][105620] Updated weights for policy 1, policy_version 1116642 (0.0008) [2023-12-26 23:29:52,843][105692] Updated weights for policy 0, policy_version 1115395 (0.0008) [2023-12-26 23:29:52,868][105620] Updated weights for policy 1, policy_version 1116652 (0.0008) [2023-12-26 23:29:52,904][105692] Updated weights for policy 0, policy_version 1115405 (0.0007) [2023-12-26 23:29:52,915][105620] Updated weights for policy 1, policy_version 1116662 (0.0007) [2023-12-26 23:29:52,962][105692] Updated weights for policy 0, policy_version 1115415 (0.0008) [2023-12-26 23:29:53,685][105620] Updated weights for policy 1, policy_version 1116672 (0.0010) [2023-12-26 23:29:53,707][105692] Updated weights for policy 0, policy_version 1115425 (0.0008) [2023-12-26 23:29:53,743][105620] Updated weights for policy 1, policy_version 1116682 (0.0009) [2023-12-26 23:29:53,761][105692] Updated weights for policy 0, policy_version 1115435 (0.0007) [2023-12-26 23:29:53,801][105620] Updated weights for policy 1, policy_version 1116692 (0.0005) [2023-12-26 23:29:53,817][105692] Updated weights for policy 0, policy_version 1115445 (0.0008) [2023-12-26 23:29:53,869][105692] Updated weights for policy 0, policy_version 1115455 (0.0010) [2023-12-26 23:29:54,342][105620] Updated weights for policy 1, policy_version 1116702 (0.0006) [2023-12-26 23:29:54,393][105620] Updated weights for policy 1, policy_version 1116712 (0.0007) [2023-12-26 23:29:54,450][105620] Updated weights for policy 1, policy_version 1116722 (0.0008) [2023-12-26 23:29:54,626][105692] Updated weights for policy 0, policy_version 1115465 (0.0010) [2023-12-26 23:29:54,681][105692] Updated weights for policy 0, policy_version 1115475 (0.0009) [2023-12-26 23:29:54,735][105692] Updated weights for policy 0, policy_version 1115485 (0.0009) [2023-12-26 23:29:55,197][105586] KL-divergence is very high: 103.7953 [2023-12-26 23:29:55,214][105620] Updated weights for policy 1, policy_version 1116732 (0.0010) [2023-12-26 23:29:55,273][105620] Updated weights for policy 1, policy_version 1116742 (0.0010) [2023-12-26 23:29:55,317][105620] Updated weights for policy 1, policy_version 1116752 (0.0010) [2023-12-26 23:29:55,405][105692] Updated weights for policy 0, policy_version 1115495 (0.0009) [2023-12-26 23:29:55,467][105692] Updated weights for policy 0, policy_version 1115505 (0.0008) [2023-12-26 23:29:55,526][105692] Updated weights for policy 0, policy_version 1115515 (0.0008) [2023-12-26 23:29:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 571539456. Throughput: 0: 9824.5, 1: 9941.7. Samples: 571551928. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-26 23:29:56,063][104569] Avg episode reward: [(0, '9197.644'), (1, '8898.265')] [2023-12-26 23:29:56,084][105620] Updated weights for policy 1, policy_version 1116762 (0.0010) [2023-12-26 23:29:56,132][105620] Updated weights for policy 1, policy_version 1116772 (0.0010) [2023-12-26 23:29:56,187][105620] Updated weights for policy 1, policy_version 1116782 (0.0010) [2023-12-26 23:29:56,244][105620] Updated weights for policy 1, policy_version 1116792 (0.0010) [2023-12-26 23:29:56,275][105692] Updated weights for policy 0, policy_version 1115525 (0.0008) [2023-12-26 23:29:56,326][105692] Updated weights for policy 0, policy_version 1115535 (0.0008) [2023-12-26 23:29:56,385][105692] Updated weights for policy 0, policy_version 1115545 (0.0008) [2023-12-26 23:29:56,985][105620] Updated weights for policy 1, policy_version 1116802 (0.0010) [2023-12-26 23:29:57,043][105620] Updated weights for policy 1, policy_version 1116812 (0.0010) [2023-12-26 23:29:57,086][105620] Updated weights for policy 1, policy_version 1116822 (0.0010) [2023-12-26 23:29:57,162][105692] Updated weights for policy 0, policy_version 1115555 (0.0009) [2023-12-26 23:29:57,207][105692] Updated weights for policy 0, policy_version 1115565 (0.0006) [2023-12-26 23:29:57,259][105692] Updated weights for policy 0, policy_version 1115575 (0.0006) [2023-12-26 23:29:57,806][105620] Updated weights for policy 1, policy_version 1116832 (0.0008) [2023-12-26 23:29:57,854][105620] Updated weights for policy 1, policy_version 1116842 (0.0009) [2023-12-26 23:29:57,909][105620] Updated weights for policy 1, policy_version 1116852 (0.0007) [2023-12-26 23:29:58,031][105692] Updated weights for policy 0, policy_version 1115585 (0.0008) [2023-12-26 23:29:58,088][105692] Updated weights for policy 0, policy_version 1115595 (0.0010) [2023-12-26 23:29:58,141][105692] Updated weights for policy 0, policy_version 1115605 (0.0010) [2023-12-26 23:29:58,220][105692] Updated weights for policy 0, policy_version 1115615 (0.0009) [2023-12-26 23:29:58,568][105620] Updated weights for policy 1, policy_version 1116862 (0.0007) [2023-12-26 23:29:58,629][105620] Updated weights for policy 1, policy_version 1116872 (0.0008) [2023-12-26 23:29:58,695][105620] Updated weights for policy 1, policy_version 1116882 (0.0009) [2023-12-26 23:29:59,090][105692] Updated weights for policy 0, policy_version 1115625 (0.0008) [2023-12-26 23:29:59,155][105692] Updated weights for policy 0, policy_version 1115635 (0.0009) [2023-12-26 23:29:59,221][105692] Updated weights for policy 0, policy_version 1115645 (0.0009) [2023-12-26 23:29:59,546][105620] Updated weights for policy 1, policy_version 1116892 (0.0008) [2023-12-26 23:29:59,602][105620] Updated weights for policy 1, policy_version 1116902 (0.0005) [2023-12-26 23:29:59,664][105620] Updated weights for policy 1, policy_version 1116912 (0.0005) [2023-12-26 23:30:00,032][105692] Updated weights for policy 0, policy_version 1115655 (0.0007) [2023-12-26 23:30:00,097][105692] Updated weights for policy 0, policy_version 1115665 (0.0005) [2023-12-26 23:30:00,154][105692] Updated weights for policy 0, policy_version 1115675 (0.0005) [2023-12-26 23:30:00,288][105620] Updated weights for policy 1, policy_version 1116922 (0.0005) [2023-12-26 23:30:00,347][105620] Updated weights for policy 1, policy_version 1116932 (0.0006) [2023-12-26 23:30:00,406][105620] Updated weights for policy 1, policy_version 1116942 (0.0008) [2023-12-26 23:30:00,461][105620] Updated weights for policy 1, policy_version 1116952 (0.0009) [2023-12-26 23:30:00,860][105692] Updated weights for policy 0, policy_version 1115685 (0.0008) [2023-12-26 23:30:00,910][105692] Updated weights for policy 0, policy_version 1115695 (0.0009) [2023-12-26 23:30:00,956][105692] Updated weights for policy 0, policy_version 1115705 (0.0008) [2023-12-26 23:30:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 571637760. Throughput: 0: 9769.4, 1: 9963.7. Samples: 571607856. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:30:01,063][104569] Avg episode reward: [(0, '9014.490'), (1, '8715.153')] [2023-12-26 23:30:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001116952_285974528.pth... [2023-12-26 23:30:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001115712_285663232.pth... [2023-12-26 23:30:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001115832_285687808.pth [2023-12-26 23:30:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001114592_285376512.pth [2023-12-26 23:30:01,175][105620] Updated weights for policy 1, policy_version 1116962 (0.0008) [2023-12-26 23:30:01,232][105620] Updated weights for policy 1, policy_version 1116972 (0.0008) [2023-12-26 23:30:01,293][105620] Updated weights for policy 1, policy_version 1116982 (0.0009) [2023-12-26 23:30:01,733][105692] Updated weights for policy 0, policy_version 1115715 (0.0009) [2023-12-26 23:30:01,793][105692] Updated weights for policy 0, policy_version 1115725 (0.0009) [2023-12-26 23:30:01,848][105692] Updated weights for policy 0, policy_version 1115735 (0.0010) [2023-12-26 23:30:02,047][105620] Updated weights for policy 1, policy_version 1116992 (0.0009) [2023-12-26 23:30:02,105][105620] Updated weights for policy 1, policy_version 1117002 (0.0007) [2023-12-26 23:30:02,163][105620] Updated weights for policy 1, policy_version 1117012 (0.0007) [2023-12-26 23:30:02,625][105692] Updated weights for policy 0, policy_version 1115745 (0.0009) [2023-12-26 23:30:02,677][105692] Updated weights for policy 0, policy_version 1115755 (0.0007) [2023-12-26 23:30:02,740][105692] Updated weights for policy 0, policy_version 1115765 (0.0008) [2023-12-26 23:30:02,802][105692] Updated weights for policy 0, policy_version 1115775 (0.0008) [2023-12-26 23:30:02,920][105620] Updated weights for policy 1, policy_version 1117022 (0.0009) [2023-12-26 23:30:02,973][105620] Updated weights for policy 1, policy_version 1117032 (0.0009) [2023-12-26 23:30:03,024][105620] Updated weights for policy 1, policy_version 1117042 (0.0009) [2023-12-26 23:30:03,401][105692] Updated weights for policy 0, policy_version 1115785 (0.0009) [2023-12-26 23:30:03,448][105692] Updated weights for policy 0, policy_version 1115795 (0.0009) [2023-12-26 23:30:03,494][105692] Updated weights for policy 0, policy_version 1115805 (0.0009) [2023-12-26 23:30:03,752][105620] Updated weights for policy 1, policy_version 1117052 (0.0008) [2023-12-26 23:30:03,798][105620] Updated weights for policy 1, policy_version 1117062 (0.0005) [2023-12-26 23:30:03,846][105620] Updated weights for policy 1, policy_version 1117072 (0.0006) [2023-12-26 23:30:04,340][105692] Updated weights for policy 0, policy_version 1115815 (0.0007) [2023-12-26 23:30:04,405][105692] Updated weights for policy 0, policy_version 1115825 (0.0008) [2023-12-26 23:30:04,458][105620] Updated weights for policy 1, policy_version 1117082 (0.0009) [2023-12-26 23:30:04,471][105692] Updated weights for policy 0, policy_version 1115835 (0.0006) [2023-12-26 23:30:04,516][105620] Updated weights for policy 1, policy_version 1117092 (0.0010) [2023-12-26 23:30:04,569][105620] Updated weights for policy 1, policy_version 1117102 (0.0010) [2023-12-26 23:30:04,641][105620] Updated weights for policy 1, policy_version 1117112 (0.0008) [2023-12-26 23:30:05,077][105692] Updated weights for policy 0, policy_version 1115845 (0.0008) [2023-12-26 23:30:05,130][105692] Updated weights for policy 0, policy_version 1115855 (0.0009) [2023-12-26 23:30:05,186][105692] Updated weights for policy 0, policy_version 1115865 (0.0009) [2023-12-26 23:30:05,387][105620] Updated weights for policy 1, policy_version 1117122 (0.0005) [2023-12-26 23:30:05,449][105620] Updated weights for policy 1, policy_version 1117132 (0.0005) [2023-12-26 23:30:05,504][105620] Updated weights for policy 1, policy_version 1117142 (0.0005) [2023-12-26 23:30:06,002][105620] Updated weights for policy 1, policy_version 1117152 (0.0008) [2023-12-26 23:30:06,061][105620] Updated weights for policy 1, policy_version 1117162 (0.0005) [2023-12-26 23:30:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 571727872. Throughput: 0: 9627.7, 1: 9859.6. Samples: 571721896. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:30:06,062][104569] Avg episode reward: [(0, '9076.412'), (1, '8893.323')] [2023-12-26 23:30:06,078][105692] Updated weights for policy 0, policy_version 1115875 (0.0008) [2023-12-26 23:30:06,126][105620] Updated weights for policy 1, policy_version 1117172 (0.0006) [2023-12-26 23:30:06,148][105692] Updated weights for policy 0, policy_version 1115885 (0.0008) [2023-12-26 23:30:06,215][105692] Updated weights for policy 0, policy_version 1115895 (0.0008) [2023-12-26 23:30:06,818][105620] Updated weights for policy 1, policy_version 1117182 (0.0008) [2023-12-26 23:30:06,829][105692] Updated weights for policy 0, policy_version 1115905 (0.0008) [2023-12-26 23:30:06,881][105620] Updated weights for policy 1, policy_version 1117192 (0.0006) [2023-12-26 23:30:06,893][105692] Updated weights for policy 0, policy_version 1115915 (0.0011) [2023-12-26 23:30:06,932][105620] Updated weights for policy 1, policy_version 1117202 (0.0006) [2023-12-26 23:30:06,954][105692] Updated weights for policy 0, policy_version 1115925 (0.0010) [2023-12-26 23:30:07,016][105692] Updated weights for policy 0, policy_version 1115935 (0.0007) [2023-12-26 23:30:07,559][105620] Updated weights for policy 1, policy_version 1117212 (0.0007) [2023-12-26 23:30:07,625][105620] Updated weights for policy 1, policy_version 1117222 (0.0011) [2023-12-26 23:30:07,680][105620] Updated weights for policy 1, policy_version 1117232 (0.0010) [2023-12-26 23:30:07,688][105692] Updated weights for policy 0, policy_version 1115945 (0.0010) [2023-12-26 23:30:07,748][105692] Updated weights for policy 0, policy_version 1115955 (0.0011) [2023-12-26 23:30:07,807][105692] Updated weights for policy 0, policy_version 1115965 (0.0010) [2023-12-26 23:30:08,317][105620] Updated weights for policy 1, policy_version 1117242 (0.0010) [2023-12-26 23:30:08,374][105692] Updated weights for policy 0, policy_version 1115975 (0.0009) [2023-12-26 23:30:08,382][105620] Updated weights for policy 1, policy_version 1117252 (0.0008) [2023-12-26 23:30:08,436][105692] Updated weights for policy 0, policy_version 1115985 (0.0007) [2023-12-26 23:30:08,442][105620] Updated weights for policy 1, policy_version 1117262 (0.0007) [2023-12-26 23:30:08,489][105692] Updated weights for policy 0, policy_version 1115995 (0.0006) [2023-12-26 23:30:08,499][105620] Updated weights for policy 1, policy_version 1117272 (0.0008) [2023-12-26 23:30:09,237][105620] Updated weights for policy 1, policy_version 1117282 (0.0008) [2023-12-26 23:30:09,259][105692] Updated weights for policy 0, policy_version 1116005 (0.0010) [2023-12-26 23:30:09,294][105620] Updated weights for policy 1, policy_version 1117292 (0.0006) [2023-12-26 23:30:09,308][105692] Updated weights for policy 0, policy_version 1116015 (0.0010) [2023-12-26 23:30:09,347][105620] Updated weights for policy 1, policy_version 1117302 (0.0006) [2023-12-26 23:30:09,369][105692] Updated weights for policy 0, policy_version 1116025 (0.0010) [2023-12-26 23:30:10,153][105620] Updated weights for policy 1, policy_version 1117312 (0.0008) [2023-12-26 23:30:10,191][105692] Updated weights for policy 0, policy_version 1116035 (0.0009) [2023-12-26 23:30:10,219][105620] Updated weights for policy 1, policy_version 1117322 (0.0005) [2023-12-26 23:30:10,248][105692] Updated weights for policy 0, policy_version 1116045 (0.0009) [2023-12-26 23:30:10,279][105620] Updated weights for policy 1, policy_version 1117332 (0.0005) [2023-12-26 23:30:10,306][105692] Updated weights for policy 0, policy_version 1116055 (0.0009) [2023-12-26 23:30:10,840][105620] Updated weights for policy 1, policy_version 1117342 (0.0007) [2023-12-26 23:30:10,904][105620] Updated weights for policy 1, policy_version 1117352 (0.0011) [2023-12-26 23:30:10,911][105692] Updated weights for policy 0, policy_version 1116065 (0.0008) [2023-12-26 23:30:10,957][105620] Updated weights for policy 1, policy_version 1117362 (0.0010) [2023-12-26 23:30:10,974][105692] Updated weights for policy 0, policy_version 1116075 (0.0011) [2023-12-26 23:30:11,035][105692] Updated weights for policy 0, policy_version 1116085 (0.0010) [2023-12-26 23:30:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 571834368. Throughput: 0: 9617.6, 1: 9859.2. Samples: 571843780. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:30:11,062][104569] Avg episode reward: [(0, '9169.452'), (1, '9166.915')] [2023-12-26 23:30:11,103][105692] Updated weights for policy 0, policy_version 1116095 (0.0008) [2023-12-26 23:30:11,706][105620] Updated weights for policy 1, policy_version 1117372 (0.0010) [2023-12-26 23:30:11,777][105620] Updated weights for policy 1, policy_version 1117382 (0.0010) [2023-12-26 23:30:11,809][105692] Updated weights for policy 0, policy_version 1116105 (0.0009) [2023-12-26 23:30:11,841][105620] Updated weights for policy 1, policy_version 1117392 (0.0010) [2023-12-26 23:30:11,872][105692] Updated weights for policy 0, policy_version 1116115 (0.0010) [2023-12-26 23:30:11,940][105692] Updated weights for policy 0, policy_version 1116125 (0.0011) [2023-12-26 23:30:12,525][105620] Updated weights for policy 1, policy_version 1117402 (0.0010) [2023-12-26 23:30:12,594][105620] Updated weights for policy 1, policy_version 1117412 (0.0007) [2023-12-26 23:30:12,655][105620] Updated weights for policy 1, policy_version 1117422 (0.0009) [2023-12-26 23:30:12,665][105692] Updated weights for policy 0, policy_version 1116135 (0.0007) [2023-12-26 23:30:12,722][105620] Updated weights for policy 1, policy_version 1117432 (0.0008) [2023-12-26 23:30:12,724][105692] Updated weights for policy 0, policy_version 1116145 (0.0006) [2023-12-26 23:30:12,794][105692] Updated weights for policy 0, policy_version 1116155 (0.0010) [2023-12-26 23:30:13,354][105620] Updated weights for policy 1, policy_version 1117442 (0.0007) [2023-12-26 23:30:13,408][105620] Updated weights for policy 1, policy_version 1117452 (0.0007) [2023-12-26 23:30:13,460][105620] Updated weights for policy 1, policy_version 1117462 (0.0008) [2023-12-26 23:30:13,493][105692] Updated weights for policy 0, policy_version 1116165 (0.0011) [2023-12-26 23:30:13,538][105692] Updated weights for policy 0, policy_version 1116175 (0.0010) [2023-12-26 23:30:13,597][105692] Updated weights for policy 0, policy_version 1116185 (0.0010) [2023-12-26 23:30:14,127][105620] Updated weights for policy 1, policy_version 1117472 (0.0010) [2023-12-26 23:30:14,191][105620] Updated weights for policy 1, policy_version 1117482 (0.0010) [2023-12-26 23:30:14,254][105620] Updated weights for policy 1, policy_version 1117492 (0.0008) [2023-12-26 23:30:14,356][105692] Updated weights for policy 0, policy_version 1116195 (0.0010) [2023-12-26 23:30:14,415][105692] Updated weights for policy 0, policy_version 1116205 (0.0011) [2023-12-26 23:30:14,474][105692] Updated weights for policy 0, policy_version 1116215 (0.0011) [2023-12-26 23:30:14,937][105620] Updated weights for policy 1, policy_version 1117502 (0.0009) [2023-12-26 23:30:14,995][105620] Updated weights for policy 1, policy_version 1117512 (0.0009) [2023-12-26 23:30:15,062][105620] Updated weights for policy 1, policy_version 1117522 (0.0007) [2023-12-26 23:30:15,174][105692] Updated weights for policy 0, policy_version 1116225 (0.0010) [2023-12-26 23:30:15,234][105692] Updated weights for policy 0, policy_version 1116235 (0.0007) [2023-12-26 23:30:15,299][105692] Updated weights for policy 0, policy_version 1116245 (0.0009) [2023-12-26 23:30:15,355][105692] Updated weights for policy 0, policy_version 1116255 (0.0005) [2023-12-26 23:30:15,887][105620] Updated weights for policy 1, policy_version 1117532 (0.0009) [2023-12-26 23:30:15,946][105620] Updated weights for policy 1, policy_version 1117542 (0.0008) [2023-12-26 23:30:15,955][105692] Updated weights for policy 0, policy_version 1116265 (0.0006) [2023-12-26 23:30:16,002][105620] Updated weights for policy 1, policy_version 1117552 (0.0008) [2023-12-26 23:30:16,005][105692] Updated weights for policy 0, policy_version 1116275 (0.0007) [2023-12-26 23:30:16,058][105692] Updated weights for policy 0, policy_version 1116285 (0.0009) [2023-12-26 23:30:16,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.9, 300 sec: 19660.8). Total num frames: 571932672. Throughput: 0: 9548.6, 1: 9838.6. Samples: 571903392. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:30:16,062][104569] Avg episode reward: [(0, '8899.105'), (1, '9259.900')] [2023-12-26 23:30:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001117560_286130176.pth... [2023-12-26 23:30:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001116408_285835264.pth [2023-12-26 23:30:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001116288_285810688.pth... [2023-12-26 23:30:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001115168_285523968.pth [2023-12-26 23:30:16,650][105692] Updated weights for policy 0, policy_version 1116295 (0.0007) [2023-12-26 23:30:16,708][105692] Updated weights for policy 0, policy_version 1116305 (0.0005) [2023-12-26 23:30:16,757][105692] Updated weights for policy 0, policy_version 1116315 (0.0005) [2023-12-26 23:30:16,840][105620] Updated weights for policy 1, policy_version 1117562 (0.0006) [2023-12-26 23:30:16,896][105620] Updated weights for policy 1, policy_version 1117572 (0.0006) [2023-12-26 23:30:16,951][105620] Updated weights for policy 1, policy_version 1117582 (0.0005) [2023-12-26 23:30:17,003][105620] Updated weights for policy 1, policy_version 1117592 (0.0005) [2023-12-26 23:30:17,327][105692] Updated weights for policy 0, policy_version 1116325 (0.0005) [2023-12-26 23:30:17,388][105692] Updated weights for policy 0, policy_version 1116335 (0.0005) [2023-12-26 23:30:17,446][105692] Updated weights for policy 0, policy_version 1116345 (0.0005) [2023-12-26 23:30:17,624][105620] Updated weights for policy 1, policy_version 1117602 (0.0010) [2023-12-26 23:30:17,680][105620] Updated weights for policy 1, policy_version 1117612 (0.0010) [2023-12-26 23:30:17,738][105620] Updated weights for policy 1, policy_version 1117622 (0.0009) [2023-12-26 23:30:18,011][105692] Updated weights for policy 0, policy_version 1116355 (0.0007) [2023-12-26 23:30:18,070][105692] Updated weights for policy 0, policy_version 1116365 (0.0010) [2023-12-26 23:30:18,128][105692] Updated weights for policy 0, policy_version 1116375 (0.0010) [2023-12-26 23:30:18,417][105620] Updated weights for policy 1, policy_version 1117632 (0.0010) [2023-12-26 23:30:18,472][105620] Updated weights for policy 1, policy_version 1117642 (0.0010) [2023-12-26 23:30:18,520][105620] Updated weights for policy 1, policy_version 1117652 (0.0010) [2023-12-26 23:30:18,891][105692] Updated weights for policy 0, policy_version 1116385 (0.0010) [2023-12-26 23:30:18,953][105692] Updated weights for policy 0, policy_version 1116395 (0.0009) [2023-12-26 23:30:19,015][105692] Updated weights for policy 0, policy_version 1116405 (0.0008) [2023-12-26 23:30:19,073][105692] Updated weights for policy 0, policy_version 1116415 (0.0009) [2023-12-26 23:30:19,252][105620] Updated weights for policy 1, policy_version 1117662 (0.0012) [2023-12-26 23:30:19,308][105620] Updated weights for policy 1, policy_version 1117672 (0.0008) [2023-12-26 23:30:19,373][105620] Updated weights for policy 1, policy_version 1117682 (0.0009) [2023-12-26 23:30:19,851][105692] Updated weights for policy 0, policy_version 1116425 (0.0009) [2023-12-26 23:30:19,916][105692] Updated weights for policy 0, policy_version 1116435 (0.0010) [2023-12-26 23:30:19,986][105692] Updated weights for policy 0, policy_version 1116445 (0.0007) [2023-12-26 23:30:20,126][105620] Updated weights for policy 1, policy_version 1117692 (0.0008) [2023-12-26 23:30:20,198][105620] Updated weights for policy 1, policy_version 1117702 (0.0006) [2023-12-26 23:30:20,268][105620] Updated weights for policy 1, policy_version 1117712 (0.0006) [2023-12-26 23:30:20,629][105692] Updated weights for policy 0, policy_version 1116455 (0.0009) [2023-12-26 23:30:20,685][105692] Updated weights for policy 0, policy_version 1116465 (0.0009) [2023-12-26 23:30:20,740][105692] Updated weights for policy 0, policy_version 1116475 (0.0009) [2023-12-26 23:30:20,916][105620] Updated weights for policy 1, policy_version 1117722 (0.0009) [2023-12-26 23:30:20,972][105620] Updated weights for policy 1, policy_version 1117732 (0.0009) [2023-12-26 23:30:21,038][105620] Updated weights for policy 1, policy_version 1117742 (0.0009) [2023-12-26 23:30:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 572030976. Throughput: 0: 9627.0, 1: 9800.7. Samples: 572023084. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:30:21,062][104569] Avg episode reward: [(0, '8720.043'), (1, '9260.796')] [2023-12-26 23:30:21,093][105620] Updated weights for policy 1, policy_version 1117752 (0.0008) [2023-12-26 23:30:21,512][105692] Updated weights for policy 0, policy_version 1116485 (0.0009) [2023-12-26 23:30:21,576][105692] Updated weights for policy 0, policy_version 1116495 (0.0009) [2023-12-26 23:30:21,639][105692] Updated weights for policy 0, policy_version 1116505 (0.0010) [2023-12-26 23:30:21,874][105620] Updated weights for policy 1, policy_version 1117762 (0.0009) [2023-12-26 23:30:21,941][105620] Updated weights for policy 1, policy_version 1117772 (0.0009) [2023-12-26 23:30:22,004][105620] Updated weights for policy 1, policy_version 1117782 (0.0010) [2023-12-26 23:30:22,374][105692] Updated weights for policy 0, policy_version 1116515 (0.0009) [2023-12-26 23:30:22,441][105692] Updated weights for policy 0, policy_version 1116525 (0.0007) [2023-12-26 23:30:22,503][105692] Updated weights for policy 0, policy_version 1116535 (0.0006) [2023-12-26 23:30:22,850][105620] Updated weights for policy 1, policy_version 1117792 (0.0010) [2023-12-26 23:30:22,915][105620] Updated weights for policy 1, policy_version 1117802 (0.0010) [2023-12-26 23:30:22,969][105620] Updated weights for policy 1, policy_version 1117812 (0.0009) [2023-12-26 23:30:23,091][105692] Updated weights for policy 0, policy_version 1116545 (0.0006) [2023-12-26 23:30:23,144][105692] Updated weights for policy 0, policy_version 1116555 (0.0009) [2023-12-26 23:30:23,198][105692] Updated weights for policy 0, policy_version 1116565 (0.0009) [2023-12-26 23:30:23,259][105692] Updated weights for policy 0, policy_version 1116575 (0.0010) [2023-12-26 23:30:23,602][105620] Updated weights for policy 1, policy_version 1117822 (0.0007) [2023-12-26 23:30:23,663][105620] Updated weights for policy 1, policy_version 1117832 (0.0009) [2023-12-26 23:30:23,718][105620] Updated weights for policy 1, policy_version 1117842 (0.0009) [2023-12-26 23:30:23,941][105692] Updated weights for policy 0, policy_version 1116585 (0.0007) [2023-12-26 23:30:23,999][105692] Updated weights for policy 0, policy_version 1116595 (0.0009) [2023-12-26 23:30:24,051][105692] Updated weights for policy 0, policy_version 1116605 (0.0005) [2023-12-26 23:30:24,410][105620] Updated weights for policy 1, policy_version 1117852 (0.0009) [2023-12-26 23:30:24,464][105620] Updated weights for policy 1, policy_version 1117862 (0.0009) [2023-12-26 23:30:24,518][105620] Updated weights for policy 1, policy_version 1117872 (0.0009) [2023-12-26 23:30:24,739][105692] Updated weights for policy 0, policy_version 1116615 (0.0008) [2023-12-26 23:30:24,805][105692] Updated weights for policy 0, policy_version 1116625 (0.0009) [2023-12-26 23:30:24,867][105692] Updated weights for policy 0, policy_version 1116635 (0.0009) [2023-12-26 23:30:25,231][105620] Updated weights for policy 1, policy_version 1117882 (0.0008) [2023-12-26 23:30:25,282][105620] Updated weights for policy 1, policy_version 1117892 (0.0006) [2023-12-26 23:30:25,341][105620] Updated weights for policy 1, policy_version 1117902 (0.0006) [2023-12-26 23:30:25,386][105620] Updated weights for policy 1, policy_version 1117912 (0.0005) [2023-12-26 23:30:25,526][105692] Updated weights for policy 0, policy_version 1116645 (0.0009) [2023-12-26 23:30:25,592][105692] Updated weights for policy 0, policy_version 1116655 (0.0009) [2023-12-26 23:30:25,653][105692] Updated weights for policy 0, policy_version 1116665 (0.0009) [2023-12-26 23:30:25,981][105620] Updated weights for policy 1, policy_version 1117922 (0.0007) [2023-12-26 23:30:26,042][105620] Updated weights for policy 1, policy_version 1117932 (0.0009) [2023-12-26 23:30:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 572129280. Throughput: 0: 9637.7, 1: 9815.6. Samples: 572142096. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:30:26,063][104569] Avg episode reward: [(0, '8994.162'), (1, '9075.110')] [2023-12-26 23:30:26,108][105620] Updated weights for policy 1, policy_version 1117942 (0.0009) [2023-12-26 23:30:26,507][105692] Updated weights for policy 0, policy_version 1116675 (0.0010) [2023-12-26 23:30:26,571][105692] Updated weights for policy 0, policy_version 1116685 (0.0010) [2023-12-26 23:30:26,618][105692] Updated weights for policy 0, policy_version 1116695 (0.0010) [2023-12-26 23:30:26,709][105620] Updated weights for policy 1, policy_version 1117952 (0.0008) [2023-12-26 23:30:26,753][105620] Updated weights for policy 1, policy_version 1117962 (0.0008) [2023-12-26 23:30:26,800][105620] Updated weights for policy 1, policy_version 1117972 (0.0008) [2023-12-26 23:30:27,354][105692] Updated weights for policy 0, policy_version 1116705 (0.0010) [2023-12-26 23:30:27,415][105692] Updated weights for policy 0, policy_version 1116715 (0.0010) [2023-12-26 23:30:27,468][105692] Updated weights for policy 0, policy_version 1116725 (0.0010) [2023-12-26 23:30:27,532][105692] Updated weights for policy 0, policy_version 1116735 (0.0010) [2023-12-26 23:30:27,559][105620] Updated weights for policy 1, policy_version 1117982 (0.0008) [2023-12-26 23:30:27,607][105620] Updated weights for policy 1, policy_version 1117992 (0.0009) [2023-12-26 23:30:27,654][105620] Updated weights for policy 1, policy_version 1118002 (0.0010) [2023-12-26 23:30:28,141][105692] Updated weights for policy 0, policy_version 1116745 (0.0010) [2023-12-26 23:30:28,188][105692] Updated weights for policy 0, policy_version 1116755 (0.0010) [2023-12-26 23:30:28,237][105692] Updated weights for policy 0, policy_version 1116765 (0.0010) [2023-12-26 23:30:28,255][105620] Updated weights for policy 1, policy_version 1118012 (0.0010) [2023-12-26 23:30:28,308][105620] Updated weights for policy 1, policy_version 1118022 (0.0010) [2023-12-26 23:30:28,378][105620] Updated weights for policy 1, policy_version 1118032 (0.0007) [2023-12-26 23:30:29,002][105692] Updated weights for policy 0, policy_version 1116775 (0.0010) [2023-12-26 23:30:29,025][105620] Updated weights for policy 1, policy_version 1118042 (0.0009) [2023-12-26 23:30:29,056][105692] Updated weights for policy 0, policy_version 1116785 (0.0010) [2023-12-26 23:30:29,078][105620] Updated weights for policy 1, policy_version 1118052 (0.0007) [2023-12-26 23:30:29,107][105692] Updated weights for policy 0, policy_version 1116795 (0.0010) [2023-12-26 23:30:29,133][105620] Updated weights for policy 1, policy_version 1118062 (0.0005) [2023-12-26 23:30:29,180][105620] Updated weights for policy 1, policy_version 1118072 (0.0006) [2023-12-26 23:30:29,858][105692] Updated weights for policy 0, policy_version 1116805 (0.0010) [2023-12-26 23:30:29,908][105692] Updated weights for policy 0, policy_version 1116815 (0.0011) [2023-12-26 23:30:29,923][105620] Updated weights for policy 1, policy_version 1118082 (0.0007) [2023-12-26 23:30:29,971][105692] Updated weights for policy 0, policy_version 1116825 (0.0011) [2023-12-26 23:30:29,984][105620] Updated weights for policy 1, policy_version 1118092 (0.0006) [2023-12-26 23:30:30,042][105620] Updated weights for policy 1, policy_version 1118102 (0.0007) [2023-12-26 23:30:30,723][105692] Updated weights for policy 0, policy_version 1116835 (0.0011) [2023-12-26 23:30:30,770][105692] Updated weights for policy 0, policy_version 1116845 (0.0010) [2023-12-26 23:30:30,788][105620] Updated weights for policy 1, policy_version 1118112 (0.0009) [2023-12-26 23:30:30,821][105692] Updated weights for policy 0, policy_version 1116855 (0.0010) [2023-12-26 23:30:30,839][105620] Updated weights for policy 1, policy_version 1118122 (0.0007) [2023-12-26 23:30:30,901][105620] Updated weights for policy 1, policy_version 1118132 (0.0007) [2023-12-26 23:30:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 572235776. Throughput: 0: 9659.1, 1: 9890.5. Samples: 572202544. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:30:31,062][104569] Avg episode reward: [(0, '9086.652'), (1, '8711.688')] [2023-12-26 23:30:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001116864_285958144.pth... [2023-12-26 23:30:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001118136_286277632.pth... [2023-12-26 23:30:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001115712_285663232.pth [2023-12-26 23:30:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001116952_285974528.pth [2023-12-26 23:30:31,584][105692] Updated weights for policy 0, policy_version 1116865 (0.0010) [2023-12-26 23:30:31,650][105692] Updated weights for policy 0, policy_version 1116875 (0.0009) [2023-12-26 23:30:31,672][105620] Updated weights for policy 1, policy_version 1118142 (0.0006) [2023-12-26 23:30:31,705][105692] Updated weights for policy 0, policy_version 1116885 (0.0010) [2023-12-26 23:30:31,732][105620] Updated weights for policy 1, policy_version 1118152 (0.0006) [2023-12-26 23:30:31,768][105692] Updated weights for policy 0, policy_version 1116895 (0.0010) [2023-12-26 23:30:31,794][105620] Updated weights for policy 1, policy_version 1118162 (0.0008) [2023-12-26 23:30:32,515][105692] Updated weights for policy 0, policy_version 1116905 (0.0007) [2023-12-26 23:30:32,553][105620] Updated weights for policy 1, policy_version 1118172 (0.0008) [2023-12-26 23:30:32,573][105692] Updated weights for policy 0, policy_version 1116915 (0.0005) [2023-12-26 23:30:32,609][105620] Updated weights for policy 1, policy_version 1118182 (0.0008) [2023-12-26 23:30:32,629][105692] Updated weights for policy 0, policy_version 1116925 (0.0005) [2023-12-26 23:30:32,662][105620] Updated weights for policy 1, policy_version 1118192 (0.0009) [2023-12-26 23:30:33,196][105692] Updated weights for policy 0, policy_version 1116935 (0.0008) [2023-12-26 23:30:33,250][105692] Updated weights for policy 0, policy_version 1116945 (0.0009) [2023-12-26 23:30:33,298][105692] Updated weights for policy 0, policy_version 1116955 (0.0006) [2023-12-26 23:30:33,489][105620] Updated weights for policy 1, policy_version 1118202 (0.0010) [2023-12-26 23:30:33,548][105620] Updated weights for policy 1, policy_version 1118212 (0.0008) [2023-12-26 23:30:33,604][105620] Updated weights for policy 1, policy_version 1118222 (0.0008) [2023-12-26 23:30:33,655][105620] Updated weights for policy 1, policy_version 1118232 (0.0005) [2023-12-26 23:30:34,009][105692] Updated weights for policy 0, policy_version 1116965 (0.0006) [2023-12-26 23:30:34,060][105692] Updated weights for policy 0, policy_version 1116975 (0.0005) [2023-12-26 23:30:34,111][105692] Updated weights for policy 0, policy_version 1116985 (0.0005) [2023-12-26 23:30:34,402][105620] Updated weights for policy 1, policy_version 1118242 (0.0010) [2023-12-26 23:30:34,462][105620] Updated weights for policy 1, policy_version 1118252 (0.0011) [2023-12-26 23:30:34,518][105620] Updated weights for policy 1, policy_version 1118262 (0.0011) [2023-12-26 23:30:34,727][105692] Updated weights for policy 0, policy_version 1116995 (0.0006) [2023-12-26 23:30:34,777][105692] Updated weights for policy 0, policy_version 1117005 (0.0005) [2023-12-26 23:30:34,825][105692] Updated weights for policy 0, policy_version 1117015 (0.0006) [2023-12-26 23:30:35,196][105620] Updated weights for policy 1, policy_version 1118272 (0.0006) [2023-12-26 23:30:35,254][105620] Updated weights for policy 1, policy_version 1118282 (0.0007) [2023-12-26 23:30:35,309][105620] Updated weights for policy 1, policy_version 1118292 (0.0005) [2023-12-26 23:30:35,492][105692] Updated weights for policy 0, policy_version 1117026 (0.0009) [2023-12-26 23:30:35,541][105692] Updated weights for policy 0, policy_version 1117036 (0.0008) [2023-12-26 23:30:35,586][105692] Updated weights for policy 0, policy_version 1117046 (0.0008) [2023-12-26 23:30:35,645][105692] Updated weights for policy 0, policy_version 1117056 (0.0008) [2023-12-26 23:30:35,955][105620] Updated weights for policy 1, policy_version 1118302 (0.0005) [2023-12-26 23:30:36,004][105620] Updated weights for policy 1, policy_version 1118312 (0.0005) [2023-12-26 23:30:36,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.8, 300 sec: 19660.8). Total num frames: 572325888. Throughput: 0: 9694.6, 1: 9844.8. Samples: 572317624. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:30:36,062][104569] Avg episode reward: [(0, '8903.754'), (1, '8896.139')] [2023-12-26 23:30:36,063][105620] Updated weights for policy 1, policy_version 1118322 (0.0008) [2023-12-26 23:30:36,468][105692] Updated weights for policy 0, policy_version 1117066 (0.0007) [2023-12-26 23:30:36,528][105692] Updated weights for policy 0, policy_version 1117076 (0.0008) [2023-12-26 23:30:36,584][105692] Updated weights for policy 0, policy_version 1117086 (0.0008) [2023-12-26 23:30:36,767][105620] Updated weights for policy 1, policy_version 1118332 (0.0009) [2023-12-26 23:30:36,830][105620] Updated weights for policy 1, policy_version 1118342 (0.0011) [2023-12-26 23:30:36,894][105620] Updated weights for policy 1, policy_version 1118352 (0.0011) [2023-12-26 23:30:37,376][105692] Updated weights for policy 0, policy_version 1117096 (0.0008) [2023-12-26 23:30:37,433][105692] Updated weights for policy 0, policy_version 1117106 (0.0008) [2023-12-26 23:30:37,491][105692] Updated weights for policy 0, policy_version 1117116 (0.0008) [2023-12-26 23:30:37,635][105620] Updated weights for policy 1, policy_version 1118362 (0.0011) [2023-12-26 23:30:37,694][105620] Updated weights for policy 1, policy_version 1118372 (0.0010) [2023-12-26 23:30:37,759][105620] Updated weights for policy 1, policy_version 1118382 (0.0011) [2023-12-26 23:30:37,825][105620] Updated weights for policy 1, policy_version 1118392 (0.0011) [2023-12-26 23:30:38,258][105692] Updated weights for policy 0, policy_version 1117126 (0.0008) [2023-12-26 23:30:38,311][105692] Updated weights for policy 0, policy_version 1117136 (0.0008) [2023-12-26 23:30:38,373][105692] Updated weights for policy 0, policy_version 1117146 (0.0008) [2023-12-26 23:30:38,584][105620] Updated weights for policy 1, policy_version 1118402 (0.0011) [2023-12-26 23:30:38,649][105620] Updated weights for policy 1, policy_version 1118412 (0.0010) [2023-12-26 23:30:38,704][105620] Updated weights for policy 1, policy_version 1118422 (0.0010) [2023-12-26 23:30:39,136][105692] Updated weights for policy 0, policy_version 1117156 (0.0008) [2023-12-26 23:30:39,188][105692] Updated weights for policy 0, policy_version 1117166 (0.0008) [2023-12-26 23:30:39,251][105692] Updated weights for policy 0, policy_version 1117176 (0.0008) [2023-12-26 23:30:39,478][105620] Updated weights for policy 1, policy_version 1118432 (0.0011) [2023-12-26 23:30:39,538][105620] Updated weights for policy 1, policy_version 1118442 (0.0011) [2023-12-26 23:30:39,604][105620] Updated weights for policy 1, policy_version 1118452 (0.0011) [2023-12-26 23:30:40,086][105692] Updated weights for policy 0, policy_version 1117186 (0.0008) [2023-12-26 23:30:40,151][105692] Updated weights for policy 0, policy_version 1117196 (0.0008) [2023-12-26 23:30:40,218][105692] Updated weights for policy 0, policy_version 1117206 (0.0008) [2023-12-26 23:30:40,279][105692] Updated weights for policy 0, policy_version 1117216 (0.0008) [2023-12-26 23:30:40,355][105620] Updated weights for policy 1, policy_version 1118462 (0.0011) [2023-12-26 23:30:40,415][105620] Updated weights for policy 1, policy_version 1118472 (0.0011) [2023-12-26 23:30:40,464][105620] Updated weights for policy 1, policy_version 1118482 (0.0009) [2023-12-26 23:30:41,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19251.2, 300 sec: 19688.6). Total num frames: 572416000. Throughput: 0: 9715.5, 1: 9811.3. Samples: 572430636. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:30:41,063][104569] Avg episode reward: [(0, '9172.005'), (1, '8989.324')] [2023-12-26 23:30:41,076][105585] KL-divergence is very high: 131.5734 [2023-12-26 23:30:41,091][105585] KL-divergence is very high: 127.8863 [2023-12-26 23:30:41,097][105692] Updated weights for policy 0, policy_version 1117226 (0.0008) [2023-12-26 23:30:41,129][105620] Updated weights for policy 1, policy_version 1118492 (0.0008) [2023-12-26 23:30:41,132][105585] KL-divergence is very high: 251.8445 [2023-12-26 23:30:41,144][105585] KL-divergence is very high: 201.5674 [2023-12-26 23:30:41,160][105692] Updated weights for policy 0, policy_version 1117236 (0.0007) [2023-12-26 23:30:41,181][105585] KL-divergence is very high: 282.1383 [2023-12-26 23:30:41,193][105620] Updated weights for policy 1, policy_version 1118502 (0.0009) [2023-12-26 23:30:41,193][105585] KL-divergence is very high: 215.6421 [2023-12-26 23:30:41,220][105692] Updated weights for policy 0, policy_version 1117246 (0.0008) [2023-12-26 23:30:41,228][105585] KL-divergence is very high: 241.6543 [2023-12-26 23:30:41,257][105620] Updated weights for policy 1, policy_version 1118512 (0.0009) [2023-12-26 23:30:42,011][105692] Updated weights for policy 0, policy_version 1117256 (0.0008) [2023-12-26 23:30:42,053][105620] Updated weights for policy 1, policy_version 1118522 (0.0008) [2023-12-26 23:30:42,073][105692] Updated weights for policy 0, policy_version 1117266 (0.0008) [2023-12-26 23:30:42,115][105620] Updated weights for policy 1, policy_version 1118532 (0.0007) [2023-12-26 23:30:42,134][105692] Updated weights for policy 0, policy_version 1117276 (0.0008) [2023-12-26 23:30:42,175][105620] Updated weights for policy 1, policy_version 1118542 (0.0008) [2023-12-26 23:30:42,234][105620] Updated weights for policy 1, policy_version 1118552 (0.0009) [2023-12-26 23:30:42,938][105620] Updated weights for policy 1, policy_version 1118562 (0.0007) [2023-12-26 23:30:42,940][105692] Updated weights for policy 0, policy_version 1117286 (0.0008) [2023-12-26 23:30:42,997][105692] Updated weights for policy 0, policy_version 1117296 (0.0007) [2023-12-26 23:30:42,998][105620] Updated weights for policy 1, policy_version 1118572 (0.0007) [2023-12-26 23:30:43,049][105692] Updated weights for policy 0, policy_version 1117306 (0.0006) [2023-12-26 23:30:43,055][105620] Updated weights for policy 1, policy_version 1118582 (0.0007) [2023-12-26 23:30:43,708][105620] Updated weights for policy 1, policy_version 1118592 (0.0009) [2023-12-26 23:30:43,758][105620] Updated weights for policy 1, policy_version 1118602 (0.0006) [2023-12-26 23:30:43,807][105620] Updated weights for policy 1, policy_version 1118612 (0.0008) [2023-12-26 23:30:43,852][105692] Updated weights for policy 0, policy_version 1117316 (0.0009) [2023-12-26 23:30:43,899][105692] Updated weights for policy 0, policy_version 1117326 (0.0009) [2023-12-26 23:30:43,945][105692] Updated weights for policy 0, policy_version 1117336 (0.0009) [2023-12-26 23:30:44,528][105620] Updated weights for policy 1, policy_version 1118622 (0.0009) [2023-12-26 23:30:44,587][105620] Updated weights for policy 1, policy_version 1118632 (0.0009) [2023-12-26 23:30:44,649][105620] Updated weights for policy 1, policy_version 1118642 (0.0009) [2023-12-26 23:30:44,713][105692] Updated weights for policy 0, policy_version 1117346 (0.0008) [2023-12-26 23:30:44,782][105692] Updated weights for policy 0, policy_version 1117356 (0.0007) [2023-12-26 23:30:44,848][105692] Updated weights for policy 0, policy_version 1117366 (0.0009) [2023-12-26 23:30:44,913][105692] Updated weights for policy 0, policy_version 1117376 (0.0009) [2023-12-26 23:30:45,417][105620] Updated weights for policy 1, policy_version 1118652 (0.0008) [2023-12-26 23:30:45,479][105620] Updated weights for policy 1, policy_version 1118662 (0.0006) [2023-12-26 23:30:45,536][105620] Updated weights for policy 1, policy_version 1118672 (0.0008) [2023-12-26 23:30:45,602][105692] Updated weights for policy 0, policy_version 1117386 (0.0006) [2023-12-26 23:30:45,652][105692] Updated weights for policy 0, policy_version 1117396 (0.0009) [2023-12-26 23:30:45,705][105692] Updated weights for policy 0, policy_version 1117407 (0.0009) [2023-12-26 23:30:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19688.6). Total num frames: 572514304. Throughput: 0: 9686.5, 1: 9815.9. Samples: 572485460. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:30:46,062][104569] Avg episode reward: [(0, '9259.896'), (1, '8988.454')] [2023-12-26 23:30:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001117408_286097408.pth... [2023-12-26 23:30:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001118680_286416896.pth... [2023-12-26 23:30:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001116288_285810688.pth [2023-12-26 23:30:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001117560_286130176.pth [2023-12-26 23:30:46,169][105620] Updated weights for policy 1, policy_version 1118682 (0.0007) [2023-12-26 23:30:46,232][105620] Updated weights for policy 1, policy_version 1118692 (0.0009) [2023-12-26 23:30:46,283][105620] Updated weights for policy 1, policy_version 1118702 (0.0009) [2023-12-26 23:30:46,337][105620] Updated weights for policy 1, policy_version 1118712 (0.0007) [2023-12-26 23:30:46,351][105692] Updated weights for policy 0, policy_version 1117417 (0.0006) [2023-12-26 23:30:46,418][105692] Updated weights for policy 0, policy_version 1117427 (0.0006) [2023-12-26 23:30:46,479][105692] Updated weights for policy 0, policy_version 1117437 (0.0009) [2023-12-26 23:30:47,021][105620] Updated weights for policy 1, policy_version 1118722 (0.0010) [2023-12-26 23:30:47,067][105620] Updated weights for policy 1, policy_version 1118732 (0.0007) [2023-12-26 23:30:47,114][105620] Updated weights for policy 1, policy_version 1118742 (0.0005) [2023-12-26 23:30:47,206][105692] Updated weights for policy 0, policy_version 1117448 (0.0009) [2023-12-26 23:30:47,254][105692] Updated weights for policy 0, policy_version 1117458 (0.0009) [2023-12-26 23:30:47,302][105692] Updated weights for policy 0, policy_version 1117468 (0.0008) [2023-12-26 23:30:47,900][105692] Updated weights for policy 0, policy_version 1117478 (0.0009) [2023-12-26 23:30:47,941][105620] Updated weights for policy 1, policy_version 1118752 (0.0009) [2023-12-26 23:30:47,955][105692] Updated weights for policy 0, policy_version 1117488 (0.0007) [2023-12-26 23:30:48,001][105620] Updated weights for policy 1, policy_version 1118762 (0.0005) [2023-12-26 23:30:48,011][105692] Updated weights for policy 0, policy_version 1117498 (0.0010) [2023-12-26 23:30:48,060][105620] Updated weights for policy 1, policy_version 1118772 (0.0006) [2023-12-26 23:30:48,705][105692] Updated weights for policy 0, policy_version 1117508 (0.0009) [2023-12-26 23:30:48,770][105692] Updated weights for policy 0, policy_version 1117518 (0.0010) [2023-12-26 23:30:48,826][105692] Updated weights for policy 0, policy_version 1117528 (0.0009) [2023-12-26 23:30:48,849][105620] Updated weights for policy 1, policy_version 1118782 (0.0007) [2023-12-26 23:30:48,918][105620] Updated weights for policy 1, policy_version 1118792 (0.0008) [2023-12-26 23:30:48,983][105620] Updated weights for policy 1, policy_version 1118802 (0.0010) [2023-12-26 23:30:49,539][105692] Updated weights for policy 0, policy_version 1117538 (0.0008) [2023-12-26 23:30:49,594][105692] Updated weights for policy 0, policy_version 1117548 (0.0005) [2023-12-26 23:30:49,652][105692] Updated weights for policy 0, policy_version 1117558 (0.0007) [2023-12-26 23:30:49,722][105692] Updated weights for policy 0, policy_version 1117568 (0.0008) [2023-12-26 23:30:49,735][105620] Updated weights for policy 1, policy_version 1118812 (0.0010) [2023-12-26 23:30:49,796][105620] Updated weights for policy 1, policy_version 1118822 (0.0005) [2023-12-26 23:30:49,863][105620] Updated weights for policy 1, policy_version 1118832 (0.0007) [2023-12-26 23:30:50,455][105692] Updated weights for policy 0, policy_version 1117578 (0.0010) [2023-12-26 23:30:50,518][105692] Updated weights for policy 0, policy_version 1117588 (0.0010) [2023-12-26 23:30:50,535][105620] Updated weights for policy 1, policy_version 1118842 (0.0008) [2023-12-26 23:30:50,582][105692] Updated weights for policy 0, policy_version 1117598 (0.0008) [2023-12-26 23:30:50,601][105620] Updated weights for policy 1, policy_version 1118852 (0.0010) [2023-12-26 23:30:50,670][105620] Updated weights for policy 1, policy_version 1118862 (0.0011) [2023-12-26 23:30:50,730][105620] Updated weights for policy 1, policy_version 1118872 (0.0011) [2023-12-26 23:30:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19660.8). Total num frames: 572612608. Throughput: 0: 9791.0, 1: 9785.2. Samples: 572602828. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:30:51,063][104569] Avg episode reward: [(0, '8626.470'), (1, '9258.814')] [2023-12-26 23:30:51,418][105692] Updated weights for policy 0, policy_version 1117608 (0.0008) [2023-12-26 23:30:51,456][105620] Updated weights for policy 1, policy_version 1118882 (0.0010) [2023-12-26 23:30:51,478][105692] Updated weights for policy 0, policy_version 1117618 (0.0006) [2023-12-26 23:30:51,509][105620] Updated weights for policy 1, policy_version 1118892 (0.0011) [2023-12-26 23:30:51,541][105692] Updated weights for policy 0, policy_version 1117628 (0.0006) [2023-12-26 23:30:51,559][105620] Updated weights for policy 1, policy_version 1118902 (0.0010) [2023-12-26 23:30:52,209][105692] Updated weights for policy 0, policy_version 1117638 (0.0009) [2023-12-26 23:30:52,270][105692] Updated weights for policy 0, policy_version 1117648 (0.0011) [2023-12-26 23:30:52,327][105692] Updated weights for policy 0, policy_version 1117658 (0.0011) [2023-12-26 23:30:52,352][105620] Updated weights for policy 1, policy_version 1118912 (0.0008) [2023-12-26 23:30:52,421][105620] Updated weights for policy 1, policy_version 1118922 (0.0010) [2023-12-26 23:30:52,473][105620] Updated weights for policy 1, policy_version 1118932 (0.0008) [2023-12-26 23:30:53,106][105692] Updated weights for policy 0, policy_version 1117668 (0.0011) [2023-12-26 23:30:53,164][105692] Updated weights for policy 0, policy_version 1117678 (0.0010) [2023-12-26 23:30:53,226][105692] Updated weights for policy 0, policy_version 1117688 (0.0011) [2023-12-26 23:30:53,278][105620] Updated weights for policy 1, policy_version 1118942 (0.0007) [2023-12-26 23:30:53,326][105620] Updated weights for policy 1, policy_version 1118952 (0.0007) [2023-12-26 23:30:53,390][105620] Updated weights for policy 1, policy_version 1118962 (0.0005) [2023-12-26 23:30:53,842][105692] Updated weights for policy 0, policy_version 1117698 (0.0011) [2023-12-26 23:30:53,904][105692] Updated weights for policy 0, policy_version 1117708 (0.0010) [2023-12-26 23:30:53,963][105692] Updated weights for policy 0, policy_version 1117718 (0.0010) [2023-12-26 23:30:53,997][105620] Updated weights for policy 1, policy_version 1118972 (0.0006) [2023-12-26 23:30:54,020][105692] Updated weights for policy 0, policy_version 1117728 (0.0011) [2023-12-26 23:30:54,052][105620] Updated weights for policy 1, policy_version 1118982 (0.0006) [2023-12-26 23:30:54,107][105620] Updated weights for policy 1, policy_version 1118992 (0.0007) [2023-12-26 23:30:54,684][105692] Updated weights for policy 0, policy_version 1117738 (0.0009) [2023-12-26 23:30:54,741][105692] Updated weights for policy 0, policy_version 1117748 (0.0009) [2023-12-26 23:30:54,792][105692] Updated weights for policy 0, policy_version 1117758 (0.0008) [2023-12-26 23:30:54,835][105620] Updated weights for policy 1, policy_version 1119002 (0.0008) [2023-12-26 23:30:54,882][105620] Updated weights for policy 1, policy_version 1119012 (0.0009) [2023-12-26 23:30:54,930][105620] Updated weights for policy 1, policy_version 1119022 (0.0009) [2023-12-26 23:30:54,986][105620] Updated weights for policy 1, policy_version 1119032 (0.0009) [2023-12-26 23:30:55,552][105692] Updated weights for policy 0, policy_version 1117768 (0.0009) [2023-12-26 23:30:55,613][105692] Updated weights for policy 0, policy_version 1117778 (0.0006) [2023-12-26 23:30:55,673][105692] Updated weights for policy 0, policy_version 1117788 (0.0005) [2023-12-26 23:30:55,796][105620] Updated weights for policy 1, policy_version 1119042 (0.0009) [2023-12-26 23:30:55,847][105620] Updated weights for policy 1, policy_version 1119052 (0.0008) [2023-12-26 23:30:55,914][105620] Updated weights for policy 1, policy_version 1119062 (0.0008) [2023-12-26 23:30:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 572710912. Throughput: 0: 9757.9, 1: 9653.2. Samples: 572717276. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:30:56,062][104569] Avg episode reward: [(0, '8264.431'), (1, '9006.737')] [2023-12-26 23:30:56,281][105692] Updated weights for policy 0, policy_version 1117798 (0.0007) [2023-12-26 23:30:56,349][105692] Updated weights for policy 0, policy_version 1117808 (0.0009) [2023-12-26 23:30:56,394][105692] Updated weights for policy 0, policy_version 1117818 (0.0011) [2023-12-26 23:30:56,547][105620] Updated weights for policy 1, policy_version 1119072 (0.0008) [2023-12-26 23:30:56,608][105620] Updated weights for policy 1, policy_version 1119082 (0.0005) [2023-12-26 23:30:56,668][105620] Updated weights for policy 1, policy_version 1119092 (0.0005) [2023-12-26 23:30:57,094][105692] Updated weights for policy 0, policy_version 1117828 (0.0011) [2023-12-26 23:30:57,152][105692] Updated weights for policy 0, policy_version 1117838 (0.0011) [2023-12-26 23:30:57,204][105692] Updated weights for policy 0, policy_version 1117848 (0.0010) [2023-12-26 23:30:57,284][105620] Updated weights for policy 1, policy_version 1119102 (0.0009) [2023-12-26 23:30:57,351][105620] Updated weights for policy 1, policy_version 1119112 (0.0007) [2023-12-26 23:30:57,416][105620] Updated weights for policy 1, policy_version 1119122 (0.0009) [2023-12-26 23:30:57,965][105692] Updated weights for policy 0, policy_version 1117858 (0.0011) [2023-12-26 23:30:58,027][105692] Updated weights for policy 0, policy_version 1117868 (0.0010) [2023-12-26 23:30:58,082][105620] Updated weights for policy 1, policy_version 1119132 (0.0011) [2023-12-26 23:30:58,093][105692] Updated weights for policy 0, policy_version 1117878 (0.0011) [2023-12-26 23:30:58,145][105620] Updated weights for policy 1, policy_version 1119142 (0.0011) [2023-12-26 23:30:58,152][105692] Updated weights for policy 0, policy_version 1117888 (0.0010) [2023-12-26 23:30:58,215][105620] Updated weights for policy 1, policy_version 1119152 (0.0011) [2023-12-26 23:30:58,949][105692] Updated weights for policy 0, policy_version 1117898 (0.0008) [2023-12-26 23:30:58,999][105620] Updated weights for policy 1, policy_version 1119162 (0.0011) [2023-12-26 23:30:59,016][105692] Updated weights for policy 0, policy_version 1117908 (0.0008) [2023-12-26 23:30:59,062][105620] Updated weights for policy 1, policy_version 1119172 (0.0011) [2023-12-26 23:30:59,073][105692] Updated weights for policy 0, policy_version 1117918 (0.0008) [2023-12-26 23:30:59,122][105620] Updated weights for policy 1, policy_version 1119182 (0.0009) [2023-12-26 23:30:59,183][105620] Updated weights for policy 1, policy_version 1119192 (0.0010) [2023-12-26 23:30:59,865][105692] Updated weights for policy 0, policy_version 1117928 (0.0008) [2023-12-26 23:30:59,928][105692] Updated weights for policy 0, policy_version 1117938 (0.0006) [2023-12-26 23:30:59,952][105620] Updated weights for policy 1, policy_version 1119202 (0.0011) [2023-12-26 23:30:59,997][105692] Updated weights for policy 0, policy_version 1117948 (0.0008) [2023-12-26 23:31:00,017][105620] Updated weights for policy 1, policy_version 1119212 (0.0008) [2023-12-26 23:31:00,077][105620] Updated weights for policy 1, policy_version 1119222 (0.0009) [2023-12-26 23:31:00,757][105620] Updated weights for policy 1, policy_version 1119232 (0.0006) [2023-12-26 23:31:00,785][105692] Updated weights for policy 0, policy_version 1117958 (0.0009) [2023-12-26 23:31:00,803][105620] Updated weights for policy 1, policy_version 1119242 (0.0006) [2023-12-26 23:31:00,849][105692] Updated weights for policy 0, policy_version 1117968 (0.0009) [2023-12-26 23:31:00,865][105620] Updated weights for policy 1, policy_version 1119252 (0.0009) [2023-12-26 23:31:00,914][105692] Updated weights for policy 0, policy_version 1117978 (0.0007) [2023-12-26 23:31:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 572809216. Throughput: 0: 9761.3, 1: 9669.4. Samples: 572777776. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:31:01,063][104569] Avg episode reward: [(0, '8718.987'), (1, '8572.774')] [2023-12-26 23:31:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001117984_286244864.pth... [2023-12-26 23:31:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001119256_286564352.pth... [2023-12-26 23:31:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001118136_286277632.pth [2023-12-26 23:31:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001116864_285958144.pth [2023-12-26 23:31:01,563][105620] Updated weights for policy 1, policy_version 1119262 (0.0007) [2023-12-26 23:31:01,626][105692] Updated weights for policy 0, policy_version 1117988 (0.0011) [2023-12-26 23:31:01,626][105620] Updated weights for policy 1, policy_version 1119272 (0.0011) [2023-12-26 23:31:01,680][105620] Updated weights for policy 1, policy_version 1119282 (0.0010) [2023-12-26 23:31:01,685][105692] Updated weights for policy 0, policy_version 1117998 (0.0011) [2023-12-26 23:31:01,745][105692] Updated weights for policy 0, policy_version 1118008 (0.0009) [2023-12-26 23:31:02,434][105620] Updated weights for policy 1, policy_version 1119292 (0.0011) [2023-12-26 23:31:02,493][105620] Updated weights for policy 1, policy_version 1119302 (0.0010) [2023-12-26 23:31:02,497][105692] Updated weights for policy 0, policy_version 1118018 (0.0009) [2023-12-26 23:31:02,548][105692] Updated weights for policy 0, policy_version 1118028 (0.0007) [2023-12-26 23:31:02,549][105620] Updated weights for policy 1, policy_version 1119312 (0.0009) [2023-12-26 23:31:02,600][105692] Updated weights for policy 0, policy_version 1118038 (0.0005) [2023-12-26 23:31:02,650][105692] Updated weights for policy 0, policy_version 1118048 (0.0005) [2023-12-26 23:31:03,169][105620] Updated weights for policy 1, policy_version 1119322 (0.0005) [2023-12-26 23:31:03,203][105692] Updated weights for policy 0, policy_version 1118058 (0.0006) [2023-12-26 23:31:03,227][105620] Updated weights for policy 1, policy_version 1119332 (0.0007) [2023-12-26 23:31:03,252][105692] Updated weights for policy 0, policy_version 1118068 (0.0007) [2023-12-26 23:31:03,281][105620] Updated weights for policy 1, policy_version 1119342 (0.0006) [2023-12-26 23:31:03,300][105692] Updated weights for policy 0, policy_version 1118078 (0.0007) [2023-12-26 23:31:03,331][105620] Updated weights for policy 1, policy_version 1119352 (0.0010) [2023-12-26 23:31:03,936][105620] Updated weights for policy 1, policy_version 1119362 (0.0008) [2023-12-26 23:31:03,999][105620] Updated weights for policy 1, policy_version 1119372 (0.0008) [2023-12-26 23:31:04,052][105692] Updated weights for policy 0, policy_version 1118088 (0.0010) [2023-12-26 23:31:04,059][105620] Updated weights for policy 1, policy_version 1119382 (0.0008) [2023-12-26 23:31:04,116][105692] Updated weights for policy 0, policy_version 1118098 (0.0010) [2023-12-26 23:31:04,180][105692] Updated weights for policy 0, policy_version 1118108 (0.0011) [2023-12-26 23:31:04,836][105620] Updated weights for policy 1, policy_version 1119392 (0.0008) [2023-12-26 23:31:04,897][105620] Updated weights for policy 1, policy_version 1119402 (0.0008) [2023-12-26 23:31:04,929][105692] Updated weights for policy 0, policy_version 1118118 (0.0010) [2023-12-26 23:31:04,955][105620] Updated weights for policy 1, policy_version 1119412 (0.0008) [2023-12-26 23:31:04,987][105692] Updated weights for policy 0, policy_version 1118128 (0.0010) [2023-12-26 23:31:05,039][105692] Updated weights for policy 0, policy_version 1118138 (0.0010) [2023-12-26 23:31:05,622][105620] Updated weights for policy 1, policy_version 1119422 (0.0005) [2023-12-26 23:31:05,672][105620] Updated weights for policy 1, policy_version 1119432 (0.0005) [2023-12-26 23:31:05,726][105620] Updated weights for policy 1, policy_version 1119442 (0.0005) [2023-12-26 23:31:05,788][105692] Updated weights for policy 0, policy_version 1118148 (0.0010) [2023-12-26 23:31:05,849][105692] Updated weights for policy 0, policy_version 1118158 (0.0010) [2023-12-26 23:31:05,911][105692] Updated weights for policy 0, policy_version 1118168 (0.0010) [2023-12-26 23:31:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.7, 300 sec: 19688.6). Total num frames: 572907520. Throughput: 0: 9629.7, 1: 9707.7. Samples: 572893272. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:31:06,063][104569] Avg episode reward: [(0, '8624.762'), (1, '8605.715')] [2023-12-26 23:31:06,468][105620] Updated weights for policy 1, policy_version 1119452 (0.0009) [2023-12-26 23:31:06,536][105620] Updated weights for policy 1, policy_version 1119462 (0.0008) [2023-12-26 23:31:06,599][105620] Updated weights for policy 1, policy_version 1119472 (0.0008) [2023-12-26 23:31:06,689][105692] Updated weights for policy 0, policy_version 1118178 (0.0010) [2023-12-26 23:31:06,759][105692] Updated weights for policy 0, policy_version 1118188 (0.0011) [2023-12-26 23:31:06,816][105692] Updated weights for policy 0, policy_version 1118198 (0.0011) [2023-12-26 23:31:06,884][105692] Updated weights for policy 0, policy_version 1118208 (0.0011) [2023-12-26 23:31:07,276][105620] Updated weights for policy 1, policy_version 1119482 (0.0008) [2023-12-26 23:31:07,341][105620] Updated weights for policy 1, policy_version 1119492 (0.0008) [2023-12-26 23:31:07,405][105620] Updated weights for policy 1, policy_version 1119502 (0.0008) [2023-12-26 23:31:07,466][105620] Updated weights for policy 1, policy_version 1119512 (0.0007) [2023-12-26 23:31:07,590][105692] Updated weights for policy 0, policy_version 1118218 (0.0011) [2023-12-26 23:31:07,656][105692] Updated weights for policy 0, policy_version 1118228 (0.0010) [2023-12-26 23:31:07,721][105692] Updated weights for policy 0, policy_version 1118238 (0.0011) [2023-12-26 23:31:08,131][105620] Updated weights for policy 1, policy_version 1119522 (0.0009) [2023-12-26 23:31:08,192][105620] Updated weights for policy 1, policy_version 1119532 (0.0009) [2023-12-26 23:31:08,253][105620] Updated weights for policy 1, policy_version 1119542 (0.0009) [2023-12-26 23:31:08,418][105692] Updated weights for policy 0, policy_version 1118248 (0.0010) [2023-12-26 23:31:08,479][105692] Updated weights for policy 0, policy_version 1118258 (0.0008) [2023-12-26 23:31:08,538][105692] Updated weights for policy 0, policy_version 1118268 (0.0010) [2023-12-26 23:31:08,996][105620] Updated weights for policy 1, policy_version 1119552 (0.0009) [2023-12-26 23:31:09,053][105620] Updated weights for policy 1, policy_version 1119562 (0.0009) [2023-12-26 23:31:09,106][105620] Updated weights for policy 1, policy_version 1119572 (0.0009) [2023-12-26 23:31:09,296][105692] Updated weights for policy 0, policy_version 1118278 (0.0009) [2023-12-26 23:31:09,355][105692] Updated weights for policy 0, policy_version 1118288 (0.0010) [2023-12-26 23:31:09,440][105692] Updated weights for policy 0, policy_version 1118298 (0.0009) [2023-12-26 23:31:09,840][105620] Updated weights for policy 1, policy_version 1119582 (0.0009) [2023-12-26 23:31:09,906][105620] Updated weights for policy 1, policy_version 1119592 (0.0008) [2023-12-26 23:31:09,973][105620] Updated weights for policy 1, policy_version 1119602 (0.0007) [2023-12-26 23:31:10,192][105692] Updated weights for policy 0, policy_version 1118308 (0.0009) [2023-12-26 23:31:10,252][105692] Updated weights for policy 0, policy_version 1118318 (0.0010) [2023-12-26 23:31:10,307][105692] Updated weights for policy 0, policy_version 1118328 (0.0008) [2023-12-26 23:31:10,712][105620] Updated weights for policy 1, policy_version 1119612 (0.0008) [2023-12-26 23:31:10,779][105620] Updated weights for policy 1, policy_version 1119622 (0.0010) [2023-12-26 23:31:10,833][105620] Updated weights for policy 1, policy_version 1119632 (0.0010) [2023-12-26 23:31:10,906][105692] Updated weights for policy 0, policy_version 1118338 (0.0011) [2023-12-26 23:31:10,957][105692] Updated weights for policy 0, policy_version 1118348 (0.0010) [2023-12-26 23:31:11,012][105692] Updated weights for policy 0, policy_version 1118358 (0.0010) [2023-12-26 23:31:11,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 572997632. Throughput: 0: 9549.0, 1: 9676.6. Samples: 573007252. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:31:11,063][104569] Avg episode reward: [(0, '8987.117'), (1, '8409.523')] [2023-12-26 23:31:11,076][105692] Updated weights for policy 0, policy_version 1118368 (0.0011) [2023-12-26 23:31:11,663][105620] Updated weights for policy 1, policy_version 1119642 (0.0009) [2023-12-26 23:31:11,732][105620] Updated weights for policy 1, policy_version 1119652 (0.0008) [2023-12-26 23:31:11,791][105620] Updated weights for policy 1, policy_version 1119662 (0.0008) [2023-12-26 23:31:11,845][105620] Updated weights for policy 1, policy_version 1119672 (0.0006) [2023-12-26 23:31:11,863][105692] Updated weights for policy 0, policy_version 1118378 (0.0011) [2023-12-26 23:31:11,919][105692] Updated weights for policy 0, policy_version 1118388 (0.0007) [2023-12-26 23:31:11,982][105692] Updated weights for policy 0, policy_version 1118398 (0.0006) [2023-12-26 23:31:12,588][105692] Updated weights for policy 0, policy_version 1118408 (0.0010) [2023-12-26 23:31:12,648][105692] Updated weights for policy 0, policy_version 1118418 (0.0011) [2023-12-26 23:31:12,686][105620] Updated weights for policy 1, policy_version 1119682 (0.0006) [2023-12-26 23:31:12,701][105692] Updated weights for policy 0, policy_version 1118428 (0.0009) [2023-12-26 23:31:12,745][105620] Updated weights for policy 1, policy_version 1119692 (0.0006) [2023-12-26 23:31:12,809][105620] Updated weights for policy 1, policy_version 1119702 (0.0008) [2023-12-26 23:31:13,497][105692] Updated weights for policy 0, policy_version 1118438 (0.0010) [2023-12-26 23:31:13,515][105620] Updated weights for policy 1, policy_version 1119712 (0.0010) [2023-12-26 23:31:13,554][105692] Updated weights for policy 0, policy_version 1118448 (0.0007) [2023-12-26 23:31:13,572][105620] Updated weights for policy 1, policy_version 1119722 (0.0007) [2023-12-26 23:31:13,614][105692] Updated weights for policy 0, policy_version 1118458 (0.0008) [2023-12-26 23:31:13,630][105620] Updated weights for policy 1, policy_version 1119732 (0.0006) [2023-12-26 23:31:14,176][105620] Updated weights for policy 1, policy_version 1119742 (0.0007) [2023-12-26 23:31:14,239][105620] Updated weights for policy 1, policy_version 1119752 (0.0007) [2023-12-26 23:31:14,284][105620] Updated weights for policy 1, policy_version 1119762 (0.0005) [2023-12-26 23:31:14,520][105692] Updated weights for policy 0, policy_version 1118468 (0.0009) [2023-12-26 23:31:14,574][105692] Updated weights for policy 0, policy_version 1118478 (0.0010) [2023-12-26 23:31:14,643][105692] Updated weights for policy 0, policy_version 1118488 (0.0010) [2023-12-26 23:31:14,862][105620] Updated weights for policy 1, policy_version 1119772 (0.0007) [2023-12-26 23:31:14,930][105620] Updated weights for policy 1, policy_version 1119782 (0.0011) [2023-12-26 23:31:15,001][105620] Updated weights for policy 1, policy_version 1119792 (0.0011) [2023-12-26 23:31:15,367][105692] Updated weights for policy 0, policy_version 1118498 (0.0009) [2023-12-26 23:31:15,428][105692] Updated weights for policy 0, policy_version 1118508 (0.0009) [2023-12-26 23:31:15,476][105692] Updated weights for policy 0, policy_version 1118518 (0.0008) [2023-12-26 23:31:15,531][105692] Updated weights for policy 0, policy_version 1118528 (0.0008) [2023-12-26 23:31:15,704][105620] Updated weights for policy 1, policy_version 1119802 (0.0009) [2023-12-26 23:31:15,754][105620] Updated weights for policy 1, policy_version 1119812 (0.0005) [2023-12-26 23:31:15,812][105620] Updated weights for policy 1, policy_version 1119822 (0.0005) [2023-12-26 23:31:15,866][105620] Updated weights for policy 1, policy_version 1119832 (0.0005) [2023-12-26 23:31:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 573095936. Throughput: 0: 9570.1, 1: 9606.0. Samples: 573065472. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:31:16,063][104569] Avg episode reward: [(0, '9171.714'), (1, '8815.706')] [2023-12-26 23:31:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001119832_286711808.pth... [2023-12-26 23:31:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001118528_286384128.pth... [2023-12-26 23:31:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001118680_286416896.pth [2023-12-26 23:31:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001117408_286097408.pth [2023-12-26 23:31:16,403][105692] Updated weights for policy 0, policy_version 1118538 (0.0008) [2023-12-26 23:31:16,443][105620] Updated weights for policy 1, policy_version 1119842 (0.0007) [2023-12-26 23:31:16,470][105692] Updated weights for policy 0, policy_version 1118548 (0.0007) [2023-12-26 23:31:16,497][105620] Updated weights for policy 1, policy_version 1119852 (0.0007) [2023-12-26 23:31:16,536][105692] Updated weights for policy 0, policy_version 1118558 (0.0008) [2023-12-26 23:31:16,548][105620] Updated weights for policy 1, policy_version 1119862 (0.0005) [2023-12-26 23:31:17,226][105620] Updated weights for policy 1, policy_version 1119872 (0.0006) [2023-12-26 23:31:17,248][105692] Updated weights for policy 0, policy_version 1118568 (0.0007) [2023-12-26 23:31:17,289][105620] Updated weights for policy 1, policy_version 1119882 (0.0005) [2023-12-26 23:31:17,311][105692] Updated weights for policy 0, policy_version 1118578 (0.0010) [2023-12-26 23:31:17,341][105620] Updated weights for policy 1, policy_version 1119892 (0.0005) [2023-12-26 23:31:17,377][105692] Updated weights for policy 0, policy_version 1118588 (0.0011) [2023-12-26 23:31:18,025][105620] Updated weights for policy 1, policy_version 1119902 (0.0009) [2023-12-26 23:31:18,050][105692] Updated weights for policy 0, policy_version 1118598 (0.0011) [2023-12-26 23:31:18,084][105620] Updated weights for policy 1, policy_version 1119912 (0.0010) [2023-12-26 23:31:18,098][105692] Updated weights for policy 0, policy_version 1118608 (0.0006) [2023-12-26 23:31:18,145][105620] Updated weights for policy 1, policy_version 1119922 (0.0011) [2023-12-26 23:31:18,148][105692] Updated weights for policy 0, policy_version 1118618 (0.0011) [2023-12-26 23:31:18,790][105692] Updated weights for policy 0, policy_version 1118628 (0.0011) [2023-12-26 23:31:18,846][105692] Updated weights for policy 0, policy_version 1118638 (0.0011) [2023-12-26 23:31:18,874][105620] Updated weights for policy 1, policy_version 1119932 (0.0010) [2023-12-26 23:31:18,906][105692] Updated weights for policy 0, policy_version 1118648 (0.0011) [2023-12-26 23:31:18,933][105620] Updated weights for policy 1, policy_version 1119942 (0.0008) [2023-12-26 23:31:18,997][105620] Updated weights for policy 1, policy_version 1119952 (0.0007) [2023-12-26 23:31:19,668][105692] Updated weights for policy 0, policy_version 1118658 (0.0010) [2023-12-26 23:31:19,724][105692] Updated weights for policy 0, policy_version 1118668 (0.0009) [2023-12-26 23:31:19,764][105620] Updated weights for policy 1, policy_version 1119962 (0.0008) [2023-12-26 23:31:19,779][105692] Updated weights for policy 0, policy_version 1118678 (0.0009) [2023-12-26 23:31:19,833][105620] Updated weights for policy 1, policy_version 1119972 (0.0007) [2023-12-26 23:31:19,849][105692] Updated weights for policy 0, policy_version 1118688 (0.0008) [2023-12-26 23:31:19,906][105620] Updated weights for policy 1, policy_version 1119982 (0.0007) [2023-12-26 23:31:19,965][105620] Updated weights for policy 1, policy_version 1119992 (0.0009) [2023-12-26 23:31:20,570][105620] Updated weights for policy 1, policy_version 1120002 (0.0008) [2023-12-26 23:31:20,639][105620] Updated weights for policy 1, policy_version 1120012 (0.0009) [2023-12-26 23:31:20,650][105692] Updated weights for policy 0, policy_version 1118698 (0.0008) [2023-12-26 23:31:20,702][105620] Updated weights for policy 1, policy_version 1120022 (0.0009) [2023-12-26 23:31:20,713][105692] Updated weights for policy 0, policy_version 1118708 (0.0006) [2023-12-26 23:31:20,784][105692] Updated weights for policy 0, policy_version 1118718 (0.0007) [2023-12-26 23:31:21,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 573194240. Throughput: 0: 9485.7, 1: 9736.5. Samples: 573182624. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:31:21,062][104569] Avg episode reward: [(0, '8810.712'), (1, '8991.586')] [2023-12-26 23:31:21,495][105692] Updated weights for policy 0, policy_version 1118728 (0.0007) [2023-12-26 23:31:21,497][105620] Updated weights for policy 1, policy_version 1120032 (0.0006) [2023-12-26 23:31:21,553][105692] Updated weights for policy 0, policy_version 1118738 (0.0006) [2023-12-26 23:31:21,555][105620] Updated weights for policy 1, policy_version 1120042 (0.0008) [2023-12-26 23:31:21,604][105692] Updated weights for policy 0, policy_version 1118748 (0.0007) [2023-12-26 23:31:21,606][105620] Updated weights for policy 1, policy_version 1120052 (0.0006) [2023-12-26 23:31:22,428][105692] Updated weights for policy 0, policy_version 1118758 (0.0007) [2023-12-26 23:31:22,431][105620] Updated weights for policy 1, policy_version 1120062 (0.0008) [2023-12-26 23:31:22,490][105692] Updated weights for policy 0, policy_version 1118768 (0.0007) [2023-12-26 23:31:22,501][105620] Updated weights for policy 1, policy_version 1120072 (0.0006) [2023-12-26 23:31:22,552][105692] Updated weights for policy 0, policy_version 1118778 (0.0007) [2023-12-26 23:31:22,558][105620] Updated weights for policy 1, policy_version 1120082 (0.0009) [2023-12-26 23:31:23,273][105692] Updated weights for policy 0, policy_version 1118788 (0.0007) [2023-12-26 23:31:23,296][105620] Updated weights for policy 1, policy_version 1120092 (0.0008) [2023-12-26 23:31:23,322][105692] Updated weights for policy 0, policy_version 1118798 (0.0009) [2023-12-26 23:31:23,353][105620] Updated weights for policy 1, policy_version 1120102 (0.0007) [2023-12-26 23:31:23,376][105692] Updated weights for policy 0, policy_version 1118808 (0.0007) [2023-12-26 23:31:23,401][105620] Updated weights for policy 1, policy_version 1120112 (0.0006) [2023-12-26 23:31:24,042][105620] Updated weights for policy 1, policy_version 1120122 (0.0009) [2023-12-26 23:31:24,107][105620] Updated weights for policy 1, policy_version 1120132 (0.0010) [2023-12-26 23:31:24,168][105620] Updated weights for policy 1, policy_version 1120142 (0.0010) [2023-12-26 23:31:24,189][105692] Updated weights for policy 0, policy_version 1118818 (0.0008) [2023-12-26 23:31:24,223][105620] Updated weights for policy 1, policy_version 1120152 (0.0009) [2023-12-26 23:31:24,242][105692] Updated weights for policy 0, policy_version 1118828 (0.0006) [2023-12-26 23:31:24,290][105692] Updated weights for policy 0, policy_version 1118838 (0.0008) [2023-12-26 23:31:24,338][105692] Updated weights for policy 0, policy_version 1118848 (0.0008) [2023-12-26 23:31:24,955][105620] Updated weights for policy 1, policy_version 1120162 (0.0010) [2023-12-26 23:31:25,017][105620] Updated weights for policy 1, policy_version 1120172 (0.0010) [2023-12-26 23:31:25,074][105620] Updated weights for policy 1, policy_version 1120182 (0.0010) [2023-12-26 23:31:25,121][105692] Updated weights for policy 0, policy_version 1118858 (0.0007) [2023-12-26 23:31:25,180][105692] Updated weights for policy 0, policy_version 1118868 (0.0008) [2023-12-26 23:31:25,228][105692] Updated weights for policy 0, policy_version 1118878 (0.0008) [2023-12-26 23:31:25,800][105620] Updated weights for policy 1, policy_version 1120192 (0.0010) [2023-12-26 23:31:25,862][105620] Updated weights for policy 1, policy_version 1120202 (0.0010) [2023-12-26 23:31:25,921][105620] Updated weights for policy 1, policy_version 1120212 (0.0010) [2023-12-26 23:31:25,995][105692] Updated weights for policy 0, policy_version 1118888 (0.0009) [2023-12-26 23:31:26,043][105692] Updated weights for policy 0, policy_version 1118898 (0.0008) [2023-12-26 23:31:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 573284352. Throughput: 0: 9469.5, 1: 9728.8. Samples: 573294560. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:31:26,063][104569] Avg episode reward: [(0, '8721.847'), (1, '8988.305')] [2023-12-26 23:31:26,092][105692] Updated weights for policy 0, policy_version 1118908 (0.0008) [2023-12-26 23:31:26,653][105620] Updated weights for policy 1, policy_version 1120222 (0.0010) [2023-12-26 23:31:26,697][105620] Updated weights for policy 1, policy_version 1120232 (0.0010) [2023-12-26 23:31:26,750][105620] Updated weights for policy 1, policy_version 1120242 (0.0010) [2023-12-26 23:31:26,884][105692] Updated weights for policy 0, policy_version 1118918 (0.0008) [2023-12-26 23:31:26,943][105692] Updated weights for policy 0, policy_version 1118928 (0.0008) [2023-12-26 23:31:26,992][105692] Updated weights for policy 0, policy_version 1118938 (0.0008) [2023-12-26 23:31:27,470][105620] Updated weights for policy 1, policy_version 1120252 (0.0010) [2023-12-26 23:31:27,518][105620] Updated weights for policy 1, policy_version 1120262 (0.0010) [2023-12-26 23:31:27,572][105620] Updated weights for policy 1, policy_version 1120272 (0.0010) [2023-12-26 23:31:27,672][105692] Updated weights for policy 0, policy_version 1118948 (0.0007) [2023-12-26 23:31:27,705][105585] KL-divergence is very high: 156.3054 [2023-12-26 23:31:27,722][105585] KL-divergence is very high: 132.2113 [2023-12-26 23:31:27,729][105692] Updated weights for policy 0, policy_version 1118958 (0.0007) [2023-12-26 23:31:27,751][105585] KL-divergence is very high: 247.4550 [2023-12-26 23:31:27,757][105585] KL-divergence is very high: 104.5729 [2023-12-26 23:31:27,768][105585] KL-divergence is very high: 134.0734 [2023-12-26 23:31:27,784][105692] Updated weights for policy 0, policy_version 1118968 (0.0007) [2023-12-26 23:31:27,797][105585] KL-divergence is very high: 229.5233 [2023-12-26 23:31:28,325][105620] Updated weights for policy 1, policy_version 1120282 (0.0010) [2023-12-26 23:31:28,389][105620] Updated weights for policy 1, policy_version 1120292 (0.0011) [2023-12-26 23:31:28,450][105620] Updated weights for policy 1, policy_version 1120302 (0.0010) [2023-12-26 23:31:28,495][105620] Updated weights for policy 1, policy_version 1120312 (0.0010) [2023-12-26 23:31:28,512][105692] Updated weights for policy 0, policy_version 1118978 (0.0008) [2023-12-26 23:31:28,559][105692] Updated weights for policy 0, policy_version 1118988 (0.0008) [2023-12-26 23:31:28,609][105692] Updated weights for policy 0, policy_version 1118998 (0.0008) [2023-12-26 23:31:28,662][105692] Updated weights for policy 0, policy_version 1119008 (0.0008) [2023-12-26 23:31:29,203][105620] Updated weights for policy 1, policy_version 1120322 (0.0010) [2023-12-26 23:31:29,269][105620] Updated weights for policy 1, policy_version 1120332 (0.0008) [2023-12-26 23:31:29,337][105620] Updated weights for policy 1, policy_version 1120342 (0.0008) [2023-12-26 23:31:29,523][105692] Updated weights for policy 0, policy_version 1119020 (0.0011) [2023-12-26 23:31:29,578][105692] Updated weights for policy 0, policy_version 1119032 (0.0011) [2023-12-26 23:31:29,974][105620] Updated weights for policy 1, policy_version 1120352 (0.0008) [2023-12-26 23:31:30,036][105620] Updated weights for policy 1, policy_version 1120362 (0.0008) [2023-12-26 23:31:30,103][105620] Updated weights for policy 1, policy_version 1120372 (0.0007) [2023-12-26 23:31:30,460][105692] Updated weights for policy 0, policy_version 1119043 (0.0010) [2023-12-26 23:31:30,526][105692] Updated weights for policy 0, policy_version 1119053 (0.0010) [2023-12-26 23:31:30,590][105692] Updated weights for policy 0, policy_version 1119063 (0.0010) [2023-12-26 23:31:30,726][105620] Updated weights for policy 1, policy_version 1120382 (0.0009) [2023-12-26 23:31:30,777][105620] Updated weights for policy 1, policy_version 1120392 (0.0009) [2023-12-26 23:31:30,829][105620] Updated weights for policy 1, policy_version 1120402 (0.0010) [2023-12-26 23:31:31,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19114.6, 300 sec: 19605.3). Total num frames: 573382656. Throughput: 0: 9533.5, 1: 9732.8. Samples: 573352448. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:31:31,063][104569] Avg episode reward: [(0, '8628.515'), (1, '9082.336')] [2023-12-26 23:31:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001119072_286523392.pth... [2023-12-26 23:31:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001120408_286859264.pth... [2023-12-26 23:31:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001117984_286244864.pth [2023-12-26 23:31:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001119256_286564352.pth [2023-12-26 23:31:31,272][105692] Updated weights for policy 0, policy_version 1119073 (0.0009) [2023-12-26 23:31:31,321][105692] Updated weights for policy 0, policy_version 1119083 (0.0008) [2023-12-26 23:31:31,383][105692] Updated weights for policy 0, policy_version 1119093 (0.0008) [2023-12-26 23:31:31,454][105692] Updated weights for policy 0, policy_version 1119103 (0.0008) [2023-12-26 23:31:31,514][105620] Updated weights for policy 1, policy_version 1120412 (0.0008) [2023-12-26 23:31:31,574][105620] Updated weights for policy 1, policy_version 1120422 (0.0006) [2023-12-26 23:31:31,634][105620] Updated weights for policy 1, policy_version 1120432 (0.0008) [2023-12-26 23:31:32,183][105692] Updated weights for policy 0, policy_version 1119113 (0.0006) [2023-12-26 23:31:32,249][105692] Updated weights for policy 0, policy_version 1119123 (0.0009) [2023-12-26 23:31:32,307][105620] Updated weights for policy 1, policy_version 1120442 (0.0008) [2023-12-26 23:31:32,311][105692] Updated weights for policy 0, policy_version 1119133 (0.0008) [2023-12-26 23:31:32,375][105620] Updated weights for policy 1, policy_version 1120452 (0.0008) [2023-12-26 23:31:32,436][105620] Updated weights for policy 1, policy_version 1120462 (0.0008) [2023-12-26 23:31:32,505][105620] Updated weights for policy 1, policy_version 1120472 (0.0007) [2023-12-26 23:31:33,096][105692] Updated weights for policy 0, policy_version 1119143 (0.0008) [2023-12-26 23:31:33,139][105620] Updated weights for policy 1, policy_version 1120482 (0.0008) [2023-12-26 23:31:33,145][105692] Updated weights for policy 0, policy_version 1119153 (0.0007) [2023-12-26 23:31:33,191][105620] Updated weights for policy 1, policy_version 1120492 (0.0007) [2023-12-26 23:31:33,193][105692] Updated weights for policy 0, policy_version 1119163 (0.0005) [2023-12-26 23:31:33,252][105620] Updated weights for policy 1, policy_version 1120502 (0.0009) [2023-12-26 23:31:33,939][105692] Updated weights for policy 0, policy_version 1119173 (0.0007) [2023-12-26 23:31:33,964][105620] Updated weights for policy 1, policy_version 1120512 (0.0007) [2023-12-26 23:31:33,986][105692] Updated weights for policy 0, policy_version 1119183 (0.0007) [2023-12-26 23:31:34,016][105620] Updated weights for policy 1, policy_version 1120522 (0.0008) [2023-12-26 23:31:34,047][105692] Updated weights for policy 0, policy_version 1119193 (0.0007) [2023-12-26 23:31:34,082][105620] Updated weights for policy 1, policy_version 1120532 (0.0006) [2023-12-26 23:31:34,705][105620] Updated weights for policy 1, policy_version 1120542 (0.0009) [2023-12-26 23:31:34,753][105620] Updated weights for policy 1, policy_version 1120552 (0.0009) [2023-12-26 23:31:34,815][105620] Updated weights for policy 1, policy_version 1120562 (0.0007) [2023-12-26 23:31:34,889][105692] Updated weights for policy 0, policy_version 1119203 (0.0009) [2023-12-26 23:31:34,941][105692] Updated weights for policy 0, policy_version 1119214 (0.0009) [2023-12-26 23:31:34,993][105692] Updated weights for policy 0, policy_version 1119224 (0.0009) [2023-12-26 23:31:35,495][105620] Updated weights for policy 1, policy_version 1120572 (0.0006) [2023-12-26 23:31:35,549][105620] Updated weights for policy 1, policy_version 1120582 (0.0006) [2023-12-26 23:31:35,608][105620] Updated weights for policy 1, policy_version 1120592 (0.0006) [2023-12-26 23:31:35,846][105692] Updated weights for policy 0, policy_version 1119234 (0.0009) [2023-12-26 23:31:35,909][105692] Updated weights for policy 0, policy_version 1119244 (0.0009) [2023-12-26 23:31:35,971][105692] Updated weights for policy 0, policy_version 1119254 (0.0009) [2023-12-26 23:31:36,029][105692] Updated weights for policy 0, policy_version 1119264 (0.0009) [2023-12-26 23:31:36,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19251.1, 300 sec: 19633.0). Total num frames: 573480960. Throughput: 0: 9400.3, 1: 9838.2. Samples: 573468564. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:31:36,063][104569] Avg episode reward: [(0, '8626.635'), (1, '8990.895')] [2023-12-26 23:31:36,266][105620] Updated weights for policy 1, policy_version 1120602 (0.0006) [2023-12-26 23:31:36,334][105620] Updated weights for policy 1, policy_version 1120612 (0.0009) [2023-12-26 23:31:36,400][105620] Updated weights for policy 1, policy_version 1120622 (0.0011) [2023-12-26 23:31:36,459][105620] Updated weights for policy 1, policy_version 1120632 (0.0010) [2023-12-26 23:31:36,807][105692] Updated weights for policy 0, policy_version 1119274 (0.0008) [2023-12-26 23:31:36,865][105692] Updated weights for policy 0, policy_version 1119284 (0.0009) [2023-12-26 23:31:36,925][105692] Updated weights for policy 0, policy_version 1119295 (0.0010) [2023-12-26 23:31:37,124][105620] Updated weights for policy 1, policy_version 1120642 (0.0010) [2023-12-26 23:31:37,184][105620] Updated weights for policy 1, policy_version 1120652 (0.0010) [2023-12-26 23:31:37,243][105620] Updated weights for policy 1, policy_version 1120662 (0.0011) [2023-12-26 23:31:37,714][105692] Updated weights for policy 0, policy_version 1119305 (0.0006) [2023-12-26 23:31:37,773][105692] Updated weights for policy 0, policy_version 1119315 (0.0005) [2023-12-26 23:31:37,826][105692] Updated weights for policy 0, policy_version 1119325 (0.0005) [2023-12-26 23:31:37,972][105620] Updated weights for policy 1, policy_version 1120672 (0.0006) [2023-12-26 23:31:38,031][105620] Updated weights for policy 1, policy_version 1120682 (0.0006) [2023-12-26 23:31:38,086][105620] Updated weights for policy 1, policy_version 1120692 (0.0006) [2023-12-26 23:31:38,555][105692] Updated weights for policy 0, policy_version 1119335 (0.0009) [2023-12-26 23:31:38,618][105692] Updated weights for policy 0, policy_version 1119345 (0.0006) [2023-12-26 23:31:38,678][105692] Updated weights for policy 0, policy_version 1119355 (0.0005) [2023-12-26 23:31:38,700][105620] Updated weights for policy 1, policy_version 1120702 (0.0008) [2023-12-26 23:31:38,758][105620] Updated weights for policy 1, policy_version 1120712 (0.0010) [2023-12-26 23:31:38,821][105620] Updated weights for policy 1, policy_version 1120722 (0.0010) [2023-12-26 23:31:39,397][105692] Updated weights for policy 0, policy_version 1119365 (0.0007) [2023-12-26 23:31:39,458][105692] Updated weights for policy 0, policy_version 1119375 (0.0007) [2023-12-26 23:31:39,522][105692] Updated weights for policy 0, policy_version 1119385 (0.0006) [2023-12-26 23:31:39,574][105620] Updated weights for policy 1, policy_version 1120732 (0.0008) [2023-12-26 23:31:39,645][105620] Updated weights for policy 1, policy_version 1120742 (0.0010) [2023-12-26 23:31:39,711][105620] Updated weights for policy 1, policy_version 1120752 (0.0009) [2023-12-26 23:31:40,163][105692] Updated weights for policy 0, policy_version 1119395 (0.0006) [2023-12-26 23:31:40,229][105692] Updated weights for policy 0, policy_version 1119405 (0.0007) [2023-12-26 23:31:40,292][105692] Updated weights for policy 0, policy_version 1119415 (0.0007) [2023-12-26 23:31:40,469][105620] Updated weights for policy 1, policy_version 1120762 (0.0009) [2023-12-26 23:31:40,530][105620] Updated weights for policy 1, policy_version 1120772 (0.0008) [2023-12-26 23:31:40,588][105620] Updated weights for policy 1, policy_version 1120782 (0.0008) [2023-12-26 23:31:40,649][105620] Updated weights for policy 1, policy_version 1120792 (0.0009) [2023-12-26 23:31:40,986][105692] Updated weights for policy 0, policy_version 1119425 (0.0009) [2023-12-26 23:31:41,055][105692] Updated weights for policy 0, policy_version 1119435 (0.0009) [2023-12-26 23:31:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 573571072. Throughput: 0: 9365.4, 1: 9871.7. Samples: 573582948. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:31:41,063][104569] Avg episode reward: [(0, '8717.620'), (1, '8989.131')] [2023-12-26 23:31:41,117][105692] Updated weights for policy 0, policy_version 1119445 (0.0009) [2023-12-26 23:31:41,187][105692] Updated weights for policy 0, policy_version 1119455 (0.0009) [2023-12-26 23:31:41,444][105620] Updated weights for policy 1, policy_version 1120802 (0.0008) [2023-12-26 23:31:41,505][105620] Updated weights for policy 1, policy_version 1120812 (0.0006) [2023-12-26 23:31:41,576][105620] Updated weights for policy 1, policy_version 1120822 (0.0006) [2023-12-26 23:31:41,948][105692] Updated weights for policy 0, policy_version 1119465 (0.0009) [2023-12-26 23:31:42,000][105692] Updated weights for policy 0, policy_version 1119475 (0.0009) [2023-12-26 23:31:42,059][105692] Updated weights for policy 0, policy_version 1119485 (0.0009) [2023-12-26 23:31:42,256][105620] Updated weights for policy 1, policy_version 1120832 (0.0008) [2023-12-26 23:31:42,323][105620] Updated weights for policy 1, policy_version 1120842 (0.0008) [2023-12-26 23:31:42,393][105620] Updated weights for policy 1, policy_version 1120852 (0.0007) [2023-12-26 23:31:42,804][105692] Updated weights for policy 0, policy_version 1119495 (0.0009) [2023-12-26 23:31:42,869][105692] Updated weights for policy 0, policy_version 1119505 (0.0007) [2023-12-26 23:31:42,936][105692] Updated weights for policy 0, policy_version 1119515 (0.0008) [2023-12-26 23:31:43,116][105620] Updated weights for policy 1, policy_version 1120862 (0.0010) [2023-12-26 23:31:43,192][105620] Updated weights for policy 1, policy_version 1120872 (0.0008) [2023-12-26 23:31:43,258][105620] Updated weights for policy 1, policy_version 1120882 (0.0009) [2023-12-26 23:31:43,558][105692] Updated weights for policy 0, policy_version 1119526 (0.0009) [2023-12-26 23:31:43,616][105692] Updated weights for policy 0, policy_version 1119536 (0.0010) [2023-12-26 23:31:43,674][105692] Updated weights for policy 0, policy_version 1119546 (0.0010) [2023-12-26 23:31:43,887][105620] Updated weights for policy 1, policy_version 1120892 (0.0008) [2023-12-26 23:31:43,940][105620] Updated weights for policy 1, policy_version 1120902 (0.0010) [2023-12-26 23:31:43,997][105620] Updated weights for policy 1, policy_version 1120912 (0.0010) [2023-12-26 23:31:44,428][105692] Updated weights for policy 0, policy_version 1119556 (0.0009) [2023-12-26 23:31:44,491][105692] Updated weights for policy 0, policy_version 1119566 (0.0009) [2023-12-26 23:31:44,549][105692] Updated weights for policy 0, policy_version 1119576 (0.0009) [2023-12-26 23:31:44,745][105620] Updated weights for policy 1, policy_version 1120922 (0.0009) [2023-12-26 23:31:44,805][105620] Updated weights for policy 1, policy_version 1120932 (0.0008) [2023-12-26 23:31:44,860][105620] Updated weights for policy 1, policy_version 1120942 (0.0007) [2023-12-26 23:31:44,907][105620] Updated weights for policy 1, policy_version 1120952 (0.0009) [2023-12-26 23:31:45,330][105692] Updated weights for policy 0, policy_version 1119586 (0.0009) [2023-12-26 23:31:45,394][105692] Updated weights for policy 0, policy_version 1119596 (0.0009) [2023-12-26 23:31:45,462][105692] Updated weights for policy 0, policy_version 1119606 (0.0009) [2023-12-26 23:31:45,528][105692] Updated weights for policy 0, policy_version 1119616 (0.0009) [2023-12-26 23:31:45,619][105620] Updated weights for policy 1, policy_version 1120962 (0.0008) [2023-12-26 23:31:45,685][105620] Updated weights for policy 1, policy_version 1120972 (0.0008) [2023-12-26 23:31:45,748][105620] Updated weights for policy 1, policy_version 1120982 (0.0008) [2023-12-26 23:31:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.1, 300 sec: 19633.0). Total num frames: 573669376. Throughput: 0: 9342.5, 1: 9842.6. Samples: 573641112. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:31:46,063][104569] Avg episode reward: [(0, '8901.619'), (1, '9080.615')] [2023-12-26 23:31:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001119616_286662656.pth... [2023-12-26 23:31:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001120984_287006720.pth... [2023-12-26 23:31:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001118528_286384128.pth [2023-12-26 23:31:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001119832_286711808.pth [2023-12-26 23:31:46,309][105692] Updated weights for policy 0, policy_version 1119626 (0.0005) [2023-12-26 23:31:46,323][105620] Updated weights for policy 1, policy_version 1120992 (0.0010) [2023-12-26 23:31:46,361][105692] Updated weights for policy 0, policy_version 1119636 (0.0006) [2023-12-26 23:31:46,378][105620] Updated weights for policy 1, policy_version 1121002 (0.0010) [2023-12-26 23:31:46,420][105692] Updated weights for policy 0, policy_version 1119646 (0.0006) [2023-12-26 23:31:46,434][105620] Updated weights for policy 1, policy_version 1121012 (0.0011) [2023-12-26 23:31:47,160][105692] Updated weights for policy 0, policy_version 1119656 (0.0008) [2023-12-26 23:31:47,180][105620] Updated weights for policy 1, policy_version 1121022 (0.0010) [2023-12-26 23:31:47,215][105692] Updated weights for policy 0, policy_version 1119666 (0.0009) [2023-12-26 23:31:47,241][105620] Updated weights for policy 1, policy_version 1121032 (0.0009) [2023-12-26 23:31:47,274][105692] Updated weights for policy 0, policy_version 1119676 (0.0007) [2023-12-26 23:31:47,296][105620] Updated weights for policy 1, policy_version 1121042 (0.0010) [2023-12-26 23:31:47,978][105620] Updated weights for policy 1, policy_version 1121052 (0.0010) [2023-12-26 23:31:48,035][105620] Updated weights for policy 1, policy_version 1121062 (0.0008) [2023-12-26 23:31:48,061][105692] Updated weights for policy 0, policy_version 1119686 (0.0007) [2023-12-26 23:31:48,095][105620] Updated weights for policy 1, policy_version 1121072 (0.0008) [2023-12-26 23:31:48,127][105692] Updated weights for policy 0, policy_version 1119696 (0.0008) [2023-12-26 23:31:48,185][105692] Updated weights for policy 0, policy_version 1119706 (0.0008) [2023-12-26 23:31:48,869][105620] Updated weights for policy 1, policy_version 1121082 (0.0009) [2023-12-26 23:31:48,931][105620] Updated weights for policy 1, policy_version 1121092 (0.0008) [2023-12-26 23:31:48,941][105692] Updated weights for policy 0, policy_version 1119716 (0.0009) [2023-12-26 23:31:48,994][105620] Updated weights for policy 1, policy_version 1121102 (0.0005) [2023-12-26 23:31:48,996][105692] Updated weights for policy 0, policy_version 1119726 (0.0011) [2023-12-26 23:31:49,047][105620] Updated weights for policy 1, policy_version 1121112 (0.0006) [2023-12-26 23:31:49,059][105692] Updated weights for policy 0, policy_version 1119736 (0.0010) [2023-12-26 23:31:49,715][105620] Updated weights for policy 1, policy_version 1121122 (0.0008) [2023-12-26 23:31:49,779][105620] Updated weights for policy 1, policy_version 1121132 (0.0008) [2023-12-26 23:31:49,848][105620] Updated weights for policy 1, policy_version 1121142 (0.0007) [2023-12-26 23:31:49,891][105692] Updated weights for policy 0, policy_version 1119746 (0.0010) [2023-12-26 23:31:49,961][105692] Updated weights for policy 0, policy_version 1119756 (0.0009) [2023-12-26 23:31:50,026][105692] Updated weights for policy 0, policy_version 1119766 (0.0011) [2023-12-26 23:31:50,089][105692] Updated weights for policy 0, policy_version 1119776 (0.0011) [2023-12-26 23:31:50,683][105620] Updated weights for policy 1, policy_version 1121152 (0.0008) [2023-12-26 23:31:50,708][105692] Updated weights for policy 0, policy_version 1119786 (0.0008) [2023-12-26 23:31:50,735][105620] Updated weights for policy 1, policy_version 1121162 (0.0007) [2023-12-26 23:31:50,771][105692] Updated weights for policy 0, policy_version 1119796 (0.0007) [2023-12-26 23:31:50,792][105620] Updated weights for policy 1, policy_version 1121172 (0.0007) [2023-12-26 23:31:50,825][105692] Updated weights for policy 0, policy_version 1119806 (0.0007) [2023-12-26 23:31:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 573767680. Throughput: 0: 9285.5, 1: 9847.9. Samples: 573754268. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:31:51,062][104569] Avg episode reward: [(0, '8993.296'), (1, '8986.817')] [2023-12-26 23:31:51,538][105620] Updated weights for policy 1, policy_version 1121182 (0.0008) [2023-12-26 23:31:51,595][105620] Updated weights for policy 1, policy_version 1121192 (0.0008) [2023-12-26 23:31:51,618][105692] Updated weights for policy 0, policy_version 1119816 (0.0007) [2023-12-26 23:31:51,658][105620] Updated weights for policy 1, policy_version 1121202 (0.0008) [2023-12-26 23:31:51,681][105692] Updated weights for policy 0, policy_version 1119826 (0.0008) [2023-12-26 23:31:51,735][105692] Updated weights for policy 0, policy_version 1119836 (0.0009) [2023-12-26 23:31:52,448][105692] Updated weights for policy 0, policy_version 1119846 (0.0010) [2023-12-26 23:31:52,482][105620] Updated weights for policy 1, policy_version 1121212 (0.0009) [2023-12-26 23:31:52,502][105692] Updated weights for policy 0, policy_version 1119856 (0.0009) [2023-12-26 23:31:52,546][105620] Updated weights for policy 1, policy_version 1121222 (0.0011) [2023-12-26 23:31:52,563][105692] Updated weights for policy 0, policy_version 1119866 (0.0005) [2023-12-26 23:31:52,612][105620] Updated weights for policy 1, policy_version 1121232 (0.0011) [2023-12-26 23:31:53,203][105692] Updated weights for policy 0, policy_version 1119876 (0.0005) [2023-12-26 23:31:53,255][105692] Updated weights for policy 0, policy_version 1119886 (0.0009) [2023-12-26 23:31:53,306][105620] Updated weights for policy 1, policy_version 1121242 (0.0011) [2023-12-26 23:31:53,310][105692] Updated weights for policy 0, policy_version 1119896 (0.0010) [2023-12-26 23:31:53,362][105620] Updated weights for policy 1, policy_version 1121252 (0.0011) [2023-12-26 23:31:53,418][105620] Updated weights for policy 1, policy_version 1121262 (0.0011) [2023-12-26 23:31:53,473][105620] Updated weights for policy 1, policy_version 1121272 (0.0010) [2023-12-26 23:31:53,983][105692] Updated weights for policy 0, policy_version 1119906 (0.0011) [2023-12-26 23:31:54,038][105692] Updated weights for policy 0, policy_version 1119916 (0.0010) [2023-12-26 23:31:54,094][105692] Updated weights for policy 0, policy_version 1119926 (0.0010) [2023-12-26 23:31:54,144][105692] Updated weights for policy 0, policy_version 1119936 (0.0010) [2023-12-26 23:31:54,205][105620] Updated weights for policy 1, policy_version 1121282 (0.0005) [2023-12-26 23:31:54,262][105620] Updated weights for policy 1, policy_version 1121292 (0.0005) [2023-12-26 23:31:54,318][105620] Updated weights for policy 1, policy_version 1121302 (0.0005) [2023-12-26 23:31:54,865][105620] Updated weights for policy 1, policy_version 1121312 (0.0009) [2023-12-26 23:31:54,916][105692] Updated weights for policy 0, policy_version 1119946 (0.0007) [2023-12-26 23:31:54,919][105620] Updated weights for policy 1, policy_version 1121322 (0.0009) [2023-12-26 23:31:54,981][105692] Updated weights for policy 0, policy_version 1119956 (0.0006) [2023-12-26 23:31:54,982][105620] Updated weights for policy 1, policy_version 1121332 (0.0006) [2023-12-26 23:31:55,041][105692] Updated weights for policy 0, policy_version 1119966 (0.0007) [2023-12-26 23:31:55,664][105620] Updated weights for policy 1, policy_version 1121342 (0.0007) [2023-12-26 23:31:55,722][105692] Updated weights for policy 0, policy_version 1119976 (0.0006) [2023-12-26 23:31:55,727][105620] Updated weights for policy 1, policy_version 1121352 (0.0008) [2023-12-26 23:31:55,768][105692] Updated weights for policy 0, policy_version 1119986 (0.0009) [2023-12-26 23:31:55,777][105620] Updated weights for policy 1, policy_version 1121362 (0.0005) [2023-12-26 23:31:55,826][105692] Updated weights for policy 0, policy_version 1119996 (0.0009) [2023-12-26 23:31:56,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 573865984. Throughput: 0: 9341.5, 1: 9863.3. Samples: 573871468. Policy #0 lag: (min: 6.0, avg: 12.8, max: 38.0) [2023-12-26 23:31:56,063][104569] Avg episode reward: [(0, '8902.169'), (1, '8897.002')] [2023-12-26 23:31:56,509][105692] Updated weights for policy 0, policy_version 1120006 (0.0008) [2023-12-26 23:31:56,534][105620] Updated weights for policy 1, policy_version 1121372 (0.0007) [2023-12-26 23:31:56,569][105692] Updated weights for policy 0, policy_version 1120016 (0.0009) [2023-12-26 23:31:56,590][105620] Updated weights for policy 1, policy_version 1121382 (0.0008) [2023-12-26 23:31:56,633][105692] Updated weights for policy 0, policy_version 1120026 (0.0007) [2023-12-26 23:31:56,650][105620] Updated weights for policy 1, policy_version 1121392 (0.0009) [2023-12-26 23:31:57,323][105692] Updated weights for policy 0, policy_version 1120036 (0.0008) [2023-12-26 23:31:57,380][105692] Updated weights for policy 0, policy_version 1120046 (0.0006) [2023-12-26 23:31:57,416][105620] Updated weights for policy 1, policy_version 1121402 (0.0008) [2023-12-26 23:31:57,434][105692] Updated weights for policy 0, policy_version 1120056 (0.0008) [2023-12-26 23:31:57,461][105620] Updated weights for policy 1, policy_version 1121412 (0.0006) [2023-12-26 23:31:57,504][105620] Updated weights for policy 1, policy_version 1121422 (0.0008) [2023-12-26 23:31:57,549][105620] Updated weights for policy 1, policy_version 1121432 (0.0006) [2023-12-26 23:31:58,147][105620] Updated weights for policy 1, policy_version 1121442 (0.0010) [2023-12-26 23:31:58,212][105620] Updated weights for policy 1, policy_version 1121452 (0.0010) [2023-12-26 23:31:58,248][105692] Updated weights for policy 0, policy_version 1120066 (0.0007) [2023-12-26 23:31:58,277][105620] Updated weights for policy 1, policy_version 1121462 (0.0008) [2023-12-26 23:31:58,310][105692] Updated weights for policy 0, policy_version 1120076 (0.0007) [2023-12-26 23:31:58,377][105692] Updated weights for policy 0, policy_version 1120086 (0.0008) [2023-12-26 23:31:58,440][105692] Updated weights for policy 0, policy_version 1120096 (0.0009) [2023-12-26 23:31:59,031][105620] Updated weights for policy 1, policy_version 1121472 (0.0006) [2023-12-26 23:31:59,102][105620] Updated weights for policy 1, policy_version 1121482 (0.0005) [2023-12-26 23:31:59,161][105620] Updated weights for policy 1, policy_version 1121492 (0.0006) [2023-12-26 23:31:59,241][105692] Updated weights for policy 0, policy_version 1120106 (0.0008) [2023-12-26 23:31:59,294][105692] Updated weights for policy 0, policy_version 1120116 (0.0006) [2023-12-26 23:31:59,357][105692] Updated weights for policy 0, policy_version 1120126 (0.0009) [2023-12-26 23:31:59,813][105620] Updated weights for policy 1, policy_version 1121502 (0.0008) [2023-12-26 23:31:59,876][105620] Updated weights for policy 1, policy_version 1121512 (0.0008) [2023-12-26 23:31:59,927][105620] Updated weights for policy 1, policy_version 1121522 (0.0008) [2023-12-26 23:32:00,118][105692] Updated weights for policy 0, policy_version 1120136 (0.0010) [2023-12-26 23:32:00,171][105692] Updated weights for policy 0, policy_version 1120146 (0.0011) [2023-12-26 23:32:00,220][105692] Updated weights for policy 0, policy_version 1120156 (0.0008) [2023-12-26 23:32:00,661][105620] Updated weights for policy 1, policy_version 1121532 (0.0007) [2023-12-26 23:32:00,726][105620] Updated weights for policy 1, policy_version 1121542 (0.0008) [2023-12-26 23:32:00,773][105620] Updated weights for policy 1, policy_version 1121552 (0.0008) [2023-12-26 23:32:00,944][105692] Updated weights for policy 0, policy_version 1120166 (0.0009) [2023-12-26 23:32:00,952][105585] KL-divergence is very high: 113.9901 [2023-12-26 23:32:00,987][105585] KL-divergence is very high: 186.1406 [2023-12-26 23:32:00,988][105692] Updated weights for policy 0, policy_version 1120176 (0.0010) [2023-12-26 23:32:01,028][105585] KL-divergence is very high: 167.7080 [2023-12-26 23:32:01,040][105692] Updated weights for policy 0, policy_version 1120186 (0.0010) [2023-12-26 23:32:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19549.7). Total num frames: 573956096. Throughput: 0: 9318.6, 1: 9868.1. Samples: 573928872. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:32:01,062][104569] Avg episode reward: [(0, '8807.892'), (1, '9171.566')] [2023-12-26 23:32:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001121560_287154176.pth... [2023-12-26 23:32:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001120408_286859264.pth [2023-12-26 23:32:01,078][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001120192_286810112.pth... [2023-12-26 23:32:01,082][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001119072_286523392.pth [2023-12-26 23:32:01,541][105620] Updated weights for policy 1, policy_version 1121562 (0.0006) [2023-12-26 23:32:01,591][105620] Updated weights for policy 1, policy_version 1121572 (0.0008) [2023-12-26 23:32:01,646][105620] Updated weights for policy 1, policy_version 1121582 (0.0008) [2023-12-26 23:32:01,702][105620] Updated weights for policy 1, policy_version 1121592 (0.0008) [2023-12-26 23:32:01,831][105692] Updated weights for policy 0, policy_version 1120196 (0.0009) [2023-12-26 23:32:01,893][105692] Updated weights for policy 0, policy_version 1120206 (0.0010) [2023-12-26 23:32:01,962][105692] Updated weights for policy 0, policy_version 1120216 (0.0011) [2023-12-26 23:32:02,454][105620] Updated weights for policy 1, policy_version 1121602 (0.0009) [2023-12-26 23:32:02,507][105620] Updated weights for policy 1, policy_version 1121612 (0.0009) [2023-12-26 23:32:02,541][105692] Updated weights for policy 0, policy_version 1120226 (0.0010) [2023-12-26 23:32:02,567][105620] Updated weights for policy 1, policy_version 1121622 (0.0009) [2023-12-26 23:32:02,593][105692] Updated weights for policy 0, policy_version 1120236 (0.0005) [2023-12-26 23:32:02,649][105692] Updated weights for policy 0, policy_version 1120246 (0.0005) [2023-12-26 23:32:02,708][105692] Updated weights for policy 0, policy_version 1120256 (0.0005) [2023-12-26 23:32:03,306][105620] Updated weights for policy 1, policy_version 1121632 (0.0006) [2023-12-26 23:32:03,349][105692] Updated weights for policy 0, policy_version 1120266 (0.0005) [2023-12-26 23:32:03,351][105620] Updated weights for policy 1, policy_version 1121642 (0.0005) [2023-12-26 23:32:03,417][105620] Updated weights for policy 1, policy_version 1121652 (0.0005) [2023-12-26 23:32:03,418][105692] Updated weights for policy 0, policy_version 1120276 (0.0005) [2023-12-26 23:32:03,478][105692] Updated weights for policy 0, policy_version 1120286 (0.0006) [2023-12-26 23:32:03,998][105620] Updated weights for policy 1, policy_version 1121662 (0.0007) [2023-12-26 23:32:04,012][105692] Updated weights for policy 0, policy_version 1120296 (0.0006) [2023-12-26 23:32:04,060][105620] Updated weights for policy 1, policy_version 1121672 (0.0008) [2023-12-26 23:32:04,063][105692] Updated weights for policy 0, policy_version 1120306 (0.0007) [2023-12-26 23:32:04,116][105692] Updated weights for policy 0, policy_version 1120316 (0.0006) [2023-12-26 23:32:04,118][105620] Updated weights for policy 1, policy_version 1121682 (0.0010) [2023-12-26 23:32:04,810][105620] Updated weights for policy 1, policy_version 1121692 (0.0007) [2023-12-26 23:32:04,878][105620] Updated weights for policy 1, policy_version 1121702 (0.0006) [2023-12-26 23:32:04,882][105692] Updated weights for policy 0, policy_version 1120326 (0.0007) [2023-12-26 23:32:04,930][105692] Updated weights for policy 0, policy_version 1120336 (0.0007) [2023-12-26 23:32:04,937][105620] Updated weights for policy 1, policy_version 1121712 (0.0006) [2023-12-26 23:32:04,984][105692] Updated weights for policy 0, policy_version 1120346 (0.0008) [2023-12-26 23:32:05,523][105620] Updated weights for policy 1, policy_version 1121722 (0.0006) [2023-12-26 23:32:05,578][105620] Updated weights for policy 1, policy_version 1121732 (0.0005) [2023-12-26 23:32:05,633][105620] Updated weights for policy 1, policy_version 1121742 (0.0005) [2023-12-26 23:32:05,691][105620] Updated weights for policy 1, policy_version 1121752 (0.0005) [2023-12-26 23:32:05,754][105692] Updated weights for policy 0, policy_version 1120357 (0.0010) [2023-12-26 23:32:05,822][105692] Updated weights for policy 0, policy_version 1120367 (0.0009) [2023-12-26 23:32:05,858][105585] KL-divergence is very high: 143.1899 [2023-12-26 23:32:05,887][105585] KL-divergence is very high: 192.2077 [2023-12-26 23:32:05,889][105692] Updated weights for policy 0, policy_version 1120377 (0.0010) [2023-12-26 23:32:05,905][105585] KL-divergence is very high: 253.5910 [2023-12-26 23:32:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.3, 300 sec: 19577.5). Total num frames: 574062592. Throughput: 0: 9427.0, 1: 9824.8. Samples: 574048956. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:32:06,062][104569] Avg episode reward: [(0, '8718.405'), (1, '9170.407')] [2023-12-26 23:32:06,223][105620] Updated weights for policy 1, policy_version 1121762 (0.0010) [2023-12-26 23:32:06,276][105620] Updated weights for policy 1, policy_version 1121772 (0.0008) [2023-12-26 23:32:06,343][105620] Updated weights for policy 1, policy_version 1121782 (0.0010) [2023-12-26 23:32:06,579][105585] KL-divergence is very high: 257.1595 [2023-12-26 23:32:06,591][105692] Updated weights for policy 0, policy_version 1120387 (0.0008) [2023-12-26 23:32:06,625][105585] KL-divergence is very high: 203.8385 [2023-12-26 23:32:06,647][105692] Updated weights for policy 0, policy_version 1120397 (0.0007) [2023-12-26 23:32:06,668][105585] KL-divergence is very high: 130.1661 [2023-12-26 23:32:06,702][105692] Updated weights for policy 0, policy_version 1120407 (0.0005) [2023-12-26 23:32:07,212][105620] Updated weights for policy 1, policy_version 1121792 (0.0009) [2023-12-26 23:32:07,264][105692] Updated weights for policy 0, policy_version 1120417 (0.0006) [2023-12-26 23:32:07,275][105620] Updated weights for policy 1, policy_version 1121802 (0.0008) [2023-12-26 23:32:07,329][105692] Updated weights for policy 0, policy_version 1120427 (0.0010) [2023-12-26 23:32:07,339][105620] Updated weights for policy 1, policy_version 1121812 (0.0006) [2023-12-26 23:32:07,392][105692] Updated weights for policy 0, policy_version 1120437 (0.0011) [2023-12-26 23:32:07,456][105692] Updated weights for policy 0, policy_version 1120447 (0.0011) [2023-12-26 23:32:08,021][105620] Updated weights for policy 1, policy_version 1121822 (0.0008) [2023-12-26 23:32:08,083][105620] Updated weights for policy 1, policy_version 1121832 (0.0007) [2023-12-26 23:32:08,101][105692] Updated weights for policy 0, policy_version 1120457 (0.0010) [2023-12-26 23:32:08,123][105585] KL-divergence is very high: 370.8365 [2023-12-26 23:32:08,139][105620] Updated weights for policy 1, policy_version 1121842 (0.0006) [2023-12-26 23:32:08,151][105692] Updated weights for policy 0, policy_version 1120467 (0.0008) [2023-12-26 23:32:08,164][105585] KL-divergence is very high: 689.2692 [2023-12-26 23:32:08,210][105692] Updated weights for policy 0, policy_version 1120477 (0.0007) [2023-12-26 23:32:08,212][105585] KL-divergence is very high: 769.2555 [2023-12-26 23:32:08,827][105620] Updated weights for policy 1, policy_version 1121852 (0.0009) [2023-12-26 23:32:08,884][105620] Updated weights for policy 1, policy_version 1121862 (0.0009) [2023-12-26 23:32:08,940][105620] Updated weights for policy 1, policy_version 1121872 (0.0009) [2023-12-26 23:32:08,978][105692] Updated weights for policy 0, policy_version 1120487 (0.0006) [2023-12-26 23:32:09,039][105692] Updated weights for policy 0, policy_version 1120497 (0.0009) [2023-12-26 23:32:09,091][105692] Updated weights for policy 0, policy_version 1120507 (0.0009) [2023-12-26 23:32:09,743][105620] Updated weights for policy 1, policy_version 1121882 (0.0009) [2023-12-26 23:32:09,793][105620] Updated weights for policy 1, policy_version 1121892 (0.0008) [2023-12-26 23:32:09,850][105692] Updated weights for policy 0, policy_version 1120517 (0.0010) [2023-12-26 23:32:09,856][105620] Updated weights for policy 1, policy_version 1121902 (0.0009) [2023-12-26 23:32:09,903][105692] Updated weights for policy 0, policy_version 1120527 (0.0011) [2023-12-26 23:32:09,919][105620] Updated weights for policy 1, policy_version 1121912 (0.0008) [2023-12-26 23:32:09,956][105692] Updated weights for policy 0, policy_version 1120537 (0.0010) [2023-12-26 23:32:10,522][105620] Updated weights for policy 1, policy_version 1121922 (0.0008) [2023-12-26 23:32:10,585][105620] Updated weights for policy 1, policy_version 1121932 (0.0008) [2023-12-26 23:32:10,645][105620] Updated weights for policy 1, policy_version 1121942 (0.0008) [2023-12-26 23:32:10,728][105692] Updated weights for policy 0, policy_version 1120547 (0.0010) [2023-12-26 23:32:10,786][105692] Updated weights for policy 0, policy_version 1120557 (0.0005) [2023-12-26 23:32:10,842][105692] Updated weights for policy 0, policy_version 1120567 (0.0007) [2023-12-26 23:32:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 574160896. Throughput: 0: 9513.6, 1: 9883.2. Samples: 574167416. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:32:11,062][104569] Avg episode reward: [(0, '7898.664'), (1, '9078.168')] [2023-12-26 23:32:11,347][105620] Updated weights for policy 1, policy_version 1121952 (0.0008) [2023-12-26 23:32:11,413][105620] Updated weights for policy 1, policy_version 1121962 (0.0007) [2023-12-26 23:32:11,483][105620] Updated weights for policy 1, policy_version 1121972 (0.0006) [2023-12-26 23:32:11,548][105692] Updated weights for policy 0, policy_version 1120577 (0.0011) [2023-12-26 23:32:11,614][105692] Updated weights for policy 0, policy_version 1120587 (0.0007) [2023-12-26 23:32:11,647][105585] KL-divergence is very high: 121.3512 [2023-12-26 23:32:11,684][105692] Updated weights for policy 0, policy_version 1120597 (0.0008) [2023-12-26 23:32:11,697][105585] KL-divergence is very high: 195.2368 [2023-12-26 23:32:11,751][105692] Updated weights for policy 0, policy_version 1120607 (0.0008) [2023-12-26 23:32:11,753][105585] KL-divergence is very high: 192.3273 [2023-12-26 23:32:12,199][105620] Updated weights for policy 1, policy_version 1121982 (0.0007) [2023-12-26 23:32:12,262][105620] Updated weights for policy 1, policy_version 1121992 (0.0009) [2023-12-26 23:32:12,327][105620] Updated weights for policy 1, policy_version 1122002 (0.0009) [2023-12-26 23:32:12,458][105585] KL-divergence is very high: 126.1105 [2023-12-26 23:32:12,472][105692] Updated weights for policy 0, policy_version 1120617 (0.0007) [2023-12-26 23:32:12,503][105585] KL-divergence is very high: 112.5413 [2023-12-26 23:32:12,534][105692] Updated weights for policy 0, policy_version 1120627 (0.0006) [2023-12-26 23:32:12,554][105585] KL-divergence is very high: 126.9833 [2023-12-26 23:32:12,598][105692] Updated weights for policy 0, policy_version 1120637 (0.0009) [2023-12-26 23:32:12,997][105620] Updated weights for policy 1, policy_version 1122012 (0.0007) [2023-12-26 23:32:13,056][105620] Updated weights for policy 1, policy_version 1122022 (0.0005) [2023-12-26 23:32:13,128][105620] Updated weights for policy 1, policy_version 1122032 (0.0006) [2023-12-26 23:32:13,449][105692] Updated weights for policy 0, policy_version 1120647 (0.0007) [2023-12-26 23:32:13,508][105692] Updated weights for policy 0, policy_version 1120657 (0.0005) [2023-12-26 23:32:13,570][105692] Updated weights for policy 0, policy_version 1120667 (0.0006) [2023-12-26 23:32:13,728][105620] Updated weights for policy 1, policy_version 1122042 (0.0006) [2023-12-26 23:32:13,800][105620] Updated weights for policy 1, policy_version 1122052 (0.0010) [2023-12-26 23:32:13,854][105620] Updated weights for policy 1, policy_version 1122062 (0.0009) [2023-12-26 23:32:13,908][105620] Updated weights for policy 1, policy_version 1122072 (0.0009) [2023-12-26 23:32:14,201][105692] Updated weights for policy 0, policy_version 1120677 (0.0007) [2023-12-26 23:32:14,219][105585] KL-divergence is very high: 137.4851 [2023-12-26 23:32:14,239][105585] KL-divergence is very high: 137.8788 [2023-12-26 23:32:14,254][105692] Updated weights for policy 0, policy_version 1120688 (0.0010) [2023-12-26 23:32:14,261][105585] KL-divergence is very high: 218.1475 [2023-12-26 23:32:14,281][105585] KL-divergence is very high: 133.0637 [2023-12-26 23:32:14,300][105585] KL-divergence is very high: 168.8878 [2023-12-26 23:32:14,305][105692] Updated weights for policy 0, policy_version 1120698 (0.0010) [2023-12-26 23:32:14,577][105620] Updated weights for policy 1, policy_version 1122082 (0.0007) [2023-12-26 23:32:14,637][105620] Updated weights for policy 1, policy_version 1122092 (0.0005) [2023-12-26 23:32:14,696][105620] Updated weights for policy 1, policy_version 1122102 (0.0009) [2023-12-26 23:32:15,035][105692] Updated weights for policy 0, policy_version 1120708 (0.0010) [2023-12-26 23:32:15,092][105692] Updated weights for policy 0, policy_version 1120718 (0.0010) [2023-12-26 23:32:15,151][105692] Updated weights for policy 0, policy_version 1120728 (0.0010) [2023-12-26 23:32:15,513][105620] Updated weights for policy 1, policy_version 1122112 (0.0009) [2023-12-26 23:32:15,570][105620] Updated weights for policy 1, policy_version 1122123 (0.0009) [2023-12-26 23:32:15,625][105620] Updated weights for policy 1, policy_version 1122134 (0.0010) [2023-12-26 23:32:15,776][105692] Updated weights for policy 0, policy_version 1120738 (0.0010) [2023-12-26 23:32:15,836][105692] Updated weights for policy 0, policy_version 1120748 (0.0008) [2023-12-26 23:32:15,891][105692] Updated weights for policy 0, policy_version 1120758 (0.0010) [2023-12-26 23:32:15,948][105692] Updated weights for policy 0, policy_version 1120768 (0.0008) [2023-12-26 23:32:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19387.7, 300 sec: 19605.2). Total num frames: 574259200. Throughput: 0: 9495.4, 1: 9906.0. Samples: 574225516. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:32:16,063][104569] Avg episode reward: [(0, '7434.838'), (1, '9082.907')] [2023-12-26 23:32:16,074][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001120768_286957568.pth... [2023-12-26 23:32:16,074][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001122136_287301632.pth... [2023-12-26 23:32:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001119616_286662656.pth [2023-12-26 23:32:16,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001120984_287006720.pth [2023-12-26 23:32:16,318][105620] Updated weights for policy 1, policy_version 1122144 (0.0006) [2023-12-26 23:32:16,373][105620] Updated weights for policy 1, policy_version 1122154 (0.0005) [2023-12-26 23:32:16,439][105620] Updated weights for policy 1, policy_version 1122164 (0.0008) [2023-12-26 23:32:16,669][105692] Updated weights for policy 0, policy_version 1120778 (0.0005) [2023-12-26 23:32:16,724][105692] Updated weights for policy 0, policy_version 1120788 (0.0005) [2023-12-26 23:32:16,783][105692] Updated weights for policy 0, policy_version 1120798 (0.0005) [2023-12-26 23:32:16,792][105585] KL-divergence is very high: 141.7845 [2023-12-26 23:32:17,180][105620] Updated weights for policy 1, policy_version 1122174 (0.0008) [2023-12-26 23:32:17,234][105620] Updated weights for policy 1, policy_version 1122184 (0.0008) [2023-12-26 23:32:17,285][105620] Updated weights for policy 1, policy_version 1122194 (0.0007) [2023-12-26 23:32:17,376][105692] Updated weights for policy 0, policy_version 1120808 (0.0009) [2023-12-26 23:32:17,434][105692] Updated weights for policy 0, policy_version 1120818 (0.0010) [2023-12-26 23:32:17,498][105692] Updated weights for policy 0, policy_version 1120828 (0.0010) [2023-12-26 23:32:17,968][105620] Updated weights for policy 1, policy_version 1122204 (0.0007) [2023-12-26 23:32:18,022][105620] Updated weights for policy 1, policy_version 1122214 (0.0005) [2023-12-26 23:32:18,076][105620] Updated weights for policy 1, policy_version 1122224 (0.0005) [2023-12-26 23:32:18,235][105692] Updated weights for policy 0, policy_version 1120838 (0.0010) [2023-12-26 23:32:18,286][105692] Updated weights for policy 0, policy_version 1120848 (0.0010) [2023-12-26 23:32:18,352][105692] Updated weights for policy 0, policy_version 1120858 (0.0009) [2023-12-26 23:32:18,613][105620] Updated weights for policy 1, policy_version 1122234 (0.0006) [2023-12-26 23:32:18,668][105620] Updated weights for policy 1, policy_version 1122244 (0.0005) [2023-12-26 23:32:18,732][105620] Updated weights for policy 1, policy_version 1122254 (0.0006) [2023-12-26 23:32:18,785][105620] Updated weights for policy 1, policy_version 1122264 (0.0008) [2023-12-26 23:32:19,092][105692] Updated weights for policy 0, policy_version 1120868 (0.0009) [2023-12-26 23:32:19,143][105692] Updated weights for policy 0, policy_version 1120878 (0.0010) [2023-12-26 23:32:19,190][105692] Updated weights for policy 0, policy_version 1120888 (0.0010) [2023-12-26 23:32:19,535][105620] Updated weights for policy 1, policy_version 1122274 (0.0008) [2023-12-26 23:32:19,592][105620] Updated weights for policy 1, policy_version 1122284 (0.0010) [2023-12-26 23:32:19,657][105620] Updated weights for policy 1, policy_version 1122294 (0.0009) [2023-12-26 23:32:19,901][105692] Updated weights for policy 0, policy_version 1120898 (0.0010) [2023-12-26 23:32:19,969][105692] Updated weights for policy 0, policy_version 1120908 (0.0011) [2023-12-26 23:32:20,026][105692] Updated weights for policy 0, policy_version 1120918 (0.0010) [2023-12-26 23:32:20,085][105692] Updated weights for policy 0, policy_version 1120928 (0.0011) [2023-12-26 23:32:20,391][105620] Updated weights for policy 1, policy_version 1122304 (0.0009) [2023-12-26 23:32:20,444][105620] Updated weights for policy 1, policy_version 1122314 (0.0006) [2023-12-26 23:32:20,504][105620] Updated weights for policy 1, policy_version 1122324 (0.0006) [2023-12-26 23:32:20,861][105692] Updated weights for policy 0, policy_version 1120938 (0.0011) [2023-12-26 23:32:20,924][105692] Updated weights for policy 0, policy_version 1120948 (0.0011) [2023-12-26 23:32:20,994][105692] Updated weights for policy 0, policy_version 1120958 (0.0011) [2023-12-26 23:32:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 574357504. Throughput: 0: 9615.2, 1: 9856.9. Samples: 574344804. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:32:21,063][104569] Avg episode reward: [(0, '7891.159'), (1, '9084.061')] [2023-12-26 23:32:21,182][105620] Updated weights for policy 1, policy_version 1122334 (0.0007) [2023-12-26 23:32:21,243][105620] Updated weights for policy 1, policy_version 1122344 (0.0009) [2023-12-26 23:32:21,307][105620] Updated weights for policy 1, policy_version 1122354 (0.0009) [2023-12-26 23:32:21,852][105692] Updated weights for policy 0, policy_version 1120968 (0.0011) [2023-12-26 23:32:21,903][105692] Updated weights for policy 0, policy_version 1120978 (0.0010) [2023-12-26 23:32:21,964][105692] Updated weights for policy 0, policy_version 1120988 (0.0009) [2023-12-26 23:32:22,113][105620] Updated weights for policy 1, policy_version 1122364 (0.0009) [2023-12-26 23:32:22,171][105620] Updated weights for policy 1, policy_version 1122374 (0.0009) [2023-12-26 23:32:22,233][105620] Updated weights for policy 1, policy_version 1122384 (0.0008) [2023-12-26 23:32:22,767][105692] Updated weights for policy 0, policy_version 1120998 (0.0008) [2023-12-26 23:32:22,832][105692] Updated weights for policy 0, policy_version 1121008 (0.0009) [2023-12-26 23:32:22,881][105620] Updated weights for policy 1, policy_version 1122394 (0.0009) [2023-12-26 23:32:22,894][105692] Updated weights for policy 0, policy_version 1121018 (0.0008) [2023-12-26 23:32:22,938][105620] Updated weights for policy 1, policy_version 1122404 (0.0007) [2023-12-26 23:32:22,998][105620] Updated weights for policy 1, policy_version 1122414 (0.0009) [2023-12-26 23:32:23,055][105620] Updated weights for policy 1, policy_version 1122424 (0.0008) [2023-12-26 23:32:23,515][105692] Updated weights for policy 0, policy_version 1121028 (0.0006) [2023-12-26 23:32:23,573][105692] Updated weights for policy 0, policy_version 1121038 (0.0008) [2023-12-26 23:32:23,596][105585] KL-divergence is very high: 115.6922 [2023-12-26 23:32:23,631][105692] Updated weights for policy 0, policy_version 1121048 (0.0006) [2023-12-26 23:32:23,653][105585] KL-divergence is very high: 101.6489 [2023-12-26 23:32:23,773][105620] Updated weights for policy 1, policy_version 1122434 (0.0008) [2023-12-26 23:32:23,829][105620] Updated weights for policy 1, policy_version 1122444 (0.0008) [2023-12-26 23:32:23,884][105620] Updated weights for policy 1, policy_version 1122454 (0.0008) [2023-12-26 23:32:24,365][105692] Updated weights for policy 0, policy_version 1121058 (0.0009) [2023-12-26 23:32:24,428][105692] Updated weights for policy 0, policy_version 1121068 (0.0010) [2023-12-26 23:32:24,483][105692] Updated weights for policy 0, policy_version 1121078 (0.0010) [2023-12-26 23:32:24,541][105692] Updated weights for policy 0, policy_version 1121088 (0.0010) [2023-12-26 23:32:24,651][105620] Updated weights for policy 1, policy_version 1122464 (0.0008) [2023-12-26 23:32:24,710][105620] Updated weights for policy 1, policy_version 1122474 (0.0008) [2023-12-26 23:32:24,773][105620] Updated weights for policy 1, policy_version 1122484 (0.0008) [2023-12-26 23:32:25,298][105692] Updated weights for policy 0, policy_version 1121098 (0.0011) [2023-12-26 23:32:25,359][105692] Updated weights for policy 0, policy_version 1121108 (0.0010) [2023-12-26 23:32:25,417][105692] Updated weights for policy 0, policy_version 1121118 (0.0010) [2023-12-26 23:32:25,528][105620] Updated weights for policy 1, policy_version 1122494 (0.0008) [2023-12-26 23:32:25,593][105620] Updated weights for policy 1, policy_version 1122504 (0.0008) [2023-12-26 23:32:25,689][105620] Updated weights for policy 1, policy_version 1122514 (0.0009) [2023-12-26 23:32:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 574447616. Throughput: 0: 9625.1, 1: 9834.2. Samples: 574458620. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:32:26,063][104569] Avg episode reward: [(0, '7429.068'), (1, '8989.959')] [2023-12-26 23:32:26,108][105692] Updated weights for policy 0, policy_version 1121128 (0.0007) [2023-12-26 23:32:26,152][105692] Updated weights for policy 0, policy_version 1121138 (0.0005) [2023-12-26 23:32:26,211][105692] Updated weights for policy 0, policy_version 1121148 (0.0007) [2023-12-26 23:32:26,308][105620] Updated weights for policy 1, policy_version 1122524 (0.0010) [2023-12-26 23:32:26,366][105620] Updated weights for policy 1, policy_version 1122534 (0.0010) [2023-12-26 23:32:26,422][105620] Updated weights for policy 1, policy_version 1122544 (0.0009) [2023-12-26 23:32:26,738][105692] Updated weights for policy 0, policy_version 1121158 (0.0007) [2023-12-26 23:32:26,793][105692] Updated weights for policy 0, policy_version 1121168 (0.0005) [2023-12-26 23:32:26,861][105692] Updated weights for policy 0, policy_version 1121178 (0.0005) [2023-12-26 23:32:27,274][105620] Updated weights for policy 1, policy_version 1122554 (0.0010) [2023-12-26 23:32:27,333][105620] Updated weights for policy 1, policy_version 1122564 (0.0009) [2023-12-26 23:32:27,396][105692] Updated weights for policy 0, policy_version 1121188 (0.0007) [2023-12-26 23:32:27,397][105620] Updated weights for policy 1, policy_version 1122574 (0.0006) [2023-12-26 23:32:27,451][105692] Updated weights for policy 0, policy_version 1121198 (0.0005) [2023-12-26 23:32:27,456][105620] Updated weights for policy 1, policy_version 1122584 (0.0005) [2023-12-26 23:32:27,499][105692] Updated weights for policy 0, policy_version 1121208 (0.0005) [2023-12-26 23:32:28,082][105692] Updated weights for policy 0, policy_version 1121218 (0.0006) [2023-12-26 23:32:28,108][105620] Updated weights for policy 1, policy_version 1122594 (0.0006) [2023-12-26 23:32:28,140][105692] Updated weights for policy 0, policy_version 1121228 (0.0010) [2023-12-26 23:32:28,162][105620] Updated weights for policy 1, policy_version 1122604 (0.0007) [2023-12-26 23:32:28,190][105692] Updated weights for policy 0, policy_version 1121238 (0.0010) [2023-12-26 23:32:28,220][105620] Updated weights for policy 1, policy_version 1122614 (0.0006) [2023-12-26 23:32:28,234][105692] Updated weights for policy 0, policy_version 1121248 (0.0010) [2023-12-26 23:32:28,931][105620] Updated weights for policy 1, policy_version 1122624 (0.0007) [2023-12-26 23:32:28,982][105620] Updated weights for policy 1, policy_version 1122634 (0.0006) [2023-12-26 23:32:28,983][105692] Updated weights for policy 0, policy_version 1121258 (0.0010) [2023-12-26 23:32:29,031][105620] Updated weights for policy 1, policy_version 1122644 (0.0009) [2023-12-26 23:32:29,045][105692] Updated weights for policy 0, policy_version 1121268 (0.0010) [2023-12-26 23:32:29,103][105692] Updated weights for policy 0, policy_version 1121278 (0.0010) [2023-12-26 23:32:29,798][105620] Updated weights for policy 1, policy_version 1122654 (0.0006) [2023-12-26 23:32:29,836][105692] Updated weights for policy 0, policy_version 1121288 (0.0010) [2023-12-26 23:32:29,864][105620] Updated weights for policy 1, policy_version 1122664 (0.0008) [2023-12-26 23:32:29,901][105692] Updated weights for policy 0, policy_version 1121298 (0.0009) [2023-12-26 23:32:29,927][105620] Updated weights for policy 1, policy_version 1122674 (0.0009) [2023-12-26 23:32:29,961][105692] Updated weights for policy 0, policy_version 1121308 (0.0008) [2023-12-26 23:32:30,572][105692] Updated weights for policy 0, policy_version 1121318 (0.0009) [2023-12-26 23:32:30,623][105692] Updated weights for policy 0, policy_version 1121328 (0.0009) [2023-12-26 23:32:30,670][105692] Updated weights for policy 0, policy_version 1121338 (0.0009) [2023-12-26 23:32:30,707][105620] Updated weights for policy 1, policy_version 1122684 (0.0007) [2023-12-26 23:32:30,758][105620] Updated weights for policy 1, policy_version 1122694 (0.0009) [2023-12-26 23:32:30,812][105620] Updated weights for policy 1, policy_version 1122704 (0.0009) [2023-12-26 23:32:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 574554112. Throughput: 0: 9749.3, 1: 9833.7. Samples: 574522340. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:32:31,062][104569] Avg episode reward: [(0, '8439.499'), (1, '9078.754')] [2023-12-26 23:32:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001122712_287449088.pth... [2023-12-26 23:32:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001121344_287105024.pth... [2023-12-26 23:32:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001121560_287154176.pth [2023-12-26 23:32:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001120192_286810112.pth [2023-12-26 23:32:31,294][105692] Updated weights for policy 0, policy_version 1121348 (0.0011) [2023-12-26 23:32:31,350][105692] Updated weights for policy 0, policy_version 1121358 (0.0011) [2023-12-26 23:32:31,414][105692] Updated weights for policy 0, policy_version 1121368 (0.0010) [2023-12-26 23:32:31,634][105620] Updated weights for policy 1, policy_version 1122715 (0.0010) [2023-12-26 23:32:31,696][105620] Updated weights for policy 1, policy_version 1122725 (0.0008) [2023-12-26 23:32:31,765][105620] Updated weights for policy 1, policy_version 1122735 (0.0008) [2023-12-26 23:32:32,047][105692] Updated weights for policy 0, policy_version 1121378 (0.0006) [2023-12-26 23:32:32,109][105692] Updated weights for policy 0, policy_version 1121388 (0.0011) [2023-12-26 23:32:32,157][105692] Updated weights for policy 0, policy_version 1121398 (0.0010) [2023-12-26 23:32:32,202][105692] Updated weights for policy 0, policy_version 1121408 (0.0010) [2023-12-26 23:32:32,460][105620] Updated weights for policy 1, policy_version 1122745 (0.0007) [2023-12-26 23:32:32,511][105620] Updated weights for policy 1, policy_version 1122755 (0.0008) [2023-12-26 23:32:32,559][105620] Updated weights for policy 1, policy_version 1122765 (0.0008) [2023-12-26 23:32:32,566][105586] KL-divergence is very high: 102.1020 [2023-12-26 23:32:32,609][105586] KL-divergence is very high: 102.3245 [2023-12-26 23:32:32,610][105620] Updated weights for policy 1, policy_version 1122775 (0.0008) [2023-12-26 23:32:32,934][105692] Updated weights for policy 0, policy_version 1121418 (0.0011) [2023-12-26 23:32:32,968][105585] KL-divergence is very high: 206.9272 [2023-12-26 23:32:32,985][105585] KL-divergence is very high: 245.7133 [2023-12-26 23:32:32,992][105692] Updated weights for policy 0, policy_version 1121428 (0.0010) [2023-12-26 23:32:33,011][105585] KL-divergence is very high: 349.9485 [2023-12-26 23:32:33,026][105585] KL-divergence is very high: 314.3227 [2023-12-26 23:32:33,040][105692] Updated weights for policy 0, policy_version 1121438 (0.0010) [2023-12-26 23:32:33,369][105620] Updated weights for policy 1, policy_version 1122785 (0.0005) [2023-12-26 23:32:33,425][105620] Updated weights for policy 1, policy_version 1122795 (0.0005) [2023-12-26 23:32:33,492][105620] Updated weights for policy 1, policy_version 1122805 (0.0006) [2023-12-26 23:32:33,688][105585] KL-divergence is very high: 286.2832 [2023-12-26 23:32:33,719][105692] Updated weights for policy 0, policy_version 1121448 (0.0006) [2023-12-26 23:32:33,725][105585] KL-divergence is very high: 282.8947 [2023-12-26 23:32:33,762][105585] KL-divergence is very high: 258.7376 [2023-12-26 23:32:33,765][105692] Updated weights for policy 0, policy_version 1121458 (0.0005) [2023-12-26 23:32:33,801][105585] KL-divergence is very high: 274.6185 [2023-12-26 23:32:33,816][105692] Updated weights for policy 0, policy_version 1121468 (0.0005) [2023-12-26 23:32:34,168][105620] Updated weights for policy 1, policy_version 1122815 (0.0010) [2023-12-26 23:32:34,228][105620] Updated weights for policy 1, policy_version 1122825 (0.0010) [2023-12-26 23:32:34,293][105620] Updated weights for policy 1, policy_version 1122835 (0.0010) [2023-12-26 23:32:34,433][105692] Updated weights for policy 0, policy_version 1121478 (0.0008) [2023-12-26 23:32:34,494][105692] Updated weights for policy 0, policy_version 1121488 (0.0008) [2023-12-26 23:32:34,551][105692] Updated weights for policy 0, policy_version 1121498 (0.0008) [2023-12-26 23:32:34,966][105620] Updated weights for policy 1, policy_version 1122845 (0.0009) [2023-12-26 23:32:35,024][105620] Updated weights for policy 1, policy_version 1122855 (0.0010) [2023-12-26 23:32:35,082][105620] Updated weights for policy 1, policy_version 1122865 (0.0010) [2023-12-26 23:32:35,367][105692] Updated weights for policy 0, policy_version 1121508 (0.0008) [2023-12-26 23:32:35,412][105692] Updated weights for policy 0, policy_version 1121518 (0.0008) [2023-12-26 23:32:35,464][105692] Updated weights for policy 0, policy_version 1121528 (0.0007) [2023-12-26 23:32:35,805][105620] Updated weights for policy 1, policy_version 1122875 (0.0010) [2023-12-26 23:32:35,857][105620] Updated weights for policy 1, policy_version 1122885 (0.0010) [2023-12-26 23:32:35,920][105620] Updated weights for policy 1, policy_version 1122895 (0.0010) [2023-12-26 23:32:36,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19524.4, 300 sec: 19605.3). Total num frames: 574652416. Throughput: 0: 9917.9, 1: 9788.9. Samples: 574641072. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:32:36,062][104569] Avg episode reward: [(0, '8164.712'), (1, '8985.744')] [2023-12-26 23:32:36,155][105692] Updated weights for policy 0, policy_version 1121538 (0.0006) [2023-12-26 23:32:36,243][105692] Updated weights for policy 0, policy_version 1121548 (0.0008) [2023-12-26 23:32:36,301][105692] Updated weights for policy 0, policy_version 1121558 (0.0006) [2023-12-26 23:32:36,357][105692] Updated weights for policy 0, policy_version 1121568 (0.0006) [2023-12-26 23:32:36,592][105620] Updated weights for policy 1, policy_version 1122905 (0.0009) [2023-12-26 23:32:36,657][105620] Updated weights for policy 1, policy_version 1122915 (0.0009) [2023-12-26 23:32:36,718][105620] Updated weights for policy 1, policy_version 1122925 (0.0009) [2023-12-26 23:32:36,792][105620] Updated weights for policy 1, policy_version 1122935 (0.0010) [2023-12-26 23:32:37,060][105692] Updated weights for policy 0, policy_version 1121578 (0.0010) [2023-12-26 23:32:37,128][105692] Updated weights for policy 0, policy_version 1121588 (0.0010) [2023-12-26 23:32:37,186][105692] Updated weights for policy 0, policy_version 1121598 (0.0009) [2023-12-26 23:32:37,521][105620] Updated weights for policy 1, policy_version 1122945 (0.0008) [2023-12-26 23:32:37,580][105620] Updated weights for policy 1, policy_version 1122955 (0.0008) [2023-12-26 23:32:37,644][105620] Updated weights for policy 1, policy_version 1122965 (0.0008) [2023-12-26 23:32:37,972][105692] Updated weights for policy 0, policy_version 1121608 (0.0010) [2023-12-26 23:32:38,041][105692] Updated weights for policy 0, policy_version 1121618 (0.0007) [2023-12-26 23:32:38,093][105692] Updated weights for policy 0, policy_version 1121628 (0.0005) [2023-12-26 23:32:38,437][105620] Updated weights for policy 1, policy_version 1122975 (0.0008) [2023-12-26 23:32:38,500][105620] Updated weights for policy 1, policy_version 1122985 (0.0008) [2023-12-26 23:32:38,554][105620] Updated weights for policy 1, policy_version 1122995 (0.0006) [2023-12-26 23:32:38,786][105692] Updated weights for policy 0, policy_version 1121638 (0.0009) [2023-12-26 23:32:38,835][105692] Updated weights for policy 0, policy_version 1121648 (0.0010) [2023-12-26 23:32:38,890][105692] Updated weights for policy 0, policy_version 1121658 (0.0010) [2023-12-26 23:32:39,124][105620] Updated weights for policy 1, policy_version 1123005 (0.0007) [2023-12-26 23:32:39,172][105620] Updated weights for policy 1, policy_version 1123015 (0.0008) [2023-12-26 23:32:39,216][105620] Updated weights for policy 1, policy_version 1123025 (0.0008) [2023-12-26 23:32:39,684][105692] Updated weights for policy 0, policy_version 1121668 (0.0010) [2023-12-26 23:32:39,734][105692] Updated weights for policy 0, policy_version 1121678 (0.0008) [2023-12-26 23:32:39,797][105692] Updated weights for policy 0, policy_version 1121688 (0.0009) [2023-12-26 23:32:40,011][105620] Updated weights for policy 1, policy_version 1123035 (0.0009) [2023-12-26 23:32:40,080][105620] Updated weights for policy 1, policy_version 1123045 (0.0009) [2023-12-26 23:32:40,146][105620] Updated weights for policy 1, policy_version 1123055 (0.0009) [2023-12-26 23:32:40,486][105692] Updated weights for policy 0, policy_version 1121698 (0.0008) [2023-12-26 23:32:40,549][105692] Updated weights for policy 0, policy_version 1121708 (0.0005) [2023-12-26 23:32:40,615][105692] Updated weights for policy 0, policy_version 1121718 (0.0007) [2023-12-26 23:32:40,676][105692] Updated weights for policy 0, policy_version 1121728 (0.0009) [2023-12-26 23:32:40,961][105620] Updated weights for policy 1, policy_version 1123065 (0.0009) [2023-12-26 23:32:41,023][105620] Updated weights for policy 1, policy_version 1123075 (0.0010) [2023-12-26 23:32:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 574742528. Throughput: 0: 9876.7, 1: 9759.3. Samples: 574755088. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:32:41,063][104569] Avg episode reward: [(0, '7888.591'), (1, '8985.303')] [2023-12-26 23:32:41,094][105620] Updated weights for policy 1, policy_version 1123085 (0.0008) [2023-12-26 23:32:41,164][105620] Updated weights for policy 1, policy_version 1123095 (0.0009) [2023-12-26 23:32:41,317][105692] Updated weights for policy 0, policy_version 1121738 (0.0009) [2023-12-26 23:32:41,383][105692] Updated weights for policy 0, policy_version 1121748 (0.0009) [2023-12-26 23:32:41,446][105692] Updated weights for policy 0, policy_version 1121758 (0.0010) [2023-12-26 23:32:41,825][105620] Updated weights for policy 1, policy_version 1123105 (0.0007) [2023-12-26 23:32:41,873][105620] Updated weights for policy 1, policy_version 1123115 (0.0008) [2023-12-26 23:32:41,926][105620] Updated weights for policy 1, policy_version 1123125 (0.0008) [2023-12-26 23:32:42,255][105692] Updated weights for policy 0, policy_version 1121769 (0.0008) [2023-12-26 23:32:42,318][105692] Updated weights for policy 0, policy_version 1121779 (0.0011) [2023-12-26 23:32:42,384][105692] Updated weights for policy 0, policy_version 1121789 (0.0011) [2023-12-26 23:32:42,706][105620] Updated weights for policy 1, policy_version 1123135 (0.0009) [2023-12-26 23:32:42,764][105620] Updated weights for policy 1, policy_version 1123145 (0.0009) [2023-12-26 23:32:42,839][105620] Updated weights for policy 1, policy_version 1123155 (0.0010) [2023-12-26 23:32:42,966][105692] Updated weights for policy 0, policy_version 1121799 (0.0007) [2023-12-26 23:32:43,031][105692] Updated weights for policy 0, policy_version 1121809 (0.0005) [2023-12-26 23:32:43,083][105692] Updated weights for policy 0, policy_version 1121819 (0.0005) [2023-12-26 23:32:43,593][105692] Updated weights for policy 0, policy_version 1121829 (0.0005) [2023-12-26 23:32:43,611][105620] Updated weights for policy 1, policy_version 1123165 (0.0010) [2023-12-26 23:32:43,655][105692] Updated weights for policy 0, policy_version 1121839 (0.0005) [2023-12-26 23:32:43,661][105620] Updated weights for policy 1, policy_version 1123175 (0.0009) [2023-12-26 23:32:43,716][105620] Updated weights for policy 1, policy_version 1123185 (0.0006) [2023-12-26 23:32:43,719][105692] Updated weights for policy 0, policy_version 1121849 (0.0005) [2023-12-26 23:32:44,226][105692] Updated weights for policy 0, policy_version 1121859 (0.0005) [2023-12-26 23:32:44,290][105692] Updated weights for policy 0, policy_version 1121869 (0.0006) [2023-12-26 23:32:44,292][105620] Updated weights for policy 1, policy_version 1123195 (0.0006) [2023-12-26 23:32:44,348][105692] Updated weights for policy 0, policy_version 1121879 (0.0006) [2023-12-26 23:32:44,356][105620] Updated weights for policy 1, policy_version 1123205 (0.0006) [2023-12-26 23:32:44,414][105620] Updated weights for policy 1, policy_version 1123215 (0.0008) [2023-12-26 23:32:44,945][105692] Updated weights for policy 0, policy_version 1121889 (0.0006) [2023-12-26 23:32:44,986][105620] Updated weights for policy 1, policy_version 1123225 (0.0008) [2023-12-26 23:32:45,009][105692] Updated weights for policy 0, policy_version 1121899 (0.0008) [2023-12-26 23:32:45,048][105620] Updated weights for policy 1, policy_version 1123235 (0.0010) [2023-12-26 23:32:45,072][105692] Updated weights for policy 0, policy_version 1121909 (0.0008) [2023-12-26 23:32:45,103][105620] Updated weights for policy 1, policy_version 1123245 (0.0008) [2023-12-26 23:32:45,136][105692] Updated weights for policy 0, policy_version 1121919 (0.0008) [2023-12-26 23:32:45,154][105620] Updated weights for policy 1, policy_version 1123255 (0.0008) [2023-12-26 23:32:45,890][105620] Updated weights for policy 1, policy_version 1123265 (0.0007) [2023-12-26 23:32:45,908][105692] Updated weights for policy 0, policy_version 1121929 (0.0007) [2023-12-26 23:32:45,945][105620] Updated weights for policy 1, policy_version 1123275 (0.0010) [2023-12-26 23:32:45,964][105692] Updated weights for policy 0, policy_version 1121939 (0.0006) [2023-12-26 23:32:45,994][105620] Updated weights for policy 1, policy_version 1123285 (0.0008) [2023-12-26 23:32:46,012][105692] Updated weights for policy 0, policy_version 1121949 (0.0007) [2023-12-26 23:32:46,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 574857216. Throughput: 0: 9948.7, 1: 9745.7. Samples: 574815120. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:32:46,063][104569] Avg episode reward: [(0, '7889.850'), (1, '8894.379')] [2023-12-26 23:32:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001123288_287596544.pth... [2023-12-26 23:32:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001121952_287260672.pth... [2023-12-26 23:32:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001122136_287301632.pth [2023-12-26 23:32:46,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001120768_286957568.pth [2023-12-26 23:32:46,652][105692] Updated weights for policy 0, policy_version 1121959 (0.0006) [2023-12-26 23:32:46,691][105620] Updated weights for policy 1, policy_version 1123295 (0.0010) [2023-12-26 23:32:46,702][105692] Updated weights for policy 0, policy_version 1121969 (0.0006) [2023-12-26 23:32:46,750][105620] Updated weights for policy 1, policy_version 1123305 (0.0010) [2023-12-26 23:32:46,754][105692] Updated weights for policy 0, policy_version 1121979 (0.0008) [2023-12-26 23:32:46,813][105620] Updated weights for policy 1, policy_version 1123315 (0.0010) [2023-12-26 23:32:47,473][105692] Updated weights for policy 0, policy_version 1121989 (0.0009) [2023-12-26 23:32:47,522][105692] Updated weights for policy 0, policy_version 1121999 (0.0008) [2023-12-26 23:32:47,551][105620] Updated weights for policy 1, policy_version 1123325 (0.0010) [2023-12-26 23:32:47,566][105692] Updated weights for policy 0, policy_version 1122009 (0.0007) [2023-12-26 23:32:47,609][105620] Updated weights for policy 1, policy_version 1123335 (0.0010) [2023-12-26 23:32:47,666][105620] Updated weights for policy 1, policy_version 1123345 (0.0010) [2023-12-26 23:32:48,293][105692] Updated weights for policy 0, policy_version 1122019 (0.0008) [2023-12-26 23:32:48,335][105620] Updated weights for policy 1, policy_version 1123355 (0.0010) [2023-12-26 23:32:48,354][105692] Updated weights for policy 0, policy_version 1122029 (0.0009) [2023-12-26 23:32:48,395][105620] Updated weights for policy 1, policy_version 1123365 (0.0006) [2023-12-26 23:32:48,416][105692] Updated weights for policy 0, policy_version 1122039 (0.0010) [2023-12-26 23:32:48,449][105620] Updated weights for policy 1, policy_version 1123375 (0.0008) [2023-12-26 23:32:49,099][105692] Updated weights for policy 0, policy_version 1122049 (0.0010) [2023-12-26 23:32:49,151][105692] Updated weights for policy 0, policy_version 1122059 (0.0005) [2023-12-26 23:32:49,175][105620] Updated weights for policy 1, policy_version 1123385 (0.0010) [2023-12-26 23:32:49,213][105692] Updated weights for policy 0, policy_version 1122069 (0.0006) [2023-12-26 23:32:49,244][105620] Updated weights for policy 1, policy_version 1123395 (0.0010) [2023-12-26 23:32:49,276][105692] Updated weights for policy 0, policy_version 1122079 (0.0010) [2023-12-26 23:32:49,299][105620] Updated weights for policy 1, policy_version 1123405 (0.0010) [2023-12-26 23:32:49,359][105620] Updated weights for policy 1, policy_version 1123415 (0.0012) [2023-12-26 23:32:50,045][105692] Updated weights for policy 0, policy_version 1122089 (0.0011) [2023-12-26 23:32:50,107][105692] Updated weights for policy 0, policy_version 1122099 (0.0010) [2023-12-26 23:32:50,151][105620] Updated weights for policy 1, policy_version 1123425 (0.0010) [2023-12-26 23:32:50,162][105692] Updated weights for policy 0, policy_version 1122109 (0.0010) [2023-12-26 23:32:50,210][105620] Updated weights for policy 1, policy_version 1123435 (0.0011) [2023-12-26 23:32:50,272][105620] Updated weights for policy 1, policy_version 1123445 (0.0010) [2023-12-26 23:32:50,901][105692] Updated weights for policy 0, policy_version 1122119 (0.0010) [2023-12-26 23:32:50,956][105692] Updated weights for policy 0, policy_version 1122129 (0.0010) [2023-12-26 23:32:51,015][105692] Updated weights for policy 0, policy_version 1122139 (0.0010) [2023-12-26 23:32:51,039][105620] Updated weights for policy 1, policy_version 1123455 (0.0011) [2023-12-26 23:32:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 574947328. Throughput: 0: 9979.6, 1: 9762.1. Samples: 574937336. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:32:51,063][104569] Avg episode reward: [(0, '8346.637'), (1, '9168.758')] [2023-12-26 23:32:51,099][105620] Updated weights for policy 1, policy_version 1123465 (0.0006) [2023-12-26 23:32:51,165][105620] Updated weights for policy 1, policy_version 1123475 (0.0009) [2023-12-26 23:32:51,756][105692] Updated weights for policy 0, policy_version 1122149 (0.0010) [2023-12-26 23:32:51,822][105692] Updated weights for policy 0, policy_version 1122159 (0.0009) [2023-12-26 23:32:51,873][105692] Updated weights for policy 0, policy_version 1122169 (0.0008) [2023-12-26 23:32:51,955][105620] Updated weights for policy 1, policy_version 1123485 (0.0008) [2023-12-26 23:32:52,013][105620] Updated weights for policy 1, policy_version 1123495 (0.0009) [2023-12-26 23:32:52,061][105620] Updated weights for policy 1, policy_version 1123505 (0.0008) [2023-12-26 23:32:52,595][105692] Updated weights for policy 0, policy_version 1122179 (0.0009) [2023-12-26 23:32:52,655][105692] Updated weights for policy 0, policy_version 1122189 (0.0006) [2023-12-26 23:32:52,716][105692] Updated weights for policy 0, policy_version 1122199 (0.0009) [2023-12-26 23:32:52,827][105620] Updated weights for policy 1, policy_version 1123515 (0.0009) [2023-12-26 23:32:52,885][105620] Updated weights for policy 1, policy_version 1123525 (0.0009) [2023-12-26 23:32:52,935][105620] Updated weights for policy 1, policy_version 1123535 (0.0008) [2023-12-26 23:32:53,446][105692] Updated weights for policy 0, policy_version 1122209 (0.0008) [2023-12-26 23:32:53,491][105692] Updated weights for policy 0, policy_version 1122219 (0.0010) [2023-12-26 23:32:53,535][105692] Updated weights for policy 0, policy_version 1122229 (0.0010) [2023-12-26 23:32:53,583][105692] Updated weights for policy 0, policy_version 1122239 (0.0010) [2023-12-26 23:32:53,712][105620] Updated weights for policy 1, policy_version 1123545 (0.0008) [2023-12-26 23:32:53,765][105620] Updated weights for policy 1, policy_version 1123555 (0.0009) [2023-12-26 23:32:53,820][105620] Updated weights for policy 1, policy_version 1123565 (0.0009) [2023-12-26 23:32:53,874][105620] Updated weights for policy 1, policy_version 1123575 (0.0007) [2023-12-26 23:32:54,214][105692] Updated weights for policy 0, policy_version 1122249 (0.0010) [2023-12-26 23:32:54,268][105692] Updated weights for policy 0, policy_version 1122259 (0.0008) [2023-12-26 23:32:54,331][105692] Updated weights for policy 0, policy_version 1122269 (0.0005) [2023-12-26 23:32:54,599][105620] Updated weights for policy 1, policy_version 1123585 (0.0005) [2023-12-26 23:32:54,654][105620] Updated weights for policy 1, policy_version 1123595 (0.0005) [2023-12-26 23:32:54,723][105620] Updated weights for policy 1, policy_version 1123605 (0.0005) [2023-12-26 23:32:54,907][105692] Updated weights for policy 0, policy_version 1122279 (0.0008) [2023-12-26 23:32:54,966][105692] Updated weights for policy 0, policy_version 1122289 (0.0009) [2023-12-26 23:32:55,034][105692] Updated weights for policy 0, policy_version 1122299 (0.0007) [2023-12-26 23:32:55,274][105620] Updated weights for policy 1, policy_version 1123615 (0.0006) [2023-12-26 23:32:55,332][105620] Updated weights for policy 1, policy_version 1123625 (0.0009) [2023-12-26 23:32:55,389][105620] Updated weights for policy 1, policy_version 1123635 (0.0008) [2023-12-26 23:32:55,600][105692] Updated weights for policy 0, policy_version 1122309 (0.0007) [2023-12-26 23:32:55,665][105692] Updated weights for policy 0, policy_version 1122319 (0.0005) [2023-12-26 23:32:55,726][105692] Updated weights for policy 0, policy_version 1122329 (0.0010) [2023-12-26 23:32:56,037][105620] Updated weights for policy 1, policy_version 1123645 (0.0008) [2023-12-26 23:32:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 575045632. Throughput: 0: 10041.9, 1: 9711.8. Samples: 575056332. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:32:56,062][104569] Avg episode reward: [(0, '8709.472'), (1, '9261.122')] [2023-12-26 23:32:56,092][105620] Updated weights for policy 1, policy_version 1123655 (0.0006) [2023-12-26 23:32:56,144][105620] Updated weights for policy 1, policy_version 1123665 (0.0010) [2023-12-26 23:32:56,389][105692] Updated weights for policy 0, policy_version 1122339 (0.0009) [2023-12-26 23:32:56,441][105692] Updated weights for policy 0, policy_version 1122349 (0.0010) [2023-12-26 23:32:56,491][105692] Updated weights for policy 0, policy_version 1122359 (0.0010) [2023-12-26 23:32:56,772][105620] Updated weights for policy 1, policy_version 1123675 (0.0009) [2023-12-26 23:32:56,834][105620] Updated weights for policy 1, policy_version 1123685 (0.0005) [2023-12-26 23:32:56,886][105620] Updated weights for policy 1, policy_version 1123695 (0.0005) [2023-12-26 23:32:57,095][105692] Updated weights for policy 0, policy_version 1122369 (0.0005) [2023-12-26 23:32:57,155][105692] Updated weights for policy 0, policy_version 1122379 (0.0005) [2023-12-26 23:32:57,220][105692] Updated weights for policy 0, policy_version 1122389 (0.0005) [2023-12-26 23:32:57,276][105692] Updated weights for policy 0, policy_version 1122399 (0.0005) [2023-12-26 23:32:57,485][105620] Updated weights for policy 1, policy_version 1123705 (0.0008) [2023-12-26 23:32:57,531][105620] Updated weights for policy 1, policy_version 1123715 (0.0005) [2023-12-26 23:32:57,584][105620] Updated weights for policy 1, policy_version 1123725 (0.0005) [2023-12-26 23:32:57,645][105620] Updated weights for policy 1, policy_version 1123735 (0.0005) [2023-12-26 23:32:57,805][105692] Updated weights for policy 0, policy_version 1122409 (0.0010) [2023-12-26 23:32:57,875][105692] Updated weights for policy 0, policy_version 1122419 (0.0006) [2023-12-26 23:32:57,927][105692] Updated weights for policy 0, policy_version 1122429 (0.0010) [2023-12-26 23:32:58,281][105620] Updated weights for policy 1, policy_version 1123745 (0.0006) [2023-12-26 23:32:58,357][105620] Updated weights for policy 1, policy_version 1123755 (0.0007) [2023-12-26 23:32:58,418][105620] Updated weights for policy 1, policy_version 1123765 (0.0011) [2023-12-26 23:32:58,592][105692] Updated weights for policy 0, policy_version 1122439 (0.0007) [2023-12-26 23:32:58,660][105692] Updated weights for policy 0, policy_version 1122449 (0.0010) [2023-12-26 23:32:58,719][105692] Updated weights for policy 0, policy_version 1122459 (0.0010) [2023-12-26 23:32:59,208][105620] Updated weights for policy 1, policy_version 1123775 (0.0008) [2023-12-26 23:32:59,284][105620] Updated weights for policy 1, policy_version 1123785 (0.0008) [2023-12-26 23:32:59,350][105620] Updated weights for policy 1, policy_version 1123795 (0.0009) [2023-12-26 23:32:59,536][105692] Updated weights for policy 0, policy_version 1122469 (0.0010) [2023-12-26 23:32:59,602][105692] Updated weights for policy 0, policy_version 1122479 (0.0009) [2023-12-26 23:32:59,660][105692] Updated weights for policy 0, policy_version 1122489 (0.0010) [2023-12-26 23:33:00,131][105620] Updated weights for policy 1, policy_version 1123805 (0.0008) [2023-12-26 23:33:00,175][105620] Updated weights for policy 1, policy_version 1123815 (0.0007) [2023-12-26 23:33:00,223][105620] Updated weights for policy 1, policy_version 1123825 (0.0008) [2023-12-26 23:33:00,401][105692] Updated weights for policy 0, policy_version 1122499 (0.0010) [2023-12-26 23:33:00,452][105692] Updated weights for policy 0, policy_version 1122509 (0.0010) [2023-12-26 23:33:00,504][105692] Updated weights for policy 0, policy_version 1122519 (0.0010) [2023-12-26 23:33:01,007][105620] Updated weights for policy 1, policy_version 1123835 (0.0007) [2023-12-26 23:33:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 575143936. Throughput: 0: 10151.0, 1: 9748.1. Samples: 575120972. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:33:01,063][104569] Avg episode reward: [(0, '8531.142'), (1, '9260.917')] [2023-12-26 23:33:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001122528_287408128.pth... [2023-12-26 23:33:01,070][105620] Updated weights for policy 1, policy_version 1123845 (0.0008) [2023-12-26 23:33:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001121344_287105024.pth [2023-12-26 23:33:01,128][105620] Updated weights for policy 1, policy_version 1123855 (0.0008) [2023-12-26 23:33:01,174][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001123864_287744000.pth... [2023-12-26 23:33:01,178][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001122712_287449088.pth [2023-12-26 23:33:01,261][105692] Updated weights for policy 0, policy_version 1122529 (0.0010) [2023-12-26 23:33:01,318][105692] Updated weights for policy 0, policy_version 1122539 (0.0011) [2023-12-26 23:33:01,385][105692] Updated weights for policy 0, policy_version 1122549 (0.0009) [2023-12-26 23:33:01,440][105692] Updated weights for policy 0, policy_version 1122559 (0.0009) [2023-12-26 23:33:01,859][105620] Updated weights for policy 1, policy_version 1123865 (0.0008) [2023-12-26 23:33:01,915][105620] Updated weights for policy 1, policy_version 1123875 (0.0005) [2023-12-26 23:33:01,961][105620] Updated weights for policy 1, policy_version 1123885 (0.0005) [2023-12-26 23:33:02,007][105620] Updated weights for policy 1, policy_version 1123895 (0.0005) [2023-12-26 23:33:02,228][105692] Updated weights for policy 0, policy_version 1122569 (0.0011) [2023-12-26 23:33:02,292][105692] Updated weights for policy 0, policy_version 1122579 (0.0011) [2023-12-26 23:33:02,297][105585] KL-divergence is very high: 117.9042 [2023-12-26 23:33:02,346][105585] KL-divergence is very high: 128.3610 [2023-12-26 23:33:02,353][105692] Updated weights for policy 0, policy_version 1122589 (0.0011) [2023-12-26 23:33:02,671][105620] Updated weights for policy 1, policy_version 1123905 (0.0008) [2023-12-26 23:33:02,731][105620] Updated weights for policy 1, policy_version 1123915 (0.0008) [2023-12-26 23:33:02,792][105620] Updated weights for policy 1, policy_version 1123925 (0.0008) [2023-12-26 23:33:03,130][105692] Updated weights for policy 0, policy_version 1122599 (0.0011) [2023-12-26 23:33:03,184][105692] Updated weights for policy 0, policy_version 1122609 (0.0010) [2023-12-26 23:33:03,231][105692] Updated weights for policy 0, policy_version 1122619 (0.0010) [2023-12-26 23:33:03,548][105620] Updated weights for policy 1, policy_version 1123935 (0.0009) [2023-12-26 23:33:03,603][105620] Updated weights for policy 1, policy_version 1123945 (0.0009) [2023-12-26 23:33:03,657][105620] Updated weights for policy 1, policy_version 1123955 (0.0005) [2023-12-26 23:33:03,850][105692] Updated weights for policy 0, policy_version 1122629 (0.0008) [2023-12-26 23:33:03,912][105692] Updated weights for policy 0, policy_version 1122639 (0.0010) [2023-12-26 23:33:03,978][105692] Updated weights for policy 0, policy_version 1122649 (0.0011) [2023-12-26 23:33:04,417][105620] Updated weights for policy 1, policy_version 1123965 (0.0009) [2023-12-26 23:33:04,469][105620] Updated weights for policy 1, policy_version 1123975 (0.0008) [2023-12-26 23:33:04,525][105620] Updated weights for policy 1, policy_version 1123985 (0.0008) [2023-12-26 23:33:04,684][105692] Updated weights for policy 0, policy_version 1122659 (0.0009) [2023-12-26 23:33:04,734][105692] Updated weights for policy 0, policy_version 1122669 (0.0005) [2023-12-26 23:33:04,785][105692] Updated weights for policy 0, policy_version 1122679 (0.0005) [2023-12-26 23:33:05,239][105620] Updated weights for policy 1, policy_version 1123995 (0.0007) [2023-12-26 23:33:05,301][105692] Updated weights for policy 0, policy_version 1122689 (0.0005) [2023-12-26 23:33:05,310][105620] Updated weights for policy 1, policy_version 1124005 (0.0005) [2023-12-26 23:33:05,364][105692] Updated weights for policy 0, policy_version 1122699 (0.0008) [2023-12-26 23:33:05,365][105620] Updated weights for policy 1, policy_version 1124015 (0.0005) [2023-12-26 23:33:05,416][105692] Updated weights for policy 0, policy_version 1122709 (0.0010) [2023-12-26 23:33:05,467][105692] Updated weights for policy 0, policy_version 1122719 (0.0010) [2023-12-26 23:33:06,021][105620] Updated weights for policy 1, policy_version 1124025 (0.0005) [2023-12-26 23:33:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 575242240. Throughput: 0: 10075.6, 1: 9664.7. Samples: 575233116. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:33:06,063][104569] Avg episode reward: [(0, '8263.302'), (1, '9168.816')] [2023-12-26 23:33:06,075][105620] Updated weights for policy 1, policy_version 1124035 (0.0006) [2023-12-26 23:33:06,133][105620] Updated weights for policy 1, policy_version 1124045 (0.0008) [2023-12-26 23:33:06,195][105620] Updated weights for policy 1, policy_version 1124055 (0.0008) [2023-12-26 23:33:06,201][105692] Updated weights for policy 0, policy_version 1122729 (0.0011) [2023-12-26 23:33:06,268][105692] Updated weights for policy 0, policy_version 1122739 (0.0010) [2023-12-26 23:33:06,333][105692] Updated weights for policy 0, policy_version 1122749 (0.0006) [2023-12-26 23:33:06,851][105620] Updated weights for policy 1, policy_version 1124065 (0.0006) [2023-12-26 23:33:06,916][105620] Updated weights for policy 1, policy_version 1124075 (0.0006) [2023-12-26 23:33:06,943][105692] Updated weights for policy 0, policy_version 1122759 (0.0009) [2023-12-26 23:33:06,978][105620] Updated weights for policy 1, policy_version 1124085 (0.0007) [2023-12-26 23:33:07,003][105692] Updated weights for policy 0, policy_version 1122769 (0.0011) [2023-12-26 23:33:07,070][105692] Updated weights for policy 0, policy_version 1122779 (0.0011) [2023-12-26 23:33:07,555][105620] Updated weights for policy 1, policy_version 1124095 (0.0007) [2023-12-26 23:33:07,604][105620] Updated weights for policy 1, policy_version 1124105 (0.0008) [2023-12-26 23:33:07,656][105620] Updated weights for policy 1, policy_version 1124115 (0.0008) [2023-12-26 23:33:07,806][105692] Updated weights for policy 0, policy_version 1122789 (0.0011) [2023-12-26 23:33:07,858][105692] Updated weights for policy 0, policy_version 1122799 (0.0010) [2023-12-26 23:33:07,910][105692] Updated weights for policy 0, policy_version 1122809 (0.0011) [2023-12-26 23:33:08,454][105620] Updated weights for policy 1, policy_version 1124125 (0.0008) [2023-12-26 23:33:08,514][105620] Updated weights for policy 1, policy_version 1124135 (0.0008) [2023-12-26 23:33:08,580][105620] Updated weights for policy 1, policy_version 1124145 (0.0009) [2023-12-26 23:33:08,667][105692] Updated weights for policy 0, policy_version 1122819 (0.0009) [2023-12-26 23:33:08,697][105585] KL-divergence is very high: 244.1775 [2023-12-26 23:33:08,725][105692] Updated weights for policy 0, policy_version 1122829 (0.0005) [2023-12-26 23:33:08,743][105585] KL-divergence is very high: 432.9971 [2023-12-26 23:33:08,786][105692] Updated weights for policy 0, policy_version 1122839 (0.0008) [2023-12-26 23:33:08,792][105585] KL-divergence is very high: 463.4254 [2023-12-26 23:33:09,355][105620] Updated weights for policy 1, policy_version 1124155 (0.0008) [2023-12-26 23:33:09,417][105692] Updated weights for policy 0, policy_version 1122849 (0.0010) [2023-12-26 23:33:09,420][105620] Updated weights for policy 1, policy_version 1124165 (0.0008) [2023-12-26 23:33:09,474][105620] Updated weights for policy 1, policy_version 1124175 (0.0009) [2023-12-26 23:33:09,474][105692] Updated weights for policy 0, policy_version 1122859 (0.0009) [2023-12-26 23:33:09,528][105692] Updated weights for policy 0, policy_version 1122869 (0.0009) [2023-12-26 23:33:09,584][105692] Updated weights for policy 0, policy_version 1122879 (0.0008) [2023-12-26 23:33:10,226][105692] Updated weights for policy 0, policy_version 1122889 (0.0006) [2023-12-26 23:33:10,255][105620] Updated weights for policy 1, policy_version 1124185 (0.0008) [2023-12-26 23:33:10,289][105692] Updated weights for policy 0, policy_version 1122899 (0.0006) [2023-12-26 23:33:10,318][105620] Updated weights for policy 1, policy_version 1124195 (0.0009) [2023-12-26 23:33:10,357][105692] Updated weights for policy 0, policy_version 1122909 (0.0006) [2023-12-26 23:33:10,377][105620] Updated weights for policy 1, policy_version 1124205 (0.0007) [2023-12-26 23:33:10,444][105620] Updated weights for policy 1, policy_version 1124215 (0.0010) [2023-12-26 23:33:10,986][105692] Updated weights for policy 0, policy_version 1122919 (0.0007) [2023-12-26 23:33:11,049][105692] Updated weights for policy 0, policy_version 1122929 (0.0007) [2023-12-26 23:33:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 575340544. Throughput: 0: 10210.6, 1: 9687.0. Samples: 575354012. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:33:11,063][104569] Avg episode reward: [(0, '8354.837'), (1, '9168.428')] [2023-12-26 23:33:11,119][105692] Updated weights for policy 0, policy_version 1122939 (0.0007) [2023-12-26 23:33:11,216][105620] Updated weights for policy 1, policy_version 1124225 (0.0010) [2023-12-26 23:33:11,277][105620] Updated weights for policy 1, policy_version 1124235 (0.0008) [2023-12-26 23:33:11,346][105620] Updated weights for policy 1, policy_version 1124245 (0.0008) [2023-12-26 23:33:11,803][105692] Updated weights for policy 0, policy_version 1122949 (0.0009) [2023-12-26 23:33:11,861][105692] Updated weights for policy 0, policy_version 1122959 (0.0007) [2023-12-26 23:33:11,920][105692] Updated weights for policy 0, policy_version 1122969 (0.0009) [2023-12-26 23:33:12,070][105620] Updated weights for policy 1, policy_version 1124255 (0.0009) [2023-12-26 23:33:12,126][105620] Updated weights for policy 1, policy_version 1124265 (0.0009) [2023-12-26 23:33:12,188][105620] Updated weights for policy 1, policy_version 1124275 (0.0008) [2023-12-26 23:33:12,649][105692] Updated weights for policy 0, policy_version 1122979 (0.0009) [2023-12-26 23:33:12,710][105692] Updated weights for policy 0, policy_version 1122989 (0.0008) [2023-12-26 23:33:12,773][105692] Updated weights for policy 0, policy_version 1122999 (0.0009) [2023-12-26 23:33:12,909][105620] Updated weights for policy 1, policy_version 1124285 (0.0009) [2023-12-26 23:33:12,980][105620] Updated weights for policy 1, policy_version 1124295 (0.0010) [2023-12-26 23:33:13,049][105620] Updated weights for policy 1, policy_version 1124305 (0.0009) [2023-12-26 23:33:13,387][105692] Updated weights for policy 0, policy_version 1123009 (0.0009) [2023-12-26 23:33:13,445][105692] Updated weights for policy 0, policy_version 1123019 (0.0008) [2023-12-26 23:33:13,502][105692] Updated weights for policy 0, policy_version 1123029 (0.0009) [2023-12-26 23:33:13,573][105692] Updated weights for policy 0, policy_version 1123039 (0.0009) [2023-12-26 23:33:13,783][105620] Updated weights for policy 1, policy_version 1124315 (0.0010) [2023-12-26 23:33:13,834][105620] Updated weights for policy 1, policy_version 1124325 (0.0009) [2023-12-26 23:33:13,885][105620] Updated weights for policy 1, policy_version 1124335 (0.0005) [2023-12-26 23:33:14,210][105692] Updated weights for policy 0, policy_version 1123049 (0.0008) [2023-12-26 23:33:14,260][105692] Updated weights for policy 0, policy_version 1123059 (0.0006) [2023-12-26 23:33:14,306][105692] Updated weights for policy 0, policy_version 1123069 (0.0005) [2023-12-26 23:33:14,444][105620] Updated weights for policy 1, policy_version 1124345 (0.0005) [2023-12-26 23:33:14,507][105620] Updated weights for policy 1, policy_version 1124355 (0.0006) [2023-12-26 23:33:14,561][105620] Updated weights for policy 1, policy_version 1124365 (0.0010) [2023-12-26 23:33:14,621][105620] Updated weights for policy 1, policy_version 1124375 (0.0008) [2023-12-26 23:33:14,962][105692] Updated weights for policy 0, policy_version 1123079 (0.0009) [2023-12-26 23:33:15,029][105692] Updated weights for policy 0, policy_version 1123089 (0.0011) [2023-12-26 23:33:15,088][105692] Updated weights for policy 0, policy_version 1123099 (0.0010) [2023-12-26 23:33:15,260][105620] Updated weights for policy 1, policy_version 1124385 (0.0008) [2023-12-26 23:33:15,325][105620] Updated weights for policy 1, policy_version 1124395 (0.0008) [2023-12-26 23:33:15,391][105620] Updated weights for policy 1, policy_version 1124405 (0.0009) [2023-12-26 23:33:15,829][105692] Updated weights for policy 0, policy_version 1123109 (0.0010) [2023-12-26 23:33:15,891][105692] Updated weights for policy 0, policy_version 1123119 (0.0010) [2023-12-26 23:33:15,958][105692] Updated weights for policy 0, policy_version 1123129 (0.0010) [2023-12-26 23:33:16,040][105620] Updated weights for policy 1, policy_version 1124415 (0.0006) [2023-12-26 23:33:16,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 575447040. Throughput: 0: 10104.5, 1: 9665.8. Samples: 575412004. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:33:16,062][104569] Avg episode reward: [(0, '8627.598'), (1, '9259.390')] [2023-12-26 23:33:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001123136_287563776.pth... [2023-12-26 23:33:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001121952_287260672.pth [2023-12-26 23:33:16,092][105620] Updated weights for policy 1, policy_version 1124425 (0.0006) [2023-12-26 23:33:16,151][105620] Updated weights for policy 1, policy_version 1124435 (0.0007) [2023-12-26 23:33:16,187][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001124440_287891456.pth... [2023-12-26 23:33:16,192][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001123288_287596544.pth [2023-12-26 23:33:16,629][105692] Updated weights for policy 0, policy_version 1123139 (0.0009) [2023-12-26 23:33:16,688][105692] Updated weights for policy 0, policy_version 1123149 (0.0007) [2023-12-26 23:33:16,750][105692] Updated weights for policy 0, policy_version 1123159 (0.0008) [2023-12-26 23:33:16,753][105620] Updated weights for policy 1, policy_version 1124445 (0.0010) [2023-12-26 23:33:16,804][105620] Updated weights for policy 1, policy_version 1124455 (0.0010) [2023-12-26 23:33:16,865][105620] Updated weights for policy 1, policy_version 1124465 (0.0010) [2023-12-26 23:33:17,346][105692] Updated weights for policy 0, policy_version 1123169 (0.0005) [2023-12-26 23:33:17,417][105692] Updated weights for policy 0, policy_version 1123179 (0.0005) [2023-12-26 23:33:17,475][105692] Updated weights for policy 0, policy_version 1123189 (0.0005) [2023-12-26 23:33:17,512][105620] Updated weights for policy 1, policy_version 1124475 (0.0009) [2023-12-26 23:33:17,528][105692] Updated weights for policy 0, policy_version 1123199 (0.0008) [2023-12-26 23:33:17,563][105620] Updated weights for policy 1, policy_version 1124485 (0.0007) [2023-12-26 23:33:17,619][105620] Updated weights for policy 1, policy_version 1124495 (0.0005) [2023-12-26 23:33:18,186][105692] Updated weights for policy 0, policy_version 1123209 (0.0008) [2023-12-26 23:33:18,235][105692] Updated weights for policy 0, policy_version 1123219 (0.0008) [2023-12-26 23:33:18,287][105692] Updated weights for policy 0, policy_version 1123229 (0.0008) [2023-12-26 23:33:18,334][105620] Updated weights for policy 1, policy_version 1124505 (0.0010) [2023-12-26 23:33:18,396][105620] Updated weights for policy 1, policy_version 1124515 (0.0009) [2023-12-26 23:33:18,451][105620] Updated weights for policy 1, policy_version 1124525 (0.0010) [2023-12-26 23:33:18,506][105620] Updated weights for policy 1, policy_version 1124535 (0.0010) [2023-12-26 23:33:19,088][105692] Updated weights for policy 0, policy_version 1123239 (0.0006) [2023-12-26 23:33:19,140][105692] Updated weights for policy 0, policy_version 1123249 (0.0005) [2023-12-26 23:33:19,205][105692] Updated weights for policy 0, policy_version 1123259 (0.0005) [2023-12-26 23:33:19,268][105620] Updated weights for policy 1, policy_version 1124545 (0.0011) [2023-12-26 23:33:19,324][105620] Updated weights for policy 1, policy_version 1124555 (0.0010) [2023-12-26 23:33:19,394][105620] Updated weights for policy 1, policy_version 1124565 (0.0011) [2023-12-26 23:33:19,815][105692] Updated weights for policy 0, policy_version 1123269 (0.0008) [2023-12-26 23:33:19,879][105692] Updated weights for policy 0, policy_version 1123279 (0.0009) [2023-12-26 23:33:19,941][105692] Updated weights for policy 0, policy_version 1123289 (0.0008) [2023-12-26 23:33:20,112][105620] Updated weights for policy 1, policy_version 1124575 (0.0009) [2023-12-26 23:33:20,176][105620] Updated weights for policy 1, policy_version 1124585 (0.0011) [2023-12-26 23:33:20,243][105620] Updated weights for policy 1, policy_version 1124595 (0.0011) [2023-12-26 23:33:20,668][105692] Updated weights for policy 0, policy_version 1123299 (0.0006) [2023-12-26 23:33:20,736][105692] Updated weights for policy 0, policy_version 1123309 (0.0008) [2023-12-26 23:33:20,802][105692] Updated weights for policy 0, policy_version 1123319 (0.0008) [2023-12-26 23:33:20,985][105620] Updated weights for policy 1, policy_version 1124605 (0.0011) [2023-12-26 23:33:21,045][105620] Updated weights for policy 1, policy_version 1124615 (0.0010) [2023-12-26 23:33:21,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 575545344. Throughput: 0: 10102.6, 1: 9785.4. Samples: 575536032. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:33:21,062][104569] Avg episode reward: [(0, '8531.105'), (1, '9258.617')] [2023-12-26 23:33:21,105][105620] Updated weights for policy 1, policy_version 1124625 (0.0006) [2023-12-26 23:33:21,605][105692] Updated weights for policy 0, policy_version 1123329 (0.0009) [2023-12-26 23:33:21,677][105692] Updated weights for policy 0, policy_version 1123339 (0.0008) [2023-12-26 23:33:21,749][105692] Updated weights for policy 0, policy_version 1123349 (0.0009) [2023-12-26 23:33:21,806][105620] Updated weights for policy 1, policy_version 1124635 (0.0008) [2023-12-26 23:33:21,812][105692] Updated weights for policy 0, policy_version 1123359 (0.0009) [2023-12-26 23:33:21,874][105620] Updated weights for policy 1, policy_version 1124645 (0.0008) [2023-12-26 23:33:21,939][105620] Updated weights for policy 1, policy_version 1124655 (0.0009) [2023-12-26 23:33:22,586][105692] Updated weights for policy 0, policy_version 1123369 (0.0009) [2023-12-26 23:33:22,634][105620] Updated weights for policy 1, policy_version 1124665 (0.0008) [2023-12-26 23:33:22,640][105692] Updated weights for policy 0, policy_version 1123379 (0.0008) [2023-12-26 23:33:22,696][105620] Updated weights for policy 1, policy_version 1124675 (0.0007) [2023-12-26 23:33:22,700][105692] Updated weights for policy 0, policy_version 1123389 (0.0008) [2023-12-26 23:33:22,754][105620] Updated weights for policy 1, policy_version 1124685 (0.0006) [2023-12-26 23:33:22,808][105620] Updated weights for policy 1, policy_version 1124695 (0.0009) [2023-12-26 23:33:23,482][105692] Updated weights for policy 0, policy_version 1123399 (0.0008) [2023-12-26 23:33:23,537][105620] Updated weights for policy 1, policy_version 1124705 (0.0007) [2023-12-26 23:33:23,542][105692] Updated weights for policy 0, policy_version 1123409 (0.0007) [2023-12-26 23:33:23,586][105620] Updated weights for policy 1, policy_version 1124715 (0.0006) [2023-12-26 23:33:23,595][105692] Updated weights for policy 0, policy_version 1123419 (0.0009) [2023-12-26 23:33:23,639][105620] Updated weights for policy 1, policy_version 1124725 (0.0008) [2023-12-26 23:33:24,365][105692] Updated weights for policy 0, policy_version 1123429 (0.0007) [2023-12-26 23:33:24,396][105620] Updated weights for policy 1, policy_version 1124735 (0.0007) [2023-12-26 23:33:24,424][105692] Updated weights for policy 0, policy_version 1123439 (0.0009) [2023-12-26 23:33:24,454][105620] Updated weights for policy 1, policy_version 1124745 (0.0007) [2023-12-26 23:33:24,476][105692] Updated weights for policy 0, policy_version 1123449 (0.0006) [2023-12-26 23:33:24,515][105620] Updated weights for policy 1, policy_version 1124755 (0.0008) [2023-12-26 23:33:25,197][105620] Updated weights for policy 1, policy_version 1124765 (0.0007) [2023-12-26 23:33:25,252][105620] Updated weights for policy 1, policy_version 1124775 (0.0006) [2023-12-26 23:33:25,263][105692] Updated weights for policy 0, policy_version 1123459 (0.0009) [2023-12-26 23:33:25,308][105620] Updated weights for policy 1, policy_version 1124785 (0.0006) [2023-12-26 23:33:25,322][105692] Updated weights for policy 0, policy_version 1123469 (0.0008) [2023-12-26 23:33:25,381][105692] Updated weights for policy 0, policy_version 1123479 (0.0009) [2023-12-26 23:33:25,994][105620] Updated weights for policy 1, policy_version 1124795 (0.0006) [2023-12-26 23:33:26,045][105620] Updated weights for policy 1, policy_version 1124805 (0.0009) [2023-12-26 23:33:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 575635456. Throughput: 0: 10057.3, 1: 9813.8. Samples: 575649284. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:33:26,062][104569] Avg episode reward: [(0, '8529.799'), (1, '9259.176')] [2023-12-26 23:33:26,104][105620] Updated weights for policy 1, policy_version 1124815 (0.0009) [2023-12-26 23:33:26,126][105692] Updated weights for policy 0, policy_version 1123489 (0.0009) [2023-12-26 23:33:26,172][105692] Updated weights for policy 0, policy_version 1123499 (0.0008) [2023-12-26 23:33:26,220][105692] Updated weights for policy 0, policy_version 1123509 (0.0009) [2023-12-26 23:33:26,275][105692] Updated weights for policy 0, policy_version 1123519 (0.0010) [2023-12-26 23:33:26,730][105620] Updated weights for policy 1, policy_version 1124825 (0.0008) [2023-12-26 23:33:26,785][105620] Updated weights for policy 1, policy_version 1124835 (0.0005) [2023-12-26 23:33:26,837][105620] Updated weights for policy 1, policy_version 1124845 (0.0005) [2023-12-26 23:33:26,900][105620] Updated weights for policy 1, policy_version 1124855 (0.0008) [2023-12-26 23:33:27,144][105692] Updated weights for policy 0, policy_version 1123529 (0.0009) [2023-12-26 23:33:27,191][105692] Updated weights for policy 0, policy_version 1123539 (0.0009) [2023-12-26 23:33:27,246][105692] Updated weights for policy 0, policy_version 1123550 (0.0009) [2023-12-26 23:33:27,560][105620] Updated weights for policy 1, policy_version 1124865 (0.0009) [2023-12-26 23:33:27,621][105620] Updated weights for policy 1, policy_version 1124875 (0.0009) [2023-12-26 23:33:27,673][105620] Updated weights for policy 1, policy_version 1124885 (0.0009) [2023-12-26 23:33:28,021][105692] Updated weights for policy 0, policy_version 1123560 (0.0009) [2023-12-26 23:33:28,074][105692] Updated weights for policy 0, policy_version 1123571 (0.0009) [2023-12-26 23:33:28,126][105692] Updated weights for policy 0, policy_version 1123582 (0.0009) [2023-12-26 23:33:28,311][105620] Updated weights for policy 1, policy_version 1124895 (0.0007) [2023-12-26 23:33:28,378][105620] Updated weights for policy 1, policy_version 1124905 (0.0008) [2023-12-26 23:33:28,438][105620] Updated weights for policy 1, policy_version 1124915 (0.0006) [2023-12-26 23:33:29,006][105692] Updated weights for policy 0, policy_version 1123592 (0.0009) [2023-12-26 23:33:29,061][105692] Updated weights for policy 0, policy_version 1123602 (0.0010) [2023-12-26 23:33:29,075][105620] Updated weights for policy 1, policy_version 1124925 (0.0006) [2023-12-26 23:33:29,109][105692] Updated weights for policy 0, policy_version 1123612 (0.0008) [2023-12-26 23:33:29,128][105620] Updated weights for policy 1, policy_version 1124935 (0.0005) [2023-12-26 23:33:29,177][105620] Updated weights for policy 1, policy_version 1124946 (0.0009) [2023-12-26 23:33:29,884][105692] Updated weights for policy 0, policy_version 1123622 (0.0008) [2023-12-26 23:33:29,918][105620] Updated weights for policy 1, policy_version 1124956 (0.0008) [2023-12-26 23:33:29,946][105692] Updated weights for policy 0, policy_version 1123632 (0.0007) [2023-12-26 23:33:29,975][105620] Updated weights for policy 1, policy_version 1124966 (0.0007) [2023-12-26 23:33:30,006][105692] Updated weights for policy 0, policy_version 1123642 (0.0007) [2023-12-26 23:33:30,029][105620] Updated weights for policy 1, policy_version 1124976 (0.0006) [2023-12-26 23:33:30,688][105692] Updated weights for policy 0, policy_version 1123652 (0.0006) [2023-12-26 23:33:30,749][105692] Updated weights for policy 0, policy_version 1123662 (0.0008) [2023-12-26 23:33:30,810][105620] Updated weights for policy 1, policy_version 1124986 (0.0008) [2023-12-26 23:33:30,812][105692] Updated weights for policy 0, policy_version 1123672 (0.0008) [2023-12-26 23:33:30,861][105620] Updated weights for policy 1, policy_version 1124996 (0.0007) [2023-12-26 23:33:30,908][105620] Updated weights for policy 1, policy_version 1125006 (0.0009) [2023-12-26 23:33:30,959][105620] Updated weights for policy 1, policy_version 1125016 (0.0006) [2023-12-26 23:33:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 575741952. Throughput: 0: 9944.6, 1: 9891.0. Samples: 575707720. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:33:31,062][104569] Avg episode reward: [(0, '8624.066'), (1, '9349.731')] [2023-12-26 23:33:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001123680_287703040.pth... [2023-12-26 23:33:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001125016_288038912.pth... [2023-12-26 23:33:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001122528_287408128.pth [2023-12-26 23:33:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001123864_287744000.pth [2023-12-26 23:33:31,577][105692] Updated weights for policy 0, policy_version 1123682 (0.0009) [2023-12-26 23:33:31,641][105692] Updated weights for policy 0, policy_version 1123692 (0.0007) [2023-12-26 23:33:31,646][105620] Updated weights for policy 1, policy_version 1125026 (0.0007) [2023-12-26 23:33:31,701][105692] Updated weights for policy 0, policy_version 1123702 (0.0007) [2023-12-26 23:33:31,703][105620] Updated weights for policy 1, policy_version 1125036 (0.0007) [2023-12-26 23:33:31,765][105692] Updated weights for policy 0, policy_version 1123712 (0.0007) [2023-12-26 23:33:31,767][105620] Updated weights for policy 1, policy_version 1125046 (0.0008) [2023-12-26 23:33:32,493][105620] Updated weights for policy 1, policy_version 1125056 (0.0006) [2023-12-26 23:33:32,531][105692] Updated weights for policy 0, policy_version 1123722 (0.0008) [2023-12-26 23:33:32,551][105620] Updated weights for policy 1, policy_version 1125066 (0.0006) [2023-12-26 23:33:32,596][105692] Updated weights for policy 0, policy_version 1123732 (0.0008) [2023-12-26 23:33:32,600][105620] Updated weights for policy 1, policy_version 1125076 (0.0005) [2023-12-26 23:33:32,650][105692] Updated weights for policy 0, policy_version 1123742 (0.0008) [2023-12-26 23:33:33,266][105692] Updated weights for policy 0, policy_version 1123752 (0.0009) [2023-12-26 23:33:33,327][105692] Updated weights for policy 0, policy_version 1123762 (0.0009) [2023-12-26 23:33:33,355][105620] Updated weights for policy 1, policy_version 1125086 (0.0006) [2023-12-26 23:33:33,382][105692] Updated weights for policy 0, policy_version 1123772 (0.0008) [2023-12-26 23:33:33,405][105620] Updated weights for policy 1, policy_version 1125096 (0.0005) [2023-12-26 23:33:33,452][105620] Updated weights for policy 1, policy_version 1125106 (0.0008) [2023-12-26 23:33:34,089][105620] Updated weights for policy 1, policy_version 1125116 (0.0007) [2023-12-26 23:33:34,136][105620] Updated weights for policy 1, policy_version 1125126 (0.0009) [2023-12-26 23:33:34,187][105692] Updated weights for policy 0, policy_version 1123782 (0.0007) [2023-12-26 23:33:34,197][105620] Updated weights for policy 1, policy_version 1125136 (0.0007) [2023-12-26 23:33:34,247][105692] Updated weights for policy 0, policy_version 1123792 (0.0008) [2023-12-26 23:33:34,310][105692] Updated weights for policy 0, policy_version 1123802 (0.0010) [2023-12-26 23:33:34,808][105620] Updated weights for policy 1, policy_version 1125146 (0.0006) [2023-12-26 23:33:34,863][105620] Updated weights for policy 1, policy_version 1125156 (0.0009) [2023-12-26 23:33:34,915][105620] Updated weights for policy 1, policy_version 1125166 (0.0010) [2023-12-26 23:33:35,116][105692] Updated weights for policy 0, policy_version 1123812 (0.0009) [2023-12-26 23:33:35,174][105692] Updated weights for policy 0, policy_version 1123822 (0.0009) [2023-12-26 23:33:35,227][105692] Updated weights for policy 0, policy_version 1123832 (0.0008) [2023-12-26 23:33:35,649][105620] Updated weights for policy 1, policy_version 1125177 (0.0009) [2023-12-26 23:33:35,712][105620] Updated weights for policy 1, policy_version 1125187 (0.0009) [2023-12-26 23:33:35,766][105620] Updated weights for policy 1, policy_version 1125197 (0.0008) [2023-12-26 23:33:35,830][105620] Updated weights for policy 1, policy_version 1125207 (0.0007) [2023-12-26 23:33:35,977][105692] Updated weights for policy 0, policy_version 1123842 (0.0009) [2023-12-26 23:33:36,044][105692] Updated weights for policy 0, policy_version 1123852 (0.0010) [2023-12-26 23:33:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 575832064. Throughput: 0: 9798.3, 1: 9874.7. Samples: 575822620. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:33:36,062][104569] Avg episode reward: [(0, '8443.902'), (1, '9346.791')] [2023-12-26 23:33:36,102][105692] Updated weights for policy 0, policy_version 1123862 (0.0009) [2023-12-26 23:33:36,165][105692] Updated weights for policy 0, policy_version 1123872 (0.0009) [2023-12-26 23:33:36,520][105620] Updated weights for policy 1, policy_version 1125217 (0.0009) [2023-12-26 23:33:36,585][105620] Updated weights for policy 1, policy_version 1125227 (0.0007) [2023-12-26 23:33:36,641][105620] Updated weights for policy 1, policy_version 1125237 (0.0008) [2023-12-26 23:33:36,886][105692] Updated weights for policy 0, policy_version 1123882 (0.0006) [2023-12-26 23:33:36,947][105692] Updated weights for policy 0, policy_version 1123892 (0.0005) [2023-12-26 23:33:37,016][105692] Updated weights for policy 0, policy_version 1123902 (0.0005) [2023-12-26 23:33:37,322][105620] Updated weights for policy 1, policy_version 1125247 (0.0008) [2023-12-26 23:33:37,370][105620] Updated weights for policy 1, policy_version 1125257 (0.0009) [2023-12-26 23:33:37,418][105620] Updated weights for policy 1, policy_version 1125267 (0.0009) [2023-12-26 23:33:37,696][105692] Updated weights for policy 0, policy_version 1123912 (0.0009) [2023-12-26 23:33:37,759][105692] Updated weights for policy 0, policy_version 1123922 (0.0008) [2023-12-26 23:33:37,811][105692] Updated weights for policy 0, policy_version 1123932 (0.0008) [2023-12-26 23:33:38,134][105620] Updated weights for policy 1, policy_version 1125277 (0.0010) [2023-12-26 23:33:38,194][105620] Updated weights for policy 1, policy_version 1125287 (0.0011) [2023-12-26 23:33:38,257][105620] Updated weights for policy 1, policy_version 1125297 (0.0011) [2023-12-26 23:33:38,505][105692] Updated weights for policy 0, policy_version 1123942 (0.0006) [2023-12-26 23:33:38,560][105692] Updated weights for policy 0, policy_version 1123952 (0.0005) [2023-12-26 23:33:38,608][105692] Updated weights for policy 0, policy_version 1123962 (0.0005) [2023-12-26 23:33:38,958][105620] Updated weights for policy 1, policy_version 1125307 (0.0009) [2023-12-26 23:33:39,028][105620] Updated weights for policy 1, policy_version 1125317 (0.0008) [2023-12-26 23:33:39,103][105620] Updated weights for policy 1, policy_version 1125327 (0.0006) [2023-12-26 23:33:39,243][105692] Updated weights for policy 0, policy_version 1123972 (0.0007) [2023-12-26 23:33:39,306][105692] Updated weights for policy 0, policy_version 1123982 (0.0008) [2023-12-26 23:33:39,370][105692] Updated weights for policy 0, policy_version 1123992 (0.0008) [2023-12-26 23:33:39,712][105620] Updated weights for policy 1, policy_version 1125337 (0.0007) [2023-12-26 23:33:39,771][105620] Updated weights for policy 1, policy_version 1125347 (0.0006) [2023-12-26 23:33:39,834][105620] Updated weights for policy 1, policy_version 1125357 (0.0009) [2023-12-26 23:33:39,898][105620] Updated weights for policy 1, policy_version 1125367 (0.0009) [2023-12-26 23:33:40,190][105692] Updated weights for policy 0, policy_version 1124002 (0.0009) [2023-12-26 23:33:40,256][105692] Updated weights for policy 0, policy_version 1124012 (0.0007) [2023-12-26 23:33:40,315][105692] Updated weights for policy 0, policy_version 1124022 (0.0009) [2023-12-26 23:33:40,373][105692] Updated weights for policy 0, policy_version 1124032 (0.0007) [2023-12-26 23:33:40,685][105620] Updated weights for policy 1, policy_version 1125377 (0.0010) [2023-12-26 23:33:40,748][105620] Updated weights for policy 1, policy_version 1125387 (0.0010) [2023-12-26 23:33:40,814][105620] Updated weights for policy 1, policy_version 1125397 (0.0009) [2023-12-26 23:33:40,968][105692] Updated weights for policy 0, policy_version 1124042 (0.0005) [2023-12-26 23:33:41,025][105692] Updated weights for policy 0, policy_version 1124052 (0.0006) [2023-12-26 23:33:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 575930368. Throughput: 0: 9729.6, 1: 9894.1. Samples: 575939396. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:33:41,062][104569] Avg episode reward: [(0, '8440.523'), (1, '9253.926')] [2023-12-26 23:33:41,084][105692] Updated weights for policy 0, policy_version 1124062 (0.0008) [2023-12-26 23:33:41,673][105620] Updated weights for policy 1, policy_version 1125407 (0.0009) [2023-12-26 23:33:41,743][105620] Updated weights for policy 1, policy_version 1125417 (0.0010) [2023-12-26 23:33:41,806][105620] Updated weights for policy 1, policy_version 1125427 (0.0011) [2023-12-26 23:33:41,828][105692] Updated weights for policy 0, policy_version 1124072 (0.0010) [2023-12-26 23:33:41,888][105692] Updated weights for policy 0, policy_version 1124082 (0.0011) [2023-12-26 23:33:41,951][105692] Updated weights for policy 0, policy_version 1124092 (0.0011) [2023-12-26 23:33:42,535][105620] Updated weights for policy 1, policy_version 1125437 (0.0010) [2023-12-26 23:33:42,605][105620] Updated weights for policy 1, policy_version 1125447 (0.0010) [2023-12-26 23:33:42,661][105620] Updated weights for policy 1, policy_version 1125457 (0.0009) [2023-12-26 23:33:42,671][105692] Updated weights for policy 0, policy_version 1124102 (0.0008) [2023-12-26 23:33:42,730][105692] Updated weights for policy 0, policy_version 1124112 (0.0009) [2023-12-26 23:33:42,788][105692] Updated weights for policy 0, policy_version 1124122 (0.0008) [2023-12-26 23:33:43,334][105620] Updated weights for policy 1, policy_version 1125467 (0.0007) [2023-12-26 23:33:43,386][105692] Updated weights for policy 0, policy_version 1124132 (0.0008) [2023-12-26 23:33:43,394][105620] Updated weights for policy 1, policy_version 1125477 (0.0008) [2023-12-26 23:33:43,435][105692] Updated weights for policy 0, policy_version 1124142 (0.0007) [2023-12-26 23:33:43,448][105620] Updated weights for policy 1, policy_version 1125487 (0.0010) [2023-12-26 23:33:43,490][105692] Updated weights for policy 0, policy_version 1124152 (0.0005) [2023-12-26 23:33:44,086][105620] Updated weights for policy 1, policy_version 1125497 (0.0010) [2023-12-26 23:33:44,096][105692] Updated weights for policy 0, policy_version 1124162 (0.0006) [2023-12-26 23:33:44,143][105620] Updated weights for policy 1, policy_version 1125507 (0.0010) [2023-12-26 23:33:44,155][105692] Updated weights for policy 0, policy_version 1124172 (0.0009) [2023-12-26 23:33:44,189][105620] Updated weights for policy 1, policy_version 1125517 (0.0008) [2023-12-26 23:33:44,217][105692] Updated weights for policy 0, policy_version 1124182 (0.0010) [2023-12-26 23:33:44,240][105620] Updated weights for policy 1, policy_version 1125527 (0.0009) [2023-12-26 23:33:44,283][105692] Updated weights for policy 0, policy_version 1124192 (0.0010) [2023-12-26 23:33:44,907][105692] Updated weights for policy 0, policy_version 1124202 (0.0010) [2023-12-26 23:33:44,972][105692] Updated weights for policy 0, policy_version 1124212 (0.0010) [2023-12-26 23:33:44,977][105620] Updated weights for policy 1, policy_version 1125537 (0.0010) [2023-12-26 23:33:45,035][105692] Updated weights for policy 0, policy_version 1124222 (0.0011) [2023-12-26 23:33:45,040][105620] Updated weights for policy 1, policy_version 1125547 (0.0010) [2023-12-26 23:33:45,095][105620] Updated weights for policy 1, policy_version 1125557 (0.0010) [2023-12-26 23:33:45,677][105692] Updated weights for policy 0, policy_version 1124232 (0.0006) [2023-12-26 23:33:45,734][105692] Updated weights for policy 0, policy_version 1124242 (0.0005) [2023-12-26 23:33:45,790][105692] Updated weights for policy 0, policy_version 1124252 (0.0005) [2023-12-26 23:33:45,830][105620] Updated weights for policy 1, policy_version 1125567 (0.0010) [2023-12-26 23:33:45,877][105620] Updated weights for policy 1, policy_version 1125577 (0.0010) [2023-12-26 23:33:45,936][105620] Updated weights for policy 1, policy_version 1125587 (0.0010) [2023-12-26 23:33:46,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 576036864. Throughput: 0: 9692.7, 1: 9840.1. Samples: 575999948. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:33:46,062][104569] Avg episode reward: [(0, '8714.641'), (1, '8982.395')] [2023-12-26 23:33:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001125592_288186368.pth... [2023-12-26 23:33:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001124256_287850496.pth... [2023-12-26 23:33:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001124440_287891456.pth [2023-12-26 23:33:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001123136_287563776.pth [2023-12-26 23:33:46,438][105692] Updated weights for policy 0, policy_version 1124262 (0.0009) [2023-12-26 23:33:46,486][105692] Updated weights for policy 0, policy_version 1124272 (0.0010) [2023-12-26 23:33:46,547][105692] Updated weights for policy 0, policy_version 1124282 (0.0010) [2023-12-26 23:33:46,566][105620] Updated weights for policy 1, policy_version 1125597 (0.0008) [2023-12-26 23:33:46,615][105620] Updated weights for policy 1, policy_version 1125607 (0.0006) [2023-12-26 23:33:46,677][105620] Updated weights for policy 1, policy_version 1125617 (0.0007) [2023-12-26 23:33:47,286][105620] Updated weights for policy 1, policy_version 1125627 (0.0007) [2023-12-26 23:33:47,296][105692] Updated weights for policy 0, policy_version 1124292 (0.0010) [2023-12-26 23:33:47,342][105620] Updated weights for policy 1, policy_version 1125637 (0.0006) [2023-12-26 23:33:47,351][105692] Updated weights for policy 0, policy_version 1124302 (0.0010) [2023-12-26 23:33:47,393][105620] Updated weights for policy 1, policy_version 1125647 (0.0005) [2023-12-26 23:33:47,399][105692] Updated weights for policy 0, policy_version 1124312 (0.0010) [2023-12-26 23:33:48,152][105692] Updated weights for policy 0, policy_version 1124322 (0.0010) [2023-12-26 23:33:48,169][105620] Updated weights for policy 1, policy_version 1125657 (0.0008) [2023-12-26 23:33:48,208][105692] Updated weights for policy 0, policy_version 1124332 (0.0006) [2023-12-26 23:33:48,220][105620] Updated weights for policy 1, policy_version 1125667 (0.0009) [2023-12-26 23:33:48,259][105692] Updated weights for policy 0, policy_version 1124342 (0.0005) [2023-12-26 23:33:48,272][105620] Updated weights for policy 1, policy_version 1125678 (0.0009) [2023-12-26 23:33:48,303][105692] Updated weights for policy 0, policy_version 1124352 (0.0005) [2023-12-26 23:33:48,323][105620] Updated weights for policy 1, policy_version 1125688 (0.0007) [2023-12-26 23:33:48,931][105692] Updated weights for policy 0, policy_version 1124362 (0.0011) [2023-12-26 23:33:48,994][105692] Updated weights for policy 0, policy_version 1124372 (0.0011) [2023-12-26 23:33:49,053][105692] Updated weights for policy 0, policy_version 1124382 (0.0011) [2023-12-26 23:33:49,096][105620] Updated weights for policy 1, policy_version 1125698 (0.0011) [2023-12-26 23:33:49,161][105620] Updated weights for policy 1, policy_version 1125708 (0.0010) [2023-12-26 23:33:49,220][105620] Updated weights for policy 1, policy_version 1125718 (0.0011) [2023-12-26 23:33:49,802][105692] Updated weights for policy 0, policy_version 1124392 (0.0010) [2023-12-26 23:33:49,861][105692] Updated weights for policy 0, policy_version 1124402 (0.0011) [2023-12-26 23:33:49,927][105692] Updated weights for policy 0, policy_version 1124412 (0.0010) [2023-12-26 23:33:49,988][105620] Updated weights for policy 1, policy_version 1125728 (0.0009) [2023-12-26 23:33:50,050][105620] Updated weights for policy 1, policy_version 1125738 (0.0008) [2023-12-26 23:33:50,105][105620] Updated weights for policy 1, policy_version 1125748 (0.0008) [2023-12-26 23:33:50,717][105692] Updated weights for policy 0, policy_version 1124422 (0.0011) [2023-12-26 23:33:50,774][105692] Updated weights for policy 0, policy_version 1124432 (0.0011) [2023-12-26 23:33:50,808][105620] Updated weights for policy 1, policy_version 1125758 (0.0007) [2023-12-26 23:33:50,834][105692] Updated weights for policy 0, policy_version 1124442 (0.0011) [2023-12-26 23:33:50,873][105620] Updated weights for policy 1, policy_version 1125768 (0.0007) [2023-12-26 23:33:50,934][105620] Updated weights for policy 1, policy_version 1125778 (0.0009) [2023-12-26 23:33:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 576135168. Throughput: 0: 9812.7, 1: 9898.3. Samples: 576120112. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:33:51,062][104569] Avg episode reward: [(0, '8808.929'), (1, '8985.419')] [2023-12-26 23:33:51,498][105692] Updated weights for policy 0, policy_version 1124452 (0.0010) [2023-12-26 23:33:51,557][105692] Updated weights for policy 0, policy_version 1124462 (0.0011) [2023-12-26 23:33:51,621][105692] Updated weights for policy 0, policy_version 1124472 (0.0011) [2023-12-26 23:33:51,740][105620] Updated weights for policy 1, policy_version 1125788 (0.0009) [2023-12-26 23:33:51,796][105620] Updated weights for policy 1, policy_version 1125798 (0.0008) [2023-12-26 23:33:51,856][105620] Updated weights for policy 1, policy_version 1125808 (0.0008) [2023-12-26 23:33:52,346][105692] Updated weights for policy 0, policy_version 1124482 (0.0011) [2023-12-26 23:33:52,412][105692] Updated weights for policy 0, policy_version 1124492 (0.0011) [2023-12-26 23:33:52,464][105692] Updated weights for policy 0, policy_version 1124502 (0.0011) [2023-12-26 23:33:52,515][105692] Updated weights for policy 0, policy_version 1124512 (0.0010) [2023-12-26 23:33:52,574][105620] Updated weights for policy 1, policy_version 1125818 (0.0007) [2023-12-26 23:33:52,638][105620] Updated weights for policy 1, policy_version 1125828 (0.0009) [2023-12-26 23:33:52,696][105620] Updated weights for policy 1, policy_version 1125838 (0.0010) [2023-12-26 23:33:52,749][105620] Updated weights for policy 1, policy_version 1125848 (0.0009) [2023-12-26 23:33:53,198][105692] Updated weights for policy 0, policy_version 1124522 (0.0009) [2023-12-26 23:33:53,253][105692] Updated weights for policy 0, policy_version 1124532 (0.0009) [2023-12-26 23:33:53,300][105692] Updated weights for policy 0, policy_version 1124542 (0.0009) [2023-12-26 23:33:53,525][105620] Updated weights for policy 1, policy_version 1125858 (0.0008) [2023-12-26 23:33:53,582][105620] Updated weights for policy 1, policy_version 1125868 (0.0005) [2023-12-26 23:33:53,639][105620] Updated weights for policy 1, policy_version 1125878 (0.0006) [2023-12-26 23:33:54,130][105692] Updated weights for policy 0, policy_version 1124552 (0.0008) [2023-12-26 23:33:54,185][105692] Updated weights for policy 0, policy_version 1124562 (0.0009) [2023-12-26 23:33:54,220][105620] Updated weights for policy 1, policy_version 1125888 (0.0008) [2023-12-26 23:33:54,241][105692] Updated weights for policy 0, policy_version 1124572 (0.0008) [2023-12-26 23:33:54,272][105620] Updated weights for policy 1, policy_version 1125898 (0.0008) [2023-12-26 23:33:54,325][105620] Updated weights for policy 1, policy_version 1125908 (0.0008) [2023-12-26 23:33:54,982][105692] Updated weights for policy 0, policy_version 1124582 (0.0007) [2023-12-26 23:33:55,037][105692] Updated weights for policy 0, policy_version 1124592 (0.0007) [2023-12-26 23:33:55,086][105620] Updated weights for policy 1, policy_version 1125918 (0.0010) [2023-12-26 23:33:55,086][105692] Updated weights for policy 0, policy_version 1124602 (0.0009) [2023-12-26 23:33:55,142][105620] Updated weights for policy 1, policy_version 1125928 (0.0006) [2023-12-26 23:33:55,200][105620] Updated weights for policy 1, policy_version 1125938 (0.0009) [2023-12-26 23:33:55,784][105692] Updated weights for policy 0, policy_version 1124612 (0.0010) [2023-12-26 23:33:55,843][105692] Updated weights for policy 0, policy_version 1124622 (0.0010) [2023-12-26 23:33:55,895][105692] Updated weights for policy 0, policy_version 1124632 (0.0010) [2023-12-26 23:33:55,903][105620] Updated weights for policy 1, policy_version 1125948 (0.0007) [2023-12-26 23:33:55,952][105620] Updated weights for policy 1, policy_version 1125958 (0.0005) [2023-12-26 23:33:56,018][105620] Updated weights for policy 1, policy_version 1125968 (0.0005) [2023-12-26 23:33:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 576225280. Throughput: 0: 9697.0, 1: 9885.1. Samples: 576235204. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-26 23:33:56,062][104569] Avg episode reward: [(0, '8537.687'), (1, '9259.892')] [2023-12-26 23:33:56,587][105692] Updated weights for policy 0, policy_version 1124642 (0.0011) [2023-12-26 23:33:56,636][105620] Updated weights for policy 1, policy_version 1125978 (0.0005) [2023-12-26 23:33:56,645][105692] Updated weights for policy 0, policy_version 1124652 (0.0011) [2023-12-26 23:33:56,693][105620] Updated weights for policy 1, policy_version 1125988 (0.0008) [2023-12-26 23:33:56,693][105692] Updated weights for policy 0, policy_version 1124662 (0.0006) [2023-12-26 23:33:56,750][105620] Updated weights for policy 1, policy_version 1125998 (0.0007) [2023-12-26 23:33:56,754][105692] Updated weights for policy 0, policy_version 1124672 (0.0008) [2023-12-26 23:33:56,802][105620] Updated weights for policy 1, policy_version 1126008 (0.0009) [2023-12-26 23:33:57,425][105692] Updated weights for policy 0, policy_version 1124682 (0.0005) [2023-12-26 23:33:57,469][105692] Updated weights for policy 0, policy_version 1124692 (0.0005) [2023-12-26 23:33:57,517][105692] Updated weights for policy 0, policy_version 1124702 (0.0007) [2023-12-26 23:33:57,581][105620] Updated weights for policy 1, policy_version 1126018 (0.0009) [2023-12-26 23:33:57,630][105620] Updated weights for policy 1, policy_version 1126028 (0.0008) [2023-12-26 23:33:57,680][105620] Updated weights for policy 1, policy_version 1126038 (0.0009) [2023-12-26 23:33:58,273][105692] Updated weights for policy 0, policy_version 1124712 (0.0009) [2023-12-26 23:33:58,339][105692] Updated weights for policy 0, policy_version 1124722 (0.0010) [2023-12-26 23:33:58,406][105692] Updated weights for policy 0, policy_version 1124732 (0.0008) [2023-12-26 23:33:58,454][105620] Updated weights for policy 1, policy_version 1126048 (0.0009) [2023-12-26 23:33:58,523][105620] Updated weights for policy 1, policy_version 1126058 (0.0009) [2023-12-26 23:33:58,588][105620] Updated weights for policy 1, policy_version 1126068 (0.0009) [2023-12-26 23:33:59,152][105692] Updated weights for policy 0, policy_version 1124742 (0.0007) [2023-12-26 23:33:59,203][105692] Updated weights for policy 0, policy_version 1124752 (0.0007) [2023-12-26 23:33:59,273][105692] Updated weights for policy 0, policy_version 1124762 (0.0008) [2023-12-26 23:33:59,286][105585] KL-divergence is very high: 257.9847 [2023-12-26 23:33:59,304][105585] KL-divergence is very high: 240.3960 [2023-12-26 23:33:59,451][105620] Updated weights for policy 1, policy_version 1126078 (0.0011) [2023-12-26 23:33:59,517][105620] Updated weights for policy 1, policy_version 1126088 (0.0010) [2023-12-26 23:33:59,576][105620] Updated weights for policy 1, policy_version 1126098 (0.0010) [2023-12-26 23:33:59,922][105692] Updated weights for policy 0, policy_version 1124772 (0.0009) [2023-12-26 23:33:59,986][105692] Updated weights for policy 0, policy_version 1124782 (0.0009) [2023-12-26 23:34:00,052][105692] Updated weights for policy 0, policy_version 1124792 (0.0008) [2023-12-26 23:34:00,252][105620] Updated weights for policy 1, policy_version 1126108 (0.0010) [2023-12-26 23:34:00,309][105620] Updated weights for policy 1, policy_version 1126118 (0.0009) [2023-12-26 23:34:00,368][105620] Updated weights for policy 1, policy_version 1126129 (0.0009) [2023-12-26 23:34:00,609][105692] Updated weights for policy 0, policy_version 1124802 (0.0007) [2023-12-26 23:34:00,660][105692] Updated weights for policy 0, policy_version 1124812 (0.0005) [2023-12-26 23:34:00,719][105692] Updated weights for policy 0, policy_version 1124822 (0.0005) [2023-12-26 23:34:00,767][105692] Updated weights for policy 0, policy_version 1124832 (0.0005) [2023-12-26 23:34:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 576323584. Throughput: 0: 9696.4, 1: 9883.2. Samples: 576293084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:34:01,062][104569] Avg episode reward: [(0, '8170.462'), (1, '9258.303')] [2023-12-26 23:34:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001124832_287997952.pth... [2023-12-26 23:34:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001126136_288325632.pth... [2023-12-26 23:34:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001123680_287703040.pth [2023-12-26 23:34:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001125016_288038912.pth [2023-12-26 23:34:01,118][105620] Updated weights for policy 1, policy_version 1126139 (0.0009) [2023-12-26 23:34:01,182][105620] Updated weights for policy 1, policy_version 1126149 (0.0007) [2023-12-26 23:34:01,237][105620] Updated weights for policy 1, policy_version 1126159 (0.0010) [2023-12-26 23:34:01,385][105692] Updated weights for policy 0, policy_version 1124842 (0.0009) [2023-12-26 23:34:01,436][105692] Updated weights for policy 0, policy_version 1124852 (0.0010) [2023-12-26 23:34:01,491][105692] Updated weights for policy 0, policy_version 1124862 (0.0009) [2023-12-26 23:34:01,916][105620] Updated weights for policy 1, policy_version 1126169 (0.0007) [2023-12-26 23:34:01,974][105620] Updated weights for policy 1, policy_version 1126179 (0.0010) [2023-12-26 23:34:02,030][105620] Updated weights for policy 1, policy_version 1126189 (0.0011) [2023-12-26 23:34:02,089][105620] Updated weights for policy 1, policy_version 1126199 (0.0008) [2023-12-26 23:34:02,300][105692] Updated weights for policy 0, policy_version 1124872 (0.0009) [2023-12-26 23:34:02,359][105692] Updated weights for policy 0, policy_version 1124882 (0.0008) [2023-12-26 23:34:02,424][105692] Updated weights for policy 0, policy_version 1124892 (0.0009) [2023-12-26 23:34:02,743][105620] Updated weights for policy 1, policy_version 1126209 (0.0010) [2023-12-26 23:34:02,800][105620] Updated weights for policy 1, policy_version 1126219 (0.0010) [2023-12-26 23:34:02,858][105620] Updated weights for policy 1, policy_version 1126229 (0.0010) [2023-12-26 23:34:03,101][105692] Updated weights for policy 0, policy_version 1124902 (0.0007) [2023-12-26 23:34:03,162][105692] Updated weights for policy 0, policy_version 1124912 (0.0006) [2023-12-26 23:34:03,215][105692] Updated weights for policy 0, policy_version 1124922 (0.0005) [2023-12-26 23:34:03,598][105620] Updated weights for policy 1, policy_version 1126239 (0.0010) [2023-12-26 23:34:03,650][105620] Updated weights for policy 1, policy_version 1126249 (0.0010) [2023-12-26 23:34:03,706][105620] Updated weights for policy 1, policy_version 1126259 (0.0009) [2023-12-26 23:34:03,799][105692] Updated weights for policy 0, policy_version 1124932 (0.0005) [2023-12-26 23:34:03,863][105692] Updated weights for policy 0, policy_version 1124942 (0.0007) [2023-12-26 23:34:03,928][105692] Updated weights for policy 0, policy_version 1124952 (0.0007) [2023-12-26 23:34:04,469][105620] Updated weights for policy 1, policy_version 1126269 (0.0008) [2023-12-26 23:34:04,526][105620] Updated weights for policy 1, policy_version 1126279 (0.0008) [2023-12-26 23:34:04,574][105620] Updated weights for policy 1, policy_version 1126289 (0.0008) [2023-12-26 23:34:04,657][105692] Updated weights for policy 0, policy_version 1124962 (0.0011) [2023-12-26 23:34:04,719][105692] Updated weights for policy 0, policy_version 1124972 (0.0009) [2023-12-26 23:34:04,775][105692] Updated weights for policy 0, policy_version 1124982 (0.0011) [2023-12-26 23:34:04,824][105692] Updated weights for policy 0, policy_version 1124992 (0.0011) [2023-12-26 23:34:05,337][105620] Updated weights for policy 1, policy_version 1126299 (0.0008) [2023-12-26 23:34:05,393][105620] Updated weights for policy 1, policy_version 1126309 (0.0008) [2023-12-26 23:34:05,441][105620] Updated weights for policy 1, policy_version 1126319 (0.0008) [2023-12-26 23:34:05,582][105692] Updated weights for policy 0, policy_version 1125002 (0.0011) [2023-12-26 23:34:05,640][105692] Updated weights for policy 0, policy_version 1125012 (0.0010) [2023-12-26 23:34:05,702][105692] Updated weights for policy 0, policy_version 1125022 (0.0010) [2023-12-26 23:34:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 576421888. Throughput: 0: 9695.2, 1: 9777.3. Samples: 576412296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:34:06,062][104569] Avg episode reward: [(0, '8352.838'), (1, '9347.519')] [2023-12-26 23:34:06,180][105620] Updated weights for policy 1, policy_version 1126329 (0.0007) [2023-12-26 23:34:06,244][105620] Updated weights for policy 1, policy_version 1126339 (0.0008) [2023-12-26 23:34:06,297][105620] Updated weights for policy 1, policy_version 1126349 (0.0008) [2023-12-26 23:34:06,359][105620] Updated weights for policy 1, policy_version 1126359 (0.0008) [2023-12-26 23:34:06,453][105692] Updated weights for policy 0, policy_version 1125032 (0.0011) [2023-12-26 23:34:06,516][105692] Updated weights for policy 0, policy_version 1125042 (0.0011) [2023-12-26 23:34:06,583][105692] Updated weights for policy 0, policy_version 1125052 (0.0011) [2023-12-26 23:34:07,124][105620] Updated weights for policy 1, policy_version 1126369 (0.0008) [2023-12-26 23:34:07,182][105620] Updated weights for policy 1, policy_version 1126379 (0.0010) [2023-12-26 23:34:07,235][105620] Updated weights for policy 1, policy_version 1126389 (0.0008) [2023-12-26 23:34:07,266][105692] Updated weights for policy 0, policy_version 1125062 (0.0008) [2023-12-26 23:34:07,329][105692] Updated weights for policy 0, policy_version 1125072 (0.0005) [2023-12-26 23:34:07,389][105692] Updated weights for policy 0, policy_version 1125082 (0.0005) [2023-12-26 23:34:07,944][105692] Updated weights for policy 0, policy_version 1125092 (0.0007) [2023-12-26 23:34:07,996][105692] Updated weights for policy 0, policy_version 1125102 (0.0005) [2023-12-26 23:34:08,050][105692] Updated weights for policy 0, policy_version 1125112 (0.0005) [2023-12-26 23:34:08,103][105620] Updated weights for policy 1, policy_version 1126399 (0.0009) [2023-12-26 23:34:08,156][105620] Updated weights for policy 1, policy_version 1126409 (0.0010) [2023-12-26 23:34:08,208][105620] Updated weights for policy 1, policy_version 1126419 (0.0009) [2023-12-26 23:34:08,677][105692] Updated weights for policy 0, policy_version 1125122 (0.0005) [2023-12-26 23:34:08,743][105692] Updated weights for policy 0, policy_version 1125132 (0.0005) [2023-12-26 23:34:08,800][105692] Updated weights for policy 0, policy_version 1125142 (0.0005) [2023-12-26 23:34:08,872][105692] Updated weights for policy 0, policy_version 1125152 (0.0010) [2023-12-26 23:34:08,918][105620] Updated weights for policy 1, policy_version 1126429 (0.0009) [2023-12-26 23:34:08,988][105620] Updated weights for policy 1, policy_version 1126439 (0.0006) [2023-12-26 23:34:09,045][105620] Updated weights for policy 1, policy_version 1126449 (0.0006) [2023-12-26 23:34:09,591][105692] Updated weights for policy 0, policy_version 1125162 (0.0006) [2023-12-26 23:34:09,656][105692] Updated weights for policy 0, policy_version 1125172 (0.0008) [2023-12-26 23:34:09,719][105620] Updated weights for policy 1, policy_version 1126459 (0.0006) [2023-12-26 23:34:09,721][105692] Updated weights for policy 0, policy_version 1125182 (0.0007) [2023-12-26 23:34:09,774][105620] Updated weights for policy 1, policy_version 1126469 (0.0008) [2023-12-26 23:34:09,833][105620] Updated weights for policy 1, policy_version 1126479 (0.0008) [2023-12-26 23:34:10,488][105692] Updated weights for policy 0, policy_version 1125192 (0.0008) [2023-12-26 23:34:10,547][105692] Updated weights for policy 0, policy_version 1125202 (0.0009) [2023-12-26 23:34:10,549][105620] Updated weights for policy 1, policy_version 1126489 (0.0008) [2023-12-26 23:34:10,606][105692] Updated weights for policy 0, policy_version 1125212 (0.0007) [2023-12-26 23:34:10,608][105620] Updated weights for policy 1, policy_version 1126499 (0.0007) [2023-12-26 23:34:10,664][105620] Updated weights for policy 1, policy_version 1126509 (0.0007) [2023-12-26 23:34:10,716][105620] Updated weights for policy 1, policy_version 1126519 (0.0009) [2023-12-26 23:34:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 576520192. Throughput: 0: 9789.9, 1: 9740.3. Samples: 576528140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:34:11,062][104569] Avg episode reward: [(0, '8812.904'), (1, '9256.391')] [2023-12-26 23:34:11,383][105620] Updated weights for policy 1, policy_version 1126529 (0.0009) [2023-12-26 23:34:11,434][105692] Updated weights for policy 0, policy_version 1125222 (0.0009) [2023-12-26 23:34:11,445][105620] Updated weights for policy 1, policy_version 1126539 (0.0008) [2023-12-26 23:34:11,488][105692] Updated weights for policy 0, policy_version 1125232 (0.0007) [2023-12-26 23:34:11,502][105620] Updated weights for policy 1, policy_version 1126549 (0.0007) [2023-12-26 23:34:11,550][105692] Updated weights for policy 0, policy_version 1125242 (0.0007) [2023-12-26 23:34:12,301][105692] Updated weights for policy 0, policy_version 1125252 (0.0008) [2023-12-26 23:34:12,307][105620] Updated weights for policy 1, policy_version 1126559 (0.0007) [2023-12-26 23:34:12,361][105692] Updated weights for policy 0, policy_version 1125262 (0.0007) [2023-12-26 23:34:12,363][105620] Updated weights for policy 1, policy_version 1126569 (0.0008) [2023-12-26 23:34:12,425][105620] Updated weights for policy 1, policy_version 1126579 (0.0008) [2023-12-26 23:34:12,426][105692] Updated weights for policy 0, policy_version 1125272 (0.0008) [2023-12-26 23:34:13,177][105620] Updated weights for policy 1, policy_version 1126589 (0.0008) [2023-12-26 23:34:13,226][105692] Updated weights for policy 0, policy_version 1125282 (0.0008) [2023-12-26 23:34:13,231][105620] Updated weights for policy 1, policy_version 1126599 (0.0006) [2023-12-26 23:34:13,277][105620] Updated weights for policy 1, policy_version 1126609 (0.0005) [2023-12-26 23:34:13,278][105692] Updated weights for policy 0, policy_version 1125292 (0.0009) [2023-12-26 23:34:13,297][105585] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000005 [2023-12-26 23:34:13,815][105620] Updated weights for policy 1, policy_version 1126619 (0.0005) [2023-12-26 23:34:13,873][105620] Updated weights for policy 1, policy_version 1126629 (0.0005) [2023-12-26 23:34:13,927][105620] Updated weights for policy 1, policy_version 1126639 (0.0005) [2023-12-26 23:34:14,237][105692] Updated weights for policy 0, policy_version 1125302 (0.0007) [2023-12-26 23:34:14,299][105692] Updated weights for policy 0, policy_version 1125312 (0.0007) [2023-12-26 23:34:14,355][105692] Updated weights for policy 0, policy_version 1125322 (0.0009) [2023-12-26 23:34:14,536][105620] Updated weights for policy 1, policy_version 1126649 (0.0005) [2023-12-26 23:34:14,597][105620] Updated weights for policy 1, policy_version 1126659 (0.0006) [2023-12-26 23:34:14,648][105620] Updated weights for policy 1, policy_version 1126669 (0.0005) [2023-12-26 23:34:14,693][105620] Updated weights for policy 1, policy_version 1126679 (0.0005) [2023-12-26 23:34:15,189][105692] Updated weights for policy 0, policy_version 1125332 (0.0009) [2023-12-26 23:34:15,252][105692] Updated weights for policy 0, policy_version 1125342 (0.0009) [2023-12-26 23:34:15,259][105620] Updated weights for policy 1, policy_version 1126689 (0.0005) [2023-12-26 23:34:15,306][105692] Updated weights for policy 0, policy_version 1125352 (0.0007) [2023-12-26 23:34:15,309][105620] Updated weights for policy 1, policy_version 1126699 (0.0007) [2023-12-26 23:34:15,363][105620] Updated weights for policy 1, policy_version 1126709 (0.0008) [2023-12-26 23:34:16,052][105620] Updated weights for policy 1, policy_version 1126719 (0.0010) [2023-12-26 23:34:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 576610304. Throughput: 0: 9788.4, 1: 9715.6. Samples: 576585404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:34:16,062][104569] Avg episode reward: [(0, '8991.287'), (1, '9258.376')] [2023-12-26 23:34:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001125360_288137216.pth... [2023-12-26 23:34:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001124256_287850496.pth [2023-12-26 23:34:16,107][105620] Updated weights for policy 1, policy_version 1126729 (0.0011) [2023-12-26 23:34:16,117][105692] Updated weights for policy 0, policy_version 1125362 (0.0006) [2023-12-26 23:34:16,155][105620] Updated weights for policy 1, policy_version 1126739 (0.0010) [2023-12-26 23:34:16,176][105692] Updated weights for policy 0, policy_version 1125372 (0.0007) [2023-12-26 23:34:16,177][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001126744_288481280.pth... [2023-12-26 23:34:16,180][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001125592_288186368.pth [2023-12-26 23:34:16,233][105692] Updated weights for policy 0, policy_version 1125382 (0.0008) [2023-12-26 23:34:16,285][105692] Updated weights for policy 0, policy_version 1125392 (0.0008) [2023-12-26 23:34:16,905][105620] Updated weights for policy 1, policy_version 1126749 (0.0010) [2023-12-26 23:34:16,968][105620] Updated weights for policy 1, policy_version 1126759 (0.0010) [2023-12-26 23:34:17,021][105620] Updated weights for policy 1, policy_version 1126769 (0.0010) [2023-12-26 23:34:17,044][105692] Updated weights for policy 0, policy_version 1125402 (0.0006) [2023-12-26 23:34:17,105][105692] Updated weights for policy 0, policy_version 1125412 (0.0006) [2023-12-26 23:34:17,163][105692] Updated weights for policy 0, policy_version 1125422 (0.0006) [2023-12-26 23:34:17,634][105620] Updated weights for policy 1, policy_version 1126779 (0.0009) [2023-12-26 23:34:17,689][105620] Updated weights for policy 1, policy_version 1126789 (0.0005) [2023-12-26 23:34:17,721][105692] Updated weights for policy 0, policy_version 1125432 (0.0008) [2023-12-26 23:34:17,754][105620] Updated weights for policy 1, policy_version 1126799 (0.0009) [2023-12-26 23:34:17,776][105692] Updated weights for policy 0, policy_version 1125442 (0.0005) [2023-12-26 23:34:17,835][105692] Updated weights for policy 0, policy_version 1125452 (0.0007) [2023-12-26 23:34:18,471][105620] Updated weights for policy 1, policy_version 1126809 (0.0011) [2023-12-26 23:34:18,521][105692] Updated weights for policy 0, policy_version 1125462 (0.0008) [2023-12-26 23:34:18,524][105620] Updated weights for policy 1, policy_version 1126819 (0.0011) [2023-12-26 23:34:18,573][105620] Updated weights for policy 1, policy_version 1126829 (0.0010) [2023-12-26 23:34:18,583][105692] Updated weights for policy 0, policy_version 1125472 (0.0007) [2023-12-26 23:34:18,628][105620] Updated weights for policy 1, policy_version 1126839 (0.0011) [2023-12-26 23:34:18,639][105692] Updated weights for policy 0, policy_version 1125482 (0.0006) [2023-12-26 23:34:19,395][105692] Updated weights for policy 0, policy_version 1125492 (0.0008) [2023-12-26 23:34:19,432][105620] Updated weights for policy 1, policy_version 1126849 (0.0009) [2023-12-26 23:34:19,450][105692] Updated weights for policy 0, policy_version 1125502 (0.0007) [2023-12-26 23:34:19,494][105620] Updated weights for policy 1, policy_version 1126859 (0.0011) [2023-12-26 23:34:19,522][105692] Updated weights for policy 0, policy_version 1125512 (0.0007) [2023-12-26 23:34:19,556][105620] Updated weights for policy 1, policy_version 1126869 (0.0011) [2023-12-26 23:34:20,148][105692] Updated weights for policy 0, policy_version 1125522 (0.0009) [2023-12-26 23:34:20,207][105692] Updated weights for policy 0, policy_version 1125532 (0.0009) [2023-12-26 23:34:20,263][105692] Updated weights for policy 0, policy_version 1125542 (0.0007) [2023-12-26 23:34:20,306][105620] Updated weights for policy 1, policy_version 1126879 (0.0009) [2023-12-26 23:34:20,321][105692] Updated weights for policy 0, policy_version 1125552 (0.0005) [2023-12-26 23:34:20,362][105620] Updated weights for policy 1, policy_version 1126889 (0.0009) [2023-12-26 23:34:20,424][105620] Updated weights for policy 1, policy_version 1126899 (0.0009) [2023-12-26 23:34:21,055][105692] Updated weights for policy 0, policy_version 1125562 (0.0009) [2023-12-26 23:34:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 576708608. Throughput: 0: 9812.7, 1: 9739.9. Samples: 576702488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:34:21,062][104569] Avg episode reward: [(0, '8714.199'), (1, '9350.831')] [2023-12-26 23:34:21,123][105692] Updated weights for policy 0, policy_version 1125572 (0.0008) [2023-12-26 23:34:21,187][105692] Updated weights for policy 0, policy_version 1125582 (0.0009) [2023-12-26 23:34:21,218][105620] Updated weights for policy 1, policy_version 1126909 (0.0008) [2023-12-26 23:34:21,287][105620] Updated weights for policy 1, policy_version 1126919 (0.0009) [2023-12-26 23:34:21,366][105620] Updated weights for policy 1, policy_version 1126929 (0.0009) [2023-12-26 23:34:22,005][105692] Updated weights for policy 0, policy_version 1125592 (0.0008) [2023-12-26 23:34:22,035][105620] Updated weights for policy 1, policy_version 1126939 (0.0007) [2023-12-26 23:34:22,058][105692] Updated weights for policy 0, policy_version 1125602 (0.0009) [2023-12-26 23:34:22,105][105620] Updated weights for policy 1, policy_version 1126949 (0.0006) [2023-12-26 23:34:22,121][105692] Updated weights for policy 0, policy_version 1125612 (0.0008) [2023-12-26 23:34:22,174][105620] Updated weights for policy 1, policy_version 1126959 (0.0007) [2023-12-26 23:34:22,779][105620] Updated weights for policy 1, policy_version 1126969 (0.0009) [2023-12-26 23:34:22,838][105620] Updated weights for policy 1, policy_version 1126979 (0.0006) [2023-12-26 23:34:22,906][105620] Updated weights for policy 1, policy_version 1126989 (0.0008) [2023-12-26 23:34:22,927][105692] Updated weights for policy 0, policy_version 1125622 (0.0007) [2023-12-26 23:34:22,965][105620] Updated weights for policy 1, policy_version 1126999 (0.0008) [2023-12-26 23:34:22,988][105692] Updated weights for policy 0, policy_version 1125632 (0.0007) [2023-12-26 23:34:23,044][105692] Updated weights for policy 0, policy_version 1125642 (0.0009) [2023-12-26 23:34:23,589][105620] Updated weights for policy 1, policy_version 1127009 (0.0007) [2023-12-26 23:34:23,651][105620] Updated weights for policy 1, policy_version 1127019 (0.0009) [2023-12-26 23:34:23,709][105620] Updated weights for policy 1, policy_version 1127029 (0.0009) [2023-12-26 23:34:23,725][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000008 [2023-12-26 23:34:23,850][105692] Updated weights for policy 0, policy_version 1125652 (0.0010) [2023-12-26 23:34:23,905][105692] Updated weights for policy 0, policy_version 1125662 (0.0009) [2023-12-26 23:34:23,959][105692] Updated weights for policy 0, policy_version 1125672 (0.0010) [2023-12-26 23:34:24,400][105620] Updated weights for policy 1, policy_version 1127039 (0.0009) [2023-12-26 23:34:24,462][105620] Updated weights for policy 1, policy_version 1127049 (0.0009) [2023-12-26 23:34:24,515][105620] Updated weights for policy 1, policy_version 1127059 (0.0008) [2023-12-26 23:34:24,750][105692] Updated weights for policy 0, policy_version 1125682 (0.0010) [2023-12-26 23:34:24,819][105692] Updated weights for policy 0, policy_version 1125692 (0.0010) [2023-12-26 23:34:24,884][105692] Updated weights for policy 0, policy_version 1125702 (0.0009) [2023-12-26 23:34:24,942][105692] Updated weights for policy 0, policy_version 1125712 (0.0009) [2023-12-26 23:34:25,223][105620] Updated weights for policy 1, policy_version 1127069 (0.0009) [2023-12-26 23:34:25,283][105620] Updated weights for policy 1, policy_version 1127079 (0.0008) [2023-12-26 23:34:25,344][105620] Updated weights for policy 1, policy_version 1127089 (0.0009) [2023-12-26 23:34:25,590][105692] Updated weights for policy 0, policy_version 1125722 (0.0008) [2023-12-26 23:34:25,652][105692] Updated weights for policy 0, policy_version 1125732 (0.0009) [2023-12-26 23:34:25,713][105692] Updated weights for policy 0, policy_version 1125742 (0.0009) [2023-12-26 23:34:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 576806912. Throughput: 0: 9761.6, 1: 9749.6. Samples: 576817400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:34:26,062][104569] Avg episode reward: [(0, '8716.409'), (1, '9350.977')] [2023-12-26 23:34:26,095][105620] Updated weights for policy 1, policy_version 1127099 (0.0009) [2023-12-26 23:34:26,147][105620] Updated weights for policy 1, policy_version 1127109 (0.0009) [2023-12-26 23:34:26,203][105620] Updated weights for policy 1, policy_version 1127119 (0.0009) [2023-12-26 23:34:26,451][105692] Updated weights for policy 0, policy_version 1125752 (0.0009) [2023-12-26 23:34:26,480][105585] KL-divergence is very high: 131.6297 [2023-12-26 23:34:26,491][105585] KL-divergence is very high: 212.0650 [2023-12-26 23:34:26,508][105692] Updated weights for policy 0, policy_version 1125762 (0.0008) [2023-12-26 23:34:26,531][105585] KL-divergence is very high: 247.3738 [2023-12-26 23:34:26,543][105585] KL-divergence is very high: 323.8917 [2023-12-26 23:34:26,572][105692] Updated weights for policy 0, policy_version 1125772 (0.0008) [2023-12-26 23:34:26,577][105585] KL-divergence is very high: 247.3177 [2023-12-26 23:34:26,591][105585] KL-divergence is very high: 283.6357 [2023-12-26 23:34:26,994][105620] Updated weights for policy 1, policy_version 1127129 (0.0009) [2023-12-26 23:34:27,045][105620] Updated weights for policy 1, policy_version 1127139 (0.0009) [2023-12-26 23:34:27,102][105620] Updated weights for policy 1, policy_version 1127149 (0.0009) [2023-12-26 23:34:27,156][105620] Updated weights for policy 1, policy_version 1127160 (0.0010) [2023-12-26 23:34:27,189][105585] KL-divergence is very high: 100.8693 [2023-12-26 23:34:27,219][105692] Updated weights for policy 0, policy_version 1125782 (0.0008) [2023-12-26 23:34:27,273][105692] Updated weights for policy 0, policy_version 1125792 (0.0009) [2023-12-26 23:34:27,331][105692] Updated weights for policy 0, policy_version 1125802 (0.0009) [2023-12-26 23:34:27,909][105620] Updated weights for policy 1, policy_version 1127170 (0.0009) [2023-12-26 23:34:27,972][105620] Updated weights for policy 1, policy_version 1127180 (0.0009) [2023-12-26 23:34:28,029][105620] Updated weights for policy 1, policy_version 1127190 (0.0008) [2023-12-26 23:34:28,088][105692] Updated weights for policy 0, policy_version 1125812 (0.0009) [2023-12-26 23:34:28,142][105692] Updated weights for policy 0, policy_version 1125822 (0.0009) [2023-12-26 23:34:28,189][105692] Updated weights for policy 0, policy_version 1125832 (0.0009) [2023-12-26 23:34:28,760][105620] Updated weights for policy 1, policy_version 1127200 (0.0010) [2023-12-26 23:34:28,825][105620] Updated weights for policy 1, policy_version 1127210 (0.0009) [2023-12-26 23:34:28,885][105620] Updated weights for policy 1, policy_version 1127220 (0.0008) [2023-12-26 23:34:28,910][105692] Updated weights for policy 0, policy_version 1125842 (0.0008) [2023-12-26 23:34:28,981][105692] Updated weights for policy 0, policy_version 1125852 (0.0010) [2023-12-26 23:34:29,042][105692] Updated weights for policy 0, policy_version 1125862 (0.0010) [2023-12-26 23:34:29,099][105692] Updated weights for policy 0, policy_version 1125872 (0.0009) [2023-12-26 23:34:29,486][105620] Updated weights for policy 1, policy_version 1127230 (0.0006) [2023-12-26 23:34:29,557][105620] Updated weights for policy 1, policy_version 1127240 (0.0005) [2023-12-26 23:34:29,627][105620] Updated weights for policy 1, policy_version 1127250 (0.0006) [2023-12-26 23:34:29,961][105692] Updated weights for policy 0, policy_version 1125882 (0.0008) [2023-12-26 23:34:30,016][105692] Updated weights for policy 0, policy_version 1125892 (0.0006) [2023-12-26 23:34:30,064][105692] Updated weights for policy 0, policy_version 1125902 (0.0005) [2023-12-26 23:34:30,344][105620] Updated weights for policy 1, policy_version 1127260 (0.0006) [2023-12-26 23:34:30,400][105620] Updated weights for policy 1, policy_version 1127270 (0.0009) [2023-12-26 23:34:30,457][105620] Updated weights for policy 1, policy_version 1127280 (0.0010) [2023-12-26 23:34:30,659][105692] Updated weights for policy 0, policy_version 1125912 (0.0008) [2023-12-26 23:34:30,707][105692] Updated weights for policy 0, policy_version 1125923 (0.0009) [2023-12-26 23:34:30,757][105692] Updated weights for policy 0, policy_version 1125933 (0.0009) [2023-12-26 23:34:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 576905216. Throughput: 0: 9704.7, 1: 9707.6. Samples: 576873500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:34:31,062][104569] Avg episode reward: [(0, '8534.062'), (1, '9258.675')] [2023-12-26 23:34:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001125936_288284672.pth... [2023-12-26 23:34:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001127288_288620544.pth... [2023-12-26 23:34:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001124832_287997952.pth [2023-12-26 23:34:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001126136_288325632.pth [2023-12-26 23:34:31,213][105620] Updated weights for policy 1, policy_version 1127290 (0.0010) [2023-12-26 23:34:31,277][105620] Updated weights for policy 1, policy_version 1127300 (0.0009) [2023-12-26 23:34:31,336][105620] Updated weights for policy 1, policy_version 1127310 (0.0010) [2023-12-26 23:34:31,404][105620] Updated weights for policy 1, policy_version 1127320 (0.0009) [2023-12-26 23:34:31,501][105692] Updated weights for policy 0, policy_version 1125943 (0.0009) [2023-12-26 23:34:31,564][105692] Updated weights for policy 0, policy_version 1125953 (0.0009) [2023-12-26 23:34:31,620][105692] Updated weights for policy 0, policy_version 1125963 (0.0008) [2023-12-26 23:34:32,124][105620] Updated weights for policy 1, policy_version 1127330 (0.0009) [2023-12-26 23:34:32,174][105620] Updated weights for policy 1, policy_version 1127340 (0.0009) [2023-12-26 23:34:32,231][105620] Updated weights for policy 1, policy_version 1127350 (0.0009) [2023-12-26 23:34:32,416][105692] Updated weights for policy 0, policy_version 1125973 (0.0009) [2023-12-26 23:34:32,474][105692] Updated weights for policy 0, policy_version 1125983 (0.0011) [2023-12-26 23:34:32,534][105692] Updated weights for policy 0, policy_version 1125993 (0.0008) [2023-12-26 23:34:32,981][105620] Updated weights for policy 1, policy_version 1127360 (0.0008) [2023-12-26 23:34:33,041][105620] Updated weights for policy 1, policy_version 1127370 (0.0009) [2023-12-26 23:34:33,099][105620] Updated weights for policy 1, policy_version 1127380 (0.0008) [2023-12-26 23:34:33,294][105692] Updated weights for policy 0, policy_version 1126003 (0.0009) [2023-12-26 23:34:33,341][105692] Updated weights for policy 0, policy_version 1126013 (0.0009) [2023-12-26 23:34:33,387][105692] Updated weights for policy 0, policy_version 1126023 (0.0008) [2023-12-26 23:34:33,827][105620] Updated weights for policy 1, policy_version 1127390 (0.0009) [2023-12-26 23:34:33,879][105620] Updated weights for policy 1, policy_version 1127400 (0.0008) [2023-12-26 23:34:33,936][105620] Updated weights for policy 1, policy_version 1127410 (0.0009) [2023-12-26 23:34:34,155][105692] Updated weights for policy 0, policy_version 1126033 (0.0008) [2023-12-26 23:34:34,220][105692] Updated weights for policy 0, policy_version 1126043 (0.0009) [2023-12-26 23:34:34,280][105692] Updated weights for policy 0, policy_version 1126053 (0.0007) [2023-12-26 23:34:34,338][105692] Updated weights for policy 0, policy_version 1126063 (0.0006) [2023-12-26 23:34:34,767][105620] Updated weights for policy 1, policy_version 1127420 (0.0009) [2023-12-26 23:34:34,833][105620] Updated weights for policy 1, policy_version 1127430 (0.0009) [2023-12-26 23:34:34,890][105620] Updated weights for policy 1, policy_version 1127440 (0.0009) [2023-12-26 23:34:34,932][105692] Updated weights for policy 0, policy_version 1126073 (0.0006) [2023-12-26 23:34:34,977][105692] Updated weights for policy 0, policy_version 1126083 (0.0005) [2023-12-26 23:34:35,023][105692] Updated weights for policy 0, policy_version 1126093 (0.0005) [2023-12-26 23:34:35,572][105692] Updated weights for policy 0, policy_version 1126103 (0.0008) [2023-12-26 23:34:35,628][105692] Updated weights for policy 0, policy_version 1126113 (0.0009) [2023-12-26 23:34:35,680][105620] Updated weights for policy 1, policy_version 1127450 (0.0009) [2023-12-26 23:34:35,681][105692] Updated weights for policy 0, policy_version 1126123 (0.0010) [2023-12-26 23:34:35,724][105620] Updated weights for policy 1, policy_version 1127460 (0.0010) [2023-12-26 23:34:35,778][105620] Updated weights for policy 1, policy_version 1127470 (0.0007) [2023-12-26 23:34:35,829][105620] Updated weights for policy 1, policy_version 1127480 (0.0010) [2023-12-26 23:34:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 577003520. Throughput: 0: 9604.1, 1: 9686.6. Samples: 576988196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:34:36,063][104569] Avg episode reward: [(0, '8811.549'), (1, '9256.652')] [2023-12-26 23:34:36,494][105692] Updated weights for policy 0, policy_version 1126133 (0.0009) [2023-12-26 23:34:36,543][105692] Updated weights for policy 0, policy_version 1126143 (0.0007) [2023-12-26 23:34:36,563][105620] Updated weights for policy 1, policy_version 1127490 (0.0007) [2023-12-26 23:34:36,595][105692] Updated weights for policy 0, policy_version 1126153 (0.0006) [2023-12-26 23:34:36,624][105620] Updated weights for policy 1, policy_version 1127500 (0.0009) [2023-12-26 23:34:36,684][105620] Updated weights for policy 1, policy_version 1127510 (0.0007) [2023-12-26 23:34:37,350][105692] Updated weights for policy 0, policy_version 1126163 (0.0008) [2023-12-26 23:34:37,403][105692] Updated weights for policy 0, policy_version 1126173 (0.0006) [2023-12-26 23:34:37,435][105620] Updated weights for policy 1, policy_version 1127520 (0.0009) [2023-12-26 23:34:37,456][105692] Updated weights for policy 0, policy_version 1126184 (0.0007) [2023-12-26 23:34:37,494][105620] Updated weights for policy 1, policy_version 1127530 (0.0010) [2023-12-26 23:34:37,556][105620] Updated weights for policy 1, policy_version 1127540 (0.0010) [2023-12-26 23:34:38,079][105692] Updated weights for policy 0, policy_version 1126194 (0.0006) [2023-12-26 23:34:38,133][105692] Updated weights for policy 0, policy_version 1126204 (0.0006) [2023-12-26 23:34:38,186][105692] Updated weights for policy 0, policy_version 1126214 (0.0007) [2023-12-26 23:34:38,243][105692] Updated weights for policy 0, policy_version 1126224 (0.0008) [2023-12-26 23:34:38,250][105620] Updated weights for policy 1, policy_version 1127550 (0.0010) [2023-12-26 23:34:38,304][105620] Updated weights for policy 1, policy_version 1127560 (0.0010) [2023-12-26 23:34:38,365][105620] Updated weights for policy 1, policy_version 1127570 (0.0010) [2023-12-26 23:34:38,881][105692] Updated weights for policy 0, policy_version 1126234 (0.0008) [2023-12-26 23:34:38,934][105692] Updated weights for policy 0, policy_version 1126244 (0.0007) [2023-12-26 23:34:38,997][105692] Updated weights for policy 0, policy_version 1126254 (0.0007) [2023-12-26 23:34:39,146][105620] Updated weights for policy 1, policy_version 1127580 (0.0007) [2023-12-26 23:34:39,200][105620] Updated weights for policy 1, policy_version 1127590 (0.0009) [2023-12-26 23:34:39,265][105620] Updated weights for policy 1, policy_version 1127600 (0.0008) [2023-12-26 23:34:39,686][105692] Updated weights for policy 0, policy_version 1126264 (0.0006) [2023-12-26 23:34:39,750][105692] Updated weights for policy 0, policy_version 1126274 (0.0006) [2023-12-26 23:34:39,813][105692] Updated weights for policy 0, policy_version 1126284 (0.0005) [2023-12-26 23:34:40,115][105620] Updated weights for policy 1, policy_version 1127610 (0.0009) [2023-12-26 23:34:40,180][105620] Updated weights for policy 1, policy_version 1127620 (0.0010) [2023-12-26 23:34:40,232][105620] Updated weights for policy 1, policy_version 1127630 (0.0009) [2023-12-26 23:34:40,291][105620] Updated weights for policy 1, policy_version 1127640 (0.0010) [2023-12-26 23:34:40,385][105692] Updated weights for policy 0, policy_version 1126294 (0.0008) [2023-12-26 23:34:40,444][105692] Updated weights for policy 0, policy_version 1126304 (0.0005) [2023-12-26 23:34:40,510][105692] Updated weights for policy 0, policy_version 1126314 (0.0006) [2023-12-26 23:34:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 577093632. Throughput: 0: 9743.1, 1: 9622.9. Samples: 577106672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:34:41,062][104569] Avg episode reward: [(0, '8808.416'), (1, '9345.816')] [2023-12-26 23:34:41,103][105620] Updated weights for policy 1, policy_version 1127650 (0.0008) [2023-12-26 23:34:41,169][105620] Updated weights for policy 1, policy_version 1127660 (0.0009) [2023-12-26 23:34:41,224][105692] Updated weights for policy 0, policy_version 1126324 (0.0009) [2023-12-26 23:34:41,232][105620] Updated weights for policy 1, policy_version 1127670 (0.0009) [2023-12-26 23:34:41,290][105692] Updated weights for policy 0, policy_version 1126334 (0.0009) [2023-12-26 23:34:41,350][105692] Updated weights for policy 0, policy_version 1126344 (0.0009) [2023-12-26 23:34:42,024][105692] Updated weights for policy 0, policy_version 1126354 (0.0009) [2023-12-26 23:34:42,038][105620] Updated weights for policy 1, policy_version 1127680 (0.0010) [2023-12-26 23:34:42,077][105692] Updated weights for policy 0, policy_version 1126364 (0.0006) [2023-12-26 23:34:42,087][105620] Updated weights for policy 1, policy_version 1127690 (0.0010) [2023-12-26 23:34:42,129][105692] Updated weights for policy 0, policy_version 1126374 (0.0006) [2023-12-26 23:34:42,143][105620] Updated weights for policy 1, policy_version 1127700 (0.0010) [2023-12-26 23:34:42,182][105692] Updated weights for policy 0, policy_version 1126384 (0.0008) [2023-12-26 23:34:42,834][105620] Updated weights for policy 1, policy_version 1127710 (0.0007) [2023-12-26 23:34:42,889][105620] Updated weights for policy 1, policy_version 1127720 (0.0006) [2023-12-26 23:34:42,963][105620] Updated weights for policy 1, policy_version 1127730 (0.0011) [2023-12-26 23:34:43,043][105692] Updated weights for policy 0, policy_version 1126394 (0.0007) [2023-12-26 23:34:43,107][105692] Updated weights for policy 0, policy_version 1126404 (0.0008) [2023-12-26 23:34:43,170][105692] Updated weights for policy 0, policy_version 1126414 (0.0008) [2023-12-26 23:34:43,616][105620] Updated weights for policy 1, policy_version 1127740 (0.0011) [2023-12-26 23:34:43,671][105620] Updated weights for policy 1, policy_version 1127750 (0.0010) [2023-12-26 23:34:43,726][105620] Updated weights for policy 1, policy_version 1127760 (0.0010) [2023-12-26 23:34:43,935][105692] Updated weights for policy 0, policy_version 1126424 (0.0008) [2023-12-26 23:34:43,979][105692] Updated weights for policy 0, policy_version 1126434 (0.0007) [2023-12-26 23:34:44,028][105692] Updated weights for policy 0, policy_version 1126444 (0.0007) [2023-12-26 23:34:44,465][105620] Updated weights for policy 1, policy_version 1127770 (0.0009) [2023-12-26 23:34:44,518][105620] Updated weights for policy 1, policy_version 1127780 (0.0007) [2023-12-26 23:34:44,581][105620] Updated weights for policy 1, policy_version 1127790 (0.0010) [2023-12-26 23:34:44,630][105620] Updated weights for policy 1, policy_version 1127800 (0.0010) [2023-12-26 23:34:44,694][105692] Updated weights for policy 0, policy_version 1126454 (0.0007) [2023-12-26 23:34:44,738][105692] Updated weights for policy 0, policy_version 1126464 (0.0008) [2023-12-26 23:34:44,805][105692] Updated weights for policy 0, policy_version 1126474 (0.0009) [2023-12-26 23:34:45,349][105620] Updated weights for policy 1, policy_version 1127810 (0.0010) [2023-12-26 23:34:45,410][105620] Updated weights for policy 1, policy_version 1127820 (0.0009) [2023-12-26 23:34:45,469][105620] Updated weights for policy 1, policy_version 1127830 (0.0009) [2023-12-26 23:34:45,490][105692] Updated weights for policy 0, policy_version 1126484 (0.0008) [2023-12-26 23:34:45,554][105692] Updated weights for policy 0, policy_version 1126494 (0.0010) [2023-12-26 23:34:45,620][105692] Updated weights for policy 0, policy_version 1126504 (0.0009) [2023-12-26 23:34:46,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19251.1, 300 sec: 19494.2). Total num frames: 577191936. Throughput: 0: 9698.0, 1: 9634.2. Samples: 577163040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:34:46,063][104569] Avg episode reward: [(0, '8805.888'), (1, '9253.432')] [2023-12-26 23:34:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001127832_288759808.pth... [2023-12-26 23:34:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001126512_288432128.pth... [2023-12-26 23:34:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001126744_288481280.pth [2023-12-26 23:34:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001125360_288137216.pth [2023-12-26 23:34:46,227][105620] Updated weights for policy 1, policy_version 1127840 (0.0006) [2023-12-26 23:34:46,273][105620] Updated weights for policy 1, policy_version 1127850 (0.0005) [2023-12-26 23:34:46,319][105620] Updated weights for policy 1, policy_version 1127860 (0.0005) [2023-12-26 23:34:46,365][105692] Updated weights for policy 0, policy_version 1126514 (0.0009) [2023-12-26 23:34:46,432][105692] Updated weights for policy 0, policy_version 1126524 (0.0009) [2023-12-26 23:34:46,494][105692] Updated weights for policy 0, policy_version 1126534 (0.0009) [2023-12-26 23:34:46,553][105692] Updated weights for policy 0, policy_version 1126544 (0.0008) [2023-12-26 23:34:46,950][105620] Updated weights for policy 1, policy_version 1127870 (0.0008) [2023-12-26 23:34:47,004][105620] Updated weights for policy 1, policy_version 1127880 (0.0009) [2023-12-26 23:34:47,056][105620] Updated weights for policy 1, policy_version 1127890 (0.0009) [2023-12-26 23:34:47,303][105692] Updated weights for policy 0, policy_version 1126554 (0.0009) [2023-12-26 23:34:47,356][105692] Updated weights for policy 0, policy_version 1126564 (0.0007) [2023-12-26 23:34:47,421][105692] Updated weights for policy 0, policy_version 1126574 (0.0005) [2023-12-26 23:34:47,871][105620] Updated weights for policy 1, policy_version 1127900 (0.0009) [2023-12-26 23:34:47,926][105620] Updated weights for policy 1, policy_version 1127910 (0.0010) [2023-12-26 23:34:47,979][105620] Updated weights for policy 1, policy_version 1127920 (0.0009) [2023-12-26 23:34:48,011][105692] Updated weights for policy 0, policy_version 1126584 (0.0007) [2023-12-26 23:34:48,060][105692] Updated weights for policy 0, policy_version 1126594 (0.0008) [2023-12-26 23:34:48,111][105692] Updated weights for policy 0, policy_version 1126604 (0.0009) [2023-12-26 23:34:48,734][105620] Updated weights for policy 1, policy_version 1127930 (0.0008) [2023-12-26 23:34:48,804][105620] Updated weights for policy 1, policy_version 1127940 (0.0006) [2023-12-26 23:34:48,809][105692] Updated weights for policy 0, policy_version 1126614 (0.0008) [2023-12-26 23:34:48,868][105620] Updated weights for policy 1, policy_version 1127950 (0.0007) [2023-12-26 23:34:48,874][105692] Updated weights for policy 0, policy_version 1126624 (0.0008) [2023-12-26 23:34:48,928][105620] Updated weights for policy 1, policy_version 1127960 (0.0005) [2023-12-26 23:34:48,934][105692] Updated weights for policy 0, policy_version 1126634 (0.0011) [2023-12-26 23:34:49,618][105692] Updated weights for policy 0, policy_version 1126644 (0.0010) [2023-12-26 23:34:49,679][105692] Updated weights for policy 0, policy_version 1126654 (0.0007) [2023-12-26 23:34:49,690][105620] Updated weights for policy 1, policy_version 1127970 (0.0007) [2023-12-26 23:34:49,738][105692] Updated weights for policy 0, policy_version 1126664 (0.0009) [2023-12-26 23:34:49,751][105620] Updated weights for policy 1, policy_version 1127980 (0.0008) [2023-12-26 23:34:49,806][105620] Updated weights for policy 1, policy_version 1127990 (0.0007) [2023-12-26 23:34:50,367][105692] Updated weights for policy 0, policy_version 1126674 (0.0007) [2023-12-26 23:34:50,434][105692] Updated weights for policy 0, policy_version 1126684 (0.0006) [2023-12-26 23:34:50,487][105692] Updated weights for policy 0, policy_version 1126694 (0.0008) [2023-12-26 23:34:50,550][105692] Updated weights for policy 0, policy_version 1126704 (0.0008) [2023-12-26 23:34:50,586][105620] Updated weights for policy 1, policy_version 1128000 (0.0009) [2023-12-26 23:34:50,642][105620] Updated weights for policy 1, policy_version 1128010 (0.0009) [2023-12-26 23:34:50,694][105620] Updated weights for policy 1, policy_version 1128020 (0.0009) [2023-12-26 23:34:51,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 577290240. Throughput: 0: 9666.2, 1: 9613.8. Samples: 577279900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:34:51,063][104569] Avg episode reward: [(0, '8715.034'), (1, '9256.722')] [2023-12-26 23:34:51,233][105692] Updated weights for policy 0, policy_version 1126714 (0.0010) [2023-12-26 23:34:51,304][105692] Updated weights for policy 0, policy_version 1126724 (0.0006) [2023-12-26 23:34:51,378][105692] Updated weights for policy 0, policy_version 1126734 (0.0010) [2023-12-26 23:34:51,482][105620] Updated weights for policy 1, policy_version 1128030 (0.0010) [2023-12-26 23:34:51,530][105620] Updated weights for policy 1, policy_version 1128040 (0.0009) [2023-12-26 23:34:51,591][105620] Updated weights for policy 1, policy_version 1128050 (0.0011) [2023-12-26 23:34:52,188][105692] Updated weights for policy 0, policy_version 1126744 (0.0009) [2023-12-26 23:34:52,252][105692] Updated weights for policy 0, policy_version 1126754 (0.0007) [2023-12-26 23:34:52,257][105620] Updated weights for policy 1, policy_version 1128060 (0.0011) [2023-12-26 23:34:52,306][105692] Updated weights for policy 0, policy_version 1126764 (0.0009) [2023-12-26 23:34:52,317][105620] Updated weights for policy 1, policy_version 1128070 (0.0011) [2023-12-26 23:34:52,377][105620] Updated weights for policy 1, policy_version 1128080 (0.0011) [2023-12-26 23:34:53,038][105620] Updated weights for policy 1, policy_version 1128090 (0.0009) [2023-12-26 23:34:53,053][105692] Updated weights for policy 0, policy_version 1126774 (0.0009) [2023-12-26 23:34:53,099][105620] Updated weights for policy 1, policy_version 1128100 (0.0006) [2023-12-26 23:34:53,118][105692] Updated weights for policy 0, policy_version 1126784 (0.0010) [2023-12-26 23:34:53,163][105620] Updated weights for policy 1, policy_version 1128110 (0.0006) [2023-12-26 23:34:53,176][105692] Updated weights for policy 0, policy_version 1126794 (0.0010) [2023-12-26 23:34:53,211][105620] Updated weights for policy 1, policy_version 1128120 (0.0009) [2023-12-26 23:34:53,839][105620] Updated weights for policy 1, policy_version 1128130 (0.0009) [2023-12-26 23:34:53,897][105692] Updated weights for policy 0, policy_version 1126804 (0.0009) [2023-12-26 23:34:53,902][105620] Updated weights for policy 1, policy_version 1128140 (0.0008) [2023-12-26 23:34:53,953][105692] Updated weights for policy 0, policy_version 1126814 (0.0006) [2023-12-26 23:34:53,966][105620] Updated weights for policy 1, policy_version 1128150 (0.0008) [2023-12-26 23:34:53,971][105585] KL-divergence is very high: 184.2641 [2023-12-26 23:34:54,008][105692] Updated weights for policy 0, policy_version 1126824 (0.0008) [2023-12-26 23:34:54,015][105585] KL-divergence is very high: 210.9249 [2023-12-26 23:34:54,620][105692] Updated weights for policy 0, policy_version 1126834 (0.0006) [2023-12-26 23:34:54,670][105692] Updated weights for policy 0, policy_version 1126844 (0.0007) [2023-12-26 23:34:54,725][105692] Updated weights for policy 0, policy_version 1126854 (0.0009) [2023-12-26 23:34:54,785][105692] Updated weights for policy 0, policy_version 1126864 (0.0007) [2023-12-26 23:34:54,797][105620] Updated weights for policy 1, policy_version 1128160 (0.0008) [2023-12-26 23:34:54,856][105620] Updated weights for policy 1, policy_version 1128171 (0.0010) [2023-12-26 23:34:54,915][105620] Updated weights for policy 1, policy_version 1128181 (0.0010) [2023-12-26 23:34:55,409][105692] Updated weights for policy 0, policy_version 1126874 (0.0005) [2023-12-26 23:34:55,466][105692] Updated weights for policy 0, policy_version 1126884 (0.0006) [2023-12-26 23:34:55,515][105692] Updated weights for policy 0, policy_version 1126894 (0.0008) [2023-12-26 23:34:55,701][105620] Updated weights for policy 1, policy_version 1128192 (0.0006) [2023-12-26 23:34:55,746][105620] Updated weights for policy 1, policy_version 1128202 (0.0005) [2023-12-26 23:34:55,772][105586] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000004 [2023-12-26 23:34:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 577388544. Throughput: 0: 9675.1, 1: 9658.6. Samples: 577398160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:34:56,063][104569] Avg episode reward: [(0, '8259.357'), (1, '9075.357')] [2023-12-26 23:34:56,206][105692] Updated weights for policy 0, policy_version 1126904 (0.0006) [2023-12-26 23:34:56,256][105692] Updated weights for policy 0, policy_version 1126914 (0.0005) [2023-12-26 23:34:56,315][105692] Updated weights for policy 0, policy_version 1126924 (0.0005) [2023-12-26 23:34:56,508][105620] Updated weights for policy 1, policy_version 1128212 (0.0006) [2023-12-26 23:34:56,556][105620] Updated weights for policy 1, policy_version 1128222 (0.0008) [2023-12-26 23:34:56,604][105620] Updated weights for policy 1, policy_version 1128232 (0.0008) [2023-12-26 23:34:56,910][105692] Updated weights for policy 0, policy_version 1126934 (0.0005) [2023-12-26 23:34:56,963][105692] Updated weights for policy 0, policy_version 1126944 (0.0005) [2023-12-26 23:34:57,016][105692] Updated weights for policy 0, policy_version 1126954 (0.0009) [2023-12-26 23:34:57,449][105620] Updated weights for policy 1, policy_version 1128242 (0.0010) [2023-12-26 23:34:57,506][105620] Updated weights for policy 1, policy_version 1128252 (0.0009) [2023-12-26 23:34:57,555][105620] Updated weights for policy 1, policy_version 1128262 (0.0009) [2023-12-26 23:34:57,603][105620] Updated weights for policy 1, policy_version 1128272 (0.0010) [2023-12-26 23:34:57,684][105692] Updated weights for policy 0, policy_version 1126964 (0.0008) [2023-12-26 23:34:57,731][105692] Updated weights for policy 0, policy_version 1126974 (0.0008) [2023-12-26 23:34:57,782][105692] Updated weights for policy 0, policy_version 1126984 (0.0007) [2023-12-26 23:34:58,272][105620] Updated weights for policy 1, policy_version 1128282 (0.0011) [2023-12-26 23:34:58,338][105620] Updated weights for policy 1, policy_version 1128292 (0.0010) [2023-12-26 23:34:58,407][105620] Updated weights for policy 1, policy_version 1128302 (0.0009) [2023-12-26 23:34:58,616][105692] Updated weights for policy 0, policy_version 1126994 (0.0008) [2023-12-26 23:34:58,680][105692] Updated weights for policy 0, policy_version 1127004 (0.0010) [2023-12-26 23:34:58,737][105692] Updated weights for policy 0, policy_version 1127014 (0.0007) [2023-12-26 23:34:58,808][105692] Updated weights for policy 0, policy_version 1127024 (0.0009) [2023-12-26 23:34:59,199][105620] Updated weights for policy 1, policy_version 1128312 (0.0008) [2023-12-26 23:34:59,269][105620] Updated weights for policy 1, policy_version 1128322 (0.0008) [2023-12-26 23:34:59,330][105620] Updated weights for policy 1, policy_version 1128332 (0.0009) [2023-12-26 23:34:59,691][105692] Updated weights for policy 0, policy_version 1127034 (0.0010) [2023-12-26 23:34:59,760][105692] Updated weights for policy 0, policy_version 1127044 (0.0010) [2023-12-26 23:34:59,818][105692] Updated weights for policy 0, policy_version 1127054 (0.0009) [2023-12-26 23:34:59,970][105620] Updated weights for policy 1, policy_version 1128342 (0.0008) [2023-12-26 23:35:00,032][105620] Updated weights for policy 1, policy_version 1128352 (0.0008) [2023-12-26 23:35:00,095][105620] Updated weights for policy 1, policy_version 1128362 (0.0008) [2023-12-26 23:35:00,612][105692] Updated weights for policy 0, policy_version 1127064 (0.0008) [2023-12-26 23:35:00,661][105692] Updated weights for policy 0, policy_version 1127075 (0.0009) [2023-12-26 23:35:00,709][105692] Updated weights for policy 0, policy_version 1127085 (0.0009) [2023-12-26 23:35:00,778][105620] Updated weights for policy 1, policy_version 1128372 (0.0007) [2023-12-26 23:35:00,831][105620] Updated weights for policy 1, policy_version 1128382 (0.0006) [2023-12-26 23:35:00,884][105620] Updated weights for policy 1, policy_version 1128392 (0.0007) [2023-12-26 23:35:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 577486848. Throughput: 0: 9757.4, 1: 9591.4. Samples: 577456104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:35:01,063][104569] Avg episode reward: [(0, '7729.607'), (1, '8982.522')] [2023-12-26 23:35:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001127088_288579584.pth... [2023-12-26 23:35:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001128400_288907264.pth... [2023-12-26 23:35:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001125936_288284672.pth [2023-12-26 23:35:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001127288_288620544.pth [2023-12-26 23:35:01,527][105620] Updated weights for policy 1, policy_version 1128402 (0.0009) [2023-12-26 23:35:01,577][105620] Updated weights for policy 1, policy_version 1128412 (0.0008) [2023-12-26 23:35:01,613][105692] Updated weights for policy 0, policy_version 1127095 (0.0009) [2023-12-26 23:35:01,639][105620] Updated weights for policy 1, policy_version 1128422 (0.0007) [2023-12-26 23:35:01,678][105692] Updated weights for policy 0, policy_version 1127105 (0.0008) [2023-12-26 23:35:01,696][105620] Updated weights for policy 1, policy_version 1128432 (0.0006) [2023-12-26 23:35:01,739][105692] Updated weights for policy 0, policy_version 1127115 (0.0009) [2023-12-26 23:35:02,284][105620] Updated weights for policy 1, policy_version 1128442 (0.0006) [2023-12-26 23:35:02,337][105620] Updated weights for policy 1, policy_version 1128452 (0.0006) [2023-12-26 23:35:02,425][105620] Updated weights for policy 1, policy_version 1128462 (0.0009) [2023-12-26 23:35:02,612][105692] Updated weights for policy 0, policy_version 1127125 (0.0010) [2023-12-26 23:35:02,661][105692] Updated weights for policy 0, policy_version 1127135 (0.0009) [2023-12-26 23:35:02,715][105692] Updated weights for policy 0, policy_version 1127145 (0.0009) [2023-12-26 23:35:03,053][105620] Updated weights for policy 1, policy_version 1128472 (0.0009) [2023-12-26 23:35:03,099][105620] Updated weights for policy 1, policy_version 1128482 (0.0008) [2023-12-26 23:35:03,152][105620] Updated weights for policy 1, policy_version 1128492 (0.0008) [2023-12-26 23:35:03,548][105692] Updated weights for policy 0, policy_version 1127155 (0.0009) [2023-12-26 23:35:03,597][105692] Updated weights for policy 0, policy_version 1127165 (0.0008) [2023-12-26 23:35:03,648][105692] Updated weights for policy 0, policy_version 1127175 (0.0008) [2023-12-26 23:35:03,829][105620] Updated weights for policy 1, policy_version 1128502 (0.0007) [2023-12-26 23:35:03,889][105620] Updated weights for policy 1, policy_version 1128512 (0.0008) [2023-12-26 23:35:03,952][105620] Updated weights for policy 1, policy_version 1128522 (0.0010) [2023-12-26 23:35:04,402][105692] Updated weights for policy 0, policy_version 1127185 (0.0009) [2023-12-26 23:35:04,472][105692] Updated weights for policy 0, policy_version 1127195 (0.0009) [2023-12-26 23:35:04,540][105692] Updated weights for policy 0, policy_version 1127205 (0.0010) [2023-12-26 23:35:04,605][105692] Updated weights for policy 0, policy_version 1127215 (0.0010) [2023-12-26 23:35:04,641][105620] Updated weights for policy 1, policy_version 1128532 (0.0009) [2023-12-26 23:35:04,696][105620] Updated weights for policy 1, policy_version 1128542 (0.0005) [2023-12-26 23:35:04,747][105620] Updated weights for policy 1, policy_version 1128552 (0.0008) [2023-12-26 23:35:05,343][105692] Updated weights for policy 0, policy_version 1127225 (0.0009) [2023-12-26 23:35:05,398][105692] Updated weights for policy 0, policy_version 1127235 (0.0009) [2023-12-26 23:35:05,453][105692] Updated weights for policy 0, policy_version 1127245 (0.0009) [2023-12-26 23:35:05,484][105620] Updated weights for policy 1, policy_version 1128562 (0.0009) [2023-12-26 23:35:05,534][105620] Updated weights for policy 1, policy_version 1128572 (0.0008) [2023-12-26 23:35:05,586][105620] Updated weights for policy 1, policy_version 1128582 (0.0008) [2023-12-26 23:35:05,635][105620] Updated weights for policy 1, policy_version 1128592 (0.0008) [2023-12-26 23:35:06,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 577576960. Throughput: 0: 9628.7, 1: 9626.4. Samples: 577568968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:35:06,062][104569] Avg episode reward: [(0, '8273.242'), (1, '9163.979')] [2023-12-26 23:35:06,172][105692] Updated weights for policy 0, policy_version 1127255 (0.0008) [2023-12-26 23:35:06,232][105692] Updated weights for policy 0, policy_version 1127265 (0.0011) [2023-12-26 23:35:06,295][105692] Updated weights for policy 0, policy_version 1127275 (0.0011) [2023-12-26 23:35:06,443][105620] Updated weights for policy 1, policy_version 1128602 (0.0008) [2023-12-26 23:35:06,503][105620] Updated weights for policy 1, policy_version 1128612 (0.0008) [2023-12-26 23:35:06,570][105620] Updated weights for policy 1, policy_version 1128622 (0.0008) [2023-12-26 23:35:06,952][105692] Updated weights for policy 0, policy_version 1127285 (0.0010) [2023-12-26 23:35:07,009][105692] Updated weights for policy 0, policy_version 1127295 (0.0007) [2023-12-26 23:35:07,063][105692] Updated weights for policy 0, policy_version 1127305 (0.0009) [2023-12-26 23:35:07,388][105620] Updated weights for policy 1, policy_version 1128632 (0.0009) [2023-12-26 23:35:07,449][105620] Updated weights for policy 1, policy_version 1128642 (0.0009) [2023-12-26 23:35:07,507][105620] Updated weights for policy 1, policy_version 1128652 (0.0009) [2023-12-26 23:35:07,779][105692] Updated weights for policy 0, policy_version 1127315 (0.0009) [2023-12-26 23:35:07,841][105692] Updated weights for policy 0, policy_version 1127325 (0.0011) [2023-12-26 23:35:07,903][105692] Updated weights for policy 0, policy_version 1127335 (0.0010) [2023-12-26 23:35:08,191][105620] Updated weights for policy 1, policy_version 1128662 (0.0007) [2023-12-26 23:35:08,252][105620] Updated weights for policy 1, policy_version 1128672 (0.0005) [2023-12-26 23:35:08,305][105620] Updated weights for policy 1, policy_version 1128682 (0.0005) [2023-12-26 23:35:08,670][105692] Updated weights for policy 0, policy_version 1127345 (0.0010) [2023-12-26 23:35:08,727][105692] Updated weights for policy 0, policy_version 1127355 (0.0011) [2023-12-26 23:35:08,779][105692] Updated weights for policy 0, policy_version 1127365 (0.0011) [2023-12-26 23:35:08,840][105692] Updated weights for policy 0, policy_version 1127375 (0.0007) [2023-12-26 23:35:08,950][105620] Updated weights for policy 1, policy_version 1128692 (0.0008) [2023-12-26 23:35:09,016][105620] Updated weights for policy 1, policy_version 1128702 (0.0010) [2023-12-26 23:35:09,082][105620] Updated weights for policy 1, policy_version 1128712 (0.0010) [2023-12-26 23:35:09,529][105692] Updated weights for policy 0, policy_version 1127385 (0.0006) [2023-12-26 23:35:09,597][105692] Updated weights for policy 0, policy_version 1127395 (0.0006) [2023-12-26 23:35:09,660][105692] Updated weights for policy 0, policy_version 1127405 (0.0009) [2023-12-26 23:35:09,800][105620] Updated weights for policy 1, policy_version 1128722 (0.0008) [2023-12-26 23:35:09,877][105620] Updated weights for policy 1, policy_version 1128732 (0.0009) [2023-12-26 23:35:09,942][105620] Updated weights for policy 1, policy_version 1128742 (0.0008) [2023-12-26 23:35:10,000][105620] Updated weights for policy 1, policy_version 1128752 (0.0010) [2023-12-26 23:35:10,294][105692] Updated weights for policy 0, policy_version 1127415 (0.0007) [2023-12-26 23:35:10,349][105692] Updated weights for policy 0, policy_version 1127425 (0.0005) [2023-12-26 23:35:10,404][105692] Updated weights for policy 0, policy_version 1127435 (0.0005) [2023-12-26 23:35:10,744][105620] Updated weights for policy 1, policy_version 1128762 (0.0010) [2023-12-26 23:35:10,805][105620] Updated weights for policy 1, policy_version 1128772 (0.0011) [2023-12-26 23:35:10,862][105620] Updated weights for policy 1, policy_version 1128782 (0.0010) [2023-12-26 23:35:11,001][105692] Updated weights for policy 0, policy_version 1127445 (0.0006) [2023-12-26 23:35:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 577675264. Throughput: 0: 9716.3, 1: 9562.7. Samples: 577684952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:35:11,063][104569] Avg episode reward: [(0, '8996.859'), (1, '8981.006')] [2023-12-26 23:35:11,063][105692] Updated weights for policy 0, policy_version 1127455 (0.0010) [2023-12-26 23:35:11,132][105692] Updated weights for policy 0, policy_version 1127465 (0.0009) [2023-12-26 23:35:11,606][105620] Updated weights for policy 1, policy_version 1128792 (0.0008) [2023-12-26 23:35:11,675][105620] Updated weights for policy 1, policy_version 1128802 (0.0009) [2023-12-26 23:35:11,743][105620] Updated weights for policy 1, policy_version 1128812 (0.0009) [2023-12-26 23:35:11,839][105692] Updated weights for policy 0, policy_version 1127475 (0.0008) [2023-12-26 23:35:11,883][105692] Updated weights for policy 0, policy_version 1127485 (0.0010) [2023-12-26 23:35:11,949][105692] Updated weights for policy 0, policy_version 1127495 (0.0010) [2023-12-26 23:35:12,470][105620] Updated weights for policy 1, policy_version 1128822 (0.0009) [2023-12-26 23:35:12,533][105620] Updated weights for policy 1, policy_version 1128832 (0.0010) [2023-12-26 23:35:12,595][105620] Updated weights for policy 1, policy_version 1128842 (0.0010) [2023-12-26 23:35:12,732][105692] Updated weights for policy 0, policy_version 1127505 (0.0010) [2023-12-26 23:35:12,797][105692] Updated weights for policy 0, policy_version 1127515 (0.0006) [2023-12-26 23:35:12,855][105692] Updated weights for policy 0, policy_version 1127526 (0.0010) [2023-12-26 23:35:12,909][105692] Updated weights for policy 0, policy_version 1127536 (0.0009) [2023-12-26 23:35:13,192][105620] Updated weights for policy 1, policy_version 1128852 (0.0008) [2023-12-26 23:35:13,250][105620] Updated weights for policy 1, policy_version 1128862 (0.0005) [2023-12-26 23:35:13,314][105620] Updated weights for policy 1, policy_version 1128872 (0.0005) [2023-12-26 23:35:13,809][105620] Updated weights for policy 1, policy_version 1128882 (0.0006) [2023-12-26 23:35:13,823][105692] Updated weights for policy 0, policy_version 1127546 (0.0009) [2023-12-26 23:35:13,873][105620] Updated weights for policy 1, policy_version 1128892 (0.0005) [2023-12-26 23:35:13,884][105692] Updated weights for policy 0, policy_version 1127556 (0.0009) [2023-12-26 23:35:13,935][105620] Updated weights for policy 1, policy_version 1128902 (0.0005) [2023-12-26 23:35:13,949][105692] Updated weights for policy 0, policy_version 1127566 (0.0010) [2023-12-26 23:35:13,989][105620] Updated weights for policy 1, policy_version 1128912 (0.0010) [2023-12-26 23:35:14,703][105620] Updated weights for policy 1, policy_version 1128922 (0.0008) [2023-12-26 23:35:14,709][105692] Updated weights for policy 0, policy_version 1127576 (0.0008) [2023-12-26 23:35:14,760][105620] Updated weights for policy 1, policy_version 1128932 (0.0006) [2023-12-26 23:35:14,763][105692] Updated weights for policy 0, policy_version 1127586 (0.0007) [2023-12-26 23:35:14,826][105692] Updated weights for policy 0, policy_version 1127596 (0.0009) [2023-12-26 23:35:14,837][105620] Updated weights for policy 1, policy_version 1128942 (0.0007) [2023-12-26 23:35:15,466][105620] Updated weights for policy 1, policy_version 1128952 (0.0008) [2023-12-26 23:35:15,520][105620] Updated weights for policy 1, policy_version 1128962 (0.0009) [2023-12-26 23:35:15,571][105620] Updated weights for policy 1, policy_version 1128972 (0.0009) [2023-12-26 23:35:15,624][105692] Updated weights for policy 0, policy_version 1127606 (0.0009) [2023-12-26 23:35:15,688][105692] Updated weights for policy 0, policy_version 1127616 (0.0009) [2023-12-26 23:35:15,754][105692] Updated weights for policy 0, policy_version 1127626 (0.0009) [2023-12-26 23:35:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 577773568. Throughput: 0: 9697.7, 1: 9671.7. Samples: 577745128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:35:16,062][104569] Avg episode reward: [(0, '8912.257'), (1, '8890.335')] [2023-12-26 23:35:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001127632_288718848.pth... [2023-12-26 23:35:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001128976_289054720.pth... [2023-12-26 23:35:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001126512_288432128.pth [2023-12-26 23:35:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001127832_288759808.pth [2023-12-26 23:35:16,298][105620] Updated weights for policy 1, policy_version 1128982 (0.0010) [2023-12-26 23:35:16,357][105620] Updated weights for policy 1, policy_version 1128992 (0.0010) [2023-12-26 23:35:16,418][105620] Updated weights for policy 1, policy_version 1129002 (0.0010) [2023-12-26 23:35:16,521][105692] Updated weights for policy 0, policy_version 1127636 (0.0009) [2023-12-26 23:35:16,577][105692] Updated weights for policy 0, policy_version 1127646 (0.0008) [2023-12-26 23:35:16,635][105692] Updated weights for policy 0, policy_version 1127656 (0.0008) [2023-12-26 23:35:17,156][105620] Updated weights for policy 1, policy_version 1129012 (0.0008) [2023-12-26 23:35:17,211][105620] Updated weights for policy 1, policy_version 1129022 (0.0005) [2023-12-26 23:35:17,268][105620] Updated weights for policy 1, policy_version 1129032 (0.0005) [2023-12-26 23:35:17,447][105692] Updated weights for policy 0, policy_version 1127666 (0.0008) [2023-12-26 23:35:17,505][105692] Updated weights for policy 0, policy_version 1127676 (0.0008) [2023-12-26 23:35:17,567][105692] Updated weights for policy 0, policy_version 1127686 (0.0009) [2023-12-26 23:35:17,618][105692] Updated weights for policy 0, policy_version 1127696 (0.0008) [2023-12-26 23:35:17,897][105620] Updated weights for policy 1, policy_version 1129042 (0.0006) [2023-12-26 23:35:17,959][105620] Updated weights for policy 1, policy_version 1129052 (0.0010) [2023-12-26 23:35:18,017][105620] Updated weights for policy 1, policy_version 1129062 (0.0010) [2023-12-26 23:35:18,084][105620] Updated weights for policy 1, policy_version 1129072 (0.0011) [2023-12-26 23:35:18,393][105692] Updated weights for policy 0, policy_version 1127706 (0.0008) [2023-12-26 23:35:18,445][105692] Updated weights for policy 0, policy_version 1127716 (0.0008) [2023-12-26 23:35:18,493][105692] Updated weights for policy 0, policy_version 1127726 (0.0007) [2023-12-26 23:35:18,833][105620] Updated weights for policy 1, policy_version 1129082 (0.0011) [2023-12-26 23:35:18,898][105620] Updated weights for policy 1, policy_version 1129092 (0.0010) [2023-12-26 23:35:18,960][105620] Updated weights for policy 1, policy_version 1129102 (0.0010) [2023-12-26 23:35:19,292][105692] Updated weights for policy 0, policy_version 1127736 (0.0007) [2023-12-26 23:35:19,352][105692] Updated weights for policy 0, policy_version 1127746 (0.0007) [2023-12-26 23:35:19,418][105692] Updated weights for policy 0, policy_version 1127756 (0.0008) [2023-12-26 23:35:19,676][105620] Updated weights for policy 1, policy_version 1129112 (0.0006) [2023-12-26 23:35:19,746][105620] Updated weights for policy 1, policy_version 1129122 (0.0007) [2023-12-26 23:35:19,813][105620] Updated weights for policy 1, policy_version 1129132 (0.0010) [2023-12-26 23:35:20,116][105692] Updated weights for policy 0, policy_version 1127766 (0.0008) [2023-12-26 23:35:20,180][105692] Updated weights for policy 0, policy_version 1127776 (0.0009) [2023-12-26 23:35:20,245][105692] Updated weights for policy 0, policy_version 1127786 (0.0010) [2023-12-26 23:35:20,445][105620] Updated weights for policy 1, policy_version 1129142 (0.0010) [2023-12-26 23:35:20,512][105620] Updated weights for policy 1, policy_version 1129152 (0.0010) [2023-12-26 23:35:20,574][105620] Updated weights for policy 1, policy_version 1129162 (0.0011) [2023-12-26 23:35:21,042][105692] Updated weights for policy 0, policy_version 1127796 (0.0009) [2023-12-26 23:35:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19438.7). Total num frames: 577863680. Throughput: 0: 9614.3, 1: 9708.7. Samples: 577857732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:35:21,062][104569] Avg episode reward: [(0, '8904.613'), (1, '8983.075')] [2023-12-26 23:35:21,104][105692] Updated weights for policy 0, policy_version 1127806 (0.0009) [2023-12-26 23:35:21,170][105692] Updated weights for policy 0, policy_version 1127816 (0.0009) [2023-12-26 23:35:21,243][105620] Updated weights for policy 1, policy_version 1129172 (0.0009) [2023-12-26 23:35:21,302][105620] Updated weights for policy 1, policy_version 1129182 (0.0010) [2023-12-26 23:35:21,371][105620] Updated weights for policy 1, policy_version 1129192 (0.0010) [2023-12-26 23:35:21,965][105692] Updated weights for policy 0, policy_version 1127826 (0.0010) [2023-12-26 23:35:22,028][105692] Updated weights for policy 0, policy_version 1127836 (0.0008) [2023-12-26 23:35:22,051][105620] Updated weights for policy 1, policy_version 1129202 (0.0008) [2023-12-26 23:35:22,090][105692] Updated weights for policy 0, policy_version 1127846 (0.0007) [2023-12-26 23:35:22,105][105620] Updated weights for policy 1, policy_version 1129212 (0.0006) [2023-12-26 23:35:22,147][105692] Updated weights for policy 0, policy_version 1127856 (0.0008) [2023-12-26 23:35:22,163][105620] Updated weights for policy 1, policy_version 1129222 (0.0006) [2023-12-26 23:35:22,222][105620] Updated weights for policy 1, policy_version 1129232 (0.0008) [2023-12-26 23:35:22,888][105692] Updated weights for policy 0, policy_version 1127866 (0.0010) [2023-12-26 23:35:22,949][105692] Updated weights for policy 0, policy_version 1127876 (0.0009) [2023-12-26 23:35:22,963][105620] Updated weights for policy 1, policy_version 1129242 (0.0006) [2023-12-26 23:35:23,008][105692] Updated weights for policy 0, policy_version 1127886 (0.0008) [2023-12-26 23:35:23,021][105620] Updated weights for policy 1, policy_version 1129252 (0.0007) [2023-12-26 23:35:23,077][105620] Updated weights for policy 1, policy_version 1129262 (0.0008) [2023-12-26 23:35:23,793][105692] Updated weights for policy 0, policy_version 1127896 (0.0010) [2023-12-26 23:35:23,798][105620] Updated weights for policy 1, policy_version 1129272 (0.0006) [2023-12-26 23:35:23,850][105620] Updated weights for policy 1, policy_version 1129282 (0.0005) [2023-12-26 23:35:23,849][105692] Updated weights for policy 0, policy_version 1127906 (0.0011) [2023-12-26 23:35:23,904][105620] Updated weights for policy 1, policy_version 1129292 (0.0006) [2023-12-26 23:35:23,906][105692] Updated weights for policy 0, policy_version 1127916 (0.0011) [2023-12-26 23:35:24,594][105620] Updated weights for policy 1, policy_version 1129302 (0.0007) [2023-12-26 23:35:24,647][105692] Updated weights for policy 0, policy_version 1127926 (0.0011) [2023-12-26 23:35:24,654][105620] Updated weights for policy 1, policy_version 1129312 (0.0007) [2023-12-26 23:35:24,706][105692] Updated weights for policy 0, policy_version 1127936 (0.0010) [2023-12-26 23:35:24,708][105620] Updated weights for policy 1, policy_version 1129322 (0.0005) [2023-12-26 23:35:24,762][105692] Updated weights for policy 0, policy_version 1127946 (0.0010) [2023-12-26 23:35:25,367][105692] Updated weights for policy 0, policy_version 1127956 (0.0008) [2023-12-26 23:35:25,429][105692] Updated weights for policy 0, policy_version 1127966 (0.0009) [2023-12-26 23:35:25,492][105692] Updated weights for policy 0, policy_version 1127976 (0.0005) [2023-12-26 23:35:25,508][105620] Updated weights for policy 1, policy_version 1129332 (0.0007) [2023-12-26 23:35:25,567][105620] Updated weights for policy 1, policy_version 1129342 (0.0010) [2023-12-26 23:35:25,624][105620] Updated weights for policy 1, policy_version 1129352 (0.0009) [2023-12-26 23:35:26,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.1, 300 sec: 19410.9). Total num frames: 577961984. Throughput: 0: 9459.4, 1: 9755.7. Samples: 577971356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:35:26,063][104569] Avg episode reward: [(0, '8993.395'), (1, '9167.001')] [2023-12-26 23:35:26,103][105692] Updated weights for policy 0, policy_version 1127986 (0.0006) [2023-12-26 23:35:26,154][105692] Updated weights for policy 0, policy_version 1127996 (0.0009) [2023-12-26 23:35:26,206][105692] Updated weights for policy 0, policy_version 1128006 (0.0009) [2023-12-26 23:35:26,258][105692] Updated weights for policy 0, policy_version 1128016 (0.0009) [2023-12-26 23:35:26,409][105620] Updated weights for policy 1, policy_version 1129362 (0.0010) [2023-12-26 23:35:26,456][105620] Updated weights for policy 1, policy_version 1129372 (0.0009) [2023-12-26 23:35:26,511][105620] Updated weights for policy 1, policy_version 1129382 (0.0009) [2023-12-26 23:35:26,571][105620] Updated weights for policy 1, policy_version 1129392 (0.0008) [2023-12-26 23:35:27,029][105692] Updated weights for policy 0, policy_version 1128026 (0.0010) [2023-12-26 23:35:27,082][105692] Updated weights for policy 0, policy_version 1128036 (0.0010) [2023-12-26 23:35:27,139][105692] Updated weights for policy 0, policy_version 1128046 (0.0010) [2023-12-26 23:35:27,207][105620] Updated weights for policy 1, policy_version 1129402 (0.0006) [2023-12-26 23:35:27,255][105620] Updated weights for policy 1, policy_version 1129412 (0.0005) [2023-12-26 23:35:27,306][105620] Updated weights for policy 1, policy_version 1129422 (0.0006) [2023-12-26 23:35:27,849][105692] Updated weights for policy 0, policy_version 1128057 (0.0010) [2023-12-26 23:35:27,902][105692] Updated weights for policy 0, policy_version 1128068 (0.0010) [2023-12-26 23:35:27,952][105692] Updated weights for policy 0, policy_version 1128078 (0.0009) [2023-12-26 23:35:27,961][105620] Updated weights for policy 1, policy_version 1129432 (0.0006) [2023-12-26 23:35:28,017][105620] Updated weights for policy 1, policy_version 1129442 (0.0005) [2023-12-26 23:35:28,069][105620] Updated weights for policy 1, policy_version 1129452 (0.0005) [2023-12-26 23:35:28,663][105620] Updated weights for policy 1, policy_version 1129462 (0.0008) [2023-12-26 23:35:28,680][105692] Updated weights for policy 0, policy_version 1128088 (0.0007) [2023-12-26 23:35:28,724][105620] Updated weights for policy 1, policy_version 1129472 (0.0008) [2023-12-26 23:35:28,743][105692] Updated weights for policy 0, policy_version 1128098 (0.0006) [2023-12-26 23:35:28,786][105620] Updated weights for policy 1, policy_version 1129482 (0.0009) [2023-12-26 23:35:28,807][105692] Updated weights for policy 0, policy_version 1128108 (0.0006) [2023-12-26 23:35:29,366][105692] Updated weights for policy 0, policy_version 1128118 (0.0008) [2023-12-26 23:35:29,410][105620] Updated weights for policy 1, policy_version 1129492 (0.0009) [2023-12-26 23:35:29,433][105692] Updated weights for policy 0, policy_version 1128128 (0.0007) [2023-12-26 23:35:29,458][105620] Updated weights for policy 1, policy_version 1129502 (0.0008) [2023-12-26 23:35:29,495][105692] Updated weights for policy 0, policy_version 1128138 (0.0005) [2023-12-26 23:35:29,522][105620] Updated weights for policy 1, policy_version 1129512 (0.0007) [2023-12-26 23:35:30,164][105692] Updated weights for policy 0, policy_version 1128148 (0.0010) [2023-12-26 23:35:30,178][105620] Updated weights for policy 1, policy_version 1129522 (0.0005) [2023-12-26 23:35:30,229][105692] Updated weights for policy 0, policy_version 1128158 (0.0009) [2023-12-26 23:35:30,240][105620] Updated weights for policy 1, policy_version 1129532 (0.0005) [2023-12-26 23:35:30,286][105692] Updated weights for policy 0, policy_version 1128168 (0.0009) [2023-12-26 23:35:30,308][105620] Updated weights for policy 1, policy_version 1129542 (0.0005) [2023-12-26 23:35:30,369][105620] Updated weights for policy 1, policy_version 1129552 (0.0009) [2023-12-26 23:35:30,907][105692] Updated weights for policy 0, policy_version 1128178 (0.0008) [2023-12-26 23:35:30,961][105692] Updated weights for policy 0, policy_version 1128188 (0.0005) [2023-12-26 23:35:31,017][105692] Updated weights for policy 0, policy_version 1128198 (0.0008) [2023-12-26 23:35:31,029][105620] Updated weights for policy 1, policy_version 1129562 (0.0006) [2023-12-26 23:35:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 578060288. Throughput: 0: 9511.3, 1: 9829.8. Samples: 578033380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:35:31,062][104569] Avg episode reward: [(0, '9086.863'), (1, '9167.354')] [2023-12-26 23:35:31,075][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001128208_288866304.pth... [2023-12-26 23:35:31,076][105692] Updated weights for policy 0, policy_version 1128208 (0.0011) [2023-12-26 23:35:31,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001127088_288579584.pth [2023-12-26 23:35:31,094][105620] Updated weights for policy 1, policy_version 1129572 (0.0009) [2023-12-26 23:35:31,167][105620] Updated weights for policy 1, policy_version 1129582 (0.0012) [2023-12-26 23:35:31,176][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001129584_289210368.pth... [2023-12-26 23:35:31,179][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001128400_288907264.pth [2023-12-26 23:35:31,814][105692] Updated weights for policy 0, policy_version 1128218 (0.0011) [2023-12-26 23:35:31,876][105692] Updated weights for policy 0, policy_version 1128228 (0.0011) [2023-12-26 23:35:31,886][105620] Updated weights for policy 1, policy_version 1129592 (0.0011) [2023-12-26 23:35:31,935][105692] Updated weights for policy 0, policy_version 1128238 (0.0010) [2023-12-26 23:35:31,945][105620] Updated weights for policy 1, policy_version 1129602 (0.0010) [2023-12-26 23:35:32,003][105620] Updated weights for policy 1, policy_version 1129612 (0.0010) [2023-12-26 23:35:32,675][105692] Updated weights for policy 0, policy_version 1128248 (0.0006) [2023-12-26 23:35:32,728][105692] Updated weights for policy 0, policy_version 1128258 (0.0006) [2023-12-26 23:35:32,759][105620] Updated weights for policy 1, policy_version 1129622 (0.0010) [2023-12-26 23:35:32,775][105692] Updated weights for policy 0, policy_version 1128268 (0.0005) [2023-12-26 23:35:32,811][105620] Updated weights for policy 1, policy_version 1129632 (0.0010) [2023-12-26 23:35:32,860][105620] Updated weights for policy 1, policy_version 1129642 (0.0010) [2023-12-26 23:35:33,321][105692] Updated weights for policy 0, policy_version 1128278 (0.0005) [2023-12-26 23:35:33,385][105692] Updated weights for policy 0, policy_version 1128288 (0.0005) [2023-12-26 23:35:33,414][105585] KL-divergence is very high: 153.4561 [2023-12-26 23:35:33,450][105692] Updated weights for policy 0, policy_version 1128298 (0.0005) [2023-12-26 23:35:33,466][105585] KL-divergence is very high: 155.8832 [2023-12-26 23:35:33,629][105620] Updated weights for policy 1, policy_version 1129652 (0.0010) [2023-12-26 23:35:33,676][105620] Updated weights for policy 1, policy_version 1129662 (0.0010) [2023-12-26 23:35:33,727][105620] Updated weights for policy 1, policy_version 1129672 (0.0010) [2023-12-26 23:35:33,966][105692] Updated weights for policy 0, policy_version 1128308 (0.0005) [2023-12-26 23:35:34,027][105692] Updated weights for policy 0, policy_version 1128318 (0.0005) [2023-12-26 23:35:34,087][105692] Updated weights for policy 0, policy_version 1128328 (0.0005) [2023-12-26 23:35:34,473][105620] Updated weights for policy 1, policy_version 1129682 (0.0010) [2023-12-26 23:35:34,522][105620] Updated weights for policy 1, policy_version 1129692 (0.0010) [2023-12-26 23:35:34,585][105620] Updated weights for policy 1, policy_version 1129702 (0.0010) [2023-12-26 23:35:34,644][105620] Updated weights for policy 1, policy_version 1129712 (0.0010) [2023-12-26 23:35:34,758][105692] Updated weights for policy 0, policy_version 1128338 (0.0006) [2023-12-26 23:35:34,821][105692] Updated weights for policy 0, policy_version 1128348 (0.0011) [2023-12-26 23:35:34,879][105692] Updated weights for policy 0, policy_version 1128358 (0.0010) [2023-12-26 23:35:34,941][105692] Updated weights for policy 0, policy_version 1128368 (0.0010) [2023-12-26 23:35:35,356][105620] Updated weights for policy 1, policy_version 1129722 (0.0011) [2023-12-26 23:35:35,408][105620] Updated weights for policy 1, policy_version 1129732 (0.0010) [2023-12-26 23:35:35,454][105620] Updated weights for policy 1, policy_version 1129742 (0.0010) [2023-12-26 23:35:35,602][105692] Updated weights for policy 0, policy_version 1128378 (0.0005) [2023-12-26 23:35:35,655][105692] Updated weights for policy 0, policy_version 1128388 (0.0007) [2023-12-26 23:35:35,711][105692] Updated weights for policy 0, policy_version 1128398 (0.0009) [2023-12-26 23:35:36,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 578166784. Throughput: 0: 9606.2, 1: 9883.9. Samples: 578156952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:35:36,063][104569] Avg episode reward: [(0, '8818.762'), (1, '9168.926')] [2023-12-26 23:35:36,242][105620] Updated weights for policy 1, policy_version 1129752 (0.0008) [2023-12-26 23:35:36,300][105620] Updated weights for policy 1, policy_version 1129762 (0.0009) [2023-12-26 23:35:36,352][105620] Updated weights for policy 1, policy_version 1129772 (0.0008) [2023-12-26 23:35:36,357][105692] Updated weights for policy 0, policy_version 1128408 (0.0008) [2023-12-26 23:35:36,412][105692] Updated weights for policy 0, policy_version 1128418 (0.0008) [2023-12-26 23:35:36,460][105692] Updated weights for policy 0, policy_version 1128428 (0.0009) [2023-12-26 23:35:37,110][105620] Updated weights for policy 1, policy_version 1129782 (0.0008) [2023-12-26 23:35:37,168][105620] Updated weights for policy 1, policy_version 1129792 (0.0009) [2023-12-26 23:35:37,221][105692] Updated weights for policy 0, policy_version 1128438 (0.0007) [2023-12-26 23:35:37,224][105620] Updated weights for policy 1, policy_version 1129802 (0.0009) [2023-12-26 23:35:37,276][105692] Updated weights for policy 0, policy_version 1128448 (0.0007) [2023-12-26 23:35:37,324][105692] Updated weights for policy 0, policy_version 1128458 (0.0009) [2023-12-26 23:35:37,984][105620] Updated weights for policy 1, policy_version 1129812 (0.0007) [2023-12-26 23:35:38,044][105620] Updated weights for policy 1, policy_version 1129822 (0.0008) [2023-12-26 23:35:38,094][105692] Updated weights for policy 0, policy_version 1128468 (0.0007) [2023-12-26 23:35:38,098][105620] Updated weights for policy 1, policy_version 1129832 (0.0006) [2023-12-26 23:35:38,185][105692] Updated weights for policy 0, policy_version 1128478 (0.0009) [2023-12-26 23:35:38,249][105692] Updated weights for policy 0, policy_version 1128488 (0.0009) [2023-12-26 23:35:38,749][105620] Updated weights for policy 1, policy_version 1129842 (0.0007) [2023-12-26 23:35:38,822][105620] Updated weights for policy 1, policy_version 1129852 (0.0008) [2023-12-26 23:35:38,898][105620] Updated weights for policy 1, policy_version 1129862 (0.0009) [2023-12-26 23:35:38,955][105620] Updated weights for policy 1, policy_version 1129872 (0.0008) [2023-12-26 23:35:38,974][105692] Updated weights for policy 0, policy_version 1128498 (0.0009) [2023-12-26 23:35:39,026][105692] Updated weights for policy 0, policy_version 1128508 (0.0009) [2023-12-26 23:35:39,087][105692] Updated weights for policy 0, policy_version 1128518 (0.0009) [2023-12-26 23:35:39,153][105692] Updated weights for policy 0, policy_version 1128528 (0.0008) [2023-12-26 23:35:39,647][105620] Updated weights for policy 1, policy_version 1129882 (0.0010) [2023-12-26 23:35:39,696][105620] Updated weights for policy 1, policy_version 1129892 (0.0010) [2023-12-26 23:35:39,750][105620] Updated weights for policy 1, policy_version 1129902 (0.0011) [2023-12-26 23:35:39,869][105692] Updated weights for policy 0, policy_version 1128538 (0.0009) [2023-12-26 23:35:39,935][105692] Updated weights for policy 0, policy_version 1128548 (0.0007) [2023-12-26 23:35:39,996][105692] Updated weights for policy 0, policy_version 1128558 (0.0006) [2023-12-26 23:35:40,523][105620] Updated weights for policy 1, policy_version 1129912 (0.0008) [2023-12-26 23:35:40,586][105620] Updated weights for policy 1, policy_version 1129922 (0.0008) [2023-12-26 23:35:40,635][105620] Updated weights for policy 1, policy_version 1129932 (0.0008) [2023-12-26 23:35:40,752][105692] Updated weights for policy 0, policy_version 1128568 (0.0010) [2023-12-26 23:35:40,813][105692] Updated weights for policy 0, policy_version 1128578 (0.0010) [2023-12-26 23:35:40,866][105692] Updated weights for policy 0, policy_version 1128588 (0.0010) [2023-12-26 23:35:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 578265088. Throughput: 0: 9556.5, 1: 9843.8. Samples: 578271168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:35:41,062][104569] Avg episode reward: [(0, '8827.148'), (1, '9077.973')] [2023-12-26 23:35:41,472][105620] Updated weights for policy 1, policy_version 1129942 (0.0008) [2023-12-26 23:35:41,536][105620] Updated weights for policy 1, policy_version 1129952 (0.0006) [2023-12-26 23:35:41,600][105620] Updated weights for policy 1, policy_version 1129962 (0.0008) [2023-12-26 23:35:41,661][105692] Updated weights for policy 0, policy_version 1128598 (0.0010) [2023-12-26 23:35:41,720][105692] Updated weights for policy 0, policy_version 1128608 (0.0006) [2023-12-26 23:35:41,789][105692] Updated weights for policy 0, policy_version 1128618 (0.0006) [2023-12-26 23:35:42,250][105620] Updated weights for policy 1, policy_version 1129972 (0.0008) [2023-12-26 23:35:42,321][105620] Updated weights for policy 1, policy_version 1129982 (0.0009) [2023-12-26 23:35:42,383][105620] Updated weights for policy 1, policy_version 1129992 (0.0008) [2023-12-26 23:35:42,463][105692] Updated weights for policy 0, policy_version 1128628 (0.0008) [2023-12-26 23:35:42,518][105692] Updated weights for policy 0, policy_version 1128638 (0.0005) [2023-12-26 23:35:42,564][105692] Updated weights for policy 0, policy_version 1128648 (0.0005) [2023-12-26 23:35:43,035][105620] Updated weights for policy 1, policy_version 1130002 (0.0008) [2023-12-26 23:35:43,087][105620] Updated weights for policy 1, policy_version 1130012 (0.0008) [2023-12-26 23:35:43,144][105620] Updated weights for policy 1, policy_version 1130022 (0.0006) [2023-12-26 23:35:43,202][105620] Updated weights for policy 1, policy_version 1130032 (0.0006) [2023-12-26 23:35:43,283][105692] Updated weights for policy 0, policy_version 1128658 (0.0008) [2023-12-26 23:35:43,347][105692] Updated weights for policy 0, policy_version 1128668 (0.0010) [2023-12-26 23:35:43,405][105692] Updated weights for policy 0, policy_version 1128678 (0.0010) [2023-12-26 23:35:43,464][105692] Updated weights for policy 0, policy_version 1128688 (0.0010) [2023-12-26 23:35:43,845][105620] Updated weights for policy 1, policy_version 1130042 (0.0008) [2023-12-26 23:35:43,913][105620] Updated weights for policy 1, policy_version 1130052 (0.0009) [2023-12-26 23:35:43,978][105620] Updated weights for policy 1, policy_version 1130062 (0.0009) [2023-12-26 23:35:44,189][105692] Updated weights for policy 0, policy_version 1128698 (0.0009) [2023-12-26 23:35:44,255][105692] Updated weights for policy 0, policy_version 1128708 (0.0009) [2023-12-26 23:35:44,306][105692] Updated weights for policy 0, policy_version 1128718 (0.0008) [2023-12-26 23:35:44,570][105620] Updated weights for policy 1, policy_version 1130072 (0.0006) [2023-12-26 23:35:44,626][105620] Updated weights for policy 1, policy_version 1130082 (0.0005) [2023-12-26 23:35:44,684][105620] Updated weights for policy 1, policy_version 1130092 (0.0005) [2023-12-26 23:35:45,056][105692] Updated weights for policy 0, policy_version 1128728 (0.0005) [2023-12-26 23:35:45,122][105692] Updated weights for policy 0, policy_version 1128738 (0.0005) [2023-12-26 23:35:45,185][105692] Updated weights for policy 0, policy_version 1128748 (0.0005) [2023-12-26 23:35:45,390][105620] Updated weights for policy 1, policy_version 1130102 (0.0009) [2023-12-26 23:35:45,453][105620] Updated weights for policy 1, policy_version 1130112 (0.0010) [2023-12-26 23:35:45,520][105620] Updated weights for policy 1, policy_version 1130122 (0.0011) [2023-12-26 23:35:45,776][105692] Updated weights for policy 0, policy_version 1128758 (0.0007) [2023-12-26 23:35:45,836][105692] Updated weights for policy 0, policy_version 1128768 (0.0007) [2023-12-26 23:35:45,892][105692] Updated weights for policy 0, policy_version 1128779 (0.0006) [2023-12-26 23:35:46,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 578363392. Throughput: 0: 9522.2, 1: 9902.1. Samples: 578330200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:35:46,063][104569] Avg episode reward: [(0, '9086.499'), (1, '9167.452')] [2023-12-26 23:35:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001128784_289013760.pth... [2023-12-26 23:35:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001130128_289349632.pth... [2023-12-26 23:35:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001127632_288718848.pth [2023-12-26 23:35:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001128976_289054720.pth [2023-12-26 23:35:46,277][105620] Updated weights for policy 1, policy_version 1130132 (0.0010) [2023-12-26 23:35:46,331][105620] Updated weights for policy 1, policy_version 1130142 (0.0010) [2023-12-26 23:35:46,383][105620] Updated weights for policy 1, policy_version 1130152 (0.0010) [2023-12-26 23:35:46,534][105692] Updated weights for policy 0, policy_version 1128789 (0.0007) [2023-12-26 23:35:46,586][105692] Updated weights for policy 0, policy_version 1128799 (0.0009) [2023-12-26 23:35:46,636][105692] Updated weights for policy 0, policy_version 1128810 (0.0010) [2023-12-26 23:35:47,014][105620] Updated weights for policy 1, policy_version 1130162 (0.0010) [2023-12-26 23:35:47,060][105620] Updated weights for policy 1, policy_version 1130172 (0.0005) [2023-12-26 23:35:47,112][105620] Updated weights for policy 1, policy_version 1130182 (0.0005) [2023-12-26 23:35:47,165][105620] Updated weights for policy 1, policy_version 1130192 (0.0005) [2023-12-26 23:35:47,274][105692] Updated weights for policy 0, policy_version 1128820 (0.0008) [2023-12-26 23:35:47,340][105692] Updated weights for policy 0, policy_version 1128830 (0.0005) [2023-12-26 23:35:47,408][105692] Updated weights for policy 0, policy_version 1128840 (0.0006) [2023-12-26 23:35:47,717][105620] Updated weights for policy 1, policy_version 1130202 (0.0006) [2023-12-26 23:35:47,763][105620] Updated weights for policy 1, policy_version 1130212 (0.0006) [2023-12-26 23:35:47,810][105620] Updated weights for policy 1, policy_version 1130222 (0.0005) [2023-12-26 23:35:47,913][105692] Updated weights for policy 0, policy_version 1128850 (0.0005) [2023-12-26 23:35:47,966][105692] Updated weights for policy 0, policy_version 1128860 (0.0006) [2023-12-26 23:35:48,014][105692] Updated weights for policy 0, policy_version 1128870 (0.0010) [2023-12-26 23:35:48,058][105692] Updated weights for policy 0, policy_version 1128880 (0.0010) [2023-12-26 23:35:48,522][105620] Updated weights for policy 1, policy_version 1130232 (0.0010) [2023-12-26 23:35:48,590][105620] Updated weights for policy 1, policy_version 1130242 (0.0011) [2023-12-26 23:35:48,653][105620] Updated weights for policy 1, policy_version 1130252 (0.0011) [2023-12-26 23:35:48,768][105692] Updated weights for policy 0, policy_version 1128890 (0.0008) [2023-12-26 23:35:48,830][105692] Updated weights for policy 0, policy_version 1128900 (0.0009) [2023-12-26 23:35:48,889][105692] Updated weights for policy 0, policy_version 1128910 (0.0009) [2023-12-26 23:35:49,266][105620] Updated weights for policy 1, policy_version 1130262 (0.0008) [2023-12-26 23:35:49,329][105620] Updated weights for policy 1, policy_version 1130272 (0.0006) [2023-12-26 23:35:49,398][105620] Updated weights for policy 1, policy_version 1130282 (0.0007) [2023-12-26 23:35:49,715][105692] Updated weights for policy 0, policy_version 1128920 (0.0011) [2023-12-26 23:35:49,774][105692] Updated weights for policy 0, policy_version 1128930 (0.0011) [2023-12-26 23:35:49,840][105692] Updated weights for policy 0, policy_version 1128940 (0.0011) [2023-12-26 23:35:50,075][105620] Updated weights for policy 1, policy_version 1130292 (0.0008) [2023-12-26 23:35:50,142][105620] Updated weights for policy 1, policy_version 1130302 (0.0008) [2023-12-26 23:35:50,202][105620] Updated weights for policy 1, policy_version 1130312 (0.0008) [2023-12-26 23:35:50,546][105692] Updated weights for policy 0, policy_version 1128950 (0.0011) [2023-12-26 23:35:50,604][105692] Updated weights for policy 0, policy_version 1128960 (0.0009) [2023-12-26 23:35:50,663][105692] Updated weights for policy 0, policy_version 1128970 (0.0007) [2023-12-26 23:35:50,928][105620] Updated weights for policy 1, policy_version 1130322 (0.0009) [2023-12-26 23:35:50,991][105620] Updated weights for policy 1, policy_version 1130332 (0.0010) [2023-12-26 23:35:51,062][105620] Updated weights for policy 1, policy_version 1130342 (0.0010) [2023-12-26 23:35:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 578461696. Throughput: 0: 9772.4, 1: 9922.9. Samples: 578455260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:35:51,062][104569] Avg episode reward: [(0, '9259.643'), (1, '9258.256')] [2023-12-26 23:35:51,133][105620] Updated weights for policy 1, policy_version 1130352 (0.0010) [2023-12-26 23:35:51,433][105692] Updated weights for policy 0, policy_version 1128980 (0.0009) [2023-12-26 23:35:51,487][105692] Updated weights for policy 0, policy_version 1128990 (0.0006) [2023-12-26 23:35:51,540][105692] Updated weights for policy 0, policy_version 1129000 (0.0006) [2023-12-26 23:35:51,871][105620] Updated weights for policy 1, policy_version 1130362 (0.0008) [2023-12-26 23:35:51,931][105620] Updated weights for policy 1, policy_version 1130372 (0.0008) [2023-12-26 23:35:51,981][105620] Updated weights for policy 1, policy_version 1130382 (0.0009) [2023-12-26 23:35:52,214][105692] Updated weights for policy 0, policy_version 1129010 (0.0009) [2023-12-26 23:35:52,268][105692] Updated weights for policy 0, policy_version 1129020 (0.0009) [2023-12-26 23:35:52,322][105692] Updated weights for policy 0, policy_version 1129030 (0.0009) [2023-12-26 23:35:52,380][105692] Updated weights for policy 0, policy_version 1129040 (0.0010) [2023-12-26 23:35:52,759][105620] Updated weights for policy 1, policy_version 1130392 (0.0009) [2023-12-26 23:35:52,814][105620] Updated weights for policy 1, policy_version 1130402 (0.0008) [2023-12-26 23:35:52,860][105620] Updated weights for policy 1, policy_version 1130412 (0.0008) [2023-12-26 23:35:53,069][105692] Updated weights for policy 0, policy_version 1129050 (0.0008) [2023-12-26 23:35:53,132][105692] Updated weights for policy 0, policy_version 1129060 (0.0006) [2023-12-26 23:35:53,197][105692] Updated weights for policy 0, policy_version 1129070 (0.0008) [2023-12-26 23:35:53,668][105620] Updated weights for policy 1, policy_version 1130422 (0.0010) [2023-12-26 23:35:53,716][105620] Updated weights for policy 1, policy_version 1130432 (0.0010) [2023-12-26 23:35:53,775][105620] Updated weights for policy 1, policy_version 1130442 (0.0010) [2023-12-26 23:35:53,881][105692] Updated weights for policy 0, policy_version 1129080 (0.0006) [2023-12-26 23:35:53,930][105692] Updated weights for policy 0, policy_version 1129090 (0.0008) [2023-12-26 23:35:53,977][105692] Updated weights for policy 0, policy_version 1129100 (0.0008) [2023-12-26 23:35:54,483][105620] Updated weights for policy 1, policy_version 1130452 (0.0010) [2023-12-26 23:35:54,550][105620] Updated weights for policy 1, policy_version 1130462 (0.0008) [2023-12-26 23:35:54,616][105620] Updated weights for policy 1, policy_version 1130472 (0.0008) [2023-12-26 23:35:54,649][105692] Updated weights for policy 0, policy_version 1129110 (0.0007) [2023-12-26 23:35:54,703][105692] Updated weights for policy 0, policy_version 1129120 (0.0009) [2023-12-26 23:35:54,759][105692] Updated weights for policy 0, policy_version 1129130 (0.0008) [2023-12-26 23:35:55,227][105620] Updated weights for policy 1, policy_version 1130482 (0.0006) [2023-12-26 23:35:55,289][105620] Updated weights for policy 1, policy_version 1130492 (0.0011) [2023-12-26 23:35:55,351][105620] Updated weights for policy 1, policy_version 1130502 (0.0010) [2023-12-26 23:35:55,409][105620] Updated weights for policy 1, policy_version 1130512 (0.0010) [2023-12-26 23:35:55,434][105692] Updated weights for policy 0, policy_version 1129140 (0.0008) [2023-12-26 23:35:55,483][105692] Updated weights for policy 0, policy_version 1129150 (0.0007) [2023-12-26 23:35:55,535][105692] Updated weights for policy 0, policy_version 1129160 (0.0005) [2023-12-26 23:35:56,052][105620] Updated weights for policy 1, policy_version 1130522 (0.0005) [2023-12-26 23:35:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 578560000. Throughput: 0: 9773.0, 1: 9951.3. Samples: 578572552. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:35:56,063][104569] Avg episode reward: [(0, '9079.079'), (1, '9350.595')] [2023-12-26 23:35:56,101][105620] Updated weights for policy 1, policy_version 1130532 (0.0005) [2023-12-26 23:35:56,115][105692] Updated weights for policy 0, policy_version 1129170 (0.0006) [2023-12-26 23:35:56,151][105620] Updated weights for policy 1, policy_version 1130542 (0.0008) [2023-12-26 23:35:56,169][105692] Updated weights for policy 0, policy_version 1129180 (0.0010) [2023-12-26 23:35:56,224][105692] Updated weights for policy 0, policy_version 1129190 (0.0008) [2023-12-26 23:35:56,292][105692] Updated weights for policy 0, policy_version 1129200 (0.0005) [2023-12-26 23:35:56,815][105692] Updated weights for policy 0, policy_version 1129210 (0.0008) [2023-12-26 23:35:56,815][105620] Updated weights for policy 1, policy_version 1130552 (0.0007) [2023-12-26 23:35:56,870][105620] Updated weights for policy 1, policy_version 1130562 (0.0010) [2023-12-26 23:35:56,873][105692] Updated weights for policy 0, policy_version 1129220 (0.0010) [2023-12-26 23:35:56,929][105620] Updated weights for policy 1, policy_version 1130572 (0.0011) [2023-12-26 23:35:56,933][105692] Updated weights for policy 0, policy_version 1129230 (0.0010) [2023-12-26 23:35:57,520][105692] Updated weights for policy 0, policy_version 1129240 (0.0005) [2023-12-26 23:35:57,563][105692] Updated weights for policy 0, policy_version 1129250 (0.0005) [2023-12-26 23:35:57,610][105692] Updated weights for policy 0, policy_version 1129260 (0.0005) [2023-12-26 23:35:57,642][105620] Updated weights for policy 1, policy_version 1130582 (0.0011) [2023-12-26 23:35:57,687][105620] Updated weights for policy 1, policy_version 1130592 (0.0010) [2023-12-26 23:35:57,738][105620] Updated weights for policy 1, policy_version 1130602 (0.0010) [2023-12-26 23:35:58,195][105692] Updated weights for policy 0, policy_version 1129270 (0.0008) [2023-12-26 23:35:58,257][105692] Updated weights for policy 0, policy_version 1129280 (0.0010) [2023-12-26 23:35:58,329][105692] Updated weights for policy 0, policy_version 1129290 (0.0011) [2023-12-26 23:35:58,484][105620] Updated weights for policy 1, policy_version 1130612 (0.0009) [2023-12-26 23:35:58,540][105620] Updated weights for policy 1, policy_version 1130622 (0.0008) [2023-12-26 23:35:58,591][105620] Updated weights for policy 1, policy_version 1130632 (0.0006) [2023-12-26 23:35:59,173][105692] Updated weights for policy 0, policy_version 1129300 (0.0010) [2023-12-26 23:35:59,238][105692] Updated weights for policy 0, policy_version 1129310 (0.0008) [2023-12-26 23:35:59,304][105692] Updated weights for policy 0, policy_version 1129320 (0.0009) [2023-12-26 23:35:59,404][105620] Updated weights for policy 1, policy_version 1130642 (0.0008) [2023-12-26 23:35:59,458][105620] Updated weights for policy 1, policy_version 1130653 (0.0010) [2023-12-26 23:35:59,518][105620] Updated weights for policy 1, policy_version 1130663 (0.0006) [2023-12-26 23:36:00,019][105692] Updated weights for policy 0, policy_version 1129330 (0.0009) [2023-12-26 23:36:00,073][105692] Updated weights for policy 0, policy_version 1129340 (0.0006) [2023-12-26 23:36:00,128][105692] Updated weights for policy 0, policy_version 1129350 (0.0010) [2023-12-26 23:36:00,177][105620] Updated weights for policy 1, policy_version 1130673 (0.0006) [2023-12-26 23:36:00,187][105692] Updated weights for policy 0, policy_version 1129360 (0.0010) [2023-12-26 23:36:00,228][105620] Updated weights for policy 1, policy_version 1130683 (0.0008) [2023-12-26 23:36:00,281][105620] Updated weights for policy 1, policy_version 1130693 (0.0010) [2023-12-26 23:36:00,330][105620] Updated weights for policy 1, policy_version 1130703 (0.0007) [2023-12-26 23:36:00,869][105692] Updated weights for policy 0, policy_version 1129370 (0.0006) [2023-12-26 23:36:00,921][105692] Updated weights for policy 0, policy_version 1129380 (0.0006) [2023-12-26 23:36:00,972][105692] Updated weights for policy 0, policy_version 1129390 (0.0005) [2023-12-26 23:36:01,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 578666496. Throughput: 0: 9922.7, 1: 9879.7. Samples: 578636236. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:36:01,063][104569] Avg episode reward: [(0, '9258.196'), (1, '9076.711')] [2023-12-26 23:36:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001129392_289169408.pth... [2023-12-26 23:36:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001130704_289497088.pth... [2023-12-26 23:36:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001128208_288866304.pth [2023-12-26 23:36:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001129584_289210368.pth [2023-12-26 23:36:01,179][105620] Updated weights for policy 1, policy_version 1130713 (0.0010) [2023-12-26 23:36:01,242][105620] Updated weights for policy 1, policy_version 1130723 (0.0009) [2023-12-26 23:36:01,306][105620] Updated weights for policy 1, policy_version 1130733 (0.0009) [2023-12-26 23:36:01,629][105692] Updated weights for policy 0, policy_version 1129400 (0.0008) [2023-12-26 23:36:01,696][105692] Updated weights for policy 0, policy_version 1129410 (0.0009) [2023-12-26 23:36:01,761][105692] Updated weights for policy 0, policy_version 1129420 (0.0009) [2023-12-26 23:36:02,074][105620] Updated weights for policy 1, policy_version 1130743 (0.0007) [2023-12-26 23:36:02,132][105620] Updated weights for policy 1, policy_version 1130753 (0.0005) [2023-12-26 23:36:02,183][105620] Updated weights for policy 1, policy_version 1130763 (0.0007) [2023-12-26 23:36:02,406][105692] Updated weights for policy 0, policy_version 1129430 (0.0010) [2023-12-26 23:36:02,464][105692] Updated weights for policy 0, policy_version 1129440 (0.0008) [2023-12-26 23:36:02,523][105692] Updated weights for policy 0, policy_version 1129450 (0.0007) [2023-12-26 23:36:02,983][105620] Updated weights for policy 1, policy_version 1130775 (0.0010) [2023-12-26 23:36:03,033][105620] Updated weights for policy 1, policy_version 1130785 (0.0009) [2023-12-26 23:36:03,087][105692] Updated weights for policy 0, policy_version 1129460 (0.0006) [2023-12-26 23:36:03,095][105620] Updated weights for policy 1, policy_version 1130795 (0.0009) [2023-12-26 23:36:03,134][105692] Updated weights for policy 0, policy_version 1129470 (0.0006) [2023-12-26 23:36:03,187][105692] Updated weights for policy 0, policy_version 1129480 (0.0008) [2023-12-26 23:36:03,817][105692] Updated weights for policy 0, policy_version 1129490 (0.0007) [2023-12-26 23:36:03,849][105620] Updated weights for policy 1, policy_version 1130805 (0.0008) [2023-12-26 23:36:03,886][105692] Updated weights for policy 0, policy_version 1129500 (0.0010) [2023-12-26 23:36:03,908][105620] Updated weights for policy 1, policy_version 1130815 (0.0011) [2023-12-26 23:36:03,944][105692] Updated weights for policy 0, policy_version 1129510 (0.0007) [2023-12-26 23:36:03,967][105620] Updated weights for policy 1, policy_version 1130825 (0.0007) [2023-12-26 23:36:03,999][105692] Updated weights for policy 0, policy_version 1129520 (0.0007) [2023-12-26 23:36:04,738][105620] Updated weights for policy 1, policy_version 1130835 (0.0006) [2023-12-26 23:36:04,756][105692] Updated weights for policy 0, policy_version 1129530 (0.0007) [2023-12-26 23:36:04,784][105620] Updated weights for policy 1, policy_version 1130845 (0.0007) [2023-12-26 23:36:04,814][105692] Updated weights for policy 0, policy_version 1129540 (0.0007) [2023-12-26 23:36:04,841][105620] Updated weights for policy 1, policy_version 1130856 (0.0009) [2023-12-26 23:36:04,868][105692] Updated weights for policy 0, policy_version 1129550 (0.0005) [2023-12-26 23:36:05,418][105620] Updated weights for policy 1, policy_version 1130866 (0.0005) [2023-12-26 23:36:05,450][105692] Updated weights for policy 0, policy_version 1129560 (0.0005) [2023-12-26 23:36:05,475][105620] Updated weights for policy 1, policy_version 1130876 (0.0005) [2023-12-26 23:36:05,504][105692] Updated weights for policy 0, policy_version 1129570 (0.0005) [2023-12-26 23:36:05,536][105620] Updated weights for policy 1, policy_version 1130886 (0.0005) [2023-12-26 23:36:05,555][105692] Updated weights for policy 0, policy_version 1129580 (0.0008) [2023-12-26 23:36:05,588][105620] Updated weights for policy 1, policy_version 1130896 (0.0006) [2023-12-26 23:36:06,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 578764800. Throughput: 0: 10087.2, 1: 9787.6. Samples: 578752100. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:36:06,062][104569] Avg episode reward: [(0, '8996.720'), (1, '9076.138')] [2023-12-26 23:36:06,103][105692] Updated weights for policy 0, policy_version 1129590 (0.0008) [2023-12-26 23:36:06,174][105692] Updated weights for policy 0, policy_version 1129600 (0.0010) [2023-12-26 23:36:06,235][105692] Updated weights for policy 0, policy_version 1129610 (0.0008) [2023-12-26 23:36:06,374][105620] Updated weights for policy 1, policy_version 1130906 (0.0006) [2023-12-26 23:36:06,435][105620] Updated weights for policy 1, policy_version 1130916 (0.0007) [2023-12-26 23:36:06,496][105620] Updated weights for policy 1, policy_version 1130926 (0.0009) [2023-12-26 23:36:06,988][105692] Updated weights for policy 0, policy_version 1129620 (0.0009) [2023-12-26 23:36:07,036][105692] Updated weights for policy 0, policy_version 1129630 (0.0008) [2023-12-26 23:36:07,091][105692] Updated weights for policy 0, policy_version 1129640 (0.0009) [2023-12-26 23:36:07,221][105620] Updated weights for policy 1, policy_version 1130936 (0.0009) [2023-12-26 23:36:07,285][105620] Updated weights for policy 1, policy_version 1130946 (0.0008) [2023-12-26 23:36:07,347][105620] Updated weights for policy 1, policy_version 1130956 (0.0009) [2023-12-26 23:36:07,915][105692] Updated weights for policy 0, policy_version 1129650 (0.0009) [2023-12-26 23:36:07,964][105692] Updated weights for policy 0, policy_version 1129660 (0.0008) [2023-12-26 23:36:08,011][105692] Updated weights for policy 0, policy_version 1129670 (0.0009) [2023-12-26 23:36:08,037][105620] Updated weights for policy 1, policy_version 1130966 (0.0007) [2023-12-26 23:36:08,067][105692] Updated weights for policy 0, policy_version 1129680 (0.0008) [2023-12-26 23:36:08,095][105620] Updated weights for policy 1, policy_version 1130976 (0.0005) [2023-12-26 23:36:08,157][105620] Updated weights for policy 1, policy_version 1130986 (0.0005) [2023-12-26 23:36:08,821][105620] Updated weights for policy 1, policy_version 1130996 (0.0008) [2023-12-26 23:36:08,877][105620] Updated weights for policy 1, policy_version 1131006 (0.0011) [2023-12-26 23:36:08,920][105692] Updated weights for policy 0, policy_version 1129690 (0.0006) [2023-12-26 23:36:08,940][105620] Updated weights for policy 1, policy_version 1131016 (0.0011) [2023-12-26 23:36:08,971][105692] Updated weights for policy 0, policy_version 1129700 (0.0007) [2023-12-26 23:36:09,022][105692] Updated weights for policy 0, policy_version 1129710 (0.0008) [2023-12-26 23:36:09,681][105620] Updated weights for policy 1, policy_version 1131026 (0.0010) [2023-12-26 23:36:09,735][105620] Updated weights for policy 1, policy_version 1131036 (0.0009) [2023-12-26 23:36:09,794][105620] Updated weights for policy 1, policy_version 1131046 (0.0007) [2023-12-26 23:36:09,848][105692] Updated weights for policy 0, policy_version 1129720 (0.0007) [2023-12-26 23:36:09,853][105620] Updated weights for policy 1, policy_version 1131056 (0.0008) [2023-12-26 23:36:09,902][105692] Updated weights for policy 0, policy_version 1129730 (0.0008) [2023-12-26 23:36:09,955][105692] Updated weights for policy 0, policy_version 1129740 (0.0008) [2023-12-26 23:36:10,631][105620] Updated weights for policy 1, policy_version 1131066 (0.0009) [2023-12-26 23:36:10,685][105692] Updated weights for policy 0, policy_version 1129750 (0.0008) [2023-12-26 23:36:10,693][105620] Updated weights for policy 1, policy_version 1131076 (0.0009) [2023-12-26 23:36:10,745][105692] Updated weights for policy 0, policy_version 1129760 (0.0010) [2023-12-26 23:36:10,747][105620] Updated weights for policy 1, policy_version 1131086 (0.0008) [2023-12-26 23:36:10,797][105692] Updated weights for policy 0, policy_version 1129770 (0.0008) [2023-12-26 23:36:11,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 578863104. Throughput: 0: 10130.1, 1: 9827.1. Samples: 578869424. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:36:11,062][104569] Avg episode reward: [(0, '8556.611'), (1, '9349.258')] [2023-12-26 23:36:11,531][105620] Updated weights for policy 1, policy_version 1131096 (0.0008) [2023-12-26 23:36:11,554][105692] Updated weights for policy 0, policy_version 1129780 (0.0010) [2023-12-26 23:36:11,583][105620] Updated weights for policy 1, policy_version 1131106 (0.0005) [2023-12-26 23:36:11,621][105692] Updated weights for policy 0, policy_version 1129790 (0.0008) [2023-12-26 23:36:11,647][105620] Updated weights for policy 1, policy_version 1131116 (0.0007) [2023-12-26 23:36:11,686][105692] Updated weights for policy 0, policy_version 1129800 (0.0008) [2023-12-26 23:36:12,421][105620] Updated weights for policy 1, policy_version 1131126 (0.0008) [2023-12-26 23:36:12,472][105692] Updated weights for policy 0, policy_version 1129810 (0.0009) [2023-12-26 23:36:12,489][105620] Updated weights for policy 1, policy_version 1131136 (0.0005) [2023-12-26 23:36:12,537][105692] Updated weights for policy 0, policy_version 1129820 (0.0008) [2023-12-26 23:36:12,558][105620] Updated weights for policy 1, policy_version 1131146 (0.0006) [2023-12-26 23:36:12,601][105692] Updated weights for policy 0, policy_version 1129830 (0.0008) [2023-12-26 23:36:12,667][105692] Updated weights for policy 0, policy_version 1129840 (0.0007) [2023-12-26 23:36:13,241][105692] Updated weights for policy 0, policy_version 1129850 (0.0005) [2023-12-26 23:36:13,255][105620] Updated weights for policy 1, policy_version 1131156 (0.0009) [2023-12-26 23:36:13,299][105692] Updated weights for policy 0, policy_version 1129860 (0.0005) [2023-12-26 23:36:13,311][105620] Updated weights for policy 1, policy_version 1131166 (0.0008) [2023-12-26 23:36:13,350][105692] Updated weights for policy 0, policy_version 1129870 (0.0007) [2023-12-26 23:36:13,373][105620] Updated weights for policy 1, policy_version 1131176 (0.0009) [2023-12-26 23:36:13,885][105692] Updated weights for policy 0, policy_version 1129880 (0.0007) [2023-12-26 23:36:13,936][105692] Updated weights for policy 0, policy_version 1129890 (0.0005) [2023-12-26 23:36:14,000][105692] Updated weights for policy 0, policy_version 1129900 (0.0005) [2023-12-26 23:36:14,091][105620] Updated weights for policy 1, policy_version 1131186 (0.0009) [2023-12-26 23:36:14,153][105620] Updated weights for policy 1, policy_version 1131196 (0.0006) [2023-12-26 23:36:14,220][105620] Updated weights for policy 1, policy_version 1131206 (0.0008) [2023-12-26 23:36:14,281][105620] Updated weights for policy 1, policy_version 1131216 (0.0008) [2023-12-26 23:36:14,557][105692] Updated weights for policy 0, policy_version 1129910 (0.0006) [2023-12-26 23:36:14,617][105692] Updated weights for policy 0, policy_version 1129920 (0.0005) [2023-12-26 23:36:14,667][105692] Updated weights for policy 0, policy_version 1129930 (0.0005) [2023-12-26 23:36:14,964][105620] Updated weights for policy 1, policy_version 1131226 (0.0009) [2023-12-26 23:36:15,029][105620] Updated weights for policy 1, policy_version 1131236 (0.0011) [2023-12-26 23:36:15,086][105620] Updated weights for policy 1, policy_version 1131246 (0.0011) [2023-12-26 23:36:15,261][105692] Updated weights for policy 0, policy_version 1129940 (0.0006) [2023-12-26 23:36:15,321][105692] Updated weights for policy 0, policy_version 1129950 (0.0007) [2023-12-26 23:36:15,386][105692] Updated weights for policy 0, policy_version 1129960 (0.0008) [2023-12-26 23:36:15,861][105620] Updated weights for policy 1, policy_version 1131256 (0.0010) [2023-12-26 23:36:15,908][105620] Updated weights for policy 1, policy_version 1131266 (0.0010) [2023-12-26 23:36:15,960][105620] Updated weights for policy 1, policy_version 1131276 (0.0010) [2023-12-26 23:36:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 578961408. Throughput: 0: 10131.6, 1: 9736.9. Samples: 578927464. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:36:16,063][104569] Avg episode reward: [(0, '8290.710'), (1, '9258.360')] [2023-12-26 23:36:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001129968_289316864.pth... [2023-12-26 23:36:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001131280_289644544.pth... [2023-12-26 23:36:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001128784_289013760.pth [2023-12-26 23:36:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001130128_289349632.pth [2023-12-26 23:36:16,142][105692] Updated weights for policy 0, policy_version 1129970 (0.0008) [2023-12-26 23:36:16,198][105692] Updated weights for policy 0, policy_version 1129980 (0.0008) [2023-12-26 23:36:16,254][105692] Updated weights for policy 0, policy_version 1129990 (0.0008) [2023-12-26 23:36:16,311][105692] Updated weights for policy 0, policy_version 1130000 (0.0008) [2023-12-26 23:36:16,720][105620] Updated weights for policy 1, policy_version 1131286 (0.0007) [2023-12-26 23:36:16,768][105620] Updated weights for policy 1, policy_version 1131296 (0.0005) [2023-12-26 23:36:16,824][105620] Updated weights for policy 1, policy_version 1131306 (0.0005) [2023-12-26 23:36:16,911][105692] Updated weights for policy 0, policy_version 1130010 (0.0005) [2023-12-26 23:36:16,971][105692] Updated weights for policy 0, policy_version 1130020 (0.0006) [2023-12-26 23:36:17,039][105692] Updated weights for policy 0, policy_version 1130030 (0.0006) [2023-12-26 23:36:17,418][105620] Updated weights for policy 1, policy_version 1131316 (0.0005) [2023-12-26 23:36:17,468][105620] Updated weights for policy 1, policy_version 1131326 (0.0005) [2023-12-26 23:36:17,518][105620] Updated weights for policy 1, policy_version 1131336 (0.0006) [2023-12-26 23:36:17,646][105692] Updated weights for policy 0, policy_version 1130040 (0.0008) [2023-12-26 23:36:17,703][105692] Updated weights for policy 0, policy_version 1130050 (0.0009) [2023-12-26 23:36:17,764][105692] Updated weights for policy 0, policy_version 1130060 (0.0010) [2023-12-26 23:36:18,214][105620] Updated weights for policy 1, policy_version 1131346 (0.0007) [2023-12-26 23:36:18,265][105620] Updated weights for policy 1, policy_version 1131356 (0.0009) [2023-12-26 23:36:18,320][105620] Updated weights for policy 1, policy_version 1131366 (0.0010) [2023-12-26 23:36:18,382][105620] Updated weights for policy 1, policy_version 1131376 (0.0006) [2023-12-26 23:36:18,487][105692] Updated weights for policy 0, policy_version 1130070 (0.0009) [2023-12-26 23:36:18,537][105692] Updated weights for policy 0, policy_version 1130080 (0.0009) [2023-12-26 23:36:18,591][105692] Updated weights for policy 0, policy_version 1130090 (0.0005) [2023-12-26 23:36:19,032][105620] Updated weights for policy 1, policy_version 1131386 (0.0009) [2023-12-26 23:36:19,090][105620] Updated weights for policy 1, policy_version 1131396 (0.0009) [2023-12-26 23:36:19,148][105620] Updated weights for policy 1, policy_version 1131406 (0.0009) [2023-12-26 23:36:19,375][105692] Updated weights for policy 0, policy_version 1130100 (0.0007) [2023-12-26 23:36:19,424][105692] Updated weights for policy 0, policy_version 1130110 (0.0008) [2023-12-26 23:36:19,487][105692] Updated weights for policy 0, policy_version 1130120 (0.0008) [2023-12-26 23:36:19,923][105620] Updated weights for policy 1, policy_version 1131416 (0.0010) [2023-12-26 23:36:19,980][105620] Updated weights for policy 1, policy_version 1131426 (0.0011) [2023-12-26 23:36:20,044][105620] Updated weights for policy 1, policy_version 1131436 (0.0011) [2023-12-26 23:36:20,167][105692] Updated weights for policy 0, policy_version 1130130 (0.0006) [2023-12-26 23:36:20,232][105692] Updated weights for policy 0, policy_version 1130140 (0.0006) [2023-12-26 23:36:20,292][105692] Updated weights for policy 0, policy_version 1130150 (0.0007) [2023-12-26 23:36:20,337][105692] Updated weights for policy 0, policy_version 1130160 (0.0005) [2023-12-26 23:36:20,805][105620] Updated weights for policy 1, policy_version 1131446 (0.0011) [2023-12-26 23:36:20,868][105620] Updated weights for policy 1, policy_version 1131456 (0.0010) [2023-12-26 23:36:20,930][105620] Updated weights for policy 1, policy_version 1131466 (0.0010) [2023-12-26 23:36:21,008][105692] Updated weights for policy 0, policy_version 1130170 (0.0008) [2023-12-26 23:36:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19933.8, 300 sec: 19577.5). Total num frames: 579059712. Throughput: 0: 10092.5, 1: 9733.1. Samples: 579049108. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:36:21,063][104569] Avg episode reward: [(0, '8548.014'), (1, '9166.374')] [2023-12-26 23:36:21,075][105692] Updated weights for policy 0, policy_version 1130180 (0.0008) [2023-12-26 23:36:21,140][105692] Updated weights for policy 0, policy_version 1130190 (0.0009) [2023-12-26 23:36:21,695][105620] Updated weights for policy 1, policy_version 1131476 (0.0010) [2023-12-26 23:36:21,759][105620] Updated weights for policy 1, policy_version 1131486 (0.0008) [2023-12-26 23:36:21,807][105692] Updated weights for policy 0, policy_version 1130200 (0.0006) [2023-12-26 23:36:21,821][105620] Updated weights for policy 1, policy_version 1131496 (0.0008) [2023-12-26 23:36:21,858][105692] Updated weights for policy 0, policy_version 1130210 (0.0006) [2023-12-26 23:36:21,922][105692] Updated weights for policy 0, policy_version 1130220 (0.0008) [2023-12-26 23:36:22,573][105620] Updated weights for policy 1, policy_version 1131506 (0.0008) [2023-12-26 23:36:22,630][105620] Updated weights for policy 1, policy_version 1131516 (0.0008) [2023-12-26 23:36:22,651][105692] Updated weights for policy 0, policy_version 1130230 (0.0006) [2023-12-26 23:36:22,687][105620] Updated weights for policy 1, policy_version 1131526 (0.0007) [2023-12-26 23:36:22,717][105692] Updated weights for policy 0, policy_version 1130240 (0.0008) [2023-12-26 23:36:22,739][105620] Updated weights for policy 1, policy_version 1131536 (0.0007) [2023-12-26 23:36:22,780][105692] Updated weights for policy 0, policy_version 1130250 (0.0008) [2023-12-26 23:36:23,457][105692] Updated weights for policy 0, policy_version 1130260 (0.0009) [2023-12-26 23:36:23,504][105692] Updated weights for policy 0, policy_version 1130270 (0.0007) [2023-12-26 23:36:23,540][105620] Updated weights for policy 1, policy_version 1131546 (0.0009) [2023-12-26 23:36:23,560][105692] Updated weights for policy 0, policy_version 1130280 (0.0005) [2023-12-26 23:36:23,599][105620] Updated weights for policy 1, policy_version 1131556 (0.0008) [2023-12-26 23:36:23,663][105620] Updated weights for policy 1, policy_version 1131566 (0.0010) [2023-12-26 23:36:24,227][105692] Updated weights for policy 0, policy_version 1130290 (0.0006) [2023-12-26 23:36:24,277][105692] Updated weights for policy 0, policy_version 1130300 (0.0008) [2023-12-26 23:36:24,320][105620] Updated weights for policy 1, policy_version 1131576 (0.0008) [2023-12-26 23:36:24,327][105692] Updated weights for policy 0, policy_version 1130310 (0.0006) [2023-12-26 23:36:24,377][105620] Updated weights for policy 1, policy_version 1131586 (0.0007) [2023-12-26 23:36:24,384][105692] Updated weights for policy 0, policy_version 1130320 (0.0006) [2023-12-26 23:36:24,429][105620] Updated weights for policy 1, policy_version 1131596 (0.0009) [2023-12-26 23:36:25,091][105692] Updated weights for policy 0, policy_version 1130330 (0.0009) [2023-12-26 23:36:25,152][105692] Updated weights for policy 0, policy_version 1130340 (0.0009) [2023-12-26 23:36:25,202][105692] Updated weights for policy 0, policy_version 1130350 (0.0008) [2023-12-26 23:36:25,220][105620] Updated weights for policy 1, policy_version 1131606 (0.0007) [2023-12-26 23:36:25,288][105620] Updated weights for policy 1, policy_version 1131616 (0.0005) [2023-12-26 23:36:25,357][105620] Updated weights for policy 1, policy_version 1131626 (0.0006) [2023-12-26 23:36:25,990][105692] Updated weights for policy 0, policy_version 1130360 (0.0009) [2023-12-26 23:36:26,039][105620] Updated weights for policy 1, policy_version 1131636 (0.0007) [2023-12-26 23:36:26,052][105692] Updated weights for policy 0, policy_version 1130370 (0.0009) [2023-12-26 23:36:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 579149824. Throughput: 0: 10159.7, 1: 9704.8. Samples: 579165072. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:36:26,063][104569] Avg episode reward: [(0, '9084.311'), (1, '9164.954')] [2023-12-26 23:36:26,098][105620] Updated weights for policy 1, policy_version 1131646 (0.0007) [2023-12-26 23:36:26,117][105692] Updated weights for policy 0, policy_version 1130380 (0.0009) [2023-12-26 23:36:26,144][105620] Updated weights for policy 1, policy_version 1131656 (0.0007) [2023-12-26 23:36:26,785][105692] Updated weights for policy 0, policy_version 1130390 (0.0006) [2023-12-26 23:36:26,835][105620] Updated weights for policy 1, policy_version 1131666 (0.0008) [2023-12-26 23:36:26,847][105692] Updated weights for policy 0, policy_version 1130400 (0.0005) [2023-12-26 23:36:26,886][105620] Updated weights for policy 1, policy_version 1131676 (0.0005) [2023-12-26 23:36:26,891][105692] Updated weights for policy 0, policy_version 1130410 (0.0005) [2023-12-26 23:36:26,933][105620] Updated weights for policy 1, policy_version 1131686 (0.0006) [2023-12-26 23:36:26,984][105620] Updated weights for policy 1, policy_version 1131696 (0.0005) [2023-12-26 23:36:27,416][105692] Updated weights for policy 0, policy_version 1130420 (0.0007) [2023-12-26 23:36:27,467][105692] Updated weights for policy 0, policy_version 1130430 (0.0010) [2023-12-26 23:36:27,518][105692] Updated weights for policy 0, policy_version 1130440 (0.0010) [2023-12-26 23:36:27,682][105620] Updated weights for policy 1, policy_version 1131706 (0.0010) [2023-12-26 23:36:27,730][105620] Updated weights for policy 1, policy_version 1131716 (0.0009) [2023-12-26 23:36:27,778][105620] Updated weights for policy 1, policy_version 1131726 (0.0008) [2023-12-26 23:36:28,100][105692] Updated weights for policy 0, policy_version 1130450 (0.0006) [2023-12-26 23:36:28,143][105692] Updated weights for policy 0, policy_version 1130460 (0.0005) [2023-12-26 23:36:28,190][105692] Updated weights for policy 0, policy_version 1130470 (0.0010) [2023-12-26 23:36:28,241][105692] Updated weights for policy 0, policy_version 1130480 (0.0010) [2023-12-26 23:36:28,616][105620] Updated weights for policy 1, policy_version 1131736 (0.0006) [2023-12-26 23:36:28,670][105620] Updated weights for policy 1, policy_version 1131746 (0.0009) [2023-12-26 23:36:28,725][105620] Updated weights for policy 1, policy_version 1131756 (0.0007) [2023-12-26 23:36:28,932][105692] Updated weights for policy 0, policy_version 1130490 (0.0006) [2023-12-26 23:36:28,996][105692] Updated weights for policy 0, policy_version 1130500 (0.0005) [2023-12-26 23:36:29,054][105692] Updated weights for policy 0, policy_version 1130510 (0.0006) [2023-12-26 23:36:29,387][105620] Updated weights for policy 1, policy_version 1131766 (0.0008) [2023-12-26 23:36:29,446][105620] Updated weights for policy 1, policy_version 1131776 (0.0008) [2023-12-26 23:36:29,505][105620] Updated weights for policy 1, policy_version 1131786 (0.0009) [2023-12-26 23:36:29,737][105692] Updated weights for policy 0, policy_version 1130520 (0.0009) [2023-12-26 23:36:29,795][105692] Updated weights for policy 0, policy_version 1130530 (0.0010) [2023-12-26 23:36:29,852][105692] Updated weights for policy 0, policy_version 1130540 (0.0009) [2023-12-26 23:36:30,296][105620] Updated weights for policy 1, policy_version 1131796 (0.0008) [2023-12-26 23:36:30,353][105620] Updated weights for policy 1, policy_version 1131806 (0.0008) [2023-12-26 23:36:30,406][105620] Updated weights for policy 1, policy_version 1131816 (0.0008) [2023-12-26 23:36:30,558][105692] Updated weights for policy 0, policy_version 1130550 (0.0010) [2023-12-26 23:36:30,619][105692] Updated weights for policy 0, policy_version 1130560 (0.0010) [2023-12-26 23:36:30,680][105692] Updated weights for policy 0, policy_version 1130570 (0.0010) [2023-12-26 23:36:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 579256320. Throughput: 0: 10250.2, 1: 9668.2. Samples: 579226524. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:36:31,063][104569] Avg episode reward: [(0, '9172.765'), (1, '9346.807')] [2023-12-26 23:36:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001130576_289472512.pth... [2023-12-26 23:36:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001131824_289783808.pth... [2023-12-26 23:36:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001129392_289169408.pth [2023-12-26 23:36:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001130704_289497088.pth [2023-12-26 23:36:31,178][105620] Updated weights for policy 1, policy_version 1131826 (0.0008) [2023-12-26 23:36:31,237][105620] Updated weights for policy 1, policy_version 1131836 (0.0008) [2023-12-26 23:36:31,296][105620] Updated weights for policy 1, policy_version 1131846 (0.0008) [2023-12-26 23:36:31,328][105692] Updated weights for policy 0, policy_version 1130580 (0.0010) [2023-12-26 23:36:31,358][105620] Updated weights for policy 1, policy_version 1131856 (0.0010) [2023-12-26 23:36:31,400][105692] Updated weights for policy 0, policy_version 1130590 (0.0011) [2023-12-26 23:36:31,457][105692] Updated weights for policy 0, policy_version 1130600 (0.0010) [2023-12-26 23:36:32,058][105620] Updated weights for policy 1, policy_version 1131866 (0.0006) [2023-12-26 23:36:32,090][105692] Updated weights for policy 0, policy_version 1130610 (0.0009) [2023-12-26 23:36:32,112][105620] Updated weights for policy 1, policy_version 1131876 (0.0009) [2023-12-26 23:36:32,139][105692] Updated weights for policy 0, policy_version 1130620 (0.0007) [2023-12-26 23:36:32,162][105620] Updated weights for policy 1, policy_version 1131886 (0.0007) [2023-12-26 23:36:32,196][105692] Updated weights for policy 0, policy_version 1130630 (0.0011) [2023-12-26 23:36:32,261][105692] Updated weights for policy 0, policy_version 1130640 (0.0006) [2023-12-26 23:36:32,861][105692] Updated weights for policy 0, policy_version 1130650 (0.0010) [2023-12-26 23:36:32,918][105620] Updated weights for policy 1, policy_version 1131896 (0.0006) [2023-12-26 23:36:32,919][105692] Updated weights for policy 0, policy_version 1130660 (0.0010) [2023-12-26 23:36:32,973][105620] Updated weights for policy 1, policy_version 1131906 (0.0008) [2023-12-26 23:36:32,980][105692] Updated weights for policy 0, policy_version 1130670 (0.0010) [2023-12-26 23:36:33,033][105620] Updated weights for policy 1, policy_version 1131916 (0.0010) [2023-12-26 23:36:33,609][105692] Updated weights for policy 0, policy_version 1130680 (0.0010) [2023-12-26 23:36:33,675][105692] Updated weights for policy 0, policy_version 1130690 (0.0010) [2023-12-26 23:36:33,720][105620] Updated weights for policy 1, policy_version 1131926 (0.0007) [2023-12-26 23:36:33,730][105692] Updated weights for policy 0, policy_version 1130700 (0.0010) [2023-12-26 23:36:33,773][105620] Updated weights for policy 1, policy_version 1131936 (0.0008) [2023-12-26 23:36:33,824][105620] Updated weights for policy 1, policy_version 1131946 (0.0008) [2023-12-26 23:36:34,481][105692] Updated weights for policy 0, policy_version 1130710 (0.0011) [2023-12-26 23:36:34,532][105692] Updated weights for policy 0, policy_version 1130720 (0.0010) [2023-12-26 23:36:34,555][105620] Updated weights for policy 1, policy_version 1131956 (0.0007) [2023-12-26 23:36:34,593][105692] Updated weights for policy 0, policy_version 1130730 (0.0011) [2023-12-26 23:36:34,611][105620] Updated weights for policy 1, policy_version 1131966 (0.0005) [2023-12-26 23:36:34,666][105620] Updated weights for policy 1, policy_version 1131976 (0.0007) [2023-12-26 23:36:35,318][105620] Updated weights for policy 1, policy_version 1131986 (0.0008) [2023-12-26 23:36:35,360][105692] Updated weights for policy 0, policy_version 1130740 (0.0010) [2023-12-26 23:36:35,364][105620] Updated weights for policy 1, policy_version 1131996 (0.0005) [2023-12-26 23:36:35,408][105692] Updated weights for policy 0, policy_version 1130750 (0.0010) [2023-12-26 23:36:35,423][105620] Updated weights for policy 1, policy_version 1132006 (0.0007) [2023-12-26 23:36:35,456][105692] Updated weights for policy 0, policy_version 1130760 (0.0010) [2023-12-26 23:36:35,475][105620] Updated weights for policy 1, policy_version 1132016 (0.0006) [2023-12-26 23:36:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 579354624. Throughput: 0: 10253.2, 1: 9546.6. Samples: 579346256. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:36:36,063][104569] Avg episode reward: [(0, '9170.810'), (1, '9256.938')] [2023-12-26 23:36:36,115][105620] Updated weights for policy 1, policy_version 1132026 (0.0012) [2023-12-26 23:36:36,171][105620] Updated weights for policy 1, policy_version 1132036 (0.0008) [2023-12-26 23:36:36,196][105692] Updated weights for policy 0, policy_version 1130770 (0.0010) [2023-12-26 23:36:36,225][105620] Updated weights for policy 1, policy_version 1132046 (0.0009) [2023-12-26 23:36:36,250][105692] Updated weights for policy 0, policy_version 1130780 (0.0007) [2023-12-26 23:36:36,303][105692] Updated weights for policy 0, policy_version 1130790 (0.0009) [2023-12-26 23:36:36,357][105692] Updated weights for policy 0, policy_version 1130800 (0.0009) [2023-12-26 23:36:36,903][105620] Updated weights for policy 1, policy_version 1132056 (0.0006) [2023-12-26 23:36:36,966][105620] Updated weights for policy 1, policy_version 1132066 (0.0010) [2023-12-26 23:36:37,036][105620] Updated weights for policy 1, policy_version 1132076 (0.0007) [2023-12-26 23:36:37,113][105692] Updated weights for policy 0, policy_version 1130810 (0.0008) [2023-12-26 23:36:37,183][105692] Updated weights for policy 0, policy_version 1130820 (0.0007) [2023-12-26 23:36:37,249][105692] Updated weights for policy 0, policy_version 1130830 (0.0010) [2023-12-26 23:36:37,584][105620] Updated weights for policy 1, policy_version 1132086 (0.0009) [2023-12-26 23:36:37,640][105620] Updated weights for policy 1, policy_version 1132096 (0.0009) [2023-12-26 23:36:37,690][105620] Updated weights for policy 1, policy_version 1132106 (0.0008) [2023-12-26 23:36:38,007][105692] Updated weights for policy 0, policy_version 1130840 (0.0009) [2023-12-26 23:36:38,066][105692] Updated weights for policy 0, policy_version 1130850 (0.0009) [2023-12-26 23:36:38,131][105692] Updated weights for policy 0, policy_version 1130860 (0.0010) [2023-12-26 23:36:38,340][105620] Updated weights for policy 1, policy_version 1132116 (0.0008) [2023-12-26 23:36:38,402][105620] Updated weights for policy 1, policy_version 1132126 (0.0011) [2023-12-26 23:36:38,461][105620] Updated weights for policy 1, policy_version 1132136 (0.0011) [2023-12-26 23:36:38,779][105692] Updated weights for policy 0, policy_version 1130870 (0.0009) [2023-12-26 23:36:38,835][105692] Updated weights for policy 0, policy_version 1130880 (0.0008) [2023-12-26 23:36:38,886][105692] Updated weights for policy 0, policy_version 1130890 (0.0008) [2023-12-26 23:36:39,083][105620] Updated weights for policy 1, policy_version 1132146 (0.0009) [2023-12-26 23:36:39,155][105620] Updated weights for policy 1, policy_version 1132156 (0.0005) [2023-12-26 23:36:39,224][105620] Updated weights for policy 1, policy_version 1132166 (0.0006) [2023-12-26 23:36:39,290][105620] Updated weights for policy 1, policy_version 1132176 (0.0008) [2023-12-26 23:36:39,603][105692] Updated weights for policy 0, policy_version 1130900 (0.0009) [2023-12-26 23:36:39,666][105692] Updated weights for policy 0, policy_version 1130910 (0.0009) [2023-12-26 23:36:39,724][105692] Updated weights for policy 0, policy_version 1130920 (0.0008) [2023-12-26 23:36:40,013][105620] Updated weights for policy 1, policy_version 1132186 (0.0008) [2023-12-26 23:36:40,075][105620] Updated weights for policy 1, policy_version 1132196 (0.0009) [2023-12-26 23:36:40,134][105620] Updated weights for policy 1, policy_version 1132206 (0.0009) [2023-12-26 23:36:40,477][105692] Updated weights for policy 0, policy_version 1130930 (0.0009) [2023-12-26 23:36:40,535][105692] Updated weights for policy 0, policy_version 1130940 (0.0006) [2023-12-26 23:36:40,596][105692] Updated weights for policy 0, policy_version 1130950 (0.0005) [2023-12-26 23:36:40,664][105692] Updated weights for policy 0, policy_version 1130960 (0.0005) [2023-12-26 23:36:40,961][105620] Updated weights for policy 1, policy_version 1132216 (0.0009) [2023-12-26 23:36:41,018][105620] Updated weights for policy 1, policy_version 1132226 (0.0009) [2023-12-26 23:36:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 579452928. Throughput: 0: 10195.6, 1: 9663.2. Samples: 579466196. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:36:41,062][104569] Avg episode reward: [(0, '9167.891'), (1, '9077.354')] [2023-12-26 23:36:41,088][105620] Updated weights for policy 1, policy_version 1132236 (0.0010) [2023-12-26 23:36:41,286][105692] Updated weights for policy 0, policy_version 1130970 (0.0009) [2023-12-26 23:36:41,341][105692] Updated weights for policy 0, policy_version 1130980 (0.0009) [2023-12-26 23:36:41,411][105692] Updated weights for policy 0, policy_version 1130990 (0.0009) [2023-12-26 23:36:41,818][105620] Updated weights for policy 1, policy_version 1132246 (0.0009) [2023-12-26 23:36:41,869][105620] Updated weights for policy 1, policy_version 1132256 (0.0009) [2023-12-26 23:36:41,917][105620] Updated weights for policy 1, policy_version 1132266 (0.0008) [2023-12-26 23:36:42,184][105692] Updated weights for policy 0, policy_version 1131000 (0.0009) [2023-12-26 23:36:42,238][105692] Updated weights for policy 0, policy_version 1131010 (0.0009) [2023-12-26 23:36:42,300][105692] Updated weights for policy 0, policy_version 1131020 (0.0008) [2023-12-26 23:36:42,708][105620] Updated weights for policy 1, policy_version 1132276 (0.0009) [2023-12-26 23:36:42,759][105620] Updated weights for policy 1, policy_version 1132286 (0.0009) [2023-12-26 23:36:42,813][105620] Updated weights for policy 1, policy_version 1132296 (0.0009) [2023-12-26 23:36:43,032][105692] Updated weights for policy 0, policy_version 1131030 (0.0007) [2023-12-26 23:36:43,083][105692] Updated weights for policy 0, policy_version 1131040 (0.0005) [2023-12-26 23:36:43,133][105692] Updated weights for policy 0, policy_version 1131050 (0.0005) [2023-12-26 23:36:43,599][105620] Updated weights for policy 1, policy_version 1132306 (0.0008) [2023-12-26 23:36:43,652][105620] Updated weights for policy 1, policy_version 1132316 (0.0005) [2023-12-26 23:36:43,710][105620] Updated weights for policy 1, policy_version 1132326 (0.0008) [2023-12-26 23:36:43,756][105620] Updated weights for policy 1, policy_version 1132336 (0.0009) [2023-12-26 23:36:43,822][105692] Updated weights for policy 0, policy_version 1131060 (0.0007) [2023-12-26 23:36:43,879][105692] Updated weights for policy 0, policy_version 1131070 (0.0009) [2023-12-26 23:36:43,940][105692] Updated weights for policy 0, policy_version 1131080 (0.0008) [2023-12-26 23:36:44,453][105620] Updated weights for policy 1, policy_version 1132346 (0.0008) [2023-12-26 23:36:44,501][105620] Updated weights for policy 1, policy_version 1132356 (0.0005) [2023-12-26 23:36:44,562][105620] Updated weights for policy 1, policy_version 1132366 (0.0005) [2023-12-26 23:36:44,736][105692] Updated weights for policy 0, policy_version 1131090 (0.0009) [2023-12-26 23:36:44,797][105692] Updated weights for policy 0, policy_version 1131100 (0.0011) [2023-12-26 23:36:44,846][105692] Updated weights for policy 0, policy_version 1131110 (0.0010) [2023-12-26 23:36:44,920][105692] Updated weights for policy 0, policy_version 1131120 (0.0011) [2023-12-26 23:36:45,160][105620] Updated weights for policy 1, policy_version 1132376 (0.0009) [2023-12-26 23:36:45,213][105620] Updated weights for policy 1, policy_version 1132386 (0.0010) [2023-12-26 23:36:45,269][105620] Updated weights for policy 1, policy_version 1132396 (0.0010) [2023-12-26 23:36:45,662][105692] Updated weights for policy 0, policy_version 1131130 (0.0006) [2023-12-26 23:36:45,722][105692] Updated weights for policy 0, policy_version 1131140 (0.0009) [2023-12-26 23:36:45,777][105692] Updated weights for policy 0, policy_version 1131150 (0.0011) [2023-12-26 23:36:45,971][105620] Updated weights for policy 1, policy_version 1132406 (0.0007) [2023-12-26 23:36:46,021][105620] Updated weights for policy 1, policy_version 1132416 (0.0005) [2023-12-26 23:36:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.4, 300 sec: 19605.2). Total num frames: 579551232. Throughput: 0: 10084.0, 1: 9631.1. Samples: 579523416. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:36:46,063][104569] Avg episode reward: [(0, '8982.627'), (1, '9077.450')] [2023-12-26 23:36:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001131152_289619968.pth... [2023-12-26 23:36:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001129968_289316864.pth [2023-12-26 23:36:46,083][105620] Updated weights for policy 1, policy_version 1132426 (0.0005) [2023-12-26 23:36:46,115][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001132432_289939456.pth... [2023-12-26 23:36:46,118][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001131280_289644544.pth [2023-12-26 23:36:46,378][105692] Updated weights for policy 0, policy_version 1131160 (0.0009) [2023-12-26 23:36:46,430][105692] Updated weights for policy 0, policy_version 1131170 (0.0008) [2023-12-26 23:36:46,478][105692] Updated weights for policy 0, policy_version 1131180 (0.0007) [2023-12-26 23:36:46,707][105620] Updated weights for policy 1, policy_version 1132436 (0.0009) [2023-12-26 23:36:46,758][105620] Updated weights for policy 1, policy_version 1132446 (0.0009) [2023-12-26 23:36:46,809][105620] Updated weights for policy 1, policy_version 1132456 (0.0005) [2023-12-26 23:36:47,330][105692] Updated weights for policy 0, policy_version 1131190 (0.0009) [2023-12-26 23:36:47,377][105692] Updated weights for policy 0, policy_version 1131200 (0.0008) [2023-12-26 23:36:47,387][105585] KL-divergence is very high: 102.7612 [2023-12-26 23:36:47,406][105620] Updated weights for policy 1, policy_version 1132466 (0.0006) [2023-12-26 23:36:47,426][105585] KL-divergence is very high: 109.1712 [2023-12-26 23:36:47,428][105692] Updated weights for policy 0, policy_version 1131210 (0.0008) [2023-12-26 23:36:47,468][105620] Updated weights for policy 1, policy_version 1132476 (0.0011) [2023-12-26 23:36:47,524][105620] Updated weights for policy 1, policy_version 1132486 (0.0010) [2023-12-26 23:36:47,576][105620] Updated weights for policy 1, policy_version 1132496 (0.0010) [2023-12-26 23:36:48,165][105692] Updated weights for policy 0, policy_version 1131220 (0.0006) [2023-12-26 23:36:48,223][105692] Updated weights for policy 0, policy_version 1131230 (0.0005) [2023-12-26 23:36:48,280][105692] Updated weights for policy 0, policy_version 1131240 (0.0005) [2023-12-26 23:36:48,298][105620] Updated weights for policy 1, policy_version 1132506 (0.0010) [2023-12-26 23:36:48,359][105620] Updated weights for policy 1, policy_version 1132516 (0.0008) [2023-12-26 23:36:48,418][105620] Updated weights for policy 1, policy_version 1132526 (0.0011) [2023-12-26 23:36:48,972][105692] Updated weights for policy 0, policy_version 1131250 (0.0007) [2023-12-26 23:36:49,017][105692] Updated weights for policy 0, policy_version 1131260 (0.0008) [2023-12-26 23:36:49,061][105692] Updated weights for policy 0, policy_version 1131270 (0.0008) [2023-12-26 23:36:49,104][105692] Updated weights for policy 0, policy_version 1131280 (0.0007) [2023-12-26 23:36:49,161][105620] Updated weights for policy 1, policy_version 1132536 (0.0010) [2023-12-26 23:36:49,205][105620] Updated weights for policy 1, policy_version 1132546 (0.0010) [2023-12-26 23:36:49,269][105620] Updated weights for policy 1, policy_version 1132556 (0.0011) [2023-12-26 23:36:49,912][105692] Updated weights for policy 0, policy_version 1131290 (0.0008) [2023-12-26 23:36:49,976][105692] Updated weights for policy 0, policy_version 1131300 (0.0007) [2023-12-26 23:36:50,042][105692] Updated weights for policy 0, policy_version 1131310 (0.0007) [2023-12-26 23:36:50,054][105620] Updated weights for policy 1, policy_version 1132566 (0.0010) [2023-12-26 23:36:50,111][105620] Updated weights for policy 1, policy_version 1132576 (0.0010) [2023-12-26 23:36:50,168][105620] Updated weights for policy 1, policy_version 1132586 (0.0011) [2023-12-26 23:36:50,821][105692] Updated weights for policy 0, policy_version 1131320 (0.0007) [2023-12-26 23:36:50,828][105620] Updated weights for policy 1, policy_version 1132596 (0.0009) [2023-12-26 23:36:50,880][105692] Updated weights for policy 0, policy_version 1131330 (0.0006) [2023-12-26 23:36:50,885][105620] Updated weights for policy 1, policy_version 1132606 (0.0011) [2023-12-26 23:36:50,944][105692] Updated weights for policy 0, policy_version 1131340 (0.0005) [2023-12-26 23:36:50,955][105620] Updated weights for policy 1, policy_version 1132616 (0.0011) [2023-12-26 23:36:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 579657728. Throughput: 0: 9991.9, 1: 9762.3. Samples: 579641040. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:36:51,062][104569] Avg episode reward: [(0, '8986.435'), (1, '9166.004')] [2023-12-26 23:36:51,638][105692] Updated weights for policy 0, policy_version 1131350 (0.0007) [2023-12-26 23:36:51,672][105620] Updated weights for policy 1, policy_version 1132626 (0.0007) [2023-12-26 23:36:51,698][105692] Updated weights for policy 0, policy_version 1131360 (0.0006) [2023-12-26 23:36:51,735][105620] Updated weights for policy 1, policy_version 1132636 (0.0008) [2023-12-26 23:36:51,770][105692] Updated weights for policy 0, policy_version 1131370 (0.0008) [2023-12-26 23:36:51,797][105620] Updated weights for policy 1, policy_version 1132646 (0.0007) [2023-12-26 23:36:51,862][105620] Updated weights for policy 1, policy_version 1132656 (0.0009) [2023-12-26 23:36:52,425][105692] Updated weights for policy 0, policy_version 1131380 (0.0008) [2023-12-26 23:36:52,483][105692] Updated weights for policy 0, policy_version 1131390 (0.0009) [2023-12-26 23:36:52,545][105692] Updated weights for policy 0, policy_version 1131400 (0.0009) [2023-12-26 23:36:52,652][105620] Updated weights for policy 1, policy_version 1132666 (0.0010) [2023-12-26 23:36:52,707][105620] Updated weights for policy 1, policy_version 1132676 (0.0010) [2023-12-26 23:36:52,760][105620] Updated weights for policy 1, policy_version 1132686 (0.0008) [2023-12-26 23:36:53,238][105692] Updated weights for policy 0, policy_version 1131410 (0.0008) [2023-12-26 23:36:53,291][105692] Updated weights for policy 0, policy_version 1131420 (0.0008) [2023-12-26 23:36:53,348][105692] Updated weights for policy 0, policy_version 1131430 (0.0009) [2023-12-26 23:36:53,402][105692] Updated weights for policy 0, policy_version 1131440 (0.0009) [2023-12-26 23:36:53,512][105620] Updated weights for policy 1, policy_version 1132696 (0.0009) [2023-12-26 23:36:53,570][105620] Updated weights for policy 1, policy_version 1132706 (0.0008) [2023-12-26 23:36:53,627][105620] Updated weights for policy 1, policy_version 1132716 (0.0009) [2023-12-26 23:36:54,070][105692] Updated weights for policy 0, policy_version 1131450 (0.0005) [2023-12-26 23:36:54,120][105692] Updated weights for policy 0, policy_version 1131460 (0.0009) [2023-12-26 23:36:54,172][105692] Updated weights for policy 0, policy_version 1131470 (0.0009) [2023-12-26 23:36:54,400][105620] Updated weights for policy 1, policy_version 1132726 (0.0009) [2023-12-26 23:36:54,458][105620] Updated weights for policy 1, policy_version 1132736 (0.0009) [2023-12-26 23:36:54,519][105620] Updated weights for policy 1, policy_version 1132746 (0.0008) [2023-12-26 23:36:54,917][105692] Updated weights for policy 0, policy_version 1131480 (0.0006) [2023-12-26 23:36:54,981][105692] Updated weights for policy 0, policy_version 1131490 (0.0009) [2023-12-26 23:36:55,040][105692] Updated weights for policy 0, policy_version 1131500 (0.0009) [2023-12-26 23:36:55,219][105620] Updated weights for policy 1, policy_version 1132756 (0.0007) [2023-12-26 23:36:55,276][105620] Updated weights for policy 1, policy_version 1132766 (0.0006) [2023-12-26 23:36:55,330][105620] Updated weights for policy 1, policy_version 1132776 (0.0008) [2023-12-26 23:36:55,819][105692] Updated weights for policy 0, policy_version 1131510 (0.0009) [2023-12-26 23:36:55,870][105692] Updated weights for policy 0, policy_version 1131520 (0.0008) [2023-12-26 23:36:55,923][105692] Updated weights for policy 0, policy_version 1131530 (0.0006) [2023-12-26 23:36:56,022][105620] Updated weights for policy 1, policy_version 1132786 (0.0006) [2023-12-26 23:36:56,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 579747840. Throughput: 0: 9987.0, 1: 9727.2. Samples: 579756572. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:36:56,064][104569] Avg episode reward: [(0, '9171.673'), (1, '9073.733')] [2023-12-26 23:36:56,079][105620] Updated weights for policy 1, policy_version 1132796 (0.0008) [2023-12-26 23:36:56,145][105620] Updated weights for policy 1, policy_version 1132806 (0.0007) [2023-12-26 23:36:56,204][105620] Updated weights for policy 1, policy_version 1132816 (0.0006) [2023-12-26 23:36:56,611][105692] Updated weights for policy 0, policy_version 1131540 (0.0005) [2023-12-26 23:36:56,665][105692] Updated weights for policy 0, policy_version 1131550 (0.0007) [2023-12-26 23:36:56,722][105692] Updated weights for policy 0, policy_version 1131560 (0.0007) [2023-12-26 23:36:56,803][105620] Updated weights for policy 1, policy_version 1132826 (0.0005) [2023-12-26 23:36:56,848][105620] Updated weights for policy 1, policy_version 1132836 (0.0005) [2023-12-26 23:36:56,897][105620] Updated weights for policy 1, policy_version 1132846 (0.0005) [2023-12-26 23:36:57,425][105620] Updated weights for policy 1, policy_version 1132856 (0.0006) [2023-12-26 23:36:57,445][105692] Updated weights for policy 0, policy_version 1131570 (0.0008) [2023-12-26 23:36:57,486][105620] Updated weights for policy 1, policy_version 1132866 (0.0007) [2023-12-26 23:36:57,496][105692] Updated weights for policy 0, policy_version 1131580 (0.0010) [2023-12-26 23:36:57,539][105620] Updated weights for policy 1, policy_version 1132876 (0.0010) [2023-12-26 23:36:57,542][105692] Updated weights for policy 0, policy_version 1131590 (0.0009) [2023-12-26 23:36:57,595][105692] Updated weights for policy 0, policy_version 1131600 (0.0008) [2023-12-26 23:36:58,195][105620] Updated weights for policy 1, policy_version 1132886 (0.0009) [2023-12-26 23:36:58,257][105620] Updated weights for policy 1, policy_version 1132896 (0.0010) [2023-12-26 23:36:58,264][105692] Updated weights for policy 0, policy_version 1131610 (0.0008) [2023-12-26 23:36:58,316][105620] Updated weights for policy 1, policy_version 1132906 (0.0010) [2023-12-26 23:36:58,323][105692] Updated weights for policy 0, policy_version 1131620 (0.0009) [2023-12-26 23:36:58,382][105692] Updated weights for policy 0, policy_version 1131630 (0.0007) [2023-12-26 23:36:59,135][105620] Updated weights for policy 1, policy_version 1132916 (0.0009) [2023-12-26 23:36:59,192][105692] Updated weights for policy 0, policy_version 1131640 (0.0006) [2023-12-26 23:36:59,197][105620] Updated weights for policy 1, policy_version 1132926 (0.0009) [2023-12-26 23:36:59,262][105692] Updated weights for policy 0, policy_version 1131650 (0.0009) [2023-12-26 23:36:59,273][105620] Updated weights for policy 1, policy_version 1132936 (0.0009) [2023-12-26 23:36:59,324][105692] Updated weights for policy 0, policy_version 1131660 (0.0007) [2023-12-26 23:36:59,881][105620] Updated weights for policy 1, policy_version 1132946 (0.0007) [2023-12-26 23:36:59,946][105620] Updated weights for policy 1, policy_version 1132956 (0.0008) [2023-12-26 23:37:00,001][105620] Updated weights for policy 1, policy_version 1132966 (0.0008) [2023-12-26 23:37:00,060][105620] Updated weights for policy 1, policy_version 1132976 (0.0008) [2023-12-26 23:37:00,098][105692] Updated weights for policy 0, policy_version 1131670 (0.0008) [2023-12-26 23:37:00,153][105692] Updated weights for policy 0, policy_version 1131680 (0.0007) [2023-12-26 23:37:00,208][105692] Updated weights for policy 0, policy_version 1131690 (0.0008) [2023-12-26 23:37:00,761][105620] Updated weights for policy 1, policy_version 1132986 (0.0008) [2023-12-26 23:37:00,820][105620] Updated weights for policy 1, policy_version 1132996 (0.0008) [2023-12-26 23:37:00,879][105620] Updated weights for policy 1, policy_version 1133006 (0.0009) [2023-12-26 23:37:00,980][105692] Updated weights for policy 0, policy_version 1131700 (0.0007) [2023-12-26 23:37:01,032][105692] Updated weights for policy 0, policy_version 1131710 (0.0006) [2023-12-26 23:37:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 579846144. Throughput: 0: 9974.8, 1: 9815.1. Samples: 579818008. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:37:01,063][104569] Avg episode reward: [(0, '9263.562'), (1, '9254.416')] [2023-12-26 23:37:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001133008_290086912.pth... [2023-12-26 23:37:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001131824_289783808.pth [2023-12-26 23:37:01,101][105692] Updated weights for policy 0, policy_version 1131720 (0.0009) [2023-12-26 23:37:01,154][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001131728_289767424.pth... [2023-12-26 23:37:01,159][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001130576_289472512.pth [2023-12-26 23:37:01,576][105620] Updated weights for policy 1, policy_version 1133016 (0.0010) [2023-12-26 23:37:01,642][105620] Updated weights for policy 1, policy_version 1133026 (0.0009) [2023-12-26 23:37:01,694][105620] Updated weights for policy 1, policy_version 1133036 (0.0009) [2023-12-26 23:37:01,831][105692] Updated weights for policy 0, policy_version 1131730 (0.0009) [2023-12-26 23:37:01,881][105692] Updated weights for policy 0, policy_version 1131740 (0.0008) [2023-12-26 23:37:01,927][105692] Updated weights for policy 0, policy_version 1131750 (0.0008) [2023-12-26 23:37:01,978][105692] Updated weights for policy 0, policy_version 1131760 (0.0009) [2023-12-26 23:37:02,447][105620] Updated weights for policy 1, policy_version 1133046 (0.0007) [2023-12-26 23:37:02,507][105620] Updated weights for policy 1, policy_version 1133056 (0.0006) [2023-12-26 23:37:02,560][105620] Updated weights for policy 1, policy_version 1133066 (0.0008) [2023-12-26 23:37:02,798][105692] Updated weights for policy 0, policy_version 1131771 (0.0010) [2023-12-26 23:37:02,850][105692] Updated weights for policy 0, policy_version 1131781 (0.0008) [2023-12-26 23:37:02,912][105692] Updated weights for policy 0, policy_version 1131791 (0.0005) [2023-12-26 23:37:03,140][105620] Updated weights for policy 1, policy_version 1133076 (0.0009) [2023-12-26 23:37:03,202][105620] Updated weights for policy 1, policy_version 1133086 (0.0010) [2023-12-26 23:37:03,263][105620] Updated weights for policy 1, policy_version 1133096 (0.0009) [2023-12-26 23:37:03,519][105692] Updated weights for policy 0, policy_version 1131801 (0.0008) [2023-12-26 23:37:03,565][105692] Updated weights for policy 0, policy_version 1131811 (0.0009) [2023-12-26 23:37:03,616][105692] Updated weights for policy 0, policy_version 1131821 (0.0009) [2023-12-26 23:37:04,020][105620] Updated weights for policy 1, policy_version 1133106 (0.0009) [2023-12-26 23:37:04,084][105620] Updated weights for policy 1, policy_version 1133116 (0.0009) [2023-12-26 23:37:04,145][105620] Updated weights for policy 1, policy_version 1133126 (0.0009) [2023-12-26 23:37:04,208][105620] Updated weights for policy 1, policy_version 1133136 (0.0009) [2023-12-26 23:37:04,400][105692] Updated weights for policy 0, policy_version 1131831 (0.0009) [2023-12-26 23:37:04,462][105692] Updated weights for policy 0, policy_version 1131841 (0.0009) [2023-12-26 23:37:04,530][105692] Updated weights for policy 0, policy_version 1131851 (0.0008) [2023-12-26 23:37:04,970][105620] Updated weights for policy 1, policy_version 1133146 (0.0010) [2023-12-26 23:37:05,024][105620] Updated weights for policy 1, policy_version 1133156 (0.0010) [2023-12-26 23:37:05,077][105620] Updated weights for policy 1, policy_version 1133166 (0.0009) [2023-12-26 23:37:05,202][105692] Updated weights for policy 0, policy_version 1131861 (0.0007) [2023-12-26 23:37:05,261][105692] Updated weights for policy 0, policy_version 1131871 (0.0005) [2023-12-26 23:37:05,314][105692] Updated weights for policy 0, policy_version 1131881 (0.0005) [2023-12-26 23:37:05,809][105620] Updated weights for policy 1, policy_version 1133176 (0.0006) [2023-12-26 23:37:05,868][105620] Updated weights for policy 1, policy_version 1133186 (0.0008) [2023-12-26 23:37:05,913][105620] Updated weights for policy 1, policy_version 1133196 (0.0008) [2023-12-26 23:37:05,981][105692] Updated weights for policy 0, policy_version 1131891 (0.0006) [2023-12-26 23:37:06,040][105692] Updated weights for policy 0, policy_version 1131901 (0.0009) [2023-12-26 23:37:06,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 579944448. Throughput: 0: 9826.6, 1: 9818.5. Samples: 579933136. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:37:06,063][104569] Avg episode reward: [(0, '9077.711'), (1, '9343.583')] [2023-12-26 23:37:06,113][105692] Updated weights for policy 0, policy_version 1131911 (0.0009) [2023-12-26 23:37:06,712][105620] Updated weights for policy 1, policy_version 1133206 (0.0008) [2023-12-26 23:37:06,777][105620] Updated weights for policy 1, policy_version 1133216 (0.0009) [2023-12-26 23:37:06,838][105620] Updated weights for policy 1, policy_version 1133226 (0.0008) [2023-12-26 23:37:06,871][105692] Updated weights for policy 0, policy_version 1131921 (0.0008) [2023-12-26 23:37:06,934][105692] Updated weights for policy 0, policy_version 1131931 (0.0009) [2023-12-26 23:37:06,996][105692] Updated weights for policy 0, policy_version 1131941 (0.0009) [2023-12-26 23:37:07,056][105692] Updated weights for policy 0, policy_version 1131951 (0.0009) [2023-12-26 23:37:07,584][105620] Updated weights for policy 1, policy_version 1133236 (0.0009) [2023-12-26 23:37:07,632][105620] Updated weights for policy 1, policy_version 1133246 (0.0008) [2023-12-26 23:37:07,687][105620] Updated weights for policy 1, policy_version 1133256 (0.0008) [2023-12-26 23:37:07,822][105692] Updated weights for policy 0, policy_version 1131961 (0.0010) [2023-12-26 23:37:07,877][105692] Updated weights for policy 0, policy_version 1131971 (0.0010) [2023-12-26 23:37:07,932][105692] Updated weights for policy 0, policy_version 1131981 (0.0010) [2023-12-26 23:37:08,459][105620] Updated weights for policy 1, policy_version 1133266 (0.0008) [2023-12-26 23:37:08,522][105620] Updated weights for policy 1, policy_version 1133276 (0.0008) [2023-12-26 23:37:08,578][105620] Updated weights for policy 1, policy_version 1133286 (0.0008) [2023-12-26 23:37:08,638][105620] Updated weights for policy 1, policy_version 1133296 (0.0008) [2023-12-26 23:37:08,701][105692] Updated weights for policy 0, policy_version 1131991 (0.0010) [2023-12-26 23:37:08,753][105692] Updated weights for policy 0, policy_version 1132001 (0.0010) [2023-12-26 23:37:08,812][105692] Updated weights for policy 0, policy_version 1132011 (0.0010) [2023-12-26 23:37:09,410][105620] Updated weights for policy 1, policy_version 1133306 (0.0008) [2023-12-26 23:37:09,474][105620] Updated weights for policy 1, policy_version 1133316 (0.0008) [2023-12-26 23:37:09,526][105620] Updated weights for policy 1, policy_version 1133326 (0.0008) [2023-12-26 23:37:09,576][105692] Updated weights for policy 0, policy_version 1132021 (0.0011) [2023-12-26 23:37:09,639][105692] Updated weights for policy 0, policy_version 1132031 (0.0010) [2023-12-26 23:37:09,704][105692] Updated weights for policy 0, policy_version 1132041 (0.0010) [2023-12-26 23:37:10,310][105620] Updated weights for policy 1, policy_version 1133336 (0.0009) [2023-12-26 23:37:10,365][105620] Updated weights for policy 1, policy_version 1133346 (0.0010) [2023-12-26 23:37:10,420][105620] Updated weights for policy 1, policy_version 1133356 (0.0010) [2023-12-26 23:37:10,431][105692] Updated weights for policy 0, policy_version 1132051 (0.0009) [2023-12-26 23:37:10,490][105692] Updated weights for policy 0, policy_version 1132061 (0.0006) [2023-12-26 23:37:10,551][105692] Updated weights for policy 0, policy_version 1132071 (0.0011) [2023-12-26 23:37:11,009][105620] Updated weights for policy 1, policy_version 1133366 (0.0008) [2023-12-26 23:37:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 580034560. Throughput: 0: 9759.8, 1: 9838.1. Samples: 580046980. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:37:11,063][104569] Avg episode reward: [(0, '9076.559'), (1, '9344.012')] [2023-12-26 23:37:11,078][105620] Updated weights for policy 1, policy_version 1133376 (0.0011) [2023-12-26 23:37:11,148][105620] Updated weights for policy 1, policy_version 1133386 (0.0012) [2023-12-26 23:37:11,198][105692] Updated weights for policy 0, policy_version 1132081 (0.0011) [2023-12-26 23:37:11,255][105692] Updated weights for policy 0, policy_version 1132091 (0.0011) [2023-12-26 23:37:11,319][105692] Updated weights for policy 0, policy_version 1132101 (0.0008) [2023-12-26 23:37:11,388][105692] Updated weights for policy 0, policy_version 1132111 (0.0012) [2023-12-26 23:37:11,905][105620] Updated weights for policy 1, policy_version 1133396 (0.0008) [2023-12-26 23:37:11,967][105620] Updated weights for policy 1, policy_version 1133406 (0.0008) [2023-12-26 23:37:12,029][105620] Updated weights for policy 1, policy_version 1133416 (0.0010) [2023-12-26 23:37:12,137][105692] Updated weights for policy 0, policy_version 1132121 (0.0006) [2023-12-26 23:37:12,200][105692] Updated weights for policy 0, policy_version 1132131 (0.0005) [2023-12-26 23:37:12,258][105692] Updated weights for policy 0, policy_version 1132141 (0.0006) [2023-12-26 23:37:12,790][105620] Updated weights for policy 1, policy_version 1133426 (0.0009) [2023-12-26 23:37:12,852][105620] Updated weights for policy 1, policy_version 1133436 (0.0008) [2023-12-26 23:37:12,921][105620] Updated weights for policy 1, policy_version 1133446 (0.0008) [2023-12-26 23:37:12,945][105692] Updated weights for policy 0, policy_version 1132151 (0.0009) [2023-12-26 23:37:12,979][105620] Updated weights for policy 1, policy_version 1133456 (0.0009) [2023-12-26 23:37:12,996][105692] Updated weights for policy 0, policy_version 1132161 (0.0010) [2023-12-26 23:37:13,044][105692] Updated weights for policy 0, policy_version 1132171 (0.0010) [2023-12-26 23:37:13,709][105620] Updated weights for policy 1, policy_version 1133466 (0.0008) [2023-12-26 23:37:13,768][105620] Updated weights for policy 1, policy_version 1133476 (0.0008) [2023-12-26 23:37:13,806][105692] Updated weights for policy 0, policy_version 1132181 (0.0010) [2023-12-26 23:37:13,817][105620] Updated weights for policy 1, policy_version 1133486 (0.0006) [2023-12-26 23:37:13,861][105692] Updated weights for policy 0, policy_version 1132191 (0.0010) [2023-12-26 23:37:13,922][105692] Updated weights for policy 0, policy_version 1132201 (0.0010) [2023-12-26 23:37:14,572][105620] Updated weights for policy 1, policy_version 1133496 (0.0007) [2023-12-26 23:37:14,624][105620] Updated weights for policy 1, policy_version 1133506 (0.0008) [2023-12-26 23:37:14,652][105692] Updated weights for policy 0, policy_version 1132211 (0.0010) [2023-12-26 23:37:14,681][105620] Updated weights for policy 1, policy_version 1133516 (0.0006) [2023-12-26 23:37:14,712][105692] Updated weights for policy 0, policy_version 1132221 (0.0010) [2023-12-26 23:37:14,781][105692] Updated weights for policy 0, policy_version 1132231 (0.0010) [2023-12-26 23:37:15,464][105620] Updated weights for policy 1, policy_version 1133526 (0.0008) [2023-12-26 23:37:15,524][105620] Updated weights for policy 1, policy_version 1133536 (0.0008) [2023-12-26 23:37:15,533][105692] Updated weights for policy 0, policy_version 1132241 (0.0010) [2023-12-26 23:37:15,583][105620] Updated weights for policy 1, policy_version 1133546 (0.0006) [2023-12-26 23:37:15,584][105692] Updated weights for policy 0, policy_version 1132251 (0.0010) [2023-12-26 23:37:15,642][105692] Updated weights for policy 0, policy_version 1132261 (0.0010) [2023-12-26 23:37:15,697][105692] Updated weights for policy 0, policy_version 1132271 (0.0010) [2023-12-26 23:37:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 580132864. Throughput: 0: 9685.8, 1: 9815.6. Samples: 580104088. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:37:16,062][104569] Avg episode reward: [(0, '9167.121'), (1, '9347.397')] [2023-12-26 23:37:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001133552_290226176.pth... [2023-12-26 23:37:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001132272_289906688.pth... [2023-12-26 23:37:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001132432_289939456.pth [2023-12-26 23:37:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001131152_289619968.pth [2023-12-26 23:37:16,344][105620] Updated weights for policy 1, policy_version 1133556 (0.0007) [2023-12-26 23:37:16,407][105620] Updated weights for policy 1, policy_version 1133566 (0.0009) [2023-12-26 23:37:16,442][105692] Updated weights for policy 0, policy_version 1132281 (0.0006) [2023-12-26 23:37:16,462][105620] Updated weights for policy 1, policy_version 1133576 (0.0007) [2023-12-26 23:37:16,495][105692] Updated weights for policy 0, policy_version 1132291 (0.0009) [2023-12-26 23:37:16,556][105692] Updated weights for policy 0, policy_version 1132301 (0.0008) [2023-12-26 23:37:17,218][105620] Updated weights for policy 1, policy_version 1133586 (0.0008) [2023-12-26 23:37:17,271][105620] Updated weights for policy 1, policy_version 1133596 (0.0008) [2023-12-26 23:37:17,313][105620] Updated weights for policy 1, policy_version 1133606 (0.0006) [2023-12-26 23:37:17,315][105692] Updated weights for policy 0, policy_version 1132311 (0.0008) [2023-12-26 23:37:17,360][105692] Updated weights for policy 0, policy_version 1132321 (0.0005) [2023-12-26 23:37:17,366][105620] Updated weights for policy 1, policy_version 1133616 (0.0007) [2023-12-26 23:37:17,413][105692] Updated weights for policy 0, policy_version 1132331 (0.0009) [2023-12-26 23:37:18,113][105620] Updated weights for policy 1, policy_version 1133626 (0.0008) [2023-12-26 23:37:18,166][105620] Updated weights for policy 1, policy_version 1133636 (0.0008) [2023-12-26 23:37:18,175][105692] Updated weights for policy 0, policy_version 1132341 (0.0010) [2023-12-26 23:37:18,210][105620] Updated weights for policy 1, policy_version 1133646 (0.0008) [2023-12-26 23:37:18,233][105692] Updated weights for policy 0, policy_version 1132351 (0.0010) [2023-12-26 23:37:18,285][105692] Updated weights for policy 0, policy_version 1132361 (0.0010) [2023-12-26 23:37:18,984][105620] Updated weights for policy 1, policy_version 1133656 (0.0009) [2023-12-26 23:37:19,028][105620] Updated weights for policy 1, policy_version 1133666 (0.0008) [2023-12-26 23:37:19,055][105692] Updated weights for policy 0, policy_version 1132371 (0.0010) [2023-12-26 23:37:19,073][105620] Updated weights for policy 1, policy_version 1133676 (0.0007) [2023-12-26 23:37:19,113][105692] Updated weights for policy 0, policy_version 1132381 (0.0010) [2023-12-26 23:37:19,170][105692] Updated weights for policy 0, policy_version 1132391 (0.0010) [2023-12-26 23:37:19,901][105620] Updated weights for policy 1, policy_version 1133686 (0.0009) [2023-12-26 23:37:19,910][105692] Updated weights for policy 0, policy_version 1132401 (0.0008) [2023-12-26 23:37:19,961][105620] Updated weights for policy 1, policy_version 1133696 (0.0008) [2023-12-26 23:37:19,970][105692] Updated weights for policy 0, policy_version 1132411 (0.0008) [2023-12-26 23:37:20,019][105692] Updated weights for policy 0, policy_version 1132421 (0.0008) [2023-12-26 23:37:20,023][105620] Updated weights for policy 1, policy_version 1133706 (0.0007) [2023-12-26 23:37:20,071][105692] Updated weights for policy 0, policy_version 1132431 (0.0008) [2023-12-26 23:37:20,709][105620] Updated weights for policy 1, policy_version 1133716 (0.0008) [2023-12-26 23:37:20,772][105620] Updated weights for policy 1, policy_version 1133726 (0.0009) [2023-12-26 23:37:20,826][105620] Updated weights for policy 1, policy_version 1133736 (0.0008) [2023-12-26 23:37:20,905][105692] Updated weights for policy 0, policy_version 1132441 (0.0009) [2023-12-26 23:37:20,968][105692] Updated weights for policy 0, policy_version 1132451 (0.0009) [2023-12-26 23:37:21,031][105692] Updated weights for policy 0, policy_version 1132461 (0.0009) [2023-12-26 23:37:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 580231168. Throughput: 0: 9553.6, 1: 9766.6. Samples: 580215664. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:37:21,063][104569] Avg episode reward: [(0, '9166.523'), (1, '9257.856')] [2023-12-26 23:37:21,591][105620] Updated weights for policy 1, policy_version 1133746 (0.0008) [2023-12-26 23:37:21,655][105620] Updated weights for policy 1, policy_version 1133756 (0.0008) [2023-12-26 23:37:21,717][105620] Updated weights for policy 1, policy_version 1133766 (0.0008) [2023-12-26 23:37:21,783][105620] Updated weights for policy 1, policy_version 1133776 (0.0008) [2023-12-26 23:37:21,833][105692] Updated weights for policy 0, policy_version 1132471 (0.0006) [2023-12-26 23:37:21,887][105692] Updated weights for policy 0, policy_version 1132481 (0.0007) [2023-12-26 23:37:21,942][105692] Updated weights for policy 0, policy_version 1132491 (0.0009) [2023-12-26 23:37:22,480][105620] Updated weights for policy 1, policy_version 1133786 (0.0009) [2023-12-26 23:37:22,538][105620] Updated weights for policy 1, policy_version 1133796 (0.0009) [2023-12-26 23:37:22,587][105620] Updated weights for policy 1, policy_version 1133806 (0.0008) [2023-12-26 23:37:22,706][105692] Updated weights for policy 0, policy_version 1132501 (0.0009) [2023-12-26 23:37:22,765][105692] Updated weights for policy 0, policy_version 1132511 (0.0009) [2023-12-26 23:37:22,825][105692] Updated weights for policy 0, policy_version 1132521 (0.0009) [2023-12-26 23:37:23,368][105620] Updated weights for policy 1, policy_version 1133816 (0.0006) [2023-12-26 23:37:23,421][105620] Updated weights for policy 1, policy_version 1133826 (0.0005) [2023-12-26 23:37:23,488][105620] Updated weights for policy 1, policy_version 1133836 (0.0005) [2023-12-26 23:37:23,564][105692] Updated weights for policy 0, policy_version 1132531 (0.0009) [2023-12-26 23:37:23,613][105692] Updated weights for policy 0, policy_version 1132541 (0.0009) [2023-12-26 23:37:23,676][105692] Updated weights for policy 0, policy_version 1132551 (0.0009) [2023-12-26 23:37:24,075][105620] Updated weights for policy 1, policy_version 1133846 (0.0008) [2023-12-26 23:37:24,125][105620] Updated weights for policy 1, policy_version 1133856 (0.0008) [2023-12-26 23:37:24,179][105620] Updated weights for policy 1, policy_version 1133866 (0.0009) [2023-12-26 23:37:24,521][105692] Updated weights for policy 0, policy_version 1132561 (0.0010) [2023-12-26 23:37:24,573][105692] Updated weights for policy 0, policy_version 1132572 (0.0009) [2023-12-26 23:37:24,627][105692] Updated weights for policy 0, policy_version 1132582 (0.0009) [2023-12-26 23:37:24,685][105692] Updated weights for policy 0, policy_version 1132592 (0.0009) [2023-12-26 23:37:24,814][105620] Updated weights for policy 1, policy_version 1133876 (0.0008) [2023-12-26 23:37:24,877][105620] Updated weights for policy 1, policy_version 1133886 (0.0009) [2023-12-26 23:37:24,945][105620] Updated weights for policy 1, policy_version 1133896 (0.0009) [2023-12-26 23:37:25,450][105692] Updated weights for policy 0, policy_version 1132602 (0.0009) [2023-12-26 23:37:25,513][105692] Updated weights for policy 0, policy_version 1132612 (0.0009) [2023-12-26 23:37:25,538][105620] Updated weights for policy 1, policy_version 1133906 (0.0009) [2023-12-26 23:37:25,568][105692] Updated weights for policy 0, policy_version 1132622 (0.0008) [2023-12-26 23:37:25,583][105620] Updated weights for policy 1, policy_version 1133916 (0.0007) [2023-12-26 23:37:25,634][105620] Updated weights for policy 1, policy_version 1133926 (0.0009) [2023-12-26 23:37:26,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 580321280. Throughput: 0: 9476.1, 1: 9713.2. Samples: 580329716. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:37:26,063][104569] Avg episode reward: [(0, '9258.118'), (1, '9165.095')] [2023-12-26 23:37:26,332][105620] Updated weights for policy 1, policy_version 1133937 (0.0007) [2023-12-26 23:37:26,338][105692] Updated weights for policy 0, policy_version 1132632 (0.0006) [2023-12-26 23:37:26,393][105692] Updated weights for policy 0, policy_version 1132642 (0.0006) [2023-12-26 23:37:26,395][105620] Updated weights for policy 1, policy_version 1133947 (0.0008) [2023-12-26 23:37:26,440][105692] Updated weights for policy 0, policy_version 1132652 (0.0008) [2023-12-26 23:37:26,452][105620] Updated weights for policy 1, policy_version 1133957 (0.0006) [2023-12-26 23:37:26,513][105620] Updated weights for policy 1, policy_version 1133967 (0.0006) [2023-12-26 23:37:27,034][105620] Updated weights for policy 1, policy_version 1133977 (0.0009) [2023-12-26 23:37:27,081][105620] Updated weights for policy 1, policy_version 1133987 (0.0009) [2023-12-26 23:37:27,135][105620] Updated weights for policy 1, policy_version 1133997 (0.0009) [2023-12-26 23:37:27,244][105692] Updated weights for policy 0, policy_version 1132662 (0.0009) [2023-12-26 23:37:27,295][105692] Updated weights for policy 0, policy_version 1132672 (0.0009) [2023-12-26 23:37:27,353][105692] Updated weights for policy 0, policy_version 1132682 (0.0007) [2023-12-26 23:37:27,872][105620] Updated weights for policy 1, policy_version 1134007 (0.0006) [2023-12-26 23:37:27,932][105620] Updated weights for policy 1, policy_version 1134017 (0.0005) [2023-12-26 23:37:27,983][105620] Updated weights for policy 1, policy_version 1134027 (0.0005) [2023-12-26 23:37:28,078][105692] Updated weights for policy 0, policy_version 1132692 (0.0006) [2023-12-26 23:37:28,124][105692] Updated weights for policy 0, policy_version 1132702 (0.0009) [2023-12-26 23:37:28,171][105692] Updated weights for policy 0, policy_version 1132712 (0.0008) [2023-12-26 23:37:28,582][105620] Updated weights for policy 1, policy_version 1134037 (0.0006) [2023-12-26 23:37:28,638][105620] Updated weights for policy 1, policy_version 1134047 (0.0005) [2023-12-26 23:37:28,710][105620] Updated weights for policy 1, policy_version 1134057 (0.0005) [2023-12-26 23:37:28,964][105692] Updated weights for policy 0, policy_version 1132722 (0.0008) [2023-12-26 23:37:29,022][105692] Updated weights for policy 0, policy_version 1132732 (0.0005) [2023-12-26 23:37:29,078][105692] Updated weights for policy 0, policy_version 1132742 (0.0005) [2023-12-26 23:37:29,135][105692] Updated weights for policy 0, policy_version 1132752 (0.0008) [2023-12-26 23:37:29,430][105620] Updated weights for policy 1, policy_version 1134067 (0.0008) [2023-12-26 23:37:29,479][105620] Updated weights for policy 1, policy_version 1134077 (0.0005) [2023-12-26 23:37:29,539][105620] Updated weights for policy 1, policy_version 1134087 (0.0006) [2023-12-26 23:37:29,864][105692] Updated weights for policy 0, policy_version 1132762 (0.0008) [2023-12-26 23:37:29,927][105692] Updated weights for policy 0, policy_version 1132772 (0.0006) [2023-12-26 23:37:29,995][105692] Updated weights for policy 0, policy_version 1132782 (0.0006) [2023-12-26 23:37:30,282][105620] Updated weights for policy 1, policy_version 1134097 (0.0007) [2023-12-26 23:37:30,339][105620] Updated weights for policy 1, policy_version 1134107 (0.0009) [2023-12-26 23:37:30,395][105620] Updated weights for policy 1, policy_version 1134117 (0.0008) [2023-12-26 23:37:30,456][105620] Updated weights for policy 1, policy_version 1134127 (0.0009) [2023-12-26 23:37:30,579][105692] Updated weights for policy 0, policy_version 1132792 (0.0009) [2023-12-26 23:37:30,623][105692] Updated weights for policy 0, policy_version 1132802 (0.0010) [2023-12-26 23:37:30,667][105692] Updated weights for policy 0, policy_version 1132812 (0.0010) [2023-12-26 23:37:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 580419584. Throughput: 0: 9430.6, 1: 9827.9. Samples: 580390044. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:37:31,062][104569] Avg episode reward: [(0, '9166.408'), (1, '8888.653')] [2023-12-26 23:37:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001132816_290045952.pth... [2023-12-26 23:37:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001134128_290373632.pth... [2023-12-26 23:37:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001131728_289767424.pth [2023-12-26 23:37:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001133008_290086912.pth [2023-12-26 23:37:31,212][105620] Updated weights for policy 1, policy_version 1134137 (0.0007) [2023-12-26 23:37:31,276][105620] Updated weights for policy 1, policy_version 1134147 (0.0007) [2023-12-26 23:37:31,335][105620] Updated weights for policy 1, policy_version 1134157 (0.0008) [2023-12-26 23:37:31,439][105692] Updated weights for policy 0, policy_version 1132822 (0.0010) [2023-12-26 23:37:31,494][105692] Updated weights for policy 0, policy_version 1132832 (0.0009) [2023-12-26 23:37:31,552][105692] Updated weights for policy 0, policy_version 1132842 (0.0010) [2023-12-26 23:37:32,032][105620] Updated weights for policy 1, policy_version 1134167 (0.0008) [2023-12-26 23:37:32,091][105620] Updated weights for policy 1, policy_version 1134177 (0.0008) [2023-12-26 23:37:32,150][105620] Updated weights for policy 1, policy_version 1134187 (0.0008) [2023-12-26 23:37:32,288][105692] Updated weights for policy 0, policy_version 1132852 (0.0008) [2023-12-26 23:37:32,353][105692] Updated weights for policy 0, policy_version 1132862 (0.0010) [2023-12-26 23:37:32,367][105585] KL-divergence is very high: 167.9328 [2023-12-26 23:37:32,418][105692] Updated weights for policy 0, policy_version 1132872 (0.0010) [2023-12-26 23:37:32,420][105585] KL-divergence is very high: 177.8969 [2023-12-26 23:37:32,911][105620] Updated weights for policy 1, policy_version 1134197 (0.0008) [2023-12-26 23:37:32,970][105620] Updated weights for policy 1, policy_version 1134207 (0.0009) [2023-12-26 23:37:33,019][105620] Updated weights for policy 1, policy_version 1134217 (0.0007) [2023-12-26 23:37:33,147][105692] Updated weights for policy 0, policy_version 1132882 (0.0009) [2023-12-26 23:37:33,204][105692] Updated weights for policy 0, policy_version 1132892 (0.0008) [2023-12-26 23:37:33,270][105692] Updated weights for policy 0, policy_version 1132902 (0.0010) [2023-12-26 23:37:33,331][105692] Updated weights for policy 0, policy_version 1132912 (0.0010) [2023-12-26 23:37:33,805][105620] Updated weights for policy 1, policy_version 1134227 (0.0008) [2023-12-26 23:37:33,865][105620] Updated weights for policy 1, policy_version 1134237 (0.0008) [2023-12-26 23:37:33,871][105692] Updated weights for policy 0, policy_version 1132922 (0.0007) [2023-12-26 23:37:33,919][105620] Updated weights for policy 1, policy_version 1134247 (0.0005) [2023-12-26 23:37:33,922][105692] Updated weights for policy 0, policy_version 1132932 (0.0010) [2023-12-26 23:37:33,983][105692] Updated weights for policy 0, policy_version 1132942 (0.0010) [2023-12-26 23:37:34,690][105620] Updated weights for policy 1, policy_version 1134257 (0.0006) [2023-12-26 23:37:34,729][105692] Updated weights for policy 0, policy_version 1132952 (0.0010) [2023-12-26 23:37:34,747][105620] Updated weights for policy 1, policy_version 1134267 (0.0010) [2023-12-26 23:37:34,781][105692] Updated weights for policy 0, policy_version 1132962 (0.0011) [2023-12-26 23:37:34,807][105620] Updated weights for policy 1, policy_version 1134277 (0.0011) [2023-12-26 23:37:34,831][105692] Updated weights for policy 0, policy_version 1132972 (0.0011) [2023-12-26 23:37:34,865][105620] Updated weights for policy 1, policy_version 1134287 (0.0010) [2023-12-26 23:37:35,495][105620] Updated weights for policy 1, policy_version 1134297 (0.0007) [2023-12-26 23:37:35,554][105620] Updated weights for policy 1, policy_version 1134307 (0.0011) [2023-12-26 23:37:35,566][105692] Updated weights for policy 0, policy_version 1132982 (0.0007) [2023-12-26 23:37:35,605][105620] Updated weights for policy 1, policy_version 1134317 (0.0010) [2023-12-26 23:37:35,614][105692] Updated weights for policy 0, policy_version 1132992 (0.0010) [2023-12-26 23:37:35,662][105692] Updated weights for policy 0, policy_version 1133002 (0.0010) [2023-12-26 23:37:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 580517888. Throughput: 0: 9510.9, 1: 9705.6. Samples: 580505788. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:37:36,063][104569] Avg episode reward: [(0, '9077.768'), (1, '8979.285')] [2023-12-26 23:37:36,245][105620] Updated weights for policy 1, policy_version 1134327 (0.0007) [2023-12-26 23:37:36,310][105620] Updated weights for policy 1, policy_version 1134337 (0.0005) [2023-12-26 23:37:36,376][105620] Updated weights for policy 1, policy_version 1134347 (0.0007) [2023-12-26 23:37:36,409][105692] Updated weights for policy 0, policy_version 1133012 (0.0010) [2023-12-26 23:37:36,468][105692] Updated weights for policy 0, policy_version 1133022 (0.0010) [2023-12-26 23:37:36,531][105692] Updated weights for policy 0, policy_version 1133032 (0.0011) [2023-12-26 23:37:36,961][105620] Updated weights for policy 1, policy_version 1134357 (0.0011) [2023-12-26 23:37:37,027][105620] Updated weights for policy 1, policy_version 1134367 (0.0008) [2023-12-26 23:37:37,085][105620] Updated weights for policy 1, policy_version 1134377 (0.0008) [2023-12-26 23:37:37,287][105692] Updated weights for policy 0, policy_version 1133042 (0.0011) [2023-12-26 23:37:37,345][105692] Updated weights for policy 0, policy_version 1133052 (0.0010) [2023-12-26 23:37:37,407][105692] Updated weights for policy 0, policy_version 1133062 (0.0010) [2023-12-26 23:37:37,480][105692] Updated weights for policy 0, policy_version 1133072 (0.0010) [2023-12-26 23:37:37,715][105620] Updated weights for policy 1, policy_version 1134387 (0.0007) [2023-12-26 23:37:37,789][105620] Updated weights for policy 1, policy_version 1134397 (0.0007) [2023-12-26 23:37:37,848][105620] Updated weights for policy 1, policy_version 1134407 (0.0008) [2023-12-26 23:37:38,193][105692] Updated weights for policy 0, policy_version 1133082 (0.0006) [2023-12-26 23:37:38,239][105692] Updated weights for policy 0, policy_version 1133092 (0.0005) [2023-12-26 23:37:38,283][105692] Updated weights for policy 0, policy_version 1133102 (0.0007) [2023-12-26 23:37:38,623][105620] Updated weights for policy 1, policy_version 1134417 (0.0009) [2023-12-26 23:37:38,671][105620] Updated weights for policy 1, policy_version 1134427 (0.0010) [2023-12-26 23:37:38,731][105620] Updated weights for policy 1, policy_version 1134437 (0.0008) [2023-12-26 23:37:38,790][105620] Updated weights for policy 1, policy_version 1134447 (0.0010) [2023-12-26 23:37:38,957][105692] Updated weights for policy 0, policy_version 1133112 (0.0007) [2023-12-26 23:37:39,012][105692] Updated weights for policy 0, policy_version 1133122 (0.0006) [2023-12-26 23:37:39,075][105692] Updated weights for policy 0, policy_version 1133132 (0.0008) [2023-12-26 23:37:39,558][105620] Updated weights for policy 1, policy_version 1134457 (0.0011) [2023-12-26 23:37:39,618][105620] Updated weights for policy 1, policy_version 1134467 (0.0011) [2023-12-26 23:37:39,669][105620] Updated weights for policy 1, policy_version 1134477 (0.0010) [2023-12-26 23:37:39,795][105692] Updated weights for policy 0, policy_version 1133142 (0.0009) [2023-12-26 23:37:39,864][105692] Updated weights for policy 0, policy_version 1133152 (0.0008) [2023-12-26 23:37:39,938][105692] Updated weights for policy 0, policy_version 1133162 (0.0010) [2023-12-26 23:37:40,408][105620] Updated weights for policy 1, policy_version 1134487 (0.0009) [2023-12-26 23:37:40,471][105620] Updated weights for policy 1, policy_version 1134497 (0.0010) [2023-12-26 23:37:40,537][105620] Updated weights for policy 1, policy_version 1134507 (0.0009) [2023-12-26 23:37:40,668][105692] Updated weights for policy 0, policy_version 1133172 (0.0009) [2023-12-26 23:37:40,734][105692] Updated weights for policy 0, policy_version 1133182 (0.0010) [2023-12-26 23:37:40,798][105692] Updated weights for policy 0, policy_version 1133192 (0.0007) [2023-12-26 23:37:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 580616192. Throughput: 0: 9498.1, 1: 9769.2. Samples: 580623596. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:37:41,062][104569] Avg episode reward: [(0, '9173.135'), (1, '9073.168')] [2023-12-26 23:37:41,336][105620] Updated weights for policy 1, policy_version 1134517 (0.0009) [2023-12-26 23:37:41,402][105620] Updated weights for policy 1, policy_version 1134527 (0.0009) [2023-12-26 23:37:41,456][105620] Updated weights for policy 1, policy_version 1134537 (0.0011) [2023-12-26 23:37:41,480][105692] Updated weights for policy 0, policy_version 1133202 (0.0009) [2023-12-26 23:37:41,529][105692] Updated weights for policy 0, policy_version 1133212 (0.0005) [2023-12-26 23:37:41,586][105692] Updated weights for policy 0, policy_version 1133222 (0.0009) [2023-12-26 23:37:41,654][105692] Updated weights for policy 0, policy_version 1133232 (0.0011) [2023-12-26 23:37:42,242][105620] Updated weights for policy 1, policy_version 1134547 (0.0011) [2023-12-26 23:37:42,314][105620] Updated weights for policy 1, policy_version 1134557 (0.0010) [2023-12-26 23:37:42,382][105620] Updated weights for policy 1, policy_version 1134567 (0.0008) [2023-12-26 23:37:42,387][105692] Updated weights for policy 0, policy_version 1133242 (0.0008) [2023-12-26 23:37:42,454][105692] Updated weights for policy 0, policy_version 1133252 (0.0006) [2023-12-26 23:37:42,526][105692] Updated weights for policy 0, policy_version 1133262 (0.0006) [2023-12-26 23:37:43,146][105620] Updated weights for policy 1, policy_version 1134577 (0.0009) [2023-12-26 23:37:43,151][105692] Updated weights for policy 0, policy_version 1133272 (0.0006) [2023-12-26 23:37:43,212][105620] Updated weights for policy 1, policy_version 1134587 (0.0007) [2023-12-26 23:37:43,214][105692] Updated weights for policy 0, policy_version 1133282 (0.0007) [2023-12-26 23:37:43,271][105620] Updated weights for policy 1, policy_version 1134597 (0.0006) [2023-12-26 23:37:43,282][105692] Updated weights for policy 0, policy_version 1133292 (0.0008) [2023-12-26 23:37:43,330][105620] Updated weights for policy 1, policy_version 1134607 (0.0006) [2023-12-26 23:37:43,863][105692] Updated weights for policy 0, policy_version 1133302 (0.0006) [2023-12-26 23:37:43,920][105692] Updated weights for policy 0, policy_version 1133312 (0.0007) [2023-12-26 23:37:43,932][105620] Updated weights for policy 1, policy_version 1134617 (0.0008) [2023-12-26 23:37:43,969][105692] Updated weights for policy 0, policy_version 1133322 (0.0007) [2023-12-26 23:37:43,982][105620] Updated weights for policy 1, policy_version 1134627 (0.0008) [2023-12-26 23:37:44,035][105620] Updated weights for policy 1, policy_version 1134637 (0.0005) [2023-12-26 23:37:44,654][105692] Updated weights for policy 0, policy_version 1133332 (0.0007) [2023-12-26 23:37:44,703][105692] Updated weights for policy 0, policy_version 1133342 (0.0005) [2023-12-26 23:37:44,759][105692] Updated weights for policy 0, policy_version 1133352 (0.0006) [2023-12-26 23:37:44,803][105620] Updated weights for policy 1, policy_version 1134647 (0.0007) [2023-12-26 23:37:44,861][105620] Updated weights for policy 1, policy_version 1134657 (0.0008) [2023-12-26 23:37:44,921][105620] Updated weights for policy 1, policy_version 1134667 (0.0008) [2023-12-26 23:37:45,441][105692] Updated weights for policy 0, policy_version 1133362 (0.0007) [2023-12-26 23:37:45,496][105692] Updated weights for policy 0, policy_version 1133372 (0.0005) [2023-12-26 23:37:45,560][105692] Updated weights for policy 0, policy_version 1133382 (0.0005) [2023-12-26 23:37:45,617][105692] Updated weights for policy 0, policy_version 1133392 (0.0005) [2023-12-26 23:37:45,690][105620] Updated weights for policy 1, policy_version 1134677 (0.0009) [2023-12-26 23:37:45,741][105620] Updated weights for policy 1, policy_version 1134687 (0.0010) [2023-12-26 23:37:45,792][105620] Updated weights for policy 1, policy_version 1134697 (0.0010) [2023-12-26 23:37:46,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 580714496. Throughput: 0: 9515.2, 1: 9690.8. Samples: 580682280. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:37:46,063][104569] Avg episode reward: [(0, '9357.360'), (1, '9073.481')] [2023-12-26 23:37:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001133392_290193408.pth... [2023-12-26 23:37:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001134704_290521088.pth... [2023-12-26 23:37:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001132272_289906688.pth [2023-12-26 23:37:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001133552_290226176.pth [2023-12-26 23:37:46,265][105692] Updated weights for policy 0, policy_version 1133402 (0.0008) [2023-12-26 23:37:46,331][105692] Updated weights for policy 0, policy_version 1133412 (0.0008) [2023-12-26 23:37:46,390][105692] Updated weights for policy 0, policy_version 1133422 (0.0008) [2023-12-26 23:37:46,510][105620] Updated weights for policy 1, policy_version 1134707 (0.0010) [2023-12-26 23:37:46,571][105620] Updated weights for policy 1, policy_version 1134717 (0.0010) [2023-12-26 23:37:46,629][105620] Updated weights for policy 1, policy_version 1134727 (0.0010) [2023-12-26 23:37:47,085][105692] Updated weights for policy 0, policy_version 1133432 (0.0009) [2023-12-26 23:37:47,139][105692] Updated weights for policy 0, policy_version 1133442 (0.0010) [2023-12-26 23:37:47,188][105692] Updated weights for policy 0, policy_version 1133452 (0.0008) [2023-12-26 23:37:47,268][105620] Updated weights for policy 1, policy_version 1134737 (0.0010) [2023-12-26 23:37:47,326][105620] Updated weights for policy 1, policy_version 1134747 (0.0010) [2023-12-26 23:37:47,370][105620] Updated weights for policy 1, policy_version 1134757 (0.0010) [2023-12-26 23:37:47,420][105620] Updated weights for policy 1, policy_version 1134767 (0.0009) [2023-12-26 23:37:47,924][105692] Updated weights for policy 0, policy_version 1133462 (0.0009) [2023-12-26 23:37:47,984][105692] Updated weights for policy 0, policy_version 1133472 (0.0009) [2023-12-26 23:37:48,039][105692] Updated weights for policy 0, policy_version 1133482 (0.0006) [2023-12-26 23:37:48,153][105620] Updated weights for policy 1, policy_version 1134777 (0.0008) [2023-12-26 23:37:48,231][105620] Updated weights for policy 1, policy_version 1134787 (0.0009) [2023-12-26 23:37:48,293][105620] Updated weights for policy 1, policy_version 1134797 (0.0005) [2023-12-26 23:37:48,774][105692] Updated weights for policy 0, policy_version 1133492 (0.0005) [2023-12-26 23:37:48,832][105692] Updated weights for policy 0, policy_version 1133502 (0.0005) [2023-12-26 23:37:48,882][105692] Updated weights for policy 0, policy_version 1133512 (0.0005) [2023-12-26 23:37:49,037][105620] Updated weights for policy 1, policy_version 1134807 (0.0008) [2023-12-26 23:37:49,091][105620] Updated weights for policy 1, policy_version 1134817 (0.0008) [2023-12-26 23:37:49,142][105620] Updated weights for policy 1, policy_version 1134827 (0.0009) [2023-12-26 23:37:49,560][105692] Updated weights for policy 0, policy_version 1133522 (0.0006) [2023-12-26 23:37:49,623][105692] Updated weights for policy 0, policy_version 1133532 (0.0006) [2023-12-26 23:37:49,687][105692] Updated weights for policy 0, policy_version 1133542 (0.0008) [2023-12-26 23:37:49,750][105692] Updated weights for policy 0, policy_version 1133552 (0.0007) [2023-12-26 23:37:49,906][105620] Updated weights for policy 1, policy_version 1134837 (0.0009) [2023-12-26 23:37:49,973][105620] Updated weights for policy 1, policy_version 1134847 (0.0008) [2023-12-26 23:37:50,024][105620] Updated weights for policy 1, policy_version 1134857 (0.0009) [2023-12-26 23:37:50,440][105692] Updated weights for policy 0, policy_version 1133562 (0.0005) [2023-12-26 23:37:50,498][105692] Updated weights for policy 0, policy_version 1133572 (0.0005) [2023-12-26 23:37:50,562][105692] Updated weights for policy 0, policy_version 1133582 (0.0007) [2023-12-26 23:37:50,827][105620] Updated weights for policy 1, policy_version 1134867 (0.0009) [2023-12-26 23:37:50,874][105620] Updated weights for policy 1, policy_version 1134877 (0.0008) [2023-12-26 23:37:50,931][105620] Updated weights for policy 1, policy_version 1134887 (0.0009) [2023-12-26 23:37:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 580812800. Throughput: 0: 9608.2, 1: 9651.6. Samples: 580799824. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:37:51,062][104569] Avg episode reward: [(0, '9354.296'), (1, '9072.356')] [2023-12-26 23:37:51,220][105692] Updated weights for policy 0, policy_version 1133592 (0.0009) [2023-12-26 23:37:51,285][105692] Updated weights for policy 0, policy_version 1133602 (0.0010) [2023-12-26 23:37:51,351][105692] Updated weights for policy 0, policy_version 1133612 (0.0009) [2023-12-26 23:37:51,813][105620] Updated weights for policy 1, policy_version 1134897 (0.0010) [2023-12-26 23:37:51,867][105620] Updated weights for policy 1, policy_version 1134907 (0.0009) [2023-12-26 23:37:51,915][105620] Updated weights for policy 1, policy_version 1134917 (0.0008) [2023-12-26 23:37:51,972][105620] Updated weights for policy 1, policy_version 1134927 (0.0009) [2023-12-26 23:37:52,016][105692] Updated weights for policy 0, policy_version 1133622 (0.0007) [2023-12-26 23:37:52,066][105692] Updated weights for policy 0, policy_version 1133632 (0.0006) [2023-12-26 23:37:52,124][105692] Updated weights for policy 0, policy_version 1133642 (0.0006) [2023-12-26 23:37:52,761][105692] Updated weights for policy 0, policy_version 1133652 (0.0008) [2023-12-26 23:37:52,822][105620] Updated weights for policy 1, policy_version 1134937 (0.0006) [2023-12-26 23:37:52,824][105692] Updated weights for policy 0, policy_version 1133662 (0.0011) [2023-12-26 23:37:52,880][105692] Updated weights for policy 0, policy_version 1133672 (0.0011) [2023-12-26 23:37:52,882][105620] Updated weights for policy 1, policy_version 1134947 (0.0005) [2023-12-26 23:37:52,940][105620] Updated weights for policy 1, policy_version 1134957 (0.0007) [2023-12-26 23:37:53,522][105620] Updated weights for policy 1, policy_version 1134967 (0.0007) [2023-12-26 23:37:53,586][105620] Updated weights for policy 1, policy_version 1134977 (0.0008) [2023-12-26 23:37:53,591][105692] Updated weights for policy 0, policy_version 1133682 (0.0011) [2023-12-26 23:37:53,647][105620] Updated weights for policy 1, policy_version 1134987 (0.0006) [2023-12-26 23:37:53,652][105692] Updated weights for policy 0, policy_version 1133692 (0.0010) [2023-12-26 23:37:53,708][105692] Updated weights for policy 0, policy_version 1133702 (0.0011) [2023-12-26 23:37:53,760][105692] Updated weights for policy 0, policy_version 1133712 (0.0010) [2023-12-26 23:37:54,383][105620] Updated weights for policy 1, policy_version 1134997 (0.0007) [2023-12-26 23:37:54,435][105620] Updated weights for policy 1, policy_version 1135007 (0.0008) [2023-12-26 23:37:54,488][105620] Updated weights for policy 1, policy_version 1135017 (0.0006) [2023-12-26 23:37:54,494][105692] Updated weights for policy 0, policy_version 1133722 (0.0010) [2023-12-26 23:37:54,538][105692] Updated weights for policy 0, policy_version 1133732 (0.0010) [2023-12-26 23:37:54,559][105585] KL-divergence is very high: 105.6641 [2023-12-26 23:37:54,583][105692] Updated weights for policy 0, policy_version 1133742 (0.0010) [2023-12-26 23:37:55,251][105620] Updated weights for policy 1, policy_version 1135027 (0.0007) [2023-12-26 23:37:55,309][105620] Updated weights for policy 1, policy_version 1135037 (0.0008) [2023-12-26 23:37:55,362][105692] Updated weights for policy 0, policy_version 1133752 (0.0010) [2023-12-26 23:37:55,369][105620] Updated weights for policy 1, policy_version 1135047 (0.0007) [2023-12-26 23:37:55,422][105692] Updated weights for policy 0, policy_version 1133762 (0.0011) [2023-12-26 23:37:55,492][105692] Updated weights for policy 0, policy_version 1133772 (0.0011) [2023-12-26 23:37:56,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.3, 300 sec: 19522.0). Total num frames: 580902912. Throughput: 0: 9649.5, 1: 9619.5. Samples: 580914084. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-12-26 23:37:56,062][104569] Avg episode reward: [(0, '9080.135'), (1, '9256.447')] [2023-12-26 23:37:56,129][105620] Updated weights for policy 1, policy_version 1135057 (0.0006) [2023-12-26 23:37:56,180][105620] Updated weights for policy 1, policy_version 1135067 (0.0008) [2023-12-26 23:37:56,226][105692] Updated weights for policy 0, policy_version 1133782 (0.0010) [2023-12-26 23:37:56,233][105620] Updated weights for policy 1, policy_version 1135077 (0.0007) [2023-12-26 23:37:56,288][105692] Updated weights for policy 0, policy_version 1133792 (0.0010) [2023-12-26 23:37:56,294][105620] Updated weights for policy 1, policy_version 1135087 (0.0006) [2023-12-26 23:37:56,343][105692] Updated weights for policy 0, policy_version 1133802 (0.0010) [2023-12-26 23:37:57,032][105620] Updated weights for policy 1, policy_version 1135097 (0.0007) [2023-12-26 23:37:57,078][105620] Updated weights for policy 1, policy_version 1135107 (0.0008) [2023-12-26 23:37:57,086][105692] Updated weights for policy 0, policy_version 1133812 (0.0010) [2023-12-26 23:37:57,123][105620] Updated weights for policy 1, policy_version 1135117 (0.0008) [2023-12-26 23:37:57,130][105692] Updated weights for policy 0, policy_version 1133822 (0.0010) [2023-12-26 23:37:57,177][105692] Updated weights for policy 0, policy_version 1133832 (0.0010) [2023-12-26 23:37:57,907][105620] Updated weights for policy 1, policy_version 1135127 (0.0008) [2023-12-26 23:37:57,935][105692] Updated weights for policy 0, policy_version 1133842 (0.0010) [2023-12-26 23:37:57,959][105620] Updated weights for policy 1, policy_version 1135137 (0.0010) [2023-12-26 23:37:57,994][105692] Updated weights for policy 0, policy_version 1133852 (0.0010) [2023-12-26 23:37:58,018][105620] Updated weights for policy 1, policy_version 1135147 (0.0010) [2023-12-26 23:37:58,055][105692] Updated weights for policy 0, policy_version 1133862 (0.0010) [2023-12-26 23:37:58,111][105692] Updated weights for policy 0, policy_version 1133872 (0.0010) [2023-12-26 23:37:58,831][105620] Updated weights for policy 1, policy_version 1135157 (0.0012) [2023-12-26 23:37:58,905][105620] Updated weights for policy 1, policy_version 1135167 (0.0008) [2023-12-26 23:37:58,923][105692] Updated weights for policy 0, policy_version 1133882 (0.0010) [2023-12-26 23:37:58,966][105620] Updated weights for policy 1, policy_version 1135177 (0.0006) [2023-12-26 23:37:58,984][105692] Updated weights for policy 0, policy_version 1133892 (0.0010) [2023-12-26 23:37:59,051][105692] Updated weights for policy 0, policy_version 1133902 (0.0007) [2023-12-26 23:37:59,738][105620] Updated weights for policy 1, policy_version 1135187 (0.0008) [2023-12-26 23:37:59,799][105620] Updated weights for policy 1, policy_version 1135197 (0.0005) [2023-12-26 23:37:59,803][105692] Updated weights for policy 0, policy_version 1133912 (0.0009) [2023-12-26 23:37:59,864][105692] Updated weights for policy 0, policy_version 1133922 (0.0009) [2023-12-26 23:37:59,867][105620] Updated weights for policy 1, policy_version 1135207 (0.0007) [2023-12-26 23:37:59,928][105692] Updated weights for policy 0, policy_version 1133932 (0.0009) [2023-12-26 23:38:00,577][105620] Updated weights for policy 1, policy_version 1135217 (0.0008) [2023-12-26 23:38:00,632][105620] Updated weights for policy 1, policy_version 1135227 (0.0010) [2023-12-26 23:38:00,663][105692] Updated weights for policy 0, policy_version 1133942 (0.0006) [2023-12-26 23:38:00,705][105620] Updated weights for policy 1, policy_version 1135237 (0.0008) [2023-12-26 23:38:00,715][105692] Updated weights for policy 0, policy_version 1133952 (0.0006) [2023-12-26 23:38:00,768][105620] Updated weights for policy 1, policy_version 1135247 (0.0008) [2023-12-26 23:38:00,778][105692] Updated weights for policy 0, policy_version 1133962 (0.0006) [2023-12-26 23:38:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 581001216. Throughput: 0: 9622.7, 1: 9614.3. Samples: 580969752. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:38:01,062][104569] Avg episode reward: [(0, '8991.204'), (1, '9166.493')] [2023-12-26 23:38:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001133968_290340864.pth... [2023-12-26 23:38:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001135248_290660352.pth... [2023-12-26 23:38:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001132816_290045952.pth [2023-12-26 23:38:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001134128_290373632.pth [2023-12-26 23:38:01,484][105692] Updated weights for policy 0, policy_version 1133972 (0.0009) [2023-12-26 23:38:01,543][105692] Updated weights for policy 0, policy_version 1133982 (0.0009) [2023-12-26 23:38:01,593][105620] Updated weights for policy 1, policy_version 1135257 (0.0006) [2023-12-26 23:38:01,596][105692] Updated weights for policy 0, policy_version 1133992 (0.0007) [2023-12-26 23:38:01,654][105620] Updated weights for policy 1, policy_version 1135267 (0.0008) [2023-12-26 23:38:01,712][105620] Updated weights for policy 1, policy_version 1135277 (0.0009) [2023-12-26 23:38:02,307][105692] Updated weights for policy 0, policy_version 1134002 (0.0009) [2023-12-26 23:38:02,373][105692] Updated weights for policy 0, policy_version 1134012 (0.0008) [2023-12-26 23:38:02,424][105692] Updated weights for policy 0, policy_version 1134022 (0.0007) [2023-12-26 23:38:02,453][105620] Updated weights for policy 1, policy_version 1135287 (0.0009) [2023-12-26 23:38:02,479][105692] Updated weights for policy 0, policy_version 1134032 (0.0006) [2023-12-26 23:38:02,515][105620] Updated weights for policy 1, policy_version 1135297 (0.0008) [2023-12-26 23:38:02,570][105620] Updated weights for policy 1, policy_version 1135307 (0.0008) [2023-12-26 23:38:03,185][105620] Updated weights for policy 1, policy_version 1135317 (0.0008) [2023-12-26 23:38:03,235][105620] Updated weights for policy 1, policy_version 1135327 (0.0006) [2023-12-26 23:38:03,251][105692] Updated weights for policy 0, policy_version 1134042 (0.0010) [2023-12-26 23:38:03,282][105620] Updated weights for policy 1, policy_version 1135337 (0.0005) [2023-12-26 23:38:03,295][105692] Updated weights for policy 0, policy_version 1134052 (0.0010) [2023-12-26 23:38:03,340][105692] Updated weights for policy 0, policy_version 1134062 (0.0010) [2023-12-26 23:38:04,039][105692] Updated weights for policy 0, policy_version 1134072 (0.0010) [2023-12-26 23:38:04,059][105620] Updated weights for policy 1, policy_version 1135347 (0.0005) [2023-12-26 23:38:04,102][105692] Updated weights for policy 0, policy_version 1134082 (0.0011) [2023-12-26 23:38:04,121][105620] Updated weights for policy 1, policy_version 1135357 (0.0006) [2023-12-26 23:38:04,166][105692] Updated weights for policy 0, policy_version 1134092 (0.0009) [2023-12-26 23:38:04,186][105620] Updated weights for policy 1, policy_version 1135367 (0.0007) [2023-12-26 23:38:04,812][105620] Updated weights for policy 1, policy_version 1135377 (0.0006) [2023-12-26 23:38:04,852][105692] Updated weights for policy 0, policy_version 1134102 (0.0008) [2023-12-26 23:38:04,876][105620] Updated weights for policy 1, policy_version 1135387 (0.0007) [2023-12-26 23:38:04,910][105692] Updated weights for policy 0, policy_version 1134112 (0.0005) [2023-12-26 23:38:04,931][105620] Updated weights for policy 1, policy_version 1135397 (0.0006) [2023-12-26 23:38:04,967][105692] Updated weights for policy 0, policy_version 1134122 (0.0005) [2023-12-26 23:38:04,981][105620] Updated weights for policy 1, policy_version 1135407 (0.0007) [2023-12-26 23:38:05,498][105692] Updated weights for policy 0, policy_version 1134132 (0.0005) [2023-12-26 23:38:05,560][105692] Updated weights for policy 0, policy_version 1134142 (0.0007) [2023-12-26 23:38:05,607][105692] Updated weights for policy 0, policy_version 1134152 (0.0009) [2023-12-26 23:38:05,751][105620] Updated weights for policy 1, policy_version 1135417 (0.0009) [2023-12-26 23:38:05,801][105620] Updated weights for policy 1, policy_version 1135427 (0.0009) [2023-12-26 23:38:05,849][105620] Updated weights for policy 1, policy_version 1135437 (0.0009) [2023-12-26 23:38:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 581099520. Throughput: 0: 9643.0, 1: 9668.3. Samples: 581084668. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:38:06,062][104569] Avg episode reward: [(0, '8989.178'), (1, '9166.958')] [2023-12-26 23:38:06,261][105692] Updated weights for policy 0, policy_version 1134162 (0.0009) [2023-12-26 23:38:06,320][105692] Updated weights for policy 0, policy_version 1134172 (0.0007) [2023-12-26 23:38:06,381][105692] Updated weights for policy 0, policy_version 1134182 (0.0010) [2023-12-26 23:38:06,449][105692] Updated weights for policy 0, policy_version 1134192 (0.0010) [2023-12-26 23:38:06,642][105620] Updated weights for policy 1, policy_version 1135447 (0.0008) [2023-12-26 23:38:06,710][105620] Updated weights for policy 1, policy_version 1135457 (0.0008) [2023-12-26 23:38:06,774][105620] Updated weights for policy 1, policy_version 1135467 (0.0008) [2023-12-26 23:38:07,193][105692] Updated weights for policy 0, policy_version 1134202 (0.0011) [2023-12-26 23:38:07,259][105692] Updated weights for policy 0, policy_version 1134212 (0.0010) [2023-12-26 23:38:07,323][105692] Updated weights for policy 0, policy_version 1134222 (0.0010) [2023-12-26 23:38:07,351][105620] Updated weights for policy 1, policy_version 1135477 (0.0007) [2023-12-26 23:38:07,417][105620] Updated weights for policy 1, policy_version 1135487 (0.0005) [2023-12-26 23:38:07,466][105620] Updated weights for policy 1, policy_version 1135497 (0.0008) [2023-12-26 23:38:08,033][105620] Updated weights for policy 1, policy_version 1135507 (0.0007) [2023-12-26 23:38:08,069][105692] Updated weights for policy 0, policy_version 1134232 (0.0009) [2023-12-26 23:38:08,082][105620] Updated weights for policy 1, policy_version 1135517 (0.0005) [2023-12-26 23:38:08,083][105585] KL-divergence is very high: 105.8707 [2023-12-26 23:38:08,117][105692] Updated weights for policy 0, policy_version 1134242 (0.0009) [2023-12-26 23:38:08,119][105585] KL-divergence is very high: 154.0456 [2023-12-26 23:38:08,126][105620] Updated weights for policy 1, policy_version 1135527 (0.0005) [2023-12-26 23:38:08,159][105585] KL-divergence is very high: 125.9762 [2023-12-26 23:38:08,166][105692] Updated weights for policy 0, policy_version 1134252 (0.0009) [2023-12-26 23:38:08,748][105620] Updated weights for policy 1, policy_version 1135537 (0.0006) [2023-12-26 23:38:08,811][105620] Updated weights for policy 1, policy_version 1135547 (0.0007) [2023-12-26 23:38:08,871][105620] Updated weights for policy 1, policy_version 1135557 (0.0010) [2023-12-26 23:38:08,942][105620] Updated weights for policy 1, policy_version 1135567 (0.0011) [2023-12-26 23:38:08,950][105692] Updated weights for policy 0, policy_version 1134262 (0.0010) [2023-12-26 23:38:09,006][105692] Updated weights for policy 0, policy_version 1134272 (0.0011) [2023-12-26 23:38:09,072][105692] Updated weights for policy 0, policy_version 1134282 (0.0010) [2023-12-26 23:38:09,695][105620] Updated weights for policy 1, policy_version 1135577 (0.0010) [2023-12-26 23:38:09,755][105620] Updated weights for policy 1, policy_version 1135587 (0.0009) [2023-12-26 23:38:09,791][105692] Updated weights for policy 0, policy_version 1134292 (0.0009) [2023-12-26 23:38:09,811][105620] Updated weights for policy 1, policy_version 1135597 (0.0008) [2023-12-26 23:38:09,862][105692] Updated weights for policy 0, policy_version 1134302 (0.0008) [2023-12-26 23:38:09,921][105692] Updated weights for policy 0, policy_version 1134312 (0.0008) [2023-12-26 23:38:10,598][105620] Updated weights for policy 1, policy_version 1135607 (0.0010) [2023-12-26 23:38:10,623][105692] Updated weights for policy 0, policy_version 1134322 (0.0008) [2023-12-26 23:38:10,659][105620] Updated weights for policy 1, policy_version 1135617 (0.0011) [2023-12-26 23:38:10,679][105692] Updated weights for policy 0, policy_version 1134332 (0.0005) [2023-12-26 23:38:10,720][105620] Updated weights for policy 1, policy_version 1135627 (0.0011) [2023-12-26 23:38:10,737][105692] Updated weights for policy 0, policy_version 1134342 (0.0007) [2023-12-26 23:38:10,800][105692] Updated weights for policy 0, policy_version 1134352 (0.0008) [2023-12-26 23:38:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 581197824. Throughput: 0: 9774.5, 1: 9650.2. Samples: 581203824. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:38:11,062][104569] Avg episode reward: [(0, '8984.296'), (1, '9167.298')] [2023-12-26 23:38:11,512][105620] Updated weights for policy 1, policy_version 1135637 (0.0008) [2023-12-26 23:38:11,551][105692] Updated weights for policy 0, policy_version 1134362 (0.0006) [2023-12-26 23:38:11,571][105620] Updated weights for policy 1, policy_version 1135647 (0.0006) [2023-12-26 23:38:11,619][105692] Updated weights for policy 0, policy_version 1134372 (0.0008) [2023-12-26 23:38:11,640][105620] Updated weights for policy 1, policy_version 1135657 (0.0010) [2023-12-26 23:38:11,685][105692] Updated weights for policy 0, policy_version 1134382 (0.0008) [2023-12-26 23:38:12,395][105620] Updated weights for policy 1, policy_version 1135667 (0.0010) [2023-12-26 23:38:12,410][105692] Updated weights for policy 0, policy_version 1134392 (0.0010) [2023-12-26 23:38:12,454][105620] Updated weights for policy 1, policy_version 1135677 (0.0010) [2023-12-26 23:38:12,470][105692] Updated weights for policy 0, policy_version 1134402 (0.0011) [2023-12-26 23:38:12,517][105620] Updated weights for policy 1, policy_version 1135687 (0.0010) [2023-12-26 23:38:12,526][105692] Updated weights for policy 0, policy_version 1134412 (0.0010) [2023-12-26 23:38:13,237][105620] Updated weights for policy 1, policy_version 1135697 (0.0010) [2023-12-26 23:38:13,297][105620] Updated weights for policy 1, policy_version 1135707 (0.0007) [2023-12-26 23:38:13,303][105692] Updated weights for policy 0, policy_version 1134422 (0.0008) [2023-12-26 23:38:13,353][105620] Updated weights for policy 1, policy_version 1135717 (0.0008) [2023-12-26 23:38:13,367][105692] Updated weights for policy 0, policy_version 1134432 (0.0006) [2023-12-26 23:38:13,423][105620] Updated weights for policy 1, policy_version 1135727 (0.0010) [2023-12-26 23:38:13,433][105692] Updated weights for policy 0, policy_version 1134442 (0.0005) [2023-12-26 23:38:14,081][105692] Updated weights for policy 0, policy_version 1134452 (0.0007) [2023-12-26 23:38:14,136][105692] Updated weights for policy 0, policy_version 1134462 (0.0009) [2023-12-26 23:38:14,189][105620] Updated weights for policy 1, policy_version 1135737 (0.0008) [2023-12-26 23:38:14,200][105692] Updated weights for policy 0, policy_version 1134472 (0.0006) [2023-12-26 23:38:14,246][105620] Updated weights for policy 1, policy_version 1135747 (0.0008) [2023-12-26 23:38:14,291][105586] KL-divergence is very high: 100.9582 [2023-12-26 23:38:14,303][105620] Updated weights for policy 1, policy_version 1135757 (0.0009) [2023-12-26 23:38:14,968][105692] Updated weights for policy 0, policy_version 1134482 (0.0006) [2023-12-26 23:38:15,011][105620] Updated weights for policy 1, policy_version 1135767 (0.0009) [2023-12-26 23:38:15,022][105692] Updated weights for policy 0, policy_version 1134492 (0.0008) [2023-12-26 23:38:15,073][105620] Updated weights for policy 1, policy_version 1135777 (0.0008) [2023-12-26 23:38:15,083][105692] Updated weights for policy 0, policy_version 1134502 (0.0006) [2023-12-26 23:38:15,131][105620] Updated weights for policy 1, policy_version 1135787 (0.0007) [2023-12-26 23:38:15,146][105692] Updated weights for policy 0, policy_version 1134512 (0.0008) [2023-12-26 23:38:15,861][105692] Updated weights for policy 0, policy_version 1134522 (0.0009) [2023-12-26 23:38:15,901][105620] Updated weights for policy 1, policy_version 1135797 (0.0008) [2023-12-26 23:38:15,919][105692] Updated weights for policy 0, policy_version 1134532 (0.0009) [2023-12-26 23:38:15,962][105620] Updated weights for policy 1, policy_version 1135807 (0.0007) [2023-12-26 23:38:15,979][105692] Updated weights for policy 0, policy_version 1134542 (0.0009) [2023-12-26 23:38:16,020][105620] Updated weights for policy 1, policy_version 1135817 (0.0008) [2023-12-26 23:38:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 581296128. Throughput: 0: 9816.7, 1: 9539.2. Samples: 581261060. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:38:16,063][104569] Avg episode reward: [(0, '9260.767'), (1, '9076.219')] [2023-12-26 23:38:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001134544_290488320.pth... [2023-12-26 23:38:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001135824_290807808.pth... [2023-12-26 23:38:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001134704_290521088.pth [2023-12-26 23:38:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001133392_290193408.pth [2023-12-26 23:38:16,729][105692] Updated weights for policy 0, policy_version 1134552 (0.0009) [2023-12-26 23:38:16,762][105620] Updated weights for policy 1, policy_version 1135827 (0.0009) [2023-12-26 23:38:16,784][105692] Updated weights for policy 0, policy_version 1134562 (0.0008) [2023-12-26 23:38:16,817][105620] Updated weights for policy 1, policy_version 1135837 (0.0007) [2023-12-26 23:38:16,847][105692] Updated weights for policy 0, policy_version 1134572 (0.0005) [2023-12-26 23:38:16,873][105620] Updated weights for policy 1, policy_version 1135847 (0.0009) [2023-12-26 23:38:17,398][105692] Updated weights for policy 0, policy_version 1134582 (0.0007) [2023-12-26 23:38:17,457][105692] Updated weights for policy 0, policy_version 1134592 (0.0009) [2023-12-26 23:38:17,515][105692] Updated weights for policy 0, policy_version 1134602 (0.0009) [2023-12-26 23:38:17,703][105620] Updated weights for policy 1, policy_version 1135857 (0.0010) [2023-12-26 23:38:17,757][105620] Updated weights for policy 1, policy_version 1135867 (0.0009) [2023-12-26 23:38:17,803][105620] Updated weights for policy 1, policy_version 1135877 (0.0008) [2023-12-26 23:38:17,854][105620] Updated weights for policy 1, policy_version 1135887 (0.0009) [2023-12-26 23:38:18,266][105692] Updated weights for policy 0, policy_version 1134612 (0.0009) [2023-12-26 23:38:18,326][105692] Updated weights for policy 0, policy_version 1134622 (0.0008) [2023-12-26 23:38:18,393][105692] Updated weights for policy 0, policy_version 1134632 (0.0006) [2023-12-26 23:38:18,660][105620] Updated weights for policy 1, policy_version 1135897 (0.0009) [2023-12-26 23:38:18,719][105620] Updated weights for policy 1, policy_version 1135907 (0.0009) [2023-12-26 23:38:18,779][105620] Updated weights for policy 1, policy_version 1135917 (0.0009) [2023-12-26 23:38:19,119][105692] Updated weights for policy 0, policy_version 1134642 (0.0006) [2023-12-26 23:38:19,174][105692] Updated weights for policy 0, policy_version 1134652 (0.0008) [2023-12-26 23:38:19,230][105692] Updated weights for policy 0, policy_version 1134662 (0.0009) [2023-12-26 23:38:19,291][105692] Updated weights for policy 0, policy_version 1134672 (0.0010) [2023-12-26 23:38:19,475][105620] Updated weights for policy 1, policy_version 1135927 (0.0009) [2023-12-26 23:38:19,536][105620] Updated weights for policy 1, policy_version 1135937 (0.0009) [2023-12-26 23:38:19,597][105620] Updated weights for policy 1, policy_version 1135947 (0.0009) [2023-12-26 23:38:20,076][105692] Updated weights for policy 0, policy_version 1134682 (0.0011) [2023-12-26 23:38:20,129][105692] Updated weights for policy 0, policy_version 1134692 (0.0011) [2023-12-26 23:38:20,196][105692] Updated weights for policy 0, policy_version 1134702 (0.0011) [2023-12-26 23:38:20,358][105620] Updated weights for policy 1, policy_version 1135957 (0.0010) [2023-12-26 23:38:20,411][105620] Updated weights for policy 1, policy_version 1135967 (0.0010) [2023-12-26 23:38:20,472][105620] Updated weights for policy 1, policy_version 1135977 (0.0009) [2023-12-26 23:38:20,860][105692] Updated weights for policy 0, policy_version 1134712 (0.0006) [2023-12-26 23:38:20,924][105692] Updated weights for policy 0, policy_version 1134722 (0.0008) [2023-12-26 23:38:20,977][105692] Updated weights for policy 0, policy_version 1134732 (0.0009) [2023-12-26 23:38:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 581386240. Throughput: 0: 9765.9, 1: 9535.1. Samples: 581374332. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:38:21,063][104569] Avg episode reward: [(0, '9171.774'), (1, '9054.354')] [2023-12-26 23:38:21,314][105620] Updated weights for policy 1, policy_version 1135987 (0.0010) [2023-12-26 23:38:21,389][105620] Updated weights for policy 1, policy_version 1135997 (0.0009) [2023-12-26 23:38:21,461][105620] Updated weights for policy 1, policy_version 1136007 (0.0009) [2023-12-26 23:38:21,630][105692] Updated weights for policy 0, policy_version 1134742 (0.0008) [2023-12-26 23:38:21,688][105692] Updated weights for policy 0, policy_version 1134752 (0.0009) [2023-12-26 23:38:21,760][105692] Updated weights for policy 0, policy_version 1134762 (0.0009) [2023-12-26 23:38:22,222][105620] Updated weights for policy 1, policy_version 1136017 (0.0010) [2023-12-26 23:38:22,291][105620] Updated weights for policy 1, policy_version 1136027 (0.0009) [2023-12-26 23:38:22,346][105620] Updated weights for policy 1, policy_version 1136037 (0.0008) [2023-12-26 23:38:22,415][105620] Updated weights for policy 1, policy_version 1136047 (0.0006) [2023-12-26 23:38:22,593][105692] Updated weights for policy 0, policy_version 1134772 (0.0009) [2023-12-26 23:38:22,650][105692] Updated weights for policy 0, policy_version 1134782 (0.0009) [2023-12-26 23:38:22,711][105692] Updated weights for policy 0, policy_version 1134792 (0.0009) [2023-12-26 23:38:23,010][105620] Updated weights for policy 1, policy_version 1136057 (0.0005) [2023-12-26 23:38:23,058][105620] Updated weights for policy 1, policy_version 1136067 (0.0005) [2023-12-26 23:38:23,105][105620] Updated weights for policy 1, policy_version 1136077 (0.0008) [2023-12-26 23:38:23,563][105692] Updated weights for policy 0, policy_version 1134802 (0.0007) [2023-12-26 23:38:23,608][105692] Updated weights for policy 0, policy_version 1134812 (0.0006) [2023-12-26 23:38:23,664][105692] Updated weights for policy 0, policy_version 1134822 (0.0005) [2023-12-26 23:38:23,679][105620] Updated weights for policy 1, policy_version 1136087 (0.0005) [2023-12-26 23:38:23,712][105692] Updated weights for policy 0, policy_version 1134832 (0.0006) [2023-12-26 23:38:23,727][105620] Updated weights for policy 1, policy_version 1136097 (0.0007) [2023-12-26 23:38:23,778][105620] Updated weights for policy 1, policy_version 1136107 (0.0010) [2023-12-26 23:38:24,297][105692] Updated weights for policy 0, policy_version 1134842 (0.0006) [2023-12-26 23:38:24,349][105692] Updated weights for policy 0, policy_version 1134852 (0.0010) [2023-12-26 23:38:24,382][105620] Updated weights for policy 1, policy_version 1136117 (0.0009) [2023-12-26 23:38:24,407][105692] Updated weights for policy 0, policy_version 1134862 (0.0010) [2023-12-26 23:38:24,448][105620] Updated weights for policy 1, policy_version 1136127 (0.0008) [2023-12-26 23:38:24,514][105620] Updated weights for policy 1, policy_version 1136137 (0.0008) [2023-12-26 23:38:24,989][105692] Updated weights for policy 0, policy_version 1134872 (0.0006) [2023-12-26 23:38:25,047][105692] Updated weights for policy 0, policy_version 1134882 (0.0010) [2023-12-26 23:38:25,108][105692] Updated weights for policy 0, policy_version 1134892 (0.0011) [2023-12-26 23:38:25,173][105620] Updated weights for policy 1, policy_version 1136147 (0.0006) [2023-12-26 23:38:25,223][105620] Updated weights for policy 1, policy_version 1136157 (0.0006) [2023-12-26 23:38:25,277][105620] Updated weights for policy 1, policy_version 1136167 (0.0005) [2023-12-26 23:38:25,729][105692] Updated weights for policy 0, policy_version 1134902 (0.0007) [2023-12-26 23:38:25,796][105692] Updated weights for policy 0, policy_version 1134912 (0.0006) [2023-12-26 23:38:25,866][105692] Updated weights for policy 0, policy_version 1134922 (0.0008) [2023-12-26 23:38:25,912][105620] Updated weights for policy 1, policy_version 1136177 (0.0006) [2023-12-26 23:38:25,968][105620] Updated weights for policy 1, policy_version 1136187 (0.0010) [2023-12-26 23:38:26,028][105620] Updated weights for policy 1, policy_version 1136197 (0.0010) [2023-12-26 23:38:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 581484544. Throughput: 0: 9817.2, 1: 9549.6. Samples: 581495104. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:38:26,062][104569] Avg episode reward: [(0, '8989.591'), (1, '7073.812')] [2023-12-26 23:38:26,087][105620] Updated weights for policy 1, policy_version 1136207 (0.0010) [2023-12-26 23:38:26,580][105692] Updated weights for policy 0, policy_version 1134932 (0.0007) [2023-12-26 23:38:26,636][105692] Updated weights for policy 0, policy_version 1134942 (0.0005) [2023-12-26 23:38:26,689][105692] Updated weights for policy 0, policy_version 1134952 (0.0005) [2023-12-26 23:38:26,796][105620] Updated weights for policy 1, policy_version 1136217 (0.0011) [2023-12-26 23:38:26,851][105620] Updated weights for policy 1, policy_version 1136227 (0.0010) [2023-12-26 23:38:26,907][105620] Updated weights for policy 1, policy_version 1136237 (0.0010) [2023-12-26 23:38:27,229][105692] Updated weights for policy 0, policy_version 1134962 (0.0006) [2023-12-26 23:38:27,273][105692] Updated weights for policy 0, policy_version 1134972 (0.0007) [2023-12-26 23:38:27,326][105692] Updated weights for policy 0, policy_version 1134982 (0.0006) [2023-12-26 23:38:27,385][105692] Updated weights for policy 0, policy_version 1134992 (0.0009) [2023-12-26 23:38:27,628][105620] Updated weights for policy 1, policy_version 1136247 (0.0008) [2023-12-26 23:38:27,673][105620] Updated weights for policy 1, policy_version 1136257 (0.0008) [2023-12-26 23:38:27,730][105620] Updated weights for policy 1, policy_version 1136267 (0.0008) [2023-12-26 23:38:27,983][105692] Updated weights for policy 0, policy_version 1135002 (0.0009) [2023-12-26 23:38:28,036][105692] Updated weights for policy 0, policy_version 1135012 (0.0009) [2023-12-26 23:38:28,100][105692] Updated weights for policy 0, policy_version 1135022 (0.0008) [2023-12-26 23:38:28,307][105620] Updated weights for policy 1, policy_version 1136277 (0.0005) [2023-12-26 23:38:28,369][105620] Updated weights for policy 1, policy_version 1136287 (0.0009) [2023-12-26 23:38:28,432][105620] Updated weights for policy 1, policy_version 1136297 (0.0006) [2023-12-26 23:38:28,757][105692] Updated weights for policy 0, policy_version 1135032 (0.0010) [2023-12-26 23:38:28,819][105692] Updated weights for policy 0, policy_version 1135042 (0.0011) [2023-12-26 23:38:28,885][105692] Updated weights for policy 0, policy_version 1135052 (0.0011) [2023-12-26 23:38:29,009][105620] Updated weights for policy 1, policy_version 1136307 (0.0009) [2023-12-26 23:38:29,056][105620] Updated weights for policy 1, policy_version 1136317 (0.0005) [2023-12-26 23:38:29,101][105620] Updated weights for policy 1, policy_version 1136327 (0.0005) [2023-12-26 23:38:29,674][105692] Updated weights for policy 0, policy_version 1135062 (0.0009) [2023-12-26 23:38:29,741][105692] Updated weights for policy 0, policy_version 1135072 (0.0009) [2023-12-26 23:38:29,765][105620] Updated weights for policy 1, policy_version 1136337 (0.0006) [2023-12-26 23:38:29,795][105692] Updated weights for policy 0, policy_version 1135082 (0.0007) [2023-12-26 23:38:29,822][105620] Updated weights for policy 1, policy_version 1136347 (0.0009) [2023-12-26 23:38:29,879][105620] Updated weights for policy 1, policy_version 1136357 (0.0007) [2023-12-26 23:38:29,942][105620] Updated weights for policy 1, policy_version 1136367 (0.0007) [2023-12-26 23:38:30,484][105692] Updated weights for policy 0, policy_version 1135092 (0.0009) [2023-12-26 23:38:30,541][105692] Updated weights for policy 0, policy_version 1135102 (0.0008) [2023-12-26 23:38:30,612][105692] Updated weights for policy 0, policy_version 1135112 (0.0005) [2023-12-26 23:38:30,627][105620] Updated weights for policy 1, policy_version 1136377 (0.0009) [2023-12-26 23:38:30,677][105620] Updated weights for policy 1, policy_version 1136388 (0.0009) [2023-12-26 23:38:30,731][105620] Updated weights for policy 1, policy_version 1136399 (0.0010) [2023-12-26 23:38:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 581591040. Throughput: 0: 9879.1, 1: 9613.2. Samples: 581559432. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:38:31,062][104569] Avg episode reward: [(0, '9080.197'), (1, '7248.689')] [2023-12-26 23:38:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001135120_290635776.pth... [2023-12-26 23:38:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001136400_290955264.pth... [2023-12-26 23:38:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001133968_290340864.pth [2023-12-26 23:38:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001135248_290660352.pth [2023-12-26 23:38:31,309][105692] Updated weights for policy 0, policy_version 1135122 (0.0006) [2023-12-26 23:38:31,372][105692] Updated weights for policy 0, policy_version 1135132 (0.0009) [2023-12-26 23:38:31,425][105692] Updated weights for policy 0, policy_version 1135142 (0.0009) [2023-12-26 23:38:31,468][105620] Updated weights for policy 1, policy_version 1136409 (0.0008) [2023-12-26 23:38:31,482][105692] Updated weights for policy 0, policy_version 1135152 (0.0006) [2023-12-26 23:38:31,531][105620] Updated weights for policy 1, policy_version 1136419 (0.0008) [2023-12-26 23:38:31,600][105620] Updated weights for policy 1, policy_version 1136429 (0.0009) [2023-12-26 23:38:32,222][105692] Updated weights for policy 0, policy_version 1135162 (0.0008) [2023-12-26 23:38:32,280][105692] Updated weights for policy 0, policy_version 1135172 (0.0007) [2023-12-26 23:38:32,338][105692] Updated weights for policy 0, policy_version 1135182 (0.0008) [2023-12-26 23:38:32,366][105620] Updated weights for policy 1, policy_version 1136439 (0.0010) [2023-12-26 23:38:32,426][105620] Updated weights for policy 1, policy_version 1136449 (0.0011) [2023-12-26 23:38:32,489][105620] Updated weights for policy 1, policy_version 1136459 (0.0011) [2023-12-26 23:38:33,097][105692] Updated weights for policy 0, policy_version 1135192 (0.0008) [2023-12-26 23:38:33,148][105692] Updated weights for policy 0, policy_version 1135202 (0.0008) [2023-12-26 23:38:33,191][105692] Updated weights for policy 0, policy_version 1135212 (0.0008) [2023-12-26 23:38:33,234][105620] Updated weights for policy 1, policy_version 1136469 (0.0010) [2023-12-26 23:38:33,299][105620] Updated weights for policy 1, policy_version 1136479 (0.0010) [2023-12-26 23:38:33,349][105620] Updated weights for policy 1, policy_version 1136489 (0.0010) [2023-12-26 23:38:33,972][105692] Updated weights for policy 0, policy_version 1135222 (0.0007) [2023-12-26 23:38:34,023][105692] Updated weights for policy 0, policy_version 1135232 (0.0008) [2023-12-26 23:38:34,067][105692] Updated weights for policy 0, policy_version 1135242 (0.0008) [2023-12-26 23:38:34,089][105620] Updated weights for policy 1, policy_version 1136499 (0.0010) [2023-12-26 23:38:34,154][105620] Updated weights for policy 1, policy_version 1136509 (0.0010) [2023-12-26 23:38:34,216][105620] Updated weights for policy 1, policy_version 1136519 (0.0011) [2023-12-26 23:38:34,848][105692] Updated weights for policy 0, policy_version 1135252 (0.0006) [2023-12-26 23:38:34,903][105692] Updated weights for policy 0, policy_version 1135262 (0.0009) [2023-12-26 23:38:34,966][105692] Updated weights for policy 0, policy_version 1135272 (0.0008) [2023-12-26 23:38:34,982][105620] Updated weights for policy 1, policy_version 1136529 (0.0011) [2023-12-26 23:38:35,038][105620] Updated weights for policy 1, policy_version 1136539 (0.0011) [2023-12-26 23:38:35,089][105620] Updated weights for policy 1, policy_version 1136549 (0.0010) [2023-12-26 23:38:35,137][105620] Updated weights for policy 1, policy_version 1136559 (0.0010) [2023-12-26 23:38:35,733][105692] Updated weights for policy 0, policy_version 1135282 (0.0008) [2023-12-26 23:38:35,783][105692] Updated weights for policy 0, policy_version 1135292 (0.0008) [2023-12-26 23:38:35,831][105692] Updated weights for policy 0, policy_version 1135302 (0.0005) [2023-12-26 23:38:35,881][105692] Updated weights for policy 0, policy_version 1135312 (0.0005) [2023-12-26 23:38:35,912][105620] Updated weights for policy 1, policy_version 1136569 (0.0008) [2023-12-26 23:38:35,968][105620] Updated weights for policy 1, policy_version 1136579 (0.0008) [2023-12-26 23:38:36,021][105620] Updated weights for policy 1, policy_version 1136589 (0.0009) [2023-12-26 23:38:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 581689344. Throughput: 0: 9786.3, 1: 9636.3. Samples: 581673844. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:38:36,063][104569] Avg episode reward: [(0, '9078.892'), (1, '8627.143')] [2023-12-26 23:38:36,529][105692] Updated weights for policy 0, policy_version 1135322 (0.0011) [2023-12-26 23:38:36,592][105692] Updated weights for policy 0, policy_version 1135332 (0.0011) [2023-12-26 23:38:36,652][105692] Updated weights for policy 0, policy_version 1135342 (0.0011) [2023-12-26 23:38:36,695][105620] Updated weights for policy 1, policy_version 1136599 (0.0008) [2023-12-26 23:38:36,759][105620] Updated weights for policy 1, policy_version 1136609 (0.0008) [2023-12-26 23:38:36,818][105620] Updated weights for policy 1, policy_version 1136619 (0.0008) [2023-12-26 23:38:37,390][105692] Updated weights for policy 0, policy_version 1135352 (0.0011) [2023-12-26 23:38:37,442][105692] Updated weights for policy 0, policy_version 1135362 (0.0011) [2023-12-26 23:38:37,495][105692] Updated weights for policy 0, policy_version 1135372 (0.0011) [2023-12-26 23:38:37,554][105620] Updated weights for policy 1, policy_version 1136629 (0.0008) [2023-12-26 23:38:37,606][105620] Updated weights for policy 1, policy_version 1136639 (0.0008) [2023-12-26 23:38:37,654][105620] Updated weights for policy 1, policy_version 1136649 (0.0008) [2023-12-26 23:38:38,138][105692] Updated weights for policy 0, policy_version 1135382 (0.0008) [2023-12-26 23:38:38,203][105692] Updated weights for policy 0, policy_version 1135392 (0.0007) [2023-12-26 23:38:38,247][105692] Updated weights for policy 0, policy_version 1135402 (0.0010) [2023-12-26 23:38:38,518][105620] Updated weights for policy 1, policy_version 1136659 (0.0008) [2023-12-26 23:38:38,582][105620] Updated weights for policy 1, policy_version 1136669 (0.0007) [2023-12-26 23:38:38,644][105620] Updated weights for policy 1, policy_version 1136679 (0.0005) [2023-12-26 23:38:38,988][105692] Updated weights for policy 0, policy_version 1135412 (0.0008) [2023-12-26 23:38:39,065][105692] Updated weights for policy 0, policy_version 1135422 (0.0006) [2023-12-26 23:38:39,132][105692] Updated weights for policy 0, policy_version 1135432 (0.0006) [2023-12-26 23:38:39,296][105620] Updated weights for policy 1, policy_version 1136689 (0.0006) [2023-12-26 23:38:39,361][105620] Updated weights for policy 1, policy_version 1136699 (0.0010) [2023-12-26 23:38:39,434][105620] Updated weights for policy 1, policy_version 1136709 (0.0010) [2023-12-26 23:38:39,495][105620] Updated weights for policy 1, policy_version 1136719 (0.0011) [2023-12-26 23:38:39,824][105692] Updated weights for policy 0, policy_version 1135442 (0.0008) [2023-12-26 23:38:39,890][105692] Updated weights for policy 0, policy_version 1135452 (0.0009) [2023-12-26 23:38:39,956][105692] Updated weights for policy 0, policy_version 1135462 (0.0008) [2023-12-26 23:38:40,021][105692] Updated weights for policy 0, policy_version 1135472 (0.0009) [2023-12-26 23:38:40,245][105620] Updated weights for policy 1, policy_version 1136729 (0.0011) [2023-12-26 23:38:40,312][105620] Updated weights for policy 1, policy_version 1136739 (0.0010) [2023-12-26 23:38:40,371][105620] Updated weights for policy 1, policy_version 1136749 (0.0011) [2023-12-26 23:38:40,774][105692] Updated weights for policy 0, policy_version 1135482 (0.0008) [2023-12-26 23:38:40,835][105692] Updated weights for policy 0, policy_version 1135492 (0.0008) [2023-12-26 23:38:40,901][105692] Updated weights for policy 0, policy_version 1135502 (0.0008) [2023-12-26 23:38:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 581779456. Throughput: 0: 9776.8, 1: 9671.8. Samples: 581789272. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:38:41,062][104569] Avg episode reward: [(0, '9261.207'), (1, '9347.409')] [2023-12-26 23:38:41,100][105620] Updated weights for policy 1, policy_version 1136759 (0.0011) [2023-12-26 23:38:41,162][105620] Updated weights for policy 1, policy_version 1136769 (0.0011) [2023-12-26 23:38:41,225][105620] Updated weights for policy 1, policy_version 1136779 (0.0006) [2023-12-26 23:38:41,655][105692] Updated weights for policy 0, policy_version 1135512 (0.0008) [2023-12-26 23:38:41,703][105692] Updated weights for policy 0, policy_version 1135522 (0.0008) [2023-12-26 23:38:41,768][105692] Updated weights for policy 0, policy_version 1135532 (0.0008) [2023-12-26 23:38:41,922][105620] Updated weights for policy 1, policy_version 1136789 (0.0011) [2023-12-26 23:38:41,979][105620] Updated weights for policy 1, policy_version 1136799 (0.0011) [2023-12-26 23:38:42,036][105620] Updated weights for policy 1, policy_version 1136809 (0.0010) [2023-12-26 23:38:42,538][105692] Updated weights for policy 0, policy_version 1135542 (0.0009) [2023-12-26 23:38:42,589][105692] Updated weights for policy 0, policy_version 1135552 (0.0007) [2023-12-26 23:38:42,634][105692] Updated weights for policy 0, policy_version 1135562 (0.0008) [2023-12-26 23:38:42,791][105620] Updated weights for policy 1, policy_version 1136819 (0.0008) [2023-12-26 23:38:42,855][105620] Updated weights for policy 1, policy_version 1136829 (0.0007) [2023-12-26 23:38:42,912][105620] Updated weights for policy 1, policy_version 1136839 (0.0006) [2023-12-26 23:38:43,454][105692] Updated weights for policy 0, policy_version 1135572 (0.0008) [2023-12-26 23:38:43,512][105692] Updated weights for policy 0, policy_version 1135582 (0.0008) [2023-12-26 23:38:43,572][105692] Updated weights for policy 0, policy_version 1135592 (0.0008) [2023-12-26 23:38:43,587][105620] Updated weights for policy 1, policy_version 1136849 (0.0008) [2023-12-26 23:38:43,632][105620] Updated weights for policy 1, policy_version 1136859 (0.0010) [2023-12-26 23:38:43,681][105620] Updated weights for policy 1, policy_version 1136869 (0.0010) [2023-12-26 23:38:43,729][105620] Updated weights for policy 1, policy_version 1136879 (0.0010) [2023-12-26 23:38:44,271][105692] Updated weights for policy 0, policy_version 1135602 (0.0006) [2023-12-26 23:38:44,322][105692] Updated weights for policy 0, policy_version 1135612 (0.0005) [2023-12-26 23:38:44,379][105692] Updated weights for policy 0, policy_version 1135622 (0.0005) [2023-12-26 23:38:44,434][105692] Updated weights for policy 0, policy_version 1135632 (0.0006) [2023-12-26 23:38:44,494][105620] Updated weights for policy 1, policy_version 1136889 (0.0008) [2023-12-26 23:38:44,555][105620] Updated weights for policy 1, policy_version 1136899 (0.0008) [2023-12-26 23:38:44,623][105620] Updated weights for policy 1, policy_version 1136909 (0.0009) [2023-12-26 23:38:45,030][105692] Updated weights for policy 0, policy_version 1135642 (0.0009) [2023-12-26 23:38:45,081][105692] Updated weights for policy 0, policy_version 1135652 (0.0009) [2023-12-26 23:38:45,134][105692] Updated weights for policy 0, policy_version 1135662 (0.0008) [2023-12-26 23:38:45,356][105620] Updated weights for policy 1, policy_version 1136919 (0.0008) [2023-12-26 23:38:45,416][105620] Updated weights for policy 1, policy_version 1136929 (0.0009) [2023-12-26 23:38:45,478][105620] Updated weights for policy 1, policy_version 1136939 (0.0009) [2023-12-26 23:38:45,863][105692] Updated weights for policy 0, policy_version 1135672 (0.0008) [2023-12-26 23:38:45,925][105692] Updated weights for policy 0, policy_version 1135682 (0.0009) [2023-12-26 23:38:45,981][105692] Updated weights for policy 0, policy_version 1135692 (0.0009) [2023-12-26 23:38:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 581877760. Throughput: 0: 9738.2, 1: 9710.7. Samples: 581844956. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:38:46,063][104569] Avg episode reward: [(0, '9261.831'), (1, '9079.412')] [2023-12-26 23:38:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001135696_290783232.pth... [2023-12-26 23:38:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001136944_291094528.pth... [2023-12-26 23:38:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001134544_290488320.pth [2023-12-26 23:38:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001135824_290807808.pth [2023-12-26 23:38:46,197][105620] Updated weights for policy 1, policy_version 1136949 (0.0009) [2023-12-26 23:38:46,251][105620] Updated weights for policy 1, policy_version 1136960 (0.0010) [2023-12-26 23:38:46,311][105620] Updated weights for policy 1, policy_version 1136970 (0.0009) [2023-12-26 23:38:46,575][105692] Updated weights for policy 0, policy_version 1135702 (0.0008) [2023-12-26 23:38:46,634][105692] Updated weights for policy 0, policy_version 1135712 (0.0009) [2023-12-26 23:38:46,689][105692] Updated weights for policy 0, policy_version 1135722 (0.0009) [2023-12-26 23:38:47,034][105620] Updated weights for policy 1, policy_version 1136980 (0.0007) [2023-12-26 23:38:47,089][105620] Updated weights for policy 1, policy_version 1136991 (0.0009) [2023-12-26 23:38:47,137][105620] Updated weights for policy 1, policy_version 1137001 (0.0009) [2023-12-26 23:38:47,436][105692] Updated weights for policy 0, policy_version 1135732 (0.0009) [2023-12-26 23:38:47,494][105692] Updated weights for policy 0, policy_version 1135742 (0.0009) [2023-12-26 23:38:47,553][105692] Updated weights for policy 0, policy_version 1135752 (0.0009) [2023-12-26 23:38:47,816][105620] Updated weights for policy 1, policy_version 1137011 (0.0009) [2023-12-26 23:38:47,867][105620] Updated weights for policy 1, policy_version 1137021 (0.0008) [2023-12-26 23:38:47,914][105620] Updated weights for policy 1, policy_version 1137031 (0.0008) [2023-12-26 23:38:48,376][105692] Updated weights for policy 0, policy_version 1135762 (0.0009) [2023-12-26 23:38:48,424][105692] Updated weights for policy 0, policy_version 1135772 (0.0009) [2023-12-26 23:38:48,487][105692] Updated weights for policy 0, policy_version 1135782 (0.0010) [2023-12-26 23:38:48,550][105692] Updated weights for policy 0, policy_version 1135792 (0.0010) [2023-12-26 23:38:48,583][105620] Updated weights for policy 1, policy_version 1137041 (0.0005) [2023-12-26 23:38:48,637][105620] Updated weights for policy 1, policy_version 1137051 (0.0007) [2023-12-26 23:38:48,688][105620] Updated weights for policy 1, policy_version 1137061 (0.0009) [2023-12-26 23:38:48,746][105620] Updated weights for policy 1, policy_version 1137071 (0.0009) [2023-12-26 23:38:49,257][105692] Updated weights for policy 0, policy_version 1135802 (0.0009) [2023-12-26 23:38:49,324][105692] Updated weights for policy 0, policy_version 1135812 (0.0008) [2023-12-26 23:38:49,382][105692] Updated weights for policy 0, policy_version 1135822 (0.0007) [2023-12-26 23:38:49,515][105620] Updated weights for policy 1, policy_version 1137081 (0.0010) [2023-12-26 23:38:49,576][105620] Updated weights for policy 1, policy_version 1137091 (0.0010) [2023-12-26 23:38:49,638][105620] Updated weights for policy 1, policy_version 1137101 (0.0010) [2023-12-26 23:38:50,206][105692] Updated weights for policy 0, policy_version 1135832 (0.0006) [2023-12-26 23:38:50,252][105692] Updated weights for policy 0, policy_version 1135842 (0.0008) [2023-12-26 23:38:50,310][105692] Updated weights for policy 0, policy_version 1135852 (0.0007) [2023-12-26 23:38:50,367][105620] Updated weights for policy 1, policy_version 1137111 (0.0011) [2023-12-26 23:38:50,439][105620] Updated weights for policy 1, policy_version 1137121 (0.0010) [2023-12-26 23:38:50,502][105620] Updated weights for policy 1, policy_version 1137131 (0.0010) [2023-12-26 23:38:50,990][105692] Updated weights for policy 0, policy_version 1135862 (0.0005) [2023-12-26 23:38:51,051][105692] Updated weights for policy 0, policy_version 1135872 (0.0008) [2023-12-26 23:38:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 581967872. Throughput: 0: 9793.6, 1: 9732.6. Samples: 581963348. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:38:51,062][104569] Avg episode reward: [(0, '9170.213'), (1, '8813.107')] [2023-12-26 23:38:51,107][105692] Updated weights for policy 0, policy_version 1135882 (0.0008) [2023-12-26 23:38:51,227][105620] Updated weights for policy 1, policy_version 1137141 (0.0008) [2023-12-26 23:38:51,292][105620] Updated weights for policy 1, policy_version 1137151 (0.0010) [2023-12-26 23:38:51,354][105620] Updated weights for policy 1, policy_version 1137161 (0.0011) [2023-12-26 23:38:51,844][105692] Updated weights for policy 0, policy_version 1135892 (0.0008) [2023-12-26 23:38:51,909][105692] Updated weights for policy 0, policy_version 1135902 (0.0007) [2023-12-26 23:38:51,971][105692] Updated weights for policy 0, policy_version 1135912 (0.0007) [2023-12-26 23:38:52,096][105620] Updated weights for policy 1, policy_version 1137171 (0.0010) [2023-12-26 23:38:52,160][105620] Updated weights for policy 1, policy_version 1137181 (0.0008) [2023-12-26 23:38:52,220][105620] Updated weights for policy 1, policy_version 1137191 (0.0010) [2023-12-26 23:38:52,759][105692] Updated weights for policy 0, policy_version 1135922 (0.0010) [2023-12-26 23:38:52,819][105692] Updated weights for policy 0, policy_version 1135932 (0.0008) [2023-12-26 23:38:52,881][105692] Updated weights for policy 0, policy_version 1135942 (0.0008) [2023-12-26 23:38:52,941][105692] Updated weights for policy 0, policy_version 1135952 (0.0009) [2023-12-26 23:38:52,954][105620] Updated weights for policy 1, policy_version 1137201 (0.0009) [2023-12-26 23:38:53,016][105620] Updated weights for policy 1, policy_version 1137211 (0.0009) [2023-12-26 23:38:53,063][105620] Updated weights for policy 1, policy_version 1137221 (0.0009) [2023-12-26 23:38:53,110][105620] Updated weights for policy 1, policy_version 1137231 (0.0009) [2023-12-26 23:38:53,677][105692] Updated weights for policy 0, policy_version 1135962 (0.0005) [2023-12-26 23:38:53,726][105692] Updated weights for policy 0, policy_version 1135972 (0.0005) [2023-12-26 23:38:53,780][105692] Updated weights for policy 0, policy_version 1135982 (0.0007) [2023-12-26 23:38:53,907][105620] Updated weights for policy 1, policy_version 1137241 (0.0005) [2023-12-26 23:38:53,958][105620] Updated weights for policy 1, policy_version 1137251 (0.0005) [2023-12-26 23:38:54,007][105620] Updated weights for policy 1, policy_version 1137261 (0.0008) [2023-12-26 23:38:54,567][105692] Updated weights for policy 0, policy_version 1135992 (0.0009) [2023-12-26 23:38:54,609][105620] Updated weights for policy 1, policy_version 1137271 (0.0007) [2023-12-26 23:38:54,619][105692] Updated weights for policy 0, policy_version 1136002 (0.0007) [2023-12-26 23:38:54,659][105620] Updated weights for policy 1, policy_version 1137281 (0.0006) [2023-12-26 23:38:54,672][105692] Updated weights for policy 0, policy_version 1136012 (0.0008) [2023-12-26 23:38:54,707][105620] Updated weights for policy 1, policy_version 1137291 (0.0006) [2023-12-26 23:38:55,413][105692] Updated weights for policy 0, policy_version 1136022 (0.0007) [2023-12-26 23:38:55,426][105620] Updated weights for policy 1, policy_version 1137301 (0.0008) [2023-12-26 23:38:55,476][105692] Updated weights for policy 0, policy_version 1136032 (0.0007) [2023-12-26 23:38:55,490][105620] Updated weights for policy 1, policy_version 1137311 (0.0007) [2023-12-26 23:38:55,530][105692] Updated weights for policy 0, policy_version 1136042 (0.0006) [2023-12-26 23:38:55,545][105620] Updated weights for policy 1, policy_version 1137321 (0.0007) [2023-12-26 23:38:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.6, 300 sec: 19466.4). Total num frames: 582066176. Throughput: 0: 9717.5, 1: 9701.5. Samples: 582077684. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:38:56,063][104569] Avg episode reward: [(0, '9080.695'), (1, '8813.100')] [2023-12-26 23:38:56,237][105620] Updated weights for policy 1, policy_version 1137331 (0.0008) [2023-12-26 23:38:56,283][105620] Updated weights for policy 1, policy_version 1137341 (0.0009) [2023-12-26 23:38:56,298][105692] Updated weights for policy 0, policy_version 1136052 (0.0007) [2023-12-26 23:38:56,344][105620] Updated weights for policy 1, policy_version 1137351 (0.0008) [2023-12-26 23:38:56,355][105692] Updated weights for policy 0, policy_version 1136062 (0.0007) [2023-12-26 23:38:56,413][105692] Updated weights for policy 0, policy_version 1136072 (0.0007) [2023-12-26 23:38:57,106][105620] Updated weights for policy 1, policy_version 1137361 (0.0006) [2023-12-26 23:38:57,149][105692] Updated weights for policy 0, policy_version 1136082 (0.0007) [2023-12-26 23:38:57,155][105620] Updated weights for policy 1, policy_version 1137371 (0.0008) [2023-12-26 23:38:57,199][105692] Updated weights for policy 0, policy_version 1136092 (0.0006) [2023-12-26 23:38:57,201][105620] Updated weights for policy 1, policy_version 1137381 (0.0008) [2023-12-26 23:38:57,254][105692] Updated weights for policy 0, policy_version 1136102 (0.0006) [2023-12-26 23:38:57,260][105620] Updated weights for policy 1, policy_version 1137391 (0.0008) [2023-12-26 23:38:57,316][105692] Updated weights for policy 0, policy_version 1136112 (0.0008) [2023-12-26 23:38:57,869][105620] Updated weights for policy 1, policy_version 1137401 (0.0006) [2023-12-26 23:38:57,924][105620] Updated weights for policy 1, policy_version 1137411 (0.0008) [2023-12-26 23:38:57,937][105692] Updated weights for policy 0, policy_version 1136122 (0.0005) [2023-12-26 23:38:57,979][105620] Updated weights for policy 1, policy_version 1137421 (0.0008) [2023-12-26 23:38:57,991][105692] Updated weights for policy 0, policy_version 1136132 (0.0005) [2023-12-26 23:38:58,043][105692] Updated weights for policy 0, policy_version 1136142 (0.0007) [2023-12-26 23:38:58,649][105620] Updated weights for policy 1, policy_version 1137431 (0.0008) [2023-12-26 23:38:58,712][105620] Updated weights for policy 1, policy_version 1137441 (0.0006) [2023-12-26 23:38:58,784][105692] Updated weights for policy 0, policy_version 1136152 (0.0009) [2023-12-26 23:38:58,795][105620] Updated weights for policy 1, policy_version 1137451 (0.0007) [2023-12-26 23:38:58,855][105692] Updated weights for policy 0, policy_version 1136162 (0.0009) [2023-12-26 23:38:58,916][105692] Updated weights for policy 0, policy_version 1136172 (0.0008) [2023-12-26 23:38:59,596][105620] Updated weights for policy 1, policy_version 1137461 (0.0009) [2023-12-26 23:38:59,646][105620] Updated weights for policy 1, policy_version 1137471 (0.0007) [2023-12-26 23:38:59,702][105620] Updated weights for policy 1, policy_version 1137481 (0.0006) [2023-12-26 23:38:59,708][105692] Updated weights for policy 0, policy_version 1136182 (0.0009) [2023-12-26 23:38:59,755][105692] Updated weights for policy 0, policy_version 1136192 (0.0007) [2023-12-26 23:38:59,818][105692] Updated weights for policy 0, policy_version 1136202 (0.0008) [2023-12-26 23:39:00,366][105620] Updated weights for policy 1, policy_version 1137491 (0.0007) [2023-12-26 23:39:00,415][105620] Updated weights for policy 1, policy_version 1137501 (0.0009) [2023-12-26 23:39:00,463][105620] Updated weights for policy 1, policy_version 1137511 (0.0009) [2023-12-26 23:39:00,559][105692] Updated weights for policy 0, policy_version 1136212 (0.0008) [2023-12-26 23:39:00,608][105692] Updated weights for policy 0, policy_version 1136222 (0.0009) [2023-12-26 23:39:00,658][105692] Updated weights for policy 0, policy_version 1136232 (0.0009) [2023-12-26 23:39:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 582164480. Throughput: 0: 9720.8, 1: 9746.7. Samples: 582137096. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:39:01,063][104569] Avg episode reward: [(0, '9081.602'), (1, '8990.666')] [2023-12-26 23:39:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001136240_290922496.pth... [2023-12-26 23:39:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001137520_291241984.pth... [2023-12-26 23:39:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001135120_290635776.pth [2023-12-26 23:39:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001136400_290955264.pth [2023-12-26 23:39:01,251][105620] Updated weights for policy 1, policy_version 1137521 (0.0009) [2023-12-26 23:39:01,306][105620] Updated weights for policy 1, policy_version 1137531 (0.0010) [2023-12-26 23:39:01,369][105620] Updated weights for policy 1, policy_version 1137541 (0.0010) [2023-12-26 23:39:01,420][105692] Updated weights for policy 0, policy_version 1136242 (0.0008) [2023-12-26 23:39:01,437][105620] Updated weights for policy 1, policy_version 1137551 (0.0011) [2023-12-26 23:39:01,470][105692] Updated weights for policy 0, policy_version 1136252 (0.0007) [2023-12-26 23:39:01,520][105692] Updated weights for policy 0, policy_version 1136262 (0.0008) [2023-12-26 23:39:01,566][105692] Updated weights for policy 0, policy_version 1136272 (0.0008) [2023-12-26 23:39:02,196][105620] Updated weights for policy 1, policy_version 1137561 (0.0010) [2023-12-26 23:39:02,253][105620] Updated weights for policy 1, policy_version 1137571 (0.0009) [2023-12-26 23:39:02,289][105692] Updated weights for policy 0, policy_version 1136282 (0.0007) [2023-12-26 23:39:02,313][105620] Updated weights for policy 1, policy_version 1137581 (0.0006) [2023-12-26 23:39:02,354][105692] Updated weights for policy 0, policy_version 1136292 (0.0010) [2023-12-26 23:39:02,417][105692] Updated weights for policy 0, policy_version 1136302 (0.0010) [2023-12-26 23:39:02,959][105620] Updated weights for policy 1, policy_version 1137591 (0.0009) [2023-12-26 23:39:03,016][105620] Updated weights for policy 1, policy_version 1137601 (0.0010) [2023-12-26 23:39:03,066][105620] Updated weights for policy 1, policy_version 1137611 (0.0005) [2023-12-26 23:39:03,160][105692] Updated weights for policy 0, policy_version 1136312 (0.0006) [2023-12-26 23:39:03,218][105692] Updated weights for policy 0, policy_version 1136322 (0.0006) [2023-12-26 23:39:03,274][105692] Updated weights for policy 0, policy_version 1136332 (0.0008) [2023-12-26 23:39:03,705][105620] Updated weights for policy 1, policy_version 1137621 (0.0008) [2023-12-26 23:39:03,756][105620] Updated weights for policy 1, policy_version 1137631 (0.0010) [2023-12-26 23:39:03,803][105692] Updated weights for policy 0, policy_version 1136342 (0.0007) [2023-12-26 23:39:03,814][105620] Updated weights for policy 1, policy_version 1137641 (0.0009) [2023-12-26 23:39:03,868][105692] Updated weights for policy 0, policy_version 1136352 (0.0010) [2023-12-26 23:39:03,916][105692] Updated weights for policy 0, policy_version 1136362 (0.0010) [2023-12-26 23:39:04,603][105620] Updated weights for policy 1, policy_version 1137651 (0.0007) [2023-12-26 23:39:04,603][105692] Updated weights for policy 0, policy_version 1136372 (0.0009) [2023-12-26 23:39:04,651][105692] Updated weights for policy 0, policy_version 1136382 (0.0007) [2023-12-26 23:39:04,653][105620] Updated weights for policy 1, policy_version 1137661 (0.0009) [2023-12-26 23:39:04,704][105692] Updated weights for policy 0, policy_version 1136392 (0.0007) [2023-12-26 23:39:04,708][105620] Updated weights for policy 1, policy_version 1137671 (0.0006) [2023-12-26 23:39:05,310][105620] Updated weights for policy 1, policy_version 1137681 (0.0006) [2023-12-26 23:39:05,370][105620] Updated weights for policy 1, policy_version 1137691 (0.0009) [2023-12-26 23:39:05,429][105620] Updated weights for policy 1, policy_version 1137701 (0.0011) [2023-12-26 23:39:05,443][105692] Updated weights for policy 0, policy_version 1136402 (0.0009) [2023-12-26 23:39:05,485][105620] Updated weights for policy 1, policy_version 1137711 (0.0009) [2023-12-26 23:39:05,503][105692] Updated weights for policy 0, policy_version 1136412 (0.0007) [2023-12-26 23:39:05,526][105585] KL-divergence is very high: 111.7634 [2023-12-26 23:39:05,556][105692] Updated weights for policy 0, policy_version 1136422 (0.0008) [2023-12-26 23:39:05,609][105692] Updated weights for policy 0, policy_version 1136432 (0.0008) [2023-12-26 23:39:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.6, 300 sec: 19466.4). Total num frames: 582262784. Throughput: 0: 9730.5, 1: 9811.5. Samples: 582253728. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:39:06,063][104569] Avg episode reward: [(0, '9079.178'), (1, '8810.015')] [2023-12-26 23:39:06,125][105620] Updated weights for policy 1, policy_version 1137721 (0.0011) [2023-12-26 23:39:06,180][105620] Updated weights for policy 1, policy_version 1137731 (0.0007) [2023-12-26 23:39:06,244][105620] Updated weights for policy 1, policy_version 1137741 (0.0006) [2023-12-26 23:39:06,478][105692] Updated weights for policy 0, policy_version 1136442 (0.0008) [2023-12-26 23:39:06,547][105692] Updated weights for policy 0, policy_version 1136452 (0.0008) [2023-12-26 23:39:06,619][105692] Updated weights for policy 0, policy_version 1136462 (0.0009) [2023-12-26 23:39:06,868][105620] Updated weights for policy 1, policy_version 1137751 (0.0009) [2023-12-26 23:39:06,913][105620] Updated weights for policy 1, policy_version 1137761 (0.0010) [2023-12-26 23:39:06,962][105620] Updated weights for policy 1, policy_version 1137771 (0.0010) [2023-12-26 23:39:07,366][105692] Updated weights for policy 0, policy_version 1136472 (0.0009) [2023-12-26 23:39:07,430][105692] Updated weights for policy 0, policy_version 1136482 (0.0009) [2023-12-26 23:39:07,488][105692] Updated weights for policy 0, policy_version 1136492 (0.0009) [2023-12-26 23:39:07,719][105620] Updated weights for policy 1, policy_version 1137781 (0.0010) [2023-12-26 23:39:07,766][105620] Updated weights for policy 1, policy_version 1137791 (0.0009) [2023-12-26 23:39:07,812][105620] Updated weights for policy 1, policy_version 1137801 (0.0008) [2023-12-26 23:39:08,220][105692] Updated weights for policy 0, policy_version 1136502 (0.0009) [2023-12-26 23:39:08,267][105692] Updated weights for policy 0, policy_version 1136512 (0.0008) [2023-12-26 23:39:08,318][105692] Updated weights for policy 0, policy_version 1136522 (0.0009) [2023-12-26 23:39:08,619][105620] Updated weights for policy 1, policy_version 1137811 (0.0009) [2023-12-26 23:39:08,683][105620] Updated weights for policy 1, policy_version 1137821 (0.0009) [2023-12-26 23:39:08,739][105620] Updated weights for policy 1, policy_version 1137831 (0.0009) [2023-12-26 23:39:08,989][105692] Updated weights for policy 0, policy_version 1136532 (0.0008) [2023-12-26 23:39:09,043][105692] Updated weights for policy 0, policy_version 1136542 (0.0008) [2023-12-26 23:39:09,097][105692] Updated weights for policy 0, policy_version 1136552 (0.0010) [2023-12-26 23:39:09,483][105620] Updated weights for policy 1, policy_version 1137841 (0.0008) [2023-12-26 23:39:09,539][105620] Updated weights for policy 1, policy_version 1137851 (0.0008) [2023-12-26 23:39:09,591][105620] Updated weights for policy 1, policy_version 1137861 (0.0008) [2023-12-26 23:39:09,637][105620] Updated weights for policy 1, policy_version 1137871 (0.0008) [2023-12-26 23:39:09,903][105692] Updated weights for policy 0, policy_version 1136562 (0.0010) [2023-12-26 23:39:09,962][105692] Updated weights for policy 0, policy_version 1136572 (0.0009) [2023-12-26 23:39:10,024][105692] Updated weights for policy 0, policy_version 1136582 (0.0009) [2023-12-26 23:39:10,089][105692] Updated weights for policy 0, policy_version 1136592 (0.0009) [2023-12-26 23:39:10,467][105620] Updated weights for policy 1, policy_version 1137881 (0.0009) [2023-12-26 23:39:10,514][105620] Updated weights for policy 1, policy_version 1137891 (0.0009) [2023-12-26 23:39:10,561][105620] Updated weights for policy 1, policy_version 1137901 (0.0008) [2023-12-26 23:39:10,731][105692] Updated weights for policy 0, policy_version 1136602 (0.0006) [2023-12-26 23:39:10,785][105692] Updated weights for policy 0, policy_version 1136612 (0.0005) [2023-12-26 23:39:10,847][105692] Updated weights for policy 0, policy_version 1136622 (0.0008) [2023-12-26 23:39:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 582361088. Throughput: 0: 9647.6, 1: 9756.5. Samples: 582368284. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:39:11,062][104569] Avg episode reward: [(0, '8897.911'), (1, '8264.666')] [2023-12-26 23:39:11,393][105620] Updated weights for policy 1, policy_version 1137911 (0.0008) [2023-12-26 23:39:11,459][105620] Updated weights for policy 1, policy_version 1137921 (0.0008) [2023-12-26 23:39:11,515][105620] Updated weights for policy 1, policy_version 1137931 (0.0008) [2023-12-26 23:39:11,654][105692] Updated weights for policy 0, policy_version 1136632 (0.0010) [2023-12-26 23:39:11,726][105692] Updated weights for policy 0, policy_version 1136642 (0.0017) [2023-12-26 23:39:11,780][105692] Updated weights for policy 0, policy_version 1136652 (0.0011) [2023-12-26 23:39:12,330][105620] Updated weights for policy 1, policy_version 1137941 (0.0008) [2023-12-26 23:39:12,396][105620] Updated weights for policy 1, policy_version 1137951 (0.0006) [2023-12-26 23:39:12,442][105620] Updated weights for policy 1, policy_version 1137961 (0.0008) [2023-12-26 23:39:12,532][105692] Updated weights for policy 0, policy_version 1136662 (0.0007) [2023-12-26 23:39:12,594][105692] Updated weights for policy 0, policy_version 1136672 (0.0006) [2023-12-26 23:39:12,663][105692] Updated weights for policy 0, policy_version 1136682 (0.0009) [2023-12-26 23:39:13,105][105620] Updated weights for policy 1, policy_version 1137971 (0.0007) [2023-12-26 23:39:13,160][105620] Updated weights for policy 1, policy_version 1137981 (0.0007) [2023-12-26 23:39:13,208][105620] Updated weights for policy 1, policy_version 1137991 (0.0010) [2023-12-26 23:39:13,367][105692] Updated weights for policy 0, policy_version 1136692 (0.0010) [2023-12-26 23:39:13,421][105692] Updated weights for policy 0, policy_version 1136702 (0.0005) [2023-12-26 23:39:13,467][105692] Updated weights for policy 0, policy_version 1136712 (0.0005) [2023-12-26 23:39:13,906][105620] Updated weights for policy 1, policy_version 1138001 (0.0010) [2023-12-26 23:39:13,954][105620] Updated weights for policy 1, policy_version 1138011 (0.0005) [2023-12-26 23:39:14,001][105620] Updated weights for policy 1, policy_version 1138021 (0.0006) [2023-12-26 23:39:14,057][105620] Updated weights for policy 1, policy_version 1138031 (0.0005) [2023-12-26 23:39:14,135][105692] Updated weights for policy 0, policy_version 1136722 (0.0006) [2023-12-26 23:39:14,194][105692] Updated weights for policy 0, policy_version 1136732 (0.0010) [2023-12-26 23:39:14,266][105692] Updated weights for policy 0, policy_version 1136742 (0.0010) [2023-12-26 23:39:14,326][105692] Updated weights for policy 0, policy_version 1136752 (0.0010) [2023-12-26 23:39:14,682][105620] Updated weights for policy 1, policy_version 1138041 (0.0005) [2023-12-26 23:39:14,743][105620] Updated weights for policy 1, policy_version 1138051 (0.0005) [2023-12-26 23:39:14,801][105620] Updated weights for policy 1, policy_version 1138061 (0.0008) [2023-12-26 23:39:14,909][105692] Updated weights for policy 0, policy_version 1136762 (0.0009) [2023-12-26 23:39:14,962][105692] Updated weights for policy 0, policy_version 1136772 (0.0011) [2023-12-26 23:39:15,025][105692] Updated weights for policy 0, policy_version 1136782 (0.0010) [2023-12-26 23:39:15,449][105620] Updated weights for policy 1, policy_version 1138071 (0.0008) [2023-12-26 23:39:15,502][105620] Updated weights for policy 1, policy_version 1138081 (0.0008) [2023-12-26 23:39:15,559][105620] Updated weights for policy 1, policy_version 1138091 (0.0008) [2023-12-26 23:39:15,797][105692] Updated weights for policy 0, policy_version 1136792 (0.0010) [2023-12-26 23:39:15,852][105692] Updated weights for policy 0, policy_version 1136802 (0.0010) [2023-12-26 23:39:15,908][105692] Updated weights for policy 0, policy_version 1136812 (0.0009) [2023-12-26 23:39:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 582459392. Throughput: 0: 9550.4, 1: 9700.3. Samples: 582425716. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:39:16,063][104569] Avg episode reward: [(0, '9081.860'), (1, '3006.859')] [2023-12-26 23:39:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001136816_291069952.pth... [2023-12-26 23:39:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001138096_291389440.pth... [2023-12-26 23:39:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001136944_291094528.pth [2023-12-26 23:39:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001135696_290783232.pth [2023-12-26 23:39:16,195][105620] Updated weights for policy 1, policy_version 1138101 (0.0007) [2023-12-26 23:39:16,258][105620] Updated weights for policy 1, policy_version 1138111 (0.0005) [2023-12-26 23:39:16,315][105620] Updated weights for policy 1, policy_version 1138121 (0.0005) [2023-12-26 23:39:16,830][105692] Updated weights for policy 0, policy_version 1136823 (0.0007) [2023-12-26 23:39:16,835][105620] Updated weights for policy 1, policy_version 1138131 (0.0007) [2023-12-26 23:39:16,886][105692] Updated weights for policy 0, policy_version 1136833 (0.0005) [2023-12-26 23:39:16,887][105620] Updated weights for policy 1, policy_version 1138141 (0.0007) [2023-12-26 23:39:16,938][105692] Updated weights for policy 0, policy_version 1136843 (0.0007) [2023-12-26 23:39:16,951][105620] Updated weights for policy 1, policy_version 1138151 (0.0005) [2023-12-26 23:39:17,597][105692] Updated weights for policy 0, policy_version 1136853 (0.0007) [2023-12-26 23:39:17,657][105692] Updated weights for policy 0, policy_version 1136863 (0.0007) [2023-12-26 23:39:17,670][105620] Updated weights for policy 1, policy_version 1138161 (0.0006) [2023-12-26 23:39:17,715][105692] Updated weights for policy 0, policy_version 1136873 (0.0008) [2023-12-26 23:39:17,726][105620] Updated weights for policy 1, policy_version 1138171 (0.0007) [2023-12-26 23:39:17,773][105620] Updated weights for policy 1, policy_version 1138181 (0.0007) [2023-12-26 23:39:17,820][105620] Updated weights for policy 1, policy_version 1138191 (0.0008) [2023-12-26 23:39:18,437][105692] Updated weights for policy 0, policy_version 1136883 (0.0007) [2023-12-26 23:39:18,502][105692] Updated weights for policy 0, policy_version 1136893 (0.0007) [2023-12-26 23:39:18,508][105620] Updated weights for policy 1, policy_version 1138201 (0.0010) [2023-12-26 23:39:18,566][105692] Updated weights for policy 0, policy_version 1136903 (0.0006) [2023-12-26 23:39:18,567][105620] Updated weights for policy 1, policy_version 1138211 (0.0010) [2023-12-26 23:39:18,627][105620] Updated weights for policy 1, policy_version 1138221 (0.0011) [2023-12-26 23:39:19,306][105692] Updated weights for policy 0, policy_version 1136913 (0.0006) [2023-12-26 23:39:19,358][105620] Updated weights for policy 1, policy_version 1138231 (0.0010) [2023-12-26 23:39:19,378][105692] Updated weights for policy 0, policy_version 1136923 (0.0009) [2023-12-26 23:39:19,420][105620] Updated weights for policy 1, policy_version 1138241 (0.0008) [2023-12-26 23:39:19,434][105692] Updated weights for policy 0, policy_version 1136933 (0.0007) [2023-12-26 23:39:19,478][105620] Updated weights for policy 1, policy_version 1138251 (0.0008) [2023-12-26 23:39:19,493][105692] Updated weights for policy 0, policy_version 1136943 (0.0008) [2023-12-26 23:39:20,199][105692] Updated weights for policy 0, policy_version 1136953 (0.0006) [2023-12-26 23:39:20,258][105620] Updated weights for policy 1, policy_version 1138261 (0.0009) [2023-12-26 23:39:20,262][105692] Updated weights for policy 0, policy_version 1136963 (0.0006) [2023-12-26 23:39:20,316][105620] Updated weights for policy 1, policy_version 1138271 (0.0007) [2023-12-26 23:39:20,319][105692] Updated weights for policy 0, policy_version 1136973 (0.0006) [2023-12-26 23:39:20,378][105620] Updated weights for policy 1, policy_version 1138281 (0.0010) [2023-12-26 23:39:20,948][105692] Updated weights for policy 0, policy_version 1136983 (0.0006) [2023-12-26 23:39:21,018][105692] Updated weights for policy 0, policy_version 1136993 (0.0006) [2023-12-26 23:39:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 582549504. Throughput: 0: 9569.0, 1: 9793.7. Samples: 582545164. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:39:21,063][104569] Avg episode reward: [(0, '9082.622'), (1, '5711.729')] [2023-12-26 23:39:21,082][105692] Updated weights for policy 0, policy_version 1137003 (0.0009) [2023-12-26 23:39:21,244][105620] Updated weights for policy 1, policy_version 1138291 (0.0009) [2023-12-26 23:39:21,313][105620] Updated weights for policy 1, policy_version 1138301 (0.0007) [2023-12-26 23:39:21,389][105620] Updated weights for policy 1, policy_version 1138311 (0.0008) [2023-12-26 23:39:21,903][105692] Updated weights for policy 0, policy_version 1137013 (0.0009) [2023-12-26 23:39:21,975][105692] Updated weights for policy 0, policy_version 1137023 (0.0008) [2023-12-26 23:39:22,032][105692] Updated weights for policy 0, policy_version 1137033 (0.0009) [2023-12-26 23:39:22,060][105620] Updated weights for policy 1, policy_version 1138321 (0.0006) [2023-12-26 23:39:22,110][105620] Updated weights for policy 1, policy_version 1138331 (0.0008) [2023-12-26 23:39:22,166][105620] Updated weights for policy 1, policy_version 1138341 (0.0010) [2023-12-26 23:39:22,220][105620] Updated weights for policy 1, policy_version 1138351 (0.0008) [2023-12-26 23:39:22,805][105692] Updated weights for policy 0, policy_version 1137043 (0.0008) [2023-12-26 23:39:22,871][105692] Updated weights for policy 0, policy_version 1137053 (0.0010) [2023-12-26 23:39:22,935][105692] Updated weights for policy 0, policy_version 1137063 (0.0009) [2023-12-26 23:39:23,007][105620] Updated weights for policy 1, policy_version 1138361 (0.0007) [2023-12-26 23:39:23,070][105620] Updated weights for policy 1, policy_version 1138371 (0.0008) [2023-12-26 23:39:23,126][105620] Updated weights for policy 1, policy_version 1138381 (0.0007) [2023-12-26 23:39:23,717][105620] Updated weights for policy 1, policy_version 1138391 (0.0011) [2023-12-26 23:39:23,782][105620] Updated weights for policy 1, policy_version 1138401 (0.0010) [2023-12-26 23:39:23,784][105692] Updated weights for policy 0, policy_version 1137073 (0.0009) [2023-12-26 23:39:23,838][105692] Updated weights for policy 0, policy_version 1137083 (0.0008) [2023-12-26 23:39:23,841][105620] Updated weights for policy 1, policy_version 1138411 (0.0011) [2023-12-26 23:39:23,891][105692] Updated weights for policy 0, policy_version 1137093 (0.0008) [2023-12-26 23:39:23,938][105692] Updated weights for policy 0, policy_version 1137103 (0.0009) [2023-12-26 23:39:24,561][105692] Updated weights for policy 0, policy_version 1137113 (0.0010) [2023-12-26 23:39:24,595][105620] Updated weights for policy 1, policy_version 1138421 (0.0008) [2023-12-26 23:39:24,620][105692] Updated weights for policy 0, policy_version 1137123 (0.0010) [2023-12-26 23:39:24,654][105620] Updated weights for policy 1, policy_version 1138431 (0.0005) [2023-12-26 23:39:24,679][105692] Updated weights for policy 0, policy_version 1137133 (0.0010) [2023-12-26 23:39:24,713][105620] Updated weights for policy 1, policy_version 1138441 (0.0006) [2023-12-26 23:39:25,328][105620] Updated weights for policy 1, policy_version 1138451 (0.0006) [2023-12-26 23:39:25,389][105620] Updated weights for policy 1, policy_version 1138461 (0.0010) [2023-12-26 23:39:25,391][105692] Updated weights for policy 0, policy_version 1137143 (0.0007) [2023-12-26 23:39:25,444][105692] Updated weights for policy 0, policy_version 1137153 (0.0006) [2023-12-26 23:39:25,450][105620] Updated weights for policy 1, policy_version 1138471 (0.0009) [2023-12-26 23:39:25,494][105692] Updated weights for policy 0, policy_version 1137163 (0.0006) [2023-12-26 23:39:26,037][105620] Updated weights for policy 1, policy_version 1138481 (0.0009) [2023-12-26 23:39:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 582647808. Throughput: 0: 9542.1, 1: 9844.1. Samples: 582661652. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:39:26,063][104569] Avg episode reward: [(0, '9081.920'), (1, '7828.821')] [2023-12-26 23:39:26,105][105620] Updated weights for policy 1, policy_version 1138491 (0.0007) [2023-12-26 23:39:26,150][105692] Updated weights for policy 0, policy_version 1137173 (0.0006) [2023-12-26 23:39:26,175][105620] Updated weights for policy 1, policy_version 1138501 (0.0005) [2023-12-26 23:39:26,205][105692] Updated weights for policy 0, policy_version 1137183 (0.0006) [2023-12-26 23:39:26,239][105620] Updated weights for policy 1, policy_version 1138511 (0.0008) [2023-12-26 23:39:26,255][105692] Updated weights for policy 0, policy_version 1137193 (0.0005) [2023-12-26 23:39:26,794][105620] Updated weights for policy 1, policy_version 1138521 (0.0006) [2023-12-26 23:39:26,851][105620] Updated weights for policy 1, policy_version 1138531 (0.0006) [2023-12-26 23:39:26,911][105620] Updated weights for policy 1, policy_version 1138541 (0.0005) [2023-12-26 23:39:26,984][105692] Updated weights for policy 0, policy_version 1137203 (0.0009) [2023-12-26 23:39:27,034][105692] Updated weights for policy 0, policy_version 1137214 (0.0008) [2023-12-26 23:39:27,096][105692] Updated weights for policy 0, policy_version 1137224 (0.0009) [2023-12-26 23:39:27,521][105620] Updated weights for policy 1, policy_version 1138551 (0.0008) [2023-12-26 23:39:27,567][105620] Updated weights for policy 1, policy_version 1138561 (0.0005) [2023-12-26 23:39:27,635][105620] Updated weights for policy 1, policy_version 1138571 (0.0005) [2023-12-26 23:39:27,775][105692] Updated weights for policy 0, policy_version 1137234 (0.0009) [2023-12-26 23:39:27,828][105692] Updated weights for policy 0, policy_version 1137244 (0.0009) [2023-12-26 23:39:27,890][105692] Updated weights for policy 0, policy_version 1137254 (0.0006) [2023-12-26 23:39:27,940][105692] Updated weights for policy 0, policy_version 1137264 (0.0005) [2023-12-26 23:39:28,165][105620] Updated weights for policy 1, policy_version 1138581 (0.0006) [2023-12-26 23:39:28,217][105620] Updated weights for policy 1, policy_version 1138591 (0.0008) [2023-12-26 23:39:28,276][105620] Updated weights for policy 1, policy_version 1138601 (0.0005) [2023-12-26 23:39:28,529][105692] Updated weights for policy 0, policy_version 1137274 (0.0006) [2023-12-26 23:39:28,587][105692] Updated weights for policy 0, policy_version 1137284 (0.0010) [2023-12-26 23:39:28,641][105692] Updated weights for policy 0, policy_version 1137294 (0.0010) [2023-12-26 23:39:28,936][105620] Updated weights for policy 1, policy_version 1138611 (0.0007) [2023-12-26 23:39:28,986][105620] Updated weights for policy 1, policy_version 1138621 (0.0008) [2023-12-26 23:39:29,032][105620] Updated weights for policy 1, policy_version 1138631 (0.0008) [2023-12-26 23:39:29,387][105692] Updated weights for policy 0, policy_version 1137304 (0.0010) [2023-12-26 23:39:29,438][105692] Updated weights for policy 0, policy_version 1137314 (0.0010) [2023-12-26 23:39:29,493][105692] Updated weights for policy 0, policy_version 1137324 (0.0010) [2023-12-26 23:39:29,779][105620] Updated weights for policy 1, policy_version 1138641 (0.0009) [2023-12-26 23:39:29,848][105620] Updated weights for policy 1, policy_version 1138651 (0.0008) [2023-12-26 23:39:29,912][105620] Updated weights for policy 1, policy_version 1138661 (0.0006) [2023-12-26 23:39:29,974][105620] Updated weights for policy 1, policy_version 1138671 (0.0007) [2023-12-26 23:39:30,214][105692] Updated weights for policy 0, policy_version 1137334 (0.0010) [2023-12-26 23:39:30,276][105692] Updated weights for policy 0, policy_version 1137344 (0.0010) [2023-12-26 23:39:30,338][105692] Updated weights for policy 0, policy_version 1137354 (0.0008) [2023-12-26 23:39:30,686][105620] Updated weights for policy 1, policy_version 1138681 (0.0009) [2023-12-26 23:39:30,745][105620] Updated weights for policy 1, policy_version 1138691 (0.0009) [2023-12-26 23:39:30,799][105620] Updated weights for policy 1, policy_version 1138701 (0.0009) [2023-12-26 23:39:30,961][105692] Updated weights for policy 0, policy_version 1137364 (0.0010) [2023-12-26 23:39:31,018][105692] Updated weights for policy 0, policy_version 1137374 (0.0009) [2023-12-26 23:39:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 582754304. Throughput: 0: 9654.2, 1: 9949.8. Samples: 582727132. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:39:31,062][104569] Avg episode reward: [(0, '9262.061'), (1, '8990.214')] [2023-12-26 23:39:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001138704_291545088.pth... [2023-12-26 23:39:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001137520_291241984.pth [2023-12-26 23:39:31,081][105692] Updated weights for policy 0, policy_version 1137384 (0.0008) [2023-12-26 23:39:31,122][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001137392_291217408.pth... [2023-12-26 23:39:31,127][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001136240_290922496.pth [2023-12-26 23:39:31,584][105620] Updated weights for policy 1, policy_version 1138711 (0.0009) [2023-12-26 23:39:31,652][105620] Updated weights for policy 1, policy_version 1138721 (0.0009) [2023-12-26 23:39:31,712][105620] Updated weights for policy 1, policy_version 1138731 (0.0009) [2023-12-26 23:39:31,882][105692] Updated weights for policy 0, policy_version 1137394 (0.0008) [2023-12-26 23:39:31,933][105692] Updated weights for policy 0, policy_version 1137404 (0.0009) [2023-12-26 23:39:31,978][105692] Updated weights for policy 0, policy_version 1137414 (0.0008) [2023-12-26 23:39:32,025][105692] Updated weights for policy 0, policy_version 1137424 (0.0008) [2023-12-26 23:39:32,433][105620] Updated weights for policy 1, policy_version 1138741 (0.0008) [2023-12-26 23:39:32,492][105620] Updated weights for policy 1, policy_version 1138751 (0.0006) [2023-12-26 23:39:32,553][105620] Updated weights for policy 1, policy_version 1138761 (0.0008) [2023-12-26 23:39:32,728][105692] Updated weights for policy 0, policy_version 1137434 (0.0009) [2023-12-26 23:39:32,780][105692] Updated weights for policy 0, policy_version 1137444 (0.0010) [2023-12-26 23:39:32,834][105692] Updated weights for policy 0, policy_version 1137454 (0.0007) [2023-12-26 23:39:33,161][105620] Updated weights for policy 1, policy_version 1138771 (0.0009) [2023-12-26 23:39:33,207][105620] Updated weights for policy 1, policy_version 1138781 (0.0010) [2023-12-26 23:39:33,254][105620] Updated weights for policy 1, policy_version 1138791 (0.0010) [2023-12-26 23:39:33,443][105692] Updated weights for policy 0, policy_version 1137464 (0.0005) [2023-12-26 23:39:33,493][105692] Updated weights for policy 0, policy_version 1137474 (0.0009) [2023-12-26 23:39:33,550][105692] Updated weights for policy 0, policy_version 1137484 (0.0006) [2023-12-26 23:39:33,941][105620] Updated weights for policy 1, policy_version 1138801 (0.0010) [2023-12-26 23:39:33,991][105620] Updated weights for policy 1, policy_version 1138811 (0.0006) [2023-12-26 23:39:34,041][105620] Updated weights for policy 1, policy_version 1138821 (0.0005) [2023-12-26 23:39:34,088][105620] Updated weights for policy 1, policy_version 1138831 (0.0005) [2023-12-26 23:39:34,130][105692] Updated weights for policy 0, policy_version 1137494 (0.0005) [2023-12-26 23:39:34,201][105692] Updated weights for policy 0, policy_version 1137504 (0.0008) [2023-12-26 23:39:34,254][105692] Updated weights for policy 0, policy_version 1137514 (0.0010) [2023-12-26 23:39:34,725][105620] Updated weights for policy 1, policy_version 1138841 (0.0009) [2023-12-26 23:39:34,779][105620] Updated weights for policy 1, policy_version 1138851 (0.0008) [2023-12-26 23:39:34,841][105620] Updated weights for policy 1, policy_version 1138861 (0.0009) [2023-12-26 23:39:35,007][105692] Updated weights for policy 0, policy_version 1137524 (0.0009) [2023-12-26 23:39:35,066][105692] Updated weights for policy 0, policy_version 1137534 (0.0009) [2023-12-26 23:39:35,119][105692] Updated weights for policy 0, policy_version 1137544 (0.0009) [2023-12-26 23:39:35,587][105620] Updated weights for policy 1, policy_version 1138871 (0.0009) [2023-12-26 23:39:35,641][105620] Updated weights for policy 1, policy_version 1138881 (0.0010) [2023-12-26 23:39:35,696][105620] Updated weights for policy 1, policy_version 1138891 (0.0009) [2023-12-26 23:39:35,752][105692] Updated weights for policy 0, policy_version 1137554 (0.0008) [2023-12-26 23:39:35,806][105692] Updated weights for policy 0, policy_version 1137564 (0.0009) [2023-12-26 23:39:35,872][105692] Updated weights for policy 0, policy_version 1137574 (0.0009) [2023-12-26 23:39:35,919][105692] Updated weights for policy 0, policy_version 1137584 (0.0009) [2023-12-26 23:39:36,062][104569] Fps is (10 sec: 21299.3, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 582860800. Throughput: 0: 9674.4, 1: 9955.5. Samples: 582846696. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:39:36,063][104569] Avg episode reward: [(0, '9172.782'), (1, '9079.541')] [2023-12-26 23:39:36,405][105620] Updated weights for policy 1, policy_version 1138901 (0.0008) [2023-12-26 23:39:36,460][105620] Updated weights for policy 1, policy_version 1138911 (0.0009) [2023-12-26 23:39:36,508][105620] Updated weights for policy 1, policy_version 1138921 (0.0009) [2023-12-26 23:39:36,728][105692] Updated weights for policy 0, policy_version 1137594 (0.0011) [2023-12-26 23:39:36,791][105692] Updated weights for policy 0, policy_version 1137604 (0.0010) [2023-12-26 23:39:36,854][105692] Updated weights for policy 0, policy_version 1137614 (0.0011) [2023-12-26 23:39:37,277][105620] Updated weights for policy 1, policy_version 1138931 (0.0008) [2023-12-26 23:39:37,333][105620] Updated weights for policy 1, policy_version 1138941 (0.0005) [2023-12-26 23:39:37,390][105620] Updated weights for policy 1, policy_version 1138951 (0.0006) [2023-12-26 23:39:37,577][105692] Updated weights for policy 0, policy_version 1137624 (0.0011) [2023-12-26 23:39:37,647][105692] Updated weights for policy 0, policy_version 1137634 (0.0008) [2023-12-26 23:39:37,716][105692] Updated weights for policy 0, policy_version 1137644 (0.0005) [2023-12-26 23:39:37,960][105620] Updated weights for policy 1, policy_version 1138961 (0.0006) [2023-12-26 23:39:38,004][105620] Updated weights for policy 1, policy_version 1138971 (0.0009) [2023-12-26 23:39:38,059][105620] Updated weights for policy 1, policy_version 1138981 (0.0010) [2023-12-26 23:39:38,117][105620] Updated weights for policy 1, policy_version 1138991 (0.0010) [2023-12-26 23:39:38,311][105692] Updated weights for policy 0, policy_version 1137654 (0.0005) [2023-12-26 23:39:38,379][105692] Updated weights for policy 0, policy_version 1137664 (0.0009) [2023-12-26 23:39:38,439][105692] Updated weights for policy 0, policy_version 1137674 (0.0011) [2023-12-26 23:39:38,885][105620] Updated weights for policy 1, policy_version 1139001 (0.0008) [2023-12-26 23:39:38,944][105620] Updated weights for policy 1, policy_version 1139011 (0.0007) [2023-12-26 23:39:39,002][105620] Updated weights for policy 1, policy_version 1139021 (0.0007) [2023-12-26 23:39:39,165][105692] Updated weights for policy 0, policy_version 1137684 (0.0011) [2023-12-26 23:39:39,232][105692] Updated weights for policy 0, policy_version 1137694 (0.0011) [2023-12-26 23:39:39,298][105692] Updated weights for policy 0, policy_version 1137704 (0.0011) [2023-12-26 23:39:39,627][105620] Updated weights for policy 1, policy_version 1139031 (0.0007) [2023-12-26 23:39:39,686][105620] Updated weights for policy 1, policy_version 1139041 (0.0009) [2023-12-26 23:39:39,748][105620] Updated weights for policy 1, policy_version 1139051 (0.0009) [2023-12-26 23:39:40,105][105692] Updated weights for policy 0, policy_version 1137714 (0.0010) [2023-12-26 23:39:40,168][105692] Updated weights for policy 0, policy_version 1137724 (0.0009) [2023-12-26 23:39:40,236][105692] Updated weights for policy 0, policy_version 1137734 (0.0010) [2023-12-26 23:39:40,294][105692] Updated weights for policy 0, policy_version 1137744 (0.0009) [2023-12-26 23:39:40,467][105620] Updated weights for policy 1, policy_version 1139061 (0.0009) [2023-12-26 23:39:40,519][105620] Updated weights for policy 1, policy_version 1139071 (0.0008) [2023-12-26 23:39:40,566][105620] Updated weights for policy 1, policy_version 1139081 (0.0008) [2023-12-26 23:39:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 582950912. Throughput: 0: 9697.5, 1: 9990.6. Samples: 582963644. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:39:41,062][104569] Avg episode reward: [(0, '8989.565'), (1, '9076.116')] [2023-12-26 23:39:41,063][105692] Updated weights for policy 0, policy_version 1137754 (0.0011) [2023-12-26 23:39:41,133][105692] Updated weights for policy 0, policy_version 1137764 (0.0011) [2023-12-26 23:39:41,193][105692] Updated weights for policy 0, policy_version 1137774 (0.0010) [2023-12-26 23:39:41,399][105620] Updated weights for policy 1, policy_version 1139091 (0.0009) [2023-12-26 23:39:41,467][105620] Updated weights for policy 1, policy_version 1139101 (0.0009) [2023-12-26 23:39:41,536][105620] Updated weights for policy 1, policy_version 1139111 (0.0008) [2023-12-26 23:39:41,956][105692] Updated weights for policy 0, policy_version 1137784 (0.0010) [2023-12-26 23:39:42,022][105692] Updated weights for policy 0, policy_version 1137794 (0.0010) [2023-12-26 23:39:42,086][105692] Updated weights for policy 0, policy_version 1137804 (0.0010) [2023-12-26 23:39:42,293][105620] Updated weights for policy 1, policy_version 1139121 (0.0008) [2023-12-26 23:39:42,354][105620] Updated weights for policy 1, policy_version 1139131 (0.0008) [2023-12-26 23:39:42,421][105620] Updated weights for policy 1, policy_version 1139141 (0.0008) [2023-12-26 23:39:42,480][105620] Updated weights for policy 1, policy_version 1139151 (0.0008) [2023-12-26 23:39:42,825][105692] Updated weights for policy 0, policy_version 1137814 (0.0010) [2023-12-26 23:39:42,891][105692] Updated weights for policy 0, policy_version 1137824 (0.0010) [2023-12-26 23:39:42,950][105692] Updated weights for policy 0, policy_version 1137834 (0.0010) [2023-12-26 23:39:43,256][105620] Updated weights for policy 1, policy_version 1139161 (0.0008) [2023-12-26 23:39:43,306][105620] Updated weights for policy 1, policy_version 1139171 (0.0008) [2023-12-26 23:39:43,361][105620] Updated weights for policy 1, policy_version 1139182 (0.0009) [2023-12-26 23:39:43,702][105692] Updated weights for policy 0, policy_version 1137844 (0.0010) [2023-12-26 23:39:43,749][105692] Updated weights for policy 0, policy_version 1137854 (0.0010) [2023-12-26 23:39:43,800][105692] Updated weights for policy 0, policy_version 1137864 (0.0010) [2023-12-26 23:39:44,156][105620] Updated weights for policy 1, policy_version 1139192 (0.0007) [2023-12-26 23:39:44,213][105620] Updated weights for policy 1, policy_version 1139202 (0.0008) [2023-12-26 23:39:44,274][105620] Updated weights for policy 1, policy_version 1139212 (0.0008) [2023-12-26 23:39:44,560][105692] Updated weights for policy 0, policy_version 1137874 (0.0010) [2023-12-26 23:39:44,615][105692] Updated weights for policy 0, policy_version 1137884 (0.0010) [2023-12-26 23:39:44,665][105692] Updated weights for policy 0, policy_version 1137894 (0.0010) [2023-12-26 23:39:44,716][105692] Updated weights for policy 0, policy_version 1137904 (0.0010) [2023-12-26 23:39:45,050][105620] Updated weights for policy 1, policy_version 1139222 (0.0008) [2023-12-26 23:39:45,116][105620] Updated weights for policy 1, policy_version 1139232 (0.0009) [2023-12-26 23:39:45,179][105620] Updated weights for policy 1, policy_version 1139242 (0.0009) [2023-12-26 23:39:45,419][105692] Updated weights for policy 0, policy_version 1137914 (0.0009) [2023-12-26 23:39:45,471][105692] Updated weights for policy 0, policy_version 1137924 (0.0009) [2023-12-26 23:39:45,523][105692] Updated weights for policy 0, policy_version 1137934 (0.0009) [2023-12-26 23:39:45,877][105620] Updated weights for policy 1, policy_version 1139252 (0.0008) [2023-12-26 23:39:45,925][105620] Updated weights for policy 1, policy_version 1139262 (0.0005) [2023-12-26 23:39:45,978][105620] Updated weights for policy 1, policy_version 1139272 (0.0005) [2023-12-26 23:39:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 583049216. Throughput: 0: 9653.4, 1: 9927.3. Samples: 583018232. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:39:46,063][104569] Avg episode reward: [(0, '8896.460'), (1, '9073.925')] [2023-12-26 23:39:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001137936_291356672.pth... [2023-12-26 23:39:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001139280_291692544.pth... [2023-12-26 23:39:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001136816_291069952.pth [2023-12-26 23:39:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001138096_291389440.pth [2023-12-26 23:39:46,253][105692] Updated weights for policy 0, policy_version 1137944 (0.0006) [2023-12-26 23:39:46,307][105692] Updated weights for policy 0, policy_version 1137954 (0.0006) [2023-12-26 23:39:46,365][105692] Updated weights for policy 0, policy_version 1137964 (0.0008) [2023-12-26 23:39:46,592][105620] Updated weights for policy 1, policy_version 1139282 (0.0007) [2023-12-26 23:39:46,646][105620] Updated weights for policy 1, policy_version 1139293 (0.0010) [2023-12-26 23:39:46,704][105620] Updated weights for policy 1, policy_version 1139306 (0.0010) [2023-12-26 23:39:46,947][105692] Updated weights for policy 0, policy_version 1137974 (0.0010) [2023-12-26 23:39:47,006][105692] Updated weights for policy 0, policy_version 1137984 (0.0011) [2023-12-26 23:39:47,061][105692] Updated weights for policy 0, policy_version 1137994 (0.0010) [2023-12-26 23:39:47,380][105620] Updated weights for policy 1, policy_version 1139316 (0.0006) [2023-12-26 23:39:47,432][105620] Updated weights for policy 1, policy_version 1139326 (0.0005) [2023-12-26 23:39:47,499][105620] Updated weights for policy 1, policy_version 1139336 (0.0005) [2023-12-26 23:39:47,749][105692] Updated weights for policy 0, policy_version 1138004 (0.0008) [2023-12-26 23:39:47,801][105692] Updated weights for policy 0, policy_version 1138014 (0.0005) [2023-12-26 23:39:47,852][105692] Updated weights for policy 0, policy_version 1138024 (0.0005) [2023-12-26 23:39:48,192][105620] Updated weights for policy 1, policy_version 1139346 (0.0010) [2023-12-26 23:39:48,240][105620] Updated weights for policy 1, policy_version 1139356 (0.0010) [2023-12-26 23:39:48,289][105620] Updated weights for policy 1, policy_version 1139366 (0.0010) [2023-12-26 23:39:48,338][105620] Updated weights for policy 1, policy_version 1139376 (0.0010) [2023-12-26 23:39:48,545][105692] Updated weights for policy 0, policy_version 1138034 (0.0007) [2023-12-26 23:39:48,600][105692] Updated weights for policy 0, policy_version 1138044 (0.0010) [2023-12-26 23:39:48,652][105692] Updated weights for policy 0, policy_version 1138054 (0.0010) [2023-12-26 23:39:48,704][105692] Updated weights for policy 0, policy_version 1138064 (0.0010) [2023-12-26 23:39:49,012][105620] Updated weights for policy 1, policy_version 1139386 (0.0010) [2023-12-26 23:39:49,071][105620] Updated weights for policy 1, policy_version 1139396 (0.0010) [2023-12-26 23:39:49,130][105620] Updated weights for policy 1, policy_version 1139406 (0.0010) [2023-12-26 23:39:49,463][105692] Updated weights for policy 0, policy_version 1138074 (0.0006) [2023-12-26 23:39:49,523][105692] Updated weights for policy 0, policy_version 1138084 (0.0006) [2023-12-26 23:39:49,581][105692] Updated weights for policy 0, policy_version 1138094 (0.0007) [2023-12-26 23:39:49,821][105620] Updated weights for policy 1, policy_version 1139416 (0.0008) [2023-12-26 23:39:49,881][105620] Updated weights for policy 1, policy_version 1139426 (0.0009) [2023-12-26 23:39:49,939][105620] Updated weights for policy 1, policy_version 1139436 (0.0008) [2023-12-26 23:39:50,331][105692] Updated weights for policy 0, policy_version 1138104 (0.0010) [2023-12-26 23:39:50,396][105692] Updated weights for policy 0, policy_version 1138114 (0.0010) [2023-12-26 23:39:50,456][105692] Updated weights for policy 0, policy_version 1138124 (0.0010) [2023-12-26 23:39:50,582][105620] Updated weights for policy 1, policy_version 1139446 (0.0008) [2023-12-26 23:39:50,656][105620] Updated weights for policy 1, policy_version 1139456 (0.0007) [2023-12-26 23:39:50,718][105620] Updated weights for policy 1, policy_version 1139466 (0.0009) [2023-12-26 23:39:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 583147520. Throughput: 0: 9691.3, 1: 9969.0. Samples: 583138436. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:39:51,063][104569] Avg episode reward: [(0, '9170.463'), (1, '9080.423')] [2023-12-26 23:39:51,238][105692] Updated weights for policy 0, policy_version 1138134 (0.0010) [2023-12-26 23:39:51,301][105692] Updated weights for policy 0, policy_version 1138144 (0.0008) [2023-12-26 23:39:51,369][105692] Updated weights for policy 0, policy_version 1138154 (0.0008) [2023-12-26 23:39:51,456][105620] Updated weights for policy 1, policy_version 1139476 (0.0009) [2023-12-26 23:39:51,515][105620] Updated weights for policy 1, policy_version 1139486 (0.0010) [2023-12-26 23:39:51,574][105620] Updated weights for policy 1, policy_version 1139496 (0.0011) [2023-12-26 23:39:52,131][105692] Updated weights for policy 0, policy_version 1138164 (0.0009) [2023-12-26 23:39:52,194][105692] Updated weights for policy 0, policy_version 1138174 (0.0009) [2023-12-26 23:39:52,252][105692] Updated weights for policy 0, policy_version 1138184 (0.0008) [2023-12-26 23:39:52,352][105620] Updated weights for policy 1, policy_version 1139506 (0.0011) [2023-12-26 23:39:52,421][105620] Updated weights for policy 1, policy_version 1139516 (0.0011) [2023-12-26 23:39:52,477][105620] Updated weights for policy 1, policy_version 1139526 (0.0011) [2023-12-26 23:39:52,530][105620] Updated weights for policy 1, policy_version 1139536 (0.0011) [2023-12-26 23:39:53,032][105692] Updated weights for policy 0, policy_version 1138194 (0.0009) [2023-12-26 23:39:53,083][105692] Updated weights for policy 0, policy_version 1138204 (0.0009) [2023-12-26 23:39:53,138][105692] Updated weights for policy 0, policy_version 1138214 (0.0008) [2023-12-26 23:39:53,185][105692] Updated weights for policy 0, policy_version 1138224 (0.0009) [2023-12-26 23:39:53,263][105620] Updated weights for policy 1, policy_version 1139546 (0.0009) [2023-12-26 23:39:53,322][105620] Updated weights for policy 1, policy_version 1139556 (0.0009) [2023-12-26 23:39:53,387][105620] Updated weights for policy 1, policy_version 1139566 (0.0009) [2023-12-26 23:39:53,947][105692] Updated weights for policy 0, policy_version 1138234 (0.0008) [2023-12-26 23:39:54,001][105692] Updated weights for policy 0, policy_version 1138244 (0.0009) [2023-12-26 23:39:54,054][105692] Updated weights for policy 0, policy_version 1138254 (0.0008) [2023-12-26 23:39:54,148][105620] Updated weights for policy 1, policy_version 1139577 (0.0009) [2023-12-26 23:39:54,199][105620] Updated weights for policy 1, policy_version 1139587 (0.0005) [2023-12-26 23:39:54,243][105620] Updated weights for policy 1, policy_version 1139597 (0.0005) [2023-12-26 23:39:54,846][105692] Updated weights for policy 0, policy_version 1138264 (0.0009) [2023-12-26 23:39:54,896][105692] Updated weights for policy 0, policy_version 1138274 (0.0009) [2023-12-26 23:39:54,933][105620] Updated weights for policy 1, policy_version 1139607 (0.0005) [2023-12-26 23:39:54,945][105692] Updated weights for policy 0, policy_version 1138284 (0.0009) [2023-12-26 23:39:54,987][105620] Updated weights for policy 1, policy_version 1139617 (0.0006) [2023-12-26 23:39:55,041][105620] Updated weights for policy 1, policy_version 1139627 (0.0009) [2023-12-26 23:39:55,718][105692] Updated weights for policy 0, policy_version 1138294 (0.0009) [2023-12-26 23:39:55,766][105692] Updated weights for policy 0, policy_version 1138304 (0.0008) [2023-12-26 23:39:55,771][105620] Updated weights for policy 1, policy_version 1139637 (0.0009) [2023-12-26 23:39:55,832][105692] Updated weights for policy 0, policy_version 1138314 (0.0007) [2023-12-26 23:39:55,835][105620] Updated weights for policy 1, policy_version 1139647 (0.0007) [2023-12-26 23:39:55,904][105620] Updated weights for policy 1, policy_version 1139657 (0.0009) [2023-12-26 23:39:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 583245824. Throughput: 0: 9657.8, 1: 9951.7. Samples: 583250716. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:39:56,063][104569] Avg episode reward: [(0, '9355.198'), (1, '9076.556')] [2023-12-26 23:39:56,607][105692] Updated weights for policy 0, policy_version 1138324 (0.0008) [2023-12-26 23:39:56,644][105620] Updated weights for policy 1, policy_version 1139667 (0.0007) [2023-12-26 23:39:56,666][105692] Updated weights for policy 0, policy_version 1138334 (0.0008) [2023-12-26 23:39:56,700][105620] Updated weights for policy 1, policy_version 1139677 (0.0005) [2023-12-26 23:39:56,714][105692] Updated weights for policy 0, policy_version 1138344 (0.0009) [2023-12-26 23:39:56,748][105620] Updated weights for policy 1, policy_version 1139687 (0.0005) [2023-12-26 23:39:57,394][105620] Updated weights for policy 1, policy_version 1139697 (0.0006) [2023-12-26 23:39:57,463][105620] Updated weights for policy 1, policy_version 1139707 (0.0006) [2023-12-26 23:39:57,469][105692] Updated weights for policy 0, policy_version 1138354 (0.0008) [2023-12-26 23:39:57,511][105620] Updated weights for policy 1, policy_version 1139717 (0.0009) [2023-12-26 23:39:57,519][105692] Updated weights for policy 0, policy_version 1138364 (0.0005) [2023-12-26 23:39:57,559][105620] Updated weights for policy 1, policy_version 1139727 (0.0008) [2023-12-26 23:39:57,573][105692] Updated weights for policy 0, policy_version 1138374 (0.0005) [2023-12-26 23:39:57,627][105692] Updated weights for policy 0, policy_version 1138384 (0.0007) [2023-12-26 23:39:58,314][105620] Updated weights for policy 1, policy_version 1139737 (0.0007) [2023-12-26 23:39:58,331][105692] Updated weights for policy 0, policy_version 1138394 (0.0007) [2023-12-26 23:39:58,383][105620] Updated weights for policy 1, policy_version 1139747 (0.0008) [2023-12-26 23:39:58,395][105692] Updated weights for policy 0, policy_version 1138404 (0.0010) [2023-12-26 23:39:58,451][105620] Updated weights for policy 1, policy_version 1139757 (0.0008) [2023-12-26 23:39:58,460][105692] Updated weights for policy 0, policy_version 1138414 (0.0007) [2023-12-26 23:39:59,214][105692] Updated weights for policy 0, policy_version 1138424 (0.0008) [2023-12-26 23:39:59,231][105620] Updated weights for policy 1, policy_version 1139767 (0.0006) [2023-12-26 23:39:59,275][105692] Updated weights for policy 0, policy_version 1138434 (0.0008) [2023-12-26 23:39:59,294][105620] Updated weights for policy 1, policy_version 1139777 (0.0007) [2023-12-26 23:39:59,330][105692] Updated weights for policy 0, policy_version 1138444 (0.0007) [2023-12-26 23:39:59,355][105620] Updated weights for policy 1, policy_version 1139787 (0.0007) [2023-12-26 23:40:00,012][105620] Updated weights for policy 1, policy_version 1139797 (0.0008) [2023-12-26 23:40:00,061][105620] Updated weights for policy 1, policy_version 1139807 (0.0008) [2023-12-26 23:40:00,110][105620] Updated weights for policy 1, policy_version 1139817 (0.0008) [2023-12-26 23:40:00,172][105692] Updated weights for policy 0, policy_version 1138454 (0.0008) [2023-12-26 23:40:00,234][105692] Updated weights for policy 0, policy_version 1138464 (0.0008) [2023-12-26 23:40:00,296][105692] Updated weights for policy 0, policy_version 1138474 (0.0007) [2023-12-26 23:40:00,883][105620] Updated weights for policy 1, policy_version 1139827 (0.0008) [2023-12-26 23:40:00,934][105692] Updated weights for policy 0, policy_version 1138484 (0.0007) [2023-12-26 23:40:00,939][105620] Updated weights for policy 1, policy_version 1139837 (0.0006) [2023-12-26 23:40:00,982][105692] Updated weights for policy 0, policy_version 1138494 (0.0008) [2023-12-26 23:40:00,989][105620] Updated weights for policy 1, policy_version 1139847 (0.0005) [2023-12-26 23:40:01,038][105692] Updated weights for policy 0, policy_version 1138504 (0.0008) [2023-12-26 23:40:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 583335936. Throughput: 0: 9646.0, 1: 9951.1. Samples: 583307580. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-12-26 23:40:01,062][104569] Avg episode reward: [(0, '9356.030'), (1, '9178.847')] [2023-12-26 23:40:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001139856_291840000.pth... [2023-12-26 23:40:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001138704_291545088.pth [2023-12-26 23:40:01,090][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001138512_291504128.pth... [2023-12-26 23:40:01,096][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001137392_291217408.pth [2023-12-26 23:40:01,706][105620] Updated weights for policy 1, policy_version 1139857 (0.0006) [2023-12-26 23:40:01,772][105620] Updated weights for policy 1, policy_version 1139867 (0.0008) [2023-12-26 23:40:01,834][105620] Updated weights for policy 1, policy_version 1139877 (0.0010) [2023-12-26 23:40:01,854][105692] Updated weights for policy 0, policy_version 1138514 (0.0009) [2023-12-26 23:40:01,890][105620] Updated weights for policy 1, policy_version 1139887 (0.0005) [2023-12-26 23:40:01,910][105692] Updated weights for policy 0, policy_version 1138524 (0.0008) [2023-12-26 23:40:01,966][105692] Updated weights for policy 0, policy_version 1138534 (0.0009) [2023-12-26 23:40:02,024][105692] Updated weights for policy 0, policy_version 1138544 (0.0009) [2023-12-26 23:40:02,555][105620] Updated weights for policy 1, policy_version 1139897 (0.0005) [2023-12-26 23:40:02,622][105620] Updated weights for policy 1, policy_version 1139907 (0.0005) [2023-12-26 23:40:02,678][105620] Updated weights for policy 1, policy_version 1139917 (0.0005) [2023-12-26 23:40:02,802][105692] Updated weights for policy 0, policy_version 1138554 (0.0009) [2023-12-26 23:40:02,855][105692] Updated weights for policy 0, policy_version 1138564 (0.0010) [2023-12-26 23:40:02,913][105692] Updated weights for policy 0, policy_version 1138574 (0.0009) [2023-12-26 23:40:03,223][105620] Updated weights for policy 1, policy_version 1139927 (0.0005) [2023-12-26 23:40:03,285][105620] Updated weights for policy 1, policy_version 1139937 (0.0005) [2023-12-26 23:40:03,337][105620] Updated weights for policy 1, policy_version 1139947 (0.0005) [2023-12-26 23:40:03,784][105692] Updated weights for policy 0, policy_version 1138585 (0.0010) [2023-12-26 23:40:03,836][105692] Updated weights for policy 0, policy_version 1138596 (0.0010) [2023-12-26 23:40:03,889][105692] Updated weights for policy 0, policy_version 1138606 (0.0008) [2023-12-26 23:40:03,897][105620] Updated weights for policy 1, policy_version 1139957 (0.0007) [2023-12-26 23:40:03,951][105620] Updated weights for policy 1, policy_version 1139967 (0.0008) [2023-12-26 23:40:04,007][105620] Updated weights for policy 1, policy_version 1139977 (0.0008) [2023-12-26 23:40:04,631][105692] Updated weights for policy 0, policy_version 1138616 (0.0007) [2023-12-26 23:40:04,677][105692] Updated weights for policy 0, policy_version 1138626 (0.0005) [2023-12-26 23:40:04,723][105692] Updated weights for policy 0, policy_version 1138636 (0.0005) [2023-12-26 23:40:04,833][105620] Updated weights for policy 1, policy_version 1139987 (0.0007) [2023-12-26 23:40:04,883][105620] Updated weights for policy 1, policy_version 1139997 (0.0005) [2023-12-26 23:40:04,947][105620] Updated weights for policy 1, policy_version 1140007 (0.0005) [2023-12-26 23:40:05,283][105692] Updated weights for policy 0, policy_version 1138646 (0.0009) [2023-12-26 23:40:05,340][105692] Updated weights for policy 0, policy_version 1138656 (0.0011) [2023-12-26 23:40:05,409][105692] Updated weights for policy 0, policy_version 1138666 (0.0008) [2023-12-26 23:40:05,529][105620] Updated weights for policy 1, policy_version 1140017 (0.0006) [2023-12-26 23:40:05,584][105620] Updated weights for policy 1, policy_version 1140027 (0.0006) [2023-12-26 23:40:05,631][105620] Updated weights for policy 1, policy_version 1140037 (0.0009) [2023-12-26 23:40:05,677][105620] Updated weights for policy 1, policy_version 1140047 (0.0006) [2023-12-26 23:40:05,979][105692] Updated weights for policy 0, policy_version 1138676 (0.0006) [2023-12-26 23:40:06,048][105692] Updated weights for policy 0, policy_version 1138686 (0.0006) [2023-12-26 23:40:06,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.4, 300 sec: 19522.0). Total num frames: 583434240. Throughput: 0: 9594.9, 1: 9912.1. Samples: 583422976. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:40:06,062][104569] Avg episode reward: [(0, '9356.426'), (1, '8913.204')] [2023-12-26 23:40:06,117][105692] Updated weights for policy 0, policy_version 1138696 (0.0010) [2023-12-26 23:40:06,300][105620] Updated weights for policy 1, policy_version 1140057 (0.0008) [2023-12-26 23:40:06,369][105620] Updated weights for policy 1, policy_version 1140067 (0.0008) [2023-12-26 23:40:06,430][105620] Updated weights for policy 1, policy_version 1140077 (0.0008) [2023-12-26 23:40:06,821][105692] Updated weights for policy 0, policy_version 1138706 (0.0010) [2023-12-26 23:40:06,873][105692] Updated weights for policy 0, policy_version 1138716 (0.0005) [2023-12-26 23:40:06,933][105692] Updated weights for policy 0, policy_version 1138726 (0.0005) [2023-12-26 23:40:06,983][105692] Updated weights for policy 0, policy_version 1138736 (0.0005) [2023-12-26 23:40:07,200][105620] Updated weights for policy 1, policy_version 1140087 (0.0009) [2023-12-26 23:40:07,253][105620] Updated weights for policy 1, policy_version 1140097 (0.0010) [2023-12-26 23:40:07,304][105620] Updated weights for policy 1, policy_version 1140107 (0.0009) [2023-12-26 23:40:07,604][105692] Updated weights for policy 0, policy_version 1138746 (0.0008) [2023-12-26 23:40:07,669][105692] Updated weights for policy 0, policy_version 1138756 (0.0008) [2023-12-26 23:40:07,736][105692] Updated weights for policy 0, policy_version 1138766 (0.0008) [2023-12-26 23:40:08,174][105620] Updated weights for policy 1, policy_version 1140117 (0.0010) [2023-12-26 23:40:08,227][105620] Updated weights for policy 1, policy_version 1140127 (0.0010) [2023-12-26 23:40:08,281][105692] Updated weights for policy 0, policy_version 1138776 (0.0006) [2023-12-26 23:40:08,284][105620] Updated weights for policy 1, policy_version 1140137 (0.0009) [2023-12-26 23:40:08,316][105585] KL-divergence is very high: 124.2699 [2023-12-26 23:40:08,354][105692] Updated weights for policy 0, policy_version 1138786 (0.0007) [2023-12-26 23:40:08,375][105585] KL-divergence is very high: 160.2844 [2023-12-26 23:40:08,417][105692] Updated weights for policy 0, policy_version 1138796 (0.0010) [2023-12-26 23:40:08,420][105585] KL-divergence is very high: 120.1898 [2023-12-26 23:40:09,079][105692] Updated weights for policy 0, policy_version 1138806 (0.0010) [2023-12-26 23:40:09,094][105620] Updated weights for policy 1, policy_version 1140147 (0.0009) [2023-12-26 23:40:09,138][105692] Updated weights for policy 0, policy_version 1138816 (0.0007) [2023-12-26 23:40:09,149][105620] Updated weights for policy 1, policy_version 1140157 (0.0006) [2023-12-26 23:40:09,192][105692] Updated weights for policy 0, policy_version 1138826 (0.0007) [2023-12-26 23:40:09,198][105620] Updated weights for policy 1, policy_version 1140167 (0.0008) [2023-12-26 23:40:09,893][105620] Updated weights for policy 1, policy_version 1140177 (0.0008) [2023-12-26 23:40:09,957][105620] Updated weights for policy 1, policy_version 1140187 (0.0009) [2023-12-26 23:40:10,022][105620] Updated weights for policy 1, policy_version 1140197 (0.0007) [2023-12-26 23:40:10,028][105692] Updated weights for policy 0, policy_version 1138836 (0.0007) [2023-12-26 23:40:10,074][105620] Updated weights for policy 1, policy_version 1140207 (0.0007) [2023-12-26 23:40:10,092][105692] Updated weights for policy 0, policy_version 1138846 (0.0006) [2023-12-26 23:40:10,157][105692] Updated weights for policy 0, policy_version 1138856 (0.0007) [2023-12-26 23:40:10,821][105620] Updated weights for policy 1, policy_version 1140217 (0.0007) [2023-12-26 23:40:10,850][105692] Updated weights for policy 0, policy_version 1138866 (0.0007) [2023-12-26 23:40:10,879][105620] Updated weights for policy 1, policy_version 1140227 (0.0007) [2023-12-26 23:40:10,908][105692] Updated weights for policy 0, policy_version 1138876 (0.0009) [2023-12-26 23:40:10,939][105620] Updated weights for policy 1, policy_version 1140237 (0.0005) [2023-12-26 23:40:10,957][105692] Updated weights for policy 0, policy_version 1138886 (0.0011) [2023-12-26 23:40:11,009][105692] Updated weights for policy 0, policy_version 1138896 (0.0010) [2023-12-26 23:40:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 583540736. Throughput: 0: 9717.3, 1: 9876.8. Samples: 583543384. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:40:11,062][104569] Avg episode reward: [(0, '9080.559'), (1, '8824.648')] [2023-12-26 23:40:11,700][105620] Updated weights for policy 1, policy_version 1140247 (0.0006) [2023-12-26 23:40:11,770][105620] Updated weights for policy 1, policy_version 1140257 (0.0006) [2023-12-26 23:40:11,834][105620] Updated weights for policy 1, policy_version 1140267 (0.0007) [2023-12-26 23:40:11,857][105692] Updated weights for policy 0, policy_version 1138906 (0.0008) [2023-12-26 23:40:11,910][105692] Updated weights for policy 0, policy_version 1138916 (0.0009) [2023-12-26 23:40:11,965][105692] Updated weights for policy 0, policy_version 1138926 (0.0010) [2023-12-26 23:40:12,483][105620] Updated weights for policy 1, policy_version 1140277 (0.0006) [2023-12-26 23:40:12,542][105620] Updated weights for policy 1, policy_version 1140287 (0.0008) [2023-12-26 23:40:12,601][105620] Updated weights for policy 1, policy_version 1140297 (0.0009) [2023-12-26 23:40:12,731][105692] Updated weights for policy 0, policy_version 1138936 (0.0009) [2023-12-26 23:40:12,782][105692] Updated weights for policy 0, policy_version 1138946 (0.0008) [2023-12-26 23:40:12,835][105692] Updated weights for policy 0, policy_version 1138956 (0.0009) [2023-12-26 23:40:13,341][105620] Updated weights for policy 1, policy_version 1140307 (0.0009) [2023-12-26 23:40:13,399][105620] Updated weights for policy 1, policy_version 1140317 (0.0005) [2023-12-26 23:40:13,454][105620] Updated weights for policy 1, policy_version 1140327 (0.0006) [2023-12-26 23:40:13,549][105692] Updated weights for policy 0, policy_version 1138966 (0.0007) [2023-12-26 23:40:13,618][105692] Updated weights for policy 0, policy_version 1138976 (0.0005) [2023-12-26 23:40:13,689][105692] Updated weights for policy 0, policy_version 1138986 (0.0005) [2023-12-26 23:40:14,081][105620] Updated weights for policy 1, policy_version 1140337 (0.0008) [2023-12-26 23:40:14,142][105620] Updated weights for policy 1, policy_version 1140347 (0.0009) [2023-12-26 23:40:14,206][105692] Updated weights for policy 0, policy_version 1138996 (0.0008) [2023-12-26 23:40:14,222][105620] Updated weights for policy 1, policy_version 1140357 (0.0008) [2023-12-26 23:40:14,255][105692] Updated weights for policy 0, policy_version 1139006 (0.0010) [2023-12-26 23:40:14,267][105620] Updated weights for policy 1, policy_version 1140367 (0.0005) [2023-12-26 23:40:14,304][105692] Updated weights for policy 0, policy_version 1139016 (0.0010) [2023-12-26 23:40:14,867][105620] Updated weights for policy 1, policy_version 1140377 (0.0006) [2023-12-26 23:40:14,939][105620] Updated weights for policy 1, policy_version 1140387 (0.0006) [2023-12-26 23:40:15,003][105620] Updated weights for policy 1, policy_version 1140397 (0.0006) [2023-12-26 23:40:15,015][105692] Updated weights for policy 0, policy_version 1139026 (0.0010) [2023-12-26 23:40:15,067][105692] Updated weights for policy 0, policy_version 1139036 (0.0010) [2023-12-26 23:40:15,118][105692] Updated weights for policy 0, policy_version 1139046 (0.0006) [2023-12-26 23:40:15,176][105692] Updated weights for policy 0, policy_version 1139056 (0.0007) [2023-12-26 23:40:15,566][105620] Updated weights for policy 1, policy_version 1140407 (0.0009) [2023-12-26 23:40:15,619][105620] Updated weights for policy 1, policy_version 1140417 (0.0011) [2023-12-26 23:40:15,685][105620] Updated weights for policy 1, policy_version 1140427 (0.0011) [2023-12-26 23:40:15,750][105692] Updated weights for policy 0, policy_version 1139066 (0.0010) [2023-12-26 23:40:15,796][105692] Updated weights for policy 0, policy_version 1139076 (0.0010) [2023-12-26 23:40:15,856][105692] Updated weights for policy 0, policy_version 1139086 (0.0010) [2023-12-26 23:40:16,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 583639040. Throughput: 0: 9644.4, 1: 9785.1. Samples: 583601460. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:40:16,065][104569] Avg episode reward: [(0, '8807.191'), (1, '9083.057')] [2023-12-26 23:40:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001139088_291651584.pth... [2023-12-26 23:40:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001140432_291987456.pth... [2023-12-26 23:40:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001137936_291356672.pth [2023-12-26 23:40:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001139280_291692544.pth [2023-12-26 23:40:16,478][105620] Updated weights for policy 1, policy_version 1140437 (0.0009) [2023-12-26 23:40:16,495][105692] Updated weights for policy 0, policy_version 1139096 (0.0009) [2023-12-26 23:40:16,533][105620] Updated weights for policy 1, policy_version 1140447 (0.0007) [2023-12-26 23:40:16,552][105692] Updated weights for policy 0, policy_version 1139106 (0.0010) [2023-12-26 23:40:16,582][105620] Updated weights for policy 1, policy_version 1140457 (0.0007) [2023-12-26 23:40:16,600][105692] Updated weights for policy 0, policy_version 1139116 (0.0010) [2023-12-26 23:40:17,192][105620] Updated weights for policy 1, policy_version 1140467 (0.0009) [2023-12-26 23:40:17,240][105620] Updated weights for policy 1, policy_version 1140477 (0.0010) [2023-12-26 23:40:17,288][105620] Updated weights for policy 1, policy_version 1140487 (0.0010) [2023-12-26 23:40:17,348][105692] Updated weights for policy 0, policy_version 1139126 (0.0010) [2023-12-26 23:40:17,413][105692] Updated weights for policy 0, policy_version 1139136 (0.0010) [2023-12-26 23:40:17,477][105692] Updated weights for policy 0, policy_version 1139146 (0.0010) [2023-12-26 23:40:17,883][105620] Updated weights for policy 1, policy_version 1140497 (0.0010) [2023-12-26 23:40:17,945][105620] Updated weights for policy 1, policy_version 1140507 (0.0005) [2023-12-26 23:40:18,004][105620] Updated weights for policy 1, policy_version 1140517 (0.0006) [2023-12-26 23:40:18,055][105620] Updated weights for policy 1, policy_version 1140527 (0.0005) [2023-12-26 23:40:18,194][105692] Updated weights for policy 0, policy_version 1139156 (0.0008) [2023-12-26 23:40:18,248][105692] Updated weights for policy 0, policy_version 1139166 (0.0006) [2023-12-26 23:40:18,306][105692] Updated weights for policy 0, policy_version 1139176 (0.0008) [2023-12-26 23:40:18,604][105620] Updated weights for policy 1, policy_version 1140537 (0.0009) [2023-12-26 23:40:18,656][105620] Updated weights for policy 1, policy_version 1140547 (0.0010) [2023-12-26 23:40:18,711][105620] Updated weights for policy 1, policy_version 1140557 (0.0008) [2023-12-26 23:40:18,909][105692] Updated weights for policy 0, policy_version 1139186 (0.0010) [2023-12-26 23:40:18,973][105692] Updated weights for policy 0, policy_version 1139196 (0.0005) [2023-12-26 23:40:19,037][105692] Updated weights for policy 0, policy_version 1139206 (0.0005) [2023-12-26 23:40:19,086][105692] Updated weights for policy 0, policy_version 1139216 (0.0009) [2023-12-26 23:40:19,424][105620] Updated weights for policy 1, policy_version 1140567 (0.0008) [2023-12-26 23:40:19,490][105620] Updated weights for policy 1, policy_version 1140577 (0.0006) [2023-12-26 23:40:19,553][105620] Updated weights for policy 1, policy_version 1140587 (0.0008) [2023-12-26 23:40:19,755][105692] Updated weights for policy 0, policy_version 1139226 (0.0010) [2023-12-26 23:40:19,820][105692] Updated weights for policy 0, policy_version 1139236 (0.0009) [2023-12-26 23:40:19,888][105692] Updated weights for policy 0, policy_version 1139246 (0.0007) [2023-12-26 23:40:20,196][105620] Updated weights for policy 1, policy_version 1140597 (0.0009) [2023-12-26 23:40:20,257][105620] Updated weights for policy 1, policy_version 1140607 (0.0008) [2023-12-26 23:40:20,318][105620] Updated weights for policy 1, policy_version 1140617 (0.0005) [2023-12-26 23:40:20,571][105692] Updated weights for policy 0, policy_version 1139256 (0.0009) [2023-12-26 23:40:20,633][105692] Updated weights for policy 0, policy_version 1139266 (0.0009) [2023-12-26 23:40:20,695][105692] Updated weights for policy 0, policy_version 1139276 (0.0008) [2023-12-26 23:40:20,975][105620] Updated weights for policy 1, policy_version 1140627 (0.0007) [2023-12-26 23:40:21,037][105620] Updated weights for policy 1, policy_version 1140637 (0.0008) [2023-12-26 23:40:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 583737344. Throughput: 0: 9707.2, 1: 9897.9. Samples: 583728924. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:40:21,062][104569] Avg episode reward: [(0, '8991.884'), (1, '9260.874')] [2023-12-26 23:40:21,107][105620] Updated weights for policy 1, policy_version 1140647 (0.0009) [2023-12-26 23:40:21,519][105692] Updated weights for policy 0, policy_version 1139286 (0.0008) [2023-12-26 23:40:21,573][105692] Updated weights for policy 0, policy_version 1139296 (0.0008) [2023-12-26 23:40:21,631][105692] Updated weights for policy 0, policy_version 1139306 (0.0008) [2023-12-26 23:40:21,916][105620] Updated weights for policy 1, policy_version 1140657 (0.0009) [2023-12-26 23:40:21,984][105620] Updated weights for policy 1, policy_version 1140667 (0.0007) [2023-12-26 23:40:22,053][105620] Updated weights for policy 1, policy_version 1140677 (0.0007) [2023-12-26 23:40:22,124][105620] Updated weights for policy 1, policy_version 1140687 (0.0009) [2023-12-26 23:40:22,341][105692] Updated weights for policy 0, policy_version 1139316 (0.0009) [2023-12-26 23:40:22,403][105692] Updated weights for policy 0, policy_version 1139327 (0.0009) [2023-12-26 23:40:22,458][105692] Updated weights for policy 0, policy_version 1139339 (0.0011) [2023-12-26 23:40:22,752][105620] Updated weights for policy 1, policy_version 1140697 (0.0008) [2023-12-26 23:40:22,803][105620] Updated weights for policy 1, policy_version 1140707 (0.0009) [2023-12-26 23:40:22,862][105620] Updated weights for policy 1, policy_version 1140717 (0.0009) [2023-12-26 23:40:23,197][105692] Updated weights for policy 0, policy_version 1139349 (0.0010) [2023-12-26 23:40:23,263][105692] Updated weights for policy 0, policy_version 1139359 (0.0010) [2023-12-26 23:40:23,332][105692] Updated weights for policy 0, policy_version 1139369 (0.0011) [2023-12-26 23:40:23,573][105620] Updated weights for policy 1, policy_version 1140727 (0.0007) [2023-12-26 23:40:23,626][105620] Updated weights for policy 1, policy_version 1140737 (0.0007) [2023-12-26 23:40:23,680][105620] Updated weights for policy 1, policy_version 1140747 (0.0009) [2023-12-26 23:40:24,076][105692] Updated weights for policy 0, policy_version 1139379 (0.0011) [2023-12-26 23:40:24,143][105692] Updated weights for policy 0, policy_version 1139389 (0.0011) [2023-12-26 23:40:24,200][105692] Updated weights for policy 0, policy_version 1139399 (0.0008) [2023-12-26 23:40:24,429][105620] Updated weights for policy 1, policy_version 1140757 (0.0009) [2023-12-26 23:40:24,499][105620] Updated weights for policy 1, policy_version 1140767 (0.0006) [2023-12-26 23:40:24,569][105620] Updated weights for policy 1, policy_version 1140777 (0.0006) [2023-12-26 23:40:24,923][105692] Updated weights for policy 0, policy_version 1139409 (0.0009) [2023-12-26 23:40:24,971][105692] Updated weights for policy 0, policy_version 1139419 (0.0010) [2023-12-26 23:40:25,022][105692] Updated weights for policy 0, policy_version 1139429 (0.0010) [2023-12-26 23:40:25,073][105692] Updated weights for policy 0, policy_version 1139439 (0.0010) [2023-12-26 23:40:25,180][105620] Updated weights for policy 1, policy_version 1140787 (0.0008) [2023-12-26 23:40:25,230][105620] Updated weights for policy 1, policy_version 1140797 (0.0005) [2023-12-26 23:40:25,283][105620] Updated weights for policy 1, policy_version 1140807 (0.0005) [2023-12-26 23:40:25,789][105692] Updated weights for policy 0, policy_version 1139449 (0.0009) [2023-12-26 23:40:25,846][105692] Updated weights for policy 0, policy_version 1139461 (0.0011) [2023-12-26 23:40:25,900][105692] Updated weights for policy 0, policy_version 1139472 (0.0010) [2023-12-26 23:40:25,901][105620] Updated weights for policy 1, policy_version 1140817 (0.0005) [2023-12-26 23:40:25,961][105620] Updated weights for policy 1, policy_version 1140827 (0.0005) [2023-12-26 23:40:26,008][105620] Updated weights for policy 1, policy_version 1140837 (0.0009) [2023-12-26 23:40:26,055][105620] Updated weights for policy 1, policy_version 1140847 (0.0008) [2023-12-26 23:40:26,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 583843840. Throughput: 0: 9699.2, 1: 9912.5. Samples: 583846168. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:40:26,062][104569] Avg episode reward: [(0, '9176.102'), (1, '9172.923')] [2023-12-26 23:40:26,702][105692] Updated weights for policy 0, policy_version 1139482 (0.0006) [2023-12-26 23:40:26,754][105692] Updated weights for policy 0, policy_version 1139492 (0.0005) [2023-12-26 23:40:26,765][105620] Updated weights for policy 1, policy_version 1140857 (0.0008) [2023-12-26 23:40:26,812][105692] Updated weights for policy 0, policy_version 1139502 (0.0006) [2023-12-26 23:40:26,814][105620] Updated weights for policy 1, policy_version 1140867 (0.0009) [2023-12-26 23:40:26,870][105620] Updated weights for policy 1, policy_version 1140877 (0.0010) [2023-12-26 23:40:27,334][105692] Updated weights for policy 0, policy_version 1139512 (0.0007) [2023-12-26 23:40:27,393][105692] Updated weights for policy 0, policy_version 1139522 (0.0008) [2023-12-26 23:40:27,456][105692] Updated weights for policy 0, policy_version 1139532 (0.0008) [2023-12-26 23:40:27,654][105620] Updated weights for policy 1, policy_version 1140887 (0.0008) [2023-12-26 23:40:27,707][105620] Updated weights for policy 1, policy_version 1140897 (0.0009) [2023-12-26 23:40:27,754][105620] Updated weights for policy 1, policy_version 1140907 (0.0009) [2023-12-26 23:40:28,136][105692] Updated weights for policy 0, policy_version 1139542 (0.0007) [2023-12-26 23:40:28,185][105692] Updated weights for policy 0, policy_version 1139552 (0.0005) [2023-12-26 23:40:28,236][105692] Updated weights for policy 0, policy_version 1139562 (0.0005) [2023-12-26 23:40:28,447][105620] Updated weights for policy 1, policy_version 1140917 (0.0009) [2023-12-26 23:40:28,505][105620] Updated weights for policy 1, policy_version 1140927 (0.0009) [2023-12-26 23:40:28,566][105620] Updated weights for policy 1, policy_version 1140937 (0.0009) [2023-12-26 23:40:28,917][105692] Updated weights for policy 0, policy_version 1139572 (0.0006) [2023-12-26 23:40:28,971][105692] Updated weights for policy 0, policy_version 1139582 (0.0010) [2023-12-26 23:40:29,028][105692] Updated weights for policy 0, policy_version 1139592 (0.0010) [2023-12-26 23:40:29,339][105620] Updated weights for policy 1, policy_version 1140947 (0.0009) [2023-12-26 23:40:29,402][105620] Updated weights for policy 1, policy_version 1140957 (0.0009) [2023-12-26 23:40:29,463][105620] Updated weights for policy 1, policy_version 1140967 (0.0009) [2023-12-26 23:40:29,736][105692] Updated weights for policy 0, policy_version 1139602 (0.0009) [2023-12-26 23:40:29,795][105692] Updated weights for policy 0, policy_version 1139612 (0.0008) [2023-12-26 23:40:29,859][105692] Updated weights for policy 0, policy_version 1139622 (0.0010) [2023-12-26 23:40:29,918][105692] Updated weights for policy 0, policy_version 1139632 (0.0009) [2023-12-26 23:40:30,214][105620] Updated weights for policy 1, policy_version 1140977 (0.0008) [2023-12-26 23:40:30,274][105620] Updated weights for policy 1, policy_version 1140987 (0.0005) [2023-12-26 23:40:30,328][105620] Updated weights for policy 1, policy_version 1140997 (0.0009) [2023-12-26 23:40:30,389][105620] Updated weights for policy 1, policy_version 1141007 (0.0009) [2023-12-26 23:40:30,663][105692] Updated weights for policy 0, policy_version 1139642 (0.0005) [2023-12-26 23:40:30,728][105692] Updated weights for policy 0, policy_version 1139652 (0.0009) [2023-12-26 23:40:30,789][105692] Updated weights for policy 0, policy_version 1139662 (0.0010) [2023-12-26 23:40:30,962][105620] Updated weights for policy 1, policy_version 1141017 (0.0008) [2023-12-26 23:40:31,019][105620] Updated weights for policy 1, policy_version 1141027 (0.0008) [2023-12-26 23:40:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 583933952. Throughput: 0: 9786.1, 1: 9956.2. Samples: 583906632. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:40:31,062][104569] Avg episode reward: [(0, '9176.309'), (1, '9172.645')] [2023-12-26 23:40:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001139664_291799040.pth... [2023-12-26 23:40:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001138512_291504128.pth [2023-12-26 23:40:31,081][105620] Updated weights for policy 1, policy_version 1141037 (0.0008) [2023-12-26 23:40:31,095][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001141040_292143104.pth... [2023-12-26 23:40:31,098][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001139856_291840000.pth [2023-12-26 23:40:31,459][105692] Updated weights for policy 0, policy_version 1139672 (0.0009) [2023-12-26 23:40:31,521][105692] Updated weights for policy 0, policy_version 1139682 (0.0008) [2023-12-26 23:40:31,584][105692] Updated weights for policy 0, policy_version 1139692 (0.0009) [2023-12-26 23:40:31,879][105620] Updated weights for policy 1, policy_version 1141047 (0.0006) [2023-12-26 23:40:31,940][105620] Updated weights for policy 1, policy_version 1141057 (0.0005) [2023-12-26 23:40:31,999][105620] Updated weights for policy 1, policy_version 1141067 (0.0005) [2023-12-26 23:40:32,296][105692] Updated weights for policy 0, policy_version 1139702 (0.0010) [2023-12-26 23:40:32,362][105692] Updated weights for policy 0, policy_version 1139712 (0.0010) [2023-12-26 23:40:32,421][105692] Updated weights for policy 0, policy_version 1139722 (0.0009) [2023-12-26 23:40:32,594][105620] Updated weights for policy 1, policy_version 1141077 (0.0006) [2023-12-26 23:40:32,649][105620] Updated weights for policy 1, policy_version 1141087 (0.0008) [2023-12-26 23:40:32,711][105620] Updated weights for policy 1, policy_version 1141097 (0.0007) [2023-12-26 23:40:33,125][105692] Updated weights for policy 0, policy_version 1139732 (0.0010) [2023-12-26 23:40:33,169][105692] Updated weights for policy 0, policy_version 1139742 (0.0010) [2023-12-26 23:40:33,216][105585] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000001 [2023-12-26 23:40:33,219][105692] Updated weights for policy 0, policy_version 1139752 (0.0010) [2023-12-26 23:40:33,267][105620] Updated weights for policy 1, policy_version 1141107 (0.0007) [2023-12-26 23:40:33,322][105620] Updated weights for policy 1, policy_version 1141117 (0.0005) [2023-12-26 23:40:33,372][105620] Updated weights for policy 1, policy_version 1141127 (0.0005) [2023-12-26 23:40:33,876][105692] Updated weights for policy 0, policy_version 1139762 (0.0008) [2023-12-26 23:40:33,923][105692] Updated weights for policy 0, policy_version 1139772 (0.0008) [2023-12-26 23:40:33,931][105620] Updated weights for policy 1, policy_version 1141137 (0.0008) [2023-12-26 23:40:33,971][105692] Updated weights for policy 0, policy_version 1139782 (0.0005) [2023-12-26 23:40:33,981][105620] Updated weights for policy 1, policy_version 1141147 (0.0010) [2023-12-26 23:40:34,041][105620] Updated weights for policy 1, policy_version 1141157 (0.0007) [2023-12-26 23:40:34,108][105620] Updated weights for policy 1, policy_version 1141167 (0.0005) [2023-12-26 23:40:34,639][105692] Updated weights for policy 0, policy_version 1139792 (0.0008) [2023-12-26 23:40:34,695][105692] Updated weights for policy 0, policy_version 1139802 (0.0009) [2023-12-26 23:40:34,737][105620] Updated weights for policy 1, policy_version 1141177 (0.0008) [2023-12-26 23:40:34,751][105692] Updated weights for policy 0, policy_version 1139812 (0.0008) [2023-12-26 23:40:34,792][105620] Updated weights for policy 1, policy_version 1141187 (0.0007) [2023-12-26 23:40:34,861][105620] Updated weights for policy 1, policy_version 1141197 (0.0009) [2023-12-26 23:40:35,508][105620] Updated weights for policy 1, policy_version 1141207 (0.0009) [2023-12-26 23:40:35,514][105692] Updated weights for policy 0, policy_version 1139822 (0.0007) [2023-12-26 23:40:35,553][105620] Updated weights for policy 1, policy_version 1141217 (0.0005) [2023-12-26 23:40:35,563][105692] Updated weights for policy 0, policy_version 1139832 (0.0007) [2023-12-26 23:40:35,609][105620] Updated weights for policy 1, policy_version 1141227 (0.0006) [2023-12-26 23:40:35,614][105692] Updated weights for policy 0, policy_version 1139842 (0.0008) [2023-12-26 23:40:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 584040448. Throughput: 0: 9786.2, 1: 10013.9. Samples: 584029440. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:40:36,062][104569] Avg episode reward: [(0, '9173.373'), (1, '8810.753')] [2023-12-26 23:40:36,356][105620] Updated weights for policy 1, policy_version 1141237 (0.0009) [2023-12-26 23:40:36,377][105692] Updated weights for policy 0, policy_version 1139852 (0.0010) [2023-12-26 23:40:36,416][105620] Updated weights for policy 1, policy_version 1141247 (0.0007) [2023-12-26 23:40:36,435][105692] Updated weights for policy 0, policy_version 1139862 (0.0007) [2023-12-26 23:40:36,475][105620] Updated weights for policy 1, policy_version 1141257 (0.0005) [2023-12-26 23:40:36,497][105692] Updated weights for policy 0, policy_version 1139872 (0.0009) [2023-12-26 23:40:37,153][105620] Updated weights for policy 1, policy_version 1141267 (0.0006) [2023-12-26 23:40:37,211][105620] Updated weights for policy 1, policy_version 1141277 (0.0006) [2023-12-26 23:40:37,278][105620] Updated weights for policy 1, policy_version 1141287 (0.0007) [2023-12-26 23:40:37,311][105692] Updated weights for policy 0, policy_version 1139882 (0.0009) [2023-12-26 23:40:37,364][105692] Updated weights for policy 0, policy_version 1139892 (0.0009) [2023-12-26 23:40:37,422][105692] Updated weights for policy 0, policy_version 1139902 (0.0005) [2023-12-26 23:40:37,489][105692] Updated weights for policy 0, policy_version 1139912 (0.0005) [2023-12-26 23:40:37,817][105620] Updated weights for policy 1, policy_version 1141297 (0.0007) [2023-12-26 23:40:37,870][105620] Updated weights for policy 1, policy_version 1141307 (0.0010) [2023-12-26 23:40:37,919][105620] Updated weights for policy 1, policy_version 1141317 (0.0010) [2023-12-26 23:40:37,972][105620] Updated weights for policy 1, policy_version 1141327 (0.0010) [2023-12-26 23:40:38,033][105692] Updated weights for policy 0, policy_version 1139922 (0.0007) [2023-12-26 23:40:38,098][105692] Updated weights for policy 0, policy_version 1139932 (0.0005) [2023-12-26 23:40:38,168][105692] Updated weights for policy 0, policy_version 1139942 (0.0006) [2023-12-26 23:40:38,697][105620] Updated weights for policy 1, policy_version 1141337 (0.0011) [2023-12-26 23:40:38,757][105620] Updated weights for policy 1, policy_version 1141347 (0.0011) [2023-12-26 23:40:38,817][105620] Updated weights for policy 1, policy_version 1141357 (0.0011) [2023-12-26 23:40:38,834][105692] Updated weights for policy 0, policy_version 1139952 (0.0010) [2023-12-26 23:40:38,893][105692] Updated weights for policy 0, policy_version 1139962 (0.0011) [2023-12-26 23:40:38,956][105692] Updated weights for policy 0, policy_version 1139972 (0.0011) [2023-12-26 23:40:39,548][105620] Updated weights for policy 1, policy_version 1141367 (0.0011) [2023-12-26 23:40:39,601][105620] Updated weights for policy 1, policy_version 1141377 (0.0010) [2023-12-26 23:40:39,654][105620] Updated weights for policy 1, policy_version 1141387 (0.0011) [2023-12-26 23:40:39,722][105692] Updated weights for policy 0, policy_version 1139982 (0.0009) [2023-12-26 23:40:39,782][105692] Updated weights for policy 0, policy_version 1139992 (0.0008) [2023-12-26 23:40:39,844][105692] Updated weights for policy 0, policy_version 1140002 (0.0009) [2023-12-26 23:40:40,423][105620] Updated weights for policy 1, policy_version 1141397 (0.0011) [2023-12-26 23:40:40,480][105620] Updated weights for policy 1, policy_version 1141407 (0.0011) [2023-12-26 23:40:40,543][105620] Updated weights for policy 1, policy_version 1141417 (0.0011) [2023-12-26 23:40:40,564][105692] Updated weights for policy 0, policy_version 1140012 (0.0007) [2023-12-26 23:40:40,625][105692] Updated weights for policy 0, policy_version 1140022 (0.0007) [2023-12-26 23:40:40,682][105692] Updated weights for policy 0, policy_version 1140032 (0.0007) [2023-12-26 23:40:41,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 584138752. Throughput: 0: 9855.9, 1: 10067.2. Samples: 584147256. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:40:41,063][104569] Avg episode reward: [(0, '9265.022'), (1, '8717.088')] [2023-12-26 23:40:41,288][105620] Updated weights for policy 1, policy_version 1141427 (0.0011) [2023-12-26 23:40:41,355][105620] Updated weights for policy 1, policy_version 1141437 (0.0011) [2023-12-26 23:40:41,390][105692] Updated weights for policy 0, policy_version 1140042 (0.0008) [2023-12-26 23:40:41,427][105620] Updated weights for policy 1, policy_version 1141447 (0.0009) [2023-12-26 23:40:41,453][105692] Updated weights for policy 0, policy_version 1140052 (0.0006) [2023-12-26 23:40:41,512][105692] Updated weights for policy 0, policy_version 1140062 (0.0006) [2023-12-26 23:40:41,575][105692] Updated weights for policy 0, policy_version 1140072 (0.0007) [2023-12-26 23:40:42,191][105620] Updated weights for policy 1, policy_version 1141457 (0.0010) [2023-12-26 23:40:42,255][105620] Updated weights for policy 1, policy_version 1141467 (0.0010) [2023-12-26 23:40:42,321][105620] Updated weights for policy 1, policy_version 1141477 (0.0010) [2023-12-26 23:40:42,356][105692] Updated weights for policy 0, policy_version 1140082 (0.0006) [2023-12-26 23:40:42,385][105620] Updated weights for policy 1, policy_version 1141487 (0.0009) [2023-12-26 23:40:42,421][105692] Updated weights for policy 0, policy_version 1140092 (0.0007) [2023-12-26 23:40:42,489][105692] Updated weights for policy 0, policy_version 1140102 (0.0009) [2023-12-26 23:40:43,008][105620] Updated weights for policy 1, policy_version 1141497 (0.0010) [2023-12-26 23:40:43,057][105620] Updated weights for policy 1, policy_version 1141507 (0.0010) [2023-12-26 23:40:43,115][105620] Updated weights for policy 1, policy_version 1141517 (0.0010) [2023-12-26 23:40:43,218][105692] Updated weights for policy 0, policy_version 1140112 (0.0006) [2023-12-26 23:40:43,275][105692] Updated weights for policy 0, policy_version 1140122 (0.0008) [2023-12-26 23:40:43,334][105692] Updated weights for policy 0, policy_version 1140132 (0.0008) [2023-12-26 23:40:43,758][105620] Updated weights for policy 1, policy_version 1141527 (0.0007) [2023-12-26 23:40:43,822][105620] Updated weights for policy 1, policy_version 1141537 (0.0005) [2023-12-26 23:40:43,887][105620] Updated weights for policy 1, policy_version 1141547 (0.0005) [2023-12-26 23:40:44,061][105692] Updated weights for policy 0, policy_version 1140142 (0.0008) [2023-12-26 23:40:44,117][105692] Updated weights for policy 0, policy_version 1140152 (0.0009) [2023-12-26 23:40:44,167][105585] KL-divergence is very high: 151.5725 [2023-12-26 23:40:44,169][105692] Updated weights for policy 0, policy_version 1140162 (0.0009) [2023-12-26 23:40:44,438][105620] Updated weights for policy 1, policy_version 1141557 (0.0006) [2023-12-26 23:40:44,496][105620] Updated weights for policy 1, policy_version 1141567 (0.0006) [2023-12-26 23:40:44,551][105620] Updated weights for policy 1, policy_version 1141577 (0.0006) [2023-12-26 23:40:45,020][105692] Updated weights for policy 0, policy_version 1140172 (0.0009) [2023-12-26 23:40:45,088][105692] Updated weights for policy 0, policy_version 1140182 (0.0007) [2023-12-26 23:40:45,151][105692] Updated weights for policy 0, policy_version 1140192 (0.0009) [2023-12-26 23:40:45,213][105620] Updated weights for policy 1, policy_version 1141587 (0.0007) [2023-12-26 23:40:45,269][105620] Updated weights for policy 1, policy_version 1141597 (0.0009) [2023-12-26 23:40:45,331][105620] Updated weights for policy 1, policy_version 1141607 (0.0009) [2023-12-26 23:40:45,883][105692] Updated weights for policy 0, policy_version 1140202 (0.0009) [2023-12-26 23:40:45,935][105692] Updated weights for policy 0, policy_version 1140212 (0.0010) [2023-12-26 23:40:45,990][105692] Updated weights for policy 0, policy_version 1140222 (0.0009) [2023-12-26 23:40:46,038][105692] Updated weights for policy 0, policy_version 1140232 (0.0009) [2023-12-26 23:40:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 584237056. Throughput: 0: 9866.1, 1: 10103.0. Samples: 584206192. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:40:46,063][104569] Avg episode reward: [(0, '9083.293'), (1, '8908.197')] [2023-12-26 23:40:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001140232_291946496.pth... [2023-12-26 23:40:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001141616_292290560.pth... [2023-12-26 23:40:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001139088_291651584.pth [2023-12-26 23:40:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001140432_291987456.pth [2023-12-26 23:40:46,095][105620] Updated weights for policy 1, policy_version 1141617 (0.0009) [2023-12-26 23:40:46,150][105620] Updated weights for policy 1, policy_version 1141627 (0.0009) [2023-12-26 23:40:46,199][105620] Updated weights for policy 1, policy_version 1141637 (0.0007) [2023-12-26 23:40:46,249][105620] Updated weights for policy 1, policy_version 1141647 (0.0008) [2023-12-26 23:40:46,767][105692] Updated weights for policy 0, policy_version 1140242 (0.0007) [2023-12-26 23:40:46,818][105692] Updated weights for policy 0, policy_version 1140252 (0.0005) [2023-12-26 23:40:46,864][105692] Updated weights for policy 0, policy_version 1140262 (0.0005) [2023-12-26 23:40:47,093][105620] Updated weights for policy 1, policy_version 1141657 (0.0010) [2023-12-26 23:40:47,146][105620] Updated weights for policy 1, policy_version 1141667 (0.0009) [2023-12-26 23:40:47,203][105620] Updated weights for policy 1, policy_version 1141677 (0.0010) [2023-12-26 23:40:47,385][105692] Updated weights for policy 0, policy_version 1140272 (0.0008) [2023-12-26 23:40:47,451][105692] Updated weights for policy 0, policy_version 1140282 (0.0009) [2023-12-26 23:40:47,520][105692] Updated weights for policy 0, policy_version 1140292 (0.0009) [2023-12-26 23:40:47,977][105620] Updated weights for policy 1, policy_version 1141688 (0.0009) [2023-12-26 23:40:48,037][105620] Updated weights for policy 1, policy_version 1141698 (0.0007) [2023-12-26 23:40:48,098][105620] Updated weights for policy 1, policy_version 1141708 (0.0006) [2023-12-26 23:40:48,211][105692] Updated weights for policy 0, policy_version 1140302 (0.0007) [2023-12-26 23:40:48,272][105692] Updated weights for policy 0, policy_version 1140312 (0.0009) [2023-12-26 23:40:48,316][105692] Updated weights for policy 0, policy_version 1140322 (0.0007) [2023-12-26 23:40:48,776][105620] Updated weights for policy 1, policy_version 1141718 (0.0009) [2023-12-26 23:40:48,836][105620] Updated weights for policy 1, policy_version 1141728 (0.0009) [2023-12-26 23:40:48,884][105620] Updated weights for policy 1, policy_version 1141738 (0.0009) [2023-12-26 23:40:49,024][105692] Updated weights for policy 0, policy_version 1140332 (0.0006) [2023-12-26 23:40:49,084][105692] Updated weights for policy 0, policy_version 1140342 (0.0006) [2023-12-26 23:40:49,151][105692] Updated weights for policy 0, policy_version 1140352 (0.0005) [2023-12-26 23:40:49,724][105620] Updated weights for policy 1, policy_version 1141748 (0.0008) [2023-12-26 23:40:49,775][105620] Updated weights for policy 1, policy_version 1141758 (0.0009) [2023-12-26 23:40:49,778][105692] Updated weights for policy 0, policy_version 1140362 (0.0006) [2023-12-26 23:40:49,831][105620] Updated weights for policy 1, policy_version 1141768 (0.0007) [2023-12-26 23:40:49,838][105692] Updated weights for policy 0, policy_version 1140372 (0.0006) [2023-12-26 23:40:49,851][105585] KL-divergence is very high: 109.2032 [2023-12-26 23:40:49,904][105692] Updated weights for policy 0, policy_version 1140382 (0.0006) [2023-12-26 23:40:49,905][105585] KL-divergence is very high: 187.1149 [2023-12-26 23:40:49,959][105585] KL-divergence is very high: 194.3670 [2023-12-26 23:40:49,970][105692] Updated weights for policy 0, policy_version 1140392 (0.0008) [2023-12-26 23:40:50,615][105620] Updated weights for policy 1, policy_version 1141779 (0.0008) [2023-12-26 23:40:50,666][105620] Updated weights for policy 1, policy_version 1141789 (0.0007) [2023-12-26 23:40:50,680][105692] Updated weights for policy 0, policy_version 1140402 (0.0006) [2023-12-26 23:40:50,727][105620] Updated weights for policy 1, policy_version 1141799 (0.0007) [2023-12-26 23:40:50,735][105692] Updated weights for policy 0, policy_version 1140412 (0.0006) [2023-12-26 23:40:50,795][105692] Updated weights for policy 0, policy_version 1140422 (0.0006) [2023-12-26 23:40:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 584335360. Throughput: 0: 9969.8, 1: 10013.6. Samples: 584322228. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:40:51,062][104569] Avg episode reward: [(0, '8990.509'), (1, '9090.044')] [2023-12-26 23:40:51,411][105692] Updated weights for policy 0, policy_version 1140432 (0.0008) [2023-12-26 23:40:51,458][105620] Updated weights for policy 1, policy_version 1141809 (0.0008) [2023-12-26 23:40:51,479][105692] Updated weights for policy 0, policy_version 1140442 (0.0006) [2023-12-26 23:40:51,520][105620] Updated weights for policy 1, policy_version 1141819 (0.0007) [2023-12-26 23:40:51,526][105692] Updated weights for policy 0, policy_version 1140452 (0.0006) [2023-12-26 23:40:51,572][105620] Updated weights for policy 1, policy_version 1141829 (0.0007) [2023-12-26 23:40:51,623][105620] Updated weights for policy 1, policy_version 1141839 (0.0008) [2023-12-26 23:40:52,203][105692] Updated weights for policy 0, policy_version 1140462 (0.0006) [2023-12-26 23:40:52,257][105692] Updated weights for policy 0, policy_version 1140472 (0.0005) [2023-12-26 23:40:52,320][105692] Updated weights for policy 0, policy_version 1140482 (0.0008) [2023-12-26 23:40:52,397][105620] Updated weights for policy 1, policy_version 1141849 (0.0008) [2023-12-26 23:40:52,449][105620] Updated weights for policy 1, policy_version 1141859 (0.0008) [2023-12-26 23:40:52,505][105620] Updated weights for policy 1, policy_version 1141869 (0.0008) [2023-12-26 23:40:52,996][105692] Updated weights for policy 0, policy_version 1140492 (0.0008) [2023-12-26 23:40:53,058][105692] Updated weights for policy 0, policy_version 1140502 (0.0008) [2023-12-26 23:40:53,110][105692] Updated weights for policy 0, policy_version 1140512 (0.0008) [2023-12-26 23:40:53,261][105620] Updated weights for policy 1, policy_version 1141879 (0.0006) [2023-12-26 23:40:53,307][105620] Updated weights for policy 1, policy_version 1141889 (0.0005) [2023-12-26 23:40:53,353][105620] Updated weights for policy 1, policy_version 1141899 (0.0005) [2023-12-26 23:40:53,804][105692] Updated weights for policy 0, policy_version 1140522 (0.0009) [2023-12-26 23:40:53,852][105692] Updated weights for policy 0, policy_version 1140532 (0.0009) [2023-12-26 23:40:53,906][105692] Updated weights for policy 0, policy_version 1140542 (0.0009) [2023-12-26 23:40:53,967][105692] Updated weights for policy 0, policy_version 1140552 (0.0008) [2023-12-26 23:40:54,028][105620] Updated weights for policy 1, policy_version 1141909 (0.0007) [2023-12-26 23:40:54,076][105620] Updated weights for policy 1, policy_version 1141919 (0.0009) [2023-12-26 23:40:54,123][105620] Updated weights for policy 1, policy_version 1141929 (0.0007) [2023-12-26 23:40:54,760][105692] Updated weights for policy 0, policy_version 1140562 (0.0010) [2023-12-26 23:40:54,811][105692] Updated weights for policy 0, policy_version 1140572 (0.0010) [2023-12-26 23:40:54,843][105620] Updated weights for policy 1, policy_version 1141939 (0.0006) [2023-12-26 23:40:54,863][105692] Updated weights for policy 0, policy_version 1140582 (0.0010) [2023-12-26 23:40:54,901][105620] Updated weights for policy 1, policy_version 1141949 (0.0005) [2023-12-26 23:40:54,948][105620] Updated weights for policy 1, policy_version 1141959 (0.0007) [2023-12-26 23:40:55,573][105620] Updated weights for policy 1, policy_version 1141969 (0.0009) [2023-12-26 23:40:55,617][105692] Updated weights for policy 0, policy_version 1140592 (0.0007) [2023-12-26 23:40:55,638][105620] Updated weights for policy 1, policy_version 1141979 (0.0007) [2023-12-26 23:40:55,677][105692] Updated weights for policy 0, policy_version 1140602 (0.0005) [2023-12-26 23:40:55,706][105620] Updated weights for policy 1, policy_version 1141989 (0.0005) [2023-12-26 23:40:55,738][105692] Updated weights for policy 0, policy_version 1140612 (0.0006) [2023-12-26 23:40:55,774][105620] Updated weights for policy 1, policy_version 1141999 (0.0005) [2023-12-26 23:40:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 584433664. Throughput: 0: 9886.8, 1: 10065.4. Samples: 584441236. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:40:56,063][104569] Avg episode reward: [(0, '9173.714'), (1, '9171.996')] [2023-12-26 23:40:56,359][105692] Updated weights for policy 0, policy_version 1140622 (0.0006) [2023-12-26 23:40:56,406][105692] Updated weights for policy 0, policy_version 1140632 (0.0005) [2023-12-26 23:40:56,423][105620] Updated weights for policy 1, policy_version 1142009 (0.0006) [2023-12-26 23:40:56,453][105692] Updated weights for policy 0, policy_version 1140642 (0.0005) [2023-12-26 23:40:56,480][105620] Updated weights for policy 1, policy_version 1142019 (0.0005) [2023-12-26 23:40:56,535][105620] Updated weights for policy 1, policy_version 1142029 (0.0006) [2023-12-26 23:40:57,057][105692] Updated weights for policy 0, policy_version 1140652 (0.0006) [2023-12-26 23:40:57,125][105692] Updated weights for policy 0, policy_version 1140662 (0.0006) [2023-12-26 23:40:57,191][105692] Updated weights for policy 0, policy_version 1140672 (0.0006) [2023-12-26 23:40:57,267][105620] Updated weights for policy 1, policy_version 1142040 (0.0010) [2023-12-26 23:40:57,331][105620] Updated weights for policy 1, policy_version 1142050 (0.0010) [2023-12-26 23:40:57,406][105620] Updated weights for policy 1, policy_version 1142061 (0.0009) [2023-12-26 23:40:57,738][105692] Updated weights for policy 0, policy_version 1140682 (0.0005) [2023-12-26 23:40:57,795][105692] Updated weights for policy 0, policy_version 1140692 (0.0006) [2023-12-26 23:40:57,847][105692] Updated weights for policy 0, policy_version 1140702 (0.0006) [2023-12-26 23:40:57,908][105692] Updated weights for policy 0, policy_version 1140712 (0.0007) [2023-12-26 23:40:58,015][105620] Updated weights for policy 1, policy_version 1142071 (0.0007) [2023-12-26 23:40:58,072][105620] Updated weights for policy 1, policy_version 1142081 (0.0007) [2023-12-26 23:40:58,132][105620] Updated weights for policy 1, policy_version 1142091 (0.0009) [2023-12-26 23:40:58,590][105692] Updated weights for policy 0, policy_version 1140722 (0.0009) [2023-12-26 23:40:58,647][105692] Updated weights for policy 0, policy_version 1140732 (0.0008) [2023-12-26 23:40:58,704][105692] Updated weights for policy 0, policy_version 1140742 (0.0008) [2023-12-26 23:40:58,964][105620] Updated weights for policy 1, policy_version 1142101 (0.0009) [2023-12-26 23:40:59,027][105620] Updated weights for policy 1, policy_version 1142111 (0.0008) [2023-12-26 23:40:59,081][105620] Updated weights for policy 1, policy_version 1142121 (0.0008) [2023-12-26 23:40:59,538][105692] Updated weights for policy 0, policy_version 1140752 (0.0008) [2023-12-26 23:40:59,585][105692] Updated weights for policy 0, policy_version 1140762 (0.0008) [2023-12-26 23:40:59,642][105692] Updated weights for policy 0, policy_version 1140772 (0.0008) [2023-12-26 23:40:59,861][105620] Updated weights for policy 1, policy_version 1142131 (0.0010) [2023-12-26 23:40:59,921][105620] Updated weights for policy 1, policy_version 1142141 (0.0011) [2023-12-26 23:40:59,991][105620] Updated weights for policy 1, policy_version 1142151 (0.0011) [2023-12-26 23:41:00,449][105692] Updated weights for policy 0, policy_version 1140782 (0.0008) [2023-12-26 23:41:00,508][105692] Updated weights for policy 0, policy_version 1140792 (0.0007) [2023-12-26 23:41:00,572][105692] Updated weights for policy 0, policy_version 1140802 (0.0005) [2023-12-26 23:41:00,616][105620] Updated weights for policy 1, policy_version 1142161 (0.0008) [2023-12-26 23:41:00,667][105620] Updated weights for policy 1, policy_version 1142171 (0.0010) [2023-12-26 23:41:00,715][105620] Updated weights for policy 1, policy_version 1142181 (0.0009) [2023-12-26 23:41:00,768][105620] Updated weights for policy 1, policy_version 1142191 (0.0005) [2023-12-26 23:41:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19933.8, 300 sec: 19549.7). Total num frames: 584531968. Throughput: 0: 9998.2, 1: 10045.4. Samples: 584503424. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:41:01,062][104569] Avg episode reward: [(0, '9173.128'), (1, '9258.617')] [2023-12-26 23:41:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001140808_292093952.pth... [2023-12-26 23:41:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001142192_292438016.pth... [2023-12-26 23:41:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001139664_291799040.pth [2023-12-26 23:41:01,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001141040_292143104.pth [2023-12-26 23:41:01,181][105692] Updated weights for policy 0, policy_version 1140812 (0.0007) [2023-12-26 23:41:01,234][105692] Updated weights for policy 0, policy_version 1140822 (0.0006) [2023-12-26 23:41:01,291][105692] Updated weights for policy 0, policy_version 1140832 (0.0006) [2023-12-26 23:41:01,481][105620] Updated weights for policy 1, policy_version 1142201 (0.0006) [2023-12-26 23:41:01,540][105620] Updated weights for policy 1, policy_version 1142211 (0.0008) [2023-12-26 23:41:01,592][105620] Updated weights for policy 1, policy_version 1142221 (0.0008) [2023-12-26 23:41:01,998][105692] Updated weights for policy 0, policy_version 1140842 (0.0007) [2023-12-26 23:41:02,067][105692] Updated weights for policy 0, policy_version 1140852 (0.0010) [2023-12-26 23:41:02,132][105692] Updated weights for policy 0, policy_version 1140862 (0.0010) [2023-12-26 23:41:02,197][105692] Updated weights for policy 0, policy_version 1140872 (0.0010) [2023-12-26 23:41:02,288][105620] Updated weights for policy 1, policy_version 1142231 (0.0006) [2023-12-26 23:41:02,347][105620] Updated weights for policy 1, policy_version 1142241 (0.0006) [2023-12-26 23:41:02,405][105620] Updated weights for policy 1, policy_version 1142251 (0.0006) [2023-12-26 23:41:02,890][105692] Updated weights for policy 0, policy_version 1140882 (0.0010) [2023-12-26 23:41:02,944][105692] Updated weights for policy 0, policy_version 1140892 (0.0010) [2023-12-26 23:41:03,002][105692] Updated weights for policy 0, policy_version 1140902 (0.0010) [2023-12-26 23:41:03,043][105620] Updated weights for policy 1, policy_version 1142261 (0.0007) [2023-12-26 23:41:03,102][105620] Updated weights for policy 1, policy_version 1142271 (0.0008) [2023-12-26 23:41:03,152][105620] Updated weights for policy 1, policy_version 1142281 (0.0009) [2023-12-26 23:41:03,599][105692] Updated weights for policy 0, policy_version 1140912 (0.0008) [2023-12-26 23:41:03,663][105692] Updated weights for policy 0, policy_version 1140922 (0.0007) [2023-12-26 23:41:03,718][105692] Updated weights for policy 0, policy_version 1140932 (0.0010) [2023-12-26 23:41:03,868][105620] Updated weights for policy 1, policy_version 1142291 (0.0010) [2023-12-26 23:41:03,929][105620] Updated weights for policy 1, policy_version 1142301 (0.0006) [2023-12-26 23:41:04,000][105620] Updated weights for policy 1, policy_version 1142311 (0.0008) [2023-12-26 23:41:04,409][105692] Updated weights for policy 0, policy_version 1140942 (0.0010) [2023-12-26 23:41:04,470][105692] Updated weights for policy 0, policy_version 1140952 (0.0010) [2023-12-26 23:41:04,527][105692] Updated weights for policy 0, policy_version 1140962 (0.0008) [2023-12-26 23:41:04,721][105620] Updated weights for policy 1, policy_version 1142321 (0.0010) [2023-12-26 23:41:04,782][105620] Updated weights for policy 1, policy_version 1142331 (0.0009) [2023-12-26 23:41:04,843][105620] Updated weights for policy 1, policy_version 1142341 (0.0009) [2023-12-26 23:41:04,910][105620] Updated weights for policy 1, policy_version 1142351 (0.0005) [2023-12-26 23:41:05,271][105692] Updated weights for policy 0, policy_version 1140972 (0.0009) [2023-12-26 23:41:05,335][105692] Updated weights for policy 0, policy_version 1140982 (0.0010) [2023-12-26 23:41:05,399][105692] Updated weights for policy 0, policy_version 1140992 (0.0010) [2023-12-26 23:41:05,580][105620] Updated weights for policy 1, policy_version 1142361 (0.0008) [2023-12-26 23:41:05,638][105620] Updated weights for policy 1, policy_version 1142371 (0.0008) [2023-12-26 23:41:05,687][105620] Updated weights for policy 1, policy_version 1142381 (0.0008) [2023-12-26 23:41:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19933.8, 300 sec: 19549.7). Total num frames: 584630272. Throughput: 0: 9901.1, 1: 9945.2. Samples: 584622012. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:41:06,063][104569] Avg episode reward: [(0, '8681.678'), (1, '8895.753')] [2023-12-26 23:41:06,122][105692] Updated weights for policy 0, policy_version 1141002 (0.0010) [2023-12-26 23:41:06,186][105692] Updated weights for policy 0, policy_version 1141012 (0.0011) [2023-12-26 23:41:06,245][105692] Updated weights for policy 0, policy_version 1141022 (0.0010) [2023-12-26 23:41:06,305][105692] Updated weights for policy 0, policy_version 1141032 (0.0010) [2023-12-26 23:41:06,455][105620] Updated weights for policy 1, policy_version 1142391 (0.0008) [2023-12-26 23:41:06,515][105620] Updated weights for policy 1, policy_version 1142401 (0.0008) [2023-12-26 23:41:06,582][105620] Updated weights for policy 1, policy_version 1142411 (0.0008) [2023-12-26 23:41:07,031][105692] Updated weights for policy 0, policy_version 1141042 (0.0008) [2023-12-26 23:41:07,095][105692] Updated weights for policy 0, policy_version 1141052 (0.0008) [2023-12-26 23:41:07,152][105692] Updated weights for policy 0, policy_version 1141062 (0.0008) [2023-12-26 23:41:07,346][105620] Updated weights for policy 1, policy_version 1142421 (0.0009) [2023-12-26 23:41:07,398][105620] Updated weights for policy 1, policy_version 1142431 (0.0010) [2023-12-26 23:41:07,450][105620] Updated weights for policy 1, policy_version 1142441 (0.0007) [2023-12-26 23:41:07,730][105692] Updated weights for policy 0, policy_version 1141072 (0.0007) [2023-12-26 23:41:07,792][105692] Updated weights for policy 0, policy_version 1141082 (0.0009) [2023-12-26 23:41:07,841][105692] Updated weights for policy 0, policy_version 1141092 (0.0009) [2023-12-26 23:41:08,156][105620] Updated weights for policy 1, policy_version 1142452 (0.0009) [2023-12-26 23:41:08,221][105620] Updated weights for policy 1, policy_version 1142462 (0.0006) [2023-12-26 23:41:08,281][105620] Updated weights for policy 1, policy_version 1142472 (0.0005) [2023-12-26 23:41:08,567][105692] Updated weights for policy 0, policy_version 1141103 (0.0007) [2023-12-26 23:41:08,634][105692] Updated weights for policy 0, policy_version 1141113 (0.0006) [2023-12-26 23:41:08,694][105692] Updated weights for policy 0, policy_version 1141123 (0.0007) [2023-12-26 23:41:08,859][105620] Updated weights for policy 1, policy_version 1142482 (0.0007) [2023-12-26 23:41:08,924][105620] Updated weights for policy 1, policy_version 1142492 (0.0009) [2023-12-26 23:41:08,972][105620] Updated weights for policy 1, policy_version 1142502 (0.0009) [2023-12-26 23:41:09,028][105620] Updated weights for policy 1, policy_version 1142512 (0.0009) [2023-12-26 23:41:09,240][105692] Updated weights for policy 0, policy_version 1141133 (0.0007) [2023-12-26 23:41:09,299][105692] Updated weights for policy 0, policy_version 1141143 (0.0008) [2023-12-26 23:41:09,365][105692] Updated weights for policy 0, policy_version 1141153 (0.0007) [2023-12-26 23:41:09,804][105620] Updated weights for policy 1, policy_version 1142522 (0.0010) [2023-12-26 23:41:09,865][105620] Updated weights for policy 1, policy_version 1142532 (0.0009) [2023-12-26 23:41:09,933][105620] Updated weights for policy 1, policy_version 1142542 (0.0007) [2023-12-26 23:41:10,136][105692] Updated weights for policy 0, policy_version 1141163 (0.0009) [2023-12-26 23:41:10,193][105692] Updated weights for policy 0, policy_version 1141173 (0.0009) [2023-12-26 23:41:10,246][105692] Updated weights for policy 0, policy_version 1141183 (0.0009) [2023-12-26 23:41:10,619][105620] Updated weights for policy 1, policy_version 1142552 (0.0007) [2023-12-26 23:41:10,672][105620] Updated weights for policy 1, policy_version 1142562 (0.0006) [2023-12-26 23:41:10,727][105620] Updated weights for policy 1, policy_version 1142572 (0.0006) [2023-12-26 23:41:10,981][105692] Updated weights for policy 0, policy_version 1141193 (0.0009) [2023-12-26 23:41:11,049][105692] Updated weights for policy 0, policy_version 1141203 (0.0010) [2023-12-26 23:41:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 584728576. Throughput: 0: 9956.0, 1: 9909.7. Samples: 584740124. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:41:11,063][104569] Avg episode reward: [(0, '8723.032'), (1, '9078.195')] [2023-12-26 23:41:11,109][105692] Updated weights for policy 0, policy_version 1141213 (0.0010) [2023-12-26 23:41:11,177][105692] Updated weights for policy 0, policy_version 1141223 (0.0012) [2023-12-26 23:41:11,403][105620] Updated weights for policy 1, policy_version 1142582 (0.0008) [2023-12-26 23:41:11,456][105620] Updated weights for policy 1, policy_version 1142592 (0.0007) [2023-12-26 23:41:11,510][105620] Updated weights for policy 1, policy_version 1142602 (0.0007) [2023-12-26 23:41:11,946][105692] Updated weights for policy 0, policy_version 1141233 (0.0010) [2023-12-26 23:41:11,998][105692] Updated weights for policy 0, policy_version 1141243 (0.0010) [2023-12-26 23:41:12,064][105692] Updated weights for policy 0, policy_version 1141253 (0.0010) [2023-12-26 23:41:12,259][105620] Updated weights for policy 1, policy_version 1142612 (0.0008) [2023-12-26 23:41:12,317][105620] Updated weights for policy 1, policy_version 1142622 (0.0008) [2023-12-26 23:41:12,386][105620] Updated weights for policy 1, policy_version 1142632 (0.0008) [2023-12-26 23:41:12,724][105692] Updated weights for policy 0, policy_version 1141263 (0.0005) [2023-12-26 23:41:12,786][105692] Updated weights for policy 0, policy_version 1141273 (0.0007) [2023-12-26 23:41:12,835][105692] Updated weights for policy 0, policy_version 1141283 (0.0005) [2023-12-26 23:41:13,027][105620] Updated weights for policy 1, policy_version 1142642 (0.0007) [2023-12-26 23:41:13,085][105620] Updated weights for policy 1, policy_version 1142652 (0.0010) [2023-12-26 23:41:13,133][105620] Updated weights for policy 1, policy_version 1142662 (0.0010) [2023-12-26 23:41:13,186][105620] Updated weights for policy 1, policy_version 1142672 (0.0009) [2023-12-26 23:41:13,533][105692] Updated weights for policy 0, policy_version 1141293 (0.0007) [2023-12-26 23:41:13,592][105692] Updated weights for policy 0, policy_version 1141303 (0.0007) [2023-12-26 23:41:13,653][105692] Updated weights for policy 0, policy_version 1141313 (0.0006) [2023-12-26 23:41:13,846][105620] Updated weights for policy 1, policy_version 1142682 (0.0006) [2023-12-26 23:41:13,898][105620] Updated weights for policy 1, policy_version 1142692 (0.0005) [2023-12-26 23:41:13,950][105620] Updated weights for policy 1, policy_version 1142702 (0.0005) [2023-12-26 23:41:14,423][105692] Updated weights for policy 0, policy_version 1141323 (0.0009) [2023-12-26 23:41:14,481][105692] Updated weights for policy 0, policy_version 1141333 (0.0009) [2023-12-26 23:41:14,551][105692] Updated weights for policy 0, policy_version 1141343 (0.0009) [2023-12-26 23:41:14,558][105620] Updated weights for policy 1, policy_version 1142712 (0.0005) [2023-12-26 23:41:14,625][105620] Updated weights for policy 1, policy_version 1142722 (0.0007) [2023-12-26 23:41:14,677][105620] Updated weights for policy 1, policy_version 1142732 (0.0009) [2023-12-26 23:41:15,271][105692] Updated weights for policy 0, policy_version 1141353 (0.0008) [2023-12-26 23:41:15,318][105692] Updated weights for policy 0, policy_version 1141363 (0.0005) [2023-12-26 23:41:15,376][105692] Updated weights for policy 0, policy_version 1141373 (0.0008) [2023-12-26 23:41:15,396][105620] Updated weights for policy 1, policy_version 1142742 (0.0008) [2023-12-26 23:41:15,429][105692] Updated weights for policy 0, policy_version 1141383 (0.0011) [2023-12-26 23:41:15,459][105620] Updated weights for policy 1, policy_version 1142752 (0.0008) [2023-12-26 23:41:15,527][105620] Updated weights for policy 1, policy_version 1142762 (0.0008) [2023-12-26 23:41:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 584826880. Throughput: 0: 9915.6, 1: 9950.9. Samples: 584800624. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:41:16,062][104569] Avg episode reward: [(0, '9176.463'), (1, '9169.477')] [2023-12-26 23:41:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001141384_292241408.pth... [2023-12-26 23:41:16,072][105620] Updated weights for policy 1, policy_version 1142772 (0.0008) [2023-12-26 23:41:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001140232_291946496.pth [2023-12-26 23:41:16,134][105692] Updated weights for policy 0, policy_version 1141393 (0.0010) [2023-12-26 23:41:16,137][105620] Updated weights for policy 1, policy_version 1142782 (0.0008) [2023-12-26 23:41:16,199][105620] Updated weights for policy 1, policy_version 1142792 (0.0007) [2023-12-26 23:41:16,200][105692] Updated weights for policy 0, policy_version 1141403 (0.0010) [2023-12-26 23:41:16,243][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001142800_292593664.pth... [2023-12-26 23:41:16,246][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001141616_292290560.pth [2023-12-26 23:41:16,252][105692] Updated weights for policy 0, policy_version 1141413 (0.0010) [2023-12-26 23:41:16,763][105620] Updated weights for policy 1, policy_version 1142802 (0.0007) [2023-12-26 23:41:16,819][105620] Updated weights for policy 1, policy_version 1142812 (0.0008) [2023-12-26 23:41:16,878][105620] Updated weights for policy 1, policy_version 1142822 (0.0007) [2023-12-26 23:41:16,928][105620] Updated weights for policy 1, policy_version 1142832 (0.0008) [2023-12-26 23:41:16,994][105692] Updated weights for policy 0, policy_version 1141423 (0.0010) [2023-12-26 23:41:17,049][105692] Updated weights for policy 0, policy_version 1141433 (0.0010) [2023-12-26 23:41:17,104][105692] Updated weights for policy 0, policy_version 1141443 (0.0010) [2023-12-26 23:41:17,682][105620] Updated weights for policy 1, policy_version 1142842 (0.0005) [2023-12-26 23:41:17,734][105620] Updated weights for policy 1, policy_version 1142852 (0.0005) [2023-12-26 23:41:17,790][105620] Updated weights for policy 1, policy_version 1142862 (0.0005) [2023-12-26 23:41:17,886][105692] Updated weights for policy 0, policy_version 1141453 (0.0011) [2023-12-26 23:41:17,944][105692] Updated weights for policy 0, policy_version 1141463 (0.0010) [2023-12-26 23:41:18,019][105692] Updated weights for policy 0, policy_version 1141473 (0.0010) [2023-12-26 23:41:18,389][105620] Updated weights for policy 1, policy_version 1142872 (0.0008) [2023-12-26 23:41:18,442][105620] Updated weights for policy 1, policy_version 1142882 (0.0011) [2023-12-26 23:41:18,491][105620] Updated weights for policy 1, policy_version 1142892 (0.0010) [2023-12-26 23:41:18,686][105692] Updated weights for policy 0, policy_version 1141483 (0.0007) [2023-12-26 23:41:18,741][105692] Updated weights for policy 0, policy_version 1141493 (0.0010) [2023-12-26 23:41:18,789][105692] Updated weights for policy 0, policy_version 1141503 (0.0010) [2023-12-26 23:41:19,148][105620] Updated weights for policy 1, policy_version 1142902 (0.0009) [2023-12-26 23:41:19,201][105620] Updated weights for policy 1, policy_version 1142912 (0.0008) [2023-12-26 23:41:19,277][105620] Updated weights for policy 1, policy_version 1142922 (0.0009) [2023-12-26 23:41:19,496][105692] Updated weights for policy 0, policy_version 1141513 (0.0010) [2023-12-26 23:41:19,553][105692] Updated weights for policy 0, policy_version 1141523 (0.0006) [2023-12-26 23:41:19,615][105692] Updated weights for policy 0, policy_version 1141533 (0.0007) [2023-12-26 23:41:19,679][105692] Updated weights for policy 0, policy_version 1141543 (0.0010) [2023-12-26 23:41:20,039][105620] Updated weights for policy 1, policy_version 1142932 (0.0007) [2023-12-26 23:41:20,102][105620] Updated weights for policy 1, policy_version 1142942 (0.0007) [2023-12-26 23:41:20,154][105620] Updated weights for policy 1, policy_version 1142952 (0.0008) [2023-12-26 23:41:20,360][105692] Updated weights for policy 0, policy_version 1141553 (0.0011) [2023-12-26 23:41:20,409][105692] Updated weights for policy 0, policy_version 1141563 (0.0010) [2023-12-26 23:41:20,462][105692] Updated weights for policy 0, policy_version 1141573 (0.0008) [2023-12-26 23:41:20,906][105620] Updated weights for policy 1, policy_version 1142962 (0.0009) [2023-12-26 23:41:20,976][105620] Updated weights for policy 1, policy_version 1142972 (0.0010) [2023-12-26 23:41:21,043][105620] Updated weights for policy 1, policy_version 1142982 (0.0008) [2023-12-26 23:41:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 584925184. Throughput: 0: 9854.8, 1: 9964.8. Samples: 584921324. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:41:21,063][104569] Avg episode reward: [(0, '9353.871'), (1, '9077.831')] [2023-12-26 23:41:21,105][105620] Updated weights for policy 1, policy_version 1142992 (0.0008) [2023-12-26 23:41:21,235][105692] Updated weights for policy 0, policy_version 1141583 (0.0010) [2023-12-26 23:41:21,297][105692] Updated weights for policy 0, policy_version 1141593 (0.0009) [2023-12-26 23:41:21,364][105692] Updated weights for policy 0, policy_version 1141603 (0.0011) [2023-12-26 23:41:21,917][105620] Updated weights for policy 1, policy_version 1143002 (0.0009) [2023-12-26 23:41:21,974][105620] Updated weights for policy 1, policy_version 1143012 (0.0008) [2023-12-26 23:41:22,033][105620] Updated weights for policy 1, policy_version 1143022 (0.0008) [2023-12-26 23:41:22,127][105692] Updated weights for policy 0, policy_version 1141613 (0.0011) [2023-12-26 23:41:22,180][105692] Updated weights for policy 0, policy_version 1141623 (0.0010) [2023-12-26 23:41:22,233][105692] Updated weights for policy 0, policy_version 1141633 (0.0010) [2023-12-26 23:41:22,730][105620] Updated weights for policy 1, policy_version 1143032 (0.0006) [2023-12-26 23:41:22,799][105620] Updated weights for policy 1, policy_version 1143042 (0.0008) [2023-12-26 23:41:22,864][105620] Updated weights for policy 1, policy_version 1143052 (0.0006) [2023-12-26 23:41:23,023][105692] Updated weights for policy 0, policy_version 1141643 (0.0009) [2023-12-26 23:41:23,079][105692] Updated weights for policy 0, policy_version 1141653 (0.0005) [2023-12-26 23:41:23,129][105692] Updated weights for policy 0, policy_version 1141663 (0.0006) [2023-12-26 23:41:23,509][105620] Updated weights for policy 1, policy_version 1143062 (0.0009) [2023-12-26 23:41:23,557][105620] Updated weights for policy 1, policy_version 1143072 (0.0010) [2023-12-26 23:41:23,606][105620] Updated weights for policy 1, policy_version 1143082 (0.0010) [2023-12-26 23:41:23,658][105692] Updated weights for policy 0, policy_version 1141673 (0.0006) [2023-12-26 23:41:23,713][105692] Updated weights for policy 0, policy_version 1141683 (0.0005) [2023-12-26 23:41:23,760][105692] Updated weights for policy 0, policy_version 1141693 (0.0005) [2023-12-26 23:41:23,805][105692] Updated weights for policy 0, policy_version 1141703 (0.0005) [2023-12-26 23:41:24,176][105620] Updated weights for policy 1, policy_version 1143092 (0.0008) [2023-12-26 23:41:24,237][105620] Updated weights for policy 1, policy_version 1143102 (0.0005) [2023-12-26 23:41:24,301][105620] Updated weights for policy 1, policy_version 1143112 (0.0006) [2023-12-26 23:41:24,480][105692] Updated weights for policy 0, policy_version 1141713 (0.0010) [2023-12-26 23:41:24,528][105692] Updated weights for policy 0, policy_version 1141723 (0.0010) [2023-12-26 23:41:24,591][105692] Updated weights for policy 0, policy_version 1141733 (0.0010) [2023-12-26 23:41:24,882][105620] Updated weights for policy 1, policy_version 1143122 (0.0007) [2023-12-26 23:41:24,943][105620] Updated weights for policy 1, policy_version 1143132 (0.0010) [2023-12-26 23:41:25,005][105620] Updated weights for policy 1, policy_version 1143142 (0.0010) [2023-12-26 23:41:25,065][105620] Updated weights for policy 1, policy_version 1143152 (0.0010) [2023-12-26 23:41:25,291][105692] Updated weights for policy 0, policy_version 1141743 (0.0007) [2023-12-26 23:41:25,358][105692] Updated weights for policy 0, policy_version 1141753 (0.0005) [2023-12-26 23:41:25,414][105692] Updated weights for policy 0, policy_version 1141763 (0.0006) [2023-12-26 23:41:25,798][105620] Updated weights for policy 1, policy_version 1143162 (0.0005) [2023-12-26 23:41:25,841][105620] Updated weights for policy 1, policy_version 1143172 (0.0005) [2023-12-26 23:41:25,884][105620] Updated weights for policy 1, policy_version 1143182 (0.0005) [2023-12-26 23:41:26,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 585031680. Throughput: 0: 9904.6, 1: 9974.4. Samples: 585041812. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:41:26,062][104569] Avg episode reward: [(0, '9170.411'), (1, '9348.120')] [2023-12-26 23:41:26,068][105692] Updated weights for policy 0, policy_version 1141773 (0.0006) [2023-12-26 23:41:26,131][105692] Updated weights for policy 0, policy_version 1141783 (0.0006) [2023-12-26 23:41:26,185][105692] Updated weights for policy 0, policy_version 1141793 (0.0008) [2023-12-26 23:41:26,481][105620] Updated weights for policy 1, policy_version 1143192 (0.0009) [2023-12-26 23:41:26,539][105620] Updated weights for policy 1, policy_version 1143202 (0.0010) [2023-12-26 23:41:26,597][105620] Updated weights for policy 1, policy_version 1143212 (0.0010) [2023-12-26 23:41:26,770][105692] Updated weights for policy 0, policy_version 1141803 (0.0007) [2023-12-26 23:41:26,818][105692] Updated weights for policy 0, policy_version 1141813 (0.0008) [2023-12-26 23:41:26,873][105692] Updated weights for policy 0, policy_version 1141823 (0.0008) [2023-12-26 23:41:27,303][105620] Updated weights for policy 1, policy_version 1143222 (0.0010) [2023-12-26 23:41:27,366][105620] Updated weights for policy 1, policy_version 1143232 (0.0010) [2023-12-26 23:41:27,418][105620] Updated weights for policy 1, policy_version 1143242 (0.0010) [2023-12-26 23:41:27,654][105692] Updated weights for policy 0, policy_version 1141833 (0.0008) [2023-12-26 23:41:27,711][105692] Updated weights for policy 0, policy_version 1141843 (0.0010) [2023-12-26 23:41:27,776][105692] Updated weights for policy 0, policy_version 1141853 (0.0005) [2023-12-26 23:41:27,833][105692] Updated weights for policy 0, policy_version 1141863 (0.0005) [2023-12-26 23:41:28,105][105620] Updated weights for policy 1, policy_version 1143252 (0.0008) [2023-12-26 23:41:28,155][105620] Updated weights for policy 1, policy_version 1143262 (0.0005) [2023-12-26 23:41:28,222][105620] Updated weights for policy 1, policy_version 1143272 (0.0007) [2023-12-26 23:41:28,421][105692] Updated weights for policy 0, policy_version 1141873 (0.0005) [2023-12-26 23:41:28,488][105692] Updated weights for policy 0, policy_version 1141883 (0.0005) [2023-12-26 23:41:28,558][105692] Updated weights for policy 0, policy_version 1141893 (0.0005) [2023-12-26 23:41:28,900][105620] Updated weights for policy 1, policy_version 1143282 (0.0010) [2023-12-26 23:41:28,965][105620] Updated weights for policy 1, policy_version 1143292 (0.0010) [2023-12-26 23:41:29,030][105620] Updated weights for policy 1, policy_version 1143302 (0.0010) [2023-12-26 23:41:29,073][105692] Updated weights for policy 0, policy_version 1141903 (0.0006) [2023-12-26 23:41:29,079][105620] Updated weights for policy 1, policy_version 1143312 (0.0010) [2023-12-26 23:41:29,137][105692] Updated weights for policy 0, policy_version 1141913 (0.0010) [2023-12-26 23:41:29,202][105692] Updated weights for policy 0, policy_version 1141923 (0.0010) [2023-12-26 23:41:29,829][105620] Updated weights for policy 1, policy_version 1143322 (0.0010) [2023-12-26 23:41:29,875][105692] Updated weights for policy 0, policy_version 1141933 (0.0009) [2023-12-26 23:41:29,896][105620] Updated weights for policy 1, policy_version 1143332 (0.0010) [2023-12-26 23:41:29,939][105692] Updated weights for policy 0, policy_version 1141943 (0.0011) [2023-12-26 23:41:29,960][105620] Updated weights for policy 1, policy_version 1143342 (0.0011) [2023-12-26 23:41:29,994][105692] Updated weights for policy 0, policy_version 1141953 (0.0010) [2023-12-26 23:41:30,671][105620] Updated weights for policy 1, policy_version 1143352 (0.0009) [2023-12-26 23:41:30,704][105692] Updated weights for policy 0, policy_version 1141963 (0.0011) [2023-12-26 23:41:30,725][105620] Updated weights for policy 1, policy_version 1143362 (0.0009) [2023-12-26 23:41:30,766][105692] Updated weights for policy 0, policy_version 1141973 (0.0010) [2023-12-26 23:41:30,781][105620] Updated weights for policy 1, policy_version 1143372 (0.0006) [2023-12-26 23:41:30,816][105692] Updated weights for policy 0, policy_version 1141983 (0.0010) [2023-12-26 23:41:31,062][104569] Fps is (10 sec: 21299.2, 60 sec: 20070.4, 300 sec: 19605.3). Total num frames: 585138176. Throughput: 0: 9970.0, 1: 9981.7. Samples: 585104016. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:41:31,062][104569] Avg episode reward: [(0, '8892.036'), (1, '9258.978')] [2023-12-26 23:41:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001141992_292397056.pth... [2023-12-26 23:41:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001143376_292741120.pth... [2023-12-26 23:41:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001142192_292438016.pth [2023-12-26 23:41:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001140808_292093952.pth [2023-12-26 23:41:31,454][105620] Updated weights for policy 1, policy_version 1143382 (0.0005) [2023-12-26 23:41:31,511][105620] Updated weights for policy 1, policy_version 1143392 (0.0005) [2023-12-26 23:41:31,553][105692] Updated weights for policy 0, policy_version 1141993 (0.0010) [2023-12-26 23:41:31,566][105620] Updated weights for policy 1, policy_version 1143402 (0.0005) [2023-12-26 23:41:31,621][105692] Updated weights for policy 0, policy_version 1142003 (0.0010) [2023-12-26 23:41:31,673][105692] Updated weights for policy 0, policy_version 1142013 (0.0006) [2023-12-26 23:41:31,736][105692] Updated weights for policy 0, policy_version 1142023 (0.0008) [2023-12-26 23:41:32,280][105692] Updated weights for policy 0, policy_version 1142033 (0.0007) [2023-12-26 23:41:32,295][105620] Updated weights for policy 1, policy_version 1143412 (0.0008) [2023-12-26 23:41:32,338][105692] Updated weights for policy 0, policy_version 1142043 (0.0005) [2023-12-26 23:41:32,347][105620] Updated weights for policy 1, policy_version 1143422 (0.0009) [2023-12-26 23:41:32,392][105692] Updated weights for policy 0, policy_version 1142053 (0.0008) [2023-12-26 23:41:32,413][105620] Updated weights for policy 1, policy_version 1143432 (0.0006) [2023-12-26 23:41:32,986][105692] Updated weights for policy 0, policy_version 1142063 (0.0006) [2023-12-26 23:41:33,056][105692] Updated weights for policy 0, policy_version 1142073 (0.0005) [2023-12-26 23:41:33,120][105692] Updated weights for policy 0, policy_version 1142083 (0.0009) [2023-12-26 23:41:33,225][105620] Updated weights for policy 1, policy_version 1143442 (0.0008) [2023-12-26 23:41:33,286][105620] Updated weights for policy 1, policy_version 1143452 (0.0009) [2023-12-26 23:41:33,348][105620] Updated weights for policy 1, policy_version 1143462 (0.0009) [2023-12-26 23:41:33,417][105620] Updated weights for policy 1, policy_version 1143472 (0.0009) [2023-12-26 23:41:33,658][105692] Updated weights for policy 0, policy_version 1142093 (0.0007) [2023-12-26 23:41:33,712][105692] Updated weights for policy 0, policy_version 1142103 (0.0008) [2023-12-26 23:41:33,763][105692] Updated weights for policy 0, policy_version 1142113 (0.0009) [2023-12-26 23:41:34,255][105620] Updated weights for policy 1, policy_version 1143482 (0.0008) [2023-12-26 23:41:34,306][105620] Updated weights for policy 1, policy_version 1143492 (0.0007) [2023-12-26 23:41:34,354][105620] Updated weights for policy 1, policy_version 1143502 (0.0008) [2023-12-26 23:41:34,417][105692] Updated weights for policy 0, policy_version 1142123 (0.0008) [2023-12-26 23:41:34,476][105692] Updated weights for policy 0, policy_version 1142133 (0.0009) [2023-12-26 23:41:34,534][105692] Updated weights for policy 0, policy_version 1142143 (0.0005) [2023-12-26 23:41:35,151][105620] Updated weights for policy 1, policy_version 1143512 (0.0009) [2023-12-26 23:41:35,202][105692] Updated weights for policy 0, policy_version 1142153 (0.0008) [2023-12-26 23:41:35,205][105620] Updated weights for policy 1, policy_version 1143523 (0.0008) [2023-12-26 23:41:35,252][105692] Updated weights for policy 0, policy_version 1142163 (0.0006) [2023-12-26 23:41:35,263][105620] Updated weights for policy 1, policy_version 1143533 (0.0008) [2023-12-26 23:41:35,307][105692] Updated weights for policy 0, policy_version 1142173 (0.0005) [2023-12-26 23:41:35,359][105692] Updated weights for policy 0, policy_version 1142183 (0.0007) [2023-12-26 23:41:35,914][105692] Updated weights for policy 0, policy_version 1142193 (0.0009) [2023-12-26 23:41:35,983][105692] Updated weights for policy 0, policy_version 1142203 (0.0008) [2023-12-26 23:41:36,048][105692] Updated weights for policy 0, policy_version 1142213 (0.0009) [2023-12-26 23:41:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 585228288. Throughput: 0: 10098.1, 1: 9942.8. Samples: 585224072. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:41:36,062][104569] Avg episode reward: [(0, '9077.498'), (1, '9258.764')] [2023-12-26 23:41:36,127][105620] Updated weights for policy 1, policy_version 1143543 (0.0009) [2023-12-26 23:41:36,183][105620] Updated weights for policy 1, policy_version 1143553 (0.0006) [2023-12-26 23:41:36,247][105620] Updated weights for policy 1, policy_version 1143563 (0.0007) [2023-12-26 23:41:36,797][105692] Updated weights for policy 0, policy_version 1142223 (0.0008) [2023-12-26 23:41:36,850][105692] Updated weights for policy 0, policy_version 1142233 (0.0009) [2023-12-26 23:41:36,900][105692] Updated weights for policy 0, policy_version 1142243 (0.0008) [2023-12-26 23:41:36,936][105620] Updated weights for policy 1, policy_version 1143573 (0.0008) [2023-12-26 23:41:36,993][105620] Updated weights for policy 1, policy_version 1143583 (0.0008) [2023-12-26 23:41:37,052][105620] Updated weights for policy 1, policy_version 1143593 (0.0009) [2023-12-26 23:41:37,676][105692] Updated weights for policy 0, policy_version 1142253 (0.0008) [2023-12-26 23:41:37,741][105692] Updated weights for policy 0, policy_version 1142263 (0.0009) [2023-12-26 23:41:37,795][105692] Updated weights for policy 0, policy_version 1142273 (0.0009) [2023-12-26 23:41:37,807][105620] Updated weights for policy 1, policy_version 1143603 (0.0008) [2023-12-26 23:41:37,866][105620] Updated weights for policy 1, policy_version 1143613 (0.0007) [2023-12-26 23:41:37,923][105620] Updated weights for policy 1, policy_version 1143623 (0.0008) [2023-12-26 23:41:38,579][105692] Updated weights for policy 0, policy_version 1142283 (0.0007) [2023-12-26 23:41:38,639][105692] Updated weights for policy 0, policy_version 1142293 (0.0009) [2023-12-26 23:41:38,702][105692] Updated weights for policy 0, policy_version 1142303 (0.0009) [2023-12-26 23:41:38,703][105620] Updated weights for policy 1, policy_version 1143633 (0.0009) [2023-12-26 23:41:38,762][105620] Updated weights for policy 1, policy_version 1143643 (0.0007) [2023-12-26 23:41:38,818][105620] Updated weights for policy 1, policy_version 1143653 (0.0008) [2023-12-26 23:41:38,872][105620] Updated weights for policy 1, policy_version 1143663 (0.0009) [2023-12-26 23:41:39,458][105692] Updated weights for policy 0, policy_version 1142313 (0.0010) [2023-12-26 23:41:39,516][105692] Updated weights for policy 0, policy_version 1142323 (0.0009) [2023-12-26 23:41:39,575][105692] Updated weights for policy 0, policy_version 1142333 (0.0009) [2023-12-26 23:41:39,629][105692] Updated weights for policy 0, policy_version 1142343 (0.0009) [2023-12-26 23:41:39,643][105620] Updated weights for policy 1, policy_version 1143673 (0.0006) [2023-12-26 23:41:39,705][105620] Updated weights for policy 1, policy_version 1143683 (0.0009) [2023-12-26 23:41:39,765][105620] Updated weights for policy 1, policy_version 1143693 (0.0009) [2023-12-26 23:41:40,338][105692] Updated weights for policy 0, policy_version 1142353 (0.0009) [2023-12-26 23:41:40,393][105692] Updated weights for policy 0, policy_version 1142363 (0.0009) [2023-12-26 23:41:40,442][105692] Updated weights for policy 0, policy_version 1142373 (0.0009) [2023-12-26 23:41:40,589][105620] Updated weights for policy 1, policy_version 1143703 (0.0010) [2023-12-26 23:41:40,657][105620] Updated weights for policy 1, policy_version 1143713 (0.0009) [2023-12-26 23:41:40,720][105620] Updated weights for policy 1, policy_version 1143723 (0.0009) [2023-12-26 23:41:41,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 585326592. Throughput: 0: 10075.7, 1: 9806.5. Samples: 585335940. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:41:41,063][104569] Avg episode reward: [(0, '9263.490'), (1, '8196.922')] [2023-12-26 23:41:41,136][105692] Updated weights for policy 0, policy_version 1142383 (0.0008) [2023-12-26 23:41:41,193][105692] Updated weights for policy 0, policy_version 1142393 (0.0007) [2023-12-26 23:41:41,244][105692] Updated weights for policy 0, policy_version 1142403 (0.0008) [2023-12-26 23:41:41,541][105620] Updated weights for policy 1, policy_version 1143733 (0.0008) [2023-12-26 23:41:41,603][105620] Updated weights for policy 1, policy_version 1143743 (0.0008) [2023-12-26 23:41:41,684][105620] Updated weights for policy 1, policy_version 1143753 (0.0007) [2023-12-26 23:41:42,034][105692] Updated weights for policy 0, policy_version 1142413 (0.0007) [2023-12-26 23:41:42,102][105692] Updated weights for policy 0, policy_version 1142423 (0.0008) [2023-12-26 23:41:42,170][105692] Updated weights for policy 0, policy_version 1142433 (0.0008) [2023-12-26 23:41:42,356][105620] Updated weights for policy 1, policy_version 1143763 (0.0008) [2023-12-26 23:41:42,423][105620] Updated weights for policy 1, policy_version 1143773 (0.0007) [2023-12-26 23:41:42,483][105620] Updated weights for policy 1, policy_version 1143783 (0.0008) [2023-12-26 23:41:42,865][105692] Updated weights for policy 0, policy_version 1142443 (0.0008) [2023-12-26 23:41:42,926][105692] Updated weights for policy 0, policy_version 1142453 (0.0008) [2023-12-26 23:41:42,979][105692] Updated weights for policy 0, policy_version 1142463 (0.0008) [2023-12-26 23:41:43,235][105620] Updated weights for policy 1, policy_version 1143793 (0.0008) [2023-12-26 23:41:43,286][105620] Updated weights for policy 1, policy_version 1143803 (0.0008) [2023-12-26 23:41:43,333][105620] Updated weights for policy 1, policy_version 1143813 (0.0009) [2023-12-26 23:41:43,387][105620] Updated weights for policy 1, policy_version 1143823 (0.0008) [2023-12-26 23:41:43,746][105692] Updated weights for policy 0, policy_version 1142473 (0.0009) [2023-12-26 23:41:43,808][105692] Updated weights for policy 0, policy_version 1142483 (0.0005) [2023-12-26 23:41:43,857][105692] Updated weights for policy 0, policy_version 1142493 (0.0005) [2023-12-26 23:41:43,901][105692] Updated weights for policy 0, policy_version 1142503 (0.0005) [2023-12-26 23:41:44,103][105620] Updated weights for policy 1, policy_version 1143833 (0.0010) [2023-12-26 23:41:44,160][105620] Updated weights for policy 1, policy_version 1143844 (0.0009) [2023-12-26 23:41:44,217][105620] Updated weights for policy 1, policy_version 1143855 (0.0009) [2023-12-26 23:41:44,440][105692] Updated weights for policy 0, policy_version 1142513 (0.0010) [2023-12-26 23:41:44,492][105692] Updated weights for policy 0, policy_version 1142523 (0.0010) [2023-12-26 23:41:44,540][105692] Updated weights for policy 0, policy_version 1142533 (0.0010) [2023-12-26 23:41:45,004][105620] Updated weights for policy 1, policy_version 1143865 (0.0008) [2023-12-26 23:41:45,068][105620] Updated weights for policy 1, policy_version 1143875 (0.0006) [2023-12-26 23:41:45,136][105620] Updated weights for policy 1, policy_version 1143885 (0.0006) [2023-12-26 23:41:45,283][105692] Updated weights for policy 0, policy_version 1142543 (0.0010) [2023-12-26 23:41:45,348][105692] Updated weights for policy 0, policy_version 1142553 (0.0008) [2023-12-26 23:41:45,413][105692] Updated weights for policy 0, policy_version 1142563 (0.0011) [2023-12-26 23:41:45,757][105620] Updated weights for policy 1, policy_version 1143895 (0.0008) [2023-12-26 23:41:45,817][105620] Updated weights for policy 1, policy_version 1143905 (0.0008) [2023-12-26 23:41:45,872][105620] Updated weights for policy 1, policy_version 1143915 (0.0008) [2023-12-26 23:41:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 585424896. Throughput: 0: 9973.9, 1: 9806.6. Samples: 585393552. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:41:46,063][104569] Avg episode reward: [(0, '9357.218'), (1, '8087.447')] [2023-12-26 23:41:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001143920_292880384.pth... [2023-12-26 23:41:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001142800_292593664.pth [2023-12-26 23:41:46,073][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_001143920_292880384.pth [2023-12-26 23:41:46,089][105692] Updated weights for policy 0, policy_version 1142573 (0.0010) [2023-12-26 23:41:46,141][105692] Updated weights for policy 0, policy_version 1142583 (0.0010) [2023-12-26 23:41:46,199][105692] Updated weights for policy 0, policy_version 1142593 (0.0010) [2023-12-26 23:41:46,239][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001142600_292552704.pth... [2023-12-26 23:41:46,243][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001141384_292241408.pth [2023-12-26 23:41:46,244][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_001142600_292552704.pth [2023-12-26 23:41:46,662][105620] Updated weights for policy 1, policy_version 1143925 (0.0008) [2023-12-26 23:41:46,714][105620] Updated weights for policy 1, policy_version 1143935 (0.0007) [2023-12-26 23:41:46,773][105620] Updated weights for policy 1, policy_version 1143946 (0.0008) [2023-12-26 23:41:46,795][105692] Updated weights for policy 0, policy_version 1142603 (0.0010) [2023-12-26 23:41:46,842][105692] Updated weights for policy 0, policy_version 1142613 (0.0009) [2023-12-26 23:41:46,889][105692] Updated weights for policy 0, policy_version 1142623 (0.0009) [2023-12-26 23:41:47,463][105692] Updated weights for policy 0, policy_version 1142633 (0.0008) [2023-12-26 23:41:47,517][105692] Updated weights for policy 0, policy_version 1142643 (0.0005) [2023-12-26 23:41:47,576][105692] Updated weights for policy 0, policy_version 1142653 (0.0005) [2023-12-26 23:41:47,600][105620] Updated weights for policy 1, policy_version 1143956 (0.0010) [2023-12-26 23:41:47,635][105692] Updated weights for policy 0, policy_version 1142663 (0.0005) [2023-12-26 23:41:47,657][105620] Updated weights for policy 1, policy_version 1143966 (0.0009) [2023-12-26 23:41:47,716][105620] Updated weights for policy 1, policy_version 1143976 (0.0010) [2023-12-26 23:41:48,224][105692] Updated weights for policy 0, policy_version 1142673 (0.0006) [2023-12-26 23:41:48,292][105692] Updated weights for policy 0, policy_version 1142683 (0.0006) [2023-12-26 23:41:48,365][105692] Updated weights for policy 0, policy_version 1142693 (0.0007) [2023-12-26 23:41:48,426][105620] Updated weights for policy 1, policy_version 1143986 (0.0009) [2023-12-26 23:41:48,489][105620] Updated weights for policy 1, policy_version 1143996 (0.0011) [2023-12-26 23:41:48,552][105620] Updated weights for policy 1, policy_version 1144006 (0.0010) [2023-12-26 23:41:48,621][105620] Updated weights for policy 1, policy_version 1144016 (0.0011) [2023-12-26 23:41:49,090][105692] Updated weights for policy 0, policy_version 1142703 (0.0009) [2023-12-26 23:41:49,139][105692] Updated weights for policy 0, policy_version 1142713 (0.0006) [2023-12-26 23:41:49,193][105692] Updated weights for policy 0, policy_version 1142723 (0.0008) [2023-12-26 23:41:49,314][105620] Updated weights for policy 1, policy_version 1144026 (0.0006) [2023-12-26 23:41:49,378][105620] Updated weights for policy 1, policy_version 1144036 (0.0008) [2023-12-26 23:41:49,441][105620] Updated weights for policy 1, policy_version 1144046 (0.0005) [2023-12-26 23:41:49,940][105692] Updated weights for policy 0, policy_version 1142733 (0.0009) [2023-12-26 23:41:49,999][105692] Updated weights for policy 0, policy_version 1142743 (0.0010) [2023-12-26 23:41:50,061][105692] Updated weights for policy 0, policy_version 1142753 (0.0010) [2023-12-26 23:41:50,083][105620] Updated weights for policy 1, policy_version 1144056 (0.0007) [2023-12-26 23:41:50,146][105620] Updated weights for policy 1, policy_version 1144066 (0.0007) [2023-12-26 23:41:50,215][105620] Updated weights for policy 1, policy_version 1144076 (0.0006) [2023-12-26 23:41:50,802][105692] Updated weights for policy 0, policy_version 1142763 (0.0007) [2023-12-26 23:41:50,820][105620] Updated weights for policy 1, policy_version 1144086 (0.0007) [2023-12-26 23:41:50,864][105692] Updated weights for policy 0, policy_version 1142773 (0.0008) [2023-12-26 23:41:50,878][105620] Updated weights for policy 1, policy_version 1144096 (0.0006) [2023-12-26 23:41:50,915][105692] Updated weights for policy 0, policy_version 1142783 (0.0008) [2023-12-26 23:41:50,927][105620] Updated weights for policy 1, policy_version 1144106 (0.0006) [2023-12-26 23:41:51,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 585531392. Throughput: 0: 10083.8, 1: 9749.6. Samples: 585514512. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:41:51,062][104569] Avg episode reward: [(0, '9082.055'), (1, '9163.092')] [2023-12-26 23:41:51,664][105620] Updated weights for policy 1, policy_version 1144116 (0.0010) [2023-12-26 23:41:51,706][105692] Updated weights for policy 0, policy_version 1142793 (0.0007) [2023-12-26 23:41:51,733][105620] Updated weights for policy 1, policy_version 1144126 (0.0010) [2023-12-26 23:41:51,771][105692] Updated weights for policy 0, policy_version 1142803 (0.0008) [2023-12-26 23:41:51,790][105620] Updated weights for policy 1, policy_version 1144136 (0.0011) [2023-12-26 23:41:51,821][105692] Updated weights for policy 0, policy_version 1142813 (0.0006) [2023-12-26 23:41:51,880][105692] Updated weights for policy 0, policy_version 1142823 (0.0009) [2023-12-26 23:41:52,494][105620] Updated weights for policy 1, policy_version 1144146 (0.0011) [2023-12-26 23:41:52,558][105620] Updated weights for policy 1, policy_version 1144156 (0.0006) [2023-12-26 23:41:52,626][105620] Updated weights for policy 1, policy_version 1144166 (0.0006) [2023-12-26 23:41:52,673][105692] Updated weights for policy 0, policy_version 1142833 (0.0008) [2023-12-26 23:41:52,690][105620] Updated weights for policy 1, policy_version 1144176 (0.0006) [2023-12-26 23:41:52,727][105692] Updated weights for policy 0, policy_version 1142843 (0.0009) [2023-12-26 23:41:52,781][105692] Updated weights for policy 0, policy_version 1142853 (0.0009) [2023-12-26 23:41:53,296][105620] Updated weights for policy 1, policy_version 1144186 (0.0008) [2023-12-26 23:41:53,359][105620] Updated weights for policy 1, policy_version 1144196 (0.0008) [2023-12-26 23:41:53,414][105620] Updated weights for policy 1, policy_version 1144206 (0.0008) [2023-12-26 23:41:53,530][105692] Updated weights for policy 0, policy_version 1142863 (0.0010) [2023-12-26 23:41:53,588][105692] Updated weights for policy 0, policy_version 1142873 (0.0010) [2023-12-26 23:41:53,639][105692] Updated weights for policy 0, policy_version 1142883 (0.0010) [2023-12-26 23:41:54,002][105620] Updated weights for policy 1, policy_version 1144216 (0.0006) [2023-12-26 23:41:54,048][105620] Updated weights for policy 1, policy_version 1144226 (0.0005) [2023-12-26 23:41:54,103][105620] Updated weights for policy 1, policy_version 1144236 (0.0006) [2023-12-26 23:41:54,344][105692] Updated weights for policy 0, policy_version 1142893 (0.0008) [2023-12-26 23:41:54,407][105692] Updated weights for policy 0, policy_version 1142903 (0.0005) [2023-12-26 23:41:54,471][105692] Updated weights for policy 0, policy_version 1142913 (0.0005) [2023-12-26 23:41:54,756][105620] Updated weights for policy 1, policy_version 1144246 (0.0005) [2023-12-26 23:41:54,820][105620] Updated weights for policy 1, policy_version 1144256 (0.0005) [2023-12-26 23:41:54,871][105620] Updated weights for policy 1, policy_version 1144266 (0.0005) [2023-12-26 23:41:55,044][105692] Updated weights for policy 0, policy_version 1142923 (0.0008) [2023-12-26 23:41:55,098][105692] Updated weights for policy 0, policy_version 1142933 (0.0010) [2023-12-26 23:41:55,150][105692] Updated weights for policy 0, policy_version 1142943 (0.0009) [2023-12-26 23:41:55,413][105620] Updated weights for policy 1, policy_version 1144276 (0.0007) [2023-12-26 23:41:55,463][105620] Updated weights for policy 1, policy_version 1144286 (0.0008) [2023-12-26 23:41:55,517][105620] Updated weights for policy 1, policy_version 1144296 (0.0009) [2023-12-26 23:41:55,852][105692] Updated weights for policy 0, policy_version 1142954 (0.0010) [2023-12-26 23:41:55,916][105692] Updated weights for policy 0, policy_version 1142964 (0.0009) [2023-12-26 23:41:55,953][105585] KL-divergence is very high: 101.1673 [2023-12-26 23:41:55,985][105692] Updated weights for policy 0, policy_version 1142974 (0.0009) [2023-12-26 23:41:56,004][105585] KL-divergence is very high: 152.0568 [2023-12-26 23:41:56,048][105692] Updated weights for policy 0, policy_version 1142984 (0.0008) [2023-12-26 23:41:56,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 585629696. Throughput: 0: 10034.8, 1: 9840.4. Samples: 585634504. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:41:56,063][104569] Avg episode reward: [(0, '8808.608'), (1, '9076.575')] [2023-12-26 23:41:56,249][105620] Updated weights for policy 1, policy_version 1144306 (0.0007) [2023-12-26 23:41:56,300][105620] Updated weights for policy 1, policy_version 1144316 (0.0008) [2023-12-26 23:41:56,359][105620] Updated weights for policy 1, policy_version 1144326 (0.0009) [2023-12-26 23:41:56,410][105620] Updated weights for policy 1, policy_version 1144336 (0.0009) [2023-12-26 23:41:56,809][105692] Updated weights for policy 0, policy_version 1142994 (0.0009) [2023-12-26 23:41:56,860][105692] Updated weights for policy 0, policy_version 1143004 (0.0009) [2023-12-26 23:41:56,910][105692] Updated weights for policy 0, policy_version 1143014 (0.0008) [2023-12-26 23:41:57,102][105620] Updated weights for policy 1, policy_version 1144346 (0.0005) [2023-12-26 23:41:57,149][105620] Updated weights for policy 1, policy_version 1144356 (0.0005) [2023-12-26 23:41:57,201][105620] Updated weights for policy 1, policy_version 1144366 (0.0006) [2023-12-26 23:41:57,644][105692] Updated weights for policy 0, policy_version 1143024 (0.0006) [2023-12-26 23:41:57,716][105692] Updated weights for policy 0, policy_version 1143034 (0.0005) [2023-12-26 23:41:57,777][105692] Updated weights for policy 0, policy_version 1143044 (0.0010) [2023-12-26 23:41:57,799][105620] Updated weights for policy 1, policy_version 1144376 (0.0006) [2023-12-26 23:41:57,860][105620] Updated weights for policy 1, policy_version 1144386 (0.0006) [2023-12-26 23:41:57,923][105620] Updated weights for policy 1, policy_version 1144396 (0.0005) [2023-12-26 23:41:58,403][105692] Updated weights for policy 0, policy_version 1143054 (0.0008) [2023-12-26 23:41:58,473][105692] Updated weights for policy 0, policy_version 1143064 (0.0006) [2023-12-26 23:41:58,537][105692] Updated weights for policy 0, policy_version 1143074 (0.0008) [2023-12-26 23:41:58,658][105620] Updated weights for policy 1, policy_version 1144406 (0.0009) [2023-12-26 23:41:58,722][105620] Updated weights for policy 1, policy_version 1144416 (0.0011) [2023-12-26 23:41:58,787][105620] Updated weights for policy 1, policy_version 1144426 (0.0010) [2023-12-26 23:41:59,275][105692] Updated weights for policy 0, policy_version 1143084 (0.0006) [2023-12-26 23:41:59,356][105692] Updated weights for policy 0, policy_version 1143094 (0.0009) [2023-12-26 23:41:59,412][105692] Updated weights for policy 0, policy_version 1143104 (0.0008) [2023-12-26 23:41:59,543][105620] Updated weights for policy 1, policy_version 1144436 (0.0007) [2023-12-26 23:41:59,605][105620] Updated weights for policy 1, policy_version 1144446 (0.0008) [2023-12-26 23:41:59,662][105620] Updated weights for policy 1, policy_version 1144456 (0.0010) [2023-12-26 23:42:00,106][105692] Updated weights for policy 0, policy_version 1143114 (0.0010) [2023-12-26 23:42:00,168][105692] Updated weights for policy 0, policy_version 1143124 (0.0009) [2023-12-26 23:42:00,234][105692] Updated weights for policy 0, policy_version 1143134 (0.0009) [2023-12-26 23:42:00,290][105692] Updated weights for policy 0, policy_version 1143144 (0.0009) [2023-12-26 23:42:00,370][105620] Updated weights for policy 1, policy_version 1144466 (0.0008) [2023-12-26 23:42:00,431][105620] Updated weights for policy 1, policy_version 1144476 (0.0008) [2023-12-26 23:42:00,497][105620] Updated weights for policy 1, policy_version 1144486 (0.0008) [2023-12-26 23:42:00,558][105620] Updated weights for policy 1, policy_version 1144496 (0.0008) [2023-12-26 23:42:01,062][105692] Updated weights for policy 0, policy_version 1143154 (0.0009) [2023-12-26 23:42:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 585719808. Throughput: 0: 10042.5, 1: 9843.6. Samples: 585695500. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-12-26 23:42:01,063][104569] Avg episode reward: [(0, '8992.669'), (1, '8899.565')] [2023-12-26 23:42:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001144496_293027840.pth... [2023-12-26 23:42:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001143376_292741120.pth [2023-12-26 23:42:01,130][105692] Updated weights for policy 0, policy_version 1143164 (0.0008) [2023-12-26 23:42:01,183][105620] Updated weights for policy 1, policy_version 1144506 (0.0007) [2023-12-26 23:42:01,198][105692] Updated weights for policy 0, policy_version 1143174 (0.0008) [2023-12-26 23:42:01,207][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001143176_292700160.pth... [2023-12-26 23:42:01,211][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001141992_292397056.pth [2023-12-26 23:42:01,252][105620] Updated weights for policy 1, policy_version 1144516 (0.0007) [2023-12-26 23:42:01,308][105620] Updated weights for policy 1, policy_version 1144526 (0.0006) [2023-12-26 23:42:01,967][105692] Updated weights for policy 0, policy_version 1143184 (0.0009) [2023-12-26 23:42:02,023][105620] Updated weights for policy 1, policy_version 1144536 (0.0007) [2023-12-26 23:42:02,025][105692] Updated weights for policy 0, policy_version 1143194 (0.0008) [2023-12-26 23:42:02,081][105692] Updated weights for policy 0, policy_version 1143204 (0.0009) [2023-12-26 23:42:02,083][105620] Updated weights for policy 1, policy_version 1144546 (0.0005) [2023-12-26 23:42:02,136][105620] Updated weights for policy 1, policy_version 1144556 (0.0005) [2023-12-26 23:42:02,727][105620] Updated weights for policy 1, policy_version 1144566 (0.0005) [2023-12-26 23:42:02,796][105620] Updated weights for policy 1, policy_version 1144576 (0.0005) [2023-12-26 23:42:02,862][105620] Updated weights for policy 1, policy_version 1144586 (0.0008) [2023-12-26 23:42:02,895][105692] Updated weights for policy 0, policy_version 1143214 (0.0007) [2023-12-26 23:42:02,944][105692] Updated weights for policy 0, policy_version 1143224 (0.0008) [2023-12-26 23:42:02,988][105692] Updated weights for policy 0, policy_version 1143234 (0.0007) [2023-12-26 23:42:03,503][105620] Updated weights for policy 1, policy_version 1144596 (0.0010) [2023-12-26 23:42:03,547][105620] Updated weights for policy 1, policy_version 1144606 (0.0010) [2023-12-26 23:42:03,602][105620] Updated weights for policy 1, policy_version 1144616 (0.0010) [2023-12-26 23:42:03,805][105692] Updated weights for policy 0, policy_version 1143245 (0.0009) [2023-12-26 23:42:03,867][105692] Updated weights for policy 0, policy_version 1143256 (0.0010) [2023-12-26 23:42:03,920][105692] Updated weights for policy 0, policy_version 1143267 (0.0010) [2023-12-26 23:42:04,203][105620] Updated weights for policy 1, policy_version 1144626 (0.0006) [2023-12-26 23:42:04,259][105620] Updated weights for policy 1, policy_version 1144636 (0.0010) [2023-12-26 23:42:04,311][105620] Updated weights for policy 1, policy_version 1144646 (0.0010) [2023-12-26 23:42:04,367][105620] Updated weights for policy 1, policy_version 1144656 (0.0010) [2023-12-26 23:42:04,779][105692] Updated weights for policy 0, policy_version 1143277 (0.0009) [2023-12-26 23:42:04,837][105692] Updated weights for policy 0, policy_version 1143287 (0.0010) [2023-12-26 23:42:04,838][105585] KL-divergence is very high: 111.4093 [2023-12-26 23:42:04,883][105585] KL-divergence is very high: 176.2675 [2023-12-26 23:42:04,895][105692] Updated weights for policy 0, policy_version 1143297 (0.0009) [2023-12-26 23:42:04,927][105585] KL-divergence is very high: 158.7020 [2023-12-26 23:42:04,987][105620] Updated weights for policy 1, policy_version 1144666 (0.0005) [2023-12-26 23:42:05,047][105620] Updated weights for policy 1, policy_version 1144676 (0.0009) [2023-12-26 23:42:05,104][105620] Updated weights for policy 1, policy_version 1144686 (0.0010) [2023-12-26 23:42:05,581][105692] Updated weights for policy 0, policy_version 1143307 (0.0009) [2023-12-26 23:42:05,645][105692] Updated weights for policy 0, policy_version 1143317 (0.0005) [2023-12-26 23:42:05,693][105620] Updated weights for policy 1, policy_version 1144696 (0.0007) [2023-12-26 23:42:05,705][105692] Updated weights for policy 0, policy_version 1143327 (0.0006) [2023-12-26 23:42:05,755][105620] Updated weights for policy 1, policy_version 1144706 (0.0005) [2023-12-26 23:42:05,809][105620] Updated weights for policy 1, policy_version 1144716 (0.0005) [2023-12-26 23:42:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19933.8, 300 sec: 19633.0). Total num frames: 585826304. Throughput: 0: 9972.8, 1: 9846.5. Samples: 585813192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:42:06,063][104569] Avg episode reward: [(0, '9264.867'), (1, '9079.620')] [2023-12-26 23:42:06,378][105692] Updated weights for policy 0, policy_version 1143337 (0.0006) [2023-12-26 23:42:06,399][105620] Updated weights for policy 1, policy_version 1144726 (0.0009) [2023-12-26 23:42:06,440][105692] Updated weights for policy 0, policy_version 1143347 (0.0010) [2023-12-26 23:42:06,461][105620] Updated weights for policy 1, policy_version 1144736 (0.0011) [2023-12-26 23:42:06,508][105692] Updated weights for policy 0, policy_version 1143357 (0.0006) [2023-12-26 23:42:06,524][105620] Updated weights for policy 1, policy_version 1144746 (0.0010) [2023-12-26 23:42:06,572][105692] Updated weights for policy 0, policy_version 1143367 (0.0006) [2023-12-26 23:42:07,214][105620] Updated weights for policy 1, policy_version 1144756 (0.0009) [2023-12-26 23:42:07,275][105620] Updated weights for policy 1, policy_version 1144766 (0.0008) [2023-12-26 23:42:07,302][105692] Updated weights for policy 0, policy_version 1143377 (0.0006) [2023-12-26 23:42:07,333][105620] Updated weights for policy 1, policy_version 1144776 (0.0007) [2023-12-26 23:42:07,367][105692] Updated weights for policy 0, policy_version 1143387 (0.0007) [2023-12-26 23:42:07,426][105692] Updated weights for policy 0, policy_version 1143397 (0.0007) [2023-12-26 23:42:07,958][105620] Updated weights for policy 1, policy_version 1144786 (0.0005) [2023-12-26 23:42:08,012][105620] Updated weights for policy 1, policy_version 1144796 (0.0005) [2023-12-26 23:42:08,084][105620] Updated weights for policy 1, policy_version 1144806 (0.0006) [2023-12-26 23:42:08,105][105692] Updated weights for policy 0, policy_version 1143407 (0.0006) [2023-12-26 23:42:08,147][105620] Updated weights for policy 1, policy_version 1144816 (0.0006) [2023-12-26 23:42:08,162][105692] Updated weights for policy 0, policy_version 1143417 (0.0005) [2023-12-26 23:42:08,211][105692] Updated weights for policy 0, policy_version 1143427 (0.0007) [2023-12-26 23:42:08,784][105620] Updated weights for policy 1, policy_version 1144826 (0.0009) [2023-12-26 23:42:08,842][105620] Updated weights for policy 1, policy_version 1144836 (0.0009) [2023-12-26 23:42:08,886][105692] Updated weights for policy 0, policy_version 1143437 (0.0007) [2023-12-26 23:42:08,906][105620] Updated weights for policy 1, policy_version 1144846 (0.0008) [2023-12-26 23:42:08,942][105692] Updated weights for policy 0, policy_version 1143447 (0.0008) [2023-12-26 23:42:08,992][105692] Updated weights for policy 0, policy_version 1143457 (0.0009) [2023-12-26 23:42:09,706][105620] Updated weights for policy 1, policy_version 1144856 (0.0009) [2023-12-26 23:42:09,740][105692] Updated weights for policy 0, policy_version 1143467 (0.0009) [2023-12-26 23:42:09,755][105620] Updated weights for policy 1, policy_version 1144866 (0.0008) [2023-12-26 23:42:09,795][105692] Updated weights for policy 0, policy_version 1143477 (0.0008) [2023-12-26 23:42:09,811][105620] Updated weights for policy 1, policy_version 1144876 (0.0006) [2023-12-26 23:42:09,858][105692] Updated weights for policy 0, policy_version 1143487 (0.0009) [2023-12-26 23:42:10,534][105692] Updated weights for policy 0, policy_version 1143497 (0.0009) [2023-12-26 23:42:10,585][105692] Updated weights for policy 0, policy_version 1143507 (0.0009) [2023-12-26 23:42:10,612][105620] Updated weights for policy 1, policy_version 1144886 (0.0007) [2023-12-26 23:42:10,642][105692] Updated weights for policy 0, policy_version 1143517 (0.0007) [2023-12-26 23:42:10,672][105620] Updated weights for policy 1, policy_version 1144896 (0.0007) [2023-12-26 23:42:10,698][105692] Updated weights for policy 0, policy_version 1143527 (0.0007) [2023-12-26 23:42:10,734][105620] Updated weights for policy 1, policy_version 1144906 (0.0010) [2023-12-26 23:42:11,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 585924608. Throughput: 0: 9956.4, 1: 9848.4. Samples: 585933028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:42:11,063][104569] Avg episode reward: [(0, '9172.022'), (1, '9346.647')] [2023-12-26 23:42:11,478][105620] Updated weights for policy 1, policy_version 1144916 (0.0008) [2023-12-26 23:42:11,528][105692] Updated weights for policy 0, policy_version 1143537 (0.0008) [2023-12-26 23:42:11,540][105620] Updated weights for policy 1, policy_version 1144926 (0.0009) [2023-12-26 23:42:11,588][105692] Updated weights for policy 0, policy_version 1143547 (0.0007) [2023-12-26 23:42:11,602][105620] Updated weights for policy 1, policy_version 1144936 (0.0009) [2023-12-26 23:42:11,655][105692] Updated weights for policy 0, policy_version 1143557 (0.0007) [2023-12-26 23:42:12,367][105620] Updated weights for policy 1, policy_version 1144946 (0.0008) [2023-12-26 23:42:12,430][105620] Updated weights for policy 1, policy_version 1144956 (0.0008) [2023-12-26 23:42:12,455][105692] Updated weights for policy 0, policy_version 1143567 (0.0010) [2023-12-26 23:42:12,493][105620] Updated weights for policy 1, policy_version 1144966 (0.0007) [2023-12-26 23:42:12,509][105692] Updated weights for policy 0, policy_version 1143577 (0.0010) [2023-12-26 23:42:12,550][105620] Updated weights for policy 1, policy_version 1144976 (0.0008) [2023-12-26 23:42:12,566][105692] Updated weights for policy 0, policy_version 1143587 (0.0007) [2023-12-26 23:42:13,249][105692] Updated weights for policy 0, policy_version 1143597 (0.0009) [2023-12-26 23:42:13,309][105692] Updated weights for policy 0, policy_version 1143607 (0.0006) [2023-12-26 23:42:13,315][105620] Updated weights for policy 1, policy_version 1144986 (0.0008) [2023-12-26 23:42:13,371][105692] Updated weights for policy 0, policy_version 1143617 (0.0007) [2023-12-26 23:42:13,377][105620] Updated weights for policy 1, policy_version 1144996 (0.0006) [2023-12-26 23:42:13,434][105620] Updated weights for policy 1, policy_version 1145006 (0.0008) [2023-12-26 23:42:14,031][105692] Updated weights for policy 0, policy_version 1143627 (0.0007) [2023-12-26 23:42:14,095][105692] Updated weights for policy 0, policy_version 1143637 (0.0009) [2023-12-26 23:42:14,158][105692] Updated weights for policy 0, policy_version 1143647 (0.0009) [2023-12-26 23:42:14,209][105620] Updated weights for policy 1, policy_version 1145016 (0.0008) [2023-12-26 23:42:14,269][105620] Updated weights for policy 1, policy_version 1145026 (0.0009) [2023-12-26 23:42:14,322][105620] Updated weights for policy 1, policy_version 1145036 (0.0010) [2023-12-26 23:42:14,835][105692] Updated weights for policy 0, policy_version 1143657 (0.0006) [2023-12-26 23:42:14,900][105692] Updated weights for policy 0, policy_version 1143667 (0.0009) [2023-12-26 23:42:14,954][105692] Updated weights for policy 0, policy_version 1143677 (0.0009) [2023-12-26 23:42:15,017][105692] Updated weights for policy 0, policy_version 1143687 (0.0009) [2023-12-26 23:42:15,060][105620] Updated weights for policy 1, policy_version 1145046 (0.0009) [2023-12-26 23:42:15,112][105620] Updated weights for policy 1, policy_version 1145056 (0.0009) [2023-12-26 23:42:15,168][105620] Updated weights for policy 1, policy_version 1145066 (0.0009) [2023-12-26 23:42:15,687][105692] Updated weights for policy 0, policy_version 1143697 (0.0009) [2023-12-26 23:42:15,746][105692] Updated weights for policy 0, policy_version 1143707 (0.0008) [2023-12-26 23:42:15,793][105692] Updated weights for policy 0, policy_version 1143717 (0.0008) [2023-12-26 23:42:15,997][105620] Updated weights for policy 1, policy_version 1145076 (0.0010) [2023-12-26 23:42:16,054][105620] Updated weights for policy 1, policy_version 1145086 (0.0009) [2023-12-26 23:42:16,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 586014720. Throughput: 0: 9876.0, 1: 9774.1. Samples: 585988272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:42:16,062][104569] Avg episode reward: [(0, '9264.329'), (1, '9170.825')] [2023-12-26 23:42:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001143720_292839424.pth... [2023-12-26 23:42:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001142600_292552704.pth [2023-12-26 23:42:16,106][105620] Updated weights for policy 1, policy_version 1145097 (0.0009) [2023-12-26 23:42:16,139][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001145104_293183488.pth... [2023-12-26 23:42:16,143][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001143920_292880384.pth [2023-12-26 23:42:16,479][105692] Updated weights for policy 0, policy_version 1143727 (0.0008) [2023-12-26 23:42:16,529][105692] Updated weights for policy 0, policy_version 1143737 (0.0009) [2023-12-26 23:42:16,580][105692] Updated weights for policy 0, policy_version 1143747 (0.0009) [2023-12-26 23:42:16,866][105620] Updated weights for policy 1, policy_version 1145107 (0.0008) [2023-12-26 23:42:16,920][105620] Updated weights for policy 1, policy_version 1145117 (0.0009) [2023-12-26 23:42:16,984][105620] Updated weights for policy 1, policy_version 1145127 (0.0008) [2023-12-26 23:42:17,303][105692] Updated weights for policy 0, policy_version 1143757 (0.0009) [2023-12-26 23:42:17,356][105692] Updated weights for policy 0, policy_version 1143767 (0.0006) [2023-12-26 23:42:17,421][105692] Updated weights for policy 0, policy_version 1143777 (0.0005) [2023-12-26 23:42:17,797][105620] Updated weights for policy 1, policy_version 1145137 (0.0009) [2023-12-26 23:42:17,857][105620] Updated weights for policy 1, policy_version 1145147 (0.0008) [2023-12-26 23:42:17,921][105620] Updated weights for policy 1, policy_version 1145157 (0.0010) [2023-12-26 23:42:17,987][105620] Updated weights for policy 1, policy_version 1145167 (0.0010) [2023-12-26 23:42:18,053][105692] Updated weights for policy 0, policy_version 1143787 (0.0008) [2023-12-26 23:42:18,108][105692] Updated weights for policy 0, policy_version 1143797 (0.0009) [2023-12-26 23:42:18,156][105692] Updated weights for policy 0, policy_version 1143807 (0.0009) [2023-12-26 23:42:18,759][105620] Updated weights for policy 1, policy_version 1145177 (0.0009) [2023-12-26 23:42:18,820][105620] Updated weights for policy 1, policy_version 1145187 (0.0009) [2023-12-26 23:42:18,878][105692] Updated weights for policy 0, policy_version 1143817 (0.0007) [2023-12-26 23:42:18,884][105620] Updated weights for policy 1, policy_version 1145197 (0.0009) [2023-12-26 23:42:18,931][105692] Updated weights for policy 0, policy_version 1143827 (0.0008) [2023-12-26 23:42:18,987][105692] Updated weights for policy 0, policy_version 1143837 (0.0009) [2023-12-26 23:42:19,043][105692] Updated weights for policy 0, policy_version 1143847 (0.0009) [2023-12-26 23:42:19,665][105620] Updated weights for policy 1, policy_version 1145207 (0.0007) [2023-12-26 23:42:19,731][105620] Updated weights for policy 1, policy_version 1145217 (0.0009) [2023-12-26 23:42:19,797][105620] Updated weights for policy 1, policy_version 1145227 (0.0009) [2023-12-26 23:42:19,803][105692] Updated weights for policy 0, policy_version 1143857 (0.0007) [2023-12-26 23:42:19,868][105692] Updated weights for policy 0, policy_version 1143867 (0.0007) [2023-12-26 23:42:19,940][105692] Updated weights for policy 0, policy_version 1143877 (0.0006) [2023-12-26 23:42:20,536][105620] Updated weights for policy 1, policy_version 1145237 (0.0008) [2023-12-26 23:42:20,594][105620] Updated weights for policy 1, policy_version 1145247 (0.0007) [2023-12-26 23:42:20,659][105620] Updated weights for policy 1, policy_version 1145257 (0.0009) [2023-12-26 23:42:20,673][105692] Updated weights for policy 0, policy_version 1143887 (0.0005) [2023-12-26 23:42:20,742][105692] Updated weights for policy 0, policy_version 1143897 (0.0006) [2023-12-26 23:42:20,811][105692] Updated weights for policy 0, policy_version 1143907 (0.0007) [2023-12-26 23:42:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 586113024. Throughput: 0: 9762.9, 1: 9747.2. Samples: 586102028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:42:21,063][104569] Avg episode reward: [(0, '9173.519'), (1, '9082.276')] [2023-12-26 23:42:21,458][105620] Updated weights for policy 1, policy_version 1145267 (0.0008) [2023-12-26 23:42:21,501][105692] Updated weights for policy 0, policy_version 1143917 (0.0007) [2023-12-26 23:42:21,511][105620] Updated weights for policy 1, policy_version 1145277 (0.0008) [2023-12-26 23:42:21,554][105692] Updated weights for policy 0, policy_version 1143927 (0.0008) [2023-12-26 23:42:21,567][105620] Updated weights for policy 1, policy_version 1145287 (0.0007) [2023-12-26 23:42:21,615][105692] Updated weights for policy 0, policy_version 1143937 (0.0007) [2023-12-26 23:42:22,333][105692] Updated weights for policy 0, policy_version 1143947 (0.0008) [2023-12-26 23:42:22,376][105620] Updated weights for policy 1, policy_version 1145297 (0.0007) [2023-12-26 23:42:22,406][105692] Updated weights for policy 0, policy_version 1143957 (0.0010) [2023-12-26 23:42:22,441][105620] Updated weights for policy 1, policy_version 1145307 (0.0008) [2023-12-26 23:42:22,461][105692] Updated weights for policy 0, policy_version 1143967 (0.0006) [2023-12-26 23:42:22,507][105620] Updated weights for policy 1, policy_version 1145317 (0.0009) [2023-12-26 23:42:22,570][105620] Updated weights for policy 1, policy_version 1145327 (0.0009) [2023-12-26 23:42:23,192][105692] Updated weights for policy 0, policy_version 1143977 (0.0006) [2023-12-26 23:42:23,242][105692] Updated weights for policy 0, policy_version 1143987 (0.0007) [2023-12-26 23:42:23,288][105692] Updated weights for policy 0, policy_version 1143997 (0.0009) [2023-12-26 23:42:23,335][105620] Updated weights for policy 1, policy_version 1145337 (0.0007) [2023-12-26 23:42:23,344][105692] Updated weights for policy 0, policy_version 1144007 (0.0008) [2023-12-26 23:42:23,383][105620] Updated weights for policy 1, policy_version 1145347 (0.0009) [2023-12-26 23:42:23,430][105620] Updated weights for policy 1, policy_version 1145357 (0.0009) [2023-12-26 23:42:24,011][105692] Updated weights for policy 0, policy_version 1144017 (0.0009) [2023-12-26 23:42:24,061][105692] Updated weights for policy 0, policy_version 1144027 (0.0009) [2023-12-26 23:42:24,115][105692] Updated weights for policy 0, policy_version 1144037 (0.0009) [2023-12-26 23:42:24,230][105620] Updated weights for policy 1, policy_version 1145367 (0.0009) [2023-12-26 23:42:24,281][105620] Updated weights for policy 1, policy_version 1145377 (0.0009) [2023-12-26 23:42:24,327][105620] Updated weights for policy 1, policy_version 1145387 (0.0008) [2023-12-26 23:42:24,766][105692] Updated weights for policy 0, policy_version 1144047 (0.0008) [2023-12-26 23:42:24,820][105692] Updated weights for policy 0, policy_version 1144057 (0.0007) [2023-12-26 23:42:24,866][105692] Updated weights for policy 0, policy_version 1144067 (0.0005) [2023-12-26 23:42:25,210][105620] Updated weights for policy 1, policy_version 1145397 (0.0008) [2023-12-26 23:42:25,270][105620] Updated weights for policy 1, policy_version 1145407 (0.0008) [2023-12-26 23:42:25,335][105620] Updated weights for policy 1, policy_version 1145417 (0.0009) [2023-12-26 23:42:25,492][105692] Updated weights for policy 0, policy_version 1144077 (0.0005) [2023-12-26 23:42:25,549][105692] Updated weights for policy 0, policy_version 1144087 (0.0005) [2023-12-26 23:42:25,602][105692] Updated weights for policy 0, policy_version 1144097 (0.0005) [2023-12-26 23:42:26,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.2, 300 sec: 19605.2). Total num frames: 586203136. Throughput: 0: 9817.5, 1: 9733.1. Samples: 586215720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:42:26,063][104569] Avg episode reward: [(0, '9173.813'), (1, '9079.800')] [2023-12-26 23:42:26,142][105620] Updated weights for policy 1, policy_version 1145427 (0.0009) [2023-12-26 23:42:26,198][105620] Updated weights for policy 1, policy_version 1145437 (0.0009) [2023-12-26 23:42:26,246][105692] Updated weights for policy 0, policy_version 1144107 (0.0006) [2023-12-26 23:42:26,253][105620] Updated weights for policy 1, policy_version 1145447 (0.0007) [2023-12-26 23:42:26,309][105692] Updated weights for policy 0, policy_version 1144117 (0.0008) [2023-12-26 23:42:26,365][105692] Updated weights for policy 0, policy_version 1144127 (0.0009) [2023-12-26 23:42:27,017][105620] Updated weights for policy 1, policy_version 1145457 (0.0007) [2023-12-26 23:42:27,077][105620] Updated weights for policy 1, policy_version 1145467 (0.0010) [2023-12-26 23:42:27,081][105692] Updated weights for policy 0, policy_version 1144137 (0.0009) [2023-12-26 23:42:27,128][105620] Updated weights for policy 1, policy_version 1145477 (0.0007) [2023-12-26 23:42:27,138][105692] Updated weights for policy 0, policy_version 1144147 (0.0008) [2023-12-26 23:42:27,177][105620] Updated weights for policy 1, policy_version 1145487 (0.0007) [2023-12-26 23:42:27,183][105692] Updated weights for policy 0, policy_version 1144157 (0.0006) [2023-12-26 23:42:27,230][105692] Updated weights for policy 0, policy_version 1144167 (0.0009) [2023-12-26 23:42:27,946][105692] Updated weights for policy 0, policy_version 1144177 (0.0007) [2023-12-26 23:42:27,996][105692] Updated weights for policy 0, policy_version 1144187 (0.0008) [2023-12-26 23:42:28,002][105620] Updated weights for policy 1, policy_version 1145497 (0.0009) [2023-12-26 23:42:28,052][105692] Updated weights for policy 0, policy_version 1144197 (0.0006) [2023-12-26 23:42:28,062][105620] Updated weights for policy 1, policy_version 1145507 (0.0009) [2023-12-26 23:42:28,114][105620] Updated weights for policy 1, policy_version 1145517 (0.0009) [2023-12-26 23:42:28,725][105692] Updated weights for policy 0, policy_version 1144207 (0.0005) [2023-12-26 23:42:28,786][105692] Updated weights for policy 0, policy_version 1144217 (0.0006) [2023-12-26 23:42:28,845][105692] Updated weights for policy 0, policy_version 1144227 (0.0008) [2023-12-26 23:42:28,921][105620] Updated weights for policy 1, policy_version 1145527 (0.0009) [2023-12-26 23:42:28,971][105620] Updated weights for policy 1, policy_version 1145538 (0.0009) [2023-12-26 23:42:29,018][105620] Updated weights for policy 1, policy_version 1145548 (0.0009) [2023-12-26 23:42:29,509][105692] Updated weights for policy 0, policy_version 1144237 (0.0008) [2023-12-26 23:42:29,570][105692] Updated weights for policy 0, policy_version 1144247 (0.0008) [2023-12-26 23:42:29,625][105692] Updated weights for policy 0, policy_version 1144257 (0.0007) [2023-12-26 23:42:29,890][105620] Updated weights for policy 1, policy_version 1145558 (0.0008) [2023-12-26 23:42:29,960][105620] Updated weights for policy 1, policy_version 1145568 (0.0006) [2023-12-26 23:42:30,027][105620] Updated weights for policy 1, policy_version 1145578 (0.0005) [2023-12-26 23:42:30,242][105692] Updated weights for policy 0, policy_version 1144267 (0.0007) [2023-12-26 23:42:30,298][105692] Updated weights for policy 0, policy_version 1144277 (0.0011) [2023-12-26 23:42:30,356][105692] Updated weights for policy 0, policy_version 1144287 (0.0010) [2023-12-26 23:42:30,687][105620] Updated weights for policy 1, policy_version 1145588 (0.0006) [2023-12-26 23:42:30,743][105620] Updated weights for policy 1, policy_version 1145598 (0.0008) [2023-12-26 23:42:30,792][105620] Updated weights for policy 1, policy_version 1145608 (0.0009) [2023-12-26 23:42:31,019][105692] Updated weights for policy 0, policy_version 1144297 (0.0010) [2023-12-26 23:42:31,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 586301440. Throughput: 0: 9848.7, 1: 9681.5. Samples: 586272412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:42:31,063][104569] Avg episode reward: [(0, '9264.823'), (1, '9165.780')] [2023-12-26 23:42:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001145616_293314560.pth... [2023-12-26 23:42:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001144496_293027840.pth [2023-12-26 23:42:31,087][105692] Updated weights for policy 0, policy_version 1144307 (0.0009) [2023-12-26 23:42:31,151][105692] Updated weights for policy 0, policy_version 1144317 (0.0011) [2023-12-26 23:42:31,211][105692] Updated weights for policy 0, policy_version 1144327 (0.0009) [2023-12-26 23:42:31,213][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001144328_292995072.pth... [2023-12-26 23:42:31,216][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001143176_292700160.pth [2023-12-26 23:42:31,507][105620] Updated weights for policy 1, policy_version 1145618 (0.0010) [2023-12-26 23:42:31,561][105620] Updated weights for policy 1, policy_version 1145628 (0.0009) [2023-12-26 23:42:31,625][105620] Updated weights for policy 1, policy_version 1145638 (0.0008) [2023-12-26 23:42:31,677][105620] Updated weights for policy 1, policy_version 1145648 (0.0010) [2023-12-26 23:42:31,969][105692] Updated weights for policy 0, policy_version 1144337 (0.0008) [2023-12-26 23:42:32,026][105692] Updated weights for policy 0, policy_version 1144347 (0.0008) [2023-12-26 23:42:32,096][105692] Updated weights for policy 0, policy_version 1144357 (0.0008) [2023-12-26 23:42:32,394][105620] Updated weights for policy 1, policy_version 1145658 (0.0011) [2023-12-26 23:42:32,442][105620] Updated weights for policy 1, policy_version 1145668 (0.0010) [2023-12-26 23:42:32,494][105620] Updated weights for policy 1, policy_version 1145678 (0.0010) [2023-12-26 23:42:32,852][105692] Updated weights for policy 0, policy_version 1144367 (0.0008) [2023-12-26 23:42:32,913][105692] Updated weights for policy 0, policy_version 1144377 (0.0007) [2023-12-26 23:42:32,978][105692] Updated weights for policy 0, policy_version 1144387 (0.0006) [2023-12-26 23:42:33,258][105620] Updated weights for policy 1, policy_version 1145688 (0.0010) [2023-12-26 23:42:33,312][105620] Updated weights for policy 1, policy_version 1145698 (0.0010) [2023-12-26 23:42:33,369][105620] Updated weights for policy 1, policy_version 1145708 (0.0010) [2023-12-26 23:42:33,554][105692] Updated weights for policy 0, policy_version 1144397 (0.0005) [2023-12-26 23:42:33,610][105692] Updated weights for policy 0, policy_version 1144407 (0.0005) [2023-12-26 23:42:33,656][105692] Updated weights for policy 0, policy_version 1144417 (0.0005) [2023-12-26 23:42:33,956][105620] Updated weights for policy 1, policy_version 1145718 (0.0008) [2023-12-26 23:42:34,001][105620] Updated weights for policy 1, policy_version 1145728 (0.0010) [2023-12-26 23:42:34,055][105620] Updated weights for policy 1, policy_version 1145738 (0.0010) [2023-12-26 23:42:34,238][105692] Updated weights for policy 0, policy_version 1144427 (0.0007) [2023-12-26 23:42:34,290][105692] Updated weights for policy 0, policy_version 1144437 (0.0010) [2023-12-26 23:42:34,353][105692] Updated weights for policy 0, policy_version 1144447 (0.0010) [2023-12-26 23:42:34,719][105620] Updated weights for policy 1, policy_version 1145748 (0.0008) [2023-12-26 23:42:34,791][105620] Updated weights for policy 1, policy_version 1145758 (0.0005) [2023-12-26 23:42:34,862][105620] Updated weights for policy 1, policy_version 1145768 (0.0009) [2023-12-26 23:42:35,106][105692] Updated weights for policy 0, policy_version 1144457 (0.0009) [2023-12-26 23:42:35,165][105692] Updated weights for policy 0, policy_version 1144467 (0.0008) [2023-12-26 23:42:35,221][105692] Updated weights for policy 0, policy_version 1144477 (0.0007) [2023-12-26 23:42:35,276][105692] Updated weights for policy 0, policy_version 1144487 (0.0005) [2023-12-26 23:42:35,567][105620] Updated weights for policy 1, policy_version 1145778 (0.0009) [2023-12-26 23:42:35,637][105620] Updated weights for policy 1, policy_version 1145788 (0.0009) [2023-12-26 23:42:35,698][105620] Updated weights for policy 1, policy_version 1145798 (0.0009) [2023-12-26 23:42:35,770][105620] Updated weights for policy 1, policy_version 1145808 (0.0009) [2023-12-26 23:42:35,851][105692] Updated weights for policy 0, policy_version 1144497 (0.0005) [2023-12-26 23:42:35,899][105692] Updated weights for policy 0, policy_version 1144507 (0.0005) [2023-12-26 23:42:35,952][105692] Updated weights for policy 0, policy_version 1144517 (0.0005) [2023-12-26 23:42:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 586407936. Throughput: 0: 9793.9, 1: 9717.7. Samples: 586392540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:42:36,063][104569] Avg episode reward: [(0, '9263.273'), (1, '9168.293')] [2023-12-26 23:42:36,518][105620] Updated weights for policy 1, policy_version 1145818 (0.0011) [2023-12-26 23:42:36,567][105692] Updated weights for policy 0, policy_version 1144527 (0.0005) [2023-12-26 23:42:36,571][105620] Updated weights for policy 1, policy_version 1145828 (0.0010) [2023-12-26 23:42:36,625][105692] Updated weights for policy 0, policy_version 1144537 (0.0006) [2023-12-26 23:42:36,638][105620] Updated weights for policy 1, policy_version 1145838 (0.0011) [2023-12-26 23:42:36,688][105692] Updated weights for policy 0, policy_version 1144547 (0.0009) [2023-12-26 23:42:37,317][105620] Updated weights for policy 1, policy_version 1145848 (0.0010) [2023-12-26 23:42:37,375][105620] Updated weights for policy 1, policy_version 1145858 (0.0010) [2023-12-26 23:42:37,434][105620] Updated weights for policy 1, policy_version 1145868 (0.0010) [2023-12-26 23:42:37,437][105692] Updated weights for policy 0, policy_version 1144557 (0.0010) [2023-12-26 23:42:37,490][105692] Updated weights for policy 0, policy_version 1144567 (0.0008) [2023-12-26 23:42:37,544][105692] Updated weights for policy 0, policy_version 1144577 (0.0005) [2023-12-26 23:42:38,160][105620] Updated weights for policy 1, policy_version 1145878 (0.0011) [2023-12-26 23:42:38,221][105620] Updated weights for policy 1, policy_version 1145888 (0.0011) [2023-12-26 23:42:38,271][105692] Updated weights for policy 0, policy_version 1144587 (0.0008) [2023-12-26 23:42:38,286][105620] Updated weights for policy 1, policy_version 1145898 (0.0006) [2023-12-26 23:42:38,324][105692] Updated weights for policy 0, policy_version 1144597 (0.0011) [2023-12-26 23:42:38,388][105692] Updated weights for policy 0, policy_version 1144607 (0.0010) [2023-12-26 23:42:38,916][105620] Updated weights for policy 1, policy_version 1145908 (0.0009) [2023-12-26 23:42:38,967][105620] Updated weights for policy 1, policy_version 1145918 (0.0010) [2023-12-26 23:42:39,025][105620] Updated weights for policy 1, policy_version 1145928 (0.0010) [2023-12-26 23:42:39,134][105692] Updated weights for policy 0, policy_version 1144617 (0.0011) [2023-12-26 23:42:39,185][105692] Updated weights for policy 0, policy_version 1144627 (0.0010) [2023-12-26 23:42:39,245][105692] Updated weights for policy 0, policy_version 1144637 (0.0011) [2023-12-26 23:42:39,315][105692] Updated weights for policy 0, policy_version 1144647 (0.0011) [2023-12-26 23:42:39,870][105620] Updated weights for policy 1, policy_version 1145938 (0.0010) [2023-12-26 23:42:39,927][105620] Updated weights for policy 1, policy_version 1145948 (0.0008) [2023-12-26 23:42:39,981][105620] Updated weights for policy 1, policy_version 1145958 (0.0009) [2023-12-26 23:42:40,006][105692] Updated weights for policy 0, policy_version 1144657 (0.0007) [2023-12-26 23:42:40,037][105620] Updated weights for policy 1, policy_version 1145968 (0.0006) [2023-12-26 23:42:40,058][105692] Updated weights for policy 0, policy_version 1144667 (0.0008) [2023-12-26 23:42:40,110][105692] Updated weights for policy 0, policy_version 1144677 (0.0009) [2023-12-26 23:42:40,782][105620] Updated weights for policy 1, policy_version 1145978 (0.0007) [2023-12-26 23:42:40,803][105692] Updated weights for policy 0, policy_version 1144687 (0.0009) [2023-12-26 23:42:40,842][105620] Updated weights for policy 1, policy_version 1145988 (0.0007) [2023-12-26 23:42:40,860][105692] Updated weights for policy 0, policy_version 1144697 (0.0006) [2023-12-26 23:42:40,897][105620] Updated weights for policy 1, policy_version 1145998 (0.0009) [2023-12-26 23:42:40,923][105692] Updated weights for policy 0, policy_version 1144707 (0.0005) [2023-12-26 23:42:41,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19660.9, 300 sec: 19633.0). Total num frames: 586506240. Throughput: 0: 9856.2, 1: 9583.3. Samples: 586509280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:42:41,062][104569] Avg episode reward: [(0, '9355.354'), (1, '8985.646')] [2023-12-26 23:42:41,622][105692] Updated weights for policy 0, policy_version 1144717 (0.0007) [2023-12-26 23:42:41,683][105692] Updated weights for policy 0, policy_version 1144727 (0.0008) [2023-12-26 23:42:41,734][105620] Updated weights for policy 1, policy_version 1146008 (0.0009) [2023-12-26 23:42:41,749][105692] Updated weights for policy 0, policy_version 1144737 (0.0007) [2023-12-26 23:42:41,795][105620] Updated weights for policy 1, policy_version 1146018 (0.0008) [2023-12-26 23:42:41,857][105620] Updated weights for policy 1, policy_version 1146028 (0.0009) [2023-12-26 23:42:42,523][105692] Updated weights for policy 0, policy_version 1144747 (0.0006) [2023-12-26 23:42:42,590][105692] Updated weights for policy 0, policy_version 1144757 (0.0009) [2023-12-26 23:42:42,618][105620] Updated weights for policy 1, policy_version 1146038 (0.0009) [2023-12-26 23:42:42,641][105692] Updated weights for policy 0, policy_version 1144767 (0.0008) [2023-12-26 23:42:42,676][105620] Updated weights for policy 1, policy_version 1146048 (0.0009) [2023-12-26 23:42:42,734][105620] Updated weights for policy 1, policy_version 1146058 (0.0008) [2023-12-26 23:42:43,352][105692] Updated weights for policy 0, policy_version 1144777 (0.0006) [2023-12-26 23:42:43,398][105692] Updated weights for policy 0, policy_version 1144787 (0.0009) [2023-12-26 23:42:43,446][105692] Updated weights for policy 0, policy_version 1144797 (0.0008) [2023-12-26 23:42:43,498][105692] Updated weights for policy 0, policy_version 1144807 (0.0008) [2023-12-26 23:42:43,512][105620] Updated weights for policy 1, policy_version 1146068 (0.0010) [2023-12-26 23:42:43,568][105620] Updated weights for policy 1, policy_version 1146078 (0.0010) [2023-12-26 23:42:43,622][105620] Updated weights for policy 1, policy_version 1146088 (0.0010) [2023-12-26 23:42:44,179][105692] Updated weights for policy 0, policy_version 1144817 (0.0008) [2023-12-26 23:42:44,230][105692] Updated weights for policy 0, policy_version 1144827 (0.0008) [2023-12-26 23:42:44,293][105692] Updated weights for policy 0, policy_version 1144837 (0.0008) [2023-12-26 23:42:44,385][105620] Updated weights for policy 1, policy_version 1146098 (0.0010) [2023-12-26 23:42:44,434][105620] Updated weights for policy 1, policy_version 1146108 (0.0010) [2023-12-26 23:42:44,480][105620] Updated weights for policy 1, policy_version 1146118 (0.0008) [2023-12-26 23:42:44,526][105620] Updated weights for policy 1, policy_version 1146128 (0.0007) [2023-12-26 23:42:45,086][105692] Updated weights for policy 0, policy_version 1144847 (0.0008) [2023-12-26 23:42:45,147][105692] Updated weights for policy 0, policy_version 1144857 (0.0008) [2023-12-26 23:42:45,209][105692] Updated weights for policy 0, policy_version 1144867 (0.0008) [2023-12-26 23:42:45,220][105620] Updated weights for policy 1, policy_version 1146138 (0.0008) [2023-12-26 23:42:45,284][105620] Updated weights for policy 1, policy_version 1146148 (0.0011) [2023-12-26 23:42:45,351][105620] Updated weights for policy 1, policy_version 1146158 (0.0011) [2023-12-26 23:42:45,962][105692] Updated weights for policy 0, policy_version 1144877 (0.0008) [2023-12-26 23:42:46,009][105692] Updated weights for policy 0, policy_version 1144887 (0.0009) [2023-12-26 23:42:46,055][105692] Updated weights for policy 0, policy_version 1144897 (0.0007) [2023-12-26 23:42:46,061][105620] Updated weights for policy 1, policy_version 1146168 (0.0007) [2023-12-26 23:42:46,062][104569] Fps is (10 sec: 18022.6, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 586588160. Throughput: 0: 9831.0, 1: 9492.8. Samples: 586565072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:42:46,062][104569] Avg episode reward: [(0, '9356.581'), (1, '8894.338')] [2023-12-26 23:42:46,084][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001144904_293142528.pth... [2023-12-26 23:42:46,087][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001143720_292839424.pth [2023-12-26 23:42:46,105][105620] Updated weights for policy 1, policy_version 1146178 (0.0007) [2023-12-26 23:42:46,152][105620] Updated weights for policy 1, policy_version 1146188 (0.0009) [2023-12-26 23:42:46,172][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001146192_293462016.pth... [2023-12-26 23:42:46,176][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001145104_293183488.pth [2023-12-26 23:42:46,813][105692] Updated weights for policy 0, policy_version 1144907 (0.0005) [2023-12-26 23:42:46,861][105692] Updated weights for policy 0, policy_version 1144917 (0.0005) [2023-12-26 23:42:46,911][105692] Updated weights for policy 0, policy_version 1144927 (0.0005) [2023-12-26 23:42:46,949][105620] Updated weights for policy 1, policy_version 1146198 (0.0008) [2023-12-26 23:42:47,007][105620] Updated weights for policy 1, policy_version 1146208 (0.0009) [2023-12-26 23:42:47,071][105620] Updated weights for policy 1, policy_version 1146218 (0.0009) [2023-12-26 23:42:47,551][105692] Updated weights for policy 0, policy_version 1144937 (0.0008) [2023-12-26 23:42:47,607][105692] Updated weights for policy 0, policy_version 1144947 (0.0009) [2023-12-26 23:42:47,664][105692] Updated weights for policy 0, policy_version 1144957 (0.0009) [2023-12-26 23:42:47,723][105692] Updated weights for policy 0, policy_version 1144967 (0.0009) [2023-12-26 23:42:47,833][105620] Updated weights for policy 1, policy_version 1146228 (0.0009) [2023-12-26 23:42:47,878][105620] Updated weights for policy 1, policy_version 1146238 (0.0008) [2023-12-26 23:42:47,924][105620] Updated weights for policy 1, policy_version 1146248 (0.0008) [2023-12-26 23:42:48,509][105692] Updated weights for policy 0, policy_version 1144977 (0.0006) [2023-12-26 23:42:48,570][105692] Updated weights for policy 0, policy_version 1144987 (0.0007) [2023-12-26 23:42:48,635][105692] Updated weights for policy 0, policy_version 1144997 (0.0008) [2023-12-26 23:42:48,672][105620] Updated weights for policy 1, policy_version 1146258 (0.0008) [2023-12-26 23:42:48,740][105620] Updated weights for policy 1, policy_version 1146268 (0.0008) [2023-12-26 23:42:48,799][105620] Updated weights for policy 1, policy_version 1146278 (0.0009) [2023-12-26 23:42:48,865][105620] Updated weights for policy 1, policy_version 1146288 (0.0009) [2023-12-26 23:42:49,360][105692] Updated weights for policy 0, policy_version 1145007 (0.0010) [2023-12-26 23:42:49,420][105585] KL-divergence is very high: 243.5519 [2023-12-26 23:42:49,425][105692] Updated weights for policy 0, policy_version 1145017 (0.0009) [2023-12-26 23:42:49,445][105585] KL-divergence is very high: 340.3897 [2023-12-26 23:42:49,470][105585] KL-divergence is very high: 393.6401 [2023-12-26 23:42:49,487][105692] Updated weights for policy 0, policy_version 1145027 (0.0009) [2023-12-26 23:42:49,494][105585] KL-divergence is very high: 376.2762 [2023-12-26 23:42:49,643][105620] Updated weights for policy 1, policy_version 1146298 (0.0009) [2023-12-26 23:42:49,701][105620] Updated weights for policy 1, policy_version 1146308 (0.0009) [2023-12-26 23:42:49,756][105620] Updated weights for policy 1, policy_version 1146318 (0.0009) [2023-12-26 23:42:50,200][105692] Updated weights for policy 0, policy_version 1145037 (0.0007) [2023-12-26 23:42:50,266][105692] Updated weights for policy 0, policy_version 1145047 (0.0009) [2023-12-26 23:42:50,321][105692] Updated weights for policy 0, policy_version 1145057 (0.0009) [2023-12-26 23:42:50,561][105620] Updated weights for policy 1, policy_version 1146328 (0.0007) [2023-12-26 23:42:50,623][105620] Updated weights for policy 1, policy_version 1146338 (0.0009) [2023-12-26 23:42:50,679][105620] Updated weights for policy 1, policy_version 1146348 (0.0008) [2023-12-26 23:42:51,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 586686464. Throughput: 0: 9901.8, 1: 9333.6. Samples: 586678780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:42:51,062][104569] Avg episode reward: [(0, '9039.727'), (1, '9081.071')] [2023-12-26 23:42:51,084][105692] Updated weights for policy 0, policy_version 1145067 (0.0008) [2023-12-26 23:42:51,150][105692] Updated weights for policy 0, policy_version 1145077 (0.0009) [2023-12-26 23:42:51,211][105692] Updated weights for policy 0, policy_version 1145087 (0.0006) [2023-12-26 23:42:51,451][105620] Updated weights for policy 1, policy_version 1146358 (0.0009) [2023-12-26 23:42:51,516][105620] Updated weights for policy 1, policy_version 1146368 (0.0009) [2023-12-26 23:42:51,577][105620] Updated weights for policy 1, policy_version 1146378 (0.0009) [2023-12-26 23:42:51,970][105692] Updated weights for policy 0, policy_version 1145097 (0.0008) [2023-12-26 23:42:52,023][105692] Updated weights for policy 0, policy_version 1145107 (0.0011) [2023-12-26 23:42:52,078][105692] Updated weights for policy 0, policy_version 1145117 (0.0011) [2023-12-26 23:42:52,132][105692] Updated weights for policy 0, policy_version 1145127 (0.0008) [2023-12-26 23:42:52,388][105620] Updated weights for policy 1, policy_version 1146388 (0.0010) [2023-12-26 23:42:52,444][105620] Updated weights for policy 1, policy_version 1146398 (0.0010) [2023-12-26 23:42:52,500][105620] Updated weights for policy 1, policy_version 1146408 (0.0008) [2023-12-26 23:42:52,801][105692] Updated weights for policy 0, policy_version 1145137 (0.0008) [2023-12-26 23:42:52,858][105692] Updated weights for policy 0, policy_version 1145147 (0.0009) [2023-12-26 23:42:52,926][105692] Updated weights for policy 0, policy_version 1145157 (0.0006) [2023-12-26 23:42:53,324][105620] Updated weights for policy 1, policy_version 1146418 (0.0009) [2023-12-26 23:42:53,380][105620] Updated weights for policy 1, policy_version 1146428 (0.0007) [2023-12-26 23:42:53,436][105620] Updated weights for policy 1, policy_version 1146438 (0.0006) [2023-12-26 23:42:53,488][105620] Updated weights for policy 1, policy_version 1146448 (0.0010) [2023-12-26 23:42:53,489][105692] Updated weights for policy 0, policy_version 1145167 (0.0006) [2023-12-26 23:42:53,550][105692] Updated weights for policy 0, policy_version 1145177 (0.0006) [2023-12-26 23:42:53,607][105692] Updated weights for policy 0, policy_version 1145187 (0.0005) [2023-12-26 23:42:54,116][105620] Updated weights for policy 1, policy_version 1146458 (0.0006) [2023-12-26 23:42:54,169][105620] Updated weights for policy 1, policy_version 1146468 (0.0006) [2023-12-26 23:42:54,230][105620] Updated weights for policy 1, policy_version 1146478 (0.0010) [2023-12-26 23:42:54,248][105692] Updated weights for policy 0, policy_version 1145197 (0.0006) [2023-12-26 23:42:54,304][105692] Updated weights for policy 0, policy_version 1145207 (0.0008) [2023-12-26 23:42:54,368][105692] Updated weights for policy 0, policy_version 1145217 (0.0007) [2023-12-26 23:42:54,792][105620] Updated weights for policy 1, policy_version 1146488 (0.0006) [2023-12-26 23:42:54,846][105620] Updated weights for policy 1, policy_version 1146498 (0.0005) [2023-12-26 23:42:54,893][105620] Updated weights for policy 1, policy_version 1146508 (0.0005) [2023-12-26 23:42:54,916][105692] Updated weights for policy 0, policy_version 1145227 (0.0006) [2023-12-26 23:42:54,976][105692] Updated weights for policy 0, policy_version 1145237 (0.0006) [2023-12-26 23:42:55,039][105692] Updated weights for policy 0, policy_version 1145247 (0.0005) [2023-12-26 23:42:55,591][105620] Updated weights for policy 1, policy_version 1146518 (0.0005) [2023-12-26 23:42:55,642][105620] Updated weights for policy 1, policy_version 1146528 (0.0005) [2023-12-26 23:42:55,689][105620] Updated weights for policy 1, policy_version 1146538 (0.0005) [2023-12-26 23:42:55,692][105692] Updated weights for policy 0, policy_version 1145257 (0.0007) [2023-12-26 23:42:55,747][105692] Updated weights for policy 0, policy_version 1145268 (0.0010) [2023-12-26 23:42:55,801][105692] Updated weights for policy 0, policy_version 1145279 (0.0010) [2023-12-26 23:42:56,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 586792960. Throughput: 0: 9977.4, 1: 9298.5. Samples: 586800444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:42:56,063][104569] Avg episode reward: [(0, '8758.223'), (1, '8994.006')] [2023-12-26 23:42:56,362][105620] Updated weights for policy 1, policy_version 1146548 (0.0007) [2023-12-26 23:42:56,418][105620] Updated weights for policy 1, policy_version 1146558 (0.0010) [2023-12-26 23:42:56,475][105620] Updated weights for policy 1, policy_version 1146568 (0.0010) [2023-12-26 23:42:56,602][105692] Updated weights for policy 0, policy_version 1145290 (0.0008) [2023-12-26 23:42:56,658][105692] Updated weights for policy 0, policy_version 1145300 (0.0005) [2023-12-26 23:42:56,720][105692] Updated weights for policy 0, policy_version 1145310 (0.0005) [2023-12-26 23:42:56,785][105692] Updated weights for policy 0, policy_version 1145320 (0.0005) [2023-12-26 23:42:57,253][105620] Updated weights for policy 1, policy_version 1146578 (0.0009) [2023-12-26 23:42:57,321][105692] Updated weights for policy 0, policy_version 1145330 (0.0009) [2023-12-26 23:42:57,321][105620] Updated weights for policy 1, policy_version 1146588 (0.0006) [2023-12-26 23:42:57,373][105692] Updated weights for policy 0, policy_version 1145340 (0.0007) [2023-12-26 23:42:57,375][105620] Updated weights for policy 1, policy_version 1146598 (0.0006) [2023-12-26 23:42:57,425][105620] Updated weights for policy 1, policy_version 1146608 (0.0008) [2023-12-26 23:42:57,430][105692] Updated weights for policy 0, policy_version 1145350 (0.0005) [2023-12-26 23:42:58,045][105692] Updated weights for policy 0, policy_version 1145361 (0.0008) [2023-12-26 23:42:58,099][105692] Updated weights for policy 0, policy_version 1145372 (0.0010) [2023-12-26 23:42:58,147][105692] Updated weights for policy 0, policy_version 1145382 (0.0009) [2023-12-26 23:42:58,274][105620] Updated weights for policy 1, policy_version 1146618 (0.0011) [2023-12-26 23:42:58,346][105620] Updated weights for policy 1, policy_version 1146628 (0.0011) [2023-12-26 23:42:58,418][105620] Updated weights for policy 1, policy_version 1146638 (0.0009) [2023-12-26 23:42:59,020][105692] Updated weights for policy 0, policy_version 1145392 (0.0006) [2023-12-26 23:42:59,080][105692] Updated weights for policy 0, policy_version 1145402 (0.0008) [2023-12-26 23:42:59,133][105620] Updated weights for policy 1, policy_version 1146648 (0.0008) [2023-12-26 23:42:59,141][105692] Updated weights for policy 0, policy_version 1145412 (0.0007) [2023-12-26 23:42:59,192][105620] Updated weights for policy 1, policy_version 1146658 (0.0008) [2023-12-26 23:42:59,264][105620] Updated weights for policy 1, policy_version 1146668 (0.0008) [2023-12-26 23:42:59,930][105692] Updated weights for policy 0, policy_version 1145422 (0.0009) [2023-12-26 23:42:59,958][105620] Updated weights for policy 1, policy_version 1146678 (0.0008) [2023-12-26 23:42:59,985][105692] Updated weights for policy 0, policy_version 1145432 (0.0006) [2023-12-26 23:43:00,016][105620] Updated weights for policy 1, policy_version 1146688 (0.0008) [2023-12-26 23:43:00,040][105692] Updated weights for policy 0, policy_version 1145442 (0.0006) [2023-12-26 23:43:00,072][105620] Updated weights for policy 1, policy_version 1146698 (0.0008) [2023-12-26 23:43:00,704][105692] Updated weights for policy 0, policy_version 1145452 (0.0005) [2023-12-26 23:43:00,755][105620] Updated weights for policy 1, policy_version 1146708 (0.0007) [2023-12-26 23:43:00,763][105692] Updated weights for policy 0, policy_version 1145462 (0.0008) [2023-12-26 23:43:00,816][105620] Updated weights for policy 1, policy_version 1146718 (0.0005) [2023-12-26 23:43:00,819][105692] Updated weights for policy 0, policy_version 1145472 (0.0007) [2023-12-26 23:43:00,877][105620] Updated weights for policy 1, policy_version 1146728 (0.0006) [2023-12-26 23:43:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 586891264. Throughput: 0: 10035.7, 1: 9283.7. Samples: 586857648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:43:01,062][104569] Avg episode reward: [(0, '8892.391'), (1, '8990.204')] [2023-12-26 23:43:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001145480_293289984.pth... [2023-12-26 23:43:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001146736_293601280.pth... [2023-12-26 23:43:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001144328_292995072.pth [2023-12-26 23:43:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001145616_293314560.pth [2023-12-26 23:43:01,468][105692] Updated weights for policy 0, policy_version 1145482 (0.0007) [2023-12-26 23:43:01,532][105692] Updated weights for policy 0, policy_version 1145492 (0.0006) [2023-12-26 23:43:01,600][105692] Updated weights for policy 0, policy_version 1145502 (0.0007) [2023-12-26 23:43:01,602][105620] Updated weights for policy 1, policy_version 1146738 (0.0007) [2023-12-26 23:43:01,663][105692] Updated weights for policy 0, policy_version 1145512 (0.0008) [2023-12-26 23:43:01,671][105620] Updated weights for policy 1, policy_version 1146748 (0.0007) [2023-12-26 23:43:01,734][105620] Updated weights for policy 1, policy_version 1146758 (0.0007) [2023-12-26 23:43:01,788][105620] Updated weights for policy 1, policy_version 1146768 (0.0009) [2023-12-26 23:43:02,374][105692] Updated weights for policy 0, policy_version 1145522 (0.0010) [2023-12-26 23:43:02,433][105692] Updated weights for policy 0, policy_version 1145532 (0.0009) [2023-12-26 23:43:02,491][105692] Updated weights for policy 0, policy_version 1145542 (0.0008) [2023-12-26 23:43:02,505][105620] Updated weights for policy 1, policy_version 1146778 (0.0006) [2023-12-26 23:43:02,566][105620] Updated weights for policy 1, policy_version 1146788 (0.0010) [2023-12-26 23:43:02,626][105620] Updated weights for policy 1, policy_version 1146798 (0.0010) [2023-12-26 23:43:03,265][105620] Updated weights for policy 1, policy_version 1146808 (0.0009) [2023-12-26 23:43:03,284][105692] Updated weights for policy 0, policy_version 1145552 (0.0008) [2023-12-26 23:43:03,318][105620] Updated weights for policy 1, policy_version 1146818 (0.0007) [2023-12-26 23:43:03,332][105692] Updated weights for policy 0, policy_version 1145562 (0.0007) [2023-12-26 23:43:03,381][105620] Updated weights for policy 1, policy_version 1146828 (0.0008) [2023-12-26 23:43:03,384][105692] Updated weights for policy 0, policy_version 1145572 (0.0006) [2023-12-26 23:43:03,949][105692] Updated weights for policy 0, policy_version 1145582 (0.0008) [2023-12-26 23:43:04,000][105692] Updated weights for policy 0, policy_version 1145592 (0.0008) [2023-12-26 23:43:04,005][105620] Updated weights for policy 1, policy_version 1146838 (0.0006) [2023-12-26 23:43:04,056][105620] Updated weights for policy 1, policy_version 1146848 (0.0008) [2023-12-26 23:43:04,058][105692] Updated weights for policy 0, policy_version 1145602 (0.0008) [2023-12-26 23:43:04,120][105620] Updated weights for policy 1, policy_version 1146858 (0.0006) [2023-12-26 23:43:04,828][105692] Updated weights for policy 0, policy_version 1145612 (0.0009) [2023-12-26 23:43:04,860][105620] Updated weights for policy 1, policy_version 1146868 (0.0005) [2023-12-26 23:43:04,881][105692] Updated weights for policy 0, policy_version 1145622 (0.0008) [2023-12-26 23:43:04,918][105620] Updated weights for policy 1, policy_version 1146878 (0.0009) [2023-12-26 23:43:04,930][105692] Updated weights for policy 0, policy_version 1145632 (0.0010) [2023-12-26 23:43:04,978][105620] Updated weights for policy 1, policy_version 1146888 (0.0009) [2023-12-26 23:43:05,525][105692] Updated weights for policy 0, policy_version 1145642 (0.0006) [2023-12-26 23:43:05,586][105692] Updated weights for policy 0, policy_version 1145652 (0.0010) [2023-12-26 23:43:05,639][105692] Updated weights for policy 0, policy_version 1145663 (0.0010) [2023-12-26 23:43:05,654][105620] Updated weights for policy 1, policy_version 1146898 (0.0010) [2023-12-26 23:43:05,704][105620] Updated weights for policy 1, policy_version 1146908 (0.0007) [2023-12-26 23:43:05,758][105620] Updated weights for policy 1, policy_version 1146918 (0.0009) [2023-12-26 23:43:05,810][105620] Updated weights for policy 1, policy_version 1146928 (0.0010) [2023-12-26 23:43:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 586989568. Throughput: 0: 10001.7, 1: 9418.6. Samples: 586975944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:43:06,063][104569] Avg episode reward: [(0, '8708.809'), (1, '9076.928')] [2023-12-26 23:43:06,420][105692] Updated weights for policy 0, policy_version 1145673 (0.0008) [2023-12-26 23:43:06,479][105692] Updated weights for policy 0, policy_version 1145683 (0.0009) [2023-12-26 23:43:06,537][105692] Updated weights for policy 0, policy_version 1145693 (0.0008) [2023-12-26 23:43:06,568][105620] Updated weights for policy 1, policy_version 1146938 (0.0007) [2023-12-26 23:43:06,596][105692] Updated weights for policy 0, policy_version 1145703 (0.0008) [2023-12-26 23:43:06,623][105620] Updated weights for policy 1, policy_version 1146948 (0.0005) [2023-12-26 23:43:06,679][105620] Updated weights for policy 1, policy_version 1146958 (0.0005) [2023-12-26 23:43:07,365][105692] Updated weights for policy 0, policy_version 1145713 (0.0009) [2023-12-26 23:43:07,384][105620] Updated weights for policy 1, policy_version 1146968 (0.0007) [2023-12-26 23:43:07,416][105692] Updated weights for policy 0, policy_version 1145723 (0.0006) [2023-12-26 23:43:07,440][105620] Updated weights for policy 1, policy_version 1146978 (0.0007) [2023-12-26 23:43:07,483][105692] Updated weights for policy 0, policy_version 1145733 (0.0010) [2023-12-26 23:43:07,510][105620] Updated weights for policy 1, policy_version 1146988 (0.0005) [2023-12-26 23:43:08,145][105692] Updated weights for policy 0, policy_version 1145743 (0.0008) [2023-12-26 23:43:08,194][105692] Updated weights for policy 0, policy_version 1145753 (0.0009) [2023-12-26 23:43:08,214][105620] Updated weights for policy 1, policy_version 1146998 (0.0005) [2023-12-26 23:43:08,246][105692] Updated weights for policy 0, policy_version 1145763 (0.0007) [2023-12-26 23:43:08,268][105620] Updated weights for policy 1, policy_version 1147008 (0.0008) [2023-12-26 23:43:08,327][105620] Updated weights for policy 1, policy_version 1147018 (0.0008) [2023-12-26 23:43:08,941][105692] Updated weights for policy 0, policy_version 1145773 (0.0006) [2023-12-26 23:43:09,007][105692] Updated weights for policy 0, policy_version 1145783 (0.0005) [2023-12-26 23:43:09,056][105620] Updated weights for policy 1, policy_version 1147028 (0.0008) [2023-12-26 23:43:09,076][105692] Updated weights for policy 0, policy_version 1145793 (0.0006) [2023-12-26 23:43:09,120][105620] Updated weights for policy 1, policy_version 1147038 (0.0011) [2023-12-26 23:43:09,187][105620] Updated weights for policy 1, policy_version 1147048 (0.0011) [2023-12-26 23:43:09,745][105692] Updated weights for policy 0, policy_version 1145803 (0.0007) [2023-12-26 23:43:09,806][105692] Updated weights for policy 0, policy_version 1145813 (0.0010) [2023-12-26 23:43:09,842][105620] Updated weights for policy 1, policy_version 1147058 (0.0010) [2023-12-26 23:43:09,876][105692] Updated weights for policy 0, policy_version 1145823 (0.0008) [2023-12-26 23:43:09,912][105620] Updated weights for policy 1, policy_version 1147068 (0.0011) [2023-12-26 23:43:09,973][105620] Updated weights for policy 1, policy_version 1147078 (0.0007) [2023-12-26 23:43:10,037][105620] Updated weights for policy 1, policy_version 1147088 (0.0011) [2023-12-26 23:43:10,668][105692] Updated weights for policy 0, policy_version 1145833 (0.0007) [2023-12-26 23:43:10,727][105692] Updated weights for policy 0, policy_version 1145843 (0.0006) [2023-12-26 23:43:10,732][105620] Updated weights for policy 1, policy_version 1147098 (0.0005) [2023-12-26 23:43:10,779][105692] Updated weights for policy 0, policy_version 1145853 (0.0006) [2023-12-26 23:43:10,791][105620] Updated weights for policy 1, policy_version 1147108 (0.0008) [2023-12-26 23:43:10,840][105692] Updated weights for policy 0, policy_version 1145863 (0.0005) [2023-12-26 23:43:10,857][105620] Updated weights for policy 1, policy_version 1147118 (0.0010) [2023-12-26 23:43:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 587087872. Throughput: 0: 9946.1, 1: 9555.3. Samples: 587093276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:43:11,062][104569] Avg episode reward: [(0, '8894.902'), (1, '8683.821')] [2023-12-26 23:43:11,538][105692] Updated weights for policy 0, policy_version 1145873 (0.0008) [2023-12-26 23:43:11,597][105692] Updated weights for policy 0, policy_version 1145883 (0.0008) [2023-12-26 23:43:11,632][105620] Updated weights for policy 1, policy_version 1147128 (0.0010) [2023-12-26 23:43:11,662][105692] Updated weights for policy 0, policy_version 1145893 (0.0008) [2023-12-26 23:43:11,695][105620] Updated weights for policy 1, policy_version 1147138 (0.0011) [2023-12-26 23:43:11,759][105620] Updated weights for policy 1, policy_version 1147148 (0.0010) [2023-12-26 23:43:12,340][105692] Updated weights for policy 0, policy_version 1145903 (0.0008) [2023-12-26 23:43:12,406][105692] Updated weights for policy 0, policy_version 1145913 (0.0008) [2023-12-26 23:43:12,466][105692] Updated weights for policy 0, policy_version 1145923 (0.0008) [2023-12-26 23:43:12,508][105620] Updated weights for policy 1, policy_version 1147158 (0.0010) [2023-12-26 23:43:12,561][105620] Updated weights for policy 1, policy_version 1147168 (0.0005) [2023-12-26 23:43:12,610][105620] Updated weights for policy 1, policy_version 1147178 (0.0005) [2023-12-26 23:43:13,154][105692] Updated weights for policy 0, policy_version 1145933 (0.0007) [2023-12-26 23:43:13,211][105692] Updated weights for policy 0, policy_version 1145943 (0.0007) [2023-12-26 23:43:13,277][105692] Updated weights for policy 0, policy_version 1145953 (0.0005) [2023-12-26 23:43:13,282][105620] Updated weights for policy 1, policy_version 1147188 (0.0008) [2023-12-26 23:43:13,331][105620] Updated weights for policy 1, policy_version 1147198 (0.0010) [2023-12-26 23:43:13,389][105620] Updated weights for policy 1, policy_version 1147208 (0.0010) [2023-12-26 23:43:13,911][105692] Updated weights for policy 0, policy_version 1145963 (0.0006) [2023-12-26 23:43:13,971][105692] Updated weights for policy 0, policy_version 1145973 (0.0008) [2023-12-26 23:43:14,025][105692] Updated weights for policy 0, policy_version 1145983 (0.0005) [2023-12-26 23:43:14,154][105620] Updated weights for policy 1, policy_version 1147218 (0.0011) [2023-12-26 23:43:14,217][105620] Updated weights for policy 1, policy_version 1147228 (0.0011) [2023-12-26 23:43:14,261][105620] Updated weights for policy 1, policy_version 1147238 (0.0010) [2023-12-26 23:43:14,310][105620] Updated weights for policy 1, policy_version 1147248 (0.0010) [2023-12-26 23:43:14,588][105692] Updated weights for policy 0, policy_version 1145993 (0.0005) [2023-12-26 23:43:14,651][105692] Updated weights for policy 0, policy_version 1146003 (0.0005) [2023-12-26 23:43:14,709][105692] Updated weights for policy 0, policy_version 1146013 (0.0005) [2023-12-26 23:43:14,754][105692] Updated weights for policy 0, policy_version 1146023 (0.0005) [2023-12-26 23:43:15,033][105620] Updated weights for policy 1, policy_version 1147258 (0.0008) [2023-12-26 23:43:15,097][105620] Updated weights for policy 1, policy_version 1147268 (0.0009) [2023-12-26 23:43:15,162][105620] Updated weights for policy 1, policy_version 1147278 (0.0008) [2023-12-26 23:43:15,464][105692] Updated weights for policy 0, policy_version 1146033 (0.0009) [2023-12-26 23:43:15,527][105692] Updated weights for policy 0, policy_version 1146043 (0.0011) [2023-12-26 23:43:15,587][105692] Updated weights for policy 0, policy_version 1146053 (0.0011) [2023-12-26 23:43:15,870][105620] Updated weights for policy 1, policy_version 1147288 (0.0008) [2023-12-26 23:43:15,932][105620] Updated weights for policy 1, policy_version 1147298 (0.0006) [2023-12-26 23:43:16,000][105620] Updated weights for policy 1, policy_version 1147308 (0.0007) [2023-12-26 23:43:16,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 587186176. Throughput: 0: 9945.2, 1: 9592.4. Samples: 587151600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:43:16,062][104569] Avg episode reward: [(0, '9262.114'), (1, '8424.087')] [2023-12-26 23:43:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001146056_293437440.pth... [2023-12-26 23:43:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001147312_293748736.pth... [2023-12-26 23:43:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001144904_293142528.pth [2023-12-26 23:43:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001146192_293462016.pth [2023-12-26 23:43:16,296][105692] Updated weights for policy 0, policy_version 1146063 (0.0007) [2023-12-26 23:43:16,344][105692] Updated weights for policy 0, policy_version 1146073 (0.0005) [2023-12-26 23:43:16,400][105692] Updated weights for policy 0, policy_version 1146083 (0.0006) [2023-12-26 23:43:16,547][105620] Updated weights for policy 1, policy_version 1147318 (0.0007) [2023-12-26 23:43:16,599][105620] Updated weights for policy 1, policy_version 1147328 (0.0006) [2023-12-26 23:43:16,653][105620] Updated weights for policy 1, policy_version 1147338 (0.0009) [2023-12-26 23:43:16,938][105692] Updated weights for policy 0, policy_version 1146093 (0.0005) [2023-12-26 23:43:16,987][105692] Updated weights for policy 0, policy_version 1146103 (0.0005) [2023-12-26 23:43:17,048][105692] Updated weights for policy 0, policy_version 1146113 (0.0005) [2023-12-26 23:43:17,200][105620] Updated weights for policy 1, policy_version 1147348 (0.0007) [2023-12-26 23:43:17,252][105620] Updated weights for policy 1, policy_version 1147358 (0.0010) [2023-12-26 23:43:17,313][105620] Updated weights for policy 1, policy_version 1147368 (0.0010) [2023-12-26 23:43:17,670][105692] Updated weights for policy 0, policy_version 1146123 (0.0007) [2023-12-26 23:43:17,722][105692] Updated weights for policy 0, policy_version 1146133 (0.0010) [2023-12-26 23:43:17,767][105692] Updated weights for policy 0, policy_version 1146143 (0.0010) [2023-12-26 23:43:18,012][105620] Updated weights for policy 1, policy_version 1147378 (0.0010) [2023-12-26 23:43:18,060][105620] Updated weights for policy 1, policy_version 1147388 (0.0010) [2023-12-26 23:43:18,111][105620] Updated weights for policy 1, policy_version 1147398 (0.0010) [2023-12-26 23:43:18,163][105620] Updated weights for policy 1, policy_version 1147408 (0.0010) [2023-12-26 23:43:18,523][105692] Updated weights for policy 0, policy_version 1146153 (0.0010) [2023-12-26 23:43:18,582][105692] Updated weights for policy 0, policy_version 1146163 (0.0011) [2023-12-26 23:43:18,637][105692] Updated weights for policy 0, policy_version 1146173 (0.0010) [2023-12-26 23:43:18,693][105692] Updated weights for policy 0, policy_version 1146183 (0.0010) [2023-12-26 23:43:18,860][105620] Updated weights for policy 1, policy_version 1147418 (0.0011) [2023-12-26 23:43:18,919][105620] Updated weights for policy 1, policy_version 1147428 (0.0010) [2023-12-26 23:43:18,980][105620] Updated weights for policy 1, policy_version 1147438 (0.0009) [2023-12-26 23:43:19,446][105692] Updated weights for policy 0, policy_version 1146193 (0.0009) [2023-12-26 23:43:19,507][105692] Updated weights for policy 0, policy_version 1146203 (0.0009) [2023-12-26 23:43:19,565][105692] Updated weights for policy 0, policy_version 1146213 (0.0010) [2023-12-26 23:43:19,665][105620] Updated weights for policy 1, policy_version 1147448 (0.0009) [2023-12-26 23:43:19,715][105620] Updated weights for policy 1, policy_version 1147458 (0.0009) [2023-12-26 23:43:19,763][105620] Updated weights for policy 1, policy_version 1147468 (0.0009) [2023-12-26 23:43:20,314][105692] Updated weights for policy 0, policy_version 1146223 (0.0010) [2023-12-26 23:43:20,376][105692] Updated weights for policy 0, policy_version 1146233 (0.0009) [2023-12-26 23:43:20,435][105692] Updated weights for policy 0, policy_version 1146243 (0.0009) [2023-12-26 23:43:20,468][105620] Updated weights for policy 1, policy_version 1147478 (0.0007) [2023-12-26 23:43:20,530][105620] Updated weights for policy 1, policy_version 1147488 (0.0010) [2023-12-26 23:43:20,597][105620] Updated weights for policy 1, policy_version 1147498 (0.0008) [2023-12-26 23:43:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19660.8). Total num frames: 587284480. Throughput: 0: 9968.9, 1: 9658.1. Samples: 587275756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:43:21,063][104569] Avg episode reward: [(0, '9354.664'), (1, '8823.811')] [2023-12-26 23:43:21,303][105620] Updated weights for policy 1, policy_version 1147508 (0.0009) [2023-12-26 23:43:21,319][105692] Updated weights for policy 0, policy_version 1146253 (0.0008) [2023-12-26 23:43:21,376][105620] Updated weights for policy 1, policy_version 1147518 (0.0008) [2023-12-26 23:43:21,391][105692] Updated weights for policy 0, policy_version 1146263 (0.0008) [2023-12-26 23:43:21,435][105620] Updated weights for policy 1, policy_version 1147528 (0.0009) [2023-12-26 23:43:21,453][105692] Updated weights for policy 0, policy_version 1146273 (0.0006) [2023-12-26 23:43:22,134][105692] Updated weights for policy 0, policy_version 1146283 (0.0007) [2023-12-26 23:43:22,200][105692] Updated weights for policy 0, policy_version 1146293 (0.0010) [2023-12-26 23:43:22,265][105620] Updated weights for policy 1, policy_version 1147538 (0.0008) [2023-12-26 23:43:22,266][105692] Updated weights for policy 0, policy_version 1146303 (0.0011) [2023-12-26 23:43:22,331][105620] Updated weights for policy 1, policy_version 1147548 (0.0010) [2023-12-26 23:43:22,400][105620] Updated weights for policy 1, policy_version 1147558 (0.0009) [2023-12-26 23:43:22,463][105620] Updated weights for policy 1, policy_version 1147568 (0.0009) [2023-12-26 23:43:22,990][105692] Updated weights for policy 0, policy_version 1146313 (0.0010) [2023-12-26 23:43:23,056][105692] Updated weights for policy 0, policy_version 1146323 (0.0011) [2023-12-26 23:43:23,114][105692] Updated weights for policy 0, policy_version 1146333 (0.0010) [2023-12-26 23:43:23,179][105692] Updated weights for policy 0, policy_version 1146343 (0.0011) [2023-12-26 23:43:23,222][105620] Updated weights for policy 1, policy_version 1147578 (0.0007) [2023-12-26 23:43:23,274][105620] Updated weights for policy 1, policy_version 1147588 (0.0008) [2023-12-26 23:43:23,328][105620] Updated weights for policy 1, policy_version 1147598 (0.0008) [2023-12-26 23:43:23,774][105692] Updated weights for policy 0, policy_version 1146353 (0.0006) [2023-12-26 23:43:23,837][105692] Updated weights for policy 0, policy_version 1146363 (0.0005) [2023-12-26 23:43:23,887][105692] Updated weights for policy 0, policy_version 1146373 (0.0005) [2023-12-26 23:43:24,135][105620] Updated weights for policy 1, policy_version 1147608 (0.0006) [2023-12-26 23:43:24,187][105620] Updated weights for policy 1, policy_version 1147618 (0.0005) [2023-12-26 23:43:24,223][105586] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000009 [2023-12-26 23:43:24,456][105692] Updated weights for policy 0, policy_version 1146383 (0.0009) [2023-12-26 23:43:24,517][105692] Updated weights for policy 0, policy_version 1146393 (0.0009) [2023-12-26 23:43:24,590][105692] Updated weights for policy 0, policy_version 1146403 (0.0008) [2023-12-26 23:43:24,738][105620] Updated weights for policy 1, policy_version 1147628 (0.0005) [2023-12-26 23:43:24,789][105620] Updated weights for policy 1, policy_version 1147638 (0.0005) [2023-12-26 23:43:24,838][105620] Updated weights for policy 1, policy_version 1147648 (0.0005) [2023-12-26 23:43:25,314][105692] Updated weights for policy 0, policy_version 1146413 (0.0009) [2023-12-26 23:43:25,376][105692] Updated weights for policy 0, policy_version 1146423 (0.0010) [2023-12-26 23:43:25,390][105620] Updated weights for policy 1, policy_version 1147658 (0.0005) [2023-12-26 23:43:25,434][105692] Updated weights for policy 0, policy_version 1146433 (0.0010) [2023-12-26 23:43:25,448][105620] Updated weights for policy 1, policy_version 1147668 (0.0007) [2023-12-26 23:43:25,506][105620] Updated weights for policy 1, policy_version 1147678 (0.0010) [2023-12-26 23:43:25,574][105620] Updated weights for policy 1, policy_version 1147688 (0.0010) [2023-12-26 23:43:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.9, 300 sec: 19633.0). Total num frames: 587382784. Throughput: 0: 9940.7, 1: 9746.8. Samples: 587395216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:43:26,062][104569] Avg episode reward: [(0, '9356.062'), (1, '9173.533')] [2023-12-26 23:43:26,172][105692] Updated weights for policy 0, policy_version 1146443 (0.0009) [2023-12-26 23:43:26,226][105692] Updated weights for policy 0, policy_version 1146453 (0.0005) [2023-12-26 23:43:26,227][105620] Updated weights for policy 1, policy_version 1147698 (0.0006) [2023-12-26 23:43:26,276][105620] Updated weights for policy 1, policy_version 1147708 (0.0005) [2023-12-26 23:43:26,289][105692] Updated weights for policy 0, policy_version 1146463 (0.0006) [2023-12-26 23:43:26,327][105620] Updated weights for policy 1, policy_version 1147718 (0.0005) [2023-12-26 23:43:26,973][105620] Updated weights for policy 1, policy_version 1147728 (0.0009) [2023-12-26 23:43:27,000][105692] Updated weights for policy 0, policy_version 1146473 (0.0005) [2023-12-26 23:43:27,031][105620] Updated weights for policy 1, policy_version 1147738 (0.0011) [2023-12-26 23:43:27,053][105692] Updated weights for policy 0, policy_version 1146483 (0.0006) [2023-12-26 23:43:27,090][105620] Updated weights for policy 1, policy_version 1147748 (0.0010) [2023-12-26 23:43:27,115][105692] Updated weights for policy 0, policy_version 1146493 (0.0006) [2023-12-26 23:43:27,178][105692] Updated weights for policy 0, policy_version 1146503 (0.0007) [2023-12-26 23:43:27,703][105620] Updated weights for policy 1, policy_version 1147758 (0.0007) [2023-12-26 23:43:27,761][105620] Updated weights for policy 1, policy_version 1147768 (0.0010) [2023-12-26 23:43:27,785][105692] Updated weights for policy 0, policy_version 1146513 (0.0005) [2023-12-26 23:43:27,819][105620] Updated weights for policy 1, policy_version 1147778 (0.0010) [2023-12-26 23:43:27,843][105692] Updated weights for policy 0, policy_version 1146523 (0.0005) [2023-12-26 23:43:27,902][105692] Updated weights for policy 0, policy_version 1146533 (0.0005) [2023-12-26 23:43:28,505][105620] Updated weights for policy 1, policy_version 1147788 (0.0008) [2023-12-26 23:43:28,540][105692] Updated weights for policy 0, policy_version 1146543 (0.0008) [2023-12-26 23:43:28,566][105620] Updated weights for policy 1, policy_version 1147798 (0.0005) [2023-12-26 23:43:28,603][105692] Updated weights for policy 0, policy_version 1146553 (0.0009) [2023-12-26 23:43:28,626][105620] Updated weights for policy 1, policy_version 1147808 (0.0005) [2023-12-26 23:43:28,667][105692] Updated weights for policy 0, policy_version 1146563 (0.0009) [2023-12-26 23:43:29,234][105620] Updated weights for policy 1, policy_version 1147818 (0.0006) [2023-12-26 23:43:29,289][105620] Updated weights for policy 1, policy_version 1147828 (0.0007) [2023-12-26 23:43:29,353][105620] Updated weights for policy 1, policy_version 1147838 (0.0010) [2023-12-26 23:43:29,421][105620] Updated weights for policy 1, policy_version 1147848 (0.0009) [2023-12-26 23:43:29,482][105692] Updated weights for policy 0, policy_version 1146573 (0.0009) [2023-12-26 23:43:29,541][105692] Updated weights for policy 0, policy_version 1146583 (0.0009) [2023-12-26 23:43:29,599][105692] Updated weights for policy 0, policy_version 1146593 (0.0009) [2023-12-26 23:43:30,109][105620] Updated weights for policy 1, policy_version 1147858 (0.0008) [2023-12-26 23:43:30,166][105620] Updated weights for policy 1, policy_version 1147868 (0.0009) [2023-12-26 23:43:30,223][105620] Updated weights for policy 1, policy_version 1147878 (0.0009) [2023-12-26 23:43:30,405][105692] Updated weights for policy 0, policy_version 1146603 (0.0010) [2023-12-26 23:43:30,466][105692] Updated weights for policy 0, policy_version 1146613 (0.0008) [2023-12-26 23:43:30,537][105692] Updated weights for policy 0, policy_version 1146623 (0.0010) [2023-12-26 23:43:30,855][105620] Updated weights for policy 1, policy_version 1147888 (0.0009) [2023-12-26 23:43:30,912][105620] Updated weights for policy 1, policy_version 1147898 (0.0008) [2023-12-26 23:43:30,961][105620] Updated weights for policy 1, policy_version 1147908 (0.0009) [2023-12-26 23:43:31,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 587489280. Throughput: 0: 9979.6, 1: 9859.8. Samples: 587457848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:43:31,063][104569] Avg episode reward: [(0, '9264.922'), (1, '9348.621')] [2023-12-26 23:43:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001146632_293584896.pth... [2023-12-26 23:43:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001147912_293904384.pth... [2023-12-26 23:43:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001146736_293601280.pth [2023-12-26 23:43:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001145480_293289984.pth [2023-12-26 23:43:31,278][105692] Updated weights for policy 0, policy_version 1146633 (0.0010) [2023-12-26 23:43:31,346][105692] Updated weights for policy 0, policy_version 1146643 (0.0008) [2023-12-26 23:43:31,416][105692] Updated weights for policy 0, policy_version 1146653 (0.0008) [2023-12-26 23:43:31,464][105692] Updated weights for policy 0, policy_version 1146663 (0.0008) [2023-12-26 23:43:31,716][105620] Updated weights for policy 1, policy_version 1147918 (0.0009) [2023-12-26 23:43:31,778][105620] Updated weights for policy 1, policy_version 1147928 (0.0009) [2023-12-26 23:43:31,830][105620] Updated weights for policy 1, policy_version 1147938 (0.0009) [2023-12-26 23:43:32,154][105692] Updated weights for policy 0, policy_version 1146673 (0.0009) [2023-12-26 23:43:32,212][105692] Updated weights for policy 0, policy_version 1146683 (0.0010) [2023-12-26 23:43:32,275][105692] Updated weights for policy 0, policy_version 1146693 (0.0008) [2023-12-26 23:43:32,595][105620] Updated weights for policy 1, policy_version 1147948 (0.0009) [2023-12-26 23:43:32,652][105620] Updated weights for policy 1, policy_version 1147958 (0.0008) [2023-12-26 23:43:32,706][105620] Updated weights for policy 1, policy_version 1147968 (0.0009) [2023-12-26 23:43:33,108][105692] Updated weights for policy 0, policy_version 1146703 (0.0009) [2023-12-26 23:43:33,167][105692] Updated weights for policy 0, policy_version 1146713 (0.0008) [2023-12-26 23:43:33,228][105692] Updated weights for policy 0, policy_version 1146723 (0.0009) [2023-12-26 23:43:33,392][105620] Updated weights for policy 1, policy_version 1147978 (0.0009) [2023-12-26 23:43:33,442][105620] Updated weights for policy 1, policy_version 1147988 (0.0009) [2023-12-26 23:43:33,501][105620] Updated weights for policy 1, policy_version 1147998 (0.0009) [2023-12-26 23:43:33,558][105620] Updated weights for policy 1, policy_version 1148008 (0.0009) [2023-12-26 23:43:33,994][105692] Updated weights for policy 0, policy_version 1146733 (0.0009) [2023-12-26 23:43:34,049][105692] Updated weights for policy 0, policy_version 1146743 (0.0008) [2023-12-26 23:43:34,096][105692] Updated weights for policy 0, policy_version 1146753 (0.0007) [2023-12-26 23:43:34,252][105620] Updated weights for policy 1, policy_version 1148018 (0.0010) [2023-12-26 23:43:34,301][105620] Updated weights for policy 1, policy_version 1148028 (0.0010) [2023-12-26 23:43:34,353][105620] Updated weights for policy 1, policy_version 1148038 (0.0010) [2023-12-26 23:43:34,890][105692] Updated weights for policy 0, policy_version 1146763 (0.0008) [2023-12-26 23:43:34,941][105692] Updated weights for policy 0, policy_version 1146773 (0.0008) [2023-12-26 23:43:35,004][105692] Updated weights for policy 0, policy_version 1146783 (0.0008) [2023-12-26 23:43:35,120][105620] Updated weights for policy 1, policy_version 1148048 (0.0010) [2023-12-26 23:43:35,171][105620] Updated weights for policy 1, policy_version 1148058 (0.0010) [2023-12-26 23:43:35,219][105620] Updated weights for policy 1, policy_version 1148068 (0.0010) [2023-12-26 23:43:35,748][105692] Updated weights for policy 0, policy_version 1146793 (0.0008) [2023-12-26 23:43:35,802][105692] Updated weights for policy 0, policy_version 1146803 (0.0005) [2023-12-26 23:43:35,848][105692] Updated weights for policy 0, policy_version 1146813 (0.0005) [2023-12-26 23:43:35,897][105692] Updated weights for policy 0, policy_version 1146823 (0.0008) [2023-12-26 23:43:35,943][105620] Updated weights for policy 1, policy_version 1148078 (0.0010) [2023-12-26 23:43:35,998][105620] Updated weights for policy 1, policy_version 1148088 (0.0010) [2023-12-26 23:43:36,046][105620] Updated weights for policy 1, policy_version 1148098 (0.0010) [2023-12-26 23:43:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 587579392. Throughput: 0: 9908.4, 1: 9926.6. Samples: 587571360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:43:36,062][104569] Avg episode reward: [(0, '9170.850'), (1, '9167.479')] [2023-12-26 23:43:36,623][105692] Updated weights for policy 0, policy_version 1146833 (0.0008) [2023-12-26 23:43:36,677][105692] Updated weights for policy 0, policy_version 1146843 (0.0009) [2023-12-26 23:43:36,727][105692] Updated weights for policy 0, policy_version 1146853 (0.0008) [2023-12-26 23:43:36,834][105620] Updated weights for policy 1, policy_version 1148108 (0.0010) [2023-12-26 23:43:36,899][105620] Updated weights for policy 1, policy_version 1148118 (0.0010) [2023-12-26 23:43:36,958][105620] Updated weights for policy 1, policy_version 1148128 (0.0010) [2023-12-26 23:43:37,485][105692] Updated weights for policy 0, policy_version 1146863 (0.0008) [2023-12-26 23:43:37,545][105692] Updated weights for policy 0, policy_version 1146873 (0.0008) [2023-12-26 23:43:37,605][105692] Updated weights for policy 0, policy_version 1146883 (0.0009) [2023-12-26 23:43:37,670][105620] Updated weights for policy 1, policy_version 1148138 (0.0010) [2023-12-26 23:43:37,734][105620] Updated weights for policy 1, policy_version 1148148 (0.0008) [2023-12-26 23:43:37,786][105620] Updated weights for policy 1, policy_version 1148158 (0.0005) [2023-12-26 23:43:37,838][105620] Updated weights for policy 1, policy_version 1148168 (0.0005) [2023-12-26 23:43:38,332][105692] Updated weights for policy 0, policy_version 1146894 (0.0009) [2023-12-26 23:43:38,391][105692] Updated weights for policy 0, policy_version 1146904 (0.0007) [2023-12-26 23:43:38,443][105620] Updated weights for policy 1, policy_version 1148178 (0.0006) [2023-12-26 23:43:38,452][105692] Updated weights for policy 0, policy_version 1146914 (0.0008) [2023-12-26 23:43:38,511][105620] Updated weights for policy 1, policy_version 1148188 (0.0006) [2023-12-26 23:43:38,568][105620] Updated weights for policy 1, policy_version 1148198 (0.0006) [2023-12-26 23:43:39,109][105692] Updated weights for policy 0, policy_version 1146924 (0.0008) [2023-12-26 23:43:39,168][105692] Updated weights for policy 0, policy_version 1146934 (0.0008) [2023-12-26 23:43:39,234][105692] Updated weights for policy 0, policy_version 1146944 (0.0008) [2023-12-26 23:43:39,245][105620] Updated weights for policy 1, policy_version 1148208 (0.0007) [2023-12-26 23:43:39,306][105620] Updated weights for policy 1, policy_version 1148218 (0.0007) [2023-12-26 23:43:39,369][105620] Updated weights for policy 1, policy_version 1148228 (0.0008) [2023-12-26 23:43:39,982][105692] Updated weights for policy 0, policy_version 1146954 (0.0009) [2023-12-26 23:43:40,014][105620] Updated weights for policy 1, policy_version 1148238 (0.0008) [2023-12-26 23:43:40,050][105692] Updated weights for policy 0, policy_version 1146964 (0.0011) [2023-12-26 23:43:40,076][105620] Updated weights for policy 1, policy_version 1148248 (0.0008) [2023-12-26 23:43:40,110][105692] Updated weights for policy 0, policy_version 1146974 (0.0011) [2023-12-26 23:43:40,140][105620] Updated weights for policy 1, policy_version 1148258 (0.0008) [2023-12-26 23:43:40,163][105692] Updated weights for policy 0, policy_version 1146984 (0.0011) [2023-12-26 23:43:40,880][105620] Updated weights for policy 1, policy_version 1148268 (0.0007) [2023-12-26 23:43:40,936][105620] Updated weights for policy 1, policy_version 1148278 (0.0006) [2023-12-26 23:43:40,941][105692] Updated weights for policy 0, policy_version 1146994 (0.0010) [2023-12-26 23:43:40,984][105620] Updated weights for policy 1, policy_version 1148288 (0.0007) [2023-12-26 23:43:40,996][105692] Updated weights for policy 0, policy_version 1147004 (0.0010) [2023-12-26 23:43:41,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 587677696. Throughput: 0: 9780.8, 1: 9932.2. Samples: 587687528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:43:41,062][104569] Avg episode reward: [(0, '9169.870'), (1, '9166.928')] [2023-12-26 23:43:41,075][105692] Updated weights for policy 0, policy_version 1147014 (0.0009) [2023-12-26 23:43:41,809][105620] Updated weights for policy 1, policy_version 1148298 (0.0009) [2023-12-26 23:43:41,867][105692] Updated weights for policy 0, policy_version 1147024 (0.0008) [2023-12-26 23:43:41,868][105620] Updated weights for policy 1, policy_version 1148308 (0.0009) [2023-12-26 23:43:41,920][105692] Updated weights for policy 0, policy_version 1147034 (0.0007) [2023-12-26 23:43:41,932][105620] Updated weights for policy 1, policy_version 1148318 (0.0009) [2023-12-26 23:43:41,971][105692] Updated weights for policy 0, policy_version 1147044 (0.0007) [2023-12-26 23:43:41,993][105620] Updated weights for policy 1, policy_version 1148328 (0.0009) [2023-12-26 23:43:42,726][105620] Updated weights for policy 1, policy_version 1148338 (0.0008) [2023-12-26 23:43:42,782][105620] Updated weights for policy 1, policy_version 1148348 (0.0009) [2023-12-26 23:43:42,790][105692] Updated weights for policy 0, policy_version 1147054 (0.0009) [2023-12-26 23:43:42,841][105620] Updated weights for policy 1, policy_version 1148358 (0.0009) [2023-12-26 23:43:42,847][105692] Updated weights for policy 0, policy_version 1147064 (0.0010) [2023-12-26 23:43:42,910][105692] Updated weights for policy 0, policy_version 1147074 (0.0009) [2023-12-26 23:43:43,577][105620] Updated weights for policy 1, policy_version 1148368 (0.0008) [2023-12-26 23:43:43,642][105620] Updated weights for policy 1, policy_version 1148378 (0.0009) [2023-12-26 23:43:43,683][105692] Updated weights for policy 0, policy_version 1147084 (0.0008) [2023-12-26 23:43:43,699][105620] Updated weights for policy 1, policy_version 1148388 (0.0008) [2023-12-26 23:43:43,737][105692] Updated weights for policy 0, policy_version 1147094 (0.0007) [2023-12-26 23:43:43,799][105692] Updated weights for policy 0, policy_version 1147104 (0.0010) [2023-12-26 23:43:44,302][105620] Updated weights for policy 1, policy_version 1148398 (0.0010) [2023-12-26 23:43:44,348][105620] Updated weights for policy 1, policy_version 1148408 (0.0008) [2023-12-26 23:43:44,403][105620] Updated weights for policy 1, policy_version 1148418 (0.0010) [2023-12-26 23:43:44,488][105692] Updated weights for policy 0, policy_version 1147115 (0.0009) [2023-12-26 23:43:44,550][105692] Updated weights for policy 0, policy_version 1147125 (0.0006) [2023-12-26 23:43:44,615][105692] Updated weights for policy 0, policy_version 1147135 (0.0007) [2023-12-26 23:43:45,133][105620] Updated weights for policy 1, policy_version 1148428 (0.0008) [2023-12-26 23:43:45,197][105620] Updated weights for policy 1, policy_version 1148438 (0.0007) [2023-12-26 23:43:45,261][105620] Updated weights for policy 1, policy_version 1148448 (0.0011) [2023-12-26 23:43:45,354][105692] Updated weights for policy 0, policy_version 1147145 (0.0007) [2023-12-26 23:43:45,415][105692] Updated weights for policy 0, policy_version 1147155 (0.0007) [2023-12-26 23:43:45,468][105692] Updated weights for policy 0, policy_version 1147165 (0.0008) [2023-12-26 23:43:45,517][105692] Updated weights for policy 0, policy_version 1147175 (0.0009) [2023-12-26 23:43:45,968][105620] Updated weights for policy 1, policy_version 1148458 (0.0010) [2023-12-26 23:43:46,019][105620] Updated weights for policy 1, policy_version 1148468 (0.0009) [2023-12-26 23:43:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 587767808. Throughput: 0: 9698.6, 1: 9971.0. Samples: 587742780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:43:46,062][104569] Avg episode reward: [(0, '8987.200'), (1, '9167.181')] [2023-12-26 23:43:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001147176_293724160.pth... [2023-12-26 23:43:46,073][105620] Updated weights for policy 1, policy_version 1148478 (0.0007) [2023-12-26 23:43:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001146056_293437440.pth [2023-12-26 23:43:46,119][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001148488_294051840.pth... [2023-12-26 23:43:46,119][105620] Updated weights for policy 1, policy_version 1148488 (0.0008) [2023-12-26 23:43:46,122][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001147312_293748736.pth [2023-12-26 23:43:46,290][105692] Updated weights for policy 0, policy_version 1147185 (0.0010) [2023-12-26 23:43:46,344][105692] Updated weights for policy 0, policy_version 1147195 (0.0010) [2023-12-26 23:43:46,401][105692] Updated weights for policy 0, policy_version 1147205 (0.0007) [2023-12-26 23:43:46,808][105620] Updated weights for policy 1, policy_version 1148498 (0.0009) [2023-12-26 23:43:46,873][105620] Updated weights for policy 1, policy_version 1148508 (0.0008) [2023-12-26 23:43:46,939][105620] Updated weights for policy 1, policy_version 1148518 (0.0007) [2023-12-26 23:43:47,111][105692] Updated weights for policy 0, policy_version 1147215 (0.0007) [2023-12-26 23:43:47,159][105692] Updated weights for policy 0, policy_version 1147225 (0.0009) [2023-12-26 23:43:47,214][105692] Updated weights for policy 0, policy_version 1147235 (0.0007) [2023-12-26 23:43:47,624][105620] Updated weights for policy 1, policy_version 1148528 (0.0006) [2023-12-26 23:43:47,674][105620] Updated weights for policy 1, policy_version 1148538 (0.0005) [2023-12-26 23:43:47,727][105620] Updated weights for policy 1, policy_version 1148548 (0.0005) [2023-12-26 23:43:48,021][105692] Updated weights for policy 0, policy_version 1147245 (0.0009) [2023-12-26 23:43:48,079][105692] Updated weights for policy 0, policy_version 1147255 (0.0009) [2023-12-26 23:43:48,138][105692] Updated weights for policy 0, policy_version 1147265 (0.0009) [2023-12-26 23:43:48,335][105620] Updated weights for policy 1, policy_version 1148558 (0.0006) [2023-12-26 23:43:48,400][105620] Updated weights for policy 1, policy_version 1148568 (0.0009) [2023-12-26 23:43:48,448][105620] Updated weights for policy 1, policy_version 1148578 (0.0009) [2023-12-26 23:43:48,872][105692] Updated weights for policy 0, policy_version 1147275 (0.0009) [2023-12-26 23:43:48,924][105692] Updated weights for policy 0, policy_version 1147285 (0.0009) [2023-12-26 23:43:48,987][105692] Updated weights for policy 0, policy_version 1147295 (0.0009) [2023-12-26 23:43:49,180][105620] Updated weights for policy 1, policy_version 1148588 (0.0009) [2023-12-26 23:43:49,256][105620] Updated weights for policy 1, policy_version 1148598 (0.0008) [2023-12-26 23:43:49,314][105620] Updated weights for policy 1, policy_version 1148608 (0.0009) [2023-12-26 23:43:49,857][105692] Updated weights for policy 0, policy_version 1147305 (0.0009) [2023-12-26 23:43:49,912][105620] Updated weights for policy 1, policy_version 1148618 (0.0008) [2023-12-26 23:43:49,914][105692] Updated weights for policy 0, policy_version 1147315 (0.0008) [2023-12-26 23:43:49,970][105620] Updated weights for policy 1, policy_version 1148628 (0.0007) [2023-12-26 23:43:49,979][105692] Updated weights for policy 0, policy_version 1147325 (0.0007) [2023-12-26 23:43:50,021][105620] Updated weights for policy 1, policy_version 1148638 (0.0008) [2023-12-26 23:43:50,065][105692] Updated weights for policy 0, policy_version 1147335 (0.0009) [2023-12-26 23:43:50,071][105620] Updated weights for policy 1, policy_version 1148648 (0.0007) [2023-12-26 23:43:50,831][105692] Updated weights for policy 0, policy_version 1147345 (0.0008) [2023-12-26 23:43:50,847][105620] Updated weights for policy 1, policy_version 1148658 (0.0006) [2023-12-26 23:43:50,891][105692] Updated weights for policy 0, policy_version 1147355 (0.0010) [2023-12-26 23:43:50,907][105620] Updated weights for policy 1, policy_version 1148668 (0.0005) [2023-12-26 23:43:50,951][105692] Updated weights for policy 0, policy_version 1147365 (0.0009) [2023-12-26 23:43:50,956][105620] Updated weights for policy 1, policy_version 1148678 (0.0007) [2023-12-26 23:43:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 587874304. Throughput: 0: 9645.0, 1: 10018.8. Samples: 587860812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:43:51,062][104569] Avg episode reward: [(0, '8932.621'), (1, '9171.414')] [2023-12-26 23:43:51,696][105620] Updated weights for policy 1, policy_version 1148688 (0.0007) [2023-12-26 23:43:51,716][105692] Updated weights for policy 0, policy_version 1147375 (0.0009) [2023-12-26 23:43:51,766][105620] Updated weights for policy 1, policy_version 1148698 (0.0007) [2023-12-26 23:43:51,784][105692] Updated weights for policy 0, policy_version 1147385 (0.0009) [2023-12-26 23:43:51,832][105620] Updated weights for policy 1, policy_version 1148708 (0.0006) [2023-12-26 23:43:51,845][105692] Updated weights for policy 0, policy_version 1147395 (0.0007) [2023-12-26 23:43:52,541][105620] Updated weights for policy 1, policy_version 1148718 (0.0007) [2023-12-26 23:43:52,605][105620] Updated weights for policy 1, policy_version 1148728 (0.0008) [2023-12-26 23:43:52,634][105692] Updated weights for policy 0, policy_version 1147405 (0.0008) [2023-12-26 23:43:52,664][105620] Updated weights for policy 1, policy_version 1148738 (0.0008) [2023-12-26 23:43:52,698][105692] Updated weights for policy 0, policy_version 1147415 (0.0008) [2023-12-26 23:43:52,747][105692] Updated weights for policy 0, policy_version 1147425 (0.0010) [2023-12-26 23:43:53,411][105692] Updated weights for policy 0, policy_version 1147435 (0.0010) [2023-12-26 23:43:53,451][105620] Updated weights for policy 1, policy_version 1148748 (0.0008) [2023-12-26 23:43:53,470][105692] Updated weights for policy 0, policy_version 1147445 (0.0011) [2023-12-26 23:43:53,517][105620] Updated weights for policy 1, policy_version 1148758 (0.0007) [2023-12-26 23:43:53,524][105692] Updated weights for policy 0, policy_version 1147455 (0.0008) [2023-12-26 23:43:53,581][105620] Updated weights for policy 1, policy_version 1148768 (0.0007) [2023-12-26 23:43:54,262][105692] Updated weights for policy 0, policy_version 1147465 (0.0008) [2023-12-26 23:43:54,329][105692] Updated weights for policy 0, policy_version 1147475 (0.0010) [2023-12-26 23:43:54,340][105620] Updated weights for policy 1, policy_version 1148778 (0.0008) [2023-12-26 23:43:54,384][105692] Updated weights for policy 0, policy_version 1147485 (0.0011) [2023-12-26 23:43:54,395][105620] Updated weights for policy 1, policy_version 1148788 (0.0006) [2023-12-26 23:43:54,443][105692] Updated weights for policy 0, policy_version 1147495 (0.0010) [2023-12-26 23:43:54,448][105620] Updated weights for policy 1, policy_version 1148798 (0.0008) [2023-12-26 23:43:54,510][105620] Updated weights for policy 1, policy_version 1148808 (0.0008) [2023-12-26 23:43:55,114][105692] Updated weights for policy 0, policy_version 1147505 (0.0006) [2023-12-26 23:43:55,168][105692] Updated weights for policy 0, policy_version 1147515 (0.0006) [2023-12-26 23:43:55,224][105692] Updated weights for policy 0, policy_version 1147525 (0.0006) [2023-12-26 23:43:55,239][105620] Updated weights for policy 1, policy_version 1148818 (0.0008) [2023-12-26 23:43:55,299][105620] Updated weights for policy 1, policy_version 1148828 (0.0010) [2023-12-26 23:43:55,354][105620] Updated weights for policy 1, policy_version 1148838 (0.0008) [2023-12-26 23:43:55,931][105692] Updated weights for policy 0, policy_version 1147535 (0.0006) [2023-12-26 23:43:55,994][105692] Updated weights for policy 0, policy_version 1147545 (0.0005) [2023-12-26 23:43:56,058][105692] Updated weights for policy 0, policy_version 1147555 (0.0008) [2023-12-26 23:43:56,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 587956224. Throughput: 0: 9605.1, 1: 9942.7. Samples: 587972932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:43:56,062][104569] Avg episode reward: [(0, '9114.323'), (1, '8905.361')] [2023-12-26 23:43:56,134][105620] Updated weights for policy 1, policy_version 1148848 (0.0008) [2023-12-26 23:43:56,190][105620] Updated weights for policy 1, policy_version 1148858 (0.0008) [2023-12-26 23:43:56,236][105620] Updated weights for policy 1, policy_version 1148868 (0.0005) [2023-12-26 23:43:56,674][105692] Updated weights for policy 0, policy_version 1147565 (0.0008) [2023-12-26 23:43:56,738][105692] Updated weights for policy 0, policy_version 1147575 (0.0005) [2023-12-26 23:43:56,798][105692] Updated weights for policy 0, policy_version 1147585 (0.0006) [2023-12-26 23:43:56,997][105620] Updated weights for policy 1, policy_version 1148878 (0.0006) [2023-12-26 23:43:57,053][105620] Updated weights for policy 1, policy_version 1148888 (0.0005) [2023-12-26 23:43:57,121][105620] Updated weights for policy 1, policy_version 1148898 (0.0005) [2023-12-26 23:43:57,315][105692] Updated weights for policy 0, policy_version 1147595 (0.0005) [2023-12-26 23:43:57,362][105692] Updated weights for policy 0, policy_version 1147605 (0.0005) [2023-12-26 23:43:57,418][105692] Updated weights for policy 0, policy_version 1147615 (0.0005) [2023-12-26 23:43:57,680][105620] Updated weights for policy 1, policy_version 1148908 (0.0005) [2023-12-26 23:43:57,734][105620] Updated weights for policy 1, policy_version 1148918 (0.0005) [2023-12-26 23:43:57,786][105620] Updated weights for policy 1, policy_version 1148928 (0.0005) [2023-12-26 23:43:57,958][105692] Updated weights for policy 0, policy_version 1147625 (0.0006) [2023-12-26 23:43:58,012][105692] Updated weights for policy 0, policy_version 1147635 (0.0005) [2023-12-26 23:43:58,060][105692] Updated weights for policy 0, policy_version 1147645 (0.0005) [2023-12-26 23:43:58,105][105692] Updated weights for policy 0, policy_version 1147655 (0.0005) [2023-12-26 23:43:58,476][105620] Updated weights for policy 1, policy_version 1148938 (0.0006) [2023-12-26 23:43:58,547][105620] Updated weights for policy 1, policy_version 1148948 (0.0008) [2023-12-26 23:43:58,613][105620] Updated weights for policy 1, policy_version 1148958 (0.0008) [2023-12-26 23:43:58,676][105620] Updated weights for policy 1, policy_version 1148968 (0.0008) [2023-12-26 23:43:58,835][105692] Updated weights for policy 0, policy_version 1147665 (0.0008) [2023-12-26 23:43:58,895][105692] Updated weights for policy 0, policy_version 1147675 (0.0007) [2023-12-26 23:43:58,955][105692] Updated weights for policy 0, policy_version 1147685 (0.0006) [2023-12-26 23:43:59,449][105620] Updated weights for policy 1, policy_version 1148978 (0.0008) [2023-12-26 23:43:59,506][105620] Updated weights for policy 1, policy_version 1148988 (0.0008) [2023-12-26 23:43:59,571][105620] Updated weights for policy 1, policy_version 1148998 (0.0008) [2023-12-26 23:43:59,602][105692] Updated weights for policy 0, policy_version 1147695 (0.0008) [2023-12-26 23:43:59,650][105692] Updated weights for policy 0, policy_version 1147705 (0.0007) [2023-12-26 23:43:59,699][105692] Updated weights for policy 0, policy_version 1147715 (0.0007) [2023-12-26 23:44:00,253][105620] Updated weights for policy 1, policy_version 1149008 (0.0007) [2023-12-26 23:44:00,316][105620] Updated weights for policy 1, policy_version 1149018 (0.0006) [2023-12-26 23:44:00,377][105620] Updated weights for policy 1, policy_version 1149028 (0.0006) [2023-12-26 23:44:00,456][105692] Updated weights for policy 0, policy_version 1147725 (0.0009) [2023-12-26 23:44:00,517][105692] Updated weights for policy 0, policy_version 1147735 (0.0010) [2023-12-26 23:44:00,583][105692] Updated weights for policy 0, policy_version 1147745 (0.0011) [2023-12-26 23:44:00,954][105620] Updated weights for policy 1, policy_version 1149038 (0.0005) [2023-12-26 23:44:01,020][105620] Updated weights for policy 1, policy_version 1149048 (0.0008) [2023-12-26 23:44:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 588062720. Throughput: 0: 9686.6, 1: 9975.6. Samples: 588036404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:44:01,062][104569] Avg episode reward: [(0, '9165.296'), (1, '8904.568')] [2023-12-26 23:44:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001147752_293871616.pth... [2023-12-26 23:44:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001146632_293584896.pth [2023-12-26 23:44:01,075][105620] Updated weights for policy 1, policy_version 1149058 (0.0010) [2023-12-26 23:44:01,109][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001149064_294199296.pth... [2023-12-26 23:44:01,113][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001147912_293904384.pth [2023-12-26 23:44:01,309][105692] Updated weights for policy 0, policy_version 1147755 (0.0010) [2023-12-26 23:44:01,380][105692] Updated weights for policy 0, policy_version 1147765 (0.0009) [2023-12-26 23:44:01,431][105692] Updated weights for policy 0, policy_version 1147775 (0.0008) [2023-12-26 23:44:01,830][105620] Updated weights for policy 1, policy_version 1149068 (0.0009) [2023-12-26 23:44:01,897][105620] Updated weights for policy 1, policy_version 1149078 (0.0007) [2023-12-26 23:44:01,961][105620] Updated weights for policy 1, policy_version 1149088 (0.0010) [2023-12-26 23:44:02,064][105692] Updated weights for policy 0, policy_version 1147785 (0.0008) [2023-12-26 23:44:02,122][105692] Updated weights for policy 0, policy_version 1147795 (0.0005) [2023-12-26 23:44:02,175][105692] Updated weights for policy 0, policy_version 1147805 (0.0005) [2023-12-26 23:44:02,230][105692] Updated weights for policy 0, policy_version 1147815 (0.0008) [2023-12-26 23:44:02,716][105620] Updated weights for policy 1, policy_version 1149098 (0.0007) [2023-12-26 23:44:02,787][105620] Updated weights for policy 1, policy_version 1149108 (0.0007) [2023-12-26 23:44:02,851][105620] Updated weights for policy 1, policy_version 1149118 (0.0006) [2023-12-26 23:44:02,878][105692] Updated weights for policy 0, policy_version 1147825 (0.0007) [2023-12-26 23:44:02,910][105620] Updated weights for policy 1, policy_version 1149128 (0.0009) [2023-12-26 23:44:02,936][105692] Updated weights for policy 0, policy_version 1147835 (0.0009) [2023-12-26 23:44:02,999][105692] Updated weights for policy 0, policy_version 1147845 (0.0008) [2023-12-26 23:44:03,586][105620] Updated weights for policy 1, policy_version 1149138 (0.0008) [2023-12-26 23:44:03,627][105692] Updated weights for policy 0, policy_version 1147855 (0.0006) [2023-12-26 23:44:03,639][105620] Updated weights for policy 1, policy_version 1149148 (0.0006) [2023-12-26 23:44:03,693][105692] Updated weights for policy 0, policy_version 1147865 (0.0005) [2023-12-26 23:44:03,698][105620] Updated weights for policy 1, policy_version 1149158 (0.0006) [2023-12-26 23:44:03,765][105692] Updated weights for policy 0, policy_version 1147875 (0.0005) [2023-12-26 23:44:04,288][105620] Updated weights for policy 1, policy_version 1149168 (0.0010) [2023-12-26 23:44:04,350][105620] Updated weights for policy 1, policy_version 1149178 (0.0010) [2023-12-26 23:44:04,415][105620] Updated weights for policy 1, policy_version 1149188 (0.0008) [2023-12-26 23:44:04,434][105692] Updated weights for policy 0, policy_version 1147885 (0.0006) [2023-12-26 23:44:04,491][105692] Updated weights for policy 0, policy_version 1147895 (0.0008) [2023-12-26 23:44:04,547][105692] Updated weights for policy 0, policy_version 1147905 (0.0008) [2023-12-26 23:44:05,085][105620] Updated weights for policy 1, policy_version 1149198 (0.0007) [2023-12-26 23:44:05,131][105620] Updated weights for policy 1, policy_version 1149208 (0.0005) [2023-12-26 23:44:05,177][105620] Updated weights for policy 1, policy_version 1149218 (0.0005) [2023-12-26 23:44:05,276][105692] Updated weights for policy 0, policy_version 1147915 (0.0009) [2023-12-26 23:44:05,324][105692] Updated weights for policy 0, policy_version 1147925 (0.0010) [2023-12-26 23:44:05,368][105692] Updated weights for policy 0, policy_version 1147935 (0.0008) [2023-12-26 23:44:05,893][105620] Updated weights for policy 1, policy_version 1149228 (0.0007) [2023-12-26 23:44:05,961][105620] Updated weights for policy 1, policy_version 1149238 (0.0010) [2023-12-26 23:44:06,002][105692] Updated weights for policy 0, policy_version 1147945 (0.0005) [2023-12-26 23:44:06,020][105620] Updated weights for policy 1, policy_version 1149248 (0.0010) [2023-12-26 23:44:06,055][105692] Updated weights for policy 0, policy_version 1147955 (0.0006) [2023-12-26 23:44:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 588161024. Throughput: 0: 9640.6, 1: 9940.8. Samples: 588156916. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:44:06,062][104569] Avg episode reward: [(0, '9257.085'), (1, '8644.443')] [2023-12-26 23:44:06,105][105692] Updated weights for policy 0, policy_version 1147965 (0.0008) [2023-12-26 23:44:06,166][105692] Updated weights for policy 0, policy_version 1147975 (0.0008) [2023-12-26 23:44:06,723][105620] Updated weights for policy 1, policy_version 1149258 (0.0009) [2023-12-26 23:44:06,786][105620] Updated weights for policy 1, policy_version 1149268 (0.0005) [2023-12-26 23:44:06,844][105620] Updated weights for policy 1, policy_version 1149278 (0.0005) [2023-12-26 23:44:06,904][105620] Updated weights for policy 1, policy_version 1149288 (0.0009) [2023-12-26 23:44:06,910][105692] Updated weights for policy 0, policy_version 1147985 (0.0010) [2023-12-26 23:44:06,973][105692] Updated weights for policy 0, policy_version 1147995 (0.0011) [2023-12-26 23:44:07,036][105692] Updated weights for policy 0, policy_version 1148005 (0.0011) [2023-12-26 23:44:07,580][105620] Updated weights for policy 1, policy_version 1149298 (0.0011) [2023-12-26 23:44:07,643][105620] Updated weights for policy 1, policy_version 1149308 (0.0010) [2023-12-26 23:44:07,700][105620] Updated weights for policy 1, policy_version 1149318 (0.0007) [2023-12-26 23:44:07,762][105692] Updated weights for policy 0, policy_version 1148015 (0.0011) [2023-12-26 23:44:07,810][105692] Updated weights for policy 0, policy_version 1148025 (0.0010) [2023-12-26 23:44:07,861][105692] Updated weights for policy 0, policy_version 1148035 (0.0010) [2023-12-26 23:44:08,402][105620] Updated weights for policy 1, policy_version 1149328 (0.0009) [2023-12-26 23:44:08,464][105620] Updated weights for policy 1, policy_version 1149338 (0.0009) [2023-12-26 23:44:08,520][105692] Updated weights for policy 0, policy_version 1148045 (0.0010) [2023-12-26 23:44:08,526][105620] Updated weights for policy 1, policy_version 1149348 (0.0007) [2023-12-26 23:44:08,582][105692] Updated weights for policy 0, policy_version 1148055 (0.0010) [2023-12-26 23:44:08,647][105692] Updated weights for policy 0, policy_version 1148065 (0.0010) [2023-12-26 23:44:09,302][105620] Updated weights for policy 1, policy_version 1149358 (0.0007) [2023-12-26 23:44:09,367][105620] Updated weights for policy 1, policy_version 1149368 (0.0008) [2023-12-26 23:44:09,396][105692] Updated weights for policy 0, policy_version 1148075 (0.0009) [2023-12-26 23:44:09,438][105620] Updated weights for policy 1, policy_version 1149378 (0.0007) [2023-12-26 23:44:09,460][105692] Updated weights for policy 0, policy_version 1148085 (0.0008) [2023-12-26 23:44:09,525][105692] Updated weights for policy 0, policy_version 1148095 (0.0006) [2023-12-26 23:44:10,125][105620] Updated weights for policy 1, policy_version 1149388 (0.0008) [2023-12-26 23:44:10,190][105620] Updated weights for policy 1, policy_version 1149398 (0.0009) [2023-12-26 23:44:10,250][105620] Updated weights for policy 1, policy_version 1149408 (0.0009) [2023-12-26 23:44:10,294][105692] Updated weights for policy 0, policy_version 1148105 (0.0010) [2023-12-26 23:44:10,359][105692] Updated weights for policy 0, policy_version 1148115 (0.0010) [2023-12-26 23:44:10,419][105692] Updated weights for policy 0, policy_version 1148125 (0.0009) [2023-12-26 23:44:10,479][105692] Updated weights for policy 0, policy_version 1148135 (0.0006) [2023-12-26 23:44:10,976][105620] Updated weights for policy 1, policy_version 1149418 (0.0007) [2023-12-26 23:44:11,039][105620] Updated weights for policy 1, policy_version 1149428 (0.0007) [2023-12-26 23:44:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19660.8). Total num frames: 588259328. Throughput: 0: 9638.8, 1: 9889.7. Samples: 588274000. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:44:11,062][104569] Avg episode reward: [(0, '9350.027'), (1, '8477.504')] [2023-12-26 23:44:11,107][105620] Updated weights for policy 1, policy_version 1149438 (0.0008) [2023-12-26 23:44:11,179][105620] Updated weights for policy 1, policy_version 1149448 (0.0008) [2023-12-26 23:44:11,182][105692] Updated weights for policy 0, policy_version 1148145 (0.0008) [2023-12-26 23:44:11,245][105692] Updated weights for policy 0, policy_version 1148155 (0.0009) [2023-12-26 23:44:11,318][105692] Updated weights for policy 0, policy_version 1148165 (0.0009) [2023-12-26 23:44:11,822][105620] Updated weights for policy 1, policy_version 1149458 (0.0005) [2023-12-26 23:44:11,888][105620] Updated weights for policy 1, policy_version 1149468 (0.0006) [2023-12-26 23:44:11,951][105620] Updated weights for policy 1, policy_version 1149478 (0.0006) [2023-12-26 23:44:12,149][105692] Updated weights for policy 0, policy_version 1148175 (0.0010) [2023-12-26 23:44:12,207][105692] Updated weights for policy 0, policy_version 1148185 (0.0008) [2023-12-26 23:44:12,274][105692] Updated weights for policy 0, policy_version 1148195 (0.0007) [2023-12-26 23:44:12,603][105620] Updated weights for policy 1, policy_version 1149488 (0.0009) [2023-12-26 23:44:12,654][105620] Updated weights for policy 1, policy_version 1149498 (0.0009) [2023-12-26 23:44:12,712][105620] Updated weights for policy 1, policy_version 1149508 (0.0009) [2023-12-26 23:44:12,996][105692] Updated weights for policy 0, policy_version 1148205 (0.0009) [2023-12-26 23:44:13,043][105692] Updated weights for policy 0, policy_version 1148215 (0.0009) [2023-12-26 23:44:13,102][105692] Updated weights for policy 0, policy_version 1148225 (0.0009) [2023-12-26 23:44:13,511][105620] Updated weights for policy 1, policy_version 1149518 (0.0009) [2023-12-26 23:44:13,573][105620] Updated weights for policy 1, policy_version 1149528 (0.0009) [2023-12-26 23:44:13,635][105620] Updated weights for policy 1, policy_version 1149538 (0.0009) [2023-12-26 23:44:13,822][105692] Updated weights for policy 0, policy_version 1148235 (0.0009) [2023-12-26 23:44:13,884][105692] Updated weights for policy 0, policy_version 1148245 (0.0009) [2023-12-26 23:44:13,931][105692] Updated weights for policy 0, policy_version 1148255 (0.0009) [2023-12-26 23:44:14,406][105620] Updated weights for policy 1, policy_version 1149548 (0.0009) [2023-12-26 23:44:14,453][105620] Updated weights for policy 1, policy_version 1149558 (0.0007) [2023-12-26 23:44:14,503][105620] Updated weights for policy 1, policy_version 1149568 (0.0008) [2023-12-26 23:44:14,638][105692] Updated weights for policy 0, policy_version 1148265 (0.0008) [2023-12-26 23:44:14,703][105692] Updated weights for policy 0, policy_version 1148275 (0.0005) [2023-12-26 23:44:14,790][105692] Updated weights for policy 0, policy_version 1148285 (0.0008) [2023-12-26 23:44:14,846][105692] Updated weights for policy 0, policy_version 1148295 (0.0011) [2023-12-26 23:44:15,307][105620] Updated weights for policy 1, policy_version 1149578 (0.0009) [2023-12-26 23:44:15,363][105620] Updated weights for policy 1, policy_version 1149588 (0.0009) [2023-12-26 23:44:15,423][105620] Updated weights for policy 1, policy_version 1149598 (0.0008) [2023-12-26 23:44:15,474][105692] Updated weights for policy 0, policy_version 1148305 (0.0011) [2023-12-26 23:44:15,480][105620] Updated weights for policy 1, policy_version 1149608 (0.0006) [2023-12-26 23:44:15,525][105692] Updated weights for policy 0, policy_version 1148315 (0.0010) [2023-12-26 23:44:15,584][105692] Updated weights for policy 0, policy_version 1148325 (0.0006) [2023-12-26 23:44:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 588357632. Throughput: 0: 9568.4, 1: 9817.1. Samples: 588330192. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:44:16,062][104569] Avg episode reward: [(0, '9350.419'), (1, '5038.840')] [2023-12-26 23:44:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001148328_294019072.pth... [2023-12-26 23:44:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001149608_294338560.pth... [2023-12-26 23:44:16,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001147176_293724160.pth [2023-12-26 23:44:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001148488_294051840.pth [2023-12-26 23:44:16,175][105692] Updated weights for policy 0, policy_version 1148335 (0.0009) [2023-12-26 23:44:16,231][105692] Updated weights for policy 0, policy_version 1148345 (0.0011) [2023-12-26 23:44:16,300][105692] Updated weights for policy 0, policy_version 1148355 (0.0010) [2023-12-26 23:44:16,321][105620] Updated weights for policy 1, policy_version 1149618 (0.0005) [2023-12-26 23:44:16,375][105620] Updated weights for policy 1, policy_version 1149628 (0.0007) [2023-12-26 23:44:16,437][105620] Updated weights for policy 1, policy_version 1149638 (0.0008) [2023-12-26 23:44:17,032][105692] Updated weights for policy 0, policy_version 1148365 (0.0010) [2023-12-26 23:44:17,084][105692] Updated weights for policy 0, policy_version 1148375 (0.0009) [2023-12-26 23:44:17,142][105692] Updated weights for policy 0, policy_version 1148385 (0.0009) [2023-12-26 23:44:17,177][105620] Updated weights for policy 1, policy_version 1149648 (0.0008) [2023-12-26 23:44:17,223][105620] Updated weights for policy 1, policy_version 1149658 (0.0008) [2023-12-26 23:44:17,279][105620] Updated weights for policy 1, policy_version 1149668 (0.0010) [2023-12-26 23:44:17,803][105692] Updated weights for policy 0, policy_version 1148395 (0.0008) [2023-12-26 23:44:17,857][105692] Updated weights for policy 0, policy_version 1148405 (0.0007) [2023-12-26 23:44:17,913][105692] Updated weights for policy 0, policy_version 1148415 (0.0009) [2023-12-26 23:44:18,105][105620] Updated weights for policy 1, policy_version 1149678 (0.0009) [2023-12-26 23:44:18,161][105620] Updated weights for policy 1, policy_version 1149688 (0.0008) [2023-12-26 23:44:18,211][105620] Updated weights for policy 1, policy_version 1149698 (0.0009) [2023-12-26 23:44:18,614][105692] Updated weights for policy 0, policy_version 1148425 (0.0009) [2023-12-26 23:44:18,679][105692] Updated weights for policy 0, policy_version 1148435 (0.0007) [2023-12-26 23:44:18,739][105692] Updated weights for policy 0, policy_version 1148445 (0.0005) [2023-12-26 23:44:18,800][105692] Updated weights for policy 0, policy_version 1148455 (0.0009) [2023-12-26 23:44:19,020][105620] Updated weights for policy 1, policy_version 1149708 (0.0008) [2023-12-26 23:44:19,072][105620] Updated weights for policy 1, policy_version 1149718 (0.0008) [2023-12-26 23:44:19,122][105620] Updated weights for policy 1, policy_version 1149728 (0.0009) [2023-12-26 23:44:19,525][105692] Updated weights for policy 0, policy_version 1148465 (0.0008) [2023-12-26 23:44:19,575][105692] Updated weights for policy 0, policy_version 1148475 (0.0008) [2023-12-26 23:44:19,623][105692] Updated weights for policy 0, policy_version 1148485 (0.0008) [2023-12-26 23:44:19,897][105620] Updated weights for policy 1, policy_version 1149738 (0.0008) [2023-12-26 23:44:19,966][105620] Updated weights for policy 1, policy_version 1149748 (0.0008) [2023-12-26 23:44:20,031][105620] Updated weights for policy 1, policy_version 1149758 (0.0007) [2023-12-26 23:44:20,090][105620] Updated weights for policy 1, policy_version 1149768 (0.0008) [2023-12-26 23:44:20,341][105692] Updated weights for policy 0, policy_version 1148495 (0.0008) [2023-12-26 23:44:20,399][105692] Updated weights for policy 0, policy_version 1148505 (0.0005) [2023-12-26 23:44:20,453][105692] Updated weights for policy 0, policy_version 1148515 (0.0008) [2023-12-26 23:44:20,833][105620] Updated weights for policy 1, policy_version 1149778 (0.0008) [2023-12-26 23:44:20,899][105620] Updated weights for policy 1, policy_version 1149788 (0.0006) [2023-12-26 23:44:20,964][105620] Updated weights for policy 1, policy_version 1149798 (0.0008) [2023-12-26 23:44:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 588455936. Throughput: 0: 9712.3, 1: 9694.0. Samples: 588444640. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:44:21,062][104569] Avg episode reward: [(0, '9263.760'), (1, '4332.043')] [2023-12-26 23:44:21,242][105692] Updated weights for policy 0, policy_version 1148525 (0.0008) [2023-12-26 23:44:21,310][105692] Updated weights for policy 0, policy_version 1148535 (0.0006) [2023-12-26 23:44:21,383][105692] Updated weights for policy 0, policy_version 1148545 (0.0008) [2023-12-26 23:44:21,699][105620] Updated weights for policy 1, policy_version 1149808 (0.0010) [2023-12-26 23:44:21,771][105620] Updated weights for policy 1, policy_version 1149818 (0.0011) [2023-12-26 23:44:21,820][105620] Updated weights for policy 1, policy_version 1149828 (0.0010) [2023-12-26 23:44:22,156][105692] Updated weights for policy 0, policy_version 1148555 (0.0007) [2023-12-26 23:44:22,207][105692] Updated weights for policy 0, policy_version 1148565 (0.0008) [2023-12-26 23:44:22,269][105692] Updated weights for policy 0, policy_version 1148575 (0.0010) [2023-12-26 23:44:22,588][105620] Updated weights for policy 1, policy_version 1149838 (0.0009) [2023-12-26 23:44:22,654][105620] Updated weights for policy 1, policy_version 1149848 (0.0009) [2023-12-26 23:44:22,717][105620] Updated weights for policy 1, policy_version 1149858 (0.0009) [2023-12-26 23:44:23,031][105692] Updated weights for policy 0, policy_version 1148585 (0.0006) [2023-12-26 23:44:23,086][105692] Updated weights for policy 0, policy_version 1148595 (0.0009) [2023-12-26 23:44:23,145][105692] Updated weights for policy 0, policy_version 1148605 (0.0009) [2023-12-26 23:44:23,206][105692] Updated weights for policy 0, policy_version 1148615 (0.0007) [2023-12-26 23:44:23,475][105620] Updated weights for policy 1, policy_version 1149868 (0.0009) [2023-12-26 23:44:23,532][105620] Updated weights for policy 1, policy_version 1149878 (0.0012) [2023-12-26 23:44:23,586][105620] Updated weights for policy 1, policy_version 1149888 (0.0010) [2023-12-26 23:44:23,858][105692] Updated weights for policy 0, policy_version 1148625 (0.0006) [2023-12-26 23:44:23,926][105692] Updated weights for policy 0, policy_version 1148635 (0.0005) [2023-12-26 23:44:23,998][105692] Updated weights for policy 0, policy_version 1148645 (0.0005) [2023-12-26 23:44:24,324][105620] Updated weights for policy 1, policy_version 1149898 (0.0009) [2023-12-26 23:44:24,395][105620] Updated weights for policy 1, policy_version 1149908 (0.0008) [2023-12-26 23:44:24,461][105620] Updated weights for policy 1, policy_version 1149918 (0.0008) [2023-12-26 23:44:24,532][105620] Updated weights for policy 1, policy_version 1149928 (0.0008) [2023-12-26 23:44:24,693][105692] Updated weights for policy 0, policy_version 1148655 (0.0008) [2023-12-26 23:44:24,750][105692] Updated weights for policy 0, policy_version 1148665 (0.0009) [2023-12-26 23:44:24,808][105692] Updated weights for policy 0, policy_version 1148675 (0.0009) [2023-12-26 23:44:25,077][105620] Updated weights for policy 1, policy_version 1149938 (0.0005) [2023-12-26 23:44:25,128][105620] Updated weights for policy 1, policy_version 1149948 (0.0005) [2023-12-26 23:44:25,181][105620] Updated weights for policy 1, policy_version 1149958 (0.0005) [2023-12-26 23:44:25,666][105692] Updated weights for policy 0, policy_version 1148685 (0.0009) [2023-12-26 23:44:25,702][105620] Updated weights for policy 1, policy_version 1149968 (0.0006) [2023-12-26 23:44:25,733][105692] Updated weights for policy 0, policy_version 1148695 (0.0005) [2023-12-26 23:44:25,759][105620] Updated weights for policy 1, policy_version 1149978 (0.0006) [2023-12-26 23:44:25,797][105692] Updated weights for policy 0, policy_version 1148705 (0.0005) [2023-12-26 23:44:25,811][105620] Updated weights for policy 1, policy_version 1149988 (0.0006) [2023-12-26 23:44:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19660.8). Total num frames: 588554240. Throughput: 0: 9699.9, 1: 9700.7. Samples: 588560556. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:44:26,062][104569] Avg episode reward: [(0, '9086.678'), (1, '7098.757')] [2023-12-26 23:44:26,328][105692] Updated weights for policy 0, policy_version 1148715 (0.0007) [2023-12-26 23:44:26,357][105620] Updated weights for policy 1, policy_version 1149998 (0.0005) [2023-12-26 23:44:26,384][105692] Updated weights for policy 0, policy_version 1148725 (0.0009) [2023-12-26 23:44:26,402][105620] Updated weights for policy 1, policy_version 1150008 (0.0005) [2023-12-26 23:44:26,433][105692] Updated weights for policy 0, policy_version 1148735 (0.0009) [2023-12-26 23:44:26,453][105620] Updated weights for policy 1, policy_version 1150018 (0.0005) [2023-12-26 23:44:27,091][105620] Updated weights for policy 1, policy_version 1150028 (0.0008) [2023-12-26 23:44:27,111][105692] Updated weights for policy 0, policy_version 1148745 (0.0009) [2023-12-26 23:44:27,146][105620] Updated weights for policy 1, policy_version 1150038 (0.0010) [2023-12-26 23:44:27,165][105692] Updated weights for policy 0, policy_version 1148755 (0.0005) [2023-12-26 23:44:27,200][105620] Updated weights for policy 1, policy_version 1150048 (0.0010) [2023-12-26 23:44:27,215][105692] Updated weights for policy 0, policy_version 1148765 (0.0005) [2023-12-26 23:44:27,279][105692] Updated weights for policy 0, policy_version 1148775 (0.0005) [2023-12-26 23:44:27,849][105620] Updated weights for policy 1, policy_version 1150058 (0.0010) [2023-12-26 23:44:27,903][105620] Updated weights for policy 1, policy_version 1150068 (0.0010) [2023-12-26 23:44:27,966][105620] Updated weights for policy 1, policy_version 1150078 (0.0008) [2023-12-26 23:44:27,996][105692] Updated weights for policy 0, policy_version 1148785 (0.0007) [2023-12-26 23:44:28,026][105620] Updated weights for policy 1, policy_version 1150088 (0.0008) [2023-12-26 23:44:28,053][105692] Updated weights for policy 0, policy_version 1148795 (0.0005) [2023-12-26 23:44:28,109][105692] Updated weights for policy 0, policy_version 1148805 (0.0005) [2023-12-26 23:44:28,697][105692] Updated weights for policy 0, policy_version 1148815 (0.0008) [2023-12-26 23:44:28,744][105620] Updated weights for policy 1, policy_version 1150098 (0.0010) [2023-12-26 23:44:28,745][105692] Updated weights for policy 0, policy_version 1148825 (0.0009) [2023-12-26 23:44:28,787][105692] Updated weights for policy 0, policy_version 1148835 (0.0007) [2023-12-26 23:44:28,807][105620] Updated weights for policy 1, policy_version 1150108 (0.0010) [2023-12-26 23:44:28,868][105620] Updated weights for policy 1, policy_version 1150118 (0.0010) [2023-12-26 23:44:29,547][105692] Updated weights for policy 0, policy_version 1148845 (0.0007) [2023-12-26 23:44:29,599][105692] Updated weights for policy 0, policy_version 1148855 (0.0008) [2023-12-26 23:44:29,616][105620] Updated weights for policy 1, policy_version 1150128 (0.0010) [2023-12-26 23:44:29,662][105692] Updated weights for policy 0, policy_version 1148865 (0.0006) [2023-12-26 23:44:29,680][105620] Updated weights for policy 1, policy_version 1150138 (0.0010) [2023-12-26 23:44:29,742][105620] Updated weights for policy 1, policy_version 1150148 (0.0007) [2023-12-26 23:44:30,335][105692] Updated weights for policy 0, policy_version 1148875 (0.0007) [2023-12-26 23:44:30,392][105692] Updated weights for policy 0, policy_version 1148885 (0.0007) [2023-12-26 23:44:30,405][105620] Updated weights for policy 1, policy_version 1150158 (0.0008) [2023-12-26 23:44:30,453][105620] Updated weights for policy 1, policy_version 1150168 (0.0010) [2023-12-26 23:44:30,453][105692] Updated weights for policy 0, policy_version 1148895 (0.0005) [2023-12-26 23:44:30,505][105620] Updated weights for policy 1, policy_version 1150178 (0.0010) [2023-12-26 23:44:31,005][105692] Updated weights for policy 0, policy_version 1148905 (0.0005) [2023-12-26 23:44:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 588652544. Throughput: 0: 9819.9, 1: 9776.1. Samples: 588624600. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:44:31,062][104569] Avg episode reward: [(0, '8994.395'), (1, '2461.113')] [2023-12-26 23:44:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001150184_294486016.pth... [2023-12-26 23:44:31,073][105692] Updated weights for policy 0, policy_version 1148915 (0.0006) [2023-12-26 23:44:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001149064_294199296.pth [2023-12-26 23:44:31,141][105692] Updated weights for policy 0, policy_version 1148925 (0.0009) [2023-12-26 23:44:31,200][105692] Updated weights for policy 0, policy_version 1148935 (0.0008) [2023-12-26 23:44:31,207][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001148936_294174720.pth... [2023-12-26 23:44:31,211][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001147752_293871616.pth [2023-12-26 23:44:31,274][105620] Updated weights for policy 1, policy_version 1150188 (0.0011) [2023-12-26 23:44:31,337][105620] Updated weights for policy 1, policy_version 1150198 (0.0010) [2023-12-26 23:44:31,400][105620] Updated weights for policy 1, policy_version 1150208 (0.0008) [2023-12-26 23:44:31,874][105692] Updated weights for policy 0, policy_version 1148945 (0.0009) [2023-12-26 23:44:31,934][105692] Updated weights for policy 0, policy_version 1148955 (0.0009) [2023-12-26 23:44:32,000][105692] Updated weights for policy 0, policy_version 1148965 (0.0006) [2023-12-26 23:44:32,011][105620] Updated weights for policy 1, policy_version 1150218 (0.0006) [2023-12-26 23:44:32,078][105620] Updated weights for policy 1, policy_version 1150228 (0.0007) [2023-12-26 23:44:32,141][105620] Updated weights for policy 1, policy_version 1150238 (0.0010) [2023-12-26 23:44:32,203][105620] Updated weights for policy 1, policy_version 1150248 (0.0010) [2023-12-26 23:44:32,631][105692] Updated weights for policy 0, policy_version 1148975 (0.0005) [2023-12-26 23:44:32,696][105692] Updated weights for policy 0, policy_version 1148985 (0.0008) [2023-12-26 23:44:32,753][105692] Updated weights for policy 0, policy_version 1148995 (0.0009) [2023-12-26 23:44:32,951][105620] Updated weights for policy 1, policy_version 1150258 (0.0009) [2023-12-26 23:44:33,003][105620] Updated weights for policy 1, policy_version 1150268 (0.0010) [2023-12-26 23:44:33,051][105620] Updated weights for policy 1, policy_version 1150278 (0.0010) [2023-12-26 23:44:33,462][105692] Updated weights for policy 0, policy_version 1149005 (0.0008) [2023-12-26 23:44:33,533][105692] Updated weights for policy 0, policy_version 1149015 (0.0005) [2023-12-26 23:44:33,593][105692] Updated weights for policy 0, policy_version 1149025 (0.0005) [2023-12-26 23:44:33,821][105620] Updated weights for policy 1, policy_version 1150288 (0.0009) [2023-12-26 23:44:33,871][105620] Updated weights for policy 1, policy_version 1150299 (0.0009) [2023-12-26 23:44:33,917][105620] Updated weights for policy 1, policy_version 1150309 (0.0008) [2023-12-26 23:44:34,223][105692] Updated weights for policy 0, policy_version 1149035 (0.0007) [2023-12-26 23:44:34,287][105692] Updated weights for policy 0, policy_version 1149045 (0.0007) [2023-12-26 23:44:34,344][105692] Updated weights for policy 0, policy_version 1149055 (0.0009) [2023-12-26 23:44:34,682][105620] Updated weights for policy 1, policy_version 1150319 (0.0006) [2023-12-26 23:44:34,746][105620] Updated weights for policy 1, policy_version 1150329 (0.0005) [2023-12-26 23:44:34,813][105620] Updated weights for policy 1, policy_version 1150339 (0.0005) [2023-12-26 23:44:35,041][105692] Updated weights for policy 0, policy_version 1149065 (0.0009) [2023-12-26 23:44:35,101][105692] Updated weights for policy 0, policy_version 1149075 (0.0009) [2023-12-26 23:44:35,155][105692] Updated weights for policy 0, policy_version 1149085 (0.0010) [2023-12-26 23:44:35,214][105692] Updated weights for policy 0, policy_version 1149096 (0.0010) [2023-12-26 23:44:35,371][105620] Updated weights for policy 1, policy_version 1150349 (0.0006) [2023-12-26 23:44:35,433][105620] Updated weights for policy 1, policy_version 1150359 (0.0005) [2023-12-26 23:44:35,490][105620] Updated weights for policy 1, policy_version 1150369 (0.0006) [2023-12-26 23:44:36,001][105620] Updated weights for policy 1, policy_version 1150379 (0.0007) [2023-12-26 23:44:36,059][105620] Updated weights for policy 1, policy_version 1150389 (0.0010) [2023-12-26 23:44:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 588750848. Throughput: 0: 9950.3, 1: 9708.9. Samples: 588745476. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:44:36,062][104569] Avg episode reward: [(0, '8990.669'), (1, '3753.849')] [2023-12-26 23:44:36,112][105692] Updated weights for policy 0, policy_version 1149106 (0.0007) [2023-12-26 23:44:36,124][105620] Updated weights for policy 1, policy_version 1150399 (0.0009) [2023-12-26 23:44:36,169][105692] Updated weights for policy 0, policy_version 1149116 (0.0006) [2023-12-26 23:44:36,225][105692] Updated weights for policy 0, policy_version 1149126 (0.0009) [2023-12-26 23:44:36,904][105620] Updated weights for policy 1, policy_version 1150409 (0.0010) [2023-12-26 23:44:36,967][105620] Updated weights for policy 1, policy_version 1150419 (0.0011) [2023-12-26 23:44:36,998][105692] Updated weights for policy 0, policy_version 1149136 (0.0006) [2023-12-26 23:44:37,020][105620] Updated weights for policy 1, policy_version 1150429 (0.0011) [2023-12-26 23:44:37,058][105692] Updated weights for policy 0, policy_version 1149146 (0.0006) [2023-12-26 23:44:37,076][105620] Updated weights for policy 1, policy_version 1150439 (0.0011) [2023-12-26 23:44:37,115][105692] Updated weights for policy 0, policy_version 1149156 (0.0007) [2023-12-26 23:44:37,795][105692] Updated weights for policy 0, policy_version 1149166 (0.0007) [2023-12-26 23:44:37,813][105620] Updated weights for policy 1, policy_version 1150449 (0.0011) [2023-12-26 23:44:37,853][105692] Updated weights for policy 0, policy_version 1149176 (0.0006) [2023-12-26 23:44:37,873][105620] Updated weights for policy 1, policy_version 1150459 (0.0011) [2023-12-26 23:44:37,915][105692] Updated weights for policy 0, policy_version 1149186 (0.0010) [2023-12-26 23:44:37,930][105620] Updated weights for policy 1, policy_version 1150469 (0.0011) [2023-12-26 23:44:38,527][105692] Updated weights for policy 0, policy_version 1149196 (0.0009) [2023-12-26 23:44:38,561][105620] Updated weights for policy 1, policy_version 1150479 (0.0008) [2023-12-26 23:44:38,595][105692] Updated weights for policy 0, policy_version 1149206 (0.0005) [2023-12-26 23:44:38,617][105620] Updated weights for policy 1, policy_version 1150489 (0.0007) [2023-12-26 23:44:38,659][105692] Updated weights for policy 0, policy_version 1149216 (0.0005) [2023-12-26 23:44:38,677][105620] Updated weights for policy 1, policy_version 1150499 (0.0011) [2023-12-26 23:44:39,285][105692] Updated weights for policy 0, policy_version 1149226 (0.0007) [2023-12-26 23:44:39,341][105692] Updated weights for policy 0, policy_version 1149236 (0.0007) [2023-12-26 23:44:39,408][105692] Updated weights for policy 0, policy_version 1149246 (0.0008) [2023-12-26 23:44:39,430][105620] Updated weights for policy 1, policy_version 1150509 (0.0009) [2023-12-26 23:44:39,473][105692] Updated weights for policy 0, policy_version 1149256 (0.0006) [2023-12-26 23:44:39,500][105620] Updated weights for policy 1, policy_version 1150519 (0.0006) [2023-12-26 23:44:39,563][105620] Updated weights for policy 1, policy_version 1150529 (0.0006) [2023-12-26 23:44:40,235][105692] Updated weights for policy 0, policy_version 1149266 (0.0009) [2023-12-26 23:44:40,260][105620] Updated weights for policy 1, policy_version 1150539 (0.0009) [2023-12-26 23:44:40,302][105692] Updated weights for policy 0, policy_version 1149276 (0.0009) [2023-12-26 23:44:40,321][105620] Updated weights for policy 1, policy_version 1150549 (0.0006) [2023-12-26 23:44:40,369][105692] Updated weights for policy 0, policy_version 1149286 (0.0008) [2023-12-26 23:44:40,385][105620] Updated weights for policy 1, policy_version 1150559 (0.0008) [2023-12-26 23:44:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 588849152. Throughput: 0: 9973.2, 1: 9811.4. Samples: 588863236. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:44:41,062][104569] Avg episode reward: [(0, '8903.626'), (1, '6966.754')] [2023-12-26 23:44:41,104][105692] Updated weights for policy 0, policy_version 1149296 (0.0006) [2023-12-26 23:44:41,168][105692] Updated weights for policy 0, policy_version 1149306 (0.0008) [2023-12-26 23:44:41,172][105620] Updated weights for policy 1, policy_version 1150569 (0.0011) [2023-12-26 23:44:41,216][105692] Updated weights for policy 0, policy_version 1149316 (0.0009) [2023-12-26 23:44:41,230][105620] Updated weights for policy 1, policy_version 1150579 (0.0007) [2023-12-26 23:44:41,290][105620] Updated weights for policy 1, policy_version 1150589 (0.0008) [2023-12-26 23:44:41,351][105620] Updated weights for policy 1, policy_version 1150599 (0.0008) [2023-12-26 23:44:41,999][105692] Updated weights for policy 0, policy_version 1149326 (0.0011) [2023-12-26 23:44:42,052][105692] Updated weights for policy 0, policy_version 1149336 (0.0011) [2023-12-26 23:44:42,112][105692] Updated weights for policy 0, policy_version 1149346 (0.0011) [2023-12-26 23:44:42,142][105620] Updated weights for policy 1, policy_version 1150609 (0.0006) [2023-12-26 23:44:42,210][105620] Updated weights for policy 1, policy_version 1150619 (0.0008) [2023-12-26 23:44:42,275][105620] Updated weights for policy 1, policy_version 1150629 (0.0008) [2023-12-26 23:44:42,910][105692] Updated weights for policy 0, policy_version 1149356 (0.0009) [2023-12-26 23:44:42,970][105692] Updated weights for policy 0, policy_version 1149366 (0.0006) [2023-12-26 23:44:43,004][105620] Updated weights for policy 1, policy_version 1150639 (0.0006) [2023-12-26 23:44:43,036][105692] Updated weights for policy 0, policy_version 1149376 (0.0011) [2023-12-26 23:44:43,054][105620] Updated weights for policy 1, policy_version 1150649 (0.0007) [2023-12-26 23:44:43,104][105620] Updated weights for policy 1, policy_version 1150659 (0.0006) [2023-12-26 23:44:43,746][105620] Updated weights for policy 1, policy_version 1150669 (0.0005) [2023-12-26 23:44:43,748][105692] Updated weights for policy 0, policy_version 1149386 (0.0011) [2023-12-26 23:44:43,800][105692] Updated weights for policy 0, policy_version 1149396 (0.0010) [2023-12-26 23:44:43,806][105620] Updated weights for policy 1, policy_version 1150679 (0.0006) [2023-12-26 23:44:43,849][105692] Updated weights for policy 0, policy_version 1149406 (0.0010) [2023-12-26 23:44:43,863][105620] Updated weights for policy 1, policy_version 1150689 (0.0008) [2023-12-26 23:44:43,908][105692] Updated weights for policy 0, policy_version 1149416 (0.0010) [2023-12-26 23:44:44,625][105620] Updated weights for policy 1, policy_version 1150699 (0.0005) [2023-12-26 23:44:44,643][105692] Updated weights for policy 0, policy_version 1149426 (0.0009) [2023-12-26 23:44:44,685][105620] Updated weights for policy 1, policy_version 1150709 (0.0008) [2023-12-26 23:44:44,689][105692] Updated weights for policy 0, policy_version 1149436 (0.0006) [2023-12-26 23:44:44,741][105692] Updated weights for policy 0, policy_version 1149446 (0.0005) [2023-12-26 23:44:44,742][105620] Updated weights for policy 1, policy_version 1150719 (0.0009) [2023-12-26 23:44:45,485][105692] Updated weights for policy 0, policy_version 1149456 (0.0008) [2023-12-26 23:44:45,530][105620] Updated weights for policy 1, policy_version 1150729 (0.0008) [2023-12-26 23:44:45,549][105692] Updated weights for policy 0, policy_version 1149466 (0.0009) [2023-12-26 23:44:45,592][105620] Updated weights for policy 1, policy_version 1150739 (0.0009) [2023-12-26 23:44:45,610][105692] Updated weights for policy 0, policy_version 1149476 (0.0008) [2023-12-26 23:44:45,651][105620] Updated weights for policy 1, policy_version 1150749 (0.0009) [2023-12-26 23:44:45,718][105620] Updated weights for policy 1, policy_version 1150759 (0.0009) [2023-12-26 23:44:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 588947456. Throughput: 0: 9849.2, 1: 9780.2. Samples: 588919724. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:44:46,063][104569] Avg episode reward: [(0, '9084.683'), (1, '8715.945')] [2023-12-26 23:44:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001149480_294313984.pth... [2023-12-26 23:44:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001150760_294633472.pth... [2023-12-26 23:44:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001149608_294338560.pth [2023-12-26 23:44:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001148328_294019072.pth [2023-12-26 23:44:46,351][105692] Updated weights for policy 0, policy_version 1149486 (0.0010) [2023-12-26 23:44:46,404][105692] Updated weights for policy 0, policy_version 1149496 (0.0008) [2023-12-26 23:44:46,463][105692] Updated weights for policy 0, policy_version 1149506 (0.0008) [2023-12-26 23:44:46,466][105620] Updated weights for policy 1, policy_version 1150769 (0.0008) [2023-12-26 23:44:46,523][105620] Updated weights for policy 1, policy_version 1150779 (0.0007) [2023-12-26 23:44:46,581][105620] Updated weights for policy 1, policy_version 1150789 (0.0009) [2023-12-26 23:44:47,219][105692] Updated weights for policy 0, policy_version 1149516 (0.0009) [2023-12-26 23:44:47,269][105692] Updated weights for policy 0, policy_version 1149526 (0.0009) [2023-12-26 23:44:47,317][105692] Updated weights for policy 0, policy_version 1149536 (0.0007) [2023-12-26 23:44:47,319][105620] Updated weights for policy 1, policy_version 1150799 (0.0007) [2023-12-26 23:44:47,372][105620] Updated weights for policy 1, policy_version 1150809 (0.0005) [2023-12-26 23:44:47,419][105620] Updated weights for policy 1, policy_version 1150819 (0.0008) [2023-12-26 23:44:48,021][105692] Updated weights for policy 0, policy_version 1149546 (0.0009) [2023-12-26 23:44:48,079][105692] Updated weights for policy 0, policy_version 1149556 (0.0010) [2023-12-26 23:44:48,120][105620] Updated weights for policy 1, policy_version 1150829 (0.0007) [2023-12-26 23:44:48,135][105692] Updated weights for policy 0, policy_version 1149566 (0.0009) [2023-12-26 23:44:48,170][105620] Updated weights for policy 1, policy_version 1150839 (0.0005) [2023-12-26 23:44:48,191][105692] Updated weights for policy 0, policy_version 1149576 (0.0008) [2023-12-26 23:44:48,234][105620] Updated weights for policy 1, policy_version 1150849 (0.0005) [2023-12-26 23:44:48,839][105620] Updated weights for policy 1, policy_version 1150859 (0.0006) [2023-12-26 23:44:48,893][105620] Updated weights for policy 1, policy_version 1150869 (0.0009) [2023-12-26 23:44:48,952][105620] Updated weights for policy 1, policy_version 1150879 (0.0009) [2023-12-26 23:44:49,023][105692] Updated weights for policy 0, policy_version 1149586 (0.0007) [2023-12-26 23:44:49,080][105692] Updated weights for policy 0, policy_version 1149596 (0.0009) [2023-12-26 23:44:49,137][105692] Updated weights for policy 0, policy_version 1149606 (0.0008) [2023-12-26 23:44:49,753][105620] Updated weights for policy 1, policy_version 1150889 (0.0008) [2023-12-26 23:44:49,815][105620] Updated weights for policy 1, policy_version 1150899 (0.0008) [2023-12-26 23:44:49,846][105692] Updated weights for policy 0, policy_version 1149616 (0.0008) [2023-12-26 23:44:49,876][105620] Updated weights for policy 1, policy_version 1150909 (0.0008) [2023-12-26 23:44:49,909][105692] Updated weights for policy 0, policy_version 1149626 (0.0006) [2023-12-26 23:44:49,931][105620] Updated weights for policy 1, policy_version 1150919 (0.0007) [2023-12-26 23:44:49,981][105692] Updated weights for policy 0, policy_version 1149636 (0.0008) [2023-12-26 23:44:50,620][105692] Updated weights for policy 0, policy_version 1149646 (0.0008) [2023-12-26 23:44:50,671][105692] Updated weights for policy 0, policy_version 1149656 (0.0009) [2023-12-26 23:44:50,731][105692] Updated weights for policy 0, policy_version 1149666 (0.0009) [2023-12-26 23:44:50,754][105620] Updated weights for policy 1, policy_version 1150929 (0.0007) [2023-12-26 23:44:50,811][105620] Updated weights for policy 1, policy_version 1150939 (0.0008) [2023-12-26 23:44:50,859][105620] Updated weights for policy 1, policy_version 1150949 (0.0008) [2023-12-26 23:44:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 589045760. Throughput: 0: 9764.7, 1: 9708.4. Samples: 589033208. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:44:51,062][104569] Avg episode reward: [(0, '8990.850'), (1, '9257.650')] [2023-12-26 23:44:51,449][105692] Updated weights for policy 0, policy_version 1149676 (0.0007) [2023-12-26 23:44:51,505][105692] Updated weights for policy 0, policy_version 1149686 (0.0007) [2023-12-26 23:44:51,514][105585] KL-divergence is very high: 158.7538 [2023-12-26 23:44:51,522][105585] KL-divergence is very high: 104.6332 [2023-12-26 23:44:51,569][105585] KL-divergence is very high: 172.9992 [2023-12-26 23:44:51,574][105692] Updated weights for policy 0, policy_version 1149696 (0.0006) [2023-12-26 23:44:51,576][105585] KL-divergence is very high: 112.7352 [2023-12-26 23:44:51,626][105585] KL-divergence is very high: 142.5730 [2023-12-26 23:44:51,675][105620] Updated weights for policy 1, policy_version 1150959 (0.0008) [2023-12-26 23:44:51,745][105620] Updated weights for policy 1, policy_version 1150969 (0.0009) [2023-12-26 23:44:51,808][105620] Updated weights for policy 1, policy_version 1150979 (0.0008) [2023-12-26 23:44:52,252][105692] Updated weights for policy 0, policy_version 1149706 (0.0007) [2023-12-26 23:44:52,313][105692] Updated weights for policy 0, policy_version 1149716 (0.0009) [2023-12-26 23:44:52,371][105692] Updated weights for policy 0, policy_version 1149726 (0.0009) [2023-12-26 23:44:52,433][105692] Updated weights for policy 0, policy_version 1149736 (0.0008) [2023-12-26 23:44:52,652][105620] Updated weights for policy 1, policy_version 1150989 (0.0009) [2023-12-26 23:44:52,712][105620] Updated weights for policy 1, policy_version 1151000 (0.0010) [2023-12-26 23:44:52,773][105620] Updated weights for policy 1, policy_version 1151010 (0.0009) [2023-12-26 23:44:53,048][105692] Updated weights for policy 0, policy_version 1149746 (0.0005) [2023-12-26 23:44:53,107][105692] Updated weights for policy 0, policy_version 1149756 (0.0005) [2023-12-26 23:44:53,165][105692] Updated weights for policy 0, policy_version 1149766 (0.0006) [2023-12-26 23:44:53,594][105620] Updated weights for policy 1, policy_version 1151020 (0.0010) [2023-12-26 23:44:53,649][105620] Updated weights for policy 1, policy_version 1151030 (0.0009) [2023-12-26 23:44:53,705][105620] Updated weights for policy 1, policy_version 1151040 (0.0010) [2023-12-26 23:44:53,798][105692] Updated weights for policy 0, policy_version 1149776 (0.0005) [2023-12-26 23:44:53,865][105692] Updated weights for policy 0, policy_version 1149786 (0.0005) [2023-12-26 23:44:53,932][105692] Updated weights for policy 0, policy_version 1149796 (0.0005) [2023-12-26 23:44:54,488][105692] Updated weights for policy 0, policy_version 1149806 (0.0007) [2023-12-26 23:44:54,508][105620] Updated weights for policy 1, policy_version 1151050 (0.0009) [2023-12-26 23:44:54,538][105692] Updated weights for policy 0, policy_version 1149816 (0.0006) [2023-12-26 23:44:54,560][105620] Updated weights for policy 1, policy_version 1151060 (0.0010) [2023-12-26 23:44:54,583][105692] Updated weights for policy 0, policy_version 1149826 (0.0006) [2023-12-26 23:44:54,605][105620] Updated weights for policy 1, policy_version 1151070 (0.0010) [2023-12-26 23:44:54,650][105620] Updated weights for policy 1, policy_version 1151080 (0.0010) [2023-12-26 23:44:55,350][105692] Updated weights for policy 0, policy_version 1149836 (0.0007) [2023-12-26 23:44:55,395][105692] Updated weights for policy 0, policy_version 1149846 (0.0006) [2023-12-26 23:44:55,419][105620] Updated weights for policy 1, policy_version 1151090 (0.0010) [2023-12-26 23:44:55,453][105692] Updated weights for policy 0, policy_version 1149856 (0.0005) [2023-12-26 23:44:55,476][105620] Updated weights for policy 1, policy_version 1151100 (0.0010) [2023-12-26 23:44:55,539][105620] Updated weights for policy 1, policy_version 1151110 (0.0011) [2023-12-26 23:44:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 589135872. Throughput: 0: 9832.9, 1: 9588.5. Samples: 589147960. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:44:56,063][104569] Avg episode reward: [(0, '8901.333'), (1, '9075.207')] [2023-12-26 23:44:56,208][105692] Updated weights for policy 0, policy_version 1149866 (0.0007) [2023-12-26 23:44:56,264][105692] Updated weights for policy 0, policy_version 1149876 (0.0010) [2023-12-26 23:44:56,271][105620] Updated weights for policy 1, policy_version 1151120 (0.0006) [2023-12-26 23:44:56,326][105692] Updated weights for policy 0, policy_version 1149886 (0.0006) [2023-12-26 23:44:56,331][105620] Updated weights for policy 1, policy_version 1151130 (0.0011) [2023-12-26 23:44:56,381][105620] Updated weights for policy 1, policy_version 1151140 (0.0008) [2023-12-26 23:44:56,384][105692] Updated weights for policy 0, policy_version 1149896 (0.0007) [2023-12-26 23:44:56,925][105620] Updated weights for policy 1, policy_version 1151150 (0.0009) [2023-12-26 23:44:56,973][105620] Updated weights for policy 1, policy_version 1151160 (0.0010) [2023-12-26 23:44:57,021][105620] Updated weights for policy 1, policy_version 1151170 (0.0010) [2023-12-26 23:44:57,105][105692] Updated weights for policy 0, policy_version 1149906 (0.0006) [2023-12-26 23:44:57,156][105692] Updated weights for policy 0, policy_version 1149916 (0.0005) [2023-12-26 23:44:57,220][105692] Updated weights for policy 0, policy_version 1149926 (0.0005) [2023-12-26 23:44:57,706][105620] Updated weights for policy 1, policy_version 1151180 (0.0010) [2023-12-26 23:44:57,770][105620] Updated weights for policy 1, policy_version 1151190 (0.0006) [2023-12-26 23:44:57,820][105692] Updated weights for policy 0, policy_version 1149936 (0.0009) [2023-12-26 23:44:57,832][105620] Updated weights for policy 1, policy_version 1151200 (0.0006) [2023-12-26 23:44:57,872][105692] Updated weights for policy 0, policy_version 1149946 (0.0010) [2023-12-26 23:44:57,923][105692] Updated weights for policy 0, policy_version 1149956 (0.0010) [2023-12-26 23:44:58,546][105620] Updated weights for policy 1, policy_version 1151210 (0.0008) [2023-12-26 23:44:58,613][105620] Updated weights for policy 1, policy_version 1151220 (0.0010) [2023-12-26 23:44:58,639][105692] Updated weights for policy 0, policy_version 1149966 (0.0009) [2023-12-26 23:44:58,681][105620] Updated weights for policy 1, policy_version 1151230 (0.0008) [2023-12-26 23:44:58,727][105692] Updated weights for policy 0, policy_version 1149976 (0.0009) [2023-12-26 23:44:58,799][105692] Updated weights for policy 0, policy_version 1149986 (0.0007) [2023-12-26 23:44:59,527][105620] Updated weights for policy 1, policy_version 1151242 (0.0008) [2023-12-26 23:44:59,581][105620] Updated weights for policy 1, policy_version 1151252 (0.0008) [2023-12-26 23:44:59,589][105692] Updated weights for policy 0, policy_version 1149996 (0.0007) [2023-12-26 23:44:59,638][105620] Updated weights for policy 1, policy_version 1151262 (0.0008) [2023-12-26 23:44:59,645][105692] Updated weights for policy 0, policy_version 1150006 (0.0006) [2023-12-26 23:44:59,704][105620] Updated weights for policy 1, policy_version 1151272 (0.0008) [2023-12-26 23:44:59,708][105692] Updated weights for policy 0, policy_version 1150016 (0.0008) [2023-12-26 23:45:00,416][105620] Updated weights for policy 1, policy_version 1151282 (0.0009) [2023-12-26 23:45:00,466][105692] Updated weights for policy 0, policy_version 1150027 (0.0010) [2023-12-26 23:45:00,473][105620] Updated weights for policy 1, policy_version 1151292 (0.0008) [2023-12-26 23:45:00,516][105692] Updated weights for policy 0, policy_version 1150037 (0.0008) [2023-12-26 23:45:00,528][105620] Updated weights for policy 1, policy_version 1151302 (0.0007) [2023-12-26 23:45:00,576][105692] Updated weights for policy 0, policy_version 1150047 (0.0009) [2023-12-26 23:45:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 589234176. Throughput: 0: 9897.3, 1: 9623.0. Samples: 589208608. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:45:01,062][104569] Avg episode reward: [(0, '9265.665'), (1, '8894.472')] [2023-12-26 23:45:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001150056_294461440.pth... [2023-12-26 23:45:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001148936_294174720.pth [2023-12-26 23:45:01,120][105620] Updated weights for policy 1, policy_version 1151312 (0.0008) [2023-12-26 23:45:01,183][105620] Updated weights for policy 1, policy_version 1151322 (0.0009) [2023-12-26 23:45:01,238][105620] Updated weights for policy 1, policy_version 1151332 (0.0008) [2023-12-26 23:45:01,265][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001151336_294780928.pth... [2023-12-26 23:45:01,269][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001150184_294486016.pth [2023-12-26 23:45:01,364][105692] Updated weights for policy 0, policy_version 1150057 (0.0010) [2023-12-26 23:45:01,414][105692] Updated weights for policy 0, policy_version 1150067 (0.0008) [2023-12-26 23:45:01,469][105692] Updated weights for policy 0, policy_version 1150077 (0.0009) [2023-12-26 23:45:01,524][105692] Updated weights for policy 0, policy_version 1150087 (0.0009) [2023-12-26 23:45:01,962][105620] Updated weights for policy 1, policy_version 1151342 (0.0009) [2023-12-26 23:45:02,027][105620] Updated weights for policy 1, policy_version 1151352 (0.0009) [2023-12-26 23:45:02,087][105620] Updated weights for policy 1, policy_version 1151362 (0.0008) [2023-12-26 23:45:02,252][105692] Updated weights for policy 0, policy_version 1150097 (0.0010) [2023-12-26 23:45:02,314][105692] Updated weights for policy 0, policy_version 1150107 (0.0006) [2023-12-26 23:45:02,376][105692] Updated weights for policy 0, policy_version 1150117 (0.0009) [2023-12-26 23:45:02,792][105620] Updated weights for policy 1, policy_version 1151372 (0.0009) [2023-12-26 23:45:02,857][105620] Updated weights for policy 1, policy_version 1151382 (0.0009) [2023-12-26 23:45:02,922][105620] Updated weights for policy 1, policy_version 1151392 (0.0009) [2023-12-26 23:45:03,107][105692] Updated weights for policy 0, policy_version 1150127 (0.0009) [2023-12-26 23:45:03,159][105692] Updated weights for policy 0, policy_version 1150137 (0.0009) [2023-12-26 23:45:03,211][105692] Updated weights for policy 0, policy_version 1150147 (0.0005) [2023-12-26 23:45:03,659][105620] Updated weights for policy 1, policy_version 1151402 (0.0008) [2023-12-26 23:45:03,718][105620] Updated weights for policy 1, policy_version 1151412 (0.0009) [2023-12-26 23:45:03,769][105620] Updated weights for policy 1, policy_version 1151422 (0.0010) [2023-12-26 23:45:03,825][105620] Updated weights for policy 1, policy_version 1151432 (0.0010) [2023-12-26 23:45:03,860][105692] Updated weights for policy 0, policy_version 1150157 (0.0007) [2023-12-26 23:45:03,923][105692] Updated weights for policy 0, policy_version 1150167 (0.0006) [2023-12-26 23:45:03,985][105692] Updated weights for policy 0, policy_version 1150177 (0.0010) [2023-12-26 23:45:04,480][105620] Updated weights for policy 1, policy_version 1151442 (0.0008) [2023-12-26 23:45:04,534][105620] Updated weights for policy 1, policy_version 1151452 (0.0008) [2023-12-26 23:45:04,582][105620] Updated weights for policy 1, policy_version 1151462 (0.0005) [2023-12-26 23:45:04,789][105692] Updated weights for policy 0, policy_version 1150187 (0.0009) [2023-12-26 23:45:04,855][105692] Updated weights for policy 0, policy_version 1150197 (0.0008) [2023-12-26 23:45:04,916][105692] Updated weights for policy 0, policy_version 1150207 (0.0007) [2023-12-26 23:45:05,235][105620] Updated weights for policy 1, policy_version 1151472 (0.0009) [2023-12-26 23:45:05,293][105620] Updated weights for policy 1, policy_version 1151482 (0.0010) [2023-12-26 23:45:05,351][105620] Updated weights for policy 1, policy_version 1151492 (0.0010) [2023-12-26 23:45:05,709][105692] Updated weights for policy 0, policy_version 1150217 (0.0010) [2023-12-26 23:45:05,784][105692] Updated weights for policy 0, policy_version 1150227 (0.0009) [2023-12-26 23:45:05,851][105692] Updated weights for policy 0, policy_version 1150237 (0.0005) [2023-12-26 23:45:05,899][105692] Updated weights for policy 0, policy_version 1150247 (0.0005) [2023-12-26 23:45:05,985][105620] Updated weights for policy 1, policy_version 1151502 (0.0010) [2023-12-26 23:45:06,054][105620] Updated weights for policy 1, policy_version 1151512 (0.0010) [2023-12-26 23:45:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 589332480. Throughput: 0: 9789.5, 1: 9760.7. Samples: 589324400. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:45:06,063][104569] Avg episode reward: [(0, '9262.245'), (1, '9076.301')] [2023-12-26 23:45:06,121][105620] Updated weights for policy 1, policy_version 1151522 (0.0009) [2023-12-26 23:45:06,522][105692] Updated weights for policy 0, policy_version 1150257 (0.0005) [2023-12-26 23:45:06,581][105692] Updated weights for policy 0, policy_version 1150267 (0.0009) [2023-12-26 23:45:06,644][105692] Updated weights for policy 0, policy_version 1150277 (0.0009) [2023-12-26 23:45:06,786][105620] Updated weights for policy 1, policy_version 1151532 (0.0009) [2023-12-26 23:45:06,843][105620] Updated weights for policy 1, policy_version 1151542 (0.0007) [2023-12-26 23:45:06,903][105620] Updated weights for policy 1, policy_version 1151552 (0.0008) [2023-12-26 23:45:07,352][105692] Updated weights for policy 0, policy_version 1150287 (0.0010) [2023-12-26 23:45:07,417][105692] Updated weights for policy 0, policy_version 1150297 (0.0010) [2023-12-26 23:45:07,488][105692] Updated weights for policy 0, policy_version 1150307 (0.0009) [2023-12-26 23:45:07,554][105620] Updated weights for policy 1, policy_version 1151562 (0.0009) [2023-12-26 23:45:07,600][105620] Updated weights for policy 1, policy_version 1151572 (0.0005) [2023-12-26 23:45:07,649][105620] Updated weights for policy 1, policy_version 1151582 (0.0005) [2023-12-26 23:45:07,701][105620] Updated weights for policy 1, policy_version 1151592 (0.0005) [2023-12-26 23:45:08,102][105692] Updated weights for policy 0, policy_version 1150317 (0.0008) [2023-12-26 23:45:08,162][105692] Updated weights for policy 0, policy_version 1150327 (0.0005) [2023-12-26 23:45:08,234][105692] Updated weights for policy 0, policy_version 1150337 (0.0005) [2023-12-26 23:45:08,257][105620] Updated weights for policy 1, policy_version 1151602 (0.0005) [2023-12-26 23:45:08,303][105620] Updated weights for policy 1, policy_version 1151612 (0.0005) [2023-12-26 23:45:08,369][105620] Updated weights for policy 1, policy_version 1151622 (0.0008) [2023-12-26 23:45:08,891][105692] Updated weights for policy 0, policy_version 1150347 (0.0006) [2023-12-26 23:45:08,940][105692] Updated weights for policy 0, policy_version 1150357 (0.0008) [2023-12-26 23:45:09,005][105692] Updated weights for policy 0, policy_version 1150367 (0.0009) [2023-12-26 23:45:09,074][105620] Updated weights for policy 1, policy_version 1151632 (0.0009) [2023-12-26 23:45:09,136][105620] Updated weights for policy 1, policy_version 1151642 (0.0008) [2023-12-26 23:45:09,197][105620] Updated weights for policy 1, policy_version 1151652 (0.0007) [2023-12-26 23:45:09,764][105692] Updated weights for policy 0, policy_version 1150377 (0.0007) [2023-12-26 23:45:09,822][105692] Updated weights for policy 0, policy_version 1150387 (0.0009) [2023-12-26 23:45:09,886][105692] Updated weights for policy 0, policy_version 1150397 (0.0008) [2023-12-26 23:45:09,952][105692] Updated weights for policy 0, policy_version 1150407 (0.0008) [2023-12-26 23:45:10,005][105620] Updated weights for policy 1, policy_version 1151662 (0.0009) [2023-12-26 23:45:10,067][105620] Updated weights for policy 1, policy_version 1151672 (0.0009) [2023-12-26 23:45:10,131][105620] Updated weights for policy 1, policy_version 1151682 (0.0009) [2023-12-26 23:45:10,759][105692] Updated weights for policy 0, policy_version 1150417 (0.0009) [2023-12-26 23:45:10,816][105692] Updated weights for policy 0, policy_version 1150427 (0.0009) [2023-12-26 23:45:10,820][105620] Updated weights for policy 1, policy_version 1151692 (0.0008) [2023-12-26 23:45:10,871][105692] Updated weights for policy 0, policy_version 1150437 (0.0009) [2023-12-26 23:45:10,887][105620] Updated weights for policy 1, policy_version 1151702 (0.0005) [2023-12-26 23:45:10,939][105620] Updated weights for policy 1, policy_version 1151712 (0.0005) [2023-12-26 23:45:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 589438976. Throughput: 0: 9815.0, 1: 9804.4. Samples: 589443432. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:45:11,063][104569] Avg episode reward: [(0, '9078.662'), (1, '9076.118')] [2023-12-26 23:45:11,664][105692] Updated weights for policy 0, policy_version 1150447 (0.0007) [2023-12-26 23:45:11,703][105620] Updated weights for policy 1, policy_version 1151722 (0.0007) [2023-12-26 23:45:11,735][105692] Updated weights for policy 0, policy_version 1150457 (0.0008) [2023-12-26 23:45:11,770][105620] Updated weights for policy 1, policy_version 1151732 (0.0009) [2023-12-26 23:45:11,794][105692] Updated weights for policy 0, policy_version 1150467 (0.0007) [2023-12-26 23:45:11,831][105620] Updated weights for policy 1, policy_version 1151742 (0.0006) [2023-12-26 23:45:11,899][105620] Updated weights for policy 1, policy_version 1151752 (0.0005) [2023-12-26 23:45:12,563][105620] Updated weights for policy 1, policy_version 1151762 (0.0009) [2023-12-26 23:45:12,570][105692] Updated weights for policy 0, policy_version 1150477 (0.0007) [2023-12-26 23:45:12,624][105620] Updated weights for policy 1, policy_version 1151772 (0.0007) [2023-12-26 23:45:12,630][105692] Updated weights for policy 0, policy_version 1150487 (0.0007) [2023-12-26 23:45:12,638][105586] KL-divergence is very high: 117.2273 [2023-12-26 23:45:12,681][105620] Updated weights for policy 1, policy_version 1151782 (0.0007) [2023-12-26 23:45:12,681][105586] KL-divergence is very high: 185.4177 [2023-12-26 23:45:12,691][105692] Updated weights for policy 0, policy_version 1150497 (0.0007) [2023-12-26 23:45:13,312][105692] Updated weights for policy 0, policy_version 1150507 (0.0007) [2023-12-26 23:45:13,368][105692] Updated weights for policy 0, policy_version 1150517 (0.0009) [2023-12-26 23:45:13,423][105692] Updated weights for policy 0, policy_version 1150527 (0.0010) [2023-12-26 23:45:13,485][105620] Updated weights for policy 1, policy_version 1151792 (0.0007) [2023-12-26 23:45:13,543][105620] Updated weights for policy 1, policy_version 1151802 (0.0009) [2023-12-26 23:45:13,600][105620] Updated weights for policy 1, policy_version 1151812 (0.0010) [2023-12-26 23:45:14,031][105692] Updated weights for policy 0, policy_version 1150537 (0.0007) [2023-12-26 23:45:14,094][105692] Updated weights for policy 0, policy_version 1150547 (0.0005) [2023-12-26 23:45:14,154][105692] Updated weights for policy 0, policy_version 1150557 (0.0005) [2023-12-26 23:45:14,202][105692] Updated weights for policy 0, policy_version 1150567 (0.0006) [2023-12-26 23:45:14,468][105620] Updated weights for policy 1, policy_version 1151823 (0.0009) [2023-12-26 23:45:14,515][105620] Updated weights for policy 1, policy_version 1151833 (0.0009) [2023-12-26 23:45:14,562][105620] Updated weights for policy 1, policy_version 1151843 (0.0009) [2023-12-26 23:45:14,823][105692] Updated weights for policy 0, policy_version 1150577 (0.0006) [2023-12-26 23:45:14,882][105692] Updated weights for policy 0, policy_version 1150587 (0.0006) [2023-12-26 23:45:14,940][105692] Updated weights for policy 0, policy_version 1150597 (0.0005) [2023-12-26 23:45:15,265][105620] Updated weights for policy 1, policy_version 1151853 (0.0007) [2023-12-26 23:45:15,334][105620] Updated weights for policy 1, policy_version 1151863 (0.0006) [2023-12-26 23:45:15,406][105620] Updated weights for policy 1, policy_version 1151873 (0.0006) [2023-12-26 23:45:15,605][105692] Updated weights for policy 0, policy_version 1150607 (0.0006) [2023-12-26 23:45:15,671][105692] Updated weights for policy 0, policy_version 1150617 (0.0006) [2023-12-26 23:45:15,728][105692] Updated weights for policy 0, policy_version 1150627 (0.0009) [2023-12-26 23:45:15,976][105620] Updated weights for policy 1, policy_version 1151883 (0.0006) [2023-12-26 23:45:16,035][105620] Updated weights for policy 1, policy_version 1151893 (0.0005) [2023-12-26 23:45:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 589529088. Throughput: 0: 9731.7, 1: 9690.8. Samples: 589498608. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:45:16,062][104569] Avg episode reward: [(0, '9079.040'), (1, '9042.747')] [2023-12-26 23:45:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001150632_294608896.pth... [2023-12-26 23:45:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001149480_294313984.pth [2023-12-26 23:45:16,100][105620] Updated weights for policy 1, policy_version 1151903 (0.0005) [2023-12-26 23:45:16,148][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001151912_294928384.pth... [2023-12-26 23:45:16,151][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001150760_294633472.pth [2023-12-26 23:45:16,290][105692] Updated weights for policy 0, policy_version 1150637 (0.0008) [2023-12-26 23:45:16,351][105692] Updated weights for policy 0, policy_version 1150647 (0.0009) [2023-12-26 23:45:16,409][105692] Updated weights for policy 0, policy_version 1150657 (0.0006) [2023-12-26 23:45:16,603][105620] Updated weights for policy 1, policy_version 1151913 (0.0005) [2023-12-26 23:45:16,659][105620] Updated weights for policy 1, policy_version 1151923 (0.0005) [2023-12-26 23:45:16,706][105620] Updated weights for policy 1, policy_version 1151933 (0.0005) [2023-12-26 23:45:16,755][105620] Updated weights for policy 1, policy_version 1151943 (0.0005) [2023-12-26 23:45:17,226][105692] Updated weights for policy 0, policy_version 1150667 (0.0009) [2023-12-26 23:45:17,291][105692] Updated weights for policy 0, policy_version 1150677 (0.0008) [2023-12-26 23:45:17,335][105692] Updated weights for policy 0, policy_version 1150687 (0.0008) [2023-12-26 23:45:17,432][105620] Updated weights for policy 1, policy_version 1151953 (0.0010) [2023-12-26 23:45:17,483][105620] Updated weights for policy 1, policy_version 1151963 (0.0010) [2023-12-26 23:45:17,544][105620] Updated weights for policy 1, policy_version 1151973 (0.0010) [2023-12-26 23:45:18,096][105692] Updated weights for policy 0, policy_version 1150697 (0.0008) [2023-12-26 23:45:18,145][105692] Updated weights for policy 0, policy_version 1150707 (0.0008) [2023-12-26 23:45:18,205][105692] Updated weights for policy 0, policy_version 1150717 (0.0008) [2023-12-26 23:45:18,260][105692] Updated weights for policy 0, policy_version 1150727 (0.0007) [2023-12-26 23:45:18,274][105620] Updated weights for policy 1, policy_version 1151983 (0.0010) [2023-12-26 23:45:18,329][105620] Updated weights for policy 1, policy_version 1151993 (0.0010) [2023-12-26 23:45:18,393][105620] Updated weights for policy 1, policy_version 1152003 (0.0011) [2023-12-26 23:45:19,018][105692] Updated weights for policy 0, policy_version 1150737 (0.0008) [2023-12-26 23:45:19,066][105692] Updated weights for policy 0, policy_version 1150747 (0.0008) [2023-12-26 23:45:19,121][105692] Updated weights for policy 0, policy_version 1150757 (0.0008) [2023-12-26 23:45:19,146][105620] Updated weights for policy 1, policy_version 1152013 (0.0010) [2023-12-26 23:45:19,194][105620] Updated weights for policy 1, policy_version 1152023 (0.0010) [2023-12-26 23:45:19,254][105620] Updated weights for policy 1, policy_version 1152033 (0.0009) [2023-12-26 23:45:19,886][105692] Updated weights for policy 0, policy_version 1150767 (0.0008) [2023-12-26 23:45:19,948][105692] Updated weights for policy 0, policy_version 1150777 (0.0009) [2023-12-26 23:45:20,007][105692] Updated weights for policy 0, policy_version 1150787 (0.0008) [2023-12-26 23:45:20,008][105620] Updated weights for policy 1, policy_version 1152043 (0.0010) [2023-12-26 23:45:20,064][105620] Updated weights for policy 1, policy_version 1152053 (0.0008) [2023-12-26 23:45:20,119][105620] Updated weights for policy 1, policy_version 1152063 (0.0007) [2023-12-26 23:45:20,732][105692] Updated weights for policy 0, policy_version 1150797 (0.0007) [2023-12-26 23:45:20,781][105620] Updated weights for policy 1, policy_version 1152073 (0.0009) [2023-12-26 23:45:20,798][105692] Updated weights for policy 0, policy_version 1150807 (0.0009) [2023-12-26 23:45:20,845][105620] Updated weights for policy 1, policy_version 1152083 (0.0011) [2023-12-26 23:45:20,863][105692] Updated weights for policy 0, policy_version 1150817 (0.0008) [2023-12-26 23:45:20,915][105620] Updated weights for policy 1, policy_version 1152093 (0.0011) [2023-12-26 23:45:20,975][105620] Updated weights for policy 1, policy_version 1152103 (0.0011) [2023-12-26 23:45:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 589635584. Throughput: 0: 9694.7, 1: 9731.5. Samples: 589619660. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:45:21,063][104569] Avg episode reward: [(0, '9352.761'), (1, '9224.032')] [2023-12-26 23:45:21,611][105692] Updated weights for policy 0, policy_version 1150827 (0.0008) [2023-12-26 23:45:21,675][105692] Updated weights for policy 0, policy_version 1150837 (0.0008) [2023-12-26 23:45:21,724][105620] Updated weights for policy 1, policy_version 1152113 (0.0010) [2023-12-26 23:45:21,739][105692] Updated weights for policy 0, policy_version 1150847 (0.0007) [2023-12-26 23:45:21,784][105620] Updated weights for policy 1, policy_version 1152123 (0.0010) [2023-12-26 23:45:21,851][105620] Updated weights for policy 1, policy_version 1152133 (0.0011) [2023-12-26 23:45:22,434][105692] Updated weights for policy 0, policy_version 1150857 (0.0006) [2023-12-26 23:45:22,492][105692] Updated weights for policy 0, policy_version 1150867 (0.0006) [2023-12-26 23:45:22,547][105692] Updated weights for policy 0, policy_version 1150877 (0.0007) [2023-12-26 23:45:22,611][105692] Updated weights for policy 0, policy_version 1150887 (0.0008) [2023-12-26 23:45:22,672][105620] Updated weights for policy 1, policy_version 1152143 (0.0009) [2023-12-26 23:45:22,734][105620] Updated weights for policy 1, policy_version 1152153 (0.0009) [2023-12-26 23:45:22,797][105620] Updated weights for policy 1, policy_version 1152163 (0.0009) [2023-12-26 23:45:23,340][105692] Updated weights for policy 0, policy_version 1150897 (0.0008) [2023-12-26 23:45:23,402][105692] Updated weights for policy 0, policy_version 1150907 (0.0008) [2023-12-26 23:45:23,416][105620] Updated weights for policy 1, policy_version 1152173 (0.0007) [2023-12-26 23:45:23,451][105692] Updated weights for policy 0, policy_version 1150917 (0.0006) [2023-12-26 23:45:23,468][105620] Updated weights for policy 1, policy_version 1152183 (0.0006) [2023-12-26 23:45:23,513][105620] Updated weights for policy 1, policy_version 1152193 (0.0008) [2023-12-26 23:45:24,201][105692] Updated weights for policy 0, policy_version 1150927 (0.0008) [2023-12-26 23:45:24,226][105620] Updated weights for policy 1, policy_version 1152203 (0.0009) [2023-12-26 23:45:24,252][105692] Updated weights for policy 0, policy_version 1150937 (0.0008) [2023-12-26 23:45:24,278][105620] Updated weights for policy 1, policy_version 1152213 (0.0007) [2023-12-26 23:45:24,306][105692] Updated weights for policy 0, policy_version 1150947 (0.0006) [2023-12-26 23:45:24,332][105620] Updated weights for policy 1, policy_version 1152223 (0.0008) [2023-12-26 23:45:24,953][105620] Updated weights for policy 1, policy_version 1152233 (0.0009) [2023-12-26 23:45:25,008][105620] Updated weights for policy 1, policy_version 1152243 (0.0006) [2023-12-26 23:45:25,057][105620] Updated weights for policy 1, policy_version 1152253 (0.0005) [2023-12-26 23:45:25,117][105620] Updated weights for policy 1, policy_version 1152263 (0.0011) [2023-12-26 23:45:25,134][105692] Updated weights for policy 0, policy_version 1150957 (0.0007) [2023-12-26 23:45:25,183][105692] Updated weights for policy 0, policy_version 1150967 (0.0009) [2023-12-26 23:45:25,231][105692] Updated weights for policy 0, policy_version 1150977 (0.0008) [2023-12-26 23:45:25,804][105620] Updated weights for policy 1, policy_version 1152273 (0.0010) [2023-12-26 23:45:25,849][105620] Updated weights for policy 1, policy_version 1152283 (0.0010) [2023-12-26 23:45:25,894][105620] Updated weights for policy 1, policy_version 1152293 (0.0010) [2023-12-26 23:45:26,027][105692] Updated weights for policy 0, policy_version 1150987 (0.0009) [2023-12-26 23:45:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 589725696. Throughput: 0: 9663.7, 1: 9710.2. Samples: 589735064. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:45:26,062][104569] Avg episode reward: [(0, '9264.117'), (1, '9347.815')] [2023-12-26 23:45:26,071][105692] Updated weights for policy 0, policy_version 1150997 (0.0008) [2023-12-26 23:45:26,127][105692] Updated weights for policy 0, policy_version 1151007 (0.0008) [2023-12-26 23:45:26,669][105620] Updated weights for policy 1, policy_version 1152303 (0.0010) [2023-12-26 23:45:26,713][105620] Updated weights for policy 1, policy_version 1152313 (0.0010) [2023-12-26 23:45:26,760][105620] Updated weights for policy 1, policy_version 1152323 (0.0010) [2023-12-26 23:45:26,897][105692] Updated weights for policy 0, policy_version 1151017 (0.0008) [2023-12-26 23:45:26,951][105692] Updated weights for policy 0, policy_version 1151028 (0.0010) [2023-12-26 23:45:27,009][105692] Updated weights for policy 0, policy_version 1151038 (0.0010) [2023-12-26 23:45:27,066][105692] Updated weights for policy 0, policy_version 1151048 (0.0010) [2023-12-26 23:45:27,342][105620] Updated weights for policy 1, policy_version 1152333 (0.0010) [2023-12-26 23:45:27,400][105620] Updated weights for policy 1, policy_version 1152343 (0.0010) [2023-12-26 23:45:27,455][105620] Updated weights for policy 1, policy_version 1152353 (0.0010) [2023-12-26 23:45:27,863][105692] Updated weights for policy 0, policy_version 1151058 (0.0008) [2023-12-26 23:45:27,911][105692] Updated weights for policy 0, policy_version 1151068 (0.0008) [2023-12-26 23:45:27,954][105692] Updated weights for policy 0, policy_version 1151078 (0.0007) [2023-12-26 23:45:28,199][105620] Updated weights for policy 1, policy_version 1152363 (0.0010) [2023-12-26 23:45:28,251][105620] Updated weights for policy 1, policy_version 1152373 (0.0010) [2023-12-26 23:45:28,295][105620] Updated weights for policy 1, policy_version 1152383 (0.0010) [2023-12-26 23:45:28,613][105692] Updated weights for policy 0, policy_version 1151088 (0.0008) [2023-12-26 23:45:28,676][105692] Updated weights for policy 0, policy_version 1151098 (0.0008) [2023-12-26 23:45:28,736][105692] Updated weights for policy 0, policy_version 1151108 (0.0008) [2023-12-26 23:45:28,994][105620] Updated weights for policy 1, policy_version 1152393 (0.0009) [2023-12-26 23:45:29,054][105620] Updated weights for policy 1, policy_version 1152403 (0.0009) [2023-12-26 23:45:29,107][105620] Updated weights for policy 1, policy_version 1152413 (0.0010) [2023-12-26 23:45:29,166][105620] Updated weights for policy 1, policy_version 1152423 (0.0011) [2023-12-26 23:45:29,549][105692] Updated weights for policy 0, policy_version 1151118 (0.0008) [2023-12-26 23:45:29,611][105692] Updated weights for policy 0, policy_version 1151128 (0.0007) [2023-12-26 23:45:29,674][105692] Updated weights for policy 0, policy_version 1151138 (0.0008) [2023-12-26 23:45:29,846][105620] Updated weights for policy 1, policy_version 1152433 (0.0007) [2023-12-26 23:45:29,910][105620] Updated weights for policy 1, policy_version 1152443 (0.0008) [2023-12-26 23:45:29,978][105620] Updated weights for policy 1, policy_version 1152453 (0.0006) [2023-12-26 23:45:30,378][105692] Updated weights for policy 0, policy_version 1151148 (0.0006) [2023-12-26 23:45:30,427][105692] Updated weights for policy 0, policy_version 1151158 (0.0009) [2023-12-26 23:45:30,480][105692] Updated weights for policy 0, policy_version 1151168 (0.0006) [2023-12-26 23:45:30,515][105620] Updated weights for policy 1, policy_version 1152463 (0.0005) [2023-12-26 23:45:30,568][105620] Updated weights for policy 1, policy_version 1152473 (0.0006) [2023-12-26 23:45:30,615][105620] Updated weights for policy 1, policy_version 1152483 (0.0008) [2023-12-26 23:45:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 589824000. Throughput: 0: 9678.0, 1: 9757.1. Samples: 589794300. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:45:31,063][104569] Avg episode reward: [(0, '9174.758'), (1, '9350.314')] [2023-12-26 23:45:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001151176_294748160.pth... [2023-12-26 23:45:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001152488_295075840.pth... [2023-12-26 23:45:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001150056_294461440.pth [2023-12-26 23:45:31,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001151336_294780928.pth [2023-12-26 23:45:31,185][105692] Updated weights for policy 0, policy_version 1151178 (0.0010) [2023-12-26 23:45:31,226][105620] Updated weights for policy 1, policy_version 1152493 (0.0006) [2023-12-26 23:45:31,249][105692] Updated weights for policy 0, policy_version 1151188 (0.0010) [2023-12-26 23:45:31,290][105620] Updated weights for policy 1, policy_version 1152503 (0.0010) [2023-12-26 23:45:31,300][105692] Updated weights for policy 0, policy_version 1151198 (0.0007) [2023-12-26 23:45:31,343][105620] Updated weights for policy 1, policy_version 1152513 (0.0010) [2023-12-26 23:45:31,360][105692] Updated weights for policy 0, policy_version 1151208 (0.0008) [2023-12-26 23:45:32,029][105620] Updated weights for policy 1, policy_version 1152523 (0.0010) [2023-12-26 23:45:32,094][105620] Updated weights for policy 1, policy_version 1152533 (0.0009) [2023-12-26 23:45:32,100][105692] Updated weights for policy 0, policy_version 1151218 (0.0006) [2023-12-26 23:45:32,155][105620] Updated weights for policy 1, policy_version 1152543 (0.0007) [2023-12-26 23:45:32,158][105692] Updated weights for policy 0, policy_version 1151228 (0.0007) [2023-12-26 23:45:32,216][105692] Updated weights for policy 0, policy_version 1151238 (0.0007) [2023-12-26 23:45:32,853][105620] Updated weights for policy 1, policy_version 1152553 (0.0009) [2023-12-26 23:45:32,899][105620] Updated weights for policy 1, policy_version 1152563 (0.0008) [2023-12-26 23:45:32,954][105620] Updated weights for policy 1, policy_version 1152573 (0.0009) [2023-12-26 23:45:32,975][105692] Updated weights for policy 0, policy_version 1151248 (0.0008) [2023-12-26 23:45:33,023][105620] Updated weights for policy 1, policy_version 1152583 (0.0009) [2023-12-26 23:45:33,034][105692] Updated weights for policy 0, policy_version 1151258 (0.0005) [2023-12-26 23:45:33,090][105692] Updated weights for policy 0, policy_version 1151268 (0.0009) [2023-12-26 23:45:33,794][105620] Updated weights for policy 1, policy_version 1152593 (0.0007) [2023-12-26 23:45:33,800][105692] Updated weights for policy 0, policy_version 1151278 (0.0010) [2023-12-26 23:45:33,852][105620] Updated weights for policy 1, policy_version 1152603 (0.0007) [2023-12-26 23:45:33,854][105692] Updated weights for policy 0, policy_version 1151288 (0.0010) [2023-12-26 23:45:33,906][105692] Updated weights for policy 0, policy_version 1151298 (0.0011) [2023-12-26 23:45:33,908][105620] Updated weights for policy 1, policy_version 1152613 (0.0006) [2023-12-26 23:45:34,663][105620] Updated weights for policy 1, policy_version 1152623 (0.0009) [2023-12-26 23:45:34,671][105692] Updated weights for policy 0, policy_version 1151308 (0.0011) [2023-12-26 23:45:34,722][105620] Updated weights for policy 1, policy_version 1152633 (0.0010) [2023-12-26 23:45:34,723][105692] Updated weights for policy 0, policy_version 1151318 (0.0011) [2023-12-26 23:45:34,782][105692] Updated weights for policy 0, policy_version 1151328 (0.0011) [2023-12-26 23:45:34,782][105620] Updated weights for policy 1, policy_version 1152643 (0.0008) [2023-12-26 23:45:35,321][105620] Updated weights for policy 1, policy_version 1152653 (0.0005) [2023-12-26 23:45:35,383][105620] Updated weights for policy 1, policy_version 1152663 (0.0006) [2023-12-26 23:45:35,448][105620] Updated weights for policy 1, policy_version 1152673 (0.0005) [2023-12-26 23:45:35,527][105692] Updated weights for policy 0, policy_version 1151338 (0.0011) [2023-12-26 23:45:35,585][105692] Updated weights for policy 0, policy_version 1151348 (0.0010) [2023-12-26 23:45:35,640][105692] Updated weights for policy 0, policy_version 1151358 (0.0009) [2023-12-26 23:45:35,708][105692] Updated weights for policy 0, policy_version 1151368 (0.0005) [2023-12-26 23:45:36,046][105620] Updated weights for policy 1, policy_version 1152683 (0.0007) [2023-12-26 23:45:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 589922304. Throughput: 0: 9680.2, 1: 9859.5. Samples: 589912492. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:45:36,062][104569] Avg episode reward: [(0, '8811.634'), (1, '9262.312')] [2023-12-26 23:45:36,101][105620] Updated weights for policy 1, policy_version 1152693 (0.0008) [2023-12-26 23:45:36,163][105620] Updated weights for policy 1, policy_version 1152703 (0.0009) [2023-12-26 23:45:36,382][105692] Updated weights for policy 0, policy_version 1151378 (0.0011) [2023-12-26 23:45:36,445][105692] Updated weights for policy 0, policy_version 1151388 (0.0010) [2023-12-26 23:45:36,494][105692] Updated weights for policy 0, policy_version 1151398 (0.0010) [2023-12-26 23:45:36,852][105620] Updated weights for policy 1, policy_version 1152713 (0.0006) [2023-12-26 23:45:36,905][105620] Updated weights for policy 1, policy_version 1152723 (0.0010) [2023-12-26 23:45:36,960][105620] Updated weights for policy 1, policy_version 1152733 (0.0007) [2023-12-26 23:45:37,015][105620] Updated weights for policy 1, policy_version 1152743 (0.0005) [2023-12-26 23:45:37,138][105692] Updated weights for policy 0, policy_version 1151408 (0.0011) [2023-12-26 23:45:37,191][105692] Updated weights for policy 0, policy_version 1151418 (0.0011) [2023-12-26 23:45:37,250][105692] Updated weights for policy 0, policy_version 1151428 (0.0011) [2023-12-26 23:45:37,636][105620] Updated weights for policy 1, policy_version 1152753 (0.0005) [2023-12-26 23:45:37,702][105620] Updated weights for policy 1, policy_version 1152763 (0.0009) [2023-12-26 23:45:37,769][105620] Updated weights for policy 1, policy_version 1152773 (0.0007) [2023-12-26 23:45:38,046][105692] Updated weights for policy 0, policy_version 1151438 (0.0007) [2023-12-26 23:45:38,108][105692] Updated weights for policy 0, policy_version 1151448 (0.0005) [2023-12-26 23:45:38,165][105692] Updated weights for policy 0, policy_version 1151458 (0.0005) [2023-12-26 23:45:38,346][105620] Updated weights for policy 1, policy_version 1152783 (0.0007) [2023-12-26 23:45:38,409][105620] Updated weights for policy 1, policy_version 1152793 (0.0009) [2023-12-26 23:45:38,462][105620] Updated weights for policy 1, policy_version 1152803 (0.0008) [2023-12-26 23:45:38,917][105692] Updated weights for policy 0, policy_version 1151468 (0.0009) [2023-12-26 23:45:38,966][105692] Updated weights for policy 0, policy_version 1151478 (0.0009) [2023-12-26 23:45:39,013][105692] Updated weights for policy 0, policy_version 1151488 (0.0009) [2023-12-26 23:45:39,042][105620] Updated weights for policy 1, policy_version 1152813 (0.0010) [2023-12-26 23:45:39,102][105620] Updated weights for policy 1, policy_version 1152823 (0.0010) [2023-12-26 23:45:39,165][105620] Updated weights for policy 1, policy_version 1152833 (0.0011) [2023-12-26 23:45:39,789][105620] Updated weights for policy 1, policy_version 1152843 (0.0011) [2023-12-26 23:45:39,848][105620] Updated weights for policy 1, policy_version 1152853 (0.0012) [2023-12-26 23:45:39,882][105692] Updated weights for policy 0, policy_version 1151498 (0.0007) [2023-12-26 23:45:39,900][105620] Updated weights for policy 1, policy_version 1152863 (0.0008) [2023-12-26 23:45:39,952][105692] Updated weights for policy 0, policy_version 1151508 (0.0008) [2023-12-26 23:45:40,012][105692] Updated weights for policy 0, policy_version 1151518 (0.0008) [2023-12-26 23:45:40,059][105692] Updated weights for policy 0, policy_version 1151528 (0.0008) [2023-12-26 23:45:40,577][105620] Updated weights for policy 1, policy_version 1152873 (0.0008) [2023-12-26 23:45:40,632][105620] Updated weights for policy 1, policy_version 1152883 (0.0010) [2023-12-26 23:45:40,684][105620] Updated weights for policy 1, policy_version 1152893 (0.0007) [2023-12-26 23:45:40,744][105620] Updated weights for policy 1, policy_version 1152903 (0.0005) [2023-12-26 23:45:40,889][105692] Updated weights for policy 0, policy_version 1151538 (0.0007) [2023-12-26 23:45:40,933][105692] Updated weights for policy 0, policy_version 1151548 (0.0007) [2023-12-26 23:45:40,977][105692] Updated weights for policy 0, policy_version 1151558 (0.0008) [2023-12-26 23:45:41,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 590028800. Throughput: 0: 9542.4, 1: 10144.6. Samples: 590033876. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:45:41,063][104569] Avg episode reward: [(0, '8901.652'), (1, '9172.648')] [2023-12-26 23:45:41,408][105620] Updated weights for policy 1, policy_version 1152913 (0.0011) [2023-12-26 23:45:41,476][105620] Updated weights for policy 1, policy_version 1152923 (0.0011) [2023-12-26 23:45:41,544][105620] Updated weights for policy 1, policy_version 1152933 (0.0011) [2023-12-26 23:45:41,849][105692] Updated weights for policy 0, policy_version 1151568 (0.0008) [2023-12-26 23:45:41,908][105692] Updated weights for policy 0, policy_version 1151578 (0.0006) [2023-12-26 23:45:41,970][105692] Updated weights for policy 0, policy_version 1151588 (0.0006) [2023-12-26 23:45:42,303][105620] Updated weights for policy 1, policy_version 1152943 (0.0011) [2023-12-26 23:45:42,370][105620] Updated weights for policy 1, policy_version 1152953 (0.0011) [2023-12-26 23:45:42,431][105620] Updated weights for policy 1, policy_version 1152963 (0.0011) [2023-12-26 23:45:42,666][105692] Updated weights for policy 0, policy_version 1151598 (0.0006) [2023-12-26 23:45:42,720][105692] Updated weights for policy 0, policy_version 1151608 (0.0008) [2023-12-26 23:45:42,772][105692] Updated weights for policy 0, policy_version 1151618 (0.0008) [2023-12-26 23:45:43,146][105620] Updated weights for policy 1, policy_version 1152973 (0.0011) [2023-12-26 23:45:43,204][105620] Updated weights for policy 1, policy_version 1152983 (0.0010) [2023-12-26 23:45:43,267][105620] Updated weights for policy 1, policy_version 1152993 (0.0010) [2023-12-26 23:45:43,421][105692] Updated weights for policy 0, policy_version 1151628 (0.0010) [2023-12-26 23:45:43,475][105692] Updated weights for policy 0, policy_version 1151638 (0.0007) [2023-12-26 23:45:43,530][105692] Updated weights for policy 0, policy_version 1151648 (0.0005) [2023-12-26 23:45:43,916][105620] Updated weights for policy 1, policy_version 1153003 (0.0010) [2023-12-26 23:45:43,968][105620] Updated weights for policy 1, policy_version 1153013 (0.0008) [2023-12-26 23:45:44,016][105620] Updated weights for policy 1, policy_version 1153023 (0.0008) [2023-12-26 23:45:44,230][105692] Updated weights for policy 0, policy_version 1151658 (0.0006) [2023-12-26 23:45:44,296][105692] Updated weights for policy 0, policy_version 1151668 (0.0010) [2023-12-26 23:45:44,363][105692] Updated weights for policy 0, policy_version 1151678 (0.0011) [2023-12-26 23:45:44,433][105692] Updated weights for policy 0, policy_version 1151688 (0.0011) [2023-12-26 23:45:44,741][105620] Updated weights for policy 1, policy_version 1153033 (0.0008) [2023-12-26 23:45:44,800][105620] Updated weights for policy 1, policy_version 1153043 (0.0008) [2023-12-26 23:45:44,852][105620] Updated weights for policy 1, policy_version 1153053 (0.0009) [2023-12-26 23:45:44,911][105620] Updated weights for policy 1, policy_version 1153063 (0.0008) [2023-12-26 23:45:45,186][105692] Updated weights for policy 0, policy_version 1151698 (0.0011) [2023-12-26 23:45:45,234][105692] Updated weights for policy 0, policy_version 1151708 (0.0010) [2023-12-26 23:45:45,290][105692] Updated weights for policy 0, policy_version 1151718 (0.0011) [2023-12-26 23:45:45,705][105620] Updated weights for policy 1, policy_version 1153073 (0.0006) [2023-12-26 23:45:45,773][105620] Updated weights for policy 1, policy_version 1153083 (0.0005) [2023-12-26 23:45:45,834][105620] Updated weights for policy 1, policy_version 1153093 (0.0008) [2023-12-26 23:45:46,042][105692] Updated weights for policy 0, policy_version 1151728 (0.0010) [2023-12-26 23:45:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19605.2). Total num frames: 590118912. Throughput: 0: 9514.5, 1: 10120.0. Samples: 590092164. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:45:46,063][104569] Avg episode reward: [(0, '9262.980'), (1, '8987.999')] [2023-12-26 23:45:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001153096_295231488.pth... [2023-12-26 23:45:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001151912_294928384.pth [2023-12-26 23:45:46,103][105692] Updated weights for policy 0, policy_version 1151738 (0.0010) [2023-12-26 23:45:46,161][105692] Updated weights for policy 0, policy_version 1151748 (0.0010) [2023-12-26 23:45:46,184][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001151752_294895616.pth... [2023-12-26 23:45:46,187][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001150632_294608896.pth [2023-12-26 23:45:46,481][105620] Updated weights for policy 1, policy_version 1153103 (0.0009) [2023-12-26 23:45:46,536][105620] Updated weights for policy 1, policy_version 1153113 (0.0008) [2023-12-26 23:45:46,587][105620] Updated weights for policy 1, policy_version 1153123 (0.0008) [2023-12-26 23:45:46,910][105692] Updated weights for policy 0, policy_version 1151758 (0.0010) [2023-12-26 23:45:46,968][105692] Updated weights for policy 0, policy_version 1151768 (0.0010) [2023-12-26 23:45:47,022][105692] Updated weights for policy 0, policy_version 1151778 (0.0010) [2023-12-26 23:45:47,363][105620] Updated weights for policy 1, policy_version 1153133 (0.0008) [2023-12-26 23:45:47,426][105620] Updated weights for policy 1, policy_version 1153143 (0.0007) [2023-12-26 23:45:47,494][105620] Updated weights for policy 1, policy_version 1153153 (0.0005) [2023-12-26 23:45:47,764][105692] Updated weights for policy 0, policy_version 1151788 (0.0010) [2023-12-26 23:45:47,812][105692] Updated weights for policy 0, policy_version 1151798 (0.0010) [2023-12-26 23:45:47,860][105692] Updated weights for policy 0, policy_version 1151808 (0.0010) [2023-12-26 23:45:48,018][105620] Updated weights for policy 1, policy_version 1153163 (0.0006) [2023-12-26 23:45:48,070][105620] Updated weights for policy 1, policy_version 1153173 (0.0008) [2023-12-26 23:45:48,122][105620] Updated weights for policy 1, policy_version 1153183 (0.0008) [2023-12-26 23:45:48,626][105692] Updated weights for policy 0, policy_version 1151818 (0.0010) [2023-12-26 23:45:48,678][105692] Updated weights for policy 0, policy_version 1151828 (0.0010) [2023-12-26 23:45:48,727][105692] Updated weights for policy 0, policy_version 1151838 (0.0011) [2023-12-26 23:45:48,790][105692] Updated weights for policy 0, policy_version 1151848 (0.0011) [2023-12-26 23:45:48,807][105620] Updated weights for policy 1, policy_version 1153193 (0.0007) [2023-12-26 23:45:48,877][105620] Updated weights for policy 1, policy_version 1153203 (0.0006) [2023-12-26 23:45:48,946][105620] Updated weights for policy 1, policy_version 1153213 (0.0007) [2023-12-26 23:45:48,996][105620] Updated weights for policy 1, policy_version 1153223 (0.0008) [2023-12-26 23:45:49,564][105692] Updated weights for policy 0, policy_version 1151858 (0.0007) [2023-12-26 23:45:49,630][105692] Updated weights for policy 0, policy_version 1151868 (0.0008) [2023-12-26 23:45:49,676][105620] Updated weights for policy 1, policy_version 1153233 (0.0008) [2023-12-26 23:45:49,690][105692] Updated weights for policy 0, policy_version 1151878 (0.0010) [2023-12-26 23:45:49,735][105620] Updated weights for policy 1, policy_version 1153243 (0.0007) [2023-12-26 23:45:49,795][105620] Updated weights for policy 1, policy_version 1153253 (0.0005) [2023-12-26 23:45:50,366][105692] Updated weights for policy 0, policy_version 1151888 (0.0007) [2023-12-26 23:45:50,419][105620] Updated weights for policy 1, policy_version 1153263 (0.0007) [2023-12-26 23:45:50,425][105692] Updated weights for policy 0, policy_version 1151898 (0.0008) [2023-12-26 23:45:50,491][105620] Updated weights for policy 1, policy_version 1153273 (0.0009) [2023-12-26 23:45:50,494][105692] Updated weights for policy 0, policy_version 1151908 (0.0006) [2023-12-26 23:45:50,558][105620] Updated weights for policy 1, policy_version 1153283 (0.0009) [2023-12-26 23:45:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 590217216. Throughput: 0: 9524.4, 1: 10120.0. Samples: 590208396. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:45:51,062][104569] Avg episode reward: [(0, '9170.810'), (1, '7850.738')] [2023-12-26 23:45:51,163][105692] Updated weights for policy 0, policy_version 1151918 (0.0009) [2023-12-26 23:45:51,219][105692] Updated weights for policy 0, policy_version 1151928 (0.0009) [2023-12-26 23:45:51,279][105692] Updated weights for policy 0, policy_version 1151938 (0.0009) [2023-12-26 23:45:51,382][105620] Updated weights for policy 1, policy_version 1153293 (0.0008) [2023-12-26 23:45:51,449][105620] Updated weights for policy 1, policy_version 1153303 (0.0008) [2023-12-26 23:45:51,514][105620] Updated weights for policy 1, policy_version 1153313 (0.0006) [2023-12-26 23:45:52,113][105692] Updated weights for policy 0, policy_version 1151948 (0.0009) [2023-12-26 23:45:52,176][105692] Updated weights for policy 0, policy_version 1151958 (0.0009) [2023-12-26 23:45:52,234][105692] Updated weights for policy 0, policy_version 1151968 (0.0009) [2023-12-26 23:45:52,255][105620] Updated weights for policy 1, policy_version 1153323 (0.0008) [2023-12-26 23:45:52,314][105620] Updated weights for policy 1, policy_version 1153333 (0.0008) [2023-12-26 23:45:52,379][105620] Updated weights for policy 1, policy_version 1153343 (0.0008) [2023-12-26 23:45:53,020][105692] Updated weights for policy 0, policy_version 1151978 (0.0007) [2023-12-26 23:45:53,079][105692] Updated weights for policy 0, policy_version 1151988 (0.0009) [2023-12-26 23:45:53,138][105692] Updated weights for policy 0, policy_version 1151998 (0.0009) [2023-12-26 23:45:53,138][105620] Updated weights for policy 1, policy_version 1153353 (0.0008) [2023-12-26 23:45:53,184][105620] Updated weights for policy 1, policy_version 1153363 (0.0006) [2023-12-26 23:45:53,198][105692] Updated weights for policy 0, policy_version 1152008 (0.0008) [2023-12-26 23:45:53,241][105620] Updated weights for policy 1, policy_version 1153373 (0.0008) [2023-12-26 23:45:53,296][105620] Updated weights for policy 1, policy_version 1153383 (0.0009) [2023-12-26 23:45:53,863][105692] Updated weights for policy 0, policy_version 1152018 (0.0009) [2023-12-26 23:45:53,922][105692] Updated weights for policy 0, policy_version 1152028 (0.0009) [2023-12-26 23:45:53,988][105692] Updated weights for policy 0, policy_version 1152038 (0.0006) [2023-12-26 23:45:54,119][105620] Updated weights for policy 1, policy_version 1153393 (0.0008) [2023-12-26 23:45:54,164][105620] Updated weights for policy 1, policy_version 1153403 (0.0005) [2023-12-26 23:45:54,223][105620] Updated weights for policy 1, policy_version 1153413 (0.0006) [2023-12-26 23:45:54,780][105692] Updated weights for policy 0, policy_version 1152048 (0.0010) [2023-12-26 23:45:54,841][105692] Updated weights for policy 0, policy_version 1152058 (0.0008) [2023-12-26 23:45:54,891][105620] Updated weights for policy 1, policy_version 1153423 (0.0009) [2023-12-26 23:45:54,898][105692] Updated weights for policy 0, policy_version 1152068 (0.0007) [2023-12-26 23:45:54,955][105620] Updated weights for policy 1, policy_version 1153433 (0.0011) [2023-12-26 23:45:55,017][105620] Updated weights for policy 1, policy_version 1153443 (0.0011) [2023-12-26 23:45:55,571][105692] Updated weights for policy 0, policy_version 1152078 (0.0008) [2023-12-26 23:45:55,627][105692] Updated weights for policy 0, policy_version 1152088 (0.0011) [2023-12-26 23:45:55,676][105692] Updated weights for policy 0, policy_version 1152098 (0.0010) [2023-12-26 23:45:55,702][105620] Updated weights for policy 1, policy_version 1153453 (0.0009) [2023-12-26 23:45:55,768][105620] Updated weights for policy 1, policy_version 1153463 (0.0010) [2023-12-26 23:45:55,835][105620] Updated weights for policy 1, policy_version 1153473 (0.0010) [2023-12-26 23:45:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 590315520. Throughput: 0: 9510.3, 1: 10006.1. Samples: 590321668. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:45:56,062][104569] Avg episode reward: [(0, '9262.867'), (1, '8213.588')] [2023-12-26 23:45:56,350][105692] Updated weights for policy 0, policy_version 1152108 (0.0009) [2023-12-26 23:45:56,405][105692] Updated weights for policy 0, policy_version 1152118 (0.0010) [2023-12-26 23:45:56,464][105692] Updated weights for policy 0, policy_version 1152128 (0.0010) [2023-12-26 23:45:56,540][105620] Updated weights for policy 1, policy_version 1153483 (0.0010) [2023-12-26 23:45:56,588][105620] Updated weights for policy 1, policy_version 1153493 (0.0010) [2023-12-26 23:45:56,639][105620] Updated weights for policy 1, policy_version 1153503 (0.0010) [2023-12-26 23:45:57,061][105692] Updated weights for policy 0, policy_version 1152138 (0.0010) [2023-12-26 23:45:57,124][105692] Updated weights for policy 0, policy_version 1152148 (0.0009) [2023-12-26 23:45:57,181][105692] Updated weights for policy 0, policy_version 1152158 (0.0009) [2023-12-26 23:45:57,243][105692] Updated weights for policy 0, policy_version 1152168 (0.0008) [2023-12-26 23:45:57,368][105620] Updated weights for policy 1, policy_version 1153513 (0.0010) [2023-12-26 23:45:57,425][105620] Updated weights for policy 1, policy_version 1153523 (0.0007) [2023-12-26 23:45:57,484][105620] Updated weights for policy 1, policy_version 1153533 (0.0006) [2023-12-26 23:45:57,541][105620] Updated weights for policy 1, policy_version 1153543 (0.0006) [2023-12-26 23:45:57,945][105692] Updated weights for policy 0, policy_version 1152178 (0.0011) [2023-12-26 23:45:58,007][105692] Updated weights for policy 0, policy_version 1152188 (0.0010) [2023-12-26 23:45:58,060][105692] Updated weights for policy 0, policy_version 1152198 (0.0010) [2023-12-26 23:45:58,163][105620] Updated weights for policy 1, policy_version 1153553 (0.0007) [2023-12-26 23:45:58,226][105620] Updated weights for policy 1, policy_version 1153563 (0.0007) [2023-12-26 23:45:58,281][105620] Updated weights for policy 1, policy_version 1153573 (0.0008) [2023-12-26 23:45:58,888][105692] Updated weights for policy 0, policy_version 1152208 (0.0010) [2023-12-26 23:45:58,962][105692] Updated weights for policy 0, policy_version 1152218 (0.0011) [2023-12-26 23:45:59,033][105692] Updated weights for policy 0, policy_version 1152228 (0.0010) [2023-12-26 23:45:59,070][105620] Updated weights for policy 1, policy_version 1153583 (0.0007) [2023-12-26 23:45:59,132][105620] Updated weights for policy 1, policy_version 1153593 (0.0008) [2023-12-26 23:45:59,193][105620] Updated weights for policy 1, policy_version 1153603 (0.0008) [2023-12-26 23:45:59,892][105692] Updated weights for policy 0, policy_version 1152238 (0.0009) [2023-12-26 23:45:59,956][105692] Updated weights for policy 0, policy_version 1152248 (0.0007) [2023-12-26 23:46:00,018][105692] Updated weights for policy 0, policy_version 1152258 (0.0009) [2023-12-26 23:46:00,036][105620] Updated weights for policy 1, policy_version 1153613 (0.0008) [2023-12-26 23:46:00,097][105620] Updated weights for policy 1, policy_version 1153623 (0.0010) [2023-12-26 23:46:00,149][105620] Updated weights for policy 1, policy_version 1153633 (0.0009) [2023-12-26 23:46:00,745][105692] Updated weights for policy 0, policy_version 1152268 (0.0006) [2023-12-26 23:46:00,810][105692] Updated weights for policy 0, policy_version 1152278 (0.0010) [2023-12-26 23:46:00,859][105620] Updated weights for policy 1, policy_version 1153643 (0.0009) [2023-12-26 23:46:00,865][105692] Updated weights for policy 0, policy_version 1152288 (0.0008) [2023-12-26 23:46:00,910][105620] Updated weights for policy 1, policy_version 1153653 (0.0007) [2023-12-26 23:46:00,970][105620] Updated weights for policy 1, policy_version 1153663 (0.0009) [2023-12-26 23:46:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 590413824. Throughput: 0: 9542.6, 1: 10076.8. Samples: 590381484. Policy #0 lag: (min: 9.0, avg: 34.6, max: 41.0) [2023-12-26 23:46:01,063][104569] Avg episode reward: [(0, '9082.808'), (1, '9202.191')] [2023-12-26 23:46:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001152296_295034880.pth... [2023-12-26 23:46:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001153672_295378944.pth... [2023-12-26 23:46:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001151176_294748160.pth [2023-12-26 23:46:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001152488_295075840.pth [2023-12-26 23:46:01,569][105692] Updated weights for policy 0, policy_version 1152298 (0.0006) [2023-12-26 23:46:01,642][105692] Updated weights for policy 0, policy_version 1152308 (0.0009) [2023-12-26 23:46:01,697][105692] Updated weights for policy 0, policy_version 1152318 (0.0007) [2023-12-26 23:46:01,762][105692] Updated weights for policy 0, policy_version 1152328 (0.0010) [2023-12-26 23:46:01,805][105620] Updated weights for policy 1, policy_version 1153673 (0.0010) [2023-12-26 23:46:01,863][105620] Updated weights for policy 1, policy_version 1153683 (0.0009) [2023-12-26 23:46:01,924][105620] Updated weights for policy 1, policy_version 1153693 (0.0009) [2023-12-26 23:46:01,993][105620] Updated weights for policy 1, policy_version 1153703 (0.0006) [2023-12-26 23:46:02,509][105692] Updated weights for policy 0, policy_version 1152338 (0.0006) [2023-12-26 23:46:02,561][105692] Updated weights for policy 0, policy_version 1152348 (0.0005) [2023-12-26 23:46:02,622][105692] Updated weights for policy 0, policy_version 1152358 (0.0005) [2023-12-26 23:46:02,742][105620] Updated weights for policy 1, policy_version 1153713 (0.0010) [2023-12-26 23:46:02,800][105620] Updated weights for policy 1, policy_version 1153723 (0.0009) [2023-12-26 23:46:02,850][105620] Updated weights for policy 1, policy_version 1153733 (0.0009) [2023-12-26 23:46:03,288][105692] Updated weights for policy 0, policy_version 1152368 (0.0009) [2023-12-26 23:46:03,334][105692] Updated weights for policy 0, policy_version 1152378 (0.0008) [2023-12-26 23:46:03,384][105692] Updated weights for policy 0, policy_version 1152388 (0.0009) [2023-12-26 23:46:03,607][105620] Updated weights for policy 1, policy_version 1153743 (0.0009) [2023-12-26 23:46:03,655][105620] Updated weights for policy 1, policy_version 1153753 (0.0009) [2023-12-26 23:46:03,702][105620] Updated weights for policy 1, policy_version 1153763 (0.0009) [2023-12-26 23:46:04,151][105692] Updated weights for policy 0, policy_version 1152398 (0.0008) [2023-12-26 23:46:04,210][105692] Updated weights for policy 0, policy_version 1152408 (0.0008) [2023-12-26 23:46:04,270][105692] Updated weights for policy 0, policy_version 1152418 (0.0008) [2023-12-26 23:46:04,492][105620] Updated weights for policy 1, policy_version 1153773 (0.0011) [2023-12-26 23:46:04,546][105620] Updated weights for policy 1, policy_version 1153783 (0.0011) [2023-12-26 23:46:04,596][105620] Updated weights for policy 1, policy_version 1153793 (0.0011) [2023-12-26 23:46:05,050][105692] Updated weights for policy 0, policy_version 1152428 (0.0009) [2023-12-26 23:46:05,109][105692] Updated weights for policy 0, policy_version 1152438 (0.0008) [2023-12-26 23:46:05,165][105692] Updated weights for policy 0, policy_version 1152448 (0.0009) [2023-12-26 23:46:05,341][105620] Updated weights for policy 1, policy_version 1153803 (0.0011) [2023-12-26 23:46:05,390][105620] Updated weights for policy 1, policy_version 1153813 (0.0010) [2023-12-26 23:46:05,449][105620] Updated weights for policy 1, policy_version 1153823 (0.0010) [2023-12-26 23:46:05,772][105692] Updated weights for policy 0, policy_version 1152458 (0.0009) [2023-12-26 23:46:05,825][105692] Updated weights for policy 0, policy_version 1152468 (0.0009) [2023-12-26 23:46:05,887][105692] Updated weights for policy 0, policy_version 1152479 (0.0010) [2023-12-26 23:46:06,044][105620] Updated weights for policy 1, policy_version 1153833 (0.0009) [2023-12-26 23:46:06,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 590503936. Throughput: 0: 9432.7, 1: 9926.8. Samples: 590490840. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:46:06,062][104569] Avg episode reward: [(0, '8992.818'), (1, '9350.044')] [2023-12-26 23:46:06,099][105620] Updated weights for policy 1, policy_version 1153843 (0.0006) [2023-12-26 23:46:06,156][105620] Updated weights for policy 1, policy_version 1153853 (0.0009) [2023-12-26 23:46:06,218][105620] Updated weights for policy 1, policy_version 1153863 (0.0008) [2023-12-26 23:46:06,727][105692] Updated weights for policy 0, policy_version 1152489 (0.0010) [2023-12-26 23:46:06,788][105692] Updated weights for policy 0, policy_version 1152499 (0.0009) [2023-12-26 23:46:06,831][105620] Updated weights for policy 1, policy_version 1153873 (0.0007) [2023-12-26 23:46:06,849][105692] Updated weights for policy 0, policy_version 1152509 (0.0008) [2023-12-26 23:46:06,896][105620] Updated weights for policy 1, policy_version 1153883 (0.0006) [2023-12-26 23:46:06,915][105692] Updated weights for policy 0, policy_version 1152519 (0.0009) [2023-12-26 23:46:06,959][105620] Updated weights for policy 1, policy_version 1153893 (0.0009) [2023-12-26 23:46:07,661][105620] Updated weights for policy 1, policy_version 1153903 (0.0008) [2023-12-26 23:46:07,714][105620] Updated weights for policy 1, policy_version 1153913 (0.0007) [2023-12-26 23:46:07,716][105692] Updated weights for policy 0, policy_version 1152529 (0.0008) [2023-12-26 23:46:07,769][105620] Updated weights for policy 1, policy_version 1153923 (0.0006) [2023-12-26 23:46:07,776][105692] Updated weights for policy 0, policy_version 1152539 (0.0008) [2023-12-26 23:46:07,837][105692] Updated weights for policy 0, policy_version 1152549 (0.0009) [2023-12-26 23:46:08,418][105620] Updated weights for policy 1, policy_version 1153933 (0.0008) [2023-12-26 23:46:08,480][105620] Updated weights for policy 1, policy_version 1153943 (0.0009) [2023-12-26 23:46:08,543][105620] Updated weights for policy 1, policy_version 1153953 (0.0009) [2023-12-26 23:46:08,557][105692] Updated weights for policy 0, policy_version 1152559 (0.0008) [2023-12-26 23:46:08,623][105692] Updated weights for policy 0, policy_version 1152569 (0.0010) [2023-12-26 23:46:08,641][105585] KL-divergence is very high: 108.0995 [2023-12-26 23:46:08,684][105692] Updated weights for policy 0, policy_version 1152579 (0.0009) [2023-12-26 23:46:09,325][105620] Updated weights for policy 1, policy_version 1153963 (0.0007) [2023-12-26 23:46:09,391][105692] Updated weights for policy 0, policy_version 1152589 (0.0009) [2023-12-26 23:46:09,393][105620] Updated weights for policy 1, policy_version 1153973 (0.0007) [2023-12-26 23:46:09,453][105620] Updated weights for policy 1, policy_version 1153983 (0.0008) [2023-12-26 23:46:09,459][105692] Updated weights for policy 0, policy_version 1152599 (0.0008) [2023-12-26 23:46:09,523][105692] Updated weights for policy 0, policy_version 1152609 (0.0008) [2023-12-26 23:46:10,178][105620] Updated weights for policy 1, policy_version 1153993 (0.0006) [2023-12-26 23:46:10,234][105692] Updated weights for policy 0, policy_version 1152619 (0.0007) [2023-12-26 23:46:10,245][105620] Updated weights for policy 1, policy_version 1154003 (0.0009) [2023-12-26 23:46:10,290][105692] Updated weights for policy 0, policy_version 1152629 (0.0006) [2023-12-26 23:46:10,307][105620] Updated weights for policy 1, policy_version 1154013 (0.0007) [2023-12-26 23:46:10,353][105692] Updated weights for policy 0, policy_version 1152639 (0.0007) [2023-12-26 23:46:10,362][105620] Updated weights for policy 1, policy_version 1154023 (0.0008) [2023-12-26 23:46:11,000][105692] Updated weights for policy 0, policy_version 1152649 (0.0006) [2023-12-26 23:46:11,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 590594048. Throughput: 0: 9452.5, 1: 9952.1. Samples: 590608272. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:46:11,062][104569] Avg episode reward: [(0, '9141.376'), (1, '9259.861')] [2023-12-26 23:46:11,066][105692] Updated weights for policy 0, policy_version 1152659 (0.0009) [2023-12-26 23:46:11,100][105620] Updated weights for policy 1, policy_version 1154033 (0.0011) [2023-12-26 23:46:11,124][105692] Updated weights for policy 0, policy_version 1152669 (0.0006) [2023-12-26 23:46:11,166][105620] Updated weights for policy 1, policy_version 1154043 (0.0011) [2023-12-26 23:46:11,189][105692] Updated weights for policy 0, policy_version 1152679 (0.0007) [2023-12-26 23:46:11,222][105620] Updated weights for policy 1, policy_version 1154053 (0.0010) [2023-12-26 23:46:11,999][105692] Updated weights for policy 0, policy_version 1152689 (0.0008) [2023-12-26 23:46:12,030][105620] Updated weights for policy 1, policy_version 1154063 (0.0011) [2023-12-26 23:46:12,045][105692] Updated weights for policy 0, policy_version 1152699 (0.0006) [2023-12-26 23:46:12,092][105692] Updated weights for policy 0, policy_version 1152709 (0.0007) [2023-12-26 23:46:12,117][105620] Updated weights for policy 1, policy_version 1154073 (0.0011) [2023-12-26 23:46:12,177][105620] Updated weights for policy 1, policy_version 1154083 (0.0011) [2023-12-26 23:46:12,897][105692] Updated weights for policy 0, policy_version 1152719 (0.0008) [2023-12-26 23:46:12,910][105620] Updated weights for policy 1, policy_version 1154093 (0.0011) [2023-12-26 23:46:12,948][105692] Updated weights for policy 0, policy_version 1152729 (0.0005) [2023-12-26 23:46:12,962][105620] Updated weights for policy 1, policy_version 1154103 (0.0011) [2023-12-26 23:46:13,008][105692] Updated weights for policy 0, policy_version 1152739 (0.0005) [2023-12-26 23:46:13,014][105620] Updated weights for policy 1, policy_version 1154113 (0.0010) [2023-12-26 23:46:13,722][105620] Updated weights for policy 1, policy_version 1154123 (0.0010) [2023-12-26 23:46:13,776][105620] Updated weights for policy 1, policy_version 1154133 (0.0009) [2023-12-26 23:46:13,785][105692] Updated weights for policy 0, policy_version 1152749 (0.0006) [2023-12-26 23:46:13,845][105620] Updated weights for policy 1, policy_version 1154143 (0.0010) [2023-12-26 23:46:13,847][105692] Updated weights for policy 0, policy_version 1152759 (0.0005) [2023-12-26 23:46:13,909][105692] Updated weights for policy 0, policy_version 1152769 (0.0006) [2023-12-26 23:46:14,513][105692] Updated weights for policy 0, policy_version 1152779 (0.0007) [2023-12-26 23:46:14,572][105692] Updated weights for policy 0, policy_version 1152789 (0.0009) [2023-12-26 23:46:14,613][105620] Updated weights for policy 1, policy_version 1154153 (0.0010) [2023-12-26 23:46:14,625][105692] Updated weights for policy 0, policy_version 1152799 (0.0006) [2023-12-26 23:46:14,669][105620] Updated weights for policy 1, policy_version 1154163 (0.0009) [2023-12-26 23:46:14,733][105620] Updated weights for policy 1, policy_version 1154173 (0.0009) [2023-12-26 23:46:14,805][105620] Updated weights for policy 1, policy_version 1154183 (0.0008) [2023-12-26 23:46:15,452][105692] Updated weights for policy 0, policy_version 1152809 (0.0007) [2023-12-26 23:46:15,508][105692] Updated weights for policy 0, policy_version 1152819 (0.0008) [2023-12-26 23:46:15,527][105620] Updated weights for policy 1, policy_version 1154193 (0.0008) [2023-12-26 23:46:15,554][105692] Updated weights for policy 0, policy_version 1152829 (0.0006) [2023-12-26 23:46:15,579][105620] Updated weights for policy 1, policy_version 1154203 (0.0008) [2023-12-26 23:46:15,600][105692] Updated weights for policy 0, policy_version 1152839 (0.0008) [2023-12-26 23:46:15,636][105620] Updated weights for policy 1, policy_version 1154213 (0.0008) [2023-12-26 23:46:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 590692352. Throughput: 0: 9418.5, 1: 9889.1. Samples: 590663140. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:46:16,063][104569] Avg episode reward: [(0, '8973.584'), (1, '9258.435')] [2023-12-26 23:46:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001152840_295174144.pth... [2023-12-26 23:46:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001154216_295518208.pth... [2023-12-26 23:46:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001151752_294895616.pth [2023-12-26 23:46:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001153096_295231488.pth [2023-12-26 23:46:16,290][105620] Updated weights for policy 1, policy_version 1154223 (0.0009) [2023-12-26 23:46:16,354][105620] Updated weights for policy 1, policy_version 1154233 (0.0009) [2023-12-26 23:46:16,409][105692] Updated weights for policy 0, policy_version 1152849 (0.0007) [2023-12-26 23:46:16,411][105620] Updated weights for policy 1, policy_version 1154243 (0.0007) [2023-12-26 23:46:16,464][105692] Updated weights for policy 0, policy_version 1152859 (0.0007) [2023-12-26 23:46:16,511][105692] Updated weights for policy 0, policy_version 1152869 (0.0008) [2023-12-26 23:46:17,150][105620] Updated weights for policy 1, policy_version 1154253 (0.0009) [2023-12-26 23:46:17,209][105620] Updated weights for policy 1, policy_version 1154263 (0.0008) [2023-12-26 23:46:17,256][105620] Updated weights for policy 1, policy_version 1154273 (0.0008) [2023-12-26 23:46:17,268][105692] Updated weights for policy 0, policy_version 1152879 (0.0009) [2023-12-26 23:46:17,315][105692] Updated weights for policy 0, policy_version 1152889 (0.0007) [2023-12-26 23:46:17,364][105692] Updated weights for policy 0, policy_version 1152899 (0.0005) [2023-12-26 23:46:17,888][105620] Updated weights for policy 1, policy_version 1154283 (0.0007) [2023-12-26 23:46:17,933][105620] Updated weights for policy 1, policy_version 1154293 (0.0005) [2023-12-26 23:46:17,981][105620] Updated weights for policy 1, policy_version 1154303 (0.0008) [2023-12-26 23:46:18,221][105692] Updated weights for policy 0, policy_version 1152909 (0.0008) [2023-12-26 23:46:18,274][105692] Updated weights for policy 0, policy_version 1152919 (0.0009) [2023-12-26 23:46:18,330][105692] Updated weights for policy 0, policy_version 1152929 (0.0010) [2023-12-26 23:46:18,592][105620] Updated weights for policy 1, policy_version 1154313 (0.0006) [2023-12-26 23:46:18,651][105620] Updated weights for policy 1, policy_version 1154323 (0.0007) [2023-12-26 23:46:18,706][105620] Updated weights for policy 1, policy_version 1154333 (0.0005) [2023-12-26 23:46:18,760][105620] Updated weights for policy 1, policy_version 1154343 (0.0005) [2023-12-26 23:46:19,184][105692] Updated weights for policy 0, policy_version 1152939 (0.0009) [2023-12-26 23:46:19,251][105692] Updated weights for policy 0, policy_version 1152949 (0.0008) [2023-12-26 23:46:19,313][105692] Updated weights for policy 0, policy_version 1152959 (0.0009) [2023-12-26 23:46:19,414][105620] Updated weights for policy 1, policy_version 1154353 (0.0008) [2023-12-26 23:46:19,484][105620] Updated weights for policy 1, policy_version 1154363 (0.0008) [2023-12-26 23:46:19,552][105620] Updated weights for policy 1, policy_version 1154373 (0.0008) [2023-12-26 23:46:19,970][105692] Updated weights for policy 0, policy_version 1152969 (0.0009) [2023-12-26 23:46:20,035][105692] Updated weights for policy 0, policy_version 1152979 (0.0009) [2023-12-26 23:46:20,097][105692] Updated weights for policy 0, policy_version 1152989 (0.0008) [2023-12-26 23:46:20,160][105692] Updated weights for policy 0, policy_version 1152999 (0.0009) [2023-12-26 23:46:20,246][105620] Updated weights for policy 1, policy_version 1154383 (0.0010) [2023-12-26 23:46:20,303][105620] Updated weights for policy 1, policy_version 1154393 (0.0010) [2023-12-26 23:46:20,369][105620] Updated weights for policy 1, policy_version 1154403 (0.0011) [2023-12-26 23:46:20,956][105692] Updated weights for policy 0, policy_version 1153009 (0.0009) [2023-12-26 23:46:21,011][105692] Updated weights for policy 0, policy_version 1153019 (0.0012) [2023-12-26 23:46:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19494.2). Total num frames: 590782464. Throughput: 0: 9372.9, 1: 9881.1. Samples: 590778924. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:46:21,062][104569] Avg episode reward: [(0, '9003.912'), (1, '9168.270')] [2023-12-26 23:46:21,078][105692] Updated weights for policy 0, policy_version 1153029 (0.0007) [2023-12-26 23:46:21,084][105620] Updated weights for policy 1, policy_version 1154413 (0.0010) [2023-12-26 23:46:21,151][105620] Updated weights for policy 1, policy_version 1154423 (0.0010) [2023-12-26 23:46:21,210][105620] Updated weights for policy 1, policy_version 1154433 (0.0010) [2023-12-26 23:46:21,771][105692] Updated weights for policy 0, policy_version 1153039 (0.0009) [2023-12-26 23:46:21,829][105692] Updated weights for policy 0, policy_version 1153049 (0.0010) [2023-12-26 23:46:21,887][105692] Updated weights for policy 0, policy_version 1153059 (0.0010) [2023-12-26 23:46:21,918][105620] Updated weights for policy 1, policy_version 1154443 (0.0009) [2023-12-26 23:46:21,981][105620] Updated weights for policy 1, policy_version 1154453 (0.0006) [2023-12-26 23:46:22,048][105620] Updated weights for policy 1, policy_version 1154463 (0.0008) [2023-12-26 23:46:22,705][105620] Updated weights for policy 1, policy_version 1154473 (0.0008) [2023-12-26 23:46:22,755][105620] Updated weights for policy 1, policy_version 1154483 (0.0005) [2023-12-26 23:46:22,778][105692] Updated weights for policy 0, policy_version 1153069 (0.0010) [2023-12-26 23:46:22,805][105620] Updated weights for policy 1, policy_version 1154493 (0.0005) [2023-12-26 23:46:22,843][105692] Updated weights for policy 0, policy_version 1153079 (0.0009) [2023-12-26 23:46:22,864][105620] Updated weights for policy 1, policy_version 1154503 (0.0005) [2023-12-26 23:46:22,910][105692] Updated weights for policy 0, policy_version 1153089 (0.0009) [2023-12-26 23:46:23,490][105620] Updated weights for policy 1, policy_version 1154513 (0.0007) [2023-12-26 23:46:23,540][105620] Updated weights for policy 1, policy_version 1154523 (0.0008) [2023-12-26 23:46:23,600][105620] Updated weights for policy 1, policy_version 1154533 (0.0007) [2023-12-26 23:46:23,678][105692] Updated weights for policy 0, policy_version 1153099 (0.0007) [2023-12-26 23:46:23,747][105692] Updated weights for policy 0, policy_version 1153109 (0.0009) [2023-12-26 23:46:23,802][105692] Updated weights for policy 0, policy_version 1153119 (0.0008) [2023-12-26 23:46:24,198][105620] Updated weights for policy 1, policy_version 1154543 (0.0009) [2023-12-26 23:46:24,259][105620] Updated weights for policy 1, policy_version 1154553 (0.0006) [2023-12-26 23:46:24,324][105620] Updated weights for policy 1, policy_version 1154563 (0.0005) [2023-12-26 23:46:24,577][105692] Updated weights for policy 0, policy_version 1153129 (0.0008) [2023-12-26 23:46:24,632][105692] Updated weights for policy 0, policy_version 1153139 (0.0009) [2023-12-26 23:46:24,686][105692] Updated weights for policy 0, policy_version 1153149 (0.0010) [2023-12-26 23:46:24,738][105692] Updated weights for policy 0, policy_version 1153159 (0.0009) [2023-12-26 23:46:24,873][105620] Updated weights for policy 1, policy_version 1154573 (0.0005) [2023-12-26 23:46:24,939][105620] Updated weights for policy 1, policy_version 1154583 (0.0008) [2023-12-26 23:46:24,990][105620] Updated weights for policy 1, policy_version 1154593 (0.0010) [2023-12-26 23:46:25,583][105692] Updated weights for policy 0, policy_version 1153169 (0.0006) [2023-12-26 23:46:25,637][105692] Updated weights for policy 0, policy_version 1153179 (0.0005) [2023-12-26 23:46:25,662][105620] Updated weights for policy 1, policy_version 1154603 (0.0008) [2023-12-26 23:46:25,693][105692] Updated weights for policy 0, policy_version 1153189 (0.0005) [2023-12-26 23:46:25,711][105620] Updated weights for policy 1, policy_version 1154613 (0.0008) [2023-12-26 23:46:25,765][105620] Updated weights for policy 1, policy_version 1154623 (0.0009) [2023-12-26 23:46:26,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 590888960. Throughput: 0: 9338.9, 1: 9808.6. Samples: 590895508. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:46:26,062][104569] Avg episode reward: [(0, '9187.353'), (1, '9168.288')] [2023-12-26 23:46:26,369][105620] Updated weights for policy 1, policy_version 1154633 (0.0007) [2023-12-26 23:46:26,429][105692] Updated weights for policy 0, policy_version 1153199 (0.0008) [2023-12-26 23:46:26,429][105620] Updated weights for policy 1, policy_version 1154643 (0.0005) [2023-12-26 23:46:26,476][105620] Updated weights for policy 1, policy_version 1154653 (0.0008) [2023-12-26 23:46:26,490][105692] Updated weights for policy 0, policy_version 1153209 (0.0008) [2023-12-26 23:46:26,522][105620] Updated weights for policy 1, policy_version 1154663 (0.0007) [2023-12-26 23:46:26,546][105692] Updated weights for policy 0, policy_version 1153219 (0.0009) [2023-12-26 23:46:27,209][105620] Updated weights for policy 1, policy_version 1154673 (0.0006) [2023-12-26 23:46:27,264][105620] Updated weights for policy 1, policy_version 1154683 (0.0006) [2023-12-26 23:46:27,323][105620] Updated weights for policy 1, policy_version 1154693 (0.0006) [2023-12-26 23:46:27,331][105692] Updated weights for policy 0, policy_version 1153229 (0.0008) [2023-12-26 23:46:27,387][105692] Updated weights for policy 0, policy_version 1153239 (0.0009) [2023-12-26 23:46:27,431][105692] Updated weights for policy 0, policy_version 1153249 (0.0008) [2023-12-26 23:46:28,010][105620] Updated weights for policy 1, policy_version 1154703 (0.0009) [2023-12-26 23:46:28,054][105620] Updated weights for policy 1, policy_version 1154713 (0.0007) [2023-12-26 23:46:28,114][105620] Updated weights for policy 1, policy_version 1154723 (0.0009) [2023-12-26 23:46:28,205][105692] Updated weights for policy 0, policy_version 1153259 (0.0009) [2023-12-26 23:46:28,269][105692] Updated weights for policy 0, policy_version 1153269 (0.0009) [2023-12-26 23:46:28,331][105692] Updated weights for policy 0, policy_version 1153279 (0.0009) [2023-12-26 23:46:28,861][105620] Updated weights for policy 1, policy_version 1154733 (0.0008) [2023-12-26 23:46:28,908][105620] Updated weights for policy 1, policy_version 1154743 (0.0009) [2023-12-26 23:46:28,961][105620] Updated weights for policy 1, policy_version 1154754 (0.0007) [2023-12-26 23:46:29,060][105692] Updated weights for policy 0, policy_version 1153289 (0.0009) [2023-12-26 23:46:29,114][105692] Updated weights for policy 0, policy_version 1153299 (0.0009) [2023-12-26 23:46:29,162][105692] Updated weights for policy 0, policy_version 1153309 (0.0009) [2023-12-26 23:46:29,221][105692] Updated weights for policy 0, policy_version 1153319 (0.0009) [2023-12-26 23:46:29,727][105620] Updated weights for policy 1, policy_version 1154764 (0.0009) [2023-12-26 23:46:29,789][105620] Updated weights for policy 1, policy_version 1154774 (0.0007) [2023-12-26 23:46:29,855][105620] Updated weights for policy 1, policy_version 1154784 (0.0008) [2023-12-26 23:46:30,045][105692] Updated weights for policy 0, policy_version 1153329 (0.0010) [2023-12-26 23:46:30,094][105692] Updated weights for policy 0, policy_version 1153339 (0.0011) [2023-12-26 23:46:30,150][105692] Updated weights for policy 0, policy_version 1153349 (0.0011) [2023-12-26 23:46:30,513][105620] Updated weights for policy 1, policy_version 1154794 (0.0008) [2023-12-26 23:46:30,572][105620] Updated weights for policy 1, policy_version 1154804 (0.0006) [2023-12-26 23:46:30,633][105620] Updated weights for policy 1, policy_version 1154814 (0.0007) [2023-12-26 23:46:30,694][105620] Updated weights for policy 1, policy_version 1154824 (0.0006) [2023-12-26 23:46:30,830][105692] Updated weights for policy 0, policy_version 1153359 (0.0010) [2023-12-26 23:46:30,885][105692] Updated weights for policy 0, policy_version 1153369 (0.0010) [2023-12-26 23:46:30,937][105692] Updated weights for policy 0, policy_version 1153379 (0.0010) [2023-12-26 23:46:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 590987264. Throughput: 0: 9300.4, 1: 9843.9. Samples: 590953652. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:46:31,062][104569] Avg episode reward: [(0, '9262.491'), (1, '9350.638')] [2023-12-26 23:46:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001153384_295313408.pth... [2023-12-26 23:46:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001154824_295673856.pth... [2023-12-26 23:46:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001152296_295034880.pth [2023-12-26 23:46:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001153672_295378944.pth [2023-12-26 23:46:31,292][105620] Updated weights for policy 1, policy_version 1154834 (0.0006) [2023-12-26 23:46:31,349][105620] Updated weights for policy 1, policy_version 1154844 (0.0007) [2023-12-26 23:46:31,408][105620] Updated weights for policy 1, policy_version 1154854 (0.0006) [2023-12-26 23:46:31,745][105692] Updated weights for policy 0, policy_version 1153389 (0.0009) [2023-12-26 23:46:31,794][105692] Updated weights for policy 0, policy_version 1153399 (0.0005) [2023-12-26 23:46:31,848][105692] Updated weights for policy 0, policy_version 1153409 (0.0007) [2023-12-26 23:46:32,026][105620] Updated weights for policy 1, policy_version 1154864 (0.0005) [2023-12-26 23:46:32,082][105620] Updated weights for policy 1, policy_version 1154874 (0.0006) [2023-12-26 23:46:32,147][105620] Updated weights for policy 1, policy_version 1154884 (0.0007) [2023-12-26 23:46:32,509][105692] Updated weights for policy 0, policy_version 1153419 (0.0007) [2023-12-26 23:46:32,561][105692] Updated weights for policy 0, policy_version 1153429 (0.0010) [2023-12-26 23:46:32,614][105692] Updated weights for policy 0, policy_version 1153439 (0.0011) [2023-12-26 23:46:32,767][105620] Updated weights for policy 1, policy_version 1154894 (0.0005) [2023-12-26 23:46:32,818][105620] Updated weights for policy 1, policy_version 1154904 (0.0008) [2023-12-26 23:46:32,869][105620] Updated weights for policy 1, policy_version 1154914 (0.0010) [2023-12-26 23:46:33,338][105692] Updated weights for policy 0, policy_version 1153449 (0.0011) [2023-12-26 23:46:33,386][105692] Updated weights for policy 0, policy_version 1153459 (0.0010) [2023-12-26 23:46:33,437][105692] Updated weights for policy 0, policy_version 1153469 (0.0010) [2023-12-26 23:46:33,484][105692] Updated weights for policy 0, policy_version 1153479 (0.0010) [2023-12-26 23:46:33,594][105620] Updated weights for policy 1, policy_version 1154924 (0.0010) [2023-12-26 23:46:33,649][105620] Updated weights for policy 1, policy_version 1154934 (0.0010) [2023-12-26 23:46:33,700][105620] Updated weights for policy 1, policy_version 1154944 (0.0010) [2023-12-26 23:46:34,134][105692] Updated weights for policy 0, policy_version 1153489 (0.0006) [2023-12-26 23:46:34,201][105692] Updated weights for policy 0, policy_version 1153499 (0.0007) [2023-12-26 23:46:34,264][105692] Updated weights for policy 0, policy_version 1153509 (0.0008) [2023-12-26 23:46:34,427][105620] Updated weights for policy 1, policy_version 1154954 (0.0009) [2023-12-26 23:46:34,488][105620] Updated weights for policy 1, policy_version 1154964 (0.0006) [2023-12-26 23:46:34,543][105620] Updated weights for policy 1, policy_version 1154974 (0.0005) [2023-12-26 23:46:34,597][105620] Updated weights for policy 1, policy_version 1154984 (0.0005) [2023-12-26 23:46:34,985][105692] Updated weights for policy 0, policy_version 1153519 (0.0009) [2023-12-26 23:46:35,045][105692] Updated weights for policy 0, policy_version 1153529 (0.0006) [2023-12-26 23:46:35,098][105692] Updated weights for policy 0, policy_version 1153539 (0.0008) [2023-12-26 23:46:35,265][105620] Updated weights for policy 1, policy_version 1154994 (0.0009) [2023-12-26 23:46:35,311][105620] Updated weights for policy 1, policy_version 1155004 (0.0008) [2023-12-26 23:46:35,364][105620] Updated weights for policy 1, policy_version 1155014 (0.0008) [2023-12-26 23:46:35,809][105692] Updated weights for policy 0, policy_version 1153549 (0.0008) [2023-12-26 23:46:35,875][105692] Updated weights for policy 0, policy_version 1153559 (0.0009) [2023-12-26 23:46:35,931][105692] Updated weights for policy 0, policy_version 1153569 (0.0008) [2023-12-26 23:46:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 591085568. Throughput: 0: 9360.3, 1: 9887.1. Samples: 591074528. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:46:36,062][104569] Avg episode reward: [(0, '9356.897'), (1, '9350.850')] [2023-12-26 23:46:36,196][105620] Updated weights for policy 1, policy_version 1155024 (0.0008) [2023-12-26 23:46:36,262][105620] Updated weights for policy 1, policy_version 1155034 (0.0008) [2023-12-26 23:46:36,325][105620] Updated weights for policy 1, policy_version 1155044 (0.0008) [2023-12-26 23:46:36,707][105692] Updated weights for policy 0, policy_version 1153579 (0.0006) [2023-12-26 23:46:36,763][105692] Updated weights for policy 0, policy_version 1153589 (0.0010) [2023-12-26 23:46:36,826][105692] Updated weights for policy 0, policy_version 1153599 (0.0009) [2023-12-26 23:46:36,996][105620] Updated weights for policy 1, policy_version 1155054 (0.0009) [2023-12-26 23:46:37,053][105620] Updated weights for policy 1, policy_version 1155064 (0.0008) [2023-12-26 23:46:37,123][105620] Updated weights for policy 1, policy_version 1155074 (0.0011) [2023-12-26 23:46:37,611][105692] Updated weights for policy 0, policy_version 1153609 (0.0009) [2023-12-26 23:46:37,675][105692] Updated weights for policy 0, policy_version 1153619 (0.0008) [2023-12-26 23:46:37,742][105692] Updated weights for policy 0, policy_version 1153629 (0.0005) [2023-12-26 23:46:37,802][105692] Updated weights for policy 0, policy_version 1153639 (0.0008) [2023-12-26 23:46:37,863][105620] Updated weights for policy 1, policy_version 1155084 (0.0011) [2023-12-26 23:46:37,924][105620] Updated weights for policy 1, policy_version 1155094 (0.0008) [2023-12-26 23:46:37,986][105620] Updated weights for policy 1, policy_version 1155104 (0.0011) [2023-12-26 23:46:38,489][105692] Updated weights for policy 0, policy_version 1153649 (0.0008) [2023-12-26 23:46:38,540][105692] Updated weights for policy 0, policy_version 1153659 (0.0008) [2023-12-26 23:46:38,600][105692] Updated weights for policy 0, policy_version 1153669 (0.0008) [2023-12-26 23:46:38,726][105620] Updated weights for policy 1, policy_version 1155114 (0.0011) [2023-12-26 23:46:38,778][105620] Updated weights for policy 1, policy_version 1155124 (0.0011) [2023-12-26 23:46:38,830][105620] Updated weights for policy 1, policy_version 1155134 (0.0011) [2023-12-26 23:46:38,886][105620] Updated weights for policy 1, policy_version 1155144 (0.0011) [2023-12-26 23:46:39,379][105692] Updated weights for policy 0, policy_version 1153679 (0.0008) [2023-12-26 23:46:39,442][105692] Updated weights for policy 0, policy_version 1153689 (0.0008) [2023-12-26 23:46:39,494][105692] Updated weights for policy 0, policy_version 1153699 (0.0008) [2023-12-26 23:46:39,670][105620] Updated weights for policy 1, policy_version 1155154 (0.0011) [2023-12-26 23:46:39,740][105620] Updated weights for policy 1, policy_version 1155164 (0.0011) [2023-12-26 23:46:39,804][105620] Updated weights for policy 1, policy_version 1155174 (0.0011) [2023-12-26 23:46:40,249][105692] Updated weights for policy 0, policy_version 1153709 (0.0008) [2023-12-26 23:46:40,316][105692] Updated weights for policy 0, policy_version 1153719 (0.0009) [2023-12-26 23:46:40,379][105692] Updated weights for policy 0, policy_version 1153729 (0.0010) [2023-12-26 23:46:40,575][105620] Updated weights for policy 1, policy_version 1155184 (0.0007) [2023-12-26 23:46:40,636][105620] Updated weights for policy 1, policy_version 1155194 (0.0006) [2023-12-26 23:46:40,697][105620] Updated weights for policy 1, policy_version 1155204 (0.0009) [2023-12-26 23:46:41,007][105692] Updated weights for policy 0, policy_version 1153739 (0.0006) [2023-12-26 23:46:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19494.2). Total num frames: 591175680. Throughput: 0: 9347.3, 1: 9881.6. Samples: 591186968. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:46:41,062][104569] Avg episode reward: [(0, '9356.944'), (1, '9350.923')] [2023-12-26 23:46:41,099][105692] Updated weights for policy 0, policy_version 1153749 (0.0006) [2023-12-26 23:46:41,171][105692] Updated weights for policy 0, policy_version 1153759 (0.0008) [2023-12-26 23:46:41,405][105620] Updated weights for policy 1, policy_version 1155214 (0.0009) [2023-12-26 23:46:41,457][105620] Updated weights for policy 1, policy_version 1155224 (0.0008) [2023-12-26 23:46:41,502][105620] Updated weights for policy 1, policy_version 1155234 (0.0008) [2023-12-26 23:46:41,898][105692] Updated weights for policy 0, policy_version 1153769 (0.0008) [2023-12-26 23:46:41,959][105692] Updated weights for policy 0, policy_version 1153779 (0.0009) [2023-12-26 23:46:42,015][105692] Updated weights for policy 0, policy_version 1153789 (0.0010) [2023-12-26 23:46:42,065][105692] Updated weights for policy 0, policy_version 1153799 (0.0009) [2023-12-26 23:46:42,332][105620] Updated weights for policy 1, policy_version 1155244 (0.0008) [2023-12-26 23:46:42,404][105620] Updated weights for policy 1, policy_version 1155254 (0.0008) [2023-12-26 23:46:42,456][105620] Updated weights for policy 1, policy_version 1155264 (0.0010) [2023-12-26 23:46:42,805][105692] Updated weights for policy 0, policy_version 1153809 (0.0010) [2023-12-26 23:46:42,864][105692] Updated weights for policy 0, policy_version 1153819 (0.0010) [2023-12-26 23:46:42,927][105692] Updated weights for policy 0, policy_version 1153829 (0.0010) [2023-12-26 23:46:43,150][105620] Updated weights for policy 1, policy_version 1155274 (0.0009) [2023-12-26 23:46:43,200][105620] Updated weights for policy 1, policy_version 1155284 (0.0010) [2023-12-26 23:46:43,258][105620] Updated weights for policy 1, policy_version 1155294 (0.0010) [2023-12-26 23:46:43,316][105620] Updated weights for policy 1, policy_version 1155304 (0.0010) [2023-12-26 23:46:43,646][105692] Updated weights for policy 0, policy_version 1153839 (0.0007) [2023-12-26 23:46:43,700][105692] Updated weights for policy 0, policy_version 1153849 (0.0007) [2023-12-26 23:46:43,754][105692] Updated weights for policy 0, policy_version 1153859 (0.0007) [2023-12-26 23:46:44,050][105620] Updated weights for policy 1, policy_version 1155314 (0.0010) [2023-12-26 23:46:44,114][105620] Updated weights for policy 1, policy_version 1155324 (0.0007) [2023-12-26 23:46:44,168][105620] Updated weights for policy 1, policy_version 1155334 (0.0005) [2023-12-26 23:46:44,544][105692] Updated weights for policy 0, policy_version 1153869 (0.0008) [2023-12-26 23:46:44,596][105692] Updated weights for policy 0, policy_version 1153879 (0.0005) [2023-12-26 23:46:44,649][105692] Updated weights for policy 0, policy_version 1153889 (0.0005) [2023-12-26 23:46:44,787][105620] Updated weights for policy 1, policy_version 1155344 (0.0009) [2023-12-26 23:46:44,848][105620] Updated weights for policy 1, policy_version 1155354 (0.0009) [2023-12-26 23:46:44,904][105620] Updated weights for policy 1, policy_version 1155364 (0.0008) [2023-12-26 23:46:45,353][105692] Updated weights for policy 0, policy_version 1153899 (0.0008) [2023-12-26 23:46:45,405][105692] Updated weights for policy 0, policy_version 1153909 (0.0010) [2023-12-26 23:46:45,466][105692] Updated weights for policy 0, policy_version 1153919 (0.0010) [2023-12-26 23:46:45,550][105620] Updated weights for policy 1, policy_version 1155374 (0.0007) [2023-12-26 23:46:45,612][105620] Updated weights for policy 1, policy_version 1155384 (0.0008) [2023-12-26 23:46:45,674][105620] Updated weights for policy 1, policy_version 1155394 (0.0008) [2023-12-26 23:46:46,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 591273984. Throughput: 0: 9322.5, 1: 9840.1. Samples: 591243808. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:46:46,063][104569] Avg episode reward: [(0, '9356.533'), (1, '6402.007')] [2023-12-26 23:46:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001153928_295452672.pth... [2023-12-26 23:46:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001155400_295821312.pth... [2023-12-26 23:46:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001152840_295174144.pth [2023-12-26 23:46:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001154216_295518208.pth [2023-12-26 23:46:46,152][105692] Updated weights for policy 0, policy_version 1153929 (0.0010) [2023-12-26 23:46:46,213][105692] Updated weights for policy 0, policy_version 1153939 (0.0010) [2023-12-26 23:46:46,267][105692] Updated weights for policy 0, policy_version 1153949 (0.0010) [2023-12-26 23:46:46,315][105692] Updated weights for policy 0, policy_version 1153959 (0.0010) [2023-12-26 23:46:46,418][105620] Updated weights for policy 1, policy_version 1155404 (0.0007) [2023-12-26 23:46:46,473][105620] Updated weights for policy 1, policy_version 1155414 (0.0007) [2023-12-26 23:46:46,521][105620] Updated weights for policy 1, policy_version 1155424 (0.0008) [2023-12-26 23:46:47,061][105692] Updated weights for policy 0, policy_version 1153969 (0.0010) [2023-12-26 23:46:47,113][105692] Updated weights for policy 0, policy_version 1153979 (0.0010) [2023-12-26 23:46:47,169][105692] Updated weights for policy 0, policy_version 1153989 (0.0007) [2023-12-26 23:46:47,266][105620] Updated weights for policy 1, policy_version 1155434 (0.0008) [2023-12-26 23:46:47,327][105620] Updated weights for policy 1, policy_version 1155444 (0.0008) [2023-12-26 23:46:47,378][105620] Updated weights for policy 1, policy_version 1155454 (0.0006) [2023-12-26 23:46:47,429][105620] Updated weights for policy 1, policy_version 1155464 (0.0005) [2023-12-26 23:46:47,914][105692] Updated weights for policy 0, policy_version 1153999 (0.0010) [2023-12-26 23:46:47,969][105692] Updated weights for policy 0, policy_version 1154009 (0.0010) [2023-12-26 23:46:48,020][105692] Updated weights for policy 0, policy_version 1154019 (0.0010) [2023-12-26 23:46:48,050][105620] Updated weights for policy 1, policy_version 1155474 (0.0006) [2023-12-26 23:46:48,102][105620] Updated weights for policy 1, policy_version 1155484 (0.0008) [2023-12-26 23:46:48,158][105620] Updated weights for policy 1, policy_version 1155494 (0.0008) [2023-12-26 23:46:48,773][105692] Updated weights for policy 0, policy_version 1154029 (0.0010) [2023-12-26 23:46:48,838][105692] Updated weights for policy 0, policy_version 1154039 (0.0010) [2023-12-26 23:46:48,893][105692] Updated weights for policy 0, policy_version 1154049 (0.0010) [2023-12-26 23:46:48,914][105620] Updated weights for policy 1, policy_version 1155504 (0.0010) [2023-12-26 23:46:48,979][105620] Updated weights for policy 1, policy_version 1155514 (0.0006) [2023-12-26 23:46:49,029][105620] Updated weights for policy 1, policy_version 1155524 (0.0008) [2023-12-26 23:46:49,615][105692] Updated weights for policy 0, policy_version 1154059 (0.0010) [2023-12-26 23:46:49,663][105692] Updated weights for policy 0, policy_version 1154069 (0.0010) [2023-12-26 23:46:49,723][105692] Updated weights for policy 0, policy_version 1154079 (0.0005) [2023-12-26 23:46:49,798][105620] Updated weights for policy 1, policy_version 1155534 (0.0007) [2023-12-26 23:46:49,864][105620] Updated weights for policy 1, policy_version 1155544 (0.0008) [2023-12-26 23:46:49,926][105620] Updated weights for policy 1, policy_version 1155554 (0.0006) [2023-12-26 23:46:50,373][105692] Updated weights for policy 0, policy_version 1154089 (0.0009) [2023-12-26 23:46:50,436][105692] Updated weights for policy 0, policy_version 1154099 (0.0011) [2023-12-26 23:46:50,496][105692] Updated weights for policy 0, policy_version 1154109 (0.0011) [2023-12-26 23:46:50,548][105692] Updated weights for policy 0, policy_version 1154119 (0.0010) [2023-12-26 23:46:50,739][105620] Updated weights for policy 1, policy_version 1155564 (0.0008) [2023-12-26 23:46:50,789][105620] Updated weights for policy 1, policy_version 1155574 (0.0008) [2023-12-26 23:46:50,844][105620] Updated weights for policy 1, policy_version 1155584 (0.0009) [2023-12-26 23:46:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 591372288. Throughput: 0: 9370.8, 1: 9955.2. Samples: 591360508. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:46:51,063][104569] Avg episode reward: [(0, '9265.326'), (1, '3499.896')] [2023-12-26 23:46:51,187][105692] Updated weights for policy 0, policy_version 1154129 (0.0009) [2023-12-26 23:46:51,235][105692] Updated weights for policy 0, policy_version 1154139 (0.0009) [2023-12-26 23:46:51,289][105692] Updated weights for policy 0, policy_version 1154149 (0.0009) [2023-12-26 23:46:51,664][105620] Updated weights for policy 1, policy_version 1155594 (0.0009) [2023-12-26 23:46:51,719][105620] Updated weights for policy 1, policy_version 1155604 (0.0009) [2023-12-26 23:46:51,794][105620] Updated weights for policy 1, policy_version 1155614 (0.0009) [2023-12-26 23:46:51,852][105620] Updated weights for policy 1, policy_version 1155624 (0.0009) [2023-12-26 23:46:52,098][105692] Updated weights for policy 0, policy_version 1154159 (0.0007) [2023-12-26 23:46:52,150][105692] Updated weights for policy 0, policy_version 1154169 (0.0005) [2023-12-26 23:46:52,211][105692] Updated weights for policy 0, policy_version 1154179 (0.0007) [2023-12-26 23:46:52,619][105620] Updated weights for policy 1, policy_version 1155634 (0.0009) [2023-12-26 23:46:52,678][105620] Updated weights for policy 1, policy_version 1155644 (0.0008) [2023-12-26 23:46:52,736][105620] Updated weights for policy 1, policy_version 1155654 (0.0008) [2023-12-26 23:46:52,954][105692] Updated weights for policy 0, policy_version 1154189 (0.0007) [2023-12-26 23:46:53,019][105692] Updated weights for policy 0, policy_version 1154199 (0.0005) [2023-12-26 23:46:53,073][105692] Updated weights for policy 0, policy_version 1154209 (0.0005) [2023-12-26 23:46:53,569][105692] Updated weights for policy 0, policy_version 1154219 (0.0005) [2023-12-26 23:46:53,582][105620] Updated weights for policy 1, policy_version 1155664 (0.0009) [2023-12-26 23:46:53,626][105692] Updated weights for policy 0, policy_version 1154229 (0.0005) [2023-12-26 23:46:53,639][105620] Updated weights for policy 1, policy_version 1155674 (0.0009) [2023-12-26 23:46:53,687][105692] Updated weights for policy 0, policy_version 1154239 (0.0005) [2023-12-26 23:46:53,688][105620] Updated weights for policy 1, policy_version 1155684 (0.0010) [2023-12-26 23:46:54,198][105692] Updated weights for policy 0, policy_version 1154249 (0.0006) [2023-12-26 23:46:54,253][105692] Updated weights for policy 0, policy_version 1154259 (0.0011) [2023-12-26 23:46:54,308][105692] Updated weights for policy 0, policy_version 1154269 (0.0010) [2023-12-26 23:46:54,361][105692] Updated weights for policy 0, policy_version 1154279 (0.0010) [2023-12-26 23:46:54,520][105620] Updated weights for policy 1, policy_version 1155694 (0.0008) [2023-12-26 23:46:54,568][105620] Updated weights for policy 1, policy_version 1155704 (0.0008) [2023-12-26 23:46:54,616][105620] Updated weights for policy 1, policy_version 1155714 (0.0008) [2023-12-26 23:46:55,057][105692] Updated weights for policy 0, policy_version 1154289 (0.0006) [2023-12-26 23:46:55,115][105692] Updated weights for policy 0, policy_version 1154299 (0.0005) [2023-12-26 23:46:55,168][105692] Updated weights for policy 0, policy_version 1154309 (0.0007) [2023-12-26 23:46:55,501][105620] Updated weights for policy 1, policy_version 1155724 (0.0007) [2023-12-26 23:46:55,554][105620] Updated weights for policy 1, policy_version 1155734 (0.0008) [2023-12-26 23:46:55,611][105620] Updated weights for policy 1, policy_version 1155745 (0.0010) [2023-12-26 23:46:55,716][105692] Updated weights for policy 0, policy_version 1154319 (0.0006) [2023-12-26 23:46:55,784][105692] Updated weights for policy 0, policy_version 1154329 (0.0005) [2023-12-26 23:46:55,849][105692] Updated weights for policy 0, policy_version 1154339 (0.0005) [2023-12-26 23:46:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 591470592. Throughput: 0: 9535.3, 1: 9750.0. Samples: 591476112. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:46:56,062][104569] Avg episode reward: [(0, '9084.616'), (1, '6370.194')] [2023-12-26 23:46:56,428][105620] Updated weights for policy 1, policy_version 1155756 (0.0009) [2023-12-26 23:46:56,482][105620] Updated weights for policy 1, policy_version 1155766 (0.0008) [2023-12-26 23:46:56,489][105692] Updated weights for policy 0, policy_version 1154349 (0.0010) [2023-12-26 23:46:56,532][105692] Updated weights for policy 0, policy_version 1154359 (0.0007) [2023-12-26 23:46:56,534][105620] Updated weights for policy 1, policy_version 1155776 (0.0007) [2023-12-26 23:46:56,579][105692] Updated weights for policy 0, policy_version 1154369 (0.0009) [2023-12-26 23:46:57,283][105620] Updated weights for policy 1, policy_version 1155786 (0.0006) [2023-12-26 23:46:57,323][105692] Updated weights for policy 0, policy_version 1154379 (0.0010) [2023-12-26 23:46:57,344][105620] Updated weights for policy 1, policy_version 1155796 (0.0007) [2023-12-26 23:46:57,375][105692] Updated weights for policy 0, policy_version 1154389 (0.0010) [2023-12-26 23:46:57,401][105620] Updated weights for policy 1, policy_version 1155806 (0.0006) [2023-12-26 23:46:57,426][105692] Updated weights for policy 0, policy_version 1154399 (0.0010) [2023-12-26 23:46:57,448][105620] Updated weights for policy 1, policy_version 1155816 (0.0005) [2023-12-26 23:46:58,171][105692] Updated weights for policy 0, policy_version 1154409 (0.0010) [2023-12-26 23:46:58,188][105620] Updated weights for policy 1, policy_version 1155826 (0.0008) [2023-12-26 23:46:58,229][105692] Updated weights for policy 0, policy_version 1154419 (0.0011) [2023-12-26 23:46:58,241][105620] Updated weights for policy 1, policy_version 1155836 (0.0008) [2023-12-26 23:46:58,292][105692] Updated weights for policy 0, policy_version 1154429 (0.0010) [2023-12-26 23:46:58,304][105620] Updated weights for policy 1, policy_version 1155846 (0.0008) [2023-12-26 23:46:58,348][105692] Updated weights for policy 0, policy_version 1154439 (0.0009) [2023-12-26 23:46:59,063][105620] Updated weights for policy 1, policy_version 1155856 (0.0009) [2023-12-26 23:46:59,122][105620] Updated weights for policy 1, policy_version 1155866 (0.0011) [2023-12-26 23:46:59,183][105620] Updated weights for policy 1, policy_version 1155876 (0.0011) [2023-12-26 23:46:59,204][105692] Updated weights for policy 0, policy_version 1154449 (0.0007) [2023-12-26 23:46:59,266][105692] Updated weights for policy 0, policy_version 1154459 (0.0008) [2023-12-26 23:46:59,333][105692] Updated weights for policy 0, policy_version 1154469 (0.0007) [2023-12-26 23:46:59,947][105620] Updated weights for policy 1, policy_version 1155886 (0.0009) [2023-12-26 23:47:00,014][105620] Updated weights for policy 1, policy_version 1155896 (0.0007) [2023-12-26 23:47:00,023][105692] Updated weights for policy 0, policy_version 1154479 (0.0009) [2023-12-26 23:47:00,068][105692] Updated weights for policy 0, policy_version 1154489 (0.0006) [2023-12-26 23:47:00,081][105620] Updated weights for policy 1, policy_version 1155906 (0.0008) [2023-12-26 23:47:00,123][105692] Updated weights for policy 0, policy_version 1154499 (0.0005) [2023-12-26 23:47:00,627][105620] Updated weights for policy 1, policy_version 1155916 (0.0006) [2023-12-26 23:47:00,675][105620] Updated weights for policy 1, policy_version 1155926 (0.0005) [2023-12-26 23:47:00,727][105692] Updated weights for policy 0, policy_version 1154509 (0.0006) [2023-12-26 23:47:00,731][105620] Updated weights for policy 1, policy_version 1155936 (0.0005) [2023-12-26 23:47:00,781][105692] Updated weights for policy 0, policy_version 1154519 (0.0005) [2023-12-26 23:47:00,827][105692] Updated weights for policy 0, policy_version 1154529 (0.0005) [2023-12-26 23:47:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 591568896. Throughput: 0: 9580.7, 1: 9760.6. Samples: 591533500. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:47:01,063][104569] Avg episode reward: [(0, '9174.932'), (1, '8452.867')] [2023-12-26 23:47:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001154536_295608320.pth... [2023-12-26 23:47:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001155944_295960576.pth... [2023-12-26 23:47:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001153384_295313408.pth [2023-12-26 23:47:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001154824_295673856.pth [2023-12-26 23:47:01,412][105692] Updated weights for policy 0, policy_version 1154539 (0.0005) [2023-12-26 23:47:01,414][105620] Updated weights for policy 1, policy_version 1155946 (0.0008) [2023-12-26 23:47:01,472][105620] Updated weights for policy 1, policy_version 1155956 (0.0007) [2023-12-26 23:47:01,477][105692] Updated weights for policy 0, policy_version 1154549 (0.0007) [2023-12-26 23:47:01,535][105620] Updated weights for policy 1, policy_version 1155966 (0.0008) [2023-12-26 23:47:01,540][105692] Updated weights for policy 0, policy_version 1154559 (0.0006) [2023-12-26 23:47:01,587][105620] Updated weights for policy 1, policy_version 1155976 (0.0008) [2023-12-26 23:47:02,279][105692] Updated weights for policy 0, policy_version 1154569 (0.0006) [2023-12-26 23:47:02,322][105620] Updated weights for policy 1, policy_version 1155986 (0.0007) [2023-12-26 23:47:02,335][105692] Updated weights for policy 0, policy_version 1154579 (0.0007) [2023-12-26 23:47:02,393][105620] Updated weights for policy 1, policy_version 1155996 (0.0007) [2023-12-26 23:47:02,406][105692] Updated weights for policy 0, policy_version 1154589 (0.0008) [2023-12-26 23:47:02,454][105620] Updated weights for policy 1, policy_version 1156006 (0.0007) [2023-12-26 23:47:02,468][105692] Updated weights for policy 0, policy_version 1154599 (0.0009) [2023-12-26 23:47:03,139][105692] Updated weights for policy 0, policy_version 1154609 (0.0006) [2023-12-26 23:47:03,192][105692] Updated weights for policy 0, policy_version 1154619 (0.0005) [2023-12-26 23:47:03,238][105692] Updated weights for policy 0, policy_version 1154629 (0.0005) [2023-12-26 23:47:03,239][105620] Updated weights for policy 1, policy_version 1156016 (0.0008) [2023-12-26 23:47:03,287][105620] Updated weights for policy 1, policy_version 1156026 (0.0010) [2023-12-26 23:47:03,331][105620] Updated weights for policy 1, policy_version 1156036 (0.0010) [2023-12-26 23:47:03,850][105692] Updated weights for policy 0, policy_version 1154639 (0.0007) [2023-12-26 23:47:03,909][105692] Updated weights for policy 0, policy_version 1154649 (0.0008) [2023-12-26 23:47:03,965][105692] Updated weights for policy 0, policy_version 1154659 (0.0009) [2023-12-26 23:47:04,076][105620] Updated weights for policy 1, policy_version 1156046 (0.0007) [2023-12-26 23:47:04,139][105620] Updated weights for policy 1, policy_version 1156056 (0.0011) [2023-12-26 23:47:04,198][105620] Updated weights for policy 1, policy_version 1156066 (0.0010) [2023-12-26 23:47:04,614][105692] Updated weights for policy 0, policy_version 1154669 (0.0007) [2023-12-26 23:47:04,670][105692] Updated weights for policy 0, policy_version 1154679 (0.0008) [2023-12-26 23:47:04,723][105692] Updated weights for policy 0, policy_version 1154689 (0.0008) [2023-12-26 23:47:04,922][105620] Updated weights for policy 1, policy_version 1156076 (0.0009) [2023-12-26 23:47:04,988][105620] Updated weights for policy 1, policy_version 1156086 (0.0011) [2023-12-26 23:47:05,045][105620] Updated weights for policy 1, policy_version 1156096 (0.0011) [2023-12-26 23:47:05,391][105692] Updated weights for policy 0, policy_version 1154699 (0.0007) [2023-12-26 23:47:05,454][105692] Updated weights for policy 0, policy_version 1154709 (0.0005) [2023-12-26 23:47:05,521][105692] Updated weights for policy 0, policy_version 1154719 (0.0005) [2023-12-26 23:47:05,684][105620] Updated weights for policy 1, policy_version 1156106 (0.0010) [2023-12-26 23:47:05,746][105620] Updated weights for policy 1, policy_version 1156116 (0.0008) [2023-12-26 23:47:05,798][105620] Updated weights for policy 1, policy_version 1156126 (0.0010) [2023-12-26 23:47:05,853][105620] Updated weights for policy 1, policy_version 1156136 (0.0010) [2023-12-26 23:47:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 591667200. Throughput: 0: 9734.2, 1: 9697.8. Samples: 591653364. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:47:06,062][104569] Avg episode reward: [(0, '9263.604'), (1, '9258.513')] [2023-12-26 23:47:06,115][105692] Updated weights for policy 0, policy_version 1154729 (0.0006) [2023-12-26 23:47:06,177][105692] Updated weights for policy 0, policy_version 1154739 (0.0008) [2023-12-26 23:47:06,230][105692] Updated weights for policy 0, policy_version 1154749 (0.0008) [2023-12-26 23:47:06,291][105692] Updated weights for policy 0, policy_version 1154759 (0.0008) [2023-12-26 23:47:06,574][105620] Updated weights for policy 1, policy_version 1156146 (0.0010) [2023-12-26 23:47:06,627][105620] Updated weights for policy 1, policy_version 1156156 (0.0010) [2023-12-26 23:47:06,680][105620] Updated weights for policy 1, policy_version 1156166 (0.0011) [2023-12-26 23:47:07,092][105692] Updated weights for policy 0, policy_version 1154769 (0.0008) [2023-12-26 23:47:07,154][105692] Updated weights for policy 0, policy_version 1154779 (0.0008) [2023-12-26 23:47:07,215][105692] Updated weights for policy 0, policy_version 1154789 (0.0008) [2023-12-26 23:47:07,454][105620] Updated weights for policy 1, policy_version 1156176 (0.0011) [2023-12-26 23:47:07,512][105620] Updated weights for policy 1, policy_version 1156186 (0.0010) [2023-12-26 23:47:07,570][105620] Updated weights for policy 1, policy_version 1156196 (0.0010) [2023-12-26 23:47:07,985][105692] Updated weights for policy 0, policy_version 1154799 (0.0009) [2023-12-26 23:47:08,052][105692] Updated weights for policy 0, policy_version 1154809 (0.0005) [2023-12-26 23:47:08,109][105692] Updated weights for policy 0, policy_version 1154819 (0.0005) [2023-12-26 23:47:08,212][105620] Updated weights for policy 1, policy_version 1156206 (0.0007) [2023-12-26 23:47:08,268][105620] Updated weights for policy 1, policy_version 1156216 (0.0005) [2023-12-26 23:47:08,333][105620] Updated weights for policy 1, policy_version 1156226 (0.0006) [2023-12-26 23:47:08,680][105692] Updated weights for policy 0, policy_version 1154829 (0.0007) [2023-12-26 23:47:08,732][105692] Updated weights for policy 0, policy_version 1154839 (0.0007) [2023-12-26 23:47:08,782][105692] Updated weights for policy 0, policy_version 1154849 (0.0009) [2023-12-26 23:47:08,938][105620] Updated weights for policy 1, policy_version 1156236 (0.0007) [2023-12-26 23:47:09,006][105620] Updated weights for policy 1, policy_version 1156246 (0.0006) [2023-12-26 23:47:09,068][105620] Updated weights for policy 1, policy_version 1156256 (0.0011) [2023-12-26 23:47:09,403][105692] Updated weights for policy 0, policy_version 1154859 (0.0010) [2023-12-26 23:47:09,465][105692] Updated weights for policy 0, policy_version 1154869 (0.0007) [2023-12-26 23:47:09,527][105692] Updated weights for policy 0, policy_version 1154879 (0.0006) [2023-12-26 23:47:09,765][105620] Updated weights for policy 1, policy_version 1156266 (0.0010) [2023-12-26 23:47:09,833][105620] Updated weights for policy 1, policy_version 1156276 (0.0009) [2023-12-26 23:47:09,890][105620] Updated weights for policy 1, policy_version 1156286 (0.0008) [2023-12-26 23:47:09,947][105620] Updated weights for policy 1, policy_version 1156296 (0.0008) [2023-12-26 23:47:10,227][105692] Updated weights for policy 0, policy_version 1154889 (0.0006) [2023-12-26 23:47:10,286][105692] Updated weights for policy 0, policy_version 1154899 (0.0011) [2023-12-26 23:47:10,353][105692] Updated weights for policy 0, policy_version 1154909 (0.0010) [2023-12-26 23:47:10,413][105692] Updated weights for policy 0, policy_version 1154919 (0.0011) [2023-12-26 23:47:10,715][105620] Updated weights for policy 1, policy_version 1156306 (0.0006) [2023-12-26 23:47:10,771][105620] Updated weights for policy 1, policy_version 1156316 (0.0008) [2023-12-26 23:47:10,823][105620] Updated weights for policy 1, policy_version 1156326 (0.0008) [2023-12-26 23:47:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 591765504. Throughput: 0: 9882.2, 1: 9634.5. Samples: 591773760. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:47:11,063][104569] Avg episode reward: [(0, '9077.525'), (1, '9350.390')] [2023-12-26 23:47:11,129][105692] Updated weights for policy 0, policy_version 1154929 (0.0009) [2023-12-26 23:47:11,193][105692] Updated weights for policy 0, policy_version 1154939 (0.0008) [2023-12-26 23:47:11,258][105692] Updated weights for policy 0, policy_version 1154949 (0.0006) [2023-12-26 23:47:11,612][105620] Updated weights for policy 1, policy_version 1156336 (0.0010) [2023-12-26 23:47:11,680][105620] Updated weights for policy 1, policy_version 1156346 (0.0011) [2023-12-26 23:47:11,753][105620] Updated weights for policy 1, policy_version 1156356 (0.0010) [2023-12-26 23:47:11,998][105692] Updated weights for policy 0, policy_version 1154959 (0.0008) [2023-12-26 23:47:12,059][105692] Updated weights for policy 0, policy_version 1154969 (0.0008) [2023-12-26 23:47:12,130][105692] Updated weights for policy 0, policy_version 1154979 (0.0008) [2023-12-26 23:47:12,495][105620] Updated weights for policy 1, policy_version 1156366 (0.0010) [2023-12-26 23:47:12,555][105620] Updated weights for policy 1, policy_version 1156376 (0.0009) [2023-12-26 23:47:12,621][105620] Updated weights for policy 1, policy_version 1156386 (0.0009) [2023-12-26 23:47:12,838][105692] Updated weights for policy 0, policy_version 1154989 (0.0008) [2023-12-26 23:47:12,895][105692] Updated weights for policy 0, policy_version 1154999 (0.0007) [2023-12-26 23:47:12,962][105692] Updated weights for policy 0, policy_version 1155009 (0.0006) [2023-12-26 23:47:13,306][105620] Updated weights for policy 1, policy_version 1156396 (0.0009) [2023-12-26 23:47:13,368][105620] Updated weights for policy 1, policy_version 1156406 (0.0008) [2023-12-26 23:47:13,426][105620] Updated weights for policy 1, policy_version 1156416 (0.0008) [2023-12-26 23:47:13,699][105692] Updated weights for policy 0, policy_version 1155019 (0.0007) [2023-12-26 23:47:13,768][105692] Updated weights for policy 0, policy_version 1155029 (0.0005) [2023-12-26 23:47:13,832][105692] Updated weights for policy 0, policy_version 1155039 (0.0009) [2023-12-26 23:47:14,052][105620] Updated weights for policy 1, policy_version 1156426 (0.0006) [2023-12-26 23:47:14,112][105620] Updated weights for policy 1, policy_version 1156436 (0.0008) [2023-12-26 23:47:14,169][105620] Updated weights for policy 1, policy_version 1156446 (0.0009) [2023-12-26 23:47:14,438][105692] Updated weights for policy 0, policy_version 1155049 (0.0008) [2023-12-26 23:47:14,509][105692] Updated weights for policy 0, policy_version 1155059 (0.0006) [2023-12-26 23:47:14,577][105692] Updated weights for policy 0, policy_version 1155069 (0.0006) [2023-12-26 23:47:14,642][105692] Updated weights for policy 0, policy_version 1155079 (0.0006) [2023-12-26 23:47:15,038][105620] Updated weights for policy 1, policy_version 1156457 (0.0010) [2023-12-26 23:47:15,101][105620] Updated weights for policy 1, policy_version 1156467 (0.0010) [2023-12-26 23:47:15,170][105620] Updated weights for policy 1, policy_version 1156477 (0.0008) [2023-12-26 23:47:15,233][105620] Updated weights for policy 1, policy_version 1156487 (0.0008) [2023-12-26 23:47:15,302][105692] Updated weights for policy 0, policy_version 1155089 (0.0008) [2023-12-26 23:47:15,363][105692] Updated weights for policy 0, policy_version 1155099 (0.0009) [2023-12-26 23:47:15,426][105692] Updated weights for policy 0, policy_version 1155109 (0.0009) [2023-12-26 23:47:15,931][105620] Updated weights for policy 1, policy_version 1156497 (0.0006) [2023-12-26 23:47:15,991][105620] Updated weights for policy 1, policy_version 1156507 (0.0008) [2023-12-26 23:47:16,035][105620] Updated weights for policy 1, policy_version 1156517 (0.0008) [2023-12-26 23:47:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 591863808. Throughput: 0: 9900.6, 1: 9600.9. Samples: 591831220. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:47:16,063][104569] Avg episode reward: [(0, '8895.082'), (1, '9258.024')] [2023-12-26 23:47:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001155112_295755776.pth... [2023-12-26 23:47:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001156520_296108032.pth... [2023-12-26 23:47:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001153928_295452672.pth [2023-12-26 23:47:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001155400_295821312.pth [2023-12-26 23:47:16,227][105692] Updated weights for policy 0, policy_version 1155119 (0.0010) [2023-12-26 23:47:16,279][105692] Updated weights for policy 0, policy_version 1155129 (0.0010) [2023-12-26 23:47:16,326][105692] Updated weights for policy 0, policy_version 1155139 (0.0010) [2023-12-26 23:47:16,708][105620] Updated weights for policy 1, policy_version 1156527 (0.0008) [2023-12-26 23:47:16,756][105620] Updated weights for policy 1, policy_version 1156537 (0.0008) [2023-12-26 23:47:16,814][105620] Updated weights for policy 1, policy_version 1156547 (0.0010) [2023-12-26 23:47:16,994][105692] Updated weights for policy 0, policy_version 1155149 (0.0008) [2023-12-26 23:47:17,051][105692] Updated weights for policy 0, policy_version 1155159 (0.0005) [2023-12-26 23:47:17,096][105692] Updated weights for policy 0, policy_version 1155169 (0.0005) [2023-12-26 23:47:17,530][105620] Updated weights for policy 1, policy_version 1156557 (0.0010) [2023-12-26 23:47:17,585][105620] Updated weights for policy 1, policy_version 1156567 (0.0010) [2023-12-26 23:47:17,643][105620] Updated weights for policy 1, policy_version 1156577 (0.0010) [2023-12-26 23:47:17,694][105692] Updated weights for policy 0, policy_version 1155179 (0.0007) [2023-12-26 23:47:17,750][105692] Updated weights for policy 0, policy_version 1155189 (0.0011) [2023-12-26 23:47:17,813][105692] Updated weights for policy 0, policy_version 1155199 (0.0010) [2023-12-26 23:47:18,371][105620] Updated weights for policy 1, policy_version 1156587 (0.0010) [2023-12-26 23:47:18,434][105620] Updated weights for policy 1, policy_version 1156597 (0.0011) [2023-12-26 23:47:18,493][105620] Updated weights for policy 1, policy_version 1156607 (0.0008) [2023-12-26 23:47:18,571][105692] Updated weights for policy 0, policy_version 1155209 (0.0010) [2023-12-26 23:47:18,630][105692] Updated weights for policy 0, policy_version 1155219 (0.0011) [2023-12-26 23:47:18,683][105692] Updated weights for policy 0, policy_version 1155229 (0.0011) [2023-12-26 23:47:18,738][105692] Updated weights for policy 0, policy_version 1155239 (0.0011) [2023-12-26 23:47:19,210][105620] Updated weights for policy 1, policy_version 1156617 (0.0010) [2023-12-26 23:47:19,278][105620] Updated weights for policy 1, policy_version 1156627 (0.0010) [2023-12-26 23:47:19,345][105620] Updated weights for policy 1, policy_version 1156637 (0.0008) [2023-12-26 23:47:19,411][105620] Updated weights for policy 1, policy_version 1156647 (0.0009) [2023-12-26 23:47:19,514][105692] Updated weights for policy 0, policy_version 1155249 (0.0010) [2023-12-26 23:47:19,573][105692] Updated weights for policy 0, policy_version 1155259 (0.0008) [2023-12-26 23:47:19,635][105692] Updated weights for policy 0, policy_version 1155269 (0.0010) [2023-12-26 23:47:20,087][105620] Updated weights for policy 1, policy_version 1156657 (0.0006) [2023-12-26 23:47:20,143][105620] Updated weights for policy 1, policy_version 1156667 (0.0009) [2023-12-26 23:47:20,203][105620] Updated weights for policy 1, policy_version 1156677 (0.0009) [2023-12-26 23:47:20,370][105692] Updated weights for policy 0, policy_version 1155279 (0.0008) [2023-12-26 23:47:20,431][105692] Updated weights for policy 0, policy_version 1155289 (0.0009) [2023-12-26 23:47:20,493][105692] Updated weights for policy 0, policy_version 1155299 (0.0009) [2023-12-26 23:47:20,977][105620] Updated weights for policy 1, policy_version 1156687 (0.0009) [2023-12-26 23:47:21,046][105620] Updated weights for policy 1, policy_version 1156697 (0.0007) [2023-12-26 23:47:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 591953920. Throughput: 0: 9919.6, 1: 9490.3. Samples: 591947976. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:47:21,063][104569] Avg episode reward: [(0, '8991.481'), (1, '9165.921')] [2023-12-26 23:47:21,109][105620] Updated weights for policy 1, policy_version 1156707 (0.0007) [2023-12-26 23:47:21,243][105692] Updated weights for policy 0, policy_version 1155309 (0.0009) [2023-12-26 23:47:21,304][105692] Updated weights for policy 0, policy_version 1155319 (0.0009) [2023-12-26 23:47:21,374][105692] Updated weights for policy 0, policy_version 1155329 (0.0008) [2023-12-26 23:47:21,789][105620] Updated weights for policy 1, policy_version 1156717 (0.0009) [2023-12-26 23:47:21,849][105620] Updated weights for policy 1, policy_version 1156727 (0.0008) [2023-12-26 23:47:21,902][105620] Updated weights for policy 1, policy_version 1156737 (0.0008) [2023-12-26 23:47:22,053][105692] Updated weights for policy 0, policy_version 1155339 (0.0007) [2023-12-26 23:47:22,106][105692] Updated weights for policy 0, policy_version 1155349 (0.0010) [2023-12-26 23:47:22,159][105692] Updated weights for policy 0, policy_version 1155359 (0.0010) [2023-12-26 23:47:22,721][105620] Updated weights for policy 1, policy_version 1156747 (0.0009) [2023-12-26 23:47:22,780][105620] Updated weights for policy 1, policy_version 1156757 (0.0008) [2023-12-26 23:47:22,842][105620] Updated weights for policy 1, policy_version 1156767 (0.0009) [2023-12-26 23:47:22,878][105692] Updated weights for policy 0, policy_version 1155369 (0.0010) [2023-12-26 23:47:22,941][105692] Updated weights for policy 0, policy_version 1155379 (0.0011) [2023-12-26 23:47:23,000][105692] Updated weights for policy 0, policy_version 1155389 (0.0010) [2023-12-26 23:47:23,059][105692] Updated weights for policy 0, policy_version 1155399 (0.0010) [2023-12-26 23:47:23,641][105620] Updated weights for policy 1, policy_version 1156777 (0.0007) [2023-12-26 23:47:23,685][105620] Updated weights for policy 1, policy_version 1156787 (0.0005) [2023-12-26 23:47:23,731][105620] Updated weights for policy 1, policy_version 1156797 (0.0010) [2023-12-26 23:47:23,770][105692] Updated weights for policy 0, policy_version 1155409 (0.0006) [2023-12-26 23:47:23,780][105620] Updated weights for policy 1, policy_version 1156807 (0.0010) [2023-12-26 23:47:23,818][105692] Updated weights for policy 0, policy_version 1155419 (0.0007) [2023-12-26 23:47:23,871][105692] Updated weights for policy 0, policy_version 1155429 (0.0008) [2023-12-26 23:47:24,519][105620] Updated weights for policy 1, policy_version 1156817 (0.0010) [2023-12-26 23:47:24,546][105692] Updated weights for policy 0, policy_version 1155439 (0.0007) [2023-12-26 23:47:24,578][105620] Updated weights for policy 1, policy_version 1156827 (0.0010) [2023-12-26 23:47:24,604][105692] Updated weights for policy 0, policy_version 1155449 (0.0007) [2023-12-26 23:47:24,637][105620] Updated weights for policy 1, policy_version 1156837 (0.0009) [2023-12-26 23:47:24,660][105692] Updated weights for policy 0, policy_version 1155459 (0.0007) [2023-12-26 23:47:25,374][105620] Updated weights for policy 1, policy_version 1156847 (0.0010) [2023-12-26 23:47:25,431][105692] Updated weights for policy 0, policy_version 1155469 (0.0007) [2023-12-26 23:47:25,435][105620] Updated weights for policy 1, policy_version 1156857 (0.0010) [2023-12-26 23:47:25,486][105620] Updated weights for policy 1, policy_version 1156867 (0.0010) [2023-12-26 23:47:25,489][105692] Updated weights for policy 0, policy_version 1155479 (0.0006) [2023-12-26 23:47:25,551][105692] Updated weights for policy 0, policy_version 1155489 (0.0007) [2023-12-26 23:47:26,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 592052224. Throughput: 0: 9963.6, 1: 9494.6. Samples: 592062584. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:47:26,062][104569] Avg episode reward: [(0, '9082.773'), (1, '9164.878')] [2023-12-26 23:47:26,116][105692] Updated weights for policy 0, policy_version 1155499 (0.0008) [2023-12-26 23:47:26,161][105692] Updated weights for policy 0, policy_version 1155509 (0.0005) [2023-12-26 23:47:26,208][105692] Updated weights for policy 0, policy_version 1155519 (0.0005) [2023-12-26 23:47:26,224][105620] Updated weights for policy 1, policy_version 1156877 (0.0010) [2023-12-26 23:47:26,275][105620] Updated weights for policy 1, policy_version 1156887 (0.0010) [2023-12-26 23:47:26,334][105620] Updated weights for policy 1, policy_version 1156897 (0.0010) [2023-12-26 23:47:26,746][105692] Updated weights for policy 0, policy_version 1155529 (0.0005) [2023-12-26 23:47:26,806][105692] Updated weights for policy 0, policy_version 1155539 (0.0005) [2023-12-26 23:47:26,855][105692] Updated weights for policy 0, policy_version 1155549 (0.0010) [2023-12-26 23:47:26,902][105692] Updated weights for policy 0, policy_version 1155559 (0.0010) [2023-12-26 23:47:26,980][105620] Updated weights for policy 1, policy_version 1156907 (0.0009) [2023-12-26 23:47:27,028][105620] Updated weights for policy 1, policy_version 1156917 (0.0005) [2023-12-26 23:47:27,086][105620] Updated weights for policy 1, policy_version 1156927 (0.0005) [2023-12-26 23:47:27,536][105692] Updated weights for policy 0, policy_version 1155569 (0.0006) [2023-12-26 23:47:27,597][105692] Updated weights for policy 0, policy_version 1155579 (0.0005) [2023-12-26 23:47:27,656][105620] Updated weights for policy 1, policy_version 1156937 (0.0006) [2023-12-26 23:47:27,656][105692] Updated weights for policy 0, policy_version 1155589 (0.0005) [2023-12-26 23:47:27,705][105620] Updated weights for policy 1, policy_version 1156947 (0.0005) [2023-12-26 23:47:27,771][105620] Updated weights for policy 1, policy_version 1156957 (0.0006) [2023-12-26 23:47:27,836][105620] Updated weights for policy 1, policy_version 1156967 (0.0010) [2023-12-26 23:47:28,156][105692] Updated weights for policy 0, policy_version 1155599 (0.0005) [2023-12-26 23:47:28,206][105692] Updated weights for policy 0, policy_version 1155609 (0.0005) [2023-12-26 23:47:28,253][105692] Updated weights for policy 0, policy_version 1155619 (0.0006) [2023-12-26 23:47:28,506][105620] Updated weights for policy 1, policy_version 1156977 (0.0011) [2023-12-26 23:47:28,564][105620] Updated weights for policy 1, policy_version 1156987 (0.0010) [2023-12-26 23:47:28,622][105620] Updated weights for policy 1, policy_version 1156997 (0.0010) [2023-12-26 23:47:28,868][105692] Updated weights for policy 0, policy_version 1155629 (0.0006) [2023-12-26 23:47:28,932][105692] Updated weights for policy 0, policy_version 1155639 (0.0006) [2023-12-26 23:47:29,003][105692] Updated weights for policy 0, policy_version 1155649 (0.0006) [2023-12-26 23:47:29,352][105620] Updated weights for policy 1, policy_version 1157007 (0.0009) [2023-12-26 23:47:29,415][105620] Updated weights for policy 1, policy_version 1157017 (0.0008) [2023-12-26 23:47:29,473][105620] Updated weights for policy 1, policy_version 1157027 (0.0008) [2023-12-26 23:47:29,585][105692] Updated weights for policy 0, policy_version 1155659 (0.0006) [2023-12-26 23:47:29,644][105692] Updated weights for policy 0, policy_version 1155669 (0.0009) [2023-12-26 23:47:29,719][105692] Updated weights for policy 0, policy_version 1155679 (0.0009) [2023-12-26 23:47:30,143][105620] Updated weights for policy 1, policy_version 1157037 (0.0007) [2023-12-26 23:47:30,208][105620] Updated weights for policy 1, policy_version 1157047 (0.0008) [2023-12-26 23:47:30,270][105620] Updated weights for policy 1, policy_version 1157057 (0.0009) [2023-12-26 23:47:30,464][105692] Updated weights for policy 0, policy_version 1155689 (0.0009) [2023-12-26 23:47:30,515][105692] Updated weights for policy 0, policy_version 1155699 (0.0010) [2023-12-26 23:47:30,561][105692] Updated weights for policy 0, policy_version 1155709 (0.0010) [2023-12-26 23:47:30,608][105692] Updated weights for policy 0, policy_version 1155719 (0.0010) [2023-12-26 23:47:30,840][105620] Updated weights for policy 1, policy_version 1157067 (0.0008) [2023-12-26 23:47:30,888][105620] Updated weights for policy 1, policy_version 1157077 (0.0005) [2023-12-26 23:47:30,933][105620] Updated weights for policy 1, policy_version 1157087 (0.0006) [2023-12-26 23:47:31,062][104569] Fps is (10 sec: 21299.5, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 592166912. Throughput: 0: 10129.9, 1: 9574.9. Samples: 592130520. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:47:31,062][104569] Avg episode reward: [(0, '9266.219'), (1, '9256.343')] [2023-12-26 23:47:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001155720_295911424.pth... [2023-12-26 23:47:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001157096_296255488.pth... [2023-12-26 23:47:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001155944_295960576.pth [2023-12-26 23:47:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001154536_295608320.pth [2023-12-26 23:47:31,257][105692] Updated weights for policy 0, policy_version 1155729 (0.0006) [2023-12-26 23:47:31,325][105692] Updated weights for policy 0, policy_version 1155739 (0.0006) [2023-12-26 23:47:31,392][105692] Updated weights for policy 0, policy_version 1155749 (0.0009) [2023-12-26 23:47:31,618][105620] Updated weights for policy 1, policy_version 1157097 (0.0006) [2023-12-26 23:47:31,679][105620] Updated weights for policy 1, policy_version 1157107 (0.0007) [2023-12-26 23:47:31,748][105620] Updated weights for policy 1, policy_version 1157117 (0.0008) [2023-12-26 23:47:31,809][105620] Updated weights for policy 1, policy_version 1157127 (0.0009) [2023-12-26 23:47:32,056][105692] Updated weights for policy 0, policy_version 1155759 (0.0010) [2023-12-26 23:47:32,114][105692] Updated weights for policy 0, policy_version 1155769 (0.0010) [2023-12-26 23:47:32,180][105692] Updated weights for policy 0, policy_version 1155779 (0.0010) [2023-12-26 23:47:32,592][105620] Updated weights for policy 1, policy_version 1157137 (0.0009) [2023-12-26 23:47:32,639][105620] Updated weights for policy 1, policy_version 1157147 (0.0009) [2023-12-26 23:47:32,693][105620] Updated weights for policy 1, policy_version 1157157 (0.0009) [2023-12-26 23:47:32,882][105692] Updated weights for policy 0, policy_version 1155789 (0.0008) [2023-12-26 23:47:32,930][105692] Updated weights for policy 0, policy_version 1155799 (0.0009) [2023-12-26 23:47:32,976][105692] Updated weights for policy 0, policy_version 1155809 (0.0009) [2023-12-26 23:47:33,476][105620] Updated weights for policy 1, policy_version 1157167 (0.0008) [2023-12-26 23:47:33,532][105620] Updated weights for policy 1, policy_version 1157177 (0.0008) [2023-12-26 23:47:33,591][105620] Updated weights for policy 1, policy_version 1157187 (0.0008) [2023-12-26 23:47:33,734][105692] Updated weights for policy 0, policy_version 1155819 (0.0009) [2023-12-26 23:47:33,787][105692] Updated weights for policy 0, policy_version 1155829 (0.0006) [2023-12-26 23:47:33,847][105692] Updated weights for policy 0, policy_version 1155839 (0.0006) [2023-12-26 23:47:34,339][105620] Updated weights for policy 1, policy_version 1157197 (0.0008) [2023-12-26 23:47:34,386][105620] Updated weights for policy 1, policy_version 1157207 (0.0009) [2023-12-26 23:47:34,443][105620] Updated weights for policy 1, policy_version 1157217 (0.0008) [2023-12-26 23:47:34,564][105692] Updated weights for policy 0, policy_version 1155849 (0.0005) [2023-12-26 23:47:34,625][105692] Updated weights for policy 0, policy_version 1155859 (0.0006) [2023-12-26 23:47:34,692][105692] Updated weights for policy 0, policy_version 1155869 (0.0006) [2023-12-26 23:47:34,747][105692] Updated weights for policy 0, policy_version 1155879 (0.0006) [2023-12-26 23:47:35,075][105620] Updated weights for policy 1, policy_version 1157227 (0.0005) [2023-12-26 23:47:35,126][105620] Updated weights for policy 1, policy_version 1157237 (0.0006) [2023-12-26 23:47:35,176][105620] Updated weights for policy 1, policy_version 1157247 (0.0009) [2023-12-26 23:47:35,409][105692] Updated weights for policy 0, policy_version 1155889 (0.0010) [2023-12-26 23:47:35,454][105692] Updated weights for policy 0, policy_version 1155899 (0.0010) [2023-12-26 23:47:35,500][105692] Updated weights for policy 0, policy_version 1155909 (0.0006) [2023-12-26 23:47:35,938][105620] Updated weights for policy 1, policy_version 1157257 (0.0009) [2023-12-26 23:47:35,981][105620] Updated weights for policy 1, policy_version 1157267 (0.0006) [2023-12-26 23:47:36,027][105620] Updated weights for policy 1, policy_version 1157277 (0.0006) [2023-12-26 23:47:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 592257024. Throughput: 0: 10200.6, 1: 9587.8. Samples: 592250988. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:47:36,062][104569] Avg episode reward: [(0, '9358.150'), (1, '8900.849')] [2023-12-26 23:47:36,079][105620] Updated weights for policy 1, policy_version 1157287 (0.0007) [2023-12-26 23:47:36,110][105692] Updated weights for policy 0, policy_version 1155919 (0.0006) [2023-12-26 23:47:36,180][105692] Updated weights for policy 0, policy_version 1155929 (0.0008) [2023-12-26 23:47:36,249][105692] Updated weights for policy 0, policy_version 1155939 (0.0008) [2023-12-26 23:47:36,826][105692] Updated weights for policy 0, policy_version 1155949 (0.0007) [2023-12-26 23:47:36,895][105692] Updated weights for policy 0, policy_version 1155959 (0.0007) [2023-12-26 23:47:36,917][105620] Updated weights for policy 1, policy_version 1157297 (0.0008) [2023-12-26 23:47:36,956][105692] Updated weights for policy 0, policy_version 1155969 (0.0007) [2023-12-26 23:47:36,980][105620] Updated weights for policy 1, policy_version 1157307 (0.0008) [2023-12-26 23:47:37,043][105620] Updated weights for policy 1, policy_version 1157317 (0.0008) [2023-12-26 23:47:37,685][105620] Updated weights for policy 1, policy_version 1157327 (0.0009) [2023-12-26 23:47:37,738][105692] Updated weights for policy 0, policy_version 1155979 (0.0008) [2023-12-26 23:47:37,746][105620] Updated weights for policy 1, policy_version 1157337 (0.0008) [2023-12-26 23:47:37,790][105692] Updated weights for policy 0, policy_version 1155989 (0.0007) [2023-12-26 23:47:37,806][105620] Updated weights for policy 1, policy_version 1157347 (0.0008) [2023-12-26 23:47:37,837][105692] Updated weights for policy 0, policy_version 1155999 (0.0007) [2023-12-26 23:47:38,499][105620] Updated weights for policy 1, policy_version 1157357 (0.0009) [2023-12-26 23:47:38,547][105620] Updated weights for policy 1, policy_version 1157367 (0.0010) [2023-12-26 23:47:38,596][105620] Updated weights for policy 1, policy_version 1157377 (0.0010) [2023-12-26 23:47:38,663][105692] Updated weights for policy 0, policy_version 1156009 (0.0009) [2023-12-26 23:47:38,727][105692] Updated weights for policy 0, policy_version 1156019 (0.0008) [2023-12-26 23:47:38,779][105692] Updated weights for policy 0, policy_version 1156029 (0.0008) [2023-12-26 23:47:38,808][105585] KL-divergence is very high: 133.1606 [2023-12-26 23:47:38,835][105692] Updated weights for policy 0, policy_version 1156039 (0.0008) [2023-12-26 23:47:39,328][105620] Updated weights for policy 1, policy_version 1157387 (0.0010) [2023-12-26 23:47:39,402][105620] Updated weights for policy 1, policy_version 1157397 (0.0009) [2023-12-26 23:47:39,465][105620] Updated weights for policy 1, policy_version 1157407 (0.0010) [2023-12-26 23:47:39,621][105692] Updated weights for policy 0, policy_version 1156049 (0.0009) [2023-12-26 23:47:39,677][105692] Updated weights for policy 0, policy_version 1156059 (0.0007) [2023-12-26 23:47:39,732][105692] Updated weights for policy 0, policy_version 1156069 (0.0005) [2023-12-26 23:47:40,173][105620] Updated weights for policy 1, policy_version 1157417 (0.0010) [2023-12-26 23:47:40,232][105620] Updated weights for policy 1, policy_version 1157427 (0.0010) [2023-12-26 23:47:40,298][105620] Updated weights for policy 1, policy_version 1157437 (0.0009) [2023-12-26 23:47:40,357][105620] Updated weights for policy 1, policy_version 1157447 (0.0009) [2023-12-26 23:47:40,454][105692] Updated weights for policy 0, policy_version 1156079 (0.0008) [2023-12-26 23:47:40,512][105692] Updated weights for policy 0, policy_version 1156089 (0.0008) [2023-12-26 23:47:40,565][105692] Updated weights for policy 0, policy_version 1156099 (0.0009) [2023-12-26 23:47:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 592355328. Throughput: 0: 10066.2, 1: 9734.0. Samples: 592367120. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:47:41,062][104569] Avg episode reward: [(0, '9174.470'), (1, '8991.703')] [2023-12-26 23:47:41,101][105620] Updated weights for policy 1, policy_version 1157457 (0.0008) [2023-12-26 23:47:41,164][105620] Updated weights for policy 1, policy_version 1157467 (0.0009) [2023-12-26 23:47:41,222][105620] Updated weights for policy 1, policy_version 1157477 (0.0009) [2023-12-26 23:47:41,283][105692] Updated weights for policy 0, policy_version 1156109 (0.0007) [2023-12-26 23:47:41,339][105692] Updated weights for policy 0, policy_version 1156119 (0.0009) [2023-12-26 23:47:41,408][105692] Updated weights for policy 0, policy_version 1156129 (0.0009) [2023-12-26 23:47:41,964][105620] Updated weights for policy 1, policy_version 1157487 (0.0006) [2023-12-26 23:47:42,015][105620] Updated weights for policy 1, policy_version 1157497 (0.0007) [2023-12-26 23:47:42,070][105620] Updated weights for policy 1, policy_version 1157507 (0.0008) [2023-12-26 23:47:42,244][105692] Updated weights for policy 0, policy_version 1156139 (0.0010) [2023-12-26 23:47:42,305][105692] Updated weights for policy 0, policy_version 1156149 (0.0009) [2023-12-26 23:47:42,370][105692] Updated weights for policy 0, policy_version 1156159 (0.0009) [2023-12-26 23:47:42,808][105620] Updated weights for policy 1, policy_version 1157517 (0.0009) [2023-12-26 23:47:42,870][105620] Updated weights for policy 1, policy_version 1157527 (0.0009) [2023-12-26 23:47:42,935][105620] Updated weights for policy 1, policy_version 1157537 (0.0009) [2023-12-26 23:47:43,124][105692] Updated weights for policy 0, policy_version 1156169 (0.0009) [2023-12-26 23:47:43,194][105692] Updated weights for policy 0, policy_version 1156179 (0.0007) [2023-12-26 23:47:43,255][105692] Updated weights for policy 0, policy_version 1156189 (0.0005) [2023-12-26 23:47:43,307][105692] Updated weights for policy 0, policy_version 1156199 (0.0005) [2023-12-26 23:47:43,682][105620] Updated weights for policy 1, policy_version 1157547 (0.0008) [2023-12-26 23:47:43,743][105620] Updated weights for policy 1, policy_version 1157557 (0.0010) [2023-12-26 23:47:43,805][105620] Updated weights for policy 1, policy_version 1157567 (0.0010) [2023-12-26 23:47:43,879][105692] Updated weights for policy 0, policy_version 1156209 (0.0005) [2023-12-26 23:47:43,931][105692] Updated weights for policy 0, policy_version 1156219 (0.0005) [2023-12-26 23:47:43,982][105692] Updated weights for policy 0, policy_version 1156229 (0.0005) [2023-12-26 23:47:44,553][105692] Updated weights for policy 0, policy_version 1156239 (0.0006) [2023-12-26 23:47:44,618][105692] Updated weights for policy 0, policy_version 1156249 (0.0007) [2023-12-26 23:47:44,674][105692] Updated weights for policy 0, policy_version 1156259 (0.0010) [2023-12-26 23:47:44,679][105620] Updated weights for policy 1, policy_version 1157577 (0.0008) [2023-12-26 23:47:44,737][105620] Updated weights for policy 1, policy_version 1157587 (0.0007) [2023-12-26 23:47:44,805][105620] Updated weights for policy 1, policy_version 1157597 (0.0008) [2023-12-26 23:47:44,867][105620] Updated weights for policy 1, policy_version 1157607 (0.0006) [2023-12-26 23:47:45,326][105692] Updated weights for policy 0, policy_version 1156269 (0.0009) [2023-12-26 23:47:45,378][105692] Updated weights for policy 0, policy_version 1156279 (0.0008) [2023-12-26 23:47:45,438][105692] Updated weights for policy 0, policy_version 1156289 (0.0008) [2023-12-26 23:47:45,642][105620] Updated weights for policy 1, policy_version 1157617 (0.0005) [2023-12-26 23:47:45,705][105620] Updated weights for policy 1, policy_version 1157627 (0.0005) [2023-12-26 23:47:45,771][105620] Updated weights for policy 1, policy_version 1157637 (0.0006) [2023-12-26 23:47:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 592453632. Throughput: 0: 10039.3, 1: 9728.2. Samples: 592423036. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:47:46,062][104569] Avg episode reward: [(0, '8991.224'), (1, '9082.622')] [2023-12-26 23:47:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001157640_296394752.pth... [2023-12-26 23:47:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001156296_296058880.pth... [2023-12-26 23:47:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001155112_295755776.pth [2023-12-26 23:47:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001156520_296108032.pth [2023-12-26 23:47:46,219][105692] Updated weights for policy 0, policy_version 1156299 (0.0009) [2023-12-26 23:47:46,271][105692] Updated weights for policy 0, policy_version 1156309 (0.0010) [2023-12-26 23:47:46,310][105620] Updated weights for policy 1, policy_version 1157647 (0.0006) [2023-12-26 23:47:46,327][105692] Updated weights for policy 0, policy_version 1156319 (0.0010) [2023-12-26 23:47:46,373][105620] Updated weights for policy 1, policy_version 1157657 (0.0006) [2023-12-26 23:47:46,437][105620] Updated weights for policy 1, policy_version 1157667 (0.0008) [2023-12-26 23:47:47,052][105692] Updated weights for policy 0, policy_version 1156329 (0.0010) [2023-12-26 23:47:47,109][105692] Updated weights for policy 0, policy_version 1156339 (0.0010) [2023-12-26 23:47:47,168][105692] Updated weights for policy 0, policy_version 1156349 (0.0010) [2023-12-26 23:47:47,181][105620] Updated weights for policy 1, policy_version 1157677 (0.0007) [2023-12-26 23:47:47,216][105692] Updated weights for policy 0, policy_version 1156359 (0.0010) [2023-12-26 23:47:47,239][105620] Updated weights for policy 1, policy_version 1157687 (0.0006) [2023-12-26 23:47:47,300][105620] Updated weights for policy 1, policy_version 1157697 (0.0008) [2023-12-26 23:47:47,953][105692] Updated weights for policy 0, policy_version 1156369 (0.0010) [2023-12-26 23:47:47,999][105692] Updated weights for policy 0, policy_version 1156379 (0.0010) [2023-12-26 23:47:48,049][105620] Updated weights for policy 1, policy_version 1157707 (0.0008) [2023-12-26 23:47:48,057][105692] Updated weights for policy 0, policy_version 1156389 (0.0010) [2023-12-26 23:47:48,111][105620] Updated weights for policy 1, policy_version 1157717 (0.0009) [2023-12-26 23:47:48,177][105620] Updated weights for policy 1, policy_version 1157727 (0.0008) [2023-12-26 23:47:48,850][105692] Updated weights for policy 0, policy_version 1156399 (0.0007) [2023-12-26 23:47:48,907][105692] Updated weights for policy 0, policy_version 1156409 (0.0005) [2023-12-26 23:47:48,954][105620] Updated weights for policy 1, policy_version 1157737 (0.0008) [2023-12-26 23:47:48,959][105692] Updated weights for policy 0, policy_version 1156419 (0.0006) [2023-12-26 23:47:49,010][105620] Updated weights for policy 1, policy_version 1157747 (0.0007) [2023-12-26 23:47:49,067][105620] Updated weights for policy 1, policy_version 1157757 (0.0008) [2023-12-26 23:47:49,119][105620] Updated weights for policy 1, policy_version 1157767 (0.0008) [2023-12-26 23:47:49,648][105692] Updated weights for policy 0, policy_version 1156429 (0.0008) [2023-12-26 23:47:49,702][105692] Updated weights for policy 0, policy_version 1156439 (0.0005) [2023-12-26 23:47:49,760][105692] Updated weights for policy 0, policy_version 1156449 (0.0006) [2023-12-26 23:47:49,941][105620] Updated weights for policy 1, policy_version 1157777 (0.0009) [2023-12-26 23:47:50,006][105620] Updated weights for policy 1, policy_version 1157787 (0.0007) [2023-12-26 23:47:50,069][105620] Updated weights for policy 1, policy_version 1157797 (0.0007) [2023-12-26 23:47:50,401][105692] Updated weights for policy 0, policy_version 1156459 (0.0011) [2023-12-26 23:47:50,453][105692] Updated weights for policy 0, policy_version 1156469 (0.0009) [2023-12-26 23:47:50,511][105692] Updated weights for policy 0, policy_version 1156479 (0.0005) [2023-12-26 23:47:50,898][105620] Updated weights for policy 1, policy_version 1157807 (0.0010) [2023-12-26 23:47:50,952][105620] Updated weights for policy 1, policy_version 1157817 (0.0010) [2023-12-26 23:47:51,003][105620] Updated weights for policy 1, policy_version 1157827 (0.0009) [2023-12-26 23:47:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 592551936. Throughput: 0: 10013.9, 1: 9666.7. Samples: 592538988. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:47:51,062][104569] Avg episode reward: [(0, '8993.508'), (1, '8618.778')] [2023-12-26 23:47:51,100][105692] Updated weights for policy 0, policy_version 1156489 (0.0006) [2023-12-26 23:47:51,157][105692] Updated weights for policy 0, policy_version 1156499 (0.0007) [2023-12-26 23:47:51,215][105692] Updated weights for policy 0, policy_version 1156509 (0.0009) [2023-12-26 23:47:51,275][105692] Updated weights for policy 0, policy_version 1156519 (0.0011) [2023-12-26 23:47:51,767][105620] Updated weights for policy 1, policy_version 1157837 (0.0008) [2023-12-26 23:47:51,832][105620] Updated weights for policy 1, policy_version 1157847 (0.0009) [2023-12-26 23:47:51,886][105620] Updated weights for policy 1, policy_version 1157857 (0.0010) [2023-12-26 23:47:51,971][105692] Updated weights for policy 0, policy_version 1156529 (0.0006) [2023-12-26 23:47:52,030][105692] Updated weights for policy 0, policy_version 1156539 (0.0007) [2023-12-26 23:47:52,085][105692] Updated weights for policy 0, policy_version 1156549 (0.0011) [2023-12-26 23:47:52,584][105620] Updated weights for policy 1, policy_version 1157867 (0.0009) [2023-12-26 23:47:52,637][105620] Updated weights for policy 1, policy_version 1157877 (0.0006) [2023-12-26 23:47:52,701][105620] Updated weights for policy 1, policy_version 1157887 (0.0005) [2023-12-26 23:47:52,733][105692] Updated weights for policy 0, policy_version 1156559 (0.0007) [2023-12-26 23:47:52,790][105692] Updated weights for policy 0, policy_version 1156569 (0.0005) [2023-12-26 23:47:52,845][105692] Updated weights for policy 0, policy_version 1156579 (0.0005) [2023-12-26 23:47:53,387][105620] Updated weights for policy 1, policy_version 1157897 (0.0007) [2023-12-26 23:47:53,419][105692] Updated weights for policy 0, policy_version 1156589 (0.0006) [2023-12-26 23:47:53,438][105620] Updated weights for policy 1, policy_version 1157907 (0.0008) [2023-12-26 23:47:53,465][105692] Updated weights for policy 0, policy_version 1156599 (0.0006) [2023-12-26 23:47:53,483][105620] Updated weights for policy 1, policy_version 1157917 (0.0006) [2023-12-26 23:47:53,510][105692] Updated weights for policy 0, policy_version 1156609 (0.0006) [2023-12-26 23:47:53,532][105620] Updated weights for policy 1, policy_version 1157927 (0.0006) [2023-12-26 23:47:54,231][105620] Updated weights for policy 1, policy_version 1157937 (0.0005) [2023-12-26 23:47:54,291][105620] Updated weights for policy 1, policy_version 1157947 (0.0006) [2023-12-26 23:47:54,362][105620] Updated weights for policy 1, policy_version 1157957 (0.0006) [2023-12-26 23:47:54,363][105692] Updated weights for policy 0, policy_version 1156619 (0.0008) [2023-12-26 23:47:54,417][105692] Updated weights for policy 0, policy_version 1156629 (0.0009) [2023-12-26 23:47:54,470][105692] Updated weights for policy 0, policy_version 1156639 (0.0008) [2023-12-26 23:47:55,053][105692] Updated weights for policy 0, policy_version 1156649 (0.0008) [2023-12-26 23:47:55,105][105692] Updated weights for policy 0, policy_version 1156659 (0.0008) [2023-12-26 23:47:55,107][105620] Updated weights for policy 1, policy_version 1157967 (0.0007) [2023-12-26 23:47:55,154][105692] Updated weights for policy 0, policy_version 1156669 (0.0005) [2023-12-26 23:47:55,160][105620] Updated weights for policy 1, policy_version 1157977 (0.0007) [2023-12-26 23:47:55,204][105692] Updated weights for policy 0, policy_version 1156679 (0.0005) [2023-12-26 23:47:55,221][105620] Updated weights for policy 1, policy_version 1157987 (0.0009) [2023-12-26 23:47:55,828][105692] Updated weights for policy 0, policy_version 1156689 (0.0005) [2023-12-26 23:47:55,881][105692] Updated weights for policy 0, policy_version 1156699 (0.0007) [2023-12-26 23:47:55,931][105692] Updated weights for policy 0, policy_version 1156709 (0.0008) [2023-12-26 23:47:56,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 592650240. Throughput: 0: 10075.4, 1: 9584.2. Samples: 592658448. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:47:56,063][104569] Avg episode reward: [(0, '9069.615'), (1, '8438.304')] [2023-12-26 23:47:56,070][105620] Updated weights for policy 1, policy_version 1157997 (0.0009) [2023-12-26 23:47:56,128][105620] Updated weights for policy 1, policy_version 1158007 (0.0009) [2023-12-26 23:47:56,190][105620] Updated weights for policy 1, policy_version 1158017 (0.0010) [2023-12-26 23:47:56,542][105692] Updated weights for policy 0, policy_version 1156719 (0.0008) [2023-12-26 23:47:56,589][105692] Updated weights for policy 0, policy_version 1156729 (0.0009) [2023-12-26 23:47:56,639][105692] Updated weights for policy 0, policy_version 1156739 (0.0009) [2023-12-26 23:47:56,917][105620] Updated weights for policy 1, policy_version 1158027 (0.0009) [2023-12-26 23:47:56,979][105620] Updated weights for policy 1, policy_version 1158037 (0.0008) [2023-12-26 23:47:57,027][105620] Updated weights for policy 1, policy_version 1158047 (0.0008) [2023-12-26 23:47:57,424][105692] Updated weights for policy 0, policy_version 1156749 (0.0007) [2023-12-26 23:47:57,474][105692] Updated weights for policy 0, policy_version 1156759 (0.0007) [2023-12-26 23:47:57,484][105585] KL-divergence is very high: 141.5222 [2023-12-26 23:47:57,521][105585] KL-divergence is very high: 126.1080 [2023-12-26 23:47:57,521][105692] Updated weights for policy 0, policy_version 1156769 (0.0010) [2023-12-26 23:47:57,796][105620] Updated weights for policy 1, policy_version 1158057 (0.0007) [2023-12-26 23:47:57,843][105620] Updated weights for policy 1, policy_version 1158067 (0.0008) [2023-12-26 23:47:57,897][105620] Updated weights for policy 1, policy_version 1158079 (0.0010) [2023-12-26 23:47:58,148][105692] Updated weights for policy 0, policy_version 1156779 (0.0010) [2023-12-26 23:47:58,211][105692] Updated weights for policy 0, policy_version 1156789 (0.0010) [2023-12-26 23:47:58,272][105692] Updated weights for policy 0, policy_version 1156799 (0.0011) [2023-12-26 23:47:58,740][105620] Updated weights for policy 1, policy_version 1158089 (0.0009) [2023-12-26 23:47:58,804][105620] Updated weights for policy 1, policy_version 1158099 (0.0009) [2023-12-26 23:47:58,870][105620] Updated weights for policy 1, policy_version 1158109 (0.0009) [2023-12-26 23:47:58,938][105620] Updated weights for policy 1, policy_version 1158119 (0.0009) [2023-12-26 23:47:59,065][105692] Updated weights for policy 0, policy_version 1156809 (0.0010) [2023-12-26 23:47:59,123][105692] Updated weights for policy 0, policy_version 1156819 (0.0008) [2023-12-26 23:47:59,188][105692] Updated weights for policy 0, policy_version 1156829 (0.0007) [2023-12-26 23:47:59,255][105692] Updated weights for policy 0, policy_version 1156839 (0.0008) [2023-12-26 23:47:59,712][105620] Updated weights for policy 1, policy_version 1158129 (0.0008) [2023-12-26 23:47:59,778][105620] Updated weights for policy 1, policy_version 1158139 (0.0010) [2023-12-26 23:47:59,843][105620] Updated weights for policy 1, policy_version 1158149 (0.0011) [2023-12-26 23:47:59,976][105692] Updated weights for policy 0, policy_version 1156849 (0.0008) [2023-12-26 23:48:00,041][105692] Updated weights for policy 0, policy_version 1156859 (0.0008) [2023-12-26 23:48:00,098][105692] Updated weights for policy 0, policy_version 1156869 (0.0010) [2023-12-26 23:48:00,535][105620] Updated weights for policy 1, policy_version 1158159 (0.0009) [2023-12-26 23:48:00,584][105620] Updated weights for policy 1, policy_version 1158169 (0.0008) [2023-12-26 23:48:00,640][105620] Updated weights for policy 1, policy_version 1158179 (0.0006) [2023-12-26 23:48:00,838][105692] Updated weights for policy 0, policy_version 1156879 (0.0009) [2023-12-26 23:48:00,898][105692] Updated weights for policy 0, policy_version 1156889 (0.0010) [2023-12-26 23:48:00,953][105692] Updated weights for policy 0, policy_version 1156899 (0.0009) [2023-12-26 23:48:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 592748544. Throughput: 0: 10135.2, 1: 9525.7. Samples: 592715960. Policy #0 lag: (min: 26.0, avg: 46.7, max: 58.0) [2023-12-26 23:48:01,063][104569] Avg episode reward: [(0, '8926.449'), (1, '8928.218')] [2023-12-26 23:48:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001156904_296214528.pth... [2023-12-26 23:48:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001158184_296534016.pth... [2023-12-26 23:48:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001155720_295911424.pth [2023-12-26 23:48:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001157096_296255488.pth [2023-12-26 23:48:01,332][105620] Updated weights for policy 1, policy_version 1158189 (0.0006) [2023-12-26 23:48:01,397][105620] Updated weights for policy 1, policy_version 1158199 (0.0008) [2023-12-26 23:48:01,459][105620] Updated weights for policy 1, policy_version 1158209 (0.0010) [2023-12-26 23:48:01,613][105692] Updated weights for policy 0, policy_version 1156909 (0.0008) [2023-12-26 23:48:01,680][105692] Updated weights for policy 0, policy_version 1156919 (0.0011) [2023-12-26 23:48:01,746][105692] Updated weights for policy 0, policy_version 1156929 (0.0009) [2023-12-26 23:48:02,284][105620] Updated weights for policy 1, policy_version 1158220 (0.0010) [2023-12-26 23:48:02,325][105692] Updated weights for policy 0, policy_version 1156939 (0.0007) [2023-12-26 23:48:02,345][105620] Updated weights for policy 1, policy_version 1158230 (0.0007) [2023-12-26 23:48:02,388][105692] Updated weights for policy 0, policy_version 1156949 (0.0009) [2023-12-26 23:48:02,409][105620] Updated weights for policy 1, policy_version 1158240 (0.0006) [2023-12-26 23:48:02,446][105692] Updated weights for policy 0, policy_version 1156959 (0.0007) [2023-12-26 23:48:02,974][105620] Updated weights for policy 1, policy_version 1158250 (0.0005) [2023-12-26 23:48:03,047][105620] Updated weights for policy 1, policy_version 1158260 (0.0006) [2023-12-26 23:48:03,068][105692] Updated weights for policy 0, policy_version 1156969 (0.0010) [2023-12-26 23:48:03,107][105620] Updated weights for policy 1, policy_version 1158270 (0.0007) [2023-12-26 23:48:03,135][105692] Updated weights for policy 0, policy_version 1156979 (0.0005) [2023-12-26 23:48:03,181][105620] Updated weights for policy 1, policy_version 1158280 (0.0006) [2023-12-26 23:48:03,188][105692] Updated weights for policy 0, policy_version 1156989 (0.0008) [2023-12-26 23:48:03,240][105692] Updated weights for policy 0, policy_version 1156999 (0.0010) [2023-12-26 23:48:03,687][105620] Updated weights for policy 1, policy_version 1158290 (0.0005) [2023-12-26 23:48:03,743][105620] Updated weights for policy 1, policy_version 1158300 (0.0005) [2023-12-26 23:48:03,791][105620] Updated weights for policy 1, policy_version 1158310 (0.0005) [2023-12-26 23:48:03,863][105692] Updated weights for policy 0, policy_version 1157009 (0.0010) [2023-12-26 23:48:03,930][105692] Updated weights for policy 0, policy_version 1157019 (0.0007) [2023-12-26 23:48:03,987][105692] Updated weights for policy 0, policy_version 1157029 (0.0011) [2023-12-26 23:48:04,449][105620] Updated weights for policy 1, policy_version 1158320 (0.0010) [2023-12-26 23:48:04,513][105620] Updated weights for policy 1, policy_version 1158330 (0.0011) [2023-12-26 23:48:04,573][105620] Updated weights for policy 1, policy_version 1158340 (0.0011) [2023-12-26 23:48:04,731][105692] Updated weights for policy 0, policy_version 1157039 (0.0010) [2023-12-26 23:48:04,795][105692] Updated weights for policy 0, policy_version 1157049 (0.0011) [2023-12-26 23:48:04,854][105692] Updated weights for policy 0, policy_version 1157059 (0.0011) [2023-12-26 23:48:05,251][105620] Updated weights for policy 1, policy_version 1158350 (0.0010) [2023-12-26 23:48:05,304][105620] Updated weights for policy 1, policy_version 1158360 (0.0006) [2023-12-26 23:48:05,350][105620] Updated weights for policy 1, policy_version 1158370 (0.0008) [2023-12-26 23:48:05,528][105692] Updated weights for policy 0, policy_version 1157069 (0.0010) [2023-12-26 23:48:05,590][105692] Updated weights for policy 0, policy_version 1157079 (0.0011) [2023-12-26 23:48:05,648][105692] Updated weights for policy 0, policy_version 1157089 (0.0010) [2023-12-26 23:48:06,003][105620] Updated weights for policy 1, policy_version 1158380 (0.0008) [2023-12-26 23:48:06,047][105620] Updated weights for policy 1, policy_version 1158390 (0.0006) [2023-12-26 23:48:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 592846848. Throughput: 0: 10159.7, 1: 9602.6. Samples: 592837280. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:48:06,063][104569] Avg episode reward: [(0, '9086.291'), (1, '9259.765')] [2023-12-26 23:48:06,104][105620] Updated weights for policy 1, policy_version 1158400 (0.0006) [2023-12-26 23:48:06,383][105692] Updated weights for policy 0, policy_version 1157099 (0.0010) [2023-12-26 23:48:06,442][105692] Updated weights for policy 0, policy_version 1157109 (0.0011) [2023-12-26 23:48:06,504][105692] Updated weights for policy 0, policy_version 1157119 (0.0010) [2023-12-26 23:48:06,820][105620] Updated weights for policy 1, policy_version 1158410 (0.0008) [2023-12-26 23:48:06,872][105620] Updated weights for policy 1, policy_version 1158420 (0.0007) [2023-12-26 23:48:06,924][105620] Updated weights for policy 1, policy_version 1158430 (0.0008) [2023-12-26 23:48:06,972][105620] Updated weights for policy 1, policy_version 1158440 (0.0008) [2023-12-26 23:48:07,255][105692] Updated weights for policy 0, policy_version 1157129 (0.0010) [2023-12-26 23:48:07,311][105692] Updated weights for policy 0, policy_version 1157139 (0.0007) [2023-12-26 23:48:07,367][105692] Updated weights for policy 0, policy_version 1157149 (0.0005) [2023-12-26 23:48:07,423][105692] Updated weights for policy 0, policy_version 1157159 (0.0005) [2023-12-26 23:48:07,790][105620] Updated weights for policy 1, policy_version 1158450 (0.0009) [2023-12-26 23:48:07,837][105620] Updated weights for policy 1, policy_version 1158460 (0.0008) [2023-12-26 23:48:07,888][105620] Updated weights for policy 1, policy_version 1158470 (0.0009) [2023-12-26 23:48:08,053][105692] Updated weights for policy 0, policy_version 1157169 (0.0008) [2023-12-26 23:48:08,119][105692] Updated weights for policy 0, policy_version 1157179 (0.0006) [2023-12-26 23:48:08,168][105692] Updated weights for policy 0, policy_version 1157189 (0.0005) [2023-12-26 23:48:08,742][105620] Updated weights for policy 1, policy_version 1158480 (0.0009) [2023-12-26 23:48:08,792][105620] Updated weights for policy 1, policy_version 1158490 (0.0009) [2023-12-26 23:48:08,814][105692] Updated weights for policy 0, policy_version 1157199 (0.0005) [2023-12-26 23:48:08,840][105620] Updated weights for policy 1, policy_version 1158500 (0.0008) [2023-12-26 23:48:08,879][105692] Updated weights for policy 0, policy_version 1157209 (0.0005) [2023-12-26 23:48:08,935][105692] Updated weights for policy 0, policy_version 1157219 (0.0009) [2023-12-26 23:48:09,666][105620] Updated weights for policy 1, policy_version 1158510 (0.0007) [2023-12-26 23:48:09,667][105692] Updated weights for policy 0, policy_version 1157229 (0.0010) [2023-12-26 23:48:09,723][105620] Updated weights for policy 1, policy_version 1158520 (0.0006) [2023-12-26 23:48:09,725][105692] Updated weights for policy 0, policy_version 1157239 (0.0009) [2023-12-26 23:48:09,784][105692] Updated weights for policy 0, policy_version 1157249 (0.0009) [2023-12-26 23:48:09,787][105620] Updated weights for policy 1, policy_version 1158530 (0.0006) [2023-12-26 23:48:10,442][105620] Updated weights for policy 1, policy_version 1158540 (0.0008) [2023-12-26 23:48:10,504][105620] Updated weights for policy 1, policy_version 1158550 (0.0009) [2023-12-26 23:48:10,560][105620] Updated weights for policy 1, policy_version 1158560 (0.0008) [2023-12-26 23:48:10,599][105692] Updated weights for policy 0, policy_version 1157259 (0.0008) [2023-12-26 23:48:10,662][105692] Updated weights for policy 0, policy_version 1157269 (0.0010) [2023-12-26 23:48:10,723][105692] Updated weights for policy 0, policy_version 1157279 (0.0009) [2023-12-26 23:48:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 592945152. Throughput: 0: 10154.5, 1: 9635.3. Samples: 592953128. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:48:11,063][104569] Avg episode reward: [(0, '9081.473'), (1, '9260.394')] [2023-12-26 23:48:11,223][105620] Updated weights for policy 1, policy_version 1158570 (0.0005) [2023-12-26 23:48:11,290][105620] Updated weights for policy 1, policy_version 1158580 (0.0008) [2023-12-26 23:48:11,349][105620] Updated weights for policy 1, policy_version 1158590 (0.0008) [2023-12-26 23:48:11,414][105620] Updated weights for policy 1, policy_version 1158600 (0.0007) [2023-12-26 23:48:11,563][105692] Updated weights for policy 0, policy_version 1157289 (0.0008) [2023-12-26 23:48:11,616][105692] Updated weights for policy 0, policy_version 1157299 (0.0009) [2023-12-26 23:48:11,696][105692] Updated weights for policy 0, policy_version 1157311 (0.0009) [2023-12-26 23:48:12,055][105620] Updated weights for policy 1, policy_version 1158610 (0.0005) [2023-12-26 23:48:12,124][105620] Updated weights for policy 1, policy_version 1158620 (0.0007) [2023-12-26 23:48:12,182][105620] Updated weights for policy 1, policy_version 1158630 (0.0008) [2023-12-26 23:48:12,549][105692] Updated weights for policy 0, policy_version 1157321 (0.0009) [2023-12-26 23:48:12,608][105692] Updated weights for policy 0, policy_version 1157331 (0.0010) [2023-12-26 23:48:12,661][105692] Updated weights for policy 0, policy_version 1157341 (0.0007) [2023-12-26 23:48:12,716][105692] Updated weights for policy 0, policy_version 1157351 (0.0006) [2023-12-26 23:48:12,898][105620] Updated weights for policy 1, policy_version 1158640 (0.0009) [2023-12-26 23:48:12,952][105620] Updated weights for policy 1, policy_version 1158650 (0.0009) [2023-12-26 23:48:13,004][105620] Updated weights for policy 1, policy_version 1158660 (0.0010) [2023-12-26 23:48:13,278][105692] Updated weights for policy 0, policy_version 1157361 (0.0009) [2023-12-26 23:48:13,328][105692] Updated weights for policy 0, policy_version 1157371 (0.0009) [2023-12-26 23:48:13,376][105692] Updated weights for policy 0, policy_version 1157381 (0.0009) [2023-12-26 23:48:13,675][105620] Updated weights for policy 1, policy_version 1158670 (0.0007) [2023-12-26 23:48:13,727][105620] Updated weights for policy 1, policy_version 1158680 (0.0006) [2023-12-26 23:48:13,777][105620] Updated weights for policy 1, policy_version 1158690 (0.0008) [2023-12-26 23:48:14,048][105692] Updated weights for policy 0, policy_version 1157391 (0.0006) [2023-12-26 23:48:14,096][105692] Updated weights for policy 0, policy_version 1157401 (0.0005) [2023-12-26 23:48:14,143][105692] Updated weights for policy 0, policy_version 1157411 (0.0005) [2023-12-26 23:48:14,431][105620] Updated weights for policy 1, policy_version 1158700 (0.0009) [2023-12-26 23:48:14,487][105620] Updated weights for policy 1, policy_version 1158710 (0.0007) [2023-12-26 23:48:14,555][105620] Updated weights for policy 1, policy_version 1158720 (0.0006) [2023-12-26 23:48:14,806][105692] Updated weights for policy 0, policy_version 1157421 (0.0008) [2023-12-26 23:48:14,870][105692] Updated weights for policy 0, policy_version 1157431 (0.0011) [2023-12-26 23:48:14,936][105692] Updated weights for policy 0, policy_version 1157441 (0.0009) [2023-12-26 23:48:15,233][105620] Updated weights for policy 1, policy_version 1158730 (0.0006) [2023-12-26 23:48:15,299][105620] Updated weights for policy 1, policy_version 1158740 (0.0008) [2023-12-26 23:48:15,363][105620] Updated weights for policy 1, policy_version 1158750 (0.0006) [2023-12-26 23:48:15,430][105620] Updated weights for policy 1, policy_version 1158760 (0.0005) [2023-12-26 23:48:15,677][105692] Updated weights for policy 0, policy_version 1157451 (0.0010) [2023-12-26 23:48:15,736][105692] Updated weights for policy 0, policy_version 1157461 (0.0010) [2023-12-26 23:48:15,788][105692] Updated weights for policy 0, policy_version 1157471 (0.0010) [2023-12-26 23:48:16,026][105620] Updated weights for policy 1, policy_version 1158770 (0.0006) [2023-12-26 23:48:16,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.9, 300 sec: 19522.0). Total num frames: 593043456. Throughput: 0: 9951.9, 1: 9603.8. Samples: 593010528. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:48:16,062][104569] Avg episode reward: [(0, '8902.721'), (1, '9352.185')] [2023-12-26 23:48:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001157480_296361984.pth... [2023-12-26 23:48:16,075][105620] Updated weights for policy 1, policy_version 1158780 (0.0008) [2023-12-26 23:48:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001156296_296058880.pth [2023-12-26 23:48:16,136][105620] Updated weights for policy 1, policy_version 1158790 (0.0008) [2023-12-26 23:48:16,144][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001158792_296689664.pth... [2023-12-26 23:48:16,147][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001157640_296394752.pth [2023-12-26 23:48:16,481][105692] Updated weights for policy 0, policy_version 1157481 (0.0010) [2023-12-26 23:48:16,543][105692] Updated weights for policy 0, policy_version 1157491 (0.0010) [2023-12-26 23:48:16,605][105692] Updated weights for policy 0, policy_version 1157501 (0.0010) [2023-12-26 23:48:16,670][105692] Updated weights for policy 0, policy_version 1157511 (0.0010) [2023-12-26 23:48:16,888][105620] Updated weights for policy 1, policy_version 1158800 (0.0008) [2023-12-26 23:48:16,944][105620] Updated weights for policy 1, policy_version 1158810 (0.0010) [2023-12-26 23:48:17,006][105620] Updated weights for policy 1, policy_version 1158820 (0.0010) [2023-12-26 23:48:17,274][105692] Updated weights for policy 0, policy_version 1157521 (0.0006) [2023-12-26 23:48:17,322][105692] Updated weights for policy 0, policy_version 1157531 (0.0010) [2023-12-26 23:48:17,369][105692] Updated weights for policy 0, policy_version 1157541 (0.0010) [2023-12-26 23:48:17,686][105620] Updated weights for policy 1, policy_version 1158830 (0.0007) [2023-12-26 23:48:17,743][105620] Updated weights for policy 1, policy_version 1158840 (0.0005) [2023-12-26 23:48:17,794][105620] Updated weights for policy 1, policy_version 1158850 (0.0007) [2023-12-26 23:48:18,048][105692] Updated weights for policy 0, policy_version 1157551 (0.0007) [2023-12-26 23:48:18,104][105692] Updated weights for policy 0, policy_version 1157561 (0.0005) [2023-12-26 23:48:18,158][105692] Updated weights for policy 0, policy_version 1157571 (0.0006) [2023-12-26 23:48:18,374][105620] Updated weights for policy 1, policy_version 1158860 (0.0008) [2023-12-26 23:48:18,432][105620] Updated weights for policy 1, policy_version 1158870 (0.0010) [2023-12-26 23:48:18,485][105620] Updated weights for policy 1, policy_version 1158880 (0.0009) [2023-12-26 23:48:18,746][105692] Updated weights for policy 0, policy_version 1157581 (0.0007) [2023-12-26 23:48:18,805][105692] Updated weights for policy 0, policy_version 1157591 (0.0011) [2023-12-26 23:48:18,864][105692] Updated weights for policy 0, policy_version 1157601 (0.0010) [2023-12-26 23:48:19,319][105620] Updated weights for policy 1, policy_version 1158890 (0.0009) [2023-12-26 23:48:19,389][105620] Updated weights for policy 1, policy_version 1158900 (0.0010) [2023-12-26 23:48:19,454][105620] Updated weights for policy 1, policy_version 1158910 (0.0010) [2023-12-26 23:48:19,528][105620] Updated weights for policy 1, policy_version 1158920 (0.0011) [2023-12-26 23:48:19,570][105692] Updated weights for policy 0, policy_version 1157611 (0.0009) [2023-12-26 23:48:19,630][105692] Updated weights for policy 0, policy_version 1157621 (0.0005) [2023-12-26 23:48:19,689][105692] Updated weights for policy 0, policy_version 1157631 (0.0008) [2023-12-26 23:48:20,268][105620] Updated weights for policy 1, policy_version 1158930 (0.0011) [2023-12-26 23:48:20,320][105620] Updated weights for policy 1, policy_version 1158940 (0.0011) [2023-12-26 23:48:20,379][105620] Updated weights for policy 1, policy_version 1158950 (0.0011) [2023-12-26 23:48:20,390][105692] Updated weights for policy 0, policy_version 1157641 (0.0008) [2023-12-26 23:48:20,449][105692] Updated weights for policy 0, policy_version 1157651 (0.0005) [2023-12-26 23:48:20,509][105692] Updated weights for policy 0, policy_version 1157661 (0.0005) [2023-12-26 23:48:20,571][105692] Updated weights for policy 0, policy_version 1157671 (0.0005) [2023-12-26 23:48:21,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.4, 300 sec: 19522.0). Total num frames: 593141760. Throughput: 0: 10015.8, 1: 9617.9. Samples: 593134504. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:48:21,062][104569] Avg episode reward: [(0, '9085.197'), (1, '9262.659')] [2023-12-26 23:48:21,199][105620] Updated weights for policy 1, policy_version 1158960 (0.0011) [2023-12-26 23:48:21,246][105692] Updated weights for policy 0, policy_version 1157681 (0.0006) [2023-12-26 23:48:21,257][105620] Updated weights for policy 1, policy_version 1158970 (0.0011) [2023-12-26 23:48:21,308][105692] Updated weights for policy 0, policy_version 1157691 (0.0008) [2023-12-26 23:48:21,318][105620] Updated weights for policy 1, policy_version 1158980 (0.0011) [2023-12-26 23:48:21,374][105692] Updated weights for policy 0, policy_version 1157701 (0.0007) [2023-12-26 23:48:22,083][105620] Updated weights for policy 1, policy_version 1158990 (0.0009) [2023-12-26 23:48:22,142][105620] Updated weights for policy 1, policy_version 1159000 (0.0007) [2023-12-26 23:48:22,157][105692] Updated weights for policy 0, policy_version 1157711 (0.0007) [2023-12-26 23:48:22,192][105620] Updated weights for policy 1, policy_version 1159010 (0.0007) [2023-12-26 23:48:22,212][105692] Updated weights for policy 0, policy_version 1157721 (0.0008) [2023-12-26 23:48:22,271][105692] Updated weights for policy 0, policy_version 1157731 (0.0008) [2023-12-26 23:48:22,970][105620] Updated weights for policy 1, policy_version 1159020 (0.0007) [2023-12-26 23:48:23,016][105620] Updated weights for policy 1, policy_version 1159030 (0.0009) [2023-12-26 23:48:23,062][105692] Updated weights for policy 0, policy_version 1157741 (0.0009) [2023-12-26 23:48:23,064][105620] Updated weights for policy 1, policy_version 1159040 (0.0007) [2023-12-26 23:48:23,115][105692] Updated weights for policy 0, policy_version 1157751 (0.0008) [2023-12-26 23:48:23,176][105692] Updated weights for policy 0, policy_version 1157761 (0.0009) [2023-12-26 23:48:23,671][105620] Updated weights for policy 1, policy_version 1159050 (0.0006) [2023-12-26 23:48:23,719][105620] Updated weights for policy 1, policy_version 1159060 (0.0007) [2023-12-26 23:48:23,767][105620] Updated weights for policy 1, policy_version 1159070 (0.0009) [2023-12-26 23:48:23,816][105620] Updated weights for policy 1, policy_version 1159080 (0.0009) [2023-12-26 23:48:24,032][105692] Updated weights for policy 0, policy_version 1157771 (0.0009) [2023-12-26 23:48:24,078][105692] Updated weights for policy 0, policy_version 1157781 (0.0008) [2023-12-26 23:48:24,128][105692] Updated weights for policy 0, policy_version 1157791 (0.0009) [2023-12-26 23:48:24,470][105620] Updated weights for policy 1, policy_version 1159090 (0.0009) [2023-12-26 23:48:24,533][105620] Updated weights for policy 1, policy_version 1159100 (0.0007) [2023-12-26 23:48:24,591][105620] Updated weights for policy 1, policy_version 1159110 (0.0009) [2023-12-26 23:48:24,948][105692] Updated weights for policy 0, policy_version 1157801 (0.0009) [2023-12-26 23:48:25,007][105692] Updated weights for policy 0, policy_version 1157811 (0.0009) [2023-12-26 23:48:25,057][105692] Updated weights for policy 0, policy_version 1157821 (0.0008) [2023-12-26 23:48:25,108][105692] Updated weights for policy 0, policy_version 1157831 (0.0009) [2023-12-26 23:48:25,298][105620] Updated weights for policy 1, policy_version 1159120 (0.0006) [2023-12-26 23:48:25,348][105620] Updated weights for policy 1, policy_version 1159130 (0.0006) [2023-12-26 23:48:25,394][105620] Updated weights for policy 1, policy_version 1159140 (0.0005) [2023-12-26 23:48:25,898][105692] Updated weights for policy 0, policy_version 1157841 (0.0009) [2023-12-26 23:48:25,951][105692] Updated weights for policy 0, policy_version 1157851 (0.0007) [2023-12-26 23:48:26,004][105692] Updated weights for policy 0, policy_version 1157861 (0.0005) [2023-12-26 23:48:26,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19797.2, 300 sec: 19494.2). Total num frames: 593240064. Throughput: 0: 9922.8, 1: 9638.2. Samples: 593247368. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:48:26,063][104569] Avg episode reward: [(0, '9264.310'), (1, '9082.291')] [2023-12-26 23:48:26,113][105620] Updated weights for policy 1, policy_version 1159150 (0.0008) [2023-12-26 23:48:26,175][105620] Updated weights for policy 1, policy_version 1159160 (0.0009) [2023-12-26 23:48:26,226][105620] Updated weights for policy 1, policy_version 1159170 (0.0009) [2023-12-26 23:48:26,729][105692] Updated weights for policy 0, policy_version 1157871 (0.0008) [2023-12-26 23:48:26,797][105692] Updated weights for policy 0, policy_version 1157881 (0.0009) [2023-12-26 23:48:26,862][105692] Updated weights for policy 0, policy_version 1157891 (0.0009) [2023-12-26 23:48:26,970][105620] Updated weights for policy 1, policy_version 1159180 (0.0009) [2023-12-26 23:48:27,033][105620] Updated weights for policy 1, policy_version 1159190 (0.0009) [2023-12-26 23:48:27,080][105620] Updated weights for policy 1, policy_version 1159200 (0.0008) [2023-12-26 23:48:27,530][105692] Updated weights for policy 0, policy_version 1157901 (0.0007) [2023-12-26 23:48:27,587][105692] Updated weights for policy 0, policy_version 1157911 (0.0008) [2023-12-26 23:48:27,648][105692] Updated weights for policy 0, policy_version 1157921 (0.0009) [2023-12-26 23:48:27,902][105620] Updated weights for policy 1, policy_version 1159210 (0.0009) [2023-12-26 23:48:27,966][105620] Updated weights for policy 1, policy_version 1159220 (0.0009) [2023-12-26 23:48:28,024][105620] Updated weights for policy 1, policy_version 1159230 (0.0009) [2023-12-26 23:48:28,088][105620] Updated weights for policy 1, policy_version 1159240 (0.0009) [2023-12-26 23:48:28,302][105692] Updated weights for policy 0, policy_version 1157931 (0.0009) [2023-12-26 23:48:28,361][105692] Updated weights for policy 0, policy_version 1157941 (0.0009) [2023-12-26 23:48:28,422][105692] Updated weights for policy 0, policy_version 1157951 (0.0009) [2023-12-26 23:48:28,771][105620] Updated weights for policy 1, policy_version 1159250 (0.0008) [2023-12-26 23:48:28,824][105620] Updated weights for policy 1, policy_version 1159260 (0.0008) [2023-12-26 23:48:28,875][105620] Updated weights for policy 1, policy_version 1159270 (0.0009) [2023-12-26 23:48:29,167][105692] Updated weights for policy 0, policy_version 1157961 (0.0009) [2023-12-26 23:48:29,218][105692] Updated weights for policy 0, policy_version 1157971 (0.0008) [2023-12-26 23:48:29,284][105692] Updated weights for policy 0, policy_version 1157981 (0.0007) [2023-12-26 23:48:29,346][105692] Updated weights for policy 0, policy_version 1157991 (0.0006) [2023-12-26 23:48:29,599][105620] Updated weights for policy 1, policy_version 1159280 (0.0007) [2023-12-26 23:48:29,650][105620] Updated weights for policy 1, policy_version 1159290 (0.0009) [2023-12-26 23:48:29,701][105620] Updated weights for policy 1, policy_version 1159300 (0.0009) [2023-12-26 23:48:30,026][105692] Updated weights for policy 0, policy_version 1158001 (0.0007) [2023-12-26 23:48:30,091][105692] Updated weights for policy 0, policy_version 1158011 (0.0007) [2023-12-26 23:48:30,153][105692] Updated weights for policy 0, policy_version 1158021 (0.0009) [2023-12-26 23:48:30,354][105620] Updated weights for policy 1, policy_version 1159310 (0.0005) [2023-12-26 23:48:30,409][105620] Updated weights for policy 1, policy_version 1159320 (0.0006) [2023-12-26 23:48:30,456][105620] Updated weights for policy 1, policy_version 1159330 (0.0009) [2023-12-26 23:48:30,865][105692] Updated weights for policy 0, policy_version 1158031 (0.0007) [2023-12-26 23:48:30,920][105692] Updated weights for policy 0, policy_version 1158041 (0.0005) [2023-12-26 23:48:30,988][105692] Updated weights for policy 0, policy_version 1158051 (0.0006) [2023-12-26 23:48:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 593338368. Throughput: 0: 9952.7, 1: 9637.0. Samples: 593304572. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:48:31,062][104569] Avg episode reward: [(0, '9077.324'), (1, '9261.091')] [2023-12-26 23:48:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001158056_296509440.pth... [2023-12-26 23:48:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001156904_296214528.pth [2023-12-26 23:48:31,098][105620] Updated weights for policy 1, policy_version 1159340 (0.0010) [2023-12-26 23:48:31,160][105620] Updated weights for policy 1, policy_version 1159350 (0.0009) [2023-12-26 23:48:31,216][105620] Updated weights for policy 1, policy_version 1159360 (0.0010) [2023-12-26 23:48:31,265][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001159368_296837120.pth... [2023-12-26 23:48:31,269][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001158184_296534016.pth [2023-12-26 23:48:31,571][105692] Updated weights for policy 0, policy_version 1158061 (0.0009) [2023-12-26 23:48:31,626][105692] Updated weights for policy 0, policy_version 1158071 (0.0008) [2023-12-26 23:48:31,689][105692] Updated weights for policy 0, policy_version 1158081 (0.0008) [2023-12-26 23:48:31,944][105620] Updated weights for policy 1, policy_version 1159370 (0.0008) [2023-12-26 23:48:32,001][105620] Updated weights for policy 1, policy_version 1159380 (0.0007) [2023-12-26 23:48:32,055][105620] Updated weights for policy 1, policy_version 1159390 (0.0009) [2023-12-26 23:48:32,100][105620] Updated weights for policy 1, policy_version 1159400 (0.0008) [2023-12-26 23:48:32,489][105692] Updated weights for policy 0, policy_version 1158091 (0.0009) [2023-12-26 23:48:32,546][105692] Updated weights for policy 0, policy_version 1158101 (0.0005) [2023-12-26 23:48:32,608][105692] Updated weights for policy 0, policy_version 1158111 (0.0007) [2023-12-26 23:48:32,812][105620] Updated weights for policy 1, policy_version 1159410 (0.0005) [2023-12-26 23:48:32,877][105620] Updated weights for policy 1, policy_version 1159420 (0.0005) [2023-12-26 23:48:32,943][105620] Updated weights for policy 1, policy_version 1159430 (0.0005) [2023-12-26 23:48:33,264][105692] Updated weights for policy 0, policy_version 1158121 (0.0009) [2023-12-26 23:48:33,312][105692] Updated weights for policy 0, policy_version 1158131 (0.0009) [2023-12-26 23:48:33,363][105692] Updated weights for policy 0, policy_version 1158141 (0.0009) [2023-12-26 23:48:33,417][105692] Updated weights for policy 0, policy_version 1158151 (0.0009) [2023-12-26 23:48:33,599][105620] Updated weights for policy 1, policy_version 1159440 (0.0008) [2023-12-26 23:48:33,646][105620] Updated weights for policy 1, policy_version 1159450 (0.0008) [2023-12-26 23:48:33,711][105620] Updated weights for policy 1, policy_version 1159460 (0.0009) [2023-12-26 23:48:34,065][105692] Updated weights for policy 0, policy_version 1158161 (0.0010) [2023-12-26 23:48:34,124][105692] Updated weights for policy 0, policy_version 1158171 (0.0010) [2023-12-26 23:48:34,186][105692] Updated weights for policy 0, policy_version 1158181 (0.0010) [2023-12-26 23:48:34,398][105620] Updated weights for policy 1, policy_version 1159470 (0.0007) [2023-12-26 23:48:34,451][105620] Updated weights for policy 1, policy_version 1159480 (0.0005) [2023-12-26 23:48:34,521][105620] Updated weights for policy 1, policy_version 1159490 (0.0006) [2023-12-26 23:48:34,937][105692] Updated weights for policy 0, policy_version 1158191 (0.0011) [2023-12-26 23:48:34,992][105692] Updated weights for policy 0, policy_version 1158201 (0.0010) [2023-12-26 23:48:35,051][105692] Updated weights for policy 0, policy_version 1158211 (0.0011) [2023-12-26 23:48:35,090][105620] Updated weights for policy 1, policy_version 1159500 (0.0006) [2023-12-26 23:48:35,135][105620] Updated weights for policy 1, policy_version 1159510 (0.0008) [2023-12-26 23:48:35,191][105620] Updated weights for policy 1, policy_version 1159520 (0.0008) [2023-12-26 23:48:35,698][105692] Updated weights for policy 0, policy_version 1158221 (0.0008) [2023-12-26 23:48:35,763][105692] Updated weights for policy 0, policy_version 1158231 (0.0009) [2023-12-26 23:48:35,815][105692] Updated weights for policy 0, policy_version 1158241 (0.0010) [2023-12-26 23:48:35,994][105620] Updated weights for policy 1, policy_version 1159530 (0.0008) [2023-12-26 23:48:36,050][105620] Updated weights for policy 1, policy_version 1159540 (0.0009) [2023-12-26 23:48:36,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 593436672. Throughput: 0: 9942.0, 1: 9785.8. Samples: 593426740. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:48:36,062][104569] Avg episode reward: [(0, '9076.968'), (1, '9351.115')] [2023-12-26 23:48:36,102][105620] Updated weights for policy 1, policy_version 1159550 (0.0009) [2023-12-26 23:48:36,171][105620] Updated weights for policy 1, policy_version 1159560 (0.0008) [2023-12-26 23:48:36,532][105692] Updated weights for policy 0, policy_version 1158251 (0.0010) [2023-12-26 23:48:36,591][105692] Updated weights for policy 0, policy_version 1158261 (0.0011) [2023-12-26 23:48:36,650][105692] Updated weights for policy 0, policy_version 1158271 (0.0011) [2023-12-26 23:48:36,978][105620] Updated weights for policy 1, policy_version 1159570 (0.0010) [2023-12-26 23:48:37,048][105620] Updated weights for policy 1, policy_version 1159580 (0.0009) [2023-12-26 23:48:37,117][105620] Updated weights for policy 1, policy_version 1159590 (0.0009) [2023-12-26 23:48:37,331][105692] Updated weights for policy 0, policy_version 1158281 (0.0010) [2023-12-26 23:48:37,395][105692] Updated weights for policy 0, policy_version 1158291 (0.0005) [2023-12-26 23:48:37,452][105692] Updated weights for policy 0, policy_version 1158301 (0.0005) [2023-12-26 23:48:37,510][105692] Updated weights for policy 0, policy_version 1158311 (0.0005) [2023-12-26 23:48:37,994][105620] Updated weights for policy 1, policy_version 1159600 (0.0009) [2023-12-26 23:48:38,018][105692] Updated weights for policy 0, policy_version 1158321 (0.0006) [2023-12-26 23:48:38,048][105620] Updated weights for policy 1, policy_version 1159610 (0.0007) [2023-12-26 23:48:38,104][105692] Updated weights for policy 0, policy_version 1158331 (0.0007) [2023-12-26 23:48:38,106][105620] Updated weights for policy 1, policy_version 1159620 (0.0007) [2023-12-26 23:48:38,153][105692] Updated weights for policy 0, policy_version 1158341 (0.0007) [2023-12-26 23:48:38,833][105692] Updated weights for policy 0, policy_version 1158351 (0.0009) [2023-12-26 23:48:38,903][105692] Updated weights for policy 0, policy_version 1158361 (0.0009) [2023-12-26 23:48:38,940][105620] Updated weights for policy 1, policy_version 1159630 (0.0006) [2023-12-26 23:48:38,965][105692] Updated weights for policy 0, policy_version 1158371 (0.0008) [2023-12-26 23:48:39,008][105620] Updated weights for policy 1, policy_version 1159640 (0.0007) [2023-12-26 23:48:39,062][105620] Updated weights for policy 1, policy_version 1159650 (0.0009) [2023-12-26 23:48:39,721][105692] Updated weights for policy 0, policy_version 1158381 (0.0008) [2023-12-26 23:48:39,779][105692] Updated weights for policy 0, policy_version 1158391 (0.0007) [2023-12-26 23:48:39,811][105620] Updated weights for policy 1, policy_version 1159660 (0.0009) [2023-12-26 23:48:39,857][105692] Updated weights for policy 0, policy_version 1158402 (0.0007) [2023-12-26 23:48:39,895][105620] Updated weights for policy 1, policy_version 1159670 (0.0009) [2023-12-26 23:48:39,962][105620] Updated weights for policy 1, policy_version 1159680 (0.0009) [2023-12-26 23:48:40,523][105692] Updated weights for policy 0, policy_version 1158412 (0.0007) [2023-12-26 23:48:40,585][105692] Updated weights for policy 0, policy_version 1158422 (0.0007) [2023-12-26 23:48:40,644][105692] Updated weights for policy 0, policy_version 1158432 (0.0007) [2023-12-26 23:48:40,758][105620] Updated weights for policy 1, policy_version 1159690 (0.0008) [2023-12-26 23:48:40,814][105620] Updated weights for policy 1, policy_version 1159700 (0.0005) [2023-12-26 23:48:40,882][105620] Updated weights for policy 1, policy_version 1159710 (0.0010) [2023-12-26 23:48:40,946][105620] Updated weights for policy 1, policy_version 1159720 (0.0009) [2023-12-26 23:48:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 593534976. Throughput: 0: 9894.5, 1: 9701.8. Samples: 593540276. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:48:41,062][104569] Avg episode reward: [(0, '9229.527'), (1, '9260.482')] [2023-12-26 23:48:41,357][105692] Updated weights for policy 0, policy_version 1158442 (0.0009) [2023-12-26 23:48:41,433][105692] Updated weights for policy 0, policy_version 1158452 (0.0009) [2023-12-26 23:48:41,502][105692] Updated weights for policy 0, policy_version 1158462 (0.0009) [2023-12-26 23:48:41,558][105692] Updated weights for policy 0, policy_version 1158472 (0.0009) [2023-12-26 23:48:41,671][105620] Updated weights for policy 1, policy_version 1159730 (0.0007) [2023-12-26 23:48:41,742][105620] Updated weights for policy 1, policy_version 1159740 (0.0007) [2023-12-26 23:48:41,808][105620] Updated weights for policy 1, policy_version 1159750 (0.0009) [2023-12-26 23:48:42,318][105692] Updated weights for policy 0, policy_version 1158482 (0.0009) [2023-12-26 23:48:42,383][105692] Updated weights for policy 0, policy_version 1158492 (0.0009) [2023-12-26 23:48:42,442][105692] Updated weights for policy 0, policy_version 1158502 (0.0009) [2023-12-26 23:48:42,546][105620] Updated weights for policy 1, policy_version 1159760 (0.0009) [2023-12-26 23:48:42,601][105620] Updated weights for policy 1, policy_version 1159770 (0.0009) [2023-12-26 23:48:42,651][105620] Updated weights for policy 1, policy_version 1159780 (0.0009) [2023-12-26 23:48:43,180][105692] Updated weights for policy 0, policy_version 1158512 (0.0009) [2023-12-26 23:48:43,232][105692] Updated weights for policy 0, policy_version 1158522 (0.0009) [2023-12-26 23:48:43,287][105692] Updated weights for policy 0, policy_version 1158532 (0.0009) [2023-12-26 23:48:43,408][105620] Updated weights for policy 1, policy_version 1159790 (0.0008) [2023-12-26 23:48:43,462][105620] Updated weights for policy 1, policy_version 1159800 (0.0009) [2023-12-26 23:48:43,519][105620] Updated weights for policy 1, policy_version 1159810 (0.0009) [2023-12-26 23:48:44,014][105692] Updated weights for policy 0, policy_version 1158542 (0.0007) [2023-12-26 23:48:44,071][105692] Updated weights for policy 0, policy_version 1158552 (0.0006) [2023-12-26 23:48:44,115][105692] Updated weights for policy 0, policy_version 1158562 (0.0008) [2023-12-26 23:48:44,308][105620] Updated weights for policy 1, policy_version 1159820 (0.0010) [2023-12-26 23:48:44,362][105620] Updated weights for policy 1, policy_version 1159830 (0.0010) [2023-12-26 23:48:44,425][105620] Updated weights for policy 1, policy_version 1159840 (0.0008) [2023-12-26 23:48:44,811][105692] Updated weights for policy 0, policy_version 1158572 (0.0009) [2023-12-26 23:48:44,871][105692] Updated weights for policy 0, policy_version 1158582 (0.0011) [2023-12-26 23:48:44,930][105692] Updated weights for policy 0, policy_version 1158592 (0.0011) [2023-12-26 23:48:45,210][105620] Updated weights for policy 1, policy_version 1159850 (0.0008) [2023-12-26 23:48:45,262][105620] Updated weights for policy 1, policy_version 1159860 (0.0008) [2023-12-26 23:48:45,313][105620] Updated weights for policy 1, policy_version 1159870 (0.0008) [2023-12-26 23:48:45,368][105620] Updated weights for policy 1, policy_version 1159880 (0.0008) [2023-12-26 23:48:45,661][105692] Updated weights for policy 0, policy_version 1158602 (0.0010) [2023-12-26 23:48:45,724][105692] Updated weights for policy 0, policy_version 1158612 (0.0010) [2023-12-26 23:48:45,781][105692] Updated weights for policy 0, policy_version 1158622 (0.0010) [2023-12-26 23:48:45,845][105692] Updated weights for policy 0, policy_version 1158632 (0.0011) [2023-12-26 23:48:46,047][105620] Updated weights for policy 1, policy_version 1159890 (0.0008) [2023-12-26 23:48:46,062][104569] Fps is (10 sec: 18840.5, 60 sec: 19524.1, 300 sec: 19494.1). Total num frames: 593625088. Throughput: 0: 9823.4, 1: 9721.2. Samples: 593595476. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:48:46,064][104569] Avg episode reward: [(0, '9228.698'), (1, '8977.475')] [2023-12-26 23:48:46,073][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001158632_296656896.pth... [2023-12-26 23:48:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001157480_296361984.pth [2023-12-26 23:48:46,103][105620] Updated weights for policy 1, policy_version 1159900 (0.0008) [2023-12-26 23:48:46,156][105620] Updated weights for policy 1, policy_version 1159910 (0.0009) [2023-12-26 23:48:46,169][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001159912_296976384.pth... [2023-12-26 23:48:46,173][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001158792_296689664.pth [2023-12-26 23:48:46,545][105692] Updated weights for policy 0, policy_version 1158642 (0.0009) [2023-12-26 23:48:46,608][105692] Updated weights for policy 0, policy_version 1158652 (0.0009) [2023-12-26 23:48:46,656][105692] Updated weights for policy 0, policy_version 1158662 (0.0009) [2023-12-26 23:48:46,903][105620] Updated weights for policy 1, policy_version 1159920 (0.0006) [2023-12-26 23:48:46,951][105620] Updated weights for policy 1, policy_version 1159930 (0.0007) [2023-12-26 23:48:46,997][105620] Updated weights for policy 1, policy_version 1159940 (0.0007) [2023-12-26 23:48:47,399][105692] Updated weights for policy 0, policy_version 1158672 (0.0010) [2023-12-26 23:48:47,455][105692] Updated weights for policy 0, policy_version 1158682 (0.0007) [2023-12-26 23:48:47,515][105692] Updated weights for policy 0, policy_version 1158692 (0.0009) [2023-12-26 23:48:47,759][105620] Updated weights for policy 1, policy_version 1159950 (0.0008) [2023-12-26 23:48:47,810][105620] Updated weights for policy 1, policy_version 1159960 (0.0008) [2023-12-26 23:48:47,858][105620] Updated weights for policy 1, policy_version 1159970 (0.0008) [2023-12-26 23:48:48,211][105692] Updated weights for policy 0, policy_version 1158702 (0.0011) [2023-12-26 23:48:48,259][105692] Updated weights for policy 0, policy_version 1158712 (0.0010) [2023-12-26 23:48:48,307][105692] Updated weights for policy 0, policy_version 1158722 (0.0010) [2023-12-26 23:48:48,680][105620] Updated weights for policy 1, policy_version 1159980 (0.0008) [2023-12-26 23:48:48,740][105620] Updated weights for policy 1, policy_version 1159990 (0.0010) [2023-12-26 23:48:48,795][105620] Updated weights for policy 1, policy_version 1160000 (0.0009) [2023-12-26 23:48:48,923][105692] Updated weights for policy 0, policy_version 1158732 (0.0008) [2023-12-26 23:48:48,982][105692] Updated weights for policy 0, policy_version 1158742 (0.0006) [2023-12-26 23:48:49,044][105692] Updated weights for policy 0, policy_version 1158752 (0.0008) [2023-12-26 23:48:49,448][105620] Updated weights for policy 1, policy_version 1160010 (0.0008) [2023-12-26 23:48:49,516][105620] Updated weights for policy 1, policy_version 1160020 (0.0005) [2023-12-26 23:48:49,577][105620] Updated weights for policy 1, policy_version 1160030 (0.0005) [2023-12-26 23:48:49,640][105620] Updated weights for policy 1, policy_version 1160040 (0.0005) [2023-12-26 23:48:49,723][105692] Updated weights for policy 0, policy_version 1158762 (0.0011) [2023-12-26 23:48:49,792][105692] Updated weights for policy 0, policy_version 1158772 (0.0011) [2023-12-26 23:48:49,861][105692] Updated weights for policy 0, policy_version 1158782 (0.0010) [2023-12-26 23:48:49,926][105692] Updated weights for policy 0, policy_version 1158792 (0.0010) [2023-12-26 23:48:50,290][105620] Updated weights for policy 1, policy_version 1160050 (0.0008) [2023-12-26 23:48:50,350][105620] Updated weights for policy 1, policy_version 1160060 (0.0009) [2023-12-26 23:48:50,404][105620] Updated weights for policy 1, policy_version 1160070 (0.0010) [2023-12-26 23:48:50,538][105692] Updated weights for policy 0, policy_version 1158802 (0.0010) [2023-12-26 23:48:50,605][105692] Updated weights for policy 0, policy_version 1158812 (0.0009) [2023-12-26 23:48:50,668][105692] Updated weights for policy 0, policy_version 1158822 (0.0010) [2023-12-26 23:48:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 593723392. Throughput: 0: 9806.7, 1: 9665.1. Samples: 593713508. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:48:51,063][104569] Avg episode reward: [(0, '9079.825'), (1, '8974.549')] [2023-12-26 23:48:51,171][105620] Updated weights for policy 1, policy_version 1160080 (0.0007) [2023-12-26 23:48:51,237][105620] Updated weights for policy 1, policy_version 1160090 (0.0006) [2023-12-26 23:48:51,299][105620] Updated weights for policy 1, policy_version 1160100 (0.0009) [2023-12-26 23:48:51,446][105692] Updated weights for policy 0, policy_version 1158832 (0.0009) [2023-12-26 23:48:51,501][105692] Updated weights for policy 0, policy_version 1158842 (0.0009) [2023-12-26 23:48:51,566][105692] Updated weights for policy 0, policy_version 1158852 (0.0008) [2023-12-26 23:48:52,090][105620] Updated weights for policy 1, policy_version 1160110 (0.0009) [2023-12-26 23:48:52,142][105620] Updated weights for policy 1, policy_version 1160120 (0.0008) [2023-12-26 23:48:52,197][105620] Updated weights for policy 1, policy_version 1160130 (0.0009) [2023-12-26 23:48:52,252][105692] Updated weights for policy 0, policy_version 1158862 (0.0008) [2023-12-26 23:48:52,320][105692] Updated weights for policy 0, policy_version 1158872 (0.0009) [2023-12-26 23:48:52,384][105692] Updated weights for policy 0, policy_version 1158882 (0.0007) [2023-12-26 23:48:52,976][105620] Updated weights for policy 1, policy_version 1160140 (0.0009) [2023-12-26 23:48:53,025][105620] Updated weights for policy 1, policy_version 1160150 (0.0010) [2023-12-26 23:48:53,073][105620] Updated weights for policy 1, policy_version 1160160 (0.0010) [2023-12-26 23:48:53,097][105692] Updated weights for policy 0, policy_version 1158892 (0.0007) [2023-12-26 23:48:53,155][105692] Updated weights for policy 0, policy_version 1158902 (0.0007) [2023-12-26 23:48:53,208][105692] Updated weights for policy 0, policy_version 1158912 (0.0006) [2023-12-26 23:48:53,718][105620] Updated weights for policy 1, policy_version 1160170 (0.0009) [2023-12-26 23:48:53,779][105620] Updated weights for policy 1, policy_version 1160180 (0.0005) [2023-12-26 23:48:53,839][105620] Updated weights for policy 1, policy_version 1160190 (0.0005) [2023-12-26 23:48:53,902][105620] Updated weights for policy 1, policy_version 1160200 (0.0010) [2023-12-26 23:48:54,033][105692] Updated weights for policy 0, policy_version 1158922 (0.0005) [2023-12-26 23:48:54,085][105692] Updated weights for policy 0, policy_version 1158932 (0.0010) [2023-12-26 23:48:54,144][105692] Updated weights for policy 0, policy_version 1158942 (0.0009) [2023-12-26 23:48:54,197][105692] Updated weights for policy 0, policy_version 1158952 (0.0008) [2023-12-26 23:48:54,471][105620] Updated weights for policy 1, policy_version 1160210 (0.0006) [2023-12-26 23:48:54,524][105620] Updated weights for policy 1, policy_version 1160220 (0.0005) [2023-12-26 23:48:54,581][105620] Updated weights for policy 1, policy_version 1160230 (0.0005) [2023-12-26 23:48:55,106][105692] Updated weights for policy 0, policy_version 1158962 (0.0008) [2023-12-26 23:48:55,109][105620] Updated weights for policy 1, policy_version 1160240 (0.0009) [2023-12-26 23:48:55,162][105692] Updated weights for policy 0, policy_version 1158972 (0.0007) [2023-12-26 23:48:55,164][105620] Updated weights for policy 1, policy_version 1160250 (0.0010) [2023-12-26 23:48:55,216][105620] Updated weights for policy 1, policy_version 1160260 (0.0010) [2023-12-26 23:48:55,222][105692] Updated weights for policy 0, policy_version 1158982 (0.0006) [2023-12-26 23:48:55,934][105620] Updated weights for policy 1, policy_version 1160270 (0.0009) [2023-12-26 23:48:55,979][105620] Updated weights for policy 1, policy_version 1160280 (0.0008) [2023-12-26 23:48:55,999][105692] Updated weights for policy 0, policy_version 1158992 (0.0006) [2023-12-26 23:48:56,027][105620] Updated weights for policy 1, policy_version 1160290 (0.0008) [2023-12-26 23:48:56,059][105692] Updated weights for policy 0, policy_version 1159002 (0.0007) [2023-12-26 23:48:56,062][104569] Fps is (10 sec: 19661.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 593821696. Throughput: 0: 9758.9, 1: 9716.3. Samples: 593829512. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:48:56,062][104569] Avg episode reward: [(0, '9082.521'), (1, '9169.062')] [2023-12-26 23:48:56,122][105692] Updated weights for policy 0, policy_version 1159012 (0.0009) [2023-12-26 23:48:56,721][105620] Updated weights for policy 1, policy_version 1160300 (0.0008) [2023-12-26 23:48:56,772][105620] Updated weights for policy 1, policy_version 1160310 (0.0008) [2023-12-26 23:48:56,822][105620] Updated weights for policy 1, policy_version 1160320 (0.0009) [2023-12-26 23:48:56,893][105692] Updated weights for policy 0, policy_version 1159022 (0.0009) [2023-12-26 23:48:56,940][105692] Updated weights for policy 0, policy_version 1159032 (0.0009) [2023-12-26 23:48:57,001][105692] Updated weights for policy 0, policy_version 1159042 (0.0010) [2023-12-26 23:48:57,591][105620] Updated weights for policy 1, policy_version 1160330 (0.0009) [2023-12-26 23:48:57,647][105620] Updated weights for policy 1, policy_version 1160340 (0.0009) [2023-12-26 23:48:57,690][105692] Updated weights for policy 0, policy_version 1159052 (0.0008) [2023-12-26 23:48:57,700][105620] Updated weights for policy 1, policy_version 1160350 (0.0010) [2023-12-26 23:48:57,741][105692] Updated weights for policy 0, policy_version 1159062 (0.0005) [2023-12-26 23:48:57,762][105585] KL-divergence is very high: 385.2791 [2023-12-26 23:48:57,763][105620] Updated weights for policy 1, policy_version 1160360 (0.0008) [2023-12-26 23:48:57,791][105692] Updated weights for policy 0, policy_version 1159072 (0.0007) [2023-12-26 23:48:57,801][105585] KL-divergence is very high: 723.4335 [2023-12-26 23:48:58,403][105692] Updated weights for policy 0, policy_version 1159082 (0.0008) [2023-12-26 23:48:58,464][105692] Updated weights for policy 0, policy_version 1159092 (0.0008) [2023-12-26 23:48:58,523][105692] Updated weights for policy 0, policy_version 1159102 (0.0009) [2023-12-26 23:48:58,595][105692] Updated weights for policy 0, policy_version 1159112 (0.0008) [2023-12-26 23:48:58,635][105620] Updated weights for policy 1, policy_version 1160370 (0.0008) [2023-12-26 23:48:58,699][105620] Updated weights for policy 1, policy_version 1160380 (0.0008) [2023-12-26 23:48:58,766][105620] Updated weights for policy 1, policy_version 1160390 (0.0008) [2023-12-26 23:48:59,423][105692] Updated weights for policy 0, policy_version 1159122 (0.0009) [2023-12-26 23:48:59,475][105692] Updated weights for policy 0, policy_version 1159132 (0.0008) [2023-12-26 23:48:59,520][105692] Updated weights for policy 0, policy_version 1159142 (0.0006) [2023-12-26 23:48:59,578][105620] Updated weights for policy 1, policy_version 1160400 (0.0009) [2023-12-26 23:48:59,647][105620] Updated weights for policy 1, policy_version 1160410 (0.0005) [2023-12-26 23:48:59,714][105620] Updated weights for policy 1, policy_version 1160420 (0.0010) [2023-12-26 23:49:00,296][105692] Updated weights for policy 0, policy_version 1159152 (0.0007) [2023-12-26 23:49:00,341][105620] Updated weights for policy 1, policy_version 1160430 (0.0009) [2023-12-26 23:49:00,351][105692] Updated weights for policy 0, policy_version 1159162 (0.0005) [2023-12-26 23:49:00,398][105692] Updated weights for policy 0, policy_version 1159172 (0.0007) [2023-12-26 23:49:00,409][105620] Updated weights for policy 1, policy_version 1160440 (0.0007) [2023-12-26 23:49:00,467][105620] Updated weights for policy 1, policy_version 1160450 (0.0006) [2023-12-26 23:49:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 593911808. Throughput: 0: 9810.2, 1: 9647.7. Samples: 593886136. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:49:01,062][104569] Avg episode reward: [(0, '9083.289'), (1, '9078.367')] [2023-12-26 23:49:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001159176_296796160.pth... [2023-12-26 23:49:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001160456_297115648.pth... [2023-12-26 23:49:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001158056_296509440.pth [2023-12-26 23:49:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001159368_296837120.pth [2023-12-26 23:49:01,142][105620] Updated weights for policy 1, policy_version 1160460 (0.0009) [2023-12-26 23:49:01,163][105692] Updated weights for policy 0, policy_version 1159182 (0.0006) [2023-12-26 23:49:01,203][105620] Updated weights for policy 1, policy_version 1160470 (0.0008) [2023-12-26 23:49:01,219][105692] Updated weights for policy 0, policy_version 1159192 (0.0008) [2023-12-26 23:49:01,260][105620] Updated weights for policy 1, policy_version 1160480 (0.0006) [2023-12-26 23:49:01,280][105692] Updated weights for policy 0, policy_version 1159202 (0.0008) [2023-12-26 23:49:01,960][105620] Updated weights for policy 1, policy_version 1160490 (0.0007) [2023-12-26 23:49:02,024][105620] Updated weights for policy 1, policy_version 1160500 (0.0009) [2023-12-26 23:49:02,025][105692] Updated weights for policy 0, policy_version 1159212 (0.0006) [2023-12-26 23:49:02,071][105620] Updated weights for policy 1, policy_version 1160510 (0.0007) [2023-12-26 23:49:02,073][105692] Updated weights for policy 0, policy_version 1159222 (0.0008) [2023-12-26 23:49:02,121][105620] Updated weights for policy 1, policy_version 1160520 (0.0006) [2023-12-26 23:49:02,126][105692] Updated weights for policy 0, policy_version 1159232 (0.0008) [2023-12-26 23:49:02,765][105620] Updated weights for policy 1, policy_version 1160530 (0.0008) [2023-12-26 23:49:02,830][105620] Updated weights for policy 1, policy_version 1160540 (0.0010) [2023-12-26 23:49:02,880][105692] Updated weights for policy 0, policy_version 1159242 (0.0009) [2023-12-26 23:49:02,890][105620] Updated weights for policy 1, policy_version 1160550 (0.0010) [2023-12-26 23:49:02,927][105692] Updated weights for policy 0, policy_version 1159252 (0.0007) [2023-12-26 23:49:02,974][105692] Updated weights for policy 0, policy_version 1159262 (0.0007) [2023-12-26 23:49:03,018][105692] Updated weights for policy 0, policy_version 1159272 (0.0005) [2023-12-26 23:49:03,501][105620] Updated weights for policy 1, policy_version 1160560 (0.0007) [2023-12-26 23:49:03,550][105620] Updated weights for policy 1, policy_version 1160570 (0.0005) [2023-12-26 23:49:03,601][105620] Updated weights for policy 1, policy_version 1160580 (0.0005) [2023-12-26 23:49:03,775][105692] Updated weights for policy 0, policy_version 1159282 (0.0009) [2023-12-26 23:49:03,828][105692] Updated weights for policy 0, policy_version 1159292 (0.0010) [2023-12-26 23:49:03,891][105692] Updated weights for policy 0, policy_version 1159302 (0.0006) [2023-12-26 23:49:04,264][105620] Updated weights for policy 1, policy_version 1160590 (0.0007) [2023-12-26 23:49:04,331][105620] Updated weights for policy 1, policy_version 1160600 (0.0010) [2023-12-26 23:49:04,393][105620] Updated weights for policy 1, policy_version 1160610 (0.0006) [2023-12-26 23:49:04,625][105692] Updated weights for policy 0, policy_version 1159312 (0.0008) [2023-12-26 23:49:04,684][105692] Updated weights for policy 0, policy_version 1159322 (0.0009) [2023-12-26 23:49:04,744][105692] Updated weights for policy 0, policy_version 1159332 (0.0010) [2023-12-26 23:49:05,098][105620] Updated weights for policy 1, policy_version 1160620 (0.0007) [2023-12-26 23:49:05,158][105620] Updated weights for policy 1, policy_version 1160630 (0.0008) [2023-12-26 23:49:05,208][105620] Updated weights for policy 1, policy_version 1160640 (0.0008) [2023-12-26 23:49:05,497][105692] Updated weights for policy 0, policy_version 1159342 (0.0009) [2023-12-26 23:49:05,545][105692] Updated weights for policy 0, policy_version 1159352 (0.0009) [2023-12-26 23:49:05,593][105692] Updated weights for policy 0, policy_version 1159362 (0.0009) [2023-12-26 23:49:05,962][105620] Updated weights for policy 1, policy_version 1160650 (0.0009) [2023-12-26 23:49:06,027][105620] Updated weights for policy 1, policy_version 1160660 (0.0009) [2023-12-26 23:49:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 594010112. Throughput: 0: 9639.8, 1: 9664.7. Samples: 594003208. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:49:06,062][104569] Avg episode reward: [(0, '8899.343'), (1, '9080.549')] [2023-12-26 23:49:06,082][105620] Updated weights for policy 1, policy_version 1160670 (0.0009) [2023-12-26 23:49:06,141][105620] Updated weights for policy 1, policy_version 1160680 (0.0008) [2023-12-26 23:49:06,353][105692] Updated weights for policy 0, policy_version 1159372 (0.0009) [2023-12-26 23:49:06,420][105692] Updated weights for policy 0, policy_version 1159382 (0.0009) [2023-12-26 23:49:06,483][105692] Updated weights for policy 0, policy_version 1159392 (0.0009) [2023-12-26 23:49:06,926][105620] Updated weights for policy 1, policy_version 1160690 (0.0009) [2023-12-26 23:49:06,986][105620] Updated weights for policy 1, policy_version 1160700 (0.0009) [2023-12-26 23:49:07,035][105620] Updated weights for policy 1, policy_version 1160710 (0.0008) [2023-12-26 23:49:07,238][105692] Updated weights for policy 0, policy_version 1159402 (0.0010) [2023-12-26 23:49:07,301][105692] Updated weights for policy 0, policy_version 1159412 (0.0011) [2023-12-26 23:49:07,364][105692] Updated weights for policy 0, policy_version 1159422 (0.0011) [2023-12-26 23:49:07,427][105692] Updated weights for policy 0, policy_version 1159432 (0.0011) [2023-12-26 23:49:07,858][105620] Updated weights for policy 1, policy_version 1160720 (0.0009) [2023-12-26 23:49:07,916][105620] Updated weights for policy 1, policy_version 1160730 (0.0009) [2023-12-26 23:49:07,972][105620] Updated weights for policy 1, policy_version 1160740 (0.0008) [2023-12-26 23:49:08,021][105692] Updated weights for policy 0, policy_version 1159442 (0.0008) [2023-12-26 23:49:08,077][105692] Updated weights for policy 0, policy_version 1159452 (0.0007) [2023-12-26 23:49:08,130][105692] Updated weights for policy 0, policy_version 1159462 (0.0005) [2023-12-26 23:49:08,760][105620] Updated weights for policy 1, policy_version 1160750 (0.0009) [2023-12-26 23:49:08,815][105620] Updated weights for policy 1, policy_version 1160760 (0.0009) [2023-12-26 23:49:08,859][105692] Updated weights for policy 0, policy_version 1159472 (0.0007) [2023-12-26 23:49:08,874][105620] Updated weights for policy 1, policy_version 1160770 (0.0007) [2023-12-26 23:49:08,910][105692] Updated weights for policy 0, policy_version 1159482 (0.0007) [2023-12-26 23:49:08,961][105692] Updated weights for policy 0, policy_version 1159492 (0.0009) [2023-12-26 23:49:09,602][105620] Updated weights for policy 1, policy_version 1160780 (0.0006) [2023-12-26 23:49:09,662][105620] Updated weights for policy 1, policy_version 1160790 (0.0009) [2023-12-26 23:49:09,727][105620] Updated weights for policy 1, policy_version 1160800 (0.0008) [2023-12-26 23:49:09,773][105692] Updated weights for policy 0, policy_version 1159502 (0.0007) [2023-12-26 23:49:09,837][105692] Updated weights for policy 0, policy_version 1159512 (0.0009) [2023-12-26 23:49:09,912][105692] Updated weights for policy 0, policy_version 1159522 (0.0008) [2023-12-26 23:49:10,444][105620] Updated weights for policy 1, policy_version 1160810 (0.0009) [2023-12-26 23:49:10,511][105620] Updated weights for policy 1, policy_version 1160820 (0.0007) [2023-12-26 23:49:10,573][105620] Updated weights for policy 1, policy_version 1160830 (0.0009) [2023-12-26 23:49:10,642][105620] Updated weights for policy 1, policy_version 1160840 (0.0007) [2023-12-26 23:49:10,656][105692] Updated weights for policy 0, policy_version 1159532 (0.0009) [2023-12-26 23:49:10,704][105692] Updated weights for policy 0, policy_version 1159542 (0.0009) [2023-12-26 23:49:10,758][105692] Updated weights for policy 0, policy_version 1159552 (0.0009) [2023-12-26 23:49:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 594108416. Throughput: 0: 9687.8, 1: 9590.5. Samples: 594114884. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:49:11,063][104569] Avg episode reward: [(0, '8991.662'), (1, '9079.140')] [2023-12-26 23:49:11,285][105620] Updated weights for policy 1, policy_version 1160850 (0.0008) [2023-12-26 23:49:11,345][105620] Updated weights for policy 1, policy_version 1160860 (0.0009) [2023-12-26 23:49:11,413][105620] Updated weights for policy 1, policy_version 1160870 (0.0008) [2023-12-26 23:49:11,544][105692] Updated weights for policy 0, policy_version 1159562 (0.0009) [2023-12-26 23:49:11,613][105692] Updated weights for policy 0, policy_version 1159572 (0.0009) [2023-12-26 23:49:11,676][105692] Updated weights for policy 0, policy_version 1159582 (0.0008) [2023-12-26 23:49:11,739][105692] Updated weights for policy 0, policy_version 1159592 (0.0009) [2023-12-26 23:49:12,236][105620] Updated weights for policy 1, policy_version 1160880 (0.0009) [2023-12-26 23:49:12,308][105620] Updated weights for policy 1, policy_version 1160890 (0.0010) [2023-12-26 23:49:12,370][105620] Updated weights for policy 1, policy_version 1160900 (0.0008) [2023-12-26 23:49:12,438][105692] Updated weights for policy 0, policy_version 1159602 (0.0006) [2023-12-26 23:49:12,498][105692] Updated weights for policy 0, policy_version 1159612 (0.0006) [2023-12-26 23:49:12,554][105692] Updated weights for policy 0, policy_version 1159622 (0.0009) [2023-12-26 23:49:13,134][105620] Updated weights for policy 1, policy_version 1160911 (0.0010) [2023-12-26 23:49:13,187][105620] Updated weights for policy 1, policy_version 1160921 (0.0009) [2023-12-26 23:49:13,236][105620] Updated weights for policy 1, policy_version 1160931 (0.0007) [2023-12-26 23:49:13,253][105692] Updated weights for policy 0, policy_version 1159632 (0.0008) [2023-12-26 23:49:13,314][105692] Updated weights for policy 0, policy_version 1159642 (0.0008) [2023-12-26 23:49:13,380][105692] Updated weights for policy 0, policy_version 1159652 (0.0009) [2023-12-26 23:49:13,939][105692] Updated weights for policy 0, policy_version 1159662 (0.0007) [2023-12-26 23:49:13,994][105692] Updated weights for policy 0, policy_version 1159672 (0.0005) [2023-12-26 23:49:14,053][105620] Updated weights for policy 1, policy_version 1160941 (0.0007) [2023-12-26 23:49:14,053][105692] Updated weights for policy 0, policy_version 1159682 (0.0006) [2023-12-26 23:49:14,106][105620] Updated weights for policy 1, policy_version 1160951 (0.0009) [2023-12-26 23:49:14,163][105620] Updated weights for policy 1, policy_version 1160961 (0.0008) [2023-12-26 23:49:14,633][105692] Updated weights for policy 0, policy_version 1159692 (0.0007) [2023-12-26 23:49:14,681][105692] Updated weights for policy 0, policy_version 1159702 (0.0009) [2023-12-26 23:49:14,729][105692] Updated weights for policy 0, policy_version 1159712 (0.0009) [2023-12-26 23:49:14,945][105620] Updated weights for policy 1, policy_version 1160971 (0.0009) [2023-12-26 23:49:15,007][105620] Updated weights for policy 1, policy_version 1160981 (0.0009) [2023-12-26 23:49:15,075][105620] Updated weights for policy 1, policy_version 1160991 (0.0008) [2023-12-26 23:49:15,512][105692] Updated weights for policy 0, policy_version 1159722 (0.0009) [2023-12-26 23:49:15,575][105692] Updated weights for policy 0, policy_version 1159732 (0.0009) [2023-12-26 23:49:15,645][105692] Updated weights for policy 0, policy_version 1159742 (0.0009) [2023-12-26 23:49:15,702][105692] Updated weights for policy 0, policy_version 1159752 (0.0009) [2023-12-26 23:49:15,770][105620] Updated weights for policy 1, policy_version 1161001 (0.0010) [2023-12-26 23:49:15,822][105620] Updated weights for policy 1, policy_version 1161011 (0.0008) [2023-12-26 23:49:15,878][105620] Updated weights for policy 1, policy_version 1161021 (0.0005) [2023-12-26 23:49:15,926][105620] Updated weights for policy 1, policy_version 1161031 (0.0008) [2023-12-26 23:49:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 594206720. Throughput: 0: 9665.2, 1: 9583.8. Samples: 594170776. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:49:16,062][104569] Avg episode reward: [(0, '8992.577'), (1, '9168.780')] [2023-12-26 23:49:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001159752_296943616.pth... [2023-12-26 23:49:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001161032_297263104.pth... [2023-12-26 23:49:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001158632_296656896.pth [2023-12-26 23:49:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001159912_296976384.pth [2023-12-26 23:49:16,412][105692] Updated weights for policy 0, policy_version 1159762 (0.0005) [2023-12-26 23:49:16,475][105692] Updated weights for policy 0, policy_version 1159772 (0.0010) [2023-12-26 23:49:16,532][105692] Updated weights for policy 0, policy_version 1159782 (0.0009) [2023-12-26 23:49:16,740][105620] Updated weights for policy 1, policy_version 1161041 (0.0008) [2023-12-26 23:49:16,793][105620] Updated weights for policy 1, policy_version 1161051 (0.0009) [2023-12-26 23:49:16,845][105620] Updated weights for policy 1, policy_version 1161061 (0.0009) [2023-12-26 23:49:17,141][105692] Updated weights for policy 0, policy_version 1159792 (0.0008) [2023-12-26 23:49:17,201][105692] Updated weights for policy 0, policy_version 1159802 (0.0006) [2023-12-26 23:49:17,261][105692] Updated weights for policy 0, policy_version 1159812 (0.0007) [2023-12-26 23:49:17,697][105620] Updated weights for policy 1, policy_version 1161071 (0.0009) [2023-12-26 23:49:17,752][105620] Updated weights for policy 1, policy_version 1161081 (0.0006) [2023-12-26 23:49:17,806][105620] Updated weights for policy 1, policy_version 1161091 (0.0005) [2023-12-26 23:49:17,902][105692] Updated weights for policy 0, policy_version 1159822 (0.0009) [2023-12-26 23:49:17,968][105692] Updated weights for policy 0, policy_version 1159832 (0.0008) [2023-12-26 23:49:18,032][105692] Updated weights for policy 0, policy_version 1159842 (0.0007) [2023-12-26 23:49:18,562][105620] Updated weights for policy 1, policy_version 1161101 (0.0007) [2023-12-26 23:49:18,602][105692] Updated weights for policy 0, policy_version 1159852 (0.0006) [2023-12-26 23:49:18,623][105620] Updated weights for policy 1, policy_version 1161111 (0.0008) [2023-12-26 23:49:18,658][105692] Updated weights for policy 0, policy_version 1159862 (0.0007) [2023-12-26 23:49:18,678][105620] Updated weights for policy 1, policy_version 1161121 (0.0009) [2023-12-26 23:49:18,725][105692] Updated weights for policy 0, policy_version 1159872 (0.0006) [2023-12-26 23:49:19,421][105692] Updated weights for policy 0, policy_version 1159882 (0.0008) [2023-12-26 23:49:19,470][105620] Updated weights for policy 1, policy_version 1161131 (0.0008) [2023-12-26 23:49:19,477][105692] Updated weights for policy 0, policy_version 1159892 (0.0010) [2023-12-26 23:49:19,528][105620] Updated weights for policy 1, policy_version 1161141 (0.0008) [2023-12-26 23:49:19,538][105692] Updated weights for policy 0, policy_version 1159902 (0.0009) [2023-12-26 23:49:19,586][105620] Updated weights for policy 1, policy_version 1161151 (0.0008) [2023-12-26 23:49:19,593][105692] Updated weights for policy 0, policy_version 1159912 (0.0008) [2023-12-26 23:49:20,288][105620] Updated weights for policy 1, policy_version 1161161 (0.0009) [2023-12-26 23:49:20,344][105620] Updated weights for policy 1, policy_version 1161171 (0.0008) [2023-12-26 23:49:20,404][105692] Updated weights for policy 0, policy_version 1159922 (0.0008) [2023-12-26 23:49:20,413][105620] Updated weights for policy 1, policy_version 1161181 (0.0005) [2023-12-26 23:49:20,459][105692] Updated weights for policy 0, policy_version 1159932 (0.0009) [2023-12-26 23:49:20,479][105620] Updated weights for policy 1, policy_version 1161191 (0.0006) [2023-12-26 23:49:20,515][105692] Updated weights for policy 0, policy_version 1159942 (0.0010) [2023-12-26 23:49:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 594296832. Throughput: 0: 9735.7, 1: 9398.1. Samples: 594287764. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:49:21,062][104569] Avg episode reward: [(0, '8988.842'), (1, '9259.407')] [2023-12-26 23:49:21,194][105620] Updated weights for policy 1, policy_version 1161201 (0.0009) [2023-12-26 23:49:21,261][105620] Updated weights for policy 1, policy_version 1161211 (0.0009) [2023-12-26 23:49:21,282][105692] Updated weights for policy 0, policy_version 1159952 (0.0008) [2023-12-26 23:49:21,325][105620] Updated weights for policy 1, policy_version 1161221 (0.0008) [2023-12-26 23:49:21,345][105692] Updated weights for policy 0, policy_version 1159962 (0.0007) [2023-12-26 23:49:21,346][105585] KL-divergence is very high: 131.6805 [2023-12-26 23:49:21,404][105585] KL-divergence is very high: 190.2558 [2023-12-26 23:49:21,419][105692] Updated weights for policy 0, policy_version 1159972 (0.0008) [2023-12-26 23:49:22,115][105620] Updated weights for policy 1, policy_version 1161231 (0.0008) [2023-12-26 23:49:22,119][105692] Updated weights for policy 0, policy_version 1159982 (0.0007) [2023-12-26 23:49:22,177][105692] Updated weights for policy 0, policy_version 1159992 (0.0005) [2023-12-26 23:49:22,183][105620] Updated weights for policy 1, policy_version 1161241 (0.0009) [2023-12-26 23:49:22,241][105692] Updated weights for policy 0, policy_version 1160002 (0.0010) [2023-12-26 23:49:22,247][105620] Updated weights for policy 1, policy_version 1161251 (0.0006) [2023-12-26 23:49:22,871][105692] Updated weights for policy 0, policy_version 1160012 (0.0009) [2023-12-26 23:49:22,933][105692] Updated weights for policy 0, policy_version 1160022 (0.0005) [2023-12-26 23:49:22,936][105620] Updated weights for policy 1, policy_version 1161261 (0.0007) [2023-12-26 23:49:22,992][105620] Updated weights for policy 1, policy_version 1161271 (0.0006) [2023-12-26 23:49:23,000][105692] Updated weights for policy 0, policy_version 1160032 (0.0006) [2023-12-26 23:49:23,051][105620] Updated weights for policy 1, policy_version 1161281 (0.0006) [2023-12-26 23:49:23,596][105692] Updated weights for policy 0, policy_version 1160042 (0.0009) [2023-12-26 23:49:23,650][105692] Updated weights for policy 0, policy_version 1160052 (0.0006) [2023-12-26 23:49:23,683][105620] Updated weights for policy 1, policy_version 1161291 (0.0006) [2023-12-26 23:49:23,705][105692] Updated weights for policy 0, policy_version 1160062 (0.0010) [2023-12-26 23:49:23,742][105620] Updated weights for policy 1, policy_version 1161301 (0.0008) [2023-12-26 23:49:23,753][105692] Updated weights for policy 0, policy_version 1160072 (0.0006) [2023-12-26 23:49:23,793][105620] Updated weights for policy 1, policy_version 1161311 (0.0008) [2023-12-26 23:49:24,367][105692] Updated weights for policy 0, policy_version 1160082 (0.0005) [2023-12-26 23:49:24,416][105692] Updated weights for policy 0, policy_version 1160092 (0.0005) [2023-12-26 23:49:24,468][105620] Updated weights for policy 1, policy_version 1161321 (0.0009) [2023-12-26 23:49:24,468][105692] Updated weights for policy 0, policy_version 1160102 (0.0005) [2023-12-26 23:49:24,529][105620] Updated weights for policy 1, policy_version 1161331 (0.0006) [2023-12-26 23:49:24,587][105620] Updated weights for policy 1, policy_version 1161341 (0.0005) [2023-12-26 23:49:24,642][105620] Updated weights for policy 1, policy_version 1161351 (0.0005) [2023-12-26 23:49:25,205][105692] Updated weights for policy 0, policy_version 1160112 (0.0008) [2023-12-26 23:49:25,258][105620] Updated weights for policy 1, policy_version 1161361 (0.0010) [2023-12-26 23:49:25,259][105692] Updated weights for policy 0, policy_version 1160122 (0.0008) [2023-12-26 23:49:25,303][105620] Updated weights for policy 1, policy_version 1161371 (0.0010) [2023-12-26 23:49:25,313][105692] Updated weights for policy 0, policy_version 1160132 (0.0005) [2023-12-26 23:49:25,352][105620] Updated weights for policy 1, policy_version 1161381 (0.0010) [2023-12-26 23:49:26,024][105620] Updated weights for policy 1, policy_version 1161391 (0.0007) [2023-12-26 23:49:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.3, 300 sec: 19466.4). Total num frames: 594395136. Throughput: 0: 9694.7, 1: 9595.0. Samples: 594408316. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:49:26,062][104569] Avg episode reward: [(0, '9078.195'), (1, '9167.788')] [2023-12-26 23:49:26,072][105620] Updated weights for policy 1, policy_version 1161401 (0.0007) [2023-12-26 23:49:26,117][105620] Updated weights for policy 1, policy_version 1161411 (0.0010) [2023-12-26 23:49:26,122][105692] Updated weights for policy 0, policy_version 1160142 (0.0006) [2023-12-26 23:49:26,184][105692] Updated weights for policy 0, policy_version 1160152 (0.0007) [2023-12-26 23:49:26,245][105692] Updated weights for policy 0, policy_version 1160162 (0.0008) [2023-12-26 23:49:26,869][105620] Updated weights for policy 1, policy_version 1161421 (0.0010) [2023-12-26 23:49:26,925][105620] Updated weights for policy 1, policy_version 1161431 (0.0010) [2023-12-26 23:49:26,956][105692] Updated weights for policy 0, policy_version 1160172 (0.0008) [2023-12-26 23:49:26,973][105620] Updated weights for policy 1, policy_version 1161441 (0.0008) [2023-12-26 23:49:27,014][105692] Updated weights for policy 0, policy_version 1160182 (0.0009) [2023-12-26 23:49:27,076][105692] Updated weights for policy 0, policy_version 1160192 (0.0010) [2023-12-26 23:49:27,579][105620] Updated weights for policy 1, policy_version 1161451 (0.0007) [2023-12-26 23:49:27,643][105620] Updated weights for policy 1, policy_version 1161461 (0.0010) [2023-12-26 23:49:27,697][105620] Updated weights for policy 1, policy_version 1161471 (0.0010) [2023-12-26 23:49:27,886][105692] Updated weights for policy 0, policy_version 1160202 (0.0010) [2023-12-26 23:49:27,940][105692] Updated weights for policy 0, policy_version 1160212 (0.0008) [2023-12-26 23:49:27,987][105692] Updated weights for policy 0, policy_version 1160222 (0.0008) [2023-12-26 23:49:28,035][105692] Updated weights for policy 0, policy_version 1160232 (0.0008) [2023-12-26 23:49:28,391][105620] Updated weights for policy 1, policy_version 1161481 (0.0010) [2023-12-26 23:49:28,447][105620] Updated weights for policy 1, policy_version 1161491 (0.0009) [2023-12-26 23:49:28,498][105620] Updated weights for policy 1, policy_version 1161501 (0.0010) [2023-12-26 23:49:28,553][105620] Updated weights for policy 1, policy_version 1161511 (0.0010) [2023-12-26 23:49:28,773][105692] Updated weights for policy 0, policy_version 1160242 (0.0005) [2023-12-26 23:49:28,837][105692] Updated weights for policy 0, policy_version 1160252 (0.0005) [2023-12-26 23:49:28,898][105692] Updated weights for policy 0, policy_version 1160262 (0.0008) [2023-12-26 23:49:29,297][105620] Updated weights for policy 1, policy_version 1161521 (0.0010) [2023-12-26 23:49:29,360][105620] Updated weights for policy 1, policy_version 1161531 (0.0011) [2023-12-26 23:49:29,421][105620] Updated weights for policy 1, policy_version 1161541 (0.0010) [2023-12-26 23:49:29,602][105692] Updated weights for policy 0, policy_version 1160273 (0.0010) [2023-12-26 23:49:29,655][105692] Updated weights for policy 0, policy_version 1160284 (0.0009) [2023-12-26 23:49:29,708][105692] Updated weights for policy 0, policy_version 1160294 (0.0010) [2023-12-26 23:49:30,059][105620] Updated weights for policy 1, policy_version 1161551 (0.0007) [2023-12-26 23:49:30,111][105620] Updated weights for policy 1, policy_version 1161561 (0.0007) [2023-12-26 23:49:30,170][105620] Updated weights for policy 1, policy_version 1161571 (0.0006) [2023-12-26 23:49:30,590][105692] Updated weights for policy 0, policy_version 1160304 (0.0009) [2023-12-26 23:49:30,651][105692] Updated weights for policy 0, policy_version 1160314 (0.0009) [2023-12-26 23:49:30,703][105692] Updated weights for policy 0, policy_version 1160324 (0.0009) [2023-12-26 23:49:30,793][105620] Updated weights for policy 1, policy_version 1161581 (0.0007) [2023-12-26 23:49:30,843][105620] Updated weights for policy 1, policy_version 1161591 (0.0006) [2023-12-26 23:49:30,890][105620] Updated weights for policy 1, policy_version 1161601 (0.0005) [2023-12-26 23:49:31,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 594501632. Throughput: 0: 9685.2, 1: 9660.7. Samples: 594466036. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:49:31,063][104569] Avg episode reward: [(0, '9262.104'), (1, '9168.225')] [2023-12-26 23:49:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001160328_297091072.pth... [2023-12-26 23:49:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001161608_297410560.pth... [2023-12-26 23:49:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001159176_296796160.pth [2023-12-26 23:49:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001160456_297115648.pth [2023-12-26 23:49:31,512][105620] Updated weights for policy 1, policy_version 1161611 (0.0007) [2023-12-26 23:49:31,561][105692] Updated weights for policy 0, policy_version 1160334 (0.0007) [2023-12-26 23:49:31,570][105620] Updated weights for policy 1, policy_version 1161621 (0.0011) [2023-12-26 23:49:31,620][105692] Updated weights for policy 0, policy_version 1160344 (0.0006) [2023-12-26 23:49:31,626][105620] Updated weights for policy 1, policy_version 1161631 (0.0011) [2023-12-26 23:49:31,670][105692] Updated weights for policy 0, policy_version 1160354 (0.0007) [2023-12-26 23:49:32,246][105620] Updated weights for policy 1, policy_version 1161641 (0.0010) [2023-12-26 23:49:32,306][105620] Updated weights for policy 1, policy_version 1161651 (0.0008) [2023-12-26 23:49:32,320][105692] Updated weights for policy 0, policy_version 1160364 (0.0007) [2023-12-26 23:49:32,364][105620] Updated weights for policy 1, policy_version 1161661 (0.0008) [2023-12-26 23:49:32,366][105692] Updated weights for policy 0, policy_version 1160374 (0.0007) [2023-12-26 23:49:32,427][105692] Updated weights for policy 0, policy_version 1160384 (0.0006) [2023-12-26 23:49:32,429][105620] Updated weights for policy 1, policy_version 1161671 (0.0008) [2023-12-26 23:49:33,174][105620] Updated weights for policy 1, policy_version 1161681 (0.0009) [2023-12-26 23:49:33,190][105692] Updated weights for policy 0, policy_version 1160394 (0.0008) [2023-12-26 23:49:33,228][105620] Updated weights for policy 1, policy_version 1161691 (0.0007) [2023-12-26 23:49:33,239][105692] Updated weights for policy 0, policy_version 1160404 (0.0007) [2023-12-26 23:49:33,285][105620] Updated weights for policy 1, policy_version 1161701 (0.0008) [2023-12-26 23:49:33,291][105692] Updated weights for policy 0, policy_version 1160414 (0.0008) [2023-12-26 23:49:33,342][105692] Updated weights for policy 0, policy_version 1160424 (0.0010) [2023-12-26 23:49:33,934][105620] Updated weights for policy 1, policy_version 1161711 (0.0008) [2023-12-26 23:49:33,980][105620] Updated weights for policy 1, policy_version 1161721 (0.0008) [2023-12-26 23:49:34,026][105620] Updated weights for policy 1, policy_version 1161731 (0.0009) [2023-12-26 23:49:34,134][105692] Updated weights for policy 0, policy_version 1160434 (0.0005) [2023-12-26 23:49:34,196][105692] Updated weights for policy 0, policy_version 1160444 (0.0008) [2023-12-26 23:49:34,247][105692] Updated weights for policy 0, policy_version 1160454 (0.0009) [2023-12-26 23:49:34,826][105620] Updated weights for policy 1, policy_version 1161741 (0.0009) [2023-12-26 23:49:34,877][105620] Updated weights for policy 1, policy_version 1161751 (0.0009) [2023-12-26 23:49:34,938][105620] Updated weights for policy 1, policy_version 1161762 (0.0010) [2023-12-26 23:49:34,978][105692] Updated weights for policy 0, policy_version 1160464 (0.0006) [2023-12-26 23:49:35,043][105692] Updated weights for policy 0, policy_version 1160474 (0.0007) [2023-12-26 23:49:35,106][105692] Updated weights for policy 0, policy_version 1160484 (0.0008) [2023-12-26 23:49:35,609][105620] Updated weights for policy 1, policy_version 1161772 (0.0009) [2023-12-26 23:49:35,671][105620] Updated weights for policy 1, policy_version 1161782 (0.0007) [2023-12-26 23:49:35,725][105620] Updated weights for policy 1, policy_version 1161792 (0.0010) [2023-12-26 23:49:35,761][105692] Updated weights for policy 0, policy_version 1160494 (0.0009) [2023-12-26 23:49:35,824][105692] Updated weights for policy 0, policy_version 1160504 (0.0009) [2023-12-26 23:49:35,882][105692] Updated weights for policy 0, policy_version 1160514 (0.0008) [2023-12-26 23:49:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 594599936. Throughput: 0: 9586.5, 1: 9735.6. Samples: 594583004. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:49:36,063][104569] Avg episode reward: [(0, '9262.642'), (1, '9351.970')] [2023-12-26 23:49:36,459][105620] Updated weights for policy 1, policy_version 1161802 (0.0009) [2023-12-26 23:49:36,525][105620] Updated weights for policy 1, policy_version 1161812 (0.0006) [2023-12-26 23:49:36,549][105692] Updated weights for policy 0, policy_version 1160524 (0.0010) [2023-12-26 23:49:36,580][105620] Updated weights for policy 1, policy_version 1161822 (0.0007) [2023-12-26 23:49:36,600][105692] Updated weights for policy 0, policy_version 1160534 (0.0006) [2023-12-26 23:49:36,639][105620] Updated weights for policy 1, policy_version 1161832 (0.0008) [2023-12-26 23:49:36,658][105692] Updated weights for policy 0, policy_version 1160544 (0.0007) [2023-12-26 23:49:37,327][105620] Updated weights for policy 1, policy_version 1161842 (0.0005) [2023-12-26 23:49:37,389][105620] Updated weights for policy 1, policy_version 1161852 (0.0005) [2023-12-26 23:49:37,446][105620] Updated weights for policy 1, policy_version 1161862 (0.0005) [2023-12-26 23:49:37,462][105692] Updated weights for policy 0, policy_version 1160554 (0.0010) [2023-12-26 23:49:37,514][105692] Updated weights for policy 0, policy_version 1160564 (0.0009) [2023-12-26 23:49:37,564][105692] Updated weights for policy 0, policy_version 1160574 (0.0009) [2023-12-26 23:49:37,624][105692] Updated weights for policy 0, policy_version 1160584 (0.0009) [2023-12-26 23:49:38,059][105620] Updated weights for policy 1, policy_version 1161872 (0.0009) [2023-12-26 23:49:38,112][105620] Updated weights for policy 1, policy_version 1161882 (0.0010) [2023-12-26 23:49:38,163][105620] Updated weights for policy 1, policy_version 1161892 (0.0005) [2023-12-26 23:49:38,392][105692] Updated weights for policy 0, policy_version 1160594 (0.0008) [2023-12-26 23:49:38,459][105692] Updated weights for policy 0, policy_version 1160604 (0.0009) [2023-12-26 23:49:38,519][105692] Updated weights for policy 0, policy_version 1160614 (0.0009) [2023-12-26 23:49:38,832][105620] Updated weights for policy 1, policy_version 1161902 (0.0007) [2023-12-26 23:49:38,891][105620] Updated weights for policy 1, policy_version 1161912 (0.0010) [2023-12-26 23:49:38,947][105620] Updated weights for policy 1, policy_version 1161922 (0.0009) [2023-12-26 23:49:39,277][105692] Updated weights for policy 0, policy_version 1160624 (0.0009) [2023-12-26 23:49:39,339][105692] Updated weights for policy 0, policy_version 1160634 (0.0010) [2023-12-26 23:49:39,417][105692] Updated weights for policy 0, policy_version 1160644 (0.0010) [2023-12-26 23:49:39,768][105620] Updated weights for policy 1, policy_version 1161932 (0.0008) [2023-12-26 23:49:39,836][105620] Updated weights for policy 1, policy_version 1161942 (0.0007) [2023-12-26 23:49:39,898][105620] Updated weights for policy 1, policy_version 1161952 (0.0006) [2023-12-26 23:49:40,148][105692] Updated weights for policy 0, policy_version 1160654 (0.0008) [2023-12-26 23:49:40,213][105692] Updated weights for policy 0, policy_version 1160664 (0.0009) [2023-12-26 23:49:40,282][105692] Updated weights for policy 0, policy_version 1160674 (0.0005) [2023-12-26 23:49:40,622][105620] Updated weights for policy 1, policy_version 1161962 (0.0008) [2023-12-26 23:49:40,675][105620] Updated weights for policy 1, policy_version 1161972 (0.0009) [2023-12-26 23:49:40,726][105620] Updated weights for policy 1, policy_version 1161982 (0.0010) [2023-12-26 23:49:40,782][105620] Updated weights for policy 1, policy_version 1161992 (0.0005) [2023-12-26 23:49:40,877][105692] Updated weights for policy 0, policy_version 1160684 (0.0006) [2023-12-26 23:49:40,942][105692] Updated weights for policy 0, policy_version 1160694 (0.0006) [2023-12-26 23:49:41,006][105692] Updated weights for policy 0, policy_version 1160704 (0.0008) [2023-12-26 23:49:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 594690048. Throughput: 0: 9633.2, 1: 9733.4. Samples: 594701008. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:49:41,063][104569] Avg episode reward: [(0, '9173.845'), (1, '9260.636')] [2023-12-26 23:49:41,599][105620] Updated weights for policy 1, policy_version 1162002 (0.0009) [2023-12-26 23:49:41,662][105620] Updated weights for policy 1, policy_version 1162012 (0.0008) [2023-12-26 23:49:41,722][105692] Updated weights for policy 0, policy_version 1160714 (0.0008) [2023-12-26 23:49:41,727][105620] Updated weights for policy 1, policy_version 1162022 (0.0008) [2023-12-26 23:49:41,777][105692] Updated weights for policy 0, policy_version 1160724 (0.0008) [2023-12-26 23:49:41,830][105692] Updated weights for policy 0, policy_version 1160734 (0.0008) [2023-12-26 23:49:41,882][105692] Updated weights for policy 0, policy_version 1160744 (0.0008) [2023-12-26 23:49:42,558][105692] Updated weights for policy 0, policy_version 1160754 (0.0007) [2023-12-26 23:49:42,563][105620] Updated weights for policy 1, policy_version 1162032 (0.0010) [2023-12-26 23:49:42,619][105620] Updated weights for policy 1, policy_version 1162042 (0.0007) [2023-12-26 23:49:42,621][105692] Updated weights for policy 0, policy_version 1160764 (0.0006) [2023-12-26 23:49:42,672][105692] Updated weights for policy 0, policy_version 1160774 (0.0008) [2023-12-26 23:49:42,672][105620] Updated weights for policy 1, policy_version 1162052 (0.0007) [2023-12-26 23:49:43,352][105692] Updated weights for policy 0, policy_version 1160784 (0.0010) [2023-12-26 23:49:43,403][105692] Updated weights for policy 0, policy_version 1160794 (0.0010) [2023-12-26 23:49:43,456][105620] Updated weights for policy 1, policy_version 1162062 (0.0009) [2023-12-26 23:49:43,459][105692] Updated weights for policy 0, policy_version 1160804 (0.0010) [2023-12-26 23:49:43,512][105620] Updated weights for policy 1, policy_version 1162072 (0.0005) [2023-12-26 23:49:43,574][105620] Updated weights for policy 1, policy_version 1162082 (0.0006) [2023-12-26 23:49:44,066][105692] Updated weights for policy 0, policy_version 1160814 (0.0007) [2023-12-26 23:49:44,116][105692] Updated weights for policy 0, policy_version 1160824 (0.0006) [2023-12-26 23:49:44,175][105692] Updated weights for policy 0, policy_version 1160834 (0.0005) [2023-12-26 23:49:44,329][105620] Updated weights for policy 1, policy_version 1162092 (0.0007) [2023-12-26 23:49:44,390][105620] Updated weights for policy 1, policy_version 1162102 (0.0005) [2023-12-26 23:49:44,435][105620] Updated weights for policy 1, policy_version 1162112 (0.0005) [2023-12-26 23:49:44,790][105692] Updated weights for policy 0, policy_version 1160844 (0.0007) [2023-12-26 23:49:44,856][105692] Updated weights for policy 0, policy_version 1160854 (0.0006) [2023-12-26 23:49:44,922][105692] Updated weights for policy 0, policy_version 1160864 (0.0008) [2023-12-26 23:49:45,188][105620] Updated weights for policy 1, policy_version 1162122 (0.0007) [2023-12-26 23:49:45,256][105620] Updated weights for policy 1, policy_version 1162132 (0.0008) [2023-12-26 23:49:45,311][105620] Updated weights for policy 1, policy_version 1162142 (0.0009) [2023-12-26 23:49:45,363][105620] Updated weights for policy 1, policy_version 1162152 (0.0009) [2023-12-26 23:49:45,620][105692] Updated weights for policy 0, policy_version 1160874 (0.0009) [2023-12-26 23:49:45,674][105692] Updated weights for policy 0, policy_version 1160884 (0.0009) [2023-12-26 23:49:45,736][105692] Updated weights for policy 0, policy_version 1160894 (0.0008) [2023-12-26 23:49:45,797][105692] Updated weights for policy 0, policy_version 1160904 (0.0008) [2023-12-26 23:49:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.9, 300 sec: 19466.4). Total num frames: 594788352. Throughput: 0: 9651.0, 1: 9721.9. Samples: 594757920. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:49:46,062][104569] Avg episode reward: [(0, '9266.806'), (1, '9260.355')] [2023-12-26 23:49:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001160904_297238528.pth... [2023-12-26 23:49:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001159752_296943616.pth [2023-12-26 23:49:46,079][105620] Updated weights for policy 1, policy_version 1162162 (0.0005) [2023-12-26 23:49:46,137][105620] Updated weights for policy 1, policy_version 1162172 (0.0007) [2023-12-26 23:49:46,203][105620] Updated weights for policy 1, policy_version 1162182 (0.0008) [2023-12-26 23:49:46,217][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001162184_297558016.pth... [2023-12-26 23:49:46,222][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001161032_297263104.pth [2023-12-26 23:49:46,587][105692] Updated weights for policy 0, policy_version 1160914 (0.0008) [2023-12-26 23:49:46,644][105692] Updated weights for policy 0, policy_version 1160924 (0.0006) [2023-12-26 23:49:46,704][105692] Updated weights for policy 0, policy_version 1160934 (0.0005) [2023-12-26 23:49:46,973][105620] Updated weights for policy 1, policy_version 1162192 (0.0010) [2023-12-26 23:49:47,025][105620] Updated weights for policy 1, policy_version 1162202 (0.0010) [2023-12-26 23:49:47,073][105620] Updated weights for policy 1, policy_version 1162212 (0.0010) [2023-12-26 23:49:47,298][105692] Updated weights for policy 0, policy_version 1160944 (0.0006) [2023-12-26 23:49:47,361][105692] Updated weights for policy 0, policy_version 1160954 (0.0006) [2023-12-26 23:49:47,429][105692] Updated weights for policy 0, policy_version 1160964 (0.0005) [2023-12-26 23:49:47,773][105620] Updated weights for policy 1, policy_version 1162222 (0.0008) [2023-12-26 23:49:47,822][105620] Updated weights for policy 1, policy_version 1162232 (0.0005) [2023-12-26 23:49:47,871][105620] Updated weights for policy 1, policy_version 1162242 (0.0005) [2023-12-26 23:49:48,132][105692] Updated weights for policy 0, policy_version 1160974 (0.0005) [2023-12-26 23:49:48,201][105692] Updated weights for policy 0, policy_version 1160984 (0.0005) [2023-12-26 23:49:48,247][105692] Updated weights for policy 0, policy_version 1160994 (0.0005) [2023-12-26 23:49:48,481][105620] Updated weights for policy 1, policy_version 1162252 (0.0007) [2023-12-26 23:49:48,538][105620] Updated weights for policy 1, policy_version 1162262 (0.0006) [2023-12-26 23:49:48,591][105620] Updated weights for policy 1, policy_version 1162272 (0.0010) [2023-12-26 23:49:48,786][105692] Updated weights for policy 0, policy_version 1161004 (0.0005) [2023-12-26 23:49:48,855][105692] Updated weights for policy 0, policy_version 1161014 (0.0005) [2023-12-26 23:49:48,916][105692] Updated weights for policy 0, policy_version 1161024 (0.0005) [2023-12-26 23:49:49,330][105620] Updated weights for policy 1, policy_version 1162282 (0.0010) [2023-12-26 23:49:49,395][105620] Updated weights for policy 1, policy_version 1162292 (0.0010) [2023-12-26 23:49:49,457][105620] Updated weights for policy 1, policy_version 1162302 (0.0008) [2023-12-26 23:49:49,515][105620] Updated weights for policy 1, policy_version 1162312 (0.0009) [2023-12-26 23:49:49,587][105692] Updated weights for policy 0, policy_version 1161034 (0.0008) [2023-12-26 23:49:49,649][105692] Updated weights for policy 0, policy_version 1161044 (0.0006) [2023-12-26 23:49:49,706][105692] Updated weights for policy 0, policy_version 1161054 (0.0009) [2023-12-26 23:49:49,756][105692] Updated weights for policy 0, policy_version 1161064 (0.0009) [2023-12-26 23:49:50,227][105620] Updated weights for policy 1, policy_version 1162322 (0.0006) [2023-12-26 23:49:50,283][105620] Updated weights for policy 1, policy_version 1162332 (0.0005) [2023-12-26 23:49:50,343][105620] Updated weights for policy 1, policy_version 1162342 (0.0008) [2023-12-26 23:49:50,560][105692] Updated weights for policy 0, policy_version 1161074 (0.0009) [2023-12-26 23:49:50,622][105692] Updated weights for policy 0, policy_version 1161084 (0.0009) [2023-12-26 23:49:50,689][105692] Updated weights for policy 0, policy_version 1161094 (0.0009) [2023-12-26 23:49:51,051][105620] Updated weights for policy 1, policy_version 1162352 (0.0009) [2023-12-26 23:49:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 594886656. Throughput: 0: 9815.0, 1: 9639.7. Samples: 594878668. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:49:51,062][104569] Avg episode reward: [(0, '9175.583'), (1, '9075.361')] [2023-12-26 23:49:51,113][105620] Updated weights for policy 1, policy_version 1162362 (0.0008) [2023-12-26 23:49:51,177][105620] Updated weights for policy 1, policy_version 1162372 (0.0008) [2023-12-26 23:49:51,418][105692] Updated weights for policy 0, policy_version 1161104 (0.0010) [2023-12-26 23:49:51,481][105692] Updated weights for policy 0, policy_version 1161114 (0.0009) [2023-12-26 23:49:51,546][105692] Updated weights for policy 0, policy_version 1161124 (0.0010) [2023-12-26 23:49:51,917][105620] Updated weights for policy 1, policy_version 1162382 (0.0007) [2023-12-26 23:49:51,970][105620] Updated weights for policy 1, policy_version 1162392 (0.0008) [2023-12-26 23:49:52,029][105620] Updated weights for policy 1, policy_version 1162402 (0.0008) [2023-12-26 23:49:52,254][105692] Updated weights for policy 0, policy_version 1161134 (0.0011) [2023-12-26 23:49:52,314][105692] Updated weights for policy 0, policy_version 1161144 (0.0011) [2023-12-26 23:49:52,381][105692] Updated weights for policy 0, policy_version 1161154 (0.0011) [2023-12-26 23:49:52,693][105620] Updated weights for policy 1, policy_version 1162412 (0.0007) [2023-12-26 23:49:52,752][105620] Updated weights for policy 1, policy_version 1162422 (0.0008) [2023-12-26 23:49:52,807][105620] Updated weights for policy 1, policy_version 1162432 (0.0008) [2023-12-26 23:49:53,062][105692] Updated weights for policy 0, policy_version 1161164 (0.0009) [2023-12-26 23:49:53,109][105692] Updated weights for policy 0, policy_version 1161174 (0.0009) [2023-12-26 23:49:53,157][105692] Updated weights for policy 0, policy_version 1161184 (0.0005) [2023-12-26 23:49:53,606][105620] Updated weights for policy 1, policy_version 1162442 (0.0009) [2023-12-26 23:49:53,653][105620] Updated weights for policy 1, policy_version 1162452 (0.0009) [2023-12-26 23:49:53,703][105620] Updated weights for policy 1, policy_version 1162462 (0.0008) [2023-12-26 23:49:53,766][105620] Updated weights for policy 1, policy_version 1162472 (0.0009) [2023-12-26 23:49:53,908][105692] Updated weights for policy 0, policy_version 1161194 (0.0011) [2023-12-26 23:49:53,963][105692] Updated weights for policy 0, policy_version 1161204 (0.0010) [2023-12-26 23:49:54,024][105692] Updated weights for policy 0, policy_version 1161215 (0.0009) [2023-12-26 23:49:54,379][105620] Updated weights for policy 1, policy_version 1162482 (0.0005) [2023-12-26 23:49:54,428][105620] Updated weights for policy 1, policy_version 1162492 (0.0005) [2023-12-26 23:49:54,478][105620] Updated weights for policy 1, policy_version 1162502 (0.0005) [2023-12-26 23:49:54,779][105692] Updated weights for policy 0, policy_version 1161225 (0.0010) [2023-12-26 23:49:54,834][105692] Updated weights for policy 0, policy_version 1161236 (0.0010) [2023-12-26 23:49:54,882][105692] Updated weights for policy 0, policy_version 1161246 (0.0006) [2023-12-26 23:49:54,940][105692] Updated weights for policy 0, policy_version 1161256 (0.0005) [2023-12-26 23:49:55,012][105620] Updated weights for policy 1, policy_version 1162512 (0.0005) [2023-12-26 23:49:55,071][105620] Updated weights for policy 1, policy_version 1162522 (0.0005) [2023-12-26 23:49:55,130][105620] Updated weights for policy 1, policy_version 1162532 (0.0005) [2023-12-26 23:49:55,674][105692] Updated weights for policy 0, policy_version 1161266 (0.0008) [2023-12-26 23:49:55,688][105620] Updated weights for policy 1, policy_version 1162542 (0.0007) [2023-12-26 23:49:55,733][105692] Updated weights for policy 0, policy_version 1161276 (0.0008) [2023-12-26 23:49:55,743][105620] Updated weights for policy 1, policy_version 1162552 (0.0005) [2023-12-26 23:49:55,795][105692] Updated weights for policy 0, policy_version 1161286 (0.0009) [2023-12-26 23:49:55,801][105620] Updated weights for policy 1, policy_version 1162562 (0.0006) [2023-12-26 23:49:56,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 594993152. Throughput: 0: 9825.4, 1: 9791.0. Samples: 594997616. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:49:56,062][104569] Avg episode reward: [(0, '9174.469'), (1, '9168.091')] [2023-12-26 23:49:56,482][105620] Updated weights for policy 1, policy_version 1162572 (0.0008) [2023-12-26 23:49:56,508][105692] Updated weights for policy 0, policy_version 1161296 (0.0009) [2023-12-26 23:49:56,541][105620] Updated weights for policy 1, policy_version 1162582 (0.0006) [2023-12-26 23:49:56,556][105692] Updated weights for policy 0, policy_version 1161306 (0.0010) [2023-12-26 23:49:56,597][105620] Updated weights for policy 1, policy_version 1162592 (0.0007) [2023-12-26 23:49:56,604][105692] Updated weights for policy 0, policy_version 1161316 (0.0010) [2023-12-26 23:49:57,308][105620] Updated weights for policy 1, policy_version 1162602 (0.0007) [2023-12-26 23:49:57,365][105692] Updated weights for policy 0, policy_version 1161326 (0.0011) [2023-12-26 23:49:57,372][105620] Updated weights for policy 1, policy_version 1162612 (0.0007) [2023-12-26 23:49:57,423][105692] Updated weights for policy 0, policy_version 1161336 (0.0010) [2023-12-26 23:49:57,429][105620] Updated weights for policy 1, policy_version 1162622 (0.0005) [2023-12-26 23:49:57,484][105692] Updated weights for policy 0, policy_version 1161346 (0.0010) [2023-12-26 23:49:57,493][105620] Updated weights for policy 1, policy_version 1162632 (0.0006) [2023-12-26 23:49:58,137][105620] Updated weights for policy 1, policy_version 1162642 (0.0008) [2023-12-26 23:49:58,205][105620] Updated weights for policy 1, policy_version 1162652 (0.0008) [2023-12-26 23:49:58,243][105692] Updated weights for policy 0, policy_version 1161356 (0.0010) [2023-12-26 23:49:58,266][105620] Updated weights for policy 1, policy_version 1162662 (0.0006) [2023-12-26 23:49:58,307][105692] Updated weights for policy 0, policy_version 1161366 (0.0011) [2023-12-26 23:49:58,377][105692] Updated weights for policy 0, policy_version 1161376 (0.0009) [2023-12-26 23:49:59,110][105620] Updated weights for policy 1, policy_version 1162672 (0.0007) [2023-12-26 23:49:59,136][105692] Updated weights for policy 0, policy_version 1161386 (0.0009) [2023-12-26 23:49:59,173][105620] Updated weights for policy 1, policy_version 1162682 (0.0008) [2023-12-26 23:49:59,200][105692] Updated weights for policy 0, policy_version 1161396 (0.0008) [2023-12-26 23:49:59,238][105620] Updated weights for policy 1, policy_version 1162692 (0.0007) [2023-12-26 23:49:59,270][105692] Updated weights for policy 0, policy_version 1161406 (0.0009) [2023-12-26 23:49:59,334][105692] Updated weights for policy 0, policy_version 1161416 (0.0011) [2023-12-26 23:49:59,895][105620] Updated weights for policy 1, policy_version 1162702 (0.0006) [2023-12-26 23:49:59,961][105620] Updated weights for policy 1, policy_version 1162712 (0.0008) [2023-12-26 23:50:00,027][105620] Updated weights for policy 1, policy_version 1162722 (0.0006) [2023-12-26 23:50:00,123][105692] Updated weights for policy 0, policy_version 1161426 (0.0005) [2023-12-26 23:50:00,191][105692] Updated weights for policy 0, policy_version 1161436 (0.0008) [2023-12-26 23:50:00,257][105692] Updated weights for policy 0, policy_version 1161446 (0.0007) [2023-12-26 23:50:00,582][105620] Updated weights for policy 1, policy_version 1162732 (0.0009) [2023-12-26 23:50:00,633][105620] Updated weights for policy 1, policy_version 1162742 (0.0010) [2023-12-26 23:50:00,681][105620] Updated weights for policy 1, policy_version 1162752 (0.0010) [2023-12-26 23:50:00,874][105692] Updated weights for policy 0, policy_version 1161456 (0.0005) [2023-12-26 23:50:00,919][105692] Updated weights for policy 0, policy_version 1161466 (0.0008) [2023-12-26 23:50:00,980][105692] Updated weights for policy 0, policy_version 1161477 (0.0010) [2023-12-26 23:50:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 595091456. Throughput: 0: 9818.2, 1: 9832.5. Samples: 595055060. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-26 23:50:01,062][104569] Avg episode reward: [(0, '9355.200'), (1, '9261.142')] [2023-12-26 23:50:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001161480_297385984.pth... [2023-12-26 23:50:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001162760_297705472.pth... [2023-12-26 23:50:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001160328_297091072.pth [2023-12-26 23:50:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001161608_297410560.pth [2023-12-26 23:50:01,394][105620] Updated weights for policy 1, policy_version 1162762 (0.0009) [2023-12-26 23:50:01,459][105620] Updated weights for policy 1, policy_version 1162772 (0.0007) [2023-12-26 23:50:01,518][105620] Updated weights for policy 1, policy_version 1162782 (0.0010) [2023-12-26 23:50:01,577][105620] Updated weights for policy 1, policy_version 1162792 (0.0011) [2023-12-26 23:50:01,804][105692] Updated weights for policy 0, policy_version 1161487 (0.0008) [2023-12-26 23:50:01,860][105692] Updated weights for policy 0, policy_version 1161497 (0.0008) [2023-12-26 23:50:01,926][105692] Updated weights for policy 0, policy_version 1161507 (0.0010) [2023-12-26 23:50:02,234][105620] Updated weights for policy 1, policy_version 1162802 (0.0009) [2023-12-26 23:50:02,299][105620] Updated weights for policy 1, policy_version 1162812 (0.0007) [2023-12-26 23:50:02,365][105620] Updated weights for policy 1, policy_version 1162822 (0.0006) [2023-12-26 23:50:02,749][105692] Updated weights for policy 0, policy_version 1161517 (0.0010) [2023-12-26 23:50:02,797][105692] Updated weights for policy 0, policy_version 1161527 (0.0009) [2023-12-26 23:50:02,857][105692] Updated weights for policy 0, policy_version 1161537 (0.0009) [2023-12-26 23:50:02,978][105620] Updated weights for policy 1, policy_version 1162832 (0.0007) [2023-12-26 23:50:03,029][105620] Updated weights for policy 1, policy_version 1162842 (0.0005) [2023-12-26 23:50:03,079][105620] Updated weights for policy 1, policy_version 1162852 (0.0005) [2023-12-26 23:50:03,600][105620] Updated weights for policy 1, policy_version 1162862 (0.0005) [2023-12-26 23:50:03,645][105620] Updated weights for policy 1, policy_version 1162872 (0.0008) [2023-12-26 23:50:03,699][105620] Updated weights for policy 1, policy_version 1162882 (0.0009) [2023-12-26 23:50:03,728][105692] Updated weights for policy 0, policy_version 1161547 (0.0007) [2023-12-26 23:50:03,786][105692] Updated weights for policy 0, policy_version 1161557 (0.0009) [2023-12-26 23:50:03,842][105692] Updated weights for policy 0, policy_version 1161567 (0.0009) [2023-12-26 23:50:04,429][105620] Updated weights for policy 1, policy_version 1162892 (0.0007) [2023-12-26 23:50:04,489][105620] Updated weights for policy 1, policy_version 1162902 (0.0011) [2023-12-26 23:50:04,551][105620] Updated weights for policy 1, policy_version 1162912 (0.0010) [2023-12-26 23:50:04,600][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000006 [2023-12-26 23:50:04,666][105692] Updated weights for policy 0, policy_version 1161577 (0.0010) [2023-12-26 23:50:04,723][105692] Updated weights for policy 0, policy_version 1161587 (0.0008) [2023-12-26 23:50:04,776][105692] Updated weights for policy 0, policy_version 1161597 (0.0009) [2023-12-26 23:50:04,832][105692] Updated weights for policy 0, policy_version 1161607 (0.0010) [2023-12-26 23:50:05,233][105620] Updated weights for policy 1, policy_version 1162922 (0.0010) [2023-12-26 23:50:05,281][105620] Updated weights for policy 1, policy_version 1162932 (0.0006) [2023-12-26 23:50:05,327][105620] Updated weights for policy 1, policy_version 1162942 (0.0005) [2023-12-26 23:50:05,379][105620] Updated weights for policy 1, policy_version 1162952 (0.0005) [2023-12-26 23:50:05,627][105692] Updated weights for policy 0, policy_version 1161617 (0.0009) [2023-12-26 23:50:05,681][105692] Updated weights for policy 0, policy_version 1161627 (0.0009) [2023-12-26 23:50:05,733][105692] Updated weights for policy 0, policy_version 1161637 (0.0009) [2023-12-26 23:50:05,998][105620] Updated weights for policy 1, policy_version 1162962 (0.0005) [2023-12-26 23:50:06,044][105620] Updated weights for policy 1, policy_version 1162972 (0.0005) [2023-12-26 23:50:06,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 595181568. Throughput: 0: 9613.4, 1: 10056.6. Samples: 595172916. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:50:06,063][104569] Avg episode reward: [(0, '9170.892'), (1, '9352.235')] [2023-12-26 23:50:06,089][105620] Updated weights for policy 1, policy_version 1162982 (0.0006) [2023-12-26 23:50:06,635][105692] Updated weights for policy 0, policy_version 1161647 (0.0009) [2023-12-26 23:50:06,690][105620] Updated weights for policy 1, policy_version 1162992 (0.0006) [2023-12-26 23:50:06,704][105692] Updated weights for policy 0, policy_version 1161657 (0.0009) [2023-12-26 23:50:06,722][105585] KL-divergence is very high: 161.9226 [2023-12-26 23:50:06,745][105620] Updated weights for policy 1, policy_version 1163002 (0.0006) [2023-12-26 23:50:06,756][105692] Updated weights for policy 0, policy_version 1161667 (0.0007) [2023-12-26 23:50:06,764][105585] KL-divergence is very high: 201.0841 [2023-12-26 23:50:06,803][105620] Updated weights for policy 1, policy_version 1163012 (0.0008) [2023-12-26 23:50:07,487][105692] Updated weights for policy 0, policy_version 1161677 (0.0007) [2023-12-26 23:50:07,544][105692] Updated weights for policy 0, policy_version 1161687 (0.0009) [2023-12-26 23:50:07,572][105620] Updated weights for policy 1, policy_version 1163022 (0.0008) [2023-12-26 23:50:07,608][105692] Updated weights for policy 0, policy_version 1161697 (0.0007) [2023-12-26 23:50:07,622][105620] Updated weights for policy 1, policy_version 1163032 (0.0008) [2023-12-26 23:50:07,671][105620] Updated weights for policy 1, policy_version 1163042 (0.0007) [2023-12-26 23:50:08,283][105620] Updated weights for policy 1, policy_version 1163052 (0.0008) [2023-12-26 23:50:08,315][105692] Updated weights for policy 0, policy_version 1161707 (0.0008) [2023-12-26 23:50:08,346][105620] Updated weights for policy 1, policy_version 1163062 (0.0008) [2023-12-26 23:50:08,377][105692] Updated weights for policy 0, policy_version 1161717 (0.0008) [2023-12-26 23:50:08,414][105620] Updated weights for policy 1, policy_version 1163072 (0.0008) [2023-12-26 23:50:08,436][105692] Updated weights for policy 0, policy_version 1161727 (0.0007) [2023-12-26 23:50:09,083][105620] Updated weights for policy 1, policy_version 1163082 (0.0006) [2023-12-26 23:50:09,091][105692] Updated weights for policy 0, policy_version 1161737 (0.0011) [2023-12-26 23:50:09,134][105620] Updated weights for policy 1, policy_version 1163092 (0.0008) [2023-12-26 23:50:09,147][105692] Updated weights for policy 0, policy_version 1161747 (0.0010) [2023-12-26 23:50:09,191][105620] Updated weights for policy 1, policy_version 1163102 (0.0006) [2023-12-26 23:50:09,208][105692] Updated weights for policy 0, policy_version 1161757 (0.0011) [2023-12-26 23:50:09,257][105620] Updated weights for policy 1, policy_version 1163112 (0.0008) [2023-12-26 23:50:09,275][105692] Updated weights for policy 0, policy_version 1161767 (0.0010) [2023-12-26 23:50:10,028][105620] Updated weights for policy 1, policy_version 1163122 (0.0008) [2023-12-26 23:50:10,080][105620] Updated weights for policy 1, policy_version 1163132 (0.0008) [2023-12-26 23:50:10,081][105692] Updated weights for policy 0, policy_version 1161777 (0.0010) [2023-12-26 23:50:10,133][105692] Updated weights for policy 0, policy_version 1161787 (0.0010) [2023-12-26 23:50:10,135][105620] Updated weights for policy 1, policy_version 1163142 (0.0008) [2023-12-26 23:50:10,179][105692] Updated weights for policy 0, policy_version 1161797 (0.0010) [2023-12-26 23:50:10,839][105620] Updated weights for policy 1, policy_version 1163152 (0.0008) [2023-12-26 23:50:10,904][105620] Updated weights for policy 1, policy_version 1163162 (0.0007) [2023-12-26 23:50:10,964][105620] Updated weights for policy 1, policy_version 1163172 (0.0009) [2023-12-26 23:50:10,968][105692] Updated weights for policy 0, policy_version 1161807 (0.0007) [2023-12-26 23:50:11,026][105692] Updated weights for policy 0, policy_version 1161817 (0.0006) [2023-12-26 23:50:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 595279872. Throughput: 0: 9510.2, 1: 10063.4. Samples: 595289128. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:50:11,062][104569] Avg episode reward: [(0, '8990.137'), (1, '9169.803')] [2023-12-26 23:50:11,085][105692] Updated weights for policy 0, policy_version 1161827 (0.0009) [2023-12-26 23:50:11,767][105692] Updated weights for policy 0, policy_version 1161837 (0.0009) [2023-12-26 23:50:11,767][105620] Updated weights for policy 1, policy_version 1163182 (0.0009) [2023-12-26 23:50:11,828][105692] Updated weights for policy 0, policy_version 1161847 (0.0007) [2023-12-26 23:50:11,830][105620] Updated weights for policy 1, policy_version 1163192 (0.0007) [2023-12-26 23:50:11,890][105692] Updated weights for policy 0, policy_version 1161857 (0.0006) [2023-12-26 23:50:11,892][105620] Updated weights for policy 1, policy_version 1163202 (0.0007) [2023-12-26 23:50:12,587][105692] Updated weights for policy 0, policy_version 1161867 (0.0008) [2023-12-26 23:50:12,598][105620] Updated weights for policy 1, policy_version 1163212 (0.0007) [2023-12-26 23:50:12,651][105692] Updated weights for policy 0, policy_version 1161877 (0.0008) [2023-12-26 23:50:12,660][105620] Updated weights for policy 1, policy_version 1163222 (0.0007) [2023-12-26 23:50:12,704][105692] Updated weights for policy 0, policy_version 1161887 (0.0009) [2023-12-26 23:50:12,721][105620] Updated weights for policy 1, policy_version 1163232 (0.0011) [2023-12-26 23:50:13,368][105620] Updated weights for policy 1, policy_version 1163242 (0.0010) [2023-12-26 23:50:13,412][105692] Updated weights for policy 0, policy_version 1161897 (0.0007) [2023-12-26 23:50:13,429][105620] Updated weights for policy 1, policy_version 1163252 (0.0007) [2023-12-26 23:50:13,465][105692] Updated weights for policy 0, policy_version 1161907 (0.0006) [2023-12-26 23:50:13,483][105620] Updated weights for policy 1, policy_version 1163262 (0.0009) [2023-12-26 23:50:13,516][105692] Updated weights for policy 0, policy_version 1161917 (0.0006) [2023-12-26 23:50:13,542][105620] Updated weights for policy 1, policy_version 1163272 (0.0008) [2023-12-26 23:50:13,566][105692] Updated weights for policy 0, policy_version 1161927 (0.0008) [2023-12-26 23:50:14,147][105692] Updated weights for policy 0, policy_version 1161937 (0.0006) [2023-12-26 23:50:14,198][105692] Updated weights for policy 0, policy_version 1161947 (0.0006) [2023-12-26 23:50:14,249][105692] Updated weights for policy 0, policy_version 1161957 (0.0005) [2023-12-26 23:50:14,381][105620] Updated weights for policy 1, policy_version 1163282 (0.0010) [2023-12-26 23:50:14,447][105620] Updated weights for policy 1, policy_version 1163292 (0.0010) [2023-12-26 23:50:14,510][105620] Updated weights for policy 1, policy_version 1163302 (0.0009) [2023-12-26 23:50:14,841][105692] Updated weights for policy 0, policy_version 1161967 (0.0009) [2023-12-26 23:50:14,894][105692] Updated weights for policy 0, policy_version 1161977 (0.0011) [2023-12-26 23:50:14,957][105692] Updated weights for policy 0, policy_version 1161987 (0.0011) [2023-12-26 23:50:15,264][105620] Updated weights for policy 1, policy_version 1163312 (0.0010) [2023-12-26 23:50:15,324][105620] Updated weights for policy 1, policy_version 1163322 (0.0011) [2023-12-26 23:50:15,381][105620] Updated weights for policy 1, policy_version 1163332 (0.0010) [2023-12-26 23:50:15,691][105692] Updated weights for policy 0, policy_version 1161997 (0.0008) [2023-12-26 23:50:15,738][105692] Updated weights for policy 0, policy_version 1162007 (0.0010) [2023-12-26 23:50:15,793][105692] Updated weights for policy 0, policy_version 1162017 (0.0010) [2023-12-26 23:50:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 595378176. Throughput: 0: 9566.4, 1: 10021.3. Samples: 595347480. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:50:16,062][104569] Avg episode reward: [(0, '8991.988'), (1, '8905.758')] [2023-12-26 23:50:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001162024_297525248.pth... [2023-12-26 23:50:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001163336_297852928.pth... [2023-12-26 23:50:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001160904_297238528.pth [2023-12-26 23:50:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001162184_297558016.pth [2023-12-26 23:50:16,138][105620] Updated weights for policy 1, policy_version 1163342 (0.0009) [2023-12-26 23:50:16,199][105620] Updated weights for policy 1, policy_version 1163352 (0.0010) [2023-12-26 23:50:16,264][105620] Updated weights for policy 1, policy_version 1163362 (0.0010) [2023-12-26 23:50:16,432][105692] Updated weights for policy 0, policy_version 1162027 (0.0010) [2023-12-26 23:50:16,503][105692] Updated weights for policy 0, policy_version 1162037 (0.0012) [2023-12-26 23:50:16,569][105692] Updated weights for policy 0, policy_version 1162047 (0.0011) [2023-12-26 23:50:17,099][105620] Updated weights for policy 1, policy_version 1163372 (0.0010) [2023-12-26 23:50:17,152][105620] Updated weights for policy 1, policy_version 1163383 (0.0009) [2023-12-26 23:50:17,161][105692] Updated weights for policy 0, policy_version 1162057 (0.0011) [2023-12-26 23:50:17,203][105620] Updated weights for policy 1, policy_version 1163393 (0.0007) [2023-12-26 23:50:17,220][105692] Updated weights for policy 0, policy_version 1162067 (0.0010) [2023-12-26 23:50:17,270][105692] Updated weights for policy 0, policy_version 1162077 (0.0010) [2023-12-26 23:50:17,335][105692] Updated weights for policy 0, policy_version 1162087 (0.0010) [2023-12-26 23:50:17,950][105692] Updated weights for policy 0, policy_version 1162097 (0.0007) [2023-12-26 23:50:17,957][105620] Updated weights for policy 1, policy_version 1163403 (0.0008) [2023-12-26 23:50:18,014][105692] Updated weights for policy 0, policy_version 1162107 (0.0009) [2023-12-26 23:50:18,021][105620] Updated weights for policy 1, policy_version 1163413 (0.0006) [2023-12-26 23:50:18,067][105692] Updated weights for policy 0, policy_version 1162117 (0.0011) [2023-12-26 23:50:18,088][105620] Updated weights for policy 1, policy_version 1163423 (0.0006) [2023-12-26 23:50:18,744][105620] Updated weights for policy 1, policy_version 1163433 (0.0008) [2023-12-26 23:50:18,787][105692] Updated weights for policy 0, policy_version 1162127 (0.0011) [2023-12-26 23:50:18,805][105620] Updated weights for policy 1, policy_version 1163443 (0.0006) [2023-12-26 23:50:18,854][105692] Updated weights for policy 0, policy_version 1162137 (0.0011) [2023-12-26 23:50:18,864][105620] Updated weights for policy 1, policy_version 1163453 (0.0006) [2023-12-26 23:50:18,910][105692] Updated weights for policy 0, policy_version 1162147 (0.0011) [2023-12-26 23:50:18,916][105620] Updated weights for policy 1, policy_version 1163463 (0.0005) [2023-12-26 23:50:19,543][105620] Updated weights for policy 1, policy_version 1163473 (0.0009) [2023-12-26 23:50:19,592][105620] Updated weights for policy 1, policy_version 1163483 (0.0008) [2023-12-26 23:50:19,620][105692] Updated weights for policy 0, policy_version 1162157 (0.0009) [2023-12-26 23:50:19,653][105620] Updated weights for policy 1, policy_version 1163493 (0.0009) [2023-12-26 23:50:19,684][105692] Updated weights for policy 0, policy_version 1162167 (0.0008) [2023-12-26 23:50:19,755][105692] Updated weights for policy 0, policy_version 1162177 (0.0009) [2023-12-26 23:50:20,406][105620] Updated weights for policy 1, policy_version 1163503 (0.0008) [2023-12-26 23:50:20,457][105620] Updated weights for policy 1, policy_version 1163513 (0.0009) [2023-12-26 23:50:20,517][105692] Updated weights for policy 0, policy_version 1162187 (0.0009) [2023-12-26 23:50:20,518][105620] Updated weights for policy 1, policy_version 1163523 (0.0009) [2023-12-26 23:50:20,578][105692] Updated weights for policy 0, policy_version 1162197 (0.0009) [2023-12-26 23:50:20,640][105692] Updated weights for policy 0, policy_version 1162207 (0.0009) [2023-12-26 23:50:21,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 595476480. Throughput: 0: 9744.8, 1: 9914.8. Samples: 595467684. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:50:21,063][104569] Avg episode reward: [(0, '8991.414'), (1, '8899.631')] [2023-12-26 23:50:21,231][105620] Updated weights for policy 1, policy_version 1163533 (0.0009) [2023-12-26 23:50:21,295][105620] Updated weights for policy 1, policy_version 1163543 (0.0008) [2023-12-26 23:50:21,344][105692] Updated weights for policy 0, policy_version 1162217 (0.0009) [2023-12-26 23:50:21,364][105620] Updated weights for policy 1, policy_version 1163553 (0.0008) [2023-12-26 23:50:21,414][105692] Updated weights for policy 0, policy_version 1162227 (0.0007) [2023-12-26 23:50:21,473][105692] Updated weights for policy 0, policy_version 1162237 (0.0009) [2023-12-26 23:50:21,527][105692] Updated weights for policy 0, policy_version 1162247 (0.0010) [2023-12-26 23:50:22,099][105620] Updated weights for policy 1, policy_version 1163563 (0.0009) [2023-12-26 23:50:22,157][105620] Updated weights for policy 1, policy_version 1163573 (0.0008) [2023-12-26 23:50:22,219][105620] Updated weights for policy 1, policy_version 1163583 (0.0009) [2023-12-26 23:50:22,321][105692] Updated weights for policy 0, policy_version 1162257 (0.0009) [2023-12-26 23:50:22,382][105692] Updated weights for policy 0, policy_version 1162267 (0.0009) [2023-12-26 23:50:22,429][105692] Updated weights for policy 0, policy_version 1162277 (0.0008) [2023-12-26 23:50:22,993][105620] Updated weights for policy 1, policy_version 1163593 (0.0009) [2023-12-26 23:50:23,053][105620] Updated weights for policy 1, policy_version 1163603 (0.0009) [2023-12-26 23:50:23,116][105620] Updated weights for policy 1, policy_version 1163613 (0.0008) [2023-12-26 23:50:23,183][105620] Updated weights for policy 1, policy_version 1163623 (0.0008) [2023-12-26 23:50:23,216][105692] Updated weights for policy 0, policy_version 1162287 (0.0009) [2023-12-26 23:50:23,275][105692] Updated weights for policy 0, policy_version 1162297 (0.0009) [2023-12-26 23:50:23,340][105692] Updated weights for policy 0, policy_version 1162307 (0.0009) [2023-12-26 23:50:23,828][105620] Updated weights for policy 1, policy_version 1163633 (0.0008) [2023-12-26 23:50:23,898][105620] Updated weights for policy 1, policy_version 1163643 (0.0008) [2023-12-26 23:50:23,964][105620] Updated weights for policy 1, policy_version 1163653 (0.0008) [2023-12-26 23:50:24,015][105692] Updated weights for policy 0, policy_version 1162317 (0.0009) [2023-12-26 23:50:24,068][105692] Updated weights for policy 0, policy_version 1162327 (0.0010) [2023-12-26 23:50:24,122][105692] Updated weights for policy 0, policy_version 1162338 (0.0011) [2023-12-26 23:50:24,511][105620] Updated weights for policy 1, policy_version 1163663 (0.0008) [2023-12-26 23:50:24,568][105620] Updated weights for policy 1, policy_version 1163673 (0.0008) [2023-12-26 23:50:24,628][105620] Updated weights for policy 1, policy_version 1163683 (0.0009) [2023-12-26 23:50:24,865][105692] Updated weights for policy 0, policy_version 1162348 (0.0009) [2023-12-26 23:50:24,922][105692] Updated weights for policy 0, policy_version 1162358 (0.0009) [2023-12-26 23:50:24,967][105692] Updated weights for policy 0, policy_version 1162368 (0.0007) [2023-12-26 23:50:25,328][105620] Updated weights for policy 1, policy_version 1163693 (0.0010) [2023-12-26 23:50:25,374][105620] Updated weights for policy 1, policy_version 1163703 (0.0010) [2023-12-26 23:50:25,422][105620] Updated weights for policy 1, policy_version 1163713 (0.0010) [2023-12-26 23:50:25,515][105692] Updated weights for policy 0, policy_version 1162378 (0.0006) [2023-12-26 23:50:25,575][105692] Updated weights for policy 0, policy_version 1162388 (0.0008) [2023-12-26 23:50:25,623][105692] Updated weights for policy 0, policy_version 1162398 (0.0008) [2023-12-26 23:50:25,676][105692] Updated weights for policy 0, policy_version 1162408 (0.0008) [2023-12-26 23:50:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 595574784. Throughput: 0: 9744.9, 1: 9897.5. Samples: 595584916. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:50:26,062][104569] Avg episode reward: [(0, '8961.628'), (1, '9169.592')] [2023-12-26 23:50:26,116][105620] Updated weights for policy 1, policy_version 1163723 (0.0009) [2023-12-26 23:50:26,186][105620] Updated weights for policy 1, policy_version 1163733 (0.0005) [2023-12-26 23:50:26,257][105620] Updated weights for policy 1, policy_version 1163743 (0.0006) [2023-12-26 23:50:26,381][105692] Updated weights for policy 0, policy_version 1162418 (0.0005) [2023-12-26 23:50:26,432][105692] Updated weights for policy 0, policy_version 1162428 (0.0005) [2023-12-26 23:50:26,483][105692] Updated weights for policy 0, policy_version 1162438 (0.0005) [2023-12-26 23:50:27,022][105692] Updated weights for policy 0, policy_version 1162448 (0.0008) [2023-12-26 23:50:27,035][105620] Updated weights for policy 1, policy_version 1163753 (0.0009) [2023-12-26 23:50:27,076][105692] Updated weights for policy 0, policy_version 1162458 (0.0010) [2023-12-26 23:50:27,082][105620] Updated weights for policy 1, policy_version 1163763 (0.0005) [2023-12-26 23:50:27,132][105620] Updated weights for policy 1, policy_version 1163773 (0.0006) [2023-12-26 23:50:27,134][105692] Updated weights for policy 0, policy_version 1162468 (0.0010) [2023-12-26 23:50:27,196][105620] Updated weights for policy 1, policy_version 1163783 (0.0005) [2023-12-26 23:50:27,841][105692] Updated weights for policy 0, policy_version 1162478 (0.0010) [2023-12-26 23:50:27,895][105692] Updated weights for policy 0, policy_version 1162488 (0.0010) [2023-12-26 23:50:27,908][105620] Updated weights for policy 1, policy_version 1163793 (0.0006) [2023-12-26 23:50:27,939][105692] Updated weights for policy 0, policy_version 1162498 (0.0008) [2023-12-26 23:50:27,964][105620] Updated weights for policy 1, policy_version 1163803 (0.0008) [2023-12-26 23:50:28,021][105620] Updated weights for policy 1, policy_version 1163813 (0.0008) [2023-12-26 23:50:28,657][105692] Updated weights for policy 0, policy_version 1162508 (0.0007) [2023-12-26 23:50:28,712][105692] Updated weights for policy 0, policy_version 1162518 (0.0009) [2023-12-26 23:50:28,763][105620] Updated weights for policy 1, policy_version 1163823 (0.0009) [2023-12-26 23:50:28,768][105692] Updated weights for policy 0, policy_version 1162528 (0.0006) [2023-12-26 23:50:28,822][105620] Updated weights for policy 1, policy_version 1163833 (0.0009) [2023-12-26 23:50:28,884][105620] Updated weights for policy 1, policy_version 1163843 (0.0009) [2023-12-26 23:50:29,465][105692] Updated weights for policy 0, policy_version 1162538 (0.0005) [2023-12-26 23:50:29,529][105692] Updated weights for policy 0, policy_version 1162548 (0.0005) [2023-12-26 23:50:29,585][105692] Updated weights for policy 0, policy_version 1162558 (0.0005) [2023-12-26 23:50:29,617][105620] Updated weights for policy 1, policy_version 1163853 (0.0007) [2023-12-26 23:50:29,657][105692] Updated weights for policy 0, policy_version 1162568 (0.0007) [2023-12-26 23:50:29,673][105620] Updated weights for policy 1, policy_version 1163863 (0.0005) [2023-12-26 23:50:29,725][105620] Updated weights for policy 1, policy_version 1163873 (0.0005) [2023-12-26 23:50:30,298][105620] Updated weights for policy 1, policy_version 1163883 (0.0007) [2023-12-26 23:50:30,344][105620] Updated weights for policy 1, policy_version 1163893 (0.0009) [2023-12-26 23:50:30,391][105692] Updated weights for policy 0, policy_version 1162578 (0.0008) [2023-12-26 23:50:30,409][105620] Updated weights for policy 1, policy_version 1163903 (0.0006) [2023-12-26 23:50:30,440][105692] Updated weights for policy 0, policy_version 1162588 (0.0006) [2023-12-26 23:50:30,486][105692] Updated weights for policy 0, policy_version 1162598 (0.0005) [2023-12-26 23:50:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 595673088. Throughput: 0: 9776.7, 1: 9939.1. Samples: 595645132. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:50:31,062][104569] Avg episode reward: [(0, '8598.815'), (1, '9082.747')] [2023-12-26 23:50:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001162600_297672704.pth... [2023-12-26 23:50:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001163912_298000384.pth... [2023-12-26 23:50:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001161480_297385984.pth [2023-12-26 23:50:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001162760_297705472.pth [2023-12-26 23:50:31,165][105620] Updated weights for policy 1, policy_version 1163913 (0.0007) [2023-12-26 23:50:31,230][105620] Updated weights for policy 1, policy_version 1163923 (0.0007) [2023-12-26 23:50:31,259][105692] Updated weights for policy 0, policy_version 1162608 (0.0007) [2023-12-26 23:50:31,293][105620] Updated weights for policy 1, policy_version 1163933 (0.0007) [2023-12-26 23:50:31,315][105692] Updated weights for policy 0, policy_version 1162618 (0.0008) [2023-12-26 23:50:31,357][105620] Updated weights for policy 1, policy_version 1163943 (0.0006) [2023-12-26 23:50:31,370][105692] Updated weights for policy 0, policy_version 1162628 (0.0007) [2023-12-26 23:50:32,036][105692] Updated weights for policy 0, policy_version 1162638 (0.0007) [2023-12-26 23:50:32,094][105692] Updated weights for policy 0, policy_version 1162648 (0.0008) [2023-12-26 23:50:32,100][105620] Updated weights for policy 1, policy_version 1163953 (0.0006) [2023-12-26 23:50:32,139][105692] Updated weights for policy 0, policy_version 1162658 (0.0006) [2023-12-26 23:50:32,156][105620] Updated weights for policy 1, policy_version 1163963 (0.0008) [2023-12-26 23:50:32,204][105620] Updated weights for policy 1, policy_version 1163973 (0.0008) [2023-12-26 23:50:32,863][105692] Updated weights for policy 0, policy_version 1162668 (0.0008) [2023-12-26 23:50:32,928][105692] Updated weights for policy 0, policy_version 1162678 (0.0007) [2023-12-26 23:50:32,992][105692] Updated weights for policy 0, policy_version 1162688 (0.0007) [2023-12-26 23:50:33,007][105620] Updated weights for policy 1, policy_version 1163983 (0.0009) [2023-12-26 23:50:33,059][105620] Updated weights for policy 1, policy_version 1163993 (0.0009) [2023-12-26 23:50:33,112][105620] Updated weights for policy 1, policy_version 1164003 (0.0009) [2023-12-26 23:50:33,563][105692] Updated weights for policy 0, policy_version 1162698 (0.0007) [2023-12-26 23:50:33,616][105692] Updated weights for policy 0, policy_version 1162708 (0.0009) [2023-12-26 23:50:33,662][105692] Updated weights for policy 0, policy_version 1162718 (0.0008) [2023-12-26 23:50:33,715][105692] Updated weights for policy 0, policy_version 1162728 (0.0009) [2023-12-26 23:50:33,921][105620] Updated weights for policy 1, policy_version 1164013 (0.0008) [2023-12-26 23:50:33,979][105620] Updated weights for policy 1, policy_version 1164023 (0.0009) [2023-12-26 23:50:34,027][105620] Updated weights for policy 1, policy_version 1164033 (0.0009) [2023-12-26 23:50:34,422][105692] Updated weights for policy 0, policy_version 1162738 (0.0009) [2023-12-26 23:50:34,482][105692] Updated weights for policy 0, policy_version 1162748 (0.0009) [2023-12-26 23:50:34,529][105692] Updated weights for policy 0, policy_version 1162758 (0.0010) [2023-12-26 23:50:34,835][105620] Updated weights for policy 1, policy_version 1164043 (0.0008) [2023-12-26 23:50:34,898][105620] Updated weights for policy 1, policy_version 1164053 (0.0009) [2023-12-26 23:50:34,960][105620] Updated weights for policy 1, policy_version 1164063 (0.0009) [2023-12-26 23:50:35,304][105692] Updated weights for policy 0, policy_version 1162768 (0.0008) [2023-12-26 23:50:35,354][105692] Updated weights for policy 0, policy_version 1162778 (0.0009) [2023-12-26 23:50:35,404][105692] Updated weights for policy 0, policy_version 1162788 (0.0008) [2023-12-26 23:50:35,739][105620] Updated weights for policy 1, policy_version 1164073 (0.0009) [2023-12-26 23:50:35,806][105620] Updated weights for policy 1, policy_version 1164083 (0.0009) [2023-12-26 23:50:35,872][105620] Updated weights for policy 1, policy_version 1164093 (0.0006) [2023-12-26 23:50:35,919][105620] Updated weights for policy 1, policy_version 1164103 (0.0005) [2023-12-26 23:50:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 595771392. Throughput: 0: 9688.4, 1: 9916.4. Samples: 595760880. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:50:36,062][104569] Avg episode reward: [(0, '8991.832'), (1, '9174.259')] [2023-12-26 23:50:36,139][105692] Updated weights for policy 0, policy_version 1162798 (0.0009) [2023-12-26 23:50:36,196][105692] Updated weights for policy 0, policy_version 1162808 (0.0009) [2023-12-26 23:50:36,253][105692] Updated weights for policy 0, policy_version 1162818 (0.0010) [2023-12-26 23:50:36,547][105620] Updated weights for policy 1, policy_version 1164113 (0.0005) [2023-12-26 23:50:36,612][105620] Updated weights for policy 1, policy_version 1164123 (0.0010) [2023-12-26 23:50:36,682][105620] Updated weights for policy 1, policy_version 1164133 (0.0006) [2023-12-26 23:50:37,118][105692] Updated weights for policy 0, policy_version 1162828 (0.0009) [2023-12-26 23:50:37,173][105692] Updated weights for policy 0, policy_version 1162838 (0.0009) [2023-12-26 23:50:37,235][105692] Updated weights for policy 0, policy_version 1162848 (0.0009) [2023-12-26 23:50:37,304][105620] Updated weights for policy 1, policy_version 1164143 (0.0006) [2023-12-26 23:50:37,364][105620] Updated weights for policy 1, policy_version 1164153 (0.0007) [2023-12-26 23:50:37,420][105620] Updated weights for policy 1, policy_version 1164163 (0.0008) [2023-12-26 23:50:38,021][105692] Updated weights for policy 0, policy_version 1162858 (0.0007) [2023-12-26 23:50:38,072][105692] Updated weights for policy 0, policy_version 1162868 (0.0006) [2023-12-26 23:50:38,116][105692] Updated weights for policy 0, policy_version 1162878 (0.0005) [2023-12-26 23:50:38,162][105692] Updated weights for policy 0, policy_version 1162888 (0.0005) [2023-12-26 23:50:38,183][105620] Updated weights for policy 1, policy_version 1164173 (0.0007) [2023-12-26 23:50:38,237][105620] Updated weights for policy 1, policy_version 1164183 (0.0005) [2023-12-26 23:50:38,298][105620] Updated weights for policy 1, policy_version 1164193 (0.0006) [2023-12-26 23:50:38,833][105692] Updated weights for policy 0, policy_version 1162898 (0.0010) [2023-12-26 23:50:38,884][105692] Updated weights for policy 0, policy_version 1162908 (0.0008) [2023-12-26 23:50:38,935][105692] Updated weights for policy 0, policy_version 1162918 (0.0006) [2023-12-26 23:50:38,989][105620] Updated weights for policy 1, policy_version 1164203 (0.0008) [2023-12-26 23:50:39,054][105620] Updated weights for policy 1, policy_version 1164213 (0.0007) [2023-12-26 23:50:39,112][105620] Updated weights for policy 1, policy_version 1164223 (0.0011) [2023-12-26 23:50:39,749][105692] Updated weights for policy 0, policy_version 1162928 (0.0008) [2023-12-26 23:50:39,750][105620] Updated weights for policy 1, policy_version 1164233 (0.0009) [2023-12-26 23:50:39,809][105692] Updated weights for policy 0, policy_version 1162938 (0.0006) [2023-12-26 23:50:39,811][105620] Updated weights for policy 1, policy_version 1164243 (0.0010) [2023-12-26 23:50:39,875][105620] Updated weights for policy 1, policy_version 1164253 (0.0008) [2023-12-26 23:50:39,886][105692] Updated weights for policy 0, policy_version 1162948 (0.0008) [2023-12-26 23:50:39,948][105620] Updated weights for policy 1, policy_version 1164263 (0.0008) [2023-12-26 23:50:40,626][105692] Updated weights for policy 0, policy_version 1162958 (0.0009) [2023-12-26 23:50:40,674][105620] Updated weights for policy 1, policy_version 1164273 (0.0006) [2023-12-26 23:50:40,690][105692] Updated weights for policy 0, policy_version 1162968 (0.0007) [2023-12-26 23:50:40,727][105620] Updated weights for policy 1, policy_version 1164283 (0.0009) [2023-12-26 23:50:40,755][105692] Updated weights for policy 0, policy_version 1162978 (0.0006) [2023-12-26 23:50:40,789][105620] Updated weights for policy 1, policy_version 1164293 (0.0008) [2023-12-26 23:50:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 595869696. Throughput: 0: 9675.4, 1: 9852.4. Samples: 595876368. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:50:41,063][104569] Avg episode reward: [(0, '9354.802'), (1, '9349.689')] [2023-12-26 23:50:41,475][105692] Updated weights for policy 0, policy_version 1162988 (0.0007) [2023-12-26 23:50:41,501][105620] Updated weights for policy 1, policy_version 1164303 (0.0007) [2023-12-26 23:50:41,539][105692] Updated weights for policy 0, policy_version 1162998 (0.0008) [2023-12-26 23:50:41,562][105620] Updated weights for policy 1, policy_version 1164313 (0.0006) [2023-12-26 23:50:41,593][105692] Updated weights for policy 0, policy_version 1163008 (0.0007) [2023-12-26 23:50:41,627][105620] Updated weights for policy 1, policy_version 1164323 (0.0006) [2023-12-26 23:50:42,305][105620] Updated weights for policy 1, policy_version 1164333 (0.0008) [2023-12-26 23:50:42,373][105620] Updated weights for policy 1, policy_version 1164343 (0.0008) [2023-12-26 23:50:42,411][105692] Updated weights for policy 0, policy_version 1163018 (0.0009) [2023-12-26 23:50:42,439][105620] Updated weights for policy 1, policy_version 1164353 (0.0006) [2023-12-26 23:50:42,470][105692] Updated weights for policy 0, policy_version 1163028 (0.0009) [2023-12-26 23:50:42,528][105692] Updated weights for policy 0, policy_version 1163038 (0.0010) [2023-12-26 23:50:42,593][105692] Updated weights for policy 0, policy_version 1163048 (0.0011) [2023-12-26 23:50:43,032][105620] Updated weights for policy 1, policy_version 1164363 (0.0007) [2023-12-26 23:50:43,089][105620] Updated weights for policy 1, policy_version 1164373 (0.0008) [2023-12-26 23:50:43,150][105620] Updated weights for policy 1, policy_version 1164383 (0.0009) [2023-12-26 23:50:43,418][105692] Updated weights for policy 0, policy_version 1163058 (0.0009) [2023-12-26 23:50:43,471][105692] Updated weights for policy 0, policy_version 1163068 (0.0009) [2023-12-26 23:50:43,526][105692] Updated weights for policy 0, policy_version 1163078 (0.0010) [2023-12-26 23:50:43,783][105620] Updated weights for policy 1, policy_version 1164393 (0.0009) [2023-12-26 23:50:43,833][105620] Updated weights for policy 1, policy_version 1164403 (0.0006) [2023-12-26 23:50:43,890][105620] Updated weights for policy 1, policy_version 1164413 (0.0005) [2023-12-26 23:50:43,944][105620] Updated weights for policy 1, policy_version 1164423 (0.0005) [2023-12-26 23:50:44,398][105692] Updated weights for policy 0, policy_version 1163088 (0.0010) [2023-12-26 23:50:44,455][105692] Updated weights for policy 0, policy_version 1163098 (0.0010) [2023-12-26 23:50:44,510][105620] Updated weights for policy 1, policy_version 1164433 (0.0008) [2023-12-26 23:50:44,512][105692] Updated weights for policy 0, policy_version 1163108 (0.0006) [2023-12-26 23:50:44,562][105620] Updated weights for policy 1, policy_version 1164443 (0.0008) [2023-12-26 23:50:44,616][105620] Updated weights for policy 1, policy_version 1164453 (0.0006) [2023-12-26 23:50:45,266][105620] Updated weights for policy 1, policy_version 1164463 (0.0008) [2023-12-26 23:50:45,328][105620] Updated weights for policy 1, policy_version 1164473 (0.0009) [2023-12-26 23:50:45,386][105692] Updated weights for policy 0, policy_version 1163118 (0.0007) [2023-12-26 23:50:45,388][105620] Updated weights for policy 1, policy_version 1164483 (0.0007) [2023-12-26 23:50:45,442][105692] Updated weights for policy 0, policy_version 1163128 (0.0007) [2023-12-26 23:50:45,505][105692] Updated weights for policy 0, policy_version 1163138 (0.0010) [2023-12-26 23:50:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 595959808. Throughput: 0: 9625.1, 1: 9907.3. Samples: 595934024. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:50:46,063][104569] Avg episode reward: [(0, '9355.496'), (1, '9257.678')] [2023-12-26 23:50:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001164488_298147840.pth... [2023-12-26 23:50:46,069][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001163336_297852928.pth [2023-12-26 23:50:46,101][105692] Updated weights for policy 0, policy_version 1163148 (0.0007) [2023-12-26 23:50:46,145][105692] Updated weights for policy 0, policy_version 1163158 (0.0010) [2023-12-26 23:50:46,200][105692] Updated weights for policy 0, policy_version 1163168 (0.0007) [2023-12-26 23:50:46,227][105620] Updated weights for policy 1, policy_version 1164493 (0.0009) [2023-12-26 23:50:46,242][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001163176_297820160.pth... [2023-12-26 23:50:46,247][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001162024_297525248.pth [2023-12-26 23:50:46,286][105620] Updated weights for policy 1, policy_version 1164503 (0.0009) [2023-12-26 23:50:46,339][105620] Updated weights for policy 1, policy_version 1164513 (0.0010) [2023-12-26 23:50:46,733][105692] Updated weights for policy 0, policy_version 1163178 (0.0006) [2023-12-26 23:50:46,787][105692] Updated weights for policy 0, policy_version 1163188 (0.0009) [2023-12-26 23:50:46,849][105692] Updated weights for policy 0, policy_version 1163198 (0.0011) [2023-12-26 23:50:46,902][105692] Updated weights for policy 0, policy_version 1163208 (0.0010) [2023-12-26 23:50:47,168][105620] Updated weights for policy 1, policy_version 1164523 (0.0009) [2023-12-26 23:50:47,240][105620] Updated weights for policy 1, policy_version 1164533 (0.0009) [2023-12-26 23:50:47,302][105620] Updated weights for policy 1, policy_version 1164543 (0.0009) [2023-12-26 23:50:47,654][105692] Updated weights for policy 0, policy_version 1163218 (0.0009) [2023-12-26 23:50:47,707][105692] Updated weights for policy 0, policy_version 1163228 (0.0008) [2023-12-26 23:50:47,764][105692] Updated weights for policy 0, policy_version 1163238 (0.0008) [2023-12-26 23:50:48,082][105620] Updated weights for policy 1, policy_version 1164553 (0.0008) [2023-12-26 23:50:48,136][105620] Updated weights for policy 1, policy_version 1164563 (0.0005) [2023-12-26 23:50:48,190][105620] Updated weights for policy 1, policy_version 1164573 (0.0005) [2023-12-26 23:50:48,238][105620] Updated weights for policy 1, policy_version 1164583 (0.0005) [2023-12-26 23:50:48,340][105692] Updated weights for policy 0, policy_version 1163248 (0.0006) [2023-12-26 23:50:48,410][105692] Updated weights for policy 0, policy_version 1163258 (0.0008) [2023-12-26 23:50:48,472][105692] Updated weights for policy 0, policy_version 1163268 (0.0009) [2023-12-26 23:50:48,982][105620] Updated weights for policy 1, policy_version 1164593 (0.0009) [2023-12-26 23:50:49,010][105692] Updated weights for policy 0, policy_version 1163278 (0.0006) [2023-12-26 23:50:49,035][105620] Updated weights for policy 1, policy_version 1164603 (0.0008) [2023-12-26 23:50:49,061][105692] Updated weights for policy 0, policy_version 1163288 (0.0006) [2023-12-26 23:50:49,080][105620] Updated weights for policy 1, policy_version 1164613 (0.0006) [2023-12-26 23:50:49,108][105692] Updated weights for policy 0, policy_version 1163298 (0.0007) [2023-12-26 23:50:49,777][105620] Updated weights for policy 1, policy_version 1164623 (0.0008) [2023-12-26 23:50:49,837][105620] Updated weights for policy 1, policy_version 1164633 (0.0009) [2023-12-26 23:50:49,895][105620] Updated weights for policy 1, policy_version 1164643 (0.0009) [2023-12-26 23:50:49,962][105692] Updated weights for policy 0, policy_version 1163308 (0.0008) [2023-12-26 23:50:50,018][105692] Updated weights for policy 0, policy_version 1163318 (0.0011) [2023-12-26 23:50:50,081][105692] Updated weights for policy 0, policy_version 1163328 (0.0011) [2023-12-26 23:50:50,559][105620] Updated weights for policy 1, policy_version 1164653 (0.0008) [2023-12-26 23:50:50,622][105620] Updated weights for policy 1, policy_version 1164663 (0.0007) [2023-12-26 23:50:50,681][105620] Updated weights for policy 1, policy_version 1164673 (0.0008) [2023-12-26 23:50:50,808][105692] Updated weights for policy 0, policy_version 1163338 (0.0010) [2023-12-26 23:50:50,874][105692] Updated weights for policy 0, policy_version 1163348 (0.0008) [2023-12-26 23:50:50,930][105692] Updated weights for policy 0, policy_version 1163358 (0.0009) [2023-12-26 23:50:50,990][105692] Updated weights for policy 0, policy_version 1163368 (0.0010) [2023-12-26 23:50:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 596066304. Throughput: 0: 9788.7, 1: 9751.0. Samples: 596052204. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:50:51,062][104569] Avg episode reward: [(0, '9263.374'), (1, '9167.571')] [2023-12-26 23:50:51,463][105620] Updated weights for policy 1, policy_version 1164683 (0.0008) [2023-12-26 23:50:51,510][105620] Updated weights for policy 1, policy_version 1164693 (0.0009) [2023-12-26 23:50:51,566][105620] Updated weights for policy 1, policy_version 1164703 (0.0009) [2023-12-26 23:50:51,724][105692] Updated weights for policy 0, policy_version 1163378 (0.0008) [2023-12-26 23:50:51,774][105692] Updated weights for policy 0, policy_version 1163388 (0.0006) [2023-12-26 23:50:51,823][105692] Updated weights for policy 0, policy_version 1163398 (0.0007) [2023-12-26 23:50:52,344][105620] Updated weights for policy 1, policy_version 1164713 (0.0009) [2023-12-26 23:50:52,408][105620] Updated weights for policy 1, policy_version 1164723 (0.0006) [2023-12-26 23:50:52,469][105620] Updated weights for policy 1, policy_version 1164733 (0.0005) [2023-12-26 23:50:52,520][105620] Updated weights for policy 1, policy_version 1164743 (0.0005) [2023-12-26 23:50:52,651][105692] Updated weights for policy 0, policy_version 1163408 (0.0010) [2023-12-26 23:50:52,709][105692] Updated weights for policy 0, policy_version 1163418 (0.0009) [2023-12-26 23:50:52,763][105692] Updated weights for policy 0, policy_version 1163428 (0.0008) [2023-12-26 23:50:53,181][105620] Updated weights for policy 1, policy_version 1164753 (0.0009) [2023-12-26 23:50:53,234][105620] Updated weights for policy 1, policy_version 1164763 (0.0009) [2023-12-26 23:50:53,285][105620] Updated weights for policy 1, policy_version 1164773 (0.0009) [2023-12-26 23:50:53,514][105692] Updated weights for policy 0, policy_version 1163438 (0.0007) [2023-12-26 23:50:53,564][105692] Updated weights for policy 0, policy_version 1163448 (0.0007) [2023-12-26 23:50:53,612][105692] Updated weights for policy 0, policy_version 1163458 (0.0005) [2023-12-26 23:50:54,085][105620] Updated weights for policy 1, policy_version 1164783 (0.0009) [2023-12-26 23:50:54,150][105620] Updated weights for policy 1, policy_version 1164793 (0.0008) [2023-12-26 23:50:54,202][105620] Updated weights for policy 1, policy_version 1164803 (0.0008) [2023-12-26 23:50:54,315][105692] Updated weights for policy 0, policy_version 1163468 (0.0007) [2023-12-26 23:50:54,386][105692] Updated weights for policy 0, policy_version 1163478 (0.0008) [2023-12-26 23:50:54,451][105692] Updated weights for policy 0, policy_version 1163488 (0.0010) [2023-12-26 23:50:54,928][105620] Updated weights for policy 1, policy_version 1164813 (0.0008) [2023-12-26 23:50:54,984][105620] Updated weights for policy 1, policy_version 1164823 (0.0009) [2023-12-26 23:50:55,038][105620] Updated weights for policy 1, policy_version 1164833 (0.0006) [2023-12-26 23:50:55,139][105692] Updated weights for policy 0, policy_version 1163498 (0.0009) [2023-12-26 23:50:55,189][105692] Updated weights for policy 0, policy_version 1163508 (0.0008) [2023-12-26 23:50:55,233][105692] Updated weights for policy 0, policy_version 1163518 (0.0008) [2023-12-26 23:50:55,290][105692] Updated weights for policy 0, policy_version 1163528 (0.0008) [2023-12-26 23:50:55,746][105620] Updated weights for policy 1, policy_version 1164843 (0.0007) [2023-12-26 23:50:55,797][105620] Updated weights for policy 1, policy_version 1164853 (0.0009) [2023-12-26 23:50:55,848][105620] Updated weights for policy 1, policy_version 1164863 (0.0009) [2023-12-26 23:50:55,977][105692] Updated weights for policy 0, policy_version 1163538 (0.0010) [2023-12-26 23:50:56,030][105692] Updated weights for policy 0, policy_version 1163548 (0.0008) [2023-12-26 23:50:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 596156416. Throughput: 0: 9833.9, 1: 9665.5. Samples: 596166600. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:50:56,062][104569] Avg episode reward: [(0, '9263.512'), (1, '8992.939')] [2023-12-26 23:50:56,085][105692] Updated weights for policy 0, policy_version 1163558 (0.0009) [2023-12-26 23:50:56,468][105620] Updated weights for policy 1, policy_version 1164873 (0.0005) [2023-12-26 23:50:56,540][105620] Updated weights for policy 1, policy_version 1164883 (0.0005) [2023-12-26 23:50:56,609][105620] Updated weights for policy 1, policy_version 1164893 (0.0006) [2023-12-26 23:50:56,670][105620] Updated weights for policy 1, policy_version 1164903 (0.0009) [2023-12-26 23:50:56,915][105692] Updated weights for policy 0, policy_version 1163568 (0.0009) [2023-12-26 23:50:56,962][105692] Updated weights for policy 0, policy_version 1163578 (0.0010) [2023-12-26 23:50:57,017][105692] Updated weights for policy 0, policy_version 1163588 (0.0010) [2023-12-26 23:50:57,326][105620] Updated weights for policy 1, policy_version 1164913 (0.0008) [2023-12-26 23:50:57,388][105620] Updated weights for policy 1, policy_version 1164923 (0.0008) [2023-12-26 23:50:57,434][105620] Updated weights for policy 1, policy_version 1164933 (0.0008) [2023-12-26 23:50:57,741][105692] Updated weights for policy 0, policy_version 1163598 (0.0009) [2023-12-26 23:50:57,786][105692] Updated weights for policy 0, policy_version 1163608 (0.0008) [2023-12-26 23:50:57,831][105692] Updated weights for policy 0, policy_version 1163618 (0.0008) [2023-12-26 23:50:58,100][105620] Updated weights for policy 1, policy_version 1164943 (0.0006) [2023-12-26 23:50:58,159][105620] Updated weights for policy 1, policy_version 1164953 (0.0007) [2023-12-26 23:50:58,219][105620] Updated weights for policy 1, policy_version 1164963 (0.0007) [2023-12-26 23:50:58,632][105692] Updated weights for policy 0, policy_version 1163628 (0.0009) [2023-12-26 23:50:58,698][105692] Updated weights for policy 0, policy_version 1163638 (0.0007) [2023-12-26 23:50:58,768][105692] Updated weights for policy 0, policy_version 1163648 (0.0007) [2023-12-26 23:50:59,025][105620] Updated weights for policy 1, policy_version 1164973 (0.0009) [2023-12-26 23:50:59,089][105620] Updated weights for policy 1, policy_version 1164983 (0.0011) [2023-12-26 23:50:59,152][105620] Updated weights for policy 1, policy_version 1164993 (0.0010) [2023-12-26 23:50:59,515][105692] Updated weights for policy 0, policy_version 1163658 (0.0008) [2023-12-26 23:50:59,583][105692] Updated weights for policy 0, policy_version 1163668 (0.0009) [2023-12-26 23:50:59,629][105692] Updated weights for policy 0, policy_version 1163678 (0.0008) [2023-12-26 23:50:59,676][105692] Updated weights for policy 0, policy_version 1163688 (0.0009) [2023-12-26 23:50:59,817][105620] Updated weights for policy 1, policy_version 1165003 (0.0008) [2023-12-26 23:50:59,876][105620] Updated weights for policy 1, policy_version 1165013 (0.0006) [2023-12-26 23:50:59,936][105620] Updated weights for policy 1, policy_version 1165023 (0.0007) [2023-12-26 23:51:00,484][105692] Updated weights for policy 0, policy_version 1163698 (0.0009) [2023-12-26 23:51:00,531][105692] Updated weights for policy 0, policy_version 1163708 (0.0009) [2023-12-26 23:51:00,585][105692] Updated weights for policy 0, policy_version 1163718 (0.0009) [2023-12-26 23:51:00,599][105620] Updated weights for policy 1, policy_version 1165033 (0.0007) [2023-12-26 23:51:00,646][105620] Updated weights for policy 1, policy_version 1165043 (0.0009) [2023-12-26 23:51:00,704][105620] Updated weights for policy 1, policy_version 1165053 (0.0010) [2023-12-26 23:51:00,774][105620] Updated weights for policy 1, policy_version 1165063 (0.0005) [2023-12-26 23:51:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 596254720. Throughput: 0: 9804.4, 1: 9705.8. Samples: 596225440. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:51:01,063][104569] Avg episode reward: [(0, '9264.557'), (1, '8906.368')] [2023-12-26 23:51:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001163720_297959424.pth... [2023-12-26 23:51:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001165064_298295296.pth... [2023-12-26 23:51:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001162600_297672704.pth [2023-12-26 23:51:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001163912_298000384.pth [2023-12-26 23:51:01,329][105692] Updated weights for policy 0, policy_version 1163728 (0.0008) [2023-12-26 23:51:01,401][105692] Updated weights for policy 0, policy_version 1163738 (0.0008) [2023-12-26 23:51:01,456][105692] Updated weights for policy 0, policy_version 1163748 (0.0008) [2023-12-26 23:51:01,495][105620] Updated weights for policy 1, policy_version 1165073 (0.0007) [2023-12-26 23:51:01,554][105620] Updated weights for policy 1, policy_version 1165083 (0.0008) [2023-12-26 23:51:01,611][105620] Updated weights for policy 1, policy_version 1165093 (0.0007) [2023-12-26 23:51:02,236][105692] Updated weights for policy 0, policy_version 1163758 (0.0008) [2023-12-26 23:51:02,291][105692] Updated weights for policy 0, policy_version 1163768 (0.0007) [2023-12-26 23:51:02,296][105620] Updated weights for policy 1, policy_version 1165103 (0.0008) [2023-12-26 23:51:02,350][105692] Updated weights for policy 0, policy_version 1163778 (0.0006) [2023-12-26 23:51:02,362][105620] Updated weights for policy 1, policy_version 1165113 (0.0008) [2023-12-26 23:51:02,414][105620] Updated weights for policy 1, policy_version 1165123 (0.0009) [2023-12-26 23:51:03,076][105620] Updated weights for policy 1, policy_version 1165133 (0.0009) [2023-12-26 23:51:03,125][105620] Updated weights for policy 1, policy_version 1165143 (0.0008) [2023-12-26 23:51:03,146][105692] Updated weights for policy 0, policy_version 1163788 (0.0009) [2023-12-26 23:51:03,184][105620] Updated weights for policy 1, policy_version 1165153 (0.0006) [2023-12-26 23:51:03,201][105692] Updated weights for policy 0, policy_version 1163798 (0.0010) [2023-12-26 23:51:03,253][105692] Updated weights for policy 0, policy_version 1163808 (0.0010) [2023-12-26 23:51:03,932][105620] Updated weights for policy 1, policy_version 1165163 (0.0007) [2023-12-26 23:51:03,999][105620] Updated weights for policy 1, policy_version 1165173 (0.0008) [2023-12-26 23:51:04,005][105692] Updated weights for policy 0, policy_version 1163818 (0.0010) [2023-12-26 23:51:04,060][105620] Updated weights for policy 1, policy_version 1165183 (0.0006) [2023-12-26 23:51:04,062][105692] Updated weights for policy 0, policy_version 1163828 (0.0011) [2023-12-26 23:51:04,115][105692] Updated weights for policy 0, policy_version 1163838 (0.0010) [2023-12-26 23:51:04,179][105692] Updated weights for policy 0, policy_version 1163848 (0.0011) [2023-12-26 23:51:04,821][105620] Updated weights for policy 1, policy_version 1165193 (0.0007) [2023-12-26 23:51:04,883][105620] Updated weights for policy 1, policy_version 1165203 (0.0008) [2023-12-26 23:51:04,932][105620] Updated weights for policy 1, policy_version 1165213 (0.0007) [2023-12-26 23:51:04,937][105692] Updated weights for policy 0, policy_version 1163858 (0.0010) [2023-12-26 23:51:04,990][105620] Updated weights for policy 1, policy_version 1165223 (0.0006) [2023-12-26 23:51:04,999][105692] Updated weights for policy 0, policy_version 1163868 (0.0010) [2023-12-26 23:51:05,061][105692] Updated weights for policy 0, policy_version 1163878 (0.0010) [2023-12-26 23:51:05,746][105692] Updated weights for policy 0, policy_version 1163888 (0.0008) [2023-12-26 23:51:05,763][105620] Updated weights for policy 1, policy_version 1165233 (0.0006) [2023-12-26 23:51:05,804][105692] Updated weights for policy 0, policy_version 1163898 (0.0008) [2023-12-26 23:51:05,826][105620] Updated weights for policy 1, policy_version 1165243 (0.0008) [2023-12-26 23:51:05,867][105692] Updated weights for policy 0, policy_version 1163908 (0.0007) [2023-12-26 23:51:05,882][105620] Updated weights for policy 1, policy_version 1165253 (0.0006) [2023-12-26 23:51:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 596353024. Throughput: 0: 9588.1, 1: 9752.9. Samples: 596338028. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:51:06,063][104569] Avg episode reward: [(0, '9172.863'), (1, '8818.577')] [2023-12-26 23:51:06,552][105692] Updated weights for policy 0, policy_version 1163918 (0.0009) [2023-12-26 23:51:06,611][105620] Updated weights for policy 1, policy_version 1165263 (0.0006) [2023-12-26 23:51:06,615][105692] Updated weights for policy 0, policy_version 1163928 (0.0010) [2023-12-26 23:51:06,674][105620] Updated weights for policy 1, policy_version 1165273 (0.0008) [2023-12-26 23:51:06,675][105692] Updated weights for policy 0, policy_version 1163938 (0.0009) [2023-12-26 23:51:06,739][105620] Updated weights for policy 1, policy_version 1165283 (0.0009) [2023-12-26 23:51:07,331][105692] Updated weights for policy 0, policy_version 1163948 (0.0007) [2023-12-26 23:51:07,383][105692] Updated weights for policy 0, policy_version 1163958 (0.0010) [2023-12-26 23:51:07,428][105692] Updated weights for policy 0, policy_version 1163968 (0.0010) [2023-12-26 23:51:07,505][105620] Updated weights for policy 1, policy_version 1165293 (0.0011) [2023-12-26 23:51:07,553][105620] Updated weights for policy 1, policy_version 1165303 (0.0010) [2023-12-26 23:51:07,616][105620] Updated weights for policy 1, policy_version 1165313 (0.0008) [2023-12-26 23:51:08,136][105692] Updated weights for policy 0, policy_version 1163978 (0.0010) [2023-12-26 23:51:08,185][105692] Updated weights for policy 0, policy_version 1163988 (0.0010) [2023-12-26 23:51:08,227][105620] Updated weights for policy 1, policy_version 1165323 (0.0007) [2023-12-26 23:51:08,234][105692] Updated weights for policy 0, policy_version 1163998 (0.0010) [2023-12-26 23:51:08,284][105620] Updated weights for policy 1, policy_version 1165333 (0.0006) [2023-12-26 23:51:08,286][105692] Updated weights for policy 0, policy_version 1164008 (0.0011) [2023-12-26 23:51:08,346][105620] Updated weights for policy 1, policy_version 1165343 (0.0007) [2023-12-26 23:51:09,060][105620] Updated weights for policy 1, policy_version 1165353 (0.0008) [2023-12-26 23:51:09,066][105692] Updated weights for policy 0, policy_version 1164018 (0.0010) [2023-12-26 23:51:09,118][105620] Updated weights for policy 1, policy_version 1165363 (0.0005) [2023-12-26 23:51:09,127][105692] Updated weights for policy 0, policy_version 1164028 (0.0010) [2023-12-26 23:51:09,173][105620] Updated weights for policy 1, policy_version 1165373 (0.0007) [2023-12-26 23:51:09,178][105692] Updated weights for policy 0, policy_version 1164038 (0.0010) [2023-12-26 23:51:09,233][105620] Updated weights for policy 1, policy_version 1165383 (0.0008) [2023-12-26 23:51:10,010][105692] Updated weights for policy 0, policy_version 1164048 (0.0009) [2023-12-26 23:51:10,069][105692] Updated weights for policy 0, policy_version 1164058 (0.0009) [2023-12-26 23:51:10,076][105620] Updated weights for policy 1, policy_version 1165393 (0.0009) [2023-12-26 23:51:10,129][105692] Updated weights for policy 0, policy_version 1164068 (0.0008) [2023-12-26 23:51:10,140][105620] Updated weights for policy 1, policy_version 1165403 (0.0007) [2023-12-26 23:51:10,204][105620] Updated weights for policy 1, policy_version 1165413 (0.0009) [2023-12-26 23:51:10,878][105692] Updated weights for policy 0, policy_version 1164078 (0.0010) [2023-12-26 23:51:10,936][105692] Updated weights for policy 0, policy_version 1164088 (0.0005) [2023-12-26 23:51:10,970][105620] Updated weights for policy 1, policy_version 1165423 (0.0008) [2023-12-26 23:51:10,990][105692] Updated weights for policy 0, policy_version 1164098 (0.0008) [2023-12-26 23:51:11,026][105620] Updated weights for policy 1, policy_version 1165433 (0.0009) [2023-12-26 23:51:11,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 596443136. Throughput: 0: 9581.4, 1: 9701.9. Samples: 596452668. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:51:11,063][104569] Avg episode reward: [(0, '9082.293'), (1, '8635.396')] [2023-12-26 23:51:11,088][105620] Updated weights for policy 1, policy_version 1165443 (0.0009) [2023-12-26 23:51:11,850][105692] Updated weights for policy 0, policy_version 1164108 (0.0010) [2023-12-26 23:51:11,872][105620] Updated weights for policy 1, policy_version 1165453 (0.0008) [2023-12-26 23:51:11,909][105692] Updated weights for policy 0, policy_version 1164118 (0.0009) [2023-12-26 23:51:11,933][105620] Updated weights for policy 1, policy_version 1165463 (0.0008) [2023-12-26 23:51:11,973][105692] Updated weights for policy 0, policy_version 1164128 (0.0008) [2023-12-26 23:51:11,987][105620] Updated weights for policy 1, policy_version 1165473 (0.0007) [2023-12-26 23:51:12,766][105692] Updated weights for policy 0, policy_version 1164138 (0.0008) [2023-12-26 23:51:12,771][105620] Updated weights for policy 1, policy_version 1165483 (0.0007) [2023-12-26 23:51:12,825][105692] Updated weights for policy 0, policy_version 1164148 (0.0011) [2023-12-26 23:51:12,826][105620] Updated weights for policy 1, policy_version 1165493 (0.0009) [2023-12-26 23:51:12,874][105692] Updated weights for policy 0, policy_version 1164158 (0.0011) [2023-12-26 23:51:12,889][105620] Updated weights for policy 1, policy_version 1165503 (0.0010) [2023-12-26 23:51:12,925][105692] Updated weights for policy 0, policy_version 1164168 (0.0008) [2023-12-26 23:51:13,488][105692] Updated weights for policy 0, policy_version 1164178 (0.0005) [2023-12-26 23:51:13,527][105620] Updated weights for policy 1, policy_version 1165513 (0.0010) [2023-12-26 23:51:13,542][105692] Updated weights for policy 0, policy_version 1164188 (0.0005) [2023-12-26 23:51:13,588][105620] Updated weights for policy 1, policy_version 1165523 (0.0008) [2023-12-26 23:51:13,598][105692] Updated weights for policy 0, policy_version 1164198 (0.0005) [2023-12-26 23:51:13,646][105620] Updated weights for policy 1, policy_version 1165533 (0.0010) [2023-12-26 23:51:13,701][105620] Updated weights for policy 1, policy_version 1165543 (0.0010) [2023-12-26 23:51:14,123][105692] Updated weights for policy 0, policy_version 1164208 (0.0006) [2023-12-26 23:51:14,178][105692] Updated weights for policy 0, policy_version 1164218 (0.0007) [2023-12-26 23:51:14,222][105692] Updated weights for policy 0, policy_version 1164228 (0.0010) [2023-12-26 23:51:14,426][105620] Updated weights for policy 1, policy_version 1165553 (0.0011) [2023-12-26 23:51:14,481][105620] Updated weights for policy 1, policy_version 1165563 (0.0010) [2023-12-26 23:51:14,539][105620] Updated weights for policy 1, policy_version 1165573 (0.0010) [2023-12-26 23:51:14,921][105692] Updated weights for policy 0, policy_version 1164238 (0.0009) [2023-12-26 23:51:14,992][105692] Updated weights for policy 0, policy_version 1164248 (0.0008) [2023-12-26 23:51:15,053][105692] Updated weights for policy 0, policy_version 1164258 (0.0011) [2023-12-26 23:51:15,215][105620] Updated weights for policy 1, policy_version 1165583 (0.0011) [2023-12-26 23:51:15,281][105620] Updated weights for policy 1, policy_version 1165593 (0.0010) [2023-12-26 23:51:15,345][105620] Updated weights for policy 1, policy_version 1165603 (0.0008) [2023-12-26 23:51:15,704][105692] Updated weights for policy 0, policy_version 1164268 (0.0011) [2023-12-26 23:51:15,763][105692] Updated weights for policy 0, policy_version 1164278 (0.0011) [2023-12-26 23:51:15,849][105692] Updated weights for policy 0, policy_version 1164288 (0.0011) [2023-12-26 23:51:16,051][105620] Updated weights for policy 1, policy_version 1165613 (0.0011) [2023-12-26 23:51:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 596541440. Throughput: 0: 9516.0, 1: 9698.7. Samples: 596509792. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:51:16,062][104569] Avg episode reward: [(0, '9082.271'), (1, '8726.354')] [2023-12-26 23:51:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001164296_298106880.pth... [2023-12-26 23:51:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001163176_297820160.pth [2023-12-26 23:51:16,113][105620] Updated weights for policy 1, policy_version 1165623 (0.0010) [2023-12-26 23:51:16,167][105620] Updated weights for policy 1, policy_version 1165633 (0.0007) [2023-12-26 23:51:16,201][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001165640_298442752.pth... [2023-12-26 23:51:16,204][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001164488_298147840.pth [2023-12-26 23:51:16,534][105692] Updated weights for policy 0, policy_version 1164298 (0.0010) [2023-12-26 23:51:16,583][105692] Updated weights for policy 0, policy_version 1164308 (0.0008) [2023-12-26 23:51:16,628][105692] Updated weights for policy 0, policy_version 1164318 (0.0008) [2023-12-26 23:51:16,680][105692] Updated weights for policy 0, policy_version 1164328 (0.0008) [2023-12-26 23:51:16,842][105620] Updated weights for policy 1, policy_version 1165643 (0.0009) [2023-12-26 23:51:16,888][105620] Updated weights for policy 1, policy_version 1165653 (0.0005) [2023-12-26 23:51:16,943][105620] Updated weights for policy 1, policy_version 1165663 (0.0005) [2023-12-26 23:51:17,485][105692] Updated weights for policy 0, policy_version 1164338 (0.0006) [2023-12-26 23:51:17,514][105620] Updated weights for policy 1, policy_version 1165673 (0.0007) [2023-12-26 23:51:17,544][105692] Updated weights for policy 0, policy_version 1164348 (0.0006) [2023-12-26 23:51:17,570][105620] Updated weights for policy 1, policy_version 1165683 (0.0011) [2023-12-26 23:51:17,597][105692] Updated weights for policy 0, policy_version 1164358 (0.0006) [2023-12-26 23:51:17,628][105620] Updated weights for policy 1, policy_version 1165693 (0.0009) [2023-12-26 23:51:17,686][105620] Updated weights for policy 1, policy_version 1165703 (0.0006) [2023-12-26 23:51:18,333][105692] Updated weights for policy 0, policy_version 1164368 (0.0007) [2023-12-26 23:51:18,390][105692] Updated weights for policy 0, policy_version 1164378 (0.0008) [2023-12-26 23:51:18,423][105620] Updated weights for policy 1, policy_version 1165713 (0.0011) [2023-12-26 23:51:18,447][105692] Updated weights for policy 0, policy_version 1164388 (0.0009) [2023-12-26 23:51:18,479][105620] Updated weights for policy 1, policy_version 1165723 (0.0009) [2023-12-26 23:51:18,528][105620] Updated weights for policy 1, policy_version 1165733 (0.0010) [2023-12-26 23:51:19,233][105692] Updated weights for policy 0, policy_version 1164398 (0.0008) [2023-12-26 23:51:19,294][105692] Updated weights for policy 0, policy_version 1164408 (0.0007) [2023-12-26 23:51:19,296][105620] Updated weights for policy 1, policy_version 1165743 (0.0010) [2023-12-26 23:51:19,357][105692] Updated weights for policy 0, policy_version 1164418 (0.0007) [2023-12-26 23:51:19,360][105620] Updated weights for policy 1, policy_version 1165753 (0.0011) [2023-12-26 23:51:19,416][105620] Updated weights for policy 1, policy_version 1165763 (0.0010) [2023-12-26 23:51:20,112][105692] Updated weights for policy 0, policy_version 1164428 (0.0007) [2023-12-26 23:51:20,170][105692] Updated weights for policy 0, policy_version 1164438 (0.0008) [2023-12-26 23:51:20,184][105620] Updated weights for policy 1, policy_version 1165773 (0.0009) [2023-12-26 23:51:20,231][105692] Updated weights for policy 0, policy_version 1164448 (0.0008) [2023-12-26 23:51:20,244][105620] Updated weights for policy 1, policy_version 1165783 (0.0006) [2023-12-26 23:51:20,302][105620] Updated weights for policy 1, policy_version 1165793 (0.0010) [2023-12-26 23:51:20,967][105620] Updated weights for policy 1, policy_version 1165803 (0.0009) [2023-12-26 23:51:21,038][105620] Updated weights for policy 1, policy_version 1165813 (0.0007) [2023-12-26 23:51:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 596631552. Throughput: 0: 9511.1, 1: 9767.2. Samples: 596628404. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:51:21,063][104569] Avg episode reward: [(0, '9171.760'), (1, '8721.808')] [2023-12-26 23:51:21,078][105692] Updated weights for policy 0, policy_version 1164458 (0.0009) [2023-12-26 23:51:21,106][105620] Updated weights for policy 1, policy_version 1165823 (0.0009) [2023-12-26 23:51:21,147][105692] Updated weights for policy 0, policy_version 1164468 (0.0008) [2023-12-26 23:51:21,207][105692] Updated weights for policy 0, policy_version 1164478 (0.0009) [2023-12-26 23:51:21,273][105692] Updated weights for policy 0, policy_version 1164488 (0.0008) [2023-12-26 23:51:21,804][105620] Updated weights for policy 1, policy_version 1165833 (0.0009) [2023-12-26 23:51:21,863][105620] Updated weights for policy 1, policy_version 1165843 (0.0009) [2023-12-26 23:51:21,921][105620] Updated weights for policy 1, policy_version 1165853 (0.0008) [2023-12-26 23:51:21,986][105620] Updated weights for policy 1, policy_version 1165863 (0.0008) [2023-12-26 23:51:22,010][105692] Updated weights for policy 0, policy_version 1164498 (0.0008) [2023-12-26 23:51:22,072][105692] Updated weights for policy 0, policy_version 1164508 (0.0009) [2023-12-26 23:51:22,127][105692] Updated weights for policy 0, policy_version 1164518 (0.0009) [2023-12-26 23:51:22,756][105620] Updated weights for policy 1, policy_version 1165873 (0.0008) [2023-12-26 23:51:22,812][105620] Updated weights for policy 1, policy_version 1165883 (0.0009) [2023-12-26 23:51:22,862][105620] Updated weights for policy 1, policy_version 1165893 (0.0009) [2023-12-26 23:51:22,870][105692] Updated weights for policy 0, policy_version 1164528 (0.0006) [2023-12-26 23:51:22,934][105692] Updated weights for policy 0, policy_version 1164538 (0.0008) [2023-12-26 23:51:22,996][105692] Updated weights for policy 0, policy_version 1164548 (0.0009) [2023-12-26 23:51:23,621][105692] Updated weights for policy 0, policy_version 1164558 (0.0009) [2023-12-26 23:51:23,663][105620] Updated weights for policy 1, policy_version 1165903 (0.0008) [2023-12-26 23:51:23,677][105692] Updated weights for policy 0, policy_version 1164568 (0.0008) [2023-12-26 23:51:23,721][105620] Updated weights for policy 1, policy_version 1165913 (0.0008) [2023-12-26 23:51:23,723][105692] Updated weights for policy 0, policy_version 1164578 (0.0008) [2023-12-26 23:51:23,777][105620] Updated weights for policy 1, policy_version 1165923 (0.0009) [2023-12-26 23:51:24,483][105692] Updated weights for policy 0, policy_version 1164588 (0.0008) [2023-12-26 23:51:24,529][105620] Updated weights for policy 1, policy_version 1165933 (0.0008) [2023-12-26 23:51:24,538][105692] Updated weights for policy 0, policy_version 1164598 (0.0007) [2023-12-26 23:51:24,574][105620] Updated weights for policy 1, policy_version 1165943 (0.0006) [2023-12-26 23:51:24,593][105692] Updated weights for policy 0, policy_version 1164608 (0.0009) [2023-12-26 23:51:24,656][105620] Updated weights for policy 1, policy_version 1165953 (0.0007) [2023-12-26 23:51:25,231][105692] Updated weights for policy 0, policy_version 1164618 (0.0009) [2023-12-26 23:51:25,286][105692] Updated weights for policy 0, policy_version 1164628 (0.0009) [2023-12-26 23:51:25,336][105692] Updated weights for policy 0, policy_version 1164638 (0.0008) [2023-12-26 23:51:25,382][105692] Updated weights for policy 0, policy_version 1164648 (0.0009) [2023-12-26 23:51:25,431][105620] Updated weights for policy 1, policy_version 1165963 (0.0009) [2023-12-26 23:51:25,484][105620] Updated weights for policy 1, policy_version 1165973 (0.0009) [2023-12-26 23:51:25,538][105620] Updated weights for policy 1, policy_version 1165983 (0.0009) [2023-12-26 23:51:26,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19251.1, 300 sec: 19466.4). Total num frames: 596729856. Throughput: 0: 9530.6, 1: 9709.0. Samples: 596742156. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:51:26,063][104569] Avg episode reward: [(0, '9080.571'), (1, '8898.443')] [2023-12-26 23:51:26,205][105692] Updated weights for policy 0, policy_version 1164658 (0.0008) [2023-12-26 23:51:26,209][105620] Updated weights for policy 1, policy_version 1165993 (0.0009) [2023-12-26 23:51:26,264][105692] Updated weights for policy 0, policy_version 1164668 (0.0006) [2023-12-26 23:51:26,266][105620] Updated weights for policy 1, policy_version 1166003 (0.0007) [2023-12-26 23:51:26,316][105692] Updated weights for policy 0, policy_version 1164678 (0.0006) [2023-12-26 23:51:26,323][105620] Updated weights for policy 1, policy_version 1166013 (0.0007) [2023-12-26 23:51:26,376][105620] Updated weights for policy 1, policy_version 1166023 (0.0007) [2023-12-26 23:51:27,011][105692] Updated weights for policy 0, policy_version 1164688 (0.0006) [2023-12-26 23:51:27,074][105692] Updated weights for policy 0, policy_version 1164698 (0.0005) [2023-12-26 23:51:27,122][105692] Updated weights for policy 0, policy_version 1164708 (0.0005) [2023-12-26 23:51:27,131][105620] Updated weights for policy 1, policy_version 1166033 (0.0008) [2023-12-26 23:51:27,187][105620] Updated weights for policy 1, policy_version 1166043 (0.0009) [2023-12-26 23:51:27,244][105620] Updated weights for policy 1, policy_version 1166054 (0.0010) [2023-12-26 23:51:27,745][105692] Updated weights for policy 0, policy_version 1164718 (0.0005) [2023-12-26 23:51:27,798][105692] Updated weights for policy 0, policy_version 1164728 (0.0005) [2023-12-26 23:51:27,847][105692] Updated weights for policy 0, policy_version 1164738 (0.0006) [2023-12-26 23:51:28,037][105620] Updated weights for policy 1, policy_version 1166064 (0.0007) [2023-12-26 23:51:28,084][105620] Updated weights for policy 1, policy_version 1166074 (0.0009) [2023-12-26 23:51:28,132][105620] Updated weights for policy 1, policy_version 1166084 (0.0010) [2023-12-26 23:51:28,529][105692] Updated weights for policy 0, policy_version 1164748 (0.0007) [2023-12-26 23:51:28,588][105692] Updated weights for policy 0, policy_version 1164758 (0.0010) [2023-12-26 23:51:28,638][105692] Updated weights for policy 0, policy_version 1164768 (0.0010) [2023-12-26 23:51:28,844][105620] Updated weights for policy 1, policy_version 1166094 (0.0008) [2023-12-26 23:51:28,900][105620] Updated weights for policy 1, policy_version 1166104 (0.0008) [2023-12-26 23:51:28,957][105620] Updated weights for policy 1, policy_version 1166114 (0.0008) [2023-12-26 23:51:29,391][105692] Updated weights for policy 0, policy_version 1164778 (0.0010) [2023-12-26 23:51:29,446][105692] Updated weights for policy 0, policy_version 1164788 (0.0006) [2023-12-26 23:51:29,506][105692] Updated weights for policy 0, policy_version 1164798 (0.0005) [2023-12-26 23:51:29,570][105692] Updated weights for policy 0, policy_version 1164808 (0.0006) [2023-12-26 23:51:29,725][105620] Updated weights for policy 1, policy_version 1166124 (0.0009) [2023-12-26 23:51:29,773][105620] Updated weights for policy 1, policy_version 1166134 (0.0010) [2023-12-26 23:51:29,821][105620] Updated weights for policy 1, policy_version 1166144 (0.0010) [2023-12-26 23:51:30,250][105692] Updated weights for policy 0, policy_version 1164818 (0.0011) [2023-12-26 23:51:30,309][105692] Updated weights for policy 0, policy_version 1164828 (0.0011) [2023-12-26 23:51:30,360][105692] Updated weights for policy 0, policy_version 1164838 (0.0010) [2023-12-26 23:51:30,505][105620] Updated weights for policy 1, policy_version 1166154 (0.0011) [2023-12-26 23:51:30,560][105620] Updated weights for policy 1, policy_version 1166164 (0.0010) [2023-12-26 23:51:30,621][105620] Updated weights for policy 1, policy_version 1166174 (0.0010) [2023-12-26 23:51:30,681][105620] Updated weights for policy 1, policy_version 1166184 (0.0010) [2023-12-26 23:51:30,956][105692] Updated weights for policy 0, policy_version 1164848 (0.0007) [2023-12-26 23:51:31,004][105692] Updated weights for policy 0, policy_version 1164858 (0.0008) [2023-12-26 23:51:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 596828160. Throughput: 0: 9630.6, 1: 9626.4. Samples: 596800588. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:51:31,062][104569] Avg episode reward: [(0, '8730.793'), (1, '8989.647')] [2023-12-26 23:51:31,066][105692] Updated weights for policy 0, policy_version 1164868 (0.0008) [2023-12-26 23:51:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001166184_298582016.pth... [2023-12-26 23:51:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001165064_298295296.pth [2023-12-26 23:51:31,087][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001164872_298254336.pth... [2023-12-26 23:51:31,092][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001163720_297959424.pth [2023-12-26 23:51:31,435][105620] Updated weights for policy 1, policy_version 1166194 (0.0009) [2023-12-26 23:51:31,490][105620] Updated weights for policy 1, policy_version 1166204 (0.0010) [2023-12-26 23:51:31,546][105620] Updated weights for policy 1, policy_version 1166214 (0.0009) [2023-12-26 23:51:31,754][105692] Updated weights for policy 0, policy_version 1164878 (0.0007) [2023-12-26 23:51:31,808][105692] Updated weights for policy 0, policy_version 1164888 (0.0005) [2023-12-26 23:51:31,866][105692] Updated weights for policy 0, policy_version 1164898 (0.0005) [2023-12-26 23:51:32,275][105620] Updated weights for policy 1, policy_version 1166224 (0.0009) [2023-12-26 23:51:32,333][105620] Updated weights for policy 1, policy_version 1166234 (0.0010) [2023-12-26 23:51:32,392][105620] Updated weights for policy 1, policy_version 1166244 (0.0010) [2023-12-26 23:51:32,441][105692] Updated weights for policy 0, policy_version 1164908 (0.0005) [2023-12-26 23:51:32,504][105692] Updated weights for policy 0, policy_version 1164918 (0.0006) [2023-12-26 23:51:32,572][105692] Updated weights for policy 0, policy_version 1164928 (0.0005) [2023-12-26 23:51:33,042][105620] Updated weights for policy 1, policy_version 1166254 (0.0007) [2023-12-26 23:51:33,092][105620] Updated weights for policy 1, policy_version 1166264 (0.0005) [2023-12-26 23:51:33,143][105620] Updated weights for policy 1, policy_version 1166274 (0.0005) [2023-12-26 23:51:33,173][105692] Updated weights for policy 0, policy_version 1164938 (0.0006) [2023-12-26 23:51:33,225][105692] Updated weights for policy 0, policy_version 1164948 (0.0009) [2023-12-26 23:51:33,278][105692] Updated weights for policy 0, policy_version 1164958 (0.0010) [2023-12-26 23:51:33,708][105620] Updated weights for policy 1, policy_version 1166284 (0.0007) [2023-12-26 23:51:33,764][105620] Updated weights for policy 1, policy_version 1166294 (0.0008) [2023-12-26 23:51:33,814][105620] Updated weights for policy 1, policy_version 1166304 (0.0009) [2023-12-26 23:51:34,025][105692] Updated weights for policy 0, policy_version 1164969 (0.0009) [2023-12-26 23:51:34,070][105692] Updated weights for policy 0, policy_version 1164979 (0.0007) [2023-12-26 23:51:34,128][105692] Updated weights for policy 0, policy_version 1164989 (0.0009) [2023-12-26 23:51:34,190][105692] Updated weights for policy 0, policy_version 1164999 (0.0007) [2023-12-26 23:51:34,623][105620] Updated weights for policy 1, policy_version 1166314 (0.0008) [2023-12-26 23:51:34,682][105620] Updated weights for policy 1, policy_version 1166324 (0.0009) [2023-12-26 23:51:34,740][105620] Updated weights for policy 1, policy_version 1166334 (0.0006) [2023-12-26 23:51:34,810][105620] Updated weights for policy 1, policy_version 1166344 (0.0007) [2023-12-26 23:51:34,869][105692] Updated weights for policy 0, policy_version 1165009 (0.0006) [2023-12-26 23:51:34,932][105692] Updated weights for policy 0, policy_version 1165019 (0.0006) [2023-12-26 23:51:34,994][105692] Updated weights for policy 0, policy_version 1165029 (0.0009) [2023-12-26 23:51:35,481][105620] Updated weights for policy 1, policy_version 1166354 (0.0005) [2023-12-26 23:51:35,537][105620] Updated weights for policy 1, policy_version 1166364 (0.0005) [2023-12-26 23:51:35,588][105620] Updated weights for policy 1, policy_version 1166374 (0.0005) [2023-12-26 23:51:35,772][105692] Updated weights for policy 0, policy_version 1165039 (0.0010) [2023-12-26 23:51:35,826][105692] Updated weights for policy 0, policy_version 1165051 (0.0010) [2023-12-26 23:51:35,884][105692] Updated weights for policy 0, policy_version 1165063 (0.0010) [2023-12-26 23:51:36,062][104569] Fps is (10 sec: 20480.7, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 596934656. Throughput: 0: 9660.3, 1: 9676.3. Samples: 596922352. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:51:36,062][104569] Avg episode reward: [(0, '8657.979'), (1, '9168.828')] [2023-12-26 23:51:36,154][105620] Updated weights for policy 1, policy_version 1166384 (0.0007) [2023-12-26 23:51:36,221][105620] Updated weights for policy 1, policy_version 1166394 (0.0005) [2023-12-26 23:51:36,292][105620] Updated weights for policy 1, policy_version 1166404 (0.0005) [2023-12-26 23:51:36,773][105692] Updated weights for policy 0, policy_version 1165073 (0.0008) [2023-12-26 23:51:36,839][105692] Updated weights for policy 0, policy_version 1165083 (0.0007) [2023-12-26 23:51:36,873][105620] Updated weights for policy 1, policy_version 1166414 (0.0007) [2023-12-26 23:51:36,896][105692] Updated weights for policy 0, policy_version 1165093 (0.0006) [2023-12-26 23:51:36,933][105620] Updated weights for policy 1, policy_version 1166424 (0.0009) [2023-12-26 23:51:36,998][105620] Updated weights for policy 1, policy_version 1166434 (0.0009) [2023-12-26 23:51:37,651][105692] Updated weights for policy 0, policy_version 1165103 (0.0007) [2023-12-26 23:51:37,652][105620] Updated weights for policy 1, policy_version 1166444 (0.0009) [2023-12-26 23:51:37,711][105692] Updated weights for policy 0, policy_version 1165113 (0.0010) [2023-12-26 23:51:37,715][105620] Updated weights for policy 1, policy_version 1166454 (0.0006) [2023-12-26 23:51:37,774][105620] Updated weights for policy 1, policy_version 1166464 (0.0008) [2023-12-26 23:51:37,777][105692] Updated weights for policy 0, policy_version 1165123 (0.0006) [2023-12-26 23:51:38,403][105620] Updated weights for policy 1, policy_version 1166474 (0.0007) [2023-12-26 23:51:38,458][105620] Updated weights for policy 1, policy_version 1166484 (0.0010) [2023-12-26 23:51:38,510][105620] Updated weights for policy 1, policy_version 1166494 (0.0010) [2023-12-26 23:51:38,527][105692] Updated weights for policy 0, policy_version 1165133 (0.0006) [2023-12-26 23:51:38,569][105620] Updated weights for policy 1, policy_version 1166504 (0.0011) [2023-12-26 23:51:38,592][105692] Updated weights for policy 0, policy_version 1165143 (0.0006) [2023-12-26 23:51:38,652][105692] Updated weights for policy 0, policy_version 1165153 (0.0007) [2023-12-26 23:51:39,332][105620] Updated weights for policy 1, policy_version 1166514 (0.0010) [2023-12-26 23:51:39,394][105692] Updated weights for policy 0, policy_version 1165163 (0.0007) [2023-12-26 23:51:39,403][105620] Updated weights for policy 1, policy_version 1166524 (0.0010) [2023-12-26 23:51:39,463][105692] Updated weights for policy 0, policy_version 1165173 (0.0007) [2023-12-26 23:51:39,469][105620] Updated weights for policy 1, policy_version 1166534 (0.0011) [2023-12-26 23:51:39,526][105692] Updated weights for policy 0, policy_version 1165183 (0.0006) [2023-12-26 23:51:40,116][105620] Updated weights for policy 1, policy_version 1166544 (0.0008) [2023-12-26 23:51:40,184][105620] Updated weights for policy 1, policy_version 1166554 (0.0008) [2023-12-26 23:51:40,244][105692] Updated weights for policy 0, policy_version 1165193 (0.0007) [2023-12-26 23:51:40,246][105620] Updated weights for policy 1, policy_version 1166564 (0.0008) [2023-12-26 23:51:40,300][105692] Updated weights for policy 0, policy_version 1165203 (0.0007) [2023-12-26 23:51:40,351][105692] Updated weights for policy 0, policy_version 1165213 (0.0009) [2023-12-26 23:51:40,420][105692] Updated weights for policy 0, policy_version 1165223 (0.0010) [2023-12-26 23:51:40,841][105620] Updated weights for policy 1, policy_version 1166574 (0.0007) [2023-12-26 23:51:40,895][105620] Updated weights for policy 1, policy_version 1166584 (0.0008) [2023-12-26 23:51:40,957][105620] Updated weights for policy 1, policy_version 1166594 (0.0009) [2023-12-26 23:51:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 597032960. Throughput: 0: 9596.5, 1: 9811.9. Samples: 597039980. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:51:41,062][104569] Avg episode reward: [(0, '8829.848'), (1, '9009.338')] [2023-12-26 23:51:41,212][105692] Updated weights for policy 0, policy_version 1165233 (0.0009) [2023-12-26 23:51:41,270][105692] Updated weights for policy 0, policy_version 1165243 (0.0009) [2023-12-26 23:51:41,332][105692] Updated weights for policy 0, policy_version 1165253 (0.0009) [2023-12-26 23:51:41,701][105620] Updated weights for policy 1, policy_version 1166604 (0.0009) [2023-12-26 23:51:41,773][105620] Updated weights for policy 1, policy_version 1166614 (0.0009) [2023-12-26 23:51:41,836][105620] Updated weights for policy 1, policy_version 1166624 (0.0006) [2023-12-26 23:51:42,133][105692] Updated weights for policy 0, policy_version 1165263 (0.0009) [2023-12-26 23:51:42,206][105692] Updated weights for policy 0, policy_version 1165273 (0.0010) [2023-12-26 23:51:42,265][105692] Updated weights for policy 0, policy_version 1165283 (0.0008) [2023-12-26 23:51:42,581][105620] Updated weights for policy 1, policy_version 1166634 (0.0008) [2023-12-26 23:51:42,638][105620] Updated weights for policy 1, policy_version 1166644 (0.0009) [2023-12-26 23:51:42,695][105620] Updated weights for policy 1, policy_version 1166654 (0.0010) [2023-12-26 23:51:42,747][105620] Updated weights for policy 1, policy_version 1166664 (0.0008) [2023-12-26 23:51:42,992][105692] Updated weights for policy 0, policy_version 1165293 (0.0009) [2023-12-26 23:51:43,048][105692] Updated weights for policy 0, policy_version 1165303 (0.0009) [2023-12-26 23:51:43,103][105692] Updated weights for policy 0, policy_version 1165313 (0.0009) [2023-12-26 23:51:43,474][105620] Updated weights for policy 1, policy_version 1166674 (0.0010) [2023-12-26 23:51:43,532][105620] Updated weights for policy 1, policy_version 1166684 (0.0009) [2023-12-26 23:51:43,597][105620] Updated weights for policy 1, policy_version 1166694 (0.0008) [2023-12-26 23:51:43,869][105692] Updated weights for policy 0, policy_version 1165323 (0.0009) [2023-12-26 23:51:43,926][105692] Updated weights for policy 0, policy_version 1165333 (0.0009) [2023-12-26 23:51:43,984][105692] Updated weights for policy 0, policy_version 1165343 (0.0008) [2023-12-26 23:51:44,342][105620] Updated weights for policy 1, policy_version 1166704 (0.0009) [2023-12-26 23:51:44,395][105620] Updated weights for policy 1, policy_version 1166714 (0.0008) [2023-12-26 23:51:44,456][105620] Updated weights for policy 1, policy_version 1166724 (0.0008) [2023-12-26 23:51:44,766][105692] Updated weights for policy 0, policy_version 1165353 (0.0009) [2023-12-26 23:51:44,831][105692] Updated weights for policy 0, policy_version 1165363 (0.0007) [2023-12-26 23:51:44,892][105692] Updated weights for policy 0, policy_version 1165373 (0.0010) [2023-12-26 23:51:44,962][105692] Updated weights for policy 0, policy_version 1165383 (0.0009) [2023-12-26 23:51:45,158][105620] Updated weights for policy 1, policy_version 1166734 (0.0010) [2023-12-26 23:51:45,213][105620] Updated weights for policy 1, policy_version 1166744 (0.0005) [2023-12-26 23:51:45,280][105620] Updated weights for policy 1, policy_version 1166754 (0.0005) [2023-12-26 23:51:45,779][105692] Updated weights for policy 0, policy_version 1165393 (0.0009) [2023-12-26 23:51:45,833][105692] Updated weights for policy 0, policy_version 1165403 (0.0009) [2023-12-26 23:51:45,883][105692] Updated weights for policy 0, policy_version 1165413 (0.0008) [2023-12-26 23:51:45,885][105620] Updated weights for policy 1, policy_version 1166764 (0.0008) [2023-12-26 23:51:45,937][105620] Updated weights for policy 1, policy_version 1166774 (0.0005) [2023-12-26 23:51:45,985][105620] Updated weights for policy 1, policy_version 1166784 (0.0005) [2023-12-26 23:51:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 597131264. Throughput: 0: 9575.7, 1: 9758.3. Samples: 597095472. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:51:46,062][104569] Avg episode reward: [(0, '8906.051'), (1, '8939.858')] [2023-12-26 23:51:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001165416_298393600.pth... [2023-12-26 23:51:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001166792_298737664.pth... [2023-12-26 23:51:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001165640_298442752.pth [2023-12-26 23:51:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001164296_298106880.pth [2023-12-26 23:51:46,518][105692] Updated weights for policy 0, policy_version 1165423 (0.0006) [2023-12-26 23:51:46,573][105692] Updated weights for policy 0, policy_version 1165433 (0.0005) [2023-12-26 23:51:46,625][105692] Updated weights for policy 0, policy_version 1165443 (0.0007) [2023-12-26 23:51:46,799][105620] Updated weights for policy 1, policy_version 1166794 (0.0007) [2023-12-26 23:51:46,854][105620] Updated weights for policy 1, policy_version 1166804 (0.0008) [2023-12-26 23:51:46,918][105620] Updated weights for policy 1, policy_version 1166814 (0.0008) [2023-12-26 23:51:46,980][105620] Updated weights for policy 1, policy_version 1166824 (0.0006) [2023-12-26 23:51:47,350][105692] Updated weights for policy 0, policy_version 1165453 (0.0009) [2023-12-26 23:51:47,413][105692] Updated weights for policy 0, policy_version 1165463 (0.0010) [2023-12-26 23:51:47,471][105692] Updated weights for policy 0, policy_version 1165473 (0.0007) [2023-12-26 23:51:47,740][105620] Updated weights for policy 1, policy_version 1166834 (0.0009) [2023-12-26 23:51:47,799][105620] Updated weights for policy 1, policy_version 1166844 (0.0009) [2023-12-26 23:51:47,859][105620] Updated weights for policy 1, policy_version 1166854 (0.0010) [2023-12-26 23:51:48,113][105692] Updated weights for policy 0, policy_version 1165483 (0.0007) [2023-12-26 23:51:48,174][105692] Updated weights for policy 0, policy_version 1165493 (0.0008) [2023-12-26 23:51:48,235][105692] Updated weights for policy 0, policy_version 1165503 (0.0008) [2023-12-26 23:51:48,605][105620] Updated weights for policy 1, policy_version 1166864 (0.0007) [2023-12-26 23:51:48,660][105620] Updated weights for policy 1, policy_version 1166874 (0.0006) [2023-12-26 23:51:48,712][105620] Updated weights for policy 1, policy_version 1166884 (0.0006) [2023-12-26 23:51:48,869][105692] Updated weights for policy 0, policy_version 1165513 (0.0009) [2023-12-26 23:51:48,926][105692] Updated weights for policy 0, policy_version 1165523 (0.0008) [2023-12-26 23:51:48,973][105692] Updated weights for policy 0, policy_version 1165533 (0.0009) [2023-12-26 23:51:49,019][105692] Updated weights for policy 0, policy_version 1165543 (0.0008) [2023-12-26 23:51:49,449][105620] Updated weights for policy 1, policy_version 1166894 (0.0009) [2023-12-26 23:51:49,501][105620] Updated weights for policy 1, policy_version 1166904 (0.0009) [2023-12-26 23:51:49,553][105620] Updated weights for policy 1, policy_version 1166914 (0.0008) [2023-12-26 23:51:49,790][105692] Updated weights for policy 0, policy_version 1165553 (0.0010) [2023-12-26 23:51:49,850][105692] Updated weights for policy 0, policy_version 1165563 (0.0009) [2023-12-26 23:51:49,918][105692] Updated weights for policy 0, policy_version 1165573 (0.0008) [2023-12-26 23:51:50,342][105620] Updated weights for policy 1, policy_version 1166924 (0.0009) [2023-12-26 23:51:50,406][105620] Updated weights for policy 1, policy_version 1166934 (0.0006) [2023-12-26 23:51:50,463][105620] Updated weights for policy 1, policy_version 1166944 (0.0005) [2023-12-26 23:51:50,615][105692] Updated weights for policy 0, policy_version 1165583 (0.0008) [2023-12-26 23:51:50,670][105692] Updated weights for policy 0, policy_version 1165593 (0.0005) [2023-12-26 23:51:50,733][105692] Updated weights for policy 0, policy_version 1165603 (0.0005) [2023-12-26 23:51:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 597221376. Throughput: 0: 9679.4, 1: 9731.0. Samples: 597211496. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:51:51,062][104569] Avg episode reward: [(0, '8729.529'), (1, '7906.266')] [2023-12-26 23:51:51,100][105620] Updated weights for policy 1, policy_version 1166954 (0.0008) [2023-12-26 23:51:51,161][105620] Updated weights for policy 1, policy_version 1166964 (0.0009) [2023-12-26 23:51:51,216][105620] Updated weights for policy 1, policy_version 1166974 (0.0009) [2023-12-26 23:51:51,278][105620] Updated weights for policy 1, policy_version 1166984 (0.0009) [2023-12-26 23:51:51,465][105692] Updated weights for policy 0, policy_version 1165613 (0.0007) [2023-12-26 23:51:51,519][105692] Updated weights for policy 0, policy_version 1165623 (0.0008) [2023-12-26 23:51:51,574][105692] Updated weights for policy 0, policy_version 1165633 (0.0009) [2023-12-26 23:51:52,066][105620] Updated weights for policy 1, policy_version 1166994 (0.0008) [2023-12-26 23:51:52,118][105620] Updated weights for policy 1, policy_version 1167004 (0.0008) [2023-12-26 23:51:52,181][105620] Updated weights for policy 1, policy_version 1167014 (0.0008) [2023-12-26 23:51:52,362][105692] Updated weights for policy 0, policy_version 1165643 (0.0009) [2023-12-26 23:51:52,426][105692] Updated weights for policy 0, policy_version 1165653 (0.0010) [2023-12-26 23:51:52,489][105692] Updated weights for policy 0, policy_version 1165663 (0.0011) [2023-12-26 23:51:52,932][105620] Updated weights for policy 1, policy_version 1167024 (0.0010) [2023-12-26 23:51:52,989][105620] Updated weights for policy 1, policy_version 1167034 (0.0011) [2023-12-26 23:51:53,056][105620] Updated weights for policy 1, policy_version 1167044 (0.0007) [2023-12-26 23:51:53,108][105692] Updated weights for policy 0, policy_version 1165673 (0.0010) [2023-12-26 23:51:53,164][105692] Updated weights for policy 0, policy_version 1165683 (0.0010) [2023-12-26 23:51:53,223][105692] Updated weights for policy 0, policy_version 1165693 (0.0010) [2023-12-26 23:51:53,284][105692] Updated weights for policy 0, policy_version 1165703 (0.0007) [2023-12-26 23:51:53,637][105620] Updated weights for policy 1, policy_version 1167054 (0.0006) [2023-12-26 23:51:53,692][105620] Updated weights for policy 1, policy_version 1167064 (0.0006) [2023-12-26 23:51:53,755][105620] Updated weights for policy 1, policy_version 1167074 (0.0005) [2023-12-26 23:51:54,026][105692] Updated weights for policy 0, policy_version 1165713 (0.0010) [2023-12-26 23:51:54,081][105692] Updated weights for policy 0, policy_version 1165723 (0.0009) [2023-12-26 23:51:54,143][105692] Updated weights for policy 0, policy_version 1165733 (0.0009) [2023-12-26 23:51:54,293][105620] Updated weights for policy 1, policy_version 1167084 (0.0005) [2023-12-26 23:51:54,348][105620] Updated weights for policy 1, policy_version 1167094 (0.0006) [2023-12-26 23:51:54,403][105620] Updated weights for policy 1, policy_version 1167104 (0.0009) [2023-12-26 23:51:54,827][105692] Updated weights for policy 0, policy_version 1165743 (0.0008) [2023-12-26 23:51:54,875][105692] Updated weights for policy 0, policy_version 1165753 (0.0009) [2023-12-26 23:51:54,925][105692] Updated weights for policy 0, policy_version 1165763 (0.0009) [2023-12-26 23:51:55,096][105620] Updated weights for policy 1, policy_version 1167114 (0.0008) [2023-12-26 23:51:55,149][105620] Updated weights for policy 1, policy_version 1167124 (0.0009) [2023-12-26 23:51:55,206][105620] Updated weights for policy 1, policy_version 1167135 (0.0010) [2023-12-26 23:51:55,605][105692] Updated weights for policy 0, policy_version 1165773 (0.0008) [2023-12-26 23:51:55,662][105692] Updated weights for policy 0, policy_version 1165783 (0.0005) [2023-12-26 23:51:55,707][105692] Updated weights for policy 0, policy_version 1165793 (0.0005) [2023-12-26 23:51:55,828][105620] Updated weights for policy 1, policy_version 1167145 (0.0006) [2023-12-26 23:51:55,878][105620] Updated weights for policy 1, policy_version 1167155 (0.0007) [2023-12-26 23:51:55,925][105620] Updated weights for policy 1, policy_version 1167166 (0.0008) [2023-12-26 23:51:55,970][105620] Updated weights for policy 1, policy_version 1167176 (0.0008) [2023-12-26 23:51:56,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 597327872. Throughput: 0: 9708.6, 1: 9840.6. Samples: 597332384. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:51:56,063][104569] Avg episode reward: [(0, '8725.892'), (1, '8037.528')] [2023-12-26 23:51:56,328][105692] Updated weights for policy 0, policy_version 1165803 (0.0007) [2023-12-26 23:51:56,378][105692] Updated weights for policy 0, policy_version 1165813 (0.0005) [2023-12-26 23:51:56,427][105692] Updated weights for policy 0, policy_version 1165823 (0.0008) [2023-12-26 23:51:56,708][105620] Updated weights for policy 1, policy_version 1167186 (0.0010) [2023-12-26 23:51:56,758][105620] Updated weights for policy 1, policy_version 1167196 (0.0010) [2023-12-26 23:51:56,816][105620] Updated weights for policy 1, policy_version 1167206 (0.0010) [2023-12-26 23:51:57,153][105692] Updated weights for policy 0, policy_version 1165833 (0.0009) [2023-12-26 23:51:57,198][105692] Updated weights for policy 0, policy_version 1165843 (0.0008) [2023-12-26 23:51:57,245][105692] Updated weights for policy 0, policy_version 1165853 (0.0008) [2023-12-26 23:51:57,290][105692] Updated weights for policy 0, policy_version 1165863 (0.0008) [2023-12-26 23:51:57,464][105620] Updated weights for policy 1, policy_version 1167216 (0.0009) [2023-12-26 23:51:57,515][105620] Updated weights for policy 1, policy_version 1167226 (0.0009) [2023-12-26 23:51:57,578][105620] Updated weights for policy 1, policy_version 1167236 (0.0006) [2023-12-26 23:51:58,037][105692] Updated weights for policy 0, policy_version 1165873 (0.0010) [2023-12-26 23:51:58,090][105692] Updated weights for policy 0, policy_version 1165883 (0.0010) [2023-12-26 23:51:58,146][105692] Updated weights for policy 0, policy_version 1165893 (0.0009) [2023-12-26 23:51:58,209][105620] Updated weights for policy 1, policy_version 1167246 (0.0007) [2023-12-26 23:51:58,271][105620] Updated weights for policy 1, policy_version 1167256 (0.0008) [2023-12-26 23:51:58,334][105620] Updated weights for policy 1, policy_version 1167266 (0.0008) [2023-12-26 23:51:58,964][105692] Updated weights for policy 0, policy_version 1165903 (0.0009) [2023-12-26 23:51:59,032][105692] Updated weights for policy 0, policy_version 1165913 (0.0009) [2023-12-26 23:51:59,087][105620] Updated weights for policy 1, policy_version 1167276 (0.0007) [2023-12-26 23:51:59,097][105692] Updated weights for policy 0, policy_version 1165923 (0.0007) [2023-12-26 23:51:59,149][105620] Updated weights for policy 1, policy_version 1167286 (0.0008) [2023-12-26 23:51:59,210][105620] Updated weights for policy 1, policy_version 1167296 (0.0007) [2023-12-26 23:51:59,878][105692] Updated weights for policy 0, policy_version 1165933 (0.0007) [2023-12-26 23:51:59,924][105620] Updated weights for policy 1, policy_version 1167306 (0.0007) [2023-12-26 23:51:59,938][105692] Updated weights for policy 0, policy_version 1165943 (0.0008) [2023-12-26 23:51:59,984][105620] Updated weights for policy 1, policy_version 1167316 (0.0008) [2023-12-26 23:51:59,988][105692] Updated weights for policy 0, policy_version 1165953 (0.0007) [2023-12-26 23:52:00,045][105620] Updated weights for policy 1, policy_version 1167326 (0.0007) [2023-12-26 23:52:00,107][105620] Updated weights for policy 1, policy_version 1167336 (0.0008) [2023-12-26 23:52:00,713][105692] Updated weights for policy 0, policy_version 1165963 (0.0009) [2023-12-26 23:52:00,766][105692] Updated weights for policy 0, policy_version 1165973 (0.0009) [2023-12-26 23:52:00,829][105692] Updated weights for policy 0, policy_version 1165983 (0.0009) [2023-12-26 23:52:00,859][105620] Updated weights for policy 1, policy_version 1167346 (0.0006) [2023-12-26 23:52:00,906][105620] Updated weights for policy 1, policy_version 1167356 (0.0007) [2023-12-26 23:52:00,962][105620] Updated weights for policy 1, policy_version 1167366 (0.0008) [2023-12-26 23:52:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 597426176. Throughput: 0: 9735.2, 1: 9886.5. Samples: 597392768. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-12-26 23:52:01,063][104569] Avg episode reward: [(0, '8899.463'), (1, '9258.657')] [2023-12-26 23:52:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001165992_298541056.pth... [2023-12-26 23:52:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001167368_298885120.pth... [2023-12-26 23:52:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001164872_298254336.pth [2023-12-26 23:52:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001166184_298582016.pth [2023-12-26 23:52:01,601][105692] Updated weights for policy 0, policy_version 1165993 (0.0008) [2023-12-26 23:52:01,682][105692] Updated weights for policy 0, policy_version 1166003 (0.0009) [2023-12-26 23:52:01,751][105692] Updated weights for policy 0, policy_version 1166013 (0.0009) [2023-12-26 23:52:01,758][105620] Updated weights for policy 1, policy_version 1167376 (0.0008) [2023-12-26 23:52:01,808][105692] Updated weights for policy 0, policy_version 1166023 (0.0008) [2023-12-26 23:52:01,818][105620] Updated weights for policy 1, policy_version 1167386 (0.0006) [2023-12-26 23:52:01,872][105620] Updated weights for policy 1, policy_version 1167396 (0.0009) [2023-12-26 23:52:02,512][105692] Updated weights for policy 0, policy_version 1166033 (0.0010) [2023-12-26 23:52:02,570][105692] Updated weights for policy 0, policy_version 1166043 (0.0010) [2023-12-26 23:52:02,629][105692] Updated weights for policy 0, policy_version 1166053 (0.0011) [2023-12-26 23:52:02,640][105620] Updated weights for policy 1, policy_version 1167406 (0.0007) [2023-12-26 23:52:02,709][105620] Updated weights for policy 1, policy_version 1167416 (0.0005) [2023-12-26 23:52:02,762][105620] Updated weights for policy 1, policy_version 1167426 (0.0005) [2023-12-26 23:52:03,204][105692] Updated weights for policy 0, policy_version 1166063 (0.0007) [2023-12-26 23:52:03,252][105692] Updated weights for policy 0, policy_version 1166073 (0.0005) [2023-12-26 23:52:03,295][105692] Updated weights for policy 0, policy_version 1166083 (0.0005) [2023-12-26 23:52:03,375][105620] Updated weights for policy 1, policy_version 1167436 (0.0005) [2023-12-26 23:52:03,428][105620] Updated weights for policy 1, policy_version 1167446 (0.0005) [2023-12-26 23:52:03,471][105620] Updated weights for policy 1, policy_version 1167456 (0.0005) [2023-12-26 23:52:03,867][105692] Updated weights for policy 0, policy_version 1166093 (0.0007) [2023-12-26 23:52:03,926][105692] Updated weights for policy 0, policy_version 1166103 (0.0011) [2023-12-26 23:52:03,986][105692] Updated weights for policy 0, policy_version 1166113 (0.0011) [2023-12-26 23:52:04,064][105620] Updated weights for policy 1, policy_version 1167466 (0.0006) [2023-12-26 23:52:04,124][105620] Updated weights for policy 1, policy_version 1167476 (0.0006) [2023-12-26 23:52:04,196][105620] Updated weights for policy 1, policy_version 1167486 (0.0006) [2023-12-26 23:52:04,261][105620] Updated weights for policy 1, policy_version 1167496 (0.0006) [2023-12-26 23:52:04,765][105620] Updated weights for policy 1, policy_version 1167506 (0.0010) [2023-12-26 23:52:04,776][105692] Updated weights for policy 0, policy_version 1166123 (0.0010) [2023-12-26 23:52:04,808][105620] Updated weights for policy 1, policy_version 1167516 (0.0005) [2023-12-26 23:52:04,827][105692] Updated weights for policy 0, policy_version 1166133 (0.0005) [2023-12-26 23:52:04,857][105620] Updated weights for policy 1, policy_version 1167526 (0.0006) [2023-12-26 23:52:04,882][105692] Updated weights for policy 0, policy_version 1166143 (0.0006) [2023-12-26 23:52:05,579][105692] Updated weights for policy 0, policy_version 1166153 (0.0006) [2023-12-26 23:52:05,589][105620] Updated weights for policy 1, policy_version 1167536 (0.0006) [2023-12-26 23:52:05,634][105692] Updated weights for policy 0, policy_version 1166163 (0.0009) [2023-12-26 23:52:05,641][105620] Updated weights for policy 1, policy_version 1167546 (0.0005) [2023-12-26 23:52:05,681][105692] Updated weights for policy 0, policy_version 1166173 (0.0008) [2023-12-26 23:52:05,692][105620] Updated weights for policy 1, policy_version 1167556 (0.0005) [2023-12-26 23:52:05,745][105692] Updated weights for policy 0, policy_version 1166183 (0.0009) [2023-12-26 23:52:06,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 597524480. Throughput: 0: 9715.8, 1: 9913.6. Samples: 597511728. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:52:06,062][104569] Avg episode reward: [(0, '8626.213'), (1, '9261.797')] [2023-12-26 23:52:06,372][105620] Updated weights for policy 1, policy_version 1167566 (0.0009) [2023-12-26 23:52:06,396][105692] Updated weights for policy 0, policy_version 1166193 (0.0006) [2023-12-26 23:52:06,438][105620] Updated weights for policy 1, policy_version 1167576 (0.0011) [2023-12-26 23:52:06,449][105692] Updated weights for policy 0, policy_version 1166203 (0.0005) [2023-12-26 23:52:06,501][105620] Updated weights for policy 1, policy_version 1167586 (0.0011) [2023-12-26 23:52:06,502][105692] Updated weights for policy 0, policy_version 1166213 (0.0010) [2023-12-26 23:52:07,226][105692] Updated weights for policy 0, policy_version 1166223 (0.0011) [2023-12-26 23:52:07,227][105620] Updated weights for policy 1, policy_version 1167596 (0.0011) [2023-12-26 23:52:07,285][105692] Updated weights for policy 0, policy_version 1166233 (0.0010) [2023-12-26 23:52:07,287][105620] Updated weights for policy 1, policy_version 1167606 (0.0010) [2023-12-26 23:52:07,335][105620] Updated weights for policy 1, policy_version 1167616 (0.0010) [2023-12-26 23:52:07,344][105692] Updated weights for policy 0, policy_version 1166243 (0.0010) [2023-12-26 23:52:07,968][105620] Updated weights for policy 1, policy_version 1167626 (0.0009) [2023-12-26 23:52:08,027][105620] Updated weights for policy 1, policy_version 1167636 (0.0006) [2023-12-26 23:52:08,044][105692] Updated weights for policy 0, policy_version 1166253 (0.0011) [2023-12-26 23:52:08,085][105620] Updated weights for policy 1, policy_version 1167646 (0.0005) [2023-12-26 23:52:08,096][105692] Updated weights for policy 0, policy_version 1166263 (0.0010) [2023-12-26 23:52:08,142][105620] Updated weights for policy 1, policy_version 1167656 (0.0006) [2023-12-26 23:52:08,163][105692] Updated weights for policy 0, policy_version 1166273 (0.0011) [2023-12-26 23:52:08,801][105620] Updated weights for policy 1, policy_version 1167666 (0.0010) [2023-12-26 23:52:08,863][105620] Updated weights for policy 1, policy_version 1167676 (0.0010) [2023-12-26 23:52:08,878][105692] Updated weights for policy 0, policy_version 1166283 (0.0009) [2023-12-26 23:52:08,928][105620] Updated weights for policy 1, policy_version 1167686 (0.0007) [2023-12-26 23:52:08,937][105692] Updated weights for policy 0, policy_version 1166293 (0.0011) [2023-12-26 23:52:08,994][105692] Updated weights for policy 0, policy_version 1166303 (0.0009) [2023-12-26 23:52:09,723][105692] Updated weights for policy 0, policy_version 1166313 (0.0009) [2023-12-26 23:52:09,779][105620] Updated weights for policy 1, policy_version 1167696 (0.0008) [2023-12-26 23:52:09,789][105692] Updated weights for policy 0, policy_version 1166323 (0.0008) [2023-12-26 23:52:09,843][105620] Updated weights for policy 1, policy_version 1167706 (0.0008) [2023-12-26 23:52:09,856][105692] Updated weights for policy 0, policy_version 1166333 (0.0008) [2023-12-26 23:52:09,909][105620] Updated weights for policy 1, policy_version 1167716 (0.0008) [2023-12-26 23:52:09,920][105692] Updated weights for policy 0, policy_version 1166343 (0.0008) [2023-12-26 23:52:10,533][105692] Updated weights for policy 0, policy_version 1166353 (0.0007) [2023-12-26 23:52:10,606][105692] Updated weights for policy 0, policy_version 1166363 (0.0009) [2023-12-26 23:52:10,661][105620] Updated weights for policy 1, policy_version 1167726 (0.0008) [2023-12-26 23:52:10,668][105692] Updated weights for policy 0, policy_version 1166373 (0.0007) [2023-12-26 23:52:10,731][105620] Updated weights for policy 1, policy_version 1167736 (0.0009) [2023-12-26 23:52:10,781][105620] Updated weights for policy 1, policy_version 1167746 (0.0008) [2023-12-26 23:52:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 597622784. Throughput: 0: 9783.0, 1: 9954.7. Samples: 597630348. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:52:11,063][104569] Avg episode reward: [(0, '8903.129'), (1, '8987.872')] [2023-12-26 23:52:11,487][105620] Updated weights for policy 1, policy_version 1167756 (0.0007) [2023-12-26 23:52:11,497][105692] Updated weights for policy 0, policy_version 1166383 (0.0008) [2023-12-26 23:52:11,547][105620] Updated weights for policy 1, policy_version 1167766 (0.0010) [2023-12-26 23:52:11,558][105692] Updated weights for policy 0, policy_version 1166393 (0.0006) [2023-12-26 23:52:11,619][105692] Updated weights for policy 0, policy_version 1166403 (0.0007) [2023-12-26 23:52:11,619][105620] Updated weights for policy 1, policy_version 1167776 (0.0011) [2023-12-26 23:52:12,291][105620] Updated weights for policy 1, policy_version 1167786 (0.0010) [2023-12-26 23:52:12,357][105620] Updated weights for policy 1, policy_version 1167796 (0.0008) [2023-12-26 23:52:12,420][105620] Updated weights for policy 1, policy_version 1167806 (0.0006) [2023-12-26 23:52:12,426][105692] Updated weights for policy 0, policy_version 1166413 (0.0007) [2023-12-26 23:52:12,481][105620] Updated weights for policy 1, policy_version 1167816 (0.0005) [2023-12-26 23:52:12,481][105692] Updated weights for policy 0, policy_version 1166423 (0.0005) [2023-12-26 23:52:12,543][105692] Updated weights for policy 0, policy_version 1166433 (0.0005) [2023-12-26 23:52:13,112][105620] Updated weights for policy 1, policy_version 1167826 (0.0009) [2023-12-26 23:52:13,168][105620] Updated weights for policy 1, policy_version 1167836 (0.0005) [2023-12-26 23:52:13,214][105620] Updated weights for policy 1, policy_version 1167846 (0.0005) [2023-12-26 23:52:13,246][105692] Updated weights for policy 0, policy_version 1166443 (0.0009) [2023-12-26 23:52:13,300][105692] Updated weights for policy 0, policy_version 1166453 (0.0010) [2023-12-26 23:52:13,360][105692] Updated weights for policy 0, policy_version 1166465 (0.0010) [2023-12-26 23:52:13,767][105620] Updated weights for policy 1, policy_version 1167856 (0.0005) [2023-12-26 23:52:13,824][105620] Updated weights for policy 1, policy_version 1167866 (0.0007) [2023-12-26 23:52:13,886][105620] Updated weights for policy 1, policy_version 1167876 (0.0009) [2023-12-26 23:52:13,983][105692] Updated weights for policy 0, policy_version 1166475 (0.0007) [2023-12-26 23:52:14,032][105692] Updated weights for policy 0, policy_version 1166485 (0.0010) [2023-12-26 23:52:14,080][105692] Updated weights for policy 0, policy_version 1166495 (0.0010) [2023-12-26 23:52:14,555][105620] Updated weights for policy 1, policy_version 1167886 (0.0009) [2023-12-26 23:52:14,618][105620] Updated weights for policy 1, policy_version 1167896 (0.0011) [2023-12-26 23:52:14,687][105620] Updated weights for policy 1, policy_version 1167906 (0.0011) [2023-12-26 23:52:14,777][105692] Updated weights for policy 0, policy_version 1166505 (0.0010) [2023-12-26 23:52:14,842][105692] Updated weights for policy 0, policy_version 1166515 (0.0008) [2023-12-26 23:52:14,905][105692] Updated weights for policy 0, policy_version 1166525 (0.0008) [2023-12-26 23:52:14,969][105692] Updated weights for policy 0, policy_version 1166535 (0.0009) [2023-12-26 23:52:15,440][105620] Updated weights for policy 1, policy_version 1167916 (0.0011) [2023-12-26 23:52:15,488][105620] Updated weights for policy 1, policy_version 1167926 (0.0010) [2023-12-26 23:52:15,540][105620] Updated weights for policy 1, policy_version 1167936 (0.0009) [2023-12-26 23:52:15,634][105692] Updated weights for policy 0, policy_version 1166545 (0.0008) [2023-12-26 23:52:15,698][105692] Updated weights for policy 0, policy_version 1166555 (0.0008) [2023-12-26 23:52:15,757][105692] Updated weights for policy 0, policy_version 1166565 (0.0008) [2023-12-26 23:52:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 597721088. Throughput: 0: 9715.1, 1: 10023.7. Samples: 597688840. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:52:16,063][104569] Avg episode reward: [(0, '8994.935'), (1, '8805.838')] [2023-12-26 23:52:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001166568_298688512.pth... [2023-12-26 23:52:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001167944_299032576.pth... [2023-12-26 23:52:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001165416_298393600.pth [2023-12-26 23:52:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001166792_298737664.pth [2023-12-26 23:52:16,274][105620] Updated weights for policy 1, policy_version 1167946 (0.0008) [2023-12-26 23:52:16,336][105620] Updated weights for policy 1, policy_version 1167956 (0.0010) [2023-12-26 23:52:16,399][105620] Updated weights for policy 1, policy_version 1167966 (0.0010) [2023-12-26 23:52:16,457][105620] Updated weights for policy 1, policy_version 1167976 (0.0010) [2023-12-26 23:52:16,463][105692] Updated weights for policy 0, policy_version 1166575 (0.0006) [2023-12-26 23:52:16,512][105692] Updated weights for policy 0, policy_version 1166585 (0.0008) [2023-12-26 23:52:16,568][105692] Updated weights for policy 0, policy_version 1166595 (0.0008) [2023-12-26 23:52:17,163][105620] Updated weights for policy 1, policy_version 1167986 (0.0010) [2023-12-26 23:52:17,214][105620] Updated weights for policy 1, policy_version 1167996 (0.0010) [2023-12-26 23:52:17,266][105620] Updated weights for policy 1, policy_version 1168006 (0.0010) [2023-12-26 23:52:17,338][105692] Updated weights for policy 0, policy_version 1166605 (0.0008) [2023-12-26 23:52:17,393][105692] Updated weights for policy 0, policy_version 1166615 (0.0008) [2023-12-26 23:52:17,447][105692] Updated weights for policy 0, policy_version 1166625 (0.0009) [2023-12-26 23:52:18,047][105620] Updated weights for policy 1, policy_version 1168016 (0.0008) [2023-12-26 23:52:18,106][105620] Updated weights for policy 1, policy_version 1168026 (0.0005) [2023-12-26 23:52:18,164][105620] Updated weights for policy 1, policy_version 1168036 (0.0006) [2023-12-26 23:52:18,211][105692] Updated weights for policy 0, policy_version 1166635 (0.0009) [2023-12-26 23:52:18,269][105692] Updated weights for policy 0, policy_version 1166645 (0.0009) [2023-12-26 23:52:18,329][105692] Updated weights for policy 0, policy_version 1166656 (0.0010) [2023-12-26 23:52:18,838][105620] Updated weights for policy 1, policy_version 1168046 (0.0008) [2023-12-26 23:52:18,903][105620] Updated weights for policy 1, policy_version 1168056 (0.0009) [2023-12-26 23:52:18,961][105620] Updated weights for policy 1, policy_version 1168066 (0.0005) [2023-12-26 23:52:19,117][105692] Updated weights for policy 0, policy_version 1166666 (0.0009) [2023-12-26 23:52:19,170][105692] Updated weights for policy 0, policy_version 1166676 (0.0009) [2023-12-26 23:52:19,222][105692] Updated weights for policy 0, policy_version 1166687 (0.0010) [2023-12-26 23:52:19,600][105620] Updated weights for policy 1, policy_version 1168076 (0.0006) [2023-12-26 23:52:19,653][105620] Updated weights for policy 1, policy_version 1168086 (0.0006) [2023-12-26 23:52:19,700][105620] Updated weights for policy 1, policy_version 1168096 (0.0005) [2023-12-26 23:52:20,088][105692] Updated weights for policy 0, policy_version 1166697 (0.0009) [2023-12-26 23:52:20,141][105692] Updated weights for policy 0, policy_version 1166707 (0.0009) [2023-12-26 23:52:20,195][105692] Updated weights for policy 0, policy_version 1166717 (0.0008) [2023-12-26 23:52:20,249][105692] Updated weights for policy 0, policy_version 1166727 (0.0008) [2023-12-26 23:52:20,356][105620] Updated weights for policy 1, policy_version 1168106 (0.0009) [2023-12-26 23:52:20,424][105620] Updated weights for policy 1, policy_version 1168116 (0.0009) [2023-12-26 23:52:20,479][105620] Updated weights for policy 1, policy_version 1168126 (0.0009) [2023-12-26 23:52:20,540][105620] Updated weights for policy 1, policy_version 1168136 (0.0007) [2023-12-26 23:52:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 597811200. Throughput: 0: 9605.6, 1: 10009.2. Samples: 597805020. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:52:21,063][104569] Avg episode reward: [(0, '9081.795'), (1, '9080.393')] [2023-12-26 23:52:21,156][105692] Updated weights for policy 0, policy_version 1166737 (0.0008) [2023-12-26 23:52:21,173][105620] Updated weights for policy 1, policy_version 1168146 (0.0008) [2023-12-26 23:52:21,222][105692] Updated weights for policy 0, policy_version 1166747 (0.0006) [2023-12-26 23:52:21,238][105620] Updated weights for policy 1, policy_version 1168156 (0.0007) [2023-12-26 23:52:21,287][105692] Updated weights for policy 0, policy_version 1166757 (0.0007) [2023-12-26 23:52:21,303][105620] Updated weights for policy 1, policy_version 1168166 (0.0008) [2023-12-26 23:52:22,042][105692] Updated weights for policy 0, policy_version 1166767 (0.0008) [2023-12-26 23:52:22,077][105620] Updated weights for policy 1, policy_version 1168176 (0.0006) [2023-12-26 23:52:22,104][105692] Updated weights for policy 0, policy_version 1166777 (0.0007) [2023-12-26 23:52:22,142][105620] Updated weights for policy 1, policy_version 1168186 (0.0008) [2023-12-26 23:52:22,157][105692] Updated weights for policy 0, policy_version 1166787 (0.0006) [2023-12-26 23:52:22,197][105620] Updated weights for policy 1, policy_version 1168196 (0.0009) [2023-12-26 23:52:22,866][105692] Updated weights for policy 0, policy_version 1166797 (0.0006) [2023-12-26 23:52:22,930][105692] Updated weights for policy 0, policy_version 1166807 (0.0010) [2023-12-26 23:52:23,000][105692] Updated weights for policy 0, policy_version 1166817 (0.0011) [2023-12-26 23:52:23,027][105620] Updated weights for policy 1, policy_version 1168206 (0.0008) [2023-12-26 23:52:23,082][105620] Updated weights for policy 1, policy_version 1168216 (0.0008) [2023-12-26 23:52:23,134][105620] Updated weights for policy 1, policy_version 1168226 (0.0008) [2023-12-26 23:52:23,717][105692] Updated weights for policy 0, policy_version 1166827 (0.0011) [2023-12-26 23:52:23,785][105692] Updated weights for policy 0, policy_version 1166837 (0.0010) [2023-12-26 23:52:23,847][105692] Updated weights for policy 0, policy_version 1166847 (0.0009) [2023-12-26 23:52:23,909][105620] Updated weights for policy 1, policy_version 1168236 (0.0007) [2023-12-26 23:52:23,972][105620] Updated weights for policy 1, policy_version 1168246 (0.0008) [2023-12-26 23:52:24,016][105620] Updated weights for policy 1, policy_version 1168256 (0.0008) [2023-12-26 23:52:24,556][105692] Updated weights for policy 0, policy_version 1166857 (0.0009) [2023-12-26 23:52:24,610][105692] Updated weights for policy 0, policy_version 1166867 (0.0010) [2023-12-26 23:52:24,673][105692] Updated weights for policy 0, policy_version 1166877 (0.0010) [2023-12-26 23:52:24,728][105692] Updated weights for policy 0, policy_version 1166887 (0.0010) [2023-12-26 23:52:24,792][105620] Updated weights for policy 1, policy_version 1168266 (0.0008) [2023-12-26 23:52:24,851][105620] Updated weights for policy 1, policy_version 1168276 (0.0008) [2023-12-26 23:52:24,899][105620] Updated weights for policy 1, policy_version 1168286 (0.0008) [2023-12-26 23:52:24,966][105620] Updated weights for policy 1, policy_version 1168296 (0.0008) [2023-12-26 23:52:25,394][105692] Updated weights for policy 0, policy_version 1166897 (0.0006) [2023-12-26 23:52:25,450][105692] Updated weights for policy 0, policy_version 1166907 (0.0005) [2023-12-26 23:52:25,506][105692] Updated weights for policy 0, policy_version 1166917 (0.0005) [2023-12-26 23:52:25,832][105620] Updated weights for policy 1, policy_version 1168307 (0.0008) [2023-12-26 23:52:25,886][105620] Updated weights for policy 1, policy_version 1168318 (0.0010) [2023-12-26 23:52:26,002][105692] Updated weights for policy 0, policy_version 1166927 (0.0005) [2023-12-26 23:52:26,053][105692] Updated weights for policy 0, policy_version 1166937 (0.0005) [2023-12-26 23:52:26,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19660.9, 300 sec: 19466.4). Total num frames: 597909504. Throughput: 0: 9663.3, 1: 9818.8. Samples: 597916672. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:52:26,062][104569] Avg episode reward: [(0, '9259.773'), (1, '9079.376')] [2023-12-26 23:52:26,105][105692] Updated weights for policy 0, policy_version 1166947 (0.0005) [2023-12-26 23:52:26,718][105692] Updated weights for policy 0, policy_version 1166957 (0.0008) [2023-12-26 23:52:26,748][105620] Updated weights for policy 1, policy_version 1168330 (0.0009) [2023-12-26 23:52:26,762][105692] Updated weights for policy 0, policy_version 1166967 (0.0010) [2023-12-26 23:52:26,796][105620] Updated weights for policy 1, policy_version 1168340 (0.0005) [2023-12-26 23:52:26,810][105692] Updated weights for policy 0, policy_version 1166977 (0.0010) [2023-12-26 23:52:26,847][105620] Updated weights for policy 1, policy_version 1168350 (0.0005) [2023-12-26 23:52:26,890][105620] Updated weights for policy 1, policy_version 1168360 (0.0007) [2023-12-26 23:52:27,381][105692] Updated weights for policy 0, policy_version 1166987 (0.0009) [2023-12-26 23:52:27,432][105692] Updated weights for policy 0, policy_version 1166997 (0.0010) [2023-12-26 23:52:27,486][105692] Updated weights for policy 0, policy_version 1167007 (0.0010) [2023-12-26 23:52:27,754][105620] Updated weights for policy 1, policy_version 1168370 (0.0008) [2023-12-26 23:52:27,797][105620] Updated weights for policy 1, policy_version 1168380 (0.0008) [2023-12-26 23:52:27,841][105620] Updated weights for policy 1, policy_version 1168390 (0.0008) [2023-12-26 23:52:28,219][105692] Updated weights for policy 0, policy_version 1167017 (0.0010) [2023-12-26 23:52:28,262][105692] Updated weights for policy 0, policy_version 1167027 (0.0010) [2023-12-26 23:52:28,306][105692] Updated weights for policy 0, policy_version 1167037 (0.0010) [2023-12-26 23:52:28,368][105692] Updated weights for policy 0, policy_version 1167047 (0.0010) [2023-12-26 23:52:28,592][105620] Updated weights for policy 1, policy_version 1168400 (0.0008) [2023-12-26 23:52:28,648][105620] Updated weights for policy 1, policy_version 1168410 (0.0008) [2023-12-26 23:52:28,713][105620] Updated weights for policy 1, policy_version 1168420 (0.0008) [2023-12-26 23:52:29,120][105692] Updated weights for policy 0, policy_version 1167057 (0.0010) [2023-12-26 23:52:29,191][105692] Updated weights for policy 0, policy_version 1167067 (0.0010) [2023-12-26 23:52:29,247][105692] Updated weights for policy 0, policy_version 1167077 (0.0011) [2023-12-26 23:52:29,479][105620] Updated weights for policy 1, policy_version 1168430 (0.0008) [2023-12-26 23:52:29,538][105620] Updated weights for policy 1, policy_version 1168440 (0.0005) [2023-12-26 23:52:29,582][105620] Updated weights for policy 1, policy_version 1168450 (0.0008) [2023-12-26 23:52:29,976][105692] Updated weights for policy 0, policy_version 1167087 (0.0009) [2023-12-26 23:52:30,044][105692] Updated weights for policy 0, policy_version 1167097 (0.0010) [2023-12-26 23:52:30,106][105692] Updated weights for policy 0, policy_version 1167107 (0.0011) [2023-12-26 23:52:30,321][105620] Updated weights for policy 1, policy_version 1168460 (0.0008) [2023-12-26 23:52:30,373][105620] Updated weights for policy 1, policy_version 1168470 (0.0009) [2023-12-26 23:52:30,425][105620] Updated weights for policy 1, policy_version 1168480 (0.0008) [2023-12-26 23:52:30,761][105692] Updated weights for policy 0, policy_version 1167117 (0.0011) [2023-12-26 23:52:30,818][105692] Updated weights for policy 0, policy_version 1167127 (0.0010) [2023-12-26 23:52:30,872][105692] Updated weights for policy 0, policy_version 1167137 (0.0010) [2023-12-26 23:52:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 598007808. Throughput: 0: 9791.6, 1: 9791.1. Samples: 597976696. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:52:31,063][104569] Avg episode reward: [(0, '9171.280'), (1, '8988.004')] [2023-12-26 23:52:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001167144_298835968.pth... [2023-12-26 23:52:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001168488_299171840.pth... [2023-12-26 23:52:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001167368_298885120.pth [2023-12-26 23:52:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001165992_298541056.pth [2023-12-26 23:52:31,212][105620] Updated weights for policy 1, policy_version 1168490 (0.0008) [2023-12-26 23:52:31,279][105620] Updated weights for policy 1, policy_version 1168500 (0.0007) [2023-12-26 23:52:31,338][105620] Updated weights for policy 1, policy_version 1168510 (0.0008) [2023-12-26 23:52:31,408][105620] Updated weights for policy 1, policy_version 1168520 (0.0008) [2023-12-26 23:52:31,594][105692] Updated weights for policy 0, policy_version 1167147 (0.0010) [2023-12-26 23:52:31,655][105692] Updated weights for policy 0, policy_version 1167157 (0.0009) [2023-12-26 23:52:31,711][105692] Updated weights for policy 0, policy_version 1167167 (0.0009) [2023-12-26 23:52:32,165][105620] Updated weights for policy 1, policy_version 1168530 (0.0008) [2023-12-26 23:52:32,225][105620] Updated weights for policy 1, policy_version 1168540 (0.0008) [2023-12-26 23:52:32,286][105620] Updated weights for policy 1, policy_version 1168550 (0.0008) [2023-12-26 23:52:32,399][105692] Updated weights for policy 0, policy_version 1167177 (0.0008) [2023-12-26 23:52:32,445][105692] Updated weights for policy 0, policy_version 1167187 (0.0009) [2023-12-26 23:52:32,497][105692] Updated weights for policy 0, policy_version 1167197 (0.0010) [2023-12-26 23:52:32,556][105692] Updated weights for policy 0, policy_version 1167207 (0.0010) [2023-12-26 23:52:32,980][105620] Updated weights for policy 1, policy_version 1168560 (0.0008) [2023-12-26 23:52:33,026][105620] Updated weights for policy 1, policy_version 1168570 (0.0008) [2023-12-26 23:52:33,083][105620] Updated weights for policy 1, policy_version 1168580 (0.0009) [2023-12-26 23:52:33,258][105692] Updated weights for policy 0, policy_version 1167217 (0.0010) [2023-12-26 23:52:33,317][105692] Updated weights for policy 0, policy_version 1167227 (0.0010) [2023-12-26 23:52:33,372][105692] Updated weights for policy 0, policy_version 1167237 (0.0008) [2023-12-26 23:52:33,756][105620] Updated weights for policy 1, policy_version 1168590 (0.0010) [2023-12-26 23:52:33,808][105620] Updated weights for policy 1, policy_version 1168600 (0.0007) [2023-12-26 23:52:33,854][105620] Updated weights for policy 1, policy_version 1168610 (0.0005) [2023-12-26 23:52:33,907][105692] Updated weights for policy 0, policy_version 1167247 (0.0005) [2023-12-26 23:52:33,960][105692] Updated weights for policy 0, policy_version 1167257 (0.0005) [2023-12-26 23:52:34,018][105692] Updated weights for policy 0, policy_version 1167267 (0.0008) [2023-12-26 23:52:34,522][105620] Updated weights for policy 1, policy_version 1168620 (0.0007) [2023-12-26 23:52:34,577][105620] Updated weights for policy 1, policy_version 1168630 (0.0008) [2023-12-26 23:52:34,640][105620] Updated weights for policy 1, policy_version 1168640 (0.0007) [2023-12-26 23:52:34,667][105692] Updated weights for policy 0, policy_version 1167277 (0.0008) [2023-12-26 23:52:34,727][105692] Updated weights for policy 0, policy_version 1167287 (0.0011) [2023-12-26 23:52:34,785][105692] Updated weights for policy 0, policy_version 1167297 (0.0010) [2023-12-26 23:52:35,246][105620] Updated weights for policy 1, policy_version 1168650 (0.0007) [2023-12-26 23:52:35,294][105620] Updated weights for policy 1, policy_version 1168660 (0.0008) [2023-12-26 23:52:35,345][105620] Updated weights for policy 1, policy_version 1168670 (0.0006) [2023-12-26 23:52:35,394][105620] Updated weights for policy 1, policy_version 1168680 (0.0005) [2023-12-26 23:52:35,498][105692] Updated weights for policy 0, policy_version 1167307 (0.0010) [2023-12-26 23:52:35,560][105692] Updated weights for policy 0, policy_version 1167317 (0.0010) [2023-12-26 23:52:35,618][105692] Updated weights for policy 0, policy_version 1167327 (0.0010) [2023-12-26 23:52:36,052][105620] Updated weights for policy 1, policy_version 1168690 (0.0006) [2023-12-26 23:52:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 598106112. Throughput: 0: 9844.1, 1: 9818.3. Samples: 598096304. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:52:36,062][104569] Avg episode reward: [(0, '8724.523'), (1, '8805.231')] [2023-12-26 23:52:36,102][105620] Updated weights for policy 1, policy_version 1168700 (0.0006) [2023-12-26 23:52:36,169][105620] Updated weights for policy 1, policy_version 1168710 (0.0008) [2023-12-26 23:52:36,347][105692] Updated weights for policy 0, policy_version 1167337 (0.0010) [2023-12-26 23:52:36,395][105692] Updated weights for policy 0, policy_version 1167347 (0.0010) [2023-12-26 23:52:36,444][105692] Updated weights for policy 0, policy_version 1167357 (0.0010) [2023-12-26 23:52:36,504][105692] Updated weights for policy 0, policy_version 1167367 (0.0011) [2023-12-26 23:52:36,878][105620] Updated weights for policy 1, policy_version 1168720 (0.0006) [2023-12-26 23:52:36,934][105620] Updated weights for policy 1, policy_version 1168730 (0.0006) [2023-12-26 23:52:36,985][105620] Updated weights for policy 1, policy_version 1168740 (0.0005) [2023-12-26 23:52:37,279][105692] Updated weights for policy 0, policy_version 1167377 (0.0010) [2023-12-26 23:52:37,324][105692] Updated weights for policy 0, policy_version 1167387 (0.0010) [2023-12-26 23:52:37,379][105692] Updated weights for policy 0, policy_version 1167397 (0.0010) [2023-12-26 23:52:37,553][105620] Updated weights for policy 1, policy_version 1168750 (0.0005) [2023-12-26 23:52:37,626][105620] Updated weights for policy 1, policy_version 1168760 (0.0005) [2023-12-26 23:52:37,679][105620] Updated weights for policy 1, policy_version 1168770 (0.0005) [2023-12-26 23:52:38,072][105692] Updated weights for policy 0, policy_version 1167407 (0.0008) [2023-12-26 23:52:38,134][105692] Updated weights for policy 0, policy_version 1167417 (0.0009) [2023-12-26 23:52:38,189][105692] Updated weights for policy 0, policy_version 1167427 (0.0008) [2023-12-26 23:52:38,212][105620] Updated weights for policy 1, policy_version 1168780 (0.0007) [2023-12-26 23:52:38,262][105620] Updated weights for policy 1, policy_version 1168790 (0.0009) [2023-12-26 23:52:38,319][105620] Updated weights for policy 1, policy_version 1168800 (0.0007) [2023-12-26 23:52:38,939][105620] Updated weights for policy 1, policy_version 1168810 (0.0008) [2023-12-26 23:52:38,987][105620] Updated weights for policy 1, policy_version 1168820 (0.0009) [2023-12-26 23:52:39,016][105692] Updated weights for policy 0, policy_version 1167437 (0.0009) [2023-12-26 23:52:39,038][105620] Updated weights for policy 1, policy_version 1168830 (0.0008) [2023-12-26 23:52:39,073][105692] Updated weights for policy 0, policy_version 1167447 (0.0008) [2023-12-26 23:52:39,091][105620] Updated weights for policy 1, policy_version 1168840 (0.0006) [2023-12-26 23:52:39,129][105692] Updated weights for policy 0, policy_version 1167457 (0.0007) [2023-12-26 23:52:39,865][105692] Updated weights for policy 0, policy_version 1167467 (0.0008) [2023-12-26 23:52:39,927][105692] Updated weights for policy 0, policy_version 1167477 (0.0008) [2023-12-26 23:52:39,935][105620] Updated weights for policy 1, policy_version 1168850 (0.0007) [2023-12-26 23:52:39,990][105692] Updated weights for policy 0, policy_version 1167487 (0.0009) [2023-12-26 23:52:39,996][105620] Updated weights for policy 1, policy_version 1168860 (0.0009) [2023-12-26 23:52:40,063][105620] Updated weights for policy 1, policy_version 1168870 (0.0007) [2023-12-26 23:52:40,768][105620] Updated weights for policy 1, policy_version 1168880 (0.0009) [2023-12-26 23:52:40,779][105692] Updated weights for policy 0, policy_version 1167497 (0.0009) [2023-12-26 23:52:40,827][105620] Updated weights for policy 1, policy_version 1168890 (0.0008) [2023-12-26 23:52:40,838][105692] Updated weights for policy 0, policy_version 1167507 (0.0009) [2023-12-26 23:52:40,884][105620] Updated weights for policy 1, policy_version 1168900 (0.0007) [2023-12-26 23:52:40,903][105692] Updated weights for policy 0, policy_version 1167517 (0.0008) [2023-12-26 23:52:40,967][105692] Updated weights for policy 0, policy_version 1167527 (0.0009) [2023-12-26 23:52:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 598212608. Throughput: 0: 9779.3, 1: 9843.2. Samples: 598215396. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:52:41,063][104569] Avg episode reward: [(0, '8816.859'), (1, '9077.861')] [2023-12-26 23:52:41,573][105620] Updated weights for policy 1, policy_version 1168910 (0.0009) [2023-12-26 23:52:41,632][105620] Updated weights for policy 1, policy_version 1168920 (0.0011) [2023-12-26 23:52:41,708][105620] Updated weights for policy 1, policy_version 1168930 (0.0009) [2023-12-26 23:52:41,754][105692] Updated weights for policy 0, policy_version 1167537 (0.0007) [2023-12-26 23:52:41,817][105692] Updated weights for policy 0, policy_version 1167547 (0.0007) [2023-12-26 23:52:41,886][105692] Updated weights for policy 0, policy_version 1167557 (0.0005) [2023-12-26 23:52:42,461][105620] Updated weights for policy 1, policy_version 1168940 (0.0011) [2023-12-26 23:52:42,513][105620] Updated weights for policy 1, policy_version 1168950 (0.0010) [2023-12-26 23:52:42,568][105620] Updated weights for policy 1, policy_version 1168960 (0.0010) [2023-12-26 23:52:42,599][105692] Updated weights for policy 0, policy_version 1167567 (0.0008) [2023-12-26 23:52:42,664][105692] Updated weights for policy 0, policy_version 1167577 (0.0009) [2023-12-26 23:52:42,729][105692] Updated weights for policy 0, policy_version 1167587 (0.0007) [2023-12-26 23:52:43,223][105620] Updated weights for policy 1, policy_version 1168970 (0.0009) [2023-12-26 23:52:43,272][105620] Updated weights for policy 1, policy_version 1168980 (0.0005) [2023-12-26 23:52:43,324][105620] Updated weights for policy 1, policy_version 1168990 (0.0005) [2023-12-26 23:52:43,396][105620] Updated weights for policy 1, policy_version 1169000 (0.0005) [2023-12-26 23:52:43,478][105692] Updated weights for policy 0, policy_version 1167597 (0.0009) [2023-12-26 23:52:43,532][105692] Updated weights for policy 0, policy_version 1167609 (0.0010) [2023-12-26 23:52:43,591][105692] Updated weights for policy 0, policy_version 1167620 (0.0009) [2023-12-26 23:52:43,927][105620] Updated weights for policy 1, policy_version 1169010 (0.0005) [2023-12-26 23:52:43,978][105620] Updated weights for policy 1, policy_version 1169020 (0.0010) [2023-12-26 23:52:44,034][105620] Updated weights for policy 1, policy_version 1169030 (0.0011) [2023-12-26 23:52:44,346][105692] Updated weights for policy 0, policy_version 1167630 (0.0007) [2023-12-26 23:52:44,392][105692] Updated weights for policy 0, policy_version 1167640 (0.0005) [2023-12-26 23:52:44,447][105692] Updated weights for policy 0, policy_version 1167650 (0.0007) [2023-12-26 23:52:44,781][105620] Updated weights for policy 1, policy_version 1169040 (0.0011) [2023-12-26 23:52:44,837][105620] Updated weights for policy 1, policy_version 1169050 (0.0011) [2023-12-26 23:52:44,893][105620] Updated weights for policy 1, policy_version 1169060 (0.0010) [2023-12-26 23:52:45,175][105692] Updated weights for policy 0, policy_version 1167660 (0.0011) [2023-12-26 23:52:45,235][105692] Updated weights for policy 0, policy_version 1167670 (0.0011) [2023-12-26 23:52:45,285][105692] Updated weights for policy 0, policy_version 1167680 (0.0011) [2023-12-26 23:52:45,649][105620] Updated weights for policy 1, policy_version 1169070 (0.0011) [2023-12-26 23:52:45,713][105620] Updated weights for policy 1, policy_version 1169080 (0.0011) [2023-12-26 23:52:45,774][105620] Updated weights for policy 1, policy_version 1169090 (0.0010) [2023-12-26 23:52:45,995][105692] Updated weights for policy 0, policy_version 1167690 (0.0011) [2023-12-26 23:52:46,042][105692] Updated weights for policy 0, policy_version 1167700 (0.0009) [2023-12-26 23:52:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 598302720. Throughput: 0: 9717.8, 1: 9867.2. Samples: 598274096. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:52:46,063][104569] Avg episode reward: [(0, '8994.287'), (1, '9260.575')] [2023-12-26 23:52:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001169096_299327488.pth... [2023-12-26 23:52:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001167944_299032576.pth [2023-12-26 23:52:46,090][105692] Updated weights for policy 0, policy_version 1167710 (0.0009) [2023-12-26 23:52:46,137][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001167720_298983424.pth... [2023-12-26 23:52:46,138][105692] Updated weights for policy 0, policy_version 1167720 (0.0008) [2023-12-26 23:52:46,142][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001166568_298688512.pth [2023-12-26 23:52:46,514][105620] Updated weights for policy 1, policy_version 1169100 (0.0009) [2023-12-26 23:52:46,586][105620] Updated weights for policy 1, policy_version 1169110 (0.0008) [2023-12-26 23:52:46,636][105620] Updated weights for policy 1, policy_version 1169120 (0.0009) [2023-12-26 23:52:46,968][105692] Updated weights for policy 0, policy_version 1167730 (0.0009) [2023-12-26 23:52:47,023][105692] Updated weights for policy 0, policy_version 1167740 (0.0010) [2023-12-26 23:52:47,075][105692] Updated weights for policy 0, policy_version 1167750 (0.0008) [2023-12-26 23:52:47,266][105620] Updated weights for policy 1, policy_version 1169130 (0.0008) [2023-12-26 23:52:47,327][105620] Updated weights for policy 1, policy_version 1169140 (0.0009) [2023-12-26 23:52:47,390][105620] Updated weights for policy 1, policy_version 1169150 (0.0007) [2023-12-26 23:52:47,444][105620] Updated weights for policy 1, policy_version 1169160 (0.0005) [2023-12-26 23:52:47,874][105692] Updated weights for policy 0, policy_version 1167760 (0.0008) [2023-12-26 23:52:47,920][105692] Updated weights for policy 0, policy_version 1167770 (0.0008) [2023-12-26 23:52:47,976][105692] Updated weights for policy 0, policy_version 1167780 (0.0008) [2023-12-26 23:52:48,125][105620] Updated weights for policy 1, policy_version 1169170 (0.0010) [2023-12-26 23:52:48,192][105620] Updated weights for policy 1, policy_version 1169180 (0.0010) [2023-12-26 23:52:48,254][105620] Updated weights for policy 1, policy_version 1169190 (0.0010) [2023-12-26 23:52:48,777][105692] Updated weights for policy 0, policy_version 1167790 (0.0008) [2023-12-26 23:52:48,826][105692] Updated weights for policy 0, policy_version 1167800 (0.0008) [2023-12-26 23:52:48,874][105692] Updated weights for policy 0, policy_version 1167810 (0.0008) [2023-12-26 23:52:48,998][105620] Updated weights for policy 1, policy_version 1169200 (0.0010) [2023-12-26 23:52:49,051][105620] Updated weights for policy 1, policy_version 1169210 (0.0009) [2023-12-26 23:52:49,103][105620] Updated weights for policy 1, policy_version 1169220 (0.0010) [2023-12-26 23:52:49,686][105692] Updated weights for policy 0, policy_version 1167820 (0.0009) [2023-12-26 23:52:49,735][105692] Updated weights for policy 0, policy_version 1167830 (0.0008) [2023-12-26 23:52:49,782][105620] Updated weights for policy 1, policy_version 1169230 (0.0010) [2023-12-26 23:52:49,794][105692] Updated weights for policy 0, policy_version 1167840 (0.0009) [2023-12-26 23:52:49,849][105620] Updated weights for policy 1, policy_version 1169240 (0.0013) [2023-12-26 23:52:49,916][105620] Updated weights for policy 1, policy_version 1169250 (0.0011) [2023-12-26 23:52:50,589][105620] Updated weights for policy 1, policy_version 1169260 (0.0010) [2023-12-26 23:52:50,640][105692] Updated weights for policy 0, policy_version 1167850 (0.0008) [2023-12-26 23:52:50,645][105620] Updated weights for policy 1, policy_version 1169270 (0.0009) [2023-12-26 23:52:50,700][105692] Updated weights for policy 0, policy_version 1167860 (0.0005) [2023-12-26 23:52:50,701][105620] Updated weights for policy 1, policy_version 1169280 (0.0011) [2023-12-26 23:52:50,764][105692] Updated weights for policy 0, policy_version 1167870 (0.0006) [2023-12-26 23:52:50,831][105692] Updated weights for policy 0, policy_version 1167880 (0.0009) [2023-12-26 23:52:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 598401024. Throughput: 0: 9662.5, 1: 9802.6. Samples: 598387656. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:52:51,062][104569] Avg episode reward: [(0, '9266.914'), (1, '9352.840')] [2023-12-26 23:52:51,461][105620] Updated weights for policy 1, policy_version 1169290 (0.0011) [2023-12-26 23:52:51,527][105620] Updated weights for policy 1, policy_version 1169300 (0.0011) [2023-12-26 23:52:51,592][105620] Updated weights for policy 1, policy_version 1169310 (0.0010) [2023-12-26 23:52:51,645][105692] Updated weights for policy 0, policy_version 1167890 (0.0007) [2023-12-26 23:52:51,657][105620] Updated weights for policy 1, policy_version 1169320 (0.0008) [2023-12-26 23:52:51,701][105692] Updated weights for policy 0, policy_version 1167900 (0.0009) [2023-12-26 23:52:51,763][105692] Updated weights for policy 0, policy_version 1167910 (0.0008) [2023-12-26 23:52:52,284][105620] Updated weights for policy 1, policy_version 1169330 (0.0011) [2023-12-26 23:52:52,344][105620] Updated weights for policy 1, policy_version 1169340 (0.0011) [2023-12-26 23:52:52,411][105620] Updated weights for policy 1, policy_version 1169350 (0.0010) [2023-12-26 23:52:52,548][105692] Updated weights for policy 0, policy_version 1167920 (0.0008) [2023-12-26 23:52:52,602][105692] Updated weights for policy 0, policy_version 1167930 (0.0008) [2023-12-26 23:52:52,650][105692] Updated weights for policy 0, policy_version 1167940 (0.0008) [2023-12-26 23:52:53,056][105620] Updated weights for policy 1, policy_version 1169360 (0.0009) [2023-12-26 23:52:53,104][105620] Updated weights for policy 1, policy_version 1169370 (0.0010) [2023-12-26 23:52:53,148][105620] Updated weights for policy 1, policy_version 1169380 (0.0010) [2023-12-26 23:52:53,446][105692] Updated weights for policy 0, policy_version 1167950 (0.0007) [2023-12-26 23:52:53,498][105692] Updated weights for policy 0, policy_version 1167960 (0.0008) [2023-12-26 23:52:53,551][105692] Updated weights for policy 0, policy_version 1167970 (0.0008) [2023-12-26 23:52:53,830][105620] Updated weights for policy 1, policy_version 1169390 (0.0007) [2023-12-26 23:52:53,894][105620] Updated weights for policy 1, policy_version 1169400 (0.0006) [2023-12-26 23:52:53,949][105620] Updated weights for policy 1, policy_version 1169410 (0.0010) [2023-12-26 23:52:54,451][105692] Updated weights for policy 0, policy_version 1167980 (0.0008) [2023-12-26 23:52:54,506][105692] Updated weights for policy 0, policy_version 1167990 (0.0007) [2023-12-26 23:52:54,522][105620] Updated weights for policy 1, policy_version 1169420 (0.0010) [2023-12-26 23:52:54,564][105692] Updated weights for policy 0, policy_version 1168000 (0.0005) [2023-12-26 23:52:54,585][105620] Updated weights for policy 1, policy_version 1169430 (0.0010) [2023-12-26 23:52:54,646][105620] Updated weights for policy 1, policy_version 1169440 (0.0010) [2023-12-26 23:52:55,266][105620] Updated weights for policy 1, policy_version 1169450 (0.0009) [2023-12-26 23:52:55,322][105620] Updated weights for policy 1, policy_version 1169460 (0.0009) [2023-12-26 23:52:55,367][105692] Updated weights for policy 0, policy_version 1168010 (0.0006) [2023-12-26 23:52:55,375][105620] Updated weights for policy 1, policy_version 1169470 (0.0005) [2023-12-26 23:52:55,419][105692] Updated weights for policy 0, policy_version 1168020 (0.0009) [2023-12-26 23:52:55,421][105620] Updated weights for policy 1, policy_version 1169480 (0.0005) [2023-12-26 23:52:55,471][105692] Updated weights for policy 0, policy_version 1168030 (0.0009) [2023-12-26 23:52:55,539][105692] Updated weights for policy 0, policy_version 1168040 (0.0009) [2023-12-26 23:52:56,056][105620] Updated weights for policy 1, policy_version 1169490 (0.0005) [2023-12-26 23:52:56,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 598491136. Throughput: 0: 9472.1, 1: 9914.6. Samples: 598502748. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:52:56,062][104569] Avg episode reward: [(0, '9175.695'), (1, '9353.030')] [2023-12-26 23:52:56,105][105620] Updated weights for policy 1, policy_version 1169500 (0.0005) [2023-12-26 23:52:56,156][105620] Updated weights for policy 1, policy_version 1169510 (0.0008) [2023-12-26 23:52:56,361][105692] Updated weights for policy 0, policy_version 1168050 (0.0010) [2023-12-26 23:52:56,423][105692] Updated weights for policy 0, policy_version 1168060 (0.0009) [2023-12-26 23:52:56,489][105692] Updated weights for policy 0, policy_version 1168070 (0.0010) [2023-12-26 23:52:56,829][105620] Updated weights for policy 1, policy_version 1169520 (0.0009) [2023-12-26 23:52:56,886][105620] Updated weights for policy 1, policy_version 1169530 (0.0009) [2023-12-26 23:52:56,939][105620] Updated weights for policy 1, policy_version 1169540 (0.0008) [2023-12-26 23:52:57,281][105692] Updated weights for policy 0, policy_version 1168080 (0.0009) [2023-12-26 23:52:57,337][105692] Updated weights for policy 0, policy_version 1168090 (0.0009) [2023-12-26 23:52:57,388][105692] Updated weights for policy 0, policy_version 1168100 (0.0009) [2023-12-26 23:52:57,607][105620] Updated weights for policy 1, policy_version 1169550 (0.0008) [2023-12-26 23:52:57,664][105620] Updated weights for policy 1, policy_version 1169560 (0.0009) [2023-12-26 23:52:57,729][105620] Updated weights for policy 1, policy_version 1169570 (0.0009) [2023-12-26 23:52:58,174][105692] Updated weights for policy 0, policy_version 1168110 (0.0008) [2023-12-26 23:52:58,240][105692] Updated weights for policy 0, policy_version 1168120 (0.0009) [2023-12-26 23:52:58,302][105692] Updated weights for policy 0, policy_version 1168130 (0.0008) [2023-12-26 23:52:58,489][105620] Updated weights for policy 1, policy_version 1169580 (0.0009) [2023-12-26 23:52:58,547][105620] Updated weights for policy 1, policy_version 1169590 (0.0009) [2023-12-26 23:52:58,613][105620] Updated weights for policy 1, policy_version 1169600 (0.0008) [2023-12-26 23:52:59,062][105692] Updated weights for policy 0, policy_version 1168140 (0.0009) [2023-12-26 23:52:59,117][105692] Updated weights for policy 0, policy_version 1168150 (0.0008) [2023-12-26 23:52:59,181][105692] Updated weights for policy 0, policy_version 1168160 (0.0009) [2023-12-26 23:52:59,515][105620] Updated weights for policy 1, policy_version 1169610 (0.0008) [2023-12-26 23:52:59,566][105620] Updated weights for policy 1, policy_version 1169620 (0.0009) [2023-12-26 23:52:59,633][105620] Updated weights for policy 1, policy_version 1169630 (0.0009) [2023-12-26 23:52:59,691][105620] Updated weights for policy 1, policy_version 1169640 (0.0009) [2023-12-26 23:52:59,998][105692] Updated weights for policy 0, policy_version 1168170 (0.0009) [2023-12-26 23:53:00,063][105692] Updated weights for policy 0, policy_version 1168180 (0.0009) [2023-12-26 23:53:00,117][105692] Updated weights for policy 0, policy_version 1168190 (0.0010) [2023-12-26 23:53:00,171][105692] Updated weights for policy 0, policy_version 1168200 (0.0009) [2023-12-26 23:53:00,374][105620] Updated weights for policy 1, policy_version 1169650 (0.0009) [2023-12-26 23:53:00,432][105620] Updated weights for policy 1, policy_version 1169660 (0.0007) [2023-12-26 23:53:00,489][105620] Updated weights for policy 1, policy_version 1169670 (0.0005) [2023-12-26 23:53:00,903][105692] Updated weights for policy 0, policy_version 1168210 (0.0007) [2023-12-26 23:53:00,955][105692] Updated weights for policy 0, policy_version 1168220 (0.0008) [2023-12-26 23:53:01,011][105692] Updated weights for policy 0, policy_version 1168230 (0.0008) [2023-12-26 23:53:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 598589440. Throughput: 0: 9465.4, 1: 9858.5. Samples: 598558412. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:53:01,063][104569] Avg episode reward: [(0, '9265.794'), (1, '9170.977')] [2023-12-26 23:53:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001168232_299114496.pth... [2023-12-26 23:53:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001169672_299474944.pth... [2023-12-26 23:53:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001167144_298835968.pth [2023-12-26 23:53:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001168488_299171840.pth [2023-12-26 23:53:01,216][105620] Updated weights for policy 1, policy_version 1169680 (0.0007) [2023-12-26 23:53:01,274][105620] Updated weights for policy 1, policy_version 1169690 (0.0010) [2023-12-26 23:53:01,340][105620] Updated weights for policy 1, policy_version 1169700 (0.0010) [2023-12-26 23:53:01,745][105692] Updated weights for policy 0, policy_version 1168240 (0.0008) [2023-12-26 23:53:01,800][105692] Updated weights for policy 0, policy_version 1168250 (0.0009) [2023-12-26 23:53:01,862][105692] Updated weights for policy 0, policy_version 1168260 (0.0010) [2023-12-26 23:53:02,015][105620] Updated weights for policy 1, policy_version 1169710 (0.0006) [2023-12-26 23:53:02,073][105620] Updated weights for policy 1, policy_version 1169720 (0.0005) [2023-12-26 23:53:02,127][105620] Updated weights for policy 1, policy_version 1169730 (0.0005) [2023-12-26 23:53:02,697][105692] Updated weights for policy 0, policy_version 1168270 (0.0009) [2023-12-26 23:53:02,753][105692] Updated weights for policy 0, policy_version 1168280 (0.0009) [2023-12-26 23:53:02,767][105620] Updated weights for policy 1, policy_version 1169740 (0.0006) [2023-12-26 23:53:02,802][105692] Updated weights for policy 0, policy_version 1168290 (0.0006) [2023-12-26 23:53:02,823][105620] Updated weights for policy 1, policy_version 1169750 (0.0009) [2023-12-26 23:53:02,889][105620] Updated weights for policy 1, policy_version 1169760 (0.0008) [2023-12-26 23:53:03,553][105692] Updated weights for policy 0, policy_version 1168300 (0.0009) [2023-12-26 23:53:03,606][105692] Updated weights for policy 0, policy_version 1168310 (0.0009) [2023-12-26 23:53:03,629][105620] Updated weights for policy 1, policy_version 1169770 (0.0008) [2023-12-26 23:53:03,658][105692] Updated weights for policy 0, policy_version 1168320 (0.0009) [2023-12-26 23:53:03,685][105620] Updated weights for policy 1, policy_version 1169780 (0.0008) [2023-12-26 23:53:03,747][105620] Updated weights for policy 1, policy_version 1169790 (0.0008) [2023-12-26 23:53:03,807][105620] Updated weights for policy 1, policy_version 1169800 (0.0009) [2023-12-26 23:53:04,386][105692] Updated weights for policy 0, policy_version 1168330 (0.0007) [2023-12-26 23:53:04,435][105692] Updated weights for policy 0, policy_version 1168340 (0.0011) [2023-12-26 23:53:04,487][105692] Updated weights for policy 0, policy_version 1168350 (0.0011) [2023-12-26 23:53:04,539][105692] Updated weights for policy 0, policy_version 1168360 (0.0010) [2023-12-26 23:53:04,566][105620] Updated weights for policy 1, policy_version 1169810 (0.0008) [2023-12-26 23:53:04,620][105620] Updated weights for policy 1, policy_version 1169820 (0.0008) [2023-12-26 23:53:04,667][105620] Updated weights for policy 1, policy_version 1169830 (0.0008) [2023-12-26 23:53:05,248][105692] Updated weights for policy 0, policy_version 1168370 (0.0010) [2023-12-26 23:53:05,303][105692] Updated weights for policy 0, policy_version 1168380 (0.0010) [2023-12-26 23:53:05,360][105692] Updated weights for policy 0, policy_version 1168390 (0.0010) [2023-12-26 23:53:05,424][105620] Updated weights for policy 1, policy_version 1169840 (0.0008) [2023-12-26 23:53:05,475][105620] Updated weights for policy 1, policy_version 1169850 (0.0008) [2023-12-26 23:53:05,523][105620] Updated weights for policy 1, policy_version 1169860 (0.0008) [2023-12-26 23:53:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19438.7). Total num frames: 598679552. Throughput: 0: 9413.2, 1: 9840.6. Samples: 598671436. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:53:06,062][104569] Avg episode reward: [(0, '9356.924'), (1, '8987.792')] [2023-12-26 23:53:06,115][105692] Updated weights for policy 0, policy_version 1168400 (0.0010) [2023-12-26 23:53:06,187][105692] Updated weights for policy 0, policy_version 1168410 (0.0007) [2023-12-26 23:53:06,259][105692] Updated weights for policy 0, policy_version 1168420 (0.0005) [2023-12-26 23:53:06,274][105620] Updated weights for policy 1, policy_version 1169870 (0.0007) [2023-12-26 23:53:06,346][105620] Updated weights for policy 1, policy_version 1169880 (0.0006) [2023-12-26 23:53:06,419][105620] Updated weights for policy 1, policy_version 1169890 (0.0006) [2023-12-26 23:53:06,815][105692] Updated weights for policy 0, policy_version 1168430 (0.0007) [2023-12-26 23:53:06,874][105692] Updated weights for policy 0, policy_version 1168440 (0.0010) [2023-12-26 23:53:06,934][105692] Updated weights for policy 0, policy_version 1168450 (0.0011) [2023-12-26 23:53:07,022][105620] Updated weights for policy 1, policy_version 1169900 (0.0007) [2023-12-26 23:53:07,075][105620] Updated weights for policy 1, policy_version 1169910 (0.0008) [2023-12-26 23:53:07,136][105620] Updated weights for policy 1, policy_version 1169920 (0.0008) [2023-12-26 23:53:07,608][105692] Updated weights for policy 0, policy_version 1168460 (0.0010) [2023-12-26 23:53:07,656][105692] Updated weights for policy 0, policy_version 1168470 (0.0007) [2023-12-26 23:53:07,722][105692] Updated weights for policy 0, policy_version 1168480 (0.0007) [2023-12-26 23:53:07,936][105620] Updated weights for policy 1, policy_version 1169930 (0.0007) [2023-12-26 23:53:07,986][105620] Updated weights for policy 1, policy_version 1169940 (0.0005) [2023-12-26 23:53:08,051][105620] Updated weights for policy 1, policy_version 1169950 (0.0009) [2023-12-26 23:53:08,108][105620] Updated weights for policy 1, policy_version 1169960 (0.0009) [2023-12-26 23:53:08,454][105692] Updated weights for policy 0, policy_version 1168490 (0.0008) [2023-12-26 23:53:08,503][105692] Updated weights for policy 0, policy_version 1168500 (0.0009) [2023-12-26 23:53:08,557][105692] Updated weights for policy 0, policy_version 1168510 (0.0009) [2023-12-26 23:53:08,620][105692] Updated weights for policy 0, policy_version 1168520 (0.0009) [2023-12-26 23:53:08,796][105620] Updated weights for policy 1, policy_version 1169970 (0.0010) [2023-12-26 23:53:08,867][105620] Updated weights for policy 1, policy_version 1169980 (0.0009) [2023-12-26 23:53:08,936][105620] Updated weights for policy 1, policy_version 1169990 (0.0009) [2023-12-26 23:53:09,329][105692] Updated weights for policy 0, policy_version 1168530 (0.0007) [2023-12-26 23:53:09,401][105692] Updated weights for policy 0, policy_version 1168540 (0.0008) [2023-12-26 23:53:09,472][105692] Updated weights for policy 0, policy_version 1168550 (0.0009) [2023-12-26 23:53:09,710][105620] Updated weights for policy 1, policy_version 1170000 (0.0010) [2023-12-26 23:53:09,767][105620] Updated weights for policy 1, policy_version 1170010 (0.0010) [2023-12-26 23:53:09,830][105620] Updated weights for policy 1, policy_version 1170020 (0.0009) [2023-12-26 23:53:10,186][105692] Updated weights for policy 0, policy_version 1168560 (0.0009) [2023-12-26 23:53:10,234][105692] Updated weights for policy 0, policy_version 1168570 (0.0008) [2023-12-26 23:53:10,282][105692] Updated weights for policy 0, policy_version 1168580 (0.0009) [2023-12-26 23:53:10,595][105620] Updated weights for policy 1, policy_version 1170030 (0.0008) [2023-12-26 23:53:10,649][105620] Updated weights for policy 1, policy_version 1170040 (0.0009) [2023-12-26 23:53:10,711][105620] Updated weights for policy 1, policy_version 1170050 (0.0008) [2023-12-26 23:53:11,027][105692] Updated weights for policy 0, policy_version 1168590 (0.0009) [2023-12-26 23:53:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 598777856. Throughput: 0: 9475.8, 1: 9878.1. Samples: 598787600. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:53:11,063][104569] Avg episode reward: [(0, '9266.051'), (1, '8849.318')] [2023-12-26 23:53:11,087][105692] Updated weights for policy 0, policy_version 1168600 (0.0009) [2023-12-26 23:53:11,150][105692] Updated weights for policy 0, policy_version 1168610 (0.0008) [2023-12-26 23:53:11,550][105620] Updated weights for policy 1, policy_version 1170060 (0.0008) [2023-12-26 23:53:11,612][105620] Updated weights for policy 1, policy_version 1170070 (0.0009) [2023-12-26 23:53:11,678][105620] Updated weights for policy 1, policy_version 1170080 (0.0008) [2023-12-26 23:53:11,931][105692] Updated weights for policy 0, policy_version 1168620 (0.0008) [2023-12-26 23:53:11,995][105692] Updated weights for policy 0, policy_version 1168630 (0.0010) [2023-12-26 23:53:12,055][105692] Updated weights for policy 0, policy_version 1168640 (0.0009) [2023-12-26 23:53:12,462][105620] Updated weights for policy 1, policy_version 1170090 (0.0008) [2023-12-26 23:53:12,526][105620] Updated weights for policy 1, policy_version 1170100 (0.0006) [2023-12-26 23:53:12,583][105620] Updated weights for policy 1, policy_version 1170110 (0.0009) [2023-12-26 23:53:12,643][105620] Updated weights for policy 1, policy_version 1170120 (0.0009) [2023-12-26 23:53:12,727][105692] Updated weights for policy 0, policy_version 1168650 (0.0008) [2023-12-26 23:53:12,775][105692] Updated weights for policy 0, policy_version 1168660 (0.0010) [2023-12-26 23:53:12,828][105692] Updated weights for policy 0, policy_version 1168670 (0.0008) [2023-12-26 23:53:12,880][105692] Updated weights for policy 0, policy_version 1168680 (0.0005) [2023-12-26 23:53:13,406][105620] Updated weights for policy 1, policy_version 1170131 (0.0007) [2023-12-26 23:53:13,463][105620] Updated weights for policy 1, policy_version 1170141 (0.0006) [2023-12-26 23:53:13,511][105692] Updated weights for policy 0, policy_version 1168690 (0.0005) [2023-12-26 23:53:13,516][105620] Updated weights for policy 1, policy_version 1170151 (0.0009) [2023-12-26 23:53:13,573][105692] Updated weights for policy 0, policy_version 1168700 (0.0005) [2023-12-26 23:53:13,638][105692] Updated weights for policy 0, policy_version 1168710 (0.0007) [2023-12-26 23:53:14,249][105620] Updated weights for policy 1, policy_version 1170161 (0.0010) [2023-12-26 23:53:14,309][105620] Updated weights for policy 1, policy_version 1170171 (0.0009) [2023-12-26 23:53:14,311][105692] Updated weights for policy 0, policy_version 1168720 (0.0006) [2023-12-26 23:53:14,358][105692] Updated weights for policy 0, policy_version 1168730 (0.0008) [2023-12-26 23:53:14,366][105620] Updated weights for policy 1, policy_version 1170181 (0.0007) [2023-12-26 23:53:14,408][105692] Updated weights for policy 0, policy_version 1168740 (0.0008) [2023-12-26 23:53:15,071][105692] Updated weights for policy 0, policy_version 1168750 (0.0007) [2023-12-26 23:53:15,119][105692] Updated weights for policy 0, policy_version 1168760 (0.0009) [2023-12-26 23:53:15,176][105692] Updated weights for policy 0, policy_version 1168770 (0.0008) [2023-12-26 23:53:15,178][105620] Updated weights for policy 1, policy_version 1170191 (0.0007) [2023-12-26 23:53:15,237][105620] Updated weights for policy 1, policy_version 1170201 (0.0006) [2023-12-26 23:53:15,302][105620] Updated weights for policy 1, policy_version 1170211 (0.0009) [2023-12-26 23:53:15,966][105692] Updated weights for policy 0, policy_version 1168780 (0.0008) [2023-12-26 23:53:16,024][105692] Updated weights for policy 0, policy_version 1168790 (0.0009) [2023-12-26 23:53:16,055][105620] Updated weights for policy 1, policy_version 1170221 (0.0009) [2023-12-26 23:53:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19114.7, 300 sec: 19410.9). Total num frames: 598867968. Throughput: 0: 9418.1, 1: 9900.5. Samples: 598846036. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:53:16,062][104569] Avg episode reward: [(0, '9172.680'), (1, '8260.490')] [2023-12-26 23:53:16,081][105692] Updated weights for policy 0, policy_version 1168800 (0.0007) [2023-12-26 23:53:16,110][105620] Updated weights for policy 1, policy_version 1170231 (0.0007) [2023-12-26 23:53:16,122][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001168808_299261952.pth... [2023-12-26 23:53:16,126][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001167720_298983424.pth [2023-12-26 23:53:16,166][105620] Updated weights for policy 1, policy_version 1170241 (0.0008) [2023-12-26 23:53:16,204][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001170248_299622400.pth... [2023-12-26 23:53:16,208][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001169096_299327488.pth [2023-12-26 23:53:16,721][105620] Updated weights for policy 1, policy_version 1170251 (0.0005) [2023-12-26 23:53:16,775][105620] Updated weights for policy 1, policy_version 1170261 (0.0005) [2023-12-26 23:53:16,825][105620] Updated weights for policy 1, policy_version 1170271 (0.0005) [2023-12-26 23:53:16,972][105692] Updated weights for policy 0, policy_version 1168810 (0.0007) [2023-12-26 23:53:17,027][105692] Updated weights for policy 0, policy_version 1168820 (0.0009) [2023-12-26 23:53:17,089][105692] Updated weights for policy 0, policy_version 1168830 (0.0009) [2023-12-26 23:53:17,137][105692] Updated weights for policy 0, policy_version 1168840 (0.0009) [2023-12-26 23:53:17,493][105620] Updated weights for policy 1, policy_version 1170281 (0.0005) [2023-12-26 23:53:17,559][105620] Updated weights for policy 1, policy_version 1170291 (0.0007) [2023-12-26 23:53:17,618][105620] Updated weights for policy 1, policy_version 1170301 (0.0010) [2023-12-26 23:53:17,672][105620] Updated weights for policy 1, policy_version 1170311 (0.0009) [2023-12-26 23:53:17,874][105692] Updated weights for policy 0, policy_version 1168850 (0.0008) [2023-12-26 23:53:17,923][105692] Updated weights for policy 0, policy_version 1168860 (0.0005) [2023-12-26 23:53:17,979][105692] Updated weights for policy 0, policy_version 1168870 (0.0009) [2023-12-26 23:53:18,374][105620] Updated weights for policy 1, policy_version 1170321 (0.0007) [2023-12-26 23:53:18,427][105620] Updated weights for policy 1, policy_version 1170331 (0.0006) [2023-12-26 23:53:18,485][105620] Updated weights for policy 1, policy_version 1170341 (0.0005) [2023-12-26 23:53:18,763][105692] Updated weights for policy 0, policy_version 1168880 (0.0009) [2023-12-26 23:53:18,822][105692] Updated weights for policy 0, policy_version 1168890 (0.0009) [2023-12-26 23:53:18,885][105692] Updated weights for policy 0, policy_version 1168900 (0.0009) [2023-12-26 23:53:19,077][105620] Updated weights for policy 1, policy_version 1170351 (0.0005) [2023-12-26 23:53:19,131][105620] Updated weights for policy 1, policy_version 1170361 (0.0005) [2023-12-26 23:53:19,180][105620] Updated weights for policy 1, policy_version 1170371 (0.0005) [2023-12-26 23:53:19,564][105692] Updated weights for policy 0, policy_version 1168910 (0.0007) [2023-12-26 23:53:19,622][105692] Updated weights for policy 0, policy_version 1168920 (0.0006) [2023-12-26 23:53:19,676][105692] Updated weights for policy 0, policy_version 1168930 (0.0006) [2023-12-26 23:53:19,888][105620] Updated weights for policy 1, policy_version 1170381 (0.0007) [2023-12-26 23:53:19,954][105620] Updated weights for policy 1, policy_version 1170391 (0.0008) [2023-12-26 23:53:20,014][105620] Updated weights for policy 1, policy_version 1170401 (0.0009) [2023-12-26 23:53:20,441][105692] Updated weights for policy 0, policy_version 1168940 (0.0008) [2023-12-26 23:53:20,506][105692] Updated weights for policy 0, policy_version 1168950 (0.0011) [2023-12-26 23:53:20,572][105692] Updated weights for policy 0, policy_version 1168960 (0.0010) [2023-12-26 23:53:20,699][105620] Updated weights for policy 1, policy_version 1170411 (0.0009) [2023-12-26 23:53:20,753][105620] Updated weights for policy 1, policy_version 1170421 (0.0009) [2023-12-26 23:53:20,825][105620] Updated weights for policy 1, policy_version 1170431 (0.0009) [2023-12-26 23:53:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19438.7). Total num frames: 598974464. Throughput: 0: 9316.0, 1: 9943.7. Samples: 598962992. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:53:21,062][104569] Avg episode reward: [(0, '8994.592'), (1, '8580.600')] [2023-12-26 23:53:21,353][105692] Updated weights for policy 0, policy_version 1168970 (0.0009) [2023-12-26 23:53:21,422][105692] Updated weights for policy 0, policy_version 1168980 (0.0008) [2023-12-26 23:53:21,472][105692] Updated weights for policy 0, policy_version 1168990 (0.0009) [2023-12-26 23:53:21,529][105692] Updated weights for policy 0, policy_version 1169000 (0.0008) [2023-12-26 23:53:21,605][105620] Updated weights for policy 1, policy_version 1170441 (0.0006) [2023-12-26 23:53:21,666][105620] Updated weights for policy 1, policy_version 1170451 (0.0011) [2023-12-26 23:53:21,719][105620] Updated weights for policy 1, policy_version 1170461 (0.0011) [2023-12-26 23:53:21,784][105620] Updated weights for policy 1, policy_version 1170471 (0.0010) [2023-12-26 23:53:22,337][105692] Updated weights for policy 0, policy_version 1169010 (0.0008) [2023-12-26 23:53:22,408][105692] Updated weights for policy 0, policy_version 1169020 (0.0008) [2023-12-26 23:53:22,474][105692] Updated weights for policy 0, policy_version 1169030 (0.0008) [2023-12-26 23:53:22,588][105620] Updated weights for policy 1, policy_version 1170481 (0.0010) [2023-12-26 23:53:22,658][105620] Updated weights for policy 1, policy_version 1170491 (0.0009) [2023-12-26 23:53:22,719][105620] Updated weights for policy 1, policy_version 1170501 (0.0009) [2023-12-26 23:53:23,174][105692] Updated weights for policy 0, policy_version 1169040 (0.0009) [2023-12-26 23:53:23,223][105692] Updated weights for policy 0, policy_version 1169050 (0.0009) [2023-12-26 23:53:23,271][105692] Updated weights for policy 0, policy_version 1169060 (0.0009) [2023-12-26 23:53:23,470][105620] Updated weights for policy 1, policy_version 1170511 (0.0009) [2023-12-26 23:53:23,525][105620] Updated weights for policy 1, policy_version 1170521 (0.0010) [2023-12-26 23:53:23,583][105620] Updated weights for policy 1, policy_version 1170531 (0.0010) [2023-12-26 23:53:23,988][105692] Updated weights for policy 0, policy_version 1169070 (0.0008) [2023-12-26 23:53:24,039][105692] Updated weights for policy 0, policy_version 1169080 (0.0007) [2023-12-26 23:53:24,106][105692] Updated weights for policy 0, policy_version 1169090 (0.0006) [2023-12-26 23:53:24,254][105620] Updated weights for policy 1, policy_version 1170541 (0.0010) [2023-12-26 23:53:24,300][105620] Updated weights for policy 1, policy_version 1170551 (0.0010) [2023-12-26 23:53:24,349][105620] Updated weights for policy 1, policy_version 1170561 (0.0010) [2023-12-26 23:53:24,776][105692] Updated weights for policy 0, policy_version 1169100 (0.0009) [2023-12-26 23:53:24,829][105692] Updated weights for policy 0, policy_version 1169110 (0.0008) [2023-12-26 23:53:24,878][105692] Updated weights for policy 0, policy_version 1169120 (0.0008) [2023-12-26 23:53:25,103][105620] Updated weights for policy 1, policy_version 1170571 (0.0009) [2023-12-26 23:53:25,154][105620] Updated weights for policy 1, policy_version 1170581 (0.0005) [2023-12-26 23:53:25,212][105620] Updated weights for policy 1, policy_version 1170591 (0.0006) [2023-12-26 23:53:25,624][105692] Updated weights for policy 0, policy_version 1169130 (0.0008) [2023-12-26 23:53:25,685][105692] Updated weights for policy 0, policy_version 1169140 (0.0005) [2023-12-26 23:53:25,728][105692] Updated weights for policy 0, policy_version 1169150 (0.0005) [2023-12-26 23:53:25,772][105692] Updated weights for policy 0, policy_version 1169160 (0.0005) [2023-12-26 23:53:25,888][105620] Updated weights for policy 1, policy_version 1170601 (0.0005) [2023-12-26 23:53:25,943][105620] Updated weights for policy 1, policy_version 1170611 (0.0008) [2023-12-26 23:53:25,999][105620] Updated weights for policy 1, policy_version 1170621 (0.0011) [2023-12-26 23:53:26,051][105620] Updated weights for policy 1, policy_version 1170631 (0.0011) [2023-12-26 23:53:26,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 599072768. Throughput: 0: 9327.1, 1: 9825.0. Samples: 599077240. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:53:26,062][104569] Avg episode reward: [(0, '8904.163'), (1, '9086.620')] [2023-12-26 23:53:26,400][105692] Updated weights for policy 0, policy_version 1169170 (0.0006) [2023-12-26 23:53:26,462][105692] Updated weights for policy 0, policy_version 1169180 (0.0005) [2023-12-26 23:53:26,515][105692] Updated weights for policy 0, policy_version 1169190 (0.0005) [2023-12-26 23:53:26,763][105620] Updated weights for policy 1, policy_version 1170641 (0.0006) [2023-12-26 23:53:26,811][105620] Updated weights for policy 1, policy_version 1170651 (0.0010) [2023-12-26 23:53:26,858][105620] Updated weights for policy 1, policy_version 1170661 (0.0010) [2023-12-26 23:53:27,091][105692] Updated weights for policy 0, policy_version 1169200 (0.0005) [2023-12-26 23:53:27,159][105692] Updated weights for policy 0, policy_version 1169210 (0.0005) [2023-12-26 23:53:27,215][105692] Updated weights for policy 0, policy_version 1169220 (0.0005) [2023-12-26 23:53:27,447][105620] Updated weights for policy 1, policy_version 1170671 (0.0007) [2023-12-26 23:53:27,493][105620] Updated weights for policy 1, policy_version 1170681 (0.0006) [2023-12-26 23:53:27,539][105620] Updated weights for policy 1, policy_version 1170691 (0.0007) [2023-12-26 23:53:27,707][105692] Updated weights for policy 0, policy_version 1169230 (0.0007) [2023-12-26 23:53:27,761][105692] Updated weights for policy 0, policy_version 1169240 (0.0009) [2023-12-26 23:53:27,814][105692] Updated weights for policy 0, policy_version 1169250 (0.0010) [2023-12-26 23:53:28,111][105620] Updated weights for policy 1, policy_version 1170701 (0.0008) [2023-12-26 23:53:28,158][105620] Updated weights for policy 1, policy_version 1170711 (0.0006) [2023-12-26 23:53:28,206][105620] Updated weights for policy 1, policy_version 1170721 (0.0010) [2023-12-26 23:53:28,688][105692] Updated weights for policy 0, policy_version 1169260 (0.0010) [2023-12-26 23:53:28,748][105692] Updated weights for policy 0, policy_version 1169270 (0.0008) [2023-12-26 23:53:28,810][105692] Updated weights for policy 0, policy_version 1169280 (0.0008) [2023-12-26 23:53:28,953][105620] Updated weights for policy 1, policy_version 1170731 (0.0009) [2023-12-26 23:53:29,013][105620] Updated weights for policy 1, policy_version 1170741 (0.0005) [2023-12-26 23:53:29,061][105620] Updated weights for policy 1, policy_version 1170751 (0.0005) [2023-12-26 23:53:29,567][105692] Updated weights for policy 0, policy_version 1169290 (0.0007) [2023-12-26 23:53:29,616][105692] Updated weights for policy 0, policy_version 1169300 (0.0008) [2023-12-26 23:53:29,660][105692] Updated weights for policy 0, policy_version 1169310 (0.0008) [2023-12-26 23:53:29,697][105620] Updated weights for policy 1, policy_version 1170761 (0.0006) [2023-12-26 23:53:29,708][105692] Updated weights for policy 0, policy_version 1169320 (0.0007) [2023-12-26 23:53:29,752][105620] Updated weights for policy 1, policy_version 1170771 (0.0010) [2023-12-26 23:53:29,803][105620] Updated weights for policy 1, policy_version 1170781 (0.0010) [2023-12-26 23:53:29,871][105620] Updated weights for policy 1, policy_version 1170792 (0.0011) [2023-12-26 23:53:30,491][105692] Updated weights for policy 0, policy_version 1169330 (0.0005) [2023-12-26 23:53:30,545][105692] Updated weights for policy 0, policy_version 1169340 (0.0005) [2023-12-26 23:53:30,574][105620] Updated weights for policy 1, policy_version 1170802 (0.0005) [2023-12-26 23:53:30,600][105692] Updated weights for policy 0, policy_version 1169350 (0.0008) [2023-12-26 23:53:30,629][105620] Updated weights for policy 1, policy_version 1170812 (0.0007) [2023-12-26 23:53:30,676][105620] Updated weights for policy 1, policy_version 1170822 (0.0008) [2023-12-26 23:53:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 599171072. Throughput: 0: 9450.9, 1: 9841.4. Samples: 599142244. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:53:31,062][104569] Avg episode reward: [(0, '8995.363'), (1, '8813.878')] [2023-12-26 23:53:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001169352_299401216.pth... [2023-12-26 23:53:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001170824_299769856.pth... [2023-12-26 23:53:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001168232_299114496.pth [2023-12-26 23:53:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001169672_299474944.pth [2023-12-26 23:53:31,331][105692] Updated weights for policy 0, policy_version 1169360 (0.0008) [2023-12-26 23:53:31,393][105692] Updated weights for policy 0, policy_version 1169370 (0.0008) [2023-12-26 23:53:31,404][105620] Updated weights for policy 1, policy_version 1170832 (0.0009) [2023-12-26 23:53:31,451][105692] Updated weights for policy 0, policy_version 1169380 (0.0008) [2023-12-26 23:53:31,466][105620] Updated weights for policy 1, policy_version 1170842 (0.0007) [2023-12-26 23:53:31,524][105620] Updated weights for policy 1, policy_version 1170852 (0.0008) [2023-12-26 23:53:32,202][105692] Updated weights for policy 0, policy_version 1169390 (0.0008) [2023-12-26 23:53:32,237][105620] Updated weights for policy 1, policy_version 1170862 (0.0006) [2023-12-26 23:53:32,263][105692] Updated weights for policy 0, policy_version 1169400 (0.0008) [2023-12-26 23:53:32,300][105620] Updated weights for policy 1, policy_version 1170872 (0.0007) [2023-12-26 23:53:32,326][105692] Updated weights for policy 0, policy_version 1169410 (0.0008) [2023-12-26 23:53:32,369][105620] Updated weights for policy 1, policy_version 1170882 (0.0006) [2023-12-26 23:53:32,886][105692] Updated weights for policy 0, policy_version 1169420 (0.0006) [2023-12-26 23:53:32,930][105692] Updated weights for policy 0, policy_version 1169430 (0.0005) [2023-12-26 23:53:32,981][105692] Updated weights for policy 0, policy_version 1169440 (0.0005) [2023-12-26 23:53:33,116][105620] Updated weights for policy 1, policy_version 1170892 (0.0009) [2023-12-26 23:53:33,170][105620] Updated weights for policy 1, policy_version 1170902 (0.0009) [2023-12-26 23:53:33,218][105620] Updated weights for policy 1, policy_version 1170912 (0.0009) [2023-12-26 23:53:33,648][105692] Updated weights for policy 0, policy_version 1169450 (0.0006) [2023-12-26 23:53:33,702][105692] Updated weights for policy 0, policy_version 1169461 (0.0010) [2023-12-26 23:53:33,751][105692] Updated weights for policy 0, policy_version 1169471 (0.0007) [2023-12-26 23:53:33,912][105620] Updated weights for policy 1, policy_version 1170922 (0.0009) [2023-12-26 23:53:33,980][105620] Updated weights for policy 1, policy_version 1170932 (0.0005) [2023-12-26 23:53:34,029][105620] Updated weights for policy 1, policy_version 1170942 (0.0005) [2023-12-26 23:53:34,073][105620] Updated weights for policy 1, policy_version 1170952 (0.0005) [2023-12-26 23:53:34,346][105692] Updated weights for policy 0, policy_version 1169481 (0.0005) [2023-12-26 23:53:34,412][105692] Updated weights for policy 0, policy_version 1169491 (0.0011) [2023-12-26 23:53:34,473][105692] Updated weights for policy 0, policy_version 1169501 (0.0011) [2023-12-26 23:53:34,536][105692] Updated weights for policy 0, policy_version 1169511 (0.0010) [2023-12-26 23:53:34,767][105620] Updated weights for policy 1, policy_version 1170962 (0.0008) [2023-12-26 23:53:34,832][105620] Updated weights for policy 1, policy_version 1170972 (0.0008) [2023-12-26 23:53:34,885][105620] Updated weights for policy 1, policy_version 1170982 (0.0008) [2023-12-26 23:53:35,264][105692] Updated weights for policy 0, policy_version 1169521 (0.0010) [2023-12-26 23:53:35,319][105692] Updated weights for policy 0, policy_version 1169531 (0.0010) [2023-12-26 23:53:35,374][105692] Updated weights for policy 0, policy_version 1169541 (0.0010) [2023-12-26 23:53:35,686][105620] Updated weights for policy 1, policy_version 1170992 (0.0008) [2023-12-26 23:53:35,735][105620] Updated weights for policy 1, policy_version 1171002 (0.0008) [2023-12-26 23:53:35,779][105620] Updated weights for policy 1, policy_version 1171012 (0.0007) [2023-12-26 23:53:36,027][105692] Updated weights for policy 0, policy_version 1169551 (0.0007) [2023-12-26 23:53:36,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 599269376. Throughput: 0: 9546.9, 1: 9864.4. Samples: 599261168. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:53:36,063][104569] Avg episode reward: [(0, '8994.943'), (1, '8988.003')] [2023-12-26 23:53:36,084][105692] Updated weights for policy 0, policy_version 1169561 (0.0006) [2023-12-26 23:53:36,143][105692] Updated weights for policy 0, policy_version 1169571 (0.0008) [2023-12-26 23:53:36,666][105620] Updated weights for policy 1, policy_version 1171022 (0.0009) [2023-12-26 23:53:36,743][105620] Updated weights for policy 1, policy_version 1171032 (0.0008) [2023-12-26 23:53:36,754][105692] Updated weights for policy 0, policy_version 1169581 (0.0007) [2023-12-26 23:53:36,794][105620] Updated weights for policy 1, policy_version 1171042 (0.0007) [2023-12-26 23:53:36,807][105692] Updated weights for policy 0, policy_version 1169591 (0.0005) [2023-12-26 23:53:36,857][105692] Updated weights for policy 0, policy_version 1169601 (0.0006) [2023-12-26 23:53:37,436][105692] Updated weights for policy 0, policy_version 1169611 (0.0009) [2023-12-26 23:53:37,484][105692] Updated weights for policy 0, policy_version 1169621 (0.0005) [2023-12-26 23:53:37,543][105692] Updated weights for policy 0, policy_version 1169631 (0.0006) [2023-12-26 23:53:37,630][105620] Updated weights for policy 1, policy_version 1171052 (0.0008) [2023-12-26 23:53:37,689][105620] Updated weights for policy 1, policy_version 1171062 (0.0005) [2023-12-26 23:53:37,748][105620] Updated weights for policy 1, policy_version 1171072 (0.0006) [2023-12-26 23:53:38,160][105692] Updated weights for policy 0, policy_version 1169641 (0.0006) [2023-12-26 23:53:38,221][105692] Updated weights for policy 0, policy_version 1169651 (0.0010) [2023-12-26 23:53:38,272][105692] Updated weights for policy 0, policy_version 1169661 (0.0010) [2023-12-26 23:53:38,319][105692] Updated weights for policy 0, policy_version 1169671 (0.0010) [2023-12-26 23:53:38,428][105620] Updated weights for policy 1, policy_version 1171082 (0.0006) [2023-12-26 23:53:38,487][105620] Updated weights for policy 1, policy_version 1171092 (0.0010) [2023-12-26 23:53:38,538][105620] Updated weights for policy 1, policy_version 1171102 (0.0005) [2023-12-26 23:53:38,599][105620] Updated weights for policy 1, policy_version 1171112 (0.0006) [2023-12-26 23:53:39,044][105692] Updated weights for policy 0, policy_version 1169681 (0.0010) [2023-12-26 23:53:39,105][105692] Updated weights for policy 0, policy_version 1169691 (0.0010) [2023-12-26 23:53:39,165][105692] Updated weights for policy 0, policy_version 1169701 (0.0010) [2023-12-26 23:53:39,316][105620] Updated weights for policy 1, policy_version 1171122 (0.0008) [2023-12-26 23:53:39,388][105620] Updated weights for policy 1, policy_version 1171132 (0.0008) [2023-12-26 23:53:39,449][105620] Updated weights for policy 1, policy_version 1171143 (0.0007) [2023-12-26 23:53:39,922][105692] Updated weights for policy 0, policy_version 1169711 (0.0010) [2023-12-26 23:53:39,987][105692] Updated weights for policy 0, policy_version 1169721 (0.0006) [2023-12-26 23:53:40,059][105692] Updated weights for policy 0, policy_version 1169731 (0.0006) [2023-12-26 23:53:40,141][105620] Updated weights for policy 1, policy_version 1171153 (0.0010) [2023-12-26 23:53:40,209][105620] Updated weights for policy 1, policy_version 1171163 (0.0011) [2023-12-26 23:53:40,272][105620] Updated weights for policy 1, policy_version 1171173 (0.0010) [2023-12-26 23:53:40,753][105692] Updated weights for policy 0, policy_version 1169741 (0.0007) [2023-12-26 23:53:40,816][105692] Updated weights for policy 0, policy_version 1169751 (0.0008) [2023-12-26 23:53:40,868][105692] Updated weights for policy 0, policy_version 1169761 (0.0008) [2023-12-26 23:53:41,035][105620] Updated weights for policy 1, policy_version 1171183 (0.0011) [2023-12-26 23:53:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 599367680. Throughput: 0: 9777.7, 1: 9673.4. Samples: 599378052. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:53:41,063][104569] Avg episode reward: [(0, '9082.644'), (1, '9078.331')] [2023-12-26 23:53:41,102][105620] Updated weights for policy 1, policy_version 1171193 (0.0011) [2023-12-26 23:53:41,172][105620] Updated weights for policy 1, policy_version 1171203 (0.0008) [2023-12-26 23:53:41,656][105692] Updated weights for policy 0, policy_version 1169771 (0.0008) [2023-12-26 23:53:41,723][105692] Updated weights for policy 0, policy_version 1169781 (0.0009) [2023-12-26 23:53:41,787][105692] Updated weights for policy 0, policy_version 1169791 (0.0006) [2023-12-26 23:53:41,908][105620] Updated weights for policy 1, policy_version 1171213 (0.0007) [2023-12-26 23:53:41,960][105620] Updated weights for policy 1, policy_version 1171223 (0.0005) [2023-12-26 23:53:42,018][105620] Updated weights for policy 1, policy_version 1171233 (0.0008) [2023-12-26 23:53:42,544][105692] Updated weights for policy 0, policy_version 1169801 (0.0007) [2023-12-26 23:53:42,606][105692] Updated weights for policy 0, policy_version 1169811 (0.0008) [2023-12-26 23:53:42,630][105620] Updated weights for policy 1, policy_version 1171243 (0.0006) [2023-12-26 23:53:42,669][105692] Updated weights for policy 0, policy_version 1169821 (0.0009) [2023-12-26 23:53:42,689][105620] Updated weights for policy 1, policy_version 1171253 (0.0006) [2023-12-26 23:53:42,733][105692] Updated weights for policy 0, policy_version 1169831 (0.0008) [2023-12-26 23:53:42,748][105620] Updated weights for policy 1, policy_version 1171263 (0.0009) [2023-12-26 23:53:43,449][105620] Updated weights for policy 1, policy_version 1171273 (0.0009) [2023-12-26 23:53:43,472][105692] Updated weights for policy 0, policy_version 1169841 (0.0008) [2023-12-26 23:53:43,506][105620] Updated weights for policy 1, policy_version 1171283 (0.0006) [2023-12-26 23:53:43,535][105692] Updated weights for policy 0, policy_version 1169851 (0.0009) [2023-12-26 23:53:43,564][105620] Updated weights for policy 1, policy_version 1171293 (0.0007) [2023-12-26 23:53:43,601][105692] Updated weights for policy 0, policy_version 1169861 (0.0008) [2023-12-26 23:53:43,621][105620] Updated weights for policy 1, policy_version 1171303 (0.0005) [2023-12-26 23:53:44,230][105620] Updated weights for policy 1, policy_version 1171313 (0.0008) [2023-12-26 23:53:44,292][105620] Updated weights for policy 1, policy_version 1171323 (0.0005) [2023-12-26 23:53:44,345][105620] Updated weights for policy 1, policy_version 1171333 (0.0005) [2023-12-26 23:53:44,408][105692] Updated weights for policy 0, policy_version 1169871 (0.0010) [2023-12-26 23:53:44,461][105692] Updated weights for policy 0, policy_version 1169881 (0.0009) [2023-12-26 23:53:44,516][105692] Updated weights for policy 0, policy_version 1169891 (0.0010) [2023-12-26 23:53:44,914][105620] Updated weights for policy 1, policy_version 1171343 (0.0009) [2023-12-26 23:53:44,978][105620] Updated weights for policy 1, policy_version 1171353 (0.0011) [2023-12-26 23:53:45,042][105620] Updated weights for policy 1, policy_version 1171363 (0.0011) [2023-12-26 23:53:45,352][105692] Updated weights for policy 0, policy_version 1169902 (0.0011) [2023-12-26 23:53:45,414][105692] Updated weights for policy 0, policy_version 1169912 (0.0011) [2023-12-26 23:53:45,474][105692] Updated weights for policy 0, policy_version 1169922 (0.0011) [2023-12-26 23:53:45,722][105620] Updated weights for policy 1, policy_version 1171373 (0.0010) [2023-12-26 23:53:45,782][105620] Updated weights for policy 1, policy_version 1171383 (0.0011) [2023-12-26 23:53:45,839][105620] Updated weights for policy 1, policy_version 1171393 (0.0010) [2023-12-26 23:53:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 599465984. Throughput: 0: 9779.9, 1: 9705.3. Samples: 599435248. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:53:46,063][104569] Avg episode reward: [(0, '9080.577'), (1, '9170.067')] [2023-12-26 23:53:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001169928_299548672.pth... [2023-12-26 23:53:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001171400_299917312.pth... [2023-12-26 23:53:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001168808_299261952.pth [2023-12-26 23:53:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001170248_299622400.pth [2023-12-26 23:53:46,223][105692] Updated weights for policy 0, policy_version 1169932 (0.0010) [2023-12-26 23:53:46,285][105692] Updated weights for policy 0, policy_version 1169942 (0.0010) [2023-12-26 23:53:46,344][105692] Updated weights for policy 0, policy_version 1169952 (0.0010) [2023-12-26 23:53:46,602][105620] Updated weights for policy 1, policy_version 1171403 (0.0010) [2023-12-26 23:53:46,665][105620] Updated weights for policy 1, policy_version 1171413 (0.0005) [2023-12-26 23:53:46,718][105620] Updated weights for policy 1, policy_version 1171423 (0.0007) [2023-12-26 23:53:46,942][105692] Updated weights for policy 0, policy_version 1169962 (0.0009) [2023-12-26 23:53:47,009][105692] Updated weights for policy 0, policy_version 1169972 (0.0008) [2023-12-26 23:53:47,067][105692] Updated weights for policy 0, policy_version 1169982 (0.0008) [2023-12-26 23:53:47,129][105692] Updated weights for policy 0, policy_version 1169992 (0.0005) [2023-12-26 23:53:47,373][105620] Updated weights for policy 1, policy_version 1171433 (0.0009) [2023-12-26 23:53:47,429][105620] Updated weights for policy 1, policy_version 1171443 (0.0007) [2023-12-26 23:53:47,482][105620] Updated weights for policy 1, policy_version 1171453 (0.0008) [2023-12-26 23:53:47,540][105620] Updated weights for policy 1, policy_version 1171463 (0.0009) [2023-12-26 23:53:47,787][105692] Updated weights for policy 0, policy_version 1170002 (0.0009) [2023-12-26 23:53:47,842][105692] Updated weights for policy 0, policy_version 1170012 (0.0009) [2023-12-26 23:53:47,896][105692] Updated weights for policy 0, policy_version 1170022 (0.0009) [2023-12-26 23:53:48,281][105620] Updated weights for policy 1, policy_version 1171473 (0.0010) [2023-12-26 23:53:48,352][105620] Updated weights for policy 1, policy_version 1171483 (0.0009) [2023-12-26 23:53:48,414][105620] Updated weights for policy 1, policy_version 1171493 (0.0011) [2023-12-26 23:53:48,526][105692] Updated weights for policy 0, policy_version 1170032 (0.0009) [2023-12-26 23:53:48,585][105692] Updated weights for policy 0, policy_version 1170042 (0.0009) [2023-12-26 23:53:48,637][105692] Updated weights for policy 0, policy_version 1170052 (0.0009) [2023-12-26 23:53:49,179][105620] Updated weights for policy 1, policy_version 1171503 (0.0009) [2023-12-26 23:53:49,237][105620] Updated weights for policy 1, policy_version 1171513 (0.0009) [2023-12-26 23:53:49,302][105620] Updated weights for policy 1, policy_version 1171523 (0.0009) [2023-12-26 23:53:49,400][105692] Updated weights for policy 0, policy_version 1170062 (0.0008) [2023-12-26 23:53:49,467][105692] Updated weights for policy 0, policy_version 1170072 (0.0009) [2023-12-26 23:53:49,533][105692] Updated weights for policy 0, policy_version 1170082 (0.0009) [2023-12-26 23:53:50,088][105620] Updated weights for policy 1, policy_version 1171533 (0.0008) [2023-12-26 23:53:50,139][105620] Updated weights for policy 1, policy_version 1171543 (0.0008) [2023-12-26 23:53:50,191][105620] Updated weights for policy 1, policy_version 1171553 (0.0009) [2023-12-26 23:53:50,236][105692] Updated weights for policy 0, policy_version 1170092 (0.0006) [2023-12-26 23:53:50,291][105692] Updated weights for policy 0, policy_version 1170102 (0.0005) [2023-12-26 23:53:50,343][105692] Updated weights for policy 0, policy_version 1170112 (0.0005) [2023-12-26 23:53:50,973][105692] Updated weights for policy 0, policy_version 1170122 (0.0006) [2023-12-26 23:53:51,038][105692] Updated weights for policy 0, policy_version 1170132 (0.0009) [2023-12-26 23:53:51,040][105620] Updated weights for policy 1, policy_version 1171563 (0.0009) [2023-12-26 23:53:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 599556096. Throughput: 0: 9828.3, 1: 9738.1. Samples: 599551928. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:53:51,063][104569] Avg episode reward: [(0, '9172.635'), (1, '8471.973')] [2023-12-26 23:53:51,102][105692] Updated weights for policy 0, policy_version 1170142 (0.0007) [2023-12-26 23:53:51,112][105620] Updated weights for policy 1, policy_version 1171573 (0.0008) [2023-12-26 23:53:51,171][105692] Updated weights for policy 0, policy_version 1170152 (0.0007) [2023-12-26 23:53:51,176][105620] Updated weights for policy 1, policy_version 1171583 (0.0008) [2023-12-26 23:53:51,934][105620] Updated weights for policy 1, policy_version 1171593 (0.0007) [2023-12-26 23:53:51,966][105692] Updated weights for policy 0, policy_version 1170162 (0.0010) [2023-12-26 23:53:51,991][105620] Updated weights for policy 1, policy_version 1171603 (0.0010) [2023-12-26 23:53:52,022][105692] Updated weights for policy 0, policy_version 1170172 (0.0009) [2023-12-26 23:53:52,044][105620] Updated weights for policy 1, policy_version 1171613 (0.0008) [2023-12-26 23:53:52,085][105692] Updated weights for policy 0, policy_version 1170182 (0.0009) [2023-12-26 23:53:52,101][105620] Updated weights for policy 1, policy_version 1171623 (0.0009) [2023-12-26 23:53:52,829][105692] Updated weights for policy 0, policy_version 1170192 (0.0010) [2023-12-26 23:53:52,886][105620] Updated weights for policy 1, policy_version 1171633 (0.0006) [2023-12-26 23:53:52,891][105692] Updated weights for policy 0, policy_version 1170202 (0.0010) [2023-12-26 23:53:52,949][105620] Updated weights for policy 1, policy_version 1171643 (0.0006) [2023-12-26 23:53:52,950][105692] Updated weights for policy 0, policy_version 1170212 (0.0011) [2023-12-26 23:53:53,005][105620] Updated weights for policy 1, policy_version 1171653 (0.0007) [2023-12-26 23:53:53,653][105692] Updated weights for policy 0, policy_version 1170222 (0.0007) [2023-12-26 23:53:53,707][105692] Updated weights for policy 0, policy_version 1170232 (0.0005) [2023-12-26 23:53:53,767][105692] Updated weights for policy 0, policy_version 1170242 (0.0007) [2023-12-26 23:53:53,778][105620] Updated weights for policy 1, policy_version 1171663 (0.0009) [2023-12-26 23:53:53,827][105620] Updated weights for policy 1, policy_version 1171673 (0.0007) [2023-12-26 23:53:53,876][105620] Updated weights for policy 1, policy_version 1171683 (0.0008) [2023-12-26 23:53:54,457][105692] Updated weights for policy 0, policy_version 1170252 (0.0011) [2023-12-26 23:53:54,526][105692] Updated weights for policy 0, policy_version 1170262 (0.0010) [2023-12-26 23:53:54,593][105692] Updated weights for policy 0, policy_version 1170272 (0.0011) [2023-12-26 23:53:54,653][105620] Updated weights for policy 1, policy_version 1171693 (0.0008) [2023-12-26 23:53:54,712][105620] Updated weights for policy 1, policy_version 1171703 (0.0009) [2023-12-26 23:53:54,760][105620] Updated weights for policy 1, policy_version 1171713 (0.0007) [2023-12-26 23:53:55,200][105692] Updated weights for policy 0, policy_version 1170282 (0.0009) [2023-12-26 23:53:55,243][105692] Updated weights for policy 0, policy_version 1170292 (0.0007) [2023-12-26 23:53:55,294][105692] Updated weights for policy 0, policy_version 1170302 (0.0005) [2023-12-26 23:53:55,362][105692] Updated weights for policy 0, policy_version 1170312 (0.0005) [2023-12-26 23:53:55,449][105620] Updated weights for policy 1, policy_version 1171723 (0.0007) [2023-12-26 23:53:55,507][105620] Updated weights for policy 1, policy_version 1171733 (0.0009) [2023-12-26 23:53:55,553][105620] Updated weights for policy 1, policy_version 1171743 (0.0009) [2023-12-26 23:53:56,033][105692] Updated weights for policy 0, policy_version 1170322 (0.0008) [2023-12-26 23:53:56,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 599654400. Throughput: 0: 9841.9, 1: 9704.4. Samples: 599667180. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:53:56,062][104569] Avg episode reward: [(0, '9173.329'), (1, '8501.927')] [2023-12-26 23:53:56,088][105692] Updated weights for policy 0, policy_version 1170332 (0.0007) [2023-12-26 23:53:56,140][105692] Updated weights for policy 0, policy_version 1170343 (0.0010) [2023-12-26 23:53:56,250][105620] Updated weights for policy 1, policy_version 1171753 (0.0009) [2023-12-26 23:53:56,307][105620] Updated weights for policy 1, policy_version 1171763 (0.0009) [2023-12-26 23:53:56,360][105620] Updated weights for policy 1, policy_version 1171774 (0.0010) [2023-12-26 23:53:56,416][105620] Updated weights for policy 1, policy_version 1171784 (0.0010) [2023-12-26 23:53:56,766][105692] Updated weights for policy 0, policy_version 1170353 (0.0006) [2023-12-26 23:53:56,830][105692] Updated weights for policy 0, policy_version 1170363 (0.0005) [2023-12-26 23:53:56,892][105692] Updated weights for policy 0, policy_version 1170373 (0.0010) [2023-12-26 23:53:57,050][105620] Updated weights for policy 1, policy_version 1171794 (0.0005) [2023-12-26 23:53:57,096][105620] Updated weights for policy 1, policy_version 1171804 (0.0005) [2023-12-26 23:53:57,139][105620] Updated weights for policy 1, policy_version 1171814 (0.0005) [2023-12-26 23:53:57,532][105692] Updated weights for policy 0, policy_version 1170383 (0.0009) [2023-12-26 23:53:57,579][105692] Updated weights for policy 0, policy_version 1170393 (0.0008) [2023-12-26 23:53:57,640][105692] Updated weights for policy 0, policy_version 1170403 (0.0008) [2023-12-26 23:53:57,758][105620] Updated weights for policy 1, policy_version 1171824 (0.0009) [2023-12-26 23:53:57,816][105620] Updated weights for policy 1, policy_version 1171834 (0.0010) [2023-12-26 23:53:57,873][105620] Updated weights for policy 1, policy_version 1171844 (0.0010) [2023-12-26 23:53:58,411][105692] Updated weights for policy 0, policy_version 1170413 (0.0008) [2023-12-26 23:53:58,479][105692] Updated weights for policy 0, policy_version 1170423 (0.0008) [2023-12-26 23:53:58,546][105692] Updated weights for policy 0, policy_version 1170433 (0.0008) [2023-12-26 23:53:58,636][105620] Updated weights for policy 1, policy_version 1171854 (0.0009) [2023-12-26 23:53:58,703][105620] Updated weights for policy 1, policy_version 1171864 (0.0010) [2023-12-26 23:53:58,771][105620] Updated weights for policy 1, policy_version 1171874 (0.0011) [2023-12-26 23:53:59,354][105692] Updated weights for policy 0, policy_version 1170443 (0.0008) [2023-12-26 23:53:59,420][105692] Updated weights for policy 0, policy_version 1170453 (0.0008) [2023-12-26 23:53:59,483][105692] Updated weights for policy 0, policy_version 1170463 (0.0008) [2023-12-26 23:53:59,528][105620] Updated weights for policy 1, policy_version 1171884 (0.0010) [2023-12-26 23:53:59,610][105620] Updated weights for policy 1, policy_version 1171894 (0.0011) [2023-12-26 23:53:59,679][105620] Updated weights for policy 1, policy_version 1171904 (0.0011) [2023-12-26 23:54:00,141][105692] Updated weights for policy 0, policy_version 1170473 (0.0008) [2023-12-26 23:54:00,201][105692] Updated weights for policy 0, policy_version 1170483 (0.0005) [2023-12-26 23:54:00,252][105692] Updated weights for policy 0, policy_version 1170493 (0.0005) [2023-12-26 23:54:00,304][105692] Updated weights for policy 0, policy_version 1170503 (0.0005) [2023-12-26 23:54:00,349][105620] Updated weights for policy 1, policy_version 1171914 (0.0010) [2023-12-26 23:54:00,405][105620] Updated weights for policy 1, policy_version 1171924 (0.0011) [2023-12-26 23:54:00,468][105620] Updated weights for policy 1, policy_version 1171934 (0.0010) [2023-12-26 23:54:00,524][105620] Updated weights for policy 1, policy_version 1171944 (0.0011) [2023-12-26 23:54:00,827][105692] Updated weights for policy 0, policy_version 1170513 (0.0005) [2023-12-26 23:54:00,887][105692] Updated weights for policy 0, policy_version 1170523 (0.0010) [2023-12-26 23:54:00,948][105692] Updated weights for policy 0, policy_version 1170533 (0.0010) [2023-12-26 23:54:01,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 599760896. Throughput: 0: 9829.3, 1: 9757.5. Samples: 599727440. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-26 23:54:01,062][104569] Avg episode reward: [(0, '9264.706'), (1, '9316.657')] [2023-12-26 23:54:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001170536_299704320.pth... [2023-12-26 23:54:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001171944_300056576.pth... [2023-12-26 23:54:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001169352_299401216.pth [2023-12-26 23:54:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001170824_299769856.pth [2023-12-26 23:54:01,308][105620] Updated weights for policy 1, policy_version 1171954 (0.0009) [2023-12-26 23:54:01,377][105620] Updated weights for policy 1, policy_version 1171964 (0.0009) [2023-12-26 23:54:01,428][105620] Updated weights for policy 1, policy_version 1171974 (0.0009) [2023-12-26 23:54:01,590][105692] Updated weights for policy 0, policy_version 1170543 (0.0009) [2023-12-26 23:54:01,654][105692] Updated weights for policy 0, policy_version 1170553 (0.0011) [2023-12-26 23:54:01,706][105692] Updated weights for policy 0, policy_version 1170563 (0.0010) [2023-12-26 23:54:02,242][105620] Updated weights for policy 1, policy_version 1171984 (0.0008) [2023-12-26 23:54:02,291][105692] Updated weights for policy 0, policy_version 1170573 (0.0008) [2023-12-26 23:54:02,303][105620] Updated weights for policy 1, policy_version 1171994 (0.0008) [2023-12-26 23:54:02,356][105692] Updated weights for policy 0, policy_version 1170583 (0.0006) [2023-12-26 23:54:02,369][105620] Updated weights for policy 1, policy_version 1172004 (0.0009) [2023-12-26 23:54:02,420][105692] Updated weights for policy 0, policy_version 1170593 (0.0008) [2023-12-26 23:54:03,063][105692] Updated weights for policy 0, policy_version 1170603 (0.0009) [2023-12-26 23:54:03,083][105620] Updated weights for policy 1, policy_version 1172014 (0.0007) [2023-12-26 23:54:03,118][105692] Updated weights for policy 0, policy_version 1170613 (0.0006) [2023-12-26 23:54:03,139][105620] Updated weights for policy 1, policy_version 1172024 (0.0008) [2023-12-26 23:54:03,177][105692] Updated weights for policy 0, policy_version 1170623 (0.0005) [2023-12-26 23:54:03,191][105620] Updated weights for policy 1, policy_version 1172034 (0.0009) [2023-12-26 23:54:03,781][105692] Updated weights for policy 0, policy_version 1170633 (0.0007) [2023-12-26 23:54:03,830][105692] Updated weights for policy 0, policy_version 1170643 (0.0006) [2023-12-26 23:54:03,894][105692] Updated weights for policy 0, policy_version 1170653 (0.0009) [2023-12-26 23:54:03,950][105692] Updated weights for policy 0, policy_version 1170663 (0.0009) [2023-12-26 23:54:04,029][105620] Updated weights for policy 1, policy_version 1172044 (0.0008) [2023-12-26 23:54:04,082][105620] Updated weights for policy 1, policy_version 1172054 (0.0011) [2023-12-26 23:54:04,138][105620] Updated weights for policy 1, policy_version 1172064 (0.0011) [2023-12-26 23:54:04,595][105692] Updated weights for policy 0, policy_version 1170673 (0.0006) [2023-12-26 23:54:04,652][105692] Updated weights for policy 0, policy_version 1170683 (0.0005) [2023-12-26 23:54:04,709][105692] Updated weights for policy 0, policy_version 1170693 (0.0006) [2023-12-26 23:54:04,911][105620] Updated weights for policy 1, policy_version 1172074 (0.0010) [2023-12-26 23:54:04,975][105620] Updated weights for policy 1, policy_version 1172084 (0.0005) [2023-12-26 23:54:05,042][105620] Updated weights for policy 1, policy_version 1172094 (0.0010) [2023-12-26 23:54:05,104][105620] Updated weights for policy 1, policy_version 1172104 (0.0010) [2023-12-26 23:54:05,241][105692] Updated weights for policy 0, policy_version 1170703 (0.0006) [2023-12-26 23:54:05,297][105692] Updated weights for policy 0, policy_version 1170713 (0.0008) [2023-12-26 23:54:05,363][105692] Updated weights for policy 0, policy_version 1170723 (0.0008) [2023-12-26 23:54:05,806][105620] Updated weights for policy 1, policy_version 1172114 (0.0010) [2023-12-26 23:54:05,860][105620] Updated weights for policy 1, policy_version 1172124 (0.0010) [2023-12-26 23:54:05,921][105620] Updated weights for policy 1, policy_version 1172134 (0.0010) [2023-12-26 23:54:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 599859200. Throughput: 0: 10006.0, 1: 9640.5. Samples: 599847080. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:54:06,062][104569] Avg episode reward: [(0, '9173.944'), (1, '9351.885')] [2023-12-26 23:54:06,064][105692] Updated weights for policy 0, policy_version 1170733 (0.0009) [2023-12-26 23:54:06,124][105692] Updated weights for policy 0, policy_version 1170743 (0.0010) [2023-12-26 23:54:06,176][105692] Updated weights for policy 0, policy_version 1170753 (0.0010) [2023-12-26 23:54:06,646][105620] Updated weights for policy 1, policy_version 1172144 (0.0010) [2023-12-26 23:54:06,709][105620] Updated weights for policy 1, policy_version 1172154 (0.0007) [2023-12-26 23:54:06,765][105620] Updated weights for policy 1, policy_version 1172164 (0.0008) [2023-12-26 23:54:06,947][105692] Updated weights for policy 0, policy_version 1170763 (0.0009) [2023-12-26 23:54:07,013][105692] Updated weights for policy 0, policy_version 1170773 (0.0006) [2023-12-26 23:54:07,077][105692] Updated weights for policy 0, policy_version 1170783 (0.0006) [2023-12-26 23:54:07,485][105620] Updated weights for policy 1, policy_version 1172174 (0.0008) [2023-12-26 23:54:07,545][105620] Updated weights for policy 1, policy_version 1172184 (0.0008) [2023-12-26 23:54:07,593][105620] Updated weights for policy 1, policy_version 1172194 (0.0008) [2023-12-26 23:54:07,683][105692] Updated weights for policy 0, policy_version 1170793 (0.0006) [2023-12-26 23:54:07,735][105692] Updated weights for policy 0, policy_version 1170803 (0.0010) [2023-12-26 23:54:07,783][105692] Updated weights for policy 0, policy_version 1170813 (0.0010) [2023-12-26 23:54:07,833][105692] Updated weights for policy 0, policy_version 1170823 (0.0010) [2023-12-26 23:54:08,348][105620] Updated weights for policy 1, policy_version 1172205 (0.0008) [2023-12-26 23:54:08,409][105620] Updated weights for policy 1, policy_version 1172215 (0.0006) [2023-12-26 23:54:08,474][105620] Updated weights for policy 1, policy_version 1172225 (0.0006) [2023-12-26 23:54:08,611][105692] Updated weights for policy 0, policy_version 1170833 (0.0010) [2023-12-26 23:54:08,673][105692] Updated weights for policy 0, policy_version 1170843 (0.0011) [2023-12-26 23:54:08,731][105692] Updated weights for policy 0, policy_version 1170853 (0.0010) [2023-12-26 23:54:09,134][105620] Updated weights for policy 1, policy_version 1172235 (0.0007) [2023-12-26 23:54:09,195][105620] Updated weights for policy 1, policy_version 1172245 (0.0010) [2023-12-26 23:54:09,259][105620] Updated weights for policy 1, policy_version 1172255 (0.0011) [2023-12-26 23:54:09,422][105692] Updated weights for policy 0, policy_version 1170863 (0.0008) [2023-12-26 23:54:09,493][105692] Updated weights for policy 0, policy_version 1170873 (0.0007) [2023-12-26 23:54:09,555][105692] Updated weights for policy 0, policy_version 1170883 (0.0011) [2023-12-26 23:54:10,024][105620] Updated weights for policy 1, policy_version 1172265 (0.0010) [2023-12-26 23:54:10,074][105620] Updated weights for policy 1, policy_version 1172275 (0.0009) [2023-12-26 23:54:10,129][105620] Updated weights for policy 1, policy_version 1172285 (0.0009) [2023-12-26 23:54:10,183][105620] Updated weights for policy 1, policy_version 1172295 (0.0010) [2023-12-26 23:54:10,255][105692] Updated weights for policy 0, policy_version 1170893 (0.0008) [2023-12-26 23:54:10,305][105692] Updated weights for policy 0, policy_version 1170903 (0.0005) [2023-12-26 23:54:10,366][105692] Updated weights for policy 0, policy_version 1170913 (0.0006) [2023-12-26 23:54:10,974][105692] Updated weights for policy 0, policy_version 1170923 (0.0009) [2023-12-26 23:54:11,033][105692] Updated weights for policy 0, policy_version 1170933 (0.0007) [2023-12-26 23:54:11,053][105620] Updated weights for policy 1, policy_version 1172305 (0.0007) [2023-12-26 23:54:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 599949312. Throughput: 0: 10087.4, 1: 9622.0. Samples: 599964164. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:54:11,062][104569] Avg episode reward: [(0, '8991.440'), (1, '9262.749')] [2023-12-26 23:54:11,098][105692] Updated weights for policy 0, policy_version 1170943 (0.0008) [2023-12-26 23:54:11,111][105620] Updated weights for policy 1, policy_version 1172315 (0.0007) [2023-12-26 23:54:11,186][105620] Updated weights for policy 1, policy_version 1172325 (0.0007) [2023-12-26 23:54:11,904][105692] Updated weights for policy 0, policy_version 1170953 (0.0008) [2023-12-26 23:54:11,906][105620] Updated weights for policy 1, policy_version 1172335 (0.0007) [2023-12-26 23:54:11,963][105692] Updated weights for policy 0, policy_version 1170963 (0.0009) [2023-12-26 23:54:11,969][105620] Updated weights for policy 1, policy_version 1172345 (0.0006) [2023-12-26 23:54:12,017][105692] Updated weights for policy 0, policy_version 1170973 (0.0008) [2023-12-26 23:54:12,028][105620] Updated weights for policy 1, policy_version 1172355 (0.0006) [2023-12-26 23:54:12,076][105692] Updated weights for policy 0, policy_version 1170983 (0.0007) [2023-12-26 23:54:12,771][105620] Updated weights for policy 1, policy_version 1172365 (0.0006) [2023-12-26 23:54:12,783][105692] Updated weights for policy 0, policy_version 1170993 (0.0008) [2023-12-26 23:54:12,829][105620] Updated weights for policy 1, policy_version 1172375 (0.0005) [2023-12-26 23:54:12,840][105692] Updated weights for policy 0, policy_version 1171003 (0.0009) [2023-12-26 23:54:12,887][105620] Updated weights for policy 1, policy_version 1172385 (0.0006) [2023-12-26 23:54:12,897][105692] Updated weights for policy 0, policy_version 1171013 (0.0008) [2023-12-26 23:54:13,602][105620] Updated weights for policy 1, policy_version 1172395 (0.0007) [2023-12-26 23:54:13,653][105620] Updated weights for policy 1, policy_version 1172405 (0.0007) [2023-12-26 23:54:13,658][105692] Updated weights for policy 0, policy_version 1171023 (0.0007) [2023-12-26 23:54:13,709][105620] Updated weights for policy 1, policy_version 1172415 (0.0009) [2023-12-26 23:54:13,719][105692] Updated weights for policy 0, policy_version 1171033 (0.0008) [2023-12-26 23:54:13,784][105692] Updated weights for policy 0, policy_version 1171043 (0.0008) [2023-12-26 23:54:14,478][105620] Updated weights for policy 1, policy_version 1172425 (0.0007) [2023-12-26 23:54:14,525][105692] Updated weights for policy 0, policy_version 1171053 (0.0010) [2023-12-26 23:54:14,540][105620] Updated weights for policy 1, policy_version 1172435 (0.0010) [2023-12-26 23:54:14,577][105692] Updated weights for policy 0, policy_version 1171063 (0.0008) [2023-12-26 23:54:14,596][105620] Updated weights for policy 1, policy_version 1172445 (0.0010) [2023-12-26 23:54:14,629][105692] Updated weights for policy 0, policy_version 1171073 (0.0006) [2023-12-26 23:54:14,642][105620] Updated weights for policy 1, policy_version 1172455 (0.0009) [2023-12-26 23:54:15,296][105692] Updated weights for policy 0, policy_version 1171083 (0.0007) [2023-12-26 23:54:15,353][105620] Updated weights for policy 1, policy_version 1172465 (0.0010) [2023-12-26 23:54:15,356][105692] Updated weights for policy 0, policy_version 1171093 (0.0009) [2023-12-26 23:54:15,403][105620] Updated weights for policy 1, policy_version 1172475 (0.0008) [2023-12-26 23:54:15,414][105692] Updated weights for policy 0, policy_version 1171103 (0.0007) [2023-12-26 23:54:15,454][105620] Updated weights for policy 1, policy_version 1172485 (0.0007) [2023-12-26 23:54:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 600047616. Throughput: 0: 10003.4, 1: 9529.1. Samples: 600021204. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:54:16,062][104569] Avg episode reward: [(0, '9173.387'), (1, '9171.080')] [2023-12-26 23:54:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001171112_299851776.pth... [2023-12-26 23:54:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001172488_300195840.pth... [2023-12-26 23:54:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001169928_299548672.pth [2023-12-26 23:54:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001171400_299917312.pth [2023-12-26 23:54:16,195][105692] Updated weights for policy 0, policy_version 1171113 (0.0009) [2023-12-26 23:54:16,226][105620] Updated weights for policy 1, policy_version 1172495 (0.0011) [2023-12-26 23:54:16,245][105692] Updated weights for policy 0, policy_version 1171123 (0.0010) [2023-12-26 23:54:16,281][105620] Updated weights for policy 1, policy_version 1172505 (0.0011) [2023-12-26 23:54:16,290][105692] Updated weights for policy 0, policy_version 1171133 (0.0010) [2023-12-26 23:54:16,338][105620] Updated weights for policy 1, policy_version 1172515 (0.0011) [2023-12-26 23:54:16,339][105692] Updated weights for policy 0, policy_version 1171143 (0.0010) [2023-12-26 23:54:16,890][105620] Updated weights for policy 1, policy_version 1172525 (0.0008) [2023-12-26 23:54:16,947][105620] Updated weights for policy 1, policy_version 1172535 (0.0008) [2023-12-26 23:54:16,999][105620] Updated weights for policy 1, policy_version 1172545 (0.0010) [2023-12-26 23:54:17,041][105692] Updated weights for policy 0, policy_version 1171153 (0.0007) [2023-12-26 23:54:17,100][105692] Updated weights for policy 0, policy_version 1171163 (0.0007) [2023-12-26 23:54:17,158][105692] Updated weights for policy 0, policy_version 1171173 (0.0009) [2023-12-26 23:54:17,673][105620] Updated weights for policy 1, policy_version 1172555 (0.0010) [2023-12-26 23:54:17,731][105620] Updated weights for policy 1, policy_version 1172565 (0.0009) [2023-12-26 23:54:17,779][105620] Updated weights for policy 1, policy_version 1172575 (0.0009) [2023-12-26 23:54:17,928][105692] Updated weights for policy 0, policy_version 1171183 (0.0009) [2023-12-26 23:54:17,989][105692] Updated weights for policy 0, policy_version 1171193 (0.0009) [2023-12-26 23:54:18,046][105692] Updated weights for policy 0, policy_version 1171203 (0.0008) [2023-12-26 23:54:18,505][105620] Updated weights for policy 1, policy_version 1172585 (0.0008) [2023-12-26 23:54:18,572][105620] Updated weights for policy 1, policy_version 1172595 (0.0010) [2023-12-26 23:54:18,629][105620] Updated weights for policy 1, policy_version 1172605 (0.0009) [2023-12-26 23:54:18,680][105620] Updated weights for policy 1, policy_version 1172615 (0.0009) [2023-12-26 23:54:18,822][105692] Updated weights for policy 0, policy_version 1171213 (0.0008) [2023-12-26 23:54:18,870][105692] Updated weights for policy 0, policy_version 1171223 (0.0009) [2023-12-26 23:54:18,924][105692] Updated weights for policy 0, policy_version 1171233 (0.0009) [2023-12-26 23:54:19,431][105620] Updated weights for policy 1, policy_version 1172625 (0.0009) [2023-12-26 23:54:19,479][105620] Updated weights for policy 1, policy_version 1172635 (0.0009) [2023-12-26 23:54:19,533][105620] Updated weights for policy 1, policy_version 1172645 (0.0009) [2023-12-26 23:54:19,696][105692] Updated weights for policy 0, policy_version 1171243 (0.0008) [2023-12-26 23:54:19,758][105692] Updated weights for policy 0, policy_version 1171253 (0.0009) [2023-12-26 23:54:19,824][105692] Updated weights for policy 0, policy_version 1171263 (0.0009) [2023-12-26 23:54:20,353][105620] Updated weights for policy 1, policy_version 1172655 (0.0009) [2023-12-26 23:54:20,416][105620] Updated weights for policy 1, policy_version 1172665 (0.0009) [2023-12-26 23:54:20,478][105620] Updated weights for policy 1, policy_version 1172675 (0.0009) [2023-12-26 23:54:20,575][105692] Updated weights for policy 0, policy_version 1171273 (0.0009) [2023-12-26 23:54:20,640][105692] Updated weights for policy 0, policy_version 1171283 (0.0009) [2023-12-26 23:54:20,695][105692] Updated weights for policy 0, policy_version 1171293 (0.0010) [2023-12-26 23:54:20,758][105692] Updated weights for policy 0, policy_version 1171303 (0.0010) [2023-12-26 23:54:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 600145920. Throughput: 0: 9930.3, 1: 9525.9. Samples: 600136692. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:54:21,063][104569] Avg episode reward: [(0, '9264.838'), (1, '9170.025')] [2023-12-26 23:54:21,217][105620] Updated weights for policy 1, policy_version 1172685 (0.0008) [2023-12-26 23:54:21,279][105620] Updated weights for policy 1, policy_version 1172695 (0.0008) [2023-12-26 23:54:21,353][105620] Updated weights for policy 1, policy_version 1172705 (0.0009) [2023-12-26 23:54:21,574][105692] Updated weights for policy 0, policy_version 1171313 (0.0008) [2023-12-26 23:54:21,639][105692] Updated weights for policy 0, policy_version 1171323 (0.0008) [2023-12-26 23:54:21,701][105692] Updated weights for policy 0, policy_version 1171333 (0.0009) [2023-12-26 23:54:22,145][105620] Updated weights for policy 1, policy_version 1172715 (0.0009) [2023-12-26 23:54:22,199][105620] Updated weights for policy 1, policy_version 1172725 (0.0007) [2023-12-26 23:54:22,268][105620] Updated weights for policy 1, policy_version 1172735 (0.0008) [2023-12-26 23:54:22,515][105692] Updated weights for policy 0, policy_version 1171343 (0.0009) [2023-12-26 23:54:22,584][105692] Updated weights for policy 0, policy_version 1171353 (0.0009) [2023-12-26 23:54:22,652][105692] Updated weights for policy 0, policy_version 1171363 (0.0009) [2023-12-26 23:54:22,995][105620] Updated weights for policy 1, policy_version 1172745 (0.0009) [2023-12-26 23:54:23,041][105620] Updated weights for policy 1, policy_version 1172755 (0.0010) [2023-12-26 23:54:23,097][105620] Updated weights for policy 1, policy_version 1172766 (0.0010) [2023-12-26 23:54:23,154][105620] Updated weights for policy 1, policy_version 1172776 (0.0008) [2023-12-26 23:54:23,428][105692] Updated weights for policy 0, policy_version 1171373 (0.0010) [2023-12-26 23:54:23,482][105692] Updated weights for policy 0, policy_version 1171383 (0.0010) [2023-12-26 23:54:23,540][105692] Updated weights for policy 0, policy_version 1171393 (0.0010) [2023-12-26 23:54:23,806][105620] Updated weights for policy 1, policy_version 1172786 (0.0009) [2023-12-26 23:54:23,852][105620] Updated weights for policy 1, policy_version 1172796 (0.0008) [2023-12-26 23:54:23,897][105620] Updated weights for policy 1, policy_version 1172806 (0.0008) [2023-12-26 23:54:24,272][105692] Updated weights for policy 0, policy_version 1171403 (0.0009) [2023-12-26 23:54:24,334][105692] Updated weights for policy 0, policy_version 1171413 (0.0010) [2023-12-26 23:54:24,389][105692] Updated weights for policy 0, policy_version 1171423 (0.0010) [2023-12-26 23:54:24,625][105620] Updated weights for policy 1, policy_version 1172816 (0.0007) [2023-12-26 23:54:24,674][105620] Updated weights for policy 1, policy_version 1172826 (0.0005) [2023-12-26 23:54:24,720][105620] Updated weights for policy 1, policy_version 1172836 (0.0005) [2023-12-26 23:54:25,011][105692] Updated weights for policy 0, policy_version 1171433 (0.0010) [2023-12-26 23:54:25,073][105692] Updated weights for policy 0, policy_version 1171443 (0.0005) [2023-12-26 23:54:25,143][105692] Updated weights for policy 0, policy_version 1171453 (0.0005) [2023-12-26 23:54:25,199][105692] Updated weights for policy 0, policy_version 1171463 (0.0007) [2023-12-26 23:54:25,281][105620] Updated weights for policy 1, policy_version 1172846 (0.0008) [2023-12-26 23:54:25,335][105620] Updated weights for policy 1, policy_version 1172856 (0.0010) [2023-12-26 23:54:25,388][105620] Updated weights for policy 1, policy_version 1172866 (0.0010) [2023-12-26 23:54:25,872][105692] Updated weights for policy 0, policy_version 1171473 (0.0010) [2023-12-26 23:54:25,934][105692] Updated weights for policy 0, policy_version 1171483 (0.0010) [2023-12-26 23:54:25,984][105692] Updated weights for policy 0, policy_version 1171493 (0.0008) [2023-12-26 23:54:26,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 600244224. Throughput: 0: 9805.4, 1: 9625.9. Samples: 600252464. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:54:26,063][104569] Avg episode reward: [(0, '9265.300'), (1, '9079.136')] [2023-12-26 23:54:26,107][105620] Updated weights for policy 1, policy_version 1172876 (0.0008) [2023-12-26 23:54:26,154][105620] Updated weights for policy 1, policy_version 1172886 (0.0007) [2023-12-26 23:54:26,210][105620] Updated weights for policy 1, policy_version 1172896 (0.0010) [2023-12-26 23:54:26,709][105692] Updated weights for policy 0, policy_version 1171503 (0.0008) [2023-12-26 23:54:26,765][105692] Updated weights for policy 0, policy_version 1171513 (0.0009) [2023-12-26 23:54:26,825][105692] Updated weights for policy 0, policy_version 1171523 (0.0009) [2023-12-26 23:54:26,893][105620] Updated weights for policy 1, policy_version 1172906 (0.0010) [2023-12-26 23:54:26,948][105620] Updated weights for policy 1, policy_version 1172916 (0.0005) [2023-12-26 23:54:26,992][105620] Updated weights for policy 1, policy_version 1172926 (0.0005) [2023-12-26 23:54:27,038][105620] Updated weights for policy 1, policy_version 1172936 (0.0005) [2023-12-26 23:54:27,441][105692] Updated weights for policy 0, policy_version 1171533 (0.0009) [2023-12-26 23:54:27,486][105692] Updated weights for policy 0, policy_version 1171543 (0.0009) [2023-12-26 23:54:27,537][105692] Updated weights for policy 0, policy_version 1171553 (0.0010) [2023-12-26 23:54:27,755][105620] Updated weights for policy 1, policy_version 1172946 (0.0010) [2023-12-26 23:54:27,816][105620] Updated weights for policy 1, policy_version 1172956 (0.0010) [2023-12-26 23:54:27,880][105620] Updated weights for policy 1, policy_version 1172966 (0.0010) [2023-12-26 23:54:28,213][105692] Updated weights for policy 0, policy_version 1171563 (0.0009) [2023-12-26 23:54:28,263][105692] Updated weights for policy 0, policy_version 1171573 (0.0005) [2023-12-26 23:54:28,311][105692] Updated weights for policy 0, policy_version 1171583 (0.0006) [2023-12-26 23:54:28,601][105620] Updated weights for policy 1, policy_version 1172976 (0.0010) [2023-12-26 23:54:28,653][105620] Updated weights for policy 1, policy_version 1172986 (0.0010) [2023-12-26 23:54:28,710][105620] Updated weights for policy 1, policy_version 1172996 (0.0010) [2023-12-26 23:54:29,009][105692] Updated weights for policy 0, policy_version 1171593 (0.0011) [2023-12-26 23:54:29,071][105692] Updated weights for policy 0, policy_version 1171603 (0.0008) [2023-12-26 23:54:29,135][105692] Updated weights for policy 0, policy_version 1171613 (0.0009) [2023-12-26 23:54:29,187][105692] Updated weights for policy 0, policy_version 1171623 (0.0009) [2023-12-26 23:54:29,427][105620] Updated weights for policy 1, policy_version 1173006 (0.0010) [2023-12-26 23:54:29,495][105620] Updated weights for policy 1, policy_version 1173016 (0.0009) [2023-12-26 23:54:29,564][105620] Updated weights for policy 1, policy_version 1173026 (0.0006) [2023-12-26 23:54:29,857][105692] Updated weights for policy 0, policy_version 1171633 (0.0008) [2023-12-26 23:54:29,915][105692] Updated weights for policy 0, policy_version 1171643 (0.0007) [2023-12-26 23:54:29,970][105692] Updated weights for policy 0, policy_version 1171653 (0.0009) [2023-12-26 23:54:30,255][105620] Updated weights for policy 1, policy_version 1173036 (0.0008) [2023-12-26 23:54:30,312][105620] Updated weights for policy 1, policy_version 1173046 (0.0005) [2023-12-26 23:54:30,369][105620] Updated weights for policy 1, policy_version 1173056 (0.0005) [2023-12-26 23:54:30,632][105692] Updated weights for policy 0, policy_version 1171663 (0.0006) [2023-12-26 23:54:30,695][105692] Updated weights for policy 0, policy_version 1171673 (0.0005) [2023-12-26 23:54:30,747][105692] Updated weights for policy 0, policy_version 1171683 (0.0009) [2023-12-26 23:54:31,043][105620] Updated weights for policy 1, policy_version 1173066 (0.0009) [2023-12-26 23:54:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 600342528. Throughput: 0: 9890.3, 1: 9609.1. Samples: 600312720. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:54:31,062][104569] Avg episode reward: [(0, '9173.952'), (1, '8895.360')] [2023-12-26 23:54:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001171688_299999232.pth... [2023-12-26 23:54:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001170536_299704320.pth [2023-12-26 23:54:31,095][105620] Updated weights for policy 1, policy_version 1173076 (0.0009) [2023-12-26 23:54:31,151][105620] Updated weights for policy 1, policy_version 1173086 (0.0009) [2023-12-26 23:54:31,204][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001173096_300351488.pth... [2023-12-26 23:54:31,206][105620] Updated weights for policy 1, policy_version 1173096 (0.0008) [2023-12-26 23:54:31,209][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001171944_300056576.pth [2023-12-26 23:54:31,372][105692] Updated weights for policy 0, policy_version 1171693 (0.0009) [2023-12-26 23:54:31,441][105692] Updated weights for policy 0, policy_version 1171703 (0.0007) [2023-12-26 23:54:31,504][105692] Updated weights for policy 0, policy_version 1171713 (0.0006) [2023-12-26 23:54:31,966][105620] Updated weights for policy 1, policy_version 1173106 (0.0005) [2023-12-26 23:54:32,016][105620] Updated weights for policy 1, policy_version 1173116 (0.0006) [2023-12-26 23:54:32,069][105620] Updated weights for policy 1, policy_version 1173126 (0.0005) [2023-12-26 23:54:32,250][105692] Updated weights for policy 0, policy_version 1171723 (0.0007) [2023-12-26 23:54:32,306][105692] Updated weights for policy 0, policy_version 1171733 (0.0009) [2023-12-26 23:54:32,358][105692] Updated weights for policy 0, policy_version 1171743 (0.0009) [2023-12-26 23:54:32,757][105620] Updated weights for policy 1, policy_version 1173136 (0.0005) [2023-12-26 23:54:32,820][105620] Updated weights for policy 1, policy_version 1173146 (0.0006) [2023-12-26 23:54:32,875][105620] Updated weights for policy 1, policy_version 1173156 (0.0009) [2023-12-26 23:54:33,059][105692] Updated weights for policy 0, policy_version 1171753 (0.0007) [2023-12-26 23:54:33,107][105692] Updated weights for policy 0, policy_version 1171763 (0.0008) [2023-12-26 23:54:33,169][105692] Updated weights for policy 0, policy_version 1171773 (0.0009) [2023-12-26 23:54:33,233][105692] Updated weights for policy 0, policy_version 1171783 (0.0008) [2023-12-26 23:54:33,495][105620] Updated weights for policy 1, policy_version 1173166 (0.0007) [2023-12-26 23:54:33,548][105620] Updated weights for policy 1, policy_version 1173176 (0.0009) [2023-12-26 23:54:33,603][105620] Updated weights for policy 1, policy_version 1173186 (0.0009) [2023-12-26 23:54:33,931][105692] Updated weights for policy 0, policy_version 1171793 (0.0006) [2023-12-26 23:54:33,987][105692] Updated weights for policy 0, policy_version 1171803 (0.0005) [2023-12-26 23:54:34,036][105692] Updated weights for policy 0, policy_version 1171813 (0.0005) [2023-12-26 23:54:34,468][105620] Updated weights for policy 1, policy_version 1173196 (0.0009) [2023-12-26 23:54:34,534][105620] Updated weights for policy 1, policy_version 1173206 (0.0009) [2023-12-26 23:54:34,595][105620] Updated weights for policy 1, policy_version 1173216 (0.0009) [2023-12-26 23:54:34,674][105692] Updated weights for policy 0, policy_version 1171823 (0.0008) [2023-12-26 23:54:34,721][105692] Updated weights for policy 0, policy_version 1171833 (0.0008) [2023-12-26 23:54:34,771][105692] Updated weights for policy 0, policy_version 1171844 (0.0010) [2023-12-26 23:54:35,271][105620] Updated weights for policy 1, policy_version 1173226 (0.0008) [2023-12-26 23:54:35,331][105620] Updated weights for policy 1, policy_version 1173236 (0.0008) [2023-12-26 23:54:35,389][105620] Updated weights for policy 1, policy_version 1173246 (0.0008) [2023-12-26 23:54:35,451][105620] Updated weights for policy 1, policy_version 1173256 (0.0008) [2023-12-26 23:54:35,590][105692] Updated weights for policy 0, policy_version 1171854 (0.0010) [2023-12-26 23:54:35,634][105692] Updated weights for policy 0, policy_version 1171864 (0.0010) [2023-12-26 23:54:35,684][105692] Updated weights for policy 0, policy_version 1171874 (0.0010) [2023-12-26 23:54:36,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 600440832. Throughput: 0: 9965.2, 1: 9599.8. Samples: 600432348. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:54:36,062][104569] Avg episode reward: [(0, '9261.535'), (1, '8986.937')] [2023-12-26 23:54:36,210][105620] Updated weights for policy 1, policy_version 1173266 (0.0008) [2023-12-26 23:54:36,263][105620] Updated weights for policy 1, policy_version 1173276 (0.0008) [2023-12-26 23:54:36,323][105620] Updated weights for policy 1, policy_version 1173286 (0.0008) [2023-12-26 23:54:36,439][105692] Updated weights for policy 0, policy_version 1171884 (0.0010) [2023-12-26 23:54:36,491][105692] Updated weights for policy 0, policy_version 1171894 (0.0010) [2023-12-26 23:54:36,557][105692] Updated weights for policy 0, policy_version 1171904 (0.0010) [2023-12-26 23:54:37,126][105620] Updated weights for policy 1, policy_version 1173296 (0.0010) [2023-12-26 23:54:37,180][105620] Updated weights for policy 1, policy_version 1173307 (0.0010) [2023-12-26 23:54:37,225][105692] Updated weights for policy 0, policy_version 1171914 (0.0008) [2023-12-26 23:54:37,233][105620] Updated weights for policy 1, policy_version 1173317 (0.0008) [2023-12-26 23:54:37,279][105692] Updated weights for policy 0, policy_version 1171924 (0.0006) [2023-12-26 23:54:37,324][105692] Updated weights for policy 0, policy_version 1171934 (0.0005) [2023-12-26 23:54:37,375][105692] Updated weights for policy 0, policy_version 1171944 (0.0006) [2023-12-26 23:54:37,992][105620] Updated weights for policy 1, policy_version 1173327 (0.0010) [2023-12-26 23:54:38,033][105692] Updated weights for policy 0, policy_version 1171954 (0.0009) [2023-12-26 23:54:38,042][105620] Updated weights for policy 1, policy_version 1173337 (0.0006) [2023-12-26 23:54:38,088][105620] Updated weights for policy 1, policy_version 1173347 (0.0006) [2023-12-26 23:54:38,093][105692] Updated weights for policy 0, policy_version 1171964 (0.0006) [2023-12-26 23:54:38,147][105692] Updated weights for policy 0, policy_version 1171974 (0.0008) [2023-12-26 23:54:38,789][105692] Updated weights for policy 0, policy_version 1171984 (0.0010) [2023-12-26 23:54:38,831][105620] Updated weights for policy 1, policy_version 1173357 (0.0008) [2023-12-26 23:54:38,851][105692] Updated weights for policy 0, policy_version 1171994 (0.0011) [2023-12-26 23:54:38,894][105620] Updated weights for policy 1, policy_version 1173367 (0.0005) [2023-12-26 23:54:38,907][105692] Updated weights for policy 0, policy_version 1172004 (0.0011) [2023-12-26 23:54:38,944][105620] Updated weights for policy 1, policy_version 1173377 (0.0007) [2023-12-26 23:54:39,676][105692] Updated weights for policy 0, policy_version 1172014 (0.0010) [2023-12-26 23:54:39,724][105620] Updated weights for policy 1, policy_version 1173387 (0.0007) [2023-12-26 23:54:39,740][105692] Updated weights for policy 0, policy_version 1172024 (0.0011) [2023-12-26 23:54:39,783][105620] Updated weights for policy 1, policy_version 1173397 (0.0007) [2023-12-26 23:54:39,799][105692] Updated weights for policy 0, policy_version 1172034 (0.0010) [2023-12-26 23:54:39,851][105620] Updated weights for policy 1, policy_version 1173407 (0.0007) [2023-12-26 23:54:40,504][105692] Updated weights for policy 0, policy_version 1172044 (0.0008) [2023-12-26 23:54:40,537][105620] Updated weights for policy 1, policy_version 1173417 (0.0006) [2023-12-26 23:54:40,558][105692] Updated weights for policy 0, policy_version 1172054 (0.0006) [2023-12-26 23:54:40,601][105620] Updated weights for policy 1, policy_version 1173427 (0.0006) [2023-12-26 23:54:40,608][105692] Updated weights for policy 0, policy_version 1172064 (0.0006) [2023-12-26 23:54:40,665][105620] Updated weights for policy 1, policy_version 1173437 (0.0005) [2023-12-26 23:54:40,731][105620] Updated weights for policy 1, policy_version 1173447 (0.0005) [2023-12-26 23:54:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 600539136. Throughput: 0: 9945.3, 1: 9637.7. Samples: 600548416. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:54:41,063][104569] Avg episode reward: [(0, '9258.356'), (1, '9170.937')] [2023-12-26 23:54:41,229][105692] Updated weights for policy 0, policy_version 1172074 (0.0007) [2023-12-26 23:54:41,292][105692] Updated weights for policy 0, policy_version 1172084 (0.0010) [2023-12-26 23:54:41,347][105692] Updated weights for policy 0, policy_version 1172094 (0.0010) [2023-12-26 23:54:41,402][105620] Updated weights for policy 1, policy_version 1173457 (0.0007) [2023-12-26 23:54:41,408][105692] Updated weights for policy 0, policy_version 1172104 (0.0008) [2023-12-26 23:54:41,453][105620] Updated weights for policy 1, policy_version 1173467 (0.0008) [2023-12-26 23:54:41,508][105620] Updated weights for policy 1, policy_version 1173478 (0.0010) [2023-12-26 23:54:42,111][105692] Updated weights for policy 0, policy_version 1172114 (0.0010) [2023-12-26 23:54:42,160][105692] Updated weights for policy 0, policy_version 1172124 (0.0009) [2023-12-26 23:54:42,223][105692] Updated weights for policy 0, policy_version 1172134 (0.0010) [2023-12-26 23:54:42,338][105620] Updated weights for policy 1, policy_version 1173488 (0.0008) [2023-12-26 23:54:42,409][105620] Updated weights for policy 1, policy_version 1173498 (0.0008) [2023-12-26 23:54:42,475][105620] Updated weights for policy 1, policy_version 1173508 (0.0010) [2023-12-26 23:54:42,915][105692] Updated weights for policy 0, policy_version 1172144 (0.0006) [2023-12-26 23:54:42,969][105692] Updated weights for policy 0, policy_version 1172154 (0.0009) [2023-12-26 23:54:43,021][105692] Updated weights for policy 0, policy_version 1172164 (0.0009) [2023-12-26 23:54:43,271][105620] Updated weights for policy 1, policy_version 1173518 (0.0009) [2023-12-26 23:54:43,330][105620] Updated weights for policy 1, policy_version 1173528 (0.0008) [2023-12-26 23:54:43,387][105620] Updated weights for policy 1, policy_version 1173538 (0.0007) [2023-12-26 23:54:43,622][105692] Updated weights for policy 0, policy_version 1172174 (0.0008) [2023-12-26 23:54:43,668][105692] Updated weights for policy 0, policy_version 1172184 (0.0008) [2023-12-26 23:54:43,726][105692] Updated weights for policy 0, policy_version 1172194 (0.0009) [2023-12-26 23:54:44,027][105620] Updated weights for policy 1, policy_version 1173548 (0.0008) [2023-12-26 23:54:44,075][105620] Updated weights for policy 1, policy_version 1173558 (0.0005) [2023-12-26 23:54:44,123][105620] Updated weights for policy 1, policy_version 1173568 (0.0005) [2023-12-26 23:54:44,432][105692] Updated weights for policy 0, policy_version 1172204 (0.0009) [2023-12-26 23:54:44,488][105692] Updated weights for policy 0, policy_version 1172214 (0.0008) [2023-12-26 23:54:44,554][105692] Updated weights for policy 0, policy_version 1172224 (0.0006) [2023-12-26 23:54:44,732][105620] Updated weights for policy 1, policy_version 1173578 (0.0006) [2023-12-26 23:54:44,790][105620] Updated weights for policy 1, policy_version 1173588 (0.0009) [2023-12-26 23:54:44,851][105620] Updated weights for policy 1, policy_version 1173598 (0.0008) [2023-12-26 23:54:44,912][105620] Updated weights for policy 1, policy_version 1173608 (0.0008) [2023-12-26 23:54:45,211][105692] Updated weights for policy 0, policy_version 1172234 (0.0006) [2023-12-26 23:54:45,280][105692] Updated weights for policy 0, policy_version 1172244 (0.0008) [2023-12-26 23:54:45,344][105692] Updated weights for policy 0, policy_version 1172254 (0.0009) [2023-12-26 23:54:45,407][105692] Updated weights for policy 0, policy_version 1172264 (0.0009) [2023-12-26 23:54:45,637][105620] Updated weights for policy 1, policy_version 1173618 (0.0006) [2023-12-26 23:54:45,684][105620] Updated weights for policy 1, policy_version 1173628 (0.0010) [2023-12-26 23:54:45,741][105620] Updated weights for policy 1, policy_version 1173638 (0.0009) [2023-12-26 23:54:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 600637440. Throughput: 0: 9966.2, 1: 9573.9. Samples: 600606748. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:54:46,062][104569] Avg episode reward: [(0, '9167.428'), (1, '9079.517')] [2023-12-26 23:54:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001172264_300146688.pth... [2023-12-26 23:54:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001173640_300490752.pth... [2023-12-26 23:54:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001171112_299851776.pth [2023-12-26 23:54:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001172488_300195840.pth [2023-12-26 23:54:46,146][105692] Updated weights for policy 0, policy_version 1172274 (0.0009) [2023-12-26 23:54:46,196][105692] Updated weights for policy 0, policy_version 1172284 (0.0009) [2023-12-26 23:54:46,251][105692] Updated weights for policy 0, policy_version 1172294 (0.0010) [2023-12-26 23:54:46,437][105620] Updated weights for policy 1, policy_version 1173648 (0.0009) [2023-12-26 23:54:46,486][105620] Updated weights for policy 1, policy_version 1173658 (0.0009) [2023-12-26 23:54:46,531][105620] Updated weights for policy 1, policy_version 1173668 (0.0008) [2023-12-26 23:54:46,946][105692] Updated weights for policy 0, policy_version 1172304 (0.0006) [2023-12-26 23:54:47,001][105692] Updated weights for policy 0, policy_version 1172314 (0.0005) [2023-12-26 23:54:47,047][105692] Updated weights for policy 0, policy_version 1172324 (0.0005) [2023-12-26 23:54:47,268][105620] Updated weights for policy 1, policy_version 1173678 (0.0006) [2023-12-26 23:54:47,316][105620] Updated weights for policy 1, policy_version 1173688 (0.0005) [2023-12-26 23:54:47,366][105620] Updated weights for policy 1, policy_version 1173698 (0.0008) [2023-12-26 23:54:47,731][105692] Updated weights for policy 0, policy_version 1172334 (0.0008) [2023-12-26 23:54:47,785][105692] Updated weights for policy 0, policy_version 1172344 (0.0010) [2023-12-26 23:54:47,838][105692] Updated weights for policy 0, policy_version 1172354 (0.0010) [2023-12-26 23:54:47,977][105620] Updated weights for policy 1, policy_version 1173708 (0.0008) [2023-12-26 23:54:48,044][105620] Updated weights for policy 1, policy_version 1173718 (0.0005) [2023-12-26 23:54:48,110][105620] Updated weights for policy 1, policy_version 1173728 (0.0005) [2023-12-26 23:54:48,664][105692] Updated weights for policy 0, policy_version 1172364 (0.0009) [2023-12-26 23:54:48,713][105692] Updated weights for policy 0, policy_version 1172374 (0.0009) [2023-12-26 23:54:48,745][105620] Updated weights for policy 1, policy_version 1173738 (0.0006) [2023-12-26 23:54:48,775][105692] Updated weights for policy 0, policy_version 1172384 (0.0008) [2023-12-26 23:54:48,802][105620] Updated weights for policy 1, policy_version 1173748 (0.0007) [2023-12-26 23:54:48,868][105620] Updated weights for policy 1, policy_version 1173758 (0.0008) [2023-12-26 23:54:48,934][105620] Updated weights for policy 1, policy_version 1173768 (0.0009) [2023-12-26 23:54:49,567][105692] Updated weights for policy 0, policy_version 1172394 (0.0008) [2023-12-26 23:54:49,617][105692] Updated weights for policy 0, policy_version 1172404 (0.0010) [2023-12-26 23:54:49,671][105692] Updated weights for policy 0, policy_version 1172414 (0.0007) [2023-12-26 23:54:49,673][105620] Updated weights for policy 1, policy_version 1173778 (0.0007) [2023-12-26 23:54:49,725][105692] Updated weights for policy 0, policy_version 1172424 (0.0006) [2023-12-26 23:54:49,730][105620] Updated weights for policy 1, policy_version 1173788 (0.0008) [2023-12-26 23:54:49,785][105620] Updated weights for policy 1, policy_version 1173798 (0.0008) [2023-12-26 23:54:50,498][105692] Updated weights for policy 0, policy_version 1172434 (0.0008) [2023-12-26 23:54:50,541][105620] Updated weights for policy 1, policy_version 1173808 (0.0008) [2023-12-26 23:54:50,555][105692] Updated weights for policy 0, policy_version 1172444 (0.0007) [2023-12-26 23:54:50,600][105620] Updated weights for policy 1, policy_version 1173818 (0.0008) [2023-12-26 23:54:50,620][105692] Updated weights for policy 0, policy_version 1172454 (0.0008) [2023-12-26 23:54:50,654][105620] Updated weights for policy 1, policy_version 1173828 (0.0008) [2023-12-26 23:54:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 600735744. Throughput: 0: 9819.1, 1: 9705.4. Samples: 600725684. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:54:51,062][104569] Avg episode reward: [(0, '9168.138'), (1, '8896.283')] [2023-12-26 23:54:51,370][105692] Updated weights for policy 0, policy_version 1172464 (0.0008) [2023-12-26 23:54:51,427][105692] Updated weights for policy 0, policy_version 1172474 (0.0008) [2023-12-26 23:54:51,434][105620] Updated weights for policy 1, policy_version 1173838 (0.0009) [2023-12-26 23:54:51,482][105692] Updated weights for policy 0, policy_version 1172484 (0.0005) [2023-12-26 23:54:51,483][105620] Updated weights for policy 1, policy_version 1173848 (0.0010) [2023-12-26 23:54:51,535][105620] Updated weights for policy 1, policy_version 1173858 (0.0009) [2023-12-26 23:54:52,163][105692] Updated weights for policy 0, policy_version 1172494 (0.0007) [2023-12-26 23:54:52,227][105692] Updated weights for policy 0, policy_version 1172504 (0.0008) [2023-12-26 23:54:52,260][105620] Updated weights for policy 1, policy_version 1173868 (0.0010) [2023-12-26 23:54:52,294][105692] Updated weights for policy 0, policy_version 1172514 (0.0007) [2023-12-26 23:54:52,316][105620] Updated weights for policy 1, policy_version 1173878 (0.0010) [2023-12-26 23:54:52,394][105620] Updated weights for policy 1, policy_version 1173888 (0.0012) [2023-12-26 23:54:53,013][105692] Updated weights for policy 0, policy_version 1172524 (0.0006) [2023-12-26 23:54:53,059][105692] Updated weights for policy 0, policy_version 1172534 (0.0005) [2023-12-26 23:54:53,119][105692] Updated weights for policy 0, policy_version 1172544 (0.0006) [2023-12-26 23:54:53,130][105620] Updated weights for policy 1, policy_version 1173898 (0.0010) [2023-12-26 23:54:53,186][105620] Updated weights for policy 1, policy_version 1173908 (0.0006) [2023-12-26 23:54:53,249][105620] Updated weights for policy 1, policy_version 1173918 (0.0009) [2023-12-26 23:54:53,299][105620] Updated weights for policy 1, policy_version 1173928 (0.0008) [2023-12-26 23:54:53,657][105692] Updated weights for policy 0, policy_version 1172554 (0.0006) [2023-12-26 23:54:53,721][105692] Updated weights for policy 0, policy_version 1172564 (0.0010) [2023-12-26 23:54:53,783][105692] Updated weights for policy 0, policy_version 1172574 (0.0010) [2023-12-26 23:54:53,841][105692] Updated weights for policy 0, policy_version 1172584 (0.0010) [2023-12-26 23:54:54,011][105620] Updated weights for policy 1, policy_version 1173938 (0.0008) [2023-12-26 23:54:54,071][105620] Updated weights for policy 1, policy_version 1173948 (0.0008) [2023-12-26 23:54:54,130][105620] Updated weights for policy 1, policy_version 1173958 (0.0008) [2023-12-26 23:54:54,514][105692] Updated weights for policy 0, policy_version 1172594 (0.0006) [2023-12-26 23:54:54,561][105692] Updated weights for policy 0, policy_version 1172604 (0.0005) [2023-12-26 23:54:54,625][105692] Updated weights for policy 0, policy_version 1172614 (0.0008) [2023-12-26 23:54:54,762][105620] Updated weights for policy 1, policy_version 1173968 (0.0006) [2023-12-26 23:54:54,824][105620] Updated weights for policy 1, policy_version 1173978 (0.0005) [2023-12-26 23:54:54,888][105620] Updated weights for policy 1, policy_version 1173988 (0.0005) [2023-12-26 23:54:55,238][105692] Updated weights for policy 0, policy_version 1172624 (0.0006) [2023-12-26 23:54:55,283][105692] Updated weights for policy 0, policy_version 1172634 (0.0005) [2023-12-26 23:54:55,337][105692] Updated weights for policy 0, policy_version 1172644 (0.0006) [2023-12-26 23:54:55,605][105620] Updated weights for policy 1, policy_version 1173998 (0.0008) [2023-12-26 23:54:55,664][105620] Updated weights for policy 1, policy_version 1174009 (0.0011) [2023-12-26 23:54:55,716][105620] Updated weights for policy 1, policy_version 1174019 (0.0009) [2023-12-26 23:54:55,891][105692] Updated weights for policy 0, policy_version 1172654 (0.0007) [2023-12-26 23:54:55,954][105692] Updated weights for policy 0, policy_version 1172664 (0.0006) [2023-12-26 23:54:56,011][105692] Updated weights for policy 0, policy_version 1172674 (0.0005) [2023-12-26 23:54:56,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.4, 300 sec: 19494.2). Total num frames: 600842240. Throughput: 0: 9850.0, 1: 9706.2. Samples: 600844192. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:54:56,062][104569] Avg episode reward: [(0, '8985.915'), (1, '8804.719')] [2023-12-26 23:54:56,593][105692] Updated weights for policy 0, policy_version 1172684 (0.0011) [2023-12-26 23:54:56,599][105620] Updated weights for policy 1, policy_version 1174029 (0.0009) [2023-12-26 23:54:56,637][105692] Updated weights for policy 0, policy_version 1172694 (0.0008) [2023-12-26 23:54:56,646][105620] Updated weights for policy 1, policy_version 1174039 (0.0007) [2023-12-26 23:54:56,682][105692] Updated weights for policy 0, policy_version 1172704 (0.0006) [2023-12-26 23:54:56,691][105620] Updated weights for policy 1, policy_version 1174049 (0.0009) [2023-12-26 23:54:57,344][105692] Updated weights for policy 0, policy_version 1172714 (0.0007) [2023-12-26 23:54:57,401][105692] Updated weights for policy 0, policy_version 1172724 (0.0008) [2023-12-26 23:54:57,450][105692] Updated weights for policy 0, policy_version 1172734 (0.0010) [2023-12-26 23:54:57,486][105620] Updated weights for policy 1, policy_version 1174059 (0.0009) [2023-12-26 23:54:57,505][105692] Updated weights for policy 0, policy_version 1172744 (0.0010) [2023-12-26 23:54:57,543][105620] Updated weights for policy 1, policy_version 1174069 (0.0007) [2023-12-26 23:54:57,612][105620] Updated weights for policy 1, policy_version 1174079 (0.0009) [2023-12-26 23:54:58,224][105692] Updated weights for policy 0, policy_version 1172754 (0.0007) [2023-12-26 23:54:58,279][105692] Updated weights for policy 0, policy_version 1172764 (0.0006) [2023-12-26 23:54:58,342][105692] Updated weights for policy 0, policy_version 1172774 (0.0006) [2023-12-26 23:54:58,380][105620] Updated weights for policy 1, policy_version 1174089 (0.0009) [2023-12-26 23:54:58,447][105620] Updated weights for policy 1, policy_version 1174099 (0.0008) [2023-12-26 23:54:58,511][105620] Updated weights for policy 1, policy_version 1174109 (0.0009) [2023-12-26 23:54:58,576][105620] Updated weights for policy 1, policy_version 1174119 (0.0008) [2023-12-26 23:54:59,182][105692] Updated weights for policy 0, policy_version 1172784 (0.0008) [2023-12-26 23:54:59,242][105692] Updated weights for policy 0, policy_version 1172794 (0.0008) [2023-12-26 23:54:59,311][105692] Updated weights for policy 0, policy_version 1172804 (0.0008) [2023-12-26 23:54:59,384][105620] Updated weights for policy 1, policy_version 1174129 (0.0007) [2023-12-26 23:54:59,439][105620] Updated weights for policy 1, policy_version 1174139 (0.0009) [2023-12-26 23:54:59,485][105620] Updated weights for policy 1, policy_version 1174149 (0.0008) [2023-12-26 23:55:00,078][105692] Updated weights for policy 0, policy_version 1172814 (0.0007) [2023-12-26 23:55:00,141][105692] Updated weights for policy 0, policy_version 1172824 (0.0005) [2023-12-26 23:55:00,207][105692] Updated weights for policy 0, policy_version 1172834 (0.0008) [2023-12-26 23:55:00,295][105620] Updated weights for policy 1, policy_version 1174159 (0.0010) [2023-12-26 23:55:00,349][105620] Updated weights for policy 1, policy_version 1174169 (0.0008) [2023-12-26 23:55:00,414][105620] Updated weights for policy 1, policy_version 1174179 (0.0009) [2023-12-26 23:55:00,746][105692] Updated weights for policy 0, policy_version 1172844 (0.0005) [2023-12-26 23:55:00,798][105692] Updated weights for policy 0, policy_version 1172854 (0.0005) [2023-12-26 23:55:00,848][105692] Updated weights for policy 0, policy_version 1172864 (0.0007) [2023-12-26 23:55:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 600932352. Throughput: 0: 9922.3, 1: 9673.2. Samples: 600903000. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:55:01,062][104569] Avg episode reward: [(0, '8985.195'), (1, '8987.045')] [2023-12-26 23:55:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001172872_300302336.pth... [2023-12-26 23:55:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001174184_300630016.pth... [2023-12-26 23:55:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001173096_300351488.pth [2023-12-26 23:55:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001171688_299999232.pth [2023-12-26 23:55:01,203][105620] Updated weights for policy 1, policy_version 1174189 (0.0009) [2023-12-26 23:55:01,266][105620] Updated weights for policy 1, policy_version 1174199 (0.0009) [2023-12-26 23:55:01,327][105620] Updated weights for policy 1, policy_version 1174209 (0.0009) [2023-12-26 23:55:01,552][105692] Updated weights for policy 0, policy_version 1172874 (0.0008) [2023-12-26 23:55:01,607][105692] Updated weights for policy 0, policy_version 1172884 (0.0008) [2023-12-26 23:55:01,674][105692] Updated weights for policy 0, policy_version 1172894 (0.0008) [2023-12-26 23:55:01,738][105692] Updated weights for policy 0, policy_version 1172904 (0.0009) [2023-12-26 23:55:02,011][105620] Updated weights for policy 1, policy_version 1174219 (0.0009) [2023-12-26 23:55:02,061][105620] Updated weights for policy 1, policy_version 1174229 (0.0008) [2023-12-26 23:55:02,114][105620] Updated weights for policy 1, policy_version 1174239 (0.0005) [2023-12-26 23:55:02,467][105692] Updated weights for policy 0, policy_version 1172914 (0.0005) [2023-12-26 23:55:02,524][105692] Updated weights for policy 0, policy_version 1172924 (0.0005) [2023-12-26 23:55:02,570][105692] Updated weights for policy 0, policy_version 1172934 (0.0005) [2023-12-26 23:55:02,910][105620] Updated weights for policy 1, policy_version 1174249 (0.0005) [2023-12-26 23:55:02,964][105620] Updated weights for policy 1, policy_version 1174259 (0.0009) [2023-12-26 23:55:03,017][105620] Updated weights for policy 1, policy_version 1174269 (0.0009) [2023-12-26 23:55:03,063][105620] Updated weights for policy 1, policy_version 1174279 (0.0009) [2023-12-26 23:55:03,178][105692] Updated weights for policy 0, policy_version 1172944 (0.0006) [2023-12-26 23:55:03,232][105692] Updated weights for policy 0, policy_version 1172954 (0.0006) [2023-12-26 23:55:03,293][105692] Updated weights for policy 0, policy_version 1172964 (0.0007) [2023-12-26 23:55:03,848][105692] Updated weights for policy 0, policy_version 1172974 (0.0006) [2023-12-26 23:55:03,904][105620] Updated weights for policy 1, policy_version 1174289 (0.0007) [2023-12-26 23:55:03,911][105692] Updated weights for policy 0, policy_version 1172984 (0.0008) [2023-12-26 23:55:03,967][105620] Updated weights for policy 1, policy_version 1174299 (0.0008) [2023-12-26 23:55:03,972][105692] Updated weights for policy 0, policy_version 1172994 (0.0008) [2023-12-26 23:55:04,031][105620] Updated weights for policy 1, policy_version 1174309 (0.0008) [2023-12-26 23:55:04,732][105692] Updated weights for policy 0, policy_version 1173004 (0.0009) [2023-12-26 23:55:04,788][105692] Updated weights for policy 0, policy_version 1173014 (0.0011) [2023-12-26 23:55:04,798][105620] Updated weights for policy 1, policy_version 1174319 (0.0007) [2023-12-26 23:55:04,837][105692] Updated weights for policy 0, policy_version 1173024 (0.0010) [2023-12-26 23:55:04,850][105620] Updated weights for policy 1, policy_version 1174329 (0.0006) [2023-12-26 23:55:04,908][105620] Updated weights for policy 1, policy_version 1174339 (0.0009) [2023-12-26 23:55:05,396][105692] Updated weights for policy 0, policy_version 1173034 (0.0006) [2023-12-26 23:55:05,447][105692] Updated weights for policy 0, policy_version 1173044 (0.0006) [2023-12-26 23:55:05,510][105692] Updated weights for policy 0, policy_version 1173054 (0.0006) [2023-12-26 23:55:05,571][105692] Updated weights for policy 0, policy_version 1173064 (0.0005) [2023-12-26 23:55:05,809][105620] Updated weights for policy 1, policy_version 1174349 (0.0009) [2023-12-26 23:55:05,867][105620] Updated weights for policy 1, policy_version 1174360 (0.0009) [2023-12-26 23:55:05,921][105620] Updated weights for policy 1, policy_version 1174370 (0.0009) [2023-12-26 23:55:06,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 601030656. Throughput: 0: 10017.0, 1: 9559.9. Samples: 601017652. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:55:06,063][104569] Avg episode reward: [(0, '9257.579'), (1, '9172.166')] [2023-12-26 23:55:06,158][105692] Updated weights for policy 0, policy_version 1173074 (0.0008) [2023-12-26 23:55:06,221][105692] Updated weights for policy 0, policy_version 1173084 (0.0006) [2023-12-26 23:55:06,289][105692] Updated weights for policy 0, policy_version 1173094 (0.0005) [2023-12-26 23:55:06,768][105620] Updated weights for policy 1, policy_version 1174380 (0.0009) [2023-12-26 23:55:06,816][105620] Updated weights for policy 1, policy_version 1174390 (0.0008) [2023-12-26 23:55:06,865][105620] Updated weights for policy 1, policy_version 1174400 (0.0008) [2023-12-26 23:55:06,944][105692] Updated weights for policy 0, policy_version 1173104 (0.0009) [2023-12-26 23:55:07,010][105692] Updated weights for policy 0, policy_version 1173114 (0.0010) [2023-12-26 23:55:07,072][105692] Updated weights for policy 0, policy_version 1173124 (0.0011) [2023-12-26 23:55:07,657][105620] Updated weights for policy 1, policy_version 1174410 (0.0008) [2023-12-26 23:55:07,707][105620] Updated weights for policy 1, policy_version 1174420 (0.0007) [2023-12-26 23:55:07,755][105620] Updated weights for policy 1, policy_version 1174430 (0.0008) [2023-12-26 23:55:07,806][105620] Updated weights for policy 1, policy_version 1174440 (0.0007) [2023-12-26 23:55:07,809][105692] Updated weights for policy 0, policy_version 1173134 (0.0010) [2023-12-26 23:55:07,862][105692] Updated weights for policy 0, policy_version 1173144 (0.0006) [2023-12-26 23:55:07,918][105692] Updated weights for policy 0, policy_version 1173154 (0.0005) [2023-12-26 23:55:08,472][105692] Updated weights for policy 0, policy_version 1173164 (0.0006) [2023-12-26 23:55:08,536][105692] Updated weights for policy 0, policy_version 1173174 (0.0009) [2023-12-26 23:55:08,603][105692] Updated weights for policy 0, policy_version 1173184 (0.0008) [2023-12-26 23:55:08,676][105620] Updated weights for policy 1, policy_version 1174450 (0.0007) [2023-12-26 23:55:08,736][105620] Updated weights for policy 1, policy_version 1174460 (0.0008) [2023-12-26 23:55:08,794][105620] Updated weights for policy 1, policy_version 1174470 (0.0008) [2023-12-26 23:55:09,339][105692] Updated weights for policy 0, policy_version 1173194 (0.0009) [2023-12-26 23:55:09,404][105692] Updated weights for policy 0, policy_version 1173204 (0.0009) [2023-12-26 23:55:09,469][105692] Updated weights for policy 0, policy_version 1173214 (0.0009) [2023-12-26 23:55:09,533][105692] Updated weights for policy 0, policy_version 1173224 (0.0011) [2023-12-26 23:55:09,554][105620] Updated weights for policy 1, policy_version 1174480 (0.0008) [2023-12-26 23:55:09,605][105620] Updated weights for policy 1, policy_version 1174490 (0.0008) [2023-12-26 23:55:09,661][105620] Updated weights for policy 1, policy_version 1174500 (0.0008) [2023-12-26 23:55:10,292][105692] Updated weights for policy 0, policy_version 1173234 (0.0008) [2023-12-26 23:55:10,359][105692] Updated weights for policy 0, policy_version 1173244 (0.0008) [2023-12-26 23:55:10,366][105620] Updated weights for policy 1, policy_version 1174510 (0.0008) [2023-12-26 23:55:10,415][105620] Updated weights for policy 1, policy_version 1174520 (0.0008) [2023-12-26 23:55:10,421][105692] Updated weights for policy 0, policy_version 1173254 (0.0008) [2023-12-26 23:55:10,471][105620] Updated weights for policy 1, policy_version 1174530 (0.0008) [2023-12-26 23:55:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 601120768. Throughput: 0: 10141.0, 1: 9433.6. Samples: 601133316. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:55:11,062][104569] Avg episode reward: [(0, '9348.598'), (1, '8895.611')] [2023-12-26 23:55:11,072][105692] Updated weights for policy 0, policy_version 1173264 (0.0007) [2023-12-26 23:55:11,142][105692] Updated weights for policy 0, policy_version 1173274 (0.0006) [2023-12-26 23:55:11,202][105620] Updated weights for policy 1, policy_version 1174540 (0.0007) [2023-12-26 23:55:11,204][105692] Updated weights for policy 0, policy_version 1173284 (0.0009) [2023-12-26 23:55:11,261][105620] Updated weights for policy 1, policy_version 1174550 (0.0007) [2023-12-26 23:55:11,317][105620] Updated weights for policy 1, policy_version 1174560 (0.0009) [2023-12-26 23:55:11,850][105692] Updated weights for policy 0, policy_version 1173294 (0.0011) [2023-12-26 23:55:11,914][105692] Updated weights for policy 0, policy_version 1173304 (0.0011) [2023-12-26 23:55:11,981][105692] Updated weights for policy 0, policy_version 1173314 (0.0011) [2023-12-26 23:55:12,088][105620] Updated weights for policy 1, policy_version 1174570 (0.0008) [2023-12-26 23:55:12,140][105620] Updated weights for policy 1, policy_version 1174580 (0.0006) [2023-12-26 23:55:12,187][105620] Updated weights for policy 1, policy_version 1174590 (0.0005) [2023-12-26 23:55:12,236][105620] Updated weights for policy 1, policy_version 1174600 (0.0005) [2023-12-26 23:55:12,782][105692] Updated weights for policy 0, policy_version 1173324 (0.0011) [2023-12-26 23:55:12,848][105692] Updated weights for policy 0, policy_version 1173334 (0.0011) [2023-12-26 23:55:12,911][105692] Updated weights for policy 0, policy_version 1173344 (0.0010) [2023-12-26 23:55:12,993][105620] Updated weights for policy 1, policy_version 1174610 (0.0007) [2023-12-26 23:55:13,051][105620] Updated weights for policy 1, policy_version 1174620 (0.0008) [2023-12-26 23:55:13,112][105620] Updated weights for policy 1, policy_version 1174630 (0.0008) [2023-12-26 23:55:13,642][105692] Updated weights for policy 0, policy_version 1173354 (0.0011) [2023-12-26 23:55:13,711][105692] Updated weights for policy 0, policy_version 1173364 (0.0010) [2023-12-26 23:55:13,759][105692] Updated weights for policy 0, policy_version 1173374 (0.0010) [2023-12-26 23:55:13,815][105692] Updated weights for policy 0, policy_version 1173384 (0.0010) [2023-12-26 23:55:13,860][105620] Updated weights for policy 1, policy_version 1174640 (0.0008) [2023-12-26 23:55:13,929][105620] Updated weights for policy 1, policy_version 1174650 (0.0008) [2023-12-26 23:55:13,984][105620] Updated weights for policy 1, policy_version 1174660 (0.0008) [2023-12-26 23:55:14,560][105692] Updated weights for policy 0, policy_version 1173394 (0.0010) [2023-12-26 23:55:14,620][105692] Updated weights for policy 0, policy_version 1173404 (0.0010) [2023-12-26 23:55:14,681][105692] Updated weights for policy 0, policy_version 1173414 (0.0006) [2023-12-26 23:55:14,738][105620] Updated weights for policy 1, policy_version 1174670 (0.0009) [2023-12-26 23:55:14,812][105620] Updated weights for policy 1, policy_version 1174680 (0.0008) [2023-12-26 23:55:14,872][105620] Updated weights for policy 1, policy_version 1174690 (0.0009) [2023-12-26 23:55:15,289][105692] Updated weights for policy 0, policy_version 1173424 (0.0010) [2023-12-26 23:55:15,334][105692] Updated weights for policy 0, policy_version 1173434 (0.0006) [2023-12-26 23:55:15,388][105692] Updated weights for policy 0, policy_version 1173444 (0.0010) [2023-12-26 23:55:15,735][105620] Updated weights for policy 1, policy_version 1174700 (0.0007) [2023-12-26 23:55:15,794][105620] Updated weights for policy 1, policy_version 1174710 (0.0005) [2023-12-26 23:55:15,864][105620] Updated weights for policy 1, policy_version 1174720 (0.0006) [2023-12-26 23:55:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 601219072. Throughput: 0: 10105.3, 1: 9394.9. Samples: 601190232. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:55:16,063][104569] Avg episode reward: [(0, '9257.740'), (1, '8986.890')] [2023-12-26 23:55:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001174728_300769280.pth... [2023-12-26 23:55:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001173640_300490752.pth [2023-12-26 23:55:16,082][105692] Updated weights for policy 0, policy_version 1173454 (0.0010) [2023-12-26 23:55:16,143][105692] Updated weights for policy 0, policy_version 1173464 (0.0010) [2023-12-26 23:55:16,198][105692] Updated weights for policy 0, policy_version 1173474 (0.0009) [2023-12-26 23:55:16,230][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001173480_300457984.pth... [2023-12-26 23:55:16,233][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001172264_300146688.pth [2023-12-26 23:55:16,483][105620] Updated weights for policy 1, policy_version 1174730 (0.0006) [2023-12-26 23:55:16,537][105620] Updated weights for policy 1, policy_version 1174740 (0.0009) [2023-12-26 23:55:16,586][105620] Updated weights for policy 1, policy_version 1174750 (0.0008) [2023-12-26 23:55:16,648][105620] Updated weights for policy 1, policy_version 1174760 (0.0009) [2023-12-26 23:55:16,947][105692] Updated weights for policy 0, policy_version 1173484 (0.0008) [2023-12-26 23:55:16,995][105692] Updated weights for policy 0, policy_version 1173494 (0.0005) [2023-12-26 23:55:17,050][105692] Updated weights for policy 0, policy_version 1173504 (0.0005) [2023-12-26 23:55:17,280][105620] Updated weights for policy 1, policy_version 1174770 (0.0005) [2023-12-26 23:55:17,347][105620] Updated weights for policy 1, policy_version 1174780 (0.0005) [2023-12-26 23:55:17,405][105620] Updated weights for policy 1, policy_version 1174790 (0.0005) [2023-12-26 23:55:17,738][105692] Updated weights for policy 0, policy_version 1173514 (0.0006) [2023-12-26 23:55:17,803][105692] Updated weights for policy 0, policy_version 1173524 (0.0009) [2023-12-26 23:55:17,864][105692] Updated weights for policy 0, policy_version 1173534 (0.0005) [2023-12-26 23:55:17,903][105620] Updated weights for policy 1, policy_version 1174800 (0.0005) [2023-12-26 23:55:17,921][105692] Updated weights for policy 0, policy_version 1173544 (0.0005) [2023-12-26 23:55:17,974][105620] Updated weights for policy 1, policy_version 1174810 (0.0006) [2023-12-26 23:55:18,042][105620] Updated weights for policy 1, policy_version 1174820 (0.0008) [2023-12-26 23:55:18,627][105692] Updated weights for policy 0, policy_version 1173554 (0.0009) [2023-12-26 23:55:18,668][105620] Updated weights for policy 1, policy_version 1174830 (0.0007) [2023-12-26 23:55:18,682][105692] Updated weights for policy 0, policy_version 1173564 (0.0008) [2023-12-26 23:55:18,729][105620] Updated weights for policy 1, policy_version 1174840 (0.0008) [2023-12-26 23:55:18,743][105692] Updated weights for policy 0, policy_version 1173574 (0.0006) [2023-12-26 23:55:18,791][105620] Updated weights for policy 1, policy_version 1174850 (0.0008) [2023-12-26 23:55:19,553][105692] Updated weights for policy 0, policy_version 1173584 (0.0008) [2023-12-26 23:55:19,566][105620] Updated weights for policy 1, policy_version 1174860 (0.0009) [2023-12-26 23:55:19,613][105692] Updated weights for policy 0, policy_version 1173594 (0.0008) [2023-12-26 23:55:19,627][105620] Updated weights for policy 1, policy_version 1174870 (0.0009) [2023-12-26 23:55:19,674][105692] Updated weights for policy 0, policy_version 1173604 (0.0007) [2023-12-26 23:55:19,688][105620] Updated weights for policy 1, policy_version 1174880 (0.0007) [2023-12-26 23:55:20,311][105692] Updated weights for policy 0, policy_version 1173614 (0.0005) [2023-12-26 23:55:20,368][105692] Updated weights for policy 0, policy_version 1173624 (0.0007) [2023-12-26 23:55:20,434][105692] Updated weights for policy 0, policy_version 1173634 (0.0007) [2023-12-26 23:55:20,546][105620] Updated weights for policy 1, policy_version 1174890 (0.0008) [2023-12-26 23:55:20,610][105620] Updated weights for policy 1, policy_version 1174900 (0.0008) [2023-12-26 23:55:20,669][105620] Updated weights for policy 1, policy_version 1174910 (0.0008) [2023-12-26 23:55:20,733][105620] Updated weights for policy 1, policy_version 1174920 (0.0008) [2023-12-26 23:55:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 601317376. Throughput: 0: 10062.2, 1: 9414.2. Samples: 601308788. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:55:21,063][104569] Avg episode reward: [(0, '9167.020'), (1, '9172.691')] [2023-12-26 23:55:21,166][105692] Updated weights for policy 0, policy_version 1173644 (0.0011) [2023-12-26 23:55:21,232][105692] Updated weights for policy 0, policy_version 1173654 (0.0010) [2023-12-26 23:55:21,292][105692] Updated weights for policy 0, policy_version 1173664 (0.0011) [2023-12-26 23:55:21,470][105620] Updated weights for policy 1, policy_version 1174930 (0.0008) [2023-12-26 23:55:21,530][105620] Updated weights for policy 1, policy_version 1174940 (0.0008) [2023-12-26 23:55:21,594][105620] Updated weights for policy 1, policy_version 1174950 (0.0006) [2023-12-26 23:55:22,072][105692] Updated weights for policy 0, policy_version 1173674 (0.0010) [2023-12-26 23:55:22,131][105692] Updated weights for policy 0, policy_version 1173684 (0.0010) [2023-12-26 23:55:22,195][105692] Updated weights for policy 0, policy_version 1173694 (0.0011) [2023-12-26 23:55:22,214][105620] Updated weights for policy 1, policy_version 1174960 (0.0006) [2023-12-26 23:55:22,255][105692] Updated weights for policy 0, policy_version 1173704 (0.0010) [2023-12-26 23:55:22,271][105620] Updated weights for policy 1, policy_version 1174970 (0.0007) [2023-12-26 23:55:22,325][105620] Updated weights for policy 1, policy_version 1174980 (0.0009) [2023-12-26 23:55:22,959][105620] Updated weights for policy 1, policy_version 1174990 (0.0010) [2023-12-26 23:55:23,015][105620] Updated weights for policy 1, policy_version 1175000 (0.0010) [2023-12-26 23:55:23,028][105692] Updated weights for policy 0, policy_version 1173714 (0.0010) [2023-12-26 23:55:23,065][105620] Updated weights for policy 1, policy_version 1175010 (0.0010) [2023-12-26 23:55:23,084][105692] Updated weights for policy 0, policy_version 1173724 (0.0010) [2023-12-26 23:55:23,140][105692] Updated weights for policy 0, policy_version 1173734 (0.0010) [2023-12-26 23:55:23,790][105620] Updated weights for policy 1, policy_version 1175020 (0.0008) [2023-12-26 23:55:23,842][105620] Updated weights for policy 1, policy_version 1175030 (0.0005) [2023-12-26 23:55:23,861][105692] Updated weights for policy 0, policy_version 1173744 (0.0010) [2023-12-26 23:55:23,901][105620] Updated weights for policy 1, policy_version 1175040 (0.0006) [2023-12-26 23:55:23,924][105692] Updated weights for policy 0, policy_version 1173754 (0.0010) [2023-12-26 23:55:23,982][105692] Updated weights for policy 0, policy_version 1173764 (0.0011) [2023-12-26 23:55:24,570][105620] Updated weights for policy 1, policy_version 1175050 (0.0011) [2023-12-26 23:55:24,610][105692] Updated weights for policy 0, policy_version 1173774 (0.0010) [2023-12-26 23:55:24,633][105620] Updated weights for policy 1, policy_version 1175060 (0.0005) [2023-12-26 23:55:24,666][105692] Updated weights for policy 0, policy_version 1173784 (0.0008) [2023-12-26 23:55:24,688][105620] Updated weights for policy 1, policy_version 1175070 (0.0005) [2023-12-26 23:55:24,723][105692] Updated weights for policy 0, policy_version 1173794 (0.0008) [2023-12-26 23:55:24,742][105620] Updated weights for policy 1, policy_version 1175080 (0.0005) [2023-12-26 23:55:25,301][105620] Updated weights for policy 1, policy_version 1175090 (0.0005) [2023-12-26 23:55:25,366][105620] Updated weights for policy 1, policy_version 1175100 (0.0005) [2023-12-26 23:55:25,429][105620] Updated weights for policy 1, policy_version 1175110 (0.0006) [2023-12-26 23:55:25,597][105692] Updated weights for policy 0, policy_version 1173804 (0.0009) [2023-12-26 23:55:25,662][105692] Updated weights for policy 0, policy_version 1173814 (0.0009) [2023-12-26 23:55:25,728][105692] Updated weights for policy 0, policy_version 1173824 (0.0009) [2023-12-26 23:55:25,959][105620] Updated weights for policy 1, policy_version 1175120 (0.0006) [2023-12-26 23:55:26,007][105620] Updated weights for policy 1, policy_version 1175130 (0.0005) [2023-12-26 23:55:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 601415680. Throughput: 0: 10015.5, 1: 9539.0. Samples: 601428372. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:55:26,063][104569] Avg episode reward: [(0, '9075.412'), (1, '8990.105')] [2023-12-26 23:55:26,068][105620] Updated weights for policy 1, policy_version 1175140 (0.0005) [2023-12-26 23:55:26,585][105620] Updated weights for policy 1, policy_version 1175150 (0.0005) [2023-12-26 23:55:26,602][105692] Updated weights for policy 0, policy_version 1173834 (0.0009) [2023-12-26 23:55:26,645][105620] Updated weights for policy 1, policy_version 1175160 (0.0005) [2023-12-26 23:55:26,653][105692] Updated weights for policy 0, policy_version 1173844 (0.0007) [2023-12-26 23:55:26,699][105620] Updated weights for policy 1, policy_version 1175170 (0.0005) [2023-12-26 23:55:26,706][105692] Updated weights for policy 0, policy_version 1173854 (0.0009) [2023-12-26 23:55:26,757][105692] Updated weights for policy 0, policy_version 1173864 (0.0009) [2023-12-26 23:55:27,192][105620] Updated weights for policy 1, policy_version 1175180 (0.0005) [2023-12-26 23:55:27,247][105620] Updated weights for policy 1, policy_version 1175190 (0.0005) [2023-12-26 23:55:27,302][105620] Updated weights for policy 1, policy_version 1175200 (0.0006) [2023-12-26 23:55:27,642][105692] Updated weights for policy 0, policy_version 1173874 (0.0008) [2023-12-26 23:55:27,695][105692] Updated weights for policy 0, policy_version 1173884 (0.0010) [2023-12-26 23:55:27,749][105692] Updated weights for policy 0, policy_version 1173894 (0.0009) [2023-12-26 23:55:27,821][105620] Updated weights for policy 1, policy_version 1175210 (0.0006) [2023-12-26 23:55:27,879][105620] Updated weights for policy 1, policy_version 1175220 (0.0005) [2023-12-26 23:55:27,946][105620] Updated weights for policy 1, policy_version 1175230 (0.0005) [2023-12-26 23:55:28,007][105620] Updated weights for policy 1, policy_version 1175240 (0.0010) [2023-12-26 23:55:28,550][105692] Updated weights for policy 0, policy_version 1173904 (0.0009) [2023-12-26 23:55:28,599][105692] Updated weights for policy 0, policy_version 1173914 (0.0008) [2023-12-26 23:55:28,654][105692] Updated weights for policy 0, policy_version 1173924 (0.0008) [2023-12-26 23:55:28,675][105620] Updated weights for policy 1, policy_version 1175250 (0.0010) [2023-12-26 23:55:28,733][105620] Updated weights for policy 1, policy_version 1175260 (0.0010) [2023-12-26 23:55:28,780][105620] Updated weights for policy 1, policy_version 1175270 (0.0010) [2023-12-26 23:55:29,429][105692] Updated weights for policy 0, policy_version 1173934 (0.0008) [2023-12-26 23:55:29,474][105692] Updated weights for policy 0, policy_version 1173944 (0.0008) [2023-12-26 23:55:29,522][105692] Updated weights for policy 0, policy_version 1173954 (0.0008) [2023-12-26 23:55:29,539][105620] Updated weights for policy 1, policy_version 1175280 (0.0010) [2023-12-26 23:55:29,590][105620] Updated weights for policy 1, policy_version 1175290 (0.0008) [2023-12-26 23:55:29,650][105620] Updated weights for policy 1, policy_version 1175300 (0.0011) [2023-12-26 23:55:30,294][105692] Updated weights for policy 0, policy_version 1173964 (0.0007) [2023-12-26 23:55:30,342][105692] Updated weights for policy 0, policy_version 1173974 (0.0009) [2023-12-26 23:55:30,395][105692] Updated weights for policy 0, policy_version 1173984 (0.0009) [2023-12-26 23:55:30,406][105620] Updated weights for policy 1, policy_version 1175310 (0.0008) [2023-12-26 23:55:30,460][105620] Updated weights for policy 1, policy_version 1175320 (0.0007) [2023-12-26 23:55:30,507][105620] Updated weights for policy 1, policy_version 1175330 (0.0008) [2023-12-26 23:55:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 601513984. Throughput: 0: 9880.4, 1: 9719.6. Samples: 601488752. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:55:31,063][104569] Avg episode reward: [(0, '9075.179'), (1, '9172.287')] [2023-12-26 23:55:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001173992_300589056.pth... [2023-12-26 23:55:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001175336_300924928.pth... [2023-12-26 23:55:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001174184_300630016.pth [2023-12-26 23:55:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001172872_300302336.pth [2023-12-26 23:55:31,174][105620] Updated weights for policy 1, policy_version 1175340 (0.0009) [2023-12-26 23:55:31,182][105692] Updated weights for policy 0, policy_version 1173994 (0.0008) [2023-12-26 23:55:31,228][105692] Updated weights for policy 0, policy_version 1174004 (0.0006) [2023-12-26 23:55:31,229][105620] Updated weights for policy 1, policy_version 1175350 (0.0010) [2023-12-26 23:55:31,289][105692] Updated weights for policy 0, policy_version 1174014 (0.0008) [2023-12-26 23:55:31,294][105620] Updated weights for policy 1, policy_version 1175360 (0.0008) [2023-12-26 23:55:31,350][105692] Updated weights for policy 0, policy_version 1174024 (0.0007) [2023-12-26 23:55:32,032][105620] Updated weights for policy 1, policy_version 1175370 (0.0009) [2023-12-26 23:55:32,057][105692] Updated weights for policy 0, policy_version 1174034 (0.0005) [2023-12-26 23:55:32,083][105620] Updated weights for policy 1, policy_version 1175380 (0.0010) [2023-12-26 23:55:32,107][105692] Updated weights for policy 0, policy_version 1174044 (0.0005) [2023-12-26 23:55:32,132][105620] Updated weights for policy 1, policy_version 1175390 (0.0010) [2023-12-26 23:55:32,152][105692] Updated weights for policy 0, policy_version 1174054 (0.0005) [2023-12-26 23:55:32,190][105620] Updated weights for policy 1, policy_version 1175400 (0.0008) [2023-12-26 23:55:32,826][105692] Updated weights for policy 0, policy_version 1174064 (0.0007) [2023-12-26 23:55:32,877][105620] Updated weights for policy 1, policy_version 1175410 (0.0005) [2023-12-26 23:55:32,879][105692] Updated weights for policy 0, policy_version 1174075 (0.0009) [2023-12-26 23:55:32,931][105620] Updated weights for policy 1, policy_version 1175420 (0.0005) [2023-12-26 23:55:32,941][105692] Updated weights for policy 0, policy_version 1174085 (0.0007) [2023-12-26 23:55:32,998][105620] Updated weights for policy 1, policy_version 1175430 (0.0007) [2023-12-26 23:55:33,560][105692] Updated weights for policy 0, policy_version 1174095 (0.0008) [2023-12-26 23:55:33,614][105692] Updated weights for policy 0, policy_version 1174105 (0.0009) [2023-12-26 23:55:33,618][105620] Updated weights for policy 1, policy_version 1175440 (0.0006) [2023-12-26 23:55:33,665][105692] Updated weights for policy 0, policy_version 1174115 (0.0008) [2023-12-26 23:55:33,670][105620] Updated weights for policy 1, policy_version 1175450 (0.0005) [2023-12-26 23:55:33,728][105620] Updated weights for policy 1, policy_version 1175460 (0.0005) [2023-12-26 23:55:34,416][105692] Updated weights for policy 0, policy_version 1174125 (0.0009) [2023-12-26 23:55:34,433][105620] Updated weights for policy 1, policy_version 1175470 (0.0008) [2023-12-26 23:55:34,466][105692] Updated weights for policy 0, policy_version 1174135 (0.0008) [2023-12-26 23:55:34,495][105620] Updated weights for policy 1, policy_version 1175480 (0.0009) [2023-12-26 23:55:34,518][105692] Updated weights for policy 0, policy_version 1174145 (0.0008) [2023-12-26 23:55:34,549][105620] Updated weights for policy 1, policy_version 1175490 (0.0007) [2023-12-26 23:55:35,271][105620] Updated weights for policy 1, policy_version 1175500 (0.0008) [2023-12-26 23:55:35,295][105692] Updated weights for policy 0, policy_version 1174155 (0.0006) [2023-12-26 23:55:35,331][105620] Updated weights for policy 1, policy_version 1175510 (0.0009) [2023-12-26 23:55:35,349][105692] Updated weights for policy 0, policy_version 1174165 (0.0005) [2023-12-26 23:55:35,388][105620] Updated weights for policy 1, policy_version 1175520 (0.0009) [2023-12-26 23:55:35,399][105692] Updated weights for policy 0, policy_version 1174175 (0.0005) [2023-12-26 23:55:35,943][105692] Updated weights for policy 0, policy_version 1174185 (0.0006) [2023-12-26 23:55:36,001][105692] Updated weights for policy 0, policy_version 1174195 (0.0009) [2023-12-26 23:55:36,053][105692] Updated weights for policy 0, policy_version 1174205 (0.0009) [2023-12-26 23:55:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 601612288. Throughput: 0: 9883.3, 1: 9672.6. Samples: 601605704. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:55:36,063][104569] Avg episode reward: [(0, '8984.058'), (1, '9171.918')] [2023-12-26 23:55:36,120][105692] Updated weights for policy 0, policy_version 1174215 (0.0007) [2023-12-26 23:55:36,201][105620] Updated weights for policy 1, policy_version 1175530 (0.0009) [2023-12-26 23:55:36,257][105620] Updated weights for policy 1, policy_version 1175540 (0.0010) [2023-12-26 23:55:36,306][105620] Updated weights for policy 1, policy_version 1175550 (0.0010) [2023-12-26 23:55:36,372][105620] Updated weights for policy 1, policy_version 1175560 (0.0010) [2023-12-26 23:55:36,850][105692] Updated weights for policy 0, policy_version 1174225 (0.0011) [2023-12-26 23:55:36,902][105692] Updated weights for policy 0, policy_version 1174235 (0.0010) [2023-12-26 23:55:36,961][105692] Updated weights for policy 0, policy_version 1174245 (0.0010) [2023-12-26 23:55:37,115][105620] Updated weights for policy 1, policy_version 1175570 (0.0010) [2023-12-26 23:55:37,175][105620] Updated weights for policy 1, policy_version 1175580 (0.0009) [2023-12-26 23:55:37,231][105620] Updated weights for policy 1, policy_version 1175590 (0.0005) [2023-12-26 23:55:37,694][105692] Updated weights for policy 0, policy_version 1174255 (0.0007) [2023-12-26 23:55:37,763][105692] Updated weights for policy 0, policy_version 1174265 (0.0006) [2023-12-26 23:55:37,817][105620] Updated weights for policy 1, policy_version 1175600 (0.0010) [2023-12-26 23:55:37,831][105692] Updated weights for policy 0, policy_version 1174275 (0.0006) [2023-12-26 23:55:37,889][105620] Updated weights for policy 1, policy_version 1175610 (0.0010) [2023-12-26 23:55:37,942][105620] Updated weights for policy 1, policy_version 1175620 (0.0010) [2023-12-26 23:55:38,493][105692] Updated weights for policy 0, policy_version 1174285 (0.0007) [2023-12-26 23:55:38,562][105692] Updated weights for policy 0, policy_version 1174295 (0.0005) [2023-12-26 23:55:38,617][105692] Updated weights for policy 0, policy_version 1174305 (0.0006) [2023-12-26 23:55:38,671][105620] Updated weights for policy 1, policy_version 1175630 (0.0011) [2023-12-26 23:55:38,716][105620] Updated weights for policy 1, policy_version 1175640 (0.0010) [2023-12-26 23:55:38,765][105620] Updated weights for policy 1, policy_version 1175650 (0.0010) [2023-12-26 23:55:39,188][105692] Updated weights for policy 0, policy_version 1174315 (0.0005) [2023-12-26 23:55:39,258][105692] Updated weights for policy 0, policy_version 1174325 (0.0008) [2023-12-26 23:55:39,330][105692] Updated weights for policy 0, policy_version 1174335 (0.0010) [2023-12-26 23:55:39,571][105620] Updated weights for policy 1, policy_version 1175660 (0.0010) [2023-12-26 23:55:39,634][105620] Updated weights for policy 1, policy_version 1175670 (0.0010) [2023-12-26 23:55:39,701][105620] Updated weights for policy 1, policy_version 1175680 (0.0011) [2023-12-26 23:55:40,035][105692] Updated weights for policy 0, policy_version 1174345 (0.0011) [2023-12-26 23:55:40,083][105692] Updated weights for policy 0, policy_version 1174355 (0.0008) [2023-12-26 23:55:40,140][105692] Updated weights for policy 0, policy_version 1174365 (0.0008) [2023-12-26 23:55:40,200][105692] Updated weights for policy 0, policy_version 1174375 (0.0008) [2023-12-26 23:55:40,399][105620] Updated weights for policy 1, policy_version 1175690 (0.0007) [2023-12-26 23:55:40,453][105620] Updated weights for policy 1, policy_version 1175700 (0.0005) [2023-12-26 23:55:40,510][105620] Updated weights for policy 1, policy_version 1175710 (0.0007) [2023-12-26 23:55:40,567][105620] Updated weights for policy 1, policy_version 1175720 (0.0009) [2023-12-26 23:55:41,013][105692] Updated weights for policy 0, policy_version 1174385 (0.0009) [2023-12-26 23:55:41,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 601710592. Throughput: 0: 9859.5, 1: 9700.9. Samples: 601724408. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:55:41,062][104569] Avg episode reward: [(0, '9258.464'), (1, '9079.458')] [2023-12-26 23:55:41,082][105692] Updated weights for policy 0, policy_version 1174395 (0.0009) [2023-12-26 23:55:41,142][105692] Updated weights for policy 0, policy_version 1174406 (0.0010) [2023-12-26 23:55:41,302][105620] Updated weights for policy 1, policy_version 1175730 (0.0007) [2023-12-26 23:55:41,369][105620] Updated weights for policy 1, policy_version 1175740 (0.0009) [2023-12-26 23:55:41,433][105620] Updated weights for policy 1, policy_version 1175750 (0.0009) [2023-12-26 23:55:41,922][105692] Updated weights for policy 0, policy_version 1174416 (0.0007) [2023-12-26 23:55:41,972][105692] Updated weights for policy 0, policy_version 1174426 (0.0006) [2023-12-26 23:55:42,019][105692] Updated weights for policy 0, policy_version 1174436 (0.0009) [2023-12-26 23:55:42,200][105620] Updated weights for policy 1, policy_version 1175760 (0.0009) [2023-12-26 23:55:42,258][105620] Updated weights for policy 1, policy_version 1175770 (0.0008) [2023-12-26 23:55:42,309][105620] Updated weights for policy 1, policy_version 1175780 (0.0008) [2023-12-26 23:55:42,653][105692] Updated weights for policy 0, policy_version 1174446 (0.0009) [2023-12-26 23:55:42,701][105692] Updated weights for policy 0, policy_version 1174456 (0.0009) [2023-12-26 23:55:42,766][105692] Updated weights for policy 0, policy_version 1174466 (0.0008) [2023-12-26 23:55:43,110][105620] Updated weights for policy 1, policy_version 1175790 (0.0009) [2023-12-26 23:55:43,172][105620] Updated weights for policy 1, policy_version 1175800 (0.0008) [2023-12-26 23:55:43,238][105620] Updated weights for policy 1, policy_version 1175810 (0.0009) [2023-12-26 23:55:43,489][105692] Updated weights for policy 0, policy_version 1174476 (0.0009) [2023-12-26 23:55:43,541][105692] Updated weights for policy 0, policy_version 1174486 (0.0008) [2023-12-26 23:55:43,590][105692] Updated weights for policy 0, policy_version 1174496 (0.0009) [2023-12-26 23:55:44,019][105620] Updated weights for policy 1, policy_version 1175820 (0.0009) [2023-12-26 23:55:44,075][105620] Updated weights for policy 1, policy_version 1175830 (0.0010) [2023-12-26 23:55:44,129][105620] Updated weights for policy 1, policy_version 1175840 (0.0008) [2023-12-26 23:55:44,295][105692] Updated weights for policy 0, policy_version 1174506 (0.0009) [2023-12-26 23:55:44,352][105692] Updated weights for policy 0, policy_version 1174516 (0.0008) [2023-12-26 23:55:44,416][105692] Updated weights for policy 0, policy_version 1174526 (0.0010) [2023-12-26 23:55:44,475][105692] Updated weights for policy 0, policy_version 1174536 (0.0009) [2023-12-26 23:55:44,817][105620] Updated weights for policy 1, policy_version 1175850 (0.0009) [2023-12-26 23:55:44,883][105620] Updated weights for policy 1, policy_version 1175860 (0.0010) [2023-12-26 23:55:44,945][105620] Updated weights for policy 1, policy_version 1175870 (0.0010) [2023-12-26 23:55:45,009][105620] Updated weights for policy 1, policy_version 1175880 (0.0010) [2023-12-26 23:55:45,172][105692] Updated weights for policy 0, policy_version 1174546 (0.0011) [2023-12-26 23:55:45,235][105692] Updated weights for policy 0, policy_version 1174556 (0.0011) [2023-12-26 23:55:45,302][105692] Updated weights for policy 0, policy_version 1174566 (0.0011) [2023-12-26 23:55:45,687][105620] Updated weights for policy 1, policy_version 1175890 (0.0009) [2023-12-26 23:55:45,750][105620] Updated weights for policy 1, policy_version 1175900 (0.0011) [2023-12-26 23:55:45,812][105620] Updated weights for policy 1, policy_version 1175910 (0.0010) [2023-12-26 23:55:45,955][105692] Updated weights for policy 0, policy_version 1174576 (0.0009) [2023-12-26 23:55:46,015][105692] Updated weights for policy 0, policy_version 1174586 (0.0009) [2023-12-26 23:55:46,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 601808896. Throughput: 0: 9781.2, 1: 9713.7. Samples: 601780272. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:55:46,062][104569] Avg episode reward: [(0, '9349.116'), (1, '8898.438')] [2023-12-26 23:55:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001175912_301072384.pth... [2023-12-26 23:55:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001174728_300769280.pth [2023-12-26 23:55:46,080][105692] Updated weights for policy 0, policy_version 1174596 (0.0008) [2023-12-26 23:55:46,103][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001174600_300744704.pth... [2023-12-26 23:55:46,108][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001173480_300457984.pth [2023-12-26 23:55:46,535][105620] Updated weights for policy 1, policy_version 1175920 (0.0007) [2023-12-26 23:55:46,605][105620] Updated weights for policy 1, policy_version 1175930 (0.0005) [2023-12-26 23:55:46,672][105620] Updated weights for policy 1, policy_version 1175940 (0.0005) [2023-12-26 23:55:46,682][105692] Updated weights for policy 0, policy_version 1174606 (0.0006) [2023-12-26 23:55:46,733][105692] Updated weights for policy 0, policy_version 1174616 (0.0005) [2023-12-26 23:55:46,791][105692] Updated weights for policy 0, policy_version 1174626 (0.0005) [2023-12-26 23:55:47,300][105620] Updated weights for policy 1, policy_version 1175950 (0.0006) [2023-12-26 23:55:47,302][105692] Updated weights for policy 0, policy_version 1174636 (0.0006) [2023-12-26 23:55:47,354][105692] Updated weights for policy 0, policy_version 1174646 (0.0006) [2023-12-26 23:55:47,362][105620] Updated weights for policy 1, policy_version 1175960 (0.0005) [2023-12-26 23:55:47,406][105692] Updated weights for policy 0, policy_version 1174656 (0.0010) [2023-12-26 23:55:47,416][105620] Updated weights for policy 1, policy_version 1175970 (0.0005) [2023-12-26 23:55:47,991][105692] Updated weights for policy 0, policy_version 1174666 (0.0010) [2023-12-26 23:55:48,044][105692] Updated weights for policy 0, policy_version 1174676 (0.0010) [2023-12-26 23:55:48,095][105692] Updated weights for policy 0, policy_version 1174686 (0.0010) [2023-12-26 23:55:48,125][105620] Updated weights for policy 1, policy_version 1175980 (0.0008) [2023-12-26 23:55:48,154][105692] Updated weights for policy 0, policy_version 1174696 (0.0010) [2023-12-26 23:55:48,179][105620] Updated weights for policy 1, policy_version 1175990 (0.0007) [2023-12-26 23:55:48,231][105620] Updated weights for policy 1, policy_version 1176000 (0.0008) [2023-12-26 23:55:48,877][105692] Updated weights for policy 0, policy_version 1174706 (0.0006) [2023-12-26 23:55:48,933][105692] Updated weights for policy 0, policy_version 1174716 (0.0005) [2023-12-26 23:55:48,974][105620] Updated weights for policy 1, policy_version 1176010 (0.0007) [2023-12-26 23:55:48,992][105692] Updated weights for policy 0, policy_version 1174726 (0.0010) [2023-12-26 23:55:49,031][105620] Updated weights for policy 1, policy_version 1176020 (0.0007) [2023-12-26 23:55:49,079][105620] Updated weights for policy 1, policy_version 1176030 (0.0008) [2023-12-26 23:55:49,123][105620] Updated weights for policy 1, policy_version 1176040 (0.0008) [2023-12-26 23:55:49,717][105692] Updated weights for policy 0, policy_version 1174736 (0.0010) [2023-12-26 23:55:49,765][105692] Updated weights for policy 0, policy_version 1174746 (0.0010) [2023-12-26 23:55:49,829][105692] Updated weights for policy 0, policy_version 1174756 (0.0010) [2023-12-26 23:55:49,913][105620] Updated weights for policy 1, policy_version 1176050 (0.0009) [2023-12-26 23:55:49,974][105620] Updated weights for policy 1, policy_version 1176060 (0.0008) [2023-12-26 23:55:50,024][105620] Updated weights for policy 1, policy_version 1176070 (0.0009) [2023-12-26 23:55:50,572][105692] Updated weights for policy 0, policy_version 1174766 (0.0010) [2023-12-26 23:55:50,632][105692] Updated weights for policy 0, policy_version 1174776 (0.0011) [2023-12-26 23:55:50,695][105692] Updated weights for policy 0, policy_version 1174786 (0.0010) [2023-12-26 23:55:50,824][105620] Updated weights for policy 1, policy_version 1176080 (0.0007) [2023-12-26 23:55:50,890][105620] Updated weights for policy 1, policy_version 1176090 (0.0008) [2023-12-26 23:55:50,960][105620] Updated weights for policy 1, policy_version 1176100 (0.0006) [2023-12-26 23:55:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 601915392. Throughput: 0: 9845.6, 1: 9816.5. Samples: 601902444. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:55:51,062][104569] Avg episode reward: [(0, '9257.936'), (1, '8806.488')] [2023-12-26 23:55:51,460][105692] Updated weights for policy 0, policy_version 1174796 (0.0011) [2023-12-26 23:55:51,509][105692] Updated weights for policy 0, policy_version 1174806 (0.0010) [2023-12-26 23:55:51,568][105692] Updated weights for policy 0, policy_version 1174816 (0.0010) [2023-12-26 23:55:51,637][105620] Updated weights for policy 1, policy_version 1176110 (0.0007) [2023-12-26 23:55:51,692][105620] Updated weights for policy 1, policy_version 1176120 (0.0008) [2023-12-26 23:55:51,759][105620] Updated weights for policy 1, policy_version 1176130 (0.0008) [2023-12-26 23:55:52,288][105692] Updated weights for policy 0, policy_version 1174826 (0.0010) [2023-12-26 23:55:52,347][105692] Updated weights for policy 0, policy_version 1174836 (0.0010) [2023-12-26 23:55:52,418][105692] Updated weights for policy 0, policy_version 1174846 (0.0009) [2023-12-26 23:55:52,475][105620] Updated weights for policy 1, policy_version 1176140 (0.0007) [2023-12-26 23:55:52,480][105692] Updated weights for policy 0, policy_version 1174856 (0.0010) [2023-12-26 23:55:52,537][105620] Updated weights for policy 1, policy_version 1176150 (0.0008) [2023-12-26 23:55:52,593][105620] Updated weights for policy 1, policy_version 1176160 (0.0008) [2023-12-26 23:55:53,212][105692] Updated weights for policy 0, policy_version 1174866 (0.0008) [2023-12-26 23:55:53,277][105692] Updated weights for policy 0, policy_version 1174876 (0.0009) [2023-12-26 23:55:53,335][105692] Updated weights for policy 0, policy_version 1174886 (0.0008) [2023-12-26 23:55:53,356][105620] Updated weights for policy 1, policy_version 1176170 (0.0007) [2023-12-26 23:55:53,419][105620] Updated weights for policy 1, policy_version 1176180 (0.0009) [2023-12-26 23:55:53,479][105620] Updated weights for policy 1, policy_version 1176190 (0.0009) [2023-12-26 23:55:53,549][105620] Updated weights for policy 1, policy_version 1176200 (0.0010) [2023-12-26 23:55:54,042][105692] Updated weights for policy 0, policy_version 1174896 (0.0006) [2023-12-26 23:55:54,107][105692] Updated weights for policy 0, policy_version 1174906 (0.0007) [2023-12-26 23:55:54,169][105692] Updated weights for policy 0, policy_version 1174916 (0.0009) [2023-12-26 23:55:54,310][105620] Updated weights for policy 1, policy_version 1176210 (0.0009) [2023-12-26 23:55:54,368][105620] Updated weights for policy 1, policy_version 1176220 (0.0009) [2023-12-26 23:55:54,433][105620] Updated weights for policy 1, policy_version 1176230 (0.0009) [2023-12-26 23:55:54,777][105692] Updated weights for policy 0, policy_version 1174926 (0.0008) [2023-12-26 23:55:54,837][105692] Updated weights for policy 0, policy_version 1174936 (0.0009) [2023-12-26 23:55:54,899][105692] Updated weights for policy 0, policy_version 1174946 (0.0009) [2023-12-26 23:55:55,238][105620] Updated weights for policy 1, policy_version 1176240 (0.0009) [2023-12-26 23:55:55,300][105620] Updated weights for policy 1, policy_version 1176250 (0.0009) [2023-12-26 23:55:55,357][105620] Updated weights for policy 1, policy_version 1176260 (0.0009) [2023-12-26 23:55:55,623][105692] Updated weights for policy 0, policy_version 1174956 (0.0007) [2023-12-26 23:55:55,681][105692] Updated weights for policy 0, policy_version 1174966 (0.0010) [2023-12-26 23:55:55,744][105692] Updated weights for policy 0, policy_version 1174976 (0.0009) [2023-12-26 23:55:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 602005504. Throughput: 0: 9755.0, 1: 9846.6. Samples: 602015392. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:55:56,063][104569] Avg episode reward: [(0, '9167.714'), (1, '8803.713')] [2023-12-26 23:55:56,151][105620] Updated weights for policy 1, policy_version 1176270 (0.0009) [2023-12-26 23:55:56,206][105620] Updated weights for policy 1, policy_version 1176280 (0.0009) [2023-12-26 23:55:56,263][105620] Updated weights for policy 1, policy_version 1176290 (0.0009) [2023-12-26 23:55:56,365][105692] Updated weights for policy 0, policy_version 1174986 (0.0008) [2023-12-26 23:55:56,411][105692] Updated weights for policy 0, policy_version 1174996 (0.0005) [2023-12-26 23:55:56,463][105692] Updated weights for policy 0, policy_version 1175006 (0.0005) [2023-12-26 23:55:56,508][105692] Updated weights for policy 0, policy_version 1175016 (0.0005) [2023-12-26 23:55:57,059][105620] Updated weights for policy 1, policy_version 1176300 (0.0009) [2023-12-26 23:55:57,116][105620] Updated weights for policy 1, policy_version 1176310 (0.0010) [2023-12-26 23:55:57,169][105620] Updated weights for policy 1, policy_version 1176320 (0.0009) [2023-12-26 23:55:57,189][105692] Updated weights for policy 0, policy_version 1175026 (0.0005) [2023-12-26 23:55:57,237][105692] Updated weights for policy 0, policy_version 1175036 (0.0005) [2023-12-26 23:55:57,289][105692] Updated weights for policy 0, policy_version 1175046 (0.0005) [2023-12-26 23:55:57,892][105692] Updated weights for policy 0, policy_version 1175056 (0.0006) [2023-12-26 23:55:57,923][105620] Updated weights for policy 1, policy_version 1176330 (0.0008) [2023-12-26 23:55:57,949][105692] Updated weights for policy 0, policy_version 1175066 (0.0007) [2023-12-26 23:55:57,975][105620] Updated weights for policy 1, policy_version 1176340 (0.0006) [2023-12-26 23:55:58,005][105692] Updated weights for policy 0, policy_version 1175076 (0.0008) [2023-12-26 23:55:58,032][105620] Updated weights for policy 1, policy_version 1176350 (0.0005) [2023-12-26 23:55:58,086][105620] Updated weights for policy 1, policy_version 1176360 (0.0005) [2023-12-26 23:55:58,578][105692] Updated weights for policy 0, policy_version 1175086 (0.0007) [2023-12-26 23:55:58,638][105692] Updated weights for policy 0, policy_version 1175096 (0.0008) [2023-12-26 23:55:58,701][105692] Updated weights for policy 0, policy_version 1175106 (0.0008) [2023-12-26 23:55:58,814][105620] Updated weights for policy 1, policy_version 1176370 (0.0010) [2023-12-26 23:55:58,884][105620] Updated weights for policy 1, policy_version 1176380 (0.0011) [2023-12-26 23:55:58,951][105620] Updated weights for policy 1, policy_version 1176390 (0.0010) [2023-12-26 23:55:59,428][105692] Updated weights for policy 0, policy_version 1175116 (0.0008) [2023-12-26 23:55:59,477][105692] Updated weights for policy 0, policy_version 1175126 (0.0008) [2023-12-26 23:55:59,527][105692] Updated weights for policy 0, policy_version 1175136 (0.0007) [2023-12-26 23:55:59,679][105620] Updated weights for policy 1, policy_version 1176400 (0.0008) [2023-12-26 23:55:59,738][105620] Updated weights for policy 1, policy_version 1176410 (0.0007) [2023-12-26 23:55:59,806][105620] Updated weights for policy 1, policy_version 1176420 (0.0010) [2023-12-26 23:56:00,220][105692] Updated weights for policy 0, policy_version 1175146 (0.0006) [2023-12-26 23:56:00,273][105692] Updated weights for policy 0, policy_version 1175156 (0.0010) [2023-12-26 23:56:00,326][105692] Updated weights for policy 0, policy_version 1175166 (0.0010) [2023-12-26 23:56:00,451][105620] Updated weights for policy 1, policy_version 1176430 (0.0009) [2023-12-26 23:56:00,510][105620] Updated weights for policy 1, policy_version 1176440 (0.0009) [2023-12-26 23:56:00,568][105620] Updated weights for policy 1, policy_version 1176450 (0.0009) [2023-12-26 23:56:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 602103808. Throughput: 0: 9841.7, 1: 9847.4. Samples: 602076240. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:56:01,062][104569] Avg episode reward: [(0, '9261.004'), (1, '8802.636')] [2023-12-26 23:56:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001175176_300892160.pth... [2023-12-26 23:56:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001176456_301211648.pth... [2023-12-26 23:56:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001173992_300589056.pth [2023-12-26 23:56:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001175336_300924928.pth [2023-12-26 23:56:01,167][105692] Updated weights for policy 0, policy_version 1175177 (0.0010) [2023-12-26 23:56:01,224][105692] Updated weights for policy 0, policy_version 1175187 (0.0008) [2023-12-26 23:56:01,260][105620] Updated weights for policy 1, policy_version 1176460 (0.0009) [2023-12-26 23:56:01,285][105692] Updated weights for policy 0, policy_version 1175197 (0.0008) [2023-12-26 23:56:01,322][105620] Updated weights for policy 1, policy_version 1176470 (0.0011) [2023-12-26 23:56:01,348][105692] Updated weights for policy 0, policy_version 1175207 (0.0006) [2023-12-26 23:56:01,386][105620] Updated weights for policy 1, policy_version 1176480 (0.0009) [2023-12-26 23:56:02,101][105620] Updated weights for policy 1, policy_version 1176490 (0.0005) [2023-12-26 23:56:02,133][105692] Updated weights for policy 0, policy_version 1175217 (0.0009) [2023-12-26 23:56:02,161][105620] Updated weights for policy 1, policy_version 1176500 (0.0005) [2023-12-26 23:56:02,186][105692] Updated weights for policy 0, policy_version 1175227 (0.0008) [2023-12-26 23:56:02,220][105620] Updated weights for policy 1, policy_version 1176510 (0.0008) [2023-12-26 23:56:02,243][105692] Updated weights for policy 0, policy_version 1175237 (0.0008) [2023-12-26 23:56:02,278][105620] Updated weights for policy 1, policy_version 1176520 (0.0007) [2023-12-26 23:56:02,950][105692] Updated weights for policy 0, policy_version 1175247 (0.0008) [2023-12-26 23:56:03,008][105692] Updated weights for policy 0, policy_version 1175257 (0.0009) [2023-12-26 23:56:03,033][105620] Updated weights for policy 1, policy_version 1176530 (0.0007) [2023-12-26 23:56:03,058][105692] Updated weights for policy 0, policy_version 1175267 (0.0008) [2023-12-26 23:56:03,079][105620] Updated weights for policy 1, policy_version 1176540 (0.0008) [2023-12-26 23:56:03,132][105620] Updated weights for policy 1, policy_version 1176550 (0.0009) [2023-12-26 23:56:03,657][105692] Updated weights for policy 0, policy_version 1175277 (0.0005) [2023-12-26 23:56:03,713][105692] Updated weights for policy 0, policy_version 1175287 (0.0005) [2023-12-26 23:56:03,769][105692] Updated weights for policy 0, policy_version 1175297 (0.0005) [2023-12-26 23:56:03,868][105620] Updated weights for policy 1, policy_version 1176560 (0.0009) [2023-12-26 23:56:03,933][105620] Updated weights for policy 1, policy_version 1176570 (0.0010) [2023-12-26 23:56:03,997][105620] Updated weights for policy 1, policy_version 1176580 (0.0010) [2023-12-26 23:56:04,400][105692] Updated weights for policy 0, policy_version 1175307 (0.0007) [2023-12-26 23:56:04,467][105692] Updated weights for policy 0, policy_version 1175317 (0.0007) [2023-12-26 23:56:04,531][105692] Updated weights for policy 0, policy_version 1175327 (0.0008) [2023-12-26 23:56:04,801][105620] Updated weights for policy 1, policy_version 1176590 (0.0009) [2023-12-26 23:56:04,849][105620] Updated weights for policy 1, policy_version 1176600 (0.0009) [2023-12-26 23:56:04,900][105620] Updated weights for policy 1, policy_version 1176610 (0.0009) [2023-12-26 23:56:05,211][105692] Updated weights for policy 0, policy_version 1175337 (0.0009) [2023-12-26 23:56:05,269][105692] Updated weights for policy 0, policy_version 1175347 (0.0009) [2023-12-26 23:56:05,319][105692] Updated weights for policy 0, policy_version 1175357 (0.0009) [2023-12-26 23:56:05,370][105692] Updated weights for policy 0, policy_version 1175367 (0.0009) [2023-12-26 23:56:05,710][105620] Updated weights for policy 1, policy_version 1176620 (0.0010) [2023-12-26 23:56:05,763][105620] Updated weights for policy 1, policy_version 1176630 (0.0010) [2023-12-26 23:56:05,816][105620] Updated weights for policy 1, policy_version 1176640 (0.0010) [2023-12-26 23:56:06,047][105692] Updated weights for policy 0, policy_version 1175377 (0.0009) [2023-12-26 23:56:06,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 602202112. Throughput: 0: 9850.3, 1: 9786.3. Samples: 602192436. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) [2023-12-26 23:56:06,062][104569] Avg episode reward: [(0, '9353.230'), (1, '9077.013')] [2023-12-26 23:56:06,094][105692] Updated weights for policy 0, policy_version 1175387 (0.0009) [2023-12-26 23:56:06,185][105692] Updated weights for policy 0, policy_version 1175397 (0.0006) [2023-12-26 23:56:06,660][105620] Updated weights for policy 1, policy_version 1176650 (0.0009) [2023-12-26 23:56:06,719][105620] Updated weights for policy 1, policy_version 1176660 (0.0009) [2023-12-26 23:56:06,770][105620] Updated weights for policy 1, policy_version 1176670 (0.0009) [2023-12-26 23:56:06,829][105620] Updated weights for policy 1, policy_version 1176680 (0.0010) [2023-12-26 23:56:06,897][105692] Updated weights for policy 0, policy_version 1175407 (0.0010) [2023-12-26 23:56:06,958][105692] Updated weights for policy 0, policy_version 1175417 (0.0008) [2023-12-26 23:56:07,024][105692] Updated weights for policy 0, policy_version 1175427 (0.0009) [2023-12-26 23:56:07,628][105620] Updated weights for policy 1, policy_version 1176690 (0.0010) [2023-12-26 23:56:07,685][105620] Updated weights for policy 1, policy_version 1176701 (0.0009) [2023-12-26 23:56:07,690][105692] Updated weights for policy 0, policy_version 1175437 (0.0008) [2023-12-26 23:56:07,741][105620] Updated weights for policy 1, policy_version 1176711 (0.0009) [2023-12-26 23:56:07,745][105692] Updated weights for policy 0, policy_version 1175447 (0.0005) [2023-12-26 23:56:07,797][105692] Updated weights for policy 0, policy_version 1175457 (0.0006) [2023-12-26 23:56:08,411][105692] Updated weights for policy 0, policy_version 1175467 (0.0009) [2023-12-26 23:56:08,476][105692] Updated weights for policy 0, policy_version 1175477 (0.0009) [2023-12-26 23:56:08,525][105692] Updated weights for policy 0, policy_version 1175487 (0.0008) [2023-12-26 23:56:08,551][105620] Updated weights for policy 1, policy_version 1176721 (0.0008) [2023-12-26 23:56:08,601][105620] Updated weights for policy 1, policy_version 1176731 (0.0008) [2023-12-26 23:56:08,663][105620] Updated weights for policy 1, policy_version 1176741 (0.0009) [2023-12-26 23:56:09,280][105692] Updated weights for policy 0, policy_version 1175497 (0.0006) [2023-12-26 23:56:09,332][105692] Updated weights for policy 0, policy_version 1175507 (0.0009) [2023-12-26 23:56:09,404][105692] Updated weights for policy 0, policy_version 1175517 (0.0009) [2023-12-26 23:56:09,433][105620] Updated weights for policy 1, policy_version 1176751 (0.0008) [2023-12-26 23:56:09,466][105692] Updated weights for policy 0, policy_version 1175527 (0.0006) [2023-12-26 23:56:09,489][105620] Updated weights for policy 1, policy_version 1176761 (0.0008) [2023-12-26 23:56:09,547][105620] Updated weights for policy 1, policy_version 1176771 (0.0009) [2023-12-26 23:56:10,217][105692] Updated weights for policy 0, policy_version 1175537 (0.0008) [2023-12-26 23:56:10,264][105620] Updated weights for policy 1, policy_version 1176781 (0.0008) [2023-12-26 23:56:10,266][105692] Updated weights for policy 0, policy_version 1175547 (0.0008) [2023-12-26 23:56:10,316][105692] Updated weights for policy 0, policy_version 1175557 (0.0006) [2023-12-26 23:56:10,322][105620] Updated weights for policy 1, policy_version 1176791 (0.0007) [2023-12-26 23:56:10,382][105620] Updated weights for policy 1, policy_version 1176801 (0.0009) [2023-12-26 23:56:11,051][105692] Updated weights for policy 0, policy_version 1175567 (0.0009) [2023-12-26 23:56:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 602292224. Throughput: 0: 9912.5, 1: 9599.4. Samples: 602306404. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:56:11,062][104569] Avg episode reward: [(0, '9262.672'), (1, '8130.413')] [2023-12-26 23:56:11,114][105692] Updated weights for policy 0, policy_version 1175577 (0.0010) [2023-12-26 23:56:11,129][105620] Updated weights for policy 1, policy_version 1176811 (0.0009) [2023-12-26 23:56:11,174][105692] Updated weights for policy 0, policy_version 1175587 (0.0008) [2023-12-26 23:56:11,198][105620] Updated weights for policy 1, policy_version 1176821 (0.0007) [2023-12-26 23:56:11,266][105620] Updated weights for policy 1, policy_version 1176831 (0.0008) [2023-12-26 23:56:11,954][105692] Updated weights for policy 0, policy_version 1175597 (0.0009) [2023-12-26 23:56:12,011][105620] Updated weights for policy 1, policy_version 1176841 (0.0007) [2023-12-26 23:56:12,022][105692] Updated weights for policy 0, policy_version 1175607 (0.0008) [2023-12-26 23:56:12,065][105620] Updated weights for policy 1, policy_version 1176851 (0.0006) [2023-12-26 23:56:12,089][105692] Updated weights for policy 0, policy_version 1175617 (0.0009) [2023-12-26 23:56:12,121][105620] Updated weights for policy 1, policy_version 1176861 (0.0009) [2023-12-26 23:56:12,174][105620] Updated weights for policy 1, policy_version 1176871 (0.0009) [2023-12-26 23:56:12,829][105692] Updated weights for policy 0, policy_version 1175627 (0.0010) [2023-12-26 23:56:12,891][105692] Updated weights for policy 0, policy_version 1175637 (0.0009) [2023-12-26 23:56:12,946][105692] Updated weights for policy 0, policy_version 1175647 (0.0007) [2023-12-26 23:56:12,952][105620] Updated weights for policy 1, policy_version 1176881 (0.0007) [2023-12-26 23:56:13,009][105620] Updated weights for policy 1, policy_version 1176891 (0.0006) [2023-12-26 23:56:13,063][105620] Updated weights for policy 1, policy_version 1176901 (0.0009) [2023-12-26 23:56:13,727][105692] Updated weights for policy 0, policy_version 1175657 (0.0007) [2023-12-26 23:56:13,767][105620] Updated weights for policy 1, policy_version 1176911 (0.0006) [2023-12-26 23:56:13,792][105692] Updated weights for policy 0, policy_version 1175667 (0.0007) [2023-12-26 23:56:13,829][105620] Updated weights for policy 1, policy_version 1176921 (0.0007) [2023-12-26 23:56:13,858][105692] Updated weights for policy 0, policy_version 1175677 (0.0010) [2023-12-26 23:56:13,881][105620] Updated weights for policy 1, policy_version 1176931 (0.0006) [2023-12-26 23:56:13,922][105692] Updated weights for policy 0, policy_version 1175687 (0.0008) [2023-12-26 23:56:14,614][105620] Updated weights for policy 1, policy_version 1176941 (0.0008) [2023-12-26 23:56:14,662][105620] Updated weights for policy 1, policy_version 1176951 (0.0006) [2023-12-26 23:56:14,693][105692] Updated weights for policy 0, policy_version 1175697 (0.0009) [2023-12-26 23:56:14,719][105620] Updated weights for policy 1, policy_version 1176961 (0.0005) [2023-12-26 23:56:14,742][105692] Updated weights for policy 0, policy_version 1175707 (0.0008) [2023-12-26 23:56:14,807][105692] Updated weights for policy 0, policy_version 1175717 (0.0008) [2023-12-26 23:56:15,436][105620] Updated weights for policy 1, policy_version 1176971 (0.0008) [2023-12-26 23:56:15,490][105620] Updated weights for policy 1, policy_version 1176981 (0.0010) [2023-12-26 23:56:15,551][105620] Updated weights for policy 1, policy_version 1176991 (0.0010) [2023-12-26 23:56:15,601][105692] Updated weights for policy 0, policy_version 1175727 (0.0006) [2023-12-26 23:56:15,655][105692] Updated weights for policy 0, policy_version 1175737 (0.0009) [2023-12-26 23:56:15,709][105692] Updated weights for policy 0, policy_version 1175747 (0.0005) [2023-12-26 23:56:16,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 602390528. Throughput: 0: 9963.4, 1: 9440.9. Samples: 602361948. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:56:16,063][104569] Avg episode reward: [(0, '9262.399'), (1, '7064.267')] [2023-12-26 23:56:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001175752_301039616.pth... [2023-12-26 23:56:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001177000_301350912.pth... [2023-12-26 23:56:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001174600_300744704.pth [2023-12-26 23:56:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001175912_301072384.pth [2023-12-26 23:56:16,204][105620] Updated weights for policy 1, policy_version 1177001 (0.0010) [2023-12-26 23:56:16,258][105620] Updated weights for policy 1, policy_version 1177011 (0.0009) [2023-12-26 23:56:16,313][105620] Updated weights for policy 1, policy_version 1177021 (0.0009) [2023-12-26 23:56:16,364][105692] Updated weights for policy 0, policy_version 1175757 (0.0005) [2023-12-26 23:56:16,369][105620] Updated weights for policy 1, policy_version 1177031 (0.0009) [2023-12-26 23:56:16,433][105692] Updated weights for policy 0, policy_version 1175767 (0.0005) [2023-12-26 23:56:16,503][105692] Updated weights for policy 0, policy_version 1175777 (0.0005) [2023-12-26 23:56:17,163][105692] Updated weights for policy 0, policy_version 1175787 (0.0009) [2023-12-26 23:56:17,171][105620] Updated weights for policy 1, policy_version 1177041 (0.0010) [2023-12-26 23:56:17,206][105692] Updated weights for policy 0, policy_version 1175797 (0.0008) [2023-12-26 23:56:17,229][105620] Updated weights for policy 1, policy_version 1177051 (0.0010) [2023-12-26 23:56:17,252][105692] Updated weights for policy 0, policy_version 1175807 (0.0007) [2023-12-26 23:56:17,291][105620] Updated weights for policy 1, policy_version 1177061 (0.0010) [2023-12-26 23:56:18,022][105620] Updated weights for policy 1, policy_version 1177071 (0.0010) [2023-12-26 23:56:18,042][105692] Updated weights for policy 0, policy_version 1175817 (0.0007) [2023-12-26 23:56:18,080][105620] Updated weights for policy 1, policy_version 1177081 (0.0010) [2023-12-26 23:56:18,099][105692] Updated weights for policy 0, policy_version 1175827 (0.0007) [2023-12-26 23:56:18,135][105620] Updated weights for policy 1, policy_version 1177091 (0.0010) [2023-12-26 23:56:18,154][105692] Updated weights for policy 0, policy_version 1175837 (0.0006) [2023-12-26 23:56:18,198][105692] Updated weights for policy 0, policy_version 1175847 (0.0008) [2023-12-26 23:56:18,875][105620] Updated weights for policy 1, policy_version 1177101 (0.0010) [2023-12-26 23:56:18,930][105620] Updated weights for policy 1, policy_version 1177111 (0.0010) [2023-12-26 23:56:18,984][105620] Updated weights for policy 1, policy_version 1177121 (0.0010) [2023-12-26 23:56:18,990][105692] Updated weights for policy 0, policy_version 1175857 (0.0006) [2023-12-26 23:56:19,045][105692] Updated weights for policy 0, policy_version 1175867 (0.0007) [2023-12-26 23:56:19,101][105692] Updated weights for policy 0, policy_version 1175877 (0.0008) [2023-12-26 23:56:19,716][105620] Updated weights for policy 1, policy_version 1177131 (0.0009) [2023-12-26 23:56:19,765][105620] Updated weights for policy 1, policy_version 1177141 (0.0006) [2023-12-26 23:56:19,819][105620] Updated weights for policy 1, policy_version 1177151 (0.0007) [2023-12-26 23:56:19,921][105692] Updated weights for policy 0, policy_version 1175887 (0.0010) [2023-12-26 23:56:19,984][105692] Updated weights for policy 0, policy_version 1175897 (0.0010) [2023-12-26 23:56:20,047][105692] Updated weights for policy 0, policy_version 1175907 (0.0009) [2023-12-26 23:56:20,413][105620] Updated weights for policy 1, policy_version 1177161 (0.0006) [2023-12-26 23:56:20,474][105620] Updated weights for policy 1, policy_version 1177171 (0.0008) [2023-12-26 23:56:20,533][105620] Updated weights for policy 1, policy_version 1177181 (0.0008) [2023-12-26 23:56:20,590][105620] Updated weights for policy 1, policy_version 1177191 (0.0008) [2023-12-26 23:56:20,777][105692] Updated weights for policy 0, policy_version 1175917 (0.0011) [2023-12-26 23:56:20,843][105692] Updated weights for policy 0, policy_version 1175927 (0.0010) [2023-12-26 23:56:20,909][105692] Updated weights for policy 0, policy_version 1175937 (0.0010) [2023-12-26 23:56:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 602488832. Throughput: 0: 9929.8, 1: 9420.3. Samples: 602476456. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:56:21,063][104569] Avg episode reward: [(0, '8900.443'), (1, '7792.351')] [2023-12-26 23:56:21,331][105620] Updated weights for policy 1, policy_version 1177201 (0.0008) [2023-12-26 23:56:21,402][105620] Updated weights for policy 1, policy_version 1177211 (0.0009) [2023-12-26 23:56:21,462][105620] Updated weights for policy 1, policy_version 1177221 (0.0009) [2023-12-26 23:56:21,578][105692] Updated weights for policy 0, policy_version 1175947 (0.0010) [2023-12-26 23:56:21,662][105692] Updated weights for policy 0, policy_version 1175957 (0.0009) [2023-12-26 23:56:21,731][105692] Updated weights for policy 0, policy_version 1175967 (0.0007) [2023-12-26 23:56:22,306][105620] Updated weights for policy 1, policy_version 1177231 (0.0007) [2023-12-26 23:56:22,367][105620] Updated weights for policy 1, policy_version 1177241 (0.0009) [2023-12-26 23:56:22,432][105620] Updated weights for policy 1, policy_version 1177251 (0.0009) [2023-12-26 23:56:22,484][105692] Updated weights for policy 0, policy_version 1175977 (0.0009) [2023-12-26 23:56:22,536][105692] Updated weights for policy 0, policy_version 1175987 (0.0009) [2023-12-26 23:56:22,590][105692] Updated weights for policy 0, policy_version 1175997 (0.0009) [2023-12-26 23:56:22,639][105692] Updated weights for policy 0, policy_version 1176007 (0.0009) [2023-12-26 23:56:23,176][105620] Updated weights for policy 1, policy_version 1177261 (0.0009) [2023-12-26 23:56:23,230][105620] Updated weights for policy 1, policy_version 1177271 (0.0009) [2023-12-26 23:56:23,280][105620] Updated weights for policy 1, policy_version 1177281 (0.0009) [2023-12-26 23:56:23,421][105692] Updated weights for policy 0, policy_version 1176017 (0.0009) [2023-12-26 23:56:23,474][105692] Updated weights for policy 0, policy_version 1176027 (0.0006) [2023-12-26 23:56:23,538][105692] Updated weights for policy 0, policy_version 1176037 (0.0006) [2023-12-26 23:56:24,063][105620] Updated weights for policy 1, policy_version 1177291 (0.0008) [2023-12-26 23:56:24,117][105620] Updated weights for policy 1, policy_version 1177301 (0.0006) [2023-12-26 23:56:24,164][105692] Updated weights for policy 0, policy_version 1176047 (0.0009) [2023-12-26 23:56:24,176][105620] Updated weights for policy 1, policy_version 1177311 (0.0006) [2023-12-26 23:56:24,220][105692] Updated weights for policy 0, policy_version 1176057 (0.0011) [2023-12-26 23:56:24,275][105692] Updated weights for policy 0, policy_version 1176067 (0.0010) [2023-12-26 23:56:24,826][105620] Updated weights for policy 1, policy_version 1177321 (0.0005) [2023-12-26 23:56:24,888][105620] Updated weights for policy 1, policy_version 1177331 (0.0005) [2023-12-26 23:56:24,925][105692] Updated weights for policy 0, policy_version 1176077 (0.0008) [2023-12-26 23:56:24,952][105620] Updated weights for policy 1, policy_version 1177341 (0.0005) [2023-12-26 23:56:24,972][105692] Updated weights for policy 0, policy_version 1176087 (0.0005) [2023-12-26 23:56:25,001][105620] Updated weights for policy 1, policy_version 1177351 (0.0005) [2023-12-26 23:56:25,029][105692] Updated weights for policy 0, policy_version 1176097 (0.0006) [2023-12-26 23:56:25,542][105692] Updated weights for policy 0, policy_version 1176107 (0.0005) [2023-12-26 23:56:25,597][105692] Updated weights for policy 0, policy_version 1176117 (0.0005) [2023-12-26 23:56:25,652][105692] Updated weights for policy 0, policy_version 1176127 (0.0005) [2023-12-26 23:56:25,670][105620] Updated weights for policy 1, policy_version 1177361 (0.0008) [2023-12-26 23:56:25,720][105620] Updated weights for policy 1, policy_version 1177371 (0.0009) [2023-12-26 23:56:25,773][105620] Updated weights for policy 1, policy_version 1177381 (0.0010) [2023-12-26 23:56:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 602587136. Throughput: 0: 9919.5, 1: 9404.9. Samples: 602594004. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:56:26,062][104569] Avg episode reward: [(0, '8990.104'), (1, '8476.432')] [2023-12-26 23:56:26,249][105692] Updated weights for policy 0, policy_version 1176137 (0.0005) [2023-12-26 23:56:26,301][105692] Updated weights for policy 0, policy_version 1176147 (0.0005) [2023-12-26 23:56:26,355][105692] Updated weights for policy 0, policy_version 1176157 (0.0005) [2023-12-26 23:56:26,413][105692] Updated weights for policy 0, policy_version 1176167 (0.0005) [2023-12-26 23:56:26,539][105620] Updated weights for policy 1, policy_version 1177392 (0.0010) [2023-12-26 23:56:26,595][105620] Updated weights for policy 1, policy_version 1177402 (0.0011) [2023-12-26 23:56:26,645][105620] Updated weights for policy 1, policy_version 1177412 (0.0010) [2023-12-26 23:56:26,933][105692] Updated weights for policy 0, policy_version 1176177 (0.0010) [2023-12-26 23:56:26,980][105692] Updated weights for policy 0, policy_version 1176187 (0.0010) [2023-12-26 23:56:27,036][105692] Updated weights for policy 0, policy_version 1176197 (0.0009) [2023-12-26 23:56:27,296][105620] Updated weights for policy 1, policy_version 1177422 (0.0008) [2023-12-26 23:56:27,344][105620] Updated weights for policy 1, policy_version 1177432 (0.0005) [2023-12-26 23:56:27,394][105620] Updated weights for policy 1, policy_version 1177442 (0.0005) [2023-12-26 23:56:27,682][105692] Updated weights for policy 0, policy_version 1176207 (0.0005) [2023-12-26 23:56:27,735][105692] Updated weights for policy 0, policy_version 1176217 (0.0005) [2023-12-26 23:56:27,792][105692] Updated weights for policy 0, policy_version 1176227 (0.0009) [2023-12-26 23:56:27,987][105620] Updated weights for policy 1, policy_version 1177452 (0.0006) [2023-12-26 23:56:28,036][105620] Updated weights for policy 1, policy_version 1177462 (0.0008) [2023-12-26 23:56:28,094][105620] Updated weights for policy 1, policy_version 1177472 (0.0010) [2023-12-26 23:56:28,469][105692] Updated weights for policy 0, policy_version 1176237 (0.0009) [2023-12-26 23:56:28,531][105692] Updated weights for policy 0, policy_version 1176247 (0.0008) [2023-12-26 23:56:28,600][105692] Updated weights for policy 0, policy_version 1176257 (0.0008) [2023-12-26 23:56:28,764][105620] Updated weights for policy 1, policy_version 1177482 (0.0009) [2023-12-26 23:56:28,825][105620] Updated weights for policy 1, policy_version 1177492 (0.0010) [2023-12-26 23:56:28,883][105620] Updated weights for policy 1, policy_version 1177502 (0.0010) [2023-12-26 23:56:29,226][105692] Updated weights for policy 0, policy_version 1176267 (0.0009) [2023-12-26 23:56:29,291][105692] Updated weights for policy 0, policy_version 1176277 (0.0011) [2023-12-26 23:56:29,358][105692] Updated weights for policy 0, policy_version 1176287 (0.0011) [2023-12-26 23:56:29,587][105620] Updated weights for policy 1, policy_version 1177514 (0.0009) [2023-12-26 23:56:29,642][105620] Updated weights for policy 1, policy_version 1177524 (0.0008) [2023-12-26 23:56:29,698][105620] Updated weights for policy 1, policy_version 1177534 (0.0005) [2023-12-26 23:56:29,759][105620] Updated weights for policy 1, policy_version 1177544 (0.0005) [2023-12-26 23:56:30,039][105692] Updated weights for policy 0, policy_version 1176297 (0.0010) [2023-12-26 23:56:30,089][105692] Updated weights for policy 0, policy_version 1176307 (0.0005) [2023-12-26 23:56:30,141][105692] Updated weights for policy 0, policy_version 1176317 (0.0005) [2023-12-26 23:56:30,195][105692] Updated weights for policy 0, policy_version 1176327 (0.0006) [2023-12-26 23:56:30,457][105620] Updated weights for policy 1, policy_version 1177554 (0.0010) [2023-12-26 23:56:30,518][105620] Updated weights for policy 1, policy_version 1177564 (0.0009) [2023-12-26 23:56:30,578][105620] Updated weights for policy 1, policy_version 1177574 (0.0008) [2023-12-26 23:56:30,805][105692] Updated weights for policy 0, policy_version 1176337 (0.0007) [2023-12-26 23:56:30,864][105692] Updated weights for policy 0, policy_version 1176347 (0.0005) [2023-12-26 23:56:30,909][105692] Updated weights for policy 0, policy_version 1176357 (0.0005) [2023-12-26 23:56:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 602693632. Throughput: 0: 10024.9, 1: 9513.1. Samples: 602659484. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:56:31,062][104569] Avg episode reward: [(0, '9266.031'), (1, '8712.700')] [2023-12-26 23:56:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001176360_301195264.pth... [2023-12-26 23:56:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001177576_301498368.pth... [2023-12-26 23:56:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001175176_300892160.pth [2023-12-26 23:56:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001176456_301211648.pth [2023-12-26 23:56:31,360][105620] Updated weights for policy 1, policy_version 1177584 (0.0008) [2023-12-26 23:56:31,423][105620] Updated weights for policy 1, policy_version 1177594 (0.0010) [2023-12-26 23:56:31,480][105620] Updated weights for policy 1, policy_version 1177604 (0.0009) [2023-12-26 23:56:31,605][105692] Updated weights for policy 0, policy_version 1176367 (0.0008) [2023-12-26 23:56:31,661][105692] Updated weights for policy 0, policy_version 1176377 (0.0009) [2023-12-26 23:56:31,719][105692] Updated weights for policy 0, policy_version 1176387 (0.0009) [2023-12-26 23:56:32,210][105620] Updated weights for policy 1, policy_version 1177614 (0.0009) [2023-12-26 23:56:32,275][105620] Updated weights for policy 1, policy_version 1177624 (0.0010) [2023-12-26 23:56:32,331][105620] Updated weights for policy 1, policy_version 1177635 (0.0010) [2023-12-26 23:56:32,440][105692] Updated weights for policy 0, policy_version 1176397 (0.0008) [2023-12-26 23:56:32,488][105692] Updated weights for policy 0, policy_version 1176407 (0.0009) [2023-12-26 23:56:32,543][105692] Updated weights for policy 0, policy_version 1176417 (0.0010) [2023-12-26 23:56:33,062][105620] Updated weights for policy 1, policy_version 1177645 (0.0007) [2023-12-26 23:56:33,127][105620] Updated weights for policy 1, policy_version 1177655 (0.0006) [2023-12-26 23:56:33,163][105692] Updated weights for policy 0, policy_version 1176428 (0.0010) [2023-12-26 23:56:33,183][105620] Updated weights for policy 1, policy_version 1177665 (0.0006) [2023-12-26 23:56:33,221][105692] Updated weights for policy 0, policy_version 1176438 (0.0009) [2023-12-26 23:56:33,281][105692] Updated weights for policy 0, policy_version 1176448 (0.0007) [2023-12-26 23:56:33,802][105620] Updated weights for policy 1, policy_version 1177675 (0.0007) [2023-12-26 23:56:33,816][105692] Updated weights for policy 0, policy_version 1176458 (0.0009) [2023-12-26 23:56:33,850][105620] Updated weights for policy 1, policy_version 1177685 (0.0011) [2023-12-26 23:56:33,873][105692] Updated weights for policy 0, policy_version 1176468 (0.0010) [2023-12-26 23:56:33,901][105620] Updated weights for policy 1, policy_version 1177695 (0.0010) [2023-12-26 23:56:33,927][105692] Updated weights for policy 0, policy_version 1176478 (0.0010) [2023-12-26 23:56:33,982][105692] Updated weights for policy 0, policy_version 1176488 (0.0007) [2023-12-26 23:56:34,620][105620] Updated weights for policy 1, policy_version 1177705 (0.0010) [2023-12-26 23:56:34,686][105620] Updated weights for policy 1, policy_version 1177715 (0.0009) [2023-12-26 23:56:34,688][105692] Updated weights for policy 0, policy_version 1176498 (0.0007) [2023-12-26 23:56:34,749][105620] Updated weights for policy 1, policy_version 1177725 (0.0011) [2023-12-26 23:56:34,751][105692] Updated weights for policy 0, policy_version 1176508 (0.0006) [2023-12-26 23:56:34,810][105692] Updated weights for policy 0, policy_version 1176518 (0.0007) [2023-12-26 23:56:34,811][105620] Updated weights for policy 1, policy_version 1177735 (0.0010) [2023-12-26 23:56:35,494][105692] Updated weights for policy 0, policy_version 1176528 (0.0008) [2023-12-26 23:56:35,516][105620] Updated weights for policy 1, policy_version 1177745 (0.0008) [2023-12-26 23:56:35,559][105692] Updated weights for policy 0, policy_version 1176538 (0.0008) [2023-12-26 23:56:35,571][105620] Updated weights for policy 1, policy_version 1177755 (0.0005) [2023-12-26 23:56:35,612][105692] Updated weights for policy 0, policy_version 1176548 (0.0009) [2023-12-26 23:56:35,632][105620] Updated weights for policy 1, policy_version 1177765 (0.0005) [2023-12-26 23:56:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 602791936. Throughput: 0: 10025.3, 1: 9515.1. Samples: 602781764. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:56:36,063][104569] Avg episode reward: [(0, '9175.132'), (1, '9079.506')] [2023-12-26 23:56:36,233][105620] Updated weights for policy 1, policy_version 1177775 (0.0008) [2023-12-26 23:56:36,303][105620] Updated weights for policy 1, policy_version 1177785 (0.0006) [2023-12-26 23:56:36,367][105620] Updated weights for policy 1, policy_version 1177795 (0.0005) [2023-12-26 23:56:36,375][105692] Updated weights for policy 0, policy_version 1176558 (0.0009) [2023-12-26 23:56:36,439][105692] Updated weights for policy 0, policy_version 1176568 (0.0006) [2023-12-26 23:56:36,505][105692] Updated weights for policy 0, policy_version 1176578 (0.0006) [2023-12-26 23:56:37,006][105620] Updated weights for policy 1, policy_version 1177805 (0.0007) [2023-12-26 23:56:37,062][105620] Updated weights for policy 1, policy_version 1177815 (0.0008) [2023-12-26 23:56:37,118][105620] Updated weights for policy 1, policy_version 1177825 (0.0008) [2023-12-26 23:56:37,152][105692] Updated weights for policy 0, policy_version 1176588 (0.0010) [2023-12-26 23:56:37,206][105692] Updated weights for policy 0, policy_version 1176598 (0.0010) [2023-12-26 23:56:37,266][105692] Updated weights for policy 0, policy_version 1176608 (0.0005) [2023-12-26 23:56:37,832][105620] Updated weights for policy 1, policy_version 1177835 (0.0007) [2023-12-26 23:56:37,887][105620] Updated weights for policy 1, policy_version 1177845 (0.0009) [2023-12-26 23:56:37,940][105692] Updated weights for policy 0, policy_version 1176618 (0.0006) [2023-12-26 23:56:37,947][105620] Updated weights for policy 1, policy_version 1177855 (0.0008) [2023-12-26 23:56:38,001][105692] Updated weights for policy 0, policy_version 1176628 (0.0009) [2023-12-26 23:56:38,066][105692] Updated weights for policy 0, policy_version 1176638 (0.0009) [2023-12-26 23:56:38,128][105692] Updated weights for policy 0, policy_version 1176648 (0.0009) [2023-12-26 23:56:38,687][105620] Updated weights for policy 1, policy_version 1177865 (0.0007) [2023-12-26 23:56:38,737][105620] Updated weights for policy 1, policy_version 1177875 (0.0008) [2023-12-26 23:56:38,795][105620] Updated weights for policy 1, policy_version 1177885 (0.0009) [2023-12-26 23:56:38,855][105620] Updated weights for policy 1, policy_version 1177895 (0.0008) [2023-12-26 23:56:38,887][105692] Updated weights for policy 0, policy_version 1176658 (0.0009) [2023-12-26 23:56:38,944][105692] Updated weights for policy 0, policy_version 1176668 (0.0010) [2023-12-26 23:56:39,002][105692] Updated weights for policy 0, policy_version 1176678 (0.0007) [2023-12-26 23:56:39,552][105620] Updated weights for policy 1, policy_version 1177905 (0.0009) [2023-12-26 23:56:39,615][105620] Updated weights for policy 1, policy_version 1177916 (0.0011) [2023-12-26 23:56:39,644][105692] Updated weights for policy 0, policy_version 1176688 (0.0007) [2023-12-26 23:56:39,675][105620] Updated weights for policy 1, policy_version 1177926 (0.0008) [2023-12-26 23:56:39,696][105692] Updated weights for policy 0, policy_version 1176698 (0.0009) [2023-12-26 23:56:39,753][105692] Updated weights for policy 0, policy_version 1176708 (0.0009) [2023-12-26 23:56:40,360][105620] Updated weights for policy 1, policy_version 1177936 (0.0007) [2023-12-26 23:56:40,414][105620] Updated weights for policy 1, policy_version 1177946 (0.0008) [2023-12-26 23:56:40,468][105620] Updated weights for policy 1, policy_version 1177956 (0.0007) [2023-12-26 23:56:40,564][105692] Updated weights for policy 0, policy_version 1176718 (0.0008) [2023-12-26 23:56:40,625][105692] Updated weights for policy 0, policy_version 1176728 (0.0009) [2023-12-26 23:56:40,688][105692] Updated weights for policy 0, policy_version 1176738 (0.0009) [2023-12-26 23:56:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 602890240. Throughput: 0: 10036.4, 1: 9631.8. Samples: 602900460. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:56:41,063][104569] Avg episode reward: [(0, '9173.232'), (1, '9080.999')] [2023-12-26 23:56:41,199][105620] Updated weights for policy 1, policy_version 1177966 (0.0008) [2023-12-26 23:56:41,260][105620] Updated weights for policy 1, policy_version 1177976 (0.0008) [2023-12-26 23:56:41,326][105620] Updated weights for policy 1, policy_version 1177986 (0.0008) [2023-12-26 23:56:41,460][105692] Updated weights for policy 0, policy_version 1176748 (0.0008) [2023-12-26 23:56:41,523][105692] Updated weights for policy 0, policy_version 1176758 (0.0007) [2023-12-26 23:56:41,584][105692] Updated weights for policy 0, policy_version 1176768 (0.0009) [2023-12-26 23:56:42,093][105620] Updated weights for policy 1, policy_version 1177996 (0.0009) [2023-12-26 23:56:42,159][105620] Updated weights for policy 1, policy_version 1178006 (0.0009) [2023-12-26 23:56:42,225][105620] Updated weights for policy 1, policy_version 1178016 (0.0008) [2023-12-26 23:56:42,351][105692] Updated weights for policy 0, policy_version 1176778 (0.0009) [2023-12-26 23:56:42,419][105692] Updated weights for policy 0, policy_version 1176788 (0.0008) [2023-12-26 23:56:42,483][105692] Updated weights for policy 0, policy_version 1176798 (0.0008) [2023-12-26 23:56:42,546][105692] Updated weights for policy 0, policy_version 1176808 (0.0008) [2023-12-26 23:56:42,901][105620] Updated weights for policy 1, policy_version 1178026 (0.0009) [2023-12-26 23:56:42,962][105620] Updated weights for policy 1, policy_version 1178036 (0.0009) [2023-12-26 23:56:43,026][105620] Updated weights for policy 1, policy_version 1178046 (0.0009) [2023-12-26 23:56:43,088][105620] Updated weights for policy 1, policy_version 1178056 (0.0008) [2023-12-26 23:56:43,325][105692] Updated weights for policy 0, policy_version 1176818 (0.0011) [2023-12-26 23:56:43,391][105692] Updated weights for policy 0, policy_version 1176828 (0.0010) [2023-12-26 23:56:43,453][105692] Updated weights for policy 0, policy_version 1176838 (0.0010) [2023-12-26 23:56:43,853][105620] Updated weights for policy 1, policy_version 1178066 (0.0010) [2023-12-26 23:56:43,901][105620] Updated weights for policy 1, policy_version 1178076 (0.0010) [2023-12-26 23:56:43,949][105620] Updated weights for policy 1, policy_version 1178086 (0.0010) [2023-12-26 23:56:44,187][105692] Updated weights for policy 0, policy_version 1176848 (0.0008) [2023-12-26 23:56:44,254][105692] Updated weights for policy 0, policy_version 1176858 (0.0008) [2023-12-26 23:56:44,319][105692] Updated weights for policy 0, policy_version 1176868 (0.0008) [2023-12-26 23:56:44,668][105620] Updated weights for policy 1, policy_version 1178096 (0.0010) [2023-12-26 23:56:44,727][105620] Updated weights for policy 1, policy_version 1178106 (0.0011) [2023-12-26 23:56:44,785][105620] Updated weights for policy 1, policy_version 1178116 (0.0011) [2023-12-26 23:56:44,896][105692] Updated weights for policy 0, policy_version 1176878 (0.0008) [2023-12-26 23:56:44,956][105692] Updated weights for policy 0, policy_version 1176888 (0.0005) [2023-12-26 23:56:45,022][105692] Updated weights for policy 0, policy_version 1176898 (0.0006) [2023-12-26 23:56:45,537][105620] Updated weights for policy 1, policy_version 1178126 (0.0011) [2023-12-26 23:56:45,592][105620] Updated weights for policy 1, policy_version 1178136 (0.0010) [2023-12-26 23:56:45,601][105692] Updated weights for policy 0, policy_version 1176908 (0.0008) [2023-12-26 23:56:45,651][105620] Updated weights for policy 1, policy_version 1178146 (0.0010) [2023-12-26 23:56:45,656][105692] Updated weights for policy 0, policy_version 1176918 (0.0010) [2023-12-26 23:56:45,708][105692] Updated weights for policy 0, policy_version 1176928 (0.0011) [2023-12-26 23:56:46,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 602988544. Throughput: 0: 9892.9, 1: 9637.0. Samples: 602955084. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:56:46,062][104569] Avg episode reward: [(0, '9173.993'), (1, '9081.438')] [2023-12-26 23:56:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001176936_301342720.pth... [2023-12-26 23:56:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001178152_301645824.pth... [2023-12-26 23:56:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001175752_301039616.pth [2023-12-26 23:56:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001177000_301350912.pth [2023-12-26 23:56:46,412][105620] Updated weights for policy 1, policy_version 1178156 (0.0011) [2023-12-26 23:56:46,430][105692] Updated weights for policy 0, policy_version 1176938 (0.0011) [2023-12-26 23:56:46,471][105620] Updated weights for policy 1, policy_version 1178166 (0.0011) [2023-12-26 23:56:46,487][105692] Updated weights for policy 0, policy_version 1176948 (0.0006) [2023-12-26 23:56:46,530][105620] Updated weights for policy 1, policy_version 1178176 (0.0011) [2023-12-26 23:56:46,535][105692] Updated weights for policy 0, policy_version 1176958 (0.0005) [2023-12-26 23:56:46,584][105692] Updated weights for policy 0, policy_version 1176968 (0.0005) [2023-12-26 23:56:47,119][105692] Updated weights for policy 0, policy_version 1176978 (0.0005) [2023-12-26 23:56:47,165][105692] Updated weights for policy 0, policy_version 1176988 (0.0005) [2023-12-26 23:56:47,210][105692] Updated weights for policy 0, policy_version 1176998 (0.0005) [2023-12-26 23:56:47,274][105620] Updated weights for policy 1, policy_version 1178186 (0.0010) [2023-12-26 23:56:47,329][105620] Updated weights for policy 1, policy_version 1178196 (0.0010) [2023-12-26 23:56:47,387][105620] Updated weights for policy 1, policy_version 1178206 (0.0010) [2023-12-26 23:56:47,444][105620] Updated weights for policy 1, policy_version 1178216 (0.0010) [2023-12-26 23:56:47,886][105692] Updated weights for policy 0, policy_version 1177008 (0.0010) [2023-12-26 23:56:47,944][105692] Updated weights for policy 0, policy_version 1177018 (0.0010) [2023-12-26 23:56:47,992][105692] Updated weights for policy 0, policy_version 1177028 (0.0010) [2023-12-26 23:56:48,188][105620] Updated weights for policy 1, policy_version 1178226 (0.0010) [2023-12-26 23:56:48,243][105620] Updated weights for policy 1, policy_version 1178236 (0.0010) [2023-12-26 23:56:48,308][105620] Updated weights for policy 1, policy_version 1178246 (0.0010) [2023-12-26 23:56:48,749][105692] Updated weights for policy 0, policy_version 1177038 (0.0011) [2023-12-26 23:56:48,817][105692] Updated weights for policy 0, policy_version 1177048 (0.0011) [2023-12-26 23:56:48,880][105692] Updated weights for policy 0, policy_version 1177058 (0.0011) [2023-12-26 23:56:49,070][105620] Updated weights for policy 1, policy_version 1178256 (0.0009) [2023-12-26 23:56:49,119][105620] Updated weights for policy 1, policy_version 1178266 (0.0008) [2023-12-26 23:56:49,178][105620] Updated weights for policy 1, policy_version 1178276 (0.0009) [2023-12-26 23:56:49,603][105692] Updated weights for policy 0, policy_version 1177068 (0.0009) [2023-12-26 23:56:49,668][105692] Updated weights for policy 0, policy_version 1177078 (0.0006) [2023-12-26 23:56:49,726][105692] Updated weights for policy 0, policy_version 1177088 (0.0006) [2023-12-26 23:56:49,971][105620] Updated weights for policy 1, policy_version 1178286 (0.0010) [2023-12-26 23:56:50,026][105620] Updated weights for policy 1, policy_version 1178296 (0.0010) [2023-12-26 23:56:50,091][105620] Updated weights for policy 1, policy_version 1178306 (0.0010) [2023-12-26 23:56:50,418][105692] Updated weights for policy 0, policy_version 1177098 (0.0008) [2023-12-26 23:56:50,475][105692] Updated weights for policy 0, policy_version 1177108 (0.0009) [2023-12-26 23:56:50,532][105692] Updated weights for policy 0, policy_version 1177118 (0.0009) [2023-12-26 23:56:50,607][105692] Updated weights for policy 0, policy_version 1177128 (0.0008) [2023-12-26 23:56:50,783][105620] Updated weights for policy 1, policy_version 1178316 (0.0009) [2023-12-26 23:56:50,848][105620] Updated weights for policy 1, policy_version 1178326 (0.0008) [2023-12-26 23:56:50,907][105620] Updated weights for policy 1, policy_version 1178336 (0.0009) [2023-12-26 23:56:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 603086848. Throughput: 0: 9962.4, 1: 9616.1. Samples: 603073468. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:56:51,063][104569] Avg episode reward: [(0, '9354.876'), (1, '8898.221')] [2023-12-26 23:56:51,408][105692] Updated weights for policy 0, policy_version 1177138 (0.0009) [2023-12-26 23:56:51,468][105692] Updated weights for policy 0, policy_version 1177148 (0.0006) [2023-12-26 23:56:51,534][105692] Updated weights for policy 0, policy_version 1177158 (0.0007) [2023-12-26 23:56:51,690][105620] Updated weights for policy 1, policy_version 1178346 (0.0009) [2023-12-26 23:56:51,752][105620] Updated weights for policy 1, policy_version 1178356 (0.0007) [2023-12-26 23:56:51,807][105620] Updated weights for policy 1, policy_version 1178366 (0.0005) [2023-12-26 23:56:51,856][105620] Updated weights for policy 1, policy_version 1178376 (0.0005) [2023-12-26 23:56:52,316][105692] Updated weights for policy 0, policy_version 1177168 (0.0006) [2023-12-26 23:56:52,382][105692] Updated weights for policy 0, policy_version 1177178 (0.0009) [2023-12-26 23:56:52,448][105692] Updated weights for policy 0, policy_version 1177188 (0.0009) [2023-12-26 23:56:52,471][105620] Updated weights for policy 1, policy_version 1178386 (0.0008) [2023-12-26 23:56:52,523][105620] Updated weights for policy 1, policy_version 1178396 (0.0010) [2023-12-26 23:56:52,573][105620] Updated weights for policy 1, policy_version 1178406 (0.0008) [2023-12-26 23:56:53,079][105692] Updated weights for policy 0, policy_version 1177198 (0.0008) [2023-12-26 23:56:53,132][105692] Updated weights for policy 0, policy_version 1177208 (0.0009) [2023-12-26 23:56:53,189][105692] Updated weights for policy 0, policy_version 1177218 (0.0009) [2023-12-26 23:56:53,360][105620] Updated weights for policy 1, policy_version 1178416 (0.0009) [2023-12-26 23:56:53,415][105620] Updated weights for policy 1, policy_version 1178426 (0.0009) [2023-12-26 23:56:53,479][105620] Updated weights for policy 1, policy_version 1178436 (0.0009) [2023-12-26 23:56:53,837][105692] Updated weights for policy 0, policy_version 1177228 (0.0007) [2023-12-26 23:56:53,902][105692] Updated weights for policy 0, policy_version 1177238 (0.0005) [2023-12-26 23:56:53,957][105692] Updated weights for policy 0, policy_version 1177248 (0.0008) [2023-12-26 23:56:54,284][105620] Updated weights for policy 1, policy_version 1178446 (0.0009) [2023-12-26 23:56:54,339][105620] Updated weights for policy 1, policy_version 1178456 (0.0010) [2023-12-26 23:56:54,396][105620] Updated weights for policy 1, policy_version 1178466 (0.0010) [2023-12-26 23:56:54,549][105692] Updated weights for policy 0, policy_version 1177258 (0.0008) [2023-12-26 23:56:54,609][105692] Updated weights for policy 0, policy_version 1177268 (0.0007) [2023-12-26 23:56:54,671][105692] Updated weights for policy 0, policy_version 1177278 (0.0010) [2023-12-26 23:56:55,077][105620] Updated weights for policy 1, policy_version 1178476 (0.0008) [2023-12-26 23:56:55,137][105620] Updated weights for policy 1, policy_version 1178486 (0.0005) [2023-12-26 23:56:55,205][105620] Updated weights for policy 1, policy_version 1178496 (0.0005) [2023-12-26 23:56:55,394][105692] Updated weights for policy 0, policy_version 1177289 (0.0010) [2023-12-26 23:56:55,455][105692] Updated weights for policy 0, policy_version 1177299 (0.0009) [2023-12-26 23:56:55,529][105692] Updated weights for policy 0, policy_version 1177309 (0.0009) [2023-12-26 23:56:55,593][105692] Updated weights for policy 0, policy_version 1177319 (0.0009) [2023-12-26 23:56:55,738][105620] Updated weights for policy 1, policy_version 1178506 (0.0005) [2023-12-26 23:56:55,795][105620] Updated weights for policy 1, policy_version 1178516 (0.0005) [2023-12-26 23:56:55,850][105620] Updated weights for policy 1, policy_version 1178526 (0.0008) [2023-12-26 23:56:55,906][105620] Updated weights for policy 1, policy_version 1178536 (0.0008) [2023-12-26 23:56:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 603185152. Throughput: 0: 9936.2, 1: 9727.0. Samples: 603191248. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:56:56,062][104569] Avg episode reward: [(0, '9354.985'), (1, '9079.560')] [2023-12-26 23:56:56,361][105692] Updated weights for policy 0, policy_version 1177329 (0.0006) [2023-12-26 23:56:56,413][105692] Updated weights for policy 0, policy_version 1177339 (0.0006) [2023-12-26 23:56:56,457][105692] Updated weights for policy 0, policy_version 1177349 (0.0010) [2023-12-26 23:56:56,659][105620] Updated weights for policy 1, policy_version 1178546 (0.0008) [2023-12-26 23:56:56,715][105620] Updated weights for policy 1, policy_version 1178556 (0.0005) [2023-12-26 23:56:56,763][105620] Updated weights for policy 1, policy_version 1178566 (0.0005) [2023-12-26 23:56:57,150][105692] Updated weights for policy 0, policy_version 1177359 (0.0007) [2023-12-26 23:56:57,203][105692] Updated weights for policy 0, policy_version 1177369 (0.0005) [2023-12-26 23:56:57,252][105692] Updated weights for policy 0, policy_version 1177379 (0.0005) [2023-12-26 23:56:57,464][105620] Updated weights for policy 1, policy_version 1178576 (0.0008) [2023-12-26 23:56:57,518][105620] Updated weights for policy 1, policy_version 1178586 (0.0009) [2023-12-26 23:56:57,577][105620] Updated weights for policy 1, policy_version 1178596 (0.0010) [2023-12-26 23:56:57,842][105692] Updated weights for policy 0, policy_version 1177389 (0.0008) [2023-12-26 23:56:57,888][105692] Updated weights for policy 0, policy_version 1177399 (0.0010) [2023-12-26 23:56:57,932][105692] Updated weights for policy 0, policy_version 1177409 (0.0010) [2023-12-26 23:56:58,374][105620] Updated weights for policy 1, policy_version 1178606 (0.0008) [2023-12-26 23:56:58,439][105620] Updated weights for policy 1, policy_version 1178616 (0.0008) [2023-12-26 23:56:58,503][105620] Updated weights for policy 1, policy_version 1178626 (0.0008) [2023-12-26 23:56:58,723][105692] Updated weights for policy 0, policy_version 1177419 (0.0008) [2023-12-26 23:56:58,795][105692] Updated weights for policy 0, policy_version 1177429 (0.0009) [2023-12-26 23:56:58,871][105692] Updated weights for policy 0, policy_version 1177439 (0.0008) [2023-12-26 23:56:59,347][105620] Updated weights for policy 1, policy_version 1178636 (0.0008) [2023-12-26 23:56:59,418][105620] Updated weights for policy 1, policy_version 1178646 (0.0010) [2023-12-26 23:56:59,471][105620] Updated weights for policy 1, policy_version 1178656 (0.0010) [2023-12-26 23:56:59,657][105692] Updated weights for policy 0, policy_version 1177449 (0.0008) [2023-12-26 23:56:59,708][105692] Updated weights for policy 0, policy_version 1177459 (0.0009) [2023-12-26 23:56:59,757][105692] Updated weights for policy 0, policy_version 1177469 (0.0010) [2023-12-26 23:56:59,805][105692] Updated weights for policy 0, policy_version 1177479 (0.0010) [2023-12-26 23:57:00,227][105620] Updated weights for policy 1, policy_version 1178666 (0.0010) [2023-12-26 23:57:00,287][105620] Updated weights for policy 1, policy_version 1178676 (0.0006) [2023-12-26 23:57:00,355][105620] Updated weights for policy 1, policy_version 1178686 (0.0005) [2023-12-26 23:57:00,417][105620] Updated weights for policy 1, policy_version 1178696 (0.0005) [2023-12-26 23:57:00,543][105692] Updated weights for policy 0, policy_version 1177489 (0.0006) [2023-12-26 23:57:00,611][105692] Updated weights for policy 0, policy_version 1177499 (0.0006) [2023-12-26 23:57:00,669][105692] Updated weights for policy 0, policy_version 1177509 (0.0006) [2023-12-26 23:57:00,924][105620] Updated weights for policy 1, policy_version 1178706 (0.0008) [2023-12-26 23:57:00,971][105620] Updated weights for policy 1, policy_version 1178716 (0.0006) [2023-12-26 23:57:01,019][105620] Updated weights for policy 1, policy_version 1178726 (0.0007) [2023-12-26 23:57:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 603283456. Throughput: 0: 10006.8, 1: 9704.2. Samples: 603248940. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:57:01,062][104569] Avg episode reward: [(0, '9171.821'), (1, '9171.280')] [2023-12-26 23:57:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001177512_301490176.pth... [2023-12-26 23:57:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001178728_301793280.pth... [2023-12-26 23:57:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001176360_301195264.pth [2023-12-26 23:57:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001177576_301498368.pth [2023-12-26 23:57:01,286][105692] Updated weights for policy 0, policy_version 1177519 (0.0007) [2023-12-26 23:57:01,345][105692] Updated weights for policy 0, policy_version 1177529 (0.0007) [2023-12-26 23:57:01,413][105692] Updated weights for policy 0, policy_version 1177539 (0.0007) [2023-12-26 23:57:01,698][105620] Updated weights for policy 1, policy_version 1178736 (0.0006) [2023-12-26 23:57:01,768][105620] Updated weights for policy 1, policy_version 1178746 (0.0009) [2023-12-26 23:57:01,825][105620] Updated weights for policy 1, policy_version 1178756 (0.0008) [2023-12-26 23:57:02,067][105692] Updated weights for policy 0, policy_version 1177549 (0.0010) [2023-12-26 23:57:02,118][105692] Updated weights for policy 0, policy_version 1177560 (0.0007) [2023-12-26 23:57:02,168][105692] Updated weights for policy 0, policy_version 1177570 (0.0006) [2023-12-26 23:57:02,500][105620] Updated weights for policy 1, policy_version 1178766 (0.0009) [2023-12-26 23:57:02,552][105620] Updated weights for policy 1, policy_version 1178776 (0.0010) [2023-12-26 23:57:02,606][105620] Updated weights for policy 1, policy_version 1178786 (0.0010) [2023-12-26 23:57:02,891][105692] Updated weights for policy 0, policy_version 1177580 (0.0008) [2023-12-26 23:57:02,946][105692] Updated weights for policy 0, policy_version 1177590 (0.0005) [2023-12-26 23:57:03,007][105692] Updated weights for policy 0, policy_version 1177600 (0.0008) [2023-12-26 23:57:03,349][105620] Updated weights for policy 1, policy_version 1178796 (0.0008) [2023-12-26 23:57:03,397][105620] Updated weights for policy 1, policy_version 1178806 (0.0005) [2023-12-26 23:57:03,443][105620] Updated weights for policy 1, policy_version 1178816 (0.0005) [2023-12-26 23:57:03,691][105692] Updated weights for policy 0, policy_version 1177610 (0.0010) [2023-12-26 23:57:03,759][105692] Updated weights for policy 0, policy_version 1177620 (0.0010) [2023-12-26 23:57:03,823][105692] Updated weights for policy 0, policy_version 1177630 (0.0010) [2023-12-26 23:57:03,885][105692] Updated weights for policy 0, policy_version 1177640 (0.0008) [2023-12-26 23:57:04,005][105620] Updated weights for policy 1, policy_version 1178826 (0.0006) [2023-12-26 23:57:04,056][105620] Updated weights for policy 1, policy_version 1178836 (0.0010) [2023-12-26 23:57:04,118][105620] Updated weights for policy 1, policy_version 1178846 (0.0011) [2023-12-26 23:57:04,181][105620] Updated weights for policy 1, policy_version 1178856 (0.0010) [2023-12-26 23:57:04,623][105692] Updated weights for policy 0, policy_version 1177650 (0.0011) [2023-12-26 23:57:04,689][105692] Updated weights for policy 0, policy_version 1177660 (0.0011) [2023-12-26 23:57:04,738][105692] Updated weights for policy 0, policy_version 1177670 (0.0011) [2023-12-26 23:57:04,844][105620] Updated weights for policy 1, policy_version 1178866 (0.0006) [2023-12-26 23:57:04,906][105620] Updated weights for policy 1, policy_version 1178876 (0.0007) [2023-12-26 23:57:04,968][105620] Updated weights for policy 1, policy_version 1178886 (0.0005) [2023-12-26 23:57:05,490][105692] Updated weights for policy 0, policy_version 1177680 (0.0010) [2023-12-26 23:57:05,509][105620] Updated weights for policy 1, policy_version 1178896 (0.0006) [2023-12-26 23:57:05,542][105692] Updated weights for policy 0, policy_version 1177690 (0.0010) [2023-12-26 23:57:05,561][105620] Updated weights for policy 1, policy_version 1178906 (0.0009) [2023-12-26 23:57:05,600][105692] Updated weights for policy 0, policy_version 1177700 (0.0010) [2023-12-26 23:57:05,610][105620] Updated weights for policy 1, policy_version 1178916 (0.0007) [2023-12-26 23:57:06,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 603381760. Throughput: 0: 10070.3, 1: 9796.8. Samples: 603370476. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:57:06,063][104569] Avg episode reward: [(0, '9080.367'), (1, '9171.697')] [2023-12-26 23:57:06,184][105692] Updated weights for policy 0, policy_version 1177710 (0.0009) [2023-12-26 23:57:06,195][105620] Updated weights for policy 1, policy_version 1178926 (0.0007) [2023-12-26 23:57:06,239][105692] Updated weights for policy 0, policy_version 1177720 (0.0011) [2023-12-26 23:57:06,257][105620] Updated weights for policy 1, policy_version 1178936 (0.0006) [2023-12-26 23:57:06,307][105692] Updated weights for policy 0, policy_version 1177730 (0.0011) [2023-12-26 23:57:06,318][105620] Updated weights for policy 1, policy_version 1178946 (0.0006) [2023-12-26 23:57:06,978][105620] Updated weights for policy 1, policy_version 1178956 (0.0007) [2023-12-26 23:57:07,026][105620] Updated weights for policy 1, policy_version 1178966 (0.0007) [2023-12-26 23:57:07,070][105692] Updated weights for policy 0, policy_version 1177740 (0.0010) [2023-12-26 23:57:07,084][105620] Updated weights for policy 1, policy_version 1178976 (0.0008) [2023-12-26 23:57:07,132][105692] Updated weights for policy 0, policy_version 1177750 (0.0008) [2023-12-26 23:57:07,200][105692] Updated weights for policy 0, policy_version 1177760 (0.0006) [2023-12-26 23:57:07,776][105692] Updated weights for policy 0, policy_version 1177770 (0.0005) [2023-12-26 23:57:07,800][105620] Updated weights for policy 1, policy_version 1178986 (0.0007) [2023-12-26 23:57:07,826][105692] Updated weights for policy 0, policy_version 1177780 (0.0010) [2023-12-26 23:57:07,856][105620] Updated weights for policy 1, policy_version 1178996 (0.0007) [2023-12-26 23:57:07,878][105692] Updated weights for policy 0, policy_version 1177790 (0.0010) [2023-12-26 23:57:07,914][105620] Updated weights for policy 1, policy_version 1179006 (0.0006) [2023-12-26 23:57:07,937][105692] Updated weights for policy 0, policy_version 1177800 (0.0010) [2023-12-26 23:57:07,975][105620] Updated weights for policy 1, policy_version 1179016 (0.0008) [2023-12-26 23:57:08,687][105692] Updated weights for policy 0, policy_version 1177810 (0.0010) [2023-12-26 23:57:08,741][105620] Updated weights for policy 1, policy_version 1179026 (0.0006) [2023-12-26 23:57:08,746][105692] Updated weights for policy 0, policy_version 1177820 (0.0010) [2023-12-26 23:57:08,793][105620] Updated weights for policy 1, policy_version 1179036 (0.0007) [2023-12-26 23:57:08,798][105692] Updated weights for policy 0, policy_version 1177830 (0.0010) [2023-12-26 23:57:08,852][105620] Updated weights for policy 1, policy_version 1179046 (0.0009) [2023-12-26 23:57:09,572][105692] Updated weights for policy 0, policy_version 1177840 (0.0010) [2023-12-26 23:57:09,634][105692] Updated weights for policy 0, policy_version 1177850 (0.0011) [2023-12-26 23:57:09,660][105620] Updated weights for policy 1, policy_version 1179056 (0.0006) [2023-12-26 23:57:09,697][105692] Updated weights for policy 0, policy_version 1177860 (0.0010) [2023-12-26 23:57:09,719][105620] Updated weights for policy 1, policy_version 1179066 (0.0007) [2023-12-26 23:57:09,776][105620] Updated weights for policy 1, policy_version 1179076 (0.0008) [2023-12-26 23:57:10,355][105692] Updated weights for policy 0, policy_version 1177870 (0.0007) [2023-12-26 23:57:10,429][105692] Updated weights for policy 0, policy_version 1177880 (0.0007) [2023-12-26 23:57:10,475][105692] Updated weights for policy 0, policy_version 1177890 (0.0008) [2023-12-26 23:57:10,576][105620] Updated weights for policy 1, policy_version 1179086 (0.0009) [2023-12-26 23:57:10,638][105620] Updated weights for policy 1, policy_version 1179096 (0.0010) [2023-12-26 23:57:10,691][105620] Updated weights for policy 1, policy_version 1179106 (0.0008) [2023-12-26 23:57:11,039][105692] Updated weights for policy 0, policy_version 1177900 (0.0007) [2023-12-26 23:57:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 603480064. Throughput: 0: 10060.8, 1: 9834.7. Samples: 603489304. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:57:11,063][104569] Avg episode reward: [(0, '9263.001'), (1, '9262.579')] [2023-12-26 23:57:11,099][105692] Updated weights for policy 0, policy_version 1177910 (0.0010) [2023-12-26 23:57:11,164][105692] Updated weights for policy 0, policy_version 1177920 (0.0011) [2023-12-26 23:57:11,424][105620] Updated weights for policy 1, policy_version 1179116 (0.0010) [2023-12-26 23:57:11,497][105620] Updated weights for policy 1, policy_version 1179126 (0.0006) [2023-12-26 23:57:11,565][105620] Updated weights for policy 1, policy_version 1179136 (0.0005) [2023-12-26 23:57:11,918][105692] Updated weights for policy 0, policy_version 1177930 (0.0009) [2023-12-26 23:57:11,978][105692] Updated weights for policy 0, policy_version 1177940 (0.0011) [2023-12-26 23:57:12,038][105692] Updated weights for policy 0, policy_version 1177950 (0.0008) [2023-12-26 23:57:12,098][105692] Updated weights for policy 0, policy_version 1177960 (0.0005) [2023-12-26 23:57:12,283][105620] Updated weights for policy 1, policy_version 1179146 (0.0007) [2023-12-26 23:57:12,340][105620] Updated weights for policy 1, policy_version 1179156 (0.0010) [2023-12-26 23:57:12,407][105620] Updated weights for policy 1, policy_version 1179166 (0.0008) [2023-12-26 23:57:12,461][105620] Updated weights for policy 1, policy_version 1179176 (0.0007) [2023-12-26 23:57:12,732][105692] Updated weights for policy 0, policy_version 1177970 (0.0009) [2023-12-26 23:57:12,788][105692] Updated weights for policy 0, policy_version 1177980 (0.0009) [2023-12-26 23:57:12,836][105692] Updated weights for policy 0, policy_version 1177990 (0.0009) [2023-12-26 23:57:13,224][105620] Updated weights for policy 1, policy_version 1179186 (0.0006) [2023-12-26 23:57:13,288][105620] Updated weights for policy 1, policy_version 1179196 (0.0008) [2023-12-26 23:57:13,350][105620] Updated weights for policy 1, policy_version 1179206 (0.0006) [2023-12-26 23:57:13,594][105692] Updated weights for policy 0, policy_version 1178000 (0.0010) [2023-12-26 23:57:13,657][105692] Updated weights for policy 0, policy_version 1178010 (0.0010) [2023-12-26 23:57:13,710][105692] Updated weights for policy 0, policy_version 1178020 (0.0008) [2023-12-26 23:57:13,896][105620] Updated weights for policy 1, policy_version 1179216 (0.0005) [2023-12-26 23:57:13,944][105620] Updated weights for policy 1, policy_version 1179226 (0.0005) [2023-12-26 23:57:13,993][105620] Updated weights for policy 1, policy_version 1179236 (0.0009) [2023-12-26 23:57:14,410][105692] Updated weights for policy 0, policy_version 1178030 (0.0007) [2023-12-26 23:57:14,469][105692] Updated weights for policy 0, policy_version 1178040 (0.0008) [2023-12-26 23:57:14,525][105692] Updated weights for policy 0, policy_version 1178050 (0.0009) [2023-12-26 23:57:14,718][105620] Updated weights for policy 1, policy_version 1179246 (0.0009) [2023-12-26 23:57:14,766][105620] Updated weights for policy 1, policy_version 1179256 (0.0010) [2023-12-26 23:57:14,828][105620] Updated weights for policy 1, policy_version 1179266 (0.0011) [2023-12-26 23:57:15,303][105692] Updated weights for policy 0, policy_version 1178060 (0.0009) [2023-12-26 23:57:15,359][105692] Updated weights for policy 0, policy_version 1178070 (0.0008) [2023-12-26 23:57:15,420][105692] Updated weights for policy 0, policy_version 1178080 (0.0008) [2023-12-26 23:57:15,620][105620] Updated weights for policy 1, policy_version 1179276 (0.0011) [2023-12-26 23:57:15,665][105620] Updated weights for policy 1, policy_version 1179286 (0.0010) [2023-12-26 23:57:15,710][105620] Updated weights for policy 1, policy_version 1179296 (0.0010) [2023-12-26 23:57:16,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 603578368. Throughput: 0: 9991.8, 1: 9797.6. Samples: 603550004. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:57:16,062][104569] Avg episode reward: [(0, '9079.066'), (1, '9170.762')] [2023-12-26 23:57:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001179304_301940736.pth... [2023-12-26 23:57:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001178088_301637632.pth... [2023-12-26 23:57:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001178152_301645824.pth [2023-12-26 23:57:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001176936_301342720.pth [2023-12-26 23:57:16,153][105692] Updated weights for policy 0, policy_version 1178090 (0.0008) [2023-12-26 23:57:16,215][105692] Updated weights for policy 0, policy_version 1178100 (0.0005) [2023-12-26 23:57:16,276][105692] Updated weights for policy 0, policy_version 1178110 (0.0005) [2023-12-26 23:57:16,345][105692] Updated weights for policy 0, policy_version 1178120 (0.0006) [2023-12-26 23:57:16,465][105620] Updated weights for policy 1, policy_version 1179306 (0.0010) [2023-12-26 23:57:16,522][105620] Updated weights for policy 1, policy_version 1179316 (0.0008) [2023-12-26 23:57:16,577][105620] Updated weights for policy 1, policy_version 1179326 (0.0005) [2023-12-26 23:57:16,633][105620] Updated weights for policy 1, policy_version 1179336 (0.0005) [2023-12-26 23:57:17,003][105692] Updated weights for policy 0, policy_version 1178130 (0.0005) [2023-12-26 23:57:17,060][105692] Updated weights for policy 0, policy_version 1178140 (0.0008) [2023-12-26 23:57:17,106][105692] Updated weights for policy 0, policy_version 1178150 (0.0008) [2023-12-26 23:57:17,242][105620] Updated weights for policy 1, policy_version 1179346 (0.0010) [2023-12-26 23:57:17,304][105620] Updated weights for policy 1, policy_version 1179356 (0.0010) [2023-12-26 23:57:17,352][105620] Updated weights for policy 1, policy_version 1179366 (0.0009) [2023-12-26 23:57:17,675][105692] Updated weights for policy 0, policy_version 1178160 (0.0006) [2023-12-26 23:57:17,720][105692] Updated weights for policy 0, policy_version 1178170 (0.0010) [2023-12-26 23:57:17,764][105692] Updated weights for policy 0, policy_version 1178180 (0.0010) [2023-12-26 23:57:18,109][105620] Updated weights for policy 1, policy_version 1179376 (0.0010) [2023-12-26 23:57:18,174][105620] Updated weights for policy 1, policy_version 1179386 (0.0011) [2023-12-26 23:57:18,222][105620] Updated weights for policy 1, policy_version 1179396 (0.0010) [2023-12-26 23:57:18,483][105692] Updated weights for policy 0, policy_version 1178190 (0.0010) [2023-12-26 23:57:18,535][105692] Updated weights for policy 0, policy_version 1178200 (0.0010) [2023-12-26 23:57:18,591][105692] Updated weights for policy 0, policy_version 1178210 (0.0011) [2023-12-26 23:57:18,953][105620] Updated weights for policy 1, policy_version 1179406 (0.0007) [2023-12-26 23:57:19,011][105620] Updated weights for policy 1, policy_version 1179416 (0.0008) [2023-12-26 23:57:19,067][105620] Updated weights for policy 1, policy_version 1179426 (0.0011) [2023-12-26 23:57:19,181][105692] Updated weights for policy 0, policy_version 1178220 (0.0008) [2023-12-26 23:57:19,238][105692] Updated weights for policy 0, policy_version 1178230 (0.0007) [2023-12-26 23:57:19,296][105692] Updated weights for policy 0, policy_version 1178240 (0.0010) [2023-12-26 23:57:19,739][105620] Updated weights for policy 1, policy_version 1179436 (0.0009) [2023-12-26 23:57:19,799][105620] Updated weights for policy 1, policy_version 1179446 (0.0011) [2023-12-26 23:57:19,867][105620] Updated weights for policy 1, policy_version 1179456 (0.0011) [2023-12-26 23:57:20,103][105692] Updated weights for policy 0, policy_version 1178250 (0.0010) [2023-12-26 23:57:20,168][105692] Updated weights for policy 0, policy_version 1178260 (0.0009) [2023-12-26 23:57:20,233][105692] Updated weights for policy 0, policy_version 1178270 (0.0009) [2023-12-26 23:57:20,298][105692] Updated weights for policy 0, policy_version 1178280 (0.0009) [2023-12-26 23:57:20,583][105620] Updated weights for policy 1, policy_version 1179466 (0.0012) [2023-12-26 23:57:20,645][105620] Updated weights for policy 1, policy_version 1179476 (0.0010) [2023-12-26 23:57:20,717][105620] Updated weights for policy 1, policy_version 1179486 (0.0011) [2023-12-26 23:57:20,783][105620] Updated weights for policy 1, policy_version 1179496 (0.0011) [2023-12-26 23:57:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 603676672. Throughput: 0: 9923.6, 1: 9795.6. Samples: 603669128. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:57:21,063][104569] Avg episode reward: [(0, '8896.199'), (1, '8989.127')] [2023-12-26 23:57:21,079][105692] Updated weights for policy 0, policy_version 1178290 (0.0009) [2023-12-26 23:57:21,137][105692] Updated weights for policy 0, policy_version 1178300 (0.0008) [2023-12-26 23:57:21,190][105692] Updated weights for policy 0, policy_version 1178310 (0.0008) [2023-12-26 23:57:21,528][105620] Updated weights for policy 1, policy_version 1179506 (0.0005) [2023-12-26 23:57:21,594][105620] Updated weights for policy 1, policy_version 1179516 (0.0011) [2023-12-26 23:57:21,658][105620] Updated weights for policy 1, policy_version 1179526 (0.0010) [2023-12-26 23:57:21,961][105692] Updated weights for policy 0, policy_version 1178320 (0.0008) [2023-12-26 23:57:22,027][105692] Updated weights for policy 0, policy_version 1178330 (0.0006) [2023-12-26 23:57:22,095][105692] Updated weights for policy 0, policy_version 1178340 (0.0009) [2023-12-26 23:57:22,356][105620] Updated weights for policy 1, policy_version 1179536 (0.0008) [2023-12-26 23:57:22,422][105620] Updated weights for policy 1, policy_version 1179546 (0.0008) [2023-12-26 23:57:22,481][105620] Updated weights for policy 1, policy_version 1179556 (0.0008) [2023-12-26 23:57:22,763][105692] Updated weights for policy 0, policy_version 1178350 (0.0009) [2023-12-26 23:57:22,815][105692] Updated weights for policy 0, policy_version 1178360 (0.0009) [2023-12-26 23:57:22,863][105692] Updated weights for policy 0, policy_version 1178370 (0.0008) [2023-12-26 23:57:23,201][105620] Updated weights for policy 1, policy_version 1179566 (0.0008) [2023-12-26 23:57:23,252][105620] Updated weights for policy 1, policy_version 1179576 (0.0009) [2023-12-26 23:57:23,313][105620] Updated weights for policy 1, policy_version 1179586 (0.0009) [2023-12-26 23:57:23,608][105692] Updated weights for policy 0, policy_version 1178380 (0.0010) [2023-12-26 23:57:23,667][105692] Updated weights for policy 0, policy_version 1178390 (0.0009) [2023-12-26 23:57:23,734][105692] Updated weights for policy 0, policy_version 1178400 (0.0010) [2023-12-26 23:57:24,033][105620] Updated weights for policy 1, policy_version 1179596 (0.0008) [2023-12-26 23:57:24,084][105620] Updated weights for policy 1, policy_version 1179606 (0.0009) [2023-12-26 23:57:24,133][105620] Updated weights for policy 1, policy_version 1179616 (0.0009) [2023-12-26 23:57:24,513][105692] Updated weights for policy 0, policy_version 1178410 (0.0009) [2023-12-26 23:57:24,580][105692] Updated weights for policy 0, policy_version 1178420 (0.0009) [2023-12-26 23:57:24,649][105692] Updated weights for policy 0, policy_version 1178430 (0.0009) [2023-12-26 23:57:24,696][105692] Updated weights for policy 0, policy_version 1178440 (0.0009) [2023-12-26 23:57:24,861][105620] Updated weights for policy 1, policy_version 1179627 (0.0007) [2023-12-26 23:57:24,920][105620] Updated weights for policy 1, policy_version 1179637 (0.0006) [2023-12-26 23:57:24,976][105620] Updated weights for policy 1, policy_version 1179647 (0.0005) [2023-12-26 23:57:25,511][105620] Updated weights for policy 1, policy_version 1179657 (0.0005) [2023-12-26 23:57:25,558][105692] Updated weights for policy 0, policy_version 1178451 (0.0009) [2023-12-26 23:57:25,564][105620] Updated weights for policy 1, policy_version 1179667 (0.0006) [2023-12-26 23:57:25,619][105692] Updated weights for policy 0, policy_version 1178462 (0.0010) [2023-12-26 23:57:25,627][105620] Updated weights for policy 1, policy_version 1179677 (0.0009) [2023-12-26 23:57:25,672][105692] Updated weights for policy 0, policy_version 1178472 (0.0008) [2023-12-26 23:57:25,686][105620] Updated weights for policy 1, policy_version 1179687 (0.0009) [2023-12-26 23:57:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 603774976. Throughput: 0: 9830.3, 1: 9800.1. Samples: 603783828. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:57:26,062][104569] Avg episode reward: [(0, '8987.902'), (1, '9079.340')] [2023-12-26 23:57:26,248][105620] Updated weights for policy 1, policy_version 1179697 (0.0010) [2023-12-26 23:57:26,306][105620] Updated weights for policy 1, policy_version 1179707 (0.0010) [2023-12-26 23:57:26,357][105620] Updated weights for policy 1, policy_version 1179717 (0.0010) [2023-12-26 23:57:26,539][105692] Updated weights for policy 0, policy_version 1178482 (0.0008) [2023-12-26 23:57:26,599][105692] Updated weights for policy 0, policy_version 1178492 (0.0008) [2023-12-26 23:57:26,666][105692] Updated weights for policy 0, policy_version 1178502 (0.0006) [2023-12-26 23:57:26,989][105620] Updated weights for policy 1, policy_version 1179727 (0.0007) [2023-12-26 23:57:27,032][105620] Updated weights for policy 1, policy_version 1179737 (0.0005) [2023-12-26 23:57:27,082][105620] Updated weights for policy 1, policy_version 1179747 (0.0006) [2023-12-26 23:57:27,252][105692] Updated weights for policy 0, policy_version 1178512 (0.0009) [2023-12-26 23:57:27,299][105692] Updated weights for policy 0, policy_version 1178522 (0.0010) [2023-12-26 23:57:27,360][105692] Updated weights for policy 0, policy_version 1178532 (0.0010) [2023-12-26 23:57:27,706][105620] Updated weights for policy 1, policy_version 1179757 (0.0008) [2023-12-26 23:57:27,764][105620] Updated weights for policy 1, policy_version 1179767 (0.0005) [2023-12-26 23:57:27,821][105620] Updated weights for policy 1, policy_version 1179777 (0.0010) [2023-12-26 23:57:28,010][105692] Updated weights for policy 0, policy_version 1178542 (0.0009) [2023-12-26 23:57:28,057][105692] Updated weights for policy 0, policy_version 1178552 (0.0008) [2023-12-26 23:57:28,116][105692] Updated weights for policy 0, policy_version 1178562 (0.0008) [2023-12-26 23:57:28,533][105620] Updated weights for policy 1, policy_version 1179787 (0.0009) [2023-12-26 23:57:28,594][105620] Updated weights for policy 1, policy_version 1179797 (0.0006) [2023-12-26 23:57:28,659][105620] Updated weights for policy 1, policy_version 1179807 (0.0005) [2023-12-26 23:57:28,787][105692] Updated weights for policy 0, policy_version 1178572 (0.0008) [2023-12-26 23:57:28,847][105692] Updated weights for policy 0, policy_version 1178582 (0.0011) [2023-12-26 23:57:28,906][105692] Updated weights for policy 0, policy_version 1178592 (0.0011) [2023-12-26 23:57:29,267][105620] Updated weights for policy 1, policy_version 1179817 (0.0006) [2023-12-26 23:57:29,334][105620] Updated weights for policy 1, policy_version 1179827 (0.0011) [2023-12-26 23:57:29,404][105620] Updated weights for policy 1, policy_version 1179837 (0.0008) [2023-12-26 23:57:29,467][105620] Updated weights for policy 1, policy_version 1179847 (0.0009) [2023-12-26 23:57:29,625][105692] Updated weights for policy 0, policy_version 1178602 (0.0010) [2023-12-26 23:57:29,695][105692] Updated weights for policy 0, policy_version 1178612 (0.0006) [2023-12-26 23:57:29,753][105692] Updated weights for policy 0, policy_version 1178622 (0.0007) [2023-12-26 23:57:29,817][105692] Updated weights for policy 0, policy_version 1178632 (0.0006) [2023-12-26 23:57:30,215][105620] Updated weights for policy 1, policy_version 1179857 (0.0010) [2023-12-26 23:57:30,263][105620] Updated weights for policy 1, policy_version 1179867 (0.0010) [2023-12-26 23:57:30,317][105620] Updated weights for policy 1, policy_version 1179877 (0.0010) [2023-12-26 23:57:30,471][105692] Updated weights for policy 0, policy_version 1178642 (0.0009) [2023-12-26 23:57:30,518][105692] Updated weights for policy 0, policy_version 1178652 (0.0008) [2023-12-26 23:57:30,577][105692] Updated weights for policy 0, policy_version 1178662 (0.0008) [2023-12-26 23:57:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 603873280. Throughput: 0: 9912.3, 1: 9914.4. Samples: 603847288. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:57:31,062][104569] Avg episode reward: [(0, '8987.686'), (1, '9260.771')] [2023-12-26 23:57:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001178664_301785088.pth... [2023-12-26 23:57:31,072][105620] Updated weights for policy 1, policy_version 1179887 (0.0010) [2023-12-26 23:57:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001177512_301490176.pth [2023-12-26 23:57:31,132][105620] Updated weights for policy 1, policy_version 1179897 (0.0010) [2023-12-26 23:57:31,196][105620] Updated weights for policy 1, policy_version 1179907 (0.0011) [2023-12-26 23:57:31,216][105692] Updated weights for policy 0, policy_version 1178672 (0.0007) [2023-12-26 23:57:31,220][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001179912_302096384.pth... [2023-12-26 23:57:31,223][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001178728_301793280.pth [2023-12-26 23:57:31,272][105692] Updated weights for policy 0, policy_version 1178682 (0.0008) [2023-12-26 23:57:31,333][105692] Updated weights for policy 0, policy_version 1178692 (0.0008) [2023-12-26 23:57:31,899][105620] Updated weights for policy 1, policy_version 1179917 (0.0008) [2023-12-26 23:57:31,960][105620] Updated weights for policy 1, policy_version 1179927 (0.0010) [2023-12-26 23:57:32,008][105620] Updated weights for policy 1, policy_version 1179937 (0.0010) [2023-12-26 23:57:32,126][105692] Updated weights for policy 0, policy_version 1178702 (0.0006) [2023-12-26 23:57:32,190][105692] Updated weights for policy 0, policy_version 1178712 (0.0005) [2023-12-26 23:57:32,255][105692] Updated weights for policy 0, policy_version 1178722 (0.0005) [2023-12-26 23:57:32,748][105620] Updated weights for policy 1, policy_version 1179947 (0.0010) [2023-12-26 23:57:32,800][105620] Updated weights for policy 1, policy_version 1179957 (0.0010) [2023-12-26 23:57:32,848][105620] Updated weights for policy 1, policy_version 1179967 (0.0010) [2023-12-26 23:57:32,862][105692] Updated weights for policy 0, policy_version 1178732 (0.0007) [2023-12-26 23:57:32,915][105692] Updated weights for policy 0, policy_version 1178742 (0.0006) [2023-12-26 23:57:32,981][105692] Updated weights for policy 0, policy_version 1178752 (0.0008) [2023-12-26 23:57:33,597][105620] Updated weights for policy 1, policy_version 1179977 (0.0010) [2023-12-26 23:57:33,645][105620] Updated weights for policy 1, policy_version 1179987 (0.0010) [2023-12-26 23:57:33,692][105620] Updated weights for policy 1, policy_version 1179997 (0.0010) [2023-12-26 23:57:33,723][105692] Updated weights for policy 0, policy_version 1178762 (0.0007) [2023-12-26 23:57:33,750][105620] Updated weights for policy 1, policy_version 1180007 (0.0010) [2023-12-26 23:57:33,792][105692] Updated weights for policy 0, policy_version 1178772 (0.0005) [2023-12-26 23:57:33,838][105692] Updated weights for policy 0, policy_version 1178782 (0.0005) [2023-12-26 23:57:33,889][105692] Updated weights for policy 0, policy_version 1178792 (0.0005) [2023-12-26 23:57:34,500][105620] Updated weights for policy 1, policy_version 1180017 (0.0009) [2023-12-26 23:57:34,525][105692] Updated weights for policy 0, policy_version 1178802 (0.0007) [2023-12-26 23:57:34,552][105620] Updated weights for policy 1, policy_version 1180027 (0.0009) [2023-12-26 23:57:34,586][105692] Updated weights for policy 0, policy_version 1178812 (0.0006) [2023-12-26 23:57:34,612][105620] Updated weights for policy 1, policy_version 1180037 (0.0008) [2023-12-26 23:57:34,639][105692] Updated weights for policy 0, policy_version 1178822 (0.0006) [2023-12-26 23:57:35,247][105692] Updated weights for policy 0, policy_version 1178832 (0.0006) [2023-12-26 23:57:35,279][105620] Updated weights for policy 1, policy_version 1180047 (0.0007) [2023-12-26 23:57:35,306][105692] Updated weights for policy 0, policy_version 1178842 (0.0006) [2023-12-26 23:57:35,338][105620] Updated weights for policy 1, policy_version 1180057 (0.0007) [2023-12-26 23:57:35,368][105692] Updated weights for policy 0, policy_version 1178852 (0.0007) [2023-12-26 23:57:35,399][105620] Updated weights for policy 1, policy_version 1180067 (0.0006) [2023-12-26 23:57:35,926][105692] Updated weights for policy 0, policy_version 1178862 (0.0008) [2023-12-26 23:57:35,977][105692] Updated weights for policy 0, policy_version 1178872 (0.0009) [2023-12-26 23:57:36,024][105692] Updated weights for policy 0, policy_version 1178882 (0.0009) [2023-12-26 23:57:36,045][105585] KL-divergence is very high: 109.8636 [2023-12-26 23:57:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 603979776. Throughput: 0: 9862.2, 1: 9927.6. Samples: 603964008. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:57:36,062][104569] Avg episode reward: [(0, '9079.702'), (1, '9170.745')] [2023-12-26 23:57:36,178][105620] Updated weights for policy 1, policy_version 1180077 (0.0010) [2023-12-26 23:57:36,242][105620] Updated weights for policy 1, policy_version 1180087 (0.0009) [2023-12-26 23:57:36,305][105620] Updated weights for policy 1, policy_version 1180097 (0.0009) [2023-12-26 23:57:36,824][105692] Updated weights for policy 0, policy_version 1178892 (0.0009) [2023-12-26 23:57:36,878][105692] Updated weights for policy 0, policy_version 1178902 (0.0009) [2023-12-26 23:57:36,929][105692] Updated weights for policy 0, policy_version 1178912 (0.0009) [2023-12-26 23:57:36,959][105620] Updated weights for policy 1, policy_version 1180107 (0.0008) [2023-12-26 23:57:37,022][105620] Updated weights for policy 1, policy_version 1180117 (0.0008) [2023-12-26 23:57:37,083][105620] Updated weights for policy 1, policy_version 1180127 (0.0009) [2023-12-26 23:57:37,675][105692] Updated weights for policy 0, policy_version 1178922 (0.0009) [2023-12-26 23:57:37,731][105692] Updated weights for policy 0, policy_version 1178932 (0.0009) [2023-12-26 23:57:37,783][105692] Updated weights for policy 0, policy_version 1178942 (0.0008) [2023-12-26 23:57:37,842][105692] Updated weights for policy 0, policy_version 1178952 (0.0009) [2023-12-26 23:57:37,863][105620] Updated weights for policy 1, policy_version 1180137 (0.0009) [2023-12-26 23:57:37,924][105620] Updated weights for policy 1, policy_version 1180148 (0.0009) [2023-12-26 23:57:37,982][105620] Updated weights for policy 1, policy_version 1180158 (0.0010) [2023-12-26 23:57:38,037][105620] Updated weights for policy 1, policy_version 1180168 (0.0008) [2023-12-26 23:57:38,484][105692] Updated weights for policy 0, policy_version 1178962 (0.0008) [2023-12-26 23:57:38,555][105692] Updated weights for policy 0, policy_version 1178972 (0.0008) [2023-12-26 23:57:38,617][105692] Updated weights for policy 0, policy_version 1178982 (0.0009) [2023-12-26 23:57:38,888][105620] Updated weights for policy 1, policy_version 1180178 (0.0009) [2023-12-26 23:57:38,943][105620] Updated weights for policy 1, policy_version 1180188 (0.0009) [2023-12-26 23:57:38,989][105620] Updated weights for policy 1, policy_version 1180198 (0.0008) [2023-12-26 23:57:39,290][105692] Updated weights for policy 0, policy_version 1178992 (0.0008) [2023-12-26 23:57:39,370][105692] Updated weights for policy 0, policy_version 1179002 (0.0008) [2023-12-26 23:57:39,436][105692] Updated weights for policy 0, policy_version 1179012 (0.0008) [2023-12-26 23:57:39,715][105620] Updated weights for policy 1, policy_version 1180208 (0.0008) [2023-12-26 23:57:39,779][105620] Updated weights for policy 1, policy_version 1180218 (0.0009) [2023-12-26 23:57:39,842][105620] Updated weights for policy 1, policy_version 1180228 (0.0009) [2023-12-26 23:57:40,254][105692] Updated weights for policy 0, policy_version 1179022 (0.0010) [2023-12-26 23:57:40,311][105692] Updated weights for policy 0, policy_version 1179032 (0.0011) [2023-12-26 23:57:40,364][105692] Updated weights for policy 0, policy_version 1179042 (0.0011) [2023-12-26 23:57:40,600][105620] Updated weights for policy 1, policy_version 1180238 (0.0008) [2023-12-26 23:57:40,659][105620] Updated weights for policy 1, policy_version 1180248 (0.0008) [2023-12-26 23:57:40,718][105620] Updated weights for policy 1, policy_version 1180258 (0.0008) [2023-12-26 23:57:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 604069888. Throughput: 0: 9882.0, 1: 9862.1. Samples: 604079732. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:57:41,062][105692] Updated weights for policy 0, policy_version 1179052 (0.0011) [2023-12-26 23:57:41,062][104569] Avg episode reward: [(0, '9173.025'), (1, '9262.658')] [2023-12-26 23:57:41,129][105692] Updated weights for policy 0, policy_version 1179062 (0.0011) [2023-12-26 23:57:41,192][105692] Updated weights for policy 0, policy_version 1179072 (0.0011) [2023-12-26 23:57:41,499][105620] Updated weights for policy 1, policy_version 1180268 (0.0007) [2023-12-26 23:57:41,561][105620] Updated weights for policy 1, policy_version 1180278 (0.0008) [2023-12-26 23:57:41,622][105620] Updated weights for policy 1, policy_version 1180288 (0.0008) [2023-12-26 23:57:41,913][105692] Updated weights for policy 0, policy_version 1179082 (0.0007) [2023-12-26 23:57:41,967][105692] Updated weights for policy 0, policy_version 1179092 (0.0009) [2023-12-26 23:57:42,023][105692] Updated weights for policy 0, policy_version 1179102 (0.0009) [2023-12-26 23:57:42,083][105692] Updated weights for policy 0, policy_version 1179112 (0.0006) [2023-12-26 23:57:42,344][105620] Updated weights for policy 1, policy_version 1180298 (0.0009) [2023-12-26 23:57:42,412][105620] Updated weights for policy 1, policy_version 1180308 (0.0007) [2023-12-26 23:57:42,477][105620] Updated weights for policy 1, policy_version 1180318 (0.0006) [2023-12-26 23:57:42,544][105620] Updated weights for policy 1, policy_version 1180328 (0.0007) [2023-12-26 23:57:42,751][105692] Updated weights for policy 0, policy_version 1179122 (0.0006) [2023-12-26 23:57:42,810][105692] Updated weights for policy 0, policy_version 1179132 (0.0009) [2023-12-26 23:57:42,879][105692] Updated weights for policy 0, policy_version 1179142 (0.0005) [2023-12-26 23:57:43,278][105620] Updated weights for policy 1, policy_version 1180338 (0.0007) [2023-12-26 23:57:43,329][105620] Updated weights for policy 1, policy_version 1180348 (0.0007) [2023-12-26 23:57:43,376][105620] Updated weights for policy 1, policy_version 1180358 (0.0008) [2023-12-26 23:57:43,519][105692] Updated weights for policy 0, policy_version 1179152 (0.0006) [2023-12-26 23:57:43,571][105692] Updated weights for policy 0, policy_version 1179162 (0.0010) [2023-12-26 23:57:43,619][105692] Updated weights for policy 0, policy_version 1179172 (0.0010) [2023-12-26 23:57:44,186][105620] Updated weights for policy 1, policy_version 1180368 (0.0009) [2023-12-26 23:57:44,242][105620] Updated weights for policy 1, policy_version 1180378 (0.0009) [2023-12-26 23:57:44,256][105692] Updated weights for policy 0, policy_version 1179182 (0.0007) [2023-12-26 23:57:44,295][105620] Updated weights for policy 1, policy_version 1180388 (0.0006) [2023-12-26 23:57:44,318][105692] Updated weights for policy 0, policy_version 1179192 (0.0011) [2023-12-26 23:57:44,376][105692] Updated weights for policy 0, policy_version 1179202 (0.0010) [2023-12-26 23:57:45,052][105620] Updated weights for policy 1, policy_version 1180398 (0.0007) [2023-12-26 23:57:45,114][105620] Updated weights for policy 1, policy_version 1180408 (0.0008) [2023-12-26 23:57:45,117][105692] Updated weights for policy 0, policy_version 1179212 (0.0010) [2023-12-26 23:57:45,170][105692] Updated weights for policy 0, policy_version 1179222 (0.0011) [2023-12-26 23:57:45,176][105620] Updated weights for policy 1, policy_version 1180418 (0.0006) [2023-12-26 23:57:45,219][105692] Updated weights for policy 0, policy_version 1179232 (0.0011) [2023-12-26 23:57:45,934][105620] Updated weights for policy 1, policy_version 1180428 (0.0007) [2023-12-26 23:57:45,983][105692] Updated weights for policy 0, policy_version 1179242 (0.0009) [2023-12-26 23:57:45,987][105620] Updated weights for policy 1, policy_version 1180438 (0.0008) [2023-12-26 23:57:46,035][105692] Updated weights for policy 0, policy_version 1179252 (0.0010) [2023-12-26 23:57:46,045][105620] Updated weights for policy 1, policy_version 1180448 (0.0006) [2023-12-26 23:57:46,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 604160000. Throughput: 0: 9880.9, 1: 9884.4. Samples: 604138380. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:57:46,062][104569] Avg episode reward: [(0, '9082.324'), (1, '9354.264')] [2023-12-26 23:57:46,087][105692] Updated weights for policy 0, policy_version 1179262 (0.0008) [2023-12-26 23:57:46,087][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001180456_302235648.pth... [2023-12-26 23:57:46,090][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001179304_301940736.pth [2023-12-26 23:57:46,140][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001179272_301940736.pth... [2023-12-26 23:57:46,141][105692] Updated weights for policy 0, policy_version 1179272 (0.0005) [2023-12-26 23:57:46,144][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001178088_301637632.pth [2023-12-26 23:57:46,735][105692] Updated weights for policy 0, policy_version 1179282 (0.0010) [2023-12-26 23:57:46,789][105692] Updated weights for policy 0, policy_version 1179292 (0.0010) [2023-12-26 23:57:46,851][105692] Updated weights for policy 0, policy_version 1179302 (0.0008) [2023-12-26 23:57:46,880][105620] Updated weights for policy 1, policy_version 1180458 (0.0007) [2023-12-26 23:57:46,935][105620] Updated weights for policy 1, policy_version 1180468 (0.0010) [2023-12-26 23:57:46,995][105620] Updated weights for policy 1, policy_version 1180478 (0.0010) [2023-12-26 23:57:47,051][105620] Updated weights for policy 1, policy_version 1180488 (0.0011) [2023-12-26 23:57:47,485][105692] Updated weights for policy 0, policy_version 1179312 (0.0008) [2023-12-26 23:57:47,530][105692] Updated weights for policy 0, policy_version 1179322 (0.0008) [2023-12-26 23:57:47,576][105692] Updated weights for policy 0, policy_version 1179332 (0.0009) [2023-12-26 23:57:47,854][105620] Updated weights for policy 1, policy_version 1180498 (0.0009) [2023-12-26 23:57:47,911][105620] Updated weights for policy 1, policy_version 1180508 (0.0010) [2023-12-26 23:57:47,970][105620] Updated weights for policy 1, policy_version 1180518 (0.0011) [2023-12-26 23:57:48,285][105692] Updated weights for policy 0, policy_version 1179342 (0.0007) [2023-12-26 23:57:48,348][105692] Updated weights for policy 0, policy_version 1179352 (0.0009) [2023-12-26 23:57:48,410][105692] Updated weights for policy 0, policy_version 1179362 (0.0011) [2023-12-26 23:57:48,725][105620] Updated weights for policy 1, policy_version 1180528 (0.0011) [2023-12-26 23:57:48,785][105620] Updated weights for policy 1, policy_version 1180538 (0.0011) [2023-12-26 23:57:48,835][105620] Updated weights for policy 1, policy_version 1180548 (0.0011) [2023-12-26 23:57:49,061][105692] Updated weights for policy 0, policy_version 1179372 (0.0011) [2023-12-26 23:57:49,120][105692] Updated weights for policy 0, policy_version 1179382 (0.0010) [2023-12-26 23:57:49,182][105692] Updated weights for policy 0, policy_version 1179392 (0.0011) [2023-12-26 23:57:49,461][105620] Updated weights for policy 1, policy_version 1180558 (0.0009) [2023-12-26 23:57:49,520][105620] Updated weights for policy 1, policy_version 1180568 (0.0008) [2023-12-26 23:57:49,564][105620] Updated weights for policy 1, policy_version 1180578 (0.0008) [2023-12-26 23:57:49,868][105692] Updated weights for policy 0, policy_version 1179402 (0.0011) [2023-12-26 23:57:49,928][105692] Updated weights for policy 0, policy_version 1179412 (0.0009) [2023-12-26 23:57:49,988][105692] Updated weights for policy 0, policy_version 1179422 (0.0008) [2023-12-26 23:57:50,049][105692] Updated weights for policy 0, policy_version 1179432 (0.0008) [2023-12-26 23:57:50,323][105620] Updated weights for policy 1, policy_version 1180588 (0.0009) [2023-12-26 23:57:50,379][105620] Updated weights for policy 1, policy_version 1180598 (0.0010) [2023-12-26 23:57:50,442][105620] Updated weights for policy 1, policy_version 1180608 (0.0011) [2023-12-26 23:57:50,742][105692] Updated weights for policy 0, policy_version 1179442 (0.0009) [2023-12-26 23:57:50,796][105692] Updated weights for policy 0, policy_version 1179452 (0.0008) [2023-12-26 23:57:50,852][105692] Updated weights for policy 0, policy_version 1179462 (0.0007) [2023-12-26 23:57:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 604266496. Throughput: 0: 9926.8, 1: 9733.0. Samples: 604255164. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:57:51,062][104569] Avg episode reward: [(0, '9263.390'), (1, '9079.377')] [2023-12-26 23:57:51,239][105620] Updated weights for policy 1, policy_version 1180618 (0.0011) [2023-12-26 23:57:51,291][105620] Updated weights for policy 1, policy_version 1180628 (0.0009) [2023-12-26 23:57:51,348][105620] Updated weights for policy 1, policy_version 1180638 (0.0009) [2023-12-26 23:57:51,418][105620] Updated weights for policy 1, policy_version 1180648 (0.0008) [2023-12-26 23:57:51,549][105692] Updated weights for policy 0, policy_version 1179472 (0.0006) [2023-12-26 23:57:51,609][105692] Updated weights for policy 0, policy_version 1179482 (0.0009) [2023-12-26 23:57:51,668][105692] Updated weights for policy 0, policy_version 1179492 (0.0008) [2023-12-26 23:57:52,270][105620] Updated weights for policy 1, policy_version 1180658 (0.0008) [2023-12-26 23:57:52,279][105692] Updated weights for policy 0, policy_version 1179502 (0.0008) [2023-12-26 23:57:52,332][105620] Updated weights for policy 1, policy_version 1180668 (0.0006) [2023-12-26 23:57:52,342][105692] Updated weights for policy 0, policy_version 1179512 (0.0008) [2023-12-26 23:57:52,396][105620] Updated weights for policy 1, policy_version 1180678 (0.0008) [2023-12-26 23:57:52,408][105692] Updated weights for policy 0, policy_version 1179522 (0.0008) [2023-12-26 23:57:53,052][105692] Updated weights for policy 0, policy_version 1179532 (0.0010) [2023-12-26 23:57:53,111][105692] Updated weights for policy 0, policy_version 1179542 (0.0009) [2023-12-26 23:57:53,168][105692] Updated weights for policy 0, policy_version 1179552 (0.0008) [2023-12-26 23:57:53,197][105620] Updated weights for policy 1, policy_version 1180688 (0.0009) [2023-12-26 23:57:53,249][105620] Updated weights for policy 1, policy_version 1180698 (0.0009) [2023-12-26 23:57:53,300][105620] Updated weights for policy 1, policy_version 1180708 (0.0008) [2023-12-26 23:57:53,850][105692] Updated weights for policy 0, policy_version 1179562 (0.0007) [2023-12-26 23:57:53,901][105692] Updated weights for policy 0, policy_version 1179572 (0.0005) [2023-12-26 23:57:53,951][105692] Updated weights for policy 0, policy_version 1179582 (0.0007) [2023-12-26 23:57:54,002][105692] Updated weights for policy 0, policy_version 1179592 (0.0009) [2023-12-26 23:57:54,118][105620] Updated weights for policy 1, policy_version 1180718 (0.0009) [2023-12-26 23:57:54,176][105620] Updated weights for policy 1, policy_version 1180728 (0.0009) [2023-12-26 23:57:54,222][105620] Updated weights for policy 1, policy_version 1180738 (0.0008) [2023-12-26 23:57:54,741][105692] Updated weights for policy 0, policy_version 1179602 (0.0005) [2023-12-26 23:57:54,787][105692] Updated weights for policy 0, policy_version 1179612 (0.0005) [2023-12-26 23:57:54,833][105692] Updated weights for policy 0, policy_version 1179622 (0.0005) [2023-12-26 23:57:54,842][105620] Updated weights for policy 1, policy_version 1180748 (0.0007) [2023-12-26 23:57:54,885][105620] Updated weights for policy 1, policy_version 1180758 (0.0005) [2023-12-26 23:57:54,932][105620] Updated weights for policy 1, policy_version 1180768 (0.0005) [2023-12-26 23:57:55,492][105692] Updated weights for policy 0, policy_version 1179632 (0.0009) [2023-12-26 23:57:55,543][105692] Updated weights for policy 0, policy_version 1179642 (0.0009) [2023-12-26 23:57:55,595][105692] Updated weights for policy 0, policy_version 1179652 (0.0011) [2023-12-26 23:57:55,653][105620] Updated weights for policy 1, policy_version 1180778 (0.0009) [2023-12-26 23:57:55,700][105620] Updated weights for policy 1, policy_version 1180788 (0.0009) [2023-12-26 23:57:55,748][105620] Updated weights for policy 1, policy_version 1180798 (0.0009) [2023-12-26 23:57:55,794][105620] Updated weights for policy 1, policy_version 1180808 (0.0008) [2023-12-26 23:57:56,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 604364800. Throughput: 0: 9958.4, 1: 9645.2. Samples: 604371464. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:57:56,062][104569] Avg episode reward: [(0, '9172.619'), (1, '8988.152')] [2023-12-26 23:57:56,398][105692] Updated weights for policy 0, policy_version 1179662 (0.0009) [2023-12-26 23:57:56,462][105692] Updated weights for policy 0, policy_version 1179672 (0.0008) [2023-12-26 23:57:56,500][105620] Updated weights for policy 1, policy_version 1180818 (0.0007) [2023-12-26 23:57:56,518][105692] Updated weights for policy 0, policy_version 1179682 (0.0008) [2023-12-26 23:57:56,556][105620] Updated weights for policy 1, policy_version 1180828 (0.0006) [2023-12-26 23:57:56,616][105620] Updated weights for policy 1, policy_version 1180838 (0.0009) [2023-12-26 23:57:57,285][105692] Updated weights for policy 0, policy_version 1179692 (0.0008) [2023-12-26 23:57:57,327][105620] Updated weights for policy 1, policy_version 1180848 (0.0008) [2023-12-26 23:57:57,343][105692] Updated weights for policy 0, policy_version 1179702 (0.0006) [2023-12-26 23:57:57,381][105620] Updated weights for policy 1, policy_version 1180858 (0.0007) [2023-12-26 23:57:57,395][105692] Updated weights for policy 0, policy_version 1179712 (0.0008) [2023-12-26 23:57:57,437][105620] Updated weights for policy 1, policy_version 1180868 (0.0007) [2023-12-26 23:57:58,142][105620] Updated weights for policy 1, policy_version 1180878 (0.0009) [2023-12-26 23:57:58,178][105692] Updated weights for policy 0, policy_version 1179722 (0.0006) [2023-12-26 23:57:58,204][105620] Updated weights for policy 1, policy_version 1180888 (0.0007) [2023-12-26 23:57:58,234][105692] Updated weights for policy 0, policy_version 1179732 (0.0008) [2023-12-26 23:57:58,261][105620] Updated weights for policy 1, policy_version 1180898 (0.0006) [2023-12-26 23:57:58,292][105692] Updated weights for policy 0, policy_version 1179742 (0.0008) [2023-12-26 23:57:58,355][105692] Updated weights for policy 0, policy_version 1179752 (0.0008) [2023-12-26 23:57:59,096][105620] Updated weights for policy 1, policy_version 1180908 (0.0009) [2023-12-26 23:57:59,162][105620] Updated weights for policy 1, policy_version 1180918 (0.0009) [2023-12-26 23:57:59,233][105620] Updated weights for policy 1, policy_version 1180928 (0.0008) [2023-12-26 23:57:59,256][105692] Updated weights for policy 0, policy_version 1179762 (0.0008) [2023-12-26 23:57:59,316][105692] Updated weights for policy 0, policy_version 1179772 (0.0007) [2023-12-26 23:57:59,379][105692] Updated weights for policy 0, policy_version 1179782 (0.0009) [2023-12-26 23:58:00,018][105620] Updated weights for policy 1, policy_version 1180938 (0.0007) [2023-12-26 23:58:00,069][105620] Updated weights for policy 1, policy_version 1180948 (0.0009) [2023-12-26 23:58:00,130][105620] Updated weights for policy 1, policy_version 1180958 (0.0008) [2023-12-26 23:58:00,153][105692] Updated weights for policy 0, policy_version 1179792 (0.0008) [2023-12-26 23:58:00,196][105620] Updated weights for policy 1, policy_version 1180968 (0.0009) [2023-12-26 23:58:00,216][105692] Updated weights for policy 0, policy_version 1179802 (0.0005) [2023-12-26 23:58:00,278][105692] Updated weights for policy 0, policy_version 1179812 (0.0007) [2023-12-26 23:58:00,825][105692] Updated weights for policy 0, policy_version 1179822 (0.0007) [2023-12-26 23:58:00,876][105692] Updated weights for policy 0, policy_version 1179832 (0.0005) [2023-12-26 23:58:00,921][105692] Updated weights for policy 0, policy_version 1179842 (0.0005) [2023-12-26 23:58:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 604454912. Throughput: 0: 9884.9, 1: 9620.7. Samples: 604427756. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:58:01,063][104569] Avg episode reward: [(0, '9172.027'), (1, '9171.107')] [2023-12-26 23:58:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001179848_302088192.pth... [2023-12-26 23:58:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001178664_301785088.pth [2023-12-26 23:58:01,076][105620] Updated weights for policy 1, policy_version 1180978 (0.0007) [2023-12-26 23:58:01,145][105620] Updated weights for policy 1, policy_version 1180988 (0.0008) [2023-12-26 23:58:01,205][105620] Updated weights for policy 1, policy_version 1180998 (0.0009) [2023-12-26 23:58:01,213][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001181000_302374912.pth... [2023-12-26 23:58:01,217][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001179912_302096384.pth [2023-12-26 23:58:01,536][105692] Updated weights for policy 0, policy_version 1179852 (0.0007) [2023-12-26 23:58:01,601][105692] Updated weights for policy 0, policy_version 1179862 (0.0006) [2023-12-26 23:58:01,670][105692] Updated weights for policy 0, policy_version 1179872 (0.0009) [2023-12-26 23:58:01,936][105620] Updated weights for policy 1, policy_version 1181008 (0.0006) [2023-12-26 23:58:02,005][105620] Updated weights for policy 1, policy_version 1181018 (0.0005) [2023-12-26 23:58:02,072][105620] Updated weights for policy 1, policy_version 1181028 (0.0007) [2023-12-26 23:58:02,330][105692] Updated weights for policy 0, policy_version 1179882 (0.0009) [2023-12-26 23:58:02,385][105692] Updated weights for policy 0, policy_version 1179892 (0.0008) [2023-12-26 23:58:02,454][105692] Updated weights for policy 0, policy_version 1179902 (0.0009) [2023-12-26 23:58:02,514][105692] Updated weights for policy 0, policy_version 1179912 (0.0007) [2023-12-26 23:58:02,799][105620] Updated weights for policy 1, policy_version 1181038 (0.0009) [2023-12-26 23:58:02,862][105620] Updated weights for policy 1, policy_version 1181048 (0.0009) [2023-12-26 23:58:02,926][105620] Updated weights for policy 1, policy_version 1181058 (0.0009) [2023-12-26 23:58:03,115][105692] Updated weights for policy 0, policy_version 1179922 (0.0009) [2023-12-26 23:58:03,168][105692] Updated weights for policy 0, policy_version 1179932 (0.0009) [2023-12-26 23:58:03,215][105692] Updated weights for policy 0, policy_version 1179942 (0.0009) [2023-12-26 23:58:03,697][105620] Updated weights for policy 1, policy_version 1181068 (0.0009) [2023-12-26 23:58:03,750][105620] Updated weights for policy 1, policy_version 1181078 (0.0009) [2023-12-26 23:58:03,803][105620] Updated weights for policy 1, policy_version 1181088 (0.0009) [2023-12-26 23:58:03,881][105692] Updated weights for policy 0, policy_version 1179952 (0.0008) [2023-12-26 23:58:03,935][105692] Updated weights for policy 0, policy_version 1179962 (0.0005) [2023-12-26 23:58:04,000][105692] Updated weights for policy 0, policy_version 1179972 (0.0009) [2023-12-26 23:58:04,433][105620] Updated weights for policy 1, policy_version 1181099 (0.0009) [2023-12-26 23:58:04,489][105620] Updated weights for policy 1, policy_version 1181109 (0.0005) [2023-12-26 23:58:04,551][105620] Updated weights for policy 1, policy_version 1181119 (0.0007) [2023-12-26 23:58:04,740][105692] Updated weights for policy 0, policy_version 1179982 (0.0009) [2023-12-26 23:58:04,793][105692] Updated weights for policy 0, policy_version 1179992 (0.0009) [2023-12-26 23:58:04,850][105692] Updated weights for policy 0, policy_version 1180002 (0.0009) [2023-12-26 23:58:05,231][105620] Updated weights for policy 1, policy_version 1181129 (0.0008) [2023-12-26 23:58:05,282][105620] Updated weights for policy 1, policy_version 1181139 (0.0005) [2023-12-26 23:58:05,331][105620] Updated weights for policy 1, policy_version 1181149 (0.0005) [2023-12-26 23:58:05,382][105620] Updated weights for policy 1, policy_version 1181159 (0.0005) [2023-12-26 23:58:05,715][105692] Updated weights for policy 0, policy_version 1180012 (0.0009) [2023-12-26 23:58:05,777][105692] Updated weights for policy 0, policy_version 1180022 (0.0008) [2023-12-26 23:58:05,837][105692] Updated weights for policy 0, policy_version 1180032 (0.0008) [2023-12-26 23:58:06,020][105620] Updated weights for policy 1, policy_version 1181169 (0.0010) [2023-12-26 23:58:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 604553216. Throughput: 0: 9893.8, 1: 9557.2. Samples: 604544420. Policy #0 lag: (min: 28.0, avg: 30.6, max: 60.0) [2023-12-26 23:58:06,062][104569] Avg episode reward: [(0, '9351.325'), (1, '9172.126')] [2023-12-26 23:58:06,080][105620] Updated weights for policy 1, policy_version 1181179 (0.0010) [2023-12-26 23:58:06,140][105620] Updated weights for policy 1, policy_version 1181189 (0.0009) [2023-12-26 23:58:06,606][105692] Updated weights for policy 0, policy_version 1180042 (0.0008) [2023-12-26 23:58:06,662][105692] Updated weights for policy 0, policy_version 1180052 (0.0008) [2023-12-26 23:58:06,716][105692] Updated weights for policy 0, policy_version 1180062 (0.0009) [2023-12-26 23:58:06,775][105692] Updated weights for policy 0, policy_version 1180072 (0.0008) [2023-12-26 23:58:06,904][105620] Updated weights for policy 1, policy_version 1181199 (0.0009) [2023-12-26 23:58:06,949][105620] Updated weights for policy 1, policy_version 1181209 (0.0010) [2023-12-26 23:58:06,997][105620] Updated weights for policy 1, policy_version 1181219 (0.0010) [2023-12-26 23:58:07,486][105692] Updated weights for policy 0, policy_version 1180082 (0.0010) [2023-12-26 23:58:07,531][105692] Updated weights for policy 0, policy_version 1180092 (0.0010) [2023-12-26 23:58:07,586][105692] Updated weights for policy 0, policy_version 1180102 (0.0010) [2023-12-26 23:58:07,721][105620] Updated weights for policy 1, policy_version 1181229 (0.0010) [2023-12-26 23:58:07,780][105620] Updated weights for policy 1, policy_version 1181239 (0.0011) [2023-12-26 23:58:07,843][105620] Updated weights for policy 1, policy_version 1181249 (0.0011) [2023-12-26 23:58:08,261][105692] Updated weights for policy 0, policy_version 1180112 (0.0011) [2023-12-26 23:58:08,311][105692] Updated weights for policy 0, policy_version 1180122 (0.0009) [2023-12-26 23:58:08,375][105692] Updated weights for policy 0, policy_version 1180132 (0.0008) [2023-12-26 23:58:08,596][105620] Updated weights for policy 1, policy_version 1181259 (0.0011) [2023-12-26 23:58:08,651][105620] Updated weights for policy 1, policy_version 1181269 (0.0010) [2023-12-26 23:58:08,703][105620] Updated weights for policy 1, policy_version 1181279 (0.0010) [2023-12-26 23:58:09,174][105692] Updated weights for policy 0, policy_version 1180142 (0.0008) [2023-12-26 23:58:09,234][105692] Updated weights for policy 0, policy_version 1180152 (0.0008) [2023-12-26 23:58:09,293][105692] Updated weights for policy 0, policy_version 1180162 (0.0011) [2023-12-26 23:58:09,484][105620] Updated weights for policy 1, policy_version 1181289 (0.0010) [2023-12-26 23:58:09,542][105620] Updated weights for policy 1, policy_version 1181299 (0.0008) [2023-12-26 23:58:09,599][105620] Updated weights for policy 1, policy_version 1181309 (0.0007) [2023-12-26 23:58:09,657][105620] Updated weights for policy 1, policy_version 1181319 (0.0007) [2023-12-26 23:58:10,040][105692] Updated weights for policy 0, policy_version 1180172 (0.0009) [2023-12-26 23:58:10,102][105692] Updated weights for policy 0, policy_version 1180182 (0.0010) [2023-12-26 23:58:10,167][105692] Updated weights for policy 0, policy_version 1180192 (0.0005) [2023-12-26 23:58:10,428][105620] Updated weights for policy 1, policy_version 1181329 (0.0010) [2023-12-26 23:58:10,490][105620] Updated weights for policy 1, policy_version 1181339 (0.0011) [2023-12-26 23:58:10,558][105620] Updated weights for policy 1, policy_version 1181349 (0.0010) [2023-12-26 23:58:10,775][105692] Updated weights for policy 0, policy_version 1180202 (0.0010) [2023-12-26 23:58:10,834][105692] Updated weights for policy 0, policy_version 1180212 (0.0007) [2023-12-26 23:58:10,897][105692] Updated weights for policy 0, policy_version 1180222 (0.0007) [2023-12-26 23:58:10,956][105692] Updated weights for policy 0, policy_version 1180232 (0.0007) [2023-12-26 23:58:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 604651520. Throughput: 0: 9937.7, 1: 9492.4. Samples: 604658180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:58:11,062][104569] Avg episode reward: [(0, '9259.624'), (1, '9172.344')] [2023-12-26 23:58:11,310][105620] Updated weights for policy 1, policy_version 1181359 (0.0008) [2023-12-26 23:58:11,378][105620] Updated weights for policy 1, policy_version 1181369 (0.0008) [2023-12-26 23:58:11,435][105620] Updated weights for policy 1, policy_version 1181379 (0.0009) [2023-12-26 23:58:11,630][105692] Updated weights for policy 0, policy_version 1180242 (0.0009) [2023-12-26 23:58:11,708][105692] Updated weights for policy 0, policy_version 1180252 (0.0008) [2023-12-26 23:58:11,775][105692] Updated weights for policy 0, policy_version 1180262 (0.0008) [2023-12-26 23:58:12,176][105620] Updated weights for policy 1, policy_version 1181390 (0.0009) [2023-12-26 23:58:12,224][105620] Updated weights for policy 1, policy_version 1181400 (0.0010) [2023-12-26 23:58:12,276][105620] Updated weights for policy 1, policy_version 1181410 (0.0010) [2023-12-26 23:58:12,570][105692] Updated weights for policy 0, policy_version 1180272 (0.0009) [2023-12-26 23:58:12,627][105692] Updated weights for policy 0, policy_version 1180282 (0.0008) [2023-12-26 23:58:12,686][105692] Updated weights for policy 0, policy_version 1180292 (0.0008) [2023-12-26 23:58:13,040][105620] Updated weights for policy 1, policy_version 1181420 (0.0011) [2023-12-26 23:58:13,106][105620] Updated weights for policy 1, policy_version 1181430 (0.0010) [2023-12-26 23:58:13,167][105620] Updated weights for policy 1, policy_version 1181440 (0.0010) [2023-12-26 23:58:13,452][105692] Updated weights for policy 0, policy_version 1180302 (0.0008) [2023-12-26 23:58:13,501][105692] Updated weights for policy 0, policy_version 1180312 (0.0008) [2023-12-26 23:58:13,554][105692] Updated weights for policy 0, policy_version 1180322 (0.0008) [2023-12-26 23:58:13,894][105620] Updated weights for policy 1, policy_version 1181450 (0.0010) [2023-12-26 23:58:13,955][105620] Updated weights for policy 1, policy_version 1181460 (0.0010) [2023-12-26 23:58:14,016][105620] Updated weights for policy 1, policy_version 1181470 (0.0010) [2023-12-26 23:58:14,073][105620] Updated weights for policy 1, policy_version 1181480 (0.0010) [2023-12-26 23:58:14,338][105692] Updated weights for policy 0, policy_version 1180332 (0.0008) [2023-12-26 23:58:14,397][105692] Updated weights for policy 0, policy_version 1180342 (0.0008) [2023-12-26 23:58:14,455][105692] Updated weights for policy 0, policy_version 1180352 (0.0008) [2023-12-26 23:58:14,776][105620] Updated weights for policy 1, policy_version 1181490 (0.0009) [2023-12-26 23:58:14,835][105620] Updated weights for policy 1, policy_version 1181500 (0.0009) [2023-12-26 23:58:14,896][105620] Updated weights for policy 1, policy_version 1181510 (0.0009) [2023-12-26 23:58:15,161][105692] Updated weights for policy 0, policy_version 1180362 (0.0007) [2023-12-26 23:58:15,225][105692] Updated weights for policy 0, policy_version 1180372 (0.0006) [2023-12-26 23:58:15,290][105692] Updated weights for policy 0, policy_version 1180382 (0.0010) [2023-12-26 23:58:15,358][105692] Updated weights for policy 0, policy_version 1180392 (0.0010) [2023-12-26 23:58:15,584][105620] Updated weights for policy 1, policy_version 1181520 (0.0007) [2023-12-26 23:58:15,637][105620] Updated weights for policy 1, policy_version 1181530 (0.0006) [2023-12-26 23:58:15,695][105620] Updated weights for policy 1, policy_version 1181540 (0.0005) [2023-12-26 23:58:16,027][105692] Updated weights for policy 0, policy_version 1180402 (0.0005) [2023-12-26 23:58:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 604741632. Throughput: 0: 9889.9, 1: 9374.1. Samples: 604714164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:58:16,062][104569] Avg episode reward: [(0, '9167.960'), (1, '9172.159')] [2023-12-26 23:58:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001181544_302514176.pth... [2023-12-26 23:58:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001180456_302235648.pth [2023-12-26 23:58:16,076][105692] Updated weights for policy 0, policy_version 1180412 (0.0005) [2023-12-26 23:58:16,140][105692] Updated weights for policy 0, policy_version 1180422 (0.0006) [2023-12-26 23:58:16,151][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001180424_302235648.pth... [2023-12-26 23:58:16,156][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001179272_301940736.pth [2023-12-26 23:58:16,261][105620] Updated weights for policy 1, policy_version 1181550 (0.0006) [2023-12-26 23:58:16,327][105620] Updated weights for policy 1, policy_version 1181560 (0.0005) [2023-12-26 23:58:16,394][105620] Updated weights for policy 1, policy_version 1181570 (0.0005) [2023-12-26 23:58:16,842][105692] Updated weights for policy 0, policy_version 1180432 (0.0008) [2023-12-26 23:58:16,904][105692] Updated weights for policy 0, policy_version 1180442 (0.0009) [2023-12-26 23:58:16,963][105692] Updated weights for policy 0, policy_version 1180452 (0.0009) [2023-12-26 23:58:17,027][105620] Updated weights for policy 1, policy_version 1181580 (0.0009) [2023-12-26 23:58:17,084][105620] Updated weights for policy 1, policy_version 1181590 (0.0009) [2023-12-26 23:58:17,127][105620] Updated weights for policy 1, policy_version 1181600 (0.0005) [2023-12-26 23:58:17,686][105620] Updated weights for policy 1, policy_version 1181610 (0.0005) [2023-12-26 23:58:17,750][105620] Updated weights for policy 1, policy_version 1181620 (0.0006) [2023-12-26 23:58:17,758][105692] Updated weights for policy 0, policy_version 1180462 (0.0008) [2023-12-26 23:58:17,811][105692] Updated weights for policy 0, policy_version 1180472 (0.0009) [2023-12-26 23:58:17,816][105620] Updated weights for policy 1, policy_version 1181630 (0.0006) [2023-12-26 23:58:17,864][105692] Updated weights for policy 0, policy_version 1180482 (0.0009) [2023-12-26 23:58:17,875][105620] Updated weights for policy 1, policy_version 1181640 (0.0006) [2023-12-26 23:58:18,517][105620] Updated weights for policy 1, policy_version 1181650 (0.0008) [2023-12-26 23:58:18,583][105620] Updated weights for policy 1, policy_version 1181660 (0.0008) [2023-12-26 23:58:18,647][105620] Updated weights for policy 1, policy_version 1181670 (0.0009) [2023-12-26 23:58:18,682][105692] Updated weights for policy 0, policy_version 1180492 (0.0007) [2023-12-26 23:58:18,746][105692] Updated weights for policy 0, policy_version 1180502 (0.0008) [2023-12-26 23:58:18,808][105692] Updated weights for policy 0, policy_version 1180512 (0.0006) [2023-12-26 23:58:19,329][105620] Updated weights for policy 1, policy_version 1181680 (0.0010) [2023-12-26 23:58:19,387][105620] Updated weights for policy 1, policy_version 1181690 (0.0009) [2023-12-26 23:58:19,446][105620] Updated weights for policy 1, policy_version 1181700 (0.0006) [2023-12-26 23:58:19,621][105692] Updated weights for policy 0, policy_version 1180522 (0.0009) [2023-12-26 23:58:19,684][105692] Updated weights for policy 0, policy_version 1180532 (0.0011) [2023-12-26 23:58:19,746][105692] Updated weights for policy 0, policy_version 1180542 (0.0011) [2023-12-26 23:58:19,809][105692] Updated weights for policy 0, policy_version 1180552 (0.0011) [2023-12-26 23:58:20,208][105620] Updated weights for policy 1, policy_version 1181710 (0.0010) [2023-12-26 23:58:20,270][105620] Updated weights for policy 1, policy_version 1181720 (0.0011) [2023-12-26 23:58:20,333][105620] Updated weights for policy 1, policy_version 1181730 (0.0010) [2023-12-26 23:58:20,579][105692] Updated weights for policy 0, policy_version 1180562 (0.0007) [2023-12-26 23:58:20,648][105692] Updated weights for policy 0, policy_version 1180572 (0.0007) [2023-12-26 23:58:20,712][105692] Updated weights for policy 0, policy_version 1180582 (0.0006) [2023-12-26 23:58:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 604839936. Throughput: 0: 9800.8, 1: 9517.8. Samples: 604833348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:58:21,063][104569] Avg episode reward: [(0, '9350.478'), (1, '9083.344')] [2023-12-26 23:58:21,093][105620] Updated weights for policy 1, policy_version 1181740 (0.0011) [2023-12-26 23:58:21,153][105620] Updated weights for policy 1, policy_version 1181750 (0.0011) [2023-12-26 23:58:21,214][105620] Updated weights for policy 1, policy_version 1181760 (0.0008) [2023-12-26 23:58:21,440][105692] Updated weights for policy 0, policy_version 1180592 (0.0008) [2023-12-26 23:58:21,496][105692] Updated weights for policy 0, policy_version 1180602 (0.0008) [2023-12-26 23:58:21,553][105692] Updated weights for policy 0, policy_version 1180612 (0.0008) [2023-12-26 23:58:21,926][105620] Updated weights for policy 1, policy_version 1181770 (0.0010) [2023-12-26 23:58:21,985][105620] Updated weights for policy 1, policy_version 1181780 (0.0011) [2023-12-26 23:58:22,052][105620] Updated weights for policy 1, policy_version 1181790 (0.0010) [2023-12-26 23:58:22,111][105620] Updated weights for policy 1, policy_version 1181800 (0.0011) [2023-12-26 23:58:22,380][105692] Updated weights for policy 0, policy_version 1180622 (0.0009) [2023-12-26 23:58:22,441][105692] Updated weights for policy 0, policy_version 1180632 (0.0010) [2023-12-26 23:58:22,495][105692] Updated weights for policy 0, policy_version 1180642 (0.0011) [2023-12-26 23:58:22,756][105620] Updated weights for policy 1, policy_version 1181810 (0.0008) [2023-12-26 23:58:22,813][105620] Updated weights for policy 1, policy_version 1181820 (0.0008) [2023-12-26 23:58:22,880][105620] Updated weights for policy 1, policy_version 1181830 (0.0008) [2023-12-26 23:58:23,310][105692] Updated weights for policy 0, policy_version 1180652 (0.0010) [2023-12-26 23:58:23,370][105692] Updated weights for policy 0, policy_version 1180662 (0.0009) [2023-12-26 23:58:23,429][105692] Updated weights for policy 0, policy_version 1180672 (0.0009) [2023-12-26 23:58:23,600][105620] Updated weights for policy 1, policy_version 1181840 (0.0010) [2023-12-26 23:58:23,655][105620] Updated weights for policy 1, policy_version 1181850 (0.0010) [2023-12-26 23:58:23,723][105620] Updated weights for policy 1, policy_version 1181860 (0.0008) [2023-12-26 23:58:24,228][105692] Updated weights for policy 0, policy_version 1180682 (0.0008) [2023-12-26 23:58:24,286][105692] Updated weights for policy 0, policy_version 1180692 (0.0009) [2023-12-26 23:58:24,312][105620] Updated weights for policy 1, policy_version 1181870 (0.0006) [2023-12-26 23:58:24,349][105692] Updated weights for policy 0, policy_version 1180702 (0.0007) [2023-12-26 23:58:24,369][105620] Updated weights for policy 1, policy_version 1181880 (0.0007) [2023-12-26 23:58:24,414][105692] Updated weights for policy 0, policy_version 1180712 (0.0007) [2023-12-26 23:58:24,425][105620] Updated weights for policy 1, policy_version 1181890 (0.0006) [2023-12-26 23:58:25,003][105620] Updated weights for policy 1, policy_version 1181900 (0.0005) [2023-12-26 23:58:25,064][105620] Updated weights for policy 1, policy_version 1181910 (0.0005) [2023-12-26 23:58:25,125][105620] Updated weights for policy 1, policy_version 1181920 (0.0005) [2023-12-26 23:58:25,143][105692] Updated weights for policy 0, policy_version 1180722 (0.0009) [2023-12-26 23:58:25,203][105692] Updated weights for policy 0, policy_version 1180732 (0.0010) [2023-12-26 23:58:25,258][105692] Updated weights for policy 0, policy_version 1180742 (0.0012) [2023-12-26 23:58:25,702][105620] Updated weights for policy 1, policy_version 1181930 (0.0006) [2023-12-26 23:58:25,750][105620] Updated weights for policy 1, policy_version 1181940 (0.0009) [2023-12-26 23:58:25,804][105620] Updated weights for policy 1, policy_version 1181950 (0.0010) [2023-12-26 23:58:25,852][105620] Updated weights for policy 1, policy_version 1181960 (0.0010) [2023-12-26 23:58:25,982][105692] Updated weights for policy 0, policy_version 1180752 (0.0006) [2023-12-26 23:58:26,027][105692] Updated weights for policy 0, policy_version 1180762 (0.0005) [2023-12-26 23:58:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 604938240. Throughput: 0: 9663.8, 1: 9640.7. Samples: 604948436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:58:26,063][104569] Avg episode reward: [(0, '9264.159'), (1, '9266.011')] [2023-12-26 23:58:26,080][105692] Updated weights for policy 0, policy_version 1180772 (0.0006) [2023-12-26 23:58:26,518][105620] Updated weights for policy 1, policy_version 1181970 (0.0011) [2023-12-26 23:58:26,570][105620] Updated weights for policy 1, policy_version 1181980 (0.0010) [2023-12-26 23:58:26,612][105692] Updated weights for policy 0, policy_version 1180782 (0.0006) [2023-12-26 23:58:26,626][105620] Updated weights for policy 1, policy_version 1181990 (0.0010) [2023-12-26 23:58:26,674][105692] Updated weights for policy 0, policy_version 1180792 (0.0008) [2023-12-26 23:58:26,732][105692] Updated weights for policy 0, policy_version 1180802 (0.0010) [2023-12-26 23:58:27,375][105620] Updated weights for policy 1, policy_version 1182000 (0.0008) [2023-12-26 23:58:27,419][105620] Updated weights for policy 1, policy_version 1182010 (0.0008) [2023-12-26 23:58:27,436][105692] Updated weights for policy 0, policy_version 1180812 (0.0010) [2023-12-26 23:58:27,478][105620] Updated weights for policy 1, policy_version 1182020 (0.0006) [2023-12-26 23:58:27,484][105692] Updated weights for policy 0, policy_version 1180822 (0.0010) [2023-12-26 23:58:27,532][105692] Updated weights for policy 0, policy_version 1180832 (0.0010) [2023-12-26 23:58:28,161][105692] Updated weights for policy 0, policy_version 1180842 (0.0009) [2023-12-26 23:58:28,181][105620] Updated weights for policy 1, policy_version 1182030 (0.0008) [2023-12-26 23:58:28,210][105692] Updated weights for policy 0, policy_version 1180852 (0.0005) [2023-12-26 23:58:28,232][105620] Updated weights for policy 1, policy_version 1182040 (0.0010) [2023-12-26 23:58:28,258][105692] Updated weights for policy 0, policy_version 1180862 (0.0005) [2023-12-26 23:58:28,279][105620] Updated weights for policy 1, policy_version 1182050 (0.0010) [2023-12-26 23:58:28,306][105692] Updated weights for policy 0, policy_version 1180872 (0.0005) [2023-12-26 23:58:28,952][105620] Updated weights for policy 1, policy_version 1182060 (0.0010) [2023-12-26 23:58:28,995][105692] Updated weights for policy 0, policy_version 1180882 (0.0010) [2023-12-26 23:58:29,012][105620] Updated weights for policy 1, policy_version 1182070 (0.0008) [2023-12-26 23:58:29,046][105692] Updated weights for policy 0, policy_version 1180892 (0.0010) [2023-12-26 23:58:29,057][105620] Updated weights for policy 1, policy_version 1182080 (0.0005) [2023-12-26 23:58:29,097][105692] Updated weights for policy 0, policy_version 1180902 (0.0010) [2023-12-26 23:58:29,823][105620] Updated weights for policy 1, policy_version 1182090 (0.0005) [2023-12-26 23:58:29,825][105692] Updated weights for policy 0, policy_version 1180912 (0.0008) [2023-12-26 23:58:29,884][105620] Updated weights for policy 1, policy_version 1182100 (0.0006) [2023-12-26 23:58:29,889][105692] Updated weights for policy 0, policy_version 1180922 (0.0010) [2023-12-26 23:58:29,950][105620] Updated weights for policy 1, policy_version 1182110 (0.0006) [2023-12-26 23:58:29,956][105692] Updated weights for policy 0, policy_version 1180932 (0.0008) [2023-12-26 23:58:30,008][105620] Updated weights for policy 1, policy_version 1182120 (0.0005) [2023-12-26 23:58:30,553][105692] Updated weights for policy 0, policy_version 1180942 (0.0008) [2023-12-26 23:58:30,602][105692] Updated weights for policy 0, policy_version 1180952 (0.0009) [2023-12-26 23:58:30,647][105692] Updated weights for policy 0, policy_version 1180962 (0.0008) [2023-12-26 23:58:30,665][105620] Updated weights for policy 1, policy_version 1182130 (0.0007) [2023-12-26 23:58:30,722][105620] Updated weights for policy 1, policy_version 1182140 (0.0008) [2023-12-26 23:58:30,785][105620] Updated weights for policy 1, policy_version 1182150 (0.0010) [2023-12-26 23:58:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 605044736. Throughput: 0: 9726.8, 1: 9695.5. Samples: 605012388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:58:31,062][104569] Avg episode reward: [(0, '9174.449'), (1, '9354.753')] [2023-12-26 23:58:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001180968_302374912.pth... [2023-12-26 23:58:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001182152_302669824.pth... [2023-12-26 23:58:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001179848_302088192.pth [2023-12-26 23:58:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001181000_302374912.pth [2023-12-26 23:58:31,350][105692] Updated weights for policy 0, policy_version 1180972 (0.0011) [2023-12-26 23:58:31,414][105692] Updated weights for policy 0, policy_version 1180982 (0.0011) [2023-12-26 23:58:31,484][105692] Updated weights for policy 0, policy_version 1180992 (0.0011) [2023-12-26 23:58:31,511][105620] Updated weights for policy 1, policy_version 1182160 (0.0006) [2023-12-26 23:58:31,571][105620] Updated weights for policy 1, policy_version 1182170 (0.0006) [2023-12-26 23:58:31,635][105620] Updated weights for policy 1, policy_version 1182180 (0.0008) [2023-12-26 23:58:32,106][105692] Updated weights for policy 0, policy_version 1181002 (0.0010) [2023-12-26 23:58:32,163][105692] Updated weights for policy 0, policy_version 1181012 (0.0005) [2023-12-26 23:58:32,225][105692] Updated weights for policy 0, policy_version 1181022 (0.0005) [2023-12-26 23:58:32,291][105692] Updated weights for policy 0, policy_version 1181032 (0.0009) [2023-12-26 23:58:32,476][105620] Updated weights for policy 1, policy_version 1182190 (0.0008) [2023-12-26 23:58:32,538][105620] Updated weights for policy 1, policy_version 1182200 (0.0008) [2023-12-26 23:58:32,602][105620] Updated weights for policy 1, policy_version 1182210 (0.0008) [2023-12-26 23:58:33,003][105692] Updated weights for policy 0, policy_version 1181042 (0.0010) [2023-12-26 23:58:33,054][105692] Updated weights for policy 0, policy_version 1181052 (0.0010) [2023-12-26 23:58:33,102][105692] Updated weights for policy 0, policy_version 1181062 (0.0010) [2023-12-26 23:58:33,253][105620] Updated weights for policy 1, policy_version 1182220 (0.0010) [2023-12-26 23:58:33,311][105620] Updated weights for policy 1, policy_version 1182230 (0.0008) [2023-12-26 23:58:33,363][105620] Updated weights for policy 1, policy_version 1182240 (0.0006) [2023-12-26 23:58:33,758][105692] Updated weights for policy 0, policy_version 1181072 (0.0006) [2023-12-26 23:58:33,811][105692] Updated weights for policy 0, policy_version 1181082 (0.0005) [2023-12-26 23:58:33,866][105692] Updated weights for policy 0, policy_version 1181092 (0.0007) [2023-12-26 23:58:34,053][105620] Updated weights for policy 1, policy_version 1182250 (0.0006) [2023-12-26 23:58:34,105][105620] Updated weights for policy 1, policy_version 1182260 (0.0005) [2023-12-26 23:58:34,162][105620] Updated weights for policy 1, policy_version 1182270 (0.0007) [2023-12-26 23:58:34,223][105620] Updated weights for policy 1, policy_version 1182280 (0.0009) [2023-12-26 23:58:34,614][105692] Updated weights for policy 0, policy_version 1181102 (0.0009) [2023-12-26 23:58:34,675][105692] Updated weights for policy 0, policy_version 1181112 (0.0008) [2023-12-26 23:58:34,734][105692] Updated weights for policy 0, policy_version 1181122 (0.0008) [2023-12-26 23:58:34,979][105620] Updated weights for policy 1, policy_version 1182290 (0.0007) [2023-12-26 23:58:35,047][105620] Updated weights for policy 1, policy_version 1182300 (0.0007) [2023-12-26 23:58:35,109][105620] Updated weights for policy 1, policy_version 1182310 (0.0010) [2023-12-26 23:58:35,395][105692] Updated weights for policy 0, policy_version 1181132 (0.0006) [2023-12-26 23:58:35,447][105692] Updated weights for policy 0, policy_version 1181142 (0.0006) [2023-12-26 23:58:35,495][105692] Updated weights for policy 0, policy_version 1181152 (0.0008) [2023-12-26 23:58:35,507][105585] KL-divergence is very high: 101.1642 [2023-12-26 23:58:35,520][105585] KL-divergence is very high: 148.7437 [2023-12-26 23:58:35,756][105620] Updated weights for policy 1, policy_version 1182320 (0.0008) [2023-12-26 23:58:35,819][105620] Updated weights for policy 1, policy_version 1182330 (0.0009) [2023-12-26 23:58:35,872][105620] Updated weights for policy 1, policy_version 1182340 (0.0005) [2023-12-26 23:58:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 605143040. Throughput: 0: 9724.0, 1: 9743.4. Samples: 605131196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:58:36,063][104569] Avg episode reward: [(0, '9174.544'), (1, '9171.902')] [2023-12-26 23:58:36,193][105692] Updated weights for policy 0, policy_version 1181162 (0.0008) [2023-12-26 23:58:36,256][105692] Updated weights for policy 0, policy_version 1181172 (0.0006) [2023-12-26 23:58:36,317][105692] Updated weights for policy 0, policy_version 1181182 (0.0007) [2023-12-26 23:58:36,378][105692] Updated weights for policy 0, policy_version 1181192 (0.0006) [2023-12-26 23:58:36,482][105620] Updated weights for policy 1, policy_version 1182350 (0.0007) [2023-12-26 23:58:36,549][105620] Updated weights for policy 1, policy_version 1182360 (0.0011) [2023-12-26 23:58:36,615][105620] Updated weights for policy 1, policy_version 1182370 (0.0011) [2023-12-26 23:58:37,051][105692] Updated weights for policy 0, policy_version 1181202 (0.0010) [2023-12-26 23:58:37,107][105692] Updated weights for policy 0, policy_version 1181212 (0.0009) [2023-12-26 23:58:37,168][105692] Updated weights for policy 0, policy_version 1181222 (0.0009) [2023-12-26 23:58:37,171][105620] Updated weights for policy 1, policy_version 1182380 (0.0008) [2023-12-26 23:58:37,221][105620] Updated weights for policy 1, policy_version 1182390 (0.0008) [2023-12-26 23:58:37,267][105620] Updated weights for policy 1, policy_version 1182400 (0.0008) [2023-12-26 23:58:37,882][105620] Updated weights for policy 1, policy_version 1182410 (0.0009) [2023-12-26 23:58:37,929][105620] Updated weights for policy 1, policy_version 1182420 (0.0008) [2023-12-26 23:58:37,986][105620] Updated weights for policy 1, policy_version 1182430 (0.0006) [2023-12-26 23:58:38,029][105692] Updated weights for policy 0, policy_version 1181232 (0.0009) [2023-12-26 23:58:38,043][105620] Updated weights for policy 1, policy_version 1182440 (0.0006) [2023-12-26 23:58:38,084][105692] Updated weights for policy 0, policy_version 1181242 (0.0008) [2023-12-26 23:58:38,131][105692] Updated weights for policy 0, policy_version 1181252 (0.0008) [2023-12-26 23:58:38,727][105620] Updated weights for policy 1, policy_version 1182450 (0.0008) [2023-12-26 23:58:38,781][105620] Updated weights for policy 1, policy_version 1182460 (0.0008) [2023-12-26 23:58:38,841][105620] Updated weights for policy 1, policy_version 1182470 (0.0008) [2023-12-26 23:58:38,928][105692] Updated weights for policy 0, policy_version 1181262 (0.0009) [2023-12-26 23:58:38,992][105692] Updated weights for policy 0, policy_version 1181272 (0.0009) [2023-12-26 23:58:39,058][105692] Updated weights for policy 0, policy_version 1181282 (0.0009) [2023-12-26 23:58:39,642][105620] Updated weights for policy 1, policy_version 1182480 (0.0006) [2023-12-26 23:58:39,711][105620] Updated weights for policy 1, policy_version 1182490 (0.0008) [2023-12-26 23:58:39,734][105692] Updated weights for policy 0, policy_version 1181292 (0.0008) [2023-12-26 23:58:39,773][105620] Updated weights for policy 1, policy_version 1182500 (0.0008) [2023-12-26 23:58:39,800][105692] Updated weights for policy 0, policy_version 1181302 (0.0008) [2023-12-26 23:58:39,867][105692] Updated weights for policy 0, policy_version 1181312 (0.0008) [2023-12-26 23:58:40,461][105620] Updated weights for policy 1, policy_version 1182510 (0.0008) [2023-12-26 23:58:40,532][105620] Updated weights for policy 1, policy_version 1182520 (0.0009) [2023-12-26 23:58:40,584][105692] Updated weights for policy 0, policy_version 1181322 (0.0006) [2023-12-26 23:58:40,588][105620] Updated weights for policy 1, policy_version 1182530 (0.0010) [2023-12-26 23:58:40,643][105692] Updated weights for policy 0, policy_version 1181332 (0.0010) [2023-12-26 23:58:40,711][105692] Updated weights for policy 0, policy_version 1181342 (0.0010) [2023-12-26 23:58:40,780][105692] Updated weights for policy 0, policy_version 1181352 (0.0009) [2023-12-26 23:58:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 605241344. Throughput: 0: 9630.7, 1: 9904.3. Samples: 605250536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:58:41,062][104569] Avg episode reward: [(0, '9264.841'), (1, '9172.213')] [2023-12-26 23:58:41,174][105620] Updated weights for policy 1, policy_version 1182540 (0.0007) [2023-12-26 23:58:41,245][105620] Updated weights for policy 1, policy_version 1182550 (0.0011) [2023-12-26 23:58:41,311][105620] Updated weights for policy 1, policy_version 1182560 (0.0011) [2023-12-26 23:58:41,489][105692] Updated weights for policy 0, policy_version 1181362 (0.0008) [2023-12-26 23:58:41,554][105692] Updated weights for policy 0, policy_version 1181372 (0.0008) [2023-12-26 23:58:41,617][105692] Updated weights for policy 0, policy_version 1181382 (0.0008) [2023-12-26 23:58:42,050][105620] Updated weights for policy 1, policy_version 1182570 (0.0010) [2023-12-26 23:58:42,109][105620] Updated weights for policy 1, policy_version 1182580 (0.0009) [2023-12-26 23:58:42,171][105620] Updated weights for policy 1, policy_version 1182590 (0.0009) [2023-12-26 23:58:42,233][105620] Updated weights for policy 1, policy_version 1182600 (0.0008) [2023-12-26 23:58:42,337][105692] Updated weights for policy 0, policy_version 1181392 (0.0009) [2023-12-26 23:58:42,402][105692] Updated weights for policy 0, policy_version 1181402 (0.0009) [2023-12-26 23:58:42,450][105692] Updated weights for policy 0, policy_version 1181412 (0.0009) [2023-12-26 23:58:42,966][105620] Updated weights for policy 1, policy_version 1182610 (0.0009) [2023-12-26 23:58:43,014][105620] Updated weights for policy 1, policy_version 1182621 (0.0008) [2023-12-26 23:58:43,065][105620] Updated weights for policy 1, policy_version 1182631 (0.0006) [2023-12-26 23:58:43,250][105692] Updated weights for policy 0, policy_version 1181422 (0.0009) [2023-12-26 23:58:43,316][105692] Updated weights for policy 0, policy_version 1181432 (0.0009) [2023-12-26 23:58:43,364][105692] Updated weights for policy 0, policy_version 1181442 (0.0009) [2023-12-26 23:58:43,737][105620] Updated weights for policy 1, policy_version 1182641 (0.0006) [2023-12-26 23:58:43,792][105620] Updated weights for policy 1, policy_version 1182651 (0.0005) [2023-12-26 23:58:43,849][105620] Updated weights for policy 1, policy_version 1182661 (0.0005) [2023-12-26 23:58:44,211][105692] Updated weights for policy 0, policy_version 1181452 (0.0009) [2023-12-26 23:58:44,268][105692] Updated weights for policy 0, policy_version 1181462 (0.0009) [2023-12-26 23:58:44,330][105692] Updated weights for policy 0, policy_version 1181472 (0.0009) [2023-12-26 23:58:44,506][105620] Updated weights for policy 1, policy_version 1182671 (0.0008) [2023-12-26 23:58:44,568][105620] Updated weights for policy 1, policy_version 1182681 (0.0009) [2023-12-26 23:58:44,614][105620] Updated weights for policy 1, policy_version 1182691 (0.0009) [2023-12-26 23:58:45,119][105692] Updated weights for policy 0, policy_version 1181482 (0.0009) [2023-12-26 23:58:45,181][105692] Updated weights for policy 0, policy_version 1181492 (0.0009) [2023-12-26 23:58:45,241][105692] Updated weights for policy 0, policy_version 1181502 (0.0010) [2023-12-26 23:58:45,310][105692] Updated weights for policy 0, policy_version 1181512 (0.0009) [2023-12-26 23:58:45,347][105620] Updated weights for policy 1, policy_version 1182701 (0.0007) [2023-12-26 23:58:45,400][105620] Updated weights for policy 1, policy_version 1182711 (0.0005) [2023-12-26 23:58:45,460][105620] Updated weights for policy 1, policy_version 1182721 (0.0006) [2023-12-26 23:58:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 605331456. Throughput: 0: 9642.3, 1: 9916.9. Samples: 605307920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:58:46,063][104569] Avg episode reward: [(0, '9352.957'), (1, '9263.877')] [2023-12-26 23:58:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001182728_302817280.pth... [2023-12-26 23:58:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001181544_302514176.pth [2023-12-26 23:58:46,083][105692] Updated weights for policy 0, policy_version 1181522 (0.0005) [2023-12-26 23:58:46,138][105692] Updated weights for policy 0, policy_version 1181532 (0.0006) [2023-12-26 23:58:46,165][105620] Updated weights for policy 1, policy_version 1182731 (0.0008) [2023-12-26 23:58:46,188][105692] Updated weights for policy 0, policy_version 1181542 (0.0006) [2023-12-26 23:58:46,195][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001181544_302522368.pth... [2023-12-26 23:58:46,198][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001180424_302235648.pth [2023-12-26 23:58:46,225][105620] Updated weights for policy 1, policy_version 1182741 (0.0009) [2023-12-26 23:58:46,284][105620] Updated weights for policy 1, policy_version 1182751 (0.0009) [2023-12-26 23:58:46,912][105692] Updated weights for policy 0, policy_version 1181552 (0.0008) [2023-12-26 23:58:46,937][105620] Updated weights for policy 1, policy_version 1182761 (0.0009) [2023-12-26 23:58:46,966][105692] Updated weights for policy 0, policy_version 1181562 (0.0008) [2023-12-26 23:58:46,989][105620] Updated weights for policy 1, policy_version 1182771 (0.0006) [2023-12-26 23:58:47,018][105692] Updated weights for policy 0, policy_version 1181572 (0.0008) [2023-12-26 23:58:47,052][105620] Updated weights for policy 1, policy_version 1182781 (0.0006) [2023-12-26 23:58:47,115][105620] Updated weights for policy 1, policy_version 1182791 (0.0009) [2023-12-26 23:58:47,745][105692] Updated weights for policy 0, policy_version 1181582 (0.0006) [2023-12-26 23:58:47,803][105692] Updated weights for policy 0, policy_version 1181592 (0.0005) [2023-12-26 23:58:47,863][105692] Updated weights for policy 0, policy_version 1181602 (0.0005) [2023-12-26 23:58:47,928][105620] Updated weights for policy 1, policy_version 1182801 (0.0009) [2023-12-26 23:58:47,989][105620] Updated weights for policy 1, policy_version 1182812 (0.0014) [2023-12-26 23:58:48,047][105620] Updated weights for policy 1, policy_version 1182822 (0.0010) [2023-12-26 23:58:48,390][105692] Updated weights for policy 0, policy_version 1181612 (0.0006) [2023-12-26 23:58:48,457][105692] Updated weights for policy 0, policy_version 1181622 (0.0006) [2023-12-26 23:58:48,525][105692] Updated weights for policy 0, policy_version 1181632 (0.0006) [2023-12-26 23:58:48,885][105620] Updated weights for policy 1, policy_version 1182832 (0.0007) [2023-12-26 23:58:48,936][105620] Updated weights for policy 1, policy_version 1182842 (0.0005) [2023-12-26 23:58:48,980][105620] Updated weights for policy 1, policy_version 1182852 (0.0005) [2023-12-26 23:58:49,119][105692] Updated weights for policy 0, policy_version 1181642 (0.0006) [2023-12-26 23:58:49,178][105692] Updated weights for policy 0, policy_version 1181652 (0.0005) [2023-12-26 23:58:49,233][105692] Updated weights for policy 0, policy_version 1181662 (0.0007) [2023-12-26 23:58:49,303][105692] Updated weights for policy 0, policy_version 1181672 (0.0007) [2023-12-26 23:58:49,728][105620] Updated weights for policy 1, policy_version 1182862 (0.0007) [2023-12-26 23:58:49,793][105620] Updated weights for policy 1, policy_version 1182872 (0.0005) [2023-12-26 23:58:49,860][105620] Updated weights for policy 1, policy_version 1182882 (0.0007) [2023-12-26 23:58:49,880][105692] Updated weights for policy 0, policy_version 1181682 (0.0007) [2023-12-26 23:58:49,938][105692] Updated weights for policy 0, policy_version 1181692 (0.0007) [2023-12-26 23:58:49,998][105692] Updated weights for policy 0, policy_version 1181702 (0.0006) [2023-12-26 23:58:50,673][105620] Updated weights for policy 1, policy_version 1182892 (0.0007) [2023-12-26 23:58:50,686][105692] Updated weights for policy 0, policy_version 1181712 (0.0008) [2023-12-26 23:58:50,730][105620] Updated weights for policy 1, policy_version 1182902 (0.0010) [2023-12-26 23:58:50,748][105692] Updated weights for policy 0, policy_version 1181722 (0.0007) [2023-12-26 23:58:50,799][105620] Updated weights for policy 1, policy_version 1182912 (0.0007) [2023-12-26 23:58:50,809][105692] Updated weights for policy 0, policy_version 1181732 (0.0007) [2023-12-26 23:58:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 605437952. Throughput: 0: 9610.3, 1: 9940.4. Samples: 605424204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:58:51,062][104569] Avg episode reward: [(0, '9352.508'), (1, '9080.838')] [2023-12-26 23:58:51,590][105620] Updated weights for policy 1, policy_version 1182922 (0.0007) [2023-12-26 23:58:51,592][105692] Updated weights for policy 0, policy_version 1181742 (0.0008) [2023-12-26 23:58:51,647][105620] Updated weights for policy 1, policy_version 1182932 (0.0007) [2023-12-26 23:58:51,657][105692] Updated weights for policy 0, policy_version 1181752 (0.0008) [2023-12-26 23:58:51,697][105620] Updated weights for policy 1, policy_version 1182942 (0.0008) [2023-12-26 23:58:51,720][105692] Updated weights for policy 0, policy_version 1181762 (0.0007) [2023-12-26 23:58:51,764][105620] Updated weights for policy 1, policy_version 1182952 (0.0007) [2023-12-26 23:58:52,310][105692] Updated weights for policy 0, policy_version 1181772 (0.0009) [2023-12-26 23:58:52,374][105692] Updated weights for policy 0, policy_version 1181782 (0.0009) [2023-12-26 23:58:52,435][105692] Updated weights for policy 0, policy_version 1181792 (0.0010) [2023-12-26 23:58:52,646][105620] Updated weights for policy 1, policy_version 1182962 (0.0008) [2023-12-26 23:58:52,709][105620] Updated weights for policy 1, policy_version 1182972 (0.0008) [2023-12-26 23:58:52,769][105620] Updated weights for policy 1, policy_version 1182982 (0.0008) [2023-12-26 23:58:53,129][105692] Updated weights for policy 0, policy_version 1181802 (0.0010) [2023-12-26 23:58:53,183][105692] Updated weights for policy 0, policy_version 1181812 (0.0005) [2023-12-26 23:58:53,251][105692] Updated weights for policy 0, policy_version 1181822 (0.0006) [2023-12-26 23:58:53,312][105692] Updated weights for policy 0, policy_version 1181832 (0.0005) [2023-12-26 23:58:53,638][105620] Updated weights for policy 1, policy_version 1182992 (0.0009) [2023-12-26 23:58:53,692][105620] Updated weights for policy 1, policy_version 1183003 (0.0010) [2023-12-26 23:58:53,745][105620] Updated weights for policy 1, policy_version 1183013 (0.0009) [2023-12-26 23:58:53,817][105692] Updated weights for policy 0, policy_version 1181842 (0.0005) [2023-12-26 23:58:53,880][105692] Updated weights for policy 0, policy_version 1181852 (0.0006) [2023-12-26 23:58:53,929][105692] Updated weights for policy 0, policy_version 1181862 (0.0005) [2023-12-26 23:58:54,556][105692] Updated weights for policy 0, policy_version 1181872 (0.0008) [2023-12-26 23:58:54,601][105620] Updated weights for policy 1, policy_version 1183023 (0.0008) [2023-12-26 23:58:54,616][105692] Updated weights for policy 0, policy_version 1181882 (0.0007) [2023-12-26 23:58:54,664][105620] Updated weights for policy 1, policy_version 1183033 (0.0009) [2023-12-26 23:58:54,665][105692] Updated weights for policy 0, policy_version 1181892 (0.0008) [2023-12-26 23:58:54,725][105620] Updated weights for policy 1, policy_version 1183043 (0.0008) [2023-12-26 23:58:55,345][105692] Updated weights for policy 0, policy_version 1181902 (0.0008) [2023-12-26 23:58:55,399][105692] Updated weights for policy 0, policy_version 1181912 (0.0009) [2023-12-26 23:58:55,453][105620] Updated weights for policy 1, policy_version 1183053 (0.0008) [2023-12-26 23:58:55,459][105692] Updated weights for policy 0, policy_version 1181922 (0.0008) [2023-12-26 23:58:55,507][105620] Updated weights for policy 1, policy_version 1183063 (0.0008) [2023-12-26 23:58:55,565][105620] Updated weights for policy 1, policy_version 1183073 (0.0009) [2023-12-26 23:58:56,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 605528064. Throughput: 0: 9758.3, 1: 9830.1. Samples: 605539664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:58:56,063][104569] Avg episode reward: [(0, '9261.465'), (1, '9080.133')] [2023-12-26 23:58:56,168][105620] Updated weights for policy 1, policy_version 1183083 (0.0008) [2023-12-26 23:58:56,178][105692] Updated weights for policy 0, policy_version 1181932 (0.0007) [2023-12-26 23:58:56,216][105620] Updated weights for policy 1, policy_version 1183093 (0.0006) [2023-12-26 23:58:56,231][105692] Updated weights for policy 0, policy_version 1181942 (0.0007) [2023-12-26 23:58:56,268][105620] Updated weights for policy 1, policy_version 1183103 (0.0009) [2023-12-26 23:58:56,275][105692] Updated weights for policy 0, policy_version 1181952 (0.0006) [2023-12-26 23:58:56,982][105692] Updated weights for policy 0, policy_version 1181962 (0.0006) [2023-12-26 23:58:57,010][105620] Updated weights for policy 1, policy_version 1183113 (0.0007) [2023-12-26 23:58:57,036][105692] Updated weights for policy 0, policy_version 1181972 (0.0007) [2023-12-26 23:58:57,067][105620] Updated weights for policy 1, policy_version 1183123 (0.0008) [2023-12-26 23:58:57,083][105692] Updated weights for policy 0, policy_version 1181982 (0.0008) [2023-12-26 23:58:57,114][105620] Updated weights for policy 1, policy_version 1183133 (0.0007) [2023-12-26 23:58:57,133][105692] Updated weights for policy 0, policy_version 1181992 (0.0007) [2023-12-26 23:58:57,158][105620] Updated weights for policy 1, policy_version 1183143 (0.0006) [2023-12-26 23:58:57,786][105620] Updated weights for policy 1, policy_version 1183153 (0.0007) [2023-12-26 23:58:57,834][105620] Updated weights for policy 1, policy_version 1183163 (0.0005) [2023-12-26 23:58:57,879][105620] Updated weights for policy 1, policy_version 1183173 (0.0005) [2023-12-26 23:58:57,947][105692] Updated weights for policy 0, policy_version 1182002 (0.0009) [2023-12-26 23:58:58,013][105692] Updated weights for policy 0, policy_version 1182012 (0.0008) [2023-12-26 23:58:58,074][105692] Updated weights for policy 0, policy_version 1182022 (0.0005) [2023-12-26 23:58:58,586][105620] Updated weights for policy 1, policy_version 1183183 (0.0007) [2023-12-26 23:58:58,656][105620] Updated weights for policy 1, policy_version 1183193 (0.0009) [2023-12-26 23:58:58,723][105620] Updated weights for policy 1, policy_version 1183203 (0.0008) [2023-12-26 23:58:58,881][105692] Updated weights for policy 0, policy_version 1182032 (0.0007) [2023-12-26 23:58:58,941][105692] Updated weights for policy 0, policy_version 1182042 (0.0008) [2023-12-26 23:58:58,996][105692] Updated weights for policy 0, policy_version 1182052 (0.0008) [2023-12-26 23:58:59,565][105620] Updated weights for policy 1, policy_version 1183213 (0.0008) [2023-12-26 23:58:59,620][105620] Updated weights for policy 1, policy_version 1183223 (0.0009) [2023-12-26 23:58:59,676][105620] Updated weights for policy 1, policy_version 1183233 (0.0005) [2023-12-26 23:58:59,741][105692] Updated weights for policy 0, policy_version 1182062 (0.0008) [2023-12-26 23:58:59,801][105692] Updated weights for policy 0, policy_version 1182072 (0.0009) [2023-12-26 23:58:59,865][105692] Updated weights for policy 0, policy_version 1182082 (0.0008) [2023-12-26 23:59:00,449][105620] Updated weights for policy 1, policy_version 1183243 (0.0006) [2023-12-26 23:59:00,506][105620] Updated weights for policy 1, policy_version 1183253 (0.0009) [2023-12-26 23:59:00,534][105692] Updated weights for policy 0, policy_version 1182092 (0.0008) [2023-12-26 23:59:00,548][105620] Updated weights for policy 1, policy_version 1183263 (0.0006) [2023-12-26 23:59:00,589][105692] Updated weights for policy 0, policy_version 1182102 (0.0006) [2023-12-26 23:59:00,649][105692] Updated weights for policy 0, policy_version 1182112 (0.0005) [2023-12-26 23:59:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 605626368. Throughput: 0: 9768.5, 1: 9883.9. Samples: 605598524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:59:01,062][104569] Avg episode reward: [(0, '9261.426'), (1, '9171.885')] [2023-12-26 23:59:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001182120_302669824.pth... [2023-12-26 23:59:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001183272_302956544.pth... [2023-12-26 23:59:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001180968_302374912.pth [2023-12-26 23:59:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001182152_302669824.pth [2023-12-26 23:59:01,346][105692] Updated weights for policy 0, policy_version 1182122 (0.0006) [2023-12-26 23:59:01,364][105620] Updated weights for policy 1, policy_version 1183273 (0.0009) [2023-12-26 23:59:01,411][105692] Updated weights for policy 0, policy_version 1182132 (0.0007) [2023-12-26 23:59:01,425][105620] Updated weights for policy 1, policy_version 1183283 (0.0008) [2023-12-26 23:59:01,478][105692] Updated weights for policy 0, policy_version 1182142 (0.0006) [2023-12-26 23:59:01,479][105620] Updated weights for policy 1, policy_version 1183293 (0.0009) [2023-12-26 23:59:01,534][105692] Updated weights for policy 0, policy_version 1182152 (0.0006) [2023-12-26 23:59:01,535][105620] Updated weights for policy 1, policy_version 1183303 (0.0007) [2023-12-26 23:59:02,239][105692] Updated weights for policy 0, policy_version 1182162 (0.0006) [2023-12-26 23:59:02,294][105620] Updated weights for policy 1, policy_version 1183313 (0.0008) [2023-12-26 23:59:02,306][105692] Updated weights for policy 0, policy_version 1182172 (0.0007) [2023-12-26 23:59:02,355][105620] Updated weights for policy 1, policy_version 1183323 (0.0009) [2023-12-26 23:59:02,364][105692] Updated weights for policy 0, policy_version 1182182 (0.0007) [2023-12-26 23:59:02,420][105620] Updated weights for policy 1, policy_version 1183333 (0.0007) [2023-12-26 23:59:03,017][105620] Updated weights for policy 1, policy_version 1183343 (0.0007) [2023-12-26 23:59:03,063][105620] Updated weights for policy 1, policy_version 1183353 (0.0005) [2023-12-26 23:59:03,112][105620] Updated weights for policy 1, policy_version 1183363 (0.0008) [2023-12-26 23:59:03,151][105692] Updated weights for policy 0, policy_version 1182192 (0.0006) [2023-12-26 23:59:03,199][105692] Updated weights for policy 0, policy_version 1182202 (0.0008) [2023-12-26 23:59:03,256][105692] Updated weights for policy 0, policy_version 1182213 (0.0009) [2023-12-26 23:59:03,703][105620] Updated weights for policy 1, policy_version 1183373 (0.0008) [2023-12-26 23:59:03,748][105620] Updated weights for policy 1, policy_version 1183383 (0.0006) [2023-12-26 23:59:03,798][105620] Updated weights for policy 1, policy_version 1183393 (0.0005) [2023-12-26 23:59:04,044][105692] Updated weights for policy 0, policy_version 1182223 (0.0006) [2023-12-26 23:59:04,095][105692] Updated weights for policy 0, policy_version 1182233 (0.0005) [2023-12-26 23:59:04,155][105692] Updated weights for policy 0, policy_version 1182243 (0.0009) [2023-12-26 23:59:04,470][105620] Updated weights for policy 1, policy_version 1183403 (0.0007) [2023-12-26 23:59:04,530][105620] Updated weights for policy 1, policy_version 1183413 (0.0009) [2023-12-26 23:59:04,593][105620] Updated weights for policy 1, policy_version 1183423 (0.0009) [2023-12-26 23:59:04,917][105692] Updated weights for policy 0, policy_version 1182253 (0.0009) [2023-12-26 23:59:04,963][105692] Updated weights for policy 0, policy_version 1182263 (0.0008) [2023-12-26 23:59:05,018][105692] Updated weights for policy 0, policy_version 1182273 (0.0008) [2023-12-26 23:59:05,284][105620] Updated weights for policy 1, policy_version 1183433 (0.0009) [2023-12-26 23:59:05,333][105620] Updated weights for policy 1, policy_version 1183443 (0.0009) [2023-12-26 23:59:05,381][105620] Updated weights for policy 1, policy_version 1183453 (0.0010) [2023-12-26 23:59:05,432][105620] Updated weights for policy 1, policy_version 1183463 (0.0010) [2023-12-26 23:59:05,881][105692] Updated weights for policy 0, policy_version 1182283 (0.0008) [2023-12-26 23:59:05,951][105692] Updated weights for policy 0, policy_version 1182293 (0.0009) [2023-12-26 23:59:05,996][105620] Updated weights for policy 1, policy_version 1183473 (0.0007) [2023-12-26 23:59:06,016][105692] Updated weights for policy 0, policy_version 1182303 (0.0007) [2023-12-26 23:59:06,062][105620] Updated weights for policy 1, policy_version 1183483 (0.0007) [2023-12-26 23:59:06,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 605716480. Throughput: 0: 9773.7, 1: 9793.2. Samples: 605713856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:59:06,062][104569] Avg episode reward: [(0, '9261.804'), (1, '9170.771')] [2023-12-26 23:59:06,128][105620] Updated weights for policy 1, policy_version 1183493 (0.0007) [2023-12-26 23:59:06,716][105620] Updated weights for policy 1, policy_version 1183503 (0.0007) [2023-12-26 23:59:06,772][105620] Updated weights for policy 1, policy_version 1183513 (0.0011) [2023-12-26 23:59:06,807][105692] Updated weights for policy 0, policy_version 1182313 (0.0006) [2023-12-26 23:59:06,821][105620] Updated weights for policy 1, policy_version 1183523 (0.0010) [2023-12-26 23:59:06,860][105692] Updated weights for policy 0, policy_version 1182323 (0.0007) [2023-12-26 23:59:06,912][105692] Updated weights for policy 0, policy_version 1182333 (0.0008) [2023-12-26 23:59:06,971][105692] Updated weights for policy 0, policy_version 1182343 (0.0008) [2023-12-26 23:59:07,513][105620] Updated weights for policy 1, policy_version 1183533 (0.0008) [2023-12-26 23:59:07,561][105692] Updated weights for policy 0, policy_version 1182353 (0.0010) [2023-12-26 23:59:07,575][105620] Updated weights for policy 1, policy_version 1183543 (0.0008) [2023-12-26 23:59:07,619][105692] Updated weights for policy 0, policy_version 1182363 (0.0010) [2023-12-26 23:59:07,636][105620] Updated weights for policy 1, policy_version 1183553 (0.0010) [2023-12-26 23:59:07,668][105692] Updated weights for policy 0, policy_version 1182373 (0.0010) [2023-12-26 23:59:08,252][105692] Updated weights for policy 0, policy_version 1182383 (0.0006) [2023-12-26 23:59:08,306][105692] Updated weights for policy 0, policy_version 1182393 (0.0005) [2023-12-26 23:59:08,369][105692] Updated weights for policy 0, policy_version 1182403 (0.0007) [2023-12-26 23:59:08,371][105620] Updated weights for policy 1, policy_version 1183563 (0.0011) [2023-12-26 23:59:08,424][105620] Updated weights for policy 1, policy_version 1183573 (0.0011) [2023-12-26 23:59:08,473][105620] Updated weights for policy 1, policy_version 1183583 (0.0010) [2023-12-26 23:59:09,099][105692] Updated weights for policy 0, policy_version 1182413 (0.0010) [2023-12-26 23:59:09,148][105692] Updated weights for policy 0, policy_version 1182423 (0.0010) [2023-12-26 23:59:09,197][105692] Updated weights for policy 0, policy_version 1182433 (0.0010) [2023-12-26 23:59:09,229][105620] Updated weights for policy 1, policy_version 1183593 (0.0010) [2023-12-26 23:59:09,295][105620] Updated weights for policy 1, policy_version 1183603 (0.0010) [2023-12-26 23:59:09,363][105620] Updated weights for policy 1, policy_version 1183613 (0.0009) [2023-12-26 23:59:09,428][105620] Updated weights for policy 1, policy_version 1183623 (0.0009) [2023-12-26 23:59:10,022][105692] Updated weights for policy 0, policy_version 1182443 (0.0009) [2023-12-26 23:59:10,043][105620] Updated weights for policy 1, policy_version 1183633 (0.0006) [2023-12-26 23:59:10,079][105692] Updated weights for policy 0, policy_version 1182453 (0.0009) [2023-12-26 23:59:10,103][105620] Updated weights for policy 1, policy_version 1183643 (0.0007) [2023-12-26 23:59:10,130][105692] Updated weights for policy 0, policy_version 1182463 (0.0008) [2023-12-26 23:59:10,161][105620] Updated weights for policy 1, policy_version 1183653 (0.0007) [2023-12-26 23:59:10,781][105620] Updated weights for policy 1, policy_version 1183663 (0.0007) [2023-12-26 23:59:10,828][105620] Updated weights for policy 1, policy_version 1183673 (0.0009) [2023-12-26 23:59:10,887][105620] Updated weights for policy 1, policy_version 1183683 (0.0009) [2023-12-26 23:59:10,946][105692] Updated weights for policy 0, policy_version 1182473 (0.0007) [2023-12-26 23:59:11,004][105692] Updated weights for policy 0, policy_version 1182483 (0.0007) [2023-12-26 23:59:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 605822976. Throughput: 0: 9844.7, 1: 9804.1. Samples: 605832632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:59:11,063][104569] Avg episode reward: [(0, '9179.235'), (1, '8986.419')] [2023-12-26 23:59:11,070][105692] Updated weights for policy 0, policy_version 1182493 (0.0009) [2023-12-26 23:59:11,124][105692] Updated weights for policy 0, policy_version 1182503 (0.0008) [2023-12-26 23:59:11,691][105620] Updated weights for policy 1, policy_version 1183693 (0.0007) [2023-12-26 23:59:11,765][105620] Updated weights for policy 1, policy_version 1183703 (0.0008) [2023-12-26 23:59:11,821][105620] Updated weights for policy 1, policy_version 1183713 (0.0008) [2023-12-26 23:59:11,955][105692] Updated weights for policy 0, policy_version 1182513 (0.0010) [2023-12-26 23:59:12,008][105692] Updated weights for policy 0, policy_version 1182523 (0.0011) [2023-12-26 23:59:12,069][105692] Updated weights for policy 0, policy_version 1182533 (0.0011) [2023-12-26 23:59:12,528][105620] Updated weights for policy 1, policy_version 1183723 (0.0008) [2023-12-26 23:59:12,591][105620] Updated weights for policy 1, policy_version 1183733 (0.0008) [2023-12-26 23:59:12,649][105620] Updated weights for policy 1, policy_version 1183743 (0.0010) [2023-12-26 23:59:12,796][105692] Updated weights for policy 0, policy_version 1182543 (0.0011) [2023-12-26 23:59:12,850][105692] Updated weights for policy 0, policy_version 1182553 (0.0011) [2023-12-26 23:59:12,899][105692] Updated weights for policy 0, policy_version 1182563 (0.0009) [2023-12-26 23:59:13,445][105620] Updated weights for policy 1, policy_version 1183753 (0.0008) [2023-12-26 23:59:13,502][105620] Updated weights for policy 1, policy_version 1183763 (0.0005) [2023-12-26 23:59:13,548][105692] Updated weights for policy 0, policy_version 1182573 (0.0006) [2023-12-26 23:59:13,586][105620] Updated weights for policy 1, policy_version 1183773 (0.0008) [2023-12-26 23:59:13,611][105692] Updated weights for policy 0, policy_version 1182583 (0.0005) [2023-12-26 23:59:13,644][105620] Updated weights for policy 1, policy_version 1183783 (0.0009) [2023-12-26 23:59:13,670][105692] Updated weights for policy 0, policy_version 1182593 (0.0005) [2023-12-26 23:59:14,254][105692] Updated weights for policy 0, policy_version 1182603 (0.0005) [2023-12-26 23:59:14,275][105620] Updated weights for policy 1, policy_version 1183793 (0.0008) [2023-12-26 23:59:14,303][105692] Updated weights for policy 0, policy_version 1182613 (0.0005) [2023-12-26 23:59:14,333][105620] Updated weights for policy 1, policy_version 1183803 (0.0009) [2023-12-26 23:59:14,352][105692] Updated weights for policy 0, policy_version 1182623 (0.0007) [2023-12-26 23:59:14,393][105620] Updated weights for policy 1, policy_version 1183813 (0.0007) [2023-12-26 23:59:15,080][105692] Updated weights for policy 0, policy_version 1182633 (0.0010) [2023-12-26 23:59:15,141][105692] Updated weights for policy 0, policy_version 1182643 (0.0009) [2023-12-26 23:59:15,176][105620] Updated weights for policy 1, policy_version 1183823 (0.0006) [2023-12-26 23:59:15,207][105692] Updated weights for policy 0, policy_version 1182653 (0.0010) [2023-12-26 23:59:15,243][105620] Updated weights for policy 1, policy_version 1183833 (0.0007) [2023-12-26 23:59:15,269][105692] Updated weights for policy 0, policy_version 1182663 (0.0010) [2023-12-26 23:59:15,301][105620] Updated weights for policy 1, policy_version 1183843 (0.0006) [2023-12-26 23:59:15,856][105620] Updated weights for policy 1, policy_version 1183853 (0.0006) [2023-12-26 23:59:15,909][105620] Updated weights for policy 1, policy_version 1183863 (0.0005) [2023-12-26 23:59:15,960][105620] Updated weights for policy 1, policy_version 1183873 (0.0005) [2023-12-26 23:59:15,994][105692] Updated weights for policy 0, policy_version 1182673 (0.0011) [2023-12-26 23:59:16,043][105692] Updated weights for policy 0, policy_version 1182683 (0.0010) [2023-12-26 23:59:16,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 605921280. Throughput: 0: 9739.4, 1: 9767.5. Samples: 605890200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:59:16,063][104569] Avg episode reward: [(0, '9269.773'), (1, '8987.410')] [2023-12-26 23:59:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001183880_303112192.pth... [2023-12-26 23:59:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001182728_302817280.pth [2023-12-26 23:59:16,108][105692] Updated weights for policy 0, policy_version 1182693 (0.0010) [2023-12-26 23:59:16,128][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001182696_302817280.pth... [2023-12-26 23:59:16,132][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001181544_302522368.pth [2023-12-26 23:59:16,613][105620] Updated weights for policy 1, policy_version 1183883 (0.0007) [2023-12-26 23:59:16,675][105620] Updated weights for policy 1, policy_version 1183893 (0.0010) [2023-12-26 23:59:16,737][105620] Updated weights for policy 1, policy_version 1183903 (0.0010) [2023-12-26 23:59:16,845][105692] Updated weights for policy 0, policy_version 1182703 (0.0009) [2023-12-26 23:59:16,904][105692] Updated weights for policy 0, policy_version 1182713 (0.0008) [2023-12-26 23:59:16,963][105692] Updated weights for policy 0, policy_version 1182723 (0.0007) [2023-12-26 23:59:17,480][105620] Updated weights for policy 1, policy_version 1183913 (0.0010) [2023-12-26 23:59:17,535][105620] Updated weights for policy 1, policy_version 1183923 (0.0010) [2023-12-26 23:59:17,601][105620] Updated weights for policy 1, policy_version 1183933 (0.0011) [2023-12-26 23:59:17,652][105620] Updated weights for policy 1, policy_version 1183943 (0.0010) [2023-12-26 23:59:17,754][105692] Updated weights for policy 0, policy_version 1182733 (0.0009) [2023-12-26 23:59:17,822][105692] Updated weights for policy 0, policy_version 1182743 (0.0010) [2023-12-26 23:59:17,883][105692] Updated weights for policy 0, policy_version 1182753 (0.0010) [2023-12-26 23:59:18,441][105620] Updated weights for policy 1, policy_version 1183953 (0.0006) [2023-12-26 23:59:18,489][105692] Updated weights for policy 0, policy_version 1182763 (0.0010) [2023-12-26 23:59:18,509][105620] Updated weights for policy 1, policy_version 1183963 (0.0005) [2023-12-26 23:59:18,549][105692] Updated weights for policy 0, policy_version 1182773 (0.0009) [2023-12-26 23:59:18,578][105620] Updated weights for policy 1, policy_version 1183973 (0.0006) [2023-12-26 23:59:18,619][105692] Updated weights for policy 0, policy_version 1182783 (0.0011) [2023-12-26 23:59:19,257][105620] Updated weights for policy 1, policy_version 1183983 (0.0007) [2023-12-26 23:59:19,310][105620] Updated weights for policy 1, policy_version 1183993 (0.0008) [2023-12-26 23:59:19,377][105620] Updated weights for policy 1, policy_version 1184003 (0.0008) [2023-12-26 23:59:19,383][105692] Updated weights for policy 0, policy_version 1182793 (0.0011) [2023-12-26 23:59:19,438][105692] Updated weights for policy 0, policy_version 1182803 (0.0007) [2023-12-26 23:59:19,508][105692] Updated weights for policy 0, policy_version 1182813 (0.0007) [2023-12-26 23:59:19,567][105692] Updated weights for policy 0, policy_version 1182823 (0.0008) [2023-12-26 23:59:20,182][105620] Updated weights for policy 1, policy_version 1184013 (0.0008) [2023-12-26 23:59:20,243][105620] Updated weights for policy 1, policy_version 1184023 (0.0008) [2023-12-26 23:59:20,290][105692] Updated weights for policy 0, policy_version 1182833 (0.0010) [2023-12-26 23:59:20,304][105620] Updated weights for policy 1, policy_version 1184033 (0.0006) [2023-12-26 23:59:20,339][105692] Updated weights for policy 0, policy_version 1182843 (0.0011) [2023-12-26 23:59:20,401][105692] Updated weights for policy 0, policy_version 1182853 (0.0011) [2023-12-26 23:59:21,018][105692] Updated weights for policy 0, policy_version 1182863 (0.0011) [2023-12-26 23:59:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 606011392. Throughput: 0: 9685.7, 1: 9774.7. Samples: 606006912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:59:21,063][104569] Avg episode reward: [(0, '9262.521'), (1, '9078.344')] [2023-12-26 23:59:21,079][105692] Updated weights for policy 0, policy_version 1182873 (0.0010) [2023-12-26 23:59:21,141][105692] Updated weights for policy 0, policy_version 1182883 (0.0010) [2023-12-26 23:59:21,145][105620] Updated weights for policy 1, policy_version 1184043 (0.0007) [2023-12-26 23:59:21,213][105620] Updated weights for policy 1, policy_version 1184053 (0.0008) [2023-12-26 23:59:21,281][105620] Updated weights for policy 1, policy_version 1184063 (0.0007) [2023-12-26 23:59:21,942][105620] Updated weights for policy 1, policy_version 1184073 (0.0006) [2023-12-26 23:59:21,952][105692] Updated weights for policy 0, policy_version 1182893 (0.0010) [2023-12-26 23:59:22,000][105620] Updated weights for policy 1, policy_version 1184083 (0.0006) [2023-12-26 23:59:22,009][105692] Updated weights for policy 0, policy_version 1182903 (0.0008) [2023-12-26 23:59:22,057][105620] Updated weights for policy 1, policy_version 1184093 (0.0007) [2023-12-26 23:59:22,072][105692] Updated weights for policy 0, policy_version 1182913 (0.0007) [2023-12-26 23:59:22,120][105620] Updated weights for policy 1, policy_version 1184103 (0.0007) [2023-12-26 23:59:22,813][105620] Updated weights for policy 1, policy_version 1184113 (0.0009) [2023-12-26 23:59:22,872][105620] Updated weights for policy 1, policy_version 1184123 (0.0008) [2023-12-26 23:59:22,878][105692] Updated weights for policy 0, policy_version 1182923 (0.0008) [2023-12-26 23:59:22,927][105620] Updated weights for policy 1, policy_version 1184133 (0.0007) [2023-12-26 23:59:22,943][105692] Updated weights for policy 0, policy_version 1182933 (0.0009) [2023-12-26 23:59:23,005][105692] Updated weights for policy 0, policy_version 1182943 (0.0009) [2023-12-26 23:59:23,619][105620] Updated weights for policy 1, policy_version 1184143 (0.0007) [2023-12-26 23:59:23,670][105620] Updated weights for policy 1, policy_version 1184153 (0.0005) [2023-12-26 23:59:23,723][105620] Updated weights for policy 1, policy_version 1184163 (0.0008) [2023-12-26 23:59:23,831][105692] Updated weights for policy 0, policy_version 1182953 (0.0009) [2023-12-26 23:59:23,884][105692] Updated weights for policy 0, policy_version 1182963 (0.0010) [2023-12-26 23:59:23,942][105692] Updated weights for policy 0, policy_version 1182974 (0.0010) [2023-12-26 23:59:24,284][105620] Updated weights for policy 1, policy_version 1184173 (0.0009) [2023-12-26 23:59:24,335][105620] Updated weights for policy 1, policy_version 1184183 (0.0009) [2023-12-26 23:59:24,383][105620] Updated weights for policy 1, policy_version 1184193 (0.0009) [2023-12-26 23:59:24,761][105692] Updated weights for policy 0, policy_version 1182985 (0.0009) [2023-12-26 23:59:24,813][105692] Updated weights for policy 0, policy_version 1182995 (0.0010) [2023-12-26 23:59:24,861][105692] Updated weights for policy 0, policy_version 1183006 (0.0009) [2023-12-26 23:59:25,087][105620] Updated weights for policy 1, policy_version 1184203 (0.0010) [2023-12-26 23:59:25,132][105620] Updated weights for policy 1, policy_version 1184213 (0.0010) [2023-12-26 23:59:25,191][105620] Updated weights for policy 1, policy_version 1184223 (0.0010) [2023-12-26 23:59:25,751][105620] Updated weights for policy 1, policy_version 1184233 (0.0009) [2023-12-26 23:59:25,763][105692] Updated weights for policy 0, policy_version 1183018 (0.0010) [2023-12-26 23:59:25,808][105620] Updated weights for policy 1, policy_version 1184243 (0.0005) [2023-12-26 23:59:25,817][105692] Updated weights for policy 0, policy_version 1183028 (0.0008) [2023-12-26 23:59:25,865][105620] Updated weights for policy 1, policy_version 1184253 (0.0007) [2023-12-26 23:59:25,885][105692] Updated weights for policy 0, policy_version 1183038 (0.0007) [2023-12-26 23:59:25,921][105620] Updated weights for policy 1, policy_version 1184263 (0.0005) [2023-12-26 23:59:25,937][105692] Updated weights for policy 0, policy_version 1183048 (0.0009) [2023-12-26 23:59:26,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 606117888. Throughput: 0: 9622.6, 1: 9748.8. Samples: 606122248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:59:26,062][104569] Avg episode reward: [(0, '8991.666'), (1, '9259.629')] [2023-12-26 23:59:26,455][105620] Updated weights for policy 1, policy_version 1184273 (0.0005) [2023-12-26 23:59:26,506][105620] Updated weights for policy 1, policy_version 1184283 (0.0005) [2023-12-26 23:59:26,556][105620] Updated weights for policy 1, policy_version 1184293 (0.0005) [2023-12-26 23:59:26,857][105692] Updated weights for policy 0, policy_version 1183058 (0.0009) [2023-12-26 23:59:26,909][105692] Updated weights for policy 0, policy_version 1183068 (0.0010) [2023-12-26 23:59:26,963][105692] Updated weights for policy 0, policy_version 1183080 (0.0010) [2023-12-26 23:59:27,079][105620] Updated weights for policy 1, policy_version 1184303 (0.0005) [2023-12-26 23:59:27,136][105620] Updated weights for policy 1, policy_version 1184313 (0.0005) [2023-12-26 23:59:27,191][105620] Updated weights for policy 1, policy_version 1184323 (0.0008) [2023-12-26 23:59:27,706][105620] Updated weights for policy 1, policy_version 1184333 (0.0007) [2023-12-26 23:59:27,769][105620] Updated weights for policy 1, policy_version 1184343 (0.0005) [2023-12-26 23:59:27,830][105620] Updated weights for policy 1, policy_version 1184353 (0.0005) [2023-12-26 23:59:27,874][105692] Updated weights for policy 0, policy_version 1183090 (0.0009) [2023-12-26 23:59:27,922][105692] Updated weights for policy 0, policy_version 1183101 (0.0009) [2023-12-26 23:59:27,977][105692] Updated weights for policy 0, policy_version 1183111 (0.0005) [2023-12-26 23:59:28,433][105620] Updated weights for policy 1, policy_version 1184363 (0.0007) [2023-12-26 23:59:28,492][105620] Updated weights for policy 1, policy_version 1184373 (0.0010) [2023-12-26 23:59:28,557][105620] Updated weights for policy 1, policy_version 1184383 (0.0011) [2023-12-26 23:59:28,715][105692] Updated weights for policy 0, policy_version 1183121 (0.0010) [2023-12-26 23:59:28,779][105692] Updated weights for policy 0, policy_version 1183131 (0.0010) [2023-12-26 23:59:28,839][105692] Updated weights for policy 0, policy_version 1183141 (0.0008) [2023-12-26 23:59:29,285][105620] Updated weights for policy 1, policy_version 1184393 (0.0010) [2023-12-26 23:59:29,338][105620] Updated weights for policy 1, policy_version 1184403 (0.0011) [2023-12-26 23:59:29,396][105620] Updated weights for policy 1, policy_version 1184413 (0.0010) [2023-12-26 23:59:29,442][105620] Updated weights for policy 1, policy_version 1184423 (0.0009) [2023-12-26 23:59:29,568][105692] Updated weights for policy 0, policy_version 1183151 (0.0006) [2023-12-26 23:59:29,640][105692] Updated weights for policy 0, policy_version 1183161 (0.0005) [2023-12-26 23:59:29,705][105692] Updated weights for policy 0, policy_version 1183171 (0.0008) [2023-12-26 23:59:30,045][105620] Updated weights for policy 1, policy_version 1184433 (0.0008) [2023-12-26 23:59:30,095][105620] Updated weights for policy 1, policy_version 1184443 (0.0008) [2023-12-26 23:59:30,145][105620] Updated weights for policy 1, policy_version 1184453 (0.0008) [2023-12-26 23:59:30,371][105692] Updated weights for policy 0, policy_version 1183181 (0.0009) [2023-12-26 23:59:30,415][105692] Updated weights for policy 0, policy_version 1183191 (0.0008) [2023-12-26 23:59:30,463][105692] Updated weights for policy 0, policy_version 1183201 (0.0008) [2023-12-26 23:59:30,892][105620] Updated weights for policy 1, policy_version 1184463 (0.0010) [2023-12-26 23:59:30,956][105620] Updated weights for policy 1, policy_version 1184473 (0.0010) [2023-12-26 23:59:31,021][105620] Updated weights for policy 1, policy_version 1184483 (0.0010) [2023-12-26 23:59:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 606216192. Throughput: 0: 9568.0, 1: 9879.8. Samples: 606183068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:59:31,062][104569] Avg episode reward: [(0, '9082.090'), (1, '9168.421')] [2023-12-26 23:59:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001184488_303267840.pth... [2023-12-26 23:59:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001183208_302948352.pth... [2023-12-26 23:59:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001182120_302669824.pth [2023-12-26 23:59:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001183272_302956544.pth [2023-12-26 23:59:31,176][105692] Updated weights for policy 0, policy_version 1183211 (0.0007) [2023-12-26 23:59:31,227][105692] Updated weights for policy 0, policy_version 1183221 (0.0005) [2023-12-26 23:59:31,287][105692] Updated weights for policy 0, policy_version 1183231 (0.0009) [2023-12-26 23:59:31,645][105620] Updated weights for policy 1, policy_version 1184493 (0.0009) [2023-12-26 23:59:31,693][105620] Updated weights for policy 1, policy_version 1184503 (0.0005) [2023-12-26 23:59:31,752][105620] Updated weights for policy 1, policy_version 1184513 (0.0009) [2023-12-26 23:59:31,942][105692] Updated weights for policy 0, policy_version 1183241 (0.0010) [2023-12-26 23:59:31,994][105692] Updated weights for policy 0, policy_version 1183251 (0.0008) [2023-12-26 23:59:32,042][105692] Updated weights for policy 0, policy_version 1183261 (0.0007) [2023-12-26 23:59:32,091][105692] Updated weights for policy 0, policy_version 1183271 (0.0008) [2023-12-26 23:59:32,482][105620] Updated weights for policy 1, policy_version 1184523 (0.0009) [2023-12-26 23:59:32,546][105620] Updated weights for policy 1, policy_version 1184533 (0.0005) [2023-12-26 23:59:32,614][105620] Updated weights for policy 1, policy_version 1184543 (0.0006) [2023-12-26 23:59:32,902][105692] Updated weights for policy 0, policy_version 1183281 (0.0005) [2023-12-26 23:59:32,956][105692] Updated weights for policy 0, policy_version 1183291 (0.0005) [2023-12-26 23:59:33,013][105692] Updated weights for policy 0, policy_version 1183301 (0.0006) [2023-12-26 23:59:33,197][105620] Updated weights for policy 1, policy_version 1184553 (0.0008) [2023-12-26 23:59:33,246][105620] Updated weights for policy 1, policy_version 1184563 (0.0010) [2023-12-26 23:59:33,300][105620] Updated weights for policy 1, policy_version 1184573 (0.0010) [2023-12-26 23:59:33,347][105620] Updated weights for policy 1, policy_version 1184583 (0.0010) [2023-12-26 23:59:33,563][105692] Updated weights for policy 0, policy_version 1183311 (0.0005) [2023-12-26 23:59:33,621][105692] Updated weights for policy 0, policy_version 1183321 (0.0005) [2023-12-26 23:59:33,674][105692] Updated weights for policy 0, policy_version 1183331 (0.0005) [2023-12-26 23:59:34,125][105620] Updated weights for policy 1, policy_version 1184593 (0.0011) [2023-12-26 23:59:34,195][105620] Updated weights for policy 1, policy_version 1184603 (0.0011) [2023-12-26 23:59:34,261][105620] Updated weights for policy 1, policy_version 1184613 (0.0009) [2023-12-26 23:59:34,311][105692] Updated weights for policy 0, policy_version 1183341 (0.0008) [2023-12-26 23:59:34,376][105692] Updated weights for policy 0, policy_version 1183351 (0.0011) [2023-12-26 23:59:34,438][105692] Updated weights for policy 0, policy_version 1183361 (0.0010) [2023-12-26 23:59:34,990][105620] Updated weights for policy 1, policy_version 1184623 (0.0010) [2023-12-26 23:59:35,056][105620] Updated weights for policy 1, policy_version 1184633 (0.0010) [2023-12-26 23:59:35,081][105692] Updated weights for policy 0, policy_version 1183371 (0.0005) [2023-12-26 23:59:35,111][105620] Updated weights for policy 1, policy_version 1184643 (0.0010) [2023-12-26 23:59:35,139][105692] Updated weights for policy 0, policy_version 1183381 (0.0010) [2023-12-26 23:59:35,194][105692] Updated weights for policy 0, policy_version 1183391 (0.0010) [2023-12-26 23:59:35,769][105620] Updated weights for policy 1, policy_version 1184653 (0.0008) [2023-12-26 23:59:35,831][105620] Updated weights for policy 1, policy_version 1184663 (0.0005) [2023-12-26 23:59:35,842][105692] Updated weights for policy 0, policy_version 1183401 (0.0010) [2023-12-26 23:59:35,882][105620] Updated weights for policy 1, policy_version 1184673 (0.0010) [2023-12-26 23:59:35,900][105692] Updated weights for policy 0, policy_version 1183411 (0.0007) [2023-12-26 23:59:35,953][105692] Updated weights for policy 0, policy_version 1183421 (0.0007) [2023-12-26 23:59:36,007][105692] Updated weights for policy 0, policy_version 1183432 (0.0010) [2023-12-26 23:59:36,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 606322688. Throughput: 0: 9594.0, 1: 9965.6. Samples: 606304388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:59:36,063][104569] Avg episode reward: [(0, '9171.392'), (1, '9168.970')] [2023-12-26 23:59:36,595][105620] Updated weights for policy 1, policy_version 1184683 (0.0010) [2023-12-26 23:59:36,655][105620] Updated weights for policy 1, policy_version 1184693 (0.0010) [2023-12-26 23:59:36,673][105692] Updated weights for policy 0, policy_version 1183442 (0.0011) [2023-12-26 23:59:36,714][105620] Updated weights for policy 1, policy_version 1184703 (0.0010) [2023-12-26 23:59:36,728][105692] Updated weights for policy 0, policy_version 1183452 (0.0010) [2023-12-26 23:59:36,784][105692] Updated weights for policy 0, policy_version 1183462 (0.0011) [2023-12-26 23:59:37,464][105620] Updated weights for policy 1, policy_version 1184713 (0.0010) [2023-12-26 23:59:37,526][105692] Updated weights for policy 0, policy_version 1183472 (0.0011) [2023-12-26 23:59:37,527][105620] Updated weights for policy 1, policy_version 1184723 (0.0010) [2023-12-26 23:59:37,584][105692] Updated weights for policy 0, policy_version 1183482 (0.0009) [2023-12-26 23:59:37,586][105620] Updated weights for policy 1, policy_version 1184733 (0.0010) [2023-12-26 23:59:37,639][105692] Updated weights for policy 0, policy_version 1183492 (0.0010) [2023-12-26 23:59:37,642][105620] Updated weights for policy 1, policy_version 1184743 (0.0011) [2023-12-26 23:59:38,244][105692] Updated weights for policy 0, policy_version 1183502 (0.0005) [2023-12-26 23:59:38,305][105692] Updated weights for policy 0, policy_version 1183512 (0.0005) [2023-12-26 23:59:38,372][105692] Updated weights for policy 0, policy_version 1183522 (0.0009) [2023-12-26 23:59:38,404][105620] Updated weights for policy 1, policy_version 1184753 (0.0011) [2023-12-26 23:59:38,463][105620] Updated weights for policy 1, policy_version 1184763 (0.0010) [2023-12-26 23:59:38,518][105620] Updated weights for policy 1, policy_version 1184773 (0.0010) [2023-12-26 23:59:39,070][105692] Updated weights for policy 0, policy_version 1183532 (0.0010) [2023-12-26 23:59:39,121][105692] Updated weights for policy 0, policy_version 1183542 (0.0010) [2023-12-26 23:59:39,172][105692] Updated weights for policy 0, policy_version 1183552 (0.0010) [2023-12-26 23:59:39,228][105620] Updated weights for policy 1, policy_version 1184783 (0.0008) [2023-12-26 23:59:39,293][105620] Updated weights for policy 1, policy_version 1184793 (0.0006) [2023-12-26 23:59:39,357][105620] Updated weights for policy 1, policy_version 1184803 (0.0007) [2023-12-26 23:59:39,879][105692] Updated weights for policy 0, policy_version 1183562 (0.0009) [2023-12-26 23:59:39,948][105692] Updated weights for policy 0, policy_version 1183572 (0.0009) [2023-12-26 23:59:40,016][105692] Updated weights for policy 0, policy_version 1183582 (0.0008) [2023-12-26 23:59:40,063][105620] Updated weights for policy 1, policy_version 1184813 (0.0009) [2023-12-26 23:59:40,077][105692] Updated weights for policy 0, policy_version 1183592 (0.0008) [2023-12-26 23:59:40,130][105620] Updated weights for policy 1, policy_version 1184823 (0.0011) [2023-12-26 23:59:40,201][105620] Updated weights for policy 1, policy_version 1184833 (0.0011) [2023-12-26 23:59:40,726][105692] Updated weights for policy 0, policy_version 1183602 (0.0008) [2023-12-26 23:59:40,789][105692] Updated weights for policy 0, policy_version 1183612 (0.0011) [2023-12-26 23:59:40,851][105692] Updated weights for policy 0, policy_version 1183622 (0.0009) [2023-12-26 23:59:40,871][105620] Updated weights for policy 1, policy_version 1184843 (0.0010) [2023-12-26 23:59:40,937][105620] Updated weights for policy 1, policy_version 1184853 (0.0008) [2023-12-26 23:59:41,004][105620] Updated weights for policy 1, policy_version 1184863 (0.0009) [2023-12-26 23:59:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 606412800. Throughput: 0: 9574.0, 1: 10080.6. Samples: 606424116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:59:41,062][104569] Avg episode reward: [(0, '9172.351'), (1, '9261.403')] [2023-12-26 23:59:41,561][105692] Updated weights for policy 0, policy_version 1183632 (0.0009) [2023-12-26 23:59:41,623][105692] Updated weights for policy 0, policy_version 1183642 (0.0008) [2023-12-26 23:59:41,681][105692] Updated weights for policy 0, policy_version 1183652 (0.0008) [2023-12-26 23:59:41,769][105620] Updated weights for policy 1, policy_version 1184873 (0.0010) [2023-12-26 23:59:41,837][105620] Updated weights for policy 1, policy_version 1184883 (0.0008) [2023-12-26 23:59:41,903][105620] Updated weights for policy 1, policy_version 1184893 (0.0008) [2023-12-26 23:59:41,968][105620] Updated weights for policy 1, policy_version 1184903 (0.0009) [2023-12-26 23:59:42,487][105692] Updated weights for policy 0, policy_version 1183662 (0.0007) [2023-12-26 23:59:42,546][105692] Updated weights for policy 0, policy_version 1183672 (0.0008) [2023-12-26 23:59:42,609][105692] Updated weights for policy 0, policy_version 1183682 (0.0008) [2023-12-26 23:59:42,732][105620] Updated weights for policy 1, policy_version 1184913 (0.0010) [2023-12-26 23:59:42,791][105620] Updated weights for policy 1, policy_version 1184923 (0.0010) [2023-12-26 23:59:42,850][105620] Updated weights for policy 1, policy_version 1184933 (0.0011) [2023-12-26 23:59:43,408][105692] Updated weights for policy 0, policy_version 1183692 (0.0008) [2023-12-26 23:59:43,472][105692] Updated weights for policy 0, policy_version 1183702 (0.0007) [2023-12-26 23:59:43,501][105620] Updated weights for policy 1, policy_version 1184943 (0.0010) [2023-12-26 23:59:43,530][105692] Updated weights for policy 0, policy_version 1183712 (0.0006) [2023-12-26 23:59:43,559][105620] Updated weights for policy 1, policy_version 1184953 (0.0010) [2023-12-26 23:59:43,617][105620] Updated weights for policy 1, policy_version 1184963 (0.0010) [2023-12-26 23:59:44,247][105692] Updated weights for policy 0, policy_version 1183722 (0.0007) [2023-12-26 23:59:44,294][105692] Updated weights for policy 0, policy_version 1183732 (0.0008) [2023-12-26 23:59:44,347][105692] Updated weights for policy 0, policy_version 1183742 (0.0007) [2023-12-26 23:59:44,360][105620] Updated weights for policy 1, policy_version 1184973 (0.0010) [2023-12-26 23:59:44,405][105692] Updated weights for policy 0, policy_version 1183752 (0.0007) [2023-12-26 23:59:44,414][105620] Updated weights for policy 1, policy_version 1184983 (0.0010) [2023-12-26 23:59:44,467][105620] Updated weights for policy 1, policy_version 1184993 (0.0010) [2023-12-26 23:59:45,074][105692] Updated weights for policy 0, policy_version 1183762 (0.0008) [2023-12-26 23:59:45,135][105692] Updated weights for policy 0, policy_version 1183772 (0.0007) [2023-12-26 23:59:45,195][105692] Updated weights for policy 0, policy_version 1183782 (0.0007) [2023-12-26 23:59:45,234][105620] Updated weights for policy 1, policy_version 1185003 (0.0010) [2023-12-26 23:59:45,304][105620] Updated weights for policy 1, policy_version 1185013 (0.0010) [2023-12-26 23:59:45,366][105620] Updated weights for policy 1, policy_version 1185023 (0.0011) [2023-12-26 23:59:45,936][105692] Updated weights for policy 0, policy_version 1183792 (0.0009) [2023-12-26 23:59:45,990][105692] Updated weights for policy 0, policy_version 1183802 (0.0008) [2023-12-26 23:59:45,994][105620] Updated weights for policy 1, policy_version 1185033 (0.0008) [2023-12-26 23:59:46,041][105692] Updated weights for policy 0, policy_version 1183812 (0.0005) [2023-12-26 23:59:46,047][105620] Updated weights for policy 1, policy_version 1185043 (0.0005) [2023-12-26 23:59:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 606511104. Throughput: 0: 9550.3, 1: 10044.9. Samples: 606480312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:59:46,063][104569] Avg episode reward: [(0, '9268.068'), (1, '9260.827')] [2023-12-26 23:59:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001183816_303104000.pth... [2023-12-26 23:59:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001182696_302817280.pth [2023-12-26 23:59:46,103][105620] Updated weights for policy 1, policy_version 1185053 (0.0006) [2023-12-26 23:59:46,162][105620] Updated weights for policy 1, policy_version 1185063 (0.0005) [2023-12-26 23:59:46,168][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001185064_303415296.pth... [2023-12-26 23:59:46,171][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001183880_303112192.pth [2023-12-26 23:59:46,738][105620] Updated weights for policy 1, policy_version 1185073 (0.0008) [2023-12-26 23:59:46,803][105620] Updated weights for policy 1, policy_version 1185083 (0.0009) [2023-12-26 23:59:46,827][105692] Updated weights for policy 0, policy_version 1183822 (0.0007) [2023-12-26 23:59:46,858][105620] Updated weights for policy 1, policy_version 1185093 (0.0006) [2023-12-26 23:59:46,887][105692] Updated weights for policy 0, policy_version 1183832 (0.0009) [2023-12-26 23:59:46,947][105692] Updated weights for policy 0, policy_version 1183842 (0.0009) [2023-12-26 23:59:47,616][105620] Updated weights for policy 1, policy_version 1185103 (0.0009) [2023-12-26 23:59:47,655][105692] Updated weights for policy 0, policy_version 1183852 (0.0009) [2023-12-26 23:59:47,674][105620] Updated weights for policy 1, policy_version 1185113 (0.0005) [2023-12-26 23:59:47,713][105692] Updated weights for policy 0, policy_version 1183862 (0.0008) [2023-12-26 23:59:47,730][105620] Updated weights for policy 1, policy_version 1185123 (0.0006) [2023-12-26 23:59:47,770][105692] Updated weights for policy 0, policy_version 1183872 (0.0009) [2023-12-26 23:59:48,376][105620] Updated weights for policy 1, policy_version 1185133 (0.0007) [2023-12-26 23:59:48,430][105620] Updated weights for policy 1, policy_version 1185143 (0.0008) [2023-12-26 23:59:48,494][105620] Updated weights for policy 1, policy_version 1185153 (0.0009) [2023-12-26 23:59:48,580][105692] Updated weights for policy 0, policy_version 1183882 (0.0009) [2023-12-26 23:59:48,649][105692] Updated weights for policy 0, policy_version 1183892 (0.0010) [2023-12-26 23:59:48,710][105692] Updated weights for policy 0, policy_version 1183902 (0.0008) [2023-12-26 23:59:48,772][105692] Updated weights for policy 0, policy_version 1183912 (0.0009) [2023-12-26 23:59:49,222][105620] Updated weights for policy 1, policy_version 1185163 (0.0009) [2023-12-26 23:59:49,298][105620] Updated weights for policy 1, policy_version 1185173 (0.0009) [2023-12-26 23:59:49,370][105620] Updated weights for policy 1, policy_version 1185183 (0.0009) [2023-12-26 23:59:49,482][105692] Updated weights for policy 0, policy_version 1183922 (0.0007) [2023-12-26 23:59:49,539][105692] Updated weights for policy 0, policy_version 1183932 (0.0009) [2023-12-26 23:59:49,592][105692] Updated weights for policy 0, policy_version 1183942 (0.0009) [2023-12-26 23:59:50,094][105620] Updated weights for policy 1, policy_version 1185193 (0.0007) [2023-12-26 23:59:50,152][105620] Updated weights for policy 1, policy_version 1185203 (0.0010) [2023-12-26 23:59:50,212][105620] Updated weights for policy 1, policy_version 1185213 (0.0008) [2023-12-26 23:59:50,265][105620] Updated weights for policy 1, policy_version 1185223 (0.0008) [2023-12-26 23:59:50,298][105692] Updated weights for policy 0, policy_version 1183952 (0.0008) [2023-12-26 23:59:50,359][105692] Updated weights for policy 0, policy_version 1183962 (0.0008) [2023-12-26 23:59:50,423][105692] Updated weights for policy 0, policy_version 1183972 (0.0009) [2023-12-26 23:59:51,048][105620] Updated weights for policy 1, policy_version 1185233 (0.0009) [2023-12-26 23:59:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 606601216. Throughput: 0: 9569.0, 1: 10060.6. Samples: 606597188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:59:51,063][104569] Avg episode reward: [(0, '9176.121'), (1, '9261.112')] [2023-12-26 23:59:51,099][105620] Updated weights for policy 1, policy_version 1185243 (0.0008) [2023-12-26 23:59:51,164][105620] Updated weights for policy 1, policy_version 1185253 (0.0007) [2023-12-26 23:59:51,175][105692] Updated weights for policy 0, policy_version 1183982 (0.0008) [2023-12-26 23:59:51,236][105692] Updated weights for policy 0, policy_version 1183992 (0.0010) [2023-12-26 23:59:51,306][105692] Updated weights for policy 0, policy_version 1184002 (0.0009) [2023-12-26 23:59:51,900][105620] Updated weights for policy 1, policy_version 1185263 (0.0006) [2023-12-26 23:59:51,965][105620] Updated weights for policy 1, policy_version 1185273 (0.0007) [2023-12-26 23:59:52,025][105620] Updated weights for policy 1, policy_version 1185283 (0.0006) [2023-12-26 23:59:52,102][105692] Updated weights for policy 0, policy_version 1184012 (0.0009) [2023-12-26 23:59:52,149][105692] Updated weights for policy 0, policy_version 1184022 (0.0009) [2023-12-26 23:59:52,203][105692] Updated weights for policy 0, policy_version 1184032 (0.0009) [2023-12-26 23:59:52,693][105620] Updated weights for policy 1, policy_version 1185293 (0.0007) [2023-12-26 23:59:52,744][105620] Updated weights for policy 1, policy_version 1185303 (0.0005) [2023-12-26 23:59:52,794][105620] Updated weights for policy 1, policy_version 1185313 (0.0006) [2023-12-26 23:59:53,050][105692] Updated weights for policy 0, policy_version 1184042 (0.0009) [2023-12-26 23:59:53,104][105692] Updated weights for policy 0, policy_version 1184052 (0.0010) [2023-12-26 23:59:53,155][105692] Updated weights for policy 0, policy_version 1184062 (0.0009) [2023-12-26 23:59:53,206][105692] Updated weights for policy 0, policy_version 1184072 (0.0007) [2023-12-26 23:59:53,390][105620] Updated weights for policy 1, policy_version 1185323 (0.0009) [2023-12-26 23:59:53,438][105620] Updated weights for policy 1, policy_version 1185333 (0.0008) [2023-12-26 23:59:53,487][105620] Updated weights for policy 1, policy_version 1185343 (0.0006) [2023-12-26 23:59:53,960][105692] Updated weights for policy 0, policy_version 1184082 (0.0009) [2023-12-26 23:59:54,011][105692] Updated weights for policy 0, policy_version 1184092 (0.0009) [2023-12-26 23:59:54,063][105692] Updated weights for policy 0, policy_version 1184102 (0.0009) [2023-12-26 23:59:54,156][105620] Updated weights for policy 1, policy_version 1185353 (0.0006) [2023-12-26 23:59:54,209][105620] Updated weights for policy 1, policy_version 1185363 (0.0008) [2023-12-26 23:59:54,270][105620] Updated weights for policy 1, policy_version 1185373 (0.0009) [2023-12-26 23:59:54,316][105620] Updated weights for policy 1, policy_version 1185383 (0.0008) [2023-12-26 23:59:54,763][105692] Updated weights for policy 0, policy_version 1184112 (0.0007) [2023-12-26 23:59:54,826][105692] Updated weights for policy 0, policy_version 1184122 (0.0005) [2023-12-26 23:59:54,870][105692] Updated weights for policy 0, policy_version 1184132 (0.0007) [2023-12-26 23:59:55,180][105620] Updated weights for policy 1, policy_version 1185393 (0.0009) [2023-12-26 23:59:55,226][105620] Updated weights for policy 1, policy_version 1185403 (0.0008) [2023-12-26 23:59:55,273][105620] Updated weights for policy 1, policy_version 1185413 (0.0009) [2023-12-26 23:59:55,498][105692] Updated weights for policy 0, policy_version 1184142 (0.0008) [2023-12-26 23:59:55,565][105692] Updated weights for policy 0, policy_version 1184152 (0.0009) [2023-12-26 23:59:55,627][105692] Updated weights for policy 0, policy_version 1184162 (0.0009) [2023-12-26 23:59:55,993][105620] Updated weights for policy 1, policy_version 1185423 (0.0006) [2023-12-26 23:59:56,059][105620] Updated weights for policy 1, policy_version 1185433 (0.0005) [2023-12-26 23:59:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 606699520. Throughput: 0: 9566.3, 1: 9981.6. Samples: 606712292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-26 23:59:56,063][104569] Avg episode reward: [(0, '8953.933'), (1, '9261.354')] [2023-12-26 23:59:56,122][105620] Updated weights for policy 1, policy_version 1185443 (0.0007) [2023-12-26 23:59:56,383][105692] Updated weights for policy 0, policy_version 1184172 (0.0009) [2023-12-26 23:59:56,443][105692] Updated weights for policy 0, policy_version 1184182 (0.0009) [2023-12-26 23:59:56,489][105692] Updated weights for policy 0, policy_version 1184192 (0.0008) [2023-12-26 23:59:56,781][105620] Updated weights for policy 1, policy_version 1185453 (0.0009) [2023-12-26 23:59:56,834][105620] Updated weights for policy 1, policy_version 1185463 (0.0008) [2023-12-26 23:59:56,894][105620] Updated weights for policy 1, policy_version 1185473 (0.0009) [2023-12-26 23:59:57,241][105692] Updated weights for policy 0, policy_version 1184202 (0.0009) [2023-12-26 23:59:57,291][105692] Updated weights for policy 0, policy_version 1184212 (0.0009) [2023-12-26 23:59:57,349][105692] Updated weights for policy 0, policy_version 1184222 (0.0009) [2023-12-26 23:59:57,406][105692] Updated weights for policy 0, policy_version 1184232 (0.0009) [2023-12-26 23:59:57,650][105620] Updated weights for policy 1, policy_version 1185483 (0.0009) [2023-12-26 23:59:57,717][105620] Updated weights for policy 1, policy_version 1185493 (0.0009) [2023-12-26 23:59:57,778][105620] Updated weights for policy 1, policy_version 1185503 (0.0010) [2023-12-26 23:59:58,035][105692] Updated weights for policy 0, policy_version 1184242 (0.0009) [2023-12-26 23:59:58,101][105692] Updated weights for policy 0, policy_version 1184252 (0.0005) [2023-12-26 23:59:58,163][105692] Updated weights for policy 0, policy_version 1184262 (0.0007) [2023-12-26 23:59:58,449][105620] Updated weights for policy 1, policy_version 1185513 (0.0007) [2023-12-26 23:59:58,509][105620] Updated weights for policy 1, policy_version 1185523 (0.0008) [2023-12-26 23:59:58,572][105620] Updated weights for policy 1, policy_version 1185533 (0.0009) [2023-12-26 23:59:58,635][105620] Updated weights for policy 1, policy_version 1185543 (0.0010) [2023-12-26 23:59:58,971][105692] Updated weights for policy 0, policy_version 1184272 (0.0008) [2023-12-26 23:59:59,038][105692] Updated weights for policy 0, policy_version 1184282 (0.0009) [2023-12-26 23:59:59,102][105692] Updated weights for policy 0, policy_version 1184292 (0.0009) [2023-12-26 23:59:59,415][105620] Updated weights for policy 1, policy_version 1185553 (0.0008) [2023-12-26 23:59:59,467][105620] Updated weights for policy 1, policy_version 1185563 (0.0007) [2023-12-26 23:59:59,529][105620] Updated weights for policy 1, policy_version 1185573 (0.0006) [2023-12-26 23:59:59,933][105692] Updated weights for policy 0, policy_version 1184302 (0.0008) [2023-12-26 23:59:59,994][105692] Updated weights for policy 0, policy_version 1184312 (0.0010) [2023-12-27 00:00:00,064][105692] Updated weights for policy 0, policy_version 1184322 (0.0009) [2023-12-27 00:00:00,189][105620] Updated weights for policy 1, policy_version 1185583 (0.0007) [2023-12-27 00:00:00,253][105620] Updated weights for policy 1, policy_version 1185593 (0.0009) [2023-12-27 00:00:00,320][105620] Updated weights for policy 1, policy_version 1185603 (0.0009) [2023-12-27 00:00:00,862][105692] Updated weights for policy 0, policy_version 1184332 (0.0010) [2023-12-27 00:00:00,931][105692] Updated weights for policy 0, policy_version 1184342 (0.0010) [2023-12-27 00:00:00,996][105692] Updated weights for policy 0, policy_version 1184352 (0.0008) [2023-12-27 00:00:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 606797824. Throughput: 0: 9579.1, 1: 9969.5. Samples: 606769888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:00:01,063][104569] Avg episode reward: [(0, '9134.646'), (1, '9261.098')] [2023-12-27 00:00:01,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001184360_303243264.pth... [2023-12-27 00:00:01,071][105620] Updated weights for policy 1, policy_version 1185613 (0.0010) [2023-12-27 00:00:01,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001183208_302948352.pth [2023-12-27 00:00:01,140][105620] Updated weights for policy 1, policy_version 1185623 (0.0008) [2023-12-27 00:00:01,204][105620] Updated weights for policy 1, policy_version 1185633 (0.0009) [2023-12-27 00:00:01,249][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001185640_303562752.pth... [2023-12-27 00:00:01,254][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001184488_303267840.pth [2023-12-27 00:00:01,856][105692] Updated weights for policy 0, policy_version 1184362 (0.0009) [2023-12-27 00:00:01,914][105692] Updated weights for policy 0, policy_version 1184372 (0.0008) [2023-12-27 00:00:01,977][105692] Updated weights for policy 0, policy_version 1184382 (0.0009) [2023-12-27 00:00:02,036][105620] Updated weights for policy 1, policy_version 1185643 (0.0008) [2023-12-27 00:00:02,043][105692] Updated weights for policy 0, policy_version 1184392 (0.0008) [2023-12-27 00:00:02,103][105620] Updated weights for policy 1, policy_version 1185653 (0.0008) [2023-12-27 00:00:02,166][105620] Updated weights for policy 1, policy_version 1185663 (0.0009) [2023-12-27 00:00:02,914][105692] Updated weights for policy 0, policy_version 1184402 (0.0009) [2023-12-27 00:00:02,983][105692] Updated weights for policy 0, policy_version 1184412 (0.0008) [2023-12-27 00:00:03,003][105620] Updated weights for policy 1, policy_version 1185673 (0.0007) [2023-12-27 00:00:03,049][105692] Updated weights for policy 0, policy_version 1184422 (0.0008) [2023-12-27 00:00:03,069][105620] Updated weights for policy 1, policy_version 1185683 (0.0007) [2023-12-27 00:00:03,130][105620] Updated weights for policy 1, policy_version 1185693 (0.0009) [2023-12-27 00:00:03,201][105620] Updated weights for policy 1, policy_version 1185703 (0.0008) [2023-12-27 00:00:03,895][105692] Updated weights for policy 0, policy_version 1184432 (0.0007) [2023-12-27 00:00:03,967][105692] Updated weights for policy 0, policy_version 1184442 (0.0007) [2023-12-27 00:00:03,995][105620] Updated weights for policy 1, policy_version 1185713 (0.0010) [2023-12-27 00:00:04,039][105692] Updated weights for policy 0, policy_version 1184452 (0.0006) [2023-12-27 00:00:04,066][105620] Updated weights for policy 1, policy_version 1185723 (0.0011) [2023-12-27 00:00:04,132][105620] Updated weights for policy 1, policy_version 1185733 (0.0011) [2023-12-27 00:00:04,831][105692] Updated weights for policy 0, policy_version 1184462 (0.0009) [2023-12-27 00:00:04,899][105692] Updated weights for policy 0, policy_version 1184472 (0.0009) [2023-12-27 00:00:04,957][105620] Updated weights for policy 1, policy_version 1185743 (0.0009) [2023-12-27 00:00:04,963][105692] Updated weights for policy 0, policy_version 1184482 (0.0007) [2023-12-27 00:00:05,022][105620] Updated weights for policy 1, policy_version 1185753 (0.0010) [2023-12-27 00:00:05,091][105620] Updated weights for policy 1, policy_version 1185763 (0.0010) [2023-12-27 00:00:05,716][105692] Updated weights for policy 0, policy_version 1184492 (0.0008) [2023-12-27 00:00:05,779][105692] Updated weights for policy 0, policy_version 1184502 (0.0008) [2023-12-27 00:00:05,849][105692] Updated weights for policy 0, policy_version 1184512 (0.0009) [2023-12-27 00:00:05,886][105620] Updated weights for policy 1, policy_version 1185773 (0.0009) [2023-12-27 00:00:05,951][105620] Updated weights for policy 1, policy_version 1185783 (0.0010) [2023-12-27 00:00:06,008][105620] Updated weights for policy 1, policy_version 1185793 (0.0009) [2023-12-27 00:00:06,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 606887936. Throughput: 0: 9388.3, 1: 9882.3. Samples: 606874088. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:00:06,063][104569] Avg episode reward: [(0, '9121.916'), (1, '9352.049')] [2023-12-27 00:00:06,718][105692] Updated weights for policy 0, policy_version 1184522 (0.0008) [2023-12-27 00:00:06,790][105692] Updated weights for policy 0, policy_version 1184532 (0.0009) [2023-12-27 00:00:06,848][105620] Updated weights for policy 1, policy_version 1185803 (0.0008) [2023-12-27 00:00:06,862][105692] Updated weights for policy 0, policy_version 1184542 (0.0009) [2023-12-27 00:00:06,918][105620] Updated weights for policy 1, policy_version 1185813 (0.0007) [2023-12-27 00:00:06,932][105692] Updated weights for policy 0, policy_version 1184552 (0.0008) [2023-12-27 00:00:06,995][105620] Updated weights for policy 1, policy_version 1185823 (0.0010) [2023-12-27 00:00:07,669][105692] Updated weights for policy 0, policy_version 1184562 (0.0010) [2023-12-27 00:00:07,733][105692] Updated weights for policy 0, policy_version 1184572 (0.0009) [2023-12-27 00:00:07,785][105620] Updated weights for policy 1, policy_version 1185833 (0.0008) [2023-12-27 00:00:07,801][105692] Updated weights for policy 0, policy_version 1184582 (0.0009) [2023-12-27 00:00:07,845][105620] Updated weights for policy 1, policy_version 1185843 (0.0008) [2023-12-27 00:00:07,906][105620] Updated weights for policy 1, policy_version 1185853 (0.0010) [2023-12-27 00:00:07,973][105620] Updated weights for policy 1, policy_version 1185863 (0.0011) [2023-12-27 00:00:08,622][105692] Updated weights for policy 0, policy_version 1184592 (0.0009) [2023-12-27 00:00:08,690][105692] Updated weights for policy 0, policy_version 1184602 (0.0009) [2023-12-27 00:00:08,758][105692] Updated weights for policy 0, policy_version 1184612 (0.0007) [2023-12-27 00:00:08,765][105620] Updated weights for policy 1, policy_version 1185873 (0.0008) [2023-12-27 00:00:08,834][105620] Updated weights for policy 1, policy_version 1185883 (0.0008) [2023-12-27 00:00:08,896][105620] Updated weights for policy 1, policy_version 1185893 (0.0007) [2023-12-27 00:00:09,623][105692] Updated weights for policy 0, policy_version 1184622 (0.0008) [2023-12-27 00:00:09,694][105692] Updated weights for policy 0, policy_version 1184632 (0.0008) [2023-12-27 00:00:09,725][105620] Updated weights for policy 1, policy_version 1185903 (0.0008) [2023-12-27 00:00:09,763][105692] Updated weights for policy 0, policy_version 1184642 (0.0007) [2023-12-27 00:00:09,790][105620] Updated weights for policy 1, policy_version 1185913 (0.0008) [2023-12-27 00:00:09,866][105620] Updated weights for policy 1, policy_version 1185923 (0.0012) [2023-12-27 00:00:10,504][105692] Updated weights for policy 0, policy_version 1184652 (0.0007) [2023-12-27 00:00:10,571][105692] Updated weights for policy 0, policy_version 1184662 (0.0009) [2023-12-27 00:00:10,638][105692] Updated weights for policy 0, policy_version 1184672 (0.0009) [2023-12-27 00:00:10,712][105620] Updated weights for policy 1, policy_version 1185933 (0.0008) [2023-12-27 00:00:10,781][105620] Updated weights for policy 1, policy_version 1185943 (0.0008) [2023-12-27 00:00:10,837][105620] Updated weights for policy 1, policy_version 1185953 (0.0007) [2023-12-27 00:00:11,062][104569] Fps is (10 sec: 17203.3, 60 sec: 19114.7, 300 sec: 19494.2). Total num frames: 606969856. Throughput: 0: 9337.6, 1: 9653.8. Samples: 606976864. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:00:11,063][104569] Avg episode reward: [(0, '9171.365'), (1, '9261.414')] [2023-12-27 00:00:11,514][105692] Updated weights for policy 0, policy_version 1184682 (0.0009) [2023-12-27 00:00:11,584][105692] Updated weights for policy 0, policy_version 1184692 (0.0010) [2023-12-27 00:00:11,667][105692] Updated weights for policy 0, policy_version 1184702 (0.0009) [2023-12-27 00:00:11,691][105620] Updated weights for policy 1, policy_version 1185963 (0.0008) [2023-12-27 00:00:11,740][105692] Updated weights for policy 0, policy_version 1184712 (0.0008) [2023-12-27 00:00:11,764][105620] Updated weights for policy 1, policy_version 1185973 (0.0008) [2023-12-27 00:00:11,827][105620] Updated weights for policy 1, policy_version 1185983 (0.0008) [2023-12-27 00:00:12,615][105692] Updated weights for policy 0, policy_version 1184722 (0.0009) [2023-12-27 00:00:12,627][105620] Updated weights for policy 1, policy_version 1185993 (0.0009) [2023-12-27 00:00:12,684][105692] Updated weights for policy 0, policy_version 1184732 (0.0008) [2023-12-27 00:00:12,695][105620] Updated weights for policy 1, policy_version 1186003 (0.0009) [2023-12-27 00:00:12,750][105692] Updated weights for policy 0, policy_version 1184742 (0.0007) [2023-12-27 00:00:12,765][105620] Updated weights for policy 1, policy_version 1186013 (0.0011) [2023-12-27 00:00:12,832][105620] Updated weights for policy 1, policy_version 1186023 (0.0011) [2023-12-27 00:00:13,553][105692] Updated weights for policy 0, policy_version 1184752 (0.0008) [2023-12-27 00:00:13,607][105620] Updated weights for policy 1, policy_version 1186033 (0.0006) [2023-12-27 00:00:13,615][105692] Updated weights for policy 0, policy_version 1184762 (0.0010) [2023-12-27 00:00:13,673][105620] Updated weights for policy 1, policy_version 1186043 (0.0008) [2023-12-27 00:00:13,675][105692] Updated weights for policy 0, policy_version 1184772 (0.0011) [2023-12-27 00:00:13,735][105620] Updated weights for policy 1, policy_version 1186053 (0.0010) [2023-12-27 00:00:14,420][105692] Updated weights for policy 0, policy_version 1184782 (0.0011) [2023-12-27 00:00:14,480][105692] Updated weights for policy 0, policy_version 1184792 (0.0011) [2023-12-27 00:00:14,483][105620] Updated weights for policy 1, policy_version 1186063 (0.0010) [2023-12-27 00:00:14,538][105692] Updated weights for policy 0, policy_version 1184802 (0.0011) [2023-12-27 00:00:14,541][105620] Updated weights for policy 1, policy_version 1186073 (0.0011) [2023-12-27 00:00:14,606][105620] Updated weights for policy 1, policy_version 1186083 (0.0011) [2023-12-27 00:00:15,329][105692] Updated weights for policy 0, policy_version 1184812 (0.0009) [2023-12-27 00:00:15,362][105620] Updated weights for policy 1, policy_version 1186093 (0.0011) [2023-12-27 00:00:15,398][105692] Updated weights for policy 0, policy_version 1184822 (0.0011) [2023-12-27 00:00:15,430][105620] Updated weights for policy 1, policy_version 1186103 (0.0006) [2023-12-27 00:00:15,457][105692] Updated weights for policy 0, policy_version 1184832 (0.0011) [2023-12-27 00:00:15,485][105620] Updated weights for policy 1, policy_version 1186113 (0.0006) [2023-12-27 00:00:16,062][104569] Fps is (10 sec: 16384.0, 60 sec: 18841.6, 300 sec: 19438.6). Total num frames: 607051776. Throughput: 0: 9323.8, 1: 9439.5. Samples: 607027420. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:00:16,063][104569] Avg episode reward: [(0, '8899.239'), (1, '9261.374')] [2023-12-27 00:00:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001184840_303366144.pth... [2023-12-27 00:00:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001186120_303685632.pth... [2023-12-27 00:00:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001183816_303104000.pth [2023-12-27 00:00:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001185064_303415296.pth [2023-12-27 00:00:16,204][105620] Updated weights for policy 1, policy_version 1186123 (0.0007) [2023-12-27 00:00:16,228][105692] Updated weights for policy 0, policy_version 1184842 (0.0011) [2023-12-27 00:00:16,259][105620] Updated weights for policy 1, policy_version 1186133 (0.0007) [2023-12-27 00:00:16,293][105692] Updated weights for policy 0, policy_version 1184852 (0.0011) [2023-12-27 00:00:16,316][105620] Updated weights for policy 1, policy_version 1186143 (0.0006) [2023-12-27 00:00:16,350][105692] Updated weights for policy 0, policy_version 1184862 (0.0011) [2023-12-27 00:00:16,397][105692] Updated weights for policy 0, policy_version 1184872 (0.0011) [2023-12-27 00:00:16,967][105620] Updated weights for policy 1, policy_version 1186153 (0.0006) [2023-12-27 00:00:17,030][105620] Updated weights for policy 1, policy_version 1186163 (0.0008) [2023-12-27 00:00:17,089][105620] Updated weights for policy 1, policy_version 1186173 (0.0009) [2023-12-27 00:00:17,152][105620] Updated weights for policy 1, policy_version 1186183 (0.0008) [2023-12-27 00:00:17,172][105692] Updated weights for policy 0, policy_version 1184882 (0.0010) [2023-12-27 00:00:17,230][105692] Updated weights for policy 0, policy_version 1184892 (0.0008) [2023-12-27 00:00:17,292][105692] Updated weights for policy 0, policy_version 1184902 (0.0008) [2023-12-27 00:00:17,861][105620] Updated weights for policy 1, policy_version 1186193 (0.0008) [2023-12-27 00:00:17,931][105620] Updated weights for policy 1, policy_version 1186203 (0.0009) [2023-12-27 00:00:17,999][105620] Updated weights for policy 1, policy_version 1186213 (0.0009) [2023-12-27 00:00:18,025][105692] Updated weights for policy 0, policy_version 1184912 (0.0007) [2023-12-27 00:00:18,076][105692] Updated weights for policy 0, policy_version 1184922 (0.0006) [2023-12-27 00:00:18,134][105692] Updated weights for policy 0, policy_version 1184932 (0.0007) [2023-12-27 00:00:18,829][105620] Updated weights for policy 1, policy_version 1186223 (0.0007) [2023-12-27 00:00:18,831][105692] Updated weights for policy 0, policy_version 1184942 (0.0011) [2023-12-27 00:00:18,889][105620] Updated weights for policy 1, policy_version 1186233 (0.0007) [2023-12-27 00:00:18,891][105692] Updated weights for policy 0, policy_version 1184952 (0.0010) [2023-12-27 00:00:18,948][105620] Updated weights for policy 1, policy_version 1186243 (0.0007) [2023-12-27 00:00:18,954][105692] Updated weights for policy 0, policy_version 1184962 (0.0009) [2023-12-27 00:00:19,756][105620] Updated weights for policy 1, policy_version 1186253 (0.0006) [2023-12-27 00:00:19,789][105692] Updated weights for policy 0, policy_version 1184972 (0.0009) [2023-12-27 00:00:19,832][105620] Updated weights for policy 1, policy_version 1186263 (0.0007) [2023-12-27 00:00:19,869][105692] Updated weights for policy 0, policy_version 1184982 (0.0008) [2023-12-27 00:00:19,907][105620] Updated weights for policy 1, policy_version 1186273 (0.0008) [2023-12-27 00:00:19,945][105692] Updated weights for policy 0, policy_version 1184992 (0.0007) [2023-12-27 00:00:20,671][105620] Updated weights for policy 1, policy_version 1186283 (0.0008) [2023-12-27 00:00:20,740][105620] Updated weights for policy 1, policy_version 1186293 (0.0008) [2023-12-27 00:00:20,755][105692] Updated weights for policy 0, policy_version 1185002 (0.0007) [2023-12-27 00:00:20,805][105620] Updated weights for policy 1, policy_version 1186303 (0.0007) [2023-12-27 00:00:20,825][105692] Updated weights for policy 0, policy_version 1185012 (0.0007) [2023-12-27 00:00:20,892][105692] Updated weights for policy 0, policy_version 1185022 (0.0008) [2023-12-27 00:00:20,960][105692] Updated weights for policy 0, policy_version 1185032 (0.0008) [2023-12-27 00:00:21,062][104569] Fps is (10 sec: 18022.3, 60 sec: 18978.1, 300 sec: 19438.7). Total num frames: 607150080. Throughput: 0: 9195.3, 1: 9327.3. Samples: 607137904. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:00:21,063][104569] Avg episode reward: [(0, '9082.531'), (1, '9351.860')] [2023-12-27 00:00:21,647][105620] Updated weights for policy 1, policy_version 1186313 (0.0008) [2023-12-27 00:00:21,726][105620] Updated weights for policy 1, policy_version 1186323 (0.0007) [2023-12-27 00:00:21,791][105620] Updated weights for policy 1, policy_version 1186333 (0.0006) [2023-12-27 00:00:21,793][105692] Updated weights for policy 0, policy_version 1185042 (0.0008) [2023-12-27 00:00:21,855][105620] Updated weights for policy 1, policy_version 1186343 (0.0007) [2023-12-27 00:00:21,861][105692] Updated weights for policy 0, policy_version 1185052 (0.0008) [2023-12-27 00:00:21,928][105692] Updated weights for policy 0, policy_version 1185062 (0.0007) [2023-12-27 00:00:22,648][105620] Updated weights for policy 1, policy_version 1186353 (0.0009) [2023-12-27 00:00:22,709][105620] Updated weights for policy 1, policy_version 1186363 (0.0009) [2023-12-27 00:00:22,737][105692] Updated weights for policy 0, policy_version 1185072 (0.0010) [2023-12-27 00:00:22,768][105620] Updated weights for policy 1, policy_version 1186373 (0.0009) [2023-12-27 00:00:22,797][105692] Updated weights for policy 0, policy_version 1185082 (0.0007) [2023-12-27 00:00:22,865][105692] Updated weights for policy 0, policy_version 1185092 (0.0009) [2023-12-27 00:00:23,529][105620] Updated weights for policy 1, policy_version 1186383 (0.0010) [2023-12-27 00:00:23,592][105620] Updated weights for policy 1, policy_version 1186393 (0.0010) [2023-12-27 00:00:23,661][105620] Updated weights for policy 1, policy_version 1186403 (0.0011) [2023-12-27 00:00:23,664][105692] Updated weights for policy 0, policy_version 1185102 (0.0009) [2023-12-27 00:00:23,725][105692] Updated weights for policy 0, policy_version 1185112 (0.0010) [2023-12-27 00:00:23,786][105692] Updated weights for policy 0, policy_version 1185122 (0.0011) [2023-12-27 00:00:24,315][105620] Updated weights for policy 1, policy_version 1186413 (0.0011) [2023-12-27 00:00:24,379][105620] Updated weights for policy 1, policy_version 1186423 (0.0009) [2023-12-27 00:00:24,444][105620] Updated weights for policy 1, policy_version 1186433 (0.0008) [2023-12-27 00:00:24,564][105692] Updated weights for policy 0, policy_version 1185132 (0.0011) [2023-12-27 00:00:24,618][105692] Updated weights for policy 0, policy_version 1185142 (0.0011) [2023-12-27 00:00:24,680][105692] Updated weights for policy 0, policy_version 1185152 (0.0011) [2023-12-27 00:00:25,187][105620] Updated weights for policy 1, policy_version 1186443 (0.0007) [2023-12-27 00:00:25,244][105620] Updated weights for policy 1, policy_version 1186453 (0.0008) [2023-12-27 00:00:25,301][105620] Updated weights for policy 1, policy_version 1186463 (0.0008) [2023-12-27 00:00:25,436][105692] Updated weights for policy 0, policy_version 1185162 (0.0011) [2023-12-27 00:00:25,502][105692] Updated weights for policy 0, policy_version 1185172 (0.0011) [2023-12-27 00:00:25,559][105692] Updated weights for policy 0, policy_version 1185182 (0.0009) [2023-12-27 00:00:25,622][105692] Updated weights for policy 0, policy_version 1185192 (0.0009) [2023-12-27 00:00:26,062][104569] Fps is (10 sec: 18022.6, 60 sec: 18568.5, 300 sec: 19383.1). Total num frames: 607232000. Throughput: 0: 8964.1, 1: 9246.2. Samples: 607243580. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:00:26,062][104569] Avg episode reward: [(0, '9081.420'), (1, '9260.989')] [2023-12-27 00:00:26,109][105620] Updated weights for policy 1, policy_version 1186473 (0.0008) [2023-12-27 00:00:26,175][105620] Updated weights for policy 1, policy_version 1186483 (0.0009) [2023-12-27 00:00:26,242][105620] Updated weights for policy 1, policy_version 1186493 (0.0009) [2023-12-27 00:00:26,307][105620] Updated weights for policy 1, policy_version 1186503 (0.0007) [2023-12-27 00:00:26,310][105692] Updated weights for policy 0, policy_version 1185202 (0.0008) [2023-12-27 00:00:26,369][105692] Updated weights for policy 0, policy_version 1185212 (0.0009) [2023-12-27 00:00:26,422][105692] Updated weights for policy 0, policy_version 1185222 (0.0009) [2023-12-27 00:00:27,046][105692] Updated weights for policy 0, policy_version 1185232 (0.0007) [2023-12-27 00:00:27,105][105692] Updated weights for policy 0, policy_version 1185242 (0.0009) [2023-12-27 00:00:27,157][105692] Updated weights for policy 0, policy_version 1185252 (0.0008) [2023-12-27 00:00:27,164][105620] Updated weights for policy 1, policy_version 1186513 (0.0007) [2023-12-27 00:00:27,218][105620] Updated weights for policy 1, policy_version 1186523 (0.0008) [2023-12-27 00:00:27,277][105620] Updated weights for policy 1, policy_version 1186533 (0.0010) [2023-12-27 00:00:27,939][105692] Updated weights for policy 0, policy_version 1185262 (0.0008) [2023-12-27 00:00:27,991][105692] Updated weights for policy 0, policy_version 1185272 (0.0009) [2023-12-27 00:00:28,052][105692] Updated weights for policy 0, policy_version 1185282 (0.0008) [2023-12-27 00:00:28,072][105620] Updated weights for policy 1, policy_version 1186543 (0.0008) [2023-12-27 00:00:28,136][105620] Updated weights for policy 1, policy_version 1186553 (0.0008) [2023-12-27 00:00:28,188][105620] Updated weights for policy 1, policy_version 1186563 (0.0009) [2023-12-27 00:00:28,790][105692] Updated weights for policy 0, policy_version 1185292 (0.0008) [2023-12-27 00:00:28,855][105692] Updated weights for policy 0, policy_version 1185302 (0.0008) [2023-12-27 00:00:28,905][105620] Updated weights for policy 1, policy_version 1186573 (0.0008) [2023-12-27 00:00:28,913][105692] Updated weights for policy 0, policy_version 1185312 (0.0007) [2023-12-27 00:00:28,975][105620] Updated weights for policy 1, policy_version 1186583 (0.0009) [2023-12-27 00:00:29,048][105620] Updated weights for policy 1, policy_version 1186593 (0.0009) [2023-12-27 00:00:29,602][105692] Updated weights for policy 0, policy_version 1185322 (0.0007) [2023-12-27 00:00:29,668][105692] Updated weights for policy 0, policy_version 1185332 (0.0006) [2023-12-27 00:00:29,723][105692] Updated weights for policy 0, policy_version 1185342 (0.0008) [2023-12-27 00:00:29,753][105620] Updated weights for policy 1, policy_version 1186603 (0.0006) [2023-12-27 00:00:29,781][105692] Updated weights for policy 0, policy_version 1185352 (0.0008) [2023-12-27 00:00:29,818][105620] Updated weights for policy 1, policy_version 1186613 (0.0009) [2023-12-27 00:00:29,889][105620] Updated weights for policy 1, policy_version 1186623 (0.0007) [2023-12-27 00:00:30,590][105692] Updated weights for policy 0, policy_version 1185362 (0.0009) [2023-12-27 00:00:30,651][105620] Updated weights for policy 1, policy_version 1186633 (0.0009) [2023-12-27 00:00:30,653][105692] Updated weights for policy 0, policy_version 1185372 (0.0009) [2023-12-27 00:00:30,713][105620] Updated weights for policy 1, policy_version 1186643 (0.0007) [2023-12-27 00:00:30,716][105692] Updated weights for policy 0, policy_version 1185382 (0.0007) [2023-12-27 00:00:30,772][105620] Updated weights for policy 1, policy_version 1186653 (0.0009) [2023-12-27 00:00:30,827][105620] Updated weights for policy 1, policy_version 1186663 (0.0009) [2023-12-27 00:00:31,062][104569] Fps is (10 sec: 18022.5, 60 sec: 18568.5, 300 sec: 19383.1). Total num frames: 607330304. Throughput: 0: 9013.0, 1: 9201.5. Samples: 607299964. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:00:31,063][104569] Avg episode reward: [(0, '9263.443'), (1, '9077.837')] [2023-12-27 00:00:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001185384_303505408.pth... [2023-12-27 00:00:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001186664_303824896.pth... [2023-12-27 00:00:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001184360_303243264.pth [2023-12-27 00:00:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001185640_303562752.pth [2023-12-27 00:00:31,543][105692] Updated weights for policy 0, policy_version 1185392 (0.0006) [2023-12-27 00:00:31,596][105692] Updated weights for policy 0, policy_version 1185402 (0.0010) [2023-12-27 00:00:31,657][105620] Updated weights for policy 1, policy_version 1186673 (0.0008) [2023-12-27 00:00:31,661][105692] Updated weights for policy 0, policy_version 1185412 (0.0010) [2023-12-27 00:00:31,726][105620] Updated weights for policy 1, policy_version 1186683 (0.0010) [2023-12-27 00:00:31,781][105620] Updated weights for policy 1, policy_version 1186693 (0.0007) [2023-12-27 00:00:32,412][105692] Updated weights for policy 0, policy_version 1185422 (0.0010) [2023-12-27 00:00:32,469][105692] Updated weights for policy 0, policy_version 1185432 (0.0007) [2023-12-27 00:00:32,479][105620] Updated weights for policy 1, policy_version 1186703 (0.0007) [2023-12-27 00:00:32,524][105692] Updated weights for policy 0, policy_version 1185442 (0.0007) [2023-12-27 00:00:32,538][105620] Updated weights for policy 1, policy_version 1186713 (0.0009) [2023-12-27 00:00:32,598][105620] Updated weights for policy 1, policy_version 1186723 (0.0011) [2023-12-27 00:00:33,315][105692] Updated weights for policy 0, policy_version 1185452 (0.0007) [2023-12-27 00:00:33,361][105620] Updated weights for policy 1, policy_version 1186733 (0.0010) [2023-12-27 00:00:33,376][105692] Updated weights for policy 0, policy_version 1185462 (0.0006) [2023-12-27 00:00:33,419][105620] Updated weights for policy 1, policy_version 1186743 (0.0009) [2023-12-27 00:00:33,426][105692] Updated weights for policy 0, policy_version 1185472 (0.0006) [2023-12-27 00:00:33,474][105620] Updated weights for policy 1, policy_version 1186753 (0.0008) [2023-12-27 00:00:34,119][105620] Updated weights for policy 1, policy_version 1186763 (0.0009) [2023-12-27 00:00:34,190][105620] Updated weights for policy 1, policy_version 1186773 (0.0008) [2023-12-27 00:00:34,239][105620] Updated weights for policy 1, policy_version 1186783 (0.0008) [2023-12-27 00:00:34,282][105692] Updated weights for policy 0, policy_version 1185482 (0.0006) [2023-12-27 00:00:34,352][105692] Updated weights for policy 0, policy_version 1185492 (0.0009) [2023-12-27 00:00:34,427][105692] Updated weights for policy 0, policy_version 1185502 (0.0008) [2023-12-27 00:00:34,494][105692] Updated weights for policy 0, policy_version 1185512 (0.0009) [2023-12-27 00:00:34,913][105620] Updated weights for policy 1, policy_version 1186793 (0.0009) [2023-12-27 00:00:34,965][105620] Updated weights for policy 1, policy_version 1186803 (0.0006) [2023-12-27 00:00:35,017][105620] Updated weights for policy 1, policy_version 1186813 (0.0006) [2023-12-27 00:00:35,072][105620] Updated weights for policy 1, policy_version 1186823 (0.0006) [2023-12-27 00:00:35,249][105692] Updated weights for policy 0, policy_version 1185522 (0.0010) [2023-12-27 00:00:35,306][105692] Updated weights for policy 0, policy_version 1185532 (0.0010) [2023-12-27 00:00:35,372][105692] Updated weights for policy 0, policy_version 1185542 (0.0009) [2023-12-27 00:00:35,713][105620] Updated weights for policy 1, policy_version 1186833 (0.0005) [2023-12-27 00:00:35,767][105620] Updated weights for policy 1, policy_version 1186843 (0.0008) [2023-12-27 00:00:35,828][105620] Updated weights for policy 1, policy_version 1186853 (0.0009) [2023-12-27 00:00:36,062][104569] Fps is (10 sec: 18841.5, 60 sec: 18295.5, 300 sec: 19355.3). Total num frames: 607420416. Throughput: 0: 8924.8, 1: 9142.1. Samples: 607410200. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:00:36,063][104569] Avg episode reward: [(0, '9355.805'), (1, '9168.442')] [2023-12-27 00:00:36,164][105692] Updated weights for policy 0, policy_version 1185552 (0.0009) [2023-12-27 00:00:36,233][105692] Updated weights for policy 0, policy_version 1185562 (0.0008) [2023-12-27 00:00:36,299][105692] Updated weights for policy 0, policy_version 1185572 (0.0009) [2023-12-27 00:00:36,589][105620] Updated weights for policy 1, policy_version 1186863 (0.0010) [2023-12-27 00:00:36,653][105620] Updated weights for policy 1, policy_version 1186873 (0.0008) [2023-12-27 00:00:36,720][105620] Updated weights for policy 1, policy_version 1186883 (0.0008) [2023-12-27 00:00:37,088][105692] Updated weights for policy 0, policy_version 1185582 (0.0009) [2023-12-27 00:00:37,154][105692] Updated weights for policy 0, policy_version 1185592 (0.0007) [2023-12-27 00:00:37,228][105692] Updated weights for policy 0, policy_version 1185602 (0.0006) [2023-12-27 00:00:37,535][105620] Updated weights for policy 1, policy_version 1186893 (0.0008) [2023-12-27 00:00:37,603][105620] Updated weights for policy 1, policy_version 1186903 (0.0008) [2023-12-27 00:00:37,669][105620] Updated weights for policy 1, policy_version 1186913 (0.0006) [2023-12-27 00:00:37,901][105692] Updated weights for policy 0, policy_version 1185612 (0.0007) [2023-12-27 00:00:37,970][105692] Updated weights for policy 0, policy_version 1185622 (0.0009) [2023-12-27 00:00:38,028][105692] Updated weights for policy 0, policy_version 1185632 (0.0008) [2023-12-27 00:00:38,065][105585] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000007 [2023-12-27 00:00:38,414][105620] Updated weights for policy 1, policy_version 1186923 (0.0007) [2023-12-27 00:00:38,483][105620] Updated weights for policy 1, policy_version 1186933 (0.0008) [2023-12-27 00:00:38,540][105620] Updated weights for policy 1, policy_version 1186943 (0.0010) [2023-12-27 00:00:38,813][105692] Updated weights for policy 0, policy_version 1185642 (0.0009) [2023-12-27 00:00:38,875][105692] Updated weights for policy 0, policy_version 1185652 (0.0009) [2023-12-27 00:00:38,938][105692] Updated weights for policy 0, policy_version 1185662 (0.0009) [2023-12-27 00:00:38,994][105692] Updated weights for policy 0, policy_version 1185672 (0.0009) [2023-12-27 00:00:39,330][105620] Updated weights for policy 1, policy_version 1186953 (0.0009) [2023-12-27 00:00:39,401][105620] Updated weights for policy 1, policy_version 1186963 (0.0008) [2023-12-27 00:00:39,476][105620] Updated weights for policy 1, policy_version 1186973 (0.0009) [2023-12-27 00:00:39,541][105620] Updated weights for policy 1, policy_version 1186983 (0.0009) [2023-12-27 00:00:39,806][105692] Updated weights for policy 0, policy_version 1185682 (0.0008) [2023-12-27 00:00:39,877][105692] Updated weights for policy 0, policy_version 1185692 (0.0008) [2023-12-27 00:00:39,946][105692] Updated weights for policy 0, policy_version 1185702 (0.0008) [2023-12-27 00:00:40,348][105620] Updated weights for policy 1, policy_version 1186993 (0.0008) [2023-12-27 00:00:40,412][105620] Updated weights for policy 1, policy_version 1187003 (0.0010) [2023-12-27 00:00:40,480][105620] Updated weights for policy 1, policy_version 1187013 (0.0009) [2023-12-27 00:00:40,741][105692] Updated weights for policy 0, policy_version 1185712 (0.0009) [2023-12-27 00:00:40,793][105692] Updated weights for policy 0, policy_version 1185722 (0.0009) [2023-12-27 00:00:40,850][105692] Updated weights for policy 0, policy_version 1185732 (0.0009) [2023-12-27 00:00:41,062][104569] Fps is (10 sec: 18022.4, 60 sec: 18295.4, 300 sec: 19327.6). Total num frames: 607510528. Throughput: 0: 8863.9, 1: 9048.5. Samples: 607518344. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:00:41,063][104569] Avg episode reward: [(0, '9356.471'), (1, '9351.377')] [2023-12-27 00:00:41,309][105620] Updated weights for policy 1, policy_version 1187023 (0.0009) [2023-12-27 00:00:41,392][105620] Updated weights for policy 1, policy_version 1187033 (0.0009) [2023-12-27 00:00:41,464][105620] Updated weights for policy 1, policy_version 1187043 (0.0009) [2023-12-27 00:00:41,731][105692] Updated weights for policy 0, policy_version 1185742 (0.0008) [2023-12-27 00:00:41,803][105692] Updated weights for policy 0, policy_version 1185752 (0.0009) [2023-12-27 00:00:41,870][105692] Updated weights for policy 0, policy_version 1185762 (0.0010) [2023-12-27 00:00:42,238][105620] Updated weights for policy 1, policy_version 1187053 (0.0008) [2023-12-27 00:00:42,309][105620] Updated weights for policy 1, policy_version 1187063 (0.0008) [2023-12-27 00:00:42,386][105620] Updated weights for policy 1, policy_version 1187073 (0.0008) [2023-12-27 00:00:42,696][105692] Updated weights for policy 0, policy_version 1185772 (0.0009) [2023-12-27 00:00:42,768][105692] Updated weights for policy 0, policy_version 1185782 (0.0009) [2023-12-27 00:00:42,830][105692] Updated weights for policy 0, policy_version 1185792 (0.0009) [2023-12-27 00:00:43,158][105620] Updated weights for policy 1, policy_version 1187083 (0.0009) [2023-12-27 00:00:43,214][105620] Updated weights for policy 1, policy_version 1187093 (0.0009) [2023-12-27 00:00:43,267][105620] Updated weights for policy 1, policy_version 1187103 (0.0008) [2023-12-27 00:00:43,591][105692] Updated weights for policy 0, policy_version 1185802 (0.0006) [2023-12-27 00:00:43,652][105692] Updated weights for policy 0, policy_version 1185812 (0.0008) [2023-12-27 00:00:43,720][105692] Updated weights for policy 0, policy_version 1185822 (0.0008) [2023-12-27 00:00:43,751][105585] KL-divergence is very high: 100.1045 [2023-12-27 00:00:43,781][105692] Updated weights for policy 0, policy_version 1185832 (0.0006) [2023-12-27 00:00:43,966][105620] Updated weights for policy 1, policy_version 1187113 (0.0006) [2023-12-27 00:00:44,029][105620] Updated weights for policy 1, policy_version 1187123 (0.0009) [2023-12-27 00:00:44,092][105620] Updated weights for policy 1, policy_version 1187133 (0.0009) [2023-12-27 00:00:44,154][105620] Updated weights for policy 1, policy_version 1187143 (0.0009) [2023-12-27 00:00:44,489][105692] Updated weights for policy 0, policy_version 1185842 (0.0007) [2023-12-27 00:00:44,546][105692] Updated weights for policy 0, policy_version 1185852 (0.0006) [2023-12-27 00:00:44,604][105692] Updated weights for policy 0, policy_version 1185862 (0.0008) [2023-12-27 00:00:44,957][105620] Updated weights for policy 1, policy_version 1187153 (0.0009) [2023-12-27 00:00:45,022][105620] Updated weights for policy 1, policy_version 1187163 (0.0009) [2023-12-27 00:00:45,075][105620] Updated weights for policy 1, policy_version 1187173 (0.0010) [2023-12-27 00:00:45,317][105692] Updated weights for policy 0, policy_version 1185872 (0.0010) [2023-12-27 00:00:45,382][105692] Updated weights for policy 0, policy_version 1185882 (0.0011) [2023-12-27 00:00:45,443][105692] Updated weights for policy 0, policy_version 1185892 (0.0011) [2023-12-27 00:00:45,846][105620] Updated weights for policy 1, policy_version 1187183 (0.0007) [2023-12-27 00:00:45,920][105620] Updated weights for policy 1, policy_version 1187193 (0.0006) [2023-12-27 00:00:45,984][105620] Updated weights for policy 1, policy_version 1187203 (0.0010) [2023-12-27 00:00:46,062][104569] Fps is (10 sec: 18022.5, 60 sec: 18159.0, 300 sec: 19272.0). Total num frames: 607600640. Throughput: 0: 8790.5, 1: 9006.5. Samples: 607570752. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:00:46,062][104569] Avg episode reward: [(0, '9356.851'), (1, '9351.242')] [2023-12-27 00:00:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001185896_303636480.pth... [2023-12-27 00:00:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001187208_303964160.pth... [2023-12-27 00:00:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001184840_303366144.pth [2023-12-27 00:00:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001186120_303685632.pth [2023-12-27 00:00:46,252][105692] Updated weights for policy 0, policy_version 1185902 (0.0010) [2023-12-27 00:00:46,316][105692] Updated weights for policy 0, policy_version 1185912 (0.0009) [2023-12-27 00:00:46,384][105692] Updated weights for policy 0, policy_version 1185922 (0.0009) [2023-12-27 00:00:46,772][105620] Updated weights for policy 1, policy_version 1187213 (0.0009) [2023-12-27 00:00:46,835][105620] Updated weights for policy 1, policy_version 1187223 (0.0008) [2023-12-27 00:00:46,895][105620] Updated weights for policy 1, policy_version 1187233 (0.0007) [2023-12-27 00:00:47,124][105692] Updated weights for policy 0, policy_version 1185932 (0.0008) [2023-12-27 00:00:47,187][105692] Updated weights for policy 0, policy_version 1185942 (0.0007) [2023-12-27 00:00:47,251][105692] Updated weights for policy 0, policy_version 1185952 (0.0009) [2023-12-27 00:00:47,623][105620] Updated weights for policy 1, policy_version 1187243 (0.0008) [2023-12-27 00:00:47,681][105620] Updated weights for policy 1, policy_version 1187253 (0.0008) [2023-12-27 00:00:47,745][105620] Updated weights for policy 1, policy_version 1187263 (0.0009) [2023-12-27 00:00:47,981][105692] Updated weights for policy 0, policy_version 1185962 (0.0007) [2023-12-27 00:00:48,050][105692] Updated weights for policy 0, policy_version 1185972 (0.0007) [2023-12-27 00:00:48,110][105692] Updated weights for policy 0, policy_version 1185982 (0.0006) [2023-12-27 00:00:48,172][105692] Updated weights for policy 0, policy_version 1185992 (0.0007) [2023-12-27 00:00:48,584][105620] Updated weights for policy 1, policy_version 1187273 (0.0009) [2023-12-27 00:00:48,651][105620] Updated weights for policy 1, policy_version 1187283 (0.0009) [2023-12-27 00:00:48,721][105620] Updated weights for policy 1, policy_version 1187293 (0.0009) [2023-12-27 00:00:48,787][105620] Updated weights for policy 1, policy_version 1187303 (0.0009) [2023-12-27 00:00:48,901][105692] Updated weights for policy 0, policy_version 1186002 (0.0010) [2023-12-27 00:00:48,953][105692] Updated weights for policy 0, policy_version 1186012 (0.0009) [2023-12-27 00:00:49,017][105692] Updated weights for policy 0, policy_version 1186022 (0.0009) [2023-12-27 00:00:49,571][105620] Updated weights for policy 1, policy_version 1187313 (0.0010) [2023-12-27 00:00:49,638][105620] Updated weights for policy 1, policy_version 1187323 (0.0009) [2023-12-27 00:00:49,702][105620] Updated weights for policy 1, policy_version 1187333 (0.0008) [2023-12-27 00:00:49,867][105692] Updated weights for policy 0, policy_version 1186032 (0.0010) [2023-12-27 00:00:49,936][105692] Updated weights for policy 0, policy_version 1186042 (0.0011) [2023-12-27 00:00:50,009][105692] Updated weights for policy 0, policy_version 1186052 (0.0010) [2023-12-27 00:00:50,560][105620] Updated weights for policy 1, policy_version 1187343 (0.0008) [2023-12-27 00:00:50,633][105620] Updated weights for policy 1, policy_version 1187353 (0.0008) [2023-12-27 00:00:50,709][105620] Updated weights for policy 1, policy_version 1187363 (0.0009) [2023-12-27 00:00:50,820][105692] Updated weights for policy 0, policy_version 1186062 (0.0008) [2023-12-27 00:00:50,894][105692] Updated weights for policy 0, policy_version 1186072 (0.0011) [2023-12-27 00:00:50,970][105692] Updated weights for policy 0, policy_version 1186082 (0.0011) [2023-12-27 00:00:51,062][104569] Fps is (10 sec: 18022.2, 60 sec: 18158.9, 300 sec: 19272.0). Total num frames: 607690752. Throughput: 0: 8902.8, 1: 8982.1. Samples: 607678912. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:00:51,063][104569] Avg episode reward: [(0, '9356.650'), (1, '9259.773')] [2023-12-27 00:00:51,589][105620] Updated weights for policy 1, policy_version 1187373 (0.0008) [2023-12-27 00:00:51,663][105620] Updated weights for policy 1, policy_version 1187383 (0.0010) [2023-12-27 00:00:51,739][105620] Updated weights for policy 1, policy_version 1187393 (0.0008) [2023-12-27 00:00:51,877][105692] Updated weights for policy 0, policy_version 1186092 (0.0010) [2023-12-27 00:00:51,947][105692] Updated weights for policy 0, policy_version 1186102 (0.0009) [2023-12-27 00:00:52,021][105692] Updated weights for policy 0, policy_version 1186112 (0.0009) [2023-12-27 00:00:52,620][105620] Updated weights for policy 1, policy_version 1187403 (0.0008) [2023-12-27 00:00:52,692][105620] Updated weights for policy 1, policy_version 1187413 (0.0006) [2023-12-27 00:00:52,761][105620] Updated weights for policy 1, policy_version 1187423 (0.0008) [2023-12-27 00:00:52,853][105692] Updated weights for policy 0, policy_version 1186122 (0.0009) [2023-12-27 00:00:52,914][105692] Updated weights for policy 0, policy_version 1186132 (0.0009) [2023-12-27 00:00:52,974][105692] Updated weights for policy 0, policy_version 1186142 (0.0009) [2023-12-27 00:00:53,030][105692] Updated weights for policy 0, policy_version 1186152 (0.0009) [2023-12-27 00:00:53,485][105620] Updated weights for policy 1, policy_version 1187433 (0.0009) [2023-12-27 00:00:53,551][105620] Updated weights for policy 1, policy_version 1187443 (0.0009) [2023-12-27 00:00:53,618][105620] Updated weights for policy 1, policy_version 1187453 (0.0009) [2023-12-27 00:00:53,677][105620] Updated weights for policy 1, policy_version 1187463 (0.0009) [2023-12-27 00:00:53,826][105692] Updated weights for policy 0, policy_version 1186162 (0.0009) [2023-12-27 00:00:53,886][105692] Updated weights for policy 0, policy_version 1186172 (0.0009) [2023-12-27 00:00:53,939][105692] Updated weights for policy 0, policy_version 1186182 (0.0009) [2023-12-27 00:00:54,459][105620] Updated weights for policy 1, policy_version 1187473 (0.0010) [2023-12-27 00:00:54,519][105620] Updated weights for policy 1, policy_version 1187483 (0.0009) [2023-12-27 00:00:54,584][105620] Updated weights for policy 1, policy_version 1187493 (0.0010) [2023-12-27 00:00:54,748][105692] Updated weights for policy 0, policy_version 1186192 (0.0007) [2023-12-27 00:00:54,820][105692] Updated weights for policy 0, policy_version 1186202 (0.0010) [2023-12-27 00:00:54,883][105692] Updated weights for policy 0, policy_version 1186212 (0.0009) [2023-12-27 00:00:55,407][105620] Updated weights for policy 1, policy_version 1187503 (0.0008) [2023-12-27 00:00:55,470][105620] Updated weights for policy 1, policy_version 1187513 (0.0010) [2023-12-27 00:00:55,533][105620] Updated weights for policy 1, policy_version 1187523 (0.0009) [2023-12-27 00:00:55,633][105692] Updated weights for policy 0, policy_version 1186222 (0.0007) [2023-12-27 00:00:55,698][105692] Updated weights for policy 0, policy_version 1186232 (0.0008) [2023-12-27 00:00:55,765][105692] Updated weights for policy 0, policy_version 1186242 (0.0008) [2023-12-27 00:00:56,062][104569] Fps is (10 sec: 17203.2, 60 sec: 17885.9, 300 sec: 19216.5). Total num frames: 607772672. Throughput: 0: 8881.5, 1: 8977.1. Samples: 607780500. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:00:56,062][104569] Avg episode reward: [(0, '9356.379'), (1, '9259.287')] [2023-12-27 00:00:56,335][105620] Updated weights for policy 1, policy_version 1187533 (0.0008) [2023-12-27 00:00:56,399][105620] Updated weights for policy 1, policy_version 1187543 (0.0009) [2023-12-27 00:00:56,464][105620] Updated weights for policy 1, policy_version 1187553 (0.0009) [2023-12-27 00:00:56,510][105692] Updated weights for policy 0, policy_version 1186252 (0.0007) [2023-12-27 00:00:56,563][105692] Updated weights for policy 0, policy_version 1186262 (0.0009) [2023-12-27 00:00:56,588][105585] KL-divergence is very high: 112.4009 [2023-12-27 00:00:56,616][105692] Updated weights for policy 0, policy_version 1186272 (0.0009) [2023-12-27 00:00:56,629][105585] KL-divergence is very high: 117.0756 [2023-12-27 00:00:57,276][105620] Updated weights for policy 1, policy_version 1187563 (0.0009) [2023-12-27 00:00:57,334][105620] Updated weights for policy 1, policy_version 1187573 (0.0008) [2023-12-27 00:00:57,362][105692] Updated weights for policy 0, policy_version 1186282 (0.0009) [2023-12-27 00:00:57,402][105620] Updated weights for policy 1, policy_version 1187583 (0.0007) [2023-12-27 00:00:57,428][105692] Updated weights for policy 0, policy_version 1186292 (0.0011) [2023-12-27 00:00:57,496][105692] Updated weights for policy 0, policy_version 1186302 (0.0008) [2023-12-27 00:00:57,553][105692] Updated weights for policy 0, policy_version 1186312 (0.0011) [2023-12-27 00:00:58,188][105620] Updated weights for policy 1, policy_version 1187593 (0.0009) [2023-12-27 00:00:58,259][105620] Updated weights for policy 1, policy_version 1187603 (0.0008) [2023-12-27 00:00:58,332][105620] Updated weights for policy 1, policy_version 1187613 (0.0008) [2023-12-27 00:00:58,388][105692] Updated weights for policy 0, policy_version 1186322 (0.0009) [2023-12-27 00:00:58,404][105620] Updated weights for policy 1, policy_version 1187623 (0.0009) [2023-12-27 00:00:58,458][105692] Updated weights for policy 0, policy_version 1186332 (0.0009) [2023-12-27 00:00:58,531][105692] Updated weights for policy 0, policy_version 1186342 (0.0009) [2023-12-27 00:00:59,454][105620] Updated weights for policy 1, policy_version 1187633 (0.0010) [2023-12-27 00:00:59,525][105692] Updated weights for policy 0, policy_version 1186352 (0.0009) [2023-12-27 00:00:59,527][105620] Updated weights for policy 1, policy_version 1187643 (0.0009) [2023-12-27 00:00:59,600][105692] Updated weights for policy 0, policy_version 1186362 (0.0010) [2023-12-27 00:00:59,601][105620] Updated weights for policy 1, policy_version 1187653 (0.0009) [2023-12-27 00:00:59,666][105692] Updated weights for policy 0, policy_version 1186372 (0.0011) [2023-12-27 00:01:00,386][105620] Updated weights for policy 1, policy_version 1187663 (0.0011) [2023-12-27 00:01:00,456][105620] Updated weights for policy 1, policy_version 1187673 (0.0010) [2023-12-27 00:01:00,457][105692] Updated weights for policy 0, policy_version 1186382 (0.0010) [2023-12-27 00:01:00,525][105620] Updated weights for policy 1, policy_version 1187683 (0.0010) [2023-12-27 00:01:00,526][105692] Updated weights for policy 0, policy_version 1186392 (0.0011) [2023-12-27 00:01:00,590][105692] Updated weights for policy 0, policy_version 1186402 (0.0011) [2023-12-27 00:01:01,062][104569] Fps is (10 sec: 16384.1, 60 sec: 17612.8, 300 sec: 19160.9). Total num frames: 607854592. Throughput: 0: 8931.6, 1: 8954.8. Samples: 607832308. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:01:01,063][104569] Avg episode reward: [(0, '9355.718'), (1, '9259.171')] [2023-12-27 00:01:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001186408_303767552.pth... [2023-12-27 00:01:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001187688_304087040.pth... [2023-12-27 00:01:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001185384_303505408.pth [2023-12-27 00:01:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001186664_303824896.pth [2023-12-27 00:01:01,283][105620] Updated weights for policy 1, policy_version 1187693 (0.0011) [2023-12-27 00:01:01,349][105620] Updated weights for policy 1, policy_version 1187703 (0.0010) [2023-12-27 00:01:01,430][105620] Updated weights for policy 1, policy_version 1187713 (0.0009) [2023-12-27 00:01:01,437][105692] Updated weights for policy 0, policy_version 1186412 (0.0009) [2023-12-27 00:01:01,511][105692] Updated weights for policy 0, policy_version 1186422 (0.0008) [2023-12-27 00:01:01,575][105692] Updated weights for policy 0, policy_version 1186432 (0.0008) [2023-12-27 00:01:02,190][105620] Updated weights for policy 1, policy_version 1187723 (0.0008) [2023-12-27 00:01:02,260][105620] Updated weights for policy 1, policy_version 1187733 (0.0009) [2023-12-27 00:01:02,320][105620] Updated weights for policy 1, policy_version 1187743 (0.0009) [2023-12-27 00:01:02,370][105692] Updated weights for policy 0, policy_version 1186442 (0.0010) [2023-12-27 00:01:02,440][105692] Updated weights for policy 0, policy_version 1186452 (0.0009) [2023-12-27 00:01:02,503][105692] Updated weights for policy 0, policy_version 1186462 (0.0008) [2023-12-27 00:01:02,559][105692] Updated weights for policy 0, policy_version 1186472 (0.0009) [2023-12-27 00:01:03,135][105620] Updated weights for policy 1, policy_version 1187753 (0.0010) [2023-12-27 00:01:03,205][105620] Updated weights for policy 1, policy_version 1187763 (0.0009) [2023-12-27 00:01:03,274][105620] Updated weights for policy 1, policy_version 1187773 (0.0008) [2023-12-27 00:01:03,343][105620] Updated weights for policy 1, policy_version 1187783 (0.0009) [2023-12-27 00:01:03,369][105692] Updated weights for policy 0, policy_version 1186482 (0.0009) [2023-12-27 00:01:03,427][105692] Updated weights for policy 0, policy_version 1186492 (0.0009) [2023-12-27 00:01:03,488][105692] Updated weights for policy 0, policy_version 1186502 (0.0009) [2023-12-27 00:01:04,067][105620] Updated weights for policy 1, policy_version 1187793 (0.0008) [2023-12-27 00:01:04,137][105620] Updated weights for policy 1, policy_version 1187803 (0.0007) [2023-12-27 00:01:04,206][105620] Updated weights for policy 1, policy_version 1187813 (0.0009) [2023-12-27 00:01:04,303][105692] Updated weights for policy 0, policy_version 1186512 (0.0009) [2023-12-27 00:01:04,368][105692] Updated weights for policy 0, policy_version 1186522 (0.0009) [2023-12-27 00:01:04,435][105692] Updated weights for policy 0, policy_version 1186532 (0.0009) [2023-12-27 00:01:05,002][105620] Updated weights for policy 1, policy_version 1187823 (0.0009) [2023-12-27 00:01:05,070][105620] Updated weights for policy 1, policy_version 1187833 (0.0009) [2023-12-27 00:01:05,126][105620] Updated weights for policy 1, policy_version 1187843 (0.0009) [2023-12-27 00:01:05,177][105692] Updated weights for policy 0, policy_version 1186542 (0.0007) [2023-12-27 00:01:05,243][105692] Updated weights for policy 0, policy_version 1186552 (0.0010) [2023-12-27 00:01:05,304][105692] Updated weights for policy 0, policy_version 1186562 (0.0009) [2023-12-27 00:01:05,903][105620] Updated weights for policy 1, policy_version 1187853 (0.0009) [2023-12-27 00:01:05,964][105620] Updated weights for policy 1, policy_version 1187863 (0.0008) [2023-12-27 00:01:06,007][105692] Updated weights for policy 0, policy_version 1186572 (0.0008) [2023-12-27 00:01:06,028][105620] Updated weights for policy 1, policy_version 1187873 (0.0008) [2023-12-27 00:01:06,062][104569] Fps is (10 sec: 16383.8, 60 sec: 17476.3, 300 sec: 19133.2). Total num frames: 607936512. Throughput: 0: 8811.5, 1: 8874.3. Samples: 607933764. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:01:06,062][104569] Avg episode reward: [(0, '9354.859'), (1, '9258.680')] [2023-12-27 00:01:06,068][105692] Updated weights for policy 0, policy_version 1186582 (0.0006) [2023-12-27 00:01:06,144][105692] Updated weights for policy 0, policy_version 1186592 (0.0011) [2023-12-27 00:01:06,920][105620] Updated weights for policy 1, policy_version 1187883 (0.0008) [2023-12-27 00:01:06,932][105692] Updated weights for policy 0, policy_version 1186602 (0.0011) [2023-12-27 00:01:06,993][105620] Updated weights for policy 1, policy_version 1187893 (0.0009) [2023-12-27 00:01:07,001][105692] Updated weights for policy 0, policy_version 1186612 (0.0010) [2023-12-27 00:01:07,061][105620] Updated weights for policy 1, policy_version 1187903 (0.0008) [2023-12-27 00:01:07,068][105692] Updated weights for policy 0, policy_version 1186622 (0.0010) [2023-12-27 00:01:07,130][105692] Updated weights for policy 0, policy_version 1186632 (0.0011) [2023-12-27 00:01:07,827][105620] Updated weights for policy 1, policy_version 1187913 (0.0008) [2023-12-27 00:01:07,890][105620] Updated weights for policy 1, policy_version 1187923 (0.0007) [2023-12-27 00:01:07,916][105692] Updated weights for policy 0, policy_version 1186642 (0.0009) [2023-12-27 00:01:07,956][105620] Updated weights for policy 1, policy_version 1187933 (0.0007) [2023-12-27 00:01:07,979][105692] Updated weights for policy 0, policy_version 1186652 (0.0011) [2023-12-27 00:01:08,022][105620] Updated weights for policy 1, policy_version 1187943 (0.0006) [2023-12-27 00:01:08,040][105692] Updated weights for policy 0, policy_version 1186662 (0.0011) [2023-12-27 00:01:08,807][105620] Updated weights for policy 1, policy_version 1187953 (0.0008) [2023-12-27 00:01:08,844][105692] Updated weights for policy 0, policy_version 1186672 (0.0011) [2023-12-27 00:01:08,871][105620] Updated weights for policy 1, policy_version 1187963 (0.0007) [2023-12-27 00:01:08,909][105692] Updated weights for policy 0, policy_version 1186682 (0.0010) [2023-12-27 00:01:08,942][105620] Updated weights for policy 1, policy_version 1187973 (0.0006) [2023-12-27 00:01:08,977][105692] Updated weights for policy 0, policy_version 1186692 (0.0009) [2023-12-27 00:01:09,736][105620] Updated weights for policy 1, policy_version 1187983 (0.0009) [2023-12-27 00:01:09,791][105692] Updated weights for policy 0, policy_version 1186702 (0.0008) [2023-12-27 00:01:09,800][105620] Updated weights for policy 1, policy_version 1187993 (0.0008) [2023-12-27 00:01:09,859][105692] Updated weights for policy 0, policy_version 1186712 (0.0008) [2023-12-27 00:01:09,867][105620] Updated weights for policy 1, policy_version 1188003 (0.0009) [2023-12-27 00:01:09,927][105692] Updated weights for policy 0, policy_version 1186722 (0.0008) [2023-12-27 00:01:10,650][105620] Updated weights for policy 1, policy_version 1188013 (0.0008) [2023-12-27 00:01:10,705][105620] Updated weights for policy 1, policy_version 1188023 (0.0008) [2023-12-27 00:01:10,747][105692] Updated weights for policy 0, policy_version 1186732 (0.0009) [2023-12-27 00:01:10,763][105620] Updated weights for policy 1, policy_version 1188033 (0.0007) [2023-12-27 00:01:10,808][105692] Updated weights for policy 0, policy_version 1186742 (0.0007) [2023-12-27 00:01:10,876][105692] Updated weights for policy 0, policy_version 1186752 (0.0008) [2023-12-27 00:01:11,062][104569] Fps is (10 sec: 18022.6, 60 sec: 17749.4, 300 sec: 19133.2). Total num frames: 608034816. Throughput: 0: 8861.1, 1: 8829.9. Samples: 608039672. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:01:11,062][104569] Avg episode reward: [(0, '9174.235'), (1, '9258.385')] [2023-12-27 00:01:11,580][105620] Updated weights for policy 1, policy_version 1188043 (0.0008) [2023-12-27 00:01:11,650][105620] Updated weights for policy 1, policy_version 1188053 (0.0008) [2023-12-27 00:01:11,714][105692] Updated weights for policy 0, policy_version 1186762 (0.0009) [2023-12-27 00:01:11,716][105620] Updated weights for policy 1, policy_version 1188063 (0.0008) [2023-12-27 00:01:11,786][105692] Updated weights for policy 0, policy_version 1186772 (0.0008) [2023-12-27 00:01:11,816][105585] KL-divergence is very high: 166.0692 [2023-12-27 00:01:11,848][105692] Updated weights for policy 0, policy_version 1186782 (0.0009) [2023-12-27 00:01:11,867][105585] KL-divergence is very high: 201.3039 [2023-12-27 00:01:11,909][105692] Updated weights for policy 0, policy_version 1186792 (0.0010) [2023-12-27 00:01:12,499][105620] Updated weights for policy 1, policy_version 1188073 (0.0009) [2023-12-27 00:01:12,568][105620] Updated weights for policy 1, policy_version 1188083 (0.0010) [2023-12-27 00:01:12,635][105620] Updated weights for policy 1, policy_version 1188093 (0.0010) [2023-12-27 00:01:12,670][105585] KL-divergence is very high: 203.1186 [2023-12-27 00:01:12,677][105585] KL-divergence is very high: 192.9796 [2023-12-27 00:01:12,703][105620] Updated weights for policy 1, policy_version 1188103 (0.0009) [2023-12-27 00:01:12,718][105585] KL-divergence is very high: 180.2376 [2023-12-27 00:01:12,726][105585] KL-divergence is very high: 169.3552 [2023-12-27 00:01:12,732][105692] Updated weights for policy 0, policy_version 1186802 (0.0007) [2023-12-27 00:01:12,772][105585] KL-divergence is very high: 149.7540 [2023-12-27 00:01:12,780][105585] KL-divergence is very high: 139.8006 [2023-12-27 00:01:12,800][105692] Updated weights for policy 0, policy_version 1186812 (0.0009) [2023-12-27 00:01:12,827][105585] KL-divergence is very high: 120.4747 [2023-12-27 00:01:12,833][105585] KL-divergence is very high: 112.5150 [2023-12-27 00:01:12,861][105692] Updated weights for policy 0, policy_version 1186822 (0.0010) [2023-12-27 00:01:13,476][105620] Updated weights for policy 1, policy_version 1188113 (0.0008) [2023-12-27 00:01:13,545][105620] Updated weights for policy 1, policy_version 1188123 (0.0008) [2023-12-27 00:01:13,604][105620] Updated weights for policy 1, policy_version 1188133 (0.0008) [2023-12-27 00:01:13,659][105692] Updated weights for policy 0, policy_version 1186832 (0.0010) [2023-12-27 00:01:13,722][105692] Updated weights for policy 0, policy_version 1186842 (0.0011) [2023-12-27 00:01:13,786][105692] Updated weights for policy 0, policy_version 1186852 (0.0011) [2023-12-27 00:01:14,382][105620] Updated weights for policy 1, policy_version 1188143 (0.0008) [2023-12-27 00:01:14,439][105620] Updated weights for policy 1, policy_version 1188153 (0.0008) [2023-12-27 00:01:14,496][105620] Updated weights for policy 1, policy_version 1188163 (0.0008) [2023-12-27 00:01:14,548][105692] Updated weights for policy 0, policy_version 1186862 (0.0011) [2023-12-27 00:01:14,602][105692] Updated weights for policy 0, policy_version 1186872 (0.0011) [2023-12-27 00:01:14,663][105585] KL-divergence is very high: 126.9735 [2023-12-27 00:01:14,666][105692] Updated weights for policy 0, policy_version 1186882 (0.0011) [2023-12-27 00:01:15,289][105620] Updated weights for policy 1, policy_version 1188173 (0.0010) [2023-12-27 00:01:15,350][105620] Updated weights for policy 1, policy_version 1188183 (0.0011) [2023-12-27 00:01:15,415][105620] Updated weights for policy 1, policy_version 1188193 (0.0011) [2023-12-27 00:01:15,433][105692] Updated weights for policy 0, policy_version 1186892 (0.0011) [2023-12-27 00:01:15,501][105692] Updated weights for policy 0, policy_version 1186902 (0.0011) [2023-12-27 00:01:15,566][105692] Updated weights for policy 0, policy_version 1186912 (0.0011) [2023-12-27 00:01:16,062][104569] Fps is (10 sec: 18022.6, 60 sec: 17749.4, 300 sec: 19077.6). Total num frames: 608116736. Throughput: 0: 8761.1, 1: 8826.7. Samples: 608091412. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:01:16,062][104569] Avg episode reward: [(0, '9174.577'), (1, '9349.605')] [2023-12-27 00:01:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001186920_303898624.pth... [2023-12-27 00:01:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001188200_304218112.pth... [2023-12-27 00:01:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001187208_303964160.pth [2023-12-27 00:01:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001185896_303636480.pth [2023-12-27 00:01:16,200][105620] Updated weights for policy 1, policy_version 1188203 (0.0011) [2023-12-27 00:01:16,257][105620] Updated weights for policy 1, policy_version 1188213 (0.0011) [2023-12-27 00:01:16,269][105692] Updated weights for policy 0, policy_version 1186922 (0.0009) [2023-12-27 00:01:16,318][105620] Updated weights for policy 1, policy_version 1188223 (0.0011) [2023-12-27 00:01:16,331][105692] Updated weights for policy 0, policy_version 1186932 (0.0007) [2023-12-27 00:01:16,392][105692] Updated weights for policy 0, policy_version 1186942 (0.0011) [2023-12-27 00:01:16,453][105692] Updated weights for policy 0, policy_version 1186952 (0.0011) [2023-12-27 00:01:17,093][105620] Updated weights for policy 1, policy_version 1188233 (0.0011) [2023-12-27 00:01:17,154][105620] Updated weights for policy 1, policy_version 1188243 (0.0011) [2023-12-27 00:01:17,188][105692] Updated weights for policy 0, policy_version 1186962 (0.0011) [2023-12-27 00:01:17,215][105620] Updated weights for policy 1, policy_version 1188253 (0.0011) [2023-12-27 00:01:17,248][105692] Updated weights for policy 0, policy_version 1186972 (0.0011) [2023-12-27 00:01:17,279][105620] Updated weights for policy 1, policy_version 1188263 (0.0011) [2023-12-27 00:01:17,309][105692] Updated weights for policy 0, policy_version 1186982 (0.0011) [2023-12-27 00:01:18,069][105620] Updated weights for policy 1, policy_version 1188273 (0.0009) [2023-12-27 00:01:18,080][105692] Updated weights for policy 0, policy_version 1186992 (0.0009) [2023-12-27 00:01:18,140][105620] Updated weights for policy 1, policy_version 1188283 (0.0009) [2023-12-27 00:01:18,146][105692] Updated weights for policy 0, policy_version 1187002 (0.0007) [2023-12-27 00:01:18,205][105620] Updated weights for policy 1, policy_version 1188293 (0.0007) [2023-12-27 00:01:18,213][105692] Updated weights for policy 0, policy_version 1187012 (0.0007) [2023-12-27 00:01:18,985][105620] Updated weights for policy 1, policy_version 1188303 (0.0009) [2023-12-27 00:01:19,053][105620] Updated weights for policy 1, policy_version 1188313 (0.0011) [2023-12-27 00:01:19,061][105692] Updated weights for policy 0, policy_version 1187022 (0.0007) [2023-12-27 00:01:19,111][105620] Updated weights for policy 1, policy_version 1188323 (0.0011) [2023-12-27 00:01:19,118][105692] Updated weights for policy 0, policy_version 1187032 (0.0006) [2023-12-27 00:01:19,174][105692] Updated weights for policy 0, policy_version 1187042 (0.0008) [2023-12-27 00:01:19,907][105620] Updated weights for policy 1, policy_version 1188333 (0.0011) [2023-12-27 00:01:19,980][105620] Updated weights for policy 1, policy_version 1188343 (0.0010) [2023-12-27 00:01:20,033][105692] Updated weights for policy 0, policy_version 1187052 (0.0007) [2023-12-27 00:01:20,051][105620] Updated weights for policy 1, policy_version 1188353 (0.0011) [2023-12-27 00:01:20,097][105692] Updated weights for policy 0, policy_version 1187062 (0.0010) [2023-12-27 00:01:20,167][105692] Updated weights for policy 0, policy_version 1187072 (0.0008) [2023-12-27 00:01:20,856][105620] Updated weights for policy 1, policy_version 1188363 (0.0010) [2023-12-27 00:01:20,927][105620] Updated weights for policy 1, policy_version 1188373 (0.0009) [2023-12-27 00:01:20,987][105692] Updated weights for policy 0, policy_version 1187082 (0.0010) [2023-12-27 00:01:20,989][105620] Updated weights for policy 1, policy_version 1188383 (0.0008) [2023-12-27 00:01:21,060][105692] Updated weights for policy 0, policy_version 1187092 (0.0008) [2023-12-27 00:01:21,062][104569] Fps is (10 sec: 17203.0, 60 sec: 17612.8, 300 sec: 19049.9). Total num frames: 608206848. Throughput: 0: 8778.5, 1: 8748.9. Samples: 608198932. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:01:21,063][104569] Avg episode reward: [(0, '9174.076'), (1, '9258.030')] [2023-12-27 00:01:21,133][105692] Updated weights for policy 0, policy_version 1187102 (0.0009) [2023-12-27 00:01:21,210][105692] Updated weights for policy 0, policy_version 1187112 (0.0009) [2023-12-27 00:01:21,854][105620] Updated weights for policy 1, policy_version 1188393 (0.0010) [2023-12-27 00:01:21,913][105620] Updated weights for policy 1, policy_version 1188403 (0.0009) [2023-12-27 00:01:21,974][105620] Updated weights for policy 1, policy_version 1188413 (0.0009) [2023-12-27 00:01:22,036][105692] Updated weights for policy 0, policy_version 1187122 (0.0007) [2023-12-27 00:01:22,041][105620] Updated weights for policy 1, policy_version 1188423 (0.0008) [2023-12-27 00:01:22,099][105692] Updated weights for policy 0, policy_version 1187132 (0.0009) [2023-12-27 00:01:22,164][105692] Updated weights for policy 0, policy_version 1187142 (0.0009) [2023-12-27 00:01:22,900][105620] Updated weights for policy 1, policy_version 1188433 (0.0008) [2023-12-27 00:01:22,941][105692] Updated weights for policy 0, policy_version 1187152 (0.0009) [2023-12-27 00:01:22,969][105620] Updated weights for policy 1, policy_version 1188443 (0.0009) [2023-12-27 00:01:23,005][105692] Updated weights for policy 0, policy_version 1187162 (0.0007) [2023-12-27 00:01:23,033][105620] Updated weights for policy 1, policy_version 1188453 (0.0007) [2023-12-27 00:01:23,069][105692] Updated weights for policy 0, policy_version 1187172 (0.0008) [2023-12-27 00:01:23,810][105620] Updated weights for policy 1, policy_version 1188463 (0.0008) [2023-12-27 00:01:23,851][105692] Updated weights for policy 0, policy_version 1187182 (0.0008) [2023-12-27 00:01:23,871][105620] Updated weights for policy 1, policy_version 1188473 (0.0007) [2023-12-27 00:01:23,914][105692] Updated weights for policy 0, policy_version 1187192 (0.0007) [2023-12-27 00:01:23,934][105620] Updated weights for policy 1, policy_version 1188483 (0.0006) [2023-12-27 00:01:23,974][105692] Updated weights for policy 0, policy_version 1187202 (0.0010) [2023-12-27 00:01:24,661][105620] Updated weights for policy 1, policy_version 1188493 (0.0008) [2023-12-27 00:01:24,726][105620] Updated weights for policy 1, policy_version 1188503 (0.0008) [2023-12-27 00:01:24,777][105620] Updated weights for policy 1, policy_version 1188513 (0.0005) [2023-12-27 00:01:24,791][105692] Updated weights for policy 0, policy_version 1187212 (0.0009) [2023-12-27 00:01:24,853][105692] Updated weights for policy 0, policy_version 1187222 (0.0009) [2023-12-27 00:01:24,900][105692] Updated weights for policy 0, policy_version 1187232 (0.0008) [2023-12-27 00:01:25,463][105620] Updated weights for policy 1, policy_version 1188523 (0.0006) [2023-12-27 00:01:25,529][105620] Updated weights for policy 1, policy_version 1188533 (0.0011) [2023-12-27 00:01:25,582][105620] Updated weights for policy 1, policy_version 1188543 (0.0011) [2023-12-27 00:01:25,735][105692] Updated weights for policy 0, policy_version 1187242 (0.0007) [2023-12-27 00:01:25,789][105692] Updated weights for policy 0, policy_version 1187252 (0.0005) [2023-12-27 00:01:25,852][105692] Updated weights for policy 0, policy_version 1187262 (0.0007) [2023-12-27 00:01:25,913][105692] Updated weights for policy 0, policy_version 1187272 (0.0009) [2023-12-27 00:01:26,062][104569] Fps is (10 sec: 18022.3, 60 sec: 17749.3, 300 sec: 18994.3). Total num frames: 608296960. Throughput: 0: 8726.5, 1: 8719.1. Samples: 608303396. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:01:26,062][104569] Avg episode reward: [(0, '8906.904'), (1, '9083.747')] [2023-12-27 00:01:26,333][105620] Updated weights for policy 1, policy_version 1188553 (0.0011) [2023-12-27 00:01:26,389][105620] Updated weights for policy 1, policy_version 1188563 (0.0009) [2023-12-27 00:01:26,440][105620] Updated weights for policy 1, policy_version 1188573 (0.0009) [2023-12-27 00:01:26,499][105620] Updated weights for policy 1, policy_version 1188583 (0.0009) [2023-12-27 00:01:26,654][105692] Updated weights for policy 0, policy_version 1187282 (0.0009) [2023-12-27 00:01:26,710][105692] Updated weights for policy 0, policy_version 1187292 (0.0009) [2023-12-27 00:01:26,772][105692] Updated weights for policy 0, policy_version 1187302 (0.0009) [2023-12-27 00:01:27,304][105620] Updated weights for policy 1, policy_version 1188593 (0.0009) [2023-12-27 00:01:27,368][105620] Updated weights for policy 1, policy_version 1188603 (0.0007) [2023-12-27 00:01:27,433][105620] Updated weights for policy 1, policy_version 1188613 (0.0008) [2023-12-27 00:01:27,554][105692] Updated weights for policy 0, policy_version 1187312 (0.0008) [2023-12-27 00:01:27,618][105692] Updated weights for policy 0, policy_version 1187322 (0.0008) [2023-12-27 00:01:27,681][105692] Updated weights for policy 0, policy_version 1187332 (0.0009) [2023-12-27 00:01:28,182][105620] Updated weights for policy 1, policy_version 1188623 (0.0009) [2023-12-27 00:01:28,240][105620] Updated weights for policy 1, policy_version 1188633 (0.0009) [2023-12-27 00:01:28,299][105620] Updated weights for policy 1, policy_version 1188643 (0.0009) [2023-12-27 00:01:28,427][105692] Updated weights for policy 0, policy_version 1187342 (0.0009) [2023-12-27 00:01:28,491][105692] Updated weights for policy 0, policy_version 1187352 (0.0008) [2023-12-27 00:01:28,559][105692] Updated weights for policy 0, policy_version 1187362 (0.0009) [2023-12-27 00:01:29,068][105620] Updated weights for policy 1, policy_version 1188653 (0.0010) [2023-12-27 00:01:29,133][105620] Updated weights for policy 1, policy_version 1188663 (0.0011) [2023-12-27 00:01:29,197][105620] Updated weights for policy 1, policy_version 1188673 (0.0011) [2023-12-27 00:01:29,351][105692] Updated weights for policy 0, policy_version 1187372 (0.0008) [2023-12-27 00:01:29,418][105692] Updated weights for policy 0, policy_version 1187382 (0.0007) [2023-12-27 00:01:29,478][105692] Updated weights for policy 0, policy_version 1187392 (0.0007) [2023-12-27 00:01:30,001][105620] Updated weights for policy 1, policy_version 1188683 (0.0008) [2023-12-27 00:01:30,072][105620] Updated weights for policy 1, policy_version 1188693 (0.0007) [2023-12-27 00:01:30,135][105620] Updated weights for policy 1, policy_version 1188703 (0.0007) [2023-12-27 00:01:30,181][105692] Updated weights for policy 0, policy_version 1187402 (0.0006) [2023-12-27 00:01:30,249][105692] Updated weights for policy 0, policy_version 1187412 (0.0007) [2023-12-27 00:01:30,317][105692] Updated weights for policy 0, policy_version 1187422 (0.0007) [2023-12-27 00:01:30,384][105692] Updated weights for policy 0, policy_version 1187432 (0.0007) [2023-12-27 00:01:30,838][105620] Updated weights for policy 1, policy_version 1188713 (0.0006) [2023-12-27 00:01:30,897][105620] Updated weights for policy 1, policy_version 1188723 (0.0011) [2023-12-27 00:01:30,959][105620] Updated weights for policy 1, policy_version 1188733 (0.0010) [2023-12-27 00:01:31,017][105620] Updated weights for policy 1, policy_version 1188743 (0.0011) [2023-12-27 00:01:31,062][104569] Fps is (10 sec: 18022.5, 60 sec: 17612.8, 300 sec: 18966.6). Total num frames: 608387072. Throughput: 0: 8762.9, 1: 8737.7. Samples: 608358280. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:01:31,062][104569] Avg episode reward: [(0, '8907.541'), (1, '8906.885')] [2023-12-27 00:01:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001188744_304357376.pth... [2023-12-27 00:01:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001187688_304087040.pth [2023-12-27 00:01:31,075][105692] Updated weights for policy 0, policy_version 1187442 (0.0007) [2023-12-27 00:01:31,143][105692] Updated weights for policy 0, policy_version 1187452 (0.0009) [2023-12-27 00:01:31,207][105692] Updated weights for policy 0, policy_version 1187462 (0.0010) [2023-12-27 00:01:31,219][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001187464_304037888.pth... [2023-12-27 00:01:31,224][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001186408_303767552.pth [2023-12-27 00:01:31,807][105620] Updated weights for policy 1, policy_version 1188753 (0.0008) [2023-12-27 00:01:31,874][105620] Updated weights for policy 1, policy_version 1188763 (0.0009) [2023-12-27 00:01:31,943][105620] Updated weights for policy 1, policy_version 1188773 (0.0009) [2023-12-27 00:01:32,065][105692] Updated weights for policy 0, policy_version 1187472 (0.0009) [2023-12-27 00:01:32,132][105692] Updated weights for policy 0, policy_version 1187482 (0.0009) [2023-12-27 00:01:32,195][105692] Updated weights for policy 0, policy_version 1187492 (0.0009) [2023-12-27 00:01:32,729][105620] Updated weights for policy 1, policy_version 1188783 (0.0009) [2023-12-27 00:01:32,796][105620] Updated weights for policy 1, policy_version 1188793 (0.0008) [2023-12-27 00:01:32,864][105620] Updated weights for policy 1, policy_version 1188803 (0.0007) [2023-12-27 00:01:32,961][105692] Updated weights for policy 0, policy_version 1187502 (0.0010) [2023-12-27 00:01:33,023][105692] Updated weights for policy 0, policy_version 1187512 (0.0008) [2023-12-27 00:01:33,089][105692] Updated weights for policy 0, policy_version 1187522 (0.0006) [2023-12-27 00:01:33,615][105620] Updated weights for policy 1, policy_version 1188813 (0.0009) [2023-12-27 00:01:33,672][105620] Updated weights for policy 1, policy_version 1188823 (0.0010) [2023-12-27 00:01:33,722][105620] Updated weights for policy 1, policy_version 1188833 (0.0008) [2023-12-27 00:01:33,739][105692] Updated weights for policy 0, policy_version 1187532 (0.0007) [2023-12-27 00:01:33,788][105692] Updated weights for policy 0, policy_version 1187542 (0.0008) [2023-12-27 00:01:33,844][105692] Updated weights for policy 0, policy_version 1187552 (0.0009) [2023-12-27 00:01:34,563][105620] Updated weights for policy 1, policy_version 1188843 (0.0008) [2023-12-27 00:01:34,635][105620] Updated weights for policy 1, policy_version 1188853 (0.0009) [2023-12-27 00:01:34,646][105692] Updated weights for policy 0, policy_version 1187562 (0.0008) [2023-12-27 00:01:34,702][105620] Updated weights for policy 1, policy_version 1188863 (0.0007) [2023-12-27 00:01:34,708][105692] Updated weights for policy 0, policy_version 1187572 (0.0008) [2023-12-27 00:01:34,778][105692] Updated weights for policy 0, policy_version 1187582 (0.0008) [2023-12-27 00:01:34,843][105692] Updated weights for policy 0, policy_version 1187592 (0.0008) [2023-12-27 00:01:35,523][105620] Updated weights for policy 1, policy_version 1188873 (0.0008) [2023-12-27 00:01:35,589][105620] Updated weights for policy 1, policy_version 1188883 (0.0009) [2023-12-27 00:01:35,620][105692] Updated weights for policy 0, policy_version 1187602 (0.0008) [2023-12-27 00:01:35,652][105620] Updated weights for policy 1, policy_version 1188893 (0.0009) [2023-12-27 00:01:35,671][105692] Updated weights for policy 0, policy_version 1187612 (0.0007) [2023-12-27 00:01:35,707][105620] Updated weights for policy 1, policy_version 1188903 (0.0010) [2023-12-27 00:01:35,728][105692] Updated weights for policy 0, policy_version 1187622 (0.0007) [2023-12-27 00:01:36,062][104569] Fps is (10 sec: 18022.6, 60 sec: 17612.8, 300 sec: 18938.8). Total num frames: 608477184. Throughput: 0: 8774.4, 1: 8747.5. Samples: 608467392. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:01:36,062][104569] Avg episode reward: [(0, '9174.197'), (1, '8899.249')] [2023-12-27 00:01:36,524][105692] Updated weights for policy 0, policy_version 1187632 (0.0008) [2023-12-27 00:01:36,530][105620] Updated weights for policy 1, policy_version 1188913 (0.0007) [2023-12-27 00:01:36,585][105692] Updated weights for policy 0, policy_version 1187642 (0.0008) [2023-12-27 00:01:36,591][105620] Updated weights for policy 1, policy_version 1188923 (0.0008) [2023-12-27 00:01:36,650][105620] Updated weights for policy 1, policy_version 1188933 (0.0008) [2023-12-27 00:01:36,653][105692] Updated weights for policy 0, policy_version 1187652 (0.0008) [2023-12-27 00:01:37,416][105692] Updated weights for policy 0, policy_version 1187662 (0.0009) [2023-12-27 00:01:37,458][105620] Updated weights for policy 1, policy_version 1188943 (0.0010) [2023-12-27 00:01:37,479][105692] Updated weights for policy 0, policy_version 1187672 (0.0008) [2023-12-27 00:01:37,520][105620] Updated weights for policy 1, policy_version 1188953 (0.0008) [2023-12-27 00:01:37,538][105692] Updated weights for policy 0, policy_version 1187682 (0.0009) [2023-12-27 00:01:37,589][105620] Updated weights for policy 1, policy_version 1188963 (0.0009) [2023-12-27 00:01:38,307][105692] Updated weights for policy 0, policy_version 1187692 (0.0009) [2023-12-27 00:01:38,339][105620] Updated weights for policy 1, policy_version 1188973 (0.0009) [2023-12-27 00:01:38,376][105692] Updated weights for policy 0, policy_version 1187702 (0.0009) [2023-12-27 00:01:38,403][105620] Updated weights for policy 1, policy_version 1188983 (0.0007) [2023-12-27 00:01:38,439][105692] Updated weights for policy 0, policy_version 1187712 (0.0009) [2023-12-27 00:01:38,468][105620] Updated weights for policy 1, policy_version 1188993 (0.0007) [2023-12-27 00:01:39,221][105692] Updated weights for policy 0, policy_version 1187722 (0.0009) [2023-12-27 00:01:39,292][105620] Updated weights for policy 1, policy_version 1189003 (0.0008) [2023-12-27 00:01:39,297][105692] Updated weights for policy 0, policy_version 1187732 (0.0008) [2023-12-27 00:01:39,365][105620] Updated weights for policy 1, policy_version 1189013 (0.0008) [2023-12-27 00:01:39,369][105692] Updated weights for policy 0, policy_version 1187742 (0.0010) [2023-12-27 00:01:39,442][105620] Updated weights for policy 1, policy_version 1189023 (0.0009) [2023-12-27 00:01:39,443][105692] Updated weights for policy 0, policy_version 1187752 (0.0010) [2023-12-27 00:01:40,295][105620] Updated weights for policy 1, policy_version 1189033 (0.0009) [2023-12-27 00:01:40,312][105692] Updated weights for policy 0, policy_version 1187762 (0.0008) [2023-12-27 00:01:40,364][105620] Updated weights for policy 1, policy_version 1189043 (0.0008) [2023-12-27 00:01:40,382][105692] Updated weights for policy 0, policy_version 1187772 (0.0009) [2023-12-27 00:01:40,432][105620] Updated weights for policy 1, policy_version 1189053 (0.0006) [2023-12-27 00:01:40,444][105692] Updated weights for policy 0, policy_version 1187782 (0.0007) [2023-12-27 00:01:40,500][105620] Updated weights for policy 1, policy_version 1189063 (0.0009) [2023-12-27 00:01:41,062][104569] Fps is (10 sec: 17203.2, 60 sec: 17476.3, 300 sec: 18883.3). Total num frames: 608559104. Throughput: 0: 8808.9, 1: 8765.4. Samples: 608571344. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:01:41,063][104569] Avg episode reward: [(0, '9264.534'), (1, '8713.963')] [2023-12-27 00:01:41,223][105692] Updated weights for policy 0, policy_version 1187792 (0.0008) [2023-12-27 00:01:41,248][105620] Updated weights for policy 1, policy_version 1189073 (0.0009) [2023-12-27 00:01:41,288][105692] Updated weights for policy 0, policy_version 1187802 (0.0009) [2023-12-27 00:01:41,317][105620] Updated weights for policy 1, policy_version 1189083 (0.0008) [2023-12-27 00:01:41,343][105586] KL-divergence is very high: 152.7348 [2023-12-27 00:01:41,358][105692] Updated weights for policy 0, policy_version 1187812 (0.0009) [2023-12-27 00:01:41,389][105620] Updated weights for policy 1, policy_version 1189093 (0.0009) [2023-12-27 00:01:41,404][105586] KL-divergence is very high: 155.5464 [2023-12-27 00:01:42,132][105620] Updated weights for policy 1, policy_version 1189103 (0.0009) [2023-12-27 00:01:42,152][105692] Updated weights for policy 0, policy_version 1187822 (0.0010) [2023-12-27 00:01:42,198][105620] Updated weights for policy 1, policy_version 1189113 (0.0008) [2023-12-27 00:01:42,213][105692] Updated weights for policy 0, policy_version 1187832 (0.0007) [2023-12-27 00:01:42,267][105620] Updated weights for policy 1, policy_version 1189123 (0.0010) [2023-12-27 00:01:42,282][105692] Updated weights for policy 0, policy_version 1187842 (0.0009) [2023-12-27 00:01:43,060][105692] Updated weights for policy 0, policy_version 1187852 (0.0009) [2023-12-27 00:01:43,070][105620] Updated weights for policy 1, policy_version 1189133 (0.0007) [2023-12-27 00:01:43,120][105692] Updated weights for policy 0, policy_version 1187862 (0.0007) [2023-12-27 00:01:43,126][105620] Updated weights for policy 1, policy_version 1189143 (0.0007) [2023-12-27 00:01:43,183][105692] Updated weights for policy 0, policy_version 1187872 (0.0007) [2023-12-27 00:01:43,188][105620] Updated weights for policy 1, policy_version 1189153 (0.0008) [2023-12-27 00:01:43,890][105692] Updated weights for policy 0, policy_version 1187882 (0.0008) [2023-12-27 00:01:43,921][105620] Updated weights for policy 1, policy_version 1189163 (0.0008) [2023-12-27 00:01:43,948][105692] Updated weights for policy 0, policy_version 1187892 (0.0007) [2023-12-27 00:01:43,982][105620] Updated weights for policy 1, policy_version 1189173 (0.0008) [2023-12-27 00:01:44,010][105692] Updated weights for policy 0, policy_version 1187902 (0.0006) [2023-12-27 00:01:44,045][105620] Updated weights for policy 1, policy_version 1189183 (0.0008) [2023-12-27 00:01:44,069][105692] Updated weights for policy 0, policy_version 1187912 (0.0008) [2023-12-27 00:01:44,805][105620] Updated weights for policy 1, policy_version 1189193 (0.0008) [2023-12-27 00:01:44,857][105692] Updated weights for policy 0, policy_version 1187922 (0.0008) [2023-12-27 00:01:44,874][105620] Updated weights for policy 1, policy_version 1189203 (0.0008) [2023-12-27 00:01:44,927][105692] Updated weights for policy 0, policy_version 1187932 (0.0009) [2023-12-27 00:01:44,942][105620] Updated weights for policy 1, policy_version 1189213 (0.0008) [2023-12-27 00:01:44,994][105692] Updated weights for policy 0, policy_version 1187942 (0.0007) [2023-12-27 00:01:45,009][105620] Updated weights for policy 1, policy_version 1189223 (0.0008) [2023-12-27 00:01:45,776][105692] Updated weights for policy 0, policy_version 1187952 (0.0007) [2023-12-27 00:01:45,786][105620] Updated weights for policy 1, policy_version 1189233 (0.0008) [2023-12-27 00:01:45,833][105692] Updated weights for policy 0, policy_version 1187962 (0.0006) [2023-12-27 00:01:45,848][105620] Updated weights for policy 1, policy_version 1189243 (0.0008) [2023-12-27 00:01:45,896][105692] Updated weights for policy 0, policy_version 1187972 (0.0008) [2023-12-27 00:01:45,910][105620] Updated weights for policy 1, policy_version 1189253 (0.0006) [2023-12-27 00:01:46,062][104569] Fps is (10 sec: 18022.3, 60 sec: 17612.8, 300 sec: 18883.3). Total num frames: 608657408. Throughput: 0: 8799.3, 1: 8815.1. Samples: 608624956. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:01:46,062][104569] Avg episode reward: [(0, '8635.503'), (1, '8988.403')] [2023-12-27 00:01:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001187976_304168960.pth... [2023-12-27 00:01:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001189256_304488448.pth... [2023-12-27 00:01:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001188200_304218112.pth [2023-12-27 00:01:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001186920_303898624.pth [2023-12-27 00:01:46,072][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_001189256_304488448.pth [2023-12-27 00:01:46,073][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_001187976_304168960.pth [2023-12-27 00:01:46,685][105692] Updated weights for policy 0, policy_version 1187982 (0.0008) [2023-12-27 00:01:46,723][105620] Updated weights for policy 1, policy_version 1189263 (0.0009) [2023-12-27 00:01:46,747][105692] Updated weights for policy 0, policy_version 1187992 (0.0009) [2023-12-27 00:01:46,779][105620] Updated weights for policy 1, policy_version 1189273 (0.0007) [2023-12-27 00:01:46,798][105692] Updated weights for policy 0, policy_version 1188002 (0.0007) [2023-12-27 00:01:46,841][105620] Updated weights for policy 1, policy_version 1189283 (0.0008) [2023-12-27 00:01:47,491][105620] Updated weights for policy 1, policy_version 1189293 (0.0008) [2023-12-27 00:01:47,551][105620] Updated weights for policy 1, policy_version 1189303 (0.0009) [2023-12-27 00:01:47,610][105620] Updated weights for policy 1, policy_version 1189313 (0.0008) [2023-12-27 00:01:47,654][105692] Updated weights for policy 0, policy_version 1188012 (0.0008) [2023-12-27 00:01:47,713][105692] Updated weights for policy 0, policy_version 1188022 (0.0009) [2023-12-27 00:01:47,773][105692] Updated weights for policy 0, policy_version 1188032 (0.0009) [2023-12-27 00:01:48,340][105620] Updated weights for policy 1, policy_version 1189323 (0.0009) [2023-12-27 00:01:48,409][105620] Updated weights for policy 1, policy_version 1189333 (0.0008) [2023-12-27 00:01:48,473][105620] Updated weights for policy 1, policy_version 1189343 (0.0009) [2023-12-27 00:01:48,533][105692] Updated weights for policy 0, policy_version 1188042 (0.0009) [2023-12-27 00:01:48,597][105692] Updated weights for policy 0, policy_version 1188052 (0.0008) [2023-12-27 00:01:48,659][105692] Updated weights for policy 0, policy_version 1188062 (0.0008) [2023-12-27 00:01:48,721][105692] Updated weights for policy 0, policy_version 1188072 (0.0009) [2023-12-27 00:01:49,264][105620] Updated weights for policy 1, policy_version 1189353 (0.0008) [2023-12-27 00:01:49,334][105620] Updated weights for policy 1, policy_version 1189363 (0.0008) [2023-12-27 00:01:49,406][105620] Updated weights for policy 1, policy_version 1189373 (0.0010) [2023-12-27 00:01:49,475][105620] Updated weights for policy 1, policy_version 1189383 (0.0009) [2023-12-27 00:01:49,505][105692] Updated weights for policy 0, policy_version 1188082 (0.0007) [2023-12-27 00:01:49,571][105692] Updated weights for policy 0, policy_version 1188092 (0.0008) [2023-12-27 00:01:49,636][105692] Updated weights for policy 0, policy_version 1188102 (0.0006) [2023-12-27 00:01:50,291][105620] Updated weights for policy 1, policy_version 1189393 (0.0008) [2023-12-27 00:01:50,352][105620] Updated weights for policy 1, policy_version 1189403 (0.0008) [2023-12-27 00:01:50,364][105692] Updated weights for policy 0, policy_version 1188112 (0.0007) [2023-12-27 00:01:50,413][105620] Updated weights for policy 1, policy_version 1189413 (0.0007) [2023-12-27 00:01:50,425][105692] Updated weights for policy 0, policy_version 1188122 (0.0006) [2023-12-27 00:01:50,485][105692] Updated weights for policy 0, policy_version 1188132 (0.0008) [2023-12-27 00:01:51,062][104569] Fps is (10 sec: 18022.4, 60 sec: 17476.3, 300 sec: 18827.7). Total num frames: 608739328. Throughput: 0: 8888.3, 1: 8881.2. Samples: 608733388. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:01:51,063][104569] Avg episode reward: [(0, '8455.964'), (1, '8713.980')] [2023-12-27 00:01:51,202][105620] Updated weights for policy 1, policy_version 1189423 (0.0009) [2023-12-27 00:01:51,276][105620] Updated weights for policy 1, policy_version 1189433 (0.0007) [2023-12-27 00:01:51,280][105692] Updated weights for policy 0, policy_version 1188142 (0.0008) [2023-12-27 00:01:51,346][105620] Updated weights for policy 1, policy_version 1189443 (0.0009) [2023-12-27 00:01:51,348][105692] Updated weights for policy 0, policy_version 1188152 (0.0007) [2023-12-27 00:01:51,420][105692] Updated weights for policy 0, policy_version 1188162 (0.0008) [2023-12-27 00:01:52,151][105620] Updated weights for policy 1, policy_version 1189453 (0.0009) [2023-12-27 00:01:52,210][105692] Updated weights for policy 0, policy_version 1188172 (0.0009) [2023-12-27 00:01:52,217][105620] Updated weights for policy 1, policy_version 1189463 (0.0007) [2023-12-27 00:01:52,277][105692] Updated weights for policy 0, policy_version 1188182 (0.0007) [2023-12-27 00:01:52,288][105620] Updated weights for policy 1, policy_version 1189473 (0.0009) [2023-12-27 00:01:52,346][105692] Updated weights for policy 0, policy_version 1188192 (0.0009) [2023-12-27 00:01:53,050][105620] Updated weights for policy 1, policy_version 1189483 (0.0007) [2023-12-27 00:01:53,113][105620] Updated weights for policy 1, policy_version 1189493 (0.0006) [2023-12-27 00:01:53,129][105692] Updated weights for policy 0, policy_version 1188202 (0.0009) [2023-12-27 00:01:53,175][105620] Updated weights for policy 1, policy_version 1189503 (0.0006) [2023-12-27 00:01:53,192][105692] Updated weights for policy 0, policy_version 1188212 (0.0010) [2023-12-27 00:01:53,247][105692] Updated weights for policy 0, policy_version 1188222 (0.0011) [2023-12-27 00:01:53,312][105692] Updated weights for policy 0, policy_version 1188232 (0.0011) [2023-12-27 00:01:53,952][105620] Updated weights for policy 1, policy_version 1189513 (0.0006) [2023-12-27 00:01:54,016][105620] Updated weights for policy 1, policy_version 1189523 (0.0008) [2023-12-27 00:01:54,068][105692] Updated weights for policy 0, policy_version 1188242 (0.0009) [2023-12-27 00:01:54,080][105620] Updated weights for policy 1, policy_version 1189533 (0.0007) [2023-12-27 00:01:54,130][105692] Updated weights for policy 0, policy_version 1188252 (0.0010) [2023-12-27 00:01:54,148][105620] Updated weights for policy 1, policy_version 1189543 (0.0008) [2023-12-27 00:01:54,190][105692] Updated weights for policy 0, policy_version 1188262 (0.0008) [2023-12-27 00:01:54,926][105620] Updated weights for policy 1, policy_version 1189553 (0.0008) [2023-12-27 00:01:54,961][105692] Updated weights for policy 0, policy_version 1188272 (0.0007) [2023-12-27 00:01:54,991][105620] Updated weights for policy 1, policy_version 1189563 (0.0008) [2023-12-27 00:01:55,026][105692] Updated weights for policy 0, policy_version 1188282 (0.0007) [2023-12-27 00:01:55,059][105620] Updated weights for policy 1, policy_version 1189573 (0.0008) [2023-12-27 00:01:55,091][105692] Updated weights for policy 0, policy_version 1188292 (0.0006) [2023-12-27 00:01:55,758][105620] Updated weights for policy 1, policy_version 1189583 (0.0008) [2023-12-27 00:01:55,825][105620] Updated weights for policy 1, policy_version 1189593 (0.0008) [2023-12-27 00:01:55,849][105692] Updated weights for policy 0, policy_version 1188302 (0.0007) [2023-12-27 00:01:55,881][105620] Updated weights for policy 1, policy_version 1189603 (0.0008) [2023-12-27 00:01:55,916][105692] Updated weights for policy 0, policy_version 1188312 (0.0009) [2023-12-27 00:01:55,976][105692] Updated weights for policy 0, policy_version 1188322 (0.0009) [2023-12-27 00:01:56,062][104569] Fps is (10 sec: 18022.3, 60 sec: 17749.3, 300 sec: 18827.7). Total num frames: 608837632. Throughput: 0: 8902.2, 1: 8904.8. Samples: 608840988. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:01:56,063][104569] Avg episode reward: [(0, '8993.673'), (1, '8893.544')] [2023-12-27 00:01:56,651][105620] Updated weights for policy 1, policy_version 1189613 (0.0008) [2023-12-27 00:01:56,717][105620] Updated weights for policy 1, policy_version 1189623 (0.0009) [2023-12-27 00:01:56,769][105692] Updated weights for policy 0, policy_version 1188332 (0.0008) [2023-12-27 00:01:56,782][105620] Updated weights for policy 1, policy_version 1189633 (0.0009) [2023-12-27 00:01:56,831][105692] Updated weights for policy 0, policy_version 1188342 (0.0006) [2023-12-27 00:01:56,888][105692] Updated weights for policy 0, policy_version 1188352 (0.0010) [2023-12-27 00:01:57,462][105620] Updated weights for policy 1, policy_version 1189643 (0.0008) [2023-12-27 00:01:57,521][105620] Updated weights for policy 1, policy_version 1189653 (0.0008) [2023-12-27 00:01:57,577][105620] Updated weights for policy 1, policy_version 1189663 (0.0007) [2023-12-27 00:01:57,625][105692] Updated weights for policy 0, policy_version 1188362 (0.0008) [2023-12-27 00:01:57,674][105692] Updated weights for policy 0, policy_version 1188372 (0.0008) [2023-12-27 00:01:57,727][105692] Updated weights for policy 0, policy_version 1188382 (0.0009) [2023-12-27 00:01:57,784][105692] Updated weights for policy 0, policy_version 1188392 (0.0008) [2023-12-27 00:01:58,330][105620] Updated weights for policy 1, policy_version 1189673 (0.0008) [2023-12-27 00:01:58,413][105620] Updated weights for policy 1, policy_version 1189683 (0.0008) [2023-12-27 00:01:58,481][105620] Updated weights for policy 1, policy_version 1189693 (0.0008) [2023-12-27 00:01:58,547][105620] Updated weights for policy 1, policy_version 1189703 (0.0009) [2023-12-27 00:01:58,587][105692] Updated weights for policy 0, policy_version 1188402 (0.0008) [2023-12-27 00:01:58,659][105692] Updated weights for policy 0, policy_version 1188412 (0.0009) [2023-12-27 00:01:58,726][105692] Updated weights for policy 0, policy_version 1188422 (0.0008) [2023-12-27 00:01:59,440][105620] Updated weights for policy 1, policy_version 1189713 (0.0008) [2023-12-27 00:01:59,506][105692] Updated weights for policy 0, policy_version 1188432 (0.0009) [2023-12-27 00:01:59,509][105620] Updated weights for policy 1, policy_version 1189723 (0.0008) [2023-12-27 00:01:59,565][105692] Updated weights for policy 0, policy_version 1188442 (0.0007) [2023-12-27 00:01:59,572][105620] Updated weights for policy 1, policy_version 1189733 (0.0008) [2023-12-27 00:01:59,624][105692] Updated weights for policy 0, policy_version 1188452 (0.0008) [2023-12-27 00:02:00,247][105620] Updated weights for policy 1, policy_version 1189743 (0.0009) [2023-12-27 00:02:00,299][105620] Updated weights for policy 1, policy_version 1189753 (0.0010) [2023-12-27 00:02:00,359][105620] Updated weights for policy 1, policy_version 1189763 (0.0011) [2023-12-27 00:02:00,443][105692] Updated weights for policy 0, policy_version 1188462 (0.0010) [2023-12-27 00:02:00,510][105692] Updated weights for policy 0, policy_version 1188472 (0.0011) [2023-12-27 00:02:00,576][105692] Updated weights for policy 0, policy_version 1188482 (0.0010) [2023-12-27 00:02:01,062][104569] Fps is (10 sec: 18022.4, 60 sec: 17749.4, 300 sec: 18772.2). Total num frames: 608919552. Throughput: 0: 8941.4, 1: 8926.9. Samples: 608895488. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:02:01,062][104569] Avg episode reward: [(0, '9263.912'), (1, '9076.983')] [2023-12-27 00:02:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001188488_304300032.pth... [2023-12-27 00:02:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001189768_304619520.pth... [2023-12-27 00:02:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001187464_304037888.pth [2023-12-27 00:02:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001188744_304357376.pth [2023-12-27 00:02:01,158][105620] Updated weights for policy 1, policy_version 1189773 (0.0011) [2023-12-27 00:02:01,228][105620] Updated weights for policy 1, policy_version 1189783 (0.0010) [2023-12-27 00:02:01,293][105692] Updated weights for policy 0, policy_version 1188492 (0.0006) [2023-12-27 00:02:01,300][105620] Updated weights for policy 1, policy_version 1189793 (0.0011) [2023-12-27 00:02:01,365][105692] Updated weights for policy 0, policy_version 1188502 (0.0011) [2023-12-27 00:02:01,437][105692] Updated weights for policy 0, policy_version 1188512 (0.0010) [2023-12-27 00:02:02,125][105620] Updated weights for policy 1, policy_version 1189803 (0.0010) [2023-12-27 00:02:02,159][105692] Updated weights for policy 0, policy_version 1188522 (0.0011) [2023-12-27 00:02:02,191][105620] Updated weights for policy 1, policy_version 1189813 (0.0009) [2023-12-27 00:02:02,220][105692] Updated weights for policy 0, policy_version 1188532 (0.0011) [2023-12-27 00:02:02,254][105620] Updated weights for policy 1, policy_version 1189823 (0.0006) [2023-12-27 00:02:02,277][105692] Updated weights for policy 0, policy_version 1188542 (0.0011) [2023-12-27 00:02:02,341][105692] Updated weights for policy 0, policy_version 1188552 (0.0008) [2023-12-27 00:02:02,996][105620] Updated weights for policy 1, policy_version 1189833 (0.0007) [2023-12-27 00:02:03,054][105620] Updated weights for policy 1, policy_version 1189843 (0.0006) [2023-12-27 00:02:03,112][105620] Updated weights for policy 1, policy_version 1189853 (0.0005) [2023-12-27 00:02:03,169][105620] Updated weights for policy 1, policy_version 1189863 (0.0006) [2023-12-27 00:02:03,197][105692] Updated weights for policy 0, policy_version 1188562 (0.0008) [2023-12-27 00:02:03,258][105692] Updated weights for policy 0, policy_version 1188572 (0.0009) [2023-12-27 00:02:03,321][105692] Updated weights for policy 0, policy_version 1188582 (0.0009) [2023-12-27 00:02:03,899][105620] Updated weights for policy 1, policy_version 1189873 (0.0009) [2023-12-27 00:02:03,960][105620] Updated weights for policy 1, policy_version 1189883 (0.0008) [2023-12-27 00:02:04,022][105620] Updated weights for policy 1, policy_version 1189893 (0.0008) [2023-12-27 00:02:04,111][105692] Updated weights for policy 0, policy_version 1188592 (0.0009) [2023-12-27 00:02:04,175][105692] Updated weights for policy 0, policy_version 1188602 (0.0009) [2023-12-27 00:02:04,239][105692] Updated weights for policy 0, policy_version 1188612 (0.0009) [2023-12-27 00:02:04,838][105620] Updated weights for policy 1, policy_version 1189903 (0.0010) [2023-12-27 00:02:04,905][105620] Updated weights for policy 1, policy_version 1189913 (0.0011) [2023-12-27 00:02:04,968][105620] Updated weights for policy 1, policy_version 1189923 (0.0009) [2023-12-27 00:02:05,028][105692] Updated weights for policy 0, policy_version 1188622 (0.0008) [2023-12-27 00:02:05,081][105692] Updated weights for policy 0, policy_version 1188632 (0.0008) [2023-12-27 00:02:05,144][105692] Updated weights for policy 0, policy_version 1188642 (0.0007) [2023-12-27 00:02:05,622][105620] Updated weights for policy 1, policy_version 1189933 (0.0009) [2023-12-27 00:02:05,684][105620] Updated weights for policy 1, policy_version 1189943 (0.0009) [2023-12-27 00:02:05,747][105620] Updated weights for policy 1, policy_version 1189953 (0.0009) [2023-12-27 00:02:05,985][105692] Updated weights for policy 0, policy_version 1188652 (0.0009) [2023-12-27 00:02:06,047][105692] Updated weights for policy 0, policy_version 1188662 (0.0009) [2023-12-27 00:02:06,062][104569] Fps is (10 sec: 17203.2, 60 sec: 17885.9, 300 sec: 18744.4). Total num frames: 609009664. Throughput: 0: 8914.7, 1: 8946.5. Samples: 609002684. Policy #0 lag: (min: 6.0, avg: 15.1, max: 38.0) [2023-12-27 00:02:06,063][104569] Avg episode reward: [(0, '8990.419'), (1, '9087.570')] [2023-12-27 00:02:06,117][105692] Updated weights for policy 0, policy_version 1188672 (0.0009) [2023-12-27 00:02:06,517][105620] Updated weights for policy 1, policy_version 1189963 (0.0009) [2023-12-27 00:02:06,572][105620] Updated weights for policy 1, policy_version 1189973 (0.0008) [2023-12-27 00:02:06,632][105620] Updated weights for policy 1, policy_version 1189983 (0.0009) [2023-12-27 00:02:06,951][105692] Updated weights for policy 0, policy_version 1188682 (0.0008) [2023-12-27 00:02:07,012][105692] Updated weights for policy 0, policy_version 1188692 (0.0009) [2023-12-27 00:02:07,073][105692] Updated weights for policy 0, policy_version 1188702 (0.0009) [2023-12-27 00:02:07,134][105692] Updated weights for policy 0, policy_version 1188712 (0.0009) [2023-12-27 00:02:07,367][105620] Updated weights for policy 1, policy_version 1189993 (0.0009) [2023-12-27 00:02:07,435][105620] Updated weights for policy 1, policy_version 1190003 (0.0008) [2023-12-27 00:02:07,504][105620] Updated weights for policy 1, policy_version 1190013 (0.0009) [2023-12-27 00:02:07,575][105620] Updated weights for policy 1, policy_version 1190023 (0.0008) [2023-12-27 00:02:07,965][105692] Updated weights for policy 0, policy_version 1188722 (0.0008) [2023-12-27 00:02:08,033][105692] Updated weights for policy 0, policy_version 1188732 (0.0009) [2023-12-27 00:02:08,096][105692] Updated weights for policy 0, policy_version 1188742 (0.0009) [2023-12-27 00:02:08,351][105620] Updated weights for policy 1, policy_version 1190033 (0.0008) [2023-12-27 00:02:08,419][105620] Updated weights for policy 1, policy_version 1190043 (0.0009) [2023-12-27 00:02:08,478][105620] Updated weights for policy 1, policy_version 1190053 (0.0009) [2023-12-27 00:02:08,893][105692] Updated weights for policy 0, policy_version 1188752 (0.0009) [2023-12-27 00:02:08,958][105692] Updated weights for policy 0, policy_version 1188762 (0.0010) [2023-12-27 00:02:09,031][105692] Updated weights for policy 0, policy_version 1188772 (0.0008) [2023-12-27 00:02:09,258][105620] Updated weights for policy 1, policy_version 1190065 (0.0009) [2023-12-27 00:02:09,325][105620] Updated weights for policy 1, policy_version 1190075 (0.0009) [2023-12-27 00:02:09,399][105620] Updated weights for policy 1, policy_version 1190085 (0.0009) [2023-12-27 00:02:09,773][105692] Updated weights for policy 0, policy_version 1188782 (0.0008) [2023-12-27 00:02:09,841][105692] Updated weights for policy 0, policy_version 1188792 (0.0009) [2023-12-27 00:02:09,901][105692] Updated weights for policy 0, policy_version 1188802 (0.0007) [2023-12-27 00:02:10,206][105620] Updated weights for policy 1, policy_version 1190095 (0.0008) [2023-12-27 00:02:10,270][105620] Updated weights for policy 1, policy_version 1190105 (0.0008) [2023-12-27 00:02:10,334][105620] Updated weights for policy 1, policy_version 1190115 (0.0008) [2023-12-27 00:02:10,700][105692] Updated weights for policy 0, policy_version 1188812 (0.0008) [2023-12-27 00:02:10,766][105692] Updated weights for policy 0, policy_version 1188822 (0.0009) [2023-12-27 00:02:10,829][105692] Updated weights for policy 0, policy_version 1188832 (0.0010) [2023-12-27 00:02:11,062][104569] Fps is (10 sec: 18022.4, 60 sec: 17749.3, 300 sec: 18716.6). Total num frames: 609099776. Throughput: 0: 8930.8, 1: 8979.6. Samples: 609109368. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:02:11,063][104569] Avg episode reward: [(0, '9081.985'), (1, '8732.457')] [2023-12-27 00:02:11,144][105620] Updated weights for policy 1, policy_version 1190125 (0.0009) [2023-12-27 00:02:11,220][105620] Updated weights for policy 1, policy_version 1190135 (0.0009) [2023-12-27 00:02:11,293][105620] Updated weights for policy 1, policy_version 1190145 (0.0009) [2023-12-27 00:02:11,764][105692] Updated weights for policy 0, policy_version 1188842 (0.0009) [2023-12-27 00:02:11,835][105692] Updated weights for policy 0, policy_version 1188852 (0.0008) [2023-12-27 00:02:11,899][105692] Updated weights for policy 0, policy_version 1188862 (0.0009) [2023-12-27 00:02:11,962][105692] Updated weights for policy 0, policy_version 1188872 (0.0010) [2023-12-27 00:02:12,071][105620] Updated weights for policy 1, policy_version 1190155 (0.0008) [2023-12-27 00:02:12,140][105620] Updated weights for policy 1, policy_version 1190165 (0.0008) [2023-12-27 00:02:12,197][105620] Updated weights for policy 1, policy_version 1190175 (0.0009) [2023-12-27 00:02:12,832][105692] Updated weights for policy 0, policy_version 1188882 (0.0009) [2023-12-27 00:02:12,905][105692] Updated weights for policy 0, policy_version 1188892 (0.0009) [2023-12-27 00:02:12,962][105620] Updated weights for policy 1, policy_version 1190185 (0.0009) [2023-12-27 00:02:12,971][105692] Updated weights for policy 0, policy_version 1188902 (0.0010) [2023-12-27 00:02:13,028][105620] Updated weights for policy 1, policy_version 1190195 (0.0009) [2023-12-27 00:02:13,088][105620] Updated weights for policy 1, policy_version 1190205 (0.0009) [2023-12-27 00:02:13,142][105620] Updated weights for policy 1, policy_version 1190215 (0.0009) [2023-12-27 00:02:13,702][105692] Updated weights for policy 0, policy_version 1188912 (0.0009) [2023-12-27 00:02:13,757][105692] Updated weights for policy 0, policy_version 1188922 (0.0009) [2023-12-27 00:02:13,824][105692] Updated weights for policy 0, policy_version 1188932 (0.0010) [2023-12-27 00:02:13,941][105620] Updated weights for policy 1, policy_version 1190225 (0.0010) [2023-12-27 00:02:13,994][105620] Updated weights for policy 1, policy_version 1190235 (0.0008) [2023-12-27 00:02:14,044][105620] Updated weights for policy 1, policy_version 1190245 (0.0008) [2023-12-27 00:02:14,588][105692] Updated weights for policy 0, policy_version 1188942 (0.0011) [2023-12-27 00:02:14,658][105692] Updated weights for policy 0, policy_version 1188952 (0.0009) [2023-12-27 00:02:14,726][105692] Updated weights for policy 0, policy_version 1188962 (0.0008) [2023-12-27 00:02:14,816][105620] Updated weights for policy 1, policy_version 1190255 (0.0007) [2023-12-27 00:02:14,869][105620] Updated weights for policy 1, policy_version 1190265 (0.0006) [2023-12-27 00:02:14,934][105620] Updated weights for policy 1, policy_version 1190275 (0.0008) [2023-12-27 00:02:15,541][105692] Updated weights for policy 0, policy_version 1188972 (0.0011) [2023-12-27 00:02:15,602][105692] Updated weights for policy 0, policy_version 1188982 (0.0011) [2023-12-27 00:02:15,644][105620] Updated weights for policy 1, policy_version 1190285 (0.0009) [2023-12-27 00:02:15,664][105692] Updated weights for policy 0, policy_version 1188992 (0.0011) [2023-12-27 00:02:15,707][105620] Updated weights for policy 1, policy_version 1190295 (0.0010) [2023-12-27 00:02:15,777][105620] Updated weights for policy 1, policy_version 1190305 (0.0008) [2023-12-27 00:02:16,062][104569] Fps is (10 sec: 18022.5, 60 sec: 17885.9, 300 sec: 18688.9). Total num frames: 609189888. Throughput: 0: 8878.2, 1: 8950.6. Samples: 609160576. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:02:16,062][104569] Avg episode reward: [(0, '9173.972'), (1, '8753.371')] [2023-12-27 00:02:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001189000_304431104.pth... [2023-12-27 00:02:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001190312_304758784.pth... [2023-12-27 00:02:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001189256_304488448.pth [2023-12-27 00:02:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001187976_304168960.pth [2023-12-27 00:02:16,436][105692] Updated weights for policy 0, policy_version 1189002 (0.0011) [2023-12-27 00:02:16,493][105692] Updated weights for policy 0, policy_version 1189012 (0.0011) [2023-12-27 00:02:16,547][105620] Updated weights for policy 1, policy_version 1190315 (0.0009) [2023-12-27 00:02:16,552][105692] Updated weights for policy 0, policy_version 1189022 (0.0011) [2023-12-27 00:02:16,603][105620] Updated weights for policy 1, policy_version 1190325 (0.0007) [2023-12-27 00:02:16,605][105692] Updated weights for policy 0, policy_version 1189032 (0.0011) [2023-12-27 00:02:16,667][105620] Updated weights for policy 1, policy_version 1190335 (0.0008) [2023-12-27 00:02:17,368][105692] Updated weights for policy 0, policy_version 1189042 (0.0011) [2023-12-27 00:02:17,425][105692] Updated weights for policy 0, policy_version 1189052 (0.0011) [2023-12-27 00:02:17,439][105620] Updated weights for policy 1, policy_version 1190345 (0.0008) [2023-12-27 00:02:17,478][105692] Updated weights for policy 0, policy_version 1189062 (0.0011) [2023-12-27 00:02:17,501][105620] Updated weights for policy 1, policy_version 1190355 (0.0006) [2023-12-27 00:02:17,550][105620] Updated weights for policy 1, policy_version 1190365 (0.0008) [2023-12-27 00:02:17,603][105620] Updated weights for policy 1, policy_version 1190375 (0.0008) [2023-12-27 00:02:18,255][105692] Updated weights for policy 0, policy_version 1189072 (0.0011) [2023-12-27 00:02:18,316][105692] Updated weights for policy 0, policy_version 1189082 (0.0011) [2023-12-27 00:02:18,395][105620] Updated weights for policy 1, policy_version 1190385 (0.0009) [2023-12-27 00:02:18,398][105692] Updated weights for policy 0, policy_version 1189093 (0.0013) [2023-12-27 00:02:18,461][105620] Updated weights for policy 1, policy_version 1190395 (0.0008) [2023-12-27 00:02:18,529][105620] Updated weights for policy 1, policy_version 1190405 (0.0009) [2023-12-27 00:02:19,183][105692] Updated weights for policy 0, policy_version 1189103 (0.0010) [2023-12-27 00:02:19,247][105692] Updated weights for policy 0, policy_version 1189113 (0.0009) [2023-12-27 00:02:19,279][105620] Updated weights for policy 1, policy_version 1190415 (0.0009) [2023-12-27 00:02:19,318][105692] Updated weights for policy 0, policy_version 1189123 (0.0008) [2023-12-27 00:02:19,347][105620] Updated weights for policy 1, policy_version 1190425 (0.0009) [2023-12-27 00:02:19,417][105620] Updated weights for policy 1, policy_version 1190435 (0.0009) [2023-12-27 00:02:20,187][105692] Updated weights for policy 0, policy_version 1189133 (0.0009) [2023-12-27 00:02:20,215][105620] Updated weights for policy 1, policy_version 1190445 (0.0007) [2023-12-27 00:02:20,255][105692] Updated weights for policy 0, policy_version 1189143 (0.0007) [2023-12-27 00:02:20,277][105585] KL-divergence is very high: 149.7605 [2023-12-27 00:02:20,283][105620] Updated weights for policy 1, policy_version 1190455 (0.0008) [2023-12-27 00:02:20,325][105692] Updated weights for policy 0, policy_version 1189153 (0.0007) [2023-12-27 00:02:20,334][105585] KL-divergence is very high: 158.4518 [2023-12-27 00:02:20,351][105620] Updated weights for policy 1, policy_version 1190465 (0.0007) [2023-12-27 00:02:21,062][104569] Fps is (10 sec: 17203.1, 60 sec: 17749.3, 300 sec: 18633.3). Total num frames: 609271808. Throughput: 0: 8823.9, 1: 8982.5. Samples: 609268680. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:02:21,063][104569] Avg episode reward: [(0, '8994.402'), (1, '8928.834')] [2023-12-27 00:02:21,103][105692] Updated weights for policy 0, policy_version 1189163 (0.0008) [2023-12-27 00:02:21,174][105692] Updated weights for policy 0, policy_version 1189173 (0.0009) [2023-12-27 00:02:21,207][105620] Updated weights for policy 1, policy_version 1190475 (0.0009) [2023-12-27 00:02:21,246][105692] Updated weights for policy 0, policy_version 1189183 (0.0008) [2023-12-27 00:02:21,280][105620] Updated weights for policy 1, policy_version 1190485 (0.0008) [2023-12-27 00:02:21,355][105620] Updated weights for policy 1, policy_version 1190495 (0.0010) [2023-12-27 00:02:22,103][105692] Updated weights for policy 0, policy_version 1189193 (0.0009) [2023-12-27 00:02:22,130][105620] Updated weights for policy 1, policy_version 1190505 (0.0009) [2023-12-27 00:02:22,173][105692] Updated weights for policy 0, policy_version 1189203 (0.0009) [2023-12-27 00:02:22,202][105620] Updated weights for policy 1, policy_version 1190515 (0.0008) [2023-12-27 00:02:22,248][105692] Updated weights for policy 0, policy_version 1189213 (0.0009) [2023-12-27 00:02:22,278][105620] Updated weights for policy 1, policy_version 1190525 (0.0007) [2023-12-27 00:02:22,319][105692] Updated weights for policy 0, policy_version 1189223 (0.0008) [2023-12-27 00:02:22,344][105620] Updated weights for policy 1, policy_version 1190535 (0.0009) [2023-12-27 00:02:23,161][105692] Updated weights for policy 0, policy_version 1189233 (0.0008) [2023-12-27 00:02:23,228][105692] Updated weights for policy 0, policy_version 1189243 (0.0006) [2023-12-27 00:02:23,230][105620] Updated weights for policy 1, policy_version 1190545 (0.0008) [2023-12-27 00:02:23,288][105692] Updated weights for policy 0, policy_version 1189253 (0.0008) [2023-12-27 00:02:23,295][105620] Updated weights for policy 1, policy_version 1190555 (0.0006) [2023-12-27 00:02:23,369][105620] Updated weights for policy 1, policy_version 1190565 (0.0007) [2023-12-27 00:02:24,065][105692] Updated weights for policy 0, policy_version 1189263 (0.0009) [2023-12-27 00:02:24,123][105620] Updated weights for policy 1, policy_version 1190575 (0.0008) [2023-12-27 00:02:24,126][105692] Updated weights for policy 0, policy_version 1189273 (0.0007) [2023-12-27 00:02:24,182][105620] Updated weights for policy 1, policy_version 1190585 (0.0007) [2023-12-27 00:02:24,190][105692] Updated weights for policy 0, policy_version 1189283 (0.0009) [2023-12-27 00:02:24,244][105620] Updated weights for policy 1, policy_version 1190595 (0.0008) [2023-12-27 00:02:24,967][105692] Updated weights for policy 0, policy_version 1189293 (0.0007) [2023-12-27 00:02:25,013][105620] Updated weights for policy 1, policy_version 1190605 (0.0010) [2023-12-27 00:02:25,021][105692] Updated weights for policy 0, policy_version 1189303 (0.0007) [2023-12-27 00:02:25,075][105620] Updated weights for policy 1, policy_version 1190615 (0.0011) [2023-12-27 00:02:25,082][105692] Updated weights for policy 0, policy_version 1189313 (0.0007) [2023-12-27 00:02:25,140][105620] Updated weights for policy 1, policy_version 1190625 (0.0011) [2023-12-27 00:02:25,802][105692] Updated weights for policy 0, policy_version 1189323 (0.0008) [2023-12-27 00:02:25,855][105692] Updated weights for policy 0, policy_version 1189333 (0.0010) [2023-12-27 00:02:25,879][105620] Updated weights for policy 1, policy_version 1190635 (0.0011) [2023-12-27 00:02:25,915][105692] Updated weights for policy 0, policy_version 1189343 (0.0011) [2023-12-27 00:02:25,932][105620] Updated weights for policy 1, policy_version 1190645 (0.0010) [2023-12-27 00:02:25,996][105620] Updated weights for policy 1, policy_version 1190655 (0.0009) [2023-12-27 00:02:26,062][104569] Fps is (10 sec: 18022.5, 60 sec: 17885.9, 300 sec: 18633.3). Total num frames: 609370112. Throughput: 0: 8809.7, 1: 8990.8. Samples: 609372364. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:02:26,062][104569] Avg episode reward: [(0, '9085.923'), (1, '8894.192')] [2023-12-27 00:02:26,608][105692] Updated weights for policy 0, policy_version 1189353 (0.0008) [2023-12-27 00:02:26,675][105692] Updated weights for policy 0, policy_version 1189363 (0.0009) [2023-12-27 00:02:26,715][105585] KL-divergence is very high: 119.7815 [2023-12-27 00:02:26,736][105692] Updated weights for policy 0, policy_version 1189373 (0.0010) [2023-12-27 00:02:26,768][105585] KL-divergence is very high: 134.6214 [2023-12-27 00:02:26,803][105692] Updated weights for policy 0, policy_version 1189383 (0.0009) [2023-12-27 00:02:26,811][105620] Updated weights for policy 1, policy_version 1190665 (0.0008) [2023-12-27 00:02:26,876][105620] Updated weights for policy 1, policy_version 1190675 (0.0009) [2023-12-27 00:02:26,942][105620] Updated weights for policy 1, policy_version 1190685 (0.0008) [2023-12-27 00:02:27,005][105620] Updated weights for policy 1, policy_version 1190695 (0.0009) [2023-12-27 00:02:27,574][105692] Updated weights for policy 0, policy_version 1189393 (0.0008) [2023-12-27 00:02:27,642][105692] Updated weights for policy 0, policy_version 1189403 (0.0010) [2023-12-27 00:02:27,702][105692] Updated weights for policy 0, policy_version 1189413 (0.0007) [2023-12-27 00:02:27,732][105620] Updated weights for policy 1, policy_version 1190705 (0.0008) [2023-12-27 00:02:27,794][105620] Updated weights for policy 1, policy_version 1190715 (0.0010) [2023-12-27 00:02:27,849][105620] Updated weights for policy 1, policy_version 1190725 (0.0009) [2023-12-27 00:02:28,467][105692] Updated weights for policy 0, policy_version 1189423 (0.0008) [2023-12-27 00:02:28,522][105692] Updated weights for policy 0, policy_version 1189433 (0.0010) [2023-12-27 00:02:28,583][105692] Updated weights for policy 0, policy_version 1189443 (0.0010) [2023-12-27 00:02:28,599][105620] Updated weights for policy 1, policy_version 1190735 (0.0007) [2023-12-27 00:02:28,665][105620] Updated weights for policy 1, policy_version 1190745 (0.0010) [2023-12-27 00:02:28,726][105620] Updated weights for policy 1, policy_version 1190755 (0.0008) [2023-12-27 00:02:29,429][105620] Updated weights for policy 1, policy_version 1190765 (0.0009) [2023-12-27 00:02:29,446][105692] Updated weights for policy 0, policy_version 1189453 (0.0009) [2023-12-27 00:02:29,488][105620] Updated weights for policy 1, policy_version 1190775 (0.0008) [2023-12-27 00:02:29,507][105692] Updated weights for policy 0, policy_version 1189463 (0.0008) [2023-12-27 00:02:29,551][105620] Updated weights for policy 1, policy_version 1190785 (0.0007) [2023-12-27 00:02:29,568][105692] Updated weights for policy 0, policy_version 1189473 (0.0010) [2023-12-27 00:02:30,292][105620] Updated weights for policy 1, policy_version 1190795 (0.0009) [2023-12-27 00:02:30,353][105620] Updated weights for policy 1, policy_version 1190805 (0.0007) [2023-12-27 00:02:30,389][105692] Updated weights for policy 0, policy_version 1189483 (0.0007) [2023-12-27 00:02:30,412][105620] Updated weights for policy 1, policy_version 1190815 (0.0008) [2023-12-27 00:02:30,449][105692] Updated weights for policy 0, policy_version 1189493 (0.0007) [2023-12-27 00:02:30,511][105692] Updated weights for policy 0, policy_version 1189503 (0.0008) [2023-12-27 00:02:31,062][104569] Fps is (10 sec: 18022.4, 60 sec: 17749.3, 300 sec: 18550.0). Total num frames: 609452032. Throughput: 0: 8834.7, 1: 8994.1. Samples: 609427252. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:02:31,063][104569] Avg episode reward: [(0, '9267.738'), (1, '8809.744')] [2023-12-27 00:02:31,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001189512_304562176.pth... [2023-12-27 00:02:31,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001190824_304889856.pth... [2023-12-27 00:02:31,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001188488_304300032.pth [2023-12-27 00:02:31,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001189768_304619520.pth [2023-12-27 00:02:31,171][105620] Updated weights for policy 1, policy_version 1190825 (0.0007) [2023-12-27 00:02:31,238][105620] Updated weights for policy 1, policy_version 1190835 (0.0009) [2023-12-27 00:02:31,304][105620] Updated weights for policy 1, policy_version 1190845 (0.0009) [2023-12-27 00:02:31,311][105692] Updated weights for policy 0, policy_version 1189513 (0.0009) [2023-12-27 00:02:31,374][105620] Updated weights for policy 1, policy_version 1190855 (0.0008) [2023-12-27 00:02:31,382][105692] Updated weights for policy 0, policy_version 1189523 (0.0009) [2023-12-27 00:02:31,460][105692] Updated weights for policy 0, policy_version 1189533 (0.0008) [2023-12-27 00:02:31,529][105692] Updated weights for policy 0, policy_version 1189543 (0.0009) [2023-12-27 00:02:32,126][105620] Updated weights for policy 1, policy_version 1190865 (0.0007) [2023-12-27 00:02:32,183][105620] Updated weights for policy 1, policy_version 1190875 (0.0006) [2023-12-27 00:02:32,250][105620] Updated weights for policy 1, policy_version 1190885 (0.0010) [2023-12-27 00:02:32,290][105692] Updated weights for policy 0, policy_version 1189553 (0.0008) [2023-12-27 00:02:32,352][105692] Updated weights for policy 0, policy_version 1189563 (0.0008) [2023-12-27 00:02:32,415][105692] Updated weights for policy 0, policy_version 1189573 (0.0009) [2023-12-27 00:02:33,021][105620] Updated weights for policy 1, policy_version 1190895 (0.0010) [2023-12-27 00:02:33,082][105620] Updated weights for policy 1, policy_version 1190905 (0.0011) [2023-12-27 00:02:33,146][105620] Updated weights for policy 1, policy_version 1190915 (0.0011) [2023-12-27 00:02:33,196][105692] Updated weights for policy 0, policy_version 1189583 (0.0008) [2023-12-27 00:02:33,252][105692] Updated weights for policy 0, policy_version 1189593 (0.0008) [2023-12-27 00:02:33,309][105692] Updated weights for policy 0, policy_version 1189603 (0.0008) [2023-12-27 00:02:33,853][105620] Updated weights for policy 1, policy_version 1190925 (0.0011) [2023-12-27 00:02:33,915][105620] Updated weights for policy 1, policy_version 1190935 (0.0010) [2023-12-27 00:02:33,971][105620] Updated weights for policy 1, policy_version 1190945 (0.0009) [2023-12-27 00:02:34,137][105692] Updated weights for policy 0, policy_version 1189613 (0.0007) [2023-12-27 00:02:34,211][105692] Updated weights for policy 0, policy_version 1189623 (0.0007) [2023-12-27 00:02:34,278][105692] Updated weights for policy 0, policy_version 1189633 (0.0008) [2023-12-27 00:02:34,745][105620] Updated weights for policy 1, policy_version 1190955 (0.0009) [2023-12-27 00:02:34,812][105620] Updated weights for policy 1, policy_version 1190965 (0.0011) [2023-12-27 00:02:34,877][105620] Updated weights for policy 1, policy_version 1190975 (0.0011) [2023-12-27 00:02:35,069][105692] Updated weights for policy 0, policy_version 1189643 (0.0008) [2023-12-27 00:02:35,139][105692] Updated weights for policy 0, policy_version 1189653 (0.0008) [2023-12-27 00:02:35,203][105692] Updated weights for policy 0, policy_version 1189663 (0.0007) [2023-12-27 00:02:35,610][105620] Updated weights for policy 1, policy_version 1190985 (0.0011) [2023-12-27 00:02:35,671][105620] Updated weights for policy 1, policy_version 1190995 (0.0011) [2023-12-27 00:02:35,731][105620] Updated weights for policy 1, policy_version 1191005 (0.0011) [2023-12-27 00:02:35,790][105620] Updated weights for policy 1, policy_version 1191015 (0.0010) [2023-12-27 00:02:35,963][105692] Updated weights for policy 0, policy_version 1189673 (0.0008) [2023-12-27 00:02:36,022][105692] Updated weights for policy 0, policy_version 1189683 (0.0009) [2023-12-27 00:02:36,062][104569] Fps is (10 sec: 17203.2, 60 sec: 17749.3, 300 sec: 18550.0). Total num frames: 609542144. Throughput: 0: 8806.0, 1: 8998.4. Samples: 609534584. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:02:36,062][104569] Avg episode reward: [(0, '9357.713'), (1, '8899.560')] [2023-12-27 00:02:36,086][105692] Updated weights for policy 0, policy_version 1189693 (0.0010) [2023-12-27 00:02:36,150][105692] Updated weights for policy 0, policy_version 1189703 (0.0007) [2023-12-27 00:02:36,536][105620] Updated weights for policy 1, policy_version 1191025 (0.0010) [2023-12-27 00:02:36,600][105620] Updated weights for policy 1, policy_version 1191035 (0.0009) [2023-12-27 00:02:36,664][105620] Updated weights for policy 1, policy_version 1191045 (0.0010) [2023-12-27 00:02:36,982][105692] Updated weights for policy 0, policy_version 1189713 (0.0007) [2023-12-27 00:02:37,045][105692] Updated weights for policy 0, policy_version 1189723 (0.0009) [2023-12-27 00:02:37,106][105692] Updated weights for policy 0, policy_version 1189733 (0.0008) [2023-12-27 00:02:37,401][105620] Updated weights for policy 1, policy_version 1191055 (0.0011) [2023-12-27 00:02:37,467][105620] Updated weights for policy 1, policy_version 1191065 (0.0011) [2023-12-27 00:02:37,528][105620] Updated weights for policy 1, policy_version 1191075 (0.0011) [2023-12-27 00:02:37,979][105692] Updated weights for policy 0, policy_version 1189743 (0.0008) [2023-12-27 00:02:38,049][105692] Updated weights for policy 0, policy_version 1189753 (0.0008) [2023-12-27 00:02:38,114][105692] Updated weights for policy 0, policy_version 1189763 (0.0008) [2023-12-27 00:02:38,316][105620] Updated weights for policy 1, policy_version 1191085 (0.0010) [2023-12-27 00:02:38,390][105620] Updated weights for policy 1, policy_version 1191095 (0.0009) [2023-12-27 00:02:38,457][105620] Updated weights for policy 1, policy_version 1191105 (0.0010) [2023-12-27 00:02:38,895][105692] Updated weights for policy 0, policy_version 1189773 (0.0009) [2023-12-27 00:02:38,957][105692] Updated weights for policy 0, policy_version 1189783 (0.0009) [2023-12-27 00:02:39,016][105692] Updated weights for policy 0, policy_version 1189793 (0.0010) [2023-12-27 00:02:39,206][105620] Updated weights for policy 1, policy_version 1191115 (0.0009) [2023-12-27 00:02:39,276][105620] Updated weights for policy 1, policy_version 1191125 (0.0008) [2023-12-27 00:02:39,343][105620] Updated weights for policy 1, policy_version 1191135 (0.0008) [2023-12-27 00:02:39,880][105692] Updated weights for policy 0, policy_version 1189803 (0.0009) [2023-12-27 00:02:39,958][105692] Updated weights for policy 0, policy_version 1189813 (0.0008) [2023-12-27 00:02:40,028][105692] Updated weights for policy 0, policy_version 1189823 (0.0007) [2023-12-27 00:02:40,160][105620] Updated weights for policy 1, policy_version 1191145 (0.0009) [2023-12-27 00:02:40,220][105620] Updated weights for policy 1, policy_version 1191155 (0.0009) [2023-12-27 00:02:40,290][105620] Updated weights for policy 1, policy_version 1191165 (0.0009) [2023-12-27 00:02:40,361][105620] Updated weights for policy 1, policy_version 1191175 (0.0007) [2023-12-27 00:02:40,791][105692] Updated weights for policy 0, policy_version 1189833 (0.0009) [2023-12-27 00:02:40,855][105692] Updated weights for policy 0, policy_version 1189843 (0.0008) [2023-12-27 00:02:40,916][105692] Updated weights for policy 0, policy_version 1189853 (0.0011) [2023-12-27 00:02:40,976][105692] Updated weights for policy 0, policy_version 1189863 (0.0011) [2023-12-27 00:02:41,062][104569] Fps is (10 sec: 18022.3, 60 sec: 17885.8, 300 sec: 18550.0). Total num frames: 609632256. Throughput: 0: 8753.5, 1: 9023.2. Samples: 609640940. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:02:41,063][104569] Avg episode reward: [(0, '9177.869'), (1, '8809.952')] [2023-12-27 00:02:41,086][105620] Updated weights for policy 1, policy_version 1191185 (0.0010) [2023-12-27 00:02:41,159][105620] Updated weights for policy 1, policy_version 1191195 (0.0011) [2023-12-27 00:02:41,222][105620] Updated weights for policy 1, policy_version 1191205 (0.0008) [2023-12-27 00:02:41,777][105692] Updated weights for policy 0, policy_version 1189873 (0.0011) [2023-12-27 00:02:41,849][105692] Updated weights for policy 0, policy_version 1189883 (0.0009) [2023-12-27 00:02:41,911][105692] Updated weights for policy 0, policy_version 1189893 (0.0008) [2023-12-27 00:02:42,014][105620] Updated weights for policy 1, policy_version 1191215 (0.0009) [2023-12-27 00:02:42,088][105620] Updated weights for policy 1, policy_version 1191225 (0.0007) [2023-12-27 00:02:42,156][105620] Updated weights for policy 1, policy_version 1191235 (0.0008) [2023-12-27 00:02:42,744][105692] Updated weights for policy 0, policy_version 1189903 (0.0009) [2023-12-27 00:02:42,808][105692] Updated weights for policy 0, policy_version 1189913 (0.0009) [2023-12-27 00:02:42,875][105692] Updated weights for policy 0, policy_version 1189923 (0.0009) [2023-12-27 00:02:42,974][105620] Updated weights for policy 1, policy_version 1191245 (0.0010) [2023-12-27 00:02:43,042][105620] Updated weights for policy 1, policy_version 1191255 (0.0010) [2023-12-27 00:02:43,099][105620] Updated weights for policy 1, policy_version 1191265 (0.0009) [2023-12-27 00:02:43,683][105692] Updated weights for policy 0, policy_version 1189933 (0.0009) [2023-12-27 00:02:43,747][105692] Updated weights for policy 0, policy_version 1189943 (0.0008) [2023-12-27 00:02:43,814][105692] Updated weights for policy 0, policy_version 1189953 (0.0008) [2023-12-27 00:02:43,861][105620] Updated weights for policy 1, policy_version 1191275 (0.0008) [2023-12-27 00:02:43,925][105620] Updated weights for policy 1, policy_version 1191285 (0.0009) [2023-12-27 00:02:43,985][105620] Updated weights for policy 1, policy_version 1191295 (0.0008) [2023-12-27 00:02:44,519][105692] Updated weights for policy 0, policy_version 1189963 (0.0009) [2023-12-27 00:02:44,584][105692] Updated weights for policy 0, policy_version 1189973 (0.0007) [2023-12-27 00:02:44,646][105692] Updated weights for policy 0, policy_version 1189983 (0.0007) [2023-12-27 00:02:44,812][105620] Updated weights for policy 1, policy_version 1191305 (0.0009) [2023-12-27 00:02:44,883][105620] Updated weights for policy 1, policy_version 1191315 (0.0008) [2023-12-27 00:02:44,944][105620] Updated weights for policy 1, policy_version 1191325 (0.0008) [2023-12-27 00:02:45,010][105620] Updated weights for policy 1, policy_version 1191335 (0.0008) [2023-12-27 00:02:45,424][105692] Updated weights for policy 0, policy_version 1189993 (0.0007) [2023-12-27 00:02:45,494][105692] Updated weights for policy 0, policy_version 1190003 (0.0009) [2023-12-27 00:02:45,551][105692] Updated weights for policy 0, policy_version 1190013 (0.0009) [2023-12-27 00:02:45,615][105692] Updated weights for policy 0, policy_version 1190023 (0.0010) [2023-12-27 00:02:45,775][105620] Updated weights for policy 1, policy_version 1191345 (0.0009) [2023-12-27 00:02:45,840][105620] Updated weights for policy 1, policy_version 1191355 (0.0009) [2023-12-27 00:02:45,904][105620] Updated weights for policy 1, policy_version 1191365 (0.0008) [2023-12-27 00:02:46,062][104569] Fps is (10 sec: 18022.1, 60 sec: 17749.3, 300 sec: 18494.5). Total num frames: 609722368. Throughput: 0: 8726.1, 1: 9002.7. Samples: 609693288. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:02:46,063][104569] Avg episode reward: [(0, '8909.021'), (1, '8720.223')] [2023-12-27 00:02:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001190024_304693248.pth... [2023-12-27 00:02:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001191368_305029120.pth... [2023-12-27 00:02:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001189000_304431104.pth [2023-12-27 00:02:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001190312_304758784.pth [2023-12-27 00:02:46,383][105692] Updated weights for policy 0, policy_version 1190033 (0.0006) [2023-12-27 00:02:46,438][105692] Updated weights for policy 0, policy_version 1190043 (0.0007) [2023-12-27 00:02:46,497][105692] Updated weights for policy 0, policy_version 1190053 (0.0008) [2023-12-27 00:02:46,619][105620] Updated weights for policy 1, policy_version 1191375 (0.0008) [2023-12-27 00:02:46,681][105620] Updated weights for policy 1, policy_version 1191385 (0.0009) [2023-12-27 00:02:46,737][105620] Updated weights for policy 1, policy_version 1191395 (0.0009) [2023-12-27 00:02:47,242][105692] Updated weights for policy 0, policy_version 1190063 (0.0008) [2023-12-27 00:02:47,307][105692] Updated weights for policy 0, policy_version 1190073 (0.0009) [2023-12-27 00:02:47,372][105692] Updated weights for policy 0, policy_version 1190083 (0.0009) [2023-12-27 00:02:47,535][105620] Updated weights for policy 1, policy_version 1191405 (0.0008) [2023-12-27 00:02:47,596][105620] Updated weights for policy 1, policy_version 1191415 (0.0006) [2023-12-27 00:02:47,651][105620] Updated weights for policy 1, policy_version 1191425 (0.0006) [2023-12-27 00:02:48,118][105692] Updated weights for policy 0, policy_version 1190093 (0.0009) [2023-12-27 00:02:48,190][105692] Updated weights for policy 0, policy_version 1190103 (0.0010) [2023-12-27 00:02:48,260][105692] Updated weights for policy 0, policy_version 1190113 (0.0010) [2023-12-27 00:02:48,413][105620] Updated weights for policy 1, policy_version 1191435 (0.0008) [2023-12-27 00:02:48,481][105620] Updated weights for policy 1, policy_version 1191445 (0.0009) [2023-12-27 00:02:48,545][105620] Updated weights for policy 1, policy_version 1191455 (0.0009) [2023-12-27 00:02:49,051][105692] Updated weights for policy 0, policy_version 1190123 (0.0009) [2023-12-27 00:02:49,116][105692] Updated weights for policy 0, policy_version 1190133 (0.0010) [2023-12-27 00:02:49,185][105692] Updated weights for policy 0, policy_version 1190143 (0.0009) [2023-12-27 00:02:49,272][105620] Updated weights for policy 1, policy_version 1191465 (0.0009) [2023-12-27 00:02:49,340][105620] Updated weights for policy 1, policy_version 1191475 (0.0008) [2023-12-27 00:02:49,406][105620] Updated weights for policy 1, policy_version 1191485 (0.0009) [2023-12-27 00:02:49,473][105620] Updated weights for policy 1, policy_version 1191495 (0.0009) [2023-12-27 00:02:50,041][105692] Updated weights for policy 0, policy_version 1190153 (0.0009) [2023-12-27 00:02:50,094][105692] Updated weights for policy 0, policy_version 1190163 (0.0010) [2023-12-27 00:02:50,155][105692] Updated weights for policy 0, policy_version 1190173 (0.0008) [2023-12-27 00:02:50,192][105620] Updated weights for policy 1, policy_version 1191505 (0.0008) [2023-12-27 00:02:50,214][105692] Updated weights for policy 0, policy_version 1190183 (0.0009) [2023-12-27 00:02:50,253][105620] Updated weights for policy 1, policy_version 1191515 (0.0007) [2023-12-27 00:02:50,323][105620] Updated weights for policy 1, policy_version 1191525 (0.0008) [2023-12-27 00:02:51,005][105692] Updated weights for policy 0, policy_version 1190193 (0.0006) [2023-12-27 00:02:51,062][104569] Fps is (10 sec: 17203.4, 60 sec: 17749.3, 300 sec: 18438.9). Total num frames: 609804288. Throughput: 0: 8764.8, 1: 9009.3. Samples: 609802516. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:02:51,062][104569] Avg episode reward: [(0, '8737.963'), (1, '8718.807')] [2023-12-27 00:02:51,073][105620] Updated weights for policy 1, policy_version 1191535 (0.0009) [2023-12-27 00:02:51,086][105692] Updated weights for policy 0, policy_version 1190203 (0.0009) [2023-12-27 00:02:51,156][105692] Updated weights for policy 0, policy_version 1190213 (0.0008) [2023-12-27 00:02:51,158][105620] Updated weights for policy 1, policy_version 1191545 (0.0009) [2023-12-27 00:02:51,225][105620] Updated weights for policy 1, policy_version 1191555 (0.0008) [2023-12-27 00:02:51,936][105620] Updated weights for policy 1, policy_version 1191565 (0.0010) [2023-12-27 00:02:51,976][105692] Updated weights for policy 0, policy_version 1190223 (0.0008) [2023-12-27 00:02:52,001][105620] Updated weights for policy 1, policy_version 1191575 (0.0011) [2023-12-27 00:02:52,037][105692] Updated weights for policy 0, policy_version 1190233 (0.0009) [2023-12-27 00:02:52,058][105620] Updated weights for policy 1, policy_version 1191585 (0.0011) [2023-12-27 00:02:52,094][105692] Updated weights for policy 0, policy_version 1190243 (0.0006) [2023-12-27 00:02:52,809][105620] Updated weights for policy 1, policy_version 1191595 (0.0011) [2023-12-27 00:02:52,878][105620] Updated weights for policy 1, policy_version 1191605 (0.0011) [2023-12-27 00:02:52,909][105692] Updated weights for policy 0, policy_version 1190253 (0.0007) [2023-12-27 00:02:52,936][105620] Updated weights for policy 1, policy_version 1191615 (0.0010) [2023-12-27 00:02:52,965][105692] Updated weights for policy 0, policy_version 1190263 (0.0006) [2023-12-27 00:02:53,030][105692] Updated weights for policy 0, policy_version 1190273 (0.0006) [2023-12-27 00:02:53,655][105620] Updated weights for policy 1, policy_version 1191625 (0.0011) [2023-12-27 00:02:53,722][105620] Updated weights for policy 1, policy_version 1191635 (0.0009) [2023-12-27 00:02:53,780][105620] Updated weights for policy 1, policy_version 1191645 (0.0008) [2023-12-27 00:02:53,788][105692] Updated weights for policy 0, policy_version 1190283 (0.0006) [2023-12-27 00:02:53,843][105692] Updated weights for policy 0, policy_version 1190293 (0.0009) [2023-12-27 00:02:53,845][105620] Updated weights for policy 1, policy_version 1191655 (0.0009) [2023-12-27 00:02:53,896][105692] Updated weights for policy 0, policy_version 1190303 (0.0005) [2023-12-27 00:02:54,590][105620] Updated weights for policy 1, policy_version 1191665 (0.0010) [2023-12-27 00:02:54,658][105620] Updated weights for policy 1, policy_version 1191675 (0.0008) [2023-12-27 00:02:54,660][105692] Updated weights for policy 0, policy_version 1190313 (0.0009) [2023-12-27 00:02:54,720][105620] Updated weights for policy 1, policy_version 1191685 (0.0011) [2023-12-27 00:02:54,722][105692] Updated weights for policy 0, policy_version 1190323 (0.0006) [2023-12-27 00:02:54,783][105692] Updated weights for policy 0, policy_version 1190333 (0.0007) [2023-12-27 00:02:54,844][105692] Updated weights for policy 0, policy_version 1190344 (0.0009) [2023-12-27 00:02:55,524][105620] Updated weights for policy 1, policy_version 1191695 (0.0011) [2023-12-27 00:02:55,574][105620] Updated weights for policy 1, policy_version 1191705 (0.0011) [2023-12-27 00:02:55,621][105692] Updated weights for policy 0, policy_version 1190354 (0.0007) [2023-12-27 00:02:55,635][105620] Updated weights for policy 1, policy_version 1191715 (0.0011) [2023-12-27 00:02:55,679][105692] Updated weights for policy 0, policy_version 1190364 (0.0007) [2023-12-27 00:02:55,767][105692] Updated weights for policy 0, policy_version 1190374 (0.0008) [2023-12-27 00:02:56,062][104569] Fps is (10 sec: 18022.7, 60 sec: 17749.4, 300 sec: 18466.7). Total num frames: 609902592. Throughput: 0: 8783.7, 1: 9019.7. Samples: 609910516. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:02:56,062][104569] Avg episode reward: [(0, '8826.238'), (1, '8806.971')] [2023-12-27 00:02:56,413][105620] Updated weights for policy 1, policy_version 1191725 (0.0011) [2023-12-27 00:02:56,473][105620] Updated weights for policy 1, policy_version 1191735 (0.0011) [2023-12-27 00:02:56,524][105692] Updated weights for policy 0, policy_version 1190384 (0.0009) [2023-12-27 00:02:56,528][105620] Updated weights for policy 1, policy_version 1191745 (0.0011) [2023-12-27 00:02:56,581][105692] Updated weights for policy 0, policy_version 1190394 (0.0008) [2023-12-27 00:02:56,639][105692] Updated weights for policy 0, policy_version 1190404 (0.0009) [2023-12-27 00:02:57,303][105620] Updated weights for policy 1, policy_version 1191755 (0.0011) [2023-12-27 00:02:57,343][105586] KL-divergence is very high: 120.6353 [2023-12-27 00:02:57,354][105620] Updated weights for policy 1, policy_version 1191765 (0.0011) [2023-12-27 00:02:57,384][105586] KL-divergence is very high: 204.1226 [2023-12-27 00:02:57,402][105620] Updated weights for policy 1, policy_version 1191775 (0.0010) [2023-12-27 00:02:57,404][105692] Updated weights for policy 0, policy_version 1190414 (0.0008) [2023-12-27 00:02:57,427][105586] KL-divergence is very high: 197.6374 [2023-12-27 00:02:57,462][105692] Updated weights for policy 0, policy_version 1190424 (0.0011) [2023-12-27 00:02:57,524][105692] Updated weights for policy 0, policy_version 1190434 (0.0011) [2023-12-27 00:02:58,175][105620] Updated weights for policy 1, policy_version 1191785 (0.0011) [2023-12-27 00:02:58,231][105620] Updated weights for policy 1, policy_version 1191795 (0.0010) [2023-12-27 00:02:58,249][105692] Updated weights for policy 0, policy_version 1190444 (0.0009) [2023-12-27 00:02:58,297][105620] Updated weights for policy 1, policy_version 1191805 (0.0010) [2023-12-27 00:02:58,313][105692] Updated weights for policy 0, policy_version 1190454 (0.0007) [2023-12-27 00:02:58,365][105620] Updated weights for policy 1, policy_version 1191815 (0.0011) [2023-12-27 00:02:58,383][105692] Updated weights for policy 0, policy_version 1190464 (0.0008) [2023-12-27 00:02:59,219][105620] Updated weights for policy 1, policy_version 1191825 (0.0009) [2023-12-27 00:02:59,292][105620] Updated weights for policy 1, policy_version 1191835 (0.0012) [2023-12-27 00:02:59,360][105692] Updated weights for policy 0, policy_version 1190474 (0.0008) [2023-12-27 00:02:59,368][105620] Updated weights for policy 1, policy_version 1191845 (0.0008) [2023-12-27 00:02:59,430][105692] Updated weights for policy 0, policy_version 1190484 (0.0011) [2023-12-27 00:02:59,491][105692] Updated weights for policy 0, policy_version 1190494 (0.0009) [2023-12-27 00:02:59,556][105692] Updated weights for policy 0, policy_version 1190504 (0.0006) [2023-12-27 00:03:00,129][105620] Updated weights for policy 1, policy_version 1191855 (0.0008) [2023-12-27 00:03:00,199][105620] Updated weights for policy 1, policy_version 1191865 (0.0008) [2023-12-27 00:03:00,239][105692] Updated weights for policy 0, policy_version 1190514 (0.0006) [2023-12-27 00:03:00,260][105620] Updated weights for policy 1, policy_version 1191875 (0.0008) [2023-12-27 00:03:00,305][105692] Updated weights for policy 0, policy_version 1190524 (0.0006) [2023-12-27 00:03:00,329][105585] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000009 [2023-12-27 00:03:00,955][105620] Updated weights for policy 1, policy_version 1191885 (0.0007) [2023-12-27 00:03:01,023][105620] Updated weights for policy 1, policy_version 1191895 (0.0008) [2023-12-27 00:03:01,062][104569] Fps is (10 sec: 18022.4, 60 sec: 17749.3, 300 sec: 18411.2). Total num frames: 609984512. Throughput: 0: 8827.7, 1: 9035.9. Samples: 609964440. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:03:01,062][104569] Avg episode reward: [(0, '8997.054'), (1, '8987.916')] [2023-12-27 00:03:01,064][105692] Updated weights for policy 0, policy_version 1190534 (0.0008) [2023-12-27 00:03:01,087][105620] Updated weights for policy 1, policy_version 1191905 (0.0008) [2023-12-27 00:03:01,124][105692] Updated weights for policy 0, policy_version 1190544 (0.0008) [2023-12-27 00:03:01,126][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001191912_305168384.pth... [2023-12-27 00:03:01,129][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001190824_304889856.pth [2023-12-27 00:03:01,184][105692] Updated weights for policy 0, policy_version 1190554 (0.0008) [2023-12-27 00:03:01,220][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001190560_304832512.pth... [2023-12-27 00:03:01,223][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001189512_304562176.pth [2023-12-27 00:03:01,871][105620] Updated weights for policy 1, policy_version 1191915 (0.0007) [2023-12-27 00:03:01,932][105692] Updated weights for policy 0, policy_version 1190564 (0.0007) [2023-12-27 00:03:01,941][105620] Updated weights for policy 1, policy_version 1191925 (0.0006) [2023-12-27 00:03:01,994][105692] Updated weights for policy 0, policy_version 1190574 (0.0006) [2023-12-27 00:03:02,004][105620] Updated weights for policy 1, policy_version 1191935 (0.0010) [2023-12-27 00:03:02,049][105692] Updated weights for policy 0, policy_version 1190584 (0.0006) [2023-12-27 00:03:02,706][105620] Updated weights for policy 1, policy_version 1191945 (0.0010) [2023-12-27 00:03:02,762][105620] Updated weights for policy 1, policy_version 1191955 (0.0008) [2023-12-27 00:03:02,775][105692] Updated weights for policy 0, policy_version 1190594 (0.0008) [2023-12-27 00:03:02,814][105620] Updated weights for policy 1, policy_version 1191965 (0.0007) [2023-12-27 00:03:02,841][105692] Updated weights for policy 0, policy_version 1190604 (0.0007) [2023-12-27 00:03:02,871][105620] Updated weights for policy 1, policy_version 1191975 (0.0006) [2023-12-27 00:03:02,902][105692] Updated weights for policy 0, policy_version 1190614 (0.0008) [2023-12-27 00:03:02,970][105692] Updated weights for policy 0, policy_version 1190624 (0.0009) [2023-12-27 00:03:03,641][105620] Updated weights for policy 1, policy_version 1191985 (0.0009) [2023-12-27 00:03:03,699][105620] Updated weights for policy 1, policy_version 1191995 (0.0007) [2023-12-27 00:03:03,717][105692] Updated weights for policy 0, policy_version 1190634 (0.0011) [2023-12-27 00:03:03,753][105620] Updated weights for policy 1, policy_version 1192005 (0.0005) [2023-12-27 00:03:03,779][105692] Updated weights for policy 0, policy_version 1190644 (0.0010) [2023-12-27 00:03:03,844][105692] Updated weights for policy 0, policy_version 1190654 (0.0010) [2023-12-27 00:03:04,451][105620] Updated weights for policy 1, policy_version 1192015 (0.0008) [2023-12-27 00:03:04,513][105620] Updated weights for policy 1, policy_version 1192025 (0.0008) [2023-12-27 00:03:04,565][105620] Updated weights for policy 1, policy_version 1192035 (0.0008) [2023-12-27 00:03:04,610][105692] Updated weights for policy 0, policy_version 1190664 (0.0009) [2023-12-27 00:03:04,668][105692] Updated weights for policy 0, policy_version 1190674 (0.0006) [2023-12-27 00:03:04,729][105692] Updated weights for policy 0, policy_version 1190684 (0.0006) [2023-12-27 00:03:05,261][105620] Updated weights for policy 1, policy_version 1192045 (0.0007) [2023-12-27 00:03:05,313][105620] Updated weights for policy 1, policy_version 1192055 (0.0008) [2023-12-27 00:03:05,362][105620] Updated weights for policy 1, policy_version 1192065 (0.0008) [2023-12-27 00:03:05,418][105692] Updated weights for policy 0, policy_version 1190694 (0.0006) [2023-12-27 00:03:05,481][105692] Updated weights for policy 0, policy_version 1190704 (0.0006) [2023-12-27 00:03:05,542][105692] Updated weights for policy 0, policy_version 1190714 (0.0009) [2023-12-27 00:03:06,062][104569] Fps is (10 sec: 18022.4, 60 sec: 17885.9, 300 sec: 18411.2). Total num frames: 610082816. Throughput: 0: 8860.8, 1: 9072.6. Samples: 610075680. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:03:06,062][104569] Avg episode reward: [(0, '8728.474'), (1, '8913.043')] [2023-12-27 00:03:06,113][105620] Updated weights for policy 1, policy_version 1192075 (0.0008) [2023-12-27 00:03:06,166][105620] Updated weights for policy 1, policy_version 1192085 (0.0006) [2023-12-27 00:03:06,229][105620] Updated weights for policy 1, policy_version 1192095 (0.0009) [2023-12-27 00:03:06,297][105692] Updated weights for policy 0, policy_version 1190724 (0.0010) [2023-12-27 00:03:06,360][105692] Updated weights for policy 0, policy_version 1190734 (0.0010) [2023-12-27 00:03:06,421][105692] Updated weights for policy 0, policy_version 1190744 (0.0010) [2023-12-27 00:03:06,967][105620] Updated weights for policy 1, policy_version 1192105 (0.0009) [2023-12-27 00:03:07,030][105620] Updated weights for policy 1, policy_version 1192115 (0.0009) [2023-12-27 00:03:07,093][105620] Updated weights for policy 1, policy_version 1192125 (0.0008) [2023-12-27 00:03:07,128][105692] Updated weights for policy 0, policy_version 1190754 (0.0008) [2023-12-27 00:03:07,157][105620] Updated weights for policy 1, policy_version 1192135 (0.0008) [2023-12-27 00:03:07,194][105692] Updated weights for policy 0, policy_version 1190764 (0.0008) [2023-12-27 00:03:07,249][105692] Updated weights for policy 0, policy_version 1190774 (0.0010) [2023-12-27 00:03:07,302][105692] Updated weights for policy 0, policy_version 1190784 (0.0009) [2023-12-27 00:03:07,842][105620] Updated weights for policy 1, policy_version 1192145 (0.0010) [2023-12-27 00:03:07,891][105620] Updated weights for policy 1, policy_version 1192155 (0.0011) [2023-12-27 00:03:07,938][105620] Updated weights for policy 1, policy_version 1192165 (0.0010) [2023-12-27 00:03:08,043][105692] Updated weights for policy 0, policy_version 1190794 (0.0006) [2023-12-27 00:03:08,105][105692] Updated weights for policy 0, policy_version 1190804 (0.0007) [2023-12-27 00:03:08,157][105692] Updated weights for policy 0, policy_version 1190814 (0.0010) [2023-12-27 00:03:08,736][105620] Updated weights for policy 1, policy_version 1192175 (0.0010) [2023-12-27 00:03:08,801][105620] Updated weights for policy 1, policy_version 1192185 (0.0011) [2023-12-27 00:03:08,840][105692] Updated weights for policy 0, policy_version 1190824 (0.0007) [2023-12-27 00:03:08,876][105620] Updated weights for policy 1, policy_version 1192195 (0.0011) [2023-12-27 00:03:08,910][105692] Updated weights for policy 0, policy_version 1190834 (0.0006) [2023-12-27 00:03:08,970][105692] Updated weights for policy 0, policy_version 1190844 (0.0009) [2023-12-27 00:03:09,505][105620] Updated weights for policy 1, policy_version 1192205 (0.0010) [2023-12-27 00:03:09,568][105620] Updated weights for policy 1, policy_version 1192215 (0.0009) [2023-12-27 00:03:09,640][105620] Updated weights for policy 1, policy_version 1192225 (0.0009) [2023-12-27 00:03:09,736][105692] Updated weights for policy 0, policy_version 1190854 (0.0011) [2023-12-27 00:03:09,804][105692] Updated weights for policy 0, policy_version 1190864 (0.0009) [2023-12-27 00:03:09,873][105692] Updated weights for policy 0, policy_version 1190874 (0.0011) [2023-12-27 00:03:10,420][105620] Updated weights for policy 1, policy_version 1192235 (0.0007) [2023-12-27 00:03:10,479][105620] Updated weights for policy 1, policy_version 1192245 (0.0009) [2023-12-27 00:03:10,548][105620] Updated weights for policy 1, policy_version 1192255 (0.0007) [2023-12-27 00:03:10,625][105692] Updated weights for policy 0, policy_version 1190884 (0.0009) [2023-12-27 00:03:10,679][105692] Updated weights for policy 0, policy_version 1190894 (0.0007) [2023-12-27 00:03:10,736][105692] Updated weights for policy 0, policy_version 1190904 (0.0008) [2023-12-27 00:03:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 18022.4, 300 sec: 18438.9). Total num frames: 610181120. Throughput: 0: 8976.8, 1: 9183.9. Samples: 610189596. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:03:11,063][104569] Avg episode reward: [(0, '8460.789'), (1, '9001.474')] [2023-12-27 00:03:11,298][105620] Updated weights for policy 1, policy_version 1192265 (0.0008) [2023-12-27 00:03:11,364][105620] Updated weights for policy 1, policy_version 1192275 (0.0010) [2023-12-27 00:03:11,422][105692] Updated weights for policy 0, policy_version 1190914 (0.0007) [2023-12-27 00:03:11,440][105620] Updated weights for policy 1, policy_version 1192285 (0.0009) [2023-12-27 00:03:11,457][105586] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000008 [2023-12-27 00:03:11,490][105692] Updated weights for policy 0, policy_version 1190924 (0.0006) [2023-12-27 00:03:11,561][105692] Updated weights for policy 0, policy_version 1190934 (0.0006) [2023-12-27 00:03:11,632][105692] Updated weights for policy 0, policy_version 1190944 (0.0007) [2023-12-27 00:03:12,293][105620] Updated weights for policy 1, policy_version 1192295 (0.0008) [2023-12-27 00:03:12,361][105620] Updated weights for policy 1, policy_version 1192305 (0.0009) [2023-12-27 00:03:12,372][105692] Updated weights for policy 0, policy_version 1190954 (0.0007) [2023-12-27 00:03:12,436][105620] Updated weights for policy 1, policy_version 1192315 (0.0009) [2023-12-27 00:03:12,440][105692] Updated weights for policy 0, policy_version 1190964 (0.0007) [2023-12-27 00:03:12,507][105692] Updated weights for policy 0, policy_version 1190974 (0.0006) [2023-12-27 00:03:13,165][105692] Updated weights for policy 0, policy_version 1190984 (0.0006) [2023-12-27 00:03:13,233][105692] Updated weights for policy 0, policy_version 1190994 (0.0007) [2023-12-27 00:03:13,277][105620] Updated weights for policy 1, policy_version 1192325 (0.0007) [2023-12-27 00:03:13,293][105692] Updated weights for policy 0, policy_version 1191004 (0.0009) [2023-12-27 00:03:13,335][105620] Updated weights for policy 1, policy_version 1192335 (0.0006) [2023-12-27 00:03:13,365][105586] KL-divergence is very high: 139.2751 [2023-12-27 00:03:13,393][105620] Updated weights for policy 1, policy_version 1192345 (0.0008) [2023-12-27 00:03:13,410][105586] KL-divergence is very high: 138.8508 [2023-12-27 00:03:13,977][105692] Updated weights for policy 0, policy_version 1191014 (0.0007) [2023-12-27 00:03:14,034][105692] Updated weights for policy 0, policy_version 1191024 (0.0009) [2023-12-27 00:03:14,071][105620] Updated weights for policy 1, policy_version 1192355 (0.0009) [2023-12-27 00:03:14,092][105692] Updated weights for policy 0, policy_version 1191034 (0.0006) [2023-12-27 00:03:14,132][105620] Updated weights for policy 1, policy_version 1192365 (0.0009) [2023-12-27 00:03:14,194][105620] Updated weights for policy 1, policy_version 1192375 (0.0008) [2023-12-27 00:03:14,733][105692] Updated weights for policy 0, policy_version 1191044 (0.0006) [2023-12-27 00:03:14,795][105692] Updated weights for policy 0, policy_version 1191054 (0.0008) [2023-12-27 00:03:14,863][105692] Updated weights for policy 0, policy_version 1191064 (0.0008) [2023-12-27 00:03:14,981][105620] Updated weights for policy 1, policy_version 1192385 (0.0007) [2023-12-27 00:03:15,053][105620] Updated weights for policy 1, policy_version 1192395 (0.0009) [2023-12-27 00:03:15,127][105620] Updated weights for policy 1, policy_version 1192405 (0.0008) [2023-12-27 00:03:15,181][105620] Updated weights for policy 1, policy_version 1192415 (0.0008) [2023-12-27 00:03:15,629][105692] Updated weights for policy 0, policy_version 1191074 (0.0008) [2023-12-27 00:03:15,684][105692] Updated weights for policy 0, policy_version 1191084 (0.0009) [2023-12-27 00:03:15,741][105692] Updated weights for policy 0, policy_version 1191094 (0.0009) [2023-12-27 00:03:15,806][105692] Updated weights for policy 0, policy_version 1191104 (0.0007) [2023-12-27 00:03:15,891][105620] Updated weights for policy 1, policy_version 1192425 (0.0010) [2023-12-27 00:03:15,943][105620] Updated weights for policy 1, policy_version 1192435 (0.0009) [2023-12-27 00:03:15,994][105620] Updated weights for policy 1, policy_version 1192445 (0.0009) [2023-12-27 00:03:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 18158.9, 300 sec: 18438.9). Total num frames: 610279424. Throughput: 0: 9001.8, 1: 9167.7. Samples: 610244880. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:03:16,062][104569] Avg episode reward: [(0, '8814.779'), (1, '8911.726')] [2023-12-27 00:03:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001191104_304971776.pth... [2023-12-27 00:03:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001192448_305307648.pth... [2023-12-27 00:03:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001191368_305029120.pth [2023-12-27 00:03:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001190024_304693248.pth [2023-12-27 00:03:16,496][105692] Updated weights for policy 0, policy_version 1191114 (0.0009) [2023-12-27 00:03:16,559][105692] Updated weights for policy 0, policy_version 1191124 (0.0008) [2023-12-27 00:03:16,615][105692] Updated weights for policy 0, policy_version 1191134 (0.0005) [2023-12-27 00:03:16,876][105620] Updated weights for policy 1, policy_version 1192455 (0.0009) [2023-12-27 00:03:16,937][105620] Updated weights for policy 1, policy_version 1192465 (0.0010) [2023-12-27 00:03:16,991][105620] Updated weights for policy 1, policy_version 1192475 (0.0010) [2023-12-27 00:03:17,192][105692] Updated weights for policy 0, policy_version 1191144 (0.0006) [2023-12-27 00:03:17,245][105692] Updated weights for policy 0, policy_version 1191154 (0.0008) [2023-12-27 00:03:17,298][105692] Updated weights for policy 0, policy_version 1191164 (0.0011) [2023-12-27 00:03:17,837][105620] Updated weights for policy 1, policy_version 1192485 (0.0010) [2023-12-27 00:03:17,895][105620] Updated weights for policy 1, policy_version 1192495 (0.0009) [2023-12-27 00:03:17,948][105620] Updated weights for policy 1, policy_version 1192505 (0.0007) [2023-12-27 00:03:17,953][105692] Updated weights for policy 0, policy_version 1191174 (0.0011) [2023-12-27 00:03:18,017][105692] Updated weights for policy 0, policy_version 1191184 (0.0010) [2023-12-27 00:03:18,077][105692] Updated weights for policy 0, policy_version 1191194 (0.0007) [2023-12-27 00:03:18,787][105620] Updated weights for policy 1, policy_version 1192515 (0.0006) [2023-12-27 00:03:18,797][105692] Updated weights for policy 0, policy_version 1191204 (0.0007) [2023-12-27 00:03:18,850][105620] Updated weights for policy 1, policy_version 1192525 (0.0010) [2023-12-27 00:03:18,860][105692] Updated weights for policy 0, policy_version 1191214 (0.0008) [2023-12-27 00:03:18,908][105620] Updated weights for policy 1, policy_version 1192535 (0.0008) [2023-12-27 00:03:18,918][105692] Updated weights for policy 0, policy_version 1191224 (0.0006) [2023-12-27 00:03:19,654][105692] Updated weights for policy 0, policy_version 1191234 (0.0007) [2023-12-27 00:03:19,718][105692] Updated weights for policy 0, policy_version 1191244 (0.0006) [2023-12-27 00:03:19,777][105692] Updated weights for policy 0, policy_version 1191254 (0.0006) [2023-12-27 00:03:19,807][105620] Updated weights for policy 1, policy_version 1192545 (0.0007) [2023-12-27 00:03:19,844][105692] Updated weights for policy 0, policy_version 1191264 (0.0008) [2023-12-27 00:03:19,878][105620] Updated weights for policy 1, policy_version 1192555 (0.0009) [2023-12-27 00:03:19,943][105620] Updated weights for policy 1, policy_version 1192565 (0.0008) [2023-12-27 00:03:20,008][105620] Updated weights for policy 1, policy_version 1192575 (0.0008) [2023-12-27 00:03:20,590][105692] Updated weights for policy 0, policy_version 1191274 (0.0011) [2023-12-27 00:03:20,654][105692] Updated weights for policy 0, policy_version 1191284 (0.0008) [2023-12-27 00:03:20,721][105692] Updated weights for policy 0, policy_version 1191294 (0.0008) [2023-12-27 00:03:20,804][105620] Updated weights for policy 1, policy_version 1192585 (0.0009) [2023-12-27 00:03:20,863][105620] Updated weights for policy 1, policy_version 1192595 (0.0009) [2023-12-27 00:03:20,929][105620] Updated weights for policy 1, policy_version 1192605 (0.0009) [2023-12-27 00:03:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 18295.5, 300 sec: 18411.2). Total num frames: 610369536. Throughput: 0: 9185.5, 1: 9092.2. Samples: 610357084. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:03:21,063][104569] Avg episode reward: [(0, '8994.673'), (1, '9080.839')] [2023-12-27 00:03:21,483][105692] Updated weights for policy 0, policy_version 1191304 (0.0007) [2023-12-27 00:03:21,537][105692] Updated weights for policy 0, policy_version 1191314 (0.0006) [2023-12-27 00:03:21,608][105692] Updated weights for policy 0, policy_version 1191324 (0.0006) [2023-12-27 00:03:21,742][105620] Updated weights for policy 1, policy_version 1192615 (0.0009) [2023-12-27 00:03:21,809][105620] Updated weights for policy 1, policy_version 1192625 (0.0009) [2023-12-27 00:03:21,877][105620] Updated weights for policy 1, policy_version 1192635 (0.0009) [2023-12-27 00:03:22,366][105692] Updated weights for policy 0, policy_version 1191334 (0.0009) [2023-12-27 00:03:22,429][105692] Updated weights for policy 0, policy_version 1191344 (0.0009) [2023-12-27 00:03:22,497][105692] Updated weights for policy 0, policy_version 1191354 (0.0009) [2023-12-27 00:03:22,667][105620] Updated weights for policy 1, policy_version 1192645 (0.0009) [2023-12-27 00:03:22,735][105620] Updated weights for policy 1, policy_version 1192655 (0.0009) [2023-12-27 00:03:22,795][105620] Updated weights for policy 1, policy_version 1192665 (0.0009) [2023-12-27 00:03:23,208][105692] Updated weights for policy 0, policy_version 1191364 (0.0008) [2023-12-27 00:03:23,276][105692] Updated weights for policy 0, policy_version 1191374 (0.0008) [2023-12-27 00:03:23,346][105692] Updated weights for policy 0, policy_version 1191384 (0.0007) [2023-12-27 00:03:23,566][105620] Updated weights for policy 1, policy_version 1192675 (0.0010) [2023-12-27 00:03:23,630][105620] Updated weights for policy 1, policy_version 1192685 (0.0009) [2023-12-27 00:03:23,687][105620] Updated weights for policy 1, policy_version 1192695 (0.0009) [2023-12-27 00:03:24,140][105692] Updated weights for policy 0, policy_version 1191394 (0.0009) [2023-12-27 00:03:24,202][105692] Updated weights for policy 0, policy_version 1191404 (0.0009) [2023-12-27 00:03:24,260][105692] Updated weights for policy 0, policy_version 1191414 (0.0009) [2023-12-27 00:03:24,323][105692] Updated weights for policy 0, policy_version 1191424 (0.0008) [2023-12-27 00:03:24,495][105620] Updated weights for policy 1, policy_version 1192705 (0.0010) [2023-12-27 00:03:24,554][105620] Updated weights for policy 1, policy_version 1192715 (0.0010) [2023-12-27 00:03:24,621][105620] Updated weights for policy 1, policy_version 1192725 (0.0010) [2023-12-27 00:03:24,684][105620] Updated weights for policy 1, policy_version 1192735 (0.0010) [2023-12-27 00:03:24,962][105692] Updated weights for policy 0, policy_version 1191434 (0.0009) [2023-12-27 00:03:25,020][105692] Updated weights for policy 0, policy_version 1191444 (0.0007) [2023-12-27 00:03:25,072][105692] Updated weights for policy 0, policy_version 1191454 (0.0009) [2023-12-27 00:03:25,554][105620] Updated weights for policy 1, policy_version 1192745 (0.0009) [2023-12-27 00:03:25,625][105620] Updated weights for policy 1, policy_version 1192755 (0.0007) [2023-12-27 00:03:25,689][105620] Updated weights for policy 1, policy_version 1192765 (0.0005) [2023-12-27 00:03:25,827][105692] Updated weights for policy 0, policy_version 1191464 (0.0010) [2023-12-27 00:03:25,885][105692] Updated weights for policy 0, policy_version 1191474 (0.0010) [2023-12-27 00:03:25,944][105692] Updated weights for policy 0, policy_version 1191484 (0.0010) [2023-12-27 00:03:26,062][104569] Fps is (10 sec: 18021.7, 60 sec: 18158.8, 300 sec: 18355.6). Total num frames: 610459648. Throughput: 0: 9277.5, 1: 9042.8. Samples: 610465356. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:03:26,063][104569] Avg episode reward: [(0, '9176.766'), (1, '9077.253')] [2023-12-27 00:03:26,395][105620] Updated weights for policy 1, policy_version 1192775 (0.0007) [2023-12-27 00:03:26,448][105620] Updated weights for policy 1, policy_version 1192785 (0.0008) [2023-12-27 00:03:26,501][105620] Updated weights for policy 1, policy_version 1192795 (0.0009) [2023-12-27 00:03:26,703][105692] Updated weights for policy 0, policy_version 1191494 (0.0010) [2023-12-27 00:03:26,753][105692] Updated weights for policy 0, policy_version 1191504 (0.0011) [2023-12-27 00:03:26,819][105692] Updated weights for policy 0, policy_version 1191514 (0.0010) [2023-12-27 00:03:27,298][105620] Updated weights for policy 1, policy_version 1192805 (0.0007) [2023-12-27 00:03:27,359][105620] Updated weights for policy 1, policy_version 1192815 (0.0009) [2023-12-27 00:03:27,415][105620] Updated weights for policy 1, policy_version 1192825 (0.0008) [2023-12-27 00:03:27,544][105692] Updated weights for policy 0, policy_version 1191524 (0.0011) [2023-12-27 00:03:27,604][105692] Updated weights for policy 0, policy_version 1191534 (0.0010) [2023-12-27 00:03:27,663][105692] Updated weights for policy 0, policy_version 1191544 (0.0011) [2023-12-27 00:03:28,161][105620] Updated weights for policy 1, policy_version 1192835 (0.0009) [2023-12-27 00:03:28,223][105620] Updated weights for policy 1, policy_version 1192845 (0.0007) [2023-12-27 00:03:28,283][105620] Updated weights for policy 1, policy_version 1192855 (0.0009) [2023-12-27 00:03:28,443][105692] Updated weights for policy 0, policy_version 1191554 (0.0010) [2023-12-27 00:03:28,505][105692] Updated weights for policy 0, policy_version 1191564 (0.0008) [2023-12-27 00:03:28,569][105692] Updated weights for policy 0, policy_version 1191574 (0.0007) [2023-12-27 00:03:28,633][105692] Updated weights for policy 0, policy_version 1191584 (0.0009) [2023-12-27 00:03:29,053][105620] Updated weights for policy 1, policy_version 1192865 (0.0009) [2023-12-27 00:03:29,114][105620] Updated weights for policy 1, policy_version 1192875 (0.0009) [2023-12-27 00:03:29,173][105620] Updated weights for policy 1, policy_version 1192885 (0.0009) [2023-12-27 00:03:29,239][105620] Updated weights for policy 1, policy_version 1192895 (0.0009) [2023-12-27 00:03:29,358][105692] Updated weights for policy 0, policy_version 1191594 (0.0009) [2023-12-27 00:03:29,426][105692] Updated weights for policy 0, policy_version 1191604 (0.0008) [2023-12-27 00:03:29,486][105692] Updated weights for policy 0, policy_version 1191614 (0.0010) [2023-12-27 00:03:30,026][105620] Updated weights for policy 1, policy_version 1192905 (0.0009) [2023-12-27 00:03:30,086][105620] Updated weights for policy 1, policy_version 1192915 (0.0009) [2023-12-27 00:03:30,147][105620] Updated weights for policy 1, policy_version 1192925 (0.0009) [2023-12-27 00:03:30,268][105692] Updated weights for policy 0, policy_version 1191624 (0.0009) [2023-12-27 00:03:30,327][105692] Updated weights for policy 0, policy_version 1191634 (0.0008) [2023-12-27 00:03:30,389][105692] Updated weights for policy 0, policy_version 1191644 (0.0007) [2023-12-27 00:03:30,913][105620] Updated weights for policy 1, policy_version 1192935 (0.0009) [2023-12-27 00:03:30,964][105620] Updated weights for policy 1, policy_version 1192945 (0.0009) [2023-12-27 00:03:31,010][105620] Updated weights for policy 1, policy_version 1192955 (0.0008) [2023-12-27 00:03:31,062][104569] Fps is (10 sec: 18022.4, 60 sec: 18295.5, 300 sec: 18327.9). Total num frames: 610549760. Throughput: 0: 9320.1, 1: 9066.7. Samples: 610520692. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:03:31,063][104569] Avg episode reward: [(0, '8810.632'), (1, '8894.271')] [2023-12-27 00:03:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001191648_305111040.pth... [2023-12-27 00:03:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001192960_305438720.pth... [2023-12-27 00:03:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001190560_304832512.pth [2023-12-27 00:03:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001191912_305168384.pth [2023-12-27 00:03:31,148][105692] Updated weights for policy 0, policy_version 1191654 (0.0010) [2023-12-27 00:03:31,206][105692] Updated weights for policy 0, policy_version 1191664 (0.0009) [2023-12-27 00:03:31,264][105692] Updated weights for policy 0, policy_version 1191674 (0.0009) [2023-12-27 00:03:31,850][105620] Updated weights for policy 1, policy_version 1192965 (0.0010) [2023-12-27 00:03:31,913][105620] Updated weights for policy 1, policy_version 1192975 (0.0010) [2023-12-27 00:03:31,970][105620] Updated weights for policy 1, policy_version 1192985 (0.0009) [2023-12-27 00:03:31,979][105692] Updated weights for policy 0, policy_version 1191684 (0.0007) [2023-12-27 00:03:32,042][105692] Updated weights for policy 0, policy_version 1191694 (0.0007) [2023-12-27 00:03:32,101][105692] Updated weights for policy 0, policy_version 1191704 (0.0009) [2023-12-27 00:03:32,793][105620] Updated weights for policy 1, policy_version 1192995 (0.0009) [2023-12-27 00:03:32,799][105692] Updated weights for policy 0, policy_version 1191714 (0.0008) [2023-12-27 00:03:32,857][105620] Updated weights for policy 1, policy_version 1193005 (0.0010) [2023-12-27 00:03:32,857][105692] Updated weights for policy 0, policy_version 1191724 (0.0006) [2023-12-27 00:03:32,919][105586] KL-divergence is very high: 123.7603 [2023-12-27 00:03:32,919][105620] Updated weights for policy 1, policy_version 1193015 (0.0006) [2023-12-27 00:03:32,921][105692] Updated weights for policy 0, policy_version 1191734 (0.0010) [2023-12-27 00:03:32,966][105586] KL-divergence is very high: 164.1988 [2023-12-27 00:03:32,981][105692] Updated weights for policy 0, policy_version 1191744 (0.0011) [2023-12-27 00:03:33,690][105620] Updated weights for policy 1, policy_version 1193025 (0.0006) [2023-12-27 00:03:33,698][105692] Updated weights for policy 0, policy_version 1191754 (0.0011) [2023-12-27 00:03:33,744][105620] Updated weights for policy 1, policy_version 1193035 (0.0007) [2023-12-27 00:03:33,759][105692] Updated weights for policy 0, policy_version 1191764 (0.0008) [2023-12-27 00:03:33,799][105620] Updated weights for policy 1, policy_version 1193045 (0.0008) [2023-12-27 00:03:33,820][105692] Updated weights for policy 0, policy_version 1191774 (0.0011) [2023-12-27 00:03:33,862][105620] Updated weights for policy 1, policy_version 1193055 (0.0009) [2023-12-27 00:03:34,530][105692] Updated weights for policy 0, policy_version 1191784 (0.0011) [2023-12-27 00:03:34,596][105692] Updated weights for policy 0, policy_version 1191794 (0.0011) [2023-12-27 00:03:34,663][105692] Updated weights for policy 0, policy_version 1191804 (0.0007) [2023-12-27 00:03:34,681][105620] Updated weights for policy 1, policy_version 1193065 (0.0007) [2023-12-27 00:03:34,740][105620] Updated weights for policy 1, policy_version 1193075 (0.0008) [2023-12-27 00:03:34,799][105620] Updated weights for policy 1, policy_version 1193085 (0.0008) [2023-12-27 00:03:35,371][105692] Updated weights for policy 0, policy_version 1191814 (0.0011) [2023-12-27 00:03:35,430][105692] Updated weights for policy 0, policy_version 1191824 (0.0011) [2023-12-27 00:03:35,486][105692] Updated weights for policy 0, policy_version 1191834 (0.0011) [2023-12-27 00:03:35,598][105620] Updated weights for policy 1, policy_version 1193095 (0.0008) [2023-12-27 00:03:35,659][105620] Updated weights for policy 1, policy_version 1193105 (0.0008) [2023-12-27 00:03:35,717][105620] Updated weights for policy 1, policy_version 1193115 (0.0008) [2023-12-27 00:03:36,062][104569] Fps is (10 sec: 18023.1, 60 sec: 18295.5, 300 sec: 18300.1). Total num frames: 610639872. Throughput: 0: 9373.8, 1: 9015.0. Samples: 610630012. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:03:36,062][104569] Avg episode reward: [(0, '8717.542'), (1, '8635.688')] [2023-12-27 00:03:36,280][105692] Updated weights for policy 0, policy_version 1191844 (0.0011) [2023-12-27 00:03:36,347][105692] Updated weights for policy 0, policy_version 1191854 (0.0011) [2023-12-27 00:03:36,407][105692] Updated weights for policy 0, policy_version 1191864 (0.0011) [2023-12-27 00:03:36,502][105620] Updated weights for policy 1, policy_version 1193125 (0.0008) [2023-12-27 00:03:36,560][105620] Updated weights for policy 1, policy_version 1193135 (0.0008) [2023-12-27 00:03:36,625][105620] Updated weights for policy 1, policy_version 1193145 (0.0009) [2023-12-27 00:03:37,176][105692] Updated weights for policy 0, policy_version 1191874 (0.0010) [2023-12-27 00:03:37,240][105692] Updated weights for policy 0, policy_version 1191884 (0.0011) [2023-12-27 00:03:37,277][105620] Updated weights for policy 1, policy_version 1193155 (0.0007) [2023-12-27 00:03:37,304][105692] Updated weights for policy 0, policy_version 1191894 (0.0011) [2023-12-27 00:03:37,346][105620] Updated weights for policy 1, policy_version 1193165 (0.0010) [2023-12-27 00:03:37,366][105692] Updated weights for policy 0, policy_version 1191904 (0.0011) [2023-12-27 00:03:37,407][105620] Updated weights for policy 1, policy_version 1193175 (0.0007) [2023-12-27 00:03:38,120][105692] Updated weights for policy 0, policy_version 1191914 (0.0011) [2023-12-27 00:03:38,170][105620] Updated weights for policy 1, policy_version 1193185 (0.0008) [2023-12-27 00:03:38,181][105692] Updated weights for policy 0, policy_version 1191924 (0.0011) [2023-12-27 00:03:38,237][105620] Updated weights for policy 1, policy_version 1193195 (0.0007) [2023-12-27 00:03:38,249][105692] Updated weights for policy 0, policy_version 1191934 (0.0012) [2023-12-27 00:03:38,304][105620] Updated weights for policy 1, policy_version 1193205 (0.0009) [2023-12-27 00:03:38,377][105620] Updated weights for policy 1, policy_version 1193215 (0.0009) [2023-12-27 00:03:38,979][105692] Updated weights for policy 0, policy_version 1191944 (0.0009) [2023-12-27 00:03:39,032][105692] Updated weights for policy 0, policy_version 1191954 (0.0009) [2023-12-27 00:03:39,100][105692] Updated weights for policy 0, policy_version 1191964 (0.0009) [2023-12-27 00:03:39,157][105620] Updated weights for policy 1, policy_version 1193225 (0.0009) [2023-12-27 00:03:39,221][105620] Updated weights for policy 1, policy_version 1193235 (0.0007) [2023-12-27 00:03:39,286][105620] Updated weights for policy 1, policy_version 1193245 (0.0008) [2023-12-27 00:03:39,894][105692] Updated weights for policy 0, policy_version 1191974 (0.0010) [2023-12-27 00:03:39,969][105692] Updated weights for policy 0, policy_version 1191984 (0.0011) [2023-12-27 00:03:40,034][105692] Updated weights for policy 0, policy_version 1191994 (0.0011) [2023-12-27 00:03:40,061][105620] Updated weights for policy 1, policy_version 1193255 (0.0007) [2023-12-27 00:03:40,121][105620] Updated weights for policy 1, policy_version 1193265 (0.0009) [2023-12-27 00:03:40,187][105620] Updated weights for policy 1, policy_version 1193275 (0.0008) [2023-12-27 00:03:40,788][105692] Updated weights for policy 0, policy_version 1192004 (0.0011) [2023-12-27 00:03:40,836][105692] Updated weights for policy 0, policy_version 1192014 (0.0010) [2023-12-27 00:03:40,890][105692] Updated weights for policy 0, policy_version 1192024 (0.0011) [2023-12-27 00:03:40,928][105620] Updated weights for policy 1, policy_version 1193285 (0.0007) [2023-12-27 00:03:40,989][105620] Updated weights for policy 1, policy_version 1193295 (0.0008) [2023-12-27 00:03:41,057][105620] Updated weights for policy 1, policy_version 1193305 (0.0009) [2023-12-27 00:03:41,062][104569] Fps is (10 sec: 18022.4, 60 sec: 18295.5, 300 sec: 18300.1). Total num frames: 610729984. Throughput: 0: 9411.6, 1: 9013.8. Samples: 610739664. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:03:41,063][104569] Avg episode reward: [(0, '8993.062'), (1, '8723.809')] [2023-12-27 00:03:41,729][105692] Updated weights for policy 0, policy_version 1192034 (0.0010) [2023-12-27 00:03:41,801][105692] Updated weights for policy 0, policy_version 1192044 (0.0009) [2023-12-27 00:03:41,867][105692] Updated weights for policy 0, policy_version 1192054 (0.0010) [2023-12-27 00:03:41,894][105620] Updated weights for policy 1, policy_version 1193315 (0.0008) [2023-12-27 00:03:41,924][105692] Updated weights for policy 0, policy_version 1192064 (0.0008) [2023-12-27 00:03:41,958][105620] Updated weights for policy 1, policy_version 1193325 (0.0007) [2023-12-27 00:03:42,025][105620] Updated weights for policy 1, policy_version 1193335 (0.0007) [2023-12-27 00:03:42,724][105692] Updated weights for policy 0, policy_version 1192074 (0.0009) [2023-12-27 00:03:42,783][105692] Updated weights for policy 0, policy_version 1192084 (0.0009) [2023-12-27 00:03:42,784][105620] Updated weights for policy 1, policy_version 1193345 (0.0009) [2023-12-27 00:03:42,845][105692] Updated weights for policy 0, policy_version 1192094 (0.0007) [2023-12-27 00:03:42,851][105620] Updated weights for policy 1, policy_version 1193355 (0.0009) [2023-12-27 00:03:42,913][105620] Updated weights for policy 1, policy_version 1193365 (0.0009) [2023-12-27 00:03:42,975][105620] Updated weights for policy 1, policy_version 1193375 (0.0009) [2023-12-27 00:03:43,605][105692] Updated weights for policy 0, policy_version 1192104 (0.0008) [2023-12-27 00:03:43,654][105620] Updated weights for policy 1, policy_version 1193385 (0.0007) [2023-12-27 00:03:43,667][105692] Updated weights for policy 0, policy_version 1192114 (0.0008) [2023-12-27 00:03:43,712][105620] Updated weights for policy 1, policy_version 1193395 (0.0005) [2023-12-27 00:03:43,726][105692] Updated weights for policy 0, policy_version 1192124 (0.0008) [2023-12-27 00:03:43,771][105620] Updated weights for policy 1, policy_version 1193405 (0.0006) [2023-12-27 00:03:44,481][105620] Updated weights for policy 1, policy_version 1193415 (0.0007) [2023-12-27 00:03:44,507][105692] Updated weights for policy 0, policy_version 1192134 (0.0011) [2023-12-27 00:03:44,546][105620] Updated weights for policy 1, policy_version 1193425 (0.0006) [2023-12-27 00:03:44,568][105692] Updated weights for policy 0, policy_version 1192144 (0.0011) [2023-12-27 00:03:44,606][105620] Updated weights for policy 1, policy_version 1193435 (0.0006) [2023-12-27 00:03:44,624][105692] Updated weights for policy 0, policy_version 1192154 (0.0010) [2023-12-27 00:03:45,396][105620] Updated weights for policy 1, policy_version 1193445 (0.0007) [2023-12-27 00:03:45,414][105692] Updated weights for policy 0, policy_version 1192164 (0.0011) [2023-12-27 00:03:45,458][105620] Updated weights for policy 1, policy_version 1193455 (0.0006) [2023-12-27 00:03:45,475][105692] Updated weights for policy 0, policy_version 1192174 (0.0011) [2023-12-27 00:03:45,509][105620] Updated weights for policy 1, policy_version 1193465 (0.0008) [2023-12-27 00:03:45,535][105692] Updated weights for policy 0, policy_version 1192184 (0.0011) [2023-12-27 00:03:46,062][104569] Fps is (10 sec: 18022.4, 60 sec: 18295.5, 300 sec: 18244.6). Total num frames: 610820096. Throughput: 0: 9401.2, 1: 9033.2. Samples: 610793988. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:03:46,062][104569] Avg episode reward: [(0, '8913.051'), (1, '8716.801')] [2023-12-27 00:03:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001192192_305250304.pth... [2023-12-27 00:03:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001193472_305569792.pth... [2023-12-27 00:03:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001192448_305307648.pth [2023-12-27 00:03:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001191104_304971776.pth [2023-12-27 00:03:46,283][105620] Updated weights for policy 1, policy_version 1193475 (0.0008) [2023-12-27 00:03:46,301][105692] Updated weights for policy 0, policy_version 1192194 (0.0011) [2023-12-27 00:03:46,341][105620] Updated weights for policy 1, policy_version 1193485 (0.0006) [2023-12-27 00:03:46,355][105692] Updated weights for policy 0, policy_version 1192204 (0.0011) [2023-12-27 00:03:46,397][105620] Updated weights for policy 1, policy_version 1193495 (0.0006) [2023-12-27 00:03:46,416][105692] Updated weights for policy 0, policy_version 1192214 (0.0011) [2023-12-27 00:03:46,478][105692] Updated weights for policy 0, policy_version 1192224 (0.0011) [2023-12-27 00:03:47,185][105620] Updated weights for policy 1, policy_version 1193505 (0.0006) [2023-12-27 00:03:47,238][105692] Updated weights for policy 0, policy_version 1192234 (0.0011) [2023-12-27 00:03:47,242][105620] Updated weights for policy 1, policy_version 1193515 (0.0010) [2023-12-27 00:03:47,296][105692] Updated weights for policy 0, policy_version 1192244 (0.0011) [2023-12-27 00:03:47,302][105620] Updated weights for policy 1, policy_version 1193525 (0.0007) [2023-12-27 00:03:47,360][105620] Updated weights for policy 1, policy_version 1193535 (0.0006) [2023-12-27 00:03:47,361][105692] Updated weights for policy 0, policy_version 1192254 (0.0011) [2023-12-27 00:03:48,125][105692] Updated weights for policy 0, policy_version 1192264 (0.0011) [2023-12-27 00:03:48,135][105620] Updated weights for policy 1, policy_version 1193545 (0.0008) [2023-12-27 00:03:48,179][105692] Updated weights for policy 0, policy_version 1192274 (0.0011) [2023-12-27 00:03:48,190][105620] Updated weights for policy 1, policy_version 1193555 (0.0006) [2023-12-27 00:03:48,241][105692] Updated weights for policy 0, policy_version 1192284 (0.0011) [2023-12-27 00:03:48,251][105620] Updated weights for policy 1, policy_version 1193565 (0.0006) [2023-12-27 00:03:48,979][105620] Updated weights for policy 1, policy_version 1193575 (0.0007) [2023-12-27 00:03:49,027][105692] Updated weights for policy 0, policy_version 1192294 (0.0011) [2023-12-27 00:03:49,032][105620] Updated weights for policy 1, policy_version 1193585 (0.0008) [2023-12-27 00:03:49,076][105692] Updated weights for policy 0, policy_version 1192304 (0.0010) [2023-12-27 00:03:49,091][105620] Updated weights for policy 1, policy_version 1193595 (0.0006) [2023-12-27 00:03:49,142][105692] Updated weights for policy 0, policy_version 1192314 (0.0010) [2023-12-27 00:03:49,872][105620] Updated weights for policy 1, policy_version 1193605 (0.0007) [2023-12-27 00:03:49,936][105620] Updated weights for policy 1, policy_version 1193615 (0.0009) [2023-12-27 00:03:49,941][105692] Updated weights for policy 0, policy_version 1192324 (0.0009) [2023-12-27 00:03:49,993][105620] Updated weights for policy 1, policy_version 1193625 (0.0007) [2023-12-27 00:03:50,007][105692] Updated weights for policy 0, policy_version 1192334 (0.0008) [2023-12-27 00:03:50,067][105692] Updated weights for policy 0, policy_version 1192344 (0.0008) [2023-12-27 00:03:50,748][105620] Updated weights for policy 1, policy_version 1193635 (0.0008) [2023-12-27 00:03:50,812][105620] Updated weights for policy 1, policy_version 1193645 (0.0008) [2023-12-27 00:03:50,813][105692] Updated weights for policy 0, policy_version 1192354 (0.0008) [2023-12-27 00:03:50,865][105620] Updated weights for policy 1, policy_version 1193655 (0.0006) [2023-12-27 00:03:50,866][105692] Updated weights for policy 0, policy_version 1192364 (0.0008) [2023-12-27 00:03:50,929][105692] Updated weights for policy 0, policy_version 1192374 (0.0006) [2023-12-27 00:03:50,986][105692] Updated weights for policy 0, policy_version 1192384 (0.0009) [2023-12-27 00:03:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18568.5, 300 sec: 18272.3). Total num frames: 610918400. Throughput: 0: 9384.2, 1: 9001.1. Samples: 610903020. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:03:51,063][104569] Avg episode reward: [(0, '8735.228'), (1, '8803.841')] [2023-12-27 00:03:51,696][105620] Updated weights for policy 1, policy_version 1193665 (0.0008) [2023-12-27 00:03:51,767][105620] Updated weights for policy 1, policy_version 1193675 (0.0008) [2023-12-27 00:03:51,781][105692] Updated weights for policy 0, policy_version 1192394 (0.0008) [2023-12-27 00:03:51,829][105620] Updated weights for policy 1, policy_version 1193685 (0.0007) [2023-12-27 00:03:51,847][105692] Updated weights for policy 0, policy_version 1192404 (0.0010) [2023-12-27 00:03:51,897][105620] Updated weights for policy 1, policy_version 1193695 (0.0006) [2023-12-27 00:03:51,915][105692] Updated weights for policy 0, policy_version 1192414 (0.0011) [2023-12-27 00:03:52,646][105620] Updated weights for policy 1, policy_version 1193705 (0.0009) [2023-12-27 00:03:52,654][105692] Updated weights for policy 0, policy_version 1192424 (0.0007) [2023-12-27 00:03:52,711][105620] Updated weights for policy 1, policy_version 1193715 (0.0008) [2023-12-27 00:03:52,719][105692] Updated weights for policy 0, policy_version 1192434 (0.0008) [2023-12-27 00:03:52,773][105620] Updated weights for policy 1, policy_version 1193725 (0.0007) [2023-12-27 00:03:52,775][105692] Updated weights for policy 0, policy_version 1192444 (0.0007) [2023-12-27 00:03:53,544][105692] Updated weights for policy 0, policy_version 1192454 (0.0008) [2023-12-27 00:03:53,583][105620] Updated weights for policy 1, policy_version 1193735 (0.0008) [2023-12-27 00:03:53,602][105692] Updated weights for policy 0, policy_version 1192464 (0.0006) [2023-12-27 00:03:53,650][105620] Updated weights for policy 1, policy_version 1193745 (0.0007) [2023-12-27 00:03:53,652][105692] Updated weights for policy 0, policy_version 1192474 (0.0006) [2023-12-27 00:03:53,715][105620] Updated weights for policy 1, policy_version 1193755 (0.0008) [2023-12-27 00:03:54,419][105692] Updated weights for policy 0, policy_version 1192484 (0.0008) [2023-12-27 00:03:54,472][105692] Updated weights for policy 0, policy_version 1192494 (0.0010) [2023-12-27 00:03:54,507][105620] Updated weights for policy 1, policy_version 1193765 (0.0007) [2023-12-27 00:03:54,529][105692] Updated weights for policy 0, policy_version 1192504 (0.0011) [2023-12-27 00:03:54,569][105620] Updated weights for policy 1, policy_version 1193775 (0.0006) [2023-12-27 00:03:54,635][105620] Updated weights for policy 1, policy_version 1193785 (0.0009) [2023-12-27 00:03:55,329][105620] Updated weights for policy 1, policy_version 1193795 (0.0009) [2023-12-27 00:03:55,370][105692] Updated weights for policy 0, policy_version 1192514 (0.0010) [2023-12-27 00:03:55,389][105620] Updated weights for policy 1, policy_version 1193805 (0.0008) [2023-12-27 00:03:55,430][105692] Updated weights for policy 0, policy_version 1192524 (0.0007) [2023-12-27 00:03:55,444][105620] Updated weights for policy 1, policy_version 1193815 (0.0009) [2023-12-27 00:03:55,489][105692] Updated weights for policy 0, policy_version 1192534 (0.0008) [2023-12-27 00:03:55,548][105692] Updated weights for policy 0, policy_version 1192544 (0.0006) [2023-12-27 00:03:56,062][104569] Fps is (10 sec: 18022.3, 60 sec: 18295.4, 300 sec: 18216.8). Total num frames: 611000320. Throughput: 0: 9327.9, 1: 8931.3. Samples: 611011260. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:03:56,063][104569] Avg episode reward: [(0, '8728.060'), (1, '9168.951')] [2023-12-27 00:03:56,184][105620] Updated weights for policy 1, policy_version 1193825 (0.0007) [2023-12-27 00:03:56,239][105620] Updated weights for policy 1, policy_version 1193835 (0.0005) [2023-12-27 00:03:56,284][105692] Updated weights for policy 0, policy_version 1192554 (0.0008) [2023-12-27 00:03:56,291][105620] Updated weights for policy 1, policy_version 1193845 (0.0005) [2023-12-27 00:03:56,343][105692] Updated weights for policy 0, policy_version 1192564 (0.0007) [2023-12-27 00:03:56,349][105620] Updated weights for policy 1, policy_version 1193855 (0.0008) [2023-12-27 00:03:56,394][105692] Updated weights for policy 0, policy_version 1192574 (0.0006) [2023-12-27 00:03:57,035][105620] Updated weights for policy 1, policy_version 1193865 (0.0006) [2023-12-27 00:03:57,085][105620] Updated weights for policy 1, policy_version 1193875 (0.0008) [2023-12-27 00:03:57,140][105620] Updated weights for policy 1, policy_version 1193885 (0.0009) [2023-12-27 00:03:57,142][105692] Updated weights for policy 0, policy_version 1192584 (0.0006) [2023-12-27 00:03:57,207][105692] Updated weights for policy 0, policy_version 1192594 (0.0008) [2023-12-27 00:03:57,255][105692] Updated weights for policy 0, policy_version 1192604 (0.0009) [2023-12-27 00:03:57,877][105620] Updated weights for policy 1, policy_version 1193895 (0.0007) [2023-12-27 00:03:57,935][105620] Updated weights for policy 1, policy_version 1193905 (0.0008) [2023-12-27 00:03:57,982][105692] Updated weights for policy 0, policy_version 1192614 (0.0007) [2023-12-27 00:03:57,988][105620] Updated weights for policy 1, policy_version 1193915 (0.0008) [2023-12-27 00:03:58,043][105692] Updated weights for policy 0, policy_version 1192624 (0.0009) [2023-12-27 00:03:58,091][105692] Updated weights for policy 0, policy_version 1192634 (0.0009) [2023-12-27 00:03:58,828][105620] Updated weights for policy 1, policy_version 1193925 (0.0010) [2023-12-27 00:03:58,900][105620] Updated weights for policy 1, policy_version 1193935 (0.0007) [2023-12-27 00:03:58,982][105620] Updated weights for policy 1, policy_version 1193945 (0.0010) [2023-12-27 00:03:59,003][105692] Updated weights for policy 0, policy_version 1192644 (0.0010) [2023-12-27 00:03:59,058][105692] Updated weights for policy 0, policy_version 1192654 (0.0008) [2023-12-27 00:03:59,119][105692] Updated weights for policy 0, policy_version 1192664 (0.0009) [2023-12-27 00:03:59,829][105620] Updated weights for policy 1, policy_version 1193955 (0.0010) [2023-12-27 00:03:59,846][105692] Updated weights for policy 0, policy_version 1192674 (0.0008) [2023-12-27 00:03:59,893][105620] Updated weights for policy 1, policy_version 1193965 (0.0011) [2023-12-27 00:03:59,911][105692] Updated weights for policy 0, policy_version 1192684 (0.0010) [2023-12-27 00:03:59,954][105620] Updated weights for policy 1, policy_version 1193975 (0.0011) [2023-12-27 00:03:59,972][105692] Updated weights for policy 0, policy_version 1192694 (0.0011) [2023-12-27 00:04:00,032][105692] Updated weights for policy 0, policy_version 1192704 (0.0011) [2023-12-27 00:04:00,686][105620] Updated weights for policy 1, policy_version 1193985 (0.0010) [2023-12-27 00:04:00,727][105692] Updated weights for policy 0, policy_version 1192714 (0.0011) [2023-12-27 00:04:00,740][105620] Updated weights for policy 1, policy_version 1193995 (0.0007) [2023-12-27 00:04:00,781][105692] Updated weights for policy 0, policy_version 1192724 (0.0011) [2023-12-27 00:04:00,788][105620] Updated weights for policy 1, policy_version 1194005 (0.0010) [2023-12-27 00:04:00,830][105692] Updated weights for policy 0, policy_version 1192734 (0.0010) [2023-12-27 00:04:00,837][105620] Updated weights for policy 1, policy_version 1194015 (0.0010) [2023-12-27 00:04:01,062][104569] Fps is (10 sec: 18022.6, 60 sec: 18568.6, 300 sec: 18244.6). Total num frames: 611098624. Throughput: 0: 9310.5, 1: 8970.0. Samples: 611067500. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:04:01,062][104569] Avg episode reward: [(0, '8909.046'), (1, '9259.186')] [2023-12-27 00:04:01,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001192736_305389568.pth... [2023-12-27 00:04:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001194016_305709056.pth... [2023-12-27 00:04:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001191648_305111040.pth [2023-12-27 00:04:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001192960_305438720.pth [2023-12-27 00:04:01,597][105692] Updated weights for policy 0, policy_version 1192744 (0.0011) [2023-12-27 00:04:01,607][105620] Updated weights for policy 1, policy_version 1194025 (0.0011) [2023-12-27 00:04:01,664][105692] Updated weights for policy 0, policy_version 1192754 (0.0009) [2023-12-27 00:04:01,671][105620] Updated weights for policy 1, policy_version 1194035 (0.0010) [2023-12-27 00:04:01,730][105692] Updated weights for policy 0, policy_version 1192764 (0.0010) [2023-12-27 00:04:01,738][105620] Updated weights for policy 1, policy_version 1194045 (0.0009) [2023-12-27 00:04:02,440][105692] Updated weights for policy 0, policy_version 1192774 (0.0007) [2023-12-27 00:04:02,499][105620] Updated weights for policy 1, policy_version 1194055 (0.0011) [2023-12-27 00:04:02,500][105692] Updated weights for policy 0, policy_version 1192784 (0.0006) [2023-12-27 00:04:02,563][105620] Updated weights for policy 1, policy_version 1194065 (0.0011) [2023-12-27 00:04:02,564][105692] Updated weights for policy 0, policy_version 1192794 (0.0010) [2023-12-27 00:04:02,628][105620] Updated weights for policy 1, policy_version 1194075 (0.0011) [2023-12-27 00:04:03,193][105692] Updated weights for policy 0, policy_version 1192804 (0.0011) [2023-12-27 00:04:03,244][105692] Updated weights for policy 0, policy_version 1192814 (0.0010) [2023-12-27 00:04:03,308][105692] Updated weights for policy 0, policy_version 1192824 (0.0009) [2023-12-27 00:04:03,339][105620] Updated weights for policy 1, policy_version 1194085 (0.0010) [2023-12-27 00:04:03,384][105620] Updated weights for policy 1, policy_version 1194095 (0.0010) [2023-12-27 00:04:03,439][105620] Updated weights for policy 1, policy_version 1194105 (0.0010) [2023-12-27 00:04:03,929][105692] Updated weights for policy 0, policy_version 1192834 (0.0005) [2023-12-27 00:04:03,995][105692] Updated weights for policy 0, policy_version 1192844 (0.0006) [2023-12-27 00:04:04,059][105692] Updated weights for policy 0, policy_version 1192854 (0.0008) [2023-12-27 00:04:04,123][105692] Updated weights for policy 0, policy_version 1192864 (0.0008) [2023-12-27 00:04:04,223][105620] Updated weights for policy 1, policy_version 1194115 (0.0011) [2023-12-27 00:04:04,283][105620] Updated weights for policy 1, policy_version 1194125 (0.0011) [2023-12-27 00:04:04,352][105620] Updated weights for policy 1, policy_version 1194135 (0.0011) [2023-12-27 00:04:04,816][105692] Updated weights for policy 0, policy_version 1192874 (0.0008) [2023-12-27 00:04:04,860][105692] Updated weights for policy 0, policy_version 1192884 (0.0005) [2023-12-27 00:04:04,910][105692] Updated weights for policy 0, policy_version 1192894 (0.0006) [2023-12-27 00:04:05,126][105620] Updated weights for policy 1, policy_version 1194145 (0.0011) [2023-12-27 00:04:05,189][105620] Updated weights for policy 1, policy_version 1194155 (0.0011) [2023-12-27 00:04:05,238][105620] Updated weights for policy 1, policy_version 1194165 (0.0010) [2023-12-27 00:04:05,285][105620] Updated weights for policy 1, policy_version 1194175 (0.0010) [2023-12-27 00:04:05,601][105692] Updated weights for policy 0, policy_version 1192904 (0.0007) [2023-12-27 00:04:05,665][105692] Updated weights for policy 0, policy_version 1192914 (0.0007) [2023-12-27 00:04:05,727][105692] Updated weights for policy 0, policy_version 1192924 (0.0006) [2023-12-27 00:04:06,037][105620] Updated weights for policy 1, policy_version 1194185 (0.0010) [2023-12-27 00:04:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 18432.0, 300 sec: 18189.0). Total num frames: 611188736. Throughput: 0: 9265.5, 1: 9035.5. Samples: 611180628. Policy #0 lag: (min: 14.0, avg: 21.5, max: 46.0) [2023-12-27 00:04:06,062][104569] Avg episode reward: [(0, '9268.896'), (1, '8897.798')] [2023-12-27 00:04:06,099][105620] Updated weights for policy 1, policy_version 1194195 (0.0011) [2023-12-27 00:04:06,165][105620] Updated weights for policy 1, policy_version 1194205 (0.0011) [2023-12-27 00:04:06,408][105692] Updated weights for policy 0, policy_version 1192934 (0.0008) [2023-12-27 00:04:06,476][105692] Updated weights for policy 0, policy_version 1192944 (0.0008) [2023-12-27 00:04:06,536][105692] Updated weights for policy 0, policy_version 1192954 (0.0008) [2023-12-27 00:04:06,898][105620] Updated weights for policy 1, policy_version 1194215 (0.0011) [2023-12-27 00:04:06,958][105620] Updated weights for policy 1, policy_version 1194225 (0.0011) [2023-12-27 00:04:07,029][105620] Updated weights for policy 1, policy_version 1194235 (0.0011) [2023-12-27 00:04:07,280][105692] Updated weights for policy 0, policy_version 1192964 (0.0009) [2023-12-27 00:04:07,332][105692] Updated weights for policy 0, policy_version 1192974 (0.0010) [2023-12-27 00:04:07,389][105692] Updated weights for policy 0, policy_version 1192984 (0.0011) [2023-12-27 00:04:07,790][105620] Updated weights for policy 1, policy_version 1194245 (0.0011) [2023-12-27 00:04:07,853][105620] Updated weights for policy 1, policy_version 1194255 (0.0010) [2023-12-27 00:04:07,923][105620] Updated weights for policy 1, policy_version 1194265 (0.0006) [2023-12-27 00:04:08,175][105692] Updated weights for policy 0, policy_version 1192994 (0.0011) [2023-12-27 00:04:08,227][105692] Updated weights for policy 0, policy_version 1193004 (0.0011) [2023-12-27 00:04:08,279][105692] Updated weights for policy 0, policy_version 1193014 (0.0005) [2023-12-27 00:04:08,343][105692] Updated weights for policy 0, policy_version 1193024 (0.0007) [2023-12-27 00:04:08,651][105620] Updated weights for policy 1, policy_version 1194275 (0.0006) [2023-12-27 00:04:08,719][105620] Updated weights for policy 1, policy_version 1194285 (0.0008) [2023-12-27 00:04:08,779][105620] Updated weights for policy 1, policy_version 1194295 (0.0008) [2023-12-27 00:04:09,070][105692] Updated weights for policy 0, policy_version 1193034 (0.0010) [2023-12-27 00:04:09,127][105692] Updated weights for policy 0, policy_version 1193044 (0.0008) [2023-12-27 00:04:09,134][105585] KL-divergence is very high: 126.6553 [2023-12-27 00:04:09,185][105585] KL-divergence is very high: 138.4989 [2023-12-27 00:04:09,193][105692] Updated weights for policy 0, policy_version 1193054 (0.0009) [2023-12-27 00:04:09,538][105620] Updated weights for policy 1, policy_version 1194305 (0.0008) [2023-12-27 00:04:09,602][105620] Updated weights for policy 1, policy_version 1194315 (0.0009) [2023-12-27 00:04:09,657][105620] Updated weights for policy 1, policy_version 1194325 (0.0009) [2023-12-27 00:04:09,711][105620] Updated weights for policy 1, policy_version 1194335 (0.0008) [2023-12-27 00:04:10,067][105692] Updated weights for policy 0, policy_version 1193064 (0.0009) [2023-12-27 00:04:10,134][105692] Updated weights for policy 0, policy_version 1193074 (0.0009) [2023-12-27 00:04:10,195][105692] Updated weights for policy 0, policy_version 1193084 (0.0009) [2023-12-27 00:04:10,475][105620] Updated weights for policy 1, policy_version 1194345 (0.0008) [2023-12-27 00:04:10,540][105620] Updated weights for policy 1, policy_version 1194355 (0.0008) [2023-12-27 00:04:10,594][105620] Updated weights for policy 1, policy_version 1194365 (0.0009) [2023-12-27 00:04:10,981][105692] Updated weights for policy 0, policy_version 1193094 (0.0007) [2023-12-27 00:04:11,053][105692] Updated weights for policy 0, policy_version 1193104 (0.0007) [2023-12-27 00:04:11,062][104569] Fps is (10 sec: 18022.2, 60 sec: 18295.4, 300 sec: 18161.3). Total num frames: 611278848. Throughput: 0: 9247.1, 1: 9121.2. Samples: 611291924. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:04:11,063][104569] Avg episode reward: [(0, '8907.506'), (1, '8538.459')] [2023-12-27 00:04:11,120][105692] Updated weights for policy 0, policy_version 1193114 (0.0009) [2023-12-27 00:04:11,319][105620] Updated weights for policy 1, policy_version 1194375 (0.0008) [2023-12-27 00:04:11,385][105620] Updated weights for policy 1, policy_version 1194385 (0.0008) [2023-12-27 00:04:11,452][105620] Updated weights for policy 1, policy_version 1194395 (0.0008) [2023-12-27 00:04:11,936][105692] Updated weights for policy 0, policy_version 1193124 (0.0009) [2023-12-27 00:04:11,989][105692] Updated weights for policy 0, policy_version 1193134 (0.0009) [2023-12-27 00:04:12,054][105692] Updated weights for policy 0, policy_version 1193144 (0.0010) [2023-12-27 00:04:12,213][105620] Updated weights for policy 1, policy_version 1194405 (0.0008) [2023-12-27 00:04:12,277][105620] Updated weights for policy 1, policy_version 1194415 (0.0009) [2023-12-27 00:04:12,348][105620] Updated weights for policy 1, policy_version 1194425 (0.0009) [2023-12-27 00:04:12,850][105692] Updated weights for policy 0, policy_version 1193154 (0.0011) [2023-12-27 00:04:12,915][105692] Updated weights for policy 0, policy_version 1193164 (0.0011) [2023-12-27 00:04:12,975][105692] Updated weights for policy 0, policy_version 1193174 (0.0011) [2023-12-27 00:04:13,040][105692] Updated weights for policy 0, policy_version 1193184 (0.0011) [2023-12-27 00:04:13,135][105620] Updated weights for policy 1, policy_version 1194435 (0.0009) [2023-12-27 00:04:13,193][105620] Updated weights for policy 1, policy_version 1194445 (0.0011) [2023-12-27 00:04:13,250][105620] Updated weights for policy 1, policy_version 1194455 (0.0011) [2023-12-27 00:04:13,805][105692] Updated weights for policy 0, policy_version 1193194 (0.0011) [2023-12-27 00:04:13,870][105692] Updated weights for policy 0, policy_version 1193204 (0.0011) [2023-12-27 00:04:13,926][105692] Updated weights for policy 0, policy_version 1193214 (0.0011) [2023-12-27 00:04:13,928][105620] Updated weights for policy 1, policy_version 1194465 (0.0011) [2023-12-27 00:04:13,987][105620] Updated weights for policy 1, policy_version 1194475 (0.0011) [2023-12-27 00:04:14,039][105620] Updated weights for policy 1, policy_version 1194485 (0.0011) [2023-12-27 00:04:14,103][105620] Updated weights for policy 1, policy_version 1194495 (0.0011) [2023-12-27 00:04:14,675][105692] Updated weights for policy 0, policy_version 1193224 (0.0009) [2023-12-27 00:04:14,727][105692] Updated weights for policy 0, policy_version 1193234 (0.0008) [2023-12-27 00:04:14,795][105692] Updated weights for policy 0, policy_version 1193244 (0.0008) [2023-12-27 00:04:14,836][105620] Updated weights for policy 1, policy_version 1194505 (0.0009) [2023-12-27 00:04:14,899][105620] Updated weights for policy 1, policy_version 1194515 (0.0011) [2023-12-27 00:04:14,965][105620] Updated weights for policy 1, policy_version 1194525 (0.0011) [2023-12-27 00:04:15,640][105692] Updated weights for policy 0, policy_version 1193254 (0.0008) [2023-12-27 00:04:15,686][105620] Updated weights for policy 1, policy_version 1194535 (0.0011) [2023-12-27 00:04:15,697][105692] Updated weights for policy 0, policy_version 1193264 (0.0007) [2023-12-27 00:04:15,745][105620] Updated weights for policy 1, policy_version 1194545 (0.0010) [2023-12-27 00:04:15,753][105692] Updated weights for policy 0, policy_version 1193274 (0.0010) [2023-12-27 00:04:15,815][105620] Updated weights for policy 1, policy_version 1194555 (0.0008) [2023-12-27 00:04:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18295.5, 300 sec: 18189.0). Total num frames: 611377152. Throughput: 0: 9227.7, 1: 9128.6. Samples: 611346720. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:04:16,062][104569] Avg episode reward: [(0, '8907.342'), (1, '8718.378')] [2023-12-27 00:04:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001193280_305528832.pth... [2023-12-27 00:04:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001194560_305848320.pth... [2023-12-27 00:04:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001192192_305250304.pth [2023-12-27 00:04:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001193472_305569792.pth [2023-12-27 00:04:16,554][105692] Updated weights for policy 0, policy_version 1193284 (0.0008) [2023-12-27 00:04:16,594][105620] Updated weights for policy 1, policy_version 1194565 (0.0010) [2023-12-27 00:04:16,625][105692] Updated weights for policy 0, policy_version 1193294 (0.0008) [2023-12-27 00:04:16,649][105620] Updated weights for policy 1, policy_version 1194575 (0.0010) [2023-12-27 00:04:16,691][105692] Updated weights for policy 0, policy_version 1193304 (0.0008) [2023-12-27 00:04:16,703][105620] Updated weights for policy 1, policy_version 1194585 (0.0009) [2023-12-27 00:04:17,427][105620] Updated weights for policy 1, policy_version 1194595 (0.0008) [2023-12-27 00:04:17,450][105692] Updated weights for policy 0, policy_version 1193314 (0.0008) [2023-12-27 00:04:17,484][105620] Updated weights for policy 1, policy_version 1194605 (0.0007) [2023-12-27 00:04:17,500][105692] Updated weights for policy 0, policy_version 1193324 (0.0009) [2023-12-27 00:04:17,547][105620] Updated weights for policy 1, policy_version 1194615 (0.0008) [2023-12-27 00:04:17,566][105692] Updated weights for policy 0, policy_version 1193334 (0.0009) [2023-12-27 00:04:17,633][105692] Updated weights for policy 0, policy_version 1193344 (0.0009) [2023-12-27 00:04:18,193][105620] Updated weights for policy 1, policy_version 1194625 (0.0007) [2023-12-27 00:04:18,266][105620] Updated weights for policy 1, policy_version 1194635 (0.0009) [2023-12-27 00:04:18,330][105620] Updated weights for policy 1, policy_version 1194645 (0.0007) [2023-12-27 00:04:18,400][105620] Updated weights for policy 1, policy_version 1194655 (0.0009) [2023-12-27 00:04:18,485][105692] Updated weights for policy 0, policy_version 1193354 (0.0008) [2023-12-27 00:04:18,548][105692] Updated weights for policy 0, policy_version 1193364 (0.0010) [2023-12-27 00:04:18,620][105692] Updated weights for policy 0, policy_version 1193374 (0.0010) [2023-12-27 00:04:19,075][105620] Updated weights for policy 1, policy_version 1194665 (0.0009) [2023-12-27 00:04:19,143][105620] Updated weights for policy 1, policy_version 1194675 (0.0009) [2023-12-27 00:04:19,204][105620] Updated weights for policy 1, policy_version 1194685 (0.0010) [2023-12-27 00:04:19,450][105692] Updated weights for policy 0, policy_version 1193384 (0.0008) [2023-12-27 00:04:19,515][105692] Updated weights for policy 0, policy_version 1193394 (0.0007) [2023-12-27 00:04:19,588][105692] Updated weights for policy 0, policy_version 1193404 (0.0009) [2023-12-27 00:04:20,013][105620] Updated weights for policy 1, policy_version 1194695 (0.0008) [2023-12-27 00:04:20,085][105620] Updated weights for policy 1, policy_version 1194705 (0.0008) [2023-12-27 00:04:20,153][105620] Updated weights for policy 1, policy_version 1194715 (0.0008) [2023-12-27 00:04:20,421][105692] Updated weights for policy 0, policy_version 1193414 (0.0010) [2023-12-27 00:04:20,493][105692] Updated weights for policy 0, policy_version 1193424 (0.0009) [2023-12-27 00:04:20,561][105692] Updated weights for policy 0, policy_version 1193434 (0.0009) [2023-12-27 00:04:20,917][105620] Updated weights for policy 1, policy_version 1194725 (0.0009) [2023-12-27 00:04:20,990][105620] Updated weights for policy 1, policy_version 1194735 (0.0009) [2023-12-27 00:04:21,062][104569] Fps is (10 sec: 18022.5, 60 sec: 18158.9, 300 sec: 18105.7). Total num frames: 611459072. Throughput: 0: 9123.4, 1: 9234.2. Samples: 611456104. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:04:21,062][105620] Updated weights for policy 1, policy_version 1194745 (0.0009) [2023-12-27 00:04:21,063][104569] Avg episode reward: [(0, '8997.598'), (1, '9170.277')] [2023-12-27 00:04:21,481][105692] Updated weights for policy 0, policy_version 1193444 (0.0009) [2023-12-27 00:04:21,552][105692] Updated weights for policy 0, policy_version 1193454 (0.0009) [2023-12-27 00:04:21,622][105692] Updated weights for policy 0, policy_version 1193464 (0.0010) [2023-12-27 00:04:21,935][105620] Updated weights for policy 1, policy_version 1194755 (0.0010) [2023-12-27 00:04:22,009][105620] Updated weights for policy 1, policy_version 1194765 (0.0010) [2023-12-27 00:04:22,074][105620] Updated weights for policy 1, policy_version 1194775 (0.0009) [2023-12-27 00:04:22,419][105692] Updated weights for policy 0, policy_version 1193474 (0.0010) [2023-12-27 00:04:22,484][105692] Updated weights for policy 0, policy_version 1193484 (0.0009) [2023-12-27 00:04:22,548][105692] Updated weights for policy 0, policy_version 1193494 (0.0009) [2023-12-27 00:04:22,615][105692] Updated weights for policy 0, policy_version 1193504 (0.0009) [2023-12-27 00:04:22,863][105620] Updated weights for policy 1, policy_version 1194785 (0.0010) [2023-12-27 00:04:22,926][105620] Updated weights for policy 1, policy_version 1194795 (0.0010) [2023-12-27 00:04:22,987][105620] Updated weights for policy 1, policy_version 1194805 (0.0007) [2023-12-27 00:04:23,058][105620] Updated weights for policy 1, policy_version 1194815 (0.0009) [2023-12-27 00:04:23,352][105692] Updated weights for policy 0, policy_version 1193514 (0.0008) [2023-12-27 00:04:23,411][105692] Updated weights for policy 0, policy_version 1193524 (0.0009) [2023-12-27 00:04:23,467][105692] Updated weights for policy 0, policy_version 1193534 (0.0009) [2023-12-27 00:04:23,848][105620] Updated weights for policy 1, policy_version 1194825 (0.0009) [2023-12-27 00:04:23,909][105620] Updated weights for policy 1, policy_version 1194835 (0.0010) [2023-12-27 00:04:23,974][105620] Updated weights for policy 1, policy_version 1194845 (0.0009) [2023-12-27 00:04:24,219][105692] Updated weights for policy 0, policy_version 1193544 (0.0009) [2023-12-27 00:04:24,281][105692] Updated weights for policy 0, policy_version 1193554 (0.0009) [2023-12-27 00:04:24,340][105692] Updated weights for policy 0, policy_version 1193564 (0.0009) [2023-12-27 00:04:24,767][105620] Updated weights for policy 1, policy_version 1194855 (0.0009) [2023-12-27 00:04:24,833][105620] Updated weights for policy 1, policy_version 1194865 (0.0007) [2023-12-27 00:04:24,899][105620] Updated weights for policy 1, policy_version 1194875 (0.0006) [2023-12-27 00:04:25,137][105692] Updated weights for policy 0, policy_version 1193574 (0.0010) [2023-12-27 00:04:25,202][105692] Updated weights for policy 0, policy_version 1193584 (0.0009) [2023-12-27 00:04:25,270][105692] Updated weights for policy 0, policy_version 1193594 (0.0010) [2023-12-27 00:04:25,575][105620] Updated weights for policy 1, policy_version 1194885 (0.0008) [2023-12-27 00:04:25,627][105620] Updated weights for policy 1, policy_version 1194895 (0.0010) [2023-12-27 00:04:25,684][105620] Updated weights for policy 1, policy_version 1194905 (0.0005) [2023-12-27 00:04:26,058][105692] Updated weights for policy 0, policy_version 1193604 (0.0008) [2023-12-27 00:04:26,062][104569] Fps is (10 sec: 17203.1, 60 sec: 18159.0, 300 sec: 18077.9). Total num frames: 611549184. Throughput: 0: 9052.4, 1: 9185.0. Samples: 611560348. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:04:26,063][104569] Avg episode reward: [(0, '9086.319'), (1, '8785.980')] [2023-12-27 00:04:26,124][105692] Updated weights for policy 0, policy_version 1193614 (0.0009) [2023-12-27 00:04:26,181][105692] Updated weights for policy 0, policy_version 1193624 (0.0008) [2023-12-27 00:04:26,440][105620] Updated weights for policy 1, policy_version 1194915 (0.0010) [2023-12-27 00:04:26,496][105620] Updated weights for policy 1, policy_version 1194925 (0.0009) [2023-12-27 00:04:26,561][105620] Updated weights for policy 1, policy_version 1194935 (0.0011) [2023-12-27 00:04:26,956][105692] Updated weights for policy 0, policy_version 1193634 (0.0008) [2023-12-27 00:04:27,006][105692] Updated weights for policy 0, policy_version 1193644 (0.0008) [2023-12-27 00:04:27,072][105692] Updated weights for policy 0, policy_version 1193654 (0.0008) [2023-12-27 00:04:27,132][105692] Updated weights for policy 0, policy_version 1193664 (0.0008) [2023-12-27 00:04:27,205][105620] Updated weights for policy 1, policy_version 1194945 (0.0010) [2023-12-27 00:04:27,266][105620] Updated weights for policy 1, policy_version 1194955 (0.0005) [2023-12-27 00:04:27,323][105620] Updated weights for policy 1, policy_version 1194965 (0.0006) [2023-12-27 00:04:27,379][105620] Updated weights for policy 1, policy_version 1194975 (0.0007) [2023-12-27 00:04:27,872][105692] Updated weights for policy 0, policy_version 1193674 (0.0005) [2023-12-27 00:04:27,933][105692] Updated weights for policy 0, policy_version 1193684 (0.0009) [2023-12-27 00:04:27,992][105692] Updated weights for policy 0, policy_version 1193694 (0.0010) [2023-12-27 00:04:28,054][105620] Updated weights for policy 1, policy_version 1194985 (0.0010) [2023-12-27 00:04:28,099][105620] Updated weights for policy 1, policy_version 1194995 (0.0010) [2023-12-27 00:04:28,144][105620] Updated weights for policy 1, policy_version 1195005 (0.0010) [2023-12-27 00:04:28,669][105692] Updated weights for policy 0, policy_version 1193704 (0.0009) [2023-12-27 00:04:28,735][105692] Updated weights for policy 0, policy_version 1193714 (0.0008) [2023-12-27 00:04:28,803][105692] Updated weights for policy 0, policy_version 1193724 (0.0008) [2023-12-27 00:04:28,950][105620] Updated weights for policy 1, policy_version 1195015 (0.0007) [2023-12-27 00:04:29,019][105620] Updated weights for policy 1, policy_version 1195025 (0.0008) [2023-12-27 00:04:29,085][105620] Updated weights for policy 1, policy_version 1195035 (0.0008) [2023-12-27 00:04:29,609][105692] Updated weights for policy 0, policy_version 1193734 (0.0008) [2023-12-27 00:04:29,656][105692] Updated weights for policy 0, policy_version 1193744 (0.0008) [2023-12-27 00:04:29,706][105692] Updated weights for policy 0, policy_version 1193754 (0.0009) [2023-12-27 00:04:29,778][105620] Updated weights for policy 1, policy_version 1195045 (0.0008) [2023-12-27 00:04:29,835][105620] Updated weights for policy 1, policy_version 1195055 (0.0009) [2023-12-27 00:04:29,902][105620] Updated weights for policy 1, policy_version 1195065 (0.0009) [2023-12-27 00:04:30,508][105692] Updated weights for policy 0, policy_version 1193764 (0.0009) [2023-12-27 00:04:30,559][105692] Updated weights for policy 0, policy_version 1193774 (0.0009) [2023-12-27 00:04:30,621][105692] Updated weights for policy 0, policy_version 1193784 (0.0008) [2023-12-27 00:04:30,675][105620] Updated weights for policy 1, policy_version 1195075 (0.0008) [2023-12-27 00:04:30,731][105620] Updated weights for policy 1, policy_version 1195085 (0.0008) [2023-12-27 00:04:30,790][105620] Updated weights for policy 1, policy_version 1195095 (0.0009) [2023-12-27 00:04:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18295.5, 300 sec: 18050.2). Total num frames: 611647488. Throughput: 0: 9098.1, 1: 9224.9. Samples: 611618524. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:04:31,062][104569] Avg episode reward: [(0, '8999.741'), (1, '8511.583')] [2023-12-27 00:04:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001193792_305659904.pth... [2023-12-27 00:04:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001195104_305987584.pth... [2023-12-27 00:04:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001194016_305709056.pth [2023-12-27 00:04:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001192736_305389568.pth [2023-12-27 00:04:31,403][105692] Updated weights for policy 0, policy_version 1193794 (0.0010) [2023-12-27 00:04:31,473][105692] Updated weights for policy 0, policy_version 1193804 (0.0009) [2023-12-27 00:04:31,534][105692] Updated weights for policy 0, policy_version 1193814 (0.0008) [2023-12-27 00:04:31,557][105620] Updated weights for policy 1, policy_version 1195105 (0.0009) [2023-12-27 00:04:31,599][105692] Updated weights for policy 0, policy_version 1193824 (0.0008) [2023-12-27 00:04:31,616][105620] Updated weights for policy 1, policy_version 1195115 (0.0010) [2023-12-27 00:04:31,682][105620] Updated weights for policy 1, policy_version 1195125 (0.0010) [2023-12-27 00:04:31,753][105620] Updated weights for policy 1, policy_version 1195135 (0.0011) [2023-12-27 00:04:32,355][105692] Updated weights for policy 0, policy_version 1193834 (0.0008) [2023-12-27 00:04:32,420][105692] Updated weights for policy 0, policy_version 1193844 (0.0008) [2023-12-27 00:04:32,485][105692] Updated weights for policy 0, policy_version 1193854 (0.0008) [2023-12-27 00:04:32,565][105620] Updated weights for policy 1, policy_version 1195145 (0.0011) [2023-12-27 00:04:32,625][105620] Updated weights for policy 1, policy_version 1195155 (0.0010) [2023-12-27 00:04:32,692][105620] Updated weights for policy 1, policy_version 1195165 (0.0011) [2023-12-27 00:04:33,193][105692] Updated weights for policy 0, policy_version 1193864 (0.0006) [2023-12-27 00:04:33,248][105692] Updated weights for policy 0, policy_version 1193874 (0.0009) [2023-12-27 00:04:33,311][105692] Updated weights for policy 0, policy_version 1193884 (0.0009) [2023-12-27 00:04:33,341][105620] Updated weights for policy 1, policy_version 1195175 (0.0007) [2023-12-27 00:04:33,402][105620] Updated weights for policy 1, policy_version 1195185 (0.0007) [2023-12-27 00:04:33,461][105620] Updated weights for policy 1, policy_version 1195195 (0.0006) [2023-12-27 00:04:34,073][105692] Updated weights for policy 0, policy_version 1193894 (0.0007) [2023-12-27 00:04:34,121][105620] Updated weights for policy 1, policy_version 1195205 (0.0007) [2023-12-27 00:04:34,139][105692] Updated weights for policy 0, policy_version 1193904 (0.0008) [2023-12-27 00:04:34,197][105620] Updated weights for policy 1, policy_version 1195216 (0.0008) [2023-12-27 00:04:34,206][105692] Updated weights for policy 0, policy_version 1193914 (0.0011) [2023-12-27 00:04:34,260][105620] Updated weights for policy 1, policy_version 1195226 (0.0008) [2023-12-27 00:04:34,952][105692] Updated weights for policy 0, policy_version 1193924 (0.0010) [2023-12-27 00:04:35,015][105692] Updated weights for policy 0, policy_version 1193934 (0.0010) [2023-12-27 00:04:35,055][105620] Updated weights for policy 1, policy_version 1195236 (0.0007) [2023-12-27 00:04:35,074][105692] Updated weights for policy 0, policy_version 1193944 (0.0007) [2023-12-27 00:04:35,115][105620] Updated weights for policy 1, policy_version 1195246 (0.0005) [2023-12-27 00:04:35,167][105620] Updated weights for policy 1, policy_version 1195256 (0.0006) [2023-12-27 00:04:35,751][105620] Updated weights for policy 1, policy_version 1195266 (0.0006) [2023-12-27 00:04:35,790][105692] Updated weights for policy 0, policy_version 1193954 (0.0009) [2023-12-27 00:04:35,803][105620] Updated weights for policy 1, policy_version 1195276 (0.0008) [2023-12-27 00:04:35,842][105692] Updated weights for policy 0, policy_version 1193964 (0.0011) [2023-12-27 00:04:35,860][105620] Updated weights for policy 1, policy_version 1195286 (0.0005) [2023-12-27 00:04:35,905][105692] Updated weights for policy 0, policy_version 1193974 (0.0011) [2023-12-27 00:04:35,918][105620] Updated weights for policy 1, policy_version 1195296 (0.0009) [2023-12-27 00:04:35,967][105692] Updated weights for policy 0, policy_version 1193984 (0.0008) [2023-12-27 00:04:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 18431.9, 300 sec: 18077.9). Total num frames: 611745792. Throughput: 0: 9103.0, 1: 9251.2. Samples: 611728960. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:04:36,063][104569] Avg episode reward: [(0, '8910.110'), (1, '8717.878')] [2023-12-27 00:04:36,728][105620] Updated weights for policy 1, policy_version 1195306 (0.0007) [2023-12-27 00:04:36,729][105692] Updated weights for policy 0, policy_version 1193994 (0.0011) [2023-12-27 00:04:36,786][105620] Updated weights for policy 1, policy_version 1195316 (0.0007) [2023-12-27 00:04:36,800][105692] Updated weights for policy 0, policy_version 1194004 (0.0011) [2023-12-27 00:04:36,847][105620] Updated weights for policy 1, policy_version 1195326 (0.0006) [2023-12-27 00:04:36,861][105692] Updated weights for policy 0, policy_version 1194014 (0.0010) [2023-12-27 00:04:37,539][105620] Updated weights for policy 1, policy_version 1195336 (0.0009) [2023-12-27 00:04:37,591][105620] Updated weights for policy 1, policy_version 1195346 (0.0010) [2023-12-27 00:04:37,608][105692] Updated weights for policy 0, policy_version 1194024 (0.0011) [2023-12-27 00:04:37,648][105620] Updated weights for policy 1, policy_version 1195356 (0.0011) [2023-12-27 00:04:37,668][105692] Updated weights for policy 0, policy_version 1194034 (0.0011) [2023-12-27 00:04:37,728][105692] Updated weights for policy 0, policy_version 1194044 (0.0011) [2023-12-27 00:04:38,460][105620] Updated weights for policy 1, policy_version 1195366 (0.0009) [2023-12-27 00:04:38,487][105692] Updated weights for policy 0, policy_version 1194054 (0.0011) [2023-12-27 00:04:38,527][105620] Updated weights for policy 1, policy_version 1195376 (0.0008) [2023-12-27 00:04:38,547][105692] Updated weights for policy 0, policy_version 1194064 (0.0011) [2023-12-27 00:04:38,587][105620] Updated weights for policy 1, policy_version 1195386 (0.0008) [2023-12-27 00:04:38,613][105692] Updated weights for policy 0, policy_version 1194074 (0.0011) [2023-12-27 00:04:39,306][105620] Updated weights for policy 1, policy_version 1195396 (0.0009) [2023-12-27 00:04:39,392][105620] Updated weights for policy 1, policy_version 1195406 (0.0010) [2023-12-27 00:04:39,415][105692] Updated weights for policy 0, policy_version 1194084 (0.0009) [2023-12-27 00:04:39,465][105620] Updated weights for policy 1, policy_version 1195416 (0.0008) [2023-12-27 00:04:39,486][105692] Updated weights for policy 0, policy_version 1194094 (0.0011) [2023-12-27 00:04:39,550][105692] Updated weights for policy 0, policy_version 1194104 (0.0010) [2023-12-27 00:04:40,318][105620] Updated weights for policy 1, policy_version 1195426 (0.0009) [2023-12-27 00:04:40,321][105692] Updated weights for policy 0, policy_version 1194114 (0.0008) [2023-12-27 00:04:40,380][105620] Updated weights for policy 1, policy_version 1195436 (0.0008) [2023-12-27 00:04:40,386][105692] Updated weights for policy 0, policy_version 1194124 (0.0007) [2023-12-27 00:04:40,444][105620] Updated weights for policy 1, policy_version 1195446 (0.0008) [2023-12-27 00:04:40,448][105692] Updated weights for policy 0, policy_version 1194134 (0.0007) [2023-12-27 00:04:40,501][105620] Updated weights for policy 1, policy_version 1195456 (0.0009) [2023-12-27 00:04:40,509][105692] Updated weights for policy 0, policy_version 1194144 (0.0006) [2023-12-27 00:04:41,062][104569] Fps is (10 sec: 18021.7, 60 sec: 18295.4, 300 sec: 18022.4). Total num frames: 611827712. Throughput: 0: 9114.3, 1: 9286.1. Samples: 611839280. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:04:41,063][104569] Avg episode reward: [(0, '8997.450'), (1, '8628.308')] [2023-12-27 00:04:41,183][105692] Updated weights for policy 0, policy_version 1194154 (0.0010) [2023-12-27 00:04:41,245][105692] Updated weights for policy 0, policy_version 1194164 (0.0010) [2023-12-27 00:04:41,317][105692] Updated weights for policy 0, policy_version 1194174 (0.0009) [2023-12-27 00:04:41,347][105620] Updated weights for policy 1, policy_version 1195466 (0.0008) [2023-12-27 00:04:41,414][105620] Updated weights for policy 1, policy_version 1195476 (0.0008) [2023-12-27 00:04:41,477][105620] Updated weights for policy 1, policy_version 1195486 (0.0008) [2023-12-27 00:04:42,080][105692] Updated weights for policy 0, policy_version 1194184 (0.0008) [2023-12-27 00:04:42,142][105692] Updated weights for policy 0, policy_version 1194194 (0.0010) [2023-12-27 00:04:42,207][105692] Updated weights for policy 0, policy_version 1194204 (0.0007) [2023-12-27 00:04:42,213][105620] Updated weights for policy 1, policy_version 1195496 (0.0008) [2023-12-27 00:04:42,288][105620] Updated weights for policy 1, policy_version 1195506 (0.0008) [2023-12-27 00:04:42,351][105620] Updated weights for policy 1, policy_version 1195516 (0.0009) [2023-12-27 00:04:43,009][105692] Updated weights for policy 0, policy_version 1194214 (0.0006) [2023-12-27 00:04:43,065][105692] Updated weights for policy 0, policy_version 1194224 (0.0009) [2023-12-27 00:04:43,108][105620] Updated weights for policy 1, policy_version 1195526 (0.0008) [2023-12-27 00:04:43,130][105692] Updated weights for policy 0, policy_version 1194234 (0.0008) [2023-12-27 00:04:43,165][105620] Updated weights for policy 1, policy_version 1195536 (0.0007) [2023-12-27 00:04:43,220][105620] Updated weights for policy 1, policy_version 1195546 (0.0009) [2023-12-27 00:04:43,836][105692] Updated weights for policy 0, policy_version 1194244 (0.0008) [2023-12-27 00:04:43,894][105692] Updated weights for policy 0, policy_version 1194254 (0.0008) [2023-12-27 00:04:43,944][105692] Updated weights for policy 0, policy_version 1194264 (0.0005) [2023-12-27 00:04:44,026][105620] Updated weights for policy 1, policy_version 1195556 (0.0010) [2023-12-27 00:04:44,085][105620] Updated weights for policy 1, policy_version 1195566 (0.0010) [2023-12-27 00:04:44,142][105620] Updated weights for policy 1, policy_version 1195576 (0.0008) [2023-12-27 00:04:44,637][105692] Updated weights for policy 0, policy_version 1194274 (0.0006) [2023-12-27 00:04:44,699][105692] Updated weights for policy 0, policy_version 1194284 (0.0009) [2023-12-27 00:04:44,764][105692] Updated weights for policy 0, policy_version 1194294 (0.0010) [2023-12-27 00:04:44,829][105692] Updated weights for policy 0, policy_version 1194304 (0.0007) [2023-12-27 00:04:44,980][105620] Updated weights for policy 1, policy_version 1195586 (0.0009) [2023-12-27 00:04:45,048][105620] Updated weights for policy 1, policy_version 1195596 (0.0010) [2023-12-27 00:04:45,119][105620] Updated weights for policy 1, policy_version 1195606 (0.0007) [2023-12-27 00:04:45,188][105620] Updated weights for policy 1, policy_version 1195616 (0.0009) [2023-12-27 00:04:45,594][105692] Updated weights for policy 0, policy_version 1194314 (0.0010) [2023-12-27 00:04:45,657][105692] Updated weights for policy 0, policy_version 1194324 (0.0009) [2023-12-27 00:04:45,727][105692] Updated weights for policy 0, policy_version 1194334 (0.0009) [2023-12-27 00:04:45,938][105620] Updated weights for policy 1, policy_version 1195626 (0.0009) [2023-12-27 00:04:45,989][105620] Updated weights for policy 1, policy_version 1195636 (0.0008) [2023-12-27 00:04:46,047][105620] Updated weights for policy 1, policy_version 1195646 (0.0008) [2023-12-27 00:04:46,062][104569] Fps is (10 sec: 17203.4, 60 sec: 18295.5, 300 sec: 18022.4). Total num frames: 611917824. Throughput: 0: 9109.0, 1: 9251.9. Samples: 611893744. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:04:46,062][104569] Avg episode reward: [(0, '8995.994'), (1, '8628.335')] [2023-12-27 00:04:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001194336_305799168.pth... [2023-12-27 00:04:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001193280_305528832.pth [2023-12-27 00:04:46,082][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001195648_306126848.pth... [2023-12-27 00:04:46,087][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001194560_305848320.pth [2023-12-27 00:04:46,506][105692] Updated weights for policy 0, policy_version 1194344 (0.0008) [2023-12-27 00:04:46,560][105692] Updated weights for policy 0, policy_version 1194354 (0.0009) [2023-12-27 00:04:46,623][105692] Updated weights for policy 0, policy_version 1194364 (0.0010) [2023-12-27 00:04:46,786][105620] Updated weights for policy 1, policy_version 1195656 (0.0011) [2023-12-27 00:04:46,842][105620] Updated weights for policy 1, policy_version 1195666 (0.0011) [2023-12-27 00:04:46,901][105620] Updated weights for policy 1, policy_version 1195676 (0.0010) [2023-12-27 00:04:47,425][105692] Updated weights for policy 0, policy_version 1194374 (0.0009) [2023-12-27 00:04:47,483][105692] Updated weights for policy 0, policy_version 1194384 (0.0008) [2023-12-27 00:04:47,533][105692] Updated weights for policy 0, policy_version 1194394 (0.0009) [2023-12-27 00:04:47,583][105620] Updated weights for policy 1, policy_version 1195686 (0.0009) [2023-12-27 00:04:47,645][105620] Updated weights for policy 1, policy_version 1195696 (0.0008) [2023-12-27 00:04:47,700][105620] Updated weights for policy 1, policy_version 1195706 (0.0005) [2023-12-27 00:04:48,226][105692] Updated weights for policy 0, policy_version 1194404 (0.0007) [2023-12-27 00:04:48,279][105692] Updated weights for policy 0, policy_version 1194414 (0.0008) [2023-12-27 00:04:48,327][105620] Updated weights for policy 1, policy_version 1195716 (0.0010) [2023-12-27 00:04:48,343][105692] Updated weights for policy 0, policy_version 1194424 (0.0008) [2023-12-27 00:04:48,394][105620] Updated weights for policy 1, policy_version 1195726 (0.0011) [2023-12-27 00:04:48,459][105620] Updated weights for policy 1, policy_version 1195736 (0.0011) [2023-12-27 00:04:49,094][105692] Updated weights for policy 0, policy_version 1194434 (0.0010) [2023-12-27 00:04:49,147][105692] Updated weights for policy 0, policy_version 1194444 (0.0008) [2023-12-27 00:04:49,202][105620] Updated weights for policy 1, policy_version 1195746 (0.0011) [2023-12-27 00:04:49,205][105692] Updated weights for policy 0, policy_version 1194454 (0.0007) [2023-12-27 00:04:49,269][105620] Updated weights for policy 1, policy_version 1195756 (0.0010) [2023-12-27 00:04:49,279][105692] Updated weights for policy 0, policy_version 1194464 (0.0008) [2023-12-27 00:04:49,332][105620] Updated weights for policy 1, policy_version 1195766 (0.0010) [2023-12-27 00:04:49,403][105620] Updated weights for policy 1, policy_version 1195776 (0.0009) [2023-12-27 00:04:50,123][105692] Updated weights for policy 0, policy_version 1194474 (0.0009) [2023-12-27 00:04:50,181][105620] Updated weights for policy 1, policy_version 1195786 (0.0009) [2023-12-27 00:04:50,188][105692] Updated weights for policy 0, policy_version 1194484 (0.0008) [2023-12-27 00:04:50,246][105620] Updated weights for policy 1, policy_version 1195796 (0.0010) [2023-12-27 00:04:50,253][105692] Updated weights for policy 0, policy_version 1194494 (0.0008) [2023-12-27 00:04:50,311][105620] Updated weights for policy 1, policy_version 1195806 (0.0011) [2023-12-27 00:04:50,943][105692] Updated weights for policy 0, policy_version 1194504 (0.0006) [2023-12-27 00:04:51,012][105692] Updated weights for policy 0, policy_version 1194514 (0.0008) [2023-12-27 00:04:51,062][104569] Fps is (10 sec: 18023.1, 60 sec: 18159.0, 300 sec: 17994.6). Total num frames: 612007936. Throughput: 0: 9041.5, 1: 9293.0. Samples: 612005680. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:04:51,062][104569] Avg episode reward: [(0, '8822.452'), (1, '8989.608')] [2023-12-27 00:04:51,063][105620] Updated weights for policy 1, policy_version 1195816 (0.0010) [2023-12-27 00:04:51,077][105692] Updated weights for policy 0, policy_version 1194524 (0.0006) [2023-12-27 00:04:51,133][105620] Updated weights for policy 1, policy_version 1195826 (0.0008) [2023-12-27 00:04:51,201][105620] Updated weights for policy 1, policy_version 1195836 (0.0008) [2023-12-27 00:04:51,822][105692] Updated weights for policy 0, policy_version 1194534 (0.0008) [2023-12-27 00:04:51,885][105692] Updated weights for policy 0, policy_version 1194544 (0.0009) [2023-12-27 00:04:51,933][105692] Updated weights for policy 0, policy_version 1194554 (0.0009) [2023-12-27 00:04:51,999][105620] Updated weights for policy 1, policy_version 1195846 (0.0009) [2023-12-27 00:04:52,062][105620] Updated weights for policy 1, policy_version 1195856 (0.0009) [2023-12-27 00:04:52,113][105620] Updated weights for policy 1, policy_version 1195866 (0.0007) [2023-12-27 00:04:52,687][105692] Updated weights for policy 0, policy_version 1194564 (0.0009) [2023-12-27 00:04:52,751][105692] Updated weights for policy 0, policy_version 1194574 (0.0009) [2023-12-27 00:04:52,809][105692] Updated weights for policy 0, policy_version 1194584 (0.0009) [2023-12-27 00:04:52,912][105620] Updated weights for policy 1, policy_version 1195876 (0.0009) [2023-12-27 00:04:52,971][105620] Updated weights for policy 1, policy_version 1195886 (0.0008) [2023-12-27 00:04:53,036][105620] Updated weights for policy 1, policy_version 1195896 (0.0009) [2023-12-27 00:04:53,583][105692] Updated weights for policy 0, policy_version 1194594 (0.0010) [2023-12-27 00:04:53,633][105692] Updated weights for policy 0, policy_version 1194604 (0.0011) [2023-12-27 00:04:53,648][105620] Updated weights for policy 1, policy_version 1195906 (0.0008) [2023-12-27 00:04:53,682][105692] Updated weights for policy 0, policy_version 1194614 (0.0010) [2023-12-27 00:04:53,695][105620] Updated weights for policy 1, policy_version 1195916 (0.0005) [2023-12-27 00:04:53,734][105692] Updated weights for policy 0, policy_version 1194624 (0.0011) [2023-12-27 00:04:53,748][105620] Updated weights for policy 1, policy_version 1195926 (0.0006) [2023-12-27 00:04:53,807][105620] Updated weights for policy 1, policy_version 1195936 (0.0006) [2023-12-27 00:04:54,510][105692] Updated weights for policy 0, policy_version 1194634 (0.0009) [2023-12-27 00:04:54,570][105692] Updated weights for policy 0, policy_version 1194644 (0.0011) [2023-12-27 00:04:54,592][105620] Updated weights for policy 1, policy_version 1195946 (0.0006) [2023-12-27 00:04:54,630][105692] Updated weights for policy 0, policy_version 1194654 (0.0011) [2023-12-27 00:04:54,649][105620] Updated weights for policy 1, policy_version 1195956 (0.0007) [2023-12-27 00:04:54,709][105620] Updated weights for policy 1, policy_version 1195966 (0.0006) [2023-12-27 00:04:55,364][105692] Updated weights for policy 0, policy_version 1194664 (0.0010) [2023-12-27 00:04:55,425][105620] Updated weights for policy 1, policy_version 1195976 (0.0006) [2023-12-27 00:04:55,428][105692] Updated weights for policy 0, policy_version 1194674 (0.0011) [2023-12-27 00:04:55,484][105620] Updated weights for policy 1, policy_version 1195986 (0.0006) [2023-12-27 00:04:55,489][105692] Updated weights for policy 0, policy_version 1194684 (0.0011) [2023-12-27 00:04:55,546][105620] Updated weights for policy 1, policy_version 1195997 (0.0006) [2023-12-27 00:04:56,062][104569] Fps is (10 sec: 18841.8, 60 sec: 18432.0, 300 sec: 17994.6). Total num frames: 612106240. Throughput: 0: 9041.9, 1: 9286.0. Samples: 612116676. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:04:56,062][104569] Avg episode reward: [(0, '8909.646'), (1, '9261.348')] [2023-12-27 00:04:56,285][105692] Updated weights for policy 0, policy_version 1194694 (0.0009) [2023-12-27 00:04:56,296][105620] Updated weights for policy 1, policy_version 1196007 (0.0007) [2023-12-27 00:04:56,354][105692] Updated weights for policy 0, policy_version 1194704 (0.0007) [2023-12-27 00:04:56,366][105620] Updated weights for policy 1, policy_version 1196017 (0.0007) [2023-12-27 00:04:56,422][105692] Updated weights for policy 0, policy_version 1194714 (0.0006) [2023-12-27 00:04:56,424][105620] Updated weights for policy 1, policy_version 1196027 (0.0008) [2023-12-27 00:04:57,149][105692] Updated weights for policy 0, policy_version 1194724 (0.0007) [2023-12-27 00:04:57,163][105620] Updated weights for policy 1, policy_version 1196037 (0.0008) [2023-12-27 00:04:57,204][105692] Updated weights for policy 0, policy_version 1194734 (0.0006) [2023-12-27 00:04:57,225][105620] Updated weights for policy 1, policy_version 1196047 (0.0007) [2023-12-27 00:04:57,255][105692] Updated weights for policy 0, policy_version 1194744 (0.0007) [2023-12-27 00:04:57,256][105586] KL-divergence is very high: 108.7843 [2023-12-27 00:04:57,286][105620] Updated weights for policy 1, policy_version 1196057 (0.0008) [2023-12-27 00:04:57,309][105586] KL-divergence is very high: 135.4631 [2023-12-27 00:04:58,005][105692] Updated weights for policy 0, policy_version 1194754 (0.0006) [2023-12-27 00:04:58,057][105620] Updated weights for policy 1, policy_version 1196067 (0.0008) [2023-12-27 00:04:58,067][105692] Updated weights for policy 0, policy_version 1194764 (0.0007) [2023-12-27 00:04:58,111][105620] Updated weights for policy 1, policy_version 1196077 (0.0008) [2023-12-27 00:04:58,125][105692] Updated weights for policy 0, policy_version 1194774 (0.0006) [2023-12-27 00:04:58,176][105620] Updated weights for policy 1, policy_version 1196087 (0.0008) [2023-12-27 00:04:58,194][105692] Updated weights for policy 0, policy_version 1194784 (0.0008) [2023-12-27 00:04:59,035][105620] Updated weights for policy 1, policy_version 1196097 (0.0008) [2023-12-27 00:04:59,055][105692] Updated weights for policy 0, policy_version 1194794 (0.0011) [2023-12-27 00:04:59,099][105620] Updated weights for policy 1, policy_version 1196107 (0.0009) [2023-12-27 00:04:59,120][105692] Updated weights for policy 0, policy_version 1194804 (0.0011) [2023-12-27 00:04:59,160][105620] Updated weights for policy 1, policy_version 1196117 (0.0006) [2023-12-27 00:04:59,183][105692] Updated weights for policy 0, policy_version 1194814 (0.0011) [2023-12-27 00:04:59,228][105620] Updated weights for policy 1, policy_version 1196127 (0.0007) [2023-12-27 00:05:00,014][105692] Updated weights for policy 0, policy_version 1194824 (0.0009) [2023-12-27 00:05:00,070][105692] Updated weights for policy 0, policy_version 1194834 (0.0007) [2023-12-27 00:05:00,094][105620] Updated weights for policy 1, policy_version 1196137 (0.0008) [2023-12-27 00:05:00,129][105692] Updated weights for policy 0, policy_version 1194844 (0.0007) [2023-12-27 00:05:00,156][105620] Updated weights for policy 1, policy_version 1196147 (0.0009) [2023-12-27 00:05:00,213][105620] Updated weights for policy 1, policy_version 1196157 (0.0009) [2023-12-27 00:05:00,910][105692] Updated weights for policy 0, policy_version 1194854 (0.0007) [2023-12-27 00:05:00,970][105692] Updated weights for policy 0, policy_version 1194864 (0.0008) [2023-12-27 00:05:00,987][105620] Updated weights for policy 1, policy_version 1196167 (0.0009) [2023-12-27 00:05:01,032][105692] Updated weights for policy 0, policy_version 1194874 (0.0007) [2023-12-27 00:05:01,058][105620] Updated weights for policy 1, policy_version 1196177 (0.0009) [2023-12-27 00:05:01,062][104569] Fps is (10 sec: 18022.2, 60 sec: 18158.9, 300 sec: 17966.9). Total num frames: 612188160. Throughput: 0: 9055.3, 1: 9261.2. Samples: 612170964. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:05:01,063][104569] Avg episode reward: [(0, '9175.452'), (1, '9080.254')] [2023-12-27 00:05:01,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001194880_305938432.pth... [2023-12-27 00:05:01,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001193792_305659904.pth [2023-12-27 00:05:01,120][105620] Updated weights for policy 1, policy_version 1196187 (0.0009) [2023-12-27 00:05:01,152][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001196192_306266112.pth... [2023-12-27 00:05:01,156][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001195104_305987584.pth [2023-12-27 00:05:01,904][105692] Updated weights for policy 0, policy_version 1194884 (0.0008) [2023-12-27 00:05:01,918][105620] Updated weights for policy 1, policy_version 1196197 (0.0009) [2023-12-27 00:05:01,968][105692] Updated weights for policy 0, policy_version 1194894 (0.0008) [2023-12-27 00:05:01,978][105620] Updated weights for policy 1, policy_version 1196207 (0.0007) [2023-12-27 00:05:02,028][105692] Updated weights for policy 0, policy_version 1194904 (0.0008) [2023-12-27 00:05:02,037][105620] Updated weights for policy 1, policy_version 1196217 (0.0007) [2023-12-27 00:05:02,757][105620] Updated weights for policy 1, policy_version 1196227 (0.0008) [2023-12-27 00:05:02,826][105620] Updated weights for policy 1, policy_version 1196237 (0.0009) [2023-12-27 00:05:02,832][105692] Updated weights for policy 0, policy_version 1194914 (0.0009) [2023-12-27 00:05:02,885][105620] Updated weights for policy 1, policy_version 1196247 (0.0007) [2023-12-27 00:05:02,896][105692] Updated weights for policy 0, policy_version 1194924 (0.0008) [2023-12-27 00:05:02,954][105692] Updated weights for policy 0, policy_version 1194934 (0.0009) [2023-12-27 00:05:03,012][105692] Updated weights for policy 0, policy_version 1194944 (0.0007) [2023-12-27 00:05:03,636][105692] Updated weights for policy 0, policy_version 1194954 (0.0009) [2023-12-27 00:05:03,693][105620] Updated weights for policy 1, policy_version 1196257 (0.0007) [2023-12-27 00:05:03,696][105692] Updated weights for policy 0, policy_version 1194964 (0.0008) [2023-12-27 00:05:03,753][105692] Updated weights for policy 0, policy_version 1194974 (0.0007) [2023-12-27 00:05:03,753][105620] Updated weights for policy 1, policy_version 1196267 (0.0010) [2023-12-27 00:05:03,817][105620] Updated weights for policy 1, policy_version 1196277 (0.0009) [2023-12-27 00:05:03,884][105620] Updated weights for policy 1, policy_version 1196287 (0.0008) [2023-12-27 00:05:04,549][105692] Updated weights for policy 0, policy_version 1194984 (0.0009) [2023-12-27 00:05:04,610][105692] Updated weights for policy 0, policy_version 1194994 (0.0009) [2023-12-27 00:05:04,672][105692] Updated weights for policy 0, policy_version 1195004 (0.0008) [2023-12-27 00:05:04,682][105620] Updated weights for policy 1, policy_version 1196297 (0.0009) [2023-12-27 00:05:04,744][105620] Updated weights for policy 1, policy_version 1196307 (0.0009) [2023-12-27 00:05:04,802][105620] Updated weights for policy 1, policy_version 1196317 (0.0010) [2023-12-27 00:05:05,379][105692] Updated weights for policy 0, policy_version 1195014 (0.0008) [2023-12-27 00:05:05,444][105692] Updated weights for policy 0, policy_version 1195024 (0.0009) [2023-12-27 00:05:05,499][105692] Updated weights for policy 0, policy_version 1195034 (0.0007) [2023-12-27 00:05:05,616][105620] Updated weights for policy 1, policy_version 1196327 (0.0009) [2023-12-27 00:05:05,677][105620] Updated weights for policy 1, policy_version 1196337 (0.0009) [2023-12-27 00:05:05,737][105620] Updated weights for policy 1, policy_version 1196347 (0.0009) [2023-12-27 00:05:06,062][104569] Fps is (10 sec: 18022.2, 60 sec: 18295.4, 300 sec: 18022.4). Total num frames: 612286464. Throughput: 0: 9076.1, 1: 9159.3. Samples: 612276696. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:05:06,063][104569] Avg episode reward: [(0, '9177.223'), (1, '9080.937')] [2023-12-27 00:05:06,222][105692] Updated weights for policy 0, policy_version 1195044 (0.0010) [2023-12-27 00:05:06,288][105692] Updated weights for policy 0, policy_version 1195054 (0.0009) [2023-12-27 00:05:06,358][105692] Updated weights for policy 0, policy_version 1195064 (0.0008) [2023-12-27 00:05:06,609][105620] Updated weights for policy 1, policy_version 1196357 (0.0008) [2023-12-27 00:05:06,671][105620] Updated weights for policy 1, policy_version 1196367 (0.0008) [2023-12-27 00:05:06,733][105620] Updated weights for policy 1, policy_version 1196377 (0.0009) [2023-12-27 00:05:07,037][105692] Updated weights for policy 0, policy_version 1195074 (0.0010) [2023-12-27 00:05:07,093][105692] Updated weights for policy 0, policy_version 1195084 (0.0008) [2023-12-27 00:05:07,157][105692] Updated weights for policy 0, policy_version 1195094 (0.0011) [2023-12-27 00:05:07,219][105692] Updated weights for policy 0, policy_version 1195104 (0.0010) [2023-12-27 00:05:07,550][105620] Updated weights for policy 1, policy_version 1196387 (0.0009) [2023-12-27 00:05:07,610][105620] Updated weights for policy 1, policy_version 1196397 (0.0010) [2023-12-27 00:05:07,675][105620] Updated weights for policy 1, policy_version 1196407 (0.0008) [2023-12-27 00:05:07,959][105692] Updated weights for policy 0, policy_version 1195114 (0.0011) [2023-12-27 00:05:08,020][105692] Updated weights for policy 0, policy_version 1195124 (0.0011) [2023-12-27 00:05:08,084][105692] Updated weights for policy 0, policy_version 1195134 (0.0011) [2023-12-27 00:05:08,475][105620] Updated weights for policy 1, policy_version 1196417 (0.0009) [2023-12-27 00:05:08,537][105620] Updated weights for policy 1, policy_version 1196427 (0.0008) [2023-12-27 00:05:08,593][105620] Updated weights for policy 1, policy_version 1196437 (0.0007) [2023-12-27 00:05:08,650][105620] Updated weights for policy 1, policy_version 1196447 (0.0008) [2023-12-27 00:05:08,865][105692] Updated weights for policy 0, policy_version 1195144 (0.0011) [2023-12-27 00:05:08,931][105692] Updated weights for policy 0, policy_version 1195154 (0.0011) [2023-12-27 00:05:08,995][105692] Updated weights for policy 0, policy_version 1195164 (0.0011) [2023-12-27 00:05:09,477][105620] Updated weights for policy 1, policy_version 1196457 (0.0009) [2023-12-27 00:05:09,543][105620] Updated weights for policy 1, policy_version 1196467 (0.0008) [2023-12-27 00:05:09,602][105620] Updated weights for policy 1, policy_version 1196477 (0.0008) [2023-12-27 00:05:09,753][105692] Updated weights for policy 0, policy_version 1195174 (0.0011) [2023-12-27 00:05:09,821][105692] Updated weights for policy 0, policy_version 1195184 (0.0008) [2023-12-27 00:05:09,887][105692] Updated weights for policy 0, policy_version 1195194 (0.0009) [2023-12-27 00:05:10,422][105620] Updated weights for policy 1, policy_version 1196487 (0.0008) [2023-12-27 00:05:10,481][105620] Updated weights for policy 1, policy_version 1196497 (0.0009) [2023-12-27 00:05:10,540][105620] Updated weights for policy 1, policy_version 1196507 (0.0009) [2023-12-27 00:05:10,761][105692] Updated weights for policy 0, policy_version 1195204 (0.0009) [2023-12-27 00:05:10,818][105692] Updated weights for policy 0, policy_version 1195214 (0.0006) [2023-12-27 00:05:10,876][105692] Updated weights for policy 0, policy_version 1195224 (0.0010) [2023-12-27 00:05:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 18295.5, 300 sec: 18050.2). Total num frames: 612376576. Throughput: 0: 9158.4, 1: 9121.5. Samples: 612382944. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:05:11,062][104569] Avg episode reward: [(0, '9266.702'), (1, '9173.488')] [2023-12-27 00:05:11,435][105620] Updated weights for policy 1, policy_version 1196517 (0.0009) [2023-12-27 00:05:11,502][105620] Updated weights for policy 1, policy_version 1196527 (0.0008) [2023-12-27 00:05:11,566][105620] Updated weights for policy 1, policy_version 1196537 (0.0008) [2023-12-27 00:05:11,727][105692] Updated weights for policy 0, policy_version 1195234 (0.0011) [2023-12-27 00:05:11,791][105692] Updated weights for policy 0, policy_version 1195244 (0.0008) [2023-12-27 00:05:11,857][105692] Updated weights for policy 0, policy_version 1195254 (0.0008) [2023-12-27 00:05:11,924][105692] Updated weights for policy 0, policy_version 1195264 (0.0009) [2023-12-27 00:05:12,443][105620] Updated weights for policy 1, policy_version 1196547 (0.0009) [2023-12-27 00:05:12,506][105620] Updated weights for policy 1, policy_version 1196557 (0.0008) [2023-12-27 00:05:12,568][105620] Updated weights for policy 1, policy_version 1196567 (0.0008) [2023-12-27 00:05:12,667][105692] Updated weights for policy 0, policy_version 1195274 (0.0008) [2023-12-27 00:05:12,730][105692] Updated weights for policy 0, policy_version 1195284 (0.0008) [2023-12-27 00:05:12,795][105692] Updated weights for policy 0, policy_version 1195294 (0.0008) [2023-12-27 00:05:13,336][105620] Updated weights for policy 1, policy_version 1196577 (0.0008) [2023-12-27 00:05:13,396][105620] Updated weights for policy 1, policy_version 1196587 (0.0009) [2023-12-27 00:05:13,457][105620] Updated weights for policy 1, policy_version 1196597 (0.0009) [2023-12-27 00:05:13,516][105620] Updated weights for policy 1, policy_version 1196607 (0.0009) [2023-12-27 00:05:13,596][105692] Updated weights for policy 0, policy_version 1195304 (0.0009) [2023-12-27 00:05:13,652][105692] Updated weights for policy 0, policy_version 1195314 (0.0009) [2023-12-27 00:05:13,703][105692] Updated weights for policy 0, policy_version 1195324 (0.0009) [2023-12-27 00:05:14,297][105620] Updated weights for policy 1, policy_version 1196617 (0.0010) [2023-12-27 00:05:14,360][105620] Updated weights for policy 1, policy_version 1196627 (0.0009) [2023-12-27 00:05:14,425][105620] Updated weights for policy 1, policy_version 1196637 (0.0010) [2023-12-27 00:05:14,479][105692] Updated weights for policy 0, policy_version 1195334 (0.0007) [2023-12-27 00:05:14,544][105692] Updated weights for policy 0, policy_version 1195344 (0.0009) [2023-12-27 00:05:14,604][105692] Updated weights for policy 0, policy_version 1195354 (0.0009) [2023-12-27 00:05:15,222][105620] Updated weights for policy 1, policy_version 1196647 (0.0010) [2023-12-27 00:05:15,287][105620] Updated weights for policy 1, policy_version 1196657 (0.0011) [2023-12-27 00:05:15,354][105620] Updated weights for policy 1, policy_version 1196667 (0.0011) [2023-12-27 00:05:15,402][105692] Updated weights for policy 0, policy_version 1195364 (0.0008) [2023-12-27 00:05:15,464][105692] Updated weights for policy 0, policy_version 1195374 (0.0008) [2023-12-27 00:05:15,517][105692] Updated weights for policy 0, policy_version 1195384 (0.0009) [2023-12-27 00:05:16,062][104569] Fps is (10 sec: 17203.1, 60 sec: 18022.4, 300 sec: 17994.6). Total num frames: 612458496. Throughput: 0: 9108.1, 1: 9031.0. Samples: 612434784. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:05:16,063][104569] Avg episode reward: [(0, '9086.146'), (1, '9083.174')] [2023-12-27 00:05:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001195392_306069504.pth... [2023-12-27 00:05:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001194336_305799168.pth [2023-12-27 00:05:16,075][105620] Updated weights for policy 1, policy_version 1196677 (0.0008) [2023-12-27 00:05:16,140][105620] Updated weights for policy 1, policy_version 1196687 (0.0007) [2023-12-27 00:05:16,203][105620] Updated weights for policy 1, policy_version 1196697 (0.0009) [2023-12-27 00:05:16,234][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001196704_306397184.pth... [2023-12-27 00:05:16,237][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001195648_306126848.pth [2023-12-27 00:05:16,300][105692] Updated weights for policy 0, policy_version 1195394 (0.0008) [2023-12-27 00:05:16,355][105692] Updated weights for policy 0, policy_version 1195404 (0.0008) [2023-12-27 00:05:16,410][105692] Updated weights for policy 0, policy_version 1195414 (0.0009) [2023-12-27 00:05:16,473][105692] Updated weights for policy 0, policy_version 1195424 (0.0009) [2023-12-27 00:05:16,890][105620] Updated weights for policy 1, policy_version 1196707 (0.0011) [2023-12-27 00:05:16,953][105620] Updated weights for policy 1, policy_version 1196717 (0.0011) [2023-12-27 00:05:17,013][105620] Updated weights for policy 1, policy_version 1196727 (0.0011) [2023-12-27 00:05:17,244][105692] Updated weights for policy 0, policy_version 1195434 (0.0008) [2023-12-27 00:05:17,305][105692] Updated weights for policy 0, policy_version 1195444 (0.0009) [2023-12-27 00:05:17,364][105692] Updated weights for policy 0, policy_version 1195454 (0.0010) [2023-12-27 00:05:17,736][105620] Updated weights for policy 1, policy_version 1196737 (0.0011) [2023-12-27 00:05:17,796][105620] Updated weights for policy 1, policy_version 1196747 (0.0007) [2023-12-27 00:05:17,854][105620] Updated weights for policy 1, policy_version 1196757 (0.0006) [2023-12-27 00:05:17,917][105620] Updated weights for policy 1, policy_version 1196767 (0.0007) [2023-12-27 00:05:18,162][105692] Updated weights for policy 0, policy_version 1195464 (0.0006) [2023-12-27 00:05:18,222][105692] Updated weights for policy 0, policy_version 1195474 (0.0006) [2023-12-27 00:05:18,285][105692] Updated weights for policy 0, policy_version 1195484 (0.0006) [2023-12-27 00:05:18,567][105620] Updated weights for policy 1, policy_version 1196777 (0.0008) [2023-12-27 00:05:18,625][105620] Updated weights for policy 1, policy_version 1196787 (0.0009) [2023-12-27 00:05:18,685][105620] Updated weights for policy 1, policy_version 1196797 (0.0009) [2023-12-27 00:05:19,008][105692] Updated weights for policy 0, policy_version 1195494 (0.0009) [2023-12-27 00:05:19,065][105692] Updated weights for policy 0, policy_version 1195504 (0.0011) [2023-12-27 00:05:19,130][105692] Updated weights for policy 0, policy_version 1195514 (0.0011) [2023-12-27 00:05:19,474][105620] Updated weights for policy 1, policy_version 1196807 (0.0009) [2023-12-27 00:05:19,545][105620] Updated weights for policy 1, policy_version 1196817 (0.0009) [2023-12-27 00:05:19,616][105620] Updated weights for policy 1, policy_version 1196827 (0.0007) [2023-12-27 00:05:19,939][105692] Updated weights for policy 0, policy_version 1195524 (0.0010) [2023-12-27 00:05:20,013][105692] Updated weights for policy 0, policy_version 1195534 (0.0009) [2023-12-27 00:05:20,071][105692] Updated weights for policy 0, policy_version 1195544 (0.0010) [2023-12-27 00:05:20,387][105620] Updated weights for policy 1, policy_version 1196838 (0.0007) [2023-12-27 00:05:20,453][105620] Updated weights for policy 1, policy_version 1196848 (0.0009) [2023-12-27 00:05:20,517][105620] Updated weights for policy 1, policy_version 1196858 (0.0009) [2023-12-27 00:05:20,880][105692] Updated weights for policy 0, policy_version 1195554 (0.0010) [2023-12-27 00:05:20,953][105692] Updated weights for policy 0, policy_version 1195564 (0.0007) [2023-12-27 00:05:21,020][105692] Updated weights for policy 0, policy_version 1195574 (0.0009) [2023-12-27 00:05:21,062][104569] Fps is (10 sec: 17203.2, 60 sec: 18159.0, 300 sec: 18022.4). Total num frames: 612548608. Throughput: 0: 9104.0, 1: 9032.6. Samples: 612545104. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:05:21,062][104569] Avg episode reward: [(0, '9176.247'), (1, '9172.292')] [2023-12-27 00:05:21,097][105692] Updated weights for policy 0, policy_version 1195584 (0.0009) [2023-12-27 00:05:21,319][105620] Updated weights for policy 1, policy_version 1196868 (0.0008) [2023-12-27 00:05:21,397][105620] Updated weights for policy 1, policy_version 1196878 (0.0009) [2023-12-27 00:05:21,459][105620] Updated weights for policy 1, policy_version 1196888 (0.0009) [2023-12-27 00:05:21,903][105692] Updated weights for policy 0, policy_version 1195594 (0.0009) [2023-12-27 00:05:21,975][105692] Updated weights for policy 0, policy_version 1195604 (0.0009) [2023-12-27 00:05:22,036][105692] Updated weights for policy 0, policy_version 1195614 (0.0009) [2023-12-27 00:05:22,266][105620] Updated weights for policy 1, policy_version 1196898 (0.0009) [2023-12-27 00:05:22,339][105620] Updated weights for policy 1, policy_version 1196908 (0.0009) [2023-12-27 00:05:22,414][105620] Updated weights for policy 1, policy_version 1196918 (0.0009) [2023-12-27 00:05:22,488][105620] Updated weights for policy 1, policy_version 1196928 (0.0009) [2023-12-27 00:05:22,852][105692] Updated weights for policy 0, policy_version 1195624 (0.0010) [2023-12-27 00:05:22,909][105692] Updated weights for policy 0, policy_version 1195634 (0.0008) [2023-12-27 00:05:22,973][105692] Updated weights for policy 0, policy_version 1195644 (0.0007) [2023-12-27 00:05:23,224][105620] Updated weights for policy 1, policy_version 1196938 (0.0010) [2023-12-27 00:05:23,286][105620] Updated weights for policy 1, policy_version 1196948 (0.0010) [2023-12-27 00:05:23,344][105620] Updated weights for policy 1, policy_version 1196958 (0.0009) [2023-12-27 00:05:23,647][105692] Updated weights for policy 0, policy_version 1195654 (0.0007) [2023-12-27 00:05:23,712][105692] Updated weights for policy 0, policy_version 1195664 (0.0006) [2023-12-27 00:05:23,772][105692] Updated weights for policy 0, policy_version 1195674 (0.0006) [2023-12-27 00:05:24,207][105620] Updated weights for policy 1, policy_version 1196968 (0.0011) [2023-12-27 00:05:24,273][105620] Updated weights for policy 1, policy_version 1196978 (0.0010) [2023-12-27 00:05:24,333][105620] Updated weights for policy 1, policy_version 1196988 (0.0009) [2023-12-27 00:05:24,508][105692] Updated weights for policy 0, policy_version 1195684 (0.0007) [2023-12-27 00:05:24,568][105692] Updated weights for policy 0, policy_version 1195694 (0.0008) [2023-12-27 00:05:24,628][105692] Updated weights for policy 0, policy_version 1195704 (0.0006) [2023-12-27 00:05:25,146][105620] Updated weights for policy 1, policy_version 1196998 (0.0010) [2023-12-27 00:05:25,214][105620] Updated weights for policy 1, policy_version 1197008 (0.0010) [2023-12-27 00:05:25,275][105620] Updated weights for policy 1, policy_version 1197018 (0.0009) [2023-12-27 00:05:25,303][105692] Updated weights for policy 0, policy_version 1195714 (0.0007) [2023-12-27 00:05:25,367][105692] Updated weights for policy 0, policy_version 1195724 (0.0009) [2023-12-27 00:05:25,422][105692] Updated weights for policy 0, policy_version 1195734 (0.0006) [2023-12-27 00:05:25,484][105692] Updated weights for policy 0, policy_version 1195744 (0.0006) [2023-12-27 00:05:26,062][104569] Fps is (10 sec: 18022.6, 60 sec: 18158.9, 300 sec: 17994.6). Total num frames: 612638720. Throughput: 0: 9098.4, 1: 8950.5. Samples: 612651472. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:05:26,063][104569] Avg episode reward: [(0, '9356.174'), (1, '9265.243')] [2023-12-27 00:05:26,077][105692] Updated weights for policy 0, policy_version 1195754 (0.0007) [2023-12-27 00:05:26,139][105692] Updated weights for policy 0, policy_version 1195764 (0.0006) [2023-12-27 00:05:26,158][105620] Updated weights for policy 1, policy_version 1197028 (0.0008) [2023-12-27 00:05:26,198][105692] Updated weights for policy 0, policy_version 1195774 (0.0008) [2023-12-27 00:05:26,227][105620] Updated weights for policy 1, policy_version 1197038 (0.0008) [2023-12-27 00:05:26,293][105620] Updated weights for policy 1, policy_version 1197048 (0.0008) [2023-12-27 00:05:26,858][105692] Updated weights for policy 0, policy_version 1195784 (0.0007) [2023-12-27 00:05:26,920][105692] Updated weights for policy 0, policy_version 1195794 (0.0008) [2023-12-27 00:05:26,984][105692] Updated weights for policy 0, policy_version 1195804 (0.0009) [2023-12-27 00:05:27,096][105620] Updated weights for policy 1, policy_version 1197058 (0.0008) [2023-12-27 00:05:27,159][105620] Updated weights for policy 1, policy_version 1197068 (0.0009) [2023-12-27 00:05:27,219][105620] Updated weights for policy 1, policy_version 1197078 (0.0009) [2023-12-27 00:05:27,280][105620] Updated weights for policy 1, policy_version 1197088 (0.0008) [2023-12-27 00:05:27,768][105692] Updated weights for policy 0, policy_version 1195814 (0.0008) [2023-12-27 00:05:27,827][105692] Updated weights for policy 0, policy_version 1195824 (0.0009) [2023-12-27 00:05:27,883][105692] Updated weights for policy 0, policy_version 1195834 (0.0009) [2023-12-27 00:05:28,019][105620] Updated weights for policy 1, policy_version 1197098 (0.0009) [2023-12-27 00:05:28,073][105620] Updated weights for policy 1, policy_version 1197108 (0.0010) [2023-12-27 00:05:28,133][105620] Updated weights for policy 1, policy_version 1197118 (0.0011) [2023-12-27 00:05:28,675][105692] Updated weights for policy 0, policy_version 1195844 (0.0008) [2023-12-27 00:05:28,733][105692] Updated weights for policy 0, policy_version 1195854 (0.0006) [2023-12-27 00:05:28,792][105692] Updated weights for policy 0, policy_version 1195864 (0.0009) [2023-12-27 00:05:28,930][105620] Updated weights for policy 1, policy_version 1197128 (0.0010) [2023-12-27 00:05:28,995][105620] Updated weights for policy 1, policy_version 1197138 (0.0009) [2023-12-27 00:05:29,054][105620] Updated weights for policy 1, policy_version 1197148 (0.0009) [2023-12-27 00:05:29,595][105692] Updated weights for policy 0, policy_version 1195874 (0.0009) [2023-12-27 00:05:29,652][105692] Updated weights for policy 0, policy_version 1195884 (0.0008) [2023-12-27 00:05:29,714][105692] Updated weights for policy 0, policy_version 1195894 (0.0008) [2023-12-27 00:05:29,775][105692] Updated weights for policy 0, policy_version 1195904 (0.0009) [2023-12-27 00:05:29,813][105620] Updated weights for policy 1, policy_version 1197158 (0.0010) [2023-12-27 00:05:29,882][105620] Updated weights for policy 1, policy_version 1197168 (0.0011) [2023-12-27 00:05:29,947][105620] Updated weights for policy 1, policy_version 1197178 (0.0011) [2023-12-27 00:05:30,574][105692] Updated weights for policy 0, policy_version 1195914 (0.0009) [2023-12-27 00:05:30,624][105692] Updated weights for policy 0, policy_version 1195924 (0.0008) [2023-12-27 00:05:30,675][105692] Updated weights for policy 0, policy_version 1195934 (0.0008) [2023-12-27 00:05:30,683][105620] Updated weights for policy 1, policy_version 1197188 (0.0011) [2023-12-27 00:05:30,736][105620] Updated weights for policy 1, policy_version 1197198 (0.0010) [2023-12-27 00:05:30,790][105620] Updated weights for policy 1, policy_version 1197208 (0.0006) [2023-12-27 00:05:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 18158.9, 300 sec: 18022.4). Total num frames: 612737024. Throughput: 0: 9136.7, 1: 8948.1. Samples: 612707560. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:05:31,063][104569] Avg episode reward: [(0, '9354.557'), (1, '9176.803')] [2023-12-27 00:05:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001195936_306208768.pth... [2023-12-27 00:05:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001197216_306528256.pth... [2023-12-27 00:05:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001196192_306266112.pth [2023-12-27 00:05:31,098][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001194880_305938432.pth [2023-12-27 00:05:31,502][105620] Updated weights for policy 1, policy_version 1197218 (0.0007) [2023-12-27 00:05:31,561][105692] Updated weights for policy 0, policy_version 1195944 (0.0007) [2023-12-27 00:05:31,567][105620] Updated weights for policy 1, policy_version 1197228 (0.0009) [2023-12-27 00:05:31,633][105692] Updated weights for policy 0, policy_version 1195954 (0.0008) [2023-12-27 00:05:31,636][105620] Updated weights for policy 1, policy_version 1197238 (0.0009) [2023-12-27 00:05:31,698][105692] Updated weights for policy 0, policy_version 1195964 (0.0008) [2023-12-27 00:05:31,711][105620] Updated weights for policy 1, policy_version 1197248 (0.0008) [2023-12-27 00:05:32,403][105620] Updated weights for policy 1, policy_version 1197258 (0.0009) [2023-12-27 00:05:32,438][105692] Updated weights for policy 0, policy_version 1195974 (0.0010) [2023-12-27 00:05:32,467][105620] Updated weights for policy 1, policy_version 1197268 (0.0008) [2023-12-27 00:05:32,500][105692] Updated weights for policy 0, policy_version 1195984 (0.0006) [2023-12-27 00:05:32,530][105620] Updated weights for policy 1, policy_version 1197278 (0.0009) [2023-12-27 00:05:32,566][105692] Updated weights for policy 0, policy_version 1195994 (0.0007) [2023-12-27 00:05:33,328][105620] Updated weights for policy 1, policy_version 1197288 (0.0008) [2023-12-27 00:05:33,333][105692] Updated weights for policy 0, policy_version 1196004 (0.0008) [2023-12-27 00:05:33,388][105620] Updated weights for policy 1, policy_version 1197298 (0.0005) [2023-12-27 00:05:33,392][105692] Updated weights for policy 0, policy_version 1196014 (0.0007) [2023-12-27 00:05:33,446][105620] Updated weights for policy 1, policy_version 1197308 (0.0008) [2023-12-27 00:05:33,453][105692] Updated weights for policy 0, policy_version 1196024 (0.0009) [2023-12-27 00:05:34,095][105620] Updated weights for policy 1, policy_version 1197318 (0.0007) [2023-12-27 00:05:34,162][105620] Updated weights for policy 1, policy_version 1197328 (0.0008) [2023-12-27 00:05:34,225][105620] Updated weights for policy 1, policy_version 1197338 (0.0009) [2023-12-27 00:05:34,275][105692] Updated weights for policy 0, policy_version 1196034 (0.0009) [2023-12-27 00:05:34,343][105692] Updated weights for policy 0, policy_version 1196044 (0.0007) [2023-12-27 00:05:34,408][105692] Updated weights for policy 0, policy_version 1196054 (0.0008) [2023-12-27 00:05:34,473][105692] Updated weights for policy 0, policy_version 1196064 (0.0008) [2023-12-27 00:05:35,039][105620] Updated weights for policy 1, policy_version 1197348 (0.0010) [2023-12-27 00:05:35,091][105620] Updated weights for policy 1, policy_version 1197358 (0.0009) [2023-12-27 00:05:35,147][105620] Updated weights for policy 1, policy_version 1197368 (0.0010) [2023-12-27 00:05:35,219][105692] Updated weights for policy 0, policy_version 1196074 (0.0008) [2023-12-27 00:05:35,270][105692] Updated weights for policy 0, policy_version 1196084 (0.0009) [2023-12-27 00:05:35,326][105692] Updated weights for policy 0, policy_version 1196094 (0.0009) [2023-12-27 00:05:35,944][105620] Updated weights for policy 1, policy_version 1197378 (0.0008) [2023-12-27 00:05:35,997][105620] Updated weights for policy 1, policy_version 1197388 (0.0010) [2023-12-27 00:05:36,052][105620] Updated weights for policy 1, policy_version 1197398 (0.0008) [2023-12-27 00:05:36,062][104569] Fps is (10 sec: 18022.6, 60 sec: 17885.9, 300 sec: 17994.6). Total num frames: 612818944. Throughput: 0: 9067.0, 1: 8944.6. Samples: 612816204. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:05:36,062][104569] Avg episode reward: [(0, '9353.412'), (1, '9265.491')] [2023-12-27 00:05:36,119][105620] Updated weights for policy 1, policy_version 1197408 (0.0009) [2023-12-27 00:05:36,126][105692] Updated weights for policy 0, policy_version 1196104 (0.0008) [2023-12-27 00:05:36,197][105692] Updated weights for policy 0, policy_version 1196114 (0.0009) [2023-12-27 00:05:36,261][105692] Updated weights for policy 0, policy_version 1196124 (0.0008) [2023-12-27 00:05:36,940][105620] Updated weights for policy 1, policy_version 1197418 (0.0009) [2023-12-27 00:05:37,002][105620] Updated weights for policy 1, policy_version 1197428 (0.0009) [2023-12-27 00:05:37,021][105692] Updated weights for policy 0, policy_version 1196134 (0.0007) [2023-12-27 00:05:37,065][105620] Updated weights for policy 1, policy_version 1197438 (0.0008) [2023-12-27 00:05:37,076][105692] Updated weights for policy 0, policy_version 1196144 (0.0006) [2023-12-27 00:05:37,141][105692] Updated weights for policy 0, policy_version 1196154 (0.0010) [2023-12-27 00:05:37,786][105620] Updated weights for policy 1, policy_version 1197448 (0.0009) [2023-12-27 00:05:37,851][105620] Updated weights for policy 1, policy_version 1197458 (0.0009) [2023-12-27 00:05:37,912][105620] Updated weights for policy 1, policy_version 1197468 (0.0008) [2023-12-27 00:05:37,972][105692] Updated weights for policy 0, policy_version 1196164 (0.0010) [2023-12-27 00:05:38,036][105692] Updated weights for policy 0, policy_version 1196174 (0.0009) [2023-12-27 00:05:38,092][105692] Updated weights for policy 0, policy_version 1196184 (0.0009) [2023-12-27 00:05:38,643][105620] Updated weights for policy 1, policy_version 1197478 (0.0010) [2023-12-27 00:05:38,707][105620] Updated weights for policy 1, policy_version 1197488 (0.0009) [2023-12-27 00:05:38,767][105620] Updated weights for policy 1, policy_version 1197498 (0.0009) [2023-12-27 00:05:38,938][105692] Updated weights for policy 0, policy_version 1196194 (0.0009) [2023-12-27 00:05:39,002][105692] Updated weights for policy 0, policy_version 1196204 (0.0009) [2023-12-27 00:05:39,067][105692] Updated weights for policy 0, policy_version 1196214 (0.0009) [2023-12-27 00:05:39,138][105692] Updated weights for policy 0, policy_version 1196224 (0.0009) [2023-12-27 00:05:39,585][105620] Updated weights for policy 1, policy_version 1197508 (0.0008) [2023-12-27 00:05:39,655][105620] Updated weights for policy 1, policy_version 1197518 (0.0009) [2023-12-27 00:05:39,718][105620] Updated weights for policy 1, policy_version 1197528 (0.0009) [2023-12-27 00:05:40,008][105692] Updated weights for policy 0, policy_version 1196234 (0.0009) [2023-12-27 00:05:40,067][105692] Updated weights for policy 0, policy_version 1196244 (0.0009) [2023-12-27 00:05:40,130][105692] Updated weights for policy 0, policy_version 1196254 (0.0008) [2023-12-27 00:05:40,473][105620] Updated weights for policy 1, policy_version 1197538 (0.0009) [2023-12-27 00:05:40,538][105620] Updated weights for policy 1, policy_version 1197548 (0.0009) [2023-12-27 00:05:40,597][105620] Updated weights for policy 1, policy_version 1197558 (0.0009) [2023-12-27 00:05:40,947][105692] Updated weights for policy 0, policy_version 1196264 (0.0009) [2023-12-27 00:05:41,010][105692] Updated weights for policy 0, policy_version 1196274 (0.0007) [2023-12-27 00:05:41,062][104569] Fps is (10 sec: 17203.2, 60 sec: 18022.5, 300 sec: 17994.6). Total num frames: 612909056. Throughput: 0: 8998.0, 1: 8905.1. Samples: 612922320. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:05:41,063][104569] Avg episode reward: [(0, '9353.166'), (1, '9262.965')] [2023-12-27 00:05:41,078][105692] Updated weights for policy 0, policy_version 1196284 (0.0008) [2023-12-27 00:05:41,395][105620] Updated weights for policy 1, policy_version 1197570 (0.0009) [2023-12-27 00:05:41,466][105620] Updated weights for policy 1, policy_version 1197580 (0.0009) [2023-12-27 00:05:41,533][105620] Updated weights for policy 1, policy_version 1197590 (0.0008) [2023-12-27 00:05:41,598][105620] Updated weights for policy 1, policy_version 1197600 (0.0008) [2023-12-27 00:05:41,893][105692] Updated weights for policy 0, policy_version 1196294 (0.0007) [2023-12-27 00:05:41,960][105692] Updated weights for policy 0, policy_version 1196304 (0.0009) [2023-12-27 00:05:42,024][105692] Updated weights for policy 0, policy_version 1196314 (0.0008) [2023-12-27 00:05:42,394][105620] Updated weights for policy 1, policy_version 1197610 (0.0009) [2023-12-27 00:05:42,460][105620] Updated weights for policy 1, policy_version 1197620 (0.0009) [2023-12-27 00:05:42,532][105620] Updated weights for policy 1, policy_version 1197630 (0.0009) [2023-12-27 00:05:42,859][105692] Updated weights for policy 0, policy_version 1196324 (0.0009) [2023-12-27 00:05:42,928][105692] Updated weights for policy 0, policy_version 1196334 (0.0009) [2023-12-27 00:05:42,991][105692] Updated weights for policy 0, policy_version 1196344 (0.0009) [2023-12-27 00:05:43,339][105620] Updated weights for policy 1, policy_version 1197640 (0.0010) [2023-12-27 00:05:43,402][105620] Updated weights for policy 1, policy_version 1197650 (0.0009) [2023-12-27 00:05:43,459][105620] Updated weights for policy 1, policy_version 1197660 (0.0009) [2023-12-27 00:05:43,691][105692] Updated weights for policy 0, policy_version 1196354 (0.0006) [2023-12-27 00:05:43,746][105692] Updated weights for policy 0, policy_version 1196364 (0.0009) [2023-12-27 00:05:43,801][105692] Updated weights for policy 0, policy_version 1196374 (0.0010) [2023-12-27 00:05:43,869][105692] Updated weights for policy 0, policy_version 1196384 (0.0009) [2023-12-27 00:05:44,199][105620] Updated weights for policy 1, policy_version 1197670 (0.0009) [2023-12-27 00:05:44,263][105620] Updated weights for policy 1, policy_version 1197680 (0.0009) [2023-12-27 00:05:44,322][105620] Updated weights for policy 1, policy_version 1197690 (0.0009) [2023-12-27 00:05:44,661][105692] Updated weights for policy 0, policy_version 1196394 (0.0009) [2023-12-27 00:05:44,713][105692] Updated weights for policy 0, policy_version 1196404 (0.0009) [2023-12-27 00:05:44,772][105692] Updated weights for policy 0, policy_version 1196414 (0.0009) [2023-12-27 00:05:45,121][105620] Updated weights for policy 1, policy_version 1197700 (0.0009) [2023-12-27 00:05:45,191][105620] Updated weights for policy 1, policy_version 1197710 (0.0009) [2023-12-27 00:05:45,259][105620] Updated weights for policy 1, policy_version 1197720 (0.0009) [2023-12-27 00:05:45,570][105692] Updated weights for policy 0, policy_version 1196424 (0.0010) [2023-12-27 00:05:45,632][105692] Updated weights for policy 0, policy_version 1196434 (0.0009) [2023-12-27 00:05:45,685][105692] Updated weights for policy 0, policy_version 1196444 (0.0009) [2023-12-27 00:05:46,001][105620] Updated weights for policy 1, policy_version 1197730 (0.0009) [2023-12-27 00:05:46,061][105620] Updated weights for policy 1, policy_version 1197740 (0.0009) [2023-12-27 00:05:46,062][104569] Fps is (10 sec: 18022.0, 60 sec: 18022.4, 300 sec: 17994.6). Total num frames: 612999168. Throughput: 0: 8970.9, 1: 8899.3. Samples: 612975124. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:05:46,063][104569] Avg episode reward: [(0, '9353.903'), (1, '9263.075')] [2023-12-27 00:05:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001196448_306339840.pth... [2023-12-27 00:05:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001195392_306069504.pth [2023-12-27 00:05:46,129][105620] Updated weights for policy 1, policy_version 1197750 (0.0007) [2023-12-27 00:05:46,181][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001197760_306667520.pth... [2023-12-27 00:05:46,182][105620] Updated weights for policy 1, policy_version 1197760 (0.0009) [2023-12-27 00:05:46,186][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001196704_306397184.pth [2023-12-27 00:05:46,494][105692] Updated weights for policy 0, policy_version 1196454 (0.0009) [2023-12-27 00:05:46,558][105692] Updated weights for policy 0, policy_version 1196464 (0.0009) [2023-12-27 00:05:46,617][105692] Updated weights for policy 0, policy_version 1196474 (0.0009) [2023-12-27 00:05:46,947][105620] Updated weights for policy 1, policy_version 1197770 (0.0009) [2023-12-27 00:05:47,008][105620] Updated weights for policy 1, policy_version 1197780 (0.0009) [2023-12-27 00:05:47,070][105620] Updated weights for policy 1, policy_version 1197790 (0.0008) [2023-12-27 00:05:47,405][105692] Updated weights for policy 0, policy_version 1196484 (0.0010) [2023-12-27 00:05:47,472][105692] Updated weights for policy 0, policy_version 1196494 (0.0009) [2023-12-27 00:05:47,533][105692] Updated weights for policy 0, policy_version 1196504 (0.0007) [2023-12-27 00:05:47,826][105620] Updated weights for policy 1, policy_version 1197800 (0.0008) [2023-12-27 00:05:47,878][105620] Updated weights for policy 1, policy_version 1197810 (0.0006) [2023-12-27 00:05:47,932][105620] Updated weights for policy 1, policy_version 1197820 (0.0005) [2023-12-27 00:05:48,301][105692] Updated weights for policy 0, policy_version 1196514 (0.0009) [2023-12-27 00:05:48,373][105692] Updated weights for policy 0, policy_version 1196524 (0.0007) [2023-12-27 00:05:48,438][105692] Updated weights for policy 0, policy_version 1196534 (0.0009) [2023-12-27 00:05:48,504][105692] Updated weights for policy 0, policy_version 1196544 (0.0008) [2023-12-27 00:05:48,730][105620] Updated weights for policy 1, policy_version 1197830 (0.0010) [2023-12-27 00:05:48,794][105620] Updated weights for policy 1, policy_version 1197840 (0.0011) [2023-12-27 00:05:48,866][105620] Updated weights for policy 1, policy_version 1197850 (0.0011) [2023-12-27 00:05:49,292][105692] Updated weights for policy 0, policy_version 1196554 (0.0007) [2023-12-27 00:05:49,363][105692] Updated weights for policy 0, policy_version 1196564 (0.0011) [2023-12-27 00:05:49,432][105692] Updated weights for policy 0, policy_version 1196574 (0.0010) [2023-12-27 00:05:49,612][105620] Updated weights for policy 1, policy_version 1197860 (0.0011) [2023-12-27 00:05:49,677][105620] Updated weights for policy 1, policy_version 1197870 (0.0011) [2023-12-27 00:05:49,742][105620] Updated weights for policy 1, policy_version 1197880 (0.0011) [2023-12-27 00:05:50,212][105692] Updated weights for policy 0, policy_version 1196584 (0.0010) [2023-12-27 00:05:50,279][105692] Updated weights for policy 0, policy_version 1196594 (0.0011) [2023-12-27 00:05:50,349][105692] Updated weights for policy 0, policy_version 1196604 (0.0008) [2023-12-27 00:05:50,493][105620] Updated weights for policy 1, policy_version 1197890 (0.0010) [2023-12-27 00:05:50,559][105620] Updated weights for policy 1, policy_version 1197900 (0.0008) [2023-12-27 00:05:50,640][105620] Updated weights for policy 1, policy_version 1197910 (0.0008) [2023-12-27 00:05:50,706][105620] Updated weights for policy 1, policy_version 1197920 (0.0008) [2023-12-27 00:05:51,062][104569] Fps is (10 sec: 18022.4, 60 sec: 18022.4, 300 sec: 18022.4). Total num frames: 613089280. Throughput: 0: 8969.8, 1: 8936.9. Samples: 613082496. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:05:51,063][104569] Avg episode reward: [(0, '9355.608'), (1, '9263.605')] [2023-12-27 00:05:51,108][105692] Updated weights for policy 0, policy_version 1196614 (0.0009) [2023-12-27 00:05:51,184][105692] Updated weights for policy 0, policy_version 1196624 (0.0011) [2023-12-27 00:05:51,253][105692] Updated weights for policy 0, policy_version 1196634 (0.0008) [2023-12-27 00:05:51,515][105620] Updated weights for policy 1, policy_version 1197930 (0.0009) [2023-12-27 00:05:51,583][105620] Updated weights for policy 1, policy_version 1197940 (0.0009) [2023-12-27 00:05:51,652][105620] Updated weights for policy 1, policy_version 1197950 (0.0008) [2023-12-27 00:05:52,050][105692] Updated weights for policy 0, policy_version 1196644 (0.0009) [2023-12-27 00:05:52,115][105692] Updated weights for policy 0, policy_version 1196654 (0.0008) [2023-12-27 00:05:52,176][105692] Updated weights for policy 0, policy_version 1196664 (0.0008) [2023-12-27 00:05:52,481][105620] Updated weights for policy 1, policy_version 1197960 (0.0008) [2023-12-27 00:05:52,546][105620] Updated weights for policy 1, policy_version 1197970 (0.0007) [2023-12-27 00:05:52,609][105620] Updated weights for policy 1, policy_version 1197980 (0.0006) [2023-12-27 00:05:52,954][105692] Updated weights for policy 0, policy_version 1196674 (0.0009) [2023-12-27 00:05:53,015][105692] Updated weights for policy 0, policy_version 1196684 (0.0009) [2023-12-27 00:05:53,079][105692] Updated weights for policy 0, policy_version 1196694 (0.0009) [2023-12-27 00:05:53,141][105692] Updated weights for policy 0, policy_version 1196704 (0.0007) [2023-12-27 00:05:53,376][105620] Updated weights for policy 1, policy_version 1197990 (0.0006) [2023-12-27 00:05:53,430][105620] Updated weights for policy 1, policy_version 1198000 (0.0007) [2023-12-27 00:05:53,490][105620] Updated weights for policy 1, policy_version 1198010 (0.0009) [2023-12-27 00:05:53,813][105692] Updated weights for policy 0, policy_version 1196714 (0.0009) [2023-12-27 00:05:53,878][105692] Updated weights for policy 0, policy_version 1196724 (0.0010) [2023-12-27 00:05:53,941][105692] Updated weights for policy 0, policy_version 1196734 (0.0009) [2023-12-27 00:05:54,228][105620] Updated weights for policy 1, policy_version 1198020 (0.0008) [2023-12-27 00:05:54,280][105620] Updated weights for policy 1, policy_version 1198030 (0.0007) [2023-12-27 00:05:54,340][105620] Updated weights for policy 1, policy_version 1198040 (0.0011) [2023-12-27 00:05:54,760][105692] Updated weights for policy 0, policy_version 1196744 (0.0009) [2023-12-27 00:05:54,824][105692] Updated weights for policy 0, policy_version 1196754 (0.0008) [2023-12-27 00:05:54,889][105692] Updated weights for policy 0, policy_version 1196764 (0.0008) [2023-12-27 00:05:55,122][105620] Updated weights for policy 1, policy_version 1198050 (0.0011) [2023-12-27 00:05:55,183][105620] Updated weights for policy 1, policy_version 1198060 (0.0011) [2023-12-27 00:05:55,248][105620] Updated weights for policy 1, policy_version 1198070 (0.0011) [2023-12-27 00:05:55,301][105620] Updated weights for policy 1, policy_version 1198080 (0.0010) [2023-12-27 00:05:55,664][105692] Updated weights for policy 0, policy_version 1196774 (0.0008) [2023-12-27 00:05:55,721][105692] Updated weights for policy 0, policy_version 1196784 (0.0008) [2023-12-27 00:05:55,785][105692] Updated weights for policy 0, policy_version 1196794 (0.0008) [2023-12-27 00:05:55,995][105620] Updated weights for policy 1, policy_version 1198090 (0.0006) [2023-12-27 00:05:56,061][105620] Updated weights for policy 1, policy_version 1198100 (0.0007) [2023-12-27 00:05:56,062][104569] Fps is (10 sec: 18022.7, 60 sec: 17885.9, 300 sec: 18050.2). Total num frames: 613179392. Throughput: 0: 8940.5, 1: 9013.5. Samples: 613190876. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:05:56,062][104569] Avg episode reward: [(0, '9357.004'), (1, '8989.182')] [2023-12-27 00:05:56,125][105620] Updated weights for policy 1, policy_version 1198110 (0.0010) [2023-12-27 00:05:56,577][105692] Updated weights for policy 0, policy_version 1196804 (0.0008) [2023-12-27 00:05:56,642][105692] Updated weights for policy 0, policy_version 1196814 (0.0011) [2023-12-27 00:05:56,705][105692] Updated weights for policy 0, policy_version 1196824 (0.0011) [2023-12-27 00:05:56,875][105620] Updated weights for policy 1, policy_version 1198120 (0.0009) [2023-12-27 00:05:56,944][105620] Updated weights for policy 1, policy_version 1198130 (0.0011) [2023-12-27 00:05:57,005][105620] Updated weights for policy 1, policy_version 1198140 (0.0011) [2023-12-27 00:05:57,440][105692] Updated weights for policy 0, policy_version 1196834 (0.0011) [2023-12-27 00:05:57,493][105692] Updated weights for policy 0, policy_version 1196844 (0.0011) [2023-12-27 00:05:57,549][105692] Updated weights for policy 0, policy_version 1196854 (0.0010) [2023-12-27 00:05:57,609][105692] Updated weights for policy 0, policy_version 1196864 (0.0006) [2023-12-27 00:05:57,790][105620] Updated weights for policy 1, policy_version 1198150 (0.0010) [2023-12-27 00:05:57,847][105620] Updated weights for policy 1, policy_version 1198160 (0.0009) [2023-12-27 00:05:57,907][105620] Updated weights for policy 1, policy_version 1198170 (0.0009) [2023-12-27 00:05:58,306][105692] Updated weights for policy 0, policy_version 1196874 (0.0010) [2023-12-27 00:05:58,393][105692] Updated weights for policy 0, policy_version 1196884 (0.0010) [2023-12-27 00:05:58,466][105692] Updated weights for policy 0, policy_version 1196894 (0.0010) [2023-12-27 00:05:58,819][105620] Updated weights for policy 1, policy_version 1198180 (0.0009) [2023-12-27 00:05:58,900][105620] Updated weights for policy 1, policy_version 1198190 (0.0008) [2023-12-27 00:05:58,972][105620] Updated weights for policy 1, policy_version 1198200 (0.0008) [2023-12-27 00:05:59,377][105692] Updated weights for policy 0, policy_version 1196904 (0.0008) [2023-12-27 00:05:59,445][105692] Updated weights for policy 0, policy_version 1196914 (0.0007) [2023-12-27 00:05:59,514][105692] Updated weights for policy 0, policy_version 1196924 (0.0008) [2023-12-27 00:05:59,817][105620] Updated weights for policy 1, policy_version 1198210 (0.0008) [2023-12-27 00:05:59,882][105620] Updated weights for policy 1, policy_version 1198220 (0.0008) [2023-12-27 00:05:59,956][105620] Updated weights for policy 1, policy_version 1198230 (0.0009) [2023-12-27 00:06:00,020][105620] Updated weights for policy 1, policy_version 1198240 (0.0009) [2023-12-27 00:06:00,153][105692] Updated weights for policy 0, policy_version 1196934 (0.0006) [2023-12-27 00:06:00,216][105692] Updated weights for policy 0, policy_version 1196944 (0.0008) [2023-12-27 00:06:00,276][105692] Updated weights for policy 0, policy_version 1196954 (0.0009) [2023-12-27 00:06:00,811][105620] Updated weights for policy 1, policy_version 1198250 (0.0009) [2023-12-27 00:06:00,868][105620] Updated weights for policy 1, policy_version 1198260 (0.0008) [2023-12-27 00:06:00,921][105620] Updated weights for policy 1, policy_version 1198270 (0.0009) [2023-12-27 00:06:00,974][105692] Updated weights for policy 0, policy_version 1196964 (0.0010) [2023-12-27 00:06:01,035][105692] Updated weights for policy 0, policy_version 1196974 (0.0010) [2023-12-27 00:06:01,062][104569] Fps is (10 sec: 18022.5, 60 sec: 18022.4, 300 sec: 18077.9). Total num frames: 613269504. Throughput: 0: 8960.9, 1: 9019.1. Samples: 613243884. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:06:01,062][104569] Avg episode reward: [(0, '9177.666'), (1, '8714.499')] [2023-12-27 00:06:01,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001198272_306798592.pth... [2023-12-27 00:06:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001197216_306528256.pth [2023-12-27 00:06:01,102][105692] Updated weights for policy 0, policy_version 1196984 (0.0009) [2023-12-27 00:06:01,158][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001196992_306479104.pth... [2023-12-27 00:06:01,162][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001195936_306208768.pth [2023-12-27 00:06:01,673][105620] Updated weights for policy 1, policy_version 1198280 (0.0008) [2023-12-27 00:06:01,743][105620] Updated weights for policy 1, policy_version 1198290 (0.0008) [2023-12-27 00:06:01,800][105620] Updated weights for policy 1, policy_version 1198300 (0.0009) [2023-12-27 00:06:01,950][105692] Updated weights for policy 0, policy_version 1196994 (0.0008) [2023-12-27 00:06:02,016][105692] Updated weights for policy 0, policy_version 1197004 (0.0008) [2023-12-27 00:06:02,077][105692] Updated weights for policy 0, policy_version 1197014 (0.0009) [2023-12-27 00:06:02,138][105692] Updated weights for policy 0, policy_version 1197024 (0.0009) [2023-12-27 00:06:02,502][105620] Updated weights for policy 1, policy_version 1198310 (0.0009) [2023-12-27 00:06:02,572][105620] Updated weights for policy 1, policy_version 1198320 (0.0008) [2023-12-27 00:06:02,632][105620] Updated weights for policy 1, policy_version 1198330 (0.0009) [2023-12-27 00:06:02,976][105692] Updated weights for policy 0, policy_version 1197034 (0.0010) [2023-12-27 00:06:03,035][105692] Updated weights for policy 0, policy_version 1197044 (0.0008) [2023-12-27 00:06:03,098][105692] Updated weights for policy 0, policy_version 1197054 (0.0009) [2023-12-27 00:06:03,311][105620] Updated weights for policy 1, policy_version 1198340 (0.0007) [2023-12-27 00:06:03,375][105620] Updated weights for policy 1, policy_version 1198350 (0.0007) [2023-12-27 00:06:03,436][105620] Updated weights for policy 1, policy_version 1198360 (0.0008) [2023-12-27 00:06:03,947][105692] Updated weights for policy 0, policy_version 1197064 (0.0010) [2023-12-27 00:06:04,014][105692] Updated weights for policy 0, policy_version 1197074 (0.0009) [2023-12-27 00:06:04,086][105692] Updated weights for policy 0, policy_version 1197084 (0.0010) [2023-12-27 00:06:04,124][105620] Updated weights for policy 1, policy_version 1198370 (0.0008) [2023-12-27 00:06:04,193][105620] Updated weights for policy 1, policy_version 1198380 (0.0009) [2023-12-27 00:06:04,261][105620] Updated weights for policy 1, policy_version 1198390 (0.0008) [2023-12-27 00:06:04,327][105620] Updated weights for policy 1, policy_version 1198400 (0.0007) [2023-12-27 00:06:04,911][105692] Updated weights for policy 0, policy_version 1197094 (0.0009) [2023-12-27 00:06:04,974][105692] Updated weights for policy 0, policy_version 1197104 (0.0009) [2023-12-27 00:06:05,038][105692] Updated weights for policy 0, policy_version 1197114 (0.0009) [2023-12-27 00:06:05,048][105620] Updated weights for policy 1, policy_version 1198410 (0.0006) [2023-12-27 00:06:05,105][105620] Updated weights for policy 1, policy_version 1198420 (0.0006) [2023-12-27 00:06:05,164][105620] Updated weights for policy 1, policy_version 1198430 (0.0009) [2023-12-27 00:06:05,845][105692] Updated weights for policy 0, policy_version 1197124 (0.0009) [2023-12-27 00:06:05,859][105620] Updated weights for policy 1, policy_version 1198440 (0.0008) [2023-12-27 00:06:05,891][105692] Updated weights for policy 0, policy_version 1197134 (0.0006) [2023-12-27 00:06:05,915][105620] Updated weights for policy 1, policy_version 1198450 (0.0007) [2023-12-27 00:06:05,947][105692] Updated weights for policy 0, policy_version 1197144 (0.0008) [2023-12-27 00:06:05,978][105620] Updated weights for policy 1, policy_version 1198460 (0.0007) [2023-12-27 00:06:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18022.4, 300 sec: 18077.9). Total num frames: 613367808. Throughput: 0: 8928.4, 1: 9010.2. Samples: 613352340. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-12-27 00:06:06,062][104569] Avg episode reward: [(0, '8996.480'), (1, '8988.052')] [2023-12-27 00:06:06,749][105692] Updated weights for policy 0, policy_version 1197154 (0.0006) [2023-12-27 00:06:06,780][105620] Updated weights for policy 1, policy_version 1198470 (0.0009) [2023-12-27 00:06:06,817][105692] Updated weights for policy 0, policy_version 1197164 (0.0007) [2023-12-27 00:06:06,840][105620] Updated weights for policy 1, policy_version 1198480 (0.0007) [2023-12-27 00:06:06,884][105692] Updated weights for policy 0, policy_version 1197174 (0.0006) [2023-12-27 00:06:06,903][105620] Updated weights for policy 1, policy_version 1198490 (0.0007) [2023-12-27 00:06:06,946][105692] Updated weights for policy 0, policy_version 1197184 (0.0007) [2023-12-27 00:06:07,622][105620] Updated weights for policy 1, policy_version 1198500 (0.0007) [2023-12-27 00:06:07,675][105620] Updated weights for policy 1, policy_version 1198510 (0.0009) [2023-12-27 00:06:07,742][105620] Updated weights for policy 1, policy_version 1198520 (0.0008) [2023-12-27 00:06:07,748][105692] Updated weights for policy 0, policy_version 1197194 (0.0008) [2023-12-27 00:06:07,809][105692] Updated weights for policy 0, policy_version 1197204 (0.0007) [2023-12-27 00:06:07,873][105692] Updated weights for policy 0, policy_version 1197214 (0.0007) [2023-12-27 00:06:08,514][105692] Updated weights for policy 0, policy_version 1197224 (0.0008) [2023-12-27 00:06:08,554][105620] Updated weights for policy 1, policy_version 1198530 (0.0007) [2023-12-27 00:06:08,567][105692] Updated weights for policy 0, policy_version 1197234 (0.0009) [2023-12-27 00:06:08,617][105620] Updated weights for policy 1, policy_version 1198540 (0.0006) [2023-12-27 00:06:08,628][105692] Updated weights for policy 0, policy_version 1197244 (0.0007) [2023-12-27 00:06:08,676][105620] Updated weights for policy 1, policy_version 1198550 (0.0007) [2023-12-27 00:06:08,736][105620] Updated weights for policy 1, policy_version 1198560 (0.0007) [2023-12-27 00:06:09,378][105692] Updated weights for policy 0, policy_version 1197254 (0.0008) [2023-12-27 00:06:09,448][105692] Updated weights for policy 0, policy_version 1197264 (0.0009) [2023-12-27 00:06:09,495][105620] Updated weights for policy 1, policy_version 1198570 (0.0008) [2023-12-27 00:06:09,512][105692] Updated weights for policy 0, policy_version 1197274 (0.0009) [2023-12-27 00:06:09,567][105620] Updated weights for policy 1, policy_version 1198580 (0.0009) [2023-12-27 00:06:09,636][105620] Updated weights for policy 1, policy_version 1198590 (0.0009) [2023-12-27 00:06:10,351][105692] Updated weights for policy 0, policy_version 1197284 (0.0010) [2023-12-27 00:06:10,414][105692] Updated weights for policy 0, policy_version 1197294 (0.0007) [2023-12-27 00:06:10,416][105620] Updated weights for policy 1, policy_version 1198600 (0.0009) [2023-12-27 00:06:10,477][105692] Updated weights for policy 0, policy_version 1197304 (0.0007) [2023-12-27 00:06:10,487][105620] Updated weights for policy 1, policy_version 1198610 (0.0009) [2023-12-27 00:06:10,551][105620] Updated weights for policy 1, policy_version 1198620 (0.0007) [2023-12-27 00:06:11,062][104569] Fps is (10 sec: 18022.4, 60 sec: 17885.9, 300 sec: 18077.9). Total num frames: 613449728. Throughput: 0: 8909.4, 1: 9071.9. Samples: 613460632. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:06:11,062][104569] Avg episode reward: [(0, '9084.290'), (1, '9262.755')] [2023-12-27 00:06:11,298][105692] Updated weights for policy 0, policy_version 1197314 (0.0007) [2023-12-27 00:06:11,339][105620] Updated weights for policy 1, policy_version 1198630 (0.0008) [2023-12-27 00:06:11,370][105692] Updated weights for policy 0, policy_version 1197324 (0.0008) [2023-12-27 00:06:11,412][105620] Updated weights for policy 1, policy_version 1198640 (0.0009) [2023-12-27 00:06:11,438][105692] Updated weights for policy 0, policy_version 1197334 (0.0009) [2023-12-27 00:06:11,481][105620] Updated weights for policy 1, policy_version 1198650 (0.0008) [2023-12-27 00:06:11,499][105692] Updated weights for policy 0, policy_version 1197344 (0.0007) [2023-12-27 00:06:12,268][105620] Updated weights for policy 1, policy_version 1198660 (0.0009) [2023-12-27 00:06:12,301][105692] Updated weights for policy 0, policy_version 1197354 (0.0008) [2023-12-27 00:06:12,340][105620] Updated weights for policy 1, policy_version 1198670 (0.0008) [2023-12-27 00:06:12,374][105692] Updated weights for policy 0, policy_version 1197364 (0.0009) [2023-12-27 00:06:12,418][105620] Updated weights for policy 1, policy_version 1198680 (0.0008) [2023-12-27 00:06:12,449][105692] Updated weights for policy 0, policy_version 1197374 (0.0008) [2023-12-27 00:06:13,194][105620] Updated weights for policy 1, policy_version 1198690 (0.0007) [2023-12-27 00:06:13,244][105692] Updated weights for policy 0, policy_version 1197384 (0.0007) [2023-12-27 00:06:13,257][105620] Updated weights for policy 1, policy_version 1198700 (0.0010) [2023-12-27 00:06:13,301][105692] Updated weights for policy 0, policy_version 1197394 (0.0006) [2023-12-27 00:06:13,311][105620] Updated weights for policy 1, policy_version 1198710 (0.0008) [2023-12-27 00:06:13,365][105692] Updated weights for policy 0, policy_version 1197404 (0.0007) [2023-12-27 00:06:13,374][105620] Updated weights for policy 1, policy_version 1198720 (0.0009) [2023-12-27 00:06:14,077][105692] Updated weights for policy 0, policy_version 1197414 (0.0009) [2023-12-27 00:06:14,134][105692] Updated weights for policy 0, policy_version 1197424 (0.0008) [2023-12-27 00:06:14,173][105620] Updated weights for policy 1, policy_version 1198730 (0.0008) [2023-12-27 00:06:14,195][105692] Updated weights for policy 0, policy_version 1197434 (0.0007) [2023-12-27 00:06:14,238][105620] Updated weights for policy 1, policy_version 1198740 (0.0007) [2023-12-27 00:06:14,304][105620] Updated weights for policy 1, policy_version 1198750 (0.0008) [2023-12-27 00:06:14,979][105692] Updated weights for policy 0, policy_version 1197444 (0.0009) [2023-12-27 00:06:15,038][105692] Updated weights for policy 0, policy_version 1197454 (0.0009) [2023-12-27 00:06:15,098][105620] Updated weights for policy 1, policy_version 1198760 (0.0007) [2023-12-27 00:06:15,103][105692] Updated weights for policy 0, policy_version 1197464 (0.0008) [2023-12-27 00:06:15,171][105620] Updated weights for policy 1, policy_version 1198770 (0.0009) [2023-12-27 00:06:15,233][105620] Updated weights for policy 1, policy_version 1198780 (0.0009) [2023-12-27 00:06:15,828][105692] Updated weights for policy 0, policy_version 1197474 (0.0006) [2023-12-27 00:06:15,886][105692] Updated weights for policy 0, policy_version 1197484 (0.0008) [2023-12-27 00:06:15,948][105692] Updated weights for policy 0, policy_version 1197494 (0.0009) [2023-12-27 00:06:16,015][105692] Updated weights for policy 0, policy_version 1197504 (0.0008) [2023-12-27 00:06:16,041][105620] Updated weights for policy 1, policy_version 1198790 (0.0008) [2023-12-27 00:06:16,062][104569] Fps is (10 sec: 17203.2, 60 sec: 18022.4, 300 sec: 18077.9). Total num frames: 613539840. Throughput: 0: 8846.9, 1: 9055.9. Samples: 613513188. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:06:16,062][104569] Avg episode reward: [(0, '9173.831'), (1, '8993.590')] [2023-12-27 00:06:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001197504_306610176.pth... [2023-12-27 00:06:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001196448_306339840.pth [2023-12-27 00:06:16,096][105620] Updated weights for policy 1, policy_version 1198800 (0.0009) [2023-12-27 00:06:16,156][105620] Updated weights for policy 1, policy_version 1198810 (0.0009) [2023-12-27 00:06:16,187][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001198816_306937856.pth... [2023-12-27 00:06:16,191][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001197760_306667520.pth [2023-12-27 00:06:16,752][105692] Updated weights for policy 0, policy_version 1197514 (0.0010) [2023-12-27 00:06:16,816][105692] Updated weights for policy 0, policy_version 1197524 (0.0010) [2023-12-27 00:06:16,869][105692] Updated weights for policy 0, policy_version 1197534 (0.0009) [2023-12-27 00:06:16,914][105620] Updated weights for policy 1, policy_version 1198820 (0.0008) [2023-12-27 00:06:16,973][105620] Updated weights for policy 1, policy_version 1198830 (0.0006) [2023-12-27 00:06:17,035][105620] Updated weights for policy 1, policy_version 1198840 (0.0008) [2023-12-27 00:06:17,671][105692] Updated weights for policy 0, policy_version 1197544 (0.0009) [2023-12-27 00:06:17,738][105692] Updated weights for policy 0, policy_version 1197554 (0.0009) [2023-12-27 00:06:17,780][105620] Updated weights for policy 1, policy_version 1198850 (0.0009) [2023-12-27 00:06:17,803][105692] Updated weights for policy 0, policy_version 1197564 (0.0008) [2023-12-27 00:06:17,842][105620] Updated weights for policy 1, policy_version 1198860 (0.0007) [2023-12-27 00:06:17,894][105620] Updated weights for policy 1, policy_version 1198870 (0.0009) [2023-12-27 00:06:17,955][105620] Updated weights for policy 1, policy_version 1198880 (0.0009) [2023-12-27 00:06:18,568][105692] Updated weights for policy 0, policy_version 1197574 (0.0008) [2023-12-27 00:06:18,627][105692] Updated weights for policy 0, policy_version 1197584 (0.0009) [2023-12-27 00:06:18,691][105692] Updated weights for policy 0, policy_version 1197594 (0.0009) [2023-12-27 00:06:18,752][105620] Updated weights for policy 1, policy_version 1198890 (0.0008) [2023-12-27 00:06:18,821][105620] Updated weights for policy 1, policy_version 1198900 (0.0011) [2023-12-27 00:06:18,883][105620] Updated weights for policy 1, policy_version 1198910 (0.0011) [2023-12-27 00:06:19,499][105692] Updated weights for policy 0, policy_version 1197604 (0.0007) [2023-12-27 00:06:19,560][105692] Updated weights for policy 0, policy_version 1197614 (0.0011) [2023-12-27 00:06:19,626][105692] Updated weights for policy 0, policy_version 1197624 (0.0011) [2023-12-27 00:06:19,680][105620] Updated weights for policy 1, policy_version 1198920 (0.0011) [2023-12-27 00:06:19,751][105620] Updated weights for policy 1, policy_version 1198930 (0.0010) [2023-12-27 00:06:19,820][105620] Updated weights for policy 1, policy_version 1198940 (0.0011) [2023-12-27 00:06:20,402][105692] Updated weights for policy 0, policy_version 1197634 (0.0011) [2023-12-27 00:06:20,462][105692] Updated weights for policy 0, policy_version 1197644 (0.0011) [2023-12-27 00:06:20,527][105692] Updated weights for policy 0, policy_version 1197654 (0.0011) [2023-12-27 00:06:20,588][105620] Updated weights for policy 1, policy_version 1198950 (0.0009) [2023-12-27 00:06:20,596][105692] Updated weights for policy 0, policy_version 1197664 (0.0009) [2023-12-27 00:06:20,657][105620] Updated weights for policy 1, policy_version 1198960 (0.0010) [2023-12-27 00:06:20,725][105620] Updated weights for policy 1, policy_version 1198970 (0.0010) [2023-12-27 00:06:21,062][104569] Fps is (10 sec: 18022.3, 60 sec: 18022.4, 300 sec: 18077.9). Total num frames: 613629952. Throughput: 0: 8890.2, 1: 8992.2. Samples: 613620912. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:06:21,063][104569] Avg episode reward: [(0, '8904.284'), (1, '8901.912')] [2023-12-27 00:06:21,392][105692] Updated weights for policy 0, policy_version 1197674 (0.0010) [2023-12-27 00:06:21,461][105692] Updated weights for policy 0, policy_version 1197684 (0.0009) [2023-12-27 00:06:21,508][105620] Updated weights for policy 1, policy_version 1198980 (0.0011) [2023-12-27 00:06:21,524][105692] Updated weights for policy 0, policy_version 1197694 (0.0010) [2023-12-27 00:06:21,570][105620] Updated weights for policy 1, policy_version 1198990 (0.0011) [2023-12-27 00:06:21,640][105620] Updated weights for policy 1, policy_version 1199000 (0.0013) [2023-12-27 00:06:22,355][105692] Updated weights for policy 0, policy_version 1197704 (0.0011) [2023-12-27 00:06:22,424][105620] Updated weights for policy 1, policy_version 1199010 (0.0008) [2023-12-27 00:06:22,428][105692] Updated weights for policy 0, policy_version 1197714 (0.0011) [2023-12-27 00:06:22,484][105620] Updated weights for policy 1, policy_version 1199020 (0.0007) [2023-12-27 00:06:22,498][105692] Updated weights for policy 0, policy_version 1197724 (0.0011) [2023-12-27 00:06:22,548][105620] Updated weights for policy 1, policy_version 1199030 (0.0009) [2023-12-27 00:06:22,611][105620] Updated weights for policy 1, policy_version 1199040 (0.0011) [2023-12-27 00:06:23,305][105692] Updated weights for policy 0, policy_version 1197734 (0.0011) [2023-12-27 00:06:23,336][105620] Updated weights for policy 1, policy_version 1199050 (0.0011) [2023-12-27 00:06:23,370][105692] Updated weights for policy 0, policy_version 1197744 (0.0011) [2023-12-27 00:06:23,398][105620] Updated weights for policy 1, policy_version 1199060 (0.0011) [2023-12-27 00:06:23,436][105692] Updated weights for policy 0, policy_version 1197754 (0.0011) [2023-12-27 00:06:23,464][105620] Updated weights for policy 1, policy_version 1199070 (0.0011) [2023-12-27 00:06:24,200][105620] Updated weights for policy 1, policy_version 1199080 (0.0010) [2023-12-27 00:06:24,204][105692] Updated weights for policy 0, policy_version 1197764 (0.0011) [2023-12-27 00:06:24,254][105620] Updated weights for policy 1, policy_version 1199090 (0.0011) [2023-12-27 00:06:24,265][105692] Updated weights for policy 0, policy_version 1197774 (0.0011) [2023-12-27 00:06:24,316][105620] Updated weights for policy 1, policy_version 1199100 (0.0010) [2023-12-27 00:06:24,324][105692] Updated weights for policy 0, policy_version 1197784 (0.0011) [2023-12-27 00:06:25,122][105620] Updated weights for policy 1, policy_version 1199110 (0.0011) [2023-12-27 00:06:25,129][105692] Updated weights for policy 0, policy_version 1197794 (0.0011) [2023-12-27 00:06:25,183][105620] Updated weights for policy 1, policy_version 1199120 (0.0011) [2023-12-27 00:06:25,189][105692] Updated weights for policy 0, policy_version 1197804 (0.0011) [2023-12-27 00:06:25,246][105620] Updated weights for policy 1, policy_version 1199130 (0.0011) [2023-12-27 00:06:25,254][105692] Updated weights for policy 0, policy_version 1197814 (0.0009) [2023-12-27 00:06:25,321][105692] Updated weights for policy 0, policy_version 1197824 (0.0011) [2023-12-27 00:06:25,957][105692] Updated weights for policy 0, policy_version 1197834 (0.0008) [2023-12-27 00:06:26,018][105692] Updated weights for policy 0, policy_version 1197844 (0.0008) [2023-12-27 00:06:26,062][104569] Fps is (10 sec: 17203.2, 60 sec: 17885.9, 300 sec: 18050.2). Total num frames: 613711872. Throughput: 0: 8898.2, 1: 8973.0. Samples: 613726524. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:06:26,062][104569] Avg episode reward: [(0, '8723.536'), (1, '9082.260')] [2023-12-27 00:06:26,073][105620] Updated weights for policy 1, policy_version 1199140 (0.0009) [2023-12-27 00:06:26,078][105692] Updated weights for policy 0, policy_version 1197854 (0.0009) [2023-12-27 00:06:26,139][105620] Updated weights for policy 1, policy_version 1199150 (0.0008) [2023-12-27 00:06:26,194][105620] Updated weights for policy 1, policy_version 1199160 (0.0009) [2023-12-27 00:06:26,836][105692] Updated weights for policy 0, policy_version 1197864 (0.0010) [2023-12-27 00:06:26,899][105692] Updated weights for policy 0, policy_version 1197874 (0.0011) [2023-12-27 00:06:26,924][105620] Updated weights for policy 1, policy_version 1199170 (0.0009) [2023-12-27 00:06:26,964][105692] Updated weights for policy 0, policy_version 1197884 (0.0011) [2023-12-27 00:06:26,985][105620] Updated weights for policy 1, policy_version 1199180 (0.0006) [2023-12-27 00:06:27,045][105620] Updated weights for policy 1, policy_version 1199190 (0.0008) [2023-12-27 00:06:27,103][105620] Updated weights for policy 1, policy_version 1199200 (0.0010) [2023-12-27 00:06:27,676][105692] Updated weights for policy 0, policy_version 1197894 (0.0011) [2023-12-27 00:06:27,737][105692] Updated weights for policy 0, policy_version 1197904 (0.0011) [2023-12-27 00:06:27,791][105692] Updated weights for policy 0, policy_version 1197914 (0.0011) [2023-12-27 00:06:27,858][105620] Updated weights for policy 1, policy_version 1199210 (0.0008) [2023-12-27 00:06:27,915][105620] Updated weights for policy 1, policy_version 1199220 (0.0008) [2023-12-27 00:06:27,974][105620] Updated weights for policy 1, policy_version 1199230 (0.0007) [2023-12-27 00:06:28,573][105692] Updated weights for policy 0, policy_version 1197924 (0.0011) [2023-12-27 00:06:28,626][105692] Updated weights for policy 0, policy_version 1197934 (0.0011) [2023-12-27 00:06:28,686][105692] Updated weights for policy 0, policy_version 1197944 (0.0010) [2023-12-27 00:06:28,700][105620] Updated weights for policy 1, policy_version 1199240 (0.0008) [2023-12-27 00:06:28,767][105620] Updated weights for policy 1, policy_version 1199250 (0.0007) [2023-12-27 00:06:28,833][105620] Updated weights for policy 1, policy_version 1199260 (0.0008) [2023-12-27 00:06:29,513][105692] Updated weights for policy 0, policy_version 1197954 (0.0009) [2023-12-27 00:06:29,581][105692] Updated weights for policy 0, policy_version 1197964 (0.0006) [2023-12-27 00:06:29,647][105692] Updated weights for policy 0, policy_version 1197974 (0.0007) [2023-12-27 00:06:29,657][105620] Updated weights for policy 1, policy_version 1199270 (0.0010) [2023-12-27 00:06:29,710][105692] Updated weights for policy 0, policy_version 1197984 (0.0006) [2023-12-27 00:06:29,716][105620] Updated weights for policy 1, policy_version 1199280 (0.0009) [2023-12-27 00:06:29,780][105620] Updated weights for policy 1, policy_version 1199290 (0.0010) [2023-12-27 00:06:30,465][105620] Updated weights for policy 1, policy_version 1199300 (0.0008) [2023-12-27 00:06:30,506][105692] Updated weights for policy 0, policy_version 1197994 (0.0008) [2023-12-27 00:06:30,530][105620] Updated weights for policy 1, policy_version 1199310 (0.0008) [2023-12-27 00:06:30,560][105692] Updated weights for policy 0, policy_version 1198004 (0.0008) [2023-12-27 00:06:30,592][105620] Updated weights for policy 1, policy_version 1199320 (0.0007) [2023-12-27 00:06:30,623][105692] Updated weights for policy 0, policy_version 1198014 (0.0007) [2023-12-27 00:06:31,062][104569] Fps is (10 sec: 18022.3, 60 sec: 17885.9, 300 sec: 18077.9). Total num frames: 613810176. Throughput: 0: 8954.0, 1: 9002.2. Samples: 613783152. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:06:31,063][104569] Avg episode reward: [(0, '8814.003'), (1, '9079.079')] [2023-12-27 00:06:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001198016_306741248.pth... [2023-12-27 00:06:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001199328_307068928.pth... [2023-12-27 00:06:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001196992_306479104.pth [2023-12-27 00:06:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001198272_306798592.pth [2023-12-27 00:06:31,325][105620] Updated weights for policy 1, policy_version 1199330 (0.0006) [2023-12-27 00:06:31,389][105620] Updated weights for policy 1, policy_version 1199340 (0.0007) [2023-12-27 00:06:31,434][105692] Updated weights for policy 0, policy_version 1198024 (0.0008) [2023-12-27 00:06:31,458][105620] Updated weights for policy 1, policy_version 1199350 (0.0007) [2023-12-27 00:06:31,498][105692] Updated weights for policy 0, policy_version 1198034 (0.0009) [2023-12-27 00:06:31,524][105620] Updated weights for policy 1, policy_version 1199360 (0.0005) [2023-12-27 00:06:31,561][105692] Updated weights for policy 0, policy_version 1198044 (0.0010) [2023-12-27 00:06:32,214][105620] Updated weights for policy 1, policy_version 1199370 (0.0007) [2023-12-27 00:06:32,284][105620] Updated weights for policy 1, policy_version 1199380 (0.0008) [2023-12-27 00:06:32,356][105620] Updated weights for policy 1, policy_version 1199390 (0.0009) [2023-12-27 00:06:32,403][105692] Updated weights for policy 0, policy_version 1198054 (0.0008) [2023-12-27 00:06:32,465][105692] Updated weights for policy 0, policy_version 1198064 (0.0008) [2023-12-27 00:06:32,527][105692] Updated weights for policy 0, policy_version 1198074 (0.0008) [2023-12-27 00:06:33,093][105620] Updated weights for policy 1, policy_version 1199400 (0.0008) [2023-12-27 00:06:33,151][105620] Updated weights for policy 1, policy_version 1199410 (0.0008) [2023-12-27 00:06:33,212][105620] Updated weights for policy 1, policy_version 1199420 (0.0008) [2023-12-27 00:06:33,303][105692] Updated weights for policy 0, policy_version 1198084 (0.0009) [2023-12-27 00:06:33,359][105692] Updated weights for policy 0, policy_version 1198094 (0.0009) [2023-12-27 00:06:33,414][105692] Updated weights for policy 0, policy_version 1198104 (0.0009) [2023-12-27 00:06:33,948][105620] Updated weights for policy 1, policy_version 1199430 (0.0009) [2023-12-27 00:06:34,018][105620] Updated weights for policy 1, policy_version 1199440 (0.0009) [2023-12-27 00:06:34,086][105620] Updated weights for policy 1, policy_version 1199450 (0.0009) [2023-12-27 00:06:34,123][105692] Updated weights for policy 0, policy_version 1198114 (0.0006) [2023-12-27 00:06:34,189][105692] Updated weights for policy 0, policy_version 1198124 (0.0009) [2023-12-27 00:06:34,258][105692] Updated weights for policy 0, policy_version 1198134 (0.0007) [2023-12-27 00:06:34,328][105692] Updated weights for policy 0, policy_version 1198144 (0.0007) [2023-12-27 00:06:34,907][105620] Updated weights for policy 1, policy_version 1199460 (0.0007) [2023-12-27 00:06:34,966][105620] Updated weights for policy 1, policy_version 1199470 (0.0008) [2023-12-27 00:06:35,024][105620] Updated weights for policy 1, policy_version 1199480 (0.0008) [2023-12-27 00:06:35,052][105692] Updated weights for policy 0, policy_version 1198154 (0.0008) [2023-12-27 00:06:35,109][105692] Updated weights for policy 0, policy_version 1198164 (0.0008) [2023-12-27 00:06:35,166][105692] Updated weights for policy 0, policy_version 1198174 (0.0009) [2023-12-27 00:06:35,799][105620] Updated weights for policy 1, policy_version 1199490 (0.0007) [2023-12-27 00:06:35,863][105620] Updated weights for policy 1, policy_version 1199500 (0.0008) [2023-12-27 00:06:35,865][105692] Updated weights for policy 0, policy_version 1198184 (0.0009) [2023-12-27 00:06:35,916][105692] Updated weights for policy 0, policy_version 1198194 (0.0006) [2023-12-27 00:06:35,918][105620] Updated weights for policy 1, policy_version 1199510 (0.0006) [2023-12-27 00:06:35,978][105620] Updated weights for policy 1, policy_version 1199520 (0.0008) [2023-12-27 00:06:35,985][105692] Updated weights for policy 0, policy_version 1198204 (0.0008) [2023-12-27 00:06:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 18158.9, 300 sec: 18133.5). Total num frames: 613908480. Throughput: 0: 8950.0, 1: 9027.2. Samples: 613891468. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:06:36,062][104569] Avg episode reward: [(0, '9000.246'), (1, '8721.633')] [2023-12-27 00:06:36,774][105620] Updated weights for policy 1, policy_version 1199530 (0.0008) [2023-12-27 00:06:36,777][105692] Updated weights for policy 0, policy_version 1198214 (0.0007) [2023-12-27 00:06:36,837][105620] Updated weights for policy 1, policy_version 1199540 (0.0009) [2023-12-27 00:06:36,845][105692] Updated weights for policy 0, policy_version 1198224 (0.0008) [2023-12-27 00:06:36,902][105620] Updated weights for policy 1, policy_version 1199550 (0.0009) [2023-12-27 00:06:36,908][105692] Updated weights for policy 0, policy_version 1198234 (0.0007) [2023-12-27 00:06:37,632][105620] Updated weights for policy 1, policy_version 1199560 (0.0009) [2023-12-27 00:06:37,671][105692] Updated weights for policy 0, policy_version 1198244 (0.0008) [2023-12-27 00:06:37,698][105620] Updated weights for policy 1, policy_version 1199570 (0.0009) [2023-12-27 00:06:37,734][105692] Updated weights for policy 0, policy_version 1198254 (0.0006) [2023-12-27 00:06:37,760][105620] Updated weights for policy 1, policy_version 1199580 (0.0008) [2023-12-27 00:06:37,796][105692] Updated weights for policy 0, policy_version 1198264 (0.0006) [2023-12-27 00:06:38,513][105620] Updated weights for policy 1, policy_version 1199590 (0.0009) [2023-12-27 00:06:38,525][105692] Updated weights for policy 0, policy_version 1198274 (0.0008) [2023-12-27 00:06:38,579][105620] Updated weights for policy 1, policy_version 1199600 (0.0010) [2023-12-27 00:06:38,589][105692] Updated weights for policy 0, policy_version 1198284 (0.0008) [2023-12-27 00:06:38,644][105620] Updated weights for policy 1, policy_version 1199610 (0.0009) [2023-12-27 00:06:38,652][105692] Updated weights for policy 0, policy_version 1198294 (0.0009) [2023-12-27 00:06:38,715][105692] Updated weights for policy 0, policy_version 1198304 (0.0009) [2023-12-27 00:06:39,425][105620] Updated weights for policy 1, policy_version 1199620 (0.0009) [2023-12-27 00:06:39,495][105620] Updated weights for policy 1, policy_version 1199630 (0.0007) [2023-12-27 00:06:39,513][105692] Updated weights for policy 0, policy_version 1198314 (0.0010) [2023-12-27 00:06:39,553][105620] Updated weights for policy 1, policy_version 1199640 (0.0006) [2023-12-27 00:06:39,575][105692] Updated weights for policy 0, policy_version 1198324 (0.0010) [2023-12-27 00:06:39,637][105692] Updated weights for policy 0, policy_version 1198334 (0.0008) [2023-12-27 00:06:40,376][105620] Updated weights for policy 1, policy_version 1199650 (0.0007) [2023-12-27 00:06:40,442][105692] Updated weights for policy 0, policy_version 1198344 (0.0007) [2023-12-27 00:06:40,447][105620] Updated weights for policy 1, policy_version 1199660 (0.0007) [2023-12-27 00:06:40,496][105692] Updated weights for policy 0, policy_version 1198354 (0.0007) [2023-12-27 00:06:40,515][105620] Updated weights for policy 1, policy_version 1199670 (0.0008) [2023-12-27 00:06:40,562][105692] Updated weights for policy 0, policy_version 1198364 (0.0008) [2023-12-27 00:06:40,577][105620] Updated weights for policy 1, policy_version 1199680 (0.0008) [2023-12-27 00:06:41,062][104569] Fps is (10 sec: 18022.3, 60 sec: 18022.4, 300 sec: 18077.9). Total num frames: 613990400. Throughput: 0: 8965.5, 1: 9009.7. Samples: 613999764. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:06:41,063][104569] Avg episode reward: [(0, '9088.845'), (1, '8812.026')] [2023-12-27 00:06:41,367][105620] Updated weights for policy 1, policy_version 1199690 (0.0009) [2023-12-27 00:06:41,368][105692] Updated weights for policy 0, policy_version 1198374 (0.0009) [2023-12-27 00:06:41,438][105692] Updated weights for policy 0, policy_version 1198384 (0.0007) [2023-12-27 00:06:41,439][105620] Updated weights for policy 1, policy_version 1199700 (0.0008) [2023-12-27 00:06:41,503][105620] Updated weights for policy 1, policy_version 1199710 (0.0008) [2023-12-27 00:06:41,506][105692] Updated weights for policy 0, policy_version 1198394 (0.0006) [2023-12-27 00:06:42,313][105692] Updated weights for policy 0, policy_version 1198404 (0.0007) [2023-12-27 00:06:42,318][105620] Updated weights for policy 1, policy_version 1199720 (0.0009) [2023-12-27 00:06:42,387][105692] Updated weights for policy 0, policy_version 1198414 (0.0009) [2023-12-27 00:06:42,392][105620] Updated weights for policy 1, policy_version 1199730 (0.0009) [2023-12-27 00:06:42,457][105692] Updated weights for policy 0, policy_version 1198424 (0.0008) [2023-12-27 00:06:42,463][105620] Updated weights for policy 1, policy_version 1199740 (0.0007) [2023-12-27 00:06:43,169][105620] Updated weights for policy 1, policy_version 1199750 (0.0008) [2023-12-27 00:06:43,176][105692] Updated weights for policy 0, policy_version 1198434 (0.0009) [2023-12-27 00:06:43,229][105620] Updated weights for policy 1, policy_version 1199760 (0.0007) [2023-12-27 00:06:43,243][105692] Updated weights for policy 0, policy_version 1198444 (0.0006) [2023-12-27 00:06:43,292][105620] Updated weights for policy 1, policy_version 1199770 (0.0009) [2023-12-27 00:06:43,304][105692] Updated weights for policy 0, policy_version 1198454 (0.0008) [2023-12-27 00:06:43,354][105692] Updated weights for policy 0, policy_version 1198464 (0.0008) [2023-12-27 00:06:44,019][105620] Updated weights for policy 1, policy_version 1199780 (0.0009) [2023-12-27 00:06:44,080][105620] Updated weights for policy 1, policy_version 1199790 (0.0005) [2023-12-27 00:06:44,086][105692] Updated weights for policy 0, policy_version 1198474 (0.0011) [2023-12-27 00:06:44,138][105620] Updated weights for policy 1, policy_version 1199800 (0.0006) [2023-12-27 00:06:44,144][105692] Updated weights for policy 0, policy_version 1198484 (0.0011) [2023-12-27 00:06:44,210][105692] Updated weights for policy 0, policy_version 1198494 (0.0010) [2023-12-27 00:06:44,908][105620] Updated weights for policy 1, policy_version 1199810 (0.0008) [2023-12-27 00:06:44,973][105620] Updated weights for policy 1, policy_version 1199820 (0.0008) [2023-12-27 00:06:44,994][105692] Updated weights for policy 0, policy_version 1198504 (0.0009) [2023-12-27 00:06:45,037][105620] Updated weights for policy 1, policy_version 1199830 (0.0008) [2023-12-27 00:06:45,063][105692] Updated weights for policy 0, policy_version 1198514 (0.0008) [2023-12-27 00:06:45,106][105620] Updated weights for policy 1, policy_version 1199840 (0.0008) [2023-12-27 00:06:45,128][105692] Updated weights for policy 0, policy_version 1198524 (0.0008) [2023-12-27 00:06:45,857][105620] Updated weights for policy 1, policy_version 1199850 (0.0011) [2023-12-27 00:06:45,919][105620] Updated weights for policy 1, policy_version 1199860 (0.0011) [2023-12-27 00:06:45,925][105692] Updated weights for policy 0, policy_version 1198534 (0.0007) [2023-12-27 00:06:45,979][105620] Updated weights for policy 1, policy_version 1199870 (0.0011) [2023-12-27 00:06:45,985][105692] Updated weights for policy 0, policy_version 1198544 (0.0006) [2023-12-27 00:06:46,042][105692] Updated weights for policy 0, policy_version 1198554 (0.0008) [2023-12-27 00:06:46,062][104569] Fps is (10 sec: 17203.2, 60 sec: 18022.4, 300 sec: 18105.7). Total num frames: 614080512. Throughput: 0: 8963.8, 1: 9044.8. Samples: 614054272. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:06:46,062][104569] Avg episode reward: [(0, '9083.892'), (1, '9169.485')] [2023-12-27 00:06:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001199872_307208192.pth... [2023-12-27 00:06:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001198816_306937856.pth [2023-12-27 00:06:46,079][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001198560_306880512.pth... [2023-12-27 00:06:46,083][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001197504_306610176.pth [2023-12-27 00:06:46,734][105692] Updated weights for policy 0, policy_version 1198564 (0.0009) [2023-12-27 00:06:46,765][105620] Updated weights for policy 1, policy_version 1199880 (0.0009) [2023-12-27 00:06:46,803][105692] Updated weights for policy 0, policy_version 1198574 (0.0011) [2023-12-27 00:06:46,826][105620] Updated weights for policy 1, policy_version 1199890 (0.0009) [2023-12-27 00:06:46,869][105692] Updated weights for policy 0, policy_version 1198584 (0.0009) [2023-12-27 00:06:46,888][105620] Updated weights for policy 1, policy_version 1199900 (0.0006) [2023-12-27 00:06:47,616][105692] Updated weights for policy 0, policy_version 1198594 (0.0008) [2023-12-27 00:06:47,649][105620] Updated weights for policy 1, policy_version 1199910 (0.0006) [2023-12-27 00:06:47,675][105692] Updated weights for policy 0, policy_version 1198604 (0.0008) [2023-12-27 00:06:47,703][105620] Updated weights for policy 1, policy_version 1199920 (0.0006) [2023-12-27 00:06:47,747][105692] Updated weights for policy 0, policy_version 1198614 (0.0008) [2023-12-27 00:06:47,762][105620] Updated weights for policy 1, policy_version 1199930 (0.0008) [2023-12-27 00:06:47,810][105692] Updated weights for policy 0, policy_version 1198624 (0.0008) [2023-12-27 00:06:48,520][105620] Updated weights for policy 1, policy_version 1199940 (0.0008) [2023-12-27 00:06:48,585][105620] Updated weights for policy 1, policy_version 1199950 (0.0009) [2023-12-27 00:06:48,638][105692] Updated weights for policy 0, policy_version 1198634 (0.0011) [2023-12-27 00:06:48,647][105620] Updated weights for policy 1, policy_version 1199960 (0.0011) [2023-12-27 00:06:48,699][105692] Updated weights for policy 0, policy_version 1198644 (0.0011) [2023-12-27 00:06:48,767][105692] Updated weights for policy 0, policy_version 1198654 (0.0011) [2023-12-27 00:06:49,443][105620] Updated weights for policy 1, policy_version 1199970 (0.0011) [2023-12-27 00:06:49,511][105620] Updated weights for policy 1, policy_version 1199980 (0.0011) [2023-12-27 00:06:49,540][105692] Updated weights for policy 0, policy_version 1198664 (0.0010) [2023-12-27 00:06:49,573][105620] Updated weights for policy 1, policy_version 1199990 (0.0011) [2023-12-27 00:06:49,605][105692] Updated weights for policy 0, policy_version 1198674 (0.0009) [2023-12-27 00:06:49,634][105620] Updated weights for policy 1, policy_version 1200000 (0.0011) [2023-12-27 00:06:49,671][105692] Updated weights for policy 0, policy_version 1198684 (0.0009) [2023-12-27 00:06:50,367][105620] Updated weights for policy 1, policy_version 1200010 (0.0011) [2023-12-27 00:06:50,439][105620] Updated weights for policy 1, policy_version 1200020 (0.0011) [2023-12-27 00:06:50,475][105692] Updated weights for policy 0, policy_version 1198694 (0.0011) [2023-12-27 00:06:50,508][105620] Updated weights for policy 1, policy_version 1200030 (0.0009) [2023-12-27 00:06:50,539][105692] Updated weights for policy 0, policy_version 1198704 (0.0011) [2023-12-27 00:06:50,606][105692] Updated weights for policy 0, policy_version 1198714 (0.0011) [2023-12-27 00:06:51,062][104569] Fps is (10 sec: 18022.6, 60 sec: 18022.4, 300 sec: 18077.9). Total num frames: 614170624. Throughput: 0: 8996.1, 1: 8997.3. Samples: 614162044. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:06:51,063][104569] Avg episode reward: [(0, '9175.615'), (1, '8991.582')] [2023-12-27 00:06:51,295][105620] Updated weights for policy 1, policy_version 1200040 (0.0009) [2023-12-27 00:06:51,366][105620] Updated weights for policy 1, policy_version 1200050 (0.0009) [2023-12-27 00:06:51,369][105692] Updated weights for policy 0, policy_version 1198724 (0.0011) [2023-12-27 00:06:51,434][105692] Updated weights for policy 0, policy_version 1198734 (0.0010) [2023-12-27 00:06:51,440][105620] Updated weights for policy 1, policy_version 1200060 (0.0008) [2023-12-27 00:06:51,499][105692] Updated weights for policy 0, policy_version 1198744 (0.0011) [2023-12-27 00:06:52,242][105620] Updated weights for policy 1, policy_version 1200070 (0.0007) [2023-12-27 00:06:52,270][105692] Updated weights for policy 0, policy_version 1198754 (0.0009) [2023-12-27 00:06:52,311][105620] Updated weights for policy 1, policy_version 1200080 (0.0008) [2023-12-27 00:06:52,334][105692] Updated weights for policy 0, policy_version 1198764 (0.0009) [2023-12-27 00:06:52,385][105620] Updated weights for policy 1, policy_version 1200090 (0.0008) [2023-12-27 00:06:52,398][105692] Updated weights for policy 0, policy_version 1198774 (0.0008) [2023-12-27 00:06:52,450][105692] Updated weights for policy 0, policy_version 1198784 (0.0009) [2023-12-27 00:06:53,114][105620] Updated weights for policy 1, policy_version 1200100 (0.0009) [2023-12-27 00:06:53,166][105620] Updated weights for policy 1, policy_version 1200110 (0.0009) [2023-12-27 00:06:53,227][105620] Updated weights for policy 1, policy_version 1200120 (0.0008) [2023-12-27 00:06:53,236][105692] Updated weights for policy 0, policy_version 1198794 (0.0011) [2023-12-27 00:06:53,296][105692] Updated weights for policy 0, policy_version 1198804 (0.0007) [2023-12-27 00:06:53,358][105692] Updated weights for policy 0, policy_version 1198814 (0.0008) [2023-12-27 00:06:54,044][105620] Updated weights for policy 1, policy_version 1200130 (0.0006) [2023-12-27 00:06:54,058][105692] Updated weights for policy 0, policy_version 1198824 (0.0009) [2023-12-27 00:06:54,107][105620] Updated weights for policy 1, policy_version 1200140 (0.0007) [2023-12-27 00:06:54,118][105692] Updated weights for policy 0, policy_version 1198834 (0.0006) [2023-12-27 00:06:54,171][105620] Updated weights for policy 1, policy_version 1200150 (0.0007) [2023-12-27 00:06:54,183][105692] Updated weights for policy 0, policy_version 1198844 (0.0007) [2023-12-27 00:06:54,238][105620] Updated weights for policy 1, policy_version 1200160 (0.0008) [2023-12-27 00:06:54,987][105692] Updated weights for policy 0, policy_version 1198854 (0.0009) [2023-12-27 00:06:54,988][105620] Updated weights for policy 1, policy_version 1200170 (0.0009) [2023-12-27 00:06:55,054][105620] Updated weights for policy 1, policy_version 1200180 (0.0008) [2023-12-27 00:06:55,057][105692] Updated weights for policy 0, policy_version 1198864 (0.0008) [2023-12-27 00:06:55,122][105692] Updated weights for policy 0, policy_version 1198874 (0.0007) [2023-12-27 00:06:55,122][105620] Updated weights for policy 1, policy_version 1200190 (0.0007) [2023-12-27 00:06:55,876][105620] Updated weights for policy 1, policy_version 1200200 (0.0009) [2023-12-27 00:06:55,932][105620] Updated weights for policy 1, policy_version 1200210 (0.0008) [2023-12-27 00:06:55,943][105692] Updated weights for policy 0, policy_version 1198884 (0.0008) [2023-12-27 00:06:55,995][105620] Updated weights for policy 1, policy_version 1200220 (0.0007) [2023-12-27 00:06:55,997][105692] Updated weights for policy 0, policy_version 1198894 (0.0006) [2023-12-27 00:06:56,058][105692] Updated weights for policy 0, policy_version 1198904 (0.0007) [2023-12-27 00:06:56,062][104569] Fps is (10 sec: 18022.5, 60 sec: 18022.4, 300 sec: 18105.7). Total num frames: 614260736. Throughput: 0: 8983.7, 1: 8984.6. Samples: 614269208. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:06:56,062][104569] Avg episode reward: [(0, '9181.812'), (1, '8991.560')] [2023-12-27 00:06:56,730][105620] Updated weights for policy 1, policy_version 1200230 (0.0009) [2023-12-27 00:06:56,780][105620] Updated weights for policy 1, policy_version 1200240 (0.0007) [2023-12-27 00:06:56,843][105620] Updated weights for policy 1, policy_version 1200250 (0.0006) [2023-12-27 00:06:56,875][105692] Updated weights for policy 0, policy_version 1198914 (0.0009) [2023-12-27 00:06:56,929][105692] Updated weights for policy 0, policy_version 1198924 (0.0009) [2023-12-27 00:06:56,996][105692] Updated weights for policy 0, policy_version 1198934 (0.0009) [2023-12-27 00:06:57,058][105692] Updated weights for policy 0, policy_version 1198944 (0.0007) [2023-12-27 00:06:57,544][105620] Updated weights for policy 1, policy_version 1200260 (0.0007) [2023-12-27 00:06:57,605][105620] Updated weights for policy 1, policy_version 1200270 (0.0006) [2023-12-27 00:06:57,665][105620] Updated weights for policy 1, policy_version 1200280 (0.0006) [2023-12-27 00:06:57,876][105692] Updated weights for policy 0, policy_version 1198954 (0.0008) [2023-12-27 00:06:57,939][105692] Updated weights for policy 0, policy_version 1198964 (0.0008) [2023-12-27 00:06:57,993][105692] Updated weights for policy 0, policy_version 1198974 (0.0008) [2023-12-27 00:06:58,525][105620] Updated weights for policy 1, policy_version 1200290 (0.0009) [2023-12-27 00:06:58,596][105620] Updated weights for policy 1, policy_version 1200300 (0.0007) [2023-12-27 00:06:58,671][105620] Updated weights for policy 1, policy_version 1200310 (0.0007) [2023-12-27 00:06:58,741][105620] Updated weights for policy 1, policy_version 1200320 (0.0008) [2023-12-27 00:06:58,876][105692] Updated weights for policy 0, policy_version 1198984 (0.0008) [2023-12-27 00:06:58,949][105692] Updated weights for policy 0, policy_version 1198994 (0.0008) [2023-12-27 00:06:59,018][105692] Updated weights for policy 0, policy_version 1199004 (0.0008) [2023-12-27 00:06:59,527][105620] Updated weights for policy 1, policy_version 1200330 (0.0009) [2023-12-27 00:06:59,586][105620] Updated weights for policy 1, policy_version 1200340 (0.0011) [2023-12-27 00:06:59,651][105620] Updated weights for policy 1, policy_version 1200350 (0.0010) [2023-12-27 00:06:59,831][105692] Updated weights for policy 0, policy_version 1199014 (0.0008) [2023-12-27 00:06:59,904][105692] Updated weights for policy 0, policy_version 1199024 (0.0007) [2023-12-27 00:06:59,978][105692] Updated weights for policy 0, policy_version 1199034 (0.0008) [2023-12-27 00:07:00,465][105620] Updated weights for policy 1, policy_version 1200360 (0.0010) [2023-12-27 00:07:00,529][105620] Updated weights for policy 1, policy_version 1200370 (0.0009) [2023-12-27 00:07:00,590][105620] Updated weights for policy 1, policy_version 1200380 (0.0008) [2023-12-27 00:07:00,638][105692] Updated weights for policy 0, policy_version 1199044 (0.0008) [2023-12-27 00:07:00,699][105692] Updated weights for policy 0, policy_version 1199054 (0.0009) [2023-12-27 00:07:00,765][105692] Updated weights for policy 0, policy_version 1199064 (0.0008) [2023-12-27 00:07:01,062][104569] Fps is (10 sec: 18022.4, 60 sec: 18022.4, 300 sec: 18105.7). Total num frames: 614350848. Throughput: 0: 8966.5, 1: 9024.4. Samples: 614322780. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:07:01,062][104569] Avg episode reward: [(0, '8908.476'), (1, '9259.400')] [2023-12-27 00:07:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001199072_307011584.pth... [2023-12-27 00:07:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001200384_307339264.pth... [2023-12-27 00:07:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001198016_306741248.pth [2023-12-27 00:07:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001199328_307068928.pth [2023-12-27 00:07:01,385][105620] Updated weights for policy 1, policy_version 1200390 (0.0009) [2023-12-27 00:07:01,450][105620] Updated weights for policy 1, policy_version 1200400 (0.0008) [2023-12-27 00:07:01,483][105692] Updated weights for policy 0, policy_version 1199074 (0.0008) [2023-12-27 00:07:01,513][105620] Updated weights for policy 1, policy_version 1200410 (0.0008) [2023-12-27 00:07:01,554][105692] Updated weights for policy 0, policy_version 1199084 (0.0008) [2023-12-27 00:07:01,618][105692] Updated weights for policy 0, policy_version 1199094 (0.0007) [2023-12-27 00:07:01,671][105692] Updated weights for policy 0, policy_version 1199104 (0.0009) [2023-12-27 00:07:02,296][105620] Updated weights for policy 1, policy_version 1200420 (0.0008) [2023-12-27 00:07:02,360][105620] Updated weights for policy 1, policy_version 1200430 (0.0008) [2023-12-27 00:07:02,431][105620] Updated weights for policy 1, policy_version 1200440 (0.0008) [2023-12-27 00:07:02,439][105692] Updated weights for policy 0, policy_version 1199114 (0.0007) [2023-12-27 00:07:02,498][105692] Updated weights for policy 0, policy_version 1199124 (0.0006) [2023-12-27 00:07:02,556][105692] Updated weights for policy 0, policy_version 1199134 (0.0008) [2023-12-27 00:07:03,166][105620] Updated weights for policy 1, policy_version 1200450 (0.0009) [2023-12-27 00:07:03,219][105620] Updated weights for policy 1, policy_version 1200460 (0.0010) [2023-12-27 00:07:03,273][105692] Updated weights for policy 0, policy_version 1199144 (0.0007) [2023-12-27 00:07:03,277][105620] Updated weights for policy 1, policy_version 1200470 (0.0010) [2023-12-27 00:07:03,331][105692] Updated weights for policy 0, policy_version 1199154 (0.0007) [2023-12-27 00:07:03,340][105620] Updated weights for policy 1, policy_version 1200480 (0.0008) [2023-12-27 00:07:03,381][105692] Updated weights for policy 0, policy_version 1199164 (0.0007) [2023-12-27 00:07:04,031][105692] Updated weights for policy 0, policy_version 1199174 (0.0007) [2023-12-27 00:07:04,098][105692] Updated weights for policy 0, policy_version 1199184 (0.0009) [2023-12-27 00:07:04,140][105620] Updated weights for policy 1, policy_version 1200490 (0.0008) [2023-12-27 00:07:04,165][105692] Updated weights for policy 0, policy_version 1199194 (0.0008) [2023-12-27 00:07:04,204][105620] Updated weights for policy 1, policy_version 1200500 (0.0010) [2023-12-27 00:07:04,269][105620] Updated weights for policy 1, policy_version 1200510 (0.0009) [2023-12-27 00:07:04,958][105692] Updated weights for policy 0, policy_version 1199204 (0.0007) [2023-12-27 00:07:05,016][105692] Updated weights for policy 0, policy_version 1199214 (0.0006) [2023-12-27 00:07:05,031][105620] Updated weights for policy 1, policy_version 1200520 (0.0009) [2023-12-27 00:07:05,082][105692] Updated weights for policy 0, policy_version 1199224 (0.0007) [2023-12-27 00:07:05,093][105620] Updated weights for policy 1, policy_version 1200530 (0.0008) [2023-12-27 00:07:05,149][105620] Updated weights for policy 1, policy_version 1200540 (0.0009) [2023-12-27 00:07:05,743][105692] Updated weights for policy 0, policy_version 1199234 (0.0007) [2023-12-27 00:07:05,809][105692] Updated weights for policy 0, policy_version 1199244 (0.0008) [2023-12-27 00:07:05,873][105692] Updated weights for policy 0, policy_version 1199254 (0.0008) [2023-12-27 00:07:05,934][105692] Updated weights for policy 0, policy_version 1199264 (0.0010) [2023-12-27 00:07:05,973][105620] Updated weights for policy 1, policy_version 1200550 (0.0008) [2023-12-27 00:07:06,039][105620] Updated weights for policy 1, policy_version 1200560 (0.0009) [2023-12-27 00:07:06,062][104569] Fps is (10 sec: 18022.4, 60 sec: 17885.9, 300 sec: 18105.7). Total num frames: 614440960. Throughput: 0: 9000.5, 1: 9026.9. Samples: 614432140. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:07:06,062][104569] Avg episode reward: [(0, '9084.178'), (1, '9350.687')] [2023-12-27 00:07:06,101][105620] Updated weights for policy 1, policy_version 1200570 (0.0009) [2023-12-27 00:07:06,739][105692] Updated weights for policy 0, policy_version 1199274 (0.0010) [2023-12-27 00:07:06,803][105692] Updated weights for policy 0, policy_version 1199284 (0.0010) [2023-12-27 00:07:06,871][105692] Updated weights for policy 0, policy_version 1199294 (0.0009) [2023-12-27 00:07:06,895][105620] Updated weights for policy 1, policy_version 1200580 (0.0008) [2023-12-27 00:07:06,959][105620] Updated weights for policy 1, policy_version 1200590 (0.0009) [2023-12-27 00:07:07,018][105620] Updated weights for policy 1, policy_version 1200600 (0.0008) [2023-12-27 00:07:07,632][105692] Updated weights for policy 0, policy_version 1199304 (0.0006) [2023-12-27 00:07:07,698][105692] Updated weights for policy 0, policy_version 1199314 (0.0008) [2023-12-27 00:07:07,763][105692] Updated weights for policy 0, policy_version 1199324 (0.0009) [2023-12-27 00:07:07,827][105620] Updated weights for policy 1, policy_version 1200610 (0.0009) [2023-12-27 00:07:07,878][105620] Updated weights for policy 1, policy_version 1200620 (0.0009) [2023-12-27 00:07:07,946][105620] Updated weights for policy 1, policy_version 1200630 (0.0011) [2023-12-27 00:07:08,006][105620] Updated weights for policy 1, policy_version 1200640 (0.0011) [2023-12-27 00:07:08,550][105692] Updated weights for policy 0, policy_version 1199334 (0.0009) [2023-12-27 00:07:08,612][105692] Updated weights for policy 0, policy_version 1199344 (0.0009) [2023-12-27 00:07:08,675][105692] Updated weights for policy 0, policy_version 1199354 (0.0009) [2023-12-27 00:07:08,811][105620] Updated weights for policy 1, policy_version 1200650 (0.0009) [2023-12-27 00:07:08,868][105620] Updated weights for policy 1, policy_version 1200660 (0.0008) [2023-12-27 00:07:08,936][105620] Updated weights for policy 1, policy_version 1200670 (0.0010) [2023-12-27 00:07:09,491][105692] Updated weights for policy 0, policy_version 1199364 (0.0008) [2023-12-27 00:07:09,557][105692] Updated weights for policy 0, policy_version 1199374 (0.0009) [2023-12-27 00:07:09,623][105692] Updated weights for policy 0, policy_version 1199384 (0.0009) [2023-12-27 00:07:09,714][105620] Updated weights for policy 1, policy_version 1200680 (0.0010) [2023-12-27 00:07:09,774][105620] Updated weights for policy 1, policy_version 1200690 (0.0009) [2023-12-27 00:07:09,842][105620] Updated weights for policy 1, policy_version 1200700 (0.0009) [2023-12-27 00:07:10,495][105692] Updated weights for policy 0, policy_version 1199394 (0.0009) [2023-12-27 00:07:10,560][105692] Updated weights for policy 0, policy_version 1199404 (0.0009) [2023-12-27 00:07:10,628][105692] Updated weights for policy 0, policy_version 1199414 (0.0009) [2023-12-27 00:07:10,675][105620] Updated weights for policy 1, policy_version 1200710 (0.0011) [2023-12-27 00:07:10,689][105692] Updated weights for policy 0, policy_version 1199424 (0.0007) [2023-12-27 00:07:10,738][105620] Updated weights for policy 1, policy_version 1200720 (0.0008) [2023-12-27 00:07:10,800][105620] Updated weights for policy 1, policy_version 1200730 (0.0009) [2023-12-27 00:07:11,062][104569] Fps is (10 sec: 18022.4, 60 sec: 18022.4, 300 sec: 18105.7). Total num frames: 614531072. Throughput: 0: 9030.4, 1: 9012.7. Samples: 614538464. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:07:11,063][104569] Avg episode reward: [(0, '9356.398'), (1, '9175.168')] [2023-12-27 00:07:11,546][105692] Updated weights for policy 0, policy_version 1199434 (0.0009) [2023-12-27 00:07:11,608][105692] Updated weights for policy 0, policy_version 1199444 (0.0009) [2023-12-27 00:07:11,637][105620] Updated weights for policy 1, policy_version 1200740 (0.0009) [2023-12-27 00:07:11,683][105692] Updated weights for policy 0, policy_version 1199454 (0.0009) [2023-12-27 00:07:11,705][105620] Updated weights for policy 1, policy_version 1200750 (0.0009) [2023-12-27 00:07:11,770][105620] Updated weights for policy 1, policy_version 1200760 (0.0011) [2023-12-27 00:07:12,537][105620] Updated weights for policy 1, policy_version 1200770 (0.0011) [2023-12-27 00:07:12,548][105692] Updated weights for policy 0, policy_version 1199464 (0.0008) [2023-12-27 00:07:12,602][105620] Updated weights for policy 1, policy_version 1200780 (0.0011) [2023-12-27 00:07:12,605][105692] Updated weights for policy 0, policy_version 1199474 (0.0007) [2023-12-27 00:07:12,665][105692] Updated weights for policy 0, policy_version 1199484 (0.0007) [2023-12-27 00:07:12,670][105620] Updated weights for policy 1, policy_version 1200790 (0.0007) [2023-12-27 00:07:12,735][105620] Updated weights for policy 1, policy_version 1200800 (0.0007) [2023-12-27 00:07:13,424][105620] Updated weights for policy 1, policy_version 1200810 (0.0011) [2023-12-27 00:07:13,488][105620] Updated weights for policy 1, policy_version 1200820 (0.0011) [2023-12-27 00:07:13,491][105692] Updated weights for policy 0, policy_version 1199494 (0.0007) [2023-12-27 00:07:13,546][105620] Updated weights for policy 1, policy_version 1200830 (0.0008) [2023-12-27 00:07:13,552][105692] Updated weights for policy 0, policy_version 1199504 (0.0008) [2023-12-27 00:07:13,615][105692] Updated weights for policy 0, policy_version 1199514 (0.0010) [2023-12-27 00:07:14,230][105620] Updated weights for policy 1, policy_version 1200840 (0.0009) [2023-12-27 00:07:14,275][105620] Updated weights for policy 1, policy_version 1200850 (0.0010) [2023-12-27 00:07:14,313][105692] Updated weights for policy 0, policy_version 1199524 (0.0010) [2023-12-27 00:07:14,339][105620] Updated weights for policy 1, policy_version 1200860 (0.0010) [2023-12-27 00:07:14,379][105692] Updated weights for policy 0, policy_version 1199534 (0.0009) [2023-12-27 00:07:14,432][105692] Updated weights for policy 0, policy_version 1199544 (0.0009) [2023-12-27 00:07:15,126][105620] Updated weights for policy 1, policy_version 1200870 (0.0006) [2023-12-27 00:07:15,174][105692] Updated weights for policy 0, policy_version 1199554 (0.0008) [2023-12-27 00:07:15,192][105620] Updated weights for policy 1, policy_version 1200880 (0.0007) [2023-12-27 00:07:15,242][105692] Updated weights for policy 0, policy_version 1199564 (0.0009) [2023-12-27 00:07:15,258][105620] Updated weights for policy 1, policy_version 1200890 (0.0009) [2023-12-27 00:07:15,306][105692] Updated weights for policy 0, policy_version 1199574 (0.0006) [2023-12-27 00:07:15,366][105692] Updated weights for policy 0, policy_version 1199584 (0.0009) [2023-12-27 00:07:15,988][105620] Updated weights for policy 1, policy_version 1200900 (0.0011) [2023-12-27 00:07:16,042][105620] Updated weights for policy 1, policy_version 1200910 (0.0011) [2023-12-27 00:07:16,062][104569] Fps is (10 sec: 17203.1, 60 sec: 17885.9, 300 sec: 18105.7). Total num frames: 614612992. Throughput: 0: 8940.8, 1: 9022.0. Samples: 614591476. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:07:16,062][104569] Avg episode reward: [(0, '9357.166'), (1, '9084.097')] [2023-12-27 00:07:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001199584_307142656.pth... [2023-12-27 00:07:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001198560_306880512.pth [2023-12-27 00:07:16,107][105620] Updated weights for policy 1, policy_version 1200920 (0.0011) [2023-12-27 00:07:16,159][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001200928_307478528.pth... [2023-12-27 00:07:16,162][105692] Updated weights for policy 0, policy_version 1199594 (0.0007) [2023-12-27 00:07:16,163][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001199872_307208192.pth [2023-12-27 00:07:16,228][105692] Updated weights for policy 0, policy_version 1199604 (0.0008) [2023-12-27 00:07:16,284][105692] Updated weights for policy 0, policy_version 1199614 (0.0010) [2023-12-27 00:07:16,830][105620] Updated weights for policy 1, policy_version 1200930 (0.0010) [2023-12-27 00:07:16,897][105620] Updated weights for policy 1, policy_version 1200940 (0.0010) [2023-12-27 00:07:16,957][105620] Updated weights for policy 1, policy_version 1200950 (0.0010) [2023-12-27 00:07:17,016][105620] Updated weights for policy 1, policy_version 1200960 (0.0008) [2023-12-27 00:07:17,036][105692] Updated weights for policy 0, policy_version 1199624 (0.0011) [2023-12-27 00:07:17,096][105692] Updated weights for policy 0, policy_version 1199634 (0.0011) [2023-12-27 00:07:17,156][105692] Updated weights for policy 0, policy_version 1199644 (0.0011) [2023-12-27 00:07:17,733][105620] Updated weights for policy 1, policy_version 1200970 (0.0007) [2023-12-27 00:07:17,792][105620] Updated weights for policy 1, policy_version 1200980 (0.0008) [2023-12-27 00:07:17,846][105620] Updated weights for policy 1, policy_version 1200990 (0.0005) [2023-12-27 00:07:17,881][105692] Updated weights for policy 0, policy_version 1199654 (0.0009) [2023-12-27 00:07:17,943][105692] Updated weights for policy 0, policy_version 1199664 (0.0009) [2023-12-27 00:07:18,002][105692] Updated weights for policy 0, policy_version 1199674 (0.0009) [2023-12-27 00:07:18,568][105620] Updated weights for policy 1, policy_version 1201000 (0.0008) [2023-12-27 00:07:18,624][105620] Updated weights for policy 1, policy_version 1201010 (0.0009) [2023-12-27 00:07:18,688][105620] Updated weights for policy 1, policy_version 1201020 (0.0009) [2023-12-27 00:07:18,827][105692] Updated weights for policy 0, policy_version 1199684 (0.0010) [2023-12-27 00:07:18,891][105692] Updated weights for policy 0, policy_version 1199694 (0.0008) [2023-12-27 00:07:18,954][105692] Updated weights for policy 0, policy_version 1199704 (0.0008) [2023-12-27 00:07:19,441][105620] Updated weights for policy 1, policy_version 1201030 (0.0009) [2023-12-27 00:07:19,510][105620] Updated weights for policy 1, policy_version 1201040 (0.0007) [2023-12-27 00:07:19,581][105620] Updated weights for policy 1, policy_version 1201050 (0.0008) [2023-12-27 00:07:19,759][105692] Updated weights for policy 0, policy_version 1199714 (0.0008) [2023-12-27 00:07:19,826][105692] Updated weights for policy 0, policy_version 1199724 (0.0008) [2023-12-27 00:07:19,899][105692] Updated weights for policy 0, policy_version 1199734 (0.0009) [2023-12-27 00:07:19,971][105692] Updated weights for policy 0, policy_version 1199744 (0.0009) [2023-12-27 00:07:20,410][105620] Updated weights for policy 1, policy_version 1201060 (0.0008) [2023-12-27 00:07:20,476][105620] Updated weights for policy 1, policy_version 1201070 (0.0007) [2023-12-27 00:07:20,541][105620] Updated weights for policy 1, policy_version 1201080 (0.0008) [2023-12-27 00:07:20,689][105692] Updated weights for policy 0, policy_version 1199754 (0.0009) [2023-12-27 00:07:20,759][105692] Updated weights for policy 0, policy_version 1199764 (0.0009) [2023-12-27 00:07:20,820][105692] Updated weights for policy 0, policy_version 1199774 (0.0009) [2023-12-27 00:07:21,062][104569] Fps is (10 sec: 18022.4, 60 sec: 18022.4, 300 sec: 18105.7). Total num frames: 614711296. Throughput: 0: 8992.8, 1: 9031.3. Samples: 614702552. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:07:21,063][104569] Avg episode reward: [(0, '9178.057'), (1, '9260.350')] [2023-12-27 00:07:21,368][105620] Updated weights for policy 1, policy_version 1201090 (0.0010) [2023-12-27 00:07:21,435][105620] Updated weights for policy 1, policy_version 1201100 (0.0010) [2023-12-27 00:07:21,499][105620] Updated weights for policy 1, policy_version 1201110 (0.0011) [2023-12-27 00:07:21,565][105620] Updated weights for policy 1, policy_version 1201120 (0.0009) [2023-12-27 00:07:21,625][105692] Updated weights for policy 0, policy_version 1199784 (0.0009) [2023-12-27 00:07:21,698][105692] Updated weights for policy 0, policy_version 1199794 (0.0009) [2023-12-27 00:07:21,768][105692] Updated weights for policy 0, policy_version 1199804 (0.0009) [2023-12-27 00:07:22,411][105620] Updated weights for policy 1, policy_version 1201130 (0.0008) [2023-12-27 00:07:22,478][105620] Updated weights for policy 1, policy_version 1201140 (0.0009) [2023-12-27 00:07:22,548][105620] Updated weights for policy 1, policy_version 1201150 (0.0008) [2023-12-27 00:07:22,594][105692] Updated weights for policy 0, policy_version 1199814 (0.0011) [2023-12-27 00:07:22,656][105692] Updated weights for policy 0, policy_version 1199824 (0.0009) [2023-12-27 00:07:22,684][105585] KL-divergence is very high: 112.4273 [2023-12-27 00:07:22,724][105692] Updated weights for policy 0, policy_version 1199834 (0.0007) [2023-12-27 00:07:22,735][105585] KL-divergence is very high: 116.9819 [2023-12-27 00:07:23,284][105620] Updated weights for policy 1, policy_version 1201160 (0.0010) [2023-12-27 00:07:23,356][105620] Updated weights for policy 1, policy_version 1201170 (0.0010) [2023-12-27 00:07:23,421][105620] Updated weights for policy 1, policy_version 1201180 (0.0009) [2023-12-27 00:07:23,440][105692] Updated weights for policy 0, policy_version 1199844 (0.0010) [2023-12-27 00:07:23,508][105692] Updated weights for policy 0, policy_version 1199854 (0.0009) [2023-12-27 00:07:23,565][105692] Updated weights for policy 0, policy_version 1199864 (0.0009) [2023-12-27 00:07:24,155][105620] Updated weights for policy 1, policy_version 1201190 (0.0009) [2023-12-27 00:07:24,223][105620] Updated weights for policy 1, policy_version 1201200 (0.0010) [2023-12-27 00:07:24,295][105620] Updated weights for policy 1, policy_version 1201210 (0.0010) [2023-12-27 00:07:24,374][105692] Updated weights for policy 0, policy_version 1199874 (0.0009) [2023-12-27 00:07:24,434][105692] Updated weights for policy 0, policy_version 1199884 (0.0008) [2023-12-27 00:07:24,494][105692] Updated weights for policy 0, policy_version 1199894 (0.0008) [2023-12-27 00:07:24,552][105692] Updated weights for policy 0, policy_version 1199904 (0.0009) [2023-12-27 00:07:25,030][105620] Updated weights for policy 1, policy_version 1201220 (0.0009) [2023-12-27 00:07:25,077][105620] Updated weights for policy 1, policy_version 1201230 (0.0011) [2023-12-27 00:07:25,134][105620] Updated weights for policy 1, policy_version 1201240 (0.0011) [2023-12-27 00:07:25,348][105692] Updated weights for policy 0, policy_version 1199914 (0.0008) [2023-12-27 00:07:25,417][105692] Updated weights for policy 0, policy_version 1199924 (0.0008) [2023-12-27 00:07:25,475][105692] Updated weights for policy 0, policy_version 1199934 (0.0009) [2023-12-27 00:07:25,951][105620] Updated weights for policy 1, policy_version 1201250 (0.0011) [2023-12-27 00:07:26,016][105620] Updated weights for policy 1, policy_version 1201260 (0.0011) [2023-12-27 00:07:26,062][104569] Fps is (10 sec: 18022.5, 60 sec: 18022.4, 300 sec: 18105.7). Total num frames: 614793216. Throughput: 0: 8955.1, 1: 9008.6. Samples: 614808124. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:07:26,062][104569] Avg episode reward: [(0, '8997.001'), (1, '9169.837')] [2023-12-27 00:07:26,070][105620] Updated weights for policy 1, policy_version 1201270 (0.0011) [2023-12-27 00:07:26,131][105620] Updated weights for policy 1, policy_version 1201280 (0.0011) [2023-12-27 00:07:26,276][105692] Updated weights for policy 0, policy_version 1199944 (0.0008) [2023-12-27 00:07:26,340][105692] Updated weights for policy 0, policy_version 1199954 (0.0009) [2023-12-27 00:07:26,410][105692] Updated weights for policy 0, policy_version 1199964 (0.0009) [2023-12-27 00:07:26,816][105620] Updated weights for policy 1, policy_version 1201290 (0.0009) [2023-12-27 00:07:26,884][105620] Updated weights for policy 1, policy_version 1201300 (0.0011) [2023-12-27 00:07:26,941][105620] Updated weights for policy 1, policy_version 1201310 (0.0011) [2023-12-27 00:07:27,132][105692] Updated weights for policy 0, policy_version 1199974 (0.0007) [2023-12-27 00:07:27,201][105692] Updated weights for policy 0, policy_version 1199984 (0.0006) [2023-12-27 00:07:27,266][105692] Updated weights for policy 0, policy_version 1199994 (0.0005) [2023-12-27 00:07:27,646][105620] Updated weights for policy 1, policy_version 1201320 (0.0011) [2023-12-27 00:07:27,694][105620] Updated weights for policy 1, policy_version 1201330 (0.0010) [2023-12-27 00:07:27,747][105620] Updated weights for policy 1, policy_version 1201340 (0.0010) [2023-12-27 00:07:27,886][105692] Updated weights for policy 0, policy_version 1200004 (0.0009) [2023-12-27 00:07:27,946][105692] Updated weights for policy 0, policy_version 1200014 (0.0011) [2023-12-27 00:07:28,008][105692] Updated weights for policy 0, policy_version 1200024 (0.0011) [2023-12-27 00:07:28,527][105620] Updated weights for policy 1, policy_version 1201350 (0.0010) [2023-12-27 00:07:28,598][105620] Updated weights for policy 1, policy_version 1201360 (0.0011) [2023-12-27 00:07:28,663][105620] Updated weights for policy 1, policy_version 1201370 (0.0011) [2023-12-27 00:07:28,765][105692] Updated weights for policy 0, policy_version 1200034 (0.0011) [2023-12-27 00:07:28,827][105692] Updated weights for policy 0, policy_version 1200044 (0.0010) [2023-12-27 00:07:28,894][105692] Updated weights for policy 0, policy_version 1200054 (0.0010) [2023-12-27 00:07:28,959][105692] Updated weights for policy 0, policy_version 1200064 (0.0011) [2023-12-27 00:07:29,418][105620] Updated weights for policy 1, policy_version 1201380 (0.0011) [2023-12-27 00:07:29,473][105620] Updated weights for policy 1, policy_version 1201390 (0.0010) [2023-12-27 00:07:29,524][105620] Updated weights for policy 1, policy_version 1201400 (0.0008) [2023-12-27 00:07:29,726][105692] Updated weights for policy 0, policy_version 1200074 (0.0010) [2023-12-27 00:07:29,786][105692] Updated weights for policy 0, policy_version 1200084 (0.0011) [2023-12-27 00:07:29,856][105692] Updated weights for policy 0, policy_version 1200094 (0.0009) [2023-12-27 00:07:30,268][105620] Updated weights for policy 1, policy_version 1201410 (0.0006) [2023-12-27 00:07:30,330][105620] Updated weights for policy 1, policy_version 1201420 (0.0008) [2023-12-27 00:07:30,396][105620] Updated weights for policy 1, policy_version 1201430 (0.0008) [2023-12-27 00:07:30,465][105620] Updated weights for policy 1, policy_version 1201440 (0.0008) [2023-12-27 00:07:30,623][105692] Updated weights for policy 0, policy_version 1200104 (0.0009) [2023-12-27 00:07:30,679][105692] Updated weights for policy 0, policy_version 1200114 (0.0009) [2023-12-27 00:07:30,734][105692] Updated weights for policy 0, policy_version 1200124 (0.0009) [2023-12-27 00:07:31,062][104569] Fps is (10 sec: 18022.3, 60 sec: 18022.4, 300 sec: 18133.5). Total num frames: 614891520. Throughput: 0: 8995.0, 1: 9032.6. Samples: 614865516. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:07:31,063][104569] Avg episode reward: [(0, '9088.334'), (1, '9079.710')] [2023-12-27 00:07:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001200128_307281920.pth... [2023-12-27 00:07:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001199072_307011584.pth [2023-12-27 00:07:31,130][105620] Updated weights for policy 1, policy_version 1201450 (0.0009) [2023-12-27 00:07:31,194][105620] Updated weights for policy 1, policy_version 1201460 (0.0008) [2023-12-27 00:07:31,257][105620] Updated weights for policy 1, policy_version 1201470 (0.0009) [2023-12-27 00:07:31,270][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001201472_307617792.pth... [2023-12-27 00:07:31,273][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001200384_307339264.pth [2023-12-27 00:07:31,534][105692] Updated weights for policy 0, policy_version 1200134 (0.0008) [2023-12-27 00:07:31,594][105692] Updated weights for policy 0, policy_version 1200144 (0.0009) [2023-12-27 00:07:31,659][105692] Updated weights for policy 0, policy_version 1200154 (0.0008) [2023-12-27 00:07:32,091][105620] Updated weights for policy 1, policy_version 1201480 (0.0010) [2023-12-27 00:07:32,162][105620] Updated weights for policy 1, policy_version 1201490 (0.0009) [2023-12-27 00:07:32,224][105620] Updated weights for policy 1, policy_version 1201500 (0.0008) [2023-12-27 00:07:32,418][105692] Updated weights for policy 0, policy_version 1200164 (0.0009) [2023-12-27 00:07:32,480][105692] Updated weights for policy 0, policy_version 1200174 (0.0009) [2023-12-27 00:07:32,545][105692] Updated weights for policy 0, policy_version 1200184 (0.0008) [2023-12-27 00:07:33,000][105620] Updated weights for policy 1, policy_version 1201510 (0.0010) [2023-12-27 00:07:33,051][105620] Updated weights for policy 1, policy_version 1201520 (0.0011) [2023-12-27 00:07:33,104][105620] Updated weights for policy 1, policy_version 1201530 (0.0011) [2023-12-27 00:07:33,330][105692] Updated weights for policy 0, policy_version 1200194 (0.0009) [2023-12-27 00:07:33,389][105692] Updated weights for policy 0, policy_version 1200204 (0.0008) [2023-12-27 00:07:33,449][105692] Updated weights for policy 0, policy_version 1200214 (0.0008) [2023-12-27 00:07:33,505][105692] Updated weights for policy 0, policy_version 1200224 (0.0008) [2023-12-27 00:07:33,869][105620] Updated weights for policy 1, policy_version 1201540 (0.0011) [2023-12-27 00:07:33,925][105620] Updated weights for policy 1, policy_version 1201550 (0.0010) [2023-12-27 00:07:33,990][105620] Updated weights for policy 1, policy_version 1201560 (0.0011) [2023-12-27 00:07:34,313][105692] Updated weights for policy 0, policy_version 1200234 (0.0009) [2023-12-27 00:07:34,367][105692] Updated weights for policy 0, policy_version 1200244 (0.0009) [2023-12-27 00:07:34,433][105692] Updated weights for policy 0, policy_version 1200254 (0.0009) [2023-12-27 00:07:34,719][105620] Updated weights for policy 1, policy_version 1201570 (0.0010) [2023-12-27 00:07:34,789][105620] Updated weights for policy 1, policy_version 1201580 (0.0008) [2023-12-27 00:07:34,877][105620] Updated weights for policy 1, policy_version 1201590 (0.0009) [2023-12-27 00:07:34,945][105620] Updated weights for policy 1, policy_version 1201600 (0.0009) [2023-12-27 00:07:35,260][105692] Updated weights for policy 0, policy_version 1200264 (0.0009) [2023-12-27 00:07:35,315][105692] Updated weights for policy 0, policy_version 1200274 (0.0009) [2023-12-27 00:07:35,374][105692] Updated weights for policy 0, policy_version 1200284 (0.0009) [2023-12-27 00:07:35,633][105620] Updated weights for policy 1, policy_version 1201610 (0.0009) [2023-12-27 00:07:35,684][105620] Updated weights for policy 1, policy_version 1201620 (0.0009) [2023-12-27 00:07:35,743][105620] Updated weights for policy 1, policy_version 1201630 (0.0006) [2023-12-27 00:07:36,062][104569] Fps is (10 sec: 18841.5, 60 sec: 17885.9, 300 sec: 18133.5). Total num frames: 614981632. Throughput: 0: 8973.7, 1: 9061.9. Samples: 614973644. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:07:36,062][104569] Avg episode reward: [(0, '9180.310'), (1, '9080.751')] [2023-12-27 00:07:36,206][105692] Updated weights for policy 0, policy_version 1200294 (0.0009) [2023-12-27 00:07:36,277][105692] Updated weights for policy 0, policy_version 1200304 (0.0009) [2023-12-27 00:07:36,341][105692] Updated weights for policy 0, policy_version 1200314 (0.0010) [2023-12-27 00:07:36,501][105620] Updated weights for policy 1, policy_version 1201640 (0.0007) [2023-12-27 00:07:36,563][105620] Updated weights for policy 1, policy_version 1201650 (0.0008) [2023-12-27 00:07:36,627][105620] Updated weights for policy 1, policy_version 1201660 (0.0008) [2023-12-27 00:07:37,112][105692] Updated weights for policy 0, policy_version 1200324 (0.0009) [2023-12-27 00:07:37,164][105692] Updated weights for policy 0, policy_version 1200334 (0.0009) [2023-12-27 00:07:37,221][105692] Updated weights for policy 0, policy_version 1200344 (0.0009) [2023-12-27 00:07:37,374][105620] Updated weights for policy 1, policy_version 1201670 (0.0009) [2023-12-27 00:07:37,437][105620] Updated weights for policy 1, policy_version 1201680 (0.0010) [2023-12-27 00:07:37,496][105620] Updated weights for policy 1, policy_version 1201690 (0.0009) [2023-12-27 00:07:38,010][105692] Updated weights for policy 0, policy_version 1200354 (0.0009) [2023-12-27 00:07:38,074][105692] Updated weights for policy 0, policy_version 1200364 (0.0009) [2023-12-27 00:07:38,137][105692] Updated weights for policy 0, policy_version 1200374 (0.0008) [2023-12-27 00:07:38,204][105692] Updated weights for policy 0, policy_version 1200384 (0.0009) [2023-12-27 00:07:38,276][105620] Updated weights for policy 1, policy_version 1201700 (0.0009) [2023-12-27 00:07:38,341][105620] Updated weights for policy 1, policy_version 1201710 (0.0009) [2023-12-27 00:07:38,407][105620] Updated weights for policy 1, policy_version 1201720 (0.0007) [2023-12-27 00:07:39,010][105692] Updated weights for policy 0, policy_version 1200394 (0.0009) [2023-12-27 00:07:39,076][105692] Updated weights for policy 0, policy_version 1200404 (0.0009) [2023-12-27 00:07:39,139][105692] Updated weights for policy 0, policy_version 1200414 (0.0009) [2023-12-27 00:07:39,150][105620] Updated weights for policy 1, policy_version 1201730 (0.0008) [2023-12-27 00:07:39,216][105620] Updated weights for policy 1, policy_version 1201740 (0.0009) [2023-12-27 00:07:39,286][105620] Updated weights for policy 1, policy_version 1201750 (0.0007) [2023-12-27 00:07:39,354][105620] Updated weights for policy 1, policy_version 1201760 (0.0008) [2023-12-27 00:07:39,955][105692] Updated weights for policy 0, policy_version 1200424 (0.0009) [2023-12-27 00:07:40,023][105692] Updated weights for policy 0, policy_version 1200434 (0.0008) [2023-12-27 00:07:40,090][105692] Updated weights for policy 0, policy_version 1200444 (0.0009) [2023-12-27 00:07:40,191][105620] Updated weights for policy 1, policy_version 1201770 (0.0009) [2023-12-27 00:07:40,258][105620] Updated weights for policy 1, policy_version 1201780 (0.0008) [2023-12-27 00:07:40,324][105620] Updated weights for policy 1, policy_version 1201790 (0.0008) [2023-12-27 00:07:40,872][105692] Updated weights for policy 0, policy_version 1200454 (0.0009) [2023-12-27 00:07:40,931][105692] Updated weights for policy 0, policy_version 1200464 (0.0007) [2023-12-27 00:07:41,001][105692] Updated weights for policy 0, policy_version 1200474 (0.0008) [2023-12-27 00:07:41,062][104569] Fps is (10 sec: 18022.6, 60 sec: 18022.4, 300 sec: 18133.5). Total num frames: 615071744. Throughput: 0: 8951.6, 1: 9080.0. Samples: 615080632. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:07:41,063][104569] Avg episode reward: [(0, '9177.902'), (1, '8898.228')] [2023-12-27 00:07:41,116][105620] Updated weights for policy 1, policy_version 1201800 (0.0009) [2023-12-27 00:07:41,179][105620] Updated weights for policy 1, policy_version 1201810 (0.0009) [2023-12-27 00:07:41,234][105620] Updated weights for policy 1, policy_version 1201820 (0.0009) [2023-12-27 00:07:41,754][105692] Updated weights for policy 0, policy_version 1200484 (0.0007) [2023-12-27 00:07:41,816][105692] Updated weights for policy 0, policy_version 1200494 (0.0008) [2023-12-27 00:07:41,870][105692] Updated weights for policy 0, policy_version 1200504 (0.0008) [2023-12-27 00:07:42,085][105620] Updated weights for policy 1, policy_version 1201830 (0.0010) [2023-12-27 00:07:42,135][105620] Updated weights for policy 1, policy_version 1201840 (0.0011) [2023-12-27 00:07:42,196][105620] Updated weights for policy 1, policy_version 1201850 (0.0008) [2023-12-27 00:07:42,679][105692] Updated weights for policy 0, policy_version 1200514 (0.0007) [2023-12-27 00:07:42,737][105692] Updated weights for policy 0, policy_version 1200524 (0.0008) [2023-12-27 00:07:42,800][105692] Updated weights for policy 0, policy_version 1200534 (0.0009) [2023-12-27 00:07:42,867][105692] Updated weights for policy 0, policy_version 1200544 (0.0009) [2023-12-27 00:07:42,970][105620] Updated weights for policy 1, policy_version 1201860 (0.0008) [2023-12-27 00:07:43,026][105620] Updated weights for policy 1, policy_version 1201870 (0.0010) [2023-12-27 00:07:43,087][105620] Updated weights for policy 1, policy_version 1201880 (0.0011) [2023-12-27 00:07:43,699][105692] Updated weights for policy 0, policy_version 1200554 (0.0009) [2023-12-27 00:07:43,755][105692] Updated weights for policy 0, policy_version 1200564 (0.0009) [2023-12-27 00:07:43,806][105692] Updated weights for policy 0, policy_version 1200574 (0.0008) [2023-12-27 00:07:43,813][105620] Updated weights for policy 1, policy_version 1201890 (0.0010) [2023-12-27 00:07:43,863][105620] Updated weights for policy 1, policy_version 1201900 (0.0009) [2023-12-27 00:07:43,915][105620] Updated weights for policy 1, policy_version 1201910 (0.0008) [2023-12-27 00:07:43,966][105620] Updated weights for policy 1, policy_version 1201920 (0.0008) [2023-12-27 00:07:44,585][105692] Updated weights for policy 0, policy_version 1200584 (0.0010) [2023-12-27 00:07:44,652][105692] Updated weights for policy 0, policy_version 1200594 (0.0009) [2023-12-27 00:07:44,711][105692] Updated weights for policy 0, policy_version 1200604 (0.0006) [2023-12-27 00:07:44,717][105620] Updated weights for policy 1, policy_version 1201930 (0.0009) [2023-12-27 00:07:44,778][105620] Updated weights for policy 1, policy_version 1201940 (0.0007) [2023-12-27 00:07:44,847][105620] Updated weights for policy 1, policy_version 1201950 (0.0009) [2023-12-27 00:07:45,522][105692] Updated weights for policy 0, policy_version 1200614 (0.0006) [2023-12-27 00:07:45,537][105620] Updated weights for policy 1, policy_version 1201960 (0.0008) [2023-12-27 00:07:45,589][105692] Updated weights for policy 0, policy_version 1200624 (0.0007) [2023-12-27 00:07:45,595][105620] Updated weights for policy 1, policy_version 1201970 (0.0006) [2023-12-27 00:07:45,648][105620] Updated weights for policy 1, policy_version 1201980 (0.0008) [2023-12-27 00:07:45,655][105692] Updated weights for policy 0, policy_version 1200634 (0.0006) [2023-12-27 00:07:46,062][104569] Fps is (10 sec: 18022.1, 60 sec: 18022.4, 300 sec: 18161.2). Total num frames: 615161856. Throughput: 0: 8975.7, 1: 9055.2. Samples: 615134172. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:07:46,063][104569] Avg episode reward: [(0, '9180.345'), (1, '8810.637')] [2023-12-27 00:07:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001200640_307412992.pth... [2023-12-27 00:07:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001201984_307748864.pth... [2023-12-27 00:07:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001200928_307478528.pth [2023-12-27 00:07:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001199584_307142656.pth [2023-12-27 00:07:46,380][105692] Updated weights for policy 0, policy_version 1200644 (0.0008) [2023-12-27 00:07:46,394][105620] Updated weights for policy 1, policy_version 1201990 (0.0009) [2023-12-27 00:07:46,432][105692] Updated weights for policy 0, policy_version 1200654 (0.0008) [2023-12-27 00:07:46,454][105620] Updated weights for policy 1, policy_version 1202000 (0.0009) [2023-12-27 00:07:46,486][105692] Updated weights for policy 0, policy_version 1200664 (0.0007) [2023-12-27 00:07:46,510][105620] Updated weights for policy 1, policy_version 1202010 (0.0007) [2023-12-27 00:07:47,106][105620] Updated weights for policy 1, policy_version 1202020 (0.0008) [2023-12-27 00:07:47,160][105620] Updated weights for policy 1, policy_version 1202030 (0.0011) [2023-12-27 00:07:47,210][105620] Updated weights for policy 1, policy_version 1202040 (0.0009) [2023-12-27 00:07:47,299][105692] Updated weights for policy 0, policy_version 1200674 (0.0008) [2023-12-27 00:07:47,361][105692] Updated weights for policy 0, policy_version 1200684 (0.0009) [2023-12-27 00:07:47,422][105692] Updated weights for policy 0, policy_version 1200694 (0.0009) [2023-12-27 00:07:47,481][105692] Updated weights for policy 0, policy_version 1200704 (0.0009) [2023-12-27 00:07:47,950][105620] Updated weights for policy 1, policy_version 1202050 (0.0006) [2023-12-27 00:07:47,999][105620] Updated weights for policy 1, policy_version 1202060 (0.0008) [2023-12-27 00:07:48,057][105620] Updated weights for policy 1, policy_version 1202070 (0.0006) [2023-12-27 00:07:48,119][105620] Updated weights for policy 1, policy_version 1202080 (0.0009) [2023-12-27 00:07:48,264][105692] Updated weights for policy 0, policy_version 1200714 (0.0010) [2023-12-27 00:07:48,318][105692] Updated weights for policy 0, policy_version 1200724 (0.0010) [2023-12-27 00:07:48,375][105692] Updated weights for policy 0, policy_version 1200734 (0.0011) [2023-12-27 00:07:48,901][105620] Updated weights for policy 1, policy_version 1202090 (0.0007) [2023-12-27 00:07:48,963][105620] Updated weights for policy 1, policy_version 1202100 (0.0008) [2023-12-27 00:07:49,020][105620] Updated weights for policy 1, policy_version 1202110 (0.0008) [2023-12-27 00:07:49,159][105692] Updated weights for policy 0, policy_version 1200744 (0.0009) [2023-12-27 00:07:49,220][105692] Updated weights for policy 0, policy_version 1200754 (0.0010) [2023-12-27 00:07:49,284][105692] Updated weights for policy 0, policy_version 1200764 (0.0010) [2023-12-27 00:07:49,831][105620] Updated weights for policy 1, policy_version 1202120 (0.0009) [2023-12-27 00:07:49,903][105620] Updated weights for policy 1, policy_version 1202130 (0.0008) [2023-12-27 00:07:49,970][105620] Updated weights for policy 1, policy_version 1202140 (0.0007) [2023-12-27 00:07:50,074][105692] Updated weights for policy 0, policy_version 1200774 (0.0008) [2023-12-27 00:07:50,133][105692] Updated weights for policy 0, policy_version 1200784 (0.0008) [2023-12-27 00:07:50,201][105692] Updated weights for policy 0, policy_version 1200794 (0.0009) [2023-12-27 00:07:50,746][105620] Updated weights for policy 1, policy_version 1202150 (0.0009) [2023-12-27 00:07:50,813][105620] Updated weights for policy 1, policy_version 1202160 (0.0010) [2023-12-27 00:07:50,880][105620] Updated weights for policy 1, policy_version 1202170 (0.0009) [2023-12-27 00:07:50,968][105692] Updated weights for policy 0, policy_version 1200804 (0.0008) [2023-12-27 00:07:51,032][105692] Updated weights for policy 0, policy_version 1200814 (0.0009) [2023-12-27 00:07:51,062][104569] Fps is (10 sec: 18022.4, 60 sec: 18022.4, 300 sec: 18133.5). Total num frames: 615251968. Throughput: 0: 8926.9, 1: 9137.5. Samples: 615245040. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:07:51,063][104569] Avg episode reward: [(0, '9180.193'), (1, '8810.528')] [2023-12-27 00:07:51,099][105692] Updated weights for policy 0, policy_version 1200824 (0.0008) [2023-12-27 00:07:51,617][105620] Updated weights for policy 1, policy_version 1202180 (0.0010) [2023-12-27 00:07:51,693][105620] Updated weights for policy 1, policy_version 1202190 (0.0008) [2023-12-27 00:07:51,764][105620] Updated weights for policy 1, policy_version 1202200 (0.0008) [2023-12-27 00:07:51,889][105692] Updated weights for policy 0, policy_version 1200834 (0.0008) [2023-12-27 00:07:51,950][105692] Updated weights for policy 0, policy_version 1200844 (0.0008) [2023-12-27 00:07:52,010][105692] Updated weights for policy 0, policy_version 1200854 (0.0008) [2023-12-27 00:07:52,066][105692] Updated weights for policy 0, policy_version 1200864 (0.0008) [2023-12-27 00:07:52,431][105620] Updated weights for policy 1, policy_version 1202210 (0.0008) [2023-12-27 00:07:52,496][105620] Updated weights for policy 1, policy_version 1202220 (0.0010) [2023-12-27 00:07:52,554][105620] Updated weights for policy 1, policy_version 1202230 (0.0008) [2023-12-27 00:07:52,617][105620] Updated weights for policy 1, policy_version 1202240 (0.0009) [2023-12-27 00:07:52,859][105692] Updated weights for policy 0, policy_version 1200874 (0.0010) [2023-12-27 00:07:52,922][105692] Updated weights for policy 0, policy_version 1200884 (0.0009) [2023-12-27 00:07:52,983][105692] Updated weights for policy 0, policy_version 1200894 (0.0010) [2023-12-27 00:07:53,311][105620] Updated weights for policy 1, policy_version 1202250 (0.0006) [2023-12-27 00:07:53,372][105620] Updated weights for policy 1, policy_version 1202260 (0.0009) [2023-12-27 00:07:53,435][105620] Updated weights for policy 1, policy_version 1202270 (0.0009) [2023-12-27 00:07:53,828][105692] Updated weights for policy 0, policy_version 1200904 (0.0009) [2023-12-27 00:07:53,890][105692] Updated weights for policy 0, policy_version 1200914 (0.0009) [2023-12-27 00:07:53,949][105692] Updated weights for policy 0, policy_version 1200924 (0.0008) [2023-12-27 00:07:54,140][105620] Updated weights for policy 1, policy_version 1202280 (0.0009) [2023-12-27 00:07:54,189][105620] Updated weights for policy 1, policy_version 1202290 (0.0008) [2023-12-27 00:07:54,244][105620] Updated weights for policy 1, policy_version 1202300 (0.0009) [2023-12-27 00:07:54,654][105692] Updated weights for policy 0, policy_version 1200934 (0.0009) [2023-12-27 00:07:54,707][105692] Updated weights for policy 0, policy_version 1200944 (0.0010) [2023-12-27 00:07:54,763][105692] Updated weights for policy 0, policy_version 1200954 (0.0009) [2023-12-27 00:07:55,040][105620] Updated weights for policy 1, policy_version 1202310 (0.0009) [2023-12-27 00:07:55,104][105620] Updated weights for policy 1, policy_version 1202320 (0.0009) [2023-12-27 00:07:55,157][105620] Updated weights for policy 1, policy_version 1202330 (0.0009) [2023-12-27 00:07:55,496][105692] Updated weights for policy 0, policy_version 1200964 (0.0009) [2023-12-27 00:07:55,563][105692] Updated weights for policy 0, policy_version 1200974 (0.0010) [2023-12-27 00:07:55,629][105692] Updated weights for policy 0, policy_version 1200984 (0.0009) [2023-12-27 00:07:55,976][105620] Updated weights for policy 1, policy_version 1202340 (0.0008) [2023-12-27 00:07:56,045][105620] Updated weights for policy 1, policy_version 1202350 (0.0008) [2023-12-27 00:07:56,062][104569] Fps is (10 sec: 18022.6, 60 sec: 18022.4, 300 sec: 18161.2). Total num frames: 615342080. Throughput: 0: 8926.8, 1: 9222.8. Samples: 615355200. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:07:56,063][104569] Avg episode reward: [(0, '9175.078'), (1, '9082.750')] [2023-12-27 00:07:56,112][105620] Updated weights for policy 1, policy_version 1202360 (0.0008) [2023-12-27 00:07:56,418][105692] Updated weights for policy 0, policy_version 1200994 (0.0009) [2023-12-27 00:07:56,469][105692] Updated weights for policy 0, policy_version 1201004 (0.0009) [2023-12-27 00:07:56,522][105692] Updated weights for policy 0, policy_version 1201014 (0.0008) [2023-12-27 00:07:56,585][105692] Updated weights for policy 0, policy_version 1201024 (0.0010) [2023-12-27 00:07:56,714][105620] Updated weights for policy 1, policy_version 1202370 (0.0006) [2023-12-27 00:07:56,769][105620] Updated weights for policy 1, policy_version 1202380 (0.0009) [2023-12-27 00:07:56,827][105620] Updated weights for policy 1, policy_version 1202390 (0.0008) [2023-12-27 00:07:56,875][105620] Updated weights for policy 1, policy_version 1202400 (0.0009) [2023-12-27 00:07:57,387][105692] Updated weights for policy 0, policy_version 1201034 (0.0009) [2023-12-27 00:07:57,446][105692] Updated weights for policy 0, policy_version 1201044 (0.0009) [2023-12-27 00:07:57,505][105692] Updated weights for policy 0, policy_version 1201054 (0.0009) [2023-12-27 00:07:57,631][105620] Updated weights for policy 1, policy_version 1202410 (0.0009) [2023-12-27 00:07:57,698][105620] Updated weights for policy 1, policy_version 1202420 (0.0009) [2023-12-27 00:07:57,747][105620] Updated weights for policy 1, policy_version 1202430 (0.0008) [2023-12-27 00:07:58,287][105692] Updated weights for policy 0, policy_version 1201064 (0.0007) [2023-12-27 00:07:58,354][105692] Updated weights for policy 0, policy_version 1201074 (0.0008) [2023-12-27 00:07:58,425][105692] Updated weights for policy 0, policy_version 1201084 (0.0009) [2023-12-27 00:07:58,544][105620] Updated weights for policy 1, policy_version 1202440 (0.0008) [2023-12-27 00:07:58,614][105620] Updated weights for policy 1, policy_version 1202450 (0.0011) [2023-12-27 00:07:58,685][105620] Updated weights for policy 1, policy_version 1202460 (0.0011) [2023-12-27 00:07:59,170][105692] Updated weights for policy 0, policy_version 1201094 (0.0008) [2023-12-27 00:07:59,239][105692] Updated weights for policy 0, policy_version 1201104 (0.0008) [2023-12-27 00:07:59,304][105692] Updated weights for policy 0, policy_version 1201114 (0.0008) [2023-12-27 00:07:59,494][105620] Updated weights for policy 1, policy_version 1202470 (0.0011) [2023-12-27 00:07:59,567][105620] Updated weights for policy 1, policy_version 1202480 (0.0011) [2023-12-27 00:07:59,638][105620] Updated weights for policy 1, policy_version 1202490 (0.0009) [2023-12-27 00:08:00,107][105692] Updated weights for policy 0, policy_version 1201124 (0.0009) [2023-12-27 00:08:00,163][105692] Updated weights for policy 0, policy_version 1201134 (0.0007) [2023-12-27 00:08:00,219][105692] Updated weights for policy 0, policy_version 1201144 (0.0009) [2023-12-27 00:08:00,414][105620] Updated weights for policy 1, policy_version 1202500 (0.0010) [2023-12-27 00:08:00,474][105620] Updated weights for policy 1, policy_version 1202510 (0.0011) [2023-12-27 00:08:00,540][105620] Updated weights for policy 1, policy_version 1202520 (0.0011) [2023-12-27 00:08:01,030][105692] Updated weights for policy 0, policy_version 1201154 (0.0008) [2023-12-27 00:08:01,062][104569] Fps is (10 sec: 18022.5, 60 sec: 18022.4, 300 sec: 18133.5). Total num frames: 615432192. Throughput: 0: 8966.8, 1: 9227.9. Samples: 615410236. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:08:01,062][104569] Avg episode reward: [(0, '9175.084'), (1, '9265.704')] [2023-12-27 00:08:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001202528_307888128.pth... [2023-12-27 00:08:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001201472_307617792.pth [2023-12-27 00:08:01,097][105692] Updated weights for policy 0, policy_version 1201164 (0.0009) [2023-12-27 00:08:01,168][105692] Updated weights for policy 0, policy_version 1201174 (0.0009) [2023-12-27 00:08:01,234][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001201184_307552256.pth... [2023-12-27 00:08:01,236][105692] Updated weights for policy 0, policy_version 1201184 (0.0007) [2023-12-27 00:08:01,239][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001200128_307281920.pth [2023-12-27 00:08:01,241][105620] Updated weights for policy 1, policy_version 1202530 (0.0007) [2023-12-27 00:08:01,310][105620] Updated weights for policy 1, policy_version 1202540 (0.0008) [2023-12-27 00:08:01,374][105620] Updated weights for policy 1, policy_version 1202550 (0.0008) [2023-12-27 00:08:01,449][105620] Updated weights for policy 1, policy_version 1202560 (0.0010) [2023-12-27 00:08:01,972][105692] Updated weights for policy 0, policy_version 1201194 (0.0009) [2023-12-27 00:08:02,029][105692] Updated weights for policy 0, policy_version 1201204 (0.0008) [2023-12-27 00:08:02,096][105692] Updated weights for policy 0, policy_version 1201214 (0.0007) [2023-12-27 00:08:02,121][105620] Updated weights for policy 1, policy_version 1202570 (0.0009) [2023-12-27 00:08:02,170][105620] Updated weights for policy 1, policy_version 1202580 (0.0011) [2023-12-27 00:08:02,233][105620] Updated weights for policy 1, policy_version 1202590 (0.0011) [2023-12-27 00:08:02,891][105692] Updated weights for policy 0, policy_version 1201224 (0.0009) [2023-12-27 00:08:02,948][105692] Updated weights for policy 0, policy_version 1201234 (0.0008) [2023-12-27 00:08:03,000][105620] Updated weights for policy 1, policy_version 1202600 (0.0011) [2023-12-27 00:08:03,014][105692] Updated weights for policy 0, policy_version 1201244 (0.0009) [2023-12-27 00:08:03,058][105620] Updated weights for policy 1, policy_version 1202610 (0.0011) [2023-12-27 00:08:03,110][105620] Updated weights for policy 1, policy_version 1202620 (0.0010) [2023-12-27 00:08:03,739][105692] Updated weights for policy 0, policy_version 1201254 (0.0006) [2023-12-27 00:08:03,802][105692] Updated weights for policy 0, policy_version 1201264 (0.0008) [2023-12-27 00:08:03,847][105620] Updated weights for policy 1, policy_version 1202630 (0.0009) [2023-12-27 00:08:03,869][105692] Updated weights for policy 0, policy_version 1201274 (0.0008) [2023-12-27 00:08:03,916][105620] Updated weights for policy 1, policy_version 1202640 (0.0007) [2023-12-27 00:08:03,983][105620] Updated weights for policy 1, policy_version 1202650 (0.0009) [2023-12-27 00:08:04,601][105692] Updated weights for policy 0, policy_version 1201284 (0.0010) [2023-12-27 00:08:04,662][105692] Updated weights for policy 0, policy_version 1201294 (0.0009) [2023-12-27 00:08:04,700][105620] Updated weights for policy 1, policy_version 1202660 (0.0008) [2023-12-27 00:08:04,718][105692] Updated weights for policy 0, policy_version 1201304 (0.0009) [2023-12-27 00:08:04,759][105620] Updated weights for policy 1, policy_version 1202670 (0.0007) [2023-12-27 00:08:04,820][105620] Updated weights for policy 1, policy_version 1202680 (0.0008) [2023-12-27 00:08:05,350][105692] Updated weights for policy 0, policy_version 1201314 (0.0010) [2023-12-27 00:08:05,410][105692] Updated weights for policy 0, policy_version 1201324 (0.0011) [2023-12-27 00:08:05,424][105620] Updated weights for policy 1, policy_version 1202690 (0.0006) [2023-12-27 00:08:05,469][105692] Updated weights for policy 0, policy_version 1201334 (0.0010) [2023-12-27 00:08:05,488][105620] Updated weights for policy 1, policy_version 1202700 (0.0006) [2023-12-27 00:08:05,530][105692] Updated weights for policy 0, policy_version 1201344 (0.0011) [2023-12-27 00:08:05,538][105620] Updated weights for policy 1, policy_version 1202710 (0.0006) [2023-12-27 00:08:05,595][105620] Updated weights for policy 1, policy_version 1202720 (0.0008) [2023-12-27 00:08:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 18158.9, 300 sec: 18133.5). Total num frames: 615530496. Throughput: 0: 8946.1, 1: 9243.0. Samples: 615521060. Policy #0 lag: (min: 2.0, avg: 20.9, max: 34.0) [2023-12-27 00:08:06,062][104569] Avg episode reward: [(0, '9268.548'), (1, '9264.062')] [2023-12-27 00:08:06,198][105692] Updated weights for policy 0, policy_version 1201354 (0.0009) [2023-12-27 00:08:06,264][105692] Updated weights for policy 0, policy_version 1201364 (0.0008) [2023-12-27 00:08:06,325][105620] Updated weights for policy 1, policy_version 1202730 (0.0010) [2023-12-27 00:08:06,332][105692] Updated weights for policy 0, policy_version 1201374 (0.0008) [2023-12-27 00:08:06,390][105620] Updated weights for policy 1, policy_version 1202740 (0.0008) [2023-12-27 00:08:06,452][105620] Updated weights for policy 1, policy_version 1202750 (0.0009) [2023-12-27 00:08:07,105][105692] Updated weights for policy 0, policy_version 1201384 (0.0009) [2023-12-27 00:08:07,162][105692] Updated weights for policy 0, policy_version 1201394 (0.0009) [2023-12-27 00:08:07,215][105620] Updated weights for policy 1, policy_version 1202760 (0.0008) [2023-12-27 00:08:07,221][105692] Updated weights for policy 0, policy_version 1201404 (0.0007) [2023-12-27 00:08:07,271][105620] Updated weights for policy 1, policy_version 1202770 (0.0007) [2023-12-27 00:08:07,338][105620] Updated weights for policy 1, policy_version 1202780 (0.0009) [2023-12-27 00:08:07,951][105692] Updated weights for policy 0, policy_version 1201414 (0.0008) [2023-12-27 00:08:08,004][105692] Updated weights for policy 0, policy_version 1201424 (0.0009) [2023-12-27 00:08:08,066][105692] Updated weights for policy 0, policy_version 1201434 (0.0008) [2023-12-27 00:08:08,069][105620] Updated weights for policy 1, policy_version 1202790 (0.0008) [2023-12-27 00:08:08,128][105620] Updated weights for policy 1, policy_version 1202800 (0.0006) [2023-12-27 00:08:08,195][105620] Updated weights for policy 1, policy_version 1202810 (0.0007) [2023-12-27 00:08:08,771][105620] Updated weights for policy 1, policy_version 1202820 (0.0005) [2023-12-27 00:08:08,823][105692] Updated weights for policy 0, policy_version 1201444 (0.0008) [2023-12-27 00:08:08,832][105620] Updated weights for policy 1, policy_version 1202830 (0.0006) [2023-12-27 00:08:08,883][105620] Updated weights for policy 1, policy_version 1202840 (0.0006) [2023-12-27 00:08:08,888][105692] Updated weights for policy 0, policy_version 1201454 (0.0009) [2023-12-27 00:08:08,948][105692] Updated weights for policy 0, policy_version 1201464 (0.0008) [2023-12-27 00:08:09,617][105620] Updated weights for policy 1, policy_version 1202850 (0.0007) [2023-12-27 00:08:09,682][105620] Updated weights for policy 1, policy_version 1202860 (0.0008) [2023-12-27 00:08:09,685][105692] Updated weights for policy 0, policy_version 1201474 (0.0008) [2023-12-27 00:08:09,748][105620] Updated weights for policy 1, policy_version 1202870 (0.0008) [2023-12-27 00:08:09,754][105692] Updated weights for policy 0, policy_version 1201484 (0.0009) [2023-12-27 00:08:09,815][105620] Updated weights for policy 1, policy_version 1202880 (0.0006) [2023-12-27 00:08:09,821][105692] Updated weights for policy 0, policy_version 1201494 (0.0009) [2023-12-27 00:08:09,887][105692] Updated weights for policy 0, policy_version 1201504 (0.0009) [2023-12-27 00:08:10,568][105620] Updated weights for policy 1, policy_version 1202890 (0.0007) [2023-12-27 00:08:10,629][105620] Updated weights for policy 1, policy_version 1202900 (0.0009) [2023-12-27 00:08:10,676][105692] Updated weights for policy 0, policy_version 1201514 (0.0007) [2023-12-27 00:08:10,698][105620] Updated weights for policy 1, policy_version 1202910 (0.0009) [2023-12-27 00:08:10,740][105692] Updated weights for policy 0, policy_version 1201524 (0.0007) [2023-12-27 00:08:10,801][105692] Updated weights for policy 0, policy_version 1201534 (0.0009) [2023-12-27 00:08:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 18295.5, 300 sec: 18133.5). Total num frames: 615628800. Throughput: 0: 9034.8, 1: 9371.7. Samples: 615636420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:08:11,063][104569] Avg episode reward: [(0, '9268.255'), (1, '9172.738')] [2023-12-27 00:08:11,499][105620] Updated weights for policy 1, policy_version 1202920 (0.0009) [2023-12-27 00:08:11,566][105620] Updated weights for policy 1, policy_version 1202930 (0.0007) [2023-12-27 00:08:11,588][105692] Updated weights for policy 0, policy_version 1201544 (0.0008) [2023-12-27 00:08:11,650][105620] Updated weights for policy 1, policy_version 1202940 (0.0007) [2023-12-27 00:08:11,660][105692] Updated weights for policy 0, policy_version 1201554 (0.0008) [2023-12-27 00:08:11,730][105692] Updated weights for policy 0, policy_version 1201564 (0.0010) [2023-12-27 00:08:12,402][105620] Updated weights for policy 1, policy_version 1202950 (0.0007) [2023-12-27 00:08:12,463][105620] Updated weights for policy 1, policy_version 1202960 (0.0008) [2023-12-27 00:08:12,490][105692] Updated weights for policy 0, policy_version 1201574 (0.0009) [2023-12-27 00:08:12,527][105620] Updated weights for policy 1, policy_version 1202970 (0.0007) [2023-12-27 00:08:12,552][105692] Updated weights for policy 0, policy_version 1201584 (0.0008) [2023-12-27 00:08:12,619][105692] Updated weights for policy 0, policy_version 1201594 (0.0010) [2023-12-27 00:08:13,284][105620] Updated weights for policy 1, policy_version 1202980 (0.0009) [2023-12-27 00:08:13,339][105620] Updated weights for policy 1, policy_version 1202990 (0.0010) [2023-12-27 00:08:13,387][105692] Updated weights for policy 0, policy_version 1201604 (0.0008) [2023-12-27 00:08:13,392][105620] Updated weights for policy 1, policy_version 1203000 (0.0010) [2023-12-27 00:08:13,439][105692] Updated weights for policy 0, policy_version 1201614 (0.0006) [2023-12-27 00:08:13,500][105692] Updated weights for policy 0, policy_version 1201624 (0.0009) [2023-12-27 00:08:14,040][105620] Updated weights for policy 1, policy_version 1203010 (0.0010) [2023-12-27 00:08:14,110][105620] Updated weights for policy 1, policy_version 1203020 (0.0011) [2023-12-27 00:08:14,177][105620] Updated weights for policy 1, policy_version 1203030 (0.0011) [2023-12-27 00:08:14,183][105692] Updated weights for policy 0, policy_version 1201634 (0.0009) [2023-12-27 00:08:14,234][105620] Updated weights for policy 1, policy_version 1203040 (0.0011) [2023-12-27 00:08:14,243][105692] Updated weights for policy 0, policy_version 1201644 (0.0011) [2023-12-27 00:08:14,303][105692] Updated weights for policy 0, policy_version 1201654 (0.0011) [2023-12-27 00:08:14,363][105692] Updated weights for policy 0, policy_version 1201664 (0.0010) [2023-12-27 00:08:15,013][105620] Updated weights for policy 1, policy_version 1203050 (0.0010) [2023-12-27 00:08:15,076][105620] Updated weights for policy 1, policy_version 1203060 (0.0007) [2023-12-27 00:08:15,115][105692] Updated weights for policy 0, policy_version 1201674 (0.0008) [2023-12-27 00:08:15,136][105620] Updated weights for policy 1, policy_version 1203070 (0.0007) [2023-12-27 00:08:15,174][105692] Updated weights for policy 0, policy_version 1201684 (0.0008) [2023-12-27 00:08:15,236][105692] Updated weights for policy 0, policy_version 1201694 (0.0009) [2023-12-27 00:08:15,847][105620] Updated weights for policy 1, policy_version 1203080 (0.0010) [2023-12-27 00:08:15,911][105620] Updated weights for policy 1, policy_version 1203090 (0.0011) [2023-12-27 00:08:15,958][105692] Updated weights for policy 0, policy_version 1201704 (0.0011) [2023-12-27 00:08:15,967][105620] Updated weights for policy 1, policy_version 1203100 (0.0010) [2023-12-27 00:08:16,011][105692] Updated weights for policy 0, policy_version 1201714 (0.0010) [2023-12-27 00:08:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 18432.0, 300 sec: 18133.5). Total num frames: 615718912. Throughput: 0: 8977.4, 1: 9358.1. Samples: 615690616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:08:16,063][104569] Avg episode reward: [(0, '9266.983'), (1, '9082.612')] [2023-12-27 00:08:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001203104_308035584.pth... [2023-12-27 00:08:16,071][105692] Updated weights for policy 0, policy_version 1201724 (0.0010) [2023-12-27 00:08:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001201984_307748864.pth [2023-12-27 00:08:16,096][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001201728_307691520.pth... [2023-12-27 00:08:16,101][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001200640_307412992.pth [2023-12-27 00:08:16,683][105620] Updated weights for policy 1, policy_version 1203110 (0.0009) [2023-12-27 00:08:16,744][105620] Updated weights for policy 1, policy_version 1203120 (0.0005) [2023-12-27 00:08:16,784][105692] Updated weights for policy 0, policy_version 1201734 (0.0009) [2023-12-27 00:08:16,802][105620] Updated weights for policy 1, policy_version 1203130 (0.0005) [2023-12-27 00:08:16,834][105692] Updated weights for policy 0, policy_version 1201744 (0.0011) [2023-12-27 00:08:16,887][105692] Updated weights for policy 0, policy_version 1201754 (0.0011) [2023-12-27 00:08:17,421][105620] Updated weights for policy 1, policy_version 1203140 (0.0007) [2023-12-27 00:08:17,477][105620] Updated weights for policy 1, policy_version 1203150 (0.0010) [2023-12-27 00:08:17,530][105692] Updated weights for policy 0, policy_version 1201764 (0.0007) [2023-12-27 00:08:17,541][105620] Updated weights for policy 1, policy_version 1203160 (0.0010) [2023-12-27 00:08:17,590][105692] Updated weights for policy 0, policy_version 1201774 (0.0007) [2023-12-27 00:08:17,645][105692] Updated weights for policy 0, policy_version 1201784 (0.0009) [2023-12-27 00:08:18,216][105620] Updated weights for policy 1, policy_version 1203170 (0.0007) [2023-12-27 00:08:18,268][105620] Updated weights for policy 1, policy_version 1203180 (0.0009) [2023-12-27 00:08:18,336][105620] Updated weights for policy 1, policy_version 1203190 (0.0008) [2023-12-27 00:08:18,362][105692] Updated weights for policy 0, policy_version 1201794 (0.0008) [2023-12-27 00:08:18,395][105620] Updated weights for policy 1, policy_version 1203200 (0.0009) [2023-12-27 00:08:18,420][105692] Updated weights for policy 0, policy_version 1201804 (0.0011) [2023-12-27 00:08:18,492][105692] Updated weights for policy 0, policy_version 1201814 (0.0009) [2023-12-27 00:08:18,556][105692] Updated weights for policy 0, policy_version 1201824 (0.0011) [2023-12-27 00:08:19,181][105620] Updated weights for policy 1, policy_version 1203210 (0.0009) [2023-12-27 00:08:19,249][105620] Updated weights for policy 1, policy_version 1203220 (0.0008) [2023-12-27 00:08:19,315][105620] Updated weights for policy 1, policy_version 1203230 (0.0008) [2023-12-27 00:08:19,319][105692] Updated weights for policy 0, policy_version 1201834 (0.0009) [2023-12-27 00:08:19,385][105692] Updated weights for policy 0, policy_version 1201844 (0.0011) [2023-12-27 00:08:19,445][105692] Updated weights for policy 0, policy_version 1201854 (0.0011) [2023-12-27 00:08:20,127][105620] Updated weights for policy 1, policy_version 1203240 (0.0008) [2023-12-27 00:08:20,191][105620] Updated weights for policy 1, policy_version 1203250 (0.0007) [2023-12-27 00:08:20,223][105692] Updated weights for policy 0, policy_version 1201864 (0.0011) [2023-12-27 00:08:20,254][105620] Updated weights for policy 1, policy_version 1203260 (0.0006) [2023-12-27 00:08:20,284][105692] Updated weights for policy 0, policy_version 1201874 (0.0011) [2023-12-27 00:08:20,353][105692] Updated weights for policy 0, policy_version 1201884 (0.0011) [2023-12-27 00:08:21,002][105620] Updated weights for policy 1, policy_version 1203270 (0.0007) [2023-12-27 00:08:21,062][104569] Fps is (10 sec: 18022.4, 60 sec: 18295.5, 300 sec: 18133.5). Total num frames: 615809024. Throughput: 0: 9094.2, 1: 9412.3. Samples: 615806436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:08:21,063][104569] Avg episode reward: [(0, '9175.655'), (1, '9082.071')] [2023-12-27 00:08:21,070][105620] Updated weights for policy 1, policy_version 1203280 (0.0008) [2023-12-27 00:08:21,129][105692] Updated weights for policy 0, policy_version 1201894 (0.0009) [2023-12-27 00:08:21,139][105620] Updated weights for policy 1, policy_version 1203290 (0.0009) [2023-12-27 00:08:21,193][105692] Updated weights for policy 0, policy_version 1201904 (0.0008) [2023-12-27 00:08:21,256][105692] Updated weights for policy 0, policy_version 1201914 (0.0008) [2023-12-27 00:08:21,940][105620] Updated weights for policy 1, policy_version 1203300 (0.0008) [2023-12-27 00:08:22,008][105620] Updated weights for policy 1, policy_version 1203310 (0.0009) [2023-12-27 00:08:22,034][105692] Updated weights for policy 0, policy_version 1201924 (0.0009) [2023-12-27 00:08:22,070][105620] Updated weights for policy 1, policy_version 1203320 (0.0007) [2023-12-27 00:08:22,089][105692] Updated weights for policy 0, policy_version 1201934 (0.0011) [2023-12-27 00:08:22,144][105692] Updated weights for policy 0, policy_version 1201944 (0.0011) [2023-12-27 00:08:22,892][105620] Updated weights for policy 1, policy_version 1203330 (0.0007) [2023-12-27 00:08:22,956][105620] Updated weights for policy 1, policy_version 1203340 (0.0009) [2023-12-27 00:08:22,964][105692] Updated weights for policy 0, policy_version 1201954 (0.0010) [2023-12-27 00:08:23,023][105620] Updated weights for policy 1, policy_version 1203350 (0.0006) [2023-12-27 00:08:23,025][105692] Updated weights for policy 0, policy_version 1201964 (0.0011) [2023-12-27 00:08:23,075][105620] Updated weights for policy 1, policy_version 1203360 (0.0007) [2023-12-27 00:08:23,089][105692] Updated weights for policy 0, policy_version 1201974 (0.0008) [2023-12-27 00:08:23,151][105692] Updated weights for policy 0, policy_version 1201984 (0.0009) [2023-12-27 00:08:23,760][105620] Updated weights for policy 1, policy_version 1203370 (0.0006) [2023-12-27 00:08:23,832][105620] Updated weights for policy 1, policy_version 1203380 (0.0006) [2023-12-27 00:08:23,901][105620] Updated weights for policy 1, policy_version 1203390 (0.0007) [2023-12-27 00:08:23,963][105692] Updated weights for policy 0, policy_version 1201994 (0.0008) [2023-12-27 00:08:24,018][105692] Updated weights for policy 0, policy_version 1202004 (0.0009) [2023-12-27 00:08:24,070][105692] Updated weights for policy 0, policy_version 1202014 (0.0009) [2023-12-27 00:08:24,577][105620] Updated weights for policy 1, policy_version 1203400 (0.0010) [2023-12-27 00:08:24,634][105620] Updated weights for policy 1, policy_version 1203410 (0.0010) [2023-12-27 00:08:24,692][105620] Updated weights for policy 1, policy_version 1203420 (0.0011) [2023-12-27 00:08:24,794][105692] Updated weights for policy 0, policy_version 1202024 (0.0011) [2023-12-27 00:08:24,855][105692] Updated weights for policy 0, policy_version 1202034 (0.0010) [2023-12-27 00:08:24,912][105692] Updated weights for policy 0, policy_version 1202044 (0.0005) [2023-12-27 00:08:25,449][105620] Updated weights for policy 1, policy_version 1203430 (0.0010) [2023-12-27 00:08:25,512][105620] Updated weights for policy 1, policy_version 1203440 (0.0009) [2023-12-27 00:08:25,570][105620] Updated weights for policy 1, policy_version 1203450 (0.0009) [2023-12-27 00:08:25,654][105692] Updated weights for policy 0, policy_version 1202054 (0.0009) [2023-12-27 00:08:25,702][105692] Updated weights for policy 0, policy_version 1202064 (0.0009) [2023-12-27 00:08:25,761][105692] Updated weights for policy 0, policy_version 1202074 (0.0009) [2023-12-27 00:08:26,062][104569] Fps is (10 sec: 18841.1, 60 sec: 18568.4, 300 sec: 18161.2). Total num frames: 615907328. Throughput: 0: 9132.3, 1: 9415.9. Samples: 615915308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:08:26,063][104569] Avg episode reward: [(0, '9175.774'), (1, '9174.347')] [2023-12-27 00:08:26,286][105620] Updated weights for policy 1, policy_version 1203460 (0.0008) [2023-12-27 00:08:26,347][105620] Updated weights for policy 1, policy_version 1203470 (0.0011) [2023-12-27 00:08:26,407][105620] Updated weights for policy 1, policy_version 1203480 (0.0011) [2023-12-27 00:08:26,634][105692] Updated weights for policy 0, policy_version 1202084 (0.0009) [2023-12-27 00:08:26,695][105692] Updated weights for policy 0, policy_version 1202094 (0.0008) [2023-12-27 00:08:26,746][105692] Updated weights for policy 0, policy_version 1202104 (0.0008) [2023-12-27 00:08:27,136][105620] Updated weights for policy 1, policy_version 1203490 (0.0010) [2023-12-27 00:08:27,200][105620] Updated weights for policy 1, policy_version 1203500 (0.0008) [2023-12-27 00:08:27,262][105620] Updated weights for policy 1, policy_version 1203510 (0.0008) [2023-12-27 00:08:27,326][105620] Updated weights for policy 1, policy_version 1203520 (0.0007) [2023-12-27 00:08:27,611][105692] Updated weights for policy 0, policy_version 1202115 (0.0009) [2023-12-27 00:08:27,662][105692] Updated weights for policy 0, policy_version 1202125 (0.0008) [2023-12-27 00:08:27,721][105692] Updated weights for policy 0, policy_version 1202135 (0.0010) [2023-12-27 00:08:27,916][105620] Updated weights for policy 1, policy_version 1203530 (0.0009) [2023-12-27 00:08:27,971][105620] Updated weights for policy 1, policy_version 1203540 (0.0009) [2023-12-27 00:08:28,028][105620] Updated weights for policy 1, policy_version 1203550 (0.0009) [2023-12-27 00:08:28,450][105692] Updated weights for policy 0, policy_version 1202145 (0.0009) [2023-12-27 00:08:28,510][105692] Updated weights for policy 0, policy_version 1202155 (0.0011) [2023-12-27 00:08:28,578][105692] Updated weights for policy 0, policy_version 1202165 (0.0011) [2023-12-27 00:08:28,642][105692] Updated weights for policy 0, policy_version 1202175 (0.0011) [2023-12-27 00:08:28,829][105620] Updated weights for policy 1, policy_version 1203560 (0.0008) [2023-12-27 00:08:28,883][105620] Updated weights for policy 1, policy_version 1203570 (0.0006) [2023-12-27 00:08:28,948][105620] Updated weights for policy 1, policy_version 1203580 (0.0007) [2023-12-27 00:08:29,414][105692] Updated weights for policy 0, policy_version 1202185 (0.0009) [2023-12-27 00:08:29,483][105692] Updated weights for policy 0, policy_version 1202195 (0.0008) [2023-12-27 00:08:29,547][105692] Updated weights for policy 0, policy_version 1202205 (0.0008) [2023-12-27 00:08:29,671][105620] Updated weights for policy 1, policy_version 1203590 (0.0008) [2023-12-27 00:08:29,738][105620] Updated weights for policy 1, policy_version 1203600 (0.0011) [2023-12-27 00:08:29,798][105620] Updated weights for policy 1, policy_version 1203610 (0.0010) [2023-12-27 00:08:30,312][105692] Updated weights for policy 0, policy_version 1202215 (0.0008) [2023-12-27 00:08:30,377][105692] Updated weights for policy 0, policy_version 1202225 (0.0009) [2023-12-27 00:08:30,448][105692] Updated weights for policy 0, policy_version 1202235 (0.0009) [2023-12-27 00:08:30,570][105620] Updated weights for policy 1, policy_version 1203620 (0.0008) [2023-12-27 00:08:30,639][105620] Updated weights for policy 1, policy_version 1203630 (0.0005) [2023-12-27 00:08:30,695][105620] Updated weights for policy 1, policy_version 1203640 (0.0009) [2023-12-27 00:08:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18432.0, 300 sec: 18161.2). Total num frames: 615997440. Throughput: 0: 9119.2, 1: 9476.5. Samples: 615970976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:08:31,062][104569] Avg episode reward: [(0, '9356.511'), (1, '9266.342')] [2023-12-27 00:08:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001202240_307822592.pth... [2023-12-27 00:08:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001203648_308174848.pth... [2023-12-27 00:08:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001201184_307552256.pth [2023-12-27 00:08:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001202528_307888128.pth [2023-12-27 00:08:31,202][105692] Updated weights for policy 0, policy_version 1202245 (0.0008) [2023-12-27 00:08:31,264][105692] Updated weights for policy 0, policy_version 1202255 (0.0009) [2023-12-27 00:08:31,326][105692] Updated weights for policy 0, policy_version 1202265 (0.0010) [2023-12-27 00:08:31,379][105620] Updated weights for policy 1, policy_version 1203650 (0.0010) [2023-12-27 00:08:31,435][105620] Updated weights for policy 1, policy_version 1203660 (0.0006) [2023-12-27 00:08:31,497][105620] Updated weights for policy 1, policy_version 1203670 (0.0010) [2023-12-27 00:08:31,556][105620] Updated weights for policy 1, policy_version 1203680 (0.0008) [2023-12-27 00:08:32,010][105692] Updated weights for policy 0, policy_version 1202275 (0.0009) [2023-12-27 00:08:32,069][105692] Updated weights for policy 0, policy_version 1202285 (0.0006) [2023-12-27 00:08:32,140][105692] Updated weights for policy 0, policy_version 1202295 (0.0008) [2023-12-27 00:08:32,337][105620] Updated weights for policy 1, policy_version 1203690 (0.0008) [2023-12-27 00:08:32,400][105620] Updated weights for policy 1, policy_version 1203700 (0.0009) [2023-12-27 00:08:32,463][105620] Updated weights for policy 1, policy_version 1203710 (0.0009) [2023-12-27 00:08:32,901][105692] Updated weights for policy 0, policy_version 1202305 (0.0010) [2023-12-27 00:08:32,962][105692] Updated weights for policy 0, policy_version 1202315 (0.0006) [2023-12-27 00:08:33,013][105692] Updated weights for policy 0, policy_version 1202325 (0.0005) [2023-12-27 00:08:33,069][105692] Updated weights for policy 0, policy_version 1202335 (0.0005) [2023-12-27 00:08:33,213][105620] Updated weights for policy 1, policy_version 1203720 (0.0010) [2023-12-27 00:08:33,273][105620] Updated weights for policy 1, policy_version 1203730 (0.0010) [2023-12-27 00:08:33,332][105620] Updated weights for policy 1, policy_version 1203740 (0.0010) [2023-12-27 00:08:33,680][105692] Updated weights for policy 0, policy_version 1202345 (0.0010) [2023-12-27 00:08:33,733][105692] Updated weights for policy 0, policy_version 1202355 (0.0010) [2023-12-27 00:08:33,795][105692] Updated weights for policy 0, policy_version 1202365 (0.0010) [2023-12-27 00:08:34,097][105620] Updated weights for policy 1, policy_version 1203750 (0.0009) [2023-12-27 00:08:34,163][105620] Updated weights for policy 1, policy_version 1203760 (0.0009) [2023-12-27 00:08:34,235][105620] Updated weights for policy 1, policy_version 1203770 (0.0009) [2023-12-27 00:08:34,539][105692] Updated weights for policy 0, policy_version 1202375 (0.0008) [2023-12-27 00:08:34,613][105692] Updated weights for policy 0, policy_version 1202385 (0.0008) [2023-12-27 00:08:34,686][105692] Updated weights for policy 0, policy_version 1202395 (0.0008) [2023-12-27 00:08:34,935][105620] Updated weights for policy 1, policy_version 1203780 (0.0010) [2023-12-27 00:08:34,988][105620] Updated weights for policy 1, policy_version 1203790 (0.0011) [2023-12-27 00:08:35,039][105620] Updated weights for policy 1, policy_version 1203800 (0.0011) [2023-12-27 00:08:35,287][105692] Updated weights for policy 0, policy_version 1202405 (0.0009) [2023-12-27 00:08:35,342][105692] Updated weights for policy 0, policy_version 1202415 (0.0009) [2023-12-27 00:08:35,399][105692] Updated weights for policy 0, policy_version 1202425 (0.0009) [2023-12-27 00:08:35,724][105620] Updated weights for policy 1, policy_version 1203810 (0.0010) [2023-12-27 00:08:35,774][105620] Updated weights for policy 1, policy_version 1203820 (0.0011) [2023-12-27 00:08:35,845][105620] Updated weights for policy 1, policy_version 1203830 (0.0011) [2023-12-27 00:08:35,909][105620] Updated weights for policy 1, policy_version 1203840 (0.0011) [2023-12-27 00:08:36,062][104569] Fps is (10 sec: 18842.3, 60 sec: 18568.5, 300 sec: 18189.0). Total num frames: 616095744. Throughput: 0: 9189.2, 1: 9449.2. Samples: 616083772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:08:36,062][104569] Avg episode reward: [(0, '9356.862'), (1, '9082.925')] [2023-12-27 00:08:36,263][105692] Updated weights for policy 0, policy_version 1202435 (0.0010) [2023-12-27 00:08:36,321][105692] Updated weights for policy 0, policy_version 1202445 (0.0008) [2023-12-27 00:08:36,387][105692] Updated weights for policy 0, policy_version 1202455 (0.0009) [2023-12-27 00:08:36,660][105620] Updated weights for policy 1, policy_version 1203850 (0.0011) [2023-12-27 00:08:36,728][105620] Updated weights for policy 1, policy_version 1203860 (0.0011) [2023-12-27 00:08:36,793][105620] Updated weights for policy 1, policy_version 1203870 (0.0011) [2023-12-27 00:08:37,204][105692] Updated weights for policy 0, policy_version 1202465 (0.0009) [2023-12-27 00:08:37,259][105692] Updated weights for policy 0, policy_version 1202475 (0.0009) [2023-12-27 00:08:37,311][105692] Updated weights for policy 0, policy_version 1202485 (0.0008) [2023-12-27 00:08:37,362][105692] Updated weights for policy 0, policy_version 1202495 (0.0009) [2023-12-27 00:08:37,532][105620] Updated weights for policy 1, policy_version 1203880 (0.0010) [2023-12-27 00:08:37,592][105620] Updated weights for policy 1, policy_version 1203890 (0.0009) [2023-12-27 00:08:37,651][105620] Updated weights for policy 1, policy_version 1203900 (0.0009) [2023-12-27 00:08:38,186][105692] Updated weights for policy 0, policy_version 1202505 (0.0009) [2023-12-27 00:08:38,250][105692] Updated weights for policy 0, policy_version 1202515 (0.0010) [2023-12-27 00:08:38,307][105692] Updated weights for policy 0, policy_version 1202525 (0.0006) [2023-12-27 00:08:38,391][105620] Updated weights for policy 1, policy_version 1203910 (0.0006) [2023-12-27 00:08:38,458][105620] Updated weights for policy 1, policy_version 1203920 (0.0008) [2023-12-27 00:08:38,528][105620] Updated weights for policy 1, policy_version 1203930 (0.0008) [2023-12-27 00:08:38,988][105692] Updated weights for policy 0, policy_version 1202535 (0.0007) [2023-12-27 00:08:39,045][105692] Updated weights for policy 0, policy_version 1202545 (0.0009) [2023-12-27 00:08:39,101][105692] Updated weights for policy 0, policy_version 1202555 (0.0009) [2023-12-27 00:08:39,338][105620] Updated weights for policy 1, policy_version 1203940 (0.0009) [2023-12-27 00:08:39,401][105620] Updated weights for policy 1, policy_version 1203950 (0.0009) [2023-12-27 00:08:39,465][105620] Updated weights for policy 1, policy_version 1203960 (0.0009) [2023-12-27 00:08:39,864][105692] Updated weights for policy 0, policy_version 1202565 (0.0009) [2023-12-27 00:08:39,929][105692] Updated weights for policy 0, policy_version 1202575 (0.0008) [2023-12-27 00:08:39,997][105692] Updated weights for policy 0, policy_version 1202585 (0.0010) [2023-12-27 00:08:40,230][105620] Updated weights for policy 1, policy_version 1203970 (0.0009) [2023-12-27 00:08:40,277][105620] Updated weights for policy 1, policy_version 1203980 (0.0008) [2023-12-27 00:08:40,329][105620] Updated weights for policy 1, policy_version 1203990 (0.0007) [2023-12-27 00:08:40,377][105620] Updated weights for policy 1, policy_version 1204000 (0.0008) [2023-12-27 00:08:40,822][105692] Updated weights for policy 0, policy_version 1202595 (0.0009) [2023-12-27 00:08:40,880][105692] Updated weights for policy 0, policy_version 1202605 (0.0007) [2023-12-27 00:08:40,944][105692] Updated weights for policy 0, policy_version 1202615 (0.0007) [2023-12-27 00:08:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 18568.6, 300 sec: 18189.0). Total num frames: 616185856. Throughput: 0: 9207.6, 1: 9446.7. Samples: 616194640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:08:41,062][104569] Avg episode reward: [(0, '9356.794'), (1, '9081.240')] [2023-12-27 00:08:41,120][105620] Updated weights for policy 1, policy_version 1204010 (0.0008) [2023-12-27 00:08:41,188][105620] Updated weights for policy 1, policy_version 1204020 (0.0007) [2023-12-27 00:08:41,253][105620] Updated weights for policy 1, policy_version 1204030 (0.0006) [2023-12-27 00:08:41,778][105692] Updated weights for policy 0, policy_version 1202625 (0.0007) [2023-12-27 00:08:41,833][105692] Updated weights for policy 0, policy_version 1202635 (0.0008) [2023-12-27 00:08:41,903][105692] Updated weights for policy 0, policy_version 1202645 (0.0006) [2023-12-27 00:08:41,958][105692] Updated weights for policy 0, policy_version 1202655 (0.0006) [2023-12-27 00:08:41,986][105620] Updated weights for policy 1, policy_version 1204040 (0.0008) [2023-12-27 00:08:42,046][105620] Updated weights for policy 1, policy_version 1204050 (0.0008) [2023-12-27 00:08:42,113][105620] Updated weights for policy 1, policy_version 1204060 (0.0008) [2023-12-27 00:08:42,671][105692] Updated weights for policy 0, policy_version 1202665 (0.0010) [2023-12-27 00:08:42,722][105692] Updated weights for policy 0, policy_version 1202675 (0.0009) [2023-12-27 00:08:42,782][105692] Updated weights for policy 0, policy_version 1202685 (0.0009) [2023-12-27 00:08:42,819][105620] Updated weights for policy 1, policy_version 1204070 (0.0007) [2023-12-27 00:08:42,881][105620] Updated weights for policy 1, policy_version 1204080 (0.0006) [2023-12-27 00:08:42,937][105620] Updated weights for policy 1, policy_version 1204090 (0.0006) [2023-12-27 00:08:43,445][105692] Updated weights for policy 0, policy_version 1202695 (0.0008) [2023-12-27 00:08:43,501][105692] Updated weights for policy 0, policy_version 1202705 (0.0008) [2023-12-27 00:08:43,558][105692] Updated weights for policy 0, policy_version 1202715 (0.0008) [2023-12-27 00:08:43,597][105620] Updated weights for policy 1, policy_version 1204100 (0.0008) [2023-12-27 00:08:43,656][105620] Updated weights for policy 1, policy_version 1204110 (0.0011) [2023-12-27 00:08:43,723][105620] Updated weights for policy 1, policy_version 1204120 (0.0010) [2023-12-27 00:08:44,313][105692] Updated weights for policy 0, policy_version 1202725 (0.0008) [2023-12-27 00:08:44,380][105692] Updated weights for policy 0, policy_version 1202735 (0.0008) [2023-12-27 00:08:44,432][105692] Updated weights for policy 0, policy_version 1202745 (0.0008) [2023-12-27 00:08:44,449][105620] Updated weights for policy 1, policy_version 1204130 (0.0011) [2023-12-27 00:08:44,500][105620] Updated weights for policy 1, policy_version 1204140 (0.0010) [2023-12-27 00:08:44,548][105620] Updated weights for policy 1, policy_version 1204150 (0.0010) [2023-12-27 00:08:44,593][105620] Updated weights for policy 1, policy_version 1204160 (0.0006) [2023-12-27 00:08:45,205][105692] Updated weights for policy 0, policy_version 1202755 (0.0009) [2023-12-27 00:08:45,267][105692] Updated weights for policy 0, policy_version 1202765 (0.0006) [2023-12-27 00:08:45,298][105620] Updated weights for policy 1, policy_version 1204170 (0.0011) [2023-12-27 00:08:45,319][105692] Updated weights for policy 0, policy_version 1202775 (0.0006) [2023-12-27 00:08:45,362][105620] Updated weights for policy 1, policy_version 1204180 (0.0011) [2023-12-27 00:08:45,425][105620] Updated weights for policy 1, policy_version 1204190 (0.0010) [2023-12-27 00:08:46,029][105692] Updated weights for policy 0, policy_version 1202785 (0.0006) [2023-12-27 00:08:46,062][104569] Fps is (10 sec: 18021.9, 60 sec: 18568.5, 300 sec: 18161.2). Total num frames: 616275968. Throughput: 0: 9238.6, 1: 9463.2. Samples: 616251824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:08:46,063][104569] Avg episode reward: [(0, '9356.596'), (1, '9169.545')] [2023-12-27 00:08:46,083][105692] Updated weights for policy 0, policy_version 1202795 (0.0007) [2023-12-27 00:08:46,107][105620] Updated weights for policy 1, policy_version 1204200 (0.0010) [2023-12-27 00:08:46,141][105692] Updated weights for policy 0, policy_version 1202805 (0.0005) [2023-12-27 00:08:46,159][105620] Updated weights for policy 1, policy_version 1204210 (0.0010) [2023-12-27 00:08:46,201][105692] Updated weights for policy 0, policy_version 1202815 (0.0006) [2023-12-27 00:08:46,204][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001202816_307970048.pth... [2023-12-27 00:08:46,207][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001201728_307691520.pth [2023-12-27 00:08:46,211][105620] Updated weights for policy 1, policy_version 1204220 (0.0010) [2023-12-27 00:08:46,228][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001204224_308322304.pth... [2023-12-27 00:08:46,233][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001203104_308035584.pth [2023-12-27 00:08:46,916][105692] Updated weights for policy 0, policy_version 1202825 (0.0008) [2023-12-27 00:08:46,959][105620] Updated weights for policy 1, policy_version 1204230 (0.0010) [2023-12-27 00:08:46,977][105692] Updated weights for policy 0, policy_version 1202835 (0.0008) [2023-12-27 00:08:47,015][105620] Updated weights for policy 1, policy_version 1204240 (0.0010) [2023-12-27 00:08:47,028][105692] Updated weights for policy 0, policy_version 1202845 (0.0005) [2023-12-27 00:08:47,066][105620] Updated weights for policy 1, policy_version 1204250 (0.0011) [2023-12-27 00:08:47,731][105692] Updated weights for policy 0, policy_version 1202855 (0.0008) [2023-12-27 00:08:47,800][105692] Updated weights for policy 0, policy_version 1202865 (0.0008) [2023-12-27 00:08:47,813][105620] Updated weights for policy 1, policy_version 1204260 (0.0010) [2023-12-27 00:08:47,857][105692] Updated weights for policy 0, policy_version 1202875 (0.0005) [2023-12-27 00:08:47,874][105620] Updated weights for policy 1, policy_version 1204270 (0.0007) [2023-12-27 00:08:47,944][105620] Updated weights for policy 1, policy_version 1204280 (0.0005) [2023-12-27 00:08:48,561][105620] Updated weights for policy 1, policy_version 1204290 (0.0006) [2023-12-27 00:08:48,607][105692] Updated weights for policy 0, policy_version 1202885 (0.0007) [2023-12-27 00:08:48,626][105620] Updated weights for policy 1, policy_version 1204300 (0.0005) [2023-12-27 00:08:48,672][105692] Updated weights for policy 0, policy_version 1202895 (0.0009) [2023-12-27 00:08:48,681][105620] Updated weights for policy 1, policy_version 1204310 (0.0009) [2023-12-27 00:08:48,729][105692] Updated weights for policy 0, policy_version 1202905 (0.0007) [2023-12-27 00:08:48,730][105620] Updated weights for policy 1, policy_version 1204320 (0.0008) [2023-12-27 00:08:49,417][105620] Updated weights for policy 1, policy_version 1204330 (0.0009) [2023-12-27 00:08:49,467][105692] Updated weights for policy 0, policy_version 1202915 (0.0006) [2023-12-27 00:08:49,477][105620] Updated weights for policy 1, policy_version 1204340 (0.0009) [2023-12-27 00:08:49,529][105692] Updated weights for policy 0, policy_version 1202925 (0.0008) [2023-12-27 00:08:49,531][105620] Updated weights for policy 1, policy_version 1204350 (0.0009) [2023-12-27 00:08:49,587][105692] Updated weights for policy 0, policy_version 1202935 (0.0008) [2023-12-27 00:08:50,195][105620] Updated weights for policy 1, policy_version 1204360 (0.0008) [2023-12-27 00:08:50,256][105620] Updated weights for policy 1, policy_version 1204370 (0.0008) [2023-12-27 00:08:50,305][105620] Updated weights for policy 1, policy_version 1204380 (0.0008) [2023-12-27 00:08:50,397][105692] Updated weights for policy 0, policy_version 1202945 (0.0008) [2023-12-27 00:08:50,462][105692] Updated weights for policy 0, policy_version 1202955 (0.0010) [2023-12-27 00:08:50,528][105692] Updated weights for policy 0, policy_version 1202965 (0.0009) [2023-12-27 00:08:50,587][105692] Updated weights for policy 0, policy_version 1202975 (0.0012) [2023-12-27 00:08:51,049][105620] Updated weights for policy 1, policy_version 1204390 (0.0007) [2023-12-27 00:08:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 18705.1, 300 sec: 18216.8). Total num frames: 616374272. Throughput: 0: 9297.1, 1: 9532.5. Samples: 616368392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:08:51,063][104569] Avg episode reward: [(0, '9265.392'), (1, '9260.537')] [2023-12-27 00:08:51,116][105620] Updated weights for policy 1, policy_version 1204400 (0.0009) [2023-12-27 00:08:51,172][105620] Updated weights for policy 1, policy_version 1204410 (0.0009) [2023-12-27 00:08:51,321][105692] Updated weights for policy 0, policy_version 1202985 (0.0008) [2023-12-27 00:08:51,384][105692] Updated weights for policy 0, policy_version 1202995 (0.0009) [2023-12-27 00:08:51,443][105692] Updated weights for policy 0, policy_version 1203005 (0.0006) [2023-12-27 00:08:51,943][105620] Updated weights for policy 1, policy_version 1204420 (0.0007) [2023-12-27 00:08:52,008][105620] Updated weights for policy 1, policy_version 1204430 (0.0007) [2023-12-27 00:08:52,054][105620] Updated weights for policy 1, policy_version 1204440 (0.0007) [2023-12-27 00:08:52,151][105692] Updated weights for policy 0, policy_version 1203015 (0.0008) [2023-12-27 00:08:52,198][105692] Updated weights for policy 0, policy_version 1203025 (0.0009) [2023-12-27 00:08:52,255][105692] Updated weights for policy 0, policy_version 1203035 (0.0009) [2023-12-27 00:08:52,821][105620] Updated weights for policy 1, policy_version 1204450 (0.0009) [2023-12-27 00:08:52,872][105620] Updated weights for policy 1, policy_version 1204460 (0.0008) [2023-12-27 00:08:52,891][105692] Updated weights for policy 0, policy_version 1203045 (0.0010) [2023-12-27 00:08:52,922][105620] Updated weights for policy 1, policy_version 1204470 (0.0006) [2023-12-27 00:08:52,950][105692] Updated weights for policy 0, policy_version 1203055 (0.0011) [2023-12-27 00:08:52,983][105620] Updated weights for policy 1, policy_version 1204480 (0.0007) [2023-12-27 00:08:53,008][105692] Updated weights for policy 0, policy_version 1203065 (0.0010) [2023-12-27 00:08:53,688][105620] Updated weights for policy 1, policy_version 1204490 (0.0006) [2023-12-27 00:08:53,742][105620] Updated weights for policy 1, policy_version 1204500 (0.0007) [2023-12-27 00:08:53,751][105692] Updated weights for policy 0, policy_version 1203075 (0.0011) [2023-12-27 00:08:53,805][105620] Updated weights for policy 1, policy_version 1204510 (0.0005) [2023-12-27 00:08:53,807][105692] Updated weights for policy 0, policy_version 1203085 (0.0011) [2023-12-27 00:08:53,873][105692] Updated weights for policy 0, policy_version 1203095 (0.0010) [2023-12-27 00:08:54,380][105620] Updated weights for policy 1, policy_version 1204520 (0.0006) [2023-12-27 00:08:54,428][105620] Updated weights for policy 1, policy_version 1204530 (0.0007) [2023-12-27 00:08:54,481][105692] Updated weights for policy 0, policy_version 1203105 (0.0010) [2023-12-27 00:08:54,490][105620] Updated weights for policy 1, policy_version 1204540 (0.0007) [2023-12-27 00:08:54,537][105692] Updated weights for policy 0, policy_version 1203115 (0.0006) [2023-12-27 00:08:54,590][105692] Updated weights for policy 0, policy_version 1203125 (0.0006) [2023-12-27 00:08:54,642][105692] Updated weights for policy 0, policy_version 1203135 (0.0006) [2023-12-27 00:08:55,079][105620] Updated weights for policy 1, policy_version 1204550 (0.0007) [2023-12-27 00:08:55,136][105620] Updated weights for policy 1, policy_version 1204560 (0.0005) [2023-12-27 00:08:55,197][105620] Updated weights for policy 1, policy_version 1204570 (0.0010) [2023-12-27 00:08:55,313][105692] Updated weights for policy 0, policy_version 1203145 (0.0009) [2023-12-27 00:08:55,366][105692] Updated weights for policy 0, policy_version 1203155 (0.0010) [2023-12-27 00:08:55,419][105692] Updated weights for policy 0, policy_version 1203165 (0.0009) [2023-12-27 00:08:55,730][105620] Updated weights for policy 1, policy_version 1204580 (0.0008) [2023-12-27 00:08:55,784][105620] Updated weights for policy 1, policy_version 1204590 (0.0005) [2023-12-27 00:08:55,851][105620] Updated weights for policy 1, policy_version 1204600 (0.0009) [2023-12-27 00:08:56,062][104569] Fps is (10 sec: 20480.5, 60 sec: 18978.1, 300 sec: 18244.6). Total num frames: 616480768. Throughput: 0: 9327.1, 1: 9618.7. Samples: 616488980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:08:56,063][104569] Avg episode reward: [(0, '9175.565'), (1, '9077.501')] [2023-12-27 00:08:56,323][105692] Updated weights for policy 0, policy_version 1203175 (0.0009) [2023-12-27 00:08:56,380][105692] Updated weights for policy 0, policy_version 1203185 (0.0008) [2023-12-27 00:08:56,435][105620] Updated weights for policy 1, policy_version 1204610 (0.0009) [2023-12-27 00:08:56,436][105692] Updated weights for policy 0, policy_version 1203195 (0.0009) [2023-12-27 00:08:56,501][105620] Updated weights for policy 1, policy_version 1204620 (0.0005) [2023-12-27 00:08:56,569][105620] Updated weights for policy 1, policy_version 1204630 (0.0007) [2023-12-27 00:08:56,633][105620] Updated weights for policy 1, policy_version 1204640 (0.0009) [2023-12-27 00:08:57,152][105692] Updated weights for policy 0, policy_version 1203205 (0.0007) [2023-12-27 00:08:57,153][105620] Updated weights for policy 1, policy_version 1204650 (0.0005) [2023-12-27 00:08:57,206][105692] Updated weights for policy 0, policy_version 1203215 (0.0005) [2023-12-27 00:08:57,216][105620] Updated weights for policy 1, policy_version 1204660 (0.0006) [2023-12-27 00:08:57,264][105692] Updated weights for policy 0, policy_version 1203225 (0.0005) [2023-12-27 00:08:57,271][105620] Updated weights for policy 1, policy_version 1204670 (0.0006) [2023-12-27 00:08:57,824][105620] Updated weights for policy 1, policy_version 1204680 (0.0007) [2023-12-27 00:08:57,883][105692] Updated weights for policy 0, policy_version 1203235 (0.0005) [2023-12-27 00:08:57,884][105620] Updated weights for policy 1, policy_version 1204690 (0.0009) [2023-12-27 00:08:57,936][105620] Updated weights for policy 1, policy_version 1204700 (0.0009) [2023-12-27 00:08:57,947][105692] Updated weights for policy 0, policy_version 1203245 (0.0007) [2023-12-27 00:08:58,010][105692] Updated weights for policy 0, policy_version 1203255 (0.0010) [2023-12-27 00:08:58,610][105620] Updated weights for policy 1, policy_version 1204710 (0.0009) [2023-12-27 00:08:58,682][105620] Updated weights for policy 1, policy_version 1204720 (0.0011) [2023-12-27 00:08:58,706][105692] Updated weights for policy 0, policy_version 1203266 (0.0009) [2023-12-27 00:08:58,745][105620] Updated weights for policy 1, policy_version 1204730 (0.0010) [2023-12-27 00:08:58,765][105692] Updated weights for policy 0, policy_version 1203276 (0.0010) [2023-12-27 00:08:58,838][105692] Updated weights for policy 0, policy_version 1203286 (0.0009) [2023-12-27 00:08:58,896][105692] Updated weights for policy 0, policy_version 1203296 (0.0010) [2023-12-27 00:08:59,553][105620] Updated weights for policy 1, policy_version 1204740 (0.0011) [2023-12-27 00:08:59,619][105620] Updated weights for policy 1, policy_version 1204750 (0.0007) [2023-12-27 00:08:59,672][105692] Updated weights for policy 0, policy_version 1203306 (0.0007) [2023-12-27 00:08:59,683][105620] Updated weights for policy 1, policy_version 1204760 (0.0009) [2023-12-27 00:08:59,733][105692] Updated weights for policy 0, policy_version 1203316 (0.0005) [2023-12-27 00:08:59,797][105692] Updated weights for policy 0, policy_version 1203326 (0.0006) [2023-12-27 00:09:00,338][105620] Updated weights for policy 1, policy_version 1204770 (0.0009) [2023-12-27 00:09:00,402][105620] Updated weights for policy 1, policy_version 1204780 (0.0006) [2023-12-27 00:09:00,466][105620] Updated weights for policy 1, policy_version 1204790 (0.0005) [2023-12-27 00:09:00,477][105692] Updated weights for policy 0, policy_version 1203336 (0.0009) [2023-12-27 00:09:00,525][105620] Updated weights for policy 1, policy_version 1204800 (0.0005) [2023-12-27 00:09:00,535][105692] Updated weights for policy 0, policy_version 1203346 (0.0007) [2023-12-27 00:09:00,590][105692] Updated weights for policy 0, policy_version 1203356 (0.0010) [2023-12-27 00:09:01,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19114.7, 300 sec: 18272.3). Total num frames: 616579072. Throughput: 0: 9396.8, 1: 9736.5. Samples: 616551612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:09:01,062][104569] Avg episode reward: [(0, '9087.711'), (1, '8986.090')] [2023-12-27 00:09:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001203360_308109312.pth... [2023-12-27 00:09:01,069][105620] Updated weights for policy 1, policy_version 1204810 (0.0009) [2023-12-27 00:09:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001202240_307822592.pth [2023-12-27 00:09:01,133][105620] Updated weights for policy 1, policy_version 1204820 (0.0008) [2023-12-27 00:09:01,192][105620] Updated weights for policy 1, policy_version 1204830 (0.0007) [2023-12-27 00:09:01,203][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001204832_308477952.pth... [2023-12-27 00:09:01,207][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001203648_308174848.pth [2023-12-27 00:09:01,348][105692] Updated weights for policy 0, policy_version 1203366 (0.0011) [2023-12-27 00:09:01,421][105692] Updated weights for policy 0, policy_version 1203376 (0.0010) [2023-12-27 00:09:01,487][105692] Updated weights for policy 0, policy_version 1203386 (0.0007) [2023-12-27 00:09:01,955][105620] Updated weights for policy 1, policy_version 1204840 (0.0008) [2023-12-27 00:09:02,002][105620] Updated weights for policy 1, policy_version 1204850 (0.0009) [2023-12-27 00:09:02,060][105620] Updated weights for policy 1, policy_version 1204860 (0.0009) [2023-12-27 00:09:02,125][105692] Updated weights for policy 0, policy_version 1203396 (0.0005) [2023-12-27 00:09:02,176][105692] Updated weights for policy 0, policy_version 1203406 (0.0009) [2023-12-27 00:09:02,223][105692] Updated weights for policy 0, policy_version 1203416 (0.0009) [2023-12-27 00:09:02,807][105620] Updated weights for policy 1, policy_version 1204870 (0.0007) [2023-12-27 00:09:02,869][105620] Updated weights for policy 1, policy_version 1204880 (0.0006) [2023-12-27 00:09:02,928][105620] Updated weights for policy 1, policy_version 1204890 (0.0009) [2023-12-27 00:09:02,989][105692] Updated weights for policy 0, policy_version 1203426 (0.0008) [2023-12-27 00:09:03,047][105692] Updated weights for policy 0, policy_version 1203436 (0.0009) [2023-12-27 00:09:03,115][105692] Updated weights for policy 0, policy_version 1203446 (0.0010) [2023-12-27 00:09:03,181][105692] Updated weights for policy 0, policy_version 1203456 (0.0010) [2023-12-27 00:09:03,474][105620] Updated weights for policy 1, policy_version 1204900 (0.0008) [2023-12-27 00:09:03,533][105620] Updated weights for policy 1, policy_version 1204910 (0.0009) [2023-12-27 00:09:03,590][105620] Updated weights for policy 1, policy_version 1204920 (0.0009) [2023-12-27 00:09:03,967][105692] Updated weights for policy 0, policy_version 1203466 (0.0009) [2023-12-27 00:09:04,036][105692] Updated weights for policy 0, policy_version 1203476 (0.0009) [2023-12-27 00:09:04,102][105692] Updated weights for policy 0, policy_version 1203486 (0.0009) [2023-12-27 00:09:04,316][105620] Updated weights for policy 1, policy_version 1204930 (0.0008) [2023-12-27 00:09:04,378][105620] Updated weights for policy 1, policy_version 1204940 (0.0008) [2023-12-27 00:09:04,442][105620] Updated weights for policy 1, policy_version 1204950 (0.0011) [2023-12-27 00:09:04,502][105620] Updated weights for policy 1, policy_version 1204960 (0.0011) [2023-12-27 00:09:04,871][105692] Updated weights for policy 0, policy_version 1203496 (0.0006) [2023-12-27 00:09:04,923][105585] KL-divergence is very high: 101.3083 [2023-12-27 00:09:04,945][105692] Updated weights for policy 0, policy_version 1203506 (0.0006) [2023-12-27 00:09:04,985][105585] KL-divergence is very high: 174.5562 [2023-12-27 00:09:05,015][105692] Updated weights for policy 0, policy_version 1203516 (0.0010) [2023-12-27 00:09:05,029][105585] KL-divergence is very high: 167.1495 [2023-12-27 00:09:05,147][105620] Updated weights for policy 1, policy_version 1204970 (0.0010) [2023-12-27 00:09:05,210][105620] Updated weights for policy 1, policy_version 1204980 (0.0010) [2023-12-27 00:09:05,268][105620] Updated weights for policy 1, policy_version 1204990 (0.0010) [2023-12-27 00:09:05,758][105692] Updated weights for policy 0, policy_version 1203526 (0.0010) [2023-12-27 00:09:05,806][105620] Updated weights for policy 1, policy_version 1205000 (0.0006) [2023-12-27 00:09:05,810][105692] Updated weights for policy 0, policy_version 1203536 (0.0010) [2023-12-27 00:09:05,858][105692] Updated weights for policy 0, policy_version 1203546 (0.0010) [2023-12-27 00:09:05,862][105620] Updated weights for policy 1, policy_version 1205010 (0.0005) [2023-12-27 00:09:05,915][105620] Updated weights for policy 1, policy_version 1205020 (0.0005) [2023-12-27 00:09:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19251.2, 300 sec: 18327.9). Total num frames: 616685568. Throughput: 0: 9344.1, 1: 9812.5. Samples: 616668484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:09:06,062][104569] Avg episode reward: [(0, '8996.856'), (1, '9260.584')] [2023-12-27 00:09:06,515][105692] Updated weights for policy 0, policy_version 1203556 (0.0009) [2023-12-27 00:09:06,538][105620] Updated weights for policy 1, policy_version 1205030 (0.0009) [2023-12-27 00:09:06,572][105692] Updated weights for policy 0, policy_version 1203566 (0.0006) [2023-12-27 00:09:06,598][105620] Updated weights for policy 1, policy_version 1205040 (0.0011) [2023-12-27 00:09:06,625][105692] Updated weights for policy 0, policy_version 1203576 (0.0006) [2023-12-27 00:09:06,661][105620] Updated weights for policy 1, policy_version 1205050 (0.0011) [2023-12-27 00:09:07,275][105692] Updated weights for policy 0, policy_version 1203586 (0.0006) [2023-12-27 00:09:07,332][105692] Updated weights for policy 0, policy_version 1203596 (0.0006) [2023-12-27 00:09:07,372][105585] KL-divergence is very high: 120.6986 [2023-12-27 00:09:07,380][105692] Updated weights for policy 0, policy_version 1203606 (0.0005) [2023-12-27 00:09:07,399][105620] Updated weights for policy 1, policy_version 1205060 (0.0010) [2023-12-27 00:09:07,442][105692] Updated weights for policy 0, policy_version 1203616 (0.0008) [2023-12-27 00:09:07,461][105620] Updated weights for policy 1, policy_version 1205070 (0.0007) [2023-12-27 00:09:07,515][105620] Updated weights for policy 1, policy_version 1205080 (0.0010) [2023-12-27 00:09:07,992][105692] Updated weights for policy 0, policy_version 1203626 (0.0005) [2023-12-27 00:09:08,050][105692] Updated weights for policy 0, policy_version 1203636 (0.0007) [2023-12-27 00:09:08,100][105692] Updated weights for policy 0, policy_version 1203646 (0.0008) [2023-12-27 00:09:08,359][105620] Updated weights for policy 1, policy_version 1205091 (0.0010) [2023-12-27 00:09:08,409][105620] Updated weights for policy 1, policy_version 1205101 (0.0010) [2023-12-27 00:09:08,465][105620] Updated weights for policy 1, policy_version 1205111 (0.0009) [2023-12-27 00:09:08,767][105692] Updated weights for policy 0, policy_version 1203656 (0.0009) [2023-12-27 00:09:08,826][105692] Updated weights for policy 0, policy_version 1203666 (0.0009) [2023-12-27 00:09:08,884][105692] Updated weights for policy 0, policy_version 1203676 (0.0009) [2023-12-27 00:09:09,240][105620] Updated weights for policy 1, policy_version 1205121 (0.0008) [2023-12-27 00:09:09,303][105620] Updated weights for policy 1, policy_version 1205131 (0.0009) [2023-12-27 00:09:09,371][105620] Updated weights for policy 1, policy_version 1205141 (0.0009) [2023-12-27 00:09:09,443][105620] Updated weights for policy 1, policy_version 1205151 (0.0008) [2023-12-27 00:09:09,658][105692] Updated weights for policy 0, policy_version 1203686 (0.0009) [2023-12-27 00:09:09,711][105692] Updated weights for policy 0, policy_version 1203696 (0.0009) [2023-12-27 00:09:09,767][105692] Updated weights for policy 0, policy_version 1203706 (0.0009) [2023-12-27 00:09:10,209][105620] Updated weights for policy 1, policy_version 1205161 (0.0009) [2023-12-27 00:09:10,272][105620] Updated weights for policy 1, policy_version 1205171 (0.0009) [2023-12-27 00:09:10,331][105620] Updated weights for policy 1, policy_version 1205181 (0.0009) [2023-12-27 00:09:10,569][105692] Updated weights for policy 0, policy_version 1203716 (0.0009) [2023-12-27 00:09:10,629][105692] Updated weights for policy 0, policy_version 1203726 (0.0008) [2023-12-27 00:09:10,695][105692] Updated weights for policy 0, policy_version 1203736 (0.0009) [2023-12-27 00:09:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19114.7, 300 sec: 18300.1). Total num frames: 616775680. Throughput: 0: 9476.5, 1: 9884.3. Samples: 616786532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:09:11,062][104569] Avg episode reward: [(0, '8544.315'), (1, '9169.530')] [2023-12-27 00:09:11,097][105620] Updated weights for policy 1, policy_version 1205191 (0.0010) [2023-12-27 00:09:11,166][105620] Updated weights for policy 1, policy_version 1205201 (0.0009) [2023-12-27 00:09:11,227][105620] Updated weights for policy 1, policy_version 1205211 (0.0010) [2023-12-27 00:09:11,469][105692] Updated weights for policy 0, policy_version 1203746 (0.0010) [2023-12-27 00:09:11,534][105692] Updated weights for policy 0, policy_version 1203756 (0.0009) [2023-12-27 00:09:11,602][105692] Updated weights for policy 0, policy_version 1203766 (0.0009) [2023-12-27 00:09:11,664][105692] Updated weights for policy 0, policy_version 1203776 (0.0008) [2023-12-27 00:09:12,065][105620] Updated weights for policy 1, policy_version 1205221 (0.0009) [2023-12-27 00:09:12,124][105620] Updated weights for policy 1, policy_version 1205231 (0.0011) [2023-12-27 00:09:12,183][105620] Updated weights for policy 1, policy_version 1205241 (0.0011) [2023-12-27 00:09:12,341][105692] Updated weights for policy 0, policy_version 1203786 (0.0009) [2023-12-27 00:09:12,404][105692] Updated weights for policy 0, policy_version 1203796 (0.0008) [2023-12-27 00:09:12,468][105692] Updated weights for policy 0, policy_version 1203806 (0.0008) [2023-12-27 00:09:12,845][105620] Updated weights for policy 1, policy_version 1205251 (0.0009) [2023-12-27 00:09:12,894][105620] Updated weights for policy 1, policy_version 1205261 (0.0005) [2023-12-27 00:09:12,950][105620] Updated weights for policy 1, policy_version 1205271 (0.0005) [2023-12-27 00:09:13,305][105692] Updated weights for policy 0, policy_version 1203816 (0.0006) [2023-12-27 00:09:13,358][105692] Updated weights for policy 0, policy_version 1203826 (0.0005) [2023-12-27 00:09:13,411][105692] Updated weights for policy 0, policy_version 1203836 (0.0005) [2023-12-27 00:09:13,508][105620] Updated weights for policy 1, policy_version 1205281 (0.0005) [2023-12-27 00:09:13,558][105620] Updated weights for policy 1, policy_version 1205291 (0.0005) [2023-12-27 00:09:13,603][105620] Updated weights for policy 1, policy_version 1205301 (0.0005) [2023-12-27 00:09:13,660][105620] Updated weights for policy 1, policy_version 1205311 (0.0005) [2023-12-27 00:09:14,102][105692] Updated weights for policy 0, policy_version 1203846 (0.0008) [2023-12-27 00:09:14,164][105692] Updated weights for policy 0, policy_version 1203856 (0.0010) [2023-12-27 00:09:14,237][105692] Updated weights for policy 0, policy_version 1203866 (0.0006) [2023-12-27 00:09:14,310][105620] Updated weights for policy 1, policy_version 1205321 (0.0008) [2023-12-27 00:09:14,373][105620] Updated weights for policy 1, policy_version 1205331 (0.0007) [2023-12-27 00:09:14,430][105620] Updated weights for policy 1, policy_version 1205341 (0.0006) [2023-12-27 00:09:14,823][105692] Updated weights for policy 0, policy_version 1203876 (0.0010) [2023-12-27 00:09:14,879][105692] Updated weights for policy 0, policy_version 1203886 (0.0008) [2023-12-27 00:09:14,945][105692] Updated weights for policy 0, policy_version 1203896 (0.0009) [2023-12-27 00:09:15,212][105620] Updated weights for policy 1, policy_version 1205351 (0.0008) [2023-12-27 00:09:15,269][105620] Updated weights for policy 1, policy_version 1205361 (0.0009) [2023-12-27 00:09:15,327][105620] Updated weights for policy 1, policy_version 1205371 (0.0009) [2023-12-27 00:09:15,685][105692] Updated weights for policy 0, policy_version 1203906 (0.0008) [2023-12-27 00:09:15,733][105692] Updated weights for policy 0, policy_version 1203916 (0.0005) [2023-12-27 00:09:15,777][105692] Updated weights for policy 0, policy_version 1203926 (0.0005) [2023-12-27 00:09:15,838][105692] Updated weights for policy 0, policy_version 1203936 (0.0005) [2023-12-27 00:09:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 18355.6). Total num frames: 616873984. Throughput: 0: 9504.1, 1: 9913.6. Samples: 616844772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:09:16,063][104569] Avg episode reward: [(0, '8724.513'), (1, '9168.751')] [2023-12-27 00:09:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001203936_308256768.pth... [2023-12-27 00:09:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001205376_308617216.pth... [2023-12-27 00:09:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001202816_307970048.pth [2023-12-27 00:09:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001204224_308322304.pth [2023-12-27 00:09:16,220][105620] Updated weights for policy 1, policy_version 1205381 (0.0008) [2023-12-27 00:09:16,291][105620] Updated weights for policy 1, policy_version 1205391 (0.0010) [2023-12-27 00:09:16,357][105620] Updated weights for policy 1, policy_version 1205401 (0.0009) [2023-12-27 00:09:16,384][105692] Updated weights for policy 0, policy_version 1203946 (0.0010) [2023-12-27 00:09:16,437][105692] Updated weights for policy 0, policy_version 1203956 (0.0011) [2023-12-27 00:09:16,486][105692] Updated weights for policy 0, policy_version 1203966 (0.0011) [2023-12-27 00:09:17,109][105692] Updated weights for policy 0, policy_version 1203976 (0.0006) [2023-12-27 00:09:17,169][105692] Updated weights for policy 0, policy_version 1203986 (0.0005) [2023-12-27 00:09:17,199][105620] Updated weights for policy 1, policy_version 1205411 (0.0007) [2023-12-27 00:09:17,229][105692] Updated weights for policy 0, policy_version 1203996 (0.0009) [2023-12-27 00:09:17,255][105620] Updated weights for policy 1, policy_version 1205421 (0.0005) [2023-12-27 00:09:17,316][105620] Updated weights for policy 1, policy_version 1205431 (0.0008) [2023-12-27 00:09:17,920][105692] Updated weights for policy 0, policy_version 1204006 (0.0011) [2023-12-27 00:09:17,976][105692] Updated weights for policy 0, policy_version 1204016 (0.0010) [2023-12-27 00:09:18,013][105620] Updated weights for policy 1, policy_version 1205441 (0.0008) [2023-12-27 00:09:18,035][105692] Updated weights for policy 0, policy_version 1204026 (0.0011) [2023-12-27 00:09:18,073][105620] Updated weights for policy 1, policy_version 1205451 (0.0005) [2023-12-27 00:09:18,125][105620] Updated weights for policy 1, policy_version 1205461 (0.0009) [2023-12-27 00:09:18,173][105620] Updated weights for policy 1, policy_version 1205471 (0.0007) [2023-12-27 00:09:18,673][105692] Updated weights for policy 0, policy_version 1204036 (0.0010) [2023-12-27 00:09:18,739][105692] Updated weights for policy 0, policy_version 1204046 (0.0007) [2023-12-27 00:09:18,798][105620] Updated weights for policy 1, policy_version 1205482 (0.0006) [2023-12-27 00:09:18,804][105692] Updated weights for policy 0, policy_version 1204056 (0.0009) [2023-12-27 00:09:18,856][105620] Updated weights for policy 1, policy_version 1205492 (0.0006) [2023-12-27 00:09:18,921][105620] Updated weights for policy 1, policy_version 1205502 (0.0005) [2023-12-27 00:09:19,417][105692] Updated weights for policy 0, policy_version 1204066 (0.0009) [2023-12-27 00:09:19,473][105692] Updated weights for policy 0, policy_version 1204076 (0.0009) [2023-12-27 00:09:19,543][105692] Updated weights for policy 0, policy_version 1204086 (0.0007) [2023-12-27 00:09:19,553][105620] Updated weights for policy 1, policy_version 1205512 (0.0006) [2023-12-27 00:09:19,596][105692] Updated weights for policy 0, policy_version 1204096 (0.0010) [2023-12-27 00:09:19,617][105620] Updated weights for policy 1, policy_version 1205522 (0.0006) [2023-12-27 00:09:19,682][105620] Updated weights for policy 1, policy_version 1205532 (0.0007) [2023-12-27 00:09:20,367][105692] Updated weights for policy 0, policy_version 1204106 (0.0007) [2023-12-27 00:09:20,401][105620] Updated weights for policy 1, policy_version 1205542 (0.0007) [2023-12-27 00:09:20,426][105692] Updated weights for policy 0, policy_version 1204116 (0.0008) [2023-12-27 00:09:20,469][105620] Updated weights for policy 1, policy_version 1205552 (0.0005) [2023-12-27 00:09:20,488][105692] Updated weights for policy 0, policy_version 1204126 (0.0009) [2023-12-27 00:09:20,532][105620] Updated weights for policy 1, policy_version 1205562 (0.0006) [2023-12-27 00:09:21,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 18383.4). Total num frames: 616972288. Throughput: 0: 9655.5, 1: 9916.4. Samples: 616964512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:09:21,063][104569] Avg episode reward: [(0, '9265.662'), (1, '9167.618')] [2023-12-27 00:09:21,216][105692] Updated weights for policy 0, policy_version 1204136 (0.0009) [2023-12-27 00:09:21,281][105692] Updated weights for policy 0, policy_version 1204146 (0.0008) [2023-12-27 00:09:21,300][105620] Updated weights for policy 1, policy_version 1205573 (0.0008) [2023-12-27 00:09:21,333][105692] Updated weights for policy 0, policy_version 1204156 (0.0007) [2023-12-27 00:09:21,355][105620] Updated weights for policy 1, policy_version 1205583 (0.0008) [2023-12-27 00:09:21,421][105620] Updated weights for policy 1, policy_version 1205593 (0.0008) [2023-12-27 00:09:22,145][105620] Updated weights for policy 1, policy_version 1205603 (0.0009) [2023-12-27 00:09:22,163][105692] Updated weights for policy 0, policy_version 1204166 (0.0009) [2023-12-27 00:09:22,193][105620] Updated weights for policy 1, policy_version 1205613 (0.0006) [2023-12-27 00:09:22,217][105692] Updated weights for policy 0, policy_version 1204176 (0.0006) [2023-12-27 00:09:22,248][105620] Updated weights for policy 1, policy_version 1205623 (0.0008) [2023-12-27 00:09:22,274][105692] Updated weights for policy 0, policy_version 1204186 (0.0008) [2023-12-27 00:09:23,015][105620] Updated weights for policy 1, policy_version 1205633 (0.0009) [2023-12-27 00:09:23,057][105692] Updated weights for policy 0, policy_version 1204196 (0.0008) [2023-12-27 00:09:23,071][105620] Updated weights for policy 1, policy_version 1205643 (0.0009) [2023-12-27 00:09:23,114][105692] Updated weights for policy 0, policy_version 1204206 (0.0006) [2023-12-27 00:09:23,128][105620] Updated weights for policy 1, policy_version 1205653 (0.0007) [2023-12-27 00:09:23,168][105692] Updated weights for policy 0, policy_version 1204216 (0.0007) [2023-12-27 00:09:23,184][105620] Updated weights for policy 1, policy_version 1205663 (0.0007) [2023-12-27 00:09:23,881][105692] Updated weights for policy 0, policy_version 1204226 (0.0006) [2023-12-27 00:09:23,927][105620] Updated weights for policy 1, policy_version 1205673 (0.0009) [2023-12-27 00:09:23,941][105692] Updated weights for policy 0, policy_version 1204236 (0.0005) [2023-12-27 00:09:23,977][105620] Updated weights for policy 1, policy_version 1205683 (0.0008) [2023-12-27 00:09:23,998][105692] Updated weights for policy 0, policy_version 1204246 (0.0005) [2023-12-27 00:09:24,033][105620] Updated weights for policy 1, policy_version 1205693 (0.0008) [2023-12-27 00:09:24,051][105692] Updated weights for policy 0, policy_version 1204256 (0.0005) [2023-12-27 00:09:24,720][105692] Updated weights for policy 0, policy_version 1204266 (0.0009) [2023-12-27 00:09:24,769][105692] Updated weights for policy 0, policy_version 1204276 (0.0009) [2023-12-27 00:09:24,809][105620] Updated weights for policy 1, policy_version 1205703 (0.0007) [2023-12-27 00:09:24,820][105692] Updated weights for policy 0, policy_version 1204286 (0.0007) [2023-12-27 00:09:24,866][105620] Updated weights for policy 1, policy_version 1205713 (0.0009) [2023-12-27 00:09:24,923][105620] Updated weights for policy 1, policy_version 1205723 (0.0010) [2023-12-27 00:09:25,561][105692] Updated weights for policy 0, policy_version 1204296 (0.0009) [2023-12-27 00:09:25,625][105692] Updated weights for policy 0, policy_version 1204306 (0.0010) [2023-12-27 00:09:25,686][105692] Updated weights for policy 0, policy_version 1204316 (0.0010) [2023-12-27 00:09:25,718][105620] Updated weights for policy 1, policy_version 1205733 (0.0007) [2023-12-27 00:09:25,782][105620] Updated weights for policy 1, policy_version 1205743 (0.0005) [2023-12-27 00:09:25,848][105620] Updated weights for policy 1, policy_version 1205753 (0.0005) [2023-12-27 00:09:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.8, 300 sec: 18383.4). Total num frames: 617070592. Throughput: 0: 9706.2, 1: 9921.1. Samples: 617077876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:09:26,063][104569] Avg episode reward: [(0, '9175.856'), (1, '9258.673')] [2023-12-27 00:09:26,395][105692] Updated weights for policy 0, policy_version 1204326 (0.0006) [2023-12-27 00:09:26,445][105692] Updated weights for policy 0, policy_version 1204336 (0.0006) [2023-12-27 00:09:26,458][105620] Updated weights for policy 1, policy_version 1205763 (0.0006) [2023-12-27 00:09:26,491][105692] Updated weights for policy 0, policy_version 1204346 (0.0006) [2023-12-27 00:09:26,514][105620] Updated weights for policy 1, policy_version 1205773 (0.0011) [2023-12-27 00:09:26,574][105620] Updated weights for policy 1, policy_version 1205783 (0.0011) [2023-12-27 00:09:27,165][105692] Updated weights for policy 0, policy_version 1204356 (0.0006) [2023-12-27 00:09:27,210][105692] Updated weights for policy 0, policy_version 1204366 (0.0007) [2023-12-27 00:09:27,215][105620] Updated weights for policy 1, policy_version 1205793 (0.0011) [2023-12-27 00:09:27,259][105620] Updated weights for policy 1, policy_version 1205803 (0.0010) [2023-12-27 00:09:27,262][105692] Updated weights for policy 0, policy_version 1204376 (0.0005) [2023-12-27 00:09:27,311][105620] Updated weights for policy 1, policy_version 1205813 (0.0010) [2023-12-27 00:09:27,366][105620] Updated weights for policy 1, policy_version 1205823 (0.0011) [2023-12-27 00:09:27,800][105692] Updated weights for policy 0, policy_version 1204386 (0.0005) [2023-12-27 00:09:27,849][105692] Updated weights for policy 0, policy_version 1204396 (0.0005) [2023-12-27 00:09:27,905][105692] Updated weights for policy 0, policy_version 1204406 (0.0005) [2023-12-27 00:09:27,962][105692] Updated weights for policy 0, policy_version 1204416 (0.0005) [2023-12-27 00:09:28,152][105620] Updated weights for policy 1, policy_version 1205833 (0.0008) [2023-12-27 00:09:28,201][105620] Updated weights for policy 1, policy_version 1205843 (0.0008) [2023-12-27 00:09:28,258][105620] Updated weights for policy 1, policy_version 1205853 (0.0009) [2023-12-27 00:09:28,560][105692] Updated weights for policy 0, policy_version 1204426 (0.0005) [2023-12-27 00:09:28,623][105692] Updated weights for policy 0, policy_version 1204436 (0.0009) [2023-12-27 00:09:28,685][105692] Updated weights for policy 0, policy_version 1204446 (0.0010) [2023-12-27 00:09:28,967][105620] Updated weights for policy 1, policy_version 1205863 (0.0009) [2023-12-27 00:09:29,025][105620] Updated weights for policy 1, policy_version 1205873 (0.0009) [2023-12-27 00:09:29,080][105620] Updated weights for policy 1, policy_version 1205883 (0.0009) [2023-12-27 00:09:29,399][105692] Updated weights for policy 0, policy_version 1204456 (0.0008) [2023-12-27 00:09:29,457][105692] Updated weights for policy 0, policy_version 1204466 (0.0009) [2023-12-27 00:09:29,515][105692] Updated weights for policy 0, policy_version 1204476 (0.0010) [2023-12-27 00:09:29,829][105620] Updated weights for policy 1, policy_version 1205893 (0.0009) [2023-12-27 00:09:29,890][105620] Updated weights for policy 1, policy_version 1205903 (0.0008) [2023-12-27 00:09:29,960][105620] Updated weights for policy 1, policy_version 1205913 (0.0010) [2023-12-27 00:09:30,280][105692] Updated weights for policy 0, policy_version 1204486 (0.0009) [2023-12-27 00:09:30,346][105692] Updated weights for policy 0, policy_version 1204496 (0.0008) [2023-12-27 00:09:30,409][105692] Updated weights for policy 0, policy_version 1204506 (0.0008) [2023-12-27 00:09:30,706][105620] Updated weights for policy 1, policy_version 1205923 (0.0010) [2023-12-27 00:09:30,758][105620] Updated weights for policy 1, policy_version 1205933 (0.0009) [2023-12-27 00:09:30,806][105620] Updated weights for policy 1, policy_version 1205943 (0.0009) [2023-12-27 00:09:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 18383.4). Total num frames: 617168896. Throughput: 0: 9817.4, 1: 9941.7. Samples: 617140980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:09:31,063][104569] Avg episode reward: [(0, '9086.987'), (1, '9259.624')] [2023-12-27 00:09:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001205952_308764672.pth... [2023-12-27 00:09:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001204832_308477952.pth [2023-12-27 00:09:31,096][105692] Updated weights for policy 0, policy_version 1204516 (0.0008) [2023-12-27 00:09:31,159][105692] Updated weights for policy 0, policy_version 1204526 (0.0009) [2023-12-27 00:09:31,215][105692] Updated weights for policy 0, policy_version 1204536 (0.0009) [2023-12-27 00:09:31,255][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001204544_308412416.pth... [2023-12-27 00:09:31,260][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001203360_308109312.pth [2023-12-27 00:09:31,619][105620] Updated weights for policy 1, policy_version 1205953 (0.0010) [2023-12-27 00:09:31,683][105620] Updated weights for policy 1, policy_version 1205963 (0.0010) [2023-12-27 00:09:31,752][105620] Updated weights for policy 1, policy_version 1205973 (0.0010) [2023-12-27 00:09:31,823][105620] Updated weights for policy 1, policy_version 1205983 (0.0009) [2023-12-27 00:09:31,940][105692] Updated weights for policy 0, policy_version 1204546 (0.0008) [2023-12-27 00:09:32,004][105692] Updated weights for policy 0, policy_version 1204556 (0.0006) [2023-12-27 00:09:32,063][105692] Updated weights for policy 0, policy_version 1204566 (0.0006) [2023-12-27 00:09:32,127][105692] Updated weights for policy 0, policy_version 1204576 (0.0008) [2023-12-27 00:09:32,600][105620] Updated weights for policy 1, policy_version 1205993 (0.0009) [2023-12-27 00:09:32,652][105620] Updated weights for policy 1, policy_version 1206003 (0.0010) [2023-12-27 00:09:32,704][105620] Updated weights for policy 1, policy_version 1206014 (0.0010) [2023-12-27 00:09:32,750][105692] Updated weights for policy 0, policy_version 1204586 (0.0005) [2023-12-27 00:09:32,802][105692] Updated weights for policy 0, policy_version 1204596 (0.0006) [2023-12-27 00:09:32,851][105692] Updated weights for policy 0, policy_version 1204606 (0.0008) [2023-12-27 00:09:33,493][105692] Updated weights for policy 0, policy_version 1204616 (0.0008) [2023-12-27 00:09:33,523][105620] Updated weights for policy 1, policy_version 1206024 (0.0007) [2023-12-27 00:09:33,551][105692] Updated weights for policy 0, policy_version 1204626 (0.0006) [2023-12-27 00:09:33,569][105620] Updated weights for policy 1, policy_version 1206034 (0.0008) [2023-12-27 00:09:33,607][105692] Updated weights for policy 0, policy_version 1204636 (0.0008) [2023-12-27 00:09:33,614][105620] Updated weights for policy 1, policy_version 1206044 (0.0005) [2023-12-27 00:09:34,232][105620] Updated weights for policy 1, policy_version 1206054 (0.0008) [2023-12-27 00:09:34,298][105620] Updated weights for policy 1, policy_version 1206064 (0.0008) [2023-12-27 00:09:34,346][105620] Updated weights for policy 1, policy_version 1206074 (0.0006) [2023-12-27 00:09:34,348][105692] Updated weights for policy 0, policy_version 1204646 (0.0009) [2023-12-27 00:09:34,397][105692] Updated weights for policy 0, policy_version 1204656 (0.0011) [2023-12-27 00:09:34,452][105692] Updated weights for policy 0, policy_version 1204666 (0.0010) [2023-12-27 00:09:35,086][105692] Updated weights for policy 0, policy_version 1204676 (0.0008) [2023-12-27 00:09:35,133][105692] Updated weights for policy 0, policy_version 1204686 (0.0008) [2023-12-27 00:09:35,181][105620] Updated weights for policy 1, policy_version 1206084 (0.0007) [2023-12-27 00:09:35,185][105692] Updated weights for policy 0, policy_version 1204696 (0.0009) [2023-12-27 00:09:35,230][105620] Updated weights for policy 1, policy_version 1206094 (0.0010) [2023-12-27 00:09:35,285][105620] Updated weights for policy 1, policy_version 1206104 (0.0010) [2023-12-27 00:09:35,958][105692] Updated weights for policy 0, policy_version 1204706 (0.0009) [2023-12-27 00:09:36,009][105692] Updated weights for policy 0, policy_version 1204716 (0.0008) [2023-12-27 00:09:36,053][105620] Updated weights for policy 1, policy_version 1206114 (0.0010) [2023-12-27 00:09:36,062][104569] Fps is (10 sec: 18842.3, 60 sec: 19387.8, 300 sec: 18411.2). Total num frames: 617259008. Throughput: 0: 9859.8, 1: 9843.1. Samples: 617255020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:09:36,062][104569] Avg episode reward: [(0, '9084.626'), (1, '9167.889')] [2023-12-27 00:09:36,068][105692] Updated weights for policy 0, policy_version 1204726 (0.0009) [2023-12-27 00:09:36,107][105620] Updated weights for policy 1, policy_version 1206124 (0.0008) [2023-12-27 00:09:36,128][105692] Updated weights for policy 0, policy_version 1204736 (0.0006) [2023-12-27 00:09:36,166][105620] Updated weights for policy 1, policy_version 1206134 (0.0008) [2023-12-27 00:09:36,229][105620] Updated weights for policy 1, policy_version 1206144 (0.0009) [2023-12-27 00:09:36,903][105692] Updated weights for policy 0, policy_version 1204746 (0.0008) [2023-12-27 00:09:36,960][105692] Updated weights for policy 0, policy_version 1204756 (0.0009) [2023-12-27 00:09:36,971][105620] Updated weights for policy 1, policy_version 1206154 (0.0006) [2023-12-27 00:09:37,017][105692] Updated weights for policy 0, policy_version 1204766 (0.0009) [2023-12-27 00:09:37,034][105620] Updated weights for policy 1, policy_version 1206164 (0.0006) [2023-12-27 00:09:37,096][105620] Updated weights for policy 1, policy_version 1206174 (0.0009) [2023-12-27 00:09:37,769][105620] Updated weights for policy 1, policy_version 1206184 (0.0009) [2023-12-27 00:09:37,812][105692] Updated weights for policy 0, policy_version 1204776 (0.0009) [2023-12-27 00:09:37,819][105620] Updated weights for policy 1, policy_version 1206194 (0.0007) [2023-12-27 00:09:37,862][105692] Updated weights for policy 0, policy_version 1204786 (0.0006) [2023-12-27 00:09:37,877][105620] Updated weights for policy 1, policy_version 1206204 (0.0008) [2023-12-27 00:09:37,915][105692] Updated weights for policy 0, policy_version 1204796 (0.0009) [2023-12-27 00:09:38,580][105620] Updated weights for policy 1, policy_version 1206214 (0.0007) [2023-12-27 00:09:38,632][105620] Updated weights for policy 1, policy_version 1206224 (0.0009) [2023-12-27 00:09:38,639][105692] Updated weights for policy 0, policy_version 1204806 (0.0010) [2023-12-27 00:09:38,684][105620] Updated weights for policy 1, policy_version 1206234 (0.0006) [2023-12-27 00:09:38,701][105692] Updated weights for policy 0, policy_version 1204816 (0.0010) [2023-12-27 00:09:38,763][105692] Updated weights for policy 0, policy_version 1204826 (0.0010) [2023-12-27 00:09:39,446][105692] Updated weights for policy 0, policy_version 1204836 (0.0008) [2023-12-27 00:09:39,499][105692] Updated weights for policy 0, policy_version 1204846 (0.0008) [2023-12-27 00:09:39,532][105620] Updated weights for policy 1, policy_version 1206244 (0.0010) [2023-12-27 00:09:39,560][105692] Updated weights for policy 0, policy_version 1204856 (0.0008) [2023-12-27 00:09:39,594][105620] Updated weights for policy 1, policy_version 1206254 (0.0010) [2023-12-27 00:09:39,652][105620] Updated weights for policy 1, policy_version 1206264 (0.0008) [2023-12-27 00:09:40,349][105692] Updated weights for policy 0, policy_version 1204866 (0.0008) [2023-12-27 00:09:40,375][105620] Updated weights for policy 1, policy_version 1206274 (0.0009) [2023-12-27 00:09:40,406][105692] Updated weights for policy 0, policy_version 1204876 (0.0007) [2023-12-27 00:09:40,435][105620] Updated weights for policy 1, policy_version 1206284 (0.0006) [2023-12-27 00:09:40,469][105692] Updated weights for policy 0, policy_version 1204886 (0.0007) [2023-12-27 00:09:40,498][105620] Updated weights for policy 1, policy_version 1206294 (0.0006) [2023-12-27 00:09:40,537][105692] Updated weights for policy 0, policy_version 1204896 (0.0008) [2023-12-27 00:09:40,567][105620] Updated weights for policy 1, policy_version 1206304 (0.0006) [2023-12-27 00:09:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 18438.9). Total num frames: 617357312. Throughput: 0: 9830.6, 1: 9710.3. Samples: 617368320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:09:41,063][104569] Avg episode reward: [(0, '8908.603'), (1, '9076.360')] [2023-12-27 00:09:41,177][105692] Updated weights for policy 0, policy_version 1204906 (0.0010) [2023-12-27 00:09:41,238][105692] Updated weights for policy 0, policy_version 1204916 (0.0010) [2023-12-27 00:09:41,296][105692] Updated weights for policy 0, policy_version 1204926 (0.0009) [2023-12-27 00:09:41,298][105620] Updated weights for policy 1, policy_version 1206314 (0.0008) [2023-12-27 00:09:41,369][105620] Updated weights for policy 1, policy_version 1206324 (0.0008) [2023-12-27 00:09:41,425][105620] Updated weights for policy 1, policy_version 1206334 (0.0008) [2023-12-27 00:09:42,086][105692] Updated weights for policy 0, policy_version 1204936 (0.0011) [2023-12-27 00:09:42,149][105692] Updated weights for policy 0, policy_version 1204946 (0.0011) [2023-12-27 00:09:42,205][105692] Updated weights for policy 0, policy_version 1204956 (0.0011) [2023-12-27 00:09:42,215][105620] Updated weights for policy 1, policy_version 1206344 (0.0006) [2023-12-27 00:09:42,280][105620] Updated weights for policy 1, policy_version 1206354 (0.0008) [2023-12-27 00:09:42,345][105620] Updated weights for policy 1, policy_version 1206364 (0.0008) [2023-12-27 00:09:42,936][105692] Updated weights for policy 0, policy_version 1204966 (0.0009) [2023-12-27 00:09:43,000][105692] Updated weights for policy 0, policy_version 1204976 (0.0009) [2023-12-27 00:09:43,061][105692] Updated weights for policy 0, policy_version 1204986 (0.0008) [2023-12-27 00:09:43,090][105620] Updated weights for policy 1, policy_version 1206374 (0.0007) [2023-12-27 00:09:43,142][105620] Updated weights for policy 1, policy_version 1206384 (0.0005) [2023-12-27 00:09:43,193][105620] Updated weights for policy 1, policy_version 1206394 (0.0005) [2023-12-27 00:09:43,711][105692] Updated weights for policy 0, policy_version 1204996 (0.0007) [2023-12-27 00:09:43,768][105692] Updated weights for policy 0, policy_version 1205006 (0.0008) [2023-12-27 00:09:43,819][105692] Updated weights for policy 0, policy_version 1205016 (0.0009) [2023-12-27 00:09:43,929][105620] Updated weights for policy 1, policy_version 1206404 (0.0006) [2023-12-27 00:09:43,983][105620] Updated weights for policy 1, policy_version 1206414 (0.0009) [2023-12-27 00:09:44,030][105620] Updated weights for policy 1, policy_version 1206424 (0.0008) [2023-12-27 00:09:44,487][105692] Updated weights for policy 0, policy_version 1205026 (0.0009) [2023-12-27 00:09:44,545][105692] Updated weights for policy 0, policy_version 1205036 (0.0007) [2023-12-27 00:09:44,594][105692] Updated weights for policy 0, policy_version 1205046 (0.0006) [2023-12-27 00:09:44,648][105692] Updated weights for policy 0, policy_version 1205056 (0.0009) [2023-12-27 00:09:44,850][105620] Updated weights for policy 1, policy_version 1206434 (0.0008) [2023-12-27 00:09:44,906][105620] Updated weights for policy 1, policy_version 1206444 (0.0009) [2023-12-27 00:09:44,964][105620] Updated weights for policy 1, policy_version 1206454 (0.0009) [2023-12-27 00:09:45,018][105620] Updated weights for policy 1, policy_version 1206464 (0.0008) [2023-12-27 00:09:45,375][105692] Updated weights for policy 0, policy_version 1205066 (0.0009) [2023-12-27 00:09:45,424][105692] Updated weights for policy 0, policy_version 1205076 (0.0011) [2023-12-27 00:09:45,480][105692] Updated weights for policy 0, policy_version 1205086 (0.0011) [2023-12-27 00:09:45,798][105620] Updated weights for policy 1, policy_version 1206474 (0.0008) [2023-12-27 00:09:45,856][105620] Updated weights for policy 1, policy_version 1206484 (0.0008) [2023-12-27 00:09:45,901][105620] Updated weights for policy 1, policy_version 1206494 (0.0008) [2023-12-27 00:09:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.9, 300 sec: 18466.7). Total num frames: 617455616. Throughput: 0: 9829.5, 1: 9589.5. Samples: 617425468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:09:46,063][104569] Avg episode reward: [(0, '8911.487'), (1, '9170.640')] [2023-12-27 00:09:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001205088_308551680.pth... [2023-12-27 00:09:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001206496_308903936.pth... [2023-12-27 00:09:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001203936_308256768.pth [2023-12-27 00:09:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001205376_308617216.pth [2023-12-27 00:09:46,173][105692] Updated weights for policy 0, policy_version 1205096 (0.0011) [2023-12-27 00:09:46,233][105692] Updated weights for policy 0, policy_version 1205106 (0.0011) [2023-12-27 00:09:46,292][105692] Updated weights for policy 0, policy_version 1205116 (0.0011) [2023-12-27 00:09:46,711][105620] Updated weights for policy 1, policy_version 1206504 (0.0007) [2023-12-27 00:09:46,777][105620] Updated weights for policy 1, policy_version 1206514 (0.0005) [2023-12-27 00:09:46,833][105620] Updated weights for policy 1, policy_version 1206524 (0.0009) [2023-12-27 00:09:47,045][105692] Updated weights for policy 0, policy_version 1205126 (0.0011) [2023-12-27 00:09:47,103][105692] Updated weights for policy 0, policy_version 1205136 (0.0010) [2023-12-27 00:09:47,158][105692] Updated weights for policy 0, policy_version 1205146 (0.0010) [2023-12-27 00:09:47,459][105620] Updated weights for policy 1, policy_version 1206534 (0.0007) [2023-12-27 00:09:47,520][105620] Updated weights for policy 1, policy_version 1206544 (0.0005) [2023-12-27 00:09:47,582][105620] Updated weights for policy 1, policy_version 1206554 (0.0005) [2023-12-27 00:09:47,758][105692] Updated weights for policy 0, policy_version 1205156 (0.0011) [2023-12-27 00:09:47,820][105692] Updated weights for policy 0, policy_version 1205166 (0.0011) [2023-12-27 00:09:47,881][105692] Updated weights for policy 0, policy_version 1205176 (0.0010) [2023-12-27 00:09:48,103][105620] Updated weights for policy 1, policy_version 1206564 (0.0006) [2023-12-27 00:09:48,158][105620] Updated weights for policy 1, policy_version 1206574 (0.0005) [2023-12-27 00:09:48,221][105620] Updated weights for policy 1, policy_version 1206584 (0.0005) [2023-12-27 00:09:48,537][105692] Updated weights for policy 0, policy_version 1205186 (0.0011) [2023-12-27 00:09:48,594][105692] Updated weights for policy 0, policy_version 1205196 (0.0011) [2023-12-27 00:09:48,660][105692] Updated weights for policy 0, policy_version 1205206 (0.0011) [2023-12-27 00:09:48,716][105692] Updated weights for policy 0, policy_version 1205216 (0.0010) [2023-12-27 00:09:48,753][105620] Updated weights for policy 1, policy_version 1206594 (0.0005) [2023-12-27 00:09:48,823][105620] Updated weights for policy 1, policy_version 1206604 (0.0006) [2023-12-27 00:09:48,877][105620] Updated weights for policy 1, policy_version 1206614 (0.0010) [2023-12-27 00:09:48,940][105620] Updated weights for policy 1, policy_version 1206624 (0.0011) [2023-12-27 00:09:49,347][105692] Updated weights for policy 0, policy_version 1205226 (0.0008) [2023-12-27 00:09:49,413][105692] Updated weights for policy 0, policy_version 1205236 (0.0008) [2023-12-27 00:09:49,466][105692] Updated weights for policy 0, policy_version 1205246 (0.0009) [2023-12-27 00:09:49,662][105620] Updated weights for policy 1, policy_version 1206634 (0.0009) [2023-12-27 00:09:49,729][105620] Updated weights for policy 1, policy_version 1206644 (0.0009) [2023-12-27 00:09:49,791][105620] Updated weights for policy 1, policy_version 1206654 (0.0010) [2023-12-27 00:09:50,200][105692] Updated weights for policy 0, policy_version 1205256 (0.0006) [2023-12-27 00:09:50,251][105692] Updated weights for policy 0, policy_version 1205266 (0.0005) [2023-12-27 00:09:50,311][105692] Updated weights for policy 0, policy_version 1205276 (0.0005) [2023-12-27 00:09:50,609][105620] Updated weights for policy 1, policy_version 1206664 (0.0008) [2023-12-27 00:09:50,673][105620] Updated weights for policy 1, policy_version 1206674 (0.0008) [2023-12-27 00:09:50,730][105620] Updated weights for policy 1, policy_version 1206684 (0.0008) [2023-12-27 00:09:50,989][105692] Updated weights for policy 0, policy_version 1205286 (0.0009) [2023-12-27 00:09:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 18466.7). Total num frames: 617553920. Throughput: 0: 9946.0, 1: 9569.2. Samples: 617546664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:09:51,062][104569] Avg episode reward: [(0, '8815.927'), (1, '9170.612')] [2023-12-27 00:09:51,063][105692] Updated weights for policy 0, policy_version 1205296 (0.0009) [2023-12-27 00:09:51,130][105692] Updated weights for policy 0, policy_version 1205306 (0.0010) [2023-12-27 00:09:51,520][105620] Updated weights for policy 1, policy_version 1206694 (0.0009) [2023-12-27 00:09:51,583][105620] Updated weights for policy 1, policy_version 1206704 (0.0009) [2023-12-27 00:09:51,646][105620] Updated weights for policy 1, policy_version 1206714 (0.0010) [2023-12-27 00:09:51,911][105692] Updated weights for policy 0, policy_version 1205316 (0.0009) [2023-12-27 00:09:51,982][105692] Updated weights for policy 0, policy_version 1205326 (0.0006) [2023-12-27 00:09:52,042][105692] Updated weights for policy 0, policy_version 1205336 (0.0005) [2023-12-27 00:09:52,394][105620] Updated weights for policy 1, policy_version 1206724 (0.0008) [2023-12-27 00:09:52,444][105620] Updated weights for policy 1, policy_version 1206734 (0.0005) [2023-12-27 00:09:52,499][105620] Updated weights for policy 1, policy_version 1206744 (0.0009) [2023-12-27 00:09:52,780][105692] Updated weights for policy 0, policy_version 1205346 (0.0006) [2023-12-27 00:09:52,831][105692] Updated weights for policy 0, policy_version 1205356 (0.0006) [2023-12-27 00:09:52,893][105692] Updated weights for policy 0, policy_version 1205366 (0.0010) [2023-12-27 00:09:52,950][105692] Updated weights for policy 0, policy_version 1205376 (0.0010) [2023-12-27 00:09:53,131][105620] Updated weights for policy 1, policy_version 1206754 (0.0008) [2023-12-27 00:09:53,190][105620] Updated weights for policy 1, policy_version 1206764 (0.0009) [2023-12-27 00:09:53,254][105620] Updated weights for policy 1, policy_version 1206774 (0.0009) [2023-12-27 00:09:53,312][105620] Updated weights for policy 1, policy_version 1206784 (0.0009) [2023-12-27 00:09:53,713][105692] Updated weights for policy 0, policy_version 1205386 (0.0008) [2023-12-27 00:09:53,770][105692] Updated weights for policy 0, policy_version 1205396 (0.0009) [2023-12-27 00:09:53,834][105692] Updated weights for policy 0, policy_version 1205406 (0.0006) [2023-12-27 00:09:53,957][105620] Updated weights for policy 1, policy_version 1206794 (0.0009) [2023-12-27 00:09:54,010][105620] Updated weights for policy 1, policy_version 1206804 (0.0009) [2023-12-27 00:09:54,059][105620] Updated weights for policy 1, policy_version 1206814 (0.0007) [2023-12-27 00:09:54,634][105692] Updated weights for policy 0, policy_version 1205416 (0.0009) [2023-12-27 00:09:54,694][105692] Updated weights for policy 0, policy_version 1205426 (0.0008) [2023-12-27 00:09:54,698][105620] Updated weights for policy 1, policy_version 1206824 (0.0005) [2023-12-27 00:09:54,756][105692] Updated weights for policy 0, policy_version 1205436 (0.0009) [2023-12-27 00:09:54,762][105620] Updated weights for policy 1, policy_version 1206834 (0.0005) [2023-12-27 00:09:54,831][105620] Updated weights for policy 1, policy_version 1206844 (0.0005) [2023-12-27 00:09:55,455][105620] Updated weights for policy 1, policy_version 1206854 (0.0007) [2023-12-27 00:09:55,519][105692] Updated weights for policy 0, policy_version 1205446 (0.0007) [2023-12-27 00:09:55,523][105620] Updated weights for policy 1, policy_version 1206864 (0.0008) [2023-12-27 00:09:55,575][105692] Updated weights for policy 0, policy_version 1205456 (0.0009) [2023-12-27 00:09:55,586][105620] Updated weights for policy 1, policy_version 1206874 (0.0005) [2023-12-27 00:09:55,626][105692] Updated weights for policy 0, policy_version 1205466 (0.0009) [2023-12-27 00:09:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 18522.3). Total num frames: 617652224. Throughput: 0: 9861.4, 1: 9634.6. Samples: 617663852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:09:56,062][104569] Avg episode reward: [(0, '8903.451'), (1, '9349.579')] [2023-12-27 00:09:56,110][105620] Updated weights for policy 1, policy_version 1206884 (0.0007) [2023-12-27 00:09:56,170][105620] Updated weights for policy 1, policy_version 1206894 (0.0011) [2023-12-27 00:09:56,217][105620] Updated weights for policy 1, policy_version 1206904 (0.0010) [2023-12-27 00:09:56,489][105692] Updated weights for policy 0, policy_version 1205476 (0.0009) [2023-12-27 00:09:56,553][105692] Updated weights for policy 0, policy_version 1205486 (0.0007) [2023-12-27 00:09:56,601][105692] Updated weights for policy 0, policy_version 1205496 (0.0010) [2023-12-27 00:09:56,815][105620] Updated weights for policy 1, policy_version 1206914 (0.0006) [2023-12-27 00:09:56,866][105620] Updated weights for policy 1, policy_version 1206924 (0.0010) [2023-12-27 00:09:56,924][105620] Updated weights for policy 1, policy_version 1206934 (0.0010) [2023-12-27 00:09:56,975][105620] Updated weights for policy 1, policy_version 1206944 (0.0010) [2023-12-27 00:09:57,254][105692] Updated weights for policy 0, policy_version 1205506 (0.0010) [2023-12-27 00:09:57,307][105692] Updated weights for policy 0, policy_version 1205516 (0.0007) [2023-12-27 00:09:57,365][105692] Updated weights for policy 0, policy_version 1205526 (0.0009) [2023-12-27 00:09:57,421][105692] Updated weights for policy 0, policy_version 1205536 (0.0009) [2023-12-27 00:09:57,687][105620] Updated weights for policy 1, policy_version 1206954 (0.0005) [2023-12-27 00:09:57,748][105620] Updated weights for policy 1, policy_version 1206964 (0.0008) [2023-12-27 00:09:57,795][105620] Updated weights for policy 1, policy_version 1206974 (0.0006) [2023-12-27 00:09:58,086][105692] Updated weights for policy 0, policy_version 1205546 (0.0006) [2023-12-27 00:09:58,132][105692] Updated weights for policy 0, policy_version 1205556 (0.0005) [2023-12-27 00:09:58,194][105692] Updated weights for policy 0, policy_version 1205566 (0.0008) [2023-12-27 00:09:58,452][105620] Updated weights for policy 1, policy_version 1206984 (0.0009) [2023-12-27 00:09:58,515][105620] Updated weights for policy 1, policy_version 1206994 (0.0009) [2023-12-27 00:09:58,573][105620] Updated weights for policy 1, policy_version 1207004 (0.0009) [2023-12-27 00:09:59,008][105692] Updated weights for policy 0, policy_version 1205576 (0.0010) [2023-12-27 00:09:59,072][105692] Updated weights for policy 0, policy_version 1205586 (0.0009) [2023-12-27 00:09:59,129][105692] Updated weights for policy 0, policy_version 1205596 (0.0008) [2023-12-27 00:09:59,394][105620] Updated weights for policy 1, policy_version 1207014 (0.0009) [2023-12-27 00:09:59,451][105620] Updated weights for policy 1, policy_version 1207024 (0.0010) [2023-12-27 00:09:59,505][105620] Updated weights for policy 1, policy_version 1207034 (0.0010) [2023-12-27 00:09:59,840][105692] Updated weights for policy 0, policy_version 1205606 (0.0009) [2023-12-27 00:09:59,905][105692] Updated weights for policy 0, policy_version 1205616 (0.0011) [2023-12-27 00:09:59,961][105692] Updated weights for policy 0, policy_version 1205626 (0.0006) [2023-12-27 00:10:00,264][105620] Updated weights for policy 1, policy_version 1207044 (0.0008) [2023-12-27 00:10:00,319][105620] Updated weights for policy 1, policy_version 1207054 (0.0008) [2023-12-27 00:10:00,375][105620] Updated weights for policy 1, policy_version 1207064 (0.0008) [2023-12-27 00:10:00,687][105692] Updated weights for policy 0, policy_version 1205636 (0.0009) [2023-12-27 00:10:00,738][105692] Updated weights for policy 0, policy_version 1205646 (0.0010) [2023-12-27 00:10:00,789][105692] Updated weights for policy 0, policy_version 1205656 (0.0010) [2023-12-27 00:10:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 18522.3). Total num frames: 617750528. Throughput: 0: 9890.5, 1: 9629.7. Samples: 617723180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:10:01,063][104569] Avg episode reward: [(0, '9173.286'), (1, '9348.867')] [2023-12-27 00:10:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001205664_308699136.pth... [2023-12-27 00:10:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001207072_309051392.pth... [2023-12-27 00:10:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001205952_308764672.pth [2023-12-27 00:10:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001204544_308412416.pth [2023-12-27 00:10:01,158][105620] Updated weights for policy 1, policy_version 1207074 (0.0008) [2023-12-27 00:10:01,212][105620] Updated weights for policy 1, policy_version 1207084 (0.0009) [2023-12-27 00:10:01,270][105620] Updated weights for policy 1, policy_version 1207094 (0.0007) [2023-12-27 00:10:01,330][105620] Updated weights for policy 1, policy_version 1207104 (0.0008) [2023-12-27 00:10:01,462][105692] Updated weights for policy 0, policy_version 1205666 (0.0010) [2023-12-27 00:10:01,524][105692] Updated weights for policy 0, policy_version 1205676 (0.0011) [2023-12-27 00:10:01,580][105692] Updated weights for policy 0, policy_version 1205686 (0.0010) [2023-12-27 00:10:01,637][105692] Updated weights for policy 0, policy_version 1205696 (0.0011) [2023-12-27 00:10:02,116][105620] Updated weights for policy 1, policy_version 1207114 (0.0008) [2023-12-27 00:10:02,185][105620] Updated weights for policy 1, policy_version 1207124 (0.0009) [2023-12-27 00:10:02,249][105620] Updated weights for policy 1, policy_version 1207134 (0.0008) [2023-12-27 00:10:02,387][105692] Updated weights for policy 0, policy_version 1205706 (0.0011) [2023-12-27 00:10:02,451][105692] Updated weights for policy 0, policy_version 1205716 (0.0011) [2023-12-27 00:10:02,512][105692] Updated weights for policy 0, policy_version 1205726 (0.0007) [2023-12-27 00:10:02,992][105620] Updated weights for policy 1, policy_version 1207144 (0.0006) [2023-12-27 00:10:03,048][105620] Updated weights for policy 1, policy_version 1207154 (0.0005) [2023-12-27 00:10:03,110][105620] Updated weights for policy 1, policy_version 1207164 (0.0005) [2023-12-27 00:10:03,161][105692] Updated weights for policy 0, policy_version 1205736 (0.0009) [2023-12-27 00:10:03,223][105692] Updated weights for policy 0, policy_version 1205746 (0.0010) [2023-12-27 00:10:03,284][105692] Updated weights for policy 0, policy_version 1205756 (0.0010) [2023-12-27 00:10:03,635][105620] Updated weights for policy 1, policy_version 1207174 (0.0005) [2023-12-27 00:10:03,711][105620] Updated weights for policy 1, policy_version 1207184 (0.0006) [2023-12-27 00:10:03,774][105620] Updated weights for policy 1, policy_version 1207194 (0.0005) [2023-12-27 00:10:03,865][105692] Updated weights for policy 0, policy_version 1205766 (0.0009) [2023-12-27 00:10:03,926][105692] Updated weights for policy 0, policy_version 1205776 (0.0006) [2023-12-27 00:10:03,979][105692] Updated weights for policy 0, policy_version 1205786 (0.0007) [2023-12-27 00:10:04,395][105620] Updated weights for policy 1, policy_version 1207204 (0.0007) [2023-12-27 00:10:04,462][105620] Updated weights for policy 1, policy_version 1207214 (0.0011) [2023-12-27 00:10:04,522][105620] Updated weights for policy 1, policy_version 1207224 (0.0011) [2023-12-27 00:10:04,662][105692] Updated weights for policy 0, policy_version 1205796 (0.0008) [2023-12-27 00:10:04,728][105692] Updated weights for policy 0, policy_version 1205806 (0.0007) [2023-12-27 00:10:04,793][105692] Updated weights for policy 0, policy_version 1205816 (0.0007) [2023-12-27 00:10:05,256][105620] Updated weights for policy 1, policy_version 1207234 (0.0009) [2023-12-27 00:10:05,311][105620] Updated weights for policy 1, policy_version 1207244 (0.0010) [2023-12-27 00:10:05,373][105620] Updated weights for policy 1, policy_version 1207254 (0.0008) [2023-12-27 00:10:05,432][105620] Updated weights for policy 1, policy_version 1207264 (0.0005) [2023-12-27 00:10:05,523][105692] Updated weights for policy 0, policy_version 1205826 (0.0007) [2023-12-27 00:10:05,590][105692] Updated weights for policy 0, policy_version 1205836 (0.0011) [2023-12-27 00:10:05,642][105692] Updated weights for policy 0, policy_version 1205846 (0.0010) [2023-12-27 00:10:05,701][105692] Updated weights for policy 0, policy_version 1205856 (0.0011) [2023-12-27 00:10:06,006][105620] Updated weights for policy 1, policy_version 1207274 (0.0005) [2023-12-27 00:10:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 18550.0). Total num frames: 617848832. Throughput: 0: 9804.8, 1: 9678.8. Samples: 617841272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:10:06,062][104569] Avg episode reward: [(0, '9082.146'), (1, '9257.778')] [2023-12-27 00:10:06,071][105620] Updated weights for policy 1, policy_version 1207284 (0.0006) [2023-12-27 00:10:06,127][105620] Updated weights for policy 1, policy_version 1207294 (0.0007) [2023-12-27 00:10:06,431][105692] Updated weights for policy 0, policy_version 1205866 (0.0010) [2023-12-27 00:10:06,487][105692] Updated weights for policy 0, policy_version 1205876 (0.0010) [2023-12-27 00:10:06,577][105692] Updated weights for policy 0, policy_version 1205886 (0.0011) [2023-12-27 00:10:06,777][105620] Updated weights for policy 1, policy_version 1207304 (0.0009) [2023-12-27 00:10:06,838][105620] Updated weights for policy 1, policy_version 1207314 (0.0011) [2023-12-27 00:10:06,890][105620] Updated weights for policy 1, policy_version 1207324 (0.0011) [2023-12-27 00:10:07,271][105692] Updated weights for policy 0, policy_version 1205896 (0.0011) [2023-12-27 00:10:07,338][105692] Updated weights for policy 0, policy_version 1205906 (0.0011) [2023-12-27 00:10:07,405][105692] Updated weights for policy 0, policy_version 1205916 (0.0011) [2023-12-27 00:10:07,594][105620] Updated weights for policy 1, policy_version 1207334 (0.0011) [2023-12-27 00:10:07,650][105620] Updated weights for policy 1, policy_version 1207344 (0.0010) [2023-12-27 00:10:07,712][105620] Updated weights for policy 1, policy_version 1207354 (0.0010) [2023-12-27 00:10:08,097][105692] Updated weights for policy 0, policy_version 1205926 (0.0009) [2023-12-27 00:10:08,151][105692] Updated weights for policy 0, policy_version 1205936 (0.0010) [2023-12-27 00:10:08,202][105692] Updated weights for policy 0, policy_version 1205946 (0.0010) [2023-12-27 00:10:08,411][105620] Updated weights for policy 1, policy_version 1207364 (0.0009) [2023-12-27 00:10:08,476][105620] Updated weights for policy 1, policy_version 1207374 (0.0011) [2023-12-27 00:10:08,532][105620] Updated weights for policy 1, policy_version 1207384 (0.0010) [2023-12-27 00:10:08,911][105692] Updated weights for policy 0, policy_version 1205956 (0.0008) [2023-12-27 00:10:08,969][105692] Updated weights for policy 0, policy_version 1205966 (0.0005) [2023-12-27 00:10:09,031][105692] Updated weights for policy 0, policy_version 1205976 (0.0006) [2023-12-27 00:10:09,080][105620] Updated weights for policy 1, policy_version 1207394 (0.0007) [2023-12-27 00:10:09,134][105620] Updated weights for policy 1, policy_version 1207404 (0.0005) [2023-12-27 00:10:09,192][105620] Updated weights for policy 1, policy_version 1207414 (0.0005) [2023-12-27 00:10:09,255][105620] Updated weights for policy 1, policy_version 1207424 (0.0008) [2023-12-27 00:10:09,769][105692] Updated weights for policy 0, policy_version 1205986 (0.0006) [2023-12-27 00:10:09,837][105692] Updated weights for policy 0, policy_version 1205996 (0.0009) [2023-12-27 00:10:09,897][105692] Updated weights for policy 0, policy_version 1206006 (0.0008) [2023-12-27 00:10:09,965][105692] Updated weights for policy 0, policy_version 1206016 (0.0007) [2023-12-27 00:10:09,975][105620] Updated weights for policy 1, policy_version 1207434 (0.0007) [2023-12-27 00:10:10,033][105620] Updated weights for policy 1, policy_version 1207444 (0.0008) [2023-12-27 00:10:10,092][105620] Updated weights for policy 1, policy_version 1207454 (0.0009) [2023-12-27 00:10:10,735][105692] Updated weights for policy 0, policy_version 1206026 (0.0008) [2023-12-27 00:10:10,779][105692] Updated weights for policy 0, policy_version 1206036 (0.0008) [2023-12-27 00:10:10,831][105692] Updated weights for policy 0, policy_version 1206046 (0.0009) [2023-12-27 00:10:10,858][105620] Updated weights for policy 1, policy_version 1207464 (0.0008) [2023-12-27 00:10:10,915][105620] Updated weights for policy 1, policy_version 1207474 (0.0007) [2023-12-27 00:10:10,970][105620] Updated weights for policy 1, policy_version 1207484 (0.0010) [2023-12-27 00:10:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 18633.3). Total num frames: 617955328. Throughput: 0: 9808.7, 1: 9791.5. Samples: 617959880. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:10:11,063][104569] Avg episode reward: [(0, '8902.078'), (1, '9258.600')] [2023-12-27 00:10:11,610][105692] Updated weights for policy 0, policy_version 1206056 (0.0007) [2023-12-27 00:10:11,684][105692] Updated weights for policy 0, policy_version 1206066 (0.0008) [2023-12-27 00:10:11,704][105620] Updated weights for policy 1, policy_version 1207494 (0.0006) [2023-12-27 00:10:11,753][105692] Updated weights for policy 0, policy_version 1206076 (0.0008) [2023-12-27 00:10:11,764][105620] Updated weights for policy 1, policy_version 1207504 (0.0009) [2023-12-27 00:10:11,813][105620] Updated weights for policy 1, policy_version 1207514 (0.0009) [2023-12-27 00:10:12,439][105692] Updated weights for policy 0, policy_version 1206086 (0.0006) [2023-12-27 00:10:12,502][105692] Updated weights for policy 0, policy_version 1206096 (0.0010) [2023-12-27 00:10:12,562][105692] Updated weights for policy 0, policy_version 1206106 (0.0011) [2023-12-27 00:10:12,581][105620] Updated weights for policy 1, policy_version 1207524 (0.0009) [2023-12-27 00:10:12,635][105620] Updated weights for policy 1, policy_version 1207534 (0.0008) [2023-12-27 00:10:12,717][105620] Updated weights for policy 1, policy_version 1207544 (0.0009) [2023-12-27 00:10:13,298][105692] Updated weights for policy 0, policy_version 1206116 (0.0011) [2023-12-27 00:10:13,354][105692] Updated weights for policy 0, policy_version 1206126 (0.0010) [2023-12-27 00:10:13,376][105620] Updated weights for policy 1, policy_version 1207554 (0.0006) [2023-12-27 00:10:13,402][105692] Updated weights for policy 0, policy_version 1206136 (0.0010) [2023-12-27 00:10:13,432][105620] Updated weights for policy 1, policy_version 1207564 (0.0005) [2023-12-27 00:10:13,476][105620] Updated weights for policy 1, policy_version 1207574 (0.0007) [2023-12-27 00:10:13,530][105620] Updated weights for policy 1, policy_version 1207584 (0.0007) [2023-12-27 00:10:14,063][105692] Updated weights for policy 0, policy_version 1206146 (0.0009) [2023-12-27 00:10:14,124][105692] Updated weights for policy 0, policy_version 1206156 (0.0010) [2023-12-27 00:10:14,173][105692] Updated weights for policy 0, policy_version 1206166 (0.0010) [2023-12-27 00:10:14,230][105692] Updated weights for policy 0, policy_version 1206176 (0.0008) [2023-12-27 00:10:14,248][105620] Updated weights for policy 1, policy_version 1207594 (0.0005) [2023-12-27 00:10:14,307][105620] Updated weights for policy 1, policy_version 1207604 (0.0006) [2023-12-27 00:10:14,367][105620] Updated weights for policy 1, policy_version 1207614 (0.0005) [2023-12-27 00:10:14,916][105620] Updated weights for policy 1, policy_version 1207624 (0.0006) [2023-12-27 00:10:14,933][105692] Updated weights for policy 0, policy_version 1206186 (0.0011) [2023-12-27 00:10:14,978][105620] Updated weights for policy 1, policy_version 1207634 (0.0008) [2023-12-27 00:10:14,996][105692] Updated weights for policy 0, policy_version 1206196 (0.0011) [2023-12-27 00:10:15,038][105620] Updated weights for policy 1, policy_version 1207644 (0.0006) [2023-12-27 00:10:15,063][105692] Updated weights for policy 0, policy_version 1206206 (0.0011) [2023-12-27 00:10:15,686][105620] Updated weights for policy 1, policy_version 1207654 (0.0008) [2023-12-27 00:10:15,748][105620] Updated weights for policy 1, policy_version 1207664 (0.0008) [2023-12-27 00:10:15,798][105692] Updated weights for policy 0, policy_version 1206216 (0.0011) [2023-12-27 00:10:15,801][105620] Updated weights for policy 1, policy_version 1207674 (0.0008) [2023-12-27 00:10:15,862][105692] Updated weights for policy 0, policy_version 1206226 (0.0010) [2023-12-27 00:10:15,920][105692] Updated weights for policy 0, policy_version 1206236 (0.0010) [2023-12-27 00:10:16,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19660.8, 300 sec: 18661.1). Total num frames: 618053632. Throughput: 0: 9700.7, 1: 9754.8. Samples: 618016480. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:10:16,063][104569] Avg episode reward: [(0, '8903.113'), (1, '9259.542')] [2023-12-27 00:10:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001206240_308846592.pth... [2023-12-27 00:10:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001207680_309207040.pth... [2023-12-27 00:10:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001205088_308551680.pth [2023-12-27 00:10:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001206496_308903936.pth [2023-12-27 00:10:16,514][105620] Updated weights for policy 1, policy_version 1207684 (0.0009) [2023-12-27 00:10:16,581][105620] Updated weights for policy 1, policy_version 1207694 (0.0009) [2023-12-27 00:10:16,636][105620] Updated weights for policy 1, policy_version 1207704 (0.0008) [2023-12-27 00:10:16,652][105692] Updated weights for policy 0, policy_version 1206246 (0.0011) [2023-12-27 00:10:16,707][105692] Updated weights for policy 0, policy_version 1206256 (0.0010) [2023-12-27 00:10:16,762][105692] Updated weights for policy 0, policy_version 1206266 (0.0010) [2023-12-27 00:10:17,253][105620] Updated weights for policy 1, policy_version 1207714 (0.0006) [2023-12-27 00:10:17,309][105620] Updated weights for policy 1, policy_version 1207724 (0.0008) [2023-12-27 00:10:17,363][105620] Updated weights for policy 1, policy_version 1207734 (0.0009) [2023-12-27 00:10:17,467][105692] Updated weights for policy 0, policy_version 1206276 (0.0008) [2023-12-27 00:10:17,527][105692] Updated weights for policy 0, policy_version 1206286 (0.0006) [2023-12-27 00:10:17,587][105692] Updated weights for policy 0, policy_version 1206296 (0.0010) [2023-12-27 00:10:18,121][105620] Updated weights for policy 1, policy_version 1207745 (0.0010) [2023-12-27 00:10:18,178][105620] Updated weights for policy 1, policy_version 1207755 (0.0006) [2023-12-27 00:10:18,236][105620] Updated weights for policy 1, policy_version 1207765 (0.0006) [2023-12-27 00:10:18,289][105692] Updated weights for policy 0, policy_version 1206306 (0.0010) [2023-12-27 00:10:18,295][105620] Updated weights for policy 1, policy_version 1207775 (0.0005) [2023-12-27 00:10:18,342][105692] Updated weights for policy 0, policy_version 1206316 (0.0011) [2023-12-27 00:10:18,401][105692] Updated weights for policy 0, policy_version 1206326 (0.0011) [2023-12-27 00:10:18,468][105692] Updated weights for policy 0, policy_version 1206336 (0.0011) [2023-12-27 00:10:18,978][105620] Updated weights for policy 1, policy_version 1207785 (0.0010) [2023-12-27 00:10:19,039][105620] Updated weights for policy 1, policy_version 1207795 (0.0010) [2023-12-27 00:10:19,103][105620] Updated weights for policy 1, policy_version 1207805 (0.0010) [2023-12-27 00:10:19,205][105692] Updated weights for policy 0, policy_version 1206346 (0.0011) [2023-12-27 00:10:19,270][105692] Updated weights for policy 0, policy_version 1206356 (0.0011) [2023-12-27 00:10:19,331][105692] Updated weights for policy 0, policy_version 1206366 (0.0011) [2023-12-27 00:10:19,833][105620] Updated weights for policy 1, policy_version 1207815 (0.0009) [2023-12-27 00:10:19,892][105620] Updated weights for policy 1, policy_version 1207825 (0.0010) [2023-12-27 00:10:19,958][105620] Updated weights for policy 1, policy_version 1207835 (0.0007) [2023-12-27 00:10:20,046][105692] Updated weights for policy 0, policy_version 1206376 (0.0007) [2023-12-27 00:10:20,115][105692] Updated weights for policy 0, policy_version 1206386 (0.0006) [2023-12-27 00:10:20,182][105692] Updated weights for policy 0, policy_version 1206396 (0.0007) [2023-12-27 00:10:20,709][105620] Updated weights for policy 1, policy_version 1207845 (0.0007) [2023-12-27 00:10:20,771][105620] Updated weights for policy 1, policy_version 1207855 (0.0010) [2023-12-27 00:10:20,831][105620] Updated weights for policy 1, policy_version 1207865 (0.0009) [2023-12-27 00:10:20,860][105692] Updated weights for policy 0, policy_version 1206406 (0.0007) [2023-12-27 00:10:20,910][105692] Updated weights for policy 0, policy_version 1206416 (0.0008) [2023-12-27 00:10:20,973][105692] Updated weights for policy 0, policy_version 1206426 (0.0007) [2023-12-27 00:10:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 18688.9). Total num frames: 618151936. Throughput: 0: 9687.0, 1: 9889.5. Samples: 618135964. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:10:21,063][104569] Avg episode reward: [(0, '9174.848'), (1, '9168.359')] [2023-12-27 00:10:21,666][105620] Updated weights for policy 1, policy_version 1207875 (0.0007) [2023-12-27 00:10:21,722][105692] Updated weights for policy 0, policy_version 1206436 (0.0007) [2023-12-27 00:10:21,735][105620] Updated weights for policy 1, policy_version 1207885 (0.0008) [2023-12-27 00:10:21,781][105692] Updated weights for policy 0, policy_version 1206446 (0.0008) [2023-12-27 00:10:21,791][105620] Updated weights for policy 1, policy_version 1207895 (0.0006) [2023-12-27 00:10:21,836][105692] Updated weights for policy 0, policy_version 1206456 (0.0008) [2023-12-27 00:10:22,430][105620] Updated weights for policy 1, policy_version 1207905 (0.0008) [2023-12-27 00:10:22,500][105620] Updated weights for policy 1, policy_version 1207915 (0.0009) [2023-12-27 00:10:22,563][105620] Updated weights for policy 1, policy_version 1207925 (0.0008) [2023-12-27 00:10:22,570][105692] Updated weights for policy 0, policy_version 1206466 (0.0009) [2023-12-27 00:10:22,623][105620] Updated weights for policy 1, policy_version 1207935 (0.0007) [2023-12-27 00:10:22,628][105692] Updated weights for policy 0, policy_version 1206476 (0.0009) [2023-12-27 00:10:22,699][105692] Updated weights for policy 0, policy_version 1206486 (0.0007) [2023-12-27 00:10:22,770][105692] Updated weights for policy 0, policy_version 1206496 (0.0008) [2023-12-27 00:10:23,298][105620] Updated weights for policy 1, policy_version 1207945 (0.0007) [2023-12-27 00:10:23,356][105620] Updated weights for policy 1, policy_version 1207955 (0.0008) [2023-12-27 00:10:23,417][105620] Updated weights for policy 1, policy_version 1207965 (0.0008) [2023-12-27 00:10:23,466][105692] Updated weights for policy 0, policy_version 1206506 (0.0005) [2023-12-27 00:10:23,510][105692] Updated weights for policy 0, policy_version 1206516 (0.0005) [2023-12-27 00:10:23,555][105692] Updated weights for policy 0, policy_version 1206526 (0.0005) [2023-12-27 00:10:24,168][105620] Updated weights for policy 1, policy_version 1207975 (0.0008) [2023-12-27 00:10:24,231][105620] Updated weights for policy 1, policy_version 1207985 (0.0008) [2023-12-27 00:10:24,278][105692] Updated weights for policy 0, policy_version 1206536 (0.0009) [2023-12-27 00:10:24,293][105620] Updated weights for policy 1, policy_version 1207995 (0.0009) [2023-12-27 00:10:24,332][105692] Updated weights for policy 0, policy_version 1206546 (0.0008) [2023-12-27 00:10:24,378][105692] Updated weights for policy 0, policy_version 1206556 (0.0005) [2023-12-27 00:10:24,981][105620] Updated weights for policy 1, policy_version 1208005 (0.0006) [2023-12-27 00:10:25,031][105620] Updated weights for policy 1, policy_version 1208015 (0.0007) [2023-12-27 00:10:25,075][105620] Updated weights for policy 1, policy_version 1208025 (0.0008) [2023-12-27 00:10:25,104][105692] Updated weights for policy 0, policy_version 1206566 (0.0008) [2023-12-27 00:10:25,161][105692] Updated weights for policy 0, policy_version 1206576 (0.0006) [2023-12-27 00:10:25,220][105692] Updated weights for policy 0, policy_version 1206586 (0.0005) [2023-12-27 00:10:25,776][105620] Updated weights for policy 1, policy_version 1208035 (0.0007) [2023-12-27 00:10:25,828][105620] Updated weights for policy 1, policy_version 1208045 (0.0008) [2023-12-27 00:10:25,842][105692] Updated weights for policy 0, policy_version 1206596 (0.0006) [2023-12-27 00:10:25,884][105620] Updated weights for policy 1, policy_version 1208055 (0.0008) [2023-12-27 00:10:25,898][105692] Updated weights for policy 0, policy_version 1206606 (0.0006) [2023-12-27 00:10:25,959][105692] Updated weights for policy 0, policy_version 1206616 (0.0007) [2023-12-27 00:10:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.9, 300 sec: 18688.9). Total num frames: 618250240. Throughput: 0: 9733.5, 1: 9914.7. Samples: 618252492. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:10:26,063][104569] Avg episode reward: [(0, '8903.638'), (1, '9259.868')] [2023-12-27 00:10:26,535][105620] Updated weights for policy 1, policy_version 1208065 (0.0008) [2023-12-27 00:10:26,590][105620] Updated weights for policy 1, policy_version 1208075 (0.0009) [2023-12-27 00:10:26,641][105620] Updated weights for policy 1, policy_version 1208085 (0.0010) [2023-12-27 00:10:26,699][105620] Updated weights for policy 1, policy_version 1208095 (0.0010) [2023-12-27 00:10:26,739][105692] Updated weights for policy 0, policy_version 1206626 (0.0008) [2023-12-27 00:10:26,809][105692] Updated weights for policy 0, policy_version 1206636 (0.0007) [2023-12-27 00:10:26,861][105692] Updated weights for policy 0, policy_version 1206646 (0.0005) [2023-12-27 00:10:26,915][105692] Updated weights for policy 0, policy_version 1206656 (0.0006) [2023-12-27 00:10:27,431][105620] Updated weights for policy 1, policy_version 1208105 (0.0010) [2023-12-27 00:10:27,490][105620] Updated weights for policy 1, policy_version 1208115 (0.0011) [2023-12-27 00:10:27,552][105620] Updated weights for policy 1, policy_version 1208125 (0.0011) [2023-12-27 00:10:27,614][105692] Updated weights for policy 0, policy_version 1206666 (0.0007) [2023-12-27 00:10:27,667][105692] Updated weights for policy 0, policy_version 1206676 (0.0009) [2023-12-27 00:10:27,734][105692] Updated weights for policy 0, policy_version 1206686 (0.0008) [2023-12-27 00:10:28,240][105620] Updated weights for policy 1, policy_version 1208135 (0.0010) [2023-12-27 00:10:28,304][105620] Updated weights for policy 1, policy_version 1208145 (0.0008) [2023-12-27 00:10:28,374][105620] Updated weights for policy 1, policy_version 1208155 (0.0008) [2023-12-27 00:10:28,419][105692] Updated weights for policy 0, policy_version 1206696 (0.0010) [2023-12-27 00:10:28,478][105692] Updated weights for policy 0, policy_version 1206706 (0.0009) [2023-12-27 00:10:28,543][105692] Updated weights for policy 0, policy_version 1206716 (0.0007) [2023-12-27 00:10:28,978][105620] Updated weights for policy 1, policy_version 1208165 (0.0008) [2023-12-27 00:10:29,039][105620] Updated weights for policy 1, policy_version 1208175 (0.0007) [2023-12-27 00:10:29,103][105620] Updated weights for policy 1, policy_version 1208185 (0.0005) [2023-12-27 00:10:29,314][105692] Updated weights for policy 0, policy_version 1206726 (0.0008) [2023-12-27 00:10:29,377][105692] Updated weights for policy 0, policy_version 1206736 (0.0008) [2023-12-27 00:10:29,431][105692] Updated weights for policy 0, policy_version 1206746 (0.0009) [2023-12-27 00:10:29,757][105620] Updated weights for policy 1, policy_version 1208195 (0.0006) [2023-12-27 00:10:29,809][105620] Updated weights for policy 1, policy_version 1208205 (0.0008) [2023-12-27 00:10:29,870][105620] Updated weights for policy 1, policy_version 1208215 (0.0008) [2023-12-27 00:10:30,156][105692] Updated weights for policy 0, policy_version 1206756 (0.0008) [2023-12-27 00:10:30,214][105692] Updated weights for policy 0, policy_version 1206766 (0.0005) [2023-12-27 00:10:30,271][105692] Updated weights for policy 0, policy_version 1206776 (0.0005) [2023-12-27 00:10:30,727][105620] Updated weights for policy 1, policy_version 1208225 (0.0009) [2023-12-27 00:10:30,775][105620] Updated weights for policy 1, policy_version 1208235 (0.0005) [2023-12-27 00:10:30,831][105620] Updated weights for policy 1, policy_version 1208245 (0.0005) [2023-12-27 00:10:30,868][105692] Updated weights for policy 0, policy_version 1206786 (0.0008) [2023-12-27 00:10:30,881][105620] Updated weights for policy 1, policy_version 1208255 (0.0005) [2023-12-27 00:10:30,920][105692] Updated weights for policy 0, policy_version 1206796 (0.0005) [2023-12-27 00:10:30,969][105692] Updated weights for policy 0, policy_version 1206806 (0.0005) [2023-12-27 00:10:31,019][105692] Updated weights for policy 0, policy_version 1206816 (0.0008) [2023-12-27 00:10:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 18744.4). Total num frames: 618348544. Throughput: 0: 9729.4, 1: 9994.5. Samples: 618313044. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:10:31,063][104569] Avg episode reward: [(0, '8995.055'), (1, '9260.429')] [2023-12-27 00:10:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001208256_309354496.pth... [2023-12-27 00:10:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001206816_308994048.pth... [2023-12-27 00:10:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001207072_309051392.pth [2023-12-27 00:10:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001205664_308699136.pth [2023-12-27 00:10:31,593][105620] Updated weights for policy 1, policy_version 1208265 (0.0009) [2023-12-27 00:10:31,657][105620] Updated weights for policy 1, policy_version 1208275 (0.0007) [2023-12-27 00:10:31,725][105620] Updated weights for policy 1, policy_version 1208285 (0.0008) [2023-12-27 00:10:31,791][105692] Updated weights for policy 0, policy_version 1206826 (0.0006) [2023-12-27 00:10:31,851][105692] Updated weights for policy 0, policy_version 1206836 (0.0009) [2023-12-27 00:10:31,910][105692] Updated weights for policy 0, policy_version 1206846 (0.0010) [2023-12-27 00:10:32,485][105620] Updated weights for policy 1, policy_version 1208295 (0.0010) [2023-12-27 00:10:32,527][105692] Updated weights for policy 0, policy_version 1206856 (0.0010) [2023-12-27 00:10:32,540][105620] Updated weights for policy 1, policy_version 1208305 (0.0010) [2023-12-27 00:10:32,580][105692] Updated weights for policy 0, policy_version 1206866 (0.0011) [2023-12-27 00:10:32,602][105620] Updated weights for policy 1, policy_version 1208315 (0.0011) [2023-12-27 00:10:32,639][105692] Updated weights for policy 0, policy_version 1206876 (0.0011) [2023-12-27 00:10:33,245][105620] Updated weights for policy 1, policy_version 1208325 (0.0010) [2023-12-27 00:10:33,296][105620] Updated weights for policy 1, policy_version 1208335 (0.0010) [2023-12-27 00:10:33,353][105620] Updated weights for policy 1, policy_version 1208345 (0.0010) [2023-12-27 00:10:33,390][105692] Updated weights for policy 0, policy_version 1206886 (0.0009) [2023-12-27 00:10:33,448][105692] Updated weights for policy 0, policy_version 1206896 (0.0010) [2023-12-27 00:10:33,499][105692] Updated weights for policy 0, policy_version 1206906 (0.0010) [2023-12-27 00:10:34,060][105620] Updated weights for policy 1, policy_version 1208355 (0.0010) [2023-12-27 00:10:34,114][105620] Updated weights for policy 1, policy_version 1208365 (0.0010) [2023-12-27 00:10:34,173][105620] Updated weights for policy 1, policy_version 1208375 (0.0010) [2023-12-27 00:10:34,184][105692] Updated weights for policy 0, policy_version 1206916 (0.0009) [2023-12-27 00:10:34,244][105692] Updated weights for policy 0, policy_version 1206926 (0.0007) [2023-12-27 00:10:34,301][105692] Updated weights for policy 0, policy_version 1206936 (0.0009) [2023-12-27 00:10:34,911][105620] Updated weights for policy 1, policy_version 1208385 (0.0010) [2023-12-27 00:10:34,970][105620] Updated weights for policy 1, policy_version 1208395 (0.0005) [2023-12-27 00:10:35,027][105620] Updated weights for policy 1, policy_version 1208405 (0.0006) [2023-12-27 00:10:35,092][105620] Updated weights for policy 1, policy_version 1208415 (0.0010) [2023-12-27 00:10:35,100][105692] Updated weights for policy 0, policy_version 1206946 (0.0009) [2023-12-27 00:10:35,160][105692] Updated weights for policy 0, policy_version 1206956 (0.0008) [2023-12-27 00:10:35,223][105692] Updated weights for policy 0, policy_version 1206966 (0.0008) [2023-12-27 00:10:35,282][105692] Updated weights for policy 0, policy_version 1206976 (0.0008) [2023-12-27 00:10:35,728][105620] Updated weights for policy 1, policy_version 1208425 (0.0007) [2023-12-27 00:10:35,785][105620] Updated weights for policy 1, policy_version 1208435 (0.0010) [2023-12-27 00:10:35,842][105620] Updated weights for policy 1, policy_version 1208445 (0.0010) [2023-12-27 00:10:36,039][105692] Updated weights for policy 0, policy_version 1206986 (0.0006) [2023-12-27 00:10:36,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 18744.4). Total num frames: 618438656. Throughput: 0: 9675.4, 1: 9948.2. Samples: 618429728. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:10:36,063][104569] Avg episode reward: [(0, '9176.787'), (1, '9169.206')] [2023-12-27 00:10:36,093][105692] Updated weights for policy 0, policy_version 1206996 (0.0005) [2023-12-27 00:10:36,161][105692] Updated weights for policy 0, policy_version 1207006 (0.0008) [2023-12-27 00:10:36,566][105620] Updated weights for policy 1, policy_version 1208455 (0.0011) [2023-12-27 00:10:36,622][105620] Updated weights for policy 1, policy_version 1208465 (0.0011) [2023-12-27 00:10:36,673][105620] Updated weights for policy 1, policy_version 1208475 (0.0010) [2023-12-27 00:10:36,879][105692] Updated weights for policy 0, policy_version 1207016 (0.0008) [2023-12-27 00:10:36,931][105692] Updated weights for policy 0, policy_version 1207026 (0.0008) [2023-12-27 00:10:36,985][105692] Updated weights for policy 0, policy_version 1207036 (0.0005) [2023-12-27 00:10:37,435][105620] Updated weights for policy 1, policy_version 1208485 (0.0010) [2023-12-27 00:10:37,497][105620] Updated weights for policy 1, policy_version 1208495 (0.0010) [2023-12-27 00:10:37,563][105620] Updated weights for policy 1, policy_version 1208505 (0.0010) [2023-12-27 00:10:37,620][105692] Updated weights for policy 0, policy_version 1207046 (0.0007) [2023-12-27 00:10:37,666][105692] Updated weights for policy 0, policy_version 1207056 (0.0008) [2023-12-27 00:10:37,719][105692] Updated weights for policy 0, policy_version 1207066 (0.0008) [2023-12-27 00:10:38,196][105620] Updated weights for policy 1, policy_version 1208515 (0.0010) [2023-12-27 00:10:38,244][105620] Updated weights for policy 1, policy_version 1208525 (0.0010) [2023-12-27 00:10:38,288][105620] Updated weights for policy 1, policy_version 1208535 (0.0010) [2023-12-27 00:10:38,563][105692] Updated weights for policy 0, policy_version 1207076 (0.0008) [2023-12-27 00:10:38,626][105692] Updated weights for policy 0, policy_version 1207086 (0.0006) [2023-12-27 00:10:38,680][105692] Updated weights for policy 0, policy_version 1207096 (0.0005) [2023-12-27 00:10:39,083][105620] Updated weights for policy 1, policy_version 1208545 (0.0010) [2023-12-27 00:10:39,139][105620] Updated weights for policy 1, policy_version 1208555 (0.0009) [2023-12-27 00:10:39,209][105620] Updated weights for policy 1, policy_version 1208565 (0.0010) [2023-12-27 00:10:39,276][105620] Updated weights for policy 1, policy_version 1208575 (0.0008) [2023-12-27 00:10:39,295][105692] Updated weights for policy 0, policy_version 1207106 (0.0005) [2023-12-27 00:10:39,362][105692] Updated weights for policy 0, policy_version 1207116 (0.0008) [2023-12-27 00:10:39,427][105692] Updated weights for policy 0, policy_version 1207126 (0.0007) [2023-12-27 00:10:39,498][105692] Updated weights for policy 0, policy_version 1207136 (0.0008) [2023-12-27 00:10:39,978][105620] Updated weights for policy 1, policy_version 1208585 (0.0009) [2023-12-27 00:10:40,029][105620] Updated weights for policy 1, policy_version 1208595 (0.0009) [2023-12-27 00:10:40,084][105620] Updated weights for policy 1, policy_version 1208605 (0.0009) [2023-12-27 00:10:40,258][105692] Updated weights for policy 0, policy_version 1207146 (0.0007) [2023-12-27 00:10:40,312][105692] Updated weights for policy 0, policy_version 1207156 (0.0005) [2023-12-27 00:10:40,370][105692] Updated weights for policy 0, policy_version 1207166 (0.0007) [2023-12-27 00:10:40,793][105620] Updated weights for policy 1, policy_version 1208615 (0.0007) [2023-12-27 00:10:40,843][105620] Updated weights for policy 1, policy_version 1208625 (0.0006) [2023-12-27 00:10:40,902][105620] Updated weights for policy 1, policy_version 1208635 (0.0008) [2023-12-27 00:10:41,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19660.8, 300 sec: 18772.2). Total num frames: 618536960. Throughput: 0: 9706.2, 1: 9897.8. Samples: 618546028. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:10:41,062][104569] Avg episode reward: [(0, '9176.089'), (1, '9259.717')] [2023-12-27 00:10:41,115][105692] Updated weights for policy 0, policy_version 1207176 (0.0010) [2023-12-27 00:10:41,186][105692] Updated weights for policy 0, policy_version 1207186 (0.0007) [2023-12-27 00:10:41,257][105692] Updated weights for policy 0, policy_version 1207196 (0.0008) [2023-12-27 00:10:41,606][105620] Updated weights for policy 1, policy_version 1208645 (0.0007) [2023-12-27 00:10:41,674][105620] Updated weights for policy 1, policy_version 1208655 (0.0009) [2023-12-27 00:10:41,737][105620] Updated weights for policy 1, policy_version 1208665 (0.0008) [2023-12-27 00:10:41,938][105692] Updated weights for policy 0, policy_version 1207206 (0.0009) [2023-12-27 00:10:42,001][105692] Updated weights for policy 0, policy_version 1207216 (0.0011) [2023-12-27 00:10:42,056][105692] Updated weights for policy 0, policy_version 1207226 (0.0011) [2023-12-27 00:10:42,522][105620] Updated weights for policy 1, policy_version 1208675 (0.0008) [2023-12-27 00:10:42,584][105620] Updated weights for policy 1, policy_version 1208685 (0.0008) [2023-12-27 00:10:42,646][105620] Updated weights for policy 1, policy_version 1208695 (0.0009) [2023-12-27 00:10:42,802][105692] Updated weights for policy 0, policy_version 1207236 (0.0009) [2023-12-27 00:10:42,864][105692] Updated weights for policy 0, policy_version 1207246 (0.0008) [2023-12-27 00:10:42,922][105692] Updated weights for policy 0, policy_version 1207256 (0.0007) [2023-12-27 00:10:43,475][105620] Updated weights for policy 1, policy_version 1208705 (0.0009) [2023-12-27 00:10:43,501][105692] Updated weights for policy 0, policy_version 1207266 (0.0007) [2023-12-27 00:10:43,538][105620] Updated weights for policy 1, policy_version 1208715 (0.0009) [2023-12-27 00:10:43,551][105692] Updated weights for policy 0, policy_version 1207276 (0.0005) [2023-12-27 00:10:43,601][105692] Updated weights for policy 0, policy_version 1207286 (0.0005) [2023-12-27 00:10:43,606][105620] Updated weights for policy 1, policy_version 1208725 (0.0009) [2023-12-27 00:10:43,658][105692] Updated weights for policy 0, policy_version 1207296 (0.0005) [2023-12-27 00:10:43,670][105620] Updated weights for policy 1, policy_version 1208735 (0.0009) [2023-12-27 00:10:44,195][105692] Updated weights for policy 0, policy_version 1207306 (0.0006) [2023-12-27 00:10:44,256][105692] Updated weights for policy 0, policy_version 1207316 (0.0007) [2023-12-27 00:10:44,308][105692] Updated weights for policy 0, policy_version 1207326 (0.0005) [2023-12-27 00:10:44,494][105620] Updated weights for policy 1, policy_version 1208745 (0.0010) [2023-12-27 00:10:44,550][105620] Updated weights for policy 1, policy_version 1208755 (0.0009) [2023-12-27 00:10:44,613][105620] Updated weights for policy 1, policy_version 1208765 (0.0008) [2023-12-27 00:10:44,915][105692] Updated weights for policy 0, policy_version 1207336 (0.0007) [2023-12-27 00:10:44,982][105692] Updated weights for policy 0, policy_version 1207346 (0.0007) [2023-12-27 00:10:45,038][105692] Updated weights for policy 0, policy_version 1207356 (0.0011) [2023-12-27 00:10:45,475][105620] Updated weights for policy 1, policy_version 1208775 (0.0008) [2023-12-27 00:10:45,524][105620] Updated weights for policy 1, policy_version 1208785 (0.0008) [2023-12-27 00:10:45,592][105620] Updated weights for policy 1, policy_version 1208795 (0.0008) [2023-12-27 00:10:45,663][105692] Updated weights for policy 0, policy_version 1207366 (0.0007) [2023-12-27 00:10:45,727][105692] Updated weights for policy 0, policy_version 1207376 (0.0008) [2023-12-27 00:10:45,786][105692] Updated weights for policy 0, policy_version 1207386 (0.0011) [2023-12-27 00:10:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 18799.9). Total num frames: 618635264. Throughput: 0: 9736.1, 1: 9805.0. Samples: 618602532. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:10:46,063][104569] Avg episode reward: [(0, '9177.987'), (1, '9350.468')] [2023-12-27 00:10:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001207392_309141504.pth... [2023-12-27 00:10:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001208800_309493760.pth... [2023-12-27 00:10:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001206240_308846592.pth [2023-12-27 00:10:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001207680_309207040.pth [2023-12-27 00:10:46,342][105692] Updated weights for policy 0, policy_version 1207396 (0.0010) [2023-12-27 00:10:46,399][105692] Updated weights for policy 0, policy_version 1207406 (0.0009) [2023-12-27 00:10:46,433][105620] Updated weights for policy 1, policy_version 1208805 (0.0009) [2023-12-27 00:10:46,462][105692] Updated weights for policy 0, policy_version 1207416 (0.0007) [2023-12-27 00:10:46,488][105620] Updated weights for policy 1, policy_version 1208815 (0.0008) [2023-12-27 00:10:46,548][105620] Updated weights for policy 1, policy_version 1208825 (0.0009) [2023-12-27 00:10:47,090][105692] Updated weights for policy 0, policy_version 1207426 (0.0007) [2023-12-27 00:10:47,140][105692] Updated weights for policy 0, policy_version 1207436 (0.0009) [2023-12-27 00:10:47,194][105692] Updated weights for policy 0, policy_version 1207446 (0.0009) [2023-12-27 00:10:47,246][105692] Updated weights for policy 0, policy_version 1207456 (0.0009) [2023-12-27 00:10:47,333][105620] Updated weights for policy 1, policy_version 1208835 (0.0009) [2023-12-27 00:10:47,388][105620] Updated weights for policy 1, policy_version 1208845 (0.0009) [2023-12-27 00:10:47,456][105620] Updated weights for policy 1, policy_version 1208855 (0.0009) [2023-12-27 00:10:48,004][105692] Updated weights for policy 0, policy_version 1207466 (0.0009) [2023-12-27 00:10:48,056][105692] Updated weights for policy 0, policy_version 1207476 (0.0009) [2023-12-27 00:10:48,122][105692] Updated weights for policy 0, policy_version 1207486 (0.0006) [2023-12-27 00:10:48,196][105620] Updated weights for policy 1, policy_version 1208865 (0.0009) [2023-12-27 00:10:48,246][105620] Updated weights for policy 1, policy_version 1208876 (0.0009) [2023-12-27 00:10:48,292][105620] Updated weights for policy 1, policy_version 1208886 (0.0008) [2023-12-27 00:10:48,342][105620] Updated weights for policy 1, policy_version 1208896 (0.0008) [2023-12-27 00:10:48,736][105692] Updated weights for policy 0, policy_version 1207496 (0.0010) [2023-12-27 00:10:48,785][105692] Updated weights for policy 0, policy_version 1207506 (0.0010) [2023-12-27 00:10:48,839][105692] Updated weights for policy 0, policy_version 1207516 (0.0009) [2023-12-27 00:10:49,263][105620] Updated weights for policy 1, policy_version 1208906 (0.0008) [2023-12-27 00:10:49,327][105620] Updated weights for policy 1, policy_version 1208916 (0.0008) [2023-12-27 00:10:49,399][105620] Updated weights for policy 1, policy_version 1208926 (0.0008) [2023-12-27 00:10:49,535][105692] Updated weights for policy 0, policy_version 1207526 (0.0010) [2023-12-27 00:10:49,591][105692] Updated weights for policy 0, policy_version 1207536 (0.0010) [2023-12-27 00:10:49,644][105692] Updated weights for policy 0, policy_version 1207547 (0.0010) [2023-12-27 00:10:50,070][105620] Updated weights for policy 1, policy_version 1208936 (0.0010) [2023-12-27 00:10:50,126][105620] Updated weights for policy 1, policy_version 1208946 (0.0009) [2023-12-27 00:10:50,185][105620] Updated weights for policy 1, policy_version 1208956 (0.0010) [2023-12-27 00:10:50,402][105692] Updated weights for policy 0, policy_version 1207557 (0.0010) [2023-12-27 00:10:50,461][105692] Updated weights for policy 0, policy_version 1207567 (0.0009) [2023-12-27 00:10:50,513][105692] Updated weights for policy 0, policy_version 1207577 (0.0009) [2023-12-27 00:10:51,017][105620] Updated weights for policy 1, policy_version 1208966 (0.0009) [2023-12-27 00:10:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 18799.9). Total num frames: 618725376. Throughput: 0: 9853.2, 1: 9688.9. Samples: 618720668. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:10:51,062][104569] Avg episode reward: [(0, '9179.291'), (1, '9349.959')] [2023-12-27 00:10:51,079][105620] Updated weights for policy 1, policy_version 1208976 (0.0008) [2023-12-27 00:10:51,130][105620] Updated weights for policy 1, policy_version 1208986 (0.0007) [2023-12-27 00:10:51,216][105692] Updated weights for policy 0, policy_version 1207587 (0.0010) [2023-12-27 00:10:51,272][105692] Updated weights for policy 0, policy_version 1207597 (0.0008) [2023-12-27 00:10:51,319][105692] Updated weights for policy 0, policy_version 1207607 (0.0006) [2023-12-27 00:10:51,940][105620] Updated weights for policy 1, policy_version 1208996 (0.0008) [2023-12-27 00:10:52,003][105620] Updated weights for policy 1, policy_version 1209006 (0.0008) [2023-12-27 00:10:52,059][105620] Updated weights for policy 1, policy_version 1209016 (0.0008) [2023-12-27 00:10:52,095][105692] Updated weights for policy 0, policy_version 1207617 (0.0011) [2023-12-27 00:10:52,147][105692] Updated weights for policy 0, policy_version 1207627 (0.0007) [2023-12-27 00:10:52,206][105692] Updated weights for policy 0, policy_version 1207637 (0.0009) [2023-12-27 00:10:52,268][105692] Updated weights for policy 0, policy_version 1207647 (0.0009) [2023-12-27 00:10:52,686][105620] Updated weights for policy 1, policy_version 1209026 (0.0008) [2023-12-27 00:10:52,750][105620] Updated weights for policy 1, policy_version 1209036 (0.0007) [2023-12-27 00:10:52,807][105620] Updated weights for policy 1, policy_version 1209046 (0.0010) [2023-12-27 00:10:52,864][105620] Updated weights for policy 1, policy_version 1209056 (0.0010) [2023-12-27 00:10:52,932][105692] Updated weights for policy 0, policy_version 1207657 (0.0008) [2023-12-27 00:10:52,988][105692] Updated weights for policy 0, policy_version 1207667 (0.0008) [2023-12-27 00:10:53,055][105692] Updated weights for policy 0, policy_version 1207677 (0.0009) [2023-12-27 00:10:53,601][105620] Updated weights for policy 1, policy_version 1209066 (0.0005) [2023-12-27 00:10:53,647][105620] Updated weights for policy 1, policy_version 1209076 (0.0005) [2023-12-27 00:10:53,698][105620] Updated weights for policy 1, policy_version 1209086 (0.0005) [2023-12-27 00:10:53,756][105692] Updated weights for policy 0, policy_version 1207687 (0.0010) [2023-12-27 00:10:53,801][105692] Updated weights for policy 0, policy_version 1207697 (0.0010) [2023-12-27 00:10:53,849][105692] Updated weights for policy 0, policy_version 1207707 (0.0010) [2023-12-27 00:10:54,247][105620] Updated weights for policy 1, policy_version 1209096 (0.0007) [2023-12-27 00:10:54,292][105620] Updated weights for policy 1, policy_version 1209106 (0.0008) [2023-12-27 00:10:54,341][105620] Updated weights for policy 1, policy_version 1209116 (0.0008) [2023-12-27 00:10:54,622][105692] Updated weights for policy 0, policy_version 1207717 (0.0011) [2023-12-27 00:10:54,671][105692] Updated weights for policy 0, policy_version 1207727 (0.0010) [2023-12-27 00:10:54,727][105692] Updated weights for policy 0, policy_version 1207737 (0.0011) [2023-12-27 00:10:55,007][105620] Updated weights for policy 1, policy_version 1209126 (0.0006) [2023-12-27 00:10:55,063][105620] Updated weights for policy 1, policy_version 1209136 (0.0006) [2023-12-27 00:10:55,118][105620] Updated weights for policy 1, policy_version 1209146 (0.0006) [2023-12-27 00:10:55,456][105692] Updated weights for policy 0, policy_version 1207747 (0.0010) [2023-12-27 00:10:55,503][105692] Updated weights for policy 0, policy_version 1207757 (0.0010) [2023-12-27 00:10:55,553][105692] Updated weights for policy 0, policy_version 1207767 (0.0010) [2023-12-27 00:10:55,796][105620] Updated weights for policy 1, policy_version 1209156 (0.0007) [2023-12-27 00:10:55,864][105620] Updated weights for policy 1, policy_version 1209166 (0.0006) [2023-12-27 00:10:55,931][105620] Updated weights for policy 1, policy_version 1209176 (0.0005) [2023-12-27 00:10:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 18855.5). Total num frames: 618831872. Throughput: 0: 9871.6, 1: 9648.2. Samples: 618838272. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:10:56,062][104569] Avg episode reward: [(0, '9179.322'), (1, '9349.107')] [2023-12-27 00:10:56,155][105692] Updated weights for policy 0, policy_version 1207777 (0.0006) [2023-12-27 00:10:56,219][105692] Updated weights for policy 0, policy_version 1207787 (0.0005) [2023-12-27 00:10:56,288][105692] Updated weights for policy 0, policy_version 1207797 (0.0005) [2023-12-27 00:10:56,353][105692] Updated weights for policy 0, policy_version 1207807 (0.0005) [2023-12-27 00:10:56,501][105620] Updated weights for policy 1, policy_version 1209186 (0.0005) [2023-12-27 00:10:56,551][105620] Updated weights for policy 1, policy_version 1209196 (0.0009) [2023-12-27 00:10:56,603][105620] Updated weights for policy 1, policy_version 1209207 (0.0009) [2023-12-27 00:10:56,894][105692] Updated weights for policy 0, policy_version 1207817 (0.0008) [2023-12-27 00:10:56,954][105692] Updated weights for policy 0, policy_version 1207827 (0.0007) [2023-12-27 00:10:57,011][105692] Updated weights for policy 0, policy_version 1207837 (0.0005) [2023-12-27 00:10:57,404][105620] Updated weights for policy 1, policy_version 1209218 (0.0010) [2023-12-27 00:10:57,462][105620] Updated weights for policy 1, policy_version 1209228 (0.0008) [2023-12-27 00:10:57,520][105620] Updated weights for policy 1, policy_version 1209238 (0.0009) [2023-12-27 00:10:57,573][105620] Updated weights for policy 1, policy_version 1209248 (0.0009) [2023-12-27 00:10:57,607][105692] Updated weights for policy 0, policy_version 1207847 (0.0008) [2023-12-27 00:10:57,658][105692] Updated weights for policy 0, policy_version 1207857 (0.0009) [2023-12-27 00:10:57,706][105692] Updated weights for policy 0, policy_version 1207867 (0.0008) [2023-12-27 00:10:58,315][105620] Updated weights for policy 1, policy_version 1209258 (0.0010) [2023-12-27 00:10:58,371][105692] Updated weights for policy 0, policy_version 1207877 (0.0007) [2023-12-27 00:10:58,389][105620] Updated weights for policy 1, policy_version 1209268 (0.0010) [2023-12-27 00:10:58,434][105692] Updated weights for policy 0, policy_version 1207887 (0.0008) [2023-12-27 00:10:58,452][105620] Updated weights for policy 1, policy_version 1209278 (0.0011) [2023-12-27 00:10:58,498][105692] Updated weights for policy 0, policy_version 1207897 (0.0008) [2023-12-27 00:10:59,167][105692] Updated weights for policy 0, policy_version 1207907 (0.0008) [2023-12-27 00:10:59,213][105620] Updated weights for policy 1, policy_version 1209288 (0.0008) [2023-12-27 00:10:59,229][105692] Updated weights for policy 0, policy_version 1207917 (0.0010) [2023-12-27 00:10:59,279][105620] Updated weights for policy 1, policy_version 1209298 (0.0008) [2023-12-27 00:10:59,286][105692] Updated weights for policy 0, policy_version 1207927 (0.0010) [2023-12-27 00:10:59,335][105620] Updated weights for policy 1, policy_version 1209308 (0.0006) [2023-12-27 00:11:00,036][105692] Updated weights for policy 0, policy_version 1207937 (0.0010) [2023-12-27 00:11:00,078][105620] Updated weights for policy 1, policy_version 1209318 (0.0008) [2023-12-27 00:11:00,097][105692] Updated weights for policy 0, policy_version 1207947 (0.0010) [2023-12-27 00:11:00,145][105620] Updated weights for policy 1, policy_version 1209328 (0.0009) [2023-12-27 00:11:00,154][105692] Updated weights for policy 0, policy_version 1207957 (0.0010) [2023-12-27 00:11:00,194][105620] Updated weights for policy 1, policy_version 1209338 (0.0010) [2023-12-27 00:11:00,205][105692] Updated weights for policy 0, policy_version 1207967 (0.0010) [2023-12-27 00:11:00,886][105620] Updated weights for policy 1, policy_version 1209348 (0.0009) [2023-12-27 00:11:00,936][105620] Updated weights for policy 1, policy_version 1209358 (0.0008) [2023-12-27 00:11:00,946][105692] Updated weights for policy 0, policy_version 1207977 (0.0010) [2023-12-27 00:11:00,988][105620] Updated weights for policy 1, policy_version 1209368 (0.0007) [2023-12-27 00:11:01,001][105692] Updated weights for policy 0, policy_version 1207987 (0.0007) [2023-12-27 00:11:01,060][105692] Updated weights for policy 0, policy_version 1207997 (0.0007) [2023-12-27 00:11:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 18855.5). Total num frames: 618930176. Throughput: 0: 9990.2, 1: 9643.7. Samples: 618900000. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:11:01,062][104569] Avg episode reward: [(0, '8903.159'), (1, '9349.177')] [2023-12-27 00:11:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001209376_309641216.pth... [2023-12-27 00:11:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001208256_309354496.pth [2023-12-27 00:11:01,076][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001208000_309297152.pth... [2023-12-27 00:11:01,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001206816_308994048.pth [2023-12-27 00:11:01,705][105620] Updated weights for policy 1, policy_version 1209378 (0.0008) [2023-12-27 00:11:01,771][105620] Updated weights for policy 1, policy_version 1209388 (0.0010) [2023-12-27 00:11:01,827][105620] Updated weights for policy 1, policy_version 1209398 (0.0008) [2023-12-27 00:11:01,851][105692] Updated weights for policy 0, policy_version 1208007 (0.0008) [2023-12-27 00:11:01,884][105620] Updated weights for policy 1, policy_version 1209408 (0.0007) [2023-12-27 00:11:01,906][105692] Updated weights for policy 0, policy_version 1208017 (0.0008) [2023-12-27 00:11:01,964][105692] Updated weights for policy 0, policy_version 1208027 (0.0009) [2023-12-27 00:11:02,608][105620] Updated weights for policy 1, policy_version 1209418 (0.0009) [2023-12-27 00:11:02,656][105620] Updated weights for policy 1, policy_version 1209428 (0.0008) [2023-12-27 00:11:02,706][105692] Updated weights for policy 0, policy_version 1208037 (0.0010) [2023-12-27 00:11:02,710][105620] Updated weights for policy 1, policy_version 1209438 (0.0008) [2023-12-27 00:11:02,755][105692] Updated weights for policy 0, policy_version 1208047 (0.0010) [2023-12-27 00:11:02,799][105692] Updated weights for policy 0, policy_version 1208057 (0.0010) [2023-12-27 00:11:03,501][105692] Updated weights for policy 0, policy_version 1208067 (0.0009) [2023-12-27 00:11:03,526][105620] Updated weights for policy 1, policy_version 1209448 (0.0008) [2023-12-27 00:11:03,569][105692] Updated weights for policy 0, policy_version 1208077 (0.0007) [2023-12-27 00:11:03,580][105620] Updated weights for policy 1, policy_version 1209458 (0.0008) [2023-12-27 00:11:03,619][105692] Updated weights for policy 0, policy_version 1208087 (0.0008) [2023-12-27 00:11:03,642][105620] Updated weights for policy 1, policy_version 1209468 (0.0007) [2023-12-27 00:11:04,254][105692] Updated weights for policy 0, policy_version 1208097 (0.0009) [2023-12-27 00:11:04,318][105692] Updated weights for policy 0, policy_version 1208107 (0.0009) [2023-12-27 00:11:04,384][105692] Updated weights for policy 0, policy_version 1208117 (0.0008) [2023-12-27 00:11:04,452][105692] Updated weights for policy 0, policy_version 1208127 (0.0008) [2023-12-27 00:11:04,482][105620] Updated weights for policy 1, policy_version 1209478 (0.0007) [2023-12-27 00:11:04,532][105620] Updated weights for policy 1, policy_version 1209488 (0.0008) [2023-12-27 00:11:04,585][105620] Updated weights for policy 1, policy_version 1209498 (0.0007) [2023-12-27 00:11:05,034][105692] Updated weights for policy 0, policy_version 1208137 (0.0009) [2023-12-27 00:11:05,079][105692] Updated weights for policy 0, policy_version 1208147 (0.0010) [2023-12-27 00:11:05,144][105692] Updated weights for policy 0, policy_version 1208157 (0.0010) [2023-12-27 00:11:05,376][105620] Updated weights for policy 1, policy_version 1209508 (0.0007) [2023-12-27 00:11:05,442][105620] Updated weights for policy 1, policy_version 1209518 (0.0008) [2023-12-27 00:11:05,509][105620] Updated weights for policy 1, policy_version 1209528 (0.0010) [2023-12-27 00:11:05,804][105692] Updated weights for policy 0, policy_version 1208167 (0.0011) [2023-12-27 00:11:05,864][105692] Updated weights for policy 0, policy_version 1208177 (0.0007) [2023-12-27 00:11:05,929][105692] Updated weights for policy 0, policy_version 1208187 (0.0007) [2023-12-27 00:11:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 18911.0). Total num frames: 619028480. Throughput: 0: 9994.3, 1: 9532.4. Samples: 619014668. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:11:06,062][104569] Avg episode reward: [(0, '9085.208'), (1, '9257.889')] [2023-12-27 00:11:06,290][105620] Updated weights for policy 1, policy_version 1209538 (0.0010) [2023-12-27 00:11:06,358][105620] Updated weights for policy 1, policy_version 1209548 (0.0011) [2023-12-27 00:11:06,424][105620] Updated weights for policy 1, policy_version 1209558 (0.0011) [2023-12-27 00:11:06,489][105620] Updated weights for policy 1, policy_version 1209568 (0.0011) [2023-12-27 00:11:06,622][105692] Updated weights for policy 0, policy_version 1208197 (0.0005) [2023-12-27 00:11:06,684][105692] Updated weights for policy 0, policy_version 1208207 (0.0010) [2023-12-27 00:11:06,740][105692] Updated weights for policy 0, policy_version 1208217 (0.0010) [2023-12-27 00:11:07,153][105620] Updated weights for policy 1, policy_version 1209578 (0.0005) [2023-12-27 00:11:07,207][105620] Updated weights for policy 1, policy_version 1209588 (0.0006) [2023-12-27 00:11:07,262][105620] Updated weights for policy 1, policy_version 1209598 (0.0005) [2023-12-27 00:11:07,425][105692] Updated weights for policy 0, policy_version 1208227 (0.0009) [2023-12-27 00:11:07,482][105692] Updated weights for policy 0, policy_version 1208237 (0.0009) [2023-12-27 00:11:07,544][105692] Updated weights for policy 0, policy_version 1208247 (0.0009) [2023-12-27 00:11:07,972][105620] Updated weights for policy 1, policy_version 1209608 (0.0009) [2023-12-27 00:11:08,027][105620] Updated weights for policy 1, policy_version 1209618 (0.0009) [2023-12-27 00:11:08,064][105692] Updated weights for policy 0, policy_version 1208257 (0.0005) [2023-12-27 00:11:08,080][105620] Updated weights for policy 1, policy_version 1209628 (0.0007) [2023-12-27 00:11:08,128][105692] Updated weights for policy 0, policy_version 1208267 (0.0005) [2023-12-27 00:11:08,197][105692] Updated weights for policy 0, policy_version 1208277 (0.0005) [2023-12-27 00:11:08,255][105692] Updated weights for policy 0, policy_version 1208287 (0.0005) [2023-12-27 00:11:08,810][105692] Updated weights for policy 0, policy_version 1208297 (0.0010) [2023-12-27 00:11:08,870][105692] Updated weights for policy 0, policy_version 1208307 (0.0011) [2023-12-27 00:11:08,936][105692] Updated weights for policy 0, policy_version 1208317 (0.0010) [2023-12-27 00:11:08,948][105620] Updated weights for policy 1, policy_version 1209638 (0.0009) [2023-12-27 00:11:09,006][105620] Updated weights for policy 1, policy_version 1209648 (0.0008) [2023-12-27 00:11:09,062][105620] Updated weights for policy 1, policy_version 1209658 (0.0008) [2023-12-27 00:11:09,729][105692] Updated weights for policy 0, policy_version 1208327 (0.0010) [2023-12-27 00:11:09,780][105692] Updated weights for policy 0, policy_version 1208337 (0.0008) [2023-12-27 00:11:09,794][105620] Updated weights for policy 1, policy_version 1209668 (0.0008) [2023-12-27 00:11:09,838][105692] Updated weights for policy 0, policy_version 1208347 (0.0007) [2023-12-27 00:11:09,857][105620] Updated weights for policy 1, policy_version 1209678 (0.0008) [2023-12-27 00:11:09,928][105620] Updated weights for policy 1, policy_version 1209688 (0.0009) [2023-12-27 00:11:10,648][105692] Updated weights for policy 0, policy_version 1208357 (0.0008) [2023-12-27 00:11:10,653][105620] Updated weights for policy 1, policy_version 1209698 (0.0008) [2023-12-27 00:11:10,707][105620] Updated weights for policy 1, policy_version 1209708 (0.0009) [2023-12-27 00:11:10,708][105692] Updated weights for policy 0, policy_version 1208367 (0.0010) [2023-12-27 00:11:10,764][105620] Updated weights for policy 1, policy_version 1209718 (0.0007) [2023-12-27 00:11:10,765][105692] Updated weights for policy 0, policy_version 1208377 (0.0007) [2023-12-27 00:11:10,820][105620] Updated weights for policy 1, policy_version 1209728 (0.0007) [2023-12-27 00:11:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 18938.8). Total num frames: 619126784. Throughput: 0: 10060.9, 1: 9485.2. Samples: 619132064. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:11:11,062][104569] Avg episode reward: [(0, '8903.560'), (1, '9167.045')] [2023-12-27 00:11:11,557][105692] Updated weights for policy 0, policy_version 1208387 (0.0008) [2023-12-27 00:11:11,561][105620] Updated weights for policy 1, policy_version 1209738 (0.0006) [2023-12-27 00:11:11,620][105692] Updated weights for policy 0, policy_version 1208397 (0.0008) [2023-12-27 00:11:11,626][105620] Updated weights for policy 1, policy_version 1209748 (0.0007) [2023-12-27 00:11:11,680][105692] Updated weights for policy 0, policy_version 1208407 (0.0008) [2023-12-27 00:11:11,686][105620] Updated weights for policy 1, policy_version 1209758 (0.0007) [2023-12-27 00:11:12,424][105692] Updated weights for policy 0, policy_version 1208417 (0.0008) [2023-12-27 00:11:12,448][105620] Updated weights for policy 1, policy_version 1209768 (0.0008) [2023-12-27 00:11:12,476][105692] Updated weights for policy 0, policy_version 1208427 (0.0006) [2023-12-27 00:11:12,510][105620] Updated weights for policy 1, policy_version 1209778 (0.0008) [2023-12-27 00:11:12,534][105692] Updated weights for policy 0, policy_version 1208437 (0.0005) [2023-12-27 00:11:12,567][105620] Updated weights for policy 1, policy_version 1209788 (0.0009) [2023-12-27 00:11:12,581][105692] Updated weights for policy 0, policy_version 1208447 (0.0005) [2023-12-27 00:11:13,123][105692] Updated weights for policy 0, policy_version 1208457 (0.0007) [2023-12-27 00:11:13,184][105692] Updated weights for policy 0, policy_version 1208467 (0.0009) [2023-12-27 00:11:13,245][105692] Updated weights for policy 0, policy_version 1208477 (0.0010) [2023-12-27 00:11:13,412][105620] Updated weights for policy 1, policy_version 1209798 (0.0008) [2023-12-27 00:11:13,461][105620] Updated weights for policy 1, policy_version 1209809 (0.0010) [2023-12-27 00:11:13,508][105620] Updated weights for policy 1, policy_version 1209819 (0.0008) [2023-12-27 00:11:13,971][105692] Updated weights for policy 0, policy_version 1208487 (0.0009) [2023-12-27 00:11:14,030][105692] Updated weights for policy 0, policy_version 1208497 (0.0009) [2023-12-27 00:11:14,089][105692] Updated weights for policy 0, policy_version 1208507 (0.0009) [2023-12-27 00:11:14,282][105620] Updated weights for policy 1, policy_version 1209829 (0.0009) [2023-12-27 00:11:14,343][105620] Updated weights for policy 1, policy_version 1209839 (0.0009) [2023-12-27 00:11:14,403][105620] Updated weights for policy 1, policy_version 1209849 (0.0008) [2023-12-27 00:11:14,884][105692] Updated weights for policy 0, policy_version 1208517 (0.0008) [2023-12-27 00:11:14,940][105692] Updated weights for policy 0, policy_version 1208527 (0.0006) [2023-12-27 00:11:14,995][105692] Updated weights for policy 0, policy_version 1208537 (0.0005) [2023-12-27 00:11:15,100][105620] Updated weights for policy 1, policy_version 1209859 (0.0009) [2023-12-27 00:11:15,161][105620] Updated weights for policy 1, policy_version 1209869 (0.0009) [2023-12-27 00:11:15,223][105620] Updated weights for policy 1, policy_version 1209879 (0.0008) [2023-12-27 00:11:15,644][105692] Updated weights for policy 0, policy_version 1208547 (0.0009) [2023-12-27 00:11:15,694][105692] Updated weights for policy 0, policy_version 1208557 (0.0008) [2023-12-27 00:11:15,756][105692] Updated weights for policy 0, policy_version 1208567 (0.0007) [2023-12-27 00:11:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 18938.8). Total num frames: 619216896. Throughput: 0: 10066.8, 1: 9394.3. Samples: 619188792. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:11:16,063][104569] Avg episode reward: [(0, '8994.903'), (1, '8807.069')] [2023-12-27 00:11:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001208576_309444608.pth... [2023-12-27 00:11:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001207392_309141504.pth [2023-12-27 00:11:16,076][105620] Updated weights for policy 1, policy_version 1209889 (0.0009) [2023-12-27 00:11:16,127][105620] Updated weights for policy 1, policy_version 1209899 (0.0009) [2023-12-27 00:11:16,180][105620] Updated weights for policy 1, policy_version 1209910 (0.0010) [2023-12-27 00:11:16,234][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001209920_309780480.pth... [2023-12-27 00:11:16,237][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001208800_309493760.pth [2023-12-27 00:11:16,238][105620] Updated weights for policy 1, policy_version 1209920 (0.0010) [2023-12-27 00:11:16,321][105692] Updated weights for policy 0, policy_version 1208577 (0.0006) [2023-12-27 00:11:16,371][105692] Updated weights for policy 0, policy_version 1208587 (0.0005) [2023-12-27 00:11:16,421][105692] Updated weights for policy 0, policy_version 1208597 (0.0006) [2023-12-27 00:11:16,485][105692] Updated weights for policy 0, policy_version 1208607 (0.0009) [2023-12-27 00:11:17,101][105620] Updated weights for policy 1, policy_version 1209930 (0.0007) [2023-12-27 00:11:17,127][105692] Updated weights for policy 0, policy_version 1208617 (0.0008) [2023-12-27 00:11:17,157][105620] Updated weights for policy 1, policy_version 1209940 (0.0008) [2023-12-27 00:11:17,180][105692] Updated weights for policy 0, policy_version 1208627 (0.0007) [2023-12-27 00:11:17,212][105585] KL-divergence is very high: 107.5915 [2023-12-27 00:11:17,213][105620] Updated weights for policy 1, policy_version 1209950 (0.0008) [2023-12-27 00:11:17,236][105692] Updated weights for policy 0, policy_version 1208637 (0.0007) [2023-12-27 00:11:17,957][105620] Updated weights for policy 1, policy_version 1209960 (0.0008) [2023-12-27 00:11:17,965][105585] KL-divergence is very high: 127.4842 [2023-12-27 00:11:17,989][105692] Updated weights for policy 0, policy_version 1208647 (0.0008) [2023-12-27 00:11:18,004][105620] Updated weights for policy 1, policy_version 1209970 (0.0008) [2023-12-27 00:11:18,004][105585] KL-divergence is very high: 114.1918 [2023-12-27 00:11:18,039][105692] Updated weights for policy 0, policy_version 1208657 (0.0006) [2023-12-27 00:11:18,057][105620] Updated weights for policy 1, policy_version 1209980 (0.0007) [2023-12-27 00:11:18,093][105692] Updated weights for policy 0, policy_version 1208667 (0.0006) [2023-12-27 00:11:18,707][105692] Updated weights for policy 0, policy_version 1208677 (0.0008) [2023-12-27 00:11:18,729][105620] Updated weights for policy 1, policy_version 1209990 (0.0007) [2023-12-27 00:11:18,763][105692] Updated weights for policy 0, policy_version 1208687 (0.0005) [2023-12-27 00:11:18,794][105620] Updated weights for policy 1, policy_version 1210000 (0.0007) [2023-12-27 00:11:18,822][105692] Updated weights for policy 0, policy_version 1208697 (0.0006) [2023-12-27 00:11:18,861][105620] Updated weights for policy 1, policy_version 1210010 (0.0007) [2023-12-27 00:11:19,426][105692] Updated weights for policy 0, policy_version 1208707 (0.0007) [2023-12-27 00:11:19,490][105692] Updated weights for policy 0, policy_version 1208717 (0.0009) [2023-12-27 00:11:19,555][105692] Updated weights for policy 0, policy_version 1208727 (0.0009) [2023-12-27 00:11:19,573][105620] Updated weights for policy 1, policy_version 1210020 (0.0007) [2023-12-27 00:11:19,636][105620] Updated weights for policy 1, policy_version 1210030 (0.0008) [2023-12-27 00:11:19,699][105620] Updated weights for policy 1, policy_version 1210040 (0.0009) [2023-12-27 00:11:20,350][105620] Updated weights for policy 1, policy_version 1210050 (0.0009) [2023-12-27 00:11:20,381][105692] Updated weights for policy 0, policy_version 1208737 (0.0010) [2023-12-27 00:11:20,412][105620] Updated weights for policy 1, policy_version 1210060 (0.0007) [2023-12-27 00:11:20,443][105692] Updated weights for policy 0, policy_version 1208747 (0.0008) [2023-12-27 00:11:20,469][105620] Updated weights for policy 1, policy_version 1210070 (0.0009) [2023-12-27 00:11:20,496][105692] Updated weights for policy 0, policy_version 1208757 (0.0006) [2023-12-27 00:11:20,532][105620] Updated weights for policy 1, policy_version 1210080 (0.0007) [2023-12-27 00:11:20,556][105692] Updated weights for policy 0, policy_version 1208767 (0.0008) [2023-12-27 00:11:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 18994.3). Total num frames: 619315200. Throughput: 0: 10131.2, 1: 9344.5. Samples: 619306136. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:11:21,062][104569] Avg episode reward: [(0, '8902.542'), (1, '8807.909')] [2023-12-27 00:11:21,290][105620] Updated weights for policy 1, policy_version 1210090 (0.0009) [2023-12-27 00:11:21,346][105620] Updated weights for policy 1, policy_version 1210100 (0.0010) [2023-12-27 00:11:21,360][105692] Updated weights for policy 0, policy_version 1208777 (0.0010) [2023-12-27 00:11:21,409][105620] Updated weights for policy 1, policy_version 1210110 (0.0008) [2023-12-27 00:11:21,424][105692] Updated weights for policy 0, policy_version 1208787 (0.0008) [2023-12-27 00:11:21,488][105692] Updated weights for policy 0, policy_version 1208797 (0.0006) [2023-12-27 00:11:22,166][105620] Updated weights for policy 1, policy_version 1210120 (0.0008) [2023-12-27 00:11:22,221][105692] Updated weights for policy 0, policy_version 1208807 (0.0009) [2023-12-27 00:11:22,226][105620] Updated weights for policy 1, policy_version 1210130 (0.0008) [2023-12-27 00:11:22,281][105620] Updated weights for policy 1, policy_version 1210140 (0.0009) [2023-12-27 00:11:22,284][105692] Updated weights for policy 0, policy_version 1208817 (0.0007) [2023-12-27 00:11:22,347][105692] Updated weights for policy 0, policy_version 1208827 (0.0007) [2023-12-27 00:11:22,932][105620] Updated weights for policy 1, policy_version 1210150 (0.0009) [2023-12-27 00:11:22,983][105620] Updated weights for policy 1, policy_version 1210160 (0.0009) [2023-12-27 00:11:23,029][105620] Updated weights for policy 1, policy_version 1210170 (0.0008) [2023-12-27 00:11:23,168][105692] Updated weights for policy 0, policy_version 1208837 (0.0009) [2023-12-27 00:11:23,233][105692] Updated weights for policy 0, policy_version 1208847 (0.0008) [2023-12-27 00:11:23,299][105692] Updated weights for policy 0, policy_version 1208857 (0.0010) [2023-12-27 00:11:23,687][105620] Updated weights for policy 1, policy_version 1210180 (0.0007) [2023-12-27 00:11:23,750][105620] Updated weights for policy 1, policy_version 1210190 (0.0006) [2023-12-27 00:11:23,803][105620] Updated weights for policy 1, policy_version 1210200 (0.0008) [2023-12-27 00:11:24,091][105692] Updated weights for policy 0, policy_version 1208867 (0.0008) [2023-12-27 00:11:24,150][105692] Updated weights for policy 0, policy_version 1208877 (0.0006) [2023-12-27 00:11:24,205][105692] Updated weights for policy 0, policy_version 1208887 (0.0005) [2023-12-27 00:11:24,601][105620] Updated weights for policy 1, policy_version 1210210 (0.0009) [2023-12-27 00:11:24,674][105620] Updated weights for policy 1, policy_version 1210220 (0.0007) [2023-12-27 00:11:24,727][105620] Updated weights for policy 1, policy_version 1210230 (0.0010) [2023-12-27 00:11:24,766][105692] Updated weights for policy 0, policy_version 1208897 (0.0006) [2023-12-27 00:11:24,788][105620] Updated weights for policy 1, policy_version 1210240 (0.0009) [2023-12-27 00:11:24,822][105692] Updated weights for policy 0, policy_version 1208907 (0.0005) [2023-12-27 00:11:24,885][105692] Updated weights for policy 0, policy_version 1208917 (0.0005) [2023-12-27 00:11:24,947][105692] Updated weights for policy 0, policy_version 1208927 (0.0009) [2023-12-27 00:11:25,466][105620] Updated weights for policy 1, policy_version 1210250 (0.0011) [2023-12-27 00:11:25,503][105692] Updated weights for policy 0, policy_version 1208937 (0.0006) [2023-12-27 00:11:25,521][105620] Updated weights for policy 1, policy_version 1210260 (0.0010) [2023-12-27 00:11:25,556][105692] Updated weights for policy 0, policy_version 1208947 (0.0006) [2023-12-27 00:11:25,579][105620] Updated weights for policy 1, policy_version 1210270 (0.0010) [2023-12-27 00:11:25,613][105692] Updated weights for policy 0, policy_version 1208957 (0.0009) [2023-12-27 00:11:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 18994.3). Total num frames: 619413504. Throughput: 0: 10123.2, 1: 9375.6. Samples: 619423472. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:11:26,062][104569] Avg episode reward: [(0, '8812.231'), (1, '9261.380')] [2023-12-27 00:11:26,148][105620] Updated weights for policy 1, policy_version 1210280 (0.0006) [2023-12-27 00:11:26,215][105620] Updated weights for policy 1, policy_version 1210290 (0.0010) [2023-12-27 00:11:26,216][105692] Updated weights for policy 0, policy_version 1208967 (0.0008) [2023-12-27 00:11:26,273][105692] Updated weights for policy 0, policy_version 1208977 (0.0007) [2023-12-27 00:11:26,283][105620] Updated weights for policy 1, policy_version 1210300 (0.0006) [2023-12-27 00:11:26,341][105692] Updated weights for policy 0, policy_version 1208987 (0.0006) [2023-12-27 00:11:26,807][105620] Updated weights for policy 1, policy_version 1210310 (0.0005) [2023-12-27 00:11:26,858][105620] Updated weights for policy 1, policy_version 1210320 (0.0005) [2023-12-27 00:11:26,891][105692] Updated weights for policy 0, policy_version 1208997 (0.0005) [2023-12-27 00:11:26,916][105620] Updated weights for policy 1, policy_version 1210330 (0.0005) [2023-12-27 00:11:26,939][105692] Updated weights for policy 0, policy_version 1209007 (0.0007) [2023-12-27 00:11:26,983][105692] Updated weights for policy 0, policy_version 1209017 (0.0010) [2023-12-27 00:11:27,470][105620] Updated weights for policy 1, policy_version 1210340 (0.0007) [2023-12-27 00:11:27,515][105620] Updated weights for policy 1, policy_version 1210350 (0.0008) [2023-12-27 00:11:27,564][105620] Updated weights for policy 1, policy_version 1210360 (0.0008) [2023-12-27 00:11:27,621][105692] Updated weights for policy 0, policy_version 1209027 (0.0010) [2023-12-27 00:11:27,665][105692] Updated weights for policy 0, policy_version 1209037 (0.0010) [2023-12-27 00:11:27,720][105692] Updated weights for policy 0, policy_version 1209047 (0.0010) [2023-12-27 00:11:28,309][105620] Updated weights for policy 1, policy_version 1210370 (0.0007) [2023-12-27 00:11:28,371][105620] Updated weights for policy 1, policy_version 1210380 (0.0007) [2023-12-27 00:11:28,433][105620] Updated weights for policy 1, policy_version 1210390 (0.0006) [2023-12-27 00:11:28,487][105692] Updated weights for policy 0, policy_version 1209057 (0.0010) [2023-12-27 00:11:28,495][105620] Updated weights for policy 1, policy_version 1210400 (0.0006) [2023-12-27 00:11:28,553][105692] Updated weights for policy 0, policy_version 1209067 (0.0009) [2023-12-27 00:11:28,616][105692] Updated weights for policy 0, policy_version 1209077 (0.0010) [2023-12-27 00:11:28,681][105692] Updated weights for policy 0, policy_version 1209087 (0.0010) [2023-12-27 00:11:29,129][105620] Updated weights for policy 1, policy_version 1210410 (0.0010) [2023-12-27 00:11:29,180][105620] Updated weights for policy 1, policy_version 1210420 (0.0010) [2023-12-27 00:11:29,241][105620] Updated weights for policy 1, policy_version 1210430 (0.0008) [2023-12-27 00:11:29,379][105692] Updated weights for policy 0, policy_version 1209097 (0.0007) [2023-12-27 00:11:29,426][105692] Updated weights for policy 0, policy_version 1209107 (0.0005) [2023-12-27 00:11:29,476][105692] Updated weights for policy 0, policy_version 1209117 (0.0008) [2023-12-27 00:11:30,009][105620] Updated weights for policy 1, policy_version 1210440 (0.0009) [2023-12-27 00:11:30,060][105620] Updated weights for policy 1, policy_version 1210450 (0.0010) [2023-12-27 00:11:30,109][105620] Updated weights for policy 1, policy_version 1210460 (0.0010) [2023-12-27 00:11:30,195][105692] Updated weights for policy 0, policy_version 1209127 (0.0009) [2023-12-27 00:11:30,249][105692] Updated weights for policy 0, policy_version 1209137 (0.0008) [2023-12-27 00:11:30,306][105692] Updated weights for policy 0, policy_version 1209147 (0.0008) [2023-12-27 00:11:30,845][105620] Updated weights for policy 1, policy_version 1210470 (0.0007) [2023-12-27 00:11:30,910][105620] Updated weights for policy 1, policy_version 1210480 (0.0005) [2023-12-27 00:11:30,959][105620] Updated weights for policy 1, policy_version 1210490 (0.0005) [2023-12-27 00:11:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19022.1). Total num frames: 619520000. Throughput: 0: 10180.8, 1: 9534.7. Samples: 619489728. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:11:31,062][104569] Avg episode reward: [(0, '8819.497'), (1, '9180.354')] [2023-12-27 00:11:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001210496_309927936.pth... [2023-12-27 00:11:31,069][105692] Updated weights for policy 0, policy_version 1209157 (0.0009) [2023-12-27 00:11:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001209376_309641216.pth [2023-12-27 00:11:31,126][105692] Updated weights for policy 0, policy_version 1209167 (0.0010) [2023-12-27 00:11:31,185][105692] Updated weights for policy 0, policy_version 1209177 (0.0010) [2023-12-27 00:11:31,216][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001209184_309600256.pth... [2023-12-27 00:11:31,219][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001208000_309297152.pth [2023-12-27 00:11:31,561][105620] Updated weights for policy 1, policy_version 1210500 (0.0006) [2023-12-27 00:11:31,614][105620] Updated weights for policy 1, policy_version 1210510 (0.0007) [2023-12-27 00:11:31,674][105620] Updated weights for policy 1, policy_version 1210520 (0.0009) [2023-12-27 00:11:31,964][105692] Updated weights for policy 0, policy_version 1209187 (0.0009) [2023-12-27 00:11:32,018][105692] Updated weights for policy 0, policy_version 1209197 (0.0008) [2023-12-27 00:11:32,076][105692] Updated weights for policy 0, policy_version 1209207 (0.0008) [2023-12-27 00:11:32,284][105620] Updated weights for policy 1, policy_version 1210530 (0.0008) [2023-12-27 00:11:32,329][105620] Updated weights for policy 1, policy_version 1210540 (0.0006) [2023-12-27 00:11:32,380][105620] Updated weights for policy 1, policy_version 1210550 (0.0006) [2023-12-27 00:11:32,438][105620] Updated weights for policy 1, policy_version 1210560 (0.0007) [2023-12-27 00:11:32,723][105692] Updated weights for policy 0, policy_version 1209217 (0.0008) [2023-12-27 00:11:32,782][105692] Updated weights for policy 0, policy_version 1209227 (0.0007) [2023-12-27 00:11:32,842][105692] Updated weights for policy 0, policy_version 1209237 (0.0009) [2023-12-27 00:11:32,899][105692] Updated weights for policy 0, policy_version 1209247 (0.0010) [2023-12-27 00:11:33,074][105620] Updated weights for policy 1, policy_version 1210570 (0.0005) [2023-12-27 00:11:33,129][105620] Updated weights for policy 1, policy_version 1210580 (0.0005) [2023-12-27 00:11:33,177][105620] Updated weights for policy 1, policy_version 1210590 (0.0005) [2023-12-27 00:11:33,690][105620] Updated weights for policy 1, policy_version 1210600 (0.0009) [2023-12-27 00:11:33,728][105692] Updated weights for policy 0, policy_version 1209257 (0.0006) [2023-12-27 00:11:33,744][105620] Updated weights for policy 1, policy_version 1210610 (0.0010) [2023-12-27 00:11:33,782][105692] Updated weights for policy 0, policy_version 1209267 (0.0007) [2023-12-27 00:11:33,806][105620] Updated weights for policy 1, policy_version 1210620 (0.0010) [2023-12-27 00:11:33,842][105692] Updated weights for policy 0, policy_version 1209277 (0.0010) [2023-12-27 00:11:34,461][105620] Updated weights for policy 1, policy_version 1210630 (0.0007) [2023-12-27 00:11:34,513][105620] Updated weights for policy 1, policy_version 1210640 (0.0005) [2023-12-27 00:11:34,584][105620] Updated weights for policy 1, policy_version 1210650 (0.0006) [2023-12-27 00:11:34,700][105692] Updated weights for policy 0, policy_version 1209287 (0.0009) [2023-12-27 00:11:34,757][105692] Updated weights for policy 0, policy_version 1209297 (0.0009) [2023-12-27 00:11:34,820][105692] Updated weights for policy 0, policy_version 1209307 (0.0008) [2023-12-27 00:11:35,173][105620] Updated weights for policy 1, policy_version 1210660 (0.0007) [2023-12-27 00:11:35,227][105620] Updated weights for policy 1, policy_version 1210670 (0.0010) [2023-12-27 00:11:35,278][105620] Updated weights for policy 1, policy_version 1210680 (0.0010) [2023-12-27 00:11:35,605][105692] Updated weights for policy 0, policy_version 1209317 (0.0008) [2023-12-27 00:11:35,653][105692] Updated weights for policy 0, policy_version 1209327 (0.0008) [2023-12-27 00:11:35,718][105692] Updated weights for policy 0, policy_version 1209337 (0.0008) [2023-12-27 00:11:36,032][105620] Updated weights for policy 1, policy_version 1210690 (0.0010) [2023-12-27 00:11:36,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.8, 300 sec: 19077.6). Total num frames: 619618304. Throughput: 0: 9974.9, 1: 9788.3. Samples: 619610016. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:11:36,063][104569] Avg episode reward: [(0, '8636.779'), (1, '9014.568')] [2023-12-27 00:11:36,090][105620] Updated weights for policy 1, policy_version 1210700 (0.0010) [2023-12-27 00:11:36,154][105620] Updated weights for policy 1, policy_version 1210710 (0.0011) [2023-12-27 00:11:36,211][105620] Updated weights for policy 1, policy_version 1210720 (0.0011) [2023-12-27 00:11:36,522][105692] Updated weights for policy 0, policy_version 1209347 (0.0008) [2023-12-27 00:11:36,587][105692] Updated weights for policy 0, policy_version 1209357 (0.0009) [2023-12-27 00:11:36,651][105692] Updated weights for policy 0, policy_version 1209367 (0.0008) [2023-12-27 00:11:36,893][105620] Updated weights for policy 1, policy_version 1210730 (0.0008) [2023-12-27 00:11:36,957][105620] Updated weights for policy 1, policy_version 1210740 (0.0011) [2023-12-27 00:11:37,022][105620] Updated weights for policy 1, policy_version 1210750 (0.0011) [2023-12-27 00:11:37,432][105692] Updated weights for policy 0, policy_version 1209377 (0.0008) [2023-12-27 00:11:37,491][105692] Updated weights for policy 0, policy_version 1209387 (0.0008) [2023-12-27 00:11:37,552][105692] Updated weights for policy 0, policy_version 1209397 (0.0008) [2023-12-27 00:11:37,613][105692] Updated weights for policy 0, policy_version 1209407 (0.0008) [2023-12-27 00:11:37,771][105620] Updated weights for policy 1, policy_version 1210760 (0.0011) [2023-12-27 00:11:37,837][105620] Updated weights for policy 1, policy_version 1210770 (0.0010) [2023-12-27 00:11:37,895][105620] Updated weights for policy 1, policy_version 1210780 (0.0010) [2023-12-27 00:11:38,262][105692] Updated weights for policy 0, policy_version 1209417 (0.0008) [2023-12-27 00:11:38,325][105692] Updated weights for policy 0, policy_version 1209427 (0.0008) [2023-12-27 00:11:38,394][105692] Updated weights for policy 0, policy_version 1209437 (0.0009) [2023-12-27 00:11:38,638][105620] Updated weights for policy 1, policy_version 1210790 (0.0010) [2023-12-27 00:11:38,696][105620] Updated weights for policy 1, policy_version 1210800 (0.0011) [2023-12-27 00:11:38,760][105620] Updated weights for policy 1, policy_version 1210810 (0.0010) [2023-12-27 00:11:39,050][105692] Updated weights for policy 0, policy_version 1209447 (0.0006) [2023-12-27 00:11:39,118][105692] Updated weights for policy 0, policy_version 1209457 (0.0005) [2023-12-27 00:11:39,175][105692] Updated weights for policy 0, policy_version 1209467 (0.0007) [2023-12-27 00:11:39,517][105620] Updated weights for policy 1, policy_version 1210820 (0.0010) [2023-12-27 00:11:39,569][105620] Updated weights for policy 1, policy_version 1210830 (0.0010) [2023-12-27 00:11:39,632][105620] Updated weights for policy 1, policy_version 1210840 (0.0010) [2023-12-27 00:11:39,872][105692] Updated weights for policy 0, policy_version 1209477 (0.0009) [2023-12-27 00:11:39,937][105692] Updated weights for policy 0, policy_version 1209487 (0.0011) [2023-12-27 00:11:40,005][105692] Updated weights for policy 0, policy_version 1209497 (0.0009) [2023-12-27 00:11:40,315][105620] Updated weights for policy 1, policy_version 1210850 (0.0011) [2023-12-27 00:11:40,375][105620] Updated weights for policy 1, policy_version 1210860 (0.0011) [2023-12-27 00:11:40,435][105620] Updated weights for policy 1, policy_version 1210870 (0.0008) [2023-12-27 00:11:40,495][105620] Updated weights for policy 1, policy_version 1210880 (0.0005) [2023-12-27 00:11:40,742][105692] Updated weights for policy 0, policy_version 1209507 (0.0009) [2023-12-27 00:11:40,801][105692] Updated weights for policy 0, policy_version 1209517 (0.0010) [2023-12-27 00:11:40,856][105692] Updated weights for policy 0, policy_version 1209527 (0.0010) [2023-12-27 00:11:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19105.4). Total num frames: 619716608. Throughput: 0: 9945.2, 1: 9754.1. Samples: 619724740. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:11:41,062][104569] Avg episode reward: [(0, '8768.807'), (1, '9354.568')] [2023-12-27 00:11:41,182][105620] Updated weights for policy 1, policy_version 1210890 (0.0011) [2023-12-27 00:11:41,246][105620] Updated weights for policy 1, policy_version 1210900 (0.0011) [2023-12-27 00:11:41,306][105620] Updated weights for policy 1, policy_version 1210910 (0.0011) [2023-12-27 00:11:41,622][105692] Updated weights for policy 0, policy_version 1209537 (0.0010) [2023-12-27 00:11:41,691][105692] Updated weights for policy 0, policy_version 1209547 (0.0011) [2023-12-27 00:11:41,762][105692] Updated weights for policy 0, policy_version 1209557 (0.0009) [2023-12-27 00:11:41,822][105692] Updated weights for policy 0, policy_version 1209567 (0.0007) [2023-12-27 00:11:42,079][105620] Updated weights for policy 1, policy_version 1210920 (0.0011) [2023-12-27 00:11:42,142][105620] Updated weights for policy 1, policy_version 1210930 (0.0011) [2023-12-27 00:11:42,201][105620] Updated weights for policy 1, policy_version 1210940 (0.0010) [2023-12-27 00:11:42,536][105692] Updated weights for policy 0, policy_version 1209577 (0.0007) [2023-12-27 00:11:42,602][105692] Updated weights for policy 0, policy_version 1209587 (0.0006) [2023-12-27 00:11:42,668][105692] Updated weights for policy 0, policy_version 1209597 (0.0007) [2023-12-27 00:11:42,903][105620] Updated weights for policy 1, policy_version 1210950 (0.0007) [2023-12-27 00:11:42,968][105620] Updated weights for policy 1, policy_version 1210960 (0.0005) [2023-12-27 00:11:43,030][105620] Updated weights for policy 1, policy_version 1210970 (0.0006) [2023-12-27 00:11:43,356][105692] Updated weights for policy 0, policy_version 1209607 (0.0008) [2023-12-27 00:11:43,404][105692] Updated weights for policy 0, policy_version 1209617 (0.0008) [2023-12-27 00:11:43,464][105692] Updated weights for policy 0, policy_version 1209627 (0.0008) [2023-12-27 00:11:43,662][105620] Updated weights for policy 1, policy_version 1210980 (0.0010) [2023-12-27 00:11:43,710][105620] Updated weights for policy 1, policy_version 1210990 (0.0010) [2023-12-27 00:11:43,762][105620] Updated weights for policy 1, policy_version 1211000 (0.0010) [2023-12-27 00:11:44,156][105692] Updated weights for policy 0, policy_version 1209637 (0.0007) [2023-12-27 00:11:44,221][105692] Updated weights for policy 0, policy_version 1209647 (0.0008) [2023-12-27 00:11:44,288][105692] Updated weights for policy 0, policy_version 1209657 (0.0008) [2023-12-27 00:11:44,527][105620] Updated weights for policy 1, policy_version 1211010 (0.0010) [2023-12-27 00:11:44,588][105620] Updated weights for policy 1, policy_version 1211020 (0.0010) [2023-12-27 00:11:44,650][105620] Updated weights for policy 1, policy_version 1211030 (0.0008) [2023-12-27 00:11:44,702][105620] Updated weights for policy 1, policy_version 1211040 (0.0008) [2023-12-27 00:11:44,986][105692] Updated weights for policy 0, policy_version 1209667 (0.0009) [2023-12-27 00:11:45,052][105692] Updated weights for policy 0, policy_version 1209677 (0.0009) [2023-12-27 00:11:45,091][105585] KL-divergence is very high: 113.0589 [2023-12-27 00:11:45,115][105692] Updated weights for policy 0, policy_version 1209687 (0.0009) [2023-12-27 00:11:45,144][105585] KL-divergence is very high: 122.7716 [2023-12-27 00:11:45,448][105620] Updated weights for policy 1, policy_version 1211050 (0.0008) [2023-12-27 00:11:45,503][105620] Updated weights for policy 1, policy_version 1211060 (0.0008) [2023-12-27 00:11:45,557][105620] Updated weights for policy 1, policy_version 1211070 (0.0006) [2023-12-27 00:11:45,931][105692] Updated weights for policy 0, policy_version 1209697 (0.0010) [2023-12-27 00:11:45,989][105692] Updated weights for policy 0, policy_version 1209707 (0.0008) [2023-12-27 00:11:46,050][105692] Updated weights for policy 0, policy_version 1209717 (0.0008) [2023-12-27 00:11:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19105.4). Total num frames: 619806720. Throughput: 0: 9822.0, 1: 9766.3. Samples: 619781480. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:11:46,063][104569] Avg episode reward: [(0, '9086.574'), (1, '9261.229')] [2023-12-27 00:11:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001211072_310075392.pth... [2023-12-27 00:11:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001209920_309780480.pth [2023-12-27 00:11:46,095][105692] Updated weights for policy 0, policy_version 1209727 (0.0008) [2023-12-27 00:11:46,097][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001209728_309739520.pth... [2023-12-27 00:11:46,101][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001208576_309444608.pth [2023-12-27 00:11:46,203][105620] Updated weights for policy 1, policy_version 1211080 (0.0007) [2023-12-27 00:11:46,251][105620] Updated weights for policy 1, policy_version 1211090 (0.0010) [2023-12-27 00:11:46,308][105620] Updated weights for policy 1, policy_version 1211100 (0.0010) [2023-12-27 00:11:46,805][105692] Updated weights for policy 0, policy_version 1209737 (0.0009) [2023-12-27 00:11:46,864][105692] Updated weights for policy 0, policy_version 1209747 (0.0008) [2023-12-27 00:11:46,913][105692] Updated weights for policy 0, policy_version 1209757 (0.0008) [2023-12-27 00:11:46,998][105620] Updated weights for policy 1, policy_version 1211110 (0.0010) [2023-12-27 00:11:47,071][105620] Updated weights for policy 1, policy_version 1211120 (0.0010) [2023-12-27 00:11:47,125][105620] Updated weights for policy 1, policy_version 1211130 (0.0010) [2023-12-27 00:11:47,722][105692] Updated weights for policy 0, policy_version 1209767 (0.0008) [2023-12-27 00:11:47,766][105692] Updated weights for policy 0, policy_version 1209777 (0.0007) [2023-12-27 00:11:47,812][105692] Updated weights for policy 0, policy_version 1209787 (0.0008) [2023-12-27 00:11:47,830][105620] Updated weights for policy 1, policy_version 1211140 (0.0009) [2023-12-27 00:11:47,898][105620] Updated weights for policy 1, policy_version 1211150 (0.0010) [2023-12-27 00:11:47,955][105620] Updated weights for policy 1, policy_version 1211160 (0.0010) [2023-12-27 00:11:48,607][105620] Updated weights for policy 1, policy_version 1211170 (0.0010) [2023-12-27 00:11:48,637][105692] Updated weights for policy 0, policy_version 1209797 (0.0009) [2023-12-27 00:11:48,673][105620] Updated weights for policy 1, policy_version 1211180 (0.0011) [2023-12-27 00:11:48,709][105692] Updated weights for policy 0, policy_version 1209807 (0.0006) [2023-12-27 00:11:48,735][105620] Updated weights for policy 1, policy_version 1211190 (0.0009) [2023-12-27 00:11:48,767][105692] Updated weights for policy 0, policy_version 1209817 (0.0008) [2023-12-27 00:11:48,784][105620] Updated weights for policy 1, policy_version 1211200 (0.0005) [2023-12-27 00:11:49,433][105620] Updated weights for policy 1, policy_version 1211210 (0.0009) [2023-12-27 00:11:49,482][105620] Updated weights for policy 1, policy_version 1211220 (0.0009) [2023-12-27 00:11:49,486][105692] Updated weights for policy 0, policy_version 1209827 (0.0009) [2023-12-27 00:11:49,537][105692] Updated weights for policy 0, policy_version 1209837 (0.0010) [2023-12-27 00:11:49,541][105620] Updated weights for policy 1, policy_version 1211230 (0.0008) [2023-12-27 00:11:49,590][105692] Updated weights for policy 0, policy_version 1209847 (0.0010) [2023-12-27 00:11:50,341][105692] Updated weights for policy 0, policy_version 1209857 (0.0011) [2023-12-27 00:11:50,361][105620] Updated weights for policy 1, policy_version 1211240 (0.0011) [2023-12-27 00:11:50,401][105692] Updated weights for policy 0, policy_version 1209867 (0.0010) [2023-12-27 00:11:50,414][105620] Updated weights for policy 1, policy_version 1211250 (0.0011) [2023-12-27 00:11:50,457][105692] Updated weights for policy 0, policy_version 1209877 (0.0010) [2023-12-27 00:11:50,466][105620] Updated weights for policy 1, policy_version 1211260 (0.0011) [2023-12-27 00:11:50,518][105692] Updated weights for policy 0, policy_version 1209887 (0.0010) [2023-12-27 00:11:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19133.2). Total num frames: 619905024. Throughput: 0: 9769.8, 1: 9843.9. Samples: 619897284. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:11:51,062][104569] Avg episode reward: [(0, '8818.277'), (1, '9169.832')] [2023-12-27 00:11:51,244][105692] Updated weights for policy 0, policy_version 1209897 (0.0009) [2023-12-27 00:11:51,254][105620] Updated weights for policy 1, policy_version 1211270 (0.0008) [2023-12-27 00:11:51,302][105692] Updated weights for policy 0, policy_version 1209907 (0.0008) [2023-12-27 00:11:51,308][105620] Updated weights for policy 1, policy_version 1211280 (0.0007) [2023-12-27 00:11:51,366][105692] Updated weights for policy 0, policy_version 1209917 (0.0007) [2023-12-27 00:11:51,373][105620] Updated weights for policy 1, policy_version 1211290 (0.0008) [2023-12-27 00:11:52,024][105692] Updated weights for policy 0, policy_version 1209927 (0.0009) [2023-12-27 00:11:52,086][105692] Updated weights for policy 0, policy_version 1209937 (0.0011) [2023-12-27 00:11:52,158][105692] Updated weights for policy 0, policy_version 1209947 (0.0006) [2023-12-27 00:11:52,189][105620] Updated weights for policy 1, policy_version 1211300 (0.0008) [2023-12-27 00:11:52,260][105620] Updated weights for policy 1, policy_version 1211310 (0.0009) [2023-12-27 00:11:52,323][105620] Updated weights for policy 1, policy_version 1211320 (0.0008) [2023-12-27 00:11:52,806][105692] Updated weights for policy 0, policy_version 1209957 (0.0007) [2023-12-27 00:11:52,859][105692] Updated weights for policy 0, policy_version 1209967 (0.0011) [2023-12-27 00:11:52,907][105692] Updated weights for policy 0, policy_version 1209977 (0.0010) [2023-12-27 00:11:53,136][105620] Updated weights for policy 1, policy_version 1211330 (0.0008) [2023-12-27 00:11:53,193][105620] Updated weights for policy 1, policy_version 1211340 (0.0008) [2023-12-27 00:11:53,242][105620] Updated weights for policy 1, policy_version 1211350 (0.0009) [2023-12-27 00:11:53,288][105620] Updated weights for policy 1, policy_version 1211360 (0.0008) [2023-12-27 00:11:53,562][105692] Updated weights for policy 0, policy_version 1209987 (0.0009) [2023-12-27 00:11:53,615][105692] Updated weights for policy 0, policy_version 1209997 (0.0005) [2023-12-27 00:11:53,667][105692] Updated weights for policy 0, policy_version 1210007 (0.0005) [2023-12-27 00:11:54,061][105620] Updated weights for policy 1, policy_version 1211370 (0.0008) [2023-12-27 00:11:54,127][105620] Updated weights for policy 1, policy_version 1211380 (0.0008) [2023-12-27 00:11:54,185][105620] Updated weights for policy 1, policy_version 1211390 (0.0008) [2023-12-27 00:11:54,346][105692] Updated weights for policy 0, policy_version 1210017 (0.0006) [2023-12-27 00:11:54,394][105692] Updated weights for policy 0, policy_version 1210027 (0.0011) [2023-12-27 00:11:54,456][105692] Updated weights for policy 0, policy_version 1210037 (0.0011) [2023-12-27 00:11:54,516][105692] Updated weights for policy 0, policy_version 1210047 (0.0010) [2023-12-27 00:11:54,853][105620] Updated weights for policy 1, policy_version 1211400 (0.0008) [2023-12-27 00:11:54,904][105620] Updated weights for policy 1, policy_version 1211410 (0.0008) [2023-12-27 00:11:54,964][105620] Updated weights for policy 1, policy_version 1211420 (0.0008) [2023-12-27 00:11:55,269][105692] Updated weights for policy 0, policy_version 1210057 (0.0006) [2023-12-27 00:11:55,316][105692] Updated weights for policy 0, policy_version 1210067 (0.0005) [2023-12-27 00:11:55,361][105692] Updated weights for policy 0, policy_version 1210077 (0.0005) [2023-12-27 00:11:55,697][105620] Updated weights for policy 1, policy_version 1211430 (0.0007) [2023-12-27 00:11:55,761][105620] Updated weights for policy 1, policy_version 1211440 (0.0009) [2023-12-27 00:11:55,830][105620] Updated weights for policy 1, policy_version 1211450 (0.0009) [2023-12-27 00:11:55,995][105692] Updated weights for policy 0, policy_version 1210087 (0.0008) [2023-12-27 00:11:56,042][105692] Updated weights for policy 0, policy_version 1210097 (0.0009) [2023-12-27 00:11:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.2, 300 sec: 19160.9). Total num frames: 620003328. Throughput: 0: 9751.9, 1: 9827.6. Samples: 620013144. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:11:56,063][104569] Avg episode reward: [(0, '8817.848'), (1, '9169.310')] [2023-12-27 00:11:56,101][105692] Updated weights for policy 0, policy_version 1210107 (0.0006) [2023-12-27 00:11:56,554][105620] Updated weights for policy 1, policy_version 1211460 (0.0009) [2023-12-27 00:11:56,598][105620] Updated weights for policy 1, policy_version 1211470 (0.0010) [2023-12-27 00:11:56,653][105620] Updated weights for policy 1, policy_version 1211480 (0.0011) [2023-12-27 00:11:56,666][105692] Updated weights for policy 0, policy_version 1210117 (0.0008) [2023-12-27 00:11:56,709][105692] Updated weights for policy 0, policy_version 1210127 (0.0006) [2023-12-27 00:11:56,761][105692] Updated weights for policy 0, policy_version 1210137 (0.0005) [2023-12-27 00:11:57,344][105620] Updated weights for policy 1, policy_version 1211490 (0.0010) [2023-12-27 00:11:57,400][105620] Updated weights for policy 1, policy_version 1211500 (0.0008) [2023-12-27 00:11:57,446][105620] Updated weights for policy 1, policy_version 1211511 (0.0006) [2023-12-27 00:11:57,472][105692] Updated weights for policy 0, policy_version 1210147 (0.0006) [2023-12-27 00:11:57,519][105692] Updated weights for policy 0, policy_version 1210157 (0.0007) [2023-12-27 00:11:57,565][105692] Updated weights for policy 0, policy_version 1210167 (0.0009) [2023-12-27 00:11:58,194][105620] Updated weights for policy 1, policy_version 1211521 (0.0008) [2023-12-27 00:11:58,260][105620] Updated weights for policy 1, policy_version 1211531 (0.0006) [2023-12-27 00:11:58,328][105620] Updated weights for policy 1, policy_version 1211541 (0.0008) [2023-12-27 00:11:58,357][105692] Updated weights for policy 0, policy_version 1210177 (0.0009) [2023-12-27 00:11:58,393][105620] Updated weights for policy 1, policy_version 1211551 (0.0008) [2023-12-27 00:11:58,417][105692] Updated weights for policy 0, policy_version 1210187 (0.0008) [2023-12-27 00:11:58,477][105692] Updated weights for policy 0, policy_version 1210197 (0.0008) [2023-12-27 00:11:58,540][105692] Updated weights for policy 0, policy_version 1210207 (0.0007) [2023-12-27 00:11:59,226][105620] Updated weights for policy 1, policy_version 1211561 (0.0007) [2023-12-27 00:11:59,289][105692] Updated weights for policy 0, policy_version 1210217 (0.0009) [2023-12-27 00:11:59,292][105620] Updated weights for policy 1, policy_version 1211571 (0.0007) [2023-12-27 00:11:59,353][105692] Updated weights for policy 0, policy_version 1210227 (0.0009) [2023-12-27 00:11:59,353][105620] Updated weights for policy 1, policy_version 1211581 (0.0007) [2023-12-27 00:11:59,406][105692] Updated weights for policy 0, policy_version 1210237 (0.0009) [2023-12-27 00:12:00,057][105620] Updated weights for policy 1, policy_version 1211591 (0.0009) [2023-12-27 00:12:00,115][105620] Updated weights for policy 1, policy_version 1211601 (0.0009) [2023-12-27 00:12:00,146][105692] Updated weights for policy 0, policy_version 1210247 (0.0007) [2023-12-27 00:12:00,170][105620] Updated weights for policy 1, policy_version 1211611 (0.0007) [2023-12-27 00:12:00,198][105692] Updated weights for policy 0, policy_version 1210257 (0.0005) [2023-12-27 00:12:00,247][105692] Updated weights for policy 0, policy_version 1210267 (0.0006) [2023-12-27 00:12:00,905][105692] Updated weights for policy 0, policy_version 1210277 (0.0009) [2023-12-27 00:12:00,958][105692] Updated weights for policy 0, policy_version 1210287 (0.0008) [2023-12-27 00:12:00,981][105620] Updated weights for policy 1, policy_version 1211621 (0.0007) [2023-12-27 00:12:01,013][105692] Updated weights for policy 0, policy_version 1210297 (0.0007) [2023-12-27 00:12:01,032][105620] Updated weights for policy 1, policy_version 1211631 (0.0007) [2023-12-27 00:12:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19188.7). Total num frames: 620101632. Throughput: 0: 9770.2, 1: 9856.2. Samples: 620071980. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:12:01,062][104569] Avg episode reward: [(0, '8824.912'), (1, '9168.516')] [2023-12-27 00:12:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001210304_309886976.pth... [2023-12-27 00:12:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001209184_309600256.pth [2023-12-27 00:12:01,096][105620] Updated weights for policy 1, policy_version 1211641 (0.0008) [2023-12-27 00:12:01,139][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001211648_310222848.pth... [2023-12-27 00:12:01,143][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001210496_309927936.pth [2023-12-27 00:12:01,721][105692] Updated weights for policy 0, policy_version 1210307 (0.0007) [2023-12-27 00:12:01,784][105692] Updated weights for policy 0, policy_version 1210317 (0.0008) [2023-12-27 00:12:01,840][105692] Updated weights for policy 0, policy_version 1210327 (0.0008) [2023-12-27 00:12:01,898][105620] Updated weights for policy 1, policy_version 1211651 (0.0010) [2023-12-27 00:12:01,950][105620] Updated weights for policy 1, policy_version 1211661 (0.0010) [2023-12-27 00:12:02,012][105620] Updated weights for policy 1, policy_version 1211671 (0.0010) [2023-12-27 00:12:02,602][105692] Updated weights for policy 0, policy_version 1210337 (0.0007) [2023-12-27 00:12:02,663][105692] Updated weights for policy 0, policy_version 1210347 (0.0008) [2023-12-27 00:12:02,720][105692] Updated weights for policy 0, policy_version 1210357 (0.0009) [2023-12-27 00:12:02,770][105620] Updated weights for policy 1, policy_version 1211681 (0.0010) [2023-12-27 00:12:02,772][105692] Updated weights for policy 0, policy_version 1210367 (0.0010) [2023-12-27 00:12:02,824][105620] Updated weights for policy 1, policy_version 1211691 (0.0008) [2023-12-27 00:12:02,883][105620] Updated weights for policy 1, policy_version 1211701 (0.0008) [2023-12-27 00:12:02,941][105620] Updated weights for policy 1, policy_version 1211711 (0.0008) [2023-12-27 00:12:03,522][105692] Updated weights for policy 0, policy_version 1210377 (0.0008) [2023-12-27 00:12:03,585][105692] Updated weights for policy 0, policy_version 1210387 (0.0007) [2023-12-27 00:12:03,643][105692] Updated weights for policy 0, policy_version 1210397 (0.0008) [2023-12-27 00:12:03,650][105620] Updated weights for policy 1, policy_version 1211721 (0.0010) [2023-12-27 00:12:03,707][105620] Updated weights for policy 1, policy_version 1211731 (0.0010) [2023-12-27 00:12:03,770][105620] Updated weights for policy 1, policy_version 1211741 (0.0010) [2023-12-27 00:12:04,287][105692] Updated weights for policy 0, policy_version 1210407 (0.0009) [2023-12-27 00:12:04,352][105692] Updated weights for policy 0, policy_version 1210417 (0.0008) [2023-12-27 00:12:04,410][105692] Updated weights for policy 0, policy_version 1210427 (0.0008) [2023-12-27 00:12:04,451][105620] Updated weights for policy 1, policy_version 1211751 (0.0011) [2023-12-27 00:12:04,514][105620] Updated weights for policy 1, policy_version 1211761 (0.0010) [2023-12-27 00:12:04,579][105620] Updated weights for policy 1, policy_version 1211771 (0.0011) [2023-12-27 00:12:05,043][105692] Updated weights for policy 0, policy_version 1210437 (0.0007) [2023-12-27 00:12:05,091][105692] Updated weights for policy 0, policy_version 1210447 (0.0005) [2023-12-27 00:12:05,141][105692] Updated weights for policy 0, policy_version 1210457 (0.0007) [2023-12-27 00:12:05,307][105620] Updated weights for policy 1, policy_version 1211781 (0.0010) [2023-12-27 00:12:05,363][105620] Updated weights for policy 1, policy_version 1211791 (0.0010) [2023-12-27 00:12:05,421][105620] Updated weights for policy 1, policy_version 1211801 (0.0010) [2023-12-27 00:12:05,869][105692] Updated weights for policy 0, policy_version 1210467 (0.0008) [2023-12-27 00:12:05,924][105692] Updated weights for policy 0, policy_version 1210477 (0.0008) [2023-12-27 00:12:05,979][105692] Updated weights for policy 0, policy_version 1210487 (0.0008) [2023-12-27 00:12:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19216.5). Total num frames: 620199936. Throughput: 0: 9688.7, 1: 9864.2. Samples: 620186020. Policy #0 lag: (min: 23.0, avg: 23.7, max: 40.0) [2023-12-27 00:12:06,063][104569] Avg episode reward: [(0, '8915.169'), (1, '9168.570')] [2023-12-27 00:12:06,163][105620] Updated weights for policy 1, policy_version 1211811 (0.0010) [2023-12-27 00:12:06,220][105620] Updated weights for policy 1, policy_version 1211821 (0.0010) [2023-12-27 00:12:06,270][105620] Updated weights for policy 1, policy_version 1211831 (0.0010) [2023-12-27 00:12:06,717][105692] Updated weights for policy 0, policy_version 1210497 (0.0008) [2023-12-27 00:12:06,769][105692] Updated weights for policy 0, policy_version 1210507 (0.0010) [2023-12-27 00:12:06,824][105692] Updated weights for policy 0, policy_version 1210517 (0.0007) [2023-12-27 00:12:06,893][105692] Updated weights for policy 0, policy_version 1210527 (0.0005) [2023-12-27 00:12:07,039][105620] Updated weights for policy 1, policy_version 1211841 (0.0010) [2023-12-27 00:12:07,103][105620] Updated weights for policy 1, policy_version 1211851 (0.0010) [2023-12-27 00:12:07,166][105620] Updated weights for policy 1, policy_version 1211861 (0.0010) [2023-12-27 00:12:07,219][105620] Updated weights for policy 1, policy_version 1211871 (0.0010) [2023-12-27 00:12:07,471][105692] Updated weights for policy 0, policy_version 1210537 (0.0010) [2023-12-27 00:12:07,525][105692] Updated weights for policy 0, policy_version 1210547 (0.0010) [2023-12-27 00:12:07,590][105692] Updated weights for policy 0, policy_version 1210557 (0.0010) [2023-12-27 00:12:07,973][105620] Updated weights for policy 1, policy_version 1211881 (0.0010) [2023-12-27 00:12:08,032][105620] Updated weights for policy 1, policy_version 1211891 (0.0010) [2023-12-27 00:12:08,077][105620] Updated weights for policy 1, policy_version 1211901 (0.0010) [2023-12-27 00:12:08,219][105692] Updated weights for policy 0, policy_version 1210567 (0.0007) [2023-12-27 00:12:08,277][105692] Updated weights for policy 0, policy_version 1210577 (0.0006) [2023-12-27 00:12:08,337][105692] Updated weights for policy 0, policy_version 1210587 (0.0010) [2023-12-27 00:12:08,844][105620] Updated weights for policy 1, policy_version 1211911 (0.0010) [2023-12-27 00:12:08,899][105620] Updated weights for policy 1, policy_version 1211921 (0.0010) [2023-12-27 00:12:08,961][105620] Updated weights for policy 1, policy_version 1211931 (0.0010) [2023-12-27 00:12:09,066][105692] Updated weights for policy 0, policy_version 1210597 (0.0010) [2023-12-27 00:12:09,131][105692] Updated weights for policy 0, policy_version 1210607 (0.0011) [2023-12-27 00:12:09,195][105692] Updated weights for policy 0, policy_version 1210617 (0.0010) [2023-12-27 00:12:09,757][105620] Updated weights for policy 1, policy_version 1211941 (0.0010) [2023-12-27 00:12:09,814][105620] Updated weights for policy 1, policy_version 1211951 (0.0011) [2023-12-27 00:12:09,881][105620] Updated weights for policy 1, policy_version 1211961 (0.0011) [2023-12-27 00:12:09,987][105692] Updated weights for policy 0, policy_version 1210627 (0.0009) [2023-12-27 00:12:10,047][105692] Updated weights for policy 0, policy_version 1210637 (0.0008) [2023-12-27 00:12:10,103][105692] Updated weights for policy 0, policy_version 1210647 (0.0009) [2023-12-27 00:12:10,643][105620] Updated weights for policy 1, policy_version 1211971 (0.0010) [2023-12-27 00:12:10,697][105620] Updated weights for policy 1, policy_version 1211981 (0.0010) [2023-12-27 00:12:10,757][105620] Updated weights for policy 1, policy_version 1211991 (0.0010) [2023-12-27 00:12:10,848][105692] Updated weights for policy 0, policy_version 1210657 (0.0010) [2023-12-27 00:12:10,901][105692] Updated weights for policy 0, policy_version 1210667 (0.0009) [2023-12-27 00:12:10,952][105692] Updated weights for policy 0, policy_version 1210677 (0.0010) [2023-12-27 00:12:11,004][105692] Updated weights for policy 0, policy_version 1210687 (0.0010) [2023-12-27 00:12:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19272.0). Total num frames: 620298240. Throughput: 0: 9755.1, 1: 9762.5. Samples: 620301764. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:12:11,062][104569] Avg episode reward: [(0, '9176.440'), (1, '9169.481')] [2023-12-27 00:12:11,554][105620] Updated weights for policy 1, policy_version 1212001 (0.0010) [2023-12-27 00:12:11,622][105620] Updated weights for policy 1, policy_version 1212011 (0.0007) [2023-12-27 00:12:11,684][105620] Updated weights for policy 1, policy_version 1212021 (0.0008) [2023-12-27 00:12:11,701][105692] Updated weights for policy 0, policy_version 1210697 (0.0011) [2023-12-27 00:12:11,752][105620] Updated weights for policy 1, policy_version 1212031 (0.0006) [2023-12-27 00:12:11,767][105692] Updated weights for policy 0, policy_version 1210707 (0.0008) [2023-12-27 00:12:11,823][105692] Updated weights for policy 0, policy_version 1210717 (0.0008) [2023-12-27 00:12:12,517][105620] Updated weights for policy 1, policy_version 1212041 (0.0009) [2023-12-27 00:12:12,574][105620] Updated weights for policy 1, policy_version 1212051 (0.0009) [2023-12-27 00:12:12,592][105692] Updated weights for policy 0, policy_version 1210727 (0.0007) [2023-12-27 00:12:12,623][105620] Updated weights for policy 1, policy_version 1212061 (0.0008) [2023-12-27 00:12:12,646][105692] Updated weights for policy 0, policy_version 1210737 (0.0006) [2023-12-27 00:12:12,704][105692] Updated weights for policy 0, policy_version 1210747 (0.0008) [2023-12-27 00:12:13,384][105620] Updated weights for policy 1, policy_version 1212071 (0.0009) [2023-12-27 00:12:13,423][105692] Updated weights for policy 0, policy_version 1210757 (0.0006) [2023-12-27 00:12:13,436][105620] Updated weights for policy 1, policy_version 1212081 (0.0008) [2023-12-27 00:12:13,468][105692] Updated weights for policy 0, policy_version 1210767 (0.0005) [2023-12-27 00:12:13,499][105620] Updated weights for policy 1, policy_version 1212091 (0.0009) [2023-12-27 00:12:13,512][105692] Updated weights for policy 0, policy_version 1210777 (0.0005) [2023-12-27 00:12:14,189][105692] Updated weights for policy 0, policy_version 1210787 (0.0007) [2023-12-27 00:12:14,258][105692] Updated weights for policy 0, policy_version 1210797 (0.0009) [2023-12-27 00:12:14,284][105620] Updated weights for policy 1, policy_version 1212101 (0.0008) [2023-12-27 00:12:14,316][105692] Updated weights for policy 0, policy_version 1210807 (0.0008) [2023-12-27 00:12:14,345][105620] Updated weights for policy 1, policy_version 1212111 (0.0005) [2023-12-27 00:12:14,393][105620] Updated weights for policy 1, policy_version 1212121 (0.0005) [2023-12-27 00:12:14,969][105692] Updated weights for policy 0, policy_version 1210817 (0.0008) [2023-12-27 00:12:15,034][105692] Updated weights for policy 0, policy_version 1210827 (0.0008) [2023-12-27 00:12:15,071][105620] Updated weights for policy 1, policy_version 1212131 (0.0005) [2023-12-27 00:12:15,101][105692] Updated weights for policy 0, policy_version 1210837 (0.0008) [2023-12-27 00:12:15,136][105620] Updated weights for policy 1, policy_version 1212141 (0.0006) [2023-12-27 00:12:15,164][105692] Updated weights for policy 0, policy_version 1210847 (0.0008) [2023-12-27 00:12:15,193][105620] Updated weights for policy 1, policy_version 1212151 (0.0009) [2023-12-27 00:12:15,883][105692] Updated weights for policy 0, policy_version 1210857 (0.0010) [2023-12-27 00:12:15,935][105692] Updated weights for policy 0, policy_version 1210867 (0.0011) [2023-12-27 00:12:15,946][105620] Updated weights for policy 1, policy_version 1212161 (0.0010) [2023-12-27 00:12:15,995][105692] Updated weights for policy 0, policy_version 1210877 (0.0011) [2023-12-27 00:12:16,012][105620] Updated weights for policy 1, policy_version 1212171 (0.0006) [2023-12-27 00:12:16,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.3, 300 sec: 19244.3). Total num frames: 620388352. Throughput: 0: 9679.9, 1: 9615.8. Samples: 620358036. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:12:16,062][104569] Avg episode reward: [(0, '9175.578'), (1, '3324.009')] [2023-12-27 00:12:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001210880_310034432.pth... [2023-12-27 00:12:16,077][105620] Updated weights for policy 1, policy_version 1212181 (0.0008) [2023-12-27 00:12:16,094][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001209728_309739520.pth [2023-12-27 00:12:16,136][105620] Updated weights for policy 1, policy_version 1212191 (0.0008) [2023-12-27 00:12:16,140][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001212192_310362112.pth... [2023-12-27 00:12:16,143][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001211072_310075392.pth [2023-12-27 00:12:16,703][105692] Updated weights for policy 0, policy_version 1210887 (0.0007) [2023-12-27 00:12:16,762][105692] Updated weights for policy 0, policy_version 1210897 (0.0005) [2023-12-27 00:12:16,815][105692] Updated weights for policy 0, policy_version 1210907 (0.0005) [2023-12-27 00:12:16,919][105620] Updated weights for policy 1, policy_version 1212201 (0.0010) [2023-12-27 00:12:16,973][105620] Updated weights for policy 1, policy_version 1212211 (0.0010) [2023-12-27 00:12:17,025][105620] Updated weights for policy 1, policy_version 1212221 (0.0009) [2023-12-27 00:12:17,324][105692] Updated weights for policy 0, policy_version 1210917 (0.0005) [2023-12-27 00:12:17,388][105692] Updated weights for policy 0, policy_version 1210927 (0.0005) [2023-12-27 00:12:17,441][105692] Updated weights for policy 0, policy_version 1210937 (0.0005) [2023-12-27 00:12:17,786][105620] Updated weights for policy 1, policy_version 1212232 (0.0007) [2023-12-27 00:12:17,838][105620] Updated weights for policy 1, policy_version 1212242 (0.0005) [2023-12-27 00:12:17,889][105620] Updated weights for policy 1, policy_version 1212252 (0.0005) [2023-12-27 00:12:18,098][105692] Updated weights for policy 0, policy_version 1210947 (0.0007) [2023-12-27 00:12:18,152][105692] Updated weights for policy 0, policy_version 1210957 (0.0010) [2023-12-27 00:12:18,204][105692] Updated weights for policy 0, policy_version 1210967 (0.0009) [2023-12-27 00:12:18,472][105620] Updated weights for policy 1, policy_version 1212262 (0.0008) [2023-12-27 00:12:18,534][105620] Updated weights for policy 1, policy_version 1212272 (0.0010) [2023-12-27 00:12:18,590][105620] Updated weights for policy 1, policy_version 1212282 (0.0010) [2023-12-27 00:12:18,950][105692] Updated weights for policy 0, policy_version 1210977 (0.0008) [2023-12-27 00:12:19,015][105692] Updated weights for policy 0, policy_version 1210987 (0.0009) [2023-12-27 00:12:19,080][105692] Updated weights for policy 0, policy_version 1210997 (0.0008) [2023-12-27 00:12:19,138][105692] Updated weights for policy 0, policy_version 1211007 (0.0007) [2023-12-27 00:12:19,329][105620] Updated weights for policy 1, policy_version 1212292 (0.0008) [2023-12-27 00:12:19,390][105620] Updated weights for policy 1, policy_version 1212302 (0.0011) [2023-12-27 00:12:19,447][105620] Updated weights for policy 1, policy_version 1212312 (0.0010) [2023-12-27 00:12:19,885][105692] Updated weights for policy 0, policy_version 1211017 (0.0007) [2023-12-27 00:12:19,943][105692] Updated weights for policy 0, policy_version 1211027 (0.0008) [2023-12-27 00:12:20,000][105692] Updated weights for policy 0, policy_version 1211037 (0.0008) [2023-12-27 00:12:20,125][105620] Updated weights for policy 1, policy_version 1212322 (0.0009) [2023-12-27 00:12:20,193][105620] Updated weights for policy 1, policy_version 1212332 (0.0011) [2023-12-27 00:12:20,259][105620] Updated weights for policy 1, policy_version 1212342 (0.0011) [2023-12-27 00:12:20,326][105620] Updated weights for policy 1, policy_version 1212352 (0.0011) [2023-12-27 00:12:20,753][105692] Updated weights for policy 0, policy_version 1211047 (0.0009) [2023-12-27 00:12:20,812][105692] Updated weights for policy 0, policy_version 1211057 (0.0008) [2023-12-27 00:12:20,871][105692] Updated weights for policy 0, policy_version 1211067 (0.0009) [2023-12-27 00:12:21,001][105620] Updated weights for policy 1, policy_version 1212362 (0.0006) [2023-12-27 00:12:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19299.8). Total num frames: 620486656. Throughput: 0: 9794.8, 1: 9490.3. Samples: 620477844. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:12:21,063][104569] Avg episode reward: [(0, '9267.086'), (1, '3342.268')] [2023-12-27 00:12:21,074][105620] Updated weights for policy 1, policy_version 1212372 (0.0007) [2023-12-27 00:12:21,139][105620] Updated weights for policy 1, policy_version 1212382 (0.0008) [2023-12-27 00:12:21,663][105692] Updated weights for policy 0, policy_version 1211077 (0.0009) [2023-12-27 00:12:21,722][105692] Updated weights for policy 0, policy_version 1211087 (0.0009) [2023-12-27 00:12:21,726][105585] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000008 [2023-12-27 00:12:21,826][105620] Updated weights for policy 1, policy_version 1212392 (0.0008) [2023-12-27 00:12:21,886][105620] Updated weights for policy 1, policy_version 1212402 (0.0011) [2023-12-27 00:12:21,942][105620] Updated weights for policy 1, policy_version 1212412 (0.0011) [2023-12-27 00:12:22,591][105692] Updated weights for policy 0, policy_version 1211097 (0.0010) [2023-12-27 00:12:22,642][105692] Updated weights for policy 0, policy_version 1211107 (0.0007) [2023-12-27 00:12:22,698][105692] Updated weights for policy 0, policy_version 1211117 (0.0007) [2023-12-27 00:12:22,703][105620] Updated weights for policy 1, policy_version 1212422 (0.0011) [2023-12-27 00:12:22,755][105620] Updated weights for policy 1, policy_version 1212432 (0.0011) [2023-12-27 00:12:22,819][105620] Updated weights for policy 1, policy_version 1212442 (0.0011) [2023-12-27 00:12:23,356][105692] Updated weights for policy 0, policy_version 1211127 (0.0008) [2023-12-27 00:12:23,408][105692] Updated weights for policy 0, policy_version 1211137 (0.0008) [2023-12-27 00:12:23,456][105692] Updated weights for policy 0, policy_version 1211147 (0.0008) [2023-12-27 00:12:23,573][105620] Updated weights for policy 1, policy_version 1212452 (0.0011) [2023-12-27 00:12:23,621][105620] Updated weights for policy 1, policy_version 1212462 (0.0010) [2023-12-27 00:12:23,672][105620] Updated weights for policy 1, policy_version 1212472 (0.0010) [2023-12-27 00:12:24,137][105692] Updated weights for policy 0, policy_version 1211157 (0.0009) [2023-12-27 00:12:24,191][105692] Updated weights for policy 0, policy_version 1211167 (0.0009) [2023-12-27 00:12:24,242][105692] Updated weights for policy 0, policy_version 1211177 (0.0009) [2023-12-27 00:12:24,434][105620] Updated weights for policy 1, policy_version 1212482 (0.0010) [2023-12-27 00:12:24,497][105620] Updated weights for policy 1, policy_version 1212492 (0.0009) [2023-12-27 00:12:24,559][105620] Updated weights for policy 1, policy_version 1212502 (0.0008) [2023-12-27 00:12:24,604][105620] Updated weights for policy 1, policy_version 1212512 (0.0008) [2023-12-27 00:12:25,054][105692] Updated weights for policy 0, policy_version 1211187 (0.0009) [2023-12-27 00:12:25,116][105692] Updated weights for policy 0, policy_version 1211197 (0.0011) [2023-12-27 00:12:25,179][105692] Updated weights for policy 0, policy_version 1211207 (0.0009) [2023-12-27 00:12:25,256][105620] Updated weights for policy 1, policy_version 1212522 (0.0007) [2023-12-27 00:12:25,312][105620] Updated weights for policy 1, policy_version 1212532 (0.0005) [2023-12-27 00:12:25,377][105620] Updated weights for policy 1, policy_version 1212542 (0.0006) [2023-12-27 00:12:25,894][105692] Updated weights for policy 0, policy_version 1211217 (0.0008) [2023-12-27 00:12:25,957][105692] Updated weights for policy 0, policy_version 1211227 (0.0006) [2023-12-27 00:12:25,970][105620] Updated weights for policy 1, policy_version 1212552 (0.0009) [2023-12-27 00:12:26,014][105692] Updated weights for policy 0, policy_version 1211237 (0.0005) [2023-12-27 00:12:26,025][105620] Updated weights for policy 1, policy_version 1212562 (0.0008) [2023-12-27 00:12:26,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19272.0). Total num frames: 620576768. Throughput: 0: 9784.2, 1: 9516.3. Samples: 620593264. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:12:26,063][104569] Avg episode reward: [(0, '9085.318'), (1, '7283.239')] [2023-12-27 00:12:26,074][105692] Updated weights for policy 0, policy_version 1211247 (0.0010) [2023-12-27 00:12:26,083][105620] Updated weights for policy 1, policy_version 1212572 (0.0008) [2023-12-27 00:12:26,707][105620] Updated weights for policy 1, policy_version 1212582 (0.0006) [2023-12-27 00:12:26,730][105692] Updated weights for policy 0, policy_version 1211257 (0.0011) [2023-12-27 00:12:26,755][105620] Updated weights for policy 1, policy_version 1212592 (0.0005) [2023-12-27 00:12:26,782][105692] Updated weights for policy 0, policy_version 1211267 (0.0010) [2023-12-27 00:12:26,813][105620] Updated weights for policy 1, policy_version 1212602 (0.0006) [2023-12-27 00:12:26,837][105692] Updated weights for policy 0, policy_version 1211277 (0.0010) [2023-12-27 00:12:27,490][105620] Updated weights for policy 1, policy_version 1212612 (0.0009) [2023-12-27 00:12:27,519][105692] Updated weights for policy 0, policy_version 1211287 (0.0007) [2023-12-27 00:12:27,544][105620] Updated weights for policy 1, policy_version 1212622 (0.0010) [2023-12-27 00:12:27,574][105692] Updated weights for policy 0, policy_version 1211297 (0.0005) [2023-12-27 00:12:27,605][105620] Updated weights for policy 1, policy_version 1212632 (0.0010) [2023-12-27 00:12:27,639][105692] Updated weights for policy 0, policy_version 1211307 (0.0005) [2023-12-27 00:12:28,260][105692] Updated weights for policy 0, policy_version 1211317 (0.0005) [2023-12-27 00:12:28,310][105692] Updated weights for policy 0, policy_version 1211327 (0.0006) [2023-12-27 00:12:28,315][105585] KL-divergence is very high: 121.6888 [2023-12-27 00:12:28,323][105585] KL-divergence is very high: 150.7639 [2023-12-27 00:12:28,343][105585] KL-divergence is very high: 124.6279 [2023-12-27 00:12:28,349][105585] KL-divergence is very high: 117.2381 [2023-12-27 00:12:28,367][105620] Updated weights for policy 1, policy_version 1212642 (0.0010) [2023-12-27 00:12:28,367][105585] KL-divergence is very high: 144.6697 [2023-12-27 00:12:28,373][105585] KL-divergence is very high: 157.6263 [2023-12-27 00:12:28,373][105692] Updated weights for policy 0, policy_version 1211337 (0.0011) [2023-12-27 00:12:28,389][105585] KL-divergence is very high: 104.8167 [2023-12-27 00:12:28,428][105620] Updated weights for policy 1, policy_version 1212652 (0.0006) [2023-12-27 00:12:28,479][105620] Updated weights for policy 1, policy_version 1212662 (0.0008) [2023-12-27 00:12:28,534][105620] Updated weights for policy 1, policy_version 1212672 (0.0008) [2023-12-27 00:12:28,959][105692] Updated weights for policy 0, policy_version 1211347 (0.0011) [2023-12-27 00:12:29,020][105692] Updated weights for policy 0, policy_version 1211357 (0.0010) [2023-12-27 00:12:29,071][105692] Updated weights for policy 0, policy_version 1211367 (0.0010) [2023-12-27 00:12:29,236][105620] Updated weights for policy 1, policy_version 1212682 (0.0006) [2023-12-27 00:12:29,299][105620] Updated weights for policy 1, policy_version 1212692 (0.0007) [2023-12-27 00:12:29,361][105620] Updated weights for policy 1, policy_version 1212702 (0.0010) [2023-12-27 00:12:29,842][105692] Updated weights for policy 0, policy_version 1211377 (0.0010) [2023-12-27 00:12:29,897][105692] Updated weights for policy 0, policy_version 1211387 (0.0010) [2023-12-27 00:12:29,962][105692] Updated weights for policy 0, policy_version 1211397 (0.0011) [2023-12-27 00:12:29,997][105620] Updated weights for policy 1, policy_version 1212712 (0.0008) [2023-12-27 00:12:30,020][105692] Updated weights for policy 0, policy_version 1211407 (0.0010) [2023-12-27 00:12:30,058][105620] Updated weights for policy 1, policy_version 1212722 (0.0008) [2023-12-27 00:12:30,108][105620] Updated weights for policy 1, policy_version 1212732 (0.0008) [2023-12-27 00:12:30,699][105692] Updated weights for policy 0, policy_version 1211417 (0.0006) [2023-12-27 00:12:30,749][105692] Updated weights for policy 0, policy_version 1211427 (0.0006) [2023-12-27 00:12:30,767][105620] Updated weights for policy 1, policy_version 1212742 (0.0006) [2023-12-27 00:12:30,800][105692] Updated weights for policy 0, policy_version 1211437 (0.0005) [2023-12-27 00:12:30,821][105620] Updated weights for policy 1, policy_version 1212752 (0.0006) [2023-12-27 00:12:30,865][105620] Updated weights for policy 1, policy_version 1212762 (0.0005) [2023-12-27 00:12:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.2, 300 sec: 19355.3). Total num frames: 620691456. Throughput: 0: 9861.3, 1: 9528.8. Samples: 620654032. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:12:31,063][104569] Avg episode reward: [(0, '8816.841'), (1, '7924.294')] [2023-12-27 00:12:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001211440_310181888.pth... [2023-12-27 00:12:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001212768_310509568.pth... [2023-12-27 00:12:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001210304_309886976.pth [2023-12-27 00:12:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001211648_310222848.pth [2023-12-27 00:12:31,442][105692] Updated weights for policy 0, policy_version 1211447 (0.0008) [2023-12-27 00:12:31,471][105620] Updated weights for policy 1, policy_version 1212772 (0.0006) [2023-12-27 00:12:31,505][105692] Updated weights for policy 0, policy_version 1211457 (0.0009) [2023-12-27 00:12:31,530][105620] Updated weights for policy 1, policy_version 1212782 (0.0006) [2023-12-27 00:12:31,561][105692] Updated weights for policy 0, policy_version 1211467 (0.0008) [2023-12-27 00:12:31,587][105620] Updated weights for policy 1, policy_version 1212792 (0.0007) [2023-12-27 00:12:32,265][105692] Updated weights for policy 0, policy_version 1211477 (0.0009) [2023-12-27 00:12:32,323][105692] Updated weights for policy 0, policy_version 1211487 (0.0009) [2023-12-27 00:12:32,360][105620] Updated weights for policy 1, policy_version 1212802 (0.0007) [2023-12-27 00:12:32,388][105692] Updated weights for policy 0, policy_version 1211497 (0.0009) [2023-12-27 00:12:32,424][105620] Updated weights for policy 1, policy_version 1212812 (0.0007) [2023-12-27 00:12:32,488][105620] Updated weights for policy 1, policy_version 1212822 (0.0007) [2023-12-27 00:12:32,546][105620] Updated weights for policy 1, policy_version 1212832 (0.0007) [2023-12-27 00:12:33,081][105692] Updated weights for policy 0, policy_version 1211507 (0.0007) [2023-12-27 00:12:33,131][105692] Updated weights for policy 0, policy_version 1211517 (0.0005) [2023-12-27 00:12:33,192][105692] Updated weights for policy 0, policy_version 1211527 (0.0005) [2023-12-27 00:12:33,310][105620] Updated weights for policy 1, policy_version 1212842 (0.0008) [2023-12-27 00:12:33,363][105620] Updated weights for policy 1, policy_version 1212852 (0.0006) [2023-12-27 00:12:33,426][105620] Updated weights for policy 1, policy_version 1212862 (0.0007) [2023-12-27 00:12:33,897][105692] Updated weights for policy 0, policy_version 1211537 (0.0006) [2023-12-27 00:12:33,965][105692] Updated weights for policy 0, policy_version 1211547 (0.0010) [2023-12-27 00:12:34,020][105692] Updated weights for policy 0, policy_version 1211557 (0.0010) [2023-12-27 00:12:34,060][105620] Updated weights for policy 1, policy_version 1212872 (0.0007) [2023-12-27 00:12:34,071][105692] Updated weights for policy 0, policy_version 1211567 (0.0010) [2023-12-27 00:12:34,122][105620] Updated weights for policy 1, policy_version 1212882 (0.0007) [2023-12-27 00:12:34,199][105620] Updated weights for policy 1, policy_version 1212892 (0.0008) [2023-12-27 00:12:34,821][105692] Updated weights for policy 0, policy_version 1211577 (0.0010) [2023-12-27 00:12:34,881][105692] Updated weights for policy 0, policy_version 1211587 (0.0011) [2023-12-27 00:12:34,933][105620] Updated weights for policy 1, policy_version 1212902 (0.0007) [2023-12-27 00:12:34,935][105692] Updated weights for policy 0, policy_version 1211597 (0.0011) [2023-12-27 00:12:34,996][105620] Updated weights for policy 1, policy_version 1212912 (0.0007) [2023-12-27 00:12:35,049][105620] Updated weights for policy 1, policy_version 1212922 (0.0008) [2023-12-27 00:12:35,667][105692] Updated weights for policy 0, policy_version 1211607 (0.0010) [2023-12-27 00:12:35,718][105692] Updated weights for policy 0, policy_version 1211617 (0.0010) [2023-12-27 00:12:35,720][105620] Updated weights for policy 1, policy_version 1212932 (0.0008) [2023-12-27 00:12:35,767][105692] Updated weights for policy 0, policy_version 1211627 (0.0007) [2023-12-27 00:12:35,769][105620] Updated weights for policy 1, policy_version 1212943 (0.0007) [2023-12-27 00:12:35,816][105620] Updated weights for policy 1, policy_version 1212953 (0.0008) [2023-12-27 00:12:36,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 620789760. Throughput: 0: 9957.6, 1: 9551.6. Samples: 620775196. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:12:36,062][104569] Avg episode reward: [(0, '8996.688'), (1, '8646.831')] [2023-12-27 00:12:36,391][105692] Updated weights for policy 0, policy_version 1211637 (0.0007) [2023-12-27 00:12:36,439][105692] Updated weights for policy 0, policy_version 1211647 (0.0009) [2023-12-27 00:12:36,491][105692] Updated weights for policy 0, policy_version 1211657 (0.0009) [2023-12-27 00:12:36,587][105620] Updated weights for policy 1, policy_version 1212963 (0.0006) [2023-12-27 00:12:36,648][105620] Updated weights for policy 1, policy_version 1212973 (0.0008) [2023-12-27 00:12:36,709][105620] Updated weights for policy 1, policy_version 1212983 (0.0009) [2023-12-27 00:12:37,144][105692] Updated weights for policy 0, policy_version 1211667 (0.0007) [2023-12-27 00:12:37,197][105692] Updated weights for policy 0, policy_version 1211677 (0.0006) [2023-12-27 00:12:37,247][105692] Updated weights for policy 0, policy_version 1211687 (0.0006) [2023-12-27 00:12:37,478][105620] Updated weights for policy 1, policy_version 1212993 (0.0009) [2023-12-27 00:12:37,533][105620] Updated weights for policy 1, policy_version 1213003 (0.0010) [2023-12-27 00:12:37,585][105620] Updated weights for policy 1, policy_version 1213013 (0.0010) [2023-12-27 00:12:37,637][105620] Updated weights for policy 1, policy_version 1213023 (0.0010) [2023-12-27 00:12:37,904][105692] Updated weights for policy 0, policy_version 1211697 (0.0006) [2023-12-27 00:12:37,965][105692] Updated weights for policy 0, policy_version 1211707 (0.0009) [2023-12-27 00:12:38,024][105692] Updated weights for policy 0, policy_version 1211717 (0.0009) [2023-12-27 00:12:38,075][105692] Updated weights for policy 0, policy_version 1211727 (0.0009) [2023-12-27 00:12:38,406][105620] Updated weights for policy 1, policy_version 1213033 (0.0010) [2023-12-27 00:12:38,469][105620] Updated weights for policy 1, policy_version 1213043 (0.0010) [2023-12-27 00:12:38,528][105620] Updated weights for policy 1, policy_version 1213053 (0.0010) [2023-12-27 00:12:38,810][105692] Updated weights for policy 0, policy_version 1211737 (0.0009) [2023-12-27 00:12:38,872][105692] Updated weights for policy 0, policy_version 1211747 (0.0008) [2023-12-27 00:12:38,929][105692] Updated weights for policy 0, policy_version 1211757 (0.0008) [2023-12-27 00:12:39,307][105620] Updated weights for policy 1, policy_version 1213063 (0.0008) [2023-12-27 00:12:39,375][105620] Updated weights for policy 1, policy_version 1213073 (0.0008) [2023-12-27 00:12:39,441][105620] Updated weights for policy 1, policy_version 1213083 (0.0008) [2023-12-27 00:12:39,670][105692] Updated weights for policy 0, policy_version 1211767 (0.0007) [2023-12-27 00:12:39,728][105692] Updated weights for policy 0, policy_version 1211777 (0.0005) [2023-12-27 00:12:39,792][105692] Updated weights for policy 0, policy_version 1211787 (0.0008) [2023-12-27 00:12:40,144][105620] Updated weights for policy 1, policy_version 1213093 (0.0008) [2023-12-27 00:12:40,212][105620] Updated weights for policy 1, policy_version 1213103 (0.0008) [2023-12-27 00:12:40,277][105620] Updated weights for policy 1, policy_version 1213113 (0.0006) [2023-12-27 00:12:40,565][105692] Updated weights for policy 0, policy_version 1211797 (0.0008) [2023-12-27 00:12:40,614][105692] Updated weights for policy 0, policy_version 1211807 (0.0009) [2023-12-27 00:12:40,666][105692] Updated weights for policy 0, policy_version 1211817 (0.0009) [2023-12-27 00:12:40,975][105620] Updated weights for policy 1, policy_version 1213123 (0.0009) [2023-12-27 00:12:41,037][105620] Updated weights for policy 1, policy_version 1213133 (0.0009) [2023-12-27 00:12:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 620879872. Throughput: 0: 9918.1, 1: 9595.8. Samples: 620891264. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:12:41,063][104569] Avg episode reward: [(0, '9177.486'), (1, '8895.971')] [2023-12-27 00:12:41,103][105620] Updated weights for policy 1, policy_version 1213143 (0.0010) [2023-12-27 00:12:41,420][105692] Updated weights for policy 0, policy_version 1211827 (0.0008) [2023-12-27 00:12:41,472][105692] Updated weights for policy 0, policy_version 1211837 (0.0009) [2023-12-27 00:12:41,520][105692] Updated weights for policy 0, policy_version 1211847 (0.0009) [2023-12-27 00:12:41,922][105620] Updated weights for policy 1, policy_version 1213153 (0.0009) [2023-12-27 00:12:41,981][105620] Updated weights for policy 1, policy_version 1213163 (0.0009) [2023-12-27 00:12:42,035][105620] Updated weights for policy 1, policy_version 1213173 (0.0008) [2023-12-27 00:12:42,083][105620] Updated weights for policy 1, policy_version 1213183 (0.0008) [2023-12-27 00:12:42,269][105692] Updated weights for policy 0, policy_version 1211857 (0.0009) [2023-12-27 00:12:42,329][105692] Updated weights for policy 0, policy_version 1211867 (0.0007) [2023-12-27 00:12:42,401][105692] Updated weights for policy 0, policy_version 1211877 (0.0008) [2023-12-27 00:12:42,459][105692] Updated weights for policy 0, policy_version 1211887 (0.0006) [2023-12-27 00:12:42,904][105620] Updated weights for policy 1, policy_version 1213193 (0.0009) [2023-12-27 00:12:42,961][105620] Updated weights for policy 1, policy_version 1213203 (0.0009) [2023-12-27 00:12:43,012][105620] Updated weights for policy 1, policy_version 1213213 (0.0009) [2023-12-27 00:12:43,147][105692] Updated weights for policy 0, policy_version 1211897 (0.0009) [2023-12-27 00:12:43,205][105692] Updated weights for policy 0, policy_version 1211907 (0.0009) [2023-12-27 00:12:43,271][105692] Updated weights for policy 0, policy_version 1211917 (0.0009) [2023-12-27 00:12:43,837][105620] Updated weights for policy 1, policy_version 1213223 (0.0008) [2023-12-27 00:12:43,895][105692] Updated weights for policy 0, policy_version 1211927 (0.0009) [2023-12-27 00:12:43,897][105620] Updated weights for policy 1, policy_version 1213233 (0.0006) [2023-12-27 00:12:43,956][105692] Updated weights for policy 0, policy_version 1211937 (0.0010) [2023-12-27 00:12:43,958][105620] Updated weights for policy 1, policy_version 1213243 (0.0007) [2023-12-27 00:12:44,011][105692] Updated weights for policy 0, policy_version 1211947 (0.0010) [2023-12-27 00:12:44,687][105692] Updated weights for policy 0, policy_version 1211957 (0.0009) [2023-12-27 00:12:44,723][105620] Updated weights for policy 1, policy_version 1213253 (0.0007) [2023-12-27 00:12:44,734][105692] Updated weights for policy 0, policy_version 1211967 (0.0006) [2023-12-27 00:12:44,784][105620] Updated weights for policy 1, policy_version 1213263 (0.0007) [2023-12-27 00:12:44,788][105692] Updated weights for policy 0, policy_version 1211977 (0.0007) [2023-12-27 00:12:44,845][105620] Updated weights for policy 1, policy_version 1213273 (0.0006) [2023-12-27 00:12:45,531][105692] Updated weights for policy 0, policy_version 1211987 (0.0008) [2023-12-27 00:12:45,586][105692] Updated weights for policy 0, policy_version 1211997 (0.0009) [2023-12-27 00:12:45,611][105620] Updated weights for policy 1, policy_version 1213283 (0.0007) [2023-12-27 00:12:45,633][105692] Updated weights for policy 0, policy_version 1212007 (0.0007) [2023-12-27 00:12:45,660][105620] Updated weights for policy 1, policy_version 1213293 (0.0006) [2023-12-27 00:12:45,703][105620] Updated weights for policy 1, policy_version 1213303 (0.0007) [2023-12-27 00:12:46,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 620978176. Throughput: 0: 9903.1, 1: 9539.7. Samples: 620946912. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:12:46,063][104569] Avg episode reward: [(0, '9088.832'), (1, '8806.077')] [2023-12-27 00:12:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001212016_310329344.pth... [2023-12-27 00:12:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001213312_310648832.pth... [2023-12-27 00:12:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001212192_310362112.pth [2023-12-27 00:12:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001210880_310034432.pth [2023-12-27 00:12:46,401][105692] Updated weights for policy 0, policy_version 1212017 (0.0008) [2023-12-27 00:12:46,452][105692] Updated weights for policy 0, policy_version 1212027 (0.0009) [2023-12-27 00:12:46,473][105620] Updated weights for policy 1, policy_version 1213313 (0.0008) [2023-12-27 00:12:46,510][105692] Updated weights for policy 0, policy_version 1212037 (0.0007) [2023-12-27 00:12:46,538][105620] Updated weights for policy 1, policy_version 1213323 (0.0008) [2023-12-27 00:12:46,561][105692] Updated weights for policy 0, policy_version 1212047 (0.0009) [2023-12-27 00:12:46,594][105620] Updated weights for policy 1, policy_version 1213333 (0.0008) [2023-12-27 00:12:46,645][105620] Updated weights for policy 1, policy_version 1213344 (0.0010) [2023-12-27 00:12:47,156][105692] Updated weights for policy 0, policy_version 1212057 (0.0006) [2023-12-27 00:12:47,208][105692] Updated weights for policy 0, policy_version 1212067 (0.0005) [2023-12-27 00:12:47,263][105692] Updated weights for policy 0, policy_version 1212077 (0.0005) [2023-12-27 00:12:47,564][105620] Updated weights for policy 1, policy_version 1213354 (0.0010) [2023-12-27 00:12:47,617][105620] Updated weights for policy 1, policy_version 1213364 (0.0009) [2023-12-27 00:12:47,670][105620] Updated weights for policy 1, policy_version 1213375 (0.0009) [2023-12-27 00:12:47,787][105692] Updated weights for policy 0, policy_version 1212087 (0.0005) [2023-12-27 00:12:47,847][105692] Updated weights for policy 0, policy_version 1212097 (0.0005) [2023-12-27 00:12:47,896][105692] Updated weights for policy 0, policy_version 1212107 (0.0005) [2023-12-27 00:12:48,467][105620] Updated weights for policy 1, policy_version 1213385 (0.0008) [2023-12-27 00:12:48,490][105692] Updated weights for policy 0, policy_version 1212117 (0.0008) [2023-12-27 00:12:48,527][105620] Updated weights for policy 1, policy_version 1213395 (0.0006) [2023-12-27 00:12:48,549][105692] Updated weights for policy 0, policy_version 1212127 (0.0011) [2023-12-27 00:12:48,587][105620] Updated weights for policy 1, policy_version 1213405 (0.0005) [2023-12-27 00:12:48,616][105692] Updated weights for policy 0, policy_version 1212137 (0.0011) [2023-12-27 00:12:49,354][105620] Updated weights for policy 1, policy_version 1213415 (0.0008) [2023-12-27 00:12:49,362][105692] Updated weights for policy 0, policy_version 1212147 (0.0011) [2023-12-27 00:12:49,419][105620] Updated weights for policy 1, policy_version 1213425 (0.0008) [2023-12-27 00:12:49,427][105692] Updated weights for policy 0, policy_version 1212157 (0.0009) [2023-12-27 00:12:49,454][105585] KL-divergence is very high: 170.4124 [2023-12-27 00:12:49,483][105620] Updated weights for policy 1, policy_version 1213435 (0.0006) [2023-12-27 00:12:49,489][105692] Updated weights for policy 0, policy_version 1212167 (0.0008) [2023-12-27 00:12:49,501][105585] KL-divergence is very high: 182.5928 [2023-12-27 00:12:50,212][105692] Updated weights for policy 0, policy_version 1212177 (0.0008) [2023-12-27 00:12:50,252][105620] Updated weights for policy 1, policy_version 1213445 (0.0008) [2023-12-27 00:12:50,270][105692] Updated weights for policy 0, policy_version 1212187 (0.0007) [2023-12-27 00:12:50,311][105620] Updated weights for policy 1, policy_version 1213455 (0.0008) [2023-12-27 00:12:50,325][105692] Updated weights for policy 0, policy_version 1212197 (0.0007) [2023-12-27 00:12:50,370][105620] Updated weights for policy 1, policy_version 1213465 (0.0007) [2023-12-27 00:12:50,391][105692] Updated weights for policy 0, policy_version 1212207 (0.0005) [2023-12-27 00:12:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 621068288. Throughput: 0: 9994.4, 1: 9486.6. Samples: 621062664. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:12:51,062][104569] Avg episode reward: [(0, '8996.133'), (1, '9169.363')] [2023-12-27 00:12:51,098][105620] Updated weights for policy 1, policy_version 1213475 (0.0008) [2023-12-27 00:12:51,144][105692] Updated weights for policy 0, policy_version 1212217 (0.0007) [2023-12-27 00:12:51,159][105620] Updated weights for policy 1, policy_version 1213485 (0.0008) [2023-12-27 00:12:51,206][105692] Updated weights for policy 0, policy_version 1212227 (0.0007) [2023-12-27 00:12:51,222][105620] Updated weights for policy 1, policy_version 1213495 (0.0008) [2023-12-27 00:12:51,267][105692] Updated weights for policy 0, policy_version 1212237 (0.0008) [2023-12-27 00:12:51,870][105620] Updated weights for policy 1, policy_version 1213505 (0.0008) [2023-12-27 00:12:51,932][105620] Updated weights for policy 1, policy_version 1213515 (0.0005) [2023-12-27 00:12:52,003][105620] Updated weights for policy 1, policy_version 1213525 (0.0005) [2023-12-27 00:12:52,062][105620] Updated weights for policy 1, policy_version 1213535 (0.0008) [2023-12-27 00:12:52,090][105692] Updated weights for policy 0, policy_version 1212247 (0.0009) [2023-12-27 00:12:52,152][105692] Updated weights for policy 0, policy_version 1212257 (0.0008) [2023-12-27 00:12:52,208][105692] Updated weights for policy 0, policy_version 1212267 (0.0008) [2023-12-27 00:12:52,723][105620] Updated weights for policy 1, policy_version 1213545 (0.0009) [2023-12-27 00:12:52,769][105620] Updated weights for policy 1, policy_version 1213555 (0.0008) [2023-12-27 00:12:52,823][105620] Updated weights for policy 1, policy_version 1213565 (0.0005) [2023-12-27 00:12:52,930][105692] Updated weights for policy 0, policy_version 1212277 (0.0008) [2023-12-27 00:12:52,983][105692] Updated weights for policy 0, policy_version 1212287 (0.0009) [2023-12-27 00:12:53,041][105692] Updated weights for policy 0, policy_version 1212297 (0.0010) [2023-12-27 00:12:53,452][105620] Updated weights for policy 1, policy_version 1213575 (0.0008) [2023-12-27 00:12:53,505][105620] Updated weights for policy 1, policy_version 1213586 (0.0010) [2023-12-27 00:12:53,556][105620] Updated weights for policy 1, policy_version 1213596 (0.0009) [2023-12-27 00:12:53,687][105692] Updated weights for policy 0, policy_version 1212307 (0.0009) [2023-12-27 00:12:53,734][105692] Updated weights for policy 0, policy_version 1212317 (0.0009) [2023-12-27 00:12:53,790][105692] Updated weights for policy 0, policy_version 1212327 (0.0009) [2023-12-27 00:12:54,338][105620] Updated weights for policy 1, policy_version 1213607 (0.0010) [2023-12-27 00:12:54,387][105620] Updated weights for policy 1, policy_version 1213617 (0.0010) [2023-12-27 00:12:54,444][105620] Updated weights for policy 1, policy_version 1213627 (0.0010) [2023-12-27 00:12:54,493][105692] Updated weights for policy 0, policy_version 1212337 (0.0009) [2023-12-27 00:12:54,553][105692] Updated weights for policy 0, policy_version 1212347 (0.0008) [2023-12-27 00:12:54,608][105692] Updated weights for policy 0, policy_version 1212357 (0.0008) [2023-12-27 00:12:54,659][105692] Updated weights for policy 0, policy_version 1212367 (0.0005) [2023-12-27 00:12:55,247][105620] Updated weights for policy 1, policy_version 1213637 (0.0008) [2023-12-27 00:12:55,309][105620] Updated weights for policy 1, policy_version 1213647 (0.0009) [2023-12-27 00:12:55,362][105692] Updated weights for policy 0, policy_version 1212377 (0.0009) [2023-12-27 00:12:55,372][105620] Updated weights for policy 1, policy_version 1213657 (0.0006) [2023-12-27 00:12:55,421][105692] Updated weights for policy 0, policy_version 1212387 (0.0008) [2023-12-27 00:12:55,482][105692] Updated weights for policy 0, policy_version 1212397 (0.0010) [2023-12-27 00:12:56,062][104569] Fps is (10 sec: 18842.3, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 621166592. Throughput: 0: 9936.2, 1: 9556.6. Samples: 621178940. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:12:56,062][104569] Avg episode reward: [(0, '9176.778'), (1, '9171.514')] [2023-12-27 00:12:56,078][105620] Updated weights for policy 1, policy_version 1213667 (0.0005) [2023-12-27 00:12:56,123][105620] Updated weights for policy 1, policy_version 1213677 (0.0005) [2023-12-27 00:12:56,148][105692] Updated weights for policy 0, policy_version 1212407 (0.0005) [2023-12-27 00:12:56,171][105620] Updated weights for policy 1, policy_version 1213687 (0.0005) [2023-12-27 00:12:56,206][105692] Updated weights for policy 0, policy_version 1212417 (0.0007) [2023-12-27 00:12:56,267][105692] Updated weights for policy 0, policy_version 1212427 (0.0010) [2023-12-27 00:12:56,799][105620] Updated weights for policy 1, policy_version 1213697 (0.0006) [2023-12-27 00:12:56,859][105620] Updated weights for policy 1, policy_version 1213707 (0.0010) [2023-12-27 00:12:56,906][105620] Updated weights for policy 1, policy_version 1213717 (0.0010) [2023-12-27 00:12:56,940][105692] Updated weights for policy 0, policy_version 1212437 (0.0007) [2023-12-27 00:12:56,957][105620] Updated weights for policy 1, policy_version 1213727 (0.0010) [2023-12-27 00:12:56,997][105692] Updated weights for policy 0, policy_version 1212447 (0.0007) [2023-12-27 00:12:57,052][105692] Updated weights for policy 0, policy_version 1212457 (0.0008) [2023-12-27 00:12:57,686][105620] Updated weights for policy 1, policy_version 1213737 (0.0006) [2023-12-27 00:12:57,744][105620] Updated weights for policy 1, policy_version 1213747 (0.0005) [2023-12-27 00:12:57,790][105620] Updated weights for policy 1, policy_version 1213757 (0.0005) [2023-12-27 00:12:57,844][105692] Updated weights for policy 0, policy_version 1212467 (0.0009) [2023-12-27 00:12:57,898][105692] Updated weights for policy 0, policy_version 1212477 (0.0010) [2023-12-27 00:12:57,950][105692] Updated weights for policy 0, policy_version 1212487 (0.0010) [2023-12-27 00:12:58,395][105620] Updated weights for policy 1, policy_version 1213767 (0.0007) [2023-12-27 00:12:58,452][105620] Updated weights for policy 1, policy_version 1213777 (0.0008) [2023-12-27 00:12:58,515][105620] Updated weights for policy 1, policy_version 1213787 (0.0008) [2023-12-27 00:12:58,857][105692] Updated weights for policy 0, policy_version 1212498 (0.0009) [2023-12-27 00:12:58,922][105692] Updated weights for policy 0, policy_version 1212508 (0.0009) [2023-12-27 00:12:58,983][105692] Updated weights for policy 0, policy_version 1212518 (0.0008) [2023-12-27 00:12:59,044][105692] Updated weights for policy 0, policy_version 1212528 (0.0008) [2023-12-27 00:12:59,325][105620] Updated weights for policy 1, policy_version 1213797 (0.0008) [2023-12-27 00:12:59,397][105620] Updated weights for policy 1, policy_version 1213807 (0.0008) [2023-12-27 00:12:59,456][105620] Updated weights for policy 1, policy_version 1213817 (0.0010) [2023-12-27 00:12:59,838][105692] Updated weights for policy 0, policy_version 1212538 (0.0009) [2023-12-27 00:12:59,900][105692] Updated weights for policy 0, policy_version 1212548 (0.0008) [2023-12-27 00:12:59,964][105692] Updated weights for policy 0, policy_version 1212558 (0.0009) [2023-12-27 00:13:00,205][105620] Updated weights for policy 1, policy_version 1213827 (0.0010) [2023-12-27 00:13:00,258][105620] Updated weights for policy 1, policy_version 1213837 (0.0009) [2023-12-27 00:13:00,309][105620] Updated weights for policy 1, policy_version 1213847 (0.0009) [2023-12-27 00:13:00,630][105692] Updated weights for policy 0, policy_version 1212568 (0.0006) [2023-12-27 00:13:00,682][105692] Updated weights for policy 0, policy_version 1212578 (0.0005) [2023-12-27 00:13:00,732][105692] Updated weights for policy 0, policy_version 1212588 (0.0005) [2023-12-27 00:13:00,988][105620] Updated weights for policy 1, policy_version 1213857 (0.0008) [2023-12-27 00:13:01,044][105620] Updated weights for policy 1, policy_version 1213867 (0.0009) [2023-12-27 00:13:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 621264896. Throughput: 0: 9912.1, 1: 9639.4. Samples: 621237860. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:13:01,063][104569] Avg episode reward: [(0, '9267.169'), (1, '8989.816')] [2023-12-27 00:13:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001212592_310476800.pth... [2023-12-27 00:13:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001211440_310181888.pth [2023-12-27 00:13:01,111][105620] Updated weights for policy 1, policy_version 1213877 (0.0011) [2023-12-27 00:13:01,174][105620] Updated weights for policy 1, policy_version 1213887 (0.0011) [2023-12-27 00:13:01,180][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001213888_310796288.pth... [2023-12-27 00:13:01,184][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001212768_310509568.pth [2023-12-27 00:13:01,362][105692] Updated weights for policy 0, policy_version 1212598 (0.0009) [2023-12-27 00:13:01,430][105692] Updated weights for policy 0, policy_version 1212608 (0.0010) [2023-12-27 00:13:01,498][105692] Updated weights for policy 0, policy_version 1212618 (0.0008) [2023-12-27 00:13:01,888][105620] Updated weights for policy 1, policy_version 1213897 (0.0010) [2023-12-27 00:13:01,945][105620] Updated weights for policy 1, policy_version 1213907 (0.0010) [2023-12-27 00:13:01,994][105620] Updated weights for policy 1, policy_version 1213917 (0.0010) [2023-12-27 00:13:02,128][105692] Updated weights for policy 0, policy_version 1212628 (0.0007) [2023-12-27 00:13:02,174][105692] Updated weights for policy 0, policy_version 1212638 (0.0005) [2023-12-27 00:13:02,227][105692] Updated weights for policy 0, policy_version 1212648 (0.0005) [2023-12-27 00:13:02,719][105620] Updated weights for policy 1, policy_version 1213927 (0.0010) [2023-12-27 00:13:02,771][105620] Updated weights for policy 1, policy_version 1213937 (0.0010) [2023-12-27 00:13:02,824][105620] Updated weights for policy 1, policy_version 1213947 (0.0010) [2023-12-27 00:13:02,899][105692] Updated weights for policy 0, policy_version 1212658 (0.0006) [2023-12-27 00:13:02,957][105692] Updated weights for policy 0, policy_version 1212668 (0.0006) [2023-12-27 00:13:03,013][105692] Updated weights for policy 0, policy_version 1212678 (0.0006) [2023-12-27 00:13:03,064][105692] Updated weights for policy 0, policy_version 1212688 (0.0005) [2023-12-27 00:13:03,491][105620] Updated weights for policy 1, policy_version 1213957 (0.0008) [2023-12-27 00:13:03,541][105620] Updated weights for policy 1, policy_version 1213967 (0.0005) [2023-12-27 00:13:03,592][105620] Updated weights for policy 1, policy_version 1213977 (0.0006) [2023-12-27 00:13:03,671][105692] Updated weights for policy 0, policy_version 1212698 (0.0007) [2023-12-27 00:13:03,727][105692] Updated weights for policy 0, policy_version 1212708 (0.0010) [2023-12-27 00:13:03,779][105692] Updated weights for policy 0, policy_version 1212718 (0.0010) [2023-12-27 00:13:04,253][105620] Updated weights for policy 1, policy_version 1213987 (0.0009) [2023-12-27 00:13:04,305][105620] Updated weights for policy 1, policy_version 1213997 (0.0009) [2023-12-27 00:13:04,360][105620] Updated weights for policy 1, policy_version 1214007 (0.0010) [2023-12-27 00:13:04,518][105692] Updated weights for policy 0, policy_version 1212728 (0.0008) [2023-12-27 00:13:04,568][105692] Updated weights for policy 0, policy_version 1212738 (0.0008) [2023-12-27 00:13:04,624][105692] Updated weights for policy 0, policy_version 1212748 (0.0008) [2023-12-27 00:13:05,108][105620] Updated weights for policy 1, policy_version 1214017 (0.0007) [2023-12-27 00:13:05,162][105620] Updated weights for policy 1, policy_version 1214027 (0.0005) [2023-12-27 00:13:05,214][105620] Updated weights for policy 1, policy_version 1214037 (0.0006) [2023-12-27 00:13:05,270][105620] Updated weights for policy 1, policy_version 1214047 (0.0008) [2023-12-27 00:13:05,435][105692] Updated weights for policy 0, policy_version 1212758 (0.0009) [2023-12-27 00:13:05,495][105692] Updated weights for policy 0, policy_version 1212768 (0.0008) [2023-12-27 00:13:05,554][105692] Updated weights for policy 0, policy_version 1212778 (0.0008) [2023-12-27 00:13:05,937][105620] Updated weights for policy 1, policy_version 1214057 (0.0010) [2023-12-27 00:13:05,991][105620] Updated weights for policy 1, policy_version 1214067 (0.0010) [2023-12-27 00:13:06,039][105620] Updated weights for policy 1, policy_version 1214077 (0.0010) [2023-12-27 00:13:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 621371392. Throughput: 0: 9899.0, 1: 9641.8. Samples: 621357176. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:13:06,062][104569] Avg episode reward: [(0, '9175.042'), (1, '8980.385')] [2023-12-27 00:13:06,321][105692] Updated weights for policy 0, policy_version 1212788 (0.0009) [2023-12-27 00:13:06,375][105692] Updated weights for policy 0, policy_version 1212798 (0.0009) [2023-12-27 00:13:06,435][105692] Updated weights for policy 0, policy_version 1212808 (0.0009) [2023-12-27 00:13:06,758][105620] Updated weights for policy 1, policy_version 1214087 (0.0009) [2023-12-27 00:13:06,823][105620] Updated weights for policy 1, policy_version 1214097 (0.0009) [2023-12-27 00:13:06,887][105620] Updated weights for policy 1, policy_version 1214107 (0.0009) [2023-12-27 00:13:07,248][105692] Updated weights for policy 0, policy_version 1212818 (0.0009) [2023-12-27 00:13:07,303][105692] Updated weights for policy 0, policy_version 1212828 (0.0009) [2023-12-27 00:13:07,361][105692] Updated weights for policy 0, policy_version 1212838 (0.0010) [2023-12-27 00:13:07,420][105692] Updated weights for policy 0, policy_version 1212848 (0.0010) [2023-12-27 00:13:07,537][105620] Updated weights for policy 1, policy_version 1214117 (0.0009) [2023-12-27 00:13:07,587][105620] Updated weights for policy 1, policy_version 1214127 (0.0008) [2023-12-27 00:13:07,641][105620] Updated weights for policy 1, policy_version 1214137 (0.0009) [2023-12-27 00:13:08,204][105692] Updated weights for policy 0, policy_version 1212858 (0.0010) [2023-12-27 00:13:08,258][105692] Updated weights for policy 0, policy_version 1212870 (0.0010) [2023-12-27 00:13:08,318][105692] Updated weights for policy 0, policy_version 1212880 (0.0007) [2023-12-27 00:13:08,363][105620] Updated weights for policy 1, policy_version 1214147 (0.0009) [2023-12-27 00:13:08,425][105620] Updated weights for policy 1, policy_version 1214157 (0.0009) [2023-12-27 00:13:08,482][105620] Updated weights for policy 1, policy_version 1214167 (0.0005) [2023-12-27 00:13:09,065][105692] Updated weights for policy 0, policy_version 1212890 (0.0010) [2023-12-27 00:13:09,118][105692] Updated weights for policy 0, policy_version 1212900 (0.0010) [2023-12-27 00:13:09,172][105692] Updated weights for policy 0, policy_version 1212910 (0.0010) [2023-12-27 00:13:09,278][105620] Updated weights for policy 1, policy_version 1214177 (0.0007) [2023-12-27 00:13:09,340][105620] Updated weights for policy 1, policy_version 1214187 (0.0011) [2023-12-27 00:13:09,410][105620] Updated weights for policy 1, policy_version 1214197 (0.0009) [2023-12-27 00:13:09,465][105620] Updated weights for policy 1, policy_version 1214207 (0.0009) [2023-12-27 00:13:09,933][105692] Updated weights for policy 0, policy_version 1212920 (0.0009) [2023-12-27 00:13:10,001][105692] Updated weights for policy 0, policy_version 1212930 (0.0009) [2023-12-27 00:13:10,070][105692] Updated weights for policy 0, policy_version 1212940 (0.0009) [2023-12-27 00:13:10,275][105620] Updated weights for policy 1, policy_version 1214217 (0.0008) [2023-12-27 00:13:10,339][105620] Updated weights for policy 1, policy_version 1214227 (0.0008) [2023-12-27 00:13:10,410][105620] Updated weights for policy 1, policy_version 1214237 (0.0009) [2023-12-27 00:13:10,691][105692] Updated weights for policy 0, policy_version 1212950 (0.0009) [2023-12-27 00:13:10,758][105692] Updated weights for policy 0, policy_version 1212960 (0.0010) [2023-12-27 00:13:10,820][105692] Updated weights for policy 0, policy_version 1212970 (0.0009) [2023-12-27 00:13:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 621461504. Throughput: 0: 9883.1, 1: 9598.2. Samples: 621469924. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:13:11,062][104569] Avg episode reward: [(0, '9175.632'), (1, '5596.578')] [2023-12-27 00:13:11,215][105620] Updated weights for policy 1, policy_version 1214247 (0.0007) [2023-12-27 00:13:11,285][105620] Updated weights for policy 1, policy_version 1214257 (0.0007) [2023-12-27 00:13:11,351][105620] Updated weights for policy 1, policy_version 1214267 (0.0008) [2023-12-27 00:13:11,554][105692] Updated weights for policy 0, policy_version 1212980 (0.0008) [2023-12-27 00:13:11,603][105692] Updated weights for policy 0, policy_version 1212990 (0.0006) [2023-12-27 00:13:11,678][105692] Updated weights for policy 0, policy_version 1213000 (0.0010) [2023-12-27 00:13:12,095][105620] Updated weights for policy 1, policy_version 1214277 (0.0009) [2023-12-27 00:13:12,157][105620] Updated weights for policy 1, policy_version 1214287 (0.0009) [2023-12-27 00:13:12,209][105620] Updated weights for policy 1, policy_version 1214297 (0.0010) [2023-12-27 00:13:12,434][105692] Updated weights for policy 0, policy_version 1213010 (0.0009) [2023-12-27 00:13:12,492][105692] Updated weights for policy 0, policy_version 1213020 (0.0006) [2023-12-27 00:13:12,555][105692] Updated weights for policy 0, policy_version 1213030 (0.0008) [2023-12-27 00:13:12,617][105692] Updated weights for policy 0, policy_version 1213040 (0.0009) [2023-12-27 00:13:12,942][105620] Updated weights for policy 1, policy_version 1214307 (0.0009) [2023-12-27 00:13:13,006][105620] Updated weights for policy 1, policy_version 1214317 (0.0009) [2023-12-27 00:13:13,062][105620] Updated weights for policy 1, policy_version 1214327 (0.0013) [2023-12-27 00:13:13,302][105692] Updated weights for policy 0, policy_version 1213050 (0.0010) [2023-12-27 00:13:13,356][105692] Updated weights for policy 0, policy_version 1213060 (0.0008) [2023-12-27 00:13:13,415][105692] Updated weights for policy 0, policy_version 1213070 (0.0009) [2023-12-27 00:13:13,858][105620] Updated weights for policy 1, policy_version 1214337 (0.0009) [2023-12-27 00:13:13,904][105620] Updated weights for policy 1, policy_version 1214347 (0.0008) [2023-12-27 00:13:13,964][105620] Updated weights for policy 1, policy_version 1214358 (0.0009) [2023-12-27 00:13:14,025][105620] Updated weights for policy 1, policy_version 1214368 (0.0010) [2023-12-27 00:13:14,160][105692] Updated weights for policy 0, policy_version 1213080 (0.0009) [2023-12-27 00:13:14,211][105692] Updated weights for policy 0, policy_version 1213090 (0.0009) [2023-12-27 00:13:14,271][105692] Updated weights for policy 0, policy_version 1213100 (0.0009) [2023-12-27 00:13:14,800][105620] Updated weights for policy 1, policy_version 1214378 (0.0009) [2023-12-27 00:13:14,862][105620] Updated weights for policy 1, policy_version 1214388 (0.0009) [2023-12-27 00:13:14,931][105620] Updated weights for policy 1, policy_version 1214398 (0.0009) [2023-12-27 00:13:15,048][105692] Updated weights for policy 0, policy_version 1213110 (0.0007) [2023-12-27 00:13:15,115][105692] Updated weights for policy 0, policy_version 1213120 (0.0008) [2023-12-27 00:13:15,176][105692] Updated weights for policy 0, policy_version 1213130 (0.0008) [2023-12-27 00:13:15,671][105620] Updated weights for policy 1, policy_version 1214408 (0.0010) [2023-12-27 00:13:15,725][105620] Updated weights for policy 1, policy_version 1214418 (0.0010) [2023-12-27 00:13:15,783][105620] Updated weights for policy 1, policy_version 1214428 (0.0010) [2023-12-27 00:13:15,909][105692] Updated weights for policy 0, policy_version 1213140 (0.0008) [2023-12-27 00:13:15,958][105692] Updated weights for policy 0, policy_version 1213150 (0.0008) [2023-12-27 00:13:16,002][105692] Updated weights for policy 0, policy_version 1213160 (0.0008) [2023-12-27 00:13:16,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 621559808. Throughput: 0: 9817.7, 1: 9551.0. Samples: 621525624. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:13:16,063][104569] Avg episode reward: [(0, '9088.503'), (1, '5253.442')] [2023-12-27 00:13:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001213168_310624256.pth... [2023-12-27 00:13:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001214432_310935552.pth... [2023-12-27 00:13:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001213312_310648832.pth [2023-12-27 00:13:16,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001212016_310329344.pth [2023-12-27 00:13:16,467][105620] Updated weights for policy 1, policy_version 1214438 (0.0010) [2023-12-27 00:13:16,522][105620] Updated weights for policy 1, policy_version 1214448 (0.0009) [2023-12-27 00:13:16,573][105620] Updated weights for policy 1, policy_version 1214458 (0.0010) [2023-12-27 00:13:16,819][105692] Updated weights for policy 0, policy_version 1213170 (0.0008) [2023-12-27 00:13:16,866][105692] Updated weights for policy 0, policy_version 1213180 (0.0009) [2023-12-27 00:13:16,914][105692] Updated weights for policy 0, policy_version 1213190 (0.0009) [2023-12-27 00:13:16,963][105692] Updated weights for policy 0, policy_version 1213200 (0.0009) [2023-12-27 00:13:17,287][105620] Updated weights for policy 1, policy_version 1214468 (0.0010) [2023-12-27 00:13:17,357][105620] Updated weights for policy 1, policy_version 1214478 (0.0010) [2023-12-27 00:13:17,415][105620] Updated weights for policy 1, policy_version 1214488 (0.0009) [2023-12-27 00:13:17,729][105692] Updated weights for policy 0, policy_version 1213210 (0.0009) [2023-12-27 00:13:17,791][105692] Updated weights for policy 0, policy_version 1213220 (0.0009) [2023-12-27 00:13:17,857][105692] Updated weights for policy 0, policy_version 1213230 (0.0009) [2023-12-27 00:13:18,129][105620] Updated weights for policy 1, policy_version 1214498 (0.0009) [2023-12-27 00:13:18,183][105620] Updated weights for policy 1, policy_version 1214508 (0.0009) [2023-12-27 00:13:18,233][105620] Updated weights for policy 1, policy_version 1214518 (0.0008) [2023-12-27 00:13:18,284][105620] Updated weights for policy 1, policy_version 1214528 (0.0009) [2023-12-27 00:13:18,599][105692] Updated weights for policy 0, policy_version 1213240 (0.0009) [2023-12-27 00:13:18,650][105692] Updated weights for policy 0, policy_version 1213250 (0.0009) [2023-12-27 00:13:18,702][105692] Updated weights for policy 0, policy_version 1213260 (0.0009) [2023-12-27 00:13:19,073][105620] Updated weights for policy 1, policy_version 1214538 (0.0008) [2023-12-27 00:13:19,123][105620] Updated weights for policy 1, policy_version 1214548 (0.0008) [2023-12-27 00:13:19,174][105620] Updated weights for policy 1, policy_version 1214558 (0.0009) [2023-12-27 00:13:19,526][105692] Updated weights for policy 0, policy_version 1213270 (0.0008) [2023-12-27 00:13:19,592][105692] Updated weights for policy 0, policy_version 1213281 (0.0009) [2023-12-27 00:13:19,646][105692] Updated weights for policy 0, policy_version 1213291 (0.0009) [2023-12-27 00:13:19,971][105620] Updated weights for policy 1, policy_version 1214568 (0.0007) [2023-12-27 00:13:20,040][105620] Updated weights for policy 1, policy_version 1214578 (0.0007) [2023-12-27 00:13:20,102][105620] Updated weights for policy 1, policy_version 1214588 (0.0008) [2023-12-27 00:13:20,506][105692] Updated weights for policy 0, policy_version 1213301 (0.0009) [2023-12-27 00:13:20,571][105692] Updated weights for policy 0, policy_version 1213311 (0.0007) [2023-12-27 00:13:20,633][105692] Updated weights for policy 0, policy_version 1213321 (0.0009) [2023-12-27 00:13:20,700][105620] Updated weights for policy 1, policy_version 1214598 (0.0006) [2023-12-27 00:13:20,760][105620] Updated weights for policy 1, policy_version 1214608 (0.0006) [2023-12-27 00:13:20,816][105620] Updated weights for policy 1, policy_version 1214618 (0.0006) [2023-12-27 00:13:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 621649920. Throughput: 0: 9705.5, 1: 9463.4. Samples: 621637796. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:13:21,062][104569] Avg episode reward: [(0, '8906.525'), (1, '7106.884')] [2023-12-27 00:13:21,400][105692] Updated weights for policy 0, policy_version 1213331 (0.0009) [2023-12-27 00:13:21,464][105692] Updated weights for policy 0, policy_version 1213341 (0.0009) [2023-12-27 00:13:21,528][105692] Updated weights for policy 0, policy_version 1213351 (0.0010) [2023-12-27 00:13:21,604][105620] Updated weights for policy 1, policy_version 1214628 (0.0009) [2023-12-27 00:13:21,670][105620] Updated weights for policy 1, policy_version 1214638 (0.0009) [2023-12-27 00:13:21,738][105620] Updated weights for policy 1, policy_version 1214648 (0.0009) [2023-12-27 00:13:22,280][105692] Updated weights for policy 0, policy_version 1213361 (0.0009) [2023-12-27 00:13:22,343][105692] Updated weights for policy 0, policy_version 1213371 (0.0007) [2023-12-27 00:13:22,420][105692] Updated weights for policy 0, policy_version 1213381 (0.0010) [2023-12-27 00:13:22,445][105620] Updated weights for policy 1, policy_version 1214658 (0.0008) [2023-12-27 00:13:22,478][105692] Updated weights for policy 0, policy_version 1213391 (0.0010) [2023-12-27 00:13:22,507][105620] Updated weights for policy 1, policy_version 1214668 (0.0009) [2023-12-27 00:13:22,560][105620] Updated weights for policy 1, policy_version 1214678 (0.0008) [2023-12-27 00:13:22,613][105620] Updated weights for policy 1, policy_version 1214688 (0.0007) [2023-12-27 00:13:23,153][105692] Updated weights for policy 0, policy_version 1213401 (0.0011) [2023-12-27 00:13:23,199][105692] Updated weights for policy 0, policy_version 1213411 (0.0010) [2023-12-27 00:13:23,261][105692] Updated weights for policy 0, policy_version 1213421 (0.0011) [2023-12-27 00:13:23,375][105620] Updated weights for policy 1, policy_version 1214698 (0.0008) [2023-12-27 00:13:23,426][105620] Updated weights for policy 1, policy_version 1214708 (0.0007) [2023-12-27 00:13:23,473][105620] Updated weights for policy 1, policy_version 1214718 (0.0008) [2023-12-27 00:13:23,990][105692] Updated weights for policy 0, policy_version 1213431 (0.0011) [2023-12-27 00:13:24,038][105692] Updated weights for policy 0, policy_version 1213441 (0.0010) [2023-12-27 00:13:24,082][105692] Updated weights for policy 0, policy_version 1213451 (0.0010) [2023-12-27 00:13:24,238][105620] Updated weights for policy 1, policy_version 1214728 (0.0008) [2023-12-27 00:13:24,282][105620] Updated weights for policy 1, policy_version 1214738 (0.0008) [2023-12-27 00:13:24,331][105620] Updated weights for policy 1, policy_version 1214748 (0.0010) [2023-12-27 00:13:24,850][105692] Updated weights for policy 0, policy_version 1213461 (0.0010) [2023-12-27 00:13:24,899][105692] Updated weights for policy 0, policy_version 1213471 (0.0006) [2023-12-27 00:13:24,950][105692] Updated weights for policy 0, policy_version 1213481 (0.0010) [2023-12-27 00:13:25,100][105620] Updated weights for policy 1, policy_version 1214758 (0.0010) [2023-12-27 00:13:25,154][105620] Updated weights for policy 1, policy_version 1214768 (0.0009) [2023-12-27 00:13:25,209][105620] Updated weights for policy 1, policy_version 1214778 (0.0010) [2023-12-27 00:13:25,623][105692] Updated weights for policy 0, policy_version 1213491 (0.0009) [2023-12-27 00:13:25,684][105692] Updated weights for policy 0, policy_version 1213501 (0.0005) [2023-12-27 00:13:25,735][105692] Updated weights for policy 0, policy_version 1213511 (0.0005) [2023-12-27 00:13:25,958][105620] Updated weights for policy 1, policy_version 1214788 (0.0010) [2023-12-27 00:13:26,013][105620] Updated weights for policy 1, policy_version 1214798 (0.0010) [2023-12-27 00:13:26,062][104569] Fps is (10 sec: 18022.9, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 621740032. Throughput: 0: 9627.6, 1: 9475.4. Samples: 621750896. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:13:26,062][104569] Avg episode reward: [(0, '8539.075'), (1, '7791.357')] [2023-12-27 00:13:26,068][105620] Updated weights for policy 1, policy_version 1214808 (0.0010) [2023-12-27 00:13:26,252][105692] Updated weights for policy 0, policy_version 1213521 (0.0005) [2023-12-27 00:13:26,326][105692] Updated weights for policy 0, policy_version 1213531 (0.0005) [2023-12-27 00:13:26,394][105692] Updated weights for policy 0, policy_version 1213541 (0.0010) [2023-12-27 00:13:26,454][105692] Updated weights for policy 0, policy_version 1213551 (0.0005) [2023-12-27 00:13:26,639][105620] Updated weights for policy 1, policy_version 1214818 (0.0008) [2023-12-27 00:13:26,692][105620] Updated weights for policy 1, policy_version 1214828 (0.0005) [2023-12-27 00:13:26,755][105620] Updated weights for policy 1, policy_version 1214838 (0.0005) [2023-12-27 00:13:26,807][105620] Updated weights for policy 1, policy_version 1214848 (0.0005) [2023-12-27 00:13:26,952][105692] Updated weights for policy 0, policy_version 1213561 (0.0005) [2023-12-27 00:13:27,008][105692] Updated weights for policy 0, policy_version 1213571 (0.0005) [2023-12-27 00:13:27,074][105692] Updated weights for policy 0, policy_version 1213581 (0.0005) [2023-12-27 00:13:27,396][105620] Updated weights for policy 1, policy_version 1214858 (0.0005) [2023-12-27 00:13:27,451][105620] Updated weights for policy 1, policy_version 1214868 (0.0005) [2023-12-27 00:13:27,511][105620] Updated weights for policy 1, policy_version 1214878 (0.0005) [2023-12-27 00:13:27,590][105692] Updated weights for policy 0, policy_version 1213591 (0.0005) [2023-12-27 00:13:27,641][105692] Updated weights for policy 0, policy_version 1213601 (0.0005) [2023-12-27 00:13:27,696][105692] Updated weights for policy 0, policy_version 1213611 (0.0005) [2023-12-27 00:13:28,040][105620] Updated weights for policy 1, policy_version 1214888 (0.0005) [2023-12-27 00:13:28,086][105620] Updated weights for policy 1, policy_version 1214898 (0.0005) [2023-12-27 00:13:28,131][105620] Updated weights for policy 1, policy_version 1214908 (0.0005) [2023-12-27 00:13:28,227][105692] Updated weights for policy 0, policy_version 1213621 (0.0005) [2023-12-27 00:13:28,281][105692] Updated weights for policy 0, policy_version 1213631 (0.0005) [2023-12-27 00:13:28,335][105692] Updated weights for policy 0, policy_version 1213641 (0.0006) [2023-12-27 00:13:28,685][105620] Updated weights for policy 1, policy_version 1214918 (0.0005) [2023-12-27 00:13:28,741][105620] Updated weights for policy 1, policy_version 1214928 (0.0006) [2023-12-27 00:13:28,795][105620] Updated weights for policy 1, policy_version 1214938 (0.0010) [2023-12-27 00:13:28,964][105692] Updated weights for policy 0, policy_version 1213651 (0.0009) [2023-12-27 00:13:29,032][105692] Updated weights for policy 0, policy_version 1213661 (0.0006) [2023-12-27 00:13:29,087][105692] Updated weights for policy 0, policy_version 1213671 (0.0006) [2023-12-27 00:13:29,524][105620] Updated weights for policy 1, policy_version 1214948 (0.0010) [2023-12-27 00:13:29,586][105620] Updated weights for policy 1, policy_version 1214958 (0.0010) [2023-12-27 00:13:29,643][105620] Updated weights for policy 1, policy_version 1214968 (0.0007) [2023-12-27 00:13:29,819][105692] Updated weights for policy 0, policy_version 1213681 (0.0006) [2023-12-27 00:13:29,892][105692] Updated weights for policy 0, policy_version 1213691 (0.0007) [2023-12-27 00:13:29,965][105692] Updated weights for policy 0, policy_version 1213701 (0.0007) [2023-12-27 00:13:30,025][105692] Updated weights for policy 0, policy_version 1213711 (0.0006) [2023-12-27 00:13:30,324][105620] Updated weights for policy 1, policy_version 1214978 (0.0010) [2023-12-27 00:13:30,390][105620] Updated weights for policy 1, policy_version 1214988 (0.0009) [2023-12-27 00:13:30,457][105620] Updated weights for policy 1, policy_version 1214998 (0.0009) [2023-12-27 00:13:30,517][105620] Updated weights for policy 1, policy_version 1215008 (0.0009) [2023-12-27 00:13:30,568][105692] Updated weights for policy 0, policy_version 1213721 (0.0005) [2023-12-27 00:13:30,616][105692] Updated weights for policy 0, policy_version 1213731 (0.0005) [2023-12-27 00:13:30,667][105692] Updated weights for policy 0, policy_version 1213741 (0.0005) [2023-12-27 00:13:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 621854720. Throughput: 0: 9792.7, 1: 9686.8. Samples: 621823484. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:13:31,062][104569] Avg episode reward: [(0, '8811.012'), (1, '8822.090')] [2023-12-27 00:13:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001213744_310771712.pth... [2023-12-27 00:13:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001215008_311083008.pth... [2023-12-27 00:13:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001212592_310476800.pth [2023-12-27 00:13:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001213888_310796288.pth [2023-12-27 00:13:31,208][105692] Updated weights for policy 0, policy_version 1213751 (0.0005) [2023-12-27 00:13:31,259][105620] Updated weights for policy 1, policy_version 1215018 (0.0006) [2023-12-27 00:13:31,272][105692] Updated weights for policy 0, policy_version 1213761 (0.0007) [2023-12-27 00:13:31,319][105620] Updated weights for policy 1, policy_version 1215028 (0.0008) [2023-12-27 00:13:31,333][105692] Updated weights for policy 0, policy_version 1213771 (0.0008) [2023-12-27 00:13:31,385][105620] Updated weights for policy 1, policy_version 1215038 (0.0009) [2023-12-27 00:13:31,988][105620] Updated weights for policy 1, policy_version 1215048 (0.0009) [2023-12-27 00:13:31,998][105692] Updated weights for policy 0, policy_version 1213781 (0.0010) [2023-12-27 00:13:32,040][105620] Updated weights for policy 1, policy_version 1215058 (0.0010) [2023-12-27 00:13:32,060][105692] Updated weights for policy 0, policy_version 1213791 (0.0011) [2023-12-27 00:13:32,099][105620] Updated weights for policy 1, policy_version 1215068 (0.0010) [2023-12-27 00:13:32,119][105692] Updated weights for policy 0, policy_version 1213801 (0.0011) [2023-12-27 00:13:32,742][105620] Updated weights for policy 1, policy_version 1215078 (0.0007) [2023-12-27 00:13:32,797][105620] Updated weights for policy 1, policy_version 1215088 (0.0005) [2023-12-27 00:13:32,856][105620] Updated weights for policy 1, policy_version 1215098 (0.0005) [2023-12-27 00:13:32,861][105692] Updated weights for policy 0, policy_version 1213811 (0.0010) [2023-12-27 00:13:32,910][105692] Updated weights for policy 0, policy_version 1213821 (0.0010) [2023-12-27 00:13:32,973][105692] Updated weights for policy 0, policy_version 1213831 (0.0011) [2023-12-27 00:13:33,439][105620] Updated weights for policy 1, policy_version 1215108 (0.0006) [2023-12-27 00:13:33,495][105620] Updated weights for policy 1, policy_version 1215118 (0.0009) [2023-12-27 00:13:33,544][105620] Updated weights for policy 1, policy_version 1215128 (0.0009) [2023-12-27 00:13:33,688][105692] Updated weights for policy 0, policy_version 1213841 (0.0010) [2023-12-27 00:13:33,754][105692] Updated weights for policy 0, policy_version 1213851 (0.0009) [2023-12-27 00:13:33,804][105692] Updated weights for policy 0, policy_version 1213861 (0.0005) [2023-12-27 00:13:33,853][105692] Updated weights for policy 0, policy_version 1213871 (0.0005) [2023-12-27 00:13:34,304][105620] Updated weights for policy 1, policy_version 1215138 (0.0009) [2023-12-27 00:13:34,372][105620] Updated weights for policy 1, policy_version 1215148 (0.0009) [2023-12-27 00:13:34,434][105620] Updated weights for policy 1, policy_version 1215158 (0.0009) [2023-12-27 00:13:34,482][105692] Updated weights for policy 0, policy_version 1213881 (0.0006) [2023-12-27 00:13:34,496][105620] Updated weights for policy 1, policy_version 1215168 (0.0007) [2023-12-27 00:13:34,543][105692] Updated weights for policy 0, policy_version 1213891 (0.0008) [2023-12-27 00:13:34,603][105692] Updated weights for policy 0, policy_version 1213901 (0.0009) [2023-12-27 00:13:35,235][105620] Updated weights for policy 1, policy_version 1215178 (0.0009) [2023-12-27 00:13:35,286][105620] Updated weights for policy 1, policy_version 1215188 (0.0009) [2023-12-27 00:13:35,332][105620] Updated weights for policy 1, policy_version 1215198 (0.0008) [2023-12-27 00:13:35,358][105692] Updated weights for policy 0, policy_version 1213911 (0.0009) [2023-12-27 00:13:35,404][105692] Updated weights for policy 0, policy_version 1213921 (0.0008) [2023-12-27 00:13:35,460][105692] Updated weights for policy 0, policy_version 1213931 (0.0009) [2023-12-27 00:13:36,029][105620] Updated weights for policy 1, policy_version 1215208 (0.0009) [2023-12-27 00:13:36,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 621953024. Throughput: 0: 9801.4, 1: 9845.4. Samples: 621946772. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:13:36,062][104569] Avg episode reward: [(0, '9175.102'), (1, '9262.877')] [2023-12-27 00:13:36,096][105620] Updated weights for policy 1, policy_version 1215218 (0.0011) [2023-12-27 00:13:36,162][105620] Updated weights for policy 1, policy_version 1215228 (0.0010) [2023-12-27 00:13:36,217][105692] Updated weights for policy 0, policy_version 1213941 (0.0010) [2023-12-27 00:13:36,287][105692] Updated weights for policy 0, policy_version 1213951 (0.0011) [2023-12-27 00:13:36,350][105692] Updated weights for policy 0, policy_version 1213961 (0.0011) [2023-12-27 00:13:36,916][105620] Updated weights for policy 1, policy_version 1215238 (0.0011) [2023-12-27 00:13:36,981][105620] Updated weights for policy 1, policy_version 1215248 (0.0010) [2023-12-27 00:13:37,033][105620] Updated weights for policy 1, policy_version 1215258 (0.0010) [2023-12-27 00:13:37,078][105692] Updated weights for policy 0, policy_version 1213971 (0.0011) [2023-12-27 00:13:37,143][105692] Updated weights for policy 0, policy_version 1213981 (0.0011) [2023-12-27 00:13:37,212][105692] Updated weights for policy 0, policy_version 1213991 (0.0011) [2023-12-27 00:13:37,739][105620] Updated weights for policy 1, policy_version 1215268 (0.0009) [2023-12-27 00:13:37,801][105620] Updated weights for policy 1, policy_version 1215278 (0.0005) [2023-12-27 00:13:37,854][105620] Updated weights for policy 1, policy_version 1215288 (0.0007) [2023-12-27 00:13:37,935][105692] Updated weights for policy 0, policy_version 1214001 (0.0011) [2023-12-27 00:13:37,986][105692] Updated weights for policy 0, policy_version 1214011 (0.0010) [2023-12-27 00:13:38,037][105692] Updated weights for policy 0, policy_version 1214021 (0.0010) [2023-12-27 00:13:38,092][105692] Updated weights for policy 0, policy_version 1214031 (0.0010) [2023-12-27 00:13:38,498][105620] Updated weights for policy 1, policy_version 1215298 (0.0010) [2023-12-27 00:13:38,544][105620] Updated weights for policy 1, policy_version 1215308 (0.0008) [2023-12-27 00:13:38,604][105620] Updated weights for policy 1, policy_version 1215318 (0.0008) [2023-12-27 00:13:38,665][105620] Updated weights for policy 1, policy_version 1215328 (0.0006) [2023-12-27 00:13:38,824][105692] Updated weights for policy 0, policy_version 1214041 (0.0010) [2023-12-27 00:13:38,896][105692] Updated weights for policy 0, policy_version 1214051 (0.0010) [2023-12-27 00:13:38,959][105692] Updated weights for policy 0, policy_version 1214061 (0.0010) [2023-12-27 00:13:39,382][105620] Updated weights for policy 1, policy_version 1215338 (0.0009) [2023-12-27 00:13:39,451][105620] Updated weights for policy 1, policy_version 1215348 (0.0010) [2023-12-27 00:13:39,511][105620] Updated weights for policy 1, policy_version 1215358 (0.0011) [2023-12-27 00:13:39,644][105692] Updated weights for policy 0, policy_version 1214071 (0.0010) [2023-12-27 00:13:39,712][105692] Updated weights for policy 0, policy_version 1214081 (0.0010) [2023-12-27 00:13:39,772][105692] Updated weights for policy 0, policy_version 1214091 (0.0010) [2023-12-27 00:13:40,219][105620] Updated weights for policy 1, policy_version 1215368 (0.0011) [2023-12-27 00:13:40,272][105620] Updated weights for policy 1, policy_version 1215378 (0.0011) [2023-12-27 00:13:40,322][105620] Updated weights for policy 1, policy_version 1215388 (0.0011) [2023-12-27 00:13:40,499][105692] Updated weights for policy 0, policy_version 1214101 (0.0008) [2023-12-27 00:13:40,545][105692] Updated weights for policy 0, policy_version 1214111 (0.0006) [2023-12-27 00:13:40,602][105692] Updated weights for policy 0, policy_version 1214121 (0.0008) [2023-12-27 00:13:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 622051328. Throughput: 0: 9793.7, 1: 9836.3. Samples: 622062288. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:13:41,063][104569] Avg episode reward: [(0, '9265.031'), (1, '9261.509')] [2023-12-27 00:13:41,100][105620] Updated weights for policy 1, policy_version 1215398 (0.0010) [2023-12-27 00:13:41,166][105620] Updated weights for policy 1, policy_version 1215408 (0.0010) [2023-12-27 00:13:41,226][105620] Updated weights for policy 1, policy_version 1215418 (0.0011) [2023-12-27 00:13:41,324][105692] Updated weights for policy 0, policy_version 1214131 (0.0008) [2023-12-27 00:13:41,390][105692] Updated weights for policy 0, policy_version 1214141 (0.0008) [2023-12-27 00:13:41,446][105692] Updated weights for policy 0, policy_version 1214151 (0.0008) [2023-12-27 00:13:42,063][105620] Updated weights for policy 1, policy_version 1215428 (0.0009) [2023-12-27 00:13:42,125][105620] Updated weights for policy 1, policy_version 1215438 (0.0009) [2023-12-27 00:13:42,192][105620] Updated weights for policy 1, policy_version 1215448 (0.0009) [2023-12-27 00:13:42,256][105692] Updated weights for policy 0, policy_version 1214161 (0.0009) [2023-12-27 00:13:42,322][105692] Updated weights for policy 0, policy_version 1214171 (0.0009) [2023-12-27 00:13:42,391][105692] Updated weights for policy 0, policy_version 1214181 (0.0009) [2023-12-27 00:13:42,458][105692] Updated weights for policy 0, policy_version 1214191 (0.0010) [2023-12-27 00:13:42,960][105620] Updated weights for policy 1, policy_version 1215458 (0.0008) [2023-12-27 00:13:43,012][105620] Updated weights for policy 1, policy_version 1215469 (0.0010) [2023-12-27 00:13:43,067][105620] Updated weights for policy 1, policy_version 1215479 (0.0010) [2023-12-27 00:13:43,135][105692] Updated weights for policy 0, policy_version 1214201 (0.0006) [2023-12-27 00:13:43,185][105692] Updated weights for policy 0, policy_version 1214211 (0.0005) [2023-12-27 00:13:43,234][105692] Updated weights for policy 0, policy_version 1214221 (0.0005) [2023-12-27 00:13:43,784][105620] Updated weights for policy 1, policy_version 1215489 (0.0008) [2023-12-27 00:13:43,840][105620] Updated weights for policy 1, policy_version 1215499 (0.0010) [2023-12-27 00:13:43,900][105620] Updated weights for policy 1, policy_version 1215509 (0.0010) [2023-12-27 00:13:43,942][105692] Updated weights for policy 0, policy_version 1214231 (0.0007) [2023-12-27 00:13:43,956][105620] Updated weights for policy 1, policy_version 1215519 (0.0010) [2023-12-27 00:13:44,004][105692] Updated weights for policy 0, policy_version 1214241 (0.0007) [2023-12-27 00:13:44,071][105692] Updated weights for policy 0, policy_version 1214251 (0.0008) [2023-12-27 00:13:44,651][105620] Updated weights for policy 1, policy_version 1215529 (0.0006) [2023-12-27 00:13:44,708][105620] Updated weights for policy 1, policy_version 1215539 (0.0006) [2023-12-27 00:13:44,762][105620] Updated weights for policy 1, policy_version 1215549 (0.0006) [2023-12-27 00:13:44,909][105692] Updated weights for policy 0, policy_version 1214261 (0.0009) [2023-12-27 00:13:44,972][105692] Updated weights for policy 0, policy_version 1214271 (0.0008) [2023-12-27 00:13:45,045][105692] Updated weights for policy 0, policy_version 1214281 (0.0010) [2023-12-27 00:13:45,388][105620] Updated weights for policy 1, policy_version 1215559 (0.0010) [2023-12-27 00:13:45,455][105620] Updated weights for policy 1, policy_version 1215569 (0.0011) [2023-12-27 00:13:45,522][105620] Updated weights for policy 1, policy_version 1215579 (0.0011) [2023-12-27 00:13:45,832][105692] Updated weights for policy 0, policy_version 1214291 (0.0009) [2023-12-27 00:13:45,885][105692] Updated weights for policy 0, policy_version 1214301 (0.0008) [2023-12-27 00:13:45,935][105692] Updated weights for policy 0, policy_version 1214311 (0.0008) [2023-12-27 00:13:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 622149632. Throughput: 0: 9802.9, 1: 9759.1. Samples: 622118152. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:13:46,062][104569] Avg episode reward: [(0, '9173.983'), (1, '9350.764')] [2023-12-27 00:13:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001214320_310919168.pth... [2023-12-27 00:13:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001215584_311230464.pth... [2023-12-27 00:13:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001214432_310935552.pth [2023-12-27 00:13:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001213168_310624256.pth [2023-12-27 00:13:46,233][105620] Updated weights for policy 1, policy_version 1215589 (0.0011) [2023-12-27 00:13:46,298][105620] Updated weights for policy 1, policy_version 1215599 (0.0010) [2023-12-27 00:13:46,356][105620] Updated weights for policy 1, policy_version 1215609 (0.0010) [2023-12-27 00:13:46,687][105692] Updated weights for policy 0, policy_version 1214321 (0.0008) [2023-12-27 00:13:46,740][105692] Updated weights for policy 0, policy_version 1214331 (0.0008) [2023-12-27 00:13:46,803][105692] Updated weights for policy 0, policy_version 1214341 (0.0009) [2023-12-27 00:13:46,865][105692] Updated weights for policy 0, policy_version 1214351 (0.0008) [2023-12-27 00:13:47,094][105620] Updated weights for policy 1, policy_version 1215619 (0.0010) [2023-12-27 00:13:47,146][105620] Updated weights for policy 1, policy_version 1215629 (0.0010) [2023-12-27 00:13:47,194][105620] Updated weights for policy 1, policy_version 1215639 (0.0010) [2023-12-27 00:13:47,627][105692] Updated weights for policy 0, policy_version 1214361 (0.0008) [2023-12-27 00:13:47,679][105692] Updated weights for policy 0, policy_version 1214371 (0.0008) [2023-12-27 00:13:47,738][105692] Updated weights for policy 0, policy_version 1214381 (0.0008) [2023-12-27 00:13:47,881][105620] Updated weights for policy 1, policy_version 1215649 (0.0010) [2023-12-27 00:13:47,929][105620] Updated weights for policy 1, policy_version 1215659 (0.0010) [2023-12-27 00:13:47,984][105620] Updated weights for policy 1, policy_version 1215669 (0.0010) [2023-12-27 00:13:48,047][105620] Updated weights for policy 1, policy_version 1215679 (0.0010) [2023-12-27 00:13:48,451][105692] Updated weights for policy 0, policy_version 1214391 (0.0008) [2023-12-27 00:13:48,511][105692] Updated weights for policy 0, policy_version 1214401 (0.0008) [2023-12-27 00:13:48,566][105692] Updated weights for policy 0, policy_version 1214411 (0.0008) [2023-12-27 00:13:48,814][105620] Updated weights for policy 1, policy_version 1215689 (0.0010) [2023-12-27 00:13:48,877][105620] Updated weights for policy 1, policy_version 1215699 (0.0010) [2023-12-27 00:13:48,940][105620] Updated weights for policy 1, policy_version 1215709 (0.0011) [2023-12-27 00:13:49,354][105692] Updated weights for policy 0, policy_version 1214421 (0.0008) [2023-12-27 00:13:49,416][105692] Updated weights for policy 0, policy_version 1214431 (0.0008) [2023-12-27 00:13:49,478][105692] Updated weights for policy 0, policy_version 1214441 (0.0007) [2023-12-27 00:13:49,697][105620] Updated weights for policy 1, policy_version 1215719 (0.0011) [2023-12-27 00:13:49,746][105620] Updated weights for policy 1, policy_version 1215729 (0.0010) [2023-12-27 00:13:49,809][105620] Updated weights for policy 1, policy_version 1215739 (0.0011) [2023-12-27 00:13:50,139][105692] Updated weights for policy 0, policy_version 1214451 (0.0007) [2023-12-27 00:13:50,200][105692] Updated weights for policy 0, policy_version 1214461 (0.0011) [2023-12-27 00:13:50,253][105692] Updated weights for policy 0, policy_version 1214471 (0.0010) [2023-12-27 00:13:50,563][105620] Updated weights for policy 1, policy_version 1215749 (0.0009) [2023-12-27 00:13:50,631][105620] Updated weights for policy 1, policy_version 1215759 (0.0011) [2023-12-27 00:13:50,696][105620] Updated weights for policy 1, policy_version 1215769 (0.0011) [2023-12-27 00:13:50,984][105692] Updated weights for policy 0, policy_version 1214481 (0.0010) [2023-12-27 00:13:51,045][105692] Updated weights for policy 0, policy_version 1214491 (0.0008) [2023-12-27 00:13:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 622239744. Throughput: 0: 9672.5, 1: 9748.9. Samples: 622231140. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:13:51,063][104569] Avg episode reward: [(0, '9264.568'), (1, '9257.542')] [2023-12-27 00:13:51,118][105692] Updated weights for policy 0, policy_version 1214501 (0.0010) [2023-12-27 00:13:51,180][105692] Updated weights for policy 0, policy_version 1214511 (0.0010) [2023-12-27 00:13:51,339][105620] Updated weights for policy 1, policy_version 1215779 (0.0010) [2023-12-27 00:13:51,410][105620] Updated weights for policy 1, policy_version 1215789 (0.0009) [2023-12-27 00:13:51,475][105620] Updated weights for policy 1, policy_version 1215799 (0.0005) [2023-12-27 00:13:51,975][105692] Updated weights for policy 0, policy_version 1214521 (0.0010) [2023-12-27 00:13:52,024][105692] Updated weights for policy 0, policy_version 1214531 (0.0010) [2023-12-27 00:13:52,079][105692] Updated weights for policy 0, policy_version 1214541 (0.0010) [2023-12-27 00:13:52,155][105620] Updated weights for policy 1, policy_version 1215809 (0.0008) [2023-12-27 00:13:52,206][105620] Updated weights for policy 1, policy_version 1215819 (0.0007) [2023-12-27 00:13:52,263][105620] Updated weights for policy 1, policy_version 1215829 (0.0008) [2023-12-27 00:13:52,326][105620] Updated weights for policy 1, policy_version 1215839 (0.0010) [2023-12-27 00:13:52,771][105692] Updated weights for policy 0, policy_version 1214551 (0.0007) [2023-12-27 00:13:52,838][105692] Updated weights for policy 0, policy_version 1214561 (0.0005) [2023-12-27 00:13:52,901][105692] Updated weights for policy 0, policy_version 1214571 (0.0007) [2023-12-27 00:13:53,032][105620] Updated weights for policy 1, policy_version 1215849 (0.0006) [2023-12-27 00:13:53,090][105620] Updated weights for policy 1, policy_version 1215859 (0.0005) [2023-12-27 00:13:53,156][105620] Updated weights for policy 1, policy_version 1215869 (0.0005) [2023-12-27 00:13:53,643][105692] Updated weights for policy 0, policy_version 1214581 (0.0009) [2023-12-27 00:13:53,653][105620] Updated weights for policy 1, policy_version 1215879 (0.0007) [2023-12-27 00:13:53,704][105692] Updated weights for policy 0, policy_version 1214591 (0.0009) [2023-12-27 00:13:53,712][105620] Updated weights for policy 1, policy_version 1215889 (0.0005) [2023-12-27 00:13:53,761][105692] Updated weights for policy 0, policy_version 1214601 (0.0007) [2023-12-27 00:13:53,761][105620] Updated weights for policy 1, policy_version 1215899 (0.0008) [2023-12-27 00:13:54,494][105620] Updated weights for policy 1, policy_version 1215909 (0.0008) [2023-12-27 00:13:54,533][105692] Updated weights for policy 0, policy_version 1214611 (0.0008) [2023-12-27 00:13:54,557][105620] Updated weights for policy 1, policy_version 1215919 (0.0007) [2023-12-27 00:13:54,597][105692] Updated weights for policy 0, policy_version 1214621 (0.0007) [2023-12-27 00:13:54,624][105620] Updated weights for policy 1, policy_version 1215929 (0.0007) [2023-12-27 00:13:54,650][105692] Updated weights for policy 0, policy_version 1214631 (0.0007) [2023-12-27 00:13:55,239][105692] Updated weights for policy 0, policy_version 1214641 (0.0005) [2023-12-27 00:13:55,310][105692] Updated weights for policy 0, policy_version 1214651 (0.0008) [2023-12-27 00:13:55,322][105620] Updated weights for policy 1, policy_version 1215939 (0.0009) [2023-12-27 00:13:55,369][105692] Updated weights for policy 0, policy_version 1214661 (0.0010) [2023-12-27 00:13:55,374][105620] Updated weights for policy 1, policy_version 1215949 (0.0010) [2023-12-27 00:13:55,426][105620] Updated weights for policy 1, policy_version 1215959 (0.0010) [2023-12-27 00:13:55,428][105692] Updated weights for policy 0, policy_version 1214671 (0.0005) [2023-12-27 00:13:56,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 622338048. Throughput: 0: 9746.6, 1: 9840.9. Samples: 622351360. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:13:56,063][104569] Avg episode reward: [(0, '9355.715'), (1, '9165.601')] [2023-12-27 00:13:56,115][105620] Updated weights for policy 1, policy_version 1215969 (0.0010) [2023-12-27 00:13:56,170][105620] Updated weights for policy 1, policy_version 1215979 (0.0010) [2023-12-27 00:13:56,192][105692] Updated weights for policy 0, policy_version 1214681 (0.0006) [2023-12-27 00:13:56,226][105620] Updated weights for policy 1, policy_version 1215989 (0.0010) [2023-12-27 00:13:56,248][105692] Updated weights for policy 0, policy_version 1214691 (0.0006) [2023-12-27 00:13:56,284][105620] Updated weights for policy 1, policy_version 1215999 (0.0010) [2023-12-27 00:13:56,307][105692] Updated weights for policy 0, policy_version 1214701 (0.0006) [2023-12-27 00:13:56,930][105620] Updated weights for policy 1, policy_version 1216009 (0.0006) [2023-12-27 00:13:56,980][105620] Updated weights for policy 1, policy_version 1216019 (0.0005) [2023-12-27 00:13:57,037][105620] Updated weights for policy 1, policy_version 1216029 (0.0005) [2023-12-27 00:13:57,146][105692] Updated weights for policy 0, policy_version 1214711 (0.0008) [2023-12-27 00:13:57,192][105692] Updated weights for policy 0, policy_version 1214721 (0.0007) [2023-12-27 00:13:57,240][105692] Updated weights for policy 0, policy_version 1214731 (0.0008) [2023-12-27 00:13:57,599][105620] Updated weights for policy 1, policy_version 1216039 (0.0005) [2023-12-27 00:13:57,660][105620] Updated weights for policy 1, policy_version 1216049 (0.0006) [2023-12-27 00:13:57,710][105620] Updated weights for policy 1, policy_version 1216059 (0.0006) [2023-12-27 00:13:58,086][105692] Updated weights for policy 0, policy_version 1214741 (0.0009) [2023-12-27 00:13:58,153][105692] Updated weights for policy 0, policy_version 1214751 (0.0010) [2023-12-27 00:13:58,216][105585] KL-divergence is very high: 101.3151 [2023-12-27 00:13:58,217][105692] Updated weights for policy 0, policy_version 1214761 (0.0010) [2023-12-27 00:13:58,289][105620] Updated weights for policy 1, policy_version 1216069 (0.0010) [2023-12-27 00:13:58,352][105620] Updated weights for policy 1, policy_version 1216079 (0.0009) [2023-12-27 00:13:58,415][105620] Updated weights for policy 1, policy_version 1216089 (0.0012) [2023-12-27 00:13:59,011][105692] Updated weights for policy 0, policy_version 1214771 (0.0011) [2023-12-27 00:13:59,074][105692] Updated weights for policy 0, policy_version 1214781 (0.0011) [2023-12-27 00:13:59,136][105692] Updated weights for policy 0, policy_version 1214791 (0.0011) [2023-12-27 00:13:59,273][105620] Updated weights for policy 1, policy_version 1216099 (0.0010) [2023-12-27 00:13:59,338][105620] Updated weights for policy 1, policy_version 1216109 (0.0007) [2023-12-27 00:13:59,406][105620] Updated weights for policy 1, policy_version 1216119 (0.0007) [2023-12-27 00:13:59,868][105692] Updated weights for policy 0, policy_version 1214801 (0.0009) [2023-12-27 00:13:59,935][105692] Updated weights for policy 0, policy_version 1214811 (0.0008) [2023-12-27 00:13:59,999][105692] Updated weights for policy 0, policy_version 1214821 (0.0009) [2023-12-27 00:14:00,058][105692] Updated weights for policy 0, policy_version 1214831 (0.0009) [2023-12-27 00:14:00,170][105620] Updated weights for policy 1, policy_version 1216129 (0.0009) [2023-12-27 00:14:00,227][105620] Updated weights for policy 1, policy_version 1216139 (0.0009) [2023-12-27 00:14:00,280][105620] Updated weights for policy 1, policy_version 1216149 (0.0009) [2023-12-27 00:14:00,328][105620] Updated weights for policy 1, policy_version 1216159 (0.0009) [2023-12-27 00:14:00,813][105692] Updated weights for policy 0, policy_version 1214841 (0.0010) [2023-12-27 00:14:00,873][105692] Updated weights for policy 0, policy_version 1214851 (0.0010) [2023-12-27 00:14:00,932][105692] Updated weights for policy 0, policy_version 1214861 (0.0009) [2023-12-27 00:14:01,052][105620] Updated weights for policy 1, policy_version 1216169 (0.0008) [2023-12-27 00:14:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 622436352. Throughput: 0: 9696.4, 1: 9933.4. Samples: 622408960. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:14:01,062][104569] Avg episode reward: [(0, '9355.875'), (1, '9074.961')] [2023-12-27 00:14:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001214864_311058432.pth... [2023-12-27 00:14:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001213744_310771712.pth [2023-12-27 00:14:01,109][105620] Updated weights for policy 1, policy_version 1216179 (0.0006) [2023-12-27 00:14:01,177][105620] Updated weights for policy 1, policy_version 1216189 (0.0007) [2023-12-27 00:14:01,195][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001216192_311386112.pth... [2023-12-27 00:14:01,199][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001215008_311083008.pth [2023-12-27 00:14:01,764][105692] Updated weights for policy 0, policy_version 1214871 (0.0009) [2023-12-27 00:14:01,819][105692] Updated weights for policy 0, policy_version 1214881 (0.0009) [2023-12-27 00:14:01,858][105620] Updated weights for policy 1, policy_version 1216199 (0.0007) [2023-12-27 00:14:01,876][105692] Updated weights for policy 0, policy_version 1214891 (0.0006) [2023-12-27 00:14:01,922][105620] Updated weights for policy 1, policy_version 1216209 (0.0008) [2023-12-27 00:14:01,980][105620] Updated weights for policy 1, policy_version 1216219 (0.0009) [2023-12-27 00:14:02,653][105692] Updated weights for policy 0, policy_version 1214901 (0.0008) [2023-12-27 00:14:02,704][105692] Updated weights for policy 0, policy_version 1214911 (0.0009) [2023-12-27 00:14:02,738][105620] Updated weights for policy 1, policy_version 1216229 (0.0008) [2023-12-27 00:14:02,760][105692] Updated weights for policy 0, policy_version 1214921 (0.0008) [2023-12-27 00:14:02,789][105620] Updated weights for policy 1, policy_version 1216239 (0.0008) [2023-12-27 00:14:02,838][105620] Updated weights for policy 1, policy_version 1216249 (0.0009) [2023-12-27 00:14:03,469][105692] Updated weights for policy 0, policy_version 1214931 (0.0007) [2023-12-27 00:14:03,513][105692] Updated weights for policy 0, policy_version 1214941 (0.0005) [2023-12-27 00:14:03,561][105692] Updated weights for policy 0, policy_version 1214951 (0.0007) [2023-12-27 00:14:03,589][105620] Updated weights for policy 1, policy_version 1216259 (0.0008) [2023-12-27 00:14:03,641][105620] Updated weights for policy 1, policy_version 1216269 (0.0007) [2023-12-27 00:14:03,687][105620] Updated weights for policy 1, policy_version 1216279 (0.0009) [2023-12-27 00:14:04,305][105692] Updated weights for policy 0, policy_version 1214961 (0.0008) [2023-12-27 00:14:04,363][105692] Updated weights for policy 0, policy_version 1214971 (0.0008) [2023-12-27 00:14:04,426][105692] Updated weights for policy 0, policy_version 1214981 (0.0009) [2023-12-27 00:14:04,441][105620] Updated weights for policy 1, policy_version 1216289 (0.0009) [2023-12-27 00:14:04,490][105692] Updated weights for policy 0, policy_version 1214991 (0.0008) [2023-12-27 00:14:04,502][105620] Updated weights for policy 1, policy_version 1216299 (0.0008) [2023-12-27 00:14:04,557][105620] Updated weights for policy 1, policy_version 1216309 (0.0009) [2023-12-27 00:14:04,611][105620] Updated weights for policy 1, policy_version 1216319 (0.0008) [2023-12-27 00:14:05,236][105620] Updated weights for policy 1, policy_version 1216329 (0.0006) [2023-12-27 00:14:05,288][105620] Updated weights for policy 1, policy_version 1216339 (0.0005) [2023-12-27 00:14:05,328][105692] Updated weights for policy 0, policy_version 1215001 (0.0009) [2023-12-27 00:14:05,338][105620] Updated weights for policy 1, policy_version 1216349 (0.0005) [2023-12-27 00:14:05,381][105692] Updated weights for policy 0, policy_version 1215011 (0.0010) [2023-12-27 00:14:05,434][105692] Updated weights for policy 0, policy_version 1215021 (0.0009) [2023-12-27 00:14:05,922][105620] Updated weights for policy 1, policy_version 1216359 (0.0009) [2023-12-27 00:14:05,974][105620] Updated weights for policy 1, policy_version 1216369 (0.0011) [2023-12-27 00:14:06,030][105620] Updated weights for policy 1, policy_version 1216379 (0.0011) [2023-12-27 00:14:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 622534656. Throughput: 0: 9688.8, 1: 9937.7. Samples: 622520992. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 00:14:06,063][104569] Avg episode reward: [(0, '9090.442'), (1, '9075.664')] [2023-12-27 00:14:06,193][105692] Updated weights for policy 0, policy_version 1215031 (0.0009) [2023-12-27 00:14:06,246][105692] Updated weights for policy 0, policy_version 1215041 (0.0008) [2023-12-27 00:14:06,306][105692] Updated weights for policy 0, policy_version 1215051 (0.0008) [2023-12-27 00:14:06,821][105620] Updated weights for policy 1, policy_version 1216389 (0.0011) [2023-12-27 00:14:06,885][105620] Updated weights for policy 1, policy_version 1216399 (0.0011) [2023-12-27 00:14:06,942][105620] Updated weights for policy 1, policy_version 1216409 (0.0007) [2023-12-27 00:14:07,109][105692] Updated weights for policy 0, policy_version 1215061 (0.0009) [2023-12-27 00:14:07,168][105692] Updated weights for policy 0, policy_version 1215071 (0.0010) [2023-12-27 00:14:07,219][105692] Updated weights for policy 0, policy_version 1215081 (0.0009) [2023-12-27 00:14:07,626][105620] Updated weights for policy 1, policy_version 1216419 (0.0010) [2023-12-27 00:14:07,680][105620] Updated weights for policy 1, policy_version 1216429 (0.0009) [2023-12-27 00:14:07,742][105620] Updated weights for policy 1, policy_version 1216439 (0.0009) [2023-12-27 00:14:07,974][105692] Updated weights for policy 0, policy_version 1215091 (0.0008) [2023-12-27 00:14:08,034][105692] Updated weights for policy 0, policy_version 1215101 (0.0007) [2023-12-27 00:14:08,093][105692] Updated weights for policy 0, policy_version 1215111 (0.0010) [2023-12-27 00:14:08,141][105585] KL-divergence is very high: 111.5627 [2023-12-27 00:14:08,507][105620] Updated weights for policy 1, policy_version 1216449 (0.0008) [2023-12-27 00:14:08,559][105620] Updated weights for policy 1, policy_version 1216459 (0.0008) [2023-12-27 00:14:08,612][105620] Updated weights for policy 1, policy_version 1216469 (0.0008) [2023-12-27 00:14:08,665][105620] Updated weights for policy 1, policy_version 1216479 (0.0008) [2023-12-27 00:14:08,793][105692] Updated weights for policy 0, policy_version 1215121 (0.0009) [2023-12-27 00:14:08,838][105692] Updated weights for policy 0, policy_version 1215131 (0.0011) [2023-12-27 00:14:08,896][105692] Updated weights for policy 0, policy_version 1215141 (0.0011) [2023-12-27 00:14:08,950][105692] Updated weights for policy 0, policy_version 1215151 (0.0010) [2023-12-27 00:14:09,553][105620] Updated weights for policy 1, policy_version 1216489 (0.0009) [2023-12-27 00:14:09,621][105620] Updated weights for policy 1, policy_version 1216499 (0.0007) [2023-12-27 00:14:09,626][105692] Updated weights for policy 0, policy_version 1215161 (0.0006) [2023-12-27 00:14:09,676][105692] Updated weights for policy 0, policy_version 1215171 (0.0007) [2023-12-27 00:14:09,682][105620] Updated weights for policy 1, policy_version 1216509 (0.0008) [2023-12-27 00:14:09,741][105692] Updated weights for policy 0, policy_version 1215181 (0.0008) [2023-12-27 00:14:10,420][105620] Updated weights for policy 1, policy_version 1216519 (0.0007) [2023-12-27 00:14:10,487][105620] Updated weights for policy 1, policy_version 1216529 (0.0007) [2023-12-27 00:14:10,497][105692] Updated weights for policy 0, policy_version 1215191 (0.0009) [2023-12-27 00:14:10,543][105620] Updated weights for policy 1, policy_version 1216539 (0.0007) [2023-12-27 00:14:10,565][105692] Updated weights for policy 0, policy_version 1215201 (0.0008) [2023-12-27 00:14:10,636][105692] Updated weights for policy 0, policy_version 1215211 (0.0008) [2023-12-27 00:14:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 622624768. Throughput: 0: 9692.2, 1: 9950.7. Samples: 622634828. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:14:11,063][104569] Avg episode reward: [(0, '8552.109'), (1, '8897.225')] [2023-12-27 00:14:11,250][105620] Updated weights for policy 1, policy_version 1216549 (0.0008) [2023-12-27 00:14:11,309][105620] Updated weights for policy 1, policy_version 1216559 (0.0008) [2023-12-27 00:14:11,337][105692] Updated weights for policy 0, policy_version 1215221 (0.0009) [2023-12-27 00:14:11,386][105620] Updated weights for policy 1, policy_version 1216569 (0.0009) [2023-12-27 00:14:11,403][105692] Updated weights for policy 0, policy_version 1215231 (0.0007) [2023-12-27 00:14:11,466][105692] Updated weights for policy 0, policy_version 1215241 (0.0006) [2023-12-27 00:14:12,182][105620] Updated weights for policy 1, policy_version 1216579 (0.0007) [2023-12-27 00:14:12,226][105692] Updated weights for policy 0, policy_version 1215251 (0.0006) [2023-12-27 00:14:12,235][105620] Updated weights for policy 1, policy_version 1216589 (0.0005) [2023-12-27 00:14:12,286][105692] Updated weights for policy 0, policy_version 1215261 (0.0008) [2023-12-27 00:14:12,297][105620] Updated weights for policy 1, policy_version 1216599 (0.0006) [2023-12-27 00:14:12,349][105692] Updated weights for policy 0, policy_version 1215271 (0.0008) [2023-12-27 00:14:13,056][105620] Updated weights for policy 1, policy_version 1216609 (0.0007) [2023-12-27 00:14:13,076][105692] Updated weights for policy 0, policy_version 1215281 (0.0008) [2023-12-27 00:14:13,117][105620] Updated weights for policy 1, policy_version 1216619 (0.0007) [2023-12-27 00:14:13,134][105692] Updated weights for policy 0, policy_version 1215291 (0.0008) [2023-12-27 00:14:13,182][105620] Updated weights for policy 1, policy_version 1216629 (0.0005) [2023-12-27 00:14:13,196][105692] Updated weights for policy 0, policy_version 1215301 (0.0009) [2023-12-27 00:14:13,246][105620] Updated weights for policy 1, policy_version 1216639 (0.0005) [2023-12-27 00:14:13,252][105692] Updated weights for policy 0, policy_version 1215312 (0.0007) [2023-12-27 00:14:13,867][105692] Updated weights for policy 0, policy_version 1215322 (0.0009) [2023-12-27 00:14:13,915][105620] Updated weights for policy 1, policy_version 1216649 (0.0006) [2023-12-27 00:14:13,928][105692] Updated weights for policy 0, policy_version 1215332 (0.0011) [2023-12-27 00:14:13,980][105620] Updated weights for policy 1, policy_version 1216659 (0.0009) [2023-12-27 00:14:13,984][105692] Updated weights for policy 0, policy_version 1215342 (0.0009) [2023-12-27 00:14:14,033][105620] Updated weights for policy 1, policy_version 1216669 (0.0010) [2023-12-27 00:14:14,649][105692] Updated weights for policy 0, policy_version 1215352 (0.0006) [2023-12-27 00:14:14,710][105692] Updated weights for policy 0, policy_version 1215362 (0.0006) [2023-12-27 00:14:14,764][105692] Updated weights for policy 0, policy_version 1215372 (0.0006) [2023-12-27 00:14:14,863][105620] Updated weights for policy 1, policy_version 1216679 (0.0010) [2023-12-27 00:14:14,927][105620] Updated weights for policy 1, policy_version 1216689 (0.0008) [2023-12-27 00:14:14,992][105620] Updated weights for policy 1, policy_version 1216699 (0.0009) [2023-12-27 00:14:15,444][105692] Updated weights for policy 0, policy_version 1215382 (0.0008) [2023-12-27 00:14:15,501][105692] Updated weights for policy 0, policy_version 1215392 (0.0009) [2023-12-27 00:14:15,557][105692] Updated weights for policy 0, policy_version 1215402 (0.0009) [2023-12-27 00:14:15,728][105620] Updated weights for policy 1, policy_version 1216709 (0.0008) [2023-12-27 00:14:15,786][105620] Updated weights for policy 1, policy_version 1216719 (0.0009) [2023-12-27 00:14:15,848][105620] Updated weights for policy 1, policy_version 1216729 (0.0009) [2023-12-27 00:14:16,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 622723072. Throughput: 0: 9509.2, 1: 9786.7. Samples: 622691796. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:14:16,062][104569] Avg episode reward: [(0, '8012.004'), (1, '8896.803')] [2023-12-27 00:14:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001216736_311525376.pth... [2023-12-27 00:14:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001215408_311197696.pth... [2023-12-27 00:14:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001215584_311230464.pth [2023-12-27 00:14:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001214320_310919168.pth [2023-12-27 00:14:16,325][105692] Updated weights for policy 0, policy_version 1215412 (0.0009) [2023-12-27 00:14:16,382][105692] Updated weights for policy 0, policy_version 1215422 (0.0008) [2023-12-27 00:14:16,445][105692] Updated weights for policy 0, policy_version 1215432 (0.0009) [2023-12-27 00:14:16,497][105620] Updated weights for policy 1, policy_version 1216739 (0.0008) [2023-12-27 00:14:16,554][105620] Updated weights for policy 1, policy_version 1216749 (0.0005) [2023-12-27 00:14:16,607][105620] Updated weights for policy 1, policy_version 1216759 (0.0005) [2023-12-27 00:14:17,065][105692] Updated weights for policy 0, policy_version 1215442 (0.0009) [2023-12-27 00:14:17,126][105692] Updated weights for policy 0, policy_version 1215452 (0.0009) [2023-12-27 00:14:17,184][105692] Updated weights for policy 0, policy_version 1215462 (0.0009) [2023-12-27 00:14:17,234][105692] Updated weights for policy 0, policy_version 1215472 (0.0009) [2023-12-27 00:14:17,297][105620] Updated weights for policy 1, policy_version 1216769 (0.0006) [2023-12-27 00:14:17,347][105620] Updated weights for policy 1, policy_version 1216779 (0.0009) [2023-12-27 00:14:17,404][105620] Updated weights for policy 1, policy_version 1216789 (0.0009) [2023-12-27 00:14:17,462][105620] Updated weights for policy 1, policy_version 1216799 (0.0009) [2023-12-27 00:14:17,980][105692] Updated weights for policy 0, policy_version 1215482 (0.0008) [2023-12-27 00:14:18,026][105692] Updated weights for policy 0, policy_version 1215492 (0.0009) [2023-12-27 00:14:18,081][105692] Updated weights for policy 0, policy_version 1215502 (0.0008) [2023-12-27 00:14:18,213][105620] Updated weights for policy 1, policy_version 1216809 (0.0008) [2023-12-27 00:14:18,268][105620] Updated weights for policy 1, policy_version 1216819 (0.0008) [2023-12-27 00:14:18,316][105620] Updated weights for policy 1, policy_version 1216829 (0.0009) [2023-12-27 00:14:18,843][105692] Updated weights for policy 0, policy_version 1215512 (0.0010) [2023-12-27 00:14:18,897][105692] Updated weights for policy 0, policy_version 1215522 (0.0010) [2023-12-27 00:14:18,956][105692] Updated weights for policy 0, policy_version 1215532 (0.0010) [2023-12-27 00:14:18,987][105620] Updated weights for policy 1, policy_version 1216839 (0.0007) [2023-12-27 00:14:19,047][105620] Updated weights for policy 1, policy_version 1216849 (0.0008) [2023-12-27 00:14:19,109][105620] Updated weights for policy 1, policy_version 1216859 (0.0009) [2023-12-27 00:14:19,712][105692] Updated weights for policy 0, policy_version 1215542 (0.0008) [2023-12-27 00:14:19,760][105692] Updated weights for policy 0, policy_version 1215552 (0.0010) [2023-12-27 00:14:19,816][105692] Updated weights for policy 0, policy_version 1215562 (0.0009) [2023-12-27 00:14:19,916][105620] Updated weights for policy 1, policy_version 1216869 (0.0009) [2023-12-27 00:14:19,987][105620] Updated weights for policy 1, policy_version 1216879 (0.0006) [2023-12-27 00:14:20,055][105620] Updated weights for policy 1, policy_version 1216889 (0.0008) [2023-12-27 00:14:20,680][105692] Updated weights for policy 0, policy_version 1215572 (0.0010) [2023-12-27 00:14:20,721][105620] Updated weights for policy 1, policy_version 1216899 (0.0008) [2023-12-27 00:14:20,736][105692] Updated weights for policy 0, policy_version 1215582 (0.0009) [2023-12-27 00:14:20,778][105620] Updated weights for policy 1, policy_version 1216909 (0.0009) [2023-12-27 00:14:20,799][105692] Updated weights for policy 0, policy_version 1215592 (0.0009) [2023-12-27 00:14:20,848][105620] Updated weights for policy 1, policy_version 1216919 (0.0010) [2023-12-27 00:14:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 622821376. Throughput: 0: 9436.1, 1: 9713.9. Samples: 622808528. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:14:21,062][104569] Avg episode reward: [(0, '8551.589'), (1, '8984.697')] [2023-12-27 00:14:21,541][105620] Updated weights for policy 1, policy_version 1216929 (0.0007) [2023-12-27 00:14:21,603][105620] Updated weights for policy 1, policy_version 1216939 (0.0009) [2023-12-27 00:14:21,615][105692] Updated weights for policy 0, policy_version 1215602 (0.0007) [2023-12-27 00:14:21,663][105620] Updated weights for policy 1, policy_version 1216949 (0.0007) [2023-12-27 00:14:21,677][105692] Updated weights for policy 0, policy_version 1215612 (0.0009) [2023-12-27 00:14:21,727][105620] Updated weights for policy 1, policy_version 1216959 (0.0007) [2023-12-27 00:14:21,745][105692] Updated weights for policy 0, policy_version 1215622 (0.0008) [2023-12-27 00:14:21,808][105692] Updated weights for policy 0, policy_version 1215632 (0.0009) [2023-12-27 00:14:22,425][105620] Updated weights for policy 1, policy_version 1216969 (0.0007) [2023-12-27 00:14:22,481][105620] Updated weights for policy 1, policy_version 1216979 (0.0006) [2023-12-27 00:14:22,543][105620] Updated weights for policy 1, policy_version 1216989 (0.0006) [2023-12-27 00:14:22,583][105692] Updated weights for policy 0, policy_version 1215642 (0.0009) [2023-12-27 00:14:22,638][105692] Updated weights for policy 0, policy_version 1215652 (0.0009) [2023-12-27 00:14:22,688][105692] Updated weights for policy 0, policy_version 1215662 (0.0009) [2023-12-27 00:14:23,293][105620] Updated weights for policy 1, policy_version 1216999 (0.0008) [2023-12-27 00:14:23,352][105620] Updated weights for policy 1, policy_version 1217009 (0.0008) [2023-12-27 00:14:23,407][105620] Updated weights for policy 1, policy_version 1217019 (0.0008) [2023-12-27 00:14:23,464][105692] Updated weights for policy 0, policy_version 1215672 (0.0010) [2023-12-27 00:14:23,509][105692] Updated weights for policy 0, policy_version 1215682 (0.0010) [2023-12-27 00:14:23,560][105692] Updated weights for policy 0, policy_version 1215692 (0.0010) [2023-12-27 00:14:24,083][105620] Updated weights for policy 1, policy_version 1217029 (0.0007) [2023-12-27 00:14:24,143][105620] Updated weights for policy 1, policy_version 1217039 (0.0008) [2023-12-27 00:14:24,197][105620] Updated weights for policy 1, policy_version 1217049 (0.0009) [2023-12-27 00:14:24,325][105692] Updated weights for policy 0, policy_version 1215702 (0.0010) [2023-12-27 00:14:24,377][105692] Updated weights for policy 0, policy_version 1215712 (0.0009) [2023-12-27 00:14:24,442][105692] Updated weights for policy 0, policy_version 1215722 (0.0009) [2023-12-27 00:14:24,958][105620] Updated weights for policy 1, policy_version 1217059 (0.0008) [2023-12-27 00:14:25,014][105620] Updated weights for policy 1, policy_version 1217069 (0.0008) [2023-12-27 00:14:25,073][105620] Updated weights for policy 1, policy_version 1217079 (0.0010) [2023-12-27 00:14:25,154][105692] Updated weights for policy 0, policy_version 1215732 (0.0008) [2023-12-27 00:14:25,205][105692] Updated weights for policy 0, policy_version 1215742 (0.0010) [2023-12-27 00:14:25,249][105692] Updated weights for policy 0, policy_version 1215752 (0.0010) [2023-12-27 00:14:25,745][105620] Updated weights for policy 1, policy_version 1217089 (0.0010) [2023-12-27 00:14:25,799][105620] Updated weights for policy 1, policy_version 1217100 (0.0010) [2023-12-27 00:14:25,857][105620] Updated weights for policy 1, policy_version 1217110 (0.0010) [2023-12-27 00:14:25,922][105692] Updated weights for policy 0, policy_version 1215762 (0.0009) [2023-12-27 00:14:25,923][105620] Updated weights for policy 1, policy_version 1217120 (0.0010) [2023-12-27 00:14:25,975][105692] Updated weights for policy 0, policy_version 1215772 (0.0006) [2023-12-27 00:14:26,028][105692] Updated weights for policy 0, policy_version 1215782 (0.0009) [2023-12-27 00:14:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 622911488. Throughput: 0: 9370.7, 1: 9712.0. Samples: 622921008. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:14:26,063][104569] Avg episode reward: [(0, '8820.608'), (1, '9076.052')] [2023-12-27 00:14:26,086][105692] Updated weights for policy 0, policy_version 1215792 (0.0010) [2023-12-27 00:14:26,713][105620] Updated weights for policy 1, policy_version 1217130 (0.0007) [2023-12-27 00:14:26,769][105620] Updated weights for policy 1, policy_version 1217140 (0.0007) [2023-12-27 00:14:26,794][105692] Updated weights for policy 0, policy_version 1215802 (0.0010) [2023-12-27 00:14:26,829][105620] Updated weights for policy 1, policy_version 1217150 (0.0005) [2023-12-27 00:14:26,839][105692] Updated weights for policy 0, policy_version 1215812 (0.0010) [2023-12-27 00:14:26,886][105692] Updated weights for policy 0, policy_version 1215822 (0.0010) [2023-12-27 00:14:27,502][105620] Updated weights for policy 1, policy_version 1217160 (0.0008) [2023-12-27 00:14:27,533][105692] Updated weights for policy 0, policy_version 1215832 (0.0009) [2023-12-27 00:14:27,558][105620] Updated weights for policy 1, policy_version 1217170 (0.0006) [2023-12-27 00:14:27,580][105692] Updated weights for policy 0, policy_version 1215842 (0.0010) [2023-12-27 00:14:27,613][105620] Updated weights for policy 1, policy_version 1217180 (0.0005) [2023-12-27 00:14:27,630][105692] Updated weights for policy 0, policy_version 1215852 (0.0010) [2023-12-27 00:14:28,206][105692] Updated weights for policy 0, policy_version 1215862 (0.0009) [2023-12-27 00:14:28,267][105692] Updated weights for policy 0, policy_version 1215872 (0.0010) [2023-12-27 00:14:28,317][105692] Updated weights for policy 0, policy_version 1215882 (0.0010) [2023-12-27 00:14:28,440][105620] Updated weights for policy 1, policy_version 1217190 (0.0008) [2023-12-27 00:14:28,498][105620] Updated weights for policy 1, policy_version 1217200 (0.0010) [2023-12-27 00:14:28,553][105620] Updated weights for policy 1, policy_version 1217210 (0.0010) [2023-12-27 00:14:28,924][105692] Updated weights for policy 0, policy_version 1215892 (0.0008) [2023-12-27 00:14:28,990][105692] Updated weights for policy 0, policy_version 1215902 (0.0009) [2023-12-27 00:14:29,050][105692] Updated weights for policy 0, policy_version 1215912 (0.0011) [2023-12-27 00:14:29,341][105620] Updated weights for policy 1, policy_version 1217220 (0.0010) [2023-12-27 00:14:29,393][105620] Updated weights for policy 1, policy_version 1217230 (0.0008) [2023-12-27 00:14:29,443][105620] Updated weights for policy 1, policy_version 1217240 (0.0005) [2023-12-27 00:14:29,732][105692] Updated weights for policy 0, policy_version 1215922 (0.0009) [2023-12-27 00:14:29,794][105692] Updated weights for policy 0, policy_version 1215932 (0.0005) [2023-12-27 00:14:29,864][105692] Updated weights for policy 0, policy_version 1215942 (0.0007) [2023-12-27 00:14:29,925][105692] Updated weights for policy 0, policy_version 1215952 (0.0008) [2023-12-27 00:14:30,040][105620] Updated weights for policy 1, policy_version 1217250 (0.0006) [2023-12-27 00:14:30,105][105620] Updated weights for policy 1, policy_version 1217260 (0.0010) [2023-12-27 00:14:30,163][105620] Updated weights for policy 1, policy_version 1217270 (0.0010) [2023-12-27 00:14:30,539][105692] Updated weights for policy 0, policy_version 1215962 (0.0010) [2023-12-27 00:14:30,598][105692] Updated weights for policy 0, policy_version 1215972 (0.0009) [2023-12-27 00:14:30,657][105692] Updated weights for policy 0, policy_version 1215982 (0.0010) [2023-12-27 00:14:30,758][105620] Updated weights for policy 1, policy_version 1217281 (0.0009) [2023-12-27 00:14:30,811][105620] Updated weights for policy 1, policy_version 1217291 (0.0009) [2023-12-27 00:14:30,865][105620] Updated weights for policy 1, policy_version 1217302 (0.0010) [2023-12-27 00:14:31,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 623017984. Throughput: 0: 9468.0, 1: 9733.8. Samples: 622982232. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:14:31,062][104569] Avg episode reward: [(0, '8819.144'), (1, '9076.644')] [2023-12-27 00:14:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001215984_311345152.pth... [2023-12-27 00:14:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001217312_311672832.pth... [2023-12-27 00:14:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001216192_311386112.pth [2023-12-27 00:14:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001214864_311058432.pth [2023-12-27 00:14:31,486][105692] Updated weights for policy 0, policy_version 1215992 (0.0009) [2023-12-27 00:14:31,541][105692] Updated weights for policy 0, policy_version 1216002 (0.0009) [2023-12-27 00:14:31,567][105620] Updated weights for policy 1, policy_version 1217313 (0.0008) [2023-12-27 00:14:31,587][105692] Updated weights for policy 0, policy_version 1216012 (0.0008) [2023-12-27 00:14:31,631][105620] Updated weights for policy 1, policy_version 1217323 (0.0007) [2023-12-27 00:14:31,687][105620] Updated weights for policy 1, policy_version 1217333 (0.0009) [2023-12-27 00:14:31,753][105620] Updated weights for policy 1, policy_version 1217343 (0.0008) [2023-12-27 00:14:32,223][105692] Updated weights for policy 0, policy_version 1216022 (0.0009) [2023-12-27 00:14:32,274][105692] Updated weights for policy 0, policy_version 1216032 (0.0009) [2023-12-27 00:14:32,336][105692] Updated weights for policy 0, policy_version 1216042 (0.0007) [2023-12-27 00:14:32,568][105620] Updated weights for policy 1, policy_version 1217353 (0.0009) [2023-12-27 00:14:32,616][105620] Updated weights for policy 1, policy_version 1217363 (0.0009) [2023-12-27 00:14:32,667][105620] Updated weights for policy 1, policy_version 1217373 (0.0008) [2023-12-27 00:14:32,973][105692] Updated weights for policy 0, policy_version 1216052 (0.0008) [2023-12-27 00:14:33,031][105692] Updated weights for policy 0, policy_version 1216062 (0.0005) [2023-12-27 00:14:33,097][105692] Updated weights for policy 0, policy_version 1216072 (0.0005) [2023-12-27 00:14:33,544][105620] Updated weights for policy 1, policy_version 1217383 (0.0008) [2023-12-27 00:14:33,595][105620] Updated weights for policy 1, policy_version 1217393 (0.0008) [2023-12-27 00:14:33,645][105620] Updated weights for policy 1, policy_version 1217403 (0.0008) [2023-12-27 00:14:33,676][105692] Updated weights for policy 0, policy_version 1216082 (0.0006) [2023-12-27 00:14:33,724][105692] Updated weights for policy 0, policy_version 1216092 (0.0010) [2023-12-27 00:14:33,778][105692] Updated weights for policy 0, policy_version 1216102 (0.0010) [2023-12-27 00:14:33,829][105692] Updated weights for policy 0, policy_version 1216112 (0.0010) [2023-12-27 00:14:34,395][105620] Updated weights for policy 1, policy_version 1217413 (0.0009) [2023-12-27 00:14:34,451][105620] Updated weights for policy 1, policy_version 1217423 (0.0011) [2023-12-27 00:14:34,513][105620] Updated weights for policy 1, policy_version 1217433 (0.0007) [2023-12-27 00:14:34,519][105692] Updated weights for policy 0, policy_version 1216122 (0.0009) [2023-12-27 00:14:34,570][105692] Updated weights for policy 0, policy_version 1216132 (0.0010) [2023-12-27 00:14:34,626][105692] Updated weights for policy 0, policy_version 1216142 (0.0011) [2023-12-27 00:14:35,245][105620] Updated weights for policy 1, policy_version 1217443 (0.0006) [2023-12-27 00:14:35,295][105692] Updated weights for policy 0, policy_version 1216152 (0.0006) [2023-12-27 00:14:35,301][105620] Updated weights for policy 1, policy_version 1217453 (0.0010) [2023-12-27 00:14:35,354][105692] Updated weights for policy 0, policy_version 1216162 (0.0005) [2023-12-27 00:14:35,359][105620] Updated weights for policy 1, policy_version 1217463 (0.0010) [2023-12-27 00:14:35,406][105692] Updated weights for policy 0, policy_version 1216172 (0.0009) [2023-12-27 00:14:35,995][105692] Updated weights for policy 0, policy_version 1216182 (0.0007) [2023-12-27 00:14:36,002][105620] Updated weights for policy 1, policy_version 1217473 (0.0011) [2023-12-27 00:14:36,057][105620] Updated weights for policy 1, policy_version 1217483 (0.0010) [2023-12-27 00:14:36,060][105692] Updated weights for policy 0, policy_version 1216192 (0.0007) [2023-12-27 00:14:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 623108096. Throughput: 0: 9634.5, 1: 9708.2. Samples: 623101560. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:14:36,062][104569] Avg episode reward: [(0, '8726.561'), (1, '9167.517')] [2023-12-27 00:14:36,124][105620] Updated weights for policy 1, policy_version 1217493 (0.0010) [2023-12-27 00:14:36,140][105692] Updated weights for policy 0, policy_version 1216202 (0.0008) [2023-12-27 00:14:36,186][105620] Updated weights for policy 1, policy_version 1217503 (0.0006) [2023-12-27 00:14:36,798][105692] Updated weights for policy 0, policy_version 1216212 (0.0011) [2023-12-27 00:14:36,866][105692] Updated weights for policy 0, policy_version 1216222 (0.0011) [2023-12-27 00:14:36,879][105620] Updated weights for policy 1, policy_version 1217513 (0.0007) [2023-12-27 00:14:36,925][105692] Updated weights for policy 0, policy_version 1216232 (0.0011) [2023-12-27 00:14:36,940][105620] Updated weights for policy 1, policy_version 1217523 (0.0011) [2023-12-27 00:14:36,997][105620] Updated weights for policy 1, policy_version 1217533 (0.0011) [2023-12-27 00:14:37,667][105692] Updated weights for policy 0, policy_version 1216242 (0.0010) [2023-12-27 00:14:37,724][105692] Updated weights for policy 0, policy_version 1216252 (0.0007) [2023-12-27 00:14:37,742][105620] Updated weights for policy 1, policy_version 1217543 (0.0011) [2023-12-27 00:14:37,784][105692] Updated weights for policy 0, policy_version 1216262 (0.0006) [2023-12-27 00:14:37,795][105620] Updated weights for policy 1, policy_version 1217553 (0.0011) [2023-12-27 00:14:37,841][105692] Updated weights for policy 0, policy_version 1216272 (0.0006) [2023-12-27 00:14:37,855][105620] Updated weights for policy 1, policy_version 1217563 (0.0011) [2023-12-27 00:14:38,521][105620] Updated weights for policy 1, policy_version 1217573 (0.0009) [2023-12-27 00:14:38,581][105620] Updated weights for policy 1, policy_version 1217583 (0.0011) [2023-12-27 00:14:38,593][105692] Updated weights for policy 0, policy_version 1216282 (0.0005) [2023-12-27 00:14:38,640][105620] Updated weights for policy 1, policy_version 1217593 (0.0011) [2023-12-27 00:14:38,644][105692] Updated weights for policy 0, policy_version 1216292 (0.0008) [2023-12-27 00:14:38,693][105692] Updated weights for policy 0, policy_version 1216302 (0.0007) [2023-12-27 00:14:39,269][105620] Updated weights for policy 1, policy_version 1217603 (0.0010) [2023-12-27 00:14:39,278][105692] Updated weights for policy 0, policy_version 1216312 (0.0007) [2023-12-27 00:14:39,336][105620] Updated weights for policy 1, policy_version 1217613 (0.0008) [2023-12-27 00:14:39,341][105692] Updated weights for policy 0, policy_version 1216322 (0.0006) [2023-12-27 00:14:39,407][105620] Updated weights for policy 1, policy_version 1217623 (0.0008) [2023-12-27 00:14:39,409][105692] Updated weights for policy 0, policy_version 1216332 (0.0008) [2023-12-27 00:14:40,005][105692] Updated weights for policy 0, policy_version 1216342 (0.0008) [2023-12-27 00:14:40,065][105692] Updated weights for policy 0, policy_version 1216352 (0.0008) [2023-12-27 00:14:40,118][105692] Updated weights for policy 0, policy_version 1216362 (0.0008) [2023-12-27 00:14:40,124][105620] Updated weights for policy 1, policy_version 1217633 (0.0008) [2023-12-27 00:14:40,183][105620] Updated weights for policy 1, policy_version 1217643 (0.0008) [2023-12-27 00:14:40,250][105620] Updated weights for policy 1, policy_version 1217653 (0.0008) [2023-12-27 00:14:40,299][105620] Updated weights for policy 1, policy_version 1217663 (0.0008) [2023-12-27 00:14:40,887][105692] Updated weights for policy 0, policy_version 1216372 (0.0007) [2023-12-27 00:14:40,954][105692] Updated weights for policy 0, policy_version 1216382 (0.0011) [2023-12-27 00:14:41,012][105620] Updated weights for policy 1, policy_version 1217673 (0.0007) [2023-12-27 00:14:41,013][105692] Updated weights for policy 0, policy_version 1216392 (0.0010) [2023-12-27 00:14:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 623206400. Throughput: 0: 9699.2, 1: 9678.8. Samples: 623223368. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:14:41,063][104569] Avg episode reward: [(0, '8455.456'), (1, '9166.153')] [2023-12-27 00:14:41,080][105620] Updated weights for policy 1, policy_version 1217683 (0.0009) [2023-12-27 00:14:41,145][105620] Updated weights for policy 1, policy_version 1217693 (0.0011) [2023-12-27 00:14:41,755][105692] Updated weights for policy 0, policy_version 1216402 (0.0009) [2023-12-27 00:14:41,812][105692] Updated weights for policy 0, policy_version 1216412 (0.0011) [2023-12-27 00:14:41,861][105692] Updated weights for policy 0, policy_version 1216422 (0.0011) [2023-12-27 00:14:41,925][105692] Updated weights for policy 0, policy_version 1216432 (0.0011) [2023-12-27 00:14:41,967][105620] Updated weights for policy 1, policy_version 1217703 (0.0010) [2023-12-27 00:14:42,025][105620] Updated weights for policy 1, policy_version 1217713 (0.0006) [2023-12-27 00:14:42,078][105620] Updated weights for policy 1, policy_version 1217723 (0.0010) [2023-12-27 00:14:42,708][105692] Updated weights for policy 0, policy_version 1216442 (0.0011) [2023-12-27 00:14:42,773][105692] Updated weights for policy 0, policy_version 1216452 (0.0011) [2023-12-27 00:14:42,797][105620] Updated weights for policy 1, policy_version 1217733 (0.0010) [2023-12-27 00:14:42,832][105692] Updated weights for policy 0, policy_version 1216462 (0.0011) [2023-12-27 00:14:42,849][105620] Updated weights for policy 1, policy_version 1217743 (0.0005) [2023-12-27 00:14:42,901][105620] Updated weights for policy 1, policy_version 1217753 (0.0005) [2023-12-27 00:14:43,476][105692] Updated weights for policy 0, policy_version 1216472 (0.0007) [2023-12-27 00:14:43,523][105620] Updated weights for policy 1, policy_version 1217763 (0.0005) [2023-12-27 00:14:43,529][105692] Updated weights for policy 0, policy_version 1216482 (0.0005) [2023-12-27 00:14:43,576][105620] Updated weights for policy 1, policy_version 1217773 (0.0008) [2023-12-27 00:14:43,584][105692] Updated weights for policy 0, policy_version 1216492 (0.0008) [2023-12-27 00:14:43,631][105620] Updated weights for policy 1, policy_version 1217783 (0.0010) [2023-12-27 00:14:44,271][105692] Updated weights for policy 0, policy_version 1216502 (0.0010) [2023-12-27 00:14:44,326][105692] Updated weights for policy 0, policy_version 1216512 (0.0010) [2023-12-27 00:14:44,358][105620] Updated weights for policy 1, policy_version 1217793 (0.0010) [2023-12-27 00:14:44,382][105692] Updated weights for policy 0, policy_version 1216522 (0.0011) [2023-12-27 00:14:44,418][105620] Updated weights for policy 1, policy_version 1217803 (0.0011) [2023-12-27 00:14:44,463][105620] Updated weights for policy 1, policy_version 1217813 (0.0010) [2023-12-27 00:14:44,519][105620] Updated weights for policy 1, policy_version 1217823 (0.0010) [2023-12-27 00:14:45,047][105692] Updated weights for policy 0, policy_version 1216532 (0.0009) [2023-12-27 00:14:45,112][105692] Updated weights for policy 0, policy_version 1216542 (0.0008) [2023-12-27 00:14:45,173][105692] Updated weights for policy 0, policy_version 1216552 (0.0008) [2023-12-27 00:14:45,312][105620] Updated weights for policy 1, policy_version 1217833 (0.0011) [2023-12-27 00:14:45,373][105620] Updated weights for policy 1, policy_version 1217843 (0.0011) [2023-12-27 00:14:45,426][105620] Updated weights for policy 1, policy_version 1217853 (0.0011) [2023-12-27 00:14:45,949][105692] Updated weights for policy 0, policy_version 1216562 (0.0008) [2023-12-27 00:14:46,002][105692] Updated weights for policy 0, policy_version 1216572 (0.0008) [2023-12-27 00:14:46,054][105692] Updated weights for policy 0, policy_version 1216582 (0.0008) [2023-12-27 00:14:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 623304704. Throughput: 0: 9766.8, 1: 9632.5. Samples: 623281928. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:14:46,062][104569] Avg episode reward: [(0, '8905.064'), (1, '8984.921')] [2023-12-27 00:14:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001217856_311812096.pth... [2023-12-27 00:14:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001216736_311525376.pth [2023-12-27 00:14:46,109][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001216592_311500800.pth... [2023-12-27 00:14:46,110][105692] Updated weights for policy 0, policy_version 1216592 (0.0009) [2023-12-27 00:14:46,117][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001215408_311197696.pth [2023-12-27 00:14:46,156][105620] Updated weights for policy 1, policy_version 1217863 (0.0008) [2023-12-27 00:14:46,211][105620] Updated weights for policy 1, policy_version 1217873 (0.0006) [2023-12-27 00:14:46,274][105620] Updated weights for policy 1, policy_version 1217883 (0.0009) [2023-12-27 00:14:46,861][105620] Updated weights for policy 1, policy_version 1217893 (0.0008) [2023-12-27 00:14:46,913][105620] Updated weights for policy 1, policy_version 1217903 (0.0006) [2023-12-27 00:14:46,966][105692] Updated weights for policy 0, policy_version 1216602 (0.0006) [2023-12-27 00:14:46,971][105620] Updated weights for policy 1, policy_version 1217913 (0.0007) [2023-12-27 00:14:47,031][105692] Updated weights for policy 0, policy_version 1216612 (0.0006) [2023-12-27 00:14:47,091][105692] Updated weights for policy 0, policy_version 1216622 (0.0008) [2023-12-27 00:14:47,680][105620] Updated weights for policy 1, policy_version 1217923 (0.0008) [2023-12-27 00:14:47,737][105620] Updated weights for policy 1, policy_version 1217933 (0.0010) [2023-12-27 00:14:47,795][105620] Updated weights for policy 1, policy_version 1217943 (0.0006) [2023-12-27 00:14:47,798][105692] Updated weights for policy 0, policy_version 1216632 (0.0007) [2023-12-27 00:14:47,858][105692] Updated weights for policy 0, policy_version 1216642 (0.0007) [2023-12-27 00:14:47,912][105692] Updated weights for policy 0, policy_version 1216652 (0.0005) [2023-12-27 00:14:48,432][105620] Updated weights for policy 1, policy_version 1217953 (0.0007) [2023-12-27 00:14:48,487][105620] Updated weights for policy 1, policy_version 1217963 (0.0010) [2023-12-27 00:14:48,543][105620] Updated weights for policy 1, policy_version 1217973 (0.0010) [2023-12-27 00:14:48,585][105692] Updated weights for policy 0, policy_version 1216662 (0.0006) [2023-12-27 00:14:48,598][105620] Updated weights for policy 1, policy_version 1217983 (0.0010) [2023-12-27 00:14:48,648][105692] Updated weights for policy 0, policy_version 1216672 (0.0008) [2023-12-27 00:14:48,712][105692] Updated weights for policy 0, policy_version 1216682 (0.0008) [2023-12-27 00:14:49,348][105620] Updated weights for policy 1, policy_version 1217993 (0.0009) [2023-12-27 00:14:49,408][105692] Updated weights for policy 0, policy_version 1216692 (0.0008) [2023-12-27 00:14:49,410][105620] Updated weights for policy 1, policy_version 1218003 (0.0006) [2023-12-27 00:14:49,469][105692] Updated weights for policy 0, policy_version 1216702 (0.0008) [2023-12-27 00:14:49,471][105620] Updated weights for policy 1, policy_version 1218013 (0.0010) [2023-12-27 00:14:49,533][105585] KL-divergence is very high: 100.1235 [2023-12-27 00:14:49,534][105692] Updated weights for policy 0, policy_version 1216712 (0.0007) [2023-12-27 00:14:50,171][105620] Updated weights for policy 1, policy_version 1218023 (0.0009) [2023-12-27 00:14:50,220][105620] Updated weights for policy 1, policy_version 1218033 (0.0010) [2023-12-27 00:14:50,250][105692] Updated weights for policy 0, policy_version 1216722 (0.0008) [2023-12-27 00:14:50,275][105620] Updated weights for policy 1, policy_version 1218043 (0.0010) [2023-12-27 00:14:50,307][105692] Updated weights for policy 0, policy_version 1216732 (0.0006) [2023-12-27 00:14:50,363][105692] Updated weights for policy 0, policy_version 1216742 (0.0009) [2023-12-27 00:14:50,426][105692] Updated weights for policy 0, policy_version 1216752 (0.0008) [2023-12-27 00:14:51,053][105620] Updated weights for policy 1, policy_version 1218053 (0.0011) [2023-12-27 00:14:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 623403008. Throughput: 0: 9809.3, 1: 9675.8. Samples: 623397820. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:14:51,063][104569] Avg episode reward: [(0, '8818.735'), (1, '9168.710')] [2023-12-27 00:14:51,121][105620] Updated weights for policy 1, policy_version 1218063 (0.0011) [2023-12-27 00:14:51,181][105620] Updated weights for policy 1, policy_version 1218073 (0.0011) [2023-12-27 00:14:51,182][105692] Updated weights for policy 0, policy_version 1216762 (0.0007) [2023-12-27 00:14:51,242][105692] Updated weights for policy 0, policy_version 1216772 (0.0009) [2023-12-27 00:14:51,300][105692] Updated weights for policy 0, policy_version 1216782 (0.0009) [2023-12-27 00:14:51,938][105620] Updated weights for policy 1, policy_version 1218083 (0.0009) [2023-12-27 00:14:52,002][105620] Updated weights for policy 1, policy_version 1218093 (0.0009) [2023-12-27 00:14:52,053][105692] Updated weights for policy 0, policy_version 1216792 (0.0006) [2023-12-27 00:14:52,067][105620] Updated weights for policy 1, policy_version 1218103 (0.0009) [2023-12-27 00:14:52,111][105692] Updated weights for policy 0, policy_version 1216802 (0.0006) [2023-12-27 00:14:52,168][105692] Updated weights for policy 0, policy_version 1216812 (0.0009) [2023-12-27 00:14:52,787][105620] Updated weights for policy 1, policy_version 1218113 (0.0007) [2023-12-27 00:14:52,849][105620] Updated weights for policy 1, policy_version 1218123 (0.0010) [2023-12-27 00:14:52,901][105692] Updated weights for policy 0, policy_version 1216822 (0.0007) [2023-12-27 00:14:52,915][105620] Updated weights for policy 1, policy_version 1218133 (0.0008) [2023-12-27 00:14:52,953][105692] Updated weights for policy 0, policy_version 1216832 (0.0005) [2023-12-27 00:14:52,978][105620] Updated weights for policy 1, policy_version 1218143 (0.0008) [2023-12-27 00:14:53,008][105692] Updated weights for policy 0, policy_version 1216842 (0.0007) [2023-12-27 00:14:53,618][105620] Updated weights for policy 1, policy_version 1218153 (0.0009) [2023-12-27 00:14:53,666][105620] Updated weights for policy 1, policy_version 1218163 (0.0005) [2023-12-27 00:14:53,685][105692] Updated weights for policy 0, policy_version 1216852 (0.0010) [2023-12-27 00:14:53,723][105620] Updated weights for policy 1, policy_version 1218173 (0.0008) [2023-12-27 00:14:53,734][105692] Updated weights for policy 0, policy_version 1216862 (0.0006) [2023-12-27 00:14:53,781][105692] Updated weights for policy 0, policy_version 1216872 (0.0008) [2023-12-27 00:14:54,354][105692] Updated weights for policy 0, policy_version 1216882 (0.0005) [2023-12-27 00:14:54,387][105620] Updated weights for policy 1, policy_version 1218183 (0.0006) [2023-12-27 00:14:54,422][105692] Updated weights for policy 0, policy_version 1216892 (0.0007) [2023-12-27 00:14:54,447][105620] Updated weights for policy 1, policy_version 1218193 (0.0006) [2023-12-27 00:14:54,474][105692] Updated weights for policy 0, policy_version 1216902 (0.0007) [2023-12-27 00:14:54,510][105620] Updated weights for policy 1, policy_version 1218203 (0.0010) [2023-12-27 00:14:54,528][105692] Updated weights for policy 0, policy_version 1216912 (0.0005) [2023-12-27 00:14:55,126][105692] Updated weights for policy 0, policy_version 1216922 (0.0010) [2023-12-27 00:14:55,169][105620] Updated weights for policy 1, policy_version 1218213 (0.0010) [2023-12-27 00:14:55,178][105692] Updated weights for policy 0, policy_version 1216932 (0.0010) [2023-12-27 00:14:55,223][105620] Updated weights for policy 1, policy_version 1218223 (0.0010) [2023-12-27 00:14:55,237][105692] Updated weights for policy 0, policy_version 1216942 (0.0011) [2023-12-27 00:14:55,271][105620] Updated weights for policy 1, policy_version 1218233 (0.0010) [2023-12-27 00:14:55,906][105692] Updated weights for policy 0, policy_version 1216952 (0.0006) [2023-12-27 00:14:55,958][105692] Updated weights for policy 0, policy_version 1216962 (0.0006) [2023-12-27 00:14:56,011][105692] Updated weights for policy 0, policy_version 1216972 (0.0005) [2023-12-27 00:14:56,032][105620] Updated weights for policy 1, policy_version 1218243 (0.0010) [2023-12-27 00:14:56,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 623509504. Throughput: 0: 9913.0, 1: 9701.2. Samples: 623517472. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:14:56,063][104569] Avg episode reward: [(0, '8821.242'), (1, '9259.781')] [2023-12-27 00:14:56,090][105620] Updated weights for policy 1, policy_version 1218253 (0.0010) [2023-12-27 00:14:56,137][105620] Updated weights for policy 1, policy_version 1218263 (0.0009) [2023-12-27 00:14:56,668][105692] Updated weights for policy 0, policy_version 1216982 (0.0008) [2023-12-27 00:14:56,721][105692] Updated weights for policy 0, policy_version 1216992 (0.0010) [2023-12-27 00:14:56,774][105692] Updated weights for policy 0, policy_version 1217002 (0.0006) [2023-12-27 00:14:56,883][105620] Updated weights for policy 1, policy_version 1218273 (0.0010) [2023-12-27 00:14:56,941][105620] Updated weights for policy 1, policy_version 1218283 (0.0010) [2023-12-27 00:14:56,998][105620] Updated weights for policy 1, policy_version 1218293 (0.0009) [2023-12-27 00:14:57,046][105620] Updated weights for policy 1, policy_version 1218303 (0.0007) [2023-12-27 00:14:57,336][105692] Updated weights for policy 0, policy_version 1217012 (0.0005) [2023-12-27 00:14:57,387][105692] Updated weights for policy 0, policy_version 1217022 (0.0005) [2023-12-27 00:14:57,443][105692] Updated weights for policy 0, policy_version 1217032 (0.0005) [2023-12-27 00:14:57,776][105620] Updated weights for policy 1, policy_version 1218313 (0.0011) [2023-12-27 00:14:57,842][105620] Updated weights for policy 1, policy_version 1218323 (0.0010) [2023-12-27 00:14:57,903][105620] Updated weights for policy 1, policy_version 1218333 (0.0010) [2023-12-27 00:14:58,141][105692] Updated weights for policy 0, policy_version 1217042 (0.0006) [2023-12-27 00:14:58,208][105692] Updated weights for policy 0, policy_version 1217052 (0.0010) [2023-12-27 00:14:58,267][105692] Updated weights for policy 0, policy_version 1217062 (0.0011) [2023-12-27 00:14:58,326][105692] Updated weights for policy 0, policy_version 1217072 (0.0010) [2023-12-27 00:14:58,669][105620] Updated weights for policy 1, policy_version 1218343 (0.0009) [2023-12-27 00:14:58,729][105620] Updated weights for policy 1, policy_version 1218353 (0.0008) [2023-12-27 00:14:58,795][105620] Updated weights for policy 1, policy_version 1218363 (0.0009) [2023-12-27 00:14:59,182][105692] Updated weights for policy 0, policy_version 1217082 (0.0008) [2023-12-27 00:14:59,249][105692] Updated weights for policy 0, policy_version 1217092 (0.0009) [2023-12-27 00:14:59,316][105692] Updated weights for policy 0, policy_version 1217102 (0.0008) [2023-12-27 00:14:59,527][105620] Updated weights for policy 1, policy_version 1218373 (0.0007) [2023-12-27 00:14:59,597][105620] Updated weights for policy 1, policy_version 1218383 (0.0008) [2023-12-27 00:14:59,652][105620] Updated weights for policy 1, policy_version 1218393 (0.0009) [2023-12-27 00:15:00,057][105692] Updated weights for policy 0, policy_version 1217112 (0.0010) [2023-12-27 00:15:00,123][105692] Updated weights for policy 0, policy_version 1217122 (0.0005) [2023-12-27 00:15:00,180][105692] Updated weights for policy 0, policy_version 1217132 (0.0005) [2023-12-27 00:15:00,310][105620] Updated weights for policy 1, policy_version 1218403 (0.0008) [2023-12-27 00:15:00,371][105620] Updated weights for policy 1, policy_version 1218413 (0.0005) [2023-12-27 00:15:00,439][105620] Updated weights for policy 1, policy_version 1218423 (0.0006) [2023-12-27 00:15:00,835][105692] Updated weights for policy 0, policy_version 1217142 (0.0008) [2023-12-27 00:15:00,894][105692] Updated weights for policy 0, policy_version 1217152 (0.0006) [2023-12-27 00:15:00,950][105620] Updated weights for policy 1, policy_version 1218433 (0.0006) [2023-12-27 00:15:00,962][105692] Updated weights for policy 0, policy_version 1217162 (0.0006) [2023-12-27 00:15:01,007][105620] Updated weights for policy 1, policy_version 1218443 (0.0008) [2023-12-27 00:15:01,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 623607808. Throughput: 0: 9981.0, 1: 9696.0. Samples: 623577260. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:15:01,062][104569] Avg episode reward: [(0, '9179.614'), (1, '9170.623')] [2023-12-27 00:15:01,064][105620] Updated weights for policy 1, policy_version 1218453 (0.0009) [2023-12-27 00:15:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001217168_311648256.pth... [2023-12-27 00:15:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001215984_311345152.pth [2023-12-27 00:15:01,115][105620] Updated weights for policy 1, policy_version 1218463 (0.0008) [2023-12-27 00:15:01,119][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001218464_311967744.pth... [2023-12-27 00:15:01,122][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001217312_311672832.pth [2023-12-27 00:15:01,625][105692] Updated weights for policy 0, policy_version 1217172 (0.0007) [2023-12-27 00:15:01,688][105692] Updated weights for policy 0, policy_version 1217182 (0.0009) [2023-12-27 00:15:01,750][105692] Updated weights for policy 0, policy_version 1217192 (0.0009) [2023-12-27 00:15:01,848][105620] Updated weights for policy 1, policy_version 1218473 (0.0007) [2023-12-27 00:15:01,899][105620] Updated weights for policy 1, policy_version 1218483 (0.0009) [2023-12-27 00:15:01,951][105620] Updated weights for policy 1, policy_version 1218493 (0.0009) [2023-12-27 00:15:02,476][105692] Updated weights for policy 0, policy_version 1217202 (0.0009) [2023-12-27 00:15:02,533][105692] Updated weights for policy 0, policy_version 1217212 (0.0008) [2023-12-27 00:15:02,590][105692] Updated weights for policy 0, policy_version 1217222 (0.0009) [2023-12-27 00:15:02,655][105692] Updated weights for policy 0, policy_version 1217232 (0.0010) [2023-12-27 00:15:02,714][105620] Updated weights for policy 1, policy_version 1218503 (0.0007) [2023-12-27 00:15:02,769][105620] Updated weights for policy 1, policy_version 1218513 (0.0005) [2023-12-27 00:15:02,825][105620] Updated weights for policy 1, policy_version 1218523 (0.0005) [2023-12-27 00:15:03,381][105620] Updated weights for policy 1, policy_version 1218533 (0.0005) [2023-12-27 00:15:03,432][105620] Updated weights for policy 1, policy_version 1218543 (0.0006) [2023-12-27 00:15:03,433][105692] Updated weights for policy 0, policy_version 1217242 (0.0005) [2023-12-27 00:15:03,482][105620] Updated weights for policy 1, policy_version 1218553 (0.0010) [2023-12-27 00:15:03,491][105692] Updated weights for policy 0, policy_version 1217252 (0.0006) [2023-12-27 00:15:03,554][105692] Updated weights for policy 0, policy_version 1217262 (0.0006) [2023-12-27 00:15:04,117][105620] Updated weights for policy 1, policy_version 1218563 (0.0011) [2023-12-27 00:15:04,184][105620] Updated weights for policy 1, policy_version 1218573 (0.0009) [2023-12-27 00:15:04,203][105692] Updated weights for policy 0, policy_version 1217272 (0.0009) [2023-12-27 00:15:04,240][105620] Updated weights for policy 1, policy_version 1218583 (0.0010) [2023-12-27 00:15:04,252][105692] Updated weights for policy 0, policy_version 1217282 (0.0010) [2023-12-27 00:15:04,316][105692] Updated weights for policy 0, policy_version 1217292 (0.0007) [2023-12-27 00:15:04,854][105620] Updated weights for policy 1, policy_version 1218593 (0.0009) [2023-12-27 00:15:04,919][105620] Updated weights for policy 1, policy_version 1218603 (0.0005) [2023-12-27 00:15:04,978][105620] Updated weights for policy 1, policy_version 1218613 (0.0005) [2023-12-27 00:15:05,037][105620] Updated weights for policy 1, policy_version 1218623 (0.0005) [2023-12-27 00:15:05,058][105692] Updated weights for policy 0, policy_version 1217302 (0.0009) [2023-12-27 00:15:05,109][105692] Updated weights for policy 0, policy_version 1217312 (0.0011) [2023-12-27 00:15:05,157][105692] Updated weights for policy 0, policy_version 1217322 (0.0010) [2023-12-27 00:15:05,698][105620] Updated weights for policy 1, policy_version 1218633 (0.0010) [2023-12-27 00:15:05,753][105620] Updated weights for policy 1, policy_version 1218643 (0.0010) [2023-12-27 00:15:05,818][105620] Updated weights for policy 1, policy_version 1218653 (0.0010) [2023-12-27 00:15:05,841][105692] Updated weights for policy 0, policy_version 1217332 (0.0010) [2023-12-27 00:15:05,884][105692] Updated weights for policy 0, policy_version 1217342 (0.0010) [2023-12-27 00:15:05,949][105692] Updated weights for policy 0, policy_version 1217352 (0.0010) [2023-12-27 00:15:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.9, 300 sec: 19522.0). Total num frames: 623714304. Throughput: 0: 9941.1, 1: 9838.3. Samples: 623698600. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:15:06,062][104569] Avg episode reward: [(0, '8996.071'), (1, '9261.715')] [2023-12-27 00:15:06,424][105620] Updated weights for policy 1, policy_version 1218663 (0.0007) [2023-12-27 00:15:06,485][105620] Updated weights for policy 1, policy_version 1218673 (0.0006) [2023-12-27 00:15:06,545][105620] Updated weights for policy 1, policy_version 1218683 (0.0010) [2023-12-27 00:15:06,657][105692] Updated weights for policy 0, policy_version 1217362 (0.0009) [2023-12-27 00:15:06,720][105692] Updated weights for policy 0, policy_version 1217372 (0.0005) [2023-12-27 00:15:06,786][105692] Updated weights for policy 0, policy_version 1217382 (0.0006) [2023-12-27 00:15:06,847][105692] Updated weights for policy 0, policy_version 1217392 (0.0006) [2023-12-27 00:15:07,293][105620] Updated weights for policy 1, policy_version 1218693 (0.0010) [2023-12-27 00:15:07,356][105620] Updated weights for policy 1, policy_version 1218703 (0.0010) [2023-12-27 00:15:07,414][105620] Updated weights for policy 1, policy_version 1218713 (0.0010) [2023-12-27 00:15:07,464][105692] Updated weights for policy 0, policy_version 1217402 (0.0006) [2023-12-27 00:15:07,517][105692] Updated weights for policy 0, policy_version 1217412 (0.0008) [2023-12-27 00:15:07,569][105692] Updated weights for policy 0, policy_version 1217422 (0.0009) [2023-12-27 00:15:08,162][105620] Updated weights for policy 1, policy_version 1218723 (0.0010) [2023-12-27 00:15:08,213][105620] Updated weights for policy 1, policy_version 1218733 (0.0008) [2023-12-27 00:15:08,272][105620] Updated weights for policy 1, policy_version 1218743 (0.0009) [2023-12-27 00:15:08,343][105692] Updated weights for policy 0, policy_version 1217432 (0.0009) [2023-12-27 00:15:08,411][105692] Updated weights for policy 0, policy_version 1217442 (0.0009) [2023-12-27 00:15:08,480][105692] Updated weights for policy 0, policy_version 1217452 (0.0010) [2023-12-27 00:15:08,999][105620] Updated weights for policy 1, policy_version 1218754 (0.0009) [2023-12-27 00:15:09,053][105620] Updated weights for policy 1, policy_version 1218764 (0.0009) [2023-12-27 00:15:09,115][105620] Updated weights for policy 1, policy_version 1218774 (0.0008) [2023-12-27 00:15:09,178][105620] Updated weights for policy 1, policy_version 1218784 (0.0008) [2023-12-27 00:15:09,228][105692] Updated weights for policy 0, policy_version 1217462 (0.0008) [2023-12-27 00:15:09,294][105692] Updated weights for policy 0, policy_version 1217472 (0.0008) [2023-12-27 00:15:09,355][105692] Updated weights for policy 0, policy_version 1217482 (0.0010) [2023-12-27 00:15:09,970][105620] Updated weights for policy 1, policy_version 1218794 (0.0009) [2023-12-27 00:15:10,039][105620] Updated weights for policy 1, policy_version 1218804 (0.0006) [2023-12-27 00:15:10,100][105620] Updated weights for policy 1, policy_version 1218814 (0.0005) [2023-12-27 00:15:10,120][105692] Updated weights for policy 0, policy_version 1217492 (0.0009) [2023-12-27 00:15:10,181][105692] Updated weights for policy 0, policy_version 1217502 (0.0010) [2023-12-27 00:15:10,232][105692] Updated weights for policy 0, policy_version 1217512 (0.0009) [2023-12-27 00:15:10,718][105620] Updated weights for policy 1, policy_version 1218824 (0.0008) [2023-12-27 00:15:10,779][105620] Updated weights for policy 1, policy_version 1218834 (0.0010) [2023-12-27 00:15:10,840][105620] Updated weights for policy 1, policy_version 1218844 (0.0009) [2023-12-27 00:15:11,051][105692] Updated weights for policy 0, policy_version 1217522 (0.0009) [2023-12-27 00:15:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 623804416. Throughput: 0: 10004.1, 1: 9851.6. Samples: 623814516. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:15:11,063][104569] Avg episode reward: [(0, '8996.301'), (1, '9170.476')] [2023-12-27 00:15:11,113][105692] Updated weights for policy 0, policy_version 1217532 (0.0009) [2023-12-27 00:15:11,179][105692] Updated weights for policy 0, policy_version 1217542 (0.0008) [2023-12-27 00:15:11,238][105692] Updated weights for policy 0, policy_version 1217552 (0.0010) [2023-12-27 00:15:11,575][105620] Updated weights for policy 1, policy_version 1218854 (0.0010) [2023-12-27 00:15:11,640][105620] Updated weights for policy 1, policy_version 1218864 (0.0008) [2023-12-27 00:15:11,694][105620] Updated weights for policy 1, policy_version 1218874 (0.0008) [2023-12-27 00:15:12,072][105692] Updated weights for policy 0, policy_version 1217562 (0.0009) [2023-12-27 00:15:12,120][105692] Updated weights for policy 0, policy_version 1217572 (0.0009) [2023-12-27 00:15:12,171][105692] Updated weights for policy 0, policy_version 1217582 (0.0008) [2023-12-27 00:15:12,483][105620] Updated weights for policy 1, policy_version 1218884 (0.0009) [2023-12-27 00:15:12,537][105620] Updated weights for policy 1, policy_version 1218894 (0.0010) [2023-12-27 00:15:12,590][105620] Updated weights for policy 1, policy_version 1218904 (0.0009) [2023-12-27 00:15:12,898][105692] Updated weights for policy 0, policy_version 1217592 (0.0009) [2023-12-27 00:15:12,952][105692] Updated weights for policy 0, policy_version 1217602 (0.0007) [2023-12-27 00:15:13,008][105692] Updated weights for policy 0, policy_version 1217612 (0.0007) [2023-12-27 00:15:13,362][105620] Updated weights for policy 1, policy_version 1218914 (0.0008) [2023-12-27 00:15:13,415][105620] Updated weights for policy 1, policy_version 1218924 (0.0010) [2023-12-27 00:15:13,473][105620] Updated weights for policy 1, policy_version 1218934 (0.0010) [2023-12-27 00:15:13,525][105620] Updated weights for policy 1, policy_version 1218944 (0.0009) [2023-12-27 00:15:13,703][105692] Updated weights for policy 0, policy_version 1217622 (0.0009) [2023-12-27 00:15:13,753][105692] Updated weights for policy 0, policy_version 1217632 (0.0009) [2023-12-27 00:15:13,804][105692] Updated weights for policy 0, policy_version 1217642 (0.0008) [2023-12-27 00:15:14,304][105620] Updated weights for policy 1, policy_version 1218954 (0.0008) [2023-12-27 00:15:14,353][105620] Updated weights for policy 1, policy_version 1218964 (0.0009) [2023-12-27 00:15:14,422][105620] Updated weights for policy 1, policy_version 1218974 (0.0008) [2023-12-27 00:15:14,559][105692] Updated weights for policy 0, policy_version 1217652 (0.0009) [2023-12-27 00:15:14,622][105692] Updated weights for policy 0, policy_version 1217662 (0.0009) [2023-12-27 00:15:14,685][105692] Updated weights for policy 0, policy_version 1217672 (0.0009) [2023-12-27 00:15:15,220][105620] Updated weights for policy 1, policy_version 1218984 (0.0009) [2023-12-27 00:15:15,282][105620] Updated weights for policy 1, policy_version 1218994 (0.0009) [2023-12-27 00:15:15,336][105620] Updated weights for policy 1, policy_version 1219004 (0.0008) [2023-12-27 00:15:15,455][105692] Updated weights for policy 0, policy_version 1217682 (0.0009) [2023-12-27 00:15:15,503][105692] Updated weights for policy 0, policy_version 1217692 (0.0010) [2023-12-27 00:15:15,552][105692] Updated weights for policy 0, policy_version 1217702 (0.0009) [2023-12-27 00:15:15,602][105692] Updated weights for policy 0, policy_version 1217712 (0.0006) [2023-12-27 00:15:16,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 623894528. Throughput: 0: 9897.3, 1: 9837.8. Samples: 623870312. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:15:16,062][104569] Avg episode reward: [(0, '9088.065'), (1, '9169.900')] [2023-12-27 00:15:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001219008_312107008.pth... [2023-12-27 00:15:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001217712_311787520.pth... [2023-12-27 00:15:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001217856_311812096.pth [2023-12-27 00:15:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001216592_311500800.pth [2023-12-27 00:15:16,135][105620] Updated weights for policy 1, policy_version 1219014 (0.0009) [2023-12-27 00:15:16,193][105620] Updated weights for policy 1, policy_version 1219024 (0.0008) [2023-12-27 00:15:16,233][105692] Updated weights for policy 0, policy_version 1217722 (0.0006) [2023-12-27 00:15:16,247][105620] Updated weights for policy 1, policy_version 1219034 (0.0008) [2023-12-27 00:15:16,288][105692] Updated weights for policy 0, policy_version 1217732 (0.0010) [2023-12-27 00:15:16,346][105692] Updated weights for policy 0, policy_version 1217742 (0.0010) [2023-12-27 00:15:16,896][105692] Updated weights for policy 0, policy_version 1217752 (0.0006) [2023-12-27 00:15:16,943][105692] Updated weights for policy 0, policy_version 1217762 (0.0009) [2023-12-27 00:15:16,997][105692] Updated weights for policy 0, policy_version 1217772 (0.0005) [2023-12-27 00:15:16,997][105620] Updated weights for policy 1, policy_version 1219044 (0.0008) [2023-12-27 00:15:17,052][105620] Updated weights for policy 1, policy_version 1219054 (0.0010) [2023-12-27 00:15:17,097][105620] Updated weights for policy 1, policy_version 1219064 (0.0010) [2023-12-27 00:15:17,633][105692] Updated weights for policy 0, policy_version 1217782 (0.0007) [2023-12-27 00:15:17,688][105692] Updated weights for policy 0, policy_version 1217792 (0.0005) [2023-12-27 00:15:17,747][105692] Updated weights for policy 0, policy_version 1217802 (0.0006) [2023-12-27 00:15:17,808][105620] Updated weights for policy 1, policy_version 1219074 (0.0010) [2023-12-27 00:15:17,866][105620] Updated weights for policy 1, policy_version 1219084 (0.0008) [2023-12-27 00:15:17,920][105620] Updated weights for policy 1, policy_version 1219094 (0.0007) [2023-12-27 00:15:17,986][105620] Updated weights for policy 1, policy_version 1219104 (0.0008) [2023-12-27 00:15:18,419][105692] Updated weights for policy 0, policy_version 1217812 (0.0009) [2023-12-27 00:15:18,467][105692] Updated weights for policy 0, policy_version 1217822 (0.0005) [2023-12-27 00:15:18,530][105692] Updated weights for policy 0, policy_version 1217832 (0.0005) [2023-12-27 00:15:18,741][105620] Updated weights for policy 1, policy_version 1219114 (0.0010) [2023-12-27 00:15:18,807][105620] Updated weights for policy 1, policy_version 1219124 (0.0010) [2023-12-27 00:15:18,879][105620] Updated weights for policy 1, policy_version 1219134 (0.0010) [2023-12-27 00:15:19,090][105692] Updated weights for policy 0, policy_version 1217842 (0.0005) [2023-12-27 00:15:19,159][105692] Updated weights for policy 0, policy_version 1217852 (0.0007) [2023-12-27 00:15:19,225][105692] Updated weights for policy 0, policy_version 1217862 (0.0011) [2023-12-27 00:15:19,287][105692] Updated weights for policy 0, policy_version 1217872 (0.0010) [2023-12-27 00:15:19,685][105620] Updated weights for policy 1, policy_version 1219144 (0.0009) [2023-12-27 00:15:19,735][105620] Updated weights for policy 1, policy_version 1219154 (0.0008) [2023-12-27 00:15:19,800][105620] Updated weights for policy 1, policy_version 1219164 (0.0008) [2023-12-27 00:15:20,094][105692] Updated weights for policy 0, policy_version 1217882 (0.0011) [2023-12-27 00:15:20,150][105692] Updated weights for policy 0, policy_version 1217892 (0.0010) [2023-12-27 00:15:20,202][105692] Updated weights for policy 0, policy_version 1217902 (0.0010) [2023-12-27 00:15:20,635][105620] Updated weights for policy 1, policy_version 1219174 (0.0009) [2023-12-27 00:15:20,692][105620] Updated weights for policy 1, policy_version 1219184 (0.0010) [2023-12-27 00:15:20,745][105620] Updated weights for policy 1, policy_version 1219194 (0.0009) [2023-12-27 00:15:20,835][105692] Updated weights for policy 0, policy_version 1217912 (0.0008) [2023-12-27 00:15:20,887][105692] Updated weights for policy 0, policy_version 1217922 (0.0005) [2023-12-27 00:15:20,946][105692] Updated weights for policy 0, policy_version 1217932 (0.0010) [2023-12-27 00:15:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 624001024. Throughput: 0: 9915.6, 1: 9780.8. Samples: 623987896. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:15:21,062][104569] Avg episode reward: [(0, '8907.925'), (1, '9171.570')] [2023-12-27 00:15:21,598][105620] Updated weights for policy 1, policy_version 1219204 (0.0009) [2023-12-27 00:15:21,662][105620] Updated weights for policy 1, policy_version 1219214 (0.0008) [2023-12-27 00:15:21,670][105692] Updated weights for policy 0, policy_version 1217942 (0.0010) [2023-12-27 00:15:21,736][105620] Updated weights for policy 1, policy_version 1219224 (0.0008) [2023-12-27 00:15:21,739][105692] Updated weights for policy 0, policy_version 1217952 (0.0009) [2023-12-27 00:15:21,806][105692] Updated weights for policy 0, policy_version 1217962 (0.0009) [2023-12-27 00:15:22,420][105620] Updated weights for policy 1, policy_version 1219234 (0.0008) [2023-12-27 00:15:22,490][105620] Updated weights for policy 1, policy_version 1219244 (0.0008) [2023-12-27 00:15:22,512][105692] Updated weights for policy 0, policy_version 1217972 (0.0011) [2023-12-27 00:15:22,556][105620] Updated weights for policy 1, policy_version 1219254 (0.0008) [2023-12-27 00:15:22,568][105692] Updated weights for policy 0, policy_version 1217982 (0.0010) [2023-12-27 00:15:22,621][105620] Updated weights for policy 1, policy_version 1219264 (0.0008) [2023-12-27 00:15:22,628][105692] Updated weights for policy 0, policy_version 1217992 (0.0010) [2023-12-27 00:15:23,359][105620] Updated weights for policy 1, policy_version 1219274 (0.0008) [2023-12-27 00:15:23,383][105692] Updated weights for policy 0, policy_version 1218002 (0.0009) [2023-12-27 00:15:23,416][105620] Updated weights for policy 1, policy_version 1219284 (0.0009) [2023-12-27 00:15:23,429][105692] Updated weights for policy 0, policy_version 1218012 (0.0005) [2023-12-27 00:15:23,466][105620] Updated weights for policy 1, policy_version 1219294 (0.0009) [2023-12-27 00:15:23,490][105692] Updated weights for policy 0, policy_version 1218022 (0.0005) [2023-12-27 00:15:23,550][105692] Updated weights for policy 0, policy_version 1218032 (0.0005) [2023-12-27 00:15:24,066][105692] Updated weights for policy 0, policy_version 1218042 (0.0011) [2023-12-27 00:15:24,128][105692] Updated weights for policy 0, policy_version 1218052 (0.0006) [2023-12-27 00:15:24,189][105692] Updated weights for policy 0, policy_version 1218062 (0.0006) [2023-12-27 00:15:24,410][105620] Updated weights for policy 1, policy_version 1219304 (0.0010) [2023-12-27 00:15:24,462][105620] Updated weights for policy 1, policy_version 1219314 (0.0009) [2023-12-27 00:15:24,529][105620] Updated weights for policy 1, policy_version 1219324 (0.0010) [2023-12-27 00:15:24,709][105692] Updated weights for policy 0, policy_version 1218072 (0.0005) [2023-12-27 00:15:24,760][105692] Updated weights for policy 0, policy_version 1218082 (0.0005) [2023-12-27 00:15:24,808][105692] Updated weights for policy 0, policy_version 1218092 (0.0008) [2023-12-27 00:15:25,393][105692] Updated weights for policy 0, policy_version 1218102 (0.0009) [2023-12-27 00:15:25,426][105620] Updated weights for policy 1, policy_version 1219334 (0.0007) [2023-12-27 00:15:25,438][105692] Updated weights for policy 0, policy_version 1218112 (0.0010) [2023-12-27 00:15:25,476][105620] Updated weights for policy 1, policy_version 1219344 (0.0006) [2023-12-27 00:15:25,482][105692] Updated weights for policy 0, policy_version 1218122 (0.0010) [2023-12-27 00:15:25,535][105620] Updated weights for policy 1, policy_version 1219354 (0.0006) [2023-12-27 00:15:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.7, 300 sec: 19466.4). Total num frames: 624091136. Throughput: 0: 9956.0, 1: 9594.5. Samples: 624103144. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:15:26,063][104569] Avg episode reward: [(0, '8905.793'), (1, '9172.184')] [2023-12-27 00:15:26,146][105692] Updated weights for policy 0, policy_version 1218132 (0.0008) [2023-12-27 00:15:26,208][105692] Updated weights for policy 0, policy_version 1218142 (0.0009) [2023-12-27 00:15:26,263][105692] Updated weights for policy 0, policy_version 1218152 (0.0010) [2023-12-27 00:15:26,319][105620] Updated weights for policy 1, policy_version 1219364 (0.0008) [2023-12-27 00:15:26,363][105620] Updated weights for policy 1, policy_version 1219374 (0.0008) [2023-12-27 00:15:26,418][105620] Updated weights for policy 1, policy_version 1219384 (0.0008) [2023-12-27 00:15:26,982][105692] Updated weights for policy 0, policy_version 1218162 (0.0010) [2023-12-27 00:15:27,040][105692] Updated weights for policy 0, policy_version 1218172 (0.0010) [2023-12-27 00:15:27,091][105692] Updated weights for policy 0, policy_version 1218182 (0.0010) [2023-12-27 00:15:27,139][105692] Updated weights for policy 0, policy_version 1218192 (0.0010) [2023-12-27 00:15:27,190][105620] Updated weights for policy 1, policy_version 1219394 (0.0008) [2023-12-27 00:15:27,252][105620] Updated weights for policy 1, policy_version 1219404 (0.0008) [2023-12-27 00:15:27,327][105620] Updated weights for policy 1, policy_version 1219415 (0.0009) [2023-12-27 00:15:27,887][105692] Updated weights for policy 0, policy_version 1218202 (0.0008) [2023-12-27 00:15:27,936][105692] Updated weights for policy 0, policy_version 1218212 (0.0007) [2023-12-27 00:15:27,960][105620] Updated weights for policy 1, policy_version 1219425 (0.0007) [2023-12-27 00:15:28,005][105692] Updated weights for policy 0, policy_version 1218222 (0.0010) [2023-12-27 00:15:28,022][105620] Updated weights for policy 1, policy_version 1219435 (0.0006) [2023-12-27 00:15:28,088][105620] Updated weights for policy 1, policy_version 1219445 (0.0008) [2023-12-27 00:15:28,140][105620] Updated weights for policy 1, policy_version 1219455 (0.0008) [2023-12-27 00:15:28,743][105692] Updated weights for policy 0, policy_version 1218232 (0.0011) [2023-12-27 00:15:28,766][105620] Updated weights for policy 1, policy_version 1219465 (0.0006) [2023-12-27 00:15:28,789][105692] Updated weights for policy 0, policy_version 1218242 (0.0007) [2023-12-27 00:15:28,825][105620] Updated weights for policy 1, policy_version 1219475 (0.0008) [2023-12-27 00:15:28,839][105692] Updated weights for policy 0, policy_version 1218252 (0.0007) [2023-12-27 00:15:28,882][105620] Updated weights for policy 1, policy_version 1219485 (0.0008) [2023-12-27 00:15:29,589][105692] Updated weights for policy 0, policy_version 1218262 (0.0007) [2023-12-27 00:15:29,631][105620] Updated weights for policy 1, policy_version 1219495 (0.0008) [2023-12-27 00:15:29,637][105692] Updated weights for policy 0, policy_version 1218272 (0.0008) [2023-12-27 00:15:29,686][105692] Updated weights for policy 0, policy_version 1218282 (0.0008) [2023-12-27 00:15:29,688][105620] Updated weights for policy 1, policy_version 1219505 (0.0008) [2023-12-27 00:15:29,750][105620] Updated weights for policy 1, policy_version 1219515 (0.0010) [2023-12-27 00:15:30,395][105692] Updated weights for policy 0, policy_version 1218292 (0.0008) [2023-12-27 00:15:30,450][105692] Updated weights for policy 0, policy_version 1218302 (0.0009) [2023-12-27 00:15:30,485][105620] Updated weights for policy 1, policy_version 1219525 (0.0010) [2023-12-27 00:15:30,507][105692] Updated weights for policy 0, policy_version 1218312 (0.0006) [2023-12-27 00:15:30,537][105620] Updated weights for policy 1, policy_version 1219535 (0.0010) [2023-12-27 00:15:30,591][105620] Updated weights for policy 1, policy_version 1219545 (0.0010) [2023-12-27 00:15:31,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 624189440. Throughput: 0: 9961.4, 1: 9599.1. Samples: 624162152. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:15:31,063][104569] Avg episode reward: [(0, '8907.263'), (1, '9354.703')] [2023-12-27 00:15:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001218320_311943168.pth... [2023-12-27 00:15:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001219552_312246272.pth... [2023-12-27 00:15:31,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001218464_311967744.pth [2023-12-27 00:15:31,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001217168_311648256.pth [2023-12-27 00:15:31,128][105692] Updated weights for policy 0, policy_version 1218322 (0.0006) [2023-12-27 00:15:31,192][105692] Updated weights for policy 0, policy_version 1218332 (0.0008) [2023-12-27 00:15:31,252][105692] Updated weights for policy 0, policy_version 1218342 (0.0010) [2023-12-27 00:15:31,270][105620] Updated weights for policy 1, policy_version 1219555 (0.0009) [2023-12-27 00:15:31,313][105692] Updated weights for policy 0, policy_version 1218352 (0.0008) [2023-12-27 00:15:31,341][105620] Updated weights for policy 1, policy_version 1219565 (0.0006) [2023-12-27 00:15:31,425][105620] Updated weights for policy 1, policy_version 1219575 (0.0010) [2023-12-27 00:15:31,961][105692] Updated weights for policy 0, policy_version 1218362 (0.0009) [2023-12-27 00:15:31,977][105585] KL-divergence is very high: 102.1462 [2023-12-27 00:15:32,013][105692] Updated weights for policy 0, policy_version 1218372 (0.0005) [2023-12-27 00:15:32,018][105585] KL-divergence is very high: 109.4771 [2023-12-27 00:15:32,085][105585] KL-divergence is very high: 116.3085 [2023-12-27 00:15:32,090][105692] Updated weights for policy 0, policy_version 1218382 (0.0008) [2023-12-27 00:15:32,176][105620] Updated weights for policy 1, policy_version 1219585 (0.0009) [2023-12-27 00:15:32,232][105620] Updated weights for policy 1, policy_version 1219595 (0.0006) [2023-12-27 00:15:32,295][105620] Updated weights for policy 1, policy_version 1219605 (0.0007) [2023-12-27 00:15:32,358][105620] Updated weights for policy 1, policy_version 1219615 (0.0007) [2023-12-27 00:15:32,873][105692] Updated weights for policy 0, policy_version 1218392 (0.0010) [2023-12-27 00:15:32,916][105692] Updated weights for policy 0, policy_version 1218402 (0.0008) [2023-12-27 00:15:32,928][105620] Updated weights for policy 1, policy_version 1219625 (0.0008) [2023-12-27 00:15:32,970][105692] Updated weights for policy 0, policy_version 1218412 (0.0008) [2023-12-27 00:15:32,994][105620] Updated weights for policy 1, policy_version 1219635 (0.0007) [2023-12-27 00:15:33,052][105620] Updated weights for policy 1, policy_version 1219645 (0.0009) [2023-12-27 00:15:33,722][105620] Updated weights for policy 1, policy_version 1219655 (0.0006) [2023-12-27 00:15:33,776][105620] Updated weights for policy 1, policy_version 1219665 (0.0010) [2023-12-27 00:15:33,790][105692] Updated weights for policy 0, policy_version 1218422 (0.0007) [2023-12-27 00:15:33,831][105620] Updated weights for policy 1, policy_version 1219675 (0.0010) [2023-12-27 00:15:33,834][105692] Updated weights for policy 0, policy_version 1218432 (0.0008) [2023-12-27 00:15:33,883][105692] Updated weights for policy 0, policy_version 1218442 (0.0007) [2023-12-27 00:15:34,472][105620] Updated weights for policy 1, policy_version 1219685 (0.0008) [2023-12-27 00:15:34,536][105620] Updated weights for policy 1, policy_version 1219695 (0.0008) [2023-12-27 00:15:34,592][105620] Updated weights for policy 1, policy_version 1219705 (0.0008) [2023-12-27 00:15:34,693][105692] Updated weights for policy 0, policy_version 1218452 (0.0009) [2023-12-27 00:15:34,746][105692] Updated weights for policy 0, policy_version 1218462 (0.0010) [2023-12-27 00:15:34,795][105692] Updated weights for policy 0, policy_version 1218472 (0.0010) [2023-12-27 00:15:35,321][105620] Updated weights for policy 1, policy_version 1219715 (0.0009) [2023-12-27 00:15:35,376][105620] Updated weights for policy 1, policy_version 1219725 (0.0009) [2023-12-27 00:15:35,436][105620] Updated weights for policy 1, policy_version 1219735 (0.0005) [2023-12-27 00:15:35,564][105692] Updated weights for policy 0, policy_version 1218482 (0.0010) [2023-12-27 00:15:35,590][105585] KL-divergence is very high: 124.5627 [2023-12-27 00:15:35,599][105585] KL-divergence is very high: 183.0274 [2023-12-27 00:15:35,608][105692] Updated weights for policy 0, policy_version 1218492 (0.0010) [2023-12-27 00:15:35,629][105585] KL-divergence is very high: 322.2045 [2023-12-27 00:15:35,641][105585] KL-divergence is very high: 346.8094 [2023-12-27 00:15:35,647][105585] KL-divergence is very high: 129.4511 [2023-12-27 00:15:35,663][105692] Updated weights for policy 0, policy_version 1218502 (0.0010) [2023-12-27 00:15:35,677][105585] KL-divergence is very high: 389.0411 [2023-12-27 00:15:35,689][105585] KL-divergence is very high: 387.8860 [2023-12-27 00:15:35,696][105585] KL-divergence is very high: 117.0737 [2023-12-27 00:15:35,727][105692] Updated weights for policy 0, policy_version 1218512 (0.0010) [2023-12-27 00:15:36,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 624287744. Throughput: 0: 9977.4, 1: 9619.8. Samples: 624279688. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:15:36,062][104569] Avg episode reward: [(0, '8191.847'), (1, '9354.032')] [2023-12-27 00:15:36,117][105620] Updated weights for policy 1, policy_version 1219745 (0.0006) [2023-12-27 00:15:36,182][105620] Updated weights for policy 1, policy_version 1219755 (0.0011) [2023-12-27 00:15:36,245][105620] Updated weights for policy 1, policy_version 1219765 (0.0011) [2023-12-27 00:15:36,293][105620] Updated weights for policy 1, policy_version 1219775 (0.0010) [2023-12-27 00:15:36,490][105692] Updated weights for policy 0, policy_version 1218522 (0.0011) [2023-12-27 00:15:36,549][105692] Updated weights for policy 0, policy_version 1218532 (0.0011) [2023-12-27 00:15:36,615][105692] Updated weights for policy 0, policy_version 1218542 (0.0009) [2023-12-27 00:15:37,016][105620] Updated weights for policy 1, policy_version 1219785 (0.0010) [2023-12-27 00:15:37,072][105620] Updated weights for policy 1, policy_version 1219795 (0.0007) [2023-12-27 00:15:37,126][105620] Updated weights for policy 1, policy_version 1219805 (0.0009) [2023-12-27 00:15:37,301][105692] Updated weights for policy 0, policy_version 1218552 (0.0009) [2023-12-27 00:15:37,349][105692] Updated weights for policy 0, policy_version 1218562 (0.0010) [2023-12-27 00:15:37,404][105692] Updated weights for policy 0, policy_version 1218572 (0.0010) [2023-12-27 00:15:37,814][105620] Updated weights for policy 1, policy_version 1219815 (0.0008) [2023-12-27 00:15:37,872][105620] Updated weights for policy 1, policy_version 1219825 (0.0008) [2023-12-27 00:15:37,929][105620] Updated weights for policy 1, policy_version 1219835 (0.0010) [2023-12-27 00:15:38,166][105692] Updated weights for policy 0, policy_version 1218582 (0.0007) [2023-12-27 00:15:38,227][105692] Updated weights for policy 0, policy_version 1218592 (0.0005) [2023-12-27 00:15:38,289][105692] Updated weights for policy 0, policy_version 1218602 (0.0006) [2023-12-27 00:15:38,634][105620] Updated weights for policy 1, policy_version 1219845 (0.0010) [2023-12-27 00:15:38,688][105620] Updated weights for policy 1, policy_version 1219855 (0.0006) [2023-12-27 00:15:38,744][105620] Updated weights for policy 1, policy_version 1219865 (0.0006) [2023-12-27 00:15:38,897][105692] Updated weights for policy 0, policy_version 1218612 (0.0008) [2023-12-27 00:15:38,949][105692] Updated weights for policy 0, policy_version 1218622 (0.0010) [2023-12-27 00:15:39,001][105692] Updated weights for policy 0, policy_version 1218632 (0.0010) [2023-12-27 00:15:39,388][105620] Updated weights for policy 1, policy_version 1219875 (0.0007) [2023-12-27 00:15:39,458][105620] Updated weights for policy 1, policy_version 1219885 (0.0007) [2023-12-27 00:15:39,516][105620] Updated weights for policy 1, policy_version 1219895 (0.0008) [2023-12-27 00:15:39,776][105692] Updated weights for policy 0, policy_version 1218642 (0.0011) [2023-12-27 00:15:39,835][105692] Updated weights for policy 0, policy_version 1218652 (0.0010) [2023-12-27 00:15:39,899][105692] Updated weights for policy 0, policy_version 1218662 (0.0010) [2023-12-27 00:15:39,969][105692] Updated weights for policy 0, policy_version 1218672 (0.0009) [2023-12-27 00:15:40,327][105620] Updated weights for policy 1, policy_version 1219905 (0.0008) [2023-12-27 00:15:40,389][105620] Updated weights for policy 1, policy_version 1219915 (0.0009) [2023-12-27 00:15:40,453][105620] Updated weights for policy 1, policy_version 1219925 (0.0010) [2023-12-27 00:15:40,508][105620] Updated weights for policy 1, policy_version 1219935 (0.0009) [2023-12-27 00:15:40,697][105692] Updated weights for policy 0, policy_version 1218682 (0.0007) [2023-12-27 00:15:40,752][105692] Updated weights for policy 0, policy_version 1218692 (0.0009) [2023-12-27 00:15:40,804][105692] Updated weights for policy 0, policy_version 1218702 (0.0009) [2023-12-27 00:15:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 624386048. Throughput: 0: 9910.1, 1: 9608.2. Samples: 624395792. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:15:41,063][104569] Avg episode reward: [(0, '7385.433'), (1, '9077.987')] [2023-12-27 00:15:41,275][105620] Updated weights for policy 1, policy_version 1219945 (0.0008) [2023-12-27 00:15:41,334][105620] Updated weights for policy 1, policy_version 1219955 (0.0008) [2023-12-27 00:15:41,405][105620] Updated weights for policy 1, policy_version 1219965 (0.0008) [2023-12-27 00:15:41,567][105692] Updated weights for policy 0, policy_version 1218712 (0.0008) [2023-12-27 00:15:41,579][105585] KL-divergence is very high: 122.3628 [2023-12-27 00:15:41,631][105692] Updated weights for policy 0, policy_version 1218722 (0.0008) [2023-12-27 00:15:41,632][105585] KL-divergence is very high: 204.7981 [2023-12-27 00:15:41,683][105585] KL-divergence is very high: 230.1685 [2023-12-27 00:15:41,693][105692] Updated weights for policy 0, policy_version 1218732 (0.0008) [2023-12-27 00:15:42,195][105620] Updated weights for policy 1, policy_version 1219975 (0.0009) [2023-12-27 00:15:42,268][105620] Updated weights for policy 1, policy_version 1219985 (0.0009) [2023-12-27 00:15:42,322][105585] KL-divergence is very high: 129.4269 [2023-12-27 00:15:42,332][105620] Updated weights for policy 1, policy_version 1219995 (0.0008) [2023-12-27 00:15:42,355][105692] Updated weights for policy 0, policy_version 1218742 (0.0009) [2023-12-27 00:15:42,370][105585] KL-divergence is very high: 117.4778 [2023-12-27 00:15:42,408][105692] Updated weights for policy 0, policy_version 1218752 (0.0009) [2023-12-27 00:15:42,412][105585] KL-divergence is very high: 109.5408 [2023-12-27 00:15:42,466][105692] Updated weights for policy 0, policy_version 1218762 (0.0010) [2023-12-27 00:15:42,948][105620] Updated weights for policy 1, policy_version 1220005 (0.0008) [2023-12-27 00:15:42,999][105620] Updated weights for policy 1, policy_version 1220015 (0.0010) [2023-12-27 00:15:43,061][105620] Updated weights for policy 1, policy_version 1220025 (0.0009) [2023-12-27 00:15:43,265][105692] Updated weights for policy 0, policy_version 1218772 (0.0010) [2023-12-27 00:15:43,316][105692] Updated weights for policy 0, policy_version 1218782 (0.0009) [2023-12-27 00:15:43,378][105692] Updated weights for policy 0, policy_version 1218792 (0.0009) [2023-12-27 00:15:43,726][105620] Updated weights for policy 1, policy_version 1220035 (0.0008) [2023-12-27 00:15:43,779][105620] Updated weights for policy 1, policy_version 1220045 (0.0006) [2023-12-27 00:15:43,831][105620] Updated weights for policy 1, policy_version 1220055 (0.0005) [2023-12-27 00:15:44,157][105692] Updated weights for policy 0, policy_version 1218802 (0.0009) [2023-12-27 00:15:44,204][105692] Updated weights for policy 0, policy_version 1218812 (0.0008) [2023-12-27 00:15:44,251][105692] Updated weights for policy 0, policy_version 1218822 (0.0009) [2023-12-27 00:15:44,298][105692] Updated weights for policy 0, policy_version 1218832 (0.0008) [2023-12-27 00:15:44,574][105620] Updated weights for policy 1, policy_version 1220065 (0.0006) [2023-12-27 00:15:44,633][105620] Updated weights for policy 1, policy_version 1220075 (0.0009) [2023-12-27 00:15:44,699][105620] Updated weights for policy 1, policy_version 1220085 (0.0009) [2023-12-27 00:15:44,764][105620] Updated weights for policy 1, policy_version 1220095 (0.0008) [2023-12-27 00:15:44,995][105692] Updated weights for policy 0, policy_version 1218842 (0.0009) [2023-12-27 00:15:45,050][105692] Updated weights for policy 0, policy_version 1218852 (0.0009) [2023-12-27 00:15:45,101][105692] Updated weights for policy 0, policy_version 1218862 (0.0009) [2023-12-27 00:15:45,515][105620] Updated weights for policy 1, policy_version 1220105 (0.0009) [2023-12-27 00:15:45,564][105620] Updated weights for policy 1, policy_version 1220115 (0.0008) [2023-12-27 00:15:45,621][105620] Updated weights for policy 1, policy_version 1220125 (0.0008) [2023-12-27 00:15:45,897][105692] Updated weights for policy 0, policy_version 1218872 (0.0009) [2023-12-27 00:15:45,955][105692] Updated weights for policy 0, policy_version 1218882 (0.0008) [2023-12-27 00:15:46,015][105692] Updated weights for policy 0, policy_version 1218892 (0.0009) [2023-12-27 00:15:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 624484352. Throughput: 0: 9816.2, 1: 9647.9. Samples: 624453148. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:15:46,063][104569] Avg episode reward: [(0, '7014.575'), (1, '9076.440')] [2023-12-27 00:15:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001220128_312393728.pth... [2023-12-27 00:15:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001218896_312090624.pth... [2023-12-27 00:15:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001219008_312107008.pth [2023-12-27 00:15:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001217712_311787520.pth [2023-12-27 00:15:46,242][105620] Updated weights for policy 1, policy_version 1220135 (0.0008) [2023-12-27 00:15:46,305][105620] Updated weights for policy 1, policy_version 1220145 (0.0007) [2023-12-27 00:15:46,372][105620] Updated weights for policy 1, policy_version 1220155 (0.0007) [2023-12-27 00:15:46,639][105692] Updated weights for policy 0, policy_version 1218902 (0.0006) [2023-12-27 00:15:46,687][105692] Updated weights for policy 0, policy_version 1218912 (0.0005) [2023-12-27 00:15:46,736][105692] Updated weights for policy 0, policy_version 1218922 (0.0005) [2023-12-27 00:15:46,975][105620] Updated weights for policy 1, policy_version 1220165 (0.0006) [2023-12-27 00:15:47,031][105620] Updated weights for policy 1, policy_version 1220175 (0.0006) [2023-12-27 00:15:47,079][105620] Updated weights for policy 1, policy_version 1220185 (0.0006) [2023-12-27 00:15:47,368][105692] Updated weights for policy 0, policy_version 1218932 (0.0007) [2023-12-27 00:15:47,426][105692] Updated weights for policy 0, policy_version 1218942 (0.0010) [2023-12-27 00:15:47,484][105692] Updated weights for policy 0, policy_version 1218952 (0.0010) [2023-12-27 00:15:47,686][105620] Updated weights for policy 1, policy_version 1220195 (0.0007) [2023-12-27 00:15:47,734][105620] Updated weights for policy 1, policy_version 1220205 (0.0010) [2023-12-27 00:15:47,797][105620] Updated weights for policy 1, policy_version 1220215 (0.0010) [2023-12-27 00:15:48,275][105692] Updated weights for policy 0, policy_version 1218962 (0.0009) [2023-12-27 00:15:48,348][105692] Updated weights for policy 0, policy_version 1218972 (0.0009) [2023-12-27 00:15:48,406][105692] Updated weights for policy 0, policy_version 1218982 (0.0009) [2023-12-27 00:15:48,460][105692] Updated weights for policy 0, policy_version 1218992 (0.0009) [2023-12-27 00:15:48,492][105620] Updated weights for policy 1, policy_version 1220225 (0.0010) [2023-12-27 00:15:48,558][105620] Updated weights for policy 1, policy_version 1220235 (0.0011) [2023-12-27 00:15:48,624][105620] Updated weights for policy 1, policy_version 1220245 (0.0011) [2023-12-27 00:15:48,691][105620] Updated weights for policy 1, policy_version 1220255 (0.0011) [2023-12-27 00:15:49,194][105692] Updated weights for policy 0, policy_version 1219002 (0.0008) [2023-12-27 00:15:49,256][105692] Updated weights for policy 0, policy_version 1219012 (0.0008) [2023-12-27 00:15:49,318][105692] Updated weights for policy 0, policy_version 1219022 (0.0009) [2023-12-27 00:15:49,417][105620] Updated weights for policy 1, policy_version 1220265 (0.0010) [2023-12-27 00:15:49,462][105620] Updated weights for policy 1, policy_version 1220275 (0.0010) [2023-12-27 00:15:49,510][105620] Updated weights for policy 1, policy_version 1220285 (0.0010) [2023-12-27 00:15:50,128][105692] Updated weights for policy 0, policy_version 1219032 (0.0009) [2023-12-27 00:15:50,190][105692] Updated weights for policy 0, policy_version 1219042 (0.0009) [2023-12-27 00:15:50,223][105620] Updated weights for policy 1, policy_version 1220295 (0.0011) [2023-12-27 00:15:50,254][105692] Updated weights for policy 0, policy_version 1219052 (0.0005) [2023-12-27 00:15:50,287][105620] Updated weights for policy 1, policy_version 1220305 (0.0011) [2023-12-27 00:15:50,346][105620] Updated weights for policy 1, policy_version 1220315 (0.0010) [2023-12-27 00:15:50,976][105620] Updated weights for policy 1, policy_version 1220325 (0.0007) [2023-12-27 00:15:51,042][105620] Updated weights for policy 1, policy_version 1220335 (0.0008) [2023-12-27 00:15:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 624574464. Throughput: 0: 9831.6, 1: 9562.8. Samples: 624571348. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:15:51,062][104569] Avg episode reward: [(0, '7007.277'), (1, '9350.805')] [2023-12-27 00:15:51,105][105620] Updated weights for policy 1, policy_version 1220345 (0.0007) [2023-12-27 00:15:51,105][105692] Updated weights for policy 0, policy_version 1219062 (0.0007) [2023-12-27 00:15:51,166][105692] Updated weights for policy 0, policy_version 1219072 (0.0008) [2023-12-27 00:15:51,221][105692] Updated weights for policy 0, policy_version 1219082 (0.0009) [2023-12-27 00:15:51,818][105620] Updated weights for policy 1, policy_version 1220355 (0.0007) [2023-12-27 00:15:51,881][105620] Updated weights for policy 1, policy_version 1220365 (0.0006) [2023-12-27 00:15:51,936][105620] Updated weights for policy 1, policy_version 1220375 (0.0007) [2023-12-27 00:15:52,073][105692] Updated weights for policy 0, policy_version 1219092 (0.0010) [2023-12-27 00:15:52,118][105692] Updated weights for policy 0, policy_version 1219102 (0.0010) [2023-12-27 00:15:52,170][105692] Updated weights for policy 0, policy_version 1219112 (0.0011) [2023-12-27 00:15:52,709][105620] Updated weights for policy 1, policy_version 1220385 (0.0008) [2023-12-27 00:15:52,776][105620] Updated weights for policy 1, policy_version 1220395 (0.0008) [2023-12-27 00:15:52,830][105692] Updated weights for policy 0, policy_version 1219122 (0.0010) [2023-12-27 00:15:52,844][105620] Updated weights for policy 1, policy_version 1220405 (0.0007) [2023-12-27 00:15:52,884][105692] Updated weights for policy 0, policy_version 1219132 (0.0006) [2023-12-27 00:15:52,899][105620] Updated weights for policy 1, policy_version 1220415 (0.0009) [2023-12-27 00:15:52,944][105692] Updated weights for policy 0, policy_version 1219142 (0.0010) [2023-12-27 00:15:53,004][105692] Updated weights for policy 0, policy_version 1219152 (0.0011) [2023-12-27 00:15:53,651][105620] Updated weights for policy 1, policy_version 1220425 (0.0008) [2023-12-27 00:15:53,670][105692] Updated weights for policy 0, policy_version 1219162 (0.0006) [2023-12-27 00:15:53,709][105620] Updated weights for policy 1, policy_version 1220435 (0.0008) [2023-12-27 00:15:53,728][105692] Updated weights for policy 0, policy_version 1219172 (0.0006) [2023-12-27 00:15:53,772][105620] Updated weights for policy 1, policy_version 1220445 (0.0008) [2023-12-27 00:15:53,791][105692] Updated weights for policy 0, policy_version 1219182 (0.0007) [2023-12-27 00:15:54,540][105620] Updated weights for policy 1, policy_version 1220455 (0.0009) [2023-12-27 00:15:54,570][105692] Updated weights for policy 0, policy_version 1219192 (0.0007) [2023-12-27 00:15:54,603][105620] Updated weights for policy 1, policy_version 1220465 (0.0010) [2023-12-27 00:15:54,629][105692] Updated weights for policy 0, policy_version 1219202 (0.0005) [2023-12-27 00:15:54,669][105620] Updated weights for policy 1, policy_version 1220475 (0.0010) [2023-12-27 00:15:54,687][105692] Updated weights for policy 0, policy_version 1219212 (0.0008) [2023-12-27 00:15:55,390][105692] Updated weights for policy 0, policy_version 1219222 (0.0008) [2023-12-27 00:15:55,402][105620] Updated weights for policy 1, policy_version 1220485 (0.0011) [2023-12-27 00:15:55,444][105692] Updated weights for policy 0, policy_version 1219232 (0.0008) [2023-12-27 00:15:55,457][105620] Updated weights for policy 1, policy_version 1220495 (0.0010) [2023-12-27 00:15:55,504][105692] Updated weights for policy 0, policy_version 1219242 (0.0005) [2023-12-27 00:15:55,510][105620] Updated weights for policy 1, policy_version 1220505 (0.0010) [2023-12-27 00:15:56,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 624672768. Throughput: 0: 9791.1, 1: 9559.0. Samples: 624685272. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:15:56,063][104569] Avg episode reward: [(0, '7276.482'), (1, '9171.869')] [2023-12-27 00:15:56,127][105620] Updated weights for policy 1, policy_version 1220515 (0.0009) [2023-12-27 00:15:56,183][105620] Updated weights for policy 1, policy_version 1220525 (0.0005) [2023-12-27 00:15:56,240][105620] Updated weights for policy 1, policy_version 1220535 (0.0010) [2023-12-27 00:15:56,387][105692] Updated weights for policy 0, policy_version 1219252 (0.0007) [2023-12-27 00:15:56,458][105692] Updated weights for policy 0, policy_version 1219262 (0.0010) [2023-12-27 00:15:56,516][105692] Updated weights for policy 0, policy_version 1219272 (0.0010) [2023-12-27 00:15:56,797][105620] Updated weights for policy 1, policy_version 1220545 (0.0010) [2023-12-27 00:15:56,853][105620] Updated weights for policy 1, policy_version 1220555 (0.0010) [2023-12-27 00:15:56,911][105620] Updated weights for policy 1, policy_version 1220565 (0.0010) [2023-12-27 00:15:56,973][105620] Updated weights for policy 1, policy_version 1220575 (0.0010) [2023-12-27 00:15:57,349][105692] Updated weights for policy 0, policy_version 1219282 (0.0010) [2023-12-27 00:15:57,404][105692] Updated weights for policy 0, policy_version 1219293 (0.0010) [2023-12-27 00:15:57,469][105692] Updated weights for policy 0, policy_version 1219303 (0.0008) [2023-12-27 00:15:57,646][105620] Updated weights for policy 1, policy_version 1220585 (0.0009) [2023-12-27 00:15:57,694][105620] Updated weights for policy 1, policy_version 1220595 (0.0009) [2023-12-27 00:15:57,740][105620] Updated weights for policy 1, policy_version 1220605 (0.0008) [2023-12-27 00:15:58,266][105692] Updated weights for policy 0, policy_version 1219313 (0.0008) [2023-12-27 00:15:58,323][105692] Updated weights for policy 0, policy_version 1219323 (0.0009) [2023-12-27 00:15:58,400][105692] Updated weights for policy 0, policy_version 1219333 (0.0009) [2023-12-27 00:15:58,466][105692] Updated weights for policy 0, policy_version 1219343 (0.0010) [2023-12-27 00:15:58,525][105620] Updated weights for policy 1, policy_version 1220615 (0.0008) [2023-12-27 00:15:58,593][105620] Updated weights for policy 1, policy_version 1220625 (0.0007) [2023-12-27 00:15:58,664][105620] Updated weights for policy 1, policy_version 1220635 (0.0008) [2023-12-27 00:15:59,204][105692] Updated weights for policy 0, policy_version 1219353 (0.0009) [2023-12-27 00:15:59,262][105692] Updated weights for policy 0, policy_version 1219363 (0.0009) [2023-12-27 00:15:59,328][105692] Updated weights for policy 0, policy_version 1219373 (0.0008) [2023-12-27 00:15:59,531][105620] Updated weights for policy 1, policy_version 1220645 (0.0008) [2023-12-27 00:15:59,592][105620] Updated weights for policy 1, policy_version 1220655 (0.0008) [2023-12-27 00:15:59,639][105620] Updated weights for policy 1, policy_version 1220665 (0.0009) [2023-12-27 00:16:00,099][105692] Updated weights for policy 0, policy_version 1219383 (0.0007) [2023-12-27 00:16:00,155][105692] Updated weights for policy 0, policy_version 1219393 (0.0006) [2023-12-27 00:16:00,214][105692] Updated weights for policy 0, policy_version 1219403 (0.0007) [2023-12-27 00:16:00,467][105620] Updated weights for policy 1, policy_version 1220675 (0.0008) [2023-12-27 00:16:00,533][105620] Updated weights for policy 1, policy_version 1220685 (0.0008) [2023-12-27 00:16:00,591][105620] Updated weights for policy 1, policy_version 1220695 (0.0009) [2023-12-27 00:16:00,861][105692] Updated weights for policy 0, policy_version 1219413 (0.0008) [2023-12-27 00:16:00,922][105692] Updated weights for policy 0, policy_version 1219423 (0.0006) [2023-12-27 00:16:00,977][105692] Updated weights for policy 0, policy_version 1219433 (0.0009) [2023-12-27 00:16:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 624771072. Throughput: 0: 9725.3, 1: 9614.0. Samples: 624740584. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:16:01,063][104569] Avg episode reward: [(0, '7634.593'), (1, '8899.820')] [2023-12-27 00:16:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001219440_312229888.pth... [2023-12-27 00:16:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001220704_312541184.pth... [2023-12-27 00:16:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001218320_311943168.pth [2023-12-27 00:16:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001219552_312246272.pth [2023-12-27 00:16:01,336][105620] Updated weights for policy 1, policy_version 1220706 (0.0009) [2023-12-27 00:16:01,401][105620] Updated weights for policy 1, policy_version 1220716 (0.0009) [2023-12-27 00:16:01,464][105620] Updated weights for policy 1, policy_version 1220726 (0.0010) [2023-12-27 00:16:01,522][105620] Updated weights for policy 1, policy_version 1220736 (0.0009) [2023-12-27 00:16:01,736][105692] Updated weights for policy 0, policy_version 1219443 (0.0010) [2023-12-27 00:16:01,805][105692] Updated weights for policy 0, policy_version 1219453 (0.0010) [2023-12-27 00:16:01,870][105692] Updated weights for policy 0, policy_version 1219463 (0.0009) [2023-12-27 00:16:02,279][105620] Updated weights for policy 1, policy_version 1220746 (0.0009) [2023-12-27 00:16:02,341][105620] Updated weights for policy 1, policy_version 1220756 (0.0008) [2023-12-27 00:16:02,406][105620] Updated weights for policy 1, policy_version 1220766 (0.0008) [2023-12-27 00:16:02,548][105692] Updated weights for policy 0, policy_version 1219473 (0.0007) [2023-12-27 00:16:02,600][105692] Updated weights for policy 0, policy_version 1219483 (0.0010) [2023-12-27 00:16:02,658][105692] Updated weights for policy 0, policy_version 1219493 (0.0011) [2023-12-27 00:16:02,707][105692] Updated weights for policy 0, policy_version 1219503 (0.0010) [2023-12-27 00:16:03,124][105620] Updated weights for policy 1, policy_version 1220776 (0.0008) [2023-12-27 00:16:03,184][105620] Updated weights for policy 1, policy_version 1220786 (0.0008) [2023-12-27 00:16:03,246][105620] Updated weights for policy 1, policy_version 1220796 (0.0010) [2023-12-27 00:16:03,392][105692] Updated weights for policy 0, policy_version 1219513 (0.0009) [2023-12-27 00:16:03,407][105585] KL-divergence is very high: 123.7179 [2023-12-27 00:16:03,440][105692] Updated weights for policy 0, policy_version 1219523 (0.0008) [2023-12-27 00:16:03,448][105585] KL-divergence is very high: 234.2913 [2023-12-27 00:16:03,486][105585] KL-divergence is very high: 258.6648 [2023-12-27 00:16:03,490][105692] Updated weights for policy 0, policy_version 1219533 (0.0005) [2023-12-27 00:16:03,931][105620] Updated weights for policy 1, policy_version 1220806 (0.0010) [2023-12-27 00:16:03,995][105620] Updated weights for policy 1, policy_version 1220816 (0.0011) [2023-12-27 00:16:04,063][105620] Updated weights for policy 1, policy_version 1220826 (0.0011) [2023-12-27 00:16:04,142][105585] KL-divergence is very high: 151.9810 [2023-12-27 00:16:04,182][105692] Updated weights for policy 0, policy_version 1219543 (0.0009) [2023-12-27 00:16:04,196][105585] KL-divergence is very high: 130.4838 [2023-12-27 00:16:04,247][105692] Updated weights for policy 0, policy_version 1219553 (0.0011) [2023-12-27 00:16:04,248][105585] KL-divergence is very high: 108.7282 [2023-12-27 00:16:04,315][105692] Updated weights for policy 0, policy_version 1219563 (0.0011) [2023-12-27 00:16:04,815][105620] Updated weights for policy 1, policy_version 1220836 (0.0011) [2023-12-27 00:16:04,867][105620] Updated weights for policy 1, policy_version 1220846 (0.0010) [2023-12-27 00:16:04,916][105692] Updated weights for policy 0, policy_version 1219573 (0.0010) [2023-12-27 00:16:04,920][105620] Updated weights for policy 1, policy_version 1220856 (0.0008) [2023-12-27 00:16:04,967][105692] Updated weights for policy 0, policy_version 1219583 (0.0010) [2023-12-27 00:16:05,028][105692] Updated weights for policy 0, policy_version 1219593 (0.0010) [2023-12-27 00:16:05,651][105620] Updated weights for policy 1, policy_version 1220866 (0.0010) [2023-12-27 00:16:05,708][105620] Updated weights for policy 1, policy_version 1220876 (0.0007) [2023-12-27 00:16:05,749][105692] Updated weights for policy 0, policy_version 1219603 (0.0009) [2023-12-27 00:16:05,763][105620] Updated weights for policy 1, policy_version 1220886 (0.0008) [2023-12-27 00:16:05,806][105692] Updated weights for policy 0, policy_version 1219613 (0.0007) [2023-12-27 00:16:05,824][105620] Updated weights for policy 1, policy_version 1220896 (0.0010) [2023-12-27 00:16:05,862][105692] Updated weights for policy 0, policy_version 1219623 (0.0005) [2023-12-27 00:16:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 624869376. Throughput: 0: 9626.4, 1: 9627.2. Samples: 624854308. Policy #0 lag: (min: 25.0, avg: 50.9, max: 57.0) [2023-12-27 00:16:06,063][104569] Avg episode reward: [(0, '7638.466'), (1, '9080.231')] [2023-12-27 00:16:06,499][105692] Updated weights for policy 0, policy_version 1219633 (0.0005) [2023-12-27 00:16:06,503][105620] Updated weights for policy 1, policy_version 1220906 (0.0009) [2023-12-27 00:16:06,557][105692] Updated weights for policy 0, policy_version 1219643 (0.0008) [2023-12-27 00:16:06,564][105620] Updated weights for policy 1, policy_version 1220916 (0.0005) [2023-12-27 00:16:06,613][105692] Updated weights for policy 0, policy_version 1219653 (0.0008) [2023-12-27 00:16:06,617][105620] Updated weights for policy 1, policy_version 1220926 (0.0005) [2023-12-27 00:16:06,679][105692] Updated weights for policy 0, policy_version 1219663 (0.0008) [2023-12-27 00:16:07,307][105620] Updated weights for policy 1, policy_version 1220936 (0.0009) [2023-12-27 00:16:07,352][105692] Updated weights for policy 0, policy_version 1219673 (0.0010) [2023-12-27 00:16:07,355][105585] KL-divergence is very high: 151.4451 [2023-12-27 00:16:07,356][105620] Updated weights for policy 1, policy_version 1220946 (0.0010) [2023-12-27 00:16:07,393][105585] KL-divergence is very high: 279.4246 [2023-12-27 00:16:07,397][105692] Updated weights for policy 0, policy_version 1219683 (0.0005) [2023-12-27 00:16:07,409][105620] Updated weights for policy 1, policy_version 1220956 (0.0010) [2023-12-27 00:16:07,432][105585] KL-divergence is very high: 298.1331 [2023-12-27 00:16:07,447][105692] Updated weights for policy 0, policy_version 1219693 (0.0005) [2023-12-27 00:16:08,067][105692] Updated weights for policy 0, policy_version 1219703 (0.0009) [2023-12-27 00:16:08,079][105620] Updated weights for policy 1, policy_version 1220966 (0.0010) [2023-12-27 00:16:08,112][105692] Updated weights for policy 0, policy_version 1219713 (0.0010) [2023-12-27 00:16:08,135][105620] Updated weights for policy 1, policy_version 1220976 (0.0011) [2023-12-27 00:16:08,158][105692] Updated weights for policy 0, policy_version 1219723 (0.0010) [2023-12-27 00:16:08,191][105620] Updated weights for policy 1, policy_version 1220986 (0.0007) [2023-12-27 00:16:08,826][105620] Updated weights for policy 1, policy_version 1220996 (0.0007) [2023-12-27 00:16:08,881][105620] Updated weights for policy 1, policy_version 1221006 (0.0010) [2023-12-27 00:16:08,933][105620] Updated weights for policy 1, policy_version 1221016 (0.0010) [2023-12-27 00:16:08,952][105692] Updated weights for policy 0, policy_version 1219733 (0.0010) [2023-12-27 00:16:09,019][105692] Updated weights for policy 0, policy_version 1219743 (0.0011) [2023-12-27 00:16:09,086][105692] Updated weights for policy 0, policy_version 1219753 (0.0011) [2023-12-27 00:16:09,733][105620] Updated weights for policy 1, policy_version 1221026 (0.0010) [2023-12-27 00:16:09,794][105620] Updated weights for policy 1, policy_version 1221036 (0.0009) [2023-12-27 00:16:09,858][105620] Updated weights for policy 1, policy_version 1221046 (0.0007) [2023-12-27 00:16:09,861][105692] Updated weights for policy 0, policy_version 1219763 (0.0009) [2023-12-27 00:16:09,920][105620] Updated weights for policy 1, policy_version 1221056 (0.0006) [2023-12-27 00:16:09,927][105692] Updated weights for policy 0, policy_version 1219773 (0.0007) [2023-12-27 00:16:09,980][105692] Updated weights for policy 0, policy_version 1219783 (0.0008) [2023-12-27 00:16:10,664][105620] Updated weights for policy 1, policy_version 1221066 (0.0009) [2023-12-27 00:16:10,719][105620] Updated weights for policy 1, policy_version 1221076 (0.0009) [2023-12-27 00:16:10,726][105692] Updated weights for policy 0, policy_version 1219793 (0.0007) [2023-12-27 00:16:10,767][105620] Updated weights for policy 1, policy_version 1221086 (0.0006) [2023-12-27 00:16:10,787][105692] Updated weights for policy 0, policy_version 1219803 (0.0009) [2023-12-27 00:16:10,849][105692] Updated weights for policy 0, policy_version 1219813 (0.0010) [2023-12-27 00:16:10,911][105692] Updated weights for policy 0, policy_version 1219823 (0.0009) [2023-12-27 00:16:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 624967680. Throughput: 0: 9545.7, 1: 9800.9. Samples: 624973740. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:16:11,063][104569] Avg episode reward: [(0, '8003.363'), (1, '9168.602')] [2023-12-27 00:16:11,469][105620] Updated weights for policy 1, policy_version 1221096 (0.0010) [2023-12-27 00:16:11,536][105620] Updated weights for policy 1, policy_version 1221106 (0.0011) [2023-12-27 00:16:11,597][105620] Updated weights for policy 1, policy_version 1221116 (0.0011) [2023-12-27 00:16:11,728][105692] Updated weights for policy 0, policy_version 1219833 (0.0008) [2023-12-27 00:16:11,793][105692] Updated weights for policy 0, policy_version 1219843 (0.0008) [2023-12-27 00:16:11,854][105692] Updated weights for policy 0, policy_version 1219853 (0.0008) [2023-12-27 00:16:12,354][105620] Updated weights for policy 1, policy_version 1221126 (0.0010) [2023-12-27 00:16:12,414][105620] Updated weights for policy 1, policy_version 1221136 (0.0011) [2023-12-27 00:16:12,473][105620] Updated weights for policy 1, policy_version 1221146 (0.0011) [2023-12-27 00:16:12,564][105692] Updated weights for policy 0, policy_version 1219863 (0.0007) [2023-12-27 00:16:12,617][105692] Updated weights for policy 0, policy_version 1219873 (0.0009) [2023-12-27 00:16:12,674][105692] Updated weights for policy 0, policy_version 1219883 (0.0010) [2023-12-27 00:16:13,131][105620] Updated weights for policy 1, policy_version 1221156 (0.0009) [2023-12-27 00:16:13,186][105620] Updated weights for policy 1, policy_version 1221166 (0.0010) [2023-12-27 00:16:13,235][105620] Updated weights for policy 1, policy_version 1221176 (0.0010) [2023-12-27 00:16:13,415][105692] Updated weights for policy 0, policy_version 1219893 (0.0007) [2023-12-27 00:16:13,481][105692] Updated weights for policy 0, policy_version 1219903 (0.0005) [2023-12-27 00:16:13,538][105692] Updated weights for policy 0, policy_version 1219913 (0.0005) [2023-12-27 00:16:13,976][105620] Updated weights for policy 1, policy_version 1221186 (0.0010) [2023-12-27 00:16:14,034][105620] Updated weights for policy 1, policy_version 1221196 (0.0011) [2023-12-27 00:16:14,083][105692] Updated weights for policy 0, policy_version 1219923 (0.0005) [2023-12-27 00:16:14,104][105620] Updated weights for policy 1, policy_version 1221206 (0.0011) [2023-12-27 00:16:14,141][105692] Updated weights for policy 0, policy_version 1219933 (0.0006) [2023-12-27 00:16:14,166][105620] Updated weights for policy 1, policy_version 1221216 (0.0010) [2023-12-27 00:16:14,199][105692] Updated weights for policy 0, policy_version 1219943 (0.0008) [2023-12-27 00:16:14,825][105620] Updated weights for policy 1, policy_version 1221226 (0.0011) [2023-12-27 00:16:14,889][105620] Updated weights for policy 1, policy_version 1221236 (0.0010) [2023-12-27 00:16:14,949][105692] Updated weights for policy 0, policy_version 1219953 (0.0008) [2023-12-27 00:16:14,951][105620] Updated weights for policy 1, policy_version 1221246 (0.0011) [2023-12-27 00:16:15,011][105692] Updated weights for policy 0, policy_version 1219963 (0.0008) [2023-12-27 00:16:15,075][105692] Updated weights for policy 0, policy_version 1219973 (0.0008) [2023-12-27 00:16:15,135][105692] Updated weights for policy 0, policy_version 1219983 (0.0008) [2023-12-27 00:16:15,708][105620] Updated weights for policy 1, policy_version 1221256 (0.0011) [2023-12-27 00:16:15,772][105620] Updated weights for policy 1, policy_version 1221266 (0.0010) [2023-12-27 00:16:15,818][105692] Updated weights for policy 0, policy_version 1219993 (0.0006) [2023-12-27 00:16:15,827][105620] Updated weights for policy 1, policy_version 1221276 (0.0011) [2023-12-27 00:16:15,877][105692] Updated weights for policy 0, policy_version 1220003 (0.0006) [2023-12-27 00:16:15,925][105692] Updated weights for policy 0, policy_version 1220013 (0.0008) [2023-12-27 00:16:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 625065984. Throughput: 0: 9528.4, 1: 9791.7. Samples: 625031552. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:16:16,063][104569] Avg episode reward: [(0, '7559.830'), (1, '9075.902')] [2023-12-27 00:16:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001220016_312377344.pth... [2023-12-27 00:16:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001221280_312688640.pth... [2023-12-27 00:16:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001218896_312090624.pth [2023-12-27 00:16:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001220128_312393728.pth [2023-12-27 00:16:16,529][105692] Updated weights for policy 0, policy_version 1220023 (0.0006) [2023-12-27 00:16:16,566][105620] Updated weights for policy 1, policy_version 1221286 (0.0011) [2023-12-27 00:16:16,582][105692] Updated weights for policy 0, policy_version 1220033 (0.0005) [2023-12-27 00:16:16,625][105620] Updated weights for policy 1, policy_version 1221296 (0.0011) [2023-12-27 00:16:16,636][105692] Updated weights for policy 0, policy_version 1220043 (0.0005) [2023-12-27 00:16:16,688][105620] Updated weights for policy 1, policy_version 1221306 (0.0011) [2023-12-27 00:16:17,204][105692] Updated weights for policy 0, policy_version 1220053 (0.0007) [2023-12-27 00:16:17,256][105692] Updated weights for policy 0, policy_version 1220063 (0.0008) [2023-12-27 00:16:17,308][105692] Updated weights for policy 0, policy_version 1220073 (0.0008) [2023-12-27 00:16:17,433][105620] Updated weights for policy 1, policy_version 1221316 (0.0011) [2023-12-27 00:16:17,484][105620] Updated weights for policy 1, policy_version 1221326 (0.0010) [2023-12-27 00:16:17,552][105620] Updated weights for policy 1, policy_version 1221336 (0.0010) [2023-12-27 00:16:17,982][105692] Updated weights for policy 0, policy_version 1220083 (0.0009) [2023-12-27 00:16:18,035][105692] Updated weights for policy 0, policy_version 1220093 (0.0010) [2023-12-27 00:16:18,089][105692] Updated weights for policy 0, policy_version 1220103 (0.0010) [2023-12-27 00:16:18,144][105620] Updated weights for policy 1, policy_version 1221346 (0.0011) [2023-12-27 00:16:18,194][105620] Updated weights for policy 1, policy_version 1221356 (0.0011) [2023-12-27 00:16:18,238][105620] Updated weights for policy 1, policy_version 1221366 (0.0010) [2023-12-27 00:16:18,290][105620] Updated weights for policy 1, policy_version 1221376 (0.0010) [2023-12-27 00:16:18,885][105692] Updated weights for policy 0, policy_version 1220114 (0.0008) [2023-12-27 00:16:18,937][105692] Updated weights for policy 0, policy_version 1220124 (0.0008) [2023-12-27 00:16:18,992][105692] Updated weights for policy 0, policy_version 1220134 (0.0008) [2023-12-27 00:16:19,049][105692] Updated weights for policy 0, policy_version 1220144 (0.0006) [2023-12-27 00:16:19,063][105620] Updated weights for policy 1, policy_version 1221386 (0.0011) [2023-12-27 00:16:19,112][105620] Updated weights for policy 1, policy_version 1221396 (0.0010) [2023-12-27 00:16:19,165][105620] Updated weights for policy 1, policy_version 1221406 (0.0010) [2023-12-27 00:16:19,869][105620] Updated weights for policy 1, policy_version 1221416 (0.0008) [2023-12-27 00:16:19,900][105692] Updated weights for policy 0, policy_version 1220154 (0.0009) [2023-12-27 00:16:19,936][105620] Updated weights for policy 1, policy_version 1221426 (0.0007) [2023-12-27 00:16:19,968][105692] Updated weights for policy 0, policy_version 1220164 (0.0010) [2023-12-27 00:16:19,995][105620] Updated weights for policy 1, policy_version 1221436 (0.0006) [2023-12-27 00:16:20,028][105692] Updated weights for policy 0, policy_version 1220174 (0.0010) [2023-12-27 00:16:20,773][105620] Updated weights for policy 1, policy_version 1221446 (0.0007) [2023-12-27 00:16:20,773][105692] Updated weights for policy 0, policy_version 1220184 (0.0006) [2023-12-27 00:16:20,834][105620] Updated weights for policy 1, policy_version 1221456 (0.0009) [2023-12-27 00:16:20,842][105692] Updated weights for policy 0, policy_version 1220194 (0.0006) [2023-12-27 00:16:20,899][105620] Updated weights for policy 1, policy_version 1221466 (0.0009) [2023-12-27 00:16:20,909][105692] Updated weights for policy 0, policy_version 1220204 (0.0006) [2023-12-27 00:16:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 625164288. Throughput: 0: 9612.9, 1: 9754.5. Samples: 625151220. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:16:21,063][104569] Avg episode reward: [(0, '7190.635'), (1, '9258.905')] [2023-12-27 00:16:21,620][105692] Updated weights for policy 0, policy_version 1220214 (0.0010) [2023-12-27 00:16:21,680][105692] Updated weights for policy 0, policy_version 1220224 (0.0009) [2023-12-27 00:16:21,725][105620] Updated weights for policy 1, policy_version 1221476 (0.0008) [2023-12-27 00:16:21,752][105692] Updated weights for policy 0, policy_version 1220234 (0.0008) [2023-12-27 00:16:21,782][105620] Updated weights for policy 1, policy_version 1221486 (0.0007) [2023-12-27 00:16:21,837][105620] Updated weights for policy 1, policy_version 1221496 (0.0008) [2023-12-27 00:16:22,543][105692] Updated weights for policy 0, policy_version 1220244 (0.0009) [2023-12-27 00:16:22,593][105692] Updated weights for policy 0, policy_version 1220254 (0.0009) [2023-12-27 00:16:22,615][105620] Updated weights for policy 1, policy_version 1221506 (0.0008) [2023-12-27 00:16:22,654][105692] Updated weights for policy 0, policy_version 1220264 (0.0008) [2023-12-27 00:16:22,676][105620] Updated weights for policy 1, policy_version 1221516 (0.0009) [2023-12-27 00:16:22,741][105620] Updated weights for policy 1, policy_version 1221526 (0.0008) [2023-12-27 00:16:22,789][105620] Updated weights for policy 1, policy_version 1221536 (0.0009) [2023-12-27 00:16:23,418][105692] Updated weights for policy 0, policy_version 1220274 (0.0007) [2023-12-27 00:16:23,470][105692] Updated weights for policy 0, policy_version 1220284 (0.0009) [2023-12-27 00:16:23,525][105692] Updated weights for policy 0, policy_version 1220294 (0.0008) [2023-12-27 00:16:23,544][105620] Updated weights for policy 1, policy_version 1221546 (0.0008) [2023-12-27 00:16:23,582][105692] Updated weights for policy 0, policy_version 1220304 (0.0006) [2023-12-27 00:16:23,598][105620] Updated weights for policy 1, policy_version 1221556 (0.0006) [2023-12-27 00:16:23,653][105620] Updated weights for policy 1, policy_version 1221566 (0.0009) [2023-12-27 00:16:24,312][105620] Updated weights for policy 1, policy_version 1221576 (0.0008) [2023-12-27 00:16:24,362][105620] Updated weights for policy 1, policy_version 1221586 (0.0007) [2023-12-27 00:16:24,368][105692] Updated weights for policy 0, policy_version 1220314 (0.0007) [2023-12-27 00:16:24,421][105692] Updated weights for policy 0, policy_version 1220324 (0.0007) [2023-12-27 00:16:24,427][105620] Updated weights for policy 1, policy_version 1221596 (0.0008) [2023-12-27 00:16:24,473][105692] Updated weights for policy 0, policy_version 1220334 (0.0009) [2023-12-27 00:16:25,185][105620] Updated weights for policy 1, policy_version 1221606 (0.0008) [2023-12-27 00:16:25,237][105692] Updated weights for policy 0, policy_version 1220344 (0.0008) [2023-12-27 00:16:25,239][105620] Updated weights for policy 1, policy_version 1221616 (0.0007) [2023-12-27 00:16:25,285][105692] Updated weights for policy 0, policy_version 1220354 (0.0005) [2023-12-27 00:16:25,291][105620] Updated weights for policy 1, policy_version 1221626 (0.0008) [2023-12-27 00:16:25,338][105692] Updated weights for policy 0, policy_version 1220364 (0.0006) [2023-12-27 00:16:26,055][105620] Updated weights for policy 1, policy_version 1221636 (0.0008) [2023-12-27 00:16:26,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 625246208. Throughput: 0: 9542.7, 1: 9684.0. Samples: 625260996. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:16:26,063][104569] Avg episode reward: [(0, '7006.782'), (1, '9167.961')] [2023-12-27 00:16:26,091][105692] Updated weights for policy 0, policy_version 1220374 (0.0007) [2023-12-27 00:16:26,110][105620] Updated weights for policy 1, policy_version 1221646 (0.0008) [2023-12-27 00:16:26,144][105692] Updated weights for policy 0, policy_version 1220384 (0.0007) [2023-12-27 00:16:26,170][105620] Updated weights for policy 1, policy_version 1221656 (0.0007) [2023-12-27 00:16:26,195][105692] Updated weights for policy 0, policy_version 1220394 (0.0007) [2023-12-27 00:16:26,841][105620] Updated weights for policy 1, policy_version 1221666 (0.0009) [2023-12-27 00:16:26,860][105692] Updated weights for policy 0, policy_version 1220404 (0.0009) [2023-12-27 00:16:26,902][105620] Updated weights for policy 1, policy_version 1221676 (0.0006) [2023-12-27 00:16:26,909][105692] Updated weights for policy 0, policy_version 1220414 (0.0007) [2023-12-27 00:16:26,958][105620] Updated weights for policy 1, policy_version 1221686 (0.0006) [2023-12-27 00:16:26,963][105692] Updated weights for policy 0, policy_version 1220424 (0.0008) [2023-12-27 00:16:27,018][105620] Updated weights for policy 1, policy_version 1221696 (0.0005) [2023-12-27 00:16:27,548][105620] Updated weights for policy 1, policy_version 1221706 (0.0010) [2023-12-27 00:16:27,603][105620] Updated weights for policy 1, policy_version 1221716 (0.0010) [2023-12-27 00:16:27,660][105620] Updated weights for policy 1, policy_version 1221726 (0.0007) [2023-12-27 00:16:27,857][105692] Updated weights for policy 0, policy_version 1220434 (0.0009) [2023-12-27 00:16:27,906][105692] Updated weights for policy 0, policy_version 1220444 (0.0010) [2023-12-27 00:16:27,950][105692] Updated weights for policy 0, policy_version 1220454 (0.0010) [2023-12-27 00:16:28,005][105692] Updated weights for policy 0, policy_version 1220464 (0.0010) [2023-12-27 00:16:28,353][105620] Updated weights for policy 1, policy_version 1221736 (0.0007) [2023-12-27 00:16:28,401][105620] Updated weights for policy 1, policy_version 1221746 (0.0008) [2023-12-27 00:16:28,455][105620] Updated weights for policy 1, policy_version 1221756 (0.0008) [2023-12-27 00:16:28,701][105692] Updated weights for policy 0, policy_version 1220474 (0.0010) [2023-12-27 00:16:28,749][105692] Updated weights for policy 0, policy_version 1220484 (0.0010) [2023-12-27 00:16:28,800][105692] Updated weights for policy 0, policy_version 1220494 (0.0010) [2023-12-27 00:16:29,147][105620] Updated weights for policy 1, policy_version 1221766 (0.0007) [2023-12-27 00:16:29,195][105620] Updated weights for policy 1, policy_version 1221776 (0.0005) [2023-12-27 00:16:29,252][105620] Updated weights for policy 1, policy_version 1221786 (0.0007) [2023-12-27 00:16:29,503][105692] Updated weights for policy 0, policy_version 1220504 (0.0006) [2023-12-27 00:16:29,559][105692] Updated weights for policy 0, policy_version 1220514 (0.0005) [2023-12-27 00:16:29,614][105692] Updated weights for policy 0, policy_version 1220524 (0.0005) [2023-12-27 00:16:29,934][105620] Updated weights for policy 1, policy_version 1221796 (0.0010) [2023-12-27 00:16:29,996][105620] Updated weights for policy 1, policy_version 1221806 (0.0008) [2023-12-27 00:16:30,057][105620] Updated weights for policy 1, policy_version 1221816 (0.0006) [2023-12-27 00:16:30,393][105692] Updated weights for policy 0, policy_version 1220534 (0.0009) [2023-12-27 00:16:30,452][105692] Updated weights for policy 0, policy_version 1220545 (0.0011) [2023-12-27 00:16:30,503][105692] Updated weights for policy 0, policy_version 1220555 (0.0009) [2023-12-27 00:16:30,599][105620] Updated weights for policy 1, policy_version 1221826 (0.0006) [2023-12-27 00:16:30,653][105620] Updated weights for policy 1, policy_version 1221836 (0.0008) [2023-12-27 00:16:30,703][105620] Updated weights for policy 1, policy_version 1221846 (0.0009) [2023-12-27 00:16:30,751][105620] Updated weights for policy 1, policy_version 1221856 (0.0008) [2023-12-27 00:16:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19438.7). Total num frames: 625352704. Throughput: 0: 9564.6, 1: 9731.3. Samples: 625321464. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:16:31,063][104569] Avg episode reward: [(0, '7279.544'), (1, '8986.735')] [2023-12-27 00:16:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001220560_312516608.pth... [2023-12-27 00:16:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001221856_312836096.pth... [2023-12-27 00:16:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001219440_312229888.pth [2023-12-27 00:16:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001220704_312541184.pth [2023-12-27 00:16:31,303][105692] Updated weights for policy 0, policy_version 1220565 (0.0009) [2023-12-27 00:16:31,366][105692] Updated weights for policy 0, policy_version 1220575 (0.0009) [2023-12-27 00:16:31,402][105620] Updated weights for policy 1, policy_version 1221866 (0.0008) [2023-12-27 00:16:31,433][105692] Updated weights for policy 0, policy_version 1220585 (0.0008) [2023-12-27 00:16:31,465][105620] Updated weights for policy 1, policy_version 1221876 (0.0006) [2023-12-27 00:16:31,529][105620] Updated weights for policy 1, policy_version 1221886 (0.0005) [2023-12-27 00:16:32,072][105620] Updated weights for policy 1, policy_version 1221896 (0.0009) [2023-12-27 00:16:32,142][105620] Updated weights for policy 1, policy_version 1221906 (0.0009) [2023-12-27 00:16:32,204][105620] Updated weights for policy 1, policy_version 1221916 (0.0007) [2023-12-27 00:16:32,225][105692] Updated weights for policy 0, policy_version 1220595 (0.0008) [2023-12-27 00:16:32,288][105692] Updated weights for policy 0, policy_version 1220605 (0.0007) [2023-12-27 00:16:32,359][105692] Updated weights for policy 0, policy_version 1220615 (0.0009) [2023-12-27 00:16:32,799][105620] Updated weights for policy 1, policy_version 1221926 (0.0007) [2023-12-27 00:16:32,872][105620] Updated weights for policy 1, policy_version 1221936 (0.0006) [2023-12-27 00:16:32,929][105620] Updated weights for policy 1, policy_version 1221946 (0.0008) [2023-12-27 00:16:33,146][105692] Updated weights for policy 0, policy_version 1220625 (0.0010) [2023-12-27 00:16:33,199][105692] Updated weights for policy 0, policy_version 1220635 (0.0009) [2023-12-27 00:16:33,257][105692] Updated weights for policy 0, policy_version 1220645 (0.0010) [2023-12-27 00:16:33,330][105692] Updated weights for policy 0, policy_version 1220655 (0.0010) [2023-12-27 00:16:33,553][105620] Updated weights for policy 1, policy_version 1221956 (0.0009) [2023-12-27 00:16:33,609][105620] Updated weights for policy 1, policy_version 1221966 (0.0009) [2023-12-27 00:16:33,662][105620] Updated weights for policy 1, policy_version 1221976 (0.0009) [2023-12-27 00:16:33,911][105692] Updated weights for policy 0, policy_version 1220665 (0.0006) [2023-12-27 00:16:33,970][105692] Updated weights for policy 0, policy_version 1220675 (0.0010) [2023-12-27 00:16:34,021][105692] Updated weights for policy 0, policy_version 1220685 (0.0010) [2023-12-27 00:16:34,529][105620] Updated weights for policy 1, policy_version 1221986 (0.0011) [2023-12-27 00:16:34,593][105620] Updated weights for policy 1, policy_version 1221996 (0.0008) [2023-12-27 00:16:34,648][105620] Updated weights for policy 1, policy_version 1222006 (0.0007) [2023-12-27 00:16:34,649][105692] Updated weights for policy 0, policy_version 1220695 (0.0010) [2023-12-27 00:16:34,708][105692] Updated weights for policy 0, policy_version 1220705 (0.0011) [2023-12-27 00:16:34,710][105620] Updated weights for policy 1, policy_version 1222016 (0.0006) [2023-12-27 00:16:34,764][105692] Updated weights for policy 0, policy_version 1220715 (0.0008) [2023-12-27 00:16:35,421][105620] Updated weights for policy 1, policy_version 1222026 (0.0010) [2023-12-27 00:16:35,482][105620] Updated weights for policy 1, policy_version 1222037 (0.0010) [2023-12-27 00:16:35,500][105692] Updated weights for policy 0, policy_version 1220725 (0.0007) [2023-12-27 00:16:35,531][105620] Updated weights for policy 1, policy_version 1222047 (0.0008) [2023-12-27 00:16:35,549][105692] Updated weights for policy 0, policy_version 1220735 (0.0005) [2023-12-27 00:16:35,599][105692] Updated weights for policy 0, policy_version 1220745 (0.0007) [2023-12-27 00:16:36,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 625451008. Throughput: 0: 9553.2, 1: 9795.9. Samples: 625442056. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:16:36,062][104569] Avg episode reward: [(0, '7643.854'), (1, '9169.595')] [2023-12-27 00:16:36,230][105620] Updated weights for policy 1, policy_version 1222057 (0.0008) [2023-12-27 00:16:36,297][105620] Updated weights for policy 1, policy_version 1222067 (0.0009) [2023-12-27 00:16:36,362][105620] Updated weights for policy 1, policy_version 1222077 (0.0009) [2023-12-27 00:16:36,372][105692] Updated weights for policy 0, policy_version 1220756 (0.0008) [2023-12-27 00:16:36,428][105692] Updated weights for policy 0, policy_version 1220766 (0.0009) [2023-12-27 00:16:36,483][105692] Updated weights for policy 0, policy_version 1220776 (0.0010) [2023-12-27 00:16:37,068][105620] Updated weights for policy 1, policy_version 1222087 (0.0009) [2023-12-27 00:16:37,130][105620] Updated weights for policy 1, policy_version 1222097 (0.0009) [2023-12-27 00:16:37,181][105620] Updated weights for policy 1, policy_version 1222107 (0.0009) [2023-12-27 00:16:37,266][105692] Updated weights for policy 0, policy_version 1220786 (0.0009) [2023-12-27 00:16:37,317][105692] Updated weights for policy 0, policy_version 1220796 (0.0009) [2023-12-27 00:16:37,368][105692] Updated weights for policy 0, policy_version 1220806 (0.0006) [2023-12-27 00:16:37,420][105692] Updated weights for policy 0, policy_version 1220816 (0.0007) [2023-12-27 00:16:37,971][105620] Updated weights for policy 1, policy_version 1222117 (0.0007) [2023-12-27 00:16:38,037][105620] Updated weights for policy 1, policy_version 1222127 (0.0006) [2023-12-27 00:16:38,105][105620] Updated weights for policy 1, policy_version 1222137 (0.0005) [2023-12-27 00:16:38,111][105692] Updated weights for policy 0, policy_version 1220826 (0.0008) [2023-12-27 00:16:38,174][105692] Updated weights for policy 0, policy_version 1220836 (0.0009) [2023-12-27 00:16:38,218][105692] Updated weights for policy 0, policy_version 1220846 (0.0010) [2023-12-27 00:16:38,659][105620] Updated weights for policy 1, policy_version 1222147 (0.0006) [2023-12-27 00:16:38,724][105620] Updated weights for policy 1, policy_version 1222157 (0.0008) [2023-12-27 00:16:38,791][105620] Updated weights for policy 1, policy_version 1222167 (0.0008) [2023-12-27 00:16:38,867][105692] Updated weights for policy 0, policy_version 1220856 (0.0011) [2023-12-27 00:16:38,936][105692] Updated weights for policy 0, policy_version 1220866 (0.0010) [2023-12-27 00:16:38,986][105692] Updated weights for policy 0, policy_version 1220876 (0.0007) [2023-12-27 00:16:39,504][105620] Updated weights for policy 1, policy_version 1222177 (0.0008) [2023-12-27 00:16:39,575][105620] Updated weights for policy 1, policy_version 1222187 (0.0007) [2023-12-27 00:16:39,633][105620] Updated weights for policy 1, policy_version 1222197 (0.0005) [2023-12-27 00:16:39,638][105692] Updated weights for policy 0, policy_version 1220886 (0.0009) [2023-12-27 00:16:39,696][105620] Updated weights for policy 1, policy_version 1222207 (0.0006) [2023-12-27 00:16:39,701][105692] Updated weights for policy 0, policy_version 1220896 (0.0011) [2023-12-27 00:16:39,757][105692] Updated weights for policy 0, policy_version 1220906 (0.0011) [2023-12-27 00:16:40,371][105620] Updated weights for policy 1, policy_version 1222217 (0.0009) [2023-12-27 00:16:40,436][105620] Updated weights for policy 1, policy_version 1222227 (0.0007) [2023-12-27 00:16:40,503][105620] Updated weights for policy 1, policy_version 1222237 (0.0007) [2023-12-27 00:16:40,559][105692] Updated weights for policy 0, policy_version 1220916 (0.0010) [2023-12-27 00:16:40,615][105692] Updated weights for policy 0, policy_version 1220926 (0.0009) [2023-12-27 00:16:40,671][105692] Updated weights for policy 0, policy_version 1220936 (0.0009) [2023-12-27 00:16:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 625549312. Throughput: 0: 9612.2, 1: 9809.0. Samples: 625559224. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:16:41,062][104569] Avg episode reward: [(0, '8092.543'), (1, '9352.445')] [2023-12-27 00:16:41,148][105620] Updated weights for policy 1, policy_version 1222247 (0.0008) [2023-12-27 00:16:41,212][105620] Updated weights for policy 1, policy_version 1222257 (0.0009) [2023-12-27 00:16:41,271][105620] Updated weights for policy 1, policy_version 1222267 (0.0009) [2023-12-27 00:16:41,489][105692] Updated weights for policy 0, policy_version 1220946 (0.0008) [2023-12-27 00:16:41,555][105692] Updated weights for policy 0, policy_version 1220956 (0.0008) [2023-12-27 00:16:41,627][105692] Updated weights for policy 0, policy_version 1220966 (0.0010) [2023-12-27 00:16:41,682][105692] Updated weights for policy 0, policy_version 1220976 (0.0009) [2023-12-27 00:16:41,963][105620] Updated weights for policy 1, policy_version 1222277 (0.0009) [2023-12-27 00:16:42,025][105620] Updated weights for policy 1, policy_version 1222287 (0.0009) [2023-12-27 00:16:42,076][105620] Updated weights for policy 1, policy_version 1222297 (0.0009) [2023-12-27 00:16:42,411][105692] Updated weights for policy 0, policy_version 1220986 (0.0008) [2023-12-27 00:16:42,472][105692] Updated weights for policy 0, policy_version 1220996 (0.0009) [2023-12-27 00:16:42,526][105692] Updated weights for policy 0, policy_version 1221007 (0.0010) [2023-12-27 00:16:42,872][105620] Updated weights for policy 1, policy_version 1222307 (0.0009) [2023-12-27 00:16:42,929][105620] Updated weights for policy 1, policy_version 1222318 (0.0009) [2023-12-27 00:16:42,984][105620] Updated weights for policy 1, policy_version 1222328 (0.0009) [2023-12-27 00:16:43,187][105692] Updated weights for policy 0, policy_version 1221017 (0.0009) [2023-12-27 00:16:43,252][105692] Updated weights for policy 0, policy_version 1221027 (0.0009) [2023-12-27 00:16:43,314][105692] Updated weights for policy 0, policy_version 1221037 (0.0009) [2023-12-27 00:16:43,774][105620] Updated weights for policy 1, policy_version 1222338 (0.0009) [2023-12-27 00:16:43,832][105620] Updated weights for policy 1, policy_version 1222349 (0.0008) [2023-12-27 00:16:43,896][105620] Updated weights for policy 1, policy_version 1222359 (0.0009) [2023-12-27 00:16:43,916][105692] Updated weights for policy 0, policy_version 1221047 (0.0006) [2023-12-27 00:16:43,974][105692] Updated weights for policy 0, policy_version 1221057 (0.0007) [2023-12-27 00:16:44,036][105692] Updated weights for policy 0, policy_version 1221067 (0.0008) [2023-12-27 00:16:44,543][105620] Updated weights for policy 1, policy_version 1222369 (0.0006) [2023-12-27 00:16:44,599][105620] Updated weights for policy 1, policy_version 1222379 (0.0010) [2023-12-27 00:16:44,651][105620] Updated weights for policy 1, policy_version 1222389 (0.0010) [2023-12-27 00:16:44,709][105620] Updated weights for policy 1, policy_version 1222399 (0.0010) [2023-12-27 00:16:44,727][105692] Updated weights for policy 0, policy_version 1221077 (0.0007) [2023-12-27 00:16:44,788][105692] Updated weights for policy 0, policy_version 1221087 (0.0008) [2023-12-27 00:16:44,848][105692] Updated weights for policy 0, policy_version 1221097 (0.0008) [2023-12-27 00:16:45,475][105620] Updated weights for policy 1, policy_version 1222409 (0.0010) [2023-12-27 00:16:45,528][105620] Updated weights for policy 1, policy_version 1222419 (0.0007) [2023-12-27 00:16:45,585][105620] Updated weights for policy 1, policy_version 1222429 (0.0009) [2023-12-27 00:16:45,623][105692] Updated weights for policy 0, policy_version 1221107 (0.0009) [2023-12-27 00:16:45,673][105692] Updated weights for policy 0, policy_version 1221117 (0.0009) [2023-12-27 00:16:45,722][105692] Updated weights for policy 0, policy_version 1221127 (0.0008) [2023-12-27 00:16:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 625647616. Throughput: 0: 9679.9, 1: 9754.8. Samples: 625615148. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:16:46,063][104569] Avg episode reward: [(0, '7634.319'), (1, '9353.182')] [2023-12-27 00:16:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001221136_312664064.pth... [2023-12-27 00:16:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001222432_312983552.pth... [2023-12-27 00:16:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001220016_312377344.pth [2023-12-27 00:16:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001221280_312688640.pth [2023-12-27 00:16:46,174][105620] Updated weights for policy 1, policy_version 1222439 (0.0006) [2023-12-27 00:16:46,220][105620] Updated weights for policy 1, policy_version 1222449 (0.0005) [2023-12-27 00:16:46,272][105620] Updated weights for policy 1, policy_version 1222459 (0.0006) [2023-12-27 00:16:46,496][105692] Updated weights for policy 0, policy_version 1221137 (0.0008) [2023-12-27 00:16:46,558][105692] Updated weights for policy 0, policy_version 1221147 (0.0005) [2023-12-27 00:16:46,610][105692] Updated weights for policy 0, policy_version 1221157 (0.0005) [2023-12-27 00:16:46,662][105692] Updated weights for policy 0, policy_version 1221167 (0.0005) [2023-12-27 00:16:46,896][105620] Updated weights for policy 1, policy_version 1222469 (0.0007) [2023-12-27 00:16:46,943][105620] Updated weights for policy 1, policy_version 1222479 (0.0011) [2023-12-27 00:16:46,998][105620] Updated weights for policy 1, policy_version 1222489 (0.0010) [2023-12-27 00:16:47,238][105692] Updated weights for policy 0, policy_version 1221177 (0.0010) [2023-12-27 00:16:47,293][105692] Updated weights for policy 0, policy_version 1221187 (0.0010) [2023-12-27 00:16:47,351][105692] Updated weights for policy 0, policy_version 1221197 (0.0010) [2023-12-27 00:16:47,665][105620] Updated weights for policy 1, policy_version 1222499 (0.0010) [2023-12-27 00:16:47,713][105620] Updated weights for policy 1, policy_version 1222509 (0.0010) [2023-12-27 00:16:47,762][105620] Updated weights for policy 1, policy_version 1222519 (0.0011) [2023-12-27 00:16:47,992][105692] Updated weights for policy 0, policy_version 1221207 (0.0010) [2023-12-27 00:16:48,045][105692] Updated weights for policy 0, policy_version 1221217 (0.0011) [2023-12-27 00:16:48,096][105692] Updated weights for policy 0, policy_version 1221227 (0.0009) [2023-12-27 00:16:48,518][105620] Updated weights for policy 1, policy_version 1222529 (0.0010) [2023-12-27 00:16:48,578][105620] Updated weights for policy 1, policy_version 1222539 (0.0011) [2023-12-27 00:16:48,645][105620] Updated weights for policy 1, policy_version 1222549 (0.0011) [2023-12-27 00:16:48,697][105620] Updated weights for policy 1, policy_version 1222559 (0.0011) [2023-12-27 00:16:48,705][105692] Updated weights for policy 0, policy_version 1221237 (0.0008) [2023-12-27 00:16:48,764][105692] Updated weights for policy 0, policy_version 1221247 (0.0010) [2023-12-27 00:16:48,826][105692] Updated weights for policy 0, policy_version 1221257 (0.0011) [2023-12-27 00:16:49,442][105620] Updated weights for policy 1, policy_version 1222569 (0.0008) [2023-12-27 00:16:49,491][105620] Updated weights for policy 1, policy_version 1222579 (0.0008) [2023-12-27 00:16:49,539][105620] Updated weights for policy 1, policy_version 1222589 (0.0008) [2023-12-27 00:16:49,578][105692] Updated weights for policy 0, policy_version 1221267 (0.0010) [2023-12-27 00:16:49,642][105692] Updated weights for policy 0, policy_version 1221277 (0.0010) [2023-12-27 00:16:49,708][105692] Updated weights for policy 0, policy_version 1221287 (0.0011) [2023-12-27 00:16:50,395][105620] Updated weights for policy 1, policy_version 1222599 (0.0008) [2023-12-27 00:16:50,447][105692] Updated weights for policy 0, policy_version 1221297 (0.0007) [2023-12-27 00:16:50,453][105620] Updated weights for policy 1, policy_version 1222609 (0.0009) [2023-12-27 00:16:50,499][105620] Updated weights for policy 1, policy_version 1222619 (0.0008) [2023-12-27 00:16:50,506][105692] Updated weights for policy 0, policy_version 1221307 (0.0008) [2023-12-27 00:16:50,559][105692] Updated weights for policy 0, policy_version 1221317 (0.0011) [2023-12-27 00:16:50,624][105692] Updated weights for policy 0, policy_version 1221327 (0.0009) [2023-12-27 00:16:51,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 625745920. Throughput: 0: 9754.0, 1: 9876.7. Samples: 625737688. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:16:51,062][104569] Avg episode reward: [(0, '7822.515'), (1, '9260.211')] [2023-12-27 00:16:51,147][105620] Updated weights for policy 1, policy_version 1222629 (0.0007) [2023-12-27 00:16:51,214][105620] Updated weights for policy 1, policy_version 1222639 (0.0006) [2023-12-27 00:16:51,277][105620] Updated weights for policy 1, policy_version 1222649 (0.0008) [2023-12-27 00:16:51,399][105692] Updated weights for policy 0, policy_version 1221337 (0.0009) [2023-12-27 00:16:51,457][105692] Updated weights for policy 0, policy_version 1221347 (0.0010) [2023-12-27 00:16:51,508][105692] Updated weights for policy 0, policy_version 1221358 (0.0010) [2023-12-27 00:16:51,954][105620] Updated weights for policy 1, policy_version 1222659 (0.0009) [2023-12-27 00:16:52,016][105620] Updated weights for policy 1, policy_version 1222669 (0.0009) [2023-12-27 00:16:52,082][105620] Updated weights for policy 1, policy_version 1222679 (0.0010) [2023-12-27 00:16:52,283][105692] Updated weights for policy 0, policy_version 1221368 (0.0009) [2023-12-27 00:16:52,342][105692] Updated weights for policy 0, policy_version 1221378 (0.0009) [2023-12-27 00:16:52,404][105692] Updated weights for policy 0, policy_version 1221388 (0.0007) [2023-12-27 00:16:52,830][105620] Updated weights for policy 1, policy_version 1222689 (0.0010) [2023-12-27 00:16:52,889][105620] Updated weights for policy 1, policy_version 1222700 (0.0009) [2023-12-27 00:16:52,947][105620] Updated weights for policy 1, policy_version 1222710 (0.0009) [2023-12-27 00:16:53,001][105620] Updated weights for policy 1, policy_version 1222720 (0.0009) [2023-12-27 00:16:53,101][105692] Updated weights for policy 0, policy_version 1221398 (0.0008) [2023-12-27 00:16:53,152][105692] Updated weights for policy 0, policy_version 1221408 (0.0009) [2023-12-27 00:16:53,210][105692] Updated weights for policy 0, policy_version 1221418 (0.0009) [2023-12-27 00:16:53,759][105620] Updated weights for policy 1, policy_version 1222730 (0.0009) [2023-12-27 00:16:53,811][105620] Updated weights for policy 1, policy_version 1222741 (0.0009) [2023-12-27 00:16:53,843][105692] Updated weights for policy 0, policy_version 1221428 (0.0007) [2023-12-27 00:16:53,868][105620] Updated weights for policy 1, policy_version 1222751 (0.0009) [2023-12-27 00:16:53,902][105692] Updated weights for policy 0, policy_version 1221438 (0.0008) [2023-12-27 00:16:53,964][105692] Updated weights for policy 0, policy_version 1221448 (0.0009) [2023-12-27 00:16:54,578][105620] Updated weights for policy 1, policy_version 1222761 (0.0011) [2023-12-27 00:16:54,640][105620] Updated weights for policy 1, policy_version 1222771 (0.0010) [2023-12-27 00:16:54,704][105620] Updated weights for policy 1, policy_version 1222781 (0.0010) [2023-12-27 00:16:54,774][105692] Updated weights for policy 0, policy_version 1221458 (0.0008) [2023-12-27 00:16:54,835][105692] Updated weights for policy 0, policy_version 1221468 (0.0005) [2023-12-27 00:16:54,885][105692] Updated weights for policy 0, policy_version 1221478 (0.0005) [2023-12-27 00:16:54,947][105692] Updated weights for policy 0, policy_version 1221488 (0.0005) [2023-12-27 00:16:55,331][105620] Updated weights for policy 1, policy_version 1222791 (0.0010) [2023-12-27 00:16:55,385][105620] Updated weights for policy 1, policy_version 1222801 (0.0010) [2023-12-27 00:16:55,436][105620] Updated weights for policy 1, policy_version 1222811 (0.0010) [2023-12-27 00:16:55,625][105692] Updated weights for policy 0, policy_version 1221498 (0.0010) [2023-12-27 00:16:55,673][105692] Updated weights for policy 0, policy_version 1221508 (0.0010) [2023-12-27 00:16:55,721][105692] Updated weights for policy 0, policy_version 1221518 (0.0010) [2023-12-27 00:16:56,018][105620] Updated weights for policy 1, policy_version 1222821 (0.0009) [2023-12-27 00:16:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.4, 300 sec: 19466.4). Total num frames: 625844224. Throughput: 0: 9697.6, 1: 9895.6. Samples: 625855432. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:16:56,062][104569] Avg episode reward: [(0, '8177.689'), (1, '9168.394')] [2023-12-27 00:16:56,065][105620] Updated weights for policy 1, policy_version 1222831 (0.0005) [2023-12-27 00:16:56,113][105620] Updated weights for policy 1, policy_version 1222841 (0.0005) [2023-12-27 00:16:56,535][105692] Updated weights for policy 0, policy_version 1221528 (0.0010) [2023-12-27 00:16:56,589][105692] Updated weights for policy 0, policy_version 1221540 (0.0010) [2023-12-27 00:16:56,639][105692] Updated weights for policy 0, policy_version 1221550 (0.0009) [2023-12-27 00:16:56,646][105620] Updated weights for policy 1, policy_version 1222851 (0.0005) [2023-12-27 00:16:56,697][105620] Updated weights for policy 1, policy_version 1222861 (0.0005) [2023-12-27 00:16:56,747][105620] Updated weights for policy 1, policy_version 1222871 (0.0009) [2023-12-27 00:16:57,335][105692] Updated weights for policy 0, policy_version 1221560 (0.0007) [2023-12-27 00:16:57,387][105692] Updated weights for policy 0, policy_version 1221570 (0.0005) [2023-12-27 00:16:57,435][105692] Updated weights for policy 0, policy_version 1221580 (0.0006) [2023-12-27 00:16:57,471][105620] Updated weights for policy 1, policy_version 1222881 (0.0010) [2023-12-27 00:16:57,532][105620] Updated weights for policy 1, policy_version 1222891 (0.0010) [2023-12-27 00:16:57,579][105620] Updated weights for policy 1, policy_version 1222901 (0.0010) [2023-12-27 00:16:57,626][105620] Updated weights for policy 1, policy_version 1222911 (0.0010) [2023-12-27 00:16:58,018][105692] Updated weights for policy 0, policy_version 1221590 (0.0007) [2023-12-27 00:16:58,072][105692] Updated weights for policy 0, policy_version 1221600 (0.0008) [2023-12-27 00:16:58,127][105692] Updated weights for policy 0, policy_version 1221610 (0.0007) [2023-12-27 00:16:58,392][105620] Updated weights for policy 1, policy_version 1222921 (0.0008) [2023-12-27 00:16:58,455][105620] Updated weights for policy 1, policy_version 1222931 (0.0010) [2023-12-27 00:16:58,522][105620] Updated weights for policy 1, policy_version 1222941 (0.0010) [2023-12-27 00:16:58,914][105692] Updated weights for policy 0, policy_version 1221620 (0.0009) [2023-12-27 00:16:58,977][105692] Updated weights for policy 0, policy_version 1221630 (0.0010) [2023-12-27 00:16:59,043][105692] Updated weights for policy 0, policy_version 1221640 (0.0009) [2023-12-27 00:16:59,413][105620] Updated weights for policy 1, policy_version 1222951 (0.0010) [2023-12-27 00:16:59,471][105620] Updated weights for policy 1, policy_version 1222961 (0.0010) [2023-12-27 00:16:59,518][105620] Updated weights for policy 1, policy_version 1222971 (0.0010) [2023-12-27 00:16:59,745][105692] Updated weights for policy 0, policy_version 1221650 (0.0010) [2023-12-27 00:16:59,801][105692] Updated weights for policy 0, policy_version 1221660 (0.0010) [2023-12-27 00:16:59,863][105692] Updated weights for policy 0, policy_version 1221670 (0.0011) [2023-12-27 00:16:59,922][105692] Updated weights for policy 0, policy_version 1221680 (0.0010) [2023-12-27 00:17:00,234][105620] Updated weights for policy 1, policy_version 1222981 (0.0011) [2023-12-27 00:17:00,289][105620] Updated weights for policy 1, policy_version 1222991 (0.0010) [2023-12-27 00:17:00,340][105620] Updated weights for policy 1, policy_version 1223001 (0.0010) [2023-12-27 00:17:00,572][105692] Updated weights for policy 0, policy_version 1221690 (0.0006) [2023-12-27 00:17:00,628][105692] Updated weights for policy 0, policy_version 1221700 (0.0005) [2023-12-27 00:17:00,685][105692] Updated weights for policy 0, policy_version 1221710 (0.0005) [2023-12-27 00:17:00,992][105620] Updated weights for policy 1, policy_version 1223011 (0.0009) [2023-12-27 00:17:01,052][105620] Updated weights for policy 1, policy_version 1223021 (0.0008) [2023-12-27 00:17:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 625942528. Throughput: 0: 9719.5, 1: 9918.7. Samples: 625915272. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:17:01,062][104569] Avg episode reward: [(0, '7998.552'), (1, '9168.213')] [2023-12-27 00:17:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001221712_312811520.pth... [2023-12-27 00:17:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001220560_312516608.pth [2023-12-27 00:17:01,113][105620] Updated weights for policy 1, policy_version 1223031 (0.0008) [2023-12-27 00:17:01,166][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001223040_313139200.pth... [2023-12-27 00:17:01,171][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001221856_312836096.pth [2023-12-27 00:17:01,323][105692] Updated weights for policy 0, policy_version 1221720 (0.0009) [2023-12-27 00:17:01,387][105692] Updated weights for policy 0, policy_version 1221730 (0.0007) [2023-12-27 00:17:01,446][105692] Updated weights for policy 0, policy_version 1221740 (0.0006) [2023-12-27 00:17:01,909][105620] Updated weights for policy 1, policy_version 1223041 (0.0009) [2023-12-27 00:17:01,976][105620] Updated weights for policy 1, policy_version 1223051 (0.0010) [2023-12-27 00:17:02,036][105620] Updated weights for policy 1, policy_version 1223061 (0.0011) [2023-12-27 00:17:02,039][105692] Updated weights for policy 0, policy_version 1221750 (0.0005) [2023-12-27 00:17:02,083][105620] Updated weights for policy 1, policy_version 1223071 (0.0010) [2023-12-27 00:17:02,095][105692] Updated weights for policy 0, policy_version 1221760 (0.0005) [2023-12-27 00:17:02,161][105692] Updated weights for policy 0, policy_version 1221770 (0.0005) [2023-12-27 00:17:02,725][105692] Updated weights for policy 0, policy_version 1221780 (0.0006) [2023-12-27 00:17:02,774][105692] Updated weights for policy 0, policy_version 1221790 (0.0008) [2023-12-27 00:17:02,819][105620] Updated weights for policy 1, policy_version 1223081 (0.0010) [2023-12-27 00:17:02,830][105692] Updated weights for policy 0, policy_version 1221800 (0.0007) [2023-12-27 00:17:02,875][105620] Updated weights for policy 1, policy_version 1223091 (0.0010) [2023-12-27 00:17:02,944][105620] Updated weights for policy 1, policy_version 1223101 (0.0005) [2023-12-27 00:17:03,545][105692] Updated weights for policy 0, policy_version 1221810 (0.0007) [2023-12-27 00:17:03,594][105620] Updated weights for policy 1, policy_version 1223111 (0.0006) [2023-12-27 00:17:03,612][105692] Updated weights for policy 0, policy_version 1221820 (0.0009) [2023-12-27 00:17:03,651][105620] Updated weights for policy 1, policy_version 1223121 (0.0005) [2023-12-27 00:17:03,660][105692] Updated weights for policy 0, policy_version 1221830 (0.0006) [2023-12-27 00:17:03,713][105620] Updated weights for policy 1, policy_version 1223131 (0.0008) [2023-12-27 00:17:03,720][105692] Updated weights for policy 0, policy_version 1221840 (0.0008) [2023-12-27 00:17:04,334][105620] Updated weights for policy 1, policy_version 1223141 (0.0010) [2023-12-27 00:17:04,401][105620] Updated weights for policy 1, policy_version 1223151 (0.0011) [2023-12-27 00:17:04,462][105620] Updated weights for policy 1, policy_version 1223161 (0.0010) [2023-12-27 00:17:04,493][105692] Updated weights for policy 0, policy_version 1221850 (0.0007) [2023-12-27 00:17:04,545][105692] Updated weights for policy 0, policy_version 1221860 (0.0008) [2023-12-27 00:17:04,594][105692] Updated weights for policy 0, policy_version 1221871 (0.0008) [2023-12-27 00:17:05,177][105620] Updated weights for policy 1, policy_version 1223171 (0.0010) [2023-12-27 00:17:05,225][105620] Updated weights for policy 1, policy_version 1223181 (0.0010) [2023-12-27 00:17:05,229][105692] Updated weights for policy 0, policy_version 1221881 (0.0005) [2023-12-27 00:17:05,275][105692] Updated weights for policy 0, policy_version 1221891 (0.0005) [2023-12-27 00:17:05,276][105620] Updated weights for policy 1, policy_version 1223191 (0.0010) [2023-12-27 00:17:05,325][105692] Updated weights for policy 0, policy_version 1221901 (0.0005) [2023-12-27 00:17:05,860][105692] Updated weights for policy 0, policy_version 1221911 (0.0005) [2023-12-27 00:17:05,922][105692] Updated weights for policy 0, policy_version 1221921 (0.0005) [2023-12-27 00:17:05,971][105692] Updated weights for policy 0, policy_version 1221931 (0.0007) [2023-12-27 00:17:06,037][105620] Updated weights for policy 1, policy_version 1223201 (0.0010) [2023-12-27 00:17:06,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 626049024. Throughput: 0: 9711.7, 1: 9937.0. Samples: 626035416. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:17:06,063][104569] Avg episode reward: [(0, '7541.897'), (1, '9076.125')] [2023-12-27 00:17:06,088][105620] Updated weights for policy 1, policy_version 1223211 (0.0010) [2023-12-27 00:17:06,151][105620] Updated weights for policy 1, policy_version 1223221 (0.0010) [2023-12-27 00:17:06,207][105620] Updated weights for policy 1, policy_version 1223231 (0.0011) [2023-12-27 00:17:06,606][105692] Updated weights for policy 0, policy_version 1221941 (0.0006) [2023-12-27 00:17:06,673][105692] Updated weights for policy 0, policy_version 1221951 (0.0006) [2023-12-27 00:17:06,745][105692] Updated weights for policy 0, policy_version 1221961 (0.0006) [2023-12-27 00:17:06,999][105620] Updated weights for policy 1, policy_version 1223241 (0.0009) [2023-12-27 00:17:07,057][105620] Updated weights for policy 1, policy_version 1223251 (0.0008) [2023-12-27 00:17:07,107][105620] Updated weights for policy 1, policy_version 1223261 (0.0006) [2023-12-27 00:17:07,401][105692] Updated weights for policy 0, policy_version 1221971 (0.0006) [2023-12-27 00:17:07,463][105692] Updated weights for policy 0, policy_version 1221981 (0.0009) [2023-12-27 00:17:07,512][105692] Updated weights for policy 0, policy_version 1221991 (0.0009) [2023-12-27 00:17:07,826][105620] Updated weights for policy 1, policy_version 1223271 (0.0006) [2023-12-27 00:17:07,882][105620] Updated weights for policy 1, policy_version 1223281 (0.0005) [2023-12-27 00:17:07,934][105620] Updated weights for policy 1, policy_version 1223291 (0.0005) [2023-12-27 00:17:08,106][105692] Updated weights for policy 0, policy_version 1222001 (0.0010) [2023-12-27 00:17:08,168][105692] Updated weights for policy 0, policy_version 1222011 (0.0010) [2023-12-27 00:17:08,228][105692] Updated weights for policy 0, policy_version 1222021 (0.0007) [2023-12-27 00:17:08,284][105692] Updated weights for policy 0, policy_version 1222031 (0.0005) [2023-12-27 00:17:08,617][105620] Updated weights for policy 1, policy_version 1223301 (0.0010) [2023-12-27 00:17:08,672][105620] Updated weights for policy 1, policy_version 1223311 (0.0010) [2023-12-27 00:17:08,731][105620] Updated weights for policy 1, policy_version 1223321 (0.0010) [2023-12-27 00:17:08,913][105692] Updated weights for policy 0, policy_version 1222041 (0.0005) [2023-12-27 00:17:08,975][105692] Updated weights for policy 0, policy_version 1222051 (0.0007) [2023-12-27 00:17:09,033][105692] Updated weights for policy 0, policy_version 1222061 (0.0011) [2023-12-27 00:17:09,445][105620] Updated weights for policy 1, policy_version 1223331 (0.0009) [2023-12-27 00:17:09,496][105620] Updated weights for policy 1, policy_version 1223341 (0.0006) [2023-12-27 00:17:09,565][105620] Updated weights for policy 1, policy_version 1223351 (0.0006) [2023-12-27 00:17:09,699][105692] Updated weights for policy 0, policy_version 1222071 (0.0009) [2023-12-27 00:17:09,766][105692] Updated weights for policy 0, policy_version 1222081 (0.0011) [2023-12-27 00:17:09,835][105692] Updated weights for policy 0, policy_version 1222091 (0.0011) [2023-12-27 00:17:10,264][105620] Updated weights for policy 1, policy_version 1223361 (0.0009) [2023-12-27 00:17:10,328][105620] Updated weights for policy 1, policy_version 1223372 (0.0009) [2023-12-27 00:17:10,382][105620] Updated weights for policy 1, policy_version 1223382 (0.0008) [2023-12-27 00:17:10,442][105620] Updated weights for policy 1, policy_version 1223392 (0.0006) [2023-12-27 00:17:10,528][105692] Updated weights for policy 0, policy_version 1222101 (0.0011) [2023-12-27 00:17:10,582][105692] Updated weights for policy 0, policy_version 1222111 (0.0010) [2023-12-27 00:17:10,638][105692] Updated weights for policy 0, policy_version 1222121 (0.0011) [2023-12-27 00:17:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 626147328. Throughput: 0: 9953.5, 1: 10015.0. Samples: 626159580. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:17:11,063][104569] Avg episode reward: [(0, '7632.534'), (1, '9167.371')] [2023-12-27 00:17:11,144][105620] Updated weights for policy 1, policy_version 1223402 (0.0008) [2023-12-27 00:17:11,210][105620] Updated weights for policy 1, policy_version 1223412 (0.0009) [2023-12-27 00:17:11,278][105620] Updated weights for policy 1, policy_version 1223422 (0.0009) [2023-12-27 00:17:11,476][105692] Updated weights for policy 0, policy_version 1222131 (0.0011) [2023-12-27 00:17:11,548][105692] Updated weights for policy 0, policy_version 1222141 (0.0011) [2023-12-27 00:17:11,614][105692] Updated weights for policy 0, policy_version 1222151 (0.0010) [2023-12-27 00:17:12,046][105620] Updated weights for policy 1, policy_version 1223432 (0.0007) [2023-12-27 00:17:12,108][105620] Updated weights for policy 1, policy_version 1223442 (0.0008) [2023-12-27 00:17:12,161][105620] Updated weights for policy 1, policy_version 1223452 (0.0008) [2023-12-27 00:17:12,379][105692] Updated weights for policy 0, policy_version 1222161 (0.0010) [2023-12-27 00:17:12,441][105692] Updated weights for policy 0, policy_version 1222171 (0.0011) [2023-12-27 00:17:12,502][105692] Updated weights for policy 0, policy_version 1222181 (0.0010) [2023-12-27 00:17:12,564][105692] Updated weights for policy 0, policy_version 1222191 (0.0010) [2023-12-27 00:17:12,906][105620] Updated weights for policy 1, policy_version 1223462 (0.0008) [2023-12-27 00:17:12,962][105620] Updated weights for policy 1, policy_version 1223472 (0.0008) [2023-12-27 00:17:13,024][105620] Updated weights for policy 1, policy_version 1223482 (0.0008) [2023-12-27 00:17:13,307][105692] Updated weights for policy 0, policy_version 1222201 (0.0011) [2023-12-27 00:17:13,369][105692] Updated weights for policy 0, policy_version 1222211 (0.0011) [2023-12-27 00:17:13,427][105692] Updated weights for policy 0, policy_version 1222221 (0.0010) [2023-12-27 00:17:13,751][105620] Updated weights for policy 1, policy_version 1223492 (0.0009) [2023-12-27 00:17:13,803][105620] Updated weights for policy 1, policy_version 1223502 (0.0007) [2023-12-27 00:17:13,857][105620] Updated weights for policy 1, policy_version 1223512 (0.0005) [2023-12-27 00:17:14,074][105692] Updated weights for policy 0, policy_version 1222231 (0.0011) [2023-12-27 00:17:14,141][105692] Updated weights for policy 0, policy_version 1222241 (0.0011) [2023-12-27 00:17:14,197][105692] Updated weights for policy 0, policy_version 1222251 (0.0011) [2023-12-27 00:17:14,536][105620] Updated weights for policy 1, policy_version 1223522 (0.0006) [2023-12-27 00:17:14,596][105620] Updated weights for policy 1, policy_version 1223532 (0.0010) [2023-12-27 00:17:14,643][105620] Updated weights for policy 1, policy_version 1223542 (0.0010) [2023-12-27 00:17:14,714][105620] Updated weights for policy 1, policy_version 1223552 (0.0010) [2023-12-27 00:17:14,739][105692] Updated weights for policy 0, policy_version 1222261 (0.0007) [2023-12-27 00:17:14,805][105692] Updated weights for policy 0, policy_version 1222271 (0.0008) [2023-12-27 00:17:14,874][105692] Updated weights for policy 0, policy_version 1222281 (0.0009) [2023-12-27 00:17:15,380][105620] Updated weights for policy 1, policy_version 1223562 (0.0008) [2023-12-27 00:17:15,439][105620] Updated weights for policy 1, policy_version 1223572 (0.0007) [2023-12-27 00:17:15,486][105620] Updated weights for policy 1, policy_version 1223582 (0.0008) [2023-12-27 00:17:15,637][105692] Updated weights for policy 0, policy_version 1222291 (0.0010) [2023-12-27 00:17:15,695][105692] Updated weights for policy 0, policy_version 1222301 (0.0010) [2023-12-27 00:17:15,757][105692] Updated weights for policy 0, policy_version 1222311 (0.0010) [2023-12-27 00:17:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 626245632. Throughput: 0: 9919.4, 1: 9927.4. Samples: 626214568. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:17:16,063][104569] Avg episode reward: [(0, '7902.963'), (1, '9181.884')] [2023-12-27 00:17:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001222320_312967168.pth... [2023-12-27 00:17:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001223584_313278464.pth... [2023-12-27 00:17:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001221136_312664064.pth [2023-12-27 00:17:16,092][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001222432_312983552.pth [2023-12-27 00:17:16,156][105620] Updated weights for policy 1, policy_version 1223592 (0.0008) [2023-12-27 00:17:16,223][105620] Updated weights for policy 1, policy_version 1223602 (0.0008) [2023-12-27 00:17:16,285][105620] Updated weights for policy 1, policy_version 1223612 (0.0008) [2023-12-27 00:17:16,496][105692] Updated weights for policy 0, policy_version 1222321 (0.0010) [2023-12-27 00:17:16,546][105692] Updated weights for policy 0, policy_version 1222331 (0.0010) [2023-12-27 00:17:16,609][105692] Updated weights for policy 0, policy_version 1222341 (0.0010) [2023-12-27 00:17:16,661][105692] Updated weights for policy 0, policy_version 1222351 (0.0010) [2023-12-27 00:17:16,945][105620] Updated weights for policy 1, policy_version 1223622 (0.0008) [2023-12-27 00:17:16,996][105620] Updated weights for policy 1, policy_version 1223632 (0.0008) [2023-12-27 00:17:17,050][105620] Updated weights for policy 1, policy_version 1223642 (0.0008) [2023-12-27 00:17:17,400][105692] Updated weights for policy 0, policy_version 1222361 (0.0010) [2023-12-27 00:17:17,451][105692] Updated weights for policy 0, policy_version 1222371 (0.0010) [2023-12-27 00:17:17,505][105692] Updated weights for policy 0, policy_version 1222381 (0.0010) [2023-12-27 00:17:17,748][105620] Updated weights for policy 1, policy_version 1223652 (0.0009) [2023-12-27 00:17:17,810][105620] Updated weights for policy 1, policy_version 1223662 (0.0008) [2023-12-27 00:17:17,865][105620] Updated weights for policy 1, policy_version 1223672 (0.0005) [2023-12-27 00:17:18,145][105692] Updated weights for policy 0, policy_version 1222391 (0.0007) [2023-12-27 00:17:18,201][105692] Updated weights for policy 0, policy_version 1222401 (0.0005) [2023-12-27 00:17:18,255][105692] Updated weights for policy 0, policy_version 1222411 (0.0005) [2023-12-27 00:17:18,462][105620] Updated weights for policy 1, policy_version 1223682 (0.0005) [2023-12-27 00:17:18,528][105620] Updated weights for policy 1, policy_version 1223692 (0.0006) [2023-12-27 00:17:18,597][105620] Updated weights for policy 1, policy_version 1223702 (0.0006) [2023-12-27 00:17:18,664][105620] Updated weights for policy 1, policy_version 1223712 (0.0006) [2023-12-27 00:17:18,863][105692] Updated weights for policy 0, policy_version 1222421 (0.0005) [2023-12-27 00:17:18,907][105692] Updated weights for policy 0, policy_version 1222431 (0.0007) [2023-12-27 00:17:18,960][105692] Updated weights for policy 0, policy_version 1222441 (0.0011) [2023-12-27 00:17:19,286][105620] Updated weights for policy 1, policy_version 1223722 (0.0007) [2023-12-27 00:17:19,352][105620] Updated weights for policy 1, policy_version 1223732 (0.0007) [2023-12-27 00:17:19,415][105620] Updated weights for policy 1, policy_version 1223742 (0.0008) [2023-12-27 00:17:19,669][105692] Updated weights for policy 0, policy_version 1222451 (0.0010) [2023-12-27 00:17:19,722][105692] Updated weights for policy 0, policy_version 1222461 (0.0010) [2023-12-27 00:17:19,783][105692] Updated weights for policy 0, policy_version 1222471 (0.0009) [2023-12-27 00:17:20,077][105620] Updated weights for policy 1, policy_version 1223752 (0.0010) [2023-12-27 00:17:20,144][105620] Updated weights for policy 1, policy_version 1223762 (0.0008) [2023-12-27 00:17:20,208][105620] Updated weights for policy 1, policy_version 1223772 (0.0009) [2023-12-27 00:17:20,478][105692] Updated weights for policy 0, policy_version 1222481 (0.0007) [2023-12-27 00:17:20,546][105692] Updated weights for policy 0, policy_version 1222491 (0.0009) [2023-12-27 00:17:20,612][105692] Updated weights for policy 0, policy_version 1222501 (0.0007) [2023-12-27 00:17:20,665][105692] Updated weights for policy 0, policy_version 1222511 (0.0008) [2023-12-27 00:17:21,031][105620] Updated weights for policy 1, policy_version 1223782 (0.0009) [2023-12-27 00:17:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 626343936. Throughput: 0: 10009.0, 1: 9924.3. Samples: 626339052. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:17:21,062][104569] Avg episode reward: [(0, '8447.572'), (1, '8515.509')] [2023-12-27 00:17:21,098][105620] Updated weights for policy 1, policy_version 1223792 (0.0009) [2023-12-27 00:17:21,163][105620] Updated weights for policy 1, policy_version 1223802 (0.0010) [2023-12-27 00:17:21,372][105692] Updated weights for policy 0, policy_version 1222521 (0.0010) [2023-12-27 00:17:21,440][105692] Updated weights for policy 0, policy_version 1222531 (0.0008) [2023-12-27 00:17:21,508][105692] Updated weights for policy 0, policy_version 1222541 (0.0006) [2023-12-27 00:17:22,014][105620] Updated weights for policy 1, policy_version 1223812 (0.0009) [2023-12-27 00:17:22,085][105620] Updated weights for policy 1, policy_version 1223822 (0.0009) [2023-12-27 00:17:22,157][105620] Updated weights for policy 1, policy_version 1223832 (0.0008) [2023-12-27 00:17:22,158][105692] Updated weights for policy 0, policy_version 1222551 (0.0007) [2023-12-27 00:17:22,216][105692] Updated weights for policy 0, policy_version 1222561 (0.0009) [2023-12-27 00:17:22,285][105692] Updated weights for policy 0, policy_version 1222571 (0.0009) [2023-12-27 00:17:22,944][105620] Updated weights for policy 1, policy_version 1223842 (0.0007) [2023-12-27 00:17:23,011][105620] Updated weights for policy 1, policy_version 1223852 (0.0008) [2023-12-27 00:17:23,064][105620] Updated weights for policy 1, policy_version 1223862 (0.0008) [2023-12-27 00:17:23,088][105692] Updated weights for policy 0, policy_version 1222581 (0.0007) [2023-12-27 00:17:23,115][105620] Updated weights for policy 1, policy_version 1223872 (0.0008) [2023-12-27 00:17:23,141][105692] Updated weights for policy 0, policy_version 1222591 (0.0010) [2023-12-27 00:17:23,189][105692] Updated weights for policy 0, policy_version 1222601 (0.0010) [2023-12-27 00:17:23,863][105620] Updated weights for policy 1, policy_version 1223882 (0.0005) [2023-12-27 00:17:23,914][105620] Updated weights for policy 1, policy_version 1223892 (0.0007) [2023-12-27 00:17:23,946][105692] Updated weights for policy 0, policy_version 1222611 (0.0010) [2023-12-27 00:17:23,975][105620] Updated weights for policy 1, policy_version 1223902 (0.0008) [2023-12-27 00:17:24,003][105692] Updated weights for policy 0, policy_version 1222621 (0.0010) [2023-12-27 00:17:24,054][105692] Updated weights for policy 0, policy_version 1222631 (0.0010) [2023-12-27 00:17:24,652][105620] Updated weights for policy 1, policy_version 1223912 (0.0006) [2023-12-27 00:17:24,702][105620] Updated weights for policy 1, policy_version 1223922 (0.0007) [2023-12-27 00:17:24,734][105692] Updated weights for policy 0, policy_version 1222641 (0.0007) [2023-12-27 00:17:24,769][105620] Updated weights for policy 1, policy_version 1223932 (0.0008) [2023-12-27 00:17:24,786][105692] Updated weights for policy 0, policy_version 1222651 (0.0011) [2023-12-27 00:17:24,838][105692] Updated weights for policy 0, policy_version 1222661 (0.0011) [2023-12-27 00:17:24,890][105692] Updated weights for policy 0, policy_version 1222671 (0.0010) [2023-12-27 00:17:25,436][105620] Updated weights for policy 1, policy_version 1223942 (0.0005) [2023-12-27 00:17:25,487][105620] Updated weights for policy 1, policy_version 1223952 (0.0005) [2023-12-27 00:17:25,548][105620] Updated weights for policy 1, policy_version 1223962 (0.0007) [2023-12-27 00:17:25,648][105692] Updated weights for policy 0, policy_version 1222681 (0.0010) [2023-12-27 00:17:25,699][105692] Updated weights for policy 0, policy_version 1222691 (0.0010) [2023-12-27 00:17:25,751][105692] Updated weights for policy 0, policy_version 1222701 (0.0010) [2023-12-27 00:17:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.8, 300 sec: 19494.2). Total num frames: 626442240. Throughput: 0: 10003.4, 1: 9835.2. Samples: 626451964. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:17:26,063][104569] Avg episode reward: [(0, '8178.757'), (1, '8763.843')] [2023-12-27 00:17:26,303][105620] Updated weights for policy 1, policy_version 1223972 (0.0008) [2023-12-27 00:17:26,354][105620] Updated weights for policy 1, policy_version 1223982 (0.0008) [2023-12-27 00:17:26,405][105620] Updated weights for policy 1, policy_version 1223992 (0.0006) [2023-12-27 00:17:26,419][105692] Updated weights for policy 0, policy_version 1222711 (0.0010) [2023-12-27 00:17:26,486][105692] Updated weights for policy 0, policy_version 1222721 (0.0010) [2023-12-27 00:17:26,539][105692] Updated weights for policy 0, policy_version 1222731 (0.0010) [2023-12-27 00:17:27,070][105620] Updated weights for policy 1, policy_version 1224002 (0.0006) [2023-12-27 00:17:27,123][105620] Updated weights for policy 1, policy_version 1224012 (0.0009) [2023-12-27 00:17:27,147][105692] Updated weights for policy 0, policy_version 1222741 (0.0008) [2023-12-27 00:17:27,174][105620] Updated weights for policy 1, policy_version 1224022 (0.0008) [2023-12-27 00:17:27,202][105692] Updated weights for policy 0, policy_version 1222751 (0.0005) [2023-12-27 00:17:27,221][105620] Updated weights for policy 1, policy_version 1224032 (0.0008) [2023-12-27 00:17:27,254][105692] Updated weights for policy 0, policy_version 1222761 (0.0005) [2023-12-27 00:17:27,771][105692] Updated weights for policy 0, policy_version 1222771 (0.0005) [2023-12-27 00:17:27,831][105692] Updated weights for policy 0, policy_version 1222781 (0.0008) [2023-12-27 00:17:27,878][105692] Updated weights for policy 0, policy_version 1222791 (0.0009) [2023-12-27 00:17:28,085][105620] Updated weights for policy 1, policy_version 1224042 (0.0009) [2023-12-27 00:17:28,132][105620] Updated weights for policy 1, policy_version 1224052 (0.0007) [2023-12-27 00:17:28,180][105620] Updated weights for policy 1, policy_version 1224062 (0.0008) [2023-12-27 00:17:28,505][105692] Updated weights for policy 0, policy_version 1222801 (0.0006) [2023-12-27 00:17:28,570][105692] Updated weights for policy 0, policy_version 1222811 (0.0010) [2023-12-27 00:17:28,632][105692] Updated weights for policy 0, policy_version 1222821 (0.0010) [2023-12-27 00:17:28,684][105692] Updated weights for policy 0, policy_version 1222831 (0.0009) [2023-12-27 00:17:29,020][105620] Updated weights for policy 1, policy_version 1224072 (0.0009) [2023-12-27 00:17:29,074][105620] Updated weights for policy 1, policy_version 1224083 (0.0009) [2023-12-27 00:17:29,132][105620] Updated weights for policy 1, policy_version 1224093 (0.0008) [2023-12-27 00:17:29,273][105692] Updated weights for policy 0, policy_version 1222841 (0.0007) [2023-12-27 00:17:29,325][105692] Updated weights for policy 0, policy_version 1222851 (0.0005) [2023-12-27 00:17:29,389][105692] Updated weights for policy 0, policy_version 1222861 (0.0008) [2023-12-27 00:17:29,927][105620] Updated weights for policy 1, policy_version 1224103 (0.0007) [2023-12-27 00:17:29,989][105620] Updated weights for policy 1, policy_version 1224113 (0.0009) [2023-12-27 00:17:30,038][105620] Updated weights for policy 1, policy_version 1224123 (0.0009) [2023-12-27 00:17:30,097][105692] Updated weights for policy 0, policy_version 1222871 (0.0009) [2023-12-27 00:17:30,157][105692] Updated weights for policy 0, policy_version 1222881 (0.0008) [2023-12-27 00:17:30,215][105692] Updated weights for policy 0, policy_version 1222891 (0.0009) [2023-12-27 00:17:30,807][105620] Updated weights for policy 1, policy_version 1224133 (0.0008) [2023-12-27 00:17:30,862][105620] Updated weights for policy 1, policy_version 1224143 (0.0010) [2023-12-27 00:17:30,922][105620] Updated weights for policy 1, policy_version 1224153 (0.0008) [2023-12-27 00:17:30,939][105692] Updated weights for policy 0, policy_version 1222901 (0.0007) [2023-12-27 00:17:30,997][105692] Updated weights for policy 0, policy_version 1222911 (0.0010) [2023-12-27 00:17:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 626540544. Throughput: 0: 10122.6, 1: 9822.3. Samples: 626512668. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:17:31,062][104569] Avg episode reward: [(0, '7632.960'), (1, '9259.700')] [2023-12-27 00:17:31,065][105692] Updated weights for policy 0, policy_version 1222921 (0.0009) [2023-12-27 00:17:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001224160_313425920.pth... [2023-12-27 00:17:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001223040_313139200.pth [2023-12-27 00:17:31,102][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001222928_313122816.pth... [2023-12-27 00:17:31,106][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001221712_312811520.pth [2023-12-27 00:17:31,722][105692] Updated weights for policy 0, policy_version 1222931 (0.0009) [2023-12-27 00:17:31,773][105620] Updated weights for policy 1, policy_version 1224163 (0.0007) [2023-12-27 00:17:31,783][105692] Updated weights for policy 0, policy_version 1222941 (0.0008) [2023-12-27 00:17:31,833][105620] Updated weights for policy 1, policy_version 1224173 (0.0006) [2023-12-27 00:17:31,843][105692] Updated weights for policy 0, policy_version 1222951 (0.0007) [2023-12-27 00:17:31,896][105620] Updated weights for policy 1, policy_version 1224183 (0.0006) [2023-12-27 00:17:32,581][105692] Updated weights for policy 0, policy_version 1222961 (0.0006) [2023-12-27 00:17:32,632][105692] Updated weights for policy 0, policy_version 1222971 (0.0009) [2023-12-27 00:17:32,644][105620] Updated weights for policy 1, policy_version 1224193 (0.0009) [2023-12-27 00:17:32,689][105692] Updated weights for policy 0, policy_version 1222981 (0.0008) [2023-12-27 00:17:32,705][105620] Updated weights for policy 1, policy_version 1224203 (0.0008) [2023-12-27 00:17:32,741][105692] Updated weights for policy 0, policy_version 1222991 (0.0010) [2023-12-27 00:17:32,764][105620] Updated weights for policy 1, policy_version 1224213 (0.0008) [2023-12-27 00:17:32,818][105620] Updated weights for policy 1, policy_version 1224223 (0.0009) [2023-12-27 00:17:33,338][105692] Updated weights for policy 0, policy_version 1223001 (0.0009) [2023-12-27 00:17:33,395][105692] Updated weights for policy 0, policy_version 1223011 (0.0009) [2023-12-27 00:17:33,448][105692] Updated weights for policy 0, policy_version 1223021 (0.0008) [2023-12-27 00:17:33,607][105620] Updated weights for policy 1, policy_version 1224233 (0.0009) [2023-12-27 00:17:33,653][105620] Updated weights for policy 1, policy_version 1224243 (0.0008) [2023-12-27 00:17:33,701][105620] Updated weights for policy 1, policy_version 1224253 (0.0009) [2023-12-27 00:17:34,123][105692] Updated weights for policy 0, policy_version 1223031 (0.0006) [2023-12-27 00:17:34,186][105692] Updated weights for policy 0, policy_version 1223041 (0.0008) [2023-12-27 00:17:34,245][105692] Updated weights for policy 0, policy_version 1223051 (0.0009) [2023-12-27 00:17:34,504][105620] Updated weights for policy 1, policy_version 1224263 (0.0009) [2023-12-27 00:17:34,554][105620] Updated weights for policy 1, policy_version 1224273 (0.0009) [2023-12-27 00:17:34,615][105620] Updated weights for policy 1, policy_version 1224283 (0.0009) [2023-12-27 00:17:34,955][105692] Updated weights for policy 0, policy_version 1223061 (0.0009) [2023-12-27 00:17:35,014][105692] Updated weights for policy 0, policy_version 1223071 (0.0009) [2023-12-27 00:17:35,075][105692] Updated weights for policy 0, policy_version 1223081 (0.0008) [2023-12-27 00:17:35,380][105620] Updated weights for policy 1, policy_version 1224293 (0.0007) [2023-12-27 00:17:35,431][105620] Updated weights for policy 1, policy_version 1224303 (0.0005) [2023-12-27 00:17:35,486][105620] Updated weights for policy 1, policy_version 1224313 (0.0005) [2023-12-27 00:17:35,821][105692] Updated weights for policy 0, policy_version 1223091 (0.0009) [2023-12-27 00:17:35,885][105692] Updated weights for policy 0, policy_version 1223101 (0.0010) [2023-12-27 00:17:35,944][105692] Updated weights for policy 0, policy_version 1223111 (0.0010) [2023-12-27 00:17:36,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 626638848. Throughput: 0: 10114.9, 1: 9660.7. Samples: 626627584. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:17:36,062][104569] Avg episode reward: [(0, '8085.842'), (1, '9170.202')] [2023-12-27 00:17:36,074][105620] Updated weights for policy 1, policy_version 1224323 (0.0005) [2023-12-27 00:17:36,138][105620] Updated weights for policy 1, policy_version 1224333 (0.0009) [2023-12-27 00:17:36,205][105620] Updated weights for policy 1, policy_version 1224343 (0.0006) [2023-12-27 00:17:36,693][105692] Updated weights for policy 0, policy_version 1223121 (0.0010) [2023-12-27 00:17:36,753][105692] Updated weights for policy 0, policy_version 1223131 (0.0010) [2023-12-27 00:17:36,811][105692] Updated weights for policy 0, policy_version 1223141 (0.0010) [2023-12-27 00:17:36,832][105620] Updated weights for policy 1, policy_version 1224353 (0.0007) [2023-12-27 00:17:36,875][105692] Updated weights for policy 0, policy_version 1223151 (0.0011) [2023-12-27 00:17:36,890][105620] Updated weights for policy 1, policy_version 1224363 (0.0007) [2023-12-27 00:17:36,947][105620] Updated weights for policy 1, policy_version 1224373 (0.0008) [2023-12-27 00:17:37,011][105620] Updated weights for policy 1, policy_version 1224383 (0.0008) [2023-12-27 00:17:37,480][105692] Updated weights for policy 0, policy_version 1223161 (0.0006) [2023-12-27 00:17:37,534][105692] Updated weights for policy 0, policy_version 1223171 (0.0005) [2023-12-27 00:17:37,591][105692] Updated weights for policy 0, policy_version 1223181 (0.0005) [2023-12-27 00:17:37,853][105620] Updated weights for policy 1, policy_version 1224393 (0.0009) [2023-12-27 00:17:37,916][105620] Updated weights for policy 1, policy_version 1224403 (0.0012) [2023-12-27 00:17:37,984][105620] Updated weights for policy 1, policy_version 1224413 (0.0010) [2023-12-27 00:17:38,117][105692] Updated weights for policy 0, policy_version 1223191 (0.0009) [2023-12-27 00:17:38,178][105692] Updated weights for policy 0, policy_version 1223201 (0.0010) [2023-12-27 00:17:38,222][105692] Updated weights for policy 0, policy_version 1223211 (0.0010) [2023-12-27 00:17:38,800][105620] Updated weights for policy 1, policy_version 1224423 (0.0009) [2023-12-27 00:17:38,852][105620] Updated weights for policy 1, policy_version 1224433 (0.0008) [2023-12-27 00:17:38,914][105620] Updated weights for policy 1, policy_version 1224443 (0.0008) [2023-12-27 00:17:38,933][105692] Updated weights for policy 0, policy_version 1223221 (0.0010) [2023-12-27 00:17:38,989][105692] Updated weights for policy 0, policy_version 1223231 (0.0010) [2023-12-27 00:17:39,043][105692] Updated weights for policy 0, policy_version 1223241 (0.0010) [2023-12-27 00:17:39,596][105620] Updated weights for policy 1, policy_version 1224453 (0.0006) [2023-12-27 00:17:39,666][105620] Updated weights for policy 1, policy_version 1224463 (0.0008) [2023-12-27 00:17:39,725][105620] Updated weights for policy 1, policy_version 1224473 (0.0008) [2023-12-27 00:17:39,753][105692] Updated weights for policy 0, policy_version 1223251 (0.0010) [2023-12-27 00:17:39,821][105692] Updated weights for policy 0, policy_version 1223261 (0.0011) [2023-12-27 00:17:39,891][105692] Updated weights for policy 0, policy_version 1223271 (0.0010) [2023-12-27 00:17:40,453][105620] Updated weights for policy 1, policy_version 1224483 (0.0008) [2023-12-27 00:17:40,508][105692] Updated weights for policy 0, policy_version 1223281 (0.0009) [2023-12-27 00:17:40,512][105620] Updated weights for policy 1, policy_version 1224493 (0.0009) [2023-12-27 00:17:40,570][105692] Updated weights for policy 0, policy_version 1223291 (0.0007) [2023-12-27 00:17:40,574][105620] Updated weights for policy 1, policy_version 1224503 (0.0006) [2023-12-27 00:17:40,631][105692] Updated weights for policy 0, policy_version 1223301 (0.0005) [2023-12-27 00:17:40,691][105692] Updated weights for policy 0, policy_version 1223311 (0.0005) [2023-12-27 00:17:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 626737152. Throughput: 0: 10200.2, 1: 9605.0. Samples: 626746664. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:17:41,062][104569] Avg episode reward: [(0, '8448.449'), (1, '9261.813')] [2023-12-27 00:17:41,356][105692] Updated weights for policy 0, policy_version 1223321 (0.0011) [2023-12-27 00:17:41,394][105620] Updated weights for policy 1, policy_version 1224513 (0.0008) [2023-12-27 00:17:41,424][105692] Updated weights for policy 0, policy_version 1223331 (0.0011) [2023-12-27 00:17:41,462][105620] Updated weights for policy 1, policy_version 1224523 (0.0009) [2023-12-27 00:17:41,479][105692] Updated weights for policy 0, policy_version 1223341 (0.0009) [2023-12-27 00:17:41,519][105620] Updated weights for policy 1, policy_version 1224533 (0.0009) [2023-12-27 00:17:41,573][105620] Updated weights for policy 1, policy_version 1224543 (0.0009) [2023-12-27 00:17:42,228][105692] Updated weights for policy 0, policy_version 1223351 (0.0006) [2023-12-27 00:17:42,300][105692] Updated weights for policy 0, policy_version 1223361 (0.0009) [2023-12-27 00:17:42,365][105692] Updated weights for policy 0, policy_version 1223371 (0.0008) [2023-12-27 00:17:42,403][105620] Updated weights for policy 1, policy_version 1224553 (0.0009) [2023-12-27 00:17:42,467][105620] Updated weights for policy 1, policy_version 1224563 (0.0008) [2023-12-27 00:17:42,535][105620] Updated weights for policy 1, policy_version 1224573 (0.0008) [2023-12-27 00:17:42,971][105692] Updated weights for policy 0, policy_version 1223381 (0.0006) [2023-12-27 00:17:43,019][105692] Updated weights for policy 0, policy_version 1223391 (0.0007) [2023-12-27 00:17:43,036][105585] KL-divergence is very high: 105.3005 [2023-12-27 00:17:43,040][105585] KL-divergence is very high: 118.2027 [2023-12-27 00:17:43,059][105585] KL-divergence is very high: 106.9857 [2023-12-27 00:17:43,064][105585] KL-divergence is very high: 121.5500 [2023-12-27 00:17:43,065][105692] Updated weights for policy 0, policy_version 1223401 (0.0008) [2023-12-27 00:17:43,076][105585] KL-divergence is very high: 136.9702 [2023-12-27 00:17:43,082][105585] KL-divergence is very high: 141.5675 [2023-12-27 00:17:43,346][105620] Updated weights for policy 1, policy_version 1224583 (0.0009) [2023-12-27 00:17:43,404][105620] Updated weights for policy 1, policy_version 1224593 (0.0009) [2023-12-27 00:17:43,469][105620] Updated weights for policy 1, policy_version 1224603 (0.0009) [2023-12-27 00:17:43,768][105692] Updated weights for policy 0, policy_version 1223411 (0.0008) [2023-12-27 00:17:43,815][105692] Updated weights for policy 0, policy_version 1223421 (0.0010) [2023-12-27 00:17:43,867][105692] Updated weights for policy 0, policy_version 1223431 (0.0008) [2023-12-27 00:17:44,167][105620] Updated weights for policy 1, policy_version 1224613 (0.0008) [2023-12-27 00:17:44,229][105620] Updated weights for policy 1, policy_version 1224623 (0.0008) [2023-12-27 00:17:44,292][105620] Updated weights for policy 1, policy_version 1224633 (0.0008) [2023-12-27 00:17:44,561][105692] Updated weights for policy 0, policy_version 1223441 (0.0005) [2023-12-27 00:17:44,617][105692] Updated weights for policy 0, policy_version 1223451 (0.0007) [2023-12-27 00:17:44,666][105692] Updated weights for policy 0, policy_version 1223461 (0.0009) [2023-12-27 00:17:44,713][105692] Updated weights for policy 0, policy_version 1223471 (0.0009) [2023-12-27 00:17:45,089][105620] Updated weights for policy 1, policy_version 1224643 (0.0010) [2023-12-27 00:17:45,156][105620] Updated weights for policy 1, policy_version 1224653 (0.0010) [2023-12-27 00:17:45,219][105620] Updated weights for policy 1, policy_version 1224663 (0.0009) [2023-12-27 00:17:45,446][105692] Updated weights for policy 0, policy_version 1223481 (0.0008) [2023-12-27 00:17:45,507][105692] Updated weights for policy 0, policy_version 1223491 (0.0009) [2023-12-27 00:17:45,571][105692] Updated weights for policy 0, policy_version 1223501 (0.0009) [2023-12-27 00:17:46,027][105620] Updated weights for policy 1, policy_version 1224673 (0.0009) [2023-12-27 00:17:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.9, 300 sec: 19522.0). Total num frames: 626827264. Throughput: 0: 10204.0, 1: 9513.7. Samples: 626802568. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:17:46,062][104569] Avg episode reward: [(0, '7813.871'), (1, '9170.541')] [2023-12-27 00:17:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001223504_313270272.pth... [2023-12-27 00:17:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001222320_312967168.pth [2023-12-27 00:17:46,084][105620] Updated weights for policy 1, policy_version 1224683 (0.0009) [2023-12-27 00:17:46,144][105620] Updated weights for policy 1, policy_version 1224693 (0.0009) [2023-12-27 00:17:46,196][105692] Updated weights for policy 0, policy_version 1223511 (0.0008) [2023-12-27 00:17:46,202][105620] Updated weights for policy 1, policy_version 1224703 (0.0008) [2023-12-27 00:17:46,206][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001224704_313565184.pth... [2023-12-27 00:17:46,210][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001223584_313278464.pth [2023-12-27 00:17:46,248][105692] Updated weights for policy 0, policy_version 1223521 (0.0008) [2023-12-27 00:17:46,294][105692] Updated weights for policy 0, policy_version 1223531 (0.0008) [2023-12-27 00:17:46,879][105692] Updated weights for policy 0, policy_version 1223541 (0.0009) [2023-12-27 00:17:46,936][105692] Updated weights for policy 0, policy_version 1223551 (0.0009) [2023-12-27 00:17:47,002][105692] Updated weights for policy 0, policy_version 1223561 (0.0009) [2023-12-27 00:17:47,047][105620] Updated weights for policy 1, policy_version 1224713 (0.0007) [2023-12-27 00:17:47,101][105620] Updated weights for policy 1, policy_version 1224723 (0.0009) [2023-12-27 00:17:47,158][105620] Updated weights for policy 1, policy_version 1224733 (0.0009) [2023-12-27 00:17:47,657][105692] Updated weights for policy 0, policy_version 1223571 (0.0006) [2023-12-27 00:17:47,724][105692] Updated weights for policy 0, policy_version 1223581 (0.0010) [2023-12-27 00:17:47,779][105692] Updated weights for policy 0, policy_version 1223591 (0.0010) [2023-12-27 00:17:47,957][105620] Updated weights for policy 1, policy_version 1224743 (0.0009) [2023-12-27 00:17:48,016][105620] Updated weights for policy 1, policy_version 1224753 (0.0009) [2023-12-27 00:17:48,066][105620] Updated weights for policy 1, policy_version 1224763 (0.0009) [2023-12-27 00:17:48,470][105692] Updated weights for policy 0, policy_version 1223601 (0.0010) [2023-12-27 00:17:48,523][105692] Updated weights for policy 0, policy_version 1223611 (0.0005) [2023-12-27 00:17:48,573][105692] Updated weights for policy 0, policy_version 1223621 (0.0006) [2023-12-27 00:17:48,625][105692] Updated weights for policy 0, policy_version 1223631 (0.0005) [2023-12-27 00:17:48,927][105620] Updated weights for policy 1, policy_version 1224773 (0.0010) [2023-12-27 00:17:48,981][105620] Updated weights for policy 1, policy_version 1224783 (0.0010) [2023-12-27 00:17:49,032][105620] Updated weights for policy 1, policy_version 1224794 (0.0010) [2023-12-27 00:17:49,187][105692] Updated weights for policy 0, policy_version 1223641 (0.0006) [2023-12-27 00:17:49,253][105692] Updated weights for policy 0, policy_version 1223651 (0.0007) [2023-12-27 00:17:49,312][105692] Updated weights for policy 0, policy_version 1223661 (0.0009) [2023-12-27 00:17:49,862][105620] Updated weights for policy 1, policy_version 1224804 (0.0010) [2023-12-27 00:17:49,911][105620] Updated weights for policy 1, policy_version 1224814 (0.0010) [2023-12-27 00:17:49,971][105620] Updated weights for policy 1, policy_version 1224824 (0.0011) [2023-12-27 00:17:50,010][105692] Updated weights for policy 0, policy_version 1223671 (0.0008) [2023-12-27 00:17:50,063][105692] Updated weights for policy 0, policy_version 1223681 (0.0010) [2023-12-27 00:17:50,110][105692] Updated weights for policy 0, policy_version 1223691 (0.0008) [2023-12-27 00:17:50,693][105620] Updated weights for policy 1, policy_version 1224834 (0.0008) [2023-12-27 00:17:50,757][105620] Updated weights for policy 1, policy_version 1224844 (0.0008) [2023-12-27 00:17:50,758][105692] Updated weights for policy 0, policy_version 1223701 (0.0007) [2023-12-27 00:17:50,813][105692] Updated weights for policy 0, policy_version 1223711 (0.0009) [2023-12-27 00:17:50,822][105620] Updated weights for policy 1, policy_version 1224854 (0.0005) [2023-12-27 00:17:50,863][105692] Updated weights for policy 0, policy_version 1223721 (0.0010) [2023-12-27 00:17:50,878][105620] Updated weights for policy 1, policy_version 1224864 (0.0005) [2023-12-27 00:17:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 626933760. Throughput: 0: 10255.5, 1: 9363.6. Samples: 626918272. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:17:51,062][104569] Avg episode reward: [(0, '7182.400'), (1, '8896.816')] [2023-12-27 00:17:51,472][105620] Updated weights for policy 1, policy_version 1224874 (0.0010) [2023-12-27 00:17:51,523][105620] Updated weights for policy 1, policy_version 1224884 (0.0010) [2023-12-27 00:17:51,578][105620] Updated weights for policy 1, policy_version 1224894 (0.0010) [2023-12-27 00:17:51,699][105692] Updated weights for policy 0, policy_version 1223731 (0.0008) [2023-12-27 00:17:51,768][105692] Updated weights for policy 0, policy_version 1223741 (0.0009) [2023-12-27 00:17:51,834][105692] Updated weights for policy 0, policy_version 1223751 (0.0009) [2023-12-27 00:17:52,230][105620] Updated weights for policy 1, policy_version 1224904 (0.0009) [2023-12-27 00:17:52,298][105620] Updated weights for policy 1, policy_version 1224914 (0.0008) [2023-12-27 00:17:52,370][105620] Updated weights for policy 1, policy_version 1224924 (0.0009) [2023-12-27 00:17:52,651][105692] Updated weights for policy 0, policy_version 1223761 (0.0009) [2023-12-27 00:17:52,715][105692] Updated weights for policy 0, policy_version 1223771 (0.0008) [2023-12-27 00:17:52,770][105692] Updated weights for policy 0, policy_version 1223781 (0.0008) [2023-12-27 00:17:52,830][105692] Updated weights for policy 0, policy_version 1223791 (0.0008) [2023-12-27 00:17:53,062][105620] Updated weights for policy 1, policy_version 1224934 (0.0010) [2023-12-27 00:17:53,113][105620] Updated weights for policy 1, policy_version 1224944 (0.0009) [2023-12-27 00:17:53,173][105620] Updated weights for policy 1, policy_version 1224954 (0.0011) [2023-12-27 00:17:53,534][105692] Updated weights for policy 0, policy_version 1223801 (0.0008) [2023-12-27 00:17:53,592][105692] Updated weights for policy 0, policy_version 1223811 (0.0008) [2023-12-27 00:17:53,653][105692] Updated weights for policy 0, policy_version 1223821 (0.0006) [2023-12-27 00:17:53,917][105620] Updated weights for policy 1, policy_version 1224964 (0.0008) [2023-12-27 00:17:53,971][105620] Updated weights for policy 1, policy_version 1224974 (0.0010) [2023-12-27 00:17:54,032][105620] Updated weights for policy 1, policy_version 1224984 (0.0010) [2023-12-27 00:17:54,282][105692] Updated weights for policy 0, policy_version 1223831 (0.0009) [2023-12-27 00:17:54,344][105692] Updated weights for policy 0, policy_version 1223841 (0.0010) [2023-12-27 00:17:54,406][105692] Updated weights for policy 0, policy_version 1223851 (0.0005) [2023-12-27 00:17:54,762][105620] Updated weights for policy 1, policy_version 1224994 (0.0009) [2023-12-27 00:17:54,808][105620] Updated weights for policy 1, policy_version 1225004 (0.0005) [2023-12-27 00:17:54,863][105620] Updated weights for policy 1, policy_version 1225014 (0.0009) [2023-12-27 00:17:54,913][105620] Updated weights for policy 1, policy_version 1225024 (0.0009) [2023-12-27 00:17:54,999][105692] Updated weights for policy 0, policy_version 1223861 (0.0008) [2023-12-27 00:17:55,064][105692] Updated weights for policy 0, policy_version 1223871 (0.0011) [2023-12-27 00:17:55,130][105692] Updated weights for policy 0, policy_version 1223881 (0.0011) [2023-12-27 00:17:55,599][105620] Updated weights for policy 1, policy_version 1225034 (0.0005) [2023-12-27 00:17:55,645][105620] Updated weights for policy 1, policy_version 1225044 (0.0005) [2023-12-27 00:17:55,692][105620] Updated weights for policy 1, policy_version 1225054 (0.0005) [2023-12-27 00:17:55,775][105692] Updated weights for policy 0, policy_version 1223891 (0.0009) [2023-12-27 00:17:55,803][105585] KL-divergence is very high: 193.3294 [2023-12-27 00:17:55,810][105585] KL-divergence is very high: 286.7925 [2023-12-27 00:17:55,842][105692] Updated weights for policy 0, policy_version 1223901 (0.0010) [2023-12-27 00:17:55,851][105585] KL-divergence is very high: 398.4389 [2023-12-27 00:17:55,856][105585] KL-divergence is very high: 511.5773 [2023-12-27 00:17:55,893][105585] KL-divergence is very high: 438.6471 [2023-12-27 00:17:55,894][105692] Updated weights for policy 0, policy_version 1223911 (0.0011) [2023-12-27 00:17:55,900][105585] KL-divergence is very high: 537.2488 [2023-12-27 00:17:55,939][105585] KL-divergence is very high: 394.6662 [2023-12-27 00:17:56,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 627032064. Throughput: 0: 10113.2, 1: 9417.0. Samples: 627038440. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:17:56,063][104569] Avg episode reward: [(0, '7360.730'), (1, '8988.102')] [2023-12-27 00:17:56,276][105620] Updated weights for policy 1, policy_version 1225064 (0.0009) [2023-12-27 00:17:56,331][105620] Updated weights for policy 1, policy_version 1225074 (0.0010) [2023-12-27 00:17:56,382][105620] Updated weights for policy 1, policy_version 1225084 (0.0010) [2023-12-27 00:17:56,534][105692] Updated weights for policy 0, policy_version 1223921 (0.0010) [2023-12-27 00:17:56,585][105692] Updated weights for policy 0, policy_version 1223931 (0.0010) [2023-12-27 00:17:56,587][105585] KL-divergence is very high: 111.5970 [2023-12-27 00:17:56,593][105585] KL-divergence is very high: 101.3104 [2023-12-27 00:17:56,598][105585] KL-divergence is very high: 264.2504 [2023-12-27 00:17:56,625][105585] KL-divergence is very high: 155.7457 [2023-12-27 00:17:56,630][105585] KL-divergence is very high: 129.2709 [2023-12-27 00:17:56,635][105585] KL-divergence is very high: 322.2361 [2023-12-27 00:17:56,635][105692] Updated weights for policy 0, policy_version 1223941 (0.0010) [2023-12-27 00:17:56,662][105585] KL-divergence is very high: 147.5901 [2023-12-27 00:17:56,667][105585] KL-divergence is very high: 113.4930 [2023-12-27 00:17:56,672][105585] KL-divergence is very high: 292.1111 [2023-12-27 00:17:56,683][105692] Updated weights for policy 0, policy_version 1223951 (0.0010) [2023-12-27 00:17:56,955][105620] Updated weights for policy 1, policy_version 1225094 (0.0008) [2023-12-27 00:17:57,007][105620] Updated weights for policy 1, policy_version 1225104 (0.0008) [2023-12-27 00:17:57,056][105620] Updated weights for policy 1, policy_version 1225114 (0.0008) [2023-12-27 00:17:57,444][105692] Updated weights for policy 0, policy_version 1223961 (0.0008) [2023-12-27 00:17:57,501][105692] Updated weights for policy 0, policy_version 1223971 (0.0005) [2023-12-27 00:17:57,553][105692] Updated weights for policy 0, policy_version 1223981 (0.0005) [2023-12-27 00:17:57,900][105620] Updated weights for policy 1, policy_version 1225124 (0.0009) [2023-12-27 00:17:57,963][105620] Updated weights for policy 1, policy_version 1225134 (0.0010) [2023-12-27 00:17:58,032][105620] Updated weights for policy 1, policy_version 1225144 (0.0009) [2023-12-27 00:17:58,078][105692] Updated weights for policy 0, policy_version 1223991 (0.0010) [2023-12-27 00:17:58,082][105585] KL-divergence is very high: 104.4343 [2023-12-27 00:17:58,125][105692] Updated weights for policy 0, policy_version 1224001 (0.0010) [2023-12-27 00:17:58,184][105692] Updated weights for policy 0, policy_version 1224011 (0.0010) [2023-12-27 00:17:58,800][105620] Updated weights for policy 1, policy_version 1225154 (0.0007) [2023-12-27 00:17:58,868][105620] Updated weights for policy 1, policy_version 1225164 (0.0007) [2023-12-27 00:17:58,939][105620] Updated weights for policy 1, policy_version 1225174 (0.0008) [2023-12-27 00:17:59,003][105620] Updated weights for policy 1, policy_version 1225184 (0.0009) [2023-12-27 00:17:59,035][105692] Updated weights for policy 0, policy_version 1224021 (0.0008) [2023-12-27 00:17:59,097][105692] Updated weights for policy 0, policy_version 1224031 (0.0008) [2023-12-27 00:17:59,155][105692] Updated weights for policy 0, policy_version 1224041 (0.0009) [2023-12-27 00:17:59,767][105620] Updated weights for policy 1, policy_version 1225194 (0.0009) [2023-12-27 00:17:59,816][105620] Updated weights for policy 1, policy_version 1225204 (0.0009) [2023-12-27 00:17:59,866][105620] Updated weights for policy 1, policy_version 1225214 (0.0009) [2023-12-27 00:17:59,939][105692] Updated weights for policy 0, policy_version 1224051 (0.0007) [2023-12-27 00:17:59,997][105692] Updated weights for policy 0, policy_version 1224061 (0.0009) [2023-12-27 00:18:00,051][105692] Updated weights for policy 0, policy_version 1224071 (0.0008) [2023-12-27 00:18:00,659][105620] Updated weights for policy 1, policy_version 1225225 (0.0010) [2023-12-27 00:18:00,722][105620] Updated weights for policy 1, policy_version 1225235 (0.0009) [2023-12-27 00:18:00,788][105620] Updated weights for policy 1, policy_version 1225245 (0.0008) [2023-12-27 00:18:00,790][105692] Updated weights for policy 0, policy_version 1224081 (0.0009) [2023-12-27 00:18:00,838][105692] Updated weights for policy 0, policy_version 1224091 (0.0008) [2023-12-27 00:18:00,892][105692] Updated weights for policy 0, policy_version 1224101 (0.0009) [2023-12-27 00:18:00,939][105692] Updated weights for policy 0, policy_version 1224111 (0.0009) [2023-12-27 00:18:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19521.9). Total num frames: 627130368. Throughput: 0: 10215.8, 1: 9444.7. Samples: 627099292. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:18:01,063][104569] Avg episode reward: [(0, '7000.254'), (1, '9262.322')] [2023-12-27 00:18:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001224112_313425920.pth... [2023-12-27 00:18:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001225248_313704448.pth... [2023-12-27 00:18:01,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001224160_313425920.pth [2023-12-27 00:18:01,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001222928_313122816.pth [2023-12-27 00:18:01,495][105620] Updated weights for policy 1, policy_version 1225255 (0.0006) [2023-12-27 00:18:01,552][105620] Updated weights for policy 1, policy_version 1225265 (0.0009) [2023-12-27 00:18:01,615][105620] Updated weights for policy 1, policy_version 1225275 (0.0009) [2023-12-27 00:18:01,736][105692] Updated weights for policy 0, policy_version 1224121 (0.0008) [2023-12-27 00:18:01,791][105692] Updated weights for policy 0, policy_version 1224131 (0.0010) [2023-12-27 00:18:01,851][105692] Updated weights for policy 0, policy_version 1224141 (0.0012) [2023-12-27 00:18:02,208][105620] Updated weights for policy 1, policy_version 1225285 (0.0007) [2023-12-27 00:18:02,260][105620] Updated weights for policy 1, policy_version 1225295 (0.0006) [2023-12-27 00:18:02,314][105620] Updated weights for policy 1, policy_version 1225305 (0.0005) [2023-12-27 00:18:02,555][105692] Updated weights for policy 0, policy_version 1224151 (0.0010) [2023-12-27 00:18:02,611][105692] Updated weights for policy 0, policy_version 1224161 (0.0010) [2023-12-27 00:18:02,682][105692] Updated weights for policy 0, policy_version 1224171 (0.0011) [2023-12-27 00:18:02,932][105620] Updated weights for policy 1, policy_version 1225315 (0.0011) [2023-12-27 00:18:02,993][105620] Updated weights for policy 1, policy_version 1225325 (0.0010) [2023-12-27 00:18:03,041][105620] Updated weights for policy 1, policy_version 1225335 (0.0010) [2023-12-27 00:18:03,272][105692] Updated weights for policy 0, policy_version 1224181 (0.0008) [2023-12-27 00:18:03,347][105692] Updated weights for policy 0, policy_version 1224191 (0.0005) [2023-12-27 00:18:03,422][105692] Updated weights for policy 0, policy_version 1224201 (0.0009) [2023-12-27 00:18:03,627][105620] Updated weights for policy 1, policy_version 1225345 (0.0010) [2023-12-27 00:18:03,682][105620] Updated weights for policy 1, policy_version 1225355 (0.0010) [2023-12-27 00:18:03,746][105620] Updated weights for policy 1, policy_version 1225365 (0.0010) [2023-12-27 00:18:03,800][105620] Updated weights for policy 1, policy_version 1225375 (0.0010) [2023-12-27 00:18:03,946][105692] Updated weights for policy 0, policy_version 1224211 (0.0010) [2023-12-27 00:18:04,011][105692] Updated weights for policy 0, policy_version 1224221 (0.0008) [2023-12-27 00:18:04,076][105692] Updated weights for policy 0, policy_version 1224231 (0.0008) [2023-12-27 00:18:04,461][105620] Updated weights for policy 1, policy_version 1225385 (0.0006) [2023-12-27 00:18:04,523][105620] Updated weights for policy 1, policy_version 1225395 (0.0006) [2023-12-27 00:18:04,582][105620] Updated weights for policy 1, policy_version 1225405 (0.0011) [2023-12-27 00:18:04,733][105692] Updated weights for policy 0, policy_version 1224241 (0.0008) [2023-12-27 00:18:04,784][105692] Updated weights for policy 0, policy_version 1224251 (0.0005) [2023-12-27 00:18:04,832][105692] Updated weights for policy 0, policy_version 1224261 (0.0005) [2023-12-27 00:18:04,882][105692] Updated weights for policy 0, policy_version 1224271 (0.0006) [2023-12-27 00:18:05,253][105620] Updated weights for policy 1, policy_version 1225415 (0.0010) [2023-12-27 00:18:05,305][105620] Updated weights for policy 1, policy_version 1225425 (0.0010) [2023-12-27 00:18:05,356][105620] Updated weights for policy 1, policy_version 1225435 (0.0010) [2023-12-27 00:18:05,611][105692] Updated weights for policy 0, policy_version 1224281 (0.0009) [2023-12-27 00:18:05,677][105692] Updated weights for policy 0, policy_version 1224291 (0.0008) [2023-12-27 00:18:05,732][105692] Updated weights for policy 0, policy_version 1224301 (0.0008) [2023-12-27 00:18:06,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 627228672. Throughput: 0: 10156.5, 1: 9418.9. Samples: 627219948. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:18:06,062][104569] Avg episode reward: [(0, '7545.409'), (1, '9262.043')] [2023-12-27 00:18:06,071][105620] Updated weights for policy 1, policy_version 1225445 (0.0010) [2023-12-27 00:18:06,131][105620] Updated weights for policy 1, policy_version 1225455 (0.0009) [2023-12-27 00:18:06,188][105620] Updated weights for policy 1, policy_version 1225465 (0.0006) [2023-12-27 00:18:06,534][105692] Updated weights for policy 0, policy_version 1224311 (0.0009) [2023-12-27 00:18:06,591][105692] Updated weights for policy 0, policy_version 1224321 (0.0010) [2023-12-27 00:18:06,676][105692] Updated weights for policy 0, policy_version 1224331 (0.0009) [2023-12-27 00:18:06,799][105620] Updated weights for policy 1, policy_version 1225475 (0.0006) [2023-12-27 00:18:06,861][105620] Updated weights for policy 1, policy_version 1225485 (0.0006) [2023-12-27 00:18:06,924][105620] Updated weights for policy 1, policy_version 1225495 (0.0007) [2023-12-27 00:18:07,482][105692] Updated weights for policy 0, policy_version 1224341 (0.0009) [2023-12-27 00:18:07,530][105692] Updated weights for policy 0, policy_version 1224351 (0.0007) [2023-12-27 00:18:07,574][105692] Updated weights for policy 0, policy_version 1224361 (0.0008) [2023-12-27 00:18:07,637][105620] Updated weights for policy 1, policy_version 1225505 (0.0010) [2023-12-27 00:18:07,691][105620] Updated weights for policy 1, policy_version 1225515 (0.0008) [2023-12-27 00:18:07,748][105620] Updated weights for policy 1, policy_version 1225525 (0.0007) [2023-12-27 00:18:07,796][105620] Updated weights for policy 1, policy_version 1225535 (0.0009) [2023-12-27 00:18:08,369][105692] Updated weights for policy 0, policy_version 1224371 (0.0007) [2023-12-27 00:18:08,429][105692] Updated weights for policy 0, policy_version 1224381 (0.0008) [2023-12-27 00:18:08,488][105692] Updated weights for policy 0, policy_version 1224391 (0.0008) [2023-12-27 00:18:08,579][105620] Updated weights for policy 1, policy_version 1225545 (0.0011) [2023-12-27 00:18:08,646][105620] Updated weights for policy 1, policy_version 1225555 (0.0011) [2023-12-27 00:18:08,705][105620] Updated weights for policy 1, policy_version 1225565 (0.0011) [2023-12-27 00:18:09,106][105692] Updated weights for policy 0, policy_version 1224401 (0.0007) [2023-12-27 00:18:09,164][105692] Updated weights for policy 0, policy_version 1224411 (0.0009) [2023-12-27 00:18:09,222][105692] Updated weights for policy 0, policy_version 1224421 (0.0010) [2023-12-27 00:18:09,284][105692] Updated weights for policy 0, policy_version 1224431 (0.0006) [2023-12-27 00:18:09,450][105620] Updated weights for policy 1, policy_version 1225575 (0.0009) [2023-12-27 00:18:09,503][105620] Updated weights for policy 1, policy_version 1225585 (0.0009) [2023-12-27 00:18:09,562][105620] Updated weights for policy 1, policy_version 1225595 (0.0009) [2023-12-27 00:18:10,043][105692] Updated weights for policy 0, policy_version 1224441 (0.0009) [2023-12-27 00:18:10,102][105692] Updated weights for policy 0, policy_version 1224451 (0.0009) [2023-12-27 00:18:10,160][105692] Updated weights for policy 0, policy_version 1224461 (0.0009) [2023-12-27 00:18:10,332][105620] Updated weights for policy 1, policy_version 1225605 (0.0009) [2023-12-27 00:18:10,402][105620] Updated weights for policy 1, policy_version 1225615 (0.0007) [2023-12-27 00:18:10,472][105620] Updated weights for policy 1, policy_version 1225625 (0.0005) [2023-12-27 00:18:10,906][105692] Updated weights for policy 0, policy_version 1224471 (0.0009) [2023-12-27 00:18:10,963][105692] Updated weights for policy 0, policy_version 1224481 (0.0008) [2023-12-27 00:18:11,014][105692] Updated weights for policy 0, policy_version 1224491 (0.0009) [2023-12-27 00:18:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 627326976. Throughput: 0: 10122.7, 1: 9465.1. Samples: 627333412. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 00:18:11,063][104569] Avg episode reward: [(0, '7901.294'), (1, '9260.747')] [2023-12-27 00:18:11,235][105620] Updated weights for policy 1, policy_version 1225635 (0.0007) [2023-12-27 00:18:11,303][105620] Updated weights for policy 1, policy_version 1225645 (0.0009) [2023-12-27 00:18:11,362][105620] Updated weights for policy 1, policy_version 1225655 (0.0010) [2023-12-27 00:18:11,770][105692] Updated weights for policy 0, policy_version 1224501 (0.0009) [2023-12-27 00:18:11,818][105692] Updated weights for policy 0, policy_version 1224511 (0.0009) [2023-12-27 00:18:11,865][105692] Updated weights for policy 0, policy_version 1224521 (0.0008) [2023-12-27 00:18:12,150][105620] Updated weights for policy 1, policy_version 1225665 (0.0009) [2023-12-27 00:18:12,214][105620] Updated weights for policy 1, policy_version 1225675 (0.0009) [2023-12-27 00:18:12,279][105620] Updated weights for policy 1, policy_version 1225685 (0.0008) [2023-12-27 00:18:12,350][105620] Updated weights for policy 1, policy_version 1225695 (0.0009) [2023-12-27 00:18:12,701][105692] Updated weights for policy 0, policy_version 1224531 (0.0010) [2023-12-27 00:18:12,764][105692] Updated weights for policy 0, policy_version 1224541 (0.0011) [2023-12-27 00:18:12,819][105692] Updated weights for policy 0, policy_version 1224551 (0.0008) [2023-12-27 00:18:13,068][105620] Updated weights for policy 1, policy_version 1225705 (0.0009) [2023-12-27 00:18:13,127][105620] Updated weights for policy 1, policy_version 1225715 (0.0005) [2023-12-27 00:18:13,183][105620] Updated weights for policy 1, policy_version 1225725 (0.0005) [2023-12-27 00:18:13,549][105692] Updated weights for policy 0, policy_version 1224561 (0.0011) [2023-12-27 00:18:13,604][105692] Updated weights for policy 0, policy_version 1224571 (0.0010) [2023-12-27 00:18:13,665][105692] Updated weights for policy 0, policy_version 1224582 (0.0009) [2023-12-27 00:18:13,721][105620] Updated weights for policy 1, policy_version 1225735 (0.0006) [2023-12-27 00:18:13,727][105692] Updated weights for policy 0, policy_version 1224592 (0.0007) [2023-12-27 00:18:13,772][105620] Updated weights for policy 1, policy_version 1225745 (0.0010) [2023-12-27 00:18:13,834][105620] Updated weights for policy 1, policy_version 1225755 (0.0011) [2023-12-27 00:18:14,451][105692] Updated weights for policy 0, policy_version 1224602 (0.0008) [2023-12-27 00:18:14,507][105692] Updated weights for policy 0, policy_version 1224612 (0.0008) [2023-12-27 00:18:14,568][105692] Updated weights for policy 0, policy_version 1224622 (0.0007) [2023-12-27 00:18:14,571][105620] Updated weights for policy 1, policy_version 1225765 (0.0011) [2023-12-27 00:18:14,635][105620] Updated weights for policy 1, policy_version 1225775 (0.0010) [2023-12-27 00:18:14,693][105620] Updated weights for policy 1, policy_version 1225785 (0.0010) [2023-12-27 00:18:15,329][105692] Updated weights for policy 0, policy_version 1224632 (0.0009) [2023-12-27 00:18:15,390][105692] Updated weights for policy 0, policy_version 1224642 (0.0007) [2023-12-27 00:18:15,440][105692] Updated weights for policy 0, policy_version 1224652 (0.0008) [2023-12-27 00:18:15,460][105620] Updated weights for policy 1, policy_version 1225795 (0.0010) [2023-12-27 00:18:15,525][105620] Updated weights for policy 1, policy_version 1225805 (0.0008) [2023-12-27 00:18:15,594][105620] Updated weights for policy 1, policy_version 1225815 (0.0005) [2023-12-27 00:18:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 627417088. Throughput: 0: 9997.3, 1: 9518.7. Samples: 627390888. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:18:16,063][104569] Avg episode reward: [(0, '8083.192'), (1, '9169.126')] [2023-12-27 00:18:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001225824_313851904.pth... [2023-12-27 00:18:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001224656_313565184.pth... [2023-12-27 00:18:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001224704_313565184.pth [2023-12-27 00:18:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001223504_313270272.pth [2023-12-27 00:18:16,244][105620] Updated weights for policy 1, policy_version 1225825 (0.0010) [2023-12-27 00:18:16,267][105692] Updated weights for policy 0, policy_version 1224662 (0.0009) [2023-12-27 00:18:16,300][105620] Updated weights for policy 1, policy_version 1225835 (0.0011) [2023-12-27 00:18:16,320][105692] Updated weights for policy 0, policy_version 1224672 (0.0011) [2023-12-27 00:18:16,359][105620] Updated weights for policy 1, policy_version 1225845 (0.0010) [2023-12-27 00:18:16,372][105692] Updated weights for policy 0, policy_version 1224682 (0.0011) [2023-12-27 00:18:16,417][105620] Updated weights for policy 1, policy_version 1225855 (0.0010) [2023-12-27 00:18:16,952][105692] Updated weights for policy 0, policy_version 1224692 (0.0010) [2023-12-27 00:18:17,000][105692] Updated weights for policy 0, policy_version 1224702 (0.0010) [2023-12-27 00:18:17,053][105692] Updated weights for policy 0, policy_version 1224712 (0.0010) [2023-12-27 00:18:17,155][105620] Updated weights for policy 1, policy_version 1225865 (0.0010) [2023-12-27 00:18:17,209][105620] Updated weights for policy 1, policy_version 1225875 (0.0010) [2023-12-27 00:18:17,277][105620] Updated weights for policy 1, policy_version 1225885 (0.0010) [2023-12-27 00:18:17,732][105692] Updated weights for policy 0, policy_version 1224722 (0.0010) [2023-12-27 00:18:17,783][105692] Updated weights for policy 0, policy_version 1224732 (0.0006) [2023-12-27 00:18:17,838][105692] Updated weights for policy 0, policy_version 1224743 (0.0010) [2023-12-27 00:18:17,853][105585] KL-divergence is very high: 124.9492 [2023-12-27 00:18:17,943][105620] Updated weights for policy 1, policy_version 1225895 (0.0010) [2023-12-27 00:18:18,003][105620] Updated weights for policy 1, policy_version 1225905 (0.0010) [2023-12-27 00:18:18,054][105620] Updated weights for policy 1, policy_version 1225915 (0.0005) [2023-12-27 00:18:18,503][105692] Updated weights for policy 0, policy_version 1224753 (0.0007) [2023-12-27 00:18:18,555][105692] Updated weights for policy 0, policy_version 1224763 (0.0009) [2023-12-27 00:18:18,621][105692] Updated weights for policy 0, policy_version 1224773 (0.0010) [2023-12-27 00:18:18,687][105692] Updated weights for policy 0, policy_version 1224783 (0.0009) [2023-12-27 00:18:18,709][105620] Updated weights for policy 1, policy_version 1225925 (0.0005) [2023-12-27 00:18:18,776][105620] Updated weights for policy 1, policy_version 1225935 (0.0008) [2023-12-27 00:18:18,841][105620] Updated weights for policy 1, policy_version 1225945 (0.0009) [2023-12-27 00:18:19,441][105620] Updated weights for policy 1, policy_version 1225955 (0.0008) [2023-12-27 00:18:19,493][105692] Updated weights for policy 0, policy_version 1224793 (0.0011) [2023-12-27 00:18:19,498][105620] Updated weights for policy 1, policy_version 1225965 (0.0011) [2023-12-27 00:18:19,552][105692] Updated weights for policy 0, policy_version 1224803 (0.0011) [2023-12-27 00:18:19,562][105620] Updated weights for policy 1, policy_version 1225975 (0.0011) [2023-12-27 00:18:19,608][105692] Updated weights for policy 0, policy_version 1224813 (0.0011) [2023-12-27 00:18:20,326][105692] Updated weights for policy 0, policy_version 1224823 (0.0011) [2023-12-27 00:18:20,375][105692] Updated weights for policy 0, policy_version 1224833 (0.0010) [2023-12-27 00:18:20,391][105620] Updated weights for policy 1, policy_version 1225985 (0.0010) [2023-12-27 00:18:20,427][105692] Updated weights for policy 0, policy_version 1224843 (0.0007) [2023-12-27 00:18:20,452][105620] Updated weights for policy 1, policy_version 1225995 (0.0009) [2023-12-27 00:18:20,503][105620] Updated weights for policy 1, policy_version 1226005 (0.0009) [2023-12-27 00:18:20,559][105620] Updated weights for policy 1, policy_version 1226015 (0.0008) [2023-12-27 00:18:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 627515392. Throughput: 0: 9922.3, 1: 9658.9. Samples: 627508740. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:18:21,062][104569] Avg episode reward: [(0, '7811.590'), (1, '9168.423')] [2023-12-27 00:18:21,075][105692] Updated weights for policy 0, policy_version 1224853 (0.0011) [2023-12-27 00:18:21,135][105692] Updated weights for policy 0, policy_version 1224863 (0.0011) [2023-12-27 00:18:21,203][105692] Updated weights for policy 0, policy_version 1224873 (0.0011) [2023-12-27 00:18:21,430][105620] Updated weights for policy 1, policy_version 1226025 (0.0009) [2023-12-27 00:18:21,492][105620] Updated weights for policy 1, policy_version 1226035 (0.0010) [2023-12-27 00:18:21,547][105620] Updated weights for policy 1, policy_version 1226045 (0.0009) [2023-12-27 00:18:21,939][105692] Updated weights for policy 0, policy_version 1224883 (0.0010) [2023-12-27 00:18:22,001][105692] Updated weights for policy 0, policy_version 1224893 (0.0010) [2023-12-27 00:18:22,064][105692] Updated weights for policy 0, policy_version 1224903 (0.0009) [2023-12-27 00:18:22,334][105620] Updated weights for policy 1, policy_version 1226055 (0.0010) [2023-12-27 00:18:22,401][105620] Updated weights for policy 1, policy_version 1226065 (0.0009) [2023-12-27 00:18:22,466][105620] Updated weights for policy 1, policy_version 1226075 (0.0010) [2023-12-27 00:18:22,849][105692] Updated weights for policy 0, policy_version 1224913 (0.0009) [2023-12-27 00:18:22,907][105692] Updated weights for policy 0, policy_version 1224923 (0.0009) [2023-12-27 00:18:22,969][105692] Updated weights for policy 0, policy_version 1224933 (0.0009) [2023-12-27 00:18:23,031][105692] Updated weights for policy 0, policy_version 1224943 (0.0010) [2023-12-27 00:18:23,135][105620] Updated weights for policy 1, policy_version 1226085 (0.0009) [2023-12-27 00:18:23,189][105620] Updated weights for policy 1, policy_version 1226095 (0.0009) [2023-12-27 00:18:23,241][105620] Updated weights for policy 1, policy_version 1226105 (0.0009) [2023-12-27 00:18:23,808][105692] Updated weights for policy 0, policy_version 1224953 (0.0009) [2023-12-27 00:18:23,848][105585] KL-divergence is very high: 132.1694 [2023-12-27 00:18:23,853][105585] KL-divergence is very high: 150.5476 [2023-12-27 00:18:23,864][105692] Updated weights for policy 0, policy_version 1224963 (0.0009) [2023-12-27 00:18:23,881][105585] KL-divergence is very high: 109.6745 [2023-12-27 00:18:23,895][105585] KL-divergence is very high: 162.2486 [2023-12-27 00:18:23,902][105585] KL-divergence is very high: 174.2953 [2023-12-27 00:18:23,927][105692] Updated weights for policy 0, policy_version 1224973 (0.0009) [2023-12-27 00:18:23,935][105585] KL-divergence is very high: 105.1935 [2023-12-27 00:18:23,948][105620] Updated weights for policy 1, policy_version 1226115 (0.0007) [2023-12-27 00:18:24,004][105620] Updated weights for policy 1, policy_version 1226125 (0.0005) [2023-12-27 00:18:24,063][105620] Updated weights for policy 1, policy_version 1226135 (0.0005) [2023-12-27 00:18:24,668][105620] Updated weights for policy 1, policy_version 1226145 (0.0005) [2023-12-27 00:18:24,719][105620] Updated weights for policy 1, policy_version 1226155 (0.0005) [2023-12-27 00:18:24,767][105620] Updated weights for policy 1, policy_version 1226165 (0.0008) [2023-12-27 00:18:24,786][105692] Updated weights for policy 0, policy_version 1224983 (0.0007) [2023-12-27 00:18:24,821][105620] Updated weights for policy 1, policy_version 1226175 (0.0006) [2023-12-27 00:18:24,840][105692] Updated weights for policy 0, policy_version 1224993 (0.0007) [2023-12-27 00:18:24,894][105692] Updated weights for policy 0, policy_version 1225003 (0.0010) [2023-12-27 00:18:25,401][105620] Updated weights for policy 1, policy_version 1226185 (0.0006) [2023-12-27 00:18:25,464][105620] Updated weights for policy 1, policy_version 1226195 (0.0007) [2023-12-27 00:18:25,513][105620] Updated weights for policy 1, policy_version 1226205 (0.0005) [2023-12-27 00:18:25,776][105692] Updated weights for policy 0, policy_version 1225014 (0.0009) [2023-12-27 00:18:25,832][105692] Updated weights for policy 0, policy_version 1225024 (0.0009) [2023-12-27 00:18:25,898][105692] Updated weights for policy 0, policy_version 1225034 (0.0009) [2023-12-27 00:18:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 627613696. Throughput: 0: 9790.2, 1: 9691.9. Samples: 627623360. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:18:26,062][104569] Avg episode reward: [(0, '7453.361'), (1, '9261.038')] [2023-12-27 00:18:26,153][105620] Updated weights for policy 1, policy_version 1226215 (0.0008) [2023-12-27 00:18:26,205][105620] Updated weights for policy 1, policy_version 1226225 (0.0009) [2023-12-27 00:18:26,257][105620] Updated weights for policy 1, policy_version 1226235 (0.0007) [2023-12-27 00:18:26,691][105692] Updated weights for policy 0, policy_version 1225044 (0.0009) [2023-12-27 00:18:26,745][105692] Updated weights for policy 0, policy_version 1225056 (0.0010) [2023-12-27 00:18:26,804][105692] Updated weights for policy 0, policy_version 1225066 (0.0011) [2023-12-27 00:18:26,866][105620] Updated weights for policy 1, policy_version 1226245 (0.0008) [2023-12-27 00:18:26,914][105620] Updated weights for policy 1, policy_version 1226255 (0.0010) [2023-12-27 00:18:26,961][105620] Updated weights for policy 1, policy_version 1226265 (0.0010) [2023-12-27 00:18:27,589][105620] Updated weights for policy 1, policy_version 1226275 (0.0010) [2023-12-27 00:18:27,646][105620] Updated weights for policy 1, policy_version 1226285 (0.0010) [2023-12-27 00:18:27,652][105692] Updated weights for policy 0, policy_version 1225077 (0.0008) [2023-12-27 00:18:27,698][105692] Updated weights for policy 0, policy_version 1225087 (0.0007) [2023-12-27 00:18:27,703][105620] Updated weights for policy 1, policy_version 1226295 (0.0010) [2023-12-27 00:18:27,741][105692] Updated weights for policy 0, policy_version 1225097 (0.0008) [2023-12-27 00:18:28,339][105620] Updated weights for policy 1, policy_version 1226305 (0.0010) [2023-12-27 00:18:28,407][105620] Updated weights for policy 1, policy_version 1226315 (0.0006) [2023-12-27 00:18:28,468][105620] Updated weights for policy 1, policy_version 1226325 (0.0006) [2023-12-27 00:18:28,534][105620] Updated weights for policy 1, policy_version 1226335 (0.0005) [2023-12-27 00:18:28,595][105692] Updated weights for policy 0, policy_version 1225107 (0.0008) [2023-12-27 00:18:28,654][105692] Updated weights for policy 0, policy_version 1225117 (0.0010) [2023-12-27 00:18:28,714][105692] Updated weights for policy 0, policy_version 1225128 (0.0010) [2023-12-27 00:18:29,126][105620] Updated weights for policy 1, policy_version 1226345 (0.0008) [2023-12-27 00:18:29,177][105620] Updated weights for policy 1, policy_version 1226355 (0.0009) [2023-12-27 00:18:29,232][105620] Updated weights for policy 1, policy_version 1226365 (0.0009) [2023-12-27 00:18:29,524][105692] Updated weights for policy 0, policy_version 1225138 (0.0010) [2023-12-27 00:18:29,588][105692] Updated weights for policy 0, policy_version 1225148 (0.0010) [2023-12-27 00:18:29,650][105692] Updated weights for policy 0, policy_version 1225158 (0.0009) [2023-12-27 00:18:29,716][105692] Updated weights for policy 0, policy_version 1225168 (0.0008) [2023-12-27 00:18:29,955][105620] Updated weights for policy 1, policy_version 1226375 (0.0008) [2023-12-27 00:18:30,009][105620] Updated weights for policy 1, policy_version 1226385 (0.0008) [2023-12-27 00:18:30,053][105620] Updated weights for policy 1, policy_version 1226395 (0.0008) [2023-12-27 00:18:30,502][105692] Updated weights for policy 0, policy_version 1225178 (0.0008) [2023-12-27 00:18:30,556][105692] Updated weights for policy 0, policy_version 1225188 (0.0007) [2023-12-27 00:18:30,614][105692] Updated weights for policy 0, policy_version 1225198 (0.0008) [2023-12-27 00:18:30,741][105620] Updated weights for policy 1, policy_version 1226405 (0.0008) [2023-12-27 00:18:30,799][105620] Updated weights for policy 1, policy_version 1226415 (0.0010) [2023-12-27 00:18:30,857][105620] Updated weights for policy 1, policy_version 1226425 (0.0010) [2023-12-27 00:18:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 627712000. Throughput: 0: 9704.0, 1: 9843.0. Samples: 627682184. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:18:31,063][104569] Avg episode reward: [(0, '7452.889'), (1, '9170.180')] [2023-12-27 00:18:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001226432_314007552.pth... [2023-12-27 00:18:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001225200_313704448.pth... [2023-12-27 00:18:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001224112_313425920.pth [2023-12-27 00:18:31,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001225248_313704448.pth [2023-12-27 00:18:31,308][105692] Updated weights for policy 0, policy_version 1225208 (0.0009) [2023-12-27 00:18:31,370][105692] Updated weights for policy 0, policy_version 1225218 (0.0009) [2023-12-27 00:18:31,425][105692] Updated weights for policy 0, policy_version 1225228 (0.0009) [2023-12-27 00:18:31,628][105620] Updated weights for policy 1, policy_version 1226435 (0.0009) [2023-12-27 00:18:31,676][105620] Updated weights for policy 1, policy_version 1226445 (0.0009) [2023-12-27 00:18:31,740][105620] Updated weights for policy 1, policy_version 1226455 (0.0009) [2023-12-27 00:18:32,222][105692] Updated weights for policy 0, policy_version 1225238 (0.0009) [2023-12-27 00:18:32,280][105692] Updated weights for policy 0, policy_version 1225248 (0.0008) [2023-12-27 00:18:32,333][105692] Updated weights for policy 0, policy_version 1225258 (0.0009) [2023-12-27 00:18:32,390][105620] Updated weights for policy 1, policy_version 1226465 (0.0009) [2023-12-27 00:18:32,446][105620] Updated weights for policy 1, policy_version 1226475 (0.0010) [2023-12-27 00:18:32,495][105620] Updated weights for policy 1, policy_version 1226486 (0.0010) [2023-12-27 00:18:32,545][105620] Updated weights for policy 1, policy_version 1226496 (0.0009) [2023-12-27 00:18:33,082][105692] Updated weights for policy 0, policy_version 1225268 (0.0009) [2023-12-27 00:18:33,147][105692] Updated weights for policy 0, policy_version 1225278 (0.0010) [2023-12-27 00:18:33,211][105692] Updated weights for policy 0, policy_version 1225288 (0.0006) [2023-12-27 00:18:33,265][105620] Updated weights for policy 1, policy_version 1226506 (0.0005) [2023-12-27 00:18:33,320][105620] Updated weights for policy 1, policy_version 1226516 (0.0005) [2023-12-27 00:18:33,387][105620] Updated weights for policy 1, policy_version 1226526 (0.0008) [2023-12-27 00:18:33,829][105692] Updated weights for policy 0, policy_version 1225298 (0.0006) [2023-12-27 00:18:33,877][105692] Updated weights for policy 0, policy_version 1225308 (0.0005) [2023-12-27 00:18:33,929][105692] Updated weights for policy 0, policy_version 1225318 (0.0006) [2023-12-27 00:18:33,984][105692] Updated weights for policy 0, policy_version 1225328 (0.0009) [2023-12-27 00:18:34,097][105620] Updated weights for policy 1, policy_version 1226536 (0.0009) [2023-12-27 00:18:34,157][105620] Updated weights for policy 1, policy_version 1226546 (0.0008) [2023-12-27 00:18:34,217][105620] Updated weights for policy 1, policy_version 1226556 (0.0008) [2023-12-27 00:18:34,668][105692] Updated weights for policy 0, policy_version 1225338 (0.0010) [2023-12-27 00:18:34,703][105585] KL-divergence is very high: 114.0306 [2023-12-27 00:18:34,724][105692] Updated weights for policy 0, policy_version 1225348 (0.0010) [2023-12-27 00:18:34,725][105585] KL-divergence is very high: 102.2077 [2023-12-27 00:18:34,752][105585] KL-divergence is very high: 131.6027 [2023-12-27 00:18:34,780][105585] KL-divergence is very high: 103.2279 [2023-12-27 00:18:34,795][105692] Updated weights for policy 0, policy_version 1225358 (0.0010) [2023-12-27 00:18:34,987][105620] Updated weights for policy 1, policy_version 1226566 (0.0008) [2023-12-27 00:18:35,052][105620] Updated weights for policy 1, policy_version 1226576 (0.0009) [2023-12-27 00:18:35,119][105620] Updated weights for policy 1, policy_version 1226586 (0.0008) [2023-12-27 00:18:35,458][105692] Updated weights for policy 0, policy_version 1225368 (0.0006) [2023-12-27 00:18:35,518][105692] Updated weights for policy 0, policy_version 1225378 (0.0010) [2023-12-27 00:18:35,580][105692] Updated weights for policy 0, policy_version 1225388 (0.0010) [2023-12-27 00:18:35,833][105620] Updated weights for policy 1, policy_version 1226596 (0.0008) [2023-12-27 00:18:35,892][105620] Updated weights for policy 1, policy_version 1226606 (0.0005) [2023-12-27 00:18:35,946][105620] Updated weights for policy 1, policy_version 1226616 (0.0006) [2023-12-27 00:18:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 627810304. Throughput: 0: 9551.2, 1: 9985.2. Samples: 627797408. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:18:36,062][104569] Avg episode reward: [(0, '7723.393'), (1, '9169.684')] [2023-12-27 00:18:36,297][105692] Updated weights for policy 0, policy_version 1225398 (0.0010) [2023-12-27 00:18:36,364][105692] Updated weights for policy 0, policy_version 1225408 (0.0010) [2023-12-27 00:18:36,423][105692] Updated weights for policy 0, policy_version 1225418 (0.0010) [2023-12-27 00:18:36,546][105620] Updated weights for policy 1, policy_version 1226626 (0.0005) [2023-12-27 00:18:36,612][105620] Updated weights for policy 1, policy_version 1226636 (0.0006) [2023-12-27 00:18:36,675][105620] Updated weights for policy 1, policy_version 1226646 (0.0007) [2023-12-27 00:18:36,732][105620] Updated weights for policy 1, policy_version 1226656 (0.0005) [2023-12-27 00:18:37,167][105692] Updated weights for policy 0, policy_version 1225428 (0.0009) [2023-12-27 00:18:37,218][105692] Updated weights for policy 0, policy_version 1225438 (0.0010) [2023-12-27 00:18:37,274][105692] Updated weights for policy 0, policy_version 1225448 (0.0010) [2023-12-27 00:18:37,346][105620] Updated weights for policy 1, policy_version 1226666 (0.0007) [2023-12-27 00:18:37,401][105620] Updated weights for policy 1, policy_version 1226676 (0.0008) [2023-12-27 00:18:37,454][105620] Updated weights for policy 1, policy_version 1226686 (0.0009) [2023-12-27 00:18:38,020][105692] Updated weights for policy 0, policy_version 1225458 (0.0010) [2023-12-27 00:18:38,067][105692] Updated weights for policy 0, policy_version 1225468 (0.0010) [2023-12-27 00:18:38,125][105692] Updated weights for policy 0, policy_version 1225478 (0.0010) [2023-12-27 00:18:38,184][105692] Updated weights for policy 0, policy_version 1225488 (0.0010) [2023-12-27 00:18:38,209][105620] Updated weights for policy 1, policy_version 1226696 (0.0008) [2023-12-27 00:18:38,260][105620] Updated weights for policy 1, policy_version 1226706 (0.0008) [2023-12-27 00:18:38,308][105620] Updated weights for policy 1, policy_version 1226716 (0.0008) [2023-12-27 00:18:38,941][105692] Updated weights for policy 0, policy_version 1225498 (0.0010) [2023-12-27 00:18:38,987][105620] Updated weights for policy 1, policy_version 1226726 (0.0008) [2023-12-27 00:18:38,998][105692] Updated weights for policy 0, policy_version 1225508 (0.0007) [2023-12-27 00:18:39,041][105620] Updated weights for policy 1, policy_version 1226736 (0.0009) [2023-12-27 00:18:39,057][105692] Updated weights for policy 0, policy_version 1225518 (0.0010) [2023-12-27 00:18:39,104][105620] Updated weights for policy 1, policy_version 1226746 (0.0006) [2023-12-27 00:18:39,776][105692] Updated weights for policy 0, policy_version 1225528 (0.0008) [2023-12-27 00:18:39,838][105692] Updated weights for policy 0, policy_version 1225538 (0.0006) [2023-12-27 00:18:39,852][105620] Updated weights for policy 1, policy_version 1226756 (0.0008) [2023-12-27 00:18:39,904][105692] Updated weights for policy 0, policy_version 1225548 (0.0007) [2023-12-27 00:18:39,919][105620] Updated weights for policy 1, policy_version 1226766 (0.0010) [2023-12-27 00:18:39,984][105620] Updated weights for policy 1, policy_version 1226776 (0.0009) [2023-12-27 00:18:40,548][105692] Updated weights for policy 0, policy_version 1225558 (0.0008) [2023-12-27 00:18:40,617][105692] Updated weights for policy 0, policy_version 1225568 (0.0008) [2023-12-27 00:18:40,680][105620] Updated weights for policy 1, policy_version 1226786 (0.0008) [2023-12-27 00:18:40,681][105692] Updated weights for policy 0, policy_version 1225578 (0.0008) [2023-12-27 00:18:40,749][105620] Updated weights for policy 1, policy_version 1226796 (0.0008) [2023-12-27 00:18:40,815][105620] Updated weights for policy 1, policy_version 1226806 (0.0009) [2023-12-27 00:18:40,875][105620] Updated weights for policy 1, policy_version 1226816 (0.0008) [2023-12-27 00:18:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 627908608. Throughput: 0: 9543.6, 1: 9944.8. Samples: 627915412. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:18:41,062][104569] Avg episode reward: [(0, '7993.664'), (1, '9261.044')] [2023-12-27 00:18:41,410][105692] Updated weights for policy 0, policy_version 1225588 (0.0009) [2023-12-27 00:18:41,463][105692] Updated weights for policy 0, policy_version 1225598 (0.0009) [2023-12-27 00:18:41,532][105692] Updated weights for policy 0, policy_version 1225610 (0.0010) [2023-12-27 00:18:41,581][105620] Updated weights for policy 1, policy_version 1226826 (0.0008) [2023-12-27 00:18:41,638][105620] Updated weights for policy 1, policy_version 1226836 (0.0010) [2023-12-27 00:18:41,688][105620] Updated weights for policy 1, policy_version 1226846 (0.0010) [2023-12-27 00:18:42,368][105692] Updated weights for policy 0, policy_version 1225620 (0.0008) [2023-12-27 00:18:42,432][105692] Updated weights for policy 0, policy_version 1225630 (0.0008) [2023-12-27 00:18:42,469][105620] Updated weights for policy 1, policy_version 1226856 (0.0007) [2023-12-27 00:18:42,494][105692] Updated weights for policy 0, policy_version 1225640 (0.0007) [2023-12-27 00:18:42,534][105620] Updated weights for policy 1, policy_version 1226866 (0.0007) [2023-12-27 00:18:42,585][105620] Updated weights for policy 1, policy_version 1226876 (0.0008) [2023-12-27 00:18:43,148][105692] Updated weights for policy 0, policy_version 1225650 (0.0006) [2023-12-27 00:18:43,200][105692] Updated weights for policy 0, policy_version 1225660 (0.0007) [2023-12-27 00:18:43,256][105692] Updated weights for policy 0, policy_version 1225670 (0.0007) [2023-12-27 00:18:43,301][105692] Updated weights for policy 0, policy_version 1225680 (0.0008) [2023-12-27 00:18:43,396][105620] Updated weights for policy 1, policy_version 1226886 (0.0010) [2023-12-27 00:18:43,445][105620] Updated weights for policy 1, policy_version 1226896 (0.0010) [2023-12-27 00:18:43,493][105620] Updated weights for policy 1, policy_version 1226906 (0.0010) [2023-12-27 00:18:44,046][105692] Updated weights for policy 0, policy_version 1225690 (0.0006) [2023-12-27 00:18:44,102][105692] Updated weights for policy 0, policy_version 1225700 (0.0009) [2023-12-27 00:18:44,162][105692] Updated weights for policy 0, policy_version 1225710 (0.0008) [2023-12-27 00:18:44,210][105620] Updated weights for policy 1, policy_version 1226916 (0.0010) [2023-12-27 00:18:44,262][105620] Updated weights for policy 1, policy_version 1226926 (0.0010) [2023-12-27 00:18:44,315][105620] Updated weights for policy 1, policy_version 1226936 (0.0010) [2023-12-27 00:18:44,831][105692] Updated weights for policy 0, policy_version 1225720 (0.0008) [2023-12-27 00:18:44,900][105692] Updated weights for policy 0, policy_version 1225730 (0.0008) [2023-12-27 00:18:44,967][105692] Updated weights for policy 0, policy_version 1225740 (0.0009) [2023-12-27 00:18:45,041][105620] Updated weights for policy 1, policy_version 1226946 (0.0010) [2023-12-27 00:18:45,113][105620] Updated weights for policy 1, policy_version 1226956 (0.0009) [2023-12-27 00:18:45,187][105620] Updated weights for policy 1, policy_version 1226966 (0.0009) [2023-12-27 00:18:45,255][105620] Updated weights for policy 1, policy_version 1226976 (0.0010) [2023-12-27 00:18:45,675][105692] Updated weights for policy 0, policy_version 1225750 (0.0009) [2023-12-27 00:18:45,740][105692] Updated weights for policy 0, policy_version 1225760 (0.0008) [2023-12-27 00:18:45,811][105692] Updated weights for policy 0, policy_version 1225770 (0.0007) [2023-12-27 00:18:45,971][105620] Updated weights for policy 1, policy_version 1226986 (0.0008) [2023-12-27 00:18:46,022][105620] Updated weights for policy 1, policy_version 1226996 (0.0009) [2023-12-27 00:18:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 627998720. Throughput: 0: 9458.3, 1: 9918.2. Samples: 627971236. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:18:46,062][104569] Avg episode reward: [(0, '7544.114'), (1, '8986.896')] [2023-12-27 00:18:46,066][105620] Updated weights for policy 1, policy_version 1227006 (0.0007) [2023-12-27 00:18:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001225776_313851904.pth... [2023-12-27 00:18:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001224656_313565184.pth [2023-12-27 00:18:46,076][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001227008_314155008.pth... [2023-12-27 00:18:46,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001225824_313851904.pth [2023-12-27 00:18:46,590][105692] Updated weights for policy 0, policy_version 1225780 (0.0007) [2023-12-27 00:18:46,643][105692] Updated weights for policy 0, policy_version 1225790 (0.0009) [2023-12-27 00:18:46,690][105692] Updated weights for policy 0, policy_version 1225800 (0.0010) [2023-12-27 00:18:46,737][105620] Updated weights for policy 1, policy_version 1227016 (0.0005) [2023-12-27 00:18:46,792][105620] Updated weights for policy 1, policy_version 1227026 (0.0009) [2023-12-27 00:18:46,846][105620] Updated weights for policy 1, policy_version 1227036 (0.0008) [2023-12-27 00:18:47,304][105692] Updated weights for policy 0, policy_version 1225810 (0.0010) [2023-12-27 00:18:47,368][105692] Updated weights for policy 0, policy_version 1225820 (0.0008) [2023-12-27 00:18:47,431][105692] Updated weights for policy 0, policy_version 1225830 (0.0007) [2023-12-27 00:18:47,481][105692] Updated weights for policy 0, policy_version 1225840 (0.0007) [2023-12-27 00:18:47,567][105620] Updated weights for policy 1, policy_version 1227046 (0.0009) [2023-12-27 00:18:47,619][105620] Updated weights for policy 1, policy_version 1227056 (0.0007) [2023-12-27 00:18:47,675][105620] Updated weights for policy 1, policy_version 1227066 (0.0005) [2023-12-27 00:18:48,148][105692] Updated weights for policy 0, policy_version 1225850 (0.0010) [2023-12-27 00:18:48,196][105692] Updated weights for policy 0, policy_version 1225860 (0.0010) [2023-12-27 00:18:48,240][105692] Updated weights for policy 0, policy_version 1225870 (0.0010) [2023-12-27 00:18:48,254][105620] Updated weights for policy 1, policy_version 1227076 (0.0005) [2023-12-27 00:18:48,302][105620] Updated weights for policy 1, policy_version 1227086 (0.0006) [2023-12-27 00:18:48,367][105620] Updated weights for policy 1, policy_version 1227096 (0.0007) [2023-12-27 00:18:48,993][105692] Updated weights for policy 0, policy_version 1225880 (0.0011) [2023-12-27 00:18:49,060][105692] Updated weights for policy 0, policy_version 1225890 (0.0011) [2023-12-27 00:18:49,116][105692] Updated weights for policy 0, policy_version 1225900 (0.0010) [2023-12-27 00:18:49,138][105620] Updated weights for policy 1, policy_version 1227106 (0.0009) [2023-12-27 00:18:49,200][105620] Updated weights for policy 1, policy_version 1227116 (0.0008) [2023-12-27 00:18:49,269][105620] Updated weights for policy 1, policy_version 1227126 (0.0009) [2023-12-27 00:18:49,334][105620] Updated weights for policy 1, policy_version 1227136 (0.0008) [2023-12-27 00:18:49,871][105692] Updated weights for policy 0, policy_version 1225910 (0.0011) [2023-12-27 00:18:49,921][105692] Updated weights for policy 0, policy_version 1225920 (0.0009) [2023-12-27 00:18:49,971][105620] Updated weights for policy 1, policy_version 1227146 (0.0008) [2023-12-27 00:18:49,987][105692] Updated weights for policy 0, policy_version 1225930 (0.0010) [2023-12-27 00:18:50,036][105620] Updated weights for policy 1, policy_version 1227156 (0.0007) [2023-12-27 00:18:50,090][105620] Updated weights for policy 1, policy_version 1227166 (0.0009) [2023-12-27 00:18:50,620][105692] Updated weights for policy 0, policy_version 1225940 (0.0010) [2023-12-27 00:18:50,688][105692] Updated weights for policy 0, policy_version 1225950 (0.0010) [2023-12-27 00:18:50,754][105692] Updated weights for policy 0, policy_version 1225960 (0.0009) [2023-12-27 00:18:50,890][105620] Updated weights for policy 1, policy_version 1227176 (0.0006) [2023-12-27 00:18:50,939][105620] Updated weights for policy 1, policy_version 1227186 (0.0005) [2023-12-27 00:18:51,005][105620] Updated weights for policy 1, policy_version 1227196 (0.0007) [2023-12-27 00:18:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 628105216. Throughput: 0: 9459.5, 1: 9890.5. Samples: 628090700. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:18:51,063][104569] Avg episode reward: [(0, '7456.837'), (1, '8894.624')] [2023-12-27 00:18:51,544][105692] Updated weights for policy 0, policy_version 1225970 (0.0009) [2023-12-27 00:18:51,590][105620] Updated weights for policy 1, policy_version 1227206 (0.0008) [2023-12-27 00:18:51,609][105692] Updated weights for policy 0, policy_version 1225980 (0.0008) [2023-12-27 00:18:51,652][105620] Updated weights for policy 1, policy_version 1227216 (0.0008) [2023-12-27 00:18:51,672][105692] Updated weights for policy 0, policy_version 1225990 (0.0008) [2023-12-27 00:18:51,713][105620] Updated weights for policy 1, policy_version 1227226 (0.0007) [2023-12-27 00:18:51,737][105692] Updated weights for policy 0, policy_version 1226000 (0.0009) [2023-12-27 00:18:52,435][105692] Updated weights for policy 0, policy_version 1226010 (0.0010) [2023-12-27 00:18:52,476][105620] Updated weights for policy 1, policy_version 1227236 (0.0008) [2023-12-27 00:18:52,497][105692] Updated weights for policy 0, policy_version 1226020 (0.0008) [2023-12-27 00:18:52,541][105620] Updated weights for policy 1, policy_version 1227246 (0.0006) [2023-12-27 00:18:52,556][105692] Updated weights for policy 0, policy_version 1226030 (0.0009) [2023-12-27 00:18:52,607][105620] Updated weights for policy 1, policy_version 1227256 (0.0009) [2023-12-27 00:18:53,250][105620] Updated weights for policy 1, policy_version 1227266 (0.0008) [2023-12-27 00:18:53,313][105620] Updated weights for policy 1, policy_version 1227276 (0.0007) [2023-12-27 00:18:53,331][105692] Updated weights for policy 0, policy_version 1226040 (0.0006) [2023-12-27 00:18:53,370][105620] Updated weights for policy 1, policy_version 1227286 (0.0007) [2023-12-27 00:18:53,398][105692] Updated weights for policy 0, policy_version 1226050 (0.0005) [2023-12-27 00:18:53,427][105620] Updated weights for policy 1, policy_version 1227296 (0.0007) [2023-12-27 00:18:53,464][105692] Updated weights for policy 0, policy_version 1226060 (0.0006) [2023-12-27 00:18:53,980][105692] Updated weights for policy 0, policy_version 1226070 (0.0008) [2023-12-27 00:18:54,036][105692] Updated weights for policy 0, policy_version 1226080 (0.0010) [2023-12-27 00:18:54,074][105620] Updated weights for policy 1, policy_version 1227306 (0.0011) [2023-12-27 00:18:54,096][105692] Updated weights for policy 0, policy_version 1226090 (0.0011) [2023-12-27 00:18:54,139][105620] Updated weights for policy 1, policy_version 1227316 (0.0010) [2023-12-27 00:18:54,196][105620] Updated weights for policy 1, policy_version 1227326 (0.0007) [2023-12-27 00:18:54,765][105692] Updated weights for policy 0, policy_version 1226100 (0.0011) [2023-12-27 00:18:54,772][105620] Updated weights for policy 1, policy_version 1227336 (0.0010) [2023-12-27 00:18:54,814][105692] Updated weights for policy 0, policy_version 1226110 (0.0010) [2023-12-27 00:18:54,824][105620] Updated weights for policy 1, policy_version 1227346 (0.0010) [2023-12-27 00:18:54,866][105692] Updated weights for policy 0, policy_version 1226120 (0.0006) [2023-12-27 00:18:54,890][105620] Updated weights for policy 1, policy_version 1227356 (0.0010) [2023-12-27 00:18:55,450][105692] Updated weights for policy 0, policy_version 1226130 (0.0005) [2023-12-27 00:18:55,521][105692] Updated weights for policy 0, policy_version 1226140 (0.0005) [2023-12-27 00:18:55,583][105692] Updated weights for policy 0, policy_version 1226150 (0.0007) [2023-12-27 00:18:55,646][105692] Updated weights for policy 0, policy_version 1226160 (0.0011) [2023-12-27 00:18:55,650][105620] Updated weights for policy 1, policy_version 1227366 (0.0011) [2023-12-27 00:18:55,700][105620] Updated weights for policy 1, policy_version 1227376 (0.0011) [2023-12-27 00:18:55,752][105620] Updated weights for policy 1, policy_version 1227386 (0.0010) [2023-12-27 00:18:56,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 628203520. Throughput: 0: 9570.1, 1: 9942.8. Samples: 628211492. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:18:56,062][104569] Avg episode reward: [(0, '7727.581'), (1, '9168.494')] [2023-12-27 00:18:56,239][105692] Updated weights for policy 0, policy_version 1226170 (0.0005) [2023-12-27 00:18:56,303][105692] Updated weights for policy 0, policy_version 1226180 (0.0005) [2023-12-27 00:18:56,365][105692] Updated weights for policy 0, policy_version 1226190 (0.0005) [2023-12-27 00:18:56,538][105620] Updated weights for policy 1, policy_version 1227396 (0.0011) [2023-12-27 00:18:56,597][105620] Updated weights for policy 1, policy_version 1227406 (0.0011) [2023-12-27 00:18:56,663][105620] Updated weights for policy 1, policy_version 1227416 (0.0011) [2023-12-27 00:18:56,899][105692] Updated weights for policy 0, policy_version 1226200 (0.0007) [2023-12-27 00:18:56,959][105692] Updated weights for policy 0, policy_version 1226210 (0.0008) [2023-12-27 00:18:57,003][105692] Updated weights for policy 0, policy_version 1226220 (0.0010) [2023-12-27 00:18:57,393][105620] Updated weights for policy 1, policy_version 1227426 (0.0010) [2023-12-27 00:18:57,453][105620] Updated weights for policy 1, policy_version 1227436 (0.0006) [2023-12-27 00:18:57,499][105620] Updated weights for policy 1, policy_version 1227446 (0.0005) [2023-12-27 00:18:57,546][105620] Updated weights for policy 1, policy_version 1227456 (0.0005) [2023-12-27 00:18:57,680][105692] Updated weights for policy 0, policy_version 1226230 (0.0010) [2023-12-27 00:18:57,728][105692] Updated weights for policy 0, policy_version 1226240 (0.0010) [2023-12-27 00:18:57,775][105692] Updated weights for policy 0, policy_version 1226250 (0.0010) [2023-12-27 00:18:58,083][105620] Updated weights for policy 1, policy_version 1227466 (0.0006) [2023-12-27 00:18:58,141][105620] Updated weights for policy 1, policy_version 1227476 (0.0005) [2023-12-27 00:18:58,203][105620] Updated weights for policy 1, policy_version 1227486 (0.0008) [2023-12-27 00:18:58,444][105692] Updated weights for policy 0, policy_version 1226260 (0.0010) [2023-12-27 00:18:58,507][105692] Updated weights for policy 0, policy_version 1226270 (0.0011) [2023-12-27 00:18:58,532][105585] KL-divergence is very high: 102.3586 [2023-12-27 00:18:58,566][105585] KL-divergence is very high: 146.3428 [2023-12-27 00:18:58,582][105692] Updated weights for policy 0, policy_version 1226280 (0.0010) [2023-12-27 00:18:58,595][105585] KL-divergence is very high: 108.2595 [2023-12-27 00:18:58,622][105585] KL-divergence is very high: 141.9075 [2023-12-27 00:18:58,875][105620] Updated weights for policy 1, policy_version 1227496 (0.0008) [2023-12-27 00:18:58,940][105620] Updated weights for policy 1, policy_version 1227506 (0.0009) [2023-12-27 00:18:58,995][105620] Updated weights for policy 1, policy_version 1227516 (0.0007) [2023-12-27 00:18:59,334][105692] Updated weights for policy 0, policy_version 1226290 (0.0011) [2023-12-27 00:18:59,401][105692] Updated weights for policy 0, policy_version 1226300 (0.0011) [2023-12-27 00:18:59,402][105585] KL-divergence is very high: 174.8739 [2023-12-27 00:18:59,454][105585] KL-divergence is very high: 365.0164 [2023-12-27 00:18:59,465][105692] Updated weights for policy 0, policy_version 1226310 (0.0010) [2023-12-27 00:18:59,499][105585] KL-divergence is very high: 118.8454 [2023-12-27 00:18:59,505][105585] KL-divergence is very high: 441.0489 [2023-12-27 00:18:59,529][105692] Updated weights for policy 0, policy_version 1226320 (0.0008) [2023-12-27 00:18:59,779][105620] Updated weights for policy 1, policy_version 1227526 (0.0007) [2023-12-27 00:18:59,835][105620] Updated weights for policy 1, policy_version 1227536 (0.0010) [2023-12-27 00:18:59,889][105620] Updated weights for policy 1, policy_version 1227546 (0.0008) [2023-12-27 00:19:00,214][105692] Updated weights for policy 0, policy_version 1226330 (0.0011) [2023-12-27 00:19:00,266][105692] Updated weights for policy 0, policy_version 1226340 (0.0010) [2023-12-27 00:19:00,321][105692] Updated weights for policy 0, policy_version 1226350 (0.0011) [2023-12-27 00:19:00,593][105620] Updated weights for policy 1, policy_version 1227556 (0.0009) [2023-12-27 00:19:00,645][105620] Updated weights for policy 1, policy_version 1227566 (0.0008) [2023-12-27 00:19:00,692][105620] Updated weights for policy 1, policy_version 1227576 (0.0008) [2023-12-27 00:19:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 628301824. Throughput: 0: 9671.5, 1: 9968.8. Samples: 628274700. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:19:01,063][104569] Avg episode reward: [(0, '7907.686'), (1, '9169.581')] [2023-12-27 00:19:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001227584_314302464.pth... [2023-12-27 00:19:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001226432_314007552.pth [2023-12-27 00:19:01,095][105692] Updated weights for policy 0, policy_version 1226360 (0.0009) [2023-12-27 00:19:01,159][105692] Updated weights for policy 0, policy_version 1226370 (0.0011) [2023-12-27 00:19:01,216][105692] Updated weights for policy 0, policy_version 1226380 (0.0010) [2023-12-27 00:19:01,240][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001226384_314007552.pth... [2023-12-27 00:19:01,245][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001225200_313704448.pth [2023-12-27 00:19:01,476][105620] Updated weights for policy 1, policy_version 1227586 (0.0008) [2023-12-27 00:19:01,539][105620] Updated weights for policy 1, policy_version 1227596 (0.0008) [2023-12-27 00:19:01,601][105620] Updated weights for policy 1, policy_version 1227606 (0.0008) [2023-12-27 00:19:01,667][105620] Updated weights for policy 1, policy_version 1227616 (0.0008) [2023-12-27 00:19:01,967][105692] Updated weights for policy 0, policy_version 1226390 (0.0010) [2023-12-27 00:19:02,015][105692] Updated weights for policy 0, policy_version 1226400 (0.0010) [2023-12-27 00:19:02,065][105692] Updated weights for policy 0, policy_version 1226410 (0.0010) [2023-12-27 00:19:02,424][105620] Updated weights for policy 1, policy_version 1227626 (0.0008) [2023-12-27 00:19:02,484][105620] Updated weights for policy 1, policy_version 1227636 (0.0008) [2023-12-27 00:19:02,532][105620] Updated weights for policy 1, policy_version 1227646 (0.0008) [2023-12-27 00:19:02,835][105692] Updated weights for policy 0, policy_version 1226420 (0.0009) [2023-12-27 00:19:02,879][105692] Updated weights for policy 0, policy_version 1226430 (0.0010) [2023-12-27 00:19:02,924][105692] Updated weights for policy 0, policy_version 1226440 (0.0010) [2023-12-27 00:19:03,265][105620] Updated weights for policy 1, policy_version 1227656 (0.0008) [2023-12-27 00:19:03,309][105620] Updated weights for policy 1, policy_version 1227666 (0.0008) [2023-12-27 00:19:03,360][105620] Updated weights for policy 1, policy_version 1227677 (0.0009) [2023-12-27 00:19:03,640][105692] Updated weights for policy 0, policy_version 1226450 (0.0010) [2023-12-27 00:19:03,687][105692] Updated weights for policy 0, policy_version 1226460 (0.0009) [2023-12-27 00:19:03,734][105692] Updated weights for policy 0, policy_version 1226470 (0.0009) [2023-12-27 00:19:03,781][105692] Updated weights for policy 0, policy_version 1226480 (0.0008) [2023-12-27 00:19:04,220][105620] Updated weights for policy 1, policy_version 1227687 (0.0009) [2023-12-27 00:19:04,291][105620] Updated weights for policy 1, policy_version 1227697 (0.0010) [2023-12-27 00:19:04,363][105620] Updated weights for policy 1, policy_version 1227707 (0.0008) [2023-12-27 00:19:04,404][105692] Updated weights for policy 0, policy_version 1226490 (0.0008) [2023-12-27 00:19:04,466][105692] Updated weights for policy 0, policy_version 1226500 (0.0009) [2023-12-27 00:19:04,514][105692] Updated weights for policy 0, policy_version 1226510 (0.0009) [2023-12-27 00:19:05,142][105620] Updated weights for policy 1, policy_version 1227717 (0.0008) [2023-12-27 00:19:05,192][105692] Updated weights for policy 0, policy_version 1226520 (0.0008) [2023-12-27 00:19:05,202][105620] Updated weights for policy 1, policy_version 1227727 (0.0007) [2023-12-27 00:19:05,253][105692] Updated weights for policy 0, policy_version 1226530 (0.0006) [2023-12-27 00:19:05,270][105620] Updated weights for policy 1, policy_version 1227737 (0.0008) [2023-12-27 00:19:05,320][105692] Updated weights for policy 0, policy_version 1226540 (0.0006) [2023-12-27 00:19:05,895][105692] Updated weights for policy 0, policy_version 1226550 (0.0007) [2023-12-27 00:19:05,952][105692] Updated weights for policy 0, policy_version 1226560 (0.0006) [2023-12-27 00:19:06,016][105692] Updated weights for policy 0, policy_version 1226570 (0.0005) [2023-12-27 00:19:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 628400128. Throughput: 0: 9661.2, 1: 9858.5. Samples: 628387128. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:19:06,062][104569] Avg episode reward: [(0, '7904.045'), (1, '9170.212')] [2023-12-27 00:19:06,096][105620] Updated weights for policy 1, policy_version 1227747 (0.0007) [2023-12-27 00:19:06,161][105620] Updated weights for policy 1, policy_version 1227757 (0.0009) [2023-12-27 00:19:06,223][105620] Updated weights for policy 1, policy_version 1227767 (0.0010) [2023-12-27 00:19:06,671][105692] Updated weights for policy 0, policy_version 1226580 (0.0007) [2023-12-27 00:19:06,728][105692] Updated weights for policy 0, policy_version 1226590 (0.0009) [2023-12-27 00:19:06,790][105692] Updated weights for policy 0, policy_version 1226601 (0.0013) [2023-12-27 00:19:06,915][105620] Updated weights for policy 1, policy_version 1227777 (0.0009) [2023-12-27 00:19:06,972][105620] Updated weights for policy 1, policy_version 1227787 (0.0005) [2023-12-27 00:19:07,023][105620] Updated weights for policy 1, policy_version 1227797 (0.0005) [2023-12-27 00:19:07,070][105620] Updated weights for policy 1, policy_version 1227807 (0.0005) [2023-12-27 00:19:07,622][105692] Updated weights for policy 0, policy_version 1226611 (0.0009) [2023-12-27 00:19:07,678][105692] Updated weights for policy 0, policy_version 1226621 (0.0007) [2023-12-27 00:19:07,743][105692] Updated weights for policy 0, policy_version 1226631 (0.0010) [2023-12-27 00:19:07,767][105620] Updated weights for policy 1, policy_version 1227817 (0.0009) [2023-12-27 00:19:07,813][105620] Updated weights for policy 1, policy_version 1227827 (0.0007) [2023-12-27 00:19:07,867][105620] Updated weights for policy 1, policy_version 1227837 (0.0007) [2023-12-27 00:19:08,392][105692] Updated weights for policy 0, policy_version 1226641 (0.0010) [2023-12-27 00:19:08,458][105692] Updated weights for policy 0, policy_version 1226651 (0.0006) [2023-12-27 00:19:08,520][105692] Updated weights for policy 0, policy_version 1226661 (0.0005) [2023-12-27 00:19:08,582][105692] Updated weights for policy 0, policy_version 1226671 (0.0007) [2023-12-27 00:19:08,730][105620] Updated weights for policy 1, policy_version 1227847 (0.0009) [2023-12-27 00:19:08,791][105620] Updated weights for policy 1, policy_version 1227857 (0.0009) [2023-12-27 00:19:08,847][105620] Updated weights for policy 1, policy_version 1227867 (0.0009) [2023-12-27 00:19:09,106][105692] Updated weights for policy 0, policy_version 1226681 (0.0005) [2023-12-27 00:19:09,162][105692] Updated weights for policy 0, policy_version 1226691 (0.0005) [2023-12-27 00:19:09,214][105692] Updated weights for policy 0, policy_version 1226701 (0.0005) [2023-12-27 00:19:09,713][105620] Updated weights for policy 1, policy_version 1227877 (0.0008) [2023-12-27 00:19:09,780][105620] Updated weights for policy 1, policy_version 1227887 (0.0007) [2023-12-27 00:19:09,823][105692] Updated weights for policy 0, policy_version 1226711 (0.0009) [2023-12-27 00:19:09,846][105620] Updated weights for policy 1, policy_version 1227897 (0.0008) [2023-12-27 00:19:09,888][105692] Updated weights for policy 0, policy_version 1226721 (0.0008) [2023-12-27 00:19:09,952][105692] Updated weights for policy 0, policy_version 1226731 (0.0009) [2023-12-27 00:19:10,550][105620] Updated weights for policy 1, policy_version 1227907 (0.0007) [2023-12-27 00:19:10,611][105620] Updated weights for policy 1, policy_version 1227917 (0.0008) [2023-12-27 00:19:10,666][105620] Updated weights for policy 1, policy_version 1227927 (0.0009) [2023-12-27 00:19:10,705][105692] Updated weights for policy 0, policy_version 1226741 (0.0009) [2023-12-27 00:19:10,767][105692] Updated weights for policy 0, policy_version 1226751 (0.0008) [2023-12-27 00:19:10,830][105692] Updated weights for policy 0, policy_version 1226761 (0.0009) [2023-12-27 00:19:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 628498432. Throughput: 0: 9832.0, 1: 9750.0. Samples: 628504552. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:19:11,062][104569] Avg episode reward: [(0, '7907.938'), (1, '9170.200')] [2023-12-27 00:19:11,336][105620] Updated weights for policy 1, policy_version 1227937 (0.0007) [2023-12-27 00:19:11,413][105620] Updated weights for policy 1, policy_version 1227947 (0.0009) [2023-12-27 00:19:11,470][105620] Updated weights for policy 1, policy_version 1227957 (0.0009) [2023-12-27 00:19:11,531][105620] Updated weights for policy 1, policy_version 1227967 (0.0008) [2023-12-27 00:19:11,719][105692] Updated weights for policy 0, policy_version 1226771 (0.0008) [2023-12-27 00:19:11,785][105692] Updated weights for policy 0, policy_version 1226781 (0.0006) [2023-12-27 00:19:11,844][105692] Updated weights for policy 0, policy_version 1226791 (0.0006) [2023-12-27 00:19:12,236][105620] Updated weights for policy 1, policy_version 1227977 (0.0007) [2023-12-27 00:19:12,303][105620] Updated weights for policy 1, policy_version 1227987 (0.0006) [2023-12-27 00:19:12,367][105620] Updated weights for policy 1, policy_version 1227997 (0.0008) [2023-12-27 00:19:12,524][105692] Updated weights for policy 0, policy_version 1226801 (0.0006) [2023-12-27 00:19:12,588][105692] Updated weights for policy 0, policy_version 1226811 (0.0009) [2023-12-27 00:19:12,660][105692] Updated weights for policy 0, policy_version 1226821 (0.0010) [2023-12-27 00:19:12,732][105692] Updated weights for policy 0, policy_version 1226831 (0.0008) [2023-12-27 00:19:12,967][105620] Updated weights for policy 1, policy_version 1228007 (0.0006) [2023-12-27 00:19:13,016][105620] Updated weights for policy 1, policy_version 1228017 (0.0007) [2023-12-27 00:19:13,082][105620] Updated weights for policy 1, policy_version 1228027 (0.0005) [2023-12-27 00:19:13,417][105692] Updated weights for policy 0, policy_version 1226841 (0.0008) [2023-12-27 00:19:13,480][105692] Updated weights for policy 0, policy_version 1226851 (0.0008) [2023-12-27 00:19:13,542][105692] Updated weights for policy 0, policy_version 1226861 (0.0006) [2023-12-27 00:19:13,743][105620] Updated weights for policy 1, policy_version 1228037 (0.0005) [2023-12-27 00:19:13,807][105620] Updated weights for policy 1, policy_version 1228047 (0.0009) [2023-12-27 00:19:13,853][105620] Updated weights for policy 1, policy_version 1228057 (0.0006) [2023-12-27 00:19:14,112][105692] Updated weights for policy 0, policy_version 1226871 (0.0009) [2023-12-27 00:19:14,165][105692] Updated weights for policy 0, policy_version 1226881 (0.0010) [2023-12-27 00:19:14,224][105692] Updated weights for policy 0, policy_version 1226891 (0.0010) [2023-12-27 00:19:14,486][105620] Updated weights for policy 1, policy_version 1228067 (0.0007) [2023-12-27 00:19:14,547][105620] Updated weights for policy 1, policy_version 1228077 (0.0010) [2023-12-27 00:19:14,606][105620] Updated weights for policy 1, policy_version 1228087 (0.0011) [2023-12-27 00:19:14,983][105692] Updated weights for policy 0, policy_version 1226901 (0.0011) [2023-12-27 00:19:15,043][105692] Updated weights for policy 0, policy_version 1226911 (0.0011) [2023-12-27 00:19:15,098][105692] Updated weights for policy 0, policy_version 1226921 (0.0011) [2023-12-27 00:19:15,263][105620] Updated weights for policy 1, policy_version 1228097 (0.0010) [2023-12-27 00:19:15,326][105620] Updated weights for policy 1, policy_version 1228107 (0.0011) [2023-12-27 00:19:15,389][105620] Updated weights for policy 1, policy_version 1228117 (0.0011) [2023-12-27 00:19:15,449][105620] Updated weights for policy 1, policy_version 1228127 (0.0010) [2023-12-27 00:19:15,798][105692] Updated weights for policy 0, policy_version 1226931 (0.0009) [2023-12-27 00:19:15,861][105692] Updated weights for policy 0, policy_version 1226941 (0.0011) [2023-12-27 00:19:15,920][105692] Updated weights for policy 0, policy_version 1226951 (0.0010) [2023-12-27 00:19:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 628596736. Throughput: 0: 9873.0, 1: 9719.9. Samples: 628563864. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:19:16,062][104569] Avg episode reward: [(0, '7998.458'), (1, '9077.444')] [2023-12-27 00:19:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001228128_314441728.pth... [2023-12-27 00:19:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001226960_314155008.pth... [2023-12-27 00:19:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001227008_314155008.pth [2023-12-27 00:19:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001225776_313851904.pth [2023-12-27 00:19:16,203][105620] Updated weights for policy 1, policy_version 1228137 (0.0010) [2023-12-27 00:19:16,262][105620] Updated weights for policy 1, policy_version 1228147 (0.0010) [2023-12-27 00:19:16,324][105620] Updated weights for policy 1, policy_version 1228157 (0.0010) [2023-12-27 00:19:16,540][105692] Updated weights for policy 0, policy_version 1226961 (0.0009) [2023-12-27 00:19:16,591][105692] Updated weights for policy 0, policy_version 1226971 (0.0011) [2023-12-27 00:19:16,652][105692] Updated weights for policy 0, policy_version 1226981 (0.0011) [2023-12-27 00:19:16,716][105692] Updated weights for policy 0, policy_version 1226991 (0.0011) [2023-12-27 00:19:17,093][105620] Updated weights for policy 1, policy_version 1228167 (0.0010) [2023-12-27 00:19:17,137][105620] Updated weights for policy 1, policy_version 1228177 (0.0010) [2023-12-27 00:19:17,189][105620] Updated weights for policy 1, policy_version 1228187 (0.0010) [2023-12-27 00:19:17,420][105692] Updated weights for policy 0, policy_version 1227001 (0.0010) [2023-12-27 00:19:17,481][105692] Updated weights for policy 0, policy_version 1227011 (0.0010) [2023-12-27 00:19:17,532][105692] Updated weights for policy 0, policy_version 1227021 (0.0010) [2023-12-27 00:19:17,925][105620] Updated weights for policy 1, policy_version 1228197 (0.0008) [2023-12-27 00:19:17,983][105620] Updated weights for policy 1, policy_version 1228207 (0.0010) [2023-12-27 00:19:18,044][105620] Updated weights for policy 1, policy_version 1228217 (0.0010) [2023-12-27 00:19:18,262][105692] Updated weights for policy 0, policy_version 1227031 (0.0009) [2023-12-27 00:19:18,309][105692] Updated weights for policy 0, policy_version 1227041 (0.0010) [2023-12-27 00:19:18,368][105692] Updated weights for policy 0, policy_version 1227051 (0.0008) [2023-12-27 00:19:18,770][105620] Updated weights for policy 1, policy_version 1228227 (0.0010) [2023-12-27 00:19:18,832][105620] Updated weights for policy 1, policy_version 1228237 (0.0009) [2023-12-27 00:19:18,890][105620] Updated weights for policy 1, policy_version 1228247 (0.0009) [2023-12-27 00:19:19,069][105692] Updated weights for policy 0, policy_version 1227061 (0.0007) [2023-12-27 00:19:19,124][105692] Updated weights for policy 0, policy_version 1227071 (0.0009) [2023-12-27 00:19:19,179][105692] Updated weights for policy 0, policy_version 1227081 (0.0007) [2023-12-27 00:19:19,641][105620] Updated weights for policy 1, policy_version 1228257 (0.0009) [2023-12-27 00:19:19,703][105620] Updated weights for policy 1, policy_version 1228267 (0.0010) [2023-12-27 00:19:19,752][105620] Updated weights for policy 1, policy_version 1228277 (0.0008) [2023-12-27 00:19:19,807][105620] Updated weights for policy 1, policy_version 1228287 (0.0008) [2023-12-27 00:19:19,908][105692] Updated weights for policy 0, policy_version 1227091 (0.0009) [2023-12-27 00:19:19,972][105692] Updated weights for policy 0, policy_version 1227101 (0.0009) [2023-12-27 00:19:20,031][105692] Updated weights for policy 0, policy_version 1227111 (0.0009) [2023-12-27 00:19:20,601][105620] Updated weights for policy 1, policy_version 1228297 (0.0008) [2023-12-27 00:19:20,650][105620] Updated weights for policy 1, policy_version 1228307 (0.0008) [2023-12-27 00:19:20,713][105620] Updated weights for policy 1, policy_version 1228317 (0.0006) [2023-12-27 00:19:20,819][105692] Updated weights for policy 0, policy_version 1227121 (0.0009) [2023-12-27 00:19:20,878][105692] Updated weights for policy 0, policy_version 1227131 (0.0009) [2023-12-27 00:19:20,937][105692] Updated weights for policy 0, policy_version 1227141 (0.0009) [2023-12-27 00:19:20,986][105692] Updated weights for policy 0, policy_version 1227151 (0.0009) [2023-12-27 00:19:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 628695040. Throughput: 0: 9945.7, 1: 9693.7. Samples: 628681180. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:19:21,063][104569] Avg episode reward: [(0, '8449.610'), (1, '9076.858')] [2023-12-27 00:19:21,405][105620] Updated weights for policy 1, policy_version 1228327 (0.0009) [2023-12-27 00:19:21,464][105620] Updated weights for policy 1, policy_version 1228337 (0.0009) [2023-12-27 00:19:21,528][105620] Updated weights for policy 1, policy_version 1228347 (0.0009) [2023-12-27 00:19:21,817][105692] Updated weights for policy 0, policy_version 1227161 (0.0009) [2023-12-27 00:19:21,867][105692] Updated weights for policy 0, policy_version 1227171 (0.0008) [2023-12-27 00:19:21,915][105692] Updated weights for policy 0, policy_version 1227181 (0.0009) [2023-12-27 00:19:22,300][105620] Updated weights for policy 1, policy_version 1228357 (0.0008) [2023-12-27 00:19:22,364][105620] Updated weights for policy 1, policy_version 1228367 (0.0009) [2023-12-27 00:19:22,425][105620] Updated weights for policy 1, policy_version 1228377 (0.0009) [2023-12-27 00:19:22,630][105692] Updated weights for policy 0, policy_version 1227191 (0.0007) [2023-12-27 00:19:22,683][105692] Updated weights for policy 0, policy_version 1227201 (0.0005) [2023-12-27 00:19:22,744][105692] Updated weights for policy 0, policy_version 1227211 (0.0007) [2023-12-27 00:19:23,256][105620] Updated weights for policy 1, policy_version 1228387 (0.0008) [2023-12-27 00:19:23,317][105620] Updated weights for policy 1, policy_version 1228397 (0.0009) [2023-12-27 00:19:23,378][105620] Updated weights for policy 1, policy_version 1228407 (0.0009) [2023-12-27 00:19:23,430][105692] Updated weights for policy 0, policy_version 1227221 (0.0008) [2023-12-27 00:19:23,485][105692] Updated weights for policy 0, policy_version 1227231 (0.0008) [2023-12-27 00:19:23,547][105692] Updated weights for policy 0, policy_version 1227241 (0.0008) [2023-12-27 00:19:24,137][105692] Updated weights for policy 0, policy_version 1227251 (0.0008) [2023-12-27 00:19:24,191][105692] Updated weights for policy 0, policy_version 1227261 (0.0005) [2023-12-27 00:19:24,206][105620] Updated weights for policy 1, policy_version 1228417 (0.0009) [2023-12-27 00:19:24,239][105692] Updated weights for policy 0, policy_version 1227271 (0.0007) [2023-12-27 00:19:24,264][105620] Updated weights for policy 1, policy_version 1228427 (0.0009) [2023-12-27 00:19:24,326][105620] Updated weights for policy 1, policy_version 1228437 (0.0009) [2023-12-27 00:19:24,400][105620] Updated weights for policy 1, policy_version 1228447 (0.0010) [2023-12-27 00:19:24,920][105692] Updated weights for policy 0, policy_version 1227281 (0.0006) [2023-12-27 00:19:24,973][105692] Updated weights for policy 0, policy_version 1227291 (0.0010) [2023-12-27 00:19:25,028][105692] Updated weights for policy 0, policy_version 1227301 (0.0008) [2023-12-27 00:19:25,048][105620] Updated weights for policy 1, policy_version 1228457 (0.0006) [2023-12-27 00:19:25,077][105692] Updated weights for policy 0, policy_version 1227311 (0.0009) [2023-12-27 00:19:25,104][105620] Updated weights for policy 1, policy_version 1228467 (0.0005) [2023-12-27 00:19:25,153][105620] Updated weights for policy 1, policy_version 1228477 (0.0005) [2023-12-27 00:19:25,676][105620] Updated weights for policy 1, policy_version 1228487 (0.0008) [2023-12-27 00:19:25,723][105620] Updated weights for policy 1, policy_version 1228497 (0.0009) [2023-12-27 00:19:25,772][105620] Updated weights for policy 1, policy_version 1228507 (0.0008) [2023-12-27 00:19:25,950][105692] Updated weights for policy 0, policy_version 1227321 (0.0008) [2023-12-27 00:19:26,008][105692] Updated weights for policy 0, policy_version 1227331 (0.0009) [2023-12-27 00:19:26,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 628785152. Throughput: 0: 9924.8, 1: 9652.6. Samples: 628796396. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:19:26,063][104569] Avg episode reward: [(0, '7815.952'), (1, '9259.780')] [2023-12-27 00:19:26,063][105692] Updated weights for policy 0, policy_version 1227342 (0.0010) [2023-12-27 00:19:26,398][105620] Updated weights for policy 1, policy_version 1228517 (0.0005) [2023-12-27 00:19:26,454][105620] Updated weights for policy 1, policy_version 1228527 (0.0005) [2023-12-27 00:19:26,516][105620] Updated weights for policy 1, policy_version 1228537 (0.0008) [2023-12-27 00:19:26,889][105692] Updated weights for policy 0, policy_version 1227352 (0.0006) [2023-12-27 00:19:26,952][105692] Updated weights for policy 0, policy_version 1227362 (0.0010) [2023-12-27 00:19:27,008][105692] Updated weights for policy 0, policy_version 1227372 (0.0009) [2023-12-27 00:19:27,179][105620] Updated weights for policy 1, policy_version 1228547 (0.0008) [2023-12-27 00:19:27,235][105620] Updated weights for policy 1, policy_version 1228557 (0.0009) [2023-12-27 00:19:27,286][105620] Updated weights for policy 1, policy_version 1228567 (0.0009) [2023-12-27 00:19:27,765][105692] Updated weights for policy 0, policy_version 1227382 (0.0007) [2023-12-27 00:19:27,826][105692] Updated weights for policy 0, policy_version 1227392 (0.0005) [2023-12-27 00:19:27,884][105692] Updated weights for policy 0, policy_version 1227402 (0.0005) [2023-12-27 00:19:27,919][105620] Updated weights for policy 1, policy_version 1228577 (0.0007) [2023-12-27 00:19:27,965][105620] Updated weights for policy 1, policy_version 1228587 (0.0009) [2023-12-27 00:19:28,034][105620] Updated weights for policy 1, policy_version 1228597 (0.0009) [2023-12-27 00:19:28,100][105620] Updated weights for policy 1, policy_version 1228607 (0.0010) [2023-12-27 00:19:28,410][105692] Updated weights for policy 0, policy_version 1227412 (0.0008) [2023-12-27 00:19:28,467][105692] Updated weights for policy 0, policy_version 1227422 (0.0005) [2023-12-27 00:19:28,530][105692] Updated weights for policy 0, policy_version 1227432 (0.0006) [2023-12-27 00:19:28,858][105620] Updated weights for policy 1, policy_version 1228617 (0.0008) [2023-12-27 00:19:28,913][105620] Updated weights for policy 1, policy_version 1228627 (0.0005) [2023-12-27 00:19:28,964][105620] Updated weights for policy 1, policy_version 1228637 (0.0005) [2023-12-27 00:19:29,121][105692] Updated weights for policy 0, policy_version 1227442 (0.0011) [2023-12-27 00:19:29,169][105692] Updated weights for policy 0, policy_version 1227452 (0.0010) [2023-12-27 00:19:29,228][105692] Updated weights for policy 0, policy_version 1227462 (0.0007) [2023-12-27 00:19:29,284][105692] Updated weights for policy 0, policy_version 1227472 (0.0006) [2023-12-27 00:19:29,607][105620] Updated weights for policy 1, policy_version 1228647 (0.0008) [2023-12-27 00:19:29,661][105620] Updated weights for policy 1, policy_version 1228657 (0.0009) [2023-12-27 00:19:29,711][105620] Updated weights for policy 1, policy_version 1228667 (0.0009) [2023-12-27 00:19:30,017][105692] Updated weights for policy 0, policy_version 1227482 (0.0008) [2023-12-27 00:19:30,065][105692] Updated weights for policy 0, policy_version 1227492 (0.0006) [2023-12-27 00:19:30,124][105692] Updated weights for policy 0, policy_version 1227502 (0.0009) [2023-12-27 00:19:30,486][105620] Updated weights for policy 1, policy_version 1228677 (0.0007) [2023-12-27 00:19:30,558][105620] Updated weights for policy 1, policy_version 1228687 (0.0009) [2023-12-27 00:19:30,620][105620] Updated weights for policy 1, policy_version 1228697 (0.0008) [2023-12-27 00:19:30,762][105692] Updated weights for policy 0, policy_version 1227512 (0.0006) [2023-12-27 00:19:30,814][105692] Updated weights for policy 0, policy_version 1227522 (0.0007) [2023-12-27 00:19:30,868][105692] Updated weights for policy 0, policy_version 1227532 (0.0006) [2023-12-27 00:19:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19605.2). Total num frames: 628891648. Throughput: 0: 9961.2, 1: 9726.5. Samples: 628857188. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:19:31,063][104569] Avg episode reward: [(0, '7818.405'), (1, '9350.475')] [2023-12-27 00:19:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001228704_314589184.pth... [2023-12-27 00:19:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001227536_314302464.pth... [2023-12-27 00:19:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001227584_314302464.pth [2023-12-27 00:19:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001226384_314007552.pth [2023-12-27 00:19:31,374][105620] Updated weights for policy 1, policy_version 1228707 (0.0008) [2023-12-27 00:19:31,436][105620] Updated weights for policy 1, policy_version 1228717 (0.0008) [2023-12-27 00:19:31,500][105620] Updated weights for policy 1, policy_version 1228727 (0.0009) [2023-12-27 00:19:31,522][105692] Updated weights for policy 0, policy_version 1227542 (0.0008) [2023-12-27 00:19:31,581][105692] Updated weights for policy 0, policy_version 1227552 (0.0006) [2023-12-27 00:19:31,651][105692] Updated weights for policy 0, policy_version 1227562 (0.0008) [2023-12-27 00:19:32,133][105620] Updated weights for policy 1, policy_version 1228737 (0.0007) [2023-12-27 00:19:32,190][105620] Updated weights for policy 1, policy_version 1228747 (0.0009) [2023-12-27 00:19:32,247][105620] Updated weights for policy 1, policy_version 1228757 (0.0009) [2023-12-27 00:19:32,311][105620] Updated weights for policy 1, policy_version 1228767 (0.0009) [2023-12-27 00:19:32,402][105692] Updated weights for policy 0, policy_version 1227572 (0.0010) [2023-12-27 00:19:32,465][105692] Updated weights for policy 0, policy_version 1227582 (0.0009) [2023-12-27 00:19:32,528][105692] Updated weights for policy 0, policy_version 1227592 (0.0009) [2023-12-27 00:19:33,073][105620] Updated weights for policy 1, policy_version 1228777 (0.0008) [2023-12-27 00:19:33,119][105620] Updated weights for policy 1, policy_version 1228787 (0.0008) [2023-12-27 00:19:33,171][105620] Updated weights for policy 1, policy_version 1228797 (0.0010) [2023-12-27 00:19:33,254][105692] Updated weights for policy 0, policy_version 1227602 (0.0008) [2023-12-27 00:19:33,298][105692] Updated weights for policy 0, policy_version 1227612 (0.0005) [2023-12-27 00:19:33,343][105692] Updated weights for policy 0, policy_version 1227622 (0.0005) [2023-12-27 00:19:33,386][105692] Updated weights for policy 0, policy_version 1227632 (0.0005) [2023-12-27 00:19:33,863][105620] Updated weights for policy 1, policy_version 1228807 (0.0007) [2023-12-27 00:19:33,916][105692] Updated weights for policy 0, policy_version 1227642 (0.0005) [2023-12-27 00:19:33,922][105620] Updated weights for policy 1, policy_version 1228817 (0.0005) [2023-12-27 00:19:33,965][105692] Updated weights for policy 0, policy_version 1227652 (0.0006) [2023-12-27 00:19:33,977][105620] Updated weights for policy 1, policy_version 1228827 (0.0005) [2023-12-27 00:19:34,025][105692] Updated weights for policy 0, policy_version 1227662 (0.0007) [2023-12-27 00:19:34,540][105620] Updated weights for policy 1, policy_version 1228837 (0.0007) [2023-12-27 00:19:34,590][105620] Updated weights for policy 1, policy_version 1228847 (0.0006) [2023-12-27 00:19:34,651][105620] Updated weights for policy 1, policy_version 1228857 (0.0010) [2023-12-27 00:19:34,836][105692] Updated weights for policy 0, policy_version 1227672 (0.0009) [2023-12-27 00:19:34,887][105692] Updated weights for policy 0, policy_version 1227682 (0.0008) [2023-12-27 00:19:34,945][105692] Updated weights for policy 0, policy_version 1227692 (0.0005) [2023-12-27 00:19:35,404][105620] Updated weights for policy 1, policy_version 1228867 (0.0009) [2023-12-27 00:19:35,452][105620] Updated weights for policy 1, policy_version 1228877 (0.0008) [2023-12-27 00:19:35,510][105620] Updated weights for policy 1, policy_version 1228887 (0.0009) [2023-12-27 00:19:35,623][105692] Updated weights for policy 0, policy_version 1227702 (0.0008) [2023-12-27 00:19:35,674][105692] Updated weights for policy 0, policy_version 1227712 (0.0010) [2023-12-27 00:19:35,732][105692] Updated weights for policy 0, policy_version 1227722 (0.0010) [2023-12-27 00:19:36,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 628989952. Throughput: 0: 10020.8, 1: 9723.6. Samples: 628979196. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:19:36,063][104569] Avg episode reward: [(0, '7375.328'), (1, '9349.760')] [2023-12-27 00:19:36,231][105620] Updated weights for policy 1, policy_version 1228897 (0.0009) [2023-12-27 00:19:36,290][105620] Updated weights for policy 1, policy_version 1228907 (0.0008) [2023-12-27 00:19:36,343][105620] Updated weights for policy 1, policy_version 1228917 (0.0008) [2023-12-27 00:19:36,401][105620] Updated weights for policy 1, policy_version 1228927 (0.0008) [2023-12-27 00:19:36,461][105692] Updated weights for policy 0, policy_version 1227732 (0.0011) [2023-12-27 00:19:36,520][105692] Updated weights for policy 0, policy_version 1227742 (0.0011) [2023-12-27 00:19:36,583][105692] Updated weights for policy 0, policy_version 1227752 (0.0011) [2023-12-27 00:19:37,154][105620] Updated weights for policy 1, policy_version 1228937 (0.0006) [2023-12-27 00:19:37,213][105620] Updated weights for policy 1, policy_version 1228947 (0.0005) [2023-12-27 00:19:37,269][105620] Updated weights for policy 1, policy_version 1228957 (0.0005) [2023-12-27 00:19:37,284][105692] Updated weights for policy 0, policy_version 1227762 (0.0011) [2023-12-27 00:19:37,334][105692] Updated weights for policy 0, policy_version 1227772 (0.0010) [2023-12-27 00:19:37,383][105692] Updated weights for policy 0, policy_version 1227782 (0.0010) [2023-12-27 00:19:37,435][105692] Updated weights for policy 0, policy_version 1227792 (0.0009) [2023-12-27 00:19:37,905][105620] Updated weights for policy 1, policy_version 1228967 (0.0007) [2023-12-27 00:19:37,957][105620] Updated weights for policy 1, policy_version 1228977 (0.0008) [2023-12-27 00:19:38,009][105620] Updated weights for policy 1, policy_version 1228987 (0.0008) [2023-12-27 00:19:38,206][105692] Updated weights for policy 0, policy_version 1227802 (0.0010) [2023-12-27 00:19:38,254][105692] Updated weights for policy 0, policy_version 1227812 (0.0010) [2023-12-27 00:19:38,305][105692] Updated weights for policy 0, policy_version 1227822 (0.0010) [2023-12-27 00:19:38,800][105620] Updated weights for policy 1, policy_version 1228997 (0.0008) [2023-12-27 00:19:38,863][105620] Updated weights for policy 1, policy_version 1229007 (0.0008) [2023-12-27 00:19:38,926][105620] Updated weights for policy 1, policy_version 1229017 (0.0008) [2023-12-27 00:19:39,070][105692] Updated weights for policy 0, policy_version 1227832 (0.0010) [2023-12-27 00:19:39,124][105692] Updated weights for policy 0, policy_version 1227842 (0.0010) [2023-12-27 00:19:39,175][105692] Updated weights for policy 0, policy_version 1227852 (0.0010) [2023-12-27 00:19:39,606][105620] Updated weights for policy 1, policy_version 1229027 (0.0009) [2023-12-27 00:19:39,664][105620] Updated weights for policy 1, policy_version 1229037 (0.0010) [2023-12-27 00:19:39,727][105620] Updated weights for policy 1, policy_version 1229047 (0.0009) [2023-12-27 00:19:39,872][105692] Updated weights for policy 0, policy_version 1227862 (0.0009) [2023-12-27 00:19:39,938][105692] Updated weights for policy 0, policy_version 1227872 (0.0009) [2023-12-27 00:19:40,009][105692] Updated weights for policy 0, policy_version 1227882 (0.0007) [2023-12-27 00:19:40,435][105620] Updated weights for policy 1, policy_version 1229057 (0.0009) [2023-12-27 00:19:40,499][105620] Updated weights for policy 1, policy_version 1229067 (0.0008) [2023-12-27 00:19:40,555][105620] Updated weights for policy 1, policy_version 1229077 (0.0008) [2023-12-27 00:19:40,605][105620] Updated weights for policy 1, policy_version 1229087 (0.0008) [2023-12-27 00:19:40,683][105692] Updated weights for policy 0, policy_version 1227892 (0.0008) [2023-12-27 00:19:40,743][105692] Updated weights for policy 0, policy_version 1227902 (0.0006) [2023-12-27 00:19:40,801][105692] Updated weights for policy 0, policy_version 1227912 (0.0009) [2023-12-27 00:19:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 629088256. Throughput: 0: 9959.4, 1: 9685.9. Samples: 629095532. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:19:41,062][104569] Avg episode reward: [(0, '7099.354'), (1, '9257.487')] [2023-12-27 00:19:41,346][105620] Updated weights for policy 1, policy_version 1229097 (0.0009) [2023-12-27 00:19:41,420][105620] Updated weights for policy 1, policy_version 1229107 (0.0009) [2023-12-27 00:19:41,494][105620] Updated weights for policy 1, policy_version 1229117 (0.0010) [2023-12-27 00:19:41,536][105692] Updated weights for policy 0, policy_version 1227922 (0.0009) [2023-12-27 00:19:41,597][105692] Updated weights for policy 0, policy_version 1227932 (0.0009) [2023-12-27 00:19:41,664][105692] Updated weights for policy 0, policy_version 1227942 (0.0008) [2023-12-27 00:19:41,728][105692] Updated weights for policy 0, policy_version 1227952 (0.0008) [2023-12-27 00:19:42,272][105620] Updated weights for policy 1, policy_version 1229127 (0.0009) [2023-12-27 00:19:42,347][105620] Updated weights for policy 1, policy_version 1229137 (0.0007) [2023-12-27 00:19:42,417][105620] Updated weights for policy 1, policy_version 1229147 (0.0009) [2023-12-27 00:19:42,440][105692] Updated weights for policy 0, policy_version 1227962 (0.0007) [2023-12-27 00:19:42,501][105692] Updated weights for policy 0, policy_version 1227972 (0.0006) [2023-12-27 00:19:42,550][105692] Updated weights for policy 0, policy_version 1227982 (0.0005) [2023-12-27 00:19:43,220][105692] Updated weights for policy 0, policy_version 1227992 (0.0005) [2023-12-27 00:19:43,221][105620] Updated weights for policy 1, policy_version 1229157 (0.0008) [2023-12-27 00:19:43,272][105692] Updated weights for policy 0, policy_version 1228002 (0.0005) [2023-12-27 00:19:43,280][105620] Updated weights for policy 1, policy_version 1229167 (0.0008) [2023-12-27 00:19:43,323][105692] Updated weights for policy 0, policy_version 1228012 (0.0009) [2023-12-27 00:19:43,330][105620] Updated weights for policy 1, policy_version 1229177 (0.0006) [2023-12-27 00:19:43,910][105692] Updated weights for policy 0, policy_version 1228022 (0.0007) [2023-12-27 00:19:43,961][105692] Updated weights for policy 0, policy_version 1228032 (0.0008) [2023-12-27 00:19:44,012][105620] Updated weights for policy 1, policy_version 1229187 (0.0007) [2023-12-27 00:19:44,018][105692] Updated weights for policy 0, policy_version 1228043 (0.0010) [2023-12-27 00:19:44,072][105620] Updated weights for policy 1, policy_version 1229197 (0.0005) [2023-12-27 00:19:44,138][105620] Updated weights for policy 1, policy_version 1229207 (0.0005) [2023-12-27 00:19:44,668][105620] Updated weights for policy 1, policy_version 1229217 (0.0006) [2023-12-27 00:19:44,720][105620] Updated weights for policy 1, policy_version 1229227 (0.0010) [2023-12-27 00:19:44,750][105692] Updated weights for policy 0, policy_version 1228054 (0.0007) [2023-12-27 00:19:44,773][105620] Updated weights for policy 1, policy_version 1229237 (0.0009) [2023-12-27 00:19:44,812][105692] Updated weights for policy 0, policy_version 1228064 (0.0007) [2023-12-27 00:19:44,835][105620] Updated weights for policy 1, policy_version 1229247 (0.0007) [2023-12-27 00:19:44,879][105692] Updated weights for policy 0, policy_version 1228074 (0.0006) [2023-12-27 00:19:45,502][105692] Updated weights for policy 0, policy_version 1228084 (0.0007) [2023-12-27 00:19:45,554][105692] Updated weights for policy 0, policy_version 1228094 (0.0007) [2023-12-27 00:19:45,598][105620] Updated weights for policy 1, policy_version 1229257 (0.0008) [2023-12-27 00:19:45,601][105692] Updated weights for policy 0, policy_version 1228104 (0.0007) [2023-12-27 00:19:45,655][105620] Updated weights for policy 1, policy_version 1229267 (0.0007) [2023-12-27 00:19:45,716][105620] Updated weights for policy 1, policy_version 1229277 (0.0008) [2023-12-27 00:19:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 629186560. Throughput: 0: 9900.2, 1: 9623.3. Samples: 629153264. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:19:46,062][104569] Avg episode reward: [(0, '7638.772'), (1, '9165.112')] [2023-12-27 00:19:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001228112_314449920.pth... [2023-12-27 00:19:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001229280_314736640.pth... [2023-12-27 00:19:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001228128_314441728.pth [2023-12-27 00:19:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001226960_314155008.pth [2023-12-27 00:19:46,268][105692] Updated weights for policy 0, policy_version 1228114 (0.0008) [2023-12-27 00:19:46,317][105692] Updated weights for policy 0, policy_version 1228124 (0.0005) [2023-12-27 00:19:46,377][105692] Updated weights for policy 0, policy_version 1228134 (0.0006) [2023-12-27 00:19:46,446][105692] Updated weights for policy 0, policy_version 1228144 (0.0005) [2023-12-27 00:19:46,505][105620] Updated weights for policy 1, policy_version 1229287 (0.0007) [2023-12-27 00:19:46,573][105620] Updated weights for policy 1, policy_version 1229297 (0.0005) [2023-12-27 00:19:46,647][105620] Updated weights for policy 1, policy_version 1229307 (0.0006) [2023-12-27 00:19:47,104][105692] Updated weights for policy 0, policy_version 1228154 (0.0011) [2023-12-27 00:19:47,168][105692] Updated weights for policy 0, policy_version 1228164 (0.0009) [2023-12-27 00:19:47,228][105692] Updated weights for policy 0, policy_version 1228174 (0.0010) [2023-12-27 00:19:47,290][105620] Updated weights for policy 1, policy_version 1229317 (0.0009) [2023-12-27 00:19:47,355][105620] Updated weights for policy 1, policy_version 1229327 (0.0010) [2023-12-27 00:19:47,417][105620] Updated weights for policy 1, policy_version 1229337 (0.0010) [2023-12-27 00:19:47,876][105692] Updated weights for policy 0, policy_version 1228184 (0.0010) [2023-12-27 00:19:47,931][105692] Updated weights for policy 0, policy_version 1228194 (0.0010) [2023-12-27 00:19:47,982][105692] Updated weights for policy 0, policy_version 1228204 (0.0010) [2023-12-27 00:19:48,049][105620] Updated weights for policy 1, policy_version 1229347 (0.0010) [2023-12-27 00:19:48,104][105620] Updated weights for policy 1, policy_version 1229357 (0.0010) [2023-12-27 00:19:48,163][105620] Updated weights for policy 1, policy_version 1229367 (0.0011) [2023-12-27 00:19:48,637][105692] Updated weights for policy 0, policy_version 1228214 (0.0007) [2023-12-27 00:19:48,707][105692] Updated weights for policy 0, policy_version 1228224 (0.0005) [2023-12-27 00:19:48,765][105692] Updated weights for policy 0, policy_version 1228234 (0.0007) [2023-12-27 00:19:48,928][105620] Updated weights for policy 1, policy_version 1229377 (0.0011) [2023-12-27 00:19:48,983][105620] Updated weights for policy 1, policy_version 1229387 (0.0010) [2023-12-27 00:19:49,053][105620] Updated weights for policy 1, policy_version 1229397 (0.0011) [2023-12-27 00:19:49,112][105620] Updated weights for policy 1, policy_version 1229407 (0.0010) [2023-12-27 00:19:49,420][105692] Updated weights for policy 0, policy_version 1228244 (0.0009) [2023-12-27 00:19:49,483][105692] Updated weights for policy 0, policy_version 1228254 (0.0008) [2023-12-27 00:19:49,547][105692] Updated weights for policy 0, policy_version 1228264 (0.0008) [2023-12-27 00:19:49,869][105620] Updated weights for policy 1, policy_version 1229417 (0.0010) [2023-12-27 00:19:49,925][105620] Updated weights for policy 1, policy_version 1229427 (0.0010) [2023-12-27 00:19:49,988][105620] Updated weights for policy 1, policy_version 1229437 (0.0011) [2023-12-27 00:19:50,242][105692] Updated weights for policy 0, policy_version 1228274 (0.0009) [2023-12-27 00:19:50,297][105692] Updated weights for policy 0, policy_version 1228284 (0.0008) [2023-12-27 00:19:50,350][105692] Updated weights for policy 0, policy_version 1228294 (0.0008) [2023-12-27 00:19:50,407][105692] Updated weights for policy 0, policy_version 1228304 (0.0008) [2023-12-27 00:19:50,741][105620] Updated weights for policy 1, policy_version 1229447 (0.0010) [2023-12-27 00:19:50,804][105620] Updated weights for policy 1, policy_version 1229457 (0.0009) [2023-12-27 00:19:50,873][105620] Updated weights for policy 1, policy_version 1229467 (0.0006) [2023-12-27 00:19:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 629284864. Throughput: 0: 10014.0, 1: 9716.8. Samples: 629275016. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:19:51,063][104569] Avg episode reward: [(0, '7544.361'), (1, '8988.322')] [2023-12-27 00:19:51,135][105692] Updated weights for policy 0, policy_version 1228314 (0.0009) [2023-12-27 00:19:51,187][105692] Updated weights for policy 0, policy_version 1228324 (0.0008) [2023-12-27 00:19:51,236][105692] Updated weights for policy 0, policy_version 1228334 (0.0008) [2023-12-27 00:19:51,640][105620] Updated weights for policy 1, policy_version 1229477 (0.0008) [2023-12-27 00:19:51,700][105620] Updated weights for policy 1, policy_version 1229487 (0.0009) [2023-12-27 00:19:51,765][105620] Updated weights for policy 1, policy_version 1229497 (0.0009) [2023-12-27 00:19:52,013][105585] KL-divergence is very high: 107.7196 [2023-12-27 00:19:52,018][105692] Updated weights for policy 0, policy_version 1228344 (0.0009) [2023-12-27 00:19:52,024][105585] KL-divergence is very high: 151.4599 [2023-12-27 00:19:52,060][105585] KL-divergence is very high: 263.1262 [2023-12-27 00:19:52,073][105585] KL-divergence is very high: 289.0757 [2023-12-27 00:19:52,078][105692] Updated weights for policy 0, policy_version 1228354 (0.0010) [2023-12-27 00:19:52,112][105585] KL-divergence is very high: 337.4914 [2023-12-27 00:19:52,124][105585] KL-divergence is very high: 346.1126 [2023-12-27 00:19:52,142][105692] Updated weights for policy 0, policy_version 1228364 (0.0009) [2023-12-27 00:19:52,157][105585] KL-divergence is very high: 317.3525 [2023-12-27 00:19:52,533][105620] Updated weights for policy 1, policy_version 1229507 (0.0010) [2023-12-27 00:19:52,594][105620] Updated weights for policy 1, policy_version 1229517 (0.0009) [2023-12-27 00:19:52,655][105620] Updated weights for policy 1, policy_version 1229527 (0.0008) [2023-12-27 00:19:52,937][105692] Updated weights for policy 0, policy_version 1228374 (0.0009) [2023-12-27 00:19:52,997][105692] Updated weights for policy 0, policy_version 1228384 (0.0008) [2023-12-27 00:19:53,050][105692] Updated weights for policy 0, policy_version 1228394 (0.0008) [2023-12-27 00:19:53,400][105620] Updated weights for policy 1, policy_version 1229537 (0.0010) [2023-12-27 00:19:53,466][105620] Updated weights for policy 1, policy_version 1229547 (0.0011) [2023-12-27 00:19:53,518][105620] Updated weights for policy 1, policy_version 1229557 (0.0011) [2023-12-27 00:19:53,579][105620] Updated weights for policy 1, policy_version 1229567 (0.0010) [2023-12-27 00:19:53,747][105692] Updated weights for policy 0, policy_version 1228404 (0.0008) [2023-12-27 00:19:53,802][105692] Updated weights for policy 0, policy_version 1228414 (0.0008) [2023-12-27 00:19:53,851][105692] Updated weights for policy 0, policy_version 1228424 (0.0008) [2023-12-27 00:19:54,313][105620] Updated weights for policy 1, policy_version 1229577 (0.0009) [2023-12-27 00:19:54,370][105620] Updated weights for policy 1, policy_version 1229587 (0.0009) [2023-12-27 00:19:54,431][105620] Updated weights for policy 1, policy_version 1229597 (0.0008) [2023-12-27 00:19:54,628][105692] Updated weights for policy 0, policy_version 1228434 (0.0008) [2023-12-27 00:19:54,684][105692] Updated weights for policy 0, policy_version 1228444 (0.0010) [2023-12-27 00:19:54,737][105692] Updated weights for policy 0, policy_version 1228454 (0.0009) [2023-12-27 00:19:54,790][105692] Updated weights for policy 0, policy_version 1228464 (0.0009) [2023-12-27 00:19:55,047][105620] Updated weights for policy 1, policy_version 1229607 (0.0010) [2023-12-27 00:19:55,108][105620] Updated weights for policy 1, policy_version 1229617 (0.0008) [2023-12-27 00:19:55,167][105620] Updated weights for policy 1, policy_version 1229627 (0.0005) [2023-12-27 00:19:55,644][105692] Updated weights for policy 0, policy_version 1228474 (0.0010) [2023-12-27 00:19:55,706][105692] Updated weights for policy 0, policy_version 1228484 (0.0009) [2023-12-27 00:19:55,765][105692] Updated weights for policy 0, policy_version 1228494 (0.0009) [2023-12-27 00:19:55,823][105620] Updated weights for policy 1, policy_version 1229637 (0.0005) [2023-12-27 00:19:55,876][105620] Updated weights for policy 1, policy_version 1229647 (0.0005) [2023-12-27 00:19:55,931][105620] Updated weights for policy 1, policy_version 1229658 (0.0007) [2023-12-27 00:19:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 629383168. Throughput: 0: 9855.7, 1: 9794.6. Samples: 629388820. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:19:56,063][104569] Avg episode reward: [(0, '7279.107'), (1, '9001.525')] [2023-12-27 00:19:56,537][105692] Updated weights for policy 0, policy_version 1228504 (0.0006) [2023-12-27 00:19:56,559][105620] Updated weights for policy 1, policy_version 1229668 (0.0010) [2023-12-27 00:19:56,583][105692] Updated weights for policy 0, policy_version 1228514 (0.0005) [2023-12-27 00:19:56,617][105620] Updated weights for policy 1, policy_version 1229678 (0.0010) [2023-12-27 00:19:56,632][105692] Updated weights for policy 0, policy_version 1228524 (0.0008) [2023-12-27 00:19:56,671][105620] Updated weights for policy 1, policy_version 1229688 (0.0010) [2023-12-27 00:19:57,176][105692] Updated weights for policy 0, policy_version 1228534 (0.0007) [2023-12-27 00:19:57,219][105692] Updated weights for policy 0, policy_version 1228544 (0.0005) [2023-12-27 00:19:57,263][105692] Updated weights for policy 0, policy_version 1228554 (0.0009) [2023-12-27 00:19:57,387][105620] Updated weights for policy 1, policy_version 1229698 (0.0010) [2023-12-27 00:19:57,449][105620] Updated weights for policy 1, policy_version 1229708 (0.0008) [2023-12-27 00:19:57,504][105620] Updated weights for policy 1, policy_version 1229718 (0.0009) [2023-12-27 00:19:57,560][105620] Updated weights for policy 1, policy_version 1229728 (0.0008) [2023-12-27 00:19:58,001][105692] Updated weights for policy 0, policy_version 1228564 (0.0010) [2023-12-27 00:19:58,055][105692] Updated weights for policy 0, policy_version 1228574 (0.0010) [2023-12-27 00:19:58,109][105692] Updated weights for policy 0, policy_version 1228584 (0.0010) [2023-12-27 00:19:58,318][105620] Updated weights for policy 1, policy_version 1229738 (0.0009) [2023-12-27 00:19:58,400][105620] Updated weights for policy 1, policy_version 1229748 (0.0010) [2023-12-27 00:19:58,472][105620] Updated weights for policy 1, policy_version 1229758 (0.0011) [2023-12-27 00:19:59,015][105692] Updated weights for policy 0, policy_version 1228594 (0.0010) [2023-12-27 00:19:59,084][105692] Updated weights for policy 0, policy_version 1228604 (0.0009) [2023-12-27 00:19:59,143][105692] Updated weights for policy 0, policy_version 1228614 (0.0009) [2023-12-27 00:19:59,192][105692] Updated weights for policy 0, policy_version 1228624 (0.0009) [2023-12-27 00:19:59,276][105620] Updated weights for policy 1, policy_version 1229768 (0.0006) [2023-12-27 00:19:59,341][105620] Updated weights for policy 1, policy_version 1229778 (0.0006) [2023-12-27 00:19:59,410][105620] Updated weights for policy 1, policy_version 1229788 (0.0009) [2023-12-27 00:20:00,015][105692] Updated weights for policy 0, policy_version 1228634 (0.0008) [2023-12-27 00:20:00,076][105692] Updated weights for policy 0, policy_version 1228644 (0.0005) [2023-12-27 00:20:00,130][105620] Updated weights for policy 1, policy_version 1229798 (0.0008) [2023-12-27 00:20:00,137][105692] Updated weights for policy 0, policy_version 1228654 (0.0007) [2023-12-27 00:20:00,187][105620] Updated weights for policy 1, policy_version 1229808 (0.0009) [2023-12-27 00:20:00,252][105620] Updated weights for policy 1, policy_version 1229818 (0.0010) [2023-12-27 00:20:00,709][105692] Updated weights for policy 0, policy_version 1228664 (0.0005) [2023-12-27 00:20:00,762][105692] Updated weights for policy 0, policy_version 1228674 (0.0005) [2023-12-27 00:20:00,815][105692] Updated weights for policy 0, policy_version 1228684 (0.0006) [2023-12-27 00:20:00,982][105620] Updated weights for policy 1, policy_version 1229828 (0.0009) [2023-12-27 00:20:01,045][105620] Updated weights for policy 1, policy_version 1229838 (0.0009) [2023-12-27 00:20:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 629473280. Throughput: 0: 9893.9, 1: 9724.6. Samples: 629446700. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:20:01,062][104569] Avg episode reward: [(0, '7640.935'), (1, '8903.670')] [2023-12-27 00:20:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001228688_314597376.pth... [2023-12-27 00:20:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001227536_314302464.pth [2023-12-27 00:20:01,104][105620] Updated weights for policy 1, policy_version 1229848 (0.0009) [2023-12-27 00:20:01,155][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001229856_314884096.pth... [2023-12-27 00:20:01,160][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001228704_314589184.pth [2023-12-27 00:20:01,500][105692] Updated weights for policy 0, policy_version 1228694 (0.0007) [2023-12-27 00:20:01,570][105692] Updated weights for policy 0, policy_version 1228704 (0.0005) [2023-12-27 00:20:01,636][105692] Updated weights for policy 0, policy_version 1228714 (0.0006) [2023-12-27 00:20:01,938][105620] Updated weights for policy 1, policy_version 1229858 (0.0009) [2023-12-27 00:20:02,001][105620] Updated weights for policy 1, policy_version 1229868 (0.0009) [2023-12-27 00:20:02,072][105620] Updated weights for policy 1, policy_version 1229878 (0.0009) [2023-12-27 00:20:02,129][105620] Updated weights for policy 1, policy_version 1229888 (0.0009) [2023-12-27 00:20:02,236][105692] Updated weights for policy 0, policy_version 1228724 (0.0007) [2023-12-27 00:20:02,299][105692] Updated weights for policy 0, policy_version 1228734 (0.0010) [2023-12-27 00:20:02,356][105692] Updated weights for policy 0, policy_version 1228744 (0.0008) [2023-12-27 00:20:02,869][105620] Updated weights for policy 1, policy_version 1229898 (0.0010) [2023-12-27 00:20:02,927][105620] Updated weights for policy 1, policy_version 1229908 (0.0010) [2023-12-27 00:20:02,980][105620] Updated weights for policy 1, policy_version 1229918 (0.0010) [2023-12-27 00:20:03,009][105692] Updated weights for policy 0, policy_version 1228754 (0.0008) [2023-12-27 00:20:03,068][105692] Updated weights for policy 0, policy_version 1228764 (0.0005) [2023-12-27 00:20:03,128][105692] Updated weights for policy 0, policy_version 1228774 (0.0005) [2023-12-27 00:20:03,177][105692] Updated weights for policy 0, policy_version 1228784 (0.0005) [2023-12-27 00:20:03,758][105692] Updated weights for policy 0, policy_version 1228794 (0.0005) [2023-12-27 00:20:03,813][105692] Updated weights for policy 0, policy_version 1228804 (0.0005) [2023-12-27 00:20:03,813][105620] Updated weights for policy 1, policy_version 1229928 (0.0006) [2023-12-27 00:20:03,869][105620] Updated weights for policy 1, policy_version 1229938 (0.0007) [2023-12-27 00:20:03,870][105692] Updated weights for policy 0, policy_version 1228814 (0.0007) [2023-12-27 00:20:03,935][105620] Updated weights for policy 1, policy_version 1229948 (0.0010) [2023-12-27 00:20:04,594][105620] Updated weights for policy 1, policy_version 1229958 (0.0007) [2023-12-27 00:20:04,598][105692] Updated weights for policy 0, policy_version 1228824 (0.0006) [2023-12-27 00:20:04,658][105620] Updated weights for policy 1, policy_version 1229968 (0.0006) [2023-12-27 00:20:04,658][105692] Updated weights for policy 0, policy_version 1228834 (0.0005) [2023-12-27 00:20:04,715][105692] Updated weights for policy 0, policy_version 1228844 (0.0005) [2023-12-27 00:20:04,717][105620] Updated weights for policy 1, policy_version 1229978 (0.0006) [2023-12-27 00:20:05,272][105620] Updated weights for policy 1, policy_version 1229988 (0.0006) [2023-12-27 00:20:05,310][105692] Updated weights for policy 0, policy_version 1228854 (0.0006) [2023-12-27 00:20:05,338][105620] Updated weights for policy 1, policy_version 1229998 (0.0006) [2023-12-27 00:20:05,361][105692] Updated weights for policy 0, policy_version 1228864 (0.0005) [2023-12-27 00:20:05,402][105620] Updated weights for policy 1, policy_version 1230008 (0.0009) [2023-12-27 00:20:05,405][105692] Updated weights for policy 0, policy_version 1228874 (0.0005) [2023-12-27 00:20:05,989][105692] Updated weights for policy 0, policy_version 1228884 (0.0005) [2023-12-27 00:20:06,053][105692] Updated weights for policy 0, policy_version 1228894 (0.0006) [2023-12-27 00:20:06,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 629571584. Throughput: 0: 9920.4, 1: 9702.4. Samples: 629564208. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:20:06,062][104569] Avg episode reward: [(0, '8091.892'), (1, '8982.085')] [2023-12-27 00:20:06,114][105620] Updated weights for policy 1, policy_version 1230018 (0.0010) [2023-12-27 00:20:06,122][105692] Updated weights for policy 0, policy_version 1228904 (0.0011) [2023-12-27 00:20:06,175][105620] Updated weights for policy 1, policy_version 1230028 (0.0009) [2023-12-27 00:20:06,234][105620] Updated weights for policy 1, policy_version 1230038 (0.0011) [2023-12-27 00:20:06,296][105620] Updated weights for policy 1, policy_version 1230048 (0.0010) [2023-12-27 00:20:06,824][105692] Updated weights for policy 0, policy_version 1228914 (0.0011) [2023-12-27 00:20:06,869][105692] Updated weights for policy 0, policy_version 1228924 (0.0010) [2023-12-27 00:20:06,924][105692] Updated weights for policy 0, policy_version 1228934 (0.0010) [2023-12-27 00:20:06,951][105620] Updated weights for policy 1, policy_version 1230058 (0.0011) [2023-12-27 00:20:06,984][105692] Updated weights for policy 0, policy_version 1228944 (0.0007) [2023-12-27 00:20:07,007][105620] Updated weights for policy 1, policy_version 1230068 (0.0011) [2023-12-27 00:20:07,070][105620] Updated weights for policy 1, policy_version 1230078 (0.0011) [2023-12-27 00:20:07,638][105692] Updated weights for policy 0, policy_version 1228954 (0.0010) [2023-12-27 00:20:07,692][105692] Updated weights for policy 0, policy_version 1228964 (0.0009) [2023-12-27 00:20:07,739][105692] Updated weights for policy 0, policy_version 1228974 (0.0006) [2023-12-27 00:20:07,820][105620] Updated weights for policy 1, policy_version 1230088 (0.0010) [2023-12-27 00:20:07,878][105620] Updated weights for policy 1, policy_version 1230098 (0.0010) [2023-12-27 00:20:07,934][105620] Updated weights for policy 1, policy_version 1230108 (0.0010) [2023-12-27 00:20:08,473][105692] Updated weights for policy 0, policy_version 1228984 (0.0008) [2023-12-27 00:20:08,526][105692] Updated weights for policy 0, policy_version 1228994 (0.0008) [2023-12-27 00:20:08,589][105692] Updated weights for policy 0, policy_version 1229004 (0.0008) [2023-12-27 00:20:08,692][105620] Updated weights for policy 1, policy_version 1230118 (0.0010) [2023-12-27 00:20:08,750][105620] Updated weights for policy 1, policy_version 1230128 (0.0010) [2023-12-27 00:20:08,812][105620] Updated weights for policy 1, policy_version 1230138 (0.0010) [2023-12-27 00:20:09,344][105692] Updated weights for policy 0, policy_version 1229014 (0.0008) [2023-12-27 00:20:09,418][105692] Updated weights for policy 0, policy_version 1229024 (0.0008) [2023-12-27 00:20:09,476][105692] Updated weights for policy 0, policy_version 1229034 (0.0008) [2023-12-27 00:20:09,582][105620] Updated weights for policy 1, policy_version 1230148 (0.0008) [2023-12-27 00:20:09,650][105620] Updated weights for policy 1, policy_version 1230158 (0.0008) [2023-12-27 00:20:09,698][105620] Updated weights for policy 1, policy_version 1230168 (0.0008) [2023-12-27 00:20:10,302][105692] Updated weights for policy 0, policy_version 1229044 (0.0008) [2023-12-27 00:20:10,359][105692] Updated weights for policy 0, policy_version 1229054 (0.0008) [2023-12-27 00:20:10,370][105620] Updated weights for policy 1, policy_version 1230178 (0.0011) [2023-12-27 00:20:10,416][105692] Updated weights for policy 0, policy_version 1229064 (0.0006) [2023-12-27 00:20:10,433][105620] Updated weights for policy 1, policy_version 1230188 (0.0011) [2023-12-27 00:20:10,493][105620] Updated weights for policy 1, policy_version 1230198 (0.0011) [2023-12-27 00:20:10,556][105620] Updated weights for policy 1, policy_version 1230208 (0.0010) [2023-12-27 00:20:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 629669888. Throughput: 0: 9960.5, 1: 9727.2. Samples: 629682340. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 00:20:11,062][104569] Avg episode reward: [(0, '8000.366'), (1, '9256.316')] [2023-12-27 00:20:11,198][105620] Updated weights for policy 1, policy_version 1230218 (0.0008) [2023-12-27 00:20:11,237][105692] Updated weights for policy 0, policy_version 1229074 (0.0007) [2023-12-27 00:20:11,261][105620] Updated weights for policy 1, policy_version 1230228 (0.0008) [2023-12-27 00:20:11,297][105692] Updated weights for policy 0, policy_version 1229084 (0.0007) [2023-12-27 00:20:11,320][105620] Updated weights for policy 1, policy_version 1230238 (0.0008) [2023-12-27 00:20:11,361][105692] Updated weights for policy 0, policy_version 1229094 (0.0007) [2023-12-27 00:20:11,424][105692] Updated weights for policy 0, policy_version 1229104 (0.0009) [2023-12-27 00:20:12,067][105620] Updated weights for policy 1, policy_version 1230248 (0.0008) [2023-12-27 00:20:12,120][105620] Updated weights for policy 1, policy_version 1230258 (0.0009) [2023-12-27 00:20:12,174][105620] Updated weights for policy 1, policy_version 1230268 (0.0006) [2023-12-27 00:20:12,182][105692] Updated weights for policy 0, policy_version 1229114 (0.0008) [2023-12-27 00:20:12,239][105692] Updated weights for policy 0, policy_version 1229124 (0.0008) [2023-12-27 00:20:12,299][105692] Updated weights for policy 0, policy_version 1229134 (0.0009) [2023-12-27 00:20:12,873][105620] Updated weights for policy 1, policy_version 1230278 (0.0009) [2023-12-27 00:20:12,929][105620] Updated weights for policy 1, policy_version 1230288 (0.0009) [2023-12-27 00:20:12,994][105620] Updated weights for policy 1, policy_version 1230298 (0.0006) [2023-12-27 00:20:13,124][105692] Updated weights for policy 0, policy_version 1229144 (0.0006) [2023-12-27 00:20:13,200][105692] Updated weights for policy 0, policy_version 1229154 (0.0005) [2023-12-27 00:20:13,255][105692] Updated weights for policy 0, policy_version 1229164 (0.0006) [2023-12-27 00:20:13,709][105620] Updated weights for policy 1, policy_version 1230308 (0.0010) [2023-12-27 00:20:13,780][105620] Updated weights for policy 1, policy_version 1230318 (0.0010) [2023-12-27 00:20:13,837][105620] Updated weights for policy 1, policy_version 1230328 (0.0010) [2023-12-27 00:20:13,850][105692] Updated weights for policy 0, policy_version 1229174 (0.0009) [2023-12-27 00:20:13,910][105692] Updated weights for policy 0, policy_version 1229184 (0.0006) [2023-12-27 00:20:13,962][105692] Updated weights for policy 0, policy_version 1229194 (0.0010) [2023-12-27 00:20:14,368][105620] Updated weights for policy 1, policy_version 1230338 (0.0007) [2023-12-27 00:20:14,421][105620] Updated weights for policy 1, policy_version 1230348 (0.0005) [2023-12-27 00:20:14,476][105620] Updated weights for policy 1, policy_version 1230358 (0.0005) [2023-12-27 00:20:14,531][105620] Updated weights for policy 1, policy_version 1230368 (0.0005) [2023-12-27 00:20:14,692][105692] Updated weights for policy 0, policy_version 1229204 (0.0010) [2023-12-27 00:20:14,741][105692] Updated weights for policy 0, policy_version 1229214 (0.0010) [2023-12-27 00:20:14,814][105692] Updated weights for policy 0, policy_version 1229224 (0.0010) [2023-12-27 00:20:15,212][105620] Updated weights for policy 1, policy_version 1230378 (0.0006) [2023-12-27 00:20:15,281][105620] Updated weights for policy 1, policy_version 1230388 (0.0006) [2023-12-27 00:20:15,342][105620] Updated weights for policy 1, policy_version 1230398 (0.0008) [2023-12-27 00:20:15,537][105692] Updated weights for policy 0, policy_version 1229234 (0.0008) [2023-12-27 00:20:15,599][105692] Updated weights for policy 0, policy_version 1229244 (0.0005) [2023-12-27 00:20:15,656][105692] Updated weights for policy 0, policy_version 1229254 (0.0005) [2023-12-27 00:20:15,710][105692] Updated weights for policy 0, policy_version 1229264 (0.0005) [2023-12-27 00:20:16,029][105620] Updated weights for policy 1, policy_version 1230408 (0.0009) [2023-12-27 00:20:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 629768192. Throughput: 0: 9913.1, 1: 9681.4. Samples: 629738940. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:20:16,063][104569] Avg episode reward: [(0, '7459.112'), (1, '9165.657')] [2023-12-27 00:20:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001229264_314744832.pth... [2023-12-27 00:20:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001228112_314449920.pth [2023-12-27 00:20:16,080][105620] Updated weights for policy 1, policy_version 1230418 (0.0009) [2023-12-27 00:20:16,132][105620] Updated weights for policy 1, policy_version 1230428 (0.0009) [2023-12-27 00:20:16,151][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001230432_315031552.pth... [2023-12-27 00:20:16,156][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001229280_314736640.pth [2023-12-27 00:20:16,344][105692] Updated weights for policy 0, policy_version 1229274 (0.0007) [2023-12-27 00:20:16,399][105692] Updated weights for policy 0, policy_version 1229284 (0.0007) [2023-12-27 00:20:16,451][105692] Updated weights for policy 0, policy_version 1229294 (0.0006) [2023-12-27 00:20:16,991][105620] Updated weights for policy 1, policy_version 1230438 (0.0009) [2023-12-27 00:20:17,025][105692] Updated weights for policy 0, policy_version 1229304 (0.0008) [2023-12-27 00:20:17,055][105620] Updated weights for policy 1, policy_version 1230448 (0.0007) [2023-12-27 00:20:17,077][105692] Updated weights for policy 0, policy_version 1229314 (0.0008) [2023-12-27 00:20:17,114][105620] Updated weights for policy 1, policy_version 1230458 (0.0006) [2023-12-27 00:20:17,133][105692] Updated weights for policy 0, policy_version 1229324 (0.0008) [2023-12-27 00:20:17,735][105620] Updated weights for policy 1, policy_version 1230468 (0.0006) [2023-12-27 00:20:17,800][105620] Updated weights for policy 1, policy_version 1230478 (0.0006) [2023-12-27 00:20:17,859][105620] Updated weights for policy 1, policy_version 1230488 (0.0006) [2023-12-27 00:20:17,931][105692] Updated weights for policy 0, policy_version 1229334 (0.0010) [2023-12-27 00:20:17,986][105692] Updated weights for policy 0, policy_version 1229344 (0.0010) [2023-12-27 00:20:18,034][105692] Updated weights for policy 0, policy_version 1229354 (0.0010) [2023-12-27 00:20:18,469][105620] Updated weights for policy 1, policy_version 1230498 (0.0006) [2023-12-27 00:20:18,524][105620] Updated weights for policy 1, policy_version 1230508 (0.0007) [2023-12-27 00:20:18,573][105620] Updated weights for policy 1, policy_version 1230518 (0.0008) [2023-12-27 00:20:18,621][105620] Updated weights for policy 1, policy_version 1230528 (0.0006) [2023-12-27 00:20:18,794][105692] Updated weights for policy 0, policy_version 1229364 (0.0011) [2023-12-27 00:20:18,860][105692] Updated weights for policy 0, policy_version 1229374 (0.0008) [2023-12-27 00:20:18,909][105692] Updated weights for policy 0, policy_version 1229384 (0.0005) [2023-12-27 00:20:19,373][105620] Updated weights for policy 1, policy_version 1230538 (0.0007) [2023-12-27 00:20:19,437][105620] Updated weights for policy 1, policy_version 1230548 (0.0007) [2023-12-27 00:20:19,506][105620] Updated weights for policy 1, policy_version 1230558 (0.0008) [2023-12-27 00:20:19,551][105692] Updated weights for policy 0, policy_version 1229394 (0.0010) [2023-12-27 00:20:19,618][105692] Updated weights for policy 0, policy_version 1229404 (0.0007) [2023-12-27 00:20:19,678][105692] Updated weights for policy 0, policy_version 1229414 (0.0007) [2023-12-27 00:20:19,744][105692] Updated weights for policy 0, policy_version 1229424 (0.0011) [2023-12-27 00:20:20,173][105620] Updated weights for policy 1, policy_version 1230568 (0.0008) [2023-12-27 00:20:20,232][105620] Updated weights for policy 1, policy_version 1230578 (0.0009) [2023-12-27 00:20:20,291][105620] Updated weights for policy 1, policy_version 1230588 (0.0009) [2023-12-27 00:20:20,421][105692] Updated weights for policy 0, policy_version 1229434 (0.0009) [2023-12-27 00:20:20,488][105692] Updated weights for policy 0, policy_version 1229444 (0.0009) [2023-12-27 00:20:20,557][105692] Updated weights for policy 0, policy_version 1229454 (0.0010) [2023-12-27 00:20:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 629866496. Throughput: 0: 9890.3, 1: 9703.8. Samples: 629860932. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:20:21,062][104569] Avg episode reward: [(0, '7824.851'), (1, '9257.855')] [2023-12-27 00:20:21,069][105620] Updated weights for policy 1, policy_version 1230598 (0.0010) [2023-12-27 00:20:21,130][105620] Updated weights for policy 1, policy_version 1230608 (0.0008) [2023-12-27 00:20:21,193][105620] Updated weights for policy 1, policy_version 1230618 (0.0007) [2023-12-27 00:20:21,292][105692] Updated weights for policy 0, policy_version 1229464 (0.0009) [2023-12-27 00:20:21,362][105692] Updated weights for policy 0, policy_version 1229474 (0.0011) [2023-12-27 00:20:21,425][105692] Updated weights for policy 0, policy_version 1229484 (0.0009) [2023-12-27 00:20:22,013][105620] Updated weights for policy 1, policy_version 1230628 (0.0006) [2023-12-27 00:20:22,078][105620] Updated weights for policy 1, policy_version 1230638 (0.0008) [2023-12-27 00:20:22,134][105620] Updated weights for policy 1, policy_version 1230648 (0.0008) [2023-12-27 00:20:22,136][105692] Updated weights for policy 0, policy_version 1229494 (0.0010) [2023-12-27 00:20:22,195][105692] Updated weights for policy 0, policy_version 1229504 (0.0009) [2023-12-27 00:20:22,256][105692] Updated weights for policy 0, policy_version 1229514 (0.0010) [2023-12-27 00:20:22,869][105620] Updated weights for policy 1, policy_version 1230658 (0.0009) [2023-12-27 00:20:22,927][105620] Updated weights for policy 1, policy_version 1230668 (0.0008) [2023-12-27 00:20:22,974][105620] Updated weights for policy 1, policy_version 1230678 (0.0008) [2023-12-27 00:20:23,026][105620] Updated weights for policy 1, policy_version 1230688 (0.0008) [2023-12-27 00:20:23,028][105692] Updated weights for policy 0, policy_version 1229524 (0.0011) [2023-12-27 00:20:23,084][105692] Updated weights for policy 0, policy_version 1229534 (0.0010) [2023-12-27 00:20:23,145][105692] Updated weights for policy 0, policy_version 1229544 (0.0010) [2023-12-27 00:20:23,789][105620] Updated weights for policy 1, policy_version 1230698 (0.0010) [2023-12-27 00:20:23,846][105620] Updated weights for policy 1, policy_version 1230708 (0.0010) [2023-12-27 00:20:23,858][105692] Updated weights for policy 0, policy_version 1229554 (0.0010) [2023-12-27 00:20:23,901][105620] Updated weights for policy 1, policy_version 1230718 (0.0010) [2023-12-27 00:20:23,913][105692] Updated weights for policy 0, policy_version 1229564 (0.0010) [2023-12-27 00:20:23,968][105692] Updated weights for policy 0, policy_version 1229574 (0.0010) [2023-12-27 00:20:24,028][105692] Updated weights for policy 0, policy_version 1229584 (0.0010) [2023-12-27 00:20:24,585][105620] Updated weights for policy 1, policy_version 1230728 (0.0009) [2023-12-27 00:20:24,635][105620] Updated weights for policy 1, policy_version 1230738 (0.0005) [2023-12-27 00:20:24,675][105692] Updated weights for policy 0, policy_version 1229594 (0.0010) [2023-12-27 00:20:24,682][105620] Updated weights for policy 1, policy_version 1230748 (0.0005) [2023-12-27 00:20:24,737][105692] Updated weights for policy 0, policy_version 1229604 (0.0010) [2023-12-27 00:20:24,795][105692] Updated weights for policy 0, policy_version 1229614 (0.0010) [2023-12-27 00:20:25,371][105620] Updated weights for policy 1, policy_version 1230758 (0.0008) [2023-12-27 00:20:25,417][105692] Updated weights for policy 0, policy_version 1229624 (0.0010) [2023-12-27 00:20:25,423][105620] Updated weights for policy 1, policy_version 1230768 (0.0010) [2023-12-27 00:20:25,472][105620] Updated weights for policy 1, policy_version 1230778 (0.0010) [2023-12-27 00:20:25,472][105692] Updated weights for policy 0, policy_version 1229634 (0.0010) [2023-12-27 00:20:25,527][105692] Updated weights for policy 0, policy_version 1229644 (0.0010) [2023-12-27 00:20:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 629964800. Throughput: 0: 9903.0, 1: 9680.4. Samples: 629976788. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:20:26,062][104569] Avg episode reward: [(0, '8363.789'), (1, '9076.779')] [2023-12-27 00:20:26,174][105692] Updated weights for policy 0, policy_version 1229654 (0.0011) [2023-12-27 00:20:26,178][105620] Updated weights for policy 1, policy_version 1230788 (0.0011) [2023-12-27 00:20:26,223][105620] Updated weights for policy 1, policy_version 1230798 (0.0010) [2023-12-27 00:20:26,232][105692] Updated weights for policy 0, policy_version 1229664 (0.0010) [2023-12-27 00:20:26,269][105620] Updated weights for policy 1, policy_version 1230808 (0.0010) [2023-12-27 00:20:26,290][105692] Updated weights for policy 0, policy_version 1229674 (0.0010) [2023-12-27 00:20:26,960][105620] Updated weights for policy 1, policy_version 1230818 (0.0010) [2023-12-27 00:20:27,018][105620] Updated weights for policy 1, policy_version 1230828 (0.0005) [2023-12-27 00:20:27,076][105620] Updated weights for policy 1, policy_version 1230838 (0.0005) [2023-12-27 00:20:27,089][105692] Updated weights for policy 0, policy_version 1229684 (0.0010) [2023-12-27 00:20:27,135][105620] Updated weights for policy 1, policy_version 1230848 (0.0010) [2023-12-27 00:20:27,137][105692] Updated weights for policy 0, policy_version 1229694 (0.0007) [2023-12-27 00:20:27,187][105692] Updated weights for policy 0, policy_version 1229704 (0.0005) [2023-12-27 00:20:27,752][105692] Updated weights for policy 0, policy_version 1229714 (0.0005) [2023-12-27 00:20:27,805][105692] Updated weights for policy 0, policy_version 1229724 (0.0005) [2023-12-27 00:20:27,824][105620] Updated weights for policy 1, policy_version 1230858 (0.0010) [2023-12-27 00:20:27,857][105692] Updated weights for policy 0, policy_version 1229734 (0.0005) [2023-12-27 00:20:27,882][105620] Updated weights for policy 1, policy_version 1230868 (0.0010) [2023-12-27 00:20:27,907][105692] Updated weights for policy 0, policy_version 1229744 (0.0005) [2023-12-27 00:20:27,933][105620] Updated weights for policy 1, policy_version 1230878 (0.0010) [2023-12-27 00:20:28,489][105692] Updated weights for policy 0, policy_version 1229754 (0.0008) [2023-12-27 00:20:28,547][105692] Updated weights for policy 0, policy_version 1229764 (0.0008) [2023-12-27 00:20:28,603][105692] Updated weights for policy 0, policy_version 1229774 (0.0008) [2023-12-27 00:20:28,688][105620] Updated weights for policy 1, policy_version 1230888 (0.0010) [2023-12-27 00:20:28,740][105620] Updated weights for policy 1, policy_version 1230898 (0.0008) [2023-12-27 00:20:28,786][105620] Updated weights for policy 1, policy_version 1230908 (0.0005) [2023-12-27 00:20:29,340][105692] Updated weights for policy 0, policy_version 1229784 (0.0008) [2023-12-27 00:20:29,401][105692] Updated weights for policy 0, policy_version 1229794 (0.0008) [2023-12-27 00:20:29,460][105692] Updated weights for policy 0, policy_version 1229804 (0.0008) [2023-12-27 00:20:29,501][105620] Updated weights for policy 1, policy_version 1230918 (0.0009) [2023-12-27 00:20:29,560][105620] Updated weights for policy 1, policy_version 1230928 (0.0008) [2023-12-27 00:20:29,611][105620] Updated weights for policy 1, policy_version 1230938 (0.0010) [2023-12-27 00:20:30,231][105692] Updated weights for policy 0, policy_version 1229814 (0.0008) [2023-12-27 00:20:30,294][105692] Updated weights for policy 0, policy_version 1229824 (0.0006) [2023-12-27 00:20:30,360][105692] Updated weights for policy 0, policy_version 1229834 (0.0006) [2023-12-27 00:20:30,378][105620] Updated weights for policy 1, policy_version 1230948 (0.0011) [2023-12-27 00:20:30,445][105620] Updated weights for policy 1, policy_version 1230958 (0.0011) [2023-12-27 00:20:30,499][105620] Updated weights for policy 1, policy_version 1230968 (0.0010) [2023-12-27 00:20:31,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 630063104. Throughput: 0: 9956.4, 1: 9717.5. Samples: 630038592. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:20:31,063][104569] Avg episode reward: [(0, '8358.106'), (1, '9077.097')] [2023-12-27 00:20:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001229840_314892288.pth... [2023-12-27 00:20:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001228688_314597376.pth [2023-12-27 00:20:31,081][105620] Updated weights for policy 1, policy_version 1230978 (0.0007) [2023-12-27 00:20:31,115][105692] Updated weights for policy 0, policy_version 1229844 (0.0005) [2023-12-27 00:20:31,139][105620] Updated weights for policy 1, policy_version 1230988 (0.0010) [2023-12-27 00:20:31,178][105692] Updated weights for policy 0, policy_version 1229854 (0.0007) [2023-12-27 00:20:31,192][105620] Updated weights for policy 1, policy_version 1230998 (0.0007) [2023-12-27 00:20:31,237][105692] Updated weights for policy 0, policy_version 1229864 (0.0007) [2023-12-27 00:20:31,243][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001231008_315179008.pth... [2023-12-27 00:20:31,244][105620] Updated weights for policy 1, policy_version 1231008 (0.0008) [2023-12-27 00:20:31,247][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001229856_314884096.pth [2023-12-27 00:20:31,959][105692] Updated weights for policy 0, policy_version 1229874 (0.0007) [2023-12-27 00:20:32,006][105620] Updated weights for policy 1, policy_version 1231018 (0.0011) [2023-12-27 00:20:32,010][105692] Updated weights for policy 0, policy_version 1229884 (0.0007) [2023-12-27 00:20:32,056][105692] Updated weights for policy 0, policy_version 1229894 (0.0006) [2023-12-27 00:20:32,064][105620] Updated weights for policy 1, policy_version 1231028 (0.0010) [2023-12-27 00:20:32,118][105692] Updated weights for policy 0, policy_version 1229904 (0.0009) [2023-12-27 00:20:32,123][105620] Updated weights for policy 1, policy_version 1231038 (0.0011) [2023-12-27 00:20:32,861][105620] Updated weights for policy 1, policy_version 1231048 (0.0010) [2023-12-27 00:20:32,879][105692] Updated weights for policy 0, policy_version 1229914 (0.0006) [2023-12-27 00:20:32,910][105620] Updated weights for policy 1, policy_version 1231058 (0.0011) [2023-12-27 00:20:32,936][105692] Updated weights for policy 0, policy_version 1229924 (0.0006) [2023-12-27 00:20:32,958][105620] Updated weights for policy 1, policy_version 1231068 (0.0011) [2023-12-27 00:20:32,990][105692] Updated weights for policy 0, policy_version 1229934 (0.0007) [2023-12-27 00:20:33,525][105620] Updated weights for policy 1, policy_version 1231078 (0.0011) [2023-12-27 00:20:33,572][105620] Updated weights for policy 1, policy_version 1231088 (0.0010) [2023-12-27 00:20:33,626][105620] Updated weights for policy 1, policy_version 1231098 (0.0010) [2023-12-27 00:20:33,812][105692] Updated weights for policy 0, policy_version 1229944 (0.0010) [2023-12-27 00:20:33,868][105692] Updated weights for policy 0, policy_version 1229955 (0.0009) [2023-12-27 00:20:33,926][105692] Updated weights for policy 0, policy_version 1229966 (0.0010) [2023-12-27 00:20:34,216][105620] Updated weights for policy 1, policy_version 1231108 (0.0008) [2023-12-27 00:20:34,276][105620] Updated weights for policy 1, policy_version 1231118 (0.0006) [2023-12-27 00:20:34,340][105620] Updated weights for policy 1, policy_version 1231128 (0.0008) [2023-12-27 00:20:34,781][105692] Updated weights for policy 0, policy_version 1229976 (0.0009) [2023-12-27 00:20:34,838][105692] Updated weights for policy 0, policy_version 1229986 (0.0008) [2023-12-27 00:20:34,895][105692] Updated weights for policy 0, policy_version 1229996 (0.0009) [2023-12-27 00:20:35,017][105620] Updated weights for policy 1, policy_version 1231138 (0.0008) [2023-12-27 00:20:35,078][105620] Updated weights for policy 1, policy_version 1231148 (0.0008) [2023-12-27 00:20:35,135][105620] Updated weights for policy 1, policy_version 1231158 (0.0009) [2023-12-27 00:20:35,187][105620] Updated weights for policy 1, policy_version 1231168 (0.0009) [2023-12-27 00:20:35,690][105692] Updated weights for policy 0, policy_version 1230006 (0.0010) [2023-12-27 00:20:35,740][105692] Updated weights for policy 0, policy_version 1230016 (0.0009) [2023-12-27 00:20:35,792][105692] Updated weights for policy 0, policy_version 1230026 (0.0010) [2023-12-27 00:20:35,890][105620] Updated weights for policy 1, policy_version 1231178 (0.0009) [2023-12-27 00:20:35,943][105620] Updated weights for policy 1, policy_version 1231188 (0.0009) [2023-12-27 00:20:35,990][105620] Updated weights for policy 1, policy_version 1231198 (0.0009) [2023-12-27 00:20:36,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 630169600. Throughput: 0: 9771.0, 1: 9773.8. Samples: 630154528. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:20:36,062][104569] Avg episode reward: [(0, '8271.061'), (1, '9258.903')] [2023-12-27 00:20:36,565][105692] Updated weights for policy 0, policy_version 1230036 (0.0009) [2023-12-27 00:20:36,637][105692] Updated weights for policy 0, policy_version 1230046 (0.0010) [2023-12-27 00:20:36,692][105692] Updated weights for policy 0, policy_version 1230056 (0.0009) [2023-12-27 00:20:36,763][105620] Updated weights for policy 1, policy_version 1231208 (0.0007) [2023-12-27 00:20:36,829][105620] Updated weights for policy 1, policy_version 1231218 (0.0005) [2023-12-27 00:20:36,895][105620] Updated weights for policy 1, policy_version 1231228 (0.0005) [2023-12-27 00:20:37,400][105620] Updated weights for policy 1, policy_version 1231238 (0.0005) [2023-12-27 00:20:37,452][105620] Updated weights for policy 1, policy_version 1231248 (0.0006) [2023-12-27 00:20:37,512][105620] Updated weights for policy 1, policy_version 1231258 (0.0006) [2023-12-27 00:20:37,567][105692] Updated weights for policy 0, policy_version 1230066 (0.0008) [2023-12-27 00:20:37,629][105692] Updated weights for policy 0, policy_version 1230076 (0.0010) [2023-12-27 00:20:37,690][105692] Updated weights for policy 0, policy_version 1230086 (0.0010) [2023-12-27 00:20:37,745][105692] Updated weights for policy 0, policy_version 1230096 (0.0011) [2023-12-27 00:20:38,047][105620] Updated weights for policy 1, policy_version 1231268 (0.0005) [2023-12-27 00:20:38,097][105620] Updated weights for policy 1, policy_version 1231278 (0.0006) [2023-12-27 00:20:38,148][105620] Updated weights for policy 1, policy_version 1231288 (0.0006) [2023-12-27 00:20:38,592][105692] Updated weights for policy 0, policy_version 1230106 (0.0009) [2023-12-27 00:20:38,648][105692] Updated weights for policy 0, policy_version 1230116 (0.0009) [2023-12-27 00:20:38,702][105692] Updated weights for policy 0, policy_version 1230126 (0.0009) [2023-12-27 00:20:38,868][105620] Updated weights for policy 1, policy_version 1231298 (0.0010) [2023-12-27 00:20:38,932][105620] Updated weights for policy 1, policy_version 1231308 (0.0011) [2023-12-27 00:20:38,990][105620] Updated weights for policy 1, policy_version 1231318 (0.0010) [2023-12-27 00:20:39,054][105620] Updated weights for policy 1, policy_version 1231328 (0.0010) [2023-12-27 00:20:39,510][105692] Updated weights for policy 0, policy_version 1230136 (0.0008) [2023-12-27 00:20:39,567][105692] Updated weights for policy 0, policy_version 1230146 (0.0008) [2023-12-27 00:20:39,628][105692] Updated weights for policy 0, policy_version 1230156 (0.0008) [2023-12-27 00:20:39,805][105620] Updated weights for policy 1, policy_version 1231338 (0.0011) [2023-12-27 00:20:39,876][105620] Updated weights for policy 1, policy_version 1231348 (0.0011) [2023-12-27 00:20:39,944][105620] Updated weights for policy 1, policy_version 1231358 (0.0010) [2023-12-27 00:20:40,370][105692] Updated weights for policy 0, policy_version 1230166 (0.0007) [2023-12-27 00:20:40,427][105692] Updated weights for policy 0, policy_version 1230176 (0.0006) [2023-12-27 00:20:40,492][105692] Updated weights for policy 0, policy_version 1230186 (0.0006) [2023-12-27 00:20:40,727][105620] Updated weights for policy 1, policy_version 1231368 (0.0008) [2023-12-27 00:20:40,779][105620] Updated weights for policy 1, policy_version 1231378 (0.0008) [2023-12-27 00:20:40,834][105620] Updated weights for policy 1, policy_version 1231388 (0.0008) [2023-12-27 00:20:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 630259712. Throughput: 0: 9702.4, 1: 9816.8. Samples: 630267184. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:20:41,062][104569] Avg episode reward: [(0, '8095.420'), (1, '9167.002')] [2023-12-27 00:20:41,221][105692] Updated weights for policy 0, policy_version 1230196 (0.0010) [2023-12-27 00:20:41,294][105692] Updated weights for policy 0, policy_version 1230206 (0.0011) [2023-12-27 00:20:41,359][105692] Updated weights for policy 0, policy_version 1230216 (0.0011) [2023-12-27 00:20:41,669][105620] Updated weights for policy 1, policy_version 1231398 (0.0008) [2023-12-27 00:20:41,736][105620] Updated weights for policy 1, policy_version 1231408 (0.0008) [2023-12-27 00:20:41,796][105620] Updated weights for policy 1, policy_version 1231418 (0.0008) [2023-12-27 00:20:42,157][105692] Updated weights for policy 0, policy_version 1230226 (0.0010) [2023-12-27 00:20:42,220][105692] Updated weights for policy 0, policy_version 1230236 (0.0010) [2023-12-27 00:20:42,287][105692] Updated weights for policy 0, policy_version 1230246 (0.0010) [2023-12-27 00:20:42,344][105692] Updated weights for policy 0, policy_version 1230256 (0.0010) [2023-12-27 00:20:42,580][105620] Updated weights for policy 1, policy_version 1231428 (0.0009) [2023-12-27 00:20:42,635][105620] Updated weights for policy 1, policy_version 1231438 (0.0009) [2023-12-27 00:20:42,706][105620] Updated weights for policy 1, policy_version 1231448 (0.0009) [2023-12-27 00:20:42,988][105692] Updated weights for policy 0, policy_version 1230266 (0.0005) [2023-12-27 00:20:43,040][105692] Updated weights for policy 0, policy_version 1230276 (0.0005) [2023-12-27 00:20:43,096][105692] Updated weights for policy 0, policy_version 1230286 (0.0009) [2023-12-27 00:20:43,501][105620] Updated weights for policy 1, policy_version 1231458 (0.0009) [2023-12-27 00:20:43,552][105620] Updated weights for policy 1, policy_version 1231468 (0.0010) [2023-12-27 00:20:43,608][105620] Updated weights for policy 1, policy_version 1231478 (0.0010) [2023-12-27 00:20:43,653][105692] Updated weights for policy 0, policy_version 1230296 (0.0006) [2023-12-27 00:20:43,660][105620] Updated weights for policy 1, policy_version 1231488 (0.0010) [2023-12-27 00:20:43,719][105692] Updated weights for policy 0, policy_version 1230306 (0.0005) [2023-12-27 00:20:43,774][105692] Updated weights for policy 0, policy_version 1230316 (0.0005) [2023-12-27 00:20:44,338][105692] Updated weights for policy 0, policy_version 1230326 (0.0005) [2023-12-27 00:20:44,345][105620] Updated weights for policy 1, policy_version 1231498 (0.0007) [2023-12-27 00:20:44,395][105692] Updated weights for policy 0, policy_version 1230336 (0.0006) [2023-12-27 00:20:44,408][105620] Updated weights for policy 1, policy_version 1231508 (0.0010) [2023-12-27 00:20:44,449][105692] Updated weights for policy 0, policy_version 1230346 (0.0006) [2023-12-27 00:20:44,464][105620] Updated weights for policy 1, policy_version 1231518 (0.0010) [2023-12-27 00:20:45,006][105692] Updated weights for policy 0, policy_version 1230356 (0.0007) [2023-12-27 00:20:45,071][105692] Updated weights for policy 0, policy_version 1230366 (0.0007) [2023-12-27 00:20:45,123][105692] Updated weights for policy 0, policy_version 1230376 (0.0009) [2023-12-27 00:20:45,196][105620] Updated weights for policy 1, policy_version 1231528 (0.0008) [2023-12-27 00:20:45,254][105620] Updated weights for policy 1, policy_version 1231538 (0.0009) [2023-12-27 00:20:45,316][105620] Updated weights for policy 1, policy_version 1231548 (0.0008) [2023-12-27 00:20:45,879][105692] Updated weights for policy 0, policy_version 1230386 (0.0009) [2023-12-27 00:20:45,933][105692] Updated weights for policy 0, policy_version 1230396 (0.0009) [2023-12-27 00:20:45,987][105692] Updated weights for policy 0, policy_version 1230406 (0.0009) [2023-12-27 00:20:46,051][105692] Updated weights for policy 0, policy_version 1230416 (0.0007) [2023-12-27 00:20:46,057][105620] Updated weights for policy 1, policy_version 1231558 (0.0008) [2023-12-27 00:20:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.3, 300 sec: 19605.2). Total num frames: 630358016. Throughput: 0: 9719.7, 1: 9779.5. Samples: 630324168. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:20:46,063][104569] Avg episode reward: [(0, '8364.694'), (1, '9257.816')] [2023-12-27 00:20:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001230416_315039744.pth... [2023-12-27 00:20:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001229264_314744832.pth [2023-12-27 00:20:46,114][105620] Updated weights for policy 1, policy_version 1231568 (0.0006) [2023-12-27 00:20:46,160][105620] Updated weights for policy 1, policy_version 1231578 (0.0005) [2023-12-27 00:20:46,195][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001231584_315326464.pth... [2023-12-27 00:20:46,219][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001230432_315031552.pth [2023-12-27 00:20:46,715][105620] Updated weights for policy 1, policy_version 1231588 (0.0007) [2023-12-27 00:20:46,769][105620] Updated weights for policy 1, policy_version 1231598 (0.0009) [2023-12-27 00:20:46,833][105620] Updated weights for policy 1, policy_version 1231608 (0.0008) [2023-12-27 00:20:46,846][105692] Updated weights for policy 0, policy_version 1230426 (0.0010) [2023-12-27 00:20:46,895][105692] Updated weights for policy 0, policy_version 1230436 (0.0006) [2023-12-27 00:20:46,940][105692] Updated weights for policy 0, policy_version 1230446 (0.0008) [2023-12-27 00:20:47,631][105620] Updated weights for policy 1, policy_version 1231618 (0.0010) [2023-12-27 00:20:47,672][105692] Updated weights for policy 0, policy_version 1230456 (0.0007) [2023-12-27 00:20:47,678][105620] Updated weights for policy 1, policy_version 1231628 (0.0008) [2023-12-27 00:20:47,731][105620] Updated weights for policy 1, policy_version 1231638 (0.0006) [2023-12-27 00:20:47,735][105692] Updated weights for policy 0, policy_version 1230466 (0.0009) [2023-12-27 00:20:47,785][105620] Updated weights for policy 1, policy_version 1231648 (0.0008) [2023-12-27 00:20:47,797][105692] Updated weights for policy 0, policy_version 1230476 (0.0007) [2023-12-27 00:20:48,387][105692] Updated weights for policy 0, policy_version 1230486 (0.0007) [2023-12-27 00:20:48,456][105692] Updated weights for policy 0, policy_version 1230496 (0.0009) [2023-12-27 00:20:48,524][105692] Updated weights for policy 0, policy_version 1230506 (0.0008) [2023-12-27 00:20:48,529][105620] Updated weights for policy 1, policy_version 1231658 (0.0007) [2023-12-27 00:20:48,593][105620] Updated weights for policy 1, policy_version 1231668 (0.0010) [2023-12-27 00:20:48,660][105620] Updated weights for policy 1, policy_version 1231678 (0.0010) [2023-12-27 00:20:49,293][105692] Updated weights for policy 0, policy_version 1230516 (0.0009) [2023-12-27 00:20:49,319][105620] Updated weights for policy 1, policy_version 1231688 (0.0009) [2023-12-27 00:20:49,355][105692] Updated weights for policy 0, policy_version 1230526 (0.0007) [2023-12-27 00:20:49,386][105620] Updated weights for policy 1, policy_version 1231698 (0.0009) [2023-12-27 00:20:49,408][105692] Updated weights for policy 0, policy_version 1230536 (0.0006) [2023-12-27 00:20:49,454][105620] Updated weights for policy 1, policy_version 1231708 (0.0008) [2023-12-27 00:20:50,061][105620] Updated weights for policy 1, policy_version 1231718 (0.0008) [2023-12-27 00:20:50,118][105620] Updated weights for policy 1, policy_version 1231728 (0.0008) [2023-12-27 00:20:50,174][105620] Updated weights for policy 1, policy_version 1231738 (0.0008) [2023-12-27 00:20:50,202][105692] Updated weights for policy 0, policy_version 1230546 (0.0009) [2023-12-27 00:20:50,250][105692] Updated weights for policy 0, policy_version 1230556 (0.0010) [2023-12-27 00:20:50,294][105692] Updated weights for policy 0, policy_version 1230566 (0.0010) [2023-12-27 00:20:50,350][105692] Updated weights for policy 0, policy_version 1230576 (0.0010) [2023-12-27 00:20:50,883][105620] Updated weights for policy 1, policy_version 1231748 (0.0006) [2023-12-27 00:20:50,947][105620] Updated weights for policy 1, policy_version 1231758 (0.0005) [2023-12-27 00:20:51,003][105620] Updated weights for policy 1, policy_version 1231768 (0.0005) [2023-12-27 00:20:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 630456320. Throughput: 0: 9697.6, 1: 9872.1. Samples: 630444844. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:20:51,062][104569] Avg episode reward: [(0, '8094.228'), (1, '9167.260')] [2023-12-27 00:20:51,127][105692] Updated weights for policy 0, policy_version 1230586 (0.0011) [2023-12-27 00:20:51,189][105692] Updated weights for policy 0, policy_version 1230596 (0.0009) [2023-12-27 00:20:51,251][105692] Updated weights for policy 0, policy_version 1230606 (0.0010) [2023-12-27 00:20:51,771][105620] Updated weights for policy 1, policy_version 1231778 (0.0008) [2023-12-27 00:20:51,831][105620] Updated weights for policy 1, policy_version 1231788 (0.0008) [2023-12-27 00:20:51,894][105620] Updated weights for policy 1, policy_version 1231798 (0.0010) [2023-12-27 00:20:51,953][105620] Updated weights for policy 1, policy_version 1231808 (0.0008) [2023-12-27 00:20:51,967][105692] Updated weights for policy 0, policy_version 1230616 (0.0007) [2023-12-27 00:20:52,024][105692] Updated weights for policy 0, policy_version 1230626 (0.0006) [2023-12-27 00:20:52,083][105692] Updated weights for policy 0, policy_version 1230636 (0.0010) [2023-12-27 00:20:52,714][105620] Updated weights for policy 1, policy_version 1231818 (0.0008) [2023-12-27 00:20:52,779][105620] Updated weights for policy 1, policy_version 1231828 (0.0007) [2023-12-27 00:20:52,796][105692] Updated weights for policy 0, policy_version 1230646 (0.0010) [2023-12-27 00:20:52,832][105620] Updated weights for policy 1, policy_version 1231838 (0.0006) [2023-12-27 00:20:52,845][105692] Updated weights for policy 0, policy_version 1230656 (0.0010) [2023-12-27 00:20:52,897][105692] Updated weights for policy 0, policy_version 1230666 (0.0008) [2023-12-27 00:20:53,563][105692] Updated weights for policy 0, policy_version 1230676 (0.0005) [2023-12-27 00:20:53,604][105620] Updated weights for policy 1, policy_version 1231848 (0.0008) [2023-12-27 00:20:53,609][105692] Updated weights for policy 0, policy_version 1230686 (0.0005) [2023-12-27 00:20:53,654][105620] Updated weights for policy 1, policy_version 1231858 (0.0009) [2023-12-27 00:20:53,664][105692] Updated weights for policy 0, policy_version 1230696 (0.0009) [2023-12-27 00:20:53,706][105620] Updated weights for policy 1, policy_version 1231868 (0.0006) [2023-12-27 00:20:54,310][105692] Updated weights for policy 0, policy_version 1230706 (0.0010) [2023-12-27 00:20:54,360][105692] Updated weights for policy 0, policy_version 1230716 (0.0009) [2023-12-27 00:20:54,410][105692] Updated weights for policy 0, policy_version 1230726 (0.0009) [2023-12-27 00:20:54,449][105620] Updated weights for policy 1, policy_version 1231878 (0.0008) [2023-12-27 00:20:54,463][105692] Updated weights for policy 0, policy_version 1230736 (0.0006) [2023-12-27 00:20:54,511][105620] Updated weights for policy 1, policy_version 1231888 (0.0008) [2023-12-27 00:20:54,571][105620] Updated weights for policy 1, policy_version 1231898 (0.0009) [2023-12-27 00:20:55,193][105692] Updated weights for policy 0, policy_version 1230746 (0.0011) [2023-12-27 00:20:55,241][105692] Updated weights for policy 0, policy_version 1230756 (0.0010) [2023-12-27 00:20:55,289][105692] Updated weights for policy 0, policy_version 1230766 (0.0010) [2023-12-27 00:20:55,301][105620] Updated weights for policy 1, policy_version 1231908 (0.0010) [2023-12-27 00:20:55,352][105620] Updated weights for policy 1, policy_version 1231918 (0.0008) [2023-12-27 00:20:55,410][105620] Updated weights for policy 1, policy_version 1231928 (0.0008) [2023-12-27 00:20:56,061][105692] Updated weights for policy 0, policy_version 1230776 (0.0010) [2023-12-27 00:20:56,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 630546432. Throughput: 0: 9675.1, 1: 9820.0. Samples: 630559620. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:20:56,063][104569] Avg episode reward: [(0, '8184.569'), (1, '8900.262')] [2023-12-27 00:20:56,127][105692] Updated weights for policy 0, policy_version 1230786 (0.0009) [2023-12-27 00:20:56,131][105620] Updated weights for policy 1, policy_version 1231938 (0.0008) [2023-12-27 00:20:56,183][105620] Updated weights for policy 1, policy_version 1231948 (0.0005) [2023-12-27 00:20:56,186][105692] Updated weights for policy 0, policy_version 1230796 (0.0009) [2023-12-27 00:20:56,230][105620] Updated weights for policy 1, policy_version 1231958 (0.0005) [2023-12-27 00:20:56,278][105620] Updated weights for policy 1, policy_version 1231968 (0.0005) [2023-12-27 00:20:56,819][105620] Updated weights for policy 1, policy_version 1231978 (0.0007) [2023-12-27 00:20:56,869][105620] Updated weights for policy 1, policy_version 1231988 (0.0006) [2023-12-27 00:20:56,922][105692] Updated weights for policy 0, policy_version 1230806 (0.0009) [2023-12-27 00:20:56,928][105620] Updated weights for policy 1, policy_version 1231998 (0.0006) [2023-12-27 00:20:56,983][105692] Updated weights for policy 0, policy_version 1230816 (0.0010) [2023-12-27 00:20:56,993][105585] KL-divergence is very high: 100.7349 [2023-12-27 00:20:57,038][105585] KL-divergence is very high: 117.8506 [2023-12-27 00:20:57,038][105692] Updated weights for policy 0, policy_version 1230826 (0.0010) [2023-12-27 00:20:57,672][105692] Updated weights for policy 0, policy_version 1230836 (0.0008) [2023-12-27 00:20:57,719][105620] Updated weights for policy 1, policy_version 1232008 (0.0008) [2023-12-27 00:20:57,720][105692] Updated weights for policy 0, policy_version 1230846 (0.0005) [2023-12-27 00:20:57,763][105692] Updated weights for policy 0, policy_version 1230856 (0.0005) [2023-12-27 00:20:57,765][105620] Updated weights for policy 1, policy_version 1232018 (0.0008) [2023-12-27 00:20:57,814][105620] Updated weights for policy 1, policy_version 1232028 (0.0009) [2023-12-27 00:20:58,550][105692] Updated weights for policy 0, policy_version 1230866 (0.0006) [2023-12-27 00:20:58,611][105692] Updated weights for policy 0, policy_version 1230876 (0.0008) [2023-12-27 00:20:58,635][105620] Updated weights for policy 1, policy_version 1232038 (0.0008) [2023-12-27 00:20:58,674][105692] Updated weights for policy 0, policy_version 1230886 (0.0008) [2023-12-27 00:20:58,701][105620] Updated weights for policy 1, policy_version 1232048 (0.0008) [2023-12-27 00:20:58,731][105692] Updated weights for policy 0, policy_version 1230896 (0.0007) [2023-12-27 00:20:58,766][105620] Updated weights for policy 1, policy_version 1232058 (0.0008) [2023-12-27 00:20:59,488][105620] Updated weights for policy 1, policy_version 1232068 (0.0007) [2023-12-27 00:20:59,510][105692] Updated weights for policy 0, policy_version 1230906 (0.0009) [2023-12-27 00:20:59,550][105620] Updated weights for policy 1, policy_version 1232078 (0.0007) [2023-12-27 00:20:59,573][105692] Updated weights for policy 0, policy_version 1230916 (0.0006) [2023-12-27 00:20:59,600][105620] Updated weights for policy 1, policy_version 1232088 (0.0009) [2023-12-27 00:20:59,634][105692] Updated weights for policy 0, policy_version 1230926 (0.0009) [2023-12-27 00:21:00,291][105620] Updated weights for policy 1, policy_version 1232098 (0.0007) [2023-12-27 00:21:00,337][105620] Updated weights for policy 1, policy_version 1232108 (0.0008) [2023-12-27 00:21:00,391][105620] Updated weights for policy 1, policy_version 1232118 (0.0008) [2023-12-27 00:21:00,422][105692] Updated weights for policy 0, policy_version 1230936 (0.0010) [2023-12-27 00:21:00,440][105620] Updated weights for policy 1, policy_version 1232128 (0.0006) [2023-12-27 00:21:00,473][105692] Updated weights for policy 0, policy_version 1230946 (0.0009) [2023-12-27 00:21:00,519][105692] Updated weights for policy 0, policy_version 1230956 (0.0008) [2023-12-27 00:21:00,527][105585] KL-divergence is very high: 155.3320 [2023-12-27 00:21:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 630644736. Throughput: 0: 9714.6, 1: 9819.5. Samples: 630617972. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:21:01,062][104569] Avg episode reward: [(0, '7823.424'), (1, '8903.936')] [2023-12-27 00:21:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001230960_315179008.pth... [2023-12-27 00:21:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001232128_315465728.pth... [2023-12-27 00:21:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001229840_314892288.pth [2023-12-27 00:21:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001231008_315179008.pth [2023-12-27 00:21:01,175][105620] Updated weights for policy 1, policy_version 1232138 (0.0007) [2023-12-27 00:21:01,235][105620] Updated weights for policy 1, policy_version 1232148 (0.0009) [2023-12-27 00:21:01,268][105585] KL-divergence is very high: 195.1365 [2023-12-27 00:21:01,294][105585] KL-divergence is very high: 178.4127 [2023-12-27 00:21:01,296][105620] Updated weights for policy 1, policy_version 1232158 (0.0008) [2023-12-27 00:21:01,304][105692] Updated weights for policy 0, policy_version 1230966 (0.0007) [2023-12-27 00:21:01,320][105585] KL-divergence is very high: 200.6108 [2023-12-27 00:21:01,348][105585] KL-divergence is very high: 151.7205 [2023-12-27 00:21:01,377][105692] Updated weights for policy 0, policy_version 1230976 (0.0007) [2023-12-27 00:21:01,377][105585] KL-divergence is very high: 155.9116 [2023-12-27 00:21:01,403][105585] KL-divergence is very high: 114.1931 [2023-12-27 00:21:01,426][105585] KL-divergence is very high: 118.0262 [2023-12-27 00:21:01,437][105692] Updated weights for policy 0, policy_version 1230986 (0.0007) [2023-12-27 00:21:02,042][105620] Updated weights for policy 1, policy_version 1232168 (0.0011) [2023-12-27 00:21:02,104][105620] Updated weights for policy 1, policy_version 1232178 (0.0010) [2023-12-27 00:21:02,158][105692] Updated weights for policy 0, policy_version 1230996 (0.0007) [2023-12-27 00:21:02,168][105620] Updated weights for policy 1, policy_version 1232188 (0.0011) [2023-12-27 00:21:02,224][105692] Updated weights for policy 0, policy_version 1231006 (0.0007) [2023-12-27 00:21:02,287][105692] Updated weights for policy 0, policy_version 1231016 (0.0008) [2023-12-27 00:21:02,899][105620] Updated weights for policy 1, policy_version 1232198 (0.0010) [2023-12-27 00:21:02,950][105620] Updated weights for policy 1, policy_version 1232208 (0.0010) [2023-12-27 00:21:03,000][105620] Updated weights for policy 1, policy_version 1232218 (0.0006) [2023-12-27 00:21:03,043][105692] Updated weights for policy 0, policy_version 1231026 (0.0008) [2023-12-27 00:21:03,096][105692] Updated weights for policy 0, policy_version 1231036 (0.0009) [2023-12-27 00:21:03,147][105692] Updated weights for policy 0, policy_version 1231046 (0.0009) [2023-12-27 00:21:03,204][105692] Updated weights for policy 0, policy_version 1231056 (0.0009) [2023-12-27 00:21:03,576][105620] Updated weights for policy 1, policy_version 1232228 (0.0007) [2023-12-27 00:21:03,636][105620] Updated weights for policy 1, policy_version 1232238 (0.0009) [2023-12-27 00:21:03,702][105620] Updated weights for policy 1, policy_version 1232248 (0.0010) [2023-12-27 00:21:03,944][105692] Updated weights for policy 0, policy_version 1231066 (0.0007) [2023-12-27 00:21:04,012][105692] Updated weights for policy 0, policy_version 1231076 (0.0006) [2023-12-27 00:21:04,065][105692] Updated weights for policy 0, policy_version 1231086 (0.0009) [2023-12-27 00:21:04,434][105620] Updated weights for policy 1, policy_version 1232258 (0.0009) [2023-12-27 00:21:04,499][105620] Updated weights for policy 1, policy_version 1232268 (0.0008) [2023-12-27 00:21:04,563][105620] Updated weights for policy 1, policy_version 1232278 (0.0009) [2023-12-27 00:21:04,630][105620] Updated weights for policy 1, policy_version 1232288 (0.0006) [2023-12-27 00:21:04,818][105692] Updated weights for policy 0, policy_version 1231096 (0.0006) [2023-12-27 00:21:04,865][105692] Updated weights for policy 0, policy_version 1231106 (0.0007) [2023-12-27 00:21:04,913][105692] Updated weights for policy 0, policy_version 1231116 (0.0005) [2023-12-27 00:21:05,393][105620] Updated weights for policy 1, policy_version 1232298 (0.0009) [2023-12-27 00:21:05,459][105620] Updated weights for policy 1, policy_version 1232308 (0.0006) [2023-12-27 00:21:05,489][105692] Updated weights for policy 0, policy_version 1231126 (0.0005) [2023-12-27 00:21:05,509][105620] Updated weights for policy 1, policy_version 1232318 (0.0009) [2023-12-27 00:21:05,544][105692] Updated weights for policy 0, policy_version 1231136 (0.0008) [2023-12-27 00:21:05,603][105692] Updated weights for policy 0, policy_version 1231146 (0.0009) [2023-12-27 00:21:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 630743040. Throughput: 0: 9585.7, 1: 9773.2. Samples: 630732084. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:21:06,062][104569] Avg episode reward: [(0, '8185.703'), (1, '8991.657')] [2023-12-27 00:21:06,104][105620] Updated weights for policy 1, policy_version 1232328 (0.0006) [2023-12-27 00:21:06,165][105620] Updated weights for policy 1, policy_version 1232338 (0.0007) [2023-12-27 00:21:06,197][105692] Updated weights for policy 0, policy_version 1231156 (0.0007) [2023-12-27 00:21:06,232][105620] Updated weights for policy 1, policy_version 1232348 (0.0008) [2023-12-27 00:21:06,259][105692] Updated weights for policy 0, policy_version 1231166 (0.0006) [2023-12-27 00:21:06,316][105692] Updated weights for policy 0, policy_version 1231176 (0.0008) [2023-12-27 00:21:06,871][105620] Updated weights for policy 1, policy_version 1232358 (0.0011) [2023-12-27 00:21:06,894][105692] Updated weights for policy 0, policy_version 1231186 (0.0006) [2023-12-27 00:21:06,935][105620] Updated weights for policy 1, policy_version 1232368 (0.0009) [2023-12-27 00:21:06,951][105692] Updated weights for policy 0, policy_version 1231196 (0.0009) [2023-12-27 00:21:06,991][105620] Updated weights for policy 1, policy_version 1232378 (0.0010) [2023-12-27 00:21:07,015][105692] Updated weights for policy 0, policy_version 1231206 (0.0006) [2023-12-27 00:21:07,085][105692] Updated weights for policy 0, policy_version 1231216 (0.0005) [2023-12-27 00:21:07,653][105692] Updated weights for policy 0, policy_version 1231226 (0.0008) [2023-12-27 00:21:07,663][105620] Updated weights for policy 1, policy_version 1232388 (0.0007) [2023-12-27 00:21:07,705][105692] Updated weights for policy 0, policy_version 1231236 (0.0006) [2023-12-27 00:21:07,721][105620] Updated weights for policy 1, policy_version 1232398 (0.0010) [2023-12-27 00:21:07,753][105692] Updated weights for policy 0, policy_version 1231246 (0.0008) [2023-12-27 00:21:07,773][105620] Updated weights for policy 1, policy_version 1232408 (0.0010) [2023-12-27 00:21:08,501][105692] Updated weights for policy 0, policy_version 1231256 (0.0008) [2023-12-27 00:21:08,534][105620] Updated weights for policy 1, policy_version 1232418 (0.0010) [2023-12-27 00:21:08,562][105692] Updated weights for policy 0, policy_version 1231266 (0.0009) [2023-12-27 00:21:08,583][105586] KL-divergence is very high: 174.6075 [2023-12-27 00:21:08,590][105620] Updated weights for policy 1, policy_version 1232428 (0.0010) [2023-12-27 00:21:08,619][105692] Updated weights for policy 0, policy_version 1231276 (0.0008) [2023-12-27 00:21:08,629][105586] KL-divergence is very high: 319.2715 [2023-12-27 00:21:08,645][105620] Updated weights for policy 1, policy_version 1232438 (0.0010) [2023-12-27 00:21:08,669][105586] KL-divergence is very high: 332.9422 [2023-12-27 00:21:08,693][105620] Updated weights for policy 1, policy_version 1232448 (0.0010) [2023-12-27 00:21:09,407][105692] Updated weights for policy 0, policy_version 1231286 (0.0008) [2023-12-27 00:21:09,460][105620] Updated weights for policy 1, policy_version 1232458 (0.0006) [2023-12-27 00:21:09,469][105692] Updated weights for policy 0, policy_version 1231296 (0.0008) [2023-12-27 00:21:09,520][105620] Updated weights for policy 1, policy_version 1232468 (0.0007) [2023-12-27 00:21:09,535][105692] Updated weights for policy 0, policy_version 1231306 (0.0009) [2023-12-27 00:21:09,578][105620] Updated weights for policy 1, policy_version 1232478 (0.0007) [2023-12-27 00:21:10,317][105620] Updated weights for policy 1, policy_version 1232488 (0.0006) [2023-12-27 00:21:10,319][105692] Updated weights for policy 0, policy_version 1231316 (0.0007) [2023-12-27 00:21:10,372][105620] Updated weights for policy 1, policy_version 1232498 (0.0008) [2023-12-27 00:21:10,383][105692] Updated weights for policy 0, policy_version 1231326 (0.0007) [2023-12-27 00:21:10,422][105620] Updated weights for policy 1, policy_version 1232508 (0.0006) [2023-12-27 00:21:10,444][105692] Updated weights for policy 0, policy_version 1231336 (0.0007) [2023-12-27 00:21:11,002][105620] Updated weights for policy 1, policy_version 1232518 (0.0006) [2023-12-27 00:21:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 630841344. Throughput: 0: 9642.8, 1: 9848.2. Samples: 630853884. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:21:11,063][104569] Avg episode reward: [(0, '8996.712'), (1, '8549.274')] [2023-12-27 00:21:11,064][105620] Updated weights for policy 1, policy_version 1232528 (0.0009) [2023-12-27 00:21:11,123][105620] Updated weights for policy 1, policy_version 1232538 (0.0008) [2023-12-27 00:21:11,261][105692] Updated weights for policy 0, policy_version 1231346 (0.0009) [2023-12-27 00:21:11,310][105692] Updated weights for policy 0, policy_version 1231356 (0.0009) [2023-12-27 00:21:11,372][105692] Updated weights for policy 0, policy_version 1231366 (0.0009) [2023-12-27 00:21:11,439][105692] Updated weights for policy 0, policy_version 1231376 (0.0009) [2023-12-27 00:21:11,830][105620] Updated weights for policy 1, policy_version 1232548 (0.0007) [2023-12-27 00:21:11,883][105620] Updated weights for policy 1, policy_version 1232558 (0.0005) [2023-12-27 00:21:11,938][105620] Updated weights for policy 1, policy_version 1232568 (0.0007) [2023-12-27 00:21:12,248][105692] Updated weights for policy 0, policy_version 1231386 (0.0008) [2023-12-27 00:21:12,304][105692] Updated weights for policy 0, policy_version 1231396 (0.0008) [2023-12-27 00:21:12,364][105692] Updated weights for policy 0, policy_version 1231406 (0.0009) [2023-12-27 00:21:12,633][105620] Updated weights for policy 1, policy_version 1232578 (0.0008) [2023-12-27 00:21:12,686][105620] Updated weights for policy 1, policy_version 1232588 (0.0008) [2023-12-27 00:21:12,737][105620] Updated weights for policy 1, policy_version 1232598 (0.0007) [2023-12-27 00:21:12,793][105620] Updated weights for policy 1, policy_version 1232608 (0.0006) [2023-12-27 00:21:13,181][105692] Updated weights for policy 0, policy_version 1231416 (0.0009) [2023-12-27 00:21:13,238][105692] Updated weights for policy 0, policy_version 1231426 (0.0009) [2023-12-27 00:21:13,291][105692] Updated weights for policy 0, policy_version 1231436 (0.0009) [2023-12-27 00:21:13,469][105620] Updated weights for policy 1, policy_version 1232618 (0.0008) [2023-12-27 00:21:13,523][105620] Updated weights for policy 1, policy_version 1232629 (0.0010) [2023-12-27 00:21:13,570][105620] Updated weights for policy 1, policy_version 1232639 (0.0008) [2023-12-27 00:21:13,990][105692] Updated weights for policy 0, policy_version 1231446 (0.0007) [2023-12-27 00:21:14,057][105692] Updated weights for policy 0, policy_version 1231456 (0.0005) [2023-12-27 00:21:14,128][105692] Updated weights for policy 0, policy_version 1231466 (0.0008) [2023-12-27 00:21:14,377][105620] Updated weights for policy 1, policy_version 1232649 (0.0006) [2023-12-27 00:21:14,431][105620] Updated weights for policy 1, policy_version 1232659 (0.0007) [2023-12-27 00:21:14,486][105620] Updated weights for policy 1, policy_version 1232669 (0.0007) [2023-12-27 00:21:14,792][105692] Updated weights for policy 0, policy_version 1231476 (0.0010) [2023-12-27 00:21:14,858][105692] Updated weights for policy 0, policy_version 1231486 (0.0010) [2023-12-27 00:21:14,917][105692] Updated weights for policy 0, policy_version 1231496 (0.0011) [2023-12-27 00:21:15,173][105620] Updated weights for policy 1, policy_version 1232679 (0.0007) [2023-12-27 00:21:15,237][105620] Updated weights for policy 1, policy_version 1232689 (0.0008) [2023-12-27 00:21:15,298][105620] Updated weights for policy 1, policy_version 1232699 (0.0008) [2023-12-27 00:21:15,691][105692] Updated weights for policy 0, policy_version 1231506 (0.0010) [2023-12-27 00:21:15,757][105692] Updated weights for policy 0, policy_version 1231516 (0.0010) [2023-12-27 00:21:15,811][105692] Updated weights for policy 0, policy_version 1231526 (0.0010) [2023-12-27 00:21:15,875][105692] Updated weights for policy 0, policy_version 1231536 (0.0010) [2023-12-27 00:21:16,059][105620] Updated weights for policy 1, policy_version 1232709 (0.0008) [2023-12-27 00:21:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 630939648. Throughput: 0: 9512.4, 1: 9849.4. Samples: 630909868. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:21:16,063][104569] Avg episode reward: [(0, '9085.715'), (1, '8640.604')] [2023-12-27 00:21:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001231536_315326464.pth... [2023-12-27 00:21:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001230416_315039744.pth [2023-12-27 00:21:16,124][105620] Updated weights for policy 1, policy_version 1232719 (0.0008) [2023-12-27 00:21:16,194][105620] Updated weights for policy 1, policy_version 1232729 (0.0009) [2023-12-27 00:21:16,236][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001232736_315621376.pth... [2023-12-27 00:21:16,242][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001231584_315326464.pth [2023-12-27 00:21:16,562][105692] Updated weights for policy 0, policy_version 1231546 (0.0009) [2023-12-27 00:21:16,609][105692] Updated weights for policy 0, policy_version 1231556 (0.0005) [2023-12-27 00:21:16,660][105692] Updated weights for policy 0, policy_version 1231566 (0.0005) [2023-12-27 00:21:16,837][105620] Updated weights for policy 1, policy_version 1232739 (0.0009) [2023-12-27 00:21:16,888][105620] Updated weights for policy 1, policy_version 1232749 (0.0009) [2023-12-27 00:21:16,942][105620] Updated weights for policy 1, policy_version 1232760 (0.0010) [2023-12-27 00:21:17,246][105692] Updated weights for policy 0, policy_version 1231576 (0.0009) [2023-12-27 00:21:17,303][105692] Updated weights for policy 0, policy_version 1231586 (0.0009) [2023-12-27 00:21:17,351][105692] Updated weights for policy 0, policy_version 1231596 (0.0008) [2023-12-27 00:21:17,777][105620] Updated weights for policy 1, policy_version 1232770 (0.0009) [2023-12-27 00:21:17,837][105620] Updated weights for policy 1, policy_version 1232780 (0.0009) [2023-12-27 00:21:17,901][105620] Updated weights for policy 1, policy_version 1232790 (0.0009) [2023-12-27 00:21:17,954][105620] Updated weights for policy 1, policy_version 1232800 (0.0009) [2023-12-27 00:21:18,113][105692] Updated weights for policy 0, policy_version 1231606 (0.0009) [2023-12-27 00:21:18,177][105692] Updated weights for policy 0, policy_version 1231616 (0.0008) [2023-12-27 00:21:18,234][105692] Updated weights for policy 0, policy_version 1231626 (0.0009) [2023-12-27 00:21:18,609][105620] Updated weights for policy 1, policy_version 1232810 (0.0009) [2023-12-27 00:21:18,672][105620] Updated weights for policy 1, policy_version 1232820 (0.0008) [2023-12-27 00:21:18,730][105620] Updated weights for policy 1, policy_version 1232830 (0.0009) [2023-12-27 00:21:19,013][105692] Updated weights for policy 0, policy_version 1231636 (0.0008) [2023-12-27 00:21:19,074][105692] Updated weights for policy 0, policy_version 1231646 (0.0009) [2023-12-27 00:21:19,138][105692] Updated weights for policy 0, policy_version 1231656 (0.0009) [2023-12-27 00:21:19,474][105620] Updated weights for policy 1, policy_version 1232840 (0.0006) [2023-12-27 00:21:19,536][105620] Updated weights for policy 1, policy_version 1232850 (0.0007) [2023-12-27 00:21:19,597][105620] Updated weights for policy 1, policy_version 1232860 (0.0008) [2023-12-27 00:21:19,901][105692] Updated weights for policy 0, policy_version 1231666 (0.0009) [2023-12-27 00:21:19,967][105692] Updated weights for policy 0, policy_version 1231676 (0.0009) [2023-12-27 00:21:20,033][105692] Updated weights for policy 0, policy_version 1231686 (0.0009) [2023-12-27 00:21:20,100][105692] Updated weights for policy 0, policy_version 1231696 (0.0009) [2023-12-27 00:21:20,253][105620] Updated weights for policy 1, policy_version 1232870 (0.0007) [2023-12-27 00:21:20,303][105620] Updated weights for policy 1, policy_version 1232880 (0.0008) [2023-12-27 00:21:20,363][105620] Updated weights for policy 1, policy_version 1232890 (0.0009) [2023-12-27 00:21:20,859][105692] Updated weights for policy 0, policy_version 1231706 (0.0009) [2023-12-27 00:21:20,910][105692] Updated weights for policy 0, policy_version 1231716 (0.0009) [2023-12-27 00:21:20,973][105692] Updated weights for policy 0, policy_version 1231726 (0.0009) [2023-12-27 00:21:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 631037952. Throughput: 0: 9601.1, 1: 9776.3. Samples: 631026516. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:21:21,063][104569] Avg episode reward: [(0, '9087.686'), (1, '8721.337')] [2023-12-27 00:21:21,100][105620] Updated weights for policy 1, policy_version 1232900 (0.0009) [2023-12-27 00:21:21,170][105620] Updated weights for policy 1, policy_version 1232910 (0.0008) [2023-12-27 00:21:21,240][105620] Updated weights for policy 1, policy_version 1232920 (0.0008) [2023-12-27 00:21:21,732][105692] Updated weights for policy 0, policy_version 1231736 (0.0009) [2023-12-27 00:21:21,799][105692] Updated weights for policy 0, policy_version 1231746 (0.0010) [2023-12-27 00:21:21,870][105692] Updated weights for policy 0, policy_version 1231756 (0.0009) [2023-12-27 00:21:22,006][105620] Updated weights for policy 1, policy_version 1232930 (0.0009) [2023-12-27 00:21:22,069][105620] Updated weights for policy 1, policy_version 1232940 (0.0009) [2023-12-27 00:21:22,131][105620] Updated weights for policy 1, policy_version 1232950 (0.0010) [2023-12-27 00:21:22,190][105620] Updated weights for policy 1, policy_version 1232960 (0.0009) [2023-12-27 00:21:22,547][105692] Updated weights for policy 0, policy_version 1231766 (0.0009) [2023-12-27 00:21:22,610][105692] Updated weights for policy 0, policy_version 1231776 (0.0008) [2023-12-27 00:21:22,669][105692] Updated weights for policy 0, policy_version 1231786 (0.0009) [2023-12-27 00:21:22,992][105620] Updated weights for policy 1, policy_version 1232970 (0.0009) [2023-12-27 00:21:23,043][105620] Updated weights for policy 1, policy_version 1232980 (0.0009) [2023-12-27 00:21:23,105][105620] Updated weights for policy 1, policy_version 1232990 (0.0008) [2023-12-27 00:21:23,339][105692] Updated weights for policy 0, policy_version 1231796 (0.0008) [2023-12-27 00:21:23,393][105692] Updated weights for policy 0, policy_version 1231806 (0.0009) [2023-12-27 00:21:23,447][105692] Updated weights for policy 0, policy_version 1231816 (0.0009) [2023-12-27 00:21:23,856][105620] Updated weights for policy 1, policy_version 1233000 (0.0006) [2023-12-27 00:21:23,910][105620] Updated weights for policy 1, policy_version 1233010 (0.0005) [2023-12-27 00:21:23,968][105620] Updated weights for policy 1, policy_version 1233020 (0.0005) [2023-12-27 00:21:24,209][105692] Updated weights for policy 0, policy_version 1231826 (0.0009) [2023-12-27 00:21:24,258][105692] Updated weights for policy 0, policy_version 1231836 (0.0010) [2023-12-27 00:21:24,307][105692] Updated weights for policy 0, policy_version 1231846 (0.0007) [2023-12-27 00:21:24,373][105692] Updated weights for policy 0, policy_version 1231856 (0.0005) [2023-12-27 00:21:24,558][105620] Updated weights for policy 1, policy_version 1233030 (0.0006) [2023-12-27 00:21:24,621][105620] Updated weights for policy 1, policy_version 1233040 (0.0005) [2023-12-27 00:21:24,683][105620] Updated weights for policy 1, policy_version 1233050 (0.0007) [2023-12-27 00:21:25,013][105692] Updated weights for policy 0, policy_version 1231866 (0.0010) [2023-12-27 00:21:25,067][105692] Updated weights for policy 0, policy_version 1231876 (0.0010) [2023-12-27 00:21:25,129][105692] Updated weights for policy 0, policy_version 1231886 (0.0010) [2023-12-27 00:21:25,235][105620] Updated weights for policy 1, policy_version 1233060 (0.0008) [2023-12-27 00:21:25,281][105620] Updated weights for policy 1, policy_version 1233070 (0.0005) [2023-12-27 00:21:25,332][105620] Updated weights for policy 1, policy_version 1233080 (0.0005) [2023-12-27 00:21:25,858][105620] Updated weights for policy 1, policy_version 1233090 (0.0006) [2023-12-27 00:21:25,863][105692] Updated weights for policy 0, policy_version 1231896 (0.0010) [2023-12-27 00:21:25,911][105692] Updated weights for policy 0, policy_version 1231906 (0.0010) [2023-12-27 00:21:25,911][105620] Updated weights for policy 1, policy_version 1233100 (0.0010) [2023-12-27 00:21:25,959][105692] Updated weights for policy 0, policy_version 1231916 (0.0010) [2023-12-27 00:21:25,965][105620] Updated weights for policy 1, policy_version 1233110 (0.0010) [2023-12-27 00:21:26,019][105620] Updated weights for policy 1, policy_version 1233120 (0.0009) [2023-12-27 00:21:26,062][104569] Fps is (10 sec: 20479.3, 60 sec: 19660.7, 300 sec: 19633.0). Total num frames: 631144448. Throughput: 0: 9718.4, 1: 9799.7. Samples: 631145504. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:21:26,063][104569] Avg episode reward: [(0, '8821.786'), (1, '8813.085')] [2023-12-27 00:21:26,706][105620] Updated weights for policy 1, policy_version 1233130 (0.0010) [2023-12-27 00:21:26,712][105692] Updated weights for policy 0, policy_version 1231926 (0.0007) [2023-12-27 00:21:26,751][105620] Updated weights for policy 1, policy_version 1233140 (0.0010) [2023-12-27 00:21:26,767][105692] Updated weights for policy 0, policy_version 1231936 (0.0007) [2023-12-27 00:21:26,791][105620] Updated weights for policy 1, policy_version 1233150 (0.0010) [2023-12-27 00:21:26,826][105692] Updated weights for policy 0, policy_version 1231946 (0.0007) [2023-12-27 00:21:27,478][105692] Updated weights for policy 0, policy_version 1231956 (0.0008) [2023-12-27 00:21:27,525][105692] Updated weights for policy 0, policy_version 1231966 (0.0008) [2023-12-27 00:21:27,568][105692] Updated weights for policy 0, policy_version 1231976 (0.0007) [2023-12-27 00:21:27,575][105620] Updated weights for policy 1, policy_version 1233160 (0.0009) [2023-12-27 00:21:27,618][105620] Updated weights for policy 1, policy_version 1233170 (0.0008) [2023-12-27 00:21:27,666][105620] Updated weights for policy 1, policy_version 1233180 (0.0010) [2023-12-27 00:21:28,314][105620] Updated weights for policy 1, policy_version 1233190 (0.0007) [2023-12-27 00:21:28,385][105620] Updated weights for policy 1, policy_version 1233200 (0.0008) [2023-12-27 00:21:28,402][105692] Updated weights for policy 0, policy_version 1231986 (0.0007) [2023-12-27 00:21:28,440][105620] Updated weights for policy 1, policy_version 1233210 (0.0006) [2023-12-27 00:21:28,460][105692] Updated weights for policy 0, policy_version 1231996 (0.0005) [2023-12-27 00:21:28,521][105692] Updated weights for policy 0, policy_version 1232006 (0.0006) [2023-12-27 00:21:28,582][105692] Updated weights for policy 0, policy_version 1232016 (0.0006) [2023-12-27 00:21:29,137][105620] Updated weights for policy 1, policy_version 1233220 (0.0011) [2023-12-27 00:21:29,189][105620] Updated weights for policy 1, policy_version 1233230 (0.0010) [2023-12-27 00:21:29,248][105620] Updated weights for policy 1, policy_version 1233240 (0.0010) [2023-12-27 00:21:29,269][105692] Updated weights for policy 0, policy_version 1232026 (0.0007) [2023-12-27 00:21:29,338][105692] Updated weights for policy 0, policy_version 1232036 (0.0009) [2023-12-27 00:21:29,396][105692] Updated weights for policy 0, policy_version 1232046 (0.0008) [2023-12-27 00:21:30,019][105620] Updated weights for policy 1, policy_version 1233250 (0.0011) [2023-12-27 00:21:30,041][105692] Updated weights for policy 0, policy_version 1232056 (0.0006) [2023-12-27 00:21:30,072][105620] Updated weights for policy 1, policy_version 1233260 (0.0008) [2023-12-27 00:21:30,098][105692] Updated weights for policy 0, policy_version 1232066 (0.0005) [2023-12-27 00:21:30,132][105620] Updated weights for policy 1, policy_version 1233270 (0.0008) [2023-12-27 00:21:30,153][105692] Updated weights for policy 0, policy_version 1232076 (0.0005) [2023-12-27 00:21:30,192][105620] Updated weights for policy 1, policy_version 1233280 (0.0008) [2023-12-27 00:21:30,712][105692] Updated weights for policy 0, policy_version 1232086 (0.0005) [2023-12-27 00:21:30,763][105692] Updated weights for policy 0, policy_version 1232096 (0.0008) [2023-12-27 00:21:30,814][105692] Updated weights for policy 0, policy_version 1232107 (0.0009) [2023-12-27 00:21:30,859][105620] Updated weights for policy 1, policy_version 1233290 (0.0005) [2023-12-27 00:21:30,923][105620] Updated weights for policy 1, policy_version 1233300 (0.0005) [2023-12-27 00:21:30,979][105620] Updated weights for policy 1, policy_version 1233310 (0.0005) [2023-12-27 00:21:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 631242752. Throughput: 0: 9686.3, 1: 9882.5. Samples: 631204764. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:21:31,063][104569] Avg episode reward: [(0, '8459.524'), (1, '8985.681')] [2023-12-27 00:21:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001232112_315473920.pth... [2023-12-27 00:21:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001233312_315768832.pth... [2023-12-27 00:21:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001230960_315179008.pth [2023-12-27 00:21:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001232128_315465728.pth [2023-12-27 00:21:31,574][105585] KL-divergence is very high: 113.3940 [2023-12-27 00:21:31,610][105692] Updated weights for policy 0, policy_version 1232117 (0.0007) [2023-12-27 00:21:31,629][105585] KL-divergence is very high: 171.2917 [2023-12-27 00:21:31,669][105692] Updated weights for policy 0, policy_version 1232127 (0.0008) [2023-12-27 00:21:31,675][105585] KL-divergence is very high: 189.2188 [2023-12-27 00:21:31,698][105620] Updated weights for policy 1, policy_version 1233320 (0.0007) [2023-12-27 00:21:31,728][105585] KL-divergence is very high: 188.2508 [2023-12-27 00:21:31,734][105692] Updated weights for policy 0, policy_version 1232137 (0.0008) [2023-12-27 00:21:31,760][105620] Updated weights for policy 1, policy_version 1233330 (0.0007) [2023-12-27 00:21:31,812][105620] Updated weights for policy 1, policy_version 1233340 (0.0008) [2023-12-27 00:21:32,349][105692] Updated weights for policy 0, policy_version 1232147 (0.0008) [2023-12-27 00:21:32,411][105692] Updated weights for policy 0, policy_version 1232157 (0.0008) [2023-12-27 00:21:32,461][105692] Updated weights for policy 0, policy_version 1232167 (0.0005) [2023-12-27 00:21:32,563][105620] Updated weights for policy 1, policy_version 1233350 (0.0008) [2023-12-27 00:21:32,621][105620] Updated weights for policy 1, policy_version 1233360 (0.0007) [2023-12-27 00:21:32,682][105620] Updated weights for policy 1, policy_version 1233370 (0.0006) [2023-12-27 00:21:33,187][105692] Updated weights for policy 0, policy_version 1232177 (0.0008) [2023-12-27 00:21:33,238][105692] Updated weights for policy 0, policy_version 1232187 (0.0009) [2023-12-27 00:21:33,241][105620] Updated weights for policy 1, policy_version 1233380 (0.0006) [2023-12-27 00:21:33,285][105692] Updated weights for policy 0, policy_version 1232197 (0.0008) [2023-12-27 00:21:33,287][105620] Updated weights for policy 1, policy_version 1233390 (0.0005) [2023-12-27 00:21:33,338][105692] Updated weights for policy 0, policy_version 1232207 (0.0009) [2023-12-27 00:21:33,341][105620] Updated weights for policy 1, policy_version 1233400 (0.0005) [2023-12-27 00:21:34,088][105620] Updated weights for policy 1, policy_version 1233410 (0.0007) [2023-12-27 00:21:34,093][105692] Updated weights for policy 0, policy_version 1232217 (0.0009) [2023-12-27 00:21:34,140][105620] Updated weights for policy 1, policy_version 1233420 (0.0006) [2023-12-27 00:21:34,143][105692] Updated weights for policy 0, policy_version 1232227 (0.0008) [2023-12-27 00:21:34,198][105620] Updated weights for policy 1, policy_version 1233430 (0.0008) [2023-12-27 00:21:34,200][105692] Updated weights for policy 0, policy_version 1232237 (0.0007) [2023-12-27 00:21:34,261][105620] Updated weights for policy 1, policy_version 1233440 (0.0008) [2023-12-27 00:21:34,944][105692] Updated weights for policy 0, policy_version 1232247 (0.0007) [2023-12-27 00:21:34,991][105692] Updated weights for policy 0, policy_version 1232257 (0.0009) [2023-12-27 00:21:35,042][105692] Updated weights for policy 0, policy_version 1232267 (0.0008) [2023-12-27 00:21:35,052][105620] Updated weights for policy 1, policy_version 1233450 (0.0008) [2023-12-27 00:21:35,107][105620] Updated weights for policy 1, policy_version 1233460 (0.0010) [2023-12-27 00:21:35,162][105620] Updated weights for policy 1, policy_version 1233470 (0.0009) [2023-12-27 00:21:35,757][105692] Updated weights for policy 0, policy_version 1232277 (0.0008) [2023-12-27 00:21:35,805][105692] Updated weights for policy 0, policy_version 1232287 (0.0009) [2023-12-27 00:21:35,856][105692] Updated weights for policy 0, policy_version 1232297 (0.0009) [2023-12-27 00:21:35,928][105620] Updated weights for policy 1, policy_version 1233480 (0.0009) [2023-12-27 00:21:35,975][105620] Updated weights for policy 1, policy_version 1233490 (0.0009) [2023-12-27 00:21:36,021][105620] Updated weights for policy 1, policy_version 1233500 (0.0008) [2023-12-27 00:21:36,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 631341056. Throughput: 0: 9693.8, 1: 9836.2. Samples: 631323700. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:21:36,063][104569] Avg episode reward: [(0, '8454.423'), (1, '9169.803')] [2023-12-27 00:21:36,566][105692] Updated weights for policy 0, policy_version 1232307 (0.0009) [2023-12-27 00:21:36,614][105692] Updated weights for policy 0, policy_version 1232317 (0.0009) [2023-12-27 00:21:36,669][105692] Updated weights for policy 0, policy_version 1232327 (0.0009) [2023-12-27 00:21:36,860][105620] Updated weights for policy 1, policy_version 1233510 (0.0008) [2023-12-27 00:21:36,925][105620] Updated weights for policy 1, policy_version 1233520 (0.0010) [2023-12-27 00:21:36,987][105620] Updated weights for policy 1, policy_version 1233530 (0.0009) [2023-12-27 00:21:37,405][105692] Updated weights for policy 0, policy_version 1232337 (0.0009) [2023-12-27 00:21:37,453][105692] Updated weights for policy 0, policy_version 1232347 (0.0009) [2023-12-27 00:21:37,503][105692] Updated weights for policy 0, policy_version 1232357 (0.0008) [2023-12-27 00:21:37,559][105692] Updated weights for policy 0, policy_version 1232367 (0.0006) [2023-12-27 00:21:37,771][105620] Updated weights for policy 1, policy_version 1233540 (0.0010) [2023-12-27 00:21:37,833][105620] Updated weights for policy 1, policy_version 1233550 (0.0009) [2023-12-27 00:21:37,887][105620] Updated weights for policy 1, policy_version 1233560 (0.0008) [2023-12-27 00:21:38,264][105692] Updated weights for policy 0, policy_version 1232377 (0.0008) [2023-12-27 00:21:38,315][105692] Updated weights for policy 0, policy_version 1232387 (0.0009) [2023-12-27 00:21:38,380][105692] Updated weights for policy 0, policy_version 1232397 (0.0008) [2023-12-27 00:21:38,607][105620] Updated weights for policy 1, policy_version 1233570 (0.0008) [2023-12-27 00:21:38,661][105620] Updated weights for policy 1, policy_version 1233580 (0.0008) [2023-12-27 00:21:38,717][105620] Updated weights for policy 1, policy_version 1233590 (0.0009) [2023-12-27 00:21:38,775][105620] Updated weights for policy 1, policy_version 1233600 (0.0009) [2023-12-27 00:21:39,184][105692] Updated weights for policy 0, policy_version 1232407 (0.0009) [2023-12-27 00:21:39,248][105692] Updated weights for policy 0, policy_version 1232417 (0.0008) [2023-12-27 00:21:39,309][105692] Updated weights for policy 0, policy_version 1232427 (0.0009) [2023-12-27 00:21:39,530][105620] Updated weights for policy 1, policy_version 1233610 (0.0009) [2023-12-27 00:21:39,595][105620] Updated weights for policy 1, policy_version 1233620 (0.0008) [2023-12-27 00:21:39,658][105620] Updated weights for policy 1, policy_version 1233630 (0.0009) [2023-12-27 00:21:40,095][105692] Updated weights for policy 0, policy_version 1232437 (0.0009) [2023-12-27 00:21:40,154][105692] Updated weights for policy 0, policy_version 1232447 (0.0009) [2023-12-27 00:21:40,219][105692] Updated weights for policy 0, policy_version 1232457 (0.0009) [2023-12-27 00:21:40,359][105620] Updated weights for policy 1, policy_version 1233640 (0.0008) [2023-12-27 00:21:40,422][105620] Updated weights for policy 1, policy_version 1233650 (0.0009) [2023-12-27 00:21:40,484][105620] Updated weights for policy 1, policy_version 1233660 (0.0009) [2023-12-27 00:21:40,964][105692] Updated weights for policy 0, policy_version 1232467 (0.0008) [2023-12-27 00:21:41,016][105692] Updated weights for policy 0, policy_version 1232477 (0.0009) [2023-12-27 00:21:41,062][104569] Fps is (10 sec: 18022.6, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 631422976. Throughput: 0: 9658.5, 1: 9809.3. Samples: 631435668. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:21:41,062][104569] Avg episode reward: [(0, '8454.075'), (1, '9352.242')] [2023-12-27 00:21:41,083][105692] Updated weights for policy 0, policy_version 1232487 (0.0007) [2023-12-27 00:21:41,227][105620] Updated weights for policy 1, policy_version 1233670 (0.0007) [2023-12-27 00:21:41,292][105620] Updated weights for policy 1, policy_version 1233680 (0.0009) [2023-12-27 00:21:41,358][105620] Updated weights for policy 1, policy_version 1233690 (0.0009) [2023-12-27 00:21:41,831][105692] Updated weights for policy 0, policy_version 1232497 (0.0007) [2023-12-27 00:21:41,894][105692] Updated weights for policy 0, policy_version 1232507 (0.0009) [2023-12-27 00:21:41,952][105692] Updated weights for policy 0, policy_version 1232517 (0.0009) [2023-12-27 00:21:42,020][105692] Updated weights for policy 0, policy_version 1232527 (0.0009) [2023-12-27 00:21:42,185][105620] Updated weights for policy 1, policy_version 1233700 (0.0010) [2023-12-27 00:21:42,252][105620] Updated weights for policy 1, policy_version 1233710 (0.0009) [2023-12-27 00:21:42,315][105620] Updated weights for policy 1, policy_version 1233720 (0.0009) [2023-12-27 00:21:42,769][105692] Updated weights for policy 0, policy_version 1232537 (0.0009) [2023-12-27 00:21:42,827][105692] Updated weights for policy 0, policy_version 1232547 (0.0009) [2023-12-27 00:21:42,889][105692] Updated weights for policy 0, policy_version 1232557 (0.0009) [2023-12-27 00:21:43,077][105620] Updated weights for policy 1, policy_version 1233730 (0.0008) [2023-12-27 00:21:43,134][105620] Updated weights for policy 1, policy_version 1233741 (0.0010) [2023-12-27 00:21:43,187][105620] Updated weights for policy 1, policy_version 1233751 (0.0010) [2023-12-27 00:21:43,474][105692] Updated weights for policy 0, policy_version 1232567 (0.0006) [2023-12-27 00:21:43,530][105692] Updated weights for policy 0, policy_version 1232577 (0.0005) [2023-12-27 00:21:43,579][105692] Updated weights for policy 0, policy_version 1232587 (0.0005) [2023-12-27 00:21:43,827][105620] Updated weights for policy 1, policy_version 1233763 (0.0009) [2023-12-27 00:21:43,897][105620] Updated weights for policy 1, policy_version 1233774 (0.0006) [2023-12-27 00:21:43,949][105620] Updated weights for policy 1, policy_version 1233784 (0.0007) [2023-12-27 00:21:44,192][105692] Updated weights for policy 0, policy_version 1232597 (0.0010) [2023-12-27 00:21:44,243][105692] Updated weights for policy 0, policy_version 1232607 (0.0010) [2023-12-27 00:21:44,298][105692] Updated weights for policy 0, policy_version 1232617 (0.0010) [2023-12-27 00:21:44,616][105620] Updated weights for policy 1, policy_version 1233794 (0.0005) [2023-12-27 00:21:44,668][105620] Updated weights for policy 1, policy_version 1233804 (0.0008) [2023-12-27 00:21:44,722][105620] Updated weights for policy 1, policy_version 1233814 (0.0005) [2023-12-27 00:21:44,790][105620] Updated weights for policy 1, policy_version 1233824 (0.0007) [2023-12-27 00:21:45,000][105692] Updated weights for policy 0, policy_version 1232627 (0.0009) [2023-12-27 00:21:45,061][105692] Updated weights for policy 0, policy_version 1232637 (0.0009) [2023-12-27 00:21:45,117][105692] Updated weights for policy 0, policy_version 1232647 (0.0009) [2023-12-27 00:21:45,470][105620] Updated weights for policy 1, policy_version 1233834 (0.0011) [2023-12-27 00:21:45,533][105620] Updated weights for policy 1, policy_version 1233844 (0.0011) [2023-12-27 00:21:45,599][105620] Updated weights for policy 1, policy_version 1233854 (0.0010) [2023-12-27 00:21:45,903][105692] Updated weights for policy 0, policy_version 1232657 (0.0009) [2023-12-27 00:21:45,957][105692] Updated weights for policy 0, policy_version 1232667 (0.0009) [2023-12-27 00:21:46,011][105692] Updated weights for policy 0, policy_version 1232678 (0.0010) [2023-12-27 00:21:46,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 631529472. Throughput: 0: 9681.7, 1: 9799.5. Samples: 631494624. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:21:46,062][104569] Avg episode reward: [(0, '8812.908'), (1, '9352.133')] [2023-12-27 00:21:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001232688_315621376.pth... [2023-12-27 00:21:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001233856_315908096.pth... [2023-12-27 00:21:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001231536_315326464.pth [2023-12-27 00:21:46,073][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_001232688_315621376.pth [2023-12-27 00:21:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001232736_315621376.pth [2023-12-27 00:21:46,077][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_001233856_315908096.pth [2023-12-27 00:21:46,214][105620] Updated weights for policy 1, policy_version 1233864 (0.0009) [2023-12-27 00:21:46,267][105620] Updated weights for policy 1, policy_version 1233874 (0.0009) [2023-12-27 00:21:46,324][105620] Updated weights for policy 1, policy_version 1233884 (0.0008) [2023-12-27 00:21:46,828][105692] Updated weights for policy 0, policy_version 1232690 (0.0010) [2023-12-27 00:21:46,895][105692] Updated weights for policy 0, policy_version 1232700 (0.0008) [2023-12-27 00:21:46,943][105692] Updated weights for policy 0, policy_version 1232710 (0.0008) [2023-12-27 00:21:46,995][105692] Updated weights for policy 0, policy_version 1232720 (0.0006) [2023-12-27 00:21:47,097][105620] Updated weights for policy 1, policy_version 1233894 (0.0010) [2023-12-27 00:21:47,155][105620] Updated weights for policy 1, policy_version 1233904 (0.0010) [2023-12-27 00:21:47,214][105620] Updated weights for policy 1, policy_version 1233914 (0.0010) [2023-12-27 00:21:47,695][105692] Updated weights for policy 0, policy_version 1232730 (0.0009) [2023-12-27 00:21:47,762][105692] Updated weights for policy 0, policy_version 1232740 (0.0008) [2023-12-27 00:21:47,817][105692] Updated weights for policy 0, policy_version 1232750 (0.0008) [2023-12-27 00:21:47,946][105620] Updated weights for policy 1, policy_version 1233924 (0.0010) [2023-12-27 00:21:48,012][105620] Updated weights for policy 1, policy_version 1233934 (0.0010) [2023-12-27 00:21:48,077][105620] Updated weights for policy 1, policy_version 1233944 (0.0010) [2023-12-27 00:21:48,470][105692] Updated weights for policy 0, policy_version 1232760 (0.0008) [2023-12-27 00:21:48,519][105692] Updated weights for policy 0, policy_version 1232770 (0.0009) [2023-12-27 00:21:48,567][105692] Updated weights for policy 0, policy_version 1232780 (0.0008) [2023-12-27 00:21:48,766][105620] Updated weights for policy 1, policy_version 1233954 (0.0009) [2023-12-27 00:21:48,828][105620] Updated weights for policy 1, policy_version 1233964 (0.0005) [2023-12-27 00:21:48,893][105620] Updated weights for policy 1, policy_version 1233974 (0.0007) [2023-12-27 00:21:48,959][105620] Updated weights for policy 1, policy_version 1233984 (0.0005) [2023-12-27 00:21:49,426][105692] Updated weights for policy 0, policy_version 1232790 (0.0008) [2023-12-27 00:21:49,474][105692] Updated weights for policy 0, policy_version 1232800 (0.0008) [2023-12-27 00:21:49,527][105692] Updated weights for policy 0, policy_version 1232810 (0.0008) [2023-12-27 00:21:49,627][105620] Updated weights for policy 1, policy_version 1233994 (0.0006) [2023-12-27 00:21:49,687][105620] Updated weights for policy 1, policy_version 1234004 (0.0005) [2023-12-27 00:21:49,749][105620] Updated weights for policy 1, policy_version 1234014 (0.0011) [2023-12-27 00:21:50,331][105692] Updated weights for policy 0, policy_version 1232820 (0.0008) [2023-12-27 00:21:50,385][105692] Updated weights for policy 0, policy_version 1232830 (0.0008) [2023-12-27 00:21:50,427][105620] Updated weights for policy 1, policy_version 1234024 (0.0010) [2023-12-27 00:21:50,445][105692] Updated weights for policy 0, policy_version 1232840 (0.0006) [2023-12-27 00:21:50,479][105620] Updated weights for policy 1, policy_version 1234034 (0.0010) [2023-12-27 00:21:50,534][105620] Updated weights for policy 1, policy_version 1234044 (0.0010) [2023-12-27 00:21:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 631619584. Throughput: 0: 9724.3, 1: 9807.1. Samples: 631611000. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:21:51,063][104569] Avg episode reward: [(0, '8908.084'), (1, '9262.129')] [2023-12-27 00:21:51,120][105692] Updated weights for policy 0, policy_version 1232850 (0.0006) [2023-12-27 00:21:51,186][105692] Updated weights for policy 0, policy_version 1232860 (0.0007) [2023-12-27 00:21:51,249][105692] Updated weights for policy 0, policy_version 1232870 (0.0008) [2023-12-27 00:21:51,313][105692] Updated weights for policy 0, policy_version 1232880 (0.0008) [2023-12-27 00:21:51,340][105620] Updated weights for policy 1, policy_version 1234054 (0.0010) [2023-12-27 00:21:51,403][105620] Updated weights for policy 1, policy_version 1234064 (0.0008) [2023-12-27 00:21:51,455][105620] Updated weights for policy 1, policy_version 1234074 (0.0005) [2023-12-27 00:21:52,016][105692] Updated weights for policy 0, policy_version 1232890 (0.0009) [2023-12-27 00:21:52,063][105692] Updated weights for policy 0, policy_version 1232900 (0.0008) [2023-12-27 00:21:52,112][105692] Updated weights for policy 0, policy_version 1232910 (0.0007) [2023-12-27 00:21:52,168][105620] Updated weights for policy 1, policy_version 1234084 (0.0007) [2023-12-27 00:21:52,223][105620] Updated weights for policy 1, policy_version 1234094 (0.0009) [2023-12-27 00:21:52,280][105620] Updated weights for policy 1, policy_version 1234104 (0.0009) [2023-12-27 00:21:52,831][105692] Updated weights for policy 0, policy_version 1232920 (0.0008) [2023-12-27 00:21:52,886][105692] Updated weights for policy 0, policy_version 1232930 (0.0009) [2023-12-27 00:21:52,940][105692] Updated weights for policy 0, policy_version 1232940 (0.0009) [2023-12-27 00:21:53,101][105620] Updated weights for policy 1, policy_version 1234114 (0.0010) [2023-12-27 00:21:53,155][105620] Updated weights for policy 1, policy_version 1234124 (0.0008) [2023-12-27 00:21:53,205][105620] Updated weights for policy 1, policy_version 1234134 (0.0009) [2023-12-27 00:21:53,255][105620] Updated weights for policy 1, policy_version 1234144 (0.0008) [2023-12-27 00:21:53,678][105692] Updated weights for policy 0, policy_version 1232950 (0.0008) [2023-12-27 00:21:53,728][105692] Updated weights for policy 0, policy_version 1232960 (0.0008) [2023-12-27 00:21:53,781][105692] Updated weights for policy 0, policy_version 1232970 (0.0008) [2023-12-27 00:21:53,967][105620] Updated weights for policy 1, policy_version 1234154 (0.0005) [2023-12-27 00:21:54,023][105620] Updated weights for policy 1, policy_version 1234164 (0.0005) [2023-12-27 00:21:54,067][105620] Updated weights for policy 1, policy_version 1234174 (0.0005) [2023-12-27 00:21:54,560][105692] Updated weights for policy 0, policy_version 1232980 (0.0009) [2023-12-27 00:21:54,613][105692] Updated weights for policy 0, policy_version 1232990 (0.0008) [2023-12-27 00:21:54,674][105692] Updated weights for policy 0, policy_version 1233000 (0.0007) [2023-12-27 00:21:54,695][105620] Updated weights for policy 1, policy_version 1234184 (0.0009) [2023-12-27 00:21:54,755][105620] Updated weights for policy 1, policy_version 1234194 (0.0006) [2023-12-27 00:21:54,819][105620] Updated weights for policy 1, policy_version 1234204 (0.0010) [2023-12-27 00:21:55,376][105692] Updated weights for policy 0, policy_version 1233010 (0.0008) [2023-12-27 00:21:55,423][105692] Updated weights for policy 0, policy_version 1233020 (0.0007) [2023-12-27 00:21:55,471][105692] Updated weights for policy 0, policy_version 1233030 (0.0008) [2023-12-27 00:21:55,523][105692] Updated weights for policy 0, policy_version 1233040 (0.0005) [2023-12-27 00:21:55,533][105620] Updated weights for policy 1, policy_version 1234214 (0.0010) [2023-12-27 00:21:55,594][105620] Updated weights for policy 1, policy_version 1234224 (0.0010) [2023-12-27 00:21:55,655][105620] Updated weights for policy 1, policy_version 1234234 (0.0010) [2023-12-27 00:21:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 631717888. Throughput: 0: 9636.1, 1: 9769.4. Samples: 631727132. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:21:56,062][104569] Avg episode reward: [(0, '8635.365'), (1, '9262.035')] [2023-12-27 00:21:56,195][105692] Updated weights for policy 0, policy_version 1233050 (0.0010) [2023-12-27 00:21:56,251][105692] Updated weights for policy 0, policy_version 1233060 (0.0010) [2023-12-27 00:21:56,308][105620] Updated weights for policy 1, policy_version 1234244 (0.0010) [2023-12-27 00:21:56,315][105692] Updated weights for policy 0, policy_version 1233070 (0.0007) [2023-12-27 00:21:56,373][105620] Updated weights for policy 1, policy_version 1234254 (0.0008) [2023-12-27 00:21:56,437][105620] Updated weights for policy 1, policy_version 1234264 (0.0007) [2023-12-27 00:21:56,909][105692] Updated weights for policy 0, policy_version 1233080 (0.0006) [2023-12-27 00:21:56,962][105692] Updated weights for policy 0, policy_version 1233090 (0.0010) [2023-12-27 00:21:57,014][105692] Updated weights for policy 0, policy_version 1233100 (0.0005) [2023-12-27 00:21:57,215][105620] Updated weights for policy 1, policy_version 1234274 (0.0009) [2023-12-27 00:21:57,272][105620] Updated weights for policy 1, policy_version 1234284 (0.0009) [2023-12-27 00:21:57,326][105620] Updated weights for policy 1, policy_version 1234294 (0.0009) [2023-12-27 00:21:57,382][105620] Updated weights for policy 1, policy_version 1234304 (0.0008) [2023-12-27 00:21:57,697][105692] Updated weights for policy 0, policy_version 1233111 (0.0008) [2023-12-27 00:21:57,754][105692] Updated weights for policy 0, policy_version 1233121 (0.0009) [2023-12-27 00:21:57,810][105692] Updated weights for policy 0, policy_version 1233131 (0.0008) [2023-12-27 00:21:58,007][105620] Updated weights for policy 1, policy_version 1234314 (0.0010) [2023-12-27 00:21:58,067][105620] Updated weights for policy 1, policy_version 1234324 (0.0009) [2023-12-27 00:21:58,129][105620] Updated weights for policy 1, policy_version 1234334 (0.0010) [2023-12-27 00:21:58,471][105692] Updated weights for policy 0, policy_version 1233141 (0.0007) [2023-12-27 00:21:58,534][105692] Updated weights for policy 0, policy_version 1233151 (0.0008) [2023-12-27 00:21:58,602][105692] Updated weights for policy 0, policy_version 1233161 (0.0010) [2023-12-27 00:21:58,943][105620] Updated weights for policy 1, policy_version 1234344 (0.0010) [2023-12-27 00:21:59,004][105620] Updated weights for policy 1, policy_version 1234354 (0.0010) [2023-12-27 00:21:59,065][105620] Updated weights for policy 1, policy_version 1234364 (0.0009) [2023-12-27 00:21:59,421][105692] Updated weights for policy 0, policy_version 1233171 (0.0008) [2023-12-27 00:21:59,475][105692] Updated weights for policy 0, policy_version 1233181 (0.0009) [2023-12-27 00:21:59,529][105692] Updated weights for policy 0, policy_version 1233191 (0.0010) [2023-12-27 00:21:59,878][105620] Updated weights for policy 1, policy_version 1234374 (0.0009) [2023-12-27 00:21:59,944][105620] Updated weights for policy 1, policy_version 1234384 (0.0010) [2023-12-27 00:22:00,003][105620] Updated weights for policy 1, policy_version 1234394 (0.0010) [2023-12-27 00:22:00,317][105692] Updated weights for policy 0, policy_version 1233201 (0.0008) [2023-12-27 00:22:00,381][105692] Updated weights for policy 0, policy_version 1233211 (0.0005) [2023-12-27 00:22:00,443][105692] Updated weights for policy 0, policy_version 1233221 (0.0005) [2023-12-27 00:22:00,493][105692] Updated weights for policy 0, policy_version 1233231 (0.0005) [2023-12-27 00:22:00,651][105620] Updated weights for policy 1, policy_version 1234404 (0.0006) [2023-12-27 00:22:00,701][105620] Updated weights for policy 1, policy_version 1234414 (0.0005) [2023-12-27 00:22:00,750][105620] Updated weights for policy 1, policy_version 1234424 (0.0006) [2023-12-27 00:22:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 631816192. Throughput: 0: 9756.5, 1: 9743.5. Samples: 631787372. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:22:01,063][104569] Avg episode reward: [(0, '8631.671'), (1, '9260.746')] [2023-12-27 00:22:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001234432_316055552.pth... [2023-12-27 00:22:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001233312_315768832.pth [2023-12-27 00:22:01,091][105692] Updated weights for policy 0, policy_version 1233241 (0.0008) [2023-12-27 00:22:01,154][105692] Updated weights for policy 0, policy_version 1233251 (0.0008) [2023-12-27 00:22:01,213][105692] Updated weights for policy 0, policy_version 1233261 (0.0006) [2023-12-27 00:22:01,229][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001233264_315768832.pth... [2023-12-27 00:22:01,232][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001232112_315473920.pth [2023-12-27 00:22:01,432][105620] Updated weights for policy 1, policy_version 1234434 (0.0006) [2023-12-27 00:22:01,480][105620] Updated weights for policy 1, policy_version 1234444 (0.0007) [2023-12-27 00:22:01,529][105620] Updated weights for policy 1, policy_version 1234454 (0.0005) [2023-12-27 00:22:01,593][105620] Updated weights for policy 1, policy_version 1234464 (0.0008) [2023-12-27 00:22:01,911][105692] Updated weights for policy 0, policy_version 1233271 (0.0006) [2023-12-27 00:22:01,972][105692] Updated weights for policy 0, policy_version 1233281 (0.0005) [2023-12-27 00:22:02,041][105692] Updated weights for policy 0, policy_version 1233291 (0.0005) [2023-12-27 00:22:02,275][105620] Updated weights for policy 1, policy_version 1234474 (0.0010) [2023-12-27 00:22:02,327][105620] Updated weights for policy 1, policy_version 1234484 (0.0009) [2023-12-27 00:22:02,389][105620] Updated weights for policy 1, policy_version 1234495 (0.0010) [2023-12-27 00:22:02,623][105692] Updated weights for policy 0, policy_version 1233301 (0.0007) [2023-12-27 00:22:02,672][105692] Updated weights for policy 0, policy_version 1233311 (0.0008) [2023-12-27 00:22:02,731][105692] Updated weights for policy 0, policy_version 1233321 (0.0009) [2023-12-27 00:22:03,145][105620] Updated weights for policy 1, policy_version 1234505 (0.0009) [2023-12-27 00:22:03,203][105620] Updated weights for policy 1, policy_version 1234515 (0.0010) [2023-12-27 00:22:03,260][105620] Updated weights for policy 1, policy_version 1234525 (0.0010) [2023-12-27 00:22:03,521][105692] Updated weights for policy 0, policy_version 1233331 (0.0009) [2023-12-27 00:22:03,568][105692] Updated weights for policy 0, policy_version 1233341 (0.0008) [2023-12-27 00:22:03,612][105692] Updated weights for policy 0, policy_version 1233351 (0.0007) [2023-12-27 00:22:03,989][105620] Updated weights for policy 1, policy_version 1234535 (0.0011) [2023-12-27 00:22:04,048][105620] Updated weights for policy 1, policy_version 1234545 (0.0010) [2023-12-27 00:22:04,113][105620] Updated weights for policy 1, policy_version 1234555 (0.0007) [2023-12-27 00:22:04,367][105692] Updated weights for policy 0, policy_version 1233361 (0.0008) [2023-12-27 00:22:04,432][105692] Updated weights for policy 0, policy_version 1233371 (0.0006) [2023-12-27 00:22:04,499][105692] Updated weights for policy 0, policy_version 1233381 (0.0006) [2023-12-27 00:22:04,565][105692] Updated weights for policy 0, policy_version 1233391 (0.0006) [2023-12-27 00:22:04,777][105620] Updated weights for policy 1, policy_version 1234565 (0.0007) [2023-12-27 00:22:04,845][105620] Updated weights for policy 1, policy_version 1234575 (0.0006) [2023-12-27 00:22:04,910][105620] Updated weights for policy 1, policy_version 1234585 (0.0007) [2023-12-27 00:22:05,151][105692] Updated weights for policy 0, policy_version 1233401 (0.0008) [2023-12-27 00:22:05,209][105692] Updated weights for policy 0, policy_version 1233411 (0.0009) [2023-12-27 00:22:05,267][105692] Updated weights for policy 0, policy_version 1233421 (0.0009) [2023-12-27 00:22:05,512][105620] Updated weights for policy 1, policy_version 1234595 (0.0009) [2023-12-27 00:22:05,568][105620] Updated weights for policy 1, policy_version 1234605 (0.0005) [2023-12-27 00:22:05,637][105620] Updated weights for policy 1, policy_version 1234615 (0.0005) [2023-12-27 00:22:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 631914496. Throughput: 0: 9761.1, 1: 9771.7. Samples: 631905492. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:22:06,063][104569] Avg episode reward: [(0, '8995.427'), (1, '9260.606')] [2023-12-27 00:22:06,163][105692] Updated weights for policy 0, policy_version 1233431 (0.0008) [2023-12-27 00:22:06,164][105620] Updated weights for policy 1, policy_version 1234625 (0.0006) [2023-12-27 00:22:06,224][105620] Updated weights for policy 1, policy_version 1234635 (0.0008) [2023-12-27 00:22:06,224][105692] Updated weights for policy 0, policy_version 1233441 (0.0008) [2023-12-27 00:22:06,281][105692] Updated weights for policy 0, policy_version 1233451 (0.0006) [2023-12-27 00:22:06,291][105620] Updated weights for policy 1, policy_version 1234645 (0.0008) [2023-12-27 00:22:06,354][105620] Updated weights for policy 1, policy_version 1234655 (0.0008) [2023-12-27 00:22:07,027][105692] Updated weights for policy 0, policy_version 1233461 (0.0005) [2023-12-27 00:22:07,092][105692] Updated weights for policy 0, policy_version 1233471 (0.0006) [2023-12-27 00:22:07,131][105620] Updated weights for policy 1, policy_version 1234665 (0.0010) [2023-12-27 00:22:07,154][105692] Updated weights for policy 0, policy_version 1233481 (0.0009) [2023-12-27 00:22:07,195][105620] Updated weights for policy 1, policy_version 1234675 (0.0011) [2023-12-27 00:22:07,254][105620] Updated weights for policy 1, policy_version 1234685 (0.0011) [2023-12-27 00:22:07,848][105620] Updated weights for policy 1, policy_version 1234695 (0.0007) [2023-12-27 00:22:07,901][105620] Updated weights for policy 1, policy_version 1234705 (0.0005) [2023-12-27 00:22:07,958][105620] Updated weights for policy 1, policy_version 1234715 (0.0006) [2023-12-27 00:22:07,964][105692] Updated weights for policy 0, policy_version 1233491 (0.0008) [2023-12-27 00:22:08,019][105692] Updated weights for policy 0, policy_version 1233501 (0.0007) [2023-12-27 00:22:08,069][105692] Updated weights for policy 0, policy_version 1233511 (0.0008) [2023-12-27 00:22:08,660][105620] Updated weights for policy 1, policy_version 1234725 (0.0011) [2023-12-27 00:22:08,723][105620] Updated weights for policy 1, policy_version 1234735 (0.0011) [2023-12-27 00:22:08,786][105620] Updated weights for policy 1, policy_version 1234745 (0.0011) [2023-12-27 00:22:08,843][105692] Updated weights for policy 0, policy_version 1233521 (0.0008) [2023-12-27 00:22:08,902][105692] Updated weights for policy 0, policy_version 1233531 (0.0008) [2023-12-27 00:22:08,961][105692] Updated weights for policy 0, policy_version 1233541 (0.0008) [2023-12-27 00:22:09,020][105692] Updated weights for policy 0, policy_version 1233551 (0.0008) [2023-12-27 00:22:09,605][105620] Updated weights for policy 1, policy_version 1234755 (0.0011) [2023-12-27 00:22:09,671][105620] Updated weights for policy 1, policy_version 1234765 (0.0011) [2023-12-27 00:22:09,734][105620] Updated weights for policy 1, policy_version 1234775 (0.0011) [2023-12-27 00:22:09,761][105692] Updated weights for policy 0, policy_version 1233561 (0.0006) [2023-12-27 00:22:09,822][105692] Updated weights for policy 0, policy_version 1233571 (0.0009) [2023-12-27 00:22:09,886][105692] Updated weights for policy 0, policy_version 1233581 (0.0009) [2023-12-27 00:22:10,484][105620] Updated weights for policy 1, policy_version 1234785 (0.0009) [2023-12-27 00:22:10,540][105620] Updated weights for policy 1, policy_version 1234795 (0.0011) [2023-12-27 00:22:10,592][105620] Updated weights for policy 1, policy_version 1234805 (0.0011) [2023-12-27 00:22:10,650][105692] Updated weights for policy 0, policy_version 1233591 (0.0007) [2023-12-27 00:22:10,652][105620] Updated weights for policy 1, policy_version 1234815 (0.0011) [2023-12-27 00:22:10,714][105692] Updated weights for policy 0, policy_version 1233601 (0.0008) [2023-12-27 00:22:10,762][105692] Updated weights for policy 0, policy_version 1233611 (0.0008) [2023-12-27 00:22:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 632012800. Throughput: 0: 9684.8, 1: 9743.1. Samples: 632019752. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-12-27 00:22:11,063][104569] Avg episode reward: [(0, '9266.107'), (1, '9260.142')] [2023-12-27 00:22:11,420][105620] Updated weights for policy 1, policy_version 1234825 (0.0009) [2023-12-27 00:22:11,487][105620] Updated weights for policy 1, policy_version 1234835 (0.0009) [2023-12-27 00:22:11,548][105620] Updated weights for policy 1, policy_version 1234845 (0.0009) [2023-12-27 00:22:11,570][105692] Updated weights for policy 0, policy_version 1233621 (0.0009) [2023-12-27 00:22:11,638][105692] Updated weights for policy 0, policy_version 1233631 (0.0009) [2023-12-27 00:22:11,709][105692] Updated weights for policy 0, policy_version 1233641 (0.0008) [2023-12-27 00:22:12,270][105620] Updated weights for policy 1, policy_version 1234855 (0.0009) [2023-12-27 00:22:12,337][105620] Updated weights for policy 1, policy_version 1234865 (0.0009) [2023-12-27 00:22:12,405][105620] Updated weights for policy 1, policy_version 1234875 (0.0008) [2023-12-27 00:22:12,469][105692] Updated weights for policy 0, policy_version 1233651 (0.0008) [2023-12-27 00:22:12,531][105692] Updated weights for policy 0, policy_version 1233661 (0.0009) [2023-12-27 00:22:12,594][105692] Updated weights for policy 0, policy_version 1233671 (0.0009) [2023-12-27 00:22:13,158][105620] Updated weights for policy 1, policy_version 1234885 (0.0010) [2023-12-27 00:22:13,202][105620] Updated weights for policy 1, policy_version 1234895 (0.0010) [2023-12-27 00:22:13,250][105692] Updated weights for policy 0, policy_version 1233681 (0.0009) [2023-12-27 00:22:13,258][105620] Updated weights for policy 1, policy_version 1234905 (0.0009) [2023-12-27 00:22:13,308][105692] Updated weights for policy 0, policy_version 1233691 (0.0005) [2023-12-27 00:22:13,360][105692] Updated weights for policy 0, policy_version 1233701 (0.0005) [2023-12-27 00:22:13,417][105692] Updated weights for policy 0, policy_version 1233711 (0.0007) [2023-12-27 00:22:13,925][105620] Updated weights for policy 1, policy_version 1234915 (0.0007) [2023-12-27 00:22:13,990][105620] Updated weights for policy 1, policy_version 1234925 (0.0010) [2023-12-27 00:22:14,042][105620] Updated weights for policy 1, policy_version 1234935 (0.0008) [2023-12-27 00:22:14,102][105692] Updated weights for policy 0, policy_version 1233721 (0.0010) [2023-12-27 00:22:14,154][105692] Updated weights for policy 0, policy_version 1233731 (0.0010) [2023-12-27 00:22:14,211][105692] Updated weights for policy 0, policy_version 1233741 (0.0010) [2023-12-27 00:22:14,806][105692] Updated weights for policy 0, policy_version 1233751 (0.0011) [2023-12-27 00:22:14,865][105620] Updated weights for policy 1, policy_version 1234945 (0.0008) [2023-12-27 00:22:14,872][105692] Updated weights for policy 0, policy_version 1233761 (0.0011) [2023-12-27 00:22:14,935][105620] Updated weights for policy 1, policy_version 1234956 (0.0009) [2023-12-27 00:22:14,938][105692] Updated weights for policy 0, policy_version 1233771 (0.0007) [2023-12-27 00:22:14,999][105620] Updated weights for policy 1, policy_version 1234966 (0.0007) [2023-12-27 00:22:15,066][105620] Updated weights for policy 1, policy_version 1234976 (0.0009) [2023-12-27 00:22:15,606][105692] Updated weights for policy 0, policy_version 1233781 (0.0009) [2023-12-27 00:22:15,668][105692] Updated weights for policy 0, policy_version 1233791 (0.0009) [2023-12-27 00:22:15,736][105692] Updated weights for policy 0, policy_version 1233801 (0.0009) [2023-12-27 00:22:15,754][105620] Updated weights for policy 1, policy_version 1234986 (0.0007) [2023-12-27 00:22:15,818][105620] Updated weights for policy 1, policy_version 1234996 (0.0009) [2023-12-27 00:22:15,887][105620] Updated weights for policy 1, policy_version 1235006 (0.0009) [2023-12-27 00:22:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 632111104. Throughput: 0: 9676.5, 1: 9711.8. Samples: 632077240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:22:16,063][104569] Avg episode reward: [(0, '9268.973'), (1, '9168.648')] [2023-12-27 00:22:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001235008_316203008.pth... [2023-12-27 00:22:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001233808_315908096.pth... [2023-12-27 00:22:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001233856_315908096.pth [2023-12-27 00:22:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001232688_315621376.pth [2023-12-27 00:22:16,508][105620] Updated weights for policy 1, policy_version 1235016 (0.0006) [2023-12-27 00:22:16,561][105620] Updated weights for policy 1, policy_version 1235026 (0.0005) [2023-12-27 00:22:16,567][105692] Updated weights for policy 0, policy_version 1233811 (0.0007) [2023-12-27 00:22:16,616][105620] Updated weights for policy 1, policy_version 1235036 (0.0005) [2023-12-27 00:22:16,625][105692] Updated weights for policy 0, policy_version 1233821 (0.0010) [2023-12-27 00:22:16,673][105692] Updated weights for policy 0, policy_version 1233831 (0.0010) [2023-12-27 00:22:17,205][105620] Updated weights for policy 1, policy_version 1235046 (0.0006) [2023-12-27 00:22:17,238][105692] Updated weights for policy 0, policy_version 1233841 (0.0010) [2023-12-27 00:22:17,263][105620] Updated weights for policy 1, policy_version 1235056 (0.0005) [2023-12-27 00:22:17,293][105692] Updated weights for policy 0, policy_version 1233851 (0.0006) [2023-12-27 00:22:17,323][105620] Updated weights for policy 1, policy_version 1235066 (0.0005) [2023-12-27 00:22:17,355][105692] Updated weights for policy 0, policy_version 1233861 (0.0006) [2023-12-27 00:22:17,409][105692] Updated weights for policy 0, policy_version 1233871 (0.0010) [2023-12-27 00:22:17,922][105620] Updated weights for policy 1, policy_version 1235076 (0.0007) [2023-12-27 00:22:17,990][105620] Updated weights for policy 1, policy_version 1235086 (0.0011) [2023-12-27 00:22:18,050][105620] Updated weights for policy 1, policy_version 1235096 (0.0009) [2023-12-27 00:22:18,180][105692] Updated weights for policy 0, policy_version 1233881 (0.0009) [2023-12-27 00:22:18,229][105692] Updated weights for policy 0, policy_version 1233891 (0.0008) [2023-12-27 00:22:18,278][105692] Updated weights for policy 0, policy_version 1233901 (0.0008) [2023-12-27 00:22:18,760][105620] Updated weights for policy 1, policy_version 1235106 (0.0007) [2023-12-27 00:22:18,821][105620] Updated weights for policy 1, policy_version 1235116 (0.0010) [2023-12-27 00:22:18,886][105620] Updated weights for policy 1, policy_version 1235126 (0.0010) [2023-12-27 00:22:18,944][105620] Updated weights for policy 1, policy_version 1235136 (0.0010) [2023-12-27 00:22:19,093][105692] Updated weights for policy 0, policy_version 1233911 (0.0008) [2023-12-27 00:22:19,141][105692] Updated weights for policy 0, policy_version 1233921 (0.0008) [2023-12-27 00:22:19,208][105692] Updated weights for policy 0, policy_version 1233931 (0.0009) [2023-12-27 00:22:19,654][105620] Updated weights for policy 1, policy_version 1235146 (0.0009) [2023-12-27 00:22:19,717][105620] Updated weights for policy 1, policy_version 1235156 (0.0010) [2023-12-27 00:22:19,783][105620] Updated weights for policy 1, policy_version 1235166 (0.0008) [2023-12-27 00:22:20,025][105692] Updated weights for policy 0, policy_version 1233941 (0.0008) [2023-12-27 00:22:20,089][105692] Updated weights for policy 0, policy_version 1233951 (0.0007) [2023-12-27 00:22:20,145][105692] Updated weights for policy 0, policy_version 1233961 (0.0005) [2023-12-27 00:22:20,610][105620] Updated weights for policy 1, policy_version 1235176 (0.0009) [2023-12-27 00:22:20,672][105620] Updated weights for policy 1, policy_version 1235186 (0.0009) [2023-12-27 00:22:20,734][105620] Updated weights for policy 1, policy_version 1235196 (0.0008) [2023-12-27 00:22:20,832][105692] Updated weights for policy 0, policy_version 1233971 (0.0007) [2023-12-27 00:22:20,879][105692] Updated weights for policy 0, policy_version 1233981 (0.0009) [2023-12-27 00:22:20,926][105692] Updated weights for policy 0, policy_version 1233991 (0.0009) [2023-12-27 00:22:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 632209408. Throughput: 0: 9625.8, 1: 9736.1. Samples: 632194984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:22:21,063][104569] Avg episode reward: [(0, '9178.840'), (1, '9260.222')] [2023-12-27 00:22:21,493][105620] Updated weights for policy 1, policy_version 1235206 (0.0009) [2023-12-27 00:22:21,554][105620] Updated weights for policy 1, policy_version 1235216 (0.0006) [2023-12-27 00:22:21,613][105620] Updated weights for policy 1, policy_version 1235226 (0.0008) [2023-12-27 00:22:21,767][105692] Updated weights for policy 0, policy_version 1234001 (0.0008) [2023-12-27 00:22:21,839][105692] Updated weights for policy 0, policy_version 1234011 (0.0005) [2023-12-27 00:22:21,908][105692] Updated weights for policy 0, policy_version 1234021 (0.0006) [2023-12-27 00:22:21,971][105692] Updated weights for policy 0, policy_version 1234031 (0.0009) [2023-12-27 00:22:22,393][105620] Updated weights for policy 1, policy_version 1235236 (0.0009) [2023-12-27 00:22:22,452][105620] Updated weights for policy 1, policy_version 1235246 (0.0009) [2023-12-27 00:22:22,498][105620] Updated weights for policy 1, policy_version 1235256 (0.0008) [2023-12-27 00:22:22,657][105692] Updated weights for policy 0, policy_version 1234041 (0.0009) [2023-12-27 00:22:22,708][105692] Updated weights for policy 0, policy_version 1234051 (0.0009) [2023-12-27 00:22:22,766][105692] Updated weights for policy 0, policy_version 1234061 (0.0010) [2023-12-27 00:22:23,133][105620] Updated weights for policy 1, policy_version 1235266 (0.0009) [2023-12-27 00:22:23,191][105620] Updated weights for policy 1, policy_version 1235276 (0.0010) [2023-12-27 00:22:23,243][105620] Updated weights for policy 1, policy_version 1235286 (0.0010) [2023-12-27 00:22:23,292][105620] Updated weights for policy 1, policy_version 1235296 (0.0008) [2023-12-27 00:22:23,605][105692] Updated weights for policy 0, policy_version 1234071 (0.0010) [2023-12-27 00:22:23,653][105692] Updated weights for policy 0, policy_version 1234081 (0.0010) [2023-12-27 00:22:23,697][105692] Updated weights for policy 0, policy_version 1234091 (0.0010) [2023-12-27 00:22:23,861][105620] Updated weights for policy 1, policy_version 1235306 (0.0005) [2023-12-27 00:22:23,915][105620] Updated weights for policy 1, policy_version 1235316 (0.0005) [2023-12-27 00:22:23,969][105620] Updated weights for policy 1, policy_version 1235326 (0.0005) [2023-12-27 00:22:24,274][105692] Updated weights for policy 0, policy_version 1234101 (0.0009) [2023-12-27 00:22:24,333][105692] Updated weights for policy 0, policy_version 1234111 (0.0010) [2023-12-27 00:22:24,382][105692] Updated weights for policy 0, policy_version 1234121 (0.0010) [2023-12-27 00:22:24,640][105620] Updated weights for policy 1, policy_version 1235336 (0.0009) [2023-12-27 00:22:24,698][105620] Updated weights for policy 1, policy_version 1235346 (0.0010) [2023-12-27 00:22:24,759][105620] Updated weights for policy 1, policy_version 1235356 (0.0010) [2023-12-27 00:22:25,127][105692] Updated weights for policy 0, policy_version 1234131 (0.0010) [2023-12-27 00:22:25,175][105692] Updated weights for policy 0, policy_version 1234141 (0.0010) [2023-12-27 00:22:25,227][105692] Updated weights for policy 0, policy_version 1234151 (0.0010) [2023-12-27 00:22:25,419][105620] Updated weights for policy 1, policy_version 1235366 (0.0007) [2023-12-27 00:22:25,484][105620] Updated weights for policy 1, policy_version 1235376 (0.0005) [2023-12-27 00:22:25,533][105620] Updated weights for policy 1, policy_version 1235386 (0.0005) [2023-12-27 00:22:25,949][105692] Updated weights for policy 0, policy_version 1234161 (0.0008) [2023-12-27 00:22:26,008][105692] Updated weights for policy 0, policy_version 1234171 (0.0006) [2023-12-27 00:22:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.3, 300 sec: 19522.0). Total num frames: 632299520. Throughput: 0: 9641.4, 1: 9861.8. Samples: 632313312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:22:26,062][104569] Avg episode reward: [(0, '8861.988'), (1, '9169.700')] [2023-12-27 00:22:26,066][105692] Updated weights for policy 0, policy_version 1234181 (0.0006) [2023-12-27 00:22:26,121][105692] Updated weights for policy 0, policy_version 1234191 (0.0007) [2023-12-27 00:22:26,206][105620] Updated weights for policy 1, policy_version 1235396 (0.0009) [2023-12-27 00:22:26,260][105620] Updated weights for policy 1, policy_version 1235406 (0.0010) [2023-12-27 00:22:26,314][105620] Updated weights for policy 1, policy_version 1235416 (0.0010) [2023-12-27 00:22:26,808][105692] Updated weights for policy 0, policy_version 1234201 (0.0006) [2023-12-27 00:22:26,863][105692] Updated weights for policy 0, policy_version 1234211 (0.0008) [2023-12-27 00:22:26,913][105692] Updated weights for policy 0, policy_version 1234222 (0.0009) [2023-12-27 00:22:27,015][105620] Updated weights for policy 1, policy_version 1235426 (0.0007) [2023-12-27 00:22:27,073][105620] Updated weights for policy 1, policy_version 1235436 (0.0006) [2023-12-27 00:22:27,128][105620] Updated weights for policy 1, policy_version 1235446 (0.0006) [2023-12-27 00:22:27,183][105620] Updated weights for policy 1, policy_version 1235456 (0.0006) [2023-12-27 00:22:27,597][105692] Updated weights for policy 0, policy_version 1234232 (0.0008) [2023-12-27 00:22:27,655][105692] Updated weights for policy 0, policy_version 1234242 (0.0008) [2023-12-27 00:22:27,714][105692] Updated weights for policy 0, policy_version 1234252 (0.0008) [2023-12-27 00:22:27,744][105620] Updated weights for policy 1, policy_version 1235466 (0.0008) [2023-12-27 00:22:27,800][105620] Updated weights for policy 1, policy_version 1235476 (0.0010) [2023-12-27 00:22:27,855][105620] Updated weights for policy 1, policy_version 1235486 (0.0009) [2023-12-27 00:22:28,287][105692] Updated weights for policy 0, policy_version 1234262 (0.0005) [2023-12-27 00:22:28,341][105692] Updated weights for policy 0, policy_version 1234272 (0.0008) [2023-12-27 00:22:28,401][105692] Updated weights for policy 0, policy_version 1234282 (0.0007) [2023-12-27 00:22:28,558][105620] Updated weights for policy 1, policy_version 1235496 (0.0006) [2023-12-27 00:22:28,621][105620] Updated weights for policy 1, policy_version 1235506 (0.0005) [2023-12-27 00:22:28,681][105620] Updated weights for policy 1, policy_version 1235516 (0.0006) [2023-12-27 00:22:28,954][105692] Updated weights for policy 0, policy_version 1234292 (0.0006) [2023-12-27 00:22:29,012][105692] Updated weights for policy 0, policy_version 1234302 (0.0006) [2023-12-27 00:22:29,080][105692] Updated weights for policy 0, policy_version 1234312 (0.0005) [2023-12-27 00:22:29,268][105620] Updated weights for policy 1, policy_version 1235526 (0.0006) [2023-12-27 00:22:29,329][105620] Updated weights for policy 1, policy_version 1235536 (0.0006) [2023-12-27 00:22:29,395][105620] Updated weights for policy 1, policy_version 1235546 (0.0010) [2023-12-27 00:22:29,692][105692] Updated weights for policy 0, policy_version 1234322 (0.0005) [2023-12-27 00:22:29,746][105692] Updated weights for policy 0, policy_version 1234332 (0.0006) [2023-12-27 00:22:29,802][105692] Updated weights for policy 0, policy_version 1234342 (0.0005) [2023-12-27 00:22:29,864][105692] Updated weights for policy 0, policy_version 1234352 (0.0008) [2023-12-27 00:22:30,181][105620] Updated weights for policy 1, policy_version 1235556 (0.0009) [2023-12-27 00:22:30,245][105620] Updated weights for policy 1, policy_version 1235566 (0.0009) [2023-12-27 00:22:30,303][105620] Updated weights for policy 1, policy_version 1235576 (0.0010) [2023-12-27 00:22:30,428][105692] Updated weights for policy 0, policy_version 1234362 (0.0005) [2023-12-27 00:22:30,483][105692] Updated weights for policy 0, policy_version 1234372 (0.0006) [2023-12-27 00:22:30,543][105692] Updated weights for policy 0, policy_version 1234382 (0.0006) [2023-12-27 00:22:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 632406016. Throughput: 0: 9671.7, 1: 9929.1. Samples: 632376660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:22:31,063][104569] Avg episode reward: [(0, '8598.668'), (1, '9079.360')] [2023-12-27 00:22:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001234384_316055552.pth... [2023-12-27 00:22:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001235584_316350464.pth... [2023-12-27 00:22:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001233264_315768832.pth [2023-12-27 00:22:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001234432_316055552.pth [2023-12-27 00:22:31,142][105692] Updated weights for policy 0, policy_version 1234392 (0.0006) [2023-12-27 00:22:31,178][105620] Updated weights for policy 1, policy_version 1235586 (0.0007) [2023-12-27 00:22:31,206][105692] Updated weights for policy 0, policy_version 1234402 (0.0008) [2023-12-27 00:22:31,235][105620] Updated weights for policy 1, policy_version 1235596 (0.0007) [2023-12-27 00:22:31,270][105692] Updated weights for policy 0, policy_version 1234412 (0.0008) [2023-12-27 00:22:31,295][105620] Updated weights for policy 1, policy_version 1235606 (0.0007) [2023-12-27 00:22:31,366][105620] Updated weights for policy 1, policy_version 1235616 (0.0009) [2023-12-27 00:22:31,923][105692] Updated weights for policy 0, policy_version 1234422 (0.0006) [2023-12-27 00:22:31,982][105692] Updated weights for policy 0, policy_version 1234432 (0.0007) [2023-12-27 00:22:32,041][105692] Updated weights for policy 0, policy_version 1234442 (0.0005) [2023-12-27 00:22:32,173][105620] Updated weights for policy 1, policy_version 1235626 (0.0008) [2023-12-27 00:22:32,231][105620] Updated weights for policy 1, policy_version 1235636 (0.0008) [2023-12-27 00:22:32,285][105620] Updated weights for policy 1, policy_version 1235646 (0.0009) [2023-12-27 00:22:32,745][105692] Updated weights for policy 0, policy_version 1234452 (0.0007) [2023-12-27 00:22:32,805][105692] Updated weights for policy 0, policy_version 1234462 (0.0009) [2023-12-27 00:22:32,856][105692] Updated weights for policy 0, policy_version 1234472 (0.0006) [2023-12-27 00:22:32,988][105620] Updated weights for policy 1, policy_version 1235656 (0.0008) [2023-12-27 00:22:33,047][105620] Updated weights for policy 1, policy_version 1235666 (0.0009) [2023-12-27 00:22:33,112][105620] Updated weights for policy 1, policy_version 1235676 (0.0009) [2023-12-27 00:22:33,538][105692] Updated weights for policy 0, policy_version 1234482 (0.0007) [2023-12-27 00:22:33,599][105692] Updated weights for policy 0, policy_version 1234492 (0.0008) [2023-12-27 00:22:33,657][105692] Updated weights for policy 0, policy_version 1234502 (0.0007) [2023-12-27 00:22:33,702][105692] Updated weights for policy 0, policy_version 1234512 (0.0005) [2023-12-27 00:22:33,927][105620] Updated weights for policy 1, policy_version 1235686 (0.0009) [2023-12-27 00:22:33,987][105620] Updated weights for policy 1, policy_version 1235696 (0.0008) [2023-12-27 00:22:34,039][105620] Updated weights for policy 1, policy_version 1235706 (0.0009) [2023-12-27 00:22:34,356][105692] Updated weights for policy 0, policy_version 1234522 (0.0009) [2023-12-27 00:22:34,406][105692] Updated weights for policy 0, policy_version 1234532 (0.0011) [2023-12-27 00:22:34,458][105692] Updated weights for policy 0, policy_version 1234542 (0.0011) [2023-12-27 00:22:34,814][105620] Updated weights for policy 1, policy_version 1235716 (0.0008) [2023-12-27 00:22:34,877][105620] Updated weights for policy 1, policy_version 1235726 (0.0008) [2023-12-27 00:22:34,941][105620] Updated weights for policy 1, policy_version 1235736 (0.0008) [2023-12-27 00:22:35,213][105692] Updated weights for policy 0, policy_version 1234552 (0.0006) [2023-12-27 00:22:35,266][105692] Updated weights for policy 0, policy_version 1234562 (0.0008) [2023-12-27 00:22:35,317][105692] Updated weights for policy 0, policy_version 1234572 (0.0009) [2023-12-27 00:22:35,650][105620] Updated weights for policy 1, policy_version 1235746 (0.0008) [2023-12-27 00:22:35,712][105620] Updated weights for policy 1, policy_version 1235756 (0.0005) [2023-12-27 00:22:35,762][105620] Updated weights for policy 1, policy_version 1235766 (0.0005) [2023-12-27 00:22:35,825][105620] Updated weights for policy 1, policy_version 1235776 (0.0007) [2023-12-27 00:22:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 632504320. Throughput: 0: 9840.7, 1: 9799.1. Samples: 632494792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:22:36,062][104569] Avg episode reward: [(0, '9087.367'), (1, '9170.666')] [2023-12-27 00:22:36,139][105692] Updated weights for policy 0, policy_version 1234582 (0.0010) [2023-12-27 00:22:36,192][105692] Updated weights for policy 0, policy_version 1234592 (0.0010) [2023-12-27 00:22:36,248][105692] Updated weights for policy 0, policy_version 1234602 (0.0010) [2023-12-27 00:22:36,514][105620] Updated weights for policy 1, policy_version 1235786 (0.0008) [2023-12-27 00:22:36,573][105620] Updated weights for policy 1, policy_version 1235796 (0.0008) [2023-12-27 00:22:36,638][105620] Updated weights for policy 1, policy_version 1235806 (0.0008) [2023-12-27 00:22:37,005][105692] Updated weights for policy 0, policy_version 1234612 (0.0009) [2023-12-27 00:22:37,063][105692] Updated weights for policy 0, policy_version 1234622 (0.0008) [2023-12-27 00:22:37,123][105692] Updated weights for policy 0, policy_version 1234632 (0.0009) [2023-12-27 00:22:37,414][105620] Updated weights for policy 1, policy_version 1235816 (0.0009) [2023-12-27 00:22:37,476][105620] Updated weights for policy 1, policy_version 1235826 (0.0009) [2023-12-27 00:22:37,535][105620] Updated weights for policy 1, policy_version 1235836 (0.0009) [2023-12-27 00:22:37,813][105692] Updated weights for policy 0, policy_version 1234642 (0.0009) [2023-12-27 00:22:37,869][105692] Updated weights for policy 0, policy_version 1234652 (0.0008) [2023-12-27 00:22:37,920][105692] Updated weights for policy 0, policy_version 1234662 (0.0008) [2023-12-27 00:22:37,974][105692] Updated weights for policy 0, policy_version 1234672 (0.0005) [2023-12-27 00:22:38,403][105620] Updated weights for policy 1, policy_version 1235846 (0.0008) [2023-12-27 00:22:38,466][105620] Updated weights for policy 1, policy_version 1235856 (0.0008) [2023-12-27 00:22:38,524][105620] Updated weights for policy 1, policy_version 1235866 (0.0008) [2023-12-27 00:22:38,596][105692] Updated weights for policy 0, policy_version 1234682 (0.0008) [2023-12-27 00:22:38,643][105692] Updated weights for policy 0, policy_version 1234692 (0.0007) [2023-12-27 00:22:38,694][105692] Updated weights for policy 0, policy_version 1234702 (0.0008) [2023-12-27 00:22:39,177][105620] Updated weights for policy 1, policy_version 1235876 (0.0009) [2023-12-27 00:22:39,236][105620] Updated weights for policy 1, policy_version 1235886 (0.0011) [2023-12-27 00:22:39,298][105620] Updated weights for policy 1, policy_version 1235896 (0.0011) [2023-12-27 00:22:39,483][105692] Updated weights for policy 0, policy_version 1234712 (0.0010) [2023-12-27 00:22:39,536][105692] Updated weights for policy 0, policy_version 1234722 (0.0009) [2023-12-27 00:22:39,589][105692] Updated weights for policy 0, policy_version 1234732 (0.0010) [2023-12-27 00:22:40,080][105620] Updated weights for policy 1, policy_version 1235906 (0.0008) [2023-12-27 00:22:40,144][105620] Updated weights for policy 1, policy_version 1235916 (0.0010) [2023-12-27 00:22:40,207][105620] Updated weights for policy 1, policy_version 1235926 (0.0011) [2023-12-27 00:22:40,266][105620] Updated weights for policy 1, policy_version 1235936 (0.0010) [2023-12-27 00:22:40,343][105692] Updated weights for policy 0, policy_version 1234742 (0.0010) [2023-12-27 00:22:40,410][105692] Updated weights for policy 0, policy_version 1234752 (0.0009) [2023-12-27 00:22:40,468][105692] Updated weights for policy 0, policy_version 1234762 (0.0009) [2023-12-27 00:22:40,971][105620] Updated weights for policy 1, policy_version 1235946 (0.0009) [2023-12-27 00:22:41,033][105620] Updated weights for policy 1, policy_version 1235956 (0.0007) [2023-12-27 00:22:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 632594432. Throughput: 0: 9837.7, 1: 9746.9. Samples: 632608440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:22:41,063][104569] Avg episode reward: [(0, '9086.983'), (1, '9078.247')] [2023-12-27 00:22:41,094][105620] Updated weights for policy 1, policy_version 1235966 (0.0009) [2023-12-27 00:22:41,182][105692] Updated weights for policy 0, policy_version 1234772 (0.0007) [2023-12-27 00:22:41,244][105692] Updated weights for policy 0, policy_version 1234782 (0.0007) [2023-12-27 00:22:41,308][105692] Updated weights for policy 0, policy_version 1234792 (0.0006) [2023-12-27 00:22:41,872][105620] Updated weights for policy 1, policy_version 1235976 (0.0009) [2023-12-27 00:22:41,927][105620] Updated weights for policy 1, policy_version 1235986 (0.0009) [2023-12-27 00:22:41,993][105620] Updated weights for policy 1, policy_version 1235996 (0.0009) [2023-12-27 00:22:42,067][105692] Updated weights for policy 0, policy_version 1234802 (0.0007) [2023-12-27 00:22:42,136][105692] Updated weights for policy 0, policy_version 1234812 (0.0007) [2023-12-27 00:22:42,201][105692] Updated weights for policy 0, policy_version 1234822 (0.0005) [2023-12-27 00:22:42,270][105692] Updated weights for policy 0, policy_version 1234832 (0.0006) [2023-12-27 00:22:42,820][105620] Updated weights for policy 1, policy_version 1236006 (0.0009) [2023-12-27 00:22:42,885][105620] Updated weights for policy 1, policy_version 1236016 (0.0009) [2023-12-27 00:22:42,943][105620] Updated weights for policy 1, policy_version 1236026 (0.0008) [2023-12-27 00:22:42,973][105692] Updated weights for policy 0, policy_version 1234842 (0.0008) [2023-12-27 00:22:43,037][105692] Updated weights for policy 0, policy_version 1234852 (0.0009) [2023-12-27 00:22:43,096][105692] Updated weights for policy 0, policy_version 1234862 (0.0010) [2023-12-27 00:22:43,620][105620] Updated weights for policy 1, policy_version 1236036 (0.0006) [2023-12-27 00:22:43,670][105620] Updated weights for policy 1, policy_version 1236046 (0.0006) [2023-12-27 00:22:43,733][105620] Updated weights for policy 1, policy_version 1236056 (0.0005) [2023-12-27 00:22:43,911][105692] Updated weights for policy 0, policy_version 1234872 (0.0010) [2023-12-27 00:22:43,969][105692] Updated weights for policy 0, policy_version 1234882 (0.0010) [2023-12-27 00:22:44,025][105692] Updated weights for policy 0, policy_version 1234892 (0.0009) [2023-12-27 00:22:44,249][105620] Updated weights for policy 1, policy_version 1236066 (0.0005) [2023-12-27 00:22:44,304][105620] Updated weights for policy 1, policy_version 1236076 (0.0006) [2023-12-27 00:22:44,358][105620] Updated weights for policy 1, policy_version 1236086 (0.0005) [2023-12-27 00:22:44,408][105620] Updated weights for policy 1, policy_version 1236096 (0.0005) [2023-12-27 00:22:44,666][105692] Updated weights for policy 0, policy_version 1234902 (0.0007) [2023-12-27 00:22:44,719][105692] Updated weights for policy 0, policy_version 1234912 (0.0006) [2023-12-27 00:22:44,766][105692] Updated weights for policy 0, policy_version 1234922 (0.0008) [2023-12-27 00:22:45,118][105620] Updated weights for policy 1, policy_version 1236106 (0.0011) [2023-12-27 00:22:45,166][105620] Updated weights for policy 1, policy_version 1236116 (0.0011) [2023-12-27 00:22:45,226][105620] Updated weights for policy 1, policy_version 1236126 (0.0011) [2023-12-27 00:22:45,542][105692] Updated weights for policy 0, policy_version 1234932 (0.0007) [2023-12-27 00:22:45,610][105692] Updated weights for policy 0, policy_version 1234942 (0.0005) [2023-12-27 00:22:45,669][105692] Updated weights for policy 0, policy_version 1234952 (0.0007) [2023-12-27 00:22:45,983][105620] Updated weights for policy 1, policy_version 1236136 (0.0011) [2023-12-27 00:22:46,034][105620] Updated weights for policy 1, policy_version 1236146 (0.0010) [2023-12-27 00:22:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 632692736. Throughput: 0: 9756.1, 1: 9755.7. Samples: 632665404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:22:46,063][104569] Avg episode reward: [(0, '8906.772'), (1, '8987.546')] [2023-12-27 00:22:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001234960_316203008.pth... [2023-12-27 00:22:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001233808_315908096.pth [2023-12-27 00:22:46,089][105620] Updated weights for policy 1, policy_version 1236156 (0.0010) [2023-12-27 00:22:46,114][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001236160_316497920.pth... [2023-12-27 00:22:46,118][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001235008_316203008.pth [2023-12-27 00:22:46,280][105692] Updated weights for policy 0, policy_version 1234962 (0.0008) [2023-12-27 00:22:46,335][105692] Updated weights for policy 0, policy_version 1234973 (0.0010) [2023-12-27 00:22:46,384][105692] Updated weights for policy 0, policy_version 1234983 (0.0009) [2023-12-27 00:22:46,715][105620] Updated weights for policy 1, policy_version 1236166 (0.0007) [2023-12-27 00:22:46,780][105620] Updated weights for policy 1, policy_version 1236176 (0.0005) [2023-12-27 00:22:46,842][105620] Updated weights for policy 1, policy_version 1236186 (0.0008) [2023-12-27 00:22:47,005][105692] Updated weights for policy 0, policy_version 1234993 (0.0005) [2023-12-27 00:22:47,051][105692] Updated weights for policy 0, policy_version 1235003 (0.0005) [2023-12-27 00:22:47,095][105692] Updated weights for policy 0, policy_version 1235013 (0.0005) [2023-12-27 00:22:47,139][105692] Updated weights for policy 0, policy_version 1235023 (0.0005) [2023-12-27 00:22:47,513][105620] Updated weights for policy 1, policy_version 1236196 (0.0010) [2023-12-27 00:22:47,563][105620] Updated weights for policy 1, policy_version 1236206 (0.0009) [2023-12-27 00:22:47,625][105620] Updated weights for policy 1, policy_version 1236216 (0.0011) [2023-12-27 00:22:47,775][105692] Updated weights for policy 0, policy_version 1235033 (0.0010) [2023-12-27 00:22:47,830][105692] Updated weights for policy 0, policy_version 1235043 (0.0011) [2023-12-27 00:22:47,890][105692] Updated weights for policy 0, policy_version 1235053 (0.0011) [2023-12-27 00:22:48,410][105620] Updated weights for policy 1, policy_version 1236226 (0.0011) [2023-12-27 00:22:48,479][105620] Updated weights for policy 1, policy_version 1236236 (0.0011) [2023-12-27 00:22:48,541][105620] Updated weights for policy 1, policy_version 1236246 (0.0011) [2023-12-27 00:22:48,546][105692] Updated weights for policy 0, policy_version 1235063 (0.0009) [2023-12-27 00:22:48,598][105620] Updated weights for policy 1, policy_version 1236256 (0.0011) [2023-12-27 00:22:48,606][105692] Updated weights for policy 0, policy_version 1235073 (0.0008) [2023-12-27 00:22:48,666][105692] Updated weights for policy 0, policy_version 1235083 (0.0009) [2023-12-27 00:22:49,357][105620] Updated weights for policy 1, policy_version 1236266 (0.0010) [2023-12-27 00:22:49,373][105692] Updated weights for policy 0, policy_version 1235093 (0.0009) [2023-12-27 00:22:49,411][105620] Updated weights for policy 1, policy_version 1236276 (0.0007) [2023-12-27 00:22:49,422][105692] Updated weights for policy 0, policy_version 1235103 (0.0010) [2023-12-27 00:22:49,464][105620] Updated weights for policy 1, policy_version 1236286 (0.0006) [2023-12-27 00:22:49,481][105692] Updated weights for policy 0, policy_version 1235113 (0.0010) [2023-12-27 00:22:50,214][105620] Updated weights for policy 1, policy_version 1236296 (0.0007) [2023-12-27 00:22:50,255][105692] Updated weights for policy 0, policy_version 1235123 (0.0010) [2023-12-27 00:22:50,270][105620] Updated weights for policy 1, policy_version 1236306 (0.0008) [2023-12-27 00:22:50,314][105692] Updated weights for policy 0, policy_version 1235133 (0.0011) [2023-12-27 00:22:50,328][105620] Updated weights for policy 1, policy_version 1236316 (0.0006) [2023-12-27 00:22:50,369][105692] Updated weights for policy 0, policy_version 1235143 (0.0011) [2023-12-27 00:22:51,032][105692] Updated weights for policy 0, policy_version 1235153 (0.0011) [2023-12-27 00:22:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 632791040. Throughput: 0: 9818.9, 1: 9741.6. Samples: 632785716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:22:51,063][104569] Avg episode reward: [(0, '8997.775'), (1, '9078.838')] [2023-12-27 00:22:51,072][105620] Updated weights for policy 1, policy_version 1236326 (0.0008) [2023-12-27 00:22:51,094][105692] Updated weights for policy 0, policy_version 1235163 (0.0007) [2023-12-27 00:22:51,130][105620] Updated weights for policy 1, policy_version 1236336 (0.0008) [2023-12-27 00:22:51,159][105692] Updated weights for policy 0, policy_version 1235173 (0.0008) [2023-12-27 00:22:51,187][105620] Updated weights for policy 1, policy_version 1236346 (0.0006) [2023-12-27 00:22:51,210][105692] Updated weights for policy 0, policy_version 1235183 (0.0010) [2023-12-27 00:22:51,910][105692] Updated weights for policy 0, policy_version 1235193 (0.0008) [2023-12-27 00:22:51,965][105692] Updated weights for policy 0, policy_version 1235203 (0.0008) [2023-12-27 00:22:51,973][105620] Updated weights for policy 1, policy_version 1236356 (0.0008) [2023-12-27 00:22:52,018][105692] Updated weights for policy 0, policy_version 1235213 (0.0008) [2023-12-27 00:22:52,029][105620] Updated weights for policy 1, policy_version 1236366 (0.0007) [2023-12-27 00:22:52,088][105620] Updated weights for policy 1, policy_version 1236376 (0.0008) [2023-12-27 00:22:52,725][105692] Updated weights for policy 0, policy_version 1235223 (0.0009) [2023-12-27 00:22:52,784][105692] Updated weights for policy 0, policy_version 1235233 (0.0010) [2023-12-27 00:22:52,848][105692] Updated weights for policy 0, policy_version 1235243 (0.0011) [2023-12-27 00:22:52,896][105620] Updated weights for policy 1, policy_version 1236386 (0.0009) [2023-12-27 00:22:52,950][105620] Updated weights for policy 1, policy_version 1236396 (0.0008) [2023-12-27 00:22:53,017][105620] Updated weights for policy 1, policy_version 1236406 (0.0008) [2023-12-27 00:22:53,077][105620] Updated weights for policy 1, policy_version 1236416 (0.0006) [2023-12-27 00:22:53,593][105692] Updated weights for policy 0, policy_version 1235253 (0.0010) [2023-12-27 00:22:53,645][105692] Updated weights for policy 0, policy_version 1235263 (0.0010) [2023-12-27 00:22:53,693][105692] Updated weights for policy 0, policy_version 1235273 (0.0010) [2023-12-27 00:22:53,827][105620] Updated weights for policy 1, policy_version 1236426 (0.0008) [2023-12-27 00:22:53,875][105620] Updated weights for policy 1, policy_version 1236436 (0.0008) [2023-12-27 00:22:53,927][105620] Updated weights for policy 1, policy_version 1236446 (0.0007) [2023-12-27 00:22:54,445][105692] Updated weights for policy 0, policy_version 1235283 (0.0009) [2023-12-27 00:22:54,499][105692] Updated weights for policy 0, policy_version 1235293 (0.0008) [2023-12-27 00:22:54,558][105692] Updated weights for policy 0, policy_version 1235303 (0.0005) [2023-12-27 00:22:54,708][105620] Updated weights for policy 1, policy_version 1236456 (0.0008) [2023-12-27 00:22:54,766][105620] Updated weights for policy 1, policy_version 1236466 (0.0009) [2023-12-27 00:22:54,834][105620] Updated weights for policy 1, policy_version 1236476 (0.0009) [2023-12-27 00:22:55,189][105692] Updated weights for policy 0, policy_version 1235313 (0.0006) [2023-12-27 00:22:55,242][105692] Updated weights for policy 0, policy_version 1235323 (0.0005) [2023-12-27 00:22:55,296][105692] Updated weights for policy 0, policy_version 1235333 (0.0005) [2023-12-27 00:22:55,356][105692] Updated weights for policy 0, policy_version 1235343 (0.0005) [2023-12-27 00:22:55,624][105620] Updated weights for policy 1, policy_version 1236486 (0.0009) [2023-12-27 00:22:55,693][105620] Updated weights for policy 1, policy_version 1236496 (0.0008) [2023-12-27 00:22:55,758][105620] Updated weights for policy 1, policy_version 1236506 (0.0008) [2023-12-27 00:22:55,932][105692] Updated weights for policy 0, policy_version 1235353 (0.0005) [2023-12-27 00:22:55,987][105692] Updated weights for policy 0, policy_version 1235363 (0.0005) [2023-12-27 00:22:56,041][105692] Updated weights for policy 0, policy_version 1235373 (0.0005) [2023-12-27 00:22:56,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 632897536. Throughput: 0: 9939.6, 1: 9624.2. Samples: 632900120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:22:56,062][104569] Avg episode reward: [(0, '9085.957'), (1, '9077.762')] [2023-12-27 00:22:56,471][105620] Updated weights for policy 1, policy_version 1236516 (0.0008) [2023-12-27 00:22:56,529][105620] Updated weights for policy 1, policy_version 1236526 (0.0009) [2023-12-27 00:22:56,583][105620] Updated weights for policy 1, policy_version 1236536 (0.0009) [2023-12-27 00:22:56,676][105692] Updated weights for policy 0, policy_version 1235383 (0.0005) [2023-12-27 00:22:56,732][105692] Updated weights for policy 0, policy_version 1235393 (0.0005) [2023-12-27 00:22:56,782][105692] Updated weights for policy 0, policy_version 1235403 (0.0005) [2023-12-27 00:22:57,369][105620] Updated weights for policy 1, policy_version 1236546 (0.0010) [2023-12-27 00:22:57,430][105620] Updated weights for policy 1, policy_version 1236556 (0.0010) [2023-12-27 00:22:57,462][105692] Updated weights for policy 0, policy_version 1235413 (0.0008) [2023-12-27 00:22:57,486][105620] Updated weights for policy 1, policy_version 1236566 (0.0010) [2023-12-27 00:22:57,523][105692] Updated weights for policy 0, policy_version 1235423 (0.0007) [2023-12-27 00:22:57,540][105620] Updated weights for policy 1, policy_version 1236576 (0.0005) [2023-12-27 00:22:57,578][105692] Updated weights for policy 0, policy_version 1235433 (0.0010) [2023-12-27 00:22:58,072][105620] Updated weights for policy 1, policy_version 1236586 (0.0005) [2023-12-27 00:22:58,123][105620] Updated weights for policy 1, policy_version 1236596 (0.0005) [2023-12-27 00:22:58,189][105620] Updated weights for policy 1, policy_version 1236606 (0.0008) [2023-12-27 00:22:58,457][105692] Updated weights for policy 0, policy_version 1235443 (0.0010) [2023-12-27 00:22:58,507][105692] Updated weights for policy 0, policy_version 1235453 (0.0009) [2023-12-27 00:22:58,562][105692] Updated weights for policy 0, policy_version 1235463 (0.0009) [2023-12-27 00:22:59,033][105620] Updated weights for policy 1, policy_version 1236616 (0.0008) [2023-12-27 00:22:59,101][105620] Updated weights for policy 1, policy_version 1236626 (0.0007) [2023-12-27 00:22:59,170][105620] Updated weights for policy 1, policy_version 1236636 (0.0006) [2023-12-27 00:22:59,530][105692] Updated weights for policy 0, policy_version 1235473 (0.0009) [2023-12-27 00:22:59,596][105692] Updated weights for policy 0, policy_version 1235483 (0.0009) [2023-12-27 00:22:59,659][105692] Updated weights for policy 0, policy_version 1235493 (0.0007) [2023-12-27 00:22:59,722][105692] Updated weights for policy 0, policy_version 1235503 (0.0007) [2023-12-27 00:22:59,834][105620] Updated weights for policy 1, policy_version 1236646 (0.0008) [2023-12-27 00:22:59,904][105620] Updated weights for policy 1, policy_version 1236656 (0.0007) [2023-12-27 00:22:59,968][105620] Updated weights for policy 1, policy_version 1236666 (0.0008) [2023-12-27 00:23:00,416][105692] Updated weights for policy 0, policy_version 1235513 (0.0008) [2023-12-27 00:23:00,462][105692] Updated weights for policy 0, policy_version 1235523 (0.0008) [2023-12-27 00:23:00,517][105692] Updated weights for policy 0, policy_version 1235533 (0.0006) [2023-12-27 00:23:00,631][105620] Updated weights for policy 1, policy_version 1236676 (0.0007) [2023-12-27 00:23:00,697][105620] Updated weights for policy 1, policy_version 1236686 (0.0005) [2023-12-27 00:23:00,757][105620] Updated weights for policy 1, policy_version 1236696 (0.0010) [2023-12-27 00:23:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 632987648. Throughput: 0: 9948.5, 1: 9634.1. Samples: 632958452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:23:01,063][104569] Avg episode reward: [(0, '9086.068'), (1, '9169.461')] [2023-12-27 00:23:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001235536_316350464.pth... [2023-12-27 00:23:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001236704_316637184.pth... [2023-12-27 00:23:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001235584_316350464.pth [2023-12-27 00:23:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001234384_316055552.pth [2023-12-27 00:23:01,214][105692] Updated weights for policy 0, policy_version 1235543 (0.0009) [2023-12-27 00:23:01,272][105692] Updated weights for policy 0, policy_version 1235553 (0.0009) [2023-12-27 00:23:01,335][105692] Updated weights for policy 0, policy_version 1235563 (0.0006) [2023-12-27 00:23:01,404][105620] Updated weights for policy 1, policy_version 1236706 (0.0009) [2023-12-27 00:23:01,451][105620] Updated weights for policy 1, policy_version 1236716 (0.0005) [2023-12-27 00:23:01,508][105620] Updated weights for policy 1, policy_version 1236726 (0.0005) [2023-12-27 00:23:01,554][105620] Updated weights for policy 1, policy_version 1236736 (0.0005) [2023-12-27 00:23:02,098][105692] Updated weights for policy 0, policy_version 1235573 (0.0009) [2023-12-27 00:23:02,147][105692] Updated weights for policy 0, policy_version 1235583 (0.0010) [2023-12-27 00:23:02,162][105620] Updated weights for policy 1, policy_version 1236746 (0.0011) [2023-12-27 00:23:02,199][105692] Updated weights for policy 0, policy_version 1235593 (0.0010) [2023-12-27 00:23:02,220][105620] Updated weights for policy 1, policy_version 1236756 (0.0010) [2023-12-27 00:23:02,282][105620] Updated weights for policy 1, policy_version 1236766 (0.0011) [2023-12-27 00:23:02,926][105692] Updated weights for policy 0, policy_version 1235603 (0.0010) [2023-12-27 00:23:02,988][105692] Updated weights for policy 0, policy_version 1235613 (0.0009) [2023-12-27 00:23:03,007][105620] Updated weights for policy 1, policy_version 1236776 (0.0008) [2023-12-27 00:23:03,047][105692] Updated weights for policy 0, policy_version 1235623 (0.0010) [2023-12-27 00:23:03,063][105620] Updated weights for policy 1, policy_version 1236786 (0.0005) [2023-12-27 00:23:03,121][105620] Updated weights for policy 1, policy_version 1236796 (0.0005) [2023-12-27 00:23:03,708][105620] Updated weights for policy 1, policy_version 1236806 (0.0006) [2023-12-27 00:23:03,757][105620] Updated weights for policy 1, policy_version 1236816 (0.0005) [2023-12-27 00:23:03,811][105620] Updated weights for policy 1, policy_version 1236826 (0.0010) [2023-12-27 00:23:03,811][105692] Updated weights for policy 0, policy_version 1235633 (0.0010) [2023-12-27 00:23:03,883][105692] Updated weights for policy 0, policy_version 1235643 (0.0010) [2023-12-27 00:23:03,934][105692] Updated weights for policy 0, policy_version 1235653 (0.0010) [2023-12-27 00:23:03,993][105692] Updated weights for policy 0, policy_version 1235663 (0.0011) [2023-12-27 00:23:04,532][105620] Updated weights for policy 1, policy_version 1236836 (0.0010) [2023-12-27 00:23:04,585][105620] Updated weights for policy 1, policy_version 1236846 (0.0011) [2023-12-27 00:23:04,642][105620] Updated weights for policy 1, policy_version 1236856 (0.0011) [2023-12-27 00:23:04,654][105692] Updated weights for policy 0, policy_version 1235673 (0.0010) [2023-12-27 00:23:04,714][105692] Updated weights for policy 0, policy_version 1235683 (0.0011) [2023-12-27 00:23:04,772][105692] Updated weights for policy 0, policy_version 1235693 (0.0010) [2023-12-27 00:23:05,276][105620] Updated weights for policy 1, policy_version 1236866 (0.0010) [2023-12-27 00:23:05,335][105620] Updated weights for policy 1, policy_version 1236876 (0.0005) [2023-12-27 00:23:05,400][105620] Updated weights for policy 1, policy_version 1236886 (0.0007) [2023-12-27 00:23:05,450][105620] Updated weights for policy 1, policy_version 1236896 (0.0008) [2023-12-27 00:23:05,520][105692] Updated weights for policy 0, policy_version 1235703 (0.0010) [2023-12-27 00:23:05,579][105692] Updated weights for policy 0, policy_version 1235713 (0.0010) [2023-12-27 00:23:05,637][105692] Updated weights for policy 0, policy_version 1235723 (0.0010) [2023-12-27 00:23:06,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 633085952. Throughput: 0: 9915.1, 1: 9681.9. Samples: 633076848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:23:06,063][104569] Avg episode reward: [(0, '8815.738'), (1, '9171.697')] [2023-12-27 00:23:06,168][105620] Updated weights for policy 1, policy_version 1236906 (0.0007) [2023-12-27 00:23:06,224][105620] Updated weights for policy 1, policy_version 1236916 (0.0011) [2023-12-27 00:23:06,276][105620] Updated weights for policy 1, policy_version 1236926 (0.0011) [2023-12-27 00:23:06,347][105692] Updated weights for policy 0, policy_version 1235733 (0.0009) [2023-12-27 00:23:06,419][105692] Updated weights for policy 0, policy_version 1235743 (0.0010) [2023-12-27 00:23:06,489][105692] Updated weights for policy 0, policy_version 1235753 (0.0011) [2023-12-27 00:23:07,031][105620] Updated weights for policy 1, policy_version 1236936 (0.0010) [2023-12-27 00:23:07,094][105620] Updated weights for policy 1, policy_version 1236946 (0.0011) [2023-12-27 00:23:07,160][105620] Updated weights for policy 1, policy_version 1236956 (0.0011) [2023-12-27 00:23:07,224][105692] Updated weights for policy 0, policy_version 1235763 (0.0010) [2023-12-27 00:23:07,280][105692] Updated weights for policy 0, policy_version 1235773 (0.0008) [2023-12-27 00:23:07,329][105692] Updated weights for policy 0, policy_version 1235783 (0.0010) [2023-12-27 00:23:07,884][105620] Updated weights for policy 1, policy_version 1236966 (0.0011) [2023-12-27 00:23:07,943][105620] Updated weights for policy 1, policy_version 1236976 (0.0010) [2023-12-27 00:23:08,002][105620] Updated weights for policy 1, policy_version 1236986 (0.0010) [2023-12-27 00:23:08,087][105692] Updated weights for policy 0, policy_version 1235793 (0.0011) [2023-12-27 00:23:08,135][105692] Updated weights for policy 0, policy_version 1235803 (0.0011) [2023-12-27 00:23:08,181][105692] Updated weights for policy 0, policy_version 1235813 (0.0010) [2023-12-27 00:23:08,230][105692] Updated weights for policy 0, policy_version 1235823 (0.0011) [2023-12-27 00:23:08,689][105620] Updated weights for policy 1, policy_version 1236996 (0.0011) [2023-12-27 00:23:08,743][105620] Updated weights for policy 1, policy_version 1237006 (0.0011) [2023-12-27 00:23:08,794][105620] Updated weights for policy 1, policy_version 1237016 (0.0010) [2023-12-27 00:23:09,036][105692] Updated weights for policy 0, policy_version 1235833 (0.0011) [2023-12-27 00:23:09,098][105692] Updated weights for policy 0, policy_version 1235843 (0.0010) [2023-12-27 00:23:09,185][105692] Updated weights for policy 0, policy_version 1235853 (0.0009) [2023-12-27 00:23:09,518][105620] Updated weights for policy 1, policy_version 1237026 (0.0010) [2023-12-27 00:23:09,587][105620] Updated weights for policy 1, policy_version 1237036 (0.0005) [2023-12-27 00:23:09,650][105620] Updated weights for policy 1, policy_version 1237046 (0.0006) [2023-12-27 00:23:09,714][105620] Updated weights for policy 1, policy_version 1237056 (0.0007) [2023-12-27 00:23:10,024][105692] Updated weights for policy 0, policy_version 1235863 (0.0009) [2023-12-27 00:23:10,091][105692] Updated weights for policy 0, policy_version 1235873 (0.0010) [2023-12-27 00:23:10,154][105692] Updated weights for policy 0, policy_version 1235883 (0.0010) [2023-12-27 00:23:10,398][105620] Updated weights for policy 1, policy_version 1237066 (0.0008) [2023-12-27 00:23:10,465][105620] Updated weights for policy 1, policy_version 1237076 (0.0008) [2023-12-27 00:23:10,521][105620] Updated weights for policy 1, policy_version 1237086 (0.0008) [2023-12-27 00:23:10,905][105692] Updated weights for policy 0, policy_version 1235893 (0.0010) [2023-12-27 00:23:10,968][105692] Updated weights for policy 0, policy_version 1235903 (0.0009) [2023-12-27 00:23:11,019][105692] Updated weights for policy 0, policy_version 1235913 (0.0009) [2023-12-27 00:23:11,063][104569] Fps is (10 sec: 18839.7, 60 sec: 19387.4, 300 sec: 19521.9). Total num frames: 633176064. Throughput: 0: 9852.1, 1: 9637.9. Samples: 633190384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:23:11,063][104569] Avg episode reward: [(0, '8725.539'), (1, '9080.545')] [2023-12-27 00:23:11,280][105620] Updated weights for policy 1, policy_version 1237096 (0.0009) [2023-12-27 00:23:11,342][105620] Updated weights for policy 1, policy_version 1237106 (0.0009) [2023-12-27 00:23:11,406][105620] Updated weights for policy 1, policy_version 1237116 (0.0010) [2023-12-27 00:23:11,877][105692] Updated weights for policy 0, policy_version 1235923 (0.0008) [2023-12-27 00:23:11,940][105692] Updated weights for policy 0, policy_version 1235933 (0.0007) [2023-12-27 00:23:12,000][105692] Updated weights for policy 0, policy_version 1235943 (0.0006) [2023-12-27 00:23:12,237][105620] Updated weights for policy 1, policy_version 1237126 (0.0010) [2023-12-27 00:23:12,301][105620] Updated weights for policy 1, policy_version 1237136 (0.0007) [2023-12-27 00:23:12,367][105620] Updated weights for policy 1, policy_version 1237146 (0.0008) [2023-12-27 00:23:12,676][105692] Updated weights for policy 0, policy_version 1235953 (0.0006) [2023-12-27 00:23:12,724][105692] Updated weights for policy 0, policy_version 1235963 (0.0010) [2023-12-27 00:23:12,786][105692] Updated weights for policy 0, policy_version 1235973 (0.0007) [2023-12-27 00:23:12,852][105692] Updated weights for policy 0, policy_version 1235983 (0.0006) [2023-12-27 00:23:13,130][105620] Updated weights for policy 1, policy_version 1237156 (0.0010) [2023-12-27 00:23:13,192][105620] Updated weights for policy 1, policy_version 1237166 (0.0011) [2023-12-27 00:23:13,250][105620] Updated weights for policy 1, policy_version 1237176 (0.0010) [2023-12-27 00:23:13,488][105692] Updated weights for policy 0, policy_version 1235993 (0.0008) [2023-12-27 00:23:13,537][105692] Updated weights for policy 0, policy_version 1236003 (0.0008) [2023-12-27 00:23:13,592][105692] Updated weights for policy 0, policy_version 1236013 (0.0010) [2023-12-27 00:23:13,900][105620] Updated weights for policy 1, policy_version 1237186 (0.0010) [2023-12-27 00:23:13,965][105620] Updated weights for policy 1, policy_version 1237196 (0.0008) [2023-12-27 00:23:14,026][105620] Updated weights for policy 1, policy_version 1237206 (0.0011) [2023-12-27 00:23:14,089][105620] Updated weights for policy 1, policy_version 1237216 (0.0011) [2023-12-27 00:23:14,247][105692] Updated weights for policy 0, policy_version 1236023 (0.0010) [2023-12-27 00:23:14,299][105692] Updated weights for policy 0, policy_version 1236033 (0.0008) [2023-12-27 00:23:14,363][105692] Updated weights for policy 0, policy_version 1236043 (0.0009) [2023-12-27 00:23:14,796][105620] Updated weights for policy 1, policy_version 1237226 (0.0007) [2023-12-27 00:23:14,863][105620] Updated weights for policy 1, policy_version 1237236 (0.0009) [2023-12-27 00:23:14,928][105620] Updated weights for policy 1, policy_version 1237246 (0.0009) [2023-12-27 00:23:15,024][105692] Updated weights for policy 0, policy_version 1236053 (0.0009) [2023-12-27 00:23:15,084][105692] Updated weights for policy 0, policy_version 1236063 (0.0008) [2023-12-27 00:23:15,148][105692] Updated weights for policy 0, policy_version 1236073 (0.0008) [2023-12-27 00:23:15,625][105620] Updated weights for policy 1, policy_version 1237256 (0.0008) [2023-12-27 00:23:15,670][105620] Updated weights for policy 1, policy_version 1237266 (0.0008) [2023-12-27 00:23:15,715][105620] Updated weights for policy 1, policy_version 1237276 (0.0008) [2023-12-27 00:23:15,861][105692] Updated weights for policy 0, policy_version 1236083 (0.0008) [2023-12-27 00:23:15,919][105692] Updated weights for policy 0, policy_version 1236093 (0.0006) [2023-12-27 00:23:15,984][105692] Updated weights for policy 0, policy_version 1236103 (0.0010) [2023-12-27 00:23:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 633282560. Throughput: 0: 9788.9, 1: 9550.4. Samples: 633246928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:23:16,063][104569] Avg episode reward: [(0, '8816.098'), (1, '9168.898')] [2023-12-27 00:23:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001236112_316497920.pth... [2023-12-27 00:23:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001237280_316784640.pth... [2023-12-27 00:23:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001236160_316497920.pth [2023-12-27 00:23:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001234960_316203008.pth [2023-12-27 00:23:16,460][105620] Updated weights for policy 1, policy_version 1237286 (0.0009) [2023-12-27 00:23:16,508][105620] Updated weights for policy 1, policy_version 1237296 (0.0008) [2023-12-27 00:23:16,555][105620] Updated weights for policy 1, policy_version 1237306 (0.0008) [2023-12-27 00:23:16,699][105692] Updated weights for policy 0, policy_version 1236113 (0.0011) [2023-12-27 00:23:16,771][105692] Updated weights for policy 0, policy_version 1236123 (0.0011) [2023-12-27 00:23:16,834][105692] Updated weights for policy 0, policy_version 1236133 (0.0010) [2023-12-27 00:23:16,888][105692] Updated weights for policy 0, policy_version 1236143 (0.0010) [2023-12-27 00:23:17,271][105620] Updated weights for policy 1, policy_version 1237316 (0.0008) [2023-12-27 00:23:17,323][105620] Updated weights for policy 1, policy_version 1237326 (0.0008) [2023-12-27 00:23:17,377][105620] Updated weights for policy 1, policy_version 1237336 (0.0008) [2023-12-27 00:23:17,599][105692] Updated weights for policy 0, policy_version 1236153 (0.0006) [2023-12-27 00:23:17,645][105692] Updated weights for policy 0, policy_version 1236163 (0.0005) [2023-12-27 00:23:17,691][105692] Updated weights for policy 0, policy_version 1236173 (0.0005) [2023-12-27 00:23:18,116][105620] Updated weights for policy 1, policy_version 1237346 (0.0008) [2023-12-27 00:23:18,175][105620] Updated weights for policy 1, policy_version 1237356 (0.0009) [2023-12-27 00:23:18,237][105620] Updated weights for policy 1, policy_version 1237366 (0.0007) [2023-12-27 00:23:18,272][105692] Updated weights for policy 0, policy_version 1236183 (0.0008) [2023-12-27 00:23:18,293][105620] Updated weights for policy 1, policy_version 1237376 (0.0007) [2023-12-27 00:23:18,348][105692] Updated weights for policy 0, policy_version 1236193 (0.0009) [2023-12-27 00:23:18,405][105692] Updated weights for policy 0, policy_version 1236203 (0.0008) [2023-12-27 00:23:19,084][105620] Updated weights for policy 1, policy_version 1237386 (0.0007) [2023-12-27 00:23:19,090][105692] Updated weights for policy 0, policy_version 1236213 (0.0008) [2023-12-27 00:23:19,144][105620] Updated weights for policy 1, policy_version 1237396 (0.0009) [2023-12-27 00:23:19,148][105692] Updated weights for policy 0, policy_version 1236223 (0.0009) [2023-12-27 00:23:19,197][105620] Updated weights for policy 1, policy_version 1237406 (0.0006) [2023-12-27 00:23:19,207][105692] Updated weights for policy 0, policy_version 1236233 (0.0011) [2023-12-27 00:23:20,046][105692] Updated weights for policy 0, policy_version 1236243 (0.0009) [2023-12-27 00:23:20,065][105620] Updated weights for policy 1, policy_version 1237416 (0.0008) [2023-12-27 00:23:20,107][105692] Updated weights for policy 0, policy_version 1236253 (0.0010) [2023-12-27 00:23:20,125][105620] Updated weights for policy 1, policy_version 1237426 (0.0006) [2023-12-27 00:23:20,170][105692] Updated weights for policy 0, policy_version 1236263 (0.0011) [2023-12-27 00:23:20,189][105620] Updated weights for policy 1, policy_version 1237436 (0.0008) [2023-12-27 00:23:20,844][105692] Updated weights for policy 0, policy_version 1236273 (0.0010) [2023-12-27 00:23:20,909][105692] Updated weights for policy 0, policy_version 1236283 (0.0006) [2023-12-27 00:23:20,966][105692] Updated weights for policy 0, policy_version 1236293 (0.0011) [2023-12-27 00:23:21,015][105620] Updated weights for policy 1, policy_version 1237446 (0.0007) [2023-12-27 00:23:21,032][105692] Updated weights for policy 0, policy_version 1236303 (0.0009) [2023-12-27 00:23:21,062][104569] Fps is (10 sec: 19662.6, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 633372672. Throughput: 0: 9690.5, 1: 9604.5. Samples: 633363072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:23:21,063][104569] Avg episode reward: [(0, '8905.922'), (1, '9076.477')] [2023-12-27 00:23:21,087][105620] Updated weights for policy 1, policy_version 1237456 (0.0009) [2023-12-27 00:23:21,152][105620] Updated weights for policy 1, policy_version 1237466 (0.0009) [2023-12-27 00:23:21,821][105692] Updated weights for policy 0, policy_version 1236313 (0.0006) [2023-12-27 00:23:21,883][105692] Updated weights for policy 0, policy_version 1236323 (0.0006) [2023-12-27 00:23:21,887][105620] Updated weights for policy 1, policy_version 1237476 (0.0009) [2023-12-27 00:23:21,938][105620] Updated weights for policy 1, policy_version 1237486 (0.0006) [2023-12-27 00:23:21,943][105692] Updated weights for policy 0, policy_version 1236333 (0.0010) [2023-12-27 00:23:22,002][105620] Updated weights for policy 1, policy_version 1237496 (0.0007) [2023-12-27 00:23:22,606][105692] Updated weights for policy 0, policy_version 1236343 (0.0011) [2023-12-27 00:23:22,675][105692] Updated weights for policy 0, policy_version 1236353 (0.0006) [2023-12-27 00:23:22,746][105692] Updated weights for policy 0, policy_version 1236363 (0.0006) [2023-12-27 00:23:22,805][105620] Updated weights for policy 1, policy_version 1237506 (0.0008) [2023-12-27 00:23:22,866][105620] Updated weights for policy 1, policy_version 1237516 (0.0011) [2023-12-27 00:23:22,919][105620] Updated weights for policy 1, policy_version 1237526 (0.0010) [2023-12-27 00:23:22,981][105620] Updated weights for policy 1, policy_version 1237536 (0.0010) [2023-12-27 00:23:23,385][105692] Updated weights for policy 0, policy_version 1236373 (0.0009) [2023-12-27 00:23:23,434][105692] Updated weights for policy 0, policy_version 1236383 (0.0011) [2023-12-27 00:23:23,479][105692] Updated weights for policy 0, policy_version 1236393 (0.0010) [2023-12-27 00:23:23,729][105620] Updated weights for policy 1, policy_version 1237546 (0.0010) [2023-12-27 00:23:23,779][105620] Updated weights for policy 1, policy_version 1237556 (0.0008) [2023-12-27 00:23:23,842][105620] Updated weights for policy 1, policy_version 1237566 (0.0009) [2023-12-27 00:23:24,212][105692] Updated weights for policy 0, policy_version 1236403 (0.0009) [2023-12-27 00:23:24,270][105692] Updated weights for policy 0, policy_version 1236413 (0.0006) [2023-12-27 00:23:24,345][105692] Updated weights for policy 0, policy_version 1236423 (0.0007) [2023-12-27 00:23:24,460][105620] Updated weights for policy 1, policy_version 1237576 (0.0008) [2023-12-27 00:23:24,508][105620] Updated weights for policy 1, policy_version 1237586 (0.0009) [2023-12-27 00:23:24,562][105620] Updated weights for policy 1, policy_version 1237596 (0.0009) [2023-12-27 00:23:24,982][105692] Updated weights for policy 0, policy_version 1236433 (0.0010) [2023-12-27 00:23:25,036][105692] Updated weights for policy 0, policy_version 1236443 (0.0008) [2023-12-27 00:23:25,090][105692] Updated weights for policy 0, policy_version 1236453 (0.0009) [2023-12-27 00:23:25,141][105692] Updated weights for policy 0, policy_version 1236463 (0.0009) [2023-12-27 00:23:25,349][105620] Updated weights for policy 1, policy_version 1237606 (0.0007) [2023-12-27 00:23:25,407][105620] Updated weights for policy 1, policy_version 1237616 (0.0007) [2023-12-27 00:23:25,465][105620] Updated weights for policy 1, policy_version 1237626 (0.0008) [2023-12-27 00:23:25,825][105692] Updated weights for policy 0, policy_version 1236473 (0.0006) [2023-12-27 00:23:25,890][105692] Updated weights for policy 0, policy_version 1236483 (0.0006) [2023-12-27 00:23:25,956][105692] Updated weights for policy 0, policy_version 1236493 (0.0006) [2023-12-27 00:23:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 633470976. Throughput: 0: 9721.2, 1: 9608.4. Samples: 633478272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:23:26,063][104569] Avg episode reward: [(0, '9174.685'), (1, '8985.669')] [2023-12-27 00:23:26,078][105620] Updated weights for policy 1, policy_version 1237637 (0.0009) [2023-12-27 00:23:26,129][105620] Updated weights for policy 1, policy_version 1237647 (0.0010) [2023-12-27 00:23:26,185][105620] Updated weights for policy 1, policy_version 1237657 (0.0010) [2023-12-27 00:23:26,449][105692] Updated weights for policy 0, policy_version 1236503 (0.0005) [2023-12-27 00:23:26,498][105692] Updated weights for policy 0, policy_version 1236513 (0.0005) [2023-12-27 00:23:26,544][105692] Updated weights for policy 0, policy_version 1236523 (0.0006) [2023-12-27 00:23:26,919][105620] Updated weights for policy 1, policy_version 1237667 (0.0010) [2023-12-27 00:23:26,963][105620] Updated weights for policy 1, policy_version 1237677 (0.0010) [2023-12-27 00:23:27,026][105620] Updated weights for policy 1, policy_version 1237687 (0.0008) [2023-12-27 00:23:27,170][105692] Updated weights for policy 0, policy_version 1236533 (0.0008) [2023-12-27 00:23:27,224][105692] Updated weights for policy 0, policy_version 1236543 (0.0005) [2023-12-27 00:23:27,269][105692] Updated weights for policy 0, policy_version 1236553 (0.0005) [2023-12-27 00:23:27,701][105620] Updated weights for policy 1, policy_version 1237697 (0.0008) [2023-12-27 00:23:27,759][105620] Updated weights for policy 1, policy_version 1237707 (0.0010) [2023-12-27 00:23:27,803][105620] Updated weights for policy 1, policy_version 1237717 (0.0010) [2023-12-27 00:23:27,847][105620] Updated weights for policy 1, policy_version 1237727 (0.0010) [2023-12-27 00:23:27,933][105692] Updated weights for policy 0, policy_version 1236563 (0.0010) [2023-12-27 00:23:27,990][105692] Updated weights for policy 0, policy_version 1236573 (0.0008) [2023-12-27 00:23:28,044][105692] Updated weights for policy 0, policy_version 1236583 (0.0008) [2023-12-27 00:23:28,554][105620] Updated weights for policy 1, policy_version 1237737 (0.0006) [2023-12-27 00:23:28,607][105620] Updated weights for policy 1, policy_version 1237747 (0.0005) [2023-12-27 00:23:28,663][105620] Updated weights for policy 1, policy_version 1237757 (0.0006) [2023-12-27 00:23:28,759][105692] Updated weights for policy 0, policy_version 1236593 (0.0008) [2023-12-27 00:23:28,805][105692] Updated weights for policy 0, policy_version 1236603 (0.0006) [2023-12-27 00:23:28,853][105692] Updated weights for policy 0, policy_version 1236613 (0.0008) [2023-12-27 00:23:28,904][105692] Updated weights for policy 0, policy_version 1236623 (0.0008) [2023-12-27 00:23:29,294][105620] Updated weights for policy 1, policy_version 1237767 (0.0010) [2023-12-27 00:23:29,359][105620] Updated weights for policy 1, policy_version 1237777 (0.0009) [2023-12-27 00:23:29,417][105620] Updated weights for policy 1, policy_version 1237787 (0.0006) [2023-12-27 00:23:29,757][105692] Updated weights for policy 0, policy_version 1236633 (0.0006) [2023-12-27 00:23:29,816][105692] Updated weights for policy 0, policy_version 1236643 (0.0010) [2023-12-27 00:23:29,874][105692] Updated weights for policy 0, policy_version 1236653 (0.0007) [2023-12-27 00:23:30,095][105620] Updated weights for policy 1, policy_version 1237797 (0.0007) [2023-12-27 00:23:30,146][105620] Updated weights for policy 1, policy_version 1237807 (0.0008) [2023-12-27 00:23:30,193][105620] Updated weights for policy 1, policy_version 1237817 (0.0008) [2023-12-27 00:23:30,574][105692] Updated weights for policy 0, policy_version 1236663 (0.0009) [2023-12-27 00:23:30,641][105692] Updated weights for policy 0, policy_version 1236673 (0.0010) [2023-12-27 00:23:30,703][105692] Updated weights for policy 0, policy_version 1236683 (0.0010) [2023-12-27 00:23:30,846][105620] Updated weights for policy 1, policy_version 1237827 (0.0009) [2023-12-27 00:23:30,890][105620] Updated weights for policy 1, policy_version 1237837 (0.0010) [2023-12-27 00:23:30,937][105620] Updated weights for policy 1, policy_version 1237847 (0.0010) [2023-12-27 00:23:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 633577472. Throughput: 0: 9822.0, 1: 9648.6. Samples: 633541584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:23:31,062][104569] Avg episode reward: [(0, '9176.142'), (1, '9168.211')] [2023-12-27 00:23:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001236688_316645376.pth... [2023-12-27 00:23:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001237856_316932096.pth... [2023-12-27 00:23:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001235536_316350464.pth [2023-12-27 00:23:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001236704_316637184.pth [2023-12-27 00:23:31,429][105692] Updated weights for policy 0, policy_version 1236693 (0.0010) [2023-12-27 00:23:31,485][105692] Updated weights for policy 0, policy_version 1236703 (0.0009) [2023-12-27 00:23:31,546][105692] Updated weights for policy 0, policy_version 1236713 (0.0008) [2023-12-27 00:23:31,758][105620] Updated weights for policy 1, policy_version 1237857 (0.0010) [2023-12-27 00:23:31,810][105620] Updated weights for policy 1, policy_version 1237867 (0.0008) [2023-12-27 00:23:31,861][105620] Updated weights for policy 1, policy_version 1237877 (0.0010) [2023-12-27 00:23:31,909][105620] Updated weights for policy 1, policy_version 1237887 (0.0010) [2023-12-27 00:23:32,198][105692] Updated weights for policy 0, policy_version 1236723 (0.0007) [2023-12-27 00:23:32,266][105692] Updated weights for policy 0, policy_version 1236733 (0.0006) [2023-12-27 00:23:32,322][105692] Updated weights for policy 0, policy_version 1236743 (0.0011) [2023-12-27 00:23:32,684][105620] Updated weights for policy 1, policy_version 1237897 (0.0010) [2023-12-27 00:23:32,733][105620] Updated weights for policy 1, policy_version 1237907 (0.0010) [2023-12-27 00:23:32,777][105620] Updated weights for policy 1, policy_version 1237917 (0.0010) [2023-12-27 00:23:32,916][105692] Updated weights for policy 0, policy_version 1236753 (0.0009) [2023-12-27 00:23:32,972][105692] Updated weights for policy 0, policy_version 1236763 (0.0005) [2023-12-27 00:23:33,025][105692] Updated weights for policy 0, policy_version 1236773 (0.0005) [2023-12-27 00:23:33,083][105692] Updated weights for policy 0, policy_version 1236783 (0.0007) [2023-12-27 00:23:33,504][105620] Updated weights for policy 1, policy_version 1237927 (0.0010) [2023-12-27 00:23:33,555][105620] Updated weights for policy 1, policy_version 1237937 (0.0010) [2023-12-27 00:23:33,598][105620] Updated weights for policy 1, policy_version 1237947 (0.0010) [2023-12-27 00:23:33,664][105692] Updated weights for policy 0, policy_version 1236793 (0.0007) [2023-12-27 00:23:33,728][105692] Updated weights for policy 0, policy_version 1236803 (0.0008) [2023-12-27 00:23:33,777][105692] Updated weights for policy 0, policy_version 1236813 (0.0007) [2023-12-27 00:23:34,372][105620] Updated weights for policy 1, policy_version 1237957 (0.0010) [2023-12-27 00:23:34,431][105620] Updated weights for policy 1, policy_version 1237967 (0.0011) [2023-12-27 00:23:34,459][105692] Updated weights for policy 0, policy_version 1236823 (0.0006) [2023-12-27 00:23:34,495][105620] Updated weights for policy 1, policy_version 1237977 (0.0011) [2023-12-27 00:23:34,521][105692] Updated weights for policy 0, policy_version 1236833 (0.0006) [2023-12-27 00:23:34,584][105692] Updated weights for policy 0, policy_version 1236843 (0.0007) [2023-12-27 00:23:35,241][105620] Updated weights for policy 1, policy_version 1237987 (0.0011) [2023-12-27 00:23:35,289][105620] Updated weights for policy 1, policy_version 1237997 (0.0008) [2023-12-27 00:23:35,329][105692] Updated weights for policy 0, policy_version 1236853 (0.0007) [2023-12-27 00:23:35,348][105620] Updated weights for policy 1, policy_version 1238007 (0.0008) [2023-12-27 00:23:35,389][105692] Updated weights for policy 0, policy_version 1236863 (0.0007) [2023-12-27 00:23:35,439][105692] Updated weights for policy 0, policy_version 1236873 (0.0009) [2023-12-27 00:23:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 633667584. Throughput: 0: 9817.3, 1: 9636.2. Samples: 633661120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:23:36,062][104569] Avg episode reward: [(0, '9175.262'), (1, '9259.063')] [2023-12-27 00:23:36,094][105692] Updated weights for policy 0, policy_version 1236883 (0.0009) [2023-12-27 00:23:36,153][105692] Updated weights for policy 0, policy_version 1236893 (0.0008) [2023-12-27 00:23:36,172][105620] Updated weights for policy 1, policy_version 1238017 (0.0009) [2023-12-27 00:23:36,216][105692] Updated weights for policy 0, policy_version 1236903 (0.0007) [2023-12-27 00:23:36,230][105620] Updated weights for policy 1, policy_version 1238027 (0.0006) [2023-12-27 00:23:36,295][105620] Updated weights for policy 1, policy_version 1238037 (0.0007) [2023-12-27 00:23:36,361][105620] Updated weights for policy 1, policy_version 1238047 (0.0010) [2023-12-27 00:23:36,859][105692] Updated weights for policy 0, policy_version 1236913 (0.0007) [2023-12-27 00:23:36,910][105692] Updated weights for policy 0, policy_version 1236923 (0.0009) [2023-12-27 00:23:36,962][105692] Updated weights for policy 0, policy_version 1236934 (0.0009) [2023-12-27 00:23:37,009][105692] Updated weights for policy 0, policy_version 1236944 (0.0009) [2023-12-27 00:23:37,157][105620] Updated weights for policy 1, policy_version 1238057 (0.0006) [2023-12-27 00:23:37,210][105620] Updated weights for policy 1, policy_version 1238067 (0.0005) [2023-12-27 00:23:37,263][105620] Updated weights for policy 1, policy_version 1238077 (0.0005) [2023-12-27 00:23:37,826][105692] Updated weights for policy 0, policy_version 1236954 (0.0008) [2023-12-27 00:23:37,885][105692] Updated weights for policy 0, policy_version 1236964 (0.0008) [2023-12-27 00:23:37,948][105620] Updated weights for policy 1, policy_version 1238087 (0.0010) [2023-12-27 00:23:37,949][105692] Updated weights for policy 0, policy_version 1236974 (0.0008) [2023-12-27 00:23:38,003][105620] Updated weights for policy 1, policy_version 1238097 (0.0010) [2023-12-27 00:23:38,074][105620] Updated weights for policy 1, policy_version 1238107 (0.0011) [2023-12-27 00:23:38,693][105692] Updated weights for policy 0, policy_version 1236984 (0.0005) [2023-12-27 00:23:38,745][105692] Updated weights for policy 0, policy_version 1236994 (0.0006) [2023-12-27 00:23:38,784][105620] Updated weights for policy 1, policy_version 1238117 (0.0009) [2023-12-27 00:23:38,805][105692] Updated weights for policy 0, policy_version 1237004 (0.0006) [2023-12-27 00:23:38,848][105620] Updated weights for policy 1, policy_version 1238127 (0.0007) [2023-12-27 00:23:38,918][105620] Updated weights for policy 1, policy_version 1238137 (0.0005) [2023-12-27 00:23:39,558][105692] Updated weights for policy 0, policy_version 1237014 (0.0006) [2023-12-27 00:23:39,624][105692] Updated weights for policy 0, policy_version 1237024 (0.0006) [2023-12-27 00:23:39,632][105620] Updated weights for policy 1, policy_version 1238147 (0.0008) [2023-12-27 00:23:39,681][105620] Updated weights for policy 1, policy_version 1238157 (0.0008) [2023-12-27 00:23:39,695][105692] Updated weights for policy 0, policy_version 1237034 (0.0005) [2023-12-27 00:23:39,733][105620] Updated weights for policy 1, policy_version 1238167 (0.0009) [2023-12-27 00:23:40,339][105692] Updated weights for policy 0, policy_version 1237044 (0.0008) [2023-12-27 00:23:40,400][105692] Updated weights for policy 0, policy_version 1237054 (0.0006) [2023-12-27 00:23:40,463][105692] Updated weights for policy 0, policy_version 1237064 (0.0009) [2023-12-27 00:23:40,519][105620] Updated weights for policy 1, policy_version 1238177 (0.0010) [2023-12-27 00:23:40,579][105620] Updated weights for policy 1, policy_version 1238187 (0.0009) [2023-12-27 00:23:40,633][105620] Updated weights for policy 1, policy_version 1238197 (0.0009) [2023-12-27 00:23:40,689][105620] Updated weights for policy 1, policy_version 1238207 (0.0010) [2023-12-27 00:23:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 633765888. Throughput: 0: 9777.7, 1: 9652.5. Samples: 633774484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:23:41,063][104569] Avg episode reward: [(0, '9087.890'), (1, '9077.518')] [2023-12-27 00:23:41,075][105692] Updated weights for policy 0, policy_version 1237074 (0.0008) [2023-12-27 00:23:41,137][105692] Updated weights for policy 0, policy_version 1237084 (0.0008) [2023-12-27 00:23:41,199][105692] Updated weights for policy 0, policy_version 1237094 (0.0006) [2023-12-27 00:23:41,266][105692] Updated weights for policy 0, policy_version 1237104 (0.0007) [2023-12-27 00:23:41,537][105620] Updated weights for policy 1, policy_version 1238217 (0.0006) [2023-12-27 00:23:41,604][105620] Updated weights for policy 1, policy_version 1238227 (0.0006) [2023-12-27 00:23:41,666][105620] Updated weights for policy 1, policy_version 1238237 (0.0008) [2023-12-27 00:23:42,003][105692] Updated weights for policy 0, policy_version 1237114 (0.0010) [2023-12-27 00:23:42,061][105692] Updated weights for policy 0, policy_version 1237124 (0.0008) [2023-12-27 00:23:42,119][105692] Updated weights for policy 0, policy_version 1237134 (0.0009) [2023-12-27 00:23:42,408][105620] Updated weights for policy 1, policy_version 1238247 (0.0008) [2023-12-27 00:23:42,478][105620] Updated weights for policy 1, policy_version 1238257 (0.0009) [2023-12-27 00:23:42,545][105620] Updated weights for policy 1, policy_version 1238267 (0.0009) [2023-12-27 00:23:42,862][105692] Updated weights for policy 0, policy_version 1237144 (0.0008) [2023-12-27 00:23:42,904][105692] Updated weights for policy 0, policy_version 1237154 (0.0007) [2023-12-27 00:23:42,959][105692] Updated weights for policy 0, policy_version 1237164 (0.0006) [2023-12-27 00:23:43,240][105620] Updated weights for policy 1, policy_version 1238277 (0.0010) [2023-12-27 00:23:43,288][105620] Updated weights for policy 1, policy_version 1238287 (0.0010) [2023-12-27 00:23:43,332][105620] Updated weights for policy 1, policy_version 1238297 (0.0010) [2023-12-27 00:23:43,615][105692] Updated weights for policy 0, policy_version 1237174 (0.0009) [2023-12-27 00:23:43,669][105692] Updated weights for policy 0, policy_version 1237184 (0.0010) [2023-12-27 00:23:43,727][105692] Updated weights for policy 0, policy_version 1237194 (0.0010) [2023-12-27 00:23:44,075][105620] Updated weights for policy 1, policy_version 1238307 (0.0010) [2023-12-27 00:23:44,126][105620] Updated weights for policy 1, policy_version 1238317 (0.0011) [2023-12-27 00:23:44,170][105620] Updated weights for policy 1, policy_version 1238327 (0.0010) [2023-12-27 00:23:44,475][105692] Updated weights for policy 0, policy_version 1237204 (0.0009) [2023-12-27 00:23:44,526][105692] Updated weights for policy 0, policy_version 1237214 (0.0008) [2023-12-27 00:23:44,586][105692] Updated weights for policy 0, policy_version 1237224 (0.0008) [2023-12-27 00:23:44,927][105620] Updated weights for policy 1, policy_version 1238337 (0.0010) [2023-12-27 00:23:44,981][105620] Updated weights for policy 1, policy_version 1238347 (0.0005) [2023-12-27 00:23:45,037][105620] Updated weights for policy 1, policy_version 1238357 (0.0009) [2023-12-27 00:23:45,093][105620] Updated weights for policy 1, policy_version 1238367 (0.0010) [2023-12-27 00:23:45,308][105692] Updated weights for policy 0, policy_version 1237234 (0.0007) [2023-12-27 00:23:45,373][105692] Updated weights for policy 0, policy_version 1237244 (0.0009) [2023-12-27 00:23:45,437][105692] Updated weights for policy 0, policy_version 1237254 (0.0008) [2023-12-27 00:23:45,493][105692] Updated weights for policy 0, policy_version 1237264 (0.0008) [2023-12-27 00:23:45,790][105620] Updated weights for policy 1, policy_version 1238377 (0.0011) [2023-12-27 00:23:45,839][105620] Updated weights for policy 1, policy_version 1238387 (0.0011) [2023-12-27 00:23:45,887][105620] Updated weights for policy 1, policy_version 1238398 (0.0010) [2023-12-27 00:23:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 633864192. Throughput: 0: 9807.5, 1: 9629.4. Samples: 633833116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:23:46,062][104569] Avg episode reward: [(0, '9088.004'), (1, '9078.128')] [2023-12-27 00:23:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001238400_317071360.pth... [2023-12-27 00:23:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001237264_316792832.pth... [2023-12-27 00:23:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001236112_316497920.pth [2023-12-27 00:23:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001237280_316784640.pth [2023-12-27 00:23:46,316][105692] Updated weights for policy 0, policy_version 1237274 (0.0008) [2023-12-27 00:23:46,361][105692] Updated weights for policy 0, policy_version 1237284 (0.0008) [2023-12-27 00:23:46,419][105692] Updated weights for policy 0, policy_version 1237294 (0.0010) [2023-12-27 00:23:46,555][105620] Updated weights for policy 1, policy_version 1238408 (0.0006) [2023-12-27 00:23:46,619][105620] Updated weights for policy 1, policy_version 1238418 (0.0006) [2023-12-27 00:23:46,680][105620] Updated weights for policy 1, policy_version 1238428 (0.0005) [2023-12-27 00:23:47,207][105620] Updated weights for policy 1, policy_version 1238438 (0.0007) [2023-12-27 00:23:47,257][105620] Updated weights for policy 1, policy_version 1238448 (0.0009) [2023-12-27 00:23:47,285][105692] Updated weights for policy 0, policy_version 1237304 (0.0006) [2023-12-27 00:23:47,310][105620] Updated weights for policy 1, policy_version 1238458 (0.0009) [2023-12-27 00:23:47,341][105692] Updated weights for policy 0, policy_version 1237314 (0.0006) [2023-12-27 00:23:47,399][105692] Updated weights for policy 0, policy_version 1237324 (0.0008) [2023-12-27 00:23:48,011][105620] Updated weights for policy 1, policy_version 1238468 (0.0008) [2023-12-27 00:23:48,065][105620] Updated weights for policy 1, policy_version 1238478 (0.0009) [2023-12-27 00:23:48,125][105620] Updated weights for policy 1, policy_version 1238488 (0.0009) [2023-12-27 00:23:48,171][105692] Updated weights for policy 0, policy_version 1237334 (0.0009) [2023-12-27 00:23:48,223][105692] Updated weights for policy 0, policy_version 1237344 (0.0009) [2023-12-27 00:23:48,270][105692] Updated weights for policy 0, policy_version 1237354 (0.0009) [2023-12-27 00:23:48,804][105620] Updated weights for policy 1, policy_version 1238498 (0.0007) [2023-12-27 00:23:48,868][105620] Updated weights for policy 1, policy_version 1238508 (0.0009) [2023-12-27 00:23:48,932][105620] Updated weights for policy 1, policy_version 1238518 (0.0009) [2023-12-27 00:23:48,995][105620] Updated weights for policy 1, policy_version 1238528 (0.0009) [2023-12-27 00:23:49,164][105692] Updated weights for policy 0, policy_version 1237364 (0.0009) [2023-12-27 00:23:49,221][105692] Updated weights for policy 0, policy_version 1237374 (0.0010) [2023-12-27 00:23:49,294][105692] Updated weights for policy 0, policy_version 1237384 (0.0010) [2023-12-27 00:23:49,687][105620] Updated weights for policy 1, policy_version 1238538 (0.0006) [2023-12-27 00:23:49,748][105620] Updated weights for policy 1, policy_version 1238548 (0.0005) [2023-12-27 00:23:49,801][105620] Updated weights for policy 1, policy_version 1238558 (0.0005) [2023-12-27 00:23:50,094][105692] Updated weights for policy 0, policy_version 1237394 (0.0009) [2023-12-27 00:23:50,150][105692] Updated weights for policy 0, policy_version 1237404 (0.0008) [2023-12-27 00:23:50,217][105692] Updated weights for policy 0, policy_version 1237414 (0.0009) [2023-12-27 00:23:50,269][105692] Updated weights for policy 0, policy_version 1237424 (0.0009) [2023-12-27 00:23:50,494][105620] Updated weights for policy 1, policy_version 1238568 (0.0010) [2023-12-27 00:23:50,554][105620] Updated weights for policy 1, policy_version 1238578 (0.0010) [2023-12-27 00:23:50,613][105620] Updated weights for policy 1, policy_version 1238588 (0.0008) [2023-12-27 00:23:50,926][105692] Updated weights for policy 0, policy_version 1237434 (0.0009) [2023-12-27 00:23:50,991][105692] Updated weights for policy 0, policy_version 1237444 (0.0010) [2023-12-27 00:23:51,051][105692] Updated weights for policy 0, policy_version 1237454 (0.0009) [2023-12-27 00:23:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 633954304. Throughput: 0: 9734.6, 1: 9627.0. Samples: 633948120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:23:51,062][104569] Avg episode reward: [(0, '9174.447'), (1, '9078.015')] [2023-12-27 00:23:51,396][105620] Updated weights for policy 1, policy_version 1238598 (0.0010) [2023-12-27 00:23:51,463][105620] Updated weights for policy 1, policy_version 1238608 (0.0008) [2023-12-27 00:23:51,530][105620] Updated weights for policy 1, policy_version 1238618 (0.0007) [2023-12-27 00:23:51,818][105692] Updated weights for policy 0, policy_version 1237464 (0.0008) [2023-12-27 00:23:51,880][105692] Updated weights for policy 0, policy_version 1237474 (0.0007) [2023-12-27 00:23:51,948][105692] Updated weights for policy 0, policy_version 1237484 (0.0006) [2023-12-27 00:23:52,291][105620] Updated weights for policy 1, policy_version 1238628 (0.0009) [2023-12-27 00:23:52,359][105620] Updated weights for policy 1, policy_version 1238638 (0.0008) [2023-12-27 00:23:52,417][105620] Updated weights for policy 1, policy_version 1238648 (0.0008) [2023-12-27 00:23:52,610][105692] Updated weights for policy 0, policy_version 1237494 (0.0008) [2023-12-27 00:23:52,674][105692] Updated weights for policy 0, policy_version 1237504 (0.0005) [2023-12-27 00:23:52,737][105692] Updated weights for policy 0, policy_version 1237514 (0.0005) [2023-12-27 00:23:53,167][105620] Updated weights for policy 1, policy_version 1238658 (0.0008) [2023-12-27 00:23:53,232][105620] Updated weights for policy 1, policy_version 1238668 (0.0008) [2023-12-27 00:23:53,297][105620] Updated weights for policy 1, policy_version 1238678 (0.0008) [2023-12-27 00:23:53,351][105692] Updated weights for policy 0, policy_version 1237524 (0.0007) [2023-12-27 00:23:53,357][105620] Updated weights for policy 1, policy_version 1238688 (0.0006) [2023-12-27 00:23:53,408][105692] Updated weights for policy 0, policy_version 1237534 (0.0007) [2023-12-27 00:23:53,462][105692] Updated weights for policy 0, policy_version 1237544 (0.0005) [2023-12-27 00:23:54,068][105692] Updated weights for policy 0, policy_version 1237554 (0.0005) [2023-12-27 00:23:54,117][105692] Updated weights for policy 0, policy_version 1237564 (0.0006) [2023-12-27 00:23:54,153][105620] Updated weights for policy 1, policy_version 1238698 (0.0008) [2023-12-27 00:23:54,167][105692] Updated weights for policy 0, policy_version 1237574 (0.0007) [2023-12-27 00:23:54,219][105620] Updated weights for policy 1, policy_version 1238708 (0.0005) [2023-12-27 00:23:54,220][105692] Updated weights for policy 0, policy_version 1237584 (0.0007) [2023-12-27 00:23:54,275][105620] Updated weights for policy 1, policy_version 1238718 (0.0005) [2023-12-27 00:23:54,816][105620] Updated weights for policy 1, policy_version 1238728 (0.0006) [2023-12-27 00:23:54,878][105620] Updated weights for policy 1, policy_version 1238738 (0.0011) [2023-12-27 00:23:54,882][105692] Updated weights for policy 0, policy_version 1237594 (0.0006) [2023-12-27 00:23:54,940][105620] Updated weights for policy 1, policy_version 1238748 (0.0010) [2023-12-27 00:23:54,941][105692] Updated weights for policy 0, policy_version 1237604 (0.0005) [2023-12-27 00:23:55,009][105692] Updated weights for policy 0, policy_version 1237614 (0.0008) [2023-12-27 00:23:55,516][105620] Updated weights for policy 1, policy_version 1238758 (0.0009) [2023-12-27 00:23:55,531][105692] Updated weights for policy 0, policy_version 1237624 (0.0006) [2023-12-27 00:23:55,576][105620] Updated weights for policy 1, policy_version 1238768 (0.0010) [2023-12-27 00:23:55,585][105692] Updated weights for policy 0, policy_version 1237634 (0.0005) [2023-12-27 00:23:55,633][105620] Updated weights for policy 1, policy_version 1238778 (0.0009) [2023-12-27 00:23:55,645][105692] Updated weights for policy 0, policy_version 1237644 (0.0006) [2023-12-27 00:23:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 634060800. Throughput: 0: 9918.0, 1: 9636.8. Samples: 634070332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:23:56,063][104569] Avg episode reward: [(0, '9085.352'), (1, '9260.362')] [2023-12-27 00:23:56,192][105620] Updated weights for policy 1, policy_version 1238788 (0.0006) [2023-12-27 00:23:56,223][105692] Updated weights for policy 0, policy_version 1237654 (0.0011) [2023-12-27 00:23:56,247][105620] Updated weights for policy 1, policy_version 1238798 (0.0005) [2023-12-27 00:23:56,279][105692] Updated weights for policy 0, policy_version 1237664 (0.0010) [2023-12-27 00:23:56,305][105620] Updated weights for policy 1, policy_version 1238808 (0.0009) [2023-12-27 00:23:56,336][105692] Updated weights for policy 0, policy_version 1237674 (0.0011) [2023-12-27 00:23:56,883][105620] Updated weights for policy 1, policy_version 1238818 (0.0009) [2023-12-27 00:23:56,930][105620] Updated weights for policy 1, policy_version 1238828 (0.0005) [2023-12-27 00:23:56,976][105620] Updated weights for policy 1, policy_version 1238838 (0.0005) [2023-12-27 00:23:57,021][105620] Updated weights for policy 1, policy_version 1238848 (0.0005) [2023-12-27 00:23:57,035][105692] Updated weights for policy 0, policy_version 1237684 (0.0010) [2023-12-27 00:23:57,079][105692] Updated weights for policy 0, policy_version 1237694 (0.0010) [2023-12-27 00:23:57,127][105692] Updated weights for policy 0, policy_version 1237704 (0.0010) [2023-12-27 00:23:57,604][105620] Updated weights for policy 1, policy_version 1238858 (0.0010) [2023-12-27 00:23:57,653][105620] Updated weights for policy 1, policy_version 1238868 (0.0006) [2023-12-27 00:23:57,714][105620] Updated weights for policy 1, policy_version 1238878 (0.0005) [2023-12-27 00:23:57,822][105692] Updated weights for policy 0, policy_version 1237714 (0.0010) [2023-12-27 00:23:57,876][105692] Updated weights for policy 0, policy_version 1237724 (0.0010) [2023-12-27 00:23:57,920][105692] Updated weights for policy 0, policy_version 1237734 (0.0010) [2023-12-27 00:23:57,971][105692] Updated weights for policy 0, policy_version 1237744 (0.0010) [2023-12-27 00:23:58,265][105620] Updated weights for policy 1, policy_version 1238888 (0.0009) [2023-12-27 00:23:58,327][105620] Updated weights for policy 1, policy_version 1238898 (0.0010) [2023-12-27 00:23:58,400][105620] Updated weights for policy 1, policy_version 1238908 (0.0009) [2023-12-27 00:23:58,798][105692] Updated weights for policy 0, policy_version 1237754 (0.0010) [2023-12-27 00:23:58,863][105692] Updated weights for policy 0, policy_version 1237764 (0.0009) [2023-12-27 00:23:58,930][105692] Updated weights for policy 0, policy_version 1237774 (0.0009) [2023-12-27 00:23:59,230][105620] Updated weights for policy 1, policy_version 1238918 (0.0009) [2023-12-27 00:23:59,304][105620] Updated weights for policy 1, policy_version 1238928 (0.0009) [2023-12-27 00:23:59,367][105620] Updated weights for policy 1, policy_version 1238938 (0.0009) [2023-12-27 00:23:59,629][105692] Updated weights for policy 0, policy_version 1237784 (0.0010) [2023-12-27 00:23:59,680][105692] Updated weights for policy 0, policy_version 1237794 (0.0010) [2023-12-27 00:23:59,732][105692] Updated weights for policy 0, policy_version 1237804 (0.0010) [2023-12-27 00:24:00,147][105620] Updated weights for policy 1, policy_version 1238948 (0.0008) [2023-12-27 00:24:00,213][105620] Updated weights for policy 1, policy_version 1238958 (0.0010) [2023-12-27 00:24:00,266][105620] Updated weights for policy 1, policy_version 1238968 (0.0010) [2023-12-27 00:24:00,354][105692] Updated weights for policy 0, policy_version 1237814 (0.0007) [2023-12-27 00:24:00,406][105692] Updated weights for policy 0, policy_version 1237824 (0.0005) [2023-12-27 00:24:00,471][105692] Updated weights for policy 0, policy_version 1237834 (0.0005) [2023-12-27 00:24:00,972][105692] Updated weights for policy 0, policy_version 1237844 (0.0005) [2023-12-27 00:24:00,991][105620] Updated weights for policy 1, policy_version 1238978 (0.0010) [2023-12-27 00:24:01,026][105692] Updated weights for policy 0, policy_version 1237854 (0.0006) [2023-12-27 00:24:01,054][105620] Updated weights for policy 1, policy_version 1238988 (0.0009) [2023-12-27 00:24:01,062][104569] Fps is (10 sec: 20479.0, 60 sec: 19524.1, 300 sec: 19521.9). Total num frames: 634159104. Throughput: 0: 9953.5, 1: 9768.5. Samples: 634134428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:24:01,063][104569] Avg episode reward: [(0, '8996.468'), (1, '8497.152')] [2023-12-27 00:24:01,084][105692] Updated weights for policy 0, policy_version 1237864 (0.0008) [2023-12-27 00:24:01,113][105620] Updated weights for policy 1, policy_version 1238998 (0.0008) [2023-12-27 00:24:01,132][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001237872_316948480.pth... [2023-12-27 00:24:01,136][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001236688_316645376.pth [2023-12-27 00:24:01,173][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001239008_317227008.pth... [2023-12-27 00:24:01,177][105620] Updated weights for policy 1, policy_version 1239008 (0.0008) [2023-12-27 00:24:01,178][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001237856_316932096.pth [2023-12-27 00:24:01,859][105620] Updated weights for policy 1, policy_version 1239018 (0.0005) [2023-12-27 00:24:01,865][105692] Updated weights for policy 0, policy_version 1237874 (0.0008) [2023-12-27 00:24:01,921][105620] Updated weights for policy 1, policy_version 1239028 (0.0006) [2023-12-27 00:24:01,923][105692] Updated weights for policy 0, policy_version 1237884 (0.0008) [2023-12-27 00:24:01,976][105692] Updated weights for policy 0, policy_version 1237894 (0.0007) [2023-12-27 00:24:01,978][105620] Updated weights for policy 1, policy_version 1239038 (0.0008) [2023-12-27 00:24:02,030][105692] Updated weights for policy 0, policy_version 1237904 (0.0006) [2023-12-27 00:24:02,596][105620] Updated weights for policy 1, policy_version 1239048 (0.0006) [2023-12-27 00:24:02,645][105620] Updated weights for policy 1, policy_version 1239058 (0.0008) [2023-12-27 00:24:02,698][105620] Updated weights for policy 1, policy_version 1239068 (0.0007) [2023-12-27 00:24:02,712][105692] Updated weights for policy 0, policy_version 1237914 (0.0008) [2023-12-27 00:24:02,771][105692] Updated weights for policy 0, policy_version 1237924 (0.0009) [2023-12-27 00:24:02,818][105692] Updated weights for policy 0, policy_version 1237934 (0.0009) [2023-12-27 00:24:03,436][105620] Updated weights for policy 1, policy_version 1239078 (0.0008) [2023-12-27 00:24:03,487][105620] Updated weights for policy 1, policy_version 1239088 (0.0009) [2023-12-27 00:24:03,511][105692] Updated weights for policy 0, policy_version 1237944 (0.0006) [2023-12-27 00:24:03,539][105620] Updated weights for policy 1, policy_version 1239098 (0.0009) [2023-12-27 00:24:03,566][105692] Updated weights for policy 0, policy_version 1237954 (0.0007) [2023-12-27 00:24:03,612][105692] Updated weights for policy 0, policy_version 1237964 (0.0008) [2023-12-27 00:24:04,320][105620] Updated weights for policy 1, policy_version 1239108 (0.0009) [2023-12-27 00:24:04,337][105692] Updated weights for policy 0, policy_version 1237974 (0.0008) [2023-12-27 00:24:04,373][105620] Updated weights for policy 1, policy_version 1239118 (0.0007) [2023-12-27 00:24:04,391][105692] Updated weights for policy 0, policy_version 1237984 (0.0007) [2023-12-27 00:24:04,446][105692] Updated weights for policy 0, policy_version 1237994 (0.0007) [2023-12-27 00:24:04,462][105620] Updated weights for policy 1, policy_version 1239128 (0.0008) [2023-12-27 00:24:05,156][105692] Updated weights for policy 0, policy_version 1238004 (0.0008) [2023-12-27 00:24:05,176][105620] Updated weights for policy 1, policy_version 1239138 (0.0009) [2023-12-27 00:24:05,218][105692] Updated weights for policy 0, policy_version 1238014 (0.0005) [2023-12-27 00:24:05,225][105620] Updated weights for policy 1, policy_version 1239148 (0.0009) [2023-12-27 00:24:05,274][105620] Updated weights for policy 1, policy_version 1239158 (0.0008) [2023-12-27 00:24:05,278][105692] Updated weights for policy 0, policy_version 1238024 (0.0006) [2023-12-27 00:24:05,323][105620] Updated weights for policy 1, policy_version 1239168 (0.0009) [2023-12-27 00:24:05,916][105692] Updated weights for policy 0, policy_version 1238034 (0.0006) [2023-12-27 00:24:05,974][105692] Updated weights for policy 0, policy_version 1238044 (0.0009) [2023-12-27 00:24:06,026][105692] Updated weights for policy 0, policy_version 1238054 (0.0007) [2023-12-27 00:24:06,040][105620] Updated weights for policy 1, policy_version 1239178 (0.0007) [2023-12-27 00:24:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 634257408. Throughput: 0: 9977.7, 1: 9794.4. Samples: 634252816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:24:06,062][104569] Avg episode reward: [(0, '8994.635'), (1, '8275.010')] [2023-12-27 00:24:06,078][105692] Updated weights for policy 0, policy_version 1238064 (0.0006) [2023-12-27 00:24:06,090][105620] Updated weights for policy 1, policy_version 1239188 (0.0006) [2023-12-27 00:24:06,161][105620] Updated weights for policy 1, policy_version 1239198 (0.0007) [2023-12-27 00:24:06,747][105692] Updated weights for policy 0, policy_version 1238074 (0.0006) [2023-12-27 00:24:06,812][105692] Updated weights for policy 0, policy_version 1238084 (0.0008) [2023-12-27 00:24:06,881][105692] Updated weights for policy 0, policy_version 1238094 (0.0007) [2023-12-27 00:24:06,945][105620] Updated weights for policy 1, policy_version 1239208 (0.0008) [2023-12-27 00:24:07,006][105620] Updated weights for policy 1, policy_version 1239218 (0.0009) [2023-12-27 00:24:07,064][105620] Updated weights for policy 1, policy_version 1239228 (0.0009) [2023-12-27 00:24:07,459][105692] Updated weights for policy 0, policy_version 1238104 (0.0007) [2023-12-27 00:24:07,516][105692] Updated weights for policy 0, policy_version 1238114 (0.0005) [2023-12-27 00:24:07,572][105692] Updated weights for policy 0, policy_version 1238124 (0.0006) [2023-12-27 00:24:07,876][105620] Updated weights for policy 1, policy_version 1239238 (0.0008) [2023-12-27 00:24:07,933][105620] Updated weights for policy 1, policy_version 1239248 (0.0009) [2023-12-27 00:24:07,995][105620] Updated weights for policy 1, policy_version 1239258 (0.0009) [2023-12-27 00:24:08,287][105692] Updated weights for policy 0, policy_version 1238134 (0.0009) [2023-12-27 00:24:08,343][105692] Updated weights for policy 0, policy_version 1238144 (0.0009) [2023-12-27 00:24:08,402][105692] Updated weights for policy 0, policy_version 1238154 (0.0009) [2023-12-27 00:24:08,762][105620] Updated weights for policy 1, policy_version 1239268 (0.0008) [2023-12-27 00:24:08,837][105620] Updated weights for policy 1, policy_version 1239278 (0.0010) [2023-12-27 00:24:08,901][105620] Updated weights for policy 1, policy_version 1239288 (0.0007) [2023-12-27 00:24:09,107][105692] Updated weights for policy 0, policy_version 1238164 (0.0010) [2023-12-27 00:24:09,167][105692] Updated weights for policy 0, policy_version 1238174 (0.0009) [2023-12-27 00:24:09,226][105692] Updated weights for policy 0, policy_version 1238184 (0.0009) [2023-12-27 00:24:09,581][105620] Updated weights for policy 1, policy_version 1239298 (0.0006) [2023-12-27 00:24:09,645][105620] Updated weights for policy 1, policy_version 1239308 (0.0006) [2023-12-27 00:24:09,703][105620] Updated weights for policy 1, policy_version 1239318 (0.0005) [2023-12-27 00:24:09,757][105620] Updated weights for policy 1, policy_version 1239328 (0.0009) [2023-12-27 00:24:09,968][105692] Updated weights for policy 0, policy_version 1238194 (0.0010) [2023-12-27 00:24:10,033][105692] Updated weights for policy 0, policy_version 1238204 (0.0006) [2023-12-27 00:24:10,100][105692] Updated weights for policy 0, policy_version 1238214 (0.0006) [2023-12-27 00:24:10,169][105692] Updated weights for policy 0, policy_version 1238224 (0.0006) [2023-12-27 00:24:10,436][105620] Updated weights for policy 1, policy_version 1239338 (0.0007) [2023-12-27 00:24:10,499][105620] Updated weights for policy 1, policy_version 1239348 (0.0009) [2023-12-27 00:24:10,555][105620] Updated weights for policy 1, policy_version 1239358 (0.0009) [2023-12-27 00:24:10,859][105692] Updated weights for policy 0, policy_version 1238234 (0.0005) [2023-12-27 00:24:10,913][105692] Updated weights for policy 0, policy_version 1238244 (0.0007) [2023-12-27 00:24:10,977][105692] Updated weights for policy 0, policy_version 1238254 (0.0006) [2023-12-27 00:24:11,062][104569] Fps is (10 sec: 20481.1, 60 sec: 19797.7, 300 sec: 19549.7). Total num frames: 634363904. Throughput: 0: 10014.7, 1: 9794.8. Samples: 634369696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:24:11,062][104569] Avg episode reward: [(0, '9086.664'), (1, '8684.847')] [2023-12-27 00:24:11,282][105620] Updated weights for policy 1, policy_version 1239368 (0.0010) [2023-12-27 00:24:11,354][105620] Updated weights for policy 1, policy_version 1239378 (0.0009) [2023-12-27 00:24:11,425][105620] Updated weights for policy 1, policy_version 1239388 (0.0009) [2023-12-27 00:24:11,749][105692] Updated weights for policy 0, policy_version 1238264 (0.0008) [2023-12-27 00:24:11,809][105692] Updated weights for policy 0, policy_version 1238274 (0.0009) [2023-12-27 00:24:11,868][105692] Updated weights for policy 0, policy_version 1238284 (0.0009) [2023-12-27 00:24:12,140][105620] Updated weights for policy 1, policy_version 1239398 (0.0008) [2023-12-27 00:24:12,197][105620] Updated weights for policy 1, policy_version 1239408 (0.0010) [2023-12-27 00:24:12,256][105620] Updated weights for policy 1, policy_version 1239418 (0.0011) [2023-12-27 00:24:12,666][105692] Updated weights for policy 0, policy_version 1238294 (0.0009) [2023-12-27 00:24:12,714][105692] Updated weights for policy 0, policy_version 1238304 (0.0008) [2023-12-27 00:24:12,768][105692] Updated weights for policy 0, policy_version 1238314 (0.0005) [2023-12-27 00:24:13,047][105620] Updated weights for policy 1, policy_version 1239428 (0.0010) [2023-12-27 00:24:13,100][105620] Updated weights for policy 1, policy_version 1239438 (0.0010) [2023-12-27 00:24:13,152][105620] Updated weights for policy 1, policy_version 1239448 (0.0010) [2023-12-27 00:24:13,387][105692] Updated weights for policy 0, policy_version 1238324 (0.0005) [2023-12-27 00:24:13,431][105692] Updated weights for policy 0, policy_version 1238334 (0.0005) [2023-12-27 00:24:13,481][105692] Updated weights for policy 0, policy_version 1238344 (0.0005) [2023-12-27 00:24:13,795][105620] Updated weights for policy 1, policy_version 1239458 (0.0007) [2023-12-27 00:24:13,849][105620] Updated weights for policy 1, policy_version 1239468 (0.0005) [2023-12-27 00:24:13,908][105620] Updated weights for policy 1, policy_version 1239478 (0.0005) [2023-12-27 00:24:13,957][105620] Updated weights for policy 1, policy_version 1239488 (0.0005) [2023-12-27 00:24:14,127][105692] Updated weights for policy 0, policy_version 1238354 (0.0006) [2023-12-27 00:24:14,179][105692] Updated weights for policy 0, policy_version 1238364 (0.0010) [2023-12-27 00:24:14,227][105692] Updated weights for policy 0, policy_version 1238374 (0.0010) [2023-12-27 00:24:14,290][105692] Updated weights for policy 0, policy_version 1238384 (0.0011) [2023-12-27 00:24:14,556][105620] Updated weights for policy 1, policy_version 1239498 (0.0005) [2023-12-27 00:24:14,611][105620] Updated weights for policy 1, policy_version 1239509 (0.0010) [2023-12-27 00:24:14,663][105620] Updated weights for policy 1, policy_version 1239520 (0.0010) [2023-12-27 00:24:14,896][105692] Updated weights for policy 0, policy_version 1238394 (0.0005) [2023-12-27 00:24:14,957][105692] Updated weights for policy 0, policy_version 1238404 (0.0006) [2023-12-27 00:24:15,024][105692] Updated weights for policy 0, policy_version 1238414 (0.0011) [2023-12-27 00:24:15,438][105620] Updated weights for policy 1, policy_version 1239530 (0.0008) [2023-12-27 00:24:15,509][105620] Updated weights for policy 1, policy_version 1239540 (0.0006) [2023-12-27 00:24:15,573][105620] Updated weights for policy 1, policy_version 1239550 (0.0008) [2023-12-27 00:24:15,734][105692] Updated weights for policy 0, policy_version 1238424 (0.0006) [2023-12-27 00:24:15,784][105692] Updated weights for policy 0, policy_version 1238434 (0.0005) [2023-12-27 00:24:15,844][105692] Updated weights for policy 0, policy_version 1238444 (0.0010) [2023-12-27 00:24:16,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 634462208. Throughput: 0: 9937.0, 1: 9764.2. Samples: 634428136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:24:16,062][104569] Avg episode reward: [(0, '8906.151'), (1, '9259.193')] [2023-12-27 00:24:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001238448_317095936.pth... [2023-12-27 00:24:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001239552_317366272.pth... [2023-12-27 00:24:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001238400_317071360.pth [2023-12-27 00:24:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001237264_316792832.pth [2023-12-27 00:24:16,340][105620] Updated weights for policy 1, policy_version 1239560 (0.0009) [2023-12-27 00:24:16,391][105620] Updated weights for policy 1, policy_version 1239570 (0.0009) [2023-12-27 00:24:16,391][105692] Updated weights for policy 0, policy_version 1238454 (0.0007) [2023-12-27 00:24:16,438][105692] Updated weights for policy 0, policy_version 1238464 (0.0005) [2023-12-27 00:24:16,446][105620] Updated weights for policy 1, policy_version 1239580 (0.0009) [2023-12-27 00:24:16,483][105692] Updated weights for policy 0, policy_version 1238474 (0.0005) [2023-12-27 00:24:17,025][105692] Updated weights for policy 0, policy_version 1238484 (0.0005) [2023-12-27 00:24:17,091][105692] Updated weights for policy 0, policy_version 1238494 (0.0005) [2023-12-27 00:24:17,151][105692] Updated weights for policy 0, policy_version 1238504 (0.0005) [2023-12-27 00:24:17,387][105620] Updated weights for policy 1, policy_version 1239590 (0.0009) [2023-12-27 00:24:17,442][105620] Updated weights for policy 1, policy_version 1239600 (0.0008) [2023-12-27 00:24:17,491][105620] Updated weights for policy 1, policy_version 1239610 (0.0008) [2023-12-27 00:24:17,763][105692] Updated weights for policy 0, policy_version 1238514 (0.0006) [2023-12-27 00:24:17,813][105692] Updated weights for policy 0, policy_version 1238524 (0.0005) [2023-12-27 00:24:17,869][105692] Updated weights for policy 0, policy_version 1238534 (0.0005) [2023-12-27 00:24:17,921][105692] Updated weights for policy 0, policy_version 1238544 (0.0007) [2023-12-27 00:24:18,275][105620] Updated weights for policy 1, policy_version 1239620 (0.0008) [2023-12-27 00:24:18,338][105620] Updated weights for policy 1, policy_version 1239630 (0.0006) [2023-12-27 00:24:18,398][105620] Updated weights for policy 1, policy_version 1239640 (0.0008) [2023-12-27 00:24:18,646][105692] Updated weights for policy 0, policy_version 1238554 (0.0010) [2023-12-27 00:24:18,701][105692] Updated weights for policy 0, policy_version 1238565 (0.0009) [2023-12-27 00:24:18,750][105692] Updated weights for policy 0, policy_version 1238575 (0.0009) [2023-12-27 00:24:19,081][105620] Updated weights for policy 1, policy_version 1239650 (0.0009) [2023-12-27 00:24:19,136][105620] Updated weights for policy 1, policy_version 1239660 (0.0008) [2023-12-27 00:24:19,182][105620] Updated weights for policy 1, policy_version 1239670 (0.0008) [2023-12-27 00:24:19,244][105620] Updated weights for policy 1, policy_version 1239680 (0.0008) [2023-12-27 00:24:19,574][105692] Updated weights for policy 0, policy_version 1238585 (0.0010) [2023-12-27 00:24:19,630][105692] Updated weights for policy 0, policy_version 1238595 (0.0009) [2023-12-27 00:24:19,690][105692] Updated weights for policy 0, policy_version 1238605 (0.0009) [2023-12-27 00:24:20,027][105620] Updated weights for policy 1, policy_version 1239690 (0.0009) [2023-12-27 00:24:20,089][105620] Updated weights for policy 1, policy_version 1239700 (0.0009) [2023-12-27 00:24:20,151][105620] Updated weights for policy 1, policy_version 1239710 (0.0009) [2023-12-27 00:24:20,524][105692] Updated weights for policy 0, policy_version 1238615 (0.0009) [2023-12-27 00:24:20,580][105692] Updated weights for policy 0, policy_version 1238625 (0.0009) [2023-12-27 00:24:20,648][105692] Updated weights for policy 0, policy_version 1238635 (0.0009) [2023-12-27 00:24:20,860][105620] Updated weights for policy 1, policy_version 1239720 (0.0009) [2023-12-27 00:24:20,909][105620] Updated weights for policy 1, policy_version 1239730 (0.0006) [2023-12-27 00:24:20,963][105620] Updated weights for policy 1, policy_version 1239740 (0.0006) [2023-12-27 00:24:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 634560512. Throughput: 0: 9991.8, 1: 9690.8. Samples: 634546836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:24:21,062][104569] Avg episode reward: [(0, '8725.660'), (1, '9078.205')] [2023-12-27 00:24:21,452][105692] Updated weights for policy 0, policy_version 1238645 (0.0011) [2023-12-27 00:24:21,510][105692] Updated weights for policy 0, policy_version 1238655 (0.0010) [2023-12-27 00:24:21,560][105692] Updated weights for policy 0, policy_version 1238665 (0.0011) [2023-12-27 00:24:21,668][105620] Updated weights for policy 1, policy_version 1239750 (0.0009) [2023-12-27 00:24:21,722][105620] Updated weights for policy 1, policy_version 1239760 (0.0011) [2023-12-27 00:24:21,788][105620] Updated weights for policy 1, policy_version 1239770 (0.0009) [2023-12-27 00:24:22,398][105692] Updated weights for policy 0, policy_version 1238675 (0.0009) [2023-12-27 00:24:22,467][105692] Updated weights for policy 0, policy_version 1238685 (0.0009) [2023-12-27 00:24:22,534][105692] Updated weights for policy 0, policy_version 1238695 (0.0008) [2023-12-27 00:24:22,571][105620] Updated weights for policy 1, policy_version 1239780 (0.0009) [2023-12-27 00:24:22,630][105620] Updated weights for policy 1, policy_version 1239790 (0.0008) [2023-12-27 00:24:22,688][105620] Updated weights for policy 1, policy_version 1239800 (0.0008) [2023-12-27 00:24:23,312][105692] Updated weights for policy 0, policy_version 1238705 (0.0008) [2023-12-27 00:24:23,359][105692] Updated weights for policy 0, policy_version 1238715 (0.0009) [2023-12-27 00:24:23,366][105620] Updated weights for policy 1, policy_version 1239810 (0.0006) [2023-12-27 00:24:23,414][105692] Updated weights for policy 0, policy_version 1238725 (0.0009) [2023-12-27 00:24:23,419][105620] Updated weights for policy 1, policy_version 1239820 (0.0008) [2023-12-27 00:24:23,466][105692] Updated weights for policy 0, policy_version 1238735 (0.0007) [2023-12-27 00:24:23,472][105620] Updated weights for policy 1, policy_version 1239830 (0.0008) [2023-12-27 00:24:23,521][105620] Updated weights for policy 1, policy_version 1239840 (0.0008) [2023-12-27 00:24:24,197][105692] Updated weights for policy 0, policy_version 1238745 (0.0006) [2023-12-27 00:24:24,231][105620] Updated weights for policy 1, policy_version 1239850 (0.0008) [2023-12-27 00:24:24,253][105692] Updated weights for policy 0, policy_version 1238755 (0.0005) [2023-12-27 00:24:24,294][105620] Updated weights for policy 1, policy_version 1239860 (0.0008) [2023-12-27 00:24:24,305][105692] Updated weights for policy 0, policy_version 1238765 (0.0005) [2023-12-27 00:24:24,351][105620] Updated weights for policy 1, policy_version 1239870 (0.0008) [2023-12-27 00:24:24,987][105620] Updated weights for policy 1, policy_version 1239880 (0.0007) [2023-12-27 00:24:25,040][105620] Updated weights for policy 1, policy_version 1239890 (0.0007) [2023-12-27 00:24:25,062][105692] Updated weights for policy 0, policy_version 1238775 (0.0005) [2023-12-27 00:24:25,094][105620] Updated weights for policy 1, policy_version 1239900 (0.0006) [2023-12-27 00:24:25,122][105692] Updated weights for policy 0, policy_version 1238785 (0.0008) [2023-12-27 00:24:25,175][105692] Updated weights for policy 0, policy_version 1238795 (0.0009) [2023-12-27 00:24:25,707][105620] Updated weights for policy 1, policy_version 1239910 (0.0007) [2023-12-27 00:24:25,766][105620] Updated weights for policy 1, policy_version 1239920 (0.0009) [2023-12-27 00:24:25,816][105620] Updated weights for policy 1, policy_version 1239930 (0.0009) [2023-12-27 00:24:25,943][105692] Updated weights for policy 0, policy_version 1238805 (0.0009) [2023-12-27 00:24:26,004][105692] Updated weights for policy 0, policy_version 1238815 (0.0009) [2023-12-27 00:24:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 634650624. Throughput: 0: 9894.7, 1: 9788.9. Samples: 634660244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:24:26,062][104569] Avg episode reward: [(0, '8904.382'), (1, '8985.727')] [2023-12-27 00:24:26,074][105692] Updated weights for policy 0, policy_version 1238825 (0.0010) [2023-12-27 00:24:26,497][105620] Updated weights for policy 1, policy_version 1239940 (0.0008) [2023-12-27 00:24:26,550][105620] Updated weights for policy 1, policy_version 1239950 (0.0005) [2023-12-27 00:24:26,615][105620] Updated weights for policy 1, policy_version 1239960 (0.0005) [2023-12-27 00:24:26,831][105692] Updated weights for policy 0, policy_version 1238835 (0.0007) [2023-12-27 00:24:26,861][105585] KL-divergence is very high: 122.0137 [2023-12-27 00:24:26,878][105585] KL-divergence is very high: 138.2977 [2023-12-27 00:24:26,882][105585] KL-divergence is very high: 216.3826 [2023-12-27 00:24:26,883][105692] Updated weights for policy 0, policy_version 1238845 (0.0006) [2023-12-27 00:24:26,893][105585] KL-divergence is very high: 208.0087 [2023-12-27 00:24:26,904][105585] KL-divergence is very high: 213.6492 [2023-12-27 00:24:26,923][105585] KL-divergence is very high: 192.1794 [2023-12-27 00:24:26,929][105585] KL-divergence is very high: 281.2547 [2023-12-27 00:24:26,940][105692] Updated weights for policy 0, policy_version 1238855 (0.0007) [2023-12-27 00:24:26,942][105585] KL-divergence is very high: 252.0631 [2023-12-27 00:24:26,954][105585] KL-divergence is very high: 244.9640 [2023-12-27 00:24:26,973][105585] KL-divergence is very high: 198.6054 [2023-12-27 00:24:26,980][105585] KL-divergence is very high: 279.8779 [2023-12-27 00:24:26,993][105585] KL-divergence is very high: 235.9171 [2023-12-27 00:24:27,308][105620] Updated weights for policy 1, policy_version 1239970 (0.0006) [2023-12-27 00:24:27,366][105620] Updated weights for policy 1, policy_version 1239980 (0.0010) [2023-12-27 00:24:27,417][105620] Updated weights for policy 1, policy_version 1239990 (0.0009) [2023-12-27 00:24:27,463][105620] Updated weights for policy 1, policy_version 1240000 (0.0008) [2023-12-27 00:24:27,489][105585] KL-divergence is very high: 209.2262 [2023-12-27 00:24:27,495][105692] Updated weights for policy 0, policy_version 1238865 (0.0009) [2023-12-27 00:24:27,518][105585] KL-divergence is very high: 171.2974 [2023-12-27 00:24:27,535][105585] KL-divergence is very high: 179.1287 [2023-12-27 00:24:27,554][105692] Updated weights for policy 0, policy_version 1238875 (0.0010) [2023-12-27 00:24:27,565][105585] KL-divergence is very high: 136.2592 [2023-12-27 00:24:27,585][105585] KL-divergence is very high: 148.3557 [2023-12-27 00:24:27,617][105585] KL-divergence is very high: 114.5531 [2023-12-27 00:24:27,618][105692] Updated weights for policy 0, policy_version 1238885 (0.0010) [2023-12-27 00:24:27,637][105585] KL-divergence is very high: 133.2158 [2023-12-27 00:24:27,669][105585] KL-divergence is very high: 105.6572 [2023-12-27 00:24:27,682][105692] Updated weights for policy 0, policy_version 1238895 (0.0009) [2023-12-27 00:24:28,111][105620] Updated weights for policy 1, policy_version 1240010 (0.0010) [2023-12-27 00:24:28,162][105620] Updated weights for policy 1, policy_version 1240020 (0.0010) [2023-12-27 00:24:28,209][105620] Updated weights for policy 1, policy_version 1240030 (0.0010) [2023-12-27 00:24:28,415][105692] Updated weights for policy 0, policy_version 1238905 (0.0008) [2023-12-27 00:24:28,459][105692] Updated weights for policy 0, policy_version 1238915 (0.0008) [2023-12-27 00:24:28,503][105692] Updated weights for policy 0, policy_version 1238925 (0.0007) [2023-12-27 00:24:29,014][105620] Updated weights for policy 1, policy_version 1240040 (0.0010) [2023-12-27 00:24:29,076][105620] Updated weights for policy 1, policy_version 1240050 (0.0009) [2023-12-27 00:24:29,135][105620] Updated weights for policy 1, policy_version 1240060 (0.0008) [2023-12-27 00:24:29,293][105692] Updated weights for policy 0, policy_version 1238935 (0.0009) [2023-12-27 00:24:29,353][105692] Updated weights for policy 0, policy_version 1238945 (0.0010) [2023-12-27 00:24:29,412][105692] Updated weights for policy 0, policy_version 1238955 (0.0009) [2023-12-27 00:24:29,853][105620] Updated weights for policy 1, policy_version 1240070 (0.0009) [2023-12-27 00:24:29,902][105620] Updated weights for policy 1, policy_version 1240080 (0.0008) [2023-12-27 00:24:29,962][105620] Updated weights for policy 1, policy_version 1240090 (0.0009) [2023-12-27 00:24:30,145][105692] Updated weights for policy 0, policy_version 1238965 (0.0007) [2023-12-27 00:24:30,197][105692] Updated weights for policy 0, policy_version 1238975 (0.0005) [2023-12-27 00:24:30,251][105692] Updated weights for policy 0, policy_version 1238985 (0.0007) [2023-12-27 00:24:30,753][105620] Updated weights for policy 1, policy_version 1240100 (0.0009) [2023-12-27 00:24:30,799][105620] Updated weights for policy 1, policy_version 1240110 (0.0008) [2023-12-27 00:24:30,862][105620] Updated weights for policy 1, policy_version 1240120 (0.0008) [2023-12-27 00:24:30,959][105692] Updated weights for policy 0, policy_version 1238995 (0.0009) [2023-12-27 00:24:31,005][105692] Updated weights for policy 0, policy_version 1239005 (0.0008) [2023-12-27 00:24:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 634748928. Throughput: 0: 9881.4, 1: 9841.9. Samples: 634720660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:24:31,063][104569] Avg episode reward: [(0, '8993.775'), (1, '9258.169')] [2023-12-27 00:24:31,063][105692] Updated weights for policy 0, policy_version 1239015 (0.0009) [2023-12-27 00:24:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001240128_317513728.pth... [2023-12-27 00:24:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001239008_317227008.pth [2023-12-27 00:24:31,116][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001239024_317243392.pth... [2023-12-27 00:24:31,121][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001237872_316948480.pth [2023-12-27 00:24:31,536][105620] Updated weights for policy 1, policy_version 1240130 (0.0008) [2023-12-27 00:24:31,593][105620] Updated weights for policy 1, policy_version 1240140 (0.0005) [2023-12-27 00:24:31,656][105620] Updated weights for policy 1, policy_version 1240150 (0.0008) [2023-12-27 00:24:31,717][105620] Updated weights for policy 1, policy_version 1240160 (0.0009) [2023-12-27 00:24:31,900][105692] Updated weights for policy 0, policy_version 1239025 (0.0008) [2023-12-27 00:24:31,957][105692] Updated weights for policy 0, policy_version 1239035 (0.0005) [2023-12-27 00:24:32,012][105692] Updated weights for policy 0, policy_version 1239045 (0.0005) [2023-12-27 00:24:32,073][105692] Updated weights for policy 0, policy_version 1239055 (0.0005) [2023-12-27 00:24:32,466][105620] Updated weights for policy 1, policy_version 1240170 (0.0007) [2023-12-27 00:24:32,519][105620] Updated weights for policy 1, policy_version 1240180 (0.0008) [2023-12-27 00:24:32,580][105620] Updated weights for policy 1, policy_version 1240190 (0.0009) [2023-12-27 00:24:32,718][105692] Updated weights for policy 0, policy_version 1239065 (0.0009) [2023-12-27 00:24:32,770][105692] Updated weights for policy 0, policy_version 1239075 (0.0009) [2023-12-27 00:24:32,828][105692] Updated weights for policy 0, policy_version 1239085 (0.0009) [2023-12-27 00:24:33,308][105620] Updated weights for policy 1, policy_version 1240200 (0.0009) [2023-12-27 00:24:33,372][105620] Updated weights for policy 1, policy_version 1240210 (0.0007) [2023-12-27 00:24:33,429][105620] Updated weights for policy 1, policy_version 1240220 (0.0005) [2023-12-27 00:24:33,632][105692] Updated weights for policy 0, policy_version 1239096 (0.0010) [2023-12-27 00:24:33,684][105692] Updated weights for policy 0, policy_version 1239106 (0.0010) [2023-12-27 00:24:33,731][105692] Updated weights for policy 0, policy_version 1239116 (0.0009) [2023-12-27 00:24:34,019][105620] Updated weights for policy 1, policy_version 1240230 (0.0007) [2023-12-27 00:24:34,093][105620] Updated weights for policy 1, policy_version 1240240 (0.0010) [2023-12-27 00:24:34,161][105620] Updated weights for policy 1, policy_version 1240250 (0.0008) [2023-12-27 00:24:34,431][105692] Updated weights for policy 0, policy_version 1239126 (0.0008) [2023-12-27 00:24:34,501][105692] Updated weights for policy 0, policy_version 1239136 (0.0007) [2023-12-27 00:24:34,571][105692] Updated weights for policy 0, policy_version 1239146 (0.0008) [2023-12-27 00:24:34,895][105620] Updated weights for policy 1, policy_version 1240260 (0.0010) [2023-12-27 00:24:34,947][105620] Updated weights for policy 1, policy_version 1240270 (0.0009) [2023-12-27 00:24:34,998][105620] Updated weights for policy 1, policy_version 1240280 (0.0009) [2023-12-27 00:24:35,220][105692] Updated weights for policy 0, policy_version 1239156 (0.0008) [2023-12-27 00:24:35,271][105692] Updated weights for policy 0, policy_version 1239166 (0.0010) [2023-12-27 00:24:35,318][105692] Updated weights for policy 0, policy_version 1239176 (0.0010) [2023-12-27 00:24:35,702][105620] Updated weights for policy 1, policy_version 1240290 (0.0008) [2023-12-27 00:24:35,753][105620] Updated weights for policy 1, policy_version 1240300 (0.0005) [2023-12-27 00:24:35,806][105620] Updated weights for policy 1, policy_version 1240310 (0.0005) [2023-12-27 00:24:35,858][105620] Updated weights for policy 1, policy_version 1240320 (0.0005) [2023-12-27 00:24:36,026][105692] Updated weights for policy 0, policy_version 1239186 (0.0009) [2023-12-27 00:24:36,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 634847232. Throughput: 0: 9969.3, 1: 9768.1. Samples: 634836308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:24:36,063][104569] Avg episode reward: [(0, '8813.754'), (1, '9350.289')] [2023-12-27 00:24:36,092][105692] Updated weights for policy 0, policy_version 1239196 (0.0006) [2023-12-27 00:24:36,151][105692] Updated weights for policy 0, policy_version 1239206 (0.0009) [2023-12-27 00:24:36,203][105692] Updated weights for policy 0, policy_version 1239216 (0.0009) [2023-12-27 00:24:36,496][105620] Updated weights for policy 1, policy_version 1240330 (0.0007) [2023-12-27 00:24:36,556][105620] Updated weights for policy 1, policy_version 1240340 (0.0010) [2023-12-27 00:24:36,616][105620] Updated weights for policy 1, policy_version 1240350 (0.0005) [2023-12-27 00:24:36,999][105692] Updated weights for policy 0, policy_version 1239226 (0.0009) [2023-12-27 00:24:37,050][105692] Updated weights for policy 0, policy_version 1239236 (0.0008) [2023-12-27 00:24:37,099][105692] Updated weights for policy 0, policy_version 1239246 (0.0008) [2023-12-27 00:24:37,280][105620] Updated weights for policy 1, policy_version 1240360 (0.0010) [2023-12-27 00:24:37,332][105620] Updated weights for policy 1, policy_version 1240370 (0.0011) [2023-12-27 00:24:37,393][105620] Updated weights for policy 1, policy_version 1240380 (0.0011) [2023-12-27 00:24:37,890][105692] Updated weights for policy 0, policy_version 1239256 (0.0009) [2023-12-27 00:24:37,941][105692] Updated weights for policy 0, policy_version 1239266 (0.0010) [2023-12-27 00:24:37,977][105620] Updated weights for policy 1, policy_version 1240390 (0.0008) [2023-12-27 00:24:37,998][105692] Updated weights for policy 0, policy_version 1239276 (0.0007) [2023-12-27 00:24:38,035][105620] Updated weights for policy 1, policy_version 1240400 (0.0007) [2023-12-27 00:24:38,093][105620] Updated weights for policy 1, policy_version 1240410 (0.0010) [2023-12-27 00:24:38,677][105692] Updated weights for policy 0, policy_version 1239286 (0.0005) [2023-12-27 00:24:38,736][105692] Updated weights for policy 0, policy_version 1239296 (0.0005) [2023-12-27 00:24:38,785][105692] Updated weights for policy 0, policy_version 1239306 (0.0008) [2023-12-27 00:24:38,860][105620] Updated weights for policy 1, policy_version 1240420 (0.0010) [2023-12-27 00:24:38,926][105620] Updated weights for policy 1, policy_version 1240430 (0.0010) [2023-12-27 00:24:38,989][105620] Updated weights for policy 1, policy_version 1240440 (0.0011) [2023-12-27 00:24:39,548][105692] Updated weights for policy 0, policy_version 1239316 (0.0008) [2023-12-27 00:24:39,612][105692] Updated weights for policy 0, policy_version 1239326 (0.0009) [2023-12-27 00:24:39,669][105692] Updated weights for policy 0, policy_version 1239336 (0.0008) [2023-12-27 00:24:39,709][105620] Updated weights for policy 1, policy_version 1240450 (0.0009) [2023-12-27 00:24:39,765][105620] Updated weights for policy 1, policy_version 1240460 (0.0007) [2023-12-27 00:24:39,829][105620] Updated weights for policy 1, policy_version 1240470 (0.0006) [2023-12-27 00:24:39,897][105620] Updated weights for policy 1, policy_version 1240480 (0.0008) [2023-12-27 00:24:40,514][105692] Updated weights for policy 0, policy_version 1239346 (0.0009) [2023-12-27 00:24:40,566][105692] Updated weights for policy 0, policy_version 1239356 (0.0008) [2023-12-27 00:24:40,569][105620] Updated weights for policy 1, policy_version 1240490 (0.0008) [2023-12-27 00:24:40,624][105692] Updated weights for policy 0, policy_version 1239366 (0.0007) [2023-12-27 00:24:40,635][105620] Updated weights for policy 1, policy_version 1240500 (0.0007) [2023-12-27 00:24:40,683][105692] Updated weights for policy 0, policy_version 1239376 (0.0007) [2023-12-27 00:24:40,692][105620] Updated weights for policy 1, policy_version 1240510 (0.0006) [2023-12-27 00:24:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 634945536. Throughput: 0: 9822.4, 1: 9795.8. Samples: 634953148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:24:41,062][104569] Avg episode reward: [(0, '8639.383'), (1, '9259.450')] [2023-12-27 00:24:41,390][105620] Updated weights for policy 1, policy_version 1240520 (0.0009) [2023-12-27 00:24:41,458][105620] Updated weights for policy 1, policy_version 1240530 (0.0006) [2023-12-27 00:24:41,522][105620] Updated weights for policy 1, policy_version 1240540 (0.0008) [2023-12-27 00:24:41,528][105692] Updated weights for policy 0, policy_version 1239386 (0.0008) [2023-12-27 00:24:41,589][105692] Updated weights for policy 0, policy_version 1239396 (0.0009) [2023-12-27 00:24:41,660][105692] Updated weights for policy 0, policy_version 1239406 (0.0008) [2023-12-27 00:24:42,292][105692] Updated weights for policy 0, policy_version 1239416 (0.0008) [2023-12-27 00:24:42,336][105620] Updated weights for policy 1, policy_version 1240550 (0.0007) [2023-12-27 00:24:42,354][105692] Updated weights for policy 0, policy_version 1239426 (0.0008) [2023-12-27 00:24:42,401][105620] Updated weights for policy 1, policy_version 1240560 (0.0009) [2023-12-27 00:24:42,424][105692] Updated weights for policy 0, policy_version 1239436 (0.0007) [2023-12-27 00:24:42,460][105620] Updated weights for policy 1, policy_version 1240570 (0.0009) [2023-12-27 00:24:43,004][105692] Updated weights for policy 0, policy_version 1239446 (0.0006) [2023-12-27 00:24:43,055][105692] Updated weights for policy 0, policy_version 1239456 (0.0005) [2023-12-27 00:24:43,109][105692] Updated weights for policy 0, policy_version 1239466 (0.0006) [2023-12-27 00:24:43,343][105620] Updated weights for policy 1, policy_version 1240580 (0.0010) [2023-12-27 00:24:43,410][105620] Updated weights for policy 1, policy_version 1240590 (0.0009) [2023-12-27 00:24:43,476][105620] Updated weights for policy 1, policy_version 1240600 (0.0010) [2023-12-27 00:24:43,643][105692] Updated weights for policy 0, policy_version 1239476 (0.0006) [2023-12-27 00:24:43,690][105692] Updated weights for policy 0, policy_version 1239486 (0.0006) [2023-12-27 00:24:43,751][105692] Updated weights for policy 0, policy_version 1239496 (0.0007) [2023-12-27 00:24:44,252][105620] Updated weights for policy 1, policy_version 1240610 (0.0009) [2023-12-27 00:24:44,302][105620] Updated weights for policy 1, policy_version 1240620 (0.0009) [2023-12-27 00:24:44,357][105620] Updated weights for policy 1, policy_version 1240630 (0.0009) [2023-12-27 00:24:44,403][105620] Updated weights for policy 1, policy_version 1240640 (0.0009) [2023-12-27 00:24:44,451][105692] Updated weights for policy 0, policy_version 1239506 (0.0006) [2023-12-27 00:24:44,512][105692] Updated weights for policy 0, policy_version 1239516 (0.0009) [2023-12-27 00:24:44,568][105692] Updated weights for policy 0, policy_version 1239526 (0.0008) [2023-12-27 00:24:44,637][105692] Updated weights for policy 0, policy_version 1239536 (0.0005) [2023-12-27 00:24:45,152][105620] Updated weights for policy 1, policy_version 1240650 (0.0009) [2023-12-27 00:24:45,212][105620] Updated weights for policy 1, policy_version 1240660 (0.0008) [2023-12-27 00:24:45,274][105620] Updated weights for policy 1, policy_version 1240670 (0.0007) [2023-12-27 00:24:45,293][105692] Updated weights for policy 0, policy_version 1239546 (0.0010) [2023-12-27 00:24:45,342][105692] Updated weights for policy 0, policy_version 1239556 (0.0011) [2023-12-27 00:24:45,391][105692] Updated weights for policy 0, policy_version 1239566 (0.0010) [2023-12-27 00:24:46,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 635035648. Throughput: 0: 9846.1, 1: 9617.3. Samples: 635010272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:24:46,062][104569] Avg episode reward: [(0, '8819.941'), (1, '9259.761')] [2023-12-27 00:24:46,078][105620] Updated weights for policy 1, policy_version 1240680 (0.0008) [2023-12-27 00:24:46,114][105692] Updated weights for policy 0, policy_version 1239576 (0.0009) [2023-12-27 00:24:46,133][105620] Updated weights for policy 1, policy_version 1240690 (0.0007) [2023-12-27 00:24:46,171][105692] Updated weights for policy 0, policy_version 1239586 (0.0006) [2023-12-27 00:24:46,184][105620] Updated weights for policy 1, policy_version 1240700 (0.0008) [2023-12-27 00:24:46,202][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001240704_317661184.pth... [2023-12-27 00:24:46,206][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001239552_317366272.pth [2023-12-27 00:24:46,221][105692] Updated weights for policy 0, policy_version 1239596 (0.0005) [2023-12-27 00:24:46,240][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001239600_317390848.pth... [2023-12-27 00:24:46,246][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001238448_317095936.pth [2023-12-27 00:24:46,875][105692] Updated weights for policy 0, policy_version 1239606 (0.0009) [2023-12-27 00:24:46,922][105692] Updated weights for policy 0, policy_version 1239616 (0.0009) [2023-12-27 00:24:46,963][105620] Updated weights for policy 1, policy_version 1240711 (0.0007) [2023-12-27 00:24:46,969][105692] Updated weights for policy 0, policy_version 1239626 (0.0007) [2023-12-27 00:24:47,013][105620] Updated weights for policy 1, policy_version 1240721 (0.0007) [2023-12-27 00:24:47,059][105620] Updated weights for policy 1, policy_version 1240731 (0.0008) [2023-12-27 00:24:47,731][105620] Updated weights for policy 1, policy_version 1240741 (0.0007) [2023-12-27 00:24:47,781][105692] Updated weights for policy 0, policy_version 1239636 (0.0006) [2023-12-27 00:24:47,796][105620] Updated weights for policy 1, policy_version 1240751 (0.0006) [2023-12-27 00:24:47,844][105692] Updated weights for policy 0, policy_version 1239646 (0.0009) [2023-12-27 00:24:47,855][105620] Updated weights for policy 1, policy_version 1240761 (0.0005) [2023-12-27 00:24:47,908][105692] Updated weights for policy 0, policy_version 1239656 (0.0006) [2023-12-27 00:24:48,459][105620] Updated weights for policy 1, policy_version 1240771 (0.0007) [2023-12-27 00:24:48,514][105620] Updated weights for policy 1, policy_version 1240781 (0.0008) [2023-12-27 00:24:48,547][105692] Updated weights for policy 0, policy_version 1239666 (0.0008) [2023-12-27 00:24:48,565][105620] Updated weights for policy 1, policy_version 1240791 (0.0007) [2023-12-27 00:24:48,610][105692] Updated weights for policy 0, policy_version 1239676 (0.0009) [2023-12-27 00:24:48,671][105692] Updated weights for policy 0, policy_version 1239686 (0.0010) [2023-12-27 00:24:48,738][105692] Updated weights for policy 0, policy_version 1239696 (0.0010) [2023-12-27 00:24:49,183][105620] Updated weights for policy 1, policy_version 1240801 (0.0006) [2023-12-27 00:24:49,250][105620] Updated weights for policy 1, policy_version 1240811 (0.0009) [2023-12-27 00:24:49,312][105620] Updated weights for policy 1, policy_version 1240821 (0.0009) [2023-12-27 00:24:49,383][105620] Updated weights for policy 1, policy_version 1240831 (0.0010) [2023-12-27 00:24:49,534][105692] Updated weights for policy 0, policy_version 1239706 (0.0008) [2023-12-27 00:24:49,591][105692] Updated weights for policy 0, policy_version 1239716 (0.0008) [2023-12-27 00:24:49,656][105692] Updated weights for policy 0, policy_version 1239726 (0.0009) [2023-12-27 00:24:50,132][105620] Updated weights for policy 1, policy_version 1240841 (0.0008) [2023-12-27 00:24:50,191][105620] Updated weights for policy 1, policy_version 1240851 (0.0009) [2023-12-27 00:24:50,253][105620] Updated weights for policy 1, policy_version 1240861 (0.0009) [2023-12-27 00:24:50,412][105692] Updated weights for policy 0, policy_version 1239736 (0.0010) [2023-12-27 00:24:50,466][105692] Updated weights for policy 0, policy_version 1239746 (0.0010) [2023-12-27 00:24:50,528][105692] Updated weights for policy 0, policy_version 1239756 (0.0010) [2023-12-27 00:24:50,946][105620] Updated weights for policy 1, policy_version 1240871 (0.0009) [2023-12-27 00:24:51,009][105620] Updated weights for policy 1, policy_version 1240881 (0.0009) [2023-12-27 00:24:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 635133952. Throughput: 0: 9798.3, 1: 9663.1. Samples: 635128580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:24:51,063][104569] Avg episode reward: [(0, '8995.523'), (1, '9351.469')] [2023-12-27 00:24:51,075][105620] Updated weights for policy 1, policy_version 1240891 (0.0010) [2023-12-27 00:24:51,313][105692] Updated weights for policy 0, policy_version 1239766 (0.0009) [2023-12-27 00:24:51,377][105692] Updated weights for policy 0, policy_version 1239776 (0.0009) [2023-12-27 00:24:51,432][105692] Updated weights for policy 0, policy_version 1239786 (0.0010) [2023-12-27 00:24:51,801][105620] Updated weights for policy 1, policy_version 1240902 (0.0009) [2023-12-27 00:24:51,863][105620] Updated weights for policy 1, policy_version 1240912 (0.0008) [2023-12-27 00:24:51,921][105620] Updated weights for policy 1, policy_version 1240922 (0.0008) [2023-12-27 00:24:52,270][105692] Updated weights for policy 0, policy_version 1239796 (0.0009) [2023-12-27 00:24:52,331][105692] Updated weights for policy 0, policy_version 1239806 (0.0008) [2023-12-27 00:24:52,397][105692] Updated weights for policy 0, policy_version 1239816 (0.0008) [2023-12-27 00:24:52,627][105620] Updated weights for policy 1, policy_version 1240932 (0.0008) [2023-12-27 00:24:52,679][105620] Updated weights for policy 1, policy_version 1240942 (0.0009) [2023-12-27 00:24:52,739][105620] Updated weights for policy 1, policy_version 1240952 (0.0009) [2023-12-27 00:24:53,191][105692] Updated weights for policy 0, policy_version 1239826 (0.0009) [2023-12-27 00:24:53,245][105692] Updated weights for policy 0, policy_version 1239836 (0.0009) [2023-12-27 00:24:53,292][105692] Updated weights for policy 0, policy_version 1239846 (0.0009) [2023-12-27 00:24:53,346][105692] Updated weights for policy 0, policy_version 1239856 (0.0009) [2023-12-27 00:24:53,444][105620] Updated weights for policy 1, policy_version 1240962 (0.0009) [2023-12-27 00:24:53,494][105620] Updated weights for policy 1, policy_version 1240972 (0.0006) [2023-12-27 00:24:53,551][105620] Updated weights for policy 1, policy_version 1240982 (0.0006) [2023-12-27 00:24:53,606][105620] Updated weights for policy 1, policy_version 1240992 (0.0008) [2023-12-27 00:24:54,071][105692] Updated weights for policy 0, policy_version 1239866 (0.0007) [2023-12-27 00:24:54,123][105692] Updated weights for policy 0, policy_version 1239876 (0.0005) [2023-12-27 00:24:54,177][105692] Updated weights for policy 0, policy_version 1239886 (0.0005) [2023-12-27 00:24:54,368][105620] Updated weights for policy 1, policy_version 1241002 (0.0010) [2023-12-27 00:24:54,427][105620] Updated weights for policy 1, policy_version 1241012 (0.0011) [2023-12-27 00:24:54,483][105620] Updated weights for policy 1, policy_version 1241022 (0.0011) [2023-12-27 00:24:54,735][105692] Updated weights for policy 0, policy_version 1239896 (0.0005) [2023-12-27 00:24:54,787][105692] Updated weights for policy 0, policy_version 1239906 (0.0005) [2023-12-27 00:24:54,845][105692] Updated weights for policy 0, policy_version 1239916 (0.0005) [2023-12-27 00:24:55,225][105620] Updated weights for policy 1, policy_version 1241032 (0.0006) [2023-12-27 00:24:55,271][105620] Updated weights for policy 1, policy_version 1241042 (0.0005) [2023-12-27 00:24:55,317][105620] Updated weights for policy 1, policy_version 1241052 (0.0005) [2023-12-27 00:24:55,367][105692] Updated weights for policy 0, policy_version 1239926 (0.0005) [2023-12-27 00:24:55,425][105692] Updated weights for policy 0, policy_version 1239936 (0.0010) [2023-12-27 00:24:55,491][105692] Updated weights for policy 0, policy_version 1239946 (0.0010) [2023-12-27 00:24:55,936][105620] Updated weights for policy 1, policy_version 1241062 (0.0008) [2023-12-27 00:24:55,984][105620] Updated weights for policy 1, policy_version 1241072 (0.0010) [2023-12-27 00:24:56,028][105620] Updated weights for policy 1, policy_version 1241082 (0.0010) [2023-12-27 00:24:56,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 635240448. Throughput: 0: 9753.5, 1: 9724.6. Samples: 635246212. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:24:56,062][104569] Avg episode reward: [(0, '8998.644'), (1, '9260.026')] [2023-12-27 00:24:56,214][105692] Updated weights for policy 0, policy_version 1239956 (0.0010) [2023-12-27 00:24:56,272][105692] Updated weights for policy 0, policy_version 1239966 (0.0010) [2023-12-27 00:24:56,330][105692] Updated weights for policy 0, policy_version 1239976 (0.0010) [2023-12-27 00:24:56,798][105620] Updated weights for policy 1, policy_version 1241092 (0.0010) [2023-12-27 00:24:56,852][105620] Updated weights for policy 1, policy_version 1241102 (0.0010) [2023-12-27 00:24:56,910][105620] Updated weights for policy 1, policy_version 1241112 (0.0010) [2023-12-27 00:24:57,041][105692] Updated weights for policy 0, policy_version 1239986 (0.0010) [2023-12-27 00:24:57,089][105692] Updated weights for policy 0, policy_version 1239996 (0.0010) [2023-12-27 00:24:57,143][105692] Updated weights for policy 0, policy_version 1240006 (0.0010) [2023-12-27 00:24:57,197][105692] Updated weights for policy 0, policy_version 1240016 (0.0010) [2023-12-27 00:24:57,619][105620] Updated weights for policy 1, policy_version 1241122 (0.0010) [2023-12-27 00:24:57,683][105620] Updated weights for policy 1, policy_version 1241132 (0.0010) [2023-12-27 00:24:57,745][105620] Updated weights for policy 1, policy_version 1241142 (0.0011) [2023-12-27 00:24:57,803][105620] Updated weights for policy 1, policy_version 1241152 (0.0010) [2023-12-27 00:24:57,807][105692] Updated weights for policy 0, policy_version 1240026 (0.0005) [2023-12-27 00:24:57,866][105692] Updated weights for policy 0, policy_version 1240036 (0.0005) [2023-12-27 00:24:57,924][105692] Updated weights for policy 0, policy_version 1240046 (0.0005) [2023-12-27 00:24:58,470][105620] Updated weights for policy 1, policy_version 1241162 (0.0011) [2023-12-27 00:24:58,533][105620] Updated weights for policy 1, policy_version 1241172 (0.0011) [2023-12-27 00:24:58,600][105620] Updated weights for policy 1, policy_version 1241182 (0.0009) [2023-12-27 00:24:58,617][105692] Updated weights for policy 0, policy_version 1240056 (0.0010) [2023-12-27 00:24:58,678][105692] Updated weights for policy 0, policy_version 1240066 (0.0011) [2023-12-27 00:24:58,736][105692] Updated weights for policy 0, policy_version 1240076 (0.0011) [2023-12-27 00:24:59,391][105620] Updated weights for policy 1, policy_version 1241192 (0.0009) [2023-12-27 00:24:59,446][105620] Updated weights for policy 1, policy_version 1241202 (0.0008) [2023-12-27 00:24:59,478][105692] Updated weights for policy 0, policy_version 1240086 (0.0009) [2023-12-27 00:24:59,501][105620] Updated weights for policy 1, policy_version 1241212 (0.0008) [2023-12-27 00:24:59,537][105692] Updated weights for policy 0, policy_version 1240096 (0.0008) [2023-12-27 00:24:59,604][105692] Updated weights for policy 0, policy_version 1240106 (0.0008) [2023-12-27 00:25:00,275][105620] Updated weights for policy 1, policy_version 1241222 (0.0010) [2023-12-27 00:25:00,330][105620] Updated weights for policy 1, policy_version 1241232 (0.0010) [2023-12-27 00:25:00,348][105692] Updated weights for policy 0, policy_version 1240116 (0.0007) [2023-12-27 00:25:00,389][105620] Updated weights for policy 1, policy_version 1241242 (0.0011) [2023-12-27 00:25:00,408][105692] Updated weights for policy 0, policy_version 1240126 (0.0008) [2023-12-27 00:25:00,465][105692] Updated weights for policy 0, policy_version 1240136 (0.0007) [2023-12-27 00:25:01,051][105620] Updated weights for policy 1, policy_version 1241252 (0.0010) [2023-12-27 00:25:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.4, 300 sec: 19521.9). Total num frames: 635330560. Throughput: 0: 9784.6, 1: 9702.9. Samples: 635305076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:25:01,063][104569] Avg episode reward: [(0, '9086.664'), (1, '9077.598')] [2023-12-27 00:25:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001240144_317530112.pth... [2023-12-27 00:25:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001239024_317243392.pth [2023-12-27 00:25:01,104][105620] Updated weights for policy 1, policy_version 1241262 (0.0010) [2023-12-27 00:25:01,167][105620] Updated weights for policy 1, policy_version 1241272 (0.0009) [2023-12-27 00:25:01,181][105692] Updated weights for policy 0, policy_version 1240146 (0.0007) [2023-12-27 00:25:01,213][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001241280_317808640.pth... [2023-12-27 00:25:01,217][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001240128_317513728.pth [2023-12-27 00:25:01,228][105692] Updated weights for policy 0, policy_version 1240156 (0.0008) [2023-12-27 00:25:01,289][105692] Updated weights for policy 0, policy_version 1240166 (0.0009) [2023-12-27 00:25:01,336][105692] Updated weights for policy 0, policy_version 1240176 (0.0008) [2023-12-27 00:25:01,865][105620] Updated weights for policy 1, policy_version 1241282 (0.0008) [2023-12-27 00:25:01,920][105620] Updated weights for policy 1, policy_version 1241292 (0.0009) [2023-12-27 00:25:01,982][105620] Updated weights for policy 1, policy_version 1241302 (0.0009) [2023-12-27 00:25:02,039][105620] Updated weights for policy 1, policy_version 1241312 (0.0009) [2023-12-27 00:25:02,170][105692] Updated weights for policy 0, policy_version 1240186 (0.0009) [2023-12-27 00:25:02,228][105692] Updated weights for policy 0, policy_version 1240196 (0.0009) [2023-12-27 00:25:02,287][105692] Updated weights for policy 0, policy_version 1240206 (0.0009) [2023-12-27 00:25:02,832][105620] Updated weights for policy 1, policy_version 1241322 (0.0010) [2023-12-27 00:25:02,886][105620] Updated weights for policy 1, policy_version 1241332 (0.0010) [2023-12-27 00:25:02,936][105620] Updated weights for policy 1, policy_version 1241342 (0.0010) [2023-12-27 00:25:03,026][105692] Updated weights for policy 0, policy_version 1240216 (0.0008) [2023-12-27 00:25:03,084][105692] Updated weights for policy 0, policy_version 1240226 (0.0008) [2023-12-27 00:25:03,132][105692] Updated weights for policy 0, policy_version 1240236 (0.0007) [2023-12-27 00:25:03,583][105620] Updated weights for policy 1, policy_version 1241352 (0.0006) [2023-12-27 00:25:03,639][105620] Updated weights for policy 1, policy_version 1241362 (0.0005) [2023-12-27 00:25:03,686][105620] Updated weights for policy 1, policy_version 1241372 (0.0005) [2023-12-27 00:25:03,986][105692] Updated weights for policy 0, policy_version 1240246 (0.0008) [2023-12-27 00:25:04,039][105692] Updated weights for policy 0, policy_version 1240256 (0.0008) [2023-12-27 00:25:04,098][105692] Updated weights for policy 0, policy_version 1240266 (0.0008) [2023-12-27 00:25:04,319][105620] Updated weights for policy 1, policy_version 1241382 (0.0010) [2023-12-27 00:25:04,377][105620] Updated weights for policy 1, policy_version 1241392 (0.0009) [2023-12-27 00:25:04,437][105620] Updated weights for policy 1, policy_version 1241402 (0.0007) [2023-12-27 00:25:04,887][105692] Updated weights for policy 0, policy_version 1240276 (0.0008) [2023-12-27 00:25:04,949][105692] Updated weights for policy 0, policy_version 1240286 (0.0008) [2023-12-27 00:25:05,005][105692] Updated weights for policy 0, policy_version 1240296 (0.0008) [2023-12-27 00:25:05,175][105620] Updated weights for policy 1, policy_version 1241412 (0.0010) [2023-12-27 00:25:05,222][105620] Updated weights for policy 1, policy_version 1241422 (0.0010) [2023-12-27 00:25:05,269][105620] Updated weights for policy 1, policy_version 1241432 (0.0010) [2023-12-27 00:25:05,813][105692] Updated weights for policy 0, policy_version 1240306 (0.0008) [2023-12-27 00:25:05,879][105692] Updated weights for policy 0, policy_version 1240316 (0.0009) [2023-12-27 00:25:05,932][105620] Updated weights for policy 1, policy_version 1241442 (0.0009) [2023-12-27 00:25:05,939][105692] Updated weights for policy 0, policy_version 1240326 (0.0008) [2023-12-27 00:25:05,989][105692] Updated weights for policy 0, policy_version 1240336 (0.0007) [2023-12-27 00:25:05,990][105620] Updated weights for policy 1, policy_version 1241452 (0.0010) [2023-12-27 00:25:06,048][105620] Updated weights for policy 1, policy_version 1241462 (0.0010) [2023-12-27 00:25:06,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 635428864. Throughput: 0: 9587.6, 1: 9791.1. Samples: 635418884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:25:06,063][104569] Avg episode reward: [(0, '9085.999'), (1, '9168.972')] [2023-12-27 00:25:06,109][105620] Updated weights for policy 1, policy_version 1241472 (0.0010) [2023-12-27 00:25:06,760][105692] Updated weights for policy 0, policy_version 1240346 (0.0008) [2023-12-27 00:25:06,824][105692] Updated weights for policy 0, policy_version 1240356 (0.0008) [2023-12-27 00:25:06,884][105692] Updated weights for policy 0, policy_version 1240366 (0.0007) [2023-12-27 00:25:06,897][105620] Updated weights for policy 1, policy_version 1241482 (0.0011) [2023-12-27 00:25:06,946][105620] Updated weights for policy 1, policy_version 1241492 (0.0010) [2023-12-27 00:25:06,997][105620] Updated weights for policy 1, policy_version 1241502 (0.0009) [2023-12-27 00:25:07,644][105692] Updated weights for policy 0, policy_version 1240376 (0.0009) [2023-12-27 00:25:07,700][105692] Updated weights for policy 0, policy_version 1240386 (0.0008) [2023-12-27 00:25:07,720][105620] Updated weights for policy 1, policy_version 1241512 (0.0007) [2023-12-27 00:25:07,760][105692] Updated weights for policy 0, policy_version 1240396 (0.0008) [2023-12-27 00:25:07,770][105620] Updated weights for policy 1, policy_version 1241522 (0.0006) [2023-12-27 00:25:07,832][105620] Updated weights for policy 1, policy_version 1241532 (0.0005) [2023-12-27 00:25:07,855][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000010 [2023-12-27 00:25:08,471][105620] Updated weights for policy 1, policy_version 1241542 (0.0008) [2023-12-27 00:25:08,534][105620] Updated weights for policy 1, policy_version 1241552 (0.0009) [2023-12-27 00:25:08,590][105692] Updated weights for policy 0, policy_version 1240406 (0.0007) [2023-12-27 00:25:08,596][105620] Updated weights for policy 1, policy_version 1241562 (0.0008) [2023-12-27 00:25:08,649][105692] Updated weights for policy 0, policy_version 1240416 (0.0007) [2023-12-27 00:25:08,709][105692] Updated weights for policy 0, policy_version 1240426 (0.0008) [2023-12-27 00:25:09,219][105620] Updated weights for policy 1, policy_version 1241572 (0.0008) [2023-12-27 00:25:09,287][105620] Updated weights for policy 1, policy_version 1241582 (0.0008) [2023-12-27 00:25:09,349][105620] Updated weights for policy 1, policy_version 1241592 (0.0011) [2023-12-27 00:25:09,546][105692] Updated weights for policy 0, policy_version 1240436 (0.0008) [2023-12-27 00:25:09,599][105692] Updated weights for policy 0, policy_version 1240446 (0.0008) [2023-12-27 00:25:09,648][105692] Updated weights for policy 0, policy_version 1240456 (0.0008) [2023-12-27 00:25:10,077][105620] Updated weights for policy 1, policy_version 1241602 (0.0008) [2023-12-27 00:25:10,142][105620] Updated weights for policy 1, policy_version 1241612 (0.0006) [2023-12-27 00:25:10,198][105620] Updated weights for policy 1, policy_version 1241622 (0.0008) [2023-12-27 00:25:10,250][105620] Updated weights for policy 1, policy_version 1241632 (0.0008) [2023-12-27 00:25:10,490][105692] Updated weights for policy 0, policy_version 1240466 (0.0010) [2023-12-27 00:25:10,542][105692] Updated weights for policy 0, policy_version 1240476 (0.0006) [2023-12-27 00:25:10,604][105692] Updated weights for policy 0, policy_version 1240486 (0.0006) [2023-12-27 00:25:10,674][105692] Updated weights for policy 0, policy_version 1240496 (0.0008) [2023-12-27 00:25:10,922][105620] Updated weights for policy 1, policy_version 1241642 (0.0005) [2023-12-27 00:25:10,992][105620] Updated weights for policy 1, policy_version 1241652 (0.0006) [2023-12-27 00:25:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 635518976. Throughput: 0: 9556.6, 1: 9823.1. Samples: 635532332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:25:11,062][104569] Avg episode reward: [(0, '9268.275'), (1, '9259.629')] [2023-12-27 00:25:11,062][105620] Updated weights for policy 1, policy_version 1241662 (0.0009) [2023-12-27 00:25:11,407][105692] Updated weights for policy 0, policy_version 1240506 (0.0009) [2023-12-27 00:25:11,468][105692] Updated weights for policy 0, policy_version 1240516 (0.0009) [2023-12-27 00:25:11,531][105692] Updated weights for policy 0, policy_version 1240526 (0.0009) [2023-12-27 00:25:11,700][105620] Updated weights for policy 1, policy_version 1241672 (0.0007) [2023-12-27 00:25:11,761][105620] Updated weights for policy 1, policy_version 1241682 (0.0011) [2023-12-27 00:25:11,814][105620] Updated weights for policy 1, policy_version 1241692 (0.0010) [2023-12-27 00:25:12,309][105692] Updated weights for policy 0, policy_version 1240536 (0.0010) [2023-12-27 00:25:12,376][105692] Updated weights for policy 0, policy_version 1240546 (0.0010) [2023-12-27 00:25:12,440][105692] Updated weights for policy 0, policy_version 1240556 (0.0011) [2023-12-27 00:25:12,468][105620] Updated weights for policy 1, policy_version 1241702 (0.0011) [2023-12-27 00:25:12,528][105620] Updated weights for policy 1, policy_version 1241712 (0.0011) [2023-12-27 00:25:12,586][105620] Updated weights for policy 1, policy_version 1241722 (0.0011) [2023-12-27 00:25:13,080][105692] Updated weights for policy 0, policy_version 1240566 (0.0007) [2023-12-27 00:25:13,142][105692] Updated weights for policy 0, policy_version 1240576 (0.0005) [2023-12-27 00:25:13,191][105692] Updated weights for policy 0, policy_version 1240586 (0.0008) [2023-12-27 00:25:13,306][105620] Updated weights for policy 1, policy_version 1241732 (0.0011) [2023-12-27 00:25:13,355][105620] Updated weights for policy 1, policy_version 1241742 (0.0008) [2023-12-27 00:25:13,425][105620] Updated weights for policy 1, policy_version 1241752 (0.0011) [2023-12-27 00:25:13,870][105692] Updated weights for policy 0, policy_version 1240596 (0.0008) [2023-12-27 00:25:13,929][105692] Updated weights for policy 0, policy_version 1240606 (0.0008) [2023-12-27 00:25:13,978][105692] Updated weights for policy 0, policy_version 1240616 (0.0010) [2023-12-27 00:25:14,147][105620] Updated weights for policy 1, policy_version 1241762 (0.0011) [2023-12-27 00:25:14,210][105620] Updated weights for policy 1, policy_version 1241772 (0.0011) [2023-12-27 00:25:14,279][105620] Updated weights for policy 1, policy_version 1241782 (0.0010) [2023-12-27 00:25:14,338][105620] Updated weights for policy 1, policy_version 1241792 (0.0011) [2023-12-27 00:25:14,714][105692] Updated weights for policy 0, policy_version 1240626 (0.0010) [2023-12-27 00:25:14,780][105692] Updated weights for policy 0, policy_version 1240636 (0.0010) [2023-12-27 00:25:14,842][105692] Updated weights for policy 0, policy_version 1240646 (0.0008) [2023-12-27 00:25:14,900][105692] Updated weights for policy 0, policy_version 1240656 (0.0008) [2023-12-27 00:25:14,978][105620] Updated weights for policy 1, policy_version 1241802 (0.0011) [2023-12-27 00:25:15,040][105620] Updated weights for policy 1, policy_version 1241812 (0.0009) [2023-12-27 00:25:15,103][105620] Updated weights for policy 1, policy_version 1241822 (0.0011) [2023-12-27 00:25:15,678][105692] Updated weights for policy 0, policy_version 1240666 (0.0008) [2023-12-27 00:25:15,739][105692] Updated weights for policy 0, policy_version 1240676 (0.0008) [2023-12-27 00:25:15,802][105692] Updated weights for policy 0, policy_version 1240686 (0.0008) [2023-12-27 00:25:15,838][105620] Updated weights for policy 1, policy_version 1241832 (0.0011) [2023-12-27 00:25:15,889][105620] Updated weights for policy 1, policy_version 1241842 (0.0011) [2023-12-27 00:25:15,951][105620] Updated weights for policy 1, policy_version 1241852 (0.0010) [2023-12-27 00:25:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 635625472. Throughput: 0: 9555.1, 1: 9803.4. Samples: 635591796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:25:16,063][104569] Avg episode reward: [(0, '9357.171'), (1, '8895.541')] [2023-12-27 00:25:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001240688_317669376.pth... [2023-12-27 00:25:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001241856_317956096.pth... [2023-12-27 00:25:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001239600_317390848.pth [2023-12-27 00:25:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001240704_317661184.pth [2023-12-27 00:25:16,537][105692] Updated weights for policy 0, policy_version 1240696 (0.0008) [2023-12-27 00:25:16,585][105692] Updated weights for policy 0, policy_version 1240706 (0.0008) [2023-12-27 00:25:16,643][105692] Updated weights for policy 0, policy_version 1240716 (0.0008) [2023-12-27 00:25:16,704][105620] Updated weights for policy 1, policy_version 1241862 (0.0011) [2023-12-27 00:25:16,761][105620] Updated weights for policy 1, policy_version 1241872 (0.0010) [2023-12-27 00:25:16,813][105620] Updated weights for policy 1, policy_version 1241882 (0.0007) [2023-12-27 00:25:17,429][105692] Updated weights for policy 0, policy_version 1240726 (0.0008) [2023-12-27 00:25:17,480][105692] Updated weights for policy 0, policy_version 1240736 (0.0008) [2023-12-27 00:25:17,518][105620] Updated weights for policy 1, policy_version 1241892 (0.0007) [2023-12-27 00:25:17,532][105692] Updated weights for policy 0, policy_version 1240746 (0.0007) [2023-12-27 00:25:17,566][105620] Updated weights for policy 1, policy_version 1241902 (0.0010) [2023-12-27 00:25:17,614][105620] Updated weights for policy 1, policy_version 1241912 (0.0010) [2023-12-27 00:25:18,255][105692] Updated weights for policy 0, policy_version 1240756 (0.0005) [2023-12-27 00:25:18,288][105620] Updated weights for policy 1, policy_version 1241922 (0.0009) [2023-12-27 00:25:18,313][105692] Updated weights for policy 0, policy_version 1240766 (0.0005) [2023-12-27 00:25:18,358][105620] Updated weights for policy 1, policy_version 1241932 (0.0006) [2023-12-27 00:25:18,372][105692] Updated weights for policy 0, policy_version 1240776 (0.0009) [2023-12-27 00:25:18,420][105620] Updated weights for policy 1, policy_version 1241942 (0.0006) [2023-12-27 00:25:18,484][105620] Updated weights for policy 1, policy_version 1241952 (0.0008) [2023-12-27 00:25:19,065][105692] Updated weights for policy 0, policy_version 1240786 (0.0008) [2023-12-27 00:25:19,119][105620] Updated weights for policy 1, policy_version 1241962 (0.0006) [2023-12-27 00:25:19,120][105692] Updated weights for policy 0, policy_version 1240796 (0.0010) [2023-12-27 00:25:19,179][105692] Updated weights for policy 0, policy_version 1240806 (0.0010) [2023-12-27 00:25:19,181][105620] Updated weights for policy 1, policy_version 1241972 (0.0005) [2023-12-27 00:25:19,238][105692] Updated weights for policy 0, policy_version 1240816 (0.0010) [2023-12-27 00:25:19,240][105620] Updated weights for policy 1, policy_version 1241982 (0.0007) [2023-12-27 00:25:19,857][105620] Updated weights for policy 1, policy_version 1241992 (0.0008) [2023-12-27 00:25:19,924][105620] Updated weights for policy 1, policy_version 1242002 (0.0007) [2023-12-27 00:25:19,987][105620] Updated weights for policy 1, policy_version 1242012 (0.0008) [2023-12-27 00:25:20,017][105692] Updated weights for policy 0, policy_version 1240826 (0.0010) [2023-12-27 00:25:20,081][105692] Updated weights for policy 0, policy_version 1240836 (0.0009) [2023-12-27 00:25:20,137][105692] Updated weights for policy 0, policy_version 1240846 (0.0010) [2023-12-27 00:25:20,723][105620] Updated weights for policy 1, policy_version 1242022 (0.0008) [2023-12-27 00:25:20,785][105620] Updated weights for policy 1, policy_version 1242032 (0.0008) [2023-12-27 00:25:20,842][105620] Updated weights for policy 1, policy_version 1242042 (0.0008) [2023-12-27 00:25:20,905][105692] Updated weights for policy 0, policy_version 1240856 (0.0011) [2023-12-27 00:25:20,962][105692] Updated weights for policy 0, policy_version 1240866 (0.0011) [2023-12-27 00:25:21,015][105692] Updated weights for policy 0, policy_version 1240876 (0.0011) [2023-12-27 00:25:21,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 635723776. Throughput: 0: 9530.7, 1: 9851.2. Samples: 635708492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:25:21,063][104569] Avg episode reward: [(0, '9355.513'), (1, '8987.684')] [2023-12-27 00:25:21,643][105620] Updated weights for policy 1, policy_version 1242052 (0.0009) [2023-12-27 00:25:21,709][105620] Updated weights for policy 1, policy_version 1242062 (0.0008) [2023-12-27 00:25:21,791][105620] Updated weights for policy 1, policy_version 1242074 (0.0009) [2023-12-27 00:25:21,839][105692] Updated weights for policy 0, policy_version 1240886 (0.0011) [2023-12-27 00:25:21,898][105692] Updated weights for policy 0, policy_version 1240896 (0.0011) [2023-12-27 00:25:21,964][105692] Updated weights for policy 0, policy_version 1240906 (0.0011) [2023-12-27 00:25:22,513][105620] Updated weights for policy 1, policy_version 1242084 (0.0009) [2023-12-27 00:25:22,574][105620] Updated weights for policy 1, policy_version 1242094 (0.0010) [2023-12-27 00:25:22,636][105620] Updated weights for policy 1, policy_version 1242104 (0.0005) [2023-12-27 00:25:22,714][105692] Updated weights for policy 0, policy_version 1240916 (0.0011) [2023-12-27 00:25:22,773][105692] Updated weights for policy 0, policy_version 1240926 (0.0010) [2023-12-27 00:25:22,836][105692] Updated weights for policy 0, policy_version 1240936 (0.0010) [2023-12-27 00:25:23,295][105620] Updated weights for policy 1, policy_version 1242114 (0.0005) [2023-12-27 00:25:23,360][105620] Updated weights for policy 1, policy_version 1242124 (0.0010) [2023-12-27 00:25:23,427][105620] Updated weights for policy 1, policy_version 1242134 (0.0011) [2023-12-27 00:25:23,486][105620] Updated weights for policy 1, policy_version 1242144 (0.0011) [2023-12-27 00:25:23,588][105692] Updated weights for policy 0, policy_version 1240946 (0.0011) [2023-12-27 00:25:23,654][105692] Updated weights for policy 0, policy_version 1240956 (0.0011) [2023-12-27 00:25:23,720][105692] Updated weights for policy 0, policy_version 1240966 (0.0011) [2023-12-27 00:25:23,786][105692] Updated weights for policy 0, policy_version 1240976 (0.0011) [2023-12-27 00:25:24,217][105620] Updated weights for policy 1, policy_version 1242154 (0.0011) [2023-12-27 00:25:24,282][105620] Updated weights for policy 1, policy_version 1242164 (0.0011) [2023-12-27 00:25:24,345][105620] Updated weights for policy 1, policy_version 1242174 (0.0010) [2023-12-27 00:25:24,434][105692] Updated weights for policy 0, policy_version 1240986 (0.0010) [2023-12-27 00:25:24,499][105692] Updated weights for policy 0, policy_version 1240996 (0.0010) [2023-12-27 00:25:24,557][105692] Updated weights for policy 0, policy_version 1241006 (0.0010) [2023-12-27 00:25:25,007][105620] Updated weights for policy 1, policy_version 1242184 (0.0008) [2023-12-27 00:25:25,055][105620] Updated weights for policy 1, policy_version 1242194 (0.0008) [2023-12-27 00:25:25,111][105620] Updated weights for policy 1, policy_version 1242204 (0.0006) [2023-12-27 00:25:25,292][105692] Updated weights for policy 0, policy_version 1241016 (0.0010) [2023-12-27 00:25:25,350][105692] Updated weights for policy 0, policy_version 1241026 (0.0010) [2023-12-27 00:25:25,416][105692] Updated weights for policy 0, policy_version 1241036 (0.0010) [2023-12-27 00:25:25,826][105620] Updated weights for policy 1, policy_version 1242214 (0.0006) [2023-12-27 00:25:25,874][105620] Updated weights for policy 1, policy_version 1242224 (0.0005) [2023-12-27 00:25:25,927][105620] Updated weights for policy 1, policy_version 1242234 (0.0007) [2023-12-27 00:25:25,978][105692] Updated weights for policy 0, policy_version 1241046 (0.0007) [2023-12-27 00:25:26,041][105692] Updated weights for policy 0, policy_version 1241056 (0.0006) [2023-12-27 00:25:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 635813888. Throughput: 0: 9514.7, 1: 9788.4. Samples: 635821792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:25:26,063][104569] Avg episode reward: [(0, '9264.441'), (1, '9249.371')] [2023-12-27 00:25:26,096][105692] Updated weights for policy 0, policy_version 1241066 (0.0006) [2023-12-27 00:25:26,498][105620] Updated weights for policy 1, policy_version 1242244 (0.0006) [2023-12-27 00:25:26,543][105620] Updated weights for policy 1, policy_version 1242254 (0.0007) [2023-12-27 00:25:26,595][105620] Updated weights for policy 1, policy_version 1242264 (0.0008) [2023-12-27 00:25:26,696][105692] Updated weights for policy 0, policy_version 1241076 (0.0008) [2023-12-27 00:25:26,753][105692] Updated weights for policy 0, policy_version 1241086 (0.0010) [2023-12-27 00:25:26,796][105692] Updated weights for policy 0, policy_version 1241096 (0.0010) [2023-12-27 00:25:27,399][105620] Updated weights for policy 1, policy_version 1242274 (0.0007) [2023-12-27 00:25:27,403][105692] Updated weights for policy 0, policy_version 1241106 (0.0008) [2023-12-27 00:25:27,449][105620] Updated weights for policy 1, policy_version 1242284 (0.0005) [2023-12-27 00:25:27,454][105692] Updated weights for policy 0, policy_version 1241116 (0.0005) [2023-12-27 00:25:27,507][105620] Updated weights for policy 1, policy_version 1242294 (0.0006) [2023-12-27 00:25:27,509][105692] Updated weights for policy 0, policy_version 1241126 (0.0007) [2023-12-27 00:25:27,558][105620] Updated weights for policy 1, policy_version 1242304 (0.0006) [2023-12-27 00:25:27,568][105692] Updated weights for policy 0, policy_version 1241136 (0.0008) [2023-12-27 00:25:28,083][105620] Updated weights for policy 1, policy_version 1242314 (0.0008) [2023-12-27 00:25:28,137][105620] Updated weights for policy 1, policy_version 1242324 (0.0009) [2023-12-27 00:25:28,184][105620] Updated weights for policy 1, policy_version 1242334 (0.0010) [2023-12-27 00:25:28,340][105692] Updated weights for policy 0, policy_version 1241146 (0.0007) [2023-12-27 00:25:28,394][105692] Updated weights for policy 0, policy_version 1241156 (0.0008) [2023-12-27 00:25:28,443][105692] Updated weights for policy 0, policy_version 1241166 (0.0008) [2023-12-27 00:25:28,929][105620] Updated weights for policy 1, policy_version 1242344 (0.0010) [2023-12-27 00:25:28,981][105620] Updated weights for policy 1, policy_version 1242354 (0.0010) [2023-12-27 00:25:29,036][105620] Updated weights for policy 1, policy_version 1242364 (0.0009) [2023-12-27 00:25:29,216][105692] Updated weights for policy 0, policy_version 1241176 (0.0010) [2023-12-27 00:25:29,282][105692] Updated weights for policy 0, policy_version 1241186 (0.0011) [2023-12-27 00:25:29,347][105692] Updated weights for policy 0, policy_version 1241196 (0.0008) [2023-12-27 00:25:29,731][105620] Updated weights for policy 1, policy_version 1242374 (0.0005) [2023-12-27 00:25:29,795][105620] Updated weights for policy 1, policy_version 1242384 (0.0005) [2023-12-27 00:25:29,861][105620] Updated weights for policy 1, policy_version 1242394 (0.0008) [2023-12-27 00:25:30,035][105692] Updated weights for policy 0, policy_version 1241206 (0.0007) [2023-12-27 00:25:30,084][105692] Updated weights for policy 0, policy_version 1241216 (0.0005) [2023-12-27 00:25:30,132][105692] Updated weights for policy 0, policy_version 1241226 (0.0008) [2023-12-27 00:25:30,494][105620] Updated weights for policy 1, policy_version 1242404 (0.0007) [2023-12-27 00:25:30,542][105620] Updated weights for policy 1, policy_version 1242414 (0.0010) [2023-12-27 00:25:30,593][105620] Updated weights for policy 1, policy_version 1242424 (0.0010) [2023-12-27 00:25:30,871][105692] Updated weights for policy 0, policy_version 1241236 (0.0008) [2023-12-27 00:25:30,925][105692] Updated weights for policy 0, policy_version 1241246 (0.0007) [2023-12-27 00:25:30,983][105692] Updated weights for policy 0, policy_version 1241256 (0.0008) [2023-12-27 00:25:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 635920384. Throughput: 0: 9530.6, 1: 9920.7. Samples: 635885580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:25:31,063][104569] Avg episode reward: [(0, '9170.471'), (1, '9156.815')] [2023-12-27 00:25:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001241264_317816832.pth... [2023-12-27 00:25:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001242432_318103552.pth... [2023-12-27 00:25:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001240144_317530112.pth [2023-12-27 00:25:31,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001241280_317808640.pth [2023-12-27 00:25:31,346][105620] Updated weights for policy 1, policy_version 1242434 (0.0010) [2023-12-27 00:25:31,417][105620] Updated weights for policy 1, policy_version 1242444 (0.0010) [2023-12-27 00:25:31,469][105620] Updated weights for policy 1, policy_version 1242454 (0.0010) [2023-12-27 00:25:31,523][105620] Updated weights for policy 1, policy_version 1242464 (0.0010) [2023-12-27 00:25:31,668][105692] Updated weights for policy 0, policy_version 1241266 (0.0007) [2023-12-27 00:25:31,732][105692] Updated weights for policy 0, policy_version 1241276 (0.0007) [2023-12-27 00:25:31,791][105692] Updated weights for policy 0, policy_version 1241286 (0.0011) [2023-12-27 00:25:31,836][105692] Updated weights for policy 0, policy_version 1241296 (0.0010) [2023-12-27 00:25:32,285][105620] Updated weights for policy 1, policy_version 1242474 (0.0006) [2023-12-27 00:25:32,349][105620] Updated weights for policy 1, policy_version 1242484 (0.0007) [2023-12-27 00:25:32,408][105620] Updated weights for policy 1, policy_version 1242494 (0.0006) [2023-12-27 00:25:32,560][105692] Updated weights for policy 0, policy_version 1241306 (0.0010) [2023-12-27 00:25:32,613][105692] Updated weights for policy 0, policy_version 1241316 (0.0010) [2023-12-27 00:25:32,665][105692] Updated weights for policy 0, policy_version 1241326 (0.0007) [2023-12-27 00:25:33,135][105620] Updated weights for policy 1, policy_version 1242504 (0.0008) [2023-12-27 00:25:33,193][105620] Updated weights for policy 1, policy_version 1242514 (0.0006) [2023-12-27 00:25:33,245][105620] Updated weights for policy 1, policy_version 1242524 (0.0005) [2023-12-27 00:25:33,328][105692] Updated weights for policy 0, policy_version 1241336 (0.0006) [2023-12-27 00:25:33,376][105692] Updated weights for policy 0, policy_version 1241346 (0.0008) [2023-12-27 00:25:33,423][105692] Updated weights for policy 0, policy_version 1241356 (0.0010) [2023-12-27 00:25:33,878][105620] Updated weights for policy 1, policy_version 1242534 (0.0008) [2023-12-27 00:25:33,925][105620] Updated weights for policy 1, policy_version 1242544 (0.0008) [2023-12-27 00:25:33,976][105620] Updated weights for policy 1, policy_version 1242554 (0.0008) [2023-12-27 00:25:34,104][105692] Updated weights for policy 0, policy_version 1241366 (0.0008) [2023-12-27 00:25:34,163][105692] Updated weights for policy 0, policy_version 1241376 (0.0008) [2023-12-27 00:25:34,220][105692] Updated weights for policy 0, policy_version 1241386 (0.0008) [2023-12-27 00:25:34,769][105620] Updated weights for policy 1, policy_version 1242564 (0.0009) [2023-12-27 00:25:34,829][105620] Updated weights for policy 1, policy_version 1242574 (0.0008) [2023-12-27 00:25:34,874][105692] Updated weights for policy 0, policy_version 1241396 (0.0008) [2023-12-27 00:25:34,892][105620] Updated weights for policy 1, policy_version 1242584 (0.0008) [2023-12-27 00:25:34,927][105692] Updated weights for policy 0, policy_version 1241406 (0.0007) [2023-12-27 00:25:34,991][105692] Updated weights for policy 0, policy_version 1241416 (0.0008) [2023-12-27 00:25:35,599][105692] Updated weights for policy 0, policy_version 1241426 (0.0009) [2023-12-27 00:25:35,620][105620] Updated weights for policy 1, policy_version 1242594 (0.0007) [2023-12-27 00:25:35,650][105692] Updated weights for policy 0, policy_version 1241436 (0.0008) [2023-12-27 00:25:35,669][105620] Updated weights for policy 1, policy_version 1242604 (0.0010) [2023-12-27 00:25:35,703][105692] Updated weights for policy 0, policy_version 1241446 (0.0005) [2023-12-27 00:25:35,716][105620] Updated weights for policy 1, policy_version 1242614 (0.0008) [2023-12-27 00:25:35,756][105692] Updated weights for policy 0, policy_version 1241456 (0.0007) [2023-12-27 00:25:35,763][105620] Updated weights for policy 1, policy_version 1242624 (0.0009) [2023-12-27 00:25:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 636018688. Throughput: 0: 9551.3, 1: 9908.4. Samples: 636004264. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:25:36,062][104569] Avg episode reward: [(0, '9076.749'), (1, '9259.119')] [2023-12-27 00:25:36,444][105620] Updated weights for policy 1, policy_version 1242634 (0.0008) [2023-12-27 00:25:36,504][105620] Updated weights for policy 1, policy_version 1242644 (0.0007) [2023-12-27 00:25:36,568][105692] Updated weights for policy 0, policy_version 1241466 (0.0011) [2023-12-27 00:25:36,570][105620] Updated weights for policy 1, policy_version 1242654 (0.0008) [2023-12-27 00:25:36,635][105692] Updated weights for policy 0, policy_version 1241476 (0.0010) [2023-12-27 00:25:36,702][105692] Updated weights for policy 0, policy_version 1241486 (0.0011) [2023-12-27 00:25:37,175][105620] Updated weights for policy 1, policy_version 1242664 (0.0006) [2023-12-27 00:25:37,225][105620] Updated weights for policy 1, policy_version 1242674 (0.0005) [2023-12-27 00:25:37,288][105620] Updated weights for policy 1, policy_version 1242684 (0.0006) [2023-12-27 00:25:37,352][105692] Updated weights for policy 0, policy_version 1241496 (0.0010) [2023-12-27 00:25:37,414][105692] Updated weights for policy 0, policy_version 1241506 (0.0010) [2023-12-27 00:25:37,477][105692] Updated weights for policy 0, policy_version 1241516 (0.0009) [2023-12-27 00:25:37,911][105620] Updated weights for policy 1, policy_version 1242694 (0.0008) [2023-12-27 00:25:37,969][105620] Updated weights for policy 1, policy_version 1242704 (0.0010) [2023-12-27 00:25:38,024][105620] Updated weights for policy 1, policy_version 1242714 (0.0010) [2023-12-27 00:25:38,135][105692] Updated weights for policy 0, policy_version 1241526 (0.0007) [2023-12-27 00:25:38,187][105692] Updated weights for policy 0, policy_version 1241536 (0.0008) [2023-12-27 00:25:38,245][105692] Updated weights for policy 0, policy_version 1241546 (0.0008) [2023-12-27 00:25:38,775][105620] Updated weights for policy 1, policy_version 1242724 (0.0010) [2023-12-27 00:25:38,840][105620] Updated weights for policy 1, policy_version 1242734 (0.0010) [2023-12-27 00:25:38,905][105620] Updated weights for policy 1, policy_version 1242744 (0.0010) [2023-12-27 00:25:39,016][105692] Updated weights for policy 0, policy_version 1241556 (0.0008) [2023-12-27 00:25:39,064][105692] Updated weights for policy 0, policy_version 1241566 (0.0007) [2023-12-27 00:25:39,109][105692] Updated weights for policy 0, policy_version 1241576 (0.0008) [2023-12-27 00:25:39,651][105620] Updated weights for policy 1, policy_version 1242754 (0.0010) [2023-12-27 00:25:39,710][105620] Updated weights for policy 1, policy_version 1242764 (0.0011) [2023-12-27 00:25:39,770][105620] Updated weights for policy 1, policy_version 1242774 (0.0011) [2023-12-27 00:25:39,832][105620] Updated weights for policy 1, policy_version 1242784 (0.0011) [2023-12-27 00:25:39,978][105692] Updated weights for policy 0, policy_version 1241586 (0.0008) [2023-12-27 00:25:40,043][105692] Updated weights for policy 0, policy_version 1241596 (0.0008) [2023-12-27 00:25:40,103][105692] Updated weights for policy 0, policy_version 1241606 (0.0008) [2023-12-27 00:25:40,167][105692] Updated weights for policy 0, policy_version 1241616 (0.0008) [2023-12-27 00:25:40,608][105620] Updated weights for policy 1, policy_version 1242794 (0.0011) [2023-12-27 00:25:40,667][105620] Updated weights for policy 1, policy_version 1242804 (0.0010) [2023-12-27 00:25:40,722][105620] Updated weights for policy 1, policy_version 1242814 (0.0010) [2023-12-27 00:25:40,945][105692] Updated weights for policy 0, policy_version 1241626 (0.0007) [2023-12-27 00:25:40,994][105692] Updated weights for policy 0, policy_version 1241636 (0.0008) [2023-12-27 00:25:41,053][105692] Updated weights for policy 0, policy_version 1241646 (0.0008) [2023-12-27 00:25:41,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 636116992. Throughput: 0: 9541.0, 1: 9905.9. Samples: 636121320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:25:41,062][104569] Avg episode reward: [(0, '8897.442'), (1, '9352.487')] [2023-12-27 00:25:41,484][105620] Updated weights for policy 1, policy_version 1242824 (0.0011) [2023-12-27 00:25:41,536][105620] Updated weights for policy 1, policy_version 1242834 (0.0011) [2023-12-27 00:25:41,589][105620] Updated weights for policy 1, policy_version 1242844 (0.0010) [2023-12-27 00:25:41,871][105692] Updated weights for policy 0, policy_version 1241656 (0.0009) [2023-12-27 00:25:41,941][105692] Updated weights for policy 0, policy_version 1241666 (0.0009) [2023-12-27 00:25:42,011][105692] Updated weights for policy 0, policy_version 1241676 (0.0009) [2023-12-27 00:25:42,354][105620] Updated weights for policy 1, policy_version 1242854 (0.0010) [2023-12-27 00:25:42,410][105620] Updated weights for policy 1, policy_version 1242864 (0.0009) [2023-12-27 00:25:42,472][105620] Updated weights for policy 1, policy_version 1242874 (0.0009) [2023-12-27 00:25:42,729][105692] Updated weights for policy 0, policy_version 1241686 (0.0009) [2023-12-27 00:25:42,790][105692] Updated weights for policy 0, policy_version 1241696 (0.0007) [2023-12-27 00:25:42,849][105692] Updated weights for policy 0, policy_version 1241706 (0.0005) [2023-12-27 00:25:43,192][105620] Updated weights for policy 1, policy_version 1242884 (0.0008) [2023-12-27 00:25:43,243][105620] Updated weights for policy 1, policy_version 1242894 (0.0009) [2023-12-27 00:25:43,305][105620] Updated weights for policy 1, policy_version 1242904 (0.0009) [2023-12-27 00:25:43,569][105692] Updated weights for policy 0, policy_version 1241716 (0.0007) [2023-12-27 00:25:43,629][105692] Updated weights for policy 0, policy_version 1241726 (0.0009) [2023-12-27 00:25:43,693][105692] Updated weights for policy 0, policy_version 1241736 (0.0009) [2023-12-27 00:25:43,952][105620] Updated weights for policy 1, policy_version 1242914 (0.0009) [2023-12-27 00:25:44,016][105620] Updated weights for policy 1, policy_version 1242924 (0.0010) [2023-12-27 00:25:44,074][105620] Updated weights for policy 1, policy_version 1242934 (0.0009) [2023-12-27 00:25:44,122][105620] Updated weights for policy 1, policy_version 1242944 (0.0007) [2023-12-27 00:25:44,492][105692] Updated weights for policy 0, policy_version 1241746 (0.0010) [2023-12-27 00:25:44,549][105692] Updated weights for policy 0, policy_version 1241756 (0.0008) [2023-12-27 00:25:44,607][105692] Updated weights for policy 0, policy_version 1241766 (0.0008) [2023-12-27 00:25:44,673][105692] Updated weights for policy 0, policy_version 1241776 (0.0009) [2023-12-27 00:25:44,834][105620] Updated weights for policy 1, policy_version 1242954 (0.0007) [2023-12-27 00:25:44,900][105620] Updated weights for policy 1, policy_version 1242964 (0.0009) [2023-12-27 00:25:44,955][105620] Updated weights for policy 1, policy_version 1242974 (0.0009) [2023-12-27 00:25:45,473][105692] Updated weights for policy 0, policy_version 1241786 (0.0009) [2023-12-27 00:25:45,532][105692] Updated weights for policy 0, policy_version 1241796 (0.0009) [2023-12-27 00:25:45,626][105692] Updated weights for policy 0, policy_version 1241806 (0.0009) [2023-12-27 00:25:45,713][105620] Updated weights for policy 1, policy_version 1242984 (0.0009) [2023-12-27 00:25:45,774][105620] Updated weights for policy 1, policy_version 1242994 (0.0009) [2023-12-27 00:25:45,826][105620] Updated weights for policy 1, policy_version 1243004 (0.0010) [2023-12-27 00:25:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 636207104. Throughput: 0: 9478.1, 1: 9923.2. Samples: 636178136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:25:46,063][104569] Avg episode reward: [(0, '8987.366'), (1, '9352.905')] [2023-12-27 00:25:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001241808_317956096.pth... [2023-12-27 00:25:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001243008_318251008.pth... [2023-12-27 00:25:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001241856_317956096.pth [2023-12-27 00:25:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001240688_317669376.pth [2023-12-27 00:25:46,326][105692] Updated weights for policy 0, policy_version 1241816 (0.0010) [2023-12-27 00:25:46,372][105692] Updated weights for policy 0, policy_version 1241826 (0.0008) [2023-12-27 00:25:46,427][105692] Updated weights for policy 0, policy_version 1241836 (0.0009) [2023-12-27 00:25:46,595][105620] Updated weights for policy 1, policy_version 1243014 (0.0009) [2023-12-27 00:25:46,653][105620] Updated weights for policy 1, policy_version 1243024 (0.0009) [2023-12-27 00:25:46,710][105620] Updated weights for policy 1, policy_version 1243034 (0.0008) [2023-12-27 00:25:47,223][105692] Updated weights for policy 0, policy_version 1241846 (0.0008) [2023-12-27 00:25:47,284][105692] Updated weights for policy 0, policy_version 1241856 (0.0008) [2023-12-27 00:25:47,344][105692] Updated weights for policy 0, policy_version 1241866 (0.0005) [2023-12-27 00:25:47,413][105620] Updated weights for policy 1, policy_version 1243044 (0.0008) [2023-12-27 00:25:47,476][105620] Updated weights for policy 1, policy_version 1243054 (0.0005) [2023-12-27 00:25:47,547][105620] Updated weights for policy 1, policy_version 1243064 (0.0009) [2023-12-27 00:25:48,024][105692] Updated weights for policy 0, policy_version 1241877 (0.0007) [2023-12-27 00:25:48,075][105692] Updated weights for policy 0, policy_version 1241887 (0.0009) [2023-12-27 00:25:48,129][105692] Updated weights for policy 0, policy_version 1241897 (0.0009) [2023-12-27 00:25:48,168][105620] Updated weights for policy 1, policy_version 1243074 (0.0009) [2023-12-27 00:25:48,229][105620] Updated weights for policy 1, policy_version 1243084 (0.0005) [2023-12-27 00:25:48,299][105620] Updated weights for policy 1, policy_version 1243094 (0.0005) [2023-12-27 00:25:48,365][105620] Updated weights for policy 1, policy_version 1243104 (0.0008) [2023-12-27 00:25:48,988][105620] Updated weights for policy 1, policy_version 1243114 (0.0011) [2023-12-27 00:25:49,021][105692] Updated weights for policy 0, policy_version 1241907 (0.0009) [2023-12-27 00:25:49,050][105620] Updated weights for policy 1, policy_version 1243124 (0.0006) [2023-12-27 00:25:49,083][105692] Updated weights for policy 0, policy_version 1241917 (0.0009) [2023-12-27 00:25:49,114][105620] Updated weights for policy 1, policy_version 1243134 (0.0010) [2023-12-27 00:25:49,145][105692] Updated weights for policy 0, policy_version 1241927 (0.0007) [2023-12-27 00:25:49,770][105620] Updated weights for policy 1, policy_version 1243144 (0.0011) [2023-12-27 00:25:49,836][105620] Updated weights for policy 1, policy_version 1243154 (0.0011) [2023-12-27 00:25:49,907][105620] Updated weights for policy 1, policy_version 1243164 (0.0010) [2023-12-27 00:25:49,977][105692] Updated weights for policy 0, policy_version 1241937 (0.0008) [2023-12-27 00:25:50,033][105692] Updated weights for policy 0, policy_version 1241947 (0.0008) [2023-12-27 00:25:50,091][105692] Updated weights for policy 0, policy_version 1241957 (0.0007) [2023-12-27 00:25:50,153][105692] Updated weights for policy 0, policy_version 1241967 (0.0005) [2023-12-27 00:25:50,578][105620] Updated weights for policy 1, policy_version 1243174 (0.0010) [2023-12-27 00:25:50,637][105620] Updated weights for policy 1, policy_version 1243184 (0.0009) [2023-12-27 00:25:50,695][105620] Updated weights for policy 1, policy_version 1243194 (0.0006) [2023-12-27 00:25:50,876][105692] Updated weights for policy 0, policy_version 1241977 (0.0007) [2023-12-27 00:25:50,924][105692] Updated weights for policy 0, policy_version 1241987 (0.0008) [2023-12-27 00:25:50,980][105692] Updated weights for policy 0, policy_version 1241997 (0.0007) [2023-12-27 00:25:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 636305408. Throughput: 0: 9454.1, 1: 9928.5. Samples: 636291096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:25:51,062][104569] Avg episode reward: [(0, '9168.443'), (1, '9352.600')] [2023-12-27 00:25:51,374][105620] Updated weights for policy 1, policy_version 1243204 (0.0009) [2023-12-27 00:25:51,439][105620] Updated weights for policy 1, policy_version 1243214 (0.0007) [2023-12-27 00:25:51,499][105620] Updated weights for policy 1, policy_version 1243224 (0.0006) [2023-12-27 00:25:51,814][105692] Updated weights for policy 0, policy_version 1242007 (0.0009) [2023-12-27 00:25:51,883][105692] Updated weights for policy 0, policy_version 1242017 (0.0010) [2023-12-27 00:25:51,948][105692] Updated weights for policy 0, policy_version 1242027 (0.0009) [2023-12-27 00:25:52,168][105620] Updated weights for policy 1, policy_version 1243234 (0.0007) [2023-12-27 00:25:52,219][105620] Updated weights for policy 1, policy_version 1243244 (0.0009) [2023-12-27 00:25:52,274][105620] Updated weights for policy 1, policy_version 1243254 (0.0009) [2023-12-27 00:25:52,322][105620] Updated weights for policy 1, policy_version 1243264 (0.0007) [2023-12-27 00:25:52,758][105692] Updated weights for policy 0, policy_version 1242037 (0.0009) [2023-12-27 00:25:52,805][105692] Updated weights for policy 0, policy_version 1242047 (0.0009) [2023-12-27 00:25:52,861][105692] Updated weights for policy 0, policy_version 1242057 (0.0009) [2023-12-27 00:25:53,072][105620] Updated weights for policy 1, policy_version 1243274 (0.0005) [2023-12-27 00:25:53,133][105620] Updated weights for policy 1, policy_version 1243284 (0.0005) [2023-12-27 00:25:53,182][105620] Updated weights for policy 1, policy_version 1243294 (0.0007) [2023-12-27 00:25:53,504][105692] Updated weights for policy 0, policy_version 1242067 (0.0008) [2023-12-27 00:25:53,573][105692] Updated weights for policy 0, policy_version 1242077 (0.0009) [2023-12-27 00:25:53,635][105692] Updated weights for policy 0, policy_version 1242087 (0.0010) [2023-12-27 00:25:53,688][105620] Updated weights for policy 1, policy_version 1243304 (0.0005) [2023-12-27 00:25:53,736][105620] Updated weights for policy 1, policy_version 1243314 (0.0008) [2023-12-27 00:25:53,790][105620] Updated weights for policy 1, policy_version 1243324 (0.0005) [2023-12-27 00:25:54,348][105692] Updated weights for policy 0, policy_version 1242097 (0.0010) [2023-12-27 00:25:54,401][105692] Updated weights for policy 0, policy_version 1242107 (0.0011) [2023-12-27 00:25:54,452][105692] Updated weights for policy 0, policy_version 1242117 (0.0010) [2023-12-27 00:25:54,509][105620] Updated weights for policy 1, policy_version 1243334 (0.0008) [2023-12-27 00:25:54,513][105692] Updated weights for policy 0, policy_version 1242127 (0.0009) [2023-12-27 00:25:54,567][105620] Updated weights for policy 1, policy_version 1243344 (0.0009) [2023-12-27 00:25:54,614][105620] Updated weights for policy 1, policy_version 1243354 (0.0007) [2023-12-27 00:25:55,226][105692] Updated weights for policy 0, policy_version 1242137 (0.0010) [2023-12-27 00:25:55,290][105692] Updated weights for policy 0, policy_version 1242147 (0.0007) [2023-12-27 00:25:55,355][105692] Updated weights for policy 0, policy_version 1242157 (0.0007) [2023-12-27 00:25:55,364][105620] Updated weights for policy 1, policy_version 1243364 (0.0009) [2023-12-27 00:25:55,421][105620] Updated weights for policy 1, policy_version 1243374 (0.0010) [2023-12-27 00:25:55,483][105620] Updated weights for policy 1, policy_version 1243384 (0.0010) [2023-12-27 00:25:55,912][105692] Updated weights for policy 0, policy_version 1242167 (0.0009) [2023-12-27 00:25:55,968][105692] Updated weights for policy 0, policy_version 1242177 (0.0010) [2023-12-27 00:25:56,027][105692] Updated weights for policy 0, policy_version 1242187 (0.0007) [2023-12-27 00:25:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 636403712. Throughput: 0: 9554.5, 1: 9942.7. Samples: 636409708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:25:56,063][104569] Avg episode reward: [(0, '9168.626'), (1, '9352.231')] [2023-12-27 00:25:56,161][105620] Updated weights for policy 1, policy_version 1243394 (0.0009) [2023-12-27 00:25:56,213][105620] Updated weights for policy 1, policy_version 1243404 (0.0005) [2023-12-27 00:25:56,272][105620] Updated weights for policy 1, policy_version 1243414 (0.0005) [2023-12-27 00:25:56,338][105620] Updated weights for policy 1, policy_version 1243424 (0.0008) [2023-12-27 00:25:56,618][105692] Updated weights for policy 0, policy_version 1242197 (0.0007) [2023-12-27 00:25:56,669][105692] Updated weights for policy 0, policy_version 1242207 (0.0005) [2023-12-27 00:25:56,722][105692] Updated weights for policy 0, policy_version 1242217 (0.0005) [2023-12-27 00:25:57,010][105620] Updated weights for policy 1, policy_version 1243434 (0.0010) [2023-12-27 00:25:57,061][105620] Updated weights for policy 1, policy_version 1243444 (0.0010) [2023-12-27 00:25:57,118][105620] Updated weights for policy 1, policy_version 1243454 (0.0010) [2023-12-27 00:25:57,245][105692] Updated weights for policy 0, policy_version 1242227 (0.0005) [2023-12-27 00:25:57,296][105692] Updated weights for policy 0, policy_version 1242237 (0.0005) [2023-12-27 00:25:57,356][105692] Updated weights for policy 0, policy_version 1242247 (0.0008) [2023-12-27 00:25:57,853][105620] Updated weights for policy 1, policy_version 1243464 (0.0007) [2023-12-27 00:25:57,909][105620] Updated weights for policy 1, policy_version 1243474 (0.0005) [2023-12-27 00:25:57,953][105620] Updated weights for policy 1, policy_version 1243484 (0.0005) [2023-12-27 00:25:58,044][105692] Updated weights for policy 0, policy_version 1242257 (0.0008) [2023-12-27 00:25:58,097][105692] Updated weights for policy 0, policy_version 1242267 (0.0006) [2023-12-27 00:25:58,171][105692] Updated weights for policy 0, policy_version 1242277 (0.0007) [2023-12-27 00:25:58,232][105692] Updated weights for policy 0, policy_version 1242287 (0.0007) [2023-12-27 00:25:58,633][105620] Updated weights for policy 1, policy_version 1243494 (0.0008) [2023-12-27 00:25:58,701][105620] Updated weights for policy 1, policy_version 1243504 (0.0010) [2023-12-27 00:25:58,772][105620] Updated weights for policy 1, policy_version 1243514 (0.0009) [2023-12-27 00:25:58,960][105692] Updated weights for policy 0, policy_version 1242297 (0.0008) [2023-12-27 00:25:59,025][105692] Updated weights for policy 0, policy_version 1242307 (0.0008) [2023-12-27 00:25:59,087][105692] Updated weights for policy 0, policy_version 1242317 (0.0009) [2023-12-27 00:25:59,531][105620] Updated weights for policy 1, policy_version 1243524 (0.0009) [2023-12-27 00:25:59,590][105620] Updated weights for policy 1, policy_version 1243534 (0.0010) [2023-12-27 00:25:59,641][105620] Updated weights for policy 1, policy_version 1243544 (0.0009) [2023-12-27 00:25:59,784][105692] Updated weights for policy 0, policy_version 1242327 (0.0008) [2023-12-27 00:25:59,842][105692] Updated weights for policy 0, policy_version 1242337 (0.0009) [2023-12-27 00:25:59,906][105692] Updated weights for policy 0, policy_version 1242347 (0.0008) [2023-12-27 00:26:00,428][105620] Updated weights for policy 1, policy_version 1243554 (0.0010) [2023-12-27 00:26:00,487][105620] Updated weights for policy 1, policy_version 1243564 (0.0011) [2023-12-27 00:26:00,539][105620] Updated weights for policy 1, policy_version 1243574 (0.0011) [2023-12-27 00:26:00,602][105620] Updated weights for policy 1, policy_version 1243584 (0.0011) [2023-12-27 00:26:00,627][105692] Updated weights for policy 0, policy_version 1242357 (0.0008) [2023-12-27 00:26:00,679][105692] Updated weights for policy 0, policy_version 1242367 (0.0008) [2023-12-27 00:26:00,735][105692] Updated weights for policy 0, policy_version 1242377 (0.0008) [2023-12-27 00:26:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 636502016. Throughput: 0: 9644.5, 1: 9930.2. Samples: 636472652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:26:01,063][104569] Avg episode reward: [(0, '9259.630'), (1, '9352.029')] [2023-12-27 00:26:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001242384_318103552.pth... [2023-12-27 00:26:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001243584_318398464.pth... [2023-12-27 00:26:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001242432_318103552.pth [2023-12-27 00:26:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001241264_317816832.pth [2023-12-27 00:26:01,264][105620] Updated weights for policy 1, policy_version 1243594 (0.0009) [2023-12-27 00:26:01,330][105620] Updated weights for policy 1, policy_version 1243604 (0.0011) [2023-12-27 00:26:01,398][105620] Updated weights for policy 1, policy_version 1243614 (0.0009) [2023-12-27 00:26:01,554][105692] Updated weights for policy 0, policy_version 1242387 (0.0009) [2023-12-27 00:26:01,604][105692] Updated weights for policy 0, policy_version 1242397 (0.0008) [2023-12-27 00:26:01,677][105692] Updated weights for policy 0, policy_version 1242407 (0.0009) [2023-12-27 00:26:02,096][105620] Updated weights for policy 1, policy_version 1243624 (0.0009) [2023-12-27 00:26:02,156][105620] Updated weights for policy 1, policy_version 1243634 (0.0010) [2023-12-27 00:26:02,208][105620] Updated weights for policy 1, policy_version 1243644 (0.0007) [2023-12-27 00:26:02,374][105692] Updated weights for policy 0, policy_version 1242417 (0.0009) [2023-12-27 00:26:02,425][105692] Updated weights for policy 0, policy_version 1242427 (0.0009) [2023-12-27 00:26:02,473][105692] Updated weights for policy 0, policy_version 1242437 (0.0009) [2023-12-27 00:26:02,535][105692] Updated weights for policy 0, policy_version 1242447 (0.0009) [2023-12-27 00:26:02,941][105620] Updated weights for policy 1, policy_version 1243654 (0.0006) [2023-12-27 00:26:03,002][105620] Updated weights for policy 1, policy_version 1243664 (0.0007) [2023-12-27 00:26:03,056][105620] Updated weights for policy 1, policy_version 1243674 (0.0007) [2023-12-27 00:26:03,330][105692] Updated weights for policy 0, policy_version 1242457 (0.0009) [2023-12-27 00:26:03,380][105692] Updated weights for policy 0, policy_version 1242467 (0.0009) [2023-12-27 00:26:03,437][105692] Updated weights for policy 0, policy_version 1242477 (0.0009) [2023-12-27 00:26:03,673][105620] Updated weights for policy 1, policy_version 1243684 (0.0009) [2023-12-27 00:26:03,732][105620] Updated weights for policy 1, policy_version 1243694 (0.0009) [2023-12-27 00:26:03,793][105620] Updated weights for policy 1, policy_version 1243704 (0.0008) [2023-12-27 00:26:04,278][105692] Updated weights for policy 0, policy_version 1242487 (0.0007) [2023-12-27 00:26:04,332][105692] Updated weights for policy 0, policy_version 1242497 (0.0007) [2023-12-27 00:26:04,393][105692] Updated weights for policy 0, policy_version 1242507 (0.0009) [2023-12-27 00:26:04,550][105620] Updated weights for policy 1, policy_version 1243714 (0.0009) [2023-12-27 00:26:04,616][105620] Updated weights for policy 1, policy_version 1243724 (0.0009) [2023-12-27 00:26:04,678][105620] Updated weights for policy 1, policy_version 1243734 (0.0008) [2023-12-27 00:26:04,741][105620] Updated weights for policy 1, policy_version 1243744 (0.0009) [2023-12-27 00:26:05,138][105692] Updated weights for policy 0, policy_version 1242517 (0.0009) [2023-12-27 00:26:05,186][105692] Updated weights for policy 0, policy_version 1242527 (0.0009) [2023-12-27 00:26:05,243][105692] Updated weights for policy 0, policy_version 1242537 (0.0009) [2023-12-27 00:26:05,489][105620] Updated weights for policy 1, policy_version 1243754 (0.0009) [2023-12-27 00:26:05,548][105620] Updated weights for policy 1, policy_version 1243764 (0.0009) [2023-12-27 00:26:05,614][105620] Updated weights for policy 1, policy_version 1243774 (0.0009) [2023-12-27 00:26:06,009][105692] Updated weights for policy 0, policy_version 1242547 (0.0009) [2023-12-27 00:26:06,062][105692] Updated weights for policy 0, policy_version 1242557 (0.0008) [2023-12-27 00:26:06,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 636592128. Throughput: 0: 9631.3, 1: 9879.2. Samples: 636586464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:26:06,062][104569] Avg episode reward: [(0, '9168.698'), (1, '9167.269')] [2023-12-27 00:26:06,107][105692] Updated weights for policy 0, policy_version 1242567 (0.0008) [2023-12-27 00:26:06,378][105620] Updated weights for policy 1, policy_version 1243784 (0.0009) [2023-12-27 00:26:06,440][105620] Updated weights for policy 1, policy_version 1243794 (0.0009) [2023-12-27 00:26:06,499][105620] Updated weights for policy 1, policy_version 1243804 (0.0009) [2023-12-27 00:26:06,911][105692] Updated weights for policy 0, policy_version 1242577 (0.0010) [2023-12-27 00:26:06,969][105692] Updated weights for policy 0, policy_version 1242587 (0.0006) [2023-12-27 00:26:07,036][105692] Updated weights for policy 0, policy_version 1242597 (0.0005) [2023-12-27 00:26:07,085][105692] Updated weights for policy 0, policy_version 1242607 (0.0006) [2023-12-27 00:26:07,133][105620] Updated weights for policy 1, policy_version 1243814 (0.0008) [2023-12-27 00:26:07,186][105620] Updated weights for policy 1, policy_version 1243824 (0.0009) [2023-12-27 00:26:07,247][105620] Updated weights for policy 1, policy_version 1243834 (0.0011) [2023-12-27 00:26:07,636][105692] Updated weights for policy 0, policy_version 1242617 (0.0010) [2023-12-27 00:26:07,689][105692] Updated weights for policy 0, policy_version 1242627 (0.0010) [2023-12-27 00:26:07,749][105692] Updated weights for policy 0, policy_version 1242637 (0.0010) [2023-12-27 00:26:07,997][105620] Updated weights for policy 1, policy_version 1243844 (0.0009) [2023-12-27 00:26:08,058][105620] Updated weights for policy 1, policy_version 1243854 (0.0006) [2023-12-27 00:26:08,109][105620] Updated weights for policy 1, policy_version 1243864 (0.0007) [2023-12-27 00:26:08,532][105692] Updated weights for policy 0, policy_version 1242647 (0.0010) [2023-12-27 00:26:08,593][105692] Updated weights for policy 0, policy_version 1242657 (0.0009) [2023-12-27 00:26:08,649][105692] Updated weights for policy 0, policy_version 1242667 (0.0009) [2023-12-27 00:26:08,752][105620] Updated weights for policy 1, policy_version 1243874 (0.0007) [2023-12-27 00:26:08,816][105620] Updated weights for policy 1, policy_version 1243884 (0.0005) [2023-12-27 00:26:08,873][105620] Updated weights for policy 1, policy_version 1243894 (0.0008) [2023-12-27 00:26:08,925][105620] Updated weights for policy 1, policy_version 1243904 (0.0009) [2023-12-27 00:26:09,313][105692] Updated weights for policy 0, policy_version 1242677 (0.0008) [2023-12-27 00:26:09,377][105692] Updated weights for policy 0, policy_version 1242687 (0.0009) [2023-12-27 00:26:09,439][105692] Updated weights for policy 0, policy_version 1242697 (0.0009) [2023-12-27 00:26:09,658][105620] Updated weights for policy 1, policy_version 1243914 (0.0009) [2023-12-27 00:26:09,706][105620] Updated weights for policy 1, policy_version 1243924 (0.0009) [2023-12-27 00:26:09,767][105620] Updated weights for policy 1, policy_version 1243934 (0.0008) [2023-12-27 00:26:10,217][105692] Updated weights for policy 0, policy_version 1242707 (0.0009) [2023-12-27 00:26:10,271][105692] Updated weights for policy 0, policy_version 1242717 (0.0007) [2023-12-27 00:26:10,324][105692] Updated weights for policy 0, policy_version 1242727 (0.0006) [2023-12-27 00:26:10,505][105620] Updated weights for policy 1, policy_version 1243944 (0.0009) [2023-12-27 00:26:10,568][105620] Updated weights for policy 1, policy_version 1243954 (0.0007) [2023-12-27 00:26:10,634][105620] Updated weights for policy 1, policy_version 1243964 (0.0005) [2023-12-27 00:26:10,945][105692] Updated weights for policy 0, policy_version 1242737 (0.0009) [2023-12-27 00:26:11,009][105692] Updated weights for policy 0, policy_version 1242747 (0.0006) [2023-12-27 00:26:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 636690432. Throughput: 0: 9682.5, 1: 9921.4. Samples: 636703968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:26:11,063][104569] Avg episode reward: [(0, '9075.978'), (1, '9076.398')] [2023-12-27 00:26:11,083][105692] Updated weights for policy 0, policy_version 1242757 (0.0010) [2023-12-27 00:26:11,144][105692] Updated weights for policy 0, policy_version 1242767 (0.0010) [2023-12-27 00:26:11,225][105620] Updated weights for policy 1, policy_version 1243974 (0.0006) [2023-12-27 00:26:11,288][105620] Updated weights for policy 1, policy_version 1243984 (0.0008) [2023-12-27 00:26:11,353][105620] Updated weights for policy 1, policy_version 1243994 (0.0008) [2023-12-27 00:26:11,894][105692] Updated weights for policy 0, policy_version 1242777 (0.0009) [2023-12-27 00:26:11,952][105692] Updated weights for policy 0, policy_version 1242787 (0.0009) [2023-12-27 00:26:11,994][105620] Updated weights for policy 1, policy_version 1244004 (0.0008) [2023-12-27 00:26:12,006][105692] Updated weights for policy 0, policy_version 1242797 (0.0009) [2023-12-27 00:26:12,048][105620] Updated weights for policy 1, policy_version 1244014 (0.0007) [2023-12-27 00:26:12,110][105620] Updated weights for policy 1, policy_version 1244024 (0.0009) [2023-12-27 00:26:12,830][105692] Updated weights for policy 0, policy_version 1242807 (0.0009) [2023-12-27 00:26:12,850][105620] Updated weights for policy 1, policy_version 1244034 (0.0009) [2023-12-27 00:26:12,883][105692] Updated weights for policy 0, policy_version 1242817 (0.0009) [2023-12-27 00:26:12,915][105620] Updated weights for policy 1, policy_version 1244044 (0.0006) [2023-12-27 00:26:12,933][105692] Updated weights for policy 0, policy_version 1242827 (0.0009) [2023-12-27 00:26:12,991][105620] Updated weights for policy 1, policy_version 1244054 (0.0006) [2023-12-27 00:26:13,046][105620] Updated weights for policy 1, policy_version 1244064 (0.0006) [2023-12-27 00:26:13,536][105620] Updated weights for policy 1, policy_version 1244074 (0.0005) [2023-12-27 00:26:13,588][105620] Updated weights for policy 1, policy_version 1244084 (0.0005) [2023-12-27 00:26:13,640][105620] Updated weights for policy 1, policy_version 1244094 (0.0010) [2023-12-27 00:26:13,835][105692] Updated weights for policy 0, policy_version 1242837 (0.0010) [2023-12-27 00:26:13,893][105692] Updated weights for policy 0, policy_version 1242847 (0.0011) [2023-12-27 00:26:13,944][105692] Updated weights for policy 0, policy_version 1242857 (0.0010) [2023-12-27 00:26:14,231][105620] Updated weights for policy 1, policy_version 1244104 (0.0007) [2023-12-27 00:26:14,282][105620] Updated weights for policy 1, policy_version 1244114 (0.0005) [2023-12-27 00:26:14,337][105620] Updated weights for policy 1, policy_version 1244124 (0.0006) [2023-12-27 00:26:14,682][105692] Updated weights for policy 0, policy_version 1242867 (0.0010) [2023-12-27 00:26:14,746][105692] Updated weights for policy 0, policy_version 1242877 (0.0010) [2023-12-27 00:26:14,807][105692] Updated weights for policy 0, policy_version 1242887 (0.0007) [2023-12-27 00:26:14,911][105620] Updated weights for policy 1, policy_version 1244134 (0.0007) [2023-12-27 00:26:14,970][105620] Updated weights for policy 1, policy_version 1244144 (0.0009) [2023-12-27 00:26:15,028][105620] Updated weights for policy 1, policy_version 1244154 (0.0009) [2023-12-27 00:26:15,519][105692] Updated weights for policy 0, policy_version 1242897 (0.0005) [2023-12-27 00:26:15,588][105692] Updated weights for policy 0, policy_version 1242907 (0.0006) [2023-12-27 00:26:15,656][105692] Updated weights for policy 0, policy_version 1242917 (0.0007) [2023-12-27 00:26:15,679][105620] Updated weights for policy 1, policy_version 1244164 (0.0008) [2023-12-27 00:26:15,713][105692] Updated weights for policy 0, policy_version 1242927 (0.0008) [2023-12-27 00:26:15,735][105620] Updated weights for policy 1, policy_version 1244174 (0.0008) [2023-12-27 00:26:15,798][105620] Updated weights for policy 1, policy_version 1244184 (0.0009) [2023-12-27 00:26:16,062][104569] Fps is (10 sec: 20479.3, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 636796928. Throughput: 0: 9559.2, 1: 9948.0. Samples: 636763404. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:26:16,063][104569] Avg episode reward: [(0, '9169.242'), (1, '9167.812')] [2023-12-27 00:26:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001244192_318554112.pth... [2023-12-27 00:26:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001242928_318242816.pth... [2023-12-27 00:26:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001243008_318251008.pth [2023-12-27 00:26:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001241808_317956096.pth [2023-12-27 00:26:16,431][105692] Updated weights for policy 0, policy_version 1242937 (0.0006) [2023-12-27 00:26:16,477][105692] Updated weights for policy 0, policy_version 1242947 (0.0005) [2023-12-27 00:26:16,491][105620] Updated weights for policy 1, policy_version 1244194 (0.0006) [2023-12-27 00:26:16,532][105692] Updated weights for policy 0, policy_version 1242957 (0.0008) [2023-12-27 00:26:16,543][105620] Updated weights for policy 1, policy_version 1244204 (0.0006) [2023-12-27 00:26:16,596][105620] Updated weights for policy 1, policy_version 1244214 (0.0009) [2023-12-27 00:26:16,650][105620] Updated weights for policy 1, policy_version 1244224 (0.0008) [2023-12-27 00:26:17,155][105692] Updated weights for policy 0, policy_version 1242967 (0.0006) [2023-12-27 00:26:17,203][105692] Updated weights for policy 0, policy_version 1242977 (0.0005) [2023-12-27 00:26:17,260][105692] Updated weights for policy 0, policy_version 1242987 (0.0007) [2023-12-27 00:26:17,499][105620] Updated weights for policy 1, policy_version 1244234 (0.0006) [2023-12-27 00:26:17,559][105620] Updated weights for policy 1, policy_version 1244244 (0.0005) [2023-12-27 00:26:17,620][105620] Updated weights for policy 1, policy_version 1244254 (0.0005) [2023-12-27 00:26:17,992][105692] Updated weights for policy 0, policy_version 1242997 (0.0007) [2023-12-27 00:26:18,046][105692] Updated weights for policy 0, policy_version 1243007 (0.0009) [2023-12-27 00:26:18,107][105692] Updated weights for policy 0, policy_version 1243017 (0.0009) [2023-12-27 00:26:18,134][105620] Updated weights for policy 1, policy_version 1244264 (0.0009) [2023-12-27 00:26:18,191][105620] Updated weights for policy 1, policy_version 1244274 (0.0009) [2023-12-27 00:26:18,248][105620] Updated weights for policy 1, policy_version 1244284 (0.0009) [2023-12-27 00:26:18,797][105692] Updated weights for policy 0, policy_version 1243027 (0.0010) [2023-12-27 00:26:18,856][105692] Updated weights for policy 0, policy_version 1243037 (0.0009) [2023-12-27 00:26:18,915][105692] Updated weights for policy 0, policy_version 1243047 (0.0009) [2023-12-27 00:26:19,014][105620] Updated weights for policy 1, policy_version 1244294 (0.0010) [2023-12-27 00:26:19,073][105620] Updated weights for policy 1, policy_version 1244304 (0.0009) [2023-12-27 00:26:19,136][105620] Updated weights for policy 1, policy_version 1244314 (0.0009) [2023-12-27 00:26:19,632][105692] Updated weights for policy 0, policy_version 1243057 (0.0009) [2023-12-27 00:26:19,694][105692] Updated weights for policy 0, policy_version 1243067 (0.0008) [2023-12-27 00:26:19,751][105692] Updated weights for policy 0, policy_version 1243077 (0.0009) [2023-12-27 00:26:19,816][105692] Updated weights for policy 0, policy_version 1243087 (0.0009) [2023-12-27 00:26:19,877][105620] Updated weights for policy 1, policy_version 1244324 (0.0009) [2023-12-27 00:26:19,942][105620] Updated weights for policy 1, policy_version 1244334 (0.0009) [2023-12-27 00:26:20,010][105620] Updated weights for policy 1, policy_version 1244344 (0.0010) [2023-12-27 00:26:20,585][105692] Updated weights for policy 0, policy_version 1243097 (0.0007) [2023-12-27 00:26:20,652][105692] Updated weights for policy 0, policy_version 1243107 (0.0010) [2023-12-27 00:26:20,715][105692] Updated weights for policy 0, policy_version 1243117 (0.0008) [2023-12-27 00:26:20,834][105620] Updated weights for policy 1, policy_version 1244354 (0.0010) [2023-12-27 00:26:20,899][105620] Updated weights for policy 1, policy_version 1244364 (0.0009) [2023-12-27 00:26:20,966][105620] Updated weights for policy 1, policy_version 1244374 (0.0010) [2023-12-27 00:26:21,036][105620] Updated weights for policy 1, policy_version 1244384 (0.0010) [2023-12-27 00:26:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 636895232. Throughput: 0: 9531.5, 1: 9990.8. Samples: 636882772. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:26:21,062][104569] Avg episode reward: [(0, '8899.335'), (1, '9259.312')] [2023-12-27 00:26:21,327][105692] Updated weights for policy 0, policy_version 1243127 (0.0006) [2023-12-27 00:26:21,398][105692] Updated weights for policy 0, policy_version 1243137 (0.0008) [2023-12-27 00:26:21,462][105692] Updated weights for policy 0, policy_version 1243147 (0.0010) [2023-12-27 00:26:21,802][105620] Updated weights for policy 1, policy_version 1244394 (0.0006) [2023-12-27 00:26:21,855][105620] Updated weights for policy 1, policy_version 1244404 (0.0005) [2023-12-27 00:26:21,917][105620] Updated weights for policy 1, policy_version 1244414 (0.0006) [2023-12-27 00:26:22,291][105692] Updated weights for policy 0, policy_version 1243157 (0.0010) [2023-12-27 00:26:22,368][105692] Updated weights for policy 0, policy_version 1243168 (0.0008) [2023-12-27 00:26:22,438][105692] Updated weights for policy 0, policy_version 1243178 (0.0009) [2023-12-27 00:26:22,546][105620] Updated weights for policy 1, policy_version 1244424 (0.0008) [2023-12-27 00:26:22,605][105620] Updated weights for policy 1, policy_version 1244434 (0.0007) [2023-12-27 00:26:22,662][105620] Updated weights for policy 1, policy_version 1244444 (0.0006) [2023-12-27 00:26:23,218][105692] Updated weights for policy 0, policy_version 1243188 (0.0010) [2023-12-27 00:26:23,268][105692] Updated weights for policy 0, policy_version 1243198 (0.0008) [2023-12-27 00:26:23,278][105620] Updated weights for policy 1, policy_version 1244454 (0.0006) [2023-12-27 00:26:23,322][105620] Updated weights for policy 1, policy_version 1244464 (0.0006) [2023-12-27 00:26:23,325][105692] Updated weights for policy 0, policy_version 1243208 (0.0008) [2023-12-27 00:26:23,376][105620] Updated weights for policy 1, policy_version 1244474 (0.0005) [2023-12-27 00:26:24,031][105620] Updated weights for policy 1, policy_version 1244484 (0.0006) [2023-12-27 00:26:24,089][105620] Updated weights for policy 1, policy_version 1244494 (0.0009) [2023-12-27 00:26:24,130][105692] Updated weights for policy 0, policy_version 1243218 (0.0009) [2023-12-27 00:26:24,144][105620] Updated weights for policy 1, policy_version 1244504 (0.0008) [2023-12-27 00:26:24,184][105692] Updated weights for policy 0, policy_version 1243228 (0.0007) [2023-12-27 00:26:24,240][105692] Updated weights for policy 0, policy_version 1243238 (0.0009) [2023-12-27 00:26:24,299][105692] Updated weights for policy 0, policy_version 1243248 (0.0009) [2023-12-27 00:26:24,853][105620] Updated weights for policy 1, policy_version 1244514 (0.0008) [2023-12-27 00:26:24,917][105620] Updated weights for policy 1, policy_version 1244524 (0.0011) [2023-12-27 00:26:24,970][105620] Updated weights for policy 1, policy_version 1244534 (0.0011) [2023-12-27 00:26:24,988][105692] Updated weights for policy 0, policy_version 1243258 (0.0006) [2023-12-27 00:26:25,021][105585] KL-divergence is very high: 135.1052 [2023-12-27 00:26:25,026][105620] Updated weights for policy 1, policy_version 1244544 (0.0011) [2023-12-27 00:26:25,045][105692] Updated weights for policy 0, policy_version 1243268 (0.0007) [2023-12-27 00:26:25,066][105585] KL-divergence is very high: 224.7000 [2023-12-27 00:26:25,078][105585] KL-divergence is very high: 133.0703 [2023-12-27 00:26:25,103][105692] Updated weights for policy 0, policy_version 1243278 (0.0008) [2023-12-27 00:26:25,743][105620] Updated weights for policy 1, policy_version 1244554 (0.0008) [2023-12-27 00:26:25,789][105620] Updated weights for policy 1, policy_version 1244564 (0.0008) [2023-12-27 00:26:25,833][105692] Updated weights for policy 0, policy_version 1243288 (0.0009) [2023-12-27 00:26:25,848][105620] Updated weights for policy 1, policy_version 1244574 (0.0008) [2023-12-27 00:26:25,884][105692] Updated weights for policy 0, policy_version 1243298 (0.0008) [2023-12-27 00:26:25,935][105692] Updated weights for policy 0, policy_version 1243308 (0.0008) [2023-12-27 00:26:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 636993536. Throughput: 0: 9482.7, 1: 9992.2. Samples: 636997696. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:26:26,063][104569] Avg episode reward: [(0, '8896.973'), (1, '9078.914')] [2023-12-27 00:26:26,449][105620] Updated weights for policy 1, policy_version 1244584 (0.0008) [2023-12-27 00:26:26,503][105620] Updated weights for policy 1, policy_version 1244594 (0.0009) [2023-12-27 00:26:26,558][105620] Updated weights for policy 1, policy_version 1244604 (0.0008) [2023-12-27 00:26:26,704][105692] Updated weights for policy 0, policy_version 1243318 (0.0006) [2023-12-27 00:26:26,761][105692] Updated weights for policy 0, policy_version 1243328 (0.0005) [2023-12-27 00:26:26,825][105692] Updated weights for policy 0, policy_version 1243338 (0.0006) [2023-12-27 00:26:27,322][105620] Updated weights for policy 1, policy_version 1244614 (0.0007) [2023-12-27 00:26:27,356][105692] Updated weights for policy 0, policy_version 1243348 (0.0005) [2023-12-27 00:26:27,387][105620] Updated weights for policy 1, policy_version 1244624 (0.0008) [2023-12-27 00:26:27,406][105692] Updated weights for policy 0, policy_version 1243358 (0.0005) [2023-12-27 00:26:27,425][105585] KL-divergence is very high: 101.4894 [2023-12-27 00:26:27,450][105620] Updated weights for policy 1, policy_version 1244634 (0.0008) [2023-12-27 00:26:27,457][105692] Updated weights for policy 0, policy_version 1243368 (0.0005) [2023-12-27 00:26:28,146][105692] Updated weights for policy 0, policy_version 1243378 (0.0006) [2023-12-27 00:26:28,182][105620] Updated weights for policy 1, policy_version 1244644 (0.0009) [2023-12-27 00:26:28,202][105692] Updated weights for policy 0, policy_version 1243388 (0.0009) [2023-12-27 00:26:28,239][105620] Updated weights for policy 1, policy_version 1244654 (0.0008) [2023-12-27 00:26:28,253][105692] Updated weights for policy 0, policy_version 1243398 (0.0008) [2023-12-27 00:26:28,291][105620] Updated weights for policy 1, policy_version 1244664 (0.0008) [2023-12-27 00:26:28,301][105692] Updated weights for policy 0, policy_version 1243408 (0.0006) [2023-12-27 00:26:29,025][105620] Updated weights for policy 1, policy_version 1244674 (0.0008) [2023-12-27 00:26:29,034][105692] Updated weights for policy 0, policy_version 1243418 (0.0009) [2023-12-27 00:26:29,079][105692] Updated weights for policy 0, policy_version 1243428 (0.0006) [2023-12-27 00:26:29,080][105620] Updated weights for policy 1, policy_version 1244684 (0.0008) [2023-12-27 00:26:29,131][105692] Updated weights for policy 0, policy_version 1243438 (0.0007) [2023-12-27 00:26:29,133][105620] Updated weights for policy 1, policy_version 1244694 (0.0006) [2023-12-27 00:26:29,191][105620] Updated weights for policy 1, policy_version 1244704 (0.0008) [2023-12-27 00:26:29,896][105620] Updated weights for policy 1, policy_version 1244714 (0.0008) [2023-12-27 00:26:29,946][105692] Updated weights for policy 0, policy_version 1243448 (0.0008) [2023-12-27 00:26:29,955][105620] Updated weights for policy 1, policy_version 1244724 (0.0006) [2023-12-27 00:26:30,001][105692] Updated weights for policy 0, policy_version 1243458 (0.0009) [2023-12-27 00:26:30,005][105620] Updated weights for policy 1, policy_version 1244734 (0.0005) [2023-12-27 00:26:30,052][105692] Updated weights for policy 0, policy_version 1243468 (0.0009) [2023-12-27 00:26:30,714][105620] Updated weights for policy 1, policy_version 1244744 (0.0008) [2023-12-27 00:26:30,758][105692] Updated weights for policy 0, policy_version 1243478 (0.0008) [2023-12-27 00:26:30,777][105620] Updated weights for policy 1, policy_version 1244754 (0.0008) [2023-12-27 00:26:30,816][105692] Updated weights for policy 0, policy_version 1243488 (0.0005) [2023-12-27 00:26:30,833][105620] Updated weights for policy 1, policy_version 1244764 (0.0009) [2023-12-27 00:26:30,879][105692] Updated weights for policy 0, policy_version 1243498 (0.0005) [2023-12-27 00:26:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 637091840. Throughput: 0: 9552.6, 1: 10012.0. Samples: 637058544. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:26:31,062][104569] Avg episode reward: [(0, '8893.650'), (1, '9171.140')] [2023-12-27 00:26:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001243504_318390272.pth... [2023-12-27 00:26:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001244768_318701568.pth... [2023-12-27 00:26:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001243584_318398464.pth [2023-12-27 00:26:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001242384_318103552.pth [2023-12-27 00:26:31,480][105692] Updated weights for policy 0, policy_version 1243508 (0.0007) [2023-12-27 00:26:31,539][105692] Updated weights for policy 0, policy_version 1243518 (0.0008) [2023-12-27 00:26:31,565][105620] Updated weights for policy 1, policy_version 1244775 (0.0010) [2023-12-27 00:26:31,586][105692] Updated weights for policy 0, policy_version 1243528 (0.0007) [2023-12-27 00:26:31,626][105620] Updated weights for policy 1, policy_version 1244785 (0.0009) [2023-12-27 00:26:31,685][105620] Updated weights for policy 1, policy_version 1244795 (0.0011) [2023-12-27 00:26:32,368][105692] Updated weights for policy 0, policy_version 1243538 (0.0007) [2023-12-27 00:26:32,416][105692] Updated weights for policy 0, policy_version 1243548 (0.0007) [2023-12-27 00:26:32,425][105620] Updated weights for policy 1, policy_version 1244805 (0.0010) [2023-12-27 00:26:32,467][105692] Updated weights for policy 0, policy_version 1243558 (0.0008) [2023-12-27 00:26:32,490][105620] Updated weights for policy 1, policy_version 1244815 (0.0010) [2023-12-27 00:26:32,531][105692] Updated weights for policy 0, policy_version 1243568 (0.0007) [2023-12-27 00:26:32,545][105620] Updated weights for policy 1, policy_version 1244825 (0.0010) [2023-12-27 00:26:33,156][105692] Updated weights for policy 0, policy_version 1243578 (0.0010) [2023-12-27 00:26:33,207][105692] Updated weights for policy 0, policy_version 1243588 (0.0010) [2023-12-27 00:26:33,256][105692] Updated weights for policy 0, policy_version 1243598 (0.0007) [2023-12-27 00:26:33,271][105620] Updated weights for policy 1, policy_version 1244835 (0.0010) [2023-12-27 00:26:33,328][105620] Updated weights for policy 1, policy_version 1244845 (0.0010) [2023-12-27 00:26:33,373][105620] Updated weights for policy 1, policy_version 1244855 (0.0009) [2023-12-27 00:26:33,837][105692] Updated weights for policy 0, policy_version 1243608 (0.0005) [2023-12-27 00:26:33,885][105692] Updated weights for policy 0, policy_version 1243618 (0.0005) [2023-12-27 00:26:33,930][105692] Updated weights for policy 0, policy_version 1243628 (0.0005) [2023-12-27 00:26:34,248][105620] Updated weights for policy 1, policy_version 1244865 (0.0009) [2023-12-27 00:26:34,310][105620] Updated weights for policy 1, policy_version 1244875 (0.0009) [2023-12-27 00:26:34,371][105620] Updated weights for policy 1, policy_version 1244885 (0.0006) [2023-12-27 00:26:34,436][105620] Updated weights for policy 1, policy_version 1244895 (0.0006) [2023-12-27 00:26:34,518][105692] Updated weights for policy 0, policy_version 1243638 (0.0008) [2023-12-27 00:26:34,571][105692] Updated weights for policy 0, policy_version 1243648 (0.0007) [2023-12-27 00:26:34,638][105692] Updated weights for policy 0, policy_version 1243658 (0.0008) [2023-12-27 00:26:35,164][105620] Updated weights for policy 1, policy_version 1244905 (0.0008) [2023-12-27 00:26:35,216][105620] Updated weights for policy 1, policy_version 1244915 (0.0007) [2023-12-27 00:26:35,271][105620] Updated weights for policy 1, policy_version 1244925 (0.0008) [2023-12-27 00:26:35,366][105692] Updated weights for policy 0, policy_version 1243668 (0.0010) [2023-12-27 00:26:35,418][105692] Updated weights for policy 0, policy_version 1243678 (0.0011) [2023-12-27 00:26:35,471][105692] Updated weights for policy 0, policy_version 1243688 (0.0011) [2023-12-27 00:26:36,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 637181952. Throughput: 0: 9740.6, 1: 9939.9. Samples: 637176720. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:26:36,063][104569] Avg episode reward: [(0, '8894.341'), (1, '9169.553')] [2023-12-27 00:26:36,106][105620] Updated weights for policy 1, policy_version 1244935 (0.0009) [2023-12-27 00:26:36,125][105692] Updated weights for policy 0, policy_version 1243698 (0.0010) [2023-12-27 00:26:36,167][105620] Updated weights for policy 1, policy_version 1244945 (0.0008) [2023-12-27 00:26:36,185][105692] Updated weights for policy 0, policy_version 1243708 (0.0007) [2023-12-27 00:26:36,229][105620] Updated weights for policy 1, policy_version 1244955 (0.0006) [2023-12-27 00:26:36,237][105692] Updated weights for policy 0, policy_version 1243718 (0.0009) [2023-12-27 00:26:36,305][105692] Updated weights for policy 0, policy_version 1243728 (0.0007) [2023-12-27 00:26:36,961][105620] Updated weights for policy 1, policy_version 1244965 (0.0008) [2023-12-27 00:26:36,967][105692] Updated weights for policy 0, policy_version 1243738 (0.0010) [2023-12-27 00:26:37,016][105620] Updated weights for policy 1, policy_version 1244975 (0.0010) [2023-12-27 00:26:37,026][105692] Updated weights for policy 0, policy_version 1243748 (0.0011) [2023-12-27 00:26:37,073][105620] Updated weights for policy 1, policy_version 1244985 (0.0009) [2023-12-27 00:26:37,088][105692] Updated weights for policy 0, policy_version 1243758 (0.0011) [2023-12-27 00:26:37,742][105620] Updated weights for policy 1, policy_version 1244995 (0.0006) [2023-12-27 00:26:37,760][105692] Updated weights for policy 0, policy_version 1243768 (0.0011) [2023-12-27 00:26:37,794][105620] Updated weights for policy 1, policy_version 1245005 (0.0005) [2023-12-27 00:26:37,815][105692] Updated weights for policy 0, policy_version 1243778 (0.0010) [2023-12-27 00:26:37,858][105620] Updated weights for policy 1, policy_version 1245015 (0.0006) [2023-12-27 00:26:37,868][105692] Updated weights for policy 0, policy_version 1243788 (0.0010) [2023-12-27 00:26:38,576][105692] Updated weights for policy 0, policy_version 1243798 (0.0011) [2023-12-27 00:26:38,638][105692] Updated weights for policy 0, policy_version 1243808 (0.0011) [2023-12-27 00:26:38,645][105620] Updated weights for policy 1, policy_version 1245025 (0.0009) [2023-12-27 00:26:38,697][105692] Updated weights for policy 0, policy_version 1243818 (0.0011) [2023-12-27 00:26:38,707][105620] Updated weights for policy 1, policy_version 1245035 (0.0006) [2023-12-27 00:26:38,757][105620] Updated weights for policy 1, policy_version 1245045 (0.0007) [2023-12-27 00:26:38,812][105620] Updated weights for policy 1, policy_version 1245055 (0.0008) [2023-12-27 00:26:39,324][105692] Updated weights for policy 0, policy_version 1243828 (0.0011) [2023-12-27 00:26:39,391][105692] Updated weights for policy 0, policy_version 1243838 (0.0009) [2023-12-27 00:26:39,458][105692] Updated weights for policy 0, policy_version 1243848 (0.0009) [2023-12-27 00:26:39,640][105620] Updated weights for policy 1, policy_version 1245065 (0.0009) [2023-12-27 00:26:39,705][105620] Updated weights for policy 1, policy_version 1245075 (0.0008) [2023-12-27 00:26:39,762][105620] Updated weights for policy 1, policy_version 1245085 (0.0008) [2023-12-27 00:26:40,241][105692] Updated weights for policy 0, policy_version 1243858 (0.0010) [2023-12-27 00:26:40,301][105692] Updated weights for policy 0, policy_version 1243868 (0.0010) [2023-12-27 00:26:40,355][105692] Updated weights for policy 0, policy_version 1243878 (0.0007) [2023-12-27 00:26:40,410][105692] Updated weights for policy 0, policy_version 1243888 (0.0009) [2023-12-27 00:26:40,423][105620] Updated weights for policy 1, policy_version 1245095 (0.0007) [2023-12-27 00:26:40,489][105620] Updated weights for policy 1, policy_version 1245105 (0.0008) [2023-12-27 00:26:40,551][105620] Updated weights for policy 1, policy_version 1245115 (0.0009) [2023-12-27 00:26:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 637280256. Throughput: 0: 9794.3, 1: 9812.8. Samples: 637292028. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:26:41,062][104569] Avg episode reward: [(0, '9081.059'), (1, '9169.784')] [2023-12-27 00:26:41,191][105692] Updated weights for policy 0, policy_version 1243898 (0.0010) [2023-12-27 00:26:41,257][105692] Updated weights for policy 0, policy_version 1243908 (0.0009) [2023-12-27 00:26:41,296][105620] Updated weights for policy 1, policy_version 1245125 (0.0008) [2023-12-27 00:26:41,310][105692] Updated weights for policy 0, policy_version 1243918 (0.0007) [2023-12-27 00:26:41,359][105620] Updated weights for policy 1, policy_version 1245135 (0.0007) [2023-12-27 00:26:41,418][105620] Updated weights for policy 1, policy_version 1245145 (0.0009) [2023-12-27 00:26:42,032][105692] Updated weights for policy 0, policy_version 1243928 (0.0008) [2023-12-27 00:26:42,091][105692] Updated weights for policy 0, policy_version 1243938 (0.0009) [2023-12-27 00:26:42,150][105692] Updated weights for policy 0, policy_version 1243948 (0.0009) [2023-12-27 00:26:42,186][105620] Updated weights for policy 1, policy_version 1245155 (0.0009) [2023-12-27 00:26:42,243][105620] Updated weights for policy 1, policy_version 1245165 (0.0007) [2023-12-27 00:26:42,310][105620] Updated weights for policy 1, policy_version 1245175 (0.0009) [2023-12-27 00:26:42,831][105692] Updated weights for policy 0, policy_version 1243958 (0.0006) [2023-12-27 00:26:42,899][105692] Updated weights for policy 0, policy_version 1243968 (0.0005) [2023-12-27 00:26:42,959][105692] Updated weights for policy 0, policy_version 1243978 (0.0005) [2023-12-27 00:26:43,136][105620] Updated weights for policy 1, policy_version 1245185 (0.0008) [2023-12-27 00:26:43,195][105620] Updated weights for policy 1, policy_version 1245196 (0.0011) [2023-12-27 00:26:43,259][105620] Updated weights for policy 1, policy_version 1245206 (0.0011) [2023-12-27 00:26:43,448][105692] Updated weights for policy 0, policy_version 1243988 (0.0006) [2023-12-27 00:26:43,498][105692] Updated weights for policy 0, policy_version 1243998 (0.0005) [2023-12-27 00:26:43,549][105692] Updated weights for policy 0, policy_version 1244008 (0.0009) [2023-12-27 00:26:44,007][105620] Updated weights for policy 1, policy_version 1245217 (0.0010) [2023-12-27 00:26:44,058][105620] Updated weights for policy 1, policy_version 1245227 (0.0006) [2023-12-27 00:26:44,107][105620] Updated weights for policy 1, policy_version 1245237 (0.0006) [2023-12-27 00:26:44,167][105620] Updated weights for policy 1, policy_version 1245247 (0.0006) [2023-12-27 00:26:44,302][105692] Updated weights for policy 0, policy_version 1244018 (0.0009) [2023-12-27 00:26:44,362][105692] Updated weights for policy 0, policy_version 1244028 (0.0009) [2023-12-27 00:26:44,416][105692] Updated weights for policy 0, policy_version 1244038 (0.0010) [2023-12-27 00:26:44,476][105692] Updated weights for policy 0, policy_version 1244048 (0.0009) [2023-12-27 00:26:44,781][105620] Updated weights for policy 1, policy_version 1245257 (0.0008) [2023-12-27 00:26:44,846][105620] Updated weights for policy 1, policy_version 1245267 (0.0007) [2023-12-27 00:26:44,909][105620] Updated weights for policy 1, policy_version 1245277 (0.0009) [2023-12-27 00:26:45,214][105692] Updated weights for policy 0, policy_version 1244058 (0.0009) [2023-12-27 00:26:45,270][105692] Updated weights for policy 0, policy_version 1244068 (0.0009) [2023-12-27 00:26:45,327][105692] Updated weights for policy 0, policy_version 1244078 (0.0009) [2023-12-27 00:26:45,691][105620] Updated weights for policy 1, policy_version 1245287 (0.0009) [2023-12-27 00:26:45,749][105620] Updated weights for policy 1, policy_version 1245297 (0.0009) [2023-12-27 00:26:45,814][105620] Updated weights for policy 1, policy_version 1245307 (0.0009) [2023-12-27 00:26:46,053][105692] Updated weights for policy 0, policy_version 1244088 (0.0008) [2023-12-27 00:26:46,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 637378560. Throughput: 0: 9743.7, 1: 9751.0. Samples: 637349912. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:26:46,062][104569] Avg episode reward: [(0, '9266.787'), (1, '9169.950')] [2023-12-27 00:26:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001245312_318840832.pth... [2023-12-27 00:26:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001244192_318554112.pth [2023-12-27 00:26:46,111][105692] Updated weights for policy 0, policy_version 1244098 (0.0005) [2023-12-27 00:26:46,174][105692] Updated weights for policy 0, policy_version 1244108 (0.0005) [2023-12-27 00:26:46,200][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001244112_318545920.pth... [2023-12-27 00:26:46,205][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001242928_318242816.pth [2023-12-27 00:26:46,612][105620] Updated weights for policy 1, policy_version 1245317 (0.0009) [2023-12-27 00:26:46,671][105620] Updated weights for policy 1, policy_version 1245327 (0.0009) [2023-12-27 00:26:46,716][105620] Updated weights for policy 1, policy_version 1245337 (0.0008) [2023-12-27 00:26:46,833][105692] Updated weights for policy 0, policy_version 1244118 (0.0007) [2023-12-27 00:26:46,883][105692] Updated weights for policy 0, policy_version 1244128 (0.0009) [2023-12-27 00:26:46,941][105692] Updated weights for policy 0, policy_version 1244138 (0.0009) [2023-12-27 00:26:47,478][105620] Updated weights for policy 1, policy_version 1245347 (0.0006) [2023-12-27 00:26:47,532][105620] Updated weights for policy 1, policy_version 1245357 (0.0009) [2023-12-27 00:26:47,591][105620] Updated weights for policy 1, policy_version 1245367 (0.0009) [2023-12-27 00:26:47,686][105692] Updated weights for policy 0, policy_version 1244148 (0.0009) [2023-12-27 00:26:47,737][105692] Updated weights for policy 0, policy_version 1244158 (0.0008) [2023-12-27 00:26:47,783][105692] Updated weights for policy 0, policy_version 1244168 (0.0008) [2023-12-27 00:26:48,259][105620] Updated weights for policy 1, policy_version 1245377 (0.0009) [2023-12-27 00:26:48,311][105620] Updated weights for policy 1, policy_version 1245387 (0.0009) [2023-12-27 00:26:48,389][105620] Updated weights for policy 1, policy_version 1245397 (0.0009) [2023-12-27 00:26:48,449][105620] Updated weights for policy 1, policy_version 1245407 (0.0006) [2023-12-27 00:26:48,620][105692] Updated weights for policy 0, policy_version 1244178 (0.0010) [2023-12-27 00:26:48,681][105692] Updated weights for policy 0, policy_version 1244188 (0.0009) [2023-12-27 00:26:48,749][105692] Updated weights for policy 0, policy_version 1244198 (0.0009) [2023-12-27 00:26:48,804][105692] Updated weights for policy 0, policy_version 1244208 (0.0009) [2023-12-27 00:26:49,063][105620] Updated weights for policy 1, policy_version 1245417 (0.0007) [2023-12-27 00:26:49,122][105620] Updated weights for policy 1, policy_version 1245427 (0.0009) [2023-12-27 00:26:49,181][105620] Updated weights for policy 1, policy_version 1245437 (0.0009) [2023-12-27 00:26:49,610][105692] Updated weights for policy 0, policy_version 1244218 (0.0010) [2023-12-27 00:26:49,668][105692] Updated weights for policy 0, policy_version 1244228 (0.0006) [2023-12-27 00:26:49,727][105692] Updated weights for policy 0, policy_version 1244238 (0.0005) [2023-12-27 00:26:49,870][105620] Updated weights for policy 1, policy_version 1245447 (0.0008) [2023-12-27 00:26:49,941][105620] Updated weights for policy 1, policy_version 1245457 (0.0007) [2023-12-27 00:26:50,008][105620] Updated weights for policy 1, policy_version 1245467 (0.0008) [2023-12-27 00:26:50,496][105692] Updated weights for policy 0, policy_version 1244248 (0.0008) [2023-12-27 00:26:50,556][105692] Updated weights for policy 0, policy_version 1244258 (0.0009) [2023-12-27 00:26:50,620][105692] Updated weights for policy 0, policy_version 1244268 (0.0009) [2023-12-27 00:26:50,622][105620] Updated weights for policy 1, policy_version 1245477 (0.0007) [2023-12-27 00:26:50,676][105620] Updated weights for policy 1, policy_version 1245487 (0.0008) [2023-12-27 00:26:50,728][105620] Updated weights for policy 1, policy_version 1245497 (0.0009) [2023-12-27 00:26:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 637476864. Throughput: 0: 9751.1, 1: 9776.4. Samples: 637465204. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:26:51,063][104569] Avg episode reward: [(0, '9267.105'), (1, '9079.011')] [2023-12-27 00:26:51,395][105692] Updated weights for policy 0, policy_version 1244278 (0.0008) [2023-12-27 00:26:51,461][105620] Updated weights for policy 1, policy_version 1245507 (0.0008) [2023-12-27 00:26:51,463][105692] Updated weights for policy 0, policy_version 1244288 (0.0007) [2023-12-27 00:26:51,518][105692] Updated weights for policy 0, policy_version 1244298 (0.0007) [2023-12-27 00:26:51,521][105620] Updated weights for policy 1, policy_version 1245517 (0.0005) [2023-12-27 00:26:51,583][105620] Updated weights for policy 1, policy_version 1245527 (0.0008) [2023-12-27 00:26:52,301][105692] Updated weights for policy 0, policy_version 1244308 (0.0007) [2023-12-27 00:26:52,309][105620] Updated weights for policy 1, policy_version 1245537 (0.0009) [2023-12-27 00:26:52,363][105692] Updated weights for policy 0, policy_version 1244318 (0.0009) [2023-12-27 00:26:52,366][105620] Updated weights for policy 1, policy_version 1245547 (0.0006) [2023-12-27 00:26:52,422][105692] Updated weights for policy 0, policy_version 1244328 (0.0007) [2023-12-27 00:26:52,424][105620] Updated weights for policy 1, policy_version 1245557 (0.0009) [2023-12-27 00:26:52,477][105620] Updated weights for policy 1, policy_version 1245567 (0.0006) [2023-12-27 00:26:53,200][105692] Updated weights for policy 0, policy_version 1244338 (0.0007) [2023-12-27 00:26:53,209][105620] Updated weights for policy 1, policy_version 1245577 (0.0008) [2023-12-27 00:26:53,256][105692] Updated weights for policy 0, policy_version 1244348 (0.0010) [2023-12-27 00:26:53,271][105620] Updated weights for policy 1, policy_version 1245587 (0.0010) [2023-12-27 00:26:53,312][105692] Updated weights for policy 0, policy_version 1244358 (0.0005) [2023-12-27 00:26:53,326][105620] Updated weights for policy 1, policy_version 1245597 (0.0010) [2023-12-27 00:26:53,364][105692] Updated weights for policy 0, policy_version 1244368 (0.0006) [2023-12-27 00:26:53,986][105620] Updated weights for policy 1, policy_version 1245607 (0.0009) [2023-12-27 00:26:54,045][105620] Updated weights for policy 1, policy_version 1245617 (0.0008) [2023-12-27 00:26:54,078][105692] Updated weights for policy 0, policy_version 1244378 (0.0006) [2023-12-27 00:26:54,110][105620] Updated weights for policy 1, policy_version 1245627 (0.0008) [2023-12-27 00:26:54,138][105692] Updated weights for policy 0, policy_version 1244388 (0.0006) [2023-12-27 00:26:54,197][105692] Updated weights for policy 0, policy_version 1244398 (0.0011) [2023-12-27 00:26:54,705][105620] Updated weights for policy 1, policy_version 1245637 (0.0007) [2023-12-27 00:26:54,748][105620] Updated weights for policy 1, policy_version 1245647 (0.0010) [2023-12-27 00:26:54,793][105620] Updated weights for policy 1, policy_version 1245657 (0.0010) [2023-12-27 00:26:54,885][105692] Updated weights for policy 0, policy_version 1244408 (0.0010) [2023-12-27 00:26:54,929][105692] Updated weights for policy 0, policy_version 1244418 (0.0010) [2023-12-27 00:26:54,990][105692] Updated weights for policy 0, policy_version 1244428 (0.0010) [2023-12-27 00:26:55,575][105620] Updated weights for policy 1, policy_version 1245667 (0.0008) [2023-12-27 00:26:55,637][105620] Updated weights for policy 1, policy_version 1245677 (0.0010) [2023-12-27 00:26:55,688][105692] Updated weights for policy 0, policy_version 1244438 (0.0008) [2023-12-27 00:26:55,700][105620] Updated weights for policy 1, policy_version 1245687 (0.0010) [2023-12-27 00:26:55,745][105692] Updated weights for policy 0, policy_version 1244448 (0.0008) [2023-12-27 00:26:55,808][105692] Updated weights for policy 0, policy_version 1244458 (0.0009) [2023-12-27 00:26:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 637575168. Throughput: 0: 9727.4, 1: 9783.3. Samples: 637581948. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:26:56,063][104569] Avg episode reward: [(0, '9083.298'), (1, '9262.094')] [2023-12-27 00:26:56,345][105620] Updated weights for policy 1, policy_version 1245697 (0.0007) [2023-12-27 00:26:56,416][105620] Updated weights for policy 1, policy_version 1245707 (0.0008) [2023-12-27 00:26:56,469][105620] Updated weights for policy 1, policy_version 1245717 (0.0007) [2023-12-27 00:26:56,516][105620] Updated weights for policy 1, policy_version 1245727 (0.0008) [2023-12-27 00:26:56,603][105692] Updated weights for policy 0, policy_version 1244468 (0.0008) [2023-12-27 00:26:56,657][105692] Updated weights for policy 0, policy_version 1244478 (0.0010) [2023-12-27 00:26:56,711][105692] Updated weights for policy 0, policy_version 1244490 (0.0010) [2023-12-27 00:26:57,041][105620] Updated weights for policy 1, policy_version 1245737 (0.0005) [2023-12-27 00:26:57,096][105620] Updated weights for policy 1, policy_version 1245747 (0.0005) [2023-12-27 00:26:57,156][105620] Updated weights for policy 1, policy_version 1245757 (0.0005) [2023-12-27 00:26:57,663][105692] Updated weights for policy 0, policy_version 1244500 (0.0009) [2023-12-27 00:26:57,664][105620] Updated weights for policy 1, policy_version 1245767 (0.0007) [2023-12-27 00:26:57,711][105692] Updated weights for policy 0, policy_version 1244510 (0.0009) [2023-12-27 00:26:57,724][105620] Updated weights for policy 1, policy_version 1245777 (0.0005) [2023-12-27 00:26:57,766][105692] Updated weights for policy 0, policy_version 1244520 (0.0008) [2023-12-27 00:26:57,775][105620] Updated weights for policy 1, policy_version 1245787 (0.0005) [2023-12-27 00:26:58,486][105620] Updated weights for policy 1, policy_version 1245797 (0.0007) [2023-12-27 00:26:58,499][105692] Updated weights for policy 0, policy_version 1244530 (0.0009) [2023-12-27 00:26:58,552][105620] Updated weights for policy 1, policy_version 1245807 (0.0008) [2023-12-27 00:26:58,564][105692] Updated weights for policy 0, policy_version 1244540 (0.0009) [2023-12-27 00:26:58,612][105620] Updated weights for policy 1, policy_version 1245817 (0.0008) [2023-12-27 00:26:58,626][105692] Updated weights for policy 0, policy_version 1244550 (0.0008) [2023-12-27 00:26:58,684][105692] Updated weights for policy 0, policy_version 1244560 (0.0007) [2023-12-27 00:26:59,411][105620] Updated weights for policy 1, policy_version 1245827 (0.0008) [2023-12-27 00:26:59,471][105692] Updated weights for policy 0, policy_version 1244570 (0.0006) [2023-12-27 00:26:59,472][105620] Updated weights for policy 1, policy_version 1245837 (0.0007) [2023-12-27 00:26:59,525][105692] Updated weights for policy 0, policy_version 1244580 (0.0008) [2023-12-27 00:26:59,534][105620] Updated weights for policy 1, policy_version 1245847 (0.0008) [2023-12-27 00:26:59,581][105692] Updated weights for policy 0, policy_version 1244590 (0.0007) [2023-12-27 00:27:00,274][105620] Updated weights for policy 1, policy_version 1245857 (0.0010) [2023-12-27 00:27:00,336][105620] Updated weights for policy 1, policy_version 1245867 (0.0007) [2023-12-27 00:27:00,348][105692] Updated weights for policy 0, policy_version 1244600 (0.0009) [2023-12-27 00:27:00,395][105620] Updated weights for policy 1, policy_version 1245877 (0.0007) [2023-12-27 00:27:00,405][105692] Updated weights for policy 0, policy_version 1244610 (0.0006) [2023-12-27 00:27:00,448][105620] Updated weights for policy 1, policy_version 1245887 (0.0006) [2023-12-27 00:27:00,458][105692] Updated weights for policy 0, policy_version 1244620 (0.0006) [2023-12-27 00:27:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 637665280. Throughput: 0: 9713.0, 1: 9794.4. Samples: 637641232. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:27:01,062][104569] Avg episode reward: [(0, '8899.032'), (1, '9352.873')] [2023-12-27 00:27:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001244624_318676992.pth... [2023-12-27 00:27:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001245888_318988288.pth... [2023-12-27 00:27:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001244768_318701568.pth [2023-12-27 00:27:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001243504_318390272.pth [2023-12-27 00:27:01,156][105620] Updated weights for policy 1, policy_version 1245897 (0.0008) [2023-12-27 00:27:01,216][105620] Updated weights for policy 1, policy_version 1245907 (0.0008) [2023-12-27 00:27:01,243][105692] Updated weights for policy 0, policy_version 1244630 (0.0008) [2023-12-27 00:27:01,278][105620] Updated weights for policy 1, policy_version 1245917 (0.0007) [2023-12-27 00:27:01,305][105692] Updated weights for policy 0, policy_version 1244640 (0.0009) [2023-12-27 00:27:01,371][105692] Updated weights for policy 0, policy_version 1244650 (0.0009) [2023-12-27 00:27:01,954][105620] Updated weights for policy 1, policy_version 1245927 (0.0008) [2023-12-27 00:27:02,019][105620] Updated weights for policy 1, policy_version 1245937 (0.0009) [2023-12-27 00:27:02,081][105620] Updated weights for policy 1, policy_version 1245947 (0.0006) [2023-12-27 00:27:02,165][105692] Updated weights for policy 0, policy_version 1244660 (0.0009) [2023-12-27 00:27:02,220][105692] Updated weights for policy 0, policy_version 1244670 (0.0009) [2023-12-27 00:27:02,283][105692] Updated weights for policy 0, policy_version 1244680 (0.0009) [2023-12-27 00:27:02,865][105620] Updated weights for policy 1, policy_version 1245957 (0.0009) [2023-12-27 00:27:02,917][105620] Updated weights for policy 1, policy_version 1245967 (0.0010) [2023-12-27 00:27:02,921][105692] Updated weights for policy 0, policy_version 1244690 (0.0006) [2023-12-27 00:27:02,970][105620] Updated weights for policy 1, policy_version 1245977 (0.0009) [2023-12-27 00:27:02,978][105692] Updated weights for policy 0, policy_version 1244700 (0.0006) [2023-12-27 00:27:03,044][105692] Updated weights for policy 0, policy_version 1244710 (0.0006) [2023-12-27 00:27:03,095][105692] Updated weights for policy 0, policy_version 1244720 (0.0008) [2023-12-27 00:27:03,580][105620] Updated weights for policy 1, policy_version 1245987 (0.0007) [2023-12-27 00:27:03,644][105620] Updated weights for policy 1, policy_version 1245997 (0.0005) [2023-12-27 00:27:03,709][105620] Updated weights for policy 1, policy_version 1246007 (0.0005) [2023-12-27 00:27:03,887][105692] Updated weights for policy 0, policy_version 1244730 (0.0008) [2023-12-27 00:27:03,937][105692] Updated weights for policy 0, policy_version 1244740 (0.0010) [2023-12-27 00:27:03,985][105692] Updated weights for policy 0, policy_version 1244750 (0.0010) [2023-12-27 00:27:04,370][105620] Updated weights for policy 1, policy_version 1246017 (0.0006) [2023-12-27 00:27:04,431][105620] Updated weights for policy 1, policy_version 1246027 (0.0008) [2023-12-27 00:27:04,476][105620] Updated weights for policy 1, policy_version 1246037 (0.0011) [2023-12-27 00:27:04,538][105620] Updated weights for policy 1, policy_version 1246047 (0.0011) [2023-12-27 00:27:04,749][105692] Updated weights for policy 0, policy_version 1244760 (0.0010) [2023-12-27 00:27:04,809][105692] Updated weights for policy 0, policy_version 1244770 (0.0010) [2023-12-27 00:27:04,873][105692] Updated weights for policy 0, policy_version 1244780 (0.0008) [2023-12-27 00:27:05,218][105620] Updated weights for policy 1, policy_version 1246057 (0.0009) [2023-12-27 00:27:05,282][105620] Updated weights for policy 1, policy_version 1246067 (0.0009) [2023-12-27 00:27:05,341][105620] Updated weights for policy 1, policy_version 1246077 (0.0008) [2023-12-27 00:27:05,615][105692] Updated weights for policy 0, policy_version 1244790 (0.0008) [2023-12-27 00:27:05,670][105692] Updated weights for policy 0, policy_version 1244800 (0.0010) [2023-12-27 00:27:05,729][105692] Updated weights for policy 0, policy_version 1244810 (0.0010) [2023-12-27 00:27:06,040][105620] Updated weights for policy 1, policy_version 1246087 (0.0009) [2023-12-27 00:27:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 637763584. Throughput: 0: 9632.2, 1: 9764.5. Samples: 637755620. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:27:06,063][104569] Avg episode reward: [(0, '8989.105'), (1, '9352.725')] [2023-12-27 00:27:06,099][105620] Updated weights for policy 1, policy_version 1246097 (0.0006) [2023-12-27 00:27:06,171][105620] Updated weights for policy 1, policy_version 1246107 (0.0007) [2023-12-27 00:27:06,580][105692] Updated weights for policy 0, policy_version 1244820 (0.0009) [2023-12-27 00:27:06,632][105692] Updated weights for policy 0, policy_version 1244830 (0.0009) [2023-12-27 00:27:06,685][105692] Updated weights for policy 0, policy_version 1244840 (0.0009) [2023-12-27 00:27:06,840][105620] Updated weights for policy 1, policy_version 1246117 (0.0008) [2023-12-27 00:27:06,897][105620] Updated weights for policy 1, policy_version 1246127 (0.0008) [2023-12-27 00:27:06,948][105620] Updated weights for policy 1, policy_version 1246137 (0.0007) [2023-12-27 00:27:07,510][105620] Updated weights for policy 1, policy_version 1246147 (0.0006) [2023-12-27 00:27:07,528][105692] Updated weights for policy 0, policy_version 1244850 (0.0010) [2023-12-27 00:27:07,562][105620] Updated weights for policy 1, policy_version 1246157 (0.0007) [2023-12-27 00:27:07,585][105692] Updated weights for policy 0, policy_version 1244860 (0.0006) [2023-12-27 00:27:07,611][105620] Updated weights for policy 1, policy_version 1246167 (0.0010) [2023-12-27 00:27:07,641][105692] Updated weights for policy 0, policy_version 1244870 (0.0005) [2023-12-27 00:27:07,703][105692] Updated weights for policy 0, policy_version 1244880 (0.0009) [2023-12-27 00:27:08,156][105620] Updated weights for policy 1, policy_version 1246177 (0.0009) [2023-12-27 00:27:08,211][105620] Updated weights for policy 1, policy_version 1246187 (0.0010) [2023-12-27 00:27:08,279][105620] Updated weights for policy 1, policy_version 1246197 (0.0010) [2023-12-27 00:27:08,337][105620] Updated weights for policy 1, policy_version 1246207 (0.0010) [2023-12-27 00:27:08,384][105692] Updated weights for policy 0, policy_version 1244890 (0.0007) [2023-12-27 00:27:08,444][105692] Updated weights for policy 0, policy_version 1244900 (0.0008) [2023-12-27 00:27:08,496][105692] Updated weights for policy 0, policy_version 1244910 (0.0009) [2023-12-27 00:27:08,955][105620] Updated weights for policy 1, policy_version 1246217 (0.0009) [2023-12-27 00:27:09,003][105620] Updated weights for policy 1, policy_version 1246227 (0.0010) [2023-12-27 00:27:09,052][105620] Updated weights for policy 1, policy_version 1246237 (0.0010) [2023-12-27 00:27:09,219][105692] Updated weights for policy 0, policy_version 1244920 (0.0006) [2023-12-27 00:27:09,278][105692] Updated weights for policy 0, policy_version 1244930 (0.0009) [2023-12-27 00:27:09,339][105692] Updated weights for policy 0, policy_version 1244940 (0.0009) [2023-12-27 00:27:09,727][105620] Updated weights for policy 1, policy_version 1246247 (0.0006) [2023-12-27 00:27:09,788][105620] Updated weights for policy 1, policy_version 1246257 (0.0010) [2023-12-27 00:27:09,846][105620] Updated weights for policy 1, policy_version 1246267 (0.0010) [2023-12-27 00:27:10,135][105692] Updated weights for policy 0, policy_version 1244950 (0.0009) [2023-12-27 00:27:10,156][105585] KL-divergence is very high: 204.7425 [2023-12-27 00:27:10,169][105585] KL-divergence is very high: 190.5945 [2023-12-27 00:27:10,193][105692] Updated weights for policy 0, policy_version 1244960 (0.0009) [2023-12-27 00:27:10,206][105585] KL-divergence is very high: 354.5203 [2023-12-27 00:27:10,220][105585] KL-divergence is very high: 299.1191 [2023-12-27 00:27:10,257][105692] Updated weights for policy 0, policy_version 1244970 (0.0009) [2023-12-27 00:27:10,258][105585] KL-divergence is very high: 407.9116 [2023-12-27 00:27:10,270][105585] KL-divergence is very high: 333.6830 [2023-12-27 00:27:10,548][105620] Updated weights for policy 1, policy_version 1246277 (0.0009) [2023-12-27 00:27:10,611][105620] Updated weights for policy 1, policy_version 1246287 (0.0010) [2023-12-27 00:27:10,677][105620] Updated weights for policy 1, policy_version 1246297 (0.0009) [2023-12-27 00:27:10,901][105692] Updated weights for policy 0, policy_version 1244980 (0.0007) [2023-12-27 00:27:10,947][105692] Updated weights for policy 0, policy_version 1244990 (0.0005) [2023-12-27 00:27:10,993][105692] Updated weights for policy 0, policy_version 1245000 (0.0005) [2023-12-27 00:27:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 637870080. Throughput: 0: 9609.3, 1: 9860.8. Samples: 637873844. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:27:11,062][104569] Avg episode reward: [(0, '8897.121'), (1, '9261.953')] [2023-12-27 00:27:11,389][105620] Updated weights for policy 1, policy_version 1246307 (0.0010) [2023-12-27 00:27:11,450][105620] Updated weights for policy 1, policy_version 1246317 (0.0009) [2023-12-27 00:27:11,510][105620] Updated weights for policy 1, policy_version 1246327 (0.0009) [2023-12-27 00:27:11,727][105692] Updated weights for policy 0, policy_version 1245010 (0.0008) [2023-12-27 00:27:11,793][105692] Updated weights for policy 0, policy_version 1245020 (0.0009) [2023-12-27 00:27:11,860][105692] Updated weights for policy 0, policy_version 1245030 (0.0007) [2023-12-27 00:27:11,922][105692] Updated weights for policy 0, policy_version 1245040 (0.0009) [2023-12-27 00:27:12,319][105620] Updated weights for policy 1, policy_version 1246337 (0.0009) [2023-12-27 00:27:12,387][105620] Updated weights for policy 1, policy_version 1246347 (0.0009) [2023-12-27 00:27:12,445][105620] Updated weights for policy 1, policy_version 1246357 (0.0008) [2023-12-27 00:27:12,504][105620] Updated weights for policy 1, policy_version 1246367 (0.0008) [2023-12-27 00:27:12,662][105692] Updated weights for policy 0, policy_version 1245050 (0.0010) [2023-12-27 00:27:12,715][105692] Updated weights for policy 0, policy_version 1245060 (0.0009) [2023-12-27 00:27:12,782][105692] Updated weights for policy 0, policy_version 1245070 (0.0010) [2023-12-27 00:27:13,180][105620] Updated weights for policy 1, policy_version 1246377 (0.0009) [2023-12-27 00:27:13,228][105620] Updated weights for policy 1, policy_version 1246387 (0.0008) [2023-12-27 00:27:13,286][105620] Updated weights for policy 1, policy_version 1246397 (0.0009) [2023-12-27 00:27:13,567][105692] Updated weights for policy 0, policy_version 1245080 (0.0008) [2023-12-27 00:27:13,620][105692] Updated weights for policy 0, policy_version 1245090 (0.0009) [2023-12-27 00:27:13,680][105692] Updated weights for policy 0, policy_version 1245100 (0.0009) [2023-12-27 00:27:14,074][105620] Updated weights for policy 1, policy_version 1246407 (0.0009) [2023-12-27 00:27:14,139][105620] Updated weights for policy 1, policy_version 1246417 (0.0009) [2023-12-27 00:27:14,203][105620] Updated weights for policy 1, policy_version 1246427 (0.0009) [2023-12-27 00:27:14,351][105692] Updated weights for policy 0, policy_version 1245110 (0.0009) [2023-12-27 00:27:14,412][105692] Updated weights for policy 0, policy_version 1245120 (0.0008) [2023-12-27 00:27:14,469][105692] Updated weights for policy 0, policy_version 1245130 (0.0008) [2023-12-27 00:27:14,964][105620] Updated weights for policy 1, policy_version 1246437 (0.0009) [2023-12-27 00:27:15,017][105620] Updated weights for policy 1, policy_version 1246447 (0.0010) [2023-12-27 00:27:15,074][105620] Updated weights for policy 1, policy_version 1246457 (0.0010) [2023-12-27 00:27:15,107][105692] Updated weights for policy 0, policy_version 1245140 (0.0009) [2023-12-27 00:27:15,169][105692] Updated weights for policy 0, policy_version 1245150 (0.0008) [2023-12-27 00:27:15,235][105692] Updated weights for policy 0, policy_version 1245160 (0.0009) [2023-12-27 00:27:15,879][105692] Updated weights for policy 0, policy_version 1245170 (0.0009) [2023-12-27 00:27:15,913][105620] Updated weights for policy 1, policy_version 1246467 (0.0007) [2023-12-27 00:27:15,927][105692] Updated weights for policy 0, policy_version 1245180 (0.0008) [2023-12-27 00:27:15,961][105620] Updated weights for policy 1, policy_version 1246477 (0.0007) [2023-12-27 00:27:15,976][105692] Updated weights for policy 0, policy_version 1245190 (0.0006) [2023-12-27 00:27:16,010][105620] Updated weights for policy 1, policy_version 1246487 (0.0007) [2023-12-27 00:27:16,024][105692] Updated weights for policy 0, policy_version 1245200 (0.0007) [2023-12-27 00:27:16,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 637968384. Throughput: 0: 9567.0, 1: 9813.3. Samples: 637930660. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:27:16,063][104569] Avg episode reward: [(0, '8531.284'), (1, '9262.142')] [2023-12-27 00:27:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001246496_319143936.pth... [2023-12-27 00:27:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001245200_318824448.pth... [2023-12-27 00:27:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001245312_318840832.pth [2023-12-27 00:27:16,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001244112_318545920.pth [2023-12-27 00:27:16,723][105692] Updated weights for policy 0, policy_version 1245210 (0.0009) [2023-12-27 00:27:16,774][105692] Updated weights for policy 0, policy_version 1245220 (0.0007) [2023-12-27 00:27:16,809][105620] Updated weights for policy 1, policy_version 1246497 (0.0008) [2023-12-27 00:27:16,835][105692] Updated weights for policy 0, policy_version 1245230 (0.0006) [2023-12-27 00:27:16,862][105620] Updated weights for policy 1, policy_version 1246507 (0.0008) [2023-12-27 00:27:16,915][105620] Updated weights for policy 1, policy_version 1246518 (0.0009) [2023-12-27 00:27:16,976][105620] Updated weights for policy 1, policy_version 1246528 (0.0009) [2023-12-27 00:27:17,390][105692] Updated weights for policy 0, policy_version 1245240 (0.0006) [2023-12-27 00:27:17,451][105692] Updated weights for policy 0, policy_version 1245250 (0.0005) [2023-12-27 00:27:17,502][105692] Updated weights for policy 0, policy_version 1245260 (0.0005) [2023-12-27 00:27:17,893][105620] Updated weights for policy 1, policy_version 1246538 (0.0008) [2023-12-27 00:27:17,941][105620] Updated weights for policy 1, policy_version 1246548 (0.0008) [2023-12-27 00:27:17,995][105620] Updated weights for policy 1, policy_version 1246558 (0.0006) [2023-12-27 00:27:18,110][105692] Updated weights for policy 0, policy_version 1245270 (0.0009) [2023-12-27 00:27:18,161][105692] Updated weights for policy 0, policy_version 1245280 (0.0010) [2023-12-27 00:27:18,213][105692] Updated weights for policy 0, policy_version 1245290 (0.0010) [2023-12-27 00:27:18,666][105620] Updated weights for policy 1, policy_version 1246568 (0.0006) [2023-12-27 00:27:18,732][105620] Updated weights for policy 1, policy_version 1246578 (0.0008) [2023-12-27 00:27:18,796][105620] Updated weights for policy 1, policy_version 1246588 (0.0008) [2023-12-27 00:27:18,975][105692] Updated weights for policy 0, policy_version 1245300 (0.0010) [2023-12-27 00:27:19,037][105692] Updated weights for policy 0, policy_version 1245310 (0.0010) [2023-12-27 00:27:19,094][105692] Updated weights for policy 0, policy_version 1245320 (0.0010) [2023-12-27 00:27:19,578][105620] Updated weights for policy 1, policy_version 1246598 (0.0008) [2023-12-27 00:27:19,643][105620] Updated weights for policy 1, policy_version 1246608 (0.0008) [2023-12-27 00:27:19,703][105620] Updated weights for policy 1, policy_version 1246618 (0.0008) [2023-12-27 00:27:19,864][105692] Updated weights for policy 0, policy_version 1245330 (0.0011) [2023-12-27 00:27:19,925][105692] Updated weights for policy 0, policy_version 1245340 (0.0011) [2023-12-27 00:27:19,977][105692] Updated weights for policy 0, policy_version 1245350 (0.0010) [2023-12-27 00:27:20,030][105692] Updated weights for policy 0, policy_version 1245360 (0.0010) [2023-12-27 00:27:20,469][105620] Updated weights for policy 1, policy_version 1246628 (0.0008) [2023-12-27 00:27:20,525][105620] Updated weights for policy 1, policy_version 1246638 (0.0008) [2023-12-27 00:27:20,596][105620] Updated weights for policy 1, policy_version 1246648 (0.0008) [2023-12-27 00:27:20,869][105692] Updated weights for policy 0, policy_version 1245370 (0.0011) [2023-12-27 00:27:20,928][105692] Updated weights for policy 0, policy_version 1245380 (0.0010) [2023-12-27 00:27:20,977][105692] Updated weights for policy 0, policy_version 1245390 (0.0011) [2023-12-27 00:27:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 638058496. Throughput: 0: 9572.2, 1: 9753.5. Samples: 638046372. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:27:21,062][104569] Avg episode reward: [(0, '8436.844'), (1, '9171.328')] [2023-12-27 00:27:21,385][105620] Updated weights for policy 1, policy_version 1246658 (0.0008) [2023-12-27 00:27:21,451][105620] Updated weights for policy 1, policy_version 1246668 (0.0008) [2023-12-27 00:27:21,515][105620] Updated weights for policy 1, policy_version 1246678 (0.0008) [2023-12-27 00:27:21,575][105620] Updated weights for policy 1, policy_version 1246688 (0.0008) [2023-12-27 00:27:21,753][105692] Updated weights for policy 0, policy_version 1245400 (0.0011) [2023-12-27 00:27:21,813][105692] Updated weights for policy 0, policy_version 1245410 (0.0010) [2023-12-27 00:27:21,875][105692] Updated weights for policy 0, policy_version 1245420 (0.0010) [2023-12-27 00:27:22,335][105620] Updated weights for policy 1, policy_version 1246698 (0.0007) [2023-12-27 00:27:22,401][105620] Updated weights for policy 1, policy_version 1246708 (0.0007) [2023-12-27 00:27:22,459][105620] Updated weights for policy 1, policy_version 1246718 (0.0006) [2023-12-27 00:27:22,598][105692] Updated weights for policy 0, policy_version 1245430 (0.0008) [2023-12-27 00:27:22,668][105692] Updated weights for policy 0, policy_version 1245440 (0.0008) [2023-12-27 00:27:22,724][105692] Updated weights for policy 0, policy_version 1245450 (0.0009) [2023-12-27 00:27:23,134][105620] Updated weights for policy 1, policy_version 1246728 (0.0008) [2023-12-27 00:27:23,187][105620] Updated weights for policy 1, policy_version 1246738 (0.0008) [2023-12-27 00:27:23,243][105620] Updated weights for policy 1, policy_version 1246748 (0.0009) [2023-12-27 00:27:23,391][105692] Updated weights for policy 0, policy_version 1245460 (0.0008) [2023-12-27 00:27:23,442][105692] Updated weights for policy 0, policy_version 1245470 (0.0009) [2023-12-27 00:27:23,500][105692] Updated weights for policy 0, policy_version 1245480 (0.0010) [2023-12-27 00:27:23,904][105620] Updated weights for policy 1, policy_version 1246758 (0.0007) [2023-12-27 00:27:23,966][105620] Updated weights for policy 1, policy_version 1246768 (0.0006) [2023-12-27 00:27:24,022][105620] Updated weights for policy 1, policy_version 1246778 (0.0005) [2023-12-27 00:27:24,247][105692] Updated weights for policy 0, policy_version 1245490 (0.0010) [2023-12-27 00:27:24,312][105692] Updated weights for policy 0, policy_version 1245500 (0.0010) [2023-12-27 00:27:24,366][105692] Updated weights for policy 0, policy_version 1245510 (0.0010) [2023-12-27 00:27:24,421][105692] Updated weights for policy 0, policy_version 1245520 (0.0010) [2023-12-27 00:27:24,548][105620] Updated weights for policy 1, policy_version 1246788 (0.0005) [2023-12-27 00:27:24,614][105620] Updated weights for policy 1, policy_version 1246798 (0.0005) [2023-12-27 00:27:24,680][105620] Updated weights for policy 1, policy_version 1246808 (0.0006) [2023-12-27 00:27:25,141][105692] Updated weights for policy 0, policy_version 1245530 (0.0005) [2023-12-27 00:27:25,203][105692] Updated weights for policy 0, policy_version 1245540 (0.0005) [2023-12-27 00:27:25,258][105692] Updated weights for policy 0, policy_version 1245550 (0.0005) [2023-12-27 00:27:25,315][105620] Updated weights for policy 1, policy_version 1246818 (0.0007) [2023-12-27 00:27:25,373][105620] Updated weights for policy 1, policy_version 1246828 (0.0010) [2023-12-27 00:27:25,431][105620] Updated weights for policy 1, policy_version 1246838 (0.0010) [2023-12-27 00:27:25,485][105620] Updated weights for policy 1, policy_version 1246848 (0.0010) [2023-12-27 00:27:25,934][105692] Updated weights for policy 0, policy_version 1245560 (0.0009) [2023-12-27 00:27:25,997][105692] Updated weights for policy 0, policy_version 1245570 (0.0010) [2023-12-27 00:27:26,061][105692] Updated weights for policy 0, policy_version 1245580 (0.0009) [2023-12-27 00:27:26,062][104569] Fps is (10 sec: 18022.7, 60 sec: 19251.3, 300 sec: 19466.4). Total num frames: 638148608. Throughput: 0: 9527.7, 1: 9850.8. Samples: 638164064. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:27:26,062][104569] Avg episode reward: [(0, '8438.862'), (1, '9079.771')] [2023-12-27 00:27:26,138][105620] Updated weights for policy 1, policy_version 1246858 (0.0011) [2023-12-27 00:27:26,201][105620] Updated weights for policy 1, policy_version 1246868 (0.0011) [2023-12-27 00:27:26,267][105620] Updated weights for policy 1, policy_version 1246878 (0.0010) [2023-12-27 00:27:26,755][105692] Updated weights for policy 0, policy_version 1245590 (0.0006) [2023-12-27 00:27:26,809][105692] Updated weights for policy 0, policy_version 1245600 (0.0005) [2023-12-27 00:27:26,866][105692] Updated weights for policy 0, policy_version 1245610 (0.0005) [2023-12-27 00:27:27,003][105620] Updated weights for policy 1, policy_version 1246888 (0.0010) [2023-12-27 00:27:27,060][105620] Updated weights for policy 1, policy_version 1246898 (0.0010) [2023-12-27 00:27:27,117][105620] Updated weights for policy 1, policy_version 1246908 (0.0010) [2023-12-27 00:27:27,472][105692] Updated weights for policy 0, policy_version 1245620 (0.0005) [2023-12-27 00:27:27,518][105692] Updated weights for policy 0, policy_version 1245630 (0.0007) [2023-12-27 00:27:27,568][105692] Updated weights for policy 0, policy_version 1245640 (0.0005) [2023-12-27 00:27:27,850][105620] Updated weights for policy 1, policy_version 1246918 (0.0010) [2023-12-27 00:27:27,894][105620] Updated weights for policy 1, policy_version 1246928 (0.0010) [2023-12-27 00:27:27,938][105620] Updated weights for policy 1, policy_version 1246938 (0.0010) [2023-12-27 00:27:28,247][105692] Updated weights for policy 0, policy_version 1245650 (0.0006) [2023-12-27 00:27:28,308][105692] Updated weights for policy 0, policy_version 1245660 (0.0010) [2023-12-27 00:27:28,375][105692] Updated weights for policy 0, policy_version 1245670 (0.0010) [2023-12-27 00:27:28,439][105692] Updated weights for policy 0, policy_version 1245680 (0.0010) [2023-12-27 00:27:28,690][105620] Updated weights for policy 1, policy_version 1246948 (0.0008) [2023-12-27 00:27:28,744][105620] Updated weights for policy 1, policy_version 1246958 (0.0005) [2023-12-27 00:27:28,802][105620] Updated weights for policy 1, policy_version 1246968 (0.0005) [2023-12-27 00:27:29,182][105692] Updated weights for policy 0, policy_version 1245690 (0.0007) [2023-12-27 00:27:29,245][105692] Updated weights for policy 0, policy_version 1245700 (0.0008) [2023-12-27 00:27:29,298][105692] Updated weights for policy 0, policy_version 1245710 (0.0006) [2023-12-27 00:27:29,464][105620] Updated weights for policy 1, policy_version 1246978 (0.0005) [2023-12-27 00:27:29,528][105620] Updated weights for policy 1, policy_version 1246988 (0.0006) [2023-12-27 00:27:29,596][105620] Updated weights for policy 1, policy_version 1246998 (0.0006) [2023-12-27 00:27:29,663][105620] Updated weights for policy 1, policy_version 1247008 (0.0005) [2023-12-27 00:27:30,072][105692] Updated weights for policy 0, policy_version 1245720 (0.0008) [2023-12-27 00:27:30,130][105692] Updated weights for policy 0, policy_version 1245730 (0.0009) [2023-12-27 00:27:30,186][105692] Updated weights for policy 0, policy_version 1245740 (0.0009) [2023-12-27 00:27:30,248][105620] Updated weights for policy 1, policy_version 1247018 (0.0006) [2023-12-27 00:27:30,301][105620] Updated weights for policy 1, policy_version 1247028 (0.0005) [2023-12-27 00:27:30,356][105620] Updated weights for policy 1, policy_version 1247038 (0.0005) [2023-12-27 00:27:30,909][105692] Updated weights for policy 0, policy_version 1245750 (0.0006) [2023-12-27 00:27:30,964][105620] Updated weights for policy 1, policy_version 1247048 (0.0005) [2023-12-27 00:27:30,965][105692] Updated weights for policy 0, policy_version 1245760 (0.0005) [2023-12-27 00:27:31,016][105692] Updated weights for policy 0, policy_version 1245770 (0.0006) [2023-12-27 00:27:31,023][105620] Updated weights for policy 1, policy_version 1247058 (0.0006) [2023-12-27 00:27:31,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 638255104. Throughput: 0: 9507.4, 1: 9916.3. Samples: 638223984. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:27:31,063][104569] Avg episode reward: [(0, '8622.355'), (1, '8987.007')] [2023-12-27 00:27:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001245776_318971904.pth... [2023-12-27 00:27:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001244624_318676992.pth [2023-12-27 00:27:31,086][105620] Updated weights for policy 1, policy_version 1247068 (0.0009) [2023-12-27 00:27:31,106][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001247072_319291392.pth... [2023-12-27 00:27:31,110][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001245888_318988288.pth [2023-12-27 00:27:31,767][105620] Updated weights for policy 1, policy_version 1247078 (0.0009) [2023-12-27 00:27:31,782][105692] Updated weights for policy 0, policy_version 1245780 (0.0007) [2023-12-27 00:27:31,817][105620] Updated weights for policy 1, policy_version 1247088 (0.0008) [2023-12-27 00:27:31,831][105692] Updated weights for policy 0, policy_version 1245790 (0.0006) [2023-12-27 00:27:31,870][105620] Updated weights for policy 1, policy_version 1247098 (0.0007) [2023-12-27 00:27:31,884][105692] Updated weights for policy 0, policy_version 1245800 (0.0007) [2023-12-27 00:27:32,610][105620] Updated weights for policy 1, policy_version 1247108 (0.0008) [2023-12-27 00:27:32,675][105620] Updated weights for policy 1, policy_version 1247118 (0.0010) [2023-12-27 00:27:32,684][105692] Updated weights for policy 0, policy_version 1245810 (0.0009) [2023-12-27 00:27:32,733][105620] Updated weights for policy 1, policy_version 1247128 (0.0010) [2023-12-27 00:27:32,739][105692] Updated weights for policy 0, policy_version 1245820 (0.0006) [2023-12-27 00:27:32,800][105692] Updated weights for policy 0, policy_version 1245830 (0.0007) [2023-12-27 00:27:32,856][105692] Updated weights for policy 0, policy_version 1245840 (0.0008) [2023-12-27 00:27:33,366][105620] Updated weights for policy 1, policy_version 1247138 (0.0010) [2023-12-27 00:27:33,429][105620] Updated weights for policy 1, policy_version 1247148 (0.0007) [2023-12-27 00:27:33,482][105620] Updated weights for policy 1, policy_version 1247158 (0.0006) [2023-12-27 00:27:33,532][105620] Updated weights for policy 1, policy_version 1247168 (0.0005) [2023-12-27 00:27:33,703][105692] Updated weights for policy 0, policy_version 1245850 (0.0010) [2023-12-27 00:27:33,760][105692] Updated weights for policy 0, policy_version 1245860 (0.0009) [2023-12-27 00:27:33,821][105692] Updated weights for policy 0, policy_version 1245870 (0.0010) [2023-12-27 00:27:34,056][105620] Updated weights for policy 1, policy_version 1247178 (0.0009) [2023-12-27 00:27:34,109][105620] Updated weights for policy 1, policy_version 1247188 (0.0009) [2023-12-27 00:27:34,166][105620] Updated weights for policy 1, policy_version 1247198 (0.0008) [2023-12-27 00:27:34,661][105692] Updated weights for policy 0, policy_version 1245880 (0.0010) [2023-12-27 00:27:34,732][105692] Updated weights for policy 0, policy_version 1245890 (0.0009) [2023-12-27 00:27:34,793][105692] Updated weights for policy 0, policy_version 1245900 (0.0009) [2023-12-27 00:27:34,841][105620] Updated weights for policy 1, policy_version 1247208 (0.0008) [2023-12-27 00:27:34,901][105620] Updated weights for policy 1, policy_version 1247218 (0.0009) [2023-12-27 00:27:34,957][105620] Updated weights for policy 1, policy_version 1247228 (0.0009) [2023-12-27 00:27:35,590][105692] Updated weights for policy 0, policy_version 1245910 (0.0008) [2023-12-27 00:27:35,641][105620] Updated weights for policy 1, policy_version 1247238 (0.0008) [2023-12-27 00:27:35,643][105692] Updated weights for policy 0, policy_version 1245920 (0.0007) [2023-12-27 00:27:35,696][105692] Updated weights for policy 0, policy_version 1245930 (0.0008) [2023-12-27 00:27:35,705][105620] Updated weights for policy 1, policy_version 1247248 (0.0007) [2023-12-27 00:27:35,771][105620] Updated weights for policy 1, policy_version 1247258 (0.0009) [2023-12-27 00:27:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 638353408. Throughput: 0: 9472.5, 1: 10010.5. Samples: 638341940. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:27:36,063][104569] Avg episode reward: [(0, '9079.080'), (1, '8713.600')] [2023-12-27 00:27:36,438][105692] Updated weights for policy 0, policy_version 1245940 (0.0009) [2023-12-27 00:27:36,493][105692] Updated weights for policy 0, policy_version 1245950 (0.0011) [2023-12-27 00:27:36,552][105692] Updated weights for policy 0, policy_version 1245960 (0.0011) [2023-12-27 00:27:36,566][105620] Updated weights for policy 1, policy_version 1247268 (0.0009) [2023-12-27 00:27:36,626][105620] Updated weights for policy 1, policy_version 1247278 (0.0011) [2023-12-27 00:27:36,688][105620] Updated weights for policy 1, policy_version 1247288 (0.0007) [2023-12-27 00:27:37,221][105692] Updated weights for policy 0, policy_version 1245970 (0.0009) [2023-12-27 00:27:37,249][105620] Updated weights for policy 1, policy_version 1247298 (0.0006) [2023-12-27 00:27:37,287][105692] Updated weights for policy 0, policy_version 1245980 (0.0011) [2023-12-27 00:27:37,302][105620] Updated weights for policy 1, policy_version 1247308 (0.0005) [2023-12-27 00:27:37,342][105692] Updated weights for policy 0, policy_version 1245990 (0.0011) [2023-12-27 00:27:37,349][105620] Updated weights for policy 1, policy_version 1247318 (0.0010) [2023-12-27 00:27:37,390][105692] Updated weights for policy 0, policy_version 1246000 (0.0010) [2023-12-27 00:27:37,398][105620] Updated weights for policy 1, policy_version 1247328 (0.0010) [2023-12-27 00:27:38,081][105620] Updated weights for policy 1, policy_version 1247338 (0.0006) [2023-12-27 00:27:38,136][105620] Updated weights for policy 1, policy_version 1247348 (0.0010) [2023-12-27 00:27:38,145][105692] Updated weights for policy 0, policy_version 1246010 (0.0011) [2023-12-27 00:27:38,190][105620] Updated weights for policy 1, policy_version 1247358 (0.0011) [2023-12-27 00:27:38,202][105692] Updated weights for policy 0, policy_version 1246020 (0.0011) [2023-12-27 00:27:38,258][105692] Updated weights for policy 0, policy_version 1246030 (0.0011) [2023-12-27 00:27:38,908][105620] Updated weights for policy 1, policy_version 1247368 (0.0011) [2023-12-27 00:27:38,960][105620] Updated weights for policy 1, policy_version 1247378 (0.0010) [2023-12-27 00:27:39,008][105620] Updated weights for policy 1, policy_version 1247388 (0.0010) [2023-12-27 00:27:39,021][105692] Updated weights for policy 0, policy_version 1246040 (0.0010) [2023-12-27 00:27:39,072][105692] Updated weights for policy 0, policy_version 1246050 (0.0010) [2023-12-27 00:27:39,120][105692] Updated weights for policy 0, policy_version 1246060 (0.0010) [2023-12-27 00:27:39,788][105620] Updated weights for policy 1, policy_version 1247398 (0.0009) [2023-12-27 00:27:39,849][105692] Updated weights for policy 0, policy_version 1246070 (0.0010) [2023-12-27 00:27:39,856][105620] Updated weights for policy 1, policy_version 1247408 (0.0009) [2023-12-27 00:27:39,905][105692] Updated weights for policy 0, policy_version 1246080 (0.0010) [2023-12-27 00:27:39,912][105620] Updated weights for policy 1, policy_version 1247418 (0.0007) [2023-12-27 00:27:39,972][105692] Updated weights for policy 0, policy_version 1246090 (0.0008) [2023-12-27 00:27:40,684][105620] Updated weights for policy 1, policy_version 1247428 (0.0009) [2023-12-27 00:27:40,720][105692] Updated weights for policy 0, policy_version 1246100 (0.0009) [2023-12-27 00:27:40,739][105620] Updated weights for policy 1, policy_version 1247438 (0.0007) [2023-12-27 00:27:40,778][105692] Updated weights for policy 0, policy_version 1246110 (0.0007) [2023-12-27 00:27:40,804][105620] Updated weights for policy 1, policy_version 1247448 (0.0010) [2023-12-27 00:27:40,836][105692] Updated weights for policy 0, policy_version 1246120 (0.0007) [2023-12-27 00:27:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 638451712. Throughput: 0: 9453.0, 1: 9981.1. Samples: 638456480. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:27:41,063][104569] Avg episode reward: [(0, '8987.810'), (1, '8714.536')] [2023-12-27 00:27:41,599][105620] Updated weights for policy 1, policy_version 1247458 (0.0008) [2023-12-27 00:27:41,616][105692] Updated weights for policy 0, policy_version 1246130 (0.0008) [2023-12-27 00:27:41,666][105620] Updated weights for policy 1, policy_version 1247468 (0.0010) [2023-12-27 00:27:41,685][105692] Updated weights for policy 0, policy_version 1246140 (0.0009) [2023-12-27 00:27:41,726][105620] Updated weights for policy 1, policy_version 1247478 (0.0008) [2023-12-27 00:27:41,744][105692] Updated weights for policy 0, policy_version 1246150 (0.0009) [2023-12-27 00:27:41,789][105620] Updated weights for policy 1, policy_version 1247488 (0.0006) [2023-12-27 00:27:41,803][105692] Updated weights for policy 0, policy_version 1246160 (0.0008) [2023-12-27 00:27:42,540][105620] Updated weights for policy 1, policy_version 1247498 (0.0008) [2023-12-27 00:27:42,578][105692] Updated weights for policy 0, policy_version 1246170 (0.0009) [2023-12-27 00:27:42,593][105620] Updated weights for policy 1, policy_version 1247508 (0.0006) [2023-12-27 00:27:42,628][105692] Updated weights for policy 0, policy_version 1246180 (0.0007) [2023-12-27 00:27:42,643][105620] Updated weights for policy 1, policy_version 1247518 (0.0006) [2023-12-27 00:27:42,692][105692] Updated weights for policy 0, policy_version 1246190 (0.0008) [2023-12-27 00:27:43,359][105620] Updated weights for policy 1, policy_version 1247528 (0.0008) [2023-12-27 00:27:43,416][105692] Updated weights for policy 0, policy_version 1246200 (0.0006) [2023-12-27 00:27:43,420][105620] Updated weights for policy 1, policy_version 1247538 (0.0008) [2023-12-27 00:27:43,475][105692] Updated weights for policy 0, policy_version 1246210 (0.0008) [2023-12-27 00:27:43,486][105620] Updated weights for policy 1, policy_version 1247548 (0.0008) [2023-12-27 00:27:43,530][105692] Updated weights for policy 0, policy_version 1246220 (0.0007) [2023-12-27 00:27:44,126][105692] Updated weights for policy 0, policy_version 1246230 (0.0007) [2023-12-27 00:27:44,183][105692] Updated weights for policy 0, policy_version 1246240 (0.0006) [2023-12-27 00:27:44,249][105692] Updated weights for policy 0, policy_version 1246250 (0.0005) [2023-12-27 00:27:44,322][105620] Updated weights for policy 1, policy_version 1247558 (0.0008) [2023-12-27 00:27:44,383][105620] Updated weights for policy 1, policy_version 1247568 (0.0009) [2023-12-27 00:27:44,435][105620] Updated weights for policy 1, policy_version 1247578 (0.0008) [2023-12-27 00:27:44,899][105692] Updated weights for policy 0, policy_version 1246260 (0.0007) [2023-12-27 00:27:44,958][105692] Updated weights for policy 0, policy_version 1246270 (0.0011) [2023-12-27 00:27:45,024][105692] Updated weights for policy 0, policy_version 1246280 (0.0011) [2023-12-27 00:27:45,173][105620] Updated weights for policy 1, policy_version 1247588 (0.0009) [2023-12-27 00:27:45,236][105620] Updated weights for policy 1, policy_version 1247598 (0.0008) [2023-12-27 00:27:45,296][105620] Updated weights for policy 1, policy_version 1247608 (0.0008) [2023-12-27 00:27:45,672][105692] Updated weights for policy 0, policy_version 1246290 (0.0009) [2023-12-27 00:27:45,733][105692] Updated weights for policy 0, policy_version 1246300 (0.0009) [2023-12-27 00:27:45,791][105692] Updated weights for policy 0, policy_version 1246310 (0.0010) [2023-12-27 00:27:45,859][105692] Updated weights for policy 0, policy_version 1246320 (0.0005) [2023-12-27 00:27:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 638541824. Throughput: 0: 9490.3, 1: 9845.8. Samples: 638511356. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:27:46,062][104569] Avg episode reward: [(0, '8804.955'), (1, '8897.302')] [2023-12-27 00:27:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001246320_319111168.pth... [2023-12-27 00:27:46,070][105620] Updated weights for policy 1, policy_version 1247618 (0.0008) [2023-12-27 00:27:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001245200_318824448.pth [2023-12-27 00:27:46,134][105620] Updated weights for policy 1, policy_version 1247628 (0.0009) [2023-12-27 00:27:46,203][105620] Updated weights for policy 1, policy_version 1247638 (0.0011) [2023-12-27 00:27:46,267][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001247648_319438848.pth... [2023-12-27 00:27:46,267][105620] Updated weights for policy 1, policy_version 1247648 (0.0010) [2023-12-27 00:27:46,272][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001246496_319143936.pth [2023-12-27 00:27:46,404][105692] Updated weights for policy 0, policy_version 1246330 (0.0005) [2023-12-27 00:27:46,462][105692] Updated weights for policy 0, policy_version 1246340 (0.0005) [2023-12-27 00:27:46,516][105692] Updated weights for policy 0, policy_version 1246350 (0.0005) [2023-12-27 00:27:46,823][105620] Updated weights for policy 1, policy_version 1247658 (0.0005) [2023-12-27 00:27:46,879][105620] Updated weights for policy 1, policy_version 1247668 (0.0005) [2023-12-27 00:27:46,931][105620] Updated weights for policy 1, policy_version 1247678 (0.0005) [2023-12-27 00:27:47,101][105692] Updated weights for policy 0, policy_version 1246360 (0.0008) [2023-12-27 00:27:47,153][105692] Updated weights for policy 0, policy_version 1246370 (0.0010) [2023-12-27 00:27:47,204][105692] Updated weights for policy 0, policy_version 1246380 (0.0010) [2023-12-27 00:27:47,575][105620] Updated weights for policy 1, policy_version 1247688 (0.0009) [2023-12-27 00:27:47,633][105620] Updated weights for policy 1, policy_version 1247698 (0.0011) [2023-12-27 00:27:47,692][105620] Updated weights for policy 1, policy_version 1247708 (0.0010) [2023-12-27 00:27:47,824][105692] Updated weights for policy 0, policy_version 1246390 (0.0008) [2023-12-27 00:27:47,874][105692] Updated weights for policy 0, policy_version 1246400 (0.0008) [2023-12-27 00:27:47,922][105692] Updated weights for policy 0, policy_version 1246410 (0.0005) [2023-12-27 00:27:48,440][105620] Updated weights for policy 1, policy_version 1247718 (0.0011) [2023-12-27 00:27:48,490][105620] Updated weights for policy 1, policy_version 1247728 (0.0011) [2023-12-27 00:27:48,546][105620] Updated weights for policy 1, policy_version 1247738 (0.0011) [2023-12-27 00:27:48,599][105692] Updated weights for policy 0, policy_version 1246420 (0.0006) [2023-12-27 00:27:48,657][105692] Updated weights for policy 0, policy_version 1246430 (0.0006) [2023-12-27 00:27:48,719][105692] Updated weights for policy 0, policy_version 1246440 (0.0008) [2023-12-27 00:27:49,351][105620] Updated weights for policy 1, policy_version 1247748 (0.0011) [2023-12-27 00:27:49,424][105620] Updated weights for policy 1, policy_version 1247758 (0.0011) [2023-12-27 00:27:49,452][105692] Updated weights for policy 0, policy_version 1246450 (0.0011) [2023-12-27 00:27:49,476][105620] Updated weights for policy 1, policy_version 1247768 (0.0010) [2023-12-27 00:27:49,512][105692] Updated weights for policy 0, policy_version 1246460 (0.0010) [2023-12-27 00:27:49,571][105692] Updated weights for policy 0, policy_version 1246470 (0.0010) [2023-12-27 00:27:49,633][105692] Updated weights for policy 0, policy_version 1246480 (0.0011) [2023-12-27 00:27:50,222][105620] Updated weights for policy 1, policy_version 1247778 (0.0007) [2023-12-27 00:27:50,283][105620] Updated weights for policy 1, policy_version 1247788 (0.0010) [2023-12-27 00:27:50,343][105620] Updated weights for policy 1, policy_version 1247798 (0.0011) [2023-12-27 00:27:50,366][105692] Updated weights for policy 0, policy_version 1246490 (0.0007) [2023-12-27 00:27:50,397][105620] Updated weights for policy 1, policy_version 1247808 (0.0011) [2023-12-27 00:27:50,425][105692] Updated weights for policy 0, policy_version 1246500 (0.0007) [2023-12-27 00:27:50,486][105692] Updated weights for policy 0, policy_version 1246510 (0.0006) [2023-12-27 00:27:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 638640128. Throughput: 0: 9710.7, 1: 9815.8. Samples: 638634312. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:27:51,062][104569] Avg episode reward: [(0, '8713.680'), (1, '8988.008')] [2023-12-27 00:27:51,186][105620] Updated weights for policy 1, policy_version 1247818 (0.0006) [2023-12-27 00:27:51,204][105692] Updated weights for policy 0, policy_version 1246520 (0.0008) [2023-12-27 00:27:51,250][105620] Updated weights for policy 1, policy_version 1247828 (0.0010) [2023-12-27 00:27:51,268][105692] Updated weights for policy 0, policy_version 1246530 (0.0006) [2023-12-27 00:27:51,308][105620] Updated weights for policy 1, policy_version 1247838 (0.0011) [2023-12-27 00:27:51,327][105692] Updated weights for policy 0, policy_version 1246540 (0.0006) [2023-12-27 00:27:52,044][105692] Updated weights for policy 0, policy_version 1246550 (0.0009) [2023-12-27 00:27:52,054][105620] Updated weights for policy 1, policy_version 1247848 (0.0007) [2023-12-27 00:27:52,104][105692] Updated weights for policy 0, policy_version 1246560 (0.0009) [2023-12-27 00:27:52,112][105620] Updated weights for policy 1, policy_version 1247858 (0.0006) [2023-12-27 00:27:52,167][105692] Updated weights for policy 0, policy_version 1246570 (0.0005) [2023-12-27 00:27:52,170][105620] Updated weights for policy 1, policy_version 1247868 (0.0006) [2023-12-27 00:27:52,765][105692] Updated weights for policy 0, policy_version 1246580 (0.0006) [2023-12-27 00:27:52,805][105620] Updated weights for policy 1, policy_version 1247878 (0.0007) [2023-12-27 00:27:52,819][105692] Updated weights for policy 0, policy_version 1246590 (0.0005) [2023-12-27 00:27:52,867][105692] Updated weights for policy 0, policy_version 1246600 (0.0008) [2023-12-27 00:27:52,871][105620] Updated weights for policy 1, policy_version 1247888 (0.0008) [2023-12-27 00:27:52,939][105620] Updated weights for policy 1, policy_version 1247898 (0.0007) [2023-12-27 00:27:53,596][105692] Updated weights for policy 0, policy_version 1246610 (0.0010) [2023-12-27 00:27:53,634][105620] Updated weights for policy 1, policy_version 1247908 (0.0006) [2023-12-27 00:27:53,649][105692] Updated weights for policy 0, policy_version 1246620 (0.0009) [2023-12-27 00:27:53,684][105620] Updated weights for policy 1, policy_version 1247918 (0.0008) [2023-12-27 00:27:53,692][105692] Updated weights for policy 0, policy_version 1246630 (0.0005) [2023-12-27 00:27:53,743][105620] Updated weights for policy 1, policy_version 1247928 (0.0007) [2023-12-27 00:27:53,749][105692] Updated weights for policy 0, policy_version 1246640 (0.0011) [2023-12-27 00:27:54,399][105692] Updated weights for policy 0, policy_version 1246650 (0.0009) [2023-12-27 00:27:54,401][105620] Updated weights for policy 1, policy_version 1247938 (0.0008) [2023-12-27 00:27:54,452][105692] Updated weights for policy 0, policy_version 1246660 (0.0005) [2023-12-27 00:27:54,458][105620] Updated weights for policy 1, policy_version 1247948 (0.0009) [2023-12-27 00:27:54,505][105692] Updated weights for policy 0, policy_version 1246670 (0.0007) [2023-12-27 00:27:54,508][105620] Updated weights for policy 1, policy_version 1247958 (0.0006) [2023-12-27 00:27:54,557][105620] Updated weights for policy 1, policy_version 1247968 (0.0008) [2023-12-27 00:27:55,127][105620] Updated weights for policy 1, policy_version 1247978 (0.0006) [2023-12-27 00:27:55,181][105620] Updated weights for policy 1, policy_version 1247988 (0.0008) [2023-12-27 00:27:55,242][105620] Updated weights for policy 1, policy_version 1247998 (0.0008) [2023-12-27 00:27:55,399][105692] Updated weights for policy 0, policy_version 1246680 (0.0009) [2023-12-27 00:27:55,460][105692] Updated weights for policy 0, policy_version 1246690 (0.0010) [2023-12-27 00:27:55,516][105692] Updated weights for policy 0, policy_version 1246700 (0.0008) [2023-12-27 00:27:55,849][105620] Updated weights for policy 1, policy_version 1248008 (0.0006) [2023-12-27 00:27:55,912][105620] Updated weights for policy 1, policy_version 1248018 (0.0007) [2023-12-27 00:27:55,974][105620] Updated weights for policy 1, policy_version 1248028 (0.0009) [2023-12-27 00:27:56,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 638746624. Throughput: 0: 9775.4, 1: 9766.0. Samples: 638753208. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:27:56,062][104569] Avg episode reward: [(0, '8620.613'), (1, '8988.554')] [2023-12-27 00:27:56,232][105692] Updated weights for policy 0, policy_version 1246710 (0.0009) [2023-12-27 00:27:56,291][105692] Updated weights for policy 0, policy_version 1246720 (0.0009) [2023-12-27 00:27:56,345][105692] Updated weights for policy 0, policy_version 1246730 (0.0009) [2023-12-27 00:27:56,671][105620] Updated weights for policy 1, policy_version 1248038 (0.0007) [2023-12-27 00:27:56,726][105620] Updated weights for policy 1, policy_version 1248048 (0.0007) [2023-12-27 00:27:56,787][105620] Updated weights for policy 1, policy_version 1248058 (0.0009) [2023-12-27 00:27:57,116][105692] Updated weights for policy 0, policy_version 1246740 (0.0009) [2023-12-27 00:27:57,174][105692] Updated weights for policy 0, policy_version 1246750 (0.0008) [2023-12-27 00:27:57,228][105692] Updated weights for policy 0, policy_version 1246760 (0.0008) [2023-12-27 00:27:57,461][105620] Updated weights for policy 1, policy_version 1248068 (0.0008) [2023-12-27 00:27:57,522][105620] Updated weights for policy 1, policy_version 1248078 (0.0010) [2023-12-27 00:27:57,569][105620] Updated weights for policy 1, policy_version 1248088 (0.0006) [2023-12-27 00:27:58,041][105692] Updated weights for policy 0, policy_version 1246770 (0.0008) [2023-12-27 00:27:58,108][105692] Updated weights for policy 0, policy_version 1246781 (0.0010) [2023-12-27 00:27:58,143][105620] Updated weights for policy 1, policy_version 1248098 (0.0005) [2023-12-27 00:27:58,162][105692] Updated weights for policy 0, policy_version 1246791 (0.0009) [2023-12-27 00:27:58,203][105620] Updated weights for policy 1, policy_version 1248108 (0.0007) [2023-12-27 00:27:58,266][105620] Updated weights for policy 1, policy_version 1248118 (0.0007) [2023-12-27 00:27:58,335][105620] Updated weights for policy 1, policy_version 1248128 (0.0007) [2023-12-27 00:27:58,952][105692] Updated weights for policy 0, policy_version 1246801 (0.0009) [2023-12-27 00:27:59,014][105692] Updated weights for policy 0, policy_version 1246811 (0.0008) [2023-12-27 00:27:59,068][105692] Updated weights for policy 0, policy_version 1246821 (0.0009) [2023-12-27 00:27:59,112][105620] Updated weights for policy 1, policy_version 1248138 (0.0011) [2023-12-27 00:27:59,128][105692] Updated weights for policy 0, policy_version 1246831 (0.0008) [2023-12-27 00:27:59,182][105620] Updated weights for policy 1, policy_version 1248148 (0.0010) [2023-12-27 00:27:59,242][105620] Updated weights for policy 1, policy_version 1248158 (0.0008) [2023-12-27 00:27:59,926][105692] Updated weights for policy 0, policy_version 1246841 (0.0006) [2023-12-27 00:27:59,983][105692] Updated weights for policy 0, policy_version 1246851 (0.0008) [2023-12-27 00:27:59,990][105620] Updated weights for policy 1, policy_version 1248168 (0.0008) [2023-12-27 00:28:00,034][105692] Updated weights for policy 0, policy_version 1246861 (0.0007) [2023-12-27 00:28:00,052][105620] Updated weights for policy 1, policy_version 1248178 (0.0011) [2023-12-27 00:28:00,104][105620] Updated weights for policy 1, policy_version 1248188 (0.0010) [2023-12-27 00:28:00,696][105620] Updated weights for policy 1, policy_version 1248198 (0.0009) [2023-12-27 00:28:00,753][105620] Updated weights for policy 1, policy_version 1248208 (0.0007) [2023-12-27 00:28:00,802][105620] Updated weights for policy 1, policy_version 1248218 (0.0005) [2023-12-27 00:28:00,836][105692] Updated weights for policy 0, policy_version 1246871 (0.0009) [2023-12-27 00:28:00,881][105692] Updated weights for policy 0, policy_version 1246881 (0.0010) [2023-12-27 00:28:00,938][105692] Updated weights for policy 0, policy_version 1246891 (0.0010) [2023-12-27 00:28:01,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 638844928. Throughput: 0: 9744.5, 1: 9824.2. Samples: 638811252. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:28:01,063][104569] Avg episode reward: [(0, '8804.134'), (1, '9079.860')] [2023-12-27 00:28:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001246896_319258624.pth... [2023-12-27 00:28:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001248224_319586304.pth... [2023-12-27 00:28:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001245776_318971904.pth [2023-12-27 00:28:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001247072_319291392.pth [2023-12-27 00:28:01,520][105620] Updated weights for policy 1, policy_version 1248228 (0.0007) [2023-12-27 00:28:01,570][105620] Updated weights for policy 1, policy_version 1248238 (0.0009) [2023-12-27 00:28:01,625][105620] Updated weights for policy 1, policy_version 1248248 (0.0009) [2023-12-27 00:28:01,708][105692] Updated weights for policy 0, policy_version 1246901 (0.0009) [2023-12-27 00:28:01,770][105692] Updated weights for policy 0, policy_version 1246911 (0.0009) [2023-12-27 00:28:01,829][105692] Updated weights for policy 0, policy_version 1246921 (0.0009) [2023-12-27 00:28:02,386][105620] Updated weights for policy 1, policy_version 1248258 (0.0008) [2023-12-27 00:28:02,440][105620] Updated weights for policy 1, policy_version 1248268 (0.0008) [2023-12-27 00:28:02,488][105620] Updated weights for policy 1, policy_version 1248278 (0.0008) [2023-12-27 00:28:02,542][105620] Updated weights for policy 1, policy_version 1248288 (0.0009) [2023-12-27 00:28:02,590][105692] Updated weights for policy 0, policy_version 1246931 (0.0009) [2023-12-27 00:28:02,653][105692] Updated weights for policy 0, policy_version 1246941 (0.0009) [2023-12-27 00:28:02,705][105692] Updated weights for policy 0, policy_version 1246951 (0.0009) [2023-12-27 00:28:03,296][105620] Updated weights for policy 1, policy_version 1248298 (0.0008) [2023-12-27 00:28:03,346][105620] Updated weights for policy 1, policy_version 1248308 (0.0009) [2023-12-27 00:28:03,392][105620] Updated weights for policy 1, policy_version 1248318 (0.0009) [2023-12-27 00:28:03,467][105692] Updated weights for policy 0, policy_version 1246961 (0.0009) [2023-12-27 00:28:03,518][105692] Updated weights for policy 0, policy_version 1246971 (0.0009) [2023-12-27 00:28:03,565][105692] Updated weights for policy 0, policy_version 1246981 (0.0009) [2023-12-27 00:28:03,612][105692] Updated weights for policy 0, policy_version 1246992 (0.0009) [2023-12-27 00:28:04,085][105620] Updated weights for policy 1, policy_version 1248328 (0.0009) [2023-12-27 00:28:04,148][105620] Updated weights for policy 1, policy_version 1248338 (0.0009) [2023-12-27 00:28:04,214][105620] Updated weights for policy 1, policy_version 1248348 (0.0009) [2023-12-27 00:28:04,455][105692] Updated weights for policy 0, policy_version 1247002 (0.0009) [2023-12-27 00:28:04,515][105692] Updated weights for policy 0, policy_version 1247012 (0.0009) [2023-12-27 00:28:04,565][105692] Updated weights for policy 0, policy_version 1247022 (0.0009) [2023-12-27 00:28:04,961][105620] Updated weights for policy 1, policy_version 1248358 (0.0008) [2023-12-27 00:28:05,016][105620] Updated weights for policy 1, policy_version 1248368 (0.0005) [2023-12-27 00:28:05,069][105620] Updated weights for policy 1, policy_version 1248378 (0.0005) [2023-12-27 00:28:05,364][105692] Updated weights for policy 0, policy_version 1247032 (0.0009) [2023-12-27 00:28:05,415][105692] Updated weights for policy 0, policy_version 1247042 (0.0009) [2023-12-27 00:28:05,462][105692] Updated weights for policy 0, policy_version 1247052 (0.0009) [2023-12-27 00:28:05,725][105620] Updated weights for policy 1, policy_version 1248388 (0.0007) [2023-12-27 00:28:05,782][105620] Updated weights for policy 1, policy_version 1248398 (0.0008) [2023-12-27 00:28:05,840][105620] Updated weights for policy 1, policy_version 1248408 (0.0009) [2023-12-27 00:28:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 638935040. Throughput: 0: 9543.0, 1: 9940.4. Samples: 638923124. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:28:06,062][104569] Avg episode reward: [(0, '8809.521'), (1, '9170.680')] [2023-12-27 00:28:06,288][105692] Updated weights for policy 0, policy_version 1247062 (0.0009) [2023-12-27 00:28:06,350][105692] Updated weights for policy 0, policy_version 1247072 (0.0008) [2023-12-27 00:28:06,416][105692] Updated weights for policy 0, policy_version 1247082 (0.0006) [2023-12-27 00:28:06,578][105620] Updated weights for policy 1, policy_version 1248418 (0.0009) [2023-12-27 00:28:06,640][105620] Updated weights for policy 1, policy_version 1248428 (0.0010) [2023-12-27 00:28:06,696][105620] Updated weights for policy 1, policy_version 1248438 (0.0010) [2023-12-27 00:28:06,746][105620] Updated weights for policy 1, policy_version 1248448 (0.0008) [2023-12-27 00:28:07,082][105692] Updated weights for policy 0, policy_version 1247092 (0.0007) [2023-12-27 00:28:07,137][105692] Updated weights for policy 0, policy_version 1247102 (0.0009) [2023-12-27 00:28:07,185][105692] Updated weights for policy 0, policy_version 1247112 (0.0008) [2023-12-27 00:28:07,469][105620] Updated weights for policy 1, policy_version 1248458 (0.0008) [2023-12-27 00:28:07,527][105620] Updated weights for policy 1, policy_version 1248468 (0.0008) [2023-12-27 00:28:07,583][105620] Updated weights for policy 1, policy_version 1248478 (0.0008) [2023-12-27 00:28:08,003][105692] Updated weights for policy 0, policy_version 1247122 (0.0009) [2023-12-27 00:28:08,064][105692] Updated weights for policy 0, policy_version 1247132 (0.0009) [2023-12-27 00:28:08,122][105692] Updated weights for policy 0, policy_version 1247142 (0.0009) [2023-12-27 00:28:08,169][105692] Updated weights for policy 0, policy_version 1247152 (0.0009) [2023-12-27 00:28:08,287][105620] Updated weights for policy 1, policy_version 1248488 (0.0009) [2023-12-27 00:28:08,348][105620] Updated weights for policy 1, policy_version 1248498 (0.0009) [2023-12-27 00:28:08,405][105620] Updated weights for policy 1, policy_version 1248508 (0.0007) [2023-12-27 00:28:08,996][105692] Updated weights for policy 0, policy_version 1247162 (0.0008) [2023-12-27 00:28:09,031][105585] KL-divergence is very high: 128.8746 [2023-12-27 00:28:09,037][105620] Updated weights for policy 1, policy_version 1248518 (0.0007) [2023-12-27 00:28:09,054][105692] Updated weights for policy 0, policy_version 1247172 (0.0010) [2023-12-27 00:28:09,075][105585] KL-divergence is very high: 147.4810 [2023-12-27 00:28:09,089][105620] Updated weights for policy 1, policy_version 1248528 (0.0006) [2023-12-27 00:28:09,104][105692] Updated weights for policy 0, policy_version 1247182 (0.0007) [2023-12-27 00:28:09,140][105620] Updated weights for policy 1, policy_version 1248538 (0.0008) [2023-12-27 00:28:09,866][105692] Updated weights for policy 0, policy_version 1247192 (0.0008) [2023-12-27 00:28:09,884][105620] Updated weights for policy 1, policy_version 1248548 (0.0009) [2023-12-27 00:28:09,928][105692] Updated weights for policy 0, policy_version 1247202 (0.0008) [2023-12-27 00:28:09,952][105620] Updated weights for policy 1, policy_version 1248558 (0.0006) [2023-12-27 00:28:09,994][105692] Updated weights for policy 0, policy_version 1247212 (0.0009) [2023-12-27 00:28:10,014][105620] Updated weights for policy 1, policy_version 1248568 (0.0010) [2023-12-27 00:28:10,761][105620] Updated weights for policy 1, policy_version 1248578 (0.0009) [2023-12-27 00:28:10,771][105692] Updated weights for policy 0, policy_version 1247222 (0.0009) [2023-12-27 00:28:10,817][105620] Updated weights for policy 1, policy_version 1248588 (0.0007) [2023-12-27 00:28:10,831][105692] Updated weights for policy 0, policy_version 1247232 (0.0006) [2023-12-27 00:28:10,878][105620] Updated weights for policy 1, policy_version 1248598 (0.0007) [2023-12-27 00:28:10,892][105692] Updated weights for policy 0, policy_version 1247242 (0.0006) [2023-12-27 00:28:10,941][105620] Updated weights for policy 1, policy_version 1248608 (0.0007) [2023-12-27 00:28:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 639033344. Throughput: 0: 9479.2, 1: 9904.3. Samples: 639036324. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:28:11,063][104569] Avg episode reward: [(0, '8172.616'), (1, '9170.664')] [2023-12-27 00:28:11,659][105620] Updated weights for policy 1, policy_version 1248618 (0.0008) [2023-12-27 00:28:11,729][105620] Updated weights for policy 1, policy_version 1248628 (0.0009) [2023-12-27 00:28:11,748][105692] Updated weights for policy 0, policy_version 1247252 (0.0008) [2023-12-27 00:28:11,791][105620] Updated weights for policy 1, policy_version 1248638 (0.0008) [2023-12-27 00:28:11,804][105692] Updated weights for policy 0, policy_version 1247262 (0.0008) [2023-12-27 00:28:11,864][105692] Updated weights for policy 0, policy_version 1247272 (0.0008) [2023-12-27 00:28:12,536][105692] Updated weights for policy 0, policy_version 1247282 (0.0008) [2023-12-27 00:28:12,592][105692] Updated weights for policy 0, policy_version 1247292 (0.0008) [2023-12-27 00:28:12,599][105620] Updated weights for policy 1, policy_version 1248648 (0.0007) [2023-12-27 00:28:12,648][105692] Updated weights for policy 0, policy_version 1247302 (0.0006) [2023-12-27 00:28:12,658][105620] Updated weights for policy 1, policy_version 1248658 (0.0006) [2023-12-27 00:28:12,702][105692] Updated weights for policy 0, policy_version 1247312 (0.0007) [2023-12-27 00:28:12,725][105620] Updated weights for policy 1, policy_version 1248668 (0.0006) [2023-12-27 00:28:13,264][105692] Updated weights for policy 0, policy_version 1247322 (0.0009) [2023-12-27 00:28:13,316][105692] Updated weights for policy 0, policy_version 1247332 (0.0010) [2023-12-27 00:28:13,374][105692] Updated weights for policy 0, policy_version 1247342 (0.0010) [2023-12-27 00:28:13,431][105620] Updated weights for policy 1, policy_version 1248678 (0.0009) [2023-12-27 00:28:13,484][105620] Updated weights for policy 1, policy_version 1248688 (0.0008) [2023-12-27 00:28:13,538][105620] Updated weights for policy 1, policy_version 1248698 (0.0007) [2023-12-27 00:28:14,036][105692] Updated weights for policy 0, policy_version 1247352 (0.0009) [2023-12-27 00:28:14,091][105692] Updated weights for policy 0, policy_version 1247362 (0.0006) [2023-12-27 00:28:14,146][105692] Updated weights for policy 0, policy_version 1247372 (0.0008) [2023-12-27 00:28:14,285][105620] Updated weights for policy 1, policy_version 1248708 (0.0011) [2023-12-27 00:28:14,344][105620] Updated weights for policy 1, policy_version 1248718 (0.0010) [2023-12-27 00:28:14,402][105620] Updated weights for policy 1, policy_version 1248728 (0.0009) [2023-12-27 00:28:14,857][105692] Updated weights for policy 0, policy_version 1247382 (0.0008) [2023-12-27 00:28:14,916][105692] Updated weights for policy 0, policy_version 1247392 (0.0006) [2023-12-27 00:28:14,972][105692] Updated weights for policy 0, policy_version 1247402 (0.0008) [2023-12-27 00:28:15,153][105620] Updated weights for policy 1, policy_version 1248738 (0.0011) [2023-12-27 00:28:15,217][105620] Updated weights for policy 1, policy_version 1248748 (0.0011) [2023-12-27 00:28:15,285][105620] Updated weights for policy 1, policy_version 1248758 (0.0011) [2023-12-27 00:28:15,341][105620] Updated weights for policy 1, policy_version 1248768 (0.0011) [2023-12-27 00:28:15,713][105692] Updated weights for policy 0, policy_version 1247412 (0.0008) [2023-12-27 00:28:15,758][105692] Updated weights for policy 0, policy_version 1247422 (0.0008) [2023-12-27 00:28:15,806][105692] Updated weights for policy 0, policy_version 1247432 (0.0009) [2023-12-27 00:28:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 639123456. Throughput: 0: 9460.3, 1: 9878.9. Samples: 639094244. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:28:16,063][104569] Avg episode reward: [(0, '8176.884'), (1, '9170.068')] [2023-12-27 00:28:16,066][105620] Updated weights for policy 1, policy_version 1248778 (0.0007) [2023-12-27 00:28:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001247440_319397888.pth... [2023-12-27 00:28:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001246320_319111168.pth [2023-12-27 00:28:16,123][105620] Updated weights for policy 1, policy_version 1248788 (0.0009) [2023-12-27 00:28:16,188][105620] Updated weights for policy 1, policy_version 1248798 (0.0007) [2023-12-27 00:28:16,200][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001248800_319733760.pth... [2023-12-27 00:28:16,203][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001247648_319438848.pth [2023-12-27 00:28:16,607][105692] Updated weights for policy 0, policy_version 1247442 (0.0008) [2023-12-27 00:28:16,655][105692] Updated weights for policy 0, policy_version 1247452 (0.0008) [2023-12-27 00:28:16,709][105692] Updated weights for policy 0, policy_version 1247462 (0.0005) [2023-12-27 00:28:16,763][105692] Updated weights for policy 0, policy_version 1247472 (0.0005) [2023-12-27 00:28:16,854][105620] Updated weights for policy 1, policy_version 1248808 (0.0010) [2023-12-27 00:28:16,906][105620] Updated weights for policy 1, policy_version 1248818 (0.0010) [2023-12-27 00:28:16,955][105620] Updated weights for policy 1, policy_version 1248828 (0.0010) [2023-12-27 00:28:17,386][105692] Updated weights for policy 0, policy_version 1247482 (0.0005) [2023-12-27 00:28:17,449][105692] Updated weights for policy 0, policy_version 1247492 (0.0005) [2023-12-27 00:28:17,500][105692] Updated weights for policy 0, policy_version 1247502 (0.0005) [2023-12-27 00:28:17,690][105620] Updated weights for policy 1, policy_version 1248838 (0.0010) [2023-12-27 00:28:17,742][105620] Updated weights for policy 1, policy_version 1248848 (0.0010) [2023-12-27 00:28:17,793][105620] Updated weights for policy 1, policy_version 1248858 (0.0010) [2023-12-27 00:28:18,140][105692] Updated weights for policy 0, policy_version 1247512 (0.0007) [2023-12-27 00:28:18,192][105692] Updated weights for policy 0, policy_version 1247522 (0.0008) [2023-12-27 00:28:18,236][105692] Updated weights for policy 0, policy_version 1247532 (0.0008) [2023-12-27 00:28:18,523][105620] Updated weights for policy 1, policy_version 1248868 (0.0008) [2023-12-27 00:28:18,580][105620] Updated weights for policy 1, policy_version 1248878 (0.0005) [2023-12-27 00:28:18,645][105620] Updated weights for policy 1, policy_version 1248888 (0.0009) [2023-12-27 00:28:18,938][105692] Updated weights for policy 0, policy_version 1247542 (0.0006) [2023-12-27 00:28:18,999][105692] Updated weights for policy 0, policy_version 1247552 (0.0005) [2023-12-27 00:28:19,056][105692] Updated weights for policy 0, policy_version 1247562 (0.0006) [2023-12-27 00:28:19,328][105620] Updated weights for policy 1, policy_version 1248898 (0.0010) [2023-12-27 00:28:19,398][105620] Updated weights for policy 1, policy_version 1248908 (0.0009) [2023-12-27 00:28:19,452][105620] Updated weights for policy 1, policy_version 1248918 (0.0010) [2023-12-27 00:28:19,513][105620] Updated weights for policy 1, policy_version 1248928 (0.0010) [2023-12-27 00:28:19,753][105692] Updated weights for policy 0, policy_version 1247572 (0.0007) [2023-12-27 00:28:19,818][105692] Updated weights for policy 0, policy_version 1247582 (0.0008) [2023-12-27 00:28:19,887][105692] Updated weights for policy 0, policy_version 1247593 (0.0010) [2023-12-27 00:28:20,236][105620] Updated weights for policy 1, policy_version 1248938 (0.0009) [2023-12-27 00:28:20,295][105620] Updated weights for policy 1, policy_version 1248948 (0.0008) [2023-12-27 00:28:20,349][105620] Updated weights for policy 1, policy_version 1248958 (0.0008) [2023-12-27 00:28:20,653][105692] Updated weights for policy 0, policy_version 1247603 (0.0009) [2023-12-27 00:28:20,712][105692] Updated weights for policy 0, policy_version 1247613 (0.0009) [2023-12-27 00:28:20,768][105692] Updated weights for policy 0, policy_version 1247623 (0.0009) [2023-12-27 00:28:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 639221760. Throughput: 0: 9600.9, 1: 9750.2. Samples: 639212736. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:28:21,062][104569] Avg episode reward: [(0, '8718.146'), (1, '9260.377')] [2023-12-27 00:28:21,160][105620] Updated weights for policy 1, policy_version 1248968 (0.0009) [2023-12-27 00:28:21,223][105620] Updated weights for policy 1, policy_version 1248978 (0.0009) [2023-12-27 00:28:21,287][105620] Updated weights for policy 1, policy_version 1248988 (0.0009) [2023-12-27 00:28:21,532][105692] Updated weights for policy 0, policy_version 1247633 (0.0009) [2023-12-27 00:28:21,605][105692] Updated weights for policy 0, policy_version 1247643 (0.0007) [2023-12-27 00:28:21,676][105692] Updated weights for policy 0, policy_version 1247653 (0.0009) [2023-12-27 00:28:21,742][105692] Updated weights for policy 0, policy_version 1247663 (0.0009) [2023-12-27 00:28:22,024][105620] Updated weights for policy 1, policy_version 1248998 (0.0009) [2023-12-27 00:28:22,077][105620] Updated weights for policy 1, policy_version 1249008 (0.0009) [2023-12-27 00:28:22,126][105620] Updated weights for policy 1, policy_version 1249018 (0.0008) [2023-12-27 00:28:22,503][105692] Updated weights for policy 0, policy_version 1247673 (0.0011) [2023-12-27 00:28:22,567][105692] Updated weights for policy 0, policy_version 1247683 (0.0010) [2023-12-27 00:28:22,627][105692] Updated weights for policy 0, policy_version 1247693 (0.0011) [2023-12-27 00:28:22,964][105620] Updated weights for policy 1, policy_version 1249028 (0.0008) [2023-12-27 00:28:23,018][105620] Updated weights for policy 1, policy_version 1249038 (0.0008) [2023-12-27 00:28:23,081][105620] Updated weights for policy 1, policy_version 1249048 (0.0008) [2023-12-27 00:28:23,377][105692] Updated weights for policy 0, policy_version 1247703 (0.0011) [2023-12-27 00:28:23,429][105585] KL-divergence is very high: 110.8793 [2023-12-27 00:28:23,436][105692] Updated weights for policy 0, policy_version 1247713 (0.0010) [2023-12-27 00:28:23,454][105585] KL-divergence is very high: 111.7295 [2023-12-27 00:28:23,498][105692] Updated weights for policy 0, policy_version 1247723 (0.0010) [2023-12-27 00:28:23,894][105620] Updated weights for policy 1, policy_version 1249058 (0.0008) [2023-12-27 00:28:23,959][105620] Updated weights for policy 1, policy_version 1249068 (0.0009) [2023-12-27 00:28:24,033][105620] Updated weights for policy 1, policy_version 1249078 (0.0009) [2023-12-27 00:28:24,062][105692] Updated weights for policy 0, policy_version 1247733 (0.0008) [2023-12-27 00:28:24,095][105620] Updated weights for policy 1, policy_version 1249088 (0.0008) [2023-12-27 00:28:24,132][105692] Updated weights for policy 0, policy_version 1247743 (0.0005) [2023-12-27 00:28:24,184][105692] Updated weights for policy 0, policy_version 1247753 (0.0005) [2023-12-27 00:28:24,729][105692] Updated weights for policy 0, policy_version 1247763 (0.0005) [2023-12-27 00:28:24,782][105692] Updated weights for policy 0, policy_version 1247773 (0.0005) [2023-12-27 00:28:24,846][105692] Updated weights for policy 0, policy_version 1247783 (0.0005) [2023-12-27 00:28:24,946][105620] Updated weights for policy 1, policy_version 1249098 (0.0010) [2023-12-27 00:28:24,999][105620] Updated weights for policy 1, policy_version 1249108 (0.0010) [2023-12-27 00:28:25,052][105620] Updated weights for policy 1, policy_version 1249118 (0.0010) [2023-12-27 00:28:25,363][105692] Updated weights for policy 0, policy_version 1247793 (0.0006) [2023-12-27 00:28:25,411][105692] Updated weights for policy 0, policy_version 1247803 (0.0010) [2023-12-27 00:28:25,455][105692] Updated weights for policy 0, policy_version 1247813 (0.0010) [2023-12-27 00:28:25,499][105692] Updated weights for policy 0, policy_version 1247823 (0.0010) [2023-12-27 00:28:25,860][105620] Updated weights for policy 1, policy_version 1249128 (0.0008) [2023-12-27 00:28:25,915][105620] Updated weights for policy 1, policy_version 1249138 (0.0006) [2023-12-27 00:28:25,970][105620] Updated weights for policy 1, policy_version 1249148 (0.0005) [2023-12-27 00:28:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 639320064. Throughput: 0: 9700.2, 1: 9623.5. Samples: 639326052. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:28:26,062][104569] Avg episode reward: [(0, '8069.552'), (1, '9260.662')] [2023-12-27 00:28:26,282][105692] Updated weights for policy 0, policy_version 1247833 (0.0010) [2023-12-27 00:28:26,343][105692] Updated weights for policy 0, policy_version 1247843 (0.0010) [2023-12-27 00:28:26,400][105692] Updated weights for policy 0, policy_version 1247853 (0.0010) [2023-12-27 00:28:26,699][105620] Updated weights for policy 1, policy_version 1249158 (0.0008) [2023-12-27 00:28:26,744][105620] Updated weights for policy 1, policy_version 1249168 (0.0007) [2023-12-27 00:28:26,789][105620] Updated weights for policy 1, policy_version 1249178 (0.0008) [2023-12-27 00:28:27,117][105692] Updated weights for policy 0, policy_version 1247863 (0.0011) [2023-12-27 00:28:27,171][105692] Updated weights for policy 0, policy_version 1247873 (0.0010) [2023-12-27 00:28:27,217][105692] Updated weights for policy 0, policy_version 1247883 (0.0011) [2023-12-27 00:28:27,599][105620] Updated weights for policy 1, policy_version 1249188 (0.0008) [2023-12-27 00:28:27,655][105620] Updated weights for policy 1, policy_version 1249198 (0.0008) [2023-12-27 00:28:27,706][105620] Updated weights for policy 1, policy_version 1249208 (0.0008) [2023-12-27 00:28:27,952][105692] Updated weights for policy 0, policy_version 1247893 (0.0008) [2023-12-27 00:28:27,997][105692] Updated weights for policy 0, policy_version 1247903 (0.0005) [2023-12-27 00:28:28,054][105692] Updated weights for policy 0, policy_version 1247913 (0.0010) [2023-12-27 00:28:28,468][105620] Updated weights for policy 1, policy_version 1249218 (0.0008) [2023-12-27 00:28:28,522][105620] Updated weights for policy 1, policy_version 1249228 (0.0007) [2023-12-27 00:28:28,569][105620] Updated weights for policy 1, policy_version 1249238 (0.0008) [2023-12-27 00:28:28,617][105620] Updated weights for policy 1, policy_version 1249248 (0.0008) [2023-12-27 00:28:28,789][105692] Updated weights for policy 0, policy_version 1247923 (0.0010) [2023-12-27 00:28:28,851][105692] Updated weights for policy 0, policy_version 1247933 (0.0010) [2023-12-27 00:28:28,906][105692] Updated weights for policy 0, policy_version 1247943 (0.0010) [2023-12-27 00:28:29,395][105620] Updated weights for policy 1, policy_version 1249258 (0.0009) [2023-12-27 00:28:29,439][105620] Updated weights for policy 1, policy_version 1249268 (0.0008) [2023-12-27 00:28:29,487][105620] Updated weights for policy 1, policy_version 1249278 (0.0008) [2023-12-27 00:28:29,673][105692] Updated weights for policy 0, policy_version 1247953 (0.0010) [2023-12-27 00:28:29,731][105692] Updated weights for policy 0, policy_version 1247963 (0.0011) [2023-12-27 00:28:29,787][105692] Updated weights for policy 0, policy_version 1247973 (0.0010) [2023-12-27 00:28:29,847][105692] Updated weights for policy 0, policy_version 1247983 (0.0008) [2023-12-27 00:28:30,259][105620] Updated weights for policy 1, policy_version 1249288 (0.0007) [2023-12-27 00:28:30,314][105620] Updated weights for policy 1, policy_version 1249298 (0.0010) [2023-12-27 00:28:30,365][105620] Updated weights for policy 1, policy_version 1249308 (0.0010) [2023-12-27 00:28:30,530][105692] Updated weights for policy 0, policy_version 1247993 (0.0010) [2023-12-27 00:28:30,574][105692] Updated weights for policy 0, policy_version 1248003 (0.0010) [2023-12-27 00:28:30,632][105692] Updated weights for policy 0, policy_version 1248013 (0.0010) [2023-12-27 00:28:30,964][105620] Updated weights for policy 1, policy_version 1249318 (0.0006) [2023-12-27 00:28:31,030][105620] Updated weights for policy 1, policy_version 1249328 (0.0008) [2023-12-27 00:28:31,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 639410176. Throughput: 0: 9728.3, 1: 9638.4. Samples: 639382860. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:28:31,063][104569] Avg episode reward: [(0, '7796.298'), (1, '9261.593')] [2023-12-27 00:28:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001248016_319545344.pth... [2023-12-27 00:28:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001246896_319258624.pth [2023-12-27 00:28:31,090][105620] Updated weights for policy 1, policy_version 1249338 (0.0010) [2023-12-27 00:28:31,126][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001249344_319873024.pth... [2023-12-27 00:28:31,130][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001248224_319586304.pth [2023-12-27 00:28:31,381][105692] Updated weights for policy 0, policy_version 1248023 (0.0008) [2023-12-27 00:28:31,435][105692] Updated weights for policy 0, policy_version 1248033 (0.0009) [2023-12-27 00:28:31,486][105692] Updated weights for policy 0, policy_version 1248043 (0.0010) [2023-12-27 00:28:31,689][105620] Updated weights for policy 1, policy_version 1249348 (0.0012) [2023-12-27 00:28:31,759][105620] Updated weights for policy 1, policy_version 1249358 (0.0011) [2023-12-27 00:28:31,818][105620] Updated weights for policy 1, policy_version 1249368 (0.0011) [2023-12-27 00:28:32,154][105692] Updated weights for policy 0, policy_version 1248053 (0.0006) [2023-12-27 00:28:32,207][105692] Updated weights for policy 0, policy_version 1248063 (0.0005) [2023-12-27 00:28:32,261][105692] Updated weights for policy 0, policy_version 1248073 (0.0006) [2023-12-27 00:28:32,466][105620] Updated weights for policy 1, policy_version 1249378 (0.0010) [2023-12-27 00:28:32,523][105620] Updated weights for policy 1, policy_version 1249388 (0.0005) [2023-12-27 00:28:32,584][105620] Updated weights for policy 1, policy_version 1249398 (0.0010) [2023-12-27 00:28:32,642][105620] Updated weights for policy 1, policy_version 1249408 (0.0010) [2023-12-27 00:28:32,996][105692] Updated weights for policy 0, policy_version 1248083 (0.0006) [2023-12-27 00:28:33,052][105692] Updated weights for policy 0, policy_version 1248093 (0.0005) [2023-12-27 00:28:33,105][105692] Updated weights for policy 0, policy_version 1248103 (0.0005) [2023-12-27 00:28:33,278][105620] Updated weights for policy 1, policy_version 1249418 (0.0010) [2023-12-27 00:28:33,333][105620] Updated weights for policy 1, policy_version 1249428 (0.0010) [2023-12-27 00:28:33,394][105620] Updated weights for policy 1, policy_version 1249438 (0.0010) [2023-12-27 00:28:33,715][105692] Updated weights for policy 0, policy_version 1248113 (0.0006) [2023-12-27 00:28:33,764][105692] Updated weights for policy 0, policy_version 1248123 (0.0010) [2023-12-27 00:28:33,825][105692] Updated weights for policy 0, policy_version 1248133 (0.0008) [2023-12-27 00:28:33,889][105692] Updated weights for policy 0, policy_version 1248143 (0.0008) [2023-12-27 00:28:34,015][105620] Updated weights for policy 1, policy_version 1249448 (0.0006) [2023-12-27 00:28:34,058][105620] Updated weights for policy 1, policy_version 1249458 (0.0005) [2023-12-27 00:28:34,109][105620] Updated weights for policy 1, policy_version 1249468 (0.0005) [2023-12-27 00:28:34,538][105692] Updated weights for policy 0, policy_version 1248153 (0.0007) [2023-12-27 00:28:34,589][105692] Updated weights for policy 0, policy_version 1248163 (0.0008) [2023-12-27 00:28:34,642][105692] Updated weights for policy 0, policy_version 1248173 (0.0008) [2023-12-27 00:28:34,642][105585] KL-divergence is very high: 237.5931 [2023-12-27 00:28:34,839][105620] Updated weights for policy 1, policy_version 1249478 (0.0008) [2023-12-27 00:28:34,898][105620] Updated weights for policy 1, policy_version 1249488 (0.0011) [2023-12-27 00:28:34,960][105620] Updated weights for policy 1, policy_version 1249498 (0.0010) [2023-12-27 00:28:35,219][105585] KL-divergence is very high: 172.5650 [2023-12-27 00:28:35,233][105585] KL-divergence is very high: 280.2040 [2023-12-27 00:28:35,263][105692] Updated weights for policy 0, policy_version 1248183 (0.0007) [2023-12-27 00:28:35,277][105585] KL-divergence is very high: 184.9095 [2023-12-27 00:28:35,289][105585] KL-divergence is very high: 273.4403 [2023-12-27 00:28:35,326][105692] Updated weights for policy 0, policy_version 1248193 (0.0008) [2023-12-27 00:28:35,327][105585] KL-divergence is very high: 179.0300 [2023-12-27 00:28:35,339][105585] KL-divergence is very high: 254.8569 [2023-12-27 00:28:35,379][105585] KL-divergence is very high: 167.7220 [2023-12-27 00:28:35,391][105692] Updated weights for policy 0, policy_version 1248203 (0.0005) [2023-12-27 00:28:35,393][105585] KL-divergence is very high: 233.2317 [2023-12-27 00:28:35,669][105620] Updated weights for policy 1, policy_version 1249508 (0.0010) [2023-12-27 00:28:35,734][105620] Updated weights for policy 1, policy_version 1249518 (0.0010) [2023-12-27 00:28:35,799][105620] Updated weights for policy 1, policy_version 1249528 (0.0010) [2023-12-27 00:28:36,002][105692] Updated weights for policy 0, policy_version 1248213 (0.0008) [2023-12-27 00:28:36,059][105692] Updated weights for policy 0, policy_version 1248223 (0.0010) [2023-12-27 00:28:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 639516672. Throughput: 0: 9622.8, 1: 9734.9. Samples: 639505412. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:28:36,063][104569] Avg episode reward: [(0, '8158.123'), (1, '9079.366')] [2023-12-27 00:28:36,129][105692] Updated weights for policy 0, policy_version 1248233 (0.0010) [2023-12-27 00:28:36,484][105620] Updated weights for policy 1, policy_version 1249538 (0.0010) [2023-12-27 00:28:36,540][105620] Updated weights for policy 1, policy_version 1249548 (0.0006) [2023-12-27 00:28:36,595][105620] Updated weights for policy 1, policy_version 1249558 (0.0005) [2023-12-27 00:28:36,660][105620] Updated weights for policy 1, policy_version 1249568 (0.0008) [2023-12-27 00:28:36,863][105692] Updated weights for policy 0, policy_version 1248243 (0.0006) [2023-12-27 00:28:36,931][105692] Updated weights for policy 0, policy_version 1248253 (0.0006) [2023-12-27 00:28:36,990][105692] Updated weights for policy 0, policy_version 1248263 (0.0010) [2023-12-27 00:28:37,286][105620] Updated weights for policy 1, policy_version 1249578 (0.0011) [2023-12-27 00:28:37,345][105620] Updated weights for policy 1, policy_version 1249588 (0.0011) [2023-12-27 00:28:37,404][105620] Updated weights for policy 1, policy_version 1249598 (0.0011) [2023-12-27 00:28:37,539][105692] Updated weights for policy 0, policy_version 1248273 (0.0010) [2023-12-27 00:28:37,588][105692] Updated weights for policy 0, policy_version 1248283 (0.0005) [2023-12-27 00:28:37,640][105692] Updated weights for policy 0, policy_version 1248293 (0.0005) [2023-12-27 00:28:37,701][105692] Updated weights for policy 0, policy_version 1248303 (0.0005) [2023-12-27 00:28:38,179][105620] Updated weights for policy 1, policy_version 1249608 (0.0010) [2023-12-27 00:28:38,230][105620] Updated weights for policy 1, policy_version 1249618 (0.0010) [2023-12-27 00:28:38,238][105692] Updated weights for policy 0, policy_version 1248313 (0.0006) [2023-12-27 00:28:38,280][105620] Updated weights for policy 1, policy_version 1249628 (0.0008) [2023-12-27 00:28:38,293][105692] Updated weights for policy 0, policy_version 1248323 (0.0005) [2023-12-27 00:28:38,369][105692] Updated weights for policy 0, policy_version 1248333 (0.0008) [2023-12-27 00:28:39,070][105620] Updated weights for policy 1, policy_version 1249638 (0.0007) [2023-12-27 00:28:39,078][105692] Updated weights for policy 0, policy_version 1248343 (0.0010) [2023-12-27 00:28:39,121][105620] Updated weights for policy 1, policy_version 1249648 (0.0005) [2023-12-27 00:28:39,139][105692] Updated weights for policy 0, policy_version 1248353 (0.0009) [2023-12-27 00:28:39,181][105620] Updated weights for policy 1, policy_version 1249658 (0.0006) [2023-12-27 00:28:39,196][105692] Updated weights for policy 0, policy_version 1248363 (0.0007) [2023-12-27 00:28:39,900][105620] Updated weights for policy 1, policy_version 1249668 (0.0009) [2023-12-27 00:28:39,970][105620] Updated weights for policy 1, policy_version 1249678 (0.0008) [2023-12-27 00:28:40,010][105692] Updated weights for policy 0, policy_version 1248373 (0.0010) [2023-12-27 00:28:40,026][105620] Updated weights for policy 1, policy_version 1249688 (0.0008) [2023-12-27 00:28:40,066][105692] Updated weights for policy 0, policy_version 1248383 (0.0008) [2023-12-27 00:28:40,118][105692] Updated weights for policy 0, policy_version 1248393 (0.0008) [2023-12-27 00:28:40,803][105692] Updated weights for policy 0, policy_version 1248403 (0.0007) [2023-12-27 00:28:40,807][105620] Updated weights for policy 1, policy_version 1249698 (0.0009) [2023-12-27 00:28:40,850][105692] Updated weights for policy 0, policy_version 1248413 (0.0006) [2023-12-27 00:28:40,859][105620] Updated weights for policy 1, policy_version 1249708 (0.0011) [2023-12-27 00:28:40,908][105692] Updated weights for policy 0, policy_version 1248423 (0.0006) [2023-12-27 00:28:40,914][105620] Updated weights for policy 1, policy_version 1249718 (0.0011) [2023-12-27 00:28:40,966][105620] Updated weights for policy 1, policy_version 1249728 (0.0010) [2023-12-27 00:28:41,062][104569] Fps is (10 sec: 21299.5, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 639623168. Throughput: 0: 9726.9, 1: 9647.3. Samples: 639625048. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:28:41,062][104569] Avg episode reward: [(0, '8528.366'), (1, '9079.703')] [2023-12-27 00:28:41,571][105692] Updated weights for policy 0, policy_version 1248433 (0.0006) [2023-12-27 00:28:41,636][105692] Updated weights for policy 0, policy_version 1248443 (0.0008) [2023-12-27 00:28:41,700][105692] Updated weights for policy 0, policy_version 1248453 (0.0007) [2023-12-27 00:28:41,743][105620] Updated weights for policy 1, policy_version 1249738 (0.0011) [2023-12-27 00:28:41,771][105692] Updated weights for policy 0, policy_version 1248463 (0.0010) [2023-12-27 00:28:41,800][105620] Updated weights for policy 1, policy_version 1249748 (0.0008) [2023-12-27 00:28:41,848][105620] Updated weights for policy 1, policy_version 1249758 (0.0008) [2023-12-27 00:28:42,526][105692] Updated weights for policy 0, policy_version 1248473 (0.0007) [2023-12-27 00:28:42,583][105620] Updated weights for policy 1, policy_version 1249768 (0.0006) [2023-12-27 00:28:42,593][105692] Updated weights for policy 0, policy_version 1248483 (0.0008) [2023-12-27 00:28:42,640][105620] Updated weights for policy 1, policy_version 1249778 (0.0006) [2023-12-27 00:28:42,650][105692] Updated weights for policy 0, policy_version 1248493 (0.0008) [2023-12-27 00:28:42,695][105620] Updated weights for policy 1, policy_version 1249788 (0.0008) [2023-12-27 00:28:43,226][105692] Updated weights for policy 0, policy_version 1248503 (0.0006) [2023-12-27 00:28:43,277][105692] Updated weights for policy 0, policy_version 1248513 (0.0005) [2023-12-27 00:28:43,322][105620] Updated weights for policy 1, policy_version 1249798 (0.0008) [2023-12-27 00:28:43,332][105692] Updated weights for policy 0, policy_version 1248523 (0.0007) [2023-12-27 00:28:43,368][105620] Updated weights for policy 1, policy_version 1249808 (0.0010) [2023-12-27 00:28:43,416][105620] Updated weights for policy 1, policy_version 1249818 (0.0010) [2023-12-27 00:28:43,923][105692] Updated weights for policy 0, policy_version 1248533 (0.0010) [2023-12-27 00:28:43,991][105692] Updated weights for policy 0, policy_version 1248543 (0.0007) [2023-12-27 00:28:44,028][105620] Updated weights for policy 1, policy_version 1249828 (0.0009) [2023-12-27 00:28:44,051][105692] Updated weights for policy 0, policy_version 1248553 (0.0007) [2023-12-27 00:28:44,079][105620] Updated weights for policy 1, policy_version 1249838 (0.0006) [2023-12-27 00:28:44,133][105620] Updated weights for policy 1, policy_version 1249848 (0.0008) [2023-12-27 00:28:44,568][105692] Updated weights for policy 0, policy_version 1248563 (0.0006) [2023-12-27 00:28:44,626][105692] Updated weights for policy 0, policy_version 1248573 (0.0006) [2023-12-27 00:28:44,684][105692] Updated weights for policy 0, policy_version 1248583 (0.0006) [2023-12-27 00:28:45,004][105620] Updated weights for policy 1, policy_version 1249858 (0.0009) [2023-12-27 00:28:45,066][105620] Updated weights for policy 1, policy_version 1249868 (0.0009) [2023-12-27 00:28:45,131][105620] Updated weights for policy 1, policy_version 1249878 (0.0008) [2023-12-27 00:28:45,202][105620] Updated weights for policy 1, policy_version 1249888 (0.0009) [2023-12-27 00:28:45,274][105692] Updated weights for policy 0, policy_version 1248593 (0.0006) [2023-12-27 00:28:45,340][105692] Updated weights for policy 0, policy_version 1248603 (0.0010) [2023-12-27 00:28:45,392][105692] Updated weights for policy 0, policy_version 1248613 (0.0010) [2023-12-27 00:28:45,450][105692] Updated weights for policy 0, policy_version 1248624 (0.0010) [2023-12-27 00:28:45,872][105620] Updated weights for policy 1, policy_version 1249898 (0.0006) [2023-12-27 00:28:45,943][105620] Updated weights for policy 1, policy_version 1249908 (0.0010) [2023-12-27 00:28:46,015][105620] Updated weights for policy 1, policy_version 1249918 (0.0010) [2023-12-27 00:28:46,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 639721472. Throughput: 0: 9804.9, 1: 9660.3. Samples: 639687184. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:28:46,062][104569] Avg episode reward: [(0, '8989.405'), (1, '9170.939')] [2023-12-27 00:28:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001249920_320020480.pth... [2023-12-27 00:28:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001248800_319733760.pth [2023-12-27 00:28:46,089][105692] Updated weights for policy 0, policy_version 1248634 (0.0006) [2023-12-27 00:28:46,139][105692] Updated weights for policy 0, policy_version 1248644 (0.0008) [2023-12-27 00:28:46,192][105692] Updated weights for policy 0, policy_version 1248654 (0.0009) [2023-12-27 00:28:46,203][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001248656_319709184.pth... [2023-12-27 00:28:46,206][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001247440_319397888.pth [2023-12-27 00:28:46,666][105620] Updated weights for policy 1, policy_version 1249928 (0.0006) [2023-12-27 00:28:46,712][105620] Updated weights for policy 1, policy_version 1249938 (0.0005) [2023-12-27 00:28:46,769][105620] Updated weights for policy 1, policy_version 1249948 (0.0005) [2023-12-27 00:28:46,863][105692] Updated weights for policy 0, policy_version 1248664 (0.0005) [2023-12-27 00:28:46,916][105692] Updated weights for policy 0, policy_version 1248674 (0.0005) [2023-12-27 00:28:46,975][105692] Updated weights for policy 0, policy_version 1248684 (0.0005) [2023-12-27 00:28:47,388][105620] Updated weights for policy 1, policy_version 1249958 (0.0005) [2023-12-27 00:28:47,448][105620] Updated weights for policy 1, policy_version 1249968 (0.0005) [2023-12-27 00:28:47,500][105620] Updated weights for policy 1, policy_version 1249978 (0.0005) [2023-12-27 00:28:47,544][105692] Updated weights for policy 0, policy_version 1248694 (0.0005) [2023-12-27 00:28:47,603][105692] Updated weights for policy 0, policy_version 1248704 (0.0005) [2023-12-27 00:28:47,662][105692] Updated weights for policy 0, policy_version 1248714 (0.0005) [2023-12-27 00:28:48,037][105620] Updated weights for policy 1, policy_version 1249988 (0.0006) [2023-12-27 00:28:48,090][105620] Updated weights for policy 1, policy_version 1249998 (0.0005) [2023-12-27 00:28:48,143][105620] Updated weights for policy 1, policy_version 1250008 (0.0005) [2023-12-27 00:28:48,231][105692] Updated weights for policy 0, policy_version 1248724 (0.0007) [2023-12-27 00:28:48,299][105692] Updated weights for policy 0, policy_version 1248734 (0.0010) [2023-12-27 00:28:48,354][105692] Updated weights for policy 0, policy_version 1248744 (0.0010) [2023-12-27 00:28:48,753][105620] Updated weights for policy 1, policy_version 1250018 (0.0005) [2023-12-27 00:28:48,809][105620] Updated weights for policy 1, policy_version 1250028 (0.0006) [2023-12-27 00:28:48,862][105620] Updated weights for policy 1, policy_version 1250038 (0.0008) [2023-12-27 00:28:48,923][105620] Updated weights for policy 1, policy_version 1250048 (0.0008) [2023-12-27 00:28:49,009][105692] Updated weights for policy 0, policy_version 1248754 (0.0010) [2023-12-27 00:28:49,063][105692] Updated weights for policy 0, policy_version 1248764 (0.0010) [2023-12-27 00:28:49,125][105692] Updated weights for policy 0, policy_version 1248774 (0.0010) [2023-12-27 00:28:49,186][105692] Updated weights for policy 0, policy_version 1248784 (0.0010) [2023-12-27 00:28:49,671][105620] Updated weights for policy 1, policy_version 1250058 (0.0008) [2023-12-27 00:28:49,733][105620] Updated weights for policy 1, policy_version 1250068 (0.0008) [2023-12-27 00:28:49,795][105620] Updated weights for policy 1, policy_version 1250078 (0.0008) [2023-12-27 00:28:49,946][105692] Updated weights for policy 0, policy_version 1248794 (0.0007) [2023-12-27 00:28:50,015][105692] Updated weights for policy 0, policy_version 1248804 (0.0009) [2023-12-27 00:28:50,077][105692] Updated weights for policy 0, policy_version 1248814 (0.0010) [2023-12-27 00:28:50,636][105620] Updated weights for policy 1, policy_version 1250088 (0.0008) [2023-12-27 00:28:50,685][105692] Updated weights for policy 0, policy_version 1248824 (0.0011) [2023-12-27 00:28:50,690][105620] Updated weights for policy 1, policy_version 1250098 (0.0009) [2023-12-27 00:28:50,736][105620] Updated weights for policy 1, policy_version 1250108 (0.0007) [2023-12-27 00:28:50,748][105692] Updated weights for policy 0, policy_version 1248834 (0.0010) [2023-12-27 00:28:50,812][105692] Updated weights for policy 0, policy_version 1248844 (0.0010) [2023-12-27 00:28:51,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 639827968. Throughput: 0: 10095.2, 1: 9705.8. Samples: 639814172. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:28:51,063][104569] Avg episode reward: [(0, '9264.809'), (1, '9353.473')] [2023-12-27 00:28:51,419][105620] Updated weights for policy 1, policy_version 1250118 (0.0008) [2023-12-27 00:28:51,470][105620] Updated weights for policy 1, policy_version 1250128 (0.0008) [2023-12-27 00:28:51,525][105620] Updated weights for policy 1, policy_version 1250138 (0.0008) [2023-12-27 00:28:51,571][105692] Updated weights for policy 0, policy_version 1248854 (0.0009) [2023-12-27 00:28:51,638][105692] Updated weights for policy 0, policy_version 1248864 (0.0008) [2023-12-27 00:28:51,707][105692] Updated weights for policy 0, policy_version 1248874 (0.0007) [2023-12-27 00:28:52,238][105620] Updated weights for policy 1, policy_version 1250148 (0.0009) [2023-12-27 00:28:52,303][105620] Updated weights for policy 1, policy_version 1250158 (0.0009) [2023-12-27 00:28:52,371][105620] Updated weights for policy 1, policy_version 1250168 (0.0009) [2023-12-27 00:28:52,434][105692] Updated weights for policy 0, policy_version 1248884 (0.0007) [2023-12-27 00:28:52,496][105692] Updated weights for policy 0, policy_version 1248894 (0.0008) [2023-12-27 00:28:52,560][105692] Updated weights for policy 0, policy_version 1248904 (0.0009) [2023-12-27 00:28:53,164][105620] Updated weights for policy 1, policy_version 1250178 (0.0009) [2023-12-27 00:28:53,219][105620] Updated weights for policy 1, policy_version 1250188 (0.0009) [2023-12-27 00:28:53,281][105620] Updated weights for policy 1, policy_version 1250198 (0.0008) [2023-12-27 00:28:53,305][105692] Updated weights for policy 0, policy_version 1248914 (0.0009) [2023-12-27 00:28:53,342][105620] Updated weights for policy 1, policy_version 1250208 (0.0005) [2023-12-27 00:28:53,363][105692] Updated weights for policy 0, policy_version 1248924 (0.0009) [2023-12-27 00:28:53,420][105692] Updated weights for policy 0, policy_version 1248934 (0.0009) [2023-12-27 00:28:53,477][105692] Updated weights for policy 0, policy_version 1248944 (0.0009) [2023-12-27 00:28:54,033][105620] Updated weights for policy 1, policy_version 1250218 (0.0009) [2023-12-27 00:28:54,091][105620] Updated weights for policy 1, policy_version 1250228 (0.0010) [2023-12-27 00:28:54,153][105620] Updated weights for policy 1, policy_version 1250238 (0.0010) [2023-12-27 00:28:54,245][105692] Updated weights for policy 0, policy_version 1248954 (0.0008) [2023-12-27 00:28:54,298][105692] Updated weights for policy 0, policy_version 1248964 (0.0008) [2023-12-27 00:28:54,361][105692] Updated weights for policy 0, policy_version 1248974 (0.0008) [2023-12-27 00:28:54,870][105620] Updated weights for policy 1, policy_version 1250248 (0.0006) [2023-12-27 00:28:54,923][105620] Updated weights for policy 1, policy_version 1250258 (0.0005) [2023-12-27 00:28:54,982][105620] Updated weights for policy 1, policy_version 1250268 (0.0007) [2023-12-27 00:28:55,137][105692] Updated weights for policy 0, policy_version 1248984 (0.0009) [2023-12-27 00:28:55,191][105692] Updated weights for policy 0, policy_version 1248994 (0.0009) [2023-12-27 00:28:55,241][105692] Updated weights for policy 0, policy_version 1249004 (0.0009) [2023-12-27 00:28:55,641][105620] Updated weights for policy 1, policy_version 1250278 (0.0006) [2023-12-27 00:28:55,691][105620] Updated weights for policy 1, policy_version 1250288 (0.0005) [2023-12-27 00:28:55,750][105620] Updated weights for policy 1, policy_version 1250298 (0.0006) [2023-12-27 00:28:56,001][105692] Updated weights for policy 0, policy_version 1249014 (0.0007) [2023-12-27 00:28:56,062][105692] Updated weights for policy 0, policy_version 1249024 (0.0008) [2023-12-27 00:28:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 639918080. Throughput: 0: 10134.3, 1: 9692.8. Samples: 639928544. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:28:56,062][104569] Avg episode reward: [(0, '9173.034'), (1, '9171.404')] [2023-12-27 00:28:56,113][105692] Updated weights for policy 0, policy_version 1249034 (0.0008) [2023-12-27 00:28:56,492][105620] Updated weights for policy 1, policy_version 1250308 (0.0009) [2023-12-27 00:28:56,553][105620] Updated weights for policy 1, policy_version 1250318 (0.0010) [2023-12-27 00:28:56,611][105620] Updated weights for policy 1, policy_version 1250328 (0.0010) [2023-12-27 00:28:56,825][105692] Updated weights for policy 0, policy_version 1249044 (0.0009) [2023-12-27 00:28:56,879][105692] Updated weights for policy 0, policy_version 1249055 (0.0010) [2023-12-27 00:28:56,924][105692] Updated weights for policy 0, policy_version 1249065 (0.0006) [2023-12-27 00:28:57,223][105620] Updated weights for policy 1, policy_version 1250338 (0.0010) [2023-12-27 00:28:57,284][105620] Updated weights for policy 1, policy_version 1250348 (0.0010) [2023-12-27 00:28:57,355][105620] Updated weights for policy 1, policy_version 1250358 (0.0010) [2023-12-27 00:28:57,418][105620] Updated weights for policy 1, policy_version 1250368 (0.0010) [2023-12-27 00:28:57,596][105692] Updated weights for policy 0, policy_version 1249075 (0.0006) [2023-12-27 00:28:57,644][105692] Updated weights for policy 0, policy_version 1249086 (0.0009) [2023-12-27 00:28:57,692][105692] Updated weights for policy 0, policy_version 1249096 (0.0008) [2023-12-27 00:28:57,996][105620] Updated weights for policy 1, policy_version 1250378 (0.0008) [2023-12-27 00:28:58,056][105620] Updated weights for policy 1, policy_version 1250388 (0.0011) [2023-12-27 00:28:58,105][105620] Updated weights for policy 1, policy_version 1250398 (0.0010) [2023-12-27 00:28:58,458][105692] Updated weights for policy 0, policy_version 1249106 (0.0006) [2023-12-27 00:28:58,522][105692] Updated weights for policy 0, policy_version 1249116 (0.0011) [2023-12-27 00:28:58,590][105692] Updated weights for policy 0, policy_version 1249126 (0.0010) [2023-12-27 00:28:58,656][105692] Updated weights for policy 0, policy_version 1249136 (0.0009) [2023-12-27 00:28:58,959][105620] Updated weights for policy 1, policy_version 1250408 (0.0007) [2023-12-27 00:28:59,023][105620] Updated weights for policy 1, policy_version 1250418 (0.0006) [2023-12-27 00:28:59,088][105620] Updated weights for policy 1, policy_version 1250428 (0.0006) [2023-12-27 00:28:59,477][105692] Updated weights for policy 0, policy_version 1249146 (0.0006) [2023-12-27 00:28:59,543][105692] Updated weights for policy 0, policy_version 1249156 (0.0006) [2023-12-27 00:28:59,608][105692] Updated weights for policy 0, policy_version 1249166 (0.0006) [2023-12-27 00:28:59,822][105620] Updated weights for policy 1, policy_version 1250438 (0.0007) [2023-12-27 00:28:59,890][105620] Updated weights for policy 1, policy_version 1250448 (0.0008) [2023-12-27 00:28:59,958][105620] Updated weights for policy 1, policy_version 1250458 (0.0008) [2023-12-27 00:29:00,247][105692] Updated weights for policy 0, policy_version 1249176 (0.0006) [2023-12-27 00:29:00,305][105692] Updated weights for policy 0, policy_version 1249186 (0.0010) [2023-12-27 00:29:00,363][105692] Updated weights for policy 0, policy_version 1249196 (0.0007) [2023-12-27 00:29:00,778][105620] Updated weights for policy 1, policy_version 1250468 (0.0009) [2023-12-27 00:29:00,836][105620] Updated weights for policy 1, policy_version 1250478 (0.0010) [2023-12-27 00:29:00,886][105620] Updated weights for policy 1, policy_version 1250488 (0.0010) [2023-12-27 00:29:00,890][105692] Updated weights for policy 0, policy_version 1249206 (0.0007) [2023-12-27 00:29:00,940][105692] Updated weights for policy 0, policy_version 1249216 (0.0010) [2023-12-27 00:29:00,990][105692] Updated weights for policy 0, policy_version 1249226 (0.0010) [2023-12-27 00:29:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 640024576. Throughput: 0: 10136.9, 1: 9728.8. Samples: 639988200. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:29:01,062][104569] Avg episode reward: [(0, '8716.298'), (1, '9261.930')] [2023-12-27 00:29:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001249232_319856640.pth... [2023-12-27 00:29:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001250496_320167936.pth... [2023-12-27 00:29:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001248016_319545344.pth [2023-12-27 00:29:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001249344_319873024.pth [2023-12-27 00:29:01,682][105620] Updated weights for policy 1, policy_version 1250498 (0.0008) [2023-12-27 00:29:01,740][105620] Updated weights for policy 1, policy_version 1250508 (0.0009) [2023-12-27 00:29:01,748][105692] Updated weights for policy 0, policy_version 1249236 (0.0010) [2023-12-27 00:29:01,798][105620] Updated weights for policy 1, policy_version 1250518 (0.0008) [2023-12-27 00:29:01,812][105692] Updated weights for policy 0, policy_version 1249246 (0.0006) [2023-12-27 00:29:01,854][105620] Updated weights for policy 1, policy_version 1250528 (0.0007) [2023-12-27 00:29:01,865][105692] Updated weights for policy 0, policy_version 1249256 (0.0009) [2023-12-27 00:29:02,506][105620] Updated weights for policy 1, policy_version 1250538 (0.0008) [2023-12-27 00:29:02,569][105620] Updated weights for policy 1, policy_version 1250548 (0.0009) [2023-12-27 00:29:02,630][105620] Updated weights for policy 1, policy_version 1250558 (0.0009) [2023-12-27 00:29:02,638][105692] Updated weights for policy 0, policy_version 1249266 (0.0010) [2023-12-27 00:29:02,692][105692] Updated weights for policy 0, policy_version 1249276 (0.0009) [2023-12-27 00:29:02,740][105692] Updated weights for policy 0, policy_version 1249286 (0.0009) [2023-12-27 00:29:02,788][105692] Updated weights for policy 0, policy_version 1249296 (0.0009) [2023-12-27 00:29:03,357][105620] Updated weights for policy 1, policy_version 1250568 (0.0008) [2023-12-27 00:29:03,410][105620] Updated weights for policy 1, policy_version 1250578 (0.0009) [2023-12-27 00:29:03,457][105620] Updated weights for policy 1, policy_version 1250588 (0.0010) [2023-12-27 00:29:03,542][105692] Updated weights for policy 0, policy_version 1249306 (0.0010) [2023-12-27 00:29:03,595][105692] Updated weights for policy 0, policy_version 1249316 (0.0010) [2023-12-27 00:29:03,659][105692] Updated weights for policy 0, policy_version 1249326 (0.0010) [2023-12-27 00:29:04,208][105620] Updated weights for policy 1, policy_version 1250598 (0.0010) [2023-12-27 00:29:04,274][105620] Updated weights for policy 1, policy_version 1250608 (0.0011) [2023-12-27 00:29:04,333][105620] Updated weights for policy 1, policy_version 1250618 (0.0011) [2023-12-27 00:29:04,399][105692] Updated weights for policy 0, policy_version 1249336 (0.0008) [2023-12-27 00:29:04,456][105692] Updated weights for policy 0, policy_version 1249346 (0.0008) [2023-12-27 00:29:04,517][105692] Updated weights for policy 0, policy_version 1249356 (0.0008) [2023-12-27 00:29:05,078][105620] Updated weights for policy 1, policy_version 1250628 (0.0010) [2023-12-27 00:29:05,136][105620] Updated weights for policy 1, policy_version 1250638 (0.0010) [2023-12-27 00:29:05,184][105620] Updated weights for policy 1, policy_version 1250648 (0.0010) [2023-12-27 00:29:05,289][105692] Updated weights for policy 0, policy_version 1249366 (0.0009) [2023-12-27 00:29:05,340][105692] Updated weights for policy 0, policy_version 1249376 (0.0009) [2023-12-27 00:29:05,391][105692] Updated weights for policy 0, policy_version 1249386 (0.0009) [2023-12-27 00:29:05,955][105620] Updated weights for policy 1, policy_version 1250658 (0.0010) [2023-12-27 00:29:05,996][105692] Updated weights for policy 0, policy_version 1249396 (0.0009) [2023-12-27 00:29:06,010][105620] Updated weights for policy 1, policy_version 1250668 (0.0008) [2023-12-27 00:29:06,056][105692] Updated weights for policy 0, policy_version 1249406 (0.0007) [2023-12-27 00:29:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 640106496. Throughput: 0: 10070.1, 1: 9688.5. Samples: 640101876. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:29:06,062][104569] Avg episode reward: [(0, '8624.923'), (1, '9352.597')] [2023-12-27 00:29:06,070][105620] Updated weights for policy 1, policy_version 1250678 (0.0007) [2023-12-27 00:29:06,116][105692] Updated weights for policy 0, policy_version 1249416 (0.0008) [2023-12-27 00:29:06,130][105620] Updated weights for policy 1, policy_version 1250688 (0.0007) [2023-12-27 00:29:06,815][105692] Updated weights for policy 0, policy_version 1249426 (0.0008) [2023-12-27 00:29:06,891][105692] Updated weights for policy 0, policy_version 1249436 (0.0009) [2023-12-27 00:29:06,928][105620] Updated weights for policy 1, policy_version 1250698 (0.0006) [2023-12-27 00:29:06,952][105692] Updated weights for policy 0, policy_version 1249446 (0.0008) [2023-12-27 00:29:06,981][105620] Updated weights for policy 1, policy_version 1250708 (0.0008) [2023-12-27 00:29:07,012][105692] Updated weights for policy 0, policy_version 1249456 (0.0008) [2023-12-27 00:29:07,039][105620] Updated weights for policy 1, policy_version 1250718 (0.0007) [2023-12-27 00:29:07,743][105692] Updated weights for policy 0, policy_version 1249466 (0.0005) [2023-12-27 00:29:07,767][105620] Updated weights for policy 1, policy_version 1250728 (0.0009) [2023-12-27 00:29:07,798][105692] Updated weights for policy 0, policy_version 1249476 (0.0007) [2023-12-27 00:29:07,821][105620] Updated weights for policy 1, policy_version 1250738 (0.0007) [2023-12-27 00:29:07,857][105692] Updated weights for policy 0, policy_version 1249486 (0.0010) [2023-12-27 00:29:07,878][105620] Updated weights for policy 1, policy_version 1250748 (0.0009) [2023-12-27 00:29:08,600][105692] Updated weights for policy 0, policy_version 1249496 (0.0007) [2023-12-27 00:29:08,614][105620] Updated weights for policy 1, policy_version 1250758 (0.0010) [2023-12-27 00:29:08,660][105692] Updated weights for policy 0, policy_version 1249506 (0.0005) [2023-12-27 00:29:08,673][105620] Updated weights for policy 1, policy_version 1250768 (0.0011) [2023-12-27 00:29:08,716][105692] Updated weights for policy 0, policy_version 1249516 (0.0006) [2023-12-27 00:29:08,734][105620] Updated weights for policy 1, policy_version 1250778 (0.0011) [2023-12-27 00:29:09,464][105620] Updated weights for policy 1, policy_version 1250788 (0.0010) [2023-12-27 00:29:09,512][105692] Updated weights for policy 0, policy_version 1249526 (0.0008) [2023-12-27 00:29:09,523][105620] Updated weights for policy 1, policy_version 1250798 (0.0007) [2023-12-27 00:29:09,573][105692] Updated weights for policy 0, policy_version 1249536 (0.0007) [2023-12-27 00:29:09,584][105620] Updated weights for policy 1, policy_version 1250808 (0.0008) [2023-12-27 00:29:09,627][105692] Updated weights for policy 0, policy_version 1249546 (0.0008) [2023-12-27 00:29:10,362][105692] Updated weights for policy 0, policy_version 1249556 (0.0009) [2023-12-27 00:29:10,386][105620] Updated weights for policy 1, policy_version 1250818 (0.0007) [2023-12-27 00:29:10,422][105692] Updated weights for policy 0, policy_version 1249566 (0.0007) [2023-12-27 00:29:10,448][105620] Updated weights for policy 1, policy_version 1250828 (0.0008) [2023-12-27 00:29:10,483][105692] Updated weights for policy 0, policy_version 1249576 (0.0007) [2023-12-27 00:29:10,505][105620] Updated weights for policy 1, policy_version 1250838 (0.0007) [2023-12-27 00:29:10,559][105620] Updated weights for policy 1, policy_version 1250848 (0.0008) [2023-12-27 00:29:11,062][104569] Fps is (10 sec: 18022.6, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 640204800. Throughput: 0: 10014.5, 1: 9745.4. Samples: 640215244. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:29:11,062][104569] Avg episode reward: [(0, '8809.196'), (1, '9352.074')] [2023-12-27 00:29:11,238][105692] Updated weights for policy 0, policy_version 1249586 (0.0008) [2023-12-27 00:29:11,306][105692] Updated weights for policy 0, policy_version 1249596 (0.0008) [2023-12-27 00:29:11,321][105620] Updated weights for policy 1, policy_version 1250858 (0.0007) [2023-12-27 00:29:11,369][105692] Updated weights for policy 0, policy_version 1249606 (0.0008) [2023-12-27 00:29:11,392][105620] Updated weights for policy 1, policy_version 1250868 (0.0010) [2023-12-27 00:29:11,435][105692] Updated weights for policy 0, policy_version 1249616 (0.0008) [2023-12-27 00:29:11,450][105620] Updated weights for policy 1, policy_version 1250878 (0.0011) [2023-12-27 00:29:12,115][105692] Updated weights for policy 0, policy_version 1249626 (0.0011) [2023-12-27 00:29:12,174][105692] Updated weights for policy 0, policy_version 1249636 (0.0011) [2023-12-27 00:29:12,226][105620] Updated weights for policy 1, policy_version 1250888 (0.0009) [2023-12-27 00:29:12,237][105692] Updated weights for policy 0, policy_version 1249646 (0.0011) [2023-12-27 00:29:12,286][105620] Updated weights for policy 1, policy_version 1250898 (0.0010) [2023-12-27 00:29:12,350][105620] Updated weights for policy 1, policy_version 1250908 (0.0011) [2023-12-27 00:29:12,985][105620] Updated weights for policy 1, policy_version 1250918 (0.0008) [2023-12-27 00:29:13,037][105620] Updated weights for policy 1, policy_version 1250928 (0.0005) [2023-12-27 00:29:13,042][105692] Updated weights for policy 0, policy_version 1249656 (0.0009) [2023-12-27 00:29:13,091][105620] Updated weights for policy 1, policy_version 1250938 (0.0005) [2023-12-27 00:29:13,099][105692] Updated weights for policy 0, policy_version 1249666 (0.0009) [2023-12-27 00:29:13,166][105692] Updated weights for policy 0, policy_version 1249676 (0.0009) [2023-12-27 00:29:13,713][105620] Updated weights for policy 1, policy_version 1250948 (0.0006) [2023-12-27 00:29:13,759][105620] Updated weights for policy 1, policy_version 1250958 (0.0005) [2023-12-27 00:29:13,814][105620] Updated weights for policy 1, policy_version 1250968 (0.0005) [2023-12-27 00:29:14,013][105692] Updated weights for policy 0, policy_version 1249686 (0.0008) [2023-12-27 00:29:14,072][105692] Updated weights for policy 0, policy_version 1249696 (0.0008) [2023-12-27 00:29:14,131][105692] Updated weights for policy 0, policy_version 1249706 (0.0008) [2023-12-27 00:29:14,392][105620] Updated weights for policy 1, policy_version 1250978 (0.0005) [2023-12-27 00:29:14,443][105620] Updated weights for policy 1, policy_version 1250988 (0.0005) [2023-12-27 00:29:14,495][105620] Updated weights for policy 1, policy_version 1250998 (0.0007) [2023-12-27 00:29:14,576][105620] Updated weights for policy 1, policy_version 1251008 (0.0010) [2023-12-27 00:29:14,966][105692] Updated weights for policy 0, policy_version 1249716 (0.0009) [2023-12-27 00:29:15,033][105692] Updated weights for policy 0, policy_version 1249726 (0.0007) [2023-12-27 00:29:15,091][105692] Updated weights for policy 0, policy_version 1249736 (0.0006) [2023-12-27 00:29:15,279][105620] Updated weights for policy 1, policy_version 1251018 (0.0008) [2023-12-27 00:29:15,349][105620] Updated weights for policy 1, policy_version 1251028 (0.0008) [2023-12-27 00:29:15,416][105620] Updated weights for policy 1, policy_version 1251038 (0.0008) [2023-12-27 00:29:15,737][105692] Updated weights for policy 0, policy_version 1249746 (0.0007) [2023-12-27 00:29:15,798][105692] Updated weights for policy 0, policy_version 1249756 (0.0006) [2023-12-27 00:29:15,867][105692] Updated weights for policy 0, policy_version 1249766 (0.0006) [2023-12-27 00:29:15,921][105692] Updated weights for policy 0, policy_version 1249776 (0.0005) [2023-12-27 00:29:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 640303104. Throughput: 0: 9972.2, 1: 9812.3. Samples: 640273160. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:29:16,063][104569] Avg episode reward: [(0, '8901.264'), (1, '9351.876')] [2023-12-27 00:29:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001249776_319995904.pth... [2023-12-27 00:29:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001248656_319709184.pth [2023-12-27 00:29:16,104][105620] Updated weights for policy 1, policy_version 1251048 (0.0006) [2023-12-27 00:29:16,167][105620] Updated weights for policy 1, policy_version 1251058 (0.0005) [2023-12-27 00:29:16,223][105620] Updated weights for policy 1, policy_version 1251068 (0.0005) [2023-12-27 00:29:16,245][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001251072_320315392.pth... [2023-12-27 00:29:16,250][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001249920_320020480.pth [2023-12-27 00:29:16,569][105692] Updated weights for policy 0, policy_version 1249786 (0.0010) [2023-12-27 00:29:16,627][105692] Updated weights for policy 0, policy_version 1249796 (0.0010) [2023-12-27 00:29:16,686][105692] Updated weights for policy 0, policy_version 1249806 (0.0010) [2023-12-27 00:29:16,798][105620] Updated weights for policy 1, policy_version 1251078 (0.0007) [2023-12-27 00:29:16,857][105620] Updated weights for policy 1, policy_version 1251088 (0.0005) [2023-12-27 00:29:16,912][105620] Updated weights for policy 1, policy_version 1251098 (0.0005) [2023-12-27 00:29:17,385][105692] Updated weights for policy 0, policy_version 1249816 (0.0009) [2023-12-27 00:29:17,434][105692] Updated weights for policy 0, policy_version 1249826 (0.0008) [2023-12-27 00:29:17,480][105692] Updated weights for policy 0, policy_version 1249836 (0.0009) [2023-12-27 00:29:17,523][105620] Updated weights for policy 1, policy_version 1251108 (0.0007) [2023-12-27 00:29:17,570][105620] Updated weights for policy 1, policy_version 1251118 (0.0007) [2023-12-27 00:29:17,626][105620] Updated weights for policy 1, policy_version 1251128 (0.0005) [2023-12-27 00:29:18,125][105692] Updated weights for policy 0, policy_version 1249846 (0.0009) [2023-12-27 00:29:18,173][105692] Updated weights for policy 0, policy_version 1249856 (0.0009) [2023-12-27 00:29:18,220][105692] Updated weights for policy 0, policy_version 1249866 (0.0009) [2023-12-27 00:29:18,349][105620] Updated weights for policy 1, policy_version 1251138 (0.0006) [2023-12-27 00:29:18,414][105620] Updated weights for policy 1, policy_version 1251148 (0.0009) [2023-12-27 00:29:18,476][105620] Updated weights for policy 1, policy_version 1251158 (0.0009) [2023-12-27 00:29:18,542][105620] Updated weights for policy 1, policy_version 1251168 (0.0009) [2023-12-27 00:29:18,948][105692] Updated weights for policy 0, policy_version 1249876 (0.0009) [2023-12-27 00:29:18,999][105692] Updated weights for policy 0, policy_version 1249886 (0.0009) [2023-12-27 00:29:19,054][105692] Updated weights for policy 0, policy_version 1249896 (0.0008) [2023-12-27 00:29:19,352][105620] Updated weights for policy 1, policy_version 1251178 (0.0009) [2023-12-27 00:29:19,415][105620] Updated weights for policy 1, policy_version 1251188 (0.0008) [2023-12-27 00:29:19,503][105620] Updated weights for policy 1, policy_version 1251200 (0.0009) [2023-12-27 00:29:19,770][105692] Updated weights for policy 0, policy_version 1249906 (0.0006) [2023-12-27 00:29:19,836][105692] Updated weights for policy 0, policy_version 1249916 (0.0009) [2023-12-27 00:29:19,892][105692] Updated weights for policy 0, policy_version 1249926 (0.0006) [2023-12-27 00:29:19,958][105692] Updated weights for policy 0, policy_version 1249936 (0.0007) [2023-12-27 00:29:20,305][105620] Updated weights for policy 1, policy_version 1251210 (0.0009) [2023-12-27 00:29:20,364][105620] Updated weights for policy 1, policy_version 1251220 (0.0009) [2023-12-27 00:29:20,426][105620] Updated weights for policy 1, policy_version 1251230 (0.0005) [2023-12-27 00:29:20,620][105692] Updated weights for policy 0, policy_version 1249946 (0.0006) [2023-12-27 00:29:20,680][105692] Updated weights for policy 0, policy_version 1249956 (0.0009) [2023-12-27 00:29:20,739][105692] Updated weights for policy 0, policy_version 1249966 (0.0009) [2023-12-27 00:29:21,010][105620] Updated weights for policy 1, policy_version 1251240 (0.0010) [2023-12-27 00:29:21,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19660.7, 300 sec: 19494.2). Total num frames: 640401408. Throughput: 0: 9951.6, 1: 9754.4. Samples: 640392180. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:29:21,063][104569] Avg episode reward: [(0, '8991.638'), (1, '9352.274')] [2023-12-27 00:29:21,075][105620] Updated weights for policy 1, policy_version 1251250 (0.0009) [2023-12-27 00:29:21,143][105620] Updated weights for policy 1, policy_version 1251260 (0.0012) [2023-12-27 00:29:21,547][105692] Updated weights for policy 0, policy_version 1249976 (0.0009) [2023-12-27 00:29:21,614][105692] Updated weights for policy 0, policy_version 1249986 (0.0007) [2023-12-27 00:29:21,685][105692] Updated weights for policy 0, policy_version 1249996 (0.0008) [2023-12-27 00:29:21,889][105620] Updated weights for policy 1, policy_version 1251270 (0.0011) [2023-12-27 00:29:21,953][105620] Updated weights for policy 1, policy_version 1251280 (0.0011) [2023-12-27 00:29:22,010][105620] Updated weights for policy 1, policy_version 1251290 (0.0011) [2023-12-27 00:29:22,481][105692] Updated weights for policy 0, policy_version 1250006 (0.0009) [2023-12-27 00:29:22,534][105692] Updated weights for policy 0, policy_version 1250016 (0.0008) [2023-12-27 00:29:22,588][105692] Updated weights for policy 0, policy_version 1250027 (0.0009) [2023-12-27 00:29:22,761][105620] Updated weights for policy 1, policy_version 1251300 (0.0010) [2023-12-27 00:29:22,813][105620] Updated weights for policy 1, policy_version 1251310 (0.0010) [2023-12-27 00:29:22,857][105620] Updated weights for policy 1, policy_version 1251320 (0.0010) [2023-12-27 00:29:23,448][105620] Updated weights for policy 1, policy_version 1251330 (0.0010) [2023-12-27 00:29:23,467][105692] Updated weights for policy 0, policy_version 1250037 (0.0008) [2023-12-27 00:29:23,500][105620] Updated weights for policy 1, policy_version 1251340 (0.0010) [2023-12-27 00:29:23,513][105692] Updated weights for policy 0, policy_version 1250047 (0.0009) [2023-12-27 00:29:23,547][105620] Updated weights for policy 1, policy_version 1251350 (0.0010) [2023-12-27 00:29:23,566][105692] Updated weights for policy 0, policy_version 1250057 (0.0005) [2023-12-27 00:29:23,600][105620] Updated weights for policy 1, policy_version 1251360 (0.0010) [2023-12-27 00:29:24,196][105620] Updated weights for policy 1, policy_version 1251370 (0.0005) [2023-12-27 00:29:24,251][105620] Updated weights for policy 1, policy_version 1251380 (0.0005) [2023-12-27 00:29:24,303][105620] Updated weights for policy 1, policy_version 1251390 (0.0008) [2023-12-27 00:29:24,443][105692] Updated weights for policy 0, policy_version 1250067 (0.0006) [2023-12-27 00:29:24,499][105692] Updated weights for policy 0, policy_version 1250077 (0.0008) [2023-12-27 00:29:24,560][105692] Updated weights for policy 0, policy_version 1250087 (0.0008) [2023-12-27 00:29:24,992][105620] Updated weights for policy 1, policy_version 1251400 (0.0010) [2023-12-27 00:29:25,060][105620] Updated weights for policy 1, policy_version 1251410 (0.0010) [2023-12-27 00:29:25,125][105620] Updated weights for policy 1, policy_version 1251420 (0.0010) [2023-12-27 00:29:25,252][105692] Updated weights for policy 0, policy_version 1250097 (0.0008) [2023-12-27 00:29:25,301][105692] Updated weights for policy 0, policy_version 1250107 (0.0010) [2023-12-27 00:29:25,350][105692] Updated weights for policy 0, policy_version 1250117 (0.0010) [2023-12-27 00:29:25,405][105692] Updated weights for policy 0, policy_version 1250127 (0.0010) [2023-12-27 00:29:25,836][105620] Updated weights for policy 1, policy_version 1251430 (0.0010) [2023-12-27 00:29:25,890][105620] Updated weights for policy 1, policy_version 1251440 (0.0010) [2023-12-27 00:29:25,942][105620] Updated weights for policy 1, policy_version 1251450 (0.0010) [2023-12-27 00:29:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 640499712. Throughput: 0: 9755.5, 1: 9838.2. Samples: 640506768. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:29:26,063][104569] Avg episode reward: [(0, '8720.820'), (1, '9352.503')] [2023-12-27 00:29:26,098][105692] Updated weights for policy 0, policy_version 1250137 (0.0006) [2023-12-27 00:29:26,161][105692] Updated weights for policy 0, policy_version 1250147 (0.0006) [2023-12-27 00:29:26,210][105692] Updated weights for policy 0, policy_version 1250157 (0.0005) [2023-12-27 00:29:26,682][105620] Updated weights for policy 1, policy_version 1251460 (0.0010) [2023-12-27 00:29:26,738][105620] Updated weights for policy 1, policy_version 1251470 (0.0010) [2023-12-27 00:29:26,786][105692] Updated weights for policy 0, policy_version 1250167 (0.0005) [2023-12-27 00:29:26,799][105620] Updated weights for policy 1, policy_version 1251480 (0.0010) [2023-12-27 00:29:26,843][105692] Updated weights for policy 0, policy_version 1250177 (0.0005) [2023-12-27 00:29:26,891][105692] Updated weights for policy 0, policy_version 1250187 (0.0005) [2023-12-27 00:29:27,429][105692] Updated weights for policy 0, policy_version 1250197 (0.0005) [2023-12-27 00:29:27,487][105692] Updated weights for policy 0, policy_version 1250207 (0.0005) [2023-12-27 00:29:27,516][105620] Updated weights for policy 1, policy_version 1251490 (0.0010) [2023-12-27 00:29:27,538][105692] Updated weights for policy 0, policy_version 1250217 (0.0005) [2023-12-27 00:29:27,571][105620] Updated weights for policy 1, policy_version 1251500 (0.0010) [2023-12-27 00:29:27,632][105620] Updated weights for policy 1, policy_version 1251510 (0.0010) [2023-12-27 00:29:27,692][105620] Updated weights for policy 1, policy_version 1251520 (0.0010) [2023-12-27 00:29:28,094][105692] Updated weights for policy 0, policy_version 1250227 (0.0006) [2023-12-27 00:29:28,138][105692] Updated weights for policy 0, policy_version 1250237 (0.0008) [2023-12-27 00:29:28,187][105692] Updated weights for policy 0, policy_version 1250247 (0.0008) [2023-12-27 00:29:28,435][105620] Updated weights for policy 1, policy_version 1251530 (0.0010) [2023-12-27 00:29:28,493][105620] Updated weights for policy 1, policy_version 1251540 (0.0010) [2023-12-27 00:29:28,553][105620] Updated weights for policy 1, policy_version 1251550 (0.0011) [2023-12-27 00:29:28,948][105692] Updated weights for policy 0, policy_version 1250257 (0.0008) [2023-12-27 00:29:29,015][105692] Updated weights for policy 0, policy_version 1250267 (0.0011) [2023-12-27 00:29:29,080][105692] Updated weights for policy 0, policy_version 1250277 (0.0009) [2023-12-27 00:29:29,146][105692] Updated weights for policy 0, policy_version 1250287 (0.0010) [2023-12-27 00:29:29,208][105620] Updated weights for policy 1, policy_version 1251560 (0.0006) [2023-12-27 00:29:29,274][105620] Updated weights for policy 1, policy_version 1251570 (0.0010) [2023-12-27 00:29:29,338][105620] Updated weights for policy 1, policy_version 1251580 (0.0006) [2023-12-27 00:29:29,830][105692] Updated weights for policy 0, policy_version 1250297 (0.0010) [2023-12-27 00:29:29,894][105692] Updated weights for policy 0, policy_version 1250307 (0.0010) [2023-12-27 00:29:29,916][105620] Updated weights for policy 1, policy_version 1251590 (0.0010) [2023-12-27 00:29:29,952][105692] Updated weights for policy 0, policy_version 1250317 (0.0010) [2023-12-27 00:29:29,980][105620] Updated weights for policy 1, policy_version 1251600 (0.0007) [2023-12-27 00:29:30,039][105620] Updated weights for policy 1, policy_version 1251610 (0.0005) [2023-12-27 00:29:30,646][105692] Updated weights for policy 0, policy_version 1250327 (0.0010) [2023-12-27 00:29:30,697][105692] Updated weights for policy 0, policy_version 1250337 (0.0010) [2023-12-27 00:29:30,702][105620] Updated weights for policy 1, policy_version 1251620 (0.0009) [2023-12-27 00:29:30,743][105692] Updated weights for policy 0, policy_version 1250347 (0.0008) [2023-12-27 00:29:30,749][105620] Updated weights for policy 1, policy_version 1251630 (0.0009) [2023-12-27 00:29:30,804][105620] Updated weights for policy 1, policy_version 1251640 (0.0007) [2023-12-27 00:29:31,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19933.9, 300 sec: 19522.0). Total num frames: 640606208. Throughput: 0: 9830.8, 1: 9785.2. Samples: 640569904. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:29:31,062][104569] Avg episode reward: [(0, '8811.571'), (1, '9169.501')] [2023-12-27 00:29:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001250352_320143360.pth... [2023-12-27 00:29:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001251648_320462848.pth... [2023-12-27 00:29:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001249232_319856640.pth [2023-12-27 00:29:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001250496_320167936.pth [2023-12-27 00:29:31,452][105692] Updated weights for policy 0, policy_version 1250357 (0.0007) [2023-12-27 00:29:31,507][105692] Updated weights for policy 0, policy_version 1250367 (0.0005) [2023-12-27 00:29:31,517][105620] Updated weights for policy 1, policy_version 1251650 (0.0008) [2023-12-27 00:29:31,564][105692] Updated weights for policy 0, policy_version 1250377 (0.0005) [2023-12-27 00:29:31,569][105620] Updated weights for policy 1, policy_version 1251660 (0.0008) [2023-12-27 00:29:31,632][105620] Updated weights for policy 1, policy_version 1251670 (0.0009) [2023-12-27 00:29:31,696][105620] Updated weights for policy 1, policy_version 1251680 (0.0008) [2023-12-27 00:29:32,193][105692] Updated weights for policy 0, policy_version 1250387 (0.0005) [2023-12-27 00:29:32,251][105692] Updated weights for policy 0, policy_version 1250397 (0.0008) [2023-12-27 00:29:32,306][105692] Updated weights for policy 0, policy_version 1250407 (0.0009) [2023-12-27 00:29:32,535][105620] Updated weights for policy 1, policy_version 1251690 (0.0010) [2023-12-27 00:29:32,585][105620] Updated weights for policy 1, policy_version 1251701 (0.0009) [2023-12-27 00:29:32,638][105620] Updated weights for policy 1, policy_version 1251711 (0.0008) [2023-12-27 00:29:32,952][105692] Updated weights for policy 0, policy_version 1250417 (0.0009) [2023-12-27 00:29:33,009][105692] Updated weights for policy 0, policy_version 1250427 (0.0006) [2023-12-27 00:29:33,073][105692] Updated weights for policy 0, policy_version 1250437 (0.0005) [2023-12-27 00:29:33,128][105692] Updated weights for policy 0, policy_version 1250447 (0.0006) [2023-12-27 00:29:33,533][105620] Updated weights for policy 1, policy_version 1251721 (0.0010) [2023-12-27 00:29:33,586][105620] Updated weights for policy 1, policy_version 1251732 (0.0010) [2023-12-27 00:29:33,637][105620] Updated weights for policy 1, policy_version 1251743 (0.0009) [2023-12-27 00:29:33,677][105692] Updated weights for policy 0, policy_version 1250457 (0.0006) [2023-12-27 00:29:33,735][105692] Updated weights for policy 0, policy_version 1250467 (0.0005) [2023-12-27 00:29:33,792][105692] Updated weights for policy 0, policy_version 1250477 (0.0005) [2023-12-27 00:29:34,444][105692] Updated weights for policy 0, policy_version 1250487 (0.0007) [2023-12-27 00:29:34,471][105620] Updated weights for policy 1, policy_version 1251753 (0.0008) [2023-12-27 00:29:34,506][105692] Updated weights for policy 0, policy_version 1250497 (0.0008) [2023-12-27 00:29:34,531][105620] Updated weights for policy 1, policy_version 1251763 (0.0007) [2023-12-27 00:29:34,566][105692] Updated weights for policy 0, policy_version 1250507 (0.0008) [2023-12-27 00:29:34,592][105620] Updated weights for policy 1, policy_version 1251773 (0.0007) [2023-12-27 00:29:35,256][105692] Updated weights for policy 0, policy_version 1250517 (0.0007) [2023-12-27 00:29:35,317][105692] Updated weights for policy 0, policy_version 1250527 (0.0007) [2023-12-27 00:29:35,372][105692] Updated weights for policy 0, policy_version 1250537 (0.0010) [2023-12-27 00:29:35,389][105620] Updated weights for policy 1, policy_version 1251783 (0.0006) [2023-12-27 00:29:35,450][105620] Updated weights for policy 1, policy_version 1251793 (0.0007) [2023-12-27 00:29:35,505][105620] Updated weights for policy 1, policy_version 1251803 (0.0008) [2023-12-27 00:29:36,049][105620] Updated weights for policy 1, policy_version 1251813 (0.0006) [2023-12-27 00:29:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.9, 300 sec: 19494.2). Total num frames: 640696320. Throughput: 0: 9736.9, 1: 9697.8. Samples: 640688732. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:29:36,062][104569] Avg episode reward: [(0, '8898.166'), (1, '9078.268')] [2023-12-27 00:29:36,114][105620] Updated weights for policy 1, policy_version 1251823 (0.0007) [2023-12-27 00:29:36,135][105692] Updated weights for policy 0, policy_version 1250547 (0.0010) [2023-12-27 00:29:36,179][105620] Updated weights for policy 1, policy_version 1251833 (0.0008) [2023-12-27 00:29:36,194][105692] Updated weights for policy 0, policy_version 1250557 (0.0008) [2023-12-27 00:29:36,261][105692] Updated weights for policy 0, policy_version 1250567 (0.0008) [2023-12-27 00:29:36,907][105620] Updated weights for policy 1, policy_version 1251843 (0.0007) [2023-12-27 00:29:36,951][105692] Updated weights for policy 0, policy_version 1250577 (0.0009) [2023-12-27 00:29:36,970][105620] Updated weights for policy 1, policy_version 1251853 (0.0009) [2023-12-27 00:29:37,001][105692] Updated weights for policy 0, policy_version 1250587 (0.0008) [2023-12-27 00:29:37,029][105620] Updated weights for policy 1, policy_version 1251863 (0.0008) [2023-12-27 00:29:37,051][105692] Updated weights for policy 0, policy_version 1250597 (0.0009) [2023-12-27 00:29:37,107][105692] Updated weights for policy 0, policy_version 1250607 (0.0007) [2023-12-27 00:29:37,788][105692] Updated weights for policy 0, policy_version 1250617 (0.0007) [2023-12-27 00:29:37,843][105620] Updated weights for policy 1, policy_version 1251873 (0.0008) [2023-12-27 00:29:37,848][105692] Updated weights for policy 0, policy_version 1250627 (0.0006) [2023-12-27 00:29:37,902][105620] Updated weights for policy 1, policy_version 1251883 (0.0009) [2023-12-27 00:29:37,905][105692] Updated weights for policy 0, policy_version 1250637 (0.0006) [2023-12-27 00:29:37,958][105620] Updated weights for policy 1, policy_version 1251893 (0.0009) [2023-12-27 00:29:38,020][105620] Updated weights for policy 1, policy_version 1251903 (0.0009) [2023-12-27 00:29:38,538][105692] Updated weights for policy 0, policy_version 1250647 (0.0006) [2023-12-27 00:29:38,601][105692] Updated weights for policy 0, policy_version 1250657 (0.0005) [2023-12-27 00:29:38,658][105692] Updated weights for policy 0, policy_version 1250667 (0.0005) [2023-12-27 00:29:38,858][105620] Updated weights for policy 1, policy_version 1251913 (0.0009) [2023-12-27 00:29:38,907][105620] Updated weights for policy 1, policy_version 1251923 (0.0008) [2023-12-27 00:29:38,962][105620] Updated weights for policy 1, policy_version 1251933 (0.0008) [2023-12-27 00:29:39,324][105692] Updated weights for policy 0, policy_version 1250677 (0.0006) [2023-12-27 00:29:39,391][105692] Updated weights for policy 0, policy_version 1250687 (0.0009) [2023-12-27 00:29:39,455][105692] Updated weights for policy 0, policy_version 1250697 (0.0008) [2023-12-27 00:29:39,744][105620] Updated weights for policy 1, policy_version 1251943 (0.0007) [2023-12-27 00:29:39,810][105620] Updated weights for policy 1, policy_version 1251953 (0.0009) [2023-12-27 00:29:39,875][105620] Updated weights for policy 1, policy_version 1251963 (0.0009) [2023-12-27 00:29:40,255][105692] Updated weights for policy 0, policy_version 1250707 (0.0009) [2023-12-27 00:29:40,318][105692] Updated weights for policy 0, policy_version 1250717 (0.0010) [2023-12-27 00:29:40,382][105692] Updated weights for policy 0, policy_version 1250727 (0.0010) [2023-12-27 00:29:40,530][105620] Updated weights for policy 1, policy_version 1251973 (0.0008) [2023-12-27 00:29:40,600][105620] Updated weights for policy 1, policy_version 1251983 (0.0006) [2023-12-27 00:29:40,668][105620] Updated weights for policy 1, policy_version 1251993 (0.0006) [2023-12-27 00:29:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 640794624. Throughput: 0: 9811.4, 1: 9663.5. Samples: 640804912. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:29:41,062][104569] Avg episode reward: [(0, '8990.825'), (1, '9078.396')] [2023-12-27 00:29:41,188][105692] Updated weights for policy 0, policy_version 1250737 (0.0009) [2023-12-27 00:29:41,245][105692] Updated weights for policy 0, policy_version 1250747 (0.0010) [2023-12-27 00:29:41,310][105692] Updated weights for policy 0, policy_version 1250757 (0.0008) [2023-12-27 00:29:41,355][105620] Updated weights for policy 1, policy_version 1252003 (0.0008) [2023-12-27 00:29:41,373][105692] Updated weights for policy 0, policy_version 1250767 (0.0008) [2023-12-27 00:29:41,419][105620] Updated weights for policy 1, policy_version 1252013 (0.0006) [2023-12-27 00:29:41,478][105620] Updated weights for policy 1, policy_version 1252023 (0.0008) [2023-12-27 00:29:42,139][105620] Updated weights for policy 1, policy_version 1252033 (0.0009) [2023-12-27 00:29:42,200][105692] Updated weights for policy 0, policy_version 1250777 (0.0008) [2023-12-27 00:29:42,200][105620] Updated weights for policy 1, policy_version 1252043 (0.0008) [2023-12-27 00:29:42,262][105620] Updated weights for policy 1, policy_version 1252053 (0.0008) [2023-12-27 00:29:42,267][105692] Updated weights for policy 0, policy_version 1250787 (0.0010) [2023-12-27 00:29:42,332][105620] Updated weights for policy 1, policy_version 1252063 (0.0007) [2023-12-27 00:29:42,354][105692] Updated weights for policy 0, policy_version 1250797 (0.0008) [2023-12-27 00:29:42,925][105620] Updated weights for policy 1, policy_version 1252073 (0.0006) [2023-12-27 00:29:42,990][105620] Updated weights for policy 1, policy_version 1252083 (0.0006) [2023-12-27 00:29:43,009][105692] Updated weights for policy 0, policy_version 1250807 (0.0006) [2023-12-27 00:29:43,043][105620] Updated weights for policy 1, policy_version 1252093 (0.0005) [2023-12-27 00:29:43,072][105692] Updated weights for policy 0, policy_version 1250817 (0.0005) [2023-12-27 00:29:43,134][105692] Updated weights for policy 0, policy_version 1250827 (0.0005) [2023-12-27 00:29:43,574][105620] Updated weights for policy 1, policy_version 1252103 (0.0007) [2023-12-27 00:29:43,629][105620] Updated weights for policy 1, policy_version 1252113 (0.0008) [2023-12-27 00:29:43,681][105620] Updated weights for policy 1, policy_version 1252123 (0.0008) [2023-12-27 00:29:43,754][105692] Updated weights for policy 0, policy_version 1250837 (0.0008) [2023-12-27 00:29:43,805][105692] Updated weights for policy 0, policy_version 1250847 (0.0010) [2023-12-27 00:29:43,867][105692] Updated weights for policy 0, policy_version 1250857 (0.0010) [2023-12-27 00:29:44,421][105620] Updated weights for policy 1, policy_version 1252133 (0.0007) [2023-12-27 00:29:44,483][105620] Updated weights for policy 1, policy_version 1252143 (0.0009) [2023-12-27 00:29:44,493][105692] Updated weights for policy 0, policy_version 1250867 (0.0009) [2023-12-27 00:29:44,535][105620] Updated weights for policy 1, policy_version 1252153 (0.0008) [2023-12-27 00:29:44,542][105692] Updated weights for policy 0, policy_version 1250877 (0.0006) [2023-12-27 00:29:44,591][105692] Updated weights for policy 0, policy_version 1250887 (0.0005) [2023-12-27 00:29:45,279][105692] Updated weights for policy 0, policy_version 1250897 (0.0008) [2023-12-27 00:29:45,290][105620] Updated weights for policy 1, policy_version 1252163 (0.0009) [2023-12-27 00:29:45,343][105692] Updated weights for policy 0, policy_version 1250907 (0.0011) [2023-12-27 00:29:45,357][105620] Updated weights for policy 1, policy_version 1252173 (0.0007) [2023-12-27 00:29:45,406][105692] Updated weights for policy 0, policy_version 1250917 (0.0011) [2023-12-27 00:29:45,413][105620] Updated weights for policy 1, policy_version 1252183 (0.0007) [2023-12-27 00:29:45,462][105692] Updated weights for policy 0, policy_version 1250927 (0.0011) [2023-12-27 00:29:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 640892928. Throughput: 0: 9783.0, 1: 9702.8. Samples: 640865060. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:29:46,062][104569] Avg episode reward: [(0, '9176.847'), (1, '9169.461')] [2023-12-27 00:29:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001250928_320290816.pth... [2023-12-27 00:29:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001252192_320602112.pth... [2023-12-27 00:29:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001249776_319995904.pth [2023-12-27 00:29:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001251072_320315392.pth [2023-12-27 00:29:46,150][105692] Updated weights for policy 0, policy_version 1250937 (0.0009) [2023-12-27 00:29:46,194][105620] Updated weights for policy 1, policy_version 1252193 (0.0008) [2023-12-27 00:29:46,203][105692] Updated weights for policy 0, policy_version 1250947 (0.0009) [2023-12-27 00:29:46,246][105620] Updated weights for policy 1, policy_version 1252203 (0.0006) [2023-12-27 00:29:46,256][105692] Updated weights for policy 0, policy_version 1250957 (0.0009) [2023-12-27 00:29:46,294][105620] Updated weights for policy 1, policy_version 1252213 (0.0007) [2023-12-27 00:29:46,348][105620] Updated weights for policy 1, policy_version 1252223 (0.0009) [2023-12-27 00:29:46,917][105692] Updated weights for policy 0, policy_version 1250967 (0.0006) [2023-12-27 00:29:46,956][105620] Updated weights for policy 1, policy_version 1252233 (0.0010) [2023-12-27 00:29:46,969][105692] Updated weights for policy 0, policy_version 1250977 (0.0006) [2023-12-27 00:29:47,018][105692] Updated weights for policy 0, policy_version 1250987 (0.0005) [2023-12-27 00:29:47,019][105620] Updated weights for policy 1, policy_version 1252243 (0.0010) [2023-12-27 00:29:47,081][105620] Updated weights for policy 1, policy_version 1252253 (0.0010) [2023-12-27 00:29:47,552][105692] Updated weights for policy 0, policy_version 1250997 (0.0006) [2023-12-27 00:29:47,625][105692] Updated weights for policy 0, policy_version 1251007 (0.0007) [2023-12-27 00:29:47,692][105692] Updated weights for policy 0, policy_version 1251017 (0.0010) [2023-12-27 00:29:47,747][105620] Updated weights for policy 1, policy_version 1252263 (0.0007) [2023-12-27 00:29:47,793][105620] Updated weights for policy 1, policy_version 1252273 (0.0005) [2023-12-27 00:29:47,846][105620] Updated weights for policy 1, policy_version 1252283 (0.0005) [2023-12-27 00:29:48,300][105692] Updated weights for policy 0, policy_version 1251027 (0.0010) [2023-12-27 00:29:48,362][105692] Updated weights for policy 0, policy_version 1251037 (0.0007) [2023-12-27 00:29:48,425][105692] Updated weights for policy 0, policy_version 1251047 (0.0007) [2023-12-27 00:29:48,611][105620] Updated weights for policy 1, policy_version 1252293 (0.0005) [2023-12-27 00:29:48,666][105620] Updated weights for policy 1, policy_version 1252303 (0.0006) [2023-12-27 00:29:48,715][105620] Updated weights for policy 1, policy_version 1252313 (0.0008) [2023-12-27 00:29:48,986][105692] Updated weights for policy 0, policy_version 1251057 (0.0007) [2023-12-27 00:29:49,048][105692] Updated weights for policy 0, policy_version 1251067 (0.0006) [2023-12-27 00:29:49,116][105692] Updated weights for policy 0, policy_version 1251077 (0.0005) [2023-12-27 00:29:49,178][105692] Updated weights for policy 0, policy_version 1251087 (0.0006) [2023-12-27 00:29:49,320][105620] Updated weights for policy 1, policy_version 1252323 (0.0007) [2023-12-27 00:29:49,400][105620] Updated weights for policy 1, policy_version 1252333 (0.0009) [2023-12-27 00:29:49,455][105620] Updated weights for policy 1, policy_version 1252343 (0.0010) [2023-12-27 00:29:49,801][105692] Updated weights for policy 0, policy_version 1251097 (0.0009) [2023-12-27 00:29:49,869][105692] Updated weights for policy 0, policy_version 1251107 (0.0010) [2023-12-27 00:29:49,928][105692] Updated weights for policy 0, policy_version 1251117 (0.0009) [2023-12-27 00:29:50,227][105620] Updated weights for policy 1, policy_version 1252353 (0.0010) [2023-12-27 00:29:50,290][105620] Updated weights for policy 1, policy_version 1252363 (0.0009) [2023-12-27 00:29:50,345][105620] Updated weights for policy 1, policy_version 1252373 (0.0009) [2023-12-27 00:29:50,403][105620] Updated weights for policy 1, policy_version 1252383 (0.0006) [2023-12-27 00:29:50,693][105692] Updated weights for policy 0, policy_version 1251127 (0.0009) [2023-12-27 00:29:50,748][105692] Updated weights for policy 0, policy_version 1251137 (0.0009) [2023-12-27 00:29:50,811][105692] Updated weights for policy 0, policy_version 1251147 (0.0009) [2023-12-27 00:29:51,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 640999424. Throughput: 0: 9955.8, 1: 9779.4. Samples: 640989956. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:29:51,062][104569] Avg episode reward: [(0, '9174.951'), (1, '9261.072')] [2023-12-27 00:29:51,175][105620] Updated weights for policy 1, policy_version 1252393 (0.0009) [2023-12-27 00:29:51,225][105620] Updated weights for policy 1, policy_version 1252403 (0.0010) [2023-12-27 00:29:51,291][105620] Updated weights for policy 1, policy_version 1252413 (0.0011) [2023-12-27 00:29:51,578][105692] Updated weights for policy 0, policy_version 1251157 (0.0008) [2023-12-27 00:29:51,631][105692] Updated weights for policy 0, policy_version 1251167 (0.0008) [2023-12-27 00:29:51,684][105692] Updated weights for policy 0, policy_version 1251177 (0.0008) [2023-12-27 00:29:52,098][105620] Updated weights for policy 1, policy_version 1252423 (0.0010) [2023-12-27 00:29:52,158][105620] Updated weights for policy 1, policy_version 1252433 (0.0008) [2023-12-27 00:29:52,213][105620] Updated weights for policy 1, policy_version 1252443 (0.0008) [2023-12-27 00:29:52,419][105692] Updated weights for policy 0, policy_version 1251187 (0.0008) [2023-12-27 00:29:52,466][105692] Updated weights for policy 0, policy_version 1251197 (0.0005) [2023-12-27 00:29:52,511][105692] Updated weights for policy 0, policy_version 1251207 (0.0005) [2023-12-27 00:29:52,975][105620] Updated weights for policy 1, policy_version 1252453 (0.0009) [2023-12-27 00:29:53,036][105620] Updated weights for policy 1, policy_version 1252463 (0.0009) [2023-12-27 00:29:53,060][105692] Updated weights for policy 0, policy_version 1251217 (0.0005) [2023-12-27 00:29:53,097][105620] Updated weights for policy 1, policy_version 1252473 (0.0009) [2023-12-27 00:29:53,111][105692] Updated weights for policy 0, policy_version 1251227 (0.0006) [2023-12-27 00:29:53,170][105692] Updated weights for policy 0, policy_version 1251237 (0.0011) [2023-12-27 00:29:53,213][105692] Updated weights for policy 0, policy_version 1251247 (0.0008) [2023-12-27 00:29:53,748][105692] Updated weights for policy 0, policy_version 1251257 (0.0006) [2023-12-27 00:29:53,806][105692] Updated weights for policy 0, policy_version 1251267 (0.0009) [2023-12-27 00:29:53,849][105692] Updated weights for policy 0, policy_version 1251277 (0.0009) [2023-12-27 00:29:53,976][105620] Updated weights for policy 1, policy_version 1252483 (0.0009) [2023-12-27 00:29:54,030][105620] Updated weights for policy 1, policy_version 1252494 (0.0008) [2023-12-27 00:29:54,085][105620] Updated weights for policy 1, policy_version 1252504 (0.0009) [2023-12-27 00:29:54,433][105692] Updated weights for policy 0, policy_version 1251287 (0.0005) [2023-12-27 00:29:54,493][105692] Updated weights for policy 0, policy_version 1251297 (0.0010) [2023-12-27 00:29:54,545][105692] Updated weights for policy 0, policy_version 1251307 (0.0010) [2023-12-27 00:29:54,929][105620] Updated weights for policy 1, policy_version 1252514 (0.0009) [2023-12-27 00:29:54,991][105620] Updated weights for policy 1, policy_version 1252524 (0.0009) [2023-12-27 00:29:55,042][105620] Updated weights for policy 1, policy_version 1252534 (0.0007) [2023-12-27 00:29:55,094][105620] Updated weights for policy 1, policy_version 1252544 (0.0008) [2023-12-27 00:29:55,163][105692] Updated weights for policy 0, policy_version 1251317 (0.0008) [2023-12-27 00:29:55,226][105692] Updated weights for policy 0, policy_version 1251327 (0.0005) [2023-12-27 00:29:55,291][105692] Updated weights for policy 0, policy_version 1251337 (0.0005) [2023-12-27 00:29:55,866][105692] Updated weights for policy 0, policy_version 1251347 (0.0007) [2023-12-27 00:29:55,924][105692] Updated weights for policy 0, policy_version 1251357 (0.0006) [2023-12-27 00:29:55,926][105620] Updated weights for policy 1, policy_version 1252554 (0.0008) [2023-12-27 00:29:55,980][105692] Updated weights for policy 0, policy_version 1251367 (0.0006) [2023-12-27 00:29:55,989][105620] Updated weights for policy 1, policy_version 1252564 (0.0008) [2023-12-27 00:29:56,048][105620] Updated weights for policy 1, policy_version 1252574 (0.0007) [2023-12-27 00:29:56,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 641105920. Throughput: 0: 10087.2, 1: 9724.6. Samples: 641106776. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:29:56,062][104569] Avg episode reward: [(0, '9172.067'), (1, '9169.690')] [2023-12-27 00:29:56,672][105620] Updated weights for policy 1, policy_version 1252584 (0.0009) [2023-12-27 00:29:56,715][105620] Updated weights for policy 1, policy_version 1252594 (0.0006) [2023-12-27 00:29:56,752][105692] Updated weights for policy 0, policy_version 1251377 (0.0007) [2023-12-27 00:29:56,758][105620] Updated weights for policy 1, policy_version 1252604 (0.0005) [2023-12-27 00:29:56,798][105692] Updated weights for policy 0, policy_version 1251387 (0.0005) [2023-12-27 00:29:56,843][105692] Updated weights for policy 0, policy_version 1251397 (0.0005) [2023-12-27 00:29:56,887][105692] Updated weights for policy 0, policy_version 1251407 (0.0005) [2023-12-27 00:29:57,316][105620] Updated weights for policy 1, policy_version 1252614 (0.0006) [2023-12-27 00:29:57,367][105620] Updated weights for policy 1, policy_version 1252624 (0.0006) [2023-12-27 00:29:57,419][105620] Updated weights for policy 1, policy_version 1252634 (0.0005) [2023-12-27 00:29:57,564][105692] Updated weights for policy 0, policy_version 1251417 (0.0008) [2023-12-27 00:29:57,625][105692] Updated weights for policy 0, policy_version 1251427 (0.0008) [2023-12-27 00:29:57,681][105692] Updated weights for policy 0, policy_version 1251437 (0.0006) [2023-12-27 00:29:58,177][105620] Updated weights for policy 1, policy_version 1252644 (0.0009) [2023-12-27 00:29:58,219][105692] Updated weights for policy 0, policy_version 1251447 (0.0009) [2023-12-27 00:29:58,242][105620] Updated weights for policy 1, policy_version 1252654 (0.0006) [2023-12-27 00:29:58,280][105692] Updated weights for policy 0, policy_version 1251457 (0.0007) [2023-12-27 00:29:58,309][105620] Updated weights for policy 1, policy_version 1252664 (0.0007) [2023-12-27 00:29:58,342][105692] Updated weights for policy 0, policy_version 1251467 (0.0007) [2023-12-27 00:29:59,130][105620] Updated weights for policy 1, policy_version 1252674 (0.0008) [2023-12-27 00:29:59,134][105692] Updated weights for policy 0, policy_version 1251477 (0.0010) [2023-12-27 00:29:59,197][105620] Updated weights for policy 1, policy_version 1252684 (0.0006) [2023-12-27 00:29:59,199][105692] Updated weights for policy 0, policy_version 1251487 (0.0010) [2023-12-27 00:29:59,268][105692] Updated weights for policy 0, policy_version 1251497 (0.0009) [2023-12-27 00:29:59,275][105620] Updated weights for policy 1, policy_version 1252694 (0.0011) [2023-12-27 00:29:59,341][105620] Updated weights for policy 1, policy_version 1252704 (0.0011) [2023-12-27 00:29:59,985][105620] Updated weights for policy 1, policy_version 1252714 (0.0010) [2023-12-27 00:30:00,009][105692] Updated weights for policy 0, policy_version 1251507 (0.0008) [2023-12-27 00:30:00,046][105620] Updated weights for policy 1, policy_version 1252724 (0.0009) [2023-12-27 00:30:00,069][105692] Updated weights for policy 0, policy_version 1251517 (0.0009) [2023-12-27 00:30:00,095][105620] Updated weights for policy 1, policy_version 1252734 (0.0005) [2023-12-27 00:30:00,122][105692] Updated weights for policy 0, policy_version 1251527 (0.0008) [2023-12-27 00:30:00,693][105620] Updated weights for policy 1, policy_version 1252744 (0.0010) [2023-12-27 00:30:00,749][105620] Updated weights for policy 1, policy_version 1252754 (0.0011) [2023-12-27 00:30:00,808][105620] Updated weights for policy 1, policy_version 1252764 (0.0010) [2023-12-27 00:30:00,826][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000006 [2023-12-27 00:30:00,916][105692] Updated weights for policy 0, policy_version 1251537 (0.0009) [2023-12-27 00:30:00,960][105692] Updated weights for policy 0, policy_version 1251547 (0.0007) [2023-12-27 00:30:01,007][105692] Updated weights for policy 0, policy_version 1251557 (0.0007) [2023-12-27 00:30:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 641196032. Throughput: 0: 10180.5, 1: 9718.6. Samples: 641168616. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:30:01,062][104569] Avg episode reward: [(0, '9079.676'), (1, '9169.029')] [2023-12-27 00:30:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001252768_320749568.pth... [2023-12-27 00:30:01,067][105692] Updated weights for policy 0, policy_version 1251567 (0.0009) [2023-12-27 00:30:01,069][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001251648_320462848.pth [2023-12-27 00:30:01,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001251568_320454656.pth... [2023-12-27 00:30:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001250352_320143360.pth [2023-12-27 00:30:01,551][105620] Updated weights for policy 1, policy_version 1252774 (0.0009) [2023-12-27 00:30:01,599][105620] Updated weights for policy 1, policy_version 1252784 (0.0008) [2023-12-27 00:30:01,653][105620] Updated weights for policy 1, policy_version 1252794 (0.0009) [2023-12-27 00:30:01,858][105692] Updated weights for policy 0, policy_version 1251577 (0.0009) [2023-12-27 00:30:01,913][105692] Updated weights for policy 0, policy_version 1251587 (0.0009) [2023-12-27 00:30:01,974][105692] Updated weights for policy 0, policy_version 1251597 (0.0009) [2023-12-27 00:30:02,459][105620] Updated weights for policy 1, policy_version 1252804 (0.0009) [2023-12-27 00:30:02,519][105620] Updated weights for policy 1, policy_version 1252814 (0.0008) [2023-12-27 00:30:02,573][105620] Updated weights for policy 1, policy_version 1252824 (0.0009) [2023-12-27 00:30:02,707][105692] Updated weights for policy 0, policy_version 1251607 (0.0009) [2023-12-27 00:30:02,757][105692] Updated weights for policy 0, policy_version 1251617 (0.0008) [2023-12-27 00:30:02,808][105692] Updated weights for policy 0, policy_version 1251627 (0.0009) [2023-12-27 00:30:03,331][105620] Updated weights for policy 1, policy_version 1252834 (0.0009) [2023-12-27 00:30:03,388][105620] Updated weights for policy 1, policy_version 1252844 (0.0008) [2023-12-27 00:30:03,435][105620] Updated weights for policy 1, policy_version 1252854 (0.0009) [2023-12-27 00:30:03,490][105620] Updated weights for policy 1, policy_version 1252864 (0.0007) [2023-12-27 00:30:03,555][105692] Updated weights for policy 0, policy_version 1251637 (0.0009) [2023-12-27 00:30:03,607][105692] Updated weights for policy 0, policy_version 1251647 (0.0009) [2023-12-27 00:30:03,660][105692] Updated weights for policy 0, policy_version 1251657 (0.0009) [2023-12-27 00:30:04,150][105620] Updated weights for policy 1, policy_version 1252874 (0.0005) [2023-12-27 00:30:04,217][105620] Updated weights for policy 1, policy_version 1252884 (0.0006) [2023-12-27 00:30:04,278][105620] Updated weights for policy 1, policy_version 1252894 (0.0007) [2023-12-27 00:30:04,496][105692] Updated weights for policy 0, policy_version 1251667 (0.0009) [2023-12-27 00:30:04,558][105692] Updated weights for policy 0, policy_version 1251677 (0.0009) [2023-12-27 00:30:04,628][105692] Updated weights for policy 0, policy_version 1251687 (0.0009) [2023-12-27 00:30:04,919][105620] Updated weights for policy 1, policy_version 1252904 (0.0008) [2023-12-27 00:30:04,973][105620] Updated weights for policy 1, policy_version 1252914 (0.0010) [2023-12-27 00:30:05,029][105620] Updated weights for policy 1, policy_version 1252924 (0.0010) [2023-12-27 00:30:05,251][105692] Updated weights for policy 0, policy_version 1251697 (0.0008) [2023-12-27 00:30:05,307][105692] Updated weights for policy 0, policy_version 1251707 (0.0005) [2023-12-27 00:30:05,360][105692] Updated weights for policy 0, policy_version 1251717 (0.0005) [2023-12-27 00:30:05,418][105692] Updated weights for policy 0, policy_version 1251727 (0.0009) [2023-12-27 00:30:05,746][105620] Updated weights for policy 1, policy_version 1252934 (0.0009) [2023-12-27 00:30:05,795][105620] Updated weights for policy 1, policy_version 1252944 (0.0009) [2023-12-27 00:30:05,853][105620] Updated weights for policy 1, policy_version 1252954 (0.0009) [2023-12-27 00:30:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 641294336. Throughput: 0: 10076.2, 1: 9720.4. Samples: 641283024. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:30:06,063][104569] Avg episode reward: [(0, '8988.840'), (1, '9077.066')] [2023-12-27 00:30:06,159][105692] Updated weights for policy 0, policy_version 1251737 (0.0009) [2023-12-27 00:30:06,210][105692] Updated weights for policy 0, policy_version 1251747 (0.0009) [2023-12-27 00:30:06,264][105692] Updated weights for policy 0, policy_version 1251757 (0.0009) [2023-12-27 00:30:06,635][105620] Updated weights for policy 1, policy_version 1252964 (0.0009) [2023-12-27 00:30:06,688][105620] Updated weights for policy 1, policy_version 1252974 (0.0009) [2023-12-27 00:30:06,743][105620] Updated weights for policy 1, policy_version 1252984 (0.0009) [2023-12-27 00:30:07,060][105692] Updated weights for policy 0, policy_version 1251767 (0.0009) [2023-12-27 00:30:07,114][105692] Updated weights for policy 0, policy_version 1251777 (0.0008) [2023-12-27 00:30:07,174][105692] Updated weights for policy 0, policy_version 1251787 (0.0009) [2023-12-27 00:30:07,526][105620] Updated weights for policy 1, policy_version 1252994 (0.0009) [2023-12-27 00:30:07,576][105620] Updated weights for policy 1, policy_version 1253004 (0.0006) [2023-12-27 00:30:07,626][105620] Updated weights for policy 1, policy_version 1253014 (0.0005) [2023-12-27 00:30:07,685][105620] Updated weights for policy 1, policy_version 1253024 (0.0005) [2023-12-27 00:30:07,954][105692] Updated weights for policy 0, policy_version 1251797 (0.0010) [2023-12-27 00:30:08,018][105692] Updated weights for policy 0, policy_version 1251807 (0.0009) [2023-12-27 00:30:08,076][105692] Updated weights for policy 0, policy_version 1251817 (0.0010) [2023-12-27 00:30:08,257][105620] Updated weights for policy 1, policy_version 1253034 (0.0011) [2023-12-27 00:30:08,306][105620] Updated weights for policy 1, policy_version 1253044 (0.0011) [2023-12-27 00:30:08,379][105620] Updated weights for policy 1, policy_version 1253054 (0.0009) [2023-12-27 00:30:08,827][105692] Updated weights for policy 0, policy_version 1251827 (0.0009) [2023-12-27 00:30:08,890][105692] Updated weights for policy 0, policy_version 1251837 (0.0008) [2023-12-27 00:30:08,955][105692] Updated weights for policy 0, policy_version 1251847 (0.0008) [2023-12-27 00:30:09,120][105620] Updated weights for policy 1, policy_version 1253064 (0.0011) [2023-12-27 00:30:09,176][105620] Updated weights for policy 1, policy_version 1253074 (0.0010) [2023-12-27 00:30:09,240][105620] Updated weights for policy 1, policy_version 1253084 (0.0011) [2023-12-27 00:30:09,742][105692] Updated weights for policy 0, policy_version 1251857 (0.0008) [2023-12-27 00:30:09,805][105692] Updated weights for policy 0, policy_version 1251867 (0.0008) [2023-12-27 00:30:09,871][105692] Updated weights for policy 0, policy_version 1251877 (0.0008) [2023-12-27 00:30:09,925][105692] Updated weights for policy 0, policy_version 1251887 (0.0008) [2023-12-27 00:30:09,962][105620] Updated weights for policy 1, policy_version 1253094 (0.0010) [2023-12-27 00:30:10,014][105620] Updated weights for policy 1, policy_version 1253104 (0.0009) [2023-12-27 00:30:10,073][105620] Updated weights for policy 1, policy_version 1253114 (0.0009) [2023-12-27 00:30:10,678][105620] Updated weights for policy 1, policy_version 1253124 (0.0008) [2023-12-27 00:30:10,733][105620] Updated weights for policy 1, policy_version 1253134 (0.0006) [2023-12-27 00:30:10,784][105692] Updated weights for policy 0, policy_version 1251897 (0.0007) [2023-12-27 00:30:10,798][105620] Updated weights for policy 1, policy_version 1253144 (0.0007) [2023-12-27 00:30:10,841][105692] Updated weights for policy 0, policy_version 1251907 (0.0006) [2023-12-27 00:30:10,902][105692] Updated weights for policy 0, policy_version 1251917 (0.0009) [2023-12-27 00:30:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 641392640. Throughput: 0: 10114.9, 1: 9687.7. Samples: 641397888. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 00:30:11,063][104569] Avg episode reward: [(0, '9172.044'), (1, '8894.935')] [2023-12-27 00:30:11,470][105620] Updated weights for policy 1, policy_version 1253154 (0.0008) [2023-12-27 00:30:11,527][105620] Updated weights for policy 1, policy_version 1253164 (0.0009) [2023-12-27 00:30:11,579][105620] Updated weights for policy 1, policy_version 1253174 (0.0008) [2023-12-27 00:30:11,641][105620] Updated weights for policy 1, policy_version 1253184 (0.0009) [2023-12-27 00:30:11,724][105692] Updated weights for policy 0, policy_version 1251927 (0.0008) [2023-12-27 00:30:11,785][105692] Updated weights for policy 0, policy_version 1251937 (0.0009) [2023-12-27 00:30:11,846][105692] Updated weights for policy 0, policy_version 1251947 (0.0008) [2023-12-27 00:30:12,370][105620] Updated weights for policy 1, policy_version 1253194 (0.0009) [2023-12-27 00:30:12,423][105620] Updated weights for policy 1, policy_version 1253204 (0.0006) [2023-12-27 00:30:12,472][105620] Updated weights for policy 1, policy_version 1253214 (0.0008) [2023-12-27 00:30:12,677][105692] Updated weights for policy 0, policy_version 1251957 (0.0008) [2023-12-27 00:30:12,730][105692] Updated weights for policy 0, policy_version 1251967 (0.0008) [2023-12-27 00:30:12,786][105692] Updated weights for policy 0, policy_version 1251977 (0.0008) [2023-12-27 00:30:13,199][105620] Updated weights for policy 1, policy_version 1253224 (0.0007) [2023-12-27 00:30:13,249][105620] Updated weights for policy 1, policy_version 1253234 (0.0005) [2023-12-27 00:30:13,297][105620] Updated weights for policy 1, policy_version 1253244 (0.0008) [2023-12-27 00:30:13,469][105692] Updated weights for policy 0, policy_version 1251987 (0.0009) [2023-12-27 00:30:13,520][105692] Updated weights for policy 0, policy_version 1251997 (0.0010) [2023-12-27 00:30:13,576][105692] Updated weights for policy 0, policy_version 1252007 (0.0010) [2023-12-27 00:30:14,019][105620] Updated weights for policy 1, policy_version 1253254 (0.0010) [2023-12-27 00:30:14,070][105620] Updated weights for policy 1, policy_version 1253264 (0.0010) [2023-12-27 00:30:14,129][105620] Updated weights for policy 1, policy_version 1253274 (0.0011) [2023-12-27 00:30:14,152][105692] Updated weights for policy 0, policy_version 1252017 (0.0010) [2023-12-27 00:30:14,206][105692] Updated weights for policy 0, policy_version 1252027 (0.0011) [2023-12-27 00:30:14,254][105692] Updated weights for policy 0, policy_version 1252037 (0.0010) [2023-12-27 00:30:14,308][105692] Updated weights for policy 0, policy_version 1252047 (0.0008) [2023-12-27 00:30:14,864][105620] Updated weights for policy 1, policy_version 1253284 (0.0009) [2023-12-27 00:30:14,921][105620] Updated weights for policy 1, policy_version 1253294 (0.0009) [2023-12-27 00:30:14,978][105620] Updated weights for policy 1, policy_version 1253304 (0.0008) [2023-12-27 00:30:15,024][105692] Updated weights for policy 0, policy_version 1252057 (0.0008) [2023-12-27 00:30:15,071][105692] Updated weights for policy 0, policy_version 1252067 (0.0008) [2023-12-27 00:30:15,120][105692] Updated weights for policy 0, policy_version 1252077 (0.0008) [2023-12-27 00:30:15,777][105620] Updated weights for policy 1, policy_version 1253314 (0.0007) [2023-12-27 00:30:15,833][105620] Updated weights for policy 1, policy_version 1253324 (0.0008) [2023-12-27 00:30:15,880][105620] Updated weights for policy 1, policy_version 1253334 (0.0008) [2023-12-27 00:30:15,892][105692] Updated weights for policy 0, policy_version 1252087 (0.0010) [2023-12-27 00:30:15,927][105620] Updated weights for policy 1, policy_version 1253344 (0.0010) [2023-12-27 00:30:15,946][105692] Updated weights for policy 0, policy_version 1252097 (0.0009) [2023-12-27 00:30:15,998][105692] Updated weights for policy 0, policy_version 1252107 (0.0010) [2023-12-27 00:30:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 641490944. Throughput: 0: 9945.3, 1: 9703.5. Samples: 641454104. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:30:16,063][104569] Avg episode reward: [(0, '9171.171'), (1, '8987.112')] [2023-12-27 00:30:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001252112_320593920.pth... [2023-12-27 00:30:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001253344_320897024.pth... [2023-12-27 00:30:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001252192_320602112.pth [2023-12-27 00:30:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001250928_320290816.pth [2023-12-27 00:30:16,608][105620] Updated weights for policy 1, policy_version 1253354 (0.0008) [2023-12-27 00:30:16,658][105620] Updated weights for policy 1, policy_version 1253364 (0.0005) [2023-12-27 00:30:16,714][105620] Updated weights for policy 1, policy_version 1253374 (0.0005) [2023-12-27 00:30:16,817][105692] Updated weights for policy 0, policy_version 1252117 (0.0010) [2023-12-27 00:30:16,877][105692] Updated weights for policy 0, policy_version 1252127 (0.0009) [2023-12-27 00:30:16,932][105692] Updated weights for policy 0, policy_version 1252137 (0.0009) [2023-12-27 00:30:17,367][105620] Updated weights for policy 1, policy_version 1253384 (0.0009) [2023-12-27 00:30:17,431][105620] Updated weights for policy 1, policy_version 1253394 (0.0010) [2023-12-27 00:30:17,487][105620] Updated weights for policy 1, policy_version 1253404 (0.0009) [2023-12-27 00:30:17,643][105692] Updated weights for policy 0, policy_version 1252148 (0.0010) [2023-12-27 00:30:17,691][105692] Updated weights for policy 0, policy_version 1252158 (0.0009) [2023-12-27 00:30:17,740][105692] Updated weights for policy 0, policy_version 1252169 (0.0010) [2023-12-27 00:30:18,148][105620] Updated weights for policy 1, policy_version 1253414 (0.0007) [2023-12-27 00:30:18,217][105620] Updated weights for policy 1, policy_version 1253424 (0.0008) [2023-12-27 00:30:18,275][105620] Updated weights for policy 1, policy_version 1253434 (0.0009) [2023-12-27 00:30:18,556][105692] Updated weights for policy 0, policy_version 1252179 (0.0009) [2023-12-27 00:30:18,610][105692] Updated weights for policy 0, policy_version 1252189 (0.0010) [2023-12-27 00:30:18,663][105692] Updated weights for policy 0, policy_version 1252199 (0.0008) [2023-12-27 00:30:18,925][105620] Updated weights for policy 1, policy_version 1253444 (0.0008) [2023-12-27 00:30:18,988][105620] Updated weights for policy 1, policy_version 1253454 (0.0006) [2023-12-27 00:30:19,046][105620] Updated weights for policy 1, policy_version 1253464 (0.0009) [2023-12-27 00:30:19,451][105692] Updated weights for policy 0, policy_version 1252209 (0.0005) [2023-12-27 00:30:19,517][105692] Updated weights for policy 0, policy_version 1252219 (0.0008) [2023-12-27 00:30:19,576][105692] Updated weights for policy 0, policy_version 1252229 (0.0006) [2023-12-27 00:30:19,640][105692] Updated weights for policy 0, policy_version 1252239 (0.0007) [2023-12-27 00:30:19,764][105620] Updated weights for policy 1, policy_version 1253474 (0.0009) [2023-12-27 00:30:19,840][105620] Updated weights for policy 1, policy_version 1253484 (0.0009) [2023-12-27 00:30:19,910][105620] Updated weights for policy 1, policy_version 1253494 (0.0009) [2023-12-27 00:30:19,977][105620] Updated weights for policy 1, policy_version 1253504 (0.0009) [2023-12-27 00:30:20,367][105692] Updated weights for policy 0, policy_version 1252249 (0.0009) [2023-12-27 00:30:20,427][105692] Updated weights for policy 0, policy_version 1252259 (0.0009) [2023-12-27 00:30:20,490][105692] Updated weights for policy 0, policy_version 1252269 (0.0009) [2023-12-27 00:30:20,711][105620] Updated weights for policy 1, policy_version 1253514 (0.0010) [2023-12-27 00:30:20,774][105620] Updated weights for policy 1, policy_version 1253524 (0.0011) [2023-12-27 00:30:20,843][105620] Updated weights for policy 1, policy_version 1253534 (0.0010) [2023-12-27 00:30:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 641581056. Throughput: 0: 9835.6, 1: 9761.4. Samples: 641570600. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:30:21,063][104569] Avg episode reward: [(0, '9081.077'), (1, '9170.140')] [2023-12-27 00:30:21,176][105692] Updated weights for policy 0, policy_version 1252279 (0.0009) [2023-12-27 00:30:21,239][105692] Updated weights for policy 0, policy_version 1252289 (0.0008) [2023-12-27 00:30:21,291][105692] Updated weights for policy 0, policy_version 1252299 (0.0008) [2023-12-27 00:30:21,621][105620] Updated weights for policy 1, policy_version 1253544 (0.0010) [2023-12-27 00:30:21,681][105620] Updated weights for policy 1, policy_version 1253554 (0.0009) [2023-12-27 00:30:21,744][105620] Updated weights for policy 1, policy_version 1253564 (0.0009) [2023-12-27 00:30:22,092][105692] Updated weights for policy 0, policy_version 1252309 (0.0009) [2023-12-27 00:30:22,139][105692] Updated weights for policy 0, policy_version 1252319 (0.0009) [2023-12-27 00:30:22,206][105692] Updated weights for policy 0, policy_version 1252329 (0.0009) [2023-12-27 00:30:22,524][105620] Updated weights for policy 1, policy_version 1253574 (0.0007) [2023-12-27 00:30:22,581][105620] Updated weights for policy 1, policy_version 1253584 (0.0009) [2023-12-27 00:30:22,639][105620] Updated weights for policy 1, policy_version 1253594 (0.0009) [2023-12-27 00:30:23,013][105692] Updated weights for policy 0, policy_version 1252339 (0.0009) [2023-12-27 00:30:23,077][105692] Updated weights for policy 0, policy_version 1252349 (0.0009) [2023-12-27 00:30:23,140][105692] Updated weights for policy 0, policy_version 1252359 (0.0009) [2023-12-27 00:30:23,371][105620] Updated weights for policy 1, policy_version 1253604 (0.0009) [2023-12-27 00:30:23,424][105620] Updated weights for policy 1, policy_version 1253614 (0.0008) [2023-12-27 00:30:23,474][105620] Updated weights for policy 1, policy_version 1253624 (0.0008) [2023-12-27 00:30:23,871][105692] Updated weights for policy 0, policy_version 1252369 (0.0009) [2023-12-27 00:30:23,922][105692] Updated weights for policy 0, policy_version 1252379 (0.0009) [2023-12-27 00:30:23,977][105692] Updated weights for policy 0, policy_version 1252389 (0.0005) [2023-12-27 00:30:24,029][105692] Updated weights for policy 0, policy_version 1252399 (0.0005) [2023-12-27 00:30:24,276][105620] Updated weights for policy 1, policy_version 1253634 (0.0009) [2023-12-27 00:30:24,330][105620] Updated weights for policy 1, policy_version 1253644 (0.0009) [2023-12-27 00:30:24,376][105620] Updated weights for policy 1, policy_version 1253654 (0.0009) [2023-12-27 00:30:24,428][105620] Updated weights for policy 1, policy_version 1253664 (0.0005) [2023-12-27 00:30:24,712][105692] Updated weights for policy 0, policy_version 1252409 (0.0009) [2023-12-27 00:30:24,762][105692] Updated weights for policy 0, policy_version 1252419 (0.0009) [2023-12-27 00:30:24,812][105692] Updated weights for policy 0, policy_version 1252429 (0.0009) [2023-12-27 00:30:25,085][105620] Updated weights for policy 1, policy_version 1253674 (0.0005) [2023-12-27 00:30:25,153][105620] Updated weights for policy 1, policy_version 1253684 (0.0005) [2023-12-27 00:30:25,217][105620] Updated weights for policy 1, policy_version 1253694 (0.0005) [2023-12-27 00:30:25,514][105692] Updated weights for policy 0, policy_version 1252439 (0.0006) [2023-12-27 00:30:25,567][105692] Updated weights for policy 0, policy_version 1252449 (0.0007) [2023-12-27 00:30:25,615][105692] Updated weights for policy 0, policy_version 1252459 (0.0009) [2023-12-27 00:30:25,788][105620] Updated weights for policy 1, policy_version 1253704 (0.0005) [2023-12-27 00:30:25,853][105620] Updated weights for policy 1, policy_version 1253714 (0.0006) [2023-12-27 00:30:25,912][105620] Updated weights for policy 1, policy_version 1253724 (0.0006) [2023-12-27 00:30:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 641679360. Throughput: 0: 9782.8, 1: 9800.2. Samples: 641686148. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:30:26,063][104569] Avg episode reward: [(0, '9081.980'), (1, '8988.702')] [2023-12-27 00:30:26,323][105692] Updated weights for policy 0, policy_version 1252470 (0.0006) [2023-12-27 00:30:26,374][105692] Updated weights for policy 0, policy_version 1252480 (0.0008) [2023-12-27 00:30:26,423][105692] Updated weights for policy 0, policy_version 1252490 (0.0009) [2023-12-27 00:30:26,487][105620] Updated weights for policy 1, policy_version 1253734 (0.0007) [2023-12-27 00:30:26,543][105620] Updated weights for policy 1, policy_version 1253744 (0.0008) [2023-12-27 00:30:26,598][105620] Updated weights for policy 1, policy_version 1253754 (0.0009) [2023-12-27 00:30:27,137][105692] Updated weights for policy 0, policy_version 1252500 (0.0009) [2023-12-27 00:30:27,193][105692] Updated weights for policy 0, policy_version 1252510 (0.0009) [2023-12-27 00:30:27,246][105692] Updated weights for policy 0, policy_version 1252520 (0.0009) [2023-12-27 00:30:27,372][105620] Updated weights for policy 1, policy_version 1253764 (0.0008) [2023-12-27 00:30:27,425][105620] Updated weights for policy 1, policy_version 1253774 (0.0008) [2023-12-27 00:30:27,485][105620] Updated weights for policy 1, policy_version 1253784 (0.0009) [2023-12-27 00:30:28,032][105692] Updated weights for policy 0, policy_version 1252530 (0.0009) [2023-12-27 00:30:28,081][105692] Updated weights for policy 0, policy_version 1252540 (0.0010) [2023-12-27 00:30:28,135][105692] Updated weights for policy 0, policy_version 1252551 (0.0008) [2023-12-27 00:30:28,200][105620] Updated weights for policy 1, policy_version 1253794 (0.0008) [2023-12-27 00:30:28,252][105620] Updated weights for policy 1, policy_version 1253804 (0.0008) [2023-12-27 00:30:28,303][105620] Updated weights for policy 1, policy_version 1253814 (0.0010) [2023-12-27 00:30:28,363][105620] Updated weights for policy 1, policy_version 1253824 (0.0010) [2023-12-27 00:30:28,998][105692] Updated weights for policy 0, policy_version 1252561 (0.0010) [2023-12-27 00:30:29,002][105620] Updated weights for policy 1, policy_version 1253834 (0.0007) [2023-12-27 00:30:29,046][105620] Updated weights for policy 1, policy_version 1253844 (0.0009) [2023-12-27 00:30:29,052][105692] Updated weights for policy 0, policy_version 1252571 (0.0007) [2023-12-27 00:30:29,095][105620] Updated weights for policy 1, policy_version 1253854 (0.0010) [2023-12-27 00:30:29,097][105692] Updated weights for policy 0, policy_version 1252581 (0.0005) [2023-12-27 00:30:29,141][105692] Updated weights for policy 0, policy_version 1252591 (0.0008) [2023-12-27 00:30:29,865][105620] Updated weights for policy 1, policy_version 1253864 (0.0010) [2023-12-27 00:30:29,929][105620] Updated weights for policy 1, policy_version 1253874 (0.0011) [2023-12-27 00:30:29,931][105692] Updated weights for policy 0, policy_version 1252601 (0.0008) [2023-12-27 00:30:29,983][105692] Updated weights for policy 0, policy_version 1252611 (0.0010) [2023-12-27 00:30:29,988][105620] Updated weights for policy 1, policy_version 1253884 (0.0010) [2023-12-27 00:30:30,035][105692] Updated weights for policy 0, policy_version 1252621 (0.0010) [2023-12-27 00:30:30,580][105692] Updated weights for policy 0, policy_version 1252631 (0.0008) [2023-12-27 00:30:30,631][105692] Updated weights for policy 0, policy_version 1252641 (0.0009) [2023-12-27 00:30:30,681][105692] Updated weights for policy 0, policy_version 1252651 (0.0009) [2023-12-27 00:30:30,746][105620] Updated weights for policy 1, policy_version 1253894 (0.0010) [2023-12-27 00:30:30,803][105620] Updated weights for policy 1, policy_version 1253904 (0.0008) [2023-12-27 00:30:30,876][105620] Updated weights for policy 1, policy_version 1253914 (0.0005) [2023-12-27 00:30:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 641777664. Throughput: 0: 9775.8, 1: 9765.5. Samples: 641744420. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:30:31,062][104569] Avg episode reward: [(0, '9171.826'), (1, '9079.490')] [2023-12-27 00:30:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001252656_320733184.pth... [2023-12-27 00:30:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001253920_321044480.pth... [2023-12-27 00:30:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001251568_320454656.pth [2023-12-27 00:30:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001252768_320749568.pth [2023-12-27 00:30:31,388][105692] Updated weights for policy 0, policy_version 1252661 (0.0009) [2023-12-27 00:30:31,454][105692] Updated weights for policy 0, policy_version 1252671 (0.0010) [2023-12-27 00:30:31,518][105692] Updated weights for policy 0, policy_version 1252681 (0.0010) [2023-12-27 00:30:31,656][105620] Updated weights for policy 1, policy_version 1253924 (0.0009) [2023-12-27 00:30:31,712][105620] Updated weights for policy 1, policy_version 1253934 (0.0009) [2023-12-27 00:30:31,784][105620] Updated weights for policy 1, policy_version 1253944 (0.0009) [2023-12-27 00:30:32,185][105692] Updated weights for policy 0, policy_version 1252691 (0.0009) [2023-12-27 00:30:32,237][105692] Updated weights for policy 0, policy_version 1252701 (0.0005) [2023-12-27 00:30:32,294][105692] Updated weights for policy 0, policy_version 1252711 (0.0010) [2023-12-27 00:30:32,514][105620] Updated weights for policy 1, policy_version 1253954 (0.0008) [2023-12-27 00:30:32,574][105620] Updated weights for policy 1, policy_version 1253964 (0.0011) [2023-12-27 00:30:32,633][105620] Updated weights for policy 1, policy_version 1253974 (0.0010) [2023-12-27 00:30:32,694][105620] Updated weights for policy 1, policy_version 1253984 (0.0009) [2023-12-27 00:30:32,951][105692] Updated weights for policy 0, policy_version 1252721 (0.0009) [2023-12-27 00:30:33,008][105692] Updated weights for policy 0, policy_version 1252731 (0.0006) [2023-12-27 00:30:33,065][105692] Updated weights for policy 0, policy_version 1252741 (0.0006) [2023-12-27 00:30:33,124][105692] Updated weights for policy 0, policy_version 1252751 (0.0005) [2023-12-27 00:30:33,345][105620] Updated weights for policy 1, policy_version 1253994 (0.0006) [2023-12-27 00:30:33,391][105620] Updated weights for policy 1, policy_version 1254004 (0.0005) [2023-12-27 00:30:33,447][105620] Updated weights for policy 1, policy_version 1254014 (0.0005) [2023-12-27 00:30:33,662][105692] Updated weights for policy 0, policy_version 1252761 (0.0005) [2023-12-27 00:30:33,708][105692] Updated weights for policy 0, policy_version 1252771 (0.0005) [2023-12-27 00:30:33,764][105692] Updated weights for policy 0, policy_version 1252781 (0.0005) [2023-12-27 00:30:34,053][105620] Updated weights for policy 1, policy_version 1254024 (0.0009) [2023-12-27 00:30:34,105][105620] Updated weights for policy 1, policy_version 1254034 (0.0009) [2023-12-27 00:30:34,170][105620] Updated weights for policy 1, policy_version 1254044 (0.0007) [2023-12-27 00:30:34,359][105692] Updated weights for policy 0, policy_version 1252791 (0.0009) [2023-12-27 00:30:34,419][105692] Updated weights for policy 0, policy_version 1252801 (0.0010) [2023-12-27 00:30:34,482][105692] Updated weights for policy 0, policy_version 1252811 (0.0008) [2023-12-27 00:30:34,939][105620] Updated weights for policy 1, policy_version 1254054 (0.0009) [2023-12-27 00:30:35,007][105620] Updated weights for policy 1, policy_version 1254064 (0.0010) [2023-12-27 00:30:35,068][105620] Updated weights for policy 1, policy_version 1254074 (0.0006) [2023-12-27 00:30:35,165][105692] Updated weights for policy 0, policy_version 1252821 (0.0009) [2023-12-27 00:30:35,216][105692] Updated weights for policy 0, policy_version 1252831 (0.0008) [2023-12-27 00:30:35,266][105692] Updated weights for policy 0, policy_version 1252841 (0.0009) [2023-12-27 00:30:35,724][105620] Updated weights for policy 1, policy_version 1254084 (0.0008) [2023-12-27 00:30:35,785][105620] Updated weights for policy 1, policy_version 1254094 (0.0005) [2023-12-27 00:30:35,845][105620] Updated weights for policy 1, policy_version 1254104 (0.0007) [2023-12-27 00:30:35,949][105692] Updated weights for policy 0, policy_version 1252851 (0.0010) [2023-12-27 00:30:36,016][105692] Updated weights for policy 0, policy_version 1252861 (0.0010) [2023-12-27 00:30:36,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 641875968. Throughput: 0: 9724.0, 1: 9732.5. Samples: 641865500. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:30:36,062][104569] Avg episode reward: [(0, '9170.455'), (1, '9169.570')] [2023-12-27 00:30:36,077][105692] Updated weights for policy 0, policy_version 1252871 (0.0010) [2023-12-27 00:30:36,497][105620] Updated weights for policy 1, policy_version 1254114 (0.0009) [2023-12-27 00:30:36,559][105620] Updated weights for policy 1, policy_version 1254124 (0.0008) [2023-12-27 00:30:36,612][105620] Updated weights for policy 1, policy_version 1254134 (0.0010) [2023-12-27 00:30:36,674][105620] Updated weights for policy 1, policy_version 1254144 (0.0008) [2023-12-27 00:30:36,749][105692] Updated weights for policy 0, policy_version 1252881 (0.0009) [2023-12-27 00:30:36,800][105692] Updated weights for policy 0, policy_version 1252891 (0.0007) [2023-12-27 00:30:36,860][105692] Updated weights for policy 0, policy_version 1252901 (0.0007) [2023-12-27 00:30:36,919][105692] Updated weights for policy 0, policy_version 1252911 (0.0010) [2023-12-27 00:30:37,413][105620] Updated weights for policy 1, policy_version 1254154 (0.0010) [2023-12-27 00:30:37,482][105620] Updated weights for policy 1, policy_version 1254164 (0.0010) [2023-12-27 00:30:37,541][105620] Updated weights for policy 1, policy_version 1254174 (0.0010) [2023-12-27 00:30:37,572][105692] Updated weights for policy 0, policy_version 1252921 (0.0009) [2023-12-27 00:30:37,631][105692] Updated weights for policy 0, policy_version 1252931 (0.0011) [2023-12-27 00:30:37,688][105692] Updated weights for policy 0, policy_version 1252941 (0.0009) [2023-12-27 00:30:38,232][105620] Updated weights for policy 1, policy_version 1254184 (0.0007) [2023-12-27 00:30:38,282][105620] Updated weights for policy 1, policy_version 1254194 (0.0009) [2023-12-27 00:30:38,344][105620] Updated weights for policy 1, policy_version 1254205 (0.0009) [2023-12-27 00:30:38,371][105692] Updated weights for policy 0, policy_version 1252951 (0.0008) [2023-12-27 00:30:38,422][105692] Updated weights for policy 0, policy_version 1252961 (0.0008) [2023-12-27 00:30:38,487][105692] Updated weights for policy 0, policy_version 1252971 (0.0009) [2023-12-27 00:30:39,088][105620] Updated weights for policy 1, policy_version 1254215 (0.0009) [2023-12-27 00:30:39,148][105620] Updated weights for policy 1, policy_version 1254225 (0.0010) [2023-12-27 00:30:39,156][105692] Updated weights for policy 0, policy_version 1252981 (0.0008) [2023-12-27 00:30:39,215][105620] Updated weights for policy 1, policy_version 1254235 (0.0009) [2023-12-27 00:30:39,217][105692] Updated weights for policy 0, policy_version 1252991 (0.0008) [2023-12-27 00:30:39,291][105692] Updated weights for policy 0, policy_version 1253001 (0.0008) [2023-12-27 00:30:39,878][105620] Updated weights for policy 1, policy_version 1254245 (0.0008) [2023-12-27 00:30:39,952][105620] Updated weights for policy 1, policy_version 1254255 (0.0010) [2023-12-27 00:30:40,018][105620] Updated weights for policy 1, policy_version 1254265 (0.0011) [2023-12-27 00:30:40,073][105692] Updated weights for policy 0, policy_version 1253011 (0.0009) [2023-12-27 00:30:40,133][105692] Updated weights for policy 0, policy_version 1253021 (0.0009) [2023-12-27 00:30:40,184][105692] Updated weights for policy 0, policy_version 1253031 (0.0008) [2023-12-27 00:30:40,755][105620] Updated weights for policy 1, policy_version 1254275 (0.0010) [2023-12-27 00:30:40,817][105620] Updated weights for policy 1, policy_version 1254285 (0.0010) [2023-12-27 00:30:40,880][105620] Updated weights for policy 1, policy_version 1254295 (0.0010) [2023-12-27 00:30:40,936][105692] Updated weights for policy 0, policy_version 1253041 (0.0008) [2023-12-27 00:30:40,993][105692] Updated weights for policy 0, policy_version 1253051 (0.0008) [2023-12-27 00:30:41,058][105692] Updated weights for policy 0, policy_version 1253061 (0.0008) [2023-12-27 00:30:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 641974272. Throughput: 0: 9620.7, 1: 9855.2. Samples: 641983192. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:30:41,063][104569] Avg episode reward: [(0, '9076.345'), (1, '8987.676')] [2023-12-27 00:30:41,123][105692] Updated weights for policy 0, policy_version 1253071 (0.0008) [2023-12-27 00:30:41,633][105620] Updated weights for policy 1, policy_version 1254305 (0.0010) [2023-12-27 00:30:41,700][105620] Updated weights for policy 1, policy_version 1254315 (0.0014) [2023-12-27 00:30:41,756][105620] Updated weights for policy 1, policy_version 1254325 (0.0011) [2023-12-27 00:30:41,809][105620] Updated weights for policy 1, policy_version 1254335 (0.0011) [2023-12-27 00:30:41,942][105692] Updated weights for policy 0, policy_version 1253081 (0.0008) [2023-12-27 00:30:42,010][105692] Updated weights for policy 0, policy_version 1253091 (0.0007) [2023-12-27 00:30:42,066][105692] Updated weights for policy 0, policy_version 1253101 (0.0008) [2023-12-27 00:30:42,551][105620] Updated weights for policy 1, policy_version 1254345 (0.0008) [2023-12-27 00:30:42,610][105620] Updated weights for policy 1, policy_version 1254355 (0.0011) [2023-12-27 00:30:42,679][105620] Updated weights for policy 1, policy_version 1254365 (0.0011) [2023-12-27 00:30:42,851][105692] Updated weights for policy 0, policy_version 1253111 (0.0008) [2023-12-27 00:30:42,899][105692] Updated weights for policy 0, policy_version 1253121 (0.0008) [2023-12-27 00:30:42,948][105692] Updated weights for policy 0, policy_version 1253131 (0.0007) [2023-12-27 00:30:43,373][105620] Updated weights for policy 1, policy_version 1254375 (0.0010) [2023-12-27 00:30:43,420][105620] Updated weights for policy 1, policy_version 1254385 (0.0010) [2023-12-27 00:30:43,468][105620] Updated weights for policy 1, policy_version 1254395 (0.0010) [2023-12-27 00:30:43,693][105692] Updated weights for policy 0, policy_version 1253141 (0.0007) [2023-12-27 00:30:43,753][105692] Updated weights for policy 0, policy_version 1253151 (0.0005) [2023-12-27 00:30:43,820][105692] Updated weights for policy 0, policy_version 1253161 (0.0005) [2023-12-27 00:30:44,178][105620] Updated weights for policy 1, policy_version 1254405 (0.0010) [2023-12-27 00:30:44,226][105620] Updated weights for policy 1, policy_version 1254415 (0.0010) [2023-12-27 00:30:44,270][105620] Updated weights for policy 1, policy_version 1254425 (0.0010) [2023-12-27 00:30:44,405][105692] Updated weights for policy 0, policy_version 1253171 (0.0007) [2023-12-27 00:30:44,455][105692] Updated weights for policy 0, policy_version 1253181 (0.0009) [2023-12-27 00:30:44,499][105692] Updated weights for policy 0, policy_version 1253191 (0.0005) [2023-12-27 00:30:45,064][105692] Updated weights for policy 0, policy_version 1253201 (0.0006) [2023-12-27 00:30:45,122][105620] Updated weights for policy 1, policy_version 1254435 (0.0009) [2023-12-27 00:30:45,126][105692] Updated weights for policy 0, policy_version 1253211 (0.0008) [2023-12-27 00:30:45,177][105620] Updated weights for policy 1, policy_version 1254445 (0.0006) [2023-12-27 00:30:45,194][105692] Updated weights for policy 0, policy_version 1253221 (0.0008) [2023-12-27 00:30:45,234][105620] Updated weights for policy 1, policy_version 1254455 (0.0005) [2023-12-27 00:30:45,264][105692] Updated weights for policy 0, policy_version 1253231 (0.0008) [2023-12-27 00:30:45,841][105620] Updated weights for policy 1, policy_version 1254465 (0.0007) [2023-12-27 00:30:45,907][105620] Updated weights for policy 1, policy_version 1254475 (0.0006) [2023-12-27 00:30:45,925][105692] Updated weights for policy 0, policy_version 1253241 (0.0007) [2023-12-27 00:30:45,966][105620] Updated weights for policy 1, policy_version 1254485 (0.0009) [2023-12-27 00:30:45,980][105692] Updated weights for policy 0, policy_version 1253251 (0.0007) [2023-12-27 00:30:46,015][105620] Updated weights for policy 1, policy_version 1254495 (0.0006) [2023-12-27 00:30:46,037][105692] Updated weights for policy 0, policy_version 1253261 (0.0007) [2023-12-27 00:30:46,062][104569] Fps is (10 sec: 20479.4, 60 sec: 19797.2, 300 sec: 19577.5). Total num frames: 642080768. Throughput: 0: 9540.3, 1: 9816.7. Samples: 642039688. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:30:46,063][104569] Avg episode reward: [(0, '9168.675'), (1, '9170.672')] [2023-12-27 00:30:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001253264_320888832.pth... [2023-12-27 00:30:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001254496_321191936.pth... [2023-12-27 00:30:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001252112_320593920.pth [2023-12-27 00:30:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001253344_320897024.pth [2023-12-27 00:30:46,745][105620] Updated weights for policy 1, policy_version 1254505 (0.0005) [2023-12-27 00:30:46,777][105692] Updated weights for policy 0, policy_version 1253271 (0.0009) [2023-12-27 00:30:46,799][105620] Updated weights for policy 1, policy_version 1254515 (0.0005) [2023-12-27 00:30:46,830][105692] Updated weights for policy 0, policy_version 1253281 (0.0009) [2023-12-27 00:30:46,854][105620] Updated weights for policy 1, policy_version 1254525 (0.0007) [2023-12-27 00:30:46,876][105692] Updated weights for policy 0, policy_version 1253291 (0.0009) [2023-12-27 00:30:47,447][105620] Updated weights for policy 1, policy_version 1254535 (0.0009) [2023-12-27 00:30:47,510][105620] Updated weights for policy 1, policy_version 1254545 (0.0011) [2023-12-27 00:30:47,569][105620] Updated weights for policy 1, policy_version 1254555 (0.0008) [2023-12-27 00:30:47,724][105692] Updated weights for policy 0, policy_version 1253301 (0.0008) [2023-12-27 00:30:47,785][105692] Updated weights for policy 0, policy_version 1253311 (0.0008) [2023-12-27 00:30:47,840][105692] Updated weights for policy 0, policy_version 1253321 (0.0008) [2023-12-27 00:30:48,246][105620] Updated weights for policy 1, policy_version 1254565 (0.0007) [2023-12-27 00:30:48,307][105620] Updated weights for policy 1, policy_version 1254575 (0.0006) [2023-12-27 00:30:48,371][105620] Updated weights for policy 1, policy_version 1254585 (0.0010) [2023-12-27 00:30:48,641][105692] Updated weights for policy 0, policy_version 1253331 (0.0008) [2023-12-27 00:30:48,689][105692] Updated weights for policy 0, policy_version 1253341 (0.0008) [2023-12-27 00:30:48,734][105692] Updated weights for policy 0, policy_version 1253351 (0.0008) [2023-12-27 00:30:49,086][105620] Updated weights for policy 1, policy_version 1254595 (0.0011) [2023-12-27 00:30:49,140][105620] Updated weights for policy 1, policy_version 1254605 (0.0009) [2023-12-27 00:30:49,197][105620] Updated weights for policy 1, policy_version 1254615 (0.0006) [2023-12-27 00:30:49,580][105692] Updated weights for policy 0, policy_version 1253361 (0.0008) [2023-12-27 00:30:49,647][105692] Updated weights for policy 0, policy_version 1253371 (0.0010) [2023-12-27 00:30:49,697][105692] Updated weights for policy 0, policy_version 1253381 (0.0010) [2023-12-27 00:30:49,751][105692] Updated weights for policy 0, policy_version 1253391 (0.0009) [2023-12-27 00:30:49,829][105620] Updated weights for policy 1, policy_version 1254625 (0.0009) [2023-12-27 00:30:49,892][105620] Updated weights for policy 1, policy_version 1254635 (0.0009) [2023-12-27 00:30:49,954][105620] Updated weights for policy 1, policy_version 1254645 (0.0009) [2023-12-27 00:30:50,013][105620] Updated weights for policy 1, policy_version 1254655 (0.0009) [2023-12-27 00:30:50,536][105692] Updated weights for policy 0, policy_version 1253401 (0.0009) [2023-12-27 00:30:50,602][105692] Updated weights for policy 0, policy_version 1253411 (0.0010) [2023-12-27 00:30:50,661][105692] Updated weights for policy 0, policy_version 1253421 (0.0009) [2023-12-27 00:30:50,778][105620] Updated weights for policy 1, policy_version 1254665 (0.0007) [2023-12-27 00:30:50,831][105620] Updated weights for policy 1, policy_version 1254675 (0.0009) [2023-12-27 00:30:50,890][105620] Updated weights for policy 1, policy_version 1254685 (0.0009) [2023-12-27 00:30:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 642170880. Throughput: 0: 9631.3, 1: 9829.1. Samples: 642158748. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:30:51,063][104569] Avg episode reward: [(0, '9261.465'), (1, '9078.604')] [2023-12-27 00:30:51,454][105692] Updated weights for policy 0, policy_version 1253431 (0.0006) [2023-12-27 00:30:51,506][105692] Updated weights for policy 0, policy_version 1253441 (0.0005) [2023-12-27 00:30:51,555][105692] Updated weights for policy 0, policy_version 1253451 (0.0006) [2023-12-27 00:30:51,721][105620] Updated weights for policy 1, policy_version 1254695 (0.0009) [2023-12-27 00:30:51,784][105620] Updated weights for policy 1, policy_version 1254705 (0.0008) [2023-12-27 00:30:51,851][105620] Updated weights for policy 1, policy_version 1254715 (0.0005) [2023-12-27 00:30:52,170][105692] Updated weights for policy 0, policy_version 1253461 (0.0011) [2023-12-27 00:30:52,221][105692] Updated weights for policy 0, policy_version 1253471 (0.0009) [2023-12-27 00:30:52,280][105692] Updated weights for policy 0, policy_version 1253481 (0.0009) [2023-12-27 00:30:52,629][105620] Updated weights for policy 1, policy_version 1254725 (0.0009) [2023-12-27 00:30:52,678][105620] Updated weights for policy 1, policy_version 1254735 (0.0011) [2023-12-27 00:30:52,746][105620] Updated weights for policy 1, policy_version 1254745 (0.0011) [2023-12-27 00:30:53,078][105692] Updated weights for policy 0, policy_version 1253491 (0.0011) [2023-12-27 00:30:53,130][105692] Updated weights for policy 0, policy_version 1253501 (0.0011) [2023-12-27 00:30:53,178][105692] Updated weights for policy 0, policy_version 1253511 (0.0010) [2023-12-27 00:30:53,483][105620] Updated weights for policy 1, policy_version 1254755 (0.0011) [2023-12-27 00:30:53,543][105620] Updated weights for policy 1, policy_version 1254765 (0.0007) [2023-12-27 00:30:53,596][105620] Updated weights for policy 1, policy_version 1254775 (0.0006) [2023-12-27 00:30:53,807][105692] Updated weights for policy 0, policy_version 1253521 (0.0010) [2023-12-27 00:30:53,869][105692] Updated weights for policy 0, policy_version 1253531 (0.0007) [2023-12-27 00:30:53,934][105692] Updated weights for policy 0, policy_version 1253541 (0.0011) [2023-12-27 00:30:53,996][105692] Updated weights for policy 0, policy_version 1253551 (0.0011) [2023-12-27 00:30:54,180][105620] Updated weights for policy 1, policy_version 1254785 (0.0006) [2023-12-27 00:30:54,243][105620] Updated weights for policy 1, policy_version 1254795 (0.0008) [2023-12-27 00:30:54,306][105620] Updated weights for policy 1, policy_version 1254805 (0.0008) [2023-12-27 00:30:54,360][105620] Updated weights for policy 1, policy_version 1254815 (0.0007) [2023-12-27 00:30:54,693][105692] Updated weights for policy 0, policy_version 1253561 (0.0010) [2023-12-27 00:30:54,741][105692] Updated weights for policy 0, policy_version 1253571 (0.0010) [2023-12-27 00:30:54,797][105692] Updated weights for policy 0, policy_version 1253581 (0.0007) [2023-12-27 00:30:55,050][105620] Updated weights for policy 1, policy_version 1254825 (0.0009) [2023-12-27 00:30:55,098][105620] Updated weights for policy 1, policy_version 1254835 (0.0009) [2023-12-27 00:30:55,157][105620] Updated weights for policy 1, policy_version 1254845 (0.0009) [2023-12-27 00:30:55,434][105692] Updated weights for policy 0, policy_version 1253591 (0.0008) [2023-12-27 00:30:55,488][105692] Updated weights for policy 0, policy_version 1253601 (0.0009) [2023-12-27 00:30:55,556][105692] Updated weights for policy 0, policy_version 1253611 (0.0008) [2023-12-27 00:30:55,986][105620] Updated weights for policy 1, policy_version 1254855 (0.0008) [2023-12-27 00:30:56,035][105620] Updated weights for policy 1, policy_version 1254865 (0.0008) [2023-12-27 00:30:56,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19251.1, 300 sec: 19521.9). Total num frames: 642260992. Throughput: 0: 9704.5, 1: 9753.6. Samples: 642273504. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:30:56,063][104569] Avg episode reward: [(0, '9169.189'), (1, '9077.936')] [2023-12-27 00:30:56,093][105620] Updated weights for policy 1, policy_version 1254875 (0.0006) [2023-12-27 00:30:56,196][105692] Updated weights for policy 0, policy_version 1253621 (0.0005) [2023-12-27 00:30:56,257][105692] Updated weights for policy 0, policy_version 1253631 (0.0005) [2023-12-27 00:30:56,317][105692] Updated weights for policy 0, policy_version 1253641 (0.0005) [2023-12-27 00:30:56,776][105620] Updated weights for policy 1, policy_version 1254885 (0.0007) [2023-12-27 00:30:56,827][105620] Updated weights for policy 1, policy_version 1254895 (0.0008) [2023-12-27 00:30:56,881][105620] Updated weights for policy 1, policy_version 1254905 (0.0008) [2023-12-27 00:30:56,975][105692] Updated weights for policy 0, policy_version 1253651 (0.0007) [2023-12-27 00:30:57,032][105692] Updated weights for policy 0, policy_version 1253661 (0.0010) [2023-12-27 00:30:57,085][105692] Updated weights for policy 0, policy_version 1253671 (0.0010) [2023-12-27 00:30:57,550][105620] Updated weights for policy 1, policy_version 1254915 (0.0007) [2023-12-27 00:30:57,613][105620] Updated weights for policy 1, policy_version 1254925 (0.0005) [2023-12-27 00:30:57,668][105620] Updated weights for policy 1, policy_version 1254935 (0.0005) [2023-12-27 00:30:57,805][105692] Updated weights for policy 0, policy_version 1253681 (0.0010) [2023-12-27 00:30:57,889][105692] Updated weights for policy 0, policy_version 1253691 (0.0010) [2023-12-27 00:30:57,939][105692] Updated weights for policy 0, policy_version 1253701 (0.0010) [2023-12-27 00:30:58,004][105692] Updated weights for policy 0, policy_version 1253711 (0.0010) [2023-12-27 00:30:58,216][105620] Updated weights for policy 1, policy_version 1254945 (0.0006) [2023-12-27 00:30:58,281][105620] Updated weights for policy 1, policy_version 1254955 (0.0008) [2023-12-27 00:30:58,352][105620] Updated weights for policy 1, policy_version 1254965 (0.0008) [2023-12-27 00:30:58,424][105620] Updated weights for policy 1, policy_version 1254975 (0.0008) [2023-12-27 00:30:58,754][105692] Updated weights for policy 0, policy_version 1253721 (0.0009) [2023-12-27 00:30:58,833][105692] Updated weights for policy 0, policy_version 1253731 (0.0009) [2023-12-27 00:30:58,903][105692] Updated weights for policy 0, policy_version 1253741 (0.0008) [2023-12-27 00:30:59,258][105620] Updated weights for policy 1, policy_version 1254985 (0.0008) [2023-12-27 00:30:59,328][105620] Updated weights for policy 1, policy_version 1254995 (0.0008) [2023-12-27 00:30:59,397][105620] Updated weights for policy 1, policy_version 1255005 (0.0008) [2023-12-27 00:30:59,710][105692] Updated weights for policy 0, policy_version 1253751 (0.0009) [2023-12-27 00:30:59,764][105692] Updated weights for policy 0, policy_version 1253761 (0.0009) [2023-12-27 00:30:59,820][105692] Updated weights for policy 0, policy_version 1253771 (0.0008) [2023-12-27 00:31:00,196][105620] Updated weights for policy 1, policy_version 1255015 (0.0008) [2023-12-27 00:31:00,255][105620] Updated weights for policy 1, policy_version 1255025 (0.0009) [2023-12-27 00:31:00,314][105620] Updated weights for policy 1, policy_version 1255035 (0.0009) [2023-12-27 00:31:00,622][105692] Updated weights for policy 0, policy_version 1253782 (0.0008) [2023-12-27 00:31:00,681][105692] Updated weights for policy 0, policy_version 1253792 (0.0005) [2023-12-27 00:31:00,732][105692] Updated weights for policy 0, policy_version 1253802 (0.0005) [2023-12-27 00:31:00,903][105620] Updated weights for policy 1, policy_version 1255045 (0.0007) [2023-12-27 00:31:00,956][105620] Updated weights for policy 1, policy_version 1255055 (0.0008) [2023-12-27 00:31:01,022][105620] Updated weights for policy 1, policy_version 1255065 (0.0006) [2023-12-27 00:31:01,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 642359296. Throughput: 0: 9777.1, 1: 9790.2. Samples: 642334628. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:31:01,062][104569] Avg episode reward: [(0, '9169.402'), (1, '9078.835')] [2023-12-27 00:31:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001253808_321028096.pth... [2023-12-27 00:31:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001255072_321339392.pth... [2023-12-27 00:31:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001252656_320733184.pth [2023-12-27 00:31:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001253920_321044480.pth [2023-12-27 00:31:01,384][105692] Updated weights for policy 0, policy_version 1253812 (0.0006) [2023-12-27 00:31:01,448][105692] Updated weights for policy 0, policy_version 1253822 (0.0007) [2023-12-27 00:31:01,503][105692] Updated weights for policy 0, policy_version 1253832 (0.0007) [2023-12-27 00:31:01,759][105620] Updated weights for policy 1, policy_version 1255075 (0.0008) [2023-12-27 00:31:01,807][105620] Updated weights for policy 1, policy_version 1255085 (0.0010) [2023-12-27 00:31:01,856][105620] Updated weights for policy 1, policy_version 1255095 (0.0009) [2023-12-27 00:31:02,215][105692] Updated weights for policy 0, policy_version 1253842 (0.0007) [2023-12-27 00:31:02,278][105692] Updated weights for policy 0, policy_version 1253852 (0.0008) [2023-12-27 00:31:02,342][105692] Updated weights for policy 0, policy_version 1253862 (0.0008) [2023-12-27 00:31:02,400][105692] Updated weights for policy 0, policy_version 1253872 (0.0008) [2023-12-27 00:31:02,622][105620] Updated weights for policy 1, policy_version 1255105 (0.0010) [2023-12-27 00:31:02,673][105620] Updated weights for policy 1, policy_version 1255115 (0.0010) [2023-12-27 00:31:02,721][105620] Updated weights for policy 1, policy_version 1255125 (0.0010) [2023-12-27 00:31:02,773][105620] Updated weights for policy 1, policy_version 1255135 (0.0010) [2023-12-27 00:31:03,060][105692] Updated weights for policy 0, policy_version 1253882 (0.0008) [2023-12-27 00:31:03,117][105692] Updated weights for policy 0, policy_version 1253892 (0.0008) [2023-12-27 00:31:03,166][105692] Updated weights for policy 0, policy_version 1253902 (0.0008) [2023-12-27 00:31:03,466][105620] Updated weights for policy 1, policy_version 1255145 (0.0006) [2023-12-27 00:31:03,527][105620] Updated weights for policy 1, policy_version 1255155 (0.0008) [2023-12-27 00:31:03,585][105620] Updated weights for policy 1, policy_version 1255165 (0.0010) [2023-12-27 00:31:03,963][105692] Updated weights for policy 0, policy_version 1253912 (0.0009) [2023-12-27 00:31:04,018][105692] Updated weights for policy 0, policy_version 1253922 (0.0010) [2023-12-27 00:31:04,076][105692] Updated weights for policy 0, policy_version 1253932 (0.0008) [2023-12-27 00:31:04,254][105620] Updated weights for policy 1, policy_version 1255175 (0.0008) [2023-12-27 00:31:04,320][105620] Updated weights for policy 1, policy_version 1255185 (0.0008) [2023-12-27 00:31:04,391][105620] Updated weights for policy 1, policy_version 1255195 (0.0010) [2023-12-27 00:31:04,932][105692] Updated weights for policy 0, policy_version 1253942 (0.0009) [2023-12-27 00:31:04,993][105692] Updated weights for policy 0, policy_version 1253952 (0.0009) [2023-12-27 00:31:05,048][105692] Updated weights for policy 0, policy_version 1253963 (0.0010) [2023-12-27 00:31:05,100][105620] Updated weights for policy 1, policy_version 1255205 (0.0010) [2023-12-27 00:31:05,160][105620] Updated weights for policy 1, policy_version 1255215 (0.0006) [2023-12-27 00:31:05,212][105620] Updated weights for policy 1, policy_version 1255225 (0.0009) [2023-12-27 00:31:05,769][105692] Updated weights for policy 0, policy_version 1253973 (0.0009) [2023-12-27 00:31:05,826][105692] Updated weights for policy 0, policy_version 1253983 (0.0009) [2023-12-27 00:31:05,889][105692] Updated weights for policy 0, policy_version 1253993 (0.0009) [2023-12-27 00:31:05,990][105620] Updated weights for policy 1, policy_version 1255235 (0.0010) [2023-12-27 00:31:06,041][105620] Updated weights for policy 1, policy_version 1255245 (0.0010) [2023-12-27 00:31:06,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 642457600. Throughput: 0: 9738.7, 1: 9767.0. Samples: 642448356. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:31:06,062][104569] Avg episode reward: [(0, '8991.550'), (1, '9079.327')] [2023-12-27 00:31:06,090][105620] Updated weights for policy 1, policy_version 1255255 (0.0009) [2023-12-27 00:31:06,603][105692] Updated weights for policy 0, policy_version 1254003 (0.0007) [2023-12-27 00:31:06,667][105692] Updated weights for policy 0, policy_version 1254013 (0.0008) [2023-12-27 00:31:06,724][105692] Updated weights for policy 0, policy_version 1254023 (0.0011) [2023-12-27 00:31:06,931][105620] Updated weights for policy 1, policy_version 1255265 (0.0009) [2023-12-27 00:31:06,986][105620] Updated weights for policy 1, policy_version 1255275 (0.0008) [2023-12-27 00:31:07,051][105620] Updated weights for policy 1, policy_version 1255285 (0.0010) [2023-12-27 00:31:07,118][105620] Updated weights for policy 1, policy_version 1255295 (0.0011) [2023-12-27 00:31:07,508][105692] Updated weights for policy 0, policy_version 1254033 (0.0010) [2023-12-27 00:31:07,566][105692] Updated weights for policy 0, policy_version 1254043 (0.0011) [2023-12-27 00:31:07,628][105692] Updated weights for policy 0, policy_version 1254053 (0.0010) [2023-12-27 00:31:07,681][105692] Updated weights for policy 0, policy_version 1254063 (0.0010) [2023-12-27 00:31:07,849][105620] Updated weights for policy 1, policy_version 1255305 (0.0011) [2023-12-27 00:31:07,916][105620] Updated weights for policy 1, policy_version 1255315 (0.0011) [2023-12-27 00:31:07,986][105620] Updated weights for policy 1, policy_version 1255325 (0.0011) [2023-12-27 00:31:08,443][105692] Updated weights for policy 0, policy_version 1254073 (0.0010) [2023-12-27 00:31:08,512][105692] Updated weights for policy 0, policy_version 1254083 (0.0011) [2023-12-27 00:31:08,570][105692] Updated weights for policy 0, policy_version 1254093 (0.0010) [2023-12-27 00:31:08,736][105620] Updated weights for policy 1, policy_version 1255335 (0.0011) [2023-12-27 00:31:08,799][105620] Updated weights for policy 1, policy_version 1255345 (0.0011) [2023-12-27 00:31:08,855][105620] Updated weights for policy 1, policy_version 1255355 (0.0011) [2023-12-27 00:31:09,314][105692] Updated weights for policy 0, policy_version 1254103 (0.0011) [2023-12-27 00:31:09,382][105692] Updated weights for policy 0, policy_version 1254113 (0.0012) [2023-12-27 00:31:09,448][105692] Updated weights for policy 0, policy_version 1254123 (0.0008) [2023-12-27 00:31:09,634][105620] Updated weights for policy 1, policy_version 1255365 (0.0008) [2023-12-27 00:31:09,689][105620] Updated weights for policy 1, policy_version 1255375 (0.0006) [2023-12-27 00:31:09,750][105620] Updated weights for policy 1, policy_version 1255385 (0.0005) [2023-12-27 00:31:10,229][105692] Updated weights for policy 0, policy_version 1254133 (0.0008) [2023-12-27 00:31:10,288][105692] Updated weights for policy 0, policy_version 1254143 (0.0008) [2023-12-27 00:31:10,349][105692] Updated weights for policy 0, policy_version 1254153 (0.0008) [2023-12-27 00:31:10,464][105620] Updated weights for policy 1, policy_version 1255395 (0.0009) [2023-12-27 00:31:10,514][105620] Updated weights for policy 1, policy_version 1255405 (0.0011) [2023-12-27 00:31:10,577][105620] Updated weights for policy 1, policy_version 1255415 (0.0011) [2023-12-27 00:31:11,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 642547712. Throughput: 0: 9693.3, 1: 9686.9. Samples: 642558260. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:31:11,063][104569] Avg episode reward: [(0, '9084.309'), (1, '9171.125')] [2023-12-27 00:31:11,125][105692] Updated weights for policy 0, policy_version 1254163 (0.0009) [2023-12-27 00:31:11,194][105692] Updated weights for policy 0, policy_version 1254173 (0.0008) [2023-12-27 00:31:11,253][105692] Updated weights for policy 0, policy_version 1254183 (0.0008) [2023-12-27 00:31:11,423][105620] Updated weights for policy 1, policy_version 1255425 (0.0010) [2023-12-27 00:31:11,489][105620] Updated weights for policy 1, policy_version 1255435 (0.0008) [2023-12-27 00:31:11,552][105620] Updated weights for policy 1, policy_version 1255445 (0.0006) [2023-12-27 00:31:11,623][105620] Updated weights for policy 1, policy_version 1255455 (0.0007) [2023-12-27 00:31:12,013][105692] Updated weights for policy 0, policy_version 1254193 (0.0008) [2023-12-27 00:31:12,068][105692] Updated weights for policy 0, policy_version 1254203 (0.0008) [2023-12-27 00:31:12,124][105692] Updated weights for policy 0, policy_version 1254213 (0.0006) [2023-12-27 00:31:12,177][105692] Updated weights for policy 0, policy_version 1254223 (0.0008) [2023-12-27 00:31:12,300][105620] Updated weights for policy 1, policy_version 1255465 (0.0009) [2023-12-27 00:31:12,372][105620] Updated weights for policy 1, policy_version 1255475 (0.0008) [2023-12-27 00:31:12,434][105620] Updated weights for policy 1, policy_version 1255485 (0.0009) [2023-12-27 00:31:12,865][105692] Updated weights for policy 0, policy_version 1254233 (0.0010) [2023-12-27 00:31:12,916][105692] Updated weights for policy 0, policy_version 1254243 (0.0008) [2023-12-27 00:31:12,963][105692] Updated weights for policy 0, policy_version 1254253 (0.0005) [2023-12-27 00:31:13,254][105620] Updated weights for policy 1, policy_version 1255495 (0.0009) [2023-12-27 00:31:13,313][105620] Updated weights for policy 1, policy_version 1255505 (0.0009) [2023-12-27 00:31:13,373][105620] Updated weights for policy 1, policy_version 1255515 (0.0009) [2023-12-27 00:31:13,645][105692] Updated weights for policy 0, policy_version 1254263 (0.0007) [2023-12-27 00:31:13,693][105692] Updated weights for policy 0, policy_version 1254273 (0.0010) [2023-12-27 00:31:13,740][105692] Updated weights for policy 0, policy_version 1254283 (0.0010) [2023-12-27 00:31:14,120][105620] Updated weights for policy 1, policy_version 1255525 (0.0007) [2023-12-27 00:31:14,184][105620] Updated weights for policy 1, policy_version 1255535 (0.0006) [2023-12-27 00:31:14,254][105620] Updated weights for policy 1, policy_version 1255545 (0.0005) [2023-12-27 00:31:14,317][105692] Updated weights for policy 0, policy_version 1254293 (0.0009) [2023-12-27 00:31:14,373][105692] Updated weights for policy 0, policy_version 1254303 (0.0005) [2023-12-27 00:31:14,430][105692] Updated weights for policy 0, policy_version 1254313 (0.0005) [2023-12-27 00:31:14,989][105692] Updated weights for policy 0, policy_version 1254323 (0.0007) [2023-12-27 00:31:15,038][105620] Updated weights for policy 1, policy_version 1255555 (0.0006) [2023-12-27 00:31:15,049][105692] Updated weights for policy 0, policy_version 1254333 (0.0010) [2023-12-27 00:31:15,099][105620] Updated weights for policy 1, policy_version 1255565 (0.0007) [2023-12-27 00:31:15,101][105692] Updated weights for policy 0, policy_version 1254343 (0.0010) [2023-12-27 00:31:15,160][105620] Updated weights for policy 1, policy_version 1255575 (0.0006) [2023-12-27 00:31:15,713][105692] Updated weights for policy 0, policy_version 1254353 (0.0010) [2023-12-27 00:31:15,764][105692] Updated weights for policy 0, policy_version 1254363 (0.0005) [2023-12-27 00:31:15,817][105692] Updated weights for policy 0, policy_version 1254373 (0.0008) [2023-12-27 00:31:15,872][105692] Updated weights for policy 0, policy_version 1254383 (0.0010) [2023-12-27 00:31:15,882][105620] Updated weights for policy 1, policy_version 1255585 (0.0006) [2023-12-27 00:31:15,938][105620] Updated weights for policy 1, policy_version 1255595 (0.0008) [2023-12-27 00:31:16,000][105620] Updated weights for policy 1, policy_version 1255605 (0.0007) [2023-12-27 00:31:16,058][105620] Updated weights for policy 1, policy_version 1255615 (0.0010) [2023-12-27 00:31:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 642654208. Throughput: 0: 9715.8, 1: 9629.9. Samples: 642614976. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:31:16,062][104569] Avg episode reward: [(0, '9353.411'), (1, '9079.757')] [2023-12-27 00:31:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001254384_321175552.pth... [2023-12-27 00:31:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001255616_321478656.pth... [2023-12-27 00:31:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001253264_320888832.pth [2023-12-27 00:31:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001254496_321191936.pth [2023-12-27 00:31:16,501][105692] Updated weights for policy 0, policy_version 1254393 (0.0010) [2023-12-27 00:31:16,555][105692] Updated weights for policy 0, policy_version 1254403 (0.0007) [2023-12-27 00:31:16,601][105692] Updated weights for policy 0, policy_version 1254413 (0.0005) [2023-12-27 00:31:16,780][105620] Updated weights for policy 1, policy_version 1255625 (0.0009) [2023-12-27 00:31:16,842][105620] Updated weights for policy 1, policy_version 1255635 (0.0010) [2023-12-27 00:31:16,900][105620] Updated weights for policy 1, policy_version 1255645 (0.0010) [2023-12-27 00:31:17,139][105692] Updated weights for policy 0, policy_version 1254423 (0.0006) [2023-12-27 00:31:17,197][105692] Updated weights for policy 0, policy_version 1254433 (0.0010) [2023-12-27 00:31:17,256][105692] Updated weights for policy 0, policy_version 1254443 (0.0011) [2023-12-27 00:31:17,735][105620] Updated weights for policy 1, policy_version 1255655 (0.0009) [2023-12-27 00:31:17,793][105620] Updated weights for policy 1, policy_version 1255665 (0.0008) [2023-12-27 00:31:17,851][105620] Updated weights for policy 1, policy_version 1255675 (0.0007) [2023-12-27 00:31:17,983][105692] Updated weights for policy 0, policy_version 1254453 (0.0010) [2023-12-27 00:31:18,039][105692] Updated weights for policy 0, policy_version 1254463 (0.0006) [2023-12-27 00:31:18,093][105692] Updated weights for policy 0, policy_version 1254473 (0.0005) [2023-12-27 00:31:18,670][105620] Updated weights for policy 1, policy_version 1255685 (0.0007) [2023-12-27 00:31:18,703][105692] Updated weights for policy 0, policy_version 1254483 (0.0007) [2023-12-27 00:31:18,724][105620] Updated weights for policy 1, policy_version 1255695 (0.0006) [2023-12-27 00:31:18,759][105692] Updated weights for policy 0, policy_version 1254493 (0.0010) [2023-12-27 00:31:18,776][105620] Updated weights for policy 1, policy_version 1255705 (0.0005) [2023-12-27 00:31:18,818][105692] Updated weights for policy 0, policy_version 1254503 (0.0011) [2023-12-27 00:31:19,448][105620] Updated weights for policy 1, policy_version 1255715 (0.0006) [2023-12-27 00:31:19,508][105620] Updated weights for policy 1, policy_version 1255725 (0.0008) [2023-12-27 00:31:19,569][105620] Updated weights for policy 1, policy_version 1255735 (0.0008) [2023-12-27 00:31:19,580][105692] Updated weights for policy 0, policy_version 1254513 (0.0011) [2023-12-27 00:31:19,642][105692] Updated weights for policy 0, policy_version 1254523 (0.0011) [2023-12-27 00:31:19,701][105692] Updated weights for policy 0, policy_version 1254533 (0.0010) [2023-12-27 00:31:19,766][105692] Updated weights for policy 0, policy_version 1254543 (0.0011) [2023-12-27 00:31:20,319][105620] Updated weights for policy 1, policy_version 1255745 (0.0007) [2023-12-27 00:31:20,383][105620] Updated weights for policy 1, policy_version 1255755 (0.0008) [2023-12-27 00:31:20,446][105620] Updated weights for policy 1, policy_version 1255765 (0.0007) [2023-12-27 00:31:20,503][105620] Updated weights for policy 1, policy_version 1255775 (0.0006) [2023-12-27 00:31:20,518][105692] Updated weights for policy 0, policy_version 1254553 (0.0011) [2023-12-27 00:31:20,583][105692] Updated weights for policy 0, policy_version 1254563 (0.0010) [2023-12-27 00:31:20,648][105692] Updated weights for policy 0, policy_version 1254573 (0.0008) [2023-12-27 00:31:21,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 642744320. Throughput: 0: 9781.1, 1: 9558.2. Samples: 642735768. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:31:21,062][104569] Avg episode reward: [(0, '9350.989'), (1, '8894.442')] [2023-12-27 00:31:21,223][105620] Updated weights for policy 1, policy_version 1255785 (0.0008) [2023-12-27 00:31:21,288][105620] Updated weights for policy 1, policy_version 1255795 (0.0008) [2023-12-27 00:31:21,355][105620] Updated weights for policy 1, policy_version 1255805 (0.0009) [2023-12-27 00:31:21,391][105692] Updated weights for policy 0, policy_version 1254583 (0.0008) [2023-12-27 00:31:21,454][105692] Updated weights for policy 0, policy_version 1254593 (0.0006) [2023-12-27 00:31:21,506][105692] Updated weights for policy 0, policy_version 1254603 (0.0005) [2023-12-27 00:31:22,123][105620] Updated weights for policy 1, policy_version 1255815 (0.0009) [2023-12-27 00:31:22,187][105620] Updated weights for policy 1, policy_version 1255825 (0.0008) [2023-12-27 00:31:22,249][105620] Updated weights for policy 1, policy_version 1255835 (0.0005) [2023-12-27 00:31:22,272][105692] Updated weights for policy 0, policy_version 1254613 (0.0007) [2023-12-27 00:31:22,334][105692] Updated weights for policy 0, policy_version 1254623 (0.0009) [2023-12-27 00:31:22,404][105692] Updated weights for policy 0, policy_version 1254633 (0.0009) [2023-12-27 00:31:22,980][105620] Updated weights for policy 1, policy_version 1255845 (0.0007) [2023-12-27 00:31:23,026][105620] Updated weights for policy 1, policy_version 1255855 (0.0008) [2023-12-27 00:31:23,074][105620] Updated weights for policy 1, policy_version 1255865 (0.0009) [2023-12-27 00:31:23,148][105692] Updated weights for policy 0, policy_version 1254643 (0.0009) [2023-12-27 00:31:23,200][105692] Updated weights for policy 0, policy_version 1254653 (0.0009) [2023-12-27 00:31:23,248][105692] Updated weights for policy 0, policy_version 1254663 (0.0009) [2023-12-27 00:31:23,839][105620] Updated weights for policy 1, policy_version 1255875 (0.0009) [2023-12-27 00:31:23,894][105620] Updated weights for policy 1, policy_version 1255885 (0.0009) [2023-12-27 00:31:23,954][105620] Updated weights for policy 1, policy_version 1255896 (0.0010) [2023-12-27 00:31:23,974][105692] Updated weights for policy 0, policy_version 1254673 (0.0008) [2023-12-27 00:31:24,032][105692] Updated weights for policy 0, policy_version 1254683 (0.0008) [2023-12-27 00:31:24,097][105692] Updated weights for policy 0, policy_version 1254693 (0.0009) [2023-12-27 00:31:24,148][105692] Updated weights for policy 0, policy_version 1254703 (0.0009) [2023-12-27 00:31:24,738][105620] Updated weights for policy 1, policy_version 1255906 (0.0010) [2023-12-27 00:31:24,788][105620] Updated weights for policy 1, policy_version 1255916 (0.0009) [2023-12-27 00:31:24,833][105620] Updated weights for policy 1, policy_version 1255926 (0.0008) [2023-12-27 00:31:24,877][105692] Updated weights for policy 0, policy_version 1254713 (0.0008) [2023-12-27 00:31:24,888][105620] Updated weights for policy 1, policy_version 1255936 (0.0006) [2023-12-27 00:31:24,937][105692] Updated weights for policy 0, policy_version 1254723 (0.0008) [2023-12-27 00:31:24,998][105692] Updated weights for policy 0, policy_version 1254733 (0.0009) [2023-12-27 00:31:25,615][105620] Updated weights for policy 1, policy_version 1255946 (0.0009) [2023-12-27 00:31:25,670][105620] Updated weights for policy 1, policy_version 1255956 (0.0009) [2023-12-27 00:31:25,728][105620] Updated weights for policy 1, policy_version 1255966 (0.0007) [2023-12-27 00:31:25,738][105692] Updated weights for policy 0, policy_version 1254743 (0.0008) [2023-12-27 00:31:25,784][105692] Updated weights for policy 0, policy_version 1254753 (0.0008) [2023-12-27 00:31:25,832][105692] Updated weights for policy 0, policy_version 1254763 (0.0009) [2023-12-27 00:31:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 642842624. Throughput: 0: 9705.5, 1: 9507.0. Samples: 642847756. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:31:26,062][104569] Avg episode reward: [(0, '9259.924'), (1, '9077.552')] [2023-12-27 00:31:26,400][105620] Updated weights for policy 1, policy_version 1255976 (0.0008) [2023-12-27 00:31:26,451][105620] Updated weights for policy 1, policy_version 1255986 (0.0009) [2023-12-27 00:31:26,506][105620] Updated weights for policy 1, policy_version 1255996 (0.0009) [2023-12-27 00:31:26,653][105692] Updated weights for policy 0, policy_version 1254773 (0.0009) [2023-12-27 00:31:26,715][105692] Updated weights for policy 0, policy_version 1254783 (0.0009) [2023-12-27 00:31:26,772][105692] Updated weights for policy 0, policy_version 1254793 (0.0009) [2023-12-27 00:31:27,274][105620] Updated weights for policy 1, policy_version 1256006 (0.0009) [2023-12-27 00:31:27,341][105620] Updated weights for policy 1, policy_version 1256016 (0.0006) [2023-12-27 00:31:27,390][105620] Updated weights for policy 1, policy_version 1256026 (0.0007) [2023-12-27 00:31:27,538][105692] Updated weights for policy 0, policy_version 1254803 (0.0009) [2023-12-27 00:31:27,592][105692] Updated weights for policy 0, policy_version 1254813 (0.0009) [2023-12-27 00:31:27,647][105692] Updated weights for policy 0, policy_version 1254823 (0.0010) [2023-12-27 00:31:27,970][105620] Updated weights for policy 1, policy_version 1256036 (0.0008) [2023-12-27 00:31:28,024][105620] Updated weights for policy 1, policy_version 1256046 (0.0009) [2023-12-27 00:31:28,078][105620] Updated weights for policy 1, policy_version 1256056 (0.0010) [2023-12-27 00:31:28,351][105692] Updated weights for policy 0, policy_version 1254833 (0.0010) [2023-12-27 00:31:28,425][105692] Updated weights for policy 0, policy_version 1254843 (0.0010) [2023-12-27 00:31:28,494][105692] Updated weights for policy 0, policy_version 1254853 (0.0009) [2023-12-27 00:31:28,550][105692] Updated weights for policy 0, policy_version 1254863 (0.0006) [2023-12-27 00:31:28,816][105620] Updated weights for policy 1, policy_version 1256067 (0.0009) [2023-12-27 00:31:28,869][105620] Updated weights for policy 1, policy_version 1256077 (0.0006) [2023-12-27 00:31:28,928][105620] Updated weights for policy 1, policy_version 1256087 (0.0010) [2023-12-27 00:31:29,240][105692] Updated weights for policy 0, policy_version 1254873 (0.0006) [2023-12-27 00:31:29,301][105692] Updated weights for policy 0, policy_version 1254883 (0.0008) [2023-12-27 00:31:29,364][105692] Updated weights for policy 0, policy_version 1254893 (0.0008) [2023-12-27 00:31:29,509][105620] Updated weights for policy 1, policy_version 1256097 (0.0006) [2023-12-27 00:31:29,567][105620] Updated weights for policy 1, policy_version 1256107 (0.0006) [2023-12-27 00:31:29,628][105620] Updated weights for policy 1, policy_version 1256117 (0.0007) [2023-12-27 00:31:29,686][105620] Updated weights for policy 1, policy_version 1256127 (0.0007) [2023-12-27 00:31:30,118][105692] Updated weights for policy 0, policy_version 1254903 (0.0009) [2023-12-27 00:31:30,175][105692] Updated weights for policy 0, policy_version 1254913 (0.0007) [2023-12-27 00:31:30,232][105692] Updated weights for policy 0, policy_version 1254923 (0.0005) [2023-12-27 00:31:30,291][105620] Updated weights for policy 1, policy_version 1256137 (0.0008) [2023-12-27 00:31:30,354][105620] Updated weights for policy 1, policy_version 1256147 (0.0009) [2023-12-27 00:31:30,412][105620] Updated weights for policy 1, policy_version 1256157 (0.0009) [2023-12-27 00:31:30,958][105692] Updated weights for policy 0, policy_version 1254933 (0.0007) [2023-12-27 00:31:31,016][105692] Updated weights for policy 0, policy_version 1254943 (0.0009) [2023-12-27 00:31:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 642932736. Throughput: 0: 9708.0, 1: 9550.9. Samples: 642906332. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:31:31,062][104569] Avg episode reward: [(0, '9261.249'), (1, '9169.314')] [2023-12-27 00:31:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001256160_321617920.pth... [2023-12-27 00:31:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001255072_321339392.pth [2023-12-27 00:31:31,080][105692] Updated weights for policy 0, policy_version 1254953 (0.0010) [2023-12-27 00:31:31,123][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001254960_321323008.pth... [2023-12-27 00:31:31,128][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001253808_321028096.pth [2023-12-27 00:31:31,141][105620] Updated weights for policy 1, policy_version 1256167 (0.0007) [2023-12-27 00:31:31,202][105620] Updated weights for policy 1, policy_version 1256177 (0.0009) [2023-12-27 00:31:31,269][105620] Updated weights for policy 1, policy_version 1256187 (0.0009) [2023-12-27 00:31:31,755][105692] Updated weights for policy 0, policy_version 1254963 (0.0008) [2023-12-27 00:31:31,816][105692] Updated weights for policy 0, policy_version 1254973 (0.0008) [2023-12-27 00:31:31,874][105692] Updated weights for policy 0, policy_version 1254983 (0.0010) [2023-12-27 00:31:32,072][105620] Updated weights for policy 1, policy_version 1256197 (0.0007) [2023-12-27 00:31:32,139][105620] Updated weights for policy 1, policy_version 1256207 (0.0006) [2023-12-27 00:31:32,203][105620] Updated weights for policy 1, policy_version 1256217 (0.0008) [2023-12-27 00:31:32,514][105692] Updated weights for policy 0, policy_version 1254993 (0.0010) [2023-12-27 00:31:32,573][105692] Updated weights for policy 0, policy_version 1255003 (0.0010) [2023-12-27 00:31:32,621][105692] Updated weights for policy 0, policy_version 1255013 (0.0010) [2023-12-27 00:31:32,676][105692] Updated weights for policy 0, policy_version 1255023 (0.0010) [2023-12-27 00:31:32,881][105620] Updated weights for policy 1, policy_version 1256227 (0.0008) [2023-12-27 00:31:32,947][105620] Updated weights for policy 1, policy_version 1256237 (0.0008) [2023-12-27 00:31:33,002][105620] Updated weights for policy 1, policy_version 1256247 (0.0008) [2023-12-27 00:31:33,331][105692] Updated weights for policy 0, policy_version 1255033 (0.0008) [2023-12-27 00:31:33,389][105692] Updated weights for policy 0, policy_version 1255043 (0.0010) [2023-12-27 00:31:33,440][105692] Updated weights for policy 0, policy_version 1255053 (0.0010) [2023-12-27 00:31:33,684][105620] Updated weights for policy 1, policy_version 1256257 (0.0010) [2023-12-27 00:31:33,745][105620] Updated weights for policy 1, policy_version 1256267 (0.0010) [2023-12-27 00:31:33,807][105620] Updated weights for policy 1, policy_version 1256277 (0.0010) [2023-12-27 00:31:33,866][105620] Updated weights for policy 1, policy_version 1256287 (0.0010) [2023-12-27 00:31:34,040][105692] Updated weights for policy 0, policy_version 1255063 (0.0007) [2023-12-27 00:31:34,099][105692] Updated weights for policy 0, policy_version 1255073 (0.0005) [2023-12-27 00:31:34,172][105692] Updated weights for policy 0, policy_version 1255083 (0.0009) [2023-12-27 00:31:34,688][105620] Updated weights for policy 1, policy_version 1256297 (0.0008) [2023-12-27 00:31:34,748][105620] Updated weights for policy 1, policy_version 1256307 (0.0008) [2023-12-27 00:31:34,807][105620] Updated weights for policy 1, policy_version 1256317 (0.0008) [2023-12-27 00:31:34,898][105692] Updated weights for policy 0, policy_version 1255093 (0.0010) [2023-12-27 00:31:34,949][105692] Updated weights for policy 0, policy_version 1255103 (0.0010) [2023-12-27 00:31:35,007][105692] Updated weights for policy 0, policy_version 1255113 (0.0010) [2023-12-27 00:31:35,523][105620] Updated weights for policy 1, policy_version 1256327 (0.0007) [2023-12-27 00:31:35,581][105620] Updated weights for policy 1, policy_version 1256337 (0.0008) [2023-12-27 00:31:35,639][105620] Updated weights for policy 1, policy_version 1256347 (0.0009) [2023-12-27 00:31:35,739][105692] Updated weights for policy 0, policy_version 1255123 (0.0009) [2023-12-27 00:31:35,783][105692] Updated weights for policy 0, policy_version 1255133 (0.0010) [2023-12-27 00:31:35,834][105692] Updated weights for policy 0, policy_version 1255143 (0.0010) [2023-12-27 00:31:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 643039232. Throughput: 0: 9761.7, 1: 9501.8. Samples: 643025604. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:31:36,063][104569] Avg episode reward: [(0, '9351.576'), (1, '9352.637')] [2023-12-27 00:31:36,275][105620] Updated weights for policy 1, policy_version 1256357 (0.0008) [2023-12-27 00:31:36,332][105620] Updated weights for policy 1, policy_version 1256367 (0.0010) [2023-12-27 00:31:36,384][105620] Updated weights for policy 1, policy_version 1256377 (0.0009) [2023-12-27 00:31:36,547][105692] Updated weights for policy 0, policy_version 1255153 (0.0009) [2023-12-27 00:31:36,604][105692] Updated weights for policy 0, policy_version 1255163 (0.0009) [2023-12-27 00:31:36,664][105692] Updated weights for policy 0, policy_version 1255173 (0.0006) [2023-12-27 00:31:36,728][105692] Updated weights for policy 0, policy_version 1255183 (0.0006) [2023-12-27 00:31:37,261][105620] Updated weights for policy 1, policy_version 1256387 (0.0008) [2023-12-27 00:31:37,294][105692] Updated weights for policy 0, policy_version 1255193 (0.0006) [2023-12-27 00:31:37,318][105620] Updated weights for policy 1, policy_version 1256397 (0.0007) [2023-12-27 00:31:37,342][105692] Updated weights for policy 0, policy_version 1255203 (0.0008) [2023-12-27 00:31:37,364][105620] Updated weights for policy 1, policy_version 1256407 (0.0008) [2023-12-27 00:31:37,395][105692] Updated weights for policy 0, policy_version 1255213 (0.0006) [2023-12-27 00:31:38,052][105620] Updated weights for policy 1, policy_version 1256417 (0.0008) [2023-12-27 00:31:38,075][105692] Updated weights for policy 0, policy_version 1255223 (0.0009) [2023-12-27 00:31:38,112][105620] Updated weights for policy 1, policy_version 1256427 (0.0006) [2023-12-27 00:31:38,127][105692] Updated weights for policy 0, policy_version 1255233 (0.0009) [2023-12-27 00:31:38,168][105620] Updated weights for policy 1, policy_version 1256437 (0.0005) [2023-12-27 00:31:38,186][105692] Updated weights for policy 0, policy_version 1255243 (0.0009) [2023-12-27 00:31:38,218][105620] Updated weights for policy 1, policy_version 1256447 (0.0007) [2023-12-27 00:31:38,886][105692] Updated weights for policy 0, policy_version 1255253 (0.0008) [2023-12-27 00:31:38,941][105692] Updated weights for policy 0, policy_version 1255263 (0.0009) [2023-12-27 00:31:38,943][105620] Updated weights for policy 1, policy_version 1256457 (0.0006) [2023-12-27 00:31:38,993][105692] Updated weights for policy 0, policy_version 1255273 (0.0005) [2023-12-27 00:31:38,995][105620] Updated weights for policy 1, policy_version 1256467 (0.0007) [2023-12-27 00:31:39,059][105620] Updated weights for policy 1, policy_version 1256477 (0.0008) [2023-12-27 00:31:39,714][105692] Updated weights for policy 0, policy_version 1255283 (0.0008) [2023-12-27 00:31:39,765][105692] Updated weights for policy 0, policy_version 1255293 (0.0009) [2023-12-27 00:31:39,831][105692] Updated weights for policy 0, policy_version 1255303 (0.0009) [2023-12-27 00:31:39,870][105620] Updated weights for policy 1, policy_version 1256487 (0.0007) [2023-12-27 00:31:39,937][105620] Updated weights for policy 1, policy_version 1256497 (0.0008) [2023-12-27 00:31:40,005][105620] Updated weights for policy 1, policy_version 1256507 (0.0009) [2023-12-27 00:31:40,662][105692] Updated weights for policy 0, policy_version 1255313 (0.0008) [2023-12-27 00:31:40,718][105620] Updated weights for policy 1, policy_version 1256517 (0.0009) [2023-12-27 00:31:40,720][105692] Updated weights for policy 0, policy_version 1255323 (0.0008) [2023-12-27 00:31:40,770][105692] Updated weights for policy 0, policy_version 1255333 (0.0007) [2023-12-27 00:31:40,777][105620] Updated weights for policy 1, policy_version 1256527 (0.0009) [2023-12-27 00:31:40,829][105692] Updated weights for policy 0, policy_version 1255343 (0.0006) [2023-12-27 00:31:40,830][105620] Updated weights for policy 1, policy_version 1256537 (0.0010) [2023-12-27 00:31:41,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 643137536. Throughput: 0: 9793.2, 1: 9523.6. Samples: 643142756. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:31:41,063][104569] Avg episode reward: [(0, '9260.114'), (1, '9352.529')] [2023-12-27 00:31:41,580][105620] Updated weights for policy 1, policy_version 1256547 (0.0009) [2023-12-27 00:31:41,636][105692] Updated weights for policy 0, policy_version 1255353 (0.0009) [2023-12-27 00:31:41,649][105620] Updated weights for policy 1, policy_version 1256557 (0.0009) [2023-12-27 00:31:41,699][105692] Updated weights for policy 0, policy_version 1255363 (0.0009) [2023-12-27 00:31:41,719][105620] Updated weights for policy 1, policy_version 1256567 (0.0008) [2023-12-27 00:31:41,766][105692] Updated weights for policy 0, policy_version 1255373 (0.0006) [2023-12-27 00:31:42,498][105620] Updated weights for policy 1, policy_version 1256577 (0.0007) [2023-12-27 00:31:42,530][105692] Updated weights for policy 0, policy_version 1255383 (0.0008) [2023-12-27 00:31:42,559][105620] Updated weights for policy 1, policy_version 1256587 (0.0005) [2023-12-27 00:31:42,588][105692] Updated weights for policy 0, policy_version 1255393 (0.0009) [2023-12-27 00:31:42,617][105620] Updated weights for policy 1, policy_version 1256597 (0.0006) [2023-12-27 00:31:42,620][105585] KL-divergence is very high: 115.1700 [2023-12-27 00:31:42,638][105692] Updated weights for policy 0, policy_version 1255403 (0.0008) [2023-12-27 00:31:42,678][105620] Updated weights for policy 1, policy_version 1256607 (0.0006) [2023-12-27 00:31:43,297][105692] Updated weights for policy 0, policy_version 1255413 (0.0008) [2023-12-27 00:31:43,351][105692] Updated weights for policy 0, policy_version 1255424 (0.0008) [2023-12-27 00:31:43,366][105620] Updated weights for policy 1, policy_version 1256617 (0.0005) [2023-12-27 00:31:43,399][105692] Updated weights for policy 0, policy_version 1255434 (0.0009) [2023-12-27 00:31:43,422][105620] Updated weights for policy 1, policy_version 1256627 (0.0005) [2023-12-27 00:31:43,483][105620] Updated weights for policy 1, policy_version 1256637 (0.0008) [2023-12-27 00:31:44,038][105692] Updated weights for policy 0, policy_version 1255444 (0.0008) [2023-12-27 00:31:44,086][105692] Updated weights for policy 0, policy_version 1255454 (0.0007) [2023-12-27 00:31:44,137][105692] Updated weights for policy 0, policy_version 1255464 (0.0009) [2023-12-27 00:31:44,235][105620] Updated weights for policy 1, policy_version 1256647 (0.0007) [2023-12-27 00:31:44,280][105620] Updated weights for policy 1, policy_version 1256657 (0.0005) [2023-12-27 00:31:44,326][105620] Updated weights for policy 1, policy_version 1256667 (0.0005) [2023-12-27 00:31:44,909][105692] Updated weights for policy 0, policy_version 1255474 (0.0008) [2023-12-27 00:31:44,971][105692] Updated weights for policy 0, policy_version 1255484 (0.0008) [2023-12-27 00:31:45,000][105620] Updated weights for policy 1, policy_version 1256677 (0.0009) [2023-12-27 00:31:45,031][105692] Updated weights for policy 0, policy_version 1255494 (0.0008) [2023-12-27 00:31:45,053][105620] Updated weights for policy 1, policy_version 1256687 (0.0011) [2023-12-27 00:31:45,090][105692] Updated weights for policy 0, policy_version 1255504 (0.0008) [2023-12-27 00:31:45,105][105620] Updated weights for policy 1, policy_version 1256697 (0.0010) [2023-12-27 00:31:45,781][105620] Updated weights for policy 1, policy_version 1256707 (0.0009) [2023-12-27 00:31:45,835][105620] Updated weights for policy 1, policy_version 1256717 (0.0005) [2023-12-27 00:31:45,849][105692] Updated weights for policy 0, policy_version 1255514 (0.0010) [2023-12-27 00:31:45,885][105620] Updated weights for policy 1, policy_version 1256727 (0.0005) [2023-12-27 00:31:45,896][105692] Updated weights for policy 0, policy_version 1255524 (0.0010) [2023-12-27 00:31:45,943][105692] Updated weights for policy 0, policy_version 1255534 (0.0010) [2023-12-27 00:31:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 643235840. Throughput: 0: 9736.1, 1: 9453.9. Samples: 643198184. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:31:46,063][104569] Avg episode reward: [(0, '9077.562'), (1, '9259.816')] [2023-12-27 00:31:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001255536_321470464.pth... [2023-12-27 00:31:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001256736_321765376.pth... [2023-12-27 00:31:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001254384_321175552.pth [2023-12-27 00:31:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001255616_321478656.pth [2023-12-27 00:31:46,458][105620] Updated weights for policy 1, policy_version 1256737 (0.0009) [2023-12-27 00:31:46,513][105620] Updated weights for policy 1, policy_version 1256747 (0.0005) [2023-12-27 00:31:46,571][105620] Updated weights for policy 1, policy_version 1256757 (0.0006) [2023-12-27 00:31:46,626][105620] Updated weights for policy 1, policy_version 1256767 (0.0007) [2023-12-27 00:31:46,641][105692] Updated weights for policy 0, policy_version 1255544 (0.0007) [2023-12-27 00:31:46,698][105692] Updated weights for policy 0, policy_version 1255554 (0.0009) [2023-12-27 00:31:46,765][105692] Updated weights for policy 0, policy_version 1255564 (0.0010) [2023-12-27 00:31:47,260][105620] Updated weights for policy 1, policy_version 1256777 (0.0009) [2023-12-27 00:31:47,320][105620] Updated weights for policy 1, policy_version 1256787 (0.0009) [2023-12-27 00:31:47,378][105620] Updated weights for policy 1, policy_version 1256797 (0.0009) [2023-12-27 00:31:47,500][105692] Updated weights for policy 0, policy_version 1255574 (0.0009) [2023-12-27 00:31:47,565][105692] Updated weights for policy 0, policy_version 1255584 (0.0010) [2023-12-27 00:31:47,629][105692] Updated weights for policy 0, policy_version 1255594 (0.0009) [2023-12-27 00:31:48,035][105620] Updated weights for policy 1, policy_version 1256807 (0.0009) [2023-12-27 00:31:48,089][105620] Updated weights for policy 1, policy_version 1256817 (0.0009) [2023-12-27 00:31:48,150][105620] Updated weights for policy 1, policy_version 1256827 (0.0010) [2023-12-27 00:31:48,366][105692] Updated weights for policy 0, policy_version 1255604 (0.0010) [2023-12-27 00:31:48,429][105692] Updated weights for policy 0, policy_version 1255614 (0.0010) [2023-12-27 00:31:48,494][105692] Updated weights for policy 0, policy_version 1255624 (0.0010) [2023-12-27 00:31:48,973][105620] Updated weights for policy 1, policy_version 1256837 (0.0007) [2023-12-27 00:31:49,029][105620] Updated weights for policy 1, policy_version 1256847 (0.0008) [2023-12-27 00:31:49,077][105620] Updated weights for policy 1, policy_version 1256857 (0.0008) [2023-12-27 00:31:49,180][105692] Updated weights for policy 0, policy_version 1255634 (0.0009) [2023-12-27 00:31:49,238][105692] Updated weights for policy 0, policy_version 1255644 (0.0006) [2023-12-27 00:31:49,296][105692] Updated weights for policy 0, policy_version 1255654 (0.0008) [2023-12-27 00:31:49,362][105692] Updated weights for policy 0, policy_version 1255664 (0.0008) [2023-12-27 00:31:49,838][105620] Updated weights for policy 1, policy_version 1256867 (0.0008) [2023-12-27 00:31:49,897][105620] Updated weights for policy 1, policy_version 1256877 (0.0008) [2023-12-27 00:31:49,964][105620] Updated weights for policy 1, policy_version 1256887 (0.0009) [2023-12-27 00:31:50,060][105692] Updated weights for policy 0, policy_version 1255674 (0.0006) [2023-12-27 00:31:50,116][105692] Updated weights for policy 0, policy_version 1255684 (0.0005) [2023-12-27 00:31:50,178][105692] Updated weights for policy 0, policy_version 1255694 (0.0005) [2023-12-27 00:31:50,779][105620] Updated weights for policy 1, policy_version 1256897 (0.0008) [2023-12-27 00:31:50,784][105692] Updated weights for policy 0, policy_version 1255704 (0.0006) [2023-12-27 00:31:50,839][105620] Updated weights for policy 1, policy_version 1256907 (0.0009) [2023-12-27 00:31:50,848][105692] Updated weights for policy 0, policy_version 1255714 (0.0006) [2023-12-27 00:31:50,901][105620] Updated weights for policy 1, policy_version 1256917 (0.0009) [2023-12-27 00:31:50,905][105692] Updated weights for policy 0, policy_version 1255724 (0.0006) [2023-12-27 00:31:50,960][105620] Updated weights for policy 1, policy_version 1256927 (0.0009) [2023-12-27 00:31:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 643334144. Throughput: 0: 9806.9, 1: 9518.8. Samples: 643318016. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:31:51,063][104569] Avg episode reward: [(0, '8892.659'), (1, '8984.446')] [2023-12-27 00:31:51,627][105692] Updated weights for policy 0, policy_version 1255734 (0.0009) [2023-12-27 00:31:51,682][105692] Updated weights for policy 0, policy_version 1255744 (0.0011) [2023-12-27 00:31:51,714][105620] Updated weights for policy 1, policy_version 1256937 (0.0010) [2023-12-27 00:31:51,739][105692] Updated weights for policy 0, policy_version 1255754 (0.0011) [2023-12-27 00:31:51,807][105620] Updated weights for policy 1, policy_version 1256947 (0.0010) [2023-12-27 00:31:51,872][105620] Updated weights for policy 1, policy_version 1256957 (0.0010) [2023-12-27 00:31:52,544][105692] Updated weights for policy 0, policy_version 1255764 (0.0009) [2023-12-27 00:31:52,551][105620] Updated weights for policy 1, policy_version 1256967 (0.0007) [2023-12-27 00:31:52,595][105692] Updated weights for policy 0, policy_version 1255774 (0.0009) [2023-12-27 00:31:52,617][105620] Updated weights for policy 1, policy_version 1256977 (0.0006) [2023-12-27 00:31:52,645][105692] Updated weights for policy 0, policy_version 1255784 (0.0009) [2023-12-27 00:31:52,686][105620] Updated weights for policy 1, policy_version 1256987 (0.0007) [2023-12-27 00:31:53,368][105620] Updated weights for policy 1, policy_version 1256997 (0.0009) [2023-12-27 00:31:53,415][105620] Updated weights for policy 1, policy_version 1257007 (0.0005) [2023-12-27 00:31:53,459][105620] Updated weights for policy 1, policy_version 1257017 (0.0005) [2023-12-27 00:31:53,475][105692] Updated weights for policy 0, policy_version 1255794 (0.0009) [2023-12-27 00:31:53,523][105692] Updated weights for policy 0, policy_version 1255804 (0.0010) [2023-12-27 00:31:53,577][105692] Updated weights for policy 0, policy_version 1255814 (0.0010) [2023-12-27 00:31:53,624][105692] Updated weights for policy 0, policy_version 1255824 (0.0010) [2023-12-27 00:31:54,114][105620] Updated weights for policy 1, policy_version 1257027 (0.0007) [2023-12-27 00:31:54,173][105620] Updated weights for policy 1, policy_version 1257038 (0.0010) [2023-12-27 00:31:54,232][105620] Updated weights for policy 1, policy_version 1257048 (0.0005) [2023-12-27 00:31:54,248][105692] Updated weights for policy 0, policy_version 1255834 (0.0006) [2023-12-27 00:31:54,301][105692] Updated weights for policy 0, policy_version 1255844 (0.0006) [2023-12-27 00:31:54,354][105692] Updated weights for policy 0, policy_version 1255854 (0.0006) [2023-12-27 00:31:54,925][105620] Updated weights for policy 1, policy_version 1257058 (0.0010) [2023-12-27 00:31:54,986][105692] Updated weights for policy 0, policy_version 1255864 (0.0006) [2023-12-27 00:31:54,988][105620] Updated weights for policy 1, policy_version 1257068 (0.0010) [2023-12-27 00:31:55,043][105692] Updated weights for policy 0, policy_version 1255874 (0.0006) [2023-12-27 00:31:55,047][105620] Updated weights for policy 1, policy_version 1257078 (0.0010) [2023-12-27 00:31:55,097][105692] Updated weights for policy 0, policy_version 1255884 (0.0005) [2023-12-27 00:31:55,106][105620] Updated weights for policy 1, policy_version 1257088 (0.0010) [2023-12-27 00:31:55,722][105620] Updated weights for policy 1, policy_version 1257098 (0.0007) [2023-12-27 00:31:55,771][105620] Updated weights for policy 1, policy_version 1257108 (0.0010) [2023-12-27 00:31:55,834][105692] Updated weights for policy 0, policy_version 1255894 (0.0007) [2023-12-27 00:31:55,835][105620] Updated weights for policy 1, policy_version 1257118 (0.0010) [2023-12-27 00:31:55,888][105692] Updated weights for policy 0, policy_version 1255904 (0.0007) [2023-12-27 00:31:55,945][105692] Updated weights for policy 0, policy_version 1255914 (0.0008) [2023-12-27 00:31:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 643432448. Throughput: 0: 9907.9, 1: 9612.6. Samples: 643436684. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:31:56,063][104569] Avg episode reward: [(0, '8801.269'), (1, '8983.995')] [2023-12-27 00:31:56,432][105620] Updated weights for policy 1, policy_version 1257128 (0.0010) [2023-12-27 00:31:56,483][105620] Updated weights for policy 1, policy_version 1257138 (0.0010) [2023-12-27 00:31:56,542][105620] Updated weights for policy 1, policy_version 1257148 (0.0009) [2023-12-27 00:31:56,598][105692] Updated weights for policy 0, policy_version 1255924 (0.0009) [2023-12-27 00:31:56,654][105692] Updated weights for policy 0, policy_version 1255934 (0.0008) [2023-12-27 00:31:56,726][105692] Updated weights for policy 0, policy_version 1255944 (0.0010) [2023-12-27 00:31:57,271][105620] Updated weights for policy 1, policy_version 1257158 (0.0006) [2023-12-27 00:31:57,321][105692] Updated weights for policy 0, policy_version 1255954 (0.0009) [2023-12-27 00:31:57,329][105620] Updated weights for policy 1, policy_version 1257168 (0.0006) [2023-12-27 00:31:57,379][105692] Updated weights for policy 0, policy_version 1255964 (0.0005) [2023-12-27 00:31:57,383][105620] Updated weights for policy 1, policy_version 1257178 (0.0007) [2023-12-27 00:31:57,441][105692] Updated weights for policy 0, policy_version 1255974 (0.0008) [2023-12-27 00:31:57,506][105692] Updated weights for policy 0, policy_version 1255984 (0.0009) [2023-12-27 00:31:58,024][105620] Updated weights for policy 1, policy_version 1257188 (0.0008) [2023-12-27 00:31:58,068][105620] Updated weights for policy 1, policy_version 1257198 (0.0010) [2023-12-27 00:31:58,116][105620] Updated weights for policy 1, policy_version 1257208 (0.0010) [2023-12-27 00:31:58,230][105692] Updated weights for policy 0, policy_version 1255994 (0.0009) [2023-12-27 00:31:58,290][105692] Updated weights for policy 0, policy_version 1256004 (0.0008) [2023-12-27 00:31:58,350][105692] Updated weights for policy 0, policy_version 1256014 (0.0007) [2023-12-27 00:31:58,950][105620] Updated weights for policy 1, policy_version 1257218 (0.0010) [2023-12-27 00:31:59,008][105620] Updated weights for policy 1, policy_version 1257228 (0.0008) [2023-12-27 00:31:59,067][105620] Updated weights for policy 1, policy_version 1257238 (0.0009) [2023-12-27 00:31:59,117][105692] Updated weights for policy 0, policy_version 1256024 (0.0007) [2023-12-27 00:31:59,123][105620] Updated weights for policy 1, policy_version 1257248 (0.0006) [2023-12-27 00:31:59,174][105692] Updated weights for policy 0, policy_version 1256034 (0.0009) [2023-12-27 00:31:59,241][105692] Updated weights for policy 0, policy_version 1256044 (0.0009) [2023-12-27 00:31:59,946][105620] Updated weights for policy 1, policy_version 1257258 (0.0010) [2023-12-27 00:31:59,973][105692] Updated weights for policy 0, policy_version 1256054 (0.0007) [2023-12-27 00:32:00,008][105620] Updated weights for policy 1, policy_version 1257268 (0.0008) [2023-12-27 00:32:00,034][105692] Updated weights for policy 0, policy_version 1256064 (0.0008) [2023-12-27 00:32:00,069][105620] Updated weights for policy 1, policy_version 1257278 (0.0007) [2023-12-27 00:32:00,088][105692] Updated weights for policy 0, policy_version 1256074 (0.0007) [2023-12-27 00:32:00,800][105620] Updated weights for policy 1, policy_version 1257288 (0.0008) [2023-12-27 00:32:00,851][105692] Updated weights for policy 0, policy_version 1256084 (0.0007) [2023-12-27 00:32:00,854][105620] Updated weights for policy 1, policy_version 1257298 (0.0008) [2023-12-27 00:32:00,897][105692] Updated weights for policy 0, policy_version 1256094 (0.0005) [2023-12-27 00:32:00,903][105620] Updated weights for policy 1, policy_version 1257308 (0.0007) [2023-12-27 00:32:00,942][105692] Updated weights for policy 0, policy_version 1256104 (0.0005) [2023-12-27 00:32:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 643530752. Throughput: 0: 9929.2, 1: 9665.8. Samples: 643496752. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:32:01,062][104569] Avg episode reward: [(0, '8894.527'), (1, '9167.601')] [2023-12-27 00:32:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001256112_321617920.pth... [2023-12-27 00:32:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001257312_321912832.pth... [2023-12-27 00:32:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001254960_321323008.pth [2023-12-27 00:32:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001256160_321617920.pth [2023-12-27 00:32:01,605][105692] Updated weights for policy 0, policy_version 1256114 (0.0005) [2023-12-27 00:32:01,665][105692] Updated weights for policy 0, policy_version 1256124 (0.0008) [2023-12-27 00:32:01,728][105692] Updated weights for policy 0, policy_version 1256134 (0.0010) [2023-12-27 00:32:01,761][105620] Updated weights for policy 1, policy_version 1257318 (0.0007) [2023-12-27 00:32:01,777][105692] Updated weights for policy 0, policy_version 1256144 (0.0008) [2023-12-27 00:32:01,815][105620] Updated weights for policy 1, policy_version 1257328 (0.0009) [2023-12-27 00:32:01,878][105620] Updated weights for policy 1, policy_version 1257338 (0.0008) [2023-12-27 00:32:02,490][105692] Updated weights for policy 0, policy_version 1256154 (0.0006) [2023-12-27 00:32:02,553][105692] Updated weights for policy 0, policy_version 1256164 (0.0006) [2023-12-27 00:32:02,613][105692] Updated weights for policy 0, policy_version 1256174 (0.0006) [2023-12-27 00:32:02,710][105620] Updated weights for policy 1, policy_version 1257348 (0.0009) [2023-12-27 00:32:02,782][105620] Updated weights for policy 1, policy_version 1257358 (0.0009) [2023-12-27 00:32:02,846][105620] Updated weights for policy 1, policy_version 1257368 (0.0009) [2023-12-27 00:32:03,146][105692] Updated weights for policy 0, policy_version 1256184 (0.0006) [2023-12-27 00:32:03,195][105692] Updated weights for policy 0, policy_version 1256194 (0.0005) [2023-12-27 00:32:03,260][105692] Updated weights for policy 0, policy_version 1256204 (0.0006) [2023-12-27 00:32:03,642][105620] Updated weights for policy 1, policy_version 1257378 (0.0009) [2023-12-27 00:32:03,692][105620] Updated weights for policy 1, policy_version 1257388 (0.0009) [2023-12-27 00:32:03,744][105620] Updated weights for policy 1, policy_version 1257398 (0.0008) [2023-12-27 00:32:03,795][105620] Updated weights for policy 1, policy_version 1257408 (0.0008) [2023-12-27 00:32:03,961][105692] Updated weights for policy 0, policy_version 1256214 (0.0009) [2023-12-27 00:32:04,019][105692] Updated weights for policy 0, policy_version 1256224 (0.0009) [2023-12-27 00:32:04,073][105692] Updated weights for policy 0, policy_version 1256234 (0.0009) [2023-12-27 00:32:04,507][105620] Updated weights for policy 1, policy_version 1257418 (0.0010) [2023-12-27 00:32:04,570][105620] Updated weights for policy 1, policy_version 1257428 (0.0009) [2023-12-27 00:32:04,631][105620] Updated weights for policy 1, policy_version 1257438 (0.0008) [2023-12-27 00:32:04,803][105692] Updated weights for policy 0, policy_version 1256244 (0.0009) [2023-12-27 00:32:04,850][105692] Updated weights for policy 0, policy_version 1256254 (0.0008) [2023-12-27 00:32:04,898][105692] Updated weights for policy 0, policy_version 1256264 (0.0009) [2023-12-27 00:32:05,320][105620] Updated weights for policy 1, policy_version 1257448 (0.0006) [2023-12-27 00:32:05,368][105620] Updated weights for policy 1, policy_version 1257458 (0.0005) [2023-12-27 00:32:05,423][105620] Updated weights for policy 1, policy_version 1257468 (0.0005) [2023-12-27 00:32:05,716][105692] Updated weights for policy 0, policy_version 1256274 (0.0009) [2023-12-27 00:32:05,773][105692] Updated weights for policy 0, policy_version 1256285 (0.0006) [2023-12-27 00:32:05,833][105692] Updated weights for policy 0, policy_version 1256295 (0.0007) [2023-12-27 00:32:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 643620864. Throughput: 0: 9786.7, 1: 9625.9. Samples: 643609332. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:32:06,063][104569] Avg episode reward: [(0, '9262.364'), (1, '9077.256')] [2023-12-27 00:32:06,086][105620] Updated weights for policy 1, policy_version 1257478 (0.0005) [2023-12-27 00:32:06,154][105620] Updated weights for policy 1, policy_version 1257488 (0.0007) [2023-12-27 00:32:06,216][105620] Updated weights for policy 1, policy_version 1257498 (0.0010) [2023-12-27 00:32:06,511][105692] Updated weights for policy 0, policy_version 1256305 (0.0010) [2023-12-27 00:32:06,574][105692] Updated weights for policy 0, policy_version 1256315 (0.0006) [2023-12-27 00:32:06,633][105692] Updated weights for policy 0, policy_version 1256325 (0.0007) [2023-12-27 00:32:06,685][105692] Updated weights for policy 0, policy_version 1256335 (0.0006) [2023-12-27 00:32:07,003][105620] Updated weights for policy 1, policy_version 1257508 (0.0010) [2023-12-27 00:32:07,052][105620] Updated weights for policy 1, policy_version 1257518 (0.0010) [2023-12-27 00:32:07,105][105620] Updated weights for policy 1, policy_version 1257528 (0.0010) [2023-12-27 00:32:07,248][105692] Updated weights for policy 0, policy_version 1256345 (0.0005) [2023-12-27 00:32:07,307][105692] Updated weights for policy 0, policy_version 1256355 (0.0005) [2023-12-27 00:32:07,364][105692] Updated weights for policy 0, policy_version 1256365 (0.0008) [2023-12-27 00:32:07,776][105620] Updated weights for policy 1, policy_version 1257538 (0.0010) [2023-12-27 00:32:07,832][105620] Updated weights for policy 1, policy_version 1257548 (0.0005) [2023-12-27 00:32:07,886][105620] Updated weights for policy 1, policy_version 1257558 (0.0005) [2023-12-27 00:32:07,932][105620] Updated weights for policy 1, policy_version 1257568 (0.0005) [2023-12-27 00:32:08,066][105692] Updated weights for policy 0, policy_version 1256375 (0.0011) [2023-12-27 00:32:08,133][105692] Updated weights for policy 0, policy_version 1256385 (0.0011) [2023-12-27 00:32:08,185][105692] Updated weights for policy 0, policy_version 1256395 (0.0010) [2023-12-27 00:32:08,672][105620] Updated weights for policy 1, policy_version 1257578 (0.0010) [2023-12-27 00:32:08,731][105620] Updated weights for policy 1, policy_version 1257588 (0.0010) [2023-12-27 00:32:08,790][105620] Updated weights for policy 1, policy_version 1257598 (0.0010) [2023-12-27 00:32:08,916][105692] Updated weights for policy 0, policy_version 1256405 (0.0010) [2023-12-27 00:32:08,974][105692] Updated weights for policy 0, policy_version 1256415 (0.0010) [2023-12-27 00:32:09,029][105692] Updated weights for policy 0, policy_version 1256425 (0.0010) [2023-12-27 00:32:09,553][105620] Updated weights for policy 1, policy_version 1257608 (0.0011) [2023-12-27 00:32:09,610][105620] Updated weights for policy 1, policy_version 1257618 (0.0011) [2023-12-27 00:32:09,669][105620] Updated weights for policy 1, policy_version 1257628 (0.0011) [2023-12-27 00:32:09,787][105692] Updated weights for policy 0, policy_version 1256435 (0.0011) [2023-12-27 00:32:09,844][105692] Updated weights for policy 0, policy_version 1256445 (0.0011) [2023-12-27 00:32:09,912][105692] Updated weights for policy 0, policy_version 1256455 (0.0011) [2023-12-27 00:32:10,406][105620] Updated weights for policy 1, policy_version 1257638 (0.0011) [2023-12-27 00:32:10,469][105620] Updated weights for policy 1, policy_version 1257648 (0.0011) [2023-12-27 00:32:10,538][105620] Updated weights for policy 1, policy_version 1257658 (0.0010) [2023-12-27 00:32:10,553][105692] Updated weights for policy 0, policy_version 1256465 (0.0010) [2023-12-27 00:32:10,604][105692] Updated weights for policy 0, policy_version 1256475 (0.0006) [2023-12-27 00:32:10,664][105692] Updated weights for policy 0, policy_version 1256485 (0.0008) [2023-12-27 00:32:10,727][105692] Updated weights for policy 0, policy_version 1256495 (0.0006) [2023-12-27 00:32:11,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 643719168. Throughput: 0: 9873.5, 1: 9684.2. Samples: 643727852. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 00:32:11,063][104569] Avg episode reward: [(0, '9260.100'), (1, '9077.596')] [2023-12-27 00:32:11,302][105620] Updated weights for policy 1, policy_version 1257668 (0.0009) [2023-12-27 00:32:11,374][105620] Updated weights for policy 1, policy_version 1257678 (0.0009) [2023-12-27 00:32:11,414][105692] Updated weights for policy 0, policy_version 1256505 (0.0009) [2023-12-27 00:32:11,429][105620] Updated weights for policy 1, policy_version 1257688 (0.0006) [2023-12-27 00:32:11,472][105692] Updated weights for policy 0, policy_version 1256515 (0.0006) [2023-12-27 00:32:11,530][105692] Updated weights for policy 0, policy_version 1256525 (0.0006) [2023-12-27 00:32:12,094][105620] Updated weights for policy 1, policy_version 1257698 (0.0009) [2023-12-27 00:32:12,161][105620] Updated weights for policy 1, policy_version 1257708 (0.0005) [2023-12-27 00:32:12,231][105620] Updated weights for policy 1, policy_version 1257718 (0.0007) [2023-12-27 00:32:12,269][105692] Updated weights for policy 0, policy_version 1256535 (0.0008) [2023-12-27 00:32:12,311][105620] Updated weights for policy 1, policy_version 1257728 (0.0009) [2023-12-27 00:32:12,335][105692] Updated weights for policy 0, policy_version 1256545 (0.0013) [2023-12-27 00:32:12,402][105692] Updated weights for policy 0, policy_version 1256555 (0.0008) [2023-12-27 00:32:13,019][105620] Updated weights for policy 1, policy_version 1257738 (0.0009) [2023-12-27 00:32:13,083][105620] Updated weights for policy 1, policy_version 1257748 (0.0006) [2023-12-27 00:32:13,113][105692] Updated weights for policy 0, policy_version 1256565 (0.0007) [2023-12-27 00:32:13,149][105620] Updated weights for policy 1, policy_version 1257758 (0.0008) [2023-12-27 00:32:13,164][105692] Updated weights for policy 0, policy_version 1256575 (0.0007) [2023-12-27 00:32:13,212][105692] Updated weights for policy 0, policy_version 1256585 (0.0008) [2023-12-27 00:32:13,854][105692] Updated weights for policy 0, policy_version 1256595 (0.0007) [2023-12-27 00:32:13,908][105692] Updated weights for policy 0, policy_version 1256605 (0.0008) [2023-12-27 00:32:13,960][105692] Updated weights for policy 0, policy_version 1256615 (0.0005) [2023-12-27 00:32:13,966][105620] Updated weights for policy 1, policy_version 1257768 (0.0008) [2023-12-27 00:32:14,030][105620] Updated weights for policy 1, policy_version 1257778 (0.0008) [2023-12-27 00:32:14,081][105620] Updated weights for policy 1, policy_version 1257788 (0.0008) [2023-12-27 00:32:14,679][105692] Updated weights for policy 0, policy_version 1256625 (0.0010) [2023-12-27 00:32:14,726][105692] Updated weights for policy 0, policy_version 1256635 (0.0010) [2023-12-27 00:32:14,784][105692] Updated weights for policy 0, policy_version 1256645 (0.0010) [2023-12-27 00:32:14,819][105620] Updated weights for policy 1, policy_version 1257798 (0.0009) [2023-12-27 00:32:14,840][105692] Updated weights for policy 0, policy_version 1256655 (0.0010) [2023-12-27 00:32:14,877][105620] Updated weights for policy 1, policy_version 1257808 (0.0008) [2023-12-27 00:32:14,922][105620] Updated weights for policy 1, policy_version 1257818 (0.0008) [2023-12-27 00:32:15,644][105692] Updated weights for policy 0, policy_version 1256665 (0.0009) [2023-12-27 00:32:15,658][105620] Updated weights for policy 1, policy_version 1257828 (0.0008) [2023-12-27 00:32:15,693][105692] Updated weights for policy 0, policy_version 1256675 (0.0005) [2023-12-27 00:32:15,714][105620] Updated weights for policy 1, policy_version 1257838 (0.0008) [2023-12-27 00:32:15,746][105692] Updated weights for policy 0, policy_version 1256685 (0.0009) [2023-12-27 00:32:15,777][105620] Updated weights for policy 1, policy_version 1257848 (0.0006) [2023-12-27 00:32:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 643817472. Throughput: 0: 9900.9, 1: 9619.9. Samples: 643784768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:32:16,062][104569] Avg episode reward: [(0, '9078.924'), (1, '9260.423')] [2023-12-27 00:32:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001256688_321765376.pth... [2023-12-27 00:32:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001257856_322052096.pth... [2023-12-27 00:32:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001255536_321470464.pth [2023-12-27 00:32:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001256736_321765376.pth [2023-12-27 00:32:16,349][105620] Updated weights for policy 1, policy_version 1257858 (0.0006) [2023-12-27 00:32:16,406][105620] Updated weights for policy 1, policy_version 1257868 (0.0008) [2023-12-27 00:32:16,436][105692] Updated weights for policy 0, policy_version 1256695 (0.0010) [2023-12-27 00:32:16,459][105620] Updated weights for policy 1, policy_version 1257878 (0.0005) [2023-12-27 00:32:16,492][105692] Updated weights for policy 0, policy_version 1256705 (0.0010) [2023-12-27 00:32:16,518][105620] Updated weights for policy 1, policy_version 1257888 (0.0005) [2023-12-27 00:32:16,539][105692] Updated weights for policy 0, policy_version 1256715 (0.0010) [2023-12-27 00:32:17,216][105620] Updated weights for policy 1, policy_version 1257898 (0.0008) [2023-12-27 00:32:17,278][105620] Updated weights for policy 1, policy_version 1257908 (0.0008) [2023-12-27 00:32:17,303][105692] Updated weights for policy 0, policy_version 1256725 (0.0010) [2023-12-27 00:32:17,333][105620] Updated weights for policy 1, policy_version 1257918 (0.0009) [2023-12-27 00:32:17,364][105692] Updated weights for policy 0, policy_version 1256735 (0.0010) [2023-12-27 00:32:17,431][105692] Updated weights for policy 0, policy_version 1256745 (0.0010) [2023-12-27 00:32:17,979][105620] Updated weights for policy 1, policy_version 1257928 (0.0006) [2023-12-27 00:32:18,035][105620] Updated weights for policy 1, policy_version 1257938 (0.0010) [2023-12-27 00:32:18,080][105620] Updated weights for policy 1, policy_version 1257948 (0.0010) [2023-12-27 00:32:18,152][105692] Updated weights for policy 0, policy_version 1256755 (0.0010) [2023-12-27 00:32:18,212][105692] Updated weights for policy 0, policy_version 1256765 (0.0011) [2023-12-27 00:32:18,271][105692] Updated weights for policy 0, policy_version 1256775 (0.0010) [2023-12-27 00:32:18,838][105620] Updated weights for policy 1, policy_version 1257958 (0.0010) [2023-12-27 00:32:18,890][105620] Updated weights for policy 1, policy_version 1257968 (0.0011) [2023-12-27 00:32:18,948][105620] Updated weights for policy 1, policy_version 1257978 (0.0010) [2023-12-27 00:32:18,974][105692] Updated weights for policy 0, policy_version 1256785 (0.0010) [2023-12-27 00:32:19,021][105692] Updated weights for policy 0, policy_version 1256795 (0.0008) [2023-12-27 00:32:19,066][105692] Updated weights for policy 0, policy_version 1256805 (0.0008) [2023-12-27 00:32:19,122][105692] Updated weights for policy 0, policy_version 1256815 (0.0008) [2023-12-27 00:32:19,714][105620] Updated weights for policy 1, policy_version 1257988 (0.0011) [2023-12-27 00:32:19,774][105620] Updated weights for policy 1, policy_version 1257998 (0.0010) [2023-12-27 00:32:19,831][105620] Updated weights for policy 1, policy_version 1258008 (0.0011) [2023-12-27 00:32:19,916][105692] Updated weights for policy 0, policy_version 1256825 (0.0007) [2023-12-27 00:32:19,977][105692] Updated weights for policy 0, policy_version 1256835 (0.0009) [2023-12-27 00:32:20,038][105692] Updated weights for policy 0, policy_version 1256845 (0.0008) [2023-12-27 00:32:20,603][105620] Updated weights for policy 1, policy_version 1258018 (0.0011) [2023-12-27 00:32:20,664][105620] Updated weights for policy 1, policy_version 1258028 (0.0011) [2023-12-27 00:32:20,731][105620] Updated weights for policy 1, policy_version 1258038 (0.0010) [2023-12-27 00:32:20,793][105620] Updated weights for policy 1, policy_version 1258048 (0.0006) [2023-12-27 00:32:20,825][105692] Updated weights for policy 0, policy_version 1256855 (0.0009) [2023-12-27 00:32:20,885][105692] Updated weights for policy 0, policy_version 1256865 (0.0008) [2023-12-27 00:32:20,941][105692] Updated weights for policy 0, policy_version 1256875 (0.0008) [2023-12-27 00:32:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 643915776. Throughput: 0: 9839.5, 1: 9632.2. Samples: 643901828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:32:21,063][104569] Avg episode reward: [(0, '9173.290'), (1, '9168.272')] [2023-12-27 00:32:21,525][105620] Updated weights for policy 1, policy_version 1258058 (0.0011) [2023-12-27 00:32:21,580][105620] Updated weights for policy 1, policy_version 1258068 (0.0009) [2023-12-27 00:32:21,649][105620] Updated weights for policy 1, policy_version 1258078 (0.0009) [2023-12-27 00:32:21,712][105692] Updated weights for policy 0, policy_version 1256885 (0.0008) [2023-12-27 00:32:21,781][105692] Updated weights for policy 0, policy_version 1256895 (0.0008) [2023-12-27 00:32:21,847][105692] Updated weights for policy 0, policy_version 1256905 (0.0006) [2023-12-27 00:32:22,305][105620] Updated weights for policy 1, policy_version 1258088 (0.0007) [2023-12-27 00:32:22,367][105620] Updated weights for policy 1, policy_version 1258098 (0.0007) [2023-12-27 00:32:22,424][105620] Updated weights for policy 1, policy_version 1258108 (0.0008) [2023-12-27 00:32:22,527][105692] Updated weights for policy 0, policy_version 1256915 (0.0009) [2023-12-27 00:32:22,592][105692] Updated weights for policy 0, policy_version 1256925 (0.0009) [2023-12-27 00:32:22,654][105692] Updated weights for policy 0, policy_version 1256935 (0.0007) [2023-12-27 00:32:23,188][105620] Updated weights for policy 1, policy_version 1258118 (0.0008) [2023-12-27 00:32:23,254][105620] Updated weights for policy 1, policy_version 1258128 (0.0008) [2023-12-27 00:32:23,306][105620] Updated weights for policy 1, policy_version 1258138 (0.0008) [2023-12-27 00:32:23,363][105692] Updated weights for policy 0, policy_version 1256945 (0.0010) [2023-12-27 00:32:23,430][105692] Updated weights for policy 0, policy_version 1256955 (0.0010) [2023-12-27 00:32:23,493][105692] Updated weights for policy 0, policy_version 1256965 (0.0010) [2023-12-27 00:32:23,553][105692] Updated weights for policy 0, policy_version 1256975 (0.0011) [2023-12-27 00:32:24,106][105620] Updated weights for policy 1, policy_version 1258148 (0.0008) [2023-12-27 00:32:24,165][105620] Updated weights for policy 1, policy_version 1258158 (0.0008) [2023-12-27 00:32:24,214][105620] Updated weights for policy 1, policy_version 1258168 (0.0008) [2023-12-27 00:32:24,278][105692] Updated weights for policy 0, policy_version 1256985 (0.0011) [2023-12-27 00:32:24,334][105692] Updated weights for policy 0, policy_version 1256995 (0.0010) [2023-12-27 00:32:24,393][105692] Updated weights for policy 0, policy_version 1257005 (0.0010) [2023-12-27 00:32:24,957][105620] Updated weights for policy 1, policy_version 1258178 (0.0007) [2023-12-27 00:32:25,014][105620] Updated weights for policy 1, policy_version 1258188 (0.0008) [2023-12-27 00:32:25,073][105620] Updated weights for policy 1, policy_version 1258198 (0.0008) [2023-12-27 00:32:25,136][105620] Updated weights for policy 1, policy_version 1258208 (0.0007) [2023-12-27 00:32:25,141][105692] Updated weights for policy 0, policy_version 1257015 (0.0010) [2023-12-27 00:32:25,206][105692] Updated weights for policy 0, policy_version 1257025 (0.0010) [2023-12-27 00:32:25,261][105692] Updated weights for policy 0, policy_version 1257035 (0.0010) [2023-12-27 00:32:25,859][105620] Updated weights for policy 1, policy_version 1258218 (0.0007) [2023-12-27 00:32:25,922][105620] Updated weights for policy 1, policy_version 1258229 (0.0008) [2023-12-27 00:32:25,977][105620] Updated weights for policy 1, policy_version 1258239 (0.0005) [2023-12-27 00:32:26,003][105692] Updated weights for policy 0, policy_version 1257045 (0.0010) [2023-12-27 00:32:26,053][105692] Updated weights for policy 0, policy_version 1257055 (0.0010) [2023-12-27 00:32:26,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 644005888. Throughput: 0: 9756.2, 1: 9623.0. Samples: 644014820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:32:26,063][104569] Avg episode reward: [(0, '9175.245'), (1, '9171.461')] [2023-12-27 00:32:26,101][105692] Updated weights for policy 0, policy_version 1257065 (0.0010) [2023-12-27 00:32:26,565][105620] Updated weights for policy 1, policy_version 1258249 (0.0005) [2023-12-27 00:32:26,623][105620] Updated weights for policy 1, policy_version 1258259 (0.0010) [2023-12-27 00:32:26,680][105620] Updated weights for policy 1, policy_version 1258269 (0.0010) [2023-12-27 00:32:26,857][105692] Updated weights for policy 0, policy_version 1257075 (0.0010) [2023-12-27 00:32:26,908][105692] Updated weights for policy 0, policy_version 1257085 (0.0010) [2023-12-27 00:32:26,955][105692] Updated weights for policy 0, policy_version 1257095 (0.0010) [2023-12-27 00:32:27,241][105620] Updated weights for policy 1, policy_version 1258279 (0.0005) [2023-12-27 00:32:27,303][105620] Updated weights for policy 1, policy_version 1258289 (0.0006) [2023-12-27 00:32:27,362][105620] Updated weights for policy 1, policy_version 1258299 (0.0006) [2023-12-27 00:32:27,702][105692] Updated weights for policy 0, policy_version 1257105 (0.0010) [2023-12-27 00:32:27,747][105692] Updated weights for policy 0, policy_version 1257115 (0.0007) [2023-12-27 00:32:27,789][105692] Updated weights for policy 0, policy_version 1257125 (0.0005) [2023-12-27 00:32:27,838][105692] Updated weights for policy 0, policy_version 1257135 (0.0005) [2023-12-27 00:32:27,908][105620] Updated weights for policy 1, policy_version 1258309 (0.0006) [2023-12-27 00:32:27,971][105620] Updated weights for policy 1, policy_version 1258319 (0.0005) [2023-12-27 00:32:28,032][105620] Updated weights for policy 1, policy_version 1258329 (0.0009) [2023-12-27 00:32:28,553][105692] Updated weights for policy 0, policy_version 1257145 (0.0009) [2023-12-27 00:32:28,601][105692] Updated weights for policy 0, policy_version 1257156 (0.0009) [2023-12-27 00:32:28,646][105692] Updated weights for policy 0, policy_version 1257166 (0.0008) [2023-12-27 00:32:28,665][105620] Updated weights for policy 1, policy_version 1258339 (0.0009) [2023-12-27 00:32:28,713][105620] Updated weights for policy 1, policy_version 1258349 (0.0010) [2023-12-27 00:32:28,773][105620] Updated weights for policy 1, policy_version 1258359 (0.0010) [2023-12-27 00:32:29,315][105692] Updated weights for policy 0, policy_version 1257176 (0.0006) [2023-12-27 00:32:29,375][105692] Updated weights for policy 0, policy_version 1257186 (0.0008) [2023-12-27 00:32:29,435][105692] Updated weights for policy 0, policy_version 1257196 (0.0006) [2023-12-27 00:32:29,523][105620] Updated weights for policy 1, policy_version 1258369 (0.0010) [2023-12-27 00:32:29,587][105620] Updated weights for policy 1, policy_version 1258379 (0.0010) [2023-12-27 00:32:29,651][105620] Updated weights for policy 1, policy_version 1258389 (0.0010) [2023-12-27 00:32:29,709][105620] Updated weights for policy 1, policy_version 1258399 (0.0010) [2023-12-27 00:32:30,124][105692] Updated weights for policy 0, policy_version 1257206 (0.0007) [2023-12-27 00:32:30,189][105692] Updated weights for policy 0, policy_version 1257216 (0.0008) [2023-12-27 00:32:30,248][105692] Updated weights for policy 0, policy_version 1257226 (0.0008) [2023-12-27 00:32:30,471][105620] Updated weights for policy 1, policy_version 1258409 (0.0011) [2023-12-27 00:32:30,516][105620] Updated weights for policy 1, policy_version 1258419 (0.0010) [2023-12-27 00:32:30,569][105620] Updated weights for policy 1, policy_version 1258429 (0.0010) [2023-12-27 00:32:30,976][105692] Updated weights for policy 0, policy_version 1257236 (0.0008) [2023-12-27 00:32:31,022][105692] Updated weights for policy 0, policy_version 1257246 (0.0008) [2023-12-27 00:32:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 644104192. Throughput: 0: 9785.6, 1: 9763.8. Samples: 644077904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:32:31,063][104569] Avg episode reward: [(0, '8990.577'), (1, '8988.489')] [2023-12-27 00:32:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001258432_322199552.pth... [2023-12-27 00:32:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001257312_321912832.pth [2023-12-27 00:32:31,086][105692] Updated weights for policy 0, policy_version 1257256 (0.0009) [2023-12-27 00:32:31,133][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001257264_321912832.pth... [2023-12-27 00:32:31,137][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001256112_321617920.pth [2023-12-27 00:32:31,318][105620] Updated weights for policy 1, policy_version 1258439 (0.0011) [2023-12-27 00:32:31,392][105620] Updated weights for policy 1, policy_version 1258449 (0.0010) [2023-12-27 00:32:31,442][105620] Updated weights for policy 1, policy_version 1258459 (0.0011) [2023-12-27 00:32:31,886][105692] Updated weights for policy 0, policy_version 1257266 (0.0009) [2023-12-27 00:32:31,945][105692] Updated weights for policy 0, policy_version 1257276 (0.0008) [2023-12-27 00:32:31,997][105692] Updated weights for policy 0, policy_version 1257286 (0.0008) [2023-12-27 00:32:32,049][105692] Updated weights for policy 0, policy_version 1257296 (0.0008) [2023-12-27 00:32:32,211][105620] Updated weights for policy 1, policy_version 1258469 (0.0010) [2023-12-27 00:32:32,268][105620] Updated weights for policy 1, policy_version 1258479 (0.0009) [2023-12-27 00:32:32,322][105620] Updated weights for policy 1, policy_version 1258489 (0.0009) [2023-12-27 00:32:32,770][105692] Updated weights for policy 0, policy_version 1257306 (0.0008) [2023-12-27 00:32:32,816][105692] Updated weights for policy 0, policy_version 1257316 (0.0008) [2023-12-27 00:32:32,862][105692] Updated weights for policy 0, policy_version 1257326 (0.0008) [2023-12-27 00:32:33,100][105620] Updated weights for policy 1, policy_version 1258499 (0.0009) [2023-12-27 00:32:33,154][105620] Updated weights for policy 1, policy_version 1258509 (0.0009) [2023-12-27 00:32:33,226][105620] Updated weights for policy 1, policy_version 1258519 (0.0009) [2023-12-27 00:32:33,614][105692] Updated weights for policy 0, policy_version 1257336 (0.0009) [2023-12-27 00:32:33,675][105692] Updated weights for policy 0, policy_version 1257346 (0.0009) [2023-12-27 00:32:33,732][105692] Updated weights for policy 0, policy_version 1257356 (0.0009) [2023-12-27 00:32:33,968][105620] Updated weights for policy 1, policy_version 1258529 (0.0009) [2023-12-27 00:32:34,020][105620] Updated weights for policy 1, policy_version 1258539 (0.0009) [2023-12-27 00:32:34,078][105620] Updated weights for policy 1, policy_version 1258549 (0.0009) [2023-12-27 00:32:34,125][105620] Updated weights for policy 1, policy_version 1258559 (0.0008) [2023-12-27 00:32:34,484][105692] Updated weights for policy 0, policy_version 1257366 (0.0009) [2023-12-27 00:32:34,542][105692] Updated weights for policy 0, policy_version 1257376 (0.0009) [2023-12-27 00:32:34,601][105692] Updated weights for policy 0, policy_version 1257386 (0.0009) [2023-12-27 00:32:34,914][105620] Updated weights for policy 1, policy_version 1258569 (0.0009) [2023-12-27 00:32:34,969][105620] Updated weights for policy 1, policy_version 1258579 (0.0010) [2023-12-27 00:32:35,030][105620] Updated weights for policy 1, policy_version 1258589 (0.0009) [2023-12-27 00:32:35,241][105692] Updated weights for policy 0, policy_version 1257396 (0.0008) [2023-12-27 00:32:35,306][105692] Updated weights for policy 0, policy_version 1257406 (0.0008) [2023-12-27 00:32:35,363][105692] Updated weights for policy 0, policy_version 1257416 (0.0007) [2023-12-27 00:32:35,813][105620] Updated weights for policy 1, policy_version 1258599 (0.0010) [2023-12-27 00:32:35,867][105620] Updated weights for policy 1, policy_version 1258609 (0.0009) [2023-12-27 00:32:35,928][105620] Updated weights for policy 1, policy_version 1258619 (0.0008) [2023-12-27 00:32:35,954][105692] Updated weights for policy 0, policy_version 1257426 (0.0007) [2023-12-27 00:32:36,025][105692] Updated weights for policy 0, policy_version 1257436 (0.0005) [2023-12-27 00:32:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 644202496. Throughput: 0: 9766.6, 1: 9627.3. Samples: 644190744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:32:36,063][104569] Avg episode reward: [(0, '9169.559'), (1, '9168.803')] [2023-12-27 00:32:36,082][105692] Updated weights for policy 0, policy_version 1257446 (0.0005) [2023-12-27 00:32:36,146][105692] Updated weights for policy 0, policy_version 1257456 (0.0007) [2023-12-27 00:32:36,725][105692] Updated weights for policy 0, policy_version 1257466 (0.0006) [2023-12-27 00:32:36,739][105620] Updated weights for policy 1, policy_version 1258629 (0.0007) [2023-12-27 00:32:36,791][105692] Updated weights for policy 0, policy_version 1257476 (0.0005) [2023-12-27 00:32:36,795][105620] Updated weights for policy 1, policy_version 1258639 (0.0008) [2023-12-27 00:32:36,854][105620] Updated weights for policy 1, policy_version 1258649 (0.0007) [2023-12-27 00:32:36,858][105692] Updated weights for policy 0, policy_version 1257486 (0.0005) [2023-12-27 00:32:37,476][105692] Updated weights for policy 0, policy_version 1257496 (0.0010) [2023-12-27 00:32:37,525][105692] Updated weights for policy 0, policy_version 1257506 (0.0010) [2023-12-27 00:32:37,571][105692] Updated weights for policy 0, policy_version 1257516 (0.0006) [2023-12-27 00:32:37,589][105620] Updated weights for policy 1, policy_version 1258659 (0.0007) [2023-12-27 00:32:37,650][105620] Updated weights for policy 1, policy_version 1258669 (0.0005) [2023-12-27 00:32:37,718][105620] Updated weights for policy 1, policy_version 1258679 (0.0008) [2023-12-27 00:32:38,215][105692] Updated weights for policy 0, policy_version 1257526 (0.0010) [2023-12-27 00:32:38,277][105692] Updated weights for policy 0, policy_version 1257536 (0.0006) [2023-12-27 00:32:38,345][105692] Updated weights for policy 0, policy_version 1257546 (0.0008) [2023-12-27 00:32:38,439][105620] Updated weights for policy 1, policy_version 1258689 (0.0010) [2023-12-27 00:32:38,495][105620] Updated weights for policy 1, policy_version 1258699 (0.0010) [2023-12-27 00:32:38,551][105620] Updated weights for policy 1, policy_version 1258709 (0.0010) [2023-12-27 00:32:38,614][105620] Updated weights for policy 1, policy_version 1258719 (0.0011) [2023-12-27 00:32:38,966][105692] Updated weights for policy 0, policy_version 1257556 (0.0006) [2023-12-27 00:32:39,021][105692] Updated weights for policy 0, policy_version 1257566 (0.0005) [2023-12-27 00:32:39,077][105692] Updated weights for policy 0, policy_version 1257576 (0.0005) [2023-12-27 00:32:39,424][105620] Updated weights for policy 1, policy_version 1258729 (0.0009) [2023-12-27 00:32:39,495][105620] Updated weights for policy 1, policy_version 1258739 (0.0008) [2023-12-27 00:32:39,565][105620] Updated weights for policy 1, policy_version 1258749 (0.0009) [2023-12-27 00:32:39,761][105692] Updated weights for policy 0, policy_version 1257586 (0.0006) [2023-12-27 00:32:39,825][105692] Updated weights for policy 0, policy_version 1257596 (0.0006) [2023-12-27 00:32:39,895][105692] Updated weights for policy 0, policy_version 1257606 (0.0008) [2023-12-27 00:32:39,959][105692] Updated weights for policy 0, policy_version 1257616 (0.0007) [2023-12-27 00:32:40,255][105620] Updated weights for policy 1, policy_version 1258759 (0.0008) [2023-12-27 00:32:40,317][105620] Updated weights for policy 1, policy_version 1258769 (0.0009) [2023-12-27 00:32:40,378][105620] Updated weights for policy 1, policy_version 1258779 (0.0009) [2023-12-27 00:32:40,670][105692] Updated weights for policy 0, policy_version 1257626 (0.0009) [2023-12-27 00:32:40,731][105692] Updated weights for policy 0, policy_version 1257637 (0.0010) [2023-12-27 00:32:40,792][105692] Updated weights for policy 0, policy_version 1257647 (0.0007) [2023-12-27 00:32:41,036][105620] Updated weights for policy 1, policy_version 1258789 (0.0008) [2023-12-27 00:32:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 644300800. Throughput: 0: 9866.5, 1: 9567.7. Samples: 644311224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:32:41,063][104569] Avg episode reward: [(0, '9262.104'), (1, '9081.070')] [2023-12-27 00:32:41,101][105620] Updated weights for policy 1, policy_version 1258799 (0.0008) [2023-12-27 00:32:41,173][105620] Updated weights for policy 1, policy_version 1258809 (0.0008) [2023-12-27 00:32:41,579][105692] Updated weights for policy 0, policy_version 1257657 (0.0009) [2023-12-27 00:32:41,641][105692] Updated weights for policy 0, policy_version 1257667 (0.0007) [2023-12-27 00:32:41,707][105692] Updated weights for policy 0, policy_version 1257677 (0.0007) [2023-12-27 00:32:41,916][105620] Updated weights for policy 1, policy_version 1258819 (0.0007) [2023-12-27 00:32:41,979][105620] Updated weights for policy 1, policy_version 1258829 (0.0006) [2023-12-27 00:32:42,039][105620] Updated weights for policy 1, policy_version 1258839 (0.0007) [2023-12-27 00:32:42,394][105692] Updated weights for policy 0, policy_version 1257687 (0.0008) [2023-12-27 00:32:42,446][105692] Updated weights for policy 0, policy_version 1257697 (0.0006) [2023-12-27 00:32:42,500][105692] Updated weights for policy 0, policy_version 1257707 (0.0007) [2023-12-27 00:32:42,771][105620] Updated weights for policy 1, policy_version 1258849 (0.0010) [2023-12-27 00:32:42,824][105620] Updated weights for policy 1, policy_version 1258859 (0.0009) [2023-12-27 00:32:42,880][105620] Updated weights for policy 1, policy_version 1258869 (0.0009) [2023-12-27 00:32:42,933][105620] Updated weights for policy 1, policy_version 1258879 (0.0009) [2023-12-27 00:32:43,266][105692] Updated weights for policy 0, policy_version 1257717 (0.0009) [2023-12-27 00:32:43,327][105692] Updated weights for policy 0, policy_version 1257727 (0.0009) [2023-12-27 00:32:43,387][105692] Updated weights for policy 0, policy_version 1257737 (0.0009) [2023-12-27 00:32:43,698][105620] Updated weights for policy 1, policy_version 1258889 (0.0010) [2023-12-27 00:32:43,760][105620] Updated weights for policy 1, policy_version 1258899 (0.0010) [2023-12-27 00:32:43,812][105620] Updated weights for policy 1, policy_version 1258909 (0.0009) [2023-12-27 00:32:44,089][105692] Updated weights for policy 0, policy_version 1257747 (0.0008) [2023-12-27 00:32:44,146][105692] Updated weights for policy 0, policy_version 1257757 (0.0005) [2023-12-27 00:32:44,195][105692] Updated weights for policy 0, policy_version 1257767 (0.0005) [2023-12-27 00:32:44,571][105620] Updated weights for policy 1, policy_version 1258919 (0.0008) [2023-12-27 00:32:44,622][105620] Updated weights for policy 1, policy_version 1258929 (0.0009) [2023-12-27 00:32:44,677][105620] Updated weights for policy 1, policy_version 1258939 (0.0008) [2023-12-27 00:32:44,768][105692] Updated weights for policy 0, policy_version 1257777 (0.0006) [2023-12-27 00:32:44,824][105692] Updated weights for policy 0, policy_version 1257787 (0.0009) [2023-12-27 00:32:44,884][105692] Updated weights for policy 0, policy_version 1257797 (0.0009) [2023-12-27 00:32:44,944][105692] Updated weights for policy 0, policy_version 1257807 (0.0009) [2023-12-27 00:32:45,439][105620] Updated weights for policy 1, policy_version 1258949 (0.0009) [2023-12-27 00:32:45,495][105620] Updated weights for policy 1, policy_version 1258959 (0.0009) [2023-12-27 00:32:45,560][105620] Updated weights for policy 1, policy_version 1258969 (0.0009) [2023-12-27 00:32:45,727][105692] Updated weights for policy 0, policy_version 1257817 (0.0009) [2023-12-27 00:32:45,781][105692] Updated weights for policy 0, policy_version 1257827 (0.0009) [2023-12-27 00:32:45,832][105692] Updated weights for policy 0, policy_version 1257837 (0.0009) [2023-12-27 00:32:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19387.6, 300 sec: 19521.9). Total num frames: 644399104. Throughput: 0: 9818.2, 1: 9534.0. Samples: 644367612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:32:46,063][104569] Avg episode reward: [(0, '9170.938'), (1, '8904.707')] [2023-12-27 00:32:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001258976_322338816.pth... [2023-12-27 00:32:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001257840_322060288.pth... [2023-12-27 00:32:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001256688_321765376.pth [2023-12-27 00:32:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001257856_322052096.pth [2023-12-27 00:32:46,262][105620] Updated weights for policy 1, policy_version 1258979 (0.0009) [2023-12-27 00:32:46,321][105620] Updated weights for policy 1, policy_version 1258989 (0.0008) [2023-12-27 00:32:46,386][105620] Updated weights for policy 1, policy_version 1258999 (0.0009) [2023-12-27 00:32:46,611][105692] Updated weights for policy 0, policy_version 1257847 (0.0009) [2023-12-27 00:32:46,662][105692] Updated weights for policy 0, policy_version 1257857 (0.0009) [2023-12-27 00:32:46,720][105692] Updated weights for policy 0, policy_version 1257867 (0.0009) [2023-12-27 00:32:47,132][105620] Updated weights for policy 1, policy_version 1259009 (0.0009) [2023-12-27 00:32:47,186][105620] Updated weights for policy 1, policy_version 1259019 (0.0009) [2023-12-27 00:32:47,237][105620] Updated weights for policy 1, policy_version 1259029 (0.0009) [2023-12-27 00:32:47,286][105620] Updated weights for policy 1, policy_version 1259039 (0.0009) [2023-12-27 00:32:47,485][105692] Updated weights for policy 0, policy_version 1257877 (0.0009) [2023-12-27 00:32:47,537][105692] Updated weights for policy 0, policy_version 1257887 (0.0009) [2023-12-27 00:32:47,588][105692] Updated weights for policy 0, policy_version 1257897 (0.0009) [2023-12-27 00:32:47,979][105620] Updated weights for policy 1, policy_version 1259049 (0.0010) [2023-12-27 00:32:48,028][105620] Updated weights for policy 1, policy_version 1259059 (0.0010) [2023-12-27 00:32:48,075][105620] Updated weights for policy 1, policy_version 1259069 (0.0010) [2023-12-27 00:32:48,431][105692] Updated weights for policy 0, policy_version 1257907 (0.0008) [2023-12-27 00:32:48,491][105692] Updated weights for policy 0, policy_version 1257917 (0.0008) [2023-12-27 00:32:48,546][105692] Updated weights for policy 0, policy_version 1257927 (0.0008) [2023-12-27 00:32:48,783][105620] Updated weights for policy 1, policy_version 1259079 (0.0009) [2023-12-27 00:32:48,841][105620] Updated weights for policy 1, policy_version 1259089 (0.0010) [2023-12-27 00:32:48,907][105620] Updated weights for policy 1, policy_version 1259099 (0.0010) [2023-12-27 00:32:49,328][105692] Updated weights for policy 0, policy_version 1257937 (0.0008) [2023-12-27 00:32:49,397][105692] Updated weights for policy 0, policy_version 1257947 (0.0009) [2023-12-27 00:32:49,428][105585] KL-divergence is very high: 144.2697 [2023-12-27 00:32:49,456][105692] Updated weights for policy 0, policy_version 1257957 (0.0008) [2023-12-27 00:32:49,476][105585] KL-divergence is very high: 155.2130 [2023-12-27 00:32:49,515][105692] Updated weights for policy 0, policy_version 1257967 (0.0008) [2023-12-27 00:32:49,649][105620] Updated weights for policy 1, policy_version 1259109 (0.0011) [2023-12-27 00:32:49,715][105620] Updated weights for policy 1, policy_version 1259119 (0.0011) [2023-12-27 00:32:49,775][105620] Updated weights for policy 1, policy_version 1259129 (0.0010) [2023-12-27 00:32:50,296][105692] Updated weights for policy 0, policy_version 1257977 (0.0009) [2023-12-27 00:32:50,359][105692] Updated weights for policy 0, policy_version 1257987 (0.0009) [2023-12-27 00:32:50,419][105692] Updated weights for policy 0, policy_version 1257997 (0.0009) [2023-12-27 00:32:50,500][105620] Updated weights for policy 1, policy_version 1259139 (0.0008) [2023-12-27 00:32:50,562][105620] Updated weights for policy 1, policy_version 1259149 (0.0007) [2023-12-27 00:32:50,624][105620] Updated weights for policy 1, policy_version 1259159 (0.0010) [2023-12-27 00:32:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 644489216. Throughput: 0: 9748.2, 1: 9635.4. Samples: 644481596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:32:51,063][104569] Avg episode reward: [(0, '9168.712'), (1, '8911.546')] [2023-12-27 00:32:51,228][105692] Updated weights for policy 0, policy_version 1258007 (0.0010) [2023-12-27 00:32:51,248][105620] Updated weights for policy 1, policy_version 1259169 (0.0009) [2023-12-27 00:32:51,291][105692] Updated weights for policy 0, policy_version 1258017 (0.0008) [2023-12-27 00:32:51,304][105620] Updated weights for policy 1, policy_version 1259179 (0.0007) [2023-12-27 00:32:51,356][105620] Updated weights for policy 1, policy_version 1259189 (0.0008) [2023-12-27 00:32:51,356][105692] Updated weights for policy 0, policy_version 1258027 (0.0007) [2023-12-27 00:32:51,416][105620] Updated weights for policy 1, policy_version 1259199 (0.0009) [2023-12-27 00:32:52,118][105620] Updated weights for policy 1, policy_version 1259209 (0.0009) [2023-12-27 00:32:52,135][105692] Updated weights for policy 0, policy_version 1258037 (0.0008) [2023-12-27 00:32:52,178][105620] Updated weights for policy 1, policy_version 1259219 (0.0007) [2023-12-27 00:32:52,191][105692] Updated weights for policy 0, policy_version 1258047 (0.0006) [2023-12-27 00:32:52,233][105620] Updated weights for policy 1, policy_version 1259229 (0.0008) [2023-12-27 00:32:52,248][105692] Updated weights for policy 0, policy_version 1258057 (0.0008) [2023-12-27 00:32:52,969][105620] Updated weights for policy 1, policy_version 1259239 (0.0009) [2023-12-27 00:32:53,030][105620] Updated weights for policy 1, policy_version 1259249 (0.0009) [2023-12-27 00:32:53,043][105692] Updated weights for policy 0, policy_version 1258067 (0.0008) [2023-12-27 00:32:53,086][105620] Updated weights for policy 1, policy_version 1259259 (0.0005) [2023-12-27 00:32:53,093][105692] Updated weights for policy 0, policy_version 1258077 (0.0009) [2023-12-27 00:32:53,144][105692] Updated weights for policy 0, policy_version 1258087 (0.0008) [2023-12-27 00:32:53,698][105620] Updated weights for policy 1, policy_version 1259269 (0.0007) [2023-12-27 00:32:53,747][105620] Updated weights for policy 1, policy_version 1259279 (0.0009) [2023-12-27 00:32:53,799][105620] Updated weights for policy 1, policy_version 1259289 (0.0010) [2023-12-27 00:32:53,993][105692] Updated weights for policy 0, policy_version 1258097 (0.0010) [2023-12-27 00:32:54,046][105692] Updated weights for policy 0, policy_version 1258107 (0.0008) [2023-12-27 00:32:54,106][105692] Updated weights for policy 0, policy_version 1258117 (0.0008) [2023-12-27 00:32:54,165][105692] Updated weights for policy 0, policy_version 1258127 (0.0008) [2023-12-27 00:32:54,563][105620] Updated weights for policy 1, policy_version 1259299 (0.0010) [2023-12-27 00:32:54,617][105620] Updated weights for policy 1, policy_version 1259309 (0.0006) [2023-12-27 00:32:54,673][105620] Updated weights for policy 1, policy_version 1259319 (0.0006) [2023-12-27 00:32:54,837][105692] Updated weights for policy 0, policy_version 1258137 (0.0009) [2023-12-27 00:32:54,896][105692] Updated weights for policy 0, policy_version 1258147 (0.0008) [2023-12-27 00:32:54,960][105692] Updated weights for policy 0, policy_version 1258157 (0.0009) [2023-12-27 00:32:55,354][105620] Updated weights for policy 1, policy_version 1259329 (0.0006) [2023-12-27 00:32:55,411][105620] Updated weights for policy 1, policy_version 1259339 (0.0010) [2023-12-27 00:32:55,476][105620] Updated weights for policy 1, policy_version 1259349 (0.0010) [2023-12-27 00:32:55,533][105620] Updated weights for policy 1, policy_version 1259359 (0.0010) [2023-12-27 00:32:55,708][105692] Updated weights for policy 0, policy_version 1258167 (0.0009) [2023-12-27 00:32:55,756][105692] Updated weights for policy 0, policy_version 1258177 (0.0008) [2023-12-27 00:32:55,820][105692] Updated weights for policy 0, policy_version 1258187 (0.0008) [2023-12-27 00:32:56,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 644587520. Throughput: 0: 9628.1, 1: 9662.6. Samples: 644595940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:32:56,063][104569] Avg episode reward: [(0, '8986.195'), (1, '8905.993')] [2023-12-27 00:32:56,254][105620] Updated weights for policy 1, policy_version 1259369 (0.0011) [2023-12-27 00:32:56,309][105620] Updated weights for policy 1, policy_version 1259379 (0.0010) [2023-12-27 00:32:56,361][105620] Updated weights for policy 1, policy_version 1259389 (0.0011) [2023-12-27 00:32:56,601][105692] Updated weights for policy 0, policy_version 1258197 (0.0008) [2023-12-27 00:32:56,651][105692] Updated weights for policy 0, policy_version 1258207 (0.0008) [2023-12-27 00:32:56,695][105692] Updated weights for policy 0, policy_version 1258217 (0.0008) [2023-12-27 00:32:57,119][105620] Updated weights for policy 1, policy_version 1259399 (0.0010) [2023-12-27 00:32:57,169][105620] Updated weights for policy 1, policy_version 1259409 (0.0010) [2023-12-27 00:32:57,224][105620] Updated weights for policy 1, policy_version 1259419 (0.0010) [2023-12-27 00:32:57,490][105692] Updated weights for policy 0, policy_version 1258227 (0.0009) [2023-12-27 00:32:57,537][105692] Updated weights for policy 0, policy_version 1258238 (0.0009) [2023-12-27 00:32:57,584][105692] Updated weights for policy 0, policy_version 1258249 (0.0008) [2023-12-27 00:32:57,882][105620] Updated weights for policy 1, policy_version 1259429 (0.0010) [2023-12-27 00:32:57,932][105620] Updated weights for policy 1, policy_version 1259439 (0.0009) [2023-12-27 00:32:57,983][105620] Updated weights for policy 1, policy_version 1259449 (0.0009) [2023-12-27 00:32:58,397][105692] Updated weights for policy 0, policy_version 1258259 (0.0009) [2023-12-27 00:32:58,457][105692] Updated weights for policy 0, policy_version 1258269 (0.0011) [2023-12-27 00:32:58,516][105692] Updated weights for policy 0, policy_version 1258279 (0.0008) [2023-12-27 00:32:58,721][105620] Updated weights for policy 1, policy_version 1259459 (0.0007) [2023-12-27 00:32:58,785][105620] Updated weights for policy 1, policy_version 1259469 (0.0007) [2023-12-27 00:32:58,855][105620] Updated weights for policy 1, policy_version 1259479 (0.0009) [2023-12-27 00:32:59,268][105692] Updated weights for policy 0, policy_version 1258289 (0.0006) [2023-12-27 00:32:59,332][105692] Updated weights for policy 0, policy_version 1258299 (0.0008) [2023-12-27 00:32:59,395][105692] Updated weights for policy 0, policy_version 1258309 (0.0009) [2023-12-27 00:32:59,458][105692] Updated weights for policy 0, policy_version 1258319 (0.0008) [2023-12-27 00:32:59,606][105620] Updated weights for policy 1, policy_version 1259489 (0.0007) [2023-12-27 00:32:59,664][105620] Updated weights for policy 1, policy_version 1259499 (0.0009) [2023-12-27 00:32:59,714][105620] Updated weights for policy 1, policy_version 1259509 (0.0009) [2023-12-27 00:32:59,767][105620] Updated weights for policy 1, policy_version 1259519 (0.0009) [2023-12-27 00:33:00,167][105692] Updated weights for policy 0, policy_version 1258329 (0.0007) [2023-12-27 00:33:00,228][105692] Updated weights for policy 0, policy_version 1258339 (0.0005) [2023-12-27 00:33:00,274][105692] Updated weights for policy 0, policy_version 1258349 (0.0005) [2023-12-27 00:33:00,521][105620] Updated weights for policy 1, policy_version 1259529 (0.0008) [2023-12-27 00:33:00,586][105620] Updated weights for policy 1, policy_version 1259539 (0.0010) [2023-12-27 00:33:00,655][105620] Updated weights for policy 1, policy_version 1259549 (0.0011) [2023-12-27 00:33:00,823][105692] Updated weights for policy 0, policy_version 1258359 (0.0009) [2023-12-27 00:33:00,876][105692] Updated weights for policy 0, policy_version 1258370 (0.0010) [2023-12-27 00:33:00,938][105692] Updated weights for policy 0, policy_version 1258380 (0.0009) [2023-12-27 00:33:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 644685824. Throughput: 0: 9591.2, 1: 9680.6. Samples: 644652000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:33:01,062][104569] Avg episode reward: [(0, '8713.023'), (1, '8899.870')] [2023-12-27 00:33:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001258384_322199552.pth... [2023-12-27 00:33:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001259552_322486272.pth... [2023-12-27 00:33:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001258432_322199552.pth [2023-12-27 00:33:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001257264_321912832.pth [2023-12-27 00:33:01,343][105620] Updated weights for policy 1, policy_version 1259559 (0.0011) [2023-12-27 00:33:01,404][105620] Updated weights for policy 1, policy_version 1259569 (0.0010) [2023-12-27 00:33:01,455][105620] Updated weights for policy 1, policy_version 1259579 (0.0010) [2023-12-27 00:33:01,754][105692] Updated weights for policy 0, policy_version 1258390 (0.0008) [2023-12-27 00:33:01,806][105692] Updated weights for policy 0, policy_version 1258400 (0.0008) [2023-12-27 00:33:01,857][105692] Updated weights for policy 0, policy_version 1258410 (0.0008) [2023-12-27 00:33:02,217][105620] Updated weights for policy 1, policy_version 1259589 (0.0010) [2023-12-27 00:33:02,279][105620] Updated weights for policy 1, policy_version 1259599 (0.0010) [2023-12-27 00:33:02,340][105620] Updated weights for policy 1, policy_version 1259609 (0.0010) [2023-12-27 00:33:02,637][105692] Updated weights for policy 0, policy_version 1258420 (0.0009) [2023-12-27 00:33:02,705][105692] Updated weights for policy 0, policy_version 1258430 (0.0009) [2023-12-27 00:33:02,772][105692] Updated weights for policy 0, policy_version 1258440 (0.0010) [2023-12-27 00:33:02,980][105620] Updated weights for policy 1, policy_version 1259619 (0.0008) [2023-12-27 00:33:03,027][105620] Updated weights for policy 1, policy_version 1259629 (0.0008) [2023-12-27 00:33:03,078][105620] Updated weights for policy 1, policy_version 1259639 (0.0008) [2023-12-27 00:33:03,423][105692] Updated weights for policy 0, policy_version 1258450 (0.0010) [2023-12-27 00:33:03,478][105692] Updated weights for policy 0, policy_version 1258460 (0.0005) [2023-12-27 00:33:03,536][105692] Updated weights for policy 0, policy_version 1258470 (0.0005) [2023-12-27 00:33:03,601][105692] Updated weights for policy 0, policy_version 1258480 (0.0007) [2023-12-27 00:33:03,930][105620] Updated weights for policy 1, policy_version 1259649 (0.0009) [2023-12-27 00:33:03,985][105620] Updated weights for policy 1, policy_version 1259659 (0.0008) [2023-12-27 00:33:04,033][105620] Updated weights for policy 1, policy_version 1259669 (0.0008) [2023-12-27 00:33:04,093][105620] Updated weights for policy 1, policy_version 1259679 (0.0008) [2023-12-27 00:33:04,244][105692] Updated weights for policy 0, policy_version 1258490 (0.0011) [2023-12-27 00:33:04,293][105692] Updated weights for policy 0, policy_version 1258500 (0.0011) [2023-12-27 00:33:04,350][105692] Updated weights for policy 0, policy_version 1258510 (0.0011) [2023-12-27 00:33:04,790][105620] Updated weights for policy 1, policy_version 1259689 (0.0008) [2023-12-27 00:33:04,849][105620] Updated weights for policy 1, policy_version 1259699 (0.0008) [2023-12-27 00:33:04,905][105620] Updated weights for policy 1, policy_version 1259709 (0.0008) [2023-12-27 00:33:05,112][105692] Updated weights for policy 0, policy_version 1258520 (0.0011) [2023-12-27 00:33:05,170][105692] Updated weights for policy 0, policy_version 1258530 (0.0010) [2023-12-27 00:33:05,229][105692] Updated weights for policy 0, policy_version 1258540 (0.0010) [2023-12-27 00:33:05,569][105620] Updated weights for policy 1, policy_version 1259719 (0.0008) [2023-12-27 00:33:05,629][105620] Updated weights for policy 1, policy_version 1259729 (0.0008) [2023-12-27 00:33:05,684][105620] Updated weights for policy 1, policy_version 1259739 (0.0006) [2023-12-27 00:33:05,916][105692] Updated weights for policy 0, policy_version 1258550 (0.0009) [2023-12-27 00:33:05,989][105692] Updated weights for policy 0, policy_version 1258560 (0.0007) [2023-12-27 00:33:06,058][105692] Updated weights for policy 0, policy_version 1258570 (0.0009) [2023-12-27 00:33:06,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 644775936. Throughput: 0: 9604.2, 1: 9633.3. Samples: 644767512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:33:06,062][104569] Avg episode reward: [(0, '8713.662'), (1, '8899.454')] [2023-12-27 00:33:06,330][105620] Updated weights for policy 1, policy_version 1259749 (0.0008) [2023-12-27 00:33:06,386][105620] Updated weights for policy 1, policy_version 1259759 (0.0011) [2023-12-27 00:33:06,445][105620] Updated weights for policy 1, policy_version 1259769 (0.0011) [2023-12-27 00:33:06,717][105692] Updated weights for policy 0, policy_version 1258580 (0.0007) [2023-12-27 00:33:06,783][105692] Updated weights for policy 0, policy_version 1258590 (0.0005) [2023-12-27 00:33:06,844][105692] Updated weights for policy 0, policy_version 1258600 (0.0005) [2023-12-27 00:33:07,200][105620] Updated weights for policy 1, policy_version 1259779 (0.0011) [2023-12-27 00:33:07,258][105620] Updated weights for policy 1, policy_version 1259789 (0.0008) [2023-12-27 00:33:07,319][105620] Updated weights for policy 1, policy_version 1259799 (0.0006) [2023-12-27 00:33:07,517][105692] Updated weights for policy 0, policy_version 1258610 (0.0006) [2023-12-27 00:33:07,569][105692] Updated weights for policy 0, policy_version 1258620 (0.0010) [2023-12-27 00:33:07,618][105692] Updated weights for policy 0, policy_version 1258630 (0.0010) [2023-12-27 00:33:07,663][105692] Updated weights for policy 0, policy_version 1258640 (0.0008) [2023-12-27 00:33:07,969][105620] Updated weights for policy 1, policy_version 1259809 (0.0011) [2023-12-27 00:33:08,020][105620] Updated weights for policy 1, policy_version 1259819 (0.0010) [2023-12-27 00:33:08,081][105620] Updated weights for policy 1, policy_version 1259829 (0.0010) [2023-12-27 00:33:08,143][105620] Updated weights for policy 1, policy_version 1259839 (0.0010) [2023-12-27 00:33:08,465][105692] Updated weights for policy 0, policy_version 1258650 (0.0008) [2023-12-27 00:33:08,511][105692] Updated weights for policy 0, policy_version 1258660 (0.0008) [2023-12-27 00:33:08,567][105692] Updated weights for policy 0, policy_version 1258670 (0.0008) [2023-12-27 00:33:08,913][105620] Updated weights for policy 1, policy_version 1259849 (0.0010) [2023-12-27 00:33:08,967][105620] Updated weights for policy 1, policy_version 1259859 (0.0010) [2023-12-27 00:33:09,012][105620] Updated weights for policy 1, policy_version 1259869 (0.0010) [2023-12-27 00:33:09,381][105692] Updated weights for policy 0, policy_version 1258680 (0.0009) [2023-12-27 00:33:09,447][105692] Updated weights for policy 0, policy_version 1258690 (0.0009) [2023-12-27 00:33:09,507][105692] Updated weights for policy 0, policy_version 1258700 (0.0008) [2023-12-27 00:33:09,742][105620] Updated weights for policy 1, policy_version 1259879 (0.0010) [2023-12-27 00:33:09,805][105620] Updated weights for policy 1, policy_version 1259889 (0.0010) [2023-12-27 00:33:09,871][105620] Updated weights for policy 1, policy_version 1259899 (0.0010) [2023-12-27 00:33:10,308][105692] Updated weights for policy 0, policy_version 1258710 (0.0008) [2023-12-27 00:33:10,372][105692] Updated weights for policy 0, policy_version 1258720 (0.0009) [2023-12-27 00:33:10,427][105692] Updated weights for policy 0, policy_version 1258730 (0.0008) [2023-12-27 00:33:10,641][105620] Updated weights for policy 1, policy_version 1259909 (0.0009) [2023-12-27 00:33:10,706][105620] Updated weights for policy 1, policy_version 1259919 (0.0008) [2023-12-27 00:33:10,772][105620] Updated weights for policy 1, policy_version 1259929 (0.0006) [2023-12-27 00:33:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 644874240. Throughput: 0: 9609.0, 1: 9681.0. Samples: 644882868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:33:11,062][104569] Avg episode reward: [(0, '8806.038'), (1, '8905.362')] [2023-12-27 00:33:11,220][105692] Updated weights for policy 0, policy_version 1258740 (0.0009) [2023-12-27 00:33:11,289][105692] Updated weights for policy 0, policy_version 1258750 (0.0010) [2023-12-27 00:33:11,358][105692] Updated weights for policy 0, policy_version 1258760 (0.0008) [2023-12-27 00:33:11,458][105620] Updated weights for policy 1, policy_version 1259939 (0.0006) [2023-12-27 00:33:11,516][105620] Updated weights for policy 1, policy_version 1259949 (0.0009) [2023-12-27 00:33:11,576][105620] Updated weights for policy 1, policy_version 1259959 (0.0009) [2023-12-27 00:33:12,113][105692] Updated weights for policy 0, policy_version 1258770 (0.0008) [2023-12-27 00:33:12,171][105692] Updated weights for policy 0, policy_version 1258780 (0.0009) [2023-12-27 00:33:12,226][105692] Updated weights for policy 0, policy_version 1258790 (0.0009) [2023-12-27 00:33:12,283][105692] Updated weights for policy 0, policy_version 1258800 (0.0010) [2023-12-27 00:33:12,372][105620] Updated weights for policy 1, policy_version 1259969 (0.0008) [2023-12-27 00:33:12,434][105620] Updated weights for policy 1, policy_version 1259979 (0.0006) [2023-12-27 00:33:12,493][105620] Updated weights for policy 1, policy_version 1259989 (0.0008) [2023-12-27 00:33:12,556][105620] Updated weights for policy 1, policy_version 1259999 (0.0009) [2023-12-27 00:33:13,074][105692] Updated weights for policy 0, policy_version 1258810 (0.0011) [2023-12-27 00:33:13,122][105692] Updated weights for policy 0, policy_version 1258820 (0.0010) [2023-12-27 00:33:13,184][105692] Updated weights for policy 0, policy_version 1258830 (0.0010) [2023-12-27 00:33:13,284][105620] Updated weights for policy 1, policy_version 1260009 (0.0008) [2023-12-27 00:33:13,336][105620] Updated weights for policy 1, policy_version 1260019 (0.0008) [2023-12-27 00:33:13,384][105620] Updated weights for policy 1, policy_version 1260029 (0.0008) [2023-12-27 00:33:13,931][105692] Updated weights for policy 0, policy_version 1258840 (0.0010) [2023-12-27 00:33:13,985][105692] Updated weights for policy 0, policy_version 1258850 (0.0010) [2023-12-27 00:33:14,043][105692] Updated weights for policy 0, policy_version 1258860 (0.0010) [2023-12-27 00:33:14,155][105620] Updated weights for policy 1, policy_version 1260039 (0.0008) [2023-12-27 00:33:14,217][105620] Updated weights for policy 1, policy_version 1260049 (0.0008) [2023-12-27 00:33:14,272][105620] Updated weights for policy 1, policy_version 1260059 (0.0008) [2023-12-27 00:33:14,799][105692] Updated weights for policy 0, policy_version 1258870 (0.0008) [2023-12-27 00:33:14,867][105692] Updated weights for policy 0, policy_version 1258880 (0.0006) [2023-12-27 00:33:14,929][105692] Updated weights for policy 0, policy_version 1258890 (0.0006) [2023-12-27 00:33:15,067][105620] Updated weights for policy 1, policy_version 1260069 (0.0009) [2023-12-27 00:33:15,129][105620] Updated weights for policy 1, policy_version 1260079 (0.0007) [2023-12-27 00:33:15,195][105620] Updated weights for policy 1, policy_version 1260089 (0.0010) [2023-12-27 00:33:15,518][105692] Updated weights for policy 0, policy_version 1258900 (0.0007) [2023-12-27 00:33:15,571][105692] Updated weights for policy 0, policy_version 1258910 (0.0008) [2023-12-27 00:33:15,625][105692] Updated weights for policy 0, policy_version 1258920 (0.0007) [2023-12-27 00:33:15,998][105620] Updated weights for policy 1, policy_version 1260099 (0.0008) [2023-12-27 00:33:16,059][105620] Updated weights for policy 1, policy_version 1260109 (0.0005) [2023-12-27 00:33:16,062][104569] Fps is (10 sec: 18840.7, 60 sec: 19114.5, 300 sec: 19466.4). Total num frames: 644964352. Throughput: 0: 9570.1, 1: 9537.3. Samples: 644937744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:33:16,063][104569] Avg episode reward: [(0, '8991.361'), (1, '9085.468')] [2023-12-27 00:33:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001258928_322338816.pth... [2023-12-27 00:33:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001257840_322060288.pth [2023-12-27 00:33:16,124][105620] Updated weights for policy 1, policy_version 1260119 (0.0006) [2023-12-27 00:33:16,171][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001260128_322633728.pth... [2023-12-27 00:33:16,174][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001258976_322338816.pth [2023-12-27 00:33:16,409][105692] Updated weights for policy 0, policy_version 1258930 (0.0007) [2023-12-27 00:33:16,467][105692] Updated weights for policy 0, policy_version 1258940 (0.0009) [2023-12-27 00:33:16,529][105692] Updated weights for policy 0, policy_version 1258950 (0.0009) [2023-12-27 00:33:16,587][105692] Updated weights for policy 0, policy_version 1258960 (0.0009) [2023-12-27 00:33:16,743][105620] Updated weights for policy 1, policy_version 1260129 (0.0007) [2023-12-27 00:33:16,798][105620] Updated weights for policy 1, policy_version 1260140 (0.0009) [2023-12-27 00:33:16,845][105620] Updated weights for policy 1, policy_version 1260150 (0.0008) [2023-12-27 00:33:16,899][105620] Updated weights for policy 1, policy_version 1260160 (0.0007) [2023-12-27 00:33:17,289][105692] Updated weights for policy 0, policy_version 1258970 (0.0009) [2023-12-27 00:33:17,359][105692] Updated weights for policy 0, policy_version 1258980 (0.0009) [2023-12-27 00:33:17,409][105692] Updated weights for policy 0, policy_version 1258990 (0.0009) [2023-12-27 00:33:17,589][105620] Updated weights for policy 1, policy_version 1260170 (0.0008) [2023-12-27 00:33:17,639][105620] Updated weights for policy 1, policy_version 1260180 (0.0009) [2023-12-27 00:33:17,691][105620] Updated weights for policy 1, policy_version 1260190 (0.0009) [2023-12-27 00:33:18,128][105692] Updated weights for policy 0, policy_version 1259000 (0.0008) [2023-12-27 00:33:18,185][105692] Updated weights for policy 0, policy_version 1259010 (0.0005) [2023-12-27 00:33:18,233][105692] Updated weights for policy 0, policy_version 1259020 (0.0005) [2023-12-27 00:33:18,371][105620] Updated weights for policy 1, policy_version 1260200 (0.0008) [2023-12-27 00:33:18,429][105620] Updated weights for policy 1, policy_version 1260210 (0.0008) [2023-12-27 00:33:18,479][105620] Updated weights for policy 1, policy_version 1260220 (0.0006) [2023-12-27 00:33:18,999][105692] Updated weights for policy 0, policy_version 1259030 (0.0009) [2023-12-27 00:33:19,051][105692] Updated weights for policy 0, policy_version 1259040 (0.0010) [2023-12-27 00:33:19,105][105692] Updated weights for policy 0, policy_version 1259050 (0.0010) [2023-12-27 00:33:19,116][105620] Updated weights for policy 1, policy_version 1260230 (0.0006) [2023-12-27 00:33:19,177][105620] Updated weights for policy 1, policy_version 1260240 (0.0007) [2023-12-27 00:33:19,241][105620] Updated weights for policy 1, policy_version 1260250 (0.0008) [2023-12-27 00:33:19,844][105692] Updated weights for policy 0, policy_version 1259060 (0.0010) [2023-12-27 00:33:19,907][105692] Updated weights for policy 0, policy_version 1259070 (0.0009) [2023-12-27 00:33:19,970][105692] Updated weights for policy 0, policy_version 1259080 (0.0009) [2023-12-27 00:33:20,037][105620] Updated weights for policy 1, policy_version 1260260 (0.0009) [2023-12-27 00:33:20,093][105620] Updated weights for policy 1, policy_version 1260270 (0.0008) [2023-12-27 00:33:20,149][105620] Updated weights for policy 1, policy_version 1260280 (0.0010) [2023-12-27 00:33:20,753][105692] Updated weights for policy 0, policy_version 1259090 (0.0009) [2023-12-27 00:33:20,820][105692] Updated weights for policy 0, policy_version 1259100 (0.0009) [2023-12-27 00:33:20,883][105692] Updated weights for policy 0, policy_version 1259110 (0.0009) [2023-12-27 00:33:20,943][105692] Updated weights for policy 0, policy_version 1259120 (0.0009) [2023-12-27 00:33:20,956][105620] Updated weights for policy 1, policy_version 1260290 (0.0009) [2023-12-27 00:33:21,006][105620] Updated weights for policy 1, policy_version 1260300 (0.0009) [2023-12-27 00:33:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 645062656. Throughput: 0: 9574.1, 1: 9607.9. Samples: 645053928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:33:21,063][104569] Avg episode reward: [(0, '8993.655'), (1, '8988.009')] [2023-12-27 00:33:21,072][105620] Updated weights for policy 1, policy_version 1260310 (0.0009) [2023-12-27 00:33:21,132][105620] Updated weights for policy 1, policy_version 1260320 (0.0009) [2023-12-27 00:33:21,766][105692] Updated weights for policy 0, policy_version 1259130 (0.0009) [2023-12-27 00:33:21,830][105692] Updated weights for policy 0, policy_version 1259140 (0.0007) [2023-12-27 00:33:21,848][105620] Updated weights for policy 1, policy_version 1260330 (0.0008) [2023-12-27 00:33:21,887][105692] Updated weights for policy 0, policy_version 1259150 (0.0008) [2023-12-27 00:33:21,909][105620] Updated weights for policy 1, policy_version 1260340 (0.0007) [2023-12-27 00:33:21,964][105620] Updated weights for policy 1, policy_version 1260350 (0.0008) [2023-12-27 00:33:22,613][105692] Updated weights for policy 0, policy_version 1259160 (0.0006) [2023-12-27 00:33:22,674][105692] Updated weights for policy 0, policy_version 1259170 (0.0005) [2023-12-27 00:33:22,737][105692] Updated weights for policy 0, policy_version 1259180 (0.0006) [2023-12-27 00:33:22,751][105620] Updated weights for policy 1, policy_version 1260360 (0.0007) [2023-12-27 00:33:22,807][105620] Updated weights for policy 1, policy_version 1260370 (0.0009) [2023-12-27 00:33:22,865][105620] Updated weights for policy 1, policy_version 1260380 (0.0009) [2023-12-27 00:33:23,283][105692] Updated weights for policy 0, policy_version 1259190 (0.0006) [2023-12-27 00:33:23,336][105692] Updated weights for policy 0, policy_version 1259200 (0.0006) [2023-12-27 00:33:23,390][105692] Updated weights for policy 0, policy_version 1259210 (0.0009) [2023-12-27 00:33:23,613][105620] Updated weights for policy 1, policy_version 1260390 (0.0011) [2023-12-27 00:33:23,676][105620] Updated weights for policy 1, policy_version 1260400 (0.0010) [2023-12-27 00:33:23,745][105620] Updated weights for policy 1, policy_version 1260410 (0.0009) [2023-12-27 00:33:24,097][105692] Updated weights for policy 0, policy_version 1259220 (0.0010) [2023-12-27 00:33:24,160][105692] Updated weights for policy 0, policy_version 1259230 (0.0008) [2023-12-27 00:33:24,228][105692] Updated weights for policy 0, policy_version 1259240 (0.0006) [2023-12-27 00:33:24,327][105620] Updated weights for policy 1, policy_version 1260420 (0.0008) [2023-12-27 00:33:24,388][105620] Updated weights for policy 1, policy_version 1260430 (0.0007) [2023-12-27 00:33:24,451][105620] Updated weights for policy 1, policy_version 1260440 (0.0011) [2023-12-27 00:33:24,823][105692] Updated weights for policy 0, policy_version 1259250 (0.0008) [2023-12-27 00:33:24,881][105692] Updated weights for policy 0, policy_version 1259260 (0.0005) [2023-12-27 00:33:24,937][105692] Updated weights for policy 0, policy_version 1259270 (0.0005) [2023-12-27 00:33:24,993][105692] Updated weights for policy 0, policy_version 1259280 (0.0005) [2023-12-27 00:33:25,192][105620] Updated weights for policy 1, policy_version 1260450 (0.0010) [2023-12-27 00:33:25,240][105620] Updated weights for policy 1, policy_version 1260460 (0.0010) [2023-12-27 00:33:25,295][105620] Updated weights for policy 1, policy_version 1260470 (0.0010) [2023-12-27 00:33:25,354][105620] Updated weights for policy 1, policy_version 1260480 (0.0011) [2023-12-27 00:33:25,662][105692] Updated weights for policy 0, policy_version 1259290 (0.0010) [2023-12-27 00:33:25,726][105692] Updated weights for policy 0, policy_version 1259300 (0.0010) [2023-12-27 00:33:25,787][105692] Updated weights for policy 0, policy_version 1259310 (0.0010) [2023-12-27 00:33:26,062][104569] Fps is (10 sec: 19661.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 645160960. Throughput: 0: 9471.4, 1: 9632.0. Samples: 645170876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:33:26,063][104569] Avg episode reward: [(0, '9083.189'), (1, '8896.621')] [2023-12-27 00:33:26,118][105620] Updated weights for policy 1, policy_version 1260490 (0.0010) [2023-12-27 00:33:26,187][105620] Updated weights for policy 1, policy_version 1260500 (0.0011) [2023-12-27 00:33:26,250][105620] Updated weights for policy 1, policy_version 1260510 (0.0011) [2023-12-27 00:33:26,469][105692] Updated weights for policy 0, policy_version 1259320 (0.0006) [2023-12-27 00:33:26,517][105692] Updated weights for policy 0, policy_version 1259330 (0.0005) [2023-12-27 00:33:26,563][105692] Updated weights for policy 0, policy_version 1259340 (0.0005) [2023-12-27 00:33:26,974][105620] Updated weights for policy 1, policy_version 1260520 (0.0010) [2023-12-27 00:33:27,029][105620] Updated weights for policy 1, policy_version 1260530 (0.0010) [2023-12-27 00:33:27,084][105620] Updated weights for policy 1, policy_version 1260540 (0.0010) [2023-12-27 00:33:27,254][105692] Updated weights for policy 0, policy_version 1259350 (0.0008) [2023-12-27 00:33:27,309][105692] Updated weights for policy 0, policy_version 1259360 (0.0010) [2023-12-27 00:33:27,353][105692] Updated weights for policy 0, policy_version 1259370 (0.0010) [2023-12-27 00:33:27,830][105620] Updated weights for policy 1, policy_version 1260550 (0.0010) [2023-12-27 00:33:27,890][105620] Updated weights for policy 1, policy_version 1260560 (0.0010) [2023-12-27 00:33:27,954][105620] Updated weights for policy 1, policy_version 1260570 (0.0008) [2023-12-27 00:33:28,017][105692] Updated weights for policy 0, policy_version 1259380 (0.0010) [2023-12-27 00:33:28,071][105692] Updated weights for policy 0, policy_version 1259390 (0.0008) [2023-12-27 00:33:28,125][105692] Updated weights for policy 0, policy_version 1259400 (0.0007) [2023-12-27 00:33:28,679][105620] Updated weights for policy 1, policy_version 1260580 (0.0010) [2023-12-27 00:33:28,730][105620] Updated weights for policy 1, policy_version 1260590 (0.0010) [2023-12-27 00:33:28,789][105620] Updated weights for policy 1, policy_version 1260600 (0.0010) [2023-12-27 00:33:28,890][105692] Updated weights for policy 0, policy_version 1259410 (0.0009) [2023-12-27 00:33:28,947][105692] Updated weights for policy 0, policy_version 1259420 (0.0007) [2023-12-27 00:33:29,009][105692] Updated weights for policy 0, policy_version 1259430 (0.0008) [2023-12-27 00:33:29,065][105692] Updated weights for policy 0, policy_version 1259440 (0.0010) [2023-12-27 00:33:29,457][105620] Updated weights for policy 1, policy_version 1260610 (0.0009) [2023-12-27 00:33:29,515][105620] Updated weights for policy 1, policy_version 1260620 (0.0009) [2023-12-27 00:33:29,570][105620] Updated weights for policy 1, policy_version 1260630 (0.0010) [2023-12-27 00:33:29,618][105620] Updated weights for policy 1, policy_version 1260640 (0.0010) [2023-12-27 00:33:29,812][105692] Updated weights for policy 0, policy_version 1259450 (0.0008) [2023-12-27 00:33:29,873][105692] Updated weights for policy 0, policy_version 1259460 (0.0008) [2023-12-27 00:33:29,941][105692] Updated weights for policy 0, policy_version 1259470 (0.0007) [2023-12-27 00:33:30,397][105620] Updated weights for policy 1, policy_version 1260650 (0.0011) [2023-12-27 00:33:30,463][105620] Updated weights for policy 1, policy_version 1260660 (0.0010) [2023-12-27 00:33:30,529][105620] Updated weights for policy 1, policy_version 1260670 (0.0011) [2023-12-27 00:33:30,534][105692] Updated weights for policy 0, policy_version 1259480 (0.0007) [2023-12-27 00:33:30,596][105692] Updated weights for policy 0, policy_version 1259490 (0.0005) [2023-12-27 00:33:30,656][105692] Updated weights for policy 0, policy_version 1259500 (0.0005) [2023-12-27 00:33:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 645259264. Throughput: 0: 9521.9, 1: 9625.0. Samples: 645229212. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:33:31,063][104569] Avg episode reward: [(0, '9082.740'), (1, '8990.479')] [2023-12-27 00:33:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001259504_322486272.pth... [2023-12-27 00:33:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001260672_322772992.pth... [2023-12-27 00:33:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001259552_322486272.pth [2023-12-27 00:33:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001258384_322199552.pth [2023-12-27 00:33:31,186][105692] Updated weights for policy 0, policy_version 1259510 (0.0008) [2023-12-27 00:33:31,197][105620] Updated weights for policy 1, policy_version 1260680 (0.0011) [2023-12-27 00:33:31,238][105692] Updated weights for policy 0, policy_version 1259520 (0.0010) [2023-12-27 00:33:31,260][105620] Updated weights for policy 1, policy_version 1260690 (0.0011) [2023-12-27 00:33:31,299][105692] Updated weights for policy 0, policy_version 1259530 (0.0006) [2023-12-27 00:33:31,321][105620] Updated weights for policy 1, policy_version 1260700 (0.0009) [2023-12-27 00:33:31,987][105620] Updated weights for policy 1, policy_version 1260710 (0.0007) [2023-12-27 00:33:32,006][105692] Updated weights for policy 0, policy_version 1259540 (0.0006) [2023-12-27 00:33:32,042][105620] Updated weights for policy 1, policy_version 1260720 (0.0006) [2023-12-27 00:33:32,065][105692] Updated weights for policy 0, policy_version 1259550 (0.0009) [2023-12-27 00:33:32,108][105620] Updated weights for policy 1, policy_version 1260730 (0.0007) [2023-12-27 00:33:32,126][105692] Updated weights for policy 0, policy_version 1259560 (0.0008) [2023-12-27 00:33:32,765][105620] Updated weights for policy 1, policy_version 1260740 (0.0006) [2023-12-27 00:33:32,813][105620] Updated weights for policy 1, policy_version 1260750 (0.0005) [2023-12-27 00:33:32,861][105620] Updated weights for policy 1, policy_version 1260760 (0.0005) [2023-12-27 00:33:32,874][105692] Updated weights for policy 0, policy_version 1259570 (0.0007) [2023-12-27 00:33:32,920][105692] Updated weights for policy 0, policy_version 1259580 (0.0006) [2023-12-27 00:33:32,975][105692] Updated weights for policy 0, policy_version 1259590 (0.0007) [2023-12-27 00:33:33,035][105692] Updated weights for policy 0, policy_version 1259600 (0.0009) [2023-12-27 00:33:33,403][105620] Updated weights for policy 1, policy_version 1260770 (0.0007) [2023-12-27 00:33:33,464][105620] Updated weights for policy 1, policy_version 1260780 (0.0010) [2023-12-27 00:33:33,522][105620] Updated weights for policy 1, policy_version 1260790 (0.0007) [2023-12-27 00:33:33,577][105620] Updated weights for policy 1, policy_version 1260800 (0.0008) [2023-12-27 00:33:33,631][105692] Updated weights for policy 0, policy_version 1259610 (0.0005) [2023-12-27 00:33:33,674][105692] Updated weights for policy 0, policy_version 1259620 (0.0005) [2023-12-27 00:33:33,717][105692] Updated weights for policy 0, policy_version 1259630 (0.0005) [2023-12-27 00:33:34,245][105620] Updated weights for policy 1, policy_version 1260810 (0.0010) [2023-12-27 00:33:34,314][105620] Updated weights for policy 1, policy_version 1260820 (0.0010) [2023-12-27 00:33:34,372][105692] Updated weights for policy 0, policy_version 1259640 (0.0005) [2023-12-27 00:33:34,377][105620] Updated weights for policy 1, policy_version 1260830 (0.0010) [2023-12-27 00:33:34,432][105692] Updated weights for policy 0, policy_version 1259650 (0.0007) [2023-12-27 00:33:34,492][105692] Updated weights for policy 0, policy_version 1259660 (0.0008) [2023-12-27 00:33:35,108][105620] Updated weights for policy 1, policy_version 1260840 (0.0006) [2023-12-27 00:33:35,162][105620] Updated weights for policy 1, policy_version 1260850 (0.0006) [2023-12-27 00:33:35,220][105692] Updated weights for policy 0, policy_version 1259670 (0.0010) [2023-12-27 00:33:35,222][105620] Updated weights for policy 1, policy_version 1260860 (0.0007) [2023-12-27 00:33:35,268][105692] Updated weights for policy 0, policy_version 1259680 (0.0010) [2023-12-27 00:33:35,319][105692] Updated weights for policy 0, policy_version 1259690 (0.0010) [2023-12-27 00:33:35,928][105620] Updated weights for policy 1, policy_version 1260870 (0.0008) [2023-12-27 00:33:35,998][105620] Updated weights for policy 1, policy_version 1260880 (0.0006) [2023-12-27 00:33:36,038][105692] Updated weights for policy 0, policy_version 1259700 (0.0011) [2023-12-27 00:33:36,049][105620] Updated weights for policy 1, policy_version 1260890 (0.0006) [2023-12-27 00:33:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.3, 300 sec: 19438.6). Total num frames: 645357568. Throughput: 0: 9655.1, 1: 9713.0. Samples: 645353160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:33:36,062][104569] Avg episode reward: [(0, '9085.046'), (1, '8901.595')] [2023-12-27 00:33:36,095][105692] Updated weights for policy 0, policy_version 1259710 (0.0011) [2023-12-27 00:33:36,155][105692] Updated weights for policy 0, policy_version 1259720 (0.0011) [2023-12-27 00:33:36,759][105620] Updated weights for policy 1, policy_version 1260900 (0.0006) [2023-12-27 00:33:36,830][105620] Updated weights for policy 1, policy_version 1260910 (0.0009) [2023-12-27 00:33:36,896][105620] Updated weights for policy 1, policy_version 1260920 (0.0008) [2023-12-27 00:33:36,920][105692] Updated weights for policy 0, policy_version 1259730 (0.0010) [2023-12-27 00:33:36,980][105692] Updated weights for policy 0, policy_version 1259740 (0.0011) [2023-12-27 00:33:37,039][105692] Updated weights for policy 0, policy_version 1259750 (0.0011) [2023-12-27 00:33:37,097][105692] Updated weights for policy 0, policy_version 1259760 (0.0011) [2023-12-27 00:33:37,501][105620] Updated weights for policy 1, policy_version 1260930 (0.0011) [2023-12-27 00:33:37,563][105620] Updated weights for policy 1, policy_version 1260940 (0.0011) [2023-12-27 00:33:37,612][105620] Updated weights for policy 1, policy_version 1260950 (0.0010) [2023-12-27 00:33:37,661][105620] Updated weights for policy 1, policy_version 1260960 (0.0009) [2023-12-27 00:33:37,777][105692] Updated weights for policy 0, policy_version 1259770 (0.0010) [2023-12-27 00:33:37,839][105692] Updated weights for policy 0, policy_version 1259780 (0.0011) [2023-12-27 00:33:37,898][105692] Updated weights for policy 0, policy_version 1259790 (0.0011) [2023-12-27 00:33:38,329][105620] Updated weights for policy 1, policy_version 1260970 (0.0007) [2023-12-27 00:33:38,393][105620] Updated weights for policy 1, policy_version 1260980 (0.0009) [2023-12-27 00:33:38,453][105620] Updated weights for policy 1, policy_version 1260990 (0.0010) [2023-12-27 00:33:38,626][105692] Updated weights for policy 0, policy_version 1259800 (0.0011) [2023-12-27 00:33:38,682][105692] Updated weights for policy 0, policy_version 1259810 (0.0009) [2023-12-27 00:33:38,730][105692] Updated weights for policy 0, policy_version 1259820 (0.0010) [2023-12-27 00:33:39,058][105620] Updated weights for policy 1, policy_version 1261000 (0.0010) [2023-12-27 00:33:39,124][105620] Updated weights for policy 1, policy_version 1261010 (0.0010) [2023-12-27 00:33:39,190][105620] Updated weights for policy 1, policy_version 1261020 (0.0011) [2023-12-27 00:33:39,445][105692] Updated weights for policy 0, policy_version 1259830 (0.0009) [2023-12-27 00:33:39,505][105692] Updated weights for policy 0, policy_version 1259840 (0.0006) [2023-12-27 00:33:39,564][105692] Updated weights for policy 0, policy_version 1259850 (0.0009) [2023-12-27 00:33:39,955][105620] Updated weights for policy 1, policy_version 1261030 (0.0010) [2023-12-27 00:33:40,016][105620] Updated weights for policy 1, policy_version 1261040 (0.0008) [2023-12-27 00:33:40,079][105620] Updated weights for policy 1, policy_version 1261050 (0.0009) [2023-12-27 00:33:40,160][105692] Updated weights for policy 0, policy_version 1259860 (0.0007) [2023-12-27 00:33:40,217][105692] Updated weights for policy 0, policy_version 1259870 (0.0005) [2023-12-27 00:33:40,273][105692] Updated weights for policy 0, policy_version 1259880 (0.0010) [2023-12-27 00:33:40,828][105692] Updated weights for policy 0, policy_version 1259890 (0.0007) [2023-12-27 00:33:40,834][105620] Updated weights for policy 1, policy_version 1261060 (0.0008) [2023-12-27 00:33:40,885][105692] Updated weights for policy 0, policy_version 1259900 (0.0005) [2023-12-27 00:33:40,888][105620] Updated weights for policy 1, policy_version 1261070 (0.0008) [2023-12-27 00:33:40,940][105692] Updated weights for policy 0, policy_version 1259910 (0.0005) [2023-12-27 00:33:40,950][105620] Updated weights for policy 1, policy_version 1261080 (0.0007) [2023-12-27 00:33:41,005][105692] Updated weights for policy 0, policy_version 1259920 (0.0006) [2023-12-27 00:33:41,062][104569] Fps is (10 sec: 21299.5, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 645472256. Throughput: 0: 9777.7, 1: 9722.7. Samples: 645473452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:33:41,062][104569] Avg episode reward: [(0, '9176.611'), (1, '9172.856')] [2023-12-27 00:33:41,746][105620] Updated weights for policy 1, policy_version 1261090 (0.0008) [2023-12-27 00:33:41,754][105692] Updated weights for policy 0, policy_version 1259930 (0.0008) [2023-12-27 00:33:41,807][105620] Updated weights for policy 1, policy_version 1261100 (0.0007) [2023-12-27 00:33:41,820][105692] Updated weights for policy 0, policy_version 1259940 (0.0010) [2023-12-27 00:33:41,871][105620] Updated weights for policy 1, policy_version 1261110 (0.0008) [2023-12-27 00:33:41,887][105692] Updated weights for policy 0, policy_version 1259950 (0.0011) [2023-12-27 00:33:41,927][105620] Updated weights for policy 1, policy_version 1261120 (0.0008) [2023-12-27 00:33:42,614][105692] Updated weights for policy 0, policy_version 1259960 (0.0011) [2023-12-27 00:33:42,674][105692] Updated weights for policy 0, policy_version 1259970 (0.0011) [2023-12-27 00:33:42,721][105620] Updated weights for policy 1, policy_version 1261130 (0.0006) [2023-12-27 00:33:42,738][105692] Updated weights for policy 0, policy_version 1259980 (0.0011) [2023-12-27 00:33:42,782][105620] Updated weights for policy 1, policy_version 1261140 (0.0006) [2023-12-27 00:33:42,839][105620] Updated weights for policy 1, policy_version 1261150 (0.0009) [2023-12-27 00:33:43,347][105692] Updated weights for policy 0, policy_version 1259990 (0.0007) [2023-12-27 00:33:43,400][105692] Updated weights for policy 0, policy_version 1260000 (0.0005) [2023-12-27 00:33:43,456][105692] Updated weights for policy 0, policy_version 1260010 (0.0005) [2023-12-27 00:33:43,539][105620] Updated weights for policy 1, policy_version 1261160 (0.0008) [2023-12-27 00:33:43,595][105620] Updated weights for policy 1, policy_version 1261170 (0.0010) [2023-12-27 00:33:43,659][105620] Updated weights for policy 1, policy_version 1261180 (0.0009) [2023-12-27 00:33:44,028][105692] Updated weights for policy 0, policy_version 1260020 (0.0007) [2023-12-27 00:33:44,086][105692] Updated weights for policy 0, policy_version 1260030 (0.0009) [2023-12-27 00:33:44,141][105692] Updated weights for policy 0, policy_version 1260040 (0.0009) [2023-12-27 00:33:44,364][105620] Updated weights for policy 1, policy_version 1261190 (0.0007) [2023-12-27 00:33:44,415][105620] Updated weights for policy 1, policy_version 1261200 (0.0005) [2023-12-27 00:33:44,479][105620] Updated weights for policy 1, policy_version 1261210 (0.0005) [2023-12-27 00:33:44,951][105692] Updated weights for policy 0, policy_version 1260050 (0.0009) [2023-12-27 00:33:45,028][105692] Updated weights for policy 0, policy_version 1260060 (0.0010) [2023-12-27 00:33:45,073][105620] Updated weights for policy 1, policy_version 1261220 (0.0005) [2023-12-27 00:33:45,095][105692] Updated weights for policy 0, policy_version 1260070 (0.0008) [2023-12-27 00:33:45,134][105620] Updated weights for policy 1, policy_version 1261230 (0.0006) [2023-12-27 00:33:45,161][105692] Updated weights for policy 0, policy_version 1260080 (0.0008) [2023-12-27 00:33:45,195][105620] Updated weights for policy 1, policy_version 1261240 (0.0007) [2023-12-27 00:33:45,822][105620] Updated weights for policy 1, policy_version 1261250 (0.0007) [2023-12-27 00:33:45,883][105620] Updated weights for policy 1, policy_version 1261260 (0.0008) [2023-12-27 00:33:45,932][105620] Updated weights for policy 1, policy_version 1261270 (0.0006) [2023-12-27 00:33:45,932][105692] Updated weights for policy 0, policy_version 1260090 (0.0011) [2023-12-27 00:33:45,984][105620] Updated weights for policy 1, policy_version 1261280 (0.0009) [2023-12-27 00:33:45,992][105692] Updated weights for policy 0, policy_version 1260100 (0.0010) [2023-12-27 00:33:46,047][105692] Updated weights for policy 0, policy_version 1260110 (0.0010) [2023-12-27 00:33:46,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19524.4, 300 sec: 19466.4). Total num frames: 645570560. Throughput: 0: 9861.9, 1: 9681.9. Samples: 645531468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:33:46,062][104569] Avg episode reward: [(0, '9082.970'), (1, '9262.802')] [2023-12-27 00:33:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001260112_322641920.pth... [2023-12-27 00:33:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001261280_322928640.pth... [2023-12-27 00:33:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001258928_322338816.pth [2023-12-27 00:33:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001260128_322633728.pth [2023-12-27 00:33:46,744][105620] Updated weights for policy 1, policy_version 1261290 (0.0008) [2023-12-27 00:33:46,758][105692] Updated weights for policy 0, policy_version 1260120 (0.0008) [2023-12-27 00:33:46,800][105620] Updated weights for policy 1, policy_version 1261300 (0.0008) [2023-12-27 00:33:46,807][105692] Updated weights for policy 0, policy_version 1260130 (0.0005) [2023-12-27 00:33:46,854][105620] Updated weights for policy 1, policy_version 1261310 (0.0009) [2023-12-27 00:33:46,862][105692] Updated weights for policy 0, policy_version 1260140 (0.0005) [2023-12-27 00:33:47,489][105692] Updated weights for policy 0, policy_version 1260150 (0.0007) [2023-12-27 00:33:47,540][105692] Updated weights for policy 0, policy_version 1260160 (0.0008) [2023-12-27 00:33:47,567][105620] Updated weights for policy 1, policy_version 1261320 (0.0006) [2023-12-27 00:33:47,597][105692] Updated weights for policy 0, policy_version 1260170 (0.0005) [2023-12-27 00:33:47,635][105620] Updated weights for policy 1, policy_version 1261330 (0.0005) [2023-12-27 00:33:47,692][105620] Updated weights for policy 1, policy_version 1261340 (0.0005) [2023-12-27 00:33:48,322][105692] Updated weights for policy 0, policy_version 1260180 (0.0006) [2023-12-27 00:33:48,361][105620] Updated weights for policy 1, policy_version 1261350 (0.0006) [2023-12-27 00:33:48,383][105692] Updated weights for policy 0, policy_version 1260190 (0.0008) [2023-12-27 00:33:48,426][105620] Updated weights for policy 1, policy_version 1261360 (0.0008) [2023-12-27 00:33:48,445][105692] Updated weights for policy 0, policy_version 1260200 (0.0006) [2023-12-27 00:33:48,484][105620] Updated weights for policy 1, policy_version 1261370 (0.0007) [2023-12-27 00:33:49,173][105692] Updated weights for policy 0, policy_version 1260210 (0.0008) [2023-12-27 00:33:49,199][105620] Updated weights for policy 1, policy_version 1261380 (0.0007) [2023-12-27 00:33:49,240][105692] Updated weights for policy 0, policy_version 1260220 (0.0007) [2023-12-27 00:33:49,265][105620] Updated weights for policy 1, policy_version 1261390 (0.0010) [2023-12-27 00:33:49,297][105692] Updated weights for policy 0, policy_version 1260230 (0.0009) [2023-12-27 00:33:49,323][105620] Updated weights for policy 1, policy_version 1261400 (0.0010) [2023-12-27 00:33:49,363][105692] Updated weights for policy 0, policy_version 1260240 (0.0007) [2023-12-27 00:33:50,090][105620] Updated weights for policy 1, policy_version 1261410 (0.0010) [2023-12-27 00:33:50,133][105692] Updated weights for policy 0, policy_version 1260250 (0.0007) [2023-12-27 00:33:50,146][105620] Updated weights for policy 1, policy_version 1261420 (0.0010) [2023-12-27 00:33:50,192][105692] Updated weights for policy 0, policy_version 1260260 (0.0006) [2023-12-27 00:33:50,205][105620] Updated weights for policy 1, policy_version 1261430 (0.0010) [2023-12-27 00:33:50,251][105692] Updated weights for policy 0, policy_version 1260270 (0.0006) [2023-12-27 00:33:50,264][105620] Updated weights for policy 1, policy_version 1261440 (0.0010) [2023-12-27 00:33:51,018][105692] Updated weights for policy 0, policy_version 1260280 (0.0006) [2023-12-27 00:33:51,023][105620] Updated weights for policy 1, policy_version 1261450 (0.0011) [2023-12-27 00:33:51,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 645652480. Throughput: 0: 9841.6, 1: 9776.1. Samples: 645650308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:33:51,063][104569] Avg episode reward: [(0, '9081.791'), (1, '9172.035')] [2023-12-27 00:33:51,081][105692] Updated weights for policy 0, policy_version 1260290 (0.0008) [2023-12-27 00:33:51,090][105620] Updated weights for policy 1, policy_version 1261460 (0.0011) [2023-12-27 00:33:51,141][105692] Updated weights for policy 0, policy_version 1260300 (0.0008) [2023-12-27 00:33:51,153][105620] Updated weights for policy 1, policy_version 1261470 (0.0010) [2023-12-27 00:33:51,807][105620] Updated weights for policy 1, policy_version 1261480 (0.0010) [2023-12-27 00:33:51,868][105620] Updated weights for policy 1, policy_version 1261490 (0.0010) [2023-12-27 00:33:51,910][105692] Updated weights for policy 0, policy_version 1260310 (0.0008) [2023-12-27 00:33:51,920][105620] Updated weights for policy 1, policy_version 1261500 (0.0010) [2023-12-27 00:33:51,975][105692] Updated weights for policy 0, policy_version 1260320 (0.0006) [2023-12-27 00:33:52,042][105692] Updated weights for policy 0, policy_version 1260330 (0.0005) [2023-12-27 00:33:52,676][105692] Updated weights for policy 0, policy_version 1260340 (0.0006) [2023-12-27 00:33:52,703][105620] Updated weights for policy 1, policy_version 1261510 (0.0009) [2023-12-27 00:33:52,730][105692] Updated weights for policy 0, policy_version 1260350 (0.0009) [2023-12-27 00:33:52,756][105620] Updated weights for policy 1, policy_version 1261520 (0.0010) [2023-12-27 00:33:52,782][105692] Updated weights for policy 0, policy_version 1260360 (0.0010) [2023-12-27 00:33:52,801][105620] Updated weights for policy 1, policy_version 1261530 (0.0005) [2023-12-27 00:33:53,443][105692] Updated weights for policy 0, policy_version 1260370 (0.0009) [2023-12-27 00:33:53,504][105692] Updated weights for policy 0, policy_version 1260380 (0.0008) [2023-12-27 00:33:53,564][105692] Updated weights for policy 0, policy_version 1260390 (0.0005) [2023-12-27 00:33:53,592][105620] Updated weights for policy 1, policy_version 1261540 (0.0008) [2023-12-27 00:33:53,626][105692] Updated weights for policy 0, policy_version 1260400 (0.0009) [2023-12-27 00:33:53,650][105620] Updated weights for policy 1, policy_version 1261550 (0.0005) [2023-12-27 00:33:53,702][105620] Updated weights for policy 1, policy_version 1261560 (0.0008) [2023-12-27 00:33:54,327][105692] Updated weights for policy 0, policy_version 1260410 (0.0010) [2023-12-27 00:33:54,358][105620] Updated weights for policy 1, policy_version 1261570 (0.0008) [2023-12-27 00:33:54,393][105692] Updated weights for policy 0, policy_version 1260420 (0.0010) [2023-12-27 00:33:54,417][105620] Updated weights for policy 1, policy_version 1261580 (0.0007) [2023-12-27 00:33:54,455][105692] Updated weights for policy 0, policy_version 1260430 (0.0011) [2023-12-27 00:33:54,484][105620] Updated weights for policy 1, policy_version 1261590 (0.0006) [2023-12-27 00:33:54,544][105620] Updated weights for policy 1, policy_version 1261600 (0.0008) [2023-12-27 00:33:55,180][105692] Updated weights for policy 0, policy_version 1260440 (0.0009) [2023-12-27 00:33:55,228][105692] Updated weights for policy 0, policy_version 1260450 (0.0006) [2023-12-27 00:33:55,241][105620] Updated weights for policy 1, policy_version 1261610 (0.0007) [2023-12-27 00:33:55,283][105692] Updated weights for policy 0, policy_version 1260460 (0.0008) [2023-12-27 00:33:55,291][105620] Updated weights for policy 1, policy_version 1261620 (0.0006) [2023-12-27 00:33:55,349][105620] Updated weights for policy 1, policy_version 1261630 (0.0009) [2023-12-27 00:33:56,021][105692] Updated weights for policy 0, policy_version 1260470 (0.0009) [2023-12-27 00:33:56,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 645750784. Throughput: 0: 9868.4, 1: 9750.0. Samples: 645765692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:33:56,062][104569] Avg episode reward: [(0, '9076.606'), (1, '9172.589')] [2023-12-27 00:33:56,064][105692] Updated weights for policy 0, policy_version 1260480 (0.0007) [2023-12-27 00:33:56,090][105620] Updated weights for policy 1, policy_version 1261640 (0.0010) [2023-12-27 00:33:56,105][105692] Updated weights for policy 0, policy_version 1260490 (0.0006) [2023-12-27 00:33:56,146][105620] Updated weights for policy 1, policy_version 1261650 (0.0010) [2023-12-27 00:33:56,197][105620] Updated weights for policy 1, policy_version 1261660 (0.0010) [2023-12-27 00:33:56,886][105692] Updated weights for policy 0, policy_version 1260500 (0.0008) [2023-12-27 00:33:56,936][105692] Updated weights for policy 0, policy_version 1260510 (0.0009) [2023-12-27 00:33:56,959][105620] Updated weights for policy 1, policy_version 1261670 (0.0010) [2023-12-27 00:33:56,988][105692] Updated weights for policy 0, policy_version 1260520 (0.0005) [2023-12-27 00:33:57,020][105620] Updated weights for policy 1, policy_version 1261680 (0.0010) [2023-12-27 00:33:57,085][105620] Updated weights for policy 1, policy_version 1261690 (0.0010) [2023-12-27 00:33:57,754][105620] Updated weights for policy 1, policy_version 1261700 (0.0010) [2023-12-27 00:33:57,757][105692] Updated weights for policy 0, policy_version 1260530 (0.0007) [2023-12-27 00:33:57,806][105620] Updated weights for policy 1, policy_version 1261710 (0.0009) [2023-12-27 00:33:57,811][105692] Updated weights for policy 0, policy_version 1260540 (0.0006) [2023-12-27 00:33:57,852][105620] Updated weights for policy 1, policy_version 1261720 (0.0009) [2023-12-27 00:33:57,861][105692] Updated weights for policy 0, policy_version 1260550 (0.0009) [2023-12-27 00:33:57,912][105692] Updated weights for policy 0, policy_version 1260560 (0.0008) [2023-12-27 00:33:58,622][105620] Updated weights for policy 1, policy_version 1261730 (0.0006) [2023-12-27 00:33:58,693][105620] Updated weights for policy 1, policy_version 1261740 (0.0009) [2023-12-27 00:33:58,740][105692] Updated weights for policy 0, policy_version 1260570 (0.0009) [2023-12-27 00:33:58,752][105620] Updated weights for policy 1, policy_version 1261750 (0.0009) [2023-12-27 00:33:58,808][105692] Updated weights for policy 0, policy_version 1260580 (0.0007) [2023-12-27 00:33:58,824][105620] Updated weights for policy 1, policy_version 1261760 (0.0008) [2023-12-27 00:33:58,884][105692] Updated weights for policy 0, policy_version 1260590 (0.0009) [2023-12-27 00:33:59,602][105620] Updated weights for policy 1, policy_version 1261770 (0.0007) [2023-12-27 00:33:59,628][105692] Updated weights for policy 0, policy_version 1260600 (0.0008) [2023-12-27 00:33:59,651][105620] Updated weights for policy 1, policy_version 1261780 (0.0007) [2023-12-27 00:33:59,692][105692] Updated weights for policy 0, policy_version 1260610 (0.0007) [2023-12-27 00:33:59,701][105620] Updated weights for policy 1, policy_version 1261790 (0.0006) [2023-12-27 00:33:59,753][105692] Updated weights for policy 0, policy_version 1260620 (0.0010) [2023-12-27 00:34:00,423][105692] Updated weights for policy 0, policy_version 1260630 (0.0008) [2023-12-27 00:34:00,471][105620] Updated weights for policy 1, policy_version 1261800 (0.0006) [2023-12-27 00:34:00,478][105692] Updated weights for policy 0, policy_version 1260640 (0.0007) [2023-12-27 00:34:00,523][105620] Updated weights for policy 1, policy_version 1261810 (0.0006) [2023-12-27 00:34:00,530][105692] Updated weights for policy 0, policy_version 1260650 (0.0007) [2023-12-27 00:34:00,575][105620] Updated weights for policy 1, policy_version 1261820 (0.0009) [2023-12-27 00:34:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 645849088. Throughput: 0: 9885.3, 1: 9755.3. Samples: 645821564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:34:01,062][104569] Avg episode reward: [(0, '9076.599'), (1, '8900.075')] [2023-12-27 00:34:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001261824_323067904.pth... [2023-12-27 00:34:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001260656_322781184.pth... [2023-12-27 00:34:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001260672_322772992.pth [2023-12-27 00:34:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001259504_322486272.pth [2023-12-27 00:34:01,117][105692] Updated weights for policy 0, policy_version 1260660 (0.0007) [2023-12-27 00:34:01,180][105692] Updated weights for policy 0, policy_version 1260670 (0.0008) [2023-12-27 00:34:01,239][105692] Updated weights for policy 0, policy_version 1260680 (0.0006) [2023-12-27 00:34:01,247][105620] Updated weights for policy 1, policy_version 1261830 (0.0008) [2023-12-27 00:34:01,310][105620] Updated weights for policy 1, policy_version 1261840 (0.0006) [2023-12-27 00:34:01,374][105620] Updated weights for policy 1, policy_version 1261850 (0.0007) [2023-12-27 00:34:02,019][105692] Updated weights for policy 0, policy_version 1260690 (0.0008) [2023-12-27 00:34:02,028][105620] Updated weights for policy 1, policy_version 1261860 (0.0008) [2023-12-27 00:34:02,077][105692] Updated weights for policy 0, policy_version 1260700 (0.0009) [2023-12-27 00:34:02,091][105620] Updated weights for policy 1, policy_version 1261870 (0.0007) [2023-12-27 00:34:02,139][105692] Updated weights for policy 0, policy_version 1260710 (0.0006) [2023-12-27 00:34:02,149][105620] Updated weights for policy 1, policy_version 1261880 (0.0008) [2023-12-27 00:34:02,200][105692] Updated weights for policy 0, policy_version 1260720 (0.0006) [2023-12-27 00:34:02,872][105620] Updated weights for policy 1, policy_version 1261890 (0.0008) [2023-12-27 00:34:02,929][105620] Updated weights for policy 1, policy_version 1261900 (0.0009) [2023-12-27 00:34:02,947][105692] Updated weights for policy 0, policy_version 1260730 (0.0006) [2023-12-27 00:34:02,984][105620] Updated weights for policy 1, policy_version 1261910 (0.0008) [2023-12-27 00:34:02,998][105692] Updated weights for policy 0, policy_version 1260740 (0.0005) [2023-12-27 00:34:03,039][105620] Updated weights for policy 1, policy_version 1261920 (0.0008) [2023-12-27 00:34:03,050][105692] Updated weights for policy 0, policy_version 1260750 (0.0006) [2023-12-27 00:34:03,797][105620] Updated weights for policy 1, policy_version 1261930 (0.0009) [2023-12-27 00:34:03,811][105692] Updated weights for policy 0, policy_version 1260760 (0.0006) [2023-12-27 00:34:03,861][105620] Updated weights for policy 1, policy_version 1261940 (0.0008) [2023-12-27 00:34:03,873][105692] Updated weights for policy 0, policy_version 1260770 (0.0007) [2023-12-27 00:34:03,920][105620] Updated weights for policy 1, policy_version 1261950 (0.0007) [2023-12-27 00:34:03,929][105692] Updated weights for policy 0, policy_version 1260780 (0.0009) [2023-12-27 00:34:04,617][105692] Updated weights for policy 0, policy_version 1260790 (0.0009) [2023-12-27 00:34:04,668][105692] Updated weights for policy 0, policy_version 1260800 (0.0009) [2023-12-27 00:34:04,682][105620] Updated weights for policy 1, policy_version 1261960 (0.0009) [2023-12-27 00:34:04,717][105692] Updated weights for policy 0, policy_version 1260810 (0.0006) [2023-12-27 00:34:04,734][105620] Updated weights for policy 1, policy_version 1261970 (0.0008) [2023-12-27 00:34:04,782][105620] Updated weights for policy 1, policy_version 1261980 (0.0009) [2023-12-27 00:34:05,482][105692] Updated weights for policy 0, policy_version 1260820 (0.0008) [2023-12-27 00:34:05,534][105692] Updated weights for policy 0, policy_version 1260830 (0.0010) [2023-12-27 00:34:05,548][105620] Updated weights for policy 1, policy_version 1261990 (0.0006) [2023-12-27 00:34:05,582][105692] Updated weights for policy 0, policy_version 1260840 (0.0011) [2023-12-27 00:34:05,603][105620] Updated weights for policy 1, policy_version 1262000 (0.0006) [2023-12-27 00:34:05,654][105620] Updated weights for policy 1, policy_version 1262010 (0.0007) [2023-12-27 00:34:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 645947392. Throughput: 0: 9885.8, 1: 9740.9. Samples: 645937128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:34:06,062][104569] Avg episode reward: [(0, '8809.352'), (1, '8900.038')] [2023-12-27 00:34:06,368][105692] Updated weights for policy 0, policy_version 1260850 (0.0011) [2023-12-27 00:34:06,418][105620] Updated weights for policy 1, policy_version 1262020 (0.0008) [2023-12-27 00:34:06,425][105692] Updated weights for policy 0, policy_version 1260860 (0.0011) [2023-12-27 00:34:06,477][105692] Updated weights for policy 0, policy_version 1260870 (0.0011) [2023-12-27 00:34:06,482][105620] Updated weights for policy 1, policy_version 1262030 (0.0006) [2023-12-27 00:34:06,530][105692] Updated weights for policy 0, policy_version 1260880 (0.0011) [2023-12-27 00:34:06,539][105620] Updated weights for policy 1, policy_version 1262040 (0.0006) [2023-12-27 00:34:07,246][105620] Updated weights for policy 1, policy_version 1262050 (0.0008) [2023-12-27 00:34:07,252][105692] Updated weights for policy 0, policy_version 1260890 (0.0009) [2023-12-27 00:34:07,306][105692] Updated weights for policy 0, policy_version 1260901 (0.0009) [2023-12-27 00:34:07,310][105620] Updated weights for policy 1, policy_version 1262060 (0.0005) [2023-12-27 00:34:07,360][105692] Updated weights for policy 0, policy_version 1260911 (0.0008) [2023-12-27 00:34:07,375][105620] Updated weights for policy 1, policy_version 1262070 (0.0007) [2023-12-27 00:34:07,440][105620] Updated weights for policy 1, policy_version 1262080 (0.0007) [2023-12-27 00:34:07,978][105620] Updated weights for policy 1, policy_version 1262090 (0.0008) [2023-12-27 00:34:08,040][105620] Updated weights for policy 1, policy_version 1262100 (0.0009) [2023-12-27 00:34:08,064][105692] Updated weights for policy 0, policy_version 1260921 (0.0010) [2023-12-27 00:34:08,098][105620] Updated weights for policy 1, policy_version 1262110 (0.0007) [2023-12-27 00:34:08,113][105692] Updated weights for policy 0, policy_version 1260931 (0.0010) [2023-12-27 00:34:08,161][105692] Updated weights for policy 0, policy_version 1260941 (0.0010) [2023-12-27 00:34:08,799][105620] Updated weights for policy 1, policy_version 1262120 (0.0008) [2023-12-27 00:34:08,855][105620] Updated weights for policy 1, policy_version 1262130 (0.0009) [2023-12-27 00:34:08,894][105692] Updated weights for policy 0, policy_version 1260951 (0.0008) [2023-12-27 00:34:08,909][105620] Updated weights for policy 1, policy_version 1262140 (0.0007) [2023-12-27 00:34:08,945][105692] Updated weights for policy 0, policy_version 1260961 (0.0008) [2023-12-27 00:34:08,990][105692] Updated weights for policy 0, policy_version 1260971 (0.0005) [2023-12-27 00:34:09,715][105620] Updated weights for policy 1, policy_version 1262150 (0.0010) [2023-12-27 00:34:09,763][105620] Updated weights for policy 1, policy_version 1262160 (0.0007) [2023-12-27 00:34:09,772][105692] Updated weights for policy 0, policy_version 1260981 (0.0009) [2023-12-27 00:34:09,821][105620] Updated weights for policy 1, policy_version 1262170 (0.0006) [2023-12-27 00:34:09,836][105692] Updated weights for policy 0, policy_version 1260991 (0.0008) [2023-12-27 00:34:09,899][105692] Updated weights for policy 0, policy_version 1261001 (0.0009) [2023-12-27 00:34:10,571][105620] Updated weights for policy 1, policy_version 1262180 (0.0009) [2023-12-27 00:34:10,624][105620] Updated weights for policy 1, policy_version 1262190 (0.0008) [2023-12-27 00:34:10,626][105692] Updated weights for policy 0, policy_version 1261011 (0.0008) [2023-12-27 00:34:10,679][105692] Updated weights for policy 0, policy_version 1261021 (0.0009) [2023-12-27 00:34:10,690][105620] Updated weights for policy 1, policy_version 1262200 (0.0005) [2023-12-27 00:34:10,733][105692] Updated weights for policy 0, policy_version 1261032 (0.0009) [2023-12-27 00:34:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 646045696. Throughput: 0: 9831.5, 1: 9788.0. Samples: 646053752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:34:11,063][104569] Avg episode reward: [(0, '8813.834'), (1, '9263.137')] [2023-12-27 00:34:11,289][105620] Updated weights for policy 1, policy_version 1262210 (0.0006) [2023-12-27 00:34:11,353][105620] Updated weights for policy 1, policy_version 1262220 (0.0010) [2023-12-27 00:34:11,421][105620] Updated weights for policy 1, policy_version 1262230 (0.0012) [2023-12-27 00:34:11,489][105620] Updated weights for policy 1, policy_version 1262240 (0.0011) [2023-12-27 00:34:11,571][105692] Updated weights for policy 0, policy_version 1261042 (0.0009) [2023-12-27 00:34:11,639][105692] Updated weights for policy 0, policy_version 1261052 (0.0009) [2023-12-27 00:34:11,692][105692] Updated weights for policy 0, policy_version 1261062 (0.0008) [2023-12-27 00:34:11,767][105692] Updated weights for policy 0, policy_version 1261072 (0.0009) [2023-12-27 00:34:12,276][105620] Updated weights for policy 1, policy_version 1262250 (0.0011) [2023-12-27 00:34:12,337][105620] Updated weights for policy 1, policy_version 1262260 (0.0011) [2023-12-27 00:34:12,413][105620] Updated weights for policy 1, policy_version 1262270 (0.0009) [2023-12-27 00:34:12,533][105692] Updated weights for policy 0, policy_version 1261082 (0.0008) [2023-12-27 00:34:12,589][105692] Updated weights for policy 0, policy_version 1261092 (0.0008) [2023-12-27 00:34:12,644][105692] Updated weights for policy 0, policy_version 1261102 (0.0007) [2023-12-27 00:34:13,157][105620] Updated weights for policy 1, policy_version 1262280 (0.0011) [2023-12-27 00:34:13,216][105620] Updated weights for policy 1, policy_version 1262290 (0.0009) [2023-12-27 00:34:13,265][105620] Updated weights for policy 1, policy_version 1262300 (0.0010) [2023-12-27 00:34:13,370][105692] Updated weights for policy 0, policy_version 1261112 (0.0009) [2023-12-27 00:34:13,431][105692] Updated weights for policy 0, policy_version 1261122 (0.0009) [2023-12-27 00:34:13,483][105692] Updated weights for policy 0, policy_version 1261132 (0.0007) [2023-12-27 00:34:13,965][105620] Updated weights for policy 1, policy_version 1262310 (0.0010) [2023-12-27 00:34:14,016][105620] Updated weights for policy 1, policy_version 1262320 (0.0010) [2023-12-27 00:34:14,078][105620] Updated weights for policy 1, policy_version 1262330 (0.0010) [2023-12-27 00:34:14,256][105692] Updated weights for policy 0, policy_version 1261142 (0.0007) [2023-12-27 00:34:14,312][105692] Updated weights for policy 0, policy_version 1261152 (0.0005) [2023-12-27 00:34:14,374][105692] Updated weights for policy 0, policy_version 1261162 (0.0007) [2023-12-27 00:34:14,787][105620] Updated weights for policy 1, policy_version 1262340 (0.0010) [2023-12-27 00:34:14,855][105620] Updated weights for policy 1, policy_version 1262350 (0.0007) [2023-12-27 00:34:14,917][105620] Updated weights for policy 1, policy_version 1262360 (0.0009) [2023-12-27 00:34:15,103][105692] Updated weights for policy 0, policy_version 1261172 (0.0009) [2023-12-27 00:34:15,165][105692] Updated weights for policy 0, policy_version 1261182 (0.0008) [2023-12-27 00:34:15,229][105692] Updated weights for policy 0, policy_version 1261192 (0.0007) [2023-12-27 00:34:15,684][105620] Updated weights for policy 1, policy_version 1262370 (0.0009) [2023-12-27 00:34:15,746][105620] Updated weights for policy 1, policy_version 1262380 (0.0008) [2023-12-27 00:34:15,812][105620] Updated weights for policy 1, policy_version 1262390 (0.0010) [2023-12-27 00:34:15,842][105692] Updated weights for policy 0, policy_version 1261202 (0.0006) [2023-12-27 00:34:15,872][105620] Updated weights for policy 1, policy_version 1262400 (0.0008) [2023-12-27 00:34:15,900][105692] Updated weights for policy 0, policy_version 1261212 (0.0007) [2023-12-27 00:34:15,953][105692] Updated weights for policy 0, policy_version 1261222 (0.0009) [2023-12-27 00:34:16,014][105692] Updated weights for policy 0, policy_version 1261232 (0.0009) [2023-12-27 00:34:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.9, 300 sec: 19466.4). Total num frames: 646144000. Throughput: 0: 9755.8, 1: 9794.4. Samples: 646108972. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:34:16,062][104569] Avg episode reward: [(0, '9178.646'), (1, '9353.647')] [2023-12-27 00:34:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001261232_322928640.pth... [2023-12-27 00:34:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001262400_323215360.pth... [2023-12-27 00:34:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001260112_322641920.pth [2023-12-27 00:34:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001261280_322928640.pth [2023-12-27 00:34:16,542][105620] Updated weights for policy 1, policy_version 1262410 (0.0007) [2023-12-27 00:34:16,602][105620] Updated weights for policy 1, policy_version 1262420 (0.0008) [2023-12-27 00:34:16,654][105620] Updated weights for policy 1, policy_version 1262430 (0.0008) [2023-12-27 00:34:16,799][105692] Updated weights for policy 0, policy_version 1261242 (0.0010) [2023-12-27 00:34:16,851][105692] Updated weights for policy 0, policy_version 1261252 (0.0010) [2023-12-27 00:34:16,906][105692] Updated weights for policy 0, policy_version 1261262 (0.0010) [2023-12-27 00:34:17,402][105620] Updated weights for policy 1, policy_version 1262440 (0.0008) [2023-12-27 00:34:17,453][105620] Updated weights for policy 1, policy_version 1262450 (0.0008) [2023-12-27 00:34:17,505][105620] Updated weights for policy 1, policy_version 1262460 (0.0008) [2023-12-27 00:34:17,665][105692] Updated weights for policy 0, policy_version 1261272 (0.0009) [2023-12-27 00:34:17,723][105692] Updated weights for policy 0, policy_version 1261282 (0.0010) [2023-12-27 00:34:17,779][105692] Updated weights for policy 0, policy_version 1261293 (0.0011) [2023-12-27 00:34:18,213][105620] Updated weights for policy 1, policy_version 1262470 (0.0009) [2023-12-27 00:34:18,264][105620] Updated weights for policy 1, policy_version 1262480 (0.0009) [2023-12-27 00:34:18,317][105620] Updated weights for policy 1, policy_version 1262490 (0.0009) [2023-12-27 00:34:18,580][105692] Updated weights for policy 0, policy_version 1261304 (0.0009) [2023-12-27 00:34:18,635][105692] Updated weights for policy 0, policy_version 1261314 (0.0008) [2023-12-27 00:34:18,690][105692] Updated weights for policy 0, policy_version 1261324 (0.0008) [2023-12-27 00:34:19,054][105620] Updated weights for policy 1, policy_version 1262500 (0.0008) [2023-12-27 00:34:19,117][105620] Updated weights for policy 1, policy_version 1262510 (0.0008) [2023-12-27 00:34:19,170][105620] Updated weights for policy 1, policy_version 1262520 (0.0008) [2023-12-27 00:34:19,460][105692] Updated weights for policy 0, policy_version 1261334 (0.0010) [2023-12-27 00:34:19,525][105692] Updated weights for policy 0, policy_version 1261344 (0.0010) [2023-12-27 00:34:19,590][105692] Updated weights for policy 0, policy_version 1261354 (0.0011) [2023-12-27 00:34:19,931][105620] Updated weights for policy 1, policy_version 1262530 (0.0008) [2023-12-27 00:34:19,996][105620] Updated weights for policy 1, policy_version 1262540 (0.0008) [2023-12-27 00:34:20,057][105620] Updated weights for policy 1, policy_version 1262550 (0.0009) [2023-12-27 00:34:20,119][105620] Updated weights for policy 1, policy_version 1262560 (0.0008) [2023-12-27 00:34:20,294][105692] Updated weights for policy 0, policy_version 1261364 (0.0011) [2023-12-27 00:34:20,350][105692] Updated weights for policy 0, policy_version 1261374 (0.0011) [2023-12-27 00:34:20,407][105692] Updated weights for policy 0, policy_version 1261384 (0.0011) [2023-12-27 00:34:20,877][105620] Updated weights for policy 1, policy_version 1262570 (0.0008) [2023-12-27 00:34:20,933][105620] Updated weights for policy 1, policy_version 1262580 (0.0005) [2023-12-27 00:34:21,000][105620] Updated weights for policy 1, policy_version 1262590 (0.0005) [2023-12-27 00:34:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 646234112. Throughput: 0: 9621.2, 1: 9704.1. Samples: 646222800. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:34:21,062][104569] Avg episode reward: [(0, '9266.601'), (1, '9171.394')] [2023-12-27 00:34:21,160][105692] Updated weights for policy 0, policy_version 1261394 (0.0008) [2023-12-27 00:34:21,223][105692] Updated weights for policy 0, policy_version 1261404 (0.0011) [2023-12-27 00:34:21,289][105692] Updated weights for policy 0, policy_version 1261414 (0.0011) [2023-12-27 00:34:21,349][105692] Updated weights for policy 0, policy_version 1261424 (0.0011) [2023-12-27 00:34:21,761][105620] Updated weights for policy 1, policy_version 1262600 (0.0008) [2023-12-27 00:34:21,824][105620] Updated weights for policy 1, policy_version 1262610 (0.0007) [2023-12-27 00:34:21,891][105620] Updated weights for policy 1, policy_version 1262620 (0.0007) [2023-12-27 00:34:22,005][105692] Updated weights for policy 0, policy_version 1261434 (0.0009) [2023-12-27 00:34:22,061][105692] Updated weights for policy 0, policy_version 1261444 (0.0009) [2023-12-27 00:34:22,123][105692] Updated weights for policy 0, policy_version 1261454 (0.0009) [2023-12-27 00:34:22,669][105620] Updated weights for policy 1, policy_version 1262630 (0.0009) [2023-12-27 00:34:22,731][105620] Updated weights for policy 1, policy_version 1262640 (0.0009) [2023-12-27 00:34:22,785][105620] Updated weights for policy 1, policy_version 1262650 (0.0009) [2023-12-27 00:34:22,954][105692] Updated weights for policy 0, policy_version 1261464 (0.0009) [2023-12-27 00:34:22,955][105585] KL-divergence is very high: 366.9337 [2023-12-27 00:34:23,002][105585] KL-divergence is very high: 716.0502 [2023-12-27 00:34:23,016][105692] Updated weights for policy 0, policy_version 1261474 (0.0009) [2023-12-27 00:34:23,053][105585] KL-divergence is very high: 838.5797 [2023-12-27 00:34:23,078][105692] Updated weights for policy 0, policy_version 1261484 (0.0009) [2023-12-27 00:34:23,538][105620] Updated weights for policy 1, policy_version 1262660 (0.0009) [2023-12-27 00:34:23,590][105620] Updated weights for policy 1, policy_version 1262670 (0.0009) [2023-12-27 00:34:23,640][105620] Updated weights for policy 1, policy_version 1262680 (0.0009) [2023-12-27 00:34:23,823][105692] Updated weights for policy 0, policy_version 1261494 (0.0007) [2023-12-27 00:34:23,875][105692] Updated weights for policy 0, policy_version 1261504 (0.0006) [2023-12-27 00:34:23,922][105692] Updated weights for policy 0, policy_version 1261514 (0.0008) [2023-12-27 00:34:24,401][105620] Updated weights for policy 1, policy_version 1262690 (0.0008) [2023-12-27 00:34:24,466][105620] Updated weights for policy 1, policy_version 1262700 (0.0007) [2023-12-27 00:34:24,520][105620] Updated weights for policy 1, policy_version 1262710 (0.0009) [2023-12-27 00:34:24,582][105620] Updated weights for policy 1, policy_version 1262720 (0.0009) [2023-12-27 00:34:24,680][105692] Updated weights for policy 0, policy_version 1261524 (0.0009) [2023-12-27 00:34:24,739][105692] Updated weights for policy 0, policy_version 1261534 (0.0009) [2023-12-27 00:34:24,803][105692] Updated weights for policy 0, policy_version 1261544 (0.0009) [2023-12-27 00:34:25,322][105620] Updated weights for policy 1, policy_version 1262730 (0.0009) [2023-12-27 00:34:25,380][105620] Updated weights for policy 1, policy_version 1262741 (0.0011) [2023-12-27 00:34:25,439][105620] Updated weights for policy 1, policy_version 1262752 (0.0010) [2023-12-27 00:34:25,510][105692] Updated weights for policy 0, policy_version 1261554 (0.0008) [2023-12-27 00:34:25,563][105692] Updated weights for policy 0, policy_version 1261564 (0.0005) [2023-12-27 00:34:25,627][105692] Updated weights for policy 0, policy_version 1261574 (0.0005) [2023-12-27 00:34:25,679][105692] Updated weights for policy 0, policy_version 1261584 (0.0005) [2023-12-27 00:34:26,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 646324224. Throughput: 0: 9565.1, 1: 9573.1. Samples: 646334676. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:34:26,063][104569] Avg episode reward: [(0, '9082.195'), (1, '9171.340')] [2023-12-27 00:34:26,221][105620] Updated weights for policy 1, policy_version 1262762 (0.0009) [2023-12-27 00:34:26,274][105620] Updated weights for policy 1, policy_version 1262772 (0.0008) [2023-12-27 00:34:26,316][105692] Updated weights for policy 0, policy_version 1261594 (0.0007) [2023-12-27 00:34:26,326][105620] Updated weights for policy 1, policy_version 1262782 (0.0007) [2023-12-27 00:34:26,372][105692] Updated weights for policy 0, policy_version 1261604 (0.0008) [2023-12-27 00:34:26,426][105692] Updated weights for policy 0, policy_version 1261614 (0.0009) [2023-12-27 00:34:27,010][105692] Updated weights for policy 0, policy_version 1261624 (0.0010) [2023-12-27 00:34:27,065][105692] Updated weights for policy 0, policy_version 1261634 (0.0009) [2023-12-27 00:34:27,119][105692] Updated weights for policy 0, policy_version 1261644 (0.0005) [2023-12-27 00:34:27,184][105620] Updated weights for policy 1, policy_version 1262792 (0.0008) [2023-12-27 00:34:27,232][105620] Updated weights for policy 1, policy_version 1262802 (0.0008) [2023-12-27 00:34:27,279][105620] Updated weights for policy 1, policy_version 1262812 (0.0007) [2023-12-27 00:34:27,834][105692] Updated weights for policy 0, policy_version 1261654 (0.0009) [2023-12-27 00:34:27,891][105692] Updated weights for policy 0, policy_version 1261664 (0.0010) [2023-12-27 00:34:27,938][105692] Updated weights for policy 0, policy_version 1261674 (0.0010) [2023-12-27 00:34:28,040][105620] Updated weights for policy 1, policy_version 1262822 (0.0008) [2023-12-27 00:34:28,092][105620] Updated weights for policy 1, policy_version 1262832 (0.0006) [2023-12-27 00:34:28,152][105620] Updated weights for policy 1, policy_version 1262842 (0.0007) [2023-12-27 00:34:28,688][105692] Updated weights for policy 0, policy_version 1261684 (0.0010) [2023-12-27 00:34:28,739][105692] Updated weights for policy 0, policy_version 1261694 (0.0010) [2023-12-27 00:34:28,786][105692] Updated weights for policy 0, policy_version 1261704 (0.0010) [2023-12-27 00:34:28,889][105620] Updated weights for policy 1, policy_version 1262852 (0.0007) [2023-12-27 00:34:28,949][105620] Updated weights for policy 1, policy_version 1262862 (0.0008) [2023-12-27 00:34:28,997][105620] Updated weights for policy 1, policy_version 1262872 (0.0008) [2023-12-27 00:34:29,540][105692] Updated weights for policy 0, policy_version 1261714 (0.0010) [2023-12-27 00:34:29,591][105692] Updated weights for policy 0, policy_version 1261724 (0.0010) [2023-12-27 00:34:29,642][105692] Updated weights for policy 0, policy_version 1261734 (0.0010) [2023-12-27 00:34:29,702][105692] Updated weights for policy 0, policy_version 1261744 (0.0010) [2023-12-27 00:34:29,768][105620] Updated weights for policy 1, policy_version 1262882 (0.0008) [2023-12-27 00:34:29,833][105620] Updated weights for policy 1, policy_version 1262892 (0.0007) [2023-12-27 00:34:29,890][105620] Updated weights for policy 1, policy_version 1262902 (0.0009) [2023-12-27 00:34:29,950][105620] Updated weights for policy 1, policy_version 1262912 (0.0010) [2023-12-27 00:34:30,453][105692] Updated weights for policy 0, policy_version 1261754 (0.0008) [2023-12-27 00:34:30,521][105692] Updated weights for policy 0, policy_version 1261764 (0.0007) [2023-12-27 00:34:30,576][105692] Updated weights for policy 0, policy_version 1261774 (0.0009) [2023-12-27 00:34:30,732][105620] Updated weights for policy 1, policy_version 1262922 (0.0009) [2023-12-27 00:34:30,790][105620] Updated weights for policy 1, policy_version 1262932 (0.0009) [2023-12-27 00:34:30,838][105620] Updated weights for policy 1, policy_version 1262943 (0.0009) [2023-12-27 00:34:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 646422528. Throughput: 0: 9560.2, 1: 9601.2. Samples: 646393732. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:34:31,062][104569] Avg episode reward: [(0, '8907.711'), (1, '9171.829')] [2023-12-27 00:34:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001262944_323354624.pth... [2023-12-27 00:34:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001261776_323067904.pth... [2023-12-27 00:34:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001261824_323067904.pth [2023-12-27 00:34:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001260656_322781184.pth [2023-12-27 00:34:31,305][105692] Updated weights for policy 0, policy_version 1261784 (0.0009) [2023-12-27 00:34:31,372][105692] Updated weights for policy 0, policy_version 1261794 (0.0007) [2023-12-27 00:34:31,438][105692] Updated weights for policy 0, policy_version 1261804 (0.0007) [2023-12-27 00:34:31,543][105620] Updated weights for policy 1, policy_version 1262953 (0.0009) [2023-12-27 00:34:31,598][105620] Updated weights for policy 1, policy_version 1262963 (0.0009) [2023-12-27 00:34:31,665][105620] Updated weights for policy 1, policy_version 1262973 (0.0008) [2023-12-27 00:34:32,139][105692] Updated weights for policy 0, policy_version 1261814 (0.0008) [2023-12-27 00:34:32,191][105692] Updated weights for policy 0, policy_version 1261824 (0.0010) [2023-12-27 00:34:32,248][105692] Updated weights for policy 0, policy_version 1261834 (0.0011) [2023-12-27 00:34:32,362][105620] Updated weights for policy 1, policy_version 1262983 (0.0009) [2023-12-27 00:34:32,426][105620] Updated weights for policy 1, policy_version 1262993 (0.0010) [2023-12-27 00:34:32,486][105620] Updated weights for policy 1, policy_version 1263004 (0.0010) [2023-12-27 00:34:32,941][105692] Updated weights for policy 0, policy_version 1261844 (0.0011) [2023-12-27 00:34:32,993][105692] Updated weights for policy 0, policy_version 1261854 (0.0010) [2023-12-27 00:34:33,045][105692] Updated weights for policy 0, policy_version 1261864 (0.0009) [2023-12-27 00:34:33,143][105620] Updated weights for policy 1, policy_version 1263014 (0.0006) [2023-12-27 00:34:33,197][105620] Updated weights for policy 1, policy_version 1263024 (0.0005) [2023-12-27 00:34:33,247][105620] Updated weights for policy 1, policy_version 1263034 (0.0005) [2023-12-27 00:34:33,754][105692] Updated weights for policy 0, policy_version 1261874 (0.0006) [2023-12-27 00:34:33,802][105692] Updated weights for policy 0, policy_version 1261884 (0.0010) [2023-12-27 00:34:33,849][105692] Updated weights for policy 0, policy_version 1261894 (0.0010) [2023-12-27 00:34:33,879][105620] Updated weights for policy 1, policy_version 1263044 (0.0007) [2023-12-27 00:34:33,901][105692] Updated weights for policy 0, policy_version 1261904 (0.0010) [2023-12-27 00:34:33,933][105620] Updated weights for policy 1, policy_version 1263054 (0.0010) [2023-12-27 00:34:33,995][105620] Updated weights for policy 1, policy_version 1263064 (0.0010) [2023-12-27 00:34:34,672][105692] Updated weights for policy 0, policy_version 1261914 (0.0009) [2023-12-27 00:34:34,724][105692] Updated weights for policy 0, policy_version 1261924 (0.0010) [2023-12-27 00:34:34,733][105620] Updated weights for policy 1, policy_version 1263074 (0.0010) [2023-12-27 00:34:34,786][105692] Updated weights for policy 0, policy_version 1261934 (0.0011) [2023-12-27 00:34:34,794][105620] Updated weights for policy 1, policy_version 1263084 (0.0005) [2023-12-27 00:34:34,844][105620] Updated weights for policy 1, policy_version 1263094 (0.0008) [2023-12-27 00:34:34,896][105620] Updated weights for policy 1, policy_version 1263104 (0.0010) [2023-12-27 00:34:35,502][105620] Updated weights for policy 1, policy_version 1263114 (0.0010) [2023-12-27 00:34:35,528][105692] Updated weights for policy 0, policy_version 1261944 (0.0010) [2023-12-27 00:34:35,557][105620] Updated weights for policy 1, policy_version 1263124 (0.0010) [2023-12-27 00:34:35,584][105692] Updated weights for policy 0, policy_version 1261954 (0.0010) [2023-12-27 00:34:35,611][105620] Updated weights for policy 1, policy_version 1263134 (0.0010) [2023-12-27 00:34:35,645][105692] Updated weights for policy 0, policy_version 1261964 (0.0010) [2023-12-27 00:34:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 646520832. Throughput: 0: 9553.8, 1: 9552.1. Samples: 646510072. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:34:36,062][104569] Avg episode reward: [(0, '8373.782'), (1, '8898.892')] [2023-12-27 00:34:36,303][105620] Updated weights for policy 1, policy_version 1263144 (0.0009) [2023-12-27 00:34:36,368][105620] Updated weights for policy 1, policy_version 1263154 (0.0009) [2023-12-27 00:34:36,434][105620] Updated weights for policy 1, policy_version 1263164 (0.0008) [2023-12-27 00:34:36,458][105692] Updated weights for policy 0, policy_version 1261974 (0.0008) [2023-12-27 00:34:36,518][105692] Updated weights for policy 0, policy_version 1261984 (0.0005) [2023-12-27 00:34:36,581][105692] Updated weights for policy 0, policy_version 1261994 (0.0006) [2023-12-27 00:34:37,120][105692] Updated weights for policy 0, policy_version 1262004 (0.0005) [2023-12-27 00:34:37,171][105692] Updated weights for policy 0, policy_version 1262014 (0.0005) [2023-12-27 00:34:37,210][105620] Updated weights for policy 1, policy_version 1263174 (0.0009) [2023-12-27 00:34:37,221][105692] Updated weights for policy 0, policy_version 1262024 (0.0007) [2023-12-27 00:34:37,264][105620] Updated weights for policy 1, policy_version 1263184 (0.0007) [2023-12-27 00:34:37,330][105620] Updated weights for policy 1, policy_version 1263194 (0.0010) [2023-12-27 00:34:37,938][105692] Updated weights for policy 0, policy_version 1262034 (0.0007) [2023-12-27 00:34:37,997][105692] Updated weights for policy 0, policy_version 1262044 (0.0008) [2023-12-27 00:34:38,060][105692] Updated weights for policy 0, policy_version 1262054 (0.0008) [2023-12-27 00:34:38,084][105620] Updated weights for policy 1, policy_version 1263204 (0.0010) [2023-12-27 00:34:38,104][105692] Updated weights for policy 0, policy_version 1262064 (0.0009) [2023-12-27 00:34:38,142][105620] Updated weights for policy 1, policy_version 1263214 (0.0010) [2023-12-27 00:34:38,200][105620] Updated weights for policy 1, policy_version 1263224 (0.0010) [2023-12-27 00:34:38,890][105692] Updated weights for policy 0, policy_version 1262074 (0.0011) [2023-12-27 00:34:38,937][105620] Updated weights for policy 1, policy_version 1263234 (0.0010) [2023-12-27 00:34:38,950][105692] Updated weights for policy 0, policy_version 1262084 (0.0011) [2023-12-27 00:34:38,985][105620] Updated weights for policy 1, policy_version 1263244 (0.0011) [2023-12-27 00:34:39,006][105692] Updated weights for policy 0, policy_version 1262094 (0.0011) [2023-12-27 00:34:39,038][105620] Updated weights for policy 1, policy_version 1263254 (0.0011) [2023-12-27 00:34:39,093][105620] Updated weights for policy 1, policy_version 1263264 (0.0011) [2023-12-27 00:34:39,822][105692] Updated weights for policy 0, policy_version 1262104 (0.0007) [2023-12-27 00:34:39,883][105692] Updated weights for policy 0, policy_version 1262114 (0.0010) [2023-12-27 00:34:39,914][105620] Updated weights for policy 1, policy_version 1263274 (0.0011) [2023-12-27 00:34:39,954][105692] Updated weights for policy 0, policy_version 1262124 (0.0011) [2023-12-27 00:34:39,983][105620] Updated weights for policy 1, policy_version 1263284 (0.0010) [2023-12-27 00:34:40,033][105620] Updated weights for policy 1, policy_version 1263294 (0.0010) [2023-12-27 00:34:40,611][105692] Updated weights for policy 0, policy_version 1262134 (0.0007) [2023-12-27 00:34:40,673][105692] Updated weights for policy 0, policy_version 1262144 (0.0005) [2023-12-27 00:34:40,711][105620] Updated weights for policy 1, policy_version 1263304 (0.0010) [2023-12-27 00:34:40,730][105692] Updated weights for policy 0, policy_version 1262154 (0.0006) [2023-12-27 00:34:40,776][105620] Updated weights for policy 1, policy_version 1263314 (0.0010) [2023-12-27 00:34:40,833][105620] Updated weights for policy 1, policy_version 1263324 (0.0010) [2023-12-27 00:34:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19114.7, 300 sec: 19410.9). Total num frames: 646619136. Throughput: 0: 9563.7, 1: 9549.6. Samples: 646625792. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:34:41,062][104569] Avg episode reward: [(0, '7925.810'), (1, '9171.936')] [2023-12-27 00:34:41,319][105692] Updated weights for policy 0, policy_version 1262164 (0.0006) [2023-12-27 00:34:41,392][105692] Updated weights for policy 0, policy_version 1262174 (0.0008) [2023-12-27 00:34:41,454][105692] Updated weights for policy 0, policy_version 1262184 (0.0006) [2023-12-27 00:34:41,578][105620] Updated weights for policy 1, policy_version 1263334 (0.0011) [2023-12-27 00:34:41,645][105620] Updated weights for policy 1, policy_version 1263344 (0.0011) [2023-12-27 00:34:41,694][105620] Updated weights for policy 1, policy_version 1263354 (0.0010) [2023-12-27 00:34:42,141][105692] Updated weights for policy 0, policy_version 1262194 (0.0006) [2023-12-27 00:34:42,195][105692] Updated weights for policy 0, policy_version 1262204 (0.0010) [2023-12-27 00:34:42,249][105692] Updated weights for policy 0, policy_version 1262214 (0.0008) [2023-12-27 00:34:42,318][105692] Updated weights for policy 0, policy_version 1262224 (0.0009) [2023-12-27 00:34:42,382][105620] Updated weights for policy 1, policy_version 1263364 (0.0010) [2023-12-27 00:34:42,445][105620] Updated weights for policy 1, policy_version 1263374 (0.0011) [2023-12-27 00:34:42,508][105620] Updated weights for policy 1, policy_version 1263384 (0.0010) [2023-12-27 00:34:43,102][105692] Updated weights for policy 0, policy_version 1262234 (0.0009) [2023-12-27 00:34:43,106][105620] Updated weights for policy 1, policy_version 1263394 (0.0010) [2023-12-27 00:34:43,156][105692] Updated weights for policy 0, policy_version 1262244 (0.0009) [2023-12-27 00:34:43,159][105620] Updated weights for policy 1, policy_version 1263404 (0.0005) [2023-12-27 00:34:43,216][105620] Updated weights for policy 1, policy_version 1263414 (0.0005) [2023-12-27 00:34:43,217][105692] Updated weights for policy 0, policy_version 1262254 (0.0009) [2023-12-27 00:34:43,277][105620] Updated weights for policy 1, policy_version 1263424 (0.0010) [2023-12-27 00:34:43,973][105620] Updated weights for policy 1, policy_version 1263434 (0.0011) [2023-12-27 00:34:43,995][105692] Updated weights for policy 0, policy_version 1262264 (0.0006) [2023-12-27 00:34:44,035][105620] Updated weights for policy 1, policy_version 1263444 (0.0010) [2023-12-27 00:34:44,053][105692] Updated weights for policy 0, policy_version 1262274 (0.0006) [2023-12-27 00:34:44,086][105620] Updated weights for policy 1, policy_version 1263454 (0.0010) [2023-12-27 00:34:44,097][105692] Updated weights for policy 0, policy_version 1262284 (0.0006) [2023-12-27 00:34:44,749][105620] Updated weights for policy 1, policy_version 1263464 (0.0010) [2023-12-27 00:34:44,808][105620] Updated weights for policy 1, policy_version 1263474 (0.0010) [2023-12-27 00:34:44,870][105620] Updated weights for policy 1, policy_version 1263484 (0.0010) [2023-12-27 00:34:44,914][105692] Updated weights for policy 0, policy_version 1262294 (0.0009) [2023-12-27 00:34:44,979][105692] Updated weights for policy 0, policy_version 1262304 (0.0008) [2023-12-27 00:34:45,047][105692] Updated weights for policy 0, policy_version 1262314 (0.0008) [2023-12-27 00:34:45,628][105620] Updated weights for policy 1, policy_version 1263494 (0.0010) [2023-12-27 00:34:45,672][105620] Updated weights for policy 1, policy_version 1263504 (0.0010) [2023-12-27 00:34:45,719][105620] Updated weights for policy 1, policy_version 1263514 (0.0010) [2023-12-27 00:34:45,808][105692] Updated weights for policy 0, policy_version 1262324 (0.0008) [2023-12-27 00:34:45,862][105692] Updated weights for policy 0, policy_version 1262334 (0.0008) [2023-12-27 00:34:45,910][105692] Updated weights for policy 0, policy_version 1262344 (0.0007) [2023-12-27 00:34:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19114.7, 300 sec: 19383.1). Total num frames: 646717440. Throughput: 0: 9583.2, 1: 9600.6. Samples: 646684836. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:34:46,062][104569] Avg episode reward: [(0, '8191.788'), (1, '9080.418')] [2023-12-27 00:34:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001263520_323502080.pth... [2023-12-27 00:34:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001262352_323215360.pth... [2023-12-27 00:34:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001262400_323215360.pth [2023-12-27 00:34:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001261232_322928640.pth [2023-12-27 00:34:46,455][105620] Updated weights for policy 1, policy_version 1263524 (0.0009) [2023-12-27 00:34:46,503][105620] Updated weights for policy 1, policy_version 1263534 (0.0008) [2023-12-27 00:34:46,552][105620] Updated weights for policy 1, policy_version 1263544 (0.0010) [2023-12-27 00:34:46,718][105692] Updated weights for policy 0, policy_version 1262354 (0.0009) [2023-12-27 00:34:46,776][105692] Updated weights for policy 0, policy_version 1262365 (0.0010) [2023-12-27 00:34:46,836][105692] Updated weights for policy 0, policy_version 1262376 (0.0009) [2023-12-27 00:34:47,248][105620] Updated weights for policy 1, policy_version 1263554 (0.0010) [2023-12-27 00:34:47,306][105620] Updated weights for policy 1, policy_version 1263564 (0.0010) [2023-12-27 00:34:47,371][105620] Updated weights for policy 1, policy_version 1263574 (0.0008) [2023-12-27 00:34:47,434][105620] Updated weights for policy 1, policy_version 1263584 (0.0007) [2023-12-27 00:34:47,652][105692] Updated weights for policy 0, policy_version 1262386 (0.0008) [2023-12-27 00:34:47,713][105692] Updated weights for policy 0, policy_version 1262396 (0.0009) [2023-12-27 00:34:47,760][105692] Updated weights for policy 0, policy_version 1262406 (0.0009) [2023-12-27 00:34:47,810][105692] Updated weights for policy 0, policy_version 1262416 (0.0009) [2023-12-27 00:34:48,074][105620] Updated weights for policy 1, policy_version 1263594 (0.0008) [2023-12-27 00:34:48,131][105620] Updated weights for policy 1, policy_version 1263604 (0.0009) [2023-12-27 00:34:48,182][105620] Updated weights for policy 1, policy_version 1263614 (0.0008) [2023-12-27 00:34:48,581][105692] Updated weights for policy 0, policy_version 1262426 (0.0009) [2023-12-27 00:34:48,640][105692] Updated weights for policy 0, policy_version 1262436 (0.0009) [2023-12-27 00:34:48,697][105692] Updated weights for policy 0, policy_version 1262446 (0.0009) [2023-12-27 00:34:48,976][105620] Updated weights for policy 1, policy_version 1263624 (0.0009) [2023-12-27 00:34:49,041][105620] Updated weights for policy 1, policy_version 1263634 (0.0009) [2023-12-27 00:34:49,102][105620] Updated weights for policy 1, policy_version 1263644 (0.0009) [2023-12-27 00:34:49,421][105692] Updated weights for policy 0, policy_version 1262456 (0.0009) [2023-12-27 00:34:49,476][105692] Updated weights for policy 0, policy_version 1262466 (0.0009) [2023-12-27 00:34:49,522][105692] Updated weights for policy 0, policy_version 1262476 (0.0008) [2023-12-27 00:34:49,869][105620] Updated weights for policy 1, policy_version 1263654 (0.0009) [2023-12-27 00:34:49,935][105620] Updated weights for policy 1, policy_version 1263664 (0.0009) [2023-12-27 00:34:50,001][105620] Updated weights for policy 1, policy_version 1263674 (0.0009) [2023-12-27 00:34:50,226][105692] Updated weights for policy 0, policy_version 1262486 (0.0007) [2023-12-27 00:34:50,293][105692] Updated weights for policy 0, policy_version 1262496 (0.0006) [2023-12-27 00:34:50,358][105692] Updated weights for policy 0, policy_version 1262506 (0.0006) [2023-12-27 00:34:50,834][105620] Updated weights for policy 1, policy_version 1263685 (0.0010) [2023-12-27 00:34:50,893][105620] Updated weights for policy 1, policy_version 1263695 (0.0009) [2023-12-27 00:34:50,952][105620] Updated weights for policy 1, policy_version 1263705 (0.0009) [2023-12-27 00:34:51,017][105692] Updated weights for policy 0, policy_version 1262516 (0.0007) [2023-12-27 00:34:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19327.6). Total num frames: 646807552. Throughput: 0: 9498.5, 1: 9608.1. Samples: 646796924. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:34:51,062][104569] Avg episode reward: [(0, '8180.176'), (1, '8807.426')] [2023-12-27 00:34:51,068][105692] Updated weights for policy 0, policy_version 1262526 (0.0007) [2023-12-27 00:34:51,140][105692] Updated weights for policy 0, policy_version 1262536 (0.0007) [2023-12-27 00:34:51,627][105620] Updated weights for policy 1, policy_version 1263715 (0.0009) [2023-12-27 00:34:51,697][105620] Updated weights for policy 1, policy_version 1263725 (0.0007) [2023-12-27 00:34:51,766][105620] Updated weights for policy 1, policy_version 1263735 (0.0007) [2023-12-27 00:34:51,966][105692] Updated weights for policy 0, policy_version 1262546 (0.0007) [2023-12-27 00:34:52,018][105692] Updated weights for policy 0, policy_version 1262556 (0.0005) [2023-12-27 00:34:52,083][105692] Updated weights for policy 0, policy_version 1262566 (0.0009) [2023-12-27 00:34:52,145][105692] Updated weights for policy 0, policy_version 1262576 (0.0009) [2023-12-27 00:34:52,492][105620] Updated weights for policy 1, policy_version 1263745 (0.0009) [2023-12-27 00:34:52,553][105620] Updated weights for policy 1, policy_version 1263755 (0.0009) [2023-12-27 00:34:52,615][105620] Updated weights for policy 1, policy_version 1263765 (0.0009) [2023-12-27 00:34:52,671][105620] Updated weights for policy 1, policy_version 1263775 (0.0005) [2023-12-27 00:34:52,849][105692] Updated weights for policy 0, policy_version 1262586 (0.0008) [2023-12-27 00:34:52,912][105692] Updated weights for policy 0, policy_version 1262596 (0.0008) [2023-12-27 00:34:52,976][105692] Updated weights for policy 0, policy_version 1262606 (0.0008) [2023-12-27 00:34:53,308][105620] Updated weights for policy 1, policy_version 1263785 (0.0005) [2023-12-27 00:34:53,362][105620] Updated weights for policy 1, policy_version 1263795 (0.0007) [2023-12-27 00:34:53,418][105620] Updated weights for policy 1, policy_version 1263805 (0.0009) [2023-12-27 00:34:53,587][105692] Updated weights for policy 0, policy_version 1262616 (0.0005) [2023-12-27 00:34:53,636][105692] Updated weights for policy 0, policy_version 1262626 (0.0005) [2023-12-27 00:34:53,684][105692] Updated weights for policy 0, policy_version 1262636 (0.0005) [2023-12-27 00:34:54,167][105620] Updated weights for policy 1, policy_version 1263815 (0.0009) [2023-12-27 00:34:54,230][105620] Updated weights for policy 1, policy_version 1263825 (0.0008) [2023-12-27 00:34:54,295][105620] Updated weights for policy 1, policy_version 1263835 (0.0008) [2023-12-27 00:34:54,309][105692] Updated weights for policy 0, policy_version 1262646 (0.0006) [2023-12-27 00:34:54,371][105692] Updated weights for policy 0, policy_version 1262656 (0.0009) [2023-12-27 00:34:54,428][105692] Updated weights for policy 0, policy_version 1262666 (0.0010) [2023-12-27 00:34:54,913][105620] Updated weights for policy 1, policy_version 1263845 (0.0007) [2023-12-27 00:34:54,974][105620] Updated weights for policy 1, policy_version 1263855 (0.0007) [2023-12-27 00:34:55,026][105620] Updated weights for policy 1, policy_version 1263865 (0.0009) [2023-12-27 00:34:55,213][105692] Updated weights for policy 0, policy_version 1262676 (0.0008) [2023-12-27 00:34:55,261][105692] Updated weights for policy 0, policy_version 1262686 (0.0008) [2023-12-27 00:34:55,310][105692] Updated weights for policy 0, policy_version 1262696 (0.0008) [2023-12-27 00:34:55,676][105620] Updated weights for policy 1, policy_version 1263875 (0.0005) [2023-12-27 00:34:55,738][105620] Updated weights for policy 1, policy_version 1263885 (0.0005) [2023-12-27 00:34:55,795][105620] Updated weights for policy 1, policy_version 1263895 (0.0009) [2023-12-27 00:34:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19355.3). Total num frames: 646905856. Throughput: 0: 9536.5, 1: 9612.9. Samples: 646915472. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:34:56,062][104569] Avg episode reward: [(0, '8192.579'), (1, '8990.065')] [2023-12-27 00:34:56,165][105692] Updated weights for policy 0, policy_version 1262706 (0.0008) [2023-12-27 00:34:56,224][105692] Updated weights for policy 0, policy_version 1262716 (0.0007) [2023-12-27 00:34:56,272][105692] Updated weights for policy 0, policy_version 1262727 (0.0008) [2023-12-27 00:34:56,372][105620] Updated weights for policy 1, policy_version 1263905 (0.0010) [2023-12-27 00:34:56,433][105620] Updated weights for policy 1, policy_version 1263915 (0.0010) [2023-12-27 00:34:56,488][105620] Updated weights for policy 1, policy_version 1263925 (0.0010) [2023-12-27 00:34:56,543][105620] Updated weights for policy 1, policy_version 1263935 (0.0010) [2023-12-27 00:34:56,869][105692] Updated weights for policy 0, policy_version 1262737 (0.0006) [2023-12-27 00:34:56,921][105692] Updated weights for policy 0, policy_version 1262748 (0.0010) [2023-12-27 00:34:56,970][105692] Updated weights for policy 0, policy_version 1262758 (0.0010) [2023-12-27 00:34:57,020][105692] Updated weights for policy 0, policy_version 1262768 (0.0010) [2023-12-27 00:34:57,230][105620] Updated weights for policy 1, policy_version 1263945 (0.0006) [2023-12-27 00:34:57,286][105620] Updated weights for policy 1, policy_version 1263955 (0.0005) [2023-12-27 00:34:57,345][105620] Updated weights for policy 1, policy_version 1263965 (0.0007) [2023-12-27 00:34:57,752][105692] Updated weights for policy 0, policy_version 1262778 (0.0005) [2023-12-27 00:34:57,810][105692] Updated weights for policy 0, policy_version 1262788 (0.0005) [2023-12-27 00:34:57,863][105692] Updated weights for policy 0, policy_version 1262798 (0.0005) [2023-12-27 00:34:57,929][105620] Updated weights for policy 1, policy_version 1263975 (0.0007) [2023-12-27 00:34:57,982][105620] Updated weights for policy 1, policy_version 1263985 (0.0009) [2023-12-27 00:34:58,046][105620] Updated weights for policy 1, policy_version 1263995 (0.0008) [2023-12-27 00:34:58,559][105692] Updated weights for policy 0, policy_version 1262808 (0.0008) [2023-12-27 00:34:58,618][105692] Updated weights for policy 0, policy_version 1262818 (0.0009) [2023-12-27 00:34:58,678][105692] Updated weights for policy 0, policy_version 1262828 (0.0008) [2023-12-27 00:34:58,743][105620] Updated weights for policy 1, policy_version 1264005 (0.0009) [2023-12-27 00:34:58,808][105620] Updated weights for policy 1, policy_version 1264015 (0.0009) [2023-12-27 00:34:58,868][105620] Updated weights for policy 1, policy_version 1264025 (0.0010) [2023-12-27 00:34:59,319][105692] Updated weights for policy 0, policy_version 1262838 (0.0006) [2023-12-27 00:34:59,385][105692] Updated weights for policy 0, policy_version 1262848 (0.0007) [2023-12-27 00:34:59,451][105692] Updated weights for policy 0, policy_version 1262858 (0.0005) [2023-12-27 00:34:59,637][105620] Updated weights for policy 1, policy_version 1264035 (0.0010) [2023-12-27 00:34:59,695][105620] Updated weights for policy 1, policy_version 1264045 (0.0010) [2023-12-27 00:34:59,753][105620] Updated weights for policy 1, policy_version 1264055 (0.0010) [2023-12-27 00:35:00,085][105692] Updated weights for policy 0, policy_version 1262868 (0.0006) [2023-12-27 00:35:00,141][105692] Updated weights for policy 0, policy_version 1262878 (0.0005) [2023-12-27 00:35:00,196][105692] Updated weights for policy 0, policy_version 1262888 (0.0005) [2023-12-27 00:35:00,495][105620] Updated weights for policy 1, policy_version 1264065 (0.0008) [2023-12-27 00:35:00,543][105620] Updated weights for policy 1, policy_version 1264075 (0.0009) [2023-12-27 00:35:00,586][105620] Updated weights for policy 1, policy_version 1264085 (0.0006) [2023-12-27 00:35:00,633][105620] Updated weights for policy 1, policy_version 1264095 (0.0005) [2023-12-27 00:35:00,849][105692] Updated weights for policy 0, policy_version 1262898 (0.0006) [2023-12-27 00:35:00,903][105692] Updated weights for policy 0, policy_version 1262910 (0.0010) [2023-12-27 00:35:00,950][105692] Updated weights for policy 0, policy_version 1262920 (0.0008) [2023-12-27 00:35:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 647012352. Throughput: 0: 9611.1, 1: 9685.8. Samples: 646977332. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:35:01,062][104569] Avg episode reward: [(0, '8817.505'), (1, '9080.273')] [2023-12-27 00:35:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001262928_323362816.pth... [2023-12-27 00:35:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001264096_323649536.pth... [2023-12-27 00:35:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001261776_323067904.pth [2023-12-27 00:35:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001262944_323354624.pth [2023-12-27 00:35:01,206][105620] Updated weights for policy 1, policy_version 1264105 (0.0006) [2023-12-27 00:35:01,271][105620] Updated weights for policy 1, policy_version 1264115 (0.0007) [2023-12-27 00:35:01,327][105620] Updated weights for policy 1, policy_version 1264125 (0.0007) [2023-12-27 00:35:01,816][105692] Updated weights for policy 0, policy_version 1262931 (0.0009) [2023-12-27 00:35:01,873][105692] Updated weights for policy 0, policy_version 1262941 (0.0013) [2023-12-27 00:35:01,928][105692] Updated weights for policy 0, policy_version 1262952 (0.0009) [2023-12-27 00:35:01,984][105620] Updated weights for policy 1, policy_version 1264135 (0.0008) [2023-12-27 00:35:02,049][105620] Updated weights for policy 1, policy_version 1264145 (0.0009) [2023-12-27 00:35:02,109][105620] Updated weights for policy 1, policy_version 1264155 (0.0011) [2023-12-27 00:35:02,688][105692] Updated weights for policy 0, policy_version 1262963 (0.0010) [2023-12-27 00:35:02,694][105620] Updated weights for policy 1, policy_version 1264165 (0.0008) [2023-12-27 00:35:02,747][105692] Updated weights for policy 0, policy_version 1262973 (0.0011) [2023-12-27 00:35:02,750][105620] Updated weights for policy 1, policy_version 1264175 (0.0008) [2023-12-27 00:35:02,811][105620] Updated weights for policy 1, policy_version 1264185 (0.0005) [2023-12-27 00:35:02,814][105692] Updated weights for policy 0, policy_version 1262983 (0.0010) [2023-12-27 00:35:03,363][105620] Updated weights for policy 1, policy_version 1264195 (0.0006) [2023-12-27 00:35:03,409][105620] Updated weights for policy 1, policy_version 1264205 (0.0005) [2023-12-27 00:35:03,455][105620] Updated weights for policy 1, policy_version 1264215 (0.0005) [2023-12-27 00:35:03,601][105692] Updated weights for policy 0, policy_version 1262993 (0.0011) [2023-12-27 00:35:03,649][105692] Updated weights for policy 0, policy_version 1263003 (0.0009) [2023-12-27 00:35:03,702][105692] Updated weights for policy 0, policy_version 1263013 (0.0009) [2023-12-27 00:35:03,753][105692] Updated weights for policy 0, policy_version 1263023 (0.0009) [2023-12-27 00:35:04,108][105620] Updated weights for policy 1, policy_version 1264225 (0.0005) [2023-12-27 00:35:04,170][105620] Updated weights for policy 1, policy_version 1264235 (0.0006) [2023-12-27 00:35:04,237][105620] Updated weights for policy 1, policy_version 1264245 (0.0006) [2023-12-27 00:35:04,307][105620] Updated weights for policy 1, policy_version 1264255 (0.0006) [2023-12-27 00:35:04,523][105692] Updated weights for policy 0, policy_version 1263033 (0.0010) [2023-12-27 00:35:04,584][105692] Updated weights for policy 0, policy_version 1263043 (0.0008) [2023-12-27 00:35:04,652][105692] Updated weights for policy 0, policy_version 1263053 (0.0009) [2023-12-27 00:35:04,941][105620] Updated weights for policy 1, policy_version 1264265 (0.0008) [2023-12-27 00:35:04,996][105620] Updated weights for policy 1, policy_version 1264275 (0.0009) [2023-12-27 00:35:05,051][105620] Updated weights for policy 1, policy_version 1264285 (0.0009) [2023-12-27 00:35:05,416][105692] Updated weights for policy 0, policy_version 1263063 (0.0009) [2023-12-27 00:35:05,478][105692] Updated weights for policy 0, policy_version 1263073 (0.0010) [2023-12-27 00:35:05,540][105692] Updated weights for policy 0, policy_version 1263083 (0.0009) [2023-12-27 00:35:05,713][105620] Updated weights for policy 1, policy_version 1264295 (0.0009) [2023-12-27 00:35:05,763][105620] Updated weights for policy 1, policy_version 1264305 (0.0008) [2023-12-27 00:35:05,820][105620] Updated weights for policy 1, policy_version 1264315 (0.0009) [2023-12-27 00:35:06,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 647110656. Throughput: 0: 9642.5, 1: 9830.5. Samples: 647099088. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:35:06,063][104569] Avg episode reward: [(0, '8716.733'), (1, '9080.450')] [2023-12-27 00:35:06,308][105692] Updated weights for policy 0, policy_version 1263093 (0.0008) [2023-12-27 00:35:06,355][105692] Updated weights for policy 0, policy_version 1263103 (0.0006) [2023-12-27 00:35:06,421][105692] Updated weights for policy 0, policy_version 1263113 (0.0006) [2023-12-27 00:35:06,534][105620] Updated weights for policy 1, policy_version 1264325 (0.0009) [2023-12-27 00:35:06,590][105620] Updated weights for policy 1, policy_version 1264335 (0.0009) [2023-12-27 00:35:06,643][105620] Updated weights for policy 1, policy_version 1264346 (0.0010) [2023-12-27 00:35:07,092][105692] Updated weights for policy 0, policy_version 1263123 (0.0006) [2023-12-27 00:35:07,149][105692] Updated weights for policy 0, policy_version 1263133 (0.0009) [2023-12-27 00:35:07,209][105692] Updated weights for policy 0, policy_version 1263143 (0.0009) [2023-12-27 00:35:07,380][105620] Updated weights for policy 1, policy_version 1264356 (0.0008) [2023-12-27 00:35:07,432][105620] Updated weights for policy 1, policy_version 1264366 (0.0009) [2023-12-27 00:35:07,488][105620] Updated weights for policy 1, policy_version 1264376 (0.0009) [2023-12-27 00:35:08,024][105692] Updated weights for policy 0, policy_version 1263153 (0.0009) [2023-12-27 00:35:08,076][105692] Updated weights for policy 0, policy_version 1263163 (0.0006) [2023-12-27 00:35:08,127][105692] Updated weights for policy 0, policy_version 1263173 (0.0005) [2023-12-27 00:35:08,182][105692] Updated weights for policy 0, policy_version 1263183 (0.0006) [2023-12-27 00:35:08,185][105620] Updated weights for policy 1, policy_version 1264386 (0.0008) [2023-12-27 00:35:08,236][105620] Updated weights for policy 1, policy_version 1264396 (0.0005) [2023-12-27 00:35:08,286][105620] Updated weights for policy 1, policy_version 1264406 (0.0006) [2023-12-27 00:35:08,352][105620] Updated weights for policy 1, policy_version 1264416 (0.0010) [2023-12-27 00:35:08,815][105692] Updated weights for policy 0, policy_version 1263193 (0.0008) [2023-12-27 00:35:08,867][105692] Updated weights for policy 0, policy_version 1263203 (0.0010) [2023-12-27 00:35:08,927][105692] Updated weights for policy 0, policy_version 1263213 (0.0009) [2023-12-27 00:35:09,063][105620] Updated weights for policy 1, policy_version 1264426 (0.0008) [2023-12-27 00:35:09,124][105620] Updated weights for policy 1, policy_version 1264436 (0.0009) [2023-12-27 00:35:09,186][105620] Updated weights for policy 1, policy_version 1264446 (0.0009) [2023-12-27 00:35:09,671][105692] Updated weights for policy 0, policy_version 1263223 (0.0009) [2023-12-27 00:35:09,731][105692] Updated weights for policy 0, policy_version 1263233 (0.0010) [2023-12-27 00:35:09,795][105692] Updated weights for policy 0, policy_version 1263243 (0.0011) [2023-12-27 00:35:09,954][105620] Updated weights for policy 1, policy_version 1264456 (0.0008) [2023-12-27 00:35:10,012][105620] Updated weights for policy 1, policy_version 1264466 (0.0008) [2023-12-27 00:35:10,072][105620] Updated weights for policy 1, policy_version 1264476 (0.0008) [2023-12-27 00:35:10,552][105692] Updated weights for policy 0, policy_version 1263253 (0.0010) [2023-12-27 00:35:10,607][105692] Updated weights for policy 0, policy_version 1263263 (0.0009) [2023-12-27 00:35:10,663][105692] Updated weights for policy 0, policy_version 1263273 (0.0006) [2023-12-27 00:35:10,785][105620] Updated weights for policy 1, policy_version 1264486 (0.0007) [2023-12-27 00:35:10,841][105620] Updated weights for policy 1, policy_version 1264496 (0.0005) [2023-12-27 00:35:10,911][105620] Updated weights for policy 1, policy_version 1264506 (0.0007) [2023-12-27 00:35:11,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 647208960. Throughput: 0: 9620.5, 1: 9926.3. Samples: 647214284. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:35:11,063][104569] Avg episode reward: [(0, '8896.868'), (1, '9173.647')] [2023-12-27 00:35:11,427][105692] Updated weights for policy 0, policy_version 1263283 (0.0006) [2023-12-27 00:35:11,485][105692] Updated weights for policy 0, policy_version 1263293 (0.0007) [2023-12-27 00:35:11,552][105692] Updated weights for policy 0, policy_version 1263303 (0.0009) [2023-12-27 00:35:11,631][105620] Updated weights for policy 1, policy_version 1264516 (0.0009) [2023-12-27 00:35:11,696][105620] Updated weights for policy 1, policy_version 1264526 (0.0009) [2023-12-27 00:35:11,759][105620] Updated weights for policy 1, policy_version 1264536 (0.0009) [2023-12-27 00:35:12,205][105692] Updated weights for policy 0, policy_version 1263313 (0.0010) [2023-12-27 00:35:12,272][105692] Updated weights for policy 0, policy_version 1263323 (0.0009) [2023-12-27 00:35:12,338][105692] Updated weights for policy 0, policy_version 1263333 (0.0008) [2023-12-27 00:35:12,406][105692] Updated weights for policy 0, policy_version 1263343 (0.0009) [2023-12-27 00:35:12,604][105620] Updated weights for policy 1, policy_version 1264546 (0.0009) [2023-12-27 00:35:12,664][105620] Updated weights for policy 1, policy_version 1264556 (0.0008) [2023-12-27 00:35:12,727][105620] Updated weights for policy 1, policy_version 1264566 (0.0009) [2023-12-27 00:35:12,785][105620] Updated weights for policy 1, policy_version 1264576 (0.0009) [2023-12-27 00:35:13,152][105692] Updated weights for policy 0, policy_version 1263353 (0.0007) [2023-12-27 00:35:13,221][105692] Updated weights for policy 0, policy_version 1263363 (0.0008) [2023-12-27 00:35:13,273][105692] Updated weights for policy 0, policy_version 1263373 (0.0010) [2023-12-27 00:35:13,484][105620] Updated weights for policy 1, policy_version 1264586 (0.0005) [2023-12-27 00:35:13,531][105620] Updated weights for policy 1, policy_version 1264596 (0.0007) [2023-12-27 00:35:13,583][105620] Updated weights for policy 1, policy_version 1264606 (0.0007) [2023-12-27 00:35:13,925][105692] Updated weights for policy 0, policy_version 1263383 (0.0010) [2023-12-27 00:35:13,982][105692] Updated weights for policy 0, policy_version 1263393 (0.0006) [2023-12-27 00:35:14,040][105692] Updated weights for policy 0, policy_version 1263403 (0.0005) [2023-12-27 00:35:14,157][105620] Updated weights for policy 1, policy_version 1264616 (0.0005) [2023-12-27 00:35:14,211][105620] Updated weights for policy 1, policy_version 1264626 (0.0006) [2023-12-27 00:35:14,276][105620] Updated weights for policy 1, policy_version 1264636 (0.0005) [2023-12-27 00:35:14,659][105692] Updated weights for policy 0, policy_version 1263413 (0.0009) [2023-12-27 00:35:14,708][105692] Updated weights for policy 0, policy_version 1263423 (0.0010) [2023-12-27 00:35:14,762][105692] Updated weights for policy 0, policy_version 1263433 (0.0010) [2023-12-27 00:35:14,974][105620] Updated weights for policy 1, policy_version 1264646 (0.0007) [2023-12-27 00:35:15,043][105620] Updated weights for policy 1, policy_version 1264656 (0.0006) [2023-12-27 00:35:15,111][105620] Updated weights for policy 1, policy_version 1264666 (0.0006) [2023-12-27 00:35:15,489][105692] Updated weights for policy 0, policy_version 1263443 (0.0008) [2023-12-27 00:35:15,534][105692] Updated weights for policy 0, policy_version 1263453 (0.0010) [2023-12-27 00:35:15,585][105692] Updated weights for policy 0, policy_version 1263463 (0.0010) [2023-12-27 00:35:15,769][105620] Updated weights for policy 1, policy_version 1264676 (0.0008) [2023-12-27 00:35:15,813][105620] Updated weights for policy 1, policy_version 1264686 (0.0010) [2023-12-27 00:35:15,865][105620] Updated weights for policy 1, policy_version 1264696 (0.0010) [2023-12-27 00:35:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 647307264. Throughput: 0: 9573.7, 1: 9944.5. Samples: 647272048. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:35:16,062][104569] Avg episode reward: [(0, '9081.747'), (1, '9082.438')] [2023-12-27 00:35:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001263472_323502080.pth... [2023-12-27 00:35:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001264704_323805184.pth... [2023-12-27 00:35:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001262352_323215360.pth [2023-12-27 00:35:16,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001263520_323502080.pth [2023-12-27 00:35:16,347][105692] Updated weights for policy 0, policy_version 1263473 (0.0010) [2023-12-27 00:35:16,398][105692] Updated weights for policy 0, policy_version 1263483 (0.0010) [2023-12-27 00:35:16,452][105692] Updated weights for policy 0, policy_version 1263493 (0.0010) [2023-12-27 00:35:16,496][105692] Updated weights for policy 0, policy_version 1263503 (0.0010) [2023-12-27 00:35:16,623][105620] Updated weights for policy 1, policy_version 1264706 (0.0010) [2023-12-27 00:35:16,678][105620] Updated weights for policy 1, policy_version 1264716 (0.0008) [2023-12-27 00:35:16,733][105620] Updated weights for policy 1, policy_version 1264726 (0.0008) [2023-12-27 00:35:16,792][105620] Updated weights for policy 1, policy_version 1264736 (0.0007) [2023-12-27 00:35:17,172][105692] Updated weights for policy 0, policy_version 1263513 (0.0010) [2023-12-27 00:35:17,223][105692] Updated weights for policy 0, policy_version 1263523 (0.0010) [2023-12-27 00:35:17,268][105692] Updated weights for policy 0, policy_version 1263533 (0.0010) [2023-12-27 00:35:17,529][105620] Updated weights for policy 1, policy_version 1264746 (0.0005) [2023-12-27 00:35:17,598][105620] Updated weights for policy 1, policy_version 1264756 (0.0006) [2023-12-27 00:35:17,666][105620] Updated weights for policy 1, policy_version 1264766 (0.0005) [2023-12-27 00:35:18,055][105692] Updated weights for policy 0, policy_version 1263543 (0.0010) [2023-12-27 00:35:18,119][105692] Updated weights for policy 0, policy_version 1263553 (0.0009) [2023-12-27 00:35:18,176][105692] Updated weights for policy 0, policy_version 1263563 (0.0008) [2023-12-27 00:35:18,227][105620] Updated weights for policy 1, policy_version 1264776 (0.0005) [2023-12-27 00:35:18,289][105620] Updated weights for policy 1, policy_version 1264786 (0.0005) [2023-12-27 00:35:18,357][105620] Updated weights for policy 1, policy_version 1264796 (0.0008) [2023-12-27 00:35:18,800][105692] Updated weights for policy 0, policy_version 1263573 (0.0008) [2023-12-27 00:35:18,862][105692] Updated weights for policy 0, policy_version 1263583 (0.0010) [2023-12-27 00:35:18,925][105692] Updated weights for policy 0, policy_version 1263593 (0.0010) [2023-12-27 00:35:19,069][105620] Updated weights for policy 1, policy_version 1264806 (0.0009) [2023-12-27 00:35:19,130][105620] Updated weights for policy 1, policy_version 1264816 (0.0011) [2023-12-27 00:35:19,190][105620] Updated weights for policy 1, policy_version 1264826 (0.0011) [2023-12-27 00:35:19,695][105692] Updated weights for policy 0, policy_version 1263603 (0.0010) [2023-12-27 00:35:19,757][105692] Updated weights for policy 0, policy_version 1263613 (0.0010) [2023-12-27 00:35:19,817][105692] Updated weights for policy 0, policy_version 1263623 (0.0010) [2023-12-27 00:35:19,983][105620] Updated weights for policy 1, policy_version 1264836 (0.0010) [2023-12-27 00:35:20,045][105620] Updated weights for policy 1, policy_version 1264846 (0.0008) [2023-12-27 00:35:20,104][105620] Updated weights for policy 1, policy_version 1264856 (0.0008) [2023-12-27 00:35:20,529][105692] Updated weights for policy 0, policy_version 1263633 (0.0010) [2023-12-27 00:35:20,592][105692] Updated weights for policy 0, policy_version 1263643 (0.0008) [2023-12-27 00:35:20,656][105692] Updated weights for policy 0, policy_version 1263653 (0.0011) [2023-12-27 00:35:20,706][105692] Updated weights for policy 0, policy_version 1263663 (0.0010) [2023-12-27 00:35:20,829][105620] Updated weights for policy 1, policy_version 1264866 (0.0010) [2023-12-27 00:35:20,900][105620] Updated weights for policy 1, policy_version 1264876 (0.0009) [2023-12-27 00:35:20,970][105620] Updated weights for policy 1, policy_version 1264886 (0.0011) [2023-12-27 00:35:21,033][105620] Updated weights for policy 1, policy_version 1264896 (0.0010) [2023-12-27 00:35:21,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 647405568. Throughput: 0: 9646.8, 1: 9956.4. Samples: 647392216. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:35:21,062][104569] Avg episode reward: [(0, '8715.862'), (1, '9082.466')] [2023-12-27 00:35:21,425][105692] Updated weights for policy 0, policy_version 1263673 (0.0008) [2023-12-27 00:35:21,490][105692] Updated weights for policy 0, policy_version 1263683 (0.0008) [2023-12-27 00:35:21,560][105692] Updated weights for policy 0, policy_version 1263693 (0.0008) [2023-12-27 00:35:21,764][105620] Updated weights for policy 1, policy_version 1264906 (0.0010) [2023-12-27 00:35:21,812][105620] Updated weights for policy 1, policy_version 1264916 (0.0008) [2023-12-27 00:35:21,863][105620] Updated weights for policy 1, policy_version 1264926 (0.0008) [2023-12-27 00:35:22,302][105692] Updated weights for policy 0, policy_version 1263703 (0.0010) [2023-12-27 00:35:22,369][105692] Updated weights for policy 0, policy_version 1263713 (0.0008) [2023-12-27 00:35:22,433][105692] Updated weights for policy 0, policy_version 1263723 (0.0005) [2023-12-27 00:35:22,662][105620] Updated weights for policy 1, policy_version 1264936 (0.0010) [2023-12-27 00:35:22,725][105620] Updated weights for policy 1, policy_version 1264946 (0.0011) [2023-12-27 00:35:22,787][105620] Updated weights for policy 1, policy_version 1264956 (0.0009) [2023-12-27 00:35:23,045][105692] Updated weights for policy 0, policy_version 1263733 (0.0008) [2023-12-27 00:35:23,108][105692] Updated weights for policy 0, policy_version 1263743 (0.0010) [2023-12-27 00:35:23,168][105692] Updated weights for policy 0, policy_version 1263753 (0.0011) [2023-12-27 00:35:23,531][105620] Updated weights for policy 1, policy_version 1264966 (0.0008) [2023-12-27 00:35:23,583][105620] Updated weights for policy 1, policy_version 1264976 (0.0008) [2023-12-27 00:35:23,635][105620] Updated weights for policy 1, policy_version 1264986 (0.0008) [2023-12-27 00:35:23,908][105692] Updated weights for policy 0, policy_version 1263763 (0.0011) [2023-12-27 00:35:23,958][105692] Updated weights for policy 0, policy_version 1263773 (0.0006) [2023-12-27 00:35:24,011][105692] Updated weights for policy 0, policy_version 1263783 (0.0005) [2023-12-27 00:35:24,451][105620] Updated weights for policy 1, policy_version 1264996 (0.0008) [2023-12-27 00:35:24,509][105620] Updated weights for policy 1, policy_version 1265006 (0.0006) [2023-12-27 00:35:24,562][105620] Updated weights for policy 1, policy_version 1265016 (0.0007) [2023-12-27 00:35:24,654][105692] Updated weights for policy 0, policy_version 1263793 (0.0006) [2023-12-27 00:35:24,713][105692] Updated weights for policy 0, policy_version 1263803 (0.0009) [2023-12-27 00:35:24,768][105692] Updated weights for policy 0, policy_version 1263813 (0.0005) [2023-12-27 00:35:24,832][105692] Updated weights for policy 0, policy_version 1263823 (0.0007) [2023-12-27 00:35:25,189][105620] Updated weights for policy 1, policy_version 1265026 (0.0007) [2023-12-27 00:35:25,245][105620] Updated weights for policy 1, policy_version 1265036 (0.0008) [2023-12-27 00:35:25,306][105620] Updated weights for policy 1, policy_version 1265046 (0.0008) [2023-12-27 00:35:25,371][105620] Updated weights for policy 1, policy_version 1265056 (0.0006) [2023-12-27 00:35:25,571][105692] Updated weights for policy 0, policy_version 1263833 (0.0010) [2023-12-27 00:35:25,628][105692] Updated weights for policy 0, policy_version 1263843 (0.0010) [2023-12-27 00:35:25,696][105692] Updated weights for policy 0, policy_version 1263853 (0.0010) [2023-12-27 00:35:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 647495680. Throughput: 0: 9659.4, 1: 9927.9. Samples: 647507220. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:35:26,062][104569] Avg episode reward: [(0, '8987.836'), (1, '8806.878')] [2023-12-27 00:35:26,080][105620] Updated weights for policy 1, policy_version 1265066 (0.0010) [2023-12-27 00:35:26,128][105620] Updated weights for policy 1, policy_version 1265076 (0.0010) [2023-12-27 00:35:26,173][105620] Updated weights for policy 1, policy_version 1265086 (0.0010) [2023-12-27 00:35:26,283][105692] Updated weights for policy 0, policy_version 1263863 (0.0010) [2023-12-27 00:35:26,334][105692] Updated weights for policy 0, policy_version 1263873 (0.0010) [2023-12-27 00:35:26,385][105692] Updated weights for policy 0, policy_version 1263883 (0.0010) [2023-12-27 00:35:26,939][105620] Updated weights for policy 1, policy_version 1265096 (0.0010) [2023-12-27 00:35:26,997][105620] Updated weights for policy 1, policy_version 1265106 (0.0010) [2023-12-27 00:35:27,051][105620] Updated weights for policy 1, policy_version 1265116 (0.0010) [2023-12-27 00:35:27,135][105692] Updated weights for policy 0, policy_version 1263893 (0.0010) [2023-12-27 00:35:27,182][105692] Updated weights for policy 0, policy_version 1263903 (0.0010) [2023-12-27 00:35:27,226][105692] Updated weights for policy 0, policy_version 1263913 (0.0010) [2023-12-27 00:35:27,691][105620] Updated weights for policy 1, policy_version 1265126 (0.0007) [2023-12-27 00:35:27,735][105620] Updated weights for policy 1, policy_version 1265136 (0.0005) [2023-12-27 00:35:27,801][105692] Updated weights for policy 0, policy_version 1263923 (0.0009) [2023-12-27 00:35:27,804][105620] Updated weights for policy 1, policy_version 1265146 (0.0005) [2023-12-27 00:35:27,852][105692] Updated weights for policy 0, policy_version 1263933 (0.0005) [2023-12-27 00:35:27,904][105692] Updated weights for policy 0, policy_version 1263943 (0.0005) [2023-12-27 00:35:28,412][105620] Updated weights for policy 1, policy_version 1265156 (0.0009) [2023-12-27 00:35:28,470][105620] Updated weights for policy 1, policy_version 1265166 (0.0011) [2023-12-27 00:35:28,512][105692] Updated weights for policy 0, policy_version 1263953 (0.0006) [2023-12-27 00:35:28,520][105620] Updated weights for policy 1, policy_version 1265176 (0.0012) [2023-12-27 00:35:28,572][105692] Updated weights for policy 0, policy_version 1263963 (0.0008) [2023-12-27 00:35:28,637][105692] Updated weights for policy 0, policy_version 1263973 (0.0008) [2023-12-27 00:35:28,694][105692] Updated weights for policy 0, policy_version 1263983 (0.0008) [2023-12-27 00:35:29,267][105620] Updated weights for policy 1, policy_version 1265186 (0.0010) [2023-12-27 00:35:29,322][105620] Updated weights for policy 1, policy_version 1265196 (0.0010) [2023-12-27 00:35:29,386][105620] Updated weights for policy 1, policy_version 1265206 (0.0010) [2023-12-27 00:35:29,394][105692] Updated weights for policy 0, policy_version 1263993 (0.0009) [2023-12-27 00:35:29,437][105620] Updated weights for policy 1, policy_version 1265216 (0.0010) [2023-12-27 00:35:29,450][105692] Updated weights for policy 0, policy_version 1264003 (0.0009) [2023-12-27 00:35:29,499][105692] Updated weights for policy 0, policy_version 1264013 (0.0008) [2023-12-27 00:35:30,234][105620] Updated weights for policy 1, policy_version 1265226 (0.0009) [2023-12-27 00:35:30,249][105692] Updated weights for policy 0, policy_version 1264023 (0.0006) [2023-12-27 00:35:30,291][105620] Updated weights for policy 1, policy_version 1265236 (0.0009) [2023-12-27 00:35:30,302][105692] Updated weights for policy 0, policy_version 1264033 (0.0006) [2023-12-27 00:35:30,348][105620] Updated weights for policy 1, policy_version 1265246 (0.0007) [2023-12-27 00:35:30,364][105692] Updated weights for policy 0, policy_version 1264043 (0.0007) [2023-12-27 00:35:31,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19383.1). Total num frames: 647593984. Throughput: 0: 9747.8, 1: 9933.8. Samples: 647570512. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:35:31,063][104569] Avg episode reward: [(0, '9262.437'), (1, '8715.264')] [2023-12-27 00:35:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001264048_323649536.pth... [2023-12-27 00:35:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001262928_323362816.pth [2023-12-27 00:35:31,088][105620] Updated weights for policy 1, policy_version 1265256 (0.0009) [2023-12-27 00:35:31,120][105692] Updated weights for policy 0, policy_version 1264053 (0.0008) [2023-12-27 00:35:31,149][105620] Updated weights for policy 1, policy_version 1265266 (0.0007) [2023-12-27 00:35:31,182][105692] Updated weights for policy 0, policy_version 1264063 (0.0009) [2023-12-27 00:35:31,205][105620] Updated weights for policy 1, policy_version 1265276 (0.0006) [2023-12-27 00:35:31,226][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001265280_323952640.pth... [2023-12-27 00:35:31,235][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001264096_323649536.pth [2023-12-27 00:35:31,244][105692] Updated weights for policy 0, policy_version 1264073 (0.0007) [2023-12-27 00:35:31,953][105620] Updated weights for policy 1, policy_version 1265286 (0.0008) [2023-12-27 00:35:31,974][105692] Updated weights for policy 0, policy_version 1264083 (0.0010) [2023-12-27 00:35:32,006][105620] Updated weights for policy 1, policy_version 1265296 (0.0008) [2023-12-27 00:35:32,033][105692] Updated weights for policy 0, policy_version 1264093 (0.0008) [2023-12-27 00:35:32,055][105620] Updated weights for policy 1, policy_version 1265306 (0.0008) [2023-12-27 00:35:32,082][105692] Updated weights for policy 0, policy_version 1264103 (0.0006) [2023-12-27 00:35:32,807][105692] Updated weights for policy 0, policy_version 1264113 (0.0009) [2023-12-27 00:35:32,851][105620] Updated weights for policy 1, policy_version 1265316 (0.0006) [2023-12-27 00:35:32,872][105692] Updated weights for policy 0, policy_version 1264123 (0.0009) [2023-12-27 00:35:32,895][105620] Updated weights for policy 1, policy_version 1265326 (0.0007) [2023-12-27 00:35:32,929][105692] Updated weights for policy 0, policy_version 1264133 (0.0009) [2023-12-27 00:35:32,947][105620] Updated weights for policy 1, policy_version 1265336 (0.0007) [2023-12-27 00:35:32,974][105692] Updated weights for policy 0, policy_version 1264143 (0.0007) [2023-12-27 00:35:33,698][105692] Updated weights for policy 0, policy_version 1264153 (0.0005) [2023-12-27 00:35:33,709][105620] Updated weights for policy 1, policy_version 1265346 (0.0007) [2023-12-27 00:35:33,757][105620] Updated weights for policy 1, policy_version 1265356 (0.0008) [2023-12-27 00:35:33,758][105692] Updated weights for policy 0, policy_version 1264163 (0.0005) [2023-12-27 00:35:33,808][105692] Updated weights for policy 0, policy_version 1264173 (0.0006) [2023-12-27 00:35:33,814][105620] Updated weights for policy 1, policy_version 1265366 (0.0009) [2023-12-27 00:35:33,871][105620] Updated weights for policy 1, policy_version 1265376 (0.0009) [2023-12-27 00:35:34,397][105692] Updated weights for policy 0, policy_version 1264183 (0.0009) [2023-12-27 00:35:34,459][105692] Updated weights for policy 0, policy_version 1264193 (0.0010) [2023-12-27 00:35:34,519][105692] Updated weights for policy 0, policy_version 1264203 (0.0008) [2023-12-27 00:35:34,682][105620] Updated weights for policy 1, policy_version 1265386 (0.0009) [2023-12-27 00:35:34,737][105620] Updated weights for policy 1, policy_version 1265396 (0.0009) [2023-12-27 00:35:34,792][105620] Updated weights for policy 1, policy_version 1265406 (0.0009) [2023-12-27 00:35:35,190][105692] Updated weights for policy 0, policy_version 1264213 (0.0007) [2023-12-27 00:35:35,255][105692] Updated weights for policy 0, policy_version 1264223 (0.0006) [2023-12-27 00:35:35,304][105692] Updated weights for policy 0, policy_version 1264233 (0.0011) [2023-12-27 00:35:35,521][105620] Updated weights for policy 1, policy_version 1265416 (0.0006) [2023-12-27 00:35:35,586][105620] Updated weights for policy 1, policy_version 1265427 (0.0007) [2023-12-27 00:35:35,640][105620] Updated weights for policy 1, policy_version 1265437 (0.0010) [2023-12-27 00:35:35,857][105692] Updated weights for policy 0, policy_version 1264243 (0.0010) [2023-12-27 00:35:35,920][105692] Updated weights for policy 0, policy_version 1264253 (0.0009) [2023-12-27 00:35:35,978][105692] Updated weights for policy 0, policy_version 1264263 (0.0006) [2023-12-27 00:35:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 647700480. Throughput: 0: 9836.1, 1: 9865.2. Samples: 647683484. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:35:36,062][104569] Avg episode reward: [(0, '8990.734'), (1, '9172.299')] [2023-12-27 00:35:36,302][105620] Updated weights for policy 1, policy_version 1265447 (0.0009) [2023-12-27 00:35:36,370][105620] Updated weights for policy 1, policy_version 1265457 (0.0008) [2023-12-27 00:35:36,435][105620] Updated weights for policy 1, policy_version 1265467 (0.0009) [2023-12-27 00:35:36,593][105692] Updated weights for policy 0, policy_version 1264273 (0.0005) [2023-12-27 00:35:36,653][105692] Updated weights for policy 0, policy_version 1264283 (0.0006) [2023-12-27 00:35:36,710][105692] Updated weights for policy 0, policy_version 1264293 (0.0011) [2023-12-27 00:35:36,769][105692] Updated weights for policy 0, policy_version 1264303 (0.0011) [2023-12-27 00:35:37,015][105620] Updated weights for policy 1, policy_version 1265477 (0.0007) [2023-12-27 00:35:37,077][105620] Updated weights for policy 1, policy_version 1265487 (0.0005) [2023-12-27 00:35:37,145][105620] Updated weights for policy 1, policy_version 1265497 (0.0006) [2023-12-27 00:35:37,435][105692] Updated weights for policy 0, policy_version 1264313 (0.0011) [2023-12-27 00:35:37,491][105692] Updated weights for policy 0, policy_version 1264323 (0.0011) [2023-12-27 00:35:37,551][105692] Updated weights for policy 0, policy_version 1264333 (0.0010) [2023-12-27 00:35:37,757][105620] Updated weights for policy 1, policy_version 1265507 (0.0006) [2023-12-27 00:35:37,826][105620] Updated weights for policy 1, policy_version 1265517 (0.0005) [2023-12-27 00:35:37,890][105620] Updated weights for policy 1, policy_version 1265527 (0.0005) [2023-12-27 00:35:38,225][105692] Updated weights for policy 0, policy_version 1264343 (0.0011) [2023-12-27 00:35:38,283][105692] Updated weights for policy 0, policy_version 1264353 (0.0010) [2023-12-27 00:35:38,349][105692] Updated weights for policy 0, policy_version 1264363 (0.0011) [2023-12-27 00:35:38,411][105620] Updated weights for policy 1, policy_version 1265537 (0.0006) [2023-12-27 00:35:38,460][105620] Updated weights for policy 1, policy_version 1265547 (0.0009) [2023-12-27 00:35:38,507][105620] Updated weights for policy 1, policy_version 1265557 (0.0008) [2023-12-27 00:35:38,565][105620] Updated weights for policy 1, policy_version 1265567 (0.0009) [2023-12-27 00:35:38,985][105692] Updated weights for policy 0, policy_version 1264373 (0.0009) [2023-12-27 00:35:39,044][105692] Updated weights for policy 0, policy_version 1264383 (0.0009) [2023-12-27 00:35:39,096][105692] Updated weights for policy 0, policy_version 1264393 (0.0009) [2023-12-27 00:35:39,414][105620] Updated weights for policy 1, policy_version 1265577 (0.0008) [2023-12-27 00:35:39,467][105620] Updated weights for policy 1, policy_version 1265587 (0.0011) [2023-12-27 00:35:39,526][105620] Updated weights for policy 1, policy_version 1265597 (0.0011) [2023-12-27 00:35:39,842][105692] Updated weights for policy 0, policy_version 1264403 (0.0008) [2023-12-27 00:35:39,895][105692] Updated weights for policy 0, policy_version 1264413 (0.0008) [2023-12-27 00:35:39,955][105692] Updated weights for policy 0, policy_version 1264423 (0.0008) [2023-12-27 00:35:40,336][105620] Updated weights for policy 1, policy_version 1265607 (0.0011) [2023-12-27 00:35:40,388][105620] Updated weights for policy 1, policy_version 1265617 (0.0009) [2023-12-27 00:35:40,450][105620] Updated weights for policy 1, policy_version 1265627 (0.0010) [2023-12-27 00:35:40,761][105692] Updated weights for policy 0, policy_version 1264433 (0.0008) [2023-12-27 00:35:40,820][105692] Updated weights for policy 0, policy_version 1264443 (0.0008) [2023-12-27 00:35:40,872][105692] Updated weights for policy 0, policy_version 1264453 (0.0008) [2023-12-27 00:35:40,916][105692] Updated weights for policy 0, policy_version 1264463 (0.0010) [2023-12-27 00:35:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.7, 300 sec: 19383.1). Total num frames: 647798784. Throughput: 0: 9905.3, 1: 9895.0. Samples: 647806488. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:35:41,063][104569] Avg episode reward: [(0, '9083.791'), (1, '9263.924')] [2023-12-27 00:35:41,222][105620] Updated weights for policy 1, policy_version 1265637 (0.0010) [2023-12-27 00:35:41,293][105620] Updated weights for policy 1, policy_version 1265647 (0.0008) [2023-12-27 00:35:41,355][105620] Updated weights for policy 1, policy_version 1265657 (0.0009) [2023-12-27 00:35:41,763][105692] Updated weights for policy 0, policy_version 1264473 (0.0008) [2023-12-27 00:35:41,825][105692] Updated weights for policy 0, policy_version 1264483 (0.0008) [2023-12-27 00:35:41,880][105692] Updated weights for policy 0, policy_version 1264493 (0.0008) [2023-12-27 00:35:42,102][105620] Updated weights for policy 1, policy_version 1265667 (0.0009) [2023-12-27 00:35:42,165][105620] Updated weights for policy 1, policy_version 1265677 (0.0011) [2023-12-27 00:35:42,231][105620] Updated weights for policy 1, policy_version 1265687 (0.0011) [2023-12-27 00:35:42,660][105692] Updated weights for policy 0, policy_version 1264503 (0.0010) [2023-12-27 00:35:42,713][105692] Updated weights for policy 0, policy_version 1264513 (0.0010) [2023-12-27 00:35:42,765][105692] Updated weights for policy 0, policy_version 1264523 (0.0010) [2023-12-27 00:35:42,990][105620] Updated weights for policy 1, policy_version 1265697 (0.0009) [2023-12-27 00:35:43,049][105620] Updated weights for policy 1, policy_version 1265707 (0.0010) [2023-12-27 00:35:43,107][105620] Updated weights for policy 1, policy_version 1265717 (0.0010) [2023-12-27 00:35:43,168][105620] Updated weights for policy 1, policy_version 1265727 (0.0011) [2023-12-27 00:35:43,469][105692] Updated weights for policy 0, policy_version 1264533 (0.0011) [2023-12-27 00:35:43,530][105692] Updated weights for policy 0, policy_version 1264543 (0.0010) [2023-12-27 00:35:43,589][105692] Updated weights for policy 0, policy_version 1264553 (0.0010) [2023-12-27 00:35:43,810][105620] Updated weights for policy 1, policy_version 1265737 (0.0010) [2023-12-27 00:35:43,864][105620] Updated weights for policy 1, policy_version 1265747 (0.0010) [2023-12-27 00:35:43,916][105620] Updated weights for policy 1, policy_version 1265757 (0.0010) [2023-12-27 00:35:44,189][105692] Updated weights for policy 0, policy_version 1264563 (0.0010) [2023-12-27 00:35:44,252][105692] Updated weights for policy 0, policy_version 1264573 (0.0011) [2023-12-27 00:35:44,315][105692] Updated weights for policy 0, policy_version 1264583 (0.0011) [2023-12-27 00:35:44,649][105620] Updated weights for policy 1, policy_version 1265767 (0.0010) [2023-12-27 00:35:44,701][105620] Updated weights for policy 1, policy_version 1265777 (0.0010) [2023-12-27 00:35:44,754][105620] Updated weights for policy 1, policy_version 1265787 (0.0011) [2023-12-27 00:35:45,036][105692] Updated weights for policy 0, policy_version 1264593 (0.0010) [2023-12-27 00:35:45,088][105692] Updated weights for policy 0, policy_version 1264603 (0.0008) [2023-12-27 00:35:45,147][105692] Updated weights for policy 0, policy_version 1264613 (0.0008) [2023-12-27 00:35:45,204][105692] Updated weights for policy 0, policy_version 1264623 (0.0008) [2023-12-27 00:35:45,549][105620] Updated weights for policy 1, policy_version 1265797 (0.0011) [2023-12-27 00:35:45,613][105620] Updated weights for policy 1, policy_version 1265807 (0.0011) [2023-12-27 00:35:45,670][105620] Updated weights for policy 1, policy_version 1265817 (0.0011) [2023-12-27 00:35:45,879][105692] Updated weights for policy 0, policy_version 1264633 (0.0006) [2023-12-27 00:35:45,932][105692] Updated weights for policy 0, policy_version 1264643 (0.0006) [2023-12-27 00:35:45,986][105692] Updated weights for policy 0, policy_version 1264653 (0.0009) [2023-12-27 00:35:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 647897088. Throughput: 0: 9847.5, 1: 9816.5. Samples: 647862212. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:35:46,062][104569] Avg episode reward: [(0, '9173.618'), (1, '9355.566')] [2023-12-27 00:35:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001264656_323805184.pth... [2023-12-27 00:35:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001265824_324091904.pth... [2023-12-27 00:35:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001263472_323502080.pth [2023-12-27 00:35:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001264704_323805184.pth [2023-12-27 00:35:46,287][105620] Updated weights for policy 1, policy_version 1265827 (0.0009) [2023-12-27 00:35:46,339][105620] Updated weights for policy 1, policy_version 1265837 (0.0005) [2023-12-27 00:35:46,405][105620] Updated weights for policy 1, policy_version 1265847 (0.0005) [2023-12-27 00:35:46,628][105692] Updated weights for policy 0, policy_version 1264663 (0.0010) [2023-12-27 00:35:46,676][105692] Updated weights for policy 0, policy_version 1264673 (0.0009) [2023-12-27 00:35:46,734][105692] Updated weights for policy 0, policy_version 1264683 (0.0009) [2023-12-27 00:35:46,966][105620] Updated weights for policy 1, policy_version 1265857 (0.0008) [2023-12-27 00:35:47,014][105620] Updated weights for policy 1, policy_version 1265867 (0.0008) [2023-12-27 00:35:47,065][105620] Updated weights for policy 1, policy_version 1265877 (0.0008) [2023-12-27 00:35:47,120][105620] Updated weights for policy 1, policy_version 1265887 (0.0008) [2023-12-27 00:35:47,506][105692] Updated weights for policy 0, policy_version 1264693 (0.0011) [2023-12-27 00:35:47,568][105692] Updated weights for policy 0, policy_version 1264703 (0.0010) [2023-12-27 00:35:47,623][105692] Updated weights for policy 0, policy_version 1264713 (0.0010) [2023-12-27 00:35:47,903][105620] Updated weights for policy 1, policy_version 1265897 (0.0008) [2023-12-27 00:35:47,966][105620] Updated weights for policy 1, policy_version 1265907 (0.0009) [2023-12-27 00:35:48,023][105620] Updated weights for policy 1, policy_version 1265917 (0.0005) [2023-12-27 00:35:48,359][105692] Updated weights for policy 0, policy_version 1264723 (0.0010) [2023-12-27 00:35:48,420][105692] Updated weights for policy 0, policy_version 1264733 (0.0010) [2023-12-27 00:35:48,479][105692] Updated weights for policy 0, policy_version 1264743 (0.0007) [2023-12-27 00:35:48,636][105620] Updated weights for policy 1, policy_version 1265927 (0.0008) [2023-12-27 00:35:48,698][105620] Updated weights for policy 1, policy_version 1265937 (0.0009) [2023-12-27 00:35:48,763][105620] Updated weights for policy 1, policy_version 1265947 (0.0009) [2023-12-27 00:35:49,260][105692] Updated weights for policy 0, policy_version 1264753 (0.0009) [2023-12-27 00:35:49,313][105692] Updated weights for policy 0, policy_version 1264763 (0.0005) [2023-12-27 00:35:49,381][105692] Updated weights for policy 0, policy_version 1264773 (0.0008) [2023-12-27 00:35:49,448][105692] Updated weights for policy 0, policy_version 1264783 (0.0008) [2023-12-27 00:35:49,523][105620] Updated weights for policy 1, policy_version 1265957 (0.0007) [2023-12-27 00:35:49,593][105620] Updated weights for policy 1, policy_version 1265967 (0.0009) [2023-12-27 00:35:49,659][105620] Updated weights for policy 1, policy_version 1265977 (0.0009) [2023-12-27 00:35:50,101][105692] Updated weights for policy 0, policy_version 1264793 (0.0008) [2023-12-27 00:35:50,163][105692] Updated weights for policy 0, policy_version 1264803 (0.0009) [2023-12-27 00:35:50,225][105692] Updated weights for policy 0, policy_version 1264813 (0.0009) [2023-12-27 00:35:50,427][105620] Updated weights for policy 1, policy_version 1265987 (0.0010) [2023-12-27 00:35:50,489][105620] Updated weights for policy 1, policy_version 1265997 (0.0009) [2023-12-27 00:35:50,558][105620] Updated weights for policy 1, policy_version 1266007 (0.0010) [2023-12-27 00:35:50,978][105692] Updated weights for policy 0, policy_version 1264823 (0.0006) [2023-12-27 00:35:51,039][105692] Updated weights for policy 0, policy_version 1264833 (0.0008) [2023-12-27 00:35:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.7, 300 sec: 19410.9). Total num frames: 647987200. Throughput: 0: 9878.2, 1: 9715.7. Samples: 647980816. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:35:51,063][104569] Avg episode reward: [(0, '8993.029'), (1, '9172.946')] [2023-12-27 00:35:51,095][105692] Updated weights for policy 0, policy_version 1264843 (0.0008) [2023-12-27 00:35:51,274][105620] Updated weights for policy 1, policy_version 1266017 (0.0007) [2023-12-27 00:35:51,335][105620] Updated weights for policy 1, policy_version 1266027 (0.0012) [2023-12-27 00:35:51,400][105620] Updated weights for policy 1, policy_version 1266037 (0.0007) [2023-12-27 00:35:51,458][105620] Updated weights for policy 1, policy_version 1266047 (0.0005) [2023-12-27 00:35:51,883][105692] Updated weights for policy 0, policy_version 1264853 (0.0009) [2023-12-27 00:35:51,935][105692] Updated weights for policy 0, policy_version 1264863 (0.0009) [2023-12-27 00:35:51,994][105692] Updated weights for policy 0, policy_version 1264873 (0.0009) [2023-12-27 00:35:52,120][105620] Updated weights for policy 1, policy_version 1266057 (0.0009) [2023-12-27 00:35:52,172][105620] Updated weights for policy 1, policy_version 1266067 (0.0008) [2023-12-27 00:35:52,233][105620] Updated weights for policy 1, policy_version 1266077 (0.0008) [2023-12-27 00:35:52,808][105692] Updated weights for policy 0, policy_version 1264883 (0.0009) [2023-12-27 00:35:52,867][105692] Updated weights for policy 0, policy_version 1264893 (0.0009) [2023-12-27 00:35:52,916][105692] Updated weights for policy 0, policy_version 1264903 (0.0008) [2023-12-27 00:35:52,949][105620] Updated weights for policy 1, policy_version 1266088 (0.0008) [2023-12-27 00:35:53,007][105620] Updated weights for policy 1, policy_version 1266098 (0.0008) [2023-12-27 00:35:53,067][105620] Updated weights for policy 1, policy_version 1266108 (0.0009) [2023-12-27 00:35:53,676][105692] Updated weights for policy 0, policy_version 1264913 (0.0006) [2023-12-27 00:35:53,725][105692] Updated weights for policy 0, policy_version 1264923 (0.0009) [2023-12-27 00:35:53,786][105692] Updated weights for policy 0, policy_version 1264933 (0.0008) [2023-12-27 00:35:53,801][105620] Updated weights for policy 1, policy_version 1266118 (0.0008) [2023-12-27 00:35:53,836][105692] Updated weights for policy 0, policy_version 1264943 (0.0007) [2023-12-27 00:35:53,859][105620] Updated weights for policy 1, policy_version 1266128 (0.0008) [2023-12-27 00:35:53,908][105620] Updated weights for policy 1, policy_version 1266138 (0.0009) [2023-12-27 00:35:54,482][105692] Updated weights for policy 0, policy_version 1264953 (0.0006) [2023-12-27 00:35:54,538][105692] Updated weights for policy 0, policy_version 1264963 (0.0008) [2023-12-27 00:35:54,591][105692] Updated weights for policy 0, policy_version 1264973 (0.0008) [2023-12-27 00:35:54,673][105620] Updated weights for policy 1, policy_version 1266148 (0.0008) [2023-12-27 00:35:54,734][105620] Updated weights for policy 1, policy_version 1266158 (0.0005) [2023-12-27 00:35:54,799][105620] Updated weights for policy 1, policy_version 1266168 (0.0005) [2023-12-27 00:35:55,408][105620] Updated weights for policy 1, policy_version 1266178 (0.0006) [2023-12-27 00:35:55,422][105692] Updated weights for policy 0, policy_version 1264983 (0.0007) [2023-12-27 00:35:55,461][105620] Updated weights for policy 1, policy_version 1266188 (0.0010) [2023-12-27 00:35:55,479][105692] Updated weights for policy 0, policy_version 1264993 (0.0006) [2023-12-27 00:35:55,520][105620] Updated weights for policy 1, policy_version 1266198 (0.0011) [2023-12-27 00:35:55,538][105692] Updated weights for policy 0, policy_version 1265003 (0.0005) [2023-12-27 00:35:55,586][105620] Updated weights for policy 1, policy_version 1266208 (0.0011) [2023-12-27 00:35:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 648085504. Throughput: 0: 9873.6, 1: 9714.3. Samples: 648095740. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:35:56,063][104569] Avg episode reward: [(0, '9177.089'), (1, '8898.343')] [2023-12-27 00:35:56,275][105692] Updated weights for policy 0, policy_version 1265013 (0.0008) [2023-12-27 00:35:56,330][105620] Updated weights for policy 1, policy_version 1266218 (0.0010) [2023-12-27 00:35:56,333][105692] Updated weights for policy 0, policy_version 1265023 (0.0006) [2023-12-27 00:35:56,388][105620] Updated weights for policy 1, policy_version 1266228 (0.0010) [2023-12-27 00:35:56,389][105692] Updated weights for policy 0, policy_version 1265033 (0.0010) [2023-12-27 00:35:56,443][105620] Updated weights for policy 1, policy_version 1266238 (0.0010) [2023-12-27 00:35:57,146][105692] Updated weights for policy 0, policy_version 1265043 (0.0008) [2023-12-27 00:35:57,182][105620] Updated weights for policy 1, policy_version 1266248 (0.0010) [2023-12-27 00:35:57,195][105692] Updated weights for policy 0, policy_version 1265053 (0.0005) [2023-12-27 00:35:57,237][105620] Updated weights for policy 1, policy_version 1266258 (0.0010) [2023-12-27 00:35:57,242][105692] Updated weights for policy 0, policy_version 1265063 (0.0007) [2023-12-27 00:35:57,294][105620] Updated weights for policy 1, policy_version 1266268 (0.0010) [2023-12-27 00:35:57,977][105692] Updated weights for policy 0, policy_version 1265073 (0.0005) [2023-12-27 00:35:57,979][105620] Updated weights for policy 1, policy_version 1266278 (0.0009) [2023-12-27 00:35:58,028][105692] Updated weights for policy 0, policy_version 1265083 (0.0008) [2023-12-27 00:35:58,038][105620] Updated weights for policy 1, policy_version 1266288 (0.0008) [2023-12-27 00:35:58,084][105692] Updated weights for policy 0, policy_version 1265093 (0.0007) [2023-12-27 00:35:58,090][105620] Updated weights for policy 1, policy_version 1266298 (0.0006) [2023-12-27 00:35:58,134][105692] Updated weights for policy 0, policy_version 1265103 (0.0006) [2023-12-27 00:35:58,851][105620] Updated weights for policy 1, policy_version 1266308 (0.0008) [2023-12-27 00:35:58,918][105620] Updated weights for policy 1, policy_version 1266318 (0.0008) [2023-12-27 00:35:58,968][105692] Updated weights for policy 0, policy_version 1265113 (0.0008) [2023-12-27 00:35:58,996][105620] Updated weights for policy 1, policy_version 1266328 (0.0007) [2023-12-27 00:35:59,032][105692] Updated weights for policy 0, policy_version 1265123 (0.0008) [2023-12-27 00:35:59,094][105692] Updated weights for policy 0, policy_version 1265133 (0.0007) [2023-12-27 00:35:59,752][105620] Updated weights for policy 1, policy_version 1266338 (0.0008) [2023-12-27 00:35:59,813][105620] Updated weights for policy 1, policy_version 1266348 (0.0007) [2023-12-27 00:35:59,870][105620] Updated weights for policy 1, policy_version 1266358 (0.0010) [2023-12-27 00:35:59,878][105692] Updated weights for policy 0, policy_version 1265143 (0.0008) [2023-12-27 00:35:59,926][105620] Updated weights for policy 1, policy_version 1266368 (0.0008) [2023-12-27 00:35:59,935][105692] Updated weights for policy 0, policy_version 1265153 (0.0008) [2023-12-27 00:35:59,996][105692] Updated weights for policy 0, policy_version 1265163 (0.0008) [2023-12-27 00:36:00,529][105620] Updated weights for policy 1, policy_version 1266378 (0.0009) [2023-12-27 00:36:00,575][105620] Updated weights for policy 1, policy_version 1266388 (0.0007) [2023-12-27 00:36:00,623][105620] Updated weights for policy 1, policy_version 1266398 (0.0005) [2023-12-27 00:36:00,763][105692] Updated weights for policy 0, policy_version 1265173 (0.0009) [2023-12-27 00:36:00,819][105692] Updated weights for policy 0, policy_version 1265183 (0.0013) [2023-12-27 00:36:00,873][105692] Updated weights for policy 0, policy_version 1265194 (0.0010) [2023-12-27 00:36:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 648183808. Throughput: 0: 9859.0, 1: 9701.9. Samples: 648152288. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:36:01,063][104569] Avg episode reward: [(0, '9177.450'), (1, '8989.407')] [2023-12-27 00:36:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001265200_323944448.pth... [2023-12-27 00:36:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001266400_324239360.pth... [2023-12-27 00:36:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001265280_323952640.pth [2023-12-27 00:36:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001264048_323649536.pth [2023-12-27 00:36:01,204][105620] Updated weights for policy 1, policy_version 1266408 (0.0007) [2023-12-27 00:36:01,277][105620] Updated weights for policy 1, policy_version 1266418 (0.0009) [2023-12-27 00:36:01,343][105620] Updated weights for policy 1, policy_version 1266428 (0.0009) [2023-12-27 00:36:01,718][105692] Updated weights for policy 0, policy_version 1265204 (0.0009) [2023-12-27 00:36:01,782][105692] Updated weights for policy 0, policy_version 1265214 (0.0009) [2023-12-27 00:36:01,829][105692] Updated weights for policy 0, policy_version 1265224 (0.0009) [2023-12-27 00:36:02,037][105620] Updated weights for policy 1, policy_version 1266438 (0.0008) [2023-12-27 00:36:02,096][105620] Updated weights for policy 1, policy_version 1266448 (0.0009) [2023-12-27 00:36:02,157][105620] Updated weights for policy 1, policy_version 1266458 (0.0009) [2023-12-27 00:36:02,575][105692] Updated weights for policy 0, policy_version 1265234 (0.0008) [2023-12-27 00:36:02,628][105692] Updated weights for policy 0, policy_version 1265244 (0.0005) [2023-12-27 00:36:02,682][105692] Updated weights for policy 0, policy_version 1265254 (0.0008) [2023-12-27 00:36:02,729][105692] Updated weights for policy 0, policy_version 1265264 (0.0009) [2023-12-27 00:36:02,926][105620] Updated weights for policy 1, policy_version 1266468 (0.0009) [2023-12-27 00:36:02,987][105620] Updated weights for policy 1, policy_version 1266478 (0.0009) [2023-12-27 00:36:03,045][105620] Updated weights for policy 1, policy_version 1266488 (0.0009) [2023-12-27 00:36:03,431][105692] Updated weights for policy 0, policy_version 1265274 (0.0006) [2023-12-27 00:36:03,482][105692] Updated weights for policy 0, policy_version 1265284 (0.0005) [2023-12-27 00:36:03,536][105692] Updated weights for policy 0, policy_version 1265294 (0.0005) [2023-12-27 00:36:03,791][105620] Updated weights for policy 1, policy_version 1266498 (0.0008) [2023-12-27 00:36:03,842][105620] Updated weights for policy 1, policy_version 1266508 (0.0007) [2023-12-27 00:36:03,909][105620] Updated weights for policy 1, policy_version 1266518 (0.0006) [2023-12-27 00:36:03,969][105620] Updated weights for policy 1, policy_version 1266528 (0.0007) [2023-12-27 00:36:04,159][105692] Updated weights for policy 0, policy_version 1265304 (0.0009) [2023-12-27 00:36:04,212][105692] Updated weights for policy 0, policy_version 1265314 (0.0010) [2023-12-27 00:36:04,272][105692] Updated weights for policy 0, policy_version 1265324 (0.0010) [2023-12-27 00:36:04,606][105620] Updated weights for policy 1, policy_version 1266538 (0.0008) [2023-12-27 00:36:04,653][105620] Updated weights for policy 1, policy_version 1266548 (0.0009) [2023-12-27 00:36:04,703][105620] Updated weights for policy 1, policy_version 1266558 (0.0009) [2023-12-27 00:36:05,057][105692] Updated weights for policy 0, policy_version 1265334 (0.0007) [2023-12-27 00:36:05,114][105692] Updated weights for policy 0, policy_version 1265344 (0.0006) [2023-12-27 00:36:05,179][105692] Updated weights for policy 0, policy_version 1265354 (0.0007) [2023-12-27 00:36:05,381][105620] Updated weights for policy 1, policy_version 1266568 (0.0006) [2023-12-27 00:36:05,435][105620] Updated weights for policy 1, policy_version 1266578 (0.0006) [2023-12-27 00:36:05,493][105620] Updated weights for policy 1, policy_version 1266588 (0.0007) [2023-12-27 00:36:05,882][105692] Updated weights for policy 0, policy_version 1265364 (0.0011) [2023-12-27 00:36:05,938][105692] Updated weights for policy 0, policy_version 1265374 (0.0011) [2023-12-27 00:36:05,987][105692] Updated weights for policy 0, policy_version 1265384 (0.0011) [2023-12-27 00:36:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 648282112. Throughput: 0: 9737.2, 1: 9713.8. Samples: 648267512. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:36:06,062][104569] Avg episode reward: [(0, '9084.441'), (1, '9080.761')] [2023-12-27 00:36:06,077][105620] Updated weights for policy 1, policy_version 1266598 (0.0005) [2023-12-27 00:36:06,143][105620] Updated weights for policy 1, policy_version 1266608 (0.0008) [2023-12-27 00:36:06,213][105620] Updated weights for policy 1, policy_version 1266618 (0.0006) [2023-12-27 00:36:06,740][105692] Updated weights for policy 0, policy_version 1265394 (0.0010) [2023-12-27 00:36:06,763][105620] Updated weights for policy 1, policy_version 1266628 (0.0008) [2023-12-27 00:36:06,793][105692] Updated weights for policy 0, policy_version 1265404 (0.0011) [2023-12-27 00:36:06,826][105620] Updated weights for policy 1, policy_version 1266638 (0.0011) [2023-12-27 00:36:06,846][105692] Updated weights for policy 0, policy_version 1265414 (0.0011) [2023-12-27 00:36:06,881][105620] Updated weights for policy 1, policy_version 1266648 (0.0010) [2023-12-27 00:36:06,903][105692] Updated weights for policy 0, policy_version 1265424 (0.0009) [2023-12-27 00:36:07,571][105620] Updated weights for policy 1, policy_version 1266658 (0.0009) [2023-12-27 00:36:07,588][105692] Updated weights for policy 0, policy_version 1265434 (0.0009) [2023-12-27 00:36:07,625][105620] Updated weights for policy 1, policy_version 1266668 (0.0005) [2023-12-27 00:36:07,640][105692] Updated weights for policy 0, policy_version 1265444 (0.0009) [2023-12-27 00:36:07,674][105620] Updated weights for policy 1, policy_version 1266678 (0.0005) [2023-12-27 00:36:07,692][105692] Updated weights for policy 0, policy_version 1265454 (0.0009) [2023-12-27 00:36:07,724][105620] Updated weights for policy 1, policy_version 1266688 (0.0006) [2023-12-27 00:36:08,262][105620] Updated weights for policy 1, policy_version 1266698 (0.0006) [2023-12-27 00:36:08,316][105620] Updated weights for policy 1, policy_version 1266708 (0.0006) [2023-12-27 00:36:08,378][105620] Updated weights for policy 1, policy_version 1266718 (0.0007) [2023-12-27 00:36:08,482][105692] Updated weights for policy 0, policy_version 1265464 (0.0008) [2023-12-27 00:36:08,534][105692] Updated weights for policy 0, policy_version 1265475 (0.0008) [2023-12-27 00:36:08,577][105692] Updated weights for policy 0, policy_version 1265485 (0.0005) [2023-12-27 00:36:09,044][105620] Updated weights for policy 1, policy_version 1266728 (0.0005) [2023-12-27 00:36:09,102][105620] Updated weights for policy 1, policy_version 1266738 (0.0005) [2023-12-27 00:36:09,165][105620] Updated weights for policy 1, policy_version 1266748 (0.0005) [2023-12-27 00:36:09,327][105692] Updated weights for policy 0, policy_version 1265495 (0.0008) [2023-12-27 00:36:09,389][105692] Updated weights for policy 0, policy_version 1265505 (0.0009) [2023-12-27 00:36:09,450][105692] Updated weights for policy 0, policy_version 1265515 (0.0008) [2023-12-27 00:36:09,841][105620] Updated weights for policy 1, policy_version 1266758 (0.0009) [2023-12-27 00:36:09,912][105620] Updated weights for policy 1, policy_version 1266768 (0.0011) [2023-12-27 00:36:09,973][105620] Updated weights for policy 1, policy_version 1266778 (0.0010) [2023-12-27 00:36:10,215][105692] Updated weights for policy 0, policy_version 1265525 (0.0009) [2023-12-27 00:36:10,272][105692] Updated weights for policy 0, policy_version 1265535 (0.0008) [2023-12-27 00:36:10,334][105692] Updated weights for policy 0, policy_version 1265545 (0.0007) [2023-12-27 00:36:10,699][105620] Updated weights for policy 1, policy_version 1266788 (0.0010) [2023-12-27 00:36:10,750][105620] Updated weights for policy 1, policy_version 1266798 (0.0010) [2023-12-27 00:36:10,803][105620] Updated weights for policy 1, policy_version 1266808 (0.0009) [2023-12-27 00:36:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 648380416. Throughput: 0: 9716.1, 1: 9907.2. Samples: 648390268. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:36:11,062][104569] Avg episode reward: [(0, '9085.396'), (1, '8989.313')] [2023-12-27 00:36:11,084][105692] Updated weights for policy 0, policy_version 1265555 (0.0008) [2023-12-27 00:36:11,145][105692] Updated weights for policy 0, policy_version 1265565 (0.0008) [2023-12-27 00:36:11,205][105692] Updated weights for policy 0, policy_version 1265575 (0.0008) [2023-12-27 00:36:11,511][105620] Updated weights for policy 1, policy_version 1266818 (0.0007) [2023-12-27 00:36:11,577][105620] Updated weights for policy 1, policy_version 1266828 (0.0010) [2023-12-27 00:36:11,643][105620] Updated weights for policy 1, policy_version 1266838 (0.0010) [2023-12-27 00:36:11,710][105620] Updated weights for policy 1, policy_version 1266848 (0.0009) [2023-12-27 00:36:11,999][105692] Updated weights for policy 0, policy_version 1265585 (0.0008) [2023-12-27 00:36:12,057][105692] Updated weights for policy 0, policy_version 1265595 (0.0009) [2023-12-27 00:36:12,110][105692] Updated weights for policy 0, policy_version 1265605 (0.0008) [2023-12-27 00:36:12,166][105692] Updated weights for policy 0, policy_version 1265615 (0.0007) [2023-12-27 00:36:12,442][105620] Updated weights for policy 1, policy_version 1266858 (0.0009) [2023-12-27 00:36:12,504][105620] Updated weights for policy 1, policy_version 1266868 (0.0010) [2023-12-27 00:36:12,566][105620] Updated weights for policy 1, policy_version 1266878 (0.0010) [2023-12-27 00:36:12,915][105692] Updated weights for policy 0, policy_version 1265625 (0.0008) [2023-12-27 00:36:12,985][105692] Updated weights for policy 0, policy_version 1265635 (0.0010) [2023-12-27 00:36:13,041][105692] Updated weights for policy 0, policy_version 1265645 (0.0010) [2023-12-27 00:36:13,299][105620] Updated weights for policy 1, policy_version 1266888 (0.0010) [2023-12-27 00:36:13,351][105620] Updated weights for policy 1, policy_version 1266898 (0.0010) [2023-12-27 00:36:13,402][105620] Updated weights for policy 1, policy_version 1266908 (0.0010) [2023-12-27 00:36:13,660][105692] Updated weights for policy 0, policy_version 1265655 (0.0008) [2023-12-27 00:36:13,723][105692] Updated weights for policy 0, policy_version 1265665 (0.0005) [2023-12-27 00:36:13,775][105692] Updated weights for policy 0, policy_version 1265675 (0.0005) [2023-12-27 00:36:14,151][105620] Updated weights for policy 1, policy_version 1266918 (0.0010) [2023-12-27 00:36:14,219][105620] Updated weights for policy 1, policy_version 1266928 (0.0010) [2023-12-27 00:36:14,285][105620] Updated weights for policy 1, policy_version 1266938 (0.0010) [2023-12-27 00:36:14,363][105692] Updated weights for policy 0, policy_version 1265685 (0.0007) [2023-12-27 00:36:14,423][105692] Updated weights for policy 0, policy_version 1265695 (0.0007) [2023-12-27 00:36:14,482][105692] Updated weights for policy 0, policy_version 1265705 (0.0008) [2023-12-27 00:36:14,957][105620] Updated weights for policy 1, policy_version 1266948 (0.0011) [2023-12-27 00:36:15,014][105620] Updated weights for policy 1, policy_version 1266958 (0.0011) [2023-12-27 00:36:15,073][105620] Updated weights for policy 1, policy_version 1266968 (0.0011) [2023-12-27 00:36:15,241][105692] Updated weights for policy 0, policy_version 1265715 (0.0008) [2023-12-27 00:36:15,302][105692] Updated weights for policy 0, policy_version 1265725 (0.0008) [2023-12-27 00:36:15,368][105692] Updated weights for policy 0, policy_version 1265735 (0.0008) [2023-12-27 00:36:15,821][105620] Updated weights for policy 1, policy_version 1266978 (0.0011) [2023-12-27 00:36:15,880][105620] Updated weights for policy 1, policy_version 1266988 (0.0010) [2023-12-27 00:36:15,938][105620] Updated weights for policy 1, policy_version 1266998 (0.0010) [2023-12-27 00:36:16,000][105620] Updated weights for policy 1, policy_version 1267008 (0.0010) [2023-12-27 00:36:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 648478720. Throughput: 0: 9616.4, 1: 9861.2. Samples: 648447000. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 00:36:16,063][104569] Avg episode reward: [(0, '9082.931'), (1, '9172.536')] [2023-12-27 00:36:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001265744_324083712.pth... [2023-12-27 00:36:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001267008_324395008.pth... [2023-12-27 00:36:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001264656_323805184.pth [2023-12-27 00:36:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001265824_324091904.pth [2023-12-27 00:36:16,111][105692] Updated weights for policy 0, policy_version 1265745 (0.0008) [2023-12-27 00:36:16,174][105692] Updated weights for policy 0, policy_version 1265755 (0.0008) [2023-12-27 00:36:16,226][105692] Updated weights for policy 0, policy_version 1265765 (0.0007) [2023-12-27 00:36:16,278][105692] Updated weights for policy 0, policy_version 1265775 (0.0008) [2023-12-27 00:36:16,719][105620] Updated weights for policy 1, policy_version 1267018 (0.0007) [2023-12-27 00:36:16,779][105620] Updated weights for policy 1, policy_version 1267028 (0.0008) [2023-12-27 00:36:16,827][105620] Updated weights for policy 1, policy_version 1267038 (0.0011) [2023-12-27 00:36:17,033][105692] Updated weights for policy 0, policy_version 1265785 (0.0010) [2023-12-27 00:36:17,086][105692] Updated weights for policy 0, policy_version 1265795 (0.0008) [2023-12-27 00:36:17,145][105692] Updated weights for policy 0, policy_version 1265805 (0.0010) [2023-12-27 00:36:17,471][105620] Updated weights for policy 1, policy_version 1267048 (0.0010) [2023-12-27 00:36:17,521][105620] Updated weights for policy 1, policy_version 1267058 (0.0010) [2023-12-27 00:36:17,572][105620] Updated weights for policy 1, policy_version 1267068 (0.0010) [2023-12-27 00:36:17,907][105692] Updated weights for policy 0, policy_version 1265815 (0.0006) [2023-12-27 00:36:17,956][105692] Updated weights for policy 0, policy_version 1265825 (0.0005) [2023-12-27 00:36:18,011][105692] Updated weights for policy 0, policy_version 1265835 (0.0005) [2023-12-27 00:36:18,346][105620] Updated weights for policy 1, policy_version 1267078 (0.0009) [2023-12-27 00:36:18,412][105620] Updated weights for policy 1, policy_version 1267088 (0.0010) [2023-12-27 00:36:18,464][105620] Updated weights for policy 1, policy_version 1267098 (0.0010) [2023-12-27 00:36:18,627][105692] Updated weights for policy 0, policy_version 1265845 (0.0008) [2023-12-27 00:36:18,693][105692] Updated weights for policy 0, policy_version 1265855 (0.0011) [2023-12-27 00:36:18,750][105692] Updated weights for policy 0, policy_version 1265865 (0.0011) [2023-12-27 00:36:19,183][105620] Updated weights for policy 1, policy_version 1267108 (0.0011) [2023-12-27 00:36:19,250][105620] Updated weights for policy 1, policy_version 1267118 (0.0011) [2023-12-27 00:36:19,316][105620] Updated weights for policy 1, policy_version 1267128 (0.0011) [2023-12-27 00:36:19,469][105692] Updated weights for policy 0, policy_version 1265875 (0.0010) [2023-12-27 00:36:19,532][105692] Updated weights for policy 0, policy_version 1265885 (0.0008) [2023-12-27 00:36:19,591][105692] Updated weights for policy 0, policy_version 1265895 (0.0008) [2023-12-27 00:36:20,061][105620] Updated weights for policy 1, policy_version 1267138 (0.0010) [2023-12-27 00:36:20,117][105620] Updated weights for policy 1, policy_version 1267148 (0.0009) [2023-12-27 00:36:20,177][105620] Updated weights for policy 1, policy_version 1267158 (0.0008) [2023-12-27 00:36:20,235][105620] Updated weights for policy 1, policy_version 1267168 (0.0009) [2023-12-27 00:36:20,286][105692] Updated weights for policy 0, policy_version 1265905 (0.0006) [2023-12-27 00:36:20,351][105692] Updated weights for policy 0, policy_version 1265915 (0.0009) [2023-12-27 00:36:20,415][105692] Updated weights for policy 0, policy_version 1265925 (0.0009) [2023-12-27 00:36:20,481][105692] Updated weights for policy 0, policy_version 1265935 (0.0010) [2023-12-27 00:36:20,991][105620] Updated weights for policy 1, policy_version 1267178 (0.0008) [2023-12-27 00:36:21,057][105620] Updated weights for policy 1, policy_version 1267188 (0.0009) [2023-12-27 00:36:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 648568832. Throughput: 0: 9634.8, 1: 9928.0. Samples: 648563812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:36:21,063][104569] Avg episode reward: [(0, '9168.218'), (1, '9080.588')] [2023-12-27 00:36:21,113][105620] Updated weights for policy 1, policy_version 1267198 (0.0008) [2023-12-27 00:36:21,198][105692] Updated weights for policy 0, policy_version 1265945 (0.0008) [2023-12-27 00:36:21,264][105692] Updated weights for policy 0, policy_version 1265955 (0.0007) [2023-12-27 00:36:21,325][105692] Updated weights for policy 0, policy_version 1265965 (0.0007) [2023-12-27 00:36:21,934][105620] Updated weights for policy 1, policy_version 1267208 (0.0009) [2023-12-27 00:36:21,989][105620] Updated weights for policy 1, policy_version 1267218 (0.0010) [2023-12-27 00:36:22,042][105620] Updated weights for policy 1, policy_version 1267228 (0.0009) [2023-12-27 00:36:22,072][105692] Updated weights for policy 0, policy_version 1265975 (0.0006) [2023-12-27 00:36:22,124][105692] Updated weights for policy 0, policy_version 1265985 (0.0005) [2023-12-27 00:36:22,187][105692] Updated weights for policy 0, policy_version 1265995 (0.0005) [2023-12-27 00:36:22,858][105620] Updated weights for policy 1, policy_version 1267238 (0.0009) [2023-12-27 00:36:22,898][105692] Updated weights for policy 0, policy_version 1266005 (0.0008) [2023-12-27 00:36:22,914][105620] Updated weights for policy 1, policy_version 1267248 (0.0006) [2023-12-27 00:36:22,961][105692] Updated weights for policy 0, policy_version 1266015 (0.0007) [2023-12-27 00:36:22,974][105620] Updated weights for policy 1, policy_version 1267258 (0.0009) [2023-12-27 00:36:23,021][105692] Updated weights for policy 0, policy_version 1266025 (0.0008) [2023-12-27 00:36:23,709][105692] Updated weights for policy 0, policy_version 1266035 (0.0008) [2023-12-27 00:36:23,733][105620] Updated weights for policy 1, policy_version 1267268 (0.0007) [2023-12-27 00:36:23,772][105692] Updated weights for policy 0, policy_version 1266045 (0.0007) [2023-12-27 00:36:23,786][105620] Updated weights for policy 1, policy_version 1267278 (0.0006) [2023-12-27 00:36:23,830][105692] Updated weights for policy 0, policy_version 1266055 (0.0007) [2023-12-27 00:36:23,831][105620] Updated weights for policy 1, policy_version 1267288 (0.0005) [2023-12-27 00:36:24,417][105620] Updated weights for policy 1, policy_version 1267298 (0.0006) [2023-12-27 00:36:24,482][105620] Updated weights for policy 1, policy_version 1267308 (0.0008) [2023-12-27 00:36:24,541][105620] Updated weights for policy 1, policy_version 1267318 (0.0005) [2023-12-27 00:36:24,584][105620] Updated weights for policy 1, policy_version 1267328 (0.0005) [2023-12-27 00:36:24,692][105692] Updated weights for policy 0, policy_version 1266065 (0.0008) [2023-12-27 00:36:24,750][105692] Updated weights for policy 0, policy_version 1266075 (0.0010) [2023-12-27 00:36:24,806][105692] Updated weights for policy 0, policy_version 1266086 (0.0010) [2023-12-27 00:36:24,864][105692] Updated weights for policy 0, policy_version 1266096 (0.0009) [2023-12-27 00:36:25,222][105620] Updated weights for policy 1, policy_version 1267338 (0.0010) [2023-12-27 00:36:25,282][105620] Updated weights for policy 1, policy_version 1267348 (0.0008) [2023-12-27 00:36:25,346][105620] Updated weights for policy 1, policy_version 1267358 (0.0009) [2023-12-27 00:36:25,601][105692] Updated weights for policy 0, policy_version 1266106 (0.0009) [2023-12-27 00:36:25,656][105692] Updated weights for policy 0, policy_version 1266116 (0.0009) [2023-12-27 00:36:25,714][105692] Updated weights for policy 0, policy_version 1266126 (0.0010) [2023-12-27 00:36:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 648667136. Throughput: 0: 9506.5, 1: 9848.1. Samples: 648677444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:36:26,062][104569] Avg episode reward: [(0, '9260.723'), (1, '8827.568')] [2023-12-27 00:36:26,109][105620] Updated weights for policy 1, policy_version 1267368 (0.0009) [2023-12-27 00:36:26,166][105620] Updated weights for policy 1, policy_version 1267378 (0.0009) [2023-12-27 00:36:26,224][105620] Updated weights for policy 1, policy_version 1267388 (0.0009) [2023-12-27 00:36:26,488][105692] Updated weights for policy 0, policy_version 1266136 (0.0009) [2023-12-27 00:36:26,544][105692] Updated weights for policy 0, policy_version 1266146 (0.0009) [2023-12-27 00:36:26,598][105692] Updated weights for policy 0, policy_version 1266156 (0.0009) [2023-12-27 00:36:27,009][105620] Updated weights for policy 1, policy_version 1267398 (0.0008) [2023-12-27 00:36:27,072][105620] Updated weights for policy 1, policy_version 1267408 (0.0008) [2023-12-27 00:36:27,125][105620] Updated weights for policy 1, policy_version 1267418 (0.0008) [2023-12-27 00:36:27,285][105692] Updated weights for policy 0, policy_version 1266166 (0.0009) [2023-12-27 00:36:27,329][105692] Updated weights for policy 0, policy_version 1266176 (0.0006) [2023-12-27 00:36:27,390][105692] Updated weights for policy 0, policy_version 1266186 (0.0006) [2023-12-27 00:36:27,907][105620] Updated weights for policy 1, policy_version 1267428 (0.0008) [2023-12-27 00:36:27,961][105620] Updated weights for policy 1, policy_version 1267438 (0.0009) [2023-12-27 00:36:28,010][105620] Updated weights for policy 1, policy_version 1267448 (0.0008) [2023-12-27 00:36:28,063][105692] Updated weights for policy 0, policy_version 1266196 (0.0007) [2023-12-27 00:36:28,115][105692] Updated weights for policy 0, policy_version 1266208 (0.0010) [2023-12-27 00:36:28,166][105692] Updated weights for policy 0, policy_version 1266219 (0.0009) [2023-12-27 00:36:28,720][105620] Updated weights for policy 1, policy_version 1267458 (0.0006) [2023-12-27 00:36:28,766][105620] Updated weights for policy 1, policy_version 1267468 (0.0009) [2023-12-27 00:36:28,821][105620] Updated weights for policy 1, policy_version 1267478 (0.0009) [2023-12-27 00:36:28,875][105620] Updated weights for policy 1, policy_version 1267488 (0.0009) [2023-12-27 00:36:28,968][105692] Updated weights for policy 0, policy_version 1266229 (0.0009) [2023-12-27 00:36:29,030][105692] Updated weights for policy 0, policy_version 1266239 (0.0009) [2023-12-27 00:36:29,092][105692] Updated weights for policy 0, policy_version 1266249 (0.0009) [2023-12-27 00:36:29,723][105620] Updated weights for policy 1, policy_version 1267498 (0.0008) [2023-12-27 00:36:29,738][105692] Updated weights for policy 0, policy_version 1266259 (0.0007) [2023-12-27 00:36:29,773][105620] Updated weights for policy 1, policy_version 1267508 (0.0006) [2023-12-27 00:36:29,794][105692] Updated weights for policy 0, policy_version 1266269 (0.0007) [2023-12-27 00:36:29,821][105620] Updated weights for policy 1, policy_version 1267518 (0.0006) [2023-12-27 00:36:29,859][105692] Updated weights for policy 0, policy_version 1266279 (0.0007) [2023-12-27 00:36:30,509][105692] Updated weights for policy 0, policy_version 1266289 (0.0008) [2023-12-27 00:36:30,555][105692] Updated weights for policy 0, policy_version 1266299 (0.0009) [2023-12-27 00:36:30,602][105692] Updated weights for policy 0, policy_version 1266309 (0.0009) [2023-12-27 00:36:30,639][105620] Updated weights for policy 1, policy_version 1267528 (0.0008) [2023-12-27 00:36:30,655][105692] Updated weights for policy 0, policy_version 1266319 (0.0007) [2023-12-27 00:36:30,684][105620] Updated weights for policy 1, policy_version 1267538 (0.0007) [2023-12-27 00:36:30,730][105620] Updated weights for policy 1, policy_version 1267548 (0.0008) [2023-12-27 00:36:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 648765440. Throughput: 0: 9540.4, 1: 9831.7. Samples: 648733952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:36:31,062][104569] Avg episode reward: [(0, '9261.537'), (1, '9011.169')] [2023-12-27 00:36:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001266320_324231168.pth... [2023-12-27 00:36:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001267552_324534272.pth... [2023-12-27 00:36:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001265200_323944448.pth [2023-12-27 00:36:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001266400_324239360.pth [2023-12-27 00:36:31,445][105620] Updated weights for policy 1, policy_version 1267558 (0.0009) [2023-12-27 00:36:31,447][105692] Updated weights for policy 0, policy_version 1266329 (0.0007) [2023-12-27 00:36:31,502][105620] Updated weights for policy 1, policy_version 1267568 (0.0006) [2023-12-27 00:36:31,504][105692] Updated weights for policy 0, policy_version 1266339 (0.0007) [2023-12-27 00:36:31,561][105692] Updated weights for policy 0, policy_version 1266349 (0.0006) [2023-12-27 00:36:31,561][105620] Updated weights for policy 1, policy_version 1267578 (0.0007) [2023-12-27 00:36:32,283][105692] Updated weights for policy 0, policy_version 1266359 (0.0009) [2023-12-27 00:36:32,340][105692] Updated weights for policy 0, policy_version 1266369 (0.0008) [2023-12-27 00:36:32,345][105620] Updated weights for policy 1, policy_version 1267588 (0.0010) [2023-12-27 00:36:32,402][105692] Updated weights for policy 0, policy_version 1266379 (0.0007) [2023-12-27 00:36:32,412][105620] Updated weights for policy 1, policy_version 1267598 (0.0008) [2023-12-27 00:36:32,477][105620] Updated weights for policy 1, policy_version 1267608 (0.0007) [2023-12-27 00:36:33,094][105620] Updated weights for policy 1, policy_version 1267618 (0.0007) [2023-12-27 00:36:33,149][105620] Updated weights for policy 1, policy_version 1267628 (0.0009) [2023-12-27 00:36:33,188][105692] Updated weights for policy 0, policy_version 1266389 (0.0007) [2023-12-27 00:36:33,213][105620] Updated weights for policy 1, policy_version 1267638 (0.0008) [2023-12-27 00:36:33,234][105692] Updated weights for policy 0, policy_version 1266399 (0.0008) [2023-12-27 00:36:33,270][105620] Updated weights for policy 1, policy_version 1267648 (0.0006) [2023-12-27 00:36:33,284][105692] Updated weights for policy 0, policy_version 1266409 (0.0010) [2023-12-27 00:36:33,812][105620] Updated weights for policy 1, policy_version 1267658 (0.0005) [2023-12-27 00:36:33,865][105620] Updated weights for policy 1, policy_version 1267668 (0.0005) [2023-12-27 00:36:33,917][105620] Updated weights for policy 1, policy_version 1267678 (0.0005) [2023-12-27 00:36:34,225][105692] Updated weights for policy 0, policy_version 1266419 (0.0010) [2023-12-27 00:36:34,280][105692] Updated weights for policy 0, policy_version 1266429 (0.0009) [2023-12-27 00:36:34,343][105692] Updated weights for policy 0, policy_version 1266439 (0.0009) [2023-12-27 00:36:34,532][105620] Updated weights for policy 1, policy_version 1267688 (0.0008) [2023-12-27 00:36:34,597][105620] Updated weights for policy 1, policy_version 1267698 (0.0009) [2023-12-27 00:36:34,664][105620] Updated weights for policy 1, policy_version 1267708 (0.0010) [2023-12-27 00:36:35,053][105692] Updated weights for policy 0, policy_version 1266449 (0.0009) [2023-12-27 00:36:35,104][105692] Updated weights for policy 0, policy_version 1266459 (0.0008) [2023-12-27 00:36:35,157][105692] Updated weights for policy 0, policy_version 1266469 (0.0008) [2023-12-27 00:36:35,207][105692] Updated weights for policy 0, policy_version 1266479 (0.0007) [2023-12-27 00:36:35,450][105620] Updated weights for policy 1, policy_version 1267718 (0.0008) [2023-12-27 00:36:35,511][105620] Updated weights for policy 1, policy_version 1267728 (0.0010) [2023-12-27 00:36:35,577][105620] Updated weights for policy 1, policy_version 1267738 (0.0009) [2023-12-27 00:36:35,877][105692] Updated weights for policy 0, policy_version 1266489 (0.0005) [2023-12-27 00:36:35,946][105692] Updated weights for policy 0, policy_version 1266499 (0.0005) [2023-12-27 00:36:36,013][105692] Updated weights for policy 0, policy_version 1266509 (0.0006) [2023-12-27 00:36:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 648863744. Throughput: 0: 9470.2, 1: 9829.2. Samples: 648849284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:36:36,062][104569] Avg episode reward: [(0, '9263.350'), (1, '9172.835')] [2023-12-27 00:36:36,301][105620] Updated weights for policy 1, policy_version 1267748 (0.0010) [2023-12-27 00:36:36,365][105620] Updated weights for policy 1, policy_version 1267758 (0.0011) [2023-12-27 00:36:36,426][105620] Updated weights for policy 1, policy_version 1267768 (0.0011) [2023-12-27 00:36:36,669][105692] Updated weights for policy 0, policy_version 1266519 (0.0007) [2023-12-27 00:36:36,731][105692] Updated weights for policy 0, policy_version 1266529 (0.0008) [2023-12-27 00:36:36,787][105692] Updated weights for policy 0, policy_version 1266539 (0.0008) [2023-12-27 00:36:37,129][105620] Updated weights for policy 1, policy_version 1267778 (0.0011) [2023-12-27 00:36:37,190][105620] Updated weights for policy 1, policy_version 1267788 (0.0010) [2023-12-27 00:36:37,238][105620] Updated weights for policy 1, policy_version 1267798 (0.0010) [2023-12-27 00:36:37,286][105620] Updated weights for policy 1, policy_version 1267808 (0.0010) [2023-12-27 00:36:37,602][105692] Updated weights for policy 0, policy_version 1266549 (0.0009) [2023-12-27 00:36:37,655][105692] Updated weights for policy 0, policy_version 1266559 (0.0006) [2023-12-27 00:36:37,709][105692] Updated weights for policy 0, policy_version 1266569 (0.0006) [2023-12-27 00:36:37,946][105620] Updated weights for policy 1, policy_version 1267818 (0.0005) [2023-12-27 00:36:38,009][105620] Updated weights for policy 1, policy_version 1267828 (0.0005) [2023-12-27 00:36:38,075][105620] Updated weights for policy 1, policy_version 1267838 (0.0008) [2023-12-27 00:36:38,383][105692] Updated weights for policy 0, policy_version 1266579 (0.0006) [2023-12-27 00:36:38,448][105692] Updated weights for policy 0, policy_version 1266589 (0.0010) [2023-12-27 00:36:38,513][105692] Updated weights for policy 0, policy_version 1266599 (0.0011) [2023-12-27 00:36:38,672][105620] Updated weights for policy 1, policy_version 1267848 (0.0009) [2023-12-27 00:36:38,724][105620] Updated weights for policy 1, policy_version 1267858 (0.0008) [2023-12-27 00:36:38,777][105620] Updated weights for policy 1, policy_version 1267868 (0.0008) [2023-12-27 00:36:39,195][105692] Updated weights for policy 0, policy_version 1266609 (0.0011) [2023-12-27 00:36:39,259][105692] Updated weights for policy 0, policy_version 1266619 (0.0009) [2023-12-27 00:36:39,324][105692] Updated weights for policy 0, policy_version 1266629 (0.0008) [2023-12-27 00:36:39,393][105692] Updated weights for policy 0, policy_version 1266639 (0.0008) [2023-12-27 00:36:39,507][105620] Updated weights for policy 1, policy_version 1267878 (0.0009) [2023-12-27 00:36:39,564][105620] Updated weights for policy 1, policy_version 1267888 (0.0009) [2023-12-27 00:36:39,619][105620] Updated weights for policy 1, policy_version 1267899 (0.0010) [2023-12-27 00:36:40,041][105692] Updated weights for policy 0, policy_version 1266649 (0.0010) [2023-12-27 00:36:40,106][105692] Updated weights for policy 0, policy_version 1266659 (0.0010) [2023-12-27 00:36:40,171][105692] Updated weights for policy 0, policy_version 1266669 (0.0006) [2023-12-27 00:36:40,443][105620] Updated weights for policy 1, policy_version 1267909 (0.0009) [2023-12-27 00:36:40,492][105620] Updated weights for policy 1, policy_version 1267919 (0.0008) [2023-12-27 00:36:40,553][105620] Updated weights for policy 1, policy_version 1267929 (0.0008) [2023-12-27 00:36:40,894][105692] Updated weights for policy 0, policy_version 1266679 (0.0009) [2023-12-27 00:36:40,960][105692] Updated weights for policy 0, policy_version 1266689 (0.0011) [2023-12-27 00:36:41,035][105692] Updated weights for policy 0, policy_version 1266699 (0.0009) [2023-12-27 00:36:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.3, 300 sec: 19383.1). Total num frames: 648953856. Throughput: 0: 9561.5, 1: 9816.5. Samples: 648967748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:36:41,062][104569] Avg episode reward: [(0, '9080.356'), (1, '9093.294')] [2023-12-27 00:36:41,329][105620] Updated weights for policy 1, policy_version 1267939 (0.0008) [2023-12-27 00:36:41,395][105620] Updated weights for policy 1, policy_version 1267949 (0.0008) [2023-12-27 00:36:41,462][105620] Updated weights for policy 1, policy_version 1267959 (0.0007) [2023-12-27 00:36:41,775][105692] Updated weights for policy 0, policy_version 1266709 (0.0009) [2023-12-27 00:36:41,834][105692] Updated weights for policy 0, policy_version 1266719 (0.0009) [2023-12-27 00:36:41,898][105692] Updated weights for policy 0, policy_version 1266729 (0.0009) [2023-12-27 00:36:42,194][105620] Updated weights for policy 1, policy_version 1267969 (0.0008) [2023-12-27 00:36:42,257][105620] Updated weights for policy 1, policy_version 1267979 (0.0008) [2023-12-27 00:36:42,322][105620] Updated weights for policy 1, policy_version 1267989 (0.0008) [2023-12-27 00:36:42,392][105620] Updated weights for policy 1, policy_version 1267999 (0.0009) [2023-12-27 00:36:42,690][105692] Updated weights for policy 0, policy_version 1266739 (0.0009) [2023-12-27 00:36:42,750][105692] Updated weights for policy 0, policy_version 1266749 (0.0009) [2023-12-27 00:36:42,809][105692] Updated weights for policy 0, policy_version 1266759 (0.0009) [2023-12-27 00:36:43,125][105620] Updated weights for policy 1, policy_version 1268009 (0.0009) [2023-12-27 00:36:43,176][105620] Updated weights for policy 1, policy_version 1268019 (0.0009) [2023-12-27 00:36:43,223][105620] Updated weights for policy 1, policy_version 1268029 (0.0009) [2023-12-27 00:36:43,560][105692] Updated weights for policy 0, policy_version 1266769 (0.0009) [2023-12-27 00:36:43,611][105692] Updated weights for policy 0, policy_version 1266779 (0.0009) [2023-12-27 00:36:43,659][105692] Updated weights for policy 0, policy_version 1266789 (0.0009) [2023-12-27 00:36:43,709][105692] Updated weights for policy 0, policy_version 1266799 (0.0009) [2023-12-27 00:36:43,981][105620] Updated weights for policy 1, policy_version 1268039 (0.0009) [2023-12-27 00:36:44,048][105620] Updated weights for policy 1, policy_version 1268049 (0.0009) [2023-12-27 00:36:44,094][105620] Updated weights for policy 1, policy_version 1268059 (0.0008) [2023-12-27 00:36:44,453][105692] Updated weights for policy 0, policy_version 1266809 (0.0007) [2023-12-27 00:36:44,503][105692] Updated weights for policy 0, policy_version 1266819 (0.0008) [2023-12-27 00:36:44,554][105692] Updated weights for policy 0, policy_version 1266829 (0.0005) [2023-12-27 00:36:44,906][105620] Updated weights for policy 1, policy_version 1268069 (0.0009) [2023-12-27 00:36:44,975][105620] Updated weights for policy 1, policy_version 1268079 (0.0008) [2023-12-27 00:36:45,046][105620] Updated weights for policy 1, policy_version 1268089 (0.0008) [2023-12-27 00:36:45,191][105692] Updated weights for policy 0, policy_version 1266839 (0.0008) [2023-12-27 00:36:45,255][105692] Updated weights for policy 0, policy_version 1266849 (0.0009) [2023-12-27 00:36:45,319][105692] Updated weights for policy 0, policy_version 1266859 (0.0009) [2023-12-27 00:36:45,770][105620] Updated weights for policy 1, policy_version 1268099 (0.0007) [2023-12-27 00:36:45,825][105620] Updated weights for policy 1, policy_version 1268109 (0.0011) [2023-12-27 00:36:45,879][105620] Updated weights for policy 1, policy_version 1268119 (0.0011) [2023-12-27 00:36:46,030][105692] Updated weights for policy 0, policy_version 1266869 (0.0010) [2023-12-27 00:36:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 649052160. Throughput: 0: 9541.2, 1: 9807.6. Samples: 649022988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:36:46,062][104569] Avg episode reward: [(0, '8894.996'), (1, '9093.280')] [2023-12-27 00:36:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001268128_324681728.pth... [2023-12-27 00:36:46,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001267008_324395008.pth [2023-12-27 00:36:46,088][105692] Updated weights for policy 0, policy_version 1266879 (0.0010) [2023-12-27 00:36:46,139][105692] Updated weights for policy 0, policy_version 1266889 (0.0005) [2023-12-27 00:36:46,168][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001266896_324378624.pth... [2023-12-27 00:36:46,171][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001265744_324083712.pth [2023-12-27 00:36:46,624][105620] Updated weights for policy 1, policy_version 1268129 (0.0010) [2023-12-27 00:36:46,684][105620] Updated weights for policy 1, policy_version 1268139 (0.0010) [2023-12-27 00:36:46,750][105620] Updated weights for policy 1, policy_version 1268149 (0.0005) [2023-12-27 00:36:46,803][105620] Updated weights for policy 1, policy_version 1268159 (0.0005) [2023-12-27 00:36:46,819][105692] Updated weights for policy 0, policy_version 1266899 (0.0007) [2023-12-27 00:36:46,870][105692] Updated weights for policy 0, policy_version 1266909 (0.0010) [2023-12-27 00:36:46,934][105692] Updated weights for policy 0, policy_version 1266919 (0.0010) [2023-12-27 00:36:47,330][105620] Updated weights for policy 1, policy_version 1268169 (0.0005) [2023-12-27 00:36:47,381][105620] Updated weights for policy 1, policy_version 1268179 (0.0005) [2023-12-27 00:36:47,437][105620] Updated weights for policy 1, policy_version 1268189 (0.0005) [2023-12-27 00:36:47,684][105692] Updated weights for policy 0, policy_version 1266929 (0.0011) [2023-12-27 00:36:47,751][105692] Updated weights for policy 0, policy_version 1266939 (0.0011) [2023-12-27 00:36:47,817][105692] Updated weights for policy 0, policy_version 1266949 (0.0011) [2023-12-27 00:36:47,889][105692] Updated weights for policy 0, policy_version 1266959 (0.0010) [2023-12-27 00:36:47,974][105620] Updated weights for policy 1, policy_version 1268199 (0.0005) [2023-12-27 00:36:48,035][105620] Updated weights for policy 1, policy_version 1268209 (0.0006) [2023-12-27 00:36:48,088][105620] Updated weights for policy 1, policy_version 1268219 (0.0007) [2023-12-27 00:36:48,586][105692] Updated weights for policy 0, policy_version 1266969 (0.0011) [2023-12-27 00:36:48,637][105692] Updated weights for policy 0, policy_version 1266979 (0.0010) [2023-12-27 00:36:48,691][105692] Updated weights for policy 0, policy_version 1266989 (0.0008) [2023-12-27 00:36:48,705][105620] Updated weights for policy 1, policy_version 1268229 (0.0007) [2023-12-27 00:36:48,764][105620] Updated weights for policy 1, policy_version 1268239 (0.0008) [2023-12-27 00:36:48,816][105620] Updated weights for policy 1, policy_version 1268249 (0.0008) [2023-12-27 00:36:49,465][105692] Updated weights for policy 0, policy_version 1266999 (0.0009) [2023-12-27 00:36:49,501][105620] Updated weights for policy 1, policy_version 1268259 (0.0008) [2023-12-27 00:36:49,518][105692] Updated weights for policy 0, policy_version 1267009 (0.0008) [2023-12-27 00:36:49,557][105620] Updated weights for policy 1, policy_version 1268269 (0.0008) [2023-12-27 00:36:49,573][105692] Updated weights for policy 0, policy_version 1267019 (0.0008) [2023-12-27 00:36:49,608][105620] Updated weights for policy 1, policy_version 1268279 (0.0006) [2023-12-27 00:36:50,279][105620] Updated weights for policy 1, policy_version 1268289 (0.0008) [2023-12-27 00:36:50,344][105620] Updated weights for policy 1, policy_version 1268299 (0.0009) [2023-12-27 00:36:50,349][105692] Updated weights for policy 0, policy_version 1267029 (0.0007) [2023-12-27 00:36:50,396][105620] Updated weights for policy 1, policy_version 1268309 (0.0009) [2023-12-27 00:36:50,407][105692] Updated weights for policy 0, policy_version 1267039 (0.0006) [2023-12-27 00:36:50,450][105620] Updated weights for policy 1, policy_version 1268319 (0.0008) [2023-12-27 00:36:50,465][105692] Updated weights for policy 0, policy_version 1267049 (0.0006) [2023-12-27 00:36:51,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 649150464. Throughput: 0: 9603.5, 1: 9843.2. Samples: 649142616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:36:51,063][104569] Avg episode reward: [(0, '8986.183'), (1, '9264.673')] [2023-12-27 00:36:51,112][105620] Updated weights for policy 1, policy_version 1268329 (0.0007) [2023-12-27 00:36:51,174][105620] Updated weights for policy 1, policy_version 1268339 (0.0007) [2023-12-27 00:36:51,230][105692] Updated weights for policy 0, policy_version 1267059 (0.0009) [2023-12-27 00:36:51,232][105620] Updated weights for policy 1, policy_version 1268349 (0.0006) [2023-12-27 00:36:51,293][105692] Updated weights for policy 0, policy_version 1267069 (0.0008) [2023-12-27 00:36:51,355][105692] Updated weights for policy 0, policy_version 1267079 (0.0009) [2023-12-27 00:36:51,888][105620] Updated weights for policy 1, policy_version 1268359 (0.0006) [2023-12-27 00:36:51,940][105620] Updated weights for policy 1, policy_version 1268369 (0.0005) [2023-12-27 00:36:51,993][105620] Updated weights for policy 1, policy_version 1268379 (0.0005) [2023-12-27 00:36:52,220][105692] Updated weights for policy 0, policy_version 1267089 (0.0009) [2023-12-27 00:36:52,288][105692] Updated weights for policy 0, policy_version 1267099 (0.0008) [2023-12-27 00:36:52,354][105692] Updated weights for policy 0, policy_version 1267109 (0.0009) [2023-12-27 00:36:52,418][105692] Updated weights for policy 0, policy_version 1267119 (0.0010) [2023-12-27 00:36:52,689][105620] Updated weights for policy 1, policy_version 1268389 (0.0007) [2023-12-27 00:36:52,743][105620] Updated weights for policy 1, policy_version 1268399 (0.0007) [2023-12-27 00:36:52,801][105620] Updated weights for policy 1, policy_version 1268409 (0.0006) [2023-12-27 00:36:53,132][105692] Updated weights for policy 0, policy_version 1267129 (0.0009) [2023-12-27 00:36:53,187][105692] Updated weights for policy 0, policy_version 1267139 (0.0010) [2023-12-27 00:36:53,253][105692] Updated weights for policy 0, policy_version 1267149 (0.0010) [2023-12-27 00:36:53,529][105620] Updated weights for policy 1, policy_version 1268419 (0.0008) [2023-12-27 00:36:53,588][105620] Updated weights for policy 1, policy_version 1268429 (0.0010) [2023-12-27 00:36:53,641][105620] Updated weights for policy 1, policy_version 1268439 (0.0011) [2023-12-27 00:36:53,895][105692] Updated weights for policy 0, policy_version 1267159 (0.0008) [2023-12-27 00:36:53,953][105692] Updated weights for policy 0, policy_version 1267169 (0.0005) [2023-12-27 00:36:54,021][105692] Updated weights for policy 0, policy_version 1267179 (0.0005) [2023-12-27 00:36:54,470][105620] Updated weights for policy 1, policy_version 1268449 (0.0010) [2023-12-27 00:36:54,536][105620] Updated weights for policy 1, policy_version 1268459 (0.0011) [2023-12-27 00:36:54,595][105620] Updated weights for policy 1, policy_version 1268469 (0.0010) [2023-12-27 00:36:54,599][105692] Updated weights for policy 0, policy_version 1267189 (0.0005) [2023-12-27 00:36:54,658][105620] Updated weights for policy 1, policy_version 1268479 (0.0009) [2023-12-27 00:36:54,662][105692] Updated weights for policy 0, policy_version 1267199 (0.0008) [2023-12-27 00:36:54,717][105692] Updated weights for policy 0, policy_version 1267209 (0.0011) [2023-12-27 00:36:55,244][105620] Updated weights for policy 1, policy_version 1268489 (0.0009) [2023-12-27 00:36:55,282][105692] Updated weights for policy 0, policy_version 1267219 (0.0006) [2023-12-27 00:36:55,302][105620] Updated weights for policy 1, policy_version 1268499 (0.0010) [2023-12-27 00:36:55,329][105692] Updated weights for policy 0, policy_version 1267229 (0.0008) [2023-12-27 00:36:55,361][105620] Updated weights for policy 1, policy_version 1268509 (0.0011) [2023-12-27 00:36:55,379][105692] Updated weights for policy 0, policy_version 1267239 (0.0008) [2023-12-27 00:36:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 649248768. Throughput: 0: 9628.1, 1: 9738.7. Samples: 649261772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:36:56,062][104569] Avg episode reward: [(0, '9081.763'), (1, '9264.975')] [2023-12-27 00:36:56,092][105620] Updated weights for policy 1, policy_version 1268519 (0.0011) [2023-12-27 00:36:56,141][105620] Updated weights for policy 1, policy_version 1268529 (0.0010) [2023-12-27 00:36:56,163][105692] Updated weights for policy 0, policy_version 1267249 (0.0007) [2023-12-27 00:36:56,200][105620] Updated weights for policy 1, policy_version 1268539 (0.0010) [2023-12-27 00:36:56,222][105692] Updated weights for policy 0, policy_version 1267259 (0.0006) [2023-12-27 00:36:56,274][105692] Updated weights for policy 0, policy_version 1267269 (0.0007) [2023-12-27 00:36:56,321][105692] Updated weights for policy 0, policy_version 1267279 (0.0008) [2023-12-27 00:36:56,948][105620] Updated weights for policy 1, policy_version 1268549 (0.0010) [2023-12-27 00:36:56,992][105620] Updated weights for policy 1, policy_version 1268559 (0.0008) [2023-12-27 00:36:57,039][105620] Updated weights for policy 1, policy_version 1268569 (0.0005) [2023-12-27 00:36:57,052][105692] Updated weights for policy 0, policy_version 1267290 (0.0008) [2023-12-27 00:36:57,103][105692] Updated weights for policy 0, policy_version 1267300 (0.0009) [2023-12-27 00:36:57,156][105692] Updated weights for policy 0, policy_version 1267312 (0.0010) [2023-12-27 00:36:57,635][105620] Updated weights for policy 1, policy_version 1268579 (0.0006) [2023-12-27 00:36:57,687][105620] Updated weights for policy 1, policy_version 1268589 (0.0008) [2023-12-27 00:36:57,743][105620] Updated weights for policy 1, policy_version 1268599 (0.0010) [2023-12-27 00:36:57,942][105692] Updated weights for policy 0, policy_version 1267322 (0.0008) [2023-12-27 00:36:57,986][105692] Updated weights for policy 0, policy_version 1267332 (0.0007) [2023-12-27 00:36:58,034][105692] Updated weights for policy 0, policy_version 1267342 (0.0008) [2023-12-27 00:36:58,501][105620] Updated weights for policy 1, policy_version 1268609 (0.0010) [2023-12-27 00:36:58,561][105620] Updated weights for policy 1, policy_version 1268619 (0.0010) [2023-12-27 00:36:58,627][105620] Updated weights for policy 1, policy_version 1268629 (0.0009) [2023-12-27 00:36:58,689][105620] Updated weights for policy 1, policy_version 1268639 (0.0007) [2023-12-27 00:36:58,916][105692] Updated weights for policy 0, policy_version 1267352 (0.0009) [2023-12-27 00:36:58,983][105692] Updated weights for policy 0, policy_version 1267362 (0.0009) [2023-12-27 00:36:59,046][105692] Updated weights for policy 0, policy_version 1267372 (0.0009) [2023-12-27 00:36:59,576][105620] Updated weights for policy 1, policy_version 1268649 (0.0009) [2023-12-27 00:36:59,637][105620] Updated weights for policy 1, policy_version 1268659 (0.0007) [2023-12-27 00:36:59,698][105620] Updated weights for policy 1, policy_version 1268669 (0.0006) [2023-12-27 00:36:59,781][105692] Updated weights for policy 0, policy_version 1267382 (0.0008) [2023-12-27 00:36:59,837][105692] Updated weights for policy 0, policy_version 1267392 (0.0009) [2023-12-27 00:36:59,900][105692] Updated weights for policy 0, policy_version 1267402 (0.0009) [2023-12-27 00:37:00,347][105620] Updated weights for policy 1, policy_version 1268679 (0.0009) [2023-12-27 00:37:00,397][105620] Updated weights for policy 1, policy_version 1268689 (0.0010) [2023-12-27 00:37:00,448][105620] Updated weights for policy 1, policy_version 1268699 (0.0009) [2023-12-27 00:37:00,661][105692] Updated weights for policy 0, policy_version 1267412 (0.0008) [2023-12-27 00:37:00,713][105692] Updated weights for policy 0, policy_version 1267422 (0.0010) [2023-12-27 00:37:00,772][105692] Updated weights for policy 0, policy_version 1267432 (0.0009) [2023-12-27 00:37:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 649347072. Throughput: 0: 9620.4, 1: 9757.9. Samples: 649319024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:37:01,063][104569] Avg episode reward: [(0, '9175.508'), (1, '9265.037')] [2023-12-27 00:37:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001267440_324517888.pth... [2023-12-27 00:37:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001268704_324829184.pth... [2023-12-27 00:37:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001266320_324231168.pth [2023-12-27 00:37:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001267552_324534272.pth [2023-12-27 00:37:01,121][105620] Updated weights for policy 1, policy_version 1268709 (0.0009) [2023-12-27 00:37:01,182][105620] Updated weights for policy 1, policy_version 1268719 (0.0009) [2023-12-27 00:37:01,246][105620] Updated weights for policy 1, policy_version 1268729 (0.0008) [2023-12-27 00:37:01,609][105692] Updated weights for policy 0, policy_version 1267442 (0.0009) [2023-12-27 00:37:01,675][105692] Updated weights for policy 0, policy_version 1267452 (0.0008) [2023-12-27 00:37:01,742][105692] Updated weights for policy 0, policy_version 1267462 (0.0007) [2023-12-27 00:37:01,803][105692] Updated weights for policy 0, policy_version 1267472 (0.0010) [2023-12-27 00:37:01,964][105620] Updated weights for policy 1, policy_version 1268739 (0.0007) [2023-12-27 00:37:02,016][105620] Updated weights for policy 1, policy_version 1268749 (0.0009) [2023-12-27 00:37:02,075][105620] Updated weights for policy 1, policy_version 1268759 (0.0007) [2023-12-27 00:37:02,591][105692] Updated weights for policy 0, policy_version 1267482 (0.0008) [2023-12-27 00:37:02,655][105692] Updated weights for policy 0, policy_version 1267492 (0.0008) [2023-12-27 00:37:02,683][105620] Updated weights for policy 1, policy_version 1268769 (0.0006) [2023-12-27 00:37:02,715][105692] Updated weights for policy 0, policy_version 1267502 (0.0009) [2023-12-27 00:37:02,748][105620] Updated weights for policy 1, policy_version 1268779 (0.0010) [2023-12-27 00:37:02,806][105620] Updated weights for policy 1, policy_version 1268789 (0.0010) [2023-12-27 00:37:02,875][105620] Updated weights for policy 1, policy_version 1268799 (0.0010) [2023-12-27 00:37:03,439][105692] Updated weights for policy 0, policy_version 1267512 (0.0007) [2023-12-27 00:37:03,499][105692] Updated weights for policy 0, policy_version 1267522 (0.0008) [2023-12-27 00:37:03,540][105620] Updated weights for policy 1, policy_version 1268809 (0.0011) [2023-12-27 00:37:03,554][105692] Updated weights for policy 0, policy_version 1267532 (0.0007) [2023-12-27 00:37:03,603][105620] Updated weights for policy 1, policy_version 1268819 (0.0010) [2023-12-27 00:37:03,658][105620] Updated weights for policy 1, policy_version 1268829 (0.0011) [2023-12-27 00:37:04,281][105692] Updated weights for policy 0, policy_version 1267542 (0.0006) [2023-12-27 00:37:04,339][105692] Updated weights for policy 0, policy_version 1267552 (0.0005) [2023-12-27 00:37:04,403][105692] Updated weights for policy 0, policy_version 1267562 (0.0006) [2023-12-27 00:37:04,411][105620] Updated weights for policy 1, policy_version 1268839 (0.0008) [2023-12-27 00:37:04,466][105620] Updated weights for policy 1, policy_version 1268849 (0.0010) [2023-12-27 00:37:04,515][105620] Updated weights for policy 1, policy_version 1268859 (0.0011) [2023-12-27 00:37:05,040][105692] Updated weights for policy 0, policy_version 1267572 (0.0006) [2023-12-27 00:37:05,088][105692] Updated weights for policy 0, policy_version 1267582 (0.0007) [2023-12-27 00:37:05,138][105692] Updated weights for policy 0, policy_version 1267592 (0.0008) [2023-12-27 00:37:05,314][105620] Updated weights for policy 1, policy_version 1268869 (0.0009) [2023-12-27 00:37:05,364][105620] Updated weights for policy 1, policy_version 1268880 (0.0009) [2023-12-27 00:37:05,416][105620] Updated weights for policy 1, policy_version 1268890 (0.0009) [2023-12-27 00:37:05,808][105692] Updated weights for policy 0, policy_version 1267602 (0.0006) [2023-12-27 00:37:05,860][105692] Updated weights for policy 0, policy_version 1267612 (0.0009) [2023-12-27 00:37:05,913][105692] Updated weights for policy 0, policy_version 1267622 (0.0005) [2023-12-27 00:37:05,970][105692] Updated weights for policy 0, policy_version 1267632 (0.0005) [2023-12-27 00:37:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 649445376. Throughput: 0: 9527.7, 1: 9798.8. Samples: 649433500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:37:06,062][105620] Updated weights for policy 1, policy_version 1268900 (0.0006) [2023-12-27 00:37:06,063][104569] Avg episode reward: [(0, '9084.781'), (1, '9264.991')] [2023-12-27 00:37:06,116][105620] Updated weights for policy 1, policy_version 1268910 (0.0007) [2023-12-27 00:37:06,169][105620] Updated weights for policy 1, policy_version 1268920 (0.0009) [2023-12-27 00:37:06,597][105692] Updated weights for policy 0, policy_version 1267642 (0.0008) [2023-12-27 00:37:06,663][105692] Updated weights for policy 0, policy_version 1267652 (0.0008) [2023-12-27 00:37:06,729][105692] Updated weights for policy 0, policy_version 1267662 (0.0007) [2023-12-27 00:37:06,998][105620] Updated weights for policy 1, policy_version 1268930 (0.0009) [2023-12-27 00:37:07,069][105620] Updated weights for policy 1, policy_version 1268940 (0.0010) [2023-12-27 00:37:07,127][105620] Updated weights for policy 1, policy_version 1268950 (0.0009) [2023-12-27 00:37:07,407][105692] Updated weights for policy 0, policy_version 1267672 (0.0007) [2023-12-27 00:37:07,468][105692] Updated weights for policy 0, policy_version 1267682 (0.0007) [2023-12-27 00:37:07,516][105692] Updated weights for policy 0, policy_version 1267692 (0.0007) [2023-12-27 00:37:07,891][105620] Updated weights for policy 1, policy_version 1268961 (0.0011) [2023-12-27 00:37:07,950][105620] Updated weights for policy 1, policy_version 1268971 (0.0010) [2023-12-27 00:37:08,003][105620] Updated weights for policy 1, policy_version 1268981 (0.0010) [2023-12-27 00:37:08,062][105620] Updated weights for policy 1, policy_version 1268991 (0.0010) [2023-12-27 00:37:08,271][105692] Updated weights for policy 0, policy_version 1267702 (0.0007) [2023-12-27 00:37:08,329][105692] Updated weights for policy 0, policy_version 1267712 (0.0006) [2023-12-27 00:37:08,386][105692] Updated weights for policy 0, policy_version 1267722 (0.0008) [2023-12-27 00:37:08,827][105620] Updated weights for policy 1, policy_version 1269001 (0.0011) [2023-12-27 00:37:08,885][105620] Updated weights for policy 1, policy_version 1269011 (0.0010) [2023-12-27 00:37:08,951][105620] Updated weights for policy 1, policy_version 1269021 (0.0006) [2023-12-27 00:37:09,130][105692] Updated weights for policy 0, policy_version 1267732 (0.0009) [2023-12-27 00:37:09,189][105692] Updated weights for policy 0, policy_version 1267742 (0.0009) [2023-12-27 00:37:09,257][105692] Updated weights for policy 0, policy_version 1267752 (0.0008) [2023-12-27 00:37:09,606][105620] Updated weights for policy 1, policy_version 1269031 (0.0009) [2023-12-27 00:37:09,668][105620] Updated weights for policy 1, policy_version 1269041 (0.0011) [2023-12-27 00:37:09,728][105620] Updated weights for policy 1, policy_version 1269051 (0.0011) [2023-12-27 00:37:10,004][105692] Updated weights for policy 0, policy_version 1267762 (0.0008) [2023-12-27 00:37:10,061][105692] Updated weights for policy 0, policy_version 1267772 (0.0007) [2023-12-27 00:37:10,123][105692] Updated weights for policy 0, policy_version 1267782 (0.0006) [2023-12-27 00:37:10,192][105692] Updated weights for policy 0, policy_version 1267792 (0.0006) [2023-12-27 00:37:10,456][105620] Updated weights for policy 1, policy_version 1269061 (0.0011) [2023-12-27 00:37:10,508][105620] Updated weights for policy 1, policy_version 1269071 (0.0010) [2023-12-27 00:37:10,567][105620] Updated weights for policy 1, policy_version 1269081 (0.0011) [2023-12-27 00:37:10,851][105692] Updated weights for policy 0, policy_version 1267802 (0.0008) [2023-12-27 00:37:10,904][105692] Updated weights for policy 0, policy_version 1267812 (0.0009) [2023-12-27 00:37:10,952][105692] Updated weights for policy 0, policy_version 1267822 (0.0008) [2023-12-27 00:37:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 649543680. Throughput: 0: 9610.8, 1: 9782.0. Samples: 649550124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:37:11,063][104569] Avg episode reward: [(0, '8809.292'), (1, '9355.752')] [2023-12-27 00:37:11,339][105620] Updated weights for policy 1, policy_version 1269091 (0.0010) [2023-12-27 00:37:11,410][105620] Updated weights for policy 1, policy_version 1269101 (0.0008) [2023-12-27 00:37:11,483][105620] Updated weights for policy 1, policy_version 1269111 (0.0010) [2023-12-27 00:37:11,721][105692] Updated weights for policy 0, policy_version 1267832 (0.0007) [2023-12-27 00:37:11,788][105692] Updated weights for policy 0, policy_version 1267842 (0.0007) [2023-12-27 00:37:11,851][105692] Updated weights for policy 0, policy_version 1267852 (0.0009) [2023-12-27 00:37:12,272][105620] Updated weights for policy 1, policy_version 1269121 (0.0008) [2023-12-27 00:37:12,338][105620] Updated weights for policy 1, policy_version 1269131 (0.0008) [2023-12-27 00:37:12,406][105620] Updated weights for policy 1, policy_version 1269141 (0.0007) [2023-12-27 00:37:12,467][105620] Updated weights for policy 1, policy_version 1269151 (0.0006) [2023-12-27 00:37:12,590][105692] Updated weights for policy 0, policy_version 1267862 (0.0010) [2023-12-27 00:37:12,644][105692] Updated weights for policy 0, policy_version 1267872 (0.0009) [2023-12-27 00:37:12,699][105692] Updated weights for policy 0, policy_version 1267882 (0.0009) [2023-12-27 00:37:13,077][105620] Updated weights for policy 1, policy_version 1269161 (0.0006) [2023-12-27 00:37:13,132][105620] Updated weights for policy 1, policy_version 1269171 (0.0005) [2023-12-27 00:37:13,196][105620] Updated weights for policy 1, policy_version 1269181 (0.0007) [2023-12-27 00:37:13,540][105692] Updated weights for policy 0, policy_version 1267892 (0.0010) [2023-12-27 00:37:13,595][105692] Updated weights for policy 0, policy_version 1267903 (0.0010) [2023-12-27 00:37:13,647][105692] Updated weights for policy 0, policy_version 1267913 (0.0009) [2023-12-27 00:37:13,797][105620] Updated weights for policy 1, policy_version 1269191 (0.0006) [2023-12-27 00:37:13,850][105620] Updated weights for policy 1, policy_version 1269201 (0.0005) [2023-12-27 00:37:13,907][105620] Updated weights for policy 1, policy_version 1269211 (0.0006) [2023-12-27 00:37:14,501][105620] Updated weights for policy 1, policy_version 1269221 (0.0007) [2023-12-27 00:37:14,516][105692] Updated weights for policy 0, policy_version 1267923 (0.0008) [2023-12-27 00:37:14,558][105620] Updated weights for policy 1, policy_version 1269231 (0.0006) [2023-12-27 00:37:14,576][105692] Updated weights for policy 0, policy_version 1267933 (0.0007) [2023-12-27 00:37:14,614][105620] Updated weights for policy 1, policy_version 1269241 (0.0007) [2023-12-27 00:37:14,633][105692] Updated weights for policy 0, policy_version 1267943 (0.0006) [2023-12-27 00:37:15,293][105692] Updated weights for policy 0, policy_version 1267953 (0.0007) [2023-12-27 00:37:15,350][105692] Updated weights for policy 0, policy_version 1267963 (0.0005) [2023-12-27 00:37:15,404][105692] Updated weights for policy 0, policy_version 1267973 (0.0005) [2023-12-27 00:37:15,444][105620] Updated weights for policy 1, policy_version 1269251 (0.0007) [2023-12-27 00:37:15,463][105692] Updated weights for policy 0, policy_version 1267983 (0.0006) [2023-12-27 00:37:15,506][105620] Updated weights for policy 1, policy_version 1269261 (0.0009) [2023-12-27 00:37:15,558][105620] Updated weights for policy 1, policy_version 1269271 (0.0010) [2023-12-27 00:37:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 649633792. Throughput: 0: 9567.4, 1: 9844.8. Samples: 649607500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:37:16,062][104569] Avg episode reward: [(0, '8899.050'), (1, '9355.523')] [2023-12-27 00:37:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001269280_324976640.pth... [2023-12-27 00:37:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001268128_324681728.pth [2023-12-27 00:37:16,130][105692] Updated weights for policy 0, policy_version 1267993 (0.0009) [2023-12-27 00:37:16,185][105692] Updated weights for policy 0, policy_version 1268003 (0.0010) [2023-12-27 00:37:16,250][105692] Updated weights for policy 0, policy_version 1268013 (0.0010) [2023-12-27 00:37:16,266][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001268016_324665344.pth... [2023-12-27 00:37:16,270][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001266896_324378624.pth [2023-12-27 00:37:16,302][105620] Updated weights for policy 1, policy_version 1269281 (0.0008) [2023-12-27 00:37:16,358][105620] Updated weights for policy 1, policy_version 1269291 (0.0005) [2023-12-27 00:37:16,420][105620] Updated weights for policy 1, policy_version 1269301 (0.0007) [2023-12-27 00:37:16,482][105620] Updated weights for policy 1, policy_version 1269311 (0.0009) [2023-12-27 00:37:17,079][105692] Updated weights for policy 0, policy_version 1268023 (0.0009) [2023-12-27 00:37:17,116][105620] Updated weights for policy 1, policy_version 1269321 (0.0009) [2023-12-27 00:37:17,135][105692] Updated weights for policy 0, policy_version 1268033 (0.0005) [2023-12-27 00:37:17,165][105620] Updated weights for policy 1, policy_version 1269331 (0.0007) [2023-12-27 00:37:17,191][105692] Updated weights for policy 0, policy_version 1268043 (0.0007) [2023-12-27 00:37:17,215][105620] Updated weights for policy 1, policy_version 1269341 (0.0005) [2023-12-27 00:37:17,939][105620] Updated weights for policy 1, policy_version 1269351 (0.0008) [2023-12-27 00:37:17,956][105692] Updated weights for policy 0, policy_version 1268053 (0.0009) [2023-12-27 00:37:17,997][105620] Updated weights for policy 1, policy_version 1269361 (0.0008) [2023-12-27 00:37:18,012][105692] Updated weights for policy 0, policy_version 1268063 (0.0007) [2023-12-27 00:37:18,047][105620] Updated weights for policy 1, policy_version 1269371 (0.0006) [2023-12-27 00:37:18,061][105692] Updated weights for policy 0, policy_version 1268073 (0.0006) [2023-12-27 00:37:18,712][105620] Updated weights for policy 1, policy_version 1269381 (0.0008) [2023-12-27 00:37:18,777][105620] Updated weights for policy 1, policy_version 1269391 (0.0009) [2023-12-27 00:37:18,835][105620] Updated weights for policy 1, policy_version 1269401 (0.0009) [2023-12-27 00:37:18,882][105692] Updated weights for policy 0, policy_version 1268083 (0.0007) [2023-12-27 00:37:18,944][105692] Updated weights for policy 0, policy_version 1268093 (0.0009) [2023-12-27 00:37:19,003][105692] Updated weights for policy 0, policy_version 1268103 (0.0009) [2023-12-27 00:37:19,588][105620] Updated weights for policy 1, policy_version 1269411 (0.0009) [2023-12-27 00:37:19,651][105620] Updated weights for policy 1, policy_version 1269421 (0.0009) [2023-12-27 00:37:19,710][105620] Updated weights for policy 1, policy_version 1269431 (0.0009) [2023-12-27 00:37:19,787][105692] Updated weights for policy 0, policy_version 1268113 (0.0009) [2023-12-27 00:37:19,854][105692] Updated weights for policy 0, policy_version 1268123 (0.0009) [2023-12-27 00:37:19,920][105692] Updated weights for policy 0, policy_version 1268133 (0.0009) [2023-12-27 00:37:19,989][105692] Updated weights for policy 0, policy_version 1268143 (0.0009) [2023-12-27 00:37:20,374][105620] Updated weights for policy 1, policy_version 1269441 (0.0009) [2023-12-27 00:37:20,434][105620] Updated weights for policy 1, policy_version 1269451 (0.0007) [2023-12-27 00:37:20,495][105620] Updated weights for policy 1, policy_version 1269461 (0.0006) [2023-12-27 00:37:20,548][105620] Updated weights for policy 1, policy_version 1269471 (0.0006) [2023-12-27 00:37:20,837][105692] Updated weights for policy 0, policy_version 1268153 (0.0009) [2023-12-27 00:37:20,908][105692] Updated weights for policy 0, policy_version 1268163 (0.0008) [2023-12-27 00:37:20,974][105692] Updated weights for policy 0, policy_version 1268173 (0.0008) [2023-12-27 00:37:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 649732096. Throughput: 0: 9538.5, 1: 9822.4. Samples: 649720524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:37:21,062][104569] Avg episode reward: [(0, '9082.697'), (1, '9279.181')] [2023-12-27 00:37:21,302][105620] Updated weights for policy 1, policy_version 1269481 (0.0009) [2023-12-27 00:37:21,361][105620] Updated weights for policy 1, policy_version 1269491 (0.0009) [2023-12-27 00:37:21,439][105620] Updated weights for policy 1, policy_version 1269501 (0.0009) [2023-12-27 00:37:21,701][105692] Updated weights for policy 0, policy_version 1268183 (0.0009) [2023-12-27 00:37:21,775][105692] Updated weights for policy 0, policy_version 1268193 (0.0007) [2023-12-27 00:37:21,837][105692] Updated weights for policy 0, policy_version 1268203 (0.0008) [2023-12-27 00:37:22,204][105620] Updated weights for policy 1, policy_version 1269511 (0.0009) [2023-12-27 00:37:22,272][105620] Updated weights for policy 1, policy_version 1269521 (0.0008) [2023-12-27 00:37:22,321][105620] Updated weights for policy 1, policy_version 1269531 (0.0009) [2023-12-27 00:37:22,579][105692] Updated weights for policy 0, policy_version 1268213 (0.0009) [2023-12-27 00:37:22,638][105692] Updated weights for policy 0, policy_version 1268223 (0.0009) [2023-12-27 00:37:22,697][105692] Updated weights for policy 0, policy_version 1268233 (0.0009) [2023-12-27 00:37:23,090][105620] Updated weights for policy 1, policy_version 1269541 (0.0009) [2023-12-27 00:37:23,153][105620] Updated weights for policy 1, policy_version 1269551 (0.0009) [2023-12-27 00:37:23,215][105620] Updated weights for policy 1, policy_version 1269561 (0.0009) [2023-12-27 00:37:23,418][105692] Updated weights for policy 0, policy_version 1268243 (0.0007) [2023-12-27 00:37:23,478][105692] Updated weights for policy 0, policy_version 1268253 (0.0010) [2023-12-27 00:37:23,532][105692] Updated weights for policy 0, policy_version 1268263 (0.0010) [2023-12-27 00:37:23,811][105620] Updated weights for policy 1, policy_version 1269571 (0.0007) [2023-12-27 00:37:23,869][105620] Updated weights for policy 1, policy_version 1269581 (0.0005) [2023-12-27 00:37:23,922][105620] Updated weights for policy 1, policy_version 1269591 (0.0005) [2023-12-27 00:37:24,177][105692] Updated weights for policy 0, policy_version 1268273 (0.0006) [2023-12-27 00:37:24,225][105692] Updated weights for policy 0, policy_version 1268283 (0.0008) [2023-12-27 00:37:24,281][105692] Updated weights for policy 0, policy_version 1268293 (0.0008) [2023-12-27 00:37:24,341][105692] Updated weights for policy 0, policy_version 1268303 (0.0008) [2023-12-27 00:37:24,566][105620] Updated weights for policy 1, policy_version 1269601 (0.0007) [2023-12-27 00:37:24,624][105620] Updated weights for policy 1, policy_version 1269611 (0.0010) [2023-12-27 00:37:24,683][105620] Updated weights for policy 1, policy_version 1269621 (0.0008) [2023-12-27 00:37:24,749][105620] Updated weights for policy 1, policy_version 1269631 (0.0010) [2023-12-27 00:37:25,141][105692] Updated weights for policy 0, policy_version 1268313 (0.0009) [2023-12-27 00:37:25,190][105692] Updated weights for policy 0, policy_version 1268323 (0.0008) [2023-12-27 00:37:25,246][105692] Updated weights for policy 0, policy_version 1268333 (0.0005) [2023-12-27 00:37:25,458][105620] Updated weights for policy 1, policy_version 1269641 (0.0010) [2023-12-27 00:37:25,514][105620] Updated weights for policy 1, policy_version 1269652 (0.0009) [2023-12-27 00:37:25,573][105620] Updated weights for policy 1, policy_version 1269662 (0.0005) [2023-12-27 00:37:25,820][105692] Updated weights for policy 0, policy_version 1268343 (0.0007) [2023-12-27 00:37:25,873][105692] Updated weights for policy 0, policy_version 1268353 (0.0007) [2023-12-27 00:37:25,925][105692] Updated weights for policy 0, policy_version 1268363 (0.0005) [2023-12-27 00:37:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 649830400. Throughput: 0: 9469.4, 1: 9845.2. Samples: 649836908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:37:26,062][104569] Avg episode reward: [(0, '9354.347'), (1, '9279.024')] [2023-12-27 00:37:26,285][105620] Updated weights for policy 1, policy_version 1269672 (0.0007) [2023-12-27 00:37:26,346][105620] Updated weights for policy 1, policy_version 1269682 (0.0009) [2023-12-27 00:37:26,405][105620] Updated weights for policy 1, policy_version 1269692 (0.0008) [2023-12-27 00:37:26,632][105692] Updated weights for policy 0, policy_version 1268373 (0.0008) [2023-12-27 00:37:26,688][105692] Updated weights for policy 0, policy_version 1268383 (0.0009) [2023-12-27 00:37:26,749][105692] Updated weights for policy 0, policy_version 1268393 (0.0008) [2023-12-27 00:37:27,108][105620] Updated weights for policy 1, policy_version 1269702 (0.0005) [2023-12-27 00:37:27,172][105620] Updated weights for policy 1, policy_version 1269712 (0.0005) [2023-12-27 00:37:27,230][105620] Updated weights for policy 1, policy_version 1269722 (0.0005) [2023-12-27 00:37:27,502][105692] Updated weights for policy 0, policy_version 1268403 (0.0007) [2023-12-27 00:37:27,555][105692] Updated weights for policy 0, policy_version 1268413 (0.0009) [2023-12-27 00:37:27,609][105692] Updated weights for policy 0, policy_version 1268424 (0.0009) [2023-12-27 00:37:27,737][105620] Updated weights for policy 1, policy_version 1269732 (0.0005) [2023-12-27 00:37:27,783][105620] Updated weights for policy 1, policy_version 1269742 (0.0005) [2023-12-27 00:37:27,829][105620] Updated weights for policy 1, policy_version 1269752 (0.0005) [2023-12-27 00:37:28,321][105692] Updated weights for policy 0, policy_version 1268434 (0.0010) [2023-12-27 00:37:28,389][105692] Updated weights for policy 0, policy_version 1268444 (0.0010) [2023-12-27 00:37:28,392][105620] Updated weights for policy 1, policy_version 1269762 (0.0005) [2023-12-27 00:37:28,447][105692] Updated weights for policy 0, policy_version 1268454 (0.0010) [2023-12-27 00:37:28,448][105620] Updated weights for policy 1, policy_version 1269772 (0.0007) [2023-12-27 00:37:28,503][105692] Updated weights for policy 0, policy_version 1268464 (0.0010) [2023-12-27 00:37:28,510][105620] Updated weights for policy 1, policy_version 1269782 (0.0007) [2023-12-27 00:37:28,568][105620] Updated weights for policy 1, policy_version 1269792 (0.0010) [2023-12-27 00:37:29,073][105692] Updated weights for policy 0, policy_version 1268474 (0.0008) [2023-12-27 00:37:29,130][105692] Updated weights for policy 0, policy_version 1268484 (0.0009) [2023-12-27 00:37:29,154][105620] Updated weights for policy 1, policy_version 1269802 (0.0005) [2023-12-27 00:37:29,178][105692] Updated weights for policy 0, policy_version 1268494 (0.0010) [2023-12-27 00:37:29,208][105620] Updated weights for policy 1, policy_version 1269812 (0.0006) [2023-12-27 00:37:29,268][105620] Updated weights for policy 1, policy_version 1269822 (0.0010) [2023-12-27 00:37:29,828][105692] Updated weights for policy 0, policy_version 1268504 (0.0006) [2023-12-27 00:37:29,896][105692] Updated weights for policy 0, policy_version 1268514 (0.0008) [2023-12-27 00:37:29,955][105692] Updated weights for policy 0, policy_version 1268524 (0.0009) [2023-12-27 00:37:29,959][105620] Updated weights for policy 1, policy_version 1269832 (0.0008) [2023-12-27 00:37:30,025][105620] Updated weights for policy 1, policy_version 1269842 (0.0009) [2023-12-27 00:37:30,090][105620] Updated weights for policy 1, policy_version 1269852 (0.0010) [2023-12-27 00:37:30,650][105692] Updated weights for policy 0, policy_version 1268534 (0.0007) [2023-12-27 00:37:30,703][105692] Updated weights for policy 0, policy_version 1268544 (0.0005) [2023-12-27 00:37:30,748][105692] Updated weights for policy 0, policy_version 1268554 (0.0005) [2023-12-27 00:37:30,814][105620] Updated weights for policy 1, policy_version 1269862 (0.0010) [2023-12-27 00:37:30,871][105620] Updated weights for policy 1, policy_version 1269872 (0.0010) [2023-12-27 00:37:30,929][105620] Updated weights for policy 1, policy_version 1269882 (0.0010) [2023-12-27 00:37:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 649936896. Throughput: 0: 9515.3, 1: 9975.9. Samples: 649900092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:37:31,063][104569] Avg episode reward: [(0, '9172.368'), (1, '9080.230')] [2023-12-27 00:37:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001268560_324804608.pth... [2023-12-27 00:37:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001269888_325132288.pth... [2023-12-27 00:37:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001267440_324517888.pth [2023-12-27 00:37:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001268704_324829184.pth [2023-12-27 00:37:31,411][105692] Updated weights for policy 0, policy_version 1268564 (0.0006) [2023-12-27 00:37:31,463][105692] Updated weights for policy 0, policy_version 1268574 (0.0005) [2023-12-27 00:37:31,521][105692] Updated weights for policy 0, policy_version 1268584 (0.0005) [2023-12-27 00:37:31,687][105620] Updated weights for policy 1, policy_version 1269892 (0.0008) [2023-12-27 00:37:31,759][105620] Updated weights for policy 1, policy_version 1269902 (0.0008) [2023-12-27 00:37:31,826][105620] Updated weights for policy 1, policy_version 1269912 (0.0008) [2023-12-27 00:37:32,145][105692] Updated weights for policy 0, policy_version 1268594 (0.0005) [2023-12-27 00:37:32,199][105692] Updated weights for policy 0, policy_version 1268604 (0.0009) [2023-12-27 00:37:32,252][105692] Updated weights for policy 0, policy_version 1268614 (0.0010) [2023-12-27 00:37:32,314][105692] Updated weights for policy 0, policy_version 1268624 (0.0009) [2023-12-27 00:37:32,378][105620] Updated weights for policy 1, policy_version 1269923 (0.0009) [2023-12-27 00:37:32,435][105620] Updated weights for policy 1, policy_version 1269933 (0.0010) [2023-12-27 00:37:32,467][105586] KL-divergence is very high: 163.4884 [2023-12-27 00:37:32,497][105620] Updated weights for policy 1, policy_version 1269943 (0.0010) [2023-12-27 00:37:32,518][105586] KL-divergence is very high: 172.6283 [2023-12-27 00:37:33,127][105620] Updated weights for policy 1, policy_version 1269953 (0.0010) [2023-12-27 00:37:33,140][105692] Updated weights for policy 0, policy_version 1268634 (0.0008) [2023-12-27 00:37:33,188][105620] Updated weights for policy 1, policy_version 1269963 (0.0010) [2023-12-27 00:37:33,191][105692] Updated weights for policy 0, policy_version 1268644 (0.0006) [2023-12-27 00:37:33,245][105692] Updated weights for policy 0, policy_version 1268654 (0.0006) [2023-12-27 00:37:33,247][105620] Updated weights for policy 1, policy_version 1269973 (0.0010) [2023-12-27 00:37:33,302][105620] Updated weights for policy 1, policy_version 1269983 (0.0011) [2023-12-27 00:37:33,864][105620] Updated weights for policy 1, policy_version 1269993 (0.0006) [2023-12-27 00:37:33,924][105620] Updated weights for policy 1, policy_version 1270003 (0.0007) [2023-12-27 00:37:33,971][105620] Updated weights for policy 1, policy_version 1270013 (0.0009) [2023-12-27 00:37:34,052][105692] Updated weights for policy 0, policy_version 1268664 (0.0009) [2023-12-27 00:37:34,100][105692] Updated weights for policy 0, policy_version 1268674 (0.0009) [2023-12-27 00:37:34,156][105692] Updated weights for policy 0, policy_version 1268684 (0.0007) [2023-12-27 00:37:34,650][105620] Updated weights for policy 1, policy_version 1270023 (0.0009) [2023-12-27 00:37:34,710][105620] Updated weights for policy 1, policy_version 1270033 (0.0011) [2023-12-27 00:37:34,769][105620] Updated weights for policy 1, policy_version 1270043 (0.0010) [2023-12-27 00:37:34,868][105692] Updated weights for policy 0, policy_version 1268694 (0.0008) [2023-12-27 00:37:34,925][105692] Updated weights for policy 0, policy_version 1268704 (0.0008) [2023-12-27 00:37:34,977][105692] Updated weights for policy 0, policy_version 1268714 (0.0008) [2023-12-27 00:37:35,525][105620] Updated weights for policy 1, policy_version 1270053 (0.0010) [2023-12-27 00:37:35,566][105692] Updated weights for policy 0, policy_version 1268724 (0.0007) [2023-12-27 00:37:35,587][105620] Updated weights for policy 1, policy_version 1270063 (0.0010) [2023-12-27 00:37:35,634][105692] Updated weights for policy 0, policy_version 1268734 (0.0005) [2023-12-27 00:37:35,655][105620] Updated weights for policy 1, policy_version 1270073 (0.0010) [2023-12-27 00:37:35,698][105692] Updated weights for policy 0, policy_version 1268744 (0.0006) [2023-12-27 00:37:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 650035200. Throughput: 0: 9556.8, 1: 9999.1. Samples: 650022632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:37:36,063][104569] Avg episode reward: [(0, '8913.680'), (1, '8918.403')] [2023-12-27 00:37:36,326][105692] Updated weights for policy 0, policy_version 1268754 (0.0006) [2023-12-27 00:37:36,374][105620] Updated weights for policy 1, policy_version 1270083 (0.0009) [2023-12-27 00:37:36,383][105692] Updated weights for policy 0, policy_version 1268764 (0.0008) [2023-12-27 00:37:36,436][105620] Updated weights for policy 1, policy_version 1270093 (0.0009) [2023-12-27 00:37:36,446][105692] Updated weights for policy 0, policy_version 1268774 (0.0007) [2023-12-27 00:37:36,494][105620] Updated weights for policy 1, policy_version 1270103 (0.0007) [2023-12-27 00:37:36,504][105692] Updated weights for policy 0, policy_version 1268784 (0.0005) [2023-12-27 00:37:37,156][105620] Updated weights for policy 1, policy_version 1270113 (0.0010) [2023-12-27 00:37:37,157][105692] Updated weights for policy 0, policy_version 1268794 (0.0006) [2023-12-27 00:37:37,217][105620] Updated weights for policy 1, policy_version 1270123 (0.0009) [2023-12-27 00:37:37,229][105692] Updated weights for policy 0, policy_version 1268804 (0.0006) [2023-12-27 00:37:37,279][105620] Updated weights for policy 1, policy_version 1270133 (0.0009) [2023-12-27 00:37:37,292][105692] Updated weights for policy 0, policy_version 1268814 (0.0005) [2023-12-27 00:37:37,339][105620] Updated weights for policy 1, policy_version 1270143 (0.0007) [2023-12-27 00:37:37,925][105692] Updated weights for policy 0, policy_version 1268824 (0.0008) [2023-12-27 00:37:37,979][105692] Updated weights for policy 0, policy_version 1268834 (0.0009) [2023-12-27 00:37:38,036][105692] Updated weights for policy 0, policy_version 1268844 (0.0008) [2023-12-27 00:37:38,106][105620] Updated weights for policy 1, policy_version 1270153 (0.0010) [2023-12-27 00:37:38,161][105620] Updated weights for policy 1, policy_version 1270163 (0.0010) [2023-12-27 00:37:38,209][105620] Updated weights for policy 1, policy_version 1270173 (0.0010) [2023-12-27 00:37:38,809][105692] Updated weights for policy 0, policy_version 1268854 (0.0008) [2023-12-27 00:37:38,869][105692] Updated weights for policy 0, policy_version 1268864 (0.0008) [2023-12-27 00:37:38,918][105692] Updated weights for policy 0, policy_version 1268874 (0.0008) [2023-12-27 00:37:38,970][105620] Updated weights for policy 1, policy_version 1270183 (0.0011) [2023-12-27 00:37:39,030][105620] Updated weights for policy 1, policy_version 1270193 (0.0011) [2023-12-27 00:37:39,093][105620] Updated weights for policy 1, policy_version 1270203 (0.0011) [2023-12-27 00:37:39,703][105692] Updated weights for policy 0, policy_version 1268884 (0.0008) [2023-12-27 00:37:39,771][105692] Updated weights for policy 0, policy_version 1268894 (0.0008) [2023-12-27 00:37:39,834][105692] Updated weights for policy 0, policy_version 1268904 (0.0007) [2023-12-27 00:37:39,862][105620] Updated weights for policy 1, policy_version 1270213 (0.0009) [2023-12-27 00:37:39,925][105620] Updated weights for policy 1, policy_version 1270223 (0.0009) [2023-12-27 00:37:39,983][105620] Updated weights for policy 1, policy_version 1270233 (0.0010) [2023-12-27 00:37:40,641][105692] Updated weights for policy 0, policy_version 1268914 (0.0008) [2023-12-27 00:37:40,700][105692] Updated weights for policy 0, policy_version 1268924 (0.0010) [2023-12-27 00:37:40,710][105620] Updated weights for policy 1, policy_version 1270243 (0.0009) [2023-12-27 00:37:40,759][105692] Updated weights for policy 0, policy_version 1268934 (0.0008) [2023-12-27 00:37:40,763][105620] Updated weights for policy 1, policy_version 1270253 (0.0006) [2023-12-27 00:37:40,806][105692] Updated weights for policy 0, policy_version 1268944 (0.0009) [2023-12-27 00:37:40,811][105620] Updated weights for policy 1, policy_version 1270263 (0.0006) [2023-12-27 00:37:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19438.7). Total num frames: 650133504. Throughput: 0: 9571.5, 1: 9923.4. Samples: 650139040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:37:41,062][104569] Avg episode reward: [(0, '8911.139'), (1, '9180.991')] [2023-12-27 00:37:41,443][105620] Updated weights for policy 1, policy_version 1270273 (0.0007) [2023-12-27 00:37:41,508][105620] Updated weights for policy 1, policy_version 1270283 (0.0006) [2023-12-27 00:37:41,570][105620] Updated weights for policy 1, policy_version 1270293 (0.0006) [2023-12-27 00:37:41,633][105620] Updated weights for policy 1, policy_version 1270303 (0.0010) [2023-12-27 00:37:41,650][105692] Updated weights for policy 0, policy_version 1268954 (0.0008) [2023-12-27 00:37:41,721][105692] Updated weights for policy 0, policy_version 1268964 (0.0008) [2023-12-27 00:37:41,788][105692] Updated weights for policy 0, policy_version 1268974 (0.0008) [2023-12-27 00:37:42,337][105620] Updated weights for policy 1, policy_version 1270313 (0.0012) [2023-12-27 00:37:42,405][105620] Updated weights for policy 1, policy_version 1270323 (0.0007) [2023-12-27 00:37:42,471][105620] Updated weights for policy 1, policy_version 1270333 (0.0008) [2023-12-27 00:37:42,505][105692] Updated weights for policy 0, policy_version 1268984 (0.0008) [2023-12-27 00:37:42,560][105692] Updated weights for policy 0, policy_version 1268994 (0.0008) [2023-12-27 00:37:42,622][105692] Updated weights for policy 0, policy_version 1269004 (0.0009) [2023-12-27 00:37:43,047][105620] Updated weights for policy 1, policy_version 1270343 (0.0007) [2023-12-27 00:37:43,100][105620] Updated weights for policy 1, policy_version 1270353 (0.0005) [2023-12-27 00:37:43,147][105620] Updated weights for policy 1, policy_version 1270363 (0.0006) [2023-12-27 00:37:43,489][105692] Updated weights for policy 0, policy_version 1269015 (0.0010) [2023-12-27 00:37:43,543][105692] Updated weights for policy 0, policy_version 1269025 (0.0009) [2023-12-27 00:37:43,601][105692] Updated weights for policy 0, policy_version 1269035 (0.0009) [2023-12-27 00:37:43,729][105620] Updated weights for policy 1, policy_version 1270373 (0.0007) [2023-12-27 00:37:43,793][105620] Updated weights for policy 1, policy_version 1270383 (0.0005) [2023-12-27 00:37:43,866][105620] Updated weights for policy 1, policy_version 1270393 (0.0007) [2023-12-27 00:37:44,279][105692] Updated weights for policy 0, policy_version 1269045 (0.0007) [2023-12-27 00:37:44,334][105692] Updated weights for policy 0, policy_version 1269055 (0.0010) [2023-12-27 00:37:44,393][105692] Updated weights for policy 0, policy_version 1269065 (0.0010) [2023-12-27 00:37:44,455][105620] Updated weights for policy 1, policy_version 1270403 (0.0008) [2023-12-27 00:37:44,502][105620] Updated weights for policy 1, policy_version 1270413 (0.0010) [2023-12-27 00:37:44,564][105620] Updated weights for policy 1, policy_version 1270423 (0.0010) [2023-12-27 00:37:45,061][105692] Updated weights for policy 0, policy_version 1269075 (0.0010) [2023-12-27 00:37:45,124][105692] Updated weights for policy 0, policy_version 1269085 (0.0011) [2023-12-27 00:37:45,183][105692] Updated weights for policy 0, policy_version 1269095 (0.0011) [2023-12-27 00:37:45,311][105620] Updated weights for policy 1, policy_version 1270433 (0.0008) [2023-12-27 00:37:45,367][105620] Updated weights for policy 1, policy_version 1270443 (0.0011) [2023-12-27 00:37:45,431][105620] Updated weights for policy 1, policy_version 1270453 (0.0010) [2023-12-27 00:37:45,496][105620] Updated weights for policy 1, policy_version 1270463 (0.0005) [2023-12-27 00:37:45,938][105692] Updated weights for policy 0, policy_version 1269105 (0.0011) [2023-12-27 00:37:45,991][105692] Updated weights for policy 0, policy_version 1269115 (0.0010) [2023-12-27 00:37:46,044][105692] Updated weights for policy 0, policy_version 1269125 (0.0010) [2023-12-27 00:37:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 650223616. Throughput: 0: 9519.7, 1: 10004.5. Samples: 650197612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:37:46,063][104569] Avg episode reward: [(0, '8988.169'), (1, '9266.619')] [2023-12-27 00:37:46,096][105692] Updated weights for policy 0, policy_version 1269135 (0.0008) [2023-12-27 00:37:46,099][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001269136_324952064.pth... [2023-12-27 00:37:46,103][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001268016_324665344.pth [2023-12-27 00:37:46,111][105620] Updated weights for policy 1, policy_version 1270473 (0.0007) [2023-12-27 00:37:46,166][105620] Updated weights for policy 1, policy_version 1270483 (0.0009) [2023-12-27 00:37:46,213][105620] Updated weights for policy 1, policy_version 1270493 (0.0009) [2023-12-27 00:37:46,225][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001270496_325287936.pth... [2023-12-27 00:37:46,230][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001269280_324976640.pth [2023-12-27 00:37:46,821][105620] Updated weights for policy 1, policy_version 1270503 (0.0009) [2023-12-27 00:37:46,871][105620] Updated weights for policy 1, policy_version 1270513 (0.0008) [2023-12-27 00:37:46,922][105692] Updated weights for policy 0, policy_version 1269145 (0.0007) [2023-12-27 00:37:46,929][105620] Updated weights for policy 1, policy_version 1270523 (0.0006) [2023-12-27 00:37:46,977][105692] Updated weights for policy 0, policy_version 1269155 (0.0009) [2023-12-27 00:37:47,024][105692] Updated weights for policy 0, policy_version 1269165 (0.0009) [2023-12-27 00:37:47,709][105620] Updated weights for policy 1, policy_version 1270533 (0.0008) [2023-12-27 00:37:47,724][105692] Updated weights for policy 0, policy_version 1269175 (0.0010) [2023-12-27 00:37:47,769][105620] Updated weights for policy 1, policy_version 1270543 (0.0007) [2023-12-27 00:37:47,783][105692] Updated weights for policy 0, policy_version 1269185 (0.0008) [2023-12-27 00:37:47,827][105620] Updated weights for policy 1, policy_version 1270553 (0.0005) [2023-12-27 00:37:47,837][105692] Updated weights for policy 0, policy_version 1269195 (0.0007) [2023-12-27 00:37:48,381][105620] Updated weights for policy 1, policy_version 1270563 (0.0008) [2023-12-27 00:37:48,439][105620] Updated weights for policy 1, policy_version 1270573 (0.0009) [2023-12-27 00:37:48,494][105620] Updated weights for policy 1, policy_version 1270583 (0.0009) [2023-12-27 00:37:48,598][105692] Updated weights for policy 0, policy_version 1269205 (0.0007) [2023-12-27 00:37:48,642][105585] KL-divergence is very high: 134.7559 [2023-12-27 00:37:48,649][105692] Updated weights for policy 0, policy_version 1269215 (0.0009) [2023-12-27 00:37:48,660][105585] KL-divergence is very high: 155.8330 [2023-12-27 00:37:48,690][105585] KL-divergence is very high: 270.3822 [2023-12-27 00:37:48,708][105692] Updated weights for policy 0, policy_version 1269225 (0.0009) [2023-12-27 00:37:48,708][105585] KL-divergence is very high: 233.1253 [2023-12-27 00:37:48,738][105585] KL-divergence is very high: 315.3122 [2023-12-27 00:37:49,138][105620] Updated weights for policy 1, policy_version 1270593 (0.0006) [2023-12-27 00:37:49,184][105620] Updated weights for policy 1, policy_version 1270603 (0.0009) [2023-12-27 00:37:49,236][105620] Updated weights for policy 1, policy_version 1270613 (0.0009) [2023-12-27 00:37:49,302][105620] Updated weights for policy 1, policy_version 1270623 (0.0006) [2023-12-27 00:37:49,566][105692] Updated weights for policy 0, policy_version 1269235 (0.0009) [2023-12-27 00:37:49,632][105692] Updated weights for policy 0, policy_version 1269245 (0.0008) [2023-12-27 00:37:49,695][105692] Updated weights for policy 0, policy_version 1269255 (0.0009) [2023-12-27 00:37:50,005][105620] Updated weights for policy 1, policy_version 1270633 (0.0008) [2023-12-27 00:37:50,069][105620] Updated weights for policy 1, policy_version 1270643 (0.0007) [2023-12-27 00:37:50,133][105620] Updated weights for policy 1, policy_version 1270653 (0.0006) [2023-12-27 00:37:50,471][105692] Updated weights for policy 0, policy_version 1269265 (0.0010) [2023-12-27 00:37:50,531][105692] Updated weights for policy 0, policy_version 1269275 (0.0007) [2023-12-27 00:37:50,597][105692] Updated weights for policy 0, policy_version 1269285 (0.0007) [2023-12-27 00:37:50,659][105692] Updated weights for policy 0, policy_version 1269295 (0.0006) [2023-12-27 00:37:50,797][105620] Updated weights for policy 1, policy_version 1270663 (0.0009) [2023-12-27 00:37:50,860][105620] Updated weights for policy 1, policy_version 1270673 (0.0011) [2023-12-27 00:37:50,911][105620] Updated weights for policy 1, policy_version 1270683 (0.0011) [2023-12-27 00:37:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 650330112. Throughput: 0: 9564.1, 1: 10072.8. Samples: 650317160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:37:51,063][104569] Avg episode reward: [(0, '8992.165'), (1, '9266.608')] [2023-12-27 00:37:51,352][105692] Updated weights for policy 0, policy_version 1269305 (0.0007) [2023-12-27 00:37:51,419][105692] Updated weights for policy 0, policy_version 1269315 (0.0007) [2023-12-27 00:37:51,475][105692] Updated weights for policy 0, policy_version 1269325 (0.0008) [2023-12-27 00:37:51,685][105620] Updated weights for policy 1, policy_version 1270693 (0.0011) [2023-12-27 00:37:51,754][105620] Updated weights for policy 1, policy_version 1270703 (0.0011) [2023-12-27 00:37:51,818][105620] Updated weights for policy 1, policy_version 1270713 (0.0012) [2023-12-27 00:37:52,145][105692] Updated weights for policy 0, policy_version 1269335 (0.0008) [2023-12-27 00:37:52,198][105692] Updated weights for policy 0, policy_version 1269345 (0.0008) [2023-12-27 00:37:52,256][105692] Updated weights for policy 0, policy_version 1269355 (0.0009) [2023-12-27 00:37:52,561][105620] Updated weights for policy 1, policy_version 1270724 (0.0010) [2023-12-27 00:37:52,624][105620] Updated weights for policy 1, policy_version 1270734 (0.0011) [2023-12-27 00:37:52,683][105620] Updated weights for policy 1, policy_version 1270744 (0.0011) [2023-12-27 00:37:52,971][105692] Updated weights for policy 0, policy_version 1269365 (0.0009) [2023-12-27 00:37:53,032][105692] Updated weights for policy 0, policy_version 1269375 (0.0006) [2023-12-27 00:37:53,086][105692] Updated weights for policy 0, policy_version 1269385 (0.0005) [2023-12-27 00:37:53,430][105620] Updated weights for policy 1, policy_version 1270754 (0.0010) [2023-12-27 00:37:53,495][105620] Updated weights for policy 1, policy_version 1270764 (0.0005) [2023-12-27 00:37:53,554][105620] Updated weights for policy 1, policy_version 1270774 (0.0009) [2023-12-27 00:37:53,609][105620] Updated weights for policy 1, policy_version 1270784 (0.0009) [2023-12-27 00:37:53,654][105692] Updated weights for policy 0, policy_version 1269395 (0.0007) [2023-12-27 00:37:53,713][105692] Updated weights for policy 0, policy_version 1269405 (0.0010) [2023-12-27 00:37:53,777][105692] Updated weights for policy 0, policy_version 1269415 (0.0010) [2023-12-27 00:37:54,277][105620] Updated weights for policy 1, policy_version 1270794 (0.0010) [2023-12-27 00:37:54,329][105620] Updated weights for policy 1, policy_version 1270804 (0.0009) [2023-12-27 00:37:54,390][105620] Updated weights for policy 1, policy_version 1270814 (0.0009) [2023-12-27 00:37:54,409][105692] Updated weights for policy 0, policy_version 1269425 (0.0010) [2023-12-27 00:37:54,469][105692] Updated weights for policy 0, policy_version 1269435 (0.0008) [2023-12-27 00:37:54,525][105692] Updated weights for policy 0, policy_version 1269445 (0.0005) [2023-12-27 00:37:54,587][105692] Updated weights for policy 0, policy_version 1269455 (0.0005) [2023-12-27 00:37:55,068][105620] Updated weights for policy 1, policy_version 1270824 (0.0009) [2023-12-27 00:37:55,119][105620] Updated weights for policy 1, policy_version 1270834 (0.0009) [2023-12-27 00:37:55,165][105620] Updated weights for policy 1, policy_version 1270844 (0.0005) [2023-12-27 00:37:55,244][105692] Updated weights for policy 0, policy_version 1269465 (0.0005) [2023-12-27 00:37:55,293][105692] Updated weights for policy 0, policy_version 1269475 (0.0005) [2023-12-27 00:37:55,350][105692] Updated weights for policy 0, policy_version 1269485 (0.0005) [2023-12-27 00:37:55,727][105620] Updated weights for policy 1, policy_version 1270854 (0.0007) [2023-12-27 00:37:55,785][105620] Updated weights for policy 1, policy_version 1270864 (0.0010) [2023-12-27 00:37:55,833][105620] Updated weights for policy 1, policy_version 1270874 (0.0010) [2023-12-27 00:37:56,048][105692] Updated weights for policy 0, policy_version 1269495 (0.0006) [2023-12-27 00:37:56,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.7, 300 sec: 19466.4). Total num frames: 650428416. Throughput: 0: 9592.6, 1: 10137.6. Samples: 650437984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:37:56,063][104569] Avg episode reward: [(0, '9175.989'), (1, '9354.813')] [2023-12-27 00:37:56,108][105692] Updated weights for policy 0, policy_version 1269505 (0.0006) [2023-12-27 00:37:56,172][105692] Updated weights for policy 0, policy_version 1269515 (0.0010) [2023-12-27 00:37:56,563][105620] Updated weights for policy 1, policy_version 1270884 (0.0010) [2023-12-27 00:37:56,624][105620] Updated weights for policy 1, policy_version 1270894 (0.0010) [2023-12-27 00:37:56,693][105620] Updated weights for policy 1, policy_version 1270904 (0.0011) [2023-12-27 00:37:56,825][105692] Updated weights for policy 0, policy_version 1269525 (0.0008) [2023-12-27 00:37:56,870][105692] Updated weights for policy 0, policy_version 1269535 (0.0005) [2023-12-27 00:37:56,920][105692] Updated weights for policy 0, policy_version 1269545 (0.0005) [2023-12-27 00:37:57,344][105620] Updated weights for policy 1, policy_version 1270914 (0.0009) [2023-12-27 00:37:57,398][105620] Updated weights for policy 1, policy_version 1270924 (0.0006) [2023-12-27 00:37:57,458][105620] Updated weights for policy 1, policy_version 1270934 (0.0005) [2023-12-27 00:37:57,517][105620] Updated weights for policy 1, policy_version 1270944 (0.0005) [2023-12-27 00:37:57,569][105692] Updated weights for policy 0, policy_version 1269555 (0.0007) [2023-12-27 00:37:57,631][105692] Updated weights for policy 0, policy_version 1269565 (0.0010) [2023-12-27 00:37:57,681][105692] Updated weights for policy 0, policy_version 1269575 (0.0009) [2023-12-27 00:37:58,205][105620] Updated weights for policy 1, policy_version 1270954 (0.0010) [2023-12-27 00:37:58,267][105692] Updated weights for policy 0, policy_version 1269585 (0.0007) [2023-12-27 00:37:58,271][105620] Updated weights for policy 1, policy_version 1270964 (0.0011) [2023-12-27 00:37:58,326][105692] Updated weights for policy 0, policy_version 1269595 (0.0007) [2023-12-27 00:37:58,333][105620] Updated weights for policy 1, policy_version 1270974 (0.0010) [2023-12-27 00:37:58,392][105692] Updated weights for policy 0, policy_version 1269605 (0.0008) [2023-12-27 00:37:58,461][105692] Updated weights for policy 0, policy_version 1269615 (0.0009) [2023-12-27 00:37:59,113][105620] Updated weights for policy 1, policy_version 1270984 (0.0010) [2023-12-27 00:37:59,171][105620] Updated weights for policy 1, policy_version 1270994 (0.0010) [2023-12-27 00:37:59,174][105692] Updated weights for policy 0, policy_version 1269625 (0.0007) [2023-12-27 00:37:59,235][105620] Updated weights for policy 1, policy_version 1271004 (0.0010) [2023-12-27 00:37:59,240][105692] Updated weights for policy 0, policy_version 1269635 (0.0007) [2023-12-27 00:37:59,308][105692] Updated weights for policy 0, policy_version 1269645 (0.0008) [2023-12-27 00:37:59,951][105692] Updated weights for policy 0, policy_version 1269655 (0.0008) [2023-12-27 00:38:00,008][105692] Updated weights for policy 0, policy_version 1269665 (0.0006) [2023-12-27 00:38:00,035][105620] Updated weights for policy 1, policy_version 1271014 (0.0010) [2023-12-27 00:38:00,068][105692] Updated weights for policy 0, policy_version 1269675 (0.0009) [2023-12-27 00:38:00,094][105620] Updated weights for policy 1, policy_version 1271024 (0.0006) [2023-12-27 00:38:00,160][105620] Updated weights for policy 1, policy_version 1271034 (0.0008) [2023-12-27 00:38:00,732][105692] Updated weights for policy 0, policy_version 1269685 (0.0009) [2023-12-27 00:38:00,792][105692] Updated weights for policy 0, policy_version 1269695 (0.0009) [2023-12-27 00:38:00,852][105692] Updated weights for policy 0, policy_version 1269705 (0.0009) [2023-12-27 00:38:00,911][105620] Updated weights for policy 1, policy_version 1271044 (0.0008) [2023-12-27 00:38:00,960][105620] Updated weights for policy 1, policy_version 1271054 (0.0008) [2023-12-27 00:38:01,011][105620] Updated weights for policy 1, policy_version 1271064 (0.0008) [2023-12-27 00:38:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.4, 300 sec: 19521.9). Total num frames: 650534912. Throughput: 0: 9691.1, 1: 10119.4. Samples: 650498976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:38:01,063][104569] Avg episode reward: [(0, '9357.715'), (1, '9271.396')] [2023-12-27 00:38:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001269712_325099520.pth... [2023-12-27 00:38:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001271072_325435392.pth... [2023-12-27 00:38:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001268560_324804608.pth [2023-12-27 00:38:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001269888_325132288.pth [2023-12-27 00:38:01,545][105692] Updated weights for policy 0, policy_version 1269715 (0.0009) [2023-12-27 00:38:01,605][105692] Updated weights for policy 0, policy_version 1269725 (0.0008) [2023-12-27 00:38:01,670][105692] Updated weights for policy 0, policy_version 1269735 (0.0009) [2023-12-27 00:38:01,818][105620] Updated weights for policy 1, policy_version 1271074 (0.0008) [2023-12-27 00:38:01,868][105620] Updated weights for policy 1, policy_version 1271084 (0.0008) [2023-12-27 00:38:01,931][105620] Updated weights for policy 1, policy_version 1271094 (0.0009) [2023-12-27 00:38:01,990][105620] Updated weights for policy 1, policy_version 1271104 (0.0010) [2023-12-27 00:38:02,384][105692] Updated weights for policy 0, policy_version 1269745 (0.0009) [2023-12-27 00:38:02,430][105692] Updated weights for policy 0, policy_version 1269755 (0.0009) [2023-12-27 00:38:02,486][105692] Updated weights for policy 0, policy_version 1269765 (0.0009) [2023-12-27 00:38:02,546][105692] Updated weights for policy 0, policy_version 1269775 (0.0008) [2023-12-27 00:38:02,727][105620] Updated weights for policy 1, policy_version 1271114 (0.0008) [2023-12-27 00:38:02,773][105620] Updated weights for policy 1, policy_version 1271124 (0.0006) [2023-12-27 00:38:02,830][105620] Updated weights for policy 1, policy_version 1271134 (0.0009) [2023-12-27 00:38:03,223][105692] Updated weights for policy 0, policy_version 1269785 (0.0010) [2023-12-27 00:38:03,280][105692] Updated weights for policy 0, policy_version 1269795 (0.0010) [2023-12-27 00:38:03,337][105692] Updated weights for policy 0, policy_version 1269805 (0.0010) [2023-12-27 00:38:03,567][105620] Updated weights for policy 1, policy_version 1271144 (0.0008) [2023-12-27 00:38:03,611][105620] Updated weights for policy 1, policy_version 1271154 (0.0008) [2023-12-27 00:38:03,662][105620] Updated weights for policy 1, policy_version 1271164 (0.0008) [2023-12-27 00:38:04,052][105692] Updated weights for policy 0, policy_version 1269815 (0.0010) [2023-12-27 00:38:04,111][105692] Updated weights for policy 0, policy_version 1269825 (0.0010) [2023-12-27 00:38:04,167][105692] Updated weights for policy 0, policy_version 1269835 (0.0008) [2023-12-27 00:38:04,434][105620] Updated weights for policy 1, policy_version 1271174 (0.0008) [2023-12-27 00:38:04,488][105620] Updated weights for policy 1, policy_version 1271184 (0.0007) [2023-12-27 00:38:04,544][105620] Updated weights for policy 1, policy_version 1271194 (0.0008) [2023-12-27 00:38:04,899][105692] Updated weights for policy 0, policy_version 1269845 (0.0008) [2023-12-27 00:38:04,952][105692] Updated weights for policy 0, policy_version 1269855 (0.0005) [2023-12-27 00:38:05,015][105692] Updated weights for policy 0, policy_version 1269865 (0.0005) [2023-12-27 00:38:05,383][105620] Updated weights for policy 1, policy_version 1271204 (0.0008) [2023-12-27 00:38:05,441][105620] Updated weights for policy 1, policy_version 1271214 (0.0009) [2023-12-27 00:38:05,497][105620] Updated weights for policy 1, policy_version 1271224 (0.0009) [2023-12-27 00:38:05,519][105692] Updated weights for policy 0, policy_version 1269875 (0.0007) [2023-12-27 00:38:05,571][105692] Updated weights for policy 0, policy_version 1269885 (0.0010) [2023-12-27 00:38:05,619][105692] Updated weights for policy 0, policy_version 1269895 (0.0010) [2023-12-27 00:38:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.7, 300 sec: 19494.2). Total num frames: 650625024. Throughput: 0: 9796.4, 1: 10048.7. Samples: 650613556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:38:06,063][104569] Avg episode reward: [(0, '9267.471'), (1, '9271.079')] [2023-12-27 00:38:06,246][105620] Updated weights for policy 1, policy_version 1271234 (0.0009) [2023-12-27 00:38:06,309][105620] Updated weights for policy 1, policy_version 1271244 (0.0009) [2023-12-27 00:38:06,349][105692] Updated weights for policy 0, policy_version 1269905 (0.0010) [2023-12-27 00:38:06,372][105620] Updated weights for policy 1, policy_version 1271254 (0.0007) [2023-12-27 00:38:06,416][105692] Updated weights for policy 0, policy_version 1269915 (0.0009) [2023-12-27 00:38:06,433][105620] Updated weights for policy 1, policy_version 1271264 (0.0007) [2023-12-27 00:38:06,486][105692] Updated weights for policy 0, policy_version 1269925 (0.0009) [2023-12-27 00:38:06,549][105692] Updated weights for policy 0, policy_version 1269935 (0.0010) [2023-12-27 00:38:07,091][105620] Updated weights for policy 1, policy_version 1271274 (0.0005) [2023-12-27 00:38:07,158][105620] Updated weights for policy 1, policy_version 1271284 (0.0006) [2023-12-27 00:38:07,218][105692] Updated weights for policy 0, policy_version 1269945 (0.0006) [2023-12-27 00:38:07,224][105620] Updated weights for policy 1, policy_version 1271294 (0.0006) [2023-12-27 00:38:07,285][105692] Updated weights for policy 0, policy_version 1269955 (0.0005) [2023-12-27 00:38:07,346][105692] Updated weights for policy 0, policy_version 1269965 (0.0008) [2023-12-27 00:38:07,793][105620] Updated weights for policy 1, policy_version 1271304 (0.0008) [2023-12-27 00:38:07,848][105620] Updated weights for policy 1, policy_version 1271314 (0.0010) [2023-12-27 00:38:07,911][105620] Updated weights for policy 1, policy_version 1271324 (0.0009) [2023-12-27 00:38:08,012][105692] Updated weights for policy 0, policy_version 1269975 (0.0007) [2023-12-27 00:38:08,065][105692] Updated weights for policy 0, policy_version 1269985 (0.0006) [2023-12-27 00:38:08,125][105692] Updated weights for policy 0, policy_version 1269995 (0.0006) [2023-12-27 00:38:08,737][105620] Updated weights for policy 1, policy_version 1271334 (0.0009) [2023-12-27 00:38:08,796][105620] Updated weights for policy 1, policy_version 1271344 (0.0010) [2023-12-27 00:38:08,826][105692] Updated weights for policy 0, policy_version 1270005 (0.0005) [2023-12-27 00:38:08,853][105620] Updated weights for policy 1, policy_version 1271354 (0.0011) [2023-12-27 00:38:08,879][105692] Updated weights for policy 0, policy_version 1270015 (0.0005) [2023-12-27 00:38:08,938][105692] Updated weights for policy 0, policy_version 1270025 (0.0008) [2023-12-27 00:38:09,533][105620] Updated weights for policy 1, policy_version 1271364 (0.0009) [2023-12-27 00:38:09,591][105620] Updated weights for policy 1, policy_version 1271374 (0.0006) [2023-12-27 00:38:09,657][105620] Updated weights for policy 1, policy_version 1271384 (0.0007) [2023-12-27 00:38:09,764][105692] Updated weights for policy 0, policy_version 1270035 (0.0009) [2023-12-27 00:38:09,798][105585] KL-divergence is very high: 142.5314 [2023-12-27 00:38:09,822][105692] Updated weights for policy 0, policy_version 1270045 (0.0010) [2023-12-27 00:38:09,848][105585] KL-divergence is very high: 256.3210 [2023-12-27 00:38:09,890][105692] Updated weights for policy 0, policy_version 1270055 (0.0009) [2023-12-27 00:38:09,904][105585] KL-divergence is very high: 281.2496 [2023-12-27 00:38:10,317][105620] Updated weights for policy 1, policy_version 1271394 (0.0010) [2023-12-27 00:38:10,381][105620] Updated weights for policy 1, policy_version 1271404 (0.0011) [2023-12-27 00:38:10,435][105620] Updated weights for policy 1, policy_version 1271414 (0.0011) [2023-12-27 00:38:10,486][105620] Updated weights for policy 1, policy_version 1271424 (0.0007) [2023-12-27 00:38:10,781][105692] Updated weights for policy 0, policy_version 1270065 (0.0009) [2023-12-27 00:38:10,828][105692] Updated weights for policy 0, policy_version 1270075 (0.0009) [2023-12-27 00:38:10,874][105692] Updated weights for policy 0, policy_version 1270085 (0.0008) [2023-12-27 00:38:10,922][105692] Updated weights for policy 0, policy_version 1270095 (0.0008) [2023-12-27 00:38:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 650723328. Throughput: 0: 9836.8, 1: 10045.1. Samples: 650731592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:38:11,062][104569] Avg episode reward: [(0, '9083.865'), (1, '9270.943')] [2023-12-27 00:38:11,211][105620] Updated weights for policy 1, policy_version 1271434 (0.0008) [2023-12-27 00:38:11,275][105620] Updated weights for policy 1, policy_version 1271444 (0.0008) [2023-12-27 00:38:11,336][105620] Updated weights for policy 1, policy_version 1271454 (0.0009) [2023-12-27 00:38:11,787][105692] Updated weights for policy 0, policy_version 1270105 (0.0010) [2023-12-27 00:38:11,843][105692] Updated weights for policy 0, policy_version 1270115 (0.0009) [2023-12-27 00:38:11,890][105692] Updated weights for policy 0, policy_version 1270125 (0.0008) [2023-12-27 00:38:12,098][105620] Updated weights for policy 1, policy_version 1271464 (0.0009) [2023-12-27 00:38:12,161][105620] Updated weights for policy 1, policy_version 1271474 (0.0006) [2023-12-27 00:38:12,216][105620] Updated weights for policy 1, policy_version 1271484 (0.0006) [2023-12-27 00:38:12,732][105692] Updated weights for policy 0, policy_version 1270135 (0.0010) [2023-12-27 00:38:12,781][105692] Updated weights for policy 0, policy_version 1270145 (0.0011) [2023-12-27 00:38:12,841][105692] Updated weights for policy 0, policy_version 1270155 (0.0011) [2023-12-27 00:38:12,844][105620] Updated weights for policy 1, policy_version 1271494 (0.0006) [2023-12-27 00:38:12,898][105620] Updated weights for policy 1, policy_version 1271504 (0.0007) [2023-12-27 00:38:12,953][105620] Updated weights for policy 1, policy_version 1271514 (0.0008) [2023-12-27 00:38:13,561][105692] Updated weights for policy 0, policy_version 1270165 (0.0008) [2023-12-27 00:38:13,618][105692] Updated weights for policy 0, policy_version 1270175 (0.0005) [2023-12-27 00:38:13,620][105620] Updated weights for policy 1, policy_version 1271524 (0.0008) [2023-12-27 00:38:13,666][105692] Updated weights for policy 0, policy_version 1270185 (0.0005) [2023-12-27 00:38:13,676][105620] Updated weights for policy 1, policy_version 1271534 (0.0009) [2023-12-27 00:38:13,730][105620] Updated weights for policy 1, policy_version 1271544 (0.0009) [2023-12-27 00:38:14,255][105692] Updated weights for policy 0, policy_version 1270195 (0.0005) [2023-12-27 00:38:14,274][105620] Updated weights for policy 1, policy_version 1271554 (0.0006) [2023-12-27 00:38:14,322][105692] Updated weights for policy 0, policy_version 1270205 (0.0005) [2023-12-27 00:38:14,330][105620] Updated weights for policy 1, policy_version 1271564 (0.0005) [2023-12-27 00:38:14,387][105620] Updated weights for policy 1, policy_version 1271574 (0.0005) [2023-12-27 00:38:14,388][105692] Updated weights for policy 0, policy_version 1270215 (0.0005) [2023-12-27 00:38:14,457][105620] Updated weights for policy 1, policy_version 1271584 (0.0005) [2023-12-27 00:38:14,949][105692] Updated weights for policy 0, policy_version 1270225 (0.0006) [2023-12-27 00:38:14,986][105620] Updated weights for policy 1, policy_version 1271594 (0.0005) [2023-12-27 00:38:15,005][105692] Updated weights for policy 0, policy_version 1270235 (0.0010) [2023-12-27 00:38:15,050][105620] Updated weights for policy 1, policy_version 1271604 (0.0008) [2023-12-27 00:38:15,066][105692] Updated weights for policy 0, policy_version 1270245 (0.0011) [2023-12-27 00:38:15,115][105620] Updated weights for policy 1, policy_version 1271614 (0.0006) [2023-12-27 00:38:15,121][105692] Updated weights for policy 0, policy_version 1270255 (0.0010) [2023-12-27 00:38:15,797][105620] Updated weights for policy 1, policy_version 1271624 (0.0005) [2023-12-27 00:38:15,853][105620] Updated weights for policy 1, policy_version 1271634 (0.0006) [2023-12-27 00:38:15,882][105692] Updated weights for policy 0, policy_version 1270265 (0.0011) [2023-12-27 00:38:15,912][105620] Updated weights for policy 1, policy_version 1271644 (0.0008) [2023-12-27 00:38:15,939][105692] Updated weights for policy 0, policy_version 1270275 (0.0010) [2023-12-27 00:38:15,990][105692] Updated weights for policy 0, policy_version 1270285 (0.0010) [2023-12-27 00:38:16,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19933.8, 300 sec: 19549.7). Total num frames: 650829824. Throughput: 0: 9787.1, 1: 9963.1. Samples: 650788852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:38:16,063][104569] Avg episode reward: [(0, '9174.553'), (1, '9094.753')] [2023-12-27 00:38:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001270288_325246976.pth... [2023-12-27 00:38:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001271648_325582848.pth... [2023-12-27 00:38:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001269136_324952064.pth [2023-12-27 00:38:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001270496_325287936.pth [2023-12-27 00:38:16,533][105620] Updated weights for policy 1, policy_version 1271654 (0.0009) [2023-12-27 00:38:16,581][105620] Updated weights for policy 1, policy_version 1271664 (0.0008) [2023-12-27 00:38:16,626][105620] Updated weights for policy 1, policy_version 1271674 (0.0008) [2023-12-27 00:38:16,764][105692] Updated weights for policy 0, policy_version 1270295 (0.0009) [2023-12-27 00:38:16,822][105692] Updated weights for policy 0, policy_version 1270305 (0.0009) [2023-12-27 00:38:16,874][105692] Updated weights for policy 0, policy_version 1270315 (0.0009) [2023-12-27 00:38:17,335][105620] Updated weights for policy 1, policy_version 1271684 (0.0007) [2023-12-27 00:38:17,398][105620] Updated weights for policy 1, policy_version 1271694 (0.0006) [2023-12-27 00:38:17,464][105620] Updated weights for policy 1, policy_version 1271704 (0.0007) [2023-12-27 00:38:17,574][105692] Updated weights for policy 0, policy_version 1270325 (0.0007) [2023-12-27 00:38:17,633][105692] Updated weights for policy 0, policy_version 1270335 (0.0005) [2023-12-27 00:38:17,699][105692] Updated weights for policy 0, policy_version 1270345 (0.0008) [2023-12-27 00:38:18,221][105620] Updated weights for policy 1, policy_version 1271714 (0.0008) [2023-12-27 00:38:18,244][105692] Updated weights for policy 0, policy_version 1270355 (0.0009) [2023-12-27 00:38:18,285][105620] Updated weights for policy 1, policy_version 1271724 (0.0008) [2023-12-27 00:38:18,298][105692] Updated weights for policy 0, policy_version 1270365 (0.0005) [2023-12-27 00:38:18,346][105620] Updated weights for policy 1, policy_version 1271734 (0.0008) [2023-12-27 00:38:18,356][105692] Updated weights for policy 0, policy_version 1270375 (0.0009) [2023-12-27 00:38:18,401][105620] Updated weights for policy 1, policy_version 1271744 (0.0007) [2023-12-27 00:38:18,946][105692] Updated weights for policy 0, policy_version 1270385 (0.0010) [2023-12-27 00:38:19,008][105692] Updated weights for policy 0, policy_version 1270395 (0.0011) [2023-12-27 00:38:19,067][105692] Updated weights for policy 0, policy_version 1270405 (0.0011) [2023-12-27 00:38:19,130][105692] Updated weights for policy 0, policy_version 1270415 (0.0011) [2023-12-27 00:38:19,212][105620] Updated weights for policy 1, policy_version 1271754 (0.0008) [2023-12-27 00:38:19,284][105620] Updated weights for policy 1, policy_version 1271764 (0.0007) [2023-12-27 00:38:19,354][105620] Updated weights for policy 1, policy_version 1271774 (0.0007) [2023-12-27 00:38:19,913][105692] Updated weights for policy 0, policy_version 1270425 (0.0010) [2023-12-27 00:38:19,980][105692] Updated weights for policy 0, policy_version 1270435 (0.0010) [2023-12-27 00:38:20,039][105620] Updated weights for policy 1, policy_version 1271784 (0.0007) [2023-12-27 00:38:20,050][105692] Updated weights for policy 0, policy_version 1270445 (0.0010) [2023-12-27 00:38:20,102][105620] Updated weights for policy 1, policy_version 1271794 (0.0008) [2023-12-27 00:38:20,165][105620] Updated weights for policy 1, policy_version 1271804 (0.0008) [2023-12-27 00:38:20,794][105692] Updated weights for policy 0, policy_version 1270455 (0.0011) [2023-12-27 00:38:20,859][105692] Updated weights for policy 0, policy_version 1270465 (0.0011) [2023-12-27 00:38:20,928][105692] Updated weights for policy 0, policy_version 1270475 (0.0011) [2023-12-27 00:38:20,954][105620] Updated weights for policy 1, policy_version 1271814 (0.0009) [2023-12-27 00:38:21,018][105620] Updated weights for policy 1, policy_version 1271824 (0.0008) [2023-12-27 00:38:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19522.0). Total num frames: 650919936. Throughput: 0: 9843.3, 1: 9935.0. Samples: 650912652. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:38:21,062][104569] Avg episode reward: [(0, '9178.202'), (1, '9086.382')] [2023-12-27 00:38:21,083][105620] Updated weights for policy 1, policy_version 1271834 (0.0007) [2023-12-27 00:38:21,740][105692] Updated weights for policy 0, policy_version 1270485 (0.0010) [2023-12-27 00:38:21,802][105692] Updated weights for policy 0, policy_version 1270495 (0.0011) [2023-12-27 00:38:21,816][105620] Updated weights for policy 1, policy_version 1271844 (0.0007) [2023-12-27 00:38:21,858][105692] Updated weights for policy 0, policy_version 1270505 (0.0011) [2023-12-27 00:38:21,869][105620] Updated weights for policy 1, policy_version 1271854 (0.0006) [2023-12-27 00:38:21,929][105620] Updated weights for policy 1, policy_version 1271864 (0.0006) [2023-12-27 00:38:22,565][105692] Updated weights for policy 0, policy_version 1270515 (0.0010) [2023-12-27 00:38:22,621][105692] Updated weights for policy 0, policy_version 1270525 (0.0009) [2023-12-27 00:38:22,674][105692] Updated weights for policy 0, policy_version 1270535 (0.0008) [2023-12-27 00:38:22,744][105620] Updated weights for policy 1, policy_version 1271874 (0.0008) [2023-12-27 00:38:22,808][105620] Updated weights for policy 1, policy_version 1271884 (0.0009) [2023-12-27 00:38:22,868][105620] Updated weights for policy 1, policy_version 1271894 (0.0009) [2023-12-27 00:38:22,936][105620] Updated weights for policy 1, policy_version 1271904 (0.0009) [2023-12-27 00:38:23,309][105692] Updated weights for policy 0, policy_version 1270545 (0.0008) [2023-12-27 00:38:23,360][105692] Updated weights for policy 0, policy_version 1270555 (0.0005) [2023-12-27 00:38:23,424][105692] Updated weights for policy 0, policy_version 1270565 (0.0005) [2023-12-27 00:38:23,470][105692] Updated weights for policy 0, policy_version 1270575 (0.0005) [2023-12-27 00:38:23,820][105620] Updated weights for policy 1, policy_version 1271914 (0.0006) [2023-12-27 00:38:23,881][105620] Updated weights for policy 1, policy_version 1271924 (0.0006) [2023-12-27 00:38:23,937][105620] Updated weights for policy 1, policy_version 1271935 (0.0009) [2023-12-27 00:38:24,008][105692] Updated weights for policy 0, policy_version 1270585 (0.0007) [2023-12-27 00:38:24,066][105692] Updated weights for policy 0, policy_version 1270595 (0.0005) [2023-12-27 00:38:24,123][105692] Updated weights for policy 0, policy_version 1270605 (0.0006) [2023-12-27 00:38:24,653][105620] Updated weights for policy 1, policy_version 1271945 (0.0008) [2023-12-27 00:38:24,708][105620] Updated weights for policy 1, policy_version 1271955 (0.0008) [2023-12-27 00:38:24,760][105620] Updated weights for policy 1, policy_version 1271965 (0.0008) [2023-12-27 00:38:24,822][105692] Updated weights for policy 0, policy_version 1270615 (0.0009) [2023-12-27 00:38:24,880][105692] Updated weights for policy 0, policy_version 1270625 (0.0006) [2023-12-27 00:38:24,939][105692] Updated weights for policy 0, policy_version 1270635 (0.0007) [2023-12-27 00:38:25,528][105620] Updated weights for policy 1, policy_version 1271975 (0.0008) [2023-12-27 00:38:25,587][105620] Updated weights for policy 1, policy_version 1271985 (0.0007) [2023-12-27 00:38:25,647][105620] Updated weights for policy 1, policy_version 1271995 (0.0007) [2023-12-27 00:38:25,649][105692] Updated weights for policy 0, policy_version 1270645 (0.0010) [2023-12-27 00:38:25,710][105692] Updated weights for policy 0, policy_version 1270655 (0.0010) [2023-12-27 00:38:25,775][105692] Updated weights for policy 0, policy_version 1270665 (0.0011) [2023-12-27 00:38:26,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19797.4, 300 sec: 19522.0). Total num frames: 651018240. Throughput: 0: 9842.8, 1: 9855.8. Samples: 651025476. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:38:26,062][104569] Avg episode reward: [(0, '9089.346'), (1, '9262.516')] [2023-12-27 00:38:26,405][105620] Updated weights for policy 1, policy_version 1272005 (0.0007) [2023-12-27 00:38:26,466][105620] Updated weights for policy 1, policy_version 1272015 (0.0007) [2023-12-27 00:38:26,518][105692] Updated weights for policy 0, policy_version 1270675 (0.0011) [2023-12-27 00:38:26,528][105620] Updated weights for policy 1, policy_version 1272025 (0.0008) [2023-12-27 00:38:26,580][105692] Updated weights for policy 0, policy_version 1270685 (0.0011) [2023-12-27 00:38:26,644][105692] Updated weights for policy 0, policy_version 1270695 (0.0010) [2023-12-27 00:38:27,250][105620] Updated weights for policy 1, policy_version 1272035 (0.0007) [2023-12-27 00:38:27,308][105620] Updated weights for policy 1, policy_version 1272045 (0.0010) [2023-12-27 00:38:27,357][105692] Updated weights for policy 0, policy_version 1270705 (0.0011) [2023-12-27 00:38:27,367][105620] Updated weights for policy 1, policy_version 1272055 (0.0008) [2023-12-27 00:38:27,414][105692] Updated weights for policy 0, policy_version 1270715 (0.0010) [2023-12-27 00:38:27,461][105692] Updated weights for policy 0, policy_version 1270725 (0.0010) [2023-12-27 00:38:27,508][105692] Updated weights for policy 0, policy_version 1270735 (0.0010) [2023-12-27 00:38:28,010][105620] Updated weights for policy 1, policy_version 1272065 (0.0010) [2023-12-27 00:38:28,072][105620] Updated weights for policy 1, policy_version 1272075 (0.0007) [2023-12-27 00:38:28,132][105620] Updated weights for policy 1, policy_version 1272085 (0.0008) [2023-12-27 00:38:28,187][105620] Updated weights for policy 1, policy_version 1272095 (0.0010) [2023-12-27 00:38:28,279][105692] Updated weights for policy 0, policy_version 1270745 (0.0010) [2023-12-27 00:38:28,344][105692] Updated weights for policy 0, policy_version 1270755 (0.0010) [2023-12-27 00:38:28,398][105692] Updated weights for policy 0, policy_version 1270765 (0.0011) [2023-12-27 00:38:28,863][105620] Updated weights for policy 1, policy_version 1272105 (0.0010) [2023-12-27 00:38:28,918][105620] Updated weights for policy 1, policy_version 1272115 (0.0010) [2023-12-27 00:38:28,976][105620] Updated weights for policy 1, policy_version 1272125 (0.0010) [2023-12-27 00:38:29,052][105692] Updated weights for policy 0, policy_version 1270775 (0.0010) [2023-12-27 00:38:29,097][105692] Updated weights for policy 0, policy_version 1270785 (0.0010) [2023-12-27 00:38:29,148][105692] Updated weights for policy 0, policy_version 1270795 (0.0009) [2023-12-27 00:38:29,690][105620] Updated weights for policy 1, policy_version 1272135 (0.0009) [2023-12-27 00:38:29,746][105620] Updated weights for policy 1, policy_version 1272145 (0.0009) [2023-12-27 00:38:29,805][105620] Updated weights for policy 1, policy_version 1272155 (0.0008) [2023-12-27 00:38:29,853][105692] Updated weights for policy 0, policy_version 1270805 (0.0007) [2023-12-27 00:38:29,915][105692] Updated weights for policy 0, policy_version 1270815 (0.0009) [2023-12-27 00:38:29,967][105692] Updated weights for policy 0, policy_version 1270825 (0.0009) [2023-12-27 00:38:30,549][105692] Updated weights for policy 0, policy_version 1270835 (0.0009) [2023-12-27 00:38:30,599][105692] Updated weights for policy 0, policy_version 1270845 (0.0008) [2023-12-27 00:38:30,648][105620] Updated weights for policy 1, policy_version 1272165 (0.0008) [2023-12-27 00:38:30,654][105692] Updated weights for policy 0, policy_version 1270855 (0.0008) [2023-12-27 00:38:30,697][105620] Updated weights for policy 1, policy_version 1272175 (0.0007) [2023-12-27 00:38:30,756][105620] Updated weights for policy 1, policy_version 1272185 (0.0009) [2023-12-27 00:38:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 651116544. Throughput: 0: 9905.6, 1: 9790.4. Samples: 651083932. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:38:31,062][104569] Avg episode reward: [(0, '8996.122'), (1, '9261.987')] [2023-12-27 00:38:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001270864_325394432.pth... [2023-12-27 00:38:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001272192_325722112.pth... [2023-12-27 00:38:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001271072_325435392.pth [2023-12-27 00:38:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001269712_325099520.pth [2023-12-27 00:38:31,395][105692] Updated weights for policy 0, policy_version 1270865 (0.0007) [2023-12-27 00:38:31,446][105692] Updated weights for policy 0, policy_version 1270875 (0.0009) [2023-12-27 00:38:31,491][105692] Updated weights for policy 0, policy_version 1270885 (0.0008) [2023-12-27 00:38:31,540][105620] Updated weights for policy 1, policy_version 1272195 (0.0008) [2023-12-27 00:38:31,546][105692] Updated weights for policy 0, policy_version 1270895 (0.0007) [2023-12-27 00:38:31,594][105620] Updated weights for policy 1, policy_version 1272205 (0.0008) [2023-12-27 00:38:31,661][105620] Updated weights for policy 1, policy_version 1272215 (0.0009) [2023-12-27 00:38:32,258][105692] Updated weights for policy 0, policy_version 1270905 (0.0007) [2023-12-27 00:38:32,329][105692] Updated weights for policy 0, policy_version 1270915 (0.0008) [2023-12-27 00:38:32,392][105692] Updated weights for policy 0, policy_version 1270925 (0.0008) [2023-12-27 00:38:32,430][105620] Updated weights for policy 1, policy_version 1272225 (0.0009) [2023-12-27 00:38:32,497][105620] Updated weights for policy 1, policy_version 1272235 (0.0007) [2023-12-27 00:38:32,556][105620] Updated weights for policy 1, policy_version 1272245 (0.0009) [2023-12-27 00:38:32,607][105620] Updated weights for policy 1, policy_version 1272255 (0.0009) [2023-12-27 00:38:33,124][105692] Updated weights for policy 0, policy_version 1270935 (0.0008) [2023-12-27 00:38:33,183][105692] Updated weights for policy 0, policy_version 1270945 (0.0009) [2023-12-27 00:38:33,243][105692] Updated weights for policy 0, policy_version 1270955 (0.0009) [2023-12-27 00:38:33,260][105620] Updated weights for policy 1, policy_version 1272265 (0.0007) [2023-12-27 00:38:33,321][105620] Updated weights for policy 1, policy_version 1272275 (0.0008) [2023-12-27 00:38:33,384][105620] Updated weights for policy 1, policy_version 1272285 (0.0009) [2023-12-27 00:38:33,829][105692] Updated weights for policy 0, policy_version 1270965 (0.0005) [2023-12-27 00:38:33,883][105692] Updated weights for policy 0, policy_version 1270975 (0.0006) [2023-12-27 00:38:33,930][105692] Updated weights for policy 0, policy_version 1270985 (0.0005) [2023-12-27 00:38:34,237][105620] Updated weights for policy 1, policy_version 1272295 (0.0009) [2023-12-27 00:38:34,299][105620] Updated weights for policy 1, policy_version 1272305 (0.0010) [2023-12-27 00:38:34,362][105620] Updated weights for policy 1, policy_version 1272315 (0.0009) [2023-12-27 00:38:34,539][105692] Updated weights for policy 0, policy_version 1270995 (0.0006) [2023-12-27 00:38:34,592][105692] Updated weights for policy 0, policy_version 1271005 (0.0009) [2023-12-27 00:38:34,640][105692] Updated weights for policy 0, policy_version 1271015 (0.0009) [2023-12-27 00:38:35,129][105620] Updated weights for policy 1, policy_version 1272325 (0.0010) [2023-12-27 00:38:35,189][105620] Updated weights for policy 1, policy_version 1272335 (0.0008) [2023-12-27 00:38:35,248][105620] Updated weights for policy 1, policy_version 1272345 (0.0010) [2023-12-27 00:38:35,427][105692] Updated weights for policy 0, policy_version 1271025 (0.0009) [2023-12-27 00:38:35,477][105692] Updated weights for policy 0, policy_version 1271035 (0.0009) [2023-12-27 00:38:35,526][105692] Updated weights for policy 0, policy_version 1271045 (0.0008) [2023-12-27 00:38:35,582][105692] Updated weights for policy 0, policy_version 1271055 (0.0010) [2023-12-27 00:38:35,844][105620] Updated weights for policy 1, policy_version 1272355 (0.0008) [2023-12-27 00:38:35,889][105620] Updated weights for policy 1, policy_version 1272365 (0.0005) [2023-12-27 00:38:35,949][105620] Updated weights for policy 1, policy_version 1272375 (0.0005) [2023-12-27 00:38:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 651214848. Throughput: 0: 10029.0, 1: 9623.7. Samples: 651201532. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:38:36,062][104569] Avg episode reward: [(0, '8905.642'), (1, '9086.285')] [2023-12-27 00:38:36,480][105692] Updated weights for policy 0, policy_version 1271065 (0.0009) [2023-12-27 00:38:36,527][105620] Updated weights for policy 1, policy_version 1272385 (0.0005) [2023-12-27 00:38:36,536][105692] Updated weights for policy 0, policy_version 1271075 (0.0010) [2023-12-27 00:38:36,593][105620] Updated weights for policy 1, policy_version 1272395 (0.0005) [2023-12-27 00:38:36,603][105692] Updated weights for policy 0, policy_version 1271085 (0.0010) [2023-12-27 00:38:36,649][105620] Updated weights for policy 1, policy_version 1272405 (0.0006) [2023-12-27 00:38:36,704][105620] Updated weights for policy 1, policy_version 1272415 (0.0006) [2023-12-27 00:38:37,277][105620] Updated weights for policy 1, policy_version 1272425 (0.0009) [2023-12-27 00:38:37,324][105620] Updated weights for policy 1, policy_version 1272435 (0.0007) [2023-12-27 00:38:37,372][105620] Updated weights for policy 1, policy_version 1272445 (0.0006) [2023-12-27 00:38:37,480][105692] Updated weights for policy 0, policy_version 1271095 (0.0009) [2023-12-27 00:38:37,532][105692] Updated weights for policy 0, policy_version 1271105 (0.0009) [2023-12-27 00:38:37,585][105692] Updated weights for policy 0, policy_version 1271115 (0.0009) [2023-12-27 00:38:38,021][105620] Updated weights for policy 1, policy_version 1272455 (0.0009) [2023-12-27 00:38:38,074][105620] Updated weights for policy 1, policy_version 1272465 (0.0010) [2023-12-27 00:38:38,131][105620] Updated weights for policy 1, policy_version 1272475 (0.0009) [2023-12-27 00:38:38,431][105692] Updated weights for policy 0, policy_version 1271125 (0.0009) [2023-12-27 00:38:38,488][105692] Updated weights for policy 0, policy_version 1271135 (0.0009) [2023-12-27 00:38:38,551][105692] Updated weights for policy 0, policy_version 1271145 (0.0009) [2023-12-27 00:38:38,845][105620] Updated weights for policy 1, policy_version 1272485 (0.0008) [2023-12-27 00:38:38,898][105620] Updated weights for policy 1, policy_version 1272495 (0.0009) [2023-12-27 00:38:38,952][105620] Updated weights for policy 1, policy_version 1272505 (0.0008) [2023-12-27 00:38:39,338][105692] Updated weights for policy 0, policy_version 1271155 (0.0008) [2023-12-27 00:38:39,402][105692] Updated weights for policy 0, policy_version 1271165 (0.0008) [2023-12-27 00:38:39,464][105692] Updated weights for policy 0, policy_version 1271175 (0.0008) [2023-12-27 00:38:39,756][105620] Updated weights for policy 1, policy_version 1272515 (0.0009) [2023-12-27 00:38:39,820][105620] Updated weights for policy 1, policy_version 1272525 (0.0008) [2023-12-27 00:38:39,889][105620] Updated weights for policy 1, policy_version 1272535 (0.0008) [2023-12-27 00:38:40,216][105692] Updated weights for policy 0, policy_version 1271185 (0.0008) [2023-12-27 00:38:40,278][105692] Updated weights for policy 0, policy_version 1271195 (0.0007) [2023-12-27 00:38:40,340][105692] Updated weights for policy 0, policy_version 1271205 (0.0007) [2023-12-27 00:38:40,402][105692] Updated weights for policy 0, policy_version 1271215 (0.0010) [2023-12-27 00:38:40,656][105620] Updated weights for policy 1, policy_version 1272545 (0.0008) [2023-12-27 00:38:40,716][105620] Updated weights for policy 1, policy_version 1272555 (0.0008) [2023-12-27 00:38:40,783][105620] Updated weights for policy 1, policy_version 1272565 (0.0008) [2023-12-27 00:38:40,842][105620] Updated weights for policy 1, policy_version 1272575 (0.0008) [2023-12-27 00:38:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 651304960. Throughput: 0: 9829.5, 1: 9657.2. Samples: 651314880. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:38:41,062][104569] Avg episode reward: [(0, '9087.661'), (1, '9187.387')] [2023-12-27 00:38:41,120][105692] Updated weights for policy 0, policy_version 1271225 (0.0011) [2023-12-27 00:38:41,188][105692] Updated weights for policy 0, policy_version 1271235 (0.0011) [2023-12-27 00:38:41,251][105692] Updated weights for policy 0, policy_version 1271245 (0.0010) [2023-12-27 00:38:41,677][105620] Updated weights for policy 1, policy_version 1272585 (0.0008) [2023-12-27 00:38:41,748][105620] Updated weights for policy 1, policy_version 1272595 (0.0008) [2023-12-27 00:38:41,797][105620] Updated weights for policy 1, policy_version 1272605 (0.0009) [2023-12-27 00:38:42,025][105692] Updated weights for policy 0, policy_version 1271255 (0.0009) [2023-12-27 00:38:42,077][105692] Updated weights for policy 0, policy_version 1271265 (0.0009) [2023-12-27 00:38:42,143][105692] Updated weights for policy 0, policy_version 1271275 (0.0007) [2023-12-27 00:38:42,573][105620] Updated weights for policy 1, policy_version 1272615 (0.0009) [2023-12-27 00:38:42,633][105620] Updated weights for policy 1, policy_version 1272625 (0.0010) [2023-12-27 00:38:42,696][105620] Updated weights for policy 1, policy_version 1272635 (0.0009) [2023-12-27 00:38:42,808][105692] Updated weights for policy 0, policy_version 1271285 (0.0007) [2023-12-27 00:38:42,869][105692] Updated weights for policy 0, policy_version 1271295 (0.0009) [2023-12-27 00:38:42,930][105692] Updated weights for policy 0, policy_version 1271305 (0.0009) [2023-12-27 00:38:43,379][105620] Updated weights for policy 1, policy_version 1272645 (0.0009) [2023-12-27 00:38:43,438][105620] Updated weights for policy 1, policy_version 1272655 (0.0009) [2023-12-27 00:38:43,503][105620] Updated weights for policy 1, policy_version 1272665 (0.0009) [2023-12-27 00:38:43,703][105692] Updated weights for policy 0, policy_version 1271315 (0.0009) [2023-12-27 00:38:43,753][105692] Updated weights for policy 0, policy_version 1271325 (0.0009) [2023-12-27 00:38:43,806][105692] Updated weights for policy 0, policy_version 1271335 (0.0008) [2023-12-27 00:38:44,187][105620] Updated weights for policy 1, policy_version 1272675 (0.0009) [2023-12-27 00:38:44,242][105620] Updated weights for policy 1, policy_version 1272685 (0.0009) [2023-12-27 00:38:44,289][105620] Updated weights for policy 1, policy_version 1272695 (0.0009) [2023-12-27 00:38:44,595][105692] Updated weights for policy 0, policy_version 1271345 (0.0009) [2023-12-27 00:38:44,662][105692] Updated weights for policy 0, policy_version 1271355 (0.0008) [2023-12-27 00:38:44,715][105692] Updated weights for policy 0, policy_version 1271366 (0.0010) [2023-12-27 00:38:44,764][105692] Updated weights for policy 0, policy_version 1271376 (0.0009) [2023-12-27 00:38:44,985][105620] Updated weights for policy 1, policy_version 1272705 (0.0009) [2023-12-27 00:38:45,048][105620] Updated weights for policy 1, policy_version 1272715 (0.0009) [2023-12-27 00:38:45,116][105620] Updated weights for policy 1, policy_version 1272725 (0.0006) [2023-12-27 00:38:45,186][105620] Updated weights for policy 1, policy_version 1272735 (0.0007) [2023-12-27 00:38:45,575][105692] Updated weights for policy 0, policy_version 1271386 (0.0010) [2023-12-27 00:38:45,628][105692] Updated weights for policy 0, policy_version 1271396 (0.0010) [2023-12-27 00:38:45,693][105692] Updated weights for policy 0, policy_version 1271406 (0.0010) [2023-12-27 00:38:45,828][105620] Updated weights for policy 1, policy_version 1272745 (0.0006) [2023-12-27 00:38:45,888][105620] Updated weights for policy 1, policy_version 1272755 (0.0005) [2023-12-27 00:38:45,944][105620] Updated weights for policy 1, policy_version 1272765 (0.0005) [2023-12-27 00:38:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 651403264. Throughput: 0: 9752.8, 1: 9627.9. Samples: 651371108. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:38:46,063][104569] Avg episode reward: [(0, '9175.762'), (1, '9095.614')] [2023-12-27 00:38:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001271408_325533696.pth... [2023-12-27 00:38:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001272768_325869568.pth... [2023-12-27 00:38:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001270288_325246976.pth [2023-12-27 00:38:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001271648_325582848.pth [2023-12-27 00:38:46,513][105620] Updated weights for policy 1, policy_version 1272775 (0.0006) [2023-12-27 00:38:46,538][105692] Updated weights for policy 0, policy_version 1271416 (0.0009) [2023-12-27 00:38:46,564][105620] Updated weights for policy 1, policy_version 1272785 (0.0006) [2023-12-27 00:38:46,598][105692] Updated weights for policy 0, policy_version 1271426 (0.0008) [2023-12-27 00:38:46,631][105620] Updated weights for policy 1, policy_version 1272795 (0.0006) [2023-12-27 00:38:46,649][105692] Updated weights for policy 0, policy_version 1271436 (0.0009) [2023-12-27 00:38:47,238][105620] Updated weights for policy 1, policy_version 1272805 (0.0006) [2023-12-27 00:38:47,306][105620] Updated weights for policy 1, policy_version 1272815 (0.0006) [2023-12-27 00:38:47,358][105620] Updated weights for policy 1, policy_version 1272825 (0.0005) [2023-12-27 00:38:47,502][105692] Updated weights for policy 0, policy_version 1271446 (0.0009) [2023-12-27 00:38:47,571][105692] Updated weights for policy 0, policy_version 1271456 (0.0010) [2023-12-27 00:38:47,637][105692] Updated weights for policy 0, policy_version 1271466 (0.0010) [2023-12-27 00:38:47,892][105620] Updated weights for policy 1, policy_version 1272835 (0.0005) [2023-12-27 00:38:47,953][105620] Updated weights for policy 1, policy_version 1272845 (0.0006) [2023-12-27 00:38:48,014][105620] Updated weights for policy 1, policy_version 1272855 (0.0009) [2023-12-27 00:38:48,426][105692] Updated weights for policy 0, policy_version 1271476 (0.0010) [2023-12-27 00:38:48,481][105692] Updated weights for policy 0, policy_version 1271486 (0.0009) [2023-12-27 00:38:48,536][105692] Updated weights for policy 0, policy_version 1271496 (0.0009) [2023-12-27 00:38:48,742][105620] Updated weights for policy 1, policy_version 1272865 (0.0009) [2023-12-27 00:38:48,790][105620] Updated weights for policy 1, policy_version 1272875 (0.0011) [2023-12-27 00:38:48,846][105620] Updated weights for policy 1, policy_version 1272885 (0.0010) [2023-12-27 00:38:48,894][105620] Updated weights for policy 1, policy_version 1272895 (0.0011) [2023-12-27 00:38:49,299][105692] Updated weights for policy 0, policy_version 1271506 (0.0009) [2023-12-27 00:38:49,374][105692] Updated weights for policy 0, policy_version 1271516 (0.0009) [2023-12-27 00:38:49,440][105692] Updated weights for policy 0, policy_version 1271526 (0.0009) [2023-12-27 00:38:49,507][105692] Updated weights for policy 0, policy_version 1271536 (0.0008) [2023-12-27 00:38:49,646][105620] Updated weights for policy 1, policy_version 1272905 (0.0008) [2023-12-27 00:38:49,712][105620] Updated weights for policy 1, policy_version 1272915 (0.0005) [2023-12-27 00:38:49,767][105620] Updated weights for policy 1, policy_version 1272925 (0.0010) [2023-12-27 00:38:50,287][105692] Updated weights for policy 0, policy_version 1271546 (0.0008) [2023-12-27 00:38:50,352][105692] Updated weights for policy 0, policy_version 1271556 (0.0009) [2023-12-27 00:38:50,412][105692] Updated weights for policy 0, policy_version 1271566 (0.0009) [2023-12-27 00:38:50,435][105620] Updated weights for policy 1, policy_version 1272935 (0.0011) [2023-12-27 00:38:50,501][105620] Updated weights for policy 1, policy_version 1272945 (0.0011) [2023-12-27 00:38:50,563][105620] Updated weights for policy 1, policy_version 1272955 (0.0010) [2023-12-27 00:38:51,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 651493376. Throughput: 0: 9590.0, 1: 9804.1. Samples: 651486288. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:38:51,062][104569] Avg episode reward: [(0, '9180.485'), (1, '9018.165')] [2023-12-27 00:38:51,200][105692] Updated weights for policy 0, policy_version 1271576 (0.0008) [2023-12-27 00:38:51,257][105692] Updated weights for policy 0, policy_version 1271586 (0.0008) [2023-12-27 00:38:51,319][105692] Updated weights for policy 0, policy_version 1271596 (0.0007) [2023-12-27 00:38:51,326][105620] Updated weights for policy 1, policy_version 1272965 (0.0011) [2023-12-27 00:38:51,407][105620] Updated weights for policy 1, policy_version 1272975 (0.0009) [2023-12-27 00:38:51,473][105620] Updated weights for policy 1, policy_version 1272985 (0.0011) [2023-12-27 00:38:52,152][105692] Updated weights for policy 0, policy_version 1271606 (0.0009) [2023-12-27 00:38:52,198][105620] Updated weights for policy 1, policy_version 1272995 (0.0010) [2023-12-27 00:38:52,213][105692] Updated weights for policy 0, policy_version 1271616 (0.0008) [2023-12-27 00:38:52,249][105620] Updated weights for policy 1, policy_version 1273005 (0.0007) [2023-12-27 00:38:52,270][105692] Updated weights for policy 0, policy_version 1271626 (0.0007) [2023-12-27 00:38:52,314][105620] Updated weights for policy 1, policy_version 1273015 (0.0006) [2023-12-27 00:38:53,017][105620] Updated weights for policy 1, policy_version 1273025 (0.0008) [2023-12-27 00:38:53,071][105620] Updated weights for policy 1, policy_version 1273035 (0.0008) [2023-12-27 00:38:53,089][105692] Updated weights for policy 0, policy_version 1271636 (0.0009) [2023-12-27 00:38:53,131][105620] Updated weights for policy 1, policy_version 1273045 (0.0006) [2023-12-27 00:38:53,151][105692] Updated weights for policy 0, policy_version 1271646 (0.0009) [2023-12-27 00:38:53,182][105620] Updated weights for policy 1, policy_version 1273055 (0.0007) [2023-12-27 00:38:53,217][105692] Updated weights for policy 0, policy_version 1271656 (0.0009) [2023-12-27 00:38:53,848][105692] Updated weights for policy 0, policy_version 1271666 (0.0008) [2023-12-27 00:38:53,877][105620] Updated weights for policy 1, policy_version 1273065 (0.0007) [2023-12-27 00:38:53,908][105692] Updated weights for policy 0, policy_version 1271676 (0.0005) [2023-12-27 00:38:53,938][105620] Updated weights for policy 1, policy_version 1273075 (0.0006) [2023-12-27 00:38:53,970][105692] Updated weights for policy 0, policy_version 1271686 (0.0006) [2023-12-27 00:38:53,987][105620] Updated weights for policy 1, policy_version 1273085 (0.0005) [2023-12-27 00:38:54,020][105692] Updated weights for policy 0, policy_version 1271696 (0.0005) [2023-12-27 00:38:54,584][105692] Updated weights for policy 0, policy_version 1271706 (0.0011) [2023-12-27 00:38:54,622][105620] Updated weights for policy 1, policy_version 1273095 (0.0009) [2023-12-27 00:38:54,643][105692] Updated weights for policy 0, policy_version 1271716 (0.0010) [2023-12-27 00:38:54,682][105620] Updated weights for policy 1, policy_version 1273105 (0.0011) [2023-12-27 00:38:54,698][105692] Updated weights for policy 0, policy_version 1271726 (0.0005) [2023-12-27 00:38:54,743][105620] Updated weights for policy 1, policy_version 1273115 (0.0006) [2023-12-27 00:38:55,304][105692] Updated weights for policy 0, policy_version 1271736 (0.0006) [2023-12-27 00:38:55,307][105620] Updated weights for policy 1, policy_version 1273125 (0.0009) [2023-12-27 00:38:55,359][105692] Updated weights for policy 0, policy_version 1271746 (0.0006) [2023-12-27 00:38:55,359][105620] Updated weights for policy 1, policy_version 1273135 (0.0010) [2023-12-27 00:38:55,414][105620] Updated weights for policy 1, policy_version 1273145 (0.0010) [2023-12-27 00:38:55,415][105692] Updated weights for policy 0, policy_version 1271756 (0.0005) [2023-12-27 00:38:56,028][105692] Updated weights for policy 0, policy_version 1271766 (0.0009) [2023-12-27 00:38:56,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 651591680. Throughput: 0: 9591.2, 1: 9834.9. Samples: 651605768. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:38:56,062][104569] Avg episode reward: [(0, '9003.223'), (1, '8933.787')] [2023-12-27 00:38:56,090][105692] Updated weights for policy 0, policy_version 1271776 (0.0010) [2023-12-27 00:38:56,091][105620] Updated weights for policy 1, policy_version 1273155 (0.0009) [2023-12-27 00:38:56,136][105692] Updated weights for policy 0, policy_version 1271786 (0.0010) [2023-12-27 00:38:56,141][105620] Updated weights for policy 1, policy_version 1273165 (0.0005) [2023-12-27 00:38:56,190][105620] Updated weights for policy 1, policy_version 1273175 (0.0005) [2023-12-27 00:38:56,819][105692] Updated weights for policy 0, policy_version 1271796 (0.0008) [2023-12-27 00:38:56,862][105620] Updated weights for policy 1, policy_version 1273185 (0.0006) [2023-12-27 00:38:56,883][105692] Updated weights for policy 0, policy_version 1271806 (0.0006) [2023-12-27 00:38:56,921][105620] Updated weights for policy 1, policy_version 1273195 (0.0011) [2023-12-27 00:38:56,945][105692] Updated weights for policy 0, policy_version 1271816 (0.0008) [2023-12-27 00:38:56,980][105620] Updated weights for policy 1, policy_version 1273205 (0.0010) [2023-12-27 00:38:57,046][105620] Updated weights for policy 1, policy_version 1273215 (0.0009) [2023-12-27 00:38:57,514][105692] Updated weights for policy 0, policy_version 1271826 (0.0009) [2023-12-27 00:38:57,582][105692] Updated weights for policy 0, policy_version 1271836 (0.0005) [2023-12-27 00:38:57,645][105692] Updated weights for policy 0, policy_version 1271846 (0.0006) [2023-12-27 00:38:57,671][105620] Updated weights for policy 1, policy_version 1273225 (0.0008) [2023-12-27 00:38:57,703][105692] Updated weights for policy 0, policy_version 1271856 (0.0010) [2023-12-27 00:38:57,720][105620] Updated weights for policy 1, policy_version 1273235 (0.0005) [2023-12-27 00:38:57,781][105620] Updated weights for policy 1, policy_version 1273245 (0.0005) [2023-12-27 00:38:58,354][105620] Updated weights for policy 1, policy_version 1273255 (0.0008) [2023-12-27 00:38:58,387][105692] Updated weights for policy 0, policy_version 1271866 (0.0009) [2023-12-27 00:38:58,418][105620] Updated weights for policy 1, policy_version 1273265 (0.0007) [2023-12-27 00:38:58,449][105692] Updated weights for policy 0, policy_version 1271876 (0.0008) [2023-12-27 00:38:58,477][105620] Updated weights for policy 1, policy_version 1273275 (0.0006) [2023-12-27 00:38:58,508][105692] Updated weights for policy 0, policy_version 1271886 (0.0008) [2023-12-27 00:38:59,246][105692] Updated weights for policy 0, policy_version 1271896 (0.0008) [2023-12-27 00:38:59,263][105620] Updated weights for policy 1, policy_version 1273285 (0.0007) [2023-12-27 00:38:59,305][105692] Updated weights for policy 0, policy_version 1271906 (0.0009) [2023-12-27 00:38:59,311][105620] Updated weights for policy 1, policy_version 1273295 (0.0008) [2023-12-27 00:38:59,368][105692] Updated weights for policy 0, policy_version 1271916 (0.0007) [2023-12-27 00:38:59,374][105620] Updated weights for policy 1, policy_version 1273305 (0.0008) [2023-12-27 00:39:00,123][105620] Updated weights for policy 1, policy_version 1273315 (0.0009) [2023-12-27 00:39:00,151][105692] Updated weights for policy 0, policy_version 1271926 (0.0006) [2023-12-27 00:39:00,177][105620] Updated weights for policy 1, policy_version 1273325 (0.0010) [2023-12-27 00:39:00,208][105692] Updated weights for policy 0, policy_version 1271936 (0.0005) [2023-12-27 00:39:00,240][105620] Updated weights for policy 1, policy_version 1273335 (0.0011) [2023-12-27 00:39:00,273][105692] Updated weights for policy 0, policy_version 1271946 (0.0005) [2023-12-27 00:39:00,846][105620] Updated weights for policy 1, policy_version 1273345 (0.0011) [2023-12-27 00:39:00,904][105620] Updated weights for policy 1, policy_version 1273355 (0.0010) [2023-12-27 00:39:00,915][105692] Updated weights for policy 0, policy_version 1271956 (0.0007) [2023-12-27 00:39:00,948][105620] Updated weights for policy 1, policy_version 1273365 (0.0010) [2023-12-27 00:39:00,970][105692] Updated weights for policy 0, policy_version 1271966 (0.0005) [2023-12-27 00:39:00,996][105620] Updated weights for policy 1, policy_version 1273375 (0.0010) [2023-12-27 00:39:01,021][105692] Updated weights for policy 0, policy_version 1271976 (0.0006) [2023-12-27 00:39:01,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 651698176. Throughput: 0: 9673.5, 1: 9864.2. Samples: 651668048. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:39:01,062][104569] Avg episode reward: [(0, '8999.745'), (1, '8995.481')] [2023-12-27 00:39:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001273376_326025216.pth... [2023-12-27 00:39:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001271984_325681152.pth... [2023-12-27 00:39:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001272192_325722112.pth [2023-12-27 00:39:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001270864_325394432.pth [2023-12-27 00:39:01,717][105692] Updated weights for policy 0, policy_version 1271986 (0.0008) [2023-12-27 00:39:01,757][105620] Updated weights for policy 1, policy_version 1273385 (0.0009) [2023-12-27 00:39:01,782][105692] Updated weights for policy 0, policy_version 1271996 (0.0008) [2023-12-27 00:39:01,820][105620] Updated weights for policy 1, policy_version 1273395 (0.0010) [2023-12-27 00:39:01,842][105692] Updated weights for policy 0, policy_version 1272006 (0.0008) [2023-12-27 00:39:01,872][105620] Updated weights for policy 1, policy_version 1273405 (0.0010) [2023-12-27 00:39:01,898][105692] Updated weights for policy 0, policy_version 1272016 (0.0006) [2023-12-27 00:39:02,627][105620] Updated weights for policy 1, policy_version 1273415 (0.0010) [2023-12-27 00:39:02,649][105692] Updated weights for policy 0, policy_version 1272026 (0.0006) [2023-12-27 00:39:02,688][105620] Updated weights for policy 1, policy_version 1273425 (0.0010) [2023-12-27 00:39:02,702][105692] Updated weights for policy 0, policy_version 1272036 (0.0008) [2023-12-27 00:39:02,747][105692] Updated weights for policy 0, policy_version 1272046 (0.0007) [2023-12-27 00:39:02,750][105620] Updated weights for policy 1, policy_version 1273435 (0.0010) [2023-12-27 00:39:03,422][105620] Updated weights for policy 1, policy_version 1273445 (0.0010) [2023-12-27 00:39:03,473][105620] Updated weights for policy 1, policy_version 1273455 (0.0010) [2023-12-27 00:39:03,524][105620] Updated weights for policy 1, policy_version 1273465 (0.0010) [2023-12-27 00:39:03,538][105692] Updated weights for policy 0, policy_version 1272056 (0.0005) [2023-12-27 00:39:03,595][105692] Updated weights for policy 0, policy_version 1272066 (0.0006) [2023-12-27 00:39:03,649][105692] Updated weights for policy 0, policy_version 1272076 (0.0008) [2023-12-27 00:39:04,288][105620] Updated weights for policy 1, policy_version 1273475 (0.0010) [2023-12-27 00:39:04,344][105620] Updated weights for policy 1, policy_version 1273485 (0.0010) [2023-12-27 00:39:04,396][105620] Updated weights for policy 1, policy_version 1273495 (0.0010) [2023-12-27 00:39:04,398][105692] Updated weights for policy 0, policy_version 1272086 (0.0008) [2023-12-27 00:39:04,443][105586] KL-divergence is very high: 113.8186 [2023-12-27 00:39:04,454][105692] Updated weights for policy 0, policy_version 1272096 (0.0006) [2023-12-27 00:39:04,510][105692] Updated weights for policy 0, policy_version 1272106 (0.0009) [2023-12-27 00:39:05,083][105620] Updated weights for policy 1, policy_version 1273505 (0.0010) [2023-12-27 00:39:05,140][105620] Updated weights for policy 1, policy_version 1273515 (0.0005) [2023-12-27 00:39:05,194][105620] Updated weights for policy 1, policy_version 1273525 (0.0005) [2023-12-27 00:39:05,250][105620] Updated weights for policy 1, policy_version 1273535 (0.0005) [2023-12-27 00:39:05,338][105692] Updated weights for policy 0, policy_version 1272116 (0.0008) [2023-12-27 00:39:05,391][105692] Updated weights for policy 0, policy_version 1272126 (0.0008) [2023-12-27 00:39:05,435][105692] Updated weights for policy 0, policy_version 1272136 (0.0008) [2023-12-27 00:39:05,918][105620] Updated weights for policy 1, policy_version 1273545 (0.0010) [2023-12-27 00:39:05,969][105620] Updated weights for policy 1, policy_version 1273555 (0.0010) [2023-12-27 00:39:06,012][105620] Updated weights for policy 1, policy_version 1273565 (0.0010) [2023-12-27 00:39:06,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 651796480. Throughput: 0: 9547.0, 1: 9796.7. Samples: 651783120. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:39:06,063][104569] Avg episode reward: [(0, '9085.503'), (1, '9087.185')] [2023-12-27 00:39:06,114][105692] Updated weights for policy 0, policy_version 1272146 (0.0008) [2023-12-27 00:39:06,183][105692] Updated weights for policy 0, policy_version 1272156 (0.0007) [2023-12-27 00:39:06,243][105692] Updated weights for policy 0, policy_version 1272166 (0.0008) [2023-12-27 00:39:06,311][105692] Updated weights for policy 0, policy_version 1272176 (0.0006) [2023-12-27 00:39:06,761][105620] Updated weights for policy 1, policy_version 1273575 (0.0010) [2023-12-27 00:39:06,810][105620] Updated weights for policy 1, policy_version 1273585 (0.0010) [2023-12-27 00:39:06,859][105620] Updated weights for policy 1, policy_version 1273595 (0.0010) [2023-12-27 00:39:06,991][105692] Updated weights for policy 0, policy_version 1272186 (0.0008) [2023-12-27 00:39:07,054][105692] Updated weights for policy 0, policy_version 1272196 (0.0009) [2023-12-27 00:39:07,117][105692] Updated weights for policy 0, policy_version 1272206 (0.0009) [2023-12-27 00:39:07,594][105620] Updated weights for policy 1, policy_version 1273605 (0.0010) [2023-12-27 00:39:07,648][105620] Updated weights for policy 1, policy_version 1273615 (0.0009) [2023-12-27 00:39:07,700][105620] Updated weights for policy 1, policy_version 1273625 (0.0006) [2023-12-27 00:39:07,901][105692] Updated weights for policy 0, policy_version 1272216 (0.0009) [2023-12-27 00:39:07,962][105692] Updated weights for policy 0, policy_version 1272226 (0.0009) [2023-12-27 00:39:08,022][105692] Updated weights for policy 0, policy_version 1272236 (0.0007) [2023-12-27 00:39:08,350][105620] Updated weights for policy 1, policy_version 1273635 (0.0006) [2023-12-27 00:39:08,412][105620] Updated weights for policy 1, policy_version 1273645 (0.0008) [2023-12-27 00:39:08,473][105620] Updated weights for policy 1, policy_version 1273655 (0.0007) [2023-12-27 00:39:08,726][105692] Updated weights for policy 0, policy_version 1272246 (0.0007) [2023-12-27 00:39:08,789][105692] Updated weights for policy 0, policy_version 1272256 (0.0011) [2023-12-27 00:39:08,859][105692] Updated weights for policy 0, policy_version 1272266 (0.0010) [2023-12-27 00:39:09,113][105620] Updated weights for policy 1, policy_version 1273665 (0.0008) [2023-12-27 00:39:09,179][105620] Updated weights for policy 1, policy_version 1273675 (0.0007) [2023-12-27 00:39:09,244][105620] Updated weights for policy 1, policy_version 1273685 (0.0008) [2023-12-27 00:39:09,297][105620] Updated weights for policy 1, policy_version 1273695 (0.0009) [2023-12-27 00:39:09,634][105692] Updated weights for policy 0, policy_version 1272276 (0.0010) [2023-12-27 00:39:09,694][105692] Updated weights for policy 0, policy_version 1272286 (0.0011) [2023-12-27 00:39:09,750][105692] Updated weights for policy 0, policy_version 1272296 (0.0011) [2023-12-27 00:39:09,964][105620] Updated weights for policy 1, policy_version 1273705 (0.0008) [2023-12-27 00:39:10,018][105620] Updated weights for policy 1, policy_version 1273715 (0.0005) [2023-12-27 00:39:10,069][105620] Updated weights for policy 1, policy_version 1273725 (0.0005) [2023-12-27 00:39:10,525][105692] Updated weights for policy 0, policy_version 1272306 (0.0010) [2023-12-27 00:39:10,584][105692] Updated weights for policy 0, policy_version 1272316 (0.0008) [2023-12-27 00:39:10,639][105692] Updated weights for policy 0, policy_version 1272326 (0.0008) [2023-12-27 00:39:10,702][105692] Updated weights for policy 0, policy_version 1272336 (0.0008) [2023-12-27 00:39:10,790][105620] Updated weights for policy 1, policy_version 1273735 (0.0009) [2023-12-27 00:39:10,842][105620] Updated weights for policy 1, policy_version 1273745 (0.0010) [2023-12-27 00:39:10,904][105620] Updated weights for policy 1, policy_version 1273755 (0.0010) [2023-12-27 00:39:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 651894784. Throughput: 0: 9476.2, 1: 9965.2. Samples: 651900344. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:39:11,063][104569] Avg episode reward: [(0, '9177.407'), (1, '9087.317')] [2023-12-27 00:39:11,495][105692] Updated weights for policy 0, policy_version 1272346 (0.0011) [2023-12-27 00:39:11,552][105692] Updated weights for policy 0, policy_version 1272356 (0.0011) [2023-12-27 00:39:11,619][105692] Updated weights for policy 0, policy_version 1272366 (0.0011) [2023-12-27 00:39:11,666][105620] Updated weights for policy 1, policy_version 1273765 (0.0009) [2023-12-27 00:39:11,729][105620] Updated weights for policy 1, policy_version 1273775 (0.0010) [2023-12-27 00:39:11,779][105620] Updated weights for policy 1, policy_version 1273785 (0.0008) [2023-12-27 00:39:12,416][105692] Updated weights for policy 0, policy_version 1272376 (0.0007) [2023-12-27 00:39:12,474][105692] Updated weights for policy 0, policy_version 1272386 (0.0007) [2023-12-27 00:39:12,534][105692] Updated weights for policy 0, policy_version 1272396 (0.0011) [2023-12-27 00:39:12,564][105620] Updated weights for policy 1, policy_version 1273795 (0.0009) [2023-12-27 00:39:12,620][105620] Updated weights for policy 1, policy_version 1273805 (0.0007) [2023-12-27 00:39:12,679][105620] Updated weights for policy 1, policy_version 1273815 (0.0005) [2023-12-27 00:39:13,221][105692] Updated weights for policy 0, policy_version 1272406 (0.0009) [2023-12-27 00:39:13,284][105692] Updated weights for policy 0, policy_version 1272416 (0.0008) [2023-12-27 00:39:13,317][105620] Updated weights for policy 1, policy_version 1273825 (0.0006) [2023-12-27 00:39:13,340][105692] Updated weights for policy 0, policy_version 1272426 (0.0008) [2023-12-27 00:39:13,369][105620] Updated weights for policy 1, policy_version 1273835 (0.0010) [2023-12-27 00:39:13,426][105620] Updated weights for policy 1, policy_version 1273845 (0.0010) [2023-12-27 00:39:13,477][105620] Updated weights for policy 1, policy_version 1273855 (0.0010) [2023-12-27 00:39:14,109][105692] Updated weights for policy 0, policy_version 1272436 (0.0007) [2023-12-27 00:39:14,165][105692] Updated weights for policy 0, policy_version 1272446 (0.0008) [2023-12-27 00:39:14,218][105620] Updated weights for policy 1, policy_version 1273865 (0.0010) [2023-12-27 00:39:14,224][105692] Updated weights for policy 0, policy_version 1272456 (0.0006) [2023-12-27 00:39:14,279][105620] Updated weights for policy 1, policy_version 1273875 (0.0010) [2023-12-27 00:39:14,334][105620] Updated weights for policy 1, policy_version 1273885 (0.0010) [2023-12-27 00:39:14,867][105692] Updated weights for policy 0, policy_version 1272466 (0.0006) [2023-12-27 00:39:14,917][105692] Updated weights for policy 0, policy_version 1272476 (0.0008) [2023-12-27 00:39:14,968][105692] Updated weights for policy 0, policy_version 1272486 (0.0009) [2023-12-27 00:39:15,032][105692] Updated weights for policy 0, policy_version 1272496 (0.0009) [2023-12-27 00:39:15,067][105620] Updated weights for policy 1, policy_version 1273895 (0.0007) [2023-12-27 00:39:15,125][105620] Updated weights for policy 1, policy_version 1273905 (0.0005) [2023-12-27 00:39:15,187][105620] Updated weights for policy 1, policy_version 1273915 (0.0008) [2023-12-27 00:39:15,847][105692] Updated weights for policy 0, policy_version 1272506 (0.0008) [2023-12-27 00:39:15,890][105620] Updated weights for policy 1, policy_version 1273925 (0.0011) [2023-12-27 00:39:15,893][105692] Updated weights for policy 0, policy_version 1272516 (0.0007) [2023-12-27 00:39:15,946][105692] Updated weights for policy 0, policy_version 1272526 (0.0007) [2023-12-27 00:39:15,951][105620] Updated weights for policy 1, policy_version 1273935 (0.0010) [2023-12-27 00:39:16,006][105620] Updated weights for policy 1, policy_version 1273945 (0.0010) [2023-12-27 00:39:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 651993088. Throughput: 0: 9463.5, 1: 9945.4. Samples: 651957336. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:39:16,063][104569] Avg episode reward: [(0, '8997.758'), (1, '9263.364')] [2023-12-27 00:39:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001272528_325820416.pth... [2023-12-27 00:39:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001273952_326172672.pth... [2023-12-27 00:39:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001271408_325533696.pth [2023-12-27 00:39:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001272768_325869568.pth [2023-12-27 00:39:16,684][105620] Updated weights for policy 1, policy_version 1273955 (0.0009) [2023-12-27 00:39:16,737][105620] Updated weights for policy 1, policy_version 1273965 (0.0006) [2023-12-27 00:39:16,767][105692] Updated weights for policy 0, policy_version 1272536 (0.0008) [2023-12-27 00:39:16,782][105620] Updated weights for policy 1, policy_version 1273975 (0.0005) [2023-12-27 00:39:16,831][105692] Updated weights for policy 0, policy_version 1272546 (0.0010) [2023-12-27 00:39:16,899][105692] Updated weights for policy 0, policy_version 1272556 (0.0009) [2023-12-27 00:39:17,342][105620] Updated weights for policy 1, policy_version 1273985 (0.0005) [2023-12-27 00:39:17,395][105620] Updated weights for policy 1, policy_version 1273995 (0.0005) [2023-12-27 00:39:17,443][105620] Updated weights for policy 1, policy_version 1274005 (0.0006) [2023-12-27 00:39:17,497][105620] Updated weights for policy 1, policy_version 1274015 (0.0006) [2023-12-27 00:39:17,758][105692] Updated weights for policy 0, policy_version 1272566 (0.0008) [2023-12-27 00:39:17,815][105692] Updated weights for policy 0, policy_version 1272576 (0.0008) [2023-12-27 00:39:17,870][105692] Updated weights for policy 0, policy_version 1272586 (0.0008) [2023-12-27 00:39:18,152][105620] Updated weights for policy 1, policy_version 1274025 (0.0010) [2023-12-27 00:39:18,214][105620] Updated weights for policy 1, policy_version 1274035 (0.0010) [2023-12-27 00:39:18,280][105620] Updated weights for policy 1, policy_version 1274045 (0.0010) [2023-12-27 00:39:18,565][105692] Updated weights for policy 0, policy_version 1272596 (0.0008) [2023-12-27 00:39:18,618][105692] Updated weights for policy 0, policy_version 1272606 (0.0008) [2023-12-27 00:39:18,668][105692] Updated weights for policy 0, policy_version 1272616 (0.0008) [2023-12-27 00:39:19,009][105620] Updated weights for policy 1, policy_version 1274055 (0.0007) [2023-12-27 00:39:19,053][105620] Updated weights for policy 1, policy_version 1274065 (0.0005) [2023-12-27 00:39:19,105][105620] Updated weights for policy 1, policy_version 1274075 (0.0005) [2023-12-27 00:39:19,559][105692] Updated weights for policy 0, policy_version 1272626 (0.0008) [2023-12-27 00:39:19,624][105692] Updated weights for policy 0, policy_version 1272636 (0.0007) [2023-12-27 00:39:19,692][105692] Updated weights for policy 0, policy_version 1272646 (0.0005) [2023-12-27 00:39:19,753][105692] Updated weights for policy 0, policy_version 1272656 (0.0006) [2023-12-27 00:39:19,883][105620] Updated weights for policy 1, policy_version 1274085 (0.0005) [2023-12-27 00:39:19,948][105620] Updated weights for policy 1, policy_version 1274095 (0.0008) [2023-12-27 00:39:20,011][105620] Updated weights for policy 1, policy_version 1274105 (0.0009) [2023-12-27 00:39:20,380][105692] Updated weights for policy 0, policy_version 1272666 (0.0009) [2023-12-27 00:39:20,432][105692] Updated weights for policy 0, policy_version 1272676 (0.0009) [2023-12-27 00:39:20,485][105692] Updated weights for policy 0, policy_version 1272686 (0.0009) [2023-12-27 00:39:20,782][105620] Updated weights for policy 1, policy_version 1274115 (0.0009) [2023-12-27 00:39:20,836][105620] Updated weights for policy 1, policy_version 1274125 (0.0009) [2023-12-27 00:39:20,891][105620] Updated weights for policy 1, policy_version 1274135 (0.0008) [2023-12-27 00:39:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 652083200. Throughput: 0: 9284.6, 1: 10063.8. Samples: 652072212. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:39:21,063][104569] Avg episode reward: [(0, '8993.814'), (1, '9262.894')] [2023-12-27 00:39:21,270][105692] Updated weights for policy 0, policy_version 1272696 (0.0009) [2023-12-27 00:39:21,336][105692] Updated weights for policy 0, policy_version 1272706 (0.0009) [2023-12-27 00:39:21,406][105692] Updated weights for policy 0, policy_version 1272716 (0.0009) [2023-12-27 00:39:21,691][105620] Updated weights for policy 1, policy_version 1274145 (0.0009) [2023-12-27 00:39:21,758][105620] Updated weights for policy 1, policy_version 1274155 (0.0009) [2023-12-27 00:39:21,821][105620] Updated weights for policy 1, policy_version 1274165 (0.0009) [2023-12-27 00:39:21,882][105620] Updated weights for policy 1, policy_version 1274175 (0.0009) [2023-12-27 00:39:22,146][105692] Updated weights for policy 0, policy_version 1272726 (0.0009) [2023-12-27 00:39:22,200][105692] Updated weights for policy 0, policy_version 1272736 (0.0010) [2023-12-27 00:39:22,257][105692] Updated weights for policy 0, policy_version 1272746 (0.0009) [2023-12-27 00:39:22,612][105620] Updated weights for policy 1, policy_version 1274185 (0.0009) [2023-12-27 00:39:22,665][105620] Updated weights for policy 1, policy_version 1274195 (0.0009) [2023-12-27 00:39:22,723][105620] Updated weights for policy 1, policy_version 1274205 (0.0010) [2023-12-27 00:39:23,056][105692] Updated weights for policy 0, policy_version 1272756 (0.0009) [2023-12-27 00:39:23,114][105692] Updated weights for policy 0, policy_version 1272766 (0.0010) [2023-12-27 00:39:23,181][105692] Updated weights for policy 0, policy_version 1272776 (0.0010) [2023-12-27 00:39:23,452][105620] Updated weights for policy 1, policy_version 1274215 (0.0009) [2023-12-27 00:39:23,513][105620] Updated weights for policy 1, policy_version 1274225 (0.0009) [2023-12-27 00:39:23,569][105620] Updated weights for policy 1, policy_version 1274235 (0.0009) [2023-12-27 00:39:23,984][105692] Updated weights for policy 0, policy_version 1272787 (0.0009) [2023-12-27 00:39:24,031][105692] Updated weights for policy 0, policy_version 1272797 (0.0009) [2023-12-27 00:39:24,118][105692] Updated weights for policy 0, policy_version 1272807 (0.0009) [2023-12-27 00:39:24,288][105620] Updated weights for policy 1, policy_version 1274245 (0.0010) [2023-12-27 00:39:24,339][105620] Updated weights for policy 1, policy_version 1274255 (0.0007) [2023-12-27 00:39:24,391][105620] Updated weights for policy 1, policy_version 1274265 (0.0010) [2023-12-27 00:39:24,862][105692] Updated weights for policy 0, policy_version 1272817 (0.0008) [2023-12-27 00:39:24,927][105692] Updated weights for policy 0, policy_version 1272827 (0.0009) [2023-12-27 00:39:24,998][105692] Updated weights for policy 0, policy_version 1272837 (0.0010) [2023-12-27 00:39:25,052][105620] Updated weights for policy 1, policy_version 1274275 (0.0009) [2023-12-27 00:39:25,052][105692] Updated weights for policy 0, policy_version 1272847 (0.0009) [2023-12-27 00:39:25,106][105620] Updated weights for policy 1, policy_version 1274285 (0.0006) [2023-12-27 00:39:25,157][105620] Updated weights for policy 1, policy_version 1274295 (0.0008) [2023-12-27 00:39:25,827][105620] Updated weights for policy 1, policy_version 1274305 (0.0009) [2023-12-27 00:39:25,861][105692] Updated weights for policy 0, policy_version 1272857 (0.0008) [2023-12-27 00:39:25,886][105620] Updated weights for policy 1, policy_version 1274315 (0.0007) [2023-12-27 00:39:25,924][105692] Updated weights for policy 0, policy_version 1272867 (0.0008) [2023-12-27 00:39:25,938][105620] Updated weights for policy 1, policy_version 1274325 (0.0008) [2023-12-27 00:39:25,983][105692] Updated weights for policy 0, policy_version 1272877 (0.0008) [2023-12-27 00:39:25,997][105620] Updated weights for policy 1, policy_version 1274335 (0.0007) [2023-12-27 00:39:26,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 652181504. Throughput: 0: 9336.6, 1: 9984.2. Samples: 652184316. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:39:26,063][104569] Avg episode reward: [(0, '9083.216'), (1, '9079.738')] [2023-12-27 00:39:26,740][105620] Updated weights for policy 1, policy_version 1274345 (0.0008) [2023-12-27 00:39:26,741][105692] Updated weights for policy 0, policy_version 1272887 (0.0008) [2023-12-27 00:39:26,797][105692] Updated weights for policy 0, policy_version 1272897 (0.0007) [2023-12-27 00:39:26,798][105620] Updated weights for policy 1, policy_version 1274355 (0.0006) [2023-12-27 00:39:26,850][105692] Updated weights for policy 0, policy_version 1272907 (0.0009) [2023-12-27 00:39:26,855][105620] Updated weights for policy 1, policy_version 1274365 (0.0005) [2023-12-27 00:39:27,549][105620] Updated weights for policy 1, policy_version 1274375 (0.0007) [2023-12-27 00:39:27,587][105692] Updated weights for policy 0, policy_version 1272917 (0.0009) [2023-12-27 00:39:27,605][105620] Updated weights for policy 1, policy_version 1274385 (0.0008) [2023-12-27 00:39:27,635][105692] Updated weights for policy 0, policy_version 1272927 (0.0006) [2023-12-27 00:39:27,662][105620] Updated weights for policy 1, policy_version 1274395 (0.0008) [2023-12-27 00:39:27,677][105692] Updated weights for policy 0, policy_version 1272937 (0.0007) [2023-12-27 00:39:28,395][105620] Updated weights for policy 1, policy_version 1274405 (0.0009) [2023-12-27 00:39:28,436][105692] Updated weights for policy 0, policy_version 1272947 (0.0008) [2023-12-27 00:39:28,454][105620] Updated weights for policy 1, policy_version 1274415 (0.0008) [2023-12-27 00:39:28,485][105692] Updated weights for policy 0, policy_version 1272957 (0.0006) [2023-12-27 00:39:28,503][105620] Updated weights for policy 1, policy_version 1274425 (0.0008) [2023-12-27 00:39:28,536][105692] Updated weights for policy 0, policy_version 1272967 (0.0007) [2023-12-27 00:39:29,230][105620] Updated weights for policy 1, policy_version 1274435 (0.0006) [2023-12-27 00:39:29,293][105620] Updated weights for policy 1, policy_version 1274445 (0.0008) [2023-12-27 00:39:29,326][105692] Updated weights for policy 0, policy_version 1272977 (0.0009) [2023-12-27 00:39:29,365][105620] Updated weights for policy 1, policy_version 1274455 (0.0007) [2023-12-27 00:39:29,387][105692] Updated weights for policy 0, policy_version 1272987 (0.0008) [2023-12-27 00:39:29,446][105692] Updated weights for policy 0, policy_version 1272997 (0.0008) [2023-12-27 00:39:29,504][105692] Updated weights for policy 0, policy_version 1273007 (0.0008) [2023-12-27 00:39:29,999][105620] Updated weights for policy 1, policy_version 1274465 (0.0006) [2023-12-27 00:39:30,047][105620] Updated weights for policy 1, policy_version 1274475 (0.0005) [2023-12-27 00:39:30,096][105620] Updated weights for policy 1, policy_version 1274485 (0.0005) [2023-12-27 00:39:30,144][105620] Updated weights for policy 1, policy_version 1274495 (0.0005) [2023-12-27 00:39:30,272][105692] Updated weights for policy 0, policy_version 1273017 (0.0009) [2023-12-27 00:39:30,319][105692] Updated weights for policy 0, policy_version 1273027 (0.0009) [2023-12-27 00:39:30,369][105692] Updated weights for policy 0, policy_version 1273037 (0.0008) [2023-12-27 00:39:30,816][105620] Updated weights for policy 1, policy_version 1274505 (0.0008) [2023-12-27 00:39:30,863][105620] Updated weights for policy 1, policy_version 1274515 (0.0009) [2023-12-27 00:39:30,909][105620] Updated weights for policy 1, policy_version 1274525 (0.0008) [2023-12-27 00:39:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 652271616. Throughput: 0: 9335.4, 1: 10007.4. Samples: 652241532. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:39:31,062][104569] Avg episode reward: [(0, '9083.675'), (1, '8631.923')] [2023-12-27 00:39:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001273040_325951488.pth... [2023-12-27 00:39:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001274528_326320128.pth... [2023-12-27 00:39:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001273376_326025216.pth [2023-12-27 00:39:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001271984_325681152.pth [2023-12-27 00:39:31,140][105692] Updated weights for policy 0, policy_version 1273047 (0.0009) [2023-12-27 00:39:31,200][105692] Updated weights for policy 0, policy_version 1273057 (0.0009) [2023-12-27 00:39:31,263][105692] Updated weights for policy 0, policy_version 1273067 (0.0009) [2023-12-27 00:39:31,710][105620] Updated weights for policy 1, policy_version 1274535 (0.0010) [2023-12-27 00:39:31,771][105620] Updated weights for policy 1, policy_version 1274545 (0.0011) [2023-12-27 00:39:31,833][105620] Updated weights for policy 1, policy_version 1274555 (0.0010) [2023-12-27 00:39:32,002][105692] Updated weights for policy 0, policy_version 1273077 (0.0009) [2023-12-27 00:39:32,060][105692] Updated weights for policy 0, policy_version 1273087 (0.0009) [2023-12-27 00:39:32,115][105692] Updated weights for policy 0, policy_version 1273097 (0.0006) [2023-12-27 00:39:32,546][105620] Updated weights for policy 1, policy_version 1274565 (0.0008) [2023-12-27 00:39:32,603][105620] Updated weights for policy 1, policy_version 1274575 (0.0009) [2023-12-27 00:39:32,659][105620] Updated weights for policy 1, policy_version 1274585 (0.0011) [2023-12-27 00:39:32,847][105692] Updated weights for policy 0, policy_version 1273107 (0.0007) [2023-12-27 00:39:32,904][105692] Updated weights for policy 0, policy_version 1273117 (0.0010) [2023-12-27 00:39:32,955][105692] Updated weights for policy 0, policy_version 1273127 (0.0010) [2023-12-27 00:39:33,203][105620] Updated weights for policy 1, policy_version 1274595 (0.0009) [2023-12-27 00:39:33,252][105620] Updated weights for policy 1, policy_version 1274605 (0.0005) [2023-12-27 00:39:33,303][105620] Updated weights for policy 1, policy_version 1274615 (0.0005) [2023-12-27 00:39:33,751][105692] Updated weights for policy 0, policy_version 1273137 (0.0009) [2023-12-27 00:39:33,802][105692] Updated weights for policy 0, policy_version 1273147 (0.0009) [2023-12-27 00:39:33,853][105692] Updated weights for policy 0, policy_version 1273157 (0.0009) [2023-12-27 00:39:33,899][105692] Updated weights for policy 0, policy_version 1273167 (0.0009) [2023-12-27 00:39:33,961][105620] Updated weights for policy 1, policy_version 1274625 (0.0006) [2023-12-27 00:39:34,018][105620] Updated weights for policy 1, policy_version 1274635 (0.0009) [2023-12-27 00:39:34,065][105620] Updated weights for policy 1, policy_version 1274645 (0.0008) [2023-12-27 00:39:34,115][105620] Updated weights for policy 1, policy_version 1274655 (0.0010) [2023-12-27 00:39:34,598][105692] Updated weights for policy 0, policy_version 1273177 (0.0006) [2023-12-27 00:39:34,665][105692] Updated weights for policy 0, policy_version 1273187 (0.0006) [2023-12-27 00:39:34,721][105692] Updated weights for policy 0, policy_version 1273197 (0.0007) [2023-12-27 00:39:34,871][105620] Updated weights for policy 1, policy_version 1274665 (0.0006) [2023-12-27 00:39:34,916][105620] Updated weights for policy 1, policy_version 1274675 (0.0005) [2023-12-27 00:39:34,974][105620] Updated weights for policy 1, policy_version 1274685 (0.0005) [2023-12-27 00:39:35,248][105692] Updated weights for policy 0, policy_version 1273207 (0.0005) [2023-12-27 00:39:35,299][105692] Updated weights for policy 0, policy_version 1273217 (0.0005) [2023-12-27 00:39:35,360][105692] Updated weights for policy 0, policy_version 1273227 (0.0009) [2023-12-27 00:39:35,537][105620] Updated weights for policy 1, policy_version 1274695 (0.0007) [2023-12-27 00:39:35,585][105620] Updated weights for policy 1, policy_version 1274705 (0.0008) [2023-12-27 00:39:35,633][105620] Updated weights for policy 1, policy_version 1274715 (0.0008) [2023-12-27 00:39:36,054][105692] Updated weights for policy 0, policy_version 1273237 (0.0010) [2023-12-27 00:39:36,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.1, 300 sec: 19494.2). Total num frames: 652369920. Throughput: 0: 9432.2, 1: 9964.0. Samples: 652359120. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:39:36,063][104569] Avg episode reward: [(0, '9082.697'), (1, '8825.399')] [2023-12-27 00:39:36,106][105692] Updated weights for policy 0, policy_version 1273247 (0.0010) [2023-12-27 00:39:36,172][105692] Updated weights for policy 0, policy_version 1273257 (0.0011) [2023-12-27 00:39:36,410][105620] Updated weights for policy 1, policy_version 1274725 (0.0008) [2023-12-27 00:39:36,474][105620] Updated weights for policy 1, policy_version 1274735 (0.0009) [2023-12-27 00:39:36,531][105620] Updated weights for policy 1, policy_version 1274745 (0.0008) [2023-12-27 00:39:36,900][105692] Updated weights for policy 0, policy_version 1273267 (0.0010) [2023-12-27 00:39:36,965][105692] Updated weights for policy 0, policy_version 1273277 (0.0010) [2023-12-27 00:39:37,031][105692] Updated weights for policy 0, policy_version 1273287 (0.0010) [2023-12-27 00:39:37,142][105620] Updated weights for policy 1, policy_version 1274755 (0.0008) [2023-12-27 00:39:37,191][105620] Updated weights for policy 1, policy_version 1274765 (0.0005) [2023-12-27 00:39:37,249][105620] Updated weights for policy 1, policy_version 1274775 (0.0006) [2023-12-27 00:39:37,630][105692] Updated weights for policy 0, policy_version 1273297 (0.0009) [2023-12-27 00:39:37,695][105692] Updated weights for policy 0, policy_version 1273307 (0.0008) [2023-12-27 00:39:37,755][105692] Updated weights for policy 0, policy_version 1273317 (0.0009) [2023-12-27 00:39:37,811][105692] Updated weights for policy 0, policy_version 1273327 (0.0010) [2023-12-27 00:39:37,930][105620] Updated weights for policy 1, policy_version 1274785 (0.0007) [2023-12-27 00:39:37,989][105620] Updated weights for policy 1, policy_version 1274795 (0.0011) [2023-12-27 00:39:38,048][105620] Updated weights for policy 1, policy_version 1274805 (0.0011) [2023-12-27 00:39:38,096][105620] Updated weights for policy 1, policy_version 1274815 (0.0010) [2023-12-27 00:39:38,596][105692] Updated weights for policy 0, policy_version 1273337 (0.0008) [2023-12-27 00:39:38,660][105692] Updated weights for policy 0, policy_version 1273347 (0.0005) [2023-12-27 00:39:38,720][105692] Updated weights for policy 0, policy_version 1273357 (0.0008) [2023-12-27 00:39:38,778][105620] Updated weights for policy 1, policy_version 1274825 (0.0010) [2023-12-27 00:39:38,840][105620] Updated weights for policy 1, policy_version 1274835 (0.0010) [2023-12-27 00:39:38,900][105620] Updated weights for policy 1, policy_version 1274845 (0.0010) [2023-12-27 00:39:39,450][105692] Updated weights for policy 0, policy_version 1273367 (0.0007) [2023-12-27 00:39:39,512][105692] Updated weights for policy 0, policy_version 1273377 (0.0008) [2023-12-27 00:39:39,574][105692] Updated weights for policy 0, policy_version 1273387 (0.0008) [2023-12-27 00:39:39,688][105620] Updated weights for policy 1, policy_version 1274855 (0.0010) [2023-12-27 00:39:39,755][105620] Updated weights for policy 1, policy_version 1274865 (0.0011) [2023-12-27 00:39:39,818][105620] Updated weights for policy 1, policy_version 1274875 (0.0011) [2023-12-27 00:39:40,400][105692] Updated weights for policy 0, policy_version 1273397 (0.0008) [2023-12-27 00:39:40,441][105620] Updated weights for policy 1, policy_version 1274885 (0.0009) [2023-12-27 00:39:40,457][105692] Updated weights for policy 0, policy_version 1273407 (0.0007) [2023-12-27 00:39:40,458][105585] KL-divergence is very high: 123.6018 [2023-12-27 00:39:40,506][105620] Updated weights for policy 1, policy_version 1274895 (0.0008) [2023-12-27 00:39:40,507][105585] KL-divergence is very high: 206.7507 [2023-12-27 00:39:40,518][105692] Updated weights for policy 0, policy_version 1273417 (0.0010) [2023-12-27 00:39:40,552][105585] KL-divergence is very high: 234.8530 [2023-12-27 00:39:40,562][105620] Updated weights for policy 1, policy_version 1274905 (0.0011) [2023-12-27 00:39:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 652468224. Throughput: 0: 9417.0, 1: 9989.1. Samples: 652479040. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:39:41,062][104569] Avg episode reward: [(0, '9088.466'), (1, '9090.782')] [2023-12-27 00:39:41,229][105620] Updated weights for policy 1, policy_version 1274915 (0.0010) [2023-12-27 00:39:41,292][105620] Updated weights for policy 1, policy_version 1274925 (0.0006) [2023-12-27 00:39:41,346][105692] Updated weights for policy 0, policy_version 1273427 (0.0010) [2023-12-27 00:39:41,356][105620] Updated weights for policy 1, policy_version 1274935 (0.0008) [2023-12-27 00:39:41,411][105692] Updated weights for policy 0, policy_version 1273437 (0.0008) [2023-12-27 00:39:41,469][105692] Updated weights for policy 0, policy_version 1273447 (0.0008) [2023-12-27 00:39:42,154][105620] Updated weights for policy 1, policy_version 1274945 (0.0008) [2023-12-27 00:39:42,218][105620] Updated weights for policy 1, policy_version 1274955 (0.0009) [2023-12-27 00:39:42,246][105692] Updated weights for policy 0, policy_version 1273457 (0.0008) [2023-12-27 00:39:42,280][105620] Updated weights for policy 1, policy_version 1274965 (0.0008) [2023-12-27 00:39:42,306][105692] Updated weights for policy 0, policy_version 1273467 (0.0009) [2023-12-27 00:39:42,347][105620] Updated weights for policy 1, policy_version 1274975 (0.0007) [2023-12-27 00:39:42,369][105692] Updated weights for policy 0, policy_version 1273477 (0.0008) [2023-12-27 00:39:42,427][105692] Updated weights for policy 0, policy_version 1273487 (0.0009) [2023-12-27 00:39:43,091][105620] Updated weights for policy 1, policy_version 1274985 (0.0006) [2023-12-27 00:39:43,149][105620] Updated weights for policy 1, policy_version 1274995 (0.0007) [2023-12-27 00:39:43,152][105692] Updated weights for policy 0, policy_version 1273497 (0.0010) [2023-12-27 00:39:43,210][105620] Updated weights for policy 1, policy_version 1275005 (0.0005) [2023-12-27 00:39:43,212][105692] Updated weights for policy 0, policy_version 1273507 (0.0010) [2023-12-27 00:39:43,271][105692] Updated weights for policy 0, policy_version 1273517 (0.0010) [2023-12-27 00:39:43,935][105692] Updated weights for policy 0, policy_version 1273527 (0.0007) [2023-12-27 00:39:43,959][105620] Updated weights for policy 1, policy_version 1275015 (0.0005) [2023-12-27 00:39:43,994][105692] Updated weights for policy 0, policy_version 1273537 (0.0005) [2023-12-27 00:39:44,009][105620] Updated weights for policy 1, policy_version 1275025 (0.0009) [2023-12-27 00:39:44,057][105692] Updated weights for policy 0, policy_version 1273547 (0.0005) [2023-12-27 00:39:44,059][105620] Updated weights for policy 1, policy_version 1275035 (0.0009) [2023-12-27 00:39:44,682][105692] Updated weights for policy 0, policy_version 1273557 (0.0008) [2023-12-27 00:39:44,730][105692] Updated weights for policy 0, policy_version 1273567 (0.0011) [2023-12-27 00:39:44,732][105620] Updated weights for policy 1, policy_version 1275045 (0.0006) [2023-12-27 00:39:44,791][105692] Updated weights for policy 0, policy_version 1273577 (0.0011) [2023-12-27 00:39:44,801][105620] Updated weights for policy 1, policy_version 1275055 (0.0007) [2023-12-27 00:39:44,865][105620] Updated weights for policy 1, policy_version 1275065 (0.0008) [2023-12-27 00:39:45,575][105692] Updated weights for policy 0, policy_version 1273587 (0.0010) [2023-12-27 00:39:45,598][105620] Updated weights for policy 1, policy_version 1275075 (0.0007) [2023-12-27 00:39:45,624][105692] Updated weights for policy 0, policy_version 1273597 (0.0011) [2023-12-27 00:39:45,650][105620] Updated weights for policy 1, policy_version 1275085 (0.0005) [2023-12-27 00:39:45,685][105692] Updated weights for policy 0, policy_version 1273607 (0.0009) [2023-12-27 00:39:45,705][105620] Updated weights for policy 1, policy_version 1275095 (0.0005) [2023-12-27 00:39:46,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.8, 300 sec: 19521.9). Total num frames: 652566528. Throughput: 0: 9353.7, 1: 9905.1. Samples: 652534692. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:39:46,062][104569] Avg episode reward: [(0, '8657.178'), (1, '8818.394')] [2023-12-27 00:39:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001275104_326467584.pth... [2023-12-27 00:39:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001273616_326098944.pth... [2023-12-27 00:39:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001272528_325820416.pth [2023-12-27 00:39:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001273952_326172672.pth [2023-12-27 00:39:46,275][105692] Updated weights for policy 0, policy_version 1273617 (0.0006) [2023-12-27 00:39:46,333][105692] Updated weights for policy 0, policy_version 1273627 (0.0009) [2023-12-27 00:39:46,372][105620] Updated weights for policy 1, policy_version 1275105 (0.0006) [2023-12-27 00:39:46,384][105692] Updated weights for policy 0, policy_version 1273637 (0.0009) [2023-12-27 00:39:46,430][105620] Updated weights for policy 1, policy_version 1275115 (0.0006) [2023-12-27 00:39:46,451][105692] Updated weights for policy 0, policy_version 1273647 (0.0007) [2023-12-27 00:39:46,486][105620] Updated weights for policy 1, policy_version 1275125 (0.0005) [2023-12-27 00:39:46,539][105620] Updated weights for policy 1, policy_version 1275135 (0.0005) [2023-12-27 00:39:47,150][105692] Updated weights for policy 0, policy_version 1273657 (0.0010) [2023-12-27 00:39:47,181][105620] Updated weights for policy 1, policy_version 1275145 (0.0006) [2023-12-27 00:39:47,202][105692] Updated weights for policy 0, policy_version 1273667 (0.0010) [2023-12-27 00:39:47,233][105620] Updated weights for policy 1, policy_version 1275155 (0.0005) [2023-12-27 00:39:47,251][105692] Updated weights for policy 0, policy_version 1273677 (0.0010) [2023-12-27 00:39:47,287][105620] Updated weights for policy 1, policy_version 1275165 (0.0006) [2023-12-27 00:39:47,903][105620] Updated weights for policy 1, policy_version 1275175 (0.0007) [2023-12-27 00:39:47,934][105692] Updated weights for policy 0, policy_version 1273687 (0.0010) [2023-12-27 00:39:47,952][105620] Updated weights for policy 1, policy_version 1275185 (0.0005) [2023-12-27 00:39:47,993][105692] Updated weights for policy 0, policy_version 1273697 (0.0009) [2023-12-27 00:39:48,011][105620] Updated weights for policy 1, policy_version 1275195 (0.0008) [2023-12-27 00:39:48,042][105692] Updated weights for policy 0, policy_version 1273707 (0.0008) [2023-12-27 00:39:48,657][105620] Updated weights for policy 1, policy_version 1275205 (0.0006) [2023-12-27 00:39:48,721][105620] Updated weights for policy 1, policy_version 1275215 (0.0008) [2023-12-27 00:39:48,759][105692] Updated weights for policy 0, policy_version 1273717 (0.0010) [2023-12-27 00:39:48,785][105620] Updated weights for policy 1, policy_version 1275225 (0.0006) [2023-12-27 00:39:48,822][105692] Updated weights for policy 0, policy_version 1273727 (0.0010) [2023-12-27 00:39:48,890][105692] Updated weights for policy 0, policy_version 1273737 (0.0006) [2023-12-27 00:39:49,509][105692] Updated weights for policy 0, policy_version 1273747 (0.0006) [2023-12-27 00:39:49,550][105620] Updated weights for policy 1, policy_version 1275235 (0.0006) [2023-12-27 00:39:49,568][105692] Updated weights for policy 0, policy_version 1273757 (0.0007) [2023-12-27 00:39:49,616][105620] Updated weights for policy 1, policy_version 1275245 (0.0008) [2023-12-27 00:39:49,632][105692] Updated weights for policy 0, policy_version 1273767 (0.0007) [2023-12-27 00:39:49,680][105620] Updated weights for policy 1, policy_version 1275255 (0.0008) [2023-12-27 00:39:50,286][105620] Updated weights for policy 1, policy_version 1275265 (0.0008) [2023-12-27 00:39:50,345][105620] Updated weights for policy 1, policy_version 1275275 (0.0006) [2023-12-27 00:39:50,403][105620] Updated weights for policy 1, policy_version 1275285 (0.0008) [2023-12-27 00:39:50,427][105692] Updated weights for policy 0, policy_version 1273777 (0.0007) [2023-12-27 00:39:50,457][105620] Updated weights for policy 1, policy_version 1275295 (0.0007) [2023-12-27 00:39:50,492][105692] Updated weights for policy 0, policy_version 1273787 (0.0008) [2023-12-27 00:39:50,558][105692] Updated weights for policy 0, policy_version 1273797 (0.0009) [2023-12-27 00:39:50,619][105692] Updated weights for policy 0, policy_version 1273807 (0.0009) [2023-12-27 00:39:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 652664832. Throughput: 0: 9450.0, 1: 9975.3. Samples: 652657256. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:39:51,062][104569] Avg episode reward: [(0, '8571.425'), (1, '8818.356')] [2023-12-27 00:39:51,134][105620] Updated weights for policy 1, policy_version 1275305 (0.0009) [2023-12-27 00:39:51,185][105620] Updated weights for policy 1, policy_version 1275315 (0.0009) [2023-12-27 00:39:51,247][105620] Updated weights for policy 1, policy_version 1275325 (0.0009) [2023-12-27 00:39:51,415][105692] Updated weights for policy 0, policy_version 1273817 (0.0006) [2023-12-27 00:39:51,466][105692] Updated weights for policy 0, policy_version 1273827 (0.0005) [2023-12-27 00:39:51,521][105692] Updated weights for policy 0, policy_version 1273837 (0.0008) [2023-12-27 00:39:52,019][105620] Updated weights for policy 1, policy_version 1275335 (0.0009) [2023-12-27 00:39:52,077][105620] Updated weights for policy 1, policy_version 1275345 (0.0009) [2023-12-27 00:39:52,136][105620] Updated weights for policy 1, policy_version 1275355 (0.0007) [2023-12-27 00:39:52,272][105692] Updated weights for policy 0, policy_version 1273847 (0.0009) [2023-12-27 00:39:52,329][105692] Updated weights for policy 0, policy_version 1273858 (0.0010) [2023-12-27 00:39:52,394][105692] Updated weights for policy 0, policy_version 1273868 (0.0008) [2023-12-27 00:39:52,873][105620] Updated weights for policy 1, policy_version 1275365 (0.0007) [2023-12-27 00:39:52,928][105620] Updated weights for policy 1, policy_version 1275375 (0.0009) [2023-12-27 00:39:52,990][105620] Updated weights for policy 1, policy_version 1275385 (0.0009) [2023-12-27 00:39:53,189][105692] Updated weights for policy 0, policy_version 1273878 (0.0009) [2023-12-27 00:39:53,239][105692] Updated weights for policy 0, policy_version 1273888 (0.0008) [2023-12-27 00:39:53,290][105692] Updated weights for policy 0, policy_version 1273898 (0.0009) [2023-12-27 00:39:53,742][105620] Updated weights for policy 1, policy_version 1275395 (0.0009) [2023-12-27 00:39:53,802][105620] Updated weights for policy 1, policy_version 1275405 (0.0009) [2023-12-27 00:39:53,851][105620] Updated weights for policy 1, policy_version 1275415 (0.0009) [2023-12-27 00:39:54,027][105692] Updated weights for policy 0, policy_version 1273908 (0.0008) [2023-12-27 00:39:54,071][105692] Updated weights for policy 0, policy_version 1273918 (0.0008) [2023-12-27 00:39:54,120][105692] Updated weights for policy 0, policy_version 1273928 (0.0009) [2023-12-27 00:39:54,488][105620] Updated weights for policy 1, policy_version 1275425 (0.0009) [2023-12-27 00:39:54,543][105620] Updated weights for policy 1, policy_version 1275435 (0.0009) [2023-12-27 00:39:54,601][105620] Updated weights for policy 1, policy_version 1275445 (0.0009) [2023-12-27 00:39:54,657][105620] Updated weights for policy 1, policy_version 1275455 (0.0010) [2023-12-27 00:39:54,837][105692] Updated weights for policy 0, policy_version 1273938 (0.0009) [2023-12-27 00:39:54,891][105692] Updated weights for policy 0, policy_version 1273948 (0.0010) [2023-12-27 00:39:54,944][105692] Updated weights for policy 0, policy_version 1273958 (0.0010) [2023-12-27 00:39:54,997][105692] Updated weights for policy 0, policy_version 1273968 (0.0009) [2023-12-27 00:39:55,397][105620] Updated weights for policy 1, policy_version 1275465 (0.0010) [2023-12-27 00:39:55,455][105620] Updated weights for policy 1, policy_version 1275475 (0.0008) [2023-12-27 00:39:55,518][105620] Updated weights for policy 1, policy_version 1275485 (0.0008) [2023-12-27 00:39:55,761][105692] Updated weights for policy 0, policy_version 1273978 (0.0005) [2023-12-27 00:39:55,805][105692] Updated weights for policy 0, policy_version 1273988 (0.0005) [2023-12-27 00:39:55,857][105692] Updated weights for policy 0, policy_version 1273998 (0.0007) [2023-12-27 00:39:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 652763136. Throughput: 0: 9430.5, 1: 9939.9. Samples: 652772008. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:39:56,062][104569] Avg episode reward: [(0, '8912.254'), (1, '8989.421')] [2023-12-27 00:39:56,216][105620] Updated weights for policy 1, policy_version 1275495 (0.0006) [2023-12-27 00:39:56,264][105620] Updated weights for policy 1, policy_version 1275505 (0.0007) [2023-12-27 00:39:56,312][105620] Updated weights for policy 1, policy_version 1275515 (0.0010) [2023-12-27 00:39:56,477][105692] Updated weights for policy 0, policy_version 1274008 (0.0006) [2023-12-27 00:39:56,530][105692] Updated weights for policy 0, policy_version 1274018 (0.0005) [2023-12-27 00:39:56,577][105692] Updated weights for policy 0, policy_version 1274028 (0.0005) [2023-12-27 00:39:56,952][105620] Updated weights for policy 1, policy_version 1275525 (0.0010) [2023-12-27 00:39:57,009][105620] Updated weights for policy 1, policy_version 1275535 (0.0010) [2023-12-27 00:39:57,070][105620] Updated weights for policy 1, policy_version 1275545 (0.0007) [2023-12-27 00:39:57,158][105692] Updated weights for policy 0, policy_version 1274038 (0.0008) [2023-12-27 00:39:57,213][105692] Updated weights for policy 0, policy_version 1274048 (0.0010) [2023-12-27 00:39:57,264][105692] Updated weights for policy 0, policy_version 1274058 (0.0010) [2023-12-27 00:39:57,806][105620] Updated weights for policy 1, policy_version 1275555 (0.0008) [2023-12-27 00:39:57,851][105620] Updated weights for policy 1, policy_version 1275565 (0.0008) [2023-12-27 00:39:57,903][105620] Updated weights for policy 1, policy_version 1275575 (0.0009) [2023-12-27 00:39:57,940][105692] Updated weights for policy 0, policy_version 1274068 (0.0008) [2023-12-27 00:39:57,991][105692] Updated weights for policy 0, policy_version 1274078 (0.0009) [2023-12-27 00:39:58,040][105692] Updated weights for policy 0, policy_version 1274088 (0.0010) [2023-12-27 00:39:58,648][105620] Updated weights for policy 1, policy_version 1275585 (0.0010) [2023-12-27 00:39:58,707][105620] Updated weights for policy 1, policy_version 1275595 (0.0009) [2023-12-27 00:39:58,781][105620] Updated weights for policy 1, policy_version 1275605 (0.0009) [2023-12-27 00:39:58,844][105620] Updated weights for policy 1, policy_version 1275615 (0.0008) [2023-12-27 00:39:58,849][105692] Updated weights for policy 0, policy_version 1274098 (0.0010) [2023-12-27 00:39:58,910][105692] Updated weights for policy 0, policy_version 1274108 (0.0008) [2023-12-27 00:39:58,984][105692] Updated weights for policy 0, policy_version 1274118 (0.0007) [2023-12-27 00:39:59,048][105692] Updated weights for policy 0, policy_version 1274128 (0.0008) [2023-12-27 00:39:59,664][105620] Updated weights for policy 1, policy_version 1275625 (0.0009) [2023-12-27 00:39:59,715][105620] Updated weights for policy 1, policy_version 1275635 (0.0008) [2023-12-27 00:39:59,762][105620] Updated weights for policy 1, policy_version 1275645 (0.0009) [2023-12-27 00:39:59,778][105692] Updated weights for policy 0, policy_version 1274138 (0.0008) [2023-12-27 00:39:59,838][105692] Updated weights for policy 0, policy_version 1274148 (0.0009) [2023-12-27 00:39:59,894][105692] Updated weights for policy 0, policy_version 1274158 (0.0009) [2023-12-27 00:40:00,488][105620] Updated weights for policy 1, policy_version 1275655 (0.0009) [2023-12-27 00:40:00,533][105620] Updated weights for policy 1, policy_version 1275665 (0.0006) [2023-12-27 00:40:00,579][105620] Updated weights for policy 1, policy_version 1275675 (0.0005) [2023-12-27 00:40:00,672][105692] Updated weights for policy 0, policy_version 1274168 (0.0009) [2023-12-27 00:40:00,727][105692] Updated weights for policy 0, policy_version 1274178 (0.0008) [2023-12-27 00:40:00,789][105692] Updated weights for policy 0, policy_version 1274188 (0.0008) [2023-12-27 00:40:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 652861440. Throughput: 0: 9522.3, 1: 9933.5. Samples: 652832840. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:40:01,062][104569] Avg episode reward: [(0, '9084.646'), (1, '8988.903')] [2023-12-27 00:40:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001274192_326246400.pth... [2023-12-27 00:40:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001275680_326615040.pth... [2023-12-27 00:40:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001274528_326320128.pth [2023-12-27 00:40:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001273040_325951488.pth [2023-12-27 00:40:01,178][105620] Updated weights for policy 1, policy_version 1275685 (0.0007) [2023-12-27 00:40:01,248][105620] Updated weights for policy 1, policy_version 1275695 (0.0010) [2023-12-27 00:40:01,311][105620] Updated weights for policy 1, policy_version 1275705 (0.0009) [2023-12-27 00:40:01,514][105692] Updated weights for policy 0, policy_version 1274198 (0.0010) [2023-12-27 00:40:01,561][105692] Updated weights for policy 0, policy_version 1274208 (0.0007) [2023-12-27 00:40:01,611][105692] Updated weights for policy 0, policy_version 1274218 (0.0005) [2023-12-27 00:40:02,003][105620] Updated weights for policy 1, policy_version 1275715 (0.0009) [2023-12-27 00:40:02,070][105620] Updated weights for policy 1, policy_version 1275725 (0.0009) [2023-12-27 00:40:02,127][105620] Updated weights for policy 1, policy_version 1275735 (0.0006) [2023-12-27 00:40:02,355][105692] Updated weights for policy 0, policy_version 1274228 (0.0009) [2023-12-27 00:40:02,419][105692] Updated weights for policy 0, policy_version 1274238 (0.0006) [2023-12-27 00:40:02,478][105692] Updated weights for policy 0, policy_version 1274248 (0.0006) [2023-12-27 00:40:02,836][105620] Updated weights for policy 1, policy_version 1275745 (0.0008) [2023-12-27 00:40:02,900][105620] Updated weights for policy 1, policy_version 1275755 (0.0005) [2023-12-27 00:40:02,960][105620] Updated weights for policy 1, policy_version 1275765 (0.0005) [2023-12-27 00:40:03,012][105620] Updated weights for policy 1, policy_version 1275775 (0.0005) [2023-12-27 00:40:03,060][105692] Updated weights for policy 0, policy_version 1274258 (0.0006) [2023-12-27 00:40:03,121][105692] Updated weights for policy 0, policy_version 1274268 (0.0008) [2023-12-27 00:40:03,176][105692] Updated weights for policy 0, policy_version 1274278 (0.0005) [2023-12-27 00:40:03,232][105692] Updated weights for policy 0, policy_version 1274288 (0.0005) [2023-12-27 00:40:03,697][105620] Updated weights for policy 1, policy_version 1275787 (0.0009) [2023-12-27 00:40:03,754][105620] Updated weights for policy 1, policy_version 1275797 (0.0008) [2023-12-27 00:40:03,809][105620] Updated weights for policy 1, policy_version 1275807 (0.0009) [2023-12-27 00:40:03,903][105692] Updated weights for policy 0, policy_version 1274298 (0.0011) [2023-12-27 00:40:03,968][105692] Updated weights for policy 0, policy_version 1274308 (0.0010) [2023-12-27 00:40:04,023][105692] Updated weights for policy 0, policy_version 1274318 (0.0009) [2023-12-27 00:40:04,495][105620] Updated weights for policy 1, policy_version 1275817 (0.0008) [2023-12-27 00:40:04,551][105620] Updated weights for policy 1, policy_version 1275827 (0.0008) [2023-12-27 00:40:04,611][105620] Updated weights for policy 1, policy_version 1275837 (0.0008) [2023-12-27 00:40:04,737][105692] Updated weights for policy 0, policy_version 1274328 (0.0009) [2023-12-27 00:40:04,788][105692] Updated weights for policy 0, policy_version 1274338 (0.0010) [2023-12-27 00:40:04,831][105692] Updated weights for policy 0, policy_version 1274348 (0.0008) [2023-12-27 00:40:05,389][105692] Updated weights for policy 0, policy_version 1274358 (0.0005) [2023-12-27 00:40:05,437][105692] Updated weights for policy 0, policy_version 1274368 (0.0005) [2023-12-27 00:40:05,484][105620] Updated weights for policy 1, policy_version 1275847 (0.0008) [2023-12-27 00:40:05,493][105692] Updated weights for policy 0, policy_version 1274378 (0.0005) [2023-12-27 00:40:05,534][105620] Updated weights for policy 1, policy_version 1275857 (0.0009) [2023-12-27 00:40:05,599][105620] Updated weights for policy 1, policy_version 1275867 (0.0008) [2023-12-27 00:40:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 652959744. Throughput: 0: 9615.7, 1: 9904.2. Samples: 652950604. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:40:06,062][104569] Avg episode reward: [(0, '8912.258'), (1, '9170.722')] [2023-12-27 00:40:06,067][105692] Updated weights for policy 0, policy_version 1274388 (0.0007) [2023-12-27 00:40:06,131][105692] Updated weights for policy 0, policy_version 1274398 (0.0006) [2023-12-27 00:40:06,202][105692] Updated weights for policy 0, policy_version 1274408 (0.0010) [2023-12-27 00:40:06,341][105620] Updated weights for policy 1, policy_version 1275877 (0.0007) [2023-12-27 00:40:06,397][105620] Updated weights for policy 1, policy_version 1275887 (0.0009) [2023-12-27 00:40:06,459][105620] Updated weights for policy 1, policy_version 1275897 (0.0007) [2023-12-27 00:40:06,850][105692] Updated weights for policy 0, policy_version 1274418 (0.0009) [2023-12-27 00:40:06,898][105692] Updated weights for policy 0, policy_version 1274428 (0.0005) [2023-12-27 00:40:06,949][105692] Updated weights for policy 0, policy_version 1274438 (0.0008) [2023-12-27 00:40:07,012][105692] Updated weights for policy 0, policy_version 1274448 (0.0009) [2023-12-27 00:40:07,257][105620] Updated weights for policy 1, policy_version 1275907 (0.0008) [2023-12-27 00:40:07,318][105620] Updated weights for policy 1, policy_version 1275917 (0.0005) [2023-12-27 00:40:07,371][105620] Updated weights for policy 1, policy_version 1275927 (0.0005) [2023-12-27 00:40:07,745][105692] Updated weights for policy 0, policy_version 1274458 (0.0009) [2023-12-27 00:40:07,796][105692] Updated weights for policy 0, policy_version 1274468 (0.0009) [2023-12-27 00:40:07,860][105692] Updated weights for policy 0, policy_version 1274478 (0.0009) [2023-12-27 00:40:08,065][105620] Updated weights for policy 1, policy_version 1275937 (0.0008) [2023-12-27 00:40:08,130][105620] Updated weights for policy 1, policy_version 1275947 (0.0009) [2023-12-27 00:40:08,193][105620] Updated weights for policy 1, policy_version 1275957 (0.0011) [2023-12-27 00:40:08,250][105620] Updated weights for policy 1, policy_version 1275967 (0.0011) [2023-12-27 00:40:08,675][105692] Updated weights for policy 0, policy_version 1274488 (0.0009) [2023-12-27 00:40:08,732][105692] Updated weights for policy 0, policy_version 1274498 (0.0008) [2023-12-27 00:40:08,787][105692] Updated weights for policy 0, policy_version 1274508 (0.0009) [2023-12-27 00:40:08,870][105620] Updated weights for policy 1, policy_version 1275977 (0.0009) [2023-12-27 00:40:08,924][105620] Updated weights for policy 1, policy_version 1275987 (0.0009) [2023-12-27 00:40:08,987][105620] Updated weights for policy 1, policy_version 1275997 (0.0009) [2023-12-27 00:40:09,494][105692] Updated weights for policy 0, policy_version 1274518 (0.0010) [2023-12-27 00:40:09,553][105692] Updated weights for policy 0, policy_version 1274528 (0.0010) [2023-12-27 00:40:09,615][105692] Updated weights for policy 0, policy_version 1274538 (0.0010) [2023-12-27 00:40:09,843][105620] Updated weights for policy 1, policy_version 1276007 (0.0008) [2023-12-27 00:40:09,899][105620] Updated weights for policy 1, policy_version 1276017 (0.0009) [2023-12-27 00:40:09,967][105620] Updated weights for policy 1, policy_version 1276027 (0.0009) [2023-12-27 00:40:10,362][105692] Updated weights for policy 0, policy_version 1274548 (0.0009) [2023-12-27 00:40:10,431][105692] Updated weights for policy 0, policy_version 1274558 (0.0007) [2023-12-27 00:40:10,493][105692] Updated weights for policy 0, policy_version 1274568 (0.0011) [2023-12-27 00:40:10,807][105620] Updated weights for policy 1, policy_version 1276037 (0.0008) [2023-12-27 00:40:10,867][105620] Updated weights for policy 1, policy_version 1276047 (0.0008) [2023-12-27 00:40:10,923][105620] Updated weights for policy 1, policy_version 1276057 (0.0009) [2023-12-27 00:40:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 653058048. Throughput: 0: 9743.2, 1: 9834.6. Samples: 653065316. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:40:11,062][104569] Avg episode reward: [(0, '8818.822'), (1, '9269.984')] [2023-12-27 00:40:11,172][105692] Updated weights for policy 0, policy_version 1274578 (0.0009) [2023-12-27 00:40:11,235][105692] Updated weights for policy 0, policy_version 1274588 (0.0010) [2023-12-27 00:40:11,301][105692] Updated weights for policy 0, policy_version 1274598 (0.0011) [2023-12-27 00:40:11,368][105692] Updated weights for policy 0, policy_version 1274608 (0.0010) [2023-12-27 00:40:11,785][105620] Updated weights for policy 1, policy_version 1276067 (0.0008) [2023-12-27 00:40:11,835][105620] Updated weights for policy 1, policy_version 1276077 (0.0008) [2023-12-27 00:40:11,880][105620] Updated weights for policy 1, policy_version 1276087 (0.0008) [2023-12-27 00:40:12,092][105692] Updated weights for policy 0, policy_version 1274618 (0.0011) [2023-12-27 00:40:12,159][105692] Updated weights for policy 0, policy_version 1274628 (0.0008) [2023-12-27 00:40:12,229][105692] Updated weights for policy 0, policy_version 1274638 (0.0009) [2023-12-27 00:40:12,723][105620] Updated weights for policy 1, policy_version 1276097 (0.0008) [2023-12-27 00:40:12,777][105620] Updated weights for policy 1, policy_version 1276107 (0.0010) [2023-12-27 00:40:12,840][105620] Updated weights for policy 1, policy_version 1276117 (0.0009) [2023-12-27 00:40:12,875][105692] Updated weights for policy 0, policy_version 1274648 (0.0007) [2023-12-27 00:40:12,890][105620] Updated weights for policy 1, policy_version 1276127 (0.0006) [2023-12-27 00:40:12,926][105692] Updated weights for policy 0, policy_version 1274658 (0.0007) [2023-12-27 00:40:12,971][105692] Updated weights for policy 0, policy_version 1274668 (0.0005) [2023-12-27 00:40:13,552][105692] Updated weights for policy 0, policy_version 1274678 (0.0006) [2023-12-27 00:40:13,607][105692] Updated weights for policy 0, policy_version 1274688 (0.0009) [2023-12-27 00:40:13,668][105692] Updated weights for policy 0, policy_version 1274698 (0.0009) [2023-12-27 00:40:13,764][105620] Updated weights for policy 1, policy_version 1276137 (0.0008) [2023-12-27 00:40:13,821][105620] Updated weights for policy 1, policy_version 1276147 (0.0009) [2023-12-27 00:40:13,878][105620] Updated weights for policy 1, policy_version 1276157 (0.0009) [2023-12-27 00:40:14,331][105692] Updated weights for policy 0, policy_version 1274708 (0.0008) [2023-12-27 00:40:14,400][105692] Updated weights for policy 0, policy_version 1274718 (0.0005) [2023-12-27 00:40:14,468][105692] Updated weights for policy 0, policy_version 1274728 (0.0006) [2023-12-27 00:40:14,709][105620] Updated weights for policy 1, policy_version 1276167 (0.0009) [2023-12-27 00:40:14,765][105620] Updated weights for policy 1, policy_version 1276177 (0.0009) [2023-12-27 00:40:14,824][105620] Updated weights for policy 1, policy_version 1276187 (0.0008) [2023-12-27 00:40:15,108][105692] Updated weights for policy 0, policy_version 1274738 (0.0009) [2023-12-27 00:40:15,161][105692] Updated weights for policy 0, policy_version 1274748 (0.0009) [2023-12-27 00:40:15,211][105692] Updated weights for policy 0, policy_version 1274758 (0.0010) [2023-12-27 00:40:15,259][105692] Updated weights for policy 0, policy_version 1274768 (0.0009) [2023-12-27 00:40:15,451][105620] Updated weights for policy 1, policy_version 1276197 (0.0008) [2023-12-27 00:40:15,516][105620] Updated weights for policy 1, policy_version 1276207 (0.0009) [2023-12-27 00:40:15,567][105620] Updated weights for policy 1, policy_version 1276217 (0.0008) [2023-12-27 00:40:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 653148160. Throughput: 0: 9823.5, 1: 9746.4. Samples: 653122176. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 00:40:16,063][104569] Avg episode reward: [(0, '8455.626'), (1, '9086.834')] [2023-12-27 00:40:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001276224_326754304.pth... [2023-12-27 00:40:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001275104_326467584.pth [2023-12-27 00:40:16,076][105692] Updated weights for policy 0, policy_version 1274778 (0.0006) [2023-12-27 00:40:16,125][105692] Updated weights for policy 0, policy_version 1274788 (0.0006) [2023-12-27 00:40:16,145][105620] Updated weights for policy 1, policy_version 1276227 (0.0005) [2023-12-27 00:40:16,174][105692] Updated weights for policy 0, policy_version 1274798 (0.0006) [2023-12-27 00:40:16,185][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001274800_326402048.pth... [2023-12-27 00:40:16,189][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001273616_326098944.pth [2023-12-27 00:40:16,204][105620] Updated weights for policy 1, policy_version 1276237 (0.0005) [2023-12-27 00:40:16,250][105620] Updated weights for policy 1, policy_version 1276247 (0.0005) [2023-12-27 00:40:16,766][105692] Updated weights for policy 0, policy_version 1274808 (0.0008) [2023-12-27 00:40:16,810][105692] Updated weights for policy 0, policy_version 1274818 (0.0005) [2023-12-27 00:40:16,857][105692] Updated weights for policy 0, policy_version 1274828 (0.0005) [2023-12-27 00:40:17,017][105620] Updated weights for policy 1, policy_version 1276257 (0.0006) [2023-12-27 00:40:17,073][105620] Updated weights for policy 1, policy_version 1276267 (0.0009) [2023-12-27 00:40:17,131][105620] Updated weights for policy 1, policy_version 1276277 (0.0010) [2023-12-27 00:40:17,185][105620] Updated weights for policy 1, policy_version 1276288 (0.0010) [2023-12-27 00:40:17,456][105692] Updated weights for policy 0, policy_version 1274838 (0.0008) [2023-12-27 00:40:17,506][105692] Updated weights for policy 0, policy_version 1274848 (0.0008) [2023-12-27 00:40:17,557][105692] Updated weights for policy 0, policy_version 1274858 (0.0009) [2023-12-27 00:40:17,924][105620] Updated weights for policy 1, policy_version 1276298 (0.0008) [2023-12-27 00:40:17,971][105620] Updated weights for policy 1, policy_version 1276308 (0.0009) [2023-12-27 00:40:18,024][105620] Updated weights for policy 1, policy_version 1276318 (0.0009) [2023-12-27 00:40:18,281][105692] Updated weights for policy 0, policy_version 1274868 (0.0009) [2023-12-27 00:40:18,335][105692] Updated weights for policy 0, policy_version 1274878 (0.0008) [2023-12-27 00:40:18,400][105692] Updated weights for policy 0, policy_version 1274888 (0.0008) [2023-12-27 00:40:18,957][105620] Updated weights for policy 1, policy_version 1276328 (0.0009) [2023-12-27 00:40:18,973][105692] Updated weights for policy 0, policy_version 1274898 (0.0007) [2023-12-27 00:40:19,023][105620] Updated weights for policy 1, policy_version 1276338 (0.0008) [2023-12-27 00:40:19,025][105692] Updated weights for policy 0, policy_version 1274908 (0.0006) [2023-12-27 00:40:19,079][105620] Updated weights for policy 1, policy_version 1276348 (0.0007) [2023-12-27 00:40:19,081][105692] Updated weights for policy 0, policy_version 1274918 (0.0006) [2023-12-27 00:40:19,147][105692] Updated weights for policy 0, policy_version 1274928 (0.0008) [2023-12-27 00:40:19,728][105620] Updated weights for policy 1, policy_version 1276358 (0.0009) [2023-12-27 00:40:19,791][105620] Updated weights for policy 1, policy_version 1276368 (0.0011) [2023-12-27 00:40:19,857][105620] Updated weights for policy 1, policy_version 1276378 (0.0010) [2023-12-27 00:40:19,994][105692] Updated weights for policy 0, policy_version 1274938 (0.0009) [2023-12-27 00:40:20,052][105692] Updated weights for policy 0, policy_version 1274948 (0.0009) [2023-12-27 00:40:20,106][105692] Updated weights for policy 0, policy_version 1274958 (0.0009) [2023-12-27 00:40:20,589][105620] Updated weights for policy 1, policy_version 1276388 (0.0009) [2023-12-27 00:40:20,651][105620] Updated weights for policy 1, policy_version 1276398 (0.0009) [2023-12-27 00:40:20,721][105620] Updated weights for policy 1, policy_version 1276408 (0.0009) [2023-12-27 00:40:20,883][105692] Updated weights for policy 0, policy_version 1274968 (0.0007) [2023-12-27 00:40:20,943][105692] Updated weights for policy 0, policy_version 1274978 (0.0009) [2023-12-27 00:40:21,000][105692] Updated weights for policy 0, policy_version 1274988 (0.0009) [2023-12-27 00:40:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 653254656. Throughput: 0: 9935.1, 1: 9671.6. Samples: 653241416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:40:21,062][104569] Avg episode reward: [(0, '8730.340'), (1, '9171.103')] [2023-12-27 00:40:21,523][105620] Updated weights for policy 1, policy_version 1276418 (0.0009) [2023-12-27 00:40:21,582][105620] Updated weights for policy 1, policy_version 1276428 (0.0008) [2023-12-27 00:40:21,642][105620] Updated weights for policy 1, policy_version 1276438 (0.0008) [2023-12-27 00:40:21,709][105620] Updated weights for policy 1, policy_version 1276448 (0.0008) [2023-12-27 00:40:21,793][105692] Updated weights for policy 0, policy_version 1274998 (0.0009) [2023-12-27 00:40:21,862][105692] Updated weights for policy 0, policy_version 1275008 (0.0008) [2023-12-27 00:40:21,936][105692] Updated weights for policy 0, policy_version 1275018 (0.0008) [2023-12-27 00:40:22,400][105620] Updated weights for policy 1, policy_version 1276458 (0.0007) [2023-12-27 00:40:22,464][105620] Updated weights for policy 1, policy_version 1276468 (0.0008) [2023-12-27 00:40:22,526][105620] Updated weights for policy 1, policy_version 1276478 (0.0009) [2023-12-27 00:40:22,592][105692] Updated weights for policy 0, policy_version 1275028 (0.0009) [2023-12-27 00:40:22,648][105692] Updated weights for policy 0, policy_version 1275038 (0.0009) [2023-12-27 00:40:22,707][105692] Updated weights for policy 0, policy_version 1275048 (0.0010) [2023-12-27 00:40:23,241][105620] Updated weights for policy 1, policy_version 1276488 (0.0010) [2023-12-27 00:40:23,295][105620] Updated weights for policy 1, policy_version 1276498 (0.0009) [2023-12-27 00:40:23,352][105620] Updated weights for policy 1, policy_version 1276508 (0.0009) [2023-12-27 00:40:23,391][105692] Updated weights for policy 0, policy_version 1275058 (0.0010) [2023-12-27 00:40:23,443][105692] Updated weights for policy 0, policy_version 1275068 (0.0009) [2023-12-27 00:40:23,498][105692] Updated weights for policy 0, policy_version 1275078 (0.0009) [2023-12-27 00:40:24,027][105620] Updated weights for policy 1, policy_version 1276518 (0.0009) [2023-12-27 00:40:24,080][105620] Updated weights for policy 1, policy_version 1276528 (0.0008) [2023-12-27 00:40:24,134][105620] Updated weights for policy 1, policy_version 1276538 (0.0008) [2023-12-27 00:40:24,338][105692] Updated weights for policy 0, policy_version 1275089 (0.0009) [2023-12-27 00:40:24,389][105692] Updated weights for policy 0, policy_version 1275099 (0.0006) [2023-12-27 00:40:24,435][105692] Updated weights for policy 0, policy_version 1275109 (0.0006) [2023-12-27 00:40:24,482][105692] Updated weights for policy 0, policy_version 1275119 (0.0006) [2023-12-27 00:40:24,926][105620] Updated weights for policy 1, policy_version 1276548 (0.0008) [2023-12-27 00:40:24,978][105620] Updated weights for policy 1, policy_version 1276558 (0.0009) [2023-12-27 00:40:25,038][105620] Updated weights for policy 1, policy_version 1276568 (0.0009) [2023-12-27 00:40:25,070][105692] Updated weights for policy 0, policy_version 1275129 (0.0009) [2023-12-27 00:40:25,118][105692] Updated weights for policy 0, policy_version 1275139 (0.0009) [2023-12-27 00:40:25,170][105692] Updated weights for policy 0, policy_version 1275149 (0.0011) [2023-12-27 00:40:25,843][105620] Updated weights for policy 1, policy_version 1276578 (0.0008) [2023-12-27 00:40:25,899][105620] Updated weights for policy 1, policy_version 1276588 (0.0008) [2023-12-27 00:40:25,926][105692] Updated weights for policy 0, policy_version 1275159 (0.0007) [2023-12-27 00:40:25,959][105620] Updated weights for policy 1, policy_version 1276598 (0.0009) [2023-12-27 00:40:25,973][105692] Updated weights for policy 0, policy_version 1275169 (0.0006) [2023-12-27 00:40:26,018][105692] Updated weights for policy 0, policy_version 1275179 (0.0006) [2023-12-27 00:40:26,018][105620] Updated weights for policy 1, policy_version 1276608 (0.0009) [2023-12-27 00:40:26,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 653352960. Throughput: 0: 9919.8, 1: 9554.2. Samples: 653355372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:40:26,062][104569] Avg episode reward: [(0, '8996.297'), (1, '9171.029')] [2023-12-27 00:40:26,763][105620] Updated weights for policy 1, policy_version 1276618 (0.0009) [2023-12-27 00:40:26,766][105692] Updated weights for policy 0, policy_version 1275189 (0.0009) [2023-12-27 00:40:26,812][105620] Updated weights for policy 1, policy_version 1276628 (0.0007) [2023-12-27 00:40:26,823][105692] Updated weights for policy 0, policy_version 1275199 (0.0008) [2023-12-27 00:40:26,869][105620] Updated weights for policy 1, policy_version 1276638 (0.0006) [2023-12-27 00:40:26,882][105692] Updated weights for policy 0, policy_version 1275209 (0.0009) [2023-12-27 00:40:27,577][105692] Updated weights for policy 0, policy_version 1275219 (0.0008) [2023-12-27 00:40:27,582][105620] Updated weights for policy 1, policy_version 1276648 (0.0006) [2023-12-27 00:40:27,630][105692] Updated weights for policy 0, policy_version 1275229 (0.0005) [2023-12-27 00:40:27,636][105620] Updated weights for policy 1, policy_version 1276658 (0.0005) [2023-12-27 00:40:27,684][105692] Updated weights for policy 0, policy_version 1275239 (0.0006) [2023-12-27 00:40:27,696][105620] Updated weights for policy 1, policy_version 1276668 (0.0008) [2023-12-27 00:40:28,248][105692] Updated weights for policy 0, policy_version 1275249 (0.0006) [2023-12-27 00:40:28,277][105620] Updated weights for policy 1, policy_version 1276678 (0.0007) [2023-12-27 00:40:28,300][105692] Updated weights for policy 0, policy_version 1275259 (0.0006) [2023-12-27 00:40:28,331][105620] Updated weights for policy 1, policy_version 1276688 (0.0008) [2023-12-27 00:40:28,359][105692] Updated weights for policy 0, policy_version 1275269 (0.0007) [2023-12-27 00:40:28,387][105620] Updated weights for policy 1, policy_version 1276698 (0.0008) [2023-12-27 00:40:28,408][105692] Updated weights for policy 0, policy_version 1275279 (0.0006) [2023-12-27 00:40:28,988][105620] Updated weights for policy 1, policy_version 1276708 (0.0006) [2023-12-27 00:40:29,042][105620] Updated weights for policy 1, policy_version 1276718 (0.0005) [2023-12-27 00:40:29,093][105620] Updated weights for policy 1, policy_version 1276728 (0.0005) [2023-12-27 00:40:29,122][105692] Updated weights for policy 0, policy_version 1275289 (0.0009) [2023-12-27 00:40:29,171][105692] Updated weights for policy 0, policy_version 1275300 (0.0009) [2023-12-27 00:40:29,227][105692] Updated weights for policy 0, policy_version 1275311 (0.0010) [2023-12-27 00:40:29,678][105620] Updated weights for policy 1, policy_version 1276738 (0.0006) [2023-12-27 00:40:29,731][105620] Updated weights for policy 1, policy_version 1276748 (0.0007) [2023-12-27 00:40:29,777][105620] Updated weights for policy 1, policy_version 1276758 (0.0005) [2023-12-27 00:40:29,837][105620] Updated weights for policy 1, policy_version 1276768 (0.0006) [2023-12-27 00:40:30,085][105692] Updated weights for policy 0, policy_version 1275321 (0.0010) [2023-12-27 00:40:30,145][105692] Updated weights for policy 0, policy_version 1275331 (0.0010) [2023-12-27 00:40:30,214][105692] Updated weights for policy 0, policy_version 1275341 (0.0011) [2023-12-27 00:40:30,576][105620] Updated weights for policy 1, policy_version 1276779 (0.0010) [2023-12-27 00:40:30,632][105620] Updated weights for policy 1, policy_version 1276789 (0.0009) [2023-12-27 00:40:30,687][105620] Updated weights for policy 1, policy_version 1276799 (0.0009) [2023-12-27 00:40:30,792][105692] Updated weights for policy 0, policy_version 1275351 (0.0007) [2023-12-27 00:40:30,855][105692] Updated weights for policy 0, policy_version 1275361 (0.0006) [2023-12-27 00:40:30,925][105692] Updated weights for policy 0, policy_version 1275371 (0.0010) [2023-12-27 00:40:31,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 653451264. Throughput: 0: 9984.9, 1: 9635.6. Samples: 653417616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:40:31,063][104569] Avg episode reward: [(0, '8624.279'), (1, '9171.690')] [2023-12-27 00:40:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001275376_326549504.pth... [2023-12-27 00:40:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001276800_326901760.pth... [2023-12-27 00:40:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001274192_326246400.pth [2023-12-27 00:40:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001275680_326615040.pth [2023-12-27 00:40:31,440][105620] Updated weights for policy 1, policy_version 1276809 (0.0009) [2023-12-27 00:40:31,504][105620] Updated weights for policy 1, policy_version 1276819 (0.0010) [2023-12-27 00:40:31,547][105692] Updated weights for policy 0, policy_version 1275381 (0.0008) [2023-12-27 00:40:31,562][105620] Updated weights for policy 1, policy_version 1276829 (0.0008) [2023-12-27 00:40:31,606][105692] Updated weights for policy 0, policy_version 1275391 (0.0006) [2023-12-27 00:40:31,676][105692] Updated weights for policy 0, policy_version 1275401 (0.0009) [2023-12-27 00:40:32,298][105620] Updated weights for policy 1, policy_version 1276839 (0.0008) [2023-12-27 00:40:32,356][105620] Updated weights for policy 1, policy_version 1276849 (0.0008) [2023-12-27 00:40:32,415][105620] Updated weights for policy 1, policy_version 1276859 (0.0007) [2023-12-27 00:40:32,415][105692] Updated weights for policy 0, policy_version 1275411 (0.0011) [2023-12-27 00:40:32,475][105692] Updated weights for policy 0, policy_version 1275421 (0.0011) [2023-12-27 00:40:32,530][105692] Updated weights for policy 0, policy_version 1275431 (0.0011) [2023-12-27 00:40:33,154][105620] Updated weights for policy 1, policy_version 1276869 (0.0006) [2023-12-27 00:40:33,203][105620] Updated weights for policy 1, policy_version 1276879 (0.0005) [2023-12-27 00:40:33,253][105620] Updated weights for policy 1, policy_version 1276889 (0.0005) [2023-12-27 00:40:33,277][105692] Updated weights for policy 0, policy_version 1275441 (0.0010) [2023-12-27 00:40:33,335][105692] Updated weights for policy 0, policy_version 1275451 (0.0011) [2023-12-27 00:40:33,387][105692] Updated weights for policy 0, policy_version 1275461 (0.0009) [2023-12-27 00:40:33,431][105692] Updated weights for policy 0, policy_version 1275471 (0.0006) [2023-12-27 00:40:33,878][105620] Updated weights for policy 1, policy_version 1276899 (0.0005) [2023-12-27 00:40:33,930][105620] Updated weights for policy 1, policy_version 1276909 (0.0005) [2023-12-27 00:40:33,979][105620] Updated weights for policy 1, policy_version 1276919 (0.0008) [2023-12-27 00:40:34,008][105692] Updated weights for policy 0, policy_version 1275481 (0.0006) [2023-12-27 00:40:34,058][105692] Updated weights for policy 0, policy_version 1275491 (0.0007) [2023-12-27 00:40:34,105][105692] Updated weights for policy 0, policy_version 1275501 (0.0009) [2023-12-27 00:40:34,705][105620] Updated weights for policy 1, policy_version 1276929 (0.0009) [2023-12-27 00:40:34,763][105620] Updated weights for policy 1, policy_version 1276939 (0.0009) [2023-12-27 00:40:34,818][105620] Updated weights for policy 1, policy_version 1276949 (0.0007) [2023-12-27 00:40:34,820][105692] Updated weights for policy 0, policy_version 1275511 (0.0008) [2023-12-27 00:40:34,868][105620] Updated weights for policy 1, policy_version 1276959 (0.0005) [2023-12-27 00:40:34,872][105692] Updated weights for policy 0, policy_version 1275521 (0.0009) [2023-12-27 00:40:34,926][105692] Updated weights for policy 0, policy_version 1275531 (0.0009) [2023-12-27 00:40:35,456][105620] Updated weights for policy 1, policy_version 1276969 (0.0006) [2023-12-27 00:40:35,519][105620] Updated weights for policy 1, policy_version 1276979 (0.0006) [2023-12-27 00:40:35,525][105692] Updated weights for policy 0, policy_version 1275541 (0.0008) [2023-12-27 00:40:35,570][105620] Updated weights for policy 1, policy_version 1276989 (0.0005) [2023-12-27 00:40:35,583][105692] Updated weights for policy 0, policy_version 1275551 (0.0008) [2023-12-27 00:40:35,647][105692] Updated weights for policy 0, policy_version 1275561 (0.0007) [2023-12-27 00:40:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.9, 300 sec: 19494.2). Total num frames: 653549568. Throughput: 0: 9959.2, 1: 9604.8. Samples: 653537636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:40:36,062][104569] Avg episode reward: [(0, '8806.461'), (1, '8988.319')] [2023-12-27 00:40:36,241][105692] Updated weights for policy 0, policy_version 1275571 (0.0006) [2023-12-27 00:40:36,303][105620] Updated weights for policy 1, policy_version 1276999 (0.0010) [2023-12-27 00:40:36,306][105692] Updated weights for policy 0, policy_version 1275581 (0.0007) [2023-12-27 00:40:36,368][105692] Updated weights for policy 0, policy_version 1275591 (0.0008) [2023-12-27 00:40:36,369][105620] Updated weights for policy 1, policy_version 1277009 (0.0011) [2023-12-27 00:40:36,433][105620] Updated weights for policy 1, policy_version 1277019 (0.0011) [2023-12-27 00:40:37,067][105620] Updated weights for policy 1, policy_version 1277029 (0.0009) [2023-12-27 00:40:37,133][105620] Updated weights for policy 1, policy_version 1277039 (0.0005) [2023-12-27 00:40:37,182][105620] Updated weights for policy 1, policy_version 1277049 (0.0007) [2023-12-27 00:40:37,184][105692] Updated weights for policy 0, policy_version 1275601 (0.0008) [2023-12-27 00:40:37,252][105692] Updated weights for policy 0, policy_version 1275611 (0.0009) [2023-12-27 00:40:37,321][105692] Updated weights for policy 0, policy_version 1275621 (0.0010) [2023-12-27 00:40:37,381][105692] Updated weights for policy 0, policy_version 1275631 (0.0009) [2023-12-27 00:40:37,829][105620] Updated weights for policy 1, policy_version 1277059 (0.0007) [2023-12-27 00:40:37,897][105620] Updated weights for policy 1, policy_version 1277069 (0.0009) [2023-12-27 00:40:37,962][105620] Updated weights for policy 1, policy_version 1277079 (0.0009) [2023-12-27 00:40:38,071][105692] Updated weights for policy 0, policy_version 1275641 (0.0008) [2023-12-27 00:40:38,121][105692] Updated weights for policy 0, policy_version 1275651 (0.0009) [2023-12-27 00:40:38,182][105692] Updated weights for policy 0, policy_version 1275661 (0.0009) [2023-12-27 00:40:38,667][105620] Updated weights for policy 1, policy_version 1277089 (0.0010) [2023-12-27 00:40:38,711][105620] Updated weights for policy 1, policy_version 1277099 (0.0008) [2023-12-27 00:40:38,773][105620] Updated weights for policy 1, policy_version 1277109 (0.0008) [2023-12-27 00:40:38,839][105620] Updated weights for policy 1, policy_version 1277119 (0.0007) [2023-12-27 00:40:38,929][105692] Updated weights for policy 0, policy_version 1275671 (0.0009) [2023-12-27 00:40:38,984][105692] Updated weights for policy 0, policy_version 1275681 (0.0008) [2023-12-27 00:40:39,054][105692] Updated weights for policy 0, policy_version 1275691 (0.0008) [2023-12-27 00:40:39,600][105620] Updated weights for policy 1, policy_version 1277129 (0.0009) [2023-12-27 00:40:39,669][105620] Updated weights for policy 1, policy_version 1277139 (0.0009) [2023-12-27 00:40:39,724][105620] Updated weights for policy 1, policy_version 1277149 (0.0008) [2023-12-27 00:40:39,814][105692] Updated weights for policy 0, policy_version 1275701 (0.0010) [2023-12-27 00:40:39,875][105692] Updated weights for policy 0, policy_version 1275711 (0.0009) [2023-12-27 00:40:39,944][105692] Updated weights for policy 0, policy_version 1275721 (0.0010) [2023-12-27 00:40:40,525][105620] Updated weights for policy 1, policy_version 1277159 (0.0009) [2023-12-27 00:40:40,583][105620] Updated weights for policy 1, policy_version 1277169 (0.0008) [2023-12-27 00:40:40,650][105620] Updated weights for policy 1, policy_version 1277179 (0.0009) [2023-12-27 00:40:40,691][105692] Updated weights for policy 0, policy_version 1275731 (0.0010) [2023-12-27 00:40:40,744][105692] Updated weights for policy 0, policy_version 1275741 (0.0009) [2023-12-27 00:40:40,792][105692] Updated weights for policy 0, policy_version 1275751 (0.0009) [2023-12-27 00:40:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 653647872. Throughput: 0: 10026.6, 1: 9604.8. Samples: 653655424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:40:41,063][104569] Avg episode reward: [(0, '8987.827'), (1, '8988.639')] [2023-12-27 00:40:41,421][105620] Updated weights for policy 1, policy_version 1277189 (0.0008) [2023-12-27 00:40:41,486][105620] Updated weights for policy 1, policy_version 1277199 (0.0009) [2023-12-27 00:40:41,541][105620] Updated weights for policy 1, policy_version 1277209 (0.0010) [2023-12-27 00:40:41,632][105692] Updated weights for policy 0, policy_version 1275761 (0.0009) [2023-12-27 00:40:41,697][105692] Updated weights for policy 0, policy_version 1275771 (0.0008) [2023-12-27 00:40:41,763][105692] Updated weights for policy 0, policy_version 1275781 (0.0010) [2023-12-27 00:40:41,830][105692] Updated weights for policy 0, policy_version 1275791 (0.0009) [2023-12-27 00:40:42,298][105620] Updated weights for policy 1, policy_version 1277219 (0.0008) [2023-12-27 00:40:42,371][105620] Updated weights for policy 1, policy_version 1277229 (0.0010) [2023-12-27 00:40:42,436][105620] Updated weights for policy 1, policy_version 1277239 (0.0010) [2023-12-27 00:40:42,566][105692] Updated weights for policy 0, policy_version 1275801 (0.0009) [2023-12-27 00:40:42,626][105692] Updated weights for policy 0, policy_version 1275811 (0.0009) [2023-12-27 00:40:42,684][105692] Updated weights for policy 0, policy_version 1275821 (0.0009) [2023-12-27 00:40:43,143][105620] Updated weights for policy 1, policy_version 1277249 (0.0009) [2023-12-27 00:40:43,202][105620] Updated weights for policy 1, policy_version 1277259 (0.0007) [2023-12-27 00:40:43,252][105620] Updated weights for policy 1, policy_version 1277269 (0.0006) [2023-12-27 00:40:43,306][105620] Updated weights for policy 1, policy_version 1277279 (0.0006) [2023-12-27 00:40:43,486][105692] Updated weights for policy 0, policy_version 1275831 (0.0010) [2023-12-27 00:40:43,544][105692] Updated weights for policy 0, policy_version 1275841 (0.0010) [2023-12-27 00:40:43,603][105692] Updated weights for policy 0, policy_version 1275851 (0.0010) [2023-12-27 00:40:43,977][105620] Updated weights for policy 1, policy_version 1277289 (0.0007) [2023-12-27 00:40:44,029][105620] Updated weights for policy 1, policy_version 1277300 (0.0010) [2023-12-27 00:40:44,087][105620] Updated weights for policy 1, policy_version 1277310 (0.0009) [2023-12-27 00:40:44,162][105692] Updated weights for policy 0, policy_version 1275861 (0.0005) [2023-12-27 00:40:44,234][105692] Updated weights for policy 0, policy_version 1275871 (0.0005) [2023-12-27 00:40:44,300][105692] Updated weights for policy 0, policy_version 1275881 (0.0005) [2023-12-27 00:40:44,841][105692] Updated weights for policy 0, policy_version 1275891 (0.0007) [2023-12-27 00:40:44,873][105620] Updated weights for policy 1, policy_version 1277320 (0.0008) [2023-12-27 00:40:44,901][105692] Updated weights for policy 0, policy_version 1275901 (0.0011) [2023-12-27 00:40:44,932][105620] Updated weights for policy 1, policy_version 1277330 (0.0008) [2023-12-27 00:40:44,963][105692] Updated weights for policy 0, policy_version 1275911 (0.0010) [2023-12-27 00:40:45,001][105620] Updated weights for policy 1, policy_version 1277340 (0.0008) [2023-12-27 00:40:45,718][105620] Updated weights for policy 1, policy_version 1277350 (0.0010) [2023-12-27 00:40:45,732][105692] Updated weights for policy 0, policy_version 1275921 (0.0010) [2023-12-27 00:40:45,782][105620] Updated weights for policy 1, policy_version 1277360 (0.0005) [2023-12-27 00:40:45,788][105692] Updated weights for policy 0, policy_version 1275931 (0.0008) [2023-12-27 00:40:45,845][105692] Updated weights for policy 0, policy_version 1275941 (0.0009) [2023-12-27 00:40:45,847][105620] Updated weights for policy 1, policy_version 1277370 (0.0005) [2023-12-27 00:40:45,908][105692] Updated weights for policy 0, policy_version 1275951 (0.0009) [2023-12-27 00:40:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 653746176. Throughput: 0: 9906.9, 1: 9598.3. Samples: 653710572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:40:46,062][104569] Avg episode reward: [(0, '9079.653'), (1, '9172.066')] [2023-12-27 00:40:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001275952_326696960.pth... [2023-12-27 00:40:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001277376_327049216.pth... [2023-12-27 00:40:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001274800_326402048.pth [2023-12-27 00:40:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001276224_326754304.pth [2023-12-27 00:40:46,421][105620] Updated weights for policy 1, policy_version 1277380 (0.0006) [2023-12-27 00:40:46,472][105620] Updated weights for policy 1, policy_version 1277390 (0.0010) [2023-12-27 00:40:46,521][105620] Updated weights for policy 1, policy_version 1277400 (0.0010) [2023-12-27 00:40:46,671][105692] Updated weights for policy 0, policy_version 1275961 (0.0007) [2023-12-27 00:40:46,736][105692] Updated weights for policy 0, policy_version 1275971 (0.0008) [2023-12-27 00:40:46,798][105692] Updated weights for policy 0, policy_version 1275981 (0.0010) [2023-12-27 00:40:47,216][105620] Updated weights for policy 1, policy_version 1277410 (0.0009) [2023-12-27 00:40:47,269][105620] Updated weights for policy 1, policy_version 1277420 (0.0008) [2023-12-27 00:40:47,321][105620] Updated weights for policy 1, policy_version 1277430 (0.0010) [2023-12-27 00:40:47,375][105620] Updated weights for policy 1, policy_version 1277440 (0.0010) [2023-12-27 00:40:47,387][105692] Updated weights for policy 0, policy_version 1275991 (0.0008) [2023-12-27 00:40:47,439][105692] Updated weights for policy 0, policy_version 1276001 (0.0006) [2023-12-27 00:40:47,498][105692] Updated weights for policy 0, policy_version 1276011 (0.0005) [2023-12-27 00:40:48,125][105620] Updated weights for policy 1, policy_version 1277450 (0.0009) [2023-12-27 00:40:48,180][105620] Updated weights for policy 1, policy_version 1277460 (0.0008) [2023-12-27 00:40:48,190][105692] Updated weights for policy 0, policy_version 1276021 (0.0008) [2023-12-27 00:40:48,239][105692] Updated weights for policy 0, policy_version 1276031 (0.0006) [2023-12-27 00:40:48,240][105620] Updated weights for policy 1, policy_version 1277470 (0.0007) [2023-12-27 00:40:48,290][105692] Updated weights for policy 0, policy_version 1276041 (0.0009) [2023-12-27 00:40:48,987][105692] Updated weights for policy 0, policy_version 1276051 (0.0009) [2023-12-27 00:40:49,036][105620] Updated weights for policy 1, policy_version 1277480 (0.0007) [2023-12-27 00:40:49,047][105692] Updated weights for policy 0, policy_version 1276061 (0.0006) [2023-12-27 00:40:49,089][105620] Updated weights for policy 1, policy_version 1277490 (0.0007) [2023-12-27 00:40:49,095][105692] Updated weights for policy 0, policy_version 1276071 (0.0005) [2023-12-27 00:40:49,136][105620] Updated weights for policy 1, policy_version 1277500 (0.0008) [2023-12-27 00:40:49,782][105692] Updated weights for policy 0, policy_version 1276081 (0.0005) [2023-12-27 00:40:49,839][105692] Updated weights for policy 0, policy_version 1276091 (0.0007) [2023-12-27 00:40:49,901][105692] Updated weights for policy 0, policy_version 1276101 (0.0007) [2023-12-27 00:40:49,963][105692] Updated weights for policy 0, policy_version 1276111 (0.0008) [2023-12-27 00:40:50,000][105620] Updated weights for policy 1, policy_version 1277510 (0.0010) [2023-12-27 00:40:50,061][105620] Updated weights for policy 1, policy_version 1277520 (0.0009) [2023-12-27 00:40:50,114][105620] Updated weights for policy 1, policy_version 1277530 (0.0009) [2023-12-27 00:40:50,675][105692] Updated weights for policy 0, policy_version 1276121 (0.0009) [2023-12-27 00:40:50,734][105692] Updated weights for policy 0, policy_version 1276131 (0.0009) [2023-12-27 00:40:50,795][105692] Updated weights for policy 0, policy_version 1276141 (0.0009) [2023-12-27 00:40:50,887][105620] Updated weights for policy 1, policy_version 1277540 (0.0007) [2023-12-27 00:40:50,939][105620] Updated weights for policy 1, policy_version 1277550 (0.0008) [2023-12-27 00:40:50,994][105620] Updated weights for policy 1, policy_version 1277560 (0.0009) [2023-12-27 00:40:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 653844480. Throughput: 0: 9982.9, 1: 9551.5. Samples: 653829656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:40:51,062][104569] Avg episode reward: [(0, '9089.552'), (1, '9263.214')] [2023-12-27 00:40:51,564][105692] Updated weights for policy 0, policy_version 1276151 (0.0008) [2023-12-27 00:40:51,618][105692] Updated weights for policy 0, policy_version 1276161 (0.0008) [2023-12-27 00:40:51,675][105692] Updated weights for policy 0, policy_version 1276171 (0.0010) [2023-12-27 00:40:51,750][105620] Updated weights for policy 1, policy_version 1277570 (0.0009) [2023-12-27 00:40:51,809][105620] Updated weights for policy 1, policy_version 1277580 (0.0009) [2023-12-27 00:40:51,866][105620] Updated weights for policy 1, policy_version 1277590 (0.0008) [2023-12-27 00:40:51,930][105620] Updated weights for policy 1, policy_version 1277600 (0.0005) [2023-12-27 00:40:52,439][105692] Updated weights for policy 0, policy_version 1276181 (0.0009) [2023-12-27 00:40:52,487][105692] Updated weights for policy 0, policy_version 1276191 (0.0009) [2023-12-27 00:40:52,535][105692] Updated weights for policy 0, policy_version 1276201 (0.0009) [2023-12-27 00:40:52,692][105620] Updated weights for policy 1, policy_version 1277610 (0.0009) [2023-12-27 00:40:52,750][105620] Updated weights for policy 1, policy_version 1277620 (0.0008) [2023-12-27 00:40:52,804][105620] Updated weights for policy 1, policy_version 1277630 (0.0009) [2023-12-27 00:40:53,322][105692] Updated weights for policy 0, policy_version 1276211 (0.0009) [2023-12-27 00:40:53,370][105692] Updated weights for policy 0, policy_version 1276221 (0.0009) [2023-12-27 00:40:53,427][105692] Updated weights for policy 0, policy_version 1276231 (0.0009) [2023-12-27 00:40:53,577][105620] Updated weights for policy 1, policy_version 1277640 (0.0007) [2023-12-27 00:40:53,634][105620] Updated weights for policy 1, policy_version 1277650 (0.0005) [2023-12-27 00:40:53,690][105620] Updated weights for policy 1, policy_version 1277660 (0.0005) [2023-12-27 00:40:54,240][105692] Updated weights for policy 0, policy_version 1276241 (0.0009) [2023-12-27 00:40:54,299][105692] Updated weights for policy 0, policy_version 1276251 (0.0009) [2023-12-27 00:40:54,339][105620] Updated weights for policy 1, policy_version 1277670 (0.0008) [2023-12-27 00:40:54,358][105692] Updated weights for policy 0, policy_version 1276261 (0.0007) [2023-12-27 00:40:54,399][105620] Updated weights for policy 1, policy_version 1277680 (0.0009) [2023-12-27 00:40:54,423][105692] Updated weights for policy 0, policy_version 1276271 (0.0007) [2023-12-27 00:40:54,458][105620] Updated weights for policy 1, policy_version 1277690 (0.0008) [2023-12-27 00:40:55,156][105692] Updated weights for policy 0, policy_version 1276281 (0.0008) [2023-12-27 00:40:55,156][105620] Updated weights for policy 1, policy_version 1277700 (0.0008) [2023-12-27 00:40:55,204][105692] Updated weights for policy 0, policy_version 1276291 (0.0006) [2023-12-27 00:40:55,206][105620] Updated weights for policy 1, policy_version 1277710 (0.0008) [2023-12-27 00:40:55,252][105692] Updated weights for policy 0, policy_version 1276301 (0.0006) [2023-12-27 00:40:55,258][105620] Updated weights for policy 1, policy_version 1277720 (0.0007) [2023-12-27 00:40:55,971][105692] Updated weights for policy 0, policy_version 1276311 (0.0008) [2023-12-27 00:40:55,985][105620] Updated weights for policy 1, policy_version 1277730 (0.0009) [2023-12-27 00:40:56,024][105692] Updated weights for policy 0, policy_version 1276321 (0.0010) [2023-12-27 00:40:56,044][105620] Updated weights for policy 1, policy_version 1277740 (0.0007) [2023-12-27 00:40:56,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 653926400. Throughput: 0: 9891.2, 1: 9615.4. Samples: 653943112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:40:56,063][104569] Avg episode reward: [(0, '8817.269'), (1, '9354.584')] [2023-12-27 00:40:56,083][105692] Updated weights for policy 0, policy_version 1276331 (0.0007) [2023-12-27 00:40:56,100][105620] Updated weights for policy 1, policy_version 1277750 (0.0010) [2023-12-27 00:40:56,162][105620] Updated weights for policy 1, policy_version 1277760 (0.0006) [2023-12-27 00:40:56,771][105692] Updated weights for policy 0, policy_version 1276341 (0.0010) [2023-12-27 00:40:56,828][105692] Updated weights for policy 0, policy_version 1276351 (0.0006) [2023-12-27 00:40:56,876][105620] Updated weights for policy 1, policy_version 1277770 (0.0008) [2023-12-27 00:40:56,884][105692] Updated weights for policy 0, policy_version 1276361 (0.0007) [2023-12-27 00:40:56,932][105620] Updated weights for policy 1, policy_version 1277780 (0.0008) [2023-12-27 00:40:56,994][105620] Updated weights for policy 1, policy_version 1277790 (0.0009) [2023-12-27 00:40:57,486][105692] Updated weights for policy 0, policy_version 1276371 (0.0006) [2023-12-27 00:40:57,545][105692] Updated weights for policy 0, policy_version 1276381 (0.0006) [2023-12-27 00:40:57,601][105692] Updated weights for policy 0, policy_version 1276391 (0.0008) [2023-12-27 00:40:57,815][105620] Updated weights for policy 1, policy_version 1277800 (0.0008) [2023-12-27 00:40:57,862][105620] Updated weights for policy 1, policy_version 1277810 (0.0008) [2023-12-27 00:40:57,912][105620] Updated weights for policy 1, policy_version 1277820 (0.0008) [2023-12-27 00:40:58,314][105692] Updated weights for policy 0, policy_version 1276401 (0.0009) [2023-12-27 00:40:58,386][105692] Updated weights for policy 0, policy_version 1276411 (0.0008) [2023-12-27 00:40:58,451][105692] Updated weights for policy 0, policy_version 1276421 (0.0009) [2023-12-27 00:40:58,515][105692] Updated weights for policy 0, policy_version 1276431 (0.0008) [2023-12-27 00:40:58,755][105620] Updated weights for policy 1, policy_version 1277830 (0.0009) [2023-12-27 00:40:58,833][105620] Updated weights for policy 1, policy_version 1277840 (0.0008) [2023-12-27 00:40:58,897][105620] Updated weights for policy 1, policy_version 1277850 (0.0008) [2023-12-27 00:40:59,373][105692] Updated weights for policy 0, policy_version 1276441 (0.0007) [2023-12-27 00:40:59,420][105692] Updated weights for policy 0, policy_version 1276451 (0.0008) [2023-12-27 00:40:59,470][105692] Updated weights for policy 0, policy_version 1276461 (0.0008) [2023-12-27 00:40:59,732][105620] Updated weights for policy 1, policy_version 1277860 (0.0008) [2023-12-27 00:40:59,779][105620] Updated weights for policy 1, policy_version 1277870 (0.0009) [2023-12-27 00:40:59,830][105620] Updated weights for policy 1, policy_version 1277880 (0.0009) [2023-12-27 00:41:00,175][105692] Updated weights for policy 0, policy_version 1276471 (0.0008) [2023-12-27 00:41:00,236][105692] Updated weights for policy 0, policy_version 1276481 (0.0009) [2023-12-27 00:41:00,297][105692] Updated weights for policy 0, policy_version 1276491 (0.0009) [2023-12-27 00:41:00,592][105620] Updated weights for policy 1, policy_version 1277890 (0.0008) [2023-12-27 00:41:00,649][105620] Updated weights for policy 1, policy_version 1277900 (0.0008) [2023-12-27 00:41:00,711][105620] Updated weights for policy 1, policy_version 1277910 (0.0009) [2023-12-27 00:41:00,771][105620] Updated weights for policy 1, policy_version 1277920 (0.0009) [2023-12-27 00:41:01,027][105692] Updated weights for policy 0, policy_version 1276501 (0.0009) [2023-12-27 00:41:01,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 654024704. Throughput: 0: 9858.1, 1: 9642.1. Samples: 653999684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:41:01,063][104569] Avg episode reward: [(0, '8993.093'), (1, '9263.176')] [2023-12-27 00:41:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001277920_327188480.pth... [2023-12-27 00:41:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001276800_326901760.pth [2023-12-27 00:41:01,092][105692] Updated weights for policy 0, policy_version 1276511 (0.0009) [2023-12-27 00:41:01,152][105692] Updated weights for policy 0, policy_version 1276521 (0.0009) [2023-12-27 00:41:01,198][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001276528_326844416.pth... [2023-12-27 00:41:01,206][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001275376_326549504.pth [2023-12-27 00:41:01,561][105620] Updated weights for policy 1, policy_version 1277930 (0.0009) [2023-12-27 00:41:01,622][105620] Updated weights for policy 1, policy_version 1277940 (0.0009) [2023-12-27 00:41:01,680][105620] Updated weights for policy 1, policy_version 1277950 (0.0008) [2023-12-27 00:41:01,924][105692] Updated weights for policy 0, policy_version 1276531 (0.0009) [2023-12-27 00:41:01,983][105692] Updated weights for policy 0, policy_version 1276541 (0.0010) [2023-12-27 00:41:02,037][105692] Updated weights for policy 0, policy_version 1276551 (0.0008) [2023-12-27 00:41:02,436][105620] Updated weights for policy 1, policy_version 1277960 (0.0009) [2023-12-27 00:41:02,496][105620] Updated weights for policy 1, policy_version 1277970 (0.0009) [2023-12-27 00:41:02,554][105620] Updated weights for policy 1, policy_version 1277980 (0.0009) [2023-12-27 00:41:02,825][105692] Updated weights for policy 0, policy_version 1276561 (0.0009) [2023-12-27 00:41:02,871][105692] Updated weights for policy 0, policy_version 1276571 (0.0008) [2023-12-27 00:41:02,925][105692] Updated weights for policy 0, policy_version 1276581 (0.0009) [2023-12-27 00:41:02,972][105692] Updated weights for policy 0, policy_version 1276591 (0.0009) [2023-12-27 00:41:03,240][105620] Updated weights for policy 1, policy_version 1277990 (0.0009) [2023-12-27 00:41:03,297][105620] Updated weights for policy 1, policy_version 1278000 (0.0009) [2023-12-27 00:41:03,344][105620] Updated weights for policy 1, policy_version 1278010 (0.0009) [2023-12-27 00:41:03,684][105692] Updated weights for policy 0, policy_version 1276601 (0.0009) [2023-12-27 00:41:03,737][105692] Updated weights for policy 0, policy_version 1276611 (0.0008) [2023-12-27 00:41:03,781][105692] Updated weights for policy 0, policy_version 1276621 (0.0006) [2023-12-27 00:41:04,118][105620] Updated weights for policy 1, policy_version 1278020 (0.0009) [2023-12-27 00:41:04,170][105620] Updated weights for policy 1, policy_version 1278030 (0.0009) [2023-12-27 00:41:04,223][105620] Updated weights for policy 1, policy_version 1278040 (0.0009) [2023-12-27 00:41:04,578][105692] Updated weights for policy 0, policy_version 1276631 (0.0008) [2023-12-27 00:41:04,627][105692] Updated weights for policy 0, policy_version 1276641 (0.0009) [2023-12-27 00:41:04,679][105692] Updated weights for policy 0, policy_version 1276651 (0.0009) [2023-12-27 00:41:04,929][105620] Updated weights for policy 1, policy_version 1278050 (0.0009) [2023-12-27 00:41:04,976][105620] Updated weights for policy 1, policy_version 1278060 (0.0009) [2023-12-27 00:41:05,031][105620] Updated weights for policy 1, policy_version 1278070 (0.0009) [2023-12-27 00:41:05,090][105620] Updated weights for policy 1, policy_version 1278080 (0.0009) [2023-12-27 00:41:05,509][105692] Updated weights for policy 0, policy_version 1276661 (0.0009) [2023-12-27 00:41:05,562][105692] Updated weights for policy 0, policy_version 1276671 (0.0009) [2023-12-27 00:41:05,618][105692] Updated weights for policy 0, policy_version 1276681 (0.0009) [2023-12-27 00:41:05,759][105620] Updated weights for policy 1, policy_version 1278090 (0.0005) [2023-12-27 00:41:05,810][105620] Updated weights for policy 1, policy_version 1278100 (0.0008) [2023-12-27 00:41:05,854][105620] Updated weights for policy 1, policy_version 1278110 (0.0010) [2023-12-27 00:41:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 654123008. Throughput: 0: 9727.9, 1: 9603.6. Samples: 654111336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:41:06,062][104569] Avg episode reward: [(0, '9175.798'), (1, '9178.894')] [2023-12-27 00:41:06,435][105692] Updated weights for policy 0, policy_version 1276691 (0.0009) [2023-12-27 00:41:06,491][105692] Updated weights for policy 0, policy_version 1276701 (0.0008) [2023-12-27 00:41:06,553][105692] Updated weights for policy 0, policy_version 1276711 (0.0008) [2023-12-27 00:41:06,619][105620] Updated weights for policy 1, policy_version 1278120 (0.0011) [2023-12-27 00:41:06,677][105620] Updated weights for policy 1, policy_version 1278130 (0.0006) [2023-12-27 00:41:06,739][105620] Updated weights for policy 1, policy_version 1278140 (0.0005) [2023-12-27 00:41:07,217][105692] Updated weights for policy 0, policy_version 1276721 (0.0008) [2023-12-27 00:41:07,280][105692] Updated weights for policy 0, policy_version 1276731 (0.0008) [2023-12-27 00:41:07,333][105692] Updated weights for policy 0, policy_version 1276741 (0.0011) [2023-12-27 00:41:07,380][105620] Updated weights for policy 1, policy_version 1278150 (0.0009) [2023-12-27 00:41:07,393][105692] Updated weights for policy 0, policy_version 1276751 (0.0011) [2023-12-27 00:41:07,439][105620] Updated weights for policy 1, policy_version 1278160 (0.0005) [2023-12-27 00:41:07,496][105620] Updated weights for policy 1, policy_version 1278170 (0.0005) [2023-12-27 00:41:08,066][105620] Updated weights for policy 1, policy_version 1278180 (0.0008) [2023-12-27 00:41:08,116][105692] Updated weights for policy 0, policy_version 1276761 (0.0006) [2023-12-27 00:41:08,120][105620] Updated weights for policy 1, policy_version 1278190 (0.0009) [2023-12-27 00:41:08,171][105692] Updated weights for policy 0, policy_version 1276771 (0.0007) [2023-12-27 00:41:08,180][105620] Updated weights for policy 1, policy_version 1278200 (0.0006) [2023-12-27 00:41:08,228][105692] Updated weights for policy 0, policy_version 1276781 (0.0010) [2023-12-27 00:41:08,740][105620] Updated weights for policy 1, policy_version 1278210 (0.0006) [2023-12-27 00:41:08,791][105620] Updated weights for policy 1, policy_version 1278220 (0.0010) [2023-12-27 00:41:08,846][105620] Updated weights for policy 1, policy_version 1278230 (0.0010) [2023-12-27 00:41:08,860][105692] Updated weights for policy 0, policy_version 1276791 (0.0007) [2023-12-27 00:41:08,902][105620] Updated weights for policy 1, policy_version 1278240 (0.0010) [2023-12-27 00:41:08,918][105692] Updated weights for policy 0, policy_version 1276801 (0.0005) [2023-12-27 00:41:08,967][105692] Updated weights for policy 0, policy_version 1276811 (0.0005) [2023-12-27 00:41:09,621][105620] Updated weights for policy 1, policy_version 1278250 (0.0006) [2023-12-27 00:41:09,675][105692] Updated weights for policy 0, policy_version 1276821 (0.0006) [2023-12-27 00:41:09,680][105620] Updated weights for policy 1, policy_version 1278260 (0.0008) [2023-12-27 00:41:09,737][105620] Updated weights for policy 1, policy_version 1278270 (0.0006) [2023-12-27 00:41:09,741][105692] Updated weights for policy 0, policy_version 1276831 (0.0006) [2023-12-27 00:41:09,813][105692] Updated weights for policy 0, policy_version 1276841 (0.0009) [2023-12-27 00:41:10,444][105620] Updated weights for policy 1, policy_version 1278280 (0.0006) [2023-12-27 00:41:10,497][105620] Updated weights for policy 1, policy_version 1278290 (0.0009) [2023-12-27 00:41:10,555][105692] Updated weights for policy 0, policy_version 1276851 (0.0008) [2023-12-27 00:41:10,567][105620] Updated weights for policy 1, policy_version 1278300 (0.0009) [2023-12-27 00:41:10,610][105692] Updated weights for policy 0, policy_version 1276861 (0.0005) [2023-12-27 00:41:10,681][105692] Updated weights for policy 0, policy_version 1276871 (0.0009) [2023-12-27 00:41:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 654221312. Throughput: 0: 9721.4, 1: 9746.3. Samples: 654231420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:41:11,063][104569] Avg episode reward: [(0, '9175.486'), (1, '9180.026')] [2023-12-27 00:41:11,313][105620] Updated weights for policy 1, policy_version 1278310 (0.0008) [2023-12-27 00:41:11,371][105620] Updated weights for policy 1, policy_version 1278320 (0.0010) [2023-12-27 00:41:11,439][105620] Updated weights for policy 1, policy_version 1278330 (0.0009) [2023-12-27 00:41:11,476][105692] Updated weights for policy 0, policy_version 1276881 (0.0006) [2023-12-27 00:41:11,544][105692] Updated weights for policy 0, policy_version 1276891 (0.0009) [2023-12-27 00:41:11,608][105692] Updated weights for policy 0, policy_version 1276901 (0.0008) [2023-12-27 00:41:11,692][105692] Updated weights for policy 0, policy_version 1276911 (0.0006) [2023-12-27 00:41:12,302][105620] Updated weights for policy 1, policy_version 1278340 (0.0010) [2023-12-27 00:41:12,366][105620] Updated weights for policy 1, policy_version 1278350 (0.0011) [2023-12-27 00:41:12,429][105620] Updated weights for policy 1, policy_version 1278360 (0.0013) [2023-12-27 00:41:12,479][105692] Updated weights for policy 0, policy_version 1276921 (0.0006) [2023-12-27 00:41:12,548][105692] Updated weights for policy 0, policy_version 1276931 (0.0008) [2023-12-27 00:41:12,615][105692] Updated weights for policy 0, policy_version 1276941 (0.0008) [2023-12-27 00:41:13,101][105620] Updated weights for policy 1, policy_version 1278370 (0.0011) [2023-12-27 00:41:13,152][105620] Updated weights for policy 1, policy_version 1278380 (0.0010) [2023-12-27 00:41:13,197][105620] Updated weights for policy 1, policy_version 1278390 (0.0010) [2023-12-27 00:41:13,258][105620] Updated weights for policy 1, policy_version 1278400 (0.0006) [2023-12-27 00:41:13,412][105692] Updated weights for policy 0, policy_version 1276951 (0.0007) [2023-12-27 00:41:13,465][105692] Updated weights for policy 0, policy_version 1276961 (0.0006) [2023-12-27 00:41:13,523][105692] Updated weights for policy 0, policy_version 1276971 (0.0007) [2023-12-27 00:41:13,813][105620] Updated weights for policy 1, policy_version 1278410 (0.0005) [2023-12-27 00:41:13,866][105620] Updated weights for policy 1, policy_version 1278420 (0.0005) [2023-12-27 00:41:13,916][105620] Updated weights for policy 1, policy_version 1278430 (0.0005) [2023-12-27 00:41:14,124][105692] Updated weights for policy 0, policy_version 1276981 (0.0009) [2023-12-27 00:41:14,186][105692] Updated weights for policy 0, policy_version 1276991 (0.0010) [2023-12-27 00:41:14,243][105692] Updated weights for policy 0, policy_version 1277001 (0.0010) [2023-12-27 00:41:14,496][105620] Updated weights for policy 1, policy_version 1278440 (0.0007) [2023-12-27 00:41:14,545][105620] Updated weights for policy 1, policy_version 1278450 (0.0010) [2023-12-27 00:41:14,632][105620] Updated weights for policy 1, policy_version 1278460 (0.0010) [2023-12-27 00:41:14,954][105692] Updated weights for policy 0, policy_version 1277011 (0.0010) [2023-12-27 00:41:15,010][105692] Updated weights for policy 0, policy_version 1277021 (0.0011) [2023-12-27 00:41:15,077][105692] Updated weights for policy 0, policy_version 1277031 (0.0010) [2023-12-27 00:41:15,205][105620] Updated weights for policy 1, policy_version 1278470 (0.0009) [2023-12-27 00:41:15,254][105620] Updated weights for policy 1, policy_version 1278480 (0.0011) [2023-12-27 00:41:15,308][105620] Updated weights for policy 1, policy_version 1278490 (0.0011) [2023-12-27 00:41:15,729][105692] Updated weights for policy 0, policy_version 1277041 (0.0007) [2023-12-27 00:41:15,784][105692] Updated weights for policy 0, policy_version 1277051 (0.0010) [2023-12-27 00:41:15,837][105692] Updated weights for policy 0, policy_version 1277061 (0.0010) [2023-12-27 00:41:15,892][105692] Updated weights for policy 0, policy_version 1277071 (0.0010) [2023-12-27 00:41:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 654319616. Throughput: 0: 9622.2, 1: 9702.5. Samples: 654287228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:41:16,063][104569] Avg episode reward: [(0, '9082.654'), (1, '9264.285')] [2023-12-27 00:41:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001277072_326983680.pth... [2023-12-27 00:41:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001275952_326696960.pth [2023-12-27 00:41:16,084][105620] Updated weights for policy 1, policy_version 1278500 (0.0011) [2023-12-27 00:41:16,133][105620] Updated weights for policy 1, policy_version 1278510 (0.0011) [2023-12-27 00:41:16,182][105620] Updated weights for policy 1, policy_version 1278520 (0.0011) [2023-12-27 00:41:16,217][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001278528_327344128.pth... [2023-12-27 00:41:16,220][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001277376_327049216.pth [2023-12-27 00:41:16,475][105692] Updated weights for policy 0, policy_version 1277081 (0.0005) [2023-12-27 00:41:16,533][105692] Updated weights for policy 0, policy_version 1277091 (0.0005) [2023-12-27 00:41:16,590][105692] Updated weights for policy 0, policy_version 1277101 (0.0005) [2023-12-27 00:41:16,952][105620] Updated weights for policy 1, policy_version 1278530 (0.0010) [2023-12-27 00:41:17,014][105620] Updated weights for policy 1, policy_version 1278540 (0.0009) [2023-12-27 00:41:17,068][105620] Updated weights for policy 1, policy_version 1278550 (0.0008) [2023-12-27 00:41:17,118][105620] Updated weights for policy 1, policy_version 1278560 (0.0005) [2023-12-27 00:41:17,178][105692] Updated weights for policy 0, policy_version 1277111 (0.0009) [2023-12-27 00:41:17,229][105692] Updated weights for policy 0, policy_version 1277121 (0.0010) [2023-12-27 00:41:17,294][105692] Updated weights for policy 0, policy_version 1277131 (0.0010) [2023-12-27 00:41:17,776][105620] Updated weights for policy 1, policy_version 1278570 (0.0005) [2023-12-27 00:41:17,831][105620] Updated weights for policy 1, policy_version 1278580 (0.0006) [2023-12-27 00:41:17,897][105620] Updated weights for policy 1, policy_version 1278590 (0.0006) [2023-12-27 00:41:18,037][105692] Updated weights for policy 0, policy_version 1277141 (0.0010) [2023-12-27 00:41:18,091][105692] Updated weights for policy 0, policy_version 1277151 (0.0010) [2023-12-27 00:41:18,154][105692] Updated weights for policy 0, policy_version 1277161 (0.0006) [2023-12-27 00:41:18,508][105620] Updated weights for policy 1, policy_version 1278600 (0.0007) [2023-12-27 00:41:18,567][105620] Updated weights for policy 1, policy_version 1278610 (0.0007) [2023-12-27 00:41:18,624][105620] Updated weights for policy 1, policy_version 1278620 (0.0008) [2023-12-27 00:41:18,793][105692] Updated weights for policy 0, policy_version 1277171 (0.0006) [2023-12-27 00:41:18,863][105692] Updated weights for policy 0, policy_version 1277181 (0.0006) [2023-12-27 00:41:18,921][105692] Updated weights for policy 0, policy_version 1277191 (0.0010) [2023-12-27 00:41:19,200][105620] Updated weights for policy 1, policy_version 1278630 (0.0009) [2023-12-27 00:41:19,264][105620] Updated weights for policy 1, policy_version 1278640 (0.0008) [2023-12-27 00:41:19,317][105620] Updated weights for policy 1, policy_version 1278650 (0.0008) [2023-12-27 00:41:19,594][105692] Updated weights for policy 0, policy_version 1277201 (0.0010) [2023-12-27 00:41:19,651][105692] Updated weights for policy 0, policy_version 1277211 (0.0009) [2023-12-27 00:41:19,712][105692] Updated weights for policy 0, policy_version 1277221 (0.0009) [2023-12-27 00:41:19,771][105692] Updated weights for policy 0, policy_version 1277231 (0.0011) [2023-12-27 00:41:20,005][105620] Updated weights for policy 1, policy_version 1278660 (0.0008) [2023-12-27 00:41:20,073][105620] Updated weights for policy 1, policy_version 1278670 (0.0006) [2023-12-27 00:41:20,149][105620] Updated weights for policy 1, policy_version 1278680 (0.0005) [2023-12-27 00:41:20,514][105692] Updated weights for policy 0, policy_version 1277242 (0.0007) [2023-12-27 00:41:20,576][105692] Updated weights for policy 0, policy_version 1277252 (0.0007) [2023-12-27 00:41:20,628][105692] Updated weights for policy 0, policy_version 1277262 (0.0008) [2023-12-27 00:41:20,866][105620] Updated weights for policy 1, policy_version 1278690 (0.0009) [2023-12-27 00:41:20,930][105620] Updated weights for policy 1, policy_version 1278700 (0.0009) [2023-12-27 00:41:20,986][105620] Updated weights for policy 1, policy_version 1278710 (0.0009) [2023-12-27 00:41:21,049][105620] Updated weights for policy 1, policy_version 1278720 (0.0009) [2023-12-27 00:41:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 654426112. Throughput: 0: 9694.2, 1: 9770.7. Samples: 654413556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:41:21,062][104569] Avg episode reward: [(0, '8902.975'), (1, '9081.825')] [2023-12-27 00:41:21,336][105692] Updated weights for policy 0, policy_version 1277272 (0.0007) [2023-12-27 00:41:21,403][105692] Updated weights for policy 0, policy_version 1277282 (0.0008) [2023-12-27 00:41:21,473][105692] Updated weights for policy 0, policy_version 1277292 (0.0008) [2023-12-27 00:41:21,898][105620] Updated weights for policy 1, policy_version 1278730 (0.0010) [2023-12-27 00:41:21,958][105620] Updated weights for policy 1, policy_version 1278740 (0.0009) [2023-12-27 00:41:22,024][105620] Updated weights for policy 1, policy_version 1278750 (0.0009) [2023-12-27 00:41:22,193][105692] Updated weights for policy 0, policy_version 1277302 (0.0009) [2023-12-27 00:41:22,253][105692] Updated weights for policy 0, policy_version 1277312 (0.0009) [2023-12-27 00:41:22,319][105692] Updated weights for policy 0, policy_version 1277322 (0.0008) [2023-12-27 00:41:22,780][105620] Updated weights for policy 1, policy_version 1278760 (0.0006) [2023-12-27 00:41:22,846][105620] Updated weights for policy 1, policy_version 1278770 (0.0006) [2023-12-27 00:41:22,863][105586] KL-divergence is very high: 180.3475 [2023-12-27 00:41:22,904][105620] Updated weights for policy 1, policy_version 1278780 (0.0009) [2023-12-27 00:41:22,911][105586] KL-divergence is very high: 212.6241 [2023-12-27 00:41:23,070][105692] Updated weights for policy 0, policy_version 1277332 (0.0008) [2023-12-27 00:41:23,122][105692] Updated weights for policy 0, policy_version 1277342 (0.0005) [2023-12-27 00:41:23,174][105692] Updated weights for policy 0, policy_version 1277352 (0.0007) [2023-12-27 00:41:23,492][105620] Updated weights for policy 1, policy_version 1278790 (0.0007) [2023-12-27 00:41:23,559][105620] Updated weights for policy 1, policy_version 1278800 (0.0005) [2023-12-27 00:41:23,616][105620] Updated weights for policy 1, policy_version 1278810 (0.0006) [2023-12-27 00:41:23,852][105692] Updated weights for policy 0, policy_version 1277362 (0.0009) [2023-12-27 00:41:23,903][105692] Updated weights for policy 0, policy_version 1277373 (0.0009) [2023-12-27 00:41:23,960][105692] Updated weights for policy 0, policy_version 1277383 (0.0005) [2023-12-27 00:41:24,192][105620] Updated weights for policy 1, policy_version 1278820 (0.0009) [2023-12-27 00:41:24,242][105620] Updated weights for policy 1, policy_version 1278830 (0.0008) [2023-12-27 00:41:24,297][105620] Updated weights for policy 1, policy_version 1278840 (0.0009) [2023-12-27 00:41:24,662][105692] Updated weights for policy 0, policy_version 1277393 (0.0006) [2023-12-27 00:41:24,707][105692] Updated weights for policy 0, policy_version 1277403 (0.0009) [2023-12-27 00:41:24,766][105692] Updated weights for policy 0, policy_version 1277413 (0.0005) [2023-12-27 00:41:24,830][105692] Updated weights for policy 0, policy_version 1277423 (0.0008) [2023-12-27 00:41:24,905][105620] Updated weights for policy 1, policy_version 1278850 (0.0007) [2023-12-27 00:41:24,961][105620] Updated weights for policy 1, policy_version 1278860 (0.0005) [2023-12-27 00:41:25,009][105620] Updated weights for policy 1, policy_version 1278870 (0.0005) [2023-12-27 00:41:25,056][105620] Updated weights for policy 1, policy_version 1278880 (0.0005) [2023-12-27 00:41:25,468][105692] Updated weights for policy 0, policy_version 1277433 (0.0010) [2023-12-27 00:41:25,533][105692] Updated weights for policy 0, policy_version 1277443 (0.0010) [2023-12-27 00:41:25,587][105620] Updated weights for policy 1, policy_version 1278890 (0.0010) [2023-12-27 00:41:25,588][105692] Updated weights for policy 0, policy_version 1277453 (0.0010) [2023-12-27 00:41:25,632][105620] Updated weights for policy 1, policy_version 1278900 (0.0010) [2023-12-27 00:41:25,683][105620] Updated weights for policy 1, policy_version 1278910 (0.0010) [2023-12-27 00:41:26,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 654524416. Throughput: 0: 9695.5, 1: 9821.5. Samples: 654533692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:41:26,063][104569] Avg episode reward: [(0, '8812.469'), (1, '8818.628')] [2023-12-27 00:41:26,127][105692] Updated weights for policy 0, policy_version 1277463 (0.0007) [2023-12-27 00:41:26,177][105692] Updated weights for policy 0, policy_version 1277473 (0.0010) [2023-12-27 00:41:26,236][105692] Updated weights for policy 0, policy_version 1277483 (0.0010) [2023-12-27 00:41:26,385][105620] Updated weights for policy 1, policy_version 1278920 (0.0006) [2023-12-27 00:41:26,440][105620] Updated weights for policy 1, policy_version 1278930 (0.0006) [2023-12-27 00:41:26,502][105620] Updated weights for policy 1, policy_version 1278940 (0.0005) [2023-12-27 00:41:26,866][105692] Updated weights for policy 0, policy_version 1277493 (0.0008) [2023-12-27 00:41:26,918][105692] Updated weights for policy 0, policy_version 1277503 (0.0009) [2023-12-27 00:41:26,970][105692] Updated weights for policy 0, policy_version 1277514 (0.0010) [2023-12-27 00:41:26,987][105620] Updated weights for policy 1, policy_version 1278950 (0.0005) [2023-12-27 00:41:27,031][105620] Updated weights for policy 1, policy_version 1278960 (0.0005) [2023-12-27 00:41:27,080][105620] Updated weights for policy 1, policy_version 1278970 (0.0008) [2023-12-27 00:41:27,556][105692] Updated weights for policy 0, policy_version 1277525 (0.0008) [2023-12-27 00:41:27,603][105692] Updated weights for policy 0, policy_version 1277535 (0.0005) [2023-12-27 00:41:27,655][105692] Updated weights for policy 0, policy_version 1277545 (0.0005) [2023-12-27 00:41:27,666][105620] Updated weights for policy 1, policy_version 1278980 (0.0009) [2023-12-27 00:41:27,737][105620] Updated weights for policy 1, policy_version 1278990 (0.0007) [2023-12-27 00:41:27,788][105620] Updated weights for policy 1, policy_version 1279000 (0.0008) [2023-12-27 00:41:28,375][105692] Updated weights for policy 0, policy_version 1277555 (0.0007) [2023-12-27 00:41:28,432][105692] Updated weights for policy 0, policy_version 1277565 (0.0010) [2023-12-27 00:41:28,484][105692] Updated weights for policy 0, policy_version 1277575 (0.0010) [2023-12-27 00:41:28,503][105620] Updated weights for policy 1, policy_version 1279010 (0.0008) [2023-12-27 00:41:28,553][105620] Updated weights for policy 1, policy_version 1279020 (0.0005) [2023-12-27 00:41:28,615][105620] Updated weights for policy 1, policy_version 1279030 (0.0009) [2023-12-27 00:41:28,675][105620] Updated weights for policy 1, policy_version 1279040 (0.0008) [2023-12-27 00:41:29,204][105692] Updated weights for policy 0, policy_version 1277585 (0.0008) [2023-12-27 00:41:29,270][105692] Updated weights for policy 0, policy_version 1277595 (0.0008) [2023-12-27 00:41:29,279][105620] Updated weights for policy 1, policy_version 1279050 (0.0007) [2023-12-27 00:41:29,340][105692] Updated weights for policy 0, policy_version 1277605 (0.0008) [2023-12-27 00:41:29,340][105620] Updated weights for policy 1, policy_version 1279060 (0.0008) [2023-12-27 00:41:29,399][105692] Updated weights for policy 0, policy_version 1277615 (0.0008) [2023-12-27 00:41:29,403][105620] Updated weights for policy 1, policy_version 1279070 (0.0006) [2023-12-27 00:41:30,075][105620] Updated weights for policy 1, policy_version 1279080 (0.0005) [2023-12-27 00:41:30,076][105692] Updated weights for policy 0, policy_version 1277625 (0.0010) [2023-12-27 00:41:30,137][105692] Updated weights for policy 0, policy_version 1277635 (0.0010) [2023-12-27 00:41:30,140][105620] Updated weights for policy 1, policy_version 1279090 (0.0005) [2023-12-27 00:41:30,193][105692] Updated weights for policy 0, policy_version 1277645 (0.0010) [2023-12-27 00:41:30,195][105620] Updated weights for policy 1, policy_version 1279100 (0.0006) [2023-12-27 00:41:30,846][105620] Updated weights for policy 1, policy_version 1279110 (0.0006) [2023-12-27 00:41:30,859][105692] Updated weights for policy 0, policy_version 1277655 (0.0009) [2023-12-27 00:41:30,896][105620] Updated weights for policy 1, policy_version 1279120 (0.0006) [2023-12-27 00:41:30,905][105692] Updated weights for policy 0, policy_version 1277665 (0.0009) [2023-12-27 00:41:30,943][105620] Updated weights for policy 1, policy_version 1279130 (0.0006) [2023-12-27 00:41:30,950][105692] Updated weights for policy 0, policy_version 1277675 (0.0007) [2023-12-27 00:41:31,062][104569] Fps is (10 sec: 21299.3, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 654639104. Throughput: 0: 9836.9, 1: 9946.0. Samples: 654600800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:41:31,062][104569] Avg episode reward: [(0, '8722.655'), (1, '9173.396')] [2023-12-27 00:41:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001277680_327139328.pth... [2023-12-27 00:41:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001279136_327499776.pth... [2023-12-27 00:41:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001276528_326844416.pth [2023-12-27 00:41:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001277920_327188480.pth [2023-12-27 00:41:31,670][105620] Updated weights for policy 1, policy_version 1279140 (0.0006) [2023-12-27 00:41:31,731][105620] Updated weights for policy 1, policy_version 1279150 (0.0008) [2023-12-27 00:41:31,760][105692] Updated weights for policy 0, policy_version 1277685 (0.0009) [2023-12-27 00:41:31,793][105620] Updated weights for policy 1, policy_version 1279160 (0.0008) [2023-12-27 00:41:31,812][105692] Updated weights for policy 0, policy_version 1277695 (0.0010) [2023-12-27 00:41:31,867][105692] Updated weights for policy 0, policy_version 1277705 (0.0010) [2023-12-27 00:41:32,561][105620] Updated weights for policy 1, policy_version 1279170 (0.0007) [2023-12-27 00:41:32,609][105620] Updated weights for policy 1, policy_version 1279180 (0.0008) [2023-12-27 00:41:32,626][105692] Updated weights for policy 0, policy_version 1277715 (0.0011) [2023-12-27 00:41:32,657][105620] Updated weights for policy 1, policy_version 1279190 (0.0008) [2023-12-27 00:41:32,677][105692] Updated weights for policy 0, policy_version 1277725 (0.0010) [2023-12-27 00:41:32,703][105620] Updated weights for policy 1, policy_version 1279200 (0.0007) [2023-12-27 00:41:32,737][105692] Updated weights for policy 0, policy_version 1277735 (0.0010) [2023-12-27 00:41:33,481][105692] Updated weights for policy 0, policy_version 1277745 (0.0011) [2023-12-27 00:41:33,483][105620] Updated weights for policy 1, policy_version 1279210 (0.0008) [2023-12-27 00:41:33,537][105620] Updated weights for policy 1, policy_version 1279220 (0.0006) [2023-12-27 00:41:33,539][105692] Updated weights for policy 0, policy_version 1277755 (0.0010) [2023-12-27 00:41:33,587][105620] Updated weights for policy 1, policy_version 1279230 (0.0005) [2023-12-27 00:41:33,592][105692] Updated weights for policy 0, policy_version 1277765 (0.0010) [2023-12-27 00:41:33,640][105692] Updated weights for policy 0, policy_version 1277775 (0.0010) [2023-12-27 00:41:34,351][105620] Updated weights for policy 1, policy_version 1279240 (0.0008) [2023-12-27 00:41:34,403][105620] Updated weights for policy 1, policy_version 1279250 (0.0007) [2023-12-27 00:41:34,431][105692] Updated weights for policy 0, policy_version 1277785 (0.0010) [2023-12-27 00:41:34,463][105620] Updated weights for policy 1, policy_version 1279260 (0.0009) [2023-12-27 00:41:34,494][105692] Updated weights for policy 0, policy_version 1277795 (0.0010) [2023-12-27 00:41:34,558][105692] Updated weights for policy 0, policy_version 1277805 (0.0010) [2023-12-27 00:41:35,169][105620] Updated weights for policy 1, policy_version 1279270 (0.0007) [2023-12-27 00:41:35,204][105692] Updated weights for policy 0, policy_version 1277815 (0.0011) [2023-12-27 00:41:35,228][105620] Updated weights for policy 1, policy_version 1279280 (0.0006) [2023-12-27 00:41:35,257][105692] Updated weights for policy 0, policy_version 1277825 (0.0011) [2023-12-27 00:41:35,287][105620] Updated weights for policy 1, policy_version 1279290 (0.0011) [2023-12-27 00:41:35,312][105692] Updated weights for policy 0, policy_version 1277835 (0.0011) [2023-12-27 00:41:35,899][105620] Updated weights for policy 1, policy_version 1279300 (0.0010) [2023-12-27 00:41:35,948][105620] Updated weights for policy 1, policy_version 1279310 (0.0007) [2023-12-27 00:41:36,011][105620] Updated weights for policy 1, policy_version 1279320 (0.0005) [2023-12-27 00:41:36,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 654729216. Throughput: 0: 9736.3, 1: 9962.8. Samples: 654716116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:41:36,063][104569] Avg episode reward: [(0, '7584.741'), (1, '9261.883')] [2023-12-27 00:41:36,078][105692] Updated weights for policy 0, policy_version 1277845 (0.0010) [2023-12-27 00:41:36,142][105692] Updated weights for policy 0, policy_version 1277855 (0.0011) [2023-12-27 00:41:36,209][105692] Updated weights for policy 0, policy_version 1277865 (0.0011) [2023-12-27 00:41:36,746][105620] Updated weights for policy 1, policy_version 1279330 (0.0005) [2023-12-27 00:41:36,802][105620] Updated weights for policy 1, policy_version 1279340 (0.0006) [2023-12-27 00:41:36,856][105620] Updated weights for policy 1, policy_version 1279350 (0.0006) [2023-12-27 00:41:36,910][105692] Updated weights for policy 0, policy_version 1277875 (0.0011) [2023-12-27 00:41:36,917][105620] Updated weights for policy 1, policy_version 1279360 (0.0007) [2023-12-27 00:41:36,962][105692] Updated weights for policy 0, policy_version 1277885 (0.0011) [2023-12-27 00:41:37,016][105692] Updated weights for policy 0, policy_version 1277895 (0.0010) [2023-12-27 00:41:37,536][105620] Updated weights for policy 1, policy_version 1279370 (0.0008) [2023-12-27 00:41:37,596][105620] Updated weights for policy 1, policy_version 1279380 (0.0008) [2023-12-27 00:41:37,655][105620] Updated weights for policy 1, policy_version 1279390 (0.0008) [2023-12-27 00:41:37,768][105692] Updated weights for policy 0, policy_version 1277905 (0.0010) [2023-12-27 00:41:37,816][105692] Updated weights for policy 0, policy_version 1277915 (0.0005) [2023-12-27 00:41:37,869][105692] Updated weights for policy 0, policy_version 1277925 (0.0005) [2023-12-27 00:41:37,917][105692] Updated weights for policy 0, policy_version 1277935 (0.0006) [2023-12-27 00:41:38,396][105620] Updated weights for policy 1, policy_version 1279400 (0.0009) [2023-12-27 00:41:38,459][105620] Updated weights for policy 1, policy_version 1279410 (0.0011) [2023-12-27 00:41:38,522][105620] Updated weights for policy 1, policy_version 1279420 (0.0011) [2023-12-27 00:41:38,533][105692] Updated weights for policy 0, policy_version 1277945 (0.0006) [2023-12-27 00:41:38,588][105692] Updated weights for policy 0, policy_version 1277955 (0.0008) [2023-12-27 00:41:38,638][105692] Updated weights for policy 0, policy_version 1277965 (0.0008) [2023-12-27 00:41:39,264][105620] Updated weights for policy 1, policy_version 1279430 (0.0010) [2023-12-27 00:41:39,322][105620] Updated weights for policy 1, policy_version 1279440 (0.0011) [2023-12-27 00:41:39,376][105692] Updated weights for policy 0, policy_version 1277975 (0.0010) [2023-12-27 00:41:39,389][105620] Updated weights for policy 1, policy_version 1279450 (0.0007) [2023-12-27 00:41:39,436][105692] Updated weights for policy 0, policy_version 1277985 (0.0010) [2023-12-27 00:41:39,503][105692] Updated weights for policy 0, policy_version 1277995 (0.0011) [2023-12-27 00:41:40,155][105620] Updated weights for policy 1, policy_version 1279460 (0.0007) [2023-12-27 00:41:40,216][105620] Updated weights for policy 1, policy_version 1279470 (0.0007) [2023-12-27 00:41:40,216][105692] Updated weights for policy 0, policy_version 1278005 (0.0009) [2023-12-27 00:41:40,285][105692] Updated weights for policy 0, policy_version 1278015 (0.0008) [2023-12-27 00:41:40,287][105620] Updated weights for policy 1, policy_version 1279480 (0.0006) [2023-12-27 00:41:40,356][105692] Updated weights for policy 0, policy_version 1278025 (0.0007) [2023-12-27 00:41:40,979][105620] Updated weights for policy 1, policy_version 1279490 (0.0008) [2023-12-27 00:41:41,040][105620] Updated weights for policy 1, policy_version 1279500 (0.0011) [2023-12-27 00:41:41,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 654819328. Throughput: 0: 9791.6, 1: 10023.2. Samples: 654834776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:41:41,063][104569] Avg episode reward: [(0, '7507.661'), (1, '9353.321')] [2023-12-27 00:41:41,108][105620] Updated weights for policy 1, policy_version 1279510 (0.0010) [2023-12-27 00:41:41,147][105692] Updated weights for policy 0, policy_version 1278035 (0.0008) [2023-12-27 00:41:41,173][105620] Updated weights for policy 1, policy_version 1279520 (0.0008) [2023-12-27 00:41:41,213][105692] Updated weights for policy 0, policy_version 1278045 (0.0009) [2023-12-27 00:41:41,282][105692] Updated weights for policy 0, policy_version 1278055 (0.0009) [2023-12-27 00:41:41,919][105620] Updated weights for policy 1, policy_version 1279530 (0.0008) [2023-12-27 00:41:41,976][105620] Updated weights for policy 1, policy_version 1279540 (0.0011) [2023-12-27 00:41:42,033][105620] Updated weights for policy 1, policy_version 1279550 (0.0008) [2023-12-27 00:41:42,094][105692] Updated weights for policy 0, policy_version 1278065 (0.0008) [2023-12-27 00:41:42,147][105692] Updated weights for policy 0, policy_version 1278075 (0.0008) [2023-12-27 00:41:42,196][105692] Updated weights for policy 0, policy_version 1278085 (0.0008) [2023-12-27 00:41:42,254][105692] Updated weights for policy 0, policy_version 1278095 (0.0009) [2023-12-27 00:41:42,818][105620] Updated weights for policy 1, policy_version 1279560 (0.0010) [2023-12-27 00:41:42,883][105620] Updated weights for policy 1, policy_version 1279570 (0.0010) [2023-12-27 00:41:42,948][105620] Updated weights for policy 1, policy_version 1279580 (0.0006) [2023-12-27 00:41:43,015][105692] Updated weights for policy 0, policy_version 1278105 (0.0007) [2023-12-27 00:41:43,067][105692] Updated weights for policy 0, policy_version 1278115 (0.0005) [2023-12-27 00:41:43,125][105692] Updated weights for policy 0, policy_version 1278125 (0.0008) [2023-12-27 00:41:43,661][105620] Updated weights for policy 1, policy_version 1279590 (0.0008) [2023-12-27 00:41:43,713][105620] Updated weights for policy 1, policy_version 1279600 (0.0010) [2023-12-27 00:41:43,768][105620] Updated weights for policy 1, policy_version 1279610 (0.0010) [2023-12-27 00:41:43,852][105692] Updated weights for policy 0, policy_version 1278135 (0.0006) [2023-12-27 00:41:43,918][105692] Updated weights for policy 0, policy_version 1278145 (0.0005) [2023-12-27 00:41:43,987][105692] Updated weights for policy 0, policy_version 1278155 (0.0005) [2023-12-27 00:41:44,471][105692] Updated weights for policy 0, policy_version 1278165 (0.0008) [2023-12-27 00:41:44,484][105620] Updated weights for policy 1, policy_version 1279620 (0.0008) [2023-12-27 00:41:44,516][105692] Updated weights for policy 0, policy_version 1278175 (0.0008) [2023-12-27 00:41:44,534][105620] Updated weights for policy 1, policy_version 1279630 (0.0009) [2023-12-27 00:41:44,561][105692] Updated weights for policy 0, policy_version 1278185 (0.0005) [2023-12-27 00:41:44,590][105620] Updated weights for policy 1, policy_version 1279640 (0.0011) [2023-12-27 00:41:45,265][105620] Updated weights for policy 1, policy_version 1279650 (0.0008) [2023-12-27 00:41:45,328][105620] Updated weights for policy 1, policy_version 1279660 (0.0011) [2023-12-27 00:41:45,381][105692] Updated weights for policy 0, policy_version 1278195 (0.0006) [2023-12-27 00:41:45,392][105620] Updated weights for policy 1, policy_version 1279670 (0.0011) [2023-12-27 00:41:45,442][105692] Updated weights for policy 0, policy_version 1278205 (0.0008) [2023-12-27 00:41:45,457][105620] Updated weights for policy 1, policy_version 1279680 (0.0007) [2023-12-27 00:41:45,499][105692] Updated weights for policy 0, policy_version 1278215 (0.0009) [2023-12-27 00:41:46,061][105620] Updated weights for policy 1, policy_version 1279690 (0.0006) [2023-12-27 00:41:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 654917632. Throughput: 0: 9723.9, 1: 10050.6. Samples: 654889536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:41:46,063][104569] Avg episode reward: [(0, '8317.508'), (1, '9353.432')] [2023-12-27 00:41:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001278224_327278592.pth... [2023-12-27 00:41:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001277072_326983680.pth [2023-12-27 00:41:46,074][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_001278224_327278592.pth [2023-12-27 00:41:46,128][105620] Updated weights for policy 1, policy_version 1279700 (0.0006) [2023-12-27 00:41:46,182][105620] Updated weights for policy 1, policy_version 1279710 (0.0005) [2023-12-27 00:41:46,193][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001279712_327647232.pth... [2023-12-27 00:41:46,196][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001278528_327344128.pth [2023-12-27 00:41:46,197][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_001279712_327647232.pth [2023-12-27 00:41:46,366][105692] Updated weights for policy 0, policy_version 1278225 (0.0010) [2023-12-27 00:41:46,428][105692] Updated weights for policy 0, policy_version 1278235 (0.0009) [2023-12-27 00:41:46,492][105692] Updated weights for policy 0, policy_version 1278245 (0.0010) [2023-12-27 00:41:46,571][105692] Updated weights for policy 0, policy_version 1278255 (0.0009) [2023-12-27 00:41:46,681][105620] Updated weights for policy 1, policy_version 1279720 (0.0005) [2023-12-27 00:41:46,732][105620] Updated weights for policy 1, policy_version 1279730 (0.0011) [2023-12-27 00:41:46,780][105620] Updated weights for policy 1, policy_version 1279740 (0.0008) [2023-12-27 00:41:47,354][105620] Updated weights for policy 1, policy_version 1279750 (0.0006) [2023-12-27 00:41:47,359][105692] Updated weights for policy 0, policy_version 1278265 (0.0008) [2023-12-27 00:41:47,402][105620] Updated weights for policy 1, policy_version 1279760 (0.0007) [2023-12-27 00:41:47,417][105692] Updated weights for policy 0, policy_version 1278275 (0.0009) [2023-12-27 00:41:47,450][105620] Updated weights for policy 1, policy_version 1279770 (0.0005) [2023-12-27 00:41:47,481][105692] Updated weights for policy 0, policy_version 1278285 (0.0007) [2023-12-27 00:41:48,072][105692] Updated weights for policy 0, policy_version 1278295 (0.0007) [2023-12-27 00:41:48,127][105692] Updated weights for policy 0, policy_version 1278305 (0.0005) [2023-12-27 00:41:48,185][105692] Updated weights for policy 0, policy_version 1278315 (0.0005) [2023-12-27 00:41:48,282][105620] Updated weights for policy 1, policy_version 1279780 (0.0007) [2023-12-27 00:41:48,342][105620] Updated weights for policy 1, policy_version 1279790 (0.0009) [2023-12-27 00:41:48,415][105620] Updated weights for policy 1, policy_version 1279800 (0.0011) [2023-12-27 00:41:48,819][105692] Updated weights for policy 0, policy_version 1278325 (0.0008) [2023-12-27 00:41:48,878][105692] Updated weights for policy 0, policy_version 1278335 (0.0011) [2023-12-27 00:41:48,926][105692] Updated weights for policy 0, policy_version 1278345 (0.0010) [2023-12-27 00:41:49,160][105620] Updated weights for policy 1, policy_version 1279810 (0.0011) [2023-12-27 00:41:49,213][105620] Updated weights for policy 1, policy_version 1279820 (0.0008) [2023-12-27 00:41:49,282][105620] Updated weights for policy 1, policy_version 1279830 (0.0007) [2023-12-27 00:41:49,339][105620] Updated weights for policy 1, policy_version 1279840 (0.0008) [2023-12-27 00:41:49,701][105692] Updated weights for policy 0, policy_version 1278355 (0.0010) [2023-12-27 00:41:49,751][105692] Updated weights for policy 0, policy_version 1278365 (0.0009) [2023-12-27 00:41:49,807][105692] Updated weights for policy 0, policy_version 1278375 (0.0009) [2023-12-27 00:41:49,977][105620] Updated weights for policy 1, policy_version 1279850 (0.0007) [2023-12-27 00:41:50,039][105620] Updated weights for policy 1, policy_version 1279860 (0.0007) [2023-12-27 00:41:50,090][105620] Updated weights for policy 1, policy_version 1279870 (0.0009) [2023-12-27 00:41:50,600][105692] Updated weights for policy 0, policy_version 1278385 (0.0007) [2023-12-27 00:41:50,659][105692] Updated weights for policy 0, policy_version 1278395 (0.0006) [2023-12-27 00:41:50,733][105692] Updated weights for policy 0, policy_version 1278405 (0.0006) [2023-12-27 00:41:50,754][105620] Updated weights for policy 1, policy_version 1279880 (0.0006) [2023-12-27 00:41:50,805][105692] Updated weights for policy 0, policy_version 1278415 (0.0007) [2023-12-27 00:41:50,814][105620] Updated weights for policy 1, policy_version 1279890 (0.0006) [2023-12-27 00:41:50,879][105620] Updated weights for policy 1, policy_version 1279900 (0.0007) [2023-12-27 00:41:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 655024128. Throughput: 0: 9792.6, 1: 10200.3. Samples: 655011016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:41:51,062][104569] Avg episode reward: [(0, '8911.110'), (1, '9353.850')] [2023-12-27 00:41:51,361][105692] Updated weights for policy 0, policy_version 1278425 (0.0007) [2023-12-27 00:41:51,428][105692] Updated weights for policy 0, policy_version 1278435 (0.0008) [2023-12-27 00:41:51,503][105692] Updated weights for policy 0, policy_version 1278445 (0.0005) [2023-12-27 00:41:51,610][105620] Updated weights for policy 1, policy_version 1279910 (0.0008) [2023-12-27 00:41:51,676][105620] Updated weights for policy 1, policy_version 1279920 (0.0008) [2023-12-27 00:41:51,742][105620] Updated weights for policy 1, policy_version 1279930 (0.0009) [2023-12-27 00:41:52,219][105692] Updated weights for policy 0, policy_version 1278455 (0.0009) [2023-12-27 00:41:52,269][105692] Updated weights for policy 0, policy_version 1278465 (0.0008) [2023-12-27 00:41:52,322][105692] Updated weights for policy 0, policy_version 1278475 (0.0006) [2023-12-27 00:41:52,457][105620] Updated weights for policy 1, policy_version 1279940 (0.0009) [2023-12-27 00:41:52,515][105620] Updated weights for policy 1, policy_version 1279950 (0.0010) [2023-12-27 00:41:52,578][105620] Updated weights for policy 1, policy_version 1279960 (0.0009) [2023-12-27 00:41:52,983][105692] Updated weights for policy 0, policy_version 1278485 (0.0010) [2023-12-27 00:41:53,039][105692] Updated weights for policy 0, policy_version 1278495 (0.0009) [2023-12-27 00:41:53,093][105692] Updated weights for policy 0, policy_version 1278506 (0.0010) [2023-12-27 00:41:53,252][105620] Updated weights for policy 1, policy_version 1279970 (0.0007) [2023-12-27 00:41:53,320][105620] Updated weights for policy 1, policy_version 1279980 (0.0006) [2023-12-27 00:41:53,392][105620] Updated weights for policy 1, policy_version 1279990 (0.0005) [2023-12-27 00:41:53,463][105620] Updated weights for policy 1, policy_version 1280000 (0.0006) [2023-12-27 00:41:53,707][105692] Updated weights for policy 0, policy_version 1278516 (0.0008) [2023-12-27 00:41:53,767][105692] Updated weights for policy 0, policy_version 1278526 (0.0006) [2023-12-27 00:41:53,829][105692] Updated weights for policy 0, policy_version 1278536 (0.0005) [2023-12-27 00:41:54,038][105620] Updated weights for policy 1, policy_version 1280010 (0.0010) [2023-12-27 00:41:54,090][105620] Updated weights for policy 1, policy_version 1280020 (0.0010) [2023-12-27 00:41:54,139][105620] Updated weights for policy 1, policy_version 1280030 (0.0010) [2023-12-27 00:41:54,438][105692] Updated weights for policy 0, policy_version 1278546 (0.0009) [2023-12-27 00:41:54,493][105692] Updated weights for policy 0, policy_version 1278556 (0.0010) [2023-12-27 00:41:54,545][105692] Updated weights for policy 0, policy_version 1278566 (0.0010) [2023-12-27 00:41:54,595][105692] Updated weights for policy 0, policy_version 1278576 (0.0010) [2023-12-27 00:41:54,786][105620] Updated weights for policy 1, policy_version 1280040 (0.0008) [2023-12-27 00:41:54,839][105620] Updated weights for policy 1, policy_version 1280050 (0.0006) [2023-12-27 00:41:54,904][105620] Updated weights for policy 1, policy_version 1280060 (0.0006) [2023-12-27 00:41:55,211][105692] Updated weights for policy 0, policy_version 1278586 (0.0006) [2023-12-27 00:41:55,276][105692] Updated weights for policy 0, policy_version 1278596 (0.0005) [2023-12-27 00:41:55,341][105692] Updated weights for policy 0, policy_version 1278606 (0.0006) [2023-12-27 00:41:55,452][105620] Updated weights for policy 1, policy_version 1280070 (0.0005) [2023-12-27 00:41:55,496][105620] Updated weights for policy 1, policy_version 1280080 (0.0005) [2023-12-27 00:41:55,553][105620] Updated weights for policy 1, policy_version 1280090 (0.0007) [2023-12-27 00:41:55,861][105692] Updated weights for policy 0, policy_version 1278616 (0.0005) [2023-12-27 00:41:55,927][105692] Updated weights for policy 0, policy_version 1278626 (0.0006) [2023-12-27 00:41:55,990][105692] Updated weights for policy 0, policy_version 1278636 (0.0006) [2023-12-27 00:41:56,062][104569] Fps is (10 sec: 21299.2, 60 sec: 20070.4, 300 sec: 19605.3). Total num frames: 655130624. Throughput: 0: 9917.2, 1: 10194.0. Samples: 655136428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:41:56,063][104569] Avg episode reward: [(0, '8637.782'), (1, '9354.131')] [2023-12-27 00:41:56,243][105620] Updated weights for policy 1, policy_version 1280100 (0.0010) [2023-12-27 00:41:56,287][105620] Updated weights for policy 1, policy_version 1280110 (0.0010) [2023-12-27 00:41:56,333][105620] Updated weights for policy 1, policy_version 1280120 (0.0010) [2023-12-27 00:41:56,672][105692] Updated weights for policy 0, policy_version 1278646 (0.0006) [2023-12-27 00:41:56,729][105692] Updated weights for policy 0, policy_version 1278656 (0.0005) [2023-12-27 00:41:56,783][105692] Updated weights for policy 0, policy_version 1278666 (0.0006) [2023-12-27 00:41:57,051][105620] Updated weights for policy 1, policy_version 1280130 (0.0009) [2023-12-27 00:41:57,095][105620] Updated weights for policy 1, policy_version 1280140 (0.0005) [2023-12-27 00:41:57,141][105620] Updated weights for policy 1, policy_version 1280150 (0.0005) [2023-12-27 00:41:57,192][105620] Updated weights for policy 1, policy_version 1280160 (0.0005) [2023-12-27 00:41:57,486][105692] Updated weights for policy 0, policy_version 1278676 (0.0008) [2023-12-27 00:41:57,540][105692] Updated weights for policy 0, policy_version 1278686 (0.0010) [2023-12-27 00:41:57,587][105692] Updated weights for policy 0, policy_version 1278696 (0.0010) [2023-12-27 00:41:57,788][105620] Updated weights for policy 1, policy_version 1280170 (0.0010) [2023-12-27 00:41:57,835][105620] Updated weights for policy 1, policy_version 1280180 (0.0010) [2023-12-27 00:41:57,893][105620] Updated weights for policy 1, policy_version 1280190 (0.0010) [2023-12-27 00:41:58,306][105692] Updated weights for policy 0, policy_version 1278706 (0.0010) [2023-12-27 00:41:58,375][105692] Updated weights for policy 0, policy_version 1278716 (0.0008) [2023-12-27 00:41:58,440][105692] Updated weights for policy 0, policy_version 1278726 (0.0008) [2023-12-27 00:41:58,503][105692] Updated weights for policy 0, policy_version 1278736 (0.0008) [2023-12-27 00:41:58,653][105620] Updated weights for policy 1, policy_version 1280200 (0.0011) [2023-12-27 00:41:58,713][105620] Updated weights for policy 1, policy_version 1280210 (0.0011) [2023-12-27 00:41:58,790][105620] Updated weights for policy 1, policy_version 1280220 (0.0009) [2023-12-27 00:41:59,363][105692] Updated weights for policy 0, policy_version 1278746 (0.0008) [2023-12-27 00:41:59,432][105692] Updated weights for policy 0, policy_version 1278756 (0.0009) [2023-12-27 00:41:59,484][105692] Updated weights for policy 0, policy_version 1278768 (0.0009) [2023-12-27 00:41:59,606][105620] Updated weights for policy 1, policy_version 1280230 (0.0009) [2023-12-27 00:41:59,664][105620] Updated weights for policy 1, policy_version 1280240 (0.0007) [2023-12-27 00:41:59,719][105620] Updated weights for policy 1, policy_version 1280250 (0.0009) [2023-12-27 00:42:00,300][105692] Updated weights for policy 0, policy_version 1278778 (0.0009) [2023-12-27 00:42:00,329][105620] Updated weights for policy 1, policy_version 1280260 (0.0005) [2023-12-27 00:42:00,349][105692] Updated weights for policy 0, policy_version 1278788 (0.0009) [2023-12-27 00:42:00,392][105620] Updated weights for policy 1, policy_version 1280270 (0.0007) [2023-12-27 00:42:00,412][105692] Updated weights for policy 0, policy_version 1278798 (0.0008) [2023-12-27 00:42:00,456][105620] Updated weights for policy 1, policy_version 1280280 (0.0007) [2023-12-27 00:42:01,002][105620] Updated weights for policy 1, policy_version 1280290 (0.0006) [2023-12-27 00:42:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19933.8, 300 sec: 19577.5). Total num frames: 655220736. Throughput: 0: 10001.1, 1: 10195.3. Samples: 655196064. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:42:01,063][104569] Avg episode reward: [(0, '8727.770'), (1, '9354.387')] [2023-12-27 00:42:01,066][105620] Updated weights for policy 1, policy_version 1280300 (0.0008) [2023-12-27 00:42:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001278800_327426048.pth... [2023-12-27 00:42:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001277680_327139328.pth [2023-12-27 00:42:01,126][105620] Updated weights for policy 1, policy_version 1280310 (0.0009) [2023-12-27 00:42:01,191][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001280320_327802880.pth... [2023-12-27 00:42:01,191][105620] Updated weights for policy 1, policy_version 1280320 (0.0008) [2023-12-27 00:42:01,195][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001279136_327499776.pth [2023-12-27 00:42:01,263][105692] Updated weights for policy 0, policy_version 1278808 (0.0008) [2023-12-27 00:42:01,326][105692] Updated weights for policy 0, policy_version 1278818 (0.0009) [2023-12-27 00:42:01,385][105692] Updated weights for policy 0, policy_version 1278828 (0.0009) [2023-12-27 00:42:01,860][105620] Updated weights for policy 1, policy_version 1280330 (0.0006) [2023-12-27 00:42:01,920][105620] Updated weights for policy 1, policy_version 1280340 (0.0009) [2023-12-27 00:42:01,982][105620] Updated weights for policy 1, policy_version 1280350 (0.0009) [2023-12-27 00:42:02,210][105692] Updated weights for policy 0, policy_version 1278838 (0.0008) [2023-12-27 00:42:02,266][105692] Updated weights for policy 0, policy_version 1278848 (0.0008) [2023-12-27 00:42:02,332][105692] Updated weights for policy 0, policy_version 1278858 (0.0009) [2023-12-27 00:42:02,723][105620] Updated weights for policy 1, policy_version 1280360 (0.0009) [2023-12-27 00:42:02,779][105620] Updated weights for policy 1, policy_version 1280370 (0.0007) [2023-12-27 00:42:02,837][105620] Updated weights for policy 1, policy_version 1280380 (0.0005) [2023-12-27 00:42:02,990][105692] Updated weights for policy 0, policy_version 1278868 (0.0008) [2023-12-27 00:42:03,040][105692] Updated weights for policy 0, policy_version 1278878 (0.0005) [2023-12-27 00:42:03,093][105692] Updated weights for policy 0, policy_version 1278888 (0.0005) [2023-12-27 00:42:03,385][105620] Updated weights for policy 1, policy_version 1280390 (0.0006) [2023-12-27 00:42:03,432][105620] Updated weights for policy 1, policy_version 1280400 (0.0005) [2023-12-27 00:42:03,479][105620] Updated weights for policy 1, policy_version 1280410 (0.0005) [2023-12-27 00:42:03,719][105692] Updated weights for policy 0, policy_version 1278898 (0.0008) [2023-12-27 00:42:03,764][105692] Updated weights for policy 0, policy_version 1278908 (0.0005) [2023-12-27 00:42:03,816][105692] Updated weights for policy 0, policy_version 1278918 (0.0005) [2023-12-27 00:42:03,887][105692] Updated weights for policy 0, policy_version 1278928 (0.0010) [2023-12-27 00:42:04,088][105620] Updated weights for policy 1, policy_version 1280420 (0.0006) [2023-12-27 00:42:04,144][105620] Updated weights for policy 1, policy_version 1280430 (0.0009) [2023-12-27 00:42:04,208][105620] Updated weights for policy 1, policy_version 1280440 (0.0008) [2023-12-27 00:42:04,579][105692] Updated weights for policy 0, policy_version 1278938 (0.0009) [2023-12-27 00:42:04,636][105692] Updated weights for policy 0, policy_version 1278948 (0.0010) [2023-12-27 00:42:04,685][105692] Updated weights for policy 0, policy_version 1278958 (0.0009) [2023-12-27 00:42:04,937][105620] Updated weights for policy 1, policy_version 1280450 (0.0007) [2023-12-27 00:42:04,999][105620] Updated weights for policy 1, policy_version 1280460 (0.0005) [2023-12-27 00:42:05,062][105620] Updated weights for policy 1, policy_version 1280470 (0.0006) [2023-12-27 00:42:05,123][105620] Updated weights for policy 1, policy_version 1280480 (0.0005) [2023-12-27 00:42:05,304][105692] Updated weights for policy 0, policy_version 1278968 (0.0006) [2023-12-27 00:42:05,352][105692] Updated weights for policy 0, policy_version 1278978 (0.0005) [2023-12-27 00:42:05,408][105692] Updated weights for policy 0, policy_version 1278988 (0.0005) [2023-12-27 00:42:05,704][105620] Updated weights for policy 1, policy_version 1280490 (0.0005) [2023-12-27 00:42:05,745][105586] KL-divergence is very high: 109.0638 [2023-12-27 00:42:05,750][105620] Updated weights for policy 1, policy_version 1280500 (0.0005) [2023-12-27 00:42:05,784][105586] KL-divergence is very high: 124.6099 [2023-12-27 00:42:05,799][105620] Updated weights for policy 1, policy_version 1280510 (0.0006) [2023-12-27 00:42:05,955][105692] Updated weights for policy 0, policy_version 1278998 (0.0008) [2023-12-27 00:42:06,003][105692] Updated weights for policy 0, policy_version 1279008 (0.0010) [2023-12-27 00:42:06,054][105692] Updated weights for policy 0, policy_version 1279018 (0.0010) [2023-12-27 00:42:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 20070.4, 300 sec: 19605.3). Total num frames: 655327232. Throughput: 0: 9830.6, 1: 10203.3. Samples: 655315084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:42:06,063][104569] Avg episode reward: [(0, '8818.270'), (1, '9093.747')] [2023-12-27 00:42:06,366][105620] Updated weights for policy 1, policy_version 1280520 (0.0008) [2023-12-27 00:42:06,431][105620] Updated weights for policy 1, policy_version 1280530 (0.0010) [2023-12-27 00:42:06,491][105620] Updated weights for policy 1, policy_version 1280540 (0.0008) [2023-12-27 00:42:06,751][105692] Updated weights for policy 0, policy_version 1279028 (0.0009) [2023-12-27 00:42:06,812][105692] Updated weights for policy 0, policy_version 1279038 (0.0010) [2023-12-27 00:42:06,878][105692] Updated weights for policy 0, policy_version 1279048 (0.0011) [2023-12-27 00:42:07,145][105620] Updated weights for policy 1, policy_version 1280550 (0.0007) [2023-12-27 00:42:07,205][105620] Updated weights for policy 1, policy_version 1280560 (0.0008) [2023-12-27 00:42:07,264][105620] Updated weights for policy 1, policy_version 1280570 (0.0008) [2023-12-27 00:42:07,599][105692] Updated weights for policy 0, policy_version 1279058 (0.0010) [2023-12-27 00:42:07,657][105692] Updated weights for policy 0, policy_version 1279068 (0.0010) [2023-12-27 00:42:07,715][105692] Updated weights for policy 0, policy_version 1279078 (0.0010) [2023-12-27 00:42:07,779][105692] Updated weights for policy 0, policy_version 1279088 (0.0010) [2023-12-27 00:42:07,869][105620] Updated weights for policy 1, policy_version 1280580 (0.0005) [2023-12-27 00:42:07,930][105620] Updated weights for policy 1, policy_version 1280590 (0.0005) [2023-12-27 00:42:07,988][105620] Updated weights for policy 1, policy_version 1280600 (0.0007) [2023-12-27 00:42:08,534][105692] Updated weights for policy 0, policy_version 1279098 (0.0011) [2023-12-27 00:42:08,601][105620] Updated weights for policy 1, policy_version 1280610 (0.0008) [2023-12-27 00:42:08,603][105692] Updated weights for policy 0, policy_version 1279108 (0.0011) [2023-12-27 00:42:08,666][105692] Updated weights for policy 0, policy_version 1279118 (0.0011) [2023-12-27 00:42:08,668][105620] Updated weights for policy 1, policy_version 1280620 (0.0008) [2023-12-27 00:42:08,733][105620] Updated weights for policy 1, policy_version 1280630 (0.0008) [2023-12-27 00:42:08,801][105620] Updated weights for policy 1, policy_version 1280640 (0.0008) [2023-12-27 00:42:09,375][105692] Updated weights for policy 0, policy_version 1279128 (0.0009) [2023-12-27 00:42:09,378][105620] Updated weights for policy 1, policy_version 1280650 (0.0006) [2023-12-27 00:42:09,440][105692] Updated weights for policy 0, policy_version 1279138 (0.0008) [2023-12-27 00:42:09,448][105620] Updated weights for policy 1, policy_version 1280660 (0.0007) [2023-12-27 00:42:09,506][105692] Updated weights for policy 0, policy_version 1279148 (0.0007) [2023-12-27 00:42:09,513][105620] Updated weights for policy 1, policy_version 1280670 (0.0006) [2023-12-27 00:42:10,216][105620] Updated weights for policy 1, policy_version 1280680 (0.0008) [2023-12-27 00:42:10,270][105692] Updated weights for policy 0, policy_version 1279158 (0.0007) [2023-12-27 00:42:10,278][105620] Updated weights for policy 1, policy_version 1280690 (0.0008) [2023-12-27 00:42:10,333][105692] Updated weights for policy 0, policy_version 1279168 (0.0008) [2023-12-27 00:42:10,340][105620] Updated weights for policy 1, policy_version 1280700 (0.0007) [2023-12-27 00:42:10,392][105692] Updated weights for policy 0, policy_version 1279178 (0.0008) [2023-12-27 00:42:10,982][105620] Updated weights for policy 1, policy_version 1280710 (0.0006) [2023-12-27 00:42:11,044][105620] Updated weights for policy 1, policy_version 1280720 (0.0007) [2023-12-27 00:42:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 20070.4, 300 sec: 19633.0). Total num frames: 655425536. Throughput: 0: 9857.0, 1: 10287.6. Samples: 655440196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:42:11,063][104569] Avg episode reward: [(0, '8727.172'), (1, '8743.256')] [2023-12-27 00:42:11,102][105620] Updated weights for policy 1, policy_version 1280730 (0.0009) [2023-12-27 00:42:11,244][105692] Updated weights for policy 0, policy_version 1279188 (0.0009) [2023-12-27 00:42:11,304][105692] Updated weights for policy 0, policy_version 1279198 (0.0008) [2023-12-27 00:42:11,370][105692] Updated weights for policy 0, policy_version 1279208 (0.0008) [2023-12-27 00:42:11,774][105620] Updated weights for policy 1, policy_version 1280740 (0.0010) [2023-12-27 00:42:11,829][105620] Updated weights for policy 1, policy_version 1280750 (0.0009) [2023-12-27 00:42:11,882][105620] Updated weights for policy 1, policy_version 1280760 (0.0009) [2023-12-27 00:42:12,167][105692] Updated weights for policy 0, policy_version 1279218 (0.0008) [2023-12-27 00:42:12,228][105692] Updated weights for policy 0, policy_version 1279228 (0.0009) [2023-12-27 00:42:12,290][105692] Updated weights for policy 0, policy_version 1279238 (0.0007) [2023-12-27 00:42:12,357][105692] Updated weights for policy 0, policy_version 1279248 (0.0009) [2023-12-27 00:42:12,589][105620] Updated weights for policy 1, policy_version 1280770 (0.0009) [2023-12-27 00:42:12,654][105620] Updated weights for policy 1, policy_version 1280780 (0.0009) [2023-12-27 00:42:12,712][105620] Updated weights for policy 1, policy_version 1280790 (0.0008) [2023-12-27 00:42:12,774][105620] Updated weights for policy 1, policy_version 1280800 (0.0010) [2023-12-27 00:42:13,101][105692] Updated weights for policy 0, policy_version 1279258 (0.0008) [2023-12-27 00:42:13,165][105692] Updated weights for policy 0, policy_version 1279268 (0.0009) [2023-12-27 00:42:13,229][105692] Updated weights for policy 0, policy_version 1279278 (0.0008) [2023-12-27 00:42:13,568][105620] Updated weights for policy 1, policy_version 1280810 (0.0009) [2023-12-27 00:42:13,633][105620] Updated weights for policy 1, policy_version 1280820 (0.0009) [2023-12-27 00:42:13,693][105620] Updated weights for policy 1, policy_version 1280830 (0.0008) [2023-12-27 00:42:13,920][105692] Updated weights for policy 0, policy_version 1279288 (0.0008) [2023-12-27 00:42:13,970][105692] Updated weights for policy 0, policy_version 1279298 (0.0009) [2023-12-27 00:42:14,022][105692] Updated weights for policy 0, policy_version 1279308 (0.0009) [2023-12-27 00:42:14,465][105620] Updated weights for policy 1, policy_version 1280840 (0.0009) [2023-12-27 00:42:14,522][105620] Updated weights for policy 1, policy_version 1280850 (0.0010) [2023-12-27 00:42:14,589][105620] Updated weights for policy 1, policy_version 1280860 (0.0010) [2023-12-27 00:42:14,692][105692] Updated weights for policy 0, policy_version 1279318 (0.0009) [2023-12-27 00:42:14,750][105692] Updated weights for policy 0, policy_version 1279328 (0.0007) [2023-12-27 00:42:14,819][105692] Updated weights for policy 0, policy_version 1279338 (0.0007) [2023-12-27 00:42:15,329][105620] Updated weights for policy 1, policy_version 1280870 (0.0010) [2023-12-27 00:42:15,377][105620] Updated weights for policy 1, policy_version 1280880 (0.0009) [2023-12-27 00:42:15,425][105620] Updated weights for policy 1, policy_version 1280890 (0.0009) [2023-12-27 00:42:15,562][105692] Updated weights for policy 0, policy_version 1279348 (0.0006) [2023-12-27 00:42:15,624][105692] Updated weights for policy 0, policy_version 1279358 (0.0007) [2023-12-27 00:42:15,689][105692] Updated weights for policy 0, policy_version 1279368 (0.0011) [2023-12-27 00:42:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 20070.4, 300 sec: 19633.0). Total num frames: 655523840. Throughput: 0: 9722.6, 1: 10151.0. Samples: 655495116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:42:16,063][104569] Avg episode reward: [(0, '8727.902'), (1, '9003.806')] [2023-12-27 00:42:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001279376_327573504.pth... [2023-12-27 00:42:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001280896_327950336.pth... [2023-12-27 00:42:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001278224_327278592.pth [2023-12-27 00:42:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001279712_327647232.pth [2023-12-27 00:42:16,230][105620] Updated weights for policy 1, policy_version 1280900 (0.0008) [2023-12-27 00:42:16,289][105620] Updated weights for policy 1, policy_version 1280910 (0.0008) [2023-12-27 00:42:16,344][105620] Updated weights for policy 1, policy_version 1280920 (0.0008) [2023-12-27 00:42:16,403][105692] Updated weights for policy 0, policy_version 1279378 (0.0011) [2023-12-27 00:42:16,452][105692] Updated weights for policy 0, policy_version 1279388 (0.0010) [2023-12-27 00:42:16,513][105692] Updated weights for policy 0, policy_version 1279398 (0.0008) [2023-12-27 00:42:16,572][105692] Updated weights for policy 0, policy_version 1279408 (0.0008) [2023-12-27 00:42:17,150][105692] Updated weights for policy 0, policy_version 1279418 (0.0005) [2023-12-27 00:42:17,191][105620] Updated weights for policy 1, policy_version 1280930 (0.0007) [2023-12-27 00:42:17,208][105692] Updated weights for policy 0, policy_version 1279428 (0.0008) [2023-12-27 00:42:17,250][105620] Updated weights for policy 1, policy_version 1280940 (0.0008) [2023-12-27 00:42:17,260][105692] Updated weights for policy 0, policy_version 1279438 (0.0006) [2023-12-27 00:42:17,310][105620] Updated weights for policy 1, policy_version 1280950 (0.0009) [2023-12-27 00:42:17,367][105620] Updated weights for policy 1, policy_version 1280960 (0.0006) [2023-12-27 00:42:17,943][105692] Updated weights for policy 0, policy_version 1279448 (0.0005) [2023-12-27 00:42:17,989][105692] Updated weights for policy 0, policy_version 1279458 (0.0005) [2023-12-27 00:42:18,038][105692] Updated weights for policy 0, policy_version 1279468 (0.0005) [2023-12-27 00:42:18,052][105620] Updated weights for policy 1, policy_version 1280970 (0.0010) [2023-12-27 00:42:18,114][105620] Updated weights for policy 1, policy_version 1280980 (0.0010) [2023-12-27 00:42:18,119][105586] KL-divergence is very high: 126.9989 [2023-12-27 00:42:18,170][105586] KL-divergence is very high: 234.8725 [2023-12-27 00:42:18,176][105620] Updated weights for policy 1, policy_version 1280990 (0.0010) [2023-12-27 00:42:18,665][105692] Updated weights for policy 0, policy_version 1279478 (0.0010) [2023-12-27 00:42:18,717][105692] Updated weights for policy 0, policy_version 1279488 (0.0010) [2023-12-27 00:42:18,782][105692] Updated weights for policy 0, policy_version 1279498 (0.0011) [2023-12-27 00:42:18,928][105620] Updated weights for policy 1, policy_version 1281000 (0.0011) [2023-12-27 00:42:18,986][105620] Updated weights for policy 1, policy_version 1281010 (0.0006) [2023-12-27 00:42:19,039][105620] Updated weights for policy 1, policy_version 1281020 (0.0006) [2023-12-27 00:42:19,546][105692] Updated weights for policy 0, policy_version 1279508 (0.0011) [2023-12-27 00:42:19,614][105692] Updated weights for policy 0, policy_version 1279518 (0.0009) [2023-12-27 00:42:19,676][105692] Updated weights for policy 0, policy_version 1279528 (0.0010) [2023-12-27 00:42:19,676][105620] Updated weights for policy 1, policy_version 1281030 (0.0005) [2023-12-27 00:42:19,727][105620] Updated weights for policy 1, policy_version 1281040 (0.0005) [2023-12-27 00:42:19,776][105620] Updated weights for policy 1, policy_version 1281050 (0.0006) [2023-12-27 00:42:20,394][105692] Updated weights for policy 0, policy_version 1279538 (0.0009) [2023-12-27 00:42:20,451][105692] Updated weights for policy 0, policy_version 1279548 (0.0006) [2023-12-27 00:42:20,483][105620] Updated weights for policy 1, policy_version 1281060 (0.0006) [2023-12-27 00:42:20,510][105692] Updated weights for policy 0, policy_version 1279558 (0.0006) [2023-12-27 00:42:20,547][105620] Updated weights for policy 1, policy_version 1281070 (0.0008) [2023-12-27 00:42:20,572][105692] Updated weights for policy 0, policy_version 1279568 (0.0006) [2023-12-27 00:42:20,609][105620] Updated weights for policy 1, policy_version 1281080 (0.0010) [2023-12-27 00:42:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 655622144. Throughput: 0: 9802.8, 1: 10126.4. Samples: 655612928. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:42:21,063][104569] Avg episode reward: [(0, '8633.115'), (1, '9085.059')] [2023-12-27 00:42:21,248][105692] Updated weights for policy 0, policy_version 1279578 (0.0008) [2023-12-27 00:42:21,318][105692] Updated weights for policy 0, policy_version 1279588 (0.0008) [2023-12-27 00:42:21,392][105692] Updated weights for policy 0, policy_version 1279598 (0.0008) [2023-12-27 00:42:21,409][105620] Updated weights for policy 1, policy_version 1281090 (0.0009) [2023-12-27 00:42:21,477][105620] Updated weights for policy 1, policy_version 1281100 (0.0009) [2023-12-27 00:42:21,533][105620] Updated weights for policy 1, policy_version 1281110 (0.0009) [2023-12-27 00:42:21,595][105620] Updated weights for policy 1, policy_version 1281120 (0.0009) [2023-12-27 00:42:22,051][105692] Updated weights for policy 0, policy_version 1279608 (0.0010) [2023-12-27 00:42:22,111][105692] Updated weights for policy 0, policy_version 1279618 (0.0010) [2023-12-27 00:42:22,171][105692] Updated weights for policy 0, policy_version 1279628 (0.0010) [2023-12-27 00:42:22,389][105620] Updated weights for policy 1, policy_version 1281130 (0.0007) [2023-12-27 00:42:22,456][105620] Updated weights for policy 1, policy_version 1281140 (0.0008) [2023-12-27 00:42:22,519][105620] Updated weights for policy 1, policy_version 1281150 (0.0006) [2023-12-27 00:42:22,919][105692] Updated weights for policy 0, policy_version 1279638 (0.0010) [2023-12-27 00:42:22,989][105692] Updated weights for policy 0, policy_version 1279648 (0.0011) [2023-12-27 00:42:23,042][105692] Updated weights for policy 0, policy_version 1279658 (0.0007) [2023-12-27 00:42:23,157][105620] Updated weights for policy 1, policy_version 1281160 (0.0009) [2023-12-27 00:42:23,215][105620] Updated weights for policy 1, policy_version 1281172 (0.0010) [2023-12-27 00:42:23,261][105620] Updated weights for policy 1, policy_version 1281182 (0.0008) [2023-12-27 00:42:23,633][105692] Updated weights for policy 0, policy_version 1279668 (0.0005) [2023-12-27 00:42:23,686][105692] Updated weights for policy 0, policy_version 1279678 (0.0005) [2023-12-27 00:42:23,739][105692] Updated weights for policy 0, policy_version 1279688 (0.0008) [2023-12-27 00:42:24,182][105620] Updated weights for policy 1, policy_version 1281192 (0.0008) [2023-12-27 00:42:24,237][105620] Updated weights for policy 1, policy_version 1281202 (0.0008) [2023-12-27 00:42:24,292][105620] Updated weights for policy 1, policy_version 1281212 (0.0009) [2023-12-27 00:42:24,296][105692] Updated weights for policy 0, policy_version 1279698 (0.0007) [2023-12-27 00:42:24,341][105692] Updated weights for policy 0, policy_version 1279708 (0.0005) [2023-12-27 00:42:24,389][105692] Updated weights for policy 0, policy_version 1279718 (0.0006) [2023-12-27 00:42:24,450][105692] Updated weights for policy 0, policy_version 1279728 (0.0008) [2023-12-27 00:42:25,064][105692] Updated weights for policy 0, policy_version 1279738 (0.0005) [2023-12-27 00:42:25,109][105692] Updated weights for policy 0, policy_version 1279748 (0.0005) [2023-12-27 00:42:25,139][105620] Updated weights for policy 1, policy_version 1281222 (0.0006) [2023-12-27 00:42:25,167][105692] Updated weights for policy 0, policy_version 1279758 (0.0010) [2023-12-27 00:42:25,198][105620] Updated weights for policy 1, policy_version 1281232 (0.0006) [2023-12-27 00:42:25,250][105620] Updated weights for policy 1, policy_version 1281242 (0.0008) [2023-12-27 00:42:25,812][105692] Updated weights for policy 0, policy_version 1279768 (0.0010) [2023-12-27 00:42:25,862][105692] Updated weights for policy 0, policy_version 1279778 (0.0008) [2023-12-27 00:42:25,909][105692] Updated weights for policy 0, policy_version 1279789 (0.0009) [2023-12-27 00:42:26,049][105620] Updated weights for policy 1, policy_version 1281252 (0.0009) [2023-12-27 00:42:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 655720448. Throughput: 0: 9897.3, 1: 9977.4. Samples: 655729136. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:42:26,062][104569] Avg episode reward: [(0, '8814.381'), (1, '8737.188')] [2023-12-27 00:42:26,113][105620] Updated weights for policy 1, policy_version 1281262 (0.0009) [2023-12-27 00:42:26,164][105620] Updated weights for policy 1, policy_version 1281272 (0.0009) [2023-12-27 00:42:26,691][105692] Updated weights for policy 0, policy_version 1279799 (0.0009) [2023-12-27 00:42:26,737][105692] Updated weights for policy 0, policy_version 1279809 (0.0008) [2023-12-27 00:42:26,783][105692] Updated weights for policy 0, policy_version 1279819 (0.0009) [2023-12-27 00:42:26,921][105620] Updated weights for policy 1, policy_version 1281282 (0.0009) [2023-12-27 00:42:26,978][105620] Updated weights for policy 1, policy_version 1281292 (0.0009) [2023-12-27 00:42:27,035][105620] Updated weights for policy 1, policy_version 1281302 (0.0009) [2023-12-27 00:42:27,080][105620] Updated weights for policy 1, policy_version 1281312 (0.0008) [2023-12-27 00:42:27,482][105692] Updated weights for policy 0, policy_version 1279829 (0.0009) [2023-12-27 00:42:27,549][105692] Updated weights for policy 0, policy_version 1279839 (0.0008) [2023-12-27 00:42:27,595][105692] Updated weights for policy 0, policy_version 1279849 (0.0005) [2023-12-27 00:42:27,929][105620] Updated weights for policy 1, policy_version 1281322 (0.0010) [2023-12-27 00:42:27,991][105620] Updated weights for policy 1, policy_version 1281332 (0.0010) [2023-12-27 00:42:28,042][105620] Updated weights for policy 1, policy_version 1281342 (0.0007) [2023-12-27 00:42:28,210][105692] Updated weights for policy 0, policy_version 1279859 (0.0008) [2023-12-27 00:42:28,279][105692] Updated weights for policy 0, policy_version 1279869 (0.0008) [2023-12-27 00:42:28,346][105692] Updated weights for policy 0, policy_version 1279879 (0.0008) [2023-12-27 00:42:28,859][105620] Updated weights for policy 1, policy_version 1281352 (0.0010) [2023-12-27 00:42:28,914][105620] Updated weights for policy 1, policy_version 1281362 (0.0009) [2023-12-27 00:42:28,926][105586] KL-divergence is very high: 165.9709 [2023-12-27 00:42:28,968][105620] Updated weights for policy 1, policy_version 1281372 (0.0008) [2023-12-27 00:42:28,969][105586] KL-divergence is very high: 195.9889 [2023-12-27 00:42:28,991][105692] Updated weights for policy 0, policy_version 1279889 (0.0008) [2023-12-27 00:42:29,047][105692] Updated weights for policy 0, policy_version 1279899 (0.0009) [2023-12-27 00:42:29,108][105692] Updated weights for policy 0, policy_version 1279909 (0.0009) [2023-12-27 00:42:29,170][105692] Updated weights for policy 0, policy_version 1279919 (0.0009) [2023-12-27 00:42:29,718][105620] Updated weights for policy 1, policy_version 1281382 (0.0010) [2023-12-27 00:42:29,769][105620] Updated weights for policy 1, policy_version 1281392 (0.0009) [2023-12-27 00:42:29,826][105620] Updated weights for policy 1, policy_version 1281402 (0.0010) [2023-12-27 00:42:29,939][105692] Updated weights for policy 0, policy_version 1279929 (0.0009) [2023-12-27 00:42:29,997][105692] Updated weights for policy 0, policy_version 1279939 (0.0009) [2023-12-27 00:42:30,051][105692] Updated weights for policy 0, policy_version 1279949 (0.0005) [2023-12-27 00:42:30,594][105620] Updated weights for policy 1, policy_version 1281412 (0.0007) [2023-12-27 00:42:30,663][105620] Updated weights for policy 1, policy_version 1281422 (0.0005) [2023-12-27 00:42:30,717][105620] Updated weights for policy 1, policy_version 1281432 (0.0009) [2023-12-27 00:42:30,790][105692] Updated weights for policy 0, policy_version 1279959 (0.0007) [2023-12-27 00:42:30,844][105692] Updated weights for policy 0, policy_version 1279969 (0.0008) [2023-12-27 00:42:30,908][105692] Updated weights for policy 0, policy_version 1279979 (0.0005) [2023-12-27 00:42:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 655818752. Throughput: 0: 9973.4, 1: 9948.5. Samples: 655786020. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:42:31,063][104569] Avg episode reward: [(0, '9089.473'), (1, '8657.360')] [2023-12-27 00:42:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001279984_327729152.pth... [2023-12-27 00:42:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001281440_328089600.pth... [2023-12-27 00:42:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001278800_327426048.pth [2023-12-27 00:42:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001280320_327802880.pth [2023-12-27 00:42:31,406][105620] Updated weights for policy 1, policy_version 1281442 (0.0008) [2023-12-27 00:42:31,464][105620] Updated weights for policy 1, policy_version 1281452 (0.0005) [2023-12-27 00:42:31,520][105620] Updated weights for policy 1, policy_version 1281462 (0.0005) [2023-12-27 00:42:31,559][105692] Updated weights for policy 0, policy_version 1279989 (0.0007) [2023-12-27 00:42:31,576][105620] Updated weights for policy 1, policy_version 1281472 (0.0007) [2023-12-27 00:42:31,624][105692] Updated weights for policy 0, policy_version 1279999 (0.0007) [2023-12-27 00:42:31,690][105692] Updated weights for policy 0, policy_version 1280009 (0.0007) [2023-12-27 00:42:32,269][105620] Updated weights for policy 1, policy_version 1281482 (0.0009) [2023-12-27 00:42:32,331][105620] Updated weights for policy 1, policy_version 1281492 (0.0007) [2023-12-27 00:42:32,367][105692] Updated weights for policy 0, policy_version 1280019 (0.0008) [2023-12-27 00:42:32,394][105620] Updated weights for policy 1, policy_version 1281502 (0.0008) [2023-12-27 00:42:32,418][105692] Updated weights for policy 0, policy_version 1280029 (0.0008) [2023-12-27 00:42:32,468][105692] Updated weights for policy 0, policy_version 1280039 (0.0008) [2023-12-27 00:42:33,111][105620] Updated weights for policy 1, policy_version 1281512 (0.0009) [2023-12-27 00:42:33,153][105692] Updated weights for policy 0, policy_version 1280049 (0.0008) [2023-12-27 00:42:33,173][105620] Updated weights for policy 1, policy_version 1281522 (0.0008) [2023-12-27 00:42:33,211][105692] Updated weights for policy 0, policy_version 1280059 (0.0006) [2023-12-27 00:42:33,226][105620] Updated weights for policy 1, policy_version 1281532 (0.0009) [2023-12-27 00:42:33,260][105692] Updated weights for policy 0, policy_version 1280069 (0.0006) [2023-12-27 00:42:33,319][105692] Updated weights for policy 0, policy_version 1280079 (0.0009) [2023-12-27 00:42:33,928][105620] Updated weights for policy 1, policy_version 1281542 (0.0008) [2023-12-27 00:42:33,989][105620] Updated weights for policy 1, policy_version 1281552 (0.0006) [2023-12-27 00:42:34,042][105620] Updated weights for policy 1, policy_version 1281562 (0.0008) [2023-12-27 00:42:34,076][105692] Updated weights for policy 0, policy_version 1280089 (0.0008) [2023-12-27 00:42:34,130][105692] Updated weights for policy 0, policy_version 1280099 (0.0009) [2023-12-27 00:42:34,191][105692] Updated weights for policy 0, policy_version 1280109 (0.0008) [2023-12-27 00:42:34,770][105620] Updated weights for policy 1, policy_version 1281572 (0.0007) [2023-12-27 00:42:34,821][105620] Updated weights for policy 1, policy_version 1281582 (0.0007) [2023-12-27 00:42:34,874][105620] Updated weights for policy 1, policy_version 1281592 (0.0008) [2023-12-27 00:42:34,965][105692] Updated weights for policy 0, policy_version 1280119 (0.0009) [2023-12-27 00:42:35,012][105692] Updated weights for policy 0, policy_version 1280129 (0.0008) [2023-12-27 00:42:35,062][105692] Updated weights for policy 0, policy_version 1280139 (0.0009) [2023-12-27 00:42:35,616][105620] Updated weights for policy 1, policy_version 1281602 (0.0008) [2023-12-27 00:42:35,669][105620] Updated weights for policy 1, policy_version 1281612 (0.0010) [2023-12-27 00:42:35,717][105620] Updated weights for policy 1, policy_version 1281622 (0.0010) [2023-12-27 00:42:35,767][105620] Updated weights for policy 1, policy_version 1281632 (0.0008) [2023-12-27 00:42:35,853][105692] Updated weights for policy 0, policy_version 1280150 (0.0010) [2023-12-27 00:42:35,907][105692] Updated weights for policy 0, policy_version 1280160 (0.0009) [2023-12-27 00:42:35,968][105692] Updated weights for policy 0, policy_version 1280170 (0.0010) [2023-12-27 00:42:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 655917056. Throughput: 0: 9961.3, 1: 9851.8. Samples: 655902608. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:42:36,062][104569] Avg episode reward: [(0, '9176.737'), (1, '8660.274')] [2023-12-27 00:42:36,434][105620] Updated weights for policy 1, policy_version 1281642 (0.0011) [2023-12-27 00:42:36,497][105620] Updated weights for policy 1, policy_version 1281652 (0.0011) [2023-12-27 00:42:36,563][105620] Updated weights for policy 1, policy_version 1281662 (0.0011) [2023-12-27 00:42:36,759][105692] Updated weights for policy 0, policy_version 1280181 (0.0008) [2023-12-27 00:42:36,819][105692] Updated weights for policy 0, policy_version 1280191 (0.0008) [2023-12-27 00:42:36,878][105692] Updated weights for policy 0, policy_version 1280201 (0.0008) [2023-12-27 00:42:37,306][105620] Updated weights for policy 1, policy_version 1281672 (0.0011) [2023-12-27 00:42:37,365][105620] Updated weights for policy 1, policy_version 1281682 (0.0010) [2023-12-27 00:42:37,426][105620] Updated weights for policy 1, policy_version 1281692 (0.0010) [2023-12-27 00:42:37,608][105692] Updated weights for policy 0, policy_version 1280211 (0.0008) [2023-12-27 00:42:37,668][105692] Updated weights for policy 0, policy_version 1280221 (0.0008) [2023-12-27 00:42:37,724][105692] Updated weights for policy 0, policy_version 1280231 (0.0008) [2023-12-27 00:42:38,159][105620] Updated weights for policy 1, policy_version 1281702 (0.0010) [2023-12-27 00:42:38,217][105620] Updated weights for policy 1, policy_version 1281712 (0.0010) [2023-12-27 00:42:38,271][105620] Updated weights for policy 1, policy_version 1281722 (0.0010) [2023-12-27 00:42:38,495][105692] Updated weights for policy 0, policy_version 1280241 (0.0008) [2023-12-27 00:42:38,556][105692] Updated weights for policy 0, policy_version 1280251 (0.0008) [2023-12-27 00:42:38,616][105692] Updated weights for policy 0, policy_version 1280261 (0.0008) [2023-12-27 00:42:38,671][105692] Updated weights for policy 0, policy_version 1280271 (0.0008) [2023-12-27 00:42:38,969][105620] Updated weights for policy 1, policy_version 1281732 (0.0010) [2023-12-27 00:42:39,020][105620] Updated weights for policy 1, policy_version 1281742 (0.0010) [2023-12-27 00:42:39,062][105620] Updated weights for policy 1, policy_version 1281752 (0.0005) [2023-12-27 00:42:39,456][105692] Updated weights for policy 0, policy_version 1280281 (0.0009) [2023-12-27 00:42:39,519][105692] Updated weights for policy 0, policy_version 1280291 (0.0009) [2023-12-27 00:42:39,579][105692] Updated weights for policy 0, policy_version 1280301 (0.0008) [2023-12-27 00:42:39,732][105620] Updated weights for policy 1, policy_version 1281762 (0.0006) [2023-12-27 00:42:39,795][105620] Updated weights for policy 1, policy_version 1281772 (0.0011) [2023-12-27 00:42:39,860][105620] Updated weights for policy 1, policy_version 1281782 (0.0011) [2023-12-27 00:42:39,926][105620] Updated weights for policy 1, policy_version 1281792 (0.0009) [2023-12-27 00:42:40,351][105692] Updated weights for policy 0, policy_version 1280311 (0.0008) [2023-12-27 00:42:40,412][105692] Updated weights for policy 0, policy_version 1280321 (0.0008) [2023-12-27 00:42:40,469][105692] Updated weights for policy 0, policy_version 1280331 (0.0008) [2023-12-27 00:42:40,673][105620] Updated weights for policy 1, policy_version 1281802 (0.0010) [2023-12-27 00:42:40,734][105620] Updated weights for policy 1, policy_version 1281812 (0.0010) [2023-12-27 00:42:40,785][105620] Updated weights for policy 1, policy_version 1281822 (0.0009) [2023-12-27 00:42:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 656007168. Throughput: 0: 9775.2, 1: 9767.7. Samples: 656015852. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:42:41,062][104569] Avg episode reward: [(0, '9176.979'), (1, '8738.610')] [2023-12-27 00:42:41,320][105692] Updated weights for policy 0, policy_version 1280341 (0.0008) [2023-12-27 00:42:41,394][105692] Updated weights for policy 0, policy_version 1280351 (0.0009) [2023-12-27 00:42:41,448][105692] Updated weights for policy 0, policy_version 1280361 (0.0007) [2023-12-27 00:42:41,488][105620] Updated weights for policy 1, policy_version 1281832 (0.0010) [2023-12-27 00:42:41,541][105620] Updated weights for policy 1, policy_version 1281842 (0.0011) [2023-12-27 00:42:41,604][105620] Updated weights for policy 1, policy_version 1281852 (0.0009) [2023-12-27 00:42:42,182][105692] Updated weights for policy 0, policy_version 1280371 (0.0007) [2023-12-27 00:42:42,251][105692] Updated weights for policy 0, policy_version 1280381 (0.0008) [2023-12-27 00:42:42,319][105692] Updated weights for policy 0, policy_version 1280391 (0.0008) [2023-12-27 00:42:42,397][105620] Updated weights for policy 1, policy_version 1281862 (0.0011) [2023-12-27 00:42:42,463][105620] Updated weights for policy 1, policy_version 1281872 (0.0011) [2023-12-27 00:42:42,537][105620] Updated weights for policy 1, policy_version 1281882 (0.0011) [2023-12-27 00:42:43,003][105692] Updated weights for policy 0, policy_version 1280401 (0.0007) [2023-12-27 00:42:43,067][105692] Updated weights for policy 0, policy_version 1280411 (0.0008) [2023-12-27 00:42:43,124][105692] Updated weights for policy 0, policy_version 1280421 (0.0009) [2023-12-27 00:42:43,154][105620] Updated weights for policy 1, policy_version 1281892 (0.0010) [2023-12-27 00:42:43,181][105692] Updated weights for policy 0, policy_version 1280431 (0.0007) [2023-12-27 00:42:43,205][105620] Updated weights for policy 1, policy_version 1281902 (0.0007) [2023-12-27 00:42:43,256][105620] Updated weights for policy 1, policy_version 1281912 (0.0009) [2023-12-27 00:42:43,803][105692] Updated weights for policy 0, policy_version 1280441 (0.0009) [2023-12-27 00:42:43,863][105692] Updated weights for policy 0, policy_version 1280451 (0.0009) [2023-12-27 00:42:43,920][105692] Updated weights for policy 0, policy_version 1280461 (0.0007) [2023-12-27 00:42:44,068][105620] Updated weights for policy 1, policy_version 1281922 (0.0009) [2023-12-27 00:42:44,138][105620] Updated weights for policy 1, policy_version 1281932 (0.0010) [2023-12-27 00:42:44,206][105620] Updated weights for policy 1, policy_version 1281942 (0.0010) [2023-12-27 00:42:44,539][105692] Updated weights for policy 0, policy_version 1280472 (0.0008) [2023-12-27 00:42:44,593][105692] Updated weights for policy 0, policy_version 1280483 (0.0010) [2023-12-27 00:42:44,655][105692] Updated weights for policy 0, policy_version 1280493 (0.0009) [2023-12-27 00:42:44,932][105620] Updated weights for policy 1, policy_version 1281953 (0.0010) [2023-12-27 00:42:44,995][105620] Updated weights for policy 1, policy_version 1281963 (0.0008) [2023-12-27 00:42:45,058][105620] Updated weights for policy 1, policy_version 1281973 (0.0011) [2023-12-27 00:42:45,121][105620] Updated weights for policy 1, policy_version 1281983 (0.0011) [2023-12-27 00:42:45,404][105692] Updated weights for policy 0, policy_version 1280503 (0.0008) [2023-12-27 00:42:45,456][105692] Updated weights for policy 0, policy_version 1280513 (0.0008) [2023-12-27 00:42:45,505][105692] Updated weights for policy 0, policy_version 1280523 (0.0008) [2023-12-27 00:42:45,852][105620] Updated weights for policy 1, policy_version 1281993 (0.0011) [2023-12-27 00:42:45,910][105620] Updated weights for policy 1, policy_version 1282003 (0.0011) [2023-12-27 00:42:45,969][105620] Updated weights for policy 1, policy_version 1282013 (0.0011) [2023-12-27 00:42:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 656105472. Throughput: 0: 9741.5, 1: 9744.0. Samples: 656072912. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:42:46,062][104569] Avg episode reward: [(0, '9090.030'), (1, '9086.724')] [2023-12-27 00:42:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001280528_327868416.pth... [2023-12-27 00:42:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001282016_328237056.pth... [2023-12-27 00:42:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001279376_327573504.pth [2023-12-27 00:42:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001280896_327950336.pth [2023-12-27 00:42:46,279][105692] Updated weights for policy 0, policy_version 1280533 (0.0009) [2023-12-27 00:42:46,324][105692] Updated weights for policy 0, policy_version 1280543 (0.0010) [2023-12-27 00:42:46,368][105692] Updated weights for policy 0, policy_version 1280553 (0.0010) [2023-12-27 00:42:46,646][105620] Updated weights for policy 1, policy_version 1282023 (0.0009) [2023-12-27 00:42:46,706][105620] Updated weights for policy 1, policy_version 1282033 (0.0011) [2023-12-27 00:42:46,755][105620] Updated weights for policy 1, policy_version 1282043 (0.0011) [2023-12-27 00:42:47,134][105692] Updated weights for policy 0, policy_version 1280563 (0.0009) [2023-12-27 00:42:47,195][105692] Updated weights for policy 0, policy_version 1280573 (0.0006) [2023-12-27 00:42:47,255][105692] Updated weights for policy 0, policy_version 1280583 (0.0005) [2023-12-27 00:42:47,493][105620] Updated weights for policy 1, policy_version 1282053 (0.0009) [2023-12-27 00:42:47,555][105620] Updated weights for policy 1, policy_version 1282063 (0.0008) [2023-12-27 00:42:47,611][105620] Updated weights for policy 1, policy_version 1282073 (0.0008) [2023-12-27 00:42:47,957][105692] Updated weights for policy 0, policy_version 1280593 (0.0005) [2023-12-27 00:42:48,008][105692] Updated weights for policy 0, policy_version 1280603 (0.0006) [2023-12-27 00:42:48,062][105692] Updated weights for policy 0, policy_version 1280613 (0.0008) [2023-12-27 00:42:48,118][105692] Updated weights for policy 0, policy_version 1280623 (0.0008) [2023-12-27 00:42:48,258][105620] Updated weights for policy 1, policy_version 1282083 (0.0009) [2023-12-27 00:42:48,313][105620] Updated weights for policy 1, policy_version 1282093 (0.0006) [2023-12-27 00:42:48,378][105620] Updated weights for policy 1, policy_version 1282103 (0.0011) [2023-12-27 00:42:48,853][105692] Updated weights for policy 0, policy_version 1280633 (0.0006) [2023-12-27 00:42:48,920][105692] Updated weights for policy 0, policy_version 1280643 (0.0006) [2023-12-27 00:42:48,979][105692] Updated weights for policy 0, policy_version 1280653 (0.0006) [2023-12-27 00:42:48,979][105620] Updated weights for policy 1, policy_version 1282113 (0.0010) [2023-12-27 00:42:49,029][105620] Updated weights for policy 1, policy_version 1282123 (0.0007) [2023-12-27 00:42:49,084][105620] Updated weights for policy 1, policy_version 1282133 (0.0007) [2023-12-27 00:42:49,133][105620] Updated weights for policy 1, policy_version 1282143 (0.0009) [2023-12-27 00:42:49,593][105692] Updated weights for policy 0, policy_version 1280663 (0.0008) [2023-12-27 00:42:49,642][105692] Updated weights for policy 0, policy_version 1280673 (0.0008) [2023-12-27 00:42:49,699][105692] Updated weights for policy 0, policy_version 1280683 (0.0008) [2023-12-27 00:42:49,927][105620] Updated weights for policy 1, policy_version 1282153 (0.0008) [2023-12-27 00:42:49,993][105620] Updated weights for policy 1, policy_version 1282163 (0.0009) [2023-12-27 00:42:50,053][105620] Updated weights for policy 1, policy_version 1282173 (0.0010) [2023-12-27 00:42:50,506][105692] Updated weights for policy 0, policy_version 1280693 (0.0009) [2023-12-27 00:42:50,576][105692] Updated weights for policy 0, policy_version 1280703 (0.0010) [2023-12-27 00:42:50,643][105692] Updated weights for policy 0, policy_version 1280713 (0.0009) [2023-12-27 00:42:50,686][105620] Updated weights for policy 1, policy_version 1282183 (0.0007) [2023-12-27 00:42:50,750][105620] Updated weights for policy 1, policy_version 1282193 (0.0006) [2023-12-27 00:42:50,803][105620] Updated weights for policy 1, policy_version 1282203 (0.0006) [2023-12-27 00:42:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 656203776. Throughput: 0: 9832.8, 1: 9655.8. Samples: 656192072. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:42:51,062][104569] Avg episode reward: [(0, '8552.787'), (1, '8927.340')] [2023-12-27 00:42:51,456][105620] Updated weights for policy 1, policy_version 1282213 (0.0008) [2023-12-27 00:42:51,486][105692] Updated weights for policy 0, policy_version 1280723 (0.0008) [2023-12-27 00:42:51,515][105620] Updated weights for policy 1, policy_version 1282223 (0.0010) [2023-12-27 00:42:51,546][105692] Updated weights for policy 0, policy_version 1280733 (0.0006) [2023-12-27 00:42:51,571][105620] Updated weights for policy 1, policy_version 1282233 (0.0010) [2023-12-27 00:42:51,599][105692] Updated weights for policy 0, policy_version 1280743 (0.0008) [2023-12-27 00:42:52,345][105620] Updated weights for policy 1, policy_version 1282243 (0.0011) [2023-12-27 00:42:52,395][105692] Updated weights for policy 0, policy_version 1280753 (0.0009) [2023-12-27 00:42:52,408][105620] Updated weights for policy 1, policy_version 1282253 (0.0011) [2023-12-27 00:42:52,451][105692] Updated weights for policy 0, policy_version 1280763 (0.0007) [2023-12-27 00:42:52,463][105620] Updated weights for policy 1, policy_version 1282263 (0.0009) [2023-12-27 00:42:52,504][105692] Updated weights for policy 0, policy_version 1280773 (0.0006) [2023-12-27 00:42:52,560][105692] Updated weights for policy 0, policy_version 1280783 (0.0008) [2023-12-27 00:42:53,111][105620] Updated weights for policy 1, policy_version 1282273 (0.0008) [2023-12-27 00:42:53,175][105620] Updated weights for policy 1, policy_version 1282283 (0.0005) [2023-12-27 00:42:53,245][105620] Updated weights for policy 1, policy_version 1282293 (0.0005) [2023-12-27 00:42:53,299][105620] Updated weights for policy 1, policy_version 1282303 (0.0005) [2023-12-27 00:42:53,345][105692] Updated weights for policy 0, policy_version 1280793 (0.0010) [2023-12-27 00:42:53,400][105692] Updated weights for policy 0, policy_version 1280805 (0.0010) [2023-12-27 00:42:53,812][105620] Updated weights for policy 1, policy_version 1282313 (0.0005) [2023-12-27 00:42:53,865][105620] Updated weights for policy 1, policy_version 1282323 (0.0005) [2023-12-27 00:42:53,918][105620] Updated weights for policy 1, policy_version 1282333 (0.0005) [2023-12-27 00:42:54,331][105692] Updated weights for policy 0, policy_version 1280817 (0.0010) [2023-12-27 00:42:54,390][105692] Updated weights for policy 0, policy_version 1280827 (0.0010) [2023-12-27 00:42:54,444][105692] Updated weights for policy 0, policy_version 1280838 (0.0010) [2023-12-27 00:42:54,506][105620] Updated weights for policy 1, policy_version 1282343 (0.0005) [2023-12-27 00:42:54,552][105620] Updated weights for policy 1, policy_version 1282353 (0.0006) [2023-12-27 00:42:54,599][105620] Updated weights for policy 1, policy_version 1282363 (0.0005) [2023-12-27 00:42:55,247][105620] Updated weights for policy 1, policy_version 1282373 (0.0008) [2023-12-27 00:42:55,292][105692] Updated weights for policy 0, policy_version 1280849 (0.0008) [2023-12-27 00:42:55,306][105620] Updated weights for policy 1, policy_version 1282383 (0.0010) [2023-12-27 00:42:55,352][105692] Updated weights for policy 0, policy_version 1280859 (0.0006) [2023-12-27 00:42:55,362][105620] Updated weights for policy 1, policy_version 1282393 (0.0010) [2023-12-27 00:42:55,404][105692] Updated weights for policy 0, policy_version 1280869 (0.0006) [2023-12-27 00:42:55,453][105692] Updated weights for policy 0, policy_version 1280879 (0.0008) [2023-12-27 00:42:56,055][105620] Updated weights for policy 1, policy_version 1282403 (0.0009) [2023-12-27 00:42:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 656293888. Throughput: 0: 9641.4, 1: 9613.8. Samples: 656306676. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:42:56,063][104569] Avg episode reward: [(0, '8457.678'), (1, '8935.323')] [2023-12-27 00:42:56,123][105620] Updated weights for policy 1, policy_version 1282413 (0.0007) [2023-12-27 00:42:56,182][105620] Updated weights for policy 1, policy_version 1282423 (0.0010) [2023-12-27 00:42:56,275][105692] Updated weights for policy 0, policy_version 1280889 (0.0009) [2023-12-27 00:42:56,331][105692] Updated weights for policy 0, policy_version 1280899 (0.0009) [2023-12-27 00:42:56,390][105692] Updated weights for policy 0, policy_version 1280909 (0.0009) [2023-12-27 00:42:56,787][105620] Updated weights for policy 1, policy_version 1282433 (0.0008) [2023-12-27 00:42:56,842][105620] Updated weights for policy 1, policy_version 1282443 (0.0006) [2023-12-27 00:42:56,887][105620] Updated weights for policy 1, policy_version 1282453 (0.0008) [2023-12-27 00:42:56,940][105620] Updated weights for policy 1, policy_version 1282463 (0.0009) [2023-12-27 00:42:57,192][105692] Updated weights for policy 0, policy_version 1280919 (0.0009) [2023-12-27 00:42:57,251][105692] Updated weights for policy 0, policy_version 1280929 (0.0009) [2023-12-27 00:42:57,310][105692] Updated weights for policy 0, policy_version 1280939 (0.0009) [2023-12-27 00:42:57,620][105620] Updated weights for policy 1, policy_version 1282473 (0.0009) [2023-12-27 00:42:57,666][105620] Updated weights for policy 1, policy_version 1282483 (0.0008) [2023-12-27 00:42:57,712][105620] Updated weights for policy 1, policy_version 1282493 (0.0009) [2023-12-27 00:42:58,084][105692] Updated weights for policy 0, policy_version 1280949 (0.0009) [2023-12-27 00:42:58,140][105692] Updated weights for policy 0, policy_version 1280959 (0.0009) [2023-12-27 00:42:58,203][105692] Updated weights for policy 0, policy_version 1280969 (0.0008) [2023-12-27 00:42:58,489][105620] Updated weights for policy 1, policy_version 1282503 (0.0009) [2023-12-27 00:42:58,552][105620] Updated weights for policy 1, policy_version 1282513 (0.0009) [2023-12-27 00:42:58,621][105620] Updated weights for policy 1, policy_version 1282523 (0.0009) [2023-12-27 00:42:58,938][105692] Updated weights for policy 0, policy_version 1280979 (0.0008) [2023-12-27 00:42:58,985][105692] Updated weights for policy 0, policy_version 1280989 (0.0009) [2023-12-27 00:42:59,033][105692] Updated weights for policy 0, policy_version 1280999 (0.0009) [2023-12-27 00:42:59,338][105620] Updated weights for policy 1, policy_version 1282533 (0.0008) [2023-12-27 00:42:59,408][105620] Updated weights for policy 1, policy_version 1282543 (0.0008) [2023-12-27 00:42:59,471][105620] Updated weights for policy 1, policy_version 1282553 (0.0007) [2023-12-27 00:42:59,836][105692] Updated weights for policy 0, policy_version 1281009 (0.0009) [2023-12-27 00:42:59,904][105692] Updated weights for policy 0, policy_version 1281019 (0.0007) [2023-12-27 00:42:59,971][105692] Updated weights for policy 0, policy_version 1281029 (0.0010) [2023-12-27 00:43:00,027][105692] Updated weights for policy 0, policy_version 1281039 (0.0006) [2023-12-27 00:43:00,123][105620] Updated weights for policy 1, policy_version 1282563 (0.0007) [2023-12-27 00:43:00,177][105620] Updated weights for policy 1, policy_version 1282573 (0.0009) [2023-12-27 00:43:00,228][105620] Updated weights for policy 1, policy_version 1282583 (0.0008) [2023-12-27 00:43:00,596][105692] Updated weights for policy 0, policy_version 1281049 (0.0010) [2023-12-27 00:43:00,655][105692] Updated weights for policy 0, policy_version 1281059 (0.0010) [2023-12-27 00:43:00,703][105692] Updated weights for policy 0, policy_version 1281069 (0.0010) [2023-12-27 00:43:01,035][105620] Updated weights for policy 1, policy_version 1282594 (0.0010) [2023-12-27 00:43:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 656392192. Throughput: 0: 9630.7, 1: 9650.8. Samples: 656362780. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:43:01,062][104569] Avg episode reward: [(0, '8538.357'), (1, '8701.240')] [2023-12-27 00:43:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001281072_328007680.pth... [2023-12-27 00:43:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001279984_327729152.pth [2023-12-27 00:43:01,100][105620] Updated weights for policy 1, policy_version 1282604 (0.0008) [2023-12-27 00:43:01,171][105620] Updated weights for policy 1, policy_version 1282614 (0.0008) [2023-12-27 00:43:01,237][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001282624_328392704.pth... [2023-12-27 00:43:01,237][105620] Updated weights for policy 1, policy_version 1282624 (0.0006) [2023-12-27 00:43:01,241][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001281440_328089600.pth [2023-12-27 00:43:01,467][105692] Updated weights for policy 0, policy_version 1281079 (0.0011) [2023-12-27 00:43:01,530][105692] Updated weights for policy 0, policy_version 1281089 (0.0011) [2023-12-27 00:43:01,583][105692] Updated weights for policy 0, policy_version 1281099 (0.0010) [2023-12-27 00:43:01,905][105620] Updated weights for policy 1, policy_version 1282634 (0.0006) [2023-12-27 00:43:01,964][105620] Updated weights for policy 1, policy_version 1282644 (0.0007) [2023-12-27 00:43:02,024][105620] Updated weights for policy 1, policy_version 1282654 (0.0008) [2023-12-27 00:43:02,301][105692] Updated weights for policy 0, policy_version 1281109 (0.0010) [2023-12-27 00:43:02,369][105692] Updated weights for policy 0, policy_version 1281119 (0.0011) [2023-12-27 00:43:02,427][105692] Updated weights for policy 0, policy_version 1281129 (0.0008) [2023-12-27 00:43:02,656][105620] Updated weights for policy 1, policy_version 1282664 (0.0009) [2023-12-27 00:43:02,706][105620] Updated weights for policy 1, policy_version 1282674 (0.0008) [2023-12-27 00:43:02,764][105620] Updated weights for policy 1, policy_version 1282684 (0.0008) [2023-12-27 00:43:03,166][105692] Updated weights for policy 0, policy_version 1281139 (0.0008) [2023-12-27 00:43:03,220][105692] Updated weights for policy 0, policy_version 1281149 (0.0009) [2023-12-27 00:43:03,270][105692] Updated weights for policy 0, policy_version 1281159 (0.0009) [2023-12-27 00:43:03,549][105620] Updated weights for policy 1, policy_version 1282694 (0.0009) [2023-12-27 00:43:03,603][105620] Updated weights for policy 1, policy_version 1282704 (0.0009) [2023-12-27 00:43:03,653][105620] Updated weights for policy 1, policy_version 1282714 (0.0009) [2023-12-27 00:43:04,043][105692] Updated weights for policy 0, policy_version 1281169 (0.0009) [2023-12-27 00:43:04,101][105692] Updated weights for policy 0, policy_version 1281179 (0.0009) [2023-12-27 00:43:04,168][105692] Updated weights for policy 0, policy_version 1281189 (0.0009) [2023-12-27 00:43:04,229][105692] Updated weights for policy 0, policy_version 1281199 (0.0010) [2023-12-27 00:43:04,363][105620] Updated weights for policy 1, policy_version 1282724 (0.0008) [2023-12-27 00:43:04,426][105620] Updated weights for policy 1, policy_version 1282734 (0.0007) [2023-12-27 00:43:04,475][105620] Updated weights for policy 1, policy_version 1282744 (0.0008) [2023-12-27 00:43:04,978][105692] Updated weights for policy 0, policy_version 1281209 (0.0008) [2023-12-27 00:43:05,032][105692] Updated weights for policy 0, policy_version 1281219 (0.0008) [2023-12-27 00:43:05,095][105692] Updated weights for policy 0, policy_version 1281229 (0.0009) [2023-12-27 00:43:05,202][105620] Updated weights for policy 1, policy_version 1282754 (0.0008) [2023-12-27 00:43:05,265][105620] Updated weights for policy 1, policy_version 1282764 (0.0010) [2023-12-27 00:43:05,330][105620] Updated weights for policy 1, policy_version 1282774 (0.0010) [2023-12-27 00:43:05,398][105620] Updated weights for policy 1, policy_version 1282784 (0.0007) [2023-12-27 00:43:05,826][105692] Updated weights for policy 0, policy_version 1281239 (0.0009) [2023-12-27 00:43:05,879][105692] Updated weights for policy 0, policy_version 1281249 (0.0008) [2023-12-27 00:43:05,924][105692] Updated weights for policy 0, policy_version 1281259 (0.0008) [2023-12-27 00:43:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 656490496. Throughput: 0: 9543.8, 1: 9703.3. Samples: 656479052. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:43:06,063][104569] Avg episode reward: [(0, '8449.680'), (1, '8788.572')] [2023-12-27 00:43:06,084][105620] Updated weights for policy 1, policy_version 1282794 (0.0010) [2023-12-27 00:43:06,148][105620] Updated weights for policy 1, policy_version 1282804 (0.0009) [2023-12-27 00:43:06,206][105620] Updated weights for policy 1, policy_version 1282814 (0.0009) [2023-12-27 00:43:06,625][105692] Updated weights for policy 0, policy_version 1281269 (0.0008) [2023-12-27 00:43:06,685][105692] Updated weights for policy 0, policy_version 1281279 (0.0008) [2023-12-27 00:43:06,747][105692] Updated weights for policy 0, policy_version 1281289 (0.0008) [2023-12-27 00:43:06,978][105620] Updated weights for policy 1, policy_version 1282824 (0.0011) [2023-12-27 00:43:07,044][105620] Updated weights for policy 1, policy_version 1282834 (0.0010) [2023-12-27 00:43:07,102][105620] Updated weights for policy 1, policy_version 1282844 (0.0010) [2023-12-27 00:43:07,460][105692] Updated weights for policy 0, policy_version 1281299 (0.0006) [2023-12-27 00:43:07,517][105692] Updated weights for policy 0, policy_version 1281309 (0.0008) [2023-12-27 00:43:07,581][105692] Updated weights for policy 0, policy_version 1281319 (0.0006) [2023-12-27 00:43:07,798][105620] Updated weights for policy 1, policy_version 1282854 (0.0010) [2023-12-27 00:43:07,865][105620] Updated weights for policy 1, policy_version 1282864 (0.0011) [2023-12-27 00:43:07,922][105620] Updated weights for policy 1, policy_version 1282874 (0.0010) [2023-12-27 00:43:08,241][105692] Updated weights for policy 0, policy_version 1281329 (0.0007) [2023-12-27 00:43:08,303][105692] Updated weights for policy 0, policy_version 1281339 (0.0005) [2023-12-27 00:43:08,368][105692] Updated weights for policy 0, policy_version 1281349 (0.0008) [2023-12-27 00:43:08,431][105692] Updated weights for policy 0, policy_version 1281359 (0.0007) [2023-12-27 00:43:08,704][105620] Updated weights for policy 1, policy_version 1282884 (0.0009) [2023-12-27 00:43:08,766][105620] Updated weights for policy 1, policy_version 1282894 (0.0006) [2023-12-27 00:43:08,823][105620] Updated weights for policy 1, policy_version 1282904 (0.0006) [2023-12-27 00:43:09,095][105692] Updated weights for policy 0, policy_version 1281369 (0.0009) [2023-12-27 00:43:09,141][105692] Updated weights for policy 0, policy_version 1281379 (0.0008) [2023-12-27 00:43:09,193][105692] Updated weights for policy 0, policy_version 1281389 (0.0008) [2023-12-27 00:43:09,489][105620] Updated weights for policy 1, policy_version 1282914 (0.0007) [2023-12-27 00:43:09,551][105620] Updated weights for policy 1, policy_version 1282924 (0.0008) [2023-12-27 00:43:09,611][105620] Updated weights for policy 1, policy_version 1282934 (0.0009) [2023-12-27 00:43:09,674][105620] Updated weights for policy 1, policy_version 1282944 (0.0009) [2023-12-27 00:43:10,026][105692] Updated weights for policy 0, policy_version 1281399 (0.0008) [2023-12-27 00:43:10,085][105692] Updated weights for policy 0, policy_version 1281409 (0.0009) [2023-12-27 00:43:10,144][105692] Updated weights for policy 0, policy_version 1281419 (0.0009) [2023-12-27 00:43:10,441][105620] Updated weights for policy 1, policy_version 1282954 (0.0009) [2023-12-27 00:43:10,507][105620] Updated weights for policy 1, policy_version 1282964 (0.0009) [2023-12-27 00:43:10,566][105620] Updated weights for policy 1, policy_version 1282974 (0.0009) [2023-12-27 00:43:10,920][105692] Updated weights for policy 0, policy_version 1281429 (0.0009) [2023-12-27 00:43:10,973][105692] Updated weights for policy 0, policy_version 1281439 (0.0007) [2023-12-27 00:43:11,045][105692] Updated weights for policy 0, policy_version 1281449 (0.0008) [2023-12-27 00:43:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.3, 300 sec: 19494.2). Total num frames: 656580608. Throughput: 0: 9439.4, 1: 9780.9. Samples: 656594048. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:43:11,062][104569] Avg episode reward: [(0, '8453.850'), (1, '9193.754')] [2023-12-27 00:43:11,353][105620] Updated weights for policy 1, policy_version 1282984 (0.0009) [2023-12-27 00:43:11,414][105620] Updated weights for policy 1, policy_version 1282994 (0.0009) [2023-12-27 00:43:11,476][105620] Updated weights for policy 1, policy_version 1283004 (0.0009) [2023-12-27 00:43:11,867][105692] Updated weights for policy 0, policy_version 1281459 (0.0009) [2023-12-27 00:43:11,923][105692] Updated weights for policy 0, policy_version 1281469 (0.0009) [2023-12-27 00:43:11,986][105692] Updated weights for policy 0, policy_version 1281479 (0.0009) [2023-12-27 00:43:12,249][105620] Updated weights for policy 1, policy_version 1283014 (0.0010) [2023-12-27 00:43:12,314][105620] Updated weights for policy 1, policy_version 1283024 (0.0009) [2023-12-27 00:43:12,385][105620] Updated weights for policy 1, policy_version 1283034 (0.0010) [2023-12-27 00:43:12,806][105692] Updated weights for policy 0, policy_version 1281489 (0.0010) [2023-12-27 00:43:12,861][105692] Updated weights for policy 0, policy_version 1281499 (0.0009) [2023-12-27 00:43:12,915][105692] Updated weights for policy 0, policy_version 1281509 (0.0009) [2023-12-27 00:43:12,964][105692] Updated weights for policy 0, policy_version 1281519 (0.0009) [2023-12-27 00:43:13,063][105620] Updated weights for policy 1, policy_version 1283044 (0.0007) [2023-12-27 00:43:13,122][105620] Updated weights for policy 1, policy_version 1283054 (0.0009) [2023-12-27 00:43:13,186][105620] Updated weights for policy 1, policy_version 1283064 (0.0009) [2023-12-27 00:43:13,756][105692] Updated weights for policy 0, policy_version 1281529 (0.0009) [2023-12-27 00:43:13,810][105692] Updated weights for policy 0, policy_version 1281539 (0.0009) [2023-12-27 00:43:13,867][105692] Updated weights for policy 0, policy_version 1281549 (0.0010) [2023-12-27 00:43:13,875][105620] Updated weights for policy 1, policy_version 1283074 (0.0009) [2023-12-27 00:43:13,938][105620] Updated weights for policy 1, policy_version 1283084 (0.0010) [2023-12-27 00:43:14,004][105620] Updated weights for policy 1, policy_version 1283094 (0.0011) [2023-12-27 00:43:14,075][105620] Updated weights for policy 1, policy_version 1283104 (0.0011) [2023-12-27 00:43:14,656][105692] Updated weights for policy 0, policy_version 1281559 (0.0009) [2023-12-27 00:43:14,718][105692] Updated weights for policy 0, policy_version 1281569 (0.0010) [2023-12-27 00:43:14,779][105620] Updated weights for policy 1, policy_version 1283114 (0.0009) [2023-12-27 00:43:14,784][105692] Updated weights for policy 0, policy_version 1281579 (0.0008) [2023-12-27 00:43:14,842][105620] Updated weights for policy 1, policy_version 1283124 (0.0009) [2023-12-27 00:43:14,901][105620] Updated weights for policy 1, policy_version 1283134 (0.0010) [2023-12-27 00:43:15,563][105620] Updated weights for policy 1, policy_version 1283144 (0.0009) [2023-12-27 00:43:15,613][105692] Updated weights for policy 0, policy_version 1281589 (0.0008) [2023-12-27 00:43:15,623][105620] Updated weights for policy 1, policy_version 1283154 (0.0011) [2023-12-27 00:43:15,670][105692] Updated weights for policy 0, policy_version 1281599 (0.0006) [2023-12-27 00:43:15,680][105620] Updated weights for policy 1, policy_version 1283164 (0.0010) [2023-12-27 00:43:15,718][105692] Updated weights for policy 0, policy_version 1281609 (0.0007) [2023-12-27 00:43:16,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 656678912. Throughput: 0: 9363.9, 1: 9831.8. Samples: 656649828. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:43:16,062][104569] Avg episode reward: [(0, '8906.566'), (1, '9014.608')] [2023-12-27 00:43:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001283168_328531968.pth... [2023-12-27 00:43:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001281616_328146944.pth... [2023-12-27 00:43:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001282016_328237056.pth [2023-12-27 00:43:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001280528_327868416.pth [2023-12-27 00:43:16,314][105620] Updated weights for policy 1, policy_version 1283174 (0.0006) [2023-12-27 00:43:16,371][105620] Updated weights for policy 1, policy_version 1283184 (0.0010) [2023-12-27 00:43:16,429][105620] Updated weights for policy 1, policy_version 1283194 (0.0010) [2023-12-27 00:43:16,486][105692] Updated weights for policy 0, policy_version 1281619 (0.0008) [2023-12-27 00:43:16,544][105692] Updated weights for policy 0, policy_version 1281629 (0.0005) [2023-12-27 00:43:16,600][105692] Updated weights for policy 0, policy_version 1281639 (0.0005) [2023-12-27 00:43:17,034][105620] Updated weights for policy 1, policy_version 1283204 (0.0011) [2023-12-27 00:43:17,091][105620] Updated weights for policy 1, policy_version 1283214 (0.0010) [2023-12-27 00:43:17,152][105620] Updated weights for policy 1, policy_version 1283224 (0.0010) [2023-12-27 00:43:17,240][105692] Updated weights for policy 0, policy_version 1281649 (0.0006) [2023-12-27 00:43:17,295][105692] Updated weights for policy 0, policy_version 1281659 (0.0010) [2023-12-27 00:43:17,344][105692] Updated weights for policy 0, policy_version 1281670 (0.0011) [2023-12-27 00:43:17,395][105692] Updated weights for policy 0, policy_version 1281680 (0.0007) [2023-12-27 00:43:17,824][105620] Updated weights for policy 1, policy_version 1283234 (0.0009) [2023-12-27 00:43:17,873][105620] Updated weights for policy 1, policy_version 1283244 (0.0005) [2023-12-27 00:43:17,920][105620] Updated weights for policy 1, policy_version 1283254 (0.0005) [2023-12-27 00:43:17,973][105620] Updated weights for policy 1, policy_version 1283264 (0.0005) [2023-12-27 00:43:17,980][105692] Updated weights for policy 0, policy_version 1281690 (0.0006) [2023-12-27 00:43:18,042][105692] Updated weights for policy 0, policy_version 1281700 (0.0006) [2023-12-27 00:43:18,101][105692] Updated weights for policy 0, policy_version 1281710 (0.0006) [2023-12-27 00:43:18,676][105620] Updated weights for policy 1, policy_version 1283274 (0.0009) [2023-12-27 00:43:18,738][105620] Updated weights for policy 1, policy_version 1283284 (0.0007) [2023-12-27 00:43:18,784][105692] Updated weights for policy 0, policy_version 1281720 (0.0009) [2023-12-27 00:43:18,796][105620] Updated weights for policy 1, policy_version 1283294 (0.0008) [2023-12-27 00:43:18,847][105692] Updated weights for policy 0, policy_version 1281730 (0.0009) [2023-12-27 00:43:18,903][105692] Updated weights for policy 0, policy_version 1281740 (0.0008) [2023-12-27 00:43:19,519][105620] Updated weights for policy 1, policy_version 1283304 (0.0010) [2023-12-27 00:43:19,577][105620] Updated weights for policy 1, policy_version 1283314 (0.0009) [2023-12-27 00:43:19,639][105620] Updated weights for policy 1, policy_version 1283324 (0.0009) [2023-12-27 00:43:19,703][105692] Updated weights for policy 0, policy_version 1281750 (0.0009) [2023-12-27 00:43:19,753][105692] Updated weights for policy 0, policy_version 1281760 (0.0009) [2023-12-27 00:43:19,817][105692] Updated weights for policy 0, policy_version 1281770 (0.0009) [2023-12-27 00:43:20,410][105620] Updated weights for policy 1, policy_version 1283334 (0.0008) [2023-12-27 00:43:20,468][105620] Updated weights for policy 1, policy_version 1283344 (0.0007) [2023-12-27 00:43:20,523][105620] Updated weights for policy 1, policy_version 1283354 (0.0009) [2023-12-27 00:43:20,603][105692] Updated weights for policy 0, policy_version 1281780 (0.0009) [2023-12-27 00:43:20,655][105692] Updated weights for policy 0, policy_version 1281790 (0.0009) [2023-12-27 00:43:20,714][105692] Updated weights for policy 0, policy_version 1281800 (0.0009) [2023-12-27 00:43:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 656777216. Throughput: 0: 9347.9, 1: 9887.3. Samples: 656768192. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:43:21,062][104569] Avg episode reward: [(0, '9178.481'), (1, '8670.925')] [2023-12-27 00:43:21,265][105620] Updated weights for policy 1, policy_version 1283364 (0.0008) [2023-12-27 00:43:21,322][105620] Updated weights for policy 1, policy_version 1283374 (0.0007) [2023-12-27 00:43:21,393][105620] Updated weights for policy 1, policy_version 1283384 (0.0008) [2023-12-27 00:43:21,567][105692] Updated weights for policy 0, policy_version 1281810 (0.0010) [2023-12-27 00:43:21,623][105692] Updated weights for policy 0, policy_version 1281820 (0.0009) [2023-12-27 00:43:21,682][105692] Updated weights for policy 0, policy_version 1281830 (0.0009) [2023-12-27 00:43:21,748][105692] Updated weights for policy 0, policy_version 1281840 (0.0008) [2023-12-27 00:43:22,095][105620] Updated weights for policy 1, policy_version 1283394 (0.0007) [2023-12-27 00:43:22,157][105620] Updated weights for policy 1, policy_version 1283404 (0.0009) [2023-12-27 00:43:22,218][105620] Updated weights for policy 1, policy_version 1283414 (0.0009) [2023-12-27 00:43:22,280][105620] Updated weights for policy 1, policy_version 1283424 (0.0009) [2023-12-27 00:43:22,549][105692] Updated weights for policy 0, policy_version 1281850 (0.0009) [2023-12-27 00:43:22,607][105692] Updated weights for policy 0, policy_version 1281860 (0.0009) [2023-12-27 00:43:22,662][105692] Updated weights for policy 0, policy_version 1281870 (0.0009) [2023-12-27 00:43:23,029][105620] Updated weights for policy 1, policy_version 1283434 (0.0009) [2023-12-27 00:43:23,092][105620] Updated weights for policy 1, policy_version 1283444 (0.0009) [2023-12-27 00:43:23,155][105620] Updated weights for policy 1, policy_version 1283454 (0.0009) [2023-12-27 00:43:23,437][105692] Updated weights for policy 0, policy_version 1281880 (0.0009) [2023-12-27 00:43:23,498][105692] Updated weights for policy 0, policy_version 1281890 (0.0009) [2023-12-27 00:43:23,552][105692] Updated weights for policy 0, policy_version 1281900 (0.0009) [2023-12-27 00:43:23,855][105620] Updated weights for policy 1, policy_version 1283464 (0.0009) [2023-12-27 00:43:23,909][105620] Updated weights for policy 1, policy_version 1283474 (0.0009) [2023-12-27 00:43:23,956][105620] Updated weights for policy 1, policy_version 1283484 (0.0008) [2023-12-27 00:43:24,319][105692] Updated weights for policy 0, policy_version 1281910 (0.0009) [2023-12-27 00:43:24,381][105692] Updated weights for policy 0, policy_version 1281920 (0.0010) [2023-12-27 00:43:24,433][105692] Updated weights for policy 0, policy_version 1281930 (0.0009) [2023-12-27 00:43:24,629][105620] Updated weights for policy 1, policy_version 1283494 (0.0009) [2023-12-27 00:43:24,688][105620] Updated weights for policy 1, policy_version 1283504 (0.0009) [2023-12-27 00:43:24,751][105620] Updated weights for policy 1, policy_version 1283514 (0.0009) [2023-12-27 00:43:25,271][105692] Updated weights for policy 0, policy_version 1281940 (0.0009) [2023-12-27 00:43:25,321][105692] Updated weights for policy 0, policy_version 1281950 (0.0010) [2023-12-27 00:43:25,323][105620] Updated weights for policy 1, policy_version 1283524 (0.0007) [2023-12-27 00:43:25,369][105692] Updated weights for policy 0, policy_version 1281960 (0.0008) [2023-12-27 00:43:25,390][105620] Updated weights for policy 1, policy_version 1283534 (0.0005) [2023-12-27 00:43:25,452][105620] Updated weights for policy 1, policy_version 1283544 (0.0008) [2023-12-27 00:43:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.6, 300 sec: 19494.2). Total num frames: 656867328. Throughput: 0: 9313.9, 1: 9915.4. Samples: 656881176. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:43:26,063][104569] Avg episode reward: [(0, '9177.945'), (1, '8671.372')] [2023-12-27 00:43:26,092][105620] Updated weights for policy 1, policy_version 1283554 (0.0009) [2023-12-27 00:43:26,143][105620] Updated weights for policy 1, policy_version 1283564 (0.0009) [2023-12-27 00:43:26,159][105692] Updated weights for policy 0, policy_version 1281970 (0.0009) [2023-12-27 00:43:26,195][105620] Updated weights for policy 1, policy_version 1283574 (0.0009) [2023-12-27 00:43:26,215][105692] Updated weights for policy 0, policy_version 1281980 (0.0008) [2023-12-27 00:43:26,247][105620] Updated weights for policy 1, policy_version 1283584 (0.0006) [2023-12-27 00:43:26,258][105692] Updated weights for policy 0, policy_version 1281990 (0.0007) [2023-12-27 00:43:26,307][105692] Updated weights for policy 0, policy_version 1282000 (0.0008) [2023-12-27 00:43:27,002][105692] Updated weights for policy 0, policy_version 1282010 (0.0006) [2023-12-27 00:43:27,016][105620] Updated weights for policy 1, policy_version 1283594 (0.0009) [2023-12-27 00:43:27,054][105692] Updated weights for policy 0, policy_version 1282020 (0.0007) [2023-12-27 00:43:27,061][105620] Updated weights for policy 1, policy_version 1283604 (0.0006) [2023-12-27 00:43:27,112][105692] Updated weights for policy 0, policy_version 1282030 (0.0009) [2023-12-27 00:43:27,117][105620] Updated weights for policy 1, policy_version 1283614 (0.0008) [2023-12-27 00:43:27,751][105620] Updated weights for policy 1, policy_version 1283624 (0.0009) [2023-12-27 00:43:27,800][105620] Updated weights for policy 1, policy_version 1283634 (0.0009) [2023-12-27 00:43:27,825][105692] Updated weights for policy 0, policy_version 1282040 (0.0009) [2023-12-27 00:43:27,851][105620] Updated weights for policy 1, policy_version 1283644 (0.0005) [2023-12-27 00:43:27,870][105692] Updated weights for policy 0, policy_version 1282050 (0.0006) [2023-12-27 00:43:27,929][105692] Updated weights for policy 0, policy_version 1282060 (0.0008) [2023-12-27 00:43:28,592][105692] Updated weights for policy 0, policy_version 1282070 (0.0009) [2023-12-27 00:43:28,642][105692] Updated weights for policy 0, policy_version 1282080 (0.0007) [2023-12-27 00:43:28,644][105620] Updated weights for policy 1, policy_version 1283654 (0.0007) [2023-12-27 00:43:28,694][105692] Updated weights for policy 0, policy_version 1282090 (0.0006) [2023-12-27 00:43:28,696][105620] Updated weights for policy 1, policy_version 1283664 (0.0006) [2023-12-27 00:43:28,756][105620] Updated weights for policy 1, policy_version 1283674 (0.0007) [2023-12-27 00:43:29,388][105620] Updated weights for policy 1, policy_version 1283684 (0.0010) [2023-12-27 00:43:29,448][105620] Updated weights for policy 1, policy_version 1283694 (0.0006) [2023-12-27 00:43:29,504][105620] Updated weights for policy 1, policy_version 1283704 (0.0007) [2023-12-27 00:43:29,533][105692] Updated weights for policy 0, policy_version 1282100 (0.0006) [2023-12-27 00:43:29,563][105585] KL-divergence is very high: 101.9841 [2023-12-27 00:43:29,577][105585] KL-divergence is very high: 362.2173 [2023-12-27 00:43:29,591][105585] KL-divergence is very high: 458.8743 [2023-12-27 00:43:29,597][105692] Updated weights for policy 0, policy_version 1282110 (0.0008) [2023-12-27 00:43:29,616][105585] KL-divergence is very high: 188.0227 [2023-12-27 00:43:29,630][105585] KL-divergence is very high: 721.4458 [2023-12-27 00:43:29,644][105585] KL-divergence is very high: 687.7648 [2023-12-27 00:43:29,663][105585] KL-divergence is very high: 117.0956 [2023-12-27 00:43:29,664][105692] Updated weights for policy 0, policy_version 1282120 (0.0010) [2023-12-27 00:43:29,670][105585] KL-divergence is very high: 175.5216 [2023-12-27 00:43:29,681][105585] KL-divergence is very high: 834.8546 [2023-12-27 00:43:29,694][105585] KL-divergence is very high: 715.8686 [2023-12-27 00:43:30,100][105620] Updated weights for policy 1, policy_version 1283714 (0.0007) [2023-12-27 00:43:30,155][105620] Updated weights for policy 1, policy_version 1283724 (0.0007) [2023-12-27 00:43:30,202][105620] Updated weights for policy 1, policy_version 1283734 (0.0009) [2023-12-27 00:43:30,252][105620] Updated weights for policy 1, policy_version 1283744 (0.0009) [2023-12-27 00:43:30,498][105692] Updated weights for policy 0, policy_version 1282130 (0.0010) [2023-12-27 00:43:30,552][105692] Updated weights for policy 0, policy_version 1282140 (0.0009) [2023-12-27 00:43:30,608][105692] Updated weights for policy 0, policy_version 1282150 (0.0009) [2023-12-27 00:43:30,658][105692] Updated weights for policy 0, policy_version 1282160 (0.0009) [2023-12-27 00:43:30,976][105620] Updated weights for policy 1, policy_version 1283754 (0.0005) [2023-12-27 00:43:31,036][105620] Updated weights for policy 1, policy_version 1283764 (0.0008) [2023-12-27 00:43:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19494.2). Total num frames: 656965632. Throughput: 0: 9343.0, 1: 9921.9. Samples: 656939836. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:43:31,062][104569] Avg episode reward: [(0, '8651.045'), (1, '9007.202')] [2023-12-27 00:43:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001282160_328286208.pth... [2023-12-27 00:43:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001281072_328007680.pth [2023-12-27 00:43:31,096][105620] Updated weights for policy 1, policy_version 1283774 (0.0010) [2023-12-27 00:43:31,106][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001283776_328687616.pth... [2023-12-27 00:43:31,113][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001282624_328392704.pth [2023-12-27 00:43:31,480][105692] Updated weights for policy 0, policy_version 1282170 (0.0008) [2023-12-27 00:43:31,529][105692] Updated weights for policy 0, policy_version 1282180 (0.0006) [2023-12-27 00:43:31,580][105692] Updated weights for policy 0, policy_version 1282190 (0.0008) [2023-12-27 00:43:31,829][105620] Updated weights for policy 1, policy_version 1283784 (0.0006) [2023-12-27 00:43:31,891][105620] Updated weights for policy 1, policy_version 1283794 (0.0006) [2023-12-27 00:43:31,948][105620] Updated weights for policy 1, policy_version 1283806 (0.0009) [2023-12-27 00:43:32,301][105692] Updated weights for policy 0, policy_version 1282200 (0.0006) [2023-12-27 00:43:32,366][105692] Updated weights for policy 0, policy_version 1282210 (0.0009) [2023-12-27 00:43:32,437][105692] Updated weights for policy 0, policy_version 1282220 (0.0010) [2023-12-27 00:43:32,553][105620] Updated weights for policy 1, policy_version 1283816 (0.0006) [2023-12-27 00:43:32,613][105620] Updated weights for policy 1, policy_version 1283826 (0.0005) [2023-12-27 00:43:32,682][105620] Updated weights for policy 1, policy_version 1283836 (0.0006) [2023-12-27 00:43:33,180][105692] Updated weights for policy 0, policy_version 1282230 (0.0008) [2023-12-27 00:43:33,227][105692] Updated weights for policy 0, policy_version 1282240 (0.0009) [2023-12-27 00:43:33,276][105692] Updated weights for policy 0, policy_version 1282250 (0.0008) [2023-12-27 00:43:33,355][105620] Updated weights for policy 1, policy_version 1283846 (0.0009) [2023-12-27 00:43:33,402][105620] Updated weights for policy 1, policy_version 1283856 (0.0009) [2023-12-27 00:43:33,456][105620] Updated weights for policy 1, policy_version 1283866 (0.0009) [2023-12-27 00:43:34,037][105692] Updated weights for policy 0, policy_version 1282260 (0.0008) [2023-12-27 00:43:34,095][105692] Updated weights for policy 0, policy_version 1282270 (0.0009) [2023-12-27 00:43:34,160][105692] Updated weights for policy 0, policy_version 1282280 (0.0009) [2023-12-27 00:43:34,230][105620] Updated weights for policy 1, policy_version 1283876 (0.0009) [2023-12-27 00:43:34,296][105620] Updated weights for policy 1, policy_version 1283886 (0.0009) [2023-12-27 00:43:34,362][105620] Updated weights for policy 1, policy_version 1283896 (0.0009) [2023-12-27 00:43:34,922][105692] Updated weights for policy 0, policy_version 1282290 (0.0009) [2023-12-27 00:43:34,982][105692] Updated weights for policy 0, policy_version 1282300 (0.0010) [2023-12-27 00:43:35,041][105692] Updated weights for policy 0, policy_version 1282310 (0.0011) [2023-12-27 00:43:35,104][105692] Updated weights for policy 0, policy_version 1282320 (0.0010) [2023-12-27 00:43:35,139][105620] Updated weights for policy 1, policy_version 1283906 (0.0009) [2023-12-27 00:43:35,191][105620] Updated weights for policy 1, policy_version 1283916 (0.0008) [2023-12-27 00:43:35,243][105620] Updated weights for policy 1, policy_version 1283926 (0.0008) [2023-12-27 00:43:35,302][105620] Updated weights for policy 1, policy_version 1283936 (0.0008) [2023-12-27 00:43:35,838][105692] Updated weights for policy 0, policy_version 1282330 (0.0011) [2023-12-27 00:43:35,896][105692] Updated weights for policy 0, policy_version 1282340 (0.0010) [2023-12-27 00:43:35,910][105620] Updated weights for policy 1, policy_version 1283946 (0.0009) [2023-12-27 00:43:35,945][105692] Updated weights for policy 0, policy_version 1282350 (0.0010) [2023-12-27 00:43:35,956][105620] Updated weights for policy 1, policy_version 1283956 (0.0006) [2023-12-27 00:43:36,010][105620] Updated weights for policy 1, policy_version 1283966 (0.0007) [2023-12-27 00:43:36,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 657072128. Throughput: 0: 9213.8, 1: 9955.6. Samples: 657054692. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:43:36,062][104569] Avg episode reward: [(0, '4039.135'), (1, '9097.488')] [2023-12-27 00:43:36,656][105692] Updated weights for policy 0, policy_version 1282360 (0.0010) [2023-12-27 00:43:36,707][105692] Updated weights for policy 0, policy_version 1282370 (0.0009) [2023-12-27 00:43:36,758][105692] Updated weights for policy 0, policy_version 1282380 (0.0009) [2023-12-27 00:43:36,851][105620] Updated weights for policy 1, policy_version 1283976 (0.0009) [2023-12-27 00:43:36,900][105620] Updated weights for policy 1, policy_version 1283986 (0.0009) [2023-12-27 00:43:36,949][105620] Updated weights for policy 1, policy_version 1283996 (0.0006) [2023-12-27 00:43:37,506][105692] Updated weights for policy 0, policy_version 1282390 (0.0009) [2023-12-27 00:43:37,565][105620] Updated weights for policy 1, policy_version 1284006 (0.0008) [2023-12-27 00:43:37,567][105692] Updated weights for policy 0, policy_version 1282400 (0.0007) [2023-12-27 00:43:37,610][105585] KL-divergence is very high: 102.6705 [2023-12-27 00:43:37,619][105620] Updated weights for policy 1, policy_version 1284016 (0.0011) [2023-12-27 00:43:37,628][105692] Updated weights for policy 0, policy_version 1282410 (0.0006) [2023-12-27 00:43:37,671][105620] Updated weights for policy 1, policy_version 1284026 (0.0010) [2023-12-27 00:43:38,331][105692] Updated weights for policy 0, policy_version 1282420 (0.0008) [2023-12-27 00:43:38,387][105692] Updated weights for policy 0, policy_version 1282430 (0.0007) [2023-12-27 00:43:38,417][105620] Updated weights for policy 1, policy_version 1284036 (0.0010) [2023-12-27 00:43:38,449][105692] Updated weights for policy 0, policy_version 1282440 (0.0008) [2023-12-27 00:43:38,467][105620] Updated weights for policy 1, policy_version 1284046 (0.0006) [2023-12-27 00:43:38,514][105620] Updated weights for policy 1, policy_version 1284056 (0.0007) [2023-12-27 00:43:39,178][105620] Updated weights for policy 1, policy_version 1284066 (0.0009) [2023-12-27 00:43:39,237][105620] Updated weights for policy 1, policy_version 1284076 (0.0008) [2023-12-27 00:43:39,243][105692] Updated weights for policy 0, policy_version 1282450 (0.0007) [2023-12-27 00:43:39,296][105620] Updated weights for policy 1, policy_version 1284086 (0.0007) [2023-12-27 00:43:39,306][105692] Updated weights for policy 0, policy_version 1282460 (0.0008) [2023-12-27 00:43:39,366][105620] Updated weights for policy 1, policy_version 1284096 (0.0007) [2023-12-27 00:43:39,378][105692] Updated weights for policy 0, policy_version 1282470 (0.0008) [2023-12-27 00:43:39,449][105692] Updated weights for policy 0, policy_version 1282480 (0.0008) [2023-12-27 00:43:40,146][105620] Updated weights for policy 1, policy_version 1284106 (0.0009) [2023-12-27 00:43:40,177][105692] Updated weights for policy 0, policy_version 1282490 (0.0008) [2023-12-27 00:43:40,203][105620] Updated weights for policy 1, policy_version 1284116 (0.0007) [2023-12-27 00:43:40,236][105692] Updated weights for policy 0, policy_version 1282500 (0.0007) [2023-12-27 00:43:40,259][105620] Updated weights for policy 1, policy_version 1284126 (0.0008) [2023-12-27 00:43:40,305][105692] Updated weights for policy 0, policy_version 1282510 (0.0007) [2023-12-27 00:43:41,015][105620] Updated weights for policy 1, policy_version 1284136 (0.0008) [2023-12-27 00:43:41,043][105692] Updated weights for policy 0, policy_version 1282520 (0.0009) [2023-12-27 00:43:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19494.2). Total num frames: 657154048. Throughput: 0: 9340.0, 1: 9859.3. Samples: 657170644. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:43:41,062][104569] Avg episode reward: [(0, '2719.664'), (1, '9115.112')] [2023-12-27 00:43:41,080][105620] Updated weights for policy 1, policy_version 1284146 (0.0008) [2023-12-27 00:43:41,106][105692] Updated weights for policy 0, policy_version 1282530 (0.0008) [2023-12-27 00:43:41,148][105620] Updated weights for policy 1, policy_version 1284156 (0.0008) [2023-12-27 00:43:41,173][105692] Updated weights for policy 0, policy_version 1282540 (0.0008) [2023-12-27 00:43:41,864][105620] Updated weights for policy 1, policy_version 1284166 (0.0009) [2023-12-27 00:43:41,919][105620] Updated weights for policy 1, policy_version 1284176 (0.0009) [2023-12-27 00:43:41,956][105692] Updated weights for policy 0, policy_version 1282550 (0.0008) [2023-12-27 00:43:41,986][105620] Updated weights for policy 1, policy_version 1284186 (0.0008) [2023-12-27 00:43:42,017][105692] Updated weights for policy 0, policy_version 1282560 (0.0007) [2023-12-27 00:43:42,082][105692] Updated weights for policy 0, policy_version 1282570 (0.0009) [2023-12-27 00:43:42,724][105620] Updated weights for policy 1, policy_version 1284196 (0.0007) [2023-12-27 00:43:42,786][105620] Updated weights for policy 1, policy_version 1284206 (0.0009) [2023-12-27 00:43:42,835][105692] Updated weights for policy 0, policy_version 1282580 (0.0008) [2023-12-27 00:43:42,845][105620] Updated weights for policy 1, policy_version 1284216 (0.0008) [2023-12-27 00:43:42,888][105692] Updated weights for policy 0, policy_version 1282590 (0.0006) [2023-12-27 00:43:42,945][105692] Updated weights for policy 0, policy_version 1282600 (0.0009) [2023-12-27 00:43:43,586][105620] Updated weights for policy 1, policy_version 1284226 (0.0007) [2023-12-27 00:43:43,637][105620] Updated weights for policy 1, policy_version 1284236 (0.0009) [2023-12-27 00:43:43,690][105620] Updated weights for policy 1, policy_version 1284246 (0.0009) [2023-12-27 00:43:43,703][105692] Updated weights for policy 0, policy_version 1282610 (0.0009) [2023-12-27 00:43:43,736][105620] Updated weights for policy 1, policy_version 1284256 (0.0008) [2023-12-27 00:43:43,763][105692] Updated weights for policy 0, policy_version 1282620 (0.0008) [2023-12-27 00:43:43,821][105692] Updated weights for policy 0, policy_version 1282630 (0.0009) [2023-12-27 00:43:43,880][105692] Updated weights for policy 0, policy_version 1282640 (0.0008) [2023-12-27 00:43:44,520][105620] Updated weights for policy 1, policy_version 1284266 (0.0008) [2023-12-27 00:43:44,581][105620] Updated weights for policy 1, policy_version 1284276 (0.0009) [2023-12-27 00:43:44,623][105692] Updated weights for policy 0, policy_version 1282650 (0.0007) [2023-12-27 00:43:44,638][105620] Updated weights for policy 1, policy_version 1284286 (0.0008) [2023-12-27 00:43:44,683][105692] Updated weights for policy 0, policy_version 1282660 (0.0007) [2023-12-27 00:43:44,744][105692] Updated weights for policy 0, policy_version 1282670 (0.0009) [2023-12-27 00:43:45,357][105620] Updated weights for policy 1, policy_version 1284296 (0.0006) [2023-12-27 00:43:45,429][105620] Updated weights for policy 1, policy_version 1284306 (0.0006) [2023-12-27 00:43:45,491][105620] Updated weights for policy 1, policy_version 1284316 (0.0008) [2023-12-27 00:43:45,543][105692] Updated weights for policy 0, policy_version 1282680 (0.0008) [2023-12-27 00:43:45,605][105692] Updated weights for policy 0, policy_version 1282690 (0.0009) [2023-12-27 00:43:45,674][105692] Updated weights for policy 0, policy_version 1282700 (0.0009) [2023-12-27 00:43:46,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19114.7, 300 sec: 19522.0). Total num frames: 657252352. Throughput: 0: 9352.6, 1: 9813.6. Samples: 657225260. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:43:46,062][104569] Avg episode reward: [(0, '5395.292'), (1, '8922.874')] [2023-12-27 00:43:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001282704_328425472.pth... [2023-12-27 00:43:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001284320_328826880.pth... [2023-12-27 00:43:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001281616_328146944.pth [2023-12-27 00:43:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001283168_328531968.pth [2023-12-27 00:43:46,150][105620] Updated weights for policy 1, policy_version 1284326 (0.0008) [2023-12-27 00:43:46,205][105620] Updated weights for policy 1, policy_version 1284336 (0.0007) [2023-12-27 00:43:46,260][105620] Updated weights for policy 1, policy_version 1284346 (0.0005) [2023-12-27 00:43:46,457][105692] Updated weights for policy 0, policy_version 1282710 (0.0008) [2023-12-27 00:43:46,518][105692] Updated weights for policy 0, policy_version 1282720 (0.0009) [2023-12-27 00:43:46,571][105692] Updated weights for policy 0, policy_version 1282730 (0.0008) [2023-12-27 00:43:46,905][105620] Updated weights for policy 1, policy_version 1284356 (0.0007) [2023-12-27 00:43:46,964][105620] Updated weights for policy 1, policy_version 1284366 (0.0009) [2023-12-27 00:43:47,021][105620] Updated weights for policy 1, policy_version 1284376 (0.0009) [2023-12-27 00:43:47,351][105692] Updated weights for policy 0, policy_version 1282740 (0.0008) [2023-12-27 00:43:47,410][105692] Updated weights for policy 0, policy_version 1282750 (0.0009) [2023-12-27 00:43:47,458][105692] Updated weights for policy 0, policy_version 1282760 (0.0009) [2023-12-27 00:43:47,735][105620] Updated weights for policy 1, policy_version 1284386 (0.0009) [2023-12-27 00:43:47,824][105620] Updated weights for policy 1, policy_version 1284396 (0.0011) [2023-12-27 00:43:47,881][105620] Updated weights for policy 1, policy_version 1284406 (0.0008) [2023-12-27 00:43:47,944][105620] Updated weights for policy 1, policy_version 1284416 (0.0006) [2023-12-27 00:43:48,256][105692] Updated weights for policy 0, policy_version 1282770 (0.0008) [2023-12-27 00:43:48,320][105692] Updated weights for policy 0, policy_version 1282780 (0.0007) [2023-12-27 00:43:48,381][105692] Updated weights for policy 0, policy_version 1282790 (0.0007) [2023-12-27 00:43:48,447][105692] Updated weights for policy 0, policy_version 1282800 (0.0007) [2023-12-27 00:43:48,602][105620] Updated weights for policy 1, policy_version 1284426 (0.0007) [2023-12-27 00:43:48,667][105620] Updated weights for policy 1, policy_version 1284436 (0.0008) [2023-12-27 00:43:48,734][105620] Updated weights for policy 1, policy_version 1284446 (0.0008) [2023-12-27 00:43:49,147][105692] Updated weights for policy 0, policy_version 1282810 (0.0008) [2023-12-27 00:43:49,206][105692] Updated weights for policy 0, policy_version 1282820 (0.0008) [2023-12-27 00:43:49,266][105692] Updated weights for policy 0, policy_version 1282830 (0.0008) [2023-12-27 00:43:49,388][105620] Updated weights for policy 1, policy_version 1284456 (0.0008) [2023-12-27 00:43:49,441][105620] Updated weights for policy 1, policy_version 1284466 (0.0011) [2023-12-27 00:43:49,486][105620] Updated weights for policy 1, policy_version 1284476 (0.0011) [2023-12-27 00:43:50,039][105692] Updated weights for policy 0, policy_version 1282840 (0.0009) [2023-12-27 00:43:50,096][105692] Updated weights for policy 0, policy_version 1282850 (0.0008) [2023-12-27 00:43:50,152][105692] Updated weights for policy 0, policy_version 1282860 (0.0008) [2023-12-27 00:43:50,300][105620] Updated weights for policy 1, policy_version 1284486 (0.0011) [2023-12-27 00:43:50,360][105620] Updated weights for policy 1, policy_version 1284496 (0.0010) [2023-12-27 00:43:50,429][105620] Updated weights for policy 1, policy_version 1284506 (0.0009) [2023-12-27 00:43:50,950][105692] Updated weights for policy 0, policy_version 1282870 (0.0010) [2023-12-27 00:43:50,998][105692] Updated weights for policy 0, policy_version 1282880 (0.0011) [2023-12-27 00:43:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18978.2, 300 sec: 19494.2). Total num frames: 657342464. Throughput: 0: 9293.9, 1: 9822.8. Samples: 657339300. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:43:51,062][104569] Avg episode reward: [(0, '8352.472'), (1, '8820.209')] [2023-12-27 00:43:51,064][105692] Updated weights for policy 0, policy_version 1282890 (0.0010) [2023-12-27 00:43:51,147][105620] Updated weights for policy 1, policy_version 1284516 (0.0009) [2023-12-27 00:43:51,210][105620] Updated weights for policy 1, policy_version 1284526 (0.0011) [2023-12-27 00:43:51,275][105620] Updated weights for policy 1, policy_version 1284536 (0.0009) [2023-12-27 00:43:51,759][105692] Updated weights for policy 0, policy_version 1282900 (0.0005) [2023-12-27 00:43:51,827][105692] Updated weights for policy 0, policy_version 1282910 (0.0006) [2023-12-27 00:43:51,894][105692] Updated weights for policy 0, policy_version 1282920 (0.0005) [2023-12-27 00:43:51,975][105620] Updated weights for policy 1, policy_version 1284546 (0.0011) [2023-12-27 00:43:52,038][105620] Updated weights for policy 1, policy_version 1284556 (0.0011) [2023-12-27 00:43:52,100][105620] Updated weights for policy 1, policy_version 1284566 (0.0011) [2023-12-27 00:43:52,163][105620] Updated weights for policy 1, policy_version 1284576 (0.0011) [2023-12-27 00:43:52,563][105692] Updated weights for policy 0, policy_version 1282930 (0.0007) [2023-12-27 00:43:52,628][105692] Updated weights for policy 0, policy_version 1282940 (0.0006) [2023-12-27 00:43:52,694][105692] Updated weights for policy 0, policy_version 1282950 (0.0005) [2023-12-27 00:43:52,748][105692] Updated weights for policy 0, policy_version 1282960 (0.0006) [2023-12-27 00:43:52,880][105620] Updated weights for policy 1, policy_version 1284586 (0.0010) [2023-12-27 00:43:52,939][105620] Updated weights for policy 1, policy_version 1284596 (0.0010) [2023-12-27 00:43:53,001][105620] Updated weights for policy 1, policy_version 1284606 (0.0010) [2023-12-27 00:43:53,431][105692] Updated weights for policy 0, policy_version 1282970 (0.0010) [2023-12-27 00:43:53,498][105692] Updated weights for policy 0, policy_version 1282980 (0.0006) [2023-12-27 00:43:53,549][105692] Updated weights for policy 0, policy_version 1282990 (0.0010) [2023-12-27 00:43:53,629][105620] Updated weights for policy 1, policy_version 1284616 (0.0010) [2023-12-27 00:43:53,672][105620] Updated weights for policy 1, policy_version 1284626 (0.0009) [2023-12-27 00:43:53,731][105620] Updated weights for policy 1, policy_version 1284636 (0.0005) [2023-12-27 00:43:54,293][105692] Updated weights for policy 0, policy_version 1283000 (0.0011) [2023-12-27 00:43:54,345][105620] Updated weights for policy 1, policy_version 1284646 (0.0006) [2023-12-27 00:43:54,348][105692] Updated weights for policy 0, policy_version 1283010 (0.0010) [2023-12-27 00:43:54,401][105620] Updated weights for policy 1, policy_version 1284656 (0.0006) [2023-12-27 00:43:54,408][105692] Updated weights for policy 0, policy_version 1283020 (0.0011) [2023-12-27 00:43:54,466][105620] Updated weights for policy 1, policy_version 1284666 (0.0007) [2023-12-27 00:43:55,008][105620] Updated weights for policy 1, policy_version 1284676 (0.0007) [2023-12-27 00:43:55,069][105620] Updated weights for policy 1, policy_version 1284686 (0.0006) [2023-12-27 00:43:55,120][105620] Updated weights for policy 1, policy_version 1284696 (0.0006) [2023-12-27 00:43:55,149][105692] Updated weights for policy 0, policy_version 1283030 (0.0010) [2023-12-27 00:43:55,213][105692] Updated weights for policy 0, policy_version 1283040 (0.0011) [2023-12-27 00:43:55,277][105692] Updated weights for policy 0, policy_version 1283050 (0.0011) [2023-12-27 00:43:55,778][105620] Updated weights for policy 1, policy_version 1284706 (0.0005) [2023-12-27 00:43:55,826][105620] Updated weights for policy 1, policy_version 1284716 (0.0005) [2023-12-27 00:43:55,882][105620] Updated weights for policy 1, policy_version 1284726 (0.0005) [2023-12-27 00:43:55,929][105620] Updated weights for policy 1, policy_version 1284736 (0.0005) [2023-12-27 00:43:56,034][105692] Updated weights for policy 0, policy_version 1283060 (0.0010) [2023-12-27 00:43:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 657448960. Throughput: 0: 9265.2, 1: 9949.6. Samples: 657458712. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:43:56,063][104569] Avg episode reward: [(0, '9083.014'), (1, '8916.972')] [2023-12-27 00:43:56,098][105692] Updated weights for policy 0, policy_version 1283070 (0.0009) [2023-12-27 00:43:56,162][105692] Updated weights for policy 0, policy_version 1283080 (0.0009) [2023-12-27 00:43:56,503][105620] Updated weights for policy 1, policy_version 1284746 (0.0009) [2023-12-27 00:43:56,552][105620] Updated weights for policy 1, policy_version 1284756 (0.0007) [2023-12-27 00:43:56,604][105620] Updated weights for policy 1, policy_version 1284766 (0.0010) [2023-12-27 00:43:56,840][105692] Updated weights for policy 0, policy_version 1283090 (0.0009) [2023-12-27 00:43:56,888][105692] Updated weights for policy 0, policy_version 1283100 (0.0010) [2023-12-27 00:43:56,932][105692] Updated weights for policy 0, policy_version 1283110 (0.0010) [2023-12-27 00:43:56,979][105692] Updated weights for policy 0, policy_version 1283120 (0.0010) [2023-12-27 00:43:57,319][105620] Updated weights for policy 1, policy_version 1284776 (0.0006) [2023-12-27 00:43:57,371][105620] Updated weights for policy 1, policy_version 1284786 (0.0005) [2023-12-27 00:43:57,414][105620] Updated weights for policy 1, policy_version 1284796 (0.0005) [2023-12-27 00:43:57,663][105692] Updated weights for policy 0, policy_version 1283130 (0.0005) [2023-12-27 00:43:57,719][105692] Updated weights for policy 0, policy_version 1283140 (0.0005) [2023-12-27 00:43:57,769][105692] Updated weights for policy 0, policy_version 1283150 (0.0010) [2023-12-27 00:43:58,060][105620] Updated weights for policy 1, policy_version 1284806 (0.0006) [2023-12-27 00:43:58,114][105620] Updated weights for policy 1, policy_version 1284816 (0.0006) [2023-12-27 00:43:58,181][105620] Updated weights for policy 1, policy_version 1284826 (0.0007) [2023-12-27 00:43:58,500][105692] Updated weights for policy 0, policy_version 1283160 (0.0008) [2023-12-27 00:43:58,553][105692] Updated weights for policy 0, policy_version 1283170 (0.0008) [2023-12-27 00:43:58,610][105692] Updated weights for policy 0, policy_version 1283180 (0.0010) [2023-12-27 00:43:58,912][105620] Updated weights for policy 1, policy_version 1284836 (0.0009) [2023-12-27 00:43:58,978][105620] Updated weights for policy 1, policy_version 1284846 (0.0009) [2023-12-27 00:43:59,040][105620] Updated weights for policy 1, policy_version 1284856 (0.0009) [2023-12-27 00:43:59,375][105692] Updated weights for policy 0, policy_version 1283190 (0.0007) [2023-12-27 00:43:59,438][105692] Updated weights for policy 0, policy_version 1283200 (0.0006) [2023-12-27 00:43:59,505][105692] Updated weights for policy 0, policy_version 1283210 (0.0007) [2023-12-27 00:43:59,775][105620] Updated weights for policy 1, policy_version 1284866 (0.0009) [2023-12-27 00:43:59,842][105620] Updated weights for policy 1, policy_version 1284876 (0.0010) [2023-12-27 00:43:59,901][105620] Updated weights for policy 1, policy_version 1284886 (0.0010) [2023-12-27 00:43:59,970][105620] Updated weights for policy 1, policy_version 1284896 (0.0011) [2023-12-27 00:44:00,119][105692] Updated weights for policy 0, policy_version 1283220 (0.0008) [2023-12-27 00:44:00,175][105692] Updated weights for policy 0, policy_version 1283230 (0.0010) [2023-12-27 00:44:00,233][105692] Updated weights for policy 0, policy_version 1283240 (0.0011) [2023-12-27 00:44:00,581][105620] Updated weights for policy 1, policy_version 1284906 (0.0010) [2023-12-27 00:44:00,650][105620] Updated weights for policy 1, policy_version 1284916 (0.0010) [2023-12-27 00:44:00,711][105620] Updated weights for policy 1, policy_version 1284926 (0.0010) [2023-12-27 00:44:00,978][105692] Updated weights for policy 0, policy_version 1283250 (0.0011) [2023-12-27 00:44:01,044][105692] Updated weights for policy 0, policy_version 1283260 (0.0009) [2023-12-27 00:44:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 657547264. Throughput: 0: 9322.6, 1: 9998.5. Samples: 657519276. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:44:01,063][104569] Avg episode reward: [(0, '9354.572'), (1, '9010.620')] [2023-12-27 00:44:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001284928_328982528.pth... [2023-12-27 00:44:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001283776_328687616.pth [2023-12-27 00:44:01,107][105692] Updated weights for policy 0, policy_version 1283270 (0.0007) [2023-12-27 00:44:01,171][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001283280_328572928.pth... [2023-12-27 00:44:01,171][105692] Updated weights for policy 0, policy_version 1283280 (0.0007) [2023-12-27 00:44:01,176][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001282160_328286208.pth [2023-12-27 00:44:01,465][105620] Updated weights for policy 1, policy_version 1284936 (0.0010) [2023-12-27 00:44:01,523][105620] Updated weights for policy 1, policy_version 1284946 (0.0010) [2023-12-27 00:44:01,575][105620] Updated weights for policy 1, policy_version 1284956 (0.0010) [2023-12-27 00:44:01,798][105692] Updated weights for policy 0, policy_version 1283290 (0.0007) [2023-12-27 00:44:01,851][105692] Updated weights for policy 0, policy_version 1283300 (0.0006) [2023-12-27 00:44:01,902][105692] Updated weights for policy 0, policy_version 1283310 (0.0007) [2023-12-27 00:44:02,283][105620] Updated weights for policy 1, policy_version 1284966 (0.0009) [2023-12-27 00:44:02,340][105620] Updated weights for policy 1, policy_version 1284976 (0.0009) [2023-12-27 00:44:02,395][105620] Updated weights for policy 1, policy_version 1284986 (0.0009) [2023-12-27 00:44:02,655][105692] Updated weights for policy 0, policy_version 1283320 (0.0009) [2023-12-27 00:44:02,705][105692] Updated weights for policy 0, policy_version 1283330 (0.0009) [2023-12-27 00:44:02,753][105692] Updated weights for policy 0, policy_version 1283340 (0.0009) [2023-12-27 00:44:03,097][105620] Updated weights for policy 1, policy_version 1284996 (0.0009) [2023-12-27 00:44:03,145][105620] Updated weights for policy 1, policy_version 1285006 (0.0008) [2023-12-27 00:44:03,194][105620] Updated weights for policy 1, policy_version 1285016 (0.0007) [2023-12-27 00:44:03,486][105692] Updated weights for policy 0, policy_version 1283350 (0.0010) [2023-12-27 00:44:03,538][105692] Updated weights for policy 0, policy_version 1283360 (0.0010) [2023-12-27 00:44:03,588][105692] Updated weights for policy 0, policy_version 1283370 (0.0010) [2023-12-27 00:44:04,007][105620] Updated weights for policy 1, policy_version 1285026 (0.0007) [2023-12-27 00:44:04,057][105620] Updated weights for policy 1, policy_version 1285036 (0.0005) [2023-12-27 00:44:04,115][105620] Updated weights for policy 1, policy_version 1285046 (0.0007) [2023-12-27 00:44:04,174][105620] Updated weights for policy 1, policy_version 1285056 (0.0009) [2023-12-27 00:44:04,198][105692] Updated weights for policy 0, policy_version 1283380 (0.0009) [2023-12-27 00:44:04,255][105692] Updated weights for policy 0, policy_version 1283390 (0.0009) [2023-12-27 00:44:04,307][105692] Updated weights for policy 0, policy_version 1283400 (0.0009) [2023-12-27 00:44:04,858][105620] Updated weights for policy 1, policy_version 1285066 (0.0009) [2023-12-27 00:44:04,907][105620] Updated weights for policy 1, policy_version 1285076 (0.0009) [2023-12-27 00:44:04,955][105620] Updated weights for policy 1, policy_version 1285086 (0.0009) [2023-12-27 00:44:05,047][105692] Updated weights for policy 0, policy_version 1283410 (0.0010) [2023-12-27 00:44:05,109][105692] Updated weights for policy 0, policy_version 1283420 (0.0009) [2023-12-27 00:44:05,170][105692] Updated weights for policy 0, policy_version 1283430 (0.0009) [2023-12-27 00:44:05,225][105692] Updated weights for policy 0, policy_version 1283440 (0.0009) [2023-12-27 00:44:05,723][105620] Updated weights for policy 1, policy_version 1285096 (0.0009) [2023-12-27 00:44:05,786][105620] Updated weights for policy 1, policy_version 1285106 (0.0009) [2023-12-27 00:44:05,840][105620] Updated weights for policy 1, policy_version 1285116 (0.0008) [2023-12-27 00:44:05,969][105692] Updated weights for policy 0, policy_version 1283450 (0.0008) [2023-12-27 00:44:06,024][105692] Updated weights for policy 0, policy_version 1283460 (0.0009) [2023-12-27 00:44:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.3, 300 sec: 19494.2). Total num frames: 657645568. Throughput: 0: 9352.3, 1: 9931.3. Samples: 657635952. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:44:06,062][104569] Avg episode reward: [(0, '9089.146'), (1, '8977.378')] [2023-12-27 00:44:06,086][105692] Updated weights for policy 0, policy_version 1283470 (0.0009) [2023-12-27 00:44:06,616][105620] Updated weights for policy 1, policy_version 1285126 (0.0008) [2023-12-27 00:44:06,675][105620] Updated weights for policy 1, policy_version 1285136 (0.0009) [2023-12-27 00:44:06,735][105620] Updated weights for policy 1, policy_version 1285146 (0.0009) [2023-12-27 00:44:06,876][105692] Updated weights for policy 0, policy_version 1283480 (0.0010) [2023-12-27 00:44:06,942][105692] Updated weights for policy 0, policy_version 1283490 (0.0007) [2023-12-27 00:44:07,001][105692] Updated weights for policy 0, policy_version 1283500 (0.0008) [2023-12-27 00:44:07,495][105620] Updated weights for policy 1, policy_version 1285156 (0.0009) [2023-12-27 00:44:07,554][105620] Updated weights for policy 1, policy_version 1285166 (0.0010) [2023-12-27 00:44:07,603][105620] Updated weights for policy 1, policy_version 1285176 (0.0010) [2023-12-27 00:44:07,621][105692] Updated weights for policy 0, policy_version 1283510 (0.0007) [2023-12-27 00:44:07,677][105692] Updated weights for policy 0, policy_version 1283520 (0.0006) [2023-12-27 00:44:07,734][105692] Updated weights for policy 0, policy_version 1283530 (0.0006) [2023-12-27 00:44:08,357][105620] Updated weights for policy 1, policy_version 1285186 (0.0010) [2023-12-27 00:44:08,416][105620] Updated weights for policy 1, policy_version 1285196 (0.0011) [2023-12-27 00:44:08,451][105692] Updated weights for policy 0, policy_version 1283540 (0.0005) [2023-12-27 00:44:08,477][105620] Updated weights for policy 1, policy_version 1285206 (0.0011) [2023-12-27 00:44:08,500][105692] Updated weights for policy 0, policy_version 1283550 (0.0006) [2023-12-27 00:44:08,530][105620] Updated weights for policy 1, policy_version 1285216 (0.0011) [2023-12-27 00:44:08,546][105692] Updated weights for policy 0, policy_version 1283560 (0.0007) [2023-12-27 00:44:09,273][105692] Updated weights for policy 0, policy_version 1283570 (0.0008) [2023-12-27 00:44:09,292][105620] Updated weights for policy 1, policy_version 1285226 (0.0011) [2023-12-27 00:44:09,334][105692] Updated weights for policy 0, policy_version 1283580 (0.0008) [2023-12-27 00:44:09,359][105620] Updated weights for policy 1, policy_version 1285236 (0.0010) [2023-12-27 00:44:09,400][105692] Updated weights for policy 0, policy_version 1283590 (0.0008) [2023-12-27 00:44:09,421][105620] Updated weights for policy 1, policy_version 1285246 (0.0011) [2023-12-27 00:44:09,459][105692] Updated weights for policy 0, policy_version 1283600 (0.0008) [2023-12-27 00:44:10,167][105620] Updated weights for policy 1, policy_version 1285256 (0.0011) [2023-12-27 00:44:10,227][105620] Updated weights for policy 1, policy_version 1285266 (0.0010) [2023-12-27 00:44:10,229][105692] Updated weights for policy 0, policy_version 1283610 (0.0009) [2023-12-27 00:44:10,279][105692] Updated weights for policy 0, policy_version 1283620 (0.0008) [2023-12-27 00:44:10,290][105620] Updated weights for policy 1, policy_version 1285276 (0.0011) [2023-12-27 00:44:10,332][105692] Updated weights for policy 0, policy_version 1283630 (0.0005) [2023-12-27 00:44:10,936][105620] Updated weights for policy 1, policy_version 1285286 (0.0008) [2023-12-27 00:44:11,003][105620] Updated weights for policy 1, policy_version 1285296 (0.0006) [2023-12-27 00:44:11,033][105692] Updated weights for policy 0, policy_version 1283640 (0.0006) [2023-12-27 00:44:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 657735680. Throughput: 0: 9465.8, 1: 9861.0. Samples: 657750880. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:44:11,062][104569] Avg episode reward: [(0, '8911.686'), (1, '9138.818')] [2023-12-27 00:44:11,072][105620] Updated weights for policy 1, policy_version 1285306 (0.0007) [2023-12-27 00:44:11,094][105692] Updated weights for policy 0, policy_version 1283650 (0.0009) [2023-12-27 00:44:11,159][105692] Updated weights for policy 0, policy_version 1283660 (0.0009) [2023-12-27 00:44:11,781][105692] Updated weights for policy 0, policy_version 1283670 (0.0006) [2023-12-27 00:44:11,835][105692] Updated weights for policy 0, policy_version 1283680 (0.0006) [2023-12-27 00:44:11,869][105620] Updated weights for policy 1, policy_version 1285316 (0.0007) [2023-12-27 00:44:11,898][105692] Updated weights for policy 0, policy_version 1283690 (0.0005) [2023-12-27 00:44:11,929][105620] Updated weights for policy 1, policy_version 1285326 (0.0008) [2023-12-27 00:44:11,994][105620] Updated weights for policy 1, policy_version 1285336 (0.0005) [2023-12-27 00:44:12,545][105692] Updated weights for policy 0, policy_version 1283700 (0.0007) [2023-12-27 00:44:12,594][105692] Updated weights for policy 0, policy_version 1283710 (0.0009) [2023-12-27 00:44:12,654][105692] Updated weights for policy 0, policy_version 1283720 (0.0008) [2023-12-27 00:44:12,696][105620] Updated weights for policy 1, policy_version 1285346 (0.0006) [2023-12-27 00:44:12,751][105620] Updated weights for policy 1, policy_version 1285356 (0.0009) [2023-12-27 00:44:12,814][105620] Updated weights for policy 1, policy_version 1285366 (0.0008) [2023-12-27 00:44:12,878][105620] Updated weights for policy 1, policy_version 1285376 (0.0009) [2023-12-27 00:44:13,455][105692] Updated weights for policy 0, policy_version 1283730 (0.0008) [2023-12-27 00:44:13,517][105692] Updated weights for policy 0, policy_version 1283740 (0.0009) [2023-12-27 00:44:13,528][105620] Updated weights for policy 1, policy_version 1285386 (0.0005) [2023-12-27 00:44:13,579][105692] Updated weights for policy 0, policy_version 1283750 (0.0008) [2023-12-27 00:44:13,585][105620] Updated weights for policy 1, policy_version 1285396 (0.0006) [2023-12-27 00:44:13,641][105692] Updated weights for policy 0, policy_version 1283760 (0.0006) [2023-12-27 00:44:13,643][105620] Updated weights for policy 1, policy_version 1285406 (0.0007) [2023-12-27 00:44:14,293][105620] Updated weights for policy 1, policy_version 1285416 (0.0009) [2023-12-27 00:44:14,351][105620] Updated weights for policy 1, policy_version 1285426 (0.0009) [2023-12-27 00:44:14,407][105620] Updated weights for policy 1, policy_version 1285436 (0.0009) [2023-12-27 00:44:14,438][105692] Updated weights for policy 0, policy_version 1283770 (0.0006) [2023-12-27 00:44:14,500][105692] Updated weights for policy 0, policy_version 1283780 (0.0009) [2023-12-27 00:44:14,567][105692] Updated weights for policy 0, policy_version 1283790 (0.0009) [2023-12-27 00:44:15,085][105620] Updated weights for policy 1, policy_version 1285446 (0.0009) [2023-12-27 00:44:15,139][105620] Updated weights for policy 1, policy_version 1285456 (0.0005) [2023-12-27 00:44:15,213][105620] Updated weights for policy 1, policy_version 1285466 (0.0006) [2023-12-27 00:44:15,351][105692] Updated weights for policy 0, policy_version 1283800 (0.0009) [2023-12-27 00:44:15,414][105692] Updated weights for policy 0, policy_version 1283810 (0.0009) [2023-12-27 00:44:15,474][105692] Updated weights for policy 0, policy_version 1283820 (0.0009) [2023-12-27 00:44:15,899][105620] Updated weights for policy 1, policy_version 1285476 (0.0007) [2023-12-27 00:44:15,946][105620] Updated weights for policy 1, policy_version 1285486 (0.0008) [2023-12-27 00:44:15,994][105620] Updated weights for policy 1, policy_version 1285496 (0.0009) [2023-12-27 00:44:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 657842176. Throughput: 0: 9461.4, 1: 9865.7. Samples: 657809556. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 00:44:16,062][104569] Avg episode reward: [(0, '9088.119'), (1, '9265.402')] [2023-12-27 00:44:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001283824_328712192.pth... [2023-12-27 00:44:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001285504_329129984.pth... [2023-12-27 00:44:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001282704_328425472.pth [2023-12-27 00:44:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001284320_328826880.pth [2023-12-27 00:44:16,217][105692] Updated weights for policy 0, policy_version 1283830 (0.0008) [2023-12-27 00:44:16,276][105692] Updated weights for policy 0, policy_version 1283840 (0.0006) [2023-12-27 00:44:16,337][105692] Updated weights for policy 0, policy_version 1283850 (0.0006) [2023-12-27 00:44:16,786][105620] Updated weights for policy 1, policy_version 1285506 (0.0008) [2023-12-27 00:44:16,841][105620] Updated weights for policy 1, policy_version 1285516 (0.0011) [2023-12-27 00:44:16,893][105620] Updated weights for policy 1, policy_version 1285526 (0.0009) [2023-12-27 00:44:16,916][105692] Updated weights for policy 0, policy_version 1283860 (0.0006) [2023-12-27 00:44:16,939][105620] Updated weights for policy 1, policy_version 1285536 (0.0010) [2023-12-27 00:44:16,979][105692] Updated weights for policy 0, policy_version 1283870 (0.0006) [2023-12-27 00:44:17,036][105692] Updated weights for policy 0, policy_version 1283880 (0.0010) [2023-12-27 00:44:17,604][105692] Updated weights for policy 0, policy_version 1283890 (0.0010) [2023-12-27 00:44:17,663][105620] Updated weights for policy 1, policy_version 1285546 (0.0007) [2023-12-27 00:44:17,670][105692] Updated weights for policy 0, policy_version 1283900 (0.0011) [2023-12-27 00:44:17,710][105620] Updated weights for policy 1, policy_version 1285556 (0.0007) [2023-12-27 00:44:17,735][105692] Updated weights for policy 0, policy_version 1283910 (0.0011) [2023-12-27 00:44:17,769][105620] Updated weights for policy 1, policy_version 1285566 (0.0008) [2023-12-27 00:44:17,796][105692] Updated weights for policy 0, policy_version 1283920 (0.0010) [2023-12-27 00:44:18,425][105692] Updated weights for policy 0, policy_version 1283930 (0.0008) [2023-12-27 00:44:18,487][105692] Updated weights for policy 0, policy_version 1283940 (0.0010) [2023-12-27 00:44:18,506][105620] Updated weights for policy 1, policy_version 1285576 (0.0011) [2023-12-27 00:44:18,546][105692] Updated weights for policy 0, policy_version 1283950 (0.0011) [2023-12-27 00:44:18,558][105620] Updated weights for policy 1, policy_version 1285586 (0.0010) [2023-12-27 00:44:18,617][105620] Updated weights for policy 1, policy_version 1285596 (0.0010) [2023-12-27 00:44:19,148][105692] Updated weights for policy 0, policy_version 1283960 (0.0006) [2023-12-27 00:44:19,194][105692] Updated weights for policy 0, policy_version 1283970 (0.0005) [2023-12-27 00:44:19,260][105692] Updated weights for policy 0, policy_version 1283980 (0.0007) [2023-12-27 00:44:19,382][105620] Updated weights for policy 1, policy_version 1285606 (0.0011) [2023-12-27 00:44:19,437][105620] Updated weights for policy 1, policy_version 1285616 (0.0010) [2023-12-27 00:44:19,509][105620] Updated weights for policy 1, policy_version 1285626 (0.0010) [2023-12-27 00:44:20,014][105692] Updated weights for policy 0, policy_version 1283990 (0.0008) [2023-12-27 00:44:20,085][105692] Updated weights for policy 0, policy_version 1284000 (0.0008) [2023-12-27 00:44:20,150][105692] Updated weights for policy 0, policy_version 1284010 (0.0009) [2023-12-27 00:44:20,267][105620] Updated weights for policy 1, policy_version 1285636 (0.0009) [2023-12-27 00:44:20,324][105620] Updated weights for policy 1, policy_version 1285646 (0.0005) [2023-12-27 00:44:20,388][105620] Updated weights for policy 1, policy_version 1285656 (0.0006) [2023-12-27 00:44:20,875][105692] Updated weights for policy 0, policy_version 1284020 (0.0009) [2023-12-27 00:44:20,929][105692] Updated weights for policy 0, policy_version 1284030 (0.0011) [2023-12-27 00:44:20,994][105692] Updated weights for policy 0, policy_version 1284040 (0.0011) [2023-12-27 00:44:21,036][105620] Updated weights for policy 1, policy_version 1285666 (0.0007) [2023-12-27 00:44:21,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 657940480. Throughput: 0: 9611.5, 1: 9797.4. Samples: 657928096. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:44:21,063][104569] Avg episode reward: [(0, '8906.099'), (1, '8834.495')] [2023-12-27 00:44:21,096][105620] Updated weights for policy 1, policy_version 1285676 (0.0008) [2023-12-27 00:44:21,159][105620] Updated weights for policy 1, policy_version 1285686 (0.0009) [2023-12-27 00:44:21,218][105620] Updated weights for policy 1, policy_version 1285696 (0.0010) [2023-12-27 00:44:21,689][105692] Updated weights for policy 0, policy_version 1284050 (0.0010) [2023-12-27 00:44:21,758][105692] Updated weights for policy 0, policy_version 1284060 (0.0007) [2023-12-27 00:44:21,822][105692] Updated weights for policy 0, policy_version 1284070 (0.0011) [2023-12-27 00:44:21,886][105692] Updated weights for policy 0, policy_version 1284080 (0.0011) [2023-12-27 00:44:22,058][105620] Updated weights for policy 1, policy_version 1285706 (0.0010) [2023-12-27 00:44:22,113][105620] Updated weights for policy 1, policy_version 1285716 (0.0010) [2023-12-27 00:44:22,172][105620] Updated weights for policy 1, policy_version 1285726 (0.0009) [2023-12-27 00:44:22,617][105692] Updated weights for policy 0, policy_version 1284090 (0.0006) [2023-12-27 00:44:22,676][105692] Updated weights for policy 0, policy_version 1284100 (0.0007) [2023-12-27 00:44:22,737][105692] Updated weights for policy 0, policy_version 1284110 (0.0010) [2023-12-27 00:44:22,996][105620] Updated weights for policy 1, policy_version 1285736 (0.0008) [2023-12-27 00:44:23,063][105620] Updated weights for policy 1, policy_version 1285746 (0.0008) [2023-12-27 00:44:23,131][105620] Updated weights for policy 1, policy_version 1285756 (0.0008) [2023-12-27 00:44:23,463][105692] Updated weights for policy 0, policy_version 1284120 (0.0009) [2023-12-27 00:44:23,518][105692] Updated weights for policy 0, policy_version 1284130 (0.0008) [2023-12-27 00:44:23,583][105692] Updated weights for policy 0, policy_version 1284140 (0.0009) [2023-12-27 00:44:23,890][105620] Updated weights for policy 1, policy_version 1285766 (0.0008) [2023-12-27 00:44:23,954][105620] Updated weights for policy 1, policy_version 1285777 (0.0007) [2023-12-27 00:44:24,013][105620] Updated weights for policy 1, policy_version 1285787 (0.0006) [2023-12-27 00:44:24,257][105692] Updated weights for policy 0, policy_version 1284150 (0.0007) [2023-12-27 00:44:24,313][105692] Updated weights for policy 0, policy_version 1284160 (0.0005) [2023-12-27 00:44:24,372][105692] Updated weights for policy 0, policy_version 1284170 (0.0005) [2023-12-27 00:44:24,744][105620] Updated weights for policy 1, policy_version 1285797 (0.0009) [2023-12-27 00:44:24,801][105620] Updated weights for policy 1, policy_version 1285808 (0.0009) [2023-12-27 00:44:24,854][105620] Updated weights for policy 1, policy_version 1285818 (0.0010) [2023-12-27 00:44:24,877][105692] Updated weights for policy 0, policy_version 1284180 (0.0005) [2023-12-27 00:44:24,931][105692] Updated weights for policy 0, policy_version 1284190 (0.0005) [2023-12-27 00:44:24,983][105692] Updated weights for policy 0, policy_version 1284200 (0.0005) [2023-12-27 00:44:25,601][105620] Updated weights for policy 1, policy_version 1285828 (0.0007) [2023-12-27 00:44:25,662][105620] Updated weights for policy 1, policy_version 1285838 (0.0005) [2023-12-27 00:44:25,674][105692] Updated weights for policy 0, policy_version 1284210 (0.0009) [2023-12-27 00:44:25,721][105620] Updated weights for policy 1, policy_version 1285848 (0.0007) [2023-12-27 00:44:25,735][105692] Updated weights for policy 0, policy_version 1284220 (0.0010) [2023-12-27 00:44:25,786][105692] Updated weights for policy 0, policy_version 1284230 (0.0010) [2023-12-27 00:44:25,834][105692] Updated weights for policy 0, policy_version 1284240 (0.0010) [2023-12-27 00:44:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 658038784. Throughput: 0: 9685.4, 1: 9728.2. Samples: 658044256. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:44:26,062][104569] Avg episode reward: [(0, '8909.499'), (1, '8935.990')] [2023-12-27 00:44:26,300][105620] Updated weights for policy 1, policy_version 1285858 (0.0008) [2023-12-27 00:44:26,365][105620] Updated weights for policy 1, policy_version 1285868 (0.0010) [2023-12-27 00:44:26,429][105620] Updated weights for policy 1, policy_version 1285878 (0.0010) [2023-12-27 00:44:26,481][105620] Updated weights for policy 1, policy_version 1285888 (0.0009) [2023-12-27 00:44:26,569][105692] Updated weights for policy 0, policy_version 1284250 (0.0011) [2023-12-27 00:44:26,633][105692] Updated weights for policy 0, policy_version 1284260 (0.0008) [2023-12-27 00:44:26,696][105692] Updated weights for policy 0, policy_version 1284270 (0.0005) [2023-12-27 00:44:27,181][105620] Updated weights for policy 1, policy_version 1285898 (0.0010) [2023-12-27 00:44:27,235][105620] Updated weights for policy 1, policy_version 1285908 (0.0010) [2023-12-27 00:44:27,274][105692] Updated weights for policy 0, policy_version 1284280 (0.0009) [2023-12-27 00:44:27,294][105620] Updated weights for policy 1, policy_version 1285918 (0.0010) [2023-12-27 00:44:27,329][105692] Updated weights for policy 0, policy_version 1284290 (0.0010) [2023-12-27 00:44:27,384][105692] Updated weights for policy 0, policy_version 1284300 (0.0010) [2023-12-27 00:44:27,962][105620] Updated weights for policy 1, policy_version 1285928 (0.0008) [2023-12-27 00:44:28,013][105620] Updated weights for policy 1, policy_version 1285938 (0.0007) [2023-12-27 00:44:28,069][105620] Updated weights for policy 1, policy_version 1285948 (0.0009) [2023-12-27 00:44:28,119][105692] Updated weights for policy 0, policy_version 1284310 (0.0010) [2023-12-27 00:44:28,176][105692] Updated weights for policy 0, policy_version 1284320 (0.0010) [2023-12-27 00:44:28,220][105692] Updated weights for policy 0, policy_version 1284330 (0.0010) [2023-12-27 00:44:28,772][105620] Updated weights for policy 1, policy_version 1285958 (0.0006) [2023-12-27 00:44:28,831][105620] Updated weights for policy 1, policy_version 1285968 (0.0009) [2023-12-27 00:44:28,895][105620] Updated weights for policy 1, policy_version 1285978 (0.0010) [2023-12-27 00:44:28,933][105692] Updated weights for policy 0, policy_version 1284340 (0.0010) [2023-12-27 00:44:28,995][105692] Updated weights for policy 0, policy_version 1284350 (0.0009) [2023-12-27 00:44:29,059][105692] Updated weights for policy 0, policy_version 1284360 (0.0007) [2023-12-27 00:44:29,485][105620] Updated weights for policy 1, policy_version 1285988 (0.0006) [2023-12-27 00:44:29,549][105620] Updated weights for policy 1, policy_version 1285998 (0.0008) [2023-12-27 00:44:29,608][105620] Updated weights for policy 1, policy_version 1286008 (0.0008) [2023-12-27 00:44:29,811][105692] Updated weights for policy 0, policy_version 1284370 (0.0010) [2023-12-27 00:44:29,875][105692] Updated weights for policy 0, policy_version 1284380 (0.0007) [2023-12-27 00:44:29,938][105692] Updated weights for policy 0, policy_version 1284390 (0.0007) [2023-12-27 00:44:29,999][105692] Updated weights for policy 0, policy_version 1284400 (0.0005) [2023-12-27 00:44:30,340][105620] Updated weights for policy 1, policy_version 1286018 (0.0009) [2023-12-27 00:44:30,397][105620] Updated weights for policy 1, policy_version 1286028 (0.0005) [2023-12-27 00:44:30,452][105620] Updated weights for policy 1, policy_version 1286038 (0.0005) [2023-12-27 00:44:30,509][105620] Updated weights for policy 1, policy_version 1286048 (0.0005) [2023-12-27 00:44:30,665][105692] Updated weights for policy 0, policy_version 1284410 (0.0010) [2023-12-27 00:44:30,722][105692] Updated weights for policy 0, policy_version 1284420 (0.0010) [2023-12-27 00:44:30,776][105692] Updated weights for policy 0, policy_version 1284430 (0.0010) [2023-12-27 00:44:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 658137088. Throughput: 0: 9747.8, 1: 9793.7. Samples: 658104628. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:44:31,063][104569] Avg episode reward: [(0, '8913.385'), (1, '9211.942')] [2023-12-27 00:44:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001284432_328867840.pth... [2023-12-27 00:44:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001286048_329269248.pth... [2023-12-27 00:44:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001283280_328572928.pth [2023-12-27 00:44:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001284928_328982528.pth [2023-12-27 00:44:31,203][105620] Updated weights for policy 1, policy_version 1286058 (0.0010) [2023-12-27 00:44:31,272][105620] Updated weights for policy 1, policy_version 1286068 (0.0011) [2023-12-27 00:44:31,333][105620] Updated weights for policy 1, policy_version 1286078 (0.0006) [2023-12-27 00:44:31,457][105692] Updated weights for policy 0, policy_version 1284440 (0.0010) [2023-12-27 00:44:31,516][105692] Updated weights for policy 0, policy_version 1284450 (0.0010) [2023-12-27 00:44:31,569][105692] Updated weights for policy 0, policy_version 1284460 (0.0010) [2023-12-27 00:44:32,035][105620] Updated weights for policy 1, policy_version 1286088 (0.0010) [2023-12-27 00:44:32,091][105620] Updated weights for policy 1, policy_version 1286098 (0.0010) [2023-12-27 00:44:32,157][105620] Updated weights for policy 1, policy_version 1286108 (0.0010) [2023-12-27 00:44:32,210][105692] Updated weights for policy 0, policy_version 1284470 (0.0007) [2023-12-27 00:44:32,273][105692] Updated weights for policy 0, policy_version 1284480 (0.0008) [2023-12-27 00:44:32,337][105692] Updated weights for policy 0, policy_version 1284490 (0.0011) [2023-12-27 00:44:32,901][105620] Updated weights for policy 1, policy_version 1286118 (0.0010) [2023-12-27 00:44:32,957][105620] Updated weights for policy 1, policy_version 1286128 (0.0006) [2023-12-27 00:44:33,011][105620] Updated weights for policy 1, policy_version 1286138 (0.0009) [2023-12-27 00:44:33,017][105692] Updated weights for policy 0, policy_version 1284500 (0.0011) [2023-12-27 00:44:33,073][105692] Updated weights for policy 0, policy_version 1284510 (0.0005) [2023-12-27 00:44:33,121][105692] Updated weights for policy 0, policy_version 1284520 (0.0005) [2023-12-27 00:44:33,646][105692] Updated weights for policy 0, policy_version 1284530 (0.0005) [2023-12-27 00:44:33,705][105692] Updated weights for policy 0, policy_version 1284540 (0.0005) [2023-12-27 00:44:33,757][105692] Updated weights for policy 0, policy_version 1284550 (0.0005) [2023-12-27 00:44:33,805][105692] Updated weights for policy 0, policy_version 1284560 (0.0008) [2023-12-27 00:44:33,877][105620] Updated weights for policy 1, policy_version 1286148 (0.0009) [2023-12-27 00:44:33,926][105620] Updated weights for policy 1, policy_version 1286158 (0.0008) [2023-12-27 00:44:33,979][105620] Updated weights for policy 1, policy_version 1286168 (0.0009) [2023-12-27 00:44:34,531][105692] Updated weights for policy 0, policy_version 1284570 (0.0009) [2023-12-27 00:44:34,587][105692] Updated weights for policy 0, policy_version 1284580 (0.0008) [2023-12-27 00:44:34,647][105692] Updated weights for policy 0, policy_version 1284590 (0.0009) [2023-12-27 00:44:34,735][105620] Updated weights for policy 1, policy_version 1286178 (0.0008) [2023-12-27 00:44:34,781][105620] Updated weights for policy 1, policy_version 1286188 (0.0005) [2023-12-27 00:44:34,829][105620] Updated weights for policy 1, policy_version 1286198 (0.0006) [2023-12-27 00:44:34,878][105620] Updated weights for policy 1, policy_version 1286208 (0.0005) [2023-12-27 00:44:35,493][105620] Updated weights for policy 1, policy_version 1286218 (0.0010) [2023-12-27 00:44:35,520][105692] Updated weights for policy 0, policy_version 1284600 (0.0009) [2023-12-27 00:44:35,551][105620] Updated weights for policy 1, policy_version 1286228 (0.0010) [2023-12-27 00:44:35,568][105692] Updated weights for policy 0, policy_version 1284610 (0.0010) [2023-12-27 00:44:35,609][105620] Updated weights for policy 1, policy_version 1286238 (0.0010) [2023-12-27 00:44:35,618][105692] Updated weights for policy 0, policy_version 1284620 (0.0007) [2023-12-27 00:44:36,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 658235392. Throughput: 0: 9913.3, 1: 9758.7. Samples: 658224544. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:44:36,063][104569] Avg episode reward: [(0, '8915.087'), (1, '9106.905')] [2023-12-27 00:44:36,247][105692] Updated weights for policy 0, policy_version 1284630 (0.0008) [2023-12-27 00:44:36,259][105620] Updated weights for policy 1, policy_version 1286248 (0.0009) [2023-12-27 00:44:36,310][105692] Updated weights for policy 0, policy_version 1284640 (0.0011) [2023-12-27 00:44:36,319][105620] Updated weights for policy 1, policy_version 1286258 (0.0011) [2023-12-27 00:44:36,376][105692] Updated weights for policy 0, policy_version 1284650 (0.0011) [2023-12-27 00:44:36,380][105620] Updated weights for policy 1, policy_version 1286268 (0.0011) [2023-12-27 00:44:37,060][105620] Updated weights for policy 1, policy_version 1286278 (0.0009) [2023-12-27 00:44:37,113][105620] Updated weights for policy 1, policy_version 1286288 (0.0010) [2023-12-27 00:44:37,127][105692] Updated weights for policy 0, policy_version 1284660 (0.0010) [2023-12-27 00:44:37,166][105620] Updated weights for policy 1, policy_version 1286298 (0.0010) [2023-12-27 00:44:37,179][105692] Updated weights for policy 0, policy_version 1284670 (0.0010) [2023-12-27 00:44:37,237][105692] Updated weights for policy 0, policy_version 1284680 (0.0009) [2023-12-27 00:44:37,915][105620] Updated weights for policy 1, policy_version 1286308 (0.0011) [2023-12-27 00:44:37,935][105692] Updated weights for policy 0, policy_version 1284690 (0.0006) [2023-12-27 00:44:37,965][105620] Updated weights for policy 1, policy_version 1286318 (0.0011) [2023-12-27 00:44:37,996][105692] Updated weights for policy 0, policy_version 1284700 (0.0011) [2023-12-27 00:44:38,026][105620] Updated weights for policy 1, policy_version 1286328 (0.0011) [2023-12-27 00:44:38,049][105692] Updated weights for policy 0, policy_version 1284710 (0.0011) [2023-12-27 00:44:38,109][105692] Updated weights for policy 0, policy_version 1284720 (0.0011) [2023-12-27 00:44:38,739][105692] Updated weights for policy 0, policy_version 1284730 (0.0011) [2023-12-27 00:44:38,788][105692] Updated weights for policy 0, policy_version 1284740 (0.0008) [2023-12-27 00:44:38,843][105692] Updated weights for policy 0, policy_version 1284750 (0.0005) [2023-12-27 00:44:38,843][105620] Updated weights for policy 1, policy_version 1286338 (0.0011) [2023-12-27 00:44:38,899][105620] Updated weights for policy 1, policy_version 1286348 (0.0009) [2023-12-27 00:44:38,956][105620] Updated weights for policy 1, policy_version 1286358 (0.0009) [2023-12-27 00:44:39,010][105620] Updated weights for policy 1, policy_version 1286368 (0.0009) [2023-12-27 00:44:39,507][105692] Updated weights for policy 0, policy_version 1284760 (0.0010) [2023-12-27 00:44:39,570][105692] Updated weights for policy 0, policy_version 1284770 (0.0011) [2023-12-27 00:44:39,630][105692] Updated weights for policy 0, policy_version 1284780 (0.0011) [2023-12-27 00:44:39,712][105620] Updated weights for policy 1, policy_version 1286378 (0.0006) [2023-12-27 00:44:39,770][105620] Updated weights for policy 1, policy_version 1286388 (0.0006) [2023-12-27 00:44:39,831][105620] Updated weights for policy 1, policy_version 1286398 (0.0007) [2023-12-27 00:44:40,399][105692] Updated weights for policy 0, policy_version 1284790 (0.0011) [2023-12-27 00:44:40,458][105692] Updated weights for policy 0, policy_version 1284800 (0.0010) [2023-12-27 00:44:40,489][105620] Updated weights for policy 1, policy_version 1286408 (0.0007) [2023-12-27 00:44:40,518][105692] Updated weights for policy 0, policy_version 1284810 (0.0011) [2023-12-27 00:44:40,544][105620] Updated weights for policy 1, policy_version 1286418 (0.0009) [2023-12-27 00:44:40,601][105620] Updated weights for policy 1, policy_version 1286428 (0.0008) [2023-12-27 00:44:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 658333696. Throughput: 0: 9949.7, 1: 9696.0. Samples: 658342768. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:44:41,062][104569] Avg episode reward: [(0, '9086.606'), (1, '9079.391')] [2023-12-27 00:44:41,267][105692] Updated weights for policy 0, policy_version 1284820 (0.0010) [2023-12-27 00:44:41,271][105620] Updated weights for policy 1, policy_version 1286438 (0.0008) [2023-12-27 00:44:41,330][105692] Updated weights for policy 0, policy_version 1284830 (0.0011) [2023-12-27 00:44:41,330][105620] Updated weights for policy 1, policy_version 1286448 (0.0011) [2023-12-27 00:44:41,397][105620] Updated weights for policy 1, policy_version 1286458 (0.0009) [2023-12-27 00:44:41,403][105692] Updated weights for policy 0, policy_version 1284840 (0.0010) [2023-12-27 00:44:42,134][105620] Updated weights for policy 1, policy_version 1286468 (0.0009) [2023-12-27 00:44:42,163][105692] Updated weights for policy 0, policy_version 1284850 (0.0011) [2023-12-27 00:44:42,201][105620] Updated weights for policy 1, policy_version 1286478 (0.0009) [2023-12-27 00:44:42,219][105692] Updated weights for policy 0, policy_version 1284860 (0.0010) [2023-12-27 00:44:42,265][105620] Updated weights for policy 1, policy_version 1286488 (0.0006) [2023-12-27 00:44:42,285][105692] Updated weights for policy 0, policy_version 1284870 (0.0009) [2023-12-27 00:44:42,359][105692] Updated weights for policy 0, policy_version 1284880 (0.0007) [2023-12-27 00:44:43,012][105692] Updated weights for policy 0, policy_version 1284890 (0.0009) [2023-12-27 00:44:43,033][105620] Updated weights for policy 1, policy_version 1286498 (0.0009) [2023-12-27 00:44:43,081][105692] Updated weights for policy 0, policy_version 1284900 (0.0008) [2023-12-27 00:44:43,090][105620] Updated weights for policy 1, policy_version 1286508 (0.0008) [2023-12-27 00:44:43,146][105692] Updated weights for policy 0, policy_version 1284910 (0.0009) [2023-12-27 00:44:43,146][105620] Updated weights for policy 1, policy_version 1286518 (0.0006) [2023-12-27 00:44:43,207][105620] Updated weights for policy 1, policy_version 1286528 (0.0009) [2023-12-27 00:44:43,845][105620] Updated weights for policy 1, policy_version 1286538 (0.0005) [2023-12-27 00:44:43,869][105692] Updated weights for policy 0, policy_version 1284920 (0.0010) [2023-12-27 00:44:43,898][105620] Updated weights for policy 1, policy_version 1286548 (0.0006) [2023-12-27 00:44:43,930][105692] Updated weights for policy 0, policy_version 1284930 (0.0010) [2023-12-27 00:44:43,955][105620] Updated weights for policy 1, policy_version 1286558 (0.0008) [2023-12-27 00:44:43,985][105692] Updated weights for policy 0, policy_version 1284940 (0.0010) [2023-12-27 00:44:44,632][105620] Updated weights for policy 1, policy_version 1286568 (0.0008) [2023-12-27 00:44:44,686][105620] Updated weights for policy 1, policy_version 1286578 (0.0008) [2023-12-27 00:44:44,723][105692] Updated weights for policy 0, policy_version 1284950 (0.0011) [2023-12-27 00:44:44,738][105620] Updated weights for policy 1, policy_version 1286588 (0.0006) [2023-12-27 00:44:44,784][105692] Updated weights for policy 0, policy_version 1284960 (0.0010) [2023-12-27 00:44:44,846][105692] Updated weights for policy 0, policy_version 1284970 (0.0010) [2023-12-27 00:44:45,521][105620] Updated weights for policy 1, policy_version 1286598 (0.0008) [2023-12-27 00:44:45,573][105620] Updated weights for policy 1, policy_version 1286608 (0.0008) [2023-12-27 00:44:45,581][105692] Updated weights for policy 0, policy_version 1284980 (0.0009) [2023-12-27 00:44:45,629][105620] Updated weights for policy 1, policy_version 1286618 (0.0007) [2023-12-27 00:44:45,641][105692] Updated weights for policy 0, policy_version 1284990 (0.0009) [2023-12-27 00:44:45,696][105692] Updated weights for policy 0, policy_version 1285000 (0.0010) [2023-12-27 00:44:46,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 658432000. Throughput: 0: 9923.7, 1: 9662.5. Samples: 658400652. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:44:46,062][104569] Avg episode reward: [(0, '8812.902'), (1, '9262.834')] [2023-12-27 00:44:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001285008_329015296.pth... [2023-12-27 00:44:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001286624_329416704.pth... [2023-12-27 00:44:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001285504_329129984.pth [2023-12-27 00:44:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001283824_328712192.pth [2023-12-27 00:44:46,396][105620] Updated weights for policy 1, policy_version 1286628 (0.0009) [2023-12-27 00:44:46,441][105692] Updated weights for policy 0, policy_version 1285010 (0.0010) [2023-12-27 00:44:46,455][105620] Updated weights for policy 1, policy_version 1286638 (0.0008) [2023-12-27 00:44:46,490][105692] Updated weights for policy 0, policy_version 1285020 (0.0006) [2023-12-27 00:44:46,514][105620] Updated weights for policy 1, policy_version 1286648 (0.0008) [2023-12-27 00:44:46,536][105692] Updated weights for policy 0, policy_version 1285030 (0.0007) [2023-12-27 00:44:46,581][105692] Updated weights for policy 0, policy_version 1285040 (0.0007) [2023-12-27 00:44:47,293][105620] Updated weights for policy 1, policy_version 1286658 (0.0007) [2023-12-27 00:44:47,303][105692] Updated weights for policy 0, policy_version 1285050 (0.0008) [2023-12-27 00:44:47,349][105620] Updated weights for policy 1, policy_version 1286668 (0.0007) [2023-12-27 00:44:47,356][105692] Updated weights for policy 0, policy_version 1285060 (0.0007) [2023-12-27 00:44:47,395][105620] Updated weights for policy 1, policy_version 1286678 (0.0006) [2023-12-27 00:44:47,407][105692] Updated weights for policy 0, policy_version 1285070 (0.0006) [2023-12-27 00:44:47,440][105620] Updated weights for policy 1, policy_version 1286688 (0.0008) [2023-12-27 00:44:48,166][105620] Updated weights for policy 1, policy_version 1286698 (0.0006) [2023-12-27 00:44:48,190][105692] Updated weights for policy 0, policy_version 1285080 (0.0007) [2023-12-27 00:44:48,228][105620] Updated weights for policy 1, policy_version 1286708 (0.0009) [2023-12-27 00:44:48,251][105692] Updated weights for policy 0, policy_version 1285090 (0.0006) [2023-12-27 00:44:48,287][105620] Updated weights for policy 1, policy_version 1286718 (0.0007) [2023-12-27 00:44:48,306][105692] Updated weights for policy 0, policy_version 1285100 (0.0009) [2023-12-27 00:44:49,020][105620] Updated weights for policy 1, policy_version 1286728 (0.0008) [2023-12-27 00:44:49,060][105692] Updated weights for policy 0, policy_version 1285110 (0.0008) [2023-12-27 00:44:49,073][105620] Updated weights for policy 1, policy_version 1286738 (0.0008) [2023-12-27 00:44:49,113][105692] Updated weights for policy 0, policy_version 1285120 (0.0007) [2023-12-27 00:44:49,129][105620] Updated weights for policy 1, policy_version 1286748 (0.0009) [2023-12-27 00:44:49,164][105692] Updated weights for policy 0, policy_version 1285130 (0.0009) [2023-12-27 00:44:49,922][105620] Updated weights for policy 1, policy_version 1286758 (0.0007) [2023-12-27 00:44:49,965][105692] Updated weights for policy 0, policy_version 1285140 (0.0009) [2023-12-27 00:44:49,980][105620] Updated weights for policy 1, policy_version 1286768 (0.0007) [2023-12-27 00:44:50,024][105692] Updated weights for policy 0, policy_version 1285150 (0.0007) [2023-12-27 00:44:50,038][105620] Updated weights for policy 1, policy_version 1286778 (0.0006) [2023-12-27 00:44:50,082][105692] Updated weights for policy 0, policy_version 1285160 (0.0007) [2023-12-27 00:44:50,799][105620] Updated weights for policy 1, policy_version 1286788 (0.0007) [2023-12-27 00:44:50,845][105692] Updated weights for policy 0, policy_version 1285170 (0.0008) [2023-12-27 00:44:50,850][105620] Updated weights for policy 1, policy_version 1286798 (0.0009) [2023-12-27 00:44:50,902][105692] Updated weights for policy 0, policy_version 1285180 (0.0006) [2023-12-27 00:44:50,911][105620] Updated weights for policy 1, policy_version 1286808 (0.0008) [2023-12-27 00:44:50,959][105692] Updated weights for policy 0, policy_version 1285190 (0.0007) [2023-12-27 00:44:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 658530304. Throughput: 0: 9871.8, 1: 9633.1. Samples: 658513672. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:44:51,062][104569] Avg episode reward: [(0, '8907.936'), (1, '9262.601')] [2023-12-27 00:44:51,626][105620] Updated weights for policy 1, policy_version 1286818 (0.0007) [2023-12-27 00:44:51,691][105620] Updated weights for policy 1, policy_version 1286828 (0.0009) [2023-12-27 00:44:51,742][105692] Updated weights for policy 0, policy_version 1285201 (0.0010) [2023-12-27 00:44:51,760][105620] Updated weights for policy 1, policy_version 1286838 (0.0009) [2023-12-27 00:44:51,799][105692] Updated weights for policy 0, policy_version 1285211 (0.0007) [2023-12-27 00:44:51,806][105620] Updated weights for policy 1, policy_version 1286848 (0.0006) [2023-12-27 00:44:51,859][105692] Updated weights for policy 0, policy_version 1285221 (0.0009) [2023-12-27 00:44:51,922][105692] Updated weights for policy 0, policy_version 1285231 (0.0009) [2023-12-27 00:44:52,558][105620] Updated weights for policy 1, policy_version 1286858 (0.0009) [2023-12-27 00:44:52,606][105620] Updated weights for policy 1, policy_version 1286868 (0.0009) [2023-12-27 00:44:52,666][105620] Updated weights for policy 1, policy_version 1286878 (0.0008) [2023-12-27 00:44:52,693][105692] Updated weights for policy 0, policy_version 1285241 (0.0009) [2023-12-27 00:44:52,752][105692] Updated weights for policy 0, policy_version 1285251 (0.0007) [2023-12-27 00:44:52,805][105692] Updated weights for policy 0, policy_version 1285261 (0.0005) [2023-12-27 00:44:53,439][105692] Updated weights for policy 0, policy_version 1285271 (0.0008) [2023-12-27 00:44:53,471][105620] Updated weights for policy 1, policy_version 1286888 (0.0006) [2023-12-27 00:44:53,497][105692] Updated weights for policy 0, policy_version 1285281 (0.0007) [2023-12-27 00:44:53,534][105620] Updated weights for policy 1, policy_version 1286898 (0.0009) [2023-12-27 00:44:53,557][105692] Updated weights for policy 0, policy_version 1285291 (0.0007) [2023-12-27 00:44:53,588][105620] Updated weights for policy 1, policy_version 1286908 (0.0006) [2023-12-27 00:44:54,243][105692] Updated weights for policy 0, policy_version 1285301 (0.0008) [2023-12-27 00:44:54,297][105692] Updated weights for policy 0, policy_version 1285311 (0.0009) [2023-12-27 00:44:54,347][105692] Updated weights for policy 0, policy_version 1285321 (0.0008) [2023-12-27 00:44:54,361][105620] Updated weights for policy 1, policy_version 1286918 (0.0008) [2023-12-27 00:44:54,419][105620] Updated weights for policy 1, policy_version 1286928 (0.0008) [2023-12-27 00:44:54,482][105620] Updated weights for policy 1, policy_version 1286938 (0.0009) [2023-12-27 00:44:55,041][105692] Updated weights for policy 0, policy_version 1285331 (0.0007) [2023-12-27 00:44:55,100][105692] Updated weights for policy 0, policy_version 1285341 (0.0009) [2023-12-27 00:44:55,158][105692] Updated weights for policy 0, policy_version 1285351 (0.0009) [2023-12-27 00:44:55,274][105620] Updated weights for policy 1, policy_version 1286948 (0.0009) [2023-12-27 00:44:55,339][105620] Updated weights for policy 1, policy_version 1286958 (0.0009) [2023-12-27 00:44:55,398][105620] Updated weights for policy 1, policy_version 1286968 (0.0009) [2023-12-27 00:44:55,846][105692] Updated weights for policy 0, policy_version 1285361 (0.0009) [2023-12-27 00:44:55,904][105692] Updated weights for policy 0, policy_version 1285371 (0.0009) [2023-12-27 00:44:55,957][105692] Updated weights for policy 0, policy_version 1285381 (0.0006) [2023-12-27 00:44:56,024][105692] Updated weights for policy 0, policy_version 1285391 (0.0005) [2023-12-27 00:44:56,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 658620416. Throughput: 0: 9860.4, 1: 9584.5. Samples: 658625904. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:44:56,063][104569] Avg episode reward: [(0, '9088.288'), (1, '9262.646')] [2023-12-27 00:44:56,222][105620] Updated weights for policy 1, policy_version 1286978 (0.0009) [2023-12-27 00:44:56,280][105620] Updated weights for policy 1, policy_version 1286988 (0.0009) [2023-12-27 00:44:56,326][105620] Updated weights for policy 1, policy_version 1286998 (0.0009) [2023-12-27 00:44:56,381][105620] Updated weights for policy 1, policy_version 1287008 (0.0009) [2023-12-27 00:44:56,620][105692] Updated weights for policy 0, policy_version 1285401 (0.0008) [2023-12-27 00:44:56,686][105692] Updated weights for policy 0, policy_version 1285411 (0.0009) [2023-12-27 00:44:56,744][105692] Updated weights for policy 0, policy_version 1285421 (0.0009) [2023-12-27 00:44:57,164][105620] Updated weights for policy 1, policy_version 1287018 (0.0009) [2023-12-27 00:44:57,214][105620] Updated weights for policy 1, policy_version 1287028 (0.0009) [2023-12-27 00:44:57,264][105620] Updated weights for policy 1, policy_version 1287038 (0.0009) [2023-12-27 00:44:57,427][105692] Updated weights for policy 0, policy_version 1285431 (0.0009) [2023-12-27 00:44:57,473][105692] Updated weights for policy 0, policy_version 1285441 (0.0008) [2023-12-27 00:44:57,523][105692] Updated weights for policy 0, policy_version 1285451 (0.0008) [2023-12-27 00:44:58,028][105620] Updated weights for policy 1, policy_version 1287048 (0.0008) [2023-12-27 00:44:58,081][105620] Updated weights for policy 1, policy_version 1287058 (0.0008) [2023-12-27 00:44:58,132][105620] Updated weights for policy 1, policy_version 1287068 (0.0009) [2023-12-27 00:44:58,292][105692] Updated weights for policy 0, policy_version 1285461 (0.0007) [2023-12-27 00:44:58,360][105692] Updated weights for policy 0, policy_version 1285471 (0.0008) [2023-12-27 00:44:58,420][105692] Updated weights for policy 0, policy_version 1285481 (0.0010) [2023-12-27 00:44:59,021][105620] Updated weights for policy 1, policy_version 1287078 (0.0009) [2023-12-27 00:44:59,091][105620] Updated weights for policy 1, policy_version 1287088 (0.0009) [2023-12-27 00:44:59,158][105620] Updated weights for policy 1, policy_version 1287098 (0.0009) [2023-12-27 00:44:59,197][105692] Updated weights for policy 0, policy_version 1285491 (0.0009) [2023-12-27 00:44:59,265][105692] Updated weights for policy 0, policy_version 1285501 (0.0009) [2023-12-27 00:44:59,331][105692] Updated weights for policy 0, policy_version 1285511 (0.0009) [2023-12-27 00:44:59,926][105620] Updated weights for policy 1, policy_version 1287108 (0.0007) [2023-12-27 00:44:59,985][105620] Updated weights for policy 1, policy_version 1287118 (0.0006) [2023-12-27 00:45:00,044][105620] Updated weights for policy 1, policy_version 1287128 (0.0008) [2023-12-27 00:45:00,169][105692] Updated weights for policy 0, policy_version 1285521 (0.0008) [2023-12-27 00:45:00,219][105692] Updated weights for policy 0, policy_version 1285531 (0.0006) [2023-12-27 00:45:00,269][105692] Updated weights for policy 0, policy_version 1285541 (0.0005) [2023-12-27 00:45:00,331][105692] Updated weights for policy 0, policy_version 1285551 (0.0006) [2023-12-27 00:45:00,788][105620] Updated weights for policy 1, policy_version 1287138 (0.0009) [2023-12-27 00:45:00,836][105620] Updated weights for policy 1, policy_version 1287148 (0.0007) [2023-12-27 00:45:00,887][105692] Updated weights for policy 0, policy_version 1285561 (0.0010) [2023-12-27 00:45:00,891][105620] Updated weights for policy 1, policy_version 1287158 (0.0010) [2023-12-27 00:45:00,934][105692] Updated weights for policy 0, policy_version 1285571 (0.0007) [2023-12-27 00:45:00,939][105620] Updated weights for policy 1, policy_version 1287168 (0.0010) [2023-12-27 00:45:00,985][105692] Updated weights for policy 0, policy_version 1285581 (0.0009) [2023-12-27 00:45:01,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 658718720. Throughput: 0: 9872.0, 1: 9529.9. Samples: 658682644. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:45:01,063][104569] Avg episode reward: [(0, '8909.296'), (1, '9262.935')] [2023-12-27 00:45:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001287168_329555968.pth... [2023-12-27 00:45:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001285584_329162752.pth... [2023-12-27 00:45:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001286048_329269248.pth [2023-12-27 00:45:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001284432_328867840.pth [2023-12-27 00:45:01,612][105620] Updated weights for policy 1, policy_version 1287178 (0.0008) [2023-12-27 00:45:01,679][105620] Updated weights for policy 1, policy_version 1287188 (0.0009) [2023-12-27 00:45:01,744][105620] Updated weights for policy 1, policy_version 1287198 (0.0009) [2023-12-27 00:45:01,826][105692] Updated weights for policy 0, policy_version 1285591 (0.0007) [2023-12-27 00:45:01,893][105692] Updated weights for policy 0, policy_version 1285601 (0.0005) [2023-12-27 00:45:01,959][105692] Updated weights for policy 0, policy_version 1285611 (0.0005) [2023-12-27 00:45:02,386][105620] Updated weights for policy 1, policy_version 1287208 (0.0007) [2023-12-27 00:45:02,443][105620] Updated weights for policy 1, policy_version 1287218 (0.0009) [2023-12-27 00:45:02,500][105620] Updated weights for policy 1, policy_version 1287228 (0.0007) [2023-12-27 00:45:02,560][105692] Updated weights for policy 0, policy_version 1285621 (0.0007) [2023-12-27 00:45:02,616][105692] Updated weights for policy 0, policy_version 1285631 (0.0008) [2023-12-27 00:45:02,678][105692] Updated weights for policy 0, policy_version 1285641 (0.0007) [2023-12-27 00:45:03,116][105620] Updated weights for policy 1, policy_version 1287238 (0.0006) [2023-12-27 00:45:03,171][105620] Updated weights for policy 1, policy_version 1287248 (0.0005) [2023-12-27 00:45:03,231][105620] Updated weights for policy 1, policy_version 1287258 (0.0006) [2023-12-27 00:45:03,479][105692] Updated weights for policy 0, policy_version 1285651 (0.0009) [2023-12-27 00:45:03,539][105692] Updated weights for policy 0, policy_version 1285661 (0.0009) [2023-12-27 00:45:03,595][105692] Updated weights for policy 0, policy_version 1285671 (0.0008) [2023-12-27 00:45:03,881][105620] Updated weights for policy 1, policy_version 1287268 (0.0006) [2023-12-27 00:45:03,936][105620] Updated weights for policy 1, policy_version 1287278 (0.0009) [2023-12-27 00:45:03,984][105620] Updated weights for policy 1, policy_version 1287288 (0.0009) [2023-12-27 00:45:04,316][105692] Updated weights for policy 0, policy_version 1285681 (0.0008) [2023-12-27 00:45:04,381][105692] Updated weights for policy 0, policy_version 1285691 (0.0008) [2023-12-27 00:45:04,449][105692] Updated weights for policy 0, policy_version 1285701 (0.0007) [2023-12-27 00:45:04,451][105585] KL-divergence is very high: 117.5046 [2023-12-27 00:45:04,507][105585] KL-divergence is very high: 223.5275 [2023-12-27 00:45:04,519][105692] Updated weights for policy 0, policy_version 1285711 (0.0005) [2023-12-27 00:45:04,808][105620] Updated weights for policy 1, policy_version 1287298 (0.0009) [2023-12-27 00:45:04,865][105620] Updated weights for policy 1, policy_version 1287308 (0.0009) [2023-12-27 00:45:04,922][105620] Updated weights for policy 1, policy_version 1287318 (0.0008) [2023-12-27 00:45:04,974][105620] Updated weights for policy 1, policy_version 1287328 (0.0005) [2023-12-27 00:45:05,148][105692] Updated weights for policy 0, policy_version 1285721 (0.0010) [2023-12-27 00:45:05,202][105692] Updated weights for policy 0, policy_version 1285731 (0.0009) [2023-12-27 00:45:05,253][105692] Updated weights for policy 0, policy_version 1285741 (0.0010) [2023-12-27 00:45:05,526][105620] Updated weights for policy 1, policy_version 1287338 (0.0008) [2023-12-27 00:45:05,577][105620] Updated weights for policy 1, policy_version 1287348 (0.0009) [2023-12-27 00:45:05,634][105620] Updated weights for policy 1, policy_version 1287358 (0.0005) [2023-12-27 00:45:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 658808832. Throughput: 0: 9804.2, 1: 9558.8. Samples: 658799428. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:45:06,063][104569] Avg episode reward: [(0, '8911.236'), (1, '9262.907')] [2023-12-27 00:45:06,132][105692] Updated weights for policy 0, policy_version 1285751 (0.0008) [2023-12-27 00:45:06,188][105692] Updated weights for policy 0, policy_version 1285761 (0.0009) [2023-12-27 00:45:06,253][105692] Updated weights for policy 0, policy_version 1285771 (0.0008) [2023-12-27 00:45:06,293][105620] Updated weights for policy 1, policy_version 1287368 (0.0007) [2023-12-27 00:45:06,355][105620] Updated weights for policy 1, policy_version 1287378 (0.0009) [2023-12-27 00:45:06,419][105620] Updated weights for policy 1, policy_version 1287388 (0.0009) [2023-12-27 00:45:07,009][105692] Updated weights for policy 0, policy_version 1285781 (0.0008) [2023-12-27 00:45:07,071][105692] Updated weights for policy 0, policy_version 1285791 (0.0009) [2023-12-27 00:45:07,129][105692] Updated weights for policy 0, policy_version 1285801 (0.0009) [2023-12-27 00:45:07,169][105620] Updated weights for policy 1, policy_version 1287398 (0.0008) [2023-12-27 00:45:07,224][105620] Updated weights for policy 1, policy_version 1287408 (0.0009) [2023-12-27 00:45:07,282][105620] Updated weights for policy 1, policy_version 1287418 (0.0009) [2023-12-27 00:45:07,937][105692] Updated weights for policy 0, policy_version 1285811 (0.0006) [2023-12-27 00:45:07,948][105620] Updated weights for policy 1, policy_version 1287428 (0.0010) [2023-12-27 00:45:07,996][105620] Updated weights for policy 1, policy_version 1287438 (0.0010) [2023-12-27 00:45:08,001][105692] Updated weights for policy 0, policy_version 1285821 (0.0005) [2023-12-27 00:45:08,049][105620] Updated weights for policy 1, policy_version 1287448 (0.0010) [2023-12-27 00:45:08,059][105692] Updated weights for policy 0, policy_version 1285831 (0.0005) [2023-12-27 00:45:08,589][105692] Updated weights for policy 0, policy_version 1285841 (0.0006) [2023-12-27 00:45:08,652][105692] Updated weights for policy 0, policy_version 1285851 (0.0008) [2023-12-27 00:45:08,718][105692] Updated weights for policy 0, policy_version 1285861 (0.0009) [2023-12-27 00:45:08,782][105692] Updated weights for policy 0, policy_version 1285871 (0.0010) [2023-12-27 00:45:08,798][105620] Updated weights for policy 1, policy_version 1287458 (0.0010) [2023-12-27 00:45:08,858][105620] Updated weights for policy 1, policy_version 1287468 (0.0010) [2023-12-27 00:45:08,903][105620] Updated weights for policy 1, policy_version 1287478 (0.0010) [2023-12-27 00:45:08,951][105620] Updated weights for policy 1, policy_version 1287488 (0.0010) [2023-12-27 00:45:09,434][105692] Updated weights for policy 0, policy_version 1285881 (0.0009) [2023-12-27 00:45:09,500][105692] Updated weights for policy 0, policy_version 1285891 (0.0009) [2023-12-27 00:45:09,565][105692] Updated weights for policy 0, policy_version 1285901 (0.0008) [2023-12-27 00:45:09,767][105620] Updated weights for policy 1, policy_version 1287498 (0.0010) [2023-12-27 00:45:09,838][105620] Updated weights for policy 1, policy_version 1287508 (0.0011) [2023-12-27 00:45:09,903][105620] Updated weights for policy 1, policy_version 1287518 (0.0010) [2023-12-27 00:45:10,345][105692] Updated weights for policy 0, policy_version 1285911 (0.0008) [2023-12-27 00:45:10,412][105692] Updated weights for policy 0, policy_version 1285921 (0.0008) [2023-12-27 00:45:10,461][105692] Updated weights for policy 0, policy_version 1285931 (0.0008) [2023-12-27 00:45:10,656][105620] Updated weights for policy 1, policy_version 1287528 (0.0009) [2023-12-27 00:45:10,711][105620] Updated weights for policy 1, policy_version 1287538 (0.0010) [2023-12-27 00:45:10,767][105620] Updated weights for policy 1, policy_version 1287548 (0.0006) [2023-12-27 00:45:11,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 658907136. Throughput: 0: 9734.2, 1: 9631.6. Samples: 658915720. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:45:11,062][104569] Avg episode reward: [(0, '8909.258'), (1, '9263.446')] [2023-12-27 00:45:11,276][105692] Updated weights for policy 0, policy_version 1285941 (0.0008) [2023-12-27 00:45:11,335][105692] Updated weights for policy 0, policy_version 1285951 (0.0008) [2023-12-27 00:45:11,408][105692] Updated weights for policy 0, policy_version 1285961 (0.0009) [2023-12-27 00:45:11,454][105620] Updated weights for policy 1, policy_version 1287558 (0.0009) [2023-12-27 00:45:11,503][105620] Updated weights for policy 1, policy_version 1287568 (0.0011) [2023-12-27 00:45:11,556][105620] Updated weights for policy 1, policy_version 1287578 (0.0010) [2023-12-27 00:45:12,155][105692] Updated weights for policy 0, policy_version 1285971 (0.0006) [2023-12-27 00:45:12,212][105692] Updated weights for policy 0, policy_version 1285981 (0.0005) [2023-12-27 00:45:12,280][105692] Updated weights for policy 0, policy_version 1285991 (0.0007) [2023-12-27 00:45:12,329][105620] Updated weights for policy 1, policy_version 1287588 (0.0010) [2023-12-27 00:45:12,399][105620] Updated weights for policy 1, policy_version 1287598 (0.0009) [2023-12-27 00:45:12,455][105620] Updated weights for policy 1, policy_version 1287608 (0.0005) [2023-12-27 00:45:13,053][105620] Updated weights for policy 1, policy_version 1287618 (0.0007) [2023-12-27 00:45:13,068][105692] Updated weights for policy 0, policy_version 1286001 (0.0006) [2023-12-27 00:45:13,105][105620] Updated weights for policy 1, policy_version 1287628 (0.0010) [2023-12-27 00:45:13,119][105692] Updated weights for policy 0, policy_version 1286011 (0.0006) [2023-12-27 00:45:13,153][105620] Updated weights for policy 1, policy_version 1287638 (0.0010) [2023-12-27 00:45:13,168][105692] Updated weights for policy 0, policy_version 1286021 (0.0005) [2023-12-27 00:45:13,206][105620] Updated weights for policy 1, policy_version 1287648 (0.0010) [2023-12-27 00:45:13,217][105692] Updated weights for policy 0, policy_version 1286031 (0.0006) [2023-12-27 00:45:13,958][105620] Updated weights for policy 1, policy_version 1287658 (0.0009) [2023-12-27 00:45:14,007][105692] Updated weights for policy 0, policy_version 1286041 (0.0005) [2023-12-27 00:45:14,016][105620] Updated weights for policy 1, policy_version 1287668 (0.0011) [2023-12-27 00:45:14,058][105692] Updated weights for policy 0, policy_version 1286051 (0.0006) [2023-12-27 00:45:14,068][105620] Updated weights for policy 1, policy_version 1287678 (0.0010) [2023-12-27 00:45:14,104][105692] Updated weights for policy 0, policy_version 1286061 (0.0006) [2023-12-27 00:45:14,627][105620] Updated weights for policy 1, policy_version 1287688 (0.0008) [2023-12-27 00:45:14,686][105620] Updated weights for policy 1, policy_version 1287698 (0.0006) [2023-12-27 00:45:14,750][105620] Updated weights for policy 1, policy_version 1287708 (0.0006) [2023-12-27 00:45:14,989][105692] Updated weights for policy 0, policy_version 1286071 (0.0008) [2023-12-27 00:45:15,059][105692] Updated weights for policy 0, policy_version 1286081 (0.0008) [2023-12-27 00:45:15,126][105692] Updated weights for policy 0, policy_version 1286091 (0.0008) [2023-12-27 00:45:15,467][105620] Updated weights for policy 1, policy_version 1287718 (0.0011) [2023-12-27 00:45:15,519][105620] Updated weights for policy 1, policy_version 1287728 (0.0010) [2023-12-27 00:45:15,577][105620] Updated weights for policy 1, policy_version 1287738 (0.0010) [2023-12-27 00:45:15,886][105692] Updated weights for policy 0, policy_version 1286101 (0.0009) [2023-12-27 00:45:15,941][105692] Updated weights for policy 0, policy_version 1286111 (0.0010) [2023-12-27 00:45:15,986][105692] Updated weights for policy 0, policy_version 1286121 (0.0010) [2023-12-27 00:45:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 659005440. Throughput: 0: 9673.1, 1: 9632.7. Samples: 658973388. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:45:16,063][104569] Avg episode reward: [(0, '9090.565'), (1, '9183.251')] [2023-12-27 00:45:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001286128_329302016.pth... [2023-12-27 00:45:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001287744_329703424.pth... [2023-12-27 00:45:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001286624_329416704.pth [2023-12-27 00:45:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001285008_329015296.pth [2023-12-27 00:45:16,231][105620] Updated weights for policy 1, policy_version 1287748 (0.0011) [2023-12-27 00:45:16,289][105620] Updated weights for policy 1, policy_version 1287758 (0.0006) [2023-12-27 00:45:16,343][105620] Updated weights for policy 1, policy_version 1287768 (0.0005) [2023-12-27 00:45:16,634][105692] Updated weights for policy 0, policy_version 1286131 (0.0009) [2023-12-27 00:45:16,698][105692] Updated weights for policy 0, policy_version 1286141 (0.0008) [2023-12-27 00:45:16,758][105692] Updated weights for policy 0, policy_version 1286151 (0.0008) [2023-12-27 00:45:17,064][105620] Updated weights for policy 1, policy_version 1287778 (0.0006) [2023-12-27 00:45:17,116][105620] Updated weights for policy 1, policy_version 1287788 (0.0010) [2023-12-27 00:45:17,176][105620] Updated weights for policy 1, policy_version 1287798 (0.0011) [2023-12-27 00:45:17,235][105620] Updated weights for policy 1, policy_version 1287808 (0.0010) [2023-12-27 00:45:17,528][105692] Updated weights for policy 0, policy_version 1286161 (0.0009) [2023-12-27 00:45:17,589][105692] Updated weights for policy 0, policy_version 1286171 (0.0009) [2023-12-27 00:45:17,651][105692] Updated weights for policy 0, policy_version 1286181 (0.0008) [2023-12-27 00:45:17,715][105692] Updated weights for policy 0, policy_version 1286191 (0.0005) [2023-12-27 00:45:17,969][105620] Updated weights for policy 1, policy_version 1287818 (0.0010) [2023-12-27 00:45:18,024][105620] Updated weights for policy 1, policy_version 1287828 (0.0010) [2023-12-27 00:45:18,082][105620] Updated weights for policy 1, policy_version 1287838 (0.0011) [2023-12-27 00:45:18,382][105692] Updated weights for policy 0, policy_version 1286201 (0.0010) [2023-12-27 00:45:18,434][105692] Updated weights for policy 0, policy_version 1286211 (0.0011) [2023-12-27 00:45:18,493][105692] Updated weights for policy 0, policy_version 1286221 (0.0011) [2023-12-27 00:45:18,792][105620] Updated weights for policy 1, policy_version 1287848 (0.0008) [2023-12-27 00:45:18,851][105620] Updated weights for policy 1, policy_version 1287858 (0.0010) [2023-12-27 00:45:18,910][105620] Updated weights for policy 1, policy_version 1287868 (0.0011) [2023-12-27 00:45:19,177][105692] Updated weights for policy 0, policy_version 1286231 (0.0009) [2023-12-27 00:45:19,244][105692] Updated weights for policy 0, policy_version 1286241 (0.0008) [2023-12-27 00:45:19,309][105692] Updated weights for policy 0, policy_version 1286251 (0.0006) [2023-12-27 00:45:19,674][105620] Updated weights for policy 1, policy_version 1287878 (0.0009) [2023-12-27 00:45:19,725][105620] Updated weights for policy 1, policy_version 1287888 (0.0009) [2023-12-27 00:45:19,783][105620] Updated weights for policy 1, policy_version 1287898 (0.0010) [2023-12-27 00:45:19,958][105692] Updated weights for policy 0, policy_version 1286261 (0.0008) [2023-12-27 00:45:20,022][105692] Updated weights for policy 0, policy_version 1286271 (0.0008) [2023-12-27 00:45:20,081][105692] Updated weights for policy 0, policy_version 1286281 (0.0008) [2023-12-27 00:45:20,585][105620] Updated weights for policy 1, policy_version 1287908 (0.0009) [2023-12-27 00:45:20,648][105620] Updated weights for policy 1, policy_version 1287918 (0.0009) [2023-12-27 00:45:20,700][105620] Updated weights for policy 1, policy_version 1287928 (0.0009) [2023-12-27 00:45:20,795][105692] Updated weights for policy 0, policy_version 1286291 (0.0007) [2023-12-27 00:45:20,858][105692] Updated weights for policy 0, policy_version 1286301 (0.0007) [2023-12-27 00:45:20,919][105692] Updated weights for policy 0, policy_version 1286311 (0.0006) [2023-12-27 00:45:21,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 659103744. Throughput: 0: 9569.2, 1: 9654.2. Samples: 659089592. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:45:21,063][104569] Avg episode reward: [(0, '9086.600'), (1, '9183.441')] [2023-12-27 00:45:21,540][105692] Updated weights for policy 0, policy_version 1286321 (0.0006) [2023-12-27 00:45:21,602][105620] Updated weights for policy 1, policy_version 1287938 (0.0010) [2023-12-27 00:45:21,608][105692] Updated weights for policy 0, policy_version 1286331 (0.0006) [2023-12-27 00:45:21,670][105620] Updated weights for policy 1, policy_version 1287948 (0.0007) [2023-12-27 00:45:21,671][105692] Updated weights for policy 0, policy_version 1286341 (0.0007) [2023-12-27 00:45:21,725][105620] Updated weights for policy 1, policy_version 1287958 (0.0008) [2023-12-27 00:45:21,727][105692] Updated weights for policy 0, policy_version 1286351 (0.0006) [2023-12-27 00:45:21,792][105620] Updated weights for policy 1, policy_version 1287968 (0.0006) [2023-12-27 00:45:22,351][105692] Updated weights for policy 0, policy_version 1286361 (0.0007) [2023-12-27 00:45:22,414][105692] Updated weights for policy 0, policy_version 1286371 (0.0009) [2023-12-27 00:45:22,474][105692] Updated weights for policy 0, policy_version 1286381 (0.0006) [2023-12-27 00:45:22,635][105620] Updated weights for policy 1, policy_version 1287978 (0.0009) [2023-12-27 00:45:22,686][105620] Updated weights for policy 1, policy_version 1287988 (0.0008) [2023-12-27 00:45:22,735][105620] Updated weights for policy 1, policy_version 1287998 (0.0008) [2023-12-27 00:45:23,134][105692] Updated weights for policy 0, policy_version 1286391 (0.0005) [2023-12-27 00:45:23,187][105692] Updated weights for policy 0, policy_version 1286401 (0.0006) [2023-12-27 00:45:23,248][105692] Updated weights for policy 0, policy_version 1286411 (0.0006) [2023-12-27 00:45:23,460][105620] Updated weights for policy 1, policy_version 1288008 (0.0008) [2023-12-27 00:45:23,523][105620] Updated weights for policy 1, policy_version 1288018 (0.0006) [2023-12-27 00:45:23,574][105620] Updated weights for policy 1, policy_version 1288028 (0.0005) [2023-12-27 00:45:23,908][105692] Updated weights for policy 0, policy_version 1286421 (0.0006) [2023-12-27 00:45:23,961][105692] Updated weights for policy 0, policy_version 1286431 (0.0005) [2023-12-27 00:45:24,009][105692] Updated weights for policy 0, policy_version 1286441 (0.0005) [2023-12-27 00:45:24,243][105620] Updated weights for policy 1, policy_version 1288038 (0.0009) [2023-12-27 00:45:24,297][105620] Updated weights for policy 1, policy_version 1288048 (0.0007) [2023-12-27 00:45:24,347][105620] Updated weights for policy 1, policy_version 1288058 (0.0005) [2023-12-27 00:45:24,711][105692] Updated weights for policy 0, policy_version 1286451 (0.0008) [2023-12-27 00:45:24,770][105692] Updated weights for policy 0, policy_version 1286461 (0.0009) [2023-12-27 00:45:24,829][105692] Updated weights for policy 0, policy_version 1286471 (0.0009) [2023-12-27 00:45:25,046][105620] Updated weights for policy 1, policy_version 1288068 (0.0007) [2023-12-27 00:45:25,096][105620] Updated weights for policy 1, policy_version 1288078 (0.0009) [2023-12-27 00:45:25,142][105620] Updated weights for policy 1, policy_version 1288088 (0.0008) [2023-12-27 00:45:25,591][105692] Updated weights for policy 0, policy_version 1286481 (0.0009) [2023-12-27 00:45:25,649][105692] Updated weights for policy 0, policy_version 1286491 (0.0009) [2023-12-27 00:45:25,696][105692] Updated weights for policy 0, policy_version 1286501 (0.0009) [2023-12-27 00:45:25,751][105692] Updated weights for policy 0, policy_version 1286511 (0.0009) [2023-12-27 00:45:25,904][105620] Updated weights for policy 1, policy_version 1288098 (0.0009) [2023-12-27 00:45:25,959][105620] Updated weights for policy 1, policy_version 1288108 (0.0008) [2023-12-27 00:45:26,016][105620] Updated weights for policy 1, policy_version 1288118 (0.0009) [2023-12-27 00:45:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 659193856. Throughput: 0: 9615.8, 1: 9558.8. Samples: 659205628. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:45:26,062][104569] Avg episode reward: [(0, '8995.491'), (1, '9263.587')] [2023-12-27 00:45:26,081][105620] Updated weights for policy 1, policy_version 1288128 (0.0009) [2023-12-27 00:45:26,446][105692] Updated weights for policy 0, policy_version 1286521 (0.0008) [2023-12-27 00:45:26,495][105692] Updated weights for policy 0, policy_version 1286531 (0.0005) [2023-12-27 00:45:26,564][105692] Updated weights for policy 0, policy_version 1286541 (0.0005) [2023-12-27 00:45:26,824][105620] Updated weights for policy 1, policy_version 1288138 (0.0006) [2023-12-27 00:45:26,872][105620] Updated weights for policy 1, policy_version 1288148 (0.0010) [2023-12-27 00:45:26,917][105620] Updated weights for policy 1, policy_version 1288158 (0.0009) [2023-12-27 00:45:27,213][105692] Updated weights for policy 0, policy_version 1286551 (0.0007) [2023-12-27 00:45:27,261][105692] Updated weights for policy 0, policy_version 1286561 (0.0008) [2023-12-27 00:45:27,316][105692] Updated weights for policy 0, policy_version 1286571 (0.0008) [2023-12-27 00:45:27,655][105620] Updated weights for policy 1, policy_version 1288168 (0.0009) [2023-12-27 00:45:27,716][105620] Updated weights for policy 1, policy_version 1288178 (0.0010) [2023-12-27 00:45:27,770][105620] Updated weights for policy 1, policy_version 1288188 (0.0010) [2023-12-27 00:45:28,081][105692] Updated weights for policy 0, policy_version 1286581 (0.0008) [2023-12-27 00:45:28,133][105692] Updated weights for policy 0, policy_version 1286591 (0.0008) [2023-12-27 00:45:28,177][105692] Updated weights for policy 0, policy_version 1286601 (0.0008) [2023-12-27 00:45:28,532][105620] Updated weights for policy 1, policy_version 1288198 (0.0009) [2023-12-27 00:45:28,596][105620] Updated weights for policy 1, policy_version 1288208 (0.0009) [2023-12-27 00:45:28,661][105620] Updated weights for policy 1, policy_version 1288218 (0.0010) [2023-12-27 00:45:28,854][105692] Updated weights for policy 0, policy_version 1286611 (0.0009) [2023-12-27 00:45:28,906][105692] Updated weights for policy 0, policy_version 1286621 (0.0011) [2023-12-27 00:45:28,954][105692] Updated weights for policy 0, policy_version 1286631 (0.0010) [2023-12-27 00:45:29,386][105620] Updated weights for policy 1, policy_version 1288228 (0.0008) [2023-12-27 00:45:29,448][105620] Updated weights for policy 1, policy_version 1288238 (0.0005) [2023-12-27 00:45:29,501][105620] Updated weights for policy 1, policy_version 1288248 (0.0009) [2023-12-27 00:45:29,640][105692] Updated weights for policy 0, policy_version 1286641 (0.0010) [2023-12-27 00:45:29,699][105692] Updated weights for policy 0, policy_version 1286651 (0.0011) [2023-12-27 00:45:29,747][105692] Updated weights for policy 0, policy_version 1286661 (0.0011) [2023-12-27 00:45:29,810][105692] Updated weights for policy 0, policy_version 1286671 (0.0011) [2023-12-27 00:45:30,203][105620] Updated weights for policy 1, policy_version 1288258 (0.0009) [2023-12-27 00:45:30,259][105620] Updated weights for policy 1, policy_version 1288268 (0.0005) [2023-12-27 00:45:30,317][105620] Updated weights for policy 1, policy_version 1288278 (0.0008) [2023-12-27 00:45:30,371][105620] Updated weights for policy 1, policy_version 1288288 (0.0010) [2023-12-27 00:45:30,551][105692] Updated weights for policy 0, policy_version 1286681 (0.0010) [2023-12-27 00:45:30,617][105692] Updated weights for policy 0, policy_version 1286691 (0.0009) [2023-12-27 00:45:30,674][105692] Updated weights for policy 0, policy_version 1286701 (0.0009) [2023-12-27 00:45:30,969][105620] Updated weights for policy 1, policy_version 1288298 (0.0010) [2023-12-27 00:45:31,020][105620] Updated weights for policy 1, policy_version 1288308 (0.0010) [2023-12-27 00:45:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 659292160. Throughput: 0: 9637.8, 1: 9540.5. Samples: 659263680. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:45:31,063][104569] Avg episode reward: [(0, '8905.646'), (1, '9354.376')] [2023-12-27 00:45:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001286704_329449472.pth... [2023-12-27 00:45:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001285584_329162752.pth [2023-12-27 00:45:31,081][105620] Updated weights for policy 1, policy_version 1288318 (0.0011) [2023-12-27 00:45:31,089][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001288320_329850880.pth... [2023-12-27 00:45:31,109][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001287168_329555968.pth [2023-12-27 00:45:31,381][105692] Updated weights for policy 0, policy_version 1286711 (0.0008) [2023-12-27 00:45:31,441][105692] Updated weights for policy 0, policy_version 1286721 (0.0007) [2023-12-27 00:45:31,497][105692] Updated weights for policy 0, policy_version 1286731 (0.0008) [2023-12-27 00:45:31,846][105620] Updated weights for policy 1, policy_version 1288328 (0.0010) [2023-12-27 00:45:31,895][105620] Updated weights for policy 1, policy_version 1288338 (0.0010) [2023-12-27 00:45:31,943][105620] Updated weights for policy 1, policy_version 1288348 (0.0010) [2023-12-27 00:45:32,154][105692] Updated weights for policy 0, policy_version 1286741 (0.0007) [2023-12-27 00:45:32,215][105692] Updated weights for policy 0, policy_version 1286751 (0.0006) [2023-12-27 00:45:32,272][105692] Updated weights for policy 0, policy_version 1286761 (0.0006) [2023-12-27 00:45:32,735][105620] Updated weights for policy 1, policy_version 1288358 (0.0009) [2023-12-27 00:45:32,798][105620] Updated weights for policy 1, policy_version 1288368 (0.0006) [2023-12-27 00:45:32,807][105692] Updated weights for policy 0, policy_version 1286771 (0.0007) [2023-12-27 00:45:32,856][105620] Updated weights for policy 1, policy_version 1288378 (0.0007) [2023-12-27 00:45:32,871][105692] Updated weights for policy 0, policy_version 1286781 (0.0006) [2023-12-27 00:45:32,927][105692] Updated weights for policy 0, policy_version 1286791 (0.0008) [2023-12-27 00:45:33,541][105620] Updated weights for policy 1, policy_version 1288388 (0.0009) [2023-12-27 00:45:33,586][105620] Updated weights for policy 1, policy_version 1288398 (0.0010) [2023-12-27 00:45:33,633][105620] Updated weights for policy 1, policy_version 1288408 (0.0010) [2023-12-27 00:45:33,633][105692] Updated weights for policy 0, policy_version 1286801 (0.0008) [2023-12-27 00:45:33,692][105692] Updated weights for policy 0, policy_version 1286811 (0.0006) [2023-12-27 00:45:33,752][105692] Updated weights for policy 0, policy_version 1286821 (0.0007) [2023-12-27 00:45:33,819][105692] Updated weights for policy 0, policy_version 1286831 (0.0005) [2023-12-27 00:45:34,406][105692] Updated weights for policy 0, policy_version 1286841 (0.0007) [2023-12-27 00:45:34,411][105620] Updated weights for policy 1, policy_version 1288418 (0.0010) [2023-12-27 00:45:34,469][105692] Updated weights for policy 0, policy_version 1286851 (0.0008) [2023-12-27 00:45:34,481][105620] Updated weights for policy 1, policy_version 1288428 (0.0011) [2023-12-27 00:45:34,524][105692] Updated weights for policy 0, policy_version 1286861 (0.0008) [2023-12-27 00:45:34,543][105620] Updated weights for policy 1, policy_version 1288438 (0.0007) [2023-12-27 00:45:34,600][105620] Updated weights for policy 1, policy_version 1288448 (0.0009) [2023-12-27 00:45:35,210][105692] Updated weights for policy 0, policy_version 1286871 (0.0007) [2023-12-27 00:45:35,272][105692] Updated weights for policy 0, policy_version 1286881 (0.0010) [2023-12-27 00:45:35,333][105692] Updated weights for policy 0, policy_version 1286891 (0.0010) [2023-12-27 00:45:35,366][105620] Updated weights for policy 1, policy_version 1288458 (0.0010) [2023-12-27 00:45:35,413][105620] Updated weights for policy 1, policy_version 1288468 (0.0007) [2023-12-27 00:45:35,473][105620] Updated weights for policy 1, policy_version 1288478 (0.0007) [2023-12-27 00:45:35,993][105692] Updated weights for policy 0, policy_version 1286901 (0.0008) [2023-12-27 00:45:36,054][105692] Updated weights for policy 0, policy_version 1286911 (0.0005) [2023-12-27 00:45:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.3, 300 sec: 19466.4). Total num frames: 659390464. Throughput: 0: 9767.0, 1: 9563.1. Samples: 659383528. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:45:36,062][104569] Avg episode reward: [(0, '8907.313'), (1, '9354.338')] [2023-12-27 00:45:36,107][105692] Updated weights for policy 0, policy_version 1286921 (0.0006) [2023-12-27 00:45:36,146][105620] Updated weights for policy 1, policy_version 1288488 (0.0008) [2023-12-27 00:45:36,207][105620] Updated weights for policy 1, policy_version 1288498 (0.0008) [2023-12-27 00:45:36,273][105620] Updated weights for policy 1, policy_version 1288508 (0.0008) [2023-12-27 00:45:36,829][105692] Updated weights for policy 0, policy_version 1286931 (0.0008) [2023-12-27 00:45:36,882][105692] Updated weights for policy 0, policy_version 1286941 (0.0008) [2023-12-27 00:45:36,948][105692] Updated weights for policy 0, policy_version 1286951 (0.0007) [2023-12-27 00:45:36,962][105620] Updated weights for policy 1, policy_version 1288518 (0.0009) [2023-12-27 00:45:37,021][105620] Updated weights for policy 1, policy_version 1288528 (0.0011) [2023-12-27 00:45:37,084][105620] Updated weights for policy 1, policy_version 1288538 (0.0010) [2023-12-27 00:45:37,736][105692] Updated weights for policy 0, policy_version 1286961 (0.0006) [2023-12-27 00:45:37,789][105692] Updated weights for policy 0, policy_version 1286971 (0.0006) [2023-12-27 00:45:37,833][105620] Updated weights for policy 1, policy_version 1288548 (0.0009) [2023-12-27 00:45:37,844][105692] Updated weights for policy 0, policy_version 1286981 (0.0006) [2023-12-27 00:45:37,892][105620] Updated weights for policy 1, policy_version 1288558 (0.0011) [2023-12-27 00:45:37,895][105692] Updated weights for policy 0, policy_version 1286991 (0.0006) [2023-12-27 00:45:37,950][105620] Updated weights for policy 1, policy_version 1288568 (0.0009) [2023-12-27 00:45:38,529][105692] Updated weights for policy 0, policy_version 1287001 (0.0005) [2023-12-27 00:45:38,596][105692] Updated weights for policy 0, policy_version 1287011 (0.0005) [2023-12-27 00:45:38,664][105692] Updated weights for policy 0, policy_version 1287021 (0.0005) [2023-12-27 00:45:38,683][105620] Updated weights for policy 1, policy_version 1288578 (0.0010) [2023-12-27 00:45:38,749][105620] Updated weights for policy 1, policy_version 1288588 (0.0010) [2023-12-27 00:45:38,806][105620] Updated weights for policy 1, policy_version 1288598 (0.0009) [2023-12-27 00:45:38,858][105620] Updated weights for policy 1, policy_version 1288608 (0.0010) [2023-12-27 00:45:39,339][105692] Updated weights for policy 0, policy_version 1287031 (0.0008) [2023-12-27 00:45:39,408][105692] Updated weights for policy 0, policy_version 1287041 (0.0008) [2023-12-27 00:45:39,470][105692] Updated weights for policy 0, policy_version 1287051 (0.0009) [2023-12-27 00:45:39,556][105620] Updated weights for policy 1, policy_version 1288618 (0.0007) [2023-12-27 00:45:39,609][105620] Updated weights for policy 1, policy_version 1288628 (0.0005) [2023-12-27 00:45:39,666][105620] Updated weights for policy 1, policy_version 1288638 (0.0006) [2023-12-27 00:45:40,247][105692] Updated weights for policy 0, policy_version 1287061 (0.0009) [2023-12-27 00:45:40,306][105692] Updated weights for policy 0, policy_version 1287071 (0.0009) [2023-12-27 00:45:40,360][105692] Updated weights for policy 0, policy_version 1287081 (0.0009) [2023-12-27 00:45:40,403][105620] Updated weights for policy 1, policy_version 1288648 (0.0008) [2023-12-27 00:45:40,461][105620] Updated weights for policy 1, policy_version 1288658 (0.0009) [2023-12-27 00:45:40,524][105620] Updated weights for policy 1, policy_version 1288668 (0.0009) [2023-12-27 00:45:41,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 659488768. Throughput: 0: 9773.1, 1: 9645.4. Samples: 659499732. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:45:41,062][104569] Avg episode reward: [(0, '8650.005'), (1, '9354.126')] [2023-12-27 00:45:41,194][105692] Updated weights for policy 0, policy_version 1287091 (0.0009) [2023-12-27 00:45:41,265][105692] Updated weights for policy 0, policy_version 1287101 (0.0009) [2023-12-27 00:45:41,306][105620] Updated weights for policy 1, policy_version 1288678 (0.0008) [2023-12-27 00:45:41,321][105692] Updated weights for policy 0, policy_version 1287111 (0.0007) [2023-12-27 00:45:41,376][105620] Updated weights for policy 1, policy_version 1288688 (0.0009) [2023-12-27 00:45:41,431][105620] Updated weights for policy 1, policy_version 1288698 (0.0008) [2023-12-27 00:45:42,108][105692] Updated weights for policy 0, policy_version 1287121 (0.0007) [2023-12-27 00:45:42,171][105692] Updated weights for policy 0, policy_version 1287131 (0.0007) [2023-12-27 00:45:42,231][105692] Updated weights for policy 0, policy_version 1287141 (0.0008) [2023-12-27 00:45:42,280][105620] Updated weights for policy 1, policy_version 1288708 (0.0008) [2023-12-27 00:45:42,294][105692] Updated weights for policy 0, policy_version 1287151 (0.0008) [2023-12-27 00:45:42,339][105620] Updated weights for policy 1, policy_version 1288718 (0.0005) [2023-12-27 00:45:42,407][105620] Updated weights for policy 1, policy_version 1288728 (0.0007) [2023-12-27 00:45:43,031][105692] Updated weights for policy 0, policy_version 1287161 (0.0009) [2023-12-27 00:45:43,088][105692] Updated weights for policy 0, policy_version 1287171 (0.0008) [2023-12-27 00:45:43,111][105620] Updated weights for policy 1, policy_version 1288738 (0.0006) [2023-12-27 00:45:43,142][105692] Updated weights for policy 0, policy_version 1287181 (0.0007) [2023-12-27 00:45:43,171][105620] Updated weights for policy 1, policy_version 1288748 (0.0009) [2023-12-27 00:45:43,238][105620] Updated weights for policy 1, policy_version 1288758 (0.0009) [2023-12-27 00:45:43,299][105620] Updated weights for policy 1, policy_version 1288768 (0.0009) [2023-12-27 00:45:43,809][105692] Updated weights for policy 0, policy_version 1287191 (0.0008) [2023-12-27 00:45:43,859][105692] Updated weights for policy 0, policy_version 1287201 (0.0008) [2023-12-27 00:45:43,909][105692] Updated weights for policy 0, policy_version 1287211 (0.0009) [2023-12-27 00:45:44,024][105620] Updated weights for policy 1, policy_version 1288778 (0.0009) [2023-12-27 00:45:44,075][105620] Updated weights for policy 1, policy_version 1288788 (0.0009) [2023-12-27 00:45:44,129][105620] Updated weights for policy 1, policy_version 1288798 (0.0009) [2023-12-27 00:45:44,584][105692] Updated weights for policy 0, policy_version 1287221 (0.0008) [2023-12-27 00:45:44,642][105692] Updated weights for policy 0, policy_version 1287231 (0.0007) [2023-12-27 00:45:44,706][105692] Updated weights for policy 0, policy_version 1287241 (0.0005) [2023-12-27 00:45:45,024][105620] Updated weights for policy 1, policy_version 1288808 (0.0009) [2023-12-27 00:45:45,079][105620] Updated weights for policy 1, policy_version 1288818 (0.0009) [2023-12-27 00:45:45,134][105620] Updated weights for policy 1, policy_version 1288828 (0.0009) [2023-12-27 00:45:45,375][105692] Updated weights for policy 0, policy_version 1287251 (0.0007) [2023-12-27 00:45:45,437][105692] Updated weights for policy 0, policy_version 1287261 (0.0009) [2023-12-27 00:45:45,498][105692] Updated weights for policy 0, policy_version 1287271 (0.0009) [2023-12-27 00:45:45,908][105620] Updated weights for policy 1, policy_version 1288838 (0.0009) [2023-12-27 00:45:45,957][105620] Updated weights for policy 1, policy_version 1288848 (0.0009) [2023-12-27 00:45:46,016][105620] Updated weights for policy 1, policy_version 1288858 (0.0008) [2023-12-27 00:45:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19251.1, 300 sec: 19466.4). Total num frames: 659587072. Throughput: 0: 9740.4, 1: 9662.1. Samples: 659555760. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:45:46,063][104569] Avg episode reward: [(0, '8741.398'), (1, '9353.965')] [2023-12-27 00:45:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001287280_329596928.pth... [2023-12-27 00:45:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001288864_329990144.pth... [2023-12-27 00:45:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001287744_329703424.pth [2023-12-27 00:45:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001286128_329302016.pth [2023-12-27 00:45:46,208][105692] Updated weights for policy 0, policy_version 1287281 (0.0009) [2023-12-27 00:45:46,266][105692] Updated weights for policy 0, policy_version 1287291 (0.0009) [2023-12-27 00:45:46,326][105692] Updated weights for policy 0, policy_version 1287301 (0.0009) [2023-12-27 00:45:46,387][105692] Updated weights for policy 0, policy_version 1287311 (0.0009) [2023-12-27 00:45:46,839][105620] Updated weights for policy 1, policy_version 1288868 (0.0009) [2023-12-27 00:45:46,890][105620] Updated weights for policy 1, policy_version 1288878 (0.0008) [2023-12-27 00:45:46,946][105620] Updated weights for policy 1, policy_version 1288888 (0.0009) [2023-12-27 00:45:47,053][105692] Updated weights for policy 0, policy_version 1287321 (0.0009) [2023-12-27 00:45:47,116][105692] Updated weights for policy 0, policy_version 1287331 (0.0009) [2023-12-27 00:45:47,189][105692] Updated weights for policy 0, policy_version 1287341 (0.0010) [2023-12-27 00:45:47,683][105620] Updated weights for policy 1, policy_version 1288898 (0.0009) [2023-12-27 00:45:47,746][105620] Updated weights for policy 1, policy_version 1288908 (0.0010) [2023-12-27 00:45:47,805][105620] Updated weights for policy 1, policy_version 1288918 (0.0010) [2023-12-27 00:45:47,861][105620] Updated weights for policy 1, policy_version 1288928 (0.0007) [2023-12-27 00:45:47,908][105692] Updated weights for policy 0, policy_version 1287351 (0.0008) [2023-12-27 00:45:47,974][105692] Updated weights for policy 0, policy_version 1287361 (0.0008) [2023-12-27 00:45:48,037][105692] Updated weights for policy 0, policy_version 1287371 (0.0007) [2023-12-27 00:45:48,526][105620] Updated weights for policy 1, policy_version 1288938 (0.0009) [2023-12-27 00:45:48,574][105620] Updated weights for policy 1, policy_version 1288948 (0.0007) [2023-12-27 00:45:48,628][105620] Updated weights for policy 1, policy_version 1288958 (0.0009) [2023-12-27 00:45:48,712][105692] Updated weights for policy 0, policy_version 1287381 (0.0007) [2023-12-27 00:45:48,769][105692] Updated weights for policy 0, policy_version 1287391 (0.0008) [2023-12-27 00:45:48,834][105692] Updated weights for policy 0, policy_version 1287401 (0.0009) [2023-12-27 00:45:49,316][105620] Updated weights for policy 1, policy_version 1288968 (0.0009) [2023-12-27 00:45:49,375][105620] Updated weights for policy 1, policy_version 1288978 (0.0009) [2023-12-27 00:45:49,436][105620] Updated weights for policy 1, policy_version 1288988 (0.0009) [2023-12-27 00:45:49,629][105692] Updated weights for policy 0, policy_version 1287411 (0.0009) [2023-12-27 00:45:49,683][105692] Updated weights for policy 0, policy_version 1287421 (0.0009) [2023-12-27 00:45:49,729][105692] Updated weights for policy 0, policy_version 1287431 (0.0008) [2023-12-27 00:45:50,203][105620] Updated weights for policy 1, policy_version 1288998 (0.0009) [2023-12-27 00:45:50,274][105620] Updated weights for policy 1, policy_version 1289008 (0.0009) [2023-12-27 00:45:50,340][105620] Updated weights for policy 1, policy_version 1289018 (0.0009) [2023-12-27 00:45:50,442][105692] Updated weights for policy 0, policy_version 1287441 (0.0009) [2023-12-27 00:45:50,495][105692] Updated weights for policy 0, policy_version 1287451 (0.0010) [2023-12-27 00:45:50,549][105692] Updated weights for policy 0, policy_version 1287461 (0.0010) [2023-12-27 00:45:50,615][105692] Updated weights for policy 0, policy_version 1287471 (0.0010) [2023-12-27 00:45:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19114.6, 300 sec: 19494.2). Total num frames: 659677184. Throughput: 0: 9761.9, 1: 9583.9. Samples: 659669992. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:45:51,063][104569] Avg episode reward: [(0, '9270.295'), (1, '9353.921')] [2023-12-27 00:45:51,179][105620] Updated weights for policy 1, policy_version 1289028 (0.0007) [2023-12-27 00:45:51,241][105620] Updated weights for policy 1, policy_version 1289038 (0.0008) [2023-12-27 00:45:51,301][105620] Updated weights for policy 1, policy_version 1289048 (0.0008) [2023-12-27 00:45:51,440][105692] Updated weights for policy 0, policy_version 1287481 (0.0006) [2023-12-27 00:45:51,493][105692] Updated weights for policy 0, policy_version 1287491 (0.0005) [2023-12-27 00:45:51,554][105692] Updated weights for policy 0, policy_version 1287501 (0.0005) [2023-12-27 00:45:52,085][105620] Updated weights for policy 1, policy_version 1289058 (0.0008) [2023-12-27 00:45:52,150][105620] Updated weights for policy 1, policy_version 1289068 (0.0009) [2023-12-27 00:45:52,209][105620] Updated weights for policy 1, policy_version 1289078 (0.0009) [2023-12-27 00:45:52,275][105620] Updated weights for policy 1, policy_version 1289088 (0.0010) [2023-12-27 00:45:52,299][105692] Updated weights for policy 0, policy_version 1287511 (0.0009) [2023-12-27 00:45:52,347][105692] Updated weights for policy 0, policy_version 1287521 (0.0009) [2023-12-27 00:45:52,409][105692] Updated weights for policy 0, policy_version 1287531 (0.0009) [2023-12-27 00:45:53,033][105692] Updated weights for policy 0, policy_version 1287541 (0.0008) [2023-12-27 00:45:53,047][105620] Updated weights for policy 1, policy_version 1289098 (0.0011) [2023-12-27 00:45:53,089][105692] Updated weights for policy 0, policy_version 1287551 (0.0006) [2023-12-27 00:45:53,099][105620] Updated weights for policy 1, policy_version 1289108 (0.0011) [2023-12-27 00:45:53,145][105692] Updated weights for policy 0, policy_version 1287561 (0.0005) [2023-12-27 00:45:53,147][105620] Updated weights for policy 1, policy_version 1289118 (0.0010) [2023-12-27 00:45:53,880][105692] Updated weights for policy 0, policy_version 1287571 (0.0007) [2023-12-27 00:45:53,908][105620] Updated weights for policy 1, policy_version 1289128 (0.0010) [2023-12-27 00:45:53,938][105692] Updated weights for policy 0, policy_version 1287581 (0.0006) [2023-12-27 00:45:53,960][105620] Updated weights for policy 1, policy_version 1289138 (0.0010) [2023-12-27 00:45:53,997][105692] Updated weights for policy 0, policy_version 1287591 (0.0005) [2023-12-27 00:45:54,012][105620] Updated weights for policy 1, policy_version 1289148 (0.0010) [2023-12-27 00:45:54,559][105692] Updated weights for policy 0, policy_version 1287601 (0.0005) [2023-12-27 00:45:54,620][105692] Updated weights for policy 0, policy_version 1287611 (0.0007) [2023-12-27 00:45:54,676][105692] Updated weights for policy 0, policy_version 1287621 (0.0008) [2023-12-27 00:45:54,724][105692] Updated weights for policy 0, policy_version 1287631 (0.0008) [2023-12-27 00:45:54,795][105620] Updated weights for policy 1, policy_version 1289158 (0.0010) [2023-12-27 00:45:54,843][105620] Updated weights for policy 1, policy_version 1289168 (0.0010) [2023-12-27 00:45:54,888][105620] Updated weights for policy 1, policy_version 1289178 (0.0010) [2023-12-27 00:45:55,456][105692] Updated weights for policy 0, policy_version 1287641 (0.0006) [2023-12-27 00:45:55,508][105620] Updated weights for policy 1, policy_version 1289188 (0.0008) [2023-12-27 00:45:55,512][105692] Updated weights for policy 0, policy_version 1287651 (0.0008) [2023-12-27 00:45:55,558][105620] Updated weights for policy 1, policy_version 1289198 (0.0005) [2023-12-27 00:45:55,565][105692] Updated weights for policy 0, policy_version 1287661 (0.0008) [2023-12-27 00:45:55,616][105620] Updated weights for policy 1, policy_version 1289208 (0.0008) [2023-12-27 00:45:56,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 659775488. Throughput: 0: 9815.9, 1: 9505.3. Samples: 659785176. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:45:56,062][104569] Avg episode reward: [(0, '9178.395'), (1, '9172.298')] [2023-12-27 00:45:56,307][105692] Updated weights for policy 0, policy_version 1287671 (0.0009) [2023-12-27 00:45:56,336][105620] Updated weights for policy 1, policy_version 1289218 (0.0010) [2023-12-27 00:45:56,370][105692] Updated weights for policy 0, policy_version 1287681 (0.0010) [2023-12-27 00:45:56,384][105620] Updated weights for policy 1, policy_version 1289228 (0.0006) [2023-12-27 00:45:56,433][105692] Updated weights for policy 0, policy_version 1287691 (0.0009) [2023-12-27 00:45:56,440][105620] Updated weights for policy 1, policy_version 1289238 (0.0007) [2023-12-27 00:45:56,491][105620] Updated weights for policy 1, policy_version 1289248 (0.0007) [2023-12-27 00:45:57,070][105692] Updated weights for policy 0, policy_version 1287701 (0.0010) [2023-12-27 00:45:57,129][105692] Updated weights for policy 0, policy_version 1287711 (0.0011) [2023-12-27 00:45:57,190][105692] Updated weights for policy 0, policy_version 1287721 (0.0007) [2023-12-27 00:45:57,276][105620] Updated weights for policy 1, policy_version 1289258 (0.0006) [2023-12-27 00:45:57,333][105620] Updated weights for policy 1, policy_version 1289268 (0.0008) [2023-12-27 00:45:57,386][105620] Updated weights for policy 1, policy_version 1289278 (0.0010) [2023-12-27 00:45:57,745][105692] Updated weights for policy 0, policy_version 1287731 (0.0005) [2023-12-27 00:45:57,808][105692] Updated weights for policy 0, policy_version 1287741 (0.0009) [2023-12-27 00:45:57,874][105692] Updated weights for policy 0, policy_version 1287751 (0.0009) [2023-12-27 00:45:58,239][105620] Updated weights for policy 1, policy_version 1289288 (0.0008) [2023-12-27 00:45:58,293][105620] Updated weights for policy 1, policy_version 1289298 (0.0008) [2023-12-27 00:45:58,372][105620] Updated weights for policy 1, policy_version 1289308 (0.0009) [2023-12-27 00:45:58,614][105692] Updated weights for policy 0, policy_version 1287761 (0.0009) [2023-12-27 00:45:58,686][105692] Updated weights for policy 0, policy_version 1287771 (0.0007) [2023-12-27 00:45:58,749][105692] Updated weights for policy 0, policy_version 1287781 (0.0008) [2023-12-27 00:45:58,814][105692] Updated weights for policy 0, policy_version 1287791 (0.0008) [2023-12-27 00:45:59,249][105620] Updated weights for policy 1, policy_version 1289318 (0.0010) [2023-12-27 00:45:59,314][105620] Updated weights for policy 1, policy_version 1289328 (0.0008) [2023-12-27 00:45:59,384][105620] Updated weights for policy 1, policy_version 1289338 (0.0009) [2023-12-27 00:45:59,628][105692] Updated weights for policy 0, policy_version 1287801 (0.0007) [2023-12-27 00:45:59,692][105692] Updated weights for policy 0, policy_version 1287811 (0.0006) [2023-12-27 00:45:59,756][105692] Updated weights for policy 0, policy_version 1287821 (0.0006) [2023-12-27 00:46:00,148][105620] Updated weights for policy 1, policy_version 1289348 (0.0008) [2023-12-27 00:46:00,210][105620] Updated weights for policy 1, policy_version 1289358 (0.0009) [2023-12-27 00:46:00,268][105620] Updated weights for policy 1, policy_version 1289368 (0.0008) [2023-12-27 00:46:00,438][105692] Updated weights for policy 0, policy_version 1287831 (0.0008) [2023-12-27 00:46:00,503][105692] Updated weights for policy 0, policy_version 1287841 (0.0009) [2023-12-27 00:46:00,567][105692] Updated weights for policy 0, policy_version 1287851 (0.0009) [2023-12-27 00:46:00,956][105620] Updated weights for policy 1, policy_version 1289378 (0.0008) [2023-12-27 00:46:01,015][105620] Updated weights for policy 1, policy_version 1289388 (0.0008) [2023-12-27 00:46:01,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 659865600. Throughput: 0: 9896.6, 1: 9404.9. Samples: 659841952. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:46:01,062][104569] Avg episode reward: [(0, '9087.143'), (1, '9172.134')] [2023-12-27 00:46:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001287856_329744384.pth... [2023-12-27 00:46:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001286704_329449472.pth [2023-12-27 00:46:01,083][105620] Updated weights for policy 1, policy_version 1289398 (0.0008) [2023-12-27 00:46:01,145][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001289408_330129408.pth... [2023-12-27 00:46:01,147][105620] Updated weights for policy 1, policy_version 1289408 (0.0008) [2023-12-27 00:46:01,150][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001288320_329850880.pth [2023-12-27 00:46:01,352][105692] Updated weights for policy 0, policy_version 1287861 (0.0009) [2023-12-27 00:46:01,421][105692] Updated weights for policy 0, policy_version 1287871 (0.0009) [2023-12-27 00:46:01,476][105692] Updated weights for policy 0, policy_version 1287881 (0.0008) [2023-12-27 00:46:01,895][105620] Updated weights for policy 1, policy_version 1289418 (0.0009) [2023-12-27 00:46:01,957][105620] Updated weights for policy 1, policy_version 1289428 (0.0009) [2023-12-27 00:46:02,019][105620] Updated weights for policy 1, policy_version 1289438 (0.0009) [2023-12-27 00:46:02,222][105692] Updated weights for policy 0, policy_version 1287891 (0.0008) [2023-12-27 00:46:02,295][105692] Updated weights for policy 0, policy_version 1287901 (0.0009) [2023-12-27 00:46:02,357][105692] Updated weights for policy 0, policy_version 1287911 (0.0006) [2023-12-27 00:46:02,822][105620] Updated weights for policy 1, policy_version 1289448 (0.0010) [2023-12-27 00:46:02,890][105620] Updated weights for policy 1, policy_version 1289458 (0.0006) [2023-12-27 00:46:02,948][105692] Updated weights for policy 0, policy_version 1287921 (0.0008) [2023-12-27 00:46:02,952][105620] Updated weights for policy 1, policy_version 1289468 (0.0010) [2023-12-27 00:46:03,005][105692] Updated weights for policy 0, policy_version 1287931 (0.0008) [2023-12-27 00:46:03,062][105692] Updated weights for policy 0, policy_version 1287941 (0.0009) [2023-12-27 00:46:03,116][105692] Updated weights for policy 0, policy_version 1287951 (0.0008) [2023-12-27 00:46:03,659][105620] Updated weights for policy 1, policy_version 1289478 (0.0009) [2023-12-27 00:46:03,711][105620] Updated weights for policy 1, policy_version 1289488 (0.0011) [2023-12-27 00:46:03,760][105692] Updated weights for policy 0, policy_version 1287961 (0.0006) [2023-12-27 00:46:03,766][105620] Updated weights for policy 1, policy_version 1289498 (0.0010) [2023-12-27 00:46:03,821][105692] Updated weights for policy 0, policy_version 1287971 (0.0007) [2023-12-27 00:46:03,881][105692] Updated weights for policy 0, policy_version 1287981 (0.0008) [2023-12-27 00:46:04,547][105620] Updated weights for policy 1, policy_version 1289508 (0.0011) [2023-12-27 00:46:04,583][105692] Updated weights for policy 0, policy_version 1287991 (0.0006) [2023-12-27 00:46:04,611][105620] Updated weights for policy 1, policy_version 1289518 (0.0011) [2023-12-27 00:46:04,646][105692] Updated weights for policy 0, policy_version 1288001 (0.0007) [2023-12-27 00:46:04,671][105620] Updated weights for policy 1, policy_version 1289528 (0.0011) [2023-12-27 00:46:04,715][105692] Updated weights for policy 0, policy_version 1288011 (0.0006) [2023-12-27 00:46:05,335][105692] Updated weights for policy 0, policy_version 1288021 (0.0008) [2023-12-27 00:46:05,382][105692] Updated weights for policy 0, policy_version 1288031 (0.0010) [2023-12-27 00:46:05,412][105620] Updated weights for policy 1, policy_version 1289538 (0.0011) [2023-12-27 00:46:05,430][105692] Updated weights for policy 0, policy_version 1288041 (0.0010) [2023-12-27 00:46:05,460][105620] Updated weights for policy 1, policy_version 1289548 (0.0010) [2023-12-27 00:46:05,512][105620] Updated weights for policy 1, policy_version 1289558 (0.0010) [2023-12-27 00:46:05,560][105620] Updated weights for policy 1, policy_version 1289568 (0.0010) [2023-12-27 00:46:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 659963904. Throughput: 0: 9915.4, 1: 9321.2. Samples: 659955236. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:46:06,063][104569] Avg episode reward: [(0, '9265.671'), (1, '9261.924')] [2023-12-27 00:46:06,203][105692] Updated weights for policy 0, policy_version 1288051 (0.0010) [2023-12-27 00:46:06,272][105692] Updated weights for policy 0, policy_version 1288061 (0.0011) [2023-12-27 00:46:06,335][105620] Updated weights for policy 1, policy_version 1289578 (0.0011) [2023-12-27 00:46:06,336][105692] Updated weights for policy 0, policy_version 1288071 (0.0011) [2023-12-27 00:46:06,395][105620] Updated weights for policy 1, policy_version 1289588 (0.0011) [2023-12-27 00:46:06,454][105620] Updated weights for policy 1, policy_version 1289598 (0.0010) [2023-12-27 00:46:07,078][105692] Updated weights for policy 0, policy_version 1288081 (0.0011) [2023-12-27 00:46:07,137][105692] Updated weights for policy 0, policy_version 1288091 (0.0011) [2023-12-27 00:46:07,199][105620] Updated weights for policy 1, policy_version 1289608 (0.0011) [2023-12-27 00:46:07,202][105692] Updated weights for policy 0, policy_version 1288101 (0.0011) [2023-12-27 00:46:07,259][105620] Updated weights for policy 1, policy_version 1289618 (0.0011) [2023-12-27 00:46:07,269][105692] Updated weights for policy 0, policy_version 1288111 (0.0011) [2023-12-27 00:46:07,323][105620] Updated weights for policy 1, policy_version 1289628 (0.0011) [2023-12-27 00:46:08,014][105620] Updated weights for policy 1, policy_version 1289638 (0.0007) [2023-12-27 00:46:08,015][105692] Updated weights for policy 0, policy_version 1288121 (0.0010) [2023-12-27 00:46:08,074][105692] Updated weights for policy 0, policy_version 1288131 (0.0010) [2023-12-27 00:46:08,077][105620] Updated weights for policy 1, policy_version 1289648 (0.0005) [2023-12-27 00:46:08,130][105692] Updated weights for policy 0, policy_version 1288141 (0.0010) [2023-12-27 00:46:08,130][105620] Updated weights for policy 1, policy_version 1289658 (0.0005) [2023-12-27 00:46:08,741][105620] Updated weights for policy 1, policy_version 1289668 (0.0005) [2023-12-27 00:46:08,807][105620] Updated weights for policy 1, policy_version 1289678 (0.0009) [2023-12-27 00:46:08,872][105620] Updated weights for policy 1, policy_version 1289688 (0.0010) [2023-12-27 00:46:08,888][105692] Updated weights for policy 0, policy_version 1288151 (0.0008) [2023-12-27 00:46:08,944][105692] Updated weights for policy 0, policy_version 1288161 (0.0009) [2023-12-27 00:46:09,003][105692] Updated weights for policy 0, policy_version 1288171 (0.0008) [2023-12-27 00:46:09,608][105620] Updated weights for policy 1, policy_version 1289698 (0.0009) [2023-12-27 00:46:09,670][105620] Updated weights for policy 1, policy_version 1289708 (0.0009) [2023-12-27 00:46:09,740][105620] Updated weights for policy 1, policy_version 1289718 (0.0009) [2023-12-27 00:46:09,798][105620] Updated weights for policy 1, policy_version 1289728 (0.0010) [2023-12-27 00:46:09,808][105692] Updated weights for policy 0, policy_version 1288181 (0.0007) [2023-12-27 00:46:09,868][105692] Updated weights for policy 0, policy_version 1288191 (0.0007) [2023-12-27 00:46:09,935][105692] Updated weights for policy 0, policy_version 1288201 (0.0009) [2023-12-27 00:46:10,628][105620] Updated weights for policy 1, policy_version 1289738 (0.0006) [2023-12-27 00:46:10,653][105692] Updated weights for policy 0, policy_version 1288211 (0.0009) [2023-12-27 00:46:10,678][105620] Updated weights for policy 1, policy_version 1289748 (0.0006) [2023-12-27 00:46:10,720][105692] Updated weights for policy 0, policy_version 1288221 (0.0011) [2023-12-27 00:46:10,738][105620] Updated weights for policy 1, policy_version 1289758 (0.0006) [2023-12-27 00:46:10,775][105692] Updated weights for policy 0, policy_version 1288231 (0.0010) [2023-12-27 00:46:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 660062208. Throughput: 0: 9824.7, 1: 9378.7. Samples: 660069780. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:46:11,062][104569] Avg episode reward: [(0, '9268.721'), (1, '8988.619')] [2023-12-27 00:46:11,498][105620] Updated weights for policy 1, policy_version 1289768 (0.0007) [2023-12-27 00:46:11,499][105692] Updated weights for policy 0, policy_version 1288241 (0.0010) [2023-12-27 00:46:11,560][105620] Updated weights for policy 1, policy_version 1289778 (0.0007) [2023-12-27 00:46:11,562][105692] Updated weights for policy 0, policy_version 1288251 (0.0010) [2023-12-27 00:46:11,621][105692] Updated weights for policy 0, policy_version 1288261 (0.0010) [2023-12-27 00:46:11,622][105620] Updated weights for policy 1, policy_version 1289788 (0.0006) [2023-12-27 00:46:11,686][105692] Updated weights for policy 0, policy_version 1288271 (0.0011) [2023-12-27 00:46:12,454][105620] Updated weights for policy 1, policy_version 1289798 (0.0007) [2023-12-27 00:46:12,471][105692] Updated weights for policy 0, policy_version 1288281 (0.0010) [2023-12-27 00:46:12,513][105620] Updated weights for policy 1, policy_version 1289808 (0.0007) [2023-12-27 00:46:12,534][105692] Updated weights for policy 0, policy_version 1288291 (0.0010) [2023-12-27 00:46:12,573][105620] Updated weights for policy 1, policy_version 1289818 (0.0009) [2023-12-27 00:46:12,594][105692] Updated weights for policy 0, policy_version 1288301 (0.0011) [2023-12-27 00:46:13,213][105620] Updated weights for policy 1, policy_version 1289828 (0.0007) [2023-12-27 00:46:13,265][105620] Updated weights for policy 1, policy_version 1289838 (0.0010) [2023-12-27 00:46:13,299][105692] Updated weights for policy 0, policy_version 1288311 (0.0007) [2023-12-27 00:46:13,315][105620] Updated weights for policy 1, policy_version 1289849 (0.0007) [2023-12-27 00:46:13,357][105692] Updated weights for policy 0, policy_version 1288321 (0.0005) [2023-12-27 00:46:13,423][105692] Updated weights for policy 0, policy_version 1288331 (0.0005) [2023-12-27 00:46:14,008][105692] Updated weights for policy 0, policy_version 1288341 (0.0007) [2023-12-27 00:46:14,029][105620] Updated weights for policy 1, policy_version 1289859 (0.0007) [2023-12-27 00:46:14,054][105692] Updated weights for policy 0, policy_version 1288351 (0.0005) [2023-12-27 00:46:14,087][105620] Updated weights for policy 1, policy_version 1289869 (0.0010) [2023-12-27 00:46:14,107][105692] Updated weights for policy 0, policy_version 1288361 (0.0006) [2023-12-27 00:46:14,150][105620] Updated weights for policy 1, policy_version 1289879 (0.0007) [2023-12-27 00:46:14,733][105620] Updated weights for policy 1, policy_version 1289889 (0.0007) [2023-12-27 00:46:14,740][105692] Updated weights for policy 0, policy_version 1288371 (0.0007) [2023-12-27 00:46:14,800][105620] Updated weights for policy 1, policy_version 1289899 (0.0007) [2023-12-27 00:46:14,810][105692] Updated weights for policy 0, policy_version 1288381 (0.0008) [2023-12-27 00:46:14,861][105620] Updated weights for policy 1, policy_version 1289909 (0.0007) [2023-12-27 00:46:14,871][105692] Updated weights for policy 0, policy_version 1288391 (0.0007) [2023-12-27 00:46:14,918][105620] Updated weights for policy 1, policy_version 1289919 (0.0007) [2023-12-27 00:46:15,503][105692] Updated weights for policy 0, policy_version 1288401 (0.0007) [2023-12-27 00:46:15,563][105692] Updated weights for policy 0, policy_version 1288411 (0.0008) [2023-12-27 00:46:15,628][105692] Updated weights for policy 0, policy_version 1288421 (0.0007) [2023-12-27 00:46:15,691][105692] Updated weights for policy 0, policy_version 1288431 (0.0008) [2023-12-27 00:46:15,693][105620] Updated weights for policy 1, policy_version 1289929 (0.0007) [2023-12-27 00:46:15,745][105620] Updated weights for policy 1, policy_version 1289939 (0.0009) [2023-12-27 00:46:15,805][105620] Updated weights for policy 1, policy_version 1289949 (0.0010) [2023-12-27 00:46:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19251.1, 300 sec: 19438.6). Total num frames: 660160512. Throughput: 0: 9804.1, 1: 9374.6. Samples: 660126724. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:46:16,063][104569] Avg episode reward: [(0, '9269.612'), (1, '8567.057')] [2023-12-27 00:46:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001288432_329891840.pth... [2023-12-27 00:46:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001289952_330268672.pth... [2023-12-27 00:46:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001287280_329596928.pth [2023-12-27 00:46:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001288864_329990144.pth [2023-12-27 00:46:16,263][105692] Updated weights for policy 0, policy_version 1288441 (0.0008) [2023-12-27 00:46:16,318][105692] Updated weights for policy 0, policy_version 1288451 (0.0009) [2023-12-27 00:46:16,383][105692] Updated weights for policy 0, policy_version 1288461 (0.0008) [2023-12-27 00:46:16,623][105620] Updated weights for policy 1, policy_version 1289959 (0.0007) [2023-12-27 00:46:16,694][105620] Updated weights for policy 1, policy_version 1289969 (0.0005) [2023-12-27 00:46:16,740][105620] Updated weights for policy 1, policy_version 1289979 (0.0005) [2023-12-27 00:46:17,220][105692] Updated weights for policy 0, policy_version 1288472 (0.0010) [2023-12-27 00:46:17,266][105620] Updated weights for policy 1, policy_version 1289989 (0.0005) [2023-12-27 00:46:17,278][105692] Updated weights for policy 0, policy_version 1288482 (0.0009) [2023-12-27 00:46:17,315][105620] Updated weights for policy 1, policy_version 1289999 (0.0005) [2023-12-27 00:46:17,323][105692] Updated weights for policy 0, policy_version 1288492 (0.0008) [2023-12-27 00:46:17,368][105620] Updated weights for policy 1, policy_version 1290009 (0.0006) [2023-12-27 00:46:17,969][105692] Updated weights for policy 0, policy_version 1288502 (0.0007) [2023-12-27 00:46:18,035][105692] Updated weights for policy 0, policy_version 1288512 (0.0010) [2023-12-27 00:46:18,083][105692] Updated weights for policy 0, policy_version 1288522 (0.0008) [2023-12-27 00:46:18,134][105620] Updated weights for policy 1, policy_version 1290019 (0.0009) [2023-12-27 00:46:18,196][105620] Updated weights for policy 1, policy_version 1290029 (0.0008) [2023-12-27 00:46:18,253][105620] Updated weights for policy 1, policy_version 1290039 (0.0009) [2023-12-27 00:46:18,736][105692] Updated weights for policy 0, policy_version 1288532 (0.0009) [2023-12-27 00:46:18,795][105692] Updated weights for policy 0, policy_version 1288542 (0.0009) [2023-12-27 00:46:18,849][105692] Updated weights for policy 0, policy_version 1288552 (0.0009) [2023-12-27 00:46:19,096][105620] Updated weights for policy 1, policy_version 1290049 (0.0010) [2023-12-27 00:46:19,155][105620] Updated weights for policy 1, policy_version 1290059 (0.0009) [2023-12-27 00:46:19,217][105620] Updated weights for policy 1, policy_version 1290069 (0.0009) [2023-12-27 00:46:19,279][105620] Updated weights for policy 1, policy_version 1290079 (0.0009) [2023-12-27 00:46:19,524][105692] Updated weights for policy 0, policy_version 1288562 (0.0009) [2023-12-27 00:46:19,573][105692] Updated weights for policy 0, policy_version 1288572 (0.0009) [2023-12-27 00:46:19,621][105692] Updated weights for policy 0, policy_version 1288582 (0.0008) [2023-12-27 00:46:19,673][105692] Updated weights for policy 0, policy_version 1288592 (0.0008) [2023-12-27 00:46:20,034][105620] Updated weights for policy 1, policy_version 1290089 (0.0010) [2023-12-27 00:46:20,091][105620] Updated weights for policy 1, policy_version 1290099 (0.0011) [2023-12-27 00:46:20,147][105620] Updated weights for policy 1, policy_version 1290109 (0.0011) [2023-12-27 00:46:20,481][105692] Updated weights for policy 0, policy_version 1288602 (0.0008) [2023-12-27 00:46:20,530][105692] Updated weights for policy 0, policy_version 1288612 (0.0007) [2023-12-27 00:46:20,596][105692] Updated weights for policy 0, policy_version 1288622 (0.0007) [2023-12-27 00:46:20,884][105620] Updated weights for policy 1, policy_version 1290119 (0.0011) [2023-12-27 00:46:20,933][105620] Updated weights for policy 1, policy_version 1290129 (0.0008) [2023-12-27 00:46:20,990][105620] Updated weights for policy 1, policy_version 1290139 (0.0007) [2023-12-27 00:46:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19438.7). Total num frames: 660258816. Throughput: 0: 9807.1, 1: 9374.3. Samples: 660246692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:46:21,063][104569] Avg episode reward: [(0, '9182.344'), (1, '8839.435')] [2023-12-27 00:46:21,356][105692] Updated weights for policy 0, policy_version 1288632 (0.0013) [2023-12-27 00:46:21,419][105692] Updated weights for policy 0, policy_version 1288642 (0.0010) [2023-12-27 00:46:21,475][105692] Updated weights for policy 0, policy_version 1288652 (0.0011) [2023-12-27 00:46:21,811][105620] Updated weights for policy 1, policy_version 1290149 (0.0008) [2023-12-27 00:46:21,871][105620] Updated weights for policy 1, policy_version 1290159 (0.0008) [2023-12-27 00:46:21,921][105620] Updated weights for policy 1, policy_version 1290169 (0.0008) [2023-12-27 00:46:22,225][105692] Updated weights for policy 0, policy_version 1288662 (0.0010) [2023-12-27 00:46:22,288][105692] Updated weights for policy 0, policy_version 1288672 (0.0010) [2023-12-27 00:46:22,349][105692] Updated weights for policy 0, policy_version 1288682 (0.0009) [2023-12-27 00:46:22,617][105620] Updated weights for policy 1, policy_version 1290179 (0.0009) [2023-12-27 00:46:22,672][105620] Updated weights for policy 1, policy_version 1290189 (0.0009) [2023-12-27 00:46:22,729][105620] Updated weights for policy 1, policy_version 1290199 (0.0009) [2023-12-27 00:46:23,070][105692] Updated weights for policy 0, policy_version 1288692 (0.0009) [2023-12-27 00:46:23,125][105692] Updated weights for policy 0, policy_version 1288702 (0.0009) [2023-12-27 00:46:23,184][105692] Updated weights for policy 0, policy_version 1288712 (0.0009) [2023-12-27 00:46:23,483][105620] Updated weights for policy 1, policy_version 1290209 (0.0007) [2023-12-27 00:46:23,541][105620] Updated weights for policy 1, policy_version 1290219 (0.0009) [2023-12-27 00:46:23,599][105620] Updated weights for policy 1, policy_version 1290229 (0.0009) [2023-12-27 00:46:23,651][105620] Updated weights for policy 1, policy_version 1290239 (0.0009) [2023-12-27 00:46:23,924][105692] Updated weights for policy 0, policy_version 1288722 (0.0009) [2023-12-27 00:46:23,978][105692] Updated weights for policy 0, policy_version 1288732 (0.0009) [2023-12-27 00:46:24,036][105692] Updated weights for policy 0, policy_version 1288742 (0.0009) [2023-12-27 00:46:24,097][105692] Updated weights for policy 0, policy_version 1288752 (0.0009) [2023-12-27 00:46:24,385][105620] Updated weights for policy 1, policy_version 1290249 (0.0008) [2023-12-27 00:46:24,445][105620] Updated weights for policy 1, policy_version 1290259 (0.0008) [2023-12-27 00:46:24,504][105620] Updated weights for policy 1, policy_version 1290269 (0.0007) [2023-12-27 00:46:24,884][105692] Updated weights for policy 0, policy_version 1288762 (0.0010) [2023-12-27 00:46:24,942][105692] Updated weights for policy 0, policy_version 1288772 (0.0010) [2023-12-27 00:46:24,996][105692] Updated weights for policy 0, policy_version 1288782 (0.0010) [2023-12-27 00:46:25,155][105620] Updated weights for policy 1, policy_version 1290279 (0.0006) [2023-12-27 00:46:25,202][105620] Updated weights for policy 1, policy_version 1290289 (0.0005) [2023-12-27 00:46:25,249][105620] Updated weights for policy 1, policy_version 1290299 (0.0009) [2023-12-27 00:46:25,726][105692] Updated weights for policy 0, policy_version 1288792 (0.0011) [2023-12-27 00:46:25,780][105692] Updated weights for policy 0, policy_version 1288802 (0.0010) [2023-12-27 00:46:25,829][105692] Updated weights for policy 0, policy_version 1288812 (0.0010) [2023-12-27 00:46:25,834][105620] Updated weights for policy 1, policy_version 1290309 (0.0008) [2023-12-27 00:46:25,881][105620] Updated weights for policy 1, policy_version 1290319 (0.0005) [2023-12-27 00:46:25,932][105620] Updated weights for policy 1, policy_version 1290329 (0.0005) [2023-12-27 00:46:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.6, 300 sec: 19383.1). Total num frames: 660357120. Throughput: 0: 9766.8, 1: 9395.6. Samples: 660362044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:46:26,063][104569] Avg episode reward: [(0, '8730.061'), (1, '9078.448')] [2023-12-27 00:46:26,534][105620] Updated weights for policy 1, policy_version 1290339 (0.0005) [2023-12-27 00:46:26,575][105692] Updated weights for policy 0, policy_version 1288822 (0.0010) [2023-12-27 00:46:26,585][105620] Updated weights for policy 1, policy_version 1290349 (0.0005) [2023-12-27 00:46:26,623][105692] Updated weights for policy 0, policy_version 1288832 (0.0010) [2023-12-27 00:46:26,641][105620] Updated weights for policy 1, policy_version 1290359 (0.0006) [2023-12-27 00:46:26,671][105692] Updated weights for policy 0, policy_version 1288842 (0.0010) [2023-12-27 00:46:27,254][105620] Updated weights for policy 1, policy_version 1290369 (0.0006) [2023-12-27 00:46:27,315][105620] Updated weights for policy 1, policy_version 1290379 (0.0009) [2023-12-27 00:46:27,372][105620] Updated weights for policy 1, policy_version 1290389 (0.0010) [2023-12-27 00:46:27,421][105692] Updated weights for policy 0, policy_version 1288852 (0.0008) [2023-12-27 00:46:27,433][105620] Updated weights for policy 1, policy_version 1290399 (0.0007) [2023-12-27 00:46:27,481][105692] Updated weights for policy 0, policy_version 1288862 (0.0010) [2023-12-27 00:46:27,538][105692] Updated weights for policy 0, policy_version 1288873 (0.0010) [2023-12-27 00:46:28,098][105620] Updated weights for policy 1, policy_version 1290409 (0.0007) [2023-12-27 00:46:28,153][105620] Updated weights for policy 1, policy_version 1290419 (0.0005) [2023-12-27 00:46:28,203][105620] Updated weights for policy 1, policy_version 1290429 (0.0009) [2023-12-27 00:46:28,338][105692] Updated weights for policy 0, policy_version 1288884 (0.0010) [2023-12-27 00:46:28,396][105692] Updated weights for policy 0, policy_version 1288894 (0.0009) [2023-12-27 00:46:28,451][105692] Updated weights for policy 0, policy_version 1288904 (0.0010) [2023-12-27 00:46:28,864][105620] Updated weights for policy 1, policy_version 1290439 (0.0007) [2023-12-27 00:46:28,925][105620] Updated weights for policy 1, policy_version 1290449 (0.0010) [2023-12-27 00:46:28,985][105620] Updated weights for policy 1, policy_version 1290459 (0.0010) [2023-12-27 00:46:29,203][105692] Updated weights for policy 0, policy_version 1288914 (0.0010) [2023-12-27 00:46:29,265][105692] Updated weights for policy 0, policy_version 1288924 (0.0008) [2023-12-27 00:46:29,314][105692] Updated weights for policy 0, policy_version 1288934 (0.0010) [2023-12-27 00:46:29,378][105692] Updated weights for policy 0, policy_version 1288944 (0.0010) [2023-12-27 00:46:29,710][105620] Updated weights for policy 1, policy_version 1290469 (0.0010) [2023-12-27 00:46:29,771][105620] Updated weights for policy 1, policy_version 1290479 (0.0010) [2023-12-27 00:46:29,840][105620] Updated weights for policy 1, policy_version 1290489 (0.0011) [2023-12-27 00:46:30,015][105692] Updated weights for policy 0, policy_version 1288954 (0.0010) [2023-12-27 00:46:30,080][105692] Updated weights for policy 0, policy_version 1288964 (0.0010) [2023-12-27 00:46:30,131][105692] Updated weights for policy 0, policy_version 1288974 (0.0010) [2023-12-27 00:46:30,540][105620] Updated weights for policy 1, policy_version 1290499 (0.0009) [2023-12-27 00:46:30,586][105620] Updated weights for policy 1, policy_version 1290509 (0.0005) [2023-12-27 00:46:30,646][105620] Updated weights for policy 1, policy_version 1290519 (0.0005) [2023-12-27 00:46:30,796][105692] Updated weights for policy 0, policy_version 1288984 (0.0009) [2023-12-27 00:46:30,854][105692] Updated weights for policy 0, policy_version 1288994 (0.0010) [2023-12-27 00:46:30,908][105692] Updated weights for policy 0, policy_version 1289004 (0.0010) [2023-12-27 00:46:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 660455424. Throughput: 0: 9741.3, 1: 9512.8. Samples: 660422188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:46:31,062][104569] Avg episode reward: [(0, '8544.110'), (1, '9078.935')] [2023-12-27 00:46:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001289008_330039296.pth... [2023-12-27 00:46:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001290528_330416128.pth... [2023-12-27 00:46:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001289408_330129408.pth [2023-12-27 00:46:31,088][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001287856_329744384.pth [2023-12-27 00:46:31,324][105620] Updated weights for policy 1, policy_version 1290529 (0.0006) [2023-12-27 00:46:31,386][105620] Updated weights for policy 1, policy_version 1290539 (0.0011) [2023-12-27 00:46:31,443][105620] Updated weights for policy 1, policy_version 1290549 (0.0012) [2023-12-27 00:46:31,502][105620] Updated weights for policy 1, policy_version 1290559 (0.0011) [2023-12-27 00:46:31,636][105692] Updated weights for policy 0, policy_version 1289014 (0.0010) [2023-12-27 00:46:31,695][105692] Updated weights for policy 0, policy_version 1289026 (0.0010) [2023-12-27 00:46:31,761][105692] Updated weights for policy 0, policy_version 1289036 (0.0009) [2023-12-27 00:46:32,203][105620] Updated weights for policy 1, policy_version 1290569 (0.0010) [2023-12-27 00:46:32,261][105620] Updated weights for policy 1, policy_version 1290579 (0.0010) [2023-12-27 00:46:32,318][105620] Updated weights for policy 1, policy_version 1290589 (0.0007) [2023-12-27 00:46:32,527][105692] Updated weights for policy 0, policy_version 1289046 (0.0009) [2023-12-27 00:46:32,580][105692] Updated weights for policy 0, policy_version 1289056 (0.0010) [2023-12-27 00:46:32,639][105692] Updated weights for policy 0, policy_version 1289066 (0.0010) [2023-12-27 00:46:32,905][105620] Updated weights for policy 1, policy_version 1290599 (0.0009) [2023-12-27 00:46:32,960][105620] Updated weights for policy 1, policy_version 1290609 (0.0010) [2023-12-27 00:46:33,004][105620] Updated weights for policy 1, policy_version 1290619 (0.0010) [2023-12-27 00:46:33,312][105692] Updated weights for policy 0, policy_version 1289077 (0.0010) [2023-12-27 00:46:33,365][105692] Updated weights for policy 0, policy_version 1289087 (0.0005) [2023-12-27 00:46:33,415][105692] Updated weights for policy 0, policy_version 1289097 (0.0005) [2023-12-27 00:46:33,757][105620] Updated weights for policy 1, policy_version 1290629 (0.0010) [2023-12-27 00:46:33,808][105620] Updated weights for policy 1, policy_version 1290639 (0.0010) [2023-12-27 00:46:33,855][105620] Updated weights for policy 1, policy_version 1290649 (0.0010) [2023-12-27 00:46:33,995][105692] Updated weights for policy 0, policy_version 1289107 (0.0005) [2023-12-27 00:46:34,058][105692] Updated weights for policy 0, policy_version 1289117 (0.0006) [2023-12-27 00:46:34,114][105692] Updated weights for policy 0, policy_version 1289127 (0.0006) [2023-12-27 00:46:34,609][105620] Updated weights for policy 1, policy_version 1290659 (0.0009) [2023-12-27 00:46:34,667][105620] Updated weights for policy 1, policy_version 1290669 (0.0010) [2023-12-27 00:46:34,729][105620] Updated weights for policy 1, policy_version 1290679 (0.0011) [2023-12-27 00:46:34,774][105692] Updated weights for policy 0, policy_version 1289137 (0.0009) [2023-12-27 00:46:34,843][105692] Updated weights for policy 0, policy_version 1289147 (0.0008) [2023-12-27 00:46:34,905][105692] Updated weights for policy 0, policy_version 1289157 (0.0009) [2023-12-27 00:46:34,970][105692] Updated weights for policy 0, policy_version 1289167 (0.0008) [2023-12-27 00:46:35,349][105620] Updated weights for policy 1, policy_version 1290689 (0.0008) [2023-12-27 00:46:35,400][105620] Updated weights for policy 1, policy_version 1290699 (0.0005) [2023-12-27 00:46:35,455][105620] Updated weights for policy 1, policy_version 1290709 (0.0006) [2023-12-27 00:46:35,518][105620] Updated weights for policy 1, policy_version 1290719 (0.0006) [2023-12-27 00:46:35,667][105692] Updated weights for policy 0, policy_version 1289177 (0.0006) [2023-12-27 00:46:35,726][105692] Updated weights for policy 0, policy_version 1289187 (0.0009) [2023-12-27 00:46:35,775][105692] Updated weights for policy 0, policy_version 1289197 (0.0008) [2023-12-27 00:46:36,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 660553728. Throughput: 0: 9793.1, 1: 9605.2. Samples: 660542912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:46:36,062][104569] Avg episode reward: [(0, '8993.482'), (1, '9261.517')] [2023-12-27 00:46:36,105][105620] Updated weights for policy 1, policy_version 1290729 (0.0006) [2023-12-27 00:46:36,166][105620] Updated weights for policy 1, policy_version 1290739 (0.0009) [2023-12-27 00:46:36,222][105620] Updated weights for policy 1, policy_version 1290749 (0.0011) [2023-12-27 00:46:36,545][105692] Updated weights for policy 0, policy_version 1289207 (0.0008) [2023-12-27 00:46:36,606][105692] Updated weights for policy 0, policy_version 1289217 (0.0008) [2023-12-27 00:46:36,666][105692] Updated weights for policy 0, policy_version 1289227 (0.0008) [2023-12-27 00:46:36,935][105620] Updated weights for policy 1, policy_version 1290759 (0.0011) [2023-12-27 00:46:36,988][105620] Updated weights for policy 1, policy_version 1290769 (0.0011) [2023-12-27 00:46:37,037][105620] Updated weights for policy 1, policy_version 1290779 (0.0010) [2023-12-27 00:46:37,376][105692] Updated weights for policy 0, policy_version 1289237 (0.0009) [2023-12-27 00:46:37,440][105692] Updated weights for policy 0, policy_version 1289247 (0.0011) [2023-12-27 00:46:37,504][105692] Updated weights for policy 0, policy_version 1289257 (0.0007) [2023-12-27 00:46:37,822][105620] Updated weights for policy 1, policy_version 1290789 (0.0011) [2023-12-27 00:46:37,897][105620] Updated weights for policy 1, policy_version 1290799 (0.0010) [2023-12-27 00:46:37,961][105620] Updated weights for policy 1, policy_version 1290809 (0.0008) [2023-12-27 00:46:38,110][105692] Updated weights for policy 0, policy_version 1289267 (0.0005) [2023-12-27 00:46:38,170][105692] Updated weights for policy 0, policy_version 1289277 (0.0008) [2023-12-27 00:46:38,227][105692] Updated weights for policy 0, policy_version 1289287 (0.0009) [2023-12-27 00:46:38,577][105620] Updated weights for policy 1, policy_version 1290819 (0.0007) [2023-12-27 00:46:38,643][105620] Updated weights for policy 1, policy_version 1290829 (0.0011) [2023-12-27 00:46:38,702][105620] Updated weights for policy 1, policy_version 1290839 (0.0011) [2023-12-27 00:46:38,916][105692] Updated weights for policy 0, policy_version 1289297 (0.0009) [2023-12-27 00:46:38,981][105692] Updated weights for policy 0, policy_version 1289307 (0.0007) [2023-12-27 00:46:39,040][105692] Updated weights for policy 0, policy_version 1289317 (0.0006) [2023-12-27 00:46:39,089][105692] Updated weights for policy 0, policy_version 1289327 (0.0005) [2023-12-27 00:46:39,460][105620] Updated weights for policy 1, policy_version 1290849 (0.0010) [2023-12-27 00:46:39,526][105620] Updated weights for policy 1, policy_version 1290859 (0.0006) [2023-12-27 00:46:39,601][105620] Updated weights for policy 1, policy_version 1290869 (0.0006) [2023-12-27 00:46:39,672][105620] Updated weights for policy 1, policy_version 1290879 (0.0006) [2023-12-27 00:46:39,826][105692] Updated weights for policy 0, policy_version 1289337 (0.0006) [2023-12-27 00:46:39,894][105692] Updated weights for policy 0, policy_version 1289347 (0.0010) [2023-12-27 00:46:39,962][105692] Updated weights for policy 0, policy_version 1289357 (0.0009) [2023-12-27 00:46:40,214][105620] Updated weights for policy 1, policy_version 1290889 (0.0008) [2023-12-27 00:46:40,263][105620] Updated weights for policy 1, policy_version 1290899 (0.0009) [2023-12-27 00:46:40,311][105620] Updated weights for policy 1, policy_version 1290909 (0.0009) [2023-12-27 00:46:40,720][105692] Updated weights for policy 0, policy_version 1289367 (0.0008) [2023-12-27 00:46:40,790][105692] Updated weights for policy 0, policy_version 1289377 (0.0006) [2023-12-27 00:46:40,835][105692] Updated weights for policy 0, policy_version 1289387 (0.0008) [2023-12-27 00:46:41,037][105620] Updated weights for policy 1, policy_version 1290919 (0.0010) [2023-12-27 00:46:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19438.7). Total num frames: 660652032. Throughput: 0: 9771.3, 1: 9717.4. Samples: 660662168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:46:41,062][104569] Avg episode reward: [(0, '9266.895'), (1, '9352.461')] [2023-12-27 00:46:41,104][105620] Updated weights for policy 1, policy_version 1290929 (0.0010) [2023-12-27 00:46:41,173][105620] Updated weights for policy 1, policy_version 1290939 (0.0010) [2023-12-27 00:46:41,574][105692] Updated weights for policy 0, policy_version 1289397 (0.0007) [2023-12-27 00:46:41,638][105692] Updated weights for policy 0, policy_version 1289407 (0.0007) [2023-12-27 00:46:41,690][105692] Updated weights for policy 0, policy_version 1289417 (0.0008) [2023-12-27 00:46:41,895][105620] Updated weights for policy 1, policy_version 1290949 (0.0011) [2023-12-27 00:46:41,958][105620] Updated weights for policy 1, policy_version 1290959 (0.0011) [2023-12-27 00:46:42,024][105620] Updated weights for policy 1, policy_version 1290969 (0.0011) [2023-12-27 00:46:42,486][105692] Updated weights for policy 0, policy_version 1289427 (0.0009) [2023-12-27 00:46:42,545][105692] Updated weights for policy 0, policy_version 1289437 (0.0009) [2023-12-27 00:46:42,604][105692] Updated weights for policy 0, policy_version 1289447 (0.0010) [2023-12-27 00:46:42,718][105620] Updated weights for policy 1, policy_version 1290979 (0.0009) [2023-12-27 00:46:42,769][105620] Updated weights for policy 1, policy_version 1290989 (0.0005) [2023-12-27 00:46:42,816][105620] Updated weights for policy 1, policy_version 1290999 (0.0005) [2023-12-27 00:46:43,392][105620] Updated weights for policy 1, policy_version 1291009 (0.0009) [2023-12-27 00:46:43,416][105692] Updated weights for policy 0, policy_version 1289457 (0.0010) [2023-12-27 00:46:43,451][105620] Updated weights for policy 1, policy_version 1291019 (0.0009) [2023-12-27 00:46:43,477][105692] Updated weights for policy 0, policy_version 1289467 (0.0007) [2023-12-27 00:46:43,510][105620] Updated weights for policy 1, policy_version 1291029 (0.0010) [2023-12-27 00:46:43,541][105692] Updated weights for policy 0, policy_version 1289477 (0.0007) [2023-12-27 00:46:43,568][105620] Updated weights for policy 1, policy_version 1291039 (0.0008) [2023-12-27 00:46:43,603][105692] Updated weights for policy 0, policy_version 1289487 (0.0009) [2023-12-27 00:46:44,153][105620] Updated weights for policy 1, policy_version 1291049 (0.0010) [2023-12-27 00:46:44,208][105620] Updated weights for policy 1, policy_version 1291059 (0.0007) [2023-12-27 00:46:44,266][105620] Updated weights for policy 1, policy_version 1291069 (0.0006) [2023-12-27 00:46:44,423][105692] Updated weights for policy 0, policy_version 1289497 (0.0008) [2023-12-27 00:46:44,469][105692] Updated weights for policy 0, policy_version 1289507 (0.0008) [2023-12-27 00:46:44,513][105692] Updated weights for policy 0, policy_version 1289517 (0.0007) [2023-12-27 00:46:44,992][105620] Updated weights for policy 1, policy_version 1291079 (0.0010) [2023-12-27 00:46:45,048][105620] Updated weights for policy 1, policy_version 1291089 (0.0010) [2023-12-27 00:46:45,106][105620] Updated weights for policy 1, policy_version 1291099 (0.0009) [2023-12-27 00:46:45,261][105692] Updated weights for policy 0, policy_version 1289527 (0.0010) [2023-12-27 00:46:45,321][105692] Updated weights for policy 0, policy_version 1289537 (0.0011) [2023-12-27 00:46:45,378][105692] Updated weights for policy 0, policy_version 1289547 (0.0011) [2023-12-27 00:46:45,790][105620] Updated weights for policy 1, policy_version 1291109 (0.0006) [2023-12-27 00:46:45,850][105620] Updated weights for policy 1, policy_version 1291119 (0.0005) [2023-12-27 00:46:45,913][105620] Updated weights for policy 1, policy_version 1291129 (0.0005) [2023-12-27 00:46:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 660750336. Throughput: 0: 9684.0, 1: 9848.1. Samples: 660720896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:46:46,062][104569] Avg episode reward: [(0, '9266.819'), (1, '9352.356')] [2023-12-27 00:46:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001289552_330178560.pth... [2023-12-27 00:46:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001291136_330571776.pth... [2023-12-27 00:46:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001289952_330268672.pth [2023-12-27 00:46:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001288432_329891840.pth [2023-12-27 00:46:46,174][105692] Updated weights for policy 0, policy_version 1289557 (0.0010) [2023-12-27 00:46:46,238][105692] Updated weights for policy 0, policy_version 1289567 (0.0010) [2023-12-27 00:46:46,300][105692] Updated weights for policy 0, policy_version 1289577 (0.0009) [2023-12-27 00:46:46,432][105620] Updated weights for policy 1, policy_version 1291139 (0.0007) [2023-12-27 00:46:46,492][105620] Updated weights for policy 1, policy_version 1291149 (0.0008) [2023-12-27 00:46:46,548][105620] Updated weights for policy 1, policy_version 1291159 (0.0010) [2023-12-27 00:46:47,145][105692] Updated weights for policy 0, policy_version 1289587 (0.0010) [2023-12-27 00:46:47,151][105620] Updated weights for policy 1, policy_version 1291169 (0.0007) [2023-12-27 00:46:47,198][105692] Updated weights for policy 0, policy_version 1289597 (0.0009) [2023-12-27 00:46:47,214][105620] Updated weights for policy 1, policy_version 1291179 (0.0006) [2023-12-27 00:46:47,246][105692] Updated weights for policy 0, policy_version 1289607 (0.0008) [2023-12-27 00:46:47,278][105620] Updated weights for policy 1, policy_version 1291189 (0.0008) [2023-12-27 00:46:47,328][105620] Updated weights for policy 1, policy_version 1291199 (0.0008) [2023-12-27 00:46:47,991][105692] Updated weights for policy 0, policy_version 1289617 (0.0007) [2023-12-27 00:46:48,021][105620] Updated weights for policy 1, policy_version 1291209 (0.0010) [2023-12-27 00:46:48,043][105692] Updated weights for policy 0, policy_version 1289627 (0.0006) [2023-12-27 00:46:48,065][105620] Updated weights for policy 1, policy_version 1291219 (0.0007) [2023-12-27 00:46:48,090][105692] Updated weights for policy 0, policy_version 1289637 (0.0005) [2023-12-27 00:46:48,126][105620] Updated weights for policy 1, policy_version 1291229 (0.0010) [2023-12-27 00:46:48,137][105692] Updated weights for policy 0, policy_version 1289647 (0.0005) [2023-12-27 00:46:48,840][105620] Updated weights for policy 1, policy_version 1291239 (0.0010) [2023-12-27 00:46:48,842][105692] Updated weights for policy 0, policy_version 1289657 (0.0010) [2023-12-27 00:46:48,900][105692] Updated weights for policy 0, policy_version 1289667 (0.0010) [2023-12-27 00:46:48,901][105620] Updated weights for policy 1, policy_version 1291249 (0.0010) [2023-12-27 00:46:48,966][105692] Updated weights for policy 0, policy_version 1289677 (0.0006) [2023-12-27 00:46:48,968][105620] Updated weights for policy 1, policy_version 1291259 (0.0011) [2023-12-27 00:46:49,634][105692] Updated weights for policy 0, policy_version 1289687 (0.0005) [2023-12-27 00:46:49,663][105620] Updated weights for policy 1, policy_version 1291269 (0.0010) [2023-12-27 00:46:49,697][105692] Updated weights for policy 0, policy_version 1289697 (0.0005) [2023-12-27 00:46:49,717][105620] Updated weights for policy 1, policy_version 1291279 (0.0010) [2023-12-27 00:46:49,757][105692] Updated weights for policy 0, policy_version 1289707 (0.0007) [2023-12-27 00:46:49,770][105620] Updated weights for policy 1, policy_version 1291289 (0.0011) [2023-12-27 00:46:50,451][105692] Updated weights for policy 0, policy_version 1289717 (0.0007) [2023-12-27 00:46:50,507][105692] Updated weights for policy 0, policy_version 1289727 (0.0008) [2023-12-27 00:46:50,525][105620] Updated weights for policy 1, policy_version 1291299 (0.0008) [2023-12-27 00:46:50,555][105692] Updated weights for policy 0, policy_version 1289737 (0.0008) [2023-12-27 00:46:50,584][105620] Updated weights for policy 1, policy_version 1291309 (0.0012) [2023-12-27 00:46:50,652][105620] Updated weights for policy 1, policy_version 1291319 (0.0010) [2023-12-27 00:46:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 660848640. Throughput: 0: 9639.5, 1: 9988.3. Samples: 660838484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:46:51,062][104569] Avg episode reward: [(0, '9175.972'), (1, '9261.206')] [2023-12-27 00:46:51,279][105692] Updated weights for policy 0, policy_version 1289747 (0.0008) [2023-12-27 00:46:51,331][105692] Updated weights for policy 0, policy_version 1289757 (0.0008) [2023-12-27 00:46:51,395][105692] Updated weights for policy 0, policy_version 1289767 (0.0009) [2023-12-27 00:46:51,418][105620] Updated weights for policy 1, policy_version 1291329 (0.0011) [2023-12-27 00:46:51,478][105620] Updated weights for policy 1, policy_version 1291339 (0.0011) [2023-12-27 00:46:51,537][105620] Updated weights for policy 1, policy_version 1291349 (0.0011) [2023-12-27 00:46:51,583][105620] Updated weights for policy 1, policy_version 1291359 (0.0011) [2023-12-27 00:46:52,172][105692] Updated weights for policy 0, policy_version 1289777 (0.0007) [2023-12-27 00:46:52,228][105692] Updated weights for policy 0, policy_version 1289787 (0.0010) [2023-12-27 00:46:52,293][105692] Updated weights for policy 0, policy_version 1289797 (0.0010) [2023-12-27 00:46:52,358][105692] Updated weights for policy 0, policy_version 1289807 (0.0011) [2023-12-27 00:46:52,367][105620] Updated weights for policy 1, policy_version 1291369 (0.0009) [2023-12-27 00:46:52,431][105620] Updated weights for policy 1, policy_version 1291379 (0.0006) [2023-12-27 00:46:52,480][105620] Updated weights for policy 1, policy_version 1291389 (0.0010) [2023-12-27 00:46:52,939][105692] Updated weights for policy 0, policy_version 1289817 (0.0008) [2023-12-27 00:46:52,997][105692] Updated weights for policy 0, policy_version 1289827 (0.0007) [2023-12-27 00:46:53,049][105692] Updated weights for policy 0, policy_version 1289837 (0.0008) [2023-12-27 00:46:53,228][105620] Updated weights for policy 1, policy_version 1291399 (0.0011) [2023-12-27 00:46:53,298][105620] Updated weights for policy 1, policy_version 1291409 (0.0009) [2023-12-27 00:46:53,356][105620] Updated weights for policy 1, policy_version 1291419 (0.0010) [2023-12-27 00:46:53,742][105692] Updated weights for policy 0, policy_version 1289847 (0.0008) [2023-12-27 00:46:53,790][105692] Updated weights for policy 0, policy_version 1289857 (0.0008) [2023-12-27 00:46:53,841][105692] Updated weights for policy 0, policy_version 1289867 (0.0008) [2023-12-27 00:46:54,079][105620] Updated weights for policy 1, policy_version 1291429 (0.0011) [2023-12-27 00:46:54,134][105620] Updated weights for policy 1, policy_version 1291439 (0.0011) [2023-12-27 00:46:54,186][105620] Updated weights for policy 1, policy_version 1291449 (0.0008) [2023-12-27 00:46:54,612][105692] Updated weights for policy 0, policy_version 1289877 (0.0008) [2023-12-27 00:46:54,671][105692] Updated weights for policy 0, policy_version 1289887 (0.0008) [2023-12-27 00:46:54,725][105692] Updated weights for policy 0, policy_version 1289897 (0.0006) [2023-12-27 00:46:54,866][105620] Updated weights for policy 1, policy_version 1291459 (0.0010) [2023-12-27 00:46:54,919][105620] Updated weights for policy 1, policy_version 1291469 (0.0009) [2023-12-27 00:46:54,980][105620] Updated weights for policy 1, policy_version 1291479 (0.0007) [2023-12-27 00:46:55,447][105692] Updated weights for policy 0, policy_version 1289907 (0.0006) [2023-12-27 00:46:55,510][105692] Updated weights for policy 0, policy_version 1289917 (0.0007) [2023-12-27 00:46:55,559][105692] Updated weights for policy 0, policy_version 1289927 (0.0007) [2023-12-27 00:46:55,652][105620] Updated weights for policy 1, policy_version 1291489 (0.0011) [2023-12-27 00:46:55,713][105620] Updated weights for policy 1, policy_version 1291499 (0.0009) [2023-12-27 00:46:55,763][105620] Updated weights for policy 1, policy_version 1291509 (0.0005) [2023-12-27 00:46:55,819][105620] Updated weights for policy 1, policy_version 1291519 (0.0007) [2023-12-27 00:46:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 660946944. Throughput: 0: 9685.8, 1: 9979.3. Samples: 660954708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:46:56,063][104569] Avg episode reward: [(0, '9179.505'), (1, '8987.968')] [2023-12-27 00:46:56,375][105620] Updated weights for policy 1, policy_version 1291529 (0.0005) [2023-12-27 00:46:56,401][105692] Updated weights for policy 0, policy_version 1289937 (0.0007) [2023-12-27 00:46:56,430][105620] Updated weights for policy 1, policy_version 1291539 (0.0006) [2023-12-27 00:46:56,456][105692] Updated weights for policy 0, policy_version 1289947 (0.0008) [2023-12-27 00:46:56,489][105620] Updated weights for policy 1, policy_version 1291549 (0.0007) [2023-12-27 00:46:56,505][105692] Updated weights for policy 0, policy_version 1289957 (0.0008) [2023-12-27 00:46:56,558][105692] Updated weights for policy 0, policy_version 1289967 (0.0010) [2023-12-27 00:46:57,156][105620] Updated weights for policy 1, policy_version 1291559 (0.0006) [2023-12-27 00:46:57,211][105620] Updated weights for policy 1, policy_version 1291569 (0.0006) [2023-12-27 00:46:57,269][105620] Updated weights for policy 1, policy_version 1291579 (0.0007) [2023-12-27 00:46:57,308][105692] Updated weights for policy 0, policy_version 1289977 (0.0007) [2023-12-27 00:46:57,361][105692] Updated weights for policy 0, policy_version 1289987 (0.0008) [2023-12-27 00:46:57,417][105692] Updated weights for policy 0, policy_version 1289997 (0.0005) [2023-12-27 00:46:57,931][105620] Updated weights for policy 1, policy_version 1291589 (0.0006) [2023-12-27 00:46:57,982][105620] Updated weights for policy 1, policy_version 1291599 (0.0006) [2023-12-27 00:46:58,034][105620] Updated weights for policy 1, policy_version 1291609 (0.0006) [2023-12-27 00:46:58,183][105692] Updated weights for policy 0, policy_version 1290007 (0.0009) [2023-12-27 00:46:58,244][105692] Updated weights for policy 0, policy_version 1290017 (0.0008) [2023-12-27 00:46:58,306][105692] Updated weights for policy 0, policy_version 1290027 (0.0009) [2023-12-27 00:46:58,747][105620] Updated weights for policy 1, policy_version 1291619 (0.0006) [2023-12-27 00:46:58,821][105620] Updated weights for policy 1, policy_version 1291629 (0.0009) [2023-12-27 00:46:58,891][105620] Updated weights for policy 1, policy_version 1291639 (0.0008) [2023-12-27 00:46:59,099][105692] Updated weights for policy 0, policy_version 1290037 (0.0007) [2023-12-27 00:46:59,174][105692] Updated weights for policy 0, policy_version 1290047 (0.0007) [2023-12-27 00:46:59,241][105692] Updated weights for policy 0, policy_version 1290057 (0.0008) [2023-12-27 00:46:59,711][105620] Updated weights for policy 1, policy_version 1291649 (0.0007) [2023-12-27 00:46:59,757][105620] Updated weights for policy 1, policy_version 1291659 (0.0008) [2023-12-27 00:46:59,804][105620] Updated weights for policy 1, policy_version 1291669 (0.0009) [2023-12-27 00:46:59,863][105620] Updated weights for policy 1, policy_version 1291679 (0.0008) [2023-12-27 00:46:59,914][105692] Updated weights for policy 0, policy_version 1290067 (0.0007) [2023-12-27 00:46:59,982][105692] Updated weights for policy 0, policy_version 1290077 (0.0008) [2023-12-27 00:47:00,045][105692] Updated weights for policy 0, policy_version 1290087 (0.0005) [2023-12-27 00:47:00,531][105620] Updated weights for policy 1, policy_version 1291689 (0.0006) [2023-12-27 00:47:00,580][105692] Updated weights for policy 0, policy_version 1290097 (0.0006) [2023-12-27 00:47:00,592][105620] Updated weights for policy 1, policy_version 1291699 (0.0008) [2023-12-27 00:47:00,648][105692] Updated weights for policy 0, policy_version 1290107 (0.0008) [2023-12-27 00:47:00,661][105620] Updated weights for policy 1, policy_version 1291709 (0.0009) [2023-12-27 00:47:00,709][105692] Updated weights for policy 0, policy_version 1290117 (0.0008) [2023-12-27 00:47:00,772][105692] Updated weights for policy 0, policy_version 1290127 (0.0011) [2023-12-27 00:47:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19383.1). Total num frames: 661045248. Throughput: 0: 9645.6, 1: 10037.2. Samples: 661012444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:47:01,063][104569] Avg episode reward: [(0, '9270.604'), (1, '8896.818')] [2023-12-27 00:47:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001290128_330326016.pth... [2023-12-27 00:47:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001291712_330719232.pth... [2023-12-27 00:47:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001289008_330039296.pth [2023-12-27 00:47:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001290528_330416128.pth [2023-12-27 00:47:01,320][105620] Updated weights for policy 1, policy_version 1291719 (0.0007) [2023-12-27 00:47:01,389][105620] Updated weights for policy 1, policy_version 1291729 (0.0007) [2023-12-27 00:47:01,453][105620] Updated weights for policy 1, policy_version 1291739 (0.0008) [2023-12-27 00:47:01,489][105692] Updated weights for policy 0, policy_version 1290137 (0.0011) [2023-12-27 00:47:01,537][105692] Updated weights for policy 0, policy_version 1290147 (0.0010) [2023-12-27 00:47:01,591][105692] Updated weights for policy 0, policy_version 1290157 (0.0010) [2023-12-27 00:47:02,118][105620] Updated weights for policy 1, policy_version 1291749 (0.0006) [2023-12-27 00:47:02,175][105620] Updated weights for policy 1, policy_version 1291759 (0.0007) [2023-12-27 00:47:02,234][105620] Updated weights for policy 1, policy_version 1291769 (0.0006) [2023-12-27 00:47:02,322][105692] Updated weights for policy 0, policy_version 1290167 (0.0010) [2023-12-27 00:47:02,381][105692] Updated weights for policy 0, policy_version 1290177 (0.0011) [2023-12-27 00:47:02,436][105692] Updated weights for policy 0, policy_version 1290187 (0.0010) [2023-12-27 00:47:02,882][105620] Updated weights for policy 1, policy_version 1291779 (0.0007) [2023-12-27 00:47:02,929][105620] Updated weights for policy 1, policy_version 1291789 (0.0007) [2023-12-27 00:47:02,983][105620] Updated weights for policy 1, policy_version 1291799 (0.0008) [2023-12-27 00:47:03,190][105692] Updated weights for policy 0, policy_version 1290197 (0.0008) [2023-12-27 00:47:03,258][105692] Updated weights for policy 0, policy_version 1290207 (0.0005) [2023-12-27 00:47:03,314][105692] Updated weights for policy 0, policy_version 1290217 (0.0005) [2023-12-27 00:47:03,751][105620] Updated weights for policy 1, policy_version 1291809 (0.0007) [2023-12-27 00:47:03,810][105620] Updated weights for policy 1, policy_version 1291819 (0.0008) [2023-12-27 00:47:03,831][105692] Updated weights for policy 0, policy_version 1290227 (0.0006) [2023-12-27 00:47:03,875][105620] Updated weights for policy 1, policy_version 1291829 (0.0007) [2023-12-27 00:47:03,897][105692] Updated weights for policy 0, policy_version 1290237 (0.0008) [2023-12-27 00:47:03,939][105620] Updated weights for policy 1, policy_version 1291839 (0.0007) [2023-12-27 00:47:03,962][105692] Updated weights for policy 0, policy_version 1290247 (0.0008) [2023-12-27 00:47:04,563][105692] Updated weights for policy 0, policy_version 1290257 (0.0008) [2023-12-27 00:47:04,620][105692] Updated weights for policy 0, policy_version 1290267 (0.0005) [2023-12-27 00:47:04,674][105692] Updated weights for policy 0, policy_version 1290277 (0.0008) [2023-12-27 00:47:04,722][105692] Updated weights for policy 0, policy_version 1290287 (0.0011) [2023-12-27 00:47:04,760][105620] Updated weights for policy 1, policy_version 1291849 (0.0007) [2023-12-27 00:47:04,815][105620] Updated weights for policy 1, policy_version 1291859 (0.0007) [2023-12-27 00:47:04,875][105620] Updated weights for policy 1, policy_version 1291869 (0.0008) [2023-12-27 00:47:05,438][105692] Updated weights for policy 0, policy_version 1290297 (0.0006) [2023-12-27 00:47:05,501][105692] Updated weights for policy 0, policy_version 1290307 (0.0010) [2023-12-27 00:47:05,559][105692] Updated weights for policy 0, policy_version 1290317 (0.0010) [2023-12-27 00:47:05,627][105620] Updated weights for policy 1, policy_version 1291879 (0.0008) [2023-12-27 00:47:05,686][105620] Updated weights for policy 1, policy_version 1291889 (0.0008) [2023-12-27 00:47:05,744][105620] Updated weights for policy 1, policy_version 1291899 (0.0008) [2023-12-27 00:47:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.7, 300 sec: 19383.1). Total num frames: 661143552. Throughput: 0: 9636.1, 1: 10050.4. Samples: 661132584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:47:06,063][104569] Avg episode reward: [(0, '9358.715'), (1, '8623.477')] [2023-12-27 00:47:06,237][105692] Updated weights for policy 0, policy_version 1290327 (0.0009) [2023-12-27 00:47:06,296][105692] Updated weights for policy 0, policy_version 1290337 (0.0009) [2023-12-27 00:47:06,358][105692] Updated weights for policy 0, policy_version 1290347 (0.0007) [2023-12-27 00:47:06,535][105620] Updated weights for policy 1, policy_version 1291909 (0.0009) [2023-12-27 00:47:06,596][105620] Updated weights for policy 1, policy_version 1291919 (0.0009) [2023-12-27 00:47:06,654][105620] Updated weights for policy 1, policy_version 1291929 (0.0009) [2023-12-27 00:47:06,946][105692] Updated weights for policy 0, policy_version 1290357 (0.0009) [2023-12-27 00:47:06,999][105692] Updated weights for policy 0, policy_version 1290367 (0.0009) [2023-12-27 00:47:07,049][105692] Updated weights for policy 0, policy_version 1290377 (0.0009) [2023-12-27 00:47:07,530][105620] Updated weights for policy 1, policy_version 1291939 (0.0009) [2023-12-27 00:47:07,588][105620] Updated weights for policy 1, policy_version 1291949 (0.0009) [2023-12-27 00:47:07,649][105620] Updated weights for policy 1, policy_version 1291959 (0.0008) [2023-12-27 00:47:07,717][105692] Updated weights for policy 0, policy_version 1290387 (0.0009) [2023-12-27 00:47:07,782][105692] Updated weights for policy 0, policy_version 1290397 (0.0009) [2023-12-27 00:47:07,840][105692] Updated weights for policy 0, policy_version 1290407 (0.0006) [2023-12-27 00:47:08,402][105692] Updated weights for policy 0, policy_version 1290417 (0.0005) [2023-12-27 00:47:08,455][105692] Updated weights for policy 0, policy_version 1290427 (0.0008) [2023-12-27 00:47:08,479][105620] Updated weights for policy 1, policy_version 1291969 (0.0009) [2023-12-27 00:47:08,506][105692] Updated weights for policy 0, policy_version 1290437 (0.0007) [2023-12-27 00:47:08,541][105620] Updated weights for policy 1, policy_version 1291979 (0.0009) [2023-12-27 00:47:08,556][105692] Updated weights for policy 0, policy_version 1290447 (0.0007) [2023-12-27 00:47:08,606][105620] Updated weights for policy 1, policy_version 1291989 (0.0008) [2023-12-27 00:47:08,663][105620] Updated weights for policy 1, policy_version 1291999 (0.0009) [2023-12-27 00:47:09,272][105620] Updated weights for policy 1, policy_version 1292009 (0.0008) [2023-12-27 00:47:09,347][105620] Updated weights for policy 1, policy_version 1292019 (0.0009) [2023-12-27 00:47:09,372][105692] Updated weights for policy 0, policy_version 1290457 (0.0007) [2023-12-27 00:47:09,416][105620] Updated weights for policy 1, policy_version 1292029 (0.0009) [2023-12-27 00:47:09,434][105692] Updated weights for policy 0, policy_version 1290467 (0.0008) [2023-12-27 00:47:09,495][105692] Updated weights for policy 0, policy_version 1290477 (0.0008) [2023-12-27 00:47:10,192][105620] Updated weights for policy 1, policy_version 1292039 (0.0009) [2023-12-27 00:47:10,240][105692] Updated weights for policy 0, policy_version 1290487 (0.0008) [2023-12-27 00:47:10,243][105620] Updated weights for policy 1, policy_version 1292049 (0.0007) [2023-12-27 00:47:10,297][105620] Updated weights for policy 1, policy_version 1292059 (0.0006) [2023-12-27 00:47:10,302][105692] Updated weights for policy 0, policy_version 1290497 (0.0007) [2023-12-27 00:47:10,366][105692] Updated weights for policy 0, policy_version 1290507 (0.0008) [2023-12-27 00:47:11,025][105620] Updated weights for policy 1, policy_version 1292069 (0.0009) [2023-12-27 00:47:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19355.3). Total num frames: 661233664. Throughput: 0: 9728.0, 1: 9954.0. Samples: 661247728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:47:11,063][104569] Avg episode reward: [(0, '9180.077'), (1, '8622.493')] [2023-12-27 00:47:11,087][105620] Updated weights for policy 1, policy_version 1292079 (0.0010) [2023-12-27 00:47:11,154][105620] Updated weights for policy 1, policy_version 1292089 (0.0009) [2023-12-27 00:47:11,164][105692] Updated weights for policy 0, policy_version 1290517 (0.0008) [2023-12-27 00:47:11,222][105692] Updated weights for policy 0, policy_version 1290527 (0.0008) [2023-12-27 00:47:11,281][105692] Updated weights for policy 0, policy_version 1290537 (0.0008) [2023-12-27 00:47:11,957][105620] Updated weights for policy 1, policy_version 1292099 (0.0008) [2023-12-27 00:47:12,014][105620] Updated weights for policy 1, policy_version 1292109 (0.0010) [2023-12-27 00:47:12,052][105692] Updated weights for policy 0, policy_version 1290547 (0.0009) [2023-12-27 00:47:12,067][105620] Updated weights for policy 1, policy_version 1292119 (0.0008) [2023-12-27 00:47:12,102][105692] Updated weights for policy 0, policy_version 1290557 (0.0006) [2023-12-27 00:47:12,156][105692] Updated weights for policy 0, policy_version 1290567 (0.0008) [2023-12-27 00:47:12,789][105620] Updated weights for policy 1, policy_version 1292129 (0.0006) [2023-12-27 00:47:12,849][105620] Updated weights for policy 1, policy_version 1292139 (0.0009) [2023-12-27 00:47:12,904][105620] Updated weights for policy 1, policy_version 1292149 (0.0009) [2023-12-27 00:47:12,921][105692] Updated weights for policy 0, policy_version 1290577 (0.0006) [2023-12-27 00:47:12,956][105620] Updated weights for policy 1, policy_version 1292159 (0.0007) [2023-12-27 00:47:12,976][105692] Updated weights for policy 0, policy_version 1290587 (0.0007) [2023-12-27 00:47:13,038][105692] Updated weights for policy 0, policy_version 1290597 (0.0009) [2023-12-27 00:47:13,105][105692] Updated weights for policy 0, policy_version 1290607 (0.0009) [2023-12-27 00:47:13,762][105620] Updated weights for policy 1, policy_version 1292169 (0.0007) [2023-12-27 00:47:13,780][105692] Updated weights for policy 0, policy_version 1290617 (0.0007) [2023-12-27 00:47:13,815][105620] Updated weights for policy 1, policy_version 1292179 (0.0006) [2023-12-27 00:47:13,841][105692] Updated weights for policy 0, policy_version 1290627 (0.0009) [2023-12-27 00:47:13,873][105620] Updated weights for policy 1, policy_version 1292189 (0.0009) [2023-12-27 00:47:13,903][105692] Updated weights for policy 0, policy_version 1290637 (0.0005) [2023-12-27 00:47:14,446][105692] Updated weights for policy 0, policy_version 1290647 (0.0006) [2023-12-27 00:47:14,496][105692] Updated weights for policy 0, policy_version 1290657 (0.0009) [2023-12-27 00:47:14,544][105692] Updated weights for policy 0, policy_version 1290667 (0.0009) [2023-12-27 00:47:14,716][105620] Updated weights for policy 1, policy_version 1292199 (0.0007) [2023-12-27 00:47:14,778][105620] Updated weights for policy 1, policy_version 1292209 (0.0006) [2023-12-27 00:47:14,844][105620] Updated weights for policy 1, policy_version 1292219 (0.0008) [2023-12-27 00:47:15,312][105692] Updated weights for policy 0, policy_version 1290677 (0.0009) [2023-12-27 00:47:15,375][105692] Updated weights for policy 0, policy_version 1290687 (0.0009) [2023-12-27 00:47:15,422][105692] Updated weights for policy 0, policy_version 1290697 (0.0008) [2023-12-27 00:47:15,517][105620] Updated weights for policy 1, policy_version 1292229 (0.0009) [2023-12-27 00:47:15,566][105620] Updated weights for policy 1, policy_version 1292239 (0.0009) [2023-12-27 00:47:15,615][105620] Updated weights for policy 1, policy_version 1292249 (0.0009) [2023-12-27 00:47:16,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19355.3). Total num frames: 661331968. Throughput: 0: 9728.9, 1: 9827.7. Samples: 661302240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:47:16,063][104569] Avg episode reward: [(0, '9090.650'), (1, '8993.497')] [2023-12-27 00:47:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001290704_330473472.pth... [2023-12-27 00:47:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001292256_330858496.pth... [2023-12-27 00:47:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001289552_330178560.pth [2023-12-27 00:47:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001291136_330571776.pth [2023-12-27 00:47:16,167][105692] Updated weights for policy 0, policy_version 1290707 (0.0009) [2023-12-27 00:47:16,228][105692] Updated weights for policy 0, policy_version 1290717 (0.0009) [2023-12-27 00:47:16,276][105692] Updated weights for policy 0, policy_version 1290727 (0.0009) [2023-12-27 00:47:16,410][105620] Updated weights for policy 1, policy_version 1292259 (0.0009) [2023-12-27 00:47:16,470][105620] Updated weights for policy 1, policy_version 1292269 (0.0009) [2023-12-27 00:47:16,517][105620] Updated weights for policy 1, policy_version 1292279 (0.0008) [2023-12-27 00:47:16,986][105692] Updated weights for policy 0, policy_version 1290737 (0.0009) [2023-12-27 00:47:17,045][105692] Updated weights for policy 0, policy_version 1290747 (0.0006) [2023-12-27 00:47:17,105][105692] Updated weights for policy 0, policy_version 1290757 (0.0008) [2023-12-27 00:47:17,148][105620] Updated weights for policy 1, policy_version 1292289 (0.0008) [2023-12-27 00:47:17,155][105692] Updated weights for policy 0, policy_version 1290767 (0.0006) [2023-12-27 00:47:17,197][105620] Updated weights for policy 1, policy_version 1292299 (0.0005) [2023-12-27 00:47:17,243][105620] Updated weights for policy 1, policy_version 1292309 (0.0005) [2023-12-27 00:47:17,289][105620] Updated weights for policy 1, policy_version 1292319 (0.0005) [2023-12-27 00:47:17,791][105692] Updated weights for policy 0, policy_version 1290777 (0.0005) [2023-12-27 00:47:17,845][105692] Updated weights for policy 0, policy_version 1290787 (0.0005) [2023-12-27 00:47:17,895][105692] Updated weights for policy 0, policy_version 1290797 (0.0006) [2023-12-27 00:47:17,957][105620] Updated weights for policy 1, policy_version 1292329 (0.0005) [2023-12-27 00:47:18,018][105620] Updated weights for policy 1, policy_version 1292339 (0.0005) [2023-12-27 00:47:18,069][105620] Updated weights for policy 1, policy_version 1292349 (0.0009) [2023-12-27 00:47:18,470][105692] Updated weights for policy 0, policy_version 1290807 (0.0005) [2023-12-27 00:47:18,529][105692] Updated weights for policy 0, policy_version 1290817 (0.0009) [2023-12-27 00:47:18,589][105692] Updated weights for policy 0, policy_version 1290827 (0.0010) [2023-12-27 00:47:18,840][105620] Updated weights for policy 1, policy_version 1292360 (0.0009) [2023-12-27 00:47:18,899][105620] Updated weights for policy 1, policy_version 1292370 (0.0010) [2023-12-27 00:47:18,961][105620] Updated weights for policy 1, policy_version 1292380 (0.0009) [2023-12-27 00:47:19,257][105692] Updated weights for policy 0, policy_version 1290837 (0.0008) [2023-12-27 00:47:19,324][105692] Updated weights for policy 0, policy_version 1290847 (0.0009) [2023-12-27 00:47:19,392][105692] Updated weights for policy 0, policy_version 1290857 (0.0009) [2023-12-27 00:47:19,703][105620] Updated weights for policy 1, policy_version 1292390 (0.0010) [2023-12-27 00:47:19,778][105620] Updated weights for policy 1, policy_version 1292400 (0.0010) [2023-12-27 00:47:19,847][105620] Updated weights for policy 1, policy_version 1292410 (0.0009) [2023-12-27 00:47:20,190][105692] Updated weights for policy 0, policy_version 1290867 (0.0010) [2023-12-27 00:47:20,252][105692] Updated weights for policy 0, policy_version 1290877 (0.0009) [2023-12-27 00:47:20,314][105692] Updated weights for policy 0, policy_version 1290887 (0.0009) [2023-12-27 00:47:20,534][105620] Updated weights for policy 1, policy_version 1292420 (0.0009) [2023-12-27 00:47:20,594][105620] Updated weights for policy 1, policy_version 1292430 (0.0009) [2023-12-27 00:47:20,649][105620] Updated weights for policy 1, policy_version 1292440 (0.0008) [2023-12-27 00:47:21,056][105692] Updated weights for policy 0, policy_version 1290897 (0.0009) [2023-12-27 00:47:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19355.3). Total num frames: 661430272. Throughput: 0: 9745.2, 1: 9782.7. Samples: 661421668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:47:21,063][104569] Avg episode reward: [(0, '9178.480'), (1, '9084.680')] [2023-12-27 00:47:21,112][105692] Updated weights for policy 0, policy_version 1290907 (0.0009) [2023-12-27 00:47:21,175][105692] Updated weights for policy 0, policy_version 1290917 (0.0009) [2023-12-27 00:47:21,234][105692] Updated weights for policy 0, policy_version 1290927 (0.0009) [2023-12-27 00:47:21,443][105620] Updated weights for policy 1, policy_version 1292450 (0.0009) [2023-12-27 00:47:21,502][105620] Updated weights for policy 1, policy_version 1292460 (0.0009) [2023-12-27 00:47:21,564][105620] Updated weights for policy 1, policy_version 1292470 (0.0009) [2023-12-27 00:47:21,628][105620] Updated weights for policy 1, policy_version 1292480 (0.0008) [2023-12-27 00:47:22,076][105692] Updated weights for policy 0, policy_version 1290937 (0.0009) [2023-12-27 00:47:22,137][105692] Updated weights for policy 0, policy_version 1290947 (0.0005) [2023-12-27 00:47:22,199][105692] Updated weights for policy 0, policy_version 1290957 (0.0009) [2023-12-27 00:47:22,286][105620] Updated weights for policy 1, policy_version 1292490 (0.0008) [2023-12-27 00:47:22,348][105620] Updated weights for policy 1, policy_version 1292500 (0.0007) [2023-12-27 00:47:22,419][105620] Updated weights for policy 1, policy_version 1292510 (0.0009) [2023-12-27 00:47:22,864][105692] Updated weights for policy 0, policy_version 1290967 (0.0007) [2023-12-27 00:47:22,917][105692] Updated weights for policy 0, policy_version 1290977 (0.0009) [2023-12-27 00:47:22,969][105692] Updated weights for policy 0, policy_version 1290987 (0.0009) [2023-12-27 00:47:23,090][105620] Updated weights for policy 1, policy_version 1292520 (0.0007) [2023-12-27 00:47:23,158][105620] Updated weights for policy 1, policy_version 1292530 (0.0006) [2023-12-27 00:47:23,219][105620] Updated weights for policy 1, policy_version 1292540 (0.0006) [2023-12-27 00:47:23,730][105620] Updated weights for policy 1, policy_version 1292550 (0.0008) [2023-12-27 00:47:23,786][105620] Updated weights for policy 1, policy_version 1292560 (0.0005) [2023-12-27 00:47:23,803][105692] Updated weights for policy 0, policy_version 1290997 (0.0006) [2023-12-27 00:47:23,837][105620] Updated weights for policy 1, policy_version 1292570 (0.0005) [2023-12-27 00:47:23,860][105692] Updated weights for policy 0, policy_version 1291007 (0.0006) [2023-12-27 00:47:23,923][105692] Updated weights for policy 0, policy_version 1291017 (0.0009) [2023-12-27 00:47:24,387][105620] Updated weights for policy 1, policy_version 1292580 (0.0007) [2023-12-27 00:47:24,448][105620] Updated weights for policy 1, policy_version 1292590 (0.0009) [2023-12-27 00:47:24,507][105620] Updated weights for policy 1, policy_version 1292600 (0.0009) [2023-12-27 00:47:24,696][105692] Updated weights for policy 0, policy_version 1291027 (0.0010) [2023-12-27 00:47:24,749][105692] Updated weights for policy 0, policy_version 1291037 (0.0009) [2023-12-27 00:47:24,810][105692] Updated weights for policy 0, policy_version 1291047 (0.0009) [2023-12-27 00:47:25,214][105620] Updated weights for policy 1, policy_version 1292610 (0.0009) [2023-12-27 00:47:25,268][105620] Updated weights for policy 1, policy_version 1292620 (0.0010) [2023-12-27 00:47:25,320][105620] Updated weights for policy 1, policy_version 1292630 (0.0010) [2023-12-27 00:47:25,372][105620] Updated weights for policy 1, policy_version 1292640 (0.0010) [2023-12-27 00:47:25,626][105692] Updated weights for policy 0, policy_version 1291057 (0.0008) [2023-12-27 00:47:25,695][105692] Updated weights for policy 0, policy_version 1291067 (0.0007) [2023-12-27 00:47:25,761][105692] Updated weights for policy 0, policy_version 1291077 (0.0008) [2023-12-27 00:47:25,815][105692] Updated weights for policy 0, policy_version 1291087 (0.0008) [2023-12-27 00:47:26,032][105620] Updated weights for policy 1, policy_version 1292650 (0.0006) [2023-12-27 00:47:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.4, 300 sec: 19355.3). Total num frames: 661528576. Throughput: 0: 9664.0, 1: 9826.9. Samples: 661539256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:47:26,062][104569] Avg episode reward: [(0, '8997.409'), (1, '9169.075')] [2023-12-27 00:47:26,081][105620] Updated weights for policy 1, policy_version 1292660 (0.0005) [2023-12-27 00:47:26,137][105620] Updated weights for policy 1, policy_version 1292670 (0.0008) [2023-12-27 00:47:26,559][105692] Updated weights for policy 0, policy_version 1291097 (0.0008) [2023-12-27 00:47:26,607][105692] Updated weights for policy 0, policy_version 1291107 (0.0008) [2023-12-27 00:47:26,650][105692] Updated weights for policy 0, policy_version 1291117 (0.0007) [2023-12-27 00:47:26,841][105620] Updated weights for policy 1, policy_version 1292680 (0.0009) [2023-12-27 00:47:26,892][105620] Updated weights for policy 1, policy_version 1292690 (0.0010) [2023-12-27 00:47:26,956][105620] Updated weights for policy 1, policy_version 1292700 (0.0010) [2023-12-27 00:47:27,383][105692] Updated weights for policy 0, policy_version 1291128 (0.0009) [2023-12-27 00:47:27,430][105692] Updated weights for policy 0, policy_version 1291138 (0.0008) [2023-12-27 00:47:27,487][105692] Updated weights for policy 0, policy_version 1291149 (0.0010) [2023-12-27 00:47:27,547][105620] Updated weights for policy 1, policy_version 1292710 (0.0007) [2023-12-27 00:47:27,597][105620] Updated weights for policy 1, policy_version 1292720 (0.0010) [2023-12-27 00:47:27,654][105620] Updated weights for policy 1, policy_version 1292730 (0.0010) [2023-12-27 00:47:28,218][105620] Updated weights for policy 1, policy_version 1292740 (0.0008) [2023-12-27 00:47:28,262][105620] Updated weights for policy 1, policy_version 1292750 (0.0005) [2023-12-27 00:47:28,314][105620] Updated weights for policy 1, policy_version 1292760 (0.0010) [2023-12-27 00:47:28,366][105692] Updated weights for policy 0, policy_version 1291159 (0.0009) [2023-12-27 00:47:28,424][105692] Updated weights for policy 0, policy_version 1291169 (0.0007) [2023-12-27 00:47:28,485][105692] Updated weights for policy 0, policy_version 1291179 (0.0008) [2023-12-27 00:47:29,070][105620] Updated weights for policy 1, policy_version 1292770 (0.0010) [2023-12-27 00:47:29,123][105620] Updated weights for policy 1, policy_version 1292780 (0.0009) [2023-12-27 00:47:29,169][105620] Updated weights for policy 1, policy_version 1292790 (0.0009) [2023-12-27 00:47:29,224][105620] Updated weights for policy 1, policy_version 1292800 (0.0009) [2023-12-27 00:47:29,237][105692] Updated weights for policy 0, policy_version 1291189 (0.0009) [2023-12-27 00:47:29,293][105692] Updated weights for policy 0, policy_version 1291199 (0.0006) [2023-12-27 00:47:29,344][105692] Updated weights for policy 0, policy_version 1291209 (0.0007) [2023-12-27 00:47:29,929][105692] Updated weights for policy 0, policy_version 1291219 (0.0008) [2023-12-27 00:47:29,990][105692] Updated weights for policy 0, policy_version 1291229 (0.0008) [2023-12-27 00:47:30,044][105692] Updated weights for policy 0, policy_version 1291239 (0.0009) [2023-12-27 00:47:30,116][105620] Updated weights for policy 1, policy_version 1292810 (0.0009) [2023-12-27 00:47:30,166][105620] Updated weights for policy 1, policy_version 1292820 (0.0008) [2023-12-27 00:47:30,213][105620] Updated weights for policy 1, policy_version 1292830 (0.0009) [2023-12-27 00:47:30,727][105692] Updated weights for policy 0, policy_version 1291249 (0.0006) [2023-12-27 00:47:30,773][105692] Updated weights for policy 0, policy_version 1291259 (0.0005) [2023-12-27 00:47:30,834][105692] Updated weights for policy 0, policy_version 1291269 (0.0006) [2023-12-27 00:47:30,893][105692] Updated weights for policy 0, policy_version 1291279 (0.0005) [2023-12-27 00:47:30,976][105620] Updated weights for policy 1, policy_version 1292840 (0.0008) [2023-12-27 00:47:31,046][105620] Updated weights for policy 1, policy_version 1292850 (0.0008) [2023-12-27 00:47:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19355.3). Total num frames: 661626880. Throughput: 0: 9670.2, 1: 9833.9. Samples: 661598580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:47:31,063][104569] Avg episode reward: [(0, '8909.229'), (1, '9169.165')] [2023-12-27 00:47:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001291280_330620928.pth... [2023-12-27 00:47:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001290128_330326016.pth [2023-12-27 00:47:31,107][105620] Updated weights for policy 1, policy_version 1292860 (0.0008) [2023-12-27 00:47:31,127][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001292864_331014144.pth... [2023-12-27 00:47:31,131][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001291712_330719232.pth [2023-12-27 00:47:31,529][105692] Updated weights for policy 0, policy_version 1291289 (0.0009) [2023-12-27 00:47:31,585][105692] Updated weights for policy 0, policy_version 1291299 (0.0008) [2023-12-27 00:47:31,651][105692] Updated weights for policy 0, policy_version 1291309 (0.0010) [2023-12-27 00:47:31,793][105620] Updated weights for policy 1, policy_version 1292870 (0.0009) [2023-12-27 00:47:31,854][105620] Updated weights for policy 1, policy_version 1292880 (0.0010) [2023-12-27 00:47:31,920][105620] Updated weights for policy 1, policy_version 1292891 (0.0009) [2023-12-27 00:47:32,484][105692] Updated weights for policy 0, policy_version 1291319 (0.0009) [2023-12-27 00:47:32,543][105692] Updated weights for policy 0, policy_version 1291329 (0.0009) [2023-12-27 00:47:32,579][105620] Updated weights for policy 1, policy_version 1292901 (0.0008) [2023-12-27 00:47:32,593][105692] Updated weights for policy 0, policy_version 1291339 (0.0008) [2023-12-27 00:47:32,631][105620] Updated weights for policy 1, policy_version 1292911 (0.0006) [2023-12-27 00:47:32,683][105620] Updated weights for policy 1, policy_version 1292921 (0.0009) [2023-12-27 00:47:33,309][105620] Updated weights for policy 1, policy_version 1292931 (0.0008) [2023-12-27 00:47:33,359][105620] Updated weights for policy 1, policy_version 1292941 (0.0009) [2023-12-27 00:47:33,404][105692] Updated weights for policy 0, policy_version 1291349 (0.0006) [2023-12-27 00:47:33,410][105620] Updated weights for policy 1, policy_version 1292951 (0.0010) [2023-12-27 00:47:33,461][105692] Updated weights for policy 0, policy_version 1291359 (0.0006) [2023-12-27 00:47:33,519][105692] Updated weights for policy 0, policy_version 1291369 (0.0011) [2023-12-27 00:47:34,046][105620] Updated weights for policy 1, policy_version 1292961 (0.0009) [2023-12-27 00:47:34,102][105620] Updated weights for policy 1, policy_version 1292971 (0.0006) [2023-12-27 00:47:34,165][105620] Updated weights for policy 1, policy_version 1292981 (0.0007) [2023-12-27 00:47:34,230][105620] Updated weights for policy 1, policy_version 1292991 (0.0006) [2023-12-27 00:47:34,343][105692] Updated weights for policy 0, policy_version 1291380 (0.0011) [2023-12-27 00:47:34,405][105692] Updated weights for policy 0, policy_version 1291390 (0.0010) [2023-12-27 00:47:34,463][105692] Updated weights for policy 0, policy_version 1291400 (0.0009) [2023-12-27 00:47:34,841][105620] Updated weights for policy 1, policy_version 1293001 (0.0006) [2023-12-27 00:47:34,892][105620] Updated weights for policy 1, policy_version 1293011 (0.0005) [2023-12-27 00:47:34,943][105620] Updated weights for policy 1, policy_version 1293021 (0.0005) [2023-12-27 00:47:35,239][105692] Updated weights for policy 0, policy_version 1291410 (0.0009) [2023-12-27 00:47:35,293][105692] Updated weights for policy 0, policy_version 1291420 (0.0009) [2023-12-27 00:47:35,341][105692] Updated weights for policy 0, policy_version 1291430 (0.0008) [2023-12-27 00:47:35,390][105692] Updated weights for policy 0, policy_version 1291440 (0.0008) [2023-12-27 00:47:35,578][105620] Updated weights for policy 1, policy_version 1293031 (0.0009) [2023-12-27 00:47:35,637][105620] Updated weights for policy 1, policy_version 1293041 (0.0010) [2023-12-27 00:47:35,687][105620] Updated weights for policy 1, policy_version 1293051 (0.0009) [2023-12-27 00:47:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 661725184. Throughput: 0: 9695.2, 1: 9806.0. Samples: 661716036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:47:36,062][104569] Avg episode reward: [(0, '9181.094'), (1, '9171.678')] [2023-12-27 00:47:36,179][105692] Updated weights for policy 0, policy_version 1291450 (0.0008) [2023-12-27 00:47:36,236][105692] Updated weights for policy 0, policy_version 1291460 (0.0008) [2023-12-27 00:47:36,296][105692] Updated weights for policy 0, policy_version 1291470 (0.0008) [2023-12-27 00:47:36,420][105620] Updated weights for policy 1, policy_version 1293061 (0.0005) [2023-12-27 00:47:36,485][105620] Updated weights for policy 1, policy_version 1293071 (0.0008) [2023-12-27 00:47:36,539][105620] Updated weights for policy 1, policy_version 1293081 (0.0008) [2023-12-27 00:47:37,132][105692] Updated weights for policy 0, policy_version 1291480 (0.0009) [2023-12-27 00:47:37,161][105620] Updated weights for policy 1, policy_version 1293091 (0.0007) [2023-12-27 00:47:37,190][105692] Updated weights for policy 0, policy_version 1291490 (0.0008) [2023-12-27 00:47:37,226][105620] Updated weights for policy 1, policy_version 1293101 (0.0007) [2023-12-27 00:47:37,244][105692] Updated weights for policy 0, policy_version 1291500 (0.0006) [2023-12-27 00:47:37,279][105620] Updated weights for policy 1, policy_version 1293111 (0.0007) [2023-12-27 00:47:37,931][105692] Updated weights for policy 0, policy_version 1291510 (0.0008) [2023-12-27 00:47:37,980][105692] Updated weights for policy 0, policy_version 1291520 (0.0005) [2023-12-27 00:47:38,026][105620] Updated weights for policy 1, policy_version 1293121 (0.0009) [2023-12-27 00:47:38,037][105692] Updated weights for policy 0, policy_version 1291530 (0.0007) [2023-12-27 00:47:38,078][105620] Updated weights for policy 1, policy_version 1293131 (0.0010) [2023-12-27 00:47:38,122][105620] Updated weights for policy 1, policy_version 1293141 (0.0010) [2023-12-27 00:47:38,166][105620] Updated weights for policy 1, policy_version 1293151 (0.0010) [2023-12-27 00:47:38,783][105692] Updated weights for policy 0, policy_version 1291540 (0.0010) [2023-12-27 00:47:38,845][105692] Updated weights for policy 0, policy_version 1291550 (0.0011) [2023-12-27 00:47:38,904][105692] Updated weights for policy 0, policy_version 1291560 (0.0011) [2023-12-27 00:47:38,905][105620] Updated weights for policy 1, policy_version 1293161 (0.0011) [2023-12-27 00:47:38,957][105620] Updated weights for policy 1, policy_version 1293171 (0.0011) [2023-12-27 00:47:39,019][105620] Updated weights for policy 1, policy_version 1293181 (0.0010) [2023-12-27 00:47:39,651][105692] Updated weights for policy 0, policy_version 1291570 (0.0010) [2023-12-27 00:47:39,704][105692] Updated weights for policy 0, policy_version 1291580 (0.0010) [2023-12-27 00:47:39,706][105620] Updated weights for policy 1, policy_version 1293191 (0.0007) [2023-12-27 00:47:39,761][105692] Updated weights for policy 0, policy_version 1291590 (0.0011) [2023-12-27 00:47:39,765][105620] Updated weights for policy 1, policy_version 1293201 (0.0006) [2023-12-27 00:47:39,820][105620] Updated weights for policy 1, policy_version 1293211 (0.0006) [2023-12-27 00:47:39,825][105692] Updated weights for policy 0, policy_version 1291600 (0.0011) [2023-12-27 00:47:40,477][105620] Updated weights for policy 1, policy_version 1293221 (0.0008) [2023-12-27 00:47:40,532][105620] Updated weights for policy 1, policy_version 1293231 (0.0008) [2023-12-27 00:47:40,566][105692] Updated weights for policy 0, policy_version 1291610 (0.0011) [2023-12-27 00:47:40,591][105620] Updated weights for policy 1, policy_version 1293241 (0.0008) [2023-12-27 00:47:40,624][105692] Updated weights for policy 0, policy_version 1291620 (0.0007) [2023-12-27 00:47:40,679][105692] Updated weights for policy 0, policy_version 1291630 (0.0005) [2023-12-27 00:47:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 661823488. Throughput: 0: 9630.2, 1: 9854.1. Samples: 661831500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:47:41,062][104569] Avg episode reward: [(0, '9267.120'), (1, '9263.481')] [2023-12-27 00:47:41,330][105692] Updated weights for policy 0, policy_version 1291640 (0.0006) [2023-12-27 00:47:41,394][105692] Updated weights for policy 0, policy_version 1291650 (0.0008) [2023-12-27 00:47:41,442][105692] Updated weights for policy 0, policy_version 1291660 (0.0008) [2023-12-27 00:47:41,458][105620] Updated weights for policy 1, policy_version 1293251 (0.0008) [2023-12-27 00:47:41,514][105620] Updated weights for policy 1, policy_version 1293261 (0.0009) [2023-12-27 00:47:41,571][105620] Updated weights for policy 1, policy_version 1293271 (0.0008) [2023-12-27 00:47:42,235][105620] Updated weights for policy 1, policy_version 1293281 (0.0011) [2023-12-27 00:47:42,249][105692] Updated weights for policy 0, policy_version 1291670 (0.0007) [2023-12-27 00:47:42,298][105620] Updated weights for policy 1, policy_version 1293291 (0.0009) [2023-12-27 00:47:42,319][105692] Updated weights for policy 0, policy_version 1291680 (0.0009) [2023-12-27 00:47:42,360][105620] Updated weights for policy 1, policy_version 1293301 (0.0009) [2023-12-27 00:47:42,390][105692] Updated weights for policy 0, policy_version 1291690 (0.0006) [2023-12-27 00:47:42,427][105620] Updated weights for policy 1, policy_version 1293311 (0.0009) [2023-12-27 00:47:43,134][105692] Updated weights for policy 0, policy_version 1291700 (0.0007) [2023-12-27 00:47:43,175][105620] Updated weights for policy 1, policy_version 1293321 (0.0010) [2023-12-27 00:47:43,186][105692] Updated weights for policy 0, policy_version 1291710 (0.0006) [2023-12-27 00:47:43,237][105620] Updated weights for policy 1, policy_version 1293331 (0.0010) [2023-12-27 00:47:43,241][105692] Updated weights for policy 0, policy_version 1291720 (0.0008) [2023-12-27 00:47:43,284][105620] Updated weights for policy 1, policy_version 1293341 (0.0010) [2023-12-27 00:47:44,009][105692] Updated weights for policy 0, policy_version 1291730 (0.0009) [2023-12-27 00:47:44,039][105620] Updated weights for policy 1, policy_version 1293351 (0.0010) [2023-12-27 00:47:44,061][105692] Updated weights for policy 0, policy_version 1291740 (0.0005) [2023-12-27 00:47:44,094][105620] Updated weights for policy 1, policy_version 1293361 (0.0011) [2023-12-27 00:47:44,117][105692] Updated weights for policy 0, policy_version 1291750 (0.0008) [2023-12-27 00:47:44,156][105620] Updated weights for policy 1, policy_version 1293371 (0.0010) [2023-12-27 00:47:44,178][105692] Updated weights for policy 0, policy_version 1291760 (0.0009) [2023-12-27 00:47:44,881][105620] Updated weights for policy 1, policy_version 1293381 (0.0010) [2023-12-27 00:47:44,920][105692] Updated weights for policy 0, policy_version 1291770 (0.0006) [2023-12-27 00:47:44,937][105620] Updated weights for policy 1, policy_version 1293391 (0.0011) [2023-12-27 00:47:44,984][105692] Updated weights for policy 0, policy_version 1291780 (0.0005) [2023-12-27 00:47:44,997][105620] Updated weights for policy 1, policy_version 1293401 (0.0011) [2023-12-27 00:47:45,053][105692] Updated weights for policy 0, policy_version 1291790 (0.0006) [2023-12-27 00:47:45,758][105620] Updated weights for policy 1, policy_version 1293411 (0.0011) [2023-12-27 00:47:45,805][105620] Updated weights for policy 1, policy_version 1293421 (0.0010) [2023-12-27 00:47:45,811][105692] Updated weights for policy 0, policy_version 1291800 (0.0006) [2023-12-27 00:47:45,860][105620] Updated weights for policy 1, policy_version 1293431 (0.0010) [2023-12-27 00:47:45,870][105692] Updated weights for policy 0, policy_version 1291810 (0.0006) [2023-12-27 00:47:45,928][105692] Updated weights for policy 0, policy_version 1291820 (0.0009) [2023-12-27 00:47:46,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19524.1, 300 sec: 19383.1). Total num frames: 661921792. Throughput: 0: 9656.7, 1: 9794.5. Samples: 661887756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:47:46,063][104569] Avg episode reward: [(0, '9266.124'), (1, '9352.287')] [2023-12-27 00:47:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001291824_330760192.pth... [2023-12-27 00:47:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001293440_331161600.pth... [2023-12-27 00:47:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001290704_330473472.pth [2023-12-27 00:47:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001292256_330858496.pth [2023-12-27 00:47:46,455][105620] Updated weights for policy 1, policy_version 1293441 (0.0006) [2023-12-27 00:47:46,524][105620] Updated weights for policy 1, policy_version 1293451 (0.0008) [2023-12-27 00:47:46,588][105620] Updated weights for policy 1, policy_version 1293461 (0.0009) [2023-12-27 00:47:46,651][105620] Updated weights for policy 1, policy_version 1293471 (0.0008) [2023-12-27 00:47:46,669][105692] Updated weights for policy 0, policy_version 1291830 (0.0010) [2023-12-27 00:47:46,732][105692] Updated weights for policy 0, policy_version 1291840 (0.0010) [2023-12-27 00:47:46,791][105692] Updated weights for policy 0, policy_version 1291850 (0.0010) [2023-12-27 00:47:47,439][105620] Updated weights for policy 1, policy_version 1293481 (0.0008) [2023-12-27 00:47:47,466][105692] Updated weights for policy 0, policy_version 1291860 (0.0009) [2023-12-27 00:47:47,501][105620] Updated weights for policy 1, policy_version 1293491 (0.0009) [2023-12-27 00:47:47,521][105692] Updated weights for policy 0, policy_version 1291870 (0.0005) [2023-12-27 00:47:47,555][105620] Updated weights for policy 1, policy_version 1293501 (0.0009) [2023-12-27 00:47:47,578][105692] Updated weights for policy 0, policy_version 1291880 (0.0010) [2023-12-27 00:47:48,159][105620] Updated weights for policy 1, policy_version 1293511 (0.0009) [2023-12-27 00:47:48,226][105620] Updated weights for policy 1, policy_version 1293521 (0.0011) [2023-12-27 00:47:48,289][105620] Updated weights for policy 1, policy_version 1293531 (0.0011) [2023-12-27 00:47:48,290][105692] Updated weights for policy 0, policy_version 1291890 (0.0009) [2023-12-27 00:47:48,345][105692] Updated weights for policy 0, policy_version 1291900 (0.0006) [2023-12-27 00:47:48,412][105692] Updated weights for policy 0, policy_version 1291910 (0.0008) [2023-12-27 00:47:48,474][105692] Updated weights for policy 0, policy_version 1291920 (0.0008) [2023-12-27 00:47:49,027][105620] Updated weights for policy 1, policy_version 1293541 (0.0009) [2023-12-27 00:47:49,085][105620] Updated weights for policy 1, policy_version 1293551 (0.0009) [2023-12-27 00:47:49,134][105620] Updated weights for policy 1, policy_version 1293561 (0.0009) [2023-12-27 00:47:49,210][105692] Updated weights for policy 0, policy_version 1291930 (0.0010) [2023-12-27 00:47:49,278][105692] Updated weights for policy 0, policy_version 1291940 (0.0009) [2023-12-27 00:47:49,342][105692] Updated weights for policy 0, policy_version 1291950 (0.0009) [2023-12-27 00:47:49,927][105620] Updated weights for policy 1, policy_version 1293571 (0.0007) [2023-12-27 00:47:49,992][105620] Updated weights for policy 1, policy_version 1293581 (0.0009) [2023-12-27 00:47:50,043][105620] Updated weights for policy 1, policy_version 1293591 (0.0009) [2023-12-27 00:47:50,100][105692] Updated weights for policy 0, policy_version 1291960 (0.0007) [2023-12-27 00:47:50,146][105692] Updated weights for policy 0, policy_version 1291970 (0.0007) [2023-12-27 00:47:50,191][105692] Updated weights for policy 0, policy_version 1291980 (0.0007) [2023-12-27 00:47:50,724][105620] Updated weights for policy 1, policy_version 1293601 (0.0009) [2023-12-27 00:47:50,787][105620] Updated weights for policy 1, policy_version 1293611 (0.0009) [2023-12-27 00:47:50,851][105620] Updated weights for policy 1, policy_version 1293621 (0.0009) [2023-12-27 00:47:50,911][105620] Updated weights for policy 1, policy_version 1293631 (0.0009) [2023-12-27 00:47:50,980][105692] Updated weights for policy 0, policy_version 1291990 (0.0010) [2023-12-27 00:47:51,043][105692] Updated weights for policy 0, policy_version 1292000 (0.0007) [2023-12-27 00:47:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 662011904. Throughput: 0: 9538.3, 1: 9791.4. Samples: 662002416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:47:51,062][104569] Avg episode reward: [(0, '9177.063'), (1, '9352.168')] [2023-12-27 00:47:51,100][105692] Updated weights for policy 0, policy_version 1292010 (0.0008) [2023-12-27 00:47:51,728][105620] Updated weights for policy 1, policy_version 1293641 (0.0009) [2023-12-27 00:47:51,786][105620] Updated weights for policy 1, policy_version 1293651 (0.0009) [2023-12-27 00:47:51,833][105620] Updated weights for policy 1, policy_version 1293661 (0.0008) [2023-12-27 00:47:51,852][105692] Updated weights for policy 0, policy_version 1292020 (0.0008) [2023-12-27 00:47:51,899][105692] Updated weights for policy 0, policy_version 1292030 (0.0009) [2023-12-27 00:47:51,947][105692] Updated weights for policy 0, policy_version 1292040 (0.0008) [2023-12-27 00:47:52,584][105620] Updated weights for policy 1, policy_version 1293671 (0.0006) [2023-12-27 00:47:52,616][105692] Updated weights for policy 0, policy_version 1292050 (0.0009) [2023-12-27 00:47:52,652][105620] Updated weights for policy 1, policy_version 1293681 (0.0007) [2023-12-27 00:47:52,681][105692] Updated weights for policy 0, policy_version 1292060 (0.0008) [2023-12-27 00:47:52,711][105620] Updated weights for policy 1, policy_version 1293691 (0.0007) [2023-12-27 00:47:52,746][105692] Updated weights for policy 0, policy_version 1292070 (0.0006) [2023-12-27 00:47:52,809][105692] Updated weights for policy 0, policy_version 1292080 (0.0008) [2023-12-27 00:47:53,278][105620] Updated weights for policy 1, policy_version 1293701 (0.0006) [2023-12-27 00:47:53,345][105620] Updated weights for policy 1, policy_version 1293711 (0.0005) [2023-12-27 00:47:53,395][105620] Updated weights for policy 1, policy_version 1293721 (0.0005) [2023-12-27 00:47:53,589][105692] Updated weights for policy 0, policy_version 1292090 (0.0006) [2023-12-27 00:47:53,648][105692] Updated weights for policy 0, policy_version 1292100 (0.0007) [2023-12-27 00:47:53,692][105692] Updated weights for policy 0, policy_version 1292110 (0.0010) [2023-12-27 00:47:54,062][105620] Updated weights for policy 1, policy_version 1293731 (0.0007) [2023-12-27 00:47:54,126][105620] Updated weights for policy 1, policy_version 1293742 (0.0009) [2023-12-27 00:47:54,172][105620] Updated weights for policy 1, policy_version 1293752 (0.0008) [2023-12-27 00:47:54,345][105692] Updated weights for policy 0, policy_version 1292120 (0.0009) [2023-12-27 00:47:54,397][105692] Updated weights for policy 0, policy_version 1292130 (0.0009) [2023-12-27 00:47:54,449][105692] Updated weights for policy 0, policy_version 1292140 (0.0009) [2023-12-27 00:47:54,864][105620] Updated weights for policy 1, policy_version 1293762 (0.0009) [2023-12-27 00:47:54,917][105620] Updated weights for policy 1, policy_version 1293772 (0.0010) [2023-12-27 00:47:54,970][105620] Updated weights for policy 1, policy_version 1293783 (0.0010) [2023-12-27 00:47:55,047][105692] Updated weights for policy 0, policy_version 1292150 (0.0007) [2023-12-27 00:47:55,114][105692] Updated weights for policy 0, policy_version 1292160 (0.0006) [2023-12-27 00:47:55,173][105692] Updated weights for policy 0, policy_version 1292170 (0.0009) [2023-12-27 00:47:55,766][105620] Updated weights for policy 1, policy_version 1293793 (0.0010) [2023-12-27 00:47:55,832][105620] Updated weights for policy 1, policy_version 1293803 (0.0009) [2023-12-27 00:47:55,885][105620] Updated weights for policy 1, policy_version 1293813 (0.0010) [2023-12-27 00:47:55,903][105692] Updated weights for policy 0, policy_version 1292180 (0.0008) [2023-12-27 00:47:55,938][105620] Updated weights for policy 1, policy_version 1293824 (0.0008) [2023-12-27 00:47:55,958][105692] Updated weights for policy 0, policy_version 1292190 (0.0005) [2023-12-27 00:47:56,011][105692] Updated weights for policy 0, policy_version 1292200 (0.0005) [2023-12-27 00:47:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 662118400. Throughput: 0: 9508.8, 1: 9866.6. Samples: 662119620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:47:56,063][104569] Avg episode reward: [(0, '9177.273'), (1, '9352.137')] [2023-12-27 00:47:56,527][105692] Updated weights for policy 0, policy_version 1292210 (0.0005) [2023-12-27 00:47:56,588][105692] Updated weights for policy 0, policy_version 1292220 (0.0008) [2023-12-27 00:47:56,650][105692] Updated weights for policy 0, policy_version 1292230 (0.0009) [2023-12-27 00:47:56,701][105692] Updated weights for policy 0, policy_version 1292240 (0.0009) [2023-12-27 00:47:56,790][105620] Updated weights for policy 1, policy_version 1293834 (0.0009) [2023-12-27 00:47:56,838][105620] Updated weights for policy 1, policy_version 1293844 (0.0009) [2023-12-27 00:47:56,891][105620] Updated weights for policy 1, policy_version 1293856 (0.0010) [2023-12-27 00:47:57,285][105692] Updated weights for policy 0, policy_version 1292250 (0.0005) [2023-12-27 00:47:57,340][105692] Updated weights for policy 0, policy_version 1292260 (0.0006) [2023-12-27 00:47:57,402][105692] Updated weights for policy 0, policy_version 1292270 (0.0007) [2023-12-27 00:47:57,765][105620] Updated weights for policy 1, policy_version 1293866 (0.0010) [2023-12-27 00:47:57,814][105620] Updated weights for policy 1, policy_version 1293876 (0.0006) [2023-12-27 00:47:57,867][105620] Updated weights for policy 1, policy_version 1293886 (0.0005) [2023-12-27 00:47:57,985][105692] Updated weights for policy 0, policy_version 1292280 (0.0006) [2023-12-27 00:47:58,048][105692] Updated weights for policy 0, policy_version 1292290 (0.0008) [2023-12-27 00:47:58,098][105692] Updated weights for policy 0, policy_version 1292300 (0.0010) [2023-12-27 00:47:58,531][105620] Updated weights for policy 1, policy_version 1293896 (0.0008) [2023-12-27 00:47:58,604][105620] Updated weights for policy 1, policy_version 1293906 (0.0008) [2023-12-27 00:47:58,673][105620] Updated weights for policy 1, policy_version 1293916 (0.0008) [2023-12-27 00:47:58,775][105692] Updated weights for policy 0, policy_version 1292310 (0.0011) [2023-12-27 00:47:58,842][105692] Updated weights for policy 0, policy_version 1292320 (0.0009) [2023-12-27 00:47:58,915][105692] Updated weights for policy 0, policy_version 1292330 (0.0008) [2023-12-27 00:47:59,489][105620] Updated weights for policy 1, policy_version 1293926 (0.0010) [2023-12-27 00:47:59,542][105620] Updated weights for policy 1, policy_version 1293936 (0.0009) [2023-12-27 00:47:59,596][105620] Updated weights for policy 1, policy_version 1293946 (0.0008) [2023-12-27 00:47:59,690][105692] Updated weights for policy 0, policy_version 1292340 (0.0009) [2023-12-27 00:47:59,752][105692] Updated weights for policy 0, policy_version 1292350 (0.0009) [2023-12-27 00:47:59,810][105692] Updated weights for policy 0, policy_version 1292360 (0.0009) [2023-12-27 00:48:00,399][105620] Updated weights for policy 1, policy_version 1293956 (0.0010) [2023-12-27 00:48:00,459][105620] Updated weights for policy 1, policy_version 1293966 (0.0010) [2023-12-27 00:48:00,528][105620] Updated weights for policy 1, policy_version 1293976 (0.0009) [2023-12-27 00:48:00,530][105692] Updated weights for policy 0, policy_version 1292370 (0.0009) [2023-12-27 00:48:00,589][105692] Updated weights for policy 0, policy_version 1292380 (0.0007) [2023-12-27 00:48:00,648][105692] Updated weights for policy 0, policy_version 1292390 (0.0009) [2023-12-27 00:48:00,710][105692] Updated weights for policy 0, policy_version 1292400 (0.0009) [2023-12-27 00:48:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 662208512. Throughput: 0: 9653.3, 1: 9854.8. Samples: 662180100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:48:01,062][104569] Avg episode reward: [(0, '9083.070'), (1, '9352.041')] [2023-12-27 00:48:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001292400_330907648.pth... [2023-12-27 00:48:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001293984_331300864.pth... [2023-12-27 00:48:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001292864_331014144.pth [2023-12-27 00:48:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001291280_330620928.pth [2023-12-27 00:48:01,318][105620] Updated weights for policy 1, policy_version 1293986 (0.0008) [2023-12-27 00:48:01,385][105620] Updated weights for policy 1, policy_version 1293996 (0.0009) [2023-12-27 00:48:01,396][105692] Updated weights for policy 0, policy_version 1292410 (0.0007) [2023-12-27 00:48:01,450][105620] Updated weights for policy 1, policy_version 1294006 (0.0009) [2023-12-27 00:48:01,451][105692] Updated weights for policy 0, policy_version 1292420 (0.0005) [2023-12-27 00:48:01,507][105692] Updated weights for policy 0, policy_version 1292430 (0.0006) [2023-12-27 00:48:01,514][105620] Updated weights for policy 1, policy_version 1294016 (0.0008) [2023-12-27 00:48:02,209][105692] Updated weights for policy 0, policy_version 1292440 (0.0006) [2023-12-27 00:48:02,272][105620] Updated weights for policy 1, policy_version 1294026 (0.0009) [2023-12-27 00:48:02,275][105692] Updated weights for policy 0, policy_version 1292450 (0.0006) [2023-12-27 00:48:02,328][105620] Updated weights for policy 1, policy_version 1294036 (0.0010) [2023-12-27 00:48:02,331][105692] Updated weights for policy 0, policy_version 1292460 (0.0008) [2023-12-27 00:48:02,391][105620] Updated weights for policy 1, policy_version 1294046 (0.0009) [2023-12-27 00:48:02,986][105692] Updated weights for policy 0, policy_version 1292470 (0.0009) [2023-12-27 00:48:03,046][105692] Updated weights for policy 0, policy_version 1292480 (0.0007) [2023-12-27 00:48:03,106][105692] Updated weights for policy 0, policy_version 1292490 (0.0005) [2023-12-27 00:48:03,151][105620] Updated weights for policy 1, policy_version 1294056 (0.0006) [2023-12-27 00:48:03,205][105620] Updated weights for policy 1, policy_version 1294066 (0.0006) [2023-12-27 00:48:03,264][105620] Updated weights for policy 1, policy_version 1294076 (0.0006) [2023-12-27 00:48:03,686][105692] Updated weights for policy 0, policy_version 1292500 (0.0005) [2023-12-27 00:48:03,745][105692] Updated weights for policy 0, policy_version 1292510 (0.0005) [2023-12-27 00:48:03,801][105620] Updated weights for policy 1, policy_version 1294086 (0.0010) [2023-12-27 00:48:03,804][105692] Updated weights for policy 0, policy_version 1292520 (0.0005) [2023-12-27 00:48:03,860][105620] Updated weights for policy 1, policy_version 1294096 (0.0010) [2023-12-27 00:48:03,909][105620] Updated weights for policy 1, policy_version 1294106 (0.0010) [2023-12-27 00:48:04,435][105692] Updated weights for policy 0, policy_version 1292530 (0.0008) [2023-12-27 00:48:04,480][105692] Updated weights for policy 0, policy_version 1292540 (0.0010) [2023-12-27 00:48:04,525][105692] Updated weights for policy 0, policy_version 1292550 (0.0010) [2023-12-27 00:48:04,589][105692] Updated weights for policy 0, policy_version 1292560 (0.0010) [2023-12-27 00:48:04,624][105620] Updated weights for policy 1, policy_version 1294116 (0.0010) [2023-12-27 00:48:04,673][105620] Updated weights for policy 1, policy_version 1294126 (0.0010) [2023-12-27 00:48:04,732][105620] Updated weights for policy 1, policy_version 1294136 (0.0005) [2023-12-27 00:48:05,331][105620] Updated weights for policy 1, policy_version 1294146 (0.0006) [2023-12-27 00:48:05,360][105692] Updated weights for policy 0, policy_version 1292570 (0.0011) [2023-12-27 00:48:05,393][105620] Updated weights for policy 1, policy_version 1294156 (0.0011) [2023-12-27 00:48:05,408][105692] Updated weights for policy 0, policy_version 1292580 (0.0010) [2023-12-27 00:48:05,449][105620] Updated weights for policy 1, policy_version 1294166 (0.0010) [2023-12-27 00:48:05,456][105692] Updated weights for policy 0, policy_version 1292590 (0.0010) [2023-12-27 00:48:05,503][105620] Updated weights for policy 1, policy_version 1294176 (0.0010) [2023-12-27 00:48:06,054][105620] Updated weights for policy 1, policy_version 1294186 (0.0010) [2023-12-27 00:48:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 662306816. Throughput: 0: 9617.2, 1: 9870.5. Samples: 662298616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:48:06,063][104569] Avg episode reward: [(0, '9082.249'), (1, '9351.871')] [2023-12-27 00:48:06,124][105620] Updated weights for policy 1, policy_version 1294196 (0.0011) [2023-12-27 00:48:06,193][105620] Updated weights for policy 1, policy_version 1294206 (0.0011) [2023-12-27 00:48:06,212][105692] Updated weights for policy 0, policy_version 1292600 (0.0010) [2023-12-27 00:48:06,272][105692] Updated weights for policy 0, policy_version 1292610 (0.0011) [2023-12-27 00:48:06,332][105692] Updated weights for policy 0, policy_version 1292620 (0.0011) [2023-12-27 00:48:06,937][105620] Updated weights for policy 1, policy_version 1294216 (0.0011) [2023-12-27 00:48:06,997][105620] Updated weights for policy 1, policy_version 1294226 (0.0011) [2023-12-27 00:48:07,018][105692] Updated weights for policy 0, policy_version 1292630 (0.0010) [2023-12-27 00:48:07,057][105620] Updated weights for policy 1, policy_version 1294236 (0.0010) [2023-12-27 00:48:07,074][105692] Updated weights for policy 0, policy_version 1292640 (0.0011) [2023-12-27 00:48:07,119][105692] Updated weights for policy 0, policy_version 1292650 (0.0010) [2023-12-27 00:48:07,684][105620] Updated weights for policy 1, policy_version 1294246 (0.0009) [2023-12-27 00:48:07,742][105620] Updated weights for policy 1, policy_version 1294256 (0.0010) [2023-12-27 00:48:07,832][105692] Updated weights for policy 0, policy_version 1292660 (0.0008) [2023-12-27 00:48:07,836][105620] Updated weights for policy 1, policy_version 1294266 (0.0010) [2023-12-27 00:48:07,890][105692] Updated weights for policy 0, policy_version 1292670 (0.0005) [2023-12-27 00:48:07,955][105692] Updated weights for policy 0, policy_version 1292680 (0.0005) [2023-12-27 00:48:08,458][105620] Updated weights for policy 1, policy_version 1294276 (0.0011) [2023-12-27 00:48:08,524][105620] Updated weights for policy 1, policy_version 1294286 (0.0008) [2023-12-27 00:48:08,583][105620] Updated weights for policy 1, policy_version 1294296 (0.0005) [2023-12-27 00:48:08,655][105692] Updated weights for policy 0, policy_version 1292690 (0.0010) [2023-12-27 00:48:08,713][105692] Updated weights for policy 0, policy_version 1292700 (0.0006) [2023-12-27 00:48:08,780][105692] Updated weights for policy 0, policy_version 1292710 (0.0010) [2023-12-27 00:48:08,843][105692] Updated weights for policy 0, policy_version 1292720 (0.0008) [2023-12-27 00:48:09,169][105620] Updated weights for policy 1, policy_version 1294306 (0.0006) [2023-12-27 00:48:09,237][105620] Updated weights for policy 1, policy_version 1294316 (0.0007) [2023-12-27 00:48:09,298][105620] Updated weights for policy 1, policy_version 1294326 (0.0008) [2023-12-27 00:48:09,367][105620] Updated weights for policy 1, policy_version 1294336 (0.0008) [2023-12-27 00:48:09,578][105692] Updated weights for policy 0, policy_version 1292730 (0.0011) [2023-12-27 00:48:09,640][105692] Updated weights for policy 0, policy_version 1292740 (0.0010) [2023-12-27 00:48:09,696][105692] Updated weights for policy 0, policy_version 1292750 (0.0011) [2023-12-27 00:48:10,036][105620] Updated weights for policy 1, policy_version 1294346 (0.0008) [2023-12-27 00:48:10,103][105620] Updated weights for policy 1, policy_version 1294356 (0.0008) [2023-12-27 00:48:10,169][105620] Updated weights for policy 1, policy_version 1294366 (0.0006) [2023-12-27 00:48:10,437][105692] Updated weights for policy 0, policy_version 1292760 (0.0011) [2023-12-27 00:48:10,503][105692] Updated weights for policy 0, policy_version 1292770 (0.0009) [2023-12-27 00:48:10,567][105692] Updated weights for policy 0, policy_version 1292780 (0.0007) [2023-12-27 00:48:10,897][105620] Updated weights for policy 1, policy_version 1294376 (0.0009) [2023-12-27 00:48:10,942][105620] Updated weights for policy 1, policy_version 1294386 (0.0005) [2023-12-27 00:48:10,988][105620] Updated weights for policy 1, policy_version 1294396 (0.0006) [2023-12-27 00:48:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19438.7). Total num frames: 662413312. Throughput: 0: 9680.2, 1: 9862.5. Samples: 662418676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:48:11,062][104569] Avg episode reward: [(0, '9356.211'), (1, '9351.890')] [2023-12-27 00:48:11,287][105692] Updated weights for policy 0, policy_version 1292790 (0.0008) [2023-12-27 00:48:11,340][105692] Updated weights for policy 0, policy_version 1292800 (0.0010) [2023-12-27 00:48:11,405][105692] Updated weights for policy 0, policy_version 1292810 (0.0008) [2023-12-27 00:48:11,804][105620] Updated weights for policy 1, policy_version 1294406 (0.0010) [2023-12-27 00:48:11,861][105620] Updated weights for policy 1, policy_version 1294416 (0.0009) [2023-12-27 00:48:11,923][105620] Updated weights for policy 1, policy_version 1294426 (0.0009) [2023-12-27 00:48:12,249][105692] Updated weights for policy 0, policy_version 1292820 (0.0007) [2023-12-27 00:48:12,311][105692] Updated weights for policy 0, policy_version 1292830 (0.0009) [2023-12-27 00:48:12,372][105692] Updated weights for policy 0, policy_version 1292840 (0.0009) [2023-12-27 00:48:12,692][105620] Updated weights for policy 1, policy_version 1294436 (0.0008) [2023-12-27 00:48:12,745][105620] Updated weights for policy 1, policy_version 1294446 (0.0006) [2023-12-27 00:48:12,804][105620] Updated weights for policy 1, policy_version 1294456 (0.0006) [2023-12-27 00:48:13,229][105692] Updated weights for policy 0, policy_version 1292850 (0.0010) [2023-12-27 00:48:13,287][105692] Updated weights for policy 0, policy_version 1292860 (0.0010) [2023-12-27 00:48:13,346][105692] Updated weights for policy 0, policy_version 1292870 (0.0010) [2023-12-27 00:48:13,392][105620] Updated weights for policy 1, policy_version 1294466 (0.0007) [2023-12-27 00:48:13,404][105692] Updated weights for policy 0, policy_version 1292880 (0.0009) [2023-12-27 00:48:13,448][105620] Updated weights for policy 1, policy_version 1294476 (0.0005) [2023-12-27 00:48:13,504][105620] Updated weights for policy 1, policy_version 1294486 (0.0005) [2023-12-27 00:48:13,560][105620] Updated weights for policy 1, policy_version 1294496 (0.0005) [2023-12-27 00:48:14,128][105620] Updated weights for policy 1, policy_version 1294506 (0.0010) [2023-12-27 00:48:14,180][105620] Updated weights for policy 1, policy_version 1294516 (0.0010) [2023-12-27 00:48:14,235][105620] Updated weights for policy 1, policy_version 1294526 (0.0010) [2023-12-27 00:48:14,245][105692] Updated weights for policy 0, policy_version 1292890 (0.0005) [2023-12-27 00:48:14,293][105692] Updated weights for policy 0, policy_version 1292900 (0.0008) [2023-12-27 00:48:14,345][105692] Updated weights for policy 0, policy_version 1292910 (0.0008) [2023-12-27 00:48:14,851][105620] Updated weights for policy 1, policy_version 1294536 (0.0010) [2023-12-27 00:48:14,904][105620] Updated weights for policy 1, policy_version 1294546 (0.0011) [2023-12-27 00:48:14,964][105620] Updated weights for policy 1, policy_version 1294556 (0.0007) [2023-12-27 00:48:15,255][105692] Updated weights for policy 0, policy_version 1292920 (0.0010) [2023-12-27 00:48:15,313][105692] Updated weights for policy 0, policy_version 1292930 (0.0010) [2023-12-27 00:48:15,369][105692] Updated weights for policy 0, policy_version 1292940 (0.0008) [2023-12-27 00:48:15,537][105620] Updated weights for policy 1, policy_version 1294566 (0.0006) [2023-12-27 00:48:15,601][105620] Updated weights for policy 1, policy_version 1294576 (0.0005) [2023-12-27 00:48:15,657][105620] Updated weights for policy 1, policy_version 1294586 (0.0005) [2023-12-27 00:48:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 662503424. Throughput: 0: 9647.6, 1: 9826.5. Samples: 662474920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 00:48:16,063][104569] Avg episode reward: [(0, '9087.485'), (1, '9260.885')] [2023-12-27 00:48:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001292944_331046912.pth... [2023-12-27 00:48:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001294592_331456512.pth... [2023-12-27 00:48:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001293440_331161600.pth [2023-12-27 00:48:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001291824_330760192.pth [2023-12-27 00:48:16,162][105620] Updated weights for policy 1, policy_version 1294596 (0.0005) [2023-12-27 00:48:16,216][105620] Updated weights for policy 1, policy_version 1294606 (0.0005) [2023-12-27 00:48:16,269][105620] Updated weights for policy 1, policy_version 1294616 (0.0005) [2023-12-27 00:48:16,295][105692] Updated weights for policy 0, policy_version 1292950 (0.0009) [2023-12-27 00:48:16,376][105692] Updated weights for policy 0, policy_version 1292960 (0.0009) [2023-12-27 00:48:16,437][105692] Updated weights for policy 0, policy_version 1292970 (0.0006) [2023-12-27 00:48:16,953][105620] Updated weights for policy 1, policy_version 1294626 (0.0006) [2023-12-27 00:48:17,012][105620] Updated weights for policy 1, policy_version 1294636 (0.0009) [2023-12-27 00:48:17,063][105620] Updated weights for policy 1, policy_version 1294646 (0.0010) [2023-12-27 00:48:17,124][105620] Updated weights for policy 1, policy_version 1294656 (0.0010) [2023-12-27 00:48:17,192][105692] Updated weights for policy 0, policy_version 1292980 (0.0009) [2023-12-27 00:48:17,240][105692] Updated weights for policy 0, policy_version 1292990 (0.0008) [2023-12-27 00:48:17,288][105692] Updated weights for policy 0, policy_version 1293000 (0.0008) [2023-12-27 00:48:17,799][105620] Updated weights for policy 1, policy_version 1294666 (0.0011) [2023-12-27 00:48:17,851][105620] Updated weights for policy 1, policy_version 1294676 (0.0010) [2023-12-27 00:48:17,903][105620] Updated weights for policy 1, policy_version 1294686 (0.0010) [2023-12-27 00:48:18,080][105692] Updated weights for policy 0, policy_version 1293010 (0.0009) [2023-12-27 00:48:18,138][105692] Updated weights for policy 0, policy_version 1293020 (0.0008) [2023-12-27 00:48:18,196][105692] Updated weights for policy 0, policy_version 1293030 (0.0008) [2023-12-27 00:48:18,251][105692] Updated weights for policy 0, policy_version 1293040 (0.0008) [2023-12-27 00:48:18,680][105620] Updated weights for policy 1, policy_version 1294696 (0.0011) [2023-12-27 00:48:18,743][105620] Updated weights for policy 1, policy_version 1294706 (0.0011) [2023-12-27 00:48:18,809][105620] Updated weights for policy 1, policy_version 1294716 (0.0010) [2023-12-27 00:48:19,022][105692] Updated weights for policy 0, policy_version 1293050 (0.0008) [2023-12-27 00:48:19,081][105692] Updated weights for policy 0, policy_version 1293060 (0.0008) [2023-12-27 00:48:19,137][105692] Updated weights for policy 0, policy_version 1293070 (0.0009) [2023-12-27 00:48:19,521][105620] Updated weights for policy 1, policy_version 1294726 (0.0009) [2023-12-27 00:48:19,589][105620] Updated weights for policy 1, policy_version 1294736 (0.0007) [2023-12-27 00:48:19,658][105620] Updated weights for policy 1, policy_version 1294746 (0.0008) [2023-12-27 00:48:19,981][105692] Updated weights for policy 0, policy_version 1293080 (0.0009) [2023-12-27 00:48:20,039][105692] Updated weights for policy 0, policy_version 1293090 (0.0008) [2023-12-27 00:48:20,103][105692] Updated weights for policy 0, policy_version 1293100 (0.0009) [2023-12-27 00:48:20,400][105620] Updated weights for policy 1, policy_version 1294756 (0.0009) [2023-12-27 00:48:20,456][105620] Updated weights for policy 1, policy_version 1294766 (0.0010) [2023-12-27 00:48:20,517][105620] Updated weights for policy 1, policy_version 1294776 (0.0006) [2023-12-27 00:48:20,894][105692] Updated weights for policy 0, policy_version 1293110 (0.0009) [2023-12-27 00:48:20,947][105692] Updated weights for policy 0, policy_version 1293120 (0.0009) [2023-12-27 00:48:21,014][105692] Updated weights for policy 0, policy_version 1293130 (0.0009) [2023-12-27 00:48:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 662601728. Throughput: 0: 9524.5, 1: 9886.4. Samples: 662589524. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:48:21,062][104569] Avg episode reward: [(0, '8907.900'), (1, '9081.432')] [2023-12-27 00:48:21,253][105620] Updated weights for policy 1, policy_version 1294786 (0.0007) [2023-12-27 00:48:21,315][105620] Updated weights for policy 1, policy_version 1294796 (0.0008) [2023-12-27 00:48:21,386][105620] Updated weights for policy 1, policy_version 1294806 (0.0009) [2023-12-27 00:48:21,454][105620] Updated weights for policy 1, policy_version 1294816 (0.0010) [2023-12-27 00:48:21,815][105692] Updated weights for policy 0, policy_version 1293140 (0.0009) [2023-12-27 00:48:21,879][105692] Updated weights for policy 0, policy_version 1293150 (0.0009) [2023-12-27 00:48:21,941][105692] Updated weights for policy 0, policy_version 1293160 (0.0009) [2023-12-27 00:48:22,232][105620] Updated weights for policy 1, policy_version 1294826 (0.0008) [2023-12-27 00:48:22,295][105620] Updated weights for policy 1, policy_version 1294836 (0.0009) [2023-12-27 00:48:22,354][105620] Updated weights for policy 1, policy_version 1294846 (0.0009) [2023-12-27 00:48:22,707][105692] Updated weights for policy 0, policy_version 1293170 (0.0009) [2023-12-27 00:48:22,759][105692] Updated weights for policy 0, policy_version 1293180 (0.0009) [2023-12-27 00:48:22,810][105692] Updated weights for policy 0, policy_version 1293190 (0.0008) [2023-12-27 00:48:22,858][105692] Updated weights for policy 0, policy_version 1293200 (0.0005) [2023-12-27 00:48:23,031][105620] Updated weights for policy 1, policy_version 1294856 (0.0008) [2023-12-27 00:48:23,089][105620] Updated weights for policy 1, policy_version 1294866 (0.0010) [2023-12-27 00:48:23,136][105620] Updated weights for policy 1, policy_version 1294876 (0.0008) [2023-12-27 00:48:23,608][105692] Updated weights for policy 0, policy_version 1293210 (0.0009) [2023-12-27 00:48:23,680][105692] Updated weights for policy 0, policy_version 1293220 (0.0010) [2023-12-27 00:48:23,736][105692] Updated weights for policy 0, policy_version 1293230 (0.0009) [2023-12-27 00:48:23,753][105620] Updated weights for policy 1, policy_version 1294886 (0.0007) [2023-12-27 00:48:23,761][105586] KL-divergence is very high: 115.1644 [2023-12-27 00:48:23,797][105620] Updated weights for policy 1, policy_version 1294896 (0.0005) [2023-12-27 00:48:23,798][105586] KL-divergence is very high: 212.1054 [2023-12-27 00:48:23,835][105586] KL-divergence is very high: 225.2203 [2023-12-27 00:48:23,845][105620] Updated weights for policy 1, policy_version 1294906 (0.0005) [2023-12-27 00:48:24,518][105620] Updated weights for policy 1, policy_version 1294916 (0.0006) [2023-12-27 00:48:24,532][105692] Updated weights for policy 0, policy_version 1293240 (0.0007) [2023-12-27 00:48:24,566][105620] Updated weights for policy 1, policy_version 1294926 (0.0006) [2023-12-27 00:48:24,581][105692] Updated weights for policy 0, policy_version 1293250 (0.0007) [2023-12-27 00:48:24,613][105620] Updated weights for policy 1, policy_version 1294936 (0.0007) [2023-12-27 00:48:24,639][105692] Updated weights for policy 0, policy_version 1293260 (0.0007) [2023-12-27 00:48:25,228][105692] Updated weights for policy 0, policy_version 1293270 (0.0008) [2023-12-27 00:48:25,247][105620] Updated weights for policy 1, policy_version 1294946 (0.0008) [2023-12-27 00:48:25,287][105692] Updated weights for policy 0, policy_version 1293280 (0.0010) [2023-12-27 00:48:25,293][105620] Updated weights for policy 1, policy_version 1294956 (0.0008) [2023-12-27 00:48:25,345][105620] Updated weights for policy 1, policy_version 1294966 (0.0005) [2023-12-27 00:48:25,346][105692] Updated weights for policy 0, policy_version 1293290 (0.0011) [2023-12-27 00:48:25,397][105620] Updated weights for policy 1, policy_version 1294976 (0.0007) [2023-12-27 00:48:26,017][105692] Updated weights for policy 0, policy_version 1293300 (0.0011) [2023-12-27 00:48:26,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 662691840. Throughput: 0: 9535.1, 1: 9902.0. Samples: 662706172. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:48:26,063][104569] Avg episode reward: [(0, '9085.327'), (1, '8990.723')] [2023-12-27 00:48:26,076][105692] Updated weights for policy 0, policy_version 1293310 (0.0009) [2023-12-27 00:48:26,089][105620] Updated weights for policy 1, policy_version 1294986 (0.0008) [2023-12-27 00:48:26,137][105692] Updated weights for policy 0, policy_version 1293320 (0.0010) [2023-12-27 00:48:26,138][105620] Updated weights for policy 1, policy_version 1294996 (0.0008) [2023-12-27 00:48:26,188][105620] Updated weights for policy 1, policy_version 1295006 (0.0007) [2023-12-27 00:48:26,724][105692] Updated weights for policy 0, policy_version 1293330 (0.0009) [2023-12-27 00:48:26,767][105692] Updated weights for policy 0, policy_version 1293340 (0.0005) [2023-12-27 00:48:26,812][105692] Updated weights for policy 0, policy_version 1293350 (0.0005) [2023-12-27 00:48:26,856][105692] Updated weights for policy 0, policy_version 1293360 (0.0005) [2023-12-27 00:48:26,998][105620] Updated weights for policy 1, policy_version 1295016 (0.0010) [2023-12-27 00:48:27,050][105620] Updated weights for policy 1, policy_version 1295026 (0.0010) [2023-12-27 00:48:27,101][105620] Updated weights for policy 1, policy_version 1295036 (0.0010) [2023-12-27 00:48:27,431][105692] Updated weights for policy 0, policy_version 1293370 (0.0010) [2023-12-27 00:48:27,496][105692] Updated weights for policy 0, policy_version 1293380 (0.0010) [2023-12-27 00:48:27,551][105692] Updated weights for policy 0, policy_version 1293390 (0.0010) [2023-12-27 00:48:27,730][105620] Updated weights for policy 1, policy_version 1295046 (0.0007) [2023-12-27 00:48:27,792][105620] Updated weights for policy 1, policy_version 1295056 (0.0007) [2023-12-27 00:48:27,853][105620] Updated weights for policy 1, policy_version 1295066 (0.0008) [2023-12-27 00:48:28,242][105692] Updated weights for policy 0, policy_version 1293400 (0.0010) [2023-12-27 00:48:28,289][105692] Updated weights for policy 0, policy_version 1293410 (0.0010) [2023-12-27 00:48:28,347][105692] Updated weights for policy 0, policy_version 1293420 (0.0010) [2023-12-27 00:48:28,442][105620] Updated weights for policy 1, policy_version 1295076 (0.0008) [2023-12-27 00:48:28,498][105620] Updated weights for policy 1, policy_version 1295086 (0.0008) [2023-12-27 00:48:28,554][105620] Updated weights for policy 1, policy_version 1295096 (0.0009) [2023-12-27 00:48:29,067][105692] Updated weights for policy 0, policy_version 1293430 (0.0007) [2023-12-27 00:48:29,121][105692] Updated weights for policy 0, policy_version 1293440 (0.0005) [2023-12-27 00:48:29,169][105692] Updated weights for policy 0, policy_version 1293450 (0.0005) [2023-12-27 00:48:29,380][105620] Updated weights for policy 1, policy_version 1295106 (0.0008) [2023-12-27 00:48:29,451][105620] Updated weights for policy 1, policy_version 1295116 (0.0005) [2023-12-27 00:48:29,510][105620] Updated weights for policy 1, policy_version 1295126 (0.0009) [2023-12-27 00:48:29,566][105620] Updated weights for policy 1, policy_version 1295136 (0.0011) [2023-12-27 00:48:29,804][105692] Updated weights for policy 0, policy_version 1293460 (0.0006) [2023-12-27 00:48:29,871][105692] Updated weights for policy 0, policy_version 1293470 (0.0008) [2023-12-27 00:48:29,926][105692] Updated weights for policy 0, policy_version 1293480 (0.0008) [2023-12-27 00:48:30,279][105620] Updated weights for policy 1, policy_version 1295146 (0.0011) [2023-12-27 00:48:30,345][105620] Updated weights for policy 1, policy_version 1295156 (0.0011) [2023-12-27 00:48:30,403][105620] Updated weights for policy 1, policy_version 1295166 (0.0010) [2023-12-27 00:48:30,670][105692] Updated weights for policy 0, policy_version 1293490 (0.0009) [2023-12-27 00:48:30,726][105692] Updated weights for policy 0, policy_version 1293500 (0.0009) [2023-12-27 00:48:30,781][105692] Updated weights for policy 0, policy_version 1293510 (0.0008) [2023-12-27 00:48:30,825][105692] Updated weights for policy 0, policy_version 1293520 (0.0007) [2023-12-27 00:48:31,034][105620] Updated weights for policy 1, policy_version 1295176 (0.0010) [2023-12-27 00:48:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 662798336. Throughput: 0: 9635.5, 1: 9944.7. Samples: 662768860. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:48:31,062][104569] Avg episode reward: [(0, '9172.312'), (1, '9078.906')] [2023-12-27 00:48:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001293520_331194368.pth... [2023-12-27 00:48:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001292400_330907648.pth [2023-12-27 00:48:31,095][105620] Updated weights for policy 1, policy_version 1295186 (0.0007) [2023-12-27 00:48:31,157][105620] Updated weights for policy 1, policy_version 1295196 (0.0008) [2023-12-27 00:48:31,178][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001295200_331612160.pth... [2023-12-27 00:48:31,182][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001293984_331300864.pth [2023-12-27 00:48:31,653][105692] Updated weights for policy 0, policy_version 1293530 (0.0008) [2023-12-27 00:48:31,724][105692] Updated weights for policy 0, policy_version 1293540 (0.0008) [2023-12-27 00:48:31,780][105692] Updated weights for policy 0, policy_version 1293550 (0.0008) [2023-12-27 00:48:31,872][105620] Updated weights for policy 1, policy_version 1295206 (0.0005) [2023-12-27 00:48:31,939][105620] Updated weights for policy 1, policy_version 1295216 (0.0006) [2023-12-27 00:48:32,004][105620] Updated weights for policy 1, policy_version 1295226 (0.0007) [2023-12-27 00:48:32,602][105620] Updated weights for policy 1, policy_version 1295236 (0.0010) [2023-12-27 00:48:32,608][105692] Updated weights for policy 0, policy_version 1293560 (0.0009) [2023-12-27 00:48:32,656][105620] Updated weights for policy 1, policy_version 1295246 (0.0006) [2023-12-27 00:48:32,663][105692] Updated weights for policy 0, policy_version 1293570 (0.0008) [2023-12-27 00:48:32,709][105620] Updated weights for policy 1, policy_version 1295256 (0.0005) [2023-12-27 00:48:32,723][105692] Updated weights for policy 0, policy_version 1293580 (0.0009) [2023-12-27 00:48:33,265][105620] Updated weights for policy 1, policy_version 1295266 (0.0006) [2023-12-27 00:48:33,311][105620] Updated weights for policy 1, policy_version 1295276 (0.0008) [2023-12-27 00:48:33,352][105620] Updated weights for policy 1, policy_version 1295286 (0.0010) [2023-12-27 00:48:33,395][105620] Updated weights for policy 1, policy_version 1295296 (0.0005) [2023-12-27 00:48:33,602][105692] Updated weights for policy 0, policy_version 1293590 (0.0009) [2023-12-27 00:48:33,656][105692] Updated weights for policy 0, policy_version 1293600 (0.0012) [2023-12-27 00:48:33,710][105692] Updated weights for policy 0, policy_version 1293610 (0.0012) [2023-12-27 00:48:34,009][105620] Updated weights for policy 1, policy_version 1295306 (0.0005) [2023-12-27 00:48:34,052][105620] Updated weights for policy 1, policy_version 1295316 (0.0005) [2023-12-27 00:48:34,108][105620] Updated weights for policy 1, policy_version 1295326 (0.0006) [2023-12-27 00:48:34,362][105692] Updated weights for policy 0, policy_version 1293620 (0.0009) [2023-12-27 00:48:34,422][105692] Updated weights for policy 0, policy_version 1293630 (0.0010) [2023-12-27 00:48:34,483][105692] Updated weights for policy 0, policy_version 1293640 (0.0009) [2023-12-27 00:48:34,769][105620] Updated weights for policy 1, policy_version 1295336 (0.0009) [2023-12-27 00:48:34,839][105620] Updated weights for policy 1, policy_version 1295346 (0.0010) [2023-12-27 00:48:34,896][105620] Updated weights for policy 1, policy_version 1295356 (0.0008) [2023-12-27 00:48:35,152][105692] Updated weights for policy 0, policy_version 1293650 (0.0010) [2023-12-27 00:48:35,220][105692] Updated weights for policy 0, policy_version 1293660 (0.0010) [2023-12-27 00:48:35,279][105692] Updated weights for policy 0, policy_version 1293670 (0.0009) [2023-12-27 00:48:35,344][105692] Updated weights for policy 0, policy_version 1293680 (0.0005) [2023-12-27 00:48:35,662][105620] Updated weights for policy 1, policy_version 1295366 (0.0006) [2023-12-27 00:48:35,708][105620] Updated weights for policy 1, policy_version 1295376 (0.0005) [2023-12-27 00:48:35,767][105620] Updated weights for policy 1, policy_version 1295386 (0.0008) [2023-12-27 00:48:35,974][105692] Updated weights for policy 0, policy_version 1293690 (0.0011) [2023-12-27 00:48:36,022][105692] Updated weights for policy 0, policy_version 1293700 (0.0010) [2023-12-27 00:48:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 662896640. Throughput: 0: 9605.0, 1: 10050.8. Samples: 662886932. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:48:36,063][104569] Avg episode reward: [(0, '9267.026'), (1, '8805.892')] [2023-12-27 00:48:36,077][105692] Updated weights for policy 0, policy_version 1293710 (0.0010) [2023-12-27 00:48:36,452][105620] Updated weights for policy 1, policy_version 1295396 (0.0008) [2023-12-27 00:48:36,514][105620] Updated weights for policy 1, policy_version 1295406 (0.0009) [2023-12-27 00:48:36,572][105620] Updated weights for policy 1, policy_version 1295416 (0.0009) [2023-12-27 00:48:36,818][105692] Updated weights for policy 0, policy_version 1293720 (0.0010) [2023-12-27 00:48:36,866][105692] Updated weights for policy 0, policy_version 1293730 (0.0010) [2023-12-27 00:48:36,917][105692] Updated weights for policy 0, policy_version 1293740 (0.0010) [2023-12-27 00:48:37,361][105620] Updated weights for policy 1, policy_version 1295426 (0.0008) [2023-12-27 00:48:37,429][105620] Updated weights for policy 1, policy_version 1295436 (0.0009) [2023-12-27 00:48:37,492][105620] Updated weights for policy 1, policy_version 1295446 (0.0008) [2023-12-27 00:48:37,551][105620] Updated weights for policy 1, policy_version 1295456 (0.0008) [2023-12-27 00:48:37,697][105692] Updated weights for policy 0, policy_version 1293750 (0.0008) [2023-12-27 00:48:37,762][105692] Updated weights for policy 0, policy_version 1293760 (0.0009) [2023-12-27 00:48:37,817][105692] Updated weights for policy 0, policy_version 1293770 (0.0011) [2023-12-27 00:48:38,360][105620] Updated weights for policy 1, policy_version 1295466 (0.0008) [2023-12-27 00:48:38,389][105692] Updated weights for policy 0, policy_version 1293780 (0.0008) [2023-12-27 00:48:38,424][105620] Updated weights for policy 1, policy_version 1295476 (0.0006) [2023-12-27 00:48:38,449][105692] Updated weights for policy 0, policy_version 1293790 (0.0011) [2023-12-27 00:48:38,483][105620] Updated weights for policy 1, policy_version 1295486 (0.0006) [2023-12-27 00:48:38,508][105692] Updated weights for policy 0, policy_version 1293800 (0.0010) [2023-12-27 00:48:39,175][105692] Updated weights for policy 0, policy_version 1293810 (0.0011) [2023-12-27 00:48:39,252][105692] Updated weights for policy 0, policy_version 1293820 (0.0009) [2023-12-27 00:48:39,253][105620] Updated weights for policy 1, policy_version 1295496 (0.0008) [2023-12-27 00:48:39,304][105692] Updated weights for policy 0, policy_version 1293830 (0.0007) [2023-12-27 00:48:39,323][105620] Updated weights for policy 1, policy_version 1295506 (0.0008) [2023-12-27 00:48:39,368][105692] Updated weights for policy 0, policy_version 1293840 (0.0007) [2023-12-27 00:48:39,391][105620] Updated weights for policy 1, policy_version 1295516 (0.0009) [2023-12-27 00:48:40,017][105692] Updated weights for policy 0, policy_version 1293850 (0.0009) [2023-12-27 00:48:40,079][105692] Updated weights for policy 0, policy_version 1293860 (0.0009) [2023-12-27 00:48:40,128][105620] Updated weights for policy 1, policy_version 1295526 (0.0008) [2023-12-27 00:48:40,136][105692] Updated weights for policy 0, policy_version 1293870 (0.0008) [2023-12-27 00:48:40,190][105620] Updated weights for policy 1, policy_version 1295536 (0.0008) [2023-12-27 00:48:40,246][105620] Updated weights for policy 1, policy_version 1295546 (0.0009) [2023-12-27 00:48:40,794][105692] Updated weights for policy 0, policy_version 1293880 (0.0006) [2023-12-27 00:48:40,853][105692] Updated weights for policy 0, policy_version 1293890 (0.0005) [2023-12-27 00:48:40,907][105692] Updated weights for policy 0, policy_version 1293900 (0.0008) [2023-12-27 00:48:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 662994944. Throughput: 0: 9677.6, 1: 9965.7. Samples: 663003568. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:48:41,063][104569] Avg episode reward: [(0, '9266.380'), (1, '9079.652')] [2023-12-27 00:48:41,102][105620] Updated weights for policy 1, policy_version 1295556 (0.0010) [2023-12-27 00:48:41,172][105620] Updated weights for policy 1, policy_version 1295566 (0.0010) [2023-12-27 00:48:41,230][105620] Updated weights for policy 1, policy_version 1295576 (0.0009) [2023-12-27 00:48:41,662][105692] Updated weights for policy 0, policy_version 1293910 (0.0008) [2023-12-27 00:48:41,728][105692] Updated weights for policy 0, policy_version 1293920 (0.0009) [2023-12-27 00:48:41,795][105692] Updated weights for policy 0, policy_version 1293930 (0.0009) [2023-12-27 00:48:41,925][105620] Updated weights for policy 1, policy_version 1295586 (0.0008) [2023-12-27 00:48:41,974][105620] Updated weights for policy 1, policy_version 1295596 (0.0005) [2023-12-27 00:48:42,021][105620] Updated weights for policy 1, policy_version 1295606 (0.0005) [2023-12-27 00:48:42,067][105620] Updated weights for policy 1, policy_version 1295616 (0.0006) [2023-12-27 00:48:42,568][105692] Updated weights for policy 0, policy_version 1293940 (0.0008) [2023-12-27 00:48:42,621][105692] Updated weights for policy 0, policy_version 1293950 (0.0007) [2023-12-27 00:48:42,673][105692] Updated weights for policy 0, policy_version 1293960 (0.0005) [2023-12-27 00:48:42,750][105620] Updated weights for policy 1, policy_version 1295626 (0.0011) [2023-12-27 00:48:42,801][105620] Updated weights for policy 1, policy_version 1295636 (0.0006) [2023-12-27 00:48:42,852][105620] Updated weights for policy 1, policy_version 1295646 (0.0006) [2023-12-27 00:48:43,424][105692] Updated weights for policy 0, policy_version 1293970 (0.0007) [2023-12-27 00:48:43,463][105620] Updated weights for policy 1, policy_version 1295656 (0.0009) [2023-12-27 00:48:43,472][105692] Updated weights for policy 0, policy_version 1293980 (0.0010) [2023-12-27 00:48:43,511][105620] Updated weights for policy 1, policy_version 1295666 (0.0010) [2023-12-27 00:48:43,523][105692] Updated weights for policy 0, policy_version 1293990 (0.0010) [2023-12-27 00:48:43,556][105620] Updated weights for policy 1, policy_version 1295676 (0.0010) [2023-12-27 00:48:43,584][105692] Updated weights for policy 0, policy_version 1294000 (0.0010) [2023-12-27 00:48:44,203][105620] Updated weights for policy 1, policy_version 1295686 (0.0008) [2023-12-27 00:48:44,258][105620] Updated weights for policy 1, policy_version 1295696 (0.0010) [2023-12-27 00:48:44,316][105620] Updated weights for policy 1, policy_version 1295706 (0.0010) [2023-12-27 00:48:44,334][105692] Updated weights for policy 0, policy_version 1294010 (0.0006) [2023-12-27 00:48:44,381][105692] Updated weights for policy 0, policy_version 1294020 (0.0007) [2023-12-27 00:48:44,433][105692] Updated weights for policy 0, policy_version 1294030 (0.0008) [2023-12-27 00:48:45,066][105620] Updated weights for policy 1, policy_version 1295716 (0.0011) [2023-12-27 00:48:45,118][105620] Updated weights for policy 1, policy_version 1295726 (0.0011) [2023-12-27 00:48:45,178][105620] Updated weights for policy 1, policy_version 1295736 (0.0011) [2023-12-27 00:48:45,193][105692] Updated weights for policy 0, policy_version 1294040 (0.0010) [2023-12-27 00:48:45,250][105692] Updated weights for policy 0, policy_version 1294050 (0.0011) [2023-12-27 00:48:45,316][105692] Updated weights for policy 0, policy_version 1294060 (0.0011) [2023-12-27 00:48:45,948][105620] Updated weights for policy 1, policy_version 1295746 (0.0011) [2023-12-27 00:48:45,999][105620] Updated weights for policy 1, policy_version 1295756 (0.0010) [2023-12-27 00:48:46,048][105620] Updated weights for policy 1, policy_version 1295766 (0.0010) [2023-12-27 00:48:46,054][105692] Updated weights for policy 0, policy_version 1294070 (0.0011) [2023-12-27 00:48:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 663085056. Throughput: 0: 9540.0, 1: 10089.4. Samples: 663063424. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:48:46,063][104569] Avg episode reward: [(0, '9265.550'), (1, '9353.074')] [2023-12-27 00:48:46,094][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001295776_331759616.pth... [2023-12-27 00:48:46,096][105620] Updated weights for policy 1, policy_version 1295776 (0.0010) [2023-12-27 00:48:46,098][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001294592_331456512.pth [2023-12-27 00:48:46,106][105692] Updated weights for policy 0, policy_version 1294080 (0.0010) [2023-12-27 00:48:46,171][105692] Updated weights for policy 0, policy_version 1294090 (0.0010) [2023-12-27 00:48:46,208][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001294096_331341824.pth... [2023-12-27 00:48:46,212][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001292944_331046912.pth [2023-12-27 00:48:46,882][105620] Updated weights for policy 1, policy_version 1295786 (0.0011) [2023-12-27 00:48:46,935][105620] Updated weights for policy 1, policy_version 1295796 (0.0010) [2023-12-27 00:48:46,961][105692] Updated weights for policy 0, policy_version 1294100 (0.0010) [2023-12-27 00:48:46,991][105620] Updated weights for policy 1, policy_version 1295806 (0.0011) [2023-12-27 00:48:47,016][105692] Updated weights for policy 0, policy_version 1294110 (0.0010) [2023-12-27 00:48:47,075][105692] Updated weights for policy 0, policy_version 1294120 (0.0010) [2023-12-27 00:48:47,728][105620] Updated weights for policy 1, policy_version 1295816 (0.0009) [2023-12-27 00:48:47,786][105620] Updated weights for policy 1, policy_version 1295826 (0.0010) [2023-12-27 00:48:47,801][105692] Updated weights for policy 0, policy_version 1294130 (0.0010) [2023-12-27 00:48:47,841][105620] Updated weights for policy 1, policy_version 1295836 (0.0009) [2023-12-27 00:48:47,859][105692] Updated weights for policy 0, policy_version 1294140 (0.0006) [2023-12-27 00:48:47,929][105692] Updated weights for policy 0, policy_version 1294150 (0.0005) [2023-12-27 00:48:47,994][105692] Updated weights for policy 0, policy_version 1294160 (0.0007) [2023-12-27 00:48:48,621][105620] Updated weights for policy 1, policy_version 1295846 (0.0006) [2023-12-27 00:48:48,662][105692] Updated weights for policy 0, policy_version 1294170 (0.0011) [2023-12-27 00:48:48,674][105620] Updated weights for policy 1, policy_version 1295856 (0.0009) [2023-12-27 00:48:48,721][105692] Updated weights for policy 0, policy_version 1294180 (0.0011) [2023-12-27 00:48:48,735][105620] Updated weights for policy 1, policy_version 1295866 (0.0009) [2023-12-27 00:48:48,774][105692] Updated weights for policy 0, policy_version 1294190 (0.0011) [2023-12-27 00:48:49,489][105620] Updated weights for policy 1, policy_version 1295876 (0.0007) [2023-12-27 00:48:49,501][105692] Updated weights for policy 0, policy_version 1294200 (0.0009) [2023-12-27 00:48:49,554][105620] Updated weights for policy 1, policy_version 1295886 (0.0007) [2023-12-27 00:48:49,559][105692] Updated weights for policy 0, policy_version 1294210 (0.0007) [2023-12-27 00:48:49,615][105692] Updated weights for policy 0, policy_version 1294220 (0.0008) [2023-12-27 00:48:49,617][105620] Updated weights for policy 1, policy_version 1295896 (0.0006) [2023-12-27 00:48:50,305][105692] Updated weights for policy 0, policy_version 1294230 (0.0008) [2023-12-27 00:48:50,365][105692] Updated weights for policy 0, policy_version 1294240 (0.0007) [2023-12-27 00:48:50,413][105692] Updated weights for policy 0, policy_version 1294250 (0.0007) [2023-12-27 00:48:50,433][105620] Updated weights for policy 1, policy_version 1295906 (0.0007) [2023-12-27 00:48:50,497][105620] Updated weights for policy 1, policy_version 1295916 (0.0010) [2023-12-27 00:48:50,564][105620] Updated weights for policy 1, policy_version 1295926 (0.0009) [2023-12-27 00:48:50,622][105620] Updated weights for policy 1, policy_version 1295936 (0.0009) [2023-12-27 00:48:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 663183360. Throughput: 0: 9471.8, 1: 10025.1. Samples: 663175980. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:48:51,063][104569] Avg episode reward: [(0, '9355.685'), (1, '9260.626')] [2023-12-27 00:48:51,083][105692] Updated weights for policy 0, policy_version 1294260 (0.0006) [2023-12-27 00:48:51,152][105692] Updated weights for policy 0, policy_version 1294270 (0.0007) [2023-12-27 00:48:51,218][105692] Updated weights for policy 0, policy_version 1294280 (0.0009) [2023-12-27 00:48:51,483][105620] Updated weights for policy 1, policy_version 1295946 (0.0010) [2023-12-27 00:48:51,535][105620] Updated weights for policy 1, policy_version 1295956 (0.0009) [2023-12-27 00:48:51,593][105620] Updated weights for policy 1, policy_version 1295966 (0.0011) [2023-12-27 00:48:51,903][105692] Updated weights for policy 0, policy_version 1294290 (0.0007) [2023-12-27 00:48:51,965][105692] Updated weights for policy 0, policy_version 1294300 (0.0007) [2023-12-27 00:48:52,018][105692] Updated weights for policy 0, policy_version 1294310 (0.0007) [2023-12-27 00:48:52,077][105692] Updated weights for policy 0, policy_version 1294320 (0.0009) [2023-12-27 00:48:52,407][105620] Updated weights for policy 1, policy_version 1295976 (0.0009) [2023-12-27 00:48:52,467][105620] Updated weights for policy 1, policy_version 1295986 (0.0008) [2023-12-27 00:48:52,529][105620] Updated weights for policy 1, policy_version 1295996 (0.0009) [2023-12-27 00:48:52,747][105692] Updated weights for policy 0, policy_version 1294330 (0.0009) [2023-12-27 00:48:52,813][105692] Updated weights for policy 0, policy_version 1294340 (0.0007) [2023-12-27 00:48:52,876][105692] Updated weights for policy 0, policy_version 1294350 (0.0005) [2023-12-27 00:48:53,333][105620] Updated weights for policy 1, policy_version 1296006 (0.0009) [2023-12-27 00:48:53,391][105620] Updated weights for policy 1, policy_version 1296016 (0.0008) [2023-12-27 00:48:53,456][105620] Updated weights for policy 1, policy_version 1296026 (0.0009) [2023-12-27 00:48:53,514][105692] Updated weights for policy 0, policy_version 1294360 (0.0007) [2023-12-27 00:48:53,562][105692] Updated weights for policy 0, policy_version 1294370 (0.0009) [2023-12-27 00:48:53,609][105692] Updated weights for policy 0, policy_version 1294380 (0.0008) [2023-12-27 00:48:54,237][105620] Updated weights for policy 1, policy_version 1296036 (0.0009) [2023-12-27 00:48:54,302][105620] Updated weights for policy 1, policy_version 1296046 (0.0007) [2023-12-27 00:48:54,308][105692] Updated weights for policy 0, policy_version 1294390 (0.0010) [2023-12-27 00:48:54,361][105620] Updated weights for policy 1, policy_version 1296056 (0.0005) [2023-12-27 00:48:54,363][105692] Updated weights for policy 0, policy_version 1294400 (0.0010) [2023-12-27 00:48:54,422][105692] Updated weights for policy 0, policy_version 1294410 (0.0011) [2023-12-27 00:48:55,108][105620] Updated weights for policy 1, policy_version 1296066 (0.0006) [2023-12-27 00:48:55,160][105692] Updated weights for policy 0, policy_version 1294420 (0.0010) [2023-12-27 00:48:55,166][105620] Updated weights for policy 1, policy_version 1296076 (0.0007) [2023-12-27 00:48:55,204][105692] Updated weights for policy 0, policy_version 1294430 (0.0010) [2023-12-27 00:48:55,222][105620] Updated weights for policy 1, policy_version 1296086 (0.0005) [2023-12-27 00:48:55,249][105692] Updated weights for policy 0, policy_version 1294440 (0.0010) [2023-12-27 00:48:55,275][105620] Updated weights for policy 1, policy_version 1296096 (0.0006) [2023-12-27 00:48:55,943][105620] Updated weights for policy 1, policy_version 1296106 (0.0010) [2023-12-27 00:48:55,950][105692] Updated weights for policy 0, policy_version 1294450 (0.0010) [2023-12-27 00:48:55,995][105620] Updated weights for policy 1, policy_version 1296116 (0.0005) [2023-12-27 00:48:56,008][105692] Updated weights for policy 0, policy_version 1294460 (0.0010) [2023-12-27 00:48:56,053][105620] Updated weights for policy 1, policy_version 1296126 (0.0005) [2023-12-27 00:48:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 663273472. Throughput: 0: 9544.0, 1: 9829.5. Samples: 663290484. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:48:56,063][104569] Avg episode reward: [(0, '9176.979'), (1, '9260.905')] [2023-12-27 00:48:56,067][105692] Updated weights for policy 0, policy_version 1294470 (0.0010) [2023-12-27 00:48:56,138][105692] Updated weights for policy 0, policy_version 1294480 (0.0011) [2023-12-27 00:48:56,799][105692] Updated weights for policy 0, policy_version 1294490 (0.0005) [2023-12-27 00:48:56,814][105620] Updated weights for policy 1, policy_version 1296136 (0.0005) [2023-12-27 00:48:56,852][105692] Updated weights for policy 0, policy_version 1294500 (0.0005) [2023-12-27 00:48:56,865][105620] Updated weights for policy 1, policy_version 1296146 (0.0005) [2023-12-27 00:48:56,914][105692] Updated weights for policy 0, policy_version 1294510 (0.0007) [2023-12-27 00:48:56,922][105620] Updated weights for policy 1, policy_version 1296156 (0.0005) [2023-12-27 00:48:57,465][105620] Updated weights for policy 1, policy_version 1296166 (0.0008) [2023-12-27 00:48:57,516][105620] Updated weights for policy 1, policy_version 1296176 (0.0010) [2023-12-27 00:48:57,517][105692] Updated weights for policy 0, policy_version 1294520 (0.0009) [2023-12-27 00:48:57,565][105620] Updated weights for policy 1, policy_version 1296186 (0.0010) [2023-12-27 00:48:57,565][105692] Updated weights for policy 0, policy_version 1294530 (0.0010) [2023-12-27 00:48:57,613][105692] Updated weights for policy 0, policy_version 1294540 (0.0010) [2023-12-27 00:48:58,286][105620] Updated weights for policy 1, policy_version 1296196 (0.0009) [2023-12-27 00:48:58,353][105620] Updated weights for policy 1, policy_version 1296206 (0.0008) [2023-12-27 00:48:58,410][105692] Updated weights for policy 0, policy_version 1294550 (0.0009) [2023-12-27 00:48:58,414][105620] Updated weights for policy 1, policy_version 1296216 (0.0008) [2023-12-27 00:48:58,473][105692] Updated weights for policy 0, policy_version 1294560 (0.0008) [2023-12-27 00:48:58,541][105692] Updated weights for policy 0, policy_version 1294570 (0.0008) [2023-12-27 00:48:59,262][105692] Updated weights for policy 0, policy_version 1294580 (0.0007) [2023-12-27 00:48:59,265][105620] Updated weights for policy 1, policy_version 1296226 (0.0007) [2023-12-27 00:48:59,318][105620] Updated weights for policy 1, policy_version 1296236 (0.0008) [2023-12-27 00:48:59,327][105692] Updated weights for policy 0, policy_version 1294590 (0.0008) [2023-12-27 00:48:59,374][105620] Updated weights for policy 1, policy_version 1296246 (0.0008) [2023-12-27 00:48:59,392][105692] Updated weights for policy 0, policy_version 1294600 (0.0008) [2023-12-27 00:48:59,434][105620] Updated weights for policy 1, policy_version 1296256 (0.0009) [2023-12-27 00:49:00,092][105620] Updated weights for policy 1, policy_version 1296266 (0.0009) [2023-12-27 00:49:00,142][105692] Updated weights for policy 0, policy_version 1294610 (0.0007) [2023-12-27 00:49:00,144][105620] Updated weights for policy 1, policy_version 1296276 (0.0008) [2023-12-27 00:49:00,194][105692] Updated weights for policy 0, policy_version 1294620 (0.0007) [2023-12-27 00:49:00,204][105620] Updated weights for policy 1, policy_version 1296286 (0.0009) [2023-12-27 00:49:00,246][105692] Updated weights for policy 0, policy_version 1294630 (0.0007) [2023-12-27 00:49:00,295][105692] Updated weights for policy 0, policy_version 1294640 (0.0009) [2023-12-27 00:49:00,902][105620] Updated weights for policy 1, policy_version 1296296 (0.0008) [2023-12-27 00:49:00,948][105620] Updated weights for policy 1, policy_version 1296306 (0.0009) [2023-12-27 00:49:01,003][105620] Updated weights for policy 1, policy_version 1296316 (0.0008) [2023-12-27 00:49:01,051][105692] Updated weights for policy 0, policy_version 1294650 (0.0006) [2023-12-27 00:49:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 663379968. Throughput: 0: 9641.0, 1: 9797.6. Samples: 663349652. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:49:01,062][104569] Avg episode reward: [(0, '8914.131'), (1, '9262.903')] [2023-12-27 00:49:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001296320_331898880.pth... [2023-12-27 00:49:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001295200_331612160.pth [2023-12-27 00:49:01,112][105692] Updated weights for policy 0, policy_version 1294660 (0.0009) [2023-12-27 00:49:01,178][105692] Updated weights for policy 0, policy_version 1294670 (0.0009) [2023-12-27 00:49:01,187][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001294672_331489280.pth... [2023-12-27 00:49:01,190][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001293520_331194368.pth [2023-12-27 00:49:01,755][105620] Updated weights for policy 1, policy_version 1296326 (0.0008) [2023-12-27 00:49:01,820][105620] Updated weights for policy 1, policy_version 1296336 (0.0009) [2023-12-27 00:49:01,875][105620] Updated weights for policy 1, policy_version 1296346 (0.0008) [2023-12-27 00:49:01,910][105692] Updated weights for policy 0, policy_version 1294680 (0.0008) [2023-12-27 00:49:01,966][105692] Updated weights for policy 0, policy_version 1294690 (0.0008) [2023-12-27 00:49:02,027][105692] Updated weights for policy 0, policy_version 1294700 (0.0006) [2023-12-27 00:49:02,634][105620] Updated weights for policy 1, policy_version 1296356 (0.0008) [2023-12-27 00:49:02,685][105620] Updated weights for policy 1, policy_version 1296366 (0.0008) [2023-12-27 00:49:02,734][105620] Updated weights for policy 1, policy_version 1296376 (0.0008) [2023-12-27 00:49:02,744][105692] Updated weights for policy 0, policy_version 1294710 (0.0007) [2023-12-27 00:49:02,803][105692] Updated weights for policy 0, policy_version 1294720 (0.0007) [2023-12-27 00:49:02,850][105692] Updated weights for policy 0, policy_version 1294730 (0.0009) [2023-12-27 00:49:03,520][105692] Updated weights for policy 0, policy_version 1294740 (0.0008) [2023-12-27 00:49:03,543][105620] Updated weights for policy 1, policy_version 1296386 (0.0007) [2023-12-27 00:49:03,570][105692] Updated weights for policy 0, policy_version 1294750 (0.0007) [2023-12-27 00:49:03,592][105620] Updated weights for policy 1, policy_version 1296396 (0.0007) [2023-12-27 00:49:03,614][105692] Updated weights for policy 0, policy_version 1294760 (0.0006) [2023-12-27 00:49:03,640][105620] Updated weights for policy 1, policy_version 1296406 (0.0007) [2023-12-27 00:49:03,688][105620] Updated weights for policy 1, policy_version 1296416 (0.0007) [2023-12-27 00:49:04,333][105692] Updated weights for policy 0, policy_version 1294770 (0.0007) [2023-12-27 00:49:04,392][105692] Updated weights for policy 0, policy_version 1294780 (0.0010) [2023-12-27 00:49:04,456][105692] Updated weights for policy 0, policy_version 1294790 (0.0010) [2023-12-27 00:49:04,521][105692] Updated weights for policy 0, policy_version 1294800 (0.0008) [2023-12-27 00:49:04,527][105620] Updated weights for policy 1, policy_version 1296426 (0.0008) [2023-12-27 00:49:04,587][105620] Updated weights for policy 1, policy_version 1296436 (0.0009) [2023-12-27 00:49:04,639][105620] Updated weights for policy 1, policy_version 1296446 (0.0010) [2023-12-27 00:49:05,127][105692] Updated weights for policy 0, policy_version 1294810 (0.0005) [2023-12-27 00:49:05,188][105692] Updated weights for policy 0, policy_version 1294820 (0.0006) [2023-12-27 00:49:05,245][105692] Updated weights for policy 0, policy_version 1294830 (0.0005) [2023-12-27 00:49:05,454][105620] Updated weights for policy 1, policy_version 1296456 (0.0008) [2023-12-27 00:49:05,515][105620] Updated weights for policy 1, policy_version 1296466 (0.0008) [2023-12-27 00:49:05,582][105620] Updated weights for policy 1, policy_version 1296476 (0.0008) [2023-12-27 00:49:05,815][105692] Updated weights for policy 0, policy_version 1294840 (0.0006) [2023-12-27 00:49:05,874][105692] Updated weights for policy 0, policy_version 1294850 (0.0005) [2023-12-27 00:49:05,942][105692] Updated weights for policy 0, policy_version 1294860 (0.0007) [2023-12-27 00:49:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 663478272. Throughput: 0: 9775.4, 1: 9640.6. Samples: 663463244. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:49:06,063][104569] Avg episode reward: [(0, '8824.751'), (1, '9171.563')] [2023-12-27 00:49:06,161][105620] Updated weights for policy 1, policy_version 1296486 (0.0009) [2023-12-27 00:49:06,225][105620] Updated weights for policy 1, policy_version 1296496 (0.0007) [2023-12-27 00:49:06,288][105620] Updated weights for policy 1, policy_version 1296506 (0.0006) [2023-12-27 00:49:06,637][105692] Updated weights for policy 0, policy_version 1294870 (0.0008) [2023-12-27 00:49:06,696][105692] Updated weights for policy 0, policy_version 1294880 (0.0010) [2023-12-27 00:49:06,761][105692] Updated weights for policy 0, policy_version 1294890 (0.0010) [2023-12-27 00:49:06,903][105620] Updated weights for policy 1, policy_version 1296516 (0.0008) [2023-12-27 00:49:06,969][105620] Updated weights for policy 1, policy_version 1296526 (0.0007) [2023-12-27 00:49:07,028][105620] Updated weights for policy 1, policy_version 1296536 (0.0006) [2023-12-27 00:49:07,405][105692] Updated weights for policy 0, policy_version 1294900 (0.0010) [2023-12-27 00:49:07,457][105692] Updated weights for policy 0, policy_version 1294910 (0.0010) [2023-12-27 00:49:07,506][105692] Updated weights for policy 0, policy_version 1294920 (0.0010) [2023-12-27 00:49:07,707][105620] Updated weights for policy 1, policy_version 1296546 (0.0006) [2023-12-27 00:49:07,771][105620] Updated weights for policy 1, policy_version 1296556 (0.0006) [2023-12-27 00:49:07,836][105620] Updated weights for policy 1, policy_version 1296566 (0.0005) [2023-12-27 00:49:07,903][105620] Updated weights for policy 1, policy_version 1296576 (0.0006) [2023-12-27 00:49:08,263][105692] Updated weights for policy 0, policy_version 1294930 (0.0009) [2023-12-27 00:49:08,311][105692] Updated weights for policy 0, policy_version 1294940 (0.0006) [2023-12-27 00:49:08,379][105692] Updated weights for policy 0, policy_version 1294950 (0.0008) [2023-12-27 00:49:08,416][105620] Updated weights for policy 1, policy_version 1296586 (0.0008) [2023-12-27 00:49:08,439][105692] Updated weights for policy 0, policy_version 1294960 (0.0011) [2023-12-27 00:49:08,475][105620] Updated weights for policy 1, policy_version 1296596 (0.0008) [2023-12-27 00:49:08,530][105620] Updated weights for policy 1, policy_version 1296606 (0.0008) [2023-12-27 00:49:09,132][105620] Updated weights for policy 1, policy_version 1296616 (0.0006) [2023-12-27 00:49:09,172][105692] Updated weights for policy 0, policy_version 1294970 (0.0011) [2023-12-27 00:49:09,184][105620] Updated weights for policy 1, policy_version 1296626 (0.0005) [2023-12-27 00:49:09,229][105692] Updated weights for policy 0, policy_version 1294980 (0.0010) [2023-12-27 00:49:09,242][105620] Updated weights for policy 1, policy_version 1296636 (0.0007) [2023-12-27 00:49:09,299][105692] Updated weights for policy 0, policy_version 1294990 (0.0008) [2023-12-27 00:49:09,928][105620] Updated weights for policy 1, policy_version 1296646 (0.0009) [2023-12-27 00:49:09,989][105620] Updated weights for policy 1, policy_version 1296656 (0.0008) [2023-12-27 00:49:10,052][105620] Updated weights for policy 1, policy_version 1296666 (0.0008) [2023-12-27 00:49:10,088][105692] Updated weights for policy 0, policy_version 1295000 (0.0008) [2023-12-27 00:49:10,150][105692] Updated weights for policy 0, policy_version 1295010 (0.0008) [2023-12-27 00:49:10,209][105692] Updated weights for policy 0, policy_version 1295020 (0.0008) [2023-12-27 00:49:10,760][105620] Updated weights for policy 1, policy_version 1296676 (0.0008) [2023-12-27 00:49:10,826][105620] Updated weights for policy 1, policy_version 1296686 (0.0009) [2023-12-27 00:49:10,891][105620] Updated weights for policy 1, policy_version 1296696 (0.0009) [2023-12-27 00:49:10,978][105692] Updated weights for policy 0, policy_version 1295030 (0.0009) [2023-12-27 00:49:11,034][105692] Updated weights for policy 0, policy_version 1295040 (0.0009) [2023-12-27 00:49:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 663576576. Throughput: 0: 9845.1, 1: 9710.4. Samples: 663586168. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:49:11,063][104569] Avg episode reward: [(0, '8914.775'), (1, '9081.581')] [2023-12-27 00:49:11,089][105692] Updated weights for policy 0, policy_version 1295050 (0.0009) [2023-12-27 00:49:11,653][105620] Updated weights for policy 1, policy_version 1296706 (0.0006) [2023-12-27 00:49:11,720][105620] Updated weights for policy 1, policy_version 1296716 (0.0008) [2023-12-27 00:49:11,785][105620] Updated weights for policy 1, policy_version 1296726 (0.0008) [2023-12-27 00:49:11,850][105620] Updated weights for policy 1, policy_version 1296736 (0.0008) [2023-12-27 00:49:11,884][105692] Updated weights for policy 0, policy_version 1295060 (0.0009) [2023-12-27 00:49:11,947][105692] Updated weights for policy 0, policy_version 1295070 (0.0011) [2023-12-27 00:49:12,020][105692] Updated weights for policy 0, policy_version 1295080 (0.0011) [2023-12-27 00:49:12,536][105620] Updated weights for policy 1, policy_version 1296746 (0.0011) [2023-12-27 00:49:12,591][105620] Updated weights for policy 1, policy_version 1296756 (0.0010) [2023-12-27 00:49:12,639][105620] Updated weights for policy 1, policy_version 1296766 (0.0010) [2023-12-27 00:49:12,773][105692] Updated weights for policy 0, policy_version 1295090 (0.0009) [2023-12-27 00:49:12,834][105692] Updated weights for policy 0, policy_version 1295100 (0.0009) [2023-12-27 00:49:12,888][105692] Updated weights for policy 0, policy_version 1295110 (0.0008) [2023-12-27 00:49:12,937][105692] Updated weights for policy 0, policy_version 1295120 (0.0008) [2023-12-27 00:49:13,388][105620] Updated weights for policy 1, policy_version 1296776 (0.0006) [2023-12-27 00:49:13,447][105620] Updated weights for policy 1, policy_version 1296786 (0.0006) [2023-12-27 00:49:13,511][105620] Updated weights for policy 1, policy_version 1296796 (0.0010) [2023-12-27 00:49:13,720][105692] Updated weights for policy 0, policy_version 1295130 (0.0008) [2023-12-27 00:49:13,764][105692] Updated weights for policy 0, policy_version 1295140 (0.0008) [2023-12-27 00:49:13,812][105692] Updated weights for policy 0, policy_version 1295150 (0.0007) [2023-12-27 00:49:14,128][105620] Updated weights for policy 1, policy_version 1296806 (0.0007) [2023-12-27 00:49:14,178][105620] Updated weights for policy 1, policy_version 1296816 (0.0008) [2023-12-27 00:49:14,230][105620] Updated weights for policy 1, policy_version 1296826 (0.0008) [2023-12-27 00:49:14,685][105692] Updated weights for policy 0, policy_version 1295160 (0.0009) [2023-12-27 00:49:14,740][105692] Updated weights for policy 0, policy_version 1295170 (0.0008) [2023-12-27 00:49:14,807][105692] Updated weights for policy 0, policy_version 1295180 (0.0009) [2023-12-27 00:49:14,876][105620] Updated weights for policy 1, policy_version 1296836 (0.0008) [2023-12-27 00:49:14,932][105620] Updated weights for policy 1, policy_version 1296846 (0.0011) [2023-12-27 00:49:14,991][105620] Updated weights for policy 1, policy_version 1296856 (0.0010) [2023-12-27 00:49:15,590][105620] Updated weights for policy 1, policy_version 1296866 (0.0010) [2023-12-27 00:49:15,638][105620] Updated weights for policy 1, policy_version 1296876 (0.0011) [2023-12-27 00:49:15,673][105692] Updated weights for policy 0, policy_version 1295190 (0.0008) [2023-12-27 00:49:15,693][105620] Updated weights for policy 1, policy_version 1296886 (0.0010) [2023-12-27 00:49:15,731][105692] Updated weights for policy 0, policy_version 1295200 (0.0005) [2023-12-27 00:49:15,748][105620] Updated weights for policy 1, policy_version 1296896 (0.0011) [2023-12-27 00:49:15,793][105692] Updated weights for policy 0, policy_version 1295210 (0.0007) [2023-12-27 00:49:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.4, 300 sec: 19438.7). Total num frames: 663674880. Throughput: 0: 9730.1, 1: 9683.8. Samples: 663642484. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:49:16,062][104569] Avg episode reward: [(0, '9003.609'), (1, '9173.521')] [2023-12-27 00:49:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001295216_331628544.pth... [2023-12-27 00:49:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001296896_332046336.pth... [2023-12-27 00:49:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001294096_331341824.pth [2023-12-27 00:49:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001295776_331759616.pth [2023-12-27 00:49:16,451][105620] Updated weights for policy 1, policy_version 1296906 (0.0007) [2023-12-27 00:49:16,501][105620] Updated weights for policy 1, policy_version 1296916 (0.0006) [2023-12-27 00:49:16,551][105620] Updated weights for policy 1, policy_version 1296926 (0.0006) [2023-12-27 00:49:16,599][105692] Updated weights for policy 0, policy_version 1295220 (0.0009) [2023-12-27 00:49:16,657][105692] Updated weights for policy 0, policy_version 1295230 (0.0010) [2023-12-27 00:49:16,703][105692] Updated weights for policy 0, policy_version 1295240 (0.0011) [2023-12-27 00:49:17,127][105620] Updated weights for policy 1, policy_version 1296936 (0.0008) [2023-12-27 00:49:17,185][105620] Updated weights for policy 1, policy_version 1296946 (0.0009) [2023-12-27 00:49:17,239][105620] Updated weights for policy 1, policy_version 1296956 (0.0009) [2023-12-27 00:49:17,467][105692] Updated weights for policy 0, policy_version 1295250 (0.0009) [2023-12-27 00:49:17,527][105692] Updated weights for policy 0, policy_version 1295260 (0.0008) [2023-12-27 00:49:17,587][105692] Updated weights for policy 0, policy_version 1295270 (0.0009) [2023-12-27 00:49:17,638][105692] Updated weights for policy 0, policy_version 1295280 (0.0009) [2023-12-27 00:49:17,994][105620] Updated weights for policy 1, policy_version 1296966 (0.0009) [2023-12-27 00:49:18,045][105620] Updated weights for policy 1, policy_version 1296976 (0.0008) [2023-12-27 00:49:18,096][105620] Updated weights for policy 1, policy_version 1296986 (0.0009) [2023-12-27 00:49:18,356][105692] Updated weights for policy 0, policy_version 1295290 (0.0009) [2023-12-27 00:49:18,424][105692] Updated weights for policy 0, policy_version 1295300 (0.0010) [2023-12-27 00:49:18,477][105692] Updated weights for policy 0, policy_version 1295310 (0.0008) [2023-12-27 00:49:18,792][105620] Updated weights for policy 1, policy_version 1296996 (0.0007) [2023-12-27 00:49:18,841][105620] Updated weights for policy 1, policy_version 1297006 (0.0005) [2023-12-27 00:49:18,891][105620] Updated weights for policy 1, policy_version 1297016 (0.0005) [2023-12-27 00:49:19,374][105692] Updated weights for policy 0, policy_version 1295320 (0.0009) [2023-12-27 00:49:19,431][105692] Updated weights for policy 0, policy_version 1295330 (0.0010) [2023-12-27 00:49:19,486][105692] Updated weights for policy 0, policy_version 1295340 (0.0010) [2023-12-27 00:49:19,496][105620] Updated weights for policy 1, policy_version 1297026 (0.0007) [2023-12-27 00:49:19,559][105620] Updated weights for policy 1, policy_version 1297036 (0.0011) [2023-12-27 00:49:19,624][105620] Updated weights for policy 1, policy_version 1297046 (0.0010) [2023-12-27 00:49:19,690][105620] Updated weights for policy 1, policy_version 1297056 (0.0011) [2023-12-27 00:49:20,243][105692] Updated weights for policy 0, policy_version 1295350 (0.0009) [2023-12-27 00:49:20,304][105692] Updated weights for policy 0, policy_version 1295360 (0.0008) [2023-12-27 00:49:20,371][105692] Updated weights for policy 0, policy_version 1295370 (0.0010) [2023-12-27 00:49:20,436][105620] Updated weights for policy 1, policy_version 1297066 (0.0009) [2023-12-27 00:49:20,492][105620] Updated weights for policy 1, policy_version 1297076 (0.0009) [2023-12-27 00:49:20,549][105620] Updated weights for policy 1, policy_version 1297086 (0.0009) [2023-12-27 00:49:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 663764992. Throughput: 0: 9654.4, 1: 9707.3. Samples: 663758208. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:49:21,062][104569] Avg episode reward: [(0, '9269.205'), (1, '9178.693')] [2023-12-27 00:49:21,125][105692] Updated weights for policy 0, policy_version 1295380 (0.0009) [2023-12-27 00:49:21,187][105692] Updated weights for policy 0, policy_version 1295390 (0.0008) [2023-12-27 00:49:21,254][105692] Updated weights for policy 0, policy_version 1295400 (0.0006) [2023-12-27 00:49:21,403][105620] Updated weights for policy 1, policy_version 1297096 (0.0009) [2023-12-27 00:49:21,471][105620] Updated weights for policy 1, policy_version 1297106 (0.0008) [2023-12-27 00:49:21,538][105620] Updated weights for policy 1, policy_version 1297116 (0.0008) [2023-12-27 00:49:21,994][105692] Updated weights for policy 0, policy_version 1295410 (0.0007) [2023-12-27 00:49:22,055][105692] Updated weights for policy 0, policy_version 1295420 (0.0006) [2023-12-27 00:49:22,123][105692] Updated weights for policy 0, policy_version 1295430 (0.0009) [2023-12-27 00:49:22,183][105692] Updated weights for policy 0, policy_version 1295440 (0.0010) [2023-12-27 00:49:22,310][105620] Updated weights for policy 1, policy_version 1297126 (0.0009) [2023-12-27 00:49:22,380][105620] Updated weights for policy 1, policy_version 1297136 (0.0009) [2023-12-27 00:49:22,443][105620] Updated weights for policy 1, policy_version 1297146 (0.0009) [2023-12-27 00:49:22,944][105692] Updated weights for policy 0, policy_version 1295450 (0.0008) [2023-12-27 00:49:23,004][105692] Updated weights for policy 0, policy_version 1295460 (0.0009) [2023-12-27 00:49:23,060][105692] Updated weights for policy 0, policy_version 1295470 (0.0009) [2023-12-27 00:49:23,221][105620] Updated weights for policy 1, policy_version 1297156 (0.0009) [2023-12-27 00:49:23,273][105620] Updated weights for policy 1, policy_version 1297166 (0.0009) [2023-12-27 00:49:23,324][105620] Updated weights for policy 1, policy_version 1297176 (0.0009) [2023-12-27 00:49:23,801][105692] Updated weights for policy 0, policy_version 1295480 (0.0006) [2023-12-27 00:49:23,856][105692] Updated weights for policy 0, policy_version 1295490 (0.0005) [2023-12-27 00:49:23,907][105692] Updated weights for policy 0, policy_version 1295500 (0.0005) [2023-12-27 00:49:24,191][105620] Updated weights for policy 1, policy_version 1297186 (0.0009) [2023-12-27 00:49:24,245][105620] Updated weights for policy 1, policy_version 1297196 (0.0009) [2023-12-27 00:49:24,299][105620] Updated weights for policy 1, policy_version 1297206 (0.0009) [2023-12-27 00:49:24,353][105620] Updated weights for policy 1, policy_version 1297216 (0.0009) [2023-12-27 00:49:24,503][105692] Updated weights for policy 0, policy_version 1295510 (0.0005) [2023-12-27 00:49:24,558][105692] Updated weights for policy 0, policy_version 1295520 (0.0008) [2023-12-27 00:49:24,606][105692] Updated weights for policy 0, policy_version 1295530 (0.0009) [2023-12-27 00:49:25,150][105620] Updated weights for policy 1, policy_version 1297226 (0.0009) [2023-12-27 00:49:25,204][105620] Updated weights for policy 1, policy_version 1297236 (0.0009) [2023-12-27 00:49:25,258][105620] Updated weights for policy 1, policy_version 1297246 (0.0009) [2023-12-27 00:49:25,295][105692] Updated weights for policy 0, policy_version 1295540 (0.0009) [2023-12-27 00:49:25,346][105692] Updated weights for policy 0, policy_version 1295550 (0.0009) [2023-12-27 00:49:25,392][105692] Updated weights for policy 0, policy_version 1295560 (0.0008) [2023-12-27 00:49:25,984][105620] Updated weights for policy 1, policy_version 1297256 (0.0009) [2023-12-27 00:49:26,033][105620] Updated weights for policy 1, policy_version 1297266 (0.0008) [2023-12-27 00:49:26,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 663855104. Throughput: 0: 9556.5, 1: 9686.8. Samples: 663869520. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:49:26,063][104569] Avg episode reward: [(0, '9357.244'), (1, '8992.908')] [2023-12-27 00:49:26,080][105620] Updated weights for policy 1, policy_version 1297276 (0.0009) [2023-12-27 00:49:26,172][105692] Updated weights for policy 0, policy_version 1295570 (0.0009) [2023-12-27 00:49:26,222][105692] Updated weights for policy 0, policy_version 1295580 (0.0009) [2023-12-27 00:49:26,279][105692] Updated weights for policy 0, policy_version 1295590 (0.0009) [2023-12-27 00:49:26,325][105692] Updated weights for policy 0, policy_version 1295600 (0.0008) [2023-12-27 00:49:26,755][105620] Updated weights for policy 1, policy_version 1297286 (0.0009) [2023-12-27 00:49:26,815][105620] Updated weights for policy 1, policy_version 1297296 (0.0009) [2023-12-27 00:49:26,875][105620] Updated weights for policy 1, policy_version 1297306 (0.0008) [2023-12-27 00:49:27,138][105692] Updated weights for policy 0, policy_version 1295610 (0.0009) [2023-12-27 00:49:27,197][105692] Updated weights for policy 0, policy_version 1295620 (0.0009) [2023-12-27 00:49:27,246][105692] Updated weights for policy 0, policy_version 1295630 (0.0008) [2023-12-27 00:49:27,613][105620] Updated weights for policy 1, policy_version 1297316 (0.0009) [2023-12-27 00:49:27,667][105620] Updated weights for policy 1, policy_version 1297326 (0.0008) [2023-12-27 00:49:27,717][105620] Updated weights for policy 1, policy_version 1297336 (0.0008) [2023-12-27 00:49:27,995][105692] Updated weights for policy 0, policy_version 1295640 (0.0009) [2023-12-27 00:49:28,051][105692] Updated weights for policy 0, policy_version 1295650 (0.0009) [2023-12-27 00:49:28,105][105692] Updated weights for policy 0, policy_version 1295660 (0.0009) [2023-12-27 00:49:28,475][105620] Updated weights for policy 1, policy_version 1297346 (0.0009) [2023-12-27 00:49:28,522][105620] Updated weights for policy 1, policy_version 1297356 (0.0009) [2023-12-27 00:49:28,573][105620] Updated weights for policy 1, policy_version 1297366 (0.0008) [2023-12-27 00:49:28,637][105620] Updated weights for policy 1, policy_version 1297376 (0.0009) [2023-12-27 00:49:28,852][105692] Updated weights for policy 0, policy_version 1295670 (0.0009) [2023-12-27 00:49:28,904][105692] Updated weights for policy 0, policy_version 1295680 (0.0009) [2023-12-27 00:49:28,950][105692] Updated weights for policy 0, policy_version 1295690 (0.0008) [2023-12-27 00:49:29,360][105620] Updated weights for policy 1, policy_version 1297386 (0.0010) [2023-12-27 00:49:29,420][105620] Updated weights for policy 1, policy_version 1297396 (0.0009) [2023-12-27 00:49:29,475][105620] Updated weights for policy 1, policy_version 1297406 (0.0009) [2023-12-27 00:49:29,753][105692] Updated weights for policy 0, policy_version 1295700 (0.0008) [2023-12-27 00:49:29,812][105692] Updated weights for policy 0, policy_version 1295710 (0.0006) [2023-12-27 00:49:29,877][105692] Updated weights for policy 0, policy_version 1295720 (0.0006) [2023-12-27 00:49:30,227][105620] Updated weights for policy 1, policy_version 1297416 (0.0008) [2023-12-27 00:49:30,277][105620] Updated weights for policy 1, policy_version 1297426 (0.0008) [2023-12-27 00:49:30,323][105620] Updated weights for policy 1, policy_version 1297436 (0.0009) [2023-12-27 00:49:30,530][105692] Updated weights for policy 0, policy_version 1295730 (0.0007) [2023-12-27 00:49:30,577][105692] Updated weights for policy 0, policy_version 1295740 (0.0009) [2023-12-27 00:49:30,628][105692] Updated weights for policy 0, policy_version 1295750 (0.0009) [2023-12-27 00:49:30,674][105692] Updated weights for policy 0, policy_version 1295760 (0.0008) [2023-12-27 00:49:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 663953408. Throughput: 0: 9557.9, 1: 9614.9. Samples: 663926196. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:49:31,062][104569] Avg episode reward: [(0, '9267.074'), (1, '8992.774')] [2023-12-27 00:49:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001295760_331767808.pth... [2023-12-27 00:49:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001297440_332185600.pth... [2023-12-27 00:49:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001294672_331489280.pth [2023-12-27 00:49:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001296320_331898880.pth [2023-12-27 00:49:31,130][105620] Updated weights for policy 1, policy_version 1297446 (0.0008) [2023-12-27 00:49:31,194][105620] Updated weights for policy 1, policy_version 1297456 (0.0008) [2023-12-27 00:49:31,260][105620] Updated weights for policy 1, policy_version 1297466 (0.0009) [2023-12-27 00:49:31,441][105692] Updated weights for policy 0, policy_version 1295770 (0.0008) [2023-12-27 00:49:31,486][105692] Updated weights for policy 0, policy_version 1295780 (0.0008) [2023-12-27 00:49:31,537][105692] Updated weights for policy 0, policy_version 1295790 (0.0007) [2023-12-27 00:49:31,995][105620] Updated weights for policy 1, policy_version 1297476 (0.0009) [2023-12-27 00:49:32,042][105620] Updated weights for policy 1, policy_version 1297486 (0.0009) [2023-12-27 00:49:32,101][105620] Updated weights for policy 1, policy_version 1297496 (0.0009) [2023-12-27 00:49:32,247][105692] Updated weights for policy 0, policy_version 1295800 (0.0008) [2023-12-27 00:49:32,316][105692] Updated weights for policy 0, policy_version 1295810 (0.0005) [2023-12-27 00:49:32,381][105692] Updated weights for policy 0, policy_version 1295820 (0.0007) [2023-12-27 00:49:32,816][105620] Updated weights for policy 1, policy_version 1297506 (0.0008) [2023-12-27 00:49:32,865][105620] Updated weights for policy 1, policy_version 1297516 (0.0005) [2023-12-27 00:49:32,916][105620] Updated weights for policy 1, policy_version 1297526 (0.0005) [2023-12-27 00:49:32,970][105620] Updated weights for policy 1, policy_version 1297536 (0.0006) [2023-12-27 00:49:33,131][105692] Updated weights for policy 0, policy_version 1295830 (0.0009) [2023-12-27 00:49:33,194][105692] Updated weights for policy 0, policy_version 1295840 (0.0010) [2023-12-27 00:49:33,252][105692] Updated weights for policy 0, policy_version 1295850 (0.0009) [2023-12-27 00:49:33,534][105620] Updated weights for policy 1, policy_version 1297546 (0.0005) [2023-12-27 00:49:33,592][105620] Updated weights for policy 1, policy_version 1297556 (0.0005) [2023-12-27 00:49:33,651][105620] Updated weights for policy 1, policy_version 1297566 (0.0007) [2023-12-27 00:49:34,003][105692] Updated weights for policy 0, policy_version 1295860 (0.0007) [2023-12-27 00:49:34,061][105692] Updated weights for policy 0, policy_version 1295870 (0.0009) [2023-12-27 00:49:34,127][105692] Updated weights for policy 0, policy_version 1295880 (0.0010) [2023-12-27 00:49:34,398][105620] Updated weights for policy 1, policy_version 1297576 (0.0010) [2023-12-27 00:49:34,460][105620] Updated weights for policy 1, policy_version 1297586 (0.0008) [2023-12-27 00:49:34,522][105620] Updated weights for policy 1, policy_version 1297596 (0.0009) [2023-12-27 00:49:34,773][105692] Updated weights for policy 0, policy_version 1295890 (0.0009) [2023-12-27 00:49:34,839][105692] Updated weights for policy 0, policy_version 1295900 (0.0009) [2023-12-27 00:49:34,900][105692] Updated weights for policy 0, policy_version 1295910 (0.0009) [2023-12-27 00:49:34,959][105692] Updated weights for policy 0, policy_version 1295920 (0.0006) [2023-12-27 00:49:35,300][105620] Updated weights for policy 1, policy_version 1297606 (0.0009) [2023-12-27 00:49:35,353][105620] Updated weights for policy 1, policy_version 1297616 (0.0009) [2023-12-27 00:49:35,416][105620] Updated weights for policy 1, policy_version 1297626 (0.0008) [2023-12-27 00:49:35,629][105692] Updated weights for policy 0, policy_version 1295930 (0.0009) [2023-12-27 00:49:35,690][105692] Updated weights for policy 0, policy_version 1295940 (0.0009) [2023-12-27 00:49:35,757][105692] Updated weights for policy 0, policy_version 1295950 (0.0009) [2023-12-27 00:49:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.3, 300 sec: 19383.1). Total num frames: 664051712. Throughput: 0: 9581.5, 1: 9679.8. Samples: 664042736. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:49:36,062][104569] Avg episode reward: [(0, '9265.727'), (1, '9172.416')] [2023-12-27 00:49:36,237][105620] Updated weights for policy 1, policy_version 1297636 (0.0009) [2023-12-27 00:49:36,299][105620] Updated weights for policy 1, policy_version 1297646 (0.0009) [2023-12-27 00:49:36,356][105620] Updated weights for policy 1, policy_version 1297656 (0.0009) [2023-12-27 00:49:36,396][105692] Updated weights for policy 0, policy_version 1295960 (0.0010) [2023-12-27 00:49:36,458][105692] Updated weights for policy 0, policy_version 1295970 (0.0006) [2023-12-27 00:49:36,528][105692] Updated weights for policy 0, policy_version 1295980 (0.0005) [2023-12-27 00:49:37,106][105692] Updated weights for policy 0, policy_version 1295990 (0.0005) [2023-12-27 00:49:37,164][105692] Updated weights for policy 0, policy_version 1296000 (0.0006) [2023-12-27 00:49:37,211][105620] Updated weights for policy 1, policy_version 1297666 (0.0009) [2023-12-27 00:49:37,221][105692] Updated weights for policy 0, policy_version 1296010 (0.0009) [2023-12-27 00:49:37,261][105620] Updated weights for policy 1, policy_version 1297676 (0.0007) [2023-12-27 00:49:37,311][105620] Updated weights for policy 1, policy_version 1297686 (0.0008) [2023-12-27 00:49:37,365][105620] Updated weights for policy 1, policy_version 1297696 (0.0009) [2023-12-27 00:49:37,937][105692] Updated weights for policy 0, policy_version 1296020 (0.0007) [2023-12-27 00:49:37,995][105692] Updated weights for policy 0, policy_version 1296030 (0.0009) [2023-12-27 00:49:38,054][105692] Updated weights for policy 0, policy_version 1296040 (0.0009) [2023-12-27 00:49:38,135][105620] Updated weights for policy 1, policy_version 1297706 (0.0008) [2023-12-27 00:49:38,183][105620] Updated weights for policy 1, policy_version 1297716 (0.0009) [2023-12-27 00:49:38,234][105620] Updated weights for policy 1, policy_version 1297726 (0.0008) [2023-12-27 00:49:38,746][105692] Updated weights for policy 0, policy_version 1296050 (0.0008) [2023-12-27 00:49:38,802][105692] Updated weights for policy 0, policy_version 1296060 (0.0009) [2023-12-27 00:49:38,861][105692] Updated weights for policy 0, policy_version 1296070 (0.0007) [2023-12-27 00:49:38,914][105692] Updated weights for policy 0, policy_version 1296080 (0.0005) [2023-12-27 00:49:39,074][105620] Updated weights for policy 1, policy_version 1297736 (0.0009) [2023-12-27 00:49:39,131][105620] Updated weights for policy 1, policy_version 1297746 (0.0010) [2023-12-27 00:49:39,194][105620] Updated weights for policy 1, policy_version 1297756 (0.0010) [2023-12-27 00:49:39,626][105692] Updated weights for policy 0, policy_version 1296090 (0.0008) [2023-12-27 00:49:39,679][105692] Updated weights for policy 0, policy_version 1296100 (0.0008) [2023-12-27 00:49:39,736][105692] Updated weights for policy 0, policy_version 1296110 (0.0008) [2023-12-27 00:49:39,986][105620] Updated weights for policy 1, policy_version 1297766 (0.0011) [2023-12-27 00:49:40,039][105620] Updated weights for policy 1, policy_version 1297776 (0.0010) [2023-12-27 00:49:40,109][105620] Updated weights for policy 1, policy_version 1297786 (0.0010) [2023-12-27 00:49:40,540][105692] Updated weights for policy 0, policy_version 1296120 (0.0008) [2023-12-27 00:49:40,606][105692] Updated weights for policy 0, policy_version 1296130 (0.0008) [2023-12-27 00:49:40,668][105692] Updated weights for policy 0, policy_version 1296140 (0.0009) [2023-12-27 00:49:40,850][105620] Updated weights for policy 1, policy_version 1297796 (0.0010) [2023-12-27 00:49:40,914][105620] Updated weights for policy 1, policy_version 1297806 (0.0010) [2023-12-27 00:49:40,975][105620] Updated weights for policy 1, policy_version 1297816 (0.0010) [2023-12-27 00:49:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 664150016. Throughput: 0: 9555.8, 1: 9669.8. Samples: 664155636. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:49:41,063][104569] Avg episode reward: [(0, '9264.276'), (1, '9263.941')] [2023-12-27 00:49:41,450][105692] Updated weights for policy 0, policy_version 1296150 (0.0009) [2023-12-27 00:49:41,517][105692] Updated weights for policy 0, policy_version 1296160 (0.0008) [2023-12-27 00:49:41,571][105692] Updated weights for policy 0, policy_version 1296170 (0.0008) [2023-12-27 00:49:41,781][105620] Updated weights for policy 1, policy_version 1297826 (0.0010) [2023-12-27 00:49:41,831][105620] Updated weights for policy 1, policy_version 1297836 (0.0011) [2023-12-27 00:49:41,877][105620] Updated weights for policy 1, policy_version 1297846 (0.0010) [2023-12-27 00:49:41,931][105620] Updated weights for policy 1, policy_version 1297856 (0.0010) [2023-12-27 00:49:42,214][105692] Updated weights for policy 0, policy_version 1296180 (0.0009) [2023-12-27 00:49:42,277][105692] Updated weights for policy 0, policy_version 1296190 (0.0010) [2023-12-27 00:49:42,329][105692] Updated weights for policy 0, policy_version 1296200 (0.0010) [2023-12-27 00:49:42,622][105620] Updated weights for policy 1, policy_version 1297866 (0.0007) [2023-12-27 00:49:42,677][105620] Updated weights for policy 1, policy_version 1297876 (0.0010) [2023-12-27 00:49:42,733][105620] Updated weights for policy 1, policy_version 1297886 (0.0010) [2023-12-27 00:49:43,142][105692] Updated weights for policy 0, policy_version 1296210 (0.0008) [2023-12-27 00:49:43,204][105692] Updated weights for policy 0, policy_version 1296220 (0.0008) [2023-12-27 00:49:43,269][105692] Updated weights for policy 0, policy_version 1296230 (0.0010) [2023-12-27 00:49:43,331][105692] Updated weights for policy 0, policy_version 1296240 (0.0005) [2023-12-27 00:49:43,351][105620] Updated weights for policy 1, policy_version 1297896 (0.0006) [2023-12-27 00:49:43,417][105620] Updated weights for policy 1, policy_version 1297906 (0.0005) [2023-12-27 00:49:43,483][105620] Updated weights for policy 1, policy_version 1297916 (0.0007) [2023-12-27 00:49:43,883][105692] Updated weights for policy 0, policy_version 1296250 (0.0005) [2023-12-27 00:49:43,943][105692] Updated weights for policy 0, policy_version 1296260 (0.0005) [2023-12-27 00:49:43,996][105692] Updated weights for policy 0, policy_version 1296270 (0.0005) [2023-12-27 00:49:44,181][105620] Updated weights for policy 1, policy_version 1297926 (0.0010) [2023-12-27 00:49:44,234][105620] Updated weights for policy 1, policy_version 1297936 (0.0008) [2023-12-27 00:49:44,278][105620] Updated weights for policy 1, policy_version 1297946 (0.0009) [2023-12-27 00:49:44,524][105692] Updated weights for policy 0, policy_version 1296280 (0.0007) [2023-12-27 00:49:44,574][105692] Updated weights for policy 0, policy_version 1296290 (0.0006) [2023-12-27 00:49:44,636][105692] Updated weights for policy 0, policy_version 1296300 (0.0007) [2023-12-27 00:49:45,044][105620] Updated weights for policy 1, policy_version 1297956 (0.0010) [2023-12-27 00:49:45,110][105620] Updated weights for policy 1, policy_version 1297966 (0.0011) [2023-12-27 00:49:45,178][105620] Updated weights for policy 1, policy_version 1297976 (0.0007) [2023-12-27 00:49:45,325][105692] Updated weights for policy 0, policy_version 1296310 (0.0009) [2023-12-27 00:49:45,389][105692] Updated weights for policy 0, policy_version 1296320 (0.0011) [2023-12-27 00:49:45,449][105692] Updated weights for policy 0, policy_version 1296330 (0.0011) [2023-12-27 00:49:45,933][105620] Updated weights for policy 1, policy_version 1297986 (0.0008) [2023-12-27 00:49:46,002][105620] Updated weights for policy 1, policy_version 1297996 (0.0010) [2023-12-27 00:49:46,041][105692] Updated weights for policy 0, policy_version 1296340 (0.0009) [2023-12-27 00:49:46,058][105620] Updated weights for policy 1, policy_version 1298006 (0.0009) [2023-12-27 00:49:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19355.3). Total num frames: 664240128. Throughput: 0: 9526.3, 1: 9688.4. Samples: 664214316. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:49:46,062][104569] Avg episode reward: [(0, '9175.517'), (1, '9172.450')] [2023-12-27 00:49:46,103][105692] Updated weights for policy 0, policy_version 1296350 (0.0006) [2023-12-27 00:49:46,116][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001298016_332333056.pth... [2023-12-27 00:49:46,118][105620] Updated weights for policy 1, policy_version 1298016 (0.0007) [2023-12-27 00:49:46,119][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001296896_332046336.pth [2023-12-27 00:49:46,159][105692] Updated weights for policy 0, policy_version 1296360 (0.0007) [2023-12-27 00:49:46,209][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001296368_331923456.pth... [2023-12-27 00:49:46,215][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001295216_331628544.pth [2023-12-27 00:49:46,710][105692] Updated weights for policy 0, policy_version 1296370 (0.0007) [2023-12-27 00:49:46,772][105692] Updated weights for policy 0, policy_version 1296380 (0.0006) [2023-12-27 00:49:46,826][105692] Updated weights for policy 0, policy_version 1296390 (0.0007) [2023-12-27 00:49:46,841][105620] Updated weights for policy 1, policy_version 1298026 (0.0005) [2023-12-27 00:49:46,892][105620] Updated weights for policy 1, policy_version 1298036 (0.0005) [2023-12-27 00:49:46,952][105620] Updated weights for policy 1, policy_version 1298046 (0.0005) [2023-12-27 00:49:47,522][105692] Updated weights for policy 0, policy_version 1296401 (0.0008) [2023-12-27 00:49:47,525][105620] Updated weights for policy 1, policy_version 1298056 (0.0007) [2023-12-27 00:49:47,584][105620] Updated weights for policy 1, policy_version 1298066 (0.0005) [2023-12-27 00:49:47,586][105692] Updated weights for policy 0, policy_version 1296411 (0.0009) [2023-12-27 00:49:47,642][105692] Updated weights for policy 0, policy_version 1296421 (0.0008) [2023-12-27 00:49:47,644][105620] Updated weights for policy 1, policy_version 1298076 (0.0007) [2023-12-27 00:49:47,701][105692] Updated weights for policy 0, policy_version 1296431 (0.0008) [2023-12-27 00:49:48,251][105620] Updated weights for policy 1, policy_version 1298086 (0.0007) [2023-12-27 00:49:48,315][105620] Updated weights for policy 1, policy_version 1298096 (0.0006) [2023-12-27 00:49:48,378][105620] Updated weights for policy 1, policy_version 1298106 (0.0008) [2023-12-27 00:49:48,544][105692] Updated weights for policy 0, policy_version 1296441 (0.0009) [2023-12-27 00:49:48,601][105692] Updated weights for policy 0, policy_version 1296451 (0.0009) [2023-12-27 00:49:48,649][105692] Updated weights for policy 0, policy_version 1296461 (0.0009) [2023-12-27 00:49:49,070][105620] Updated weights for policy 1, policy_version 1298116 (0.0008) [2023-12-27 00:49:49,118][105620] Updated weights for policy 1, policy_version 1298126 (0.0009) [2023-12-27 00:49:49,165][105620] Updated weights for policy 1, policy_version 1298136 (0.0008) [2023-12-27 00:49:49,417][105692] Updated weights for policy 0, policy_version 1296471 (0.0008) [2023-12-27 00:49:49,484][105692] Updated weights for policy 0, policy_version 1296481 (0.0006) [2023-12-27 00:49:49,555][105692] Updated weights for policy 0, policy_version 1296491 (0.0006) [2023-12-27 00:49:50,026][105620] Updated weights for policy 1, policy_version 1298146 (0.0009) [2023-12-27 00:49:50,093][105620] Updated weights for policy 1, policy_version 1298156 (0.0009) [2023-12-27 00:49:50,156][105620] Updated weights for policy 1, policy_version 1298166 (0.0009) [2023-12-27 00:49:50,179][105692] Updated weights for policy 0, policy_version 1296501 (0.0007) [2023-12-27 00:49:50,222][105620] Updated weights for policy 1, policy_version 1298176 (0.0007) [2023-12-27 00:49:50,237][105692] Updated weights for policy 0, policy_version 1296511 (0.0007) [2023-12-27 00:49:50,293][105692] Updated weights for policy 0, policy_version 1296521 (0.0008) [2023-12-27 00:49:50,996][105620] Updated weights for policy 1, policy_version 1298186 (0.0008) [2023-12-27 00:49:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.3, 300 sec: 19383.1). Total num frames: 664338432. Throughput: 0: 9636.7, 1: 9746.2. Samples: 664335472. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:49:51,062][104569] Avg episode reward: [(0, '9089.468'), (1, '9263.792')] [2023-12-27 00:49:51,064][105620] Updated weights for policy 1, policy_version 1298196 (0.0008) [2023-12-27 00:49:51,070][105692] Updated weights for policy 0, policy_version 1296531 (0.0010) [2023-12-27 00:49:51,125][105620] Updated weights for policy 1, policy_version 1298206 (0.0007) [2023-12-27 00:49:51,133][105692] Updated weights for policy 0, policy_version 1296541 (0.0011) [2023-12-27 00:49:51,197][105692] Updated weights for policy 0, policy_version 1296551 (0.0011) [2023-12-27 00:49:51,919][105692] Updated weights for policy 0, policy_version 1296561 (0.0010) [2023-12-27 00:49:51,941][105620] Updated weights for policy 1, policy_version 1298216 (0.0011) [2023-12-27 00:49:51,982][105692] Updated weights for policy 0, policy_version 1296571 (0.0010) [2023-12-27 00:49:52,001][105620] Updated weights for policy 1, policy_version 1298226 (0.0011) [2023-12-27 00:49:52,045][105692] Updated weights for policy 0, policy_version 1296581 (0.0009) [2023-12-27 00:49:52,062][105620] Updated weights for policy 1, policy_version 1298236 (0.0011) [2023-12-27 00:49:52,108][105692] Updated weights for policy 0, policy_version 1296591 (0.0009) [2023-12-27 00:49:52,812][105620] Updated weights for policy 1, policy_version 1298246 (0.0007) [2023-12-27 00:49:52,876][105620] Updated weights for policy 1, policy_version 1298256 (0.0008) [2023-12-27 00:49:52,916][105692] Updated weights for policy 0, policy_version 1296601 (0.0010) [2023-12-27 00:49:52,938][105620] Updated weights for policy 1, policy_version 1298266 (0.0009) [2023-12-27 00:49:52,981][105692] Updated weights for policy 0, policy_version 1296611 (0.0008) [2023-12-27 00:49:53,041][105692] Updated weights for policy 0, policy_version 1296621 (0.0009) [2023-12-27 00:49:53,662][105620] Updated weights for policy 1, policy_version 1298276 (0.0006) [2023-12-27 00:49:53,718][105620] Updated weights for policy 1, policy_version 1298286 (0.0008) [2023-12-27 00:49:53,773][105620] Updated weights for policy 1, policy_version 1298296 (0.0009) [2023-12-27 00:49:53,790][105692] Updated weights for policy 0, policy_version 1296631 (0.0010) [2023-12-27 00:49:53,843][105692] Updated weights for policy 0, policy_version 1296641 (0.0006) [2023-12-27 00:49:53,899][105692] Updated weights for policy 0, policy_version 1296651 (0.0006) [2023-12-27 00:49:54,460][105692] Updated weights for policy 0, policy_version 1296661 (0.0005) [2023-12-27 00:49:54,529][105692] Updated weights for policy 0, policy_version 1296671 (0.0005) [2023-12-27 00:49:54,600][105692] Updated weights for policy 0, policy_version 1296681 (0.0007) [2023-12-27 00:49:54,622][105620] Updated weights for policy 1, policy_version 1298306 (0.0007) [2023-12-27 00:49:54,684][105620] Updated weights for policy 1, policy_version 1298316 (0.0006) [2023-12-27 00:49:54,749][105620] Updated weights for policy 1, policy_version 1298326 (0.0008) [2023-12-27 00:49:54,806][105620] Updated weights for policy 1, policy_version 1298336 (0.0009) [2023-12-27 00:49:55,106][105692] Updated weights for policy 0, policy_version 1296691 (0.0006) [2023-12-27 00:49:55,156][105692] Updated weights for policy 0, policy_version 1296701 (0.0005) [2023-12-27 00:49:55,215][105692] Updated weights for policy 0, policy_version 1296711 (0.0008) [2023-12-27 00:49:55,568][105620] Updated weights for policy 1, policy_version 1298346 (0.0008) [2023-12-27 00:49:55,628][105620] Updated weights for policy 1, policy_version 1298356 (0.0008) [2023-12-27 00:49:55,689][105620] Updated weights for policy 1, policy_version 1298366 (0.0009) [2023-12-27 00:49:55,864][105692] Updated weights for policy 0, policy_version 1296721 (0.0009) [2023-12-27 00:49:55,912][105692] Updated weights for policy 0, policy_version 1296731 (0.0005) [2023-12-27 00:49:55,971][105692] Updated weights for policy 0, policy_version 1296741 (0.0006) [2023-12-27 00:49:56,029][105692] Updated weights for policy 0, policy_version 1296751 (0.0005) [2023-12-27 00:49:56,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 664444928. Throughput: 0: 9652.2, 1: 9528.5. Samples: 664449300. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:49:56,063][104569] Avg episode reward: [(0, '8824.961'), (1, '9173.015')] [2023-12-27 00:49:56,434][105620] Updated weights for policy 1, policy_version 1298376 (0.0010) [2023-12-27 00:49:56,484][105620] Updated weights for policy 1, policy_version 1298386 (0.0010) [2023-12-27 00:49:56,535][105620] Updated weights for policy 1, policy_version 1298396 (0.0010) [2023-12-27 00:49:56,669][105692] Updated weights for policy 0, policy_version 1296761 (0.0005) [2023-12-27 00:49:56,720][105692] Updated weights for policy 0, policy_version 1296771 (0.0005) [2023-12-27 00:49:56,766][105692] Updated weights for policy 0, policy_version 1296781 (0.0005) [2023-12-27 00:49:57,105][105620] Updated weights for policy 1, policy_version 1298406 (0.0007) [2023-12-27 00:49:57,161][105620] Updated weights for policy 1, policy_version 1298416 (0.0005) [2023-12-27 00:49:57,210][105620] Updated weights for policy 1, policy_version 1298426 (0.0009) [2023-12-27 00:49:57,322][105692] Updated weights for policy 0, policy_version 1296791 (0.0005) [2023-12-27 00:49:57,384][105692] Updated weights for policy 0, policy_version 1296801 (0.0006) [2023-12-27 00:49:57,431][105692] Updated weights for policy 0, policy_version 1296811 (0.0010) [2023-12-27 00:49:57,909][105620] Updated weights for policy 1, policy_version 1298436 (0.0010) [2023-12-27 00:49:57,962][105620] Updated weights for policy 1, policy_version 1298446 (0.0009) [2023-12-27 00:49:58,007][105692] Updated weights for policy 0, policy_version 1296821 (0.0010) [2023-12-27 00:49:58,018][105620] Updated weights for policy 1, policy_version 1298456 (0.0008) [2023-12-27 00:49:58,067][105692] Updated weights for policy 0, policy_version 1296831 (0.0008) [2023-12-27 00:49:58,123][105692] Updated weights for policy 0, policy_version 1296841 (0.0007) [2023-12-27 00:49:58,770][105620] Updated weights for policy 1, policy_version 1298466 (0.0006) [2023-12-27 00:49:58,837][105620] Updated weights for policy 1, policy_version 1298476 (0.0008) [2023-12-27 00:49:58,914][105620] Updated weights for policy 1, policy_version 1298486 (0.0009) [2023-12-27 00:49:58,977][105692] Updated weights for policy 0, policy_version 1296851 (0.0009) [2023-12-27 00:49:59,039][105692] Updated weights for policy 0, policy_version 1296861 (0.0009) [2023-12-27 00:49:59,102][105692] Updated weights for policy 0, policy_version 1296871 (0.0009) [2023-12-27 00:49:59,759][105620] Updated weights for policy 1, policy_version 1298497 (0.0009) [2023-12-27 00:49:59,777][105692] Updated weights for policy 0, policy_version 1296881 (0.0009) [2023-12-27 00:49:59,816][105620] Updated weights for policy 1, policy_version 1298507 (0.0007) [2023-12-27 00:49:59,840][105692] Updated weights for policy 0, policy_version 1296891 (0.0008) [2023-12-27 00:49:59,878][105620] Updated weights for policy 1, policy_version 1298517 (0.0007) [2023-12-27 00:49:59,902][105692] Updated weights for policy 0, policy_version 1296901 (0.0006) [2023-12-27 00:49:59,948][105620] Updated weights for policy 1, policy_version 1298527 (0.0007) [2023-12-27 00:49:59,970][105692] Updated weights for policy 0, policy_version 1296911 (0.0007) [2023-12-27 00:50:00,628][105620] Updated weights for policy 1, policy_version 1298537 (0.0008) [2023-12-27 00:50:00,684][105620] Updated weights for policy 1, policy_version 1298547 (0.0010) [2023-12-27 00:50:00,711][105692] Updated weights for policy 0, policy_version 1296921 (0.0007) [2023-12-27 00:50:00,737][105620] Updated weights for policy 1, policy_version 1298557 (0.0010) [2023-12-27 00:50:00,764][105692] Updated weights for policy 0, policy_version 1296931 (0.0006) [2023-12-27 00:50:00,813][105692] Updated weights for policy 0, policy_version 1296941 (0.0008) [2023-12-27 00:50:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 664543232. Throughput: 0: 9784.4, 1: 9548.6. Samples: 664512468. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:50:01,062][104569] Avg episode reward: [(0, '8917.258'), (1, '8991.329')] [2023-12-27 00:50:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001298560_332472320.pth... [2023-12-27 00:50:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001296944_332070912.pth... [2023-12-27 00:50:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001295760_331767808.pth [2023-12-27 00:50:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001297440_332185600.pth [2023-12-27 00:50:01,527][105620] Updated weights for policy 1, policy_version 1298567 (0.0009) [2023-12-27 00:50:01,574][105620] Updated weights for policy 1, policy_version 1298577 (0.0006) [2023-12-27 00:50:01,586][105692] Updated weights for policy 0, policy_version 1296951 (0.0009) [2023-12-27 00:50:01,633][105620] Updated weights for policy 1, policy_version 1298587 (0.0008) [2023-12-27 00:50:01,644][105692] Updated weights for policy 0, policy_version 1296961 (0.0007) [2023-12-27 00:50:01,709][105692] Updated weights for policy 0, policy_version 1296971 (0.0008) [2023-12-27 00:50:02,335][105620] Updated weights for policy 1, policy_version 1298597 (0.0008) [2023-12-27 00:50:02,398][105620] Updated weights for policy 1, policy_version 1298607 (0.0009) [2023-12-27 00:50:02,454][105692] Updated weights for policy 0, policy_version 1296981 (0.0008) [2023-12-27 00:50:02,456][105620] Updated weights for policy 1, policy_version 1298617 (0.0007) [2023-12-27 00:50:02,516][105692] Updated weights for policy 0, policy_version 1296991 (0.0008) [2023-12-27 00:50:02,571][105692] Updated weights for policy 0, policy_version 1297001 (0.0006) [2023-12-27 00:50:03,213][105620] Updated weights for policy 1, policy_version 1298627 (0.0007) [2023-12-27 00:50:03,240][105692] Updated weights for policy 0, policy_version 1297011 (0.0006) [2023-12-27 00:50:03,262][105620] Updated weights for policy 1, policy_version 1298637 (0.0006) [2023-12-27 00:50:03,292][105692] Updated weights for policy 0, policy_version 1297021 (0.0007) [2023-12-27 00:50:03,309][105620] Updated weights for policy 1, policy_version 1298647 (0.0006) [2023-12-27 00:50:03,350][105692] Updated weights for policy 0, policy_version 1297031 (0.0008) [2023-12-27 00:50:03,997][105620] Updated weights for policy 1, policy_version 1298657 (0.0007) [2023-12-27 00:50:04,045][105620] Updated weights for policy 1, policy_version 1298667 (0.0008) [2023-12-27 00:50:04,097][105692] Updated weights for policy 0, policy_version 1297041 (0.0008) [2023-12-27 00:50:04,099][105620] Updated weights for policy 1, policy_version 1298677 (0.0009) [2023-12-27 00:50:04,154][105692] Updated weights for policy 0, policy_version 1297051 (0.0006) [2023-12-27 00:50:04,161][105620] Updated weights for policy 1, policy_version 1298687 (0.0008) [2023-12-27 00:50:04,208][105692] Updated weights for policy 0, policy_version 1297061 (0.0008) [2023-12-27 00:50:04,263][105692] Updated weights for policy 0, policy_version 1297071 (0.0009) [2023-12-27 00:50:04,902][105620] Updated weights for policy 1, policy_version 1298697 (0.0008) [2023-12-27 00:50:04,977][105620] Updated weights for policy 1, policy_version 1298707 (0.0011) [2023-12-27 00:50:05,006][105692] Updated weights for policy 0, policy_version 1297081 (0.0006) [2023-12-27 00:50:05,030][105620] Updated weights for policy 1, policy_version 1298717 (0.0011) [2023-12-27 00:50:05,061][105692] Updated weights for policy 0, policy_version 1297091 (0.0006) [2023-12-27 00:50:05,125][105692] Updated weights for policy 0, policy_version 1297101 (0.0011) [2023-12-27 00:50:05,736][105620] Updated weights for policy 1, policy_version 1298727 (0.0010) [2023-12-27 00:50:05,797][105620] Updated weights for policy 1, policy_version 1298737 (0.0005) [2023-12-27 00:50:05,798][105692] Updated weights for policy 0, policy_version 1297111 (0.0010) [2023-12-27 00:50:05,856][105692] Updated weights for policy 0, policy_version 1297121 (0.0009) [2023-12-27 00:50:05,858][105620] Updated weights for policy 1, policy_version 1298747 (0.0009) [2023-12-27 00:50:05,916][105692] Updated weights for policy 0, policy_version 1297131 (0.0007) [2023-12-27 00:50:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 664641536. Throughput: 0: 9887.2, 1: 9403.1. Samples: 664626276. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:50:06,063][104569] Avg episode reward: [(0, '9005.331'), (1, '9082.062')] [2023-12-27 00:50:06,545][105620] Updated weights for policy 1, policy_version 1298757 (0.0011) [2023-12-27 00:50:06,547][105692] Updated weights for policy 0, policy_version 1297141 (0.0007) [2023-12-27 00:50:06,604][105692] Updated weights for policy 0, policy_version 1297151 (0.0005) [2023-12-27 00:50:06,606][105620] Updated weights for policy 1, policy_version 1298767 (0.0011) [2023-12-27 00:50:06,664][105692] Updated weights for policy 0, policy_version 1297161 (0.0007) [2023-12-27 00:50:06,666][105620] Updated weights for policy 1, policy_version 1298777 (0.0011) [2023-12-27 00:50:07,388][105692] Updated weights for policy 0, policy_version 1297171 (0.0006) [2023-12-27 00:50:07,430][105620] Updated weights for policy 1, policy_version 1298787 (0.0011) [2023-12-27 00:50:07,458][105692] Updated weights for policy 0, policy_version 1297181 (0.0005) [2023-12-27 00:50:07,488][105620] Updated weights for policy 1, policy_version 1298797 (0.0009) [2023-12-27 00:50:07,523][105692] Updated weights for policy 0, policy_version 1297191 (0.0005) [2023-12-27 00:50:07,550][105620] Updated weights for policy 1, policy_version 1298807 (0.0007) [2023-12-27 00:50:08,169][105620] Updated weights for policy 1, policy_version 1298817 (0.0007) [2023-12-27 00:50:08,199][105692] Updated weights for policy 0, policy_version 1297201 (0.0007) [2023-12-27 00:50:08,234][105620] Updated weights for policy 1, policy_version 1298827 (0.0008) [2023-12-27 00:50:08,251][105692] Updated weights for policy 0, policy_version 1297211 (0.0005) [2023-12-27 00:50:08,297][105620] Updated weights for policy 1, policy_version 1298837 (0.0007) [2023-12-27 00:50:08,302][105692] Updated weights for policy 0, policy_version 1297221 (0.0007) [2023-12-27 00:50:08,362][105692] Updated weights for policy 0, policy_version 1297231 (0.0009) [2023-12-27 00:50:08,366][105620] Updated weights for policy 1, policy_version 1298847 (0.0008) [2023-12-27 00:50:09,042][105692] Updated weights for policy 0, policy_version 1297241 (0.0008) [2023-12-27 00:50:09,097][105692] Updated weights for policy 0, policy_version 1297251 (0.0007) [2023-12-27 00:50:09,105][105620] Updated weights for policy 1, policy_version 1298857 (0.0010) [2023-12-27 00:50:09,161][105692] Updated weights for policy 0, policy_version 1297261 (0.0008) [2023-12-27 00:50:09,162][105620] Updated weights for policy 1, policy_version 1298867 (0.0010) [2023-12-27 00:50:09,217][105620] Updated weights for policy 1, policy_version 1298877 (0.0006) [2023-12-27 00:50:09,949][105620] Updated weights for policy 1, policy_version 1298887 (0.0010) [2023-12-27 00:50:09,980][105692] Updated weights for policy 0, policy_version 1297271 (0.0009) [2023-12-27 00:50:10,006][105620] Updated weights for policy 1, policy_version 1298897 (0.0006) [2023-12-27 00:50:10,045][105692] Updated weights for policy 0, policy_version 1297281 (0.0009) [2023-12-27 00:50:10,054][105620] Updated weights for policy 1, policy_version 1298907 (0.0005) [2023-12-27 00:50:10,110][105692] Updated weights for policy 0, policy_version 1297291 (0.0009) [2023-12-27 00:50:10,687][105620] Updated weights for policy 1, policy_version 1298917 (0.0006) [2023-12-27 00:50:10,741][105620] Updated weights for policy 1, policy_version 1298927 (0.0008) [2023-12-27 00:50:10,800][105620] Updated weights for policy 1, policy_version 1298937 (0.0010) [2023-12-27 00:50:10,949][105692] Updated weights for policy 0, policy_version 1297301 (0.0008) [2023-12-27 00:50:11,005][105692] Updated weights for policy 0, policy_version 1297311 (0.0008) [2023-12-27 00:50:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 664731648. Throughput: 0: 9899.7, 1: 9528.5. Samples: 664743788. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:50:11,062][104569] Avg episode reward: [(0, '8914.543'), (1, '9264.150')] [2023-12-27 00:50:11,069][105692] Updated weights for policy 0, policy_version 1297321 (0.0009) [2023-12-27 00:50:11,568][105620] Updated weights for policy 1, policy_version 1298947 (0.0011) [2023-12-27 00:50:11,637][105620] Updated weights for policy 1, policy_version 1298957 (0.0009) [2023-12-27 00:50:11,699][105620] Updated weights for policy 1, policy_version 1298967 (0.0007) [2023-12-27 00:50:11,860][105692] Updated weights for policy 0, policy_version 1297331 (0.0009) [2023-12-27 00:50:11,923][105692] Updated weights for policy 0, policy_version 1297341 (0.0006) [2023-12-27 00:50:11,984][105692] Updated weights for policy 0, policy_version 1297351 (0.0007) [2023-12-27 00:50:12,392][105620] Updated weights for policy 1, policy_version 1298977 (0.0010) [2023-12-27 00:50:12,455][105620] Updated weights for policy 1, policy_version 1298987 (0.0011) [2023-12-27 00:50:12,517][105620] Updated weights for policy 1, policy_version 1298997 (0.0010) [2023-12-27 00:50:12,582][105620] Updated weights for policy 1, policy_version 1299007 (0.0010) [2023-12-27 00:50:12,729][105692] Updated weights for policy 0, policy_version 1297361 (0.0010) [2023-12-27 00:50:12,785][105692] Updated weights for policy 0, policy_version 1297371 (0.0008) [2023-12-27 00:50:12,848][105692] Updated weights for policy 0, policy_version 1297381 (0.0008) [2023-12-27 00:50:12,907][105692] Updated weights for policy 0, policy_version 1297391 (0.0008) [2023-12-27 00:50:13,320][105620] Updated weights for policy 1, policy_version 1299017 (0.0010) [2023-12-27 00:50:13,364][105620] Updated weights for policy 1, policy_version 1299027 (0.0010) [2023-12-27 00:50:13,412][105620] Updated weights for policy 1, policy_version 1299037 (0.0010) [2023-12-27 00:50:13,521][105692] Updated weights for policy 0, policy_version 1297401 (0.0005) [2023-12-27 00:50:13,578][105692] Updated weights for policy 0, policy_version 1297411 (0.0007) [2023-12-27 00:50:13,626][105692] Updated weights for policy 0, policy_version 1297421 (0.0008) [2023-12-27 00:50:14,167][105620] Updated weights for policy 1, policy_version 1299047 (0.0010) [2023-12-27 00:50:14,222][105620] Updated weights for policy 1, policy_version 1299057 (0.0010) [2023-12-27 00:50:14,267][105620] Updated weights for policy 1, policy_version 1299067 (0.0006) [2023-12-27 00:50:14,359][105692] Updated weights for policy 0, policy_version 1297431 (0.0006) [2023-12-27 00:50:14,430][105692] Updated weights for policy 0, policy_version 1297441 (0.0008) [2023-12-27 00:50:14,496][105692] Updated weights for policy 0, policy_version 1297451 (0.0010) [2023-12-27 00:50:14,904][105620] Updated weights for policy 1, policy_version 1299077 (0.0008) [2023-12-27 00:50:14,970][105620] Updated weights for policy 1, policy_version 1299087 (0.0009) [2023-12-27 00:50:15,038][105620] Updated weights for policy 1, policy_version 1299097 (0.0008) [2023-12-27 00:50:15,240][105692] Updated weights for policy 0, policy_version 1297461 (0.0009) [2023-12-27 00:50:15,309][105692] Updated weights for policy 0, policy_version 1297471 (0.0009) [2023-12-27 00:50:15,365][105692] Updated weights for policy 0, policy_version 1297481 (0.0008) [2023-12-27 00:50:15,767][105620] Updated weights for policy 1, policy_version 1299107 (0.0011) [2023-12-27 00:50:15,828][105620] Updated weights for policy 1, policy_version 1299117 (0.0010) [2023-12-27 00:50:15,882][105620] Updated weights for policy 1, policy_version 1299127 (0.0010) [2023-12-27 00:50:16,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 664829952. Throughput: 0: 9914.8, 1: 9521.8. Samples: 664800844. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 00:50:16,062][104569] Avg episode reward: [(0, '9087.540'), (1, '9355.699')] [2023-12-27 00:50:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001299136_332619776.pth... [2023-12-27 00:50:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001297488_332210176.pth... [2023-12-27 00:50:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001298016_332333056.pth [2023-12-27 00:50:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001296368_331923456.pth [2023-12-27 00:50:16,132][105692] Updated weights for policy 0, policy_version 1297491 (0.0009) [2023-12-27 00:50:16,200][105692] Updated weights for policy 0, policy_version 1297501 (0.0008) [2023-12-27 00:50:16,267][105692] Updated weights for policy 0, policy_version 1297511 (0.0008) [2023-12-27 00:50:16,619][105620] Updated weights for policy 1, policy_version 1299137 (0.0010) [2023-12-27 00:50:16,679][105620] Updated weights for policy 1, policy_version 1299147 (0.0010) [2023-12-27 00:50:16,743][105620] Updated weights for policy 1, policy_version 1299157 (0.0005) [2023-12-27 00:50:16,812][105620] Updated weights for policy 1, policy_version 1299167 (0.0005) [2023-12-27 00:50:17,036][105692] Updated weights for policy 0, policy_version 1297521 (0.0008) [2023-12-27 00:50:17,088][105692] Updated weights for policy 0, policy_version 1297531 (0.0008) [2023-12-27 00:50:17,142][105692] Updated weights for policy 0, policy_version 1297541 (0.0008) [2023-12-27 00:50:17,207][105692] Updated weights for policy 0, policy_version 1297551 (0.0009) [2023-12-27 00:50:17,428][105620] Updated weights for policy 1, policy_version 1299177 (0.0007) [2023-12-27 00:50:17,500][105620] Updated weights for policy 1, policy_version 1299188 (0.0008) [2023-12-27 00:50:17,560][105620] Updated weights for policy 1, policy_version 1299198 (0.0008) [2023-12-27 00:50:18,034][105692] Updated weights for policy 0, policy_version 1297561 (0.0009) [2023-12-27 00:50:18,089][105692] Updated weights for policy 0, policy_version 1297571 (0.0009) [2023-12-27 00:50:18,147][105692] Updated weights for policy 0, policy_version 1297581 (0.0009) [2023-12-27 00:50:18,248][105620] Updated weights for policy 1, policy_version 1299208 (0.0008) [2023-12-27 00:50:18,305][105620] Updated weights for policy 1, policy_version 1299218 (0.0008) [2023-12-27 00:50:18,371][105620] Updated weights for policy 1, policy_version 1299228 (0.0008) [2023-12-27 00:50:18,950][105620] Updated weights for policy 1, policy_version 1299238 (0.0007) [2023-12-27 00:50:18,953][105692] Updated weights for policy 0, policy_version 1297591 (0.0008) [2023-12-27 00:50:19,002][105620] Updated weights for policy 1, policy_version 1299248 (0.0006) [2023-12-27 00:50:19,008][105692] Updated weights for policy 0, policy_version 1297601 (0.0008) [2023-12-27 00:50:19,056][105620] Updated weights for policy 1, policy_version 1299258 (0.0007) [2023-12-27 00:50:19,074][105692] Updated weights for policy 0, policy_version 1297611 (0.0006) [2023-12-27 00:50:19,742][105620] Updated weights for policy 1, policy_version 1299268 (0.0009) [2023-12-27 00:50:19,798][105620] Updated weights for policy 1, policy_version 1299278 (0.0011) [2023-12-27 00:50:19,856][105692] Updated weights for policy 0, policy_version 1297621 (0.0009) [2023-12-27 00:50:19,866][105620] Updated weights for policy 1, policy_version 1299288 (0.0006) [2023-12-27 00:50:19,919][105692] Updated weights for policy 0, policy_version 1297631 (0.0008) [2023-12-27 00:50:19,982][105692] Updated weights for policy 0, policy_version 1297641 (0.0010) [2023-12-27 00:50:20,564][105620] Updated weights for policy 1, policy_version 1299298 (0.0007) [2023-12-27 00:50:20,645][105620] Updated weights for policy 1, policy_version 1299308 (0.0009) [2023-12-27 00:50:20,710][105620] Updated weights for policy 1, policy_version 1299318 (0.0011) [2023-12-27 00:50:20,775][105620] Updated weights for policy 1, policy_version 1299328 (0.0011) [2023-12-27 00:50:20,785][105692] Updated weights for policy 0, policy_version 1297651 (0.0007) [2023-12-27 00:50:20,849][105692] Updated weights for policy 0, policy_version 1297661 (0.0009) [2023-12-27 00:50:20,907][105692] Updated weights for policy 0, policy_version 1297671 (0.0009) [2023-12-27 00:50:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 664928256. Throughput: 0: 9812.7, 1: 9583.5. Samples: 664915564. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:50:21,063][104569] Avg episode reward: [(0, '9177.233'), (1, '9355.229')] [2023-12-27 00:50:21,527][105620] Updated weights for policy 1, policy_version 1299338 (0.0011) [2023-12-27 00:50:21,592][105620] Updated weights for policy 1, policy_version 1299348 (0.0011) [2023-12-27 00:50:21,657][105620] Updated weights for policy 1, policy_version 1299358 (0.0009) [2023-12-27 00:50:21,668][105692] Updated weights for policy 0, policy_version 1297681 (0.0006) [2023-12-27 00:50:21,725][105692] Updated weights for policy 0, policy_version 1297691 (0.0009) [2023-12-27 00:50:21,788][105692] Updated weights for policy 0, policy_version 1297701 (0.0008) [2023-12-27 00:50:21,836][105692] Updated weights for policy 0, policy_version 1297711 (0.0008) [2023-12-27 00:50:22,434][105620] Updated weights for policy 1, policy_version 1299368 (0.0011) [2023-12-27 00:50:22,498][105620] Updated weights for policy 1, policy_version 1299378 (0.0011) [2023-12-27 00:50:22,557][105620] Updated weights for policy 1, policy_version 1299388 (0.0011) [2023-12-27 00:50:22,682][105692] Updated weights for policy 0, policy_version 1297721 (0.0008) [2023-12-27 00:50:22,747][105692] Updated weights for policy 0, policy_version 1297731 (0.0008) [2023-12-27 00:50:22,797][105692] Updated weights for policy 0, policy_version 1297741 (0.0009) [2023-12-27 00:50:23,282][105620] Updated weights for policy 1, policy_version 1299398 (0.0011) [2023-12-27 00:50:23,333][105620] Updated weights for policy 1, policy_version 1299408 (0.0006) [2023-12-27 00:50:23,382][105620] Updated weights for policy 1, policy_version 1299418 (0.0005) [2023-12-27 00:50:23,517][105692] Updated weights for policy 0, policy_version 1297751 (0.0006) [2023-12-27 00:50:23,563][105692] Updated weights for policy 0, policy_version 1297761 (0.0005) [2023-12-27 00:50:23,609][105692] Updated weights for policy 0, policy_version 1297771 (0.0005) [2023-12-27 00:50:24,041][105620] Updated weights for policy 1, policy_version 1299428 (0.0007) [2023-12-27 00:50:24,105][105620] Updated weights for policy 1, policy_version 1299438 (0.0007) [2023-12-27 00:50:24,169][105620] Updated weights for policy 1, policy_version 1299448 (0.0008) [2023-12-27 00:50:24,248][105692] Updated weights for policy 0, policy_version 1297781 (0.0009) [2023-12-27 00:50:24,304][105692] Updated weights for policy 0, policy_version 1297791 (0.0010) [2023-12-27 00:50:24,369][105692] Updated weights for policy 0, policy_version 1297801 (0.0011) [2023-12-27 00:50:24,972][105692] Updated weights for policy 0, policy_version 1297811 (0.0010) [2023-12-27 00:50:24,997][105620] Updated weights for policy 1, policy_version 1299458 (0.0008) [2023-12-27 00:50:25,027][105692] Updated weights for policy 0, policy_version 1297821 (0.0008) [2023-12-27 00:50:25,050][105620] Updated weights for policy 1, policy_version 1299468 (0.0007) [2023-12-27 00:50:25,081][105692] Updated weights for policy 0, policy_version 1297831 (0.0007) [2023-12-27 00:50:25,111][105620] Updated weights for policy 1, policy_version 1299478 (0.0007) [2023-12-27 00:50:25,167][105620] Updated weights for policy 1, policy_version 1299488 (0.0007) [2023-12-27 00:50:25,689][105692] Updated weights for policy 0, policy_version 1297841 (0.0006) [2023-12-27 00:50:25,762][105692] Updated weights for policy 0, policy_version 1297851 (0.0005) [2023-12-27 00:50:25,830][105692] Updated weights for policy 0, policy_version 1297861 (0.0005) [2023-12-27 00:50:25,888][105692] Updated weights for policy 0, policy_version 1297871 (0.0009) [2023-12-27 00:50:25,929][105620] Updated weights for policy 1, policy_version 1299498 (0.0010) [2023-12-27 00:50:25,981][105620] Updated weights for policy 1, policy_version 1299508 (0.0011) [2023-12-27 00:50:26,033][105620] Updated weights for policy 1, policy_version 1299518 (0.0010) [2023-12-27 00:50:26,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 665026560. Throughput: 0: 9790.4, 1: 9641.4. Samples: 665030068. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:50:26,063][104569] Avg episode reward: [(0, '9268.070'), (1, '9354.814')] [2023-12-27 00:50:26,461][105692] Updated weights for policy 0, policy_version 1297881 (0.0005) [2023-12-27 00:50:26,505][105692] Updated weights for policy 0, policy_version 1297891 (0.0005) [2023-12-27 00:50:26,550][105692] Updated weights for policy 0, policy_version 1297901 (0.0005) [2023-12-27 00:50:26,600][105620] Updated weights for policy 1, policy_version 1299528 (0.0008) [2023-12-27 00:50:26,644][105620] Updated weights for policy 1, policy_version 1299538 (0.0010) [2023-12-27 00:50:26,700][105620] Updated weights for policy 1, policy_version 1299548 (0.0010) [2023-12-27 00:50:27,102][105692] Updated weights for policy 0, policy_version 1297911 (0.0005) [2023-12-27 00:50:27,159][105692] Updated weights for policy 0, policy_version 1297921 (0.0007) [2023-12-27 00:50:27,212][105692] Updated weights for policy 0, policy_version 1297932 (0.0011) [2023-12-27 00:50:27,297][105620] Updated weights for policy 1, policy_version 1299558 (0.0007) [2023-12-27 00:50:27,344][105620] Updated weights for policy 1, policy_version 1299568 (0.0010) [2023-12-27 00:50:27,400][105620] Updated weights for policy 1, policy_version 1299578 (0.0010) [2023-12-27 00:50:27,797][105692] Updated weights for policy 0, policy_version 1297942 (0.0007) [2023-12-27 00:50:27,863][105692] Updated weights for policy 0, policy_version 1297952 (0.0005) [2023-12-27 00:50:27,918][105692] Updated weights for policy 0, policy_version 1297962 (0.0005) [2023-12-27 00:50:28,133][105620] Updated weights for policy 1, policy_version 1299588 (0.0010) [2023-12-27 00:50:28,191][105620] Updated weights for policy 1, policy_version 1299598 (0.0010) [2023-12-27 00:50:28,248][105620] Updated weights for policy 1, policy_version 1299608 (0.0010) [2023-12-27 00:50:28,450][105692] Updated weights for policy 0, policy_version 1297972 (0.0005) [2023-12-27 00:50:28,505][105692] Updated weights for policy 0, policy_version 1297982 (0.0005) [2023-12-27 00:50:28,573][105692] Updated weights for policy 0, policy_version 1297992 (0.0005) [2023-12-27 00:50:28,853][105620] Updated weights for policy 1, policy_version 1299618 (0.0010) [2023-12-27 00:50:28,905][105620] Updated weights for policy 1, policy_version 1299628 (0.0010) [2023-12-27 00:50:28,950][105620] Updated weights for policy 1, policy_version 1299638 (0.0010) [2023-12-27 00:50:28,995][105620] Updated weights for policy 1, policy_version 1299648 (0.0010) [2023-12-27 00:50:29,256][105692] Updated weights for policy 0, policy_version 1298002 (0.0008) [2023-12-27 00:50:29,308][105692] Updated weights for policy 0, policy_version 1298012 (0.0009) [2023-12-27 00:50:29,367][105692] Updated weights for policy 0, policy_version 1298022 (0.0009) [2023-12-27 00:50:29,416][105692] Updated weights for policy 0, policy_version 1298032 (0.0005) [2023-12-27 00:50:29,757][105620] Updated weights for policy 1, policy_version 1299658 (0.0011) [2023-12-27 00:50:29,815][105620] Updated weights for policy 1, policy_version 1299668 (0.0010) [2023-12-27 00:50:29,879][105620] Updated weights for policy 1, policy_version 1299678 (0.0008) [2023-12-27 00:50:30,027][105692] Updated weights for policy 0, policy_version 1298042 (0.0007) [2023-12-27 00:50:30,084][105692] Updated weights for policy 0, policy_version 1298052 (0.0009) [2023-12-27 00:50:30,137][105692] Updated weights for policy 0, policy_version 1298062 (0.0010) [2023-12-27 00:50:30,598][105620] Updated weights for policy 1, policy_version 1299688 (0.0009) [2023-12-27 00:50:30,655][105620] Updated weights for policy 1, policy_version 1299698 (0.0008) [2023-12-27 00:50:30,710][105620] Updated weights for policy 1, policy_version 1299708 (0.0009) [2023-12-27 00:50:30,761][105692] Updated weights for policy 0, policy_version 1298072 (0.0006) [2023-12-27 00:50:30,814][105692] Updated weights for policy 0, policy_version 1298082 (0.0009) [2023-12-27 00:50:30,860][105692] Updated weights for policy 0, policy_version 1298092 (0.0009) [2023-12-27 00:50:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 665133056. Throughput: 0: 9944.5, 1: 9715.8. Samples: 665099032. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:50:31,062][104569] Avg episode reward: [(0, '8994.348'), (1, '9262.432')] [2023-12-27 00:50:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001298096_332365824.pth... [2023-12-27 00:50:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001299712_332767232.pth... [2023-12-27 00:50:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001298560_332472320.pth [2023-12-27 00:50:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001296944_332070912.pth [2023-12-27 00:50:31,445][105620] Updated weights for policy 1, policy_version 1299718 (0.0007) [2023-12-27 00:50:31,489][105692] Updated weights for policy 0, policy_version 1298103 (0.0010) [2023-12-27 00:50:31,504][105620] Updated weights for policy 1, policy_version 1299728 (0.0006) [2023-12-27 00:50:31,545][105692] Updated weights for policy 0, policy_version 1298113 (0.0010) [2023-12-27 00:50:31,562][105620] Updated weights for policy 1, policy_version 1299738 (0.0005) [2023-12-27 00:50:31,601][105692] Updated weights for policy 0, policy_version 1298123 (0.0008) [2023-12-27 00:50:32,233][105620] Updated weights for policy 1, policy_version 1299748 (0.0007) [2023-12-27 00:50:32,295][105620] Updated weights for policy 1, policy_version 1299758 (0.0009) [2023-12-27 00:50:32,359][105620] Updated weights for policy 1, policy_version 1299768 (0.0009) [2023-12-27 00:50:32,398][105692] Updated weights for policy 0, policy_version 1298133 (0.0008) [2023-12-27 00:50:32,450][105692] Updated weights for policy 0, policy_version 1298143 (0.0008) [2023-12-27 00:50:32,499][105692] Updated weights for policy 0, policy_version 1298153 (0.0007) [2023-12-27 00:50:33,127][105692] Updated weights for policy 0, policy_version 1298163 (0.0007) [2023-12-27 00:50:33,187][105692] Updated weights for policy 0, policy_version 1298173 (0.0005) [2023-12-27 00:50:33,221][105620] Updated weights for policy 1, policy_version 1299778 (0.0008) [2023-12-27 00:50:33,250][105692] Updated weights for policy 0, policy_version 1298183 (0.0005) [2023-12-27 00:50:33,278][105620] Updated weights for policy 1, policy_version 1299788 (0.0005) [2023-12-27 00:50:33,343][105620] Updated weights for policy 1, policy_version 1299798 (0.0006) [2023-12-27 00:50:33,406][105620] Updated weights for policy 1, policy_version 1299808 (0.0005) [2023-12-27 00:50:33,878][105692] Updated weights for policy 0, policy_version 1298193 (0.0006) [2023-12-27 00:50:33,938][105692] Updated weights for policy 0, policy_version 1298203 (0.0009) [2023-12-27 00:50:33,982][105692] Updated weights for policy 0, policy_version 1298213 (0.0008) [2023-12-27 00:50:34,033][105620] Updated weights for policy 1, policy_version 1299818 (0.0005) [2023-12-27 00:50:34,035][105692] Updated weights for policy 0, policy_version 1298223 (0.0008) [2023-12-27 00:50:34,082][105620] Updated weights for policy 1, policy_version 1299828 (0.0007) [2023-12-27 00:50:34,140][105620] Updated weights for policy 1, policy_version 1299838 (0.0006) [2023-12-27 00:50:34,771][105692] Updated weights for policy 0, policy_version 1298233 (0.0009) [2023-12-27 00:50:34,831][105692] Updated weights for policy 0, policy_version 1298243 (0.0009) [2023-12-27 00:50:34,850][105620] Updated weights for policy 1, policy_version 1299848 (0.0006) [2023-12-27 00:50:34,891][105692] Updated weights for policy 0, policy_version 1298253 (0.0008) [2023-12-27 00:50:34,909][105620] Updated weights for policy 1, policy_version 1299858 (0.0006) [2023-12-27 00:50:34,971][105620] Updated weights for policy 1, policy_version 1299868 (0.0009) [2023-12-27 00:50:35,632][105692] Updated weights for policy 0, policy_version 1298263 (0.0008) [2023-12-27 00:50:35,692][105692] Updated weights for policy 0, policy_version 1298273 (0.0009) [2023-12-27 00:50:35,707][105620] Updated weights for policy 1, policy_version 1299878 (0.0007) [2023-12-27 00:50:35,750][105692] Updated weights for policy 0, policy_version 1298283 (0.0006) [2023-12-27 00:50:35,760][105620] Updated weights for policy 1, policy_version 1299888 (0.0006) [2023-12-27 00:50:35,812][105620] Updated weights for policy 1, policy_version 1299898 (0.0008) [2023-12-27 00:50:36,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 665231360. Throughput: 0: 9948.1, 1: 9688.0. Samples: 665219096. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:50:36,063][104569] Avg episode reward: [(0, '8902.744'), (1, '9262.333')] [2023-12-27 00:50:36,391][105692] Updated weights for policy 0, policy_version 1298293 (0.0007) [2023-12-27 00:50:36,457][105692] Updated weights for policy 0, policy_version 1298303 (0.0006) [2023-12-27 00:50:36,517][105692] Updated weights for policy 0, policy_version 1298313 (0.0008) [2023-12-27 00:50:36,672][105620] Updated weights for policy 1, policy_version 1299909 (0.0009) [2023-12-27 00:50:36,731][105620] Updated weights for policy 1, policy_version 1299919 (0.0009) [2023-12-27 00:50:36,784][105620] Updated weights for policy 1, policy_version 1299929 (0.0010) [2023-12-27 00:50:37,201][105692] Updated weights for policy 0, policy_version 1298323 (0.0009) [2023-12-27 00:50:37,247][105692] Updated weights for policy 0, policy_version 1298333 (0.0008) [2023-12-27 00:50:37,299][105692] Updated weights for policy 0, policy_version 1298343 (0.0008) [2023-12-27 00:50:37,558][105620] Updated weights for policy 1, policy_version 1299939 (0.0009) [2023-12-27 00:50:37,614][105620] Updated weights for policy 1, policy_version 1299949 (0.0011) [2023-12-27 00:50:37,667][105620] Updated weights for policy 1, policy_version 1299959 (0.0010) [2023-12-27 00:50:38,019][105692] Updated weights for policy 0, policy_version 1298353 (0.0008) [2023-12-27 00:50:38,075][105692] Updated weights for policy 0, policy_version 1298363 (0.0009) [2023-12-27 00:50:38,140][105692] Updated weights for policy 0, policy_version 1298373 (0.0005) [2023-12-27 00:50:38,192][105692] Updated weights for policy 0, policy_version 1298383 (0.0008) [2023-12-27 00:50:38,367][105620] Updated weights for policy 1, policy_version 1299969 (0.0010) [2023-12-27 00:50:38,419][105620] Updated weights for policy 1, policy_version 1299979 (0.0006) [2023-12-27 00:50:38,469][105620] Updated weights for policy 1, policy_version 1299989 (0.0006) [2023-12-27 00:50:38,524][105620] Updated weights for policy 1, policy_version 1299999 (0.0006) [2023-12-27 00:50:38,986][105692] Updated weights for policy 0, policy_version 1298393 (0.0008) [2023-12-27 00:50:39,041][105692] Updated weights for policy 0, policy_version 1298403 (0.0009) [2023-12-27 00:50:39,096][105692] Updated weights for policy 0, policy_version 1298413 (0.0009) [2023-12-27 00:50:39,132][105620] Updated weights for policy 1, policy_version 1300009 (0.0007) [2023-12-27 00:50:39,182][105620] Updated weights for policy 1, policy_version 1300019 (0.0009) [2023-12-27 00:50:39,238][105620] Updated weights for policy 1, policy_version 1300029 (0.0009) [2023-12-27 00:50:39,823][105692] Updated weights for policy 0, policy_version 1298423 (0.0009) [2023-12-27 00:50:39,890][105692] Updated weights for policy 0, policy_version 1298433 (0.0009) [2023-12-27 00:50:39,956][105692] Updated weights for policy 0, policy_version 1298443 (0.0009) [2023-12-27 00:50:40,070][105620] Updated weights for policy 1, policy_version 1300039 (0.0008) [2023-12-27 00:50:40,129][105620] Updated weights for policy 1, policy_version 1300049 (0.0008) [2023-12-27 00:50:40,196][105620] Updated weights for policy 1, policy_version 1300059 (0.0008) [2023-12-27 00:50:40,788][105692] Updated weights for policy 0, policy_version 1298453 (0.0008) [2023-12-27 00:50:40,810][105620] Updated weights for policy 1, policy_version 1300069 (0.0007) [2023-12-27 00:50:40,841][105692] Updated weights for policy 0, policy_version 1298463 (0.0006) [2023-12-27 00:50:40,870][105620] Updated weights for policy 1, policy_version 1300079 (0.0009) [2023-12-27 00:50:40,901][105692] Updated weights for policy 0, policy_version 1298473 (0.0007) [2023-12-27 00:50:40,931][105620] Updated weights for policy 1, policy_version 1300089 (0.0006) [2023-12-27 00:50:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 665329664. Throughput: 0: 9881.3, 1: 9772.8. Samples: 665333732. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:50:41,062][104569] Avg episode reward: [(0, '8905.320'), (1, '9262.869')] [2023-12-27 00:50:41,654][105692] Updated weights for policy 0, policy_version 1298483 (0.0008) [2023-12-27 00:50:41,716][105692] Updated weights for policy 0, policy_version 1298493 (0.0009) [2023-12-27 00:50:41,729][105620] Updated weights for policy 1, policy_version 1300099 (0.0008) [2023-12-27 00:50:41,785][105692] Updated weights for policy 0, policy_version 1298503 (0.0008) [2023-12-27 00:50:41,801][105620] Updated weights for policy 1, policy_version 1300109 (0.0008) [2023-12-27 00:50:41,871][105620] Updated weights for policy 1, policy_version 1300119 (0.0006) [2023-12-27 00:50:42,498][105620] Updated weights for policy 1, policy_version 1300129 (0.0005) [2023-12-27 00:50:42,571][105620] Updated weights for policy 1, policy_version 1300139 (0.0005) [2023-12-27 00:50:42,583][105692] Updated weights for policy 0, policy_version 1298513 (0.0008) [2023-12-27 00:50:42,633][105620] Updated weights for policy 1, policy_version 1300149 (0.0007) [2023-12-27 00:50:42,642][105692] Updated weights for policy 0, policy_version 1298523 (0.0011) [2023-12-27 00:50:42,693][105620] Updated weights for policy 1, policy_version 1300159 (0.0006) [2023-12-27 00:50:42,707][105692] Updated weights for policy 0, policy_version 1298533 (0.0010) [2023-12-27 00:50:42,764][105692] Updated weights for policy 0, policy_version 1298543 (0.0011) [2023-12-27 00:50:43,346][105620] Updated weights for policy 1, policy_version 1300170 (0.0009) [2023-12-27 00:50:43,405][105620] Updated weights for policy 1, policy_version 1300180 (0.0009) [2023-12-27 00:50:43,418][105692] Updated weights for policy 0, policy_version 1298553 (0.0006) [2023-12-27 00:50:43,467][105620] Updated weights for policy 1, policy_version 1300190 (0.0007) [2023-12-27 00:50:43,473][105692] Updated weights for policy 0, policy_version 1298563 (0.0008) [2023-12-27 00:50:43,538][105692] Updated weights for policy 0, policy_version 1298573 (0.0010) [2023-12-27 00:50:44,158][105620] Updated weights for policy 1, policy_version 1300200 (0.0008) [2023-12-27 00:50:44,238][105620] Updated weights for policy 1, policy_version 1300210 (0.0007) [2023-12-27 00:50:44,240][105692] Updated weights for policy 0, policy_version 1298583 (0.0008) [2023-12-27 00:50:44,291][105692] Updated weights for policy 0, policy_version 1298593 (0.0008) [2023-12-27 00:50:44,300][105620] Updated weights for policy 1, policy_version 1300220 (0.0006) [2023-12-27 00:50:44,341][105692] Updated weights for policy 0, policy_version 1298603 (0.0009) [2023-12-27 00:50:44,828][105620] Updated weights for policy 1, policy_version 1300230 (0.0006) [2023-12-27 00:50:44,895][105620] Updated weights for policy 1, policy_version 1300240 (0.0008) [2023-12-27 00:50:44,964][105620] Updated weights for policy 1, policy_version 1300250 (0.0008) [2023-12-27 00:50:45,083][105692] Updated weights for policy 0, policy_version 1298614 (0.0010) [2023-12-27 00:50:45,142][105692] Updated weights for policy 0, policy_version 1298624 (0.0009) [2023-12-27 00:50:45,205][105692] Updated weights for policy 0, policy_version 1298634 (0.0009) [2023-12-27 00:50:45,691][105620] Updated weights for policy 1, policy_version 1300260 (0.0009) [2023-12-27 00:50:45,741][105620] Updated weights for policy 1, policy_version 1300270 (0.0009) [2023-12-27 00:50:45,795][105620] Updated weights for policy 1, policy_version 1300280 (0.0008) [2023-12-27 00:50:45,936][105692] Updated weights for policy 0, policy_version 1298644 (0.0007) [2023-12-27 00:50:45,992][105692] Updated weights for policy 0, policy_version 1298654 (0.0010) [2023-12-27 00:50:46,040][105692] Updated weights for policy 0, policy_version 1298664 (0.0010) [2023-12-27 00:50:46,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19660.7, 300 sec: 19466.4). Total num frames: 665419776. Throughput: 0: 9765.0, 1: 9765.4. Samples: 665391344. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:50:46,063][104569] Avg episode reward: [(0, '8814.498'), (1, '9171.380')] [2023-12-27 00:50:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001300288_332914688.pth... [2023-12-27 00:50:46,074][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001298672_332513280.pth... [2023-12-27 00:50:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001297488_332210176.pth [2023-12-27 00:50:46,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001299136_332619776.pth [2023-12-27 00:50:46,487][105620] Updated weights for policy 1, policy_version 1300291 (0.0009) [2023-12-27 00:50:46,547][105620] Updated weights for policy 1, policy_version 1300301 (0.0005) [2023-12-27 00:50:46,600][105620] Updated weights for policy 1, policy_version 1300311 (0.0005) [2023-12-27 00:50:46,652][105692] Updated weights for policy 0, policy_version 1298674 (0.0010) [2023-12-27 00:50:46,699][105692] Updated weights for policy 0, policy_version 1298684 (0.0008) [2023-12-27 00:50:46,752][105692] Updated weights for policy 0, policy_version 1298694 (0.0005) [2023-12-27 00:50:46,802][105692] Updated weights for policy 0, policy_version 1298704 (0.0005) [2023-12-27 00:50:47,268][105620] Updated weights for policy 1, policy_version 1300321 (0.0006) [2023-12-27 00:50:47,324][105620] Updated weights for policy 1, policy_version 1300331 (0.0010) [2023-12-27 00:50:47,373][105692] Updated weights for policy 0, policy_version 1298714 (0.0010) [2023-12-27 00:50:47,374][105620] Updated weights for policy 1, policy_version 1300341 (0.0009) [2023-12-27 00:50:47,431][105620] Updated weights for policy 1, policy_version 1300351 (0.0009) [2023-12-27 00:50:47,437][105692] Updated weights for policy 0, policy_version 1298724 (0.0006) [2023-12-27 00:50:47,499][105692] Updated weights for policy 0, policy_version 1298734 (0.0005) [2023-12-27 00:50:48,177][105692] Updated weights for policy 0, policy_version 1298744 (0.0008) [2023-12-27 00:50:48,207][105620] Updated weights for policy 1, policy_version 1300361 (0.0008) [2023-12-27 00:50:48,239][105692] Updated weights for policy 0, policy_version 1298754 (0.0008) [2023-12-27 00:50:48,271][105620] Updated weights for policy 1, policy_version 1300371 (0.0009) [2023-12-27 00:50:48,298][105692] Updated weights for policy 0, policy_version 1298764 (0.0008) [2023-12-27 00:50:48,334][105620] Updated weights for policy 1, policy_version 1300381 (0.0008) [2023-12-27 00:50:48,946][105692] Updated weights for policy 0, policy_version 1298774 (0.0007) [2023-12-27 00:50:49,002][105692] Updated weights for policy 0, policy_version 1298784 (0.0006) [2023-12-27 00:50:49,061][105692] Updated weights for policy 0, policy_version 1298794 (0.0006) [2023-12-27 00:50:49,106][105620] Updated weights for policy 1, policy_version 1300391 (0.0007) [2023-12-27 00:50:49,159][105620] Updated weights for policy 1, policy_version 1300401 (0.0010) [2023-12-27 00:50:49,214][105620] Updated weights for policy 1, policy_version 1300412 (0.0010) [2023-12-27 00:50:49,665][105692] Updated weights for policy 0, policy_version 1298804 (0.0006) [2023-12-27 00:50:49,736][105692] Updated weights for policy 0, policy_version 1298814 (0.0006) [2023-12-27 00:50:49,805][105692] Updated weights for policy 0, policy_version 1298824 (0.0006) [2023-12-27 00:50:49,937][105620] Updated weights for policy 1, policy_version 1300422 (0.0010) [2023-12-27 00:50:49,996][105620] Updated weights for policy 1, policy_version 1300432 (0.0010) [2023-12-27 00:50:50,063][105620] Updated weights for policy 1, policy_version 1300442 (0.0011) [2023-12-27 00:50:50,383][105692] Updated weights for policy 0, policy_version 1298834 (0.0009) [2023-12-27 00:50:50,445][105692] Updated weights for policy 0, policy_version 1298844 (0.0010) [2023-12-27 00:50:50,511][105692] Updated weights for policy 0, policy_version 1298854 (0.0008) [2023-12-27 00:50:50,580][105692] Updated weights for policy 0, policy_version 1298864 (0.0008) [2023-12-27 00:50:50,815][105620] Updated weights for policy 1, policy_version 1300452 (0.0011) [2023-12-27 00:50:50,875][105620] Updated weights for policy 1, policy_version 1300462 (0.0011) [2023-12-27 00:50:50,941][105620] Updated weights for policy 1, policy_version 1300472 (0.0011) [2023-12-27 00:50:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 665526272. Throughput: 0: 9896.5, 1: 9813.4. Samples: 665513216. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:50:51,062][104569] Avg episode reward: [(0, '8727.028'), (1, '9171.619')] [2023-12-27 00:50:51,347][105692] Updated weights for policy 0, policy_version 1298874 (0.0008) [2023-12-27 00:50:51,415][105692] Updated weights for policy 0, policy_version 1298884 (0.0008) [2023-12-27 00:50:51,474][105692] Updated weights for policy 0, policy_version 1298894 (0.0008) [2023-12-27 00:50:51,733][105620] Updated weights for policy 1, policy_version 1300482 (0.0010) [2023-12-27 00:50:51,794][105620] Updated weights for policy 1, policy_version 1300492 (0.0008) [2023-12-27 00:50:51,850][105620] Updated weights for policy 1, policy_version 1300502 (0.0008) [2023-12-27 00:50:51,905][105620] Updated weights for policy 1, policy_version 1300512 (0.0008) [2023-12-27 00:50:52,248][105692] Updated weights for policy 0, policy_version 1298904 (0.0009) [2023-12-27 00:50:52,312][105692] Updated weights for policy 0, policy_version 1298914 (0.0009) [2023-12-27 00:50:52,372][105692] Updated weights for policy 0, policy_version 1298924 (0.0008) [2023-12-27 00:50:52,670][105620] Updated weights for policy 1, policy_version 1300522 (0.0010) [2023-12-27 00:50:52,720][105620] Updated weights for policy 1, policy_version 1300532 (0.0009) [2023-12-27 00:50:52,782][105620] Updated weights for policy 1, policy_version 1300542 (0.0006) [2023-12-27 00:50:53,160][105692] Updated weights for policy 0, policy_version 1298934 (0.0008) [2023-12-27 00:50:53,217][105692] Updated weights for policy 0, policy_version 1298944 (0.0010) [2023-12-27 00:50:53,278][105692] Updated weights for policy 0, policy_version 1298954 (0.0010) [2023-12-27 00:50:53,524][105620] Updated weights for policy 1, policy_version 1300552 (0.0005) [2023-12-27 00:50:53,589][105620] Updated weights for policy 1, policy_version 1300562 (0.0005) [2023-12-27 00:50:53,648][105620] Updated weights for policy 1, policy_version 1300572 (0.0005) [2023-12-27 00:50:54,150][105692] Updated weights for policy 0, policy_version 1298964 (0.0010) [2023-12-27 00:50:54,219][105692] Updated weights for policy 0, policy_version 1298974 (0.0009) [2023-12-27 00:50:54,240][105620] Updated weights for policy 1, policy_version 1300582 (0.0006) [2023-12-27 00:50:54,277][105692] Updated weights for policy 0, policy_version 1298984 (0.0008) [2023-12-27 00:50:54,293][105620] Updated weights for policy 1, policy_version 1300592 (0.0006) [2023-12-27 00:50:54,354][105620] Updated weights for policy 1, policy_version 1300602 (0.0007) [2023-12-27 00:50:55,001][105620] Updated weights for policy 1, policy_version 1300612 (0.0008) [2023-12-27 00:50:55,055][105620] Updated weights for policy 1, policy_version 1300622 (0.0007) [2023-12-27 00:50:55,098][105692] Updated weights for policy 0, policy_version 1298994 (0.0009) [2023-12-27 00:50:55,112][105620] Updated weights for policy 1, policy_version 1300632 (0.0009) [2023-12-27 00:50:55,162][105692] Updated weights for policy 0, policy_version 1299004 (0.0008) [2023-12-27 00:50:55,221][105692] Updated weights for policy 0, policy_version 1299014 (0.0009) [2023-12-27 00:50:55,283][105692] Updated weights for policy 0, policy_version 1299024 (0.0009) [2023-12-27 00:50:55,743][105620] Updated weights for policy 1, policy_version 1300642 (0.0006) [2023-12-27 00:50:55,803][105620] Updated weights for policy 1, policy_version 1300652 (0.0008) [2023-12-27 00:50:55,861][105620] Updated weights for policy 1, policy_version 1300662 (0.0009) [2023-12-27 00:50:55,920][105620] Updated weights for policy 1, policy_version 1300672 (0.0009) [2023-12-27 00:50:56,062][104569] Fps is (10 sec: 19661.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 665616384. Throughput: 0: 9803.5, 1: 9806.0. Samples: 665626216. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:50:56,062][104569] Avg episode reward: [(0, '8728.789'), (1, '9263.416')] [2023-12-27 00:50:56,085][105692] Updated weights for policy 0, policy_version 1299034 (0.0009) [2023-12-27 00:50:56,145][105692] Updated weights for policy 0, policy_version 1299044 (0.0009) [2023-12-27 00:50:56,206][105692] Updated weights for policy 0, policy_version 1299054 (0.0009) [2023-12-27 00:50:56,547][105620] Updated weights for policy 1, policy_version 1300682 (0.0005) [2023-12-27 00:50:56,600][105620] Updated weights for policy 1, policy_version 1300692 (0.0005) [2023-12-27 00:50:56,651][105620] Updated weights for policy 1, policy_version 1300702 (0.0005) [2023-12-27 00:50:57,118][105692] Updated weights for policy 0, policy_version 1299064 (0.0009) [2023-12-27 00:50:57,180][105692] Updated weights for policy 0, policy_version 1299074 (0.0008) [2023-12-27 00:50:57,213][105620] Updated weights for policy 1, policy_version 1300712 (0.0009) [2023-12-27 00:50:57,227][105692] Updated weights for policy 0, policy_version 1299084 (0.0005) [2023-12-27 00:50:57,261][105620] Updated weights for policy 1, policy_version 1300722 (0.0008) [2023-12-27 00:50:57,326][105620] Updated weights for policy 1, policy_version 1300732 (0.0005) [2023-12-27 00:50:57,968][105620] Updated weights for policy 1, policy_version 1300742 (0.0007) [2023-12-27 00:50:58,028][105620] Updated weights for policy 1, policy_version 1300752 (0.0007) [2023-12-27 00:50:58,043][105692] Updated weights for policy 0, policy_version 1299094 (0.0006) [2023-12-27 00:50:58,078][105620] Updated weights for policy 1, policy_version 1300762 (0.0010) [2023-12-27 00:50:58,088][105692] Updated weights for policy 0, policy_version 1299104 (0.0006) [2023-12-27 00:50:58,137][105692] Updated weights for policy 0, policy_version 1299114 (0.0007) [2023-12-27 00:50:58,848][105620] Updated weights for policy 1, policy_version 1300772 (0.0009) [2023-12-27 00:50:58,912][105620] Updated weights for policy 1, policy_version 1300782 (0.0009) [2023-12-27 00:50:58,969][105620] Updated weights for policy 1, policy_version 1300792 (0.0009) [2023-12-27 00:50:59,046][105692] Updated weights for policy 0, policy_version 1299124 (0.0009) [2023-12-27 00:50:59,099][105692] Updated weights for policy 0, policy_version 1299134 (0.0009) [2023-12-27 00:50:59,152][105692] Updated weights for policy 0, policy_version 1299144 (0.0009) [2023-12-27 00:50:59,769][105620] Updated weights for policy 1, policy_version 1300802 (0.0009) [2023-12-27 00:50:59,831][105620] Updated weights for policy 1, policy_version 1300812 (0.0010) [2023-12-27 00:50:59,898][105620] Updated weights for policy 1, policy_version 1300822 (0.0011) [2023-12-27 00:50:59,920][105692] Updated weights for policy 0, policy_version 1299154 (0.0007) [2023-12-27 00:50:59,965][105620] Updated weights for policy 1, policy_version 1300832 (0.0011) [2023-12-27 00:50:59,979][105692] Updated weights for policy 0, policy_version 1299164 (0.0008) [2023-12-27 00:51:00,028][105692] Updated weights for policy 0, policy_version 1299174 (0.0008) [2023-12-27 00:51:00,073][105692] Updated weights for policy 0, policy_version 1299184 (0.0007) [2023-12-27 00:51:00,597][105620] Updated weights for policy 1, policy_version 1300842 (0.0009) [2023-12-27 00:51:00,650][105620] Updated weights for policy 1, policy_version 1300852 (0.0009) [2023-12-27 00:51:00,716][105620] Updated weights for policy 1, policy_version 1300862 (0.0008) [2023-12-27 00:51:00,862][105692] Updated weights for policy 0, policy_version 1299194 (0.0008) [2023-12-27 00:51:00,908][105692] Updated weights for policy 0, policy_version 1299204 (0.0008) [2023-12-27 00:51:00,962][105692] Updated weights for policy 0, policy_version 1299214 (0.0006) [2023-12-27 00:51:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 665714688. Throughput: 0: 9726.6, 1: 9882.9. Samples: 665683276. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:51:01,062][104569] Avg episode reward: [(0, '8462.771'), (1, '9263.246')] [2023-12-27 00:51:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001299216_332652544.pth... [2023-12-27 00:51:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001300864_333062144.pth... [2023-12-27 00:51:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001298096_332365824.pth [2023-12-27 00:51:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001299712_332767232.pth [2023-12-27 00:51:01,511][105620] Updated weights for policy 1, policy_version 1300872 (0.0008) [2023-12-27 00:51:01,566][105620] Updated weights for policy 1, policy_version 1300882 (0.0008) [2023-12-27 00:51:01,622][105620] Updated weights for policy 1, policy_version 1300892 (0.0008) [2023-12-27 00:51:01,717][105692] Updated weights for policy 0, policy_version 1299224 (0.0009) [2023-12-27 00:51:01,779][105692] Updated weights for policy 0, policy_version 1299234 (0.0009) [2023-12-27 00:51:01,841][105692] Updated weights for policy 0, policy_version 1299244 (0.0010) [2023-12-27 00:51:02,432][105620] Updated weights for policy 1, policy_version 1300902 (0.0009) [2023-12-27 00:51:02,448][105692] Updated weights for policy 0, policy_version 1299254 (0.0010) [2023-12-27 00:51:02,489][105620] Updated weights for policy 1, policy_version 1300912 (0.0008) [2023-12-27 00:51:02,503][105692] Updated weights for policy 0, policy_version 1299264 (0.0010) [2023-12-27 00:51:02,541][105620] Updated weights for policy 1, policy_version 1300922 (0.0005) [2023-12-27 00:51:02,562][105692] Updated weights for policy 0, policy_version 1299274 (0.0010) [2023-12-27 00:51:03,231][105620] Updated weights for policy 1, policy_version 1300932 (0.0006) [2023-12-27 00:51:03,296][105620] Updated weights for policy 1, policy_version 1300942 (0.0009) [2023-12-27 00:51:03,354][105620] Updated weights for policy 1, policy_version 1300952 (0.0010) [2023-12-27 00:51:03,367][105692] Updated weights for policy 0, policy_version 1299284 (0.0008) [2023-12-27 00:51:03,418][105692] Updated weights for policy 0, policy_version 1299294 (0.0006) [2023-12-27 00:51:03,479][105692] Updated weights for policy 0, policy_version 1299304 (0.0006) [2023-12-27 00:51:04,096][105692] Updated weights for policy 0, policy_version 1299314 (0.0007) [2023-12-27 00:51:04,152][105692] Updated weights for policy 0, policy_version 1299324 (0.0009) [2023-12-27 00:51:04,170][105620] Updated weights for policy 1, policy_version 1300962 (0.0011) [2023-12-27 00:51:04,211][105692] Updated weights for policy 0, policy_version 1299334 (0.0007) [2023-12-27 00:51:04,231][105620] Updated weights for policy 1, policy_version 1300972 (0.0007) [2023-12-27 00:51:04,272][105692] Updated weights for policy 0, policy_version 1299344 (0.0008) [2023-12-27 00:51:04,291][105620] Updated weights for policy 1, policy_version 1300982 (0.0009) [2023-12-27 00:51:04,355][105620] Updated weights for policy 1, policy_version 1300992 (0.0009) [2023-12-27 00:51:04,992][105692] Updated weights for policy 0, policy_version 1299354 (0.0009) [2023-12-27 00:51:05,039][105692] Updated weights for policy 0, policy_version 1299364 (0.0008) [2023-12-27 00:51:05,085][105692] Updated weights for policy 0, policy_version 1299374 (0.0008) [2023-12-27 00:51:05,128][105620] Updated weights for policy 1, policy_version 1301002 (0.0008) [2023-12-27 00:51:05,191][105620] Updated weights for policy 1, policy_version 1301012 (0.0007) [2023-12-27 00:51:05,247][105620] Updated weights for policy 1, policy_version 1301022 (0.0005) [2023-12-27 00:51:05,882][105620] Updated weights for policy 1, policy_version 1301032 (0.0006) [2023-12-27 00:51:05,893][105692] Updated weights for policy 0, policy_version 1299384 (0.0008) [2023-12-27 00:51:05,928][105620] Updated weights for policy 1, policy_version 1301042 (0.0005) [2023-12-27 00:51:05,945][105692] Updated weights for policy 0, policy_version 1299394 (0.0008) [2023-12-27 00:51:05,987][105620] Updated weights for policy 1, policy_version 1301052 (0.0009) [2023-12-27 00:51:05,990][105692] Updated weights for policy 0, policy_version 1299404 (0.0007) [2023-12-27 00:51:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 665812992. Throughput: 0: 9815.3, 1: 9752.1. Samples: 665796096. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:51:06,062][104569] Avg episode reward: [(0, '8462.301'), (1, '9354.659')] [2023-12-27 00:51:06,760][105620] Updated weights for policy 1, policy_version 1301062 (0.0009) [2023-12-27 00:51:06,765][105692] Updated weights for policy 0, policy_version 1299414 (0.0007) [2023-12-27 00:51:06,816][105620] Updated weights for policy 1, policy_version 1301072 (0.0006) [2023-12-27 00:51:06,828][105692] Updated weights for policy 0, policy_version 1299424 (0.0008) [2023-12-27 00:51:06,862][105620] Updated weights for policy 1, policy_version 1301082 (0.0007) [2023-12-27 00:51:06,887][105692] Updated weights for policy 0, policy_version 1299434 (0.0008) [2023-12-27 00:51:07,629][105620] Updated weights for policy 1, policy_version 1301092 (0.0008) [2023-12-27 00:51:07,632][105692] Updated weights for policy 0, policy_version 1299444 (0.0008) [2023-12-27 00:51:07,674][105620] Updated weights for policy 1, policy_version 1301102 (0.0007) [2023-12-27 00:51:07,681][105692] Updated weights for policy 0, policy_version 1299454 (0.0006) [2023-12-27 00:51:07,721][105620] Updated weights for policy 1, policy_version 1301112 (0.0006) [2023-12-27 00:51:07,731][105692] Updated weights for policy 0, policy_version 1299464 (0.0006) [2023-12-27 00:51:08,378][105620] Updated weights for policy 1, policy_version 1301122 (0.0007) [2023-12-27 00:51:08,436][105620] Updated weights for policy 1, policy_version 1301132 (0.0008) [2023-12-27 00:51:08,485][105620] Updated weights for policy 1, policy_version 1301142 (0.0009) [2023-12-27 00:51:08,532][105620] Updated weights for policy 1, policy_version 1301152 (0.0009) [2023-12-27 00:51:08,545][105692] Updated weights for policy 0, policy_version 1299474 (0.0007) [2023-12-27 00:51:08,592][105692] Updated weights for policy 0, policy_version 1299484 (0.0009) [2023-12-27 00:51:08,650][105692] Updated weights for policy 0, policy_version 1299494 (0.0009) [2023-12-27 00:51:08,700][105692] Updated weights for policy 0, policy_version 1299504 (0.0009) [2023-12-27 00:51:09,262][105620] Updated weights for policy 1, policy_version 1301162 (0.0006) [2023-12-27 00:51:09,333][105620] Updated weights for policy 1, policy_version 1301172 (0.0008) [2023-12-27 00:51:09,405][105620] Updated weights for policy 1, policy_version 1301182 (0.0010) [2023-12-27 00:51:09,518][105692] Updated weights for policy 0, policy_version 1299514 (0.0009) [2023-12-27 00:51:09,580][105692] Updated weights for policy 0, policy_version 1299524 (0.0010) [2023-12-27 00:51:09,639][105692] Updated weights for policy 0, policy_version 1299534 (0.0010) [2023-12-27 00:51:10,050][105620] Updated weights for policy 1, policy_version 1301192 (0.0007) [2023-12-27 00:51:10,111][105620] Updated weights for policy 1, policy_version 1301202 (0.0006) [2023-12-27 00:51:10,175][105620] Updated weights for policy 1, policy_version 1301212 (0.0011) [2023-12-27 00:51:10,490][105692] Updated weights for policy 0, policy_version 1299544 (0.0008) [2023-12-27 00:51:10,546][105692] Updated weights for policy 0, policy_version 1299554 (0.0008) [2023-12-27 00:51:10,597][105692] Updated weights for policy 0, policy_version 1299564 (0.0008) [2023-12-27 00:51:10,858][105620] Updated weights for policy 1, policy_version 1301222 (0.0010) [2023-12-27 00:51:10,910][105620] Updated weights for policy 1, policy_version 1301232 (0.0009) [2023-12-27 00:51:10,962][105620] Updated weights for policy 1, policy_version 1301242 (0.0009) [2023-12-27 00:51:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 665903104. Throughput: 0: 9700.6, 1: 9834.9. Samples: 665909160. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:51:11,062][104569] Avg episode reward: [(0, '8731.490'), (1, '9263.771')] [2023-12-27 00:51:11,358][105692] Updated weights for policy 0, policy_version 1299574 (0.0009) [2023-12-27 00:51:11,420][105692] Updated weights for policy 0, policy_version 1299584 (0.0009) [2023-12-27 00:51:11,479][105692] Updated weights for policy 0, policy_version 1299594 (0.0006) [2023-12-27 00:51:11,780][105620] Updated weights for policy 1, policy_version 1301252 (0.0008) [2023-12-27 00:51:11,843][105620] Updated weights for policy 1, policy_version 1301262 (0.0008) [2023-12-27 00:51:11,910][105620] Updated weights for policy 1, policy_version 1301272 (0.0008) [2023-12-27 00:51:12,160][105692] Updated weights for policy 0, policy_version 1299604 (0.0007) [2023-12-27 00:51:12,214][105692] Updated weights for policy 0, policy_version 1299614 (0.0009) [2023-12-27 00:51:12,266][105692] Updated weights for policy 0, policy_version 1299624 (0.0009) [2023-12-27 00:51:12,577][105620] Updated weights for policy 1, policy_version 1301282 (0.0007) [2023-12-27 00:51:12,639][105620] Updated weights for policy 1, policy_version 1301292 (0.0005) [2023-12-27 00:51:12,701][105620] Updated weights for policy 1, policy_version 1301302 (0.0006) [2023-12-27 00:51:12,760][105620] Updated weights for policy 1, policy_version 1301312 (0.0009) [2023-12-27 00:51:13,064][105692] Updated weights for policy 0, policy_version 1299634 (0.0009) [2023-12-27 00:51:13,124][105692] Updated weights for policy 0, policy_version 1299644 (0.0009) [2023-12-27 00:51:13,174][105692] Updated weights for policy 0, policy_version 1299654 (0.0009) [2023-12-27 00:51:13,221][105692] Updated weights for policy 0, policy_version 1299664 (0.0009) [2023-12-27 00:51:13,446][105620] Updated weights for policy 1, policy_version 1301322 (0.0009) [2023-12-27 00:51:13,509][105620] Updated weights for policy 1, policy_version 1301332 (0.0009) [2023-12-27 00:51:13,562][105620] Updated weights for policy 1, policy_version 1301342 (0.0009) [2023-12-27 00:51:13,913][105692] Updated weights for policy 0, policy_version 1299674 (0.0008) [2023-12-27 00:51:13,973][105692] Updated weights for policy 0, policy_version 1299684 (0.0009) [2023-12-27 00:51:14,023][105692] Updated weights for policy 0, policy_version 1299694 (0.0009) [2023-12-27 00:51:14,322][105620] Updated weights for policy 1, policy_version 1301352 (0.0009) [2023-12-27 00:51:14,377][105620] Updated weights for policy 1, policy_version 1301362 (0.0009) [2023-12-27 00:51:14,438][105620] Updated weights for policy 1, policy_version 1301372 (0.0009) [2023-12-27 00:51:14,783][105692] Updated weights for policy 0, policy_version 1299704 (0.0009) [2023-12-27 00:51:14,842][105692] Updated weights for policy 0, policy_version 1299714 (0.0009) [2023-12-27 00:51:14,895][105692] Updated weights for policy 0, policy_version 1299724 (0.0009) [2023-12-27 00:51:15,221][105620] Updated weights for policy 1, policy_version 1301382 (0.0008) [2023-12-27 00:51:15,279][105620] Updated weights for policy 1, policy_version 1301392 (0.0009) [2023-12-27 00:51:15,343][105620] Updated weights for policy 1, policy_version 1301402 (0.0008) [2023-12-27 00:51:15,739][105692] Updated weights for policy 0, policy_version 1299734 (0.0007) [2023-12-27 00:51:15,801][105692] Updated weights for policy 0, policy_version 1299744 (0.0005) [2023-12-27 00:51:15,848][105692] Updated weights for policy 0, policy_version 1299754 (0.0005) [2023-12-27 00:51:15,960][105620] Updated weights for policy 1, policy_version 1301412 (0.0005) [2023-12-27 00:51:16,024][105620] Updated weights for policy 1, policy_version 1301422 (0.0005) [2023-12-27 00:51:16,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 665993216. Throughput: 0: 9543.2, 1: 9720.3. Samples: 665965888. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:51:16,062][104569] Avg episode reward: [(0, '8819.847'), (1, '9171.165')] [2023-12-27 00:51:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001299760_332791808.pth... [2023-12-27 00:51:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001298672_332513280.pth [2023-12-27 00:51:16,080][105620] Updated weights for policy 1, policy_version 1301432 (0.0008) [2023-12-27 00:51:16,116][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001301440_333209600.pth... [2023-12-27 00:51:16,119][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001300288_332914688.pth [2023-12-27 00:51:16,379][105692] Updated weights for policy 0, policy_version 1299764 (0.0008) [2023-12-27 00:51:16,427][105692] Updated weights for policy 0, policy_version 1299774 (0.0010) [2023-12-27 00:51:16,493][105692] Updated weights for policy 0, policy_version 1299784 (0.0011) [2023-12-27 00:51:16,857][105620] Updated weights for policy 1, policy_version 1301443 (0.0009) [2023-12-27 00:51:16,917][105620] Updated weights for policy 1, policy_version 1301453 (0.0009) [2023-12-27 00:51:16,971][105620] Updated weights for policy 1, policy_version 1301463 (0.0006) [2023-12-27 00:51:17,160][105692] Updated weights for policy 0, policy_version 1299794 (0.0010) [2023-12-27 00:51:17,210][105692] Updated weights for policy 0, policy_version 1299804 (0.0005) [2023-12-27 00:51:17,256][105692] Updated weights for policy 0, policy_version 1299814 (0.0005) [2023-12-27 00:51:17,326][105692] Updated weights for policy 0, policy_version 1299824 (0.0005) [2023-12-27 00:51:17,715][105620] Updated weights for policy 1, policy_version 1301473 (0.0006) [2023-12-27 00:51:17,777][105620] Updated weights for policy 1, policy_version 1301483 (0.0007) [2023-12-27 00:51:17,830][105620] Updated weights for policy 1, policy_version 1301493 (0.0005) [2023-12-27 00:51:17,878][105620] Updated weights for policy 1, policy_version 1301503 (0.0005) [2023-12-27 00:51:17,967][105692] Updated weights for policy 0, policy_version 1299834 (0.0009) [2023-12-27 00:51:18,031][105692] Updated weights for policy 0, policy_version 1299844 (0.0008) [2023-12-27 00:51:18,100][105692] Updated weights for policy 0, policy_version 1299854 (0.0009) [2023-12-27 00:51:18,496][105620] Updated weights for policy 1, policy_version 1301513 (0.0005) [2023-12-27 00:51:18,555][105620] Updated weights for policy 1, policy_version 1301523 (0.0006) [2023-12-27 00:51:18,617][105620] Updated weights for policy 1, policy_version 1301533 (0.0007) [2023-12-27 00:51:18,944][105692] Updated weights for policy 0, policy_version 1299864 (0.0010) [2023-12-27 00:51:18,999][105692] Updated weights for policy 0, policy_version 1299874 (0.0010) [2023-12-27 00:51:19,062][105692] Updated weights for policy 0, policy_version 1299884 (0.0010) [2023-12-27 00:51:19,212][105620] Updated weights for policy 1, policy_version 1301543 (0.0007) [2023-12-27 00:51:19,275][105620] Updated weights for policy 1, policy_version 1301553 (0.0008) [2023-12-27 00:51:19,324][105620] Updated weights for policy 1, policy_version 1301563 (0.0007) [2023-12-27 00:51:19,847][105692] Updated weights for policy 0, policy_version 1299894 (0.0010) [2023-12-27 00:51:19,896][105692] Updated weights for policy 0, policy_version 1299904 (0.0009) [2023-12-27 00:51:19,962][105692] Updated weights for policy 0, policy_version 1299914 (0.0009) [2023-12-27 00:51:20,077][105620] Updated weights for policy 1, policy_version 1301573 (0.0009) [2023-12-27 00:51:20,135][105620] Updated weights for policy 1, policy_version 1301583 (0.0010) [2023-12-27 00:51:20,187][105620] Updated weights for policy 1, policy_version 1301593 (0.0008) [2023-12-27 00:51:20,711][105692] Updated weights for policy 0, policy_version 1299924 (0.0007) [2023-12-27 00:51:20,768][105692] Updated weights for policy 0, policy_version 1299934 (0.0009) [2023-12-27 00:51:20,824][105692] Updated weights for policy 0, policy_version 1299944 (0.0009) [2023-12-27 00:51:20,941][105620] Updated weights for policy 1, policy_version 1301603 (0.0009) [2023-12-27 00:51:20,999][105620] Updated weights for policy 1, policy_version 1301613 (0.0009) [2023-12-27 00:51:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19438.7). Total num frames: 666091520. Throughput: 0: 9438.2, 1: 9778.7. Samples: 666083856. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:51:21,062][104569] Avg episode reward: [(0, '8640.287'), (1, '9262.378')] [2023-12-27 00:51:21,068][105620] Updated weights for policy 1, policy_version 1301623 (0.0008) [2023-12-27 00:51:21,613][105692] Updated weights for policy 0, policy_version 1299954 (0.0009) [2023-12-27 00:51:21,676][105692] Updated weights for policy 0, policy_version 1299964 (0.0010) [2023-12-27 00:51:21,728][105692] Updated weights for policy 0, policy_version 1299974 (0.0009) [2023-12-27 00:51:21,790][105692] Updated weights for policy 0, policy_version 1299984 (0.0009) [2023-12-27 00:51:21,796][105620] Updated weights for policy 1, policy_version 1301633 (0.0006) [2023-12-27 00:51:21,867][105620] Updated weights for policy 1, policy_version 1301643 (0.0008) [2023-12-27 00:51:21,937][105620] Updated weights for policy 1, policy_version 1301653 (0.0010) [2023-12-27 00:51:22,006][105620] Updated weights for policy 1, policy_version 1301663 (0.0009) [2023-12-27 00:51:22,522][105692] Updated weights for policy 0, policy_version 1299994 (0.0008) [2023-12-27 00:51:22,580][105692] Updated weights for policy 0, policy_version 1300005 (0.0010) [2023-12-27 00:51:22,646][105692] Updated weights for policy 0, policy_version 1300015 (0.0008) [2023-12-27 00:51:22,711][105620] Updated weights for policy 1, policy_version 1301673 (0.0009) [2023-12-27 00:51:22,763][105620] Updated weights for policy 1, policy_version 1301683 (0.0009) [2023-12-27 00:51:22,814][105620] Updated weights for policy 1, policy_version 1301693 (0.0008) [2023-12-27 00:51:23,396][105692] Updated weights for policy 0, policy_version 1300025 (0.0010) [2023-12-27 00:51:23,462][105692] Updated weights for policy 0, policy_version 1300035 (0.0009) [2023-12-27 00:51:23,520][105692] Updated weights for policy 0, policy_version 1300045 (0.0007) [2023-12-27 00:51:23,522][105620] Updated weights for policy 1, policy_version 1301703 (0.0008) [2023-12-27 00:51:23,582][105620] Updated weights for policy 1, policy_version 1301713 (0.0008) [2023-12-27 00:51:23,640][105620] Updated weights for policy 1, policy_version 1301723 (0.0009) [2023-12-27 00:51:24,320][105620] Updated weights for policy 1, policy_version 1301733 (0.0009) [2023-12-27 00:51:24,334][105692] Updated weights for policy 0, policy_version 1300055 (0.0009) [2023-12-27 00:51:24,376][105620] Updated weights for policy 1, policy_version 1301743 (0.0008) [2023-12-27 00:51:24,393][105692] Updated weights for policy 0, policy_version 1300065 (0.0006) [2023-12-27 00:51:24,436][105620] Updated weights for policy 1, policy_version 1301753 (0.0009) [2023-12-27 00:51:24,452][105692] Updated weights for policy 0, policy_version 1300075 (0.0005) [2023-12-27 00:51:25,146][105620] Updated weights for policy 1, policy_version 1301763 (0.0008) [2023-12-27 00:51:25,151][105692] Updated weights for policy 0, policy_version 1300085 (0.0008) [2023-12-27 00:51:25,201][105620] Updated weights for policy 1, policy_version 1301773 (0.0006) [2023-12-27 00:51:25,211][105692] Updated weights for policy 0, policy_version 1300095 (0.0011) [2023-12-27 00:51:25,262][105620] Updated weights for policy 1, policy_version 1301783 (0.0006) [2023-12-27 00:51:25,267][105692] Updated weights for policy 0, policy_version 1300105 (0.0011) [2023-12-27 00:51:25,958][105620] Updated weights for policy 1, policy_version 1301793 (0.0007) [2023-12-27 00:51:26,004][105692] Updated weights for policy 0, policy_version 1300115 (0.0009) [2023-12-27 00:51:26,023][105620] Updated weights for policy 1, policy_version 1301803 (0.0009) [2023-12-27 00:51:26,050][105692] Updated weights for policy 0, policy_version 1300125 (0.0005) [2023-12-27 00:51:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.3, 300 sec: 19410.9). Total num frames: 666181632. Throughput: 0: 9408.8, 1: 9799.9. Samples: 666198124. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:51:26,062][104569] Avg episode reward: [(0, '8642.621'), (1, '9263.648')] [2023-12-27 00:51:26,079][105620] Updated weights for policy 1, policy_version 1301813 (0.0008) [2023-12-27 00:51:26,098][105692] Updated weights for policy 0, policy_version 1300135 (0.0005) [2023-12-27 00:51:26,132][105620] Updated weights for policy 1, policy_version 1301823 (0.0009) [2023-12-27 00:51:26,627][105692] Updated weights for policy 0, policy_version 1300145 (0.0005) [2023-12-27 00:51:26,683][105692] Updated weights for policy 0, policy_version 1300155 (0.0005) [2023-12-27 00:51:26,738][105692] Updated weights for policy 0, policy_version 1300165 (0.0006) [2023-12-27 00:51:26,790][105692] Updated weights for policy 0, policy_version 1300175 (0.0010) [2023-12-27 00:51:27,011][105620] Updated weights for policy 1, policy_version 1301833 (0.0008) [2023-12-27 00:51:27,069][105620] Updated weights for policy 1, policy_version 1301843 (0.0008) [2023-12-27 00:51:27,120][105620] Updated weights for policy 1, policy_version 1301853 (0.0007) [2023-12-27 00:51:27,478][105692] Updated weights for policy 0, policy_version 1300185 (0.0009) [2023-12-27 00:51:27,530][105692] Updated weights for policy 0, policy_version 1300195 (0.0007) [2023-12-27 00:51:27,578][105692] Updated weights for policy 0, policy_version 1300205 (0.0010) [2023-12-27 00:51:27,805][105620] Updated weights for policy 1, policy_version 1301863 (0.0008) [2023-12-27 00:51:27,849][105620] Updated weights for policy 1, policy_version 1301873 (0.0007) [2023-12-27 00:51:27,894][105620] Updated weights for policy 1, policy_version 1301883 (0.0008) [2023-12-27 00:51:28,225][105692] Updated weights for policy 0, policy_version 1300215 (0.0011) [2023-12-27 00:51:28,271][105692] Updated weights for policy 0, policy_version 1300225 (0.0006) [2023-12-27 00:51:28,327][105692] Updated weights for policy 0, policy_version 1300235 (0.0006) [2023-12-27 00:51:28,784][105620] Updated weights for policy 1, policy_version 1301894 (0.0009) [2023-12-27 00:51:28,840][105620] Updated weights for policy 1, policy_version 1301904 (0.0009) [2023-12-27 00:51:28,890][105620] Updated weights for policy 1, policy_version 1301914 (0.0008) [2023-12-27 00:51:28,904][105692] Updated weights for policy 0, policy_version 1300246 (0.0010) [2023-12-27 00:51:28,953][105692] Updated weights for policy 0, policy_version 1300256 (0.0008) [2023-12-27 00:51:28,998][105692] Updated weights for policy 0, policy_version 1300266 (0.0005) [2023-12-27 00:51:29,597][105620] Updated weights for policy 1, policy_version 1301924 (0.0006) [2023-12-27 00:51:29,630][105692] Updated weights for policy 0, policy_version 1300276 (0.0006) [2023-12-27 00:51:29,662][105620] Updated weights for policy 1, policy_version 1301934 (0.0009) [2023-12-27 00:51:29,686][105692] Updated weights for policy 0, policy_version 1300286 (0.0005) [2023-12-27 00:51:29,712][105620] Updated weights for policy 1, policy_version 1301944 (0.0009) [2023-12-27 00:51:29,744][105692] Updated weights for policy 0, policy_version 1300296 (0.0006) [2023-12-27 00:51:30,401][105620] Updated weights for policy 1, policy_version 1301954 (0.0005) [2023-12-27 00:51:30,461][105620] Updated weights for policy 1, policy_version 1301964 (0.0005) [2023-12-27 00:51:30,464][105692] Updated weights for policy 0, policy_version 1300306 (0.0009) [2023-12-27 00:51:30,520][105692] Updated weights for policy 0, policy_version 1300316 (0.0006) [2023-12-27 00:51:30,520][105620] Updated weights for policy 1, policy_version 1301974 (0.0008) [2023-12-27 00:51:30,571][105692] Updated weights for policy 0, policy_version 1300326 (0.0005) [2023-12-27 00:51:30,571][105620] Updated weights for policy 1, policy_version 1301984 (0.0010) [2023-12-27 00:51:30,615][105692] Updated weights for policy 0, policy_version 1300336 (0.0005) [2023-12-27 00:51:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 666288128. Throughput: 0: 9520.1, 1: 9720.0. Samples: 666257140. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:51:31,062][104569] Avg episode reward: [(0, '8812.179'), (1, '9263.597')] [2023-12-27 00:51:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001300336_332939264.pth... [2023-12-27 00:51:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001301984_333348864.pth... [2023-12-27 00:51:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001299216_332652544.pth [2023-12-27 00:51:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001300864_333062144.pth [2023-12-27 00:51:31,160][105692] Updated weights for policy 0, policy_version 1300346 (0.0010) [2023-12-27 00:51:31,210][105692] Updated weights for policy 0, policy_version 1300356 (0.0008) [2023-12-27 00:51:31,236][105620] Updated weights for policy 1, policy_version 1301994 (0.0009) [2023-12-27 00:51:31,272][105692] Updated weights for policy 0, policy_version 1300366 (0.0007) [2023-12-27 00:51:31,292][105620] Updated weights for policy 1, policy_version 1302004 (0.0010) [2023-12-27 00:51:31,345][105620] Updated weights for policy 1, policy_version 1302014 (0.0010) [2023-12-27 00:51:32,009][105692] Updated weights for policy 0, policy_version 1300376 (0.0010) [2023-12-27 00:51:32,060][105692] Updated weights for policy 0, policy_version 1300386 (0.0010) [2023-12-27 00:51:32,078][105620] Updated weights for policy 1, policy_version 1302024 (0.0007) [2023-12-27 00:51:32,105][105692] Updated weights for policy 0, policy_version 1300396 (0.0010) [2023-12-27 00:51:32,138][105620] Updated weights for policy 1, policy_version 1302034 (0.0006) [2023-12-27 00:51:32,201][105620] Updated weights for policy 1, policy_version 1302044 (0.0008) [2023-12-27 00:51:32,857][105692] Updated weights for policy 0, policy_version 1300406 (0.0009) [2023-12-27 00:51:32,872][105620] Updated weights for policy 1, policy_version 1302054 (0.0006) [2023-12-27 00:51:32,919][105692] Updated weights for policy 0, policy_version 1300416 (0.0008) [2023-12-27 00:51:32,923][105620] Updated weights for policy 1, policy_version 1302064 (0.0005) [2023-12-27 00:51:32,974][105620] Updated weights for policy 1, policy_version 1302074 (0.0005) [2023-12-27 00:51:32,981][105692] Updated weights for policy 0, policy_version 1300426 (0.0009) [2023-12-27 00:51:33,591][105692] Updated weights for policy 0, policy_version 1300436 (0.0008) [2023-12-27 00:51:33,602][105620] Updated weights for policy 1, policy_version 1302084 (0.0006) [2023-12-27 00:51:33,647][105692] Updated weights for policy 0, policy_version 1300446 (0.0009) [2023-12-27 00:51:33,661][105620] Updated weights for policy 1, policy_version 1302094 (0.0005) [2023-12-27 00:51:33,708][105692] Updated weights for policy 0, policy_version 1300456 (0.0009) [2023-12-27 00:51:33,709][105620] Updated weights for policy 1, policy_version 1302104 (0.0005) [2023-12-27 00:51:34,257][105620] Updated weights for policy 1, policy_version 1302114 (0.0006) [2023-12-27 00:51:34,324][105620] Updated weights for policy 1, policy_version 1302124 (0.0011) [2023-12-27 00:51:34,348][105692] Updated weights for policy 0, policy_version 1300466 (0.0005) [2023-12-27 00:51:34,385][105620] Updated weights for policy 1, policy_version 1302134 (0.0009) [2023-12-27 00:51:34,408][105692] Updated weights for policy 0, policy_version 1300476 (0.0006) [2023-12-27 00:51:34,455][105620] Updated weights for policy 1, policy_version 1302144 (0.0008) [2023-12-27 00:51:34,470][105692] Updated weights for policy 0, policy_version 1300486 (0.0007) [2023-12-27 00:51:34,529][105692] Updated weights for policy 0, policy_version 1300496 (0.0007) [2023-12-27 00:51:35,079][105620] Updated weights for policy 1, policy_version 1302154 (0.0010) [2023-12-27 00:51:35,097][105692] Updated weights for policy 0, policy_version 1300506 (0.0005) [2023-12-27 00:51:35,137][105620] Updated weights for policy 1, policy_version 1302164 (0.0009) [2023-12-27 00:51:35,148][105692] Updated weights for policy 0, policy_version 1300516 (0.0005) [2023-12-27 00:51:35,187][105620] Updated weights for policy 1, policy_version 1302174 (0.0007) [2023-12-27 00:51:35,204][105692] Updated weights for policy 0, policy_version 1300526 (0.0006) [2023-12-27 00:51:35,777][105692] Updated weights for policy 0, policy_version 1300536 (0.0005) [2023-12-27 00:51:35,833][105692] Updated weights for policy 0, policy_version 1300546 (0.0005) [2023-12-27 00:51:35,874][105620] Updated weights for policy 1, policy_version 1302184 (0.0008) [2023-12-27 00:51:35,894][105692] Updated weights for policy 0, policy_version 1300556 (0.0007) [2023-12-27 00:51:35,929][105620] Updated weights for policy 1, policy_version 1302195 (0.0006) [2023-12-27 00:51:35,981][105620] Updated weights for policy 1, policy_version 1302205 (0.0005) [2023-12-27 00:51:36,062][104569] Fps is (10 sec: 22118.0, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 666402816. Throughput: 0: 9536.9, 1: 9795.1. Samples: 666383160. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:51:36,063][104569] Avg episode reward: [(0, '8719.119'), (1, '9263.338')] [2023-12-27 00:51:36,428][105692] Updated weights for policy 0, policy_version 1300566 (0.0010) [2023-12-27 00:51:36,485][105692] Updated weights for policy 0, policy_version 1300576 (0.0010) [2023-12-27 00:51:36,549][105692] Updated weights for policy 0, policy_version 1300586 (0.0010) [2023-12-27 00:51:36,731][105620] Updated weights for policy 1, policy_version 1302215 (0.0008) [2023-12-27 00:51:36,790][105620] Updated weights for policy 1, policy_version 1302225 (0.0009) [2023-12-27 00:51:36,852][105620] Updated weights for policy 1, policy_version 1302235 (0.0009) [2023-12-27 00:51:37,314][105692] Updated weights for policy 0, policy_version 1300596 (0.0010) [2023-12-27 00:51:37,378][105692] Updated weights for policy 0, policy_version 1300606 (0.0009) [2023-12-27 00:51:37,440][105692] Updated weights for policy 0, policy_version 1300616 (0.0009) [2023-12-27 00:51:37,540][105620] Updated weights for policy 1, policy_version 1302245 (0.0009) [2023-12-27 00:51:37,595][105620] Updated weights for policy 1, policy_version 1302255 (0.0006) [2023-12-27 00:51:37,645][105620] Updated weights for policy 1, policy_version 1302265 (0.0006) [2023-12-27 00:51:38,257][105620] Updated weights for policy 1, policy_version 1302275 (0.0009) [2023-12-27 00:51:38,286][105692] Updated weights for policy 0, policy_version 1300626 (0.0008) [2023-12-27 00:51:38,322][105620] Updated weights for policy 1, policy_version 1302285 (0.0010) [2023-12-27 00:51:38,340][105692] Updated weights for policy 0, policy_version 1300636 (0.0009) [2023-12-27 00:51:38,392][105620] Updated weights for policy 1, policy_version 1302295 (0.0010) [2023-12-27 00:51:38,405][105692] Updated weights for policy 0, policy_version 1300646 (0.0008) [2023-12-27 00:51:38,429][105585] KL-divergence is very high: 132.6255 [2023-12-27 00:51:38,459][105692] Updated weights for policy 0, policy_version 1300656 (0.0007) [2023-12-27 00:51:39,110][105692] Updated weights for policy 0, policy_version 1300666 (0.0005) [2023-12-27 00:51:39,119][105620] Updated weights for policy 1, policy_version 1302305 (0.0010) [2023-12-27 00:51:39,175][105692] Updated weights for policy 0, policy_version 1300676 (0.0005) [2023-12-27 00:51:39,177][105620] Updated weights for policy 1, policy_version 1302315 (0.0010) [2023-12-27 00:51:39,239][105620] Updated weights for policy 1, policy_version 1302325 (0.0010) [2023-12-27 00:51:39,240][105692] Updated weights for policy 0, policy_version 1300686 (0.0009) [2023-12-27 00:51:39,305][105620] Updated weights for policy 1, policy_version 1302335 (0.0010) [2023-12-27 00:51:39,964][105692] Updated weights for policy 0, policy_version 1300696 (0.0009) [2023-12-27 00:51:40,021][105692] Updated weights for policy 0, policy_version 1300706 (0.0008) [2023-12-27 00:51:40,052][105620] Updated weights for policy 1, policy_version 1302345 (0.0008) [2023-12-27 00:51:40,080][105692] Updated weights for policy 0, policy_version 1300716 (0.0007) [2023-12-27 00:51:40,113][105620] Updated weights for policy 1, policy_version 1302355 (0.0005) [2023-12-27 00:51:40,182][105620] Updated weights for policy 1, policy_version 1302365 (0.0009) [2023-12-27 00:51:40,820][105620] Updated weights for policy 1, policy_version 1302375 (0.0008) [2023-12-27 00:51:40,876][105620] Updated weights for policy 1, policy_version 1302385 (0.0007) [2023-12-27 00:51:40,905][105692] Updated weights for policy 0, policy_version 1300726 (0.0008) [2023-12-27 00:51:40,927][105620] Updated weights for policy 1, policy_version 1302395 (0.0007) [2023-12-27 00:51:40,969][105692] Updated weights for policy 0, policy_version 1300736 (0.0009) [2023-12-27 00:51:41,036][105692] Updated weights for policy 0, policy_version 1300746 (0.0009) [2023-12-27 00:51:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 666492928. Throughput: 0: 9689.2, 1: 9821.0. Samples: 666504176. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:51:41,062][104569] Avg episode reward: [(0, '8184.210'), (1, '9176.062')] [2023-12-27 00:51:41,698][105620] Updated weights for policy 1, policy_version 1302405 (0.0009) [2023-12-27 00:51:41,763][105620] Updated weights for policy 1, policy_version 1302415 (0.0009) [2023-12-27 00:51:41,801][105692] Updated weights for policy 0, policy_version 1300756 (0.0009) [2023-12-27 00:51:41,817][105620] Updated weights for policy 1, policy_version 1302425 (0.0007) [2023-12-27 00:51:41,863][105692] Updated weights for policy 0, policy_version 1300766 (0.0009) [2023-12-27 00:51:41,926][105692] Updated weights for policy 0, policy_version 1300776 (0.0009) [2023-12-27 00:51:42,627][105620] Updated weights for policy 1, policy_version 1302435 (0.0007) [2023-12-27 00:51:42,633][105692] Updated weights for policy 0, policy_version 1300786 (0.0009) [2023-12-27 00:51:42,683][105620] Updated weights for policy 1, policy_version 1302445 (0.0006) [2023-12-27 00:51:42,686][105692] Updated weights for policy 0, policy_version 1300796 (0.0010) [2023-12-27 00:51:42,735][105620] Updated weights for policy 1, policy_version 1302455 (0.0008) [2023-12-27 00:51:42,742][105692] Updated weights for policy 0, policy_version 1300806 (0.0005) [2023-12-27 00:51:42,800][105692] Updated weights for policy 0, policy_version 1300816 (0.0008) [2023-12-27 00:51:43,352][105692] Updated weights for policy 0, policy_version 1300826 (0.0009) [2023-12-27 00:51:43,410][105692] Updated weights for policy 0, policy_version 1300836 (0.0010) [2023-12-27 00:51:43,472][105692] Updated weights for policy 0, policy_version 1300846 (0.0010) [2023-12-27 00:51:43,548][105620] Updated weights for policy 1, policy_version 1302465 (0.0009) [2023-12-27 00:51:43,606][105620] Updated weights for policy 1, policy_version 1302475 (0.0005) [2023-12-27 00:51:43,664][105620] Updated weights for policy 1, policy_version 1302485 (0.0006) [2023-12-27 00:51:43,708][105620] Updated weights for policy 1, policy_version 1302495 (0.0008) [2023-12-27 00:51:44,206][105692] Updated weights for policy 0, policy_version 1300856 (0.0011) [2023-12-27 00:51:44,254][105692] Updated weights for policy 0, policy_version 1300866 (0.0010) [2023-12-27 00:51:44,281][105620] Updated weights for policy 1, policy_version 1302505 (0.0006) [2023-12-27 00:51:44,305][105692] Updated weights for policy 0, policy_version 1300876 (0.0010) [2023-12-27 00:51:44,339][105620] Updated weights for policy 1, policy_version 1302515 (0.0009) [2023-12-27 00:51:44,399][105620] Updated weights for policy 1, policy_version 1302525 (0.0009) [2023-12-27 00:51:44,956][105620] Updated weights for policy 1, policy_version 1302535 (0.0005) [2023-12-27 00:51:45,003][105620] Updated weights for policy 1, policy_version 1302545 (0.0006) [2023-12-27 00:51:45,063][105620] Updated weights for policy 1, policy_version 1302555 (0.0006) [2023-12-27 00:51:45,074][105692] Updated weights for policy 0, policy_version 1300886 (0.0010) [2023-12-27 00:51:45,133][105692] Updated weights for policy 0, policy_version 1300896 (0.0010) [2023-12-27 00:51:45,190][105692] Updated weights for policy 0, policy_version 1300906 (0.0011) [2023-12-27 00:51:45,720][105620] Updated weights for policy 1, policy_version 1302565 (0.0006) [2023-12-27 00:51:45,785][105620] Updated weights for policy 1, policy_version 1302575 (0.0009) [2023-12-27 00:51:45,843][105620] Updated weights for policy 1, policy_version 1302585 (0.0010) [2023-12-27 00:51:45,950][105692] Updated weights for policy 0, policy_version 1300916 (0.0011) [2023-12-27 00:51:46,013][105692] Updated weights for policy 0, policy_version 1300926 (0.0011) [2023-12-27 00:51:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.4, 300 sec: 19466.4). Total num frames: 666591232. Throughput: 0: 9781.8, 1: 9718.9. Samples: 666560808. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:51:46,063][104569] Avg episode reward: [(0, '8368.697'), (1, '9085.115')] [2023-12-27 00:51:46,066][105692] Updated weights for policy 0, policy_version 1300936 (0.0011) [2023-12-27 00:51:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001302592_333504512.pth... [2023-12-27 00:51:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001301440_333209600.pth [2023-12-27 00:51:46,104][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001300944_333094912.pth... [2023-12-27 00:51:46,107][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001299760_332791808.pth [2023-12-27 00:51:46,501][105620] Updated weights for policy 1, policy_version 1302595 (0.0010) [2023-12-27 00:51:46,559][105620] Updated weights for policy 1, policy_version 1302605 (0.0010) [2023-12-27 00:51:46,617][105620] Updated weights for policy 1, policy_version 1302615 (0.0010) [2023-12-27 00:51:46,808][105692] Updated weights for policy 0, policy_version 1300946 (0.0011) [2023-12-27 00:51:46,873][105692] Updated weights for policy 0, policy_version 1300956 (0.0009) [2023-12-27 00:51:46,918][105692] Updated weights for policy 0, policy_version 1300966 (0.0005) [2023-12-27 00:51:46,964][105692] Updated weights for policy 0, policy_version 1300976 (0.0005) [2023-12-27 00:51:47,229][105620] Updated weights for policy 1, policy_version 1302625 (0.0010) [2023-12-27 00:51:47,277][105620] Updated weights for policy 1, policy_version 1302635 (0.0010) [2023-12-27 00:51:47,334][105620] Updated weights for policy 1, policy_version 1302645 (0.0010) [2023-12-27 00:51:47,385][105620] Updated weights for policy 1, policy_version 1302655 (0.0010) [2023-12-27 00:51:47,518][105692] Updated weights for policy 0, policy_version 1300986 (0.0010) [2023-12-27 00:51:47,566][105692] Updated weights for policy 0, policy_version 1300996 (0.0010) [2023-12-27 00:51:47,622][105692] Updated weights for policy 0, policy_version 1301006 (0.0010) [2023-12-27 00:51:48,149][105620] Updated weights for policy 1, policy_version 1302665 (0.0009) [2023-12-27 00:51:48,215][105620] Updated weights for policy 1, policy_version 1302675 (0.0008) [2023-12-27 00:51:48,273][105620] Updated weights for policy 1, policy_version 1302685 (0.0008) [2023-12-27 00:51:48,307][105692] Updated weights for policy 0, policy_version 1301016 (0.0008) [2023-12-27 00:51:48,362][105692] Updated weights for policy 0, policy_version 1301026 (0.0009) [2023-12-27 00:51:48,427][105692] Updated weights for policy 0, policy_version 1301036 (0.0009) [2023-12-27 00:51:49,063][105620] Updated weights for policy 1, policy_version 1302695 (0.0010) [2023-12-27 00:51:49,118][105620] Updated weights for policy 1, policy_version 1302705 (0.0010) [2023-12-27 00:51:49,136][105692] Updated weights for policy 0, policy_version 1301046 (0.0007) [2023-12-27 00:51:49,177][105620] Updated weights for policy 1, policy_version 1302715 (0.0009) [2023-12-27 00:51:49,191][105692] Updated weights for policy 0, policy_version 1301056 (0.0006) [2023-12-27 00:51:49,252][105692] Updated weights for policy 0, policy_version 1301066 (0.0008) [2023-12-27 00:51:49,928][105692] Updated weights for policy 0, policy_version 1301076 (0.0007) [2023-12-27 00:51:49,989][105692] Updated weights for policy 0, policy_version 1301086 (0.0008) [2023-12-27 00:51:49,991][105620] Updated weights for policy 1, policy_version 1302725 (0.0007) [2023-12-27 00:51:50,044][105620] Updated weights for policy 1, policy_version 1302735 (0.0006) [2023-12-27 00:51:50,053][105692] Updated weights for policy 0, policy_version 1301096 (0.0009) [2023-12-27 00:51:50,109][105620] Updated weights for policy 1, policy_version 1302745 (0.0007) [2023-12-27 00:51:50,816][105692] Updated weights for policy 0, policy_version 1301106 (0.0007) [2023-12-27 00:51:50,874][105692] Updated weights for policy 0, policy_version 1301116 (0.0009) [2023-12-27 00:51:50,883][105620] Updated weights for policy 1, policy_version 1302755 (0.0009) [2023-12-27 00:51:50,935][105692] Updated weights for policy 0, policy_version 1301126 (0.0008) [2023-12-27 00:51:50,946][105620] Updated weights for policy 1, policy_version 1302765 (0.0008) [2023-12-27 00:51:50,995][105692] Updated weights for policy 0, policy_version 1301136 (0.0008) [2023-12-27 00:51:51,007][105620] Updated weights for policy 1, policy_version 1302775 (0.0009) [2023-12-27 00:51:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 666689536. Throughput: 0: 9827.7, 1: 9848.5. Samples: 666681524. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:51:51,062][104569] Avg episode reward: [(0, '8365.215'), (1, '9172.110')] [2023-12-27 00:51:51,749][105692] Updated weights for policy 0, policy_version 1301146 (0.0009) [2023-12-27 00:51:51,795][105620] Updated weights for policy 1, policy_version 1302785 (0.0008) [2023-12-27 00:51:51,816][105692] Updated weights for policy 0, policy_version 1301156 (0.0010) [2023-12-27 00:51:51,860][105620] Updated weights for policy 1, policy_version 1302795 (0.0006) [2023-12-27 00:51:51,874][105692] Updated weights for policy 0, policy_version 1301166 (0.0008) [2023-12-27 00:51:51,921][105620] Updated weights for policy 1, policy_version 1302805 (0.0008) [2023-12-27 00:51:51,982][105620] Updated weights for policy 1, policy_version 1302815 (0.0009) [2023-12-27 00:51:52,647][105692] Updated weights for policy 0, policy_version 1301176 (0.0008) [2023-12-27 00:51:52,686][105620] Updated weights for policy 1, policy_version 1302825 (0.0007) [2023-12-27 00:51:52,699][105692] Updated weights for policy 0, policy_version 1301186 (0.0008) [2023-12-27 00:51:52,742][105620] Updated weights for policy 1, policy_version 1302835 (0.0006) [2023-12-27 00:51:52,749][105692] Updated weights for policy 0, policy_version 1301196 (0.0009) [2023-12-27 00:51:52,796][105620] Updated weights for policy 1, policy_version 1302845 (0.0006) [2023-12-27 00:51:53,483][105692] Updated weights for policy 0, policy_version 1301206 (0.0007) [2023-12-27 00:51:53,548][105692] Updated weights for policy 0, policy_version 1301216 (0.0009) [2023-12-27 00:51:53,581][105620] Updated weights for policy 1, policy_version 1302855 (0.0008) [2023-12-27 00:51:53,602][105692] Updated weights for policy 0, policy_version 1301226 (0.0008) [2023-12-27 00:51:53,636][105620] Updated weights for policy 1, policy_version 1302865 (0.0009) [2023-12-27 00:51:53,690][105620] Updated weights for policy 1, policy_version 1302875 (0.0010) [2023-12-27 00:51:54,267][105692] Updated weights for policy 0, policy_version 1301236 (0.0007) [2023-12-27 00:51:54,330][105692] Updated weights for policy 0, policy_version 1301246 (0.0010) [2023-12-27 00:51:54,378][105692] Updated weights for policy 0, policy_version 1301256 (0.0010) [2023-12-27 00:51:54,486][105620] Updated weights for policy 1, policy_version 1302885 (0.0010) [2023-12-27 00:51:54,530][105620] Updated weights for policy 1, policy_version 1302895 (0.0008) [2023-12-27 00:51:54,585][105620] Updated weights for policy 1, policy_version 1302905 (0.0008) [2023-12-27 00:51:55,089][105692] Updated weights for policy 0, policy_version 1301266 (0.0010) [2023-12-27 00:51:55,145][105692] Updated weights for policy 0, policy_version 1301276 (0.0010) [2023-12-27 00:51:55,206][105692] Updated weights for policy 0, policy_version 1301286 (0.0007) [2023-12-27 00:51:55,258][105692] Updated weights for policy 0, policy_version 1301296 (0.0005) [2023-12-27 00:51:55,373][105620] Updated weights for policy 1, policy_version 1302915 (0.0009) [2023-12-27 00:51:55,434][105620] Updated weights for policy 1, policy_version 1302925 (0.0010) [2023-12-27 00:51:55,492][105620] Updated weights for policy 1, policy_version 1302935 (0.0009) [2023-12-27 00:51:55,884][105692] Updated weights for policy 0, policy_version 1301306 (0.0010) [2023-12-27 00:51:55,935][105692] Updated weights for policy 0, policy_version 1301316 (0.0005) [2023-12-27 00:51:55,988][105692] Updated weights for policy 0, policy_version 1301326 (0.0010) [2023-12-27 00:51:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 666787840. Throughput: 0: 9920.9, 1: 9726.1. Samples: 666793276. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:51:56,063][104569] Avg episode reward: [(0, '8453.780'), (1, '9082.103')] [2023-12-27 00:51:56,285][105620] Updated weights for policy 1, policy_version 1302945 (0.0009) [2023-12-27 00:51:56,353][105620] Updated weights for policy 1, policy_version 1302955 (0.0008) [2023-12-27 00:51:56,407][105620] Updated weights for policy 1, policy_version 1302965 (0.0007) [2023-12-27 00:51:56,465][105620] Updated weights for policy 1, policy_version 1302975 (0.0008) [2023-12-27 00:51:56,714][105692] Updated weights for policy 0, policy_version 1301336 (0.0010) [2023-12-27 00:51:56,775][105692] Updated weights for policy 0, policy_version 1301346 (0.0011) [2023-12-27 00:51:56,830][105692] Updated weights for policy 0, policy_version 1301356 (0.0011) [2023-12-27 00:51:57,202][105620] Updated weights for policy 1, policy_version 1302985 (0.0008) [2023-12-27 00:51:57,252][105620] Updated weights for policy 1, policy_version 1302995 (0.0008) [2023-12-27 00:51:57,306][105620] Updated weights for policy 1, policy_version 1303005 (0.0008) [2023-12-27 00:51:57,575][105692] Updated weights for policy 0, policy_version 1301366 (0.0010) [2023-12-27 00:51:57,638][105692] Updated weights for policy 0, policy_version 1301376 (0.0010) [2023-12-27 00:51:57,699][105692] Updated weights for policy 0, policy_version 1301386 (0.0010) [2023-12-27 00:51:58,105][105620] Updated weights for policy 1, policy_version 1303015 (0.0009) [2023-12-27 00:51:58,164][105620] Updated weights for policy 1, policy_version 1303026 (0.0009) [2023-12-27 00:51:58,227][105620] Updated weights for policy 1, policy_version 1303036 (0.0009) [2023-12-27 00:51:58,345][105692] Updated weights for policy 0, policy_version 1301397 (0.0008) [2023-12-27 00:51:58,414][105692] Updated weights for policy 0, policy_version 1301408 (0.0008) [2023-12-27 00:51:58,479][105692] Updated weights for policy 0, policy_version 1301418 (0.0009) [2023-12-27 00:51:58,999][105620] Updated weights for policy 1, policy_version 1303046 (0.0010) [2023-12-27 00:51:59,045][105620] Updated weights for policy 1, policy_version 1303056 (0.0010) [2023-12-27 00:51:59,093][105620] Updated weights for policy 1, policy_version 1303066 (0.0010) [2023-12-27 00:51:59,307][105692] Updated weights for policy 0, policy_version 1301428 (0.0009) [2023-12-27 00:51:59,371][105692] Updated weights for policy 0, policy_version 1301438 (0.0008) [2023-12-27 00:51:59,427][105692] Updated weights for policy 0, policy_version 1301448 (0.0008) [2023-12-27 00:51:59,879][105620] Updated weights for policy 1, policy_version 1303076 (0.0011) [2023-12-27 00:51:59,942][105620] Updated weights for policy 1, policy_version 1303086 (0.0011) [2023-12-27 00:52:00,000][105620] Updated weights for policy 1, policy_version 1303096 (0.0011) [2023-12-27 00:52:00,199][105692] Updated weights for policy 0, policy_version 1301458 (0.0008) [2023-12-27 00:52:00,249][105692] Updated weights for policy 0, policy_version 1301468 (0.0009) [2023-12-27 00:52:00,307][105692] Updated weights for policy 0, policy_version 1301478 (0.0009) [2023-12-27 00:52:00,361][105692] Updated weights for policy 0, policy_version 1301488 (0.0009) [2023-12-27 00:52:00,667][105620] Updated weights for policy 1, policy_version 1303106 (0.0009) [2023-12-27 00:52:00,720][105620] Updated weights for policy 1, policy_version 1303116 (0.0005) [2023-12-27 00:52:00,775][105620] Updated weights for policy 1, policy_version 1303126 (0.0005) [2023-12-27 00:52:00,830][105620] Updated weights for policy 1, policy_version 1303136 (0.0005) [2023-12-27 00:52:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19438.7). Total num frames: 666877952. Throughput: 0: 9937.6, 1: 9706.8. Samples: 666849888. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:52:01,062][104569] Avg episode reward: [(0, '8817.015'), (1, '9081.816')] [2023-12-27 00:52:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001301488_333234176.pth... [2023-12-27 00:52:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001303136_333643776.pth... [2023-12-27 00:52:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001300336_332939264.pth [2023-12-27 00:52:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001301984_333348864.pth [2023-12-27 00:52:01,204][105692] Updated weights for policy 0, policy_version 1301498 (0.0006) [2023-12-27 00:52:01,270][105692] Updated weights for policy 0, policy_version 1301508 (0.0006) [2023-12-27 00:52:01,328][105692] Updated weights for policy 0, policy_version 1301518 (0.0008) [2023-12-27 00:52:01,504][105620] Updated weights for policy 1, policy_version 1303146 (0.0010) [2023-12-27 00:52:01,555][105620] Updated weights for policy 1, policy_version 1303156 (0.0010) [2023-12-27 00:52:01,616][105620] Updated weights for policy 1, policy_version 1303166 (0.0010) [2023-12-27 00:52:02,018][105692] Updated weights for policy 0, policy_version 1301528 (0.0006) [2023-12-27 00:52:02,089][105692] Updated weights for policy 0, policy_version 1301538 (0.0006) [2023-12-27 00:52:02,144][105692] Updated weights for policy 0, policy_version 1301548 (0.0010) [2023-12-27 00:52:02,337][105620] Updated weights for policy 1, policy_version 1303176 (0.0011) [2023-12-27 00:52:02,399][105620] Updated weights for policy 1, policy_version 1303186 (0.0008) [2023-12-27 00:52:02,457][105620] Updated weights for policy 1, policy_version 1303196 (0.0010) [2023-12-27 00:52:02,737][105692] Updated weights for policy 0, policy_version 1301558 (0.0008) [2023-12-27 00:52:02,788][105692] Updated weights for policy 0, policy_version 1301568 (0.0010) [2023-12-27 00:52:02,847][105692] Updated weights for policy 0, policy_version 1301578 (0.0011) [2023-12-27 00:52:03,145][105620] Updated weights for policy 1, policy_version 1303206 (0.0008) [2023-12-27 00:52:03,215][105620] Updated weights for policy 1, policy_version 1303216 (0.0005) [2023-12-27 00:52:03,270][105620] Updated weights for policy 1, policy_version 1303226 (0.0007) [2023-12-27 00:52:03,458][105692] Updated weights for policy 0, policy_version 1301588 (0.0010) [2023-12-27 00:52:03,532][105692] Updated weights for policy 0, policy_version 1301598 (0.0007) [2023-12-27 00:52:03,575][105585] KL-divergence is very high: 145.5917 [2023-12-27 00:52:03,588][105692] Updated weights for policy 0, policy_version 1301608 (0.0005) [2023-12-27 00:52:03,613][105585] KL-divergence is very high: 268.1579 [2023-12-27 00:52:03,925][105620] Updated weights for policy 1, policy_version 1303236 (0.0008) [2023-12-27 00:52:03,991][105620] Updated weights for policy 1, policy_version 1303246 (0.0006) [2023-12-27 00:52:04,041][105620] Updated weights for policy 1, policy_version 1303256 (0.0008) [2023-12-27 00:52:04,170][105692] Updated weights for policy 0, policy_version 1301618 (0.0005) [2023-12-27 00:52:04,170][105585] KL-divergence is very high: 171.6975 [2023-12-27 00:52:04,220][105585] KL-divergence is very high: 202.1433 [2023-12-27 00:52:04,231][105692] Updated weights for policy 0, policy_version 1301628 (0.0006) [2023-12-27 00:52:04,267][105585] KL-divergence is very high: 198.7424 [2023-12-27 00:52:04,292][105692] Updated weights for policy 0, policy_version 1301638 (0.0009) [2023-12-27 00:52:04,316][105585] KL-divergence is very high: 153.4016 [2023-12-27 00:52:04,353][105692] Updated weights for policy 0, policy_version 1301648 (0.0008) [2023-12-27 00:52:04,781][105620] Updated weights for policy 1, policy_version 1303266 (0.0008) [2023-12-27 00:52:04,843][105620] Updated weights for policy 1, policy_version 1303276 (0.0009) [2023-12-27 00:52:04,892][105692] Updated weights for policy 0, policy_version 1301658 (0.0005) [2023-12-27 00:52:04,897][105620] Updated weights for policy 1, policy_version 1303286 (0.0009) [2023-12-27 00:52:04,948][105692] Updated weights for policy 0, policy_version 1301668 (0.0005) [2023-12-27 00:52:04,951][105620] Updated weights for policy 1, policy_version 1303296 (0.0009) [2023-12-27 00:52:05,002][105692] Updated weights for policy 0, policy_version 1301678 (0.0005) [2023-12-27 00:52:05,648][105692] Updated weights for policy 0, policy_version 1301688 (0.0010) [2023-12-27 00:52:05,696][105620] Updated weights for policy 1, policy_version 1303306 (0.0005) [2023-12-27 00:52:05,704][105692] Updated weights for policy 0, policy_version 1301698 (0.0010) [2023-12-27 00:52:05,757][105620] Updated weights for policy 1, policy_version 1303316 (0.0006) [2023-12-27 00:52:05,762][105692] Updated weights for policy 0, policy_version 1301708 (0.0010) [2023-12-27 00:52:05,817][105620] Updated weights for policy 1, policy_version 1303326 (0.0007) [2023-12-27 00:52:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 666984448. Throughput: 0: 9981.2, 1: 9693.4. Samples: 666969216. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:52:06,063][104569] Avg episode reward: [(0, '8720.884'), (1, '9353.447')] [2023-12-27 00:52:06,503][105692] Updated weights for policy 0, policy_version 1301718 (0.0010) [2023-12-27 00:52:06,561][105620] Updated weights for policy 1, policy_version 1303336 (0.0006) [2023-12-27 00:52:06,562][105692] Updated weights for policy 0, policy_version 1301728 (0.0010) [2023-12-27 00:52:06,617][105620] Updated weights for policy 1, policy_version 1303346 (0.0008) [2023-12-27 00:52:06,622][105692] Updated weights for policy 0, policy_version 1301738 (0.0011) [2023-12-27 00:52:06,678][105620] Updated weights for policy 1, policy_version 1303356 (0.0009) [2023-12-27 00:52:07,362][105692] Updated weights for policy 0, policy_version 1301748 (0.0008) [2023-12-27 00:52:07,419][105692] Updated weights for policy 0, policy_version 1301758 (0.0006) [2023-12-27 00:52:07,474][105620] Updated weights for policy 1, policy_version 1303366 (0.0009) [2023-12-27 00:52:07,478][105692] Updated weights for policy 0, policy_version 1301768 (0.0011) [2023-12-27 00:52:07,530][105620] Updated weights for policy 1, policy_version 1303376 (0.0009) [2023-12-27 00:52:07,584][105620] Updated weights for policy 1, policy_version 1303386 (0.0010) [2023-12-27 00:52:08,007][105692] Updated weights for policy 0, policy_version 1301778 (0.0006) [2023-12-27 00:52:08,059][105692] Updated weights for policy 0, policy_version 1301788 (0.0005) [2023-12-27 00:52:08,112][105692] Updated weights for policy 0, policy_version 1301798 (0.0006) [2023-12-27 00:52:08,159][105692] Updated weights for policy 0, policy_version 1301808 (0.0005) [2023-12-27 00:52:08,442][105620] Updated weights for policy 1, policy_version 1303396 (0.0009) [2023-12-27 00:52:08,507][105620] Updated weights for policy 1, policy_version 1303406 (0.0008) [2023-12-27 00:52:08,565][105620] Updated weights for policy 1, policy_version 1303416 (0.0008) [2023-12-27 00:52:08,731][105692] Updated weights for policy 0, policy_version 1301818 (0.0010) [2023-12-27 00:52:08,784][105692] Updated weights for policy 0, policy_version 1301828 (0.0010) [2023-12-27 00:52:08,838][105692] Updated weights for policy 0, policy_version 1301838 (0.0008) [2023-12-27 00:52:09,293][105620] Updated weights for policy 1, policy_version 1303426 (0.0008) [2023-12-27 00:52:09,364][105620] Updated weights for policy 1, policy_version 1303436 (0.0008) [2023-12-27 00:52:09,441][105620] Updated weights for policy 1, policy_version 1303446 (0.0009) [2023-12-27 00:52:09,504][105620] Updated weights for policy 1, policy_version 1303456 (0.0007) [2023-12-27 00:52:09,635][105692] Updated weights for policy 0, policy_version 1301848 (0.0009) [2023-12-27 00:52:09,705][105692] Updated weights for policy 0, policy_version 1301858 (0.0008) [2023-12-27 00:52:09,773][105692] Updated weights for policy 0, policy_version 1301868 (0.0006) [2023-12-27 00:52:10,234][105620] Updated weights for policy 1, policy_version 1303466 (0.0008) [2023-12-27 00:52:10,300][105620] Updated weights for policy 1, policy_version 1303476 (0.0008) [2023-12-27 00:52:10,369][105620] Updated weights for policy 1, policy_version 1303486 (0.0007) [2023-12-27 00:52:10,390][105692] Updated weights for policy 0, policy_version 1301878 (0.0006) [2023-12-27 00:52:10,451][105692] Updated weights for policy 0, policy_version 1301888 (0.0007) [2023-12-27 00:52:10,520][105692] Updated weights for policy 0, policy_version 1301898 (0.0010) [2023-12-27 00:52:11,003][105620] Updated weights for policy 1, policy_version 1303496 (0.0010) [2023-12-27 00:52:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 667074560. Throughput: 0: 10134.1, 1: 9633.2. Samples: 667087656. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:52:11,063][104569] Avg episode reward: [(0, '8362.151'), (1, '9353.173')] [2023-12-27 00:52:11,067][105620] Updated weights for policy 1, policy_version 1303506 (0.0009) [2023-12-27 00:52:11,136][105620] Updated weights for policy 1, policy_version 1303516 (0.0009) [2023-12-27 00:52:11,299][105692] Updated weights for policy 0, policy_version 1301908 (0.0009) [2023-12-27 00:52:11,373][105692] Updated weights for policy 0, policy_version 1301918 (0.0009) [2023-12-27 00:52:11,438][105692] Updated weights for policy 0, policy_version 1301928 (0.0009) [2023-12-27 00:52:11,930][105620] Updated weights for policy 1, policy_version 1303526 (0.0008) [2023-12-27 00:52:11,986][105620] Updated weights for policy 1, policy_version 1303536 (0.0008) [2023-12-27 00:52:12,043][105620] Updated weights for policy 1, policy_version 1303546 (0.0008) [2023-12-27 00:52:12,217][105692] Updated weights for policy 0, policy_version 1301938 (0.0009) [2023-12-27 00:52:12,284][105692] Updated weights for policy 0, policy_version 1301948 (0.0011) [2023-12-27 00:52:12,337][105692] Updated weights for policy 0, policy_version 1301958 (0.0011) [2023-12-27 00:52:12,403][105692] Updated weights for policy 0, policy_version 1301968 (0.0013) [2023-12-27 00:52:12,805][105620] Updated weights for policy 1, policy_version 1303556 (0.0008) [2023-12-27 00:52:12,862][105620] Updated weights for policy 1, policy_version 1303566 (0.0009) [2023-12-27 00:52:12,918][105620] Updated weights for policy 1, policy_version 1303576 (0.0008) [2023-12-27 00:52:13,093][105692] Updated weights for policy 0, policy_version 1301978 (0.0011) [2023-12-27 00:52:13,158][105692] Updated weights for policy 0, policy_version 1301988 (0.0011) [2023-12-27 00:52:13,223][105692] Updated weights for policy 0, policy_version 1301998 (0.0010) [2023-12-27 00:52:13,523][105620] Updated weights for policy 1, policy_version 1303586 (0.0010) [2023-12-27 00:52:13,590][105620] Updated weights for policy 1, policy_version 1303596 (0.0009) [2023-12-27 00:52:13,653][105620] Updated weights for policy 1, policy_version 1303606 (0.0006) [2023-12-27 00:52:13,715][105620] Updated weights for policy 1, policy_version 1303616 (0.0010) [2023-12-27 00:52:13,899][105692] Updated weights for policy 0, policy_version 1302008 (0.0010) [2023-12-27 00:52:13,960][105692] Updated weights for policy 0, policy_version 1302018 (0.0010) [2023-12-27 00:52:14,019][105692] Updated weights for policy 0, policy_version 1302028 (0.0010) [2023-12-27 00:52:14,356][105620] Updated weights for policy 1, policy_version 1303626 (0.0009) [2023-12-27 00:52:14,423][105620] Updated weights for policy 1, policy_version 1303636 (0.0008) [2023-12-27 00:52:14,478][105620] Updated weights for policy 1, policy_version 1303646 (0.0010) [2023-12-27 00:52:14,672][105692] Updated weights for policy 0, policy_version 1302038 (0.0007) [2023-12-27 00:52:14,727][105692] Updated weights for policy 0, policy_version 1302048 (0.0005) [2023-12-27 00:52:14,789][105692] Updated weights for policy 0, policy_version 1302058 (0.0007) [2023-12-27 00:52:15,182][105620] Updated weights for policy 1, policy_version 1303656 (0.0011) [2023-12-27 00:52:15,246][105620] Updated weights for policy 1, policy_version 1303666 (0.0011) [2023-12-27 00:52:15,313][105620] Updated weights for policy 1, policy_version 1303676 (0.0011) [2023-12-27 00:52:15,443][105692] Updated weights for policy 0, policy_version 1302068 (0.0007) [2023-12-27 00:52:15,512][105692] Updated weights for policy 0, policy_version 1302078 (0.0008) [2023-12-27 00:52:15,575][105692] Updated weights for policy 0, policy_version 1302088 (0.0009) [2023-12-27 00:52:15,949][105620] Updated weights for policy 1, policy_version 1303686 (0.0007) [2023-12-27 00:52:16,008][105620] Updated weights for policy 1, policy_version 1303696 (0.0005) [2023-12-27 00:52:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 667172864. Throughput: 0: 10021.8, 1: 9706.6. Samples: 667144924. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 00:52:16,063][104569] Avg episode reward: [(0, '8089.369'), (1, '9171.216')] [2023-12-27 00:52:16,066][105620] Updated weights for policy 1, policy_version 1303706 (0.0006) [2023-12-27 00:52:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001302096_333389824.pth... [2023-12-27 00:52:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001300944_333094912.pth [2023-12-27 00:52:16,099][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001303712_333791232.pth... [2023-12-27 00:52:16,102][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001302592_333504512.pth [2023-12-27 00:52:16,119][105692] Updated weights for policy 0, policy_version 1302098 (0.0007) [2023-12-27 00:52:16,189][105692] Updated weights for policy 0, policy_version 1302108 (0.0009) [2023-12-27 00:52:16,248][105585] KL-divergence is very high: 127.9230 [2023-12-27 00:52:16,264][105692] Updated weights for policy 0, policy_version 1302118 (0.0010) [2023-12-27 00:52:16,300][105585] KL-divergence is very high: 167.8936 [2023-12-27 00:52:16,319][105692] Updated weights for policy 0, policy_version 1302128 (0.0010) [2023-12-27 00:52:16,660][105620] Updated weights for policy 1, policy_version 1303716 (0.0007) [2023-12-27 00:52:16,717][105620] Updated weights for policy 1, policy_version 1303726 (0.0008) [2023-12-27 00:52:16,769][105620] Updated weights for policy 1, policy_version 1303736 (0.0008) [2023-12-27 00:52:17,018][105692] Updated weights for policy 0, policy_version 1302138 (0.0011) [2023-12-27 00:52:17,066][105692] Updated weights for policy 0, policy_version 1302148 (0.0010) [2023-12-27 00:52:17,117][105692] Updated weights for policy 0, policy_version 1302158 (0.0010) [2023-12-27 00:52:17,472][105620] Updated weights for policy 1, policy_version 1303746 (0.0008) [2023-12-27 00:52:17,516][105620] Updated weights for policy 1, policy_version 1303756 (0.0010) [2023-12-27 00:52:17,568][105620] Updated weights for policy 1, policy_version 1303766 (0.0010) [2023-12-27 00:52:17,618][105620] Updated weights for policy 1, policy_version 1303776 (0.0008) [2023-12-27 00:52:17,838][105692] Updated weights for policy 0, policy_version 1302168 (0.0009) [2023-12-27 00:52:17,887][105692] Updated weights for policy 0, policy_version 1302178 (0.0009) [2023-12-27 00:52:17,935][105692] Updated weights for policy 0, policy_version 1302188 (0.0009) [2023-12-27 00:52:18,388][105620] Updated weights for policy 1, policy_version 1303786 (0.0008) [2023-12-27 00:52:18,447][105620] Updated weights for policy 1, policy_version 1303796 (0.0008) [2023-12-27 00:52:18,497][105620] Updated weights for policy 1, policy_version 1303806 (0.0008) [2023-12-27 00:52:18,654][105692] Updated weights for policy 0, policy_version 1302198 (0.0007) [2023-12-27 00:52:18,712][105692] Updated weights for policy 0, policy_version 1302208 (0.0007) [2023-12-27 00:52:18,764][105692] Updated weights for policy 0, policy_version 1302218 (0.0009) [2023-12-27 00:52:19,317][105620] Updated weights for policy 1, policy_version 1303816 (0.0009) [2023-12-27 00:52:19,388][105620] Updated weights for policy 1, policy_version 1303826 (0.0011) [2023-12-27 00:52:19,449][105692] Updated weights for policy 0, policy_version 1302228 (0.0008) [2023-12-27 00:52:19,453][105620] Updated weights for policy 1, policy_version 1303836 (0.0010) [2023-12-27 00:52:19,514][105692] Updated weights for policy 0, policy_version 1302238 (0.0008) [2023-12-27 00:52:19,561][105692] Updated weights for policy 0, policy_version 1302248 (0.0006) [2023-12-27 00:52:20,224][105620] Updated weights for policy 1, policy_version 1303846 (0.0011) [2023-12-27 00:52:20,290][105620] Updated weights for policy 1, policy_version 1303856 (0.0011) [2023-12-27 00:52:20,352][105692] Updated weights for policy 0, policy_version 1302258 (0.0009) [2023-12-27 00:52:20,357][105620] Updated weights for policy 1, policy_version 1303866 (0.0010) [2023-12-27 00:52:20,406][105692] Updated weights for policy 0, policy_version 1302268 (0.0007) [2023-12-27 00:52:20,466][105692] Updated weights for policy 0, policy_version 1302278 (0.0008) [2023-12-27 00:52:20,531][105692] Updated weights for policy 0, policy_version 1302288 (0.0008) [2023-12-27 00:52:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 667271168. Throughput: 0: 9979.9, 1: 9617.8. Samples: 667265052. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:52:21,062][104569] Avg episode reward: [(0, '8544.263'), (1, '9080.456')] [2023-12-27 00:52:21,108][105620] Updated weights for policy 1, policy_version 1303876 (0.0010) [2023-12-27 00:52:21,176][105620] Updated weights for policy 1, policy_version 1303886 (0.0011) [2023-12-27 00:52:21,244][105620] Updated weights for policy 1, policy_version 1303896 (0.0011) [2023-12-27 00:52:21,368][105692] Updated weights for policy 0, policy_version 1302298 (0.0008) [2023-12-27 00:52:21,434][105692] Updated weights for policy 0, policy_version 1302308 (0.0006) [2023-12-27 00:52:21,498][105692] Updated weights for policy 0, policy_version 1302318 (0.0006) [2023-12-27 00:52:22,000][105620] Updated weights for policy 1, policy_version 1303906 (0.0009) [2023-12-27 00:52:22,066][105620] Updated weights for policy 1, policy_version 1303917 (0.0010) [2023-12-27 00:52:22,115][105692] Updated weights for policy 0, policy_version 1302328 (0.0009) [2023-12-27 00:52:22,126][105620] Updated weights for policy 1, policy_version 1303927 (0.0008) [2023-12-27 00:52:22,170][105692] Updated weights for policy 0, policy_version 1302338 (0.0006) [2023-12-27 00:52:22,217][105692] Updated weights for policy 0, policy_version 1302348 (0.0009) [2023-12-27 00:52:22,878][105620] Updated weights for policy 1, policy_version 1303937 (0.0007) [2023-12-27 00:52:22,932][105620] Updated weights for policy 1, policy_version 1303947 (0.0007) [2023-12-27 00:52:22,988][105620] Updated weights for policy 1, policy_version 1303957 (0.0010) [2023-12-27 00:52:23,032][105692] Updated weights for policy 0, policy_version 1302358 (0.0009) [2023-12-27 00:52:23,051][105620] Updated weights for policy 1, policy_version 1303967 (0.0008) [2023-12-27 00:52:23,098][105692] Updated weights for policy 0, policy_version 1302368 (0.0008) [2023-12-27 00:52:23,154][105692] Updated weights for policy 0, policy_version 1302378 (0.0009) [2023-12-27 00:52:23,795][105620] Updated weights for policy 1, policy_version 1303977 (0.0008) [2023-12-27 00:52:23,866][105620] Updated weights for policy 1, policy_version 1303987 (0.0008) [2023-12-27 00:52:23,885][105692] Updated weights for policy 0, policy_version 1302388 (0.0008) [2023-12-27 00:52:23,932][105620] Updated weights for policy 1, policy_version 1303997 (0.0009) [2023-12-27 00:52:23,933][105692] Updated weights for policy 0, policy_version 1302398 (0.0006) [2023-12-27 00:52:23,990][105692] Updated weights for policy 0, policy_version 1302408 (0.0005) [2023-12-27 00:52:24,610][105692] Updated weights for policy 0, policy_version 1302418 (0.0006) [2023-12-27 00:52:24,662][105692] Updated weights for policy 0, policy_version 1302428 (0.0009) [2023-12-27 00:52:24,710][105692] Updated weights for policy 0, policy_version 1302438 (0.0009) [2023-12-27 00:52:24,749][105620] Updated weights for policy 1, policy_version 1304007 (0.0009) [2023-12-27 00:52:24,767][105692] Updated weights for policy 0, policy_version 1302448 (0.0007) [2023-12-27 00:52:24,801][105620] Updated weights for policy 1, policy_version 1304017 (0.0007) [2023-12-27 00:52:24,860][105620] Updated weights for policy 1, policy_version 1304027 (0.0009) [2023-12-27 00:52:25,427][105692] Updated weights for policy 0, policy_version 1302458 (0.0005) [2023-12-27 00:52:25,483][105692] Updated weights for policy 0, policy_version 1302468 (0.0005) [2023-12-27 00:52:25,547][105692] Updated weights for policy 0, policy_version 1302478 (0.0005) [2023-12-27 00:52:25,733][105620] Updated weights for policy 1, policy_version 1304037 (0.0009) [2023-12-27 00:52:25,788][105620] Updated weights for policy 1, policy_version 1304047 (0.0009) [2023-12-27 00:52:25,839][105620] Updated weights for policy 1, policy_version 1304057 (0.0009) [2023-12-27 00:52:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 667369472. Throughput: 0: 9934.7, 1: 9467.2. Samples: 667377264. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:52:26,063][104569] Avg episode reward: [(0, '8182.088'), (1, '9262.729')] [2023-12-27 00:52:26,117][105692] Updated weights for policy 0, policy_version 1302488 (0.0006) [2023-12-27 00:52:26,167][105692] Updated weights for policy 0, policy_version 1302498 (0.0007) [2023-12-27 00:52:26,225][105692] Updated weights for policy 0, policy_version 1302508 (0.0008) [2023-12-27 00:52:26,652][105620] Updated weights for policy 1, policy_version 1304067 (0.0009) [2023-12-27 00:52:26,717][105620] Updated weights for policy 1, policy_version 1304077 (0.0009) [2023-12-27 00:52:26,777][105620] Updated weights for policy 1, policy_version 1304087 (0.0008) [2023-12-27 00:52:26,904][105692] Updated weights for policy 0, policy_version 1302518 (0.0009) [2023-12-27 00:52:26,958][105692] Updated weights for policy 0, policy_version 1302528 (0.0008) [2023-12-27 00:52:27,004][105692] Updated weights for policy 0, policy_version 1302538 (0.0009) [2023-12-27 00:52:27,547][105620] Updated weights for policy 1, policy_version 1304097 (0.0009) [2023-12-27 00:52:27,599][105620] Updated weights for policy 1, policy_version 1304107 (0.0009) [2023-12-27 00:52:27,629][105692] Updated weights for policy 0, policy_version 1302548 (0.0008) [2023-12-27 00:52:27,647][105620] Updated weights for policy 1, policy_version 1304117 (0.0007) [2023-12-27 00:52:27,685][105692] Updated weights for policy 0, policy_version 1302558 (0.0007) [2023-12-27 00:52:27,703][105620] Updated weights for policy 1, policy_version 1304127 (0.0006) [2023-12-27 00:52:27,741][105692] Updated weights for policy 0, policy_version 1302568 (0.0007) [2023-12-27 00:52:28,385][105620] Updated weights for policy 1, policy_version 1304137 (0.0008) [2023-12-27 00:52:28,438][105692] Updated weights for policy 0, policy_version 1302578 (0.0006) [2023-12-27 00:52:28,440][105620] Updated weights for policy 1, policy_version 1304147 (0.0008) [2023-12-27 00:52:28,492][105620] Updated weights for policy 1, policy_version 1304157 (0.0005) [2023-12-27 00:52:28,494][105692] Updated weights for policy 0, policy_version 1302588 (0.0010) [2023-12-27 00:52:28,556][105692] Updated weights for policy 0, policy_version 1302598 (0.0010) [2023-12-27 00:52:28,611][105692] Updated weights for policy 0, policy_version 1302608 (0.0010) [2023-12-27 00:52:29,269][105620] Updated weights for policy 1, policy_version 1304167 (0.0007) [2023-12-27 00:52:29,326][105620] Updated weights for policy 1, policy_version 1304177 (0.0008) [2023-12-27 00:52:29,331][105692] Updated weights for policy 0, policy_version 1302618 (0.0007) [2023-12-27 00:52:29,385][105620] Updated weights for policy 1, policy_version 1304187 (0.0007) [2023-12-27 00:52:29,394][105692] Updated weights for policy 0, policy_version 1302628 (0.0008) [2023-12-27 00:52:29,452][105692] Updated weights for policy 0, policy_version 1302638 (0.0007) [2023-12-27 00:52:30,114][105692] Updated weights for policy 0, policy_version 1302648 (0.0008) [2023-12-27 00:52:30,166][105692] Updated weights for policy 0, policy_version 1302658 (0.0009) [2023-12-27 00:52:30,199][105620] Updated weights for policy 1, policy_version 1304197 (0.0007) [2023-12-27 00:52:30,214][105692] Updated weights for policy 0, policy_version 1302668 (0.0007) [2023-12-27 00:52:30,256][105620] Updated weights for policy 1, policy_version 1304207 (0.0009) [2023-12-27 00:52:30,312][105620] Updated weights for policy 1, policy_version 1304217 (0.0009) [2023-12-27 00:52:30,906][105692] Updated weights for policy 0, policy_version 1302678 (0.0008) [2023-12-27 00:52:30,959][105692] Updated weights for policy 0, policy_version 1302689 (0.0010) [2023-12-27 00:52:31,017][105692] Updated weights for policy 0, policy_version 1302699 (0.0009) [2023-12-27 00:52:31,051][105620] Updated weights for policy 1, policy_version 1304227 (0.0009) [2023-12-27 00:52:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 667467776. Throughput: 0: 9984.5, 1: 9490.7. Samples: 667437188. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:52:31,062][104569] Avg episode reward: [(0, '8625.990'), (1, '9263.090')] [2023-12-27 00:52:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001302704_333545472.pth... [2023-12-27 00:52:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001301488_333234176.pth [2023-12-27 00:52:31,117][105620] Updated weights for policy 1, policy_version 1304237 (0.0009) [2023-12-27 00:52:31,175][105620] Updated weights for policy 1, policy_version 1304247 (0.0006) [2023-12-27 00:52:31,218][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001304256_333930496.pth... [2023-12-27 00:52:31,221][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001303136_333643776.pth [2023-12-27 00:52:31,837][105692] Updated weights for policy 0, policy_version 1302709 (0.0009) [2023-12-27 00:52:31,892][105692] Updated weights for policy 0, policy_version 1302719 (0.0009) [2023-12-27 00:52:31,912][105620] Updated weights for policy 1, policy_version 1304257 (0.0008) [2023-12-27 00:52:31,947][105692] Updated weights for policy 0, policy_version 1302729 (0.0006) [2023-12-27 00:52:31,972][105620] Updated weights for policy 1, policy_version 1304267 (0.0009) [2023-12-27 00:52:32,040][105620] Updated weights for policy 1, policy_version 1304277 (0.0010) [2023-12-27 00:52:32,105][105620] Updated weights for policy 1, policy_version 1304287 (0.0009) [2023-12-27 00:52:32,578][105692] Updated weights for policy 0, policy_version 1302739 (0.0005) [2023-12-27 00:52:32,626][105692] Updated weights for policy 0, policy_version 1302749 (0.0005) [2023-12-27 00:52:32,686][105692] Updated weights for policy 0, policy_version 1302759 (0.0005) [2023-12-27 00:52:32,808][105620] Updated weights for policy 1, policy_version 1304297 (0.0006) [2023-12-27 00:52:32,866][105620] Updated weights for policy 1, policy_version 1304307 (0.0011) [2023-12-27 00:52:32,931][105620] Updated weights for policy 1, policy_version 1304317 (0.0010) [2023-12-27 00:52:33,271][105692] Updated weights for policy 0, policy_version 1302769 (0.0006) [2023-12-27 00:52:33,329][105692] Updated weights for policy 0, policy_version 1302779 (0.0005) [2023-12-27 00:52:33,379][105692] Updated weights for policy 0, policy_version 1302789 (0.0007) [2023-12-27 00:52:33,430][105692] Updated weights for policy 0, policy_version 1302799 (0.0009) [2023-12-27 00:52:33,549][105620] Updated weights for policy 1, policy_version 1304327 (0.0010) [2023-12-27 00:52:33,598][105620] Updated weights for policy 1, policy_version 1304337 (0.0010) [2023-12-27 00:52:33,646][105620] Updated weights for policy 1, policy_version 1304347 (0.0010) [2023-12-27 00:52:34,072][105692] Updated weights for policy 0, policy_version 1302809 (0.0006) [2023-12-27 00:52:34,140][105692] Updated weights for policy 0, policy_version 1302819 (0.0008) [2023-12-27 00:52:34,207][105692] Updated weights for policy 0, policy_version 1302829 (0.0010) [2023-12-27 00:52:34,389][105620] Updated weights for policy 1, policy_version 1304357 (0.0009) [2023-12-27 00:52:34,455][105620] Updated weights for policy 1, policy_version 1304367 (0.0010) [2023-12-27 00:52:34,519][105620] Updated weights for policy 1, policy_version 1304377 (0.0009) [2023-12-27 00:52:34,999][105692] Updated weights for policy 0, policy_version 1302839 (0.0010) [2023-12-27 00:52:35,061][105692] Updated weights for policy 0, policy_version 1302849 (0.0010) [2023-12-27 00:52:35,121][105692] Updated weights for policy 0, policy_version 1302859 (0.0009) [2023-12-27 00:52:35,124][105620] Updated weights for policy 1, policy_version 1304387 (0.0008) [2023-12-27 00:52:35,189][105620] Updated weights for policy 1, policy_version 1304397 (0.0005) [2023-12-27 00:52:35,255][105620] Updated weights for policy 1, policy_version 1304407 (0.0005) [2023-12-27 00:52:35,834][105692] Updated weights for policy 0, policy_version 1302869 (0.0009) [2023-12-27 00:52:35,885][105692] Updated weights for policy 0, policy_version 1302879 (0.0009) [2023-12-27 00:52:35,936][105692] Updated weights for policy 0, policy_version 1302889 (0.0009) [2023-12-27 00:52:35,939][105620] Updated weights for policy 1, policy_version 1304417 (0.0007) [2023-12-27 00:52:36,001][105620] Updated weights for policy 1, policy_version 1304427 (0.0007) [2023-12-27 00:52:36,060][105620] Updated weights for policy 1, policy_version 1304437 (0.0009) [2023-12-27 00:52:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 667566080. Throughput: 0: 10017.6, 1: 9432.6. Samples: 667556784. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:52:36,063][104569] Avg episode reward: [(0, '8808.360'), (1, '9173.852')] [2023-12-27 00:52:36,123][105620] Updated weights for policy 1, policy_version 1304447 (0.0009) [2023-12-27 00:52:36,782][105620] Updated weights for policy 1, policy_version 1304457 (0.0007) [2023-12-27 00:52:36,785][105692] Updated weights for policy 0, policy_version 1302899 (0.0006) [2023-12-27 00:52:36,837][105620] Updated weights for policy 1, policy_version 1304467 (0.0007) [2023-12-27 00:52:36,852][105692] Updated weights for policy 0, policy_version 1302909 (0.0006) [2023-12-27 00:52:36,898][105620] Updated weights for policy 1, policy_version 1304477 (0.0007) [2023-12-27 00:52:36,917][105692] Updated weights for policy 0, policy_version 1302919 (0.0010) [2023-12-27 00:52:37,590][105692] Updated weights for policy 0, policy_version 1302929 (0.0008) [2023-12-27 00:52:37,648][105692] Updated weights for policy 0, policy_version 1302939 (0.0005) [2023-12-27 00:52:37,671][105620] Updated weights for policy 1, policy_version 1304487 (0.0008) [2023-12-27 00:52:37,709][105692] Updated weights for policy 0, policy_version 1302949 (0.0005) [2023-12-27 00:52:37,724][105620] Updated weights for policy 1, policy_version 1304497 (0.0008) [2023-12-27 00:52:37,772][105692] Updated weights for policy 0, policy_version 1302959 (0.0006) [2023-12-27 00:52:37,776][105620] Updated weights for policy 1, policy_version 1304507 (0.0008) [2023-12-27 00:52:38,346][105692] Updated weights for policy 0, policy_version 1302969 (0.0006) [2023-12-27 00:52:38,413][105692] Updated weights for policy 0, policy_version 1302979 (0.0008) [2023-12-27 00:52:38,478][105692] Updated weights for policy 0, policy_version 1302989 (0.0008) [2023-12-27 00:52:38,660][105620] Updated weights for policy 1, policy_version 1304517 (0.0010) [2023-12-27 00:52:38,708][105620] Updated weights for policy 1, policy_version 1304527 (0.0009) [2023-12-27 00:52:38,762][105620] Updated weights for policy 1, policy_version 1304537 (0.0009) [2023-12-27 00:52:39,156][105692] Updated weights for policy 0, policy_version 1302999 (0.0008) [2023-12-27 00:52:39,223][105692] Updated weights for policy 0, policy_version 1303009 (0.0008) [2023-12-27 00:52:39,282][105692] Updated weights for policy 0, policy_version 1303019 (0.0009) [2023-12-27 00:52:39,559][105620] Updated weights for policy 1, policy_version 1304547 (0.0009) [2023-12-27 00:52:39,618][105620] Updated weights for policy 1, policy_version 1304557 (0.0009) [2023-12-27 00:52:39,666][105620] Updated weights for policy 1, policy_version 1304567 (0.0009) [2023-12-27 00:52:40,042][105692] Updated weights for policy 0, policy_version 1303029 (0.0009) [2023-12-27 00:52:40,107][105692] Updated weights for policy 0, policy_version 1303039 (0.0009) [2023-12-27 00:52:40,174][105692] Updated weights for policy 0, policy_version 1303049 (0.0009) [2023-12-27 00:52:40,463][105620] Updated weights for policy 1, policy_version 1304577 (0.0009) [2023-12-27 00:52:40,527][105620] Updated weights for policy 1, policy_version 1304587 (0.0009) [2023-12-27 00:52:40,578][105620] Updated weights for policy 1, policy_version 1304597 (0.0009) [2023-12-27 00:52:40,633][105620] Updated weights for policy 1, policy_version 1304607 (0.0009) [2023-12-27 00:52:40,899][105692] Updated weights for policy 0, policy_version 1303059 (0.0009) [2023-12-27 00:52:40,951][105692] Updated weights for policy 0, policy_version 1303069 (0.0008) [2023-12-27 00:52:40,998][105692] Updated weights for policy 0, policy_version 1303079 (0.0005) [2023-12-27 00:52:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 667664384. Throughput: 0: 10015.8, 1: 9463.0. Samples: 667669824. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:52:41,062][104569] Avg episode reward: [(0, '8353.973'), (1, '9082.614')] [2023-12-27 00:52:41,462][105620] Updated weights for policy 1, policy_version 1304617 (0.0008) [2023-12-27 00:52:41,527][105620] Updated weights for policy 1, policy_version 1304627 (0.0006) [2023-12-27 00:52:41,596][105620] Updated weights for policy 1, policy_version 1304637 (0.0007) [2023-12-27 00:52:41,713][105692] Updated weights for policy 0, policy_version 1303089 (0.0008) [2023-12-27 00:52:41,781][105692] Updated weights for policy 0, policy_version 1303099 (0.0008) [2023-12-27 00:52:41,848][105692] Updated weights for policy 0, policy_version 1303109 (0.0008) [2023-12-27 00:52:41,907][105692] Updated weights for policy 0, policy_version 1303119 (0.0008) [2023-12-27 00:52:42,193][105620] Updated weights for policy 1, policy_version 1304647 (0.0008) [2023-12-27 00:52:42,249][105620] Updated weights for policy 1, policy_version 1304657 (0.0009) [2023-12-27 00:52:42,310][105620] Updated weights for policy 1, policy_version 1304667 (0.0008) [2023-12-27 00:52:42,691][105692] Updated weights for policy 0, policy_version 1303129 (0.0010) [2023-12-27 00:52:42,739][105692] Updated weights for policy 0, policy_version 1303139 (0.0008) [2023-12-27 00:52:42,786][105692] Updated weights for policy 0, policy_version 1303149 (0.0009) [2023-12-27 00:52:43,013][105620] Updated weights for policy 1, policy_version 1304677 (0.0009) [2023-12-27 00:52:43,067][105620] Updated weights for policy 1, policy_version 1304687 (0.0009) [2023-12-27 00:52:43,120][105620] Updated weights for policy 1, policy_version 1304697 (0.0008) [2023-12-27 00:52:43,537][105692] Updated weights for policy 0, policy_version 1303159 (0.0008) [2023-12-27 00:52:43,588][105692] Updated weights for policy 0, policy_version 1303169 (0.0008) [2023-12-27 00:52:43,642][105692] Updated weights for policy 0, policy_version 1303179 (0.0009) [2023-12-27 00:52:43,860][105620] Updated weights for policy 1, policy_version 1304707 (0.0009) [2023-12-27 00:52:43,916][105620] Updated weights for policy 1, policy_version 1304717 (0.0012) [2023-12-27 00:52:43,974][105620] Updated weights for policy 1, policy_version 1304728 (0.0009) [2023-12-27 00:52:44,257][105692] Updated weights for policy 0, policy_version 1303189 (0.0007) [2023-12-27 00:52:44,309][105692] Updated weights for policy 0, policy_version 1303199 (0.0006) [2023-12-27 00:52:44,361][105692] Updated weights for policy 0, policy_version 1303209 (0.0007) [2023-12-27 00:52:44,771][105620] Updated weights for policy 1, policy_version 1304738 (0.0009) [2023-12-27 00:52:44,827][105620] Updated weights for policy 1, policy_version 1304748 (0.0008) [2023-12-27 00:52:44,883][105620] Updated weights for policy 1, policy_version 1304758 (0.0010) [2023-12-27 00:52:44,940][105620] Updated weights for policy 1, policy_version 1304768 (0.0011) [2023-12-27 00:52:45,014][105692] Updated weights for policy 0, policy_version 1303219 (0.0007) [2023-12-27 00:52:45,085][105692] Updated weights for policy 0, policy_version 1303229 (0.0005) [2023-12-27 00:52:45,154][105692] Updated weights for policy 0, policy_version 1303239 (0.0005) [2023-12-27 00:52:45,556][105620] Updated weights for policy 1, policy_version 1304778 (0.0007) [2023-12-27 00:52:45,619][105620] Updated weights for policy 1, policy_version 1304788 (0.0005) [2023-12-27 00:52:45,681][105620] Updated weights for policy 1, policy_version 1304798 (0.0005) [2023-12-27 00:52:45,831][105692] Updated weights for policy 0, policy_version 1303249 (0.0007) [2023-12-27 00:52:45,886][105692] Updated weights for policy 0, policy_version 1303259 (0.0010) [2023-12-27 00:52:45,937][105692] Updated weights for policy 0, policy_version 1303269 (0.0010) [2023-12-27 00:52:45,999][105692] Updated weights for policy 0, policy_version 1303279 (0.0010) [2023-12-27 00:52:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 667762688. Throughput: 0: 9988.6, 1: 9504.7. Samples: 667727088. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:52:46,062][104569] Avg episode reward: [(0, '8718.874'), (1, '9262.965')] [2023-12-27 00:52:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001303280_333692928.pth... [2023-12-27 00:52:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001304800_334069760.pth... [2023-12-27 00:52:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001303712_333791232.pth [2023-12-27 00:52:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001302096_333389824.pth [2023-12-27 00:52:46,259][105620] Updated weights for policy 1, policy_version 1304808 (0.0005) [2023-12-27 00:52:46,315][105620] Updated weights for policy 1, policy_version 1304818 (0.0005) [2023-12-27 00:52:46,363][105620] Updated weights for policy 1, policy_version 1304828 (0.0005) [2023-12-27 00:52:46,684][105692] Updated weights for policy 0, policy_version 1303289 (0.0010) [2023-12-27 00:52:46,739][105692] Updated weights for policy 0, policy_version 1303299 (0.0010) [2023-12-27 00:52:46,791][105692] Updated weights for policy 0, policy_version 1303309 (0.0010) [2023-12-27 00:52:47,009][105620] Updated weights for policy 1, policy_version 1304838 (0.0008) [2023-12-27 00:52:47,063][105620] Updated weights for policy 1, policy_version 1304848 (0.0010) [2023-12-27 00:52:47,123][105620] Updated weights for policy 1, policy_version 1304858 (0.0010) [2023-12-27 00:52:47,531][105692] Updated weights for policy 0, policy_version 1303319 (0.0010) [2023-12-27 00:52:47,584][105692] Updated weights for policy 0, policy_version 1303329 (0.0006) [2023-12-27 00:52:47,642][105692] Updated weights for policy 0, policy_version 1303339 (0.0005) [2023-12-27 00:52:47,667][105620] Updated weights for policy 1, policy_version 1304868 (0.0007) [2023-12-27 00:52:47,711][105620] Updated weights for policy 1, policy_version 1304878 (0.0005) [2023-12-27 00:52:47,776][105620] Updated weights for policy 1, policy_version 1304888 (0.0005) [2023-12-27 00:52:48,204][105692] Updated weights for policy 0, policy_version 1303349 (0.0005) [2023-12-27 00:52:48,251][105692] Updated weights for policy 0, policy_version 1303359 (0.0005) [2023-12-27 00:52:48,298][105692] Updated weights for policy 0, policy_version 1303369 (0.0006) [2023-12-27 00:52:48,350][105620] Updated weights for policy 1, policy_version 1304898 (0.0006) [2023-12-27 00:52:48,411][105620] Updated weights for policy 1, policy_version 1304908 (0.0008) [2023-12-27 00:52:48,466][105620] Updated weights for policy 1, policy_version 1304918 (0.0010) [2023-12-27 00:52:48,527][105620] Updated weights for policy 1, policy_version 1304928 (0.0010) [2023-12-27 00:52:48,885][105692] Updated weights for policy 0, policy_version 1303379 (0.0011) [2023-12-27 00:52:48,941][105692] Updated weights for policy 0, policy_version 1303389 (0.0011) [2023-12-27 00:52:48,999][105692] Updated weights for policy 0, policy_version 1303399 (0.0010) [2023-12-27 00:52:49,285][105620] Updated weights for policy 1, policy_version 1304938 (0.0008) [2023-12-27 00:52:49,347][105620] Updated weights for policy 1, policy_version 1304948 (0.0008) [2023-12-27 00:52:49,408][105620] Updated weights for policy 1, policy_version 1304958 (0.0008) [2023-12-27 00:52:49,771][105692] Updated weights for policy 0, policy_version 1303409 (0.0010) [2023-12-27 00:52:49,837][105692] Updated weights for policy 0, policy_version 1303419 (0.0010) [2023-12-27 00:52:49,898][105692] Updated weights for policy 0, policy_version 1303429 (0.0006) [2023-12-27 00:52:49,972][105692] Updated weights for policy 0, policy_version 1303439 (0.0007) [2023-12-27 00:52:50,201][105620] Updated weights for policy 1, policy_version 1304968 (0.0009) [2023-12-27 00:52:50,254][105620] Updated weights for policy 1, policy_version 1304978 (0.0008) [2023-12-27 00:52:50,299][105620] Updated weights for policy 1, policy_version 1304988 (0.0008) [2023-12-27 00:52:50,682][105692] Updated weights for policy 0, policy_version 1303449 (0.0010) [2023-12-27 00:52:50,735][105692] Updated weights for policy 0, policy_version 1303459 (0.0011) [2023-12-27 00:52:50,781][105692] Updated weights for policy 0, policy_version 1303469 (0.0010) [2023-12-27 00:52:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 667860992. Throughput: 0: 10059.6, 1: 9576.6. Samples: 667852848. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:52:51,063][104569] Avg episode reward: [(0, '8535.626'), (1, '9354.138')] [2023-12-27 00:52:51,096][105620] Updated weights for policy 1, policy_version 1304998 (0.0008) [2023-12-27 00:52:51,157][105620] Updated weights for policy 1, policy_version 1305008 (0.0010) [2023-12-27 00:52:51,215][105620] Updated weights for policy 1, policy_version 1305018 (0.0010) [2023-12-27 00:52:51,546][105692] Updated weights for policy 0, policy_version 1303479 (0.0008) [2023-12-27 00:52:51,617][105692] Updated weights for policy 0, policy_version 1303489 (0.0009) [2023-12-27 00:52:51,684][105692] Updated weights for policy 0, policy_version 1303499 (0.0010) [2023-12-27 00:52:52,044][105620] Updated weights for policy 1, policy_version 1305028 (0.0009) [2023-12-27 00:52:52,101][105620] Updated weights for policy 1, policy_version 1305038 (0.0008) [2023-12-27 00:52:52,158][105620] Updated weights for policy 1, policy_version 1305048 (0.0009) [2023-12-27 00:52:52,439][105692] Updated weights for policy 0, policy_version 1303509 (0.0010) [2023-12-27 00:52:52,501][105692] Updated weights for policy 0, policy_version 1303519 (0.0010) [2023-12-27 00:52:52,565][105692] Updated weights for policy 0, policy_version 1303529 (0.0008) [2023-12-27 00:52:52,956][105620] Updated weights for policy 1, policy_version 1305058 (0.0008) [2023-12-27 00:52:53,008][105620] Updated weights for policy 1, policy_version 1305068 (0.0009) [2023-12-27 00:52:53,065][105620] Updated weights for policy 1, policy_version 1305078 (0.0009) [2023-12-27 00:52:53,116][105620] Updated weights for policy 1, policy_version 1305088 (0.0008) [2023-12-27 00:52:53,219][105692] Updated weights for policy 0, policy_version 1303539 (0.0010) [2023-12-27 00:52:53,274][105692] Updated weights for policy 0, policy_version 1303549 (0.0010) [2023-12-27 00:52:53,338][105692] Updated weights for policy 0, policy_version 1303559 (0.0010) [2023-12-27 00:52:53,853][105620] Updated weights for policy 1, policy_version 1305098 (0.0010) [2023-12-27 00:52:53,896][105620] Updated weights for policy 1, policy_version 1305108 (0.0010) [2023-12-27 00:52:53,951][105620] Updated weights for policy 1, policy_version 1305118 (0.0010) [2023-12-27 00:52:54,003][105692] Updated weights for policy 0, policy_version 1303569 (0.0010) [2023-12-27 00:52:54,068][105692] Updated weights for policy 0, policy_version 1303579 (0.0007) [2023-12-27 00:52:54,126][105692] Updated weights for policy 0, policy_version 1303589 (0.0007) [2023-12-27 00:52:54,171][105692] Updated weights for policy 0, policy_version 1303599 (0.0006) [2023-12-27 00:52:54,588][105620] Updated weights for policy 1, policy_version 1305128 (0.0006) [2023-12-27 00:52:54,646][105620] Updated weights for policy 1, policy_version 1305138 (0.0006) [2023-12-27 00:52:54,713][105620] Updated weights for policy 1, policy_version 1305148 (0.0006) [2023-12-27 00:52:54,865][105692] Updated weights for policy 0, policy_version 1303609 (0.0010) [2023-12-27 00:52:54,917][105692] Updated weights for policy 0, policy_version 1303619 (0.0010) [2023-12-27 00:52:54,980][105692] Updated weights for policy 0, policy_version 1303629 (0.0010) [2023-12-27 00:52:55,246][105620] Updated weights for policy 1, policy_version 1305158 (0.0009) [2023-12-27 00:52:55,294][105620] Updated weights for policy 1, policy_version 1305168 (0.0010) [2023-12-27 00:52:55,342][105620] Updated weights for policy 1, policy_version 1305178 (0.0010) [2023-12-27 00:52:55,753][105692] Updated weights for policy 0, policy_version 1303639 (0.0010) [2023-12-27 00:52:55,818][105692] Updated weights for policy 0, policy_version 1303649 (0.0010) [2023-12-27 00:52:55,883][105692] Updated weights for policy 0, policy_version 1303659 (0.0010) [2023-12-27 00:52:55,914][105620] Updated weights for policy 1, policy_version 1305188 (0.0008) [2023-12-27 00:52:55,967][105620] Updated weights for policy 1, policy_version 1305198 (0.0006) [2023-12-27 00:52:56,017][105620] Updated weights for policy 1, policy_version 1305208 (0.0009) [2023-12-27 00:52:56,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 667967488. Throughput: 0: 9960.9, 1: 9672.6. Samples: 667971164. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:52:56,062][104569] Avg episode reward: [(0, '8353.479'), (1, '9262.788')] [2023-12-27 00:52:56,555][105692] Updated weights for policy 0, policy_version 1303669 (0.0008) [2023-12-27 00:52:56,588][105620] Updated weights for policy 1, policy_version 1305218 (0.0006) [2023-12-27 00:52:56,606][105692] Updated weights for policy 0, policy_version 1303679 (0.0006) [2023-12-27 00:52:56,642][105620] Updated weights for policy 1, policy_version 1305228 (0.0005) [2023-12-27 00:52:56,658][105692] Updated weights for policy 0, policy_version 1303689 (0.0009) [2023-12-27 00:52:56,699][105620] Updated weights for policy 1, policy_version 1305238 (0.0007) [2023-12-27 00:52:56,745][105620] Updated weights for policy 1, policy_version 1305248 (0.0007) [2023-12-27 00:52:57,275][105620] Updated weights for policy 1, policy_version 1305258 (0.0005) [2023-12-27 00:52:57,330][105620] Updated weights for policy 1, policy_version 1305268 (0.0006) [2023-12-27 00:52:57,359][105692] Updated weights for policy 0, policy_version 1303699 (0.0010) [2023-12-27 00:52:57,388][105620] Updated weights for policy 1, policy_version 1305278 (0.0005) [2023-12-27 00:52:57,420][105692] Updated weights for policy 0, policy_version 1303709 (0.0010) [2023-12-27 00:52:57,484][105692] Updated weights for policy 0, policy_version 1303719 (0.0011) [2023-12-27 00:52:57,937][105620] Updated weights for policy 1, policy_version 1305288 (0.0010) [2023-12-27 00:52:57,995][105620] Updated weights for policy 1, policy_version 1305299 (0.0005) [2023-12-27 00:52:58,051][105620] Updated weights for policy 1, policy_version 1305309 (0.0005) [2023-12-27 00:52:58,210][105692] Updated weights for policy 0, policy_version 1303729 (0.0010) [2023-12-27 00:52:58,272][105692] Updated weights for policy 0, policy_version 1303739 (0.0011) [2023-12-27 00:52:58,339][105692] Updated weights for policy 0, policy_version 1303749 (0.0010) [2023-12-27 00:52:58,410][105692] Updated weights for policy 0, policy_version 1303759 (0.0009) [2023-12-27 00:52:58,758][105620] Updated weights for policy 1, policy_version 1305319 (0.0009) [2023-12-27 00:52:58,826][105620] Updated weights for policy 1, policy_version 1305329 (0.0013) [2023-12-27 00:52:58,892][105620] Updated weights for policy 1, policy_version 1305339 (0.0007) [2023-12-27 00:52:59,201][105692] Updated weights for policy 0, policy_version 1303769 (0.0009) [2023-12-27 00:52:59,274][105692] Updated weights for policy 0, policy_version 1303779 (0.0010) [2023-12-27 00:52:59,342][105692] Updated weights for policy 0, policy_version 1303789 (0.0010) [2023-12-27 00:52:59,560][105620] Updated weights for policy 1, policy_version 1305349 (0.0007) [2023-12-27 00:52:59,609][105620] Updated weights for policy 1, policy_version 1305359 (0.0005) [2023-12-27 00:52:59,661][105620] Updated weights for policy 1, policy_version 1305369 (0.0008) [2023-12-27 00:53:00,181][105692] Updated weights for policy 0, policy_version 1303799 (0.0009) [2023-12-27 00:53:00,240][105692] Updated weights for policy 0, policy_version 1303809 (0.0009) [2023-12-27 00:53:00,284][105620] Updated weights for policy 1, policy_version 1305379 (0.0007) [2023-12-27 00:53:00,299][105692] Updated weights for policy 0, policy_version 1303819 (0.0007) [2023-12-27 00:53:00,341][105620] Updated weights for policy 1, policy_version 1305389 (0.0007) [2023-12-27 00:53:00,387][105620] Updated weights for policy 1, policy_version 1305399 (0.0009) [2023-12-27 00:53:00,984][105692] Updated weights for policy 0, policy_version 1303829 (0.0005) [2023-12-27 00:53:01,049][105692] Updated weights for policy 0, policy_version 1303839 (0.0006) [2023-12-27 00:53:01,054][105620] Updated weights for policy 1, policy_version 1305409 (0.0008) [2023-12-27 00:53:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 668057600. Throughput: 0: 9979.1, 1: 9771.8. Samples: 668033716. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:53:01,062][104569] Avg episode reward: [(0, '8809.105'), (1, '9262.949')] [2023-12-27 00:53:01,107][105692] Updated weights for policy 0, policy_version 1303849 (0.0006) [2023-12-27 00:53:01,125][105620] Updated weights for policy 1, policy_version 1305419 (0.0006) [2023-12-27 00:53:01,147][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001303856_333840384.pth... [2023-12-27 00:53:01,151][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001302704_333545472.pth [2023-12-27 00:53:01,191][105620] Updated weights for policy 1, policy_version 1305429 (0.0007) [2023-12-27 00:53:01,252][105620] Updated weights for policy 1, policy_version 1305439 (0.0006) [2023-12-27 00:53:01,259][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001305440_334233600.pth... [2023-12-27 00:53:01,263][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001304256_333930496.pth [2023-12-27 00:53:01,785][105692] Updated weights for policy 0, policy_version 1303859 (0.0007) [2023-12-27 00:53:01,844][105692] Updated weights for policy 0, policy_version 1303869 (0.0009) [2023-12-27 00:53:01,902][105692] Updated weights for policy 0, policy_version 1303879 (0.0009) [2023-12-27 00:53:01,924][105620] Updated weights for policy 1, policy_version 1305449 (0.0007) [2023-12-27 00:53:01,978][105620] Updated weights for policy 1, policy_version 1305459 (0.0007) [2023-12-27 00:53:02,027][105620] Updated weights for policy 1, policy_version 1305469 (0.0008) [2023-12-27 00:53:02,539][105692] Updated weights for policy 0, policy_version 1303889 (0.0007) [2023-12-27 00:53:02,602][105692] Updated weights for policy 0, policy_version 1303899 (0.0009) [2023-12-27 00:53:02,662][105692] Updated weights for policy 0, policy_version 1303909 (0.0009) [2023-12-27 00:53:02,716][105692] Updated weights for policy 0, policy_version 1303919 (0.0005) [2023-12-27 00:53:02,857][105620] Updated weights for policy 1, policy_version 1305479 (0.0008) [2023-12-27 00:53:02,907][105620] Updated weights for policy 1, policy_version 1305489 (0.0009) [2023-12-27 00:53:02,961][105620] Updated weights for policy 1, policy_version 1305499 (0.0009) [2023-12-27 00:53:03,421][105692] Updated weights for policy 0, policy_version 1303929 (0.0008) [2023-12-27 00:53:03,475][105692] Updated weights for policy 0, policy_version 1303939 (0.0009) [2023-12-27 00:53:03,539][105692] Updated weights for policy 0, policy_version 1303949 (0.0009) [2023-12-27 00:53:03,669][105620] Updated weights for policy 1, policy_version 1305509 (0.0008) [2023-12-27 00:53:03,715][105620] Updated weights for policy 1, policy_version 1305519 (0.0009) [2023-12-27 00:53:03,763][105620] Updated weights for policy 1, policy_version 1305529 (0.0009) [2023-12-27 00:53:04,321][105692] Updated weights for policy 0, policy_version 1303959 (0.0008) [2023-12-27 00:53:04,382][105692] Updated weights for policy 0, policy_version 1303969 (0.0009) [2023-12-27 00:53:04,438][105692] Updated weights for policy 0, policy_version 1303979 (0.0008) [2023-12-27 00:53:04,503][105620] Updated weights for policy 1, policy_version 1305539 (0.0009) [2023-12-27 00:53:04,555][105620] Updated weights for policy 1, policy_version 1305549 (0.0010) [2023-12-27 00:53:04,613][105620] Updated weights for policy 1, policy_version 1305559 (0.0010) [2023-12-27 00:53:05,207][105692] Updated weights for policy 0, policy_version 1303989 (0.0008) [2023-12-27 00:53:05,217][105620] Updated weights for policy 1, policy_version 1305569 (0.0007) [2023-12-27 00:53:05,273][105692] Updated weights for policy 0, policy_version 1303999 (0.0008) [2023-12-27 00:53:05,281][105620] Updated weights for policy 1, policy_version 1305579 (0.0006) [2023-12-27 00:53:05,335][105692] Updated weights for policy 0, policy_version 1304009 (0.0009) [2023-12-27 00:53:05,348][105620] Updated weights for policy 1, policy_version 1305589 (0.0005) [2023-12-27 00:53:05,419][105620] Updated weights for policy 1, policy_version 1305599 (0.0005) [2023-12-27 00:53:05,888][105692] Updated weights for policy 0, policy_version 1304019 (0.0007) [2023-12-27 00:53:05,922][105620] Updated weights for policy 1, policy_version 1305609 (0.0006) [2023-12-27 00:53:05,956][105692] Updated weights for policy 0, policy_version 1304029 (0.0005) [2023-12-27 00:53:05,995][105620] Updated weights for policy 1, policy_version 1305619 (0.0005) [2023-12-27 00:53:06,023][105692] Updated weights for policy 0, policy_version 1304039 (0.0005) [2023-12-27 00:53:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 668155904. Throughput: 0: 9859.6, 1: 9788.4. Samples: 668149208. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:53:06,062][104569] Avg episode reward: [(0, '8995.520'), (1, '8990.004')] [2023-12-27 00:53:06,064][105620] Updated weights for policy 1, policy_version 1305629 (0.0006) [2023-12-27 00:53:06,682][105692] Updated weights for policy 0, policy_version 1304049 (0.0007) [2023-12-27 00:53:06,728][105620] Updated weights for policy 1, policy_version 1305639 (0.0008) [2023-12-27 00:53:06,735][105692] Updated weights for policy 0, policy_version 1304059 (0.0006) [2023-12-27 00:53:06,782][105692] Updated weights for policy 0, policy_version 1304069 (0.0007) [2023-12-27 00:53:06,789][105620] Updated weights for policy 1, policy_version 1305649 (0.0007) [2023-12-27 00:53:06,829][105692] Updated weights for policy 0, policy_version 1304079 (0.0008) [2023-12-27 00:53:06,850][105620] Updated weights for policy 1, policy_version 1305659 (0.0008) [2023-12-27 00:53:07,515][105620] Updated weights for policy 1, policy_version 1305669 (0.0009) [2023-12-27 00:53:07,574][105620] Updated weights for policy 1, policy_version 1305679 (0.0008) [2023-12-27 00:53:07,641][105620] Updated weights for policy 1, policy_version 1305689 (0.0009) [2023-12-27 00:53:07,656][105692] Updated weights for policy 0, policy_version 1304089 (0.0006) [2023-12-27 00:53:07,716][105692] Updated weights for policy 0, policy_version 1304099 (0.0008) [2023-12-27 00:53:07,778][105692] Updated weights for policy 0, policy_version 1304109 (0.0009) [2023-12-27 00:53:08,257][105620] Updated weights for policy 1, policy_version 1305699 (0.0007) [2023-12-27 00:53:08,314][105620] Updated weights for policy 1, policy_version 1305709 (0.0006) [2023-12-27 00:53:08,382][105620] Updated weights for policy 1, policy_version 1305719 (0.0008) [2023-12-27 00:53:08,637][105692] Updated weights for policy 0, policy_version 1304119 (0.0009) [2023-12-27 00:53:08,695][105692] Updated weights for policy 0, policy_version 1304129 (0.0008) [2023-12-27 00:53:08,757][105692] Updated weights for policy 0, policy_version 1304139 (0.0009) [2023-12-27 00:53:09,065][105620] Updated weights for policy 1, policy_version 1305729 (0.0008) [2023-12-27 00:53:09,121][105620] Updated weights for policy 1, policy_version 1305739 (0.0006) [2023-12-27 00:53:09,186][105620] Updated weights for policy 1, policy_version 1305749 (0.0009) [2023-12-27 00:53:09,246][105620] Updated weights for policy 1, policy_version 1305759 (0.0009) [2023-12-27 00:53:09,534][105692] Updated weights for policy 0, policy_version 1304149 (0.0010) [2023-12-27 00:53:09,597][105692] Updated weights for policy 0, policy_version 1304159 (0.0008) [2023-12-27 00:53:09,660][105692] Updated weights for policy 0, policy_version 1304169 (0.0009) [2023-12-27 00:53:10,018][105620] Updated weights for policy 1, policy_version 1305769 (0.0006) [2023-12-27 00:53:10,080][105620] Updated weights for policy 1, policy_version 1305779 (0.0005) [2023-12-27 00:53:10,139][105620] Updated weights for policy 1, policy_version 1305789 (0.0007) [2023-12-27 00:53:10,469][105692] Updated weights for policy 0, policy_version 1304179 (0.0009) [2023-12-27 00:53:10,530][105692] Updated weights for policy 0, policy_version 1304189 (0.0008) [2023-12-27 00:53:10,590][105692] Updated weights for policy 0, policy_version 1304199 (0.0009) [2023-12-27 00:53:10,813][105620] Updated weights for policy 1, policy_version 1305799 (0.0009) [2023-12-27 00:53:10,871][105620] Updated weights for policy 1, policy_version 1305809 (0.0009) [2023-12-27 00:53:10,928][105620] Updated weights for policy 1, policy_version 1305819 (0.0009) [2023-12-27 00:53:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 668262400. Throughput: 0: 9802.5, 1: 9992.5. Samples: 668268040. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:53:11,062][104569] Avg episode reward: [(0, '8539.862'), (1, '8902.504')] [2023-12-27 00:53:11,341][105692] Updated weights for policy 0, policy_version 1304209 (0.0008) [2023-12-27 00:53:11,408][105692] Updated weights for policy 0, policy_version 1304219 (0.0008) [2023-12-27 00:53:11,470][105692] Updated weights for policy 0, policy_version 1304229 (0.0009) [2023-12-27 00:53:11,535][105692] Updated weights for policy 0, policy_version 1304239 (0.0008) [2023-12-27 00:53:11,739][105620] Updated weights for policy 1, policy_version 1305829 (0.0009) [2023-12-27 00:53:11,799][105620] Updated weights for policy 1, policy_version 1305839 (0.0009) [2023-12-27 00:53:11,851][105620] Updated weights for policy 1, policy_version 1305849 (0.0009) [2023-12-27 00:53:12,335][105692] Updated weights for policy 0, policy_version 1304249 (0.0009) [2023-12-27 00:53:12,402][105692] Updated weights for policy 0, policy_version 1304259 (0.0008) [2023-12-27 00:53:12,466][105692] Updated weights for policy 0, policy_version 1304269 (0.0007) [2023-12-27 00:53:12,582][105620] Updated weights for policy 1, policy_version 1305859 (0.0010) [2023-12-27 00:53:12,640][105620] Updated weights for policy 1, policy_version 1305869 (0.0009) [2023-12-27 00:53:12,701][105620] Updated weights for policy 1, policy_version 1305879 (0.0010) [2023-12-27 00:53:13,149][105692] Updated weights for policy 0, policy_version 1304279 (0.0007) [2023-12-27 00:53:13,209][105692] Updated weights for policy 0, policy_version 1304290 (0.0009) [2023-12-27 00:53:13,262][105692] Updated weights for policy 0, policy_version 1304300 (0.0009) [2023-12-27 00:53:13,338][105620] Updated weights for policy 1, policy_version 1305889 (0.0009) [2023-12-27 00:53:13,400][105620] Updated weights for policy 1, policy_version 1305899 (0.0005) [2023-12-27 00:53:13,456][105620] Updated weights for policy 1, policy_version 1305909 (0.0005) [2023-12-27 00:53:13,510][105620] Updated weights for policy 1, policy_version 1305919 (0.0005) [2023-12-27 00:53:14,071][105692] Updated weights for policy 0, policy_version 1304310 (0.0009) [2023-12-27 00:53:14,125][105692] Updated weights for policy 0, policy_version 1304320 (0.0007) [2023-12-27 00:53:14,149][105620] Updated weights for policy 1, policy_version 1305929 (0.0010) [2023-12-27 00:53:14,184][105692] Updated weights for policy 0, policy_version 1304330 (0.0007) [2023-12-27 00:53:14,211][105620] Updated weights for policy 1, policy_version 1305939 (0.0010) [2023-12-27 00:53:14,276][105620] Updated weights for policy 1, policy_version 1305949 (0.0009) [2023-12-27 00:53:14,861][105692] Updated weights for policy 0, policy_version 1304340 (0.0005) [2023-12-27 00:53:14,918][105692] Updated weights for policy 0, policy_version 1304350 (0.0005) [2023-12-27 00:53:14,980][105692] Updated weights for policy 0, policy_version 1304360 (0.0006) [2023-12-27 00:53:15,008][105620] Updated weights for policy 1, policy_version 1305959 (0.0011) [2023-12-27 00:53:15,076][105620] Updated weights for policy 1, policy_version 1305969 (0.0010) [2023-12-27 00:53:15,149][105620] Updated weights for policy 1, policy_version 1305979 (0.0010) [2023-12-27 00:53:15,613][105692] Updated weights for policy 0, policy_version 1304370 (0.0007) [2023-12-27 00:53:15,661][105692] Updated weights for policy 0, policy_version 1304380 (0.0010) [2023-12-27 00:53:15,710][105692] Updated weights for policy 0, policy_version 1304390 (0.0008) [2023-12-27 00:53:15,775][105692] Updated weights for policy 0, policy_version 1304400 (0.0009) [2023-12-27 00:53:15,852][105620] Updated weights for policy 1, policy_version 1305989 (0.0009) [2023-12-27 00:53:15,913][105620] Updated weights for policy 1, policy_version 1305999 (0.0009) [2023-12-27 00:53:15,975][105620] Updated weights for policy 1, policy_version 1306009 (0.0009) [2023-12-27 00:53:16,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.3, 300 sec: 19521.9). Total num frames: 668360704. Throughput: 0: 9706.1, 1: 10032.5. Samples: 668325428. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:53:16,063][104569] Avg episode reward: [(0, '8627.028'), (1, '9266.688')] [2023-12-27 00:53:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001304400_333979648.pth... [2023-12-27 00:53:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001306016_334381056.pth... [2023-12-27 00:53:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001304800_334069760.pth [2023-12-27 00:53:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001303280_333692928.pth [2023-12-27 00:53:16,416][105692] Updated weights for policy 0, policy_version 1304410 (0.0009) [2023-12-27 00:53:16,470][105692] Updated weights for policy 0, policy_version 1304420 (0.0009) [2023-12-27 00:53:16,519][105692] Updated weights for policy 0, policy_version 1304430 (0.0009) [2023-12-27 00:53:16,726][105620] Updated weights for policy 1, policy_version 1306019 (0.0009) [2023-12-27 00:53:16,780][105620] Updated weights for policy 1, policy_version 1306029 (0.0005) [2023-12-27 00:53:16,829][105620] Updated weights for policy 1, policy_version 1306039 (0.0005) [2023-12-27 00:53:17,295][105692] Updated weights for policy 0, policy_version 1304440 (0.0006) [2023-12-27 00:53:17,342][105692] Updated weights for policy 0, policy_version 1304450 (0.0009) [2023-12-27 00:53:17,390][105692] Updated weights for policy 0, policy_version 1304460 (0.0009) [2023-12-27 00:53:17,461][105620] Updated weights for policy 1, policy_version 1306049 (0.0006) [2023-12-27 00:53:17,515][105620] Updated weights for policy 1, policy_version 1306059 (0.0009) [2023-12-27 00:53:17,566][105620] Updated weights for policy 1, policy_version 1306069 (0.0009) [2023-12-27 00:53:17,617][105620] Updated weights for policy 1, policy_version 1306079 (0.0009) [2023-12-27 00:53:17,992][105692] Updated weights for policy 0, policy_version 1304470 (0.0007) [2023-12-27 00:53:18,039][105692] Updated weights for policy 0, policy_version 1304480 (0.0005) [2023-12-27 00:53:18,078][105585] KL-divergence is very high: 253.5852 [2023-12-27 00:53:18,095][105692] Updated weights for policy 0, policy_version 1304490 (0.0005) [2023-12-27 00:53:18,121][105585] KL-divergence is very high: 414.4216 [2023-12-27 00:53:18,367][105620] Updated weights for policy 1, policy_version 1306089 (0.0008) [2023-12-27 00:53:18,414][105620] Updated weights for policy 1, policy_version 1306099 (0.0008) [2023-12-27 00:53:18,469][105620] Updated weights for policy 1, policy_version 1306109 (0.0008) [2023-12-27 00:53:18,785][105692] Updated weights for policy 0, policy_version 1304500 (0.0007) [2023-12-27 00:53:18,847][105692] Updated weights for policy 0, policy_version 1304510 (0.0009) [2023-12-27 00:53:18,910][105692] Updated weights for policy 0, policy_version 1304520 (0.0009) [2023-12-27 00:53:19,233][105620] Updated weights for policy 1, policy_version 1306119 (0.0008) [2023-12-27 00:53:19,303][105620] Updated weights for policy 1, policy_version 1306129 (0.0009) [2023-12-27 00:53:19,375][105620] Updated weights for policy 1, policy_version 1306139 (0.0009) [2023-12-27 00:53:19,601][105692] Updated weights for policy 0, policy_version 1304530 (0.0008) [2023-12-27 00:53:19,659][105692] Updated weights for policy 0, policy_version 1304540 (0.0006) [2023-12-27 00:53:19,715][105692] Updated weights for policy 0, policy_version 1304550 (0.0008) [2023-12-27 00:53:19,768][105692] Updated weights for policy 0, policy_version 1304560 (0.0009) [2023-12-27 00:53:20,259][105620] Updated weights for policy 1, policy_version 1306149 (0.0009) [2023-12-27 00:53:20,318][105620] Updated weights for policy 1, policy_version 1306159 (0.0010) [2023-12-27 00:53:20,383][105620] Updated weights for policy 1, policy_version 1306169 (0.0009) [2023-12-27 00:53:20,543][105692] Updated weights for policy 0, policy_version 1304570 (0.0008) [2023-12-27 00:53:20,604][105692] Updated weights for policy 0, policy_version 1304580 (0.0008) [2023-12-27 00:53:20,668][105692] Updated weights for policy 0, policy_version 1304590 (0.0009) [2023-12-27 00:53:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 668450816. Throughput: 0: 9712.3, 1: 9988.5. Samples: 668443320. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:53:21,063][104569] Avg episode reward: [(0, '8806.687'), (1, '9170.862')] [2023-12-27 00:53:21,149][105620] Updated weights for policy 1, policy_version 1306179 (0.0011) [2023-12-27 00:53:21,207][105620] Updated weights for policy 1, policy_version 1306189 (0.0009) [2023-12-27 00:53:21,271][105620] Updated weights for policy 1, policy_version 1306199 (0.0009) [2023-12-27 00:53:21,430][105692] Updated weights for policy 0, policy_version 1304600 (0.0009) [2023-12-27 00:53:21,489][105692] Updated weights for policy 0, policy_version 1304610 (0.0009) [2023-12-27 00:53:21,548][105692] Updated weights for policy 0, policy_version 1304620 (0.0009) [2023-12-27 00:53:22,034][105620] Updated weights for policy 1, policy_version 1306209 (0.0009) [2023-12-27 00:53:22,096][105620] Updated weights for policy 1, policy_version 1306219 (0.0006) [2023-12-27 00:53:22,152][105620] Updated weights for policy 1, policy_version 1306229 (0.0006) [2023-12-27 00:53:22,213][105620] Updated weights for policy 1, policy_version 1306239 (0.0009) [2023-12-27 00:53:22,363][105692] Updated weights for policy 0, policy_version 1304630 (0.0010) [2023-12-27 00:53:22,418][105692] Updated weights for policy 0, policy_version 1304640 (0.0009) [2023-12-27 00:53:22,478][105692] Updated weights for policy 0, policy_version 1304650 (0.0009) [2023-12-27 00:53:22,905][105620] Updated weights for policy 1, policy_version 1306249 (0.0007) [2023-12-27 00:53:22,978][105620] Updated weights for policy 1, policy_version 1306259 (0.0008) [2023-12-27 00:53:23,040][105620] Updated weights for policy 1, policy_version 1306269 (0.0008) [2023-12-27 00:53:23,254][105692] Updated weights for policy 0, policy_version 1304660 (0.0008) [2023-12-27 00:53:23,300][105692] Updated weights for policy 0, policy_version 1304670 (0.0008) [2023-12-27 00:53:23,358][105692] Updated weights for policy 0, policy_version 1304680 (0.0009) [2023-12-27 00:53:23,749][105620] Updated weights for policy 1, policy_version 1306279 (0.0009) [2023-12-27 00:53:23,801][105620] Updated weights for policy 1, policy_version 1306289 (0.0009) [2023-12-27 00:53:23,865][105620] Updated weights for policy 1, policy_version 1306299 (0.0009) [2023-12-27 00:53:24,136][105692] Updated weights for policy 0, policy_version 1304690 (0.0009) [2023-12-27 00:53:24,197][105692] Updated weights for policy 0, policy_version 1304700 (0.0009) [2023-12-27 00:53:24,254][105692] Updated weights for policy 0, policy_version 1304710 (0.0009) [2023-12-27 00:53:24,309][105692] Updated weights for policy 0, policy_version 1304720 (0.0009) [2023-12-27 00:53:24,468][105620] Updated weights for policy 1, policy_version 1306309 (0.0008) [2023-12-27 00:53:24,527][105620] Updated weights for policy 1, policy_version 1306319 (0.0009) [2023-12-27 00:53:24,592][105620] Updated weights for policy 1, policy_version 1306329 (0.0007) [2023-12-27 00:53:25,137][105692] Updated weights for policy 0, policy_version 1304730 (0.0007) [2023-12-27 00:53:25,196][105692] Updated weights for policy 0, policy_version 1304740 (0.0005) [2023-12-27 00:53:25,228][105620] Updated weights for policy 1, policy_version 1306339 (0.0007) [2023-12-27 00:53:25,253][105692] Updated weights for policy 0, policy_version 1304750 (0.0005) [2023-12-27 00:53:25,295][105620] Updated weights for policy 1, policy_version 1306349 (0.0010) [2023-12-27 00:53:25,360][105620] Updated weights for policy 1, policy_version 1306359 (0.0010) [2023-12-27 00:53:25,954][105692] Updated weights for policy 0, policy_version 1304760 (0.0009) [2023-12-27 00:53:25,997][105620] Updated weights for policy 1, policy_version 1306369 (0.0010) [2023-12-27 00:53:26,020][105692] Updated weights for policy 0, policy_version 1304770 (0.0010) [2023-12-27 00:53:26,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 668540928. Throughput: 0: 9635.3, 1: 10067.0. Samples: 668556428. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:53:26,063][105620] Updated weights for policy 1, policy_version 1306379 (0.0007) [2023-12-27 00:53:26,063][104569] Avg episode reward: [(0, '7994.309'), (1, '9171.062')] [2023-12-27 00:53:26,085][105692] Updated weights for policy 0, policy_version 1304780 (0.0010) [2023-12-27 00:53:26,118][105620] Updated weights for policy 1, policy_version 1306389 (0.0010) [2023-12-27 00:53:26,166][105620] Updated weights for policy 1, policy_version 1306399 (0.0010) [2023-12-27 00:53:26,770][105620] Updated weights for policy 1, policy_version 1306409 (0.0010) [2023-12-27 00:53:26,798][105692] Updated weights for policy 0, policy_version 1304790 (0.0010) [2023-12-27 00:53:26,825][105620] Updated weights for policy 1, policy_version 1306419 (0.0010) [2023-12-27 00:53:26,857][105692] Updated weights for policy 0, policy_version 1304800 (0.0010) [2023-12-27 00:53:26,874][105620] Updated weights for policy 1, policy_version 1306429 (0.0010) [2023-12-27 00:53:26,912][105692] Updated weights for policy 0, policy_version 1304810 (0.0010) [2023-12-27 00:53:27,550][105620] Updated weights for policy 1, policy_version 1306439 (0.0007) [2023-12-27 00:53:27,593][105692] Updated weights for policy 0, policy_version 1304820 (0.0008) [2023-12-27 00:53:27,601][105620] Updated weights for policy 1, policy_version 1306449 (0.0010) [2023-12-27 00:53:27,641][105692] Updated weights for policy 0, policy_version 1304830 (0.0005) [2023-12-27 00:53:27,645][105620] Updated weights for policy 1, policy_version 1306459 (0.0010) [2023-12-27 00:53:27,693][105692] Updated weights for policy 0, policy_version 1304840 (0.0005) [2023-12-27 00:53:28,289][105692] Updated weights for policy 0, policy_version 1304850 (0.0007) [2023-12-27 00:53:28,333][105620] Updated weights for policy 1, policy_version 1306469 (0.0009) [2023-12-27 00:53:28,337][105692] Updated weights for policy 0, policy_version 1304860 (0.0010) [2023-12-27 00:53:28,397][105620] Updated weights for policy 1, policy_version 1306479 (0.0011) [2023-12-27 00:53:28,404][105692] Updated weights for policy 0, policy_version 1304870 (0.0007) [2023-12-27 00:53:28,459][105620] Updated weights for policy 1, policy_version 1306489 (0.0010) [2023-12-27 00:53:28,469][105692] Updated weights for policy 0, policy_version 1304880 (0.0006) [2023-12-27 00:53:29,031][105692] Updated weights for policy 0, policy_version 1304890 (0.0005) [2023-12-27 00:53:29,096][105692] Updated weights for policy 0, policy_version 1304900 (0.0005) [2023-12-27 00:53:29,161][105692] Updated weights for policy 0, policy_version 1304910 (0.0009) [2023-12-27 00:53:29,198][105620] Updated weights for policy 1, policy_version 1306499 (0.0010) [2023-12-27 00:53:29,261][105620] Updated weights for policy 1, policy_version 1306509 (0.0008) [2023-12-27 00:53:29,322][105620] Updated weights for policy 1, policy_version 1306519 (0.0011) [2023-12-27 00:53:29,831][105692] Updated weights for policy 0, policy_version 1304920 (0.0010) [2023-12-27 00:53:29,895][105692] Updated weights for policy 0, policy_version 1304930 (0.0011) [2023-12-27 00:53:29,959][105692] Updated weights for policy 0, policy_version 1304940 (0.0009) [2023-12-27 00:53:30,079][105620] Updated weights for policy 1, policy_version 1306529 (0.0009) [2023-12-27 00:53:30,131][105620] Updated weights for policy 1, policy_version 1306539 (0.0008) [2023-12-27 00:53:30,183][105620] Updated weights for policy 1, policy_version 1306549 (0.0010) [2023-12-27 00:53:30,235][105620] Updated weights for policy 1, policy_version 1306559 (0.0010) [2023-12-27 00:53:30,640][105692] Updated weights for policy 0, policy_version 1304950 (0.0007) [2023-12-27 00:53:30,687][105692] Updated weights for policy 0, policy_version 1304960 (0.0005) [2023-12-27 00:53:30,737][105692] Updated weights for policy 0, policy_version 1304970 (0.0005) [2023-12-27 00:53:30,983][105620] Updated weights for policy 1, policy_version 1306569 (0.0009) [2023-12-27 00:53:31,033][105620] Updated weights for policy 1, policy_version 1306579 (0.0008) [2023-12-27 00:53:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 668647424. Throughput: 0: 9694.2, 1: 10112.6. Samples: 668618392. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:53:31,062][104569] Avg episode reward: [(0, '8272.329'), (1, '9354.178')] [2023-12-27 00:53:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001304976_334127104.pth... [2023-12-27 00:53:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001303856_333840384.pth [2023-12-27 00:53:31,089][105620] Updated weights for policy 1, policy_version 1306589 (0.0007) [2023-12-27 00:53:31,103][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001306592_334528512.pth... [2023-12-27 00:53:31,106][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001305440_334233600.pth [2023-12-27 00:53:31,393][105692] Updated weights for policy 0, policy_version 1304980 (0.0007) [2023-12-27 00:53:31,458][105692] Updated weights for policy 0, policy_version 1304990 (0.0008) [2023-12-27 00:53:31,514][105692] Updated weights for policy 0, policy_version 1305000 (0.0008) [2023-12-27 00:53:31,779][105620] Updated weights for policy 1, policy_version 1306599 (0.0009) [2023-12-27 00:53:31,835][105620] Updated weights for policy 1, policy_version 1306609 (0.0011) [2023-12-27 00:53:31,894][105620] Updated weights for policy 1, policy_version 1306619 (0.0011) [2023-12-27 00:53:32,214][105692] Updated weights for policy 0, policy_version 1305010 (0.0006) [2023-12-27 00:53:32,274][105692] Updated weights for policy 0, policy_version 1305020 (0.0007) [2023-12-27 00:53:32,332][105692] Updated weights for policy 0, policy_version 1305030 (0.0008) [2023-12-27 00:53:32,395][105692] Updated weights for policy 0, policy_version 1305040 (0.0008) [2023-12-27 00:53:32,633][105620] Updated weights for policy 1, policy_version 1306629 (0.0009) [2023-12-27 00:53:32,701][105620] Updated weights for policy 1, policy_version 1306639 (0.0008) [2023-12-27 00:53:32,767][105620] Updated weights for policy 1, policy_version 1306649 (0.0008) [2023-12-27 00:53:33,042][105692] Updated weights for policy 0, policy_version 1305050 (0.0007) [2023-12-27 00:53:33,101][105692] Updated weights for policy 0, policy_version 1305060 (0.0005) [2023-12-27 00:53:33,159][105692] Updated weights for policy 0, policy_version 1305070 (0.0010) [2023-12-27 00:53:33,469][105620] Updated weights for policy 1, policy_version 1306659 (0.0007) [2023-12-27 00:53:33,520][105620] Updated weights for policy 1, policy_version 1306669 (0.0005) [2023-12-27 00:53:33,580][105620] Updated weights for policy 1, policy_version 1306679 (0.0007) [2023-12-27 00:53:33,710][105692] Updated weights for policy 0, policy_version 1305080 (0.0008) [2023-12-27 00:53:33,773][105692] Updated weights for policy 0, policy_version 1305090 (0.0010) [2023-12-27 00:53:33,828][105692] Updated weights for policy 0, policy_version 1305100 (0.0009) [2023-12-27 00:53:34,292][105620] Updated weights for policy 1, policy_version 1306689 (0.0009) [2023-12-27 00:53:34,345][105620] Updated weights for policy 1, policy_version 1306699 (0.0008) [2023-12-27 00:53:34,395][105620] Updated weights for policy 1, policy_version 1306709 (0.0008) [2023-12-27 00:53:34,448][105620] Updated weights for policy 1, policy_version 1306719 (0.0008) [2023-12-27 00:53:34,534][105692] Updated weights for policy 0, policy_version 1305110 (0.0007) [2023-12-27 00:53:34,588][105692] Updated weights for policy 0, policy_version 1305120 (0.0006) [2023-12-27 00:53:34,646][105692] Updated weights for policy 0, policy_version 1305130 (0.0006) [2023-12-27 00:53:35,221][105692] Updated weights for policy 0, policy_version 1305140 (0.0008) [2023-12-27 00:53:35,234][105620] Updated weights for policy 1, policy_version 1306729 (0.0005) [2023-12-27 00:53:35,276][105692] Updated weights for policy 0, policy_version 1305150 (0.0010) [2023-12-27 00:53:35,295][105620] Updated weights for policy 1, policy_version 1306739 (0.0005) [2023-12-27 00:53:35,336][105692] Updated weights for policy 0, policy_version 1305160 (0.0006) [2023-12-27 00:53:35,347][105620] Updated weights for policy 1, policy_version 1306749 (0.0008) [2023-12-27 00:53:35,924][105692] Updated weights for policy 0, policy_version 1305170 (0.0006) [2023-12-27 00:53:35,982][105692] Updated weights for policy 0, policy_version 1305180 (0.0011) [2023-12-27 00:53:36,026][105620] Updated weights for policy 1, policy_version 1306759 (0.0008) [2023-12-27 00:53:36,036][105692] Updated weights for policy 0, policy_version 1305190 (0.0006) [2023-12-27 00:53:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.7, 300 sec: 19494.2). Total num frames: 668745728. Throughput: 0: 9695.8, 1: 9976.0. Samples: 668738080. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:53:36,063][104569] Avg episode reward: [(0, '8548.140'), (1, '9354.243')] [2023-12-27 00:53:36,082][105692] Updated weights for policy 0, policy_version 1305200 (0.0005) [2023-12-27 00:53:36,094][105620] Updated weights for policy 1, policy_version 1306769 (0.0009) [2023-12-27 00:53:36,151][105620] Updated weights for policy 1, policy_version 1306779 (0.0008) [2023-12-27 00:53:36,732][105692] Updated weights for policy 0, policy_version 1305210 (0.0011) [2023-12-27 00:53:36,791][105692] Updated weights for policy 0, policy_version 1305220 (0.0011) [2023-12-27 00:53:36,850][105692] Updated weights for policy 0, policy_version 1305230 (0.0011) [2023-12-27 00:53:36,952][105620] Updated weights for policy 1, policy_version 1306789 (0.0009) [2023-12-27 00:53:37,013][105620] Updated weights for policy 1, policy_version 1306799 (0.0008) [2023-12-27 00:53:37,076][105620] Updated weights for policy 1, policy_version 1306809 (0.0008) [2023-12-27 00:53:37,516][105692] Updated weights for policy 0, policy_version 1305240 (0.0007) [2023-12-27 00:53:37,574][105692] Updated weights for policy 0, policy_version 1305250 (0.0006) [2023-12-27 00:53:37,632][105692] Updated weights for policy 0, policy_version 1305260 (0.0006) [2023-12-27 00:53:37,928][105620] Updated weights for policy 1, policy_version 1306819 (0.0009) [2023-12-27 00:53:37,973][105620] Updated weights for policy 1, policy_version 1306829 (0.0008) [2023-12-27 00:53:38,026][105620] Updated weights for policy 1, policy_version 1306839 (0.0008) [2023-12-27 00:53:38,272][105692] Updated weights for policy 0, policy_version 1305270 (0.0009) [2023-12-27 00:53:38,341][105692] Updated weights for policy 0, policy_version 1305280 (0.0010) [2023-12-27 00:53:38,399][105692] Updated weights for policy 0, policy_version 1305290 (0.0009) [2023-12-27 00:53:38,746][105620] Updated weights for policy 1, policy_version 1306849 (0.0007) [2023-12-27 00:53:38,815][105620] Updated weights for policy 1, policy_version 1306859 (0.0006) [2023-12-27 00:53:38,881][105620] Updated weights for policy 1, policy_version 1306869 (0.0006) [2023-12-27 00:53:38,940][105620] Updated weights for policy 1, policy_version 1306879 (0.0006) [2023-12-27 00:53:39,107][105692] Updated weights for policy 0, policy_version 1305300 (0.0008) [2023-12-27 00:53:39,173][105692] Updated weights for policy 0, policy_version 1305310 (0.0009) [2023-12-27 00:53:39,255][105692] Updated weights for policy 0, policy_version 1305320 (0.0007) [2023-12-27 00:53:39,545][105620] Updated weights for policy 1, policy_version 1306889 (0.0006) [2023-12-27 00:53:39,613][105620] Updated weights for policy 1, policy_version 1306899 (0.0006) [2023-12-27 00:53:39,673][105620] Updated weights for policy 1, policy_version 1306909 (0.0008) [2023-12-27 00:53:39,981][105692] Updated weights for policy 0, policy_version 1305330 (0.0008) [2023-12-27 00:53:40,048][105692] Updated weights for policy 0, policy_version 1305340 (0.0010) [2023-12-27 00:53:40,107][105692] Updated weights for policy 0, policy_version 1305350 (0.0009) [2023-12-27 00:53:40,174][105692] Updated weights for policy 0, policy_version 1305360 (0.0010) [2023-12-27 00:53:40,371][105620] Updated weights for policy 1, policy_version 1306919 (0.0008) [2023-12-27 00:53:40,432][105620] Updated weights for policy 1, policy_version 1306929 (0.0009) [2023-12-27 00:53:40,487][105620] Updated weights for policy 1, policy_version 1306939 (0.0009) [2023-12-27 00:53:40,944][105692] Updated weights for policy 0, policy_version 1305370 (0.0009) [2023-12-27 00:53:41,000][105692] Updated weights for policy 0, policy_version 1305380 (0.0011) [2023-12-27 00:53:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 668844032. Throughput: 0: 9774.1, 1: 9927.4. Samples: 668857736. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:53:41,063][104569] Avg episode reward: [(0, '8274.323'), (1, '9262.552')] [2023-12-27 00:53:41,064][105692] Updated weights for policy 0, policy_version 1305390 (0.0008) [2023-12-27 00:53:41,186][105620] Updated weights for policy 1, policy_version 1306949 (0.0009) [2023-12-27 00:53:41,245][105620] Updated weights for policy 1, policy_version 1306959 (0.0010) [2023-12-27 00:53:41,304][105620] Updated weights for policy 1, policy_version 1306969 (0.0008) [2023-12-27 00:53:41,843][105692] Updated weights for policy 0, policy_version 1305400 (0.0011) [2023-12-27 00:53:41,899][105692] Updated weights for policy 0, policy_version 1305410 (0.0010) [2023-12-27 00:53:41,956][105692] Updated weights for policy 0, policy_version 1305420 (0.0006) [2023-12-27 00:53:42,018][105620] Updated weights for policy 1, policy_version 1306979 (0.0009) [2023-12-27 00:53:42,077][105620] Updated weights for policy 1, policy_version 1306989 (0.0010) [2023-12-27 00:53:42,134][105620] Updated weights for policy 1, policy_version 1306999 (0.0008) [2023-12-27 00:53:42,630][105692] Updated weights for policy 0, policy_version 1305430 (0.0009) [2023-12-27 00:53:42,697][105692] Updated weights for policy 0, policy_version 1305440 (0.0011) [2023-12-27 00:53:42,764][105692] Updated weights for policy 0, policy_version 1305450 (0.0011) [2023-12-27 00:53:42,782][105620] Updated weights for policy 1, policy_version 1307009 (0.0007) [2023-12-27 00:53:42,838][105620] Updated weights for policy 1, policy_version 1307019 (0.0007) [2023-12-27 00:53:42,899][105620] Updated weights for policy 1, policy_version 1307029 (0.0008) [2023-12-27 00:53:42,964][105620] Updated weights for policy 1, policy_version 1307039 (0.0008) [2023-12-27 00:53:43,408][105692] Updated weights for policy 0, policy_version 1305460 (0.0011) [2023-12-27 00:53:43,460][105692] Updated weights for policy 0, policy_version 1305470 (0.0010) [2023-12-27 00:53:43,508][105692] Updated weights for policy 0, policy_version 1305480 (0.0010) [2023-12-27 00:53:43,698][105620] Updated weights for policy 1, policy_version 1307049 (0.0008) [2023-12-27 00:53:43,756][105620] Updated weights for policy 1, policy_version 1307059 (0.0006) [2023-12-27 00:53:43,817][105620] Updated weights for policy 1, policy_version 1307069 (0.0006) [2023-12-27 00:53:44,249][105692] Updated weights for policy 0, policy_version 1305490 (0.0010) [2023-12-27 00:53:44,324][105692] Updated weights for policy 0, policy_version 1305500 (0.0009) [2023-12-27 00:53:44,382][105692] Updated weights for policy 0, policy_version 1305510 (0.0009) [2023-12-27 00:53:44,442][105692] Updated weights for policy 0, policy_version 1305520 (0.0009) [2023-12-27 00:53:44,484][105620] Updated weights for policy 1, policy_version 1307079 (0.0005) [2023-12-27 00:53:44,534][105620] Updated weights for policy 1, policy_version 1307089 (0.0006) [2023-12-27 00:53:44,586][105620] Updated weights for policy 1, policy_version 1307099 (0.0005) [2023-12-27 00:53:45,130][105692] Updated weights for policy 0, policy_version 1305530 (0.0005) [2023-12-27 00:53:45,196][105692] Updated weights for policy 0, policy_version 1305540 (0.0009) [2023-12-27 00:53:45,244][105692] Updated weights for policy 0, policy_version 1305550 (0.0009) [2023-12-27 00:53:45,306][105620] Updated weights for policy 1, policy_version 1307109 (0.0009) [2023-12-27 00:53:45,365][105620] Updated weights for policy 1, policy_version 1307119 (0.0009) [2023-12-27 00:53:45,416][105620] Updated weights for policy 1, policy_version 1307129 (0.0009) [2023-12-27 00:53:45,957][105692] Updated weights for policy 0, policy_version 1305560 (0.0009) [2023-12-27 00:53:46,005][105692] Updated weights for policy 0, policy_version 1305570 (0.0009) [2023-12-27 00:53:46,060][105692] Updated weights for policy 0, policy_version 1305580 (0.0009) [2023-12-27 00:53:46,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 668942336. Throughput: 0: 9780.4, 1: 9837.0. Samples: 668916496. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:53:46,062][104569] Avg episode reward: [(0, '8273.284'), (1, '9262.265')] [2023-12-27 00:53:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001307136_334667776.pth... [2023-12-27 00:53:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001306016_334381056.pth [2023-12-27 00:53:46,083][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001305584_334282752.pth... [2023-12-27 00:53:46,086][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001304400_333979648.pth [2023-12-27 00:53:46,190][105620] Updated weights for policy 1, policy_version 1307139 (0.0009) [2023-12-27 00:53:46,236][105620] Updated weights for policy 1, policy_version 1307149 (0.0008) [2023-12-27 00:53:46,283][105620] Updated weights for policy 1, policy_version 1307159 (0.0009) [2023-12-27 00:53:46,835][105692] Updated weights for policy 0, policy_version 1305590 (0.0008) [2023-12-27 00:53:46,887][105692] Updated weights for policy 0, policy_version 1305600 (0.0007) [2023-12-27 00:53:46,933][105692] Updated weights for policy 0, policy_version 1305610 (0.0008) [2023-12-27 00:53:47,056][105620] Updated weights for policy 1, policy_version 1307169 (0.0009) [2023-12-27 00:53:47,107][105620] Updated weights for policy 1, policy_version 1307179 (0.0008) [2023-12-27 00:53:47,164][105620] Updated weights for policy 1, policy_version 1307189 (0.0009) [2023-12-27 00:53:47,225][105620] Updated weights for policy 1, policy_version 1307199 (0.0009) [2023-12-27 00:53:47,630][105692] Updated weights for policy 0, policy_version 1305620 (0.0008) [2023-12-27 00:53:47,677][105585] KL-divergence is very high: 199.2202 [2023-12-27 00:53:47,687][105692] Updated weights for policy 0, policy_version 1305630 (0.0006) [2023-12-27 00:53:47,718][105585] KL-divergence is very high: 333.9375 [2023-12-27 00:53:47,739][105692] Updated weights for policy 0, policy_version 1305640 (0.0005) [2023-12-27 00:53:47,763][105585] KL-divergence is very high: 336.5129 [2023-12-27 00:53:48,021][105620] Updated weights for policy 1, policy_version 1307209 (0.0009) [2023-12-27 00:53:48,091][105620] Updated weights for policy 1, policy_version 1307219 (0.0009) [2023-12-27 00:53:48,152][105620] Updated weights for policy 1, policy_version 1307229 (0.0010) [2023-12-27 00:53:48,348][105692] Updated weights for policy 0, policy_version 1305650 (0.0008) [2023-12-27 00:53:48,416][105692] Updated weights for policy 0, policy_version 1305660 (0.0009) [2023-12-27 00:53:48,482][105692] Updated weights for policy 0, policy_version 1305670 (0.0009) [2023-12-27 00:53:48,537][105692] Updated weights for policy 0, policy_version 1305680 (0.0010) [2023-12-27 00:53:48,878][105620] Updated weights for policy 1, policy_version 1307239 (0.0009) [2023-12-27 00:53:48,927][105620] Updated weights for policy 1, policy_version 1307249 (0.0008) [2023-12-27 00:53:48,980][105620] Updated weights for policy 1, policy_version 1307259 (0.0006) [2023-12-27 00:53:49,188][105692] Updated weights for policy 0, policy_version 1305690 (0.0010) [2023-12-27 00:53:49,252][105692] Updated weights for policy 0, policy_version 1305700 (0.0010) [2023-12-27 00:53:49,308][105692] Updated weights for policy 0, policy_version 1305710 (0.0012) [2023-12-27 00:53:49,880][105620] Updated weights for policy 1, policy_version 1307269 (0.0007) [2023-12-27 00:53:49,947][105620] Updated weights for policy 1, policy_version 1307279 (0.0008) [2023-12-27 00:53:49,962][105692] Updated weights for policy 0, policy_version 1305720 (0.0008) [2023-12-27 00:53:50,018][105620] Updated weights for policy 1, policy_version 1307289 (0.0007) [2023-12-27 00:53:50,022][105692] Updated weights for policy 0, policy_version 1305730 (0.0006) [2023-12-27 00:53:50,079][105692] Updated weights for policy 0, policy_version 1305740 (0.0006) [2023-12-27 00:53:50,638][105692] Updated weights for policy 0, policy_version 1305750 (0.0006) [2023-12-27 00:53:50,686][105692] Updated weights for policy 0, policy_version 1305760 (0.0009) [2023-12-27 00:53:50,773][105692] Updated weights for policy 0, policy_version 1305770 (0.0009) [2023-12-27 00:53:50,813][105620] Updated weights for policy 1, policy_version 1307299 (0.0007) [2023-12-27 00:53:50,882][105620] Updated weights for policy 1, policy_version 1307309 (0.0009) [2023-12-27 00:53:50,942][105620] Updated weights for policy 1, policy_version 1307319 (0.0008) [2023-12-27 00:53:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 669048832. Throughput: 0: 9873.0, 1: 9752.2. Samples: 669032340. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:53:51,063][104569] Avg episode reward: [(0, '8001.722'), (1, '9080.583')] [2023-12-27 00:53:51,575][105692] Updated weights for policy 0, policy_version 1305780 (0.0008) [2023-12-27 00:53:51,637][105692] Updated weights for policy 0, policy_version 1305790 (0.0008) [2023-12-27 00:53:51,689][105620] Updated weights for policy 1, policy_version 1307329 (0.0005) [2023-12-27 00:53:51,703][105692] Updated weights for policy 0, policy_version 1305800 (0.0008) [2023-12-27 00:53:51,754][105620] Updated weights for policy 1, policy_version 1307339 (0.0008) [2023-12-27 00:53:51,805][105620] Updated weights for policy 1, policy_version 1307349 (0.0008) [2023-12-27 00:53:51,860][105620] Updated weights for policy 1, policy_version 1307359 (0.0008) [2023-12-27 00:53:52,497][105692] Updated weights for policy 0, policy_version 1305810 (0.0008) [2023-12-27 00:53:52,560][105692] Updated weights for policy 0, policy_version 1305820 (0.0009) [2023-12-27 00:53:52,569][105620] Updated weights for policy 1, policy_version 1307369 (0.0008) [2023-12-27 00:53:52,612][105692] Updated weights for policy 0, policy_version 1305830 (0.0009) [2023-12-27 00:53:52,638][105620] Updated weights for policy 1, policy_version 1307379 (0.0007) [2023-12-27 00:53:52,669][105692] Updated weights for policy 0, policy_version 1305840 (0.0005) [2023-12-27 00:53:52,700][105620] Updated weights for policy 1, policy_version 1307389 (0.0008) [2023-12-27 00:53:53,414][105620] Updated weights for policy 1, policy_version 1307399 (0.0009) [2023-12-27 00:53:53,428][105692] Updated weights for policy 0, policy_version 1305850 (0.0007) [2023-12-27 00:53:53,476][105620] Updated weights for policy 1, policy_version 1307409 (0.0008) [2023-12-27 00:53:53,477][105692] Updated weights for policy 0, policy_version 1305860 (0.0008) [2023-12-27 00:53:53,530][105692] Updated weights for policy 0, policy_version 1305870 (0.0007) [2023-12-27 00:53:53,536][105620] Updated weights for policy 1, policy_version 1307419 (0.0008) [2023-12-27 00:53:54,264][105692] Updated weights for policy 0, policy_version 1305880 (0.0005) [2023-12-27 00:53:54,274][105620] Updated weights for policy 1, policy_version 1307429 (0.0008) [2023-12-27 00:53:54,330][105692] Updated weights for policy 0, policy_version 1305890 (0.0006) [2023-12-27 00:53:54,340][105620] Updated weights for policy 1, policy_version 1307439 (0.0009) [2023-12-27 00:53:54,392][105692] Updated weights for policy 0, policy_version 1305900 (0.0005) [2023-12-27 00:53:54,397][105620] Updated weights for policy 1, policy_version 1307449 (0.0009) [2023-12-27 00:53:55,027][105692] Updated weights for policy 0, policy_version 1305910 (0.0008) [2023-12-27 00:53:55,057][105620] Updated weights for policy 1, policy_version 1307459 (0.0007) [2023-12-27 00:53:55,083][105692] Updated weights for policy 0, policy_version 1305920 (0.0009) [2023-12-27 00:53:55,117][105620] Updated weights for policy 1, policy_version 1307469 (0.0007) [2023-12-27 00:53:55,136][105692] Updated weights for policy 0, policy_version 1305930 (0.0009) [2023-12-27 00:53:55,166][105620] Updated weights for policy 1, policy_version 1307479 (0.0006) [2023-12-27 00:53:55,736][105692] Updated weights for policy 0, policy_version 1305940 (0.0009) [2023-12-27 00:53:55,785][105692] Updated weights for policy 0, policy_version 1305950 (0.0008) [2023-12-27 00:53:55,808][105620] Updated weights for policy 1, policy_version 1307489 (0.0005) [2023-12-27 00:53:55,845][105692] Updated weights for policy 0, policy_version 1305960 (0.0008) [2023-12-27 00:53:55,859][105620] Updated weights for policy 1, policy_version 1307499 (0.0006) [2023-12-27 00:53:55,916][105620] Updated weights for policy 1, policy_version 1307509 (0.0005) [2023-12-27 00:53:55,977][105620] Updated weights for policy 1, policy_version 1307519 (0.0005) [2023-12-27 00:53:56,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 669147136. Throughput: 0: 9946.9, 1: 9656.6. Samples: 669150200. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:53:56,063][104569] Avg episode reward: [(0, '8356.424'), (1, '8903.442')] [2023-12-27 00:53:56,541][105692] Updated weights for policy 0, policy_version 1305970 (0.0009) [2023-12-27 00:53:56,594][105692] Updated weights for policy 0, policy_version 1305980 (0.0008) [2023-12-27 00:53:56,648][105692] Updated weights for policy 0, policy_version 1305990 (0.0007) [2023-12-27 00:53:56,695][105692] Updated weights for policy 0, policy_version 1306000 (0.0005) [2023-12-27 00:53:56,721][105620] Updated weights for policy 1, policy_version 1307529 (0.0009) [2023-12-27 00:53:56,773][105620] Updated weights for policy 1, policy_version 1307539 (0.0006) [2023-12-27 00:53:56,830][105620] Updated weights for policy 1, policy_version 1307549 (0.0005) [2023-12-27 00:53:57,409][105620] Updated weights for policy 1, policy_version 1307559 (0.0008) [2023-12-27 00:53:57,464][105620] Updated weights for policy 1, policy_version 1307569 (0.0008) [2023-12-27 00:53:57,495][105692] Updated weights for policy 0, policy_version 1306010 (0.0007) [2023-12-27 00:53:57,513][105620] Updated weights for policy 1, policy_version 1307579 (0.0008) [2023-12-27 00:53:57,543][105692] Updated weights for policy 0, policy_version 1306020 (0.0007) [2023-12-27 00:53:57,594][105692] Updated weights for policy 0, policy_version 1306030 (0.0009) [2023-12-27 00:53:58,162][105620] Updated weights for policy 1, policy_version 1307589 (0.0008) [2023-12-27 00:53:58,229][105620] Updated weights for policy 1, policy_version 1307599 (0.0009) [2023-12-27 00:53:58,290][105620] Updated weights for policy 1, policy_version 1307609 (0.0010) [2023-12-27 00:53:58,432][105692] Updated weights for policy 0, policy_version 1306040 (0.0009) [2023-12-27 00:53:58,490][105692] Updated weights for policy 0, policy_version 1306050 (0.0008) [2023-12-27 00:53:58,553][105692] Updated weights for policy 0, policy_version 1306060 (0.0009) [2023-12-27 00:53:59,086][105620] Updated weights for policy 1, policy_version 1307619 (0.0010) [2023-12-27 00:53:59,153][105620] Updated weights for policy 1, policy_version 1307629 (0.0008) [2023-12-27 00:53:59,222][105620] Updated weights for policy 1, policy_version 1307639 (0.0008) [2023-12-27 00:53:59,383][105692] Updated weights for policy 0, policy_version 1306070 (0.0010) [2023-12-27 00:53:59,445][105692] Updated weights for policy 0, policy_version 1306080 (0.0008) [2023-12-27 00:53:59,508][105692] Updated weights for policy 0, policy_version 1306090 (0.0009) [2023-12-27 00:53:59,841][105620] Updated weights for policy 1, policy_version 1307649 (0.0007) [2023-12-27 00:53:59,903][105620] Updated weights for policy 1, policy_version 1307659 (0.0009) [2023-12-27 00:53:59,969][105620] Updated weights for policy 1, policy_version 1307669 (0.0007) [2023-12-27 00:54:00,035][105620] Updated weights for policy 1, policy_version 1307679 (0.0008) [2023-12-27 00:54:00,237][105692] Updated weights for policy 0, policy_version 1306100 (0.0007) [2023-12-27 00:54:00,289][105692] Updated weights for policy 0, policy_version 1306110 (0.0009) [2023-12-27 00:54:00,346][105692] Updated weights for policy 0, policy_version 1306120 (0.0010) [2023-12-27 00:54:00,653][105620] Updated weights for policy 1, policy_version 1307689 (0.0008) [2023-12-27 00:54:00,700][105620] Updated weights for policy 1, policy_version 1307699 (0.0008) [2023-12-27 00:54:00,760][105620] Updated weights for policy 1, policy_version 1307709 (0.0009) [2023-12-27 00:54:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 669237248. Throughput: 0: 9958.1, 1: 9660.7. Samples: 669208272. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:54:01,062][104569] Avg episode reward: [(0, '8538.739'), (1, '8992.731')] [2023-12-27 00:54:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001307712_334815232.pth... [2023-12-27 00:54:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001306128_334422016.pth... [2023-12-27 00:54:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001304976_334127104.pth [2023-12-27 00:54:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001306592_334528512.pth [2023-12-27 00:54:01,159][105692] Updated weights for policy 0, policy_version 1306130 (0.0009) [2023-12-27 00:54:01,209][105692] Updated weights for policy 0, policy_version 1306140 (0.0006) [2023-12-27 00:54:01,273][105692] Updated weights for policy 0, policy_version 1306150 (0.0008) [2023-12-27 00:54:01,332][105692] Updated weights for policy 0, policy_version 1306160 (0.0009) [2023-12-27 00:54:01,494][105620] Updated weights for policy 1, policy_version 1307719 (0.0009) [2023-12-27 00:54:01,548][105620] Updated weights for policy 1, policy_version 1307729 (0.0009) [2023-12-27 00:54:01,612][105620] Updated weights for policy 1, policy_version 1307739 (0.0010) [2023-12-27 00:54:02,000][105692] Updated weights for policy 0, policy_version 1306170 (0.0005) [2023-12-27 00:54:02,053][105692] Updated weights for policy 0, policy_version 1306180 (0.0005) [2023-12-27 00:54:02,107][105692] Updated weights for policy 0, policy_version 1306190 (0.0006) [2023-12-27 00:54:02,290][105620] Updated weights for policy 1, policy_version 1307749 (0.0007) [2023-12-27 00:54:02,357][105620] Updated weights for policy 1, policy_version 1307759 (0.0008) [2023-12-27 00:54:02,417][105620] Updated weights for policy 1, policy_version 1307769 (0.0009) [2023-12-27 00:54:02,779][105692] Updated weights for policy 0, policy_version 1306200 (0.0009) [2023-12-27 00:54:02,832][105692] Updated weights for policy 0, policy_version 1306210 (0.0010) [2023-12-27 00:54:02,890][105692] Updated weights for policy 0, policy_version 1306220 (0.0010) [2023-12-27 00:54:03,010][105620] Updated weights for policy 1, policy_version 1307779 (0.0008) [2023-12-27 00:54:03,067][105620] Updated weights for policy 1, policy_version 1307789 (0.0005) [2023-12-27 00:54:03,119][105620] Updated weights for policy 1, policy_version 1307799 (0.0009) [2023-12-27 00:54:03,697][105692] Updated weights for policy 0, policy_version 1306230 (0.0010) [2023-12-27 00:54:03,750][105692] Updated weights for policy 0, policy_version 1306241 (0.0010) [2023-12-27 00:54:03,783][105620] Updated weights for policy 1, policy_version 1307809 (0.0010) [2023-12-27 00:54:03,808][105692] Updated weights for policy 0, policy_version 1306251 (0.0009) [2023-12-27 00:54:03,836][105620] Updated weights for policy 1, policy_version 1307819 (0.0008) [2023-12-27 00:54:03,892][105620] Updated weights for policy 1, policy_version 1307829 (0.0009) [2023-12-27 00:54:03,943][105620] Updated weights for policy 1, policy_version 1307839 (0.0009) [2023-12-27 00:54:04,614][105692] Updated weights for policy 0, policy_version 1306261 (0.0008) [2023-12-27 00:54:04,684][105692] Updated weights for policy 0, policy_version 1306271 (0.0006) [2023-12-27 00:54:04,748][105692] Updated weights for policy 0, policy_version 1306281 (0.0009) [2023-12-27 00:54:04,761][105620] Updated weights for policy 1, policy_version 1307849 (0.0007) [2023-12-27 00:54:04,824][105620] Updated weights for policy 1, policy_version 1307859 (0.0008) [2023-12-27 00:54:04,883][105620] Updated weights for policy 1, policy_version 1307869 (0.0010) [2023-12-27 00:54:05,322][105692] Updated weights for policy 0, policy_version 1306291 (0.0009) [2023-12-27 00:54:05,384][105692] Updated weights for policy 0, policy_version 1306301 (0.0010) [2023-12-27 00:54:05,438][105692] Updated weights for policy 0, policy_version 1306311 (0.0009) [2023-12-27 00:54:05,592][105620] Updated weights for policy 1, policy_version 1307879 (0.0007) [2023-12-27 00:54:05,652][105620] Updated weights for policy 1, policy_version 1307889 (0.0005) [2023-12-27 00:54:05,714][105620] Updated weights for policy 1, policy_version 1307899 (0.0008) [2023-12-27 00:54:06,032][105692] Updated weights for policy 0, policy_version 1306321 (0.0005) [2023-12-27 00:54:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 669335552. Throughput: 0: 9826.4, 1: 9748.7. Samples: 669324200. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:54:06,063][104569] Avg episode reward: [(0, '8274.253'), (1, '9078.925')] [2023-12-27 00:54:06,085][105692] Updated weights for policy 0, policy_version 1306331 (0.0005) [2023-12-27 00:54:06,151][105692] Updated weights for policy 0, policy_version 1306341 (0.0007) [2023-12-27 00:54:06,210][105692] Updated weights for policy 0, policy_version 1306351 (0.0006) [2023-12-27 00:54:06,479][105620] Updated weights for policy 1, policy_version 1307909 (0.0009) [2023-12-27 00:54:06,532][105620] Updated weights for policy 1, policy_version 1307919 (0.0010) [2023-12-27 00:54:06,590][105620] Updated weights for policy 1, policy_version 1307929 (0.0009) [2023-12-27 00:54:06,881][105692] Updated weights for policy 0, policy_version 1306361 (0.0008) [2023-12-27 00:54:06,934][105692] Updated weights for policy 0, policy_version 1306371 (0.0010) [2023-12-27 00:54:06,986][105692] Updated weights for policy 0, policy_version 1306381 (0.0009) [2023-12-27 00:54:07,281][105620] Updated weights for policy 1, policy_version 1307939 (0.0009) [2023-12-27 00:54:07,342][105620] Updated weights for policy 1, policy_version 1307949 (0.0009) [2023-12-27 00:54:07,404][105620] Updated weights for policy 1, policy_version 1307959 (0.0010) [2023-12-27 00:54:07,694][105692] Updated weights for policy 0, policy_version 1306391 (0.0010) [2023-12-27 00:54:07,749][105692] Updated weights for policy 0, policy_version 1306401 (0.0011) [2023-12-27 00:54:07,798][105692] Updated weights for policy 0, policy_version 1306411 (0.0011) [2023-12-27 00:54:08,033][105620] Updated weights for policy 1, policy_version 1307969 (0.0009) [2023-12-27 00:54:08,097][105620] Updated weights for policy 1, policy_version 1307979 (0.0007) [2023-12-27 00:54:08,161][105620] Updated weights for policy 1, policy_version 1307989 (0.0005) [2023-12-27 00:54:08,222][105620] Updated weights for policy 1, policy_version 1307999 (0.0006) [2023-12-27 00:54:08,540][105692] Updated weights for policy 0, policy_version 1306421 (0.0011) [2023-12-27 00:54:08,589][105692] Updated weights for policy 0, policy_version 1306431 (0.0010) [2023-12-27 00:54:08,651][105692] Updated weights for policy 0, policy_version 1306441 (0.0011) [2023-12-27 00:54:08,807][105620] Updated weights for policy 1, policy_version 1308009 (0.0010) [2023-12-27 00:54:08,870][105620] Updated weights for policy 1, policy_version 1308019 (0.0008) [2023-12-27 00:54:08,927][105620] Updated weights for policy 1, policy_version 1308029 (0.0006) [2023-12-27 00:54:09,301][105692] Updated weights for policy 0, policy_version 1306451 (0.0010) [2023-12-27 00:54:09,373][105692] Updated weights for policy 0, policy_version 1306462 (0.0009) [2023-12-27 00:54:09,440][105692] Updated weights for policy 0, policy_version 1306472 (0.0010) [2023-12-27 00:54:09,571][105620] Updated weights for policy 1, policy_version 1308039 (0.0009) [2023-12-27 00:54:09,632][105620] Updated weights for policy 1, policy_version 1308049 (0.0011) [2023-12-27 00:54:09,688][105620] Updated weights for policy 1, policy_version 1308059 (0.0011) [2023-12-27 00:54:10,240][105692] Updated weights for policy 0, policy_version 1306482 (0.0011) [2023-12-27 00:54:10,300][105692] Updated weights for policy 0, policy_version 1306492 (0.0010) [2023-12-27 00:54:10,357][105692] Updated weights for policy 0, policy_version 1306502 (0.0010) [2023-12-27 00:54:10,411][105692] Updated weights for policy 0, policy_version 1306512 (0.0009) [2023-12-27 00:54:10,435][105620] Updated weights for policy 1, policy_version 1308069 (0.0011) [2023-12-27 00:54:10,494][105620] Updated weights for policy 1, policy_version 1308079 (0.0011) [2023-12-27 00:54:10,551][105620] Updated weights for policy 1, policy_version 1308089 (0.0011) [2023-12-27 00:54:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 669433856. Throughput: 0: 9980.3, 1: 9783.9. Samples: 669445816. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:54:11,062][104569] Avg episode reward: [(0, '7911.859'), (1, '8996.834')] [2023-12-27 00:54:11,117][105692] Updated weights for policy 0, policy_version 1306522 (0.0007) [2023-12-27 00:54:11,181][105692] Updated weights for policy 0, policy_version 1306532 (0.0007) [2023-12-27 00:54:11,237][105692] Updated weights for policy 0, policy_version 1306542 (0.0008) [2023-12-27 00:54:11,334][105620] Updated weights for policy 1, policy_version 1308099 (0.0010) [2023-12-27 00:54:11,400][105620] Updated weights for policy 1, policy_version 1308109 (0.0006) [2023-12-27 00:54:11,456][105620] Updated weights for policy 1, policy_version 1308119 (0.0006) [2023-12-27 00:54:12,069][105692] Updated weights for policy 0, policy_version 1306552 (0.0008) [2023-12-27 00:54:12,136][105692] Updated weights for policy 0, policy_version 1306562 (0.0008) [2023-12-27 00:54:12,155][105620] Updated weights for policy 1, policy_version 1308129 (0.0006) [2023-12-27 00:54:12,197][105692] Updated weights for policy 0, policy_version 1306572 (0.0007) [2023-12-27 00:54:12,211][105620] Updated weights for policy 1, policy_version 1308139 (0.0011) [2023-12-27 00:54:12,278][105620] Updated weights for policy 1, policy_version 1308149 (0.0011) [2023-12-27 00:54:12,338][105620] Updated weights for policy 1, policy_version 1308159 (0.0009) [2023-12-27 00:54:12,977][105692] Updated weights for policy 0, policy_version 1306582 (0.0007) [2023-12-27 00:54:13,031][105692] Updated weights for policy 0, policy_version 1306592 (0.0008) [2023-12-27 00:54:13,051][105620] Updated weights for policy 1, policy_version 1308169 (0.0011) [2023-12-27 00:54:13,088][105692] Updated weights for policy 0, policy_version 1306602 (0.0007) [2023-12-27 00:54:13,117][105620] Updated weights for policy 1, policy_version 1308179 (0.0011) [2023-12-27 00:54:13,175][105620] Updated weights for policy 1, policy_version 1308189 (0.0011) [2023-12-27 00:54:13,804][105620] Updated weights for policy 1, policy_version 1308199 (0.0010) [2023-12-27 00:54:13,858][105620] Updated weights for policy 1, policy_version 1308209 (0.0005) [2023-12-27 00:54:13,894][105692] Updated weights for policy 0, policy_version 1306612 (0.0006) [2023-12-27 00:54:13,914][105620] Updated weights for policy 1, policy_version 1308219 (0.0010) [2023-12-27 00:54:13,952][105692] Updated weights for policy 0, policy_version 1306622 (0.0007) [2023-12-27 00:54:14,012][105692] Updated weights for policy 0, policy_version 1306632 (0.0008) [2023-12-27 00:54:14,506][105620] Updated weights for policy 1, policy_version 1308229 (0.0009) [2023-12-27 00:54:14,564][105620] Updated weights for policy 1, policy_version 1308239 (0.0010) [2023-12-27 00:54:14,614][105620] Updated weights for policy 1, policy_version 1308249 (0.0006) [2023-12-27 00:54:14,809][105692] Updated weights for policy 0, policy_version 1306642 (0.0009) [2023-12-27 00:54:14,867][105692] Updated weights for policy 0, policy_version 1306652 (0.0010) [2023-12-27 00:54:14,922][105692] Updated weights for policy 0, policy_version 1306662 (0.0010) [2023-12-27 00:54:14,979][105692] Updated weights for policy 0, policy_version 1306672 (0.0010) [2023-12-27 00:54:15,264][105620] Updated weights for policy 1, policy_version 1308259 (0.0008) [2023-12-27 00:54:15,320][105620] Updated weights for policy 1, policy_version 1308269 (0.0011) [2023-12-27 00:54:15,380][105620] Updated weights for policy 1, policy_version 1308279 (0.0010) [2023-12-27 00:54:15,791][105692] Updated weights for policy 0, policy_version 1306682 (0.0008) [2023-12-27 00:54:15,840][105692] Updated weights for policy 0, policy_version 1306692 (0.0008) [2023-12-27 00:54:15,888][105692] Updated weights for policy 0, policy_version 1306702 (0.0008) [2023-12-27 00:54:16,051][105620] Updated weights for policy 1, policy_version 1308289 (0.0010) [2023-12-27 00:54:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 669532160. Throughput: 0: 9899.7, 1: 9751.7. Samples: 669502708. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:54:16,062][104569] Avg episode reward: [(0, '8366.053'), (1, '8996.655')] [2023-12-27 00:54:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001306704_334569472.pth... [2023-12-27 00:54:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001305584_334282752.pth [2023-12-27 00:54:16,110][105620] Updated weights for policy 1, policy_version 1308299 (0.0005) [2023-12-27 00:54:16,160][105620] Updated weights for policy 1, policy_version 1308309 (0.0005) [2023-12-27 00:54:16,209][105620] Updated weights for policy 1, policy_version 1308319 (0.0005) [2023-12-27 00:54:16,214][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001308320_334970880.pth... [2023-12-27 00:54:16,217][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001307136_334667776.pth [2023-12-27 00:54:16,715][105692] Updated weights for policy 0, policy_version 1306712 (0.0007) [2023-12-27 00:54:16,773][105692] Updated weights for policy 0, policy_version 1306722 (0.0005) [2023-12-27 00:54:16,827][105692] Updated weights for policy 0, policy_version 1306732 (0.0005) [2023-12-27 00:54:16,903][105620] Updated weights for policy 1, policy_version 1308329 (0.0008) [2023-12-27 00:54:16,954][105620] Updated weights for policy 1, policy_version 1308339 (0.0006) [2023-12-27 00:54:17,007][105620] Updated weights for policy 1, policy_version 1308349 (0.0005) [2023-12-27 00:54:17,441][105692] Updated weights for policy 0, policy_version 1306742 (0.0005) [2023-12-27 00:54:17,493][105692] Updated weights for policy 0, policy_version 1306752 (0.0006) [2023-12-27 00:54:17,541][105692] Updated weights for policy 0, policy_version 1306762 (0.0007) [2023-12-27 00:54:17,643][105620] Updated weights for policy 1, policy_version 1308359 (0.0009) [2023-12-27 00:54:17,707][105620] Updated weights for policy 1, policy_version 1308369 (0.0010) [2023-12-27 00:54:17,756][105620] Updated weights for policy 1, policy_version 1308379 (0.0010) [2023-12-27 00:54:18,202][105692] Updated weights for policy 0, policy_version 1306772 (0.0008) [2023-12-27 00:54:18,272][105692] Updated weights for policy 0, policy_version 1306782 (0.0009) [2023-12-27 00:54:18,334][105692] Updated weights for policy 0, policy_version 1306792 (0.0009) [2023-12-27 00:54:18,487][105620] Updated weights for policy 1, policy_version 1308389 (0.0008) [2023-12-27 00:54:18,545][105620] Updated weights for policy 1, policy_version 1308399 (0.0006) [2023-12-27 00:54:18,603][105620] Updated weights for policy 1, policy_version 1308409 (0.0005) [2023-12-27 00:54:19,148][105692] Updated weights for policy 0, policy_version 1306802 (0.0009) [2023-12-27 00:54:19,199][105692] Updated weights for policy 0, policy_version 1306812 (0.0008) [2023-12-27 00:54:19,255][105692] Updated weights for policy 0, policy_version 1306822 (0.0008) [2023-12-27 00:54:19,257][105620] Updated weights for policy 1, policy_version 1308419 (0.0006) [2023-12-27 00:54:19,312][105692] Updated weights for policy 0, policy_version 1306832 (0.0008) [2023-12-27 00:54:19,314][105620] Updated weights for policy 1, policy_version 1308429 (0.0006) [2023-12-27 00:54:19,385][105620] Updated weights for policy 1, policy_version 1308439 (0.0009) [2023-12-27 00:54:20,132][105620] Updated weights for policy 1, policy_version 1308449 (0.0008) [2023-12-27 00:54:20,142][105692] Updated weights for policy 0, policy_version 1306842 (0.0008) [2023-12-27 00:54:20,193][105620] Updated weights for policy 1, policy_version 1308459 (0.0007) [2023-12-27 00:54:20,203][105692] Updated weights for policy 0, policy_version 1306852 (0.0006) [2023-12-27 00:54:20,250][105620] Updated weights for policy 1, policy_version 1308469 (0.0006) [2023-12-27 00:54:20,266][105692] Updated weights for policy 0, policy_version 1306862 (0.0007) [2023-12-27 00:54:20,316][105620] Updated weights for policy 1, policy_version 1308479 (0.0006) [2023-12-27 00:54:21,048][105692] Updated weights for policy 0, policy_version 1306872 (0.0008) [2023-12-27 00:54:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 669622272. Throughput: 0: 9716.2, 1: 9873.3. Samples: 669619604. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 00:54:21,063][104569] Avg episode reward: [(0, '8458.249'), (1, '8990.872')] [2023-12-27 00:54:21,066][105620] Updated weights for policy 1, policy_version 1308489 (0.0008) [2023-12-27 00:54:21,115][105692] Updated weights for policy 0, policy_version 1306882 (0.0007) [2023-12-27 00:54:21,122][105620] Updated weights for policy 1, policy_version 1308499 (0.0007) [2023-12-27 00:54:21,179][105620] Updated weights for policy 1, policy_version 1308509 (0.0007) [2023-12-27 00:54:21,180][105692] Updated weights for policy 0, policy_version 1306892 (0.0008) [2023-12-27 00:54:21,926][105620] Updated weights for policy 1, policy_version 1308519 (0.0008) [2023-12-27 00:54:21,937][105692] Updated weights for policy 0, policy_version 1306902 (0.0006) [2023-12-27 00:54:21,987][105692] Updated weights for policy 0, policy_version 1306912 (0.0009) [2023-12-27 00:54:21,990][105620] Updated weights for policy 1, policy_version 1308529 (0.0007) [2023-12-27 00:54:22,037][105692] Updated weights for policy 0, policy_version 1306922 (0.0006) [2023-12-27 00:54:22,053][105620] Updated weights for policy 1, policy_version 1308539 (0.0008) [2023-12-27 00:54:22,763][105692] Updated weights for policy 0, policy_version 1306932 (0.0009) [2023-12-27 00:54:22,832][105692] Updated weights for policy 0, policy_version 1306942 (0.0007) [2023-12-27 00:54:22,833][105620] Updated weights for policy 1, policy_version 1308549 (0.0008) [2023-12-27 00:54:22,883][105620] Updated weights for policy 1, policy_version 1308559 (0.0007) [2023-12-27 00:54:22,893][105692] Updated weights for policy 0, policy_version 1306952 (0.0009) [2023-12-27 00:54:22,943][105620] Updated weights for policy 1, policy_version 1308569 (0.0008) [2023-12-27 00:54:23,547][105692] Updated weights for policy 0, policy_version 1306962 (0.0006) [2023-12-27 00:54:23,601][105692] Updated weights for policy 0, policy_version 1306972 (0.0006) [2023-12-27 00:54:23,651][105692] Updated weights for policy 0, policy_version 1306982 (0.0005) [2023-12-27 00:54:23,701][105692] Updated weights for policy 0, policy_version 1306992 (0.0007) [2023-12-27 00:54:23,719][105620] Updated weights for policy 1, policy_version 1308579 (0.0009) [2023-12-27 00:54:23,769][105620] Updated weights for policy 1, policy_version 1308589 (0.0008) [2023-12-27 00:54:23,829][105620] Updated weights for policy 1, policy_version 1308599 (0.0009) [2023-12-27 00:54:24,419][105692] Updated weights for policy 0, policy_version 1307002 (0.0009) [2023-12-27 00:54:24,471][105692] Updated weights for policy 0, policy_version 1307012 (0.0009) [2023-12-27 00:54:24,530][105692] Updated weights for policy 0, policy_version 1307022 (0.0009) [2023-12-27 00:54:24,581][105620] Updated weights for policy 1, policy_version 1308609 (0.0009) [2023-12-27 00:54:24,631][105620] Updated weights for policy 1, policy_version 1308619 (0.0009) [2023-12-27 00:54:24,685][105620] Updated weights for policy 1, policy_version 1308629 (0.0009) [2023-12-27 00:54:24,738][105620] Updated weights for policy 1, policy_version 1308639 (0.0009) [2023-12-27 00:54:25,189][105692] Updated weights for policy 0, policy_version 1307032 (0.0006) [2023-12-27 00:54:25,251][105692] Updated weights for policy 0, policy_version 1307042 (0.0005) [2023-12-27 00:54:25,308][105692] Updated weights for policy 0, policy_version 1307052 (0.0005) [2023-12-27 00:54:25,619][105620] Updated weights for policy 1, policy_version 1308649 (0.0010) [2023-12-27 00:54:25,672][105620] Updated weights for policy 1, policy_version 1308659 (0.0009) [2023-12-27 00:54:25,724][105620] Updated weights for policy 1, policy_version 1308669 (0.0010) [2023-12-27 00:54:25,852][105692] Updated weights for policy 0, policy_version 1307062 (0.0005) [2023-12-27 00:54:25,906][105692] Updated weights for policy 0, policy_version 1307072 (0.0007) [2023-12-27 00:54:25,953][105692] Updated weights for policy 0, policy_version 1307082 (0.0009) [2023-12-27 00:54:26,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 669728768. Throughput: 0: 9642.4, 1: 9781.6. Samples: 669731824. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:54:26,063][104569] Avg episode reward: [(0, '8365.442'), (1, '8990.487')] [2023-12-27 00:54:26,517][105620] Updated weights for policy 1, policy_version 1308679 (0.0009) [2023-12-27 00:54:26,572][105620] Updated weights for policy 1, policy_version 1308689 (0.0009) [2023-12-27 00:54:26,621][105620] Updated weights for policy 1, policy_version 1308699 (0.0009) [2023-12-27 00:54:26,668][105692] Updated weights for policy 0, policy_version 1307092 (0.0007) [2023-12-27 00:54:26,719][105692] Updated weights for policy 0, policy_version 1307102 (0.0005) [2023-12-27 00:54:26,777][105692] Updated weights for policy 0, policy_version 1307112 (0.0005) [2023-12-27 00:54:27,351][105620] Updated weights for policy 1, policy_version 1308709 (0.0008) [2023-12-27 00:54:27,398][105620] Updated weights for policy 1, policy_version 1308719 (0.0009) [2023-12-27 00:54:27,452][105620] Updated weights for policy 1, policy_version 1308729 (0.0009) [2023-12-27 00:54:27,464][105692] Updated weights for policy 0, policy_version 1307122 (0.0006) [2023-12-27 00:54:27,515][105692] Updated weights for policy 0, policy_version 1307132 (0.0008) [2023-12-27 00:54:27,571][105692] Updated weights for policy 0, policy_version 1307143 (0.0010) [2023-12-27 00:54:28,175][105620] Updated weights for policy 1, policy_version 1308739 (0.0007) [2023-12-27 00:54:28,243][105620] Updated weights for policy 1, policy_version 1308749 (0.0009) [2023-12-27 00:54:28,301][105620] Updated weights for policy 1, policy_version 1308759 (0.0009) [2023-12-27 00:54:28,351][105692] Updated weights for policy 0, policy_version 1307154 (0.0010) [2023-12-27 00:54:28,400][105692] Updated weights for policy 0, policy_version 1307164 (0.0009) [2023-12-27 00:54:28,454][105692] Updated weights for policy 0, policy_version 1307174 (0.0009) [2023-12-27 00:54:28,510][105692] Updated weights for policy 0, policy_version 1307184 (0.0009) [2023-12-27 00:54:29,118][105620] Updated weights for policy 1, policy_version 1308769 (0.0008) [2023-12-27 00:54:29,165][105692] Updated weights for policy 0, policy_version 1307194 (0.0007) [2023-12-27 00:54:29,178][105620] Updated weights for policy 1, policy_version 1308779 (0.0009) [2023-12-27 00:54:29,232][105692] Updated weights for policy 0, policy_version 1307204 (0.0006) [2023-12-27 00:54:29,241][105620] Updated weights for policy 1, policy_version 1308789 (0.0009) [2023-12-27 00:54:29,296][105692] Updated weights for policy 0, policy_version 1307214 (0.0006) [2023-12-27 00:54:29,298][105620] Updated weights for policy 1, policy_version 1308799 (0.0008) [2023-12-27 00:54:29,993][105692] Updated weights for policy 0, policy_version 1307224 (0.0008) [2023-12-27 00:54:30,053][105620] Updated weights for policy 1, policy_version 1308809 (0.0011) [2023-12-27 00:54:30,055][105692] Updated weights for policy 0, policy_version 1307234 (0.0006) [2023-12-27 00:54:30,112][105620] Updated weights for policy 1, policy_version 1308819 (0.0010) [2023-12-27 00:54:30,113][105692] Updated weights for policy 0, policy_version 1307244 (0.0008) [2023-12-27 00:54:30,168][105620] Updated weights for policy 1, policy_version 1308829 (0.0010) [2023-12-27 00:54:30,792][105620] Updated weights for policy 1, policy_version 1308839 (0.0007) [2023-12-27 00:54:30,847][105620] Updated weights for policy 1, policy_version 1308849 (0.0005) [2023-12-27 00:54:30,889][105692] Updated weights for policy 0, policy_version 1307254 (0.0008) [2023-12-27 00:54:30,905][105620] Updated weights for policy 1, policy_version 1308859 (0.0007) [2023-12-27 00:54:30,946][105692] Updated weights for policy 0, policy_version 1307264 (0.0007) [2023-12-27 00:54:31,011][105692] Updated weights for policy 0, policy_version 1307274 (0.0005) [2023-12-27 00:54:31,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 669827072. Throughput: 0: 9656.2, 1: 9757.0. Samples: 669790096. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:54:31,063][104569] Avg episode reward: [(0, '8727.431'), (1, '9080.981')] [2023-12-27 00:54:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001307280_334716928.pth... [2023-12-27 00:54:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001308864_335110144.pth... [2023-12-27 00:54:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001306128_334422016.pth [2023-12-27 00:54:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001307712_334815232.pth [2023-12-27 00:54:31,610][105620] Updated weights for policy 1, policy_version 1308869 (0.0010) [2023-12-27 00:54:31,672][105620] Updated weights for policy 1, policy_version 1308879 (0.0007) [2023-12-27 00:54:31,729][105692] Updated weights for policy 0, policy_version 1307284 (0.0007) [2023-12-27 00:54:31,734][105620] Updated weights for policy 1, policy_version 1308889 (0.0008) [2023-12-27 00:54:31,778][105692] Updated weights for policy 0, policy_version 1307294 (0.0007) [2023-12-27 00:54:31,829][105692] Updated weights for policy 0, policy_version 1307304 (0.0008) [2023-12-27 00:54:32,418][105620] Updated weights for policy 1, policy_version 1308899 (0.0009) [2023-12-27 00:54:32,478][105620] Updated weights for policy 1, policy_version 1308909 (0.0009) [2023-12-27 00:54:32,513][105692] Updated weights for policy 0, policy_version 1307314 (0.0009) [2023-12-27 00:54:32,525][105620] Updated weights for policy 1, policy_version 1308919 (0.0008) [2023-12-27 00:54:32,572][105692] Updated weights for policy 0, policy_version 1307324 (0.0009) [2023-12-27 00:54:32,636][105692] Updated weights for policy 0, policy_version 1307334 (0.0010) [2023-12-27 00:54:32,695][105692] Updated weights for policy 0, policy_version 1307344 (0.0010) [2023-12-27 00:54:33,299][105620] Updated weights for policy 1, policy_version 1308929 (0.0007) [2023-12-27 00:54:33,318][105692] Updated weights for policy 0, policy_version 1307354 (0.0008) [2023-12-27 00:54:33,357][105620] Updated weights for policy 1, policy_version 1308939 (0.0005) [2023-12-27 00:54:33,370][105692] Updated weights for policy 0, policy_version 1307364 (0.0008) [2023-12-27 00:54:33,412][105620] Updated weights for policy 1, policy_version 1308949 (0.0005) [2023-12-27 00:54:33,423][105692] Updated weights for policy 0, policy_version 1307374 (0.0008) [2023-12-27 00:54:33,465][105620] Updated weights for policy 1, policy_version 1308959 (0.0005) [2023-12-27 00:54:34,095][105692] Updated weights for policy 0, policy_version 1307384 (0.0008) [2023-12-27 00:54:34,160][105692] Updated weights for policy 0, policy_version 1307394 (0.0008) [2023-12-27 00:54:34,177][105620] Updated weights for policy 1, policy_version 1308969 (0.0007) [2023-12-27 00:54:34,220][105692] Updated weights for policy 0, policy_version 1307404 (0.0006) [2023-12-27 00:54:34,241][105620] Updated weights for policy 1, policy_version 1308979 (0.0009) [2023-12-27 00:54:34,303][105620] Updated weights for policy 1, policy_version 1308989 (0.0009) [2023-12-27 00:54:34,995][105692] Updated weights for policy 0, policy_version 1307414 (0.0008) [2023-12-27 00:54:35,027][105620] Updated weights for policy 1, policy_version 1308999 (0.0009) [2023-12-27 00:54:35,053][105692] Updated weights for policy 0, policy_version 1307424 (0.0006) [2023-12-27 00:54:35,085][105620] Updated weights for policy 1, policy_version 1309009 (0.0009) [2023-12-27 00:54:35,113][105692] Updated weights for policy 0, policy_version 1307434 (0.0010) [2023-12-27 00:54:35,142][105620] Updated weights for policy 1, policy_version 1309019 (0.0011) [2023-12-27 00:54:35,796][105620] Updated weights for policy 1, policy_version 1309029 (0.0008) [2023-12-27 00:54:35,850][105620] Updated weights for policy 1, policy_version 1309039 (0.0005) [2023-12-27 00:54:35,900][105692] Updated weights for policy 0, policy_version 1307444 (0.0008) [2023-12-27 00:54:35,909][105620] Updated weights for policy 1, policy_version 1309049 (0.0005) [2023-12-27 00:54:35,956][105692] Updated weights for policy 0, policy_version 1307454 (0.0009) [2023-12-27 00:54:36,027][105692] Updated weights for policy 0, policy_version 1307464 (0.0010) [2023-12-27 00:54:36,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 669917184. Throughput: 0: 9648.7, 1: 9815.6. Samples: 669908232. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:54:36,063][104569] Avg episode reward: [(0, '8996.113'), (1, '9082.538')] [2023-12-27 00:54:36,460][105620] Updated weights for policy 1, policy_version 1309059 (0.0007) [2023-12-27 00:54:36,523][105620] Updated weights for policy 1, policy_version 1309069 (0.0009) [2023-12-27 00:54:36,584][105620] Updated weights for policy 1, policy_version 1309079 (0.0009) [2023-12-27 00:54:36,907][105692] Updated weights for policy 0, policy_version 1307474 (0.0010) [2023-12-27 00:54:36,969][105692] Updated weights for policy 0, policy_version 1307484 (0.0009) [2023-12-27 00:54:37,020][105692] Updated weights for policy 0, policy_version 1307494 (0.0009) [2023-12-27 00:54:37,069][105692] Updated weights for policy 0, policy_version 1307504 (0.0009) [2023-12-27 00:54:37,251][105620] Updated weights for policy 1, policy_version 1309089 (0.0009) [2023-12-27 00:54:37,310][105620] Updated weights for policy 1, policy_version 1309099 (0.0009) [2023-12-27 00:54:37,364][105620] Updated weights for policy 1, policy_version 1309109 (0.0009) [2023-12-27 00:54:37,419][105620] Updated weights for policy 1, policy_version 1309119 (0.0009) [2023-12-27 00:54:37,861][105692] Updated weights for policy 0, policy_version 1307514 (0.0009) [2023-12-27 00:54:37,932][105692] Updated weights for policy 0, policy_version 1307524 (0.0008) [2023-12-27 00:54:37,996][105692] Updated weights for policy 0, policy_version 1307534 (0.0010) [2023-12-27 00:54:38,080][105620] Updated weights for policy 1, policy_version 1309129 (0.0006) [2023-12-27 00:54:38,139][105620] Updated weights for policy 1, policy_version 1309139 (0.0005) [2023-12-27 00:54:38,197][105620] Updated weights for policy 1, policy_version 1309149 (0.0005) [2023-12-27 00:54:38,761][105692] Updated weights for policy 0, policy_version 1307545 (0.0010) [2023-12-27 00:54:38,819][105692] Updated weights for policy 0, policy_version 1307555 (0.0008) [2023-12-27 00:54:38,851][105620] Updated weights for policy 1, policy_version 1309159 (0.0009) [2023-12-27 00:54:38,876][105692] Updated weights for policy 0, policy_version 1307565 (0.0006) [2023-12-27 00:54:38,913][105620] Updated weights for policy 1, policy_version 1309169 (0.0007) [2023-12-27 00:54:38,968][105620] Updated weights for policy 1, policy_version 1309179 (0.0010) [2023-12-27 00:54:39,662][105692] Updated weights for policy 0, policy_version 1307575 (0.0008) [2023-12-27 00:54:39,698][105620] Updated weights for policy 1, policy_version 1309189 (0.0009) [2023-12-27 00:54:39,722][105692] Updated weights for policy 0, policy_version 1307585 (0.0008) [2023-12-27 00:54:39,756][105620] Updated weights for policy 1, policy_version 1309199 (0.0006) [2023-12-27 00:54:39,782][105692] Updated weights for policy 0, policy_version 1307595 (0.0007) [2023-12-27 00:54:39,821][105620] Updated weights for policy 1, policy_version 1309209 (0.0007) [2023-12-27 00:54:40,522][105620] Updated weights for policy 1, policy_version 1309219 (0.0009) [2023-12-27 00:54:40,579][105692] Updated weights for policy 0, policy_version 1307605 (0.0007) [2023-12-27 00:54:40,589][105620] Updated weights for policy 1, policy_version 1309229 (0.0008) [2023-12-27 00:54:40,636][105692] Updated weights for policy 0, policy_version 1307615 (0.0006) [2023-12-27 00:54:40,650][105620] Updated weights for policy 1, policy_version 1309239 (0.0007) [2023-12-27 00:54:40,697][105692] Updated weights for policy 0, policy_version 1307625 (0.0006) [2023-12-27 00:54:41,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 670015488. Throughput: 0: 9499.8, 1: 9883.1. Samples: 670022432. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:54:41,062][104569] Avg episode reward: [(0, '8725.810'), (1, '9085.238')] [2023-12-27 00:54:41,374][105692] Updated weights for policy 0, policy_version 1307635 (0.0007) [2023-12-27 00:54:41,436][105692] Updated weights for policy 0, policy_version 1307645 (0.0007) [2023-12-27 00:54:41,473][105620] Updated weights for policy 1, policy_version 1309249 (0.0008) [2023-12-27 00:54:41,492][105692] Updated weights for policy 0, policy_version 1307655 (0.0008) [2023-12-27 00:54:41,535][105620] Updated weights for policy 1, policy_version 1309259 (0.0009) [2023-12-27 00:54:41,600][105620] Updated weights for policy 1, policy_version 1309269 (0.0008) [2023-12-27 00:54:41,667][105620] Updated weights for policy 1, policy_version 1309279 (0.0009) [2023-12-27 00:54:42,219][105692] Updated weights for policy 0, policy_version 1307665 (0.0006) [2023-12-27 00:54:42,282][105692] Updated weights for policy 0, policy_version 1307675 (0.0008) [2023-12-27 00:54:42,350][105692] Updated weights for policy 0, policy_version 1307685 (0.0009) [2023-12-27 00:54:42,423][105692] Updated weights for policy 0, policy_version 1307695 (0.0007) [2023-12-27 00:54:42,477][105620] Updated weights for policy 1, policy_version 1309289 (0.0006) [2023-12-27 00:54:42,541][105620] Updated weights for policy 1, policy_version 1309299 (0.0006) [2023-12-27 00:54:42,600][105620] Updated weights for policy 1, policy_version 1309309 (0.0009) [2023-12-27 00:54:43,106][105692] Updated weights for policy 0, policy_version 1307705 (0.0007) [2023-12-27 00:54:43,164][105692] Updated weights for policy 0, policy_version 1307715 (0.0010) [2023-12-27 00:54:43,191][105620] Updated weights for policy 1, policy_version 1309319 (0.0011) [2023-12-27 00:54:43,229][105692] Updated weights for policy 0, policy_version 1307725 (0.0010) [2023-12-27 00:54:43,246][105620] Updated weights for policy 1, policy_version 1309329 (0.0009) [2023-12-27 00:54:43,308][105620] Updated weights for policy 1, policy_version 1309339 (0.0005) [2023-12-27 00:54:43,816][105692] Updated weights for policy 0, policy_version 1307735 (0.0010) [2023-12-27 00:54:43,870][105692] Updated weights for policy 0, policy_version 1307745 (0.0008) [2023-12-27 00:54:43,927][105692] Updated weights for policy 0, policy_version 1307755 (0.0007) [2023-12-27 00:54:44,016][105620] Updated weights for policy 1, policy_version 1309349 (0.0008) [2023-12-27 00:54:44,066][105620] Updated weights for policy 1, policy_version 1309359 (0.0010) [2023-12-27 00:54:44,118][105620] Updated weights for policy 1, policy_version 1309369 (0.0009) [2023-12-27 00:54:44,630][105692] Updated weights for policy 0, policy_version 1307765 (0.0007) [2023-12-27 00:54:44,680][105692] Updated weights for policy 0, policy_version 1307775 (0.0009) [2023-12-27 00:54:44,729][105692] Updated weights for policy 0, policy_version 1307785 (0.0010) [2023-12-27 00:54:44,789][105620] Updated weights for policy 1, policy_version 1309379 (0.0009) [2023-12-27 00:54:44,849][105620] Updated weights for policy 1, policy_version 1309389 (0.0009) [2023-12-27 00:54:44,905][105620] Updated weights for policy 1, policy_version 1309399 (0.0008) [2023-12-27 00:54:45,532][105692] Updated weights for policy 0, policy_version 1307795 (0.0010) [2023-12-27 00:54:45,595][105692] Updated weights for policy 0, policy_version 1307805 (0.0010) [2023-12-27 00:54:45,602][105620] Updated weights for policy 1, policy_version 1309409 (0.0008) [2023-12-27 00:54:45,652][105620] Updated weights for policy 1, policy_version 1309419 (0.0007) [2023-12-27 00:54:45,654][105692] Updated weights for policy 0, policy_version 1307815 (0.0011) [2023-12-27 00:54:45,707][105620] Updated weights for policy 1, policy_version 1309429 (0.0008) [2023-12-27 00:54:45,762][105620] Updated weights for policy 1, policy_version 1309439 (0.0008) [2023-12-27 00:54:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 670113792. Throughput: 0: 9552.6, 1: 9850.8. Samples: 670081428. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:54:46,063][104569] Avg episode reward: [(0, '8451.435'), (1, '9088.471')] [2023-12-27 00:54:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001307824_334856192.pth... [2023-12-27 00:54:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001309440_335257600.pth... [2023-12-27 00:54:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001306704_334569472.pth [2023-12-27 00:54:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001308320_334970880.pth [2023-12-27 00:54:46,389][105692] Updated weights for policy 0, policy_version 1307825 (0.0010) [2023-12-27 00:54:46,450][105692] Updated weights for policy 0, policy_version 1307835 (0.0010) [2023-12-27 00:54:46,483][105620] Updated weights for policy 1, policy_version 1309449 (0.0005) [2023-12-27 00:54:46,498][105692] Updated weights for policy 0, policy_version 1307845 (0.0010) [2023-12-27 00:54:46,542][105620] Updated weights for policy 1, policy_version 1309459 (0.0009) [2023-12-27 00:54:46,546][105692] Updated weights for policy 0, policy_version 1307855 (0.0010) [2023-12-27 00:54:46,596][105620] Updated weights for policy 1, policy_version 1309469 (0.0007) [2023-12-27 00:54:47,217][105620] Updated weights for policy 1, policy_version 1309479 (0.0006) [2023-12-27 00:54:47,255][105692] Updated weights for policy 0, policy_version 1307865 (0.0006) [2023-12-27 00:54:47,276][105620] Updated weights for policy 1, policy_version 1309489 (0.0005) [2023-12-27 00:54:47,313][105692] Updated weights for policy 0, policy_version 1307875 (0.0006) [2023-12-27 00:54:47,337][105620] Updated weights for policy 1, policy_version 1309499 (0.0005) [2023-12-27 00:54:47,368][105692] Updated weights for policy 0, policy_version 1307885 (0.0010) [2023-12-27 00:54:47,842][105620] Updated weights for policy 1, policy_version 1309509 (0.0006) [2023-12-27 00:54:47,905][105620] Updated weights for policy 1, policy_version 1309519 (0.0010) [2023-12-27 00:54:47,956][105692] Updated weights for policy 0, policy_version 1307895 (0.0009) [2023-12-27 00:54:47,961][105620] Updated weights for policy 1, policy_version 1309529 (0.0010) [2023-12-27 00:54:48,008][105692] Updated weights for policy 0, policy_version 1307905 (0.0010) [2023-12-27 00:54:48,058][105692] Updated weights for policy 0, policy_version 1307915 (0.0005) [2023-12-27 00:54:48,644][105620] Updated weights for policy 1, policy_version 1309539 (0.0010) [2023-12-27 00:54:48,658][105692] Updated weights for policy 0, policy_version 1307925 (0.0005) [2023-12-27 00:54:48,694][105620] Updated weights for policy 1, policy_version 1309549 (0.0011) [2023-12-27 00:54:48,716][105692] Updated weights for policy 0, policy_version 1307935 (0.0005) [2023-12-27 00:54:48,743][105620] Updated weights for policy 1, policy_version 1309559 (0.0011) [2023-12-27 00:54:48,778][105692] Updated weights for policy 0, policy_version 1307945 (0.0009) [2023-12-27 00:54:49,423][105620] Updated weights for policy 1, policy_version 1309569 (0.0011) [2023-12-27 00:54:49,491][105692] Updated weights for policy 0, policy_version 1307955 (0.0009) [2023-12-27 00:54:49,497][105620] Updated weights for policy 1, policy_version 1309579 (0.0011) [2023-12-27 00:54:49,551][105692] Updated weights for policy 0, policy_version 1307965 (0.0005) [2023-12-27 00:54:49,560][105620] Updated weights for policy 1, policy_version 1309589 (0.0011) [2023-12-27 00:54:49,609][105692] Updated weights for policy 0, policy_version 1307975 (0.0007) [2023-12-27 00:54:49,617][105620] Updated weights for policy 1, policy_version 1309599 (0.0011) [2023-12-27 00:54:50,326][105620] Updated weights for policy 1, policy_version 1309609 (0.0006) [2023-12-27 00:54:50,334][105692] Updated weights for policy 0, policy_version 1307985 (0.0010) [2023-12-27 00:54:50,383][105620] Updated weights for policy 1, policy_version 1309619 (0.0009) [2023-12-27 00:54:50,389][105692] Updated weights for policy 0, policy_version 1307995 (0.0010) [2023-12-27 00:54:50,439][105692] Updated weights for policy 0, policy_version 1308005 (0.0011) [2023-12-27 00:54:50,442][105620] Updated weights for policy 1, policy_version 1309629 (0.0006) [2023-12-27 00:54:50,491][105692] Updated weights for policy 0, policy_version 1308015 (0.0010) [2023-12-27 00:54:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 670212096. Throughput: 0: 9683.1, 1: 9893.9. Samples: 670205164. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:54:51,062][104569] Avg episode reward: [(0, '8449.684'), (1, '8914.599')] [2023-12-27 00:54:51,079][105620] Updated weights for policy 1, policy_version 1309639 (0.0008) [2023-12-27 00:54:51,145][105620] Updated weights for policy 1, policy_version 1309649 (0.0009) [2023-12-27 00:54:51,198][105620] Updated weights for policy 1, policy_version 1309659 (0.0009) [2023-12-27 00:54:51,216][105692] Updated weights for policy 0, policy_version 1308025 (0.0009) [2023-12-27 00:54:51,274][105692] Updated weights for policy 0, policy_version 1308035 (0.0009) [2023-12-27 00:54:51,334][105692] Updated weights for policy 0, policy_version 1308045 (0.0008) [2023-12-27 00:54:51,978][105620] Updated weights for policy 1, policy_version 1309669 (0.0009) [2023-12-27 00:54:52,034][105620] Updated weights for policy 1, policy_version 1309679 (0.0008) [2023-12-27 00:54:52,090][105620] Updated weights for policy 1, policy_version 1309689 (0.0007) [2023-12-27 00:54:52,127][105692] Updated weights for policy 0, policy_version 1308055 (0.0009) [2023-12-27 00:54:52,174][105692] Updated weights for policy 0, policy_version 1308065 (0.0009) [2023-12-27 00:54:52,224][105692] Updated weights for policy 0, policy_version 1308075 (0.0009) [2023-12-27 00:54:52,818][105620] Updated weights for policy 1, policy_version 1309699 (0.0006) [2023-12-27 00:54:52,881][105620] Updated weights for policy 1, policy_version 1309709 (0.0006) [2023-12-27 00:54:52,941][105620] Updated weights for policy 1, policy_version 1309719 (0.0006) [2023-12-27 00:54:53,060][105692] Updated weights for policy 0, policy_version 1308085 (0.0009) [2023-12-27 00:54:53,111][105692] Updated weights for policy 0, policy_version 1308095 (0.0009) [2023-12-27 00:54:53,157][105692] Updated weights for policy 0, policy_version 1308105 (0.0008) [2023-12-27 00:54:53,673][105620] Updated weights for policy 1, policy_version 1309729 (0.0007) [2023-12-27 00:54:53,743][105620] Updated weights for policy 1, policy_version 1309739 (0.0010) [2023-12-27 00:54:53,784][105692] Updated weights for policy 0, policy_version 1308115 (0.0008) [2023-12-27 00:54:53,813][105620] Updated weights for policy 1, policy_version 1309749 (0.0010) [2023-12-27 00:54:53,840][105692] Updated weights for policy 0, policy_version 1308125 (0.0005) [2023-12-27 00:54:53,861][105620] Updated weights for policy 1, policy_version 1309759 (0.0009) [2023-12-27 00:54:53,897][105692] Updated weights for policy 0, policy_version 1308135 (0.0007) [2023-12-27 00:54:54,502][105620] Updated weights for policy 1, policy_version 1309769 (0.0008) [2023-12-27 00:54:54,556][105620] Updated weights for policy 1, policy_version 1309779 (0.0005) [2023-12-27 00:54:54,603][105620] Updated weights for policy 1, policy_version 1309789 (0.0005) [2023-12-27 00:54:54,670][105692] Updated weights for policy 0, policy_version 1308145 (0.0009) [2023-12-27 00:54:54,726][105692] Updated weights for policy 0, policy_version 1308155 (0.0005) [2023-12-27 00:54:54,779][105692] Updated weights for policy 0, policy_version 1308165 (0.0005) [2023-12-27 00:54:54,829][105692] Updated weights for policy 0, policy_version 1308175 (0.0005) [2023-12-27 00:54:55,395][105620] Updated weights for policy 1, policy_version 1309799 (0.0007) [2023-12-27 00:54:55,414][105692] Updated weights for policy 0, policy_version 1308185 (0.0006) [2023-12-27 00:54:55,446][105620] Updated weights for policy 1, policy_version 1309809 (0.0007) [2023-12-27 00:54:55,471][105692] Updated weights for policy 0, policy_version 1308195 (0.0006) [2023-12-27 00:54:55,511][105620] Updated weights for policy 1, policy_version 1309819 (0.0005) [2023-12-27 00:54:55,520][105692] Updated weights for policy 0, policy_version 1308205 (0.0006) [2023-12-27 00:54:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 670310400. Throughput: 0: 9625.2, 1: 9855.5. Samples: 670322444. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:54:56,062][104569] Avg episode reward: [(0, '8459.482'), (1, '9002.008')] [2023-12-27 00:54:56,112][105692] Updated weights for policy 0, policy_version 1308215 (0.0006) [2023-12-27 00:54:56,147][105620] Updated weights for policy 1, policy_version 1309829 (0.0005) [2023-12-27 00:54:56,180][105692] Updated weights for policy 0, policy_version 1308225 (0.0006) [2023-12-27 00:54:56,209][105620] Updated weights for policy 1, policy_version 1309839 (0.0006) [2023-12-27 00:54:56,246][105692] Updated weights for policy 0, policy_version 1308235 (0.0006) [2023-12-27 00:54:56,268][105620] Updated weights for policy 1, policy_version 1309849 (0.0006) [2023-12-27 00:54:56,765][105692] Updated weights for policy 0, policy_version 1308245 (0.0006) [2023-12-27 00:54:56,820][105692] Updated weights for policy 0, policy_version 1308255 (0.0006) [2023-12-27 00:54:56,868][105692] Updated weights for policy 0, policy_version 1308265 (0.0006) [2023-12-27 00:54:57,034][105620] Updated weights for policy 1, policy_version 1309859 (0.0008) [2023-12-27 00:54:57,090][105620] Updated weights for policy 1, policy_version 1309869 (0.0005) [2023-12-27 00:54:57,142][105620] Updated weights for policy 1, policy_version 1309879 (0.0007) [2023-12-27 00:54:57,558][105692] Updated weights for policy 0, policy_version 1308275 (0.0008) [2023-12-27 00:54:57,617][105692] Updated weights for policy 0, policy_version 1308285 (0.0009) [2023-12-27 00:54:57,681][105692] Updated weights for policy 0, policy_version 1308295 (0.0008) [2023-12-27 00:54:57,811][105620] Updated weights for policy 1, policy_version 1309889 (0.0009) [2023-12-27 00:54:57,880][105620] Updated weights for policy 1, policy_version 1309899 (0.0005) [2023-12-27 00:54:57,945][105620] Updated weights for policy 1, policy_version 1309909 (0.0005) [2023-12-27 00:54:58,010][105620] Updated weights for policy 1, policy_version 1309919 (0.0005) [2023-12-27 00:54:58,526][105692] Updated weights for policy 0, policy_version 1308305 (0.0007) [2023-12-27 00:54:58,588][105692] Updated weights for policy 0, policy_version 1308315 (0.0009) [2023-12-27 00:54:58,610][105620] Updated weights for policy 1, policy_version 1309929 (0.0009) [2023-12-27 00:54:58,645][105692] Updated weights for policy 0, policy_version 1308325 (0.0009) [2023-12-27 00:54:58,676][105620] Updated weights for policy 1, policy_version 1309939 (0.0009) [2023-12-27 00:54:58,707][105692] Updated weights for policy 0, policy_version 1308335 (0.0009) [2023-12-27 00:54:58,736][105620] Updated weights for policy 1, policy_version 1309949 (0.0009) [2023-12-27 00:54:59,514][105620] Updated weights for policy 1, policy_version 1309959 (0.0010) [2023-12-27 00:54:59,517][105692] Updated weights for policy 0, policy_version 1308345 (0.0007) [2023-12-27 00:54:59,572][105620] Updated weights for policy 1, policy_version 1309969 (0.0006) [2023-12-27 00:54:59,584][105692] Updated weights for policy 0, policy_version 1308355 (0.0006) [2023-12-27 00:54:59,626][105620] Updated weights for policy 1, policy_version 1309979 (0.0010) [2023-12-27 00:54:59,638][105692] Updated weights for policy 0, policy_version 1308365 (0.0005) [2023-12-27 00:55:00,303][105620] Updated weights for policy 1, policy_version 1309989 (0.0008) [2023-12-27 00:55:00,336][105692] Updated weights for policy 0, policy_version 1308375 (0.0008) [2023-12-27 00:55:00,355][105620] Updated weights for policy 1, policy_version 1309999 (0.0007) [2023-12-27 00:55:00,394][105692] Updated weights for policy 0, policy_version 1308385 (0.0009) [2023-12-27 00:55:00,409][105620] Updated weights for policy 1, policy_version 1310009 (0.0006) [2023-12-27 00:55:00,454][105692] Updated weights for policy 0, policy_version 1308395 (0.0008) [2023-12-27 00:55:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 670408704. Throughput: 0: 9709.7, 1: 9843.7. Samples: 670382612. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:55:01,062][104569] Avg episode reward: [(0, '8551.052'), (1, '9355.254')] [2023-12-27 00:55:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001308400_335003648.pth... [2023-12-27 00:55:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001310016_335405056.pth... [2023-12-27 00:55:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001307280_334716928.pth [2023-12-27 00:55:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001308864_335110144.pth [2023-12-27 00:55:01,108][105620] Updated weights for policy 1, policy_version 1310019 (0.0007) [2023-12-27 00:55:01,123][105692] Updated weights for policy 0, policy_version 1308405 (0.0009) [2023-12-27 00:55:01,167][105620] Updated weights for policy 1, policy_version 1310029 (0.0007) [2023-12-27 00:55:01,184][105692] Updated weights for policy 0, policy_version 1308415 (0.0007) [2023-12-27 00:55:01,215][105620] Updated weights for policy 1, policy_version 1310039 (0.0006) [2023-12-27 00:55:01,242][105692] Updated weights for policy 0, policy_version 1308425 (0.0007) [2023-12-27 00:55:01,862][105620] Updated weights for policy 1, policy_version 1310049 (0.0007) [2023-12-27 00:55:01,910][105620] Updated weights for policy 1, policy_version 1310059 (0.0006) [2023-12-27 00:55:01,960][105620] Updated weights for policy 1, policy_version 1310069 (0.0006) [2023-12-27 00:55:02,005][105620] Updated weights for policy 1, policy_version 1310079 (0.0007) [2023-12-27 00:55:02,077][105692] Updated weights for policy 0, policy_version 1308435 (0.0008) [2023-12-27 00:55:02,132][105692] Updated weights for policy 0, policy_version 1308445 (0.0009) [2023-12-27 00:55:02,186][105692] Updated weights for policy 0, policy_version 1308455 (0.0009) [2023-12-27 00:55:02,657][105620] Updated weights for policy 1, policy_version 1310089 (0.0009) [2023-12-27 00:55:02,707][105620] Updated weights for policy 1, policy_version 1310099 (0.0008) [2023-12-27 00:55:02,756][105620] Updated weights for policy 1, policy_version 1310109 (0.0009) [2023-12-27 00:55:02,951][105692] Updated weights for policy 0, policy_version 1308465 (0.0009) [2023-12-27 00:55:03,017][105692] Updated weights for policy 0, policy_version 1308475 (0.0005) [2023-12-27 00:55:03,073][105692] Updated weights for policy 0, policy_version 1308485 (0.0005) [2023-12-27 00:55:03,131][105692] Updated weights for policy 0, policy_version 1308495 (0.0005) [2023-12-27 00:55:03,541][105620] Updated weights for policy 1, policy_version 1310119 (0.0006) [2023-12-27 00:55:03,590][105620] Updated weights for policy 1, policy_version 1310129 (0.0007) [2023-12-27 00:55:03,633][105692] Updated weights for policy 0, policy_version 1308505 (0.0006) [2023-12-27 00:55:03,640][105620] Updated weights for policy 1, policy_version 1310139 (0.0007) [2023-12-27 00:55:03,686][105692] Updated weights for policy 0, policy_version 1308515 (0.0008) [2023-12-27 00:55:03,740][105692] Updated weights for policy 0, policy_version 1308527 (0.0010) [2023-12-27 00:55:04,274][105620] Updated weights for policy 1, policy_version 1310149 (0.0006) [2023-12-27 00:55:04,337][105620] Updated weights for policy 1, policy_version 1310159 (0.0006) [2023-12-27 00:55:04,399][105620] Updated weights for policy 1, policy_version 1310169 (0.0009) [2023-12-27 00:55:04,574][105692] Updated weights for policy 0, policy_version 1308537 (0.0009) [2023-12-27 00:55:04,626][105692] Updated weights for policy 0, policy_version 1308547 (0.0009) [2023-12-27 00:55:04,685][105692] Updated weights for policy 0, policy_version 1308557 (0.0009) [2023-12-27 00:55:05,063][105620] Updated weights for policy 1, policy_version 1310179 (0.0009) [2023-12-27 00:55:05,125][105620] Updated weights for policy 1, policy_version 1310189 (0.0008) [2023-12-27 00:55:05,188][105620] Updated weights for policy 1, policy_version 1310199 (0.0008) [2023-12-27 00:55:05,488][105692] Updated weights for policy 0, policy_version 1308567 (0.0009) [2023-12-27 00:55:05,540][105692] Updated weights for policy 0, policy_version 1308577 (0.0009) [2023-12-27 00:55:05,592][105692] Updated weights for policy 0, policy_version 1308587 (0.0009) [2023-12-27 00:55:05,880][105620] Updated weights for policy 1, policy_version 1310209 (0.0008) [2023-12-27 00:55:05,938][105620] Updated weights for policy 1, policy_version 1310219 (0.0005) [2023-12-27 00:55:05,995][105620] Updated weights for policy 1, policy_version 1310229 (0.0005) [2023-12-27 00:55:06,056][105620] Updated weights for policy 1, policy_version 1310239 (0.0006) [2023-12-27 00:55:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 670515200. Throughput: 0: 9750.3, 1: 9851.1. Samples: 670501668. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:55:06,062][104569] Avg episode reward: [(0, '8813.126'), (1, '9174.669')] [2023-12-27 00:55:06,413][105692] Updated weights for policy 0, policy_version 1308597 (0.0009) [2023-12-27 00:55:06,475][105692] Updated weights for policy 0, policy_version 1308607 (0.0009) [2023-12-27 00:55:06,531][105692] Updated weights for policy 0, policy_version 1308617 (0.0006) [2023-12-27 00:55:06,741][105620] Updated weights for policy 1, policy_version 1310249 (0.0008) [2023-12-27 00:55:06,806][105620] Updated weights for policy 1, policy_version 1310259 (0.0009) [2023-12-27 00:55:06,875][105620] Updated weights for policy 1, policy_version 1310269 (0.0009) [2023-12-27 00:55:07,242][105692] Updated weights for policy 0, policy_version 1308627 (0.0007) [2023-12-27 00:55:07,296][105692] Updated weights for policy 0, policy_version 1308637 (0.0005) [2023-12-27 00:55:07,353][105692] Updated weights for policy 0, policy_version 1308647 (0.0005) [2023-12-27 00:55:07,682][105620] Updated weights for policy 1, policy_version 1310279 (0.0009) [2023-12-27 00:55:07,729][105620] Updated weights for policy 1, policy_version 1310289 (0.0008) [2023-12-27 00:55:07,775][105620] Updated weights for policy 1, policy_version 1310299 (0.0008) [2023-12-27 00:55:07,976][105692] Updated weights for policy 0, policy_version 1308657 (0.0006) [2023-12-27 00:55:08,035][105692] Updated weights for policy 0, policy_version 1308667 (0.0009) [2023-12-27 00:55:08,087][105692] Updated weights for policy 0, policy_version 1308677 (0.0009) [2023-12-27 00:55:08,154][105692] Updated weights for policy 0, policy_version 1308687 (0.0009) [2023-12-27 00:55:08,558][105620] Updated weights for policy 1, policy_version 1310309 (0.0009) [2023-12-27 00:55:08,623][105620] Updated weights for policy 1, policy_version 1310319 (0.0009) [2023-12-27 00:55:08,685][105620] Updated weights for policy 1, policy_version 1310329 (0.0009) [2023-12-27 00:55:08,921][105692] Updated weights for policy 0, policy_version 1308697 (0.0009) [2023-12-27 00:55:08,975][105692] Updated weights for policy 0, policy_version 1308707 (0.0010) [2023-12-27 00:55:09,029][105692] Updated weights for policy 0, policy_version 1308717 (0.0010) [2023-12-27 00:55:09,342][105620] Updated weights for policy 1, policy_version 1310339 (0.0008) [2023-12-27 00:55:09,410][105620] Updated weights for policy 1, policy_version 1310349 (0.0009) [2023-12-27 00:55:09,478][105620] Updated weights for policy 1, policy_version 1310359 (0.0010) [2023-12-27 00:55:09,811][105692] Updated weights for policy 0, policy_version 1308727 (0.0009) [2023-12-27 00:55:09,875][105692] Updated weights for policy 0, policy_version 1308737 (0.0009) [2023-12-27 00:55:09,941][105692] Updated weights for policy 0, policy_version 1308747 (0.0009) [2023-12-27 00:55:10,187][105620] Updated weights for policy 1, policy_version 1310369 (0.0009) [2023-12-27 00:55:10,253][105620] Updated weights for policy 1, policy_version 1310379 (0.0005) [2023-12-27 00:55:10,313][105620] Updated weights for policy 1, policy_version 1310389 (0.0006) [2023-12-27 00:55:10,371][105620] Updated weights for policy 1, policy_version 1310399 (0.0006) [2023-12-27 00:55:10,830][105692] Updated weights for policy 0, policy_version 1308757 (0.0009) [2023-12-27 00:55:10,888][105692] Updated weights for policy 0, policy_version 1308767 (0.0009) [2023-12-27 00:55:10,945][105692] Updated weights for policy 0, policy_version 1308777 (0.0009) [2023-12-27 00:55:11,013][105620] Updated weights for policy 1, policy_version 1310409 (0.0007) [2023-12-27 00:55:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 670605312. Throughput: 0: 9674.7, 1: 9949.9. Samples: 670614928. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:55:11,062][104569] Avg episode reward: [(0, '8726.231'), (1, '9086.807')] [2023-12-27 00:55:11,078][105620] Updated weights for policy 1, policy_version 1310419 (0.0008) [2023-12-27 00:55:11,138][105620] Updated weights for policy 1, policy_version 1310429 (0.0008) [2023-12-27 00:55:11,819][105692] Updated weights for policy 0, policy_version 1308787 (0.0008) [2023-12-27 00:55:11,885][105692] Updated weights for policy 0, policy_version 1308797 (0.0008) [2023-12-27 00:55:11,899][105620] Updated weights for policy 1, policy_version 1310439 (0.0007) [2023-12-27 00:55:11,947][105692] Updated weights for policy 0, policy_version 1308807 (0.0006) [2023-12-27 00:55:11,971][105620] Updated weights for policy 1, policy_version 1310449 (0.0008) [2023-12-27 00:55:12,040][105620] Updated weights for policy 1, policy_version 1310459 (0.0007) [2023-12-27 00:55:12,709][105692] Updated weights for policy 0, policy_version 1308817 (0.0008) [2023-12-27 00:55:12,730][105620] Updated weights for policy 1, policy_version 1310469 (0.0009) [2023-12-27 00:55:12,772][105692] Updated weights for policy 0, policy_version 1308827 (0.0006) [2023-12-27 00:55:12,793][105620] Updated weights for policy 1, policy_version 1310479 (0.0009) [2023-12-27 00:55:12,828][105692] Updated weights for policy 0, policy_version 1308837 (0.0007) [2023-12-27 00:55:12,855][105620] Updated weights for policy 1, policy_version 1310489 (0.0007) [2023-12-27 00:55:12,890][105692] Updated weights for policy 0, policy_version 1308847 (0.0007) [2023-12-27 00:55:13,471][105620] Updated weights for policy 1, policy_version 1310499 (0.0005) [2023-12-27 00:55:13,534][105620] Updated weights for policy 1, policy_version 1310509 (0.0005) [2023-12-27 00:55:13,590][105692] Updated weights for policy 0, policy_version 1308857 (0.0007) [2023-12-27 00:55:13,594][105620] Updated weights for policy 1, policy_version 1310519 (0.0006) [2023-12-27 00:55:13,649][105692] Updated weights for policy 0, policy_version 1308867 (0.0006) [2023-12-27 00:55:13,712][105692] Updated weights for policy 0, policy_version 1308877 (0.0009) [2023-12-27 00:55:14,272][105692] Updated weights for policy 0, policy_version 1308887 (0.0008) [2023-12-27 00:55:14,295][105620] Updated weights for policy 1, policy_version 1310529 (0.0008) [2023-12-27 00:55:14,334][105692] Updated weights for policy 0, policy_version 1308897 (0.0005) [2023-12-27 00:55:14,343][105620] Updated weights for policy 1, policy_version 1310539 (0.0009) [2023-12-27 00:55:14,391][105692] Updated weights for policy 0, policy_version 1308907 (0.0005) [2023-12-27 00:55:14,400][105620] Updated weights for policy 1, policy_version 1310549 (0.0005) [2023-12-27 00:55:14,451][105620] Updated weights for policy 1, policy_version 1310559 (0.0005) [2023-12-27 00:55:14,971][105692] Updated weights for policy 0, policy_version 1308917 (0.0007) [2023-12-27 00:55:15,020][105692] Updated weights for policy 0, policy_version 1308927 (0.0008) [2023-12-27 00:55:15,076][105692] Updated weights for policy 0, policy_version 1308937 (0.0008) [2023-12-27 00:55:15,082][105620] Updated weights for policy 1, policy_version 1310569 (0.0007) [2023-12-27 00:55:15,144][105620] Updated weights for policy 1, policy_version 1310579 (0.0008) [2023-12-27 00:55:15,203][105620] Updated weights for policy 1, policy_version 1310589 (0.0008) [2023-12-27 00:55:15,857][105692] Updated weights for policy 0, policy_version 1308947 (0.0009) [2023-12-27 00:55:15,918][105692] Updated weights for policy 0, policy_version 1308957 (0.0009) [2023-12-27 00:55:15,923][105620] Updated weights for policy 1, policy_version 1310599 (0.0008) [2023-12-27 00:55:15,970][105692] Updated weights for policy 0, policy_version 1308967 (0.0010) [2023-12-27 00:55:15,978][105620] Updated weights for policy 1, policy_version 1310609 (0.0011) [2023-12-27 00:55:16,029][105620] Updated weights for policy 1, policy_version 1310619 (0.0007) [2023-12-27 00:55:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 670711808. Throughput: 0: 9613.3, 1: 9976.8. Samples: 670671644. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:55:16,062][104569] Avg episode reward: [(0, '8643.452'), (1, '9084.517')] [2023-12-27 00:55:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001310624_335560704.pth... [2023-12-27 00:55:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001308976_335151104.pth... [2023-12-27 00:55:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001309440_335257600.pth [2023-12-27 00:55:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001307824_334856192.pth [2023-12-27 00:55:16,553][105692] Updated weights for policy 0, policy_version 1308977 (0.0010) [2023-12-27 00:55:16,606][105692] Updated weights for policy 0, policy_version 1308987 (0.0005) [2023-12-27 00:55:16,661][105692] Updated weights for policy 0, policy_version 1308997 (0.0005) [2023-12-27 00:55:16,717][105692] Updated weights for policy 0, policy_version 1309007 (0.0005) [2023-12-27 00:55:16,793][105620] Updated weights for policy 1, policy_version 1310629 (0.0009) [2023-12-27 00:55:16,848][105620] Updated weights for policy 1, policy_version 1310639 (0.0011) [2023-12-27 00:55:16,906][105620] Updated weights for policy 1, policy_version 1310649 (0.0010) [2023-12-27 00:55:17,348][105692] Updated weights for policy 0, policy_version 1309017 (0.0010) [2023-12-27 00:55:17,396][105692] Updated weights for policy 0, policy_version 1309027 (0.0010) [2023-12-27 00:55:17,443][105692] Updated weights for policy 0, policy_version 1309037 (0.0010) [2023-12-27 00:55:17,654][105620] Updated weights for policy 1, policy_version 1310659 (0.0011) [2023-12-27 00:55:17,709][105620] Updated weights for policy 1, policy_version 1310669 (0.0010) [2023-12-27 00:55:17,753][105620] Updated weights for policy 1, policy_version 1310679 (0.0010) [2023-12-27 00:55:18,124][105692] Updated weights for policy 0, policy_version 1309047 (0.0007) [2023-12-27 00:55:18,173][105692] Updated weights for policy 0, policy_version 1309057 (0.0006) [2023-12-27 00:55:18,220][105692] Updated weights for policy 0, policy_version 1309067 (0.0005) [2023-12-27 00:55:18,487][105620] Updated weights for policy 1, policy_version 1310689 (0.0010) [2023-12-27 00:55:18,541][105620] Updated weights for policy 1, policy_version 1310699 (0.0005) [2023-12-27 00:55:18,595][105620] Updated weights for policy 1, policy_version 1310709 (0.0005) [2023-12-27 00:55:18,643][105620] Updated weights for policy 1, policy_version 1310719 (0.0010) [2023-12-27 00:55:18,909][105692] Updated weights for policy 0, policy_version 1309077 (0.0008) [2023-12-27 00:55:18,966][105692] Updated weights for policy 0, policy_version 1309087 (0.0005) [2023-12-27 00:55:19,029][105692] Updated weights for policy 0, policy_version 1309097 (0.0005) [2023-12-27 00:55:19,358][105620] Updated weights for policy 1, policy_version 1310729 (0.0010) [2023-12-27 00:55:19,410][105620] Updated weights for policy 1, policy_version 1310739 (0.0011) [2023-12-27 00:55:19,476][105620] Updated weights for policy 1, policy_version 1310749 (0.0011) [2023-12-27 00:55:19,737][105692] Updated weights for policy 0, policy_version 1309107 (0.0008) [2023-12-27 00:55:19,798][105692] Updated weights for policy 0, policy_version 1309117 (0.0007) [2023-12-27 00:55:19,866][105692] Updated weights for policy 0, policy_version 1309127 (0.0008) [2023-12-27 00:55:20,167][105620] Updated weights for policy 1, policy_version 1310759 (0.0009) [2023-12-27 00:55:20,223][105620] Updated weights for policy 1, policy_version 1310769 (0.0008) [2023-12-27 00:55:20,290][105620] Updated weights for policy 1, policy_version 1310779 (0.0008) [2023-12-27 00:55:20,571][105692] Updated weights for policy 0, policy_version 1309137 (0.0006) [2023-12-27 00:55:20,631][105692] Updated weights for policy 0, policy_version 1309147 (0.0008) [2023-12-27 00:55:20,685][105692] Updated weights for policy 0, policy_version 1309157 (0.0008) [2023-12-27 00:55:20,732][105692] Updated weights for policy 0, policy_version 1309167 (0.0008) [2023-12-27 00:55:21,060][105620] Updated weights for policy 1, policy_version 1310789 (0.0009) [2023-12-27 00:55:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 670801920. Throughput: 0: 9710.5, 1: 9984.2. Samples: 670794492. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:55:21,062][104569] Avg episode reward: [(0, '8551.802'), (1, '9172.493')] [2023-12-27 00:55:21,124][105620] Updated weights for policy 1, policy_version 1310799 (0.0009) [2023-12-27 00:55:21,180][105620] Updated weights for policy 1, policy_version 1310809 (0.0009) [2023-12-27 00:55:21,575][105692] Updated weights for policy 0, policy_version 1309177 (0.0009) [2023-12-27 00:55:21,631][105692] Updated weights for policy 0, policy_version 1309187 (0.0007) [2023-12-27 00:55:21,695][105692] Updated weights for policy 0, policy_version 1309197 (0.0009) [2023-12-27 00:55:21,992][105620] Updated weights for policy 1, policy_version 1310819 (0.0009) [2023-12-27 00:55:22,058][105620] Updated weights for policy 1, policy_version 1310829 (0.0009) [2023-12-27 00:55:22,122][105620] Updated weights for policy 1, policy_version 1310839 (0.0008) [2023-12-27 00:55:22,421][105692] Updated weights for policy 0, policy_version 1309207 (0.0009) [2023-12-27 00:55:22,489][105692] Updated weights for policy 0, policy_version 1309217 (0.0010) [2023-12-27 00:55:22,553][105692] Updated weights for policy 0, policy_version 1309227 (0.0009) [2023-12-27 00:55:22,853][105620] Updated weights for policy 1, policy_version 1310849 (0.0009) [2023-12-27 00:55:22,916][105620] Updated weights for policy 1, policy_version 1310859 (0.0006) [2023-12-27 00:55:22,966][105620] Updated weights for policy 1, policy_version 1310869 (0.0005) [2023-12-27 00:55:23,019][105620] Updated weights for policy 1, policy_version 1310879 (0.0005) [2023-12-27 00:55:23,338][105692] Updated weights for policy 0, policy_version 1309237 (0.0010) [2023-12-27 00:55:23,392][105692] Updated weights for policy 0, policy_version 1309247 (0.0010) [2023-12-27 00:55:23,452][105692] Updated weights for policy 0, policy_version 1309257 (0.0007) [2023-12-27 00:55:23,629][105620] Updated weights for policy 1, policy_version 1310889 (0.0008) [2023-12-27 00:55:23,676][105620] Updated weights for policy 1, policy_version 1310899 (0.0010) [2023-12-27 00:55:23,727][105620] Updated weights for policy 1, policy_version 1310909 (0.0010) [2023-12-27 00:55:24,046][105692] Updated weights for policy 0, policy_version 1309267 (0.0007) [2023-12-27 00:55:24,104][105692] Updated weights for policy 0, policy_version 1309277 (0.0010) [2023-12-27 00:55:24,162][105692] Updated weights for policy 0, policy_version 1309287 (0.0010) [2023-12-27 00:55:24,469][105620] Updated weights for policy 1, policy_version 1310919 (0.0010) [2023-12-27 00:55:24,526][105620] Updated weights for policy 1, policy_version 1310929 (0.0009) [2023-12-27 00:55:24,578][105620] Updated weights for policy 1, policy_version 1310939 (0.0009) [2023-12-27 00:55:24,825][105692] Updated weights for policy 0, policy_version 1309297 (0.0010) [2023-12-27 00:55:24,885][105692] Updated weights for policy 0, policy_version 1309307 (0.0005) [2023-12-27 00:55:24,932][105692] Updated weights for policy 0, policy_version 1309317 (0.0005) [2023-12-27 00:55:24,979][105692] Updated weights for policy 0, policy_version 1309327 (0.0005) [2023-12-27 00:55:25,162][105620] Updated weights for policy 1, policy_version 1310949 (0.0009) [2023-12-27 00:55:25,226][105620] Updated weights for policy 1, policy_version 1310959 (0.0010) [2023-12-27 00:55:25,283][105620] Updated weights for policy 1, policy_version 1310969 (0.0010) [2023-12-27 00:55:25,536][105692] Updated weights for policy 0, policy_version 1309337 (0.0007) [2023-12-27 00:55:25,585][105692] Updated weights for policy 0, policy_version 1309347 (0.0007) [2023-12-27 00:55:25,639][105692] Updated weights for policy 0, policy_version 1309357 (0.0006) [2023-12-27 00:55:25,915][105620] Updated weights for policy 1, policy_version 1310979 (0.0009) [2023-12-27 00:55:25,970][105620] Updated weights for policy 1, policy_version 1310989 (0.0010) [2023-12-27 00:55:26,025][105620] Updated weights for policy 1, policy_version 1310999 (0.0010) [2023-12-27 00:55:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.4, 300 sec: 19549.7). Total num frames: 670900224. Throughput: 0: 9861.4, 1: 9942.7. Samples: 670913616. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:55:26,062][104569] Avg episode reward: [(0, '8908.468'), (1, '9084.099')] [2023-12-27 00:55:26,270][105692] Updated weights for policy 0, policy_version 1309367 (0.0005) [2023-12-27 00:55:26,331][105692] Updated weights for policy 0, policy_version 1309377 (0.0005) [2023-12-27 00:55:26,389][105692] Updated weights for policy 0, policy_version 1309387 (0.0005) [2023-12-27 00:55:26,769][105620] Updated weights for policy 1, policy_version 1311009 (0.0010) [2023-12-27 00:55:26,823][105620] Updated weights for policy 1, policy_version 1311019 (0.0010) [2023-12-27 00:55:26,884][105620] Updated weights for policy 1, policy_version 1311029 (0.0010) [2023-12-27 00:55:26,920][105692] Updated weights for policy 0, policy_version 1309397 (0.0005) [2023-12-27 00:55:26,934][105620] Updated weights for policy 1, policy_version 1311039 (0.0008) [2023-12-27 00:55:26,974][105692] Updated weights for policy 0, policy_version 1309407 (0.0005) [2023-12-27 00:55:27,022][105692] Updated weights for policy 0, policy_version 1309417 (0.0005) [2023-12-27 00:55:27,575][105620] Updated weights for policy 1, policy_version 1311049 (0.0010) [2023-12-27 00:55:27,620][105692] Updated weights for policy 0, policy_version 1309427 (0.0007) [2023-12-27 00:55:27,626][105620] Updated weights for policy 1, policy_version 1311059 (0.0010) [2023-12-27 00:55:27,674][105692] Updated weights for policy 0, policy_version 1309437 (0.0005) [2023-12-27 00:55:27,678][105620] Updated weights for policy 1, policy_version 1311069 (0.0010) [2023-12-27 00:55:27,730][105692] Updated weights for policy 0, policy_version 1309447 (0.0005) [2023-12-27 00:55:28,360][105692] Updated weights for policy 0, policy_version 1309457 (0.0006) [2023-12-27 00:55:28,398][105620] Updated weights for policy 1, policy_version 1311079 (0.0008) [2023-12-27 00:55:28,416][105692] Updated weights for policy 0, policy_version 1309467 (0.0006) [2023-12-27 00:55:28,458][105620] Updated weights for policy 1, policy_version 1311089 (0.0009) [2023-12-27 00:55:28,479][105692] Updated weights for policy 0, policy_version 1309477 (0.0006) [2023-12-27 00:55:28,525][105620] Updated weights for policy 1, policy_version 1311099 (0.0007) [2023-12-27 00:55:28,547][105692] Updated weights for policy 0, policy_version 1309487 (0.0006) [2023-12-27 00:55:29,177][105692] Updated weights for policy 0, policy_version 1309497 (0.0005) [2023-12-27 00:55:29,255][105692] Updated weights for policy 0, policy_version 1309507 (0.0007) [2023-12-27 00:55:29,310][105620] Updated weights for policy 1, policy_version 1311109 (0.0007) [2023-12-27 00:55:29,317][105692] Updated weights for policy 0, policy_version 1309517 (0.0008) [2023-12-27 00:55:29,374][105620] Updated weights for policy 1, policy_version 1311119 (0.0009) [2023-12-27 00:55:29,442][105620] Updated weights for policy 1, policy_version 1311129 (0.0008) [2023-12-27 00:55:30,064][105692] Updated weights for policy 0, policy_version 1309527 (0.0006) [2023-12-27 00:55:30,133][105692] Updated weights for policy 0, policy_version 1309537 (0.0006) [2023-12-27 00:55:30,182][105620] Updated weights for policy 1, policy_version 1311139 (0.0009) [2023-12-27 00:55:30,193][105692] Updated weights for policy 0, policy_version 1309547 (0.0005) [2023-12-27 00:55:30,240][105620] Updated weights for policy 1, policy_version 1311149 (0.0007) [2023-12-27 00:55:30,298][105620] Updated weights for policy 1, policy_version 1311159 (0.0008) [2023-12-27 00:55:30,889][105692] Updated weights for policy 0, policy_version 1309557 (0.0010) [2023-12-27 00:55:30,946][105692] Updated weights for policy 0, policy_version 1309567 (0.0010) [2023-12-27 00:55:30,994][105692] Updated weights for policy 0, policy_version 1309577 (0.0010) [2023-12-27 00:55:31,008][105620] Updated weights for policy 1, policy_version 1311169 (0.0007) [2023-12-27 00:55:31,062][104569] Fps is (10 sec: 20479.2, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 671006720. Throughput: 0: 9945.7, 1: 9951.7. Samples: 670976816. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:55:31,063][104569] Avg episode reward: [(0, '8817.255'), (1, '8992.175')] [2023-12-27 00:55:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001309584_335306752.pth... [2023-12-27 00:55:31,070][105620] Updated weights for policy 1, policy_version 1311179 (0.0009) [2023-12-27 00:55:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001308400_335003648.pth [2023-12-27 00:55:31,118][105620] Updated weights for policy 1, policy_version 1311189 (0.0008) [2023-12-27 00:55:31,180][105620] Updated weights for policy 1, policy_version 1311199 (0.0007) [2023-12-27 00:55:31,181][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001311200_335708160.pth... [2023-12-27 00:55:31,184][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001310016_335405056.pth [2023-12-27 00:55:31,744][105692] Updated weights for policy 0, policy_version 1309587 (0.0010) [2023-12-27 00:55:31,792][105692] Updated weights for policy 0, policy_version 1309597 (0.0010) [2023-12-27 00:55:31,847][105692] Updated weights for policy 0, policy_version 1309607 (0.0010) [2023-12-27 00:55:32,005][105620] Updated weights for policy 1, policy_version 1311209 (0.0010) [2023-12-27 00:55:32,064][105620] Updated weights for policy 1, policy_version 1311219 (0.0009) [2023-12-27 00:55:32,120][105620] Updated weights for policy 1, policy_version 1311229 (0.0009) [2023-12-27 00:55:32,460][105692] Updated weights for policy 0, policy_version 1309617 (0.0008) [2023-12-27 00:55:32,513][105692] Updated weights for policy 0, policy_version 1309627 (0.0009) [2023-12-27 00:55:32,579][105692] Updated weights for policy 0, policy_version 1309637 (0.0008) [2023-12-27 00:55:32,630][105692] Updated weights for policy 0, policy_version 1309647 (0.0007) [2023-12-27 00:55:32,976][105620] Updated weights for policy 1, policy_version 1311239 (0.0008) [2023-12-27 00:55:33,029][105620] Updated weights for policy 1, policy_version 1311249 (0.0008) [2023-12-27 00:55:33,082][105620] Updated weights for policy 1, policy_version 1311259 (0.0008) [2023-12-27 00:55:33,287][105692] Updated weights for policy 0, policy_version 1309657 (0.0010) [2023-12-27 00:55:33,330][105692] Updated weights for policy 0, policy_version 1309667 (0.0010) [2023-12-27 00:55:33,381][105692] Updated weights for policy 0, policy_version 1309677 (0.0010) [2023-12-27 00:55:33,869][105620] Updated weights for policy 1, policy_version 1311269 (0.0008) [2023-12-27 00:55:33,921][105620] Updated weights for policy 1, policy_version 1311279 (0.0008) [2023-12-27 00:55:33,965][105620] Updated weights for policy 1, policy_version 1311289 (0.0007) [2023-12-27 00:55:34,115][105692] Updated weights for policy 0, policy_version 1309687 (0.0010) [2023-12-27 00:55:34,181][105692] Updated weights for policy 0, policy_version 1309697 (0.0010) [2023-12-27 00:55:34,243][105692] Updated weights for policy 0, policy_version 1309707 (0.0007) [2023-12-27 00:55:34,713][105620] Updated weights for policy 1, policy_version 1311299 (0.0008) [2023-12-27 00:55:34,773][105620] Updated weights for policy 1, policy_version 1311309 (0.0008) [2023-12-27 00:55:34,835][105620] Updated weights for policy 1, policy_version 1311319 (0.0008) [2023-12-27 00:55:34,958][105692] Updated weights for policy 0, policy_version 1309717 (0.0010) [2023-12-27 00:55:35,016][105692] Updated weights for policy 0, policy_version 1309727 (0.0010) [2023-12-27 00:55:35,075][105692] Updated weights for policy 0, policy_version 1309737 (0.0010) [2023-12-27 00:55:35,664][105692] Updated weights for policy 0, policy_version 1309747 (0.0010) [2023-12-27 00:55:35,677][105620] Updated weights for policy 1, policy_version 1311329 (0.0009) [2023-12-27 00:55:35,723][105692] Updated weights for policy 0, policy_version 1309757 (0.0010) [2023-12-27 00:55:35,733][105620] Updated weights for policy 1, policy_version 1311339 (0.0005) [2023-12-27 00:55:35,781][105692] Updated weights for policy 0, policy_version 1309767 (0.0010) [2023-12-27 00:55:35,791][105620] Updated weights for policy 1, policy_version 1311349 (0.0005) [2023-12-27 00:55:35,847][105620] Updated weights for policy 1, policy_version 1311359 (0.0006) [2023-12-27 00:55:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 671105024. Throughput: 0: 9919.0, 1: 9765.5. Samples: 671090964. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:55:36,062][104569] Avg episode reward: [(0, '8907.559'), (1, '9172.284')] [2023-12-27 00:55:36,500][105692] Updated weights for policy 0, policy_version 1309777 (0.0010) [2023-12-27 00:55:36,563][105692] Updated weights for policy 0, policy_version 1309787 (0.0007) [2023-12-27 00:55:36,626][105692] Updated weights for policy 0, policy_version 1309797 (0.0010) [2023-12-27 00:55:36,640][105620] Updated weights for policy 1, policy_version 1311369 (0.0006) [2023-12-27 00:55:36,686][105692] Updated weights for policy 0, policy_version 1309807 (0.0011) [2023-12-27 00:55:36,700][105620] Updated weights for policy 1, policy_version 1311379 (0.0005) [2023-12-27 00:55:36,760][105620] Updated weights for policy 1, policy_version 1311389 (0.0008) [2023-12-27 00:55:37,357][105692] Updated weights for policy 0, policy_version 1309817 (0.0011) [2023-12-27 00:55:37,413][105692] Updated weights for policy 0, policy_version 1309827 (0.0011) [2023-12-27 00:55:37,469][105692] Updated weights for policy 0, policy_version 1309837 (0.0011) [2023-12-27 00:55:37,542][105620] Updated weights for policy 1, policy_version 1311399 (0.0007) [2023-12-27 00:55:37,588][105620] Updated weights for policy 1, policy_version 1311409 (0.0007) [2023-12-27 00:55:37,634][105620] Updated weights for policy 1, policy_version 1311419 (0.0010) [2023-12-27 00:55:38,225][105692] Updated weights for policy 0, policy_version 1309847 (0.0011) [2023-12-27 00:55:38,250][105620] Updated weights for policy 1, policy_version 1311429 (0.0008) [2023-12-27 00:55:38,285][105692] Updated weights for policy 0, policy_version 1309857 (0.0011) [2023-12-27 00:55:38,308][105620] Updated weights for policy 1, policy_version 1311439 (0.0006) [2023-12-27 00:55:38,343][105692] Updated weights for policy 0, policy_version 1309867 (0.0010) [2023-12-27 00:55:38,376][105620] Updated weights for policy 1, policy_version 1311449 (0.0007) [2023-12-27 00:55:38,973][105620] Updated weights for policy 1, policy_version 1311459 (0.0007) [2023-12-27 00:55:39,036][105620] Updated weights for policy 1, policy_version 1311469 (0.0011) [2023-12-27 00:55:39,093][105692] Updated weights for policy 0, policy_version 1309877 (0.0011) [2023-12-27 00:55:39,099][105620] Updated weights for policy 1, policy_version 1311479 (0.0011) [2023-12-27 00:55:39,154][105692] Updated weights for policy 0, policy_version 1309887 (0.0011) [2023-12-27 00:55:39,219][105692] Updated weights for policy 0, policy_version 1309897 (0.0010) [2023-12-27 00:55:39,815][105620] Updated weights for policy 1, policy_version 1311489 (0.0010) [2023-12-27 00:55:39,885][105620] Updated weights for policy 1, policy_version 1311499 (0.0011) [2023-12-27 00:55:39,951][105620] Updated weights for policy 1, policy_version 1311509 (0.0011) [2023-12-27 00:55:40,019][105620] Updated weights for policy 1, policy_version 1311519 (0.0011) [2023-12-27 00:55:40,029][105692] Updated weights for policy 0, policy_version 1309907 (0.0009) [2023-12-27 00:55:40,090][105692] Updated weights for policy 0, policy_version 1309917 (0.0006) [2023-12-27 00:55:40,153][105692] Updated weights for policy 0, policy_version 1309927 (0.0008) [2023-12-27 00:55:40,783][105620] Updated weights for policy 1, policy_version 1311529 (0.0010) [2023-12-27 00:55:40,841][105620] Updated weights for policy 1, policy_version 1311539 (0.0005) [2023-12-27 00:55:40,903][105620] Updated weights for policy 1, policy_version 1311549 (0.0007) [2023-12-27 00:55:40,921][105692] Updated weights for policy 0, policy_version 1309937 (0.0007) [2023-12-27 00:55:40,966][105692] Updated weights for policy 0, policy_version 1309947 (0.0008) [2023-12-27 00:55:41,027][105692] Updated weights for policy 0, policy_version 1309957 (0.0010) [2023-12-27 00:55:41,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 671195136. Throughput: 0: 9899.3, 1: 9737.7. Samples: 671206108. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:55:41,063][104569] Avg episode reward: [(0, '8632.678'), (1, '9264.194')] [2023-12-27 00:55:41,092][105692] Updated weights for policy 0, policy_version 1309967 (0.0008) [2023-12-27 00:55:41,585][105620] Updated weights for policy 1, policy_version 1311559 (0.0008) [2023-12-27 00:55:41,652][105620] Updated weights for policy 1, policy_version 1311569 (0.0009) [2023-12-27 00:55:41,727][105620] Updated weights for policy 1, policy_version 1311579 (0.0008) [2023-12-27 00:55:41,920][105692] Updated weights for policy 0, policy_version 1309977 (0.0006) [2023-12-27 00:55:41,986][105692] Updated weights for policy 0, policy_version 1309987 (0.0008) [2023-12-27 00:55:42,054][105692] Updated weights for policy 0, policy_version 1309997 (0.0007) [2023-12-27 00:55:42,441][105620] Updated weights for policy 1, policy_version 1311589 (0.0009) [2023-12-27 00:55:42,508][105620] Updated weights for policy 1, policy_version 1311599 (0.0009) [2023-12-27 00:55:42,564][105620] Updated weights for policy 1, policy_version 1311609 (0.0007) [2023-12-27 00:55:42,711][105692] Updated weights for policy 0, policy_version 1310007 (0.0009) [2023-12-27 00:55:42,770][105692] Updated weights for policy 0, policy_version 1310017 (0.0009) [2023-12-27 00:55:42,830][105692] Updated weights for policy 0, policy_version 1310027 (0.0010) [2023-12-27 00:55:43,231][105620] Updated weights for policy 1, policy_version 1311619 (0.0008) [2023-12-27 00:55:43,297][105620] Updated weights for policy 1, policy_version 1311629 (0.0005) [2023-12-27 00:55:43,352][105620] Updated weights for policy 1, policy_version 1311639 (0.0005) [2023-12-27 00:55:43,727][105692] Updated weights for policy 0, policy_version 1310037 (0.0011) [2023-12-27 00:55:43,788][105692] Updated weights for policy 0, policy_version 1310047 (0.0010) [2023-12-27 00:55:43,840][105692] Updated weights for policy 0, policy_version 1310057 (0.0009) [2023-12-27 00:55:43,849][105620] Updated weights for policy 1, policy_version 1311649 (0.0006) [2023-12-27 00:55:43,897][105620] Updated weights for policy 1, policy_version 1311659 (0.0006) [2023-12-27 00:55:43,954][105620] Updated weights for policy 1, policy_version 1311669 (0.0005) [2023-12-27 00:55:44,019][105620] Updated weights for policy 1, policy_version 1311679 (0.0005) [2023-12-27 00:55:44,594][105620] Updated weights for policy 1, policy_version 1311689 (0.0009) [2023-12-27 00:55:44,623][105692] Updated weights for policy 0, policy_version 1310067 (0.0007) [2023-12-27 00:55:44,646][105620] Updated weights for policy 1, policy_version 1311699 (0.0005) [2023-12-27 00:55:44,670][105692] Updated weights for policy 0, policy_version 1310077 (0.0008) [2023-12-27 00:55:44,699][105620] Updated weights for policy 1, policy_version 1311709 (0.0009) [2023-12-27 00:55:44,722][105692] Updated weights for policy 0, policy_version 1310087 (0.0009) [2023-12-27 00:55:45,330][105620] Updated weights for policy 1, policy_version 1311719 (0.0011) [2023-12-27 00:55:45,401][105620] Updated weights for policy 1, policy_version 1311729 (0.0007) [2023-12-27 00:55:45,471][105620] Updated weights for policy 1, policy_version 1311739 (0.0008) [2023-12-27 00:55:45,541][105692] Updated weights for policy 0, policy_version 1310097 (0.0009) [2023-12-27 00:55:45,609][105692] Updated weights for policy 0, policy_version 1310107 (0.0009) [2023-12-27 00:55:45,669][105692] Updated weights for policy 0, policy_version 1310117 (0.0008) [2023-12-27 00:55:45,728][105692] Updated weights for policy 0, policy_version 1310127 (0.0006) [2023-12-27 00:55:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 671293440. Throughput: 0: 9809.6, 1: 9790.1. Samples: 671264600. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:55:46,063][104569] Avg episode reward: [(0, '8634.188'), (1, '9177.802')] [2023-12-27 00:55:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001310128_335446016.pth... [2023-12-27 00:55:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001311744_335847424.pth... [2023-12-27 00:55:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001310624_335560704.pth [2023-12-27 00:55:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001308976_335151104.pth [2023-12-27 00:55:46,139][105620] Updated weights for policy 1, policy_version 1311749 (0.0008) [2023-12-27 00:55:46,198][105620] Updated weights for policy 1, policy_version 1311759 (0.0010) [2023-12-27 00:55:46,246][105620] Updated weights for policy 1, policy_version 1311769 (0.0010) [2023-12-27 00:55:46,452][105692] Updated weights for policy 0, policy_version 1310137 (0.0008) [2023-12-27 00:55:46,511][105692] Updated weights for policy 0, policy_version 1310147 (0.0008) [2023-12-27 00:55:46,525][105585] KL-divergence is very high: 104.4693 [2023-12-27 00:55:46,565][105585] KL-divergence is very high: 131.5782 [2023-12-27 00:55:46,565][105692] Updated weights for policy 0, policy_version 1310157 (0.0008) [2023-12-27 00:55:46,952][105620] Updated weights for policy 1, policy_version 1311779 (0.0009) [2023-12-27 00:55:47,014][105620] Updated weights for policy 1, policy_version 1311789 (0.0007) [2023-12-27 00:55:47,070][105620] Updated weights for policy 1, policy_version 1311799 (0.0006) [2023-12-27 00:55:47,336][105692] Updated weights for policy 0, policy_version 1310167 (0.0008) [2023-12-27 00:55:47,392][105692] Updated weights for policy 0, policy_version 1310177 (0.0008) [2023-12-27 00:55:47,449][105692] Updated weights for policy 0, policy_version 1310187 (0.0008) [2023-12-27 00:55:47,744][105620] Updated weights for policy 1, policy_version 1311809 (0.0006) [2023-12-27 00:55:47,802][105620] Updated weights for policy 1, policy_version 1311819 (0.0011) [2023-12-27 00:55:47,866][105620] Updated weights for policy 1, policy_version 1311829 (0.0010) [2023-12-27 00:55:47,926][105620] Updated weights for policy 1, policy_version 1311839 (0.0010) [2023-12-27 00:55:48,213][105692] Updated weights for policy 0, policy_version 1310197 (0.0008) [2023-12-27 00:55:48,272][105692] Updated weights for policy 0, policy_version 1310207 (0.0009) [2023-12-27 00:55:48,347][105692] Updated weights for policy 0, policy_version 1310217 (0.0008) [2023-12-27 00:55:48,650][105620] Updated weights for policy 1, policy_version 1311849 (0.0011) [2023-12-27 00:55:48,715][105620] Updated weights for policy 1, policy_version 1311859 (0.0011) [2023-12-27 00:55:48,784][105620] Updated weights for policy 1, policy_version 1311869 (0.0011) [2023-12-27 00:55:49,115][105692] Updated weights for policy 0, policy_version 1310227 (0.0008) [2023-12-27 00:55:49,171][105692] Updated weights for policy 0, policy_version 1310237 (0.0008) [2023-12-27 00:55:49,235][105692] Updated weights for policy 0, policy_version 1310247 (0.0008) [2023-12-27 00:55:49,507][105620] Updated weights for policy 1, policy_version 1311879 (0.0010) [2023-12-27 00:55:49,558][105620] Updated weights for policy 1, policy_version 1311889 (0.0010) [2023-12-27 00:55:49,607][105620] Updated weights for policy 1, policy_version 1311899 (0.0010) [2023-12-27 00:55:49,993][105692] Updated weights for policy 0, policy_version 1310257 (0.0008) [2023-12-27 00:55:50,057][105692] Updated weights for policy 0, policy_version 1310267 (0.0008) [2023-12-27 00:55:50,120][105692] Updated weights for policy 0, policy_version 1310277 (0.0008) [2023-12-27 00:55:50,184][105692] Updated weights for policy 0, policy_version 1310287 (0.0008) [2023-12-27 00:55:50,369][105620] Updated weights for policy 1, policy_version 1311909 (0.0011) [2023-12-27 00:55:50,421][105620] Updated weights for policy 1, policy_version 1311919 (0.0011) [2023-12-27 00:55:50,470][105620] Updated weights for policy 1, policy_version 1311929 (0.0011) [2023-12-27 00:55:50,883][105692] Updated weights for policy 0, policy_version 1310297 (0.0007) [2023-12-27 00:55:50,945][105692] Updated weights for policy 0, policy_version 1310307 (0.0007) [2023-12-27 00:55:51,007][105692] Updated weights for policy 0, policy_version 1310317 (0.0006) [2023-12-27 00:55:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 671391744. Throughput: 0: 9751.4, 1: 9751.4. Samples: 671379292. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:55:51,063][104569] Avg episode reward: [(0, '8454.468'), (1, '9089.965')] [2023-12-27 00:55:51,273][105620] Updated weights for policy 1, policy_version 1311939 (0.0010) [2023-12-27 00:55:51,338][105620] Updated weights for policy 1, policy_version 1311949 (0.0011) [2023-12-27 00:55:51,403][105620] Updated weights for policy 1, policy_version 1311959 (0.0010) [2023-12-27 00:55:51,693][105692] Updated weights for policy 0, policy_version 1310327 (0.0008) [2023-12-27 00:55:51,758][105692] Updated weights for policy 0, policy_version 1310337 (0.0009) [2023-12-27 00:55:51,819][105692] Updated weights for policy 0, policy_version 1310347 (0.0008) [2023-12-27 00:55:52,158][105620] Updated weights for policy 1, policy_version 1311969 (0.0011) [2023-12-27 00:55:52,217][105620] Updated weights for policy 1, policy_version 1311979 (0.0011) [2023-12-27 00:55:52,281][105620] Updated weights for policy 1, policy_version 1311989 (0.0011) [2023-12-27 00:55:52,349][105620] Updated weights for policy 1, policy_version 1311999 (0.0007) [2023-12-27 00:55:52,625][105692] Updated weights for policy 0, policy_version 1310357 (0.0008) [2023-12-27 00:55:52,683][105692] Updated weights for policy 0, policy_version 1310367 (0.0009) [2023-12-27 00:55:52,745][105692] Updated weights for policy 0, policy_version 1310377 (0.0009) [2023-12-27 00:55:52,962][105620] Updated weights for policy 1, policy_version 1312009 (0.0007) [2023-12-27 00:55:53,012][105620] Updated weights for policy 1, policy_version 1312019 (0.0008) [2023-12-27 00:55:53,061][105620] Updated weights for policy 1, policy_version 1312029 (0.0008) [2023-12-27 00:55:53,521][105692] Updated weights for policy 0, policy_version 1310387 (0.0010) [2023-12-27 00:55:53,585][105692] Updated weights for policy 0, policy_version 1310397 (0.0010) [2023-12-27 00:55:53,633][105692] Updated weights for policy 0, policy_version 1310407 (0.0010) [2023-12-27 00:55:53,722][105620] Updated weights for policy 1, policy_version 1312039 (0.0009) [2023-12-27 00:55:53,774][105620] Updated weights for policy 1, policy_version 1312049 (0.0005) [2023-12-27 00:55:53,833][105620] Updated weights for policy 1, policy_version 1312059 (0.0009) [2023-12-27 00:55:54,232][105692] Updated weights for policy 0, policy_version 1310417 (0.0010) [2023-12-27 00:55:54,284][105692] Updated weights for policy 0, policy_version 1310427 (0.0010) [2023-12-27 00:55:54,336][105692] Updated weights for policy 0, policy_version 1310437 (0.0010) [2023-12-27 00:55:54,398][105692] Updated weights for policy 0, policy_version 1310447 (0.0010) [2023-12-27 00:55:54,459][105620] Updated weights for policy 1, policy_version 1312069 (0.0008) [2023-12-27 00:55:54,517][105620] Updated weights for policy 1, policy_version 1312079 (0.0008) [2023-12-27 00:55:54,578][105620] Updated weights for policy 1, policy_version 1312089 (0.0010) [2023-12-27 00:55:55,072][105692] Updated weights for policy 0, policy_version 1310457 (0.0007) [2023-12-27 00:55:55,141][105692] Updated weights for policy 0, policy_version 1310467 (0.0009) [2023-12-27 00:55:55,209][105692] Updated weights for policy 0, policy_version 1310477 (0.0008) [2023-12-27 00:55:55,252][105620] Updated weights for policy 1, policy_version 1312099 (0.0006) [2023-12-27 00:55:55,300][105620] Updated weights for policy 1, policy_version 1312109 (0.0005) [2023-12-27 00:55:55,354][105620] Updated weights for policy 1, policy_version 1312119 (0.0005) [2023-12-27 00:55:55,827][105692] Updated weights for policy 0, policy_version 1310487 (0.0009) [2023-12-27 00:55:55,885][105692] Updated weights for policy 0, policy_version 1310497 (0.0010) [2023-12-27 00:55:55,946][105692] Updated weights for policy 0, policy_version 1310507 (0.0010) [2023-12-27 00:55:56,012][105620] Updated weights for policy 1, policy_version 1312129 (0.0007) [2023-12-27 00:55:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 671490048. Throughput: 0: 9834.7, 1: 9797.3. Samples: 671498372. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:55:56,063][104569] Avg episode reward: [(0, '8456.685'), (1, '9267.970')] [2023-12-27 00:55:56,071][105620] Updated weights for policy 1, policy_version 1312139 (0.0006) [2023-12-27 00:55:56,118][105620] Updated weights for policy 1, policy_version 1312149 (0.0007) [2023-12-27 00:55:56,174][105620] Updated weights for policy 1, policy_version 1312159 (0.0009) [2023-12-27 00:55:56,685][105692] Updated weights for policy 0, policy_version 1310517 (0.0010) [2023-12-27 00:55:56,746][105692] Updated weights for policy 0, policy_version 1310527 (0.0010) [2023-12-27 00:55:56,805][105692] Updated weights for policy 0, policy_version 1310537 (0.0009) [2023-12-27 00:55:56,901][105620] Updated weights for policy 1, policy_version 1312169 (0.0010) [2023-12-27 00:55:56,948][105620] Updated weights for policy 1, policy_version 1312179 (0.0010) [2023-12-27 00:55:56,995][105620] Updated weights for policy 1, policy_version 1312189 (0.0010) [2023-12-27 00:55:57,381][105692] Updated weights for policy 0, policy_version 1310547 (0.0005) [2023-12-27 00:55:57,446][105692] Updated weights for policy 0, policy_version 1310557 (0.0005) [2023-12-27 00:55:57,510][105692] Updated weights for policy 0, policy_version 1310567 (0.0005) [2023-12-27 00:55:57,675][105620] Updated weights for policy 1, policy_version 1312199 (0.0010) [2023-12-27 00:55:57,728][105620] Updated weights for policy 1, policy_version 1312209 (0.0009) [2023-12-27 00:55:57,784][105620] Updated weights for policy 1, policy_version 1312219 (0.0009) [2023-12-27 00:55:57,988][105692] Updated weights for policy 0, policy_version 1310577 (0.0005) [2023-12-27 00:55:58,043][105692] Updated weights for policy 0, policy_version 1310587 (0.0005) [2023-12-27 00:55:58,088][105692] Updated weights for policy 0, policy_version 1310597 (0.0005) [2023-12-27 00:55:58,150][105692] Updated weights for policy 0, policy_version 1310607 (0.0009) [2023-12-27 00:55:58,458][105620] Updated weights for policy 1, policy_version 1312229 (0.0010) [2023-12-27 00:55:58,531][105620] Updated weights for policy 1, policy_version 1312239 (0.0009) [2023-12-27 00:55:58,603][105620] Updated weights for policy 1, policy_version 1312249 (0.0007) [2023-12-27 00:55:59,005][105692] Updated weights for policy 0, policy_version 1310617 (0.0010) [2023-12-27 00:55:59,064][105692] Updated weights for policy 0, policy_version 1310627 (0.0010) [2023-12-27 00:55:59,126][105692] Updated weights for policy 0, policy_version 1310637 (0.0010) [2023-12-27 00:55:59,338][105620] Updated weights for policy 1, policy_version 1312259 (0.0008) [2023-12-27 00:55:59,414][105620] Updated weights for policy 1, policy_version 1312269 (0.0009) [2023-12-27 00:55:59,483][105620] Updated weights for policy 1, policy_version 1312279 (0.0008) [2023-12-27 00:55:59,862][105692] Updated weights for policy 0, policy_version 1310647 (0.0011) [2023-12-27 00:55:59,937][105692] Updated weights for policy 0, policy_version 1310657 (0.0011) [2023-12-27 00:55:59,990][105692] Updated weights for policy 0, policy_version 1310667 (0.0011) [2023-12-27 00:56:00,139][105620] Updated weights for policy 1, policy_version 1312289 (0.0007) [2023-12-27 00:56:00,199][105620] Updated weights for policy 1, policy_version 1312299 (0.0006) [2023-12-27 00:56:00,256][105620] Updated weights for policy 1, policy_version 1312309 (0.0008) [2023-12-27 00:56:00,316][105620] Updated weights for policy 1, policy_version 1312319 (0.0008) [2023-12-27 00:56:00,686][105692] Updated weights for policy 0, policy_version 1310677 (0.0008) [2023-12-27 00:56:00,753][105692] Updated weights for policy 0, policy_version 1310687 (0.0008) [2023-12-27 00:56:00,810][105692] Updated weights for policy 0, policy_version 1310697 (0.0009) [2023-12-27 00:56:00,847][105620] Updated weights for policy 1, policy_version 1312329 (0.0006) [2023-12-27 00:56:00,910][105620] Updated weights for policy 1, policy_version 1312339 (0.0009) [2023-12-27 00:56:00,964][105620] Updated weights for policy 1, policy_version 1312349 (0.0009) [2023-12-27 00:56:01,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 671596544. Throughput: 0: 9939.6, 1: 9789.0. Samples: 671559432. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:56:01,062][104569] Avg episode reward: [(0, '8547.572'), (1, '9177.664')] [2023-12-27 00:56:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001310704_335593472.pth... [2023-12-27 00:56:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001312352_336003072.pth... [2023-12-27 00:56:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001309584_335306752.pth [2023-12-27 00:56:01,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001311200_335708160.pth [2023-12-27 00:56:01,503][105692] Updated weights for policy 0, policy_version 1310707 (0.0008) [2023-12-27 00:56:01,559][105692] Updated weights for policy 0, policy_version 1310717 (0.0007) [2023-12-27 00:56:01,611][105692] Updated weights for policy 0, policy_version 1310727 (0.0008) [2023-12-27 00:56:01,710][105620] Updated weights for policy 1, policy_version 1312359 (0.0009) [2023-12-27 00:56:01,774][105620] Updated weights for policy 1, policy_version 1312369 (0.0009) [2023-12-27 00:56:01,838][105620] Updated weights for policy 1, policy_version 1312379 (0.0009) [2023-12-27 00:56:02,309][105692] Updated weights for policy 0, policy_version 1310737 (0.0010) [2023-12-27 00:56:02,372][105692] Updated weights for policy 0, policy_version 1310747 (0.0009) [2023-12-27 00:56:02,433][105692] Updated weights for policy 0, policy_version 1310757 (0.0006) [2023-12-27 00:56:02,495][105692] Updated weights for policy 0, policy_version 1310767 (0.0006) [2023-12-27 00:56:02,595][105620] Updated weights for policy 1, policy_version 1312389 (0.0010) [2023-12-27 00:56:02,661][105620] Updated weights for policy 1, policy_version 1312399 (0.0011) [2023-12-27 00:56:02,722][105620] Updated weights for policy 1, policy_version 1312409 (0.0010) [2023-12-27 00:56:03,171][105692] Updated weights for policy 0, policy_version 1310777 (0.0010) [2023-12-27 00:56:03,225][105692] Updated weights for policy 0, policy_version 1310787 (0.0010) [2023-12-27 00:56:03,290][105692] Updated weights for policy 0, policy_version 1310797 (0.0010) [2023-12-27 00:56:03,306][105620] Updated weights for policy 1, policy_version 1312419 (0.0005) [2023-12-27 00:56:03,359][105620] Updated weights for policy 1, policy_version 1312429 (0.0005) [2023-12-27 00:56:03,412][105620] Updated weights for policy 1, policy_version 1312439 (0.0007) [2023-12-27 00:56:04,058][105620] Updated weights for policy 1, policy_version 1312449 (0.0010) [2023-12-27 00:56:04,113][105620] Updated weights for policy 1, policy_version 1312459 (0.0008) [2023-12-27 00:56:04,116][105692] Updated weights for policy 0, policy_version 1310807 (0.0008) [2023-12-27 00:56:04,172][105620] Updated weights for policy 1, policy_version 1312469 (0.0008) [2023-12-27 00:56:04,174][105692] Updated weights for policy 0, policy_version 1310817 (0.0006) [2023-12-27 00:56:04,229][105692] Updated weights for policy 0, policy_version 1310827 (0.0006) [2023-12-27 00:56:04,231][105620] Updated weights for policy 1, policy_version 1312479 (0.0007) [2023-12-27 00:56:04,960][105620] Updated weights for policy 1, policy_version 1312489 (0.0006) [2023-12-27 00:56:05,009][105692] Updated weights for policy 0, policy_version 1310837 (0.0008) [2023-12-27 00:56:05,015][105620] Updated weights for policy 1, policy_version 1312499 (0.0008) [2023-12-27 00:56:05,066][105692] Updated weights for policy 0, policy_version 1310847 (0.0006) [2023-12-27 00:56:05,068][105620] Updated weights for policy 1, policy_version 1312509 (0.0007) [2023-12-27 00:56:05,117][105692] Updated weights for policy 0, policy_version 1310857 (0.0007) [2023-12-27 00:56:05,780][105620] Updated weights for policy 1, policy_version 1312519 (0.0008) [2023-12-27 00:56:05,835][105620] Updated weights for policy 1, policy_version 1312529 (0.0009) [2023-12-27 00:56:05,880][105692] Updated weights for policy 0, policy_version 1310867 (0.0008) [2023-12-27 00:56:05,894][105620] Updated weights for policy 1, policy_version 1312539 (0.0008) [2023-12-27 00:56:05,938][105692] Updated weights for policy 0, policy_version 1310877 (0.0008) [2023-12-27 00:56:05,997][105692] Updated weights for policy 0, policy_version 1310887 (0.0009) [2023-12-27 00:56:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 671694848. Throughput: 0: 9778.7, 1: 9860.9. Samples: 671678276. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:56:06,063][104569] Avg episode reward: [(0, '8996.731'), (1, '9177.739')] [2023-12-27 00:56:06,605][105620] Updated weights for policy 1, policy_version 1312549 (0.0006) [2023-12-27 00:56:06,661][105620] Updated weights for policy 1, policy_version 1312559 (0.0006) [2023-12-27 00:56:06,714][105620] Updated weights for policy 1, policy_version 1312569 (0.0005) [2023-12-27 00:56:06,838][105692] Updated weights for policy 0, policy_version 1310897 (0.0009) [2023-12-27 00:56:06,900][105692] Updated weights for policy 0, policy_version 1310907 (0.0009) [2023-12-27 00:56:06,963][105692] Updated weights for policy 0, policy_version 1310918 (0.0011) [2023-12-27 00:56:07,029][105692] Updated weights for policy 0, policy_version 1310928 (0.0010) [2023-12-27 00:56:07,239][105620] Updated weights for policy 1, policy_version 1312579 (0.0005) [2023-12-27 00:56:07,304][105620] Updated weights for policy 1, policy_version 1312589 (0.0006) [2023-12-27 00:56:07,360][105620] Updated weights for policy 1, policy_version 1312599 (0.0006) [2023-12-27 00:56:07,904][105620] Updated weights for policy 1, policy_version 1312609 (0.0006) [2023-12-27 00:56:07,945][105692] Updated weights for policy 0, policy_version 1310938 (0.0007) [2023-12-27 00:56:07,951][105620] Updated weights for policy 1, policy_version 1312619 (0.0009) [2023-12-27 00:56:08,002][105692] Updated weights for policy 0, policy_version 1310948 (0.0008) [2023-12-27 00:56:08,007][105620] Updated weights for policy 1, policy_version 1312629 (0.0007) [2023-12-27 00:56:08,063][105692] Updated weights for policy 0, policy_version 1310958 (0.0008) [2023-12-27 00:56:08,069][105620] Updated weights for policy 1, policy_version 1312639 (0.0007) [2023-12-27 00:56:08,819][105692] Updated weights for policy 0, policy_version 1310968 (0.0009) [2023-12-27 00:56:08,851][105620] Updated weights for policy 1, policy_version 1312649 (0.0006) [2023-12-27 00:56:08,884][105692] Updated weights for policy 0, policy_version 1310978 (0.0008) [2023-12-27 00:56:08,921][105620] Updated weights for policy 1, policy_version 1312659 (0.0006) [2023-12-27 00:56:08,955][105692] Updated weights for policy 0, policy_version 1310988 (0.0007) [2023-12-27 00:56:08,988][105620] Updated weights for policy 1, policy_version 1312669 (0.0006) [2023-12-27 00:56:09,630][105620] Updated weights for policy 1, policy_version 1312679 (0.0009) [2023-12-27 00:56:09,689][105620] Updated weights for policy 1, policy_version 1312689 (0.0010) [2023-12-27 00:56:09,699][105692] Updated weights for policy 0, policy_version 1310998 (0.0007) [2023-12-27 00:56:09,752][105620] Updated weights for policy 1, policy_version 1312699 (0.0011) [2023-12-27 00:56:09,758][105692] Updated weights for policy 0, policy_version 1311008 (0.0006) [2023-12-27 00:56:09,823][105692] Updated weights for policy 0, policy_version 1311018 (0.0006) [2023-12-27 00:56:10,531][105692] Updated weights for policy 0, policy_version 1311028 (0.0008) [2023-12-27 00:56:10,557][105620] Updated weights for policy 1, policy_version 1312709 (0.0010) [2023-12-27 00:56:10,585][105692] Updated weights for policy 0, policy_version 1311038 (0.0007) [2023-12-27 00:56:10,613][105620] Updated weights for policy 1, policy_version 1312719 (0.0009) [2023-12-27 00:56:10,646][105692] Updated weights for policy 0, policy_version 1311048 (0.0006) [2023-12-27 00:56:10,661][105620] Updated weights for policy 1, policy_version 1312729 (0.0010) [2023-12-27 00:56:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 671784960. Throughput: 0: 9634.8, 1: 9895.1. Samples: 671792464. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:56:11,062][104569] Avg episode reward: [(0, '8998.438'), (1, '9175.928')] [2023-12-27 00:56:11,383][105620] Updated weights for policy 1, policy_version 1312739 (0.0010) [2023-12-27 00:56:11,440][105692] Updated weights for policy 0, policy_version 1311058 (0.0006) [2023-12-27 00:56:11,452][105620] Updated weights for policy 1, policy_version 1312749 (0.0009) [2023-12-27 00:56:11,501][105692] Updated weights for policy 0, policy_version 1311068 (0.0011) [2023-12-27 00:56:11,520][105620] Updated weights for policy 1, policy_version 1312759 (0.0011) [2023-12-27 00:56:11,563][105692] Updated weights for policy 0, policy_version 1311078 (0.0011) [2023-12-27 00:56:11,631][105692] Updated weights for policy 0, policy_version 1311088 (0.0009) [2023-12-27 00:56:12,226][105620] Updated weights for policy 1, policy_version 1312769 (0.0011) [2023-12-27 00:56:12,293][105620] Updated weights for policy 1, policy_version 1312779 (0.0011) [2023-12-27 00:56:12,367][105692] Updated weights for policy 0, policy_version 1311098 (0.0008) [2023-12-27 00:56:12,368][105620] Updated weights for policy 1, policy_version 1312789 (0.0011) [2023-12-27 00:56:12,430][105620] Updated weights for policy 1, policy_version 1312799 (0.0007) [2023-12-27 00:56:12,438][105692] Updated weights for policy 0, policy_version 1311108 (0.0009) [2023-12-27 00:56:12,499][105692] Updated weights for policy 0, policy_version 1311118 (0.0009) [2023-12-27 00:56:13,137][105692] Updated weights for policy 0, policy_version 1311128 (0.0006) [2023-12-27 00:56:13,157][105620] Updated weights for policy 1, policy_version 1312809 (0.0010) [2023-12-27 00:56:13,203][105692] Updated weights for policy 0, policy_version 1311138 (0.0007) [2023-12-27 00:56:13,215][105620] Updated weights for policy 1, policy_version 1312819 (0.0010) [2023-12-27 00:56:13,261][105692] Updated weights for policy 0, policy_version 1311148 (0.0010) [2023-12-27 00:56:13,276][105620] Updated weights for policy 1, policy_version 1312829 (0.0010) [2023-12-27 00:56:13,907][105620] Updated weights for policy 1, policy_version 1312839 (0.0007) [2023-12-27 00:56:13,966][105620] Updated weights for policy 1, policy_version 1312849 (0.0006) [2023-12-27 00:56:13,973][105692] Updated weights for policy 0, policy_version 1311158 (0.0009) [2023-12-27 00:56:14,026][105692] Updated weights for policy 0, policy_version 1311168 (0.0009) [2023-12-27 00:56:14,030][105620] Updated weights for policy 1, policy_version 1312859 (0.0010) [2023-12-27 00:56:14,090][105692] Updated weights for policy 0, policy_version 1311178 (0.0006) [2023-12-27 00:56:14,614][105620] Updated weights for policy 1, policy_version 1312869 (0.0008) [2023-12-27 00:56:14,678][105620] Updated weights for policy 1, policy_version 1312879 (0.0005) [2023-12-27 00:56:14,747][105620] Updated weights for policy 1, policy_version 1312889 (0.0007) [2023-12-27 00:56:14,855][105692] Updated weights for policy 0, policy_version 1311188 (0.0010) [2023-12-27 00:56:14,919][105692] Updated weights for policy 0, policy_version 1311198 (0.0006) [2023-12-27 00:56:14,979][105692] Updated weights for policy 0, policy_version 1311208 (0.0011) [2023-12-27 00:56:15,419][105620] Updated weights for policy 1, policy_version 1312899 (0.0009) [2023-12-27 00:56:15,479][105620] Updated weights for policy 1, policy_version 1312909 (0.0010) [2023-12-27 00:56:15,533][105620] Updated weights for policy 1, policy_version 1312919 (0.0010) [2023-12-27 00:56:15,603][105692] Updated weights for policy 0, policy_version 1311218 (0.0009) [2023-12-27 00:56:15,653][105692] Updated weights for policy 0, policy_version 1311228 (0.0005) [2023-12-27 00:56:15,708][105692] Updated weights for policy 0, policy_version 1311238 (0.0005) [2023-12-27 00:56:15,754][105692] Updated weights for policy 0, policy_version 1311248 (0.0005) [2023-12-27 00:56:16,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 671883264. Throughput: 0: 9516.4, 1: 9911.7. Samples: 671851080. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:56:16,063][104569] Avg episode reward: [(0, '8361.603'), (1, '9086.629')] [2023-12-27 00:56:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001312928_336150528.pth... [2023-12-27 00:56:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001311248_335732736.pth... [2023-12-27 00:56:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001311744_335847424.pth [2023-12-27 00:56:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001310128_335446016.pth [2023-12-27 00:56:16,288][105620] Updated weights for policy 1, policy_version 1312929 (0.0010) [2023-12-27 00:56:16,354][105620] Updated weights for policy 1, policy_version 1312939 (0.0009) [2023-12-27 00:56:16,417][105620] Updated weights for policy 1, policy_version 1312949 (0.0011) [2023-12-27 00:56:16,458][105692] Updated weights for policy 0, policy_version 1311258 (0.0005) [2023-12-27 00:56:16,475][105620] Updated weights for policy 1, policy_version 1312959 (0.0010) [2023-12-27 00:56:16,508][105692] Updated weights for policy 0, policy_version 1311268 (0.0007) [2023-12-27 00:56:16,554][105692] Updated weights for policy 0, policy_version 1311278 (0.0008) [2023-12-27 00:56:17,201][105620] Updated weights for policy 1, policy_version 1312969 (0.0009) [2023-12-27 00:56:17,264][105620] Updated weights for policy 1, policy_version 1312979 (0.0005) [2023-12-27 00:56:17,321][105620] Updated weights for policy 1, policy_version 1312989 (0.0007) [2023-12-27 00:56:17,328][105692] Updated weights for policy 0, policy_version 1311288 (0.0007) [2023-12-27 00:56:17,390][105692] Updated weights for policy 0, policy_version 1311298 (0.0009) [2023-12-27 00:56:17,451][105692] Updated weights for policy 0, policy_version 1311308 (0.0010) [2023-12-27 00:56:17,887][105620] Updated weights for policy 1, policy_version 1312999 (0.0008) [2023-12-27 00:56:17,939][105620] Updated weights for policy 1, policy_version 1313009 (0.0008) [2023-12-27 00:56:18,007][105620] Updated weights for policy 1, policy_version 1313019 (0.0005) [2023-12-27 00:56:18,311][105692] Updated weights for policy 0, policy_version 1311318 (0.0010) [2023-12-27 00:56:18,388][105692] Updated weights for policy 0, policy_version 1311328 (0.0010) [2023-12-27 00:56:18,452][105692] Updated weights for policy 0, policy_version 1311338 (0.0011) [2023-12-27 00:56:18,628][105620] Updated weights for policy 1, policy_version 1313029 (0.0007) [2023-12-27 00:56:18,673][105620] Updated weights for policy 1, policy_version 1313039 (0.0008) [2023-12-27 00:56:18,722][105620] Updated weights for policy 1, policy_version 1313049 (0.0008) [2023-12-27 00:56:19,204][105692] Updated weights for policy 0, policy_version 1311348 (0.0010) [2023-12-27 00:56:19,261][105692] Updated weights for policy 0, policy_version 1311358 (0.0009) [2023-12-27 00:56:19,314][105692] Updated weights for policy 0, policy_version 1311368 (0.0011) [2023-12-27 00:56:19,328][105620] Updated weights for policy 1, policy_version 1313059 (0.0005) [2023-12-27 00:56:19,396][105620] Updated weights for policy 1, policy_version 1313069 (0.0008) [2023-12-27 00:56:19,454][105620] Updated weights for policy 1, policy_version 1313079 (0.0009) [2023-12-27 00:56:20,060][105620] Updated weights for policy 1, policy_version 1313089 (0.0008) [2023-12-27 00:56:20,123][105620] Updated weights for policy 1, policy_version 1313099 (0.0006) [2023-12-27 00:56:20,177][105692] Updated weights for policy 0, policy_version 1311378 (0.0009) [2023-12-27 00:56:20,180][105620] Updated weights for policy 1, policy_version 1313109 (0.0006) [2023-12-27 00:56:20,226][105692] Updated weights for policy 0, policy_version 1311388 (0.0008) [2023-12-27 00:56:20,243][105620] Updated weights for policy 1, policy_version 1313119 (0.0007) [2023-12-27 00:56:20,282][105692] Updated weights for policy 0, policy_version 1311398 (0.0009) [2023-12-27 00:56:20,861][105620] Updated weights for policy 1, policy_version 1313129 (0.0006) [2023-12-27 00:56:20,920][105620] Updated weights for policy 1, policy_version 1313139 (0.0006) [2023-12-27 00:56:20,974][105620] Updated weights for policy 1, policy_version 1313149 (0.0007) [2023-12-27 00:56:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 671981568. Throughput: 0: 9426.6, 1: 10112.3. Samples: 671970216. Policy #0 lag: (min: 17.0, avg: 35.6, max: 49.0) [2023-12-27 00:56:21,062][104569] Avg episode reward: [(0, '8177.729'), (1, '9175.603')] [2023-12-27 00:56:21,167][105692] Updated weights for policy 0, policy_version 1311409 (0.0009) [2023-12-27 00:56:21,227][105692] Updated weights for policy 0, policy_version 1311419 (0.0006) [2023-12-27 00:56:21,283][105692] Updated weights for policy 0, policy_version 1311429 (0.0009) [2023-12-27 00:56:21,340][105692] Updated weights for policy 0, policy_version 1311439 (0.0009) [2023-12-27 00:56:21,730][105620] Updated weights for policy 1, policy_version 1313160 (0.0010) [2023-12-27 00:56:21,800][105620] Updated weights for policy 1, policy_version 1313170 (0.0008) [2023-12-27 00:56:21,863][105620] Updated weights for policy 1, policy_version 1313180 (0.0006) [2023-12-27 00:56:22,112][105692] Updated weights for policy 0, policy_version 1311449 (0.0009) [2023-12-27 00:56:22,161][105692] Updated weights for policy 0, policy_version 1311459 (0.0008) [2023-12-27 00:56:22,229][105692] Updated weights for policy 0, policy_version 1311469 (0.0005) [2023-12-27 00:56:22,594][105620] Updated weights for policy 1, policy_version 1313190 (0.0008) [2023-12-27 00:56:22,653][105620] Updated weights for policy 1, policy_version 1313200 (0.0009) [2023-12-27 00:56:22,711][105620] Updated weights for policy 1, policy_version 1313210 (0.0008) [2023-12-27 00:56:22,889][105692] Updated weights for policy 0, policy_version 1311479 (0.0009) [2023-12-27 00:56:22,947][105692] Updated weights for policy 0, policy_version 1311489 (0.0009) [2023-12-27 00:56:23,006][105692] Updated weights for policy 0, policy_version 1311499 (0.0009) [2023-12-27 00:56:23,428][105620] Updated weights for policy 1, policy_version 1313220 (0.0009) [2023-12-27 00:56:23,475][105620] Updated weights for policy 1, policy_version 1313230 (0.0009) [2023-12-27 00:56:23,521][105620] Updated weights for policy 1, policy_version 1313240 (0.0008) [2023-12-27 00:56:23,789][105692] Updated weights for policy 0, policy_version 1311509 (0.0009) [2023-12-27 00:56:23,850][105692] Updated weights for policy 0, policy_version 1311519 (0.0009) [2023-12-27 00:56:23,898][105692] Updated weights for policy 0, policy_version 1311529 (0.0009) [2023-12-27 00:56:24,281][105620] Updated weights for policy 1, policy_version 1313250 (0.0009) [2023-12-27 00:56:24,346][105620] Updated weights for policy 1, policy_version 1313260 (0.0009) [2023-12-27 00:56:24,407][105620] Updated weights for policy 1, policy_version 1313270 (0.0009) [2023-12-27 00:56:24,468][105620] Updated weights for policy 1, policy_version 1313280 (0.0009) [2023-12-27 00:56:24,631][105692] Updated weights for policy 0, policy_version 1311539 (0.0008) [2023-12-27 00:56:24,687][105692] Updated weights for policy 0, policy_version 1311549 (0.0005) [2023-12-27 00:56:24,739][105692] Updated weights for policy 0, policy_version 1311559 (0.0007) [2023-12-27 00:56:25,174][105620] Updated weights for policy 1, policy_version 1313290 (0.0008) [2023-12-27 00:56:25,221][105620] Updated weights for policy 1, policy_version 1313300 (0.0009) [2023-12-27 00:56:25,247][105586] KL-divergence is very high: 100.0513 [2023-12-27 00:56:25,267][105620] Updated weights for policy 1, policy_version 1313310 (0.0008) [2023-12-27 00:56:25,478][105692] Updated weights for policy 0, policy_version 1311569 (0.0008) [2023-12-27 00:56:25,532][105692] Updated weights for policy 0, policy_version 1311579 (0.0009) [2023-12-27 00:56:25,586][105692] Updated weights for policy 0, policy_version 1311590 (0.0010) [2023-12-27 00:56:25,641][105692] Updated weights for policy 0, policy_version 1311600 (0.0008) [2023-12-27 00:56:25,981][105620] Updated weights for policy 1, policy_version 1313320 (0.0009) [2023-12-27 00:56:26,039][105620] Updated weights for policy 1, policy_version 1313331 (0.0010) [2023-12-27 00:56:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.2, 300 sec: 19605.2). Total num frames: 672071680. Throughput: 0: 9361.8, 1: 10145.5. Samples: 672083936. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:56:26,063][104569] Avg episode reward: [(0, '8096.607'), (1, '8812.417')] [2023-12-27 00:56:26,090][105620] Updated weights for policy 1, policy_version 1313342 (0.0010) [2023-12-27 00:56:26,358][105692] Updated weights for policy 0, policy_version 1311610 (0.0009) [2023-12-27 00:56:26,408][105692] Updated weights for policy 0, policy_version 1311620 (0.0009) [2023-12-27 00:56:26,460][105692] Updated weights for policy 0, policy_version 1311630 (0.0007) [2023-12-27 00:56:26,924][105620] Updated weights for policy 1, policy_version 1313352 (0.0006) [2023-12-27 00:56:26,993][105620] Updated weights for policy 1, policy_version 1313362 (0.0009) [2023-12-27 00:56:27,043][105620] Updated weights for policy 1, policy_version 1313372 (0.0008) [2023-12-27 00:56:27,092][105692] Updated weights for policy 0, policy_version 1311640 (0.0008) [2023-12-27 00:56:27,150][105692] Updated weights for policy 0, policy_version 1311651 (0.0011) [2023-12-27 00:56:27,206][105692] Updated weights for policy 0, policy_version 1311661 (0.0009) [2023-12-27 00:56:27,734][105620] Updated weights for policy 1, policy_version 1313382 (0.0006) [2023-12-27 00:56:27,780][105620] Updated weights for policy 1, policy_version 1313392 (0.0005) [2023-12-27 00:56:27,823][105620] Updated weights for policy 1, policy_version 1313402 (0.0005) [2023-12-27 00:56:27,904][105692] Updated weights for policy 0, policy_version 1311671 (0.0009) [2023-12-27 00:56:27,957][105692] Updated weights for policy 0, policy_version 1311683 (0.0010) [2023-12-27 00:56:28,008][105692] Updated weights for policy 0, policy_version 1311693 (0.0009) [2023-12-27 00:56:28,374][105620] Updated weights for policy 1, policy_version 1313412 (0.0006) [2023-12-27 00:56:28,436][105620] Updated weights for policy 1, policy_version 1313422 (0.0011) [2023-12-27 00:56:28,484][105620] Updated weights for policy 1, policy_version 1313432 (0.0010) [2023-12-27 00:56:28,890][105692] Updated weights for policy 0, policy_version 1311703 (0.0009) [2023-12-27 00:56:28,978][105692] Updated weights for policy 0, policy_version 1311713 (0.0009) [2023-12-27 00:56:29,040][105692] Updated weights for policy 0, policy_version 1311723 (0.0010) [2023-12-27 00:56:29,091][105620] Updated weights for policy 1, policy_version 1313442 (0.0010) [2023-12-27 00:56:29,139][105620] Updated weights for policy 1, policy_version 1313452 (0.0010) [2023-12-27 00:56:29,187][105620] Updated weights for policy 1, policy_version 1313462 (0.0010) [2023-12-27 00:56:29,244][105620] Updated weights for policy 1, policy_version 1313472 (0.0010) [2023-12-27 00:56:29,774][105692] Updated weights for policy 0, policy_version 1311733 (0.0009) [2023-12-27 00:56:29,839][105692] Updated weights for policy 0, policy_version 1311743 (0.0009) [2023-12-27 00:56:29,897][105692] Updated weights for policy 0, policy_version 1311753 (0.0009) [2023-12-27 00:56:30,015][105620] Updated weights for policy 1, policy_version 1313482 (0.0009) [2023-12-27 00:56:30,069][105620] Updated weights for policy 1, policy_version 1313492 (0.0009) [2023-12-27 00:56:30,127][105620] Updated weights for policy 1, policy_version 1313502 (0.0008) [2023-12-27 00:56:30,656][105692] Updated weights for policy 0, policy_version 1311763 (0.0009) [2023-12-27 00:56:30,713][105692] Updated weights for policy 0, policy_version 1311773 (0.0009) [2023-12-27 00:56:30,766][105692] Updated weights for policy 0, policy_version 1311783 (0.0009) [2023-12-27 00:56:30,812][105620] Updated weights for policy 1, policy_version 1313512 (0.0008) [2023-12-27 00:56:30,877][105620] Updated weights for policy 1, policy_version 1313522 (0.0009) [2023-12-27 00:56:30,933][105620] Updated weights for policy 1, policy_version 1313532 (0.0009) [2023-12-27 00:56:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.4, 300 sec: 19577.5). Total num frames: 672178176. Throughput: 0: 9421.6, 1: 10130.7. Samples: 672144452. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:56:31,063][104569] Avg episode reward: [(0, '8373.746'), (1, '7593.275')] [2023-12-27 00:56:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001311792_335872000.pth... [2023-12-27 00:56:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001313536_336306176.pth... [2023-12-27 00:56:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001310704_335593472.pth [2023-12-27 00:56:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001312352_336003072.pth [2023-12-27 00:56:31,536][105692] Updated weights for policy 0, policy_version 1311793 (0.0007) [2023-12-27 00:56:31,603][105692] Updated weights for policy 0, policy_version 1311803 (0.0006) [2023-12-27 00:56:31,674][105692] Updated weights for policy 0, policy_version 1311813 (0.0008) [2023-12-27 00:56:31,682][105620] Updated weights for policy 1, policy_version 1313542 (0.0007) [2023-12-27 00:56:31,745][105692] Updated weights for policy 0, policy_version 1311823 (0.0007) [2023-12-27 00:56:31,748][105620] Updated weights for policy 1, policy_version 1313552 (0.0008) [2023-12-27 00:56:31,810][105620] Updated weights for policy 1, policy_version 1313562 (0.0009) [2023-12-27 00:56:32,323][105692] Updated weights for policy 0, policy_version 1311833 (0.0009) [2023-12-27 00:56:32,384][105692] Updated weights for policy 0, policy_version 1311843 (0.0009) [2023-12-27 00:56:32,433][105692] Updated weights for policy 0, policy_version 1311853 (0.0009) [2023-12-27 00:56:32,562][105620] Updated weights for policy 1, policy_version 1313572 (0.0007) [2023-12-27 00:56:32,625][105620] Updated weights for policy 1, policy_version 1313582 (0.0009) [2023-12-27 00:56:32,685][105620] Updated weights for policy 1, policy_version 1313592 (0.0009) [2023-12-27 00:56:33,229][105620] Updated weights for policy 1, policy_version 1313602 (0.0007) [2023-12-27 00:56:33,279][105620] Updated weights for policy 1, policy_version 1313612 (0.0009) [2023-12-27 00:56:33,288][105692] Updated weights for policy 0, policy_version 1311863 (0.0006) [2023-12-27 00:56:33,326][105620] Updated weights for policy 1, policy_version 1313622 (0.0009) [2023-12-27 00:56:33,339][105692] Updated weights for policy 0, policy_version 1311873 (0.0005) [2023-12-27 00:56:33,373][105620] Updated weights for policy 1, policy_version 1313632 (0.0006) [2023-12-27 00:56:33,392][105692] Updated weights for policy 0, policy_version 1311883 (0.0009) [2023-12-27 00:56:34,060][105692] Updated weights for policy 0, policy_version 1311893 (0.0009) [2023-12-27 00:56:34,109][105692] Updated weights for policy 0, policy_version 1311903 (0.0008) [2023-12-27 00:56:34,164][105692] Updated weights for policy 0, policy_version 1311913 (0.0009) [2023-12-27 00:56:34,178][105620] Updated weights for policy 1, policy_version 1313642 (0.0008) [2023-12-27 00:56:34,240][105620] Updated weights for policy 1, policy_version 1313652 (0.0009) [2023-12-27 00:56:34,304][105620] Updated weights for policy 1, policy_version 1313662 (0.0008) [2023-12-27 00:56:34,946][105692] Updated weights for policy 0, policy_version 1311923 (0.0007) [2023-12-27 00:56:35,005][105692] Updated weights for policy 0, policy_version 1311933 (0.0010) [2023-12-27 00:56:35,011][105620] Updated weights for policy 1, policy_version 1313672 (0.0006) [2023-12-27 00:56:35,057][105692] Updated weights for policy 0, policy_version 1311943 (0.0006) [2023-12-27 00:56:35,060][105620] Updated weights for policy 1, policy_version 1313682 (0.0006) [2023-12-27 00:56:35,116][105620] Updated weights for policy 1, policy_version 1313692 (0.0007) [2023-12-27 00:56:35,797][105620] Updated weights for policy 1, policy_version 1313702 (0.0007) [2023-12-27 00:56:35,844][105692] Updated weights for policy 0, policy_version 1311953 (0.0007) [2023-12-27 00:56:35,846][105620] Updated weights for policy 1, policy_version 1313712 (0.0005) [2023-12-27 00:56:35,899][105620] Updated weights for policy 1, policy_version 1313722 (0.0007) [2023-12-27 00:56:35,901][105692] Updated weights for policy 0, policy_version 1311963 (0.0007) [2023-12-27 00:56:35,967][105692] Updated weights for policy 0, policy_version 1311973 (0.0006) [2023-12-27 00:56:36,030][105692] Updated weights for policy 0, policy_version 1311983 (0.0008) [2023-12-27 00:56:36,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 672276480. Throughput: 0: 9465.2, 1: 10103.2. Samples: 672259868. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:56:36,062][104569] Avg episode reward: [(0, '8461.158'), (1, '7858.814')] [2023-12-27 00:56:36,519][105620] Updated weights for policy 1, policy_version 1313732 (0.0009) [2023-12-27 00:56:36,586][105620] Updated weights for policy 1, policy_version 1313742 (0.0008) [2023-12-27 00:56:36,648][105620] Updated weights for policy 1, policy_version 1313752 (0.0010) [2023-12-27 00:56:36,855][105692] Updated weights for policy 0, policy_version 1311993 (0.0008) [2023-12-27 00:56:36,907][105692] Updated weights for policy 0, policy_version 1312003 (0.0008) [2023-12-27 00:56:36,957][105692] Updated weights for policy 0, policy_version 1312013 (0.0009) [2023-12-27 00:56:37,378][105620] Updated weights for policy 1, policy_version 1313762 (0.0010) [2023-12-27 00:56:37,429][105620] Updated weights for policy 1, policy_version 1313772 (0.0010) [2023-12-27 00:56:37,491][105620] Updated weights for policy 1, policy_version 1313782 (0.0009) [2023-12-27 00:56:37,549][105620] Updated weights for policy 1, policy_version 1313792 (0.0005) [2023-12-27 00:56:37,763][105692] Updated weights for policy 0, policy_version 1312023 (0.0009) [2023-12-27 00:56:37,816][105692] Updated weights for policy 0, policy_version 1312033 (0.0010) [2023-12-27 00:56:37,876][105692] Updated weights for policy 0, policy_version 1312044 (0.0011) [2023-12-27 00:56:38,163][105620] Updated weights for policy 1, policy_version 1313802 (0.0008) [2023-12-27 00:56:38,223][105620] Updated weights for policy 1, policy_version 1313812 (0.0005) [2023-12-27 00:56:38,276][105620] Updated weights for policy 1, policy_version 1313822 (0.0005) [2023-12-27 00:56:38,730][105692] Updated weights for policy 0, policy_version 1312054 (0.0009) [2023-12-27 00:56:38,787][105692] Updated weights for policy 0, policy_version 1312064 (0.0009) [2023-12-27 00:56:38,845][105692] Updated weights for policy 0, policy_version 1312074 (0.0009) [2023-12-27 00:56:38,902][105620] Updated weights for policy 1, policy_version 1313832 (0.0009) [2023-12-27 00:56:38,958][105620] Updated weights for policy 1, policy_version 1313842 (0.0009) [2023-12-27 00:56:39,017][105620] Updated weights for policy 1, policy_version 1313852 (0.0009) [2023-12-27 00:56:39,680][105620] Updated weights for policy 1, policy_version 1313862 (0.0007) [2023-12-27 00:56:39,720][105692] Updated weights for policy 0, policy_version 1312084 (0.0008) [2023-12-27 00:56:39,735][105620] Updated weights for policy 1, policy_version 1313872 (0.0007) [2023-12-27 00:56:39,770][105692] Updated weights for policy 0, policy_version 1312094 (0.0007) [2023-12-27 00:56:39,793][105620] Updated weights for policy 1, policy_version 1313882 (0.0008) [2023-12-27 00:56:39,830][105692] Updated weights for policy 0, policy_version 1312104 (0.0007) [2023-12-27 00:56:40,571][105620] Updated weights for policy 1, policy_version 1313892 (0.0008) [2023-12-27 00:56:40,616][105692] Updated weights for policy 0, policy_version 1312114 (0.0008) [2023-12-27 00:56:40,622][105620] Updated weights for policy 1, policy_version 1313902 (0.0008) [2023-12-27 00:56:40,680][105692] Updated weights for policy 0, policy_version 1312124 (0.0008) [2023-12-27 00:56:40,682][105620] Updated weights for policy 1, policy_version 1313912 (0.0006) [2023-12-27 00:56:40,743][105692] Updated weights for policy 0, policy_version 1312134 (0.0006) [2023-12-27 00:56:40,798][105692] Updated weights for policy 0, policy_version 1312144 (0.0009) [2023-12-27 00:56:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 672366592. Throughput: 0: 9330.8, 1: 10119.1. Samples: 672373616. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:56:41,063][104569] Avg episode reward: [(0, '8367.449'), (1, '8758.124')] [2023-12-27 00:56:41,462][105620] Updated weights for policy 1, policy_version 1313922 (0.0008) [2023-12-27 00:56:41,520][105620] Updated weights for policy 1, policy_version 1313932 (0.0007) [2023-12-27 00:56:41,570][105620] Updated weights for policy 1, policy_version 1313942 (0.0007) [2023-12-27 00:56:41,589][105692] Updated weights for policy 0, policy_version 1312154 (0.0008) [2023-12-27 00:56:41,631][105620] Updated weights for policy 1, policy_version 1313952 (0.0008) [2023-12-27 00:56:41,657][105692] Updated weights for policy 0, policy_version 1312164 (0.0009) [2023-12-27 00:56:41,716][105692] Updated weights for policy 0, policy_version 1312174 (0.0009) [2023-12-27 00:56:42,387][105620] Updated weights for policy 1, policy_version 1313962 (0.0007) [2023-12-27 00:56:42,447][105620] Updated weights for policy 1, policy_version 1313972 (0.0007) [2023-12-27 00:56:42,469][105692] Updated weights for policy 0, policy_version 1312184 (0.0010) [2023-12-27 00:56:42,507][105620] Updated weights for policy 1, policy_version 1313982 (0.0006) [2023-12-27 00:56:42,518][105692] Updated weights for policy 0, policy_version 1312194 (0.0011) [2023-12-27 00:56:42,582][105692] Updated weights for policy 0, policy_version 1312204 (0.0011) [2023-12-27 00:56:43,140][105620] Updated weights for policy 1, policy_version 1313992 (0.0009) [2023-12-27 00:56:43,207][105620] Updated weights for policy 1, policy_version 1314002 (0.0007) [2023-12-27 00:56:43,259][105620] Updated weights for policy 1, policy_version 1314012 (0.0005) [2023-12-27 00:56:43,298][105692] Updated weights for policy 0, policy_version 1312214 (0.0008) [2023-12-27 00:56:43,347][105692] Updated weights for policy 0, policy_version 1312224 (0.0008) [2023-12-27 00:56:43,402][105692] Updated weights for policy 0, policy_version 1312234 (0.0007) [2023-12-27 00:56:43,894][105620] Updated weights for policy 1, policy_version 1314022 (0.0007) [2023-12-27 00:56:43,955][105620] Updated weights for policy 1, policy_version 1314032 (0.0005) [2023-12-27 00:56:44,010][105620] Updated weights for policy 1, policy_version 1314042 (0.0005) [2023-12-27 00:56:44,130][105692] Updated weights for policy 0, policy_version 1312244 (0.0006) [2023-12-27 00:56:44,182][105692] Updated weights for policy 0, policy_version 1312254 (0.0007) [2023-12-27 00:56:44,237][105692] Updated weights for policy 0, policy_version 1312264 (0.0008) [2023-12-27 00:56:44,673][105620] Updated weights for policy 1, policy_version 1314052 (0.0006) [2023-12-27 00:56:44,726][105620] Updated weights for policy 1, policy_version 1314062 (0.0005) [2023-12-27 00:56:44,789][105620] Updated weights for policy 1, policy_version 1314072 (0.0006) [2023-12-27 00:56:44,992][105692] Updated weights for policy 0, policy_version 1312274 (0.0006) [2023-12-27 00:56:45,049][105692] Updated weights for policy 0, policy_version 1312284 (0.0009) [2023-12-27 00:56:45,105][105692] Updated weights for policy 0, policy_version 1312294 (0.0009) [2023-12-27 00:56:45,168][105692] Updated weights for policy 0, policy_version 1312304 (0.0009) [2023-12-27 00:56:45,454][105620] Updated weights for policy 1, policy_version 1314082 (0.0009) [2023-12-27 00:56:45,519][105620] Updated weights for policy 1, policy_version 1314092 (0.0008) [2023-12-27 00:56:45,570][105620] Updated weights for policy 1, policy_version 1314102 (0.0007) [2023-12-27 00:56:45,625][105620] Updated weights for policy 1, policy_version 1314112 (0.0008) [2023-12-27 00:56:45,963][105692] Updated weights for policy 0, policy_version 1312314 (0.0011) [2023-12-27 00:56:46,021][105692] Updated weights for policy 0, policy_version 1312324 (0.0011) [2023-12-27 00:56:46,062][104569] Fps is (10 sec: 18021.7, 60 sec: 19387.6, 300 sec: 19549.7). Total num frames: 672456704. Throughput: 0: 9251.6, 1: 10126.3. Samples: 672431444. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:56:46,063][104569] Avg episode reward: [(0, '8372.164'), (1, '8637.321')] [2023-12-27 00:56:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001314112_336453632.pth... [2023-12-27 00:56:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001312928_336150528.pth [2023-12-27 00:56:46,079][105692] Updated weights for policy 0, policy_version 1312334 (0.0010) [2023-12-27 00:56:46,087][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001312336_336011264.pth... [2023-12-27 00:56:46,090][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001311248_335732736.pth [2023-12-27 00:56:46,366][105620] Updated weights for policy 1, policy_version 1314122 (0.0008) [2023-12-27 00:56:46,417][105620] Updated weights for policy 1, policy_version 1314132 (0.0008) [2023-12-27 00:56:46,468][105620] Updated weights for policy 1, policy_version 1314142 (0.0007) [2023-12-27 00:56:46,782][105692] Updated weights for policy 0, policy_version 1312344 (0.0007) [2023-12-27 00:56:46,841][105692] Updated weights for policy 0, policy_version 1312354 (0.0006) [2023-12-27 00:56:46,887][105692] Updated weights for policy 0, policy_version 1312364 (0.0005) [2023-12-27 00:56:47,205][105620] Updated weights for policy 1, policy_version 1314152 (0.0006) [2023-12-27 00:56:47,264][105620] Updated weights for policy 1, policy_version 1314162 (0.0005) [2023-12-27 00:56:47,316][105620] Updated weights for policy 1, policy_version 1314172 (0.0007) [2023-12-27 00:56:47,515][105692] Updated weights for policy 0, policy_version 1312374 (0.0008) [2023-12-27 00:56:47,565][105692] Updated weights for policy 0, policy_version 1312384 (0.0005) [2023-12-27 00:56:47,629][105692] Updated weights for policy 0, policy_version 1312394 (0.0010) [2023-12-27 00:56:47,901][105620] Updated weights for policy 1, policy_version 1314182 (0.0005) [2023-12-27 00:56:47,952][105620] Updated weights for policy 1, policy_version 1314192 (0.0005) [2023-12-27 00:56:48,006][105620] Updated weights for policy 1, policy_version 1314202 (0.0006) [2023-12-27 00:56:48,200][105692] Updated weights for policy 0, policy_version 1312404 (0.0006) [2023-12-27 00:56:48,266][105692] Updated weights for policy 0, policy_version 1312414 (0.0005) [2023-12-27 00:56:48,325][105692] Updated weights for policy 0, policy_version 1312424 (0.0006) [2023-12-27 00:56:48,595][105620] Updated weights for policy 1, policy_version 1314212 (0.0006) [2023-12-27 00:56:48,655][105620] Updated weights for policy 1, policy_version 1314222 (0.0005) [2023-12-27 00:56:48,711][105620] Updated weights for policy 1, policy_version 1314232 (0.0005) [2023-12-27 00:56:48,925][105692] Updated weights for policy 0, policy_version 1312434 (0.0007) [2023-12-27 00:56:48,991][105692] Updated weights for policy 0, policy_version 1312444 (0.0010) [2023-12-27 00:56:49,043][105692] Updated weights for policy 0, policy_version 1312454 (0.0006) [2023-12-27 00:56:49,096][105692] Updated weights for policy 0, policy_version 1312464 (0.0010) [2023-12-27 00:56:49,236][105620] Updated weights for policy 1, policy_version 1314242 (0.0006) [2023-12-27 00:56:49,297][105620] Updated weights for policy 1, policy_version 1314252 (0.0010) [2023-12-27 00:56:49,349][105620] Updated weights for policy 1, policy_version 1314262 (0.0010) [2023-12-27 00:56:49,414][105620] Updated weights for policy 1, policy_version 1314272 (0.0008) [2023-12-27 00:56:49,899][105692] Updated weights for policy 0, policy_version 1312474 (0.0009) [2023-12-27 00:56:49,964][105692] Updated weights for policy 0, policy_version 1312484 (0.0009) [2023-12-27 00:56:50,022][105692] Updated weights for policy 0, policy_version 1312494 (0.0008) [2023-12-27 00:56:50,248][105620] Updated weights for policy 1, policy_version 1314282 (0.0011) [2023-12-27 00:56:50,302][105620] Updated weights for policy 1, policy_version 1314292 (0.0009) [2023-12-27 00:56:50,356][105620] Updated weights for policy 1, policy_version 1314302 (0.0005) [2023-12-27 00:56:50,760][105692] Updated weights for policy 0, policy_version 1312504 (0.0006) [2023-12-27 00:56:50,811][105692] Updated weights for policy 0, policy_version 1312514 (0.0006) [2023-12-27 00:56:50,875][105692] Updated weights for policy 0, policy_version 1312524 (0.0009) [2023-12-27 00:56:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 672563200. Throughput: 0: 9313.6, 1: 10137.6. Samples: 672553580. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:56:51,063][104569] Avg episode reward: [(0, '7916.886'), (1, '8995.793')] [2023-12-27 00:56:51,159][105620] Updated weights for policy 1, policy_version 1314312 (0.0008) [2023-12-27 00:56:51,220][105620] Updated weights for policy 1, policy_version 1314322 (0.0008) [2023-12-27 00:56:51,277][105620] Updated weights for policy 1, policy_version 1314332 (0.0008) [2023-12-27 00:56:51,549][105692] Updated weights for policy 0, policy_version 1312534 (0.0009) [2023-12-27 00:56:51,608][105692] Updated weights for policy 0, policy_version 1312544 (0.0010) [2023-12-27 00:56:51,676][105692] Updated weights for policy 0, policy_version 1312554 (0.0010) [2023-12-27 00:56:52,047][105620] Updated weights for policy 1, policy_version 1314342 (0.0008) [2023-12-27 00:56:52,107][105620] Updated weights for policy 1, policy_version 1314352 (0.0008) [2023-12-27 00:56:52,171][105620] Updated weights for policy 1, policy_version 1314362 (0.0009) [2023-12-27 00:56:52,432][105692] Updated weights for policy 0, policy_version 1312564 (0.0011) [2023-12-27 00:56:52,502][105692] Updated weights for policy 0, policy_version 1312574 (0.0011) [2023-12-27 00:56:52,551][105692] Updated weights for policy 0, policy_version 1312584 (0.0010) [2023-12-27 00:56:52,808][105620] Updated weights for policy 1, policy_version 1314372 (0.0008) [2023-12-27 00:56:52,867][105620] Updated weights for policy 1, policy_version 1314382 (0.0010) [2023-12-27 00:56:52,932][105620] Updated weights for policy 1, policy_version 1314392 (0.0008) [2023-12-27 00:56:53,210][105692] Updated weights for policy 0, policy_version 1312594 (0.0007) [2023-12-27 00:56:53,261][105692] Updated weights for policy 0, policy_version 1312604 (0.0010) [2023-12-27 00:56:53,312][105692] Updated weights for policy 0, policy_version 1312614 (0.0010) [2023-12-27 00:56:53,366][105692] Updated weights for policy 0, policy_version 1312624 (0.0010) [2023-12-27 00:56:53,608][105620] Updated weights for policy 1, policy_version 1314402 (0.0007) [2023-12-27 00:56:53,662][105620] Updated weights for policy 1, policy_version 1314412 (0.0005) [2023-12-27 00:56:53,711][105620] Updated weights for policy 1, policy_version 1314422 (0.0005) [2023-12-27 00:56:53,768][105620] Updated weights for policy 1, policy_version 1314432 (0.0008) [2023-12-27 00:56:54,118][105692] Updated weights for policy 0, policy_version 1312634 (0.0010) [2023-12-27 00:56:54,176][105692] Updated weights for policy 0, policy_version 1312644 (0.0010) [2023-12-27 00:56:54,236][105692] Updated weights for policy 0, policy_version 1312654 (0.0010) [2023-12-27 00:56:54,388][105620] Updated weights for policy 1, policy_version 1314442 (0.0009) [2023-12-27 00:56:54,440][105620] Updated weights for policy 1, policy_version 1314452 (0.0005) [2023-12-27 00:56:54,501][105620] Updated weights for policy 1, policy_version 1314462 (0.0007) [2023-12-27 00:56:54,968][105692] Updated weights for policy 0, policy_version 1312664 (0.0007) [2023-12-27 00:56:55,017][105692] Updated weights for policy 0, policy_version 1312674 (0.0006) [2023-12-27 00:56:55,067][105692] Updated weights for policy 0, policy_version 1312684 (0.0006) [2023-12-27 00:56:55,097][105620] Updated weights for policy 1, policy_version 1314472 (0.0011) [2023-12-27 00:56:55,151][105620] Updated weights for policy 1, policy_version 1314482 (0.0011) [2023-12-27 00:56:55,213][105620] Updated weights for policy 1, policy_version 1314492 (0.0011) [2023-12-27 00:56:55,690][105692] Updated weights for policy 0, policy_version 1312694 (0.0005) [2023-12-27 00:56:55,741][105692] Updated weights for policy 0, policy_version 1312704 (0.0005) [2023-12-27 00:56:55,794][105692] Updated weights for policy 0, policy_version 1312714 (0.0009) [2023-12-27 00:56:55,918][105620] Updated weights for policy 1, policy_version 1314502 (0.0008) [2023-12-27 00:56:55,988][105620] Updated weights for policy 1, policy_version 1314512 (0.0005) [2023-12-27 00:56:56,057][105620] Updated weights for policy 1, policy_version 1314522 (0.0005) [2023-12-27 00:56:56,062][104569] Fps is (10 sec: 20480.7, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 672661504. Throughput: 0: 9421.1, 1: 10129.9. Samples: 672672256. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:56:56,063][104569] Avg episode reward: [(0, '7562.806'), (1, '9175.405')] [2023-12-27 00:56:56,511][105692] Updated weights for policy 0, policy_version 1312724 (0.0010) [2023-12-27 00:56:56,535][105620] Updated weights for policy 1, policy_version 1314532 (0.0005) [2023-12-27 00:56:56,563][105692] Updated weights for policy 0, policy_version 1312734 (0.0009) [2023-12-27 00:56:56,584][105620] Updated weights for policy 1, policy_version 1314542 (0.0005) [2023-12-27 00:56:56,616][105692] Updated weights for policy 0, policy_version 1312744 (0.0006) [2023-12-27 00:56:56,642][105620] Updated weights for policy 1, policy_version 1314552 (0.0007) [2023-12-27 00:56:57,222][105692] Updated weights for policy 0, policy_version 1312754 (0.0006) [2023-12-27 00:56:57,264][105620] Updated weights for policy 1, policy_version 1314562 (0.0008) [2023-12-27 00:56:57,276][105692] Updated weights for policy 0, policy_version 1312764 (0.0010) [2023-12-27 00:56:57,319][105620] Updated weights for policy 1, policy_version 1314572 (0.0008) [2023-12-27 00:56:57,339][105692] Updated weights for policy 0, policy_version 1312774 (0.0011) [2023-12-27 00:56:57,369][105620] Updated weights for policy 1, policy_version 1314582 (0.0005) [2023-12-27 00:56:57,396][105692] Updated weights for policy 0, policy_version 1312784 (0.0010) [2023-12-27 00:56:57,416][105620] Updated weights for policy 1, policy_version 1314592 (0.0005) [2023-12-27 00:56:57,966][105692] Updated weights for policy 0, policy_version 1312794 (0.0005) [2023-12-27 00:56:58,017][105692] Updated weights for policy 0, policy_version 1312804 (0.0007) [2023-12-27 00:56:58,023][105620] Updated weights for policy 1, policy_version 1314602 (0.0009) [2023-12-27 00:56:58,069][105620] Updated weights for policy 1, policy_version 1314612 (0.0007) [2023-12-27 00:56:58,072][105692] Updated weights for policy 0, policy_version 1312814 (0.0010) [2023-12-27 00:56:58,112][105620] Updated weights for policy 1, policy_version 1314622 (0.0007) [2023-12-27 00:56:58,874][105692] Updated weights for policy 0, policy_version 1312824 (0.0009) [2023-12-27 00:56:58,935][105692] Updated weights for policy 0, policy_version 1312834 (0.0008) [2023-12-27 00:56:58,970][105620] Updated weights for policy 1, policy_version 1314632 (0.0007) [2023-12-27 00:56:58,996][105692] Updated weights for policy 0, policy_version 1312844 (0.0008) [2023-12-27 00:56:59,036][105620] Updated weights for policy 1, policy_version 1314642 (0.0007) [2023-12-27 00:56:59,095][105620] Updated weights for policy 1, policy_version 1314652 (0.0008) [2023-12-27 00:56:59,805][105620] Updated weights for policy 1, policy_version 1314662 (0.0005) [2023-12-27 00:56:59,823][105692] Updated weights for policy 0, policy_version 1312854 (0.0008) [2023-12-27 00:56:59,866][105620] Updated weights for policy 1, policy_version 1314672 (0.0007) [2023-12-27 00:56:59,887][105692] Updated weights for policy 0, policy_version 1312864 (0.0009) [2023-12-27 00:56:59,930][105620] Updated weights for policy 1, policy_version 1314682 (0.0006) [2023-12-27 00:56:59,952][105692] Updated weights for policy 0, policy_version 1312874 (0.0009) [2023-12-27 00:57:00,653][105692] Updated weights for policy 0, policy_version 1312884 (0.0010) [2023-12-27 00:57:00,664][105620] Updated weights for policy 1, policy_version 1314692 (0.0007) [2023-12-27 00:57:00,709][105620] Updated weights for policy 1, policy_version 1314702 (0.0008) [2023-12-27 00:57:00,712][105692] Updated weights for policy 0, policy_version 1312894 (0.0010) [2023-12-27 00:57:00,769][105620] Updated weights for policy 1, policy_version 1314712 (0.0005) [2023-12-27 00:57:00,770][105692] Updated weights for policy 0, policy_version 1312904 (0.0010) [2023-12-27 00:57:01,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 672768000. Throughput: 0: 9485.6, 1: 10182.3. Samples: 672736128. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:57:01,063][104569] Avg episode reward: [(0, '7691.255'), (1, '9177.197')] [2023-12-27 00:57:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001314720_336609280.pth... [2023-12-27 00:57:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001312912_336158720.pth... [2023-12-27 00:57:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001311792_335872000.pth [2023-12-27 00:57:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001313536_336306176.pth [2023-12-27 00:57:01,489][105620] Updated weights for policy 1, policy_version 1314722 (0.0006) [2023-12-27 00:57:01,549][105620] Updated weights for policy 1, policy_version 1314732 (0.0008) [2023-12-27 00:57:01,608][105620] Updated weights for policy 1, policy_version 1314742 (0.0008) [2023-12-27 00:57:01,612][105692] Updated weights for policy 0, policy_version 1312914 (0.0010) [2023-12-27 00:57:01,664][105620] Updated weights for policy 1, policy_version 1314752 (0.0008) [2023-12-27 00:57:01,678][105692] Updated weights for policy 0, policy_version 1312924 (0.0008) [2023-12-27 00:57:01,734][105692] Updated weights for policy 0, policy_version 1312934 (0.0008) [2023-12-27 00:57:01,790][105692] Updated weights for policy 0, policy_version 1312944 (0.0007) [2023-12-27 00:57:02,423][105620] Updated weights for policy 1, policy_version 1314762 (0.0006) [2023-12-27 00:57:02,456][105692] Updated weights for policy 0, policy_version 1312954 (0.0008) [2023-12-27 00:57:02,486][105620] Updated weights for policy 1, policy_version 1314772 (0.0007) [2023-12-27 00:57:02,527][105692] Updated weights for policy 0, policy_version 1312964 (0.0006) [2023-12-27 00:57:02,548][105620] Updated weights for policy 1, policy_version 1314782 (0.0006) [2023-12-27 00:57:02,588][105692] Updated weights for policy 0, policy_version 1312974 (0.0006) [2023-12-27 00:57:03,161][105620] Updated weights for policy 1, policy_version 1314792 (0.0005) [2023-12-27 00:57:03,211][105620] Updated weights for policy 1, policy_version 1314802 (0.0005) [2023-12-27 00:57:03,220][105692] Updated weights for policy 0, policy_version 1312984 (0.0007) [2023-12-27 00:57:03,256][105620] Updated weights for policy 1, policy_version 1314812 (0.0005) [2023-12-27 00:57:03,265][105692] Updated weights for policy 0, policy_version 1312994 (0.0005) [2023-12-27 00:57:03,317][105692] Updated weights for policy 0, policy_version 1313004 (0.0005) [2023-12-27 00:57:03,963][105620] Updated weights for policy 1, policy_version 1314822 (0.0008) [2023-12-27 00:57:03,981][105692] Updated weights for policy 0, policy_version 1313014 (0.0005) [2023-12-27 00:57:04,023][105620] Updated weights for policy 1, policy_version 1314832 (0.0011) [2023-12-27 00:57:04,030][105692] Updated weights for policy 0, policy_version 1313024 (0.0006) [2023-12-27 00:57:04,079][105620] Updated weights for policy 1, policy_version 1314842 (0.0010) [2023-12-27 00:57:04,085][105692] Updated weights for policy 0, policy_version 1313034 (0.0005) [2023-12-27 00:57:04,836][105620] Updated weights for policy 1, policy_version 1314852 (0.0010) [2023-12-27 00:57:04,870][105692] Updated weights for policy 0, policy_version 1313044 (0.0007) [2023-12-27 00:57:04,881][105620] Updated weights for policy 1, policy_version 1314862 (0.0008) [2023-12-27 00:57:04,927][105692] Updated weights for policy 0, policy_version 1313054 (0.0006) [2023-12-27 00:57:04,941][105620] Updated weights for policy 1, policy_version 1314872 (0.0007) [2023-12-27 00:57:04,975][105692] Updated weights for policy 0, policy_version 1313064 (0.0007) [2023-12-27 00:57:05,494][105620] Updated weights for policy 1, policy_version 1314882 (0.0005) [2023-12-27 00:57:05,545][105620] Updated weights for policy 1, policy_version 1314892 (0.0005) [2023-12-27 00:57:05,604][105620] Updated weights for policy 1, policy_version 1314902 (0.0005) [2023-12-27 00:57:05,657][105620] Updated weights for policy 1, policy_version 1314912 (0.0005) [2023-12-27 00:57:05,833][105692] Updated weights for policy 0, policy_version 1313075 (0.0009) [2023-12-27 00:57:05,889][105692] Updated weights for policy 0, policy_version 1313085 (0.0005) [2023-12-27 00:57:05,945][105692] Updated weights for policy 0, policy_version 1313095 (0.0006) [2023-12-27 00:57:06,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 672866304. Throughput: 0: 9530.3, 1: 10073.3. Samples: 672852376. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:57:06,062][104569] Avg episode reward: [(0, '7747.971'), (1, '8737.538')] [2023-12-27 00:57:06,227][105620] Updated weights for policy 1, policy_version 1314922 (0.0009) [2023-12-27 00:57:06,290][105620] Updated weights for policy 1, policy_version 1314932 (0.0007) [2023-12-27 00:57:06,361][105620] Updated weights for policy 1, policy_version 1314942 (0.0007) [2023-12-27 00:57:06,527][105692] Updated weights for policy 0, policy_version 1313105 (0.0007) [2023-12-27 00:57:06,585][105692] Updated weights for policy 0, policy_version 1313115 (0.0009) [2023-12-27 00:57:06,641][105692] Updated weights for policy 0, policy_version 1313125 (0.0009) [2023-12-27 00:57:06,703][105692] Updated weights for policy 0, policy_version 1313135 (0.0009) [2023-12-27 00:57:07,099][105620] Updated weights for policy 1, policy_version 1314952 (0.0009) [2023-12-27 00:57:07,150][105620] Updated weights for policy 1, policy_version 1314962 (0.0009) [2023-12-27 00:57:07,211][105620] Updated weights for policy 1, policy_version 1314972 (0.0009) [2023-12-27 00:57:07,470][105692] Updated weights for policy 0, policy_version 1313145 (0.0010) [2023-12-27 00:57:07,527][105692] Updated weights for policy 0, policy_version 1313156 (0.0009) [2023-12-27 00:57:07,576][105692] Updated weights for policy 0, policy_version 1313166 (0.0009) [2023-12-27 00:57:07,931][105620] Updated weights for policy 1, policy_version 1314982 (0.0010) [2023-12-27 00:57:07,982][105620] Updated weights for policy 1, policy_version 1314992 (0.0010) [2023-12-27 00:57:08,044][105620] Updated weights for policy 1, policy_version 1315002 (0.0011) [2023-12-27 00:57:08,369][105692] Updated weights for policy 0, policy_version 1313176 (0.0009) [2023-12-27 00:57:08,436][105692] Updated weights for policy 0, policy_version 1313186 (0.0006) [2023-12-27 00:57:08,510][105692] Updated weights for policy 0, policy_version 1313196 (0.0008) [2023-12-27 00:57:08,709][105620] Updated weights for policy 1, policy_version 1315012 (0.0008) [2023-12-27 00:57:08,773][105620] Updated weights for policy 1, policy_version 1315022 (0.0005) [2023-12-27 00:57:08,828][105620] Updated weights for policy 1, policy_version 1315032 (0.0006) [2023-12-27 00:57:09,244][105692] Updated weights for policy 0, policy_version 1313206 (0.0009) [2023-12-27 00:57:09,312][105692] Updated weights for policy 0, policy_version 1313216 (0.0008) [2023-12-27 00:57:09,377][105692] Updated weights for policy 0, policy_version 1313226 (0.0009) [2023-12-27 00:57:09,513][105620] Updated weights for policy 1, policy_version 1315042 (0.0010) [2023-12-27 00:57:09,565][105620] Updated weights for policy 1, policy_version 1315052 (0.0009) [2023-12-27 00:57:09,624][105620] Updated weights for policy 1, policy_version 1315062 (0.0009) [2023-12-27 00:57:09,676][105620] Updated weights for policy 1, policy_version 1315072 (0.0009) [2023-12-27 00:57:10,073][105692] Updated weights for policy 0, policy_version 1313236 (0.0010) [2023-12-27 00:57:10,133][105692] Updated weights for policy 0, policy_version 1313246 (0.0011) [2023-12-27 00:57:10,192][105692] Updated weights for policy 0, policy_version 1313256 (0.0011) [2023-12-27 00:57:10,535][105620] Updated weights for policy 1, policy_version 1315082 (0.0007) [2023-12-27 00:57:10,591][105620] Updated weights for policy 1, policy_version 1315092 (0.0006) [2023-12-27 00:57:10,659][105620] Updated weights for policy 1, policy_version 1315102 (0.0009) [2023-12-27 00:57:10,913][105692] Updated weights for policy 0, policy_version 1313266 (0.0011) [2023-12-27 00:57:10,969][105692] Updated weights for policy 0, policy_version 1313276 (0.0010) [2023-12-27 00:57:11,027][105692] Updated weights for policy 0, policy_version 1313286 (0.0010) [2023-12-27 00:57:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 672956416. Throughput: 0: 9572.4, 1: 10112.7. Samples: 672969764. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:57:11,063][104569] Avg episode reward: [(0, '7744.679'), (1, '8655.328')] [2023-12-27 00:57:11,091][105692] Updated weights for policy 0, policy_version 1313296 (0.0008) [2023-12-27 00:57:11,425][105620] Updated weights for policy 1, policy_version 1315112 (0.0007) [2023-12-27 00:57:11,477][105620] Updated weights for policy 1, policy_version 1315122 (0.0008) [2023-12-27 00:57:11,543][105620] Updated weights for policy 1, policy_version 1315132 (0.0008) [2023-12-27 00:57:11,821][105692] Updated weights for policy 0, policy_version 1313306 (0.0010) [2023-12-27 00:57:11,882][105692] Updated weights for policy 0, policy_version 1313316 (0.0011) [2023-12-27 00:57:11,951][105692] Updated weights for policy 0, policy_version 1313326 (0.0011) [2023-12-27 00:57:12,328][105620] Updated weights for policy 1, policy_version 1315142 (0.0009) [2023-12-27 00:57:12,396][105620] Updated weights for policy 1, policy_version 1315152 (0.0008) [2023-12-27 00:57:12,441][105620] Updated weights for policy 1, policy_version 1315162 (0.0008) [2023-12-27 00:57:12,730][105692] Updated weights for policy 0, policy_version 1313336 (0.0011) [2023-12-27 00:57:12,782][105692] Updated weights for policy 0, policy_version 1313346 (0.0011) [2023-12-27 00:57:12,845][105692] Updated weights for policy 0, policy_version 1313356 (0.0011) [2023-12-27 00:57:13,224][105620] Updated weights for policy 1, policy_version 1315172 (0.0008) [2023-12-27 00:57:13,283][105620] Updated weights for policy 1, policy_version 1315182 (0.0008) [2023-12-27 00:57:13,338][105620] Updated weights for policy 1, policy_version 1315192 (0.0008) [2023-12-27 00:57:13,601][105692] Updated weights for policy 0, policy_version 1313366 (0.0007) [2023-12-27 00:57:13,655][105692] Updated weights for policy 0, policy_version 1313376 (0.0005) [2023-12-27 00:57:13,702][105692] Updated weights for policy 0, policy_version 1313386 (0.0008) [2023-12-27 00:57:14,039][105620] Updated weights for policy 1, policy_version 1315202 (0.0008) [2023-12-27 00:57:14,106][105620] Updated weights for policy 1, policy_version 1315212 (0.0006) [2023-12-27 00:57:14,125][105586] KL-divergence is very high: 131.7276 [2023-12-27 00:57:14,132][105586] KL-divergence is very high: 150.4518 [2023-12-27 00:57:14,165][105620] Updated weights for policy 1, policy_version 1315222 (0.0010) [2023-12-27 00:57:14,168][105586] KL-divergence is very high: 128.2947 [2023-12-27 00:57:14,175][105586] KL-divergence is very high: 146.1947 [2023-12-27 00:57:14,219][105586] KL-divergence is very high: 105.6141 [2023-12-27 00:57:14,227][105620] Updated weights for policy 1, policy_version 1315232 (0.0010) [2023-12-27 00:57:14,418][105692] Updated weights for policy 0, policy_version 1313396 (0.0010) [2023-12-27 00:57:14,481][105692] Updated weights for policy 0, policy_version 1313406 (0.0009) [2023-12-27 00:57:14,546][105692] Updated weights for policy 0, policy_version 1313416 (0.0008) [2023-12-27 00:57:14,927][105620] Updated weights for policy 1, policy_version 1315242 (0.0011) [2023-12-27 00:57:14,986][105620] Updated weights for policy 1, policy_version 1315252 (0.0010) [2023-12-27 00:57:15,042][105620] Updated weights for policy 1, policy_version 1315262 (0.0011) [2023-12-27 00:57:15,175][105692] Updated weights for policy 0, policy_version 1313426 (0.0008) [2023-12-27 00:57:15,221][105692] Updated weights for policy 0, policy_version 1313436 (0.0011) [2023-12-27 00:57:15,267][105692] Updated weights for policy 0, policy_version 1313446 (0.0011) [2023-12-27 00:57:15,311][105692] Updated weights for policy 0, policy_version 1313456 (0.0010) [2023-12-27 00:57:15,758][105620] Updated weights for policy 1, policy_version 1315272 (0.0006) [2023-12-27 00:57:15,828][105620] Updated weights for policy 1, policy_version 1315282 (0.0008) [2023-12-27 00:57:15,885][105620] Updated weights for policy 1, policy_version 1315292 (0.0010) [2023-12-27 00:57:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.4, 300 sec: 19605.3). Total num frames: 673054720. Throughput: 0: 9535.0, 1: 10030.5. Samples: 673024900. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:57:16,062][104569] Avg episode reward: [(0, '7619.482'), (1, '9098.541')] [2023-12-27 00:57:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001315296_336756736.pth... [2023-12-27 00:57:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001314112_336453632.pth [2023-12-27 00:57:16,088][105692] Updated weights for policy 0, policy_version 1313466 (0.0008) [2023-12-27 00:57:16,151][105692] Updated weights for policy 0, policy_version 1313476 (0.0009) [2023-12-27 00:57:16,214][105692] Updated weights for policy 0, policy_version 1313486 (0.0008) [2023-12-27 00:57:16,223][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001313488_336306176.pth... [2023-12-27 00:57:16,228][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001312336_336011264.pth [2023-12-27 00:57:16,575][105620] Updated weights for policy 1, policy_version 1315302 (0.0010) [2023-12-27 00:57:16,638][105620] Updated weights for policy 1, policy_version 1315312 (0.0008) [2023-12-27 00:57:16,692][105620] Updated weights for policy 1, policy_version 1315322 (0.0008) [2023-12-27 00:57:16,894][105692] Updated weights for policy 0, policy_version 1313496 (0.0006) [2023-12-27 00:57:16,958][105692] Updated weights for policy 0, policy_version 1313506 (0.0006) [2023-12-27 00:57:17,017][105692] Updated weights for policy 0, policy_version 1313516 (0.0006) [2023-12-27 00:57:17,491][105620] Updated weights for policy 1, policy_version 1315332 (0.0008) [2023-12-27 00:57:17,547][105620] Updated weights for policy 1, policy_version 1315342 (0.0008) [2023-12-27 00:57:17,606][105620] Updated weights for policy 1, policy_version 1315352 (0.0008) [2023-12-27 00:57:17,689][105692] Updated weights for policy 0, policy_version 1313526 (0.0008) [2023-12-27 00:57:17,740][105692] Updated weights for policy 0, policy_version 1313536 (0.0010) [2023-12-27 00:57:17,794][105692] Updated weights for policy 0, policy_version 1313546 (0.0010) [2023-12-27 00:57:18,388][105620] Updated weights for policy 1, policy_version 1315362 (0.0008) [2023-12-27 00:57:18,443][105620] Updated weights for policy 1, policy_version 1315372 (0.0009) [2023-12-27 00:57:18,498][105620] Updated weights for policy 1, policy_version 1315382 (0.0009) [2023-12-27 00:57:18,517][105692] Updated weights for policy 0, policy_version 1313556 (0.0010) [2023-12-27 00:57:18,553][105620] Updated weights for policy 1, policy_version 1315392 (0.0008) [2023-12-27 00:57:18,577][105692] Updated weights for policy 0, policy_version 1313566 (0.0008) [2023-12-27 00:57:18,634][105692] Updated weights for policy 0, policy_version 1313576 (0.0008) [2023-12-27 00:57:19,303][105620] Updated weights for policy 1, policy_version 1315402 (0.0010) [2023-12-27 00:57:19,370][105620] Updated weights for policy 1, policy_version 1315412 (0.0009) [2023-12-27 00:57:19,371][105692] Updated weights for policy 0, policy_version 1313586 (0.0007) [2023-12-27 00:57:19,425][105692] Updated weights for policy 0, policy_version 1313596 (0.0006) [2023-12-27 00:57:19,432][105620] Updated weights for policy 1, policy_version 1315422 (0.0007) [2023-12-27 00:57:19,483][105692] Updated weights for policy 0, policy_version 1313606 (0.0009) [2023-12-27 00:57:19,559][105692] Updated weights for policy 0, policy_version 1313616 (0.0006) [2023-12-27 00:57:20,154][105620] Updated weights for policy 1, policy_version 1315432 (0.0007) [2023-12-27 00:57:20,207][105620] Updated weights for policy 1, policy_version 1315442 (0.0009) [2023-12-27 00:57:20,259][105620] Updated weights for policy 1, policy_version 1315452 (0.0008) [2023-12-27 00:57:20,401][105692] Updated weights for policy 0, policy_version 1313626 (0.0009) [2023-12-27 00:57:20,463][105692] Updated weights for policy 0, policy_version 1313636 (0.0009) [2023-12-27 00:57:20,525][105692] Updated weights for policy 0, policy_version 1313646 (0.0009) [2023-12-27 00:57:21,042][105620] Updated weights for policy 1, policy_version 1315462 (0.0009) [2023-12-27 00:57:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 673144832. Throughput: 0: 9589.3, 1: 9974.0. Samples: 673140220. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:57:21,062][104569] Avg episode reward: [(0, '8548.312'), (1, '8824.652')] [2023-12-27 00:57:21,112][105620] Updated weights for policy 1, policy_version 1315472 (0.0009) [2023-12-27 00:57:21,180][105620] Updated weights for policy 1, policy_version 1315482 (0.0009) [2023-12-27 00:57:21,270][105692] Updated weights for policy 0, policy_version 1313656 (0.0010) [2023-12-27 00:57:21,334][105692] Updated weights for policy 0, policy_version 1313666 (0.0009) [2023-12-27 00:57:21,404][105692] Updated weights for policy 0, policy_version 1313676 (0.0008) [2023-12-27 00:57:21,996][105620] Updated weights for policy 1, policy_version 1315492 (0.0009) [2023-12-27 00:57:22,060][105620] Updated weights for policy 1, policy_version 1315502 (0.0010) [2023-12-27 00:57:22,091][105692] Updated weights for policy 0, policy_version 1313686 (0.0008) [2023-12-27 00:57:22,126][105620] Updated weights for policy 1, policy_version 1315512 (0.0008) [2023-12-27 00:57:22,149][105692] Updated weights for policy 0, policy_version 1313696 (0.0007) [2023-12-27 00:57:22,209][105692] Updated weights for policy 0, policy_version 1313706 (0.0008) [2023-12-27 00:57:22,883][105692] Updated weights for policy 0, policy_version 1313716 (0.0007) [2023-12-27 00:57:22,933][105620] Updated weights for policy 1, policy_version 1315522 (0.0009) [2023-12-27 00:57:22,938][105692] Updated weights for policy 0, policy_version 1313726 (0.0006) [2023-12-27 00:57:22,994][105620] Updated weights for policy 1, policy_version 1315532 (0.0008) [2023-12-27 00:57:22,996][105692] Updated weights for policy 0, policy_version 1313736 (0.0008) [2023-12-27 00:57:23,059][105620] Updated weights for policy 1, policy_version 1315542 (0.0006) [2023-12-27 00:57:23,133][105620] Updated weights for policy 1, policy_version 1315552 (0.0006) [2023-12-27 00:57:23,716][105692] Updated weights for policy 0, policy_version 1313746 (0.0007) [2023-12-27 00:57:23,765][105692] Updated weights for policy 0, policy_version 1313756 (0.0006) [2023-12-27 00:57:23,770][105620] Updated weights for policy 1, policy_version 1315562 (0.0011) [2023-12-27 00:57:23,815][105692] Updated weights for policy 0, policy_version 1313766 (0.0007) [2023-12-27 00:57:23,825][105620] Updated weights for policy 1, policy_version 1315572 (0.0010) [2023-12-27 00:57:23,872][105692] Updated weights for policy 0, policy_version 1313776 (0.0009) [2023-12-27 00:57:23,884][105620] Updated weights for policy 1, policy_version 1315582 (0.0010) [2023-12-27 00:57:24,540][105620] Updated weights for policy 1, policy_version 1315592 (0.0011) [2023-12-27 00:57:24,597][105620] Updated weights for policy 1, policy_version 1315602 (0.0011) [2023-12-27 00:57:24,653][105620] Updated weights for policy 1, policy_version 1315612 (0.0011) [2023-12-27 00:57:24,659][105692] Updated weights for policy 0, policy_version 1313786 (0.0006) [2023-12-27 00:57:24,702][105692] Updated weights for policy 0, policy_version 1313796 (0.0008) [2023-12-27 00:57:24,747][105692] Updated weights for policy 0, policy_version 1313806 (0.0008) [2023-12-27 00:57:25,392][105620] Updated weights for policy 1, policy_version 1315622 (0.0010) [2023-12-27 00:57:25,450][105620] Updated weights for policy 1, policy_version 1315632 (0.0010) [2023-12-27 00:57:25,453][105692] Updated weights for policy 0, policy_version 1313816 (0.0006) [2023-12-27 00:57:25,501][105692] Updated weights for policy 0, policy_version 1313826 (0.0007) [2023-12-27 00:57:25,505][105620] Updated weights for policy 1, policy_version 1315642 (0.0010) [2023-12-27 00:57:25,552][105692] Updated weights for policy 0, policy_version 1313836 (0.0008) [2023-12-27 00:57:26,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 673243136. Throughput: 0: 9688.3, 1: 9862.3. Samples: 673253392. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:57:26,063][104569] Avg episode reward: [(0, '8555.373'), (1, '8914.699')] [2023-12-27 00:57:26,238][105620] Updated weights for policy 1, policy_version 1315652 (0.0010) [2023-12-27 00:57:26,300][105620] Updated weights for policy 1, policy_version 1315662 (0.0010) [2023-12-27 00:57:26,333][105692] Updated weights for policy 0, policy_version 1313846 (0.0006) [2023-12-27 00:57:26,362][105620] Updated weights for policy 1, policy_version 1315672 (0.0011) [2023-12-27 00:57:26,382][105692] Updated weights for policy 0, policy_version 1313856 (0.0011) [2023-12-27 00:57:26,441][105692] Updated weights for policy 0, policy_version 1313866 (0.0011) [2023-12-27 00:57:27,025][105620] Updated weights for policy 1, policy_version 1315682 (0.0009) [2023-12-27 00:57:27,080][105692] Updated weights for policy 0, policy_version 1313876 (0.0008) [2023-12-27 00:57:27,091][105620] Updated weights for policy 1, policy_version 1315692 (0.0009) [2023-12-27 00:57:27,131][105692] Updated weights for policy 0, policy_version 1313886 (0.0010) [2023-12-27 00:57:27,145][105620] Updated weights for policy 1, policy_version 1315702 (0.0010) [2023-12-27 00:57:27,175][105692] Updated weights for policy 0, policy_version 1313896 (0.0007) [2023-12-27 00:57:27,206][105620] Updated weights for policy 1, policy_version 1315712 (0.0010) [2023-12-27 00:57:27,885][105692] Updated weights for policy 0, policy_version 1313906 (0.0006) [2023-12-27 00:57:27,906][105620] Updated weights for policy 1, policy_version 1315722 (0.0010) [2023-12-27 00:57:27,941][105692] Updated weights for policy 0, policy_version 1313916 (0.0009) [2023-12-27 00:57:27,957][105620] Updated weights for policy 1, policy_version 1315732 (0.0010) [2023-12-27 00:57:28,002][105692] Updated weights for policy 0, policy_version 1313926 (0.0010) [2023-12-27 00:57:28,008][105620] Updated weights for policy 1, policy_version 1315742 (0.0010) [2023-12-27 00:57:28,052][105692] Updated weights for policy 0, policy_version 1313936 (0.0010) [2023-12-27 00:57:28,712][105620] Updated weights for policy 1, policy_version 1315752 (0.0009) [2023-12-27 00:57:28,744][105692] Updated weights for policy 0, policy_version 1313946 (0.0005) [2023-12-27 00:57:28,771][105620] Updated weights for policy 1, policy_version 1315762 (0.0008) [2023-12-27 00:57:28,805][105692] Updated weights for policy 0, policy_version 1313956 (0.0005) [2023-12-27 00:57:28,833][105620] Updated weights for policy 1, policy_version 1315772 (0.0010) [2023-12-27 00:57:28,861][105692] Updated weights for policy 0, policy_version 1313966 (0.0005) [2023-12-27 00:57:29,451][105692] Updated weights for policy 0, policy_version 1313976 (0.0008) [2023-12-27 00:57:29,500][105692] Updated weights for policy 0, policy_version 1313986 (0.0009) [2023-12-27 00:57:29,555][105692] Updated weights for policy 0, policy_version 1313996 (0.0009) [2023-12-27 00:57:29,622][105620] Updated weights for policy 1, policy_version 1315782 (0.0009) [2023-12-27 00:57:29,683][105620] Updated weights for policy 1, policy_version 1315792 (0.0009) [2023-12-27 00:57:29,741][105620] Updated weights for policy 1, policy_version 1315802 (0.0009) [2023-12-27 00:57:30,359][105692] Updated weights for policy 0, policy_version 1314006 (0.0009) [2023-12-27 00:57:30,424][105692] Updated weights for policy 0, policy_version 1314016 (0.0009) [2023-12-27 00:57:30,482][105692] Updated weights for policy 0, policy_version 1314026 (0.0008) [2023-12-27 00:57:30,492][105620] Updated weights for policy 1, policy_version 1315812 (0.0009) [2023-12-27 00:57:30,553][105620] Updated weights for policy 1, policy_version 1315822 (0.0008) [2023-12-27 00:57:30,621][105620] Updated weights for policy 1, policy_version 1315832 (0.0010) [2023-12-27 00:57:31,057][105692] Updated weights for policy 0, policy_version 1314036 (0.0007) [2023-12-27 00:57:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 673341440. Throughput: 0: 9733.5, 1: 9856.8. Samples: 673313000. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:57:31,062][104569] Avg episode reward: [(0, '8284.013'), (1, '9087.237')] [2023-12-27 00:57:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001315840_336896000.pth... [2023-12-27 00:57:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001314720_336609280.pth [2023-12-27 00:57:31,114][105692] Updated weights for policy 0, policy_version 1314046 (0.0006) [2023-12-27 00:57:31,178][105692] Updated weights for policy 0, policy_version 1314056 (0.0009) [2023-12-27 00:57:31,221][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001314064_336453632.pth... [2023-12-27 00:57:31,226][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001312912_336158720.pth [2023-12-27 00:57:31,400][105620] Updated weights for policy 1, policy_version 1315842 (0.0009) [2023-12-27 00:57:31,451][105620] Updated weights for policy 1, policy_version 1315852 (0.0008) [2023-12-27 00:57:31,503][105620] Updated weights for policy 1, policy_version 1315862 (0.0009) [2023-12-27 00:57:31,553][105620] Updated weights for policy 1, policy_version 1315872 (0.0008) [2023-12-27 00:57:31,931][105692] Updated weights for policy 0, policy_version 1314066 (0.0009) [2023-12-27 00:57:31,991][105692] Updated weights for policy 0, policy_version 1314076 (0.0010) [2023-12-27 00:57:32,047][105692] Updated weights for policy 0, policy_version 1314086 (0.0009) [2023-12-27 00:57:32,103][105692] Updated weights for policy 0, policy_version 1314096 (0.0009) [2023-12-27 00:57:32,318][105620] Updated weights for policy 1, policy_version 1315882 (0.0009) [2023-12-27 00:57:32,375][105620] Updated weights for policy 1, policy_version 1315892 (0.0008) [2023-12-27 00:57:32,436][105620] Updated weights for policy 1, policy_version 1315902 (0.0009) [2023-12-27 00:57:32,857][105692] Updated weights for policy 0, policy_version 1314106 (0.0009) [2023-12-27 00:57:32,918][105692] Updated weights for policy 0, policy_version 1314116 (0.0009) [2023-12-27 00:57:32,974][105692] Updated weights for policy 0, policy_version 1314126 (0.0008) [2023-12-27 00:57:33,205][105620] Updated weights for policy 1, policy_version 1315913 (0.0010) [2023-12-27 00:57:33,261][105620] Updated weights for policy 1, policy_version 1315923 (0.0008) [2023-12-27 00:57:33,317][105620] Updated weights for policy 1, policy_version 1315933 (0.0005) [2023-12-27 00:57:33,652][105692] Updated weights for policy 0, policy_version 1314136 (0.0006) [2023-12-27 00:57:33,703][105692] Updated weights for policy 0, policy_version 1314146 (0.0006) [2023-12-27 00:57:33,761][105692] Updated weights for policy 0, policy_version 1314156 (0.0006) [2023-12-27 00:57:34,040][105620] Updated weights for policy 1, policy_version 1315943 (0.0008) [2023-12-27 00:57:34,101][105620] Updated weights for policy 1, policy_version 1315953 (0.0007) [2023-12-27 00:57:34,167][105620] Updated weights for policy 1, policy_version 1315963 (0.0008) [2023-12-27 00:57:34,440][105692] Updated weights for policy 0, policy_version 1314166 (0.0009) [2023-12-27 00:57:34,502][105692] Updated weights for policy 0, policy_version 1314176 (0.0009) [2023-12-27 00:57:34,569][105692] Updated weights for policy 0, policy_version 1314186 (0.0008) [2023-12-27 00:57:34,922][105620] Updated weights for policy 1, policy_version 1315973 (0.0008) [2023-12-27 00:57:34,985][105620] Updated weights for policy 1, policy_version 1315983 (0.0009) [2023-12-27 00:57:35,052][105620] Updated weights for policy 1, policy_version 1315993 (0.0009) [2023-12-27 00:57:35,244][105692] Updated weights for policy 0, policy_version 1314196 (0.0007) [2023-12-27 00:57:35,289][105692] Updated weights for policy 0, policy_version 1314206 (0.0005) [2023-12-27 00:57:35,341][105692] Updated weights for policy 0, policy_version 1314216 (0.0007) [2023-12-27 00:57:35,868][105620] Updated weights for policy 1, policy_version 1316003 (0.0009) [2023-12-27 00:57:35,932][105620] Updated weights for policy 1, policy_version 1316013 (0.0009) [2023-12-27 00:57:35,981][105620] Updated weights for policy 1, policy_version 1316023 (0.0008) [2023-12-27 00:57:36,012][105692] Updated weights for policy 0, policy_version 1314226 (0.0009) [2023-12-27 00:57:36,060][105692] Updated weights for policy 0, policy_version 1314236 (0.0008) [2023-12-27 00:57:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 673439744. Throughput: 0: 9733.4, 1: 9702.1. Samples: 673428180. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:57:36,063][104569] Avg episode reward: [(0, '7757.840'), (1, '9173.071')] [2023-12-27 00:57:36,115][105692] Updated weights for policy 0, policy_version 1314246 (0.0009) [2023-12-27 00:57:36,172][105692] Updated weights for policy 0, policy_version 1314256 (0.0009) [2023-12-27 00:57:36,739][105620] Updated weights for policy 1, policy_version 1316033 (0.0009) [2023-12-27 00:57:36,788][105620] Updated weights for policy 1, policy_version 1316043 (0.0008) [2023-12-27 00:57:36,835][105620] Updated weights for policy 1, policy_version 1316053 (0.0009) [2023-12-27 00:57:36,882][105620] Updated weights for policy 1, policy_version 1316063 (0.0009) [2023-12-27 00:57:36,950][105692] Updated weights for policy 0, policy_version 1314266 (0.0009) [2023-12-27 00:57:36,999][105692] Updated weights for policy 0, policy_version 1314276 (0.0009) [2023-12-27 00:57:37,062][105692] Updated weights for policy 0, policy_version 1314286 (0.0010) [2023-12-27 00:57:37,622][105620] Updated weights for policy 1, policy_version 1316073 (0.0008) [2023-12-27 00:57:37,670][105620] Updated weights for policy 1, policy_version 1316083 (0.0009) [2023-12-27 00:57:37,721][105620] Updated weights for policy 1, policy_version 1316093 (0.0008) [2023-12-27 00:57:37,853][105692] Updated weights for policy 0, policy_version 1314296 (0.0009) [2023-12-27 00:57:37,911][105692] Updated weights for policy 0, policy_version 1314306 (0.0009) [2023-12-27 00:57:37,966][105692] Updated weights for policy 0, policy_version 1314316 (0.0009) [2023-12-27 00:57:38,524][105620] Updated weights for policy 1, policy_version 1316103 (0.0009) [2023-12-27 00:57:38,576][105620] Updated weights for policy 1, policy_version 1316113 (0.0010) [2023-12-27 00:57:38,633][105620] Updated weights for policy 1, policy_version 1316124 (0.0010) [2023-12-27 00:57:38,655][105692] Updated weights for policy 0, policy_version 1314326 (0.0009) [2023-12-27 00:57:38,708][105692] Updated weights for policy 0, policy_version 1314336 (0.0005) [2023-12-27 00:57:38,759][105692] Updated weights for policy 0, policy_version 1314346 (0.0005) [2023-12-27 00:57:39,361][105692] Updated weights for policy 0, policy_version 1314356 (0.0006) [2023-12-27 00:57:39,423][105692] Updated weights for policy 0, policy_version 1314366 (0.0009) [2023-12-27 00:57:39,483][105692] Updated weights for policy 0, policy_version 1314376 (0.0011) [2023-12-27 00:57:39,502][105620] Updated weights for policy 1, policy_version 1316134 (0.0008) [2023-12-27 00:57:39,556][105620] Updated weights for policy 1, policy_version 1316144 (0.0007) [2023-12-27 00:57:39,609][105620] Updated weights for policy 1, policy_version 1316154 (0.0008) [2023-12-27 00:57:40,190][105692] Updated weights for policy 0, policy_version 1314386 (0.0011) [2023-12-27 00:57:40,239][105692] Updated weights for policy 0, policy_version 1314396 (0.0011) [2023-12-27 00:57:40,288][105692] Updated weights for policy 0, policy_version 1314406 (0.0010) [2023-12-27 00:57:40,333][105692] Updated weights for policy 0, policy_version 1314416 (0.0010) [2023-12-27 00:57:40,437][105620] Updated weights for policy 1, policy_version 1316164 (0.0008) [2023-12-27 00:57:40,504][105620] Updated weights for policy 1, policy_version 1316174 (0.0008) [2023-12-27 00:57:40,568][105620] Updated weights for policy 1, policy_version 1316184 (0.0006) [2023-12-27 00:57:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 673529856. Throughput: 0: 9768.8, 1: 9566.3. Samples: 673542336. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:57:41,062][104569] Avg episode reward: [(0, '6584.884'), (1, '8989.502')] [2023-12-27 00:57:41,125][105692] Updated weights for policy 0, policy_version 1314426 (0.0011) [2023-12-27 00:57:41,157][105620] Updated weights for policy 1, policy_version 1316194 (0.0007) [2023-12-27 00:57:41,191][105692] Updated weights for policy 0, policy_version 1314436 (0.0011) [2023-12-27 00:57:41,221][105620] Updated weights for policy 1, policy_version 1316204 (0.0006) [2023-12-27 00:57:41,256][105692] Updated weights for policy 0, policy_version 1314446 (0.0010) [2023-12-27 00:57:41,287][105620] Updated weights for policy 1, policy_version 1316214 (0.0007) [2023-12-27 00:57:41,350][105620] Updated weights for policy 1, policy_version 1316224 (0.0008) [2023-12-27 00:57:42,014][105692] Updated weights for policy 0, policy_version 1314456 (0.0010) [2023-12-27 00:57:42,022][105620] Updated weights for policy 1, policy_version 1316234 (0.0006) [2023-12-27 00:57:42,074][105692] Updated weights for policy 0, policy_version 1314466 (0.0011) [2023-12-27 00:57:42,082][105620] Updated weights for policy 1, policy_version 1316244 (0.0008) [2023-12-27 00:57:42,131][105692] Updated weights for policy 0, policy_version 1314476 (0.0011) [2023-12-27 00:57:42,145][105620] Updated weights for policy 1, policy_version 1316254 (0.0007) [2023-12-27 00:57:42,846][105620] Updated weights for policy 1, policy_version 1316264 (0.0008) [2023-12-27 00:57:42,897][105620] Updated weights for policy 1, policy_version 1316274 (0.0007) [2023-12-27 00:57:42,940][105692] Updated weights for policy 0, policy_version 1314486 (0.0008) [2023-12-27 00:57:42,955][105620] Updated weights for policy 1, policy_version 1316284 (0.0008) [2023-12-27 00:57:43,007][105692] Updated weights for policy 0, policy_version 1314496 (0.0008) [2023-12-27 00:57:43,075][105692] Updated weights for policy 0, policy_version 1314506 (0.0008) [2023-12-27 00:57:43,603][105620] Updated weights for policy 1, policy_version 1316294 (0.0007) [2023-12-27 00:57:43,659][105620] Updated weights for policy 1, policy_version 1316304 (0.0005) [2023-12-27 00:57:43,678][105692] Updated weights for policy 0, policy_version 1314516 (0.0008) [2023-12-27 00:57:43,707][105620] Updated weights for policy 1, policy_version 1316314 (0.0005) [2023-12-27 00:57:43,740][105692] Updated weights for policy 0, policy_version 1314526 (0.0010) [2023-12-27 00:57:43,805][105692] Updated weights for policy 0, policy_version 1314536 (0.0010) [2023-12-27 00:57:44,260][105620] Updated weights for policy 1, policy_version 1316324 (0.0007) [2023-12-27 00:57:44,324][105620] Updated weights for policy 1, policy_version 1316334 (0.0009) [2023-12-27 00:57:44,403][105620] Updated weights for policy 1, policy_version 1316344 (0.0008) [2023-12-27 00:57:44,582][105692] Updated weights for policy 0, policy_version 1314546 (0.0010) [2023-12-27 00:57:44,644][105692] Updated weights for policy 0, policy_version 1314556 (0.0009) [2023-12-27 00:57:44,704][105692] Updated weights for policy 0, policy_version 1314566 (0.0009) [2023-12-27 00:57:44,760][105692] Updated weights for policy 0, policy_version 1314576 (0.0009) [2023-12-27 00:57:45,110][105620] Updated weights for policy 1, policy_version 1316354 (0.0009) [2023-12-27 00:57:45,167][105620] Updated weights for policy 1, policy_version 1316364 (0.0011) [2023-12-27 00:57:45,230][105620] Updated weights for policy 1, policy_version 1316374 (0.0011) [2023-12-27 00:57:45,283][105620] Updated weights for policy 1, policy_version 1316384 (0.0011) [2023-12-27 00:57:45,566][105692] Updated weights for policy 0, policy_version 1314586 (0.0008) [2023-12-27 00:57:45,627][105692] Updated weights for policy 0, policy_version 1314596 (0.0008) [2023-12-27 00:57:45,684][105692] Updated weights for policy 0, policy_version 1314606 (0.0008) [2023-12-27 00:57:46,062][105620] Updated weights for policy 1, policy_version 1316394 (0.0010) [2023-12-27 00:57:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 673628160. Throughput: 0: 9692.7, 1: 9547.7. Samples: 673601948. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:57:46,063][104569] Avg episode reward: [(0, '6341.112'), (1, '8810.650')] [2023-12-27 00:57:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001314608_336592896.pth... [2023-12-27 00:57:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001313488_336306176.pth [2023-12-27 00:57:46,110][105620] Updated weights for policy 1, policy_version 1316404 (0.0010) [2023-12-27 00:57:46,173][105620] Updated weights for policy 1, policy_version 1316414 (0.0010) [2023-12-27 00:57:46,184][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001316416_337043456.pth... [2023-12-27 00:57:46,188][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001315296_336756736.pth [2023-12-27 00:57:46,452][105692] Updated weights for policy 0, policy_version 1314616 (0.0008) [2023-12-27 00:57:46,511][105692] Updated weights for policy 0, policy_version 1314627 (0.0010) [2023-12-27 00:57:46,567][105692] Updated weights for policy 0, policy_version 1314640 (0.0011) [2023-12-27 00:57:46,816][105620] Updated weights for policy 1, policy_version 1316424 (0.0010) [2023-12-27 00:57:46,866][105620] Updated weights for policy 1, policy_version 1316434 (0.0009) [2023-12-27 00:57:46,925][105620] Updated weights for policy 1, policy_version 1316444 (0.0005) [2023-12-27 00:57:47,487][105692] Updated weights for policy 0, policy_version 1314650 (0.0009) [2023-12-27 00:57:47,497][105620] Updated weights for policy 1, policy_version 1316454 (0.0005) [2023-12-27 00:57:47,540][105692] Updated weights for policy 0, policy_version 1314660 (0.0009) [2023-12-27 00:57:47,551][105620] Updated weights for policy 1, policy_version 1316464 (0.0005) [2023-12-27 00:57:47,589][105692] Updated weights for policy 0, policy_version 1314670 (0.0008) [2023-12-27 00:57:47,612][105620] Updated weights for policy 1, policy_version 1316474 (0.0005) [2023-12-27 00:57:48,206][105620] Updated weights for policy 1, policy_version 1316484 (0.0007) [2023-12-27 00:57:48,267][105620] Updated weights for policy 1, policy_version 1316494 (0.0010) [2023-12-27 00:57:48,345][105620] Updated weights for policy 1, policy_version 1316504 (0.0010) [2023-12-27 00:57:48,451][105692] Updated weights for policy 0, policy_version 1314680 (0.0008) [2023-12-27 00:57:48,512][105692] Updated weights for policy 0, policy_version 1314690 (0.0008) [2023-12-27 00:57:48,558][105692] Updated weights for policy 0, policy_version 1314700 (0.0008) [2023-12-27 00:57:48,983][105620] Updated weights for policy 1, policy_version 1316514 (0.0010) [2023-12-27 00:57:49,041][105620] Updated weights for policy 1, policy_version 1316524 (0.0005) [2023-12-27 00:57:49,106][105620] Updated weights for policy 1, policy_version 1316534 (0.0006) [2023-12-27 00:57:49,160][105620] Updated weights for policy 1, policy_version 1316544 (0.0005) [2023-12-27 00:57:49,440][105692] Updated weights for policy 0, policy_version 1314710 (0.0010) [2023-12-27 00:57:49,494][105692] Updated weights for policy 0, policy_version 1314720 (0.0010) [2023-12-27 00:57:49,548][105692] Updated weights for policy 0, policy_version 1314730 (0.0010) [2023-12-27 00:57:49,725][105620] Updated weights for policy 1, policy_version 1316554 (0.0005) [2023-12-27 00:57:49,783][105620] Updated weights for policy 1, policy_version 1316564 (0.0005) [2023-12-27 00:57:49,849][105620] Updated weights for policy 1, policy_version 1316574 (0.0008) [2023-12-27 00:57:50,364][105692] Updated weights for policy 0, policy_version 1314740 (0.0008) [2023-12-27 00:57:50,427][105585] KL-divergence is very high: 233.9644 [2023-12-27 00:57:50,428][105692] Updated weights for policy 0, policy_version 1314750 (0.0009) [2023-12-27 00:57:50,472][105585] KL-divergence is very high: 350.4597 [2023-12-27 00:57:50,478][105620] Updated weights for policy 1, policy_version 1316584 (0.0008) [2023-12-27 00:57:50,484][105692] Updated weights for policy 0, policy_version 1314760 (0.0008) [2023-12-27 00:57:50,524][105585] KL-divergence is very high: 400.3741 [2023-12-27 00:57:50,541][105620] Updated weights for policy 1, policy_version 1316594 (0.0007) [2023-12-27 00:57:50,609][105620] Updated weights for policy 1, policy_version 1316604 (0.0008) [2023-12-27 00:57:51,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.8, 300 sec: 19521.9). Total num frames: 673726464. Throughput: 0: 9533.6, 1: 9679.8. Samples: 673716980. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:57:51,063][104569] Avg episode reward: [(0, '7353.461'), (1, '9085.770')] [2023-12-27 00:57:51,217][105692] Updated weights for policy 0, policy_version 1314770 (0.0009) [2023-12-27 00:57:51,286][105692] Updated weights for policy 0, policy_version 1314780 (0.0009) [2023-12-27 00:57:51,353][105692] Updated weights for policy 0, policy_version 1314790 (0.0008) [2023-12-27 00:57:51,385][105620] Updated weights for policy 1, policy_version 1316614 (0.0009) [2023-12-27 00:57:51,417][105692] Updated weights for policy 0, policy_version 1314800 (0.0009) [2023-12-27 00:57:51,448][105620] Updated weights for policy 1, policy_version 1316624 (0.0010) [2023-12-27 00:57:51,499][105620] Updated weights for policy 1, policy_version 1316634 (0.0009) [2023-12-27 00:57:52,095][105692] Updated weights for policy 0, policy_version 1314810 (0.0007) [2023-12-27 00:57:52,151][105692] Updated weights for policy 0, policy_version 1314820 (0.0009) [2023-12-27 00:57:52,214][105692] Updated weights for policy 0, policy_version 1314830 (0.0009) [2023-12-27 00:57:52,312][105620] Updated weights for policy 1, policy_version 1316644 (0.0009) [2023-12-27 00:57:52,375][105620] Updated weights for policy 1, policy_version 1316654 (0.0009) [2023-12-27 00:57:52,434][105620] Updated weights for policy 1, policy_version 1316664 (0.0009) [2023-12-27 00:57:52,956][105692] Updated weights for policy 0, policy_version 1314840 (0.0009) [2023-12-27 00:57:53,019][105692] Updated weights for policy 0, policy_version 1314850 (0.0009) [2023-12-27 00:57:53,080][105692] Updated weights for policy 0, policy_version 1314860 (0.0007) [2023-12-27 00:57:53,194][105620] Updated weights for policy 1, policy_version 1316674 (0.0009) [2023-12-27 00:57:53,246][105620] Updated weights for policy 1, policy_version 1316684 (0.0009) [2023-12-27 00:57:53,293][105620] Updated weights for policy 1, policy_version 1316694 (0.0009) [2023-12-27 00:57:53,355][105620] Updated weights for policy 1, policy_version 1316704 (0.0010) [2023-12-27 00:57:53,859][105692] Updated weights for policy 0, policy_version 1314870 (0.0009) [2023-12-27 00:57:53,914][105692] Updated weights for policy 0, policy_version 1314880 (0.0010) [2023-12-27 00:57:53,960][105692] Updated weights for policy 0, policy_version 1314890 (0.0009) [2023-12-27 00:57:54,011][105620] Updated weights for policy 1, policy_version 1316714 (0.0009) [2023-12-27 00:57:54,065][105620] Updated weights for policy 1, policy_version 1316724 (0.0009) [2023-12-27 00:57:54,125][105620] Updated weights for policy 1, policy_version 1316734 (0.0008) [2023-12-27 00:57:54,762][105620] Updated weights for policy 1, policy_version 1316744 (0.0008) [2023-12-27 00:57:54,798][105692] Updated weights for policy 0, policy_version 1314900 (0.0009) [2023-12-27 00:57:54,813][105620] Updated weights for policy 1, policy_version 1316754 (0.0006) [2023-12-27 00:57:54,860][105692] Updated weights for policy 0, policy_version 1314910 (0.0008) [2023-12-27 00:57:54,867][105620] Updated weights for policy 1, policy_version 1316764 (0.0005) [2023-12-27 00:57:54,919][105692] Updated weights for policy 0, policy_version 1314920 (0.0008) [2023-12-27 00:57:55,505][105620] Updated weights for policy 1, policy_version 1316774 (0.0008) [2023-12-27 00:57:55,570][105620] Updated weights for policy 1, policy_version 1316784 (0.0009) [2023-12-27 00:57:55,635][105620] Updated weights for policy 1, policy_version 1316794 (0.0009) [2023-12-27 00:57:55,672][105692] Updated weights for policy 0, policy_version 1314930 (0.0009) [2023-12-27 00:57:55,721][105692] Updated weights for policy 0, policy_version 1314940 (0.0009) [2023-12-27 00:57:55,773][105692] Updated weights for policy 0, policy_version 1314950 (0.0009) [2023-12-27 00:57:55,824][105692] Updated weights for policy 0, policy_version 1314960 (0.0009) [2023-12-27 00:57:56,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 673824768. Throughput: 0: 9498.2, 1: 9627.8. Samples: 673830436. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:57:56,062][104569] Avg episode reward: [(0, '8307.354'), (1, '9175.666')] [2023-12-27 00:57:56,341][105620] Updated weights for policy 1, policy_version 1316804 (0.0007) [2023-12-27 00:57:56,396][105620] Updated weights for policy 1, policy_version 1316814 (0.0008) [2023-12-27 00:57:56,458][105620] Updated weights for policy 1, policy_version 1316824 (0.0008) [2023-12-27 00:57:56,570][105692] Updated weights for policy 0, policy_version 1314970 (0.0010) [2023-12-27 00:57:56,620][105692] Updated weights for policy 0, policy_version 1314980 (0.0010) [2023-12-27 00:57:56,669][105692] Updated weights for policy 0, policy_version 1314990 (0.0006) [2023-12-27 00:57:57,188][105620] Updated weights for policy 1, policy_version 1316834 (0.0008) [2023-12-27 00:57:57,247][105620] Updated weights for policy 1, policy_version 1316844 (0.0009) [2023-12-27 00:57:57,308][105620] Updated weights for policy 1, policy_version 1316854 (0.0008) [2023-12-27 00:57:57,369][105692] Updated weights for policy 0, policy_version 1315000 (0.0006) [2023-12-27 00:57:57,371][105620] Updated weights for policy 1, policy_version 1316864 (0.0009) [2023-12-27 00:57:57,418][105692] Updated weights for policy 0, policy_version 1315010 (0.0007) [2023-12-27 00:57:57,467][105692] Updated weights for policy 0, policy_version 1315020 (0.0008) [2023-12-27 00:57:58,074][105620] Updated weights for policy 1, policy_version 1316874 (0.0010) [2023-12-27 00:57:58,089][105692] Updated weights for policy 0, policy_version 1315030 (0.0009) [2023-12-27 00:57:58,129][105620] Updated weights for policy 1, policy_version 1316884 (0.0010) [2023-12-27 00:57:58,140][105692] Updated weights for policy 0, policy_version 1315040 (0.0010) [2023-12-27 00:57:58,189][105620] Updated weights for policy 1, policy_version 1316894 (0.0010) [2023-12-27 00:57:58,210][105692] Updated weights for policy 0, policy_version 1315050 (0.0008) [2023-12-27 00:57:58,988][105692] Updated weights for policy 0, policy_version 1315060 (0.0008) [2023-12-27 00:57:59,030][105620] Updated weights for policy 1, policy_version 1316904 (0.0007) [2023-12-27 00:57:59,048][105692] Updated weights for policy 0, policy_version 1315070 (0.0007) [2023-12-27 00:57:59,098][105692] Updated weights for policy 0, policy_version 1315080 (0.0007) [2023-12-27 00:57:59,101][105620] Updated weights for policy 1, policy_version 1316914 (0.0006) [2023-12-27 00:57:59,171][105620] Updated weights for policy 1, policy_version 1316924 (0.0010) [2023-12-27 00:57:59,842][105620] Updated weights for policy 1, policy_version 1316934 (0.0009) [2023-12-27 00:57:59,855][105692] Updated weights for policy 0, policy_version 1315090 (0.0007) [2023-12-27 00:57:59,902][105620] Updated weights for policy 1, policy_version 1316944 (0.0010) [2023-12-27 00:57:59,921][105692] Updated weights for policy 0, policy_version 1315100 (0.0006) [2023-12-27 00:57:59,960][105620] Updated weights for policy 1, policy_version 1316954 (0.0007) [2023-12-27 00:57:59,979][105692] Updated weights for policy 0, policy_version 1315110 (0.0009) [2023-12-27 00:58:00,032][105692] Updated weights for policy 0, policy_version 1315120 (0.0009) [2023-12-27 00:58:00,689][105620] Updated weights for policy 1, policy_version 1316964 (0.0008) [2023-12-27 00:58:00,738][105620] Updated weights for policy 1, policy_version 1316974 (0.0009) [2023-12-27 00:58:00,778][105692] Updated weights for policy 0, policy_version 1315130 (0.0006) [2023-12-27 00:58:00,796][105620] Updated weights for policy 1, policy_version 1316984 (0.0009) [2023-12-27 00:58:00,841][105692] Updated weights for policy 0, policy_version 1315140 (0.0006) [2023-12-27 00:58:00,900][105692] Updated weights for policy 0, policy_version 1315150 (0.0008) [2023-12-27 00:58:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 673923072. Throughput: 0: 9550.9, 1: 9649.1. Samples: 673888900. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:58:01,062][104569] Avg episode reward: [(0, '8472.445'), (1, '9178.011')] [2023-12-27 00:58:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001315152_336732160.pth... [2023-12-27 00:58:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001316992_337190912.pth... [2023-12-27 00:58:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001314064_336453632.pth [2023-12-27 00:58:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001315840_336896000.pth [2023-12-27 00:58:01,568][105620] Updated weights for policy 1, policy_version 1316994 (0.0010) [2023-12-27 00:58:01,632][105620] Updated weights for policy 1, policy_version 1317004 (0.0008) [2023-12-27 00:58:01,668][105692] Updated weights for policy 0, policy_version 1315160 (0.0008) [2023-12-27 00:58:01,699][105620] Updated weights for policy 1, policy_version 1317014 (0.0007) [2023-12-27 00:58:01,731][105692] Updated weights for policy 0, policy_version 1315170 (0.0009) [2023-12-27 00:58:01,762][105620] Updated weights for policy 1, policy_version 1317024 (0.0008) [2023-12-27 00:58:01,797][105692] Updated weights for policy 0, policy_version 1315180 (0.0006) [2023-12-27 00:58:02,419][105620] Updated weights for policy 1, policy_version 1317034 (0.0009) [2023-12-27 00:58:02,421][105692] Updated weights for policy 0, policy_version 1315190 (0.0007) [2023-12-27 00:58:02,479][105620] Updated weights for policy 1, policy_version 1317044 (0.0007) [2023-12-27 00:58:02,482][105692] Updated weights for policy 0, policy_version 1315200 (0.0006) [2023-12-27 00:58:02,538][105620] Updated weights for policy 1, policy_version 1317054 (0.0007) [2023-12-27 00:58:02,544][105692] Updated weights for policy 0, policy_version 1315210 (0.0007) [2023-12-27 00:58:03,272][105692] Updated weights for policy 0, policy_version 1315220 (0.0009) [2023-12-27 00:58:03,306][105620] Updated weights for policy 1, policy_version 1317064 (0.0007) [2023-12-27 00:58:03,325][105692] Updated weights for policy 0, policy_version 1315230 (0.0008) [2023-12-27 00:58:03,350][105585] KL-divergence is very high: 134.0027 [2023-12-27 00:58:03,363][105620] Updated weights for policy 1, policy_version 1317074 (0.0008) [2023-12-27 00:58:03,377][105692] Updated weights for policy 0, policy_version 1315240 (0.0009) [2023-12-27 00:58:03,392][105585] KL-divergence is very high: 152.7830 [2023-12-27 00:58:03,422][105620] Updated weights for policy 1, policy_version 1317084 (0.0008) [2023-12-27 00:58:04,086][105620] Updated weights for policy 1, policy_version 1317094 (0.0009) [2023-12-27 00:58:04,145][105620] Updated weights for policy 1, policy_version 1317104 (0.0008) [2023-12-27 00:58:04,181][105692] Updated weights for policy 0, policy_version 1315250 (0.0006) [2023-12-27 00:58:04,212][105620] Updated weights for policy 1, policy_version 1317114 (0.0006) [2023-12-27 00:58:04,242][105692] Updated weights for policy 0, policy_version 1315260 (0.0006) [2023-12-27 00:58:04,299][105692] Updated weights for policy 0, policy_version 1315270 (0.0009) [2023-12-27 00:58:04,359][105692] Updated weights for policy 0, policy_version 1315280 (0.0007) [2023-12-27 00:58:04,870][105620] Updated weights for policy 1, policy_version 1317124 (0.0007) [2023-12-27 00:58:04,933][105620] Updated weights for policy 1, policy_version 1317134 (0.0008) [2023-12-27 00:58:04,996][105620] Updated weights for policy 1, policy_version 1317144 (0.0009) [2023-12-27 00:58:05,137][105692] Updated weights for policy 0, policy_version 1315290 (0.0008) [2023-12-27 00:58:05,191][105692] Updated weights for policy 0, policy_version 1315300 (0.0007) [2023-12-27 00:58:05,251][105692] Updated weights for policy 0, policy_version 1315310 (0.0005) [2023-12-27 00:58:05,782][105620] Updated weights for policy 1, policy_version 1317154 (0.0008) [2023-12-27 00:58:05,828][105620] Updated weights for policy 1, policy_version 1317164 (0.0006) [2023-12-27 00:58:05,851][105692] Updated weights for policy 0, policy_version 1315320 (0.0006) [2023-12-27 00:58:05,884][105620] Updated weights for policy 1, policy_version 1317174 (0.0005) [2023-12-27 00:58:05,908][105692] Updated weights for policy 0, policy_version 1315330 (0.0006) [2023-12-27 00:58:05,941][105620] Updated weights for policy 1, policy_version 1317184 (0.0005) [2023-12-27 00:58:05,962][105692] Updated weights for policy 0, policy_version 1315340 (0.0010) [2023-12-27 00:58:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 674021376. Throughput: 0: 9493.5, 1: 9696.3. Samples: 674003760. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:58:06,062][104569] Avg episode reward: [(0, '8285.132'), (1, '9086.824')] [2023-12-27 00:58:06,574][105620] Updated weights for policy 1, policy_version 1317194 (0.0006) [2023-12-27 00:58:06,644][105620] Updated weights for policy 1, policy_version 1317204 (0.0006) [2023-12-27 00:58:06,654][105692] Updated weights for policy 0, policy_version 1315350 (0.0011) [2023-12-27 00:58:06,704][105620] Updated weights for policy 1, policy_version 1317214 (0.0006) [2023-12-27 00:58:06,714][105692] Updated weights for policy 0, policy_version 1315360 (0.0011) [2023-12-27 00:58:06,780][105692] Updated weights for policy 0, policy_version 1315370 (0.0009) [2023-12-27 00:58:07,442][105692] Updated weights for policy 0, policy_version 1315380 (0.0009) [2023-12-27 00:58:07,457][105620] Updated weights for policy 1, policy_version 1317224 (0.0007) [2023-12-27 00:58:07,501][105692] Updated weights for policy 0, policy_version 1315390 (0.0011) [2023-12-27 00:58:07,520][105620] Updated weights for policy 1, policy_version 1317234 (0.0006) [2023-12-27 00:58:07,549][105692] Updated weights for policy 0, policy_version 1315400 (0.0010) [2023-12-27 00:58:07,569][105620] Updated weights for policy 1, policy_version 1317244 (0.0007) [2023-12-27 00:58:08,238][105692] Updated weights for policy 0, policy_version 1315410 (0.0009) [2023-12-27 00:58:08,290][105692] Updated weights for policy 0, policy_version 1315420 (0.0005) [2023-12-27 00:58:08,296][105620] Updated weights for policy 1, policy_version 1317254 (0.0008) [2023-12-27 00:58:08,353][105692] Updated weights for policy 0, policy_version 1315430 (0.0008) [2023-12-27 00:58:08,359][105620] Updated weights for policy 1, policy_version 1317264 (0.0007) [2023-12-27 00:58:08,411][105692] Updated weights for policy 0, policy_version 1315440 (0.0007) [2023-12-27 00:58:08,425][105620] Updated weights for policy 1, policy_version 1317274 (0.0008) [2023-12-27 00:58:08,993][105692] Updated weights for policy 0, policy_version 1315450 (0.0009) [2023-12-27 00:58:09,042][105692] Updated weights for policy 0, policy_version 1315460 (0.0009) [2023-12-27 00:58:09,049][105620] Updated weights for policy 1, policy_version 1317284 (0.0007) [2023-12-27 00:58:09,090][105692] Updated weights for policy 0, policy_version 1315470 (0.0007) [2023-12-27 00:58:09,101][105620] Updated weights for policy 1, policy_version 1317294 (0.0008) [2023-12-27 00:58:09,154][105620] Updated weights for policy 1, policy_version 1317304 (0.0009) [2023-12-27 00:58:09,816][105692] Updated weights for policy 0, policy_version 1315480 (0.0010) [2023-12-27 00:58:09,876][105692] Updated weights for policy 0, policy_version 1315490 (0.0007) [2023-12-27 00:58:09,936][105692] Updated weights for policy 0, policy_version 1315500 (0.0010) [2023-12-27 00:58:09,973][105620] Updated weights for policy 1, policy_version 1317314 (0.0009) [2023-12-27 00:58:10,029][105620] Updated weights for policy 1, policy_version 1317324 (0.0008) [2023-12-27 00:58:10,088][105620] Updated weights for policy 1, policy_version 1317334 (0.0008) [2023-12-27 00:58:10,153][105620] Updated weights for policy 1, policy_version 1317344 (0.0009) [2023-12-27 00:58:10,628][105692] Updated weights for policy 0, policy_version 1315510 (0.0011) [2023-12-27 00:58:10,680][105692] Updated weights for policy 0, policy_version 1315520 (0.0010) [2023-12-27 00:58:10,743][105692] Updated weights for policy 0, policy_version 1315530 (0.0011) [2023-12-27 00:58:10,975][105620] Updated weights for policy 1, policy_version 1317354 (0.0009) [2023-12-27 00:58:11,035][105620] Updated weights for policy 1, policy_version 1317365 (0.0010) [2023-12-27 00:58:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 674111488. Throughput: 0: 9595.2, 1: 9705.4. Samples: 674121920. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:58:11,062][104569] Avg episode reward: [(0, '8379.700'), (1, '8996.548')] [2023-12-27 00:58:11,093][105620] Updated weights for policy 1, policy_version 1317375 (0.0008) [2023-12-27 00:58:11,368][105692] Updated weights for policy 0, policy_version 1315540 (0.0008) [2023-12-27 00:58:11,428][105692] Updated weights for policy 0, policy_version 1315550 (0.0008) [2023-12-27 00:58:11,481][105692] Updated weights for policy 0, policy_version 1315560 (0.0009) [2023-12-27 00:58:11,945][105620] Updated weights for policy 1, policy_version 1317385 (0.0009) [2023-12-27 00:58:12,012][105620] Updated weights for policy 1, policy_version 1317395 (0.0009) [2023-12-27 00:58:12,076][105620] Updated weights for policy 1, policy_version 1317405 (0.0009) [2023-12-27 00:58:12,296][105692] Updated weights for policy 0, policy_version 1315570 (0.0010) [2023-12-27 00:58:12,364][105692] Updated weights for policy 0, policy_version 1315580 (0.0008) [2023-12-27 00:58:12,429][105692] Updated weights for policy 0, policy_version 1315590 (0.0009) [2023-12-27 00:58:12,495][105692] Updated weights for policy 0, policy_version 1315600 (0.0010) [2023-12-27 00:58:12,768][105620] Updated weights for policy 1, policy_version 1317415 (0.0009) [2023-12-27 00:58:12,835][105620] Updated weights for policy 1, policy_version 1317425 (0.0006) [2023-12-27 00:58:12,896][105620] Updated weights for policy 1, policy_version 1317435 (0.0005) [2023-12-27 00:58:13,279][105692] Updated weights for policy 0, policy_version 1315610 (0.0009) [2023-12-27 00:58:13,344][105692] Updated weights for policy 0, policy_version 1315620 (0.0009) [2023-12-27 00:58:13,402][105692] Updated weights for policy 0, policy_version 1315630 (0.0009) [2023-12-27 00:58:13,616][105620] Updated weights for policy 1, policy_version 1317445 (0.0008) [2023-12-27 00:58:13,669][105620] Updated weights for policy 1, policy_version 1317455 (0.0005) [2023-12-27 00:58:13,715][105620] Updated weights for policy 1, policy_version 1317465 (0.0005) [2023-12-27 00:58:14,141][105692] Updated weights for policy 0, policy_version 1315640 (0.0009) [2023-12-27 00:58:14,194][105692] Updated weights for policy 0, policy_version 1315650 (0.0008) [2023-12-27 00:58:14,248][105692] Updated weights for policy 0, policy_version 1315660 (0.0009) [2023-12-27 00:58:14,357][105620] Updated weights for policy 1, policy_version 1317475 (0.0007) [2023-12-27 00:58:14,415][105620] Updated weights for policy 1, policy_version 1317485 (0.0009) [2023-12-27 00:58:14,470][105620] Updated weights for policy 1, policy_version 1317495 (0.0009) [2023-12-27 00:58:15,014][105692] Updated weights for policy 0, policy_version 1315670 (0.0009) [2023-12-27 00:58:15,066][105692] Updated weights for policy 0, policy_version 1315680 (0.0008) [2023-12-27 00:58:15,120][105692] Updated weights for policy 0, policy_version 1315690 (0.0009) [2023-12-27 00:58:15,208][105620] Updated weights for policy 1, policy_version 1317505 (0.0008) [2023-12-27 00:58:15,271][105620] Updated weights for policy 1, policy_version 1317515 (0.0005) [2023-12-27 00:58:15,335][105620] Updated weights for policy 1, policy_version 1317525 (0.0008) [2023-12-27 00:58:15,400][105620] Updated weights for policy 1, policy_version 1317535 (0.0007) [2023-12-27 00:58:15,929][105692] Updated weights for policy 0, policy_version 1315700 (0.0010) [2023-12-27 00:58:15,986][105692] Updated weights for policy 0, policy_version 1315710 (0.0008) [2023-12-27 00:58:16,036][105692] Updated weights for policy 0, policy_version 1315720 (0.0009) [2023-12-27 00:58:16,062][104569] Fps is (10 sec: 18022.1, 60 sec: 19114.6, 300 sec: 19494.2). Total num frames: 674201600. Throughput: 0: 9537.7, 1: 9696.2. Samples: 674178524. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:58:16,063][104569] Avg episode reward: [(0, '8909.028'), (1, '9089.301')] [2023-12-27 00:58:16,073][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001315728_336879616.pth... [2023-12-27 00:58:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001314608_336592896.pth [2023-12-27 00:58:16,083][105620] Updated weights for policy 1, policy_version 1317545 (0.0010) [2023-12-27 00:58:16,141][105620] Updated weights for policy 1, policy_version 1317555 (0.0009) [2023-12-27 00:58:16,204][105620] Updated weights for policy 1, policy_version 1317565 (0.0009) [2023-12-27 00:58:16,220][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001317568_337338368.pth... [2023-12-27 00:58:16,223][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001316416_337043456.pth [2023-12-27 00:58:16,734][105692] Updated weights for policy 0, policy_version 1315730 (0.0009) [2023-12-27 00:58:16,793][105692] Updated weights for policy 0, policy_version 1315740 (0.0009) [2023-12-27 00:58:16,847][105692] Updated weights for policy 0, policy_version 1315750 (0.0009) [2023-12-27 00:58:16,902][105692] Updated weights for policy 0, policy_version 1315760 (0.0009) [2023-12-27 00:58:16,982][105620] Updated weights for policy 1, policy_version 1317575 (0.0006) [2023-12-27 00:58:17,028][105620] Updated weights for policy 1, policy_version 1317585 (0.0005) [2023-12-27 00:58:17,085][105620] Updated weights for policy 1, policy_version 1317595 (0.0010) [2023-12-27 00:58:17,599][105692] Updated weights for policy 0, policy_version 1315770 (0.0010) [2023-12-27 00:58:17,647][105692] Updated weights for policy 0, policy_version 1315780 (0.0010) [2023-12-27 00:58:17,696][105692] Updated weights for policy 0, policy_version 1315790 (0.0010) [2023-12-27 00:58:17,831][105620] Updated weights for policy 1, policy_version 1317605 (0.0008) [2023-12-27 00:58:17,887][105620] Updated weights for policy 1, policy_version 1317615 (0.0008) [2023-12-27 00:58:17,953][105620] Updated weights for policy 1, policy_version 1317625 (0.0008) [2023-12-27 00:58:18,480][105692] Updated weights for policy 0, policy_version 1315800 (0.0009) [2023-12-27 00:58:18,546][105692] Updated weights for policy 0, policy_version 1315810 (0.0008) [2023-12-27 00:58:18,611][105692] Updated weights for policy 0, policy_version 1315820 (0.0009) [2023-12-27 00:58:18,665][105620] Updated weights for policy 1, policy_version 1317635 (0.0008) [2023-12-27 00:58:18,728][105620] Updated weights for policy 1, policy_version 1317645 (0.0009) [2023-12-27 00:58:18,793][105620] Updated weights for policy 1, policy_version 1317655 (0.0009) [2023-12-27 00:58:19,298][105692] Updated weights for policy 0, policy_version 1315830 (0.0011) [2023-12-27 00:58:19,368][105692] Updated weights for policy 0, policy_version 1315840 (0.0010) [2023-12-27 00:58:19,434][105692] Updated weights for policy 0, policy_version 1315850 (0.0009) [2023-12-27 00:58:19,551][105620] Updated weights for policy 1, policy_version 1317665 (0.0008) [2023-12-27 00:58:19,612][105620] Updated weights for policy 1, policy_version 1317675 (0.0007) [2023-12-27 00:58:19,671][105620] Updated weights for policy 1, policy_version 1317685 (0.0006) [2023-12-27 00:58:19,731][105620] Updated weights for policy 1, policy_version 1317695 (0.0007) [2023-12-27 00:58:20,271][105692] Updated weights for policy 0, policy_version 1315860 (0.0010) [2023-12-27 00:58:20,323][105692] Updated weights for policy 0, policy_version 1315870 (0.0009) [2023-12-27 00:58:20,382][105692] Updated weights for policy 0, policy_version 1315880 (0.0007) [2023-12-27 00:58:20,394][105620] Updated weights for policy 1, policy_version 1317705 (0.0008) [2023-12-27 00:58:20,447][105620] Updated weights for policy 1, policy_version 1317715 (0.0009) [2023-12-27 00:58:20,503][105620] Updated weights for policy 1, policy_version 1317725 (0.0008) [2023-12-27 00:58:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 674299904. Throughput: 0: 9471.2, 1: 9731.4. Samples: 674292292. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 00:58:21,063][104569] Avg episode reward: [(0, '9174.260'), (1, '9088.509')] [2023-12-27 00:58:21,146][105692] Updated weights for policy 0, policy_version 1315890 (0.0008) [2023-12-27 00:58:21,213][105692] Updated weights for policy 0, policy_version 1315900 (0.0009) [2023-12-27 00:58:21,278][105692] Updated weights for policy 0, policy_version 1315910 (0.0008) [2023-12-27 00:58:21,305][105620] Updated weights for policy 1, policy_version 1317735 (0.0007) [2023-12-27 00:58:21,344][105692] Updated weights for policy 0, policy_version 1315920 (0.0007) [2023-12-27 00:58:21,379][105620] Updated weights for policy 1, policy_version 1317745 (0.0009) [2023-12-27 00:58:21,446][105620] Updated weights for policy 1, policy_version 1317755 (0.0008) [2023-12-27 00:58:22,121][105692] Updated weights for policy 0, policy_version 1315930 (0.0009) [2023-12-27 00:58:22,176][105692] Updated weights for policy 0, policy_version 1315940 (0.0009) [2023-12-27 00:58:22,227][105620] Updated weights for policy 1, policy_version 1317765 (0.0008) [2023-12-27 00:58:22,229][105692] Updated weights for policy 0, policy_version 1315950 (0.0008) [2023-12-27 00:58:22,296][105620] Updated weights for policy 1, policy_version 1317775 (0.0009) [2023-12-27 00:58:22,358][105620] Updated weights for policy 1, policy_version 1317785 (0.0009) [2023-12-27 00:58:23,019][105692] Updated weights for policy 0, policy_version 1315960 (0.0008) [2023-12-27 00:58:23,067][105692] Updated weights for policy 0, policy_version 1315970 (0.0009) [2023-12-27 00:58:23,115][105692] Updated weights for policy 0, policy_version 1315980 (0.0009) [2023-12-27 00:58:23,139][105620] Updated weights for policy 1, policy_version 1317795 (0.0008) [2023-12-27 00:58:23,194][105620] Updated weights for policy 1, policy_version 1317805 (0.0010) [2023-12-27 00:58:23,242][105620] Updated weights for policy 1, policy_version 1317815 (0.0009) [2023-12-27 00:58:23,905][105692] Updated weights for policy 0, policy_version 1315990 (0.0009) [2023-12-27 00:58:23,967][105692] Updated weights for policy 0, policy_version 1316000 (0.0009) [2023-12-27 00:58:24,008][105620] Updated weights for policy 1, policy_version 1317825 (0.0009) [2023-12-27 00:58:24,014][105692] Updated weights for policy 0, policy_version 1316010 (0.0010) [2023-12-27 00:58:24,055][105620] Updated weights for policy 1, policy_version 1317835 (0.0008) [2023-12-27 00:58:24,102][105620] Updated weights for policy 1, policy_version 1317845 (0.0008) [2023-12-27 00:58:24,152][105620] Updated weights for policy 1, policy_version 1317855 (0.0008) [2023-12-27 00:58:24,784][105692] Updated weights for policy 0, policy_version 1316020 (0.0007) [2023-12-27 00:58:24,831][105692] Updated weights for policy 0, policy_version 1316030 (0.0008) [2023-12-27 00:58:24,879][105692] Updated weights for policy 0, policy_version 1316040 (0.0009) [2023-12-27 00:58:24,927][105620] Updated weights for policy 1, policy_version 1317865 (0.0008) [2023-12-27 00:58:24,984][105620] Updated weights for policy 1, policy_version 1317875 (0.0009) [2023-12-27 00:58:25,031][105620] Updated weights for policy 1, policy_version 1317885 (0.0009) [2023-12-27 00:58:25,656][105692] Updated weights for policy 0, policy_version 1316050 (0.0009) [2023-12-27 00:58:25,703][105692] Updated weights for policy 0, policy_version 1316060 (0.0005) [2023-12-27 00:58:25,751][105692] Updated weights for policy 0, policy_version 1316070 (0.0008) [2023-12-27 00:58:25,798][105692] Updated weights for policy 0, policy_version 1316080 (0.0009) [2023-12-27 00:58:25,801][105620] Updated weights for policy 1, policy_version 1317895 (0.0009) [2023-12-27 00:58:25,857][105620] Updated weights for policy 1, policy_version 1317905 (0.0009) [2023-12-27 00:58:25,912][105620] Updated weights for policy 1, policy_version 1317915 (0.0009) [2023-12-27 00:58:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 674398208. Throughput: 0: 9350.0, 1: 9742.7. Samples: 674401504. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 00:58:26,062][104569] Avg episode reward: [(0, '9083.200'), (1, '9016.315')] [2023-12-27 00:58:26,555][105692] Updated weights for policy 0, policy_version 1316090 (0.0010) [2023-12-27 00:58:26,612][105692] Updated weights for policy 0, policy_version 1316100 (0.0008) [2023-12-27 00:58:26,660][105620] Updated weights for policy 1, policy_version 1317925 (0.0009) [2023-12-27 00:58:26,669][105692] Updated weights for policy 0, policy_version 1316110 (0.0007) [2023-12-27 00:58:26,721][105620] Updated weights for policy 1, policy_version 1317935 (0.0008) [2023-12-27 00:58:26,774][105620] Updated weights for policy 1, policy_version 1317945 (0.0008) [2023-12-27 00:58:27,409][105692] Updated weights for policy 0, policy_version 1316120 (0.0008) [2023-12-27 00:58:27,478][105692] Updated weights for policy 0, policy_version 1316130 (0.0010) [2023-12-27 00:58:27,512][105620] Updated weights for policy 1, policy_version 1317955 (0.0008) [2023-12-27 00:58:27,537][105692] Updated weights for policy 0, policy_version 1316140 (0.0007) [2023-12-27 00:58:27,556][105620] Updated weights for policy 1, policy_version 1317965 (0.0006) [2023-12-27 00:58:27,605][105620] Updated weights for policy 1, policy_version 1317975 (0.0008) [2023-12-27 00:58:28,235][105620] Updated weights for policy 1, policy_version 1317985 (0.0009) [2023-12-27 00:58:28,287][105620] Updated weights for policy 1, policy_version 1317995 (0.0005) [2023-12-27 00:58:28,338][105620] Updated weights for policy 1, policy_version 1318005 (0.0006) [2023-12-27 00:58:28,363][105692] Updated weights for policy 0, policy_version 1316150 (0.0009) [2023-12-27 00:58:28,401][105620] Updated weights for policy 1, policy_version 1318015 (0.0006) [2023-12-27 00:58:28,420][105692] Updated weights for policy 0, policy_version 1316160 (0.0008) [2023-12-27 00:58:28,472][105692] Updated weights for policy 0, policy_version 1316170 (0.0009) [2023-12-27 00:58:28,980][105620] Updated weights for policy 1, policy_version 1318025 (0.0009) [2023-12-27 00:58:29,033][105620] Updated weights for policy 1, policy_version 1318035 (0.0009) [2023-12-27 00:58:29,086][105620] Updated weights for policy 1, policy_version 1318045 (0.0008) [2023-12-27 00:58:29,352][105692] Updated weights for policy 0, policy_version 1316180 (0.0010) [2023-12-27 00:58:29,415][105692] Updated weights for policy 0, policy_version 1316191 (0.0010) [2023-12-27 00:58:29,469][105692] Updated weights for policy 0, policy_version 1316202 (0.0009) [2023-12-27 00:58:29,721][105620] Updated weights for policy 1, policy_version 1318055 (0.0008) [2023-12-27 00:58:29,782][105620] Updated weights for policy 1, policy_version 1318065 (0.0009) [2023-12-27 00:58:29,845][105620] Updated weights for policy 1, policy_version 1318075 (0.0008) [2023-12-27 00:58:30,298][105692] Updated weights for policy 0, policy_version 1316212 (0.0009) [2023-12-27 00:58:30,360][105692] Updated weights for policy 0, policy_version 1316222 (0.0010) [2023-12-27 00:58:30,413][105692] Updated weights for policy 0, policy_version 1316232 (0.0009) [2023-12-27 00:58:30,490][105620] Updated weights for policy 1, policy_version 1318085 (0.0009) [2023-12-27 00:58:30,546][105620] Updated weights for policy 1, policy_version 1318095 (0.0009) [2023-12-27 00:58:30,600][105620] Updated weights for policy 1, policy_version 1318105 (0.0009) [2023-12-27 00:58:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 674488320. Throughput: 0: 9325.0, 1: 9743.4. Samples: 674460020. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 00:58:31,062][104569] Avg episode reward: [(0, '9082.530'), (1, '8923.768')] [2023-12-27 00:58:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001316240_337010688.pth... [2023-12-27 00:58:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001318112_337477632.pth... [2023-12-27 00:58:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001316992_337190912.pth [2023-12-27 00:58:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001315152_336732160.pth [2023-12-27 00:58:31,177][105692] Updated weights for policy 0, policy_version 1316242 (0.0006) [2023-12-27 00:58:31,225][105692] Updated weights for policy 0, policy_version 1316252 (0.0009) [2023-12-27 00:58:31,250][105620] Updated weights for policy 1, policy_version 1318115 (0.0007) [2023-12-27 00:58:31,281][105692] Updated weights for policy 0, policy_version 1316262 (0.0010) [2023-12-27 00:58:31,311][105620] Updated weights for policy 1, policy_version 1318125 (0.0008) [2023-12-27 00:58:31,342][105692] Updated weights for policy 0, policy_version 1316272 (0.0006) [2023-12-27 00:58:31,379][105620] Updated weights for policy 1, policy_version 1318135 (0.0009) [2023-12-27 00:58:32,105][105692] Updated weights for policy 0, policy_version 1316282 (0.0009) [2023-12-27 00:58:32,155][105692] Updated weights for policy 0, policy_version 1316292 (0.0009) [2023-12-27 00:58:32,169][105620] Updated weights for policy 1, policy_version 1318145 (0.0010) [2023-12-27 00:58:32,211][105692] Updated weights for policy 0, policy_version 1316302 (0.0006) [2023-12-27 00:58:32,221][105620] Updated weights for policy 1, policy_version 1318155 (0.0008) [2023-12-27 00:58:32,276][105620] Updated weights for policy 1, policy_version 1318165 (0.0009) [2023-12-27 00:58:32,331][105620] Updated weights for policy 1, policy_version 1318175 (0.0009) [2023-12-27 00:58:32,953][105692] Updated weights for policy 0, policy_version 1316312 (0.0008) [2023-12-27 00:58:33,000][105692] Updated weights for policy 0, policy_version 1316322 (0.0009) [2023-12-27 00:58:33,060][105692] Updated weights for policy 0, policy_version 1316332 (0.0008) [2023-12-27 00:58:33,119][105620] Updated weights for policy 1, policy_version 1318185 (0.0010) [2023-12-27 00:58:33,179][105620] Updated weights for policy 1, policy_version 1318195 (0.0009) [2023-12-27 00:58:33,241][105620] Updated weights for policy 1, policy_version 1318205 (0.0009) [2023-12-27 00:58:33,848][105620] Updated weights for policy 1, policy_version 1318215 (0.0010) [2023-12-27 00:58:33,881][105692] Updated weights for policy 0, policy_version 1316342 (0.0010) [2023-12-27 00:58:33,906][105620] Updated weights for policy 1, policy_version 1318225 (0.0010) [2023-12-27 00:58:33,937][105692] Updated weights for policy 0, policy_version 1316352 (0.0006) [2023-12-27 00:58:33,966][105620] Updated weights for policy 1, policy_version 1318235 (0.0007) [2023-12-27 00:58:33,987][105692] Updated weights for policy 0, policy_version 1316362 (0.0008) [2023-12-27 00:58:34,671][105620] Updated weights for policy 1, policy_version 1318245 (0.0009) [2023-12-27 00:58:34,724][105620] Updated weights for policy 1, policy_version 1318255 (0.0010) [2023-12-27 00:58:34,772][105620] Updated weights for policy 1, policy_version 1318265 (0.0010) [2023-12-27 00:58:34,787][105692] Updated weights for policy 0, policy_version 1316372 (0.0008) [2023-12-27 00:58:34,848][105692] Updated weights for policy 0, policy_version 1316382 (0.0007) [2023-12-27 00:58:34,910][105692] Updated weights for policy 0, policy_version 1316392 (0.0007) [2023-12-27 00:58:35,521][105620] Updated weights for policy 1, policy_version 1318275 (0.0011) [2023-12-27 00:58:35,535][105692] Updated weights for policy 0, policy_version 1316402 (0.0008) [2023-12-27 00:58:35,576][105620] Updated weights for policy 1, policy_version 1318285 (0.0010) [2023-12-27 00:58:35,583][105692] Updated weights for policy 0, policy_version 1316412 (0.0010) [2023-12-27 00:58:35,631][105620] Updated weights for policy 1, policy_version 1318295 (0.0010) [2023-12-27 00:58:35,645][105692] Updated weights for policy 0, policy_version 1316422 (0.0008) [2023-12-27 00:58:35,709][105692] Updated weights for policy 0, policy_version 1316432 (0.0008) [2023-12-27 00:58:36,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 674586624. Throughput: 0: 9385.8, 1: 9648.5. Samples: 674573520. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 00:58:36,063][104569] Avg episode reward: [(0, '9087.642'), (1, '8817.842')] [2023-12-27 00:58:36,335][105692] Updated weights for policy 0, policy_version 1316442 (0.0008) [2023-12-27 00:58:36,377][105620] Updated weights for policy 1, policy_version 1318305 (0.0010) [2023-12-27 00:58:36,397][105692] Updated weights for policy 0, policy_version 1316452 (0.0006) [2023-12-27 00:58:36,436][105620] Updated weights for policy 1, policy_version 1318315 (0.0011) [2023-12-27 00:58:36,454][105692] Updated weights for policy 0, policy_version 1316462 (0.0005) [2023-12-27 00:58:36,496][105620] Updated weights for policy 1, policy_version 1318325 (0.0010) [2023-12-27 00:58:36,554][105620] Updated weights for policy 1, policy_version 1318335 (0.0007) [2023-12-27 00:58:37,088][105692] Updated weights for policy 0, policy_version 1316472 (0.0005) [2023-12-27 00:58:37,150][105692] Updated weights for policy 0, policy_version 1316482 (0.0006) [2023-12-27 00:58:37,217][105692] Updated weights for policy 0, policy_version 1316492 (0.0005) [2023-12-27 00:58:37,291][105620] Updated weights for policy 1, policy_version 1318345 (0.0010) [2023-12-27 00:58:37,349][105620] Updated weights for policy 1, policy_version 1318355 (0.0010) [2023-12-27 00:58:37,411][105620] Updated weights for policy 1, policy_version 1318365 (0.0010) [2023-12-27 00:58:37,797][105692] Updated weights for policy 0, policy_version 1316502 (0.0008) [2023-12-27 00:58:37,858][105692] Updated weights for policy 0, policy_version 1316512 (0.0008) [2023-12-27 00:58:37,915][105692] Updated weights for policy 0, policy_version 1316522 (0.0007) [2023-12-27 00:58:38,170][105620] Updated weights for policy 1, policy_version 1318375 (0.0010) [2023-12-27 00:58:38,231][105620] Updated weights for policy 1, policy_version 1318385 (0.0010) [2023-12-27 00:58:38,283][105620] Updated weights for policy 1, policy_version 1318395 (0.0010) [2023-12-27 00:58:38,555][105692] Updated weights for policy 0, policy_version 1316532 (0.0006) [2023-12-27 00:58:38,614][105692] Updated weights for policy 0, policy_version 1316542 (0.0008) [2023-12-27 00:58:38,673][105692] Updated weights for policy 0, policy_version 1316552 (0.0008) [2023-12-27 00:58:39,037][105620] Updated weights for policy 1, policy_version 1318405 (0.0011) [2023-12-27 00:58:39,095][105620] Updated weights for policy 1, policy_version 1318415 (0.0010) [2023-12-27 00:58:39,149][105620] Updated weights for policy 1, policy_version 1318425 (0.0010) [2023-12-27 00:58:39,480][105692] Updated weights for policy 0, policy_version 1316562 (0.0008) [2023-12-27 00:58:39,535][105692] Updated weights for policy 0, policy_version 1316572 (0.0009) [2023-12-27 00:58:39,590][105692] Updated weights for policy 0, policy_version 1316582 (0.0009) [2023-12-27 00:58:39,638][105692] Updated weights for policy 0, policy_version 1316592 (0.0009) [2023-12-27 00:58:39,884][105620] Updated weights for policy 1, policy_version 1318435 (0.0010) [2023-12-27 00:58:39,946][105620] Updated weights for policy 1, policy_version 1318445 (0.0007) [2023-12-27 00:58:39,999][105620] Updated weights for policy 1, policy_version 1318455 (0.0008) [2023-12-27 00:58:40,439][105692] Updated weights for policy 0, policy_version 1316602 (0.0009) [2023-12-27 00:58:40,486][105692] Updated weights for policy 0, policy_version 1316612 (0.0008) [2023-12-27 00:58:40,545][105692] Updated weights for policy 0, policy_version 1316622 (0.0009) [2023-12-27 00:58:40,750][105620] Updated weights for policy 1, policy_version 1318465 (0.0009) [2023-12-27 00:58:40,802][105620] Updated weights for policy 1, policy_version 1318475 (0.0009) [2023-12-27 00:58:40,850][105620] Updated weights for policy 1, policy_version 1318485 (0.0006) [2023-12-27 00:58:40,912][105620] Updated weights for policy 1, policy_version 1318495 (0.0006) [2023-12-27 00:58:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 674684928. Throughput: 0: 9508.6, 1: 9605.1. Samples: 674690552. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 00:58:41,062][104569] Avg episode reward: [(0, '8997.135'), (1, '8732.282')] [2023-12-27 00:58:41,412][105692] Updated weights for policy 0, policy_version 1316632 (0.0009) [2023-12-27 00:58:41,463][105692] Updated weights for policy 0, policy_version 1316642 (0.0009) [2023-12-27 00:58:41,518][105692] Updated weights for policy 0, policy_version 1316652 (0.0010) [2023-12-27 00:58:41,591][105620] Updated weights for policy 1, policy_version 1318505 (0.0007) [2023-12-27 00:58:41,658][105620] Updated weights for policy 1, policy_version 1318515 (0.0009) [2023-12-27 00:58:41,730][105620] Updated weights for policy 1, policy_version 1318525 (0.0010) [2023-12-27 00:58:42,343][105692] Updated weights for policy 0, policy_version 1316662 (0.0009) [2023-12-27 00:58:42,419][105692] Updated weights for policy 0, policy_version 1316672 (0.0008) [2023-12-27 00:58:42,474][105620] Updated weights for policy 1, policy_version 1318535 (0.0007) [2023-12-27 00:58:42,483][105692] Updated weights for policy 0, policy_version 1316682 (0.0009) [2023-12-27 00:58:42,502][105585] KL-divergence is very high: 117.1823 [2023-12-27 00:58:42,535][105620] Updated weights for policy 1, policy_version 1318545 (0.0008) [2023-12-27 00:58:42,599][105620] Updated weights for policy 1, policy_version 1318555 (0.0009) [2023-12-27 00:58:43,180][105692] Updated weights for policy 0, policy_version 1316692 (0.0009) [2023-12-27 00:58:43,237][105692] Updated weights for policy 0, policy_version 1316702 (0.0009) [2023-12-27 00:58:43,298][105692] Updated weights for policy 0, policy_version 1316713 (0.0008) [2023-12-27 00:58:43,300][105620] Updated weights for policy 1, policy_version 1318565 (0.0009) [2023-12-27 00:58:43,354][105620] Updated weights for policy 1, policy_version 1318575 (0.0008) [2023-12-27 00:58:43,409][105620] Updated weights for policy 1, policy_version 1318585 (0.0009) [2023-12-27 00:58:44,007][105692] Updated weights for policy 0, policy_version 1316723 (0.0007) [2023-12-27 00:58:44,061][105692] Updated weights for policy 0, policy_version 1316733 (0.0009) [2023-12-27 00:58:44,111][105692] Updated weights for policy 0, policy_version 1316743 (0.0008) [2023-12-27 00:58:44,175][105620] Updated weights for policy 1, policy_version 1318595 (0.0009) [2023-12-27 00:58:44,235][105620] Updated weights for policy 1, policy_version 1318605 (0.0009) [2023-12-27 00:58:44,296][105620] Updated weights for policy 1, policy_version 1318615 (0.0008) [2023-12-27 00:58:44,943][105692] Updated weights for policy 0, policy_version 1316753 (0.0009) [2023-12-27 00:58:44,959][105620] Updated weights for policy 1, policy_version 1318625 (0.0005) [2023-12-27 00:58:45,000][105692] Updated weights for policy 0, policy_version 1316763 (0.0007) [2023-12-27 00:58:45,018][105620] Updated weights for policy 1, policy_version 1318635 (0.0008) [2023-12-27 00:58:45,054][105692] Updated weights for policy 0, policy_version 1316773 (0.0007) [2023-12-27 00:58:45,073][105620] Updated weights for policy 1, policy_version 1318645 (0.0006) [2023-12-27 00:58:45,119][105692] Updated weights for policy 0, policy_version 1316783 (0.0009) [2023-12-27 00:58:45,131][105620] Updated weights for policy 1, policy_version 1318655 (0.0006) [2023-12-27 00:58:45,870][105692] Updated weights for policy 0, policy_version 1316793 (0.0008) [2023-12-27 00:58:45,901][105620] Updated weights for policy 1, policy_version 1318665 (0.0007) [2023-12-27 00:58:45,919][105692] Updated weights for policy 0, policy_version 1316803 (0.0006) [2023-12-27 00:58:45,953][105620] Updated weights for policy 1, policy_version 1318675 (0.0007) [2023-12-27 00:58:45,972][105692] Updated weights for policy 0, policy_version 1316813 (0.0006) [2023-12-27 00:58:46,012][105620] Updated weights for policy 1, policy_version 1318685 (0.0008) [2023-12-27 00:58:46,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.3, 300 sec: 19438.6). Total num frames: 674783232. Throughput: 0: 9440.5, 1: 9618.9. Samples: 674746568. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 00:58:46,062][104569] Avg episode reward: [(0, '9127.334'), (1, '8913.117')] [2023-12-27 00:58:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001316816_337158144.pth... [2023-12-27 00:58:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001318688_337625088.pth... [2023-12-27 00:58:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001315728_336879616.pth [2023-12-27 00:58:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001317568_337338368.pth [2023-12-27 00:58:46,614][105692] Updated weights for policy 0, policy_version 1316823 (0.0005) [2023-12-27 00:58:46,673][105692] Updated weights for policy 0, policy_version 1316833 (0.0005) [2023-12-27 00:58:46,728][105692] Updated weights for policy 0, policy_version 1316843 (0.0005) [2023-12-27 00:58:46,798][105620] Updated weights for policy 1, policy_version 1318695 (0.0010) [2023-12-27 00:58:46,851][105620] Updated weights for policy 1, policy_version 1318705 (0.0009) [2023-12-27 00:58:46,913][105620] Updated weights for policy 1, policy_version 1318715 (0.0010) [2023-12-27 00:58:47,429][105692] Updated weights for policy 0, policy_version 1316853 (0.0008) [2023-12-27 00:58:47,473][105692] Updated weights for policy 0, policy_version 1316863 (0.0007) [2023-12-27 00:58:47,517][105692] Updated weights for policy 0, policy_version 1316873 (0.0008) [2023-12-27 00:58:47,653][105620] Updated weights for policy 1, policy_version 1318725 (0.0010) [2023-12-27 00:58:47,711][105620] Updated weights for policy 1, policy_version 1318735 (0.0010) [2023-12-27 00:58:47,773][105620] Updated weights for policy 1, policy_version 1318745 (0.0010) [2023-12-27 00:58:48,313][105692] Updated weights for policy 0, policy_version 1316883 (0.0007) [2023-12-27 00:58:48,378][105692] Updated weights for policy 0, policy_version 1316893 (0.0008) [2023-12-27 00:58:48,438][105692] Updated weights for policy 0, policy_version 1316903 (0.0007) [2023-12-27 00:58:48,509][105620] Updated weights for policy 1, policy_version 1318755 (0.0010) [2023-12-27 00:58:48,561][105620] Updated weights for policy 1, policy_version 1318765 (0.0008) [2023-12-27 00:58:48,616][105620] Updated weights for policy 1, policy_version 1318775 (0.0009) [2023-12-27 00:58:49,187][105692] Updated weights for policy 0, policy_version 1316913 (0.0008) [2023-12-27 00:58:49,250][105692] Updated weights for policy 0, policy_version 1316923 (0.0009) [2023-12-27 00:58:49,308][105692] Updated weights for policy 0, policy_version 1316933 (0.0009) [2023-12-27 00:58:49,367][105692] Updated weights for policy 0, policy_version 1316943 (0.0008) [2023-12-27 00:58:49,401][105620] Updated weights for policy 1, policy_version 1318785 (0.0009) [2023-12-27 00:58:49,468][105620] Updated weights for policy 1, policy_version 1318795 (0.0007) [2023-12-27 00:58:49,532][105620] Updated weights for policy 1, policy_version 1318805 (0.0007) [2023-12-27 00:58:49,584][105620] Updated weights for policy 1, policy_version 1318815 (0.0009) [2023-12-27 00:58:50,120][105692] Updated weights for policy 0, policy_version 1316953 (0.0006) [2023-12-27 00:58:50,179][105692] Updated weights for policy 0, policy_version 1316963 (0.0006) [2023-12-27 00:58:50,240][105692] Updated weights for policy 0, policy_version 1316973 (0.0007) [2023-12-27 00:58:50,349][105620] Updated weights for policy 1, policy_version 1318825 (0.0010) [2023-12-27 00:58:50,410][105620] Updated weights for policy 1, policy_version 1318835 (0.0010) [2023-12-27 00:58:50,472][105620] Updated weights for policy 1, policy_version 1318845 (0.0010) [2023-12-27 00:58:50,941][105692] Updated weights for policy 0, policy_version 1316983 (0.0009) [2023-12-27 00:58:51,011][105692] Updated weights for policy 0, policy_version 1316993 (0.0010) [2023-12-27 00:58:51,062][104569] Fps is (10 sec: 18022.2, 60 sec: 18978.1, 300 sec: 19383.1). Total num frames: 674865152. Throughput: 0: 9451.7, 1: 9562.8. Samples: 674859416. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 00:58:51,063][104569] Avg episode reward: [(0, '9038.927'), (1, '9270.332')] [2023-12-27 00:58:51,077][105692] Updated weights for policy 0, policy_version 1317003 (0.0008) [2023-12-27 00:58:51,165][105620] Updated weights for policy 1, policy_version 1318855 (0.0009) [2023-12-27 00:58:51,232][105620] Updated weights for policy 1, policy_version 1318865 (0.0010) [2023-12-27 00:58:51,294][105620] Updated weights for policy 1, policy_version 1318875 (0.0009) [2023-12-27 00:58:51,776][105692] Updated weights for policy 0, policy_version 1317013 (0.0008) [2023-12-27 00:58:51,839][105692] Updated weights for policy 0, policy_version 1317023 (0.0011) [2023-12-27 00:58:51,908][105692] Updated weights for policy 0, policy_version 1317033 (0.0008) [2023-12-27 00:58:52,027][105620] Updated weights for policy 1, policy_version 1318885 (0.0007) [2023-12-27 00:58:52,076][105620] Updated weights for policy 1, policy_version 1318895 (0.0006) [2023-12-27 00:58:52,131][105620] Updated weights for policy 1, policy_version 1318905 (0.0009) [2023-12-27 00:58:52,517][105692] Updated weights for policy 0, policy_version 1317043 (0.0006) [2023-12-27 00:58:52,586][105692] Updated weights for policy 0, policy_version 1317053 (0.0006) [2023-12-27 00:58:52,644][105692] Updated weights for policy 0, policy_version 1317063 (0.0009) [2023-12-27 00:58:52,906][105620] Updated weights for policy 1, policy_version 1318915 (0.0009) [2023-12-27 00:58:52,954][105620] Updated weights for policy 1, policy_version 1318925 (0.0008) [2023-12-27 00:58:53,001][105620] Updated weights for policy 1, policy_version 1318935 (0.0008) [2023-12-27 00:58:53,326][105692] Updated weights for policy 0, policy_version 1317073 (0.0010) [2023-12-27 00:58:53,391][105692] Updated weights for policy 0, policy_version 1317083 (0.0010) [2023-12-27 00:58:53,446][105692] Updated weights for policy 0, policy_version 1317093 (0.0010) [2023-12-27 00:58:53,508][105692] Updated weights for policy 0, policy_version 1317103 (0.0011) [2023-12-27 00:58:53,768][105620] Updated weights for policy 1, policy_version 1318945 (0.0008) [2023-12-27 00:58:53,818][105620] Updated weights for policy 1, policy_version 1318955 (0.0007) [2023-12-27 00:58:53,866][105620] Updated weights for policy 1, policy_version 1318965 (0.0008) [2023-12-27 00:58:53,910][105620] Updated weights for policy 1, policy_version 1318975 (0.0008) [2023-12-27 00:58:54,256][105692] Updated weights for policy 0, policy_version 1317113 (0.0009) [2023-12-27 00:58:54,316][105692] Updated weights for policy 0, policy_version 1317123 (0.0009) [2023-12-27 00:58:54,364][105692] Updated weights for policy 0, policy_version 1317133 (0.0010) [2023-12-27 00:58:54,676][105620] Updated weights for policy 1, policy_version 1318985 (0.0008) [2023-12-27 00:58:54,739][105620] Updated weights for policy 1, policy_version 1318995 (0.0008) [2023-12-27 00:58:54,790][105620] Updated weights for policy 1, policy_version 1319005 (0.0007) [2023-12-27 00:58:55,096][105692] Updated weights for policy 0, policy_version 1317143 (0.0010) [2023-12-27 00:58:55,154][105692] Updated weights for policy 0, policy_version 1317153 (0.0010) [2023-12-27 00:58:55,218][105692] Updated weights for policy 0, policy_version 1317163 (0.0010) [2023-12-27 00:58:55,557][105620] Updated weights for policy 1, policy_version 1319015 (0.0008) [2023-12-27 00:58:55,615][105620] Updated weights for policy 1, policy_version 1319025 (0.0008) [2023-12-27 00:58:55,667][105620] Updated weights for policy 1, policy_version 1319035 (0.0008) [2023-12-27 00:58:55,955][105692] Updated weights for policy 0, policy_version 1317173 (0.0010) [2023-12-27 00:58:56,013][105692] Updated weights for policy 0, policy_version 1317183 (0.0010) [2023-12-27 00:58:56,062][104569] Fps is (10 sec: 18022.1, 60 sec: 18978.1, 300 sec: 19410.9). Total num frames: 674963456. Throughput: 0: 9398.6, 1: 9571.2. Samples: 674975564. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 00:58:56,063][104569] Avg episode reward: [(0, '8848.029'), (1, '9104.560')] [2023-12-27 00:58:56,074][105692] Updated weights for policy 0, policy_version 1317193 (0.0010) [2023-12-27 00:58:56,321][105620] Updated weights for policy 1, policy_version 1319045 (0.0009) [2023-12-27 00:58:56,382][105620] Updated weights for policy 1, policy_version 1319055 (0.0010) [2023-12-27 00:58:56,442][105620] Updated weights for policy 1, policy_version 1319065 (0.0008) [2023-12-27 00:58:56,773][105692] Updated weights for policy 0, policy_version 1317203 (0.0009) [2023-12-27 00:58:56,825][105692] Updated weights for policy 0, policy_version 1317213 (0.0005) [2023-12-27 00:58:56,879][105692] Updated weights for policy 0, policy_version 1317223 (0.0005) [2023-12-27 00:58:56,982][105620] Updated weights for policy 1, policy_version 1319075 (0.0008) [2023-12-27 00:58:57,025][105620] Updated weights for policy 1, policy_version 1319085 (0.0005) [2023-12-27 00:58:57,069][105620] Updated weights for policy 1, policy_version 1319095 (0.0005) [2023-12-27 00:58:57,382][105692] Updated weights for policy 0, policy_version 1317233 (0.0005) [2023-12-27 00:58:57,415][105585] KL-divergence is very high: 127.0425 [2023-12-27 00:58:57,424][105585] KL-divergence is very high: 105.5979 [2023-12-27 00:58:57,429][105692] Updated weights for policy 0, policy_version 1317243 (0.0009) [2023-12-27 00:58:57,450][105585] KL-divergence is very high: 177.5571 [2023-12-27 00:58:57,460][105585] KL-divergence is very high: 156.3806 [2023-12-27 00:58:57,476][105692] Updated weights for policy 0, policy_version 1317253 (0.0009) [2023-12-27 00:58:57,488][105585] KL-divergence is very high: 174.4996 [2023-12-27 00:58:57,498][105585] KL-divergence is very high: 148.5983 [2023-12-27 00:58:57,523][105692] Updated weights for policy 0, policy_version 1317263 (0.0009) [2023-12-27 00:58:57,796][105620] Updated weights for policy 1, policy_version 1319105 (0.0006) [2023-12-27 00:58:57,851][105620] Updated weights for policy 1, policy_version 1319115 (0.0009) [2023-12-27 00:58:57,901][105620] Updated weights for policy 1, policy_version 1319125 (0.0009) [2023-12-27 00:58:57,957][105620] Updated weights for policy 1, policy_version 1319135 (0.0008) [2023-12-27 00:58:58,216][105692] Updated weights for policy 0, policy_version 1317273 (0.0008) [2023-12-27 00:58:58,276][105692] Updated weights for policy 0, policy_version 1317283 (0.0008) [2023-12-27 00:58:58,349][105692] Updated weights for policy 0, policy_version 1317293 (0.0008) [2023-12-27 00:58:58,694][105620] Updated weights for policy 1, policy_version 1319145 (0.0008) [2023-12-27 00:58:58,759][105620] Updated weights for policy 1, policy_version 1319155 (0.0008) [2023-12-27 00:58:58,830][105620] Updated weights for policy 1, policy_version 1319165 (0.0009) [2023-12-27 00:58:59,197][105692] Updated weights for policy 0, policy_version 1317303 (0.0008) [2023-12-27 00:58:59,261][105692] Updated weights for policy 0, policy_version 1317313 (0.0009) [2023-12-27 00:58:59,273][105585] KL-divergence is very high: 127.1755 [2023-12-27 00:58:59,325][105585] KL-divergence is very high: 137.8740 [2023-12-27 00:58:59,325][105692] Updated weights for policy 0, policy_version 1317323 (0.0008) [2023-12-27 00:58:59,616][105620] Updated weights for policy 1, policy_version 1319175 (0.0007) [2023-12-27 00:58:59,675][105620] Updated weights for policy 1, policy_version 1319185 (0.0005) [2023-12-27 00:58:59,733][105586] KL-divergence is very high: 155.9097 [2023-12-27 00:58:59,738][105620] Updated weights for policy 1, policy_version 1319195 (0.0005) [2023-12-27 00:59:00,056][105692] Updated weights for policy 0, policy_version 1317333 (0.0008) [2023-12-27 00:59:00,115][105692] Updated weights for policy 0, policy_version 1317343 (0.0009) [2023-12-27 00:59:00,170][105692] Updated weights for policy 0, policy_version 1317353 (0.0009) [2023-12-27 00:59:00,382][105620] Updated weights for policy 1, policy_version 1319205 (0.0007) [2023-12-27 00:59:00,429][105620] Updated weights for policy 1, policy_version 1319215 (0.0008) [2023-12-27 00:59:00,483][105620] Updated weights for policy 1, policy_version 1319225 (0.0007) [2023-12-27 00:59:00,875][105692] Updated weights for policy 0, policy_version 1317363 (0.0009) [2023-12-27 00:59:00,937][105692] Updated weights for policy 0, policy_version 1317373 (0.0008) [2023-12-27 00:59:00,993][105692] Updated weights for policy 0, policy_version 1317383 (0.0008) [2023-12-27 00:59:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19114.6, 300 sec: 19438.6). Total num frames: 675069952. Throughput: 0: 9488.4, 1: 9600.6. Samples: 675037528. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 00:59:01,063][104569] Avg episode reward: [(0, '8234.419'), (1, '7658.598')] [2023-12-27 00:59:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001317392_337305600.pth... [2023-12-27 00:59:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001319232_337764352.pth... [2023-12-27 00:59:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001318112_337477632.pth [2023-12-27 00:59:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001316240_337010688.pth [2023-12-27 00:59:01,213][105620] Updated weights for policy 1, policy_version 1319235 (0.0010) [2023-12-27 00:59:01,273][105620] Updated weights for policy 1, policy_version 1319245 (0.0010) [2023-12-27 00:59:01,336][105620] Updated weights for policy 1, policy_version 1319255 (0.0010) [2023-12-27 00:59:01,782][105692] Updated weights for policy 0, policy_version 1317393 (0.0008) [2023-12-27 00:59:01,847][105692] Updated weights for policy 0, policy_version 1317403 (0.0009) [2023-12-27 00:59:01,903][105692] Updated weights for policy 0, policy_version 1317413 (0.0009) [2023-12-27 00:59:01,963][105692] Updated weights for policy 0, policy_version 1317423 (0.0008) [2023-12-27 00:59:02,081][105620] Updated weights for policy 1, policy_version 1319265 (0.0011) [2023-12-27 00:59:02,146][105620] Updated weights for policy 1, policy_version 1319275 (0.0010) [2023-12-27 00:59:02,205][105620] Updated weights for policy 1, policy_version 1319285 (0.0010) [2023-12-27 00:59:02,264][105620] Updated weights for policy 1, policy_version 1319295 (0.0010) [2023-12-27 00:59:02,707][105692] Updated weights for policy 0, policy_version 1317433 (0.0007) [2023-12-27 00:59:02,767][105692] Updated weights for policy 0, policy_version 1317443 (0.0008) [2023-12-27 00:59:02,829][105692] Updated weights for policy 0, policy_version 1317453 (0.0007) [2023-12-27 00:59:02,970][105620] Updated weights for policy 1, policy_version 1319305 (0.0010) [2023-12-27 00:59:03,027][105620] Updated weights for policy 1, policy_version 1319315 (0.0010) [2023-12-27 00:59:03,085][105620] Updated weights for policy 1, policy_version 1319325 (0.0010) [2023-12-27 00:59:03,585][105692] Updated weights for policy 0, policy_version 1317463 (0.0008) [2023-12-27 00:59:03,651][105692] Updated weights for policy 0, policy_version 1317473 (0.0006) [2023-12-27 00:59:03,709][105692] Updated weights for policy 0, policy_version 1317483 (0.0008) [2023-12-27 00:59:03,809][105620] Updated weights for policy 1, policy_version 1319335 (0.0010) [2023-12-27 00:59:03,872][105620] Updated weights for policy 1, policy_version 1319345 (0.0010) [2023-12-27 00:59:03,924][105620] Updated weights for policy 1, policy_version 1319355 (0.0010) [2023-12-27 00:59:04,442][105692] Updated weights for policy 0, policy_version 1317493 (0.0008) [2023-12-27 00:59:04,503][105692] Updated weights for policy 0, policy_version 1317503 (0.0009) [2023-12-27 00:59:04,564][105692] Updated weights for policy 0, policy_version 1317513 (0.0009) [2023-12-27 00:59:04,698][105620] Updated weights for policy 1, policy_version 1319365 (0.0009) [2023-12-27 00:59:04,756][105620] Updated weights for policy 1, policy_version 1319375 (0.0005) [2023-12-27 00:59:04,820][105620] Updated weights for policy 1, policy_version 1319385 (0.0008) [2023-12-27 00:59:05,248][105692] Updated weights for policy 0, policy_version 1317523 (0.0007) [2023-12-27 00:59:05,305][105692] Updated weights for policy 0, policy_version 1317533 (0.0005) [2023-12-27 00:59:05,366][105692] Updated weights for policy 0, policy_version 1317543 (0.0005) [2023-12-27 00:59:05,575][105620] Updated weights for policy 1, policy_version 1319395 (0.0007) [2023-12-27 00:59:05,635][105620] Updated weights for policy 1, policy_version 1319405 (0.0005) [2023-12-27 00:59:05,695][105620] Updated weights for policy 1, policy_version 1319415 (0.0005) [2023-12-27 00:59:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 18978.1, 300 sec: 19410.9). Total num frames: 675160064. Throughput: 0: 9462.7, 1: 9615.2. Samples: 675150800. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 00:59:06,062][104569] Avg episode reward: [(0, '8563.387'), (1, '4673.833')] [2023-12-27 00:59:06,082][105692] Updated weights for policy 0, policy_version 1317553 (0.0006) [2023-12-27 00:59:06,143][105692] Updated weights for policy 0, policy_version 1317563 (0.0009) [2023-12-27 00:59:06,196][105692] Updated weights for policy 0, policy_version 1317573 (0.0009) [2023-12-27 00:59:06,242][105620] Updated weights for policy 1, policy_version 1319425 (0.0005) [2023-12-27 00:59:06,256][105692] Updated weights for policy 0, policy_version 1317583 (0.0009) [2023-12-27 00:59:06,303][105620] Updated weights for policy 1, policy_version 1319435 (0.0008) [2023-12-27 00:59:06,354][105620] Updated weights for policy 1, policy_version 1319445 (0.0008) [2023-12-27 00:59:06,405][105620] Updated weights for policy 1, policy_version 1319455 (0.0009) [2023-12-27 00:59:06,991][105692] Updated weights for policy 0, policy_version 1317593 (0.0006) [2023-12-27 00:59:07,047][105692] Updated weights for policy 0, policy_version 1317603 (0.0005) [2023-12-27 00:59:07,102][105692] Updated weights for policy 0, policy_version 1317613 (0.0006) [2023-12-27 00:59:07,286][105620] Updated weights for policy 1, policy_version 1319465 (0.0009) [2023-12-27 00:59:07,334][105620] Updated weights for policy 1, policy_version 1319475 (0.0008) [2023-12-27 00:59:07,389][105620] Updated weights for policy 1, policy_version 1319485 (0.0009) [2023-12-27 00:59:07,801][105692] Updated weights for policy 0, policy_version 1317623 (0.0007) [2023-12-27 00:59:07,867][105692] Updated weights for policy 0, policy_version 1317633 (0.0005) [2023-12-27 00:59:07,926][105692] Updated weights for policy 0, policy_version 1317643 (0.0009) [2023-12-27 00:59:08,014][105620] Updated weights for policy 1, policy_version 1319495 (0.0007) [2023-12-27 00:59:08,067][105620] Updated weights for policy 1, policy_version 1319505 (0.0005) [2023-12-27 00:59:08,129][105620] Updated weights for policy 1, policy_version 1319515 (0.0005) [2023-12-27 00:59:08,589][105692] Updated weights for policy 0, policy_version 1317653 (0.0008) [2023-12-27 00:59:08,647][105692] Updated weights for policy 0, policy_version 1317663 (0.0006) [2023-12-27 00:59:08,716][105692] Updated weights for policy 0, policy_version 1317673 (0.0006) [2023-12-27 00:59:08,833][105620] Updated weights for policy 1, policy_version 1319525 (0.0008) [2023-12-27 00:59:08,892][105620] Updated weights for policy 1, policy_version 1319535 (0.0009) [2023-12-27 00:59:08,951][105620] Updated weights for policy 1, policy_version 1319545 (0.0009) [2023-12-27 00:59:09,333][105692] Updated weights for policy 0, policy_version 1317683 (0.0009) [2023-12-27 00:59:09,404][105692] Updated weights for policy 0, policy_version 1317693 (0.0008) [2023-12-27 00:59:09,468][105692] Updated weights for policy 0, policy_version 1317703 (0.0007) [2023-12-27 00:59:09,756][105620] Updated weights for policy 1, policy_version 1319555 (0.0009) [2023-12-27 00:59:09,819][105620] Updated weights for policy 1, policy_version 1319565 (0.0006) [2023-12-27 00:59:09,881][105620] Updated weights for policy 1, policy_version 1319575 (0.0008) [2023-12-27 00:59:10,195][105692] Updated weights for policy 0, policy_version 1317713 (0.0006) [2023-12-27 00:59:10,262][105692] Updated weights for policy 0, policy_version 1317723 (0.0009) [2023-12-27 00:59:10,316][105692] Updated weights for policy 0, policy_version 1317733 (0.0010) [2023-12-27 00:59:10,366][105692] Updated weights for policy 0, policy_version 1317743 (0.0009) [2023-12-27 00:59:10,541][105620] Updated weights for policy 1, policy_version 1319585 (0.0008) [2023-12-27 00:59:10,605][105620] Updated weights for policy 1, policy_version 1319595 (0.0007) [2023-12-27 00:59:10,650][105620] Updated weights for policy 1, policy_version 1319605 (0.0010) [2023-12-27 00:59:10,698][105620] Updated weights for policy 1, policy_version 1319615 (0.0011) [2023-12-27 00:59:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.6, 300 sec: 19410.9). Total num frames: 675258368. Throughput: 0: 9581.5, 1: 9708.9. Samples: 675269572. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 00:59:11,063][104569] Avg episode reward: [(0, '8898.511'), (1, '7150.041')] [2023-12-27 00:59:11,063][105692] Updated weights for policy 0, policy_version 1317753 (0.0007) [2023-12-27 00:59:11,130][105692] Updated weights for policy 0, policy_version 1317763 (0.0007) [2023-12-27 00:59:11,196][105692] Updated weights for policy 0, policy_version 1317773 (0.0006) [2023-12-27 00:59:11,423][105620] Updated weights for policy 1, policy_version 1319625 (0.0007) [2023-12-27 00:59:11,470][105620] Updated weights for policy 1, policy_version 1319635 (0.0007) [2023-12-27 00:59:11,537][105620] Updated weights for policy 1, policy_version 1319645 (0.0005) [2023-12-27 00:59:11,913][105692] Updated weights for policy 0, policy_version 1317783 (0.0010) [2023-12-27 00:59:11,973][105692] Updated weights for policy 0, policy_version 1317793 (0.0010) [2023-12-27 00:59:12,027][105692] Updated weights for policy 0, policy_version 1317803 (0.0009) [2023-12-27 00:59:12,246][105620] Updated weights for policy 1, policy_version 1319655 (0.0010) [2023-12-27 00:59:12,310][105620] Updated weights for policy 1, policy_version 1319665 (0.0011) [2023-12-27 00:59:12,370][105620] Updated weights for policy 1, policy_version 1319675 (0.0011) [2023-12-27 00:59:12,715][105692] Updated weights for policy 0, policy_version 1317813 (0.0006) [2023-12-27 00:59:12,777][105692] Updated weights for policy 0, policy_version 1317823 (0.0006) [2023-12-27 00:59:12,847][105692] Updated weights for policy 0, policy_version 1317833 (0.0007) [2023-12-27 00:59:13,060][105620] Updated weights for policy 1, policy_version 1319685 (0.0011) [2023-12-27 00:59:13,108][105620] Updated weights for policy 1, policy_version 1319695 (0.0010) [2023-12-27 00:59:13,160][105620] Updated weights for policy 1, policy_version 1319705 (0.0010) [2023-12-27 00:59:13,390][105692] Updated weights for policy 0, policy_version 1317843 (0.0005) [2023-12-27 00:59:13,446][105692] Updated weights for policy 0, policy_version 1317853 (0.0005) [2023-12-27 00:59:13,498][105692] Updated weights for policy 0, policy_version 1317863 (0.0006) [2023-12-27 00:59:13,761][105620] Updated weights for policy 1, policy_version 1319715 (0.0008) [2023-12-27 00:59:13,830][105620] Updated weights for policy 1, policy_version 1319725 (0.0005) [2023-12-27 00:59:13,893][105620] Updated weights for policy 1, policy_version 1319735 (0.0007) [2023-12-27 00:59:14,162][105692] Updated weights for policy 0, policy_version 1317873 (0.0006) [2023-12-27 00:59:14,220][105692] Updated weights for policy 0, policy_version 1317883 (0.0008) [2023-12-27 00:59:14,274][105692] Updated weights for policy 0, policy_version 1317893 (0.0008) [2023-12-27 00:59:14,322][105692] Updated weights for policy 0, policy_version 1317903 (0.0008) [2023-12-27 00:59:14,478][105620] Updated weights for policy 1, policy_version 1319745 (0.0005) [2023-12-27 00:59:14,550][105620] Updated weights for policy 1, policy_version 1319755 (0.0006) [2023-12-27 00:59:14,597][105620] Updated weights for policy 1, policy_version 1319765 (0.0010) [2023-12-27 00:59:14,650][105620] Updated weights for policy 1, policy_version 1319775 (0.0010) [2023-12-27 00:59:15,151][105692] Updated weights for policy 0, policy_version 1317913 (0.0008) [2023-12-27 00:59:15,215][105692] Updated weights for policy 0, policy_version 1317923 (0.0008) [2023-12-27 00:59:15,278][105692] Updated weights for policy 0, policy_version 1317933 (0.0008) [2023-12-27 00:59:15,393][105620] Updated weights for policy 1, policy_version 1319785 (0.0010) [2023-12-27 00:59:15,445][105620] Updated weights for policy 1, policy_version 1319795 (0.0010) [2023-12-27 00:59:15,496][105620] Updated weights for policy 1, policy_version 1319805 (0.0010) [2023-12-27 00:59:16,040][105692] Updated weights for policy 0, policy_version 1317943 (0.0006) [2023-12-27 00:59:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 675356672. Throughput: 0: 9688.6, 1: 9690.5. Samples: 675332084. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 00:59:16,063][104569] Avg episode reward: [(0, '8840.004'), (1, '8927.121')] [2023-12-27 00:59:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001319808_337911808.pth... [2023-12-27 00:59:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001318688_337625088.pth [2023-12-27 00:59:16,085][105692] Updated weights for policy 0, policy_version 1317953 (0.0005) [2023-12-27 00:59:16,138][105692] Updated weights for policy 0, policy_version 1317963 (0.0007) [2023-12-27 00:59:16,162][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001317968_337453056.pth... [2023-12-27 00:59:16,165][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001316816_337158144.pth [2023-12-27 00:59:16,257][105620] Updated weights for policy 1, policy_version 1319815 (0.0010) [2023-12-27 00:59:16,301][105620] Updated weights for policy 1, policy_version 1319825 (0.0010) [2023-12-27 00:59:16,348][105620] Updated weights for policy 1, policy_version 1319835 (0.0010) [2023-12-27 00:59:16,904][105692] Updated weights for policy 0, policy_version 1317973 (0.0009) [2023-12-27 00:59:16,959][105692] Updated weights for policy 0, policy_version 1317983 (0.0009) [2023-12-27 00:59:17,010][105692] Updated weights for policy 0, policy_version 1317993 (0.0008) [2023-12-27 00:59:17,041][105620] Updated weights for policy 1, policy_version 1319845 (0.0010) [2023-12-27 00:59:17,101][105620] Updated weights for policy 1, policy_version 1319855 (0.0010) [2023-12-27 00:59:17,152][105620] Updated weights for policy 1, policy_version 1319865 (0.0010) [2023-12-27 00:59:17,814][105692] Updated weights for policy 0, policy_version 1318003 (0.0009) [2023-12-27 00:59:17,831][105620] Updated weights for policy 1, policy_version 1319875 (0.0010) [2023-12-27 00:59:17,870][105692] Updated weights for policy 0, policy_version 1318013 (0.0009) [2023-12-27 00:59:17,885][105620] Updated weights for policy 1, policy_version 1319885 (0.0010) [2023-12-27 00:59:17,922][105692] Updated weights for policy 0, policy_version 1318023 (0.0006) [2023-12-27 00:59:17,933][105620] Updated weights for policy 1, policy_version 1319895 (0.0010) [2023-12-27 00:59:18,680][105620] Updated weights for policy 1, policy_version 1319905 (0.0010) [2023-12-27 00:59:18,726][105692] Updated weights for policy 0, policy_version 1318033 (0.0006) [2023-12-27 00:59:18,738][105620] Updated weights for policy 1, policy_version 1319915 (0.0010) [2023-12-27 00:59:18,786][105692] Updated weights for policy 0, policy_version 1318043 (0.0009) [2023-12-27 00:59:18,800][105620] Updated weights for policy 1, policy_version 1319925 (0.0010) [2023-12-27 00:59:18,846][105692] Updated weights for policy 0, policy_version 1318053 (0.0007) [2023-12-27 00:59:18,862][105620] Updated weights for policy 1, policy_version 1319935 (0.0010) [2023-12-27 00:59:18,908][105692] Updated weights for policy 0, policy_version 1318063 (0.0009) [2023-12-27 00:59:19,678][105692] Updated weights for policy 0, policy_version 1318073 (0.0008) [2023-12-27 00:59:19,688][105620] Updated weights for policy 1, policy_version 1319945 (0.0010) [2023-12-27 00:59:19,739][105692] Updated weights for policy 0, policy_version 1318083 (0.0006) [2023-12-27 00:59:19,751][105620] Updated weights for policy 1, policy_version 1319955 (0.0011) [2023-12-27 00:59:19,796][105692] Updated weights for policy 0, policy_version 1318093 (0.0007) [2023-12-27 00:59:19,813][105620] Updated weights for policy 1, policy_version 1319965 (0.0010) [2023-12-27 00:59:20,522][105620] Updated weights for policy 1, policy_version 1319975 (0.0008) [2023-12-27 00:59:20,598][105620] Updated weights for policy 1, policy_version 1319985 (0.0011) [2023-12-27 00:59:20,620][105692] Updated weights for policy 0, policy_version 1318103 (0.0009) [2023-12-27 00:59:20,663][105620] Updated weights for policy 1, policy_version 1319995 (0.0009) [2023-12-27 00:59:20,679][105692] Updated weights for policy 0, policy_version 1318113 (0.0009) [2023-12-27 00:59:20,733][105692] Updated weights for policy 0, policy_version 1318123 (0.0009) [2023-12-27 00:59:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 675454976. Throughput: 0: 9714.0, 1: 9659.6. Samples: 675445332. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 00:59:21,063][104569] Avg episode reward: [(0, '8963.247'), (1, '9001.557')] [2023-12-27 00:59:21,412][105620] Updated weights for policy 1, policy_version 1320005 (0.0009) [2023-12-27 00:59:21,472][105620] Updated weights for policy 1, policy_version 1320015 (0.0007) [2023-12-27 00:59:21,531][105620] Updated weights for policy 1, policy_version 1320025 (0.0006) [2023-12-27 00:59:21,574][105692] Updated weights for policy 0, policy_version 1318133 (0.0008) [2023-12-27 00:59:21,630][105692] Updated weights for policy 0, policy_version 1318143 (0.0008) [2023-12-27 00:59:21,700][105692] Updated weights for policy 0, policy_version 1318153 (0.0008) [2023-12-27 00:59:22,194][105620] Updated weights for policy 1, policy_version 1320035 (0.0008) [2023-12-27 00:59:22,252][105620] Updated weights for policy 1, policy_version 1320045 (0.0009) [2023-12-27 00:59:22,317][105620] Updated weights for policy 1, policy_version 1320055 (0.0008) [2023-12-27 00:59:22,466][105692] Updated weights for policy 0, policy_version 1318163 (0.0010) [2023-12-27 00:59:22,532][105692] Updated weights for policy 0, policy_version 1318173 (0.0009) [2023-12-27 00:59:22,595][105692] Updated weights for policy 0, policy_version 1318183 (0.0008) [2023-12-27 00:59:23,108][105620] Updated weights for policy 1, policy_version 1320065 (0.0009) [2023-12-27 00:59:23,170][105620] Updated weights for policy 1, policy_version 1320075 (0.0006) [2023-12-27 00:59:23,241][105620] Updated weights for policy 1, policy_version 1320085 (0.0005) [2023-12-27 00:59:23,281][105692] Updated weights for policy 0, policy_version 1318193 (0.0009) [2023-12-27 00:59:23,311][105620] Updated weights for policy 1, policy_version 1320095 (0.0005) [2023-12-27 00:59:23,340][105692] Updated weights for policy 0, policy_version 1318203 (0.0010) [2023-12-27 00:59:23,399][105692] Updated weights for policy 0, policy_version 1318213 (0.0010) [2023-12-27 00:59:23,467][105692] Updated weights for policy 0, policy_version 1318223 (0.0010) [2023-12-27 00:59:23,824][105620] Updated weights for policy 1, policy_version 1320105 (0.0006) [2023-12-27 00:59:23,870][105620] Updated weights for policy 1, policy_version 1320115 (0.0005) [2023-12-27 00:59:23,918][105620] Updated weights for policy 1, policy_version 1320125 (0.0008) [2023-12-27 00:59:24,192][105692] Updated weights for policy 0, policy_version 1318233 (0.0006) [2023-12-27 00:59:24,248][105692] Updated weights for policy 0, policy_version 1318243 (0.0005) [2023-12-27 00:59:24,307][105692] Updated weights for policy 0, policy_version 1318253 (0.0009) [2023-12-27 00:59:24,619][105620] Updated weights for policy 1, policy_version 1320135 (0.0011) [2023-12-27 00:59:24,678][105620] Updated weights for policy 1, policy_version 1320145 (0.0011) [2023-12-27 00:59:24,734][105620] Updated weights for policy 1, policy_version 1320155 (0.0011) [2023-12-27 00:59:24,996][105692] Updated weights for policy 0, policy_version 1318263 (0.0009) [2023-12-27 00:59:25,052][105692] Updated weights for policy 0, policy_version 1318273 (0.0009) [2023-12-27 00:59:25,103][105692] Updated weights for policy 0, policy_version 1318283 (0.0008) [2023-12-27 00:59:25,494][105620] Updated weights for policy 1, policy_version 1320165 (0.0010) [2023-12-27 00:59:25,551][105620] Updated weights for policy 1, policy_version 1320175 (0.0007) [2023-12-27 00:59:25,610][105620] Updated weights for policy 1, policy_version 1320185 (0.0007) [2023-12-27 00:59:25,803][105692] Updated weights for policy 0, policy_version 1318293 (0.0005) [2023-12-27 00:59:25,867][105692] Updated weights for policy 0, policy_version 1318303 (0.0005) [2023-12-27 00:59:25,930][105692] Updated weights for policy 0, policy_version 1318313 (0.0006) [2023-12-27 00:59:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 675553280. Throughput: 0: 9605.6, 1: 9730.9. Samples: 675560696. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 00:59:26,062][104569] Avg episode reward: [(0, '8820.711'), (1, '9086.664')] [2023-12-27 00:59:26,273][105620] Updated weights for policy 1, policy_version 1320195 (0.0005) [2023-12-27 00:59:26,326][105620] Updated weights for policy 1, policy_version 1320205 (0.0005) [2023-12-27 00:59:26,388][105620] Updated weights for policy 1, policy_version 1320215 (0.0007) [2023-12-27 00:59:26,520][105692] Updated weights for policy 0, policy_version 1318323 (0.0006) [2023-12-27 00:59:26,568][105692] Updated weights for policy 0, policy_version 1318333 (0.0005) [2023-12-27 00:59:26,615][105692] Updated weights for policy 0, policy_version 1318343 (0.0005) [2023-12-27 00:59:26,977][105620] Updated weights for policy 1, policy_version 1320225 (0.0009) [2023-12-27 00:59:27,038][105620] Updated weights for policy 1, policy_version 1320235 (0.0005) [2023-12-27 00:59:27,094][105620] Updated weights for policy 1, policy_version 1320245 (0.0005) [2023-12-27 00:59:27,146][105620] Updated weights for policy 1, policy_version 1320255 (0.0005) [2023-12-27 00:59:27,296][105692] Updated weights for policy 0, policy_version 1318353 (0.0006) [2023-12-27 00:59:27,357][105692] Updated weights for policy 0, policy_version 1318363 (0.0011) [2023-12-27 00:59:27,411][105692] Updated weights for policy 0, policy_version 1318373 (0.0010) [2023-12-27 00:59:27,465][105692] Updated weights for policy 0, policy_version 1318383 (0.0010) [2023-12-27 00:59:27,755][105620] Updated weights for policy 1, policy_version 1320265 (0.0008) [2023-12-27 00:59:27,799][105620] Updated weights for policy 1, policy_version 1320275 (0.0006) [2023-12-27 00:59:27,850][105620] Updated weights for policy 1, policy_version 1320285 (0.0005) [2023-12-27 00:59:28,214][105692] Updated weights for policy 0, policy_version 1318393 (0.0011) [2023-12-27 00:59:28,272][105692] Updated weights for policy 0, policy_version 1318403 (0.0010) [2023-12-27 00:59:28,330][105692] Updated weights for policy 0, policy_version 1318413 (0.0010) [2023-12-27 00:59:28,511][105620] Updated weights for policy 1, policy_version 1320295 (0.0006) [2023-12-27 00:59:28,561][105620] Updated weights for policy 1, policy_version 1320305 (0.0005) [2023-12-27 00:59:28,610][105620] Updated weights for policy 1, policy_version 1320315 (0.0008) [2023-12-27 00:59:29,066][105692] Updated weights for policy 0, policy_version 1318423 (0.0011) [2023-12-27 00:59:29,121][105692] Updated weights for policy 0, policy_version 1318433 (0.0011) [2023-12-27 00:59:29,172][105692] Updated weights for policy 0, policy_version 1318443 (0.0010) [2023-12-27 00:59:29,337][105620] Updated weights for policy 1, policy_version 1320325 (0.0009) [2023-12-27 00:59:29,401][105620] Updated weights for policy 1, policy_version 1320335 (0.0008) [2023-12-27 00:59:29,462][105620] Updated weights for policy 1, policy_version 1320345 (0.0008) [2023-12-27 00:59:29,863][105692] Updated weights for policy 0, policy_version 1318453 (0.0013) [2023-12-27 00:59:29,920][105692] Updated weights for policy 0, policy_version 1318463 (0.0006) [2023-12-27 00:59:29,975][105692] Updated weights for policy 0, policy_version 1318473 (0.0007) [2023-12-27 00:59:30,206][105620] Updated weights for policy 1, policy_version 1320355 (0.0008) [2023-12-27 00:59:30,267][105620] Updated weights for policy 1, policy_version 1320365 (0.0010) [2023-12-27 00:59:30,319][105620] Updated weights for policy 1, policy_version 1320375 (0.0008) [2023-12-27 00:59:30,616][105692] Updated weights for policy 0, policy_version 1318483 (0.0007) [2023-12-27 00:59:30,664][105692] Updated weights for policy 0, policy_version 1318493 (0.0011) [2023-12-27 00:59:30,713][105692] Updated weights for policy 0, policy_version 1318503 (0.0010) [2023-12-27 00:59:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 675651584. Throughput: 0: 9676.4, 1: 9804.7. Samples: 675623220. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 00:59:31,063][104569] Avg episode reward: [(0, '8816.363'), (1, '9001.226')] [2023-12-27 00:59:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001318512_337592320.pth... [2023-12-27 00:59:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001320384_338059264.pth... [2023-12-27 00:59:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001317392_337305600.pth [2023-12-27 00:59:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001319232_337764352.pth [2023-12-27 00:59:31,151][105620] Updated weights for policy 1, policy_version 1320385 (0.0008) [2023-12-27 00:59:31,200][105620] Updated weights for policy 1, policy_version 1320395 (0.0008) [2023-12-27 00:59:31,263][105620] Updated weights for policy 1, policy_version 1320405 (0.0008) [2023-12-27 00:59:31,319][105620] Updated weights for policy 1, policy_version 1320415 (0.0008) [2023-12-27 00:59:31,399][105692] Updated weights for policy 0, policy_version 1318513 (0.0010) [2023-12-27 00:59:31,452][105692] Updated weights for policy 0, policy_version 1318523 (0.0007) [2023-12-27 00:59:31,511][105692] Updated weights for policy 0, policy_version 1318533 (0.0009) [2023-12-27 00:59:31,570][105692] Updated weights for policy 0, policy_version 1318543 (0.0009) [2023-12-27 00:59:32,035][105620] Updated weights for policy 1, policy_version 1320425 (0.0010) [2023-12-27 00:59:32,093][105620] Updated weights for policy 1, policy_version 1320435 (0.0010) [2023-12-27 00:59:32,150][105620] Updated weights for policy 1, policy_version 1320445 (0.0010) [2023-12-27 00:59:32,311][105692] Updated weights for policy 0, policy_version 1318553 (0.0010) [2023-12-27 00:59:32,382][105692] Updated weights for policy 0, policy_version 1318563 (0.0011) [2023-12-27 00:59:32,447][105692] Updated weights for policy 0, policy_version 1318573 (0.0011) [2023-12-27 00:59:32,794][105620] Updated weights for policy 1, policy_version 1320455 (0.0007) [2023-12-27 00:59:32,850][105620] Updated weights for policy 1, policy_version 1320465 (0.0006) [2023-12-27 00:59:32,903][105620] Updated weights for policy 1, policy_version 1320475 (0.0005) [2023-12-27 00:59:33,115][105692] Updated weights for policy 0, policy_version 1318583 (0.0007) [2023-12-27 00:59:33,165][105692] Updated weights for policy 0, policy_version 1318593 (0.0006) [2023-12-27 00:59:33,218][105692] Updated weights for policy 0, policy_version 1318603 (0.0006) [2023-12-27 00:59:33,472][105620] Updated weights for policy 1, policy_version 1320485 (0.0008) [2023-12-27 00:59:33,533][105620] Updated weights for policy 1, policy_version 1320495 (0.0005) [2023-12-27 00:59:33,569][105586] KL-divergence is very high: 122.6504 [2023-12-27 00:59:33,609][105586] KL-divergence is very high: 101.3381 [2023-12-27 00:59:33,611][105620] Updated weights for policy 1, policy_version 1320505 (0.0007) [2023-12-27 00:59:33,620][105586] KL-divergence is very high: 135.2857 [2023-12-27 00:59:33,864][105692] Updated weights for policy 0, policy_version 1318613 (0.0008) [2023-12-27 00:59:33,926][105692] Updated weights for policy 0, policy_version 1318623 (0.0010) [2023-12-27 00:59:33,995][105692] Updated weights for policy 0, policy_version 1318633 (0.0011) [2023-12-27 00:59:34,311][105620] Updated weights for policy 1, policy_version 1320515 (0.0010) [2023-12-27 00:59:34,377][105620] Updated weights for policy 1, policy_version 1320525 (0.0011) [2023-12-27 00:59:34,447][105620] Updated weights for policy 1, policy_version 1320535 (0.0011) [2023-12-27 00:59:34,688][105692] Updated weights for policy 0, policy_version 1318643 (0.0009) [2023-12-27 00:59:34,741][105692] Updated weights for policy 0, policy_version 1318653 (0.0011) [2023-12-27 00:59:34,800][105692] Updated weights for policy 0, policy_version 1318663 (0.0010) [2023-12-27 00:59:35,194][105620] Updated weights for policy 1, policy_version 1320545 (0.0011) [2023-12-27 00:59:35,252][105620] Updated weights for policy 1, policy_version 1320555 (0.0010) [2023-12-27 00:59:35,309][105620] Updated weights for policy 1, policy_version 1320565 (0.0010) [2023-12-27 00:59:35,367][105620] Updated weights for policy 1, policy_version 1320575 (0.0007) [2023-12-27 00:59:35,470][105692] Updated weights for policy 0, policy_version 1318673 (0.0010) [2023-12-27 00:59:35,529][105692] Updated weights for policy 0, policy_version 1318683 (0.0006) [2023-12-27 00:59:35,592][105692] Updated weights for policy 0, policy_version 1318693 (0.0007) [2023-12-27 00:59:35,654][105692] Updated weights for policy 0, policy_version 1318703 (0.0011) [2023-12-27 00:59:36,024][105620] Updated weights for policy 1, policy_version 1320585 (0.0006) [2023-12-27 00:59:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 675749888. Throughput: 0: 9773.6, 1: 9857.2. Samples: 675742800. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 00:59:36,062][104569] Avg episode reward: [(0, '8812.003'), (1, '8744.298')] [2023-12-27 00:59:36,071][105620] Updated weights for policy 1, policy_version 1320595 (0.0005) [2023-12-27 00:59:36,131][105620] Updated weights for policy 1, policy_version 1320605 (0.0008) [2023-12-27 00:59:36,315][105692] Updated weights for policy 0, policy_version 1318713 (0.0006) [2023-12-27 00:59:36,376][105692] Updated weights for policy 0, policy_version 1318723 (0.0006) [2023-12-27 00:59:36,435][105692] Updated weights for policy 0, policy_version 1318733 (0.0006) [2023-12-27 00:59:36,885][105620] Updated weights for policy 1, policy_version 1320615 (0.0010) [2023-12-27 00:59:36,950][105620] Updated weights for policy 1, policy_version 1320625 (0.0010) [2023-12-27 00:59:37,019][105620] Updated weights for policy 1, policy_version 1320635 (0.0010) [2023-12-27 00:59:37,068][105692] Updated weights for policy 0, policy_version 1318743 (0.0009) [2023-12-27 00:59:37,120][105692] Updated weights for policy 0, policy_version 1318753 (0.0010) [2023-12-27 00:59:37,176][105692] Updated weights for policy 0, policy_version 1318763 (0.0010) [2023-12-27 00:59:37,720][105620] Updated weights for policy 1, policy_version 1320645 (0.0008) [2023-12-27 00:59:37,795][105620] Updated weights for policy 1, policy_version 1320655 (0.0006) [2023-12-27 00:59:37,866][105620] Updated weights for policy 1, policy_version 1320665 (0.0006) [2023-12-27 00:59:37,905][105692] Updated weights for policy 0, policy_version 1318773 (0.0009) [2023-12-27 00:59:37,958][105692] Updated weights for policy 0, policy_version 1318783 (0.0010) [2023-12-27 00:59:38,014][105692] Updated weights for policy 0, policy_version 1318793 (0.0011) [2023-12-27 00:59:38,461][105620] Updated weights for policy 1, policy_version 1320675 (0.0006) [2023-12-27 00:59:38,514][105620] Updated weights for policy 1, policy_version 1320685 (0.0009) [2023-12-27 00:59:38,568][105620] Updated weights for policy 1, policy_version 1320696 (0.0010) [2023-12-27 00:59:38,681][105692] Updated weights for policy 0, policy_version 1318803 (0.0009) [2023-12-27 00:59:38,738][105692] Updated weights for policy 0, policy_version 1318813 (0.0007) [2023-12-27 00:59:38,803][105692] Updated weights for policy 0, policy_version 1318823 (0.0010) [2023-12-27 00:59:39,401][105620] Updated weights for policy 1, policy_version 1320706 (0.0010) [2023-12-27 00:59:39,467][105620] Updated weights for policy 1, policy_version 1320716 (0.0009) [2023-12-27 00:59:39,530][105620] Updated weights for policy 1, policy_version 1320726 (0.0006) [2023-12-27 00:59:39,533][105692] Updated weights for policy 0, policy_version 1318833 (0.0011) [2023-12-27 00:59:39,591][105620] Updated weights for policy 1, policy_version 1320736 (0.0008) [2023-12-27 00:59:39,597][105692] Updated weights for policy 0, policy_version 1318843 (0.0010) [2023-12-27 00:59:39,658][105692] Updated weights for policy 0, policy_version 1318853 (0.0007) [2023-12-27 00:59:39,718][105692] Updated weights for policy 0, policy_version 1318863 (0.0010) [2023-12-27 00:59:40,309][105620] Updated weights for policy 1, policy_version 1320746 (0.0008) [2023-12-27 00:59:40,370][105620] Updated weights for policy 1, policy_version 1320756 (0.0008) [2023-12-27 00:59:40,436][105620] Updated weights for policy 1, policy_version 1320766 (0.0007) [2023-12-27 00:59:40,460][105692] Updated weights for policy 0, policy_version 1318873 (0.0010) [2023-12-27 00:59:40,519][105692] Updated weights for policy 0, policy_version 1318883 (0.0010) [2023-12-27 00:59:40,577][105692] Updated weights for policy 0, policy_version 1318893 (0.0009) [2023-12-27 00:59:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19438.7). Total num frames: 675848192. Throughput: 0: 9779.5, 1: 9885.8. Samples: 675860500. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 00:59:41,062][104569] Avg episode reward: [(0, '8812.959'), (1, '8922.537')] [2023-12-27 00:59:41,170][105620] Updated weights for policy 1, policy_version 1320776 (0.0008) [2023-12-27 00:59:41,235][105620] Updated weights for policy 1, policy_version 1320786 (0.0006) [2023-12-27 00:59:41,284][105692] Updated weights for policy 0, policy_version 1318903 (0.0008) [2023-12-27 00:59:41,299][105620] Updated weights for policy 1, policy_version 1320796 (0.0008) [2023-12-27 00:59:41,351][105692] Updated weights for policy 0, policy_version 1318913 (0.0008) [2023-12-27 00:59:41,423][105692] Updated weights for policy 0, policy_version 1318923 (0.0008) [2023-12-27 00:59:42,051][105620] Updated weights for policy 1, policy_version 1320806 (0.0009) [2023-12-27 00:59:42,109][105620] Updated weights for policy 1, policy_version 1320816 (0.0011) [2023-12-27 00:59:42,151][105692] Updated weights for policy 0, policy_version 1318933 (0.0007) [2023-12-27 00:59:42,172][105620] Updated weights for policy 1, policy_version 1320826 (0.0011) [2023-12-27 00:59:42,206][105692] Updated weights for policy 0, policy_version 1318943 (0.0006) [2023-12-27 00:59:42,266][105692] Updated weights for policy 0, policy_version 1318953 (0.0008) [2023-12-27 00:59:42,933][105620] Updated weights for policy 1, policy_version 1320836 (0.0009) [2023-12-27 00:59:42,955][105692] Updated weights for policy 0, policy_version 1318963 (0.0008) [2023-12-27 00:59:43,005][105620] Updated weights for policy 1, policy_version 1320846 (0.0006) [2023-12-27 00:59:43,021][105692] Updated weights for policy 0, policy_version 1318973 (0.0005) [2023-12-27 00:59:43,068][105620] Updated weights for policy 1, policy_version 1320856 (0.0006) [2023-12-27 00:59:43,085][105692] Updated weights for policy 0, policy_version 1318983 (0.0006) [2023-12-27 00:59:43,577][105620] Updated weights for policy 1, policy_version 1320866 (0.0006) [2023-12-27 00:59:43,639][105620] Updated weights for policy 1, policy_version 1320876 (0.0007) [2023-12-27 00:59:43,667][105692] Updated weights for policy 0, policy_version 1318993 (0.0006) [2023-12-27 00:59:43,699][105620] Updated weights for policy 1, policy_version 1320886 (0.0009) [2023-12-27 00:59:43,712][105692] Updated weights for policy 0, policy_version 1319003 (0.0006) [2023-12-27 00:59:43,759][105692] Updated weights for policy 0, policy_version 1319013 (0.0005) [2023-12-27 00:59:43,761][105620] Updated weights for policy 1, policy_version 1320896 (0.0008) [2023-12-27 00:59:43,810][105692] Updated weights for policy 0, policy_version 1319023 (0.0005) [2023-12-27 00:59:44,375][105692] Updated weights for policy 0, policy_version 1319033 (0.0008) [2023-12-27 00:59:44,437][105692] Updated weights for policy 0, policy_version 1319043 (0.0008) [2023-12-27 00:59:44,479][105620] Updated weights for policy 1, policy_version 1320906 (0.0010) [2023-12-27 00:59:44,494][105692] Updated weights for policy 0, policy_version 1319053 (0.0007) [2023-12-27 00:59:44,527][105620] Updated weights for policy 1, policy_version 1320916 (0.0010) [2023-12-27 00:59:44,580][105620] Updated weights for policy 1, policy_version 1320926 (0.0010) [2023-12-27 00:59:45,285][105692] Updated weights for policy 0, policy_version 1319063 (0.0007) [2023-12-27 00:59:45,340][105620] Updated weights for policy 1, policy_version 1320936 (0.0011) [2023-12-27 00:59:45,347][105692] Updated weights for policy 0, policy_version 1319073 (0.0006) [2023-12-27 00:59:45,399][105620] Updated weights for policy 1, policy_version 1320946 (0.0010) [2023-12-27 00:59:45,408][105692] Updated weights for policy 0, policy_version 1319083 (0.0008) [2023-12-27 00:59:45,459][105620] Updated weights for policy 1, policy_version 1320956 (0.0010) [2023-12-27 00:59:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 675946496. Throughput: 0: 9746.8, 1: 9880.6. Samples: 675920760. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 00:59:46,062][104569] Avg episode reward: [(0, '8724.174'), (1, '9090.122')] [2023-12-27 00:59:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001319088_337739776.pth... [2023-12-27 00:59:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001320960_338206720.pth... [2023-12-27 00:59:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001319808_337911808.pth [2023-12-27 00:59:46,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001317968_337453056.pth [2023-12-27 00:59:46,183][105620] Updated weights for policy 1, policy_version 1320966 (0.0009) [2023-12-27 00:59:46,201][105692] Updated weights for policy 0, policy_version 1319093 (0.0008) [2023-12-27 00:59:46,239][105620] Updated weights for policy 1, policy_version 1320976 (0.0007) [2023-12-27 00:59:46,253][105692] Updated weights for policy 0, policy_version 1319103 (0.0009) [2023-12-27 00:59:46,291][105620] Updated weights for policy 1, policy_version 1320986 (0.0005) [2023-12-27 00:59:46,305][105692] Updated weights for policy 0, policy_version 1319113 (0.0009) [2023-12-27 00:59:46,866][105620] Updated weights for policy 1, policy_version 1320996 (0.0005) [2023-12-27 00:59:46,919][105620] Updated weights for policy 1, policy_version 1321006 (0.0005) [2023-12-27 00:59:46,973][105620] Updated weights for policy 1, policy_version 1321016 (0.0009) [2023-12-27 00:59:47,186][105692] Updated weights for policy 0, policy_version 1319123 (0.0010) [2023-12-27 00:59:47,241][105692] Updated weights for policy 0, policy_version 1319134 (0.0010) [2023-12-27 00:59:47,287][105692] Updated weights for policy 0, policy_version 1319144 (0.0008) [2023-12-27 00:59:47,643][105620] Updated weights for policy 1, policy_version 1321026 (0.0010) [2023-12-27 00:59:47,707][105620] Updated weights for policy 1, policy_version 1321036 (0.0010) [2023-12-27 00:59:47,762][105620] Updated weights for policy 1, policy_version 1321046 (0.0010) [2023-12-27 00:59:47,812][105620] Updated weights for policy 1, policy_version 1321056 (0.0010) [2023-12-27 00:59:48,112][105692] Updated weights for policy 0, policy_version 1319154 (0.0008) [2023-12-27 00:59:48,173][105692] Updated weights for policy 0, policy_version 1319164 (0.0008) [2023-12-27 00:59:48,225][105692] Updated weights for policy 0, policy_version 1319174 (0.0008) [2023-12-27 00:59:48,274][105692] Updated weights for policy 0, policy_version 1319184 (0.0008) [2023-12-27 00:59:48,554][105620] Updated weights for policy 1, policy_version 1321066 (0.0007) [2023-12-27 00:59:48,611][105620] Updated weights for policy 1, policy_version 1321076 (0.0010) [2023-12-27 00:59:48,673][105620] Updated weights for policy 1, policy_version 1321086 (0.0010) [2023-12-27 00:59:49,108][105692] Updated weights for policy 0, policy_version 1319194 (0.0008) [2023-12-27 00:59:49,156][105692] Updated weights for policy 0, policy_version 1319204 (0.0008) [2023-12-27 00:59:49,201][105692] Updated weights for policy 0, policy_version 1319214 (0.0008) [2023-12-27 00:59:49,380][105620] Updated weights for policy 1, policy_version 1321096 (0.0010) [2023-12-27 00:59:49,426][105620] Updated weights for policy 1, policy_version 1321106 (0.0010) [2023-12-27 00:59:49,474][105620] Updated weights for policy 1, policy_version 1321116 (0.0010) [2023-12-27 00:59:49,954][105692] Updated weights for policy 0, policy_version 1319224 (0.0008) [2023-12-27 00:59:50,014][105692] Updated weights for policy 0, policy_version 1319234 (0.0008) [2023-12-27 00:59:50,080][105692] Updated weights for policy 0, policy_version 1319244 (0.0009) [2023-12-27 00:59:50,264][105620] Updated weights for policy 1, policy_version 1321126 (0.0007) [2023-12-27 00:59:50,323][105620] Updated weights for policy 1, policy_version 1321136 (0.0008) [2023-12-27 00:59:50,385][105620] Updated weights for policy 1, policy_version 1321146 (0.0007) [2023-12-27 00:59:50,910][105692] Updated weights for policy 0, policy_version 1319254 (0.0009) [2023-12-27 00:59:50,971][105692] Updated weights for policy 0, policy_version 1319264 (0.0008) [2023-12-27 00:59:51,026][105692] Updated weights for policy 0, policy_version 1319274 (0.0008) [2023-12-27 00:59:51,050][105620] Updated weights for policy 1, policy_version 1321156 (0.0007) [2023-12-27 00:59:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 676036608. Throughput: 0: 9741.6, 1: 9903.8. Samples: 676034844. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 00:59:51,063][104569] Avg episode reward: [(0, '8995.613'), (1, '8908.146')] [2023-12-27 00:59:51,115][105620] Updated weights for policy 1, policy_version 1321166 (0.0009) [2023-12-27 00:59:51,181][105620] Updated weights for policy 1, policy_version 1321176 (0.0010) [2023-12-27 00:59:51,866][105692] Updated weights for policy 0, policy_version 1319284 (0.0008) [2023-12-27 00:59:51,927][105692] Updated weights for policy 0, policy_version 1319294 (0.0009) [2023-12-27 00:59:51,956][105620] Updated weights for policy 1, policy_version 1321186 (0.0010) [2023-12-27 00:59:51,987][105692] Updated weights for policy 0, policy_version 1319304 (0.0007) [2023-12-27 00:59:52,019][105620] Updated weights for policy 1, policy_version 1321196 (0.0008) [2023-12-27 00:59:52,088][105620] Updated weights for policy 1, policy_version 1321206 (0.0008) [2023-12-27 00:59:52,152][105620] Updated weights for policy 1, policy_version 1321216 (0.0007) [2023-12-27 00:59:52,798][105620] Updated weights for policy 1, policy_version 1321226 (0.0009) [2023-12-27 00:59:52,818][105692] Updated weights for policy 0, policy_version 1319314 (0.0008) [2023-12-27 00:59:52,856][105620] Updated weights for policy 1, policy_version 1321236 (0.0007) [2023-12-27 00:59:52,875][105692] Updated weights for policy 0, policy_version 1319324 (0.0006) [2023-12-27 00:59:52,909][105620] Updated weights for policy 1, policy_version 1321246 (0.0007) [2023-12-27 00:59:52,934][105692] Updated weights for policy 0, policy_version 1319334 (0.0007) [2023-12-27 00:59:53,497][105620] Updated weights for policy 1, policy_version 1321256 (0.0005) [2023-12-27 00:59:53,562][105620] Updated weights for policy 1, policy_version 1321266 (0.0005) [2023-12-27 00:59:53,628][105620] Updated weights for policy 1, policy_version 1321276 (0.0005) [2023-12-27 00:59:53,836][105692] Updated weights for policy 0, policy_version 1319345 (0.0010) [2023-12-27 00:59:53,891][105692] Updated weights for policy 0, policy_version 1319355 (0.0010) [2023-12-27 00:59:53,946][105692] Updated weights for policy 0, policy_version 1319365 (0.0008) [2023-12-27 00:59:53,996][105692] Updated weights for policy 0, policy_version 1319375 (0.0009) [2023-12-27 00:59:54,139][105620] Updated weights for policy 1, policy_version 1321286 (0.0006) [2023-12-27 00:59:54,193][105620] Updated weights for policy 1, policy_version 1321296 (0.0005) [2023-12-27 00:59:54,247][105620] Updated weights for policy 1, policy_version 1321306 (0.0005) [2023-12-27 00:59:54,718][105692] Updated weights for policy 0, policy_version 1319385 (0.0009) [2023-12-27 00:59:54,779][105692] Updated weights for policy 0, policy_version 1319395 (0.0009) [2023-12-27 00:59:54,834][105620] Updated weights for policy 1, policy_version 1321316 (0.0006) [2023-12-27 00:59:54,836][105692] Updated weights for policy 0, policy_version 1319405 (0.0008) [2023-12-27 00:59:54,884][105620] Updated weights for policy 1, policy_version 1321327 (0.0008) [2023-12-27 00:59:54,942][105620] Updated weights for policy 1, policy_version 1321337 (0.0010) [2023-12-27 00:59:55,487][105692] Updated weights for policy 0, policy_version 1319415 (0.0006) [2023-12-27 00:59:55,546][105692] Updated weights for policy 0, policy_version 1319425 (0.0007) [2023-12-27 00:59:55,610][105692] Updated weights for policy 0, policy_version 1319435 (0.0005) [2023-12-27 00:59:55,759][105620] Updated weights for policy 1, policy_version 1321347 (0.0010) [2023-12-27 00:59:55,828][105620] Updated weights for policy 1, policy_version 1321357 (0.0010) [2023-12-27 00:59:55,899][105620] Updated weights for policy 1, policy_version 1321367 (0.0010) [2023-12-27 00:59:56,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 676143104. Throughput: 0: 9614.5, 1: 9960.6. Samples: 676150456. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 00:59:56,063][104569] Avg episode reward: [(0, '8995.456'), (1, '8908.568')] [2023-12-27 00:59:56,127][105692] Updated weights for policy 0, policy_version 1319445 (0.0007) [2023-12-27 00:59:56,175][105692] Updated weights for policy 0, policy_version 1319455 (0.0008) [2023-12-27 00:59:56,225][105692] Updated weights for policy 0, policy_version 1319465 (0.0009) [2023-12-27 00:59:56,548][105620] Updated weights for policy 1, policy_version 1321377 (0.0006) [2023-12-27 00:59:56,604][105620] Updated weights for policy 1, policy_version 1321387 (0.0010) [2023-12-27 00:59:56,663][105620] Updated weights for policy 1, policy_version 1321397 (0.0009) [2023-12-27 00:59:56,724][105620] Updated weights for policy 1, policy_version 1321407 (0.0006) [2023-12-27 00:59:56,833][105692] Updated weights for policy 0, policy_version 1319475 (0.0008) [2023-12-27 00:59:56,881][105692] Updated weights for policy 0, policy_version 1319485 (0.0006) [2023-12-27 00:59:56,937][105692] Updated weights for policy 0, policy_version 1319497 (0.0010) [2023-12-27 00:59:57,281][105620] Updated weights for policy 1, policy_version 1321417 (0.0005) [2023-12-27 00:59:57,346][105620] Updated weights for policy 1, policy_version 1321427 (0.0007) [2023-12-27 00:59:57,402][105620] Updated weights for policy 1, policy_version 1321437 (0.0007) [2023-12-27 00:59:57,606][105692] Updated weights for policy 0, policy_version 1319507 (0.0009) [2023-12-27 00:59:57,665][105692] Updated weights for policy 0, policy_version 1319517 (0.0005) [2023-12-27 00:59:57,717][105692] Updated weights for policy 0, policy_version 1319527 (0.0005) [2023-12-27 00:59:57,988][105620] Updated weights for policy 1, policy_version 1321447 (0.0006) [2023-12-27 00:59:58,044][105620] Updated weights for policy 1, policy_version 1321457 (0.0006) [2023-12-27 00:59:58,101][105620] Updated weights for policy 1, policy_version 1321467 (0.0006) [2023-12-27 00:59:58,339][105692] Updated weights for policy 0, policy_version 1319537 (0.0005) [2023-12-27 00:59:58,404][105692] Updated weights for policy 0, policy_version 1319547 (0.0009) [2023-12-27 00:59:58,467][105692] Updated weights for policy 0, policy_version 1319557 (0.0010) [2023-12-27 00:59:58,535][105692] Updated weights for policy 0, policy_version 1319567 (0.0008) [2023-12-27 00:59:58,802][105620] Updated weights for policy 1, policy_version 1321477 (0.0006) [2023-12-27 00:59:58,867][105620] Updated weights for policy 1, policy_version 1321487 (0.0008) [2023-12-27 00:59:58,922][105620] Updated weights for policy 1, policy_version 1321497 (0.0008) [2023-12-27 00:59:59,250][105692] Updated weights for policy 0, policy_version 1319577 (0.0008) [2023-12-27 00:59:59,315][105692] Updated weights for policy 0, policy_version 1319587 (0.0008) [2023-12-27 00:59:59,377][105692] Updated weights for policy 0, policy_version 1319597 (0.0008) [2023-12-27 00:59:59,771][105620] Updated weights for policy 1, policy_version 1321507 (0.0009) [2023-12-27 00:59:59,843][105620] Updated weights for policy 1, policy_version 1321517 (0.0010) [2023-12-27 00:59:59,901][105620] Updated weights for policy 1, policy_version 1321527 (0.0010) [2023-12-27 00:59:59,969][105692] Updated weights for policy 0, policy_version 1319607 (0.0009) [2023-12-27 01:00:00,034][105692] Updated weights for policy 0, policy_version 1319617 (0.0011) [2023-12-27 01:00:00,101][105692] Updated weights for policy 0, policy_version 1319627 (0.0011) [2023-12-27 01:00:00,535][105620] Updated weights for policy 1, policy_version 1321537 (0.0008) [2023-12-27 01:00:00,581][105620] Updated weights for policy 1, policy_version 1321547 (0.0006) [2023-12-27 01:00:00,624][105620] Updated weights for policy 1, policy_version 1321557 (0.0005) [2023-12-27 01:00:00,668][105620] Updated weights for policy 1, policy_version 1321567 (0.0005) [2023-12-27 01:00:00,772][105692] Updated weights for policy 0, policy_version 1319638 (0.0011) [2023-12-27 01:00:00,827][105692] Updated weights for policy 0, policy_version 1319648 (0.0010) [2023-12-27 01:00:00,885][105692] Updated weights for policy 0, policy_version 1319658 (0.0010) [2023-12-27 01:00:01,062][104569] Fps is (10 sec: 21299.0, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 676249600. Throughput: 0: 9645.4, 1: 10007.6. Samples: 676216468. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 01:00:01,063][104569] Avg episode reward: [(0, '8737.291'), (1, '9179.232')] [2023-12-27 01:00:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001319664_337887232.pth... [2023-12-27 01:00:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001321568_338362368.pth... [2023-12-27 01:00:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001318512_337592320.pth [2023-12-27 01:00:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001320384_338059264.pth [2023-12-27 01:00:01,357][105620] Updated weights for policy 1, policy_version 1321577 (0.0006) [2023-12-27 01:00:01,424][105620] Updated weights for policy 1, policy_version 1321587 (0.0007) [2023-12-27 01:00:01,489][105620] Updated weights for policy 1, policy_version 1321597 (0.0005) [2023-12-27 01:00:01,584][105692] Updated weights for policy 0, policy_version 1319668 (0.0008) [2023-12-27 01:00:01,653][105692] Updated weights for policy 0, policy_version 1319678 (0.0009) [2023-12-27 01:00:01,684][105585] KL-divergence is very high: 104.7715 [2023-12-27 01:00:01,718][105692] Updated weights for policy 0, policy_version 1319688 (0.0009) [2023-12-27 01:00:02,151][105620] Updated weights for policy 1, policy_version 1321607 (0.0006) [2023-12-27 01:00:02,216][105620] Updated weights for policy 1, policy_version 1321617 (0.0005) [2023-12-27 01:00:02,275][105620] Updated weights for policy 1, policy_version 1321627 (0.0007) [2023-12-27 01:00:02,446][105692] Updated weights for policy 0, policy_version 1319698 (0.0009) [2023-12-27 01:00:02,501][105692] Updated weights for policy 0, policy_version 1319708 (0.0006) [2023-12-27 01:00:02,557][105692] Updated weights for policy 0, policy_version 1319718 (0.0005) [2023-12-27 01:00:02,617][105692] Updated weights for policy 0, policy_version 1319728 (0.0007) [2023-12-27 01:00:02,880][105620] Updated weights for policy 1, policy_version 1321637 (0.0006) [2023-12-27 01:00:02,933][105620] Updated weights for policy 1, policy_version 1321647 (0.0005) [2023-12-27 01:00:02,985][105620] Updated weights for policy 1, policy_version 1321657 (0.0006) [2023-12-27 01:00:03,343][105692] Updated weights for policy 0, policy_version 1319738 (0.0005) [2023-12-27 01:00:03,402][105692] Updated weights for policy 0, policy_version 1319748 (0.0005) [2023-12-27 01:00:03,462][105692] Updated weights for policy 0, policy_version 1319758 (0.0006) [2023-12-27 01:00:03,746][105620] Updated weights for policy 1, policy_version 1321667 (0.0008) [2023-12-27 01:00:03,802][105620] Updated weights for policy 1, policy_version 1321677 (0.0009) [2023-12-27 01:00:03,868][105620] Updated weights for policy 1, policy_version 1321689 (0.0010) [2023-12-27 01:00:04,080][105692] Updated weights for policy 0, policy_version 1319768 (0.0007) [2023-12-27 01:00:04,153][105692] Updated weights for policy 0, policy_version 1319778 (0.0006) [2023-12-27 01:00:04,218][105692] Updated weights for policy 0, policy_version 1319788 (0.0006) [2023-12-27 01:00:04,747][105620] Updated weights for policy 1, policy_version 1321699 (0.0010) [2023-12-27 01:00:04,797][105692] Updated weights for policy 0, policy_version 1319798 (0.0008) [2023-12-27 01:00:04,799][105620] Updated weights for policy 1, policy_version 1321709 (0.0007) [2023-12-27 01:00:04,849][105620] Updated weights for policy 1, policy_version 1321719 (0.0006) [2023-12-27 01:00:04,853][105692] Updated weights for policy 0, policy_version 1319808 (0.0008) [2023-12-27 01:00:04,858][105585] KL-divergence is very high: 106.9038 [2023-12-27 01:00:04,864][105585] KL-divergence is very high: 126.5784 [2023-12-27 01:00:04,910][105692] Updated weights for policy 0, policy_version 1319818 (0.0005) [2023-12-27 01:00:05,534][105620] Updated weights for policy 1, policy_version 1321729 (0.0009) [2023-12-27 01:00:05,583][105692] Updated weights for policy 0, policy_version 1319828 (0.0006) [2023-12-27 01:00:05,590][105620] Updated weights for policy 1, policy_version 1321739 (0.0006) [2023-12-27 01:00:05,645][105620] Updated weights for policy 1, policy_version 1321749 (0.0007) [2023-12-27 01:00:05,648][105692] Updated weights for policy 0, policy_version 1319838 (0.0006) [2023-12-27 01:00:05,705][105620] Updated weights for policy 1, policy_version 1321759 (0.0005) [2023-12-27 01:00:05,711][105692] Updated weights for policy 0, policy_version 1319848 (0.0008) [2023-12-27 01:00:06,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.2, 300 sec: 19466.4). Total num frames: 676347904. Throughput: 0: 9798.4, 1: 9984.2. Samples: 676335556. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 01:00:06,063][104569] Avg episode reward: [(0, '4852.793'), (1, '9001.520')] [2023-12-27 01:00:06,250][105620] Updated weights for policy 1, policy_version 1321769 (0.0008) [2023-12-27 01:00:06,326][105620] Updated weights for policy 1, policy_version 1321779 (0.0008) [2023-12-27 01:00:06,344][105692] Updated weights for policy 0, policy_version 1319858 (0.0009) [2023-12-27 01:00:06,392][105620] Updated weights for policy 1, policy_version 1321789 (0.0008) [2023-12-27 01:00:06,412][105692] Updated weights for policy 0, policy_version 1319868 (0.0006) [2023-12-27 01:00:06,460][105585] KL-divergence is very high: 102.9672 [2023-12-27 01:00:06,478][105692] Updated weights for policy 0, policy_version 1319878 (0.0009) [2023-12-27 01:00:06,544][105692] Updated weights for policy 0, policy_version 1319888 (0.0011) [2023-12-27 01:00:07,053][105620] Updated weights for policy 1, policy_version 1321799 (0.0007) [2023-12-27 01:00:07,116][105620] Updated weights for policy 1, policy_version 1321809 (0.0008) [2023-12-27 01:00:07,173][105620] Updated weights for policy 1, policy_version 1321819 (0.0009) [2023-12-27 01:00:07,246][105692] Updated weights for policy 0, policy_version 1319898 (0.0008) [2023-12-27 01:00:07,304][105692] Updated weights for policy 0, policy_version 1319908 (0.0009) [2023-12-27 01:00:07,355][105692] Updated weights for policy 0, policy_version 1319918 (0.0010) [2023-12-27 01:00:07,884][105620] Updated weights for policy 1, policy_version 1321829 (0.0008) [2023-12-27 01:00:07,949][105620] Updated weights for policy 1, policy_version 1321839 (0.0008) [2023-12-27 01:00:08,012][105620] Updated weights for policy 1, policy_version 1321849 (0.0011) [2023-12-27 01:00:08,122][105692] Updated weights for policy 0, policy_version 1319928 (0.0009) [2023-12-27 01:00:08,183][105692] Updated weights for policy 0, policy_version 1319938 (0.0009) [2023-12-27 01:00:08,228][105692] Updated weights for policy 0, policy_version 1319948 (0.0008) [2023-12-27 01:00:08,739][105620] Updated weights for policy 1, policy_version 1321859 (0.0011) [2023-12-27 01:00:08,784][105620] Updated weights for policy 1, policy_version 1321869 (0.0010) [2023-12-27 01:00:08,840][105620] Updated weights for policy 1, policy_version 1321879 (0.0011) [2023-12-27 01:00:08,995][105692] Updated weights for policy 0, policy_version 1319958 (0.0008) [2023-12-27 01:00:09,053][105692] Updated weights for policy 0, policy_version 1319968 (0.0009) [2023-12-27 01:00:09,117][105692] Updated weights for policy 0, policy_version 1319978 (0.0009) [2023-12-27 01:00:09,635][105620] Updated weights for policy 1, policy_version 1321889 (0.0011) [2023-12-27 01:00:09,691][105620] Updated weights for policy 1, policy_version 1321899 (0.0011) [2023-12-27 01:00:09,744][105620] Updated weights for policy 1, policy_version 1321909 (0.0011) [2023-12-27 01:00:09,812][105620] Updated weights for policy 1, policy_version 1321919 (0.0011) [2023-12-27 01:00:09,913][105692] Updated weights for policy 0, policy_version 1319988 (0.0010) [2023-12-27 01:00:09,974][105692] Updated weights for policy 0, policy_version 1319998 (0.0011) [2023-12-27 01:00:10,038][105692] Updated weights for policy 0, policy_version 1320008 (0.0010) [2023-12-27 01:00:10,564][105620] Updated weights for policy 1, policy_version 1321929 (0.0008) [2023-12-27 01:00:10,620][105620] Updated weights for policy 1, policy_version 1321939 (0.0008) [2023-12-27 01:00:10,678][105620] Updated weights for policy 1, policy_version 1321949 (0.0007) [2023-12-27 01:00:10,690][105692] Updated weights for policy 0, policy_version 1320018 (0.0009) [2023-12-27 01:00:10,750][105692] Updated weights for policy 0, policy_version 1320028 (0.0010) [2023-12-27 01:00:10,795][105692] Updated weights for policy 0, policy_version 1320038 (0.0010) [2023-12-27 01:00:10,843][105692] Updated weights for policy 0, policy_version 1320048 (0.0010) [2023-12-27 01:00:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19438.6). Total num frames: 676446208. Throughput: 0: 9853.6, 1: 9990.3. Samples: 676453672. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 01:00:11,062][104569] Avg episode reward: [(0, '5705.435'), (1, '8998.443')] [2023-12-27 01:00:11,420][105620] Updated weights for policy 1, policy_version 1321959 (0.0008) [2023-12-27 01:00:11,479][105620] Updated weights for policy 1, policy_version 1321969 (0.0009) [2023-12-27 01:00:11,538][105620] Updated weights for policy 1, policy_version 1321979 (0.0009) [2023-12-27 01:00:11,624][105692] Updated weights for policy 0, policy_version 1320058 (0.0010) [2023-12-27 01:00:11,677][105692] Updated weights for policy 0, policy_version 1320068 (0.0005) [2023-12-27 01:00:11,736][105692] Updated weights for policy 0, policy_version 1320078 (0.0008) [2023-12-27 01:00:12,307][105620] Updated weights for policy 1, policy_version 1321989 (0.0008) [2023-12-27 01:00:12,381][105620] Updated weights for policy 1, policy_version 1321999 (0.0009) [2023-12-27 01:00:12,433][105620] Updated weights for policy 1, policy_version 1322009 (0.0008) [2023-12-27 01:00:12,485][105692] Updated weights for policy 0, policy_version 1320088 (0.0010) [2023-12-27 01:00:12,547][105692] Updated weights for policy 0, policy_version 1320098 (0.0010) [2023-12-27 01:00:12,602][105692] Updated weights for policy 0, policy_version 1320108 (0.0010) [2023-12-27 01:00:13,130][105620] Updated weights for policy 1, policy_version 1322019 (0.0008) [2023-12-27 01:00:13,192][105620] Updated weights for policy 1, policy_version 1322029 (0.0010) [2023-12-27 01:00:13,248][105620] Updated weights for policy 1, policy_version 1322039 (0.0009) [2023-12-27 01:00:13,273][105692] Updated weights for policy 0, policy_version 1320118 (0.0008) [2023-12-27 01:00:13,323][105692] Updated weights for policy 0, policy_version 1320128 (0.0005) [2023-12-27 01:00:13,376][105692] Updated weights for policy 0, policy_version 1320138 (0.0005) [2023-12-27 01:00:13,958][105692] Updated weights for policy 0, policy_version 1320148 (0.0007) [2023-12-27 01:00:14,003][105620] Updated weights for policy 1, policy_version 1322049 (0.0007) [2023-12-27 01:00:14,012][105692] Updated weights for policy 0, policy_version 1320158 (0.0005) [2023-12-27 01:00:14,063][105692] Updated weights for policy 0, policy_version 1320168 (0.0005) [2023-12-27 01:00:14,072][105620] Updated weights for policy 1, policy_version 1322059 (0.0008) [2023-12-27 01:00:14,134][105620] Updated weights for policy 1, policy_version 1322069 (0.0009) [2023-12-27 01:00:14,185][105620] Updated weights for policy 1, policy_version 1322079 (0.0009) [2023-12-27 01:00:14,704][105692] Updated weights for policy 0, policy_version 1320178 (0.0006) [2023-12-27 01:00:14,774][105692] Updated weights for policy 0, policy_version 1320188 (0.0006) [2023-12-27 01:00:14,841][105692] Updated weights for policy 0, policy_version 1320198 (0.0008) [2023-12-27 01:00:14,907][105692] Updated weights for policy 0, policy_version 1320208 (0.0008) [2023-12-27 01:00:14,995][105620] Updated weights for policy 1, policy_version 1322089 (0.0005) [2023-12-27 01:00:15,067][105620] Updated weights for policy 1, policy_version 1322099 (0.0007) [2023-12-27 01:00:15,133][105620] Updated weights for policy 1, policy_version 1322109 (0.0008) [2023-12-27 01:00:15,555][105692] Updated weights for policy 0, policy_version 1320218 (0.0006) [2023-12-27 01:00:15,610][105692] Updated weights for policy 0, policy_version 1320228 (0.0005) [2023-12-27 01:00:15,679][105692] Updated weights for policy 0, policy_version 1320238 (0.0005) [2023-12-27 01:00:15,885][105620] Updated weights for policy 1, policy_version 1322119 (0.0009) [2023-12-27 01:00:15,943][105620] Updated weights for policy 1, policy_version 1322129 (0.0009) [2023-12-27 01:00:15,991][105620] Updated weights for policy 1, policy_version 1322139 (0.0008) [2023-12-27 01:00:16,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 676544512. Throughput: 0: 9823.4, 1: 9900.1. Samples: 676510784. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 01:00:16,063][104569] Avg episode reward: [(0, '8096.013'), (1, '9267.140')] [2023-12-27 01:00:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001320240_338034688.pth... [2023-12-27 01:00:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001322144_338509824.pth... [2023-12-27 01:00:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001320960_338206720.pth [2023-12-27 01:00:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001319088_337739776.pth [2023-12-27 01:00:16,343][105692] Updated weights for policy 0, policy_version 1320248 (0.0009) [2023-12-27 01:00:16,398][105692] Updated weights for policy 0, policy_version 1320258 (0.0009) [2023-12-27 01:00:16,456][105692] Updated weights for policy 0, policy_version 1320268 (0.0009) [2023-12-27 01:00:16,753][105620] Updated weights for policy 1, policy_version 1322149 (0.0009) [2023-12-27 01:00:16,807][105620] Updated weights for policy 1, policy_version 1322159 (0.0008) [2023-12-27 01:00:16,866][105620] Updated weights for policy 1, policy_version 1322169 (0.0009) [2023-12-27 01:00:17,205][105692] Updated weights for policy 0, policy_version 1320278 (0.0010) [2023-12-27 01:00:17,265][105692] Updated weights for policy 0, policy_version 1320289 (0.0012) [2023-12-27 01:00:17,324][105692] Updated weights for policy 0, policy_version 1320299 (0.0009) [2023-12-27 01:00:17,572][105620] Updated weights for policy 1, policy_version 1322179 (0.0009) [2023-12-27 01:00:17,634][105620] Updated weights for policy 1, policy_version 1322189 (0.0009) [2023-12-27 01:00:17,689][105620] Updated weights for policy 1, policy_version 1322199 (0.0009) [2023-12-27 01:00:18,079][105692] Updated weights for policy 0, policy_version 1320309 (0.0009) [2023-12-27 01:00:18,133][105692] Updated weights for policy 0, policy_version 1320319 (0.0009) [2023-12-27 01:00:18,191][105692] Updated weights for policy 0, policy_version 1320329 (0.0010) [2023-12-27 01:00:18,431][105620] Updated weights for policy 1, policy_version 1322209 (0.0009) [2023-12-27 01:00:18,491][105620] Updated weights for policy 1, policy_version 1322219 (0.0010) [2023-12-27 01:00:18,547][105620] Updated weights for policy 1, policy_version 1322229 (0.0010) [2023-12-27 01:00:18,610][105620] Updated weights for policy 1, policy_version 1322239 (0.0010) [2023-12-27 01:00:18,958][105692] Updated weights for policy 0, policy_version 1320339 (0.0010) [2023-12-27 01:00:19,004][105692] Updated weights for policy 0, policy_version 1320349 (0.0011) [2023-12-27 01:00:19,050][105692] Updated weights for policy 0, policy_version 1320359 (0.0011) [2023-12-27 01:00:19,217][105620] Updated weights for policy 1, policy_version 1322249 (0.0006) [2023-12-27 01:00:19,282][105620] Updated weights for policy 1, policy_version 1322259 (0.0011) [2023-12-27 01:00:19,342][105620] Updated weights for policy 1, policy_version 1322269 (0.0011) [2023-12-27 01:00:19,866][105692] Updated weights for policy 0, policy_version 1320369 (0.0010) [2023-12-27 01:00:19,933][105692] Updated weights for policy 0, policy_version 1320379 (0.0009) [2023-12-27 01:00:19,998][105692] Updated weights for policy 0, policy_version 1320389 (0.0009) [2023-12-27 01:00:20,007][105620] Updated weights for policy 1, policy_version 1322279 (0.0007) [2023-12-27 01:00:20,055][105692] Updated weights for policy 0, policy_version 1320399 (0.0009) [2023-12-27 01:00:20,101][105620] Updated weights for policy 1, policy_version 1322289 (0.0006) [2023-12-27 01:00:20,170][105620] Updated weights for policy 1, policy_version 1322299 (0.0008) [2023-12-27 01:00:20,756][105620] Updated weights for policy 1, policy_version 1322309 (0.0009) [2023-12-27 01:00:20,812][105620] Updated weights for policy 1, policy_version 1322319 (0.0010) [2023-12-27 01:00:20,829][105586] KL-divergence is very high: 115.0920 [2023-12-27 01:00:20,869][105620] Updated weights for policy 1, policy_version 1322329 (0.0010) [2023-12-27 01:00:20,871][105692] Updated weights for policy 0, policy_version 1320409 (0.0007) [2023-12-27 01:00:20,876][105586] KL-divergence is very high: 184.4561 [2023-12-27 01:00:20,929][105692] Updated weights for policy 0, policy_version 1320419 (0.0007) [2023-12-27 01:00:20,988][105692] Updated weights for policy 0, policy_version 1320429 (0.0009) [2023-12-27 01:00:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 676642816. Throughput: 0: 9786.1, 1: 9874.3. Samples: 676627520. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-12-27 01:00:21,063][104569] Avg episode reward: [(0, '8724.901'), (1, '9269.975')] [2023-12-27 01:00:21,643][105620] Updated weights for policy 1, policy_version 1322339 (0.0011) [2023-12-27 01:00:21,707][105620] Updated weights for policy 1, policy_version 1322349 (0.0011) [2023-12-27 01:00:21,774][105620] Updated weights for policy 1, policy_version 1322359 (0.0009) [2023-12-27 01:00:21,808][105692] Updated weights for policy 0, policy_version 1320440 (0.0007) [2023-12-27 01:00:21,874][105692] Updated weights for policy 0, policy_version 1320450 (0.0007) [2023-12-27 01:00:21,938][105692] Updated weights for policy 0, policy_version 1320460 (0.0008) [2023-12-27 01:00:22,454][105620] Updated weights for policy 1, policy_version 1322369 (0.0010) [2023-12-27 01:00:22,517][105620] Updated weights for policy 1, policy_version 1322379 (0.0008) [2023-12-27 01:00:22,591][105620] Updated weights for policy 1, policy_version 1322389 (0.0008) [2023-12-27 01:00:22,646][105620] Updated weights for policy 1, policy_version 1322399 (0.0008) [2023-12-27 01:00:22,703][105692] Updated weights for policy 0, policy_version 1320470 (0.0008) [2023-12-27 01:00:22,761][105692] Updated weights for policy 0, policy_version 1320480 (0.0009) [2023-12-27 01:00:22,817][105692] Updated weights for policy 0, policy_version 1320490 (0.0008) [2023-12-27 01:00:23,416][105620] Updated weights for policy 1, policy_version 1322409 (0.0009) [2023-12-27 01:00:23,478][105620] Updated weights for policy 1, policy_version 1322419 (0.0009) [2023-12-27 01:00:23,533][105620] Updated weights for policy 1, policy_version 1322429 (0.0009) [2023-12-27 01:00:23,566][105692] Updated weights for policy 0, policy_version 1320500 (0.0008) [2023-12-27 01:00:23,618][105692] Updated weights for policy 0, policy_version 1320510 (0.0009) [2023-12-27 01:00:23,667][105692] Updated weights for policy 0, policy_version 1320520 (0.0009) [2023-12-27 01:00:24,305][105620] Updated weights for policy 1, policy_version 1322439 (0.0009) [2023-12-27 01:00:24,348][105620] Updated weights for policy 1, policy_version 1322449 (0.0007) [2023-12-27 01:00:24,395][105620] Updated weights for policy 1, policy_version 1322459 (0.0008) [2023-12-27 01:00:24,446][105692] Updated weights for policy 0, policy_version 1320530 (0.0009) [2023-12-27 01:00:24,496][105692] Updated weights for policy 0, policy_version 1320540 (0.0009) [2023-12-27 01:00:24,551][105692] Updated weights for policy 0, policy_version 1320550 (0.0009) [2023-12-27 01:00:24,605][105692] Updated weights for policy 0, policy_version 1320560 (0.0009) [2023-12-27 01:00:25,043][105620] Updated weights for policy 1, policy_version 1322469 (0.0009) [2023-12-27 01:00:25,101][105620] Updated weights for policy 1, policy_version 1322479 (0.0009) [2023-12-27 01:00:25,149][105620] Updated weights for policy 1, policy_version 1322489 (0.0009) [2023-12-27 01:00:25,424][105692] Updated weights for policy 0, policy_version 1320570 (0.0009) [2023-12-27 01:00:25,477][105692] Updated weights for policy 0, policy_version 1320580 (0.0008) [2023-12-27 01:00:25,536][105692] Updated weights for policy 0, policy_version 1320590 (0.0010) [2023-12-27 01:00:25,855][105620] Updated weights for policy 1, policy_version 1322499 (0.0008) [2023-12-27 01:00:25,908][105620] Updated weights for policy 1, policy_version 1322509 (0.0005) [2023-12-27 01:00:25,958][105620] Updated weights for policy 1, policy_version 1322519 (0.0007) [2023-12-27 01:00:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 676732928. Throughput: 0: 9641.2, 1: 9890.6. Samples: 676739436. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:00:26,063][104569] Avg episode reward: [(0, '8545.384'), (1, '9181.697')] [2023-12-27 01:00:26,296][105692] Updated weights for policy 0, policy_version 1320600 (0.0009) [2023-12-27 01:00:26,350][105692] Updated weights for policy 0, policy_version 1320610 (0.0009) [2023-12-27 01:00:26,416][105692] Updated weights for policy 0, policy_version 1320620 (0.0009) [2023-12-27 01:00:26,692][105620] Updated weights for policy 1, policy_version 1322529 (0.0009) [2023-12-27 01:00:26,739][105620] Updated weights for policy 1, policy_version 1322539 (0.0009) [2023-12-27 01:00:26,786][105620] Updated weights for policy 1, policy_version 1322549 (0.0007) [2023-12-27 01:00:26,851][105620] Updated weights for policy 1, policy_version 1322559 (0.0009) [2023-12-27 01:00:27,193][105692] Updated weights for policy 0, policy_version 1320630 (0.0008) [2023-12-27 01:00:27,236][105692] Updated weights for policy 0, policy_version 1320640 (0.0008) [2023-12-27 01:00:27,285][105692] Updated weights for policy 0, policy_version 1320650 (0.0008) [2023-12-27 01:00:27,510][105620] Updated weights for policy 1, policy_version 1322569 (0.0009) [2023-12-27 01:00:27,568][105620] Updated weights for policy 1, policy_version 1322579 (0.0009) [2023-12-27 01:00:27,615][105620] Updated weights for policy 1, policy_version 1322589 (0.0009) [2023-12-27 01:00:28,037][105692] Updated weights for policy 0, policy_version 1320660 (0.0006) [2023-12-27 01:00:28,087][105692] Updated weights for policy 0, policy_version 1320670 (0.0005) [2023-12-27 01:00:28,141][105692] Updated weights for policy 0, policy_version 1320680 (0.0010) [2023-12-27 01:00:28,383][105620] Updated weights for policy 1, policy_version 1322599 (0.0008) [2023-12-27 01:00:28,442][105620] Updated weights for policy 1, policy_version 1322609 (0.0005) [2023-12-27 01:00:28,495][105620] Updated weights for policy 1, policy_version 1322619 (0.0005) [2023-12-27 01:00:28,771][105692] Updated weights for policy 0, policy_version 1320690 (0.0009) [2023-12-27 01:00:28,825][105692] Updated weights for policy 0, policy_version 1320700 (0.0008) [2023-12-27 01:00:28,886][105692] Updated weights for policy 0, policy_version 1320710 (0.0007) [2023-12-27 01:00:28,939][105692] Updated weights for policy 0, policy_version 1320720 (0.0008) [2023-12-27 01:00:29,164][105620] Updated weights for policy 1, policy_version 1322629 (0.0008) [2023-12-27 01:00:29,222][105620] Updated weights for policy 1, policy_version 1322639 (0.0010) [2023-12-27 01:00:29,281][105620] Updated weights for policy 1, policy_version 1322649 (0.0010) [2023-12-27 01:00:29,626][105692] Updated weights for policy 0, policy_version 1320730 (0.0008) [2023-12-27 01:00:29,681][105692] Updated weights for policy 0, policy_version 1320740 (0.0008) [2023-12-27 01:00:29,732][105692] Updated weights for policy 0, policy_version 1320750 (0.0008) [2023-12-27 01:00:30,035][105620] Updated weights for policy 1, policy_version 1322659 (0.0011) [2023-12-27 01:00:30,083][105620] Updated weights for policy 1, policy_version 1322669 (0.0010) [2023-12-27 01:00:30,138][105620] Updated weights for policy 1, policy_version 1322679 (0.0010) [2023-12-27 01:00:30,503][105692] Updated weights for policy 0, policy_version 1320760 (0.0008) [2023-12-27 01:00:30,565][105692] Updated weights for policy 0, policy_version 1320770 (0.0008) [2023-12-27 01:00:30,623][105692] Updated weights for policy 0, policy_version 1320780 (0.0008) [2023-12-27 01:00:30,879][105620] Updated weights for policy 1, policy_version 1322689 (0.0010) [2023-12-27 01:00:30,950][105620] Updated weights for policy 1, policy_version 1322699 (0.0006) [2023-12-27 01:00:31,012][105620] Updated weights for policy 1, policy_version 1322709 (0.0007) [2023-12-27 01:00:31,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 676823040. Throughput: 0: 9616.1, 1: 9884.5. Samples: 676798284. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:00:31,062][104569] Avg episode reward: [(0, '8725.688'), (1, '9177.640')] [2023-12-27 01:00:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001320784_338173952.pth... [2023-12-27 01:00:31,070][105620] Updated weights for policy 1, policy_version 1322719 (0.0008) [2023-12-27 01:00:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001319664_337887232.pth [2023-12-27 01:00:31,076][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001322720_338657280.pth... [2023-12-27 01:00:31,081][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001321568_338362368.pth [2023-12-27 01:00:31,316][105692] Updated weights for policy 0, policy_version 1320790 (0.0006) [2023-12-27 01:00:31,390][105692] Updated weights for policy 0, policy_version 1320800 (0.0007) [2023-12-27 01:00:31,438][105692] Updated weights for policy 0, policy_version 1320810 (0.0006) [2023-12-27 01:00:31,805][105620] Updated weights for policy 1, policy_version 1322729 (0.0006) [2023-12-27 01:00:31,869][105620] Updated weights for policy 1, policy_version 1322739 (0.0010) [2023-12-27 01:00:31,928][105620] Updated weights for policy 1, policy_version 1322749 (0.0009) [2023-12-27 01:00:32,049][105692] Updated weights for policy 0, policy_version 1320820 (0.0005) [2023-12-27 01:00:32,111][105692] Updated weights for policy 0, policy_version 1320830 (0.0006) [2023-12-27 01:00:32,170][105692] Updated weights for policy 0, policy_version 1320840 (0.0009) [2023-12-27 01:00:32,542][105620] Updated weights for policy 1, policy_version 1322759 (0.0006) [2023-12-27 01:00:32,602][105620] Updated weights for policy 1, policy_version 1322769 (0.0008) [2023-12-27 01:00:32,657][105620] Updated weights for policy 1, policy_version 1322779 (0.0010) [2023-12-27 01:00:32,786][105692] Updated weights for policy 0, policy_version 1320850 (0.0008) [2023-12-27 01:00:32,834][105692] Updated weights for policy 0, policy_version 1320860 (0.0008) [2023-12-27 01:00:32,885][105692] Updated weights for policy 0, policy_version 1320870 (0.0008) [2023-12-27 01:00:32,934][105692] Updated weights for policy 0, policy_version 1320880 (0.0010) [2023-12-27 01:00:33,360][105620] Updated weights for policy 1, policy_version 1322789 (0.0008) [2023-12-27 01:00:33,414][105620] Updated weights for policy 1, policy_version 1322799 (0.0010) [2023-12-27 01:00:33,475][105620] Updated weights for policy 1, policy_version 1322809 (0.0010) [2023-12-27 01:00:33,653][105692] Updated weights for policy 0, policy_version 1320890 (0.0005) [2023-12-27 01:00:33,714][105692] Updated weights for policy 0, policy_version 1320900 (0.0006) [2023-12-27 01:00:33,763][105692] Updated weights for policy 0, policy_version 1320910 (0.0008) [2023-12-27 01:00:34,121][105620] Updated weights for policy 1, policy_version 1322819 (0.0009) [2023-12-27 01:00:34,188][105620] Updated weights for policy 1, policy_version 1322829 (0.0009) [2023-12-27 01:00:34,246][105620] Updated weights for policy 1, policy_version 1322839 (0.0007) [2023-12-27 01:00:34,600][105692] Updated weights for policy 0, policy_version 1320920 (0.0010) [2023-12-27 01:00:34,659][105692] Updated weights for policy 0, policy_version 1320930 (0.0009) [2023-12-27 01:00:34,719][105692] Updated weights for policy 0, policy_version 1320940 (0.0011) [2023-12-27 01:00:34,953][105620] Updated weights for policy 1, policy_version 1322849 (0.0008) [2023-12-27 01:00:35,019][105620] Updated weights for policy 1, policy_version 1322859 (0.0011) [2023-12-27 01:00:35,086][105620] Updated weights for policy 1, policy_version 1322869 (0.0011) [2023-12-27 01:00:35,143][105620] Updated weights for policy 1, policy_version 1322879 (0.0011) [2023-12-27 01:00:35,468][105692] Updated weights for policy 0, policy_version 1320950 (0.0011) [2023-12-27 01:00:35,527][105692] Updated weights for policy 0, policy_version 1320960 (0.0011) [2023-12-27 01:00:35,586][105692] Updated weights for policy 0, policy_version 1320970 (0.0011) [2023-12-27 01:00:35,874][105620] Updated weights for policy 1, policy_version 1322889 (0.0011) [2023-12-27 01:00:35,941][105620] Updated weights for policy 1, policy_version 1322899 (0.0011) [2023-12-27 01:00:36,000][105620] Updated weights for policy 1, policy_version 1322909 (0.0010) [2023-12-27 01:00:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19438.7). Total num frames: 676929536. Throughput: 0: 9728.5, 1: 9892.1. Samples: 676917768. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:00:36,062][104569] Avg episode reward: [(0, '8633.635'), (1, '9086.606')] [2023-12-27 01:00:36,308][105692] Updated weights for policy 0, policy_version 1320980 (0.0010) [2023-12-27 01:00:36,357][105692] Updated weights for policy 0, policy_version 1320990 (0.0008) [2023-12-27 01:00:36,405][105692] Updated weights for policy 0, policy_version 1321000 (0.0008) [2023-12-27 01:00:36,741][105620] Updated weights for policy 1, policy_version 1322919 (0.0011) [2023-12-27 01:00:36,808][105620] Updated weights for policy 1, policy_version 1322929 (0.0011) [2023-12-27 01:00:36,863][105620] Updated weights for policy 1, policy_version 1322939 (0.0010) [2023-12-27 01:00:37,112][105692] Updated weights for policy 0, policy_version 1321010 (0.0008) [2023-12-27 01:00:37,171][105692] Updated weights for policy 0, policy_version 1321020 (0.0008) [2023-12-27 01:00:37,233][105692] Updated weights for policy 0, policy_version 1321030 (0.0008) [2023-12-27 01:00:37,289][105692] Updated weights for policy 0, policy_version 1321040 (0.0008) [2023-12-27 01:00:37,547][105620] Updated weights for policy 1, policy_version 1322949 (0.0011) [2023-12-27 01:00:37,613][105620] Updated weights for policy 1, policy_version 1322959 (0.0011) [2023-12-27 01:00:37,675][105620] Updated weights for policy 1, policy_version 1322969 (0.0010) [2023-12-27 01:00:38,002][105692] Updated weights for policy 0, policy_version 1321050 (0.0007) [2023-12-27 01:00:38,061][105692] Updated weights for policy 0, policy_version 1321060 (0.0009) [2023-12-27 01:00:38,116][105692] Updated weights for policy 0, policy_version 1321070 (0.0009) [2023-12-27 01:00:38,434][105620] Updated weights for policy 1, policy_version 1322979 (0.0011) [2023-12-27 01:00:38,497][105620] Updated weights for policy 1, policy_version 1322989 (0.0011) [2023-12-27 01:00:38,564][105620] Updated weights for policy 1, policy_version 1322999 (0.0011) [2023-12-27 01:00:38,823][105692] Updated weights for policy 0, policy_version 1321080 (0.0008) [2023-12-27 01:00:38,883][105692] Updated weights for policy 0, policy_version 1321090 (0.0008) [2023-12-27 01:00:38,946][105692] Updated weights for policy 0, policy_version 1321100 (0.0008) [2023-12-27 01:00:39,244][105620] Updated weights for policy 1, policy_version 1323009 (0.0010) [2023-12-27 01:00:39,304][105620] Updated weights for policy 1, policy_version 1323019 (0.0010) [2023-12-27 01:00:39,376][105620] Updated weights for policy 1, policy_version 1323029 (0.0011) [2023-12-27 01:00:39,442][105620] Updated weights for policy 1, policy_version 1323039 (0.0010) [2023-12-27 01:00:39,746][105692] Updated weights for policy 0, policy_version 1321110 (0.0008) [2023-12-27 01:00:39,812][105692] Updated weights for policy 0, policy_version 1321120 (0.0006) [2023-12-27 01:00:39,874][105692] Updated weights for policy 0, policy_version 1321130 (0.0008) [2023-12-27 01:00:40,219][105620] Updated weights for policy 1, policy_version 1323049 (0.0011) [2023-12-27 01:00:40,288][105620] Updated weights for policy 1, policy_version 1323059 (0.0010) [2023-12-27 01:00:40,350][105620] Updated weights for policy 1, policy_version 1323069 (0.0010) [2023-12-27 01:00:40,584][105692] Updated weights for policy 0, policy_version 1321140 (0.0008) [2023-12-27 01:00:40,633][105692] Updated weights for policy 0, policy_version 1321150 (0.0008) [2023-12-27 01:00:40,685][105692] Updated weights for policy 0, policy_version 1321160 (0.0009) [2023-12-27 01:00:40,982][105620] Updated weights for policy 1, policy_version 1323079 (0.0008) [2023-12-27 01:00:41,042][105620] Updated weights for policy 1, policy_version 1323089 (0.0011) [2023-12-27 01:00:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 677019648. Throughput: 0: 9798.0, 1: 9801.5. Samples: 677032428. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:00:41,062][104569] Avg episode reward: [(0, '8724.525'), (1, '9090.274')] [2023-12-27 01:00:41,105][105620] Updated weights for policy 1, policy_version 1323099 (0.0011) [2023-12-27 01:00:41,533][105692] Updated weights for policy 0, policy_version 1321170 (0.0009) [2023-12-27 01:00:41,589][105692] Updated weights for policy 0, policy_version 1321180 (0.0006) [2023-12-27 01:00:41,657][105692] Updated weights for policy 0, policy_version 1321190 (0.0008) [2023-12-27 01:00:41,713][105692] Updated weights for policy 0, policy_version 1321200 (0.0009) [2023-12-27 01:00:41,747][105620] Updated weights for policy 1, policy_version 1323109 (0.0009) [2023-12-27 01:00:41,803][105620] Updated weights for policy 1, policy_version 1323119 (0.0008) [2023-12-27 01:00:41,854][105620] Updated weights for policy 1, policy_version 1323129 (0.0008) [2023-12-27 01:00:42,437][105692] Updated weights for policy 0, policy_version 1321210 (0.0006) [2023-12-27 01:00:42,505][105692] Updated weights for policy 0, policy_version 1321220 (0.0007) [2023-12-27 01:00:42,564][105692] Updated weights for policy 0, policy_version 1321230 (0.0009) [2023-12-27 01:00:42,656][105620] Updated weights for policy 1, policy_version 1323139 (0.0008) [2023-12-27 01:00:42,706][105620] Updated weights for policy 1, policy_version 1323149 (0.0009) [2023-12-27 01:00:42,763][105620] Updated weights for policy 1, policy_version 1323159 (0.0009) [2023-12-27 01:00:43,241][105692] Updated weights for policy 0, policy_version 1321240 (0.0008) [2023-12-27 01:00:43,288][105692] Updated weights for policy 0, policy_version 1321250 (0.0009) [2023-12-27 01:00:43,337][105692] Updated weights for policy 0, policy_version 1321260 (0.0008) [2023-12-27 01:00:43,488][105620] Updated weights for policy 1, policy_version 1323169 (0.0011) [2023-12-27 01:00:43,549][105620] Updated weights for policy 1, policy_version 1323179 (0.0007) [2023-12-27 01:00:43,602][105620] Updated weights for policy 1, policy_version 1323189 (0.0010) [2023-12-27 01:00:43,659][105620] Updated weights for policy 1, policy_version 1323199 (0.0010) [2023-12-27 01:00:44,006][105692] Updated weights for policy 0, policy_version 1321270 (0.0008) [2023-12-27 01:00:44,071][105692] Updated weights for policy 0, policy_version 1321280 (0.0010) [2023-12-27 01:00:44,132][105692] Updated weights for policy 0, policy_version 1321290 (0.0010) [2023-12-27 01:00:44,399][105620] Updated weights for policy 1, policy_version 1323209 (0.0011) [2023-12-27 01:00:44,454][105620] Updated weights for policy 1, policy_version 1323219 (0.0010) [2023-12-27 01:00:44,512][105620] Updated weights for policy 1, policy_version 1323229 (0.0010) [2023-12-27 01:00:44,839][105692] Updated weights for policy 0, policy_version 1321300 (0.0010) [2023-12-27 01:00:44,902][105692] Updated weights for policy 0, policy_version 1321310 (0.0006) [2023-12-27 01:00:44,959][105692] Updated weights for policy 0, policy_version 1321320 (0.0006) [2023-12-27 01:00:45,210][105620] Updated weights for policy 1, policy_version 1323239 (0.0011) [2023-12-27 01:00:45,271][105620] Updated weights for policy 1, policy_version 1323249 (0.0011) [2023-12-27 01:00:45,337][105620] Updated weights for policy 1, policy_version 1323259 (0.0011) [2023-12-27 01:00:45,562][105692] Updated weights for policy 0, policy_version 1321330 (0.0007) [2023-12-27 01:00:45,610][105692] Updated weights for policy 0, policy_version 1321340 (0.0006) [2023-12-27 01:00:45,677][105692] Updated weights for policy 0, policy_version 1321350 (0.0005) [2023-12-27 01:00:45,735][105692] Updated weights for policy 0, policy_version 1321360 (0.0005) [2023-12-27 01:00:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 677117952. Throughput: 0: 9683.2, 1: 9703.1. Samples: 677088852. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:00:46,063][104569] Avg episode reward: [(0, '8182.937'), (1, '9269.325')] [2023-12-27 01:00:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001321360_338321408.pth... [2023-12-27 01:00:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001320240_338034688.pth [2023-12-27 01:00:46,075][105620] Updated weights for policy 1, policy_version 1323269 (0.0010) [2023-12-27 01:00:46,137][105620] Updated weights for policy 1, policy_version 1323279 (0.0007) [2023-12-27 01:00:46,199][105620] Updated weights for policy 1, policy_version 1323289 (0.0009) [2023-12-27 01:00:46,244][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001323296_338804736.pth... [2023-12-27 01:00:46,249][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001322144_338509824.pth [2023-12-27 01:00:46,293][105692] Updated weights for policy 0, policy_version 1321370 (0.0005) [2023-12-27 01:00:46,343][105692] Updated weights for policy 0, policy_version 1321380 (0.0005) [2023-12-27 01:00:46,406][105692] Updated weights for policy 0, policy_version 1321390 (0.0005) [2023-12-27 01:00:46,915][105620] Updated weights for policy 1, policy_version 1323299 (0.0010) [2023-12-27 01:00:46,963][105620] Updated weights for policy 1, policy_version 1323309 (0.0010) [2023-12-27 01:00:47,007][105620] Updated weights for policy 1, policy_version 1323319 (0.0010) [2023-12-27 01:00:47,056][105692] Updated weights for policy 0, policy_version 1321400 (0.0005) [2023-12-27 01:00:47,110][105692] Updated weights for policy 0, policy_version 1321410 (0.0005) [2023-12-27 01:00:47,162][105692] Updated weights for policy 0, policy_version 1321420 (0.0007) [2023-12-27 01:00:47,724][105620] Updated weights for policy 1, policy_version 1323329 (0.0010) [2023-12-27 01:00:47,780][105620] Updated weights for policy 1, policy_version 1323339 (0.0005) [2023-12-27 01:00:47,799][105692] Updated weights for policy 0, policy_version 1321430 (0.0008) [2023-12-27 01:00:47,832][105620] Updated weights for policy 1, policy_version 1323349 (0.0008) [2023-12-27 01:00:47,847][105692] Updated weights for policy 0, policy_version 1321440 (0.0005) [2023-12-27 01:00:47,880][105620] Updated weights for policy 1, policy_version 1323359 (0.0010) [2023-12-27 01:00:47,894][105692] Updated weights for policy 0, policy_version 1321450 (0.0006) [2023-12-27 01:00:48,543][105620] Updated weights for policy 1, policy_version 1323369 (0.0009) [2023-12-27 01:00:48,605][105692] Updated weights for policy 0, policy_version 1321460 (0.0009) [2023-12-27 01:00:48,613][105620] Updated weights for policy 1, policy_version 1323379 (0.0010) [2023-12-27 01:00:48,674][105692] Updated weights for policy 0, policy_version 1321470 (0.0007) [2023-12-27 01:00:48,681][105620] Updated weights for policy 1, policy_version 1323389 (0.0008) [2023-12-27 01:00:48,742][105692] Updated weights for policy 0, policy_version 1321480 (0.0007) [2023-12-27 01:00:49,435][105692] Updated weights for policy 0, policy_version 1321490 (0.0009) [2023-12-27 01:00:49,452][105620] Updated weights for policy 1, policy_version 1323399 (0.0007) [2023-12-27 01:00:49,496][105692] Updated weights for policy 0, policy_version 1321500 (0.0006) [2023-12-27 01:00:49,512][105620] Updated weights for policy 1, policy_version 1323409 (0.0009) [2023-12-27 01:00:49,546][105692] Updated weights for policy 0, policy_version 1321510 (0.0008) [2023-12-27 01:00:49,574][105620] Updated weights for policy 1, policy_version 1323419 (0.0008) [2023-12-27 01:00:49,608][105692] Updated weights for policy 0, policy_version 1321520 (0.0006) [2023-12-27 01:00:50,303][105620] Updated weights for policy 1, policy_version 1323429 (0.0007) [2023-12-27 01:00:50,365][105620] Updated weights for policy 1, policy_version 1323439 (0.0008) [2023-12-27 01:00:50,391][105692] Updated weights for policy 0, policy_version 1321530 (0.0006) [2023-12-27 01:00:50,423][105620] Updated weights for policy 1, policy_version 1323449 (0.0008) [2023-12-27 01:00:50,450][105692] Updated weights for policy 0, policy_version 1321540 (0.0009) [2023-12-27 01:00:50,499][105692] Updated weights for policy 0, policy_version 1321550 (0.0009) [2023-12-27 01:00:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 677216256. Throughput: 0: 9708.8, 1: 9721.1. Samples: 677209892. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:00:51,062][104569] Avg episode reward: [(0, '8179.899'), (1, '9265.617')] [2023-12-27 01:00:51,207][105620] Updated weights for policy 1, policy_version 1323459 (0.0008) [2023-12-27 01:00:51,209][105692] Updated weights for policy 0, policy_version 1321560 (0.0009) [2023-12-27 01:00:51,268][105620] Updated weights for policy 1, policy_version 1323469 (0.0007) [2023-12-27 01:00:51,274][105692] Updated weights for policy 0, policy_version 1321570 (0.0008) [2023-12-27 01:00:51,328][105620] Updated weights for policy 1, policy_version 1323479 (0.0008) [2023-12-27 01:00:51,331][105692] Updated weights for policy 0, policy_version 1321580 (0.0006) [2023-12-27 01:00:52,042][105692] Updated weights for policy 0, policy_version 1321590 (0.0007) [2023-12-27 01:00:52,103][105692] Updated weights for policy 0, policy_version 1321600 (0.0006) [2023-12-27 01:00:52,129][105585] KL-divergence is very high: 157.4681 [2023-12-27 01:00:52,132][105620] Updated weights for policy 1, policy_version 1323489 (0.0009) [2023-12-27 01:00:52,158][105585] KL-divergence is very high: 157.7396 [2023-12-27 01:00:52,163][105692] Updated weights for policy 0, policy_version 1321610 (0.0005) [2023-12-27 01:00:52,175][105585] KL-divergence is very high: 186.3832 [2023-12-27 01:00:52,191][105620] Updated weights for policy 1, policy_version 1323499 (0.0008) [2023-12-27 01:00:52,247][105620] Updated weights for policy 1, policy_version 1323509 (0.0009) [2023-12-27 01:00:52,311][105620] Updated weights for policy 1, policy_version 1323519 (0.0008) [2023-12-27 01:00:52,780][105692] Updated weights for policy 0, policy_version 1321620 (0.0007) [2023-12-27 01:00:52,829][105692] Updated weights for policy 0, policy_version 1321630 (0.0009) [2023-12-27 01:00:52,895][105692] Updated weights for policy 0, policy_version 1321640 (0.0009) [2023-12-27 01:00:53,116][105620] Updated weights for policy 1, policy_version 1323529 (0.0007) [2023-12-27 01:00:53,170][105620] Updated weights for policy 1, policy_version 1323539 (0.0010) [2023-12-27 01:00:53,224][105620] Updated weights for policy 1, policy_version 1323549 (0.0008) [2023-12-27 01:00:53,631][105692] Updated weights for policy 0, policy_version 1321650 (0.0010) [2023-12-27 01:00:53,688][105692] Updated weights for policy 0, policy_version 1321660 (0.0009) [2023-12-27 01:00:53,739][105692] Updated weights for policy 0, policy_version 1321670 (0.0009) [2023-12-27 01:00:53,799][105692] Updated weights for policy 0, policy_version 1321680 (0.0009) [2023-12-27 01:00:54,023][105620] Updated weights for policy 1, policy_version 1323559 (0.0010) [2023-12-27 01:00:54,091][105620] Updated weights for policy 1, policy_version 1323569 (0.0011) [2023-12-27 01:00:54,151][105620] Updated weights for policy 1, policy_version 1323579 (0.0011) [2023-12-27 01:00:54,614][105692] Updated weights for policy 0, policy_version 1321690 (0.0008) [2023-12-27 01:00:54,677][105692] Updated weights for policy 0, policy_version 1321700 (0.0008) [2023-12-27 01:00:54,736][105692] Updated weights for policy 0, policy_version 1321710 (0.0008) [2023-12-27 01:00:54,931][105620] Updated weights for policy 1, policy_version 1323589 (0.0010) [2023-12-27 01:00:55,000][105620] Updated weights for policy 1, policy_version 1323599 (0.0008) [2023-12-27 01:00:55,066][105620] Updated weights for policy 1, policy_version 1323609 (0.0008) [2023-12-27 01:00:55,506][105692] Updated weights for policy 0, policy_version 1321720 (0.0010) [2023-12-27 01:00:55,562][105692] Updated weights for policy 0, policy_version 1321730 (0.0008) [2023-12-27 01:00:55,610][105692] Updated weights for policy 0, policy_version 1321740 (0.0009) [2023-12-27 01:00:55,733][105620] Updated weights for policy 1, policy_version 1323619 (0.0007) [2023-12-27 01:00:55,790][105620] Updated weights for policy 1, policy_version 1323629 (0.0009) [2023-12-27 01:00:55,837][105620] Updated weights for policy 1, policy_version 1323639 (0.0008) [2023-12-27 01:00:56,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19383.1). Total num frames: 677314560. Throughput: 0: 9682.2, 1: 9586.9. Samples: 677320784. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:00:56,063][104569] Avg episode reward: [(0, '8450.562'), (1, '9176.342')] [2023-12-27 01:00:56,396][105692] Updated weights for policy 0, policy_version 1321750 (0.0009) [2023-12-27 01:00:56,451][105692] Updated weights for policy 0, policy_version 1321760 (0.0007) [2023-12-27 01:00:56,496][105692] Updated weights for policy 0, policy_version 1321770 (0.0008) [2023-12-27 01:00:56,587][105620] Updated weights for policy 1, policy_version 1323649 (0.0009) [2023-12-27 01:00:56,642][105620] Updated weights for policy 1, policy_version 1323659 (0.0010) [2023-12-27 01:00:56,707][105620] Updated weights for policy 1, policy_version 1323669 (0.0010) [2023-12-27 01:00:56,759][105620] Updated weights for policy 1, policy_version 1323679 (0.0010) [2023-12-27 01:00:57,120][105692] Updated weights for policy 0, policy_version 1321780 (0.0007) [2023-12-27 01:00:57,178][105692] Updated weights for policy 0, policy_version 1321790 (0.0005) [2023-12-27 01:00:57,233][105692] Updated weights for policy 0, policy_version 1321800 (0.0005) [2023-12-27 01:00:57,472][105620] Updated weights for policy 1, policy_version 1323689 (0.0006) [2023-12-27 01:00:57,517][105620] Updated weights for policy 1, policy_version 1323699 (0.0005) [2023-12-27 01:00:57,565][105620] Updated weights for policy 1, policy_version 1323709 (0.0006) [2023-12-27 01:00:57,956][105692] Updated weights for policy 0, policy_version 1321810 (0.0005) [2023-12-27 01:00:58,019][105692] Updated weights for policy 0, policy_version 1321820 (0.0008) [2023-12-27 01:00:58,085][105692] Updated weights for policy 0, policy_version 1321830 (0.0006) [2023-12-27 01:00:58,136][105692] Updated weights for policy 0, policy_version 1321840 (0.0005) [2023-12-27 01:00:58,213][105620] Updated weights for policy 1, policy_version 1323719 (0.0006) [2023-12-27 01:00:58,275][105620] Updated weights for policy 1, policy_version 1323729 (0.0006) [2023-12-27 01:00:58,346][105620] Updated weights for policy 1, policy_version 1323739 (0.0008) [2023-12-27 01:00:58,920][105692] Updated weights for policy 0, policy_version 1321850 (0.0008) [2023-12-27 01:00:58,986][105692] Updated weights for policy 0, policy_version 1321860 (0.0008) [2023-12-27 01:00:59,050][105692] Updated weights for policy 0, policy_version 1321870 (0.0008) [2023-12-27 01:00:59,200][105620] Updated weights for policy 1, policy_version 1323749 (0.0010) [2023-12-27 01:00:59,281][105620] Updated weights for policy 1, policy_version 1323759 (0.0009) [2023-12-27 01:00:59,344][105620] Updated weights for policy 1, policy_version 1323769 (0.0009) [2023-12-27 01:00:59,830][105692] Updated weights for policy 0, policy_version 1321880 (0.0009) [2023-12-27 01:00:59,895][105692] Updated weights for policy 0, policy_version 1321890 (0.0007) [2023-12-27 01:00:59,967][105692] Updated weights for policy 0, policy_version 1321900 (0.0008) [2023-12-27 01:01:00,147][105620] Updated weights for policy 1, policy_version 1323779 (0.0009) [2023-12-27 01:01:00,209][105620] Updated weights for policy 1, policy_version 1323789 (0.0008) [2023-12-27 01:01:00,270][105620] Updated weights for policy 1, policy_version 1323799 (0.0008) [2023-12-27 01:01:00,647][105692] Updated weights for policy 0, policy_version 1321910 (0.0008) [2023-12-27 01:01:00,699][105692] Updated weights for policy 0, policy_version 1321920 (0.0009) [2023-12-27 01:01:00,753][105692] Updated weights for policy 0, policy_version 1321930 (0.0009) [2023-12-27 01:01:00,983][105620] Updated weights for policy 1, policy_version 1323809 (0.0008) [2023-12-27 01:01:01,045][105620] Updated weights for policy 1, policy_version 1323819 (0.0007) [2023-12-27 01:01:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19355.3). Total num frames: 677404672. Throughput: 0: 9699.7, 1: 9609.3. Samples: 677379688. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:01:01,063][104569] Avg episode reward: [(0, '8727.360'), (1, '9182.891')] [2023-12-27 01:01:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001321936_338468864.pth... [2023-12-27 01:01:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001320784_338173952.pth [2023-12-27 01:01:01,107][105620] Updated weights for policy 1, policy_version 1323829 (0.0008) [2023-12-27 01:01:01,177][105620] Updated weights for policy 1, policy_version 1323839 (0.0008) [2023-12-27 01:01:01,179][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001323840_338944000.pth... [2023-12-27 01:01:01,183][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001322720_338657280.pth [2023-12-27 01:01:01,651][105692] Updated weights for policy 0, policy_version 1321940 (0.0009) [2023-12-27 01:01:01,714][105692] Updated weights for policy 0, policy_version 1321950 (0.0010) [2023-12-27 01:01:01,777][105692] Updated weights for policy 0, policy_version 1321960 (0.0009) [2023-12-27 01:01:01,867][105620] Updated weights for policy 1, policy_version 1323849 (0.0009) [2023-12-27 01:01:01,929][105620] Updated weights for policy 1, policy_version 1323859 (0.0009) [2023-12-27 01:01:01,984][105620] Updated weights for policy 1, policy_version 1323869 (0.0009) [2023-12-27 01:01:02,541][105692] Updated weights for policy 0, policy_version 1321970 (0.0009) [2023-12-27 01:01:02,593][105692] Updated weights for policy 0, policy_version 1321980 (0.0009) [2023-12-27 01:01:02,650][105692] Updated weights for policy 0, policy_version 1321990 (0.0009) [2023-12-27 01:01:02,709][105620] Updated weights for policy 1, policy_version 1323879 (0.0009) [2023-12-27 01:01:02,716][105692] Updated weights for policy 0, policy_version 1322000 (0.0008) [2023-12-27 01:01:02,758][105620] Updated weights for policy 1, policy_version 1323889 (0.0007) [2023-12-27 01:01:02,810][105620] Updated weights for policy 1, policy_version 1323899 (0.0007) [2023-12-27 01:01:03,362][105692] Updated weights for policy 0, policy_version 1322010 (0.0009) [2023-12-27 01:01:03,410][105692] Updated weights for policy 0, policy_version 1322020 (0.0009) [2023-12-27 01:01:03,468][105692] Updated weights for policy 0, policy_version 1322030 (0.0009) [2023-12-27 01:01:03,591][105620] Updated weights for policy 1, policy_version 1323909 (0.0009) [2023-12-27 01:01:03,653][105620] Updated weights for policy 1, policy_version 1323919 (0.0009) [2023-12-27 01:01:03,716][105620] Updated weights for policy 1, policy_version 1323929 (0.0009) [2023-12-27 01:01:04,280][105692] Updated weights for policy 0, policy_version 1322040 (0.0007) [2023-12-27 01:01:04,343][105692] Updated weights for policy 0, policy_version 1322050 (0.0007) [2023-12-27 01:01:04,407][105692] Updated weights for policy 0, policy_version 1322060 (0.0006) [2023-12-27 01:01:04,422][105620] Updated weights for policy 1, policy_version 1323939 (0.0009) [2023-12-27 01:01:04,496][105620] Updated weights for policy 1, policy_version 1323949 (0.0009) [2023-12-27 01:01:04,563][105620] Updated weights for policy 1, policy_version 1323959 (0.0008) [2023-12-27 01:01:05,101][105692] Updated weights for policy 0, policy_version 1322070 (0.0009) [2023-12-27 01:01:05,164][105692] Updated weights for policy 0, policy_version 1322080 (0.0010) [2023-12-27 01:01:05,235][105692] Updated weights for policy 0, policy_version 1322090 (0.0010) [2023-12-27 01:01:05,282][105620] Updated weights for policy 1, policy_version 1323969 (0.0009) [2023-12-27 01:01:05,339][105620] Updated weights for policy 1, policy_version 1323979 (0.0009) [2023-12-27 01:01:05,394][105620] Updated weights for policy 1, policy_version 1323989 (0.0009) [2023-12-27 01:01:05,447][105620] Updated weights for policy 1, policy_version 1323999 (0.0009) [2023-12-27 01:01:05,996][105692] Updated weights for policy 0, policy_version 1322100 (0.0009) [2023-12-27 01:01:06,060][105692] Updated weights for policy 0, policy_version 1322110 (0.0009) [2023-12-27 01:01:06,062][104569] Fps is (10 sec: 18023.0, 60 sec: 19114.8, 300 sec: 19355.3). Total num frames: 677494784. Throughput: 0: 9595.1, 1: 9566.9. Samples: 677489808. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:01:06,062][104569] Avg episode reward: [(0, '9000.286'), (1, '9271.773')] [2023-12-27 01:01:06,125][105692] Updated weights for policy 0, policy_version 1322120 (0.0010) [2023-12-27 01:01:06,166][105620] Updated weights for policy 1, policy_version 1324009 (0.0009) [2023-12-27 01:01:06,223][105620] Updated weights for policy 1, policy_version 1324019 (0.0008) [2023-12-27 01:01:06,292][105620] Updated weights for policy 1, policy_version 1324029 (0.0008) [2023-12-27 01:01:06,902][105692] Updated weights for policy 0, policy_version 1322130 (0.0011) [2023-12-27 01:01:06,950][105692] Updated weights for policy 0, policy_version 1322140 (0.0010) [2023-12-27 01:01:06,998][105692] Updated weights for policy 0, policy_version 1322150 (0.0010) [2023-12-27 01:01:07,019][105620] Updated weights for policy 1, policy_version 1324039 (0.0009) [2023-12-27 01:01:07,044][105692] Updated weights for policy 0, policy_version 1322160 (0.0010) [2023-12-27 01:01:07,068][105620] Updated weights for policy 1, policy_version 1324049 (0.0008) [2023-12-27 01:01:07,125][105620] Updated weights for policy 1, policy_version 1324059 (0.0008) [2023-12-27 01:01:07,856][105692] Updated weights for policy 0, policy_version 1322170 (0.0009) [2023-12-27 01:01:07,891][105620] Updated weights for policy 1, policy_version 1324069 (0.0007) [2023-12-27 01:01:07,915][105692] Updated weights for policy 0, policy_version 1322180 (0.0007) [2023-12-27 01:01:07,938][105620] Updated weights for policy 1, policy_version 1324079 (0.0007) [2023-12-27 01:01:07,968][105692] Updated weights for policy 0, policy_version 1322190 (0.0008) [2023-12-27 01:01:08,004][105620] Updated weights for policy 1, policy_version 1324089 (0.0009) [2023-12-27 01:01:08,674][105692] Updated weights for policy 0, policy_version 1322200 (0.0008) [2023-12-27 01:01:08,727][105692] Updated weights for policy 0, policy_version 1322210 (0.0008) [2023-12-27 01:01:08,765][105620] Updated weights for policy 1, policy_version 1324099 (0.0007) [2023-12-27 01:01:08,787][105692] Updated weights for policy 0, policy_version 1322220 (0.0008) [2023-12-27 01:01:08,822][105620] Updated weights for policy 1, policy_version 1324109 (0.0007) [2023-12-27 01:01:08,870][105620] Updated weights for policy 1, policy_version 1324119 (0.0006) [2023-12-27 01:01:09,525][105620] Updated weights for policy 1, policy_version 1324129 (0.0006) [2023-12-27 01:01:09,593][105620] Updated weights for policy 1, policy_version 1324139 (0.0008) [2023-12-27 01:01:09,641][105692] Updated weights for policy 0, policy_version 1322230 (0.0007) [2023-12-27 01:01:09,655][105620] Updated weights for policy 1, policy_version 1324149 (0.0008) [2023-12-27 01:01:09,690][105692] Updated weights for policy 0, policy_version 1322240 (0.0006) [2023-12-27 01:01:09,722][105620] Updated weights for policy 1, policy_version 1324159 (0.0008) [2023-12-27 01:01:09,741][105692] Updated weights for policy 0, policy_version 1322250 (0.0006) [2023-12-27 01:01:10,429][105620] Updated weights for policy 1, policy_version 1324169 (0.0009) [2023-12-27 01:01:10,450][105692] Updated weights for policy 0, policy_version 1322260 (0.0008) [2023-12-27 01:01:10,486][105620] Updated weights for policy 1, policy_version 1324179 (0.0011) [2023-12-27 01:01:10,499][105692] Updated weights for policy 0, policy_version 1322270 (0.0011) [2023-12-27 01:01:10,543][105620] Updated weights for policy 1, policy_version 1324189 (0.0011) [2023-12-27 01:01:10,559][105692] Updated weights for policy 0, policy_version 1322280 (0.0010) [2023-12-27 01:01:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19114.7, 300 sec: 19355.4). Total num frames: 677593088. Throughput: 0: 9652.7, 1: 9553.2. Samples: 677603700. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:01:11,062][104569] Avg episode reward: [(0, '9178.635'), (1, '8880.604')] [2023-12-27 01:01:11,309][105620] Updated weights for policy 1, policy_version 1324199 (0.0009) [2023-12-27 01:01:11,350][105692] Updated weights for policy 0, policy_version 1322290 (0.0010) [2023-12-27 01:01:11,387][105620] Updated weights for policy 1, policy_version 1324209 (0.0009) [2023-12-27 01:01:11,417][105692] Updated weights for policy 0, policy_version 1322300 (0.0011) [2023-12-27 01:01:11,452][105620] Updated weights for policy 1, policy_version 1324219 (0.0007) [2023-12-27 01:01:11,482][105692] Updated weights for policy 0, policy_version 1322310 (0.0011) [2023-12-27 01:01:11,538][105692] Updated weights for policy 0, policy_version 1322320 (0.0011) [2023-12-27 01:01:12,228][105620] Updated weights for policy 1, policy_version 1324229 (0.0008) [2023-12-27 01:01:12,295][105620] Updated weights for policy 1, policy_version 1324239 (0.0012) [2023-12-27 01:01:12,316][105692] Updated weights for policy 0, policy_version 1322330 (0.0011) [2023-12-27 01:01:12,366][105620] Updated weights for policy 1, policy_version 1324249 (0.0010) [2023-12-27 01:01:12,379][105692] Updated weights for policy 0, policy_version 1322340 (0.0010) [2023-12-27 01:01:12,441][105692] Updated weights for policy 0, policy_version 1322350 (0.0010) [2023-12-27 01:01:13,100][105620] Updated weights for policy 1, policy_version 1324259 (0.0009) [2023-12-27 01:01:13,133][105692] Updated weights for policy 0, policy_version 1322360 (0.0011) [2023-12-27 01:01:13,162][105620] Updated weights for policy 1, policy_version 1324269 (0.0011) [2023-12-27 01:01:13,182][105692] Updated weights for policy 0, policy_version 1322370 (0.0011) [2023-12-27 01:01:13,229][105620] Updated weights for policy 1, policy_version 1324279 (0.0011) [2023-12-27 01:01:13,231][105692] Updated weights for policy 0, policy_version 1322380 (0.0011) [2023-12-27 01:01:13,956][105692] Updated weights for policy 0, policy_version 1322390 (0.0008) [2023-12-27 01:01:13,961][105620] Updated weights for policy 1, policy_version 1324289 (0.0010) [2023-12-27 01:01:14,015][105692] Updated weights for policy 0, policy_version 1322400 (0.0005) [2023-12-27 01:01:14,019][105620] Updated weights for policy 1, policy_version 1324299 (0.0010) [2023-12-27 01:01:14,067][105620] Updated weights for policy 1, policy_version 1324309 (0.0010) [2023-12-27 01:01:14,072][105692] Updated weights for policy 0, policy_version 1322410 (0.0005) [2023-12-27 01:01:14,123][105620] Updated weights for policy 1, policy_version 1324319 (0.0011) [2023-12-27 01:01:14,634][105692] Updated weights for policy 0, policy_version 1322420 (0.0006) [2023-12-27 01:01:14,693][105692] Updated weights for policy 0, policy_version 1322430 (0.0008) [2023-12-27 01:01:14,745][105692] Updated weights for policy 0, policy_version 1322440 (0.0008) [2023-12-27 01:01:14,896][105620] Updated weights for policy 1, policy_version 1324329 (0.0011) [2023-12-27 01:01:14,949][105620] Updated weights for policy 1, policy_version 1324339 (0.0011) [2023-12-27 01:01:15,010][105620] Updated weights for policy 1, policy_version 1324349 (0.0011) [2023-12-27 01:01:15,367][105692] Updated weights for policy 0, policy_version 1322450 (0.0007) [2023-12-27 01:01:15,431][105692] Updated weights for policy 0, policy_version 1322460 (0.0008) [2023-12-27 01:01:15,493][105692] Updated weights for policy 0, policy_version 1322470 (0.0008) [2023-12-27 01:01:15,560][105692] Updated weights for policy 0, policy_version 1322480 (0.0006) [2023-12-27 01:01:15,773][105620] Updated weights for policy 1, policy_version 1324359 (0.0010) [2023-12-27 01:01:15,831][105620] Updated weights for policy 1, policy_version 1324369 (0.0010) [2023-12-27 01:01:15,878][105620] Updated weights for policy 1, policy_version 1324379 (0.0010) [2023-12-27 01:01:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19114.7, 300 sec: 19355.3). Total num frames: 677691392. Throughput: 0: 9620.4, 1: 9496.4. Samples: 677658544. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:01:16,062][104569] Avg episode reward: [(0, '8903.630'), (1, '8962.288')] [2023-12-27 01:01:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001324384_339083264.pth... [2023-12-27 01:01:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001322480_338608128.pth... [2023-12-27 01:01:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001323296_338804736.pth [2023-12-27 01:01:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001321360_338321408.pth [2023-12-27 01:01:16,274][105692] Updated weights for policy 0, policy_version 1322490 (0.0008) [2023-12-27 01:01:16,330][105692] Updated weights for policy 0, policy_version 1322500 (0.0008) [2023-12-27 01:01:16,386][105692] Updated weights for policy 0, policy_version 1322510 (0.0008) [2023-12-27 01:01:16,624][105620] Updated weights for policy 1, policy_version 1324389 (0.0010) [2023-12-27 01:01:16,677][105620] Updated weights for policy 1, policy_version 1324399 (0.0010) [2023-12-27 01:01:16,733][105620] Updated weights for policy 1, policy_version 1324409 (0.0011) [2023-12-27 01:01:17,111][105692] Updated weights for policy 0, policy_version 1322520 (0.0007) [2023-12-27 01:01:17,163][105692] Updated weights for policy 0, policy_version 1322530 (0.0008) [2023-12-27 01:01:17,211][105692] Updated weights for policy 0, policy_version 1322540 (0.0008) [2023-12-27 01:01:17,497][105620] Updated weights for policy 1, policy_version 1324419 (0.0010) [2023-12-27 01:01:17,552][105620] Updated weights for policy 1, policy_version 1324429 (0.0010) [2023-12-27 01:01:17,604][105620] Updated weights for policy 1, policy_version 1324439 (0.0010) [2023-12-27 01:01:17,966][105692] Updated weights for policy 0, policy_version 1322550 (0.0008) [2023-12-27 01:01:18,026][105692] Updated weights for policy 0, policy_version 1322560 (0.0008) [2023-12-27 01:01:18,074][105692] Updated weights for policy 0, policy_version 1322570 (0.0008) [2023-12-27 01:01:18,298][105620] Updated weights for policy 1, policy_version 1324449 (0.0010) [2023-12-27 01:01:18,363][105620] Updated weights for policy 1, policy_version 1324459 (0.0011) [2023-12-27 01:01:18,427][105620] Updated weights for policy 1, policy_version 1324469 (0.0011) [2023-12-27 01:01:18,485][105620] Updated weights for policy 1, policy_version 1324479 (0.0010) [2023-12-27 01:01:18,855][105692] Updated weights for policy 0, policy_version 1322580 (0.0008) [2023-12-27 01:01:18,915][105692] Updated weights for policy 0, policy_version 1322590 (0.0008) [2023-12-27 01:01:18,967][105692] Updated weights for policy 0, policy_version 1322600 (0.0008) [2023-12-27 01:01:19,166][105620] Updated weights for policy 1, policy_version 1324489 (0.0011) [2023-12-27 01:01:19,236][105620] Updated weights for policy 1, policy_version 1324499 (0.0010) [2023-12-27 01:01:19,291][105620] Updated weights for policy 1, policy_version 1324509 (0.0010) [2023-12-27 01:01:19,763][105692] Updated weights for policy 0, policy_version 1322610 (0.0008) [2023-12-27 01:01:19,829][105692] Updated weights for policy 0, policy_version 1322620 (0.0009) [2023-12-27 01:01:19,901][105692] Updated weights for policy 0, policy_version 1322630 (0.0010) [2023-12-27 01:01:19,967][105692] Updated weights for policy 0, policy_version 1322640 (0.0009) [2023-12-27 01:01:20,007][105620] Updated weights for policy 1, policy_version 1324519 (0.0006) [2023-12-27 01:01:20,061][105620] Updated weights for policy 1, policy_version 1324529 (0.0008) [2023-12-27 01:01:20,113][105620] Updated weights for policy 1, policy_version 1324539 (0.0007) [2023-12-27 01:01:20,714][105692] Updated weights for policy 0, policy_version 1322650 (0.0009) [2023-12-27 01:01:20,766][105692] Updated weights for policy 0, policy_version 1322660 (0.0009) [2023-12-27 01:01:20,828][105692] Updated weights for policy 0, policy_version 1322670 (0.0008) [2023-12-27 01:01:20,868][105620] Updated weights for policy 1, policy_version 1324549 (0.0008) [2023-12-27 01:01:20,920][105620] Updated weights for policy 1, policy_version 1324559 (0.0008) [2023-12-27 01:01:20,968][105620] Updated weights for policy 1, policy_version 1324569 (0.0009) [2023-12-27 01:01:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19114.7, 300 sec: 19383.1). Total num frames: 677789696. Throughput: 0: 9600.8, 1: 9454.7. Samples: 677775268. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:01:21,063][104569] Avg episode reward: [(0, '8455.275'), (1, '9086.913')] [2023-12-27 01:01:21,646][105692] Updated weights for policy 0, policy_version 1322680 (0.0008) [2023-12-27 01:01:21,710][105692] Updated weights for policy 0, policy_version 1322690 (0.0009) [2023-12-27 01:01:21,783][105692] Updated weights for policy 0, policy_version 1322700 (0.0009) [2023-12-27 01:01:21,801][105620] Updated weights for policy 1, policy_version 1324579 (0.0009) [2023-12-27 01:01:21,870][105620] Updated weights for policy 1, policy_version 1324589 (0.0010) [2023-12-27 01:01:21,932][105620] Updated weights for policy 1, policy_version 1324599 (0.0008) [2023-12-27 01:01:22,562][105692] Updated weights for policy 0, policy_version 1322710 (0.0010) [2023-12-27 01:01:22,632][105692] Updated weights for policy 0, policy_version 1322720 (0.0011) [2023-12-27 01:01:22,648][105620] Updated weights for policy 1, policy_version 1324609 (0.0009) [2023-12-27 01:01:22,695][105692] Updated weights for policy 0, policy_version 1322730 (0.0011) [2023-12-27 01:01:22,713][105620] Updated weights for policy 1, policy_version 1324619 (0.0006) [2023-12-27 01:01:22,778][105620] Updated weights for policy 1, policy_version 1324629 (0.0006) [2023-12-27 01:01:22,843][105620] Updated weights for policy 1, policy_version 1324639 (0.0005) [2023-12-27 01:01:23,339][105692] Updated weights for policy 0, policy_version 1322740 (0.0011) [2023-12-27 01:01:23,399][105692] Updated weights for policy 0, policy_version 1322750 (0.0011) [2023-12-27 01:01:23,427][105585] KL-divergence is very high: 114.7688 [2023-12-27 01:01:23,452][105692] Updated weights for policy 0, policy_version 1322760 (0.0010) [2023-12-27 01:01:23,466][105585] KL-divergence is very high: 103.2322 [2023-12-27 01:01:23,494][105620] Updated weights for policy 1, policy_version 1324649 (0.0007) [2023-12-27 01:01:23,559][105620] Updated weights for policy 1, policy_version 1324659 (0.0008) [2023-12-27 01:01:23,610][105620] Updated weights for policy 1, policy_version 1324669 (0.0008) [2023-12-27 01:01:24,150][105692] Updated weights for policy 0, policy_version 1322770 (0.0011) [2023-12-27 01:01:24,211][105692] Updated weights for policy 0, policy_version 1322780 (0.0010) [2023-12-27 01:01:24,276][105692] Updated weights for policy 0, policy_version 1322790 (0.0010) [2023-12-27 01:01:24,334][105692] Updated weights for policy 0, policy_version 1322800 (0.0009) [2023-12-27 01:01:24,357][105620] Updated weights for policy 1, policy_version 1324679 (0.0010) [2023-12-27 01:01:24,411][105620] Updated weights for policy 1, policy_version 1324689 (0.0010) [2023-12-27 01:01:24,464][105620] Updated weights for policy 1, policy_version 1324699 (0.0010) [2023-12-27 01:01:25,011][105692] Updated weights for policy 0, policy_version 1322810 (0.0008) [2023-12-27 01:01:25,062][105692] Updated weights for policy 0, policy_version 1322820 (0.0010) [2023-12-27 01:01:25,120][105692] Updated weights for policy 0, policy_version 1322830 (0.0010) [2023-12-27 01:01:25,213][105620] Updated weights for policy 1, policy_version 1324709 (0.0009) [2023-12-27 01:01:25,268][105620] Updated weights for policy 1, policy_version 1324719 (0.0008) [2023-12-27 01:01:25,322][105620] Updated weights for policy 1, policy_version 1324729 (0.0007) [2023-12-27 01:01:25,829][105692] Updated weights for policy 0, policy_version 1322840 (0.0006) [2023-12-27 01:01:25,892][105692] Updated weights for policy 0, policy_version 1322851 (0.0009) [2023-12-27 01:01:25,940][105692] Updated weights for policy 0, policy_version 1322861 (0.0009) [2023-12-27 01:01:26,015][105620] Updated weights for policy 1, policy_version 1324739 (0.0009) [2023-12-27 01:01:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19327.6). Total num frames: 677879808. Throughput: 0: 9595.7, 1: 9446.3. Samples: 677889320. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:01:26,062][104569] Avg episode reward: [(0, '8452.290'), (1, '8997.223')] [2023-12-27 01:01:26,080][105620] Updated weights for policy 1, policy_version 1324749 (0.0009) [2023-12-27 01:01:26,134][105620] Updated weights for policy 1, policy_version 1324759 (0.0008) [2023-12-27 01:01:26,564][105692] Updated weights for policy 0, policy_version 1322871 (0.0008) [2023-12-27 01:01:26,613][105692] Updated weights for policy 0, policy_version 1322881 (0.0008) [2023-12-27 01:01:26,662][105692] Updated weights for policy 0, policy_version 1322891 (0.0008) [2023-12-27 01:01:26,796][105620] Updated weights for policy 1, policy_version 1324769 (0.0005) [2023-12-27 01:01:26,841][105620] Updated weights for policy 1, policy_version 1324779 (0.0005) [2023-12-27 01:01:26,887][105620] Updated weights for policy 1, policy_version 1324789 (0.0005) [2023-12-27 01:01:26,929][105620] Updated weights for policy 1, policy_version 1324799 (0.0005) [2023-12-27 01:01:27,326][105692] Updated weights for policy 0, policy_version 1322901 (0.0007) [2023-12-27 01:01:27,385][105692] Updated weights for policy 0, policy_version 1322911 (0.0005) [2023-12-27 01:01:27,448][105692] Updated weights for policy 0, policy_version 1322921 (0.0005) [2023-12-27 01:01:27,655][105620] Updated weights for policy 1, policy_version 1324809 (0.0007) [2023-12-27 01:01:27,703][105620] Updated weights for policy 1, policy_version 1324819 (0.0008) [2023-12-27 01:01:27,753][105620] Updated weights for policy 1, policy_version 1324829 (0.0008) [2023-12-27 01:01:28,023][105692] Updated weights for policy 0, policy_version 1322931 (0.0005) [2023-12-27 01:01:28,066][105692] Updated weights for policy 0, policy_version 1322941 (0.0005) [2023-12-27 01:01:28,111][105692] Updated weights for policy 0, policy_version 1322951 (0.0007) [2023-12-27 01:01:28,538][105620] Updated weights for policy 1, policy_version 1324839 (0.0006) [2023-12-27 01:01:28,593][105620] Updated weights for policy 1, policy_version 1324849 (0.0005) [2023-12-27 01:01:28,649][105620] Updated weights for policy 1, policy_version 1324859 (0.0005) [2023-12-27 01:01:28,888][105692] Updated weights for policy 0, policy_version 1322961 (0.0010) [2023-12-27 01:01:28,942][105692] Updated weights for policy 0, policy_version 1322971 (0.0009) [2023-12-27 01:01:29,004][105692] Updated weights for policy 0, policy_version 1322981 (0.0009) [2023-12-27 01:01:29,062][105692] Updated weights for policy 0, policy_version 1322991 (0.0010) [2023-12-27 01:01:29,292][105620] Updated weights for policy 1, policy_version 1324869 (0.0008) [2023-12-27 01:01:29,357][105620] Updated weights for policy 1, policy_version 1324879 (0.0009) [2023-12-27 01:01:29,426][105620] Updated weights for policy 1, policy_version 1324889 (0.0006) [2023-12-27 01:01:29,899][105692] Updated weights for policy 0, policy_version 1323001 (0.0007) [2023-12-27 01:01:29,961][105692] Updated weights for policy 0, policy_version 1323011 (0.0009) [2023-12-27 01:01:30,019][105692] Updated weights for policy 0, policy_version 1323021 (0.0009) [2023-12-27 01:01:30,054][105620] Updated weights for policy 1, policy_version 1324899 (0.0008) [2023-12-27 01:01:30,111][105620] Updated weights for policy 1, policy_version 1324909 (0.0009) [2023-12-27 01:01:30,170][105620] Updated weights for policy 1, policy_version 1324919 (0.0010) [2023-12-27 01:01:30,785][105620] Updated weights for policy 1, policy_version 1324929 (0.0009) [2023-12-27 01:01:30,796][105692] Updated weights for policy 0, policy_version 1323031 (0.0009) [2023-12-27 01:01:30,839][105620] Updated weights for policy 1, policy_version 1324939 (0.0005) [2023-12-27 01:01:30,846][105692] Updated weights for policy 0, policy_version 1323041 (0.0008) [2023-12-27 01:01:30,892][105620] Updated weights for policy 1, policy_version 1324949 (0.0005) [2023-12-27 01:01:30,895][105692] Updated weights for policy 0, policy_version 1323051 (0.0008) [2023-12-27 01:01:30,946][105620] Updated weights for policy 1, policy_version 1324959 (0.0005) [2023-12-27 01:01:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 677986304. Throughput: 0: 9689.1, 1: 9479.3. Samples: 677951428. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:01:31,062][104569] Avg episode reward: [(0, '8540.427'), (1, '8914.149')] [2023-12-27 01:01:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001323056_338755584.pth... [2023-12-27 01:01:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001324960_339230720.pth... [2023-12-27 01:01:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001321936_338468864.pth [2023-12-27 01:01:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001323840_338944000.pth [2023-12-27 01:01:31,547][105620] Updated weights for policy 1, policy_version 1324969 (0.0006) [2023-12-27 01:01:31,615][105620] Updated weights for policy 1, policy_version 1324979 (0.0006) [2023-12-27 01:01:31,677][105620] Updated weights for policy 1, policy_version 1324989 (0.0006) [2023-12-27 01:01:31,775][105692] Updated weights for policy 0, policy_version 1323061 (0.0008) [2023-12-27 01:01:31,834][105692] Updated weights for policy 0, policy_version 1323071 (0.0005) [2023-12-27 01:01:31,885][105692] Updated weights for policy 0, policy_version 1323081 (0.0005) [2023-12-27 01:01:32,292][105620] Updated weights for policy 1, policy_version 1324999 (0.0007) [2023-12-27 01:01:32,359][105620] Updated weights for policy 1, policy_version 1325009 (0.0008) [2023-12-27 01:01:32,429][105620] Updated weights for policy 1, policy_version 1325019 (0.0006) [2023-12-27 01:01:32,508][105692] Updated weights for policy 0, policy_version 1323091 (0.0006) [2023-12-27 01:01:32,578][105692] Updated weights for policy 0, policy_version 1323101 (0.0006) [2023-12-27 01:01:32,642][105692] Updated weights for policy 0, policy_version 1323111 (0.0007) [2023-12-27 01:01:33,006][105620] Updated weights for policy 1, policy_version 1325029 (0.0008) [2023-12-27 01:01:33,063][105620] Updated weights for policy 1, policy_version 1325039 (0.0010) [2023-12-27 01:01:33,129][105620] Updated weights for policy 1, policy_version 1325049 (0.0010) [2023-12-27 01:01:33,210][105692] Updated weights for policy 0, policy_version 1323121 (0.0011) [2023-12-27 01:01:33,257][105692] Updated weights for policy 0, policy_version 1323131 (0.0010) [2023-12-27 01:01:33,311][105692] Updated weights for policy 0, policy_version 1323141 (0.0010) [2023-12-27 01:01:33,366][105692] Updated weights for policy 0, policy_version 1323151 (0.0010) [2023-12-27 01:01:33,824][105620] Updated weights for policy 1, policy_version 1325059 (0.0008) [2023-12-27 01:01:33,868][105620] Updated weights for policy 1, policy_version 1325069 (0.0005) [2023-12-27 01:01:33,913][105620] Updated weights for policy 1, policy_version 1325079 (0.0005) [2023-12-27 01:01:33,994][105692] Updated weights for policy 0, policy_version 1323161 (0.0006) [2023-12-27 01:01:34,054][105692] Updated weights for policy 0, policy_version 1323171 (0.0005) [2023-12-27 01:01:34,121][105692] Updated weights for policy 0, policy_version 1323181 (0.0008) [2023-12-27 01:01:34,592][105620] Updated weights for policy 1, policy_version 1325089 (0.0007) [2023-12-27 01:01:34,655][105620] Updated weights for policy 1, policy_version 1325099 (0.0011) [2023-12-27 01:01:34,718][105620] Updated weights for policy 1, policy_version 1325109 (0.0011) [2023-12-27 01:01:34,747][105692] Updated weights for policy 0, policy_version 1323191 (0.0010) [2023-12-27 01:01:34,778][105620] Updated weights for policy 1, policy_version 1325119 (0.0011) [2023-12-27 01:01:34,799][105692] Updated weights for policy 0, policy_version 1323201 (0.0011) [2023-12-27 01:01:34,858][105692] Updated weights for policy 0, policy_version 1323211 (0.0011) [2023-12-27 01:01:35,496][105620] Updated weights for policy 1, policy_version 1325129 (0.0010) [2023-12-27 01:01:35,554][105620] Updated weights for policy 1, policy_version 1325139 (0.0010) [2023-12-27 01:01:35,617][105620] Updated weights for policy 1, policy_version 1325149 (0.0011) [2023-12-27 01:01:35,619][105692] Updated weights for policy 0, policy_version 1323221 (0.0011) [2023-12-27 01:01:35,682][105692] Updated weights for policy 0, policy_version 1323231 (0.0011) [2023-12-27 01:01:35,734][105692] Updated weights for policy 0, policy_version 1323241 (0.0011) [2023-12-27 01:01:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 678084608. Throughput: 0: 9599.5, 1: 9603.0. Samples: 678074004. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:01:36,062][104569] Avg episode reward: [(0, '8631.797'), (1, '9096.301')] [2023-12-27 01:01:36,377][105620] Updated weights for policy 1, policy_version 1325159 (0.0011) [2023-12-27 01:01:36,440][105620] Updated weights for policy 1, policy_version 1325169 (0.0010) [2023-12-27 01:01:36,497][105692] Updated weights for policy 0, policy_version 1323251 (0.0011) [2023-12-27 01:01:36,500][105620] Updated weights for policy 1, policy_version 1325179 (0.0011) [2023-12-27 01:01:36,558][105692] Updated weights for policy 0, policy_version 1323261 (0.0011) [2023-12-27 01:01:36,611][105692] Updated weights for policy 0, policy_version 1323271 (0.0011) [2023-12-27 01:01:37,227][105620] Updated weights for policy 1, policy_version 1325189 (0.0011) [2023-12-27 01:01:37,283][105620] Updated weights for policy 1, policy_version 1325199 (0.0010) [2023-12-27 01:01:37,339][105620] Updated weights for policy 1, policy_version 1325209 (0.0010) [2023-12-27 01:01:37,340][105692] Updated weights for policy 0, policy_version 1323281 (0.0011) [2023-12-27 01:01:37,399][105692] Updated weights for policy 0, policy_version 1323291 (0.0006) [2023-12-27 01:01:37,458][105692] Updated weights for policy 0, policy_version 1323301 (0.0008) [2023-12-27 01:01:37,514][105692] Updated weights for policy 0, policy_version 1323311 (0.0008) [2023-12-27 01:01:38,023][105620] Updated weights for policy 1, policy_version 1325219 (0.0008) [2023-12-27 01:01:38,085][105620] Updated weights for policy 1, policy_version 1325229 (0.0010) [2023-12-27 01:01:38,106][105586] KL-divergence is very high: 141.8727 [2023-12-27 01:01:38,142][105620] Updated weights for policy 1, policy_version 1325239 (0.0009) [2023-12-27 01:01:38,151][105586] KL-divergence is very high: 132.7232 [2023-12-27 01:01:38,255][105692] Updated weights for policy 0, policy_version 1323321 (0.0007) [2023-12-27 01:01:38,317][105692] Updated weights for policy 0, policy_version 1323331 (0.0009) [2023-12-27 01:01:38,382][105692] Updated weights for policy 0, policy_version 1323341 (0.0009) [2023-12-27 01:01:38,859][105620] Updated weights for policy 1, policy_version 1325249 (0.0009) [2023-12-27 01:01:38,915][105620] Updated weights for policy 1, policy_version 1325259 (0.0005) [2023-12-27 01:01:38,977][105620] Updated weights for policy 1, policy_version 1325269 (0.0005) [2023-12-27 01:01:39,045][105620] Updated weights for policy 1, policy_version 1325279 (0.0005) [2023-12-27 01:01:39,114][105692] Updated weights for policy 0, policy_version 1323351 (0.0007) [2023-12-27 01:01:39,164][105692] Updated weights for policy 0, policy_version 1323361 (0.0006) [2023-12-27 01:01:39,217][105692] Updated weights for policy 0, policy_version 1323371 (0.0008) [2023-12-27 01:01:39,661][105620] Updated weights for policy 1, policy_version 1325289 (0.0007) [2023-12-27 01:01:39,725][105620] Updated weights for policy 1, policy_version 1325299 (0.0007) [2023-12-27 01:01:39,787][105620] Updated weights for policy 1, policy_version 1325309 (0.0008) [2023-12-27 01:01:39,905][105692] Updated weights for policy 0, policy_version 1323381 (0.0009) [2023-12-27 01:01:39,973][105692] Updated weights for policy 0, policy_version 1323391 (0.0009) [2023-12-27 01:01:40,040][105692] Updated weights for policy 0, policy_version 1323401 (0.0010) [2023-12-27 01:01:40,421][105620] Updated weights for policy 1, policy_version 1325319 (0.0008) [2023-12-27 01:01:40,484][105620] Updated weights for policy 1, policy_version 1325329 (0.0009) [2023-12-27 01:01:40,548][105620] Updated weights for policy 1, policy_version 1325339 (0.0008) [2023-12-27 01:01:40,858][105692] Updated weights for policy 0, policy_version 1323411 (0.0009) [2023-12-27 01:01:40,910][105692] Updated weights for policy 0, policy_version 1323421 (0.0010) [2023-12-27 01:01:40,970][105692] Updated weights for policy 0, policy_version 1323431 (0.0008) [2023-12-27 01:01:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 678182912. Throughput: 0: 9576.4, 1: 9728.5. Samples: 678189500. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:01:41,062][104569] Avg episode reward: [(0, '8539.114'), (1, '9269.688')] [2023-12-27 01:01:41,243][105620] Updated weights for policy 1, policy_version 1325349 (0.0008) [2023-12-27 01:01:41,310][105620] Updated weights for policy 1, policy_version 1325359 (0.0009) [2023-12-27 01:01:41,377][105620] Updated weights for policy 1, policy_version 1325369 (0.0009) [2023-12-27 01:01:41,840][105692] Updated weights for policy 0, policy_version 1323441 (0.0009) [2023-12-27 01:01:41,895][105692] Updated weights for policy 0, policy_version 1323451 (0.0006) [2023-12-27 01:01:41,951][105692] Updated weights for policy 0, policy_version 1323461 (0.0009) [2023-12-27 01:01:42,020][105692] Updated weights for policy 0, policy_version 1323471 (0.0009) [2023-12-27 01:01:42,094][105620] Updated weights for policy 1, policy_version 1325379 (0.0008) [2023-12-27 01:01:42,156][105620] Updated weights for policy 1, policy_version 1325389 (0.0009) [2023-12-27 01:01:42,223][105620] Updated weights for policy 1, policy_version 1325399 (0.0009) [2023-12-27 01:01:42,773][105692] Updated weights for policy 0, policy_version 1323481 (0.0009) [2023-12-27 01:01:42,828][105692] Updated weights for policy 0, policy_version 1323491 (0.0008) [2023-12-27 01:01:42,891][105692] Updated weights for policy 0, policy_version 1323501 (0.0009) [2023-12-27 01:01:42,968][105620] Updated weights for policy 1, policy_version 1325409 (0.0009) [2023-12-27 01:01:43,023][105620] Updated weights for policy 1, policy_version 1325419 (0.0009) [2023-12-27 01:01:43,070][105620] Updated weights for policy 1, policy_version 1325429 (0.0009) [2023-12-27 01:01:43,128][105620] Updated weights for policy 1, policy_version 1325439 (0.0008) [2023-12-27 01:01:43,662][105692] Updated weights for policy 0, policy_version 1323511 (0.0009) [2023-12-27 01:01:43,728][105692] Updated weights for policy 0, policy_version 1323521 (0.0010) [2023-12-27 01:01:43,786][105692] Updated weights for policy 0, policy_version 1323531 (0.0009) [2023-12-27 01:01:43,843][105620] Updated weights for policy 1, policy_version 1325449 (0.0008) [2023-12-27 01:01:43,895][105620] Updated weights for policy 1, policy_version 1325459 (0.0009) [2023-12-27 01:01:43,947][105620] Updated weights for policy 1, policy_version 1325469 (0.0009) [2023-12-27 01:01:44,451][105692] Updated weights for policy 0, policy_version 1323541 (0.0009) [2023-12-27 01:01:44,503][105692] Updated weights for policy 0, policy_version 1323552 (0.0010) [2023-12-27 01:01:44,550][105692] Updated weights for policy 0, policy_version 1323562 (0.0008) [2023-12-27 01:01:44,705][105620] Updated weights for policy 1, policy_version 1325479 (0.0009) [2023-12-27 01:01:44,762][105620] Updated weights for policy 1, policy_version 1325489 (0.0008) [2023-12-27 01:01:44,824][105620] Updated weights for policy 1, policy_version 1325499 (0.0009) [2023-12-27 01:01:45,339][105692] Updated weights for policy 0, policy_version 1323572 (0.0009) [2023-12-27 01:01:45,393][105692] Updated weights for policy 0, policy_version 1323582 (0.0009) [2023-12-27 01:01:45,447][105692] Updated weights for policy 0, policy_version 1323592 (0.0010) [2023-12-27 01:01:45,537][105620] Updated weights for policy 1, policy_version 1325509 (0.0008) [2023-12-27 01:01:45,599][105620] Updated weights for policy 1, policy_version 1325519 (0.0009) [2023-12-27 01:01:45,658][105620] Updated weights for policy 1, policy_version 1325529 (0.0009) [2023-12-27 01:01:46,062][104569] Fps is (10 sec: 18840.7, 60 sec: 19251.1, 300 sec: 19355.3). Total num frames: 678273024. Throughput: 0: 9514.5, 1: 9713.1. Samples: 678244936. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:01:46,063][104569] Avg episode reward: [(0, '8809.782'), (1, '9180.497')] [2023-12-27 01:01:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001325536_339378176.pth... [2023-12-27 01:01:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001324384_339083264.pth [2023-12-27 01:01:46,075][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_001325536_339378176.pth [2023-12-27 01:01:46,091][105692] Updated weights for policy 0, policy_version 1323602 (0.0010) [2023-12-27 01:01:46,153][105692] Updated weights for policy 0, policy_version 1323612 (0.0009) [2023-12-27 01:01:46,215][105692] Updated weights for policy 0, policy_version 1323622 (0.0009) [2023-12-27 01:01:46,256][105585] KL-divergence is very high: 113.4475 [2023-12-27 01:01:46,265][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001323632_338903040.pth... [2023-12-27 01:01:46,266][105692] Updated weights for policy 0, policy_version 1323632 (0.0009) [2023-12-27 01:01:46,268][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001322480_338608128.pth [2023-12-27 01:01:46,269][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_001323632_338903040.pth [2023-12-27 01:01:46,423][105620] Updated weights for policy 1, policy_version 1325539 (0.0009) [2023-12-27 01:01:46,479][105620] Updated weights for policy 1, policy_version 1325549 (0.0010) [2023-12-27 01:01:46,528][105620] Updated weights for policy 1, policy_version 1325559 (0.0008) [2023-12-27 01:01:46,971][105692] Updated weights for policy 0, policy_version 1323642 (0.0008) [2023-12-27 01:01:47,031][105692] Updated weights for policy 0, policy_version 1323652 (0.0009) [2023-12-27 01:01:47,092][105692] Updated weights for policy 0, policy_version 1323662 (0.0009) [2023-12-27 01:01:47,291][105620] Updated weights for policy 1, policy_version 1325569 (0.0009) [2023-12-27 01:01:47,341][105620] Updated weights for policy 1, policy_version 1325579 (0.0009) [2023-12-27 01:01:47,398][105620] Updated weights for policy 1, policy_version 1325589 (0.0009) [2023-12-27 01:01:47,461][105620] Updated weights for policy 1, policy_version 1325599 (0.0009) [2023-12-27 01:01:47,832][105692] Updated weights for policy 0, policy_version 1323672 (0.0009) [2023-12-27 01:01:47,889][105692] Updated weights for policy 0, policy_version 1323682 (0.0009) [2023-12-27 01:01:47,946][105692] Updated weights for policy 0, policy_version 1323692 (0.0008) [2023-12-27 01:01:48,214][105620] Updated weights for policy 1, policy_version 1325609 (0.0008) [2023-12-27 01:01:48,270][105620] Updated weights for policy 1, policy_version 1325619 (0.0011) [2023-12-27 01:01:48,319][105620] Updated weights for policy 1, policy_version 1325629 (0.0010) [2023-12-27 01:01:48,696][105692] Updated weights for policy 0, policy_version 1323702 (0.0009) [2023-12-27 01:01:48,755][105692] Updated weights for policy 0, policy_version 1323712 (0.0008) [2023-12-27 01:01:48,819][105692] Updated weights for policy 0, policy_version 1323722 (0.0008) [2023-12-27 01:01:49,042][105620] Updated weights for policy 1, policy_version 1325639 (0.0006) [2023-12-27 01:01:49,103][105620] Updated weights for policy 1, policy_version 1325649 (0.0005) [2023-12-27 01:01:49,167][105620] Updated weights for policy 1, policy_version 1325659 (0.0006) [2023-12-27 01:01:49,613][105692] Updated weights for policy 0, policy_version 1323732 (0.0008) [2023-12-27 01:01:49,678][105692] Updated weights for policy 0, policy_version 1323742 (0.0008) [2023-12-27 01:01:49,746][105692] Updated weights for policy 0, policy_version 1323752 (0.0008) [2023-12-27 01:01:49,877][105620] Updated weights for policy 1, policy_version 1325669 (0.0011) [2023-12-27 01:01:49,938][105620] Updated weights for policy 1, policy_version 1325679 (0.0011) [2023-12-27 01:01:50,001][105620] Updated weights for policy 1, policy_version 1325689 (0.0011) [2023-12-27 01:01:50,466][105692] Updated weights for policy 0, policy_version 1323762 (0.0008) [2023-12-27 01:01:50,529][105692] Updated weights for policy 0, policy_version 1323772 (0.0008) [2023-12-27 01:01:50,582][105692] Updated weights for policy 0, policy_version 1323782 (0.0008) [2023-12-27 01:01:50,643][105692] Updated weights for policy 0, policy_version 1323792 (0.0008) [2023-12-27 01:01:50,757][105620] Updated weights for policy 1, policy_version 1325699 (0.0010) [2023-12-27 01:01:50,810][105620] Updated weights for policy 1, policy_version 1325709 (0.0011) [2023-12-27 01:01:50,865][105620] Updated weights for policy 1, policy_version 1325719 (0.0010) [2023-12-27 01:01:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19355.3). Total num frames: 678371328. Throughput: 0: 9577.1, 1: 9740.3. Samples: 678359088. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:01:51,062][104569] Avg episode reward: [(0, '8541.294'), (1, '9088.925')] [2023-12-27 01:01:51,401][105692] Updated weights for policy 0, policy_version 1323802 (0.0009) [2023-12-27 01:01:51,457][105692] Updated weights for policy 0, policy_version 1323812 (0.0009) [2023-12-27 01:01:51,509][105692] Updated weights for policy 0, policy_version 1323822 (0.0008) [2023-12-27 01:01:51,622][105620] Updated weights for policy 1, policy_version 1325729 (0.0010) [2023-12-27 01:01:51,681][105620] Updated weights for policy 1, policy_version 1325739 (0.0008) [2023-12-27 01:01:51,739][105620] Updated weights for policy 1, policy_version 1325749 (0.0008) [2023-12-27 01:01:51,797][105620] Updated weights for policy 1, policy_version 1325759 (0.0010) [2023-12-27 01:01:52,244][105692] Updated weights for policy 0, policy_version 1323832 (0.0009) [2023-12-27 01:01:52,306][105692] Updated weights for policy 0, policy_version 1323842 (0.0007) [2023-12-27 01:01:52,373][105692] Updated weights for policy 0, policy_version 1323852 (0.0009) [2023-12-27 01:01:52,596][105620] Updated weights for policy 1, policy_version 1325769 (0.0009) [2023-12-27 01:01:52,650][105620] Updated weights for policy 1, policy_version 1325779 (0.0010) [2023-12-27 01:01:52,707][105620] Updated weights for policy 1, policy_version 1325789 (0.0009) [2023-12-27 01:01:53,096][105692] Updated weights for policy 0, policy_version 1323862 (0.0009) [2023-12-27 01:01:53,155][105692] Updated weights for policy 0, policy_version 1323872 (0.0010) [2023-12-27 01:01:53,208][105692] Updated weights for policy 0, policy_version 1323882 (0.0010) [2023-12-27 01:01:53,364][105620] Updated weights for policy 1, policy_version 1325799 (0.0009) [2023-12-27 01:01:53,431][105620] Updated weights for policy 1, policy_version 1325809 (0.0009) [2023-12-27 01:01:53,493][105620] Updated weights for policy 1, policy_version 1325819 (0.0007) [2023-12-27 01:01:54,058][105692] Updated weights for policy 0, policy_version 1323892 (0.0009) [2023-12-27 01:01:54,105][105620] Updated weights for policy 1, policy_version 1325829 (0.0008) [2023-12-27 01:01:54,111][105692] Updated weights for policy 0, policy_version 1323902 (0.0007) [2023-12-27 01:01:54,164][105620] Updated weights for policy 1, policy_version 1325839 (0.0008) [2023-12-27 01:01:54,175][105692] Updated weights for policy 0, policy_version 1323912 (0.0009) [2023-12-27 01:01:54,211][105620] Updated weights for policy 1, policy_version 1325849 (0.0007) [2023-12-27 01:01:54,906][105692] Updated weights for policy 0, policy_version 1323922 (0.0008) [2023-12-27 01:01:54,967][105692] Updated weights for policy 0, policy_version 1323932 (0.0009) [2023-12-27 01:01:54,979][105620] Updated weights for policy 1, policy_version 1325859 (0.0008) [2023-12-27 01:01:55,019][105692] Updated weights for policy 0, policy_version 1323942 (0.0008) [2023-12-27 01:01:55,029][105620] Updated weights for policy 1, policy_version 1325869 (0.0009) [2023-12-27 01:01:55,068][105692] Updated weights for policy 0, policy_version 1323952 (0.0006) [2023-12-27 01:01:55,075][105620] Updated weights for policy 1, policy_version 1325879 (0.0006) [2023-12-27 01:01:55,821][105692] Updated weights for policy 0, policy_version 1323962 (0.0009) [2023-12-27 01:01:55,852][105620] Updated weights for policy 1, policy_version 1325889 (0.0009) [2023-12-27 01:01:55,875][105692] Updated weights for policy 0, policy_version 1323972 (0.0007) [2023-12-27 01:01:55,913][105620] Updated weights for policy 1, policy_version 1325899 (0.0009) [2023-12-27 01:01:55,933][105692] Updated weights for policy 0, policy_version 1323982 (0.0008) [2023-12-27 01:01:55,974][105620] Updated weights for policy 1, policy_version 1325909 (0.0009) [2023-12-27 01:01:56,026][105620] Updated weights for policy 1, policy_version 1325919 (0.0009) [2023-12-27 01:01:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19251.3, 300 sec: 19327.6). Total num frames: 678469632. Throughput: 0: 9565.2, 1: 9721.2. Samples: 678471592. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:01:56,063][104569] Avg episode reward: [(0, '8273.517'), (1, '9000.751')] [2023-12-27 01:01:56,646][105692] Updated weights for policy 0, policy_version 1323992 (0.0008) [2023-12-27 01:01:56,712][105692] Updated weights for policy 0, policy_version 1324002 (0.0009) [2023-12-27 01:01:56,773][105692] Updated weights for policy 0, policy_version 1324012 (0.0009) [2023-12-27 01:01:56,811][105620] Updated weights for policy 1, policy_version 1325929 (0.0008) [2023-12-27 01:01:56,869][105620] Updated weights for policy 1, policy_version 1325939 (0.0009) [2023-12-27 01:01:56,928][105620] Updated weights for policy 1, policy_version 1325949 (0.0009) [2023-12-27 01:01:57,527][105692] Updated weights for policy 0, policy_version 1324022 (0.0009) [2023-12-27 01:01:57,570][105620] Updated weights for policy 1, policy_version 1325959 (0.0006) [2023-12-27 01:01:57,579][105692] Updated weights for policy 0, policy_version 1324032 (0.0008) [2023-12-27 01:01:57,625][105620] Updated weights for policy 1, policy_version 1325969 (0.0006) [2023-12-27 01:01:57,627][105692] Updated weights for policy 0, policy_version 1324042 (0.0010) [2023-12-27 01:01:57,684][105620] Updated weights for policy 1, policy_version 1325979 (0.0007) [2023-12-27 01:01:58,347][105620] Updated weights for policy 1, policy_version 1325989 (0.0010) [2023-12-27 01:01:58,411][105620] Updated weights for policy 1, policy_version 1325999 (0.0011) [2023-12-27 01:01:58,471][105692] Updated weights for policy 0, policy_version 1324052 (0.0009) [2023-12-27 01:01:58,475][105620] Updated weights for policy 1, policy_version 1326009 (0.0010) [2023-12-27 01:01:58,531][105692] Updated weights for policy 0, policy_version 1324062 (0.0007) [2023-12-27 01:01:58,589][105692] Updated weights for policy 0, policy_version 1324073 (0.0008) [2023-12-27 01:01:59,222][105620] Updated weights for policy 1, policy_version 1326019 (0.0008) [2023-12-27 01:01:59,286][105620] Updated weights for policy 1, policy_version 1326029 (0.0012) [2023-12-27 01:01:59,348][105620] Updated weights for policy 1, policy_version 1326040 (0.0009) [2023-12-27 01:01:59,496][105692] Updated weights for policy 0, policy_version 1324083 (0.0008) [2023-12-27 01:01:59,547][105692] Updated weights for policy 0, policy_version 1324093 (0.0008) [2023-12-27 01:01:59,603][105692] Updated weights for policy 0, policy_version 1324103 (0.0008) [2023-12-27 01:02:00,049][105620] Updated weights for policy 1, policy_version 1326050 (0.0006) [2023-12-27 01:02:00,107][105620] Updated weights for policy 1, policy_version 1326060 (0.0007) [2023-12-27 01:02:00,169][105620] Updated weights for policy 1, policy_version 1326070 (0.0005) [2023-12-27 01:02:00,221][105620] Updated weights for policy 1, policy_version 1326080 (0.0005) [2023-12-27 01:02:00,322][105692] Updated weights for policy 0, policy_version 1324113 (0.0011) [2023-12-27 01:02:00,384][105692] Updated weights for policy 0, policy_version 1324123 (0.0009) [2023-12-27 01:02:00,443][105692] Updated weights for policy 0, policy_version 1324133 (0.0010) [2023-12-27 01:02:00,508][105692] Updated weights for policy 0, policy_version 1324143 (0.0006) [2023-12-27 01:02:00,754][105620] Updated weights for policy 1, policy_version 1326090 (0.0005) [2023-12-27 01:02:00,810][105620] Updated weights for policy 1, policy_version 1326100 (0.0005) [2023-12-27 01:02:00,868][105620] Updated weights for policy 1, policy_version 1326110 (0.0005) [2023-12-27 01:02:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19299.8). Total num frames: 678559744. Throughput: 0: 9567.5, 1: 9768.3. Samples: 678528652. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:02:01,062][104569] Avg episode reward: [(0, '8362.901'), (1, '8998.281')] [2023-12-27 01:02:01,064][105692] Updated weights for policy 0, policy_version 1324153 (0.0006) [2023-12-27 01:02:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001326112_339525632.pth... [2023-12-27 01:02:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001324960_339230720.pth [2023-12-27 01:02:01,120][105692] Updated weights for policy 0, policy_version 1324163 (0.0006) [2023-12-27 01:02:01,181][105692] Updated weights for policy 0, policy_version 1324173 (0.0009) [2023-12-27 01:02:01,193][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001324176_339042304.pth... [2023-12-27 01:02:01,197][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001323056_338755584.pth [2023-12-27 01:02:01,525][105620] Updated weights for policy 1, policy_version 1326120 (0.0008) [2023-12-27 01:02:01,578][105620] Updated weights for policy 1, policy_version 1326131 (0.0010) [2023-12-27 01:02:01,632][105620] Updated weights for policy 1, policy_version 1326141 (0.0008) [2023-12-27 01:02:01,841][105692] Updated weights for policy 0, policy_version 1324183 (0.0011) [2023-12-27 01:02:01,900][105692] Updated weights for policy 0, policy_version 1324193 (0.0009) [2023-12-27 01:02:01,956][105692] Updated weights for policy 0, policy_version 1324203 (0.0008) [2023-12-27 01:02:02,306][105620] Updated weights for policy 1, policy_version 1326151 (0.0008) [2023-12-27 01:02:02,367][105620] Updated weights for policy 1, policy_version 1326161 (0.0008) [2023-12-27 01:02:02,427][105620] Updated weights for policy 1, policy_version 1326171 (0.0008) [2023-12-27 01:02:02,698][105692] Updated weights for policy 0, policy_version 1324213 (0.0011) [2023-12-27 01:02:02,749][105692] Updated weights for policy 0, policy_version 1324223 (0.0010) [2023-12-27 01:02:02,800][105692] Updated weights for policy 0, policy_version 1324233 (0.0010) [2023-12-27 01:02:03,166][105620] Updated weights for policy 1, policy_version 1326181 (0.0008) [2023-12-27 01:02:03,231][105620] Updated weights for policy 1, policy_version 1326191 (0.0008) [2023-12-27 01:02:03,289][105620] Updated weights for policy 1, policy_version 1326201 (0.0008) [2023-12-27 01:02:03,556][105692] Updated weights for policy 0, policy_version 1324243 (0.0010) [2023-12-27 01:02:03,607][105692] Updated weights for policy 0, policy_version 1324253 (0.0010) [2023-12-27 01:02:03,651][105692] Updated weights for policy 0, policy_version 1324263 (0.0010) [2023-12-27 01:02:04,022][105620] Updated weights for policy 1, policy_version 1326211 (0.0008) [2023-12-27 01:02:04,079][105620] Updated weights for policy 1, policy_version 1326221 (0.0009) [2023-12-27 01:02:04,145][105620] Updated weights for policy 1, policy_version 1326231 (0.0009) [2023-12-27 01:02:04,417][105692] Updated weights for policy 0, policy_version 1324273 (0.0010) [2023-12-27 01:02:04,479][105692] Updated weights for policy 0, policy_version 1324283 (0.0011) [2023-12-27 01:02:04,528][105692] Updated weights for policy 0, policy_version 1324293 (0.0010) [2023-12-27 01:02:04,576][105692] Updated weights for policy 0, policy_version 1324303 (0.0010) [2023-12-27 01:02:04,814][105620] Updated weights for policy 1, policy_version 1326241 (0.0008) [2023-12-27 01:02:04,859][105620] Updated weights for policy 1, policy_version 1326251 (0.0005) [2023-12-27 01:02:04,906][105620] Updated weights for policy 1, policy_version 1326261 (0.0005) [2023-12-27 01:02:04,957][105620] Updated weights for policy 1, policy_version 1326271 (0.0005) [2023-12-27 01:02:05,350][105692] Updated weights for policy 0, policy_version 1324313 (0.0011) [2023-12-27 01:02:05,409][105692] Updated weights for policy 0, policy_version 1324323 (0.0010) [2023-12-27 01:02:05,460][105692] Updated weights for policy 0, policy_version 1324333 (0.0010) [2023-12-27 01:02:05,548][105620] Updated weights for policy 1, policy_version 1326281 (0.0008) [2023-12-27 01:02:05,592][105620] Updated weights for policy 1, policy_version 1326291 (0.0008) [2023-12-27 01:02:05,636][105620] Updated weights for policy 1, policy_version 1326301 (0.0007) [2023-12-27 01:02:06,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.8, 300 sec: 19327.6). Total num frames: 678658048. Throughput: 0: 9531.7, 1: 9858.3. Samples: 678647812. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:02:06,062][104569] Avg episode reward: [(0, '8452.681'), (1, '8992.996')] [2023-12-27 01:02:06,207][105692] Updated weights for policy 0, policy_version 1324343 (0.0011) [2023-12-27 01:02:06,272][105692] Updated weights for policy 0, policy_version 1324353 (0.0011) [2023-12-27 01:02:06,329][105692] Updated weights for policy 0, policy_version 1324363 (0.0011) [2023-12-27 01:02:06,412][105620] Updated weights for policy 1, policy_version 1326311 (0.0007) [2023-12-27 01:02:06,476][105620] Updated weights for policy 1, policy_version 1326321 (0.0008) [2023-12-27 01:02:06,532][105620] Updated weights for policy 1, policy_version 1326331 (0.0009) [2023-12-27 01:02:07,102][105692] Updated weights for policy 0, policy_version 1324373 (0.0009) [2023-12-27 01:02:07,158][105692] Updated weights for policy 0, policy_version 1324383 (0.0009) [2023-12-27 01:02:07,209][105692] Updated weights for policy 0, policy_version 1324393 (0.0008) [2023-12-27 01:02:07,285][105620] Updated weights for policy 1, policy_version 1326341 (0.0008) [2023-12-27 01:02:07,350][105620] Updated weights for policy 1, policy_version 1326351 (0.0009) [2023-12-27 01:02:07,408][105620] Updated weights for policy 1, policy_version 1326361 (0.0010) [2023-12-27 01:02:07,902][105692] Updated weights for policy 0, policy_version 1324403 (0.0009) [2023-12-27 01:02:07,949][105692] Updated weights for policy 0, policy_version 1324413 (0.0008) [2023-12-27 01:02:08,002][105692] Updated weights for policy 0, policy_version 1324423 (0.0008) [2023-12-27 01:02:08,248][105620] Updated weights for policy 1, policy_version 1326371 (0.0009) [2023-12-27 01:02:08,303][105620] Updated weights for policy 1, policy_version 1326381 (0.0009) [2023-12-27 01:02:08,365][105620] Updated weights for policy 1, policy_version 1326391 (0.0008) [2023-12-27 01:02:08,683][105692] Updated weights for policy 0, policy_version 1324433 (0.0006) [2023-12-27 01:02:08,739][105692] Updated weights for policy 0, policy_version 1324443 (0.0008) [2023-12-27 01:02:08,791][105692] Updated weights for policy 0, policy_version 1324453 (0.0008) [2023-12-27 01:02:08,842][105692] Updated weights for policy 0, policy_version 1324463 (0.0008) [2023-12-27 01:02:09,135][105620] Updated weights for policy 1, policy_version 1326401 (0.0008) [2023-12-27 01:02:09,194][105620] Updated weights for policy 1, policy_version 1326411 (0.0005) [2023-12-27 01:02:09,265][105620] Updated weights for policy 1, policy_version 1326421 (0.0008) [2023-12-27 01:02:09,328][105620] Updated weights for policy 1, policy_version 1326431 (0.0009) [2023-12-27 01:02:09,681][105692] Updated weights for policy 0, policy_version 1324473 (0.0008) [2023-12-27 01:02:09,741][105692] Updated weights for policy 0, policy_version 1324483 (0.0009) [2023-12-27 01:02:09,801][105692] Updated weights for policy 0, policy_version 1324493 (0.0011) [2023-12-27 01:02:10,030][105620] Updated weights for policy 1, policy_version 1326441 (0.0007) [2023-12-27 01:02:10,097][105620] Updated weights for policy 1, policy_version 1326451 (0.0005) [2023-12-27 01:02:10,161][105620] Updated weights for policy 1, policy_version 1326461 (0.0007) [2023-12-27 01:02:10,584][105692] Updated weights for policy 0, policy_version 1324503 (0.0008) [2023-12-27 01:02:10,645][105692] Updated weights for policy 0, policy_version 1324513 (0.0008) [2023-12-27 01:02:10,703][105692] Updated weights for policy 0, policy_version 1324523 (0.0008) [2023-12-27 01:02:10,860][105620] Updated weights for policy 1, policy_version 1326471 (0.0009) [2023-12-27 01:02:10,916][105620] Updated weights for policy 1, policy_version 1326481 (0.0008) [2023-12-27 01:02:10,968][105620] Updated weights for policy 1, policy_version 1326491 (0.0009) [2023-12-27 01:02:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19327.6). Total num frames: 678756352. Throughput: 0: 9517.4, 1: 9850.1. Samples: 678760860. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:02:11,062][104569] Avg episode reward: [(0, '8453.350'), (1, '8997.282')] [2023-12-27 01:02:11,445][105692] Updated weights for policy 0, policy_version 1324533 (0.0008) [2023-12-27 01:02:11,502][105692] Updated weights for policy 0, policy_version 1324543 (0.0009) [2023-12-27 01:02:11,562][105692] Updated weights for policy 0, policy_version 1324553 (0.0010) [2023-12-27 01:02:11,754][105620] Updated weights for policy 1, policy_version 1326501 (0.0008) [2023-12-27 01:02:11,820][105620] Updated weights for policy 1, policy_version 1326511 (0.0008) [2023-12-27 01:02:11,880][105620] Updated weights for policy 1, policy_version 1326521 (0.0005) [2023-12-27 01:02:12,373][105692] Updated weights for policy 0, policy_version 1324563 (0.0008) [2023-12-27 01:02:12,431][105692] Updated weights for policy 0, policy_version 1324573 (0.0005) [2023-12-27 01:02:12,488][105692] Updated weights for policy 0, policy_version 1324583 (0.0010) [2023-12-27 01:02:12,535][105620] Updated weights for policy 1, policy_version 1326531 (0.0007) [2023-12-27 01:02:12,605][105620] Updated weights for policy 1, policy_version 1326541 (0.0008) [2023-12-27 01:02:12,665][105620] Updated weights for policy 1, policy_version 1326551 (0.0005) [2023-12-27 01:02:13,213][105692] Updated weights for policy 0, policy_version 1324594 (0.0008) [2023-12-27 01:02:13,269][105692] Updated weights for policy 0, policy_version 1324604 (0.0005) [2023-12-27 01:02:13,302][105620] Updated weights for policy 1, policy_version 1326561 (0.0007) [2023-12-27 01:02:13,316][105692] Updated weights for policy 0, policy_version 1324614 (0.0006) [2023-12-27 01:02:13,348][105620] Updated weights for policy 1, policy_version 1326571 (0.0006) [2023-12-27 01:02:13,364][105692] Updated weights for policy 0, policy_version 1324624 (0.0007) [2023-12-27 01:02:13,401][105620] Updated weights for policy 1, policy_version 1326581 (0.0006) [2023-12-27 01:02:13,449][105620] Updated weights for policy 1, policy_version 1326591 (0.0005) [2023-12-27 01:02:14,082][105620] Updated weights for policy 1, policy_version 1326601 (0.0010) [2023-12-27 01:02:14,124][105692] Updated weights for policy 0, policy_version 1324634 (0.0006) [2023-12-27 01:02:14,138][105620] Updated weights for policy 1, policy_version 1326611 (0.0011) [2023-12-27 01:02:14,188][105692] Updated weights for policy 0, policy_version 1324644 (0.0006) [2023-12-27 01:02:14,194][105620] Updated weights for policy 1, policy_version 1326621 (0.0010) [2023-12-27 01:02:14,253][105692] Updated weights for policy 0, policy_version 1324654 (0.0007) [2023-12-27 01:02:14,885][105692] Updated weights for policy 0, policy_version 1324664 (0.0008) [2023-12-27 01:02:14,935][105620] Updated weights for policy 1, policy_version 1326631 (0.0011) [2023-12-27 01:02:14,942][105692] Updated weights for policy 0, policy_version 1324674 (0.0007) [2023-12-27 01:02:14,992][105620] Updated weights for policy 1, policy_version 1326641 (0.0010) [2023-12-27 01:02:14,993][105692] Updated weights for policy 0, policy_version 1324684 (0.0009) [2023-12-27 01:02:15,055][105620] Updated weights for policy 1, policy_version 1326651 (0.0011) [2023-12-27 01:02:15,723][105692] Updated weights for policy 0, policy_version 1324694 (0.0006) [2023-12-27 01:02:15,775][105692] Updated weights for policy 0, policy_version 1324704 (0.0005) [2023-12-27 01:02:15,800][105620] Updated weights for policy 1, policy_version 1326661 (0.0007) [2023-12-27 01:02:15,824][105692] Updated weights for policy 0, policy_version 1324714 (0.0005) [2023-12-27 01:02:15,846][105620] Updated weights for policy 1, policy_version 1326671 (0.0005) [2023-12-27 01:02:15,892][105620] Updated weights for policy 1, policy_version 1326681 (0.0005) [2023-12-27 01:02:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 678854656. Throughput: 0: 9418.2, 1: 9871.7. Samples: 678819472. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:02:16,062][104569] Avg episode reward: [(0, '8269.457'), (1, '9090.421')] [2023-12-27 01:02:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001324720_339181568.pth... [2023-12-27 01:02:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001326688_339673088.pth... [2023-12-27 01:02:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001323632_338903040.pth [2023-12-27 01:02:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001325536_339378176.pth [2023-12-27 01:02:16,489][105692] Updated weights for policy 0, policy_version 1324724 (0.0005) [2023-12-27 01:02:16,541][105692] Updated weights for policy 0, policy_version 1324734 (0.0005) [2023-12-27 01:02:16,544][105620] Updated weights for policy 1, policy_version 1326691 (0.0007) [2023-12-27 01:02:16,585][105692] Updated weights for policy 0, policy_version 1324744 (0.0005) [2023-12-27 01:02:16,610][105620] Updated weights for policy 1, policy_version 1326701 (0.0006) [2023-12-27 01:02:16,671][105620] Updated weights for policy 1, policy_version 1326711 (0.0010) [2023-12-27 01:02:17,226][105692] Updated weights for policy 0, policy_version 1324754 (0.0006) [2023-12-27 01:02:17,274][105692] Updated weights for policy 0, policy_version 1324764 (0.0010) [2023-12-27 01:02:17,308][105620] Updated weights for policy 1, policy_version 1326721 (0.0010) [2023-12-27 01:02:17,341][105692] Updated weights for policy 0, policy_version 1324774 (0.0009) [2023-12-27 01:02:17,374][105620] Updated weights for policy 1, policy_version 1326731 (0.0008) [2023-12-27 01:02:17,403][105692] Updated weights for policy 0, policy_version 1324784 (0.0008) [2023-12-27 01:02:17,435][105620] Updated weights for policy 1, policy_version 1326741 (0.0008) [2023-12-27 01:02:17,495][105620] Updated weights for policy 1, policy_version 1326751 (0.0008) [2023-12-27 01:02:18,045][105692] Updated weights for policy 0, policy_version 1324794 (0.0008) [2023-12-27 01:02:18,106][105692] Updated weights for policy 0, policy_version 1324804 (0.0009) [2023-12-27 01:02:18,125][105620] Updated weights for policy 1, policy_version 1326761 (0.0006) [2023-12-27 01:02:18,173][105692] Updated weights for policy 0, policy_version 1324814 (0.0009) [2023-12-27 01:02:18,180][105620] Updated weights for policy 1, policy_version 1326771 (0.0006) [2023-12-27 01:02:18,240][105620] Updated weights for policy 1, policy_version 1326781 (0.0007) [2023-12-27 01:02:18,885][105620] Updated weights for policy 1, policy_version 1326791 (0.0009) [2023-12-27 01:02:18,935][105692] Updated weights for policy 0, policy_version 1324824 (0.0010) [2023-12-27 01:02:18,944][105620] Updated weights for policy 1, policy_version 1326801 (0.0006) [2023-12-27 01:02:18,994][105692] Updated weights for policy 0, policy_version 1324834 (0.0010) [2023-12-27 01:02:18,997][105620] Updated weights for policy 1, policy_version 1326811 (0.0005) [2023-12-27 01:02:19,046][105692] Updated weights for policy 0, policy_version 1324844 (0.0010) [2023-12-27 01:02:19,593][105620] Updated weights for policy 1, policy_version 1326821 (0.0007) [2023-12-27 01:02:19,647][105620] Updated weights for policy 1, policy_version 1326831 (0.0008) [2023-12-27 01:02:19,707][105620] Updated weights for policy 1, policy_version 1326841 (0.0008) [2023-12-27 01:02:19,819][105692] Updated weights for policy 0, policy_version 1324854 (0.0011) [2023-12-27 01:02:19,882][105692] Updated weights for policy 0, policy_version 1324864 (0.0010) [2023-12-27 01:02:19,949][105692] Updated weights for policy 0, policy_version 1324874 (0.0011) [2023-12-27 01:02:20,446][105620] Updated weights for policy 1, policy_version 1326851 (0.0008) [2023-12-27 01:02:20,507][105620] Updated weights for policy 1, policy_version 1326861 (0.0007) [2023-12-27 01:02:20,567][105620] Updated weights for policy 1, policy_version 1326871 (0.0009) [2023-12-27 01:02:20,743][105692] Updated weights for policy 0, policy_version 1324884 (0.0011) [2023-12-27 01:02:20,814][105692] Updated weights for policy 0, policy_version 1324894 (0.0010) [2023-12-27 01:02:20,874][105692] Updated weights for policy 0, policy_version 1324904 (0.0010) [2023-12-27 01:02:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19355.3). Total num frames: 678952960. Throughput: 0: 9453.0, 1: 9817.9. Samples: 678941192. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 01:02:21,062][104569] Avg episode reward: [(0, '8544.009'), (1, '9180.356')] [2023-12-27 01:02:21,215][105620] Updated weights for policy 1, policy_version 1326881 (0.0008) [2023-12-27 01:02:21,284][105620] Updated weights for policy 1, policy_version 1326891 (0.0008) [2023-12-27 01:02:21,351][105620] Updated weights for policy 1, policy_version 1326901 (0.0008) [2023-12-27 01:02:21,416][105620] Updated weights for policy 1, policy_version 1326911 (0.0008) [2023-12-27 01:02:21,728][105692] Updated weights for policy 0, policy_version 1324914 (0.0009) [2023-12-27 01:02:21,792][105692] Updated weights for policy 0, policy_version 1324924 (0.0009) [2023-12-27 01:02:21,855][105692] Updated weights for policy 0, policy_version 1324934 (0.0009) [2023-12-27 01:02:21,902][105692] Updated weights for policy 0, policy_version 1324944 (0.0008) [2023-12-27 01:02:22,137][105620] Updated weights for policy 1, policy_version 1326921 (0.0009) [2023-12-27 01:02:22,191][105620] Updated weights for policy 1, policy_version 1326931 (0.0008) [2023-12-27 01:02:22,252][105620] Updated weights for policy 1, policy_version 1326941 (0.0009) [2023-12-27 01:02:22,618][105692] Updated weights for policy 0, policy_version 1324954 (0.0006) [2023-12-27 01:02:22,663][105692] Updated weights for policy 0, policy_version 1324964 (0.0005) [2023-12-27 01:02:22,717][105692] Updated weights for policy 0, policy_version 1324974 (0.0005) [2023-12-27 01:02:23,087][105620] Updated weights for policy 1, policy_version 1326951 (0.0009) [2023-12-27 01:02:23,141][105620] Updated weights for policy 1, policy_version 1326961 (0.0010) [2023-12-27 01:02:23,192][105620] Updated weights for policy 1, policy_version 1326971 (0.0009) [2023-12-27 01:02:23,315][105692] Updated weights for policy 0, policy_version 1324984 (0.0005) [2023-12-27 01:02:23,378][105692] Updated weights for policy 0, policy_version 1324995 (0.0010) [2023-12-27 01:02:23,432][105692] Updated weights for policy 0, policy_version 1325005 (0.0009) [2023-12-27 01:02:23,950][105620] Updated weights for policy 1, policy_version 1326981 (0.0009) [2023-12-27 01:02:24,004][105620] Updated weights for policy 1, policy_version 1326991 (0.0010) [2023-12-27 01:02:24,062][105620] Updated weights for policy 1, policy_version 1327001 (0.0010) [2023-12-27 01:02:24,116][105692] Updated weights for policy 0, policy_version 1325015 (0.0009) [2023-12-27 01:02:24,180][105692] Updated weights for policy 0, policy_version 1325025 (0.0009) [2023-12-27 01:02:24,251][105692] Updated weights for policy 0, policy_version 1325035 (0.0007) [2023-12-27 01:02:24,872][105692] Updated weights for policy 0, policy_version 1325045 (0.0006) [2023-12-27 01:02:24,880][105620] Updated weights for policy 1, policy_version 1327011 (0.0009) [2023-12-27 01:02:24,929][105620] Updated weights for policy 1, policy_version 1327021 (0.0009) [2023-12-27 01:02:24,933][105692] Updated weights for policy 0, policy_version 1325055 (0.0005) [2023-12-27 01:02:24,977][105620] Updated weights for policy 1, policy_version 1327031 (0.0008) [2023-12-27 01:02:24,978][105692] Updated weights for policy 0, policy_version 1325065 (0.0005) [2023-12-27 01:02:25,709][105692] Updated weights for policy 0, policy_version 1325075 (0.0006) [2023-12-27 01:02:25,715][105620] Updated weights for policy 1, policy_version 1327041 (0.0009) [2023-12-27 01:02:25,758][105692] Updated weights for policy 0, policy_version 1325085 (0.0005) [2023-12-27 01:02:25,767][105620] Updated weights for policy 1, policy_version 1327051 (0.0010) [2023-12-27 01:02:25,806][105692] Updated weights for policy 0, policy_version 1325095 (0.0005) [2023-12-27 01:02:25,826][105620] Updated weights for policy 1, policy_version 1327061 (0.0010) [2023-12-27 01:02:25,874][105620] Updated weights for policy 1, policy_version 1327071 (0.0010) [2023-12-27 01:02:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19355.3). Total num frames: 679051264. Throughput: 0: 9500.3, 1: 9749.8. Samples: 679055756. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:02:26,062][104569] Avg episode reward: [(0, '8906.464'), (1, '9109.542')] [2023-12-27 01:02:26,489][105620] Updated weights for policy 1, policy_version 1327081 (0.0010) [2023-12-27 01:02:26,537][105620] Updated weights for policy 1, policy_version 1327091 (0.0010) [2023-12-27 01:02:26,562][105692] Updated weights for policy 0, policy_version 1325105 (0.0008) [2023-12-27 01:02:26,589][105620] Updated weights for policy 1, policy_version 1327101 (0.0010) [2023-12-27 01:02:26,618][105692] Updated weights for policy 0, policy_version 1325115 (0.0005) [2023-12-27 01:02:26,680][105692] Updated weights for policy 0, policy_version 1325125 (0.0005) [2023-12-27 01:02:26,739][105692] Updated weights for policy 0, policy_version 1325135 (0.0006) [2023-12-27 01:02:27,241][105692] Updated weights for policy 0, policy_version 1325145 (0.0010) [2023-12-27 01:02:27,289][105692] Updated weights for policy 0, policy_version 1325155 (0.0010) [2023-12-27 01:02:27,348][105692] Updated weights for policy 0, policy_version 1325165 (0.0010) [2023-12-27 01:02:27,352][105620] Updated weights for policy 1, policy_version 1327111 (0.0007) [2023-12-27 01:02:27,405][105620] Updated weights for policy 1, policy_version 1327121 (0.0005) [2023-12-27 01:02:27,460][105620] Updated weights for policy 1, policy_version 1327131 (0.0005) [2023-12-27 01:02:28,092][105692] Updated weights for policy 0, policy_version 1325175 (0.0010) [2023-12-27 01:02:28,139][105620] Updated weights for policy 1, policy_version 1327141 (0.0006) [2023-12-27 01:02:28,144][105692] Updated weights for policy 0, policy_version 1325185 (0.0010) [2023-12-27 01:02:28,187][105620] Updated weights for policy 1, policy_version 1327151 (0.0010) [2023-12-27 01:02:28,190][105692] Updated weights for policy 0, policy_version 1325195 (0.0006) [2023-12-27 01:02:28,242][105620] Updated weights for policy 1, policy_version 1327161 (0.0011) [2023-12-27 01:02:28,908][105620] Updated weights for policy 1, policy_version 1327171 (0.0011) [2023-12-27 01:02:28,969][105620] Updated weights for policy 1, policy_version 1327181 (0.0010) [2023-12-27 01:02:28,986][105692] Updated weights for policy 0, policy_version 1325205 (0.0005) [2023-12-27 01:02:29,027][105620] Updated weights for policy 1, policy_version 1327191 (0.0010) [2023-12-27 01:02:29,042][105692] Updated weights for policy 0, policy_version 1325215 (0.0006) [2023-12-27 01:02:29,103][105692] Updated weights for policy 0, policy_version 1325225 (0.0007) [2023-12-27 01:02:29,718][105620] Updated weights for policy 1, policy_version 1327201 (0.0010) [2023-12-27 01:02:29,780][105620] Updated weights for policy 1, policy_version 1327211 (0.0007) [2023-12-27 01:02:29,837][105620] Updated weights for policy 1, policy_version 1327221 (0.0009) [2023-12-27 01:02:29,894][105692] Updated weights for policy 0, policy_version 1325235 (0.0007) [2023-12-27 01:02:29,895][105620] Updated weights for policy 1, policy_version 1327231 (0.0008) [2023-12-27 01:02:29,958][105692] Updated weights for policy 0, policy_version 1325245 (0.0008) [2023-12-27 01:02:30,011][105692] Updated weights for policy 0, policy_version 1325255 (0.0009) [2023-12-27 01:02:30,514][105620] Updated weights for policy 1, policy_version 1327241 (0.0009) [2023-12-27 01:02:30,578][105620] Updated weights for policy 1, policy_version 1327251 (0.0009) [2023-12-27 01:02:30,634][105620] Updated weights for policy 1, policy_version 1327261 (0.0005) [2023-12-27 01:02:30,855][105692] Updated weights for policy 0, policy_version 1325265 (0.0009) [2023-12-27 01:02:30,915][105692] Updated weights for policy 0, policy_version 1325275 (0.0010) [2023-12-27 01:02:30,973][105692] Updated weights for policy 0, policy_version 1325285 (0.0008) [2023-12-27 01:02:31,031][105692] Updated weights for policy 0, policy_version 1325295 (0.0009) [2023-12-27 01:02:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 679149568. Throughput: 0: 9567.8, 1: 9811.8. Samples: 679117012. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:02:31,063][104569] Avg episode reward: [(0, '8905.393'), (1, '9198.059')] [2023-12-27 01:02:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001325296_339329024.pth... [2023-12-27 01:02:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001327264_339820544.pth... [2023-12-27 01:02:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001324176_339042304.pth [2023-12-27 01:02:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001326112_339525632.pth [2023-12-27 01:02:31,314][105620] Updated weights for policy 1, policy_version 1327271 (0.0005) [2023-12-27 01:02:31,378][105620] Updated weights for policy 1, policy_version 1327281 (0.0008) [2023-12-27 01:02:31,442][105620] Updated weights for policy 1, policy_version 1327291 (0.0007) [2023-12-27 01:02:31,817][105692] Updated weights for policy 0, policy_version 1325305 (0.0010) [2023-12-27 01:02:31,835][105585] KL-divergence is very high: 193.0222 [2023-12-27 01:02:31,872][105692] Updated weights for policy 0, policy_version 1325315 (0.0010) [2023-12-27 01:02:31,877][105585] KL-divergence is very high: 336.7086 [2023-12-27 01:02:31,917][105585] KL-divergence is very high: 367.3695 [2023-12-27 01:02:31,923][105692] Updated weights for policy 0, policy_version 1325325 (0.0010) [2023-12-27 01:02:32,043][105620] Updated weights for policy 1, policy_version 1327301 (0.0006) [2023-12-27 01:02:32,097][105620] Updated weights for policy 1, policy_version 1327311 (0.0005) [2023-12-27 01:02:32,163][105620] Updated weights for policy 1, policy_version 1327321 (0.0005) [2023-12-27 01:02:32,631][105692] Updated weights for policy 0, policy_version 1325335 (0.0007) [2023-12-27 01:02:32,696][105692] Updated weights for policy 0, policy_version 1325345 (0.0008) [2023-12-27 01:02:32,707][105620] Updated weights for policy 1, policy_version 1327331 (0.0006) [2023-12-27 01:02:32,754][105692] Updated weights for policy 0, policy_version 1325355 (0.0010) [2023-12-27 01:02:32,761][105620] Updated weights for policy 1, policy_version 1327341 (0.0008) [2023-12-27 01:02:32,808][105620] Updated weights for policy 1, policy_version 1327351 (0.0008) [2023-12-27 01:02:33,329][105692] Updated weights for policy 0, policy_version 1325365 (0.0008) [2023-12-27 01:02:33,377][105692] Updated weights for policy 0, policy_version 1325375 (0.0006) [2023-12-27 01:02:33,418][105620] Updated weights for policy 1, policy_version 1327361 (0.0007) [2023-12-27 01:02:33,421][105692] Updated weights for policy 0, policy_version 1325385 (0.0007) [2023-12-27 01:02:33,464][105620] Updated weights for policy 1, policy_version 1327371 (0.0007) [2023-12-27 01:02:33,516][105620] Updated weights for policy 1, policy_version 1327382 (0.0010) [2023-12-27 01:02:33,568][105620] Updated weights for policy 1, policy_version 1327392 (0.0009) [2023-12-27 01:02:33,971][105692] Updated weights for policy 0, policy_version 1325395 (0.0006) [2023-12-27 01:02:34,022][105692] Updated weights for policy 0, policy_version 1325405 (0.0005) [2023-12-27 01:02:34,065][105692] Updated weights for policy 0, policy_version 1325415 (0.0005) [2023-12-27 01:02:34,364][105620] Updated weights for policy 1, policy_version 1327402 (0.0008) [2023-12-27 01:02:34,417][105620] Updated weights for policy 1, policy_version 1327412 (0.0008) [2023-12-27 01:02:34,475][105620] Updated weights for policy 1, policy_version 1327422 (0.0010) [2023-12-27 01:02:34,662][105692] Updated weights for policy 0, policy_version 1325425 (0.0006) [2023-12-27 01:02:34,731][105692] Updated weights for policy 0, policy_version 1325435 (0.0010) [2023-12-27 01:02:34,794][105692] Updated weights for policy 0, policy_version 1325445 (0.0011) [2023-12-27 01:02:34,856][105692] Updated weights for policy 0, policy_version 1325455 (0.0011) [2023-12-27 01:02:35,245][105620] Updated weights for policy 1, policy_version 1327432 (0.0007) [2023-12-27 01:02:35,300][105620] Updated weights for policy 1, policy_version 1327442 (0.0007) [2023-12-27 01:02:35,358][105620] Updated weights for policy 1, policy_version 1327452 (0.0008) [2023-12-27 01:02:35,584][105692] Updated weights for policy 0, policy_version 1325465 (0.0010) [2023-12-27 01:02:35,645][105692] Updated weights for policy 0, policy_version 1325475 (0.0010) [2023-12-27 01:02:35,703][105692] Updated weights for policy 0, policy_version 1325485 (0.0010) [2023-12-27 01:02:35,986][105620] Updated weights for policy 1, policy_version 1327462 (0.0008) [2023-12-27 01:02:36,041][105620] Updated weights for policy 1, policy_version 1327472 (0.0009) [2023-12-27 01:02:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 679247872. Throughput: 0: 9625.2, 1: 9933.3. Samples: 679239220. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:02:36,062][104569] Avg episode reward: [(0, '8630.983'), (1, '9269.427')] [2023-12-27 01:02:36,100][105620] Updated weights for policy 1, policy_version 1327482 (0.0009) [2023-12-27 01:02:36,415][105692] Updated weights for policy 0, policy_version 1325495 (0.0009) [2023-12-27 01:02:36,472][105692] Updated weights for policy 0, policy_version 1325505 (0.0008) [2023-12-27 01:02:36,525][105692] Updated weights for policy 0, policy_version 1325515 (0.0010) [2023-12-27 01:02:36,894][105620] Updated weights for policy 1, policy_version 1327492 (0.0009) [2023-12-27 01:02:36,957][105620] Updated weights for policy 1, policy_version 1327503 (0.0012) [2023-12-27 01:02:37,023][105620] Updated weights for policy 1, policy_version 1327513 (0.0011) [2023-12-27 01:02:37,306][105692] Updated weights for policy 0, policy_version 1325525 (0.0011) [2023-12-27 01:02:37,354][105692] Updated weights for policy 0, policy_version 1325535 (0.0010) [2023-12-27 01:02:37,407][105692] Updated weights for policy 0, policy_version 1325545 (0.0010) [2023-12-27 01:02:37,737][105620] Updated weights for policy 1, policy_version 1327523 (0.0010) [2023-12-27 01:02:37,801][105620] Updated weights for policy 1, policy_version 1327533 (0.0005) [2023-12-27 01:02:37,856][105620] Updated weights for policy 1, policy_version 1327543 (0.0005) [2023-12-27 01:02:38,168][105692] Updated weights for policy 0, policy_version 1325555 (0.0011) [2023-12-27 01:02:38,224][105692] Updated weights for policy 0, policy_version 1325565 (0.0010) [2023-12-27 01:02:38,276][105692] Updated weights for policy 0, policy_version 1325575 (0.0010) [2023-12-27 01:02:38,397][105620] Updated weights for policy 1, policy_version 1327553 (0.0006) [2023-12-27 01:02:38,455][105620] Updated weights for policy 1, policy_version 1327563 (0.0011) [2023-12-27 01:02:38,515][105620] Updated weights for policy 1, policy_version 1327573 (0.0011) [2023-12-27 01:02:38,578][105620] Updated weights for policy 1, policy_version 1327583 (0.0011) [2023-12-27 01:02:39,007][105692] Updated weights for policy 0, policy_version 1325585 (0.0010) [2023-12-27 01:02:39,063][105692] Updated weights for policy 0, policy_version 1325595 (0.0006) [2023-12-27 01:02:39,110][105692] Updated weights for policy 0, policy_version 1325605 (0.0005) [2023-12-27 01:02:39,163][105692] Updated weights for policy 0, policy_version 1325615 (0.0005) [2023-12-27 01:02:39,262][105620] Updated weights for policy 1, policy_version 1327593 (0.0008) [2023-12-27 01:02:39,321][105620] Updated weights for policy 1, policy_version 1327603 (0.0008) [2023-12-27 01:02:39,389][105620] Updated weights for policy 1, policy_version 1327613 (0.0011) [2023-12-27 01:02:39,818][105692] Updated weights for policy 0, policy_version 1325625 (0.0009) [2023-12-27 01:02:39,881][105692] Updated weights for policy 0, policy_version 1325635 (0.0007) [2023-12-27 01:02:39,946][105692] Updated weights for policy 0, policy_version 1325645 (0.0008) [2023-12-27 01:02:40,170][105620] Updated weights for policy 1, policy_version 1327623 (0.0008) [2023-12-27 01:02:40,237][105620] Updated weights for policy 1, policy_version 1327633 (0.0009) [2023-12-27 01:02:40,296][105620] Updated weights for policy 1, policy_version 1327643 (0.0009) [2023-12-27 01:02:40,664][105692] Updated weights for policy 0, policy_version 1325655 (0.0009) [2023-12-27 01:02:40,719][105692] Updated weights for policy 0, policy_version 1325665 (0.0009) [2023-12-27 01:02:40,783][105692] Updated weights for policy 0, policy_version 1325675 (0.0006) [2023-12-27 01:02:40,979][105620] Updated weights for policy 1, policy_version 1327653 (0.0009) [2023-12-27 01:02:41,031][105620] Updated weights for policy 1, policy_version 1327663 (0.0008) [2023-12-27 01:02:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 679346176. Throughput: 0: 9683.0, 1: 9991.5. Samples: 679356940. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:02:41,063][104569] Avg episode reward: [(0, '8361.888'), (1, '9269.263')] [2023-12-27 01:02:41,090][105620] Updated weights for policy 1, policy_version 1327673 (0.0008) [2023-12-27 01:02:41,531][105692] Updated weights for policy 0, policy_version 1325685 (0.0007) [2023-12-27 01:02:41,590][105692] Updated weights for policy 0, policy_version 1325695 (0.0005) [2023-12-27 01:02:41,655][105692] Updated weights for policy 0, policy_version 1325705 (0.0008) [2023-12-27 01:02:41,954][105620] Updated weights for policy 1, policy_version 1327683 (0.0009) [2023-12-27 01:02:42,017][105620] Updated weights for policy 1, policy_version 1327693 (0.0011) [2023-12-27 01:02:42,076][105620] Updated weights for policy 1, policy_version 1327703 (0.0011) [2023-12-27 01:02:42,351][105692] Updated weights for policy 0, policy_version 1325715 (0.0007) [2023-12-27 01:02:42,415][105692] Updated weights for policy 0, policy_version 1325725 (0.0008) [2023-12-27 01:02:42,476][105692] Updated weights for policy 0, policy_version 1325735 (0.0010) [2023-12-27 01:02:42,827][105620] Updated weights for policy 1, policy_version 1327713 (0.0009) [2023-12-27 01:02:42,892][105620] Updated weights for policy 1, policy_version 1327723 (0.0011) [2023-12-27 01:02:42,950][105620] Updated weights for policy 1, policy_version 1327733 (0.0010) [2023-12-27 01:02:43,005][105620] Updated weights for policy 1, policy_version 1327743 (0.0010) [2023-12-27 01:02:43,220][105692] Updated weights for policy 0, policy_version 1325745 (0.0010) [2023-12-27 01:02:43,268][105692] Updated weights for policy 0, policy_version 1325755 (0.0005) [2023-12-27 01:02:43,314][105692] Updated weights for policy 0, policy_version 1325765 (0.0005) [2023-12-27 01:02:43,378][105692] Updated weights for policy 0, policy_version 1325775 (0.0005) [2023-12-27 01:02:43,750][105620] Updated weights for policy 1, policy_version 1327753 (0.0011) [2023-12-27 01:02:43,806][105620] Updated weights for policy 1, policy_version 1327763 (0.0010) [2023-12-27 01:02:43,864][105620] Updated weights for policy 1, policy_version 1327773 (0.0008) [2023-12-27 01:02:43,905][105692] Updated weights for policy 0, policy_version 1325785 (0.0010) [2023-12-27 01:02:43,958][105692] Updated weights for policy 0, policy_version 1325795 (0.0010) [2023-12-27 01:02:44,006][105692] Updated weights for policy 0, policy_version 1325805 (0.0010) [2023-12-27 01:02:44,607][105620] Updated weights for policy 1, policy_version 1327783 (0.0007) [2023-12-27 01:02:44,655][105620] Updated weights for policy 1, policy_version 1327793 (0.0008) [2023-12-27 01:02:44,701][105620] Updated weights for policy 1, policy_version 1327803 (0.0007) [2023-12-27 01:02:44,755][105692] Updated weights for policy 0, policy_version 1325815 (0.0010) [2023-12-27 01:02:44,822][105692] Updated weights for policy 0, policy_version 1325825 (0.0011) [2023-12-27 01:02:44,886][105692] Updated weights for policy 0, policy_version 1325835 (0.0011) [2023-12-27 01:02:45,487][105620] Updated weights for policy 1, policy_version 1327813 (0.0008) [2023-12-27 01:02:45,535][105620] Updated weights for policy 1, policy_version 1327823 (0.0008) [2023-12-27 01:02:45,579][105620] Updated weights for policy 1, policy_version 1327833 (0.0007) [2023-12-27 01:02:45,637][105692] Updated weights for policy 0, policy_version 1325845 (0.0011) [2023-12-27 01:02:45,699][105692] Updated weights for policy 0, policy_version 1325855 (0.0010) [2023-12-27 01:02:45,763][105692] Updated weights for policy 0, policy_version 1325865 (0.0010) [2023-12-27 01:02:46,062][104569] Fps is (10 sec: 19660.0, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 679444480. Throughput: 0: 9726.0, 1: 9940.8. Samples: 679413668. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:02:46,063][104569] Avg episode reward: [(0, '8454.541'), (1, '9269.568')] [2023-12-27 01:02:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001325872_339476480.pth... [2023-12-27 01:02:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001327840_339968000.pth... [2023-12-27 01:02:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001324720_339181568.pth [2023-12-27 01:02:46,081][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001326688_339673088.pth [2023-12-27 01:02:46,354][105620] Updated weights for policy 1, policy_version 1327843 (0.0008) [2023-12-27 01:02:46,409][105620] Updated weights for policy 1, policy_version 1327853 (0.0007) [2023-12-27 01:02:46,477][105620] Updated weights for policy 1, policy_version 1327863 (0.0008) [2023-12-27 01:02:46,498][105692] Updated weights for policy 0, policy_version 1325875 (0.0011) [2023-12-27 01:02:46,555][105692] Updated weights for policy 0, policy_version 1325885 (0.0010) [2023-12-27 01:02:46,608][105692] Updated weights for policy 0, policy_version 1325895 (0.0010) [2023-12-27 01:02:47,213][105620] Updated weights for policy 1, policy_version 1327873 (0.0006) [2023-12-27 01:02:47,266][105620] Updated weights for policy 1, policy_version 1327883 (0.0005) [2023-12-27 01:02:47,317][105620] Updated weights for policy 1, policy_version 1327893 (0.0005) [2023-12-27 01:02:47,355][105692] Updated weights for policy 0, policy_version 1325905 (0.0010) [2023-12-27 01:02:47,375][105620] Updated weights for policy 1, policy_version 1327903 (0.0008) [2023-12-27 01:02:47,401][105692] Updated weights for policy 0, policy_version 1325915 (0.0005) [2023-12-27 01:02:47,447][105692] Updated weights for policy 0, policy_version 1325925 (0.0005) [2023-12-27 01:02:47,490][105692] Updated weights for policy 0, policy_version 1325935 (0.0005) [2023-12-27 01:02:47,973][105620] Updated weights for policy 1, policy_version 1327913 (0.0008) [2023-12-27 01:02:48,029][105620] Updated weights for policy 1, policy_version 1327923 (0.0011) [2023-12-27 01:02:48,084][105620] Updated weights for policy 1, policy_version 1327933 (0.0010) [2023-12-27 01:02:48,098][105692] Updated weights for policy 0, policy_version 1325945 (0.0010) [2023-12-27 01:02:48,155][105692] Updated weights for policy 0, policy_version 1325955 (0.0010) [2023-12-27 01:02:48,213][105692] Updated weights for policy 0, policy_version 1325965 (0.0010) [2023-12-27 01:02:48,834][105620] Updated weights for policy 1, policy_version 1327943 (0.0008) [2023-12-27 01:02:48,891][105620] Updated weights for policy 1, policy_version 1327953 (0.0007) [2023-12-27 01:02:48,942][105692] Updated weights for policy 0, policy_version 1325975 (0.0011) [2023-12-27 01:02:48,947][105620] Updated weights for policy 1, policy_version 1327963 (0.0011) [2023-12-27 01:02:48,987][105692] Updated weights for policy 0, policy_version 1325985 (0.0010) [2023-12-27 01:02:49,035][105692] Updated weights for policy 0, policy_version 1325995 (0.0010) [2023-12-27 01:02:49,675][105620] Updated weights for policy 1, policy_version 1327973 (0.0009) [2023-12-27 01:02:49,728][105620] Updated weights for policy 1, policy_version 1327983 (0.0007) [2023-12-27 01:02:49,773][105692] Updated weights for policy 0, policy_version 1326005 (0.0010) [2023-12-27 01:02:49,790][105620] Updated weights for policy 1, policy_version 1327993 (0.0008) [2023-12-27 01:02:49,834][105692] Updated weights for policy 0, policy_version 1326015 (0.0010) [2023-12-27 01:02:49,897][105692] Updated weights for policy 0, policy_version 1326025 (0.0010) [2023-12-27 01:02:50,439][105620] Updated weights for policy 1, policy_version 1328003 (0.0008) [2023-12-27 01:02:50,502][105620] Updated weights for policy 1, policy_version 1328013 (0.0011) [2023-12-27 01:02:50,548][105620] Updated weights for policy 1, policy_version 1328023 (0.0010) [2023-12-27 01:02:50,597][105692] Updated weights for policy 0, policy_version 1326035 (0.0011) [2023-12-27 01:02:50,660][105692] Updated weights for policy 0, policy_version 1326045 (0.0009) [2023-12-27 01:02:50,723][105692] Updated weights for policy 0, policy_version 1326055 (0.0006) [2023-12-27 01:02:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19383.1). Total num frames: 679542784. Throughput: 0: 9758.7, 1: 9864.9. Samples: 679530872. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:02:51,063][104569] Avg episode reward: [(0, '9085.922'), (1, '9182.062')] [2023-12-27 01:02:51,269][105620] Updated weights for policy 1, policy_version 1328033 (0.0008) [2023-12-27 01:02:51,321][105620] Updated weights for policy 1, policy_version 1328043 (0.0010) [2023-12-27 01:02:51,329][105692] Updated weights for policy 0, policy_version 1326065 (0.0006) [2023-12-27 01:02:51,390][105620] Updated weights for policy 1, policy_version 1328053 (0.0009) [2023-12-27 01:02:51,401][105692] Updated weights for policy 0, policy_version 1326075 (0.0010) [2023-12-27 01:02:51,451][105620] Updated weights for policy 1, policy_version 1328063 (0.0008) [2023-12-27 01:02:51,463][105692] Updated weights for policy 0, policy_version 1326085 (0.0007) [2023-12-27 01:02:51,523][105692] Updated weights for policy 0, policy_version 1326095 (0.0006) [2023-12-27 01:02:52,124][105620] Updated weights for policy 1, policy_version 1328073 (0.0006) [2023-12-27 01:02:52,164][105692] Updated weights for policy 0, policy_version 1326105 (0.0009) [2023-12-27 01:02:52,183][105620] Updated weights for policy 1, policy_version 1328083 (0.0007) [2023-12-27 01:02:52,228][105692] Updated weights for policy 0, policy_version 1326115 (0.0010) [2023-12-27 01:02:52,247][105620] Updated weights for policy 1, policy_version 1328093 (0.0007) [2023-12-27 01:02:52,293][105692] Updated weights for policy 0, policy_version 1326125 (0.0007) [2023-12-27 01:02:52,860][105620] Updated weights for policy 1, policy_version 1328103 (0.0009) [2023-12-27 01:02:52,919][105620] Updated weights for policy 1, policy_version 1328113 (0.0008) [2023-12-27 01:02:52,987][105620] Updated weights for policy 1, policy_version 1328123 (0.0009) [2023-12-27 01:02:53,068][105692] Updated weights for policy 0, policy_version 1326135 (0.0009) [2023-12-27 01:02:53,122][105692] Updated weights for policy 0, policy_version 1326146 (0.0010) [2023-12-27 01:02:53,152][105585] KL-divergence is very high: 144.7551 [2023-12-27 01:02:53,166][105692] Updated weights for policy 0, policy_version 1326156 (0.0008) [2023-12-27 01:02:53,184][105585] KL-divergence is very high: 174.1274 [2023-12-27 01:02:53,713][105620] Updated weights for policy 1, policy_version 1328133 (0.0011) [2023-12-27 01:02:53,772][105620] Updated weights for policy 1, policy_version 1328143 (0.0010) [2023-12-27 01:02:53,837][105620] Updated weights for policy 1, policy_version 1328153 (0.0010) [2023-12-27 01:02:53,864][105692] Updated weights for policy 0, policy_version 1326166 (0.0007) [2023-12-27 01:02:53,922][105692] Updated weights for policy 0, policy_version 1326176 (0.0008) [2023-12-27 01:02:53,985][105692] Updated weights for policy 0, policy_version 1326186 (0.0009) [2023-12-27 01:02:54,493][105620] Updated weights for policy 1, policy_version 1328163 (0.0008) [2023-12-27 01:02:54,558][105620] Updated weights for policy 1, policy_version 1328173 (0.0008) [2023-12-27 01:02:54,618][105620] Updated weights for policy 1, policy_version 1328183 (0.0011) [2023-12-27 01:02:54,686][105692] Updated weights for policy 0, policy_version 1326196 (0.0009) [2023-12-27 01:02:54,746][105692] Updated weights for policy 0, policy_version 1326206 (0.0010) [2023-12-27 01:02:54,807][105692] Updated weights for policy 0, policy_version 1326216 (0.0010) [2023-12-27 01:02:55,345][105620] Updated weights for policy 1, policy_version 1328193 (0.0011) [2023-12-27 01:02:55,406][105620] Updated weights for policy 1, policy_version 1328203 (0.0010) [2023-12-27 01:02:55,437][105692] Updated weights for policy 0, policy_version 1326226 (0.0007) [2023-12-27 01:02:55,458][105620] Updated weights for policy 1, policy_version 1328213 (0.0010) [2023-12-27 01:02:55,488][105692] Updated weights for policy 0, policy_version 1326236 (0.0010) [2023-12-27 01:02:55,505][105620] Updated weights for policy 1, policy_version 1328223 (0.0010) [2023-12-27 01:02:55,540][105692] Updated weights for policy 0, policy_version 1326246 (0.0010) [2023-12-27 01:02:55,605][105692] Updated weights for policy 0, policy_version 1326256 (0.0010) [2023-12-27 01:02:56,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 679641088. Throughput: 0: 9839.6, 1: 9951.7. Samples: 679651472. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:02:56,063][104569] Avg episode reward: [(0, '8461.470'), (1, '9178.714')] [2023-12-27 01:02:56,220][105692] Updated weights for policy 0, policy_version 1326266 (0.0005) [2023-12-27 01:02:56,257][105620] Updated weights for policy 1, policy_version 1328233 (0.0010) [2023-12-27 01:02:56,270][105692] Updated weights for policy 0, policy_version 1326276 (0.0005) [2023-12-27 01:02:56,319][105692] Updated weights for policy 0, policy_version 1326286 (0.0005) [2023-12-27 01:02:56,328][105620] Updated weights for policy 1, policy_version 1328243 (0.0010) [2023-12-27 01:02:56,383][105620] Updated weights for policy 1, policy_version 1328253 (0.0010) [2023-12-27 01:02:56,901][105692] Updated weights for policy 0, policy_version 1326296 (0.0005) [2023-12-27 01:02:56,965][105692] Updated weights for policy 0, policy_version 1326306 (0.0005) [2023-12-27 01:02:57,021][105692] Updated weights for policy 0, policy_version 1326316 (0.0005) [2023-12-27 01:02:57,111][105620] Updated weights for policy 1, policy_version 1328263 (0.0010) [2023-12-27 01:02:57,166][105620] Updated weights for policy 1, policy_version 1328273 (0.0010) [2023-12-27 01:02:57,213][105620] Updated weights for policy 1, policy_version 1328283 (0.0010) [2023-12-27 01:02:57,606][105692] Updated weights for policy 0, policy_version 1326326 (0.0007) [2023-12-27 01:02:57,659][105692] Updated weights for policy 0, policy_version 1326336 (0.0005) [2023-12-27 01:02:57,704][105692] Updated weights for policy 0, policy_version 1326346 (0.0005) [2023-12-27 01:02:57,967][105620] Updated weights for policy 1, policy_version 1328293 (0.0010) [2023-12-27 01:02:58,021][105620] Updated weights for policy 1, policy_version 1328303 (0.0010) [2023-12-27 01:02:58,072][105620] Updated weights for policy 1, policy_version 1328313 (0.0010) [2023-12-27 01:02:58,255][105692] Updated weights for policy 0, policy_version 1326356 (0.0008) [2023-12-27 01:02:58,313][105692] Updated weights for policy 0, policy_version 1326366 (0.0010) [2023-12-27 01:02:58,384][105692] Updated weights for policy 0, policy_version 1326376 (0.0008) [2023-12-27 01:02:58,776][105620] Updated weights for policy 1, policy_version 1328323 (0.0010) [2023-12-27 01:02:58,842][105620] Updated weights for policy 1, policy_version 1328333 (0.0009) [2023-12-27 01:02:58,913][105620] Updated weights for policy 1, policy_version 1328343 (0.0009) [2023-12-27 01:02:59,206][105692] Updated weights for policy 0, policy_version 1326386 (0.0011) [2023-12-27 01:02:59,279][105692] Updated weights for policy 0, policy_version 1326396 (0.0010) [2023-12-27 01:02:59,343][105692] Updated weights for policy 0, policy_version 1326406 (0.0011) [2023-12-27 01:02:59,410][105692] Updated weights for policy 0, policy_version 1326416 (0.0011) [2023-12-27 01:02:59,691][105620] Updated weights for policy 1, policy_version 1328353 (0.0011) [2023-12-27 01:02:59,761][105620] Updated weights for policy 1, policy_version 1328363 (0.0011) [2023-12-27 01:02:59,823][105620] Updated weights for policy 1, policy_version 1328373 (0.0010) [2023-12-27 01:02:59,892][105620] Updated weights for policy 1, policy_version 1328383 (0.0011) [2023-12-27 01:03:00,127][105692] Updated weights for policy 0, policy_version 1326426 (0.0006) [2023-12-27 01:03:00,190][105692] Updated weights for policy 0, policy_version 1326436 (0.0010) [2023-12-27 01:03:00,244][105692] Updated weights for policy 0, policy_version 1326446 (0.0010) [2023-12-27 01:03:00,653][105620] Updated weights for policy 1, policy_version 1328393 (0.0008) [2023-12-27 01:03:00,715][105620] Updated weights for policy 1, policy_version 1328403 (0.0009) [2023-12-27 01:03:00,775][105620] Updated weights for policy 1, policy_version 1328413 (0.0010) [2023-12-27 01:03:00,836][105692] Updated weights for policy 0, policy_version 1326456 (0.0006) [2023-12-27 01:03:00,894][105692] Updated weights for policy 0, policy_version 1326466 (0.0005) [2023-12-27 01:03:00,962][105692] Updated weights for policy 0, policy_version 1326476 (0.0010) [2023-12-27 01:03:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.3, 300 sec: 19410.9). Total num frames: 679747584. Throughput: 0: 9987.5, 1: 9894.3. Samples: 679714156. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:03:01,063][104569] Avg episode reward: [(0, '8281.082'), (1, '9265.624')] [2023-12-27 01:03:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001326480_339632128.pth... [2023-12-27 01:03:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001328416_340115456.pth... [2023-12-27 01:03:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001325296_339329024.pth [2023-12-27 01:03:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001327264_339820544.pth [2023-12-27 01:03:01,448][105620] Updated weights for policy 1, policy_version 1328423 (0.0008) [2023-12-27 01:03:01,522][105620] Updated weights for policy 1, policy_version 1328433 (0.0009) [2023-12-27 01:03:01,591][105620] Updated weights for policy 1, policy_version 1328443 (0.0006) [2023-12-27 01:03:01,634][105692] Updated weights for policy 0, policy_version 1326486 (0.0009) [2023-12-27 01:03:01,690][105692] Updated weights for policy 0, policy_version 1326496 (0.0006) [2023-12-27 01:03:01,761][105692] Updated weights for policy 0, policy_version 1326506 (0.0008) [2023-12-27 01:03:02,271][105620] Updated weights for policy 1, policy_version 1328453 (0.0008) [2023-12-27 01:03:02,327][105620] Updated weights for policy 1, policy_version 1328463 (0.0006) [2023-12-27 01:03:02,399][105620] Updated weights for policy 1, policy_version 1328473 (0.0008) [2023-12-27 01:03:02,453][105692] Updated weights for policy 0, policy_version 1326516 (0.0009) [2023-12-27 01:03:02,513][105692] Updated weights for policy 0, policy_version 1326526 (0.0009) [2023-12-27 01:03:02,572][105692] Updated weights for policy 0, policy_version 1326536 (0.0009) [2023-12-27 01:03:03,025][105620] Updated weights for policy 1, policy_version 1328483 (0.0007) [2023-12-27 01:03:03,067][105620] Updated weights for policy 1, policy_version 1328493 (0.0005) [2023-12-27 01:03:03,112][105620] Updated weights for policy 1, policy_version 1328503 (0.0005) [2023-12-27 01:03:03,187][105692] Updated weights for policy 0, policy_version 1326546 (0.0009) [2023-12-27 01:03:03,239][105692] Updated weights for policy 0, policy_version 1326556 (0.0012) [2023-12-27 01:03:03,289][105692] Updated weights for policy 0, policy_version 1326566 (0.0008) [2023-12-27 01:03:03,343][105692] Updated weights for policy 0, policy_version 1326576 (0.0005) [2023-12-27 01:03:03,678][105620] Updated weights for policy 1, policy_version 1328513 (0.0005) [2023-12-27 01:03:03,738][105620] Updated weights for policy 1, policy_version 1328523 (0.0005) [2023-12-27 01:03:03,793][105620] Updated weights for policy 1, policy_version 1328533 (0.0005) [2023-12-27 01:03:03,841][105620] Updated weights for policy 1, policy_version 1328543 (0.0006) [2023-12-27 01:03:03,961][105692] Updated weights for policy 0, policy_version 1326586 (0.0011) [2023-12-27 01:03:04,020][105692] Updated weights for policy 0, policy_version 1326596 (0.0010) [2023-12-27 01:03:04,078][105692] Updated weights for policy 0, policy_version 1326606 (0.0010) [2023-12-27 01:03:04,472][105620] Updated weights for policy 1, policy_version 1328553 (0.0008) [2023-12-27 01:03:04,520][105620] Updated weights for policy 1, policy_version 1328563 (0.0008) [2023-12-27 01:03:04,575][105620] Updated weights for policy 1, policy_version 1328573 (0.0008) [2023-12-27 01:03:04,811][105692] Updated weights for policy 0, policy_version 1326616 (0.0010) [2023-12-27 01:03:04,868][105692] Updated weights for policy 0, policy_version 1326626 (0.0010) [2023-12-27 01:03:04,915][105692] Updated weights for policy 0, policy_version 1326636 (0.0006) [2023-12-27 01:03:05,422][105620] Updated weights for policy 1, policy_version 1328583 (0.0009) [2023-12-27 01:03:05,483][105620] Updated weights for policy 1, policy_version 1328593 (0.0008) [2023-12-27 01:03:05,490][105692] Updated weights for policy 0, policy_version 1326646 (0.0006) [2023-12-27 01:03:05,535][105620] Updated weights for policy 1, policy_version 1328603 (0.0009) [2023-12-27 01:03:05,542][105692] Updated weights for policy 0, policy_version 1326656 (0.0007) [2023-12-27 01:03:05,591][105692] Updated weights for policy 0, policy_version 1326666 (0.0009) [2023-12-27 01:03:06,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19438.6). Total num frames: 679845888. Throughput: 0: 9999.5, 1: 9878.0. Samples: 679835680. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:03:06,063][104569] Avg episode reward: [(0, '8449.060'), (1, '9267.602')] [2023-12-27 01:03:06,281][105620] Updated weights for policy 1, policy_version 1328613 (0.0007) [2023-12-27 01:03:06,330][105620] Updated weights for policy 1, policy_version 1328623 (0.0009) [2023-12-27 01:03:06,337][105692] Updated weights for policy 0, policy_version 1326676 (0.0009) [2023-12-27 01:03:06,387][105620] Updated weights for policy 1, policy_version 1328633 (0.0008) [2023-12-27 01:03:06,401][105692] Updated weights for policy 0, policy_version 1326686 (0.0006) [2023-12-27 01:03:06,461][105692] Updated weights for policy 0, policy_version 1326696 (0.0007) [2023-12-27 01:03:07,110][105620] Updated weights for policy 1, policy_version 1328643 (0.0007) [2023-12-27 01:03:07,176][105620] Updated weights for policy 1, policy_version 1328653 (0.0009) [2023-12-27 01:03:07,227][105692] Updated weights for policy 0, policy_version 1326706 (0.0008) [2023-12-27 01:03:07,237][105620] Updated weights for policy 1, policy_version 1328663 (0.0008) [2023-12-27 01:03:07,284][105692] Updated weights for policy 0, policy_version 1326716 (0.0007) [2023-12-27 01:03:07,345][105692] Updated weights for policy 0, policy_version 1326726 (0.0009) [2023-12-27 01:03:07,404][105692] Updated weights for policy 0, policy_version 1326736 (0.0009) [2023-12-27 01:03:08,003][105620] Updated weights for policy 1, policy_version 1328673 (0.0007) [2023-12-27 01:03:08,067][105620] Updated weights for policy 1, policy_version 1328683 (0.0009) [2023-12-27 01:03:08,122][105692] Updated weights for policy 0, policy_version 1326746 (0.0008) [2023-12-27 01:03:08,128][105620] Updated weights for policy 1, policy_version 1328693 (0.0008) [2023-12-27 01:03:08,171][105692] Updated weights for policy 0, policy_version 1326756 (0.0005) [2023-12-27 01:03:08,189][105620] Updated weights for policy 1, policy_version 1328703 (0.0008) [2023-12-27 01:03:08,222][105692] Updated weights for policy 0, policy_version 1326766 (0.0007) [2023-12-27 01:03:08,940][105620] Updated weights for policy 1, policy_version 1328713 (0.0006) [2023-12-27 01:03:08,999][105692] Updated weights for policy 0, policy_version 1326776 (0.0009) [2023-12-27 01:03:09,007][105620] Updated weights for policy 1, policy_version 1328723 (0.0005) [2023-12-27 01:03:09,063][105692] Updated weights for policy 0, policy_version 1326786 (0.0008) [2023-12-27 01:03:09,081][105620] Updated weights for policy 1, policy_version 1328733 (0.0007) [2023-12-27 01:03:09,122][105692] Updated weights for policy 0, policy_version 1326796 (0.0008) [2023-12-27 01:03:09,866][105620] Updated weights for policy 1, policy_version 1328743 (0.0009) [2023-12-27 01:03:09,934][105620] Updated weights for policy 1, policy_version 1328753 (0.0008) [2023-12-27 01:03:09,960][105692] Updated weights for policy 0, policy_version 1326806 (0.0008) [2023-12-27 01:03:09,995][105620] Updated weights for policy 1, policy_version 1328763 (0.0008) [2023-12-27 01:03:10,013][105692] Updated weights for policy 0, policy_version 1326816 (0.0008) [2023-12-27 01:03:10,077][105692] Updated weights for policy 0, policy_version 1326826 (0.0006) [2023-12-27 01:03:10,735][105692] Updated weights for policy 0, policy_version 1326836 (0.0007) [2023-12-27 01:03:10,751][105620] Updated weights for policy 1, policy_version 1328773 (0.0008) [2023-12-27 01:03:10,790][105692] Updated weights for policy 0, policy_version 1326846 (0.0009) [2023-12-27 01:03:10,813][105620] Updated weights for policy 1, policy_version 1328783 (0.0008) [2023-12-27 01:03:10,837][105692] Updated weights for policy 0, policy_version 1326856 (0.0007) [2023-12-27 01:03:10,863][105620] Updated weights for policy 1, policy_version 1328793 (0.0006) [2023-12-27 01:03:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 679944192. Throughput: 0: 10004.6, 1: 9834.2. Samples: 679948504. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:03:11,062][104569] Avg episode reward: [(0, '8370.944'), (1, '9179.479')] [2023-12-27 01:03:11,619][105620] Updated weights for policy 1, policy_version 1328803 (0.0009) [2023-12-27 01:03:11,626][105692] Updated weights for policy 0, policy_version 1326866 (0.0006) [2023-12-27 01:03:11,677][105620] Updated weights for policy 1, policy_version 1328813 (0.0007) [2023-12-27 01:03:11,687][105692] Updated weights for policy 0, policy_version 1326876 (0.0009) [2023-12-27 01:03:11,715][105586] KL-divergence is very high: 111.9099 [2023-12-27 01:03:11,736][105620] Updated weights for policy 1, policy_version 1328823 (0.0006) [2023-12-27 01:03:11,746][105692] Updated weights for policy 0, policy_version 1326886 (0.0009) [2023-12-27 01:03:11,771][105586] KL-divergence is very high: 130.7420 [2023-12-27 01:03:11,814][105692] Updated weights for policy 0, policy_version 1326896 (0.0009) [2023-12-27 01:03:12,453][105620] Updated weights for policy 1, policy_version 1328833 (0.0008) [2023-12-27 01:03:12,508][105620] Updated weights for policy 1, policy_version 1328843 (0.0009) [2023-12-27 01:03:12,556][105620] Updated weights for policy 1, policy_version 1328853 (0.0009) [2023-12-27 01:03:12,610][105620] Updated weights for policy 1, policy_version 1328863 (0.0009) [2023-12-27 01:03:12,612][105692] Updated weights for policy 0, policy_version 1326906 (0.0008) [2023-12-27 01:03:12,677][105692] Updated weights for policy 0, policy_version 1326916 (0.0008) [2023-12-27 01:03:12,725][105692] Updated weights for policy 0, policy_version 1326926 (0.0010) [2023-12-27 01:03:13,296][105620] Updated weights for policy 1, policy_version 1328873 (0.0009) [2023-12-27 01:03:13,355][105620] Updated weights for policy 1, policy_version 1328883 (0.0010) [2023-12-27 01:03:13,413][105620] Updated weights for policy 1, policy_version 1328893 (0.0011) [2023-12-27 01:03:13,473][105692] Updated weights for policy 0, policy_version 1326936 (0.0008) [2023-12-27 01:03:13,529][105692] Updated weights for policy 0, policy_version 1326946 (0.0008) [2023-12-27 01:03:13,589][105692] Updated weights for policy 0, policy_version 1326956 (0.0008) [2023-12-27 01:03:14,151][105620] Updated weights for policy 1, policy_version 1328903 (0.0011) [2023-12-27 01:03:14,199][105620] Updated weights for policy 1, policy_version 1328913 (0.0010) [2023-12-27 01:03:14,248][105620] Updated weights for policy 1, policy_version 1328923 (0.0010) [2023-12-27 01:03:14,334][105692] Updated weights for policy 0, policy_version 1326966 (0.0008) [2023-12-27 01:03:14,392][105692] Updated weights for policy 0, policy_version 1326976 (0.0008) [2023-12-27 01:03:14,465][105692] Updated weights for policy 0, policy_version 1326986 (0.0008) [2023-12-27 01:03:14,980][105620] Updated weights for policy 1, policy_version 1328933 (0.0010) [2023-12-27 01:03:15,035][105620] Updated weights for policy 1, policy_version 1328943 (0.0009) [2023-12-27 01:03:15,100][105620] Updated weights for policy 1, policy_version 1328953 (0.0009) [2023-12-27 01:03:15,212][105692] Updated weights for policy 0, policy_version 1326996 (0.0010) [2023-12-27 01:03:15,271][105692] Updated weights for policy 0, policy_version 1327006 (0.0009) [2023-12-27 01:03:15,329][105692] Updated weights for policy 0, policy_version 1327016 (0.0009) [2023-12-27 01:03:15,869][105620] Updated weights for policy 1, policy_version 1328963 (0.0009) [2023-12-27 01:03:15,937][105620] Updated weights for policy 1, policy_version 1328973 (0.0008) [2023-12-27 01:03:16,003][105620] Updated weights for policy 1, policy_version 1328983 (0.0009) [2023-12-27 01:03:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 680034304. Throughput: 0: 9939.3, 1: 9795.1. Samples: 680005060. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:03:16,063][104569] Avg episode reward: [(0, '8461.605'), (1, '9180.381')] [2023-12-27 01:03:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001328992_340262912.pth... [2023-12-27 01:03:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001327840_339968000.pth [2023-12-27 01:03:16,080][105692] Updated weights for policy 0, policy_version 1327026 (0.0009) [2023-12-27 01:03:16,127][105585] KL-divergence is very high: 105.4637 [2023-12-27 01:03:16,133][105692] Updated weights for policy 0, policy_version 1327036 (0.0008) [2023-12-27 01:03:16,165][105585] KL-divergence is very high: 172.5280 [2023-12-27 01:03:16,180][105692] Updated weights for policy 0, policy_version 1327046 (0.0007) [2023-12-27 01:03:16,204][105585] KL-divergence is very high: 159.4259 [2023-12-27 01:03:16,225][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001327056_339779584.pth... [2023-12-27 01:03:16,226][105692] Updated weights for policy 0, policy_version 1327056 (0.0005) [2023-12-27 01:03:16,228][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001325872_339476480.pth [2023-12-27 01:03:16,779][105620] Updated weights for policy 1, policy_version 1328993 (0.0009) [2023-12-27 01:03:16,830][105620] Updated weights for policy 1, policy_version 1329003 (0.0009) [2023-12-27 01:03:16,879][105620] Updated weights for policy 1, policy_version 1329013 (0.0009) [2023-12-27 01:03:16,932][105620] Updated weights for policy 1, policy_version 1329023 (0.0009) [2023-12-27 01:03:16,943][105692] Updated weights for policy 0, policy_version 1327066 (0.0006) [2023-12-27 01:03:17,001][105692] Updated weights for policy 0, policy_version 1327076 (0.0009) [2023-12-27 01:03:17,056][105692] Updated weights for policy 0, policy_version 1327086 (0.0008) [2023-12-27 01:03:17,672][105692] Updated weights for policy 0, policy_version 1327096 (0.0007) [2023-12-27 01:03:17,723][105692] Updated weights for policy 0, policy_version 1327106 (0.0009) [2023-12-27 01:03:17,764][105620] Updated weights for policy 1, policy_version 1329033 (0.0007) [2023-12-27 01:03:17,778][105692] Updated weights for policy 0, policy_version 1327116 (0.0007) [2023-12-27 01:03:17,817][105620] Updated weights for policy 1, policy_version 1329043 (0.0007) [2023-12-27 01:03:17,879][105620] Updated weights for policy 1, policy_version 1329053 (0.0009) [2023-12-27 01:03:18,546][105620] Updated weights for policy 1, policy_version 1329063 (0.0009) [2023-12-27 01:03:18,573][105692] Updated weights for policy 0, policy_version 1327126 (0.0006) [2023-12-27 01:03:18,608][105620] Updated weights for policy 1, policy_version 1329073 (0.0008) [2023-12-27 01:03:18,627][105692] Updated weights for policy 0, policy_version 1327136 (0.0006) [2023-12-27 01:03:18,669][105620] Updated weights for policy 1, policy_version 1329083 (0.0009) [2023-12-27 01:03:18,684][105692] Updated weights for policy 0, policy_version 1327146 (0.0007) [2023-12-27 01:03:19,385][105620] Updated weights for policy 1, policy_version 1329093 (0.0008) [2023-12-27 01:03:19,432][105620] Updated weights for policy 1, policy_version 1329103 (0.0008) [2023-12-27 01:03:19,480][105620] Updated weights for policy 1, policy_version 1329113 (0.0008) [2023-12-27 01:03:19,485][105692] Updated weights for policy 0, policy_version 1327156 (0.0007) [2023-12-27 01:03:19,548][105692] Updated weights for policy 0, policy_version 1327166 (0.0006) [2023-12-27 01:03:19,617][105692] Updated weights for policy 0, policy_version 1327176 (0.0008) [2023-12-27 01:03:20,238][105620] Updated weights for policy 1, policy_version 1329123 (0.0009) [2023-12-27 01:03:20,300][105620] Updated weights for policy 1, policy_version 1329133 (0.0008) [2023-12-27 01:03:20,361][105620] Updated weights for policy 1, policy_version 1329143 (0.0006) [2023-12-27 01:03:20,393][105692] Updated weights for policy 0, policy_version 1327186 (0.0009) [2023-12-27 01:03:20,456][105692] Updated weights for policy 0, policy_version 1327196 (0.0009) [2023-12-27 01:03:20,513][105692] Updated weights for policy 0, policy_version 1327206 (0.0009) [2023-12-27 01:03:20,579][105692] Updated weights for policy 0, policy_version 1327216 (0.0009) [2023-12-27 01:03:21,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 680124416. Throughput: 0: 9874.8, 1: 9668.0. Samples: 680118648. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:03:21,062][104569] Avg episode reward: [(0, '8814.655'), (1, '9088.266')] [2023-12-27 01:03:21,073][105620] Updated weights for policy 1, policy_version 1329153 (0.0006) [2023-12-27 01:03:21,139][105620] Updated weights for policy 1, policy_version 1329163 (0.0008) [2023-12-27 01:03:21,202][105620] Updated weights for policy 1, policy_version 1329173 (0.0009) [2023-12-27 01:03:21,266][105620] Updated weights for policy 1, policy_version 1329183 (0.0009) [2023-12-27 01:03:21,406][105692] Updated weights for policy 0, policy_version 1327226 (0.0009) [2023-12-27 01:03:21,468][105692] Updated weights for policy 0, policy_version 1327236 (0.0007) [2023-12-27 01:03:21,534][105692] Updated weights for policy 0, policy_version 1327246 (0.0007) [2023-12-27 01:03:21,971][105620] Updated weights for policy 1, policy_version 1329193 (0.0009) [2023-12-27 01:03:22,028][105620] Updated weights for policy 1, policy_version 1329203 (0.0009) [2023-12-27 01:03:22,092][105620] Updated weights for policy 1, policy_version 1329213 (0.0009) [2023-12-27 01:03:22,335][105692] Updated weights for policy 0, policy_version 1327256 (0.0010) [2023-12-27 01:03:22,406][105692] Updated weights for policy 0, policy_version 1327266 (0.0007) [2023-12-27 01:03:22,463][105692] Updated weights for policy 0, policy_version 1327276 (0.0005) [2023-12-27 01:03:22,747][105620] Updated weights for policy 1, policy_version 1329223 (0.0009) [2023-12-27 01:03:22,803][105620] Updated weights for policy 1, policy_version 1329233 (0.0010) [2023-12-27 01:03:22,857][105620] Updated weights for policy 1, policy_version 1329244 (0.0008) [2023-12-27 01:03:23,169][105692] Updated weights for policy 0, policy_version 1327286 (0.0009) [2023-12-27 01:03:23,229][105692] Updated weights for policy 0, policy_version 1327296 (0.0008) [2023-12-27 01:03:23,294][105692] Updated weights for policy 0, policy_version 1327306 (0.0009) [2023-12-27 01:03:23,630][105620] Updated weights for policy 1, policy_version 1329254 (0.0008) [2023-12-27 01:03:23,688][105620] Updated weights for policy 1, policy_version 1329264 (0.0009) [2023-12-27 01:03:23,739][105620] Updated weights for policy 1, policy_version 1329274 (0.0009) [2023-12-27 01:03:23,977][105692] Updated weights for policy 0, policy_version 1327316 (0.0008) [2023-12-27 01:03:24,039][105692] Updated weights for policy 0, policy_version 1327326 (0.0010) [2023-12-27 01:03:24,097][105692] Updated weights for policy 0, policy_version 1327336 (0.0010) [2023-12-27 01:03:24,551][105620] Updated weights for policy 1, policy_version 1329284 (0.0009) [2023-12-27 01:03:24,612][105620] Updated weights for policy 1, policy_version 1329294 (0.0007) [2023-12-27 01:03:24,682][105620] Updated weights for policy 1, policy_version 1329304 (0.0006) [2023-12-27 01:03:24,804][105692] Updated weights for policy 0, policy_version 1327346 (0.0010) [2023-12-27 01:03:24,863][105692] Updated weights for policy 0, policy_version 1327356 (0.0010) [2023-12-27 01:03:24,914][105692] Updated weights for policy 0, policy_version 1327366 (0.0010) [2023-12-27 01:03:24,977][105692] Updated weights for policy 0, policy_version 1327376 (0.0011) [2023-12-27 01:03:25,368][105620] Updated weights for policy 1, policy_version 1329314 (0.0007) [2023-12-27 01:03:25,420][105620] Updated weights for policy 1, policy_version 1329324 (0.0010) [2023-12-27 01:03:25,471][105620] Updated weights for policy 1, policy_version 1329334 (0.0010) [2023-12-27 01:03:25,520][105620] Updated weights for policy 1, policy_version 1329344 (0.0006) [2023-12-27 01:03:25,653][105692] Updated weights for policy 0, policy_version 1327386 (0.0005) [2023-12-27 01:03:25,712][105692] Updated weights for policy 0, policy_version 1327396 (0.0005) [2023-12-27 01:03:25,784][105692] Updated weights for policy 0, policy_version 1327406 (0.0007) [2023-12-27 01:03:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 680222720. Throughput: 0: 9842.9, 1: 9634.8. Samples: 680233436. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:03:26,062][104569] Avg episode reward: [(0, '8270.104'), (1, '9177.038')] [2023-12-27 01:03:26,128][105620] Updated weights for policy 1, policy_version 1329354 (0.0011) [2023-12-27 01:03:26,190][105620] Updated weights for policy 1, policy_version 1329364 (0.0010) [2023-12-27 01:03:26,244][105620] Updated weights for policy 1, policy_version 1329374 (0.0010) [2023-12-27 01:03:26,397][105692] Updated weights for policy 0, policy_version 1327416 (0.0010) [2023-12-27 01:03:26,441][105692] Updated weights for policy 0, policy_version 1327426 (0.0010) [2023-12-27 01:03:26,489][105692] Updated weights for policy 0, policy_version 1327436 (0.0010) [2023-12-27 01:03:26,955][105620] Updated weights for policy 1, policy_version 1329384 (0.0010) [2023-12-27 01:03:27,013][105620] Updated weights for policy 1, policy_version 1329394 (0.0010) [2023-12-27 01:03:27,077][105620] Updated weights for policy 1, policy_version 1329404 (0.0010) [2023-12-27 01:03:27,253][105692] Updated weights for policy 0, policy_version 1327446 (0.0010) [2023-12-27 01:03:27,297][105692] Updated weights for policy 0, policy_version 1327456 (0.0010) [2023-12-27 01:03:27,359][105692] Updated weights for policy 0, policy_version 1327466 (0.0010) [2023-12-27 01:03:27,740][105620] Updated weights for policy 1, policy_version 1329414 (0.0007) [2023-12-27 01:03:27,798][105620] Updated weights for policy 1, policy_version 1329424 (0.0005) [2023-12-27 01:03:27,862][105620] Updated weights for policy 1, policy_version 1329434 (0.0005) [2023-12-27 01:03:28,100][105692] Updated weights for policy 0, policy_version 1327476 (0.0010) [2023-12-27 01:03:28,152][105692] Updated weights for policy 0, policy_version 1327486 (0.0010) [2023-12-27 01:03:28,202][105692] Updated weights for policy 0, policy_version 1327496 (0.0010) [2023-12-27 01:03:28,362][105620] Updated weights for policy 1, policy_version 1329444 (0.0006) [2023-12-27 01:03:28,419][105620] Updated weights for policy 1, policy_version 1329454 (0.0005) [2023-12-27 01:03:28,470][105620] Updated weights for policy 1, policy_version 1329464 (0.0007) [2023-12-27 01:03:29,007][105692] Updated weights for policy 0, policy_version 1327506 (0.0010) [2023-12-27 01:03:29,061][105692] Updated weights for policy 0, policy_version 1327516 (0.0007) [2023-12-27 01:03:29,120][105692] Updated weights for policy 0, policy_version 1327526 (0.0005) [2023-12-27 01:03:29,126][105620] Updated weights for policy 1, policy_version 1329474 (0.0008) [2023-12-27 01:03:29,188][105692] Updated weights for policy 0, policy_version 1327536 (0.0005) [2023-12-27 01:03:29,191][105620] Updated weights for policy 1, policy_version 1329484 (0.0008) [2023-12-27 01:03:29,253][105620] Updated weights for policy 1, policy_version 1329494 (0.0009) [2023-12-27 01:03:29,313][105620] Updated weights for policy 1, policy_version 1329504 (0.0010) [2023-12-27 01:03:29,816][105692] Updated weights for policy 0, policy_version 1327546 (0.0006) [2023-12-27 01:03:29,870][105692] Updated weights for policy 0, policy_version 1327556 (0.0008) [2023-12-27 01:03:29,921][105692] Updated weights for policy 0, policy_version 1327566 (0.0007) [2023-12-27 01:03:29,992][105620] Updated weights for policy 1, policy_version 1329514 (0.0006) [2023-12-27 01:03:30,051][105620] Updated weights for policy 1, policy_version 1329524 (0.0006) [2023-12-27 01:03:30,119][105620] Updated weights for policy 1, policy_version 1329534 (0.0006) [2023-12-27 01:03:30,570][105692] Updated weights for policy 0, policy_version 1327576 (0.0010) [2023-12-27 01:03:30,614][105692] Updated weights for policy 0, policy_version 1327586 (0.0010) [2023-12-27 01:03:30,672][105692] Updated weights for policy 0, policy_version 1327596 (0.0010) [2023-12-27 01:03:30,773][105620] Updated weights for policy 1, policy_version 1329544 (0.0009) [2023-12-27 01:03:30,820][105620] Updated weights for policy 1, policy_version 1329554 (0.0010) [2023-12-27 01:03:30,868][105620] Updated weights for policy 1, policy_version 1329564 (0.0010) [2023-12-27 01:03:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.9, 300 sec: 19466.4). Total num frames: 680329216. Throughput: 0: 9835.6, 1: 9742.9. Samples: 680294696. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:03:31,062][104569] Avg episode reward: [(0, '8552.649'), (1, '9269.028')] [2023-12-27 01:03:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001327600_339918848.pth... [2023-12-27 01:03:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001329568_340410368.pth... [2023-12-27 01:03:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001326480_339632128.pth [2023-12-27 01:03:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001328416_340115456.pth [2023-12-27 01:03:31,351][105692] Updated weights for policy 0, policy_version 1327606 (0.0009) [2023-12-27 01:03:31,417][105692] Updated weights for policy 0, policy_version 1327616 (0.0009) [2023-12-27 01:03:31,478][105692] Updated weights for policy 0, policy_version 1327626 (0.0009) [2023-12-27 01:03:31,665][105620] Updated weights for policy 1, policy_version 1329574 (0.0009) [2023-12-27 01:03:31,725][105620] Updated weights for policy 1, policy_version 1329584 (0.0009) [2023-12-27 01:03:31,792][105620] Updated weights for policy 1, policy_version 1329594 (0.0009) [2023-12-27 01:03:32,210][105692] Updated weights for policy 0, policy_version 1327636 (0.0007) [2023-12-27 01:03:32,278][105692] Updated weights for policy 0, policy_version 1327646 (0.0008) [2023-12-27 01:03:32,334][105692] Updated weights for policy 0, policy_version 1327656 (0.0008) [2023-12-27 01:03:32,544][105620] Updated weights for policy 1, policy_version 1329604 (0.0008) [2023-12-27 01:03:32,595][105620] Updated weights for policy 1, policy_version 1329614 (0.0009) [2023-12-27 01:03:32,650][105620] Updated weights for policy 1, policy_version 1329624 (0.0009) [2023-12-27 01:03:32,989][105692] Updated weights for policy 0, policy_version 1327666 (0.0007) [2023-12-27 01:03:33,060][105692] Updated weights for policy 0, policy_version 1327676 (0.0006) [2023-12-27 01:03:33,111][105692] Updated weights for policy 0, policy_version 1327686 (0.0008) [2023-12-27 01:03:33,391][105620] Updated weights for policy 1, policy_version 1329634 (0.0010) [2023-12-27 01:03:33,446][105620] Updated weights for policy 1, policy_version 1329644 (0.0010) [2023-12-27 01:03:33,493][105620] Updated weights for policy 1, policy_version 1329654 (0.0010) [2023-12-27 01:03:33,547][105620] Updated weights for policy 1, policy_version 1329664 (0.0010) [2023-12-27 01:03:33,791][105692] Updated weights for policy 0, policy_version 1327697 (0.0010) [2023-12-27 01:03:33,859][105692] Updated weights for policy 0, policy_version 1327707 (0.0010) [2023-12-27 01:03:33,923][105692] Updated weights for policy 0, policy_version 1327717 (0.0009) [2023-12-27 01:03:33,986][105692] Updated weights for policy 0, policy_version 1327727 (0.0010) [2023-12-27 01:03:34,190][105620] Updated weights for policy 1, policy_version 1329674 (0.0010) [2023-12-27 01:03:34,252][105620] Updated weights for policy 1, policy_version 1329684 (0.0010) [2023-12-27 01:03:34,311][105620] Updated weights for policy 1, policy_version 1329694 (0.0010) [2023-12-27 01:03:34,780][105692] Updated weights for policy 0, policy_version 1327737 (0.0009) [2023-12-27 01:03:34,837][105692] Updated weights for policy 0, policy_version 1327747 (0.0009) [2023-12-27 01:03:34,888][105692] Updated weights for policy 0, policy_version 1327757 (0.0009) [2023-12-27 01:03:35,030][105620] Updated weights for policy 1, policy_version 1329704 (0.0010) [2023-12-27 01:03:35,085][105620] Updated weights for policy 1, policy_version 1329714 (0.0010) [2023-12-27 01:03:35,136][105620] Updated weights for policy 1, policy_version 1329724 (0.0009) [2023-12-27 01:03:35,634][105692] Updated weights for policy 0, policy_version 1327767 (0.0008) [2023-12-27 01:03:35,692][105692] Updated weights for policy 0, policy_version 1327777 (0.0009) [2023-12-27 01:03:35,759][105692] Updated weights for policy 0, policy_version 1327787 (0.0008) [2023-12-27 01:03:35,924][105620] Updated weights for policy 1, policy_version 1329734 (0.0009) [2023-12-27 01:03:35,977][105620] Updated weights for policy 1, policy_version 1329744 (0.0009) [2023-12-27 01:03:36,037][105620] Updated weights for policy 1, policy_version 1329754 (0.0009) [2023-12-27 01:03:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 680419328. Throughput: 0: 9837.5, 1: 9768.5. Samples: 680413140. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:03:36,062][104569] Avg episode reward: [(0, '8642.062'), (1, '9357.159')] [2023-12-27 01:03:36,359][105692] Updated weights for policy 0, policy_version 1327797 (0.0007) [2023-12-27 01:03:36,425][105692] Updated weights for policy 0, policy_version 1327807 (0.0007) [2023-12-27 01:03:36,488][105692] Updated weights for policy 0, policy_version 1327817 (0.0009) [2023-12-27 01:03:36,836][105620] Updated weights for policy 1, policy_version 1329764 (0.0008) [2023-12-27 01:03:36,899][105620] Updated weights for policy 1, policy_version 1329774 (0.0008) [2023-12-27 01:03:36,959][105620] Updated weights for policy 1, policy_version 1329784 (0.0010) [2023-12-27 01:03:37,223][105692] Updated weights for policy 0, policy_version 1327827 (0.0008) [2023-12-27 01:03:37,271][105692] Updated weights for policy 0, policy_version 1327837 (0.0010) [2023-12-27 01:03:37,326][105692] Updated weights for policy 0, policy_version 1327847 (0.0011) [2023-12-27 01:03:37,708][105620] Updated weights for policy 1, policy_version 1329794 (0.0010) [2023-12-27 01:03:37,764][105620] Updated weights for policy 1, policy_version 1329804 (0.0010) [2023-12-27 01:03:37,813][105620] Updated weights for policy 1, policy_version 1329814 (0.0010) [2023-12-27 01:03:37,865][105620] Updated weights for policy 1, policy_version 1329824 (0.0010) [2023-12-27 01:03:37,994][105692] Updated weights for policy 0, policy_version 1327857 (0.0010) [2023-12-27 01:03:38,059][105692] Updated weights for policy 0, policy_version 1327867 (0.0010) [2023-12-27 01:03:38,111][105692] Updated weights for policy 0, policy_version 1327877 (0.0011) [2023-12-27 01:03:38,169][105692] Updated weights for policy 0, policy_version 1327887 (0.0010) [2023-12-27 01:03:38,541][105620] Updated weights for policy 1, policy_version 1329834 (0.0007) [2023-12-27 01:03:38,599][105620] Updated weights for policy 1, policy_version 1329844 (0.0008) [2023-12-27 01:03:38,649][105620] Updated weights for policy 1, policy_version 1329854 (0.0006) [2023-12-27 01:03:38,935][105692] Updated weights for policy 0, policy_version 1327897 (0.0010) [2023-12-27 01:03:38,942][105585] KL-divergence is very high: 132.0161 [2023-12-27 01:03:38,989][105585] KL-divergence is very high: 353.0109 [2023-12-27 01:03:38,991][105692] Updated weights for policy 0, policy_version 1327907 (0.0011) [2023-12-27 01:03:39,020][105585] KL-divergence is very high: 121.2935 [2023-12-27 01:03:39,039][105585] KL-divergence is very high: 416.2465 [2023-12-27 01:03:39,051][105692] Updated weights for policy 0, policy_version 1327917 (0.0011) [2023-12-27 01:03:39,378][105620] Updated weights for policy 1, policy_version 1329864 (0.0010) [2023-12-27 01:03:39,456][105620] Updated weights for policy 1, policy_version 1329874 (0.0011) [2023-12-27 01:03:39,519][105620] Updated weights for policy 1, policy_version 1329884 (0.0011) [2023-12-27 01:03:39,793][105692] Updated weights for policy 0, policy_version 1327927 (0.0009) [2023-12-27 01:03:39,857][105692] Updated weights for policy 0, policy_version 1327937 (0.0009) [2023-12-27 01:03:39,925][105692] Updated weights for policy 0, policy_version 1327947 (0.0009) [2023-12-27 01:03:40,308][105620] Updated weights for policy 1, policy_version 1329894 (0.0011) [2023-12-27 01:03:40,370][105620] Updated weights for policy 1, policy_version 1329904 (0.0007) [2023-12-27 01:03:40,440][105620] Updated weights for policy 1, policy_version 1329914 (0.0006) [2023-12-27 01:03:40,682][105692] Updated weights for policy 0, policy_version 1327957 (0.0008) [2023-12-27 01:03:40,733][105692] Updated weights for policy 0, policy_version 1327967 (0.0009) [2023-12-27 01:03:40,794][105692] Updated weights for policy 0, policy_version 1327977 (0.0008) [2023-12-27 01:03:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 680517632. Throughput: 0: 9784.4, 1: 9671.4. Samples: 680526980. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:03:41,062][104569] Avg episode reward: [(0, '8456.326'), (1, '9268.758')] [2023-12-27 01:03:41,084][105620] Updated weights for policy 1, policy_version 1329924 (0.0006) [2023-12-27 01:03:41,154][105620] Updated weights for policy 1, policy_version 1329934 (0.0009) [2023-12-27 01:03:41,221][105620] Updated weights for policy 1, policy_version 1329944 (0.0011) [2023-12-27 01:03:41,580][105692] Updated weights for policy 0, policy_version 1327987 (0.0008) [2023-12-27 01:03:41,649][105692] Updated weights for policy 0, policy_version 1327997 (0.0008) [2023-12-27 01:03:41,719][105692] Updated weights for policy 0, policy_version 1328007 (0.0010) [2023-12-27 01:03:41,878][105620] Updated weights for policy 1, policy_version 1329954 (0.0010) [2023-12-27 01:03:41,934][105620] Updated weights for policy 1, policy_version 1329964 (0.0011) [2023-12-27 01:03:41,987][105620] Updated weights for policy 1, policy_version 1329974 (0.0011) [2023-12-27 01:03:42,044][105620] Updated weights for policy 1, policy_version 1329984 (0.0011) [2023-12-27 01:03:42,532][105692] Updated weights for policy 0, policy_version 1328017 (0.0010) [2023-12-27 01:03:42,595][105692] Updated weights for policy 0, policy_version 1328027 (0.0008) [2023-12-27 01:03:42,644][105692] Updated weights for policy 0, policy_version 1328037 (0.0008) [2023-12-27 01:03:42,699][105692] Updated weights for policy 0, policy_version 1328047 (0.0008) [2023-12-27 01:03:42,784][105620] Updated weights for policy 1, policy_version 1329994 (0.0010) [2023-12-27 01:03:42,836][105620] Updated weights for policy 1, policy_version 1330004 (0.0010) [2023-12-27 01:03:42,885][105620] Updated weights for policy 1, policy_version 1330014 (0.0010) [2023-12-27 01:03:43,384][105692] Updated weights for policy 0, policy_version 1328057 (0.0009) [2023-12-27 01:03:43,440][105692] Updated weights for policy 0, policy_version 1328067 (0.0006) [2023-12-27 01:03:43,496][105692] Updated weights for policy 0, policy_version 1328077 (0.0008) [2023-12-27 01:03:43,638][105620] Updated weights for policy 1, policy_version 1330024 (0.0008) [2023-12-27 01:03:43,683][105620] Updated weights for policy 1, policy_version 1330034 (0.0008) [2023-12-27 01:03:43,735][105620] Updated weights for policy 1, policy_version 1330044 (0.0008) [2023-12-27 01:03:44,206][105692] Updated weights for policy 0, policy_version 1328087 (0.0010) [2023-12-27 01:03:44,263][105692] Updated weights for policy 0, policy_version 1328097 (0.0010) [2023-12-27 01:03:44,325][105692] Updated weights for policy 0, policy_version 1328107 (0.0010) [2023-12-27 01:03:44,518][105620] Updated weights for policy 1, policy_version 1330054 (0.0009) [2023-12-27 01:03:44,578][105620] Updated weights for policy 1, policy_version 1330064 (0.0010) [2023-12-27 01:03:44,633][105620] Updated weights for policy 1, policy_version 1330074 (0.0010) [2023-12-27 01:03:44,969][105692] Updated weights for policy 0, policy_version 1328117 (0.0010) [2023-12-27 01:03:45,037][105692] Updated weights for policy 0, policy_version 1328127 (0.0011) [2023-12-27 01:03:45,102][105692] Updated weights for policy 0, policy_version 1328137 (0.0010) [2023-12-27 01:03:45,498][105620] Updated weights for policy 1, policy_version 1330084 (0.0009) [2023-12-27 01:03:45,569][105620] Updated weights for policy 1, policy_version 1330094 (0.0009) [2023-12-27 01:03:45,632][105620] Updated weights for policy 1, policy_version 1330104 (0.0009) [2023-12-27 01:03:45,795][105692] Updated weights for policy 0, policy_version 1328147 (0.0009) [2023-12-27 01:03:45,851][105692] Updated weights for policy 0, policy_version 1328157 (0.0005) [2023-12-27 01:03:45,909][105692] Updated weights for policy 0, policy_version 1328167 (0.0005) [2023-12-27 01:03:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.4, 300 sec: 19494.2). Total num frames: 680615936. Throughput: 0: 9647.6, 1: 9686.9. Samples: 680584204. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:03:46,062][104569] Avg episode reward: [(0, '8358.336'), (1, '8996.803')] [2023-12-27 01:03:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001328176_340066304.pth... [2023-12-27 01:03:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001330112_340549632.pth... [2023-12-27 01:03:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001327056_339779584.pth [2023-12-27 01:03:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001328992_340262912.pth [2023-12-27 01:03:46,407][105620] Updated weights for policy 1, policy_version 1330114 (0.0008) [2023-12-27 01:03:46,474][105620] Updated weights for policy 1, policy_version 1330124 (0.0009) [2023-12-27 01:03:46,540][105620] Updated weights for policy 1, policy_version 1330134 (0.0010) [2023-12-27 01:03:46,571][105692] Updated weights for policy 0, policy_version 1328177 (0.0009) [2023-12-27 01:03:46,603][105620] Updated weights for policy 1, policy_version 1330144 (0.0009) [2023-12-27 01:03:46,628][105692] Updated weights for policy 0, policy_version 1328187 (0.0005) [2023-12-27 01:03:46,696][105692] Updated weights for policy 0, policy_version 1328197 (0.0005) [2023-12-27 01:03:46,755][105692] Updated weights for policy 0, policy_version 1328207 (0.0005) [2023-12-27 01:03:47,332][105620] Updated weights for policy 1, policy_version 1330154 (0.0007) [2023-12-27 01:03:47,343][105692] Updated weights for policy 0, policy_version 1328217 (0.0010) [2023-12-27 01:03:47,351][105585] KL-divergence is very high: 158.1151 [2023-12-27 01:03:47,387][105692] Updated weights for policy 0, policy_version 1328227 (0.0010) [2023-12-27 01:03:47,388][105585] KL-divergence is very high: 273.4182 [2023-12-27 01:03:47,390][105620] Updated weights for policy 1, policy_version 1330164 (0.0008) [2023-12-27 01:03:47,434][105585] KL-divergence is very high: 269.1601 [2023-12-27 01:03:47,447][105620] Updated weights for policy 1, policy_version 1330174 (0.0006) [2023-12-27 01:03:47,448][105692] Updated weights for policy 0, policy_version 1328237 (0.0010) [2023-12-27 01:03:48,040][105620] Updated weights for policy 1, policy_version 1330184 (0.0005) [2023-12-27 01:03:48,091][105620] Updated weights for policy 1, policy_version 1330194 (0.0010) [2023-12-27 01:03:48,140][105620] Updated weights for policy 1, policy_version 1330204 (0.0010) [2023-12-27 01:03:48,145][105692] Updated weights for policy 0, policy_version 1328247 (0.0008) [2023-12-27 01:03:48,204][105692] Updated weights for policy 0, policy_version 1328257 (0.0009) [2023-12-27 01:03:48,260][105692] Updated weights for policy 0, policy_version 1328267 (0.0005) [2023-12-27 01:03:48,879][105620] Updated weights for policy 1, policy_version 1330214 (0.0010) [2023-12-27 01:03:48,942][105620] Updated weights for policy 1, policy_version 1330224 (0.0009) [2023-12-27 01:03:48,995][105692] Updated weights for policy 0, policy_version 1328277 (0.0007) [2023-12-27 01:03:49,009][105620] Updated weights for policy 1, policy_version 1330234 (0.0006) [2023-12-27 01:03:49,052][105692] Updated weights for policy 0, policy_version 1328287 (0.0008) [2023-12-27 01:03:49,109][105692] Updated weights for policy 0, policy_version 1328297 (0.0005) [2023-12-27 01:03:49,758][105620] Updated weights for policy 1, policy_version 1330244 (0.0007) [2023-12-27 01:03:49,774][105692] Updated weights for policy 0, policy_version 1328307 (0.0007) [2023-12-27 01:03:49,817][105620] Updated weights for policy 1, policy_version 1330254 (0.0006) [2023-12-27 01:03:49,834][105692] Updated weights for policy 0, policy_version 1328317 (0.0011) [2023-12-27 01:03:49,876][105620] Updated weights for policy 1, policy_version 1330264 (0.0009) [2023-12-27 01:03:49,898][105692] Updated weights for policy 0, policy_version 1328327 (0.0008) [2023-12-27 01:03:50,467][105692] Updated weights for policy 0, policy_version 1328337 (0.0008) [2023-12-27 01:03:50,525][105692] Updated weights for policy 0, policy_version 1328347 (0.0009) [2023-12-27 01:03:50,595][105692] Updated weights for policy 0, policy_version 1328357 (0.0010) [2023-12-27 01:03:50,643][105692] Updated weights for policy 0, policy_version 1328367 (0.0008) [2023-12-27 01:03:50,718][105620] Updated weights for policy 1, policy_version 1330274 (0.0007) [2023-12-27 01:03:50,778][105620] Updated weights for policy 1, policy_version 1330284 (0.0008) [2023-12-27 01:03:50,838][105620] Updated weights for policy 1, policy_version 1330294 (0.0008) [2023-12-27 01:03:50,901][105620] Updated weights for policy 1, policy_version 1330304 (0.0008) [2023-12-27 01:03:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 680714240. Throughput: 0: 9644.1, 1: 9588.7. Samples: 680701156. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:03:51,062][104569] Avg episode reward: [(0, '8367.601'), (1, '8993.809')] [2023-12-27 01:03:51,405][105692] Updated weights for policy 0, policy_version 1328377 (0.0008) [2023-12-27 01:03:51,461][105692] Updated weights for policy 0, policy_version 1328387 (0.0008) [2023-12-27 01:03:51,516][105692] Updated weights for policy 0, policy_version 1328397 (0.0008) [2023-12-27 01:03:51,633][105620] Updated weights for policy 1, policy_version 1330314 (0.0009) [2023-12-27 01:03:51,698][105620] Updated weights for policy 1, policy_version 1330324 (0.0008) [2023-12-27 01:03:51,762][105620] Updated weights for policy 1, policy_version 1330334 (0.0009) [2023-12-27 01:03:52,236][105692] Updated weights for policy 0, policy_version 1328407 (0.0008) [2023-12-27 01:03:52,301][105692] Updated weights for policy 0, policy_version 1328417 (0.0008) [2023-12-27 01:03:52,360][105692] Updated weights for policy 0, policy_version 1328427 (0.0008) [2023-12-27 01:03:52,539][105620] Updated weights for policy 1, policy_version 1330344 (0.0007) [2023-12-27 01:03:52,589][105620] Updated weights for policy 1, policy_version 1330354 (0.0005) [2023-12-27 01:03:52,639][105620] Updated weights for policy 1, policy_version 1330364 (0.0006) [2023-12-27 01:03:53,131][105692] Updated weights for policy 0, policy_version 1328437 (0.0007) [2023-12-27 01:03:53,200][105692] Updated weights for policy 0, policy_version 1328447 (0.0009) [2023-12-27 01:03:53,253][105620] Updated weights for policy 1, policy_version 1330374 (0.0009) [2023-12-27 01:03:53,268][105692] Updated weights for policy 0, policy_version 1328457 (0.0006) [2023-12-27 01:03:53,315][105620] Updated weights for policy 1, policy_version 1330384 (0.0008) [2023-12-27 01:03:53,361][105620] Updated weights for policy 1, policy_version 1330394 (0.0005) [2023-12-27 01:03:53,851][105692] Updated weights for policy 0, policy_version 1328467 (0.0007) [2023-12-27 01:03:53,895][105692] Updated weights for policy 0, policy_version 1328477 (0.0010) [2023-12-27 01:03:53,942][105692] Updated weights for policy 0, policy_version 1328487 (0.0010) [2023-12-27 01:03:53,999][105620] Updated weights for policy 1, policy_version 1330404 (0.0007) [2023-12-27 01:03:54,055][105620] Updated weights for policy 1, policy_version 1330414 (0.0006) [2023-12-27 01:03:54,104][105620] Updated weights for policy 1, policy_version 1330424 (0.0005) [2023-12-27 01:03:54,647][105692] Updated weights for policy 0, policy_version 1328497 (0.0010) [2023-12-27 01:03:54,709][105692] Updated weights for policy 0, policy_version 1328507 (0.0011) [2023-12-27 01:03:54,768][105692] Updated weights for policy 0, policy_version 1328517 (0.0011) [2023-12-27 01:03:54,823][105692] Updated weights for policy 0, policy_version 1328527 (0.0011) [2023-12-27 01:03:54,859][105620] Updated weights for policy 1, policy_version 1330434 (0.0009) [2023-12-27 01:03:54,909][105620] Updated weights for policy 1, policy_version 1330444 (0.0007) [2023-12-27 01:03:54,962][105620] Updated weights for policy 1, policy_version 1330454 (0.0008) [2023-12-27 01:03:55,026][105620] Updated weights for policy 1, policy_version 1330464 (0.0008) [2023-12-27 01:03:55,564][105692] Updated weights for policy 0, policy_version 1328537 (0.0010) [2023-12-27 01:03:55,625][105692] Updated weights for policy 0, policy_version 1328547 (0.0007) [2023-12-27 01:03:55,693][105692] Updated weights for policy 0, policy_version 1328557 (0.0005) [2023-12-27 01:03:55,819][105620] Updated weights for policy 1, policy_version 1330474 (0.0006) [2023-12-27 01:03:55,868][105620] Updated weights for policy 1, policy_version 1330484 (0.0007) [2023-12-27 01:03:55,916][105620] Updated weights for policy 1, policy_version 1330494 (0.0010) [2023-12-27 01:03:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 680812544. Throughput: 0: 9686.0, 1: 9648.5. Samples: 680818556. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:03:56,062][104569] Avg episode reward: [(0, '8908.341'), (1, '9082.649')] [2023-12-27 01:03:56,287][105692] Updated weights for policy 0, policy_version 1328567 (0.0005) [2023-12-27 01:03:56,331][105692] Updated weights for policy 0, policy_version 1328577 (0.0005) [2023-12-27 01:03:56,378][105692] Updated weights for policy 0, policy_version 1328587 (0.0005) [2023-12-27 01:03:56,606][105620] Updated weights for policy 1, policy_version 1330504 (0.0007) [2023-12-27 01:03:56,661][105620] Updated weights for policy 1, policy_version 1330514 (0.0005) [2023-12-27 01:03:56,707][105620] Updated weights for policy 1, policy_version 1330524 (0.0005) [2023-12-27 01:03:56,984][105692] Updated weights for policy 0, policy_version 1328597 (0.0005) [2023-12-27 01:03:57,036][105692] Updated weights for policy 0, policy_version 1328607 (0.0006) [2023-12-27 01:03:57,087][105692] Updated weights for policy 0, policy_version 1328617 (0.0005) [2023-12-27 01:03:57,234][105620] Updated weights for policy 1, policy_version 1330534 (0.0006) [2023-12-27 01:03:57,280][105620] Updated weights for policy 1, policy_version 1330544 (0.0005) [2023-12-27 01:03:57,342][105620] Updated weights for policy 1, policy_version 1330554 (0.0005) [2023-12-27 01:03:57,684][105692] Updated weights for policy 0, policy_version 1328627 (0.0007) [2023-12-27 01:03:57,737][105692] Updated weights for policy 0, policy_version 1328637 (0.0007) [2023-12-27 01:03:57,783][105692] Updated weights for policy 0, policy_version 1328647 (0.0005) [2023-12-27 01:03:57,953][105620] Updated weights for policy 1, policy_version 1330564 (0.0006) [2023-12-27 01:03:58,001][105620] Updated weights for policy 1, policy_version 1330574 (0.0005) [2023-12-27 01:03:58,057][105620] Updated weights for policy 1, policy_version 1330584 (0.0005) [2023-12-27 01:03:58,398][105692] Updated weights for policy 0, policy_version 1328657 (0.0006) [2023-12-27 01:03:58,460][105692] Updated weights for policy 0, policy_version 1328667 (0.0008) [2023-12-27 01:03:58,523][105692] Updated weights for policy 0, policy_version 1328677 (0.0008) [2023-12-27 01:03:58,589][105692] Updated weights for policy 0, policy_version 1328687 (0.0008) [2023-12-27 01:03:58,761][105620] Updated weights for policy 1, policy_version 1330594 (0.0006) [2023-12-27 01:03:58,835][105620] Updated weights for policy 1, policy_version 1330604 (0.0009) [2023-12-27 01:03:58,921][105620] Updated weights for policy 1, policy_version 1330614 (0.0008) [2023-12-27 01:03:58,970][105620] Updated weights for policy 1, policy_version 1330624 (0.0009) [2023-12-27 01:03:59,383][105692] Updated weights for policy 0, policy_version 1328697 (0.0008) [2023-12-27 01:03:59,446][105692] Updated weights for policy 0, policy_version 1328707 (0.0005) [2023-12-27 01:03:59,504][105692] Updated weights for policy 0, policy_version 1328717 (0.0005) [2023-12-27 01:03:59,701][105620] Updated weights for policy 1, policy_version 1330634 (0.0008) [2023-12-27 01:03:59,762][105620] Updated weights for policy 1, policy_version 1330644 (0.0009) [2023-12-27 01:03:59,828][105620] Updated weights for policy 1, policy_version 1330654 (0.0008) [2023-12-27 01:04:00,222][105692] Updated weights for policy 0, policy_version 1328727 (0.0008) [2023-12-27 01:04:00,273][105692] Updated weights for policy 0, policy_version 1328737 (0.0009) [2023-12-27 01:04:00,320][105692] Updated weights for policy 0, policy_version 1328747 (0.0009) [2023-12-27 01:04:00,552][105620] Updated weights for policy 1, policy_version 1330664 (0.0009) [2023-12-27 01:04:00,616][105620] Updated weights for policy 1, policy_version 1330674 (0.0009) [2023-12-27 01:04:00,672][105620] Updated weights for policy 1, policy_version 1330684 (0.0009) [2023-12-27 01:04:01,041][105692] Updated weights for policy 0, policy_version 1328757 (0.0008) [2023-12-27 01:04:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 680910848. Throughput: 0: 9834.5, 1: 9715.8. Samples: 680884824. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:04:01,062][104569] Avg episode reward: [(0, '8990.101'), (1, '9173.942')] [2023-12-27 01:04:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001330688_340697088.pth... [2023-12-27 01:04:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001329568_340410368.pth [2023-12-27 01:04:01,107][105692] Updated weights for policy 0, policy_version 1328767 (0.0009) [2023-12-27 01:04:01,168][105692] Updated weights for policy 0, policy_version 1328777 (0.0010) [2023-12-27 01:04:01,201][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001328784_340221952.pth... [2023-12-27 01:04:01,206][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001327600_339918848.pth [2023-12-27 01:04:01,419][105620] Updated weights for policy 1, policy_version 1330694 (0.0009) [2023-12-27 01:04:01,478][105620] Updated weights for policy 1, policy_version 1330704 (0.0005) [2023-12-27 01:04:01,535][105620] Updated weights for policy 1, policy_version 1330714 (0.0009) [2023-12-27 01:04:01,981][105692] Updated weights for policy 0, policy_version 1328787 (0.0009) [2023-12-27 01:04:02,041][105692] Updated weights for policy 0, policy_version 1328797 (0.0011) [2023-12-27 01:04:02,100][105692] Updated weights for policy 0, policy_version 1328807 (0.0010) [2023-12-27 01:04:02,208][105620] Updated weights for policy 1, policy_version 1330724 (0.0009) [2023-12-27 01:04:02,260][105620] Updated weights for policy 1, policy_version 1330734 (0.0008) [2023-12-27 01:04:02,322][105620] Updated weights for policy 1, policy_version 1330744 (0.0008) [2023-12-27 01:04:02,762][105692] Updated weights for policy 0, policy_version 1328817 (0.0010) [2023-12-27 01:04:02,818][105692] Updated weights for policy 0, policy_version 1328827 (0.0009) [2023-12-27 01:04:02,874][105692] Updated weights for policy 0, policy_version 1328837 (0.0008) [2023-12-27 01:04:02,930][105692] Updated weights for policy 0, policy_version 1328847 (0.0005) [2023-12-27 01:04:03,137][105620] Updated weights for policy 1, policy_version 1330754 (0.0009) [2023-12-27 01:04:03,190][105620] Updated weights for policy 1, policy_version 1330764 (0.0009) [2023-12-27 01:04:03,244][105620] Updated weights for policy 1, policy_version 1330774 (0.0009) [2023-12-27 01:04:03,295][105620] Updated weights for policy 1, policy_version 1330784 (0.0009) [2023-12-27 01:04:03,642][105692] Updated weights for policy 0, policy_version 1328857 (0.0006) [2023-12-27 01:04:03,685][105692] Updated weights for policy 0, policy_version 1328867 (0.0005) [2023-12-27 01:04:03,740][105692] Updated weights for policy 0, policy_version 1328877 (0.0006) [2023-12-27 01:04:04,080][105620] Updated weights for policy 1, policy_version 1330794 (0.0008) [2023-12-27 01:04:04,149][105620] Updated weights for policy 1, policy_version 1330804 (0.0006) [2023-12-27 01:04:04,218][105620] Updated weights for policy 1, policy_version 1330814 (0.0007) [2023-12-27 01:04:04,485][105692] Updated weights for policy 0, policy_version 1328887 (0.0009) [2023-12-27 01:04:04,547][105692] Updated weights for policy 0, policy_version 1328897 (0.0009) [2023-12-27 01:04:04,599][105692] Updated weights for policy 0, policy_version 1328907 (0.0008) [2023-12-27 01:04:04,878][105620] Updated weights for policy 1, policy_version 1330824 (0.0006) [2023-12-27 01:04:04,932][105620] Updated weights for policy 1, policy_version 1330834 (0.0005) [2023-12-27 01:04:04,986][105620] Updated weights for policy 1, policy_version 1330844 (0.0005) [2023-12-27 01:04:05,372][105692] Updated weights for policy 0, policy_version 1328917 (0.0008) [2023-12-27 01:04:05,427][105692] Updated weights for policy 0, policy_version 1328927 (0.0008) [2023-12-27 01:04:05,485][105692] Updated weights for policy 0, policy_version 1328937 (0.0008) [2023-12-27 01:04:05,666][105620] Updated weights for policy 1, policy_version 1330854 (0.0008) [2023-12-27 01:04:05,721][105620] Updated weights for policy 1, policy_version 1330864 (0.0009) [2023-12-27 01:04:05,765][105620] Updated weights for policy 1, policy_version 1330874 (0.0010) [2023-12-27 01:04:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 681009152. Throughput: 0: 9835.4, 1: 9722.8. Samples: 680998768. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:04:06,062][104569] Avg episode reward: [(0, '8816.521'), (1, '9356.169')] [2023-12-27 01:04:06,186][105692] Updated weights for policy 0, policy_version 1328947 (0.0006) [2023-12-27 01:04:06,247][105692] Updated weights for policy 0, policy_version 1328957 (0.0008) [2023-12-27 01:04:06,314][105692] Updated weights for policy 0, policy_version 1328967 (0.0008) [2023-12-27 01:04:06,525][105620] Updated weights for policy 1, policy_version 1330884 (0.0010) [2023-12-27 01:04:06,591][105620] Updated weights for policy 1, policy_version 1330894 (0.0010) [2023-12-27 01:04:06,654][105620] Updated weights for policy 1, policy_version 1330904 (0.0011) [2023-12-27 01:04:07,088][105692] Updated weights for policy 0, policy_version 1328977 (0.0008) [2023-12-27 01:04:07,145][105692] Updated weights for policy 0, policy_version 1328987 (0.0008) [2023-12-27 01:04:07,215][105692] Updated weights for policy 0, policy_version 1328997 (0.0008) [2023-12-27 01:04:07,276][105692] Updated weights for policy 0, policy_version 1329007 (0.0008) [2023-12-27 01:04:07,405][105620] Updated weights for policy 1, policy_version 1330914 (0.0010) [2023-12-27 01:04:07,452][105620] Updated weights for policy 1, policy_version 1330924 (0.0010) [2023-12-27 01:04:07,500][105620] Updated weights for policy 1, policy_version 1330934 (0.0010) [2023-12-27 01:04:07,547][105620] Updated weights for policy 1, policy_version 1330944 (0.0010) [2023-12-27 01:04:08,021][105692] Updated weights for policy 0, policy_version 1329017 (0.0008) [2023-12-27 01:04:08,076][105692] Updated weights for policy 0, policy_version 1329027 (0.0008) [2023-12-27 01:04:08,121][105692] Updated weights for policy 0, policy_version 1329037 (0.0008) [2023-12-27 01:04:08,321][105620] Updated weights for policy 1, policy_version 1330954 (0.0010) [2023-12-27 01:04:08,387][105620] Updated weights for policy 1, policy_version 1330964 (0.0010) [2023-12-27 01:04:08,442][105620] Updated weights for policy 1, policy_version 1330974 (0.0010) [2023-12-27 01:04:08,911][105692] Updated weights for policy 0, policy_version 1329047 (0.0009) [2023-12-27 01:04:08,972][105692] Updated weights for policy 0, policy_version 1329057 (0.0007) [2023-12-27 01:04:09,032][105692] Updated weights for policy 0, policy_version 1329067 (0.0008) [2023-12-27 01:04:09,132][105620] Updated weights for policy 1, policy_version 1330984 (0.0007) [2023-12-27 01:04:09,188][105620] Updated weights for policy 1, policy_version 1330994 (0.0005) [2023-12-27 01:04:09,254][105620] Updated weights for policy 1, policy_version 1331004 (0.0008) [2023-12-27 01:04:09,816][105692] Updated weights for policy 0, policy_version 1329077 (0.0008) [2023-12-27 01:04:09,880][105692] Updated weights for policy 0, policy_version 1329087 (0.0008) [2023-12-27 01:04:09,948][105692] Updated weights for policy 0, policy_version 1329097 (0.0009) [2023-12-27 01:04:10,035][105620] Updated weights for policy 1, policy_version 1331014 (0.0009) [2023-12-27 01:04:10,101][105620] Updated weights for policy 1, policy_version 1331024 (0.0008) [2023-12-27 01:04:10,154][105620] Updated weights for policy 1, policy_version 1331034 (0.0008) [2023-12-27 01:04:10,780][105692] Updated weights for policy 0, policy_version 1329107 (0.0009) [2023-12-27 01:04:10,799][105620] Updated weights for policy 1, policy_version 1331044 (0.0005) [2023-12-27 01:04:10,832][105692] Updated weights for policy 0, policy_version 1329117 (0.0007) [2023-12-27 01:04:10,867][105620] Updated weights for policy 1, policy_version 1331054 (0.0006) [2023-12-27 01:04:10,884][105692] Updated weights for policy 0, policy_version 1329127 (0.0010) [2023-12-27 01:04:10,921][105620] Updated weights for policy 1, policy_version 1331064 (0.0007) [2023-12-27 01:04:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 681107456. Throughput: 0: 9795.4, 1: 9707.5. Samples: 681111072. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:04:11,063][104569] Avg episode reward: [(0, '8826.922'), (1, '9355.700')] [2023-12-27 01:04:11,701][105620] Updated weights for policy 1, policy_version 1331074 (0.0008) [2023-12-27 01:04:11,768][105620] Updated weights for policy 1, policy_version 1331084 (0.0008) [2023-12-27 01:04:11,770][105692] Updated weights for policy 0, policy_version 1329137 (0.0008) [2023-12-27 01:04:11,823][105620] Updated weights for policy 1, policy_version 1331094 (0.0007) [2023-12-27 01:04:11,833][105692] Updated weights for policy 0, policy_version 1329147 (0.0008) [2023-12-27 01:04:11,881][105620] Updated weights for policy 1, policy_version 1331104 (0.0005) [2023-12-27 01:04:11,896][105692] Updated weights for policy 0, policy_version 1329157 (0.0008) [2023-12-27 01:04:11,959][105692] Updated weights for policy 0, policy_version 1329167 (0.0008) [2023-12-27 01:04:12,471][105620] Updated weights for policy 1, policy_version 1331114 (0.0009) [2023-12-27 01:04:12,535][105620] Updated weights for policy 1, policy_version 1331124 (0.0009) [2023-12-27 01:04:12,592][105620] Updated weights for policy 1, policy_version 1331134 (0.0009) [2023-12-27 01:04:12,661][105692] Updated weights for policy 0, policy_version 1329177 (0.0006) [2023-12-27 01:04:12,713][105692] Updated weights for policy 0, policy_version 1329187 (0.0005) [2023-12-27 01:04:12,762][105692] Updated weights for policy 0, policy_version 1329197 (0.0005) [2023-12-27 01:04:13,333][105620] Updated weights for policy 1, policy_version 1331144 (0.0008) [2023-12-27 01:04:13,388][105620] Updated weights for policy 1, policy_version 1331154 (0.0008) [2023-12-27 01:04:13,441][105620] Updated weights for policy 1, policy_version 1331164 (0.0007) [2023-12-27 01:04:13,447][105692] Updated weights for policy 0, policy_version 1329207 (0.0009) [2023-12-27 01:04:13,496][105692] Updated weights for policy 0, policy_version 1329217 (0.0010) [2023-12-27 01:04:13,542][105692] Updated weights for policy 0, policy_version 1329227 (0.0005) [2023-12-27 01:04:14,222][105692] Updated weights for policy 0, policy_version 1329237 (0.0008) [2023-12-27 01:04:14,228][105620] Updated weights for policy 1, policy_version 1331174 (0.0006) [2023-12-27 01:04:14,285][105620] Updated weights for policy 1, policy_version 1331184 (0.0006) [2023-12-27 01:04:14,287][105692] Updated weights for policy 0, policy_version 1329247 (0.0010) [2023-12-27 01:04:14,341][105620] Updated weights for policy 1, policy_version 1331194 (0.0005) [2023-12-27 01:04:14,352][105692] Updated weights for policy 0, policy_version 1329257 (0.0010) [2023-12-27 01:04:14,979][105692] Updated weights for policy 0, policy_version 1329267 (0.0011) [2023-12-27 01:04:14,979][105620] Updated weights for policy 1, policy_version 1331204 (0.0007) [2023-12-27 01:04:15,039][105620] Updated weights for policy 1, policy_version 1331214 (0.0007) [2023-12-27 01:04:15,040][105692] Updated weights for policy 0, policy_version 1329277 (0.0011) [2023-12-27 01:04:15,084][105585] KL-divergence is very high: 124.1258 [2023-12-27 01:04:15,099][105620] Updated weights for policy 1, policy_version 1331224 (0.0006) [2023-12-27 01:04:15,104][105692] Updated weights for policy 0, policy_version 1329287 (0.0011) [2023-12-27 01:04:15,138][105585] KL-divergence is very high: 137.5153 [2023-12-27 01:04:15,742][105620] Updated weights for policy 1, policy_version 1331234 (0.0005) [2023-12-27 01:04:15,800][105620] Updated weights for policy 1, policy_version 1331244 (0.0006) [2023-12-27 01:04:15,851][105692] Updated weights for policy 0, policy_version 1329297 (0.0010) [2023-12-27 01:04:15,858][105620] Updated weights for policy 1, policy_version 1331254 (0.0010) [2023-12-27 01:04:15,918][105620] Updated weights for policy 1, policy_version 1331264 (0.0011) [2023-12-27 01:04:15,921][105692] Updated weights for policy 0, policy_version 1329307 (0.0005) [2023-12-27 01:04:15,983][105692] Updated weights for policy 0, policy_version 1329317 (0.0006) [2023-12-27 01:04:16,053][105692] Updated weights for policy 0, policy_version 1329327 (0.0011) [2023-12-27 01:04:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 681205760. Throughput: 0: 9769.3, 1: 9630.9. Samples: 681167708. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:04:16,063][104569] Avg episode reward: [(0, '8910.803'), (1, '9264.297')] [2023-12-27 01:04:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001329328_340361216.pth... [2023-12-27 01:04:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001331264_340844544.pth... [2023-12-27 01:04:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001328176_340066304.pth [2023-12-27 01:04:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001330112_340549632.pth [2023-12-27 01:04:16,617][105620] Updated weights for policy 1, policy_version 1331274 (0.0009) [2023-12-27 01:04:16,666][105692] Updated weights for policy 0, policy_version 1329337 (0.0011) [2023-12-27 01:04:16,677][105620] Updated weights for policy 1, policy_version 1331284 (0.0010) [2023-12-27 01:04:16,722][105692] Updated weights for policy 0, policy_version 1329347 (0.0011) [2023-12-27 01:04:16,745][105620] Updated weights for policy 1, policy_version 1331294 (0.0006) [2023-12-27 01:04:16,781][105692] Updated weights for policy 0, policy_version 1329357 (0.0011) [2023-12-27 01:04:17,383][105620] Updated weights for policy 1, policy_version 1331304 (0.0008) [2023-12-27 01:04:17,428][105620] Updated weights for policy 1, policy_version 1331314 (0.0008) [2023-12-27 01:04:17,472][105692] Updated weights for policy 0, policy_version 1329367 (0.0007) [2023-12-27 01:04:17,485][105620] Updated weights for policy 1, policy_version 1331324 (0.0008) [2023-12-27 01:04:17,521][105692] Updated weights for policy 0, policy_version 1329377 (0.0009) [2023-12-27 01:04:17,574][105692] Updated weights for policy 0, policy_version 1329387 (0.0008) [2023-12-27 01:04:18,155][105692] Updated weights for policy 0, policy_version 1329397 (0.0009) [2023-12-27 01:04:18,217][105692] Updated weights for policy 0, policy_version 1329407 (0.0009) [2023-12-27 01:04:18,267][105692] Updated weights for policy 0, policy_version 1329417 (0.0009) [2023-12-27 01:04:18,290][105620] Updated weights for policy 1, policy_version 1331334 (0.0009) [2023-12-27 01:04:18,343][105620] Updated weights for policy 1, policy_version 1331344 (0.0007) [2023-12-27 01:04:18,410][105620] Updated weights for policy 1, policy_version 1331354 (0.0008) [2023-12-27 01:04:18,980][105692] Updated weights for policy 0, policy_version 1329427 (0.0009) [2023-12-27 01:04:19,043][105692] Updated weights for policy 0, policy_version 1329437 (0.0007) [2023-12-27 01:04:19,111][105692] Updated weights for policy 0, policy_version 1329447 (0.0008) [2023-12-27 01:04:19,184][105620] Updated weights for policy 1, policy_version 1331364 (0.0009) [2023-12-27 01:04:19,249][105620] Updated weights for policy 1, policy_version 1331374 (0.0008) [2023-12-27 01:04:19,307][105620] Updated weights for policy 1, policy_version 1331384 (0.0009) [2023-12-27 01:04:19,820][105692] Updated weights for policy 0, policy_version 1329457 (0.0008) [2023-12-27 01:04:19,886][105692] Updated weights for policy 0, policy_version 1329467 (0.0010) [2023-12-27 01:04:19,948][105692] Updated weights for policy 0, policy_version 1329477 (0.0009) [2023-12-27 01:04:20,015][105692] Updated weights for policy 0, policy_version 1329487 (0.0008) [2023-12-27 01:04:20,105][105620] Updated weights for policy 1, policy_version 1331394 (0.0009) [2023-12-27 01:04:20,161][105620] Updated weights for policy 1, policy_version 1331404 (0.0008) [2023-12-27 01:04:20,221][105620] Updated weights for policy 1, policy_version 1331414 (0.0006) [2023-12-27 01:04:20,282][105620] Updated weights for policy 1, policy_version 1331424 (0.0008) [2023-12-27 01:04:20,711][105692] Updated weights for policy 0, policy_version 1329497 (0.0011) [2023-12-27 01:04:20,770][105692] Updated weights for policy 0, policy_version 1329507 (0.0011) [2023-12-27 01:04:20,836][105692] Updated weights for policy 0, policy_version 1329517 (0.0011) [2023-12-27 01:04:21,038][105620] Updated weights for policy 1, policy_version 1331434 (0.0009) [2023-12-27 01:04:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 681295872. Throughput: 0: 9813.9, 1: 9616.4. Samples: 681287508. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-12-27 01:04:21,062][104569] Avg episode reward: [(0, '8727.378'), (1, '8996.053')] [2023-12-27 01:04:21,109][105620] Updated weights for policy 1, policy_version 1331444 (0.0009) [2023-12-27 01:04:21,173][105620] Updated weights for policy 1, policy_version 1331454 (0.0008) [2023-12-27 01:04:21,559][105692] Updated weights for policy 0, policy_version 1329527 (0.0011) [2023-12-27 01:04:21,609][105692] Updated weights for policy 0, policy_version 1329537 (0.0010) [2023-12-27 01:04:21,671][105692] Updated weights for policy 0, policy_version 1329547 (0.0008) [2023-12-27 01:04:21,954][105620] Updated weights for policy 1, policy_version 1331464 (0.0009) [2023-12-27 01:04:22,015][105620] Updated weights for policy 1, policy_version 1331474 (0.0009) [2023-12-27 01:04:22,081][105620] Updated weights for policy 1, policy_version 1331484 (0.0009) [2023-12-27 01:04:22,416][105692] Updated weights for policy 0, policy_version 1329557 (0.0009) [2023-12-27 01:04:22,468][105692] Updated weights for policy 0, policy_version 1329567 (0.0009) [2023-12-27 01:04:22,534][105692] Updated weights for policy 0, policy_version 1329577 (0.0010) [2023-12-27 01:04:22,740][105620] Updated weights for policy 1, policy_version 1331494 (0.0008) [2023-12-27 01:04:22,785][105620] Updated weights for policy 1, policy_version 1331504 (0.0007) [2023-12-27 01:04:22,858][105620] Updated weights for policy 1, policy_version 1331514 (0.0007) [2023-12-27 01:04:23,381][105692] Updated weights for policy 0, policy_version 1329587 (0.0010) [2023-12-27 01:04:23,444][105692] Updated weights for policy 0, policy_version 1329597 (0.0008) [2023-12-27 01:04:23,445][105620] Updated weights for policy 1, policy_version 1331524 (0.0008) [2023-12-27 01:04:23,496][105692] Updated weights for policy 0, policy_version 1329607 (0.0005) [2023-12-27 01:04:23,501][105620] Updated weights for policy 1, policy_version 1331534 (0.0008) [2023-12-27 01:04:23,558][105620] Updated weights for policy 1, policy_version 1331544 (0.0009) [2023-12-27 01:04:24,265][105692] Updated weights for policy 0, policy_version 1329617 (0.0006) [2023-12-27 01:04:24,302][105620] Updated weights for policy 1, policy_version 1331554 (0.0008) [2023-12-27 01:04:24,321][105692] Updated weights for policy 0, policy_version 1329627 (0.0007) [2023-12-27 01:04:24,359][105620] Updated weights for policy 1, policy_version 1331564 (0.0008) [2023-12-27 01:04:24,373][105692] Updated weights for policy 0, policy_version 1329637 (0.0006) [2023-12-27 01:04:24,415][105620] Updated weights for policy 1, policy_version 1331574 (0.0007) [2023-12-27 01:04:24,433][105692] Updated weights for policy 0, policy_version 1329647 (0.0006) [2023-12-27 01:04:24,477][105620] Updated weights for policy 1, policy_version 1331584 (0.0008) [2023-12-27 01:04:25,169][105692] Updated weights for policy 0, policy_version 1329657 (0.0010) [2023-12-27 01:04:25,230][105692] Updated weights for policy 0, policy_version 1329667 (0.0008) [2023-12-27 01:04:25,248][105620] Updated weights for policy 1, policy_version 1331594 (0.0008) [2023-12-27 01:04:25,291][105692] Updated weights for policy 0, policy_version 1329677 (0.0006) [2023-12-27 01:04:25,309][105620] Updated weights for policy 1, policy_version 1331604 (0.0008) [2023-12-27 01:04:25,359][105620] Updated weights for policy 1, policy_version 1331614 (0.0008) [2023-12-27 01:04:25,919][105692] Updated weights for policy 0, policy_version 1329687 (0.0008) [2023-12-27 01:04:25,966][105692] Updated weights for policy 0, policy_version 1329697 (0.0009) [2023-12-27 01:04:26,014][105692] Updated weights for policy 0, policy_version 1329707 (0.0009) [2023-12-27 01:04:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 681394176. Throughput: 0: 9772.0, 1: 9630.5. Samples: 681400092. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:04:26,062][104569] Avg episode reward: [(0, '8822.199'), (1, '8817.322')] [2023-12-27 01:04:26,152][105620] Updated weights for policy 1, policy_version 1331624 (0.0009) [2023-12-27 01:04:26,211][105620] Updated weights for policy 1, policy_version 1331634 (0.0006) [2023-12-27 01:04:26,276][105620] Updated weights for policy 1, policy_version 1331644 (0.0005) [2023-12-27 01:04:26,826][105692] Updated weights for policy 0, policy_version 1329717 (0.0009) [2023-12-27 01:04:26,874][105692] Updated weights for policy 0, policy_version 1329727 (0.0009) [2023-12-27 01:04:26,934][105692] Updated weights for policy 0, policy_version 1329737 (0.0009) [2023-12-27 01:04:26,957][105620] Updated weights for policy 1, policy_version 1331654 (0.0006) [2023-12-27 01:04:27,006][105620] Updated weights for policy 1, policy_version 1331664 (0.0009) [2023-12-27 01:04:27,052][105620] Updated weights for policy 1, policy_version 1331674 (0.0008) [2023-12-27 01:04:27,682][105692] Updated weights for policy 0, policy_version 1329747 (0.0008) [2023-12-27 01:04:27,744][105692] Updated weights for policy 0, policy_version 1329757 (0.0009) [2023-12-27 01:04:27,798][105620] Updated weights for policy 1, policy_version 1331684 (0.0008) [2023-12-27 01:04:27,800][105692] Updated weights for policy 0, policy_version 1329767 (0.0007) [2023-12-27 01:04:27,847][105620] Updated weights for policy 1, policy_version 1331694 (0.0007) [2023-12-27 01:04:27,891][105620] Updated weights for policy 1, policy_version 1331704 (0.0008) [2023-12-27 01:04:28,559][105692] Updated weights for policy 0, policy_version 1329777 (0.0006) [2023-12-27 01:04:28,616][105692] Updated weights for policy 0, policy_version 1329787 (0.0008) [2023-12-27 01:04:28,650][105620] Updated weights for policy 1, policy_version 1331714 (0.0009) [2023-12-27 01:04:28,680][105692] Updated weights for policy 0, policy_version 1329797 (0.0006) [2023-12-27 01:04:28,709][105620] Updated weights for policy 1, policy_version 1331724 (0.0008) [2023-12-27 01:04:28,741][105692] Updated weights for policy 0, policy_version 1329807 (0.0005) [2023-12-27 01:04:28,763][105620] Updated weights for policy 1, policy_version 1331734 (0.0008) [2023-12-27 01:04:28,820][105620] Updated weights for policy 1, policy_version 1331744 (0.0009) [2023-12-27 01:04:29,469][105692] Updated weights for policy 0, policy_version 1329817 (0.0008) [2023-12-27 01:04:29,516][105692] Updated weights for policy 0, policy_version 1329827 (0.0009) [2023-12-27 01:04:29,570][105692] Updated weights for policy 0, policy_version 1329837 (0.0007) [2023-12-27 01:04:29,595][105620] Updated weights for policy 1, policy_version 1331754 (0.0009) [2023-12-27 01:04:29,652][105620] Updated weights for policy 1, policy_version 1331764 (0.0008) [2023-12-27 01:04:29,705][105620] Updated weights for policy 1, policy_version 1331774 (0.0008) [2023-12-27 01:04:30,339][105692] Updated weights for policy 0, policy_version 1329847 (0.0008) [2023-12-27 01:04:30,390][105692] Updated weights for policy 0, policy_version 1329857 (0.0009) [2023-12-27 01:04:30,436][105692] Updated weights for policy 0, policy_version 1329867 (0.0007) [2023-12-27 01:04:30,460][105620] Updated weights for policy 1, policy_version 1331784 (0.0007) [2023-12-27 01:04:30,506][105620] Updated weights for policy 1, policy_version 1331794 (0.0008) [2023-12-27 01:04:30,557][105620] Updated weights for policy 1, policy_version 1331805 (0.0010) [2023-12-27 01:04:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 681484288. Throughput: 0: 9779.8, 1: 9625.9. Samples: 681457460. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:04:31,062][104569] Avg episode reward: [(0, '8998.500'), (1, '8996.777')] [2023-12-27 01:04:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001331808_340983808.pth... [2023-12-27 01:04:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001329872_340500480.pth... [2023-12-27 01:04:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001328784_340221952.pth [2023-12-27 01:04:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001330688_340697088.pth [2023-12-27 01:04:31,153][105692] Updated weights for policy 0, policy_version 1329877 (0.0009) [2023-12-27 01:04:31,213][105692] Updated weights for policy 0, policy_version 1329887 (0.0009) [2023-12-27 01:04:31,278][105692] Updated weights for policy 0, policy_version 1329897 (0.0010) [2023-12-27 01:04:31,343][105620] Updated weights for policy 1, policy_version 1331815 (0.0007) [2023-12-27 01:04:31,411][105620] Updated weights for policy 1, policy_version 1331825 (0.0010) [2023-12-27 01:04:31,476][105620] Updated weights for policy 1, policy_version 1331835 (0.0009) [2023-12-27 01:04:32,046][105692] Updated weights for policy 0, policy_version 1329907 (0.0009) [2023-12-27 01:04:32,104][105692] Updated weights for policy 0, policy_version 1329917 (0.0009) [2023-12-27 01:04:32,155][105692] Updated weights for policy 0, policy_version 1329927 (0.0009) [2023-12-27 01:04:32,217][105620] Updated weights for policy 1, policy_version 1331845 (0.0008) [2023-12-27 01:04:32,276][105620] Updated weights for policy 1, policy_version 1331855 (0.0009) [2023-12-27 01:04:32,337][105620] Updated weights for policy 1, policy_version 1331865 (0.0009) [2023-12-27 01:04:32,923][105692] Updated weights for policy 0, policy_version 1329937 (0.0008) [2023-12-27 01:04:32,981][105692] Updated weights for policy 0, policy_version 1329947 (0.0005) [2023-12-27 01:04:33,031][105692] Updated weights for policy 0, policy_version 1329957 (0.0005) [2023-12-27 01:04:33,079][105692] Updated weights for policy 0, policy_version 1329967 (0.0005) [2023-12-27 01:04:33,126][105620] Updated weights for policy 1, policy_version 1331875 (0.0009) [2023-12-27 01:04:33,188][105620] Updated weights for policy 1, policy_version 1331885 (0.0010) [2023-12-27 01:04:33,257][105620] Updated weights for policy 1, policy_version 1331895 (0.0009) [2023-12-27 01:04:33,693][105692] Updated weights for policy 0, policy_version 1329977 (0.0008) [2023-12-27 01:04:33,755][105692] Updated weights for policy 0, policy_version 1329987 (0.0008) [2023-12-27 01:04:33,809][105692] Updated weights for policy 0, policy_version 1329997 (0.0008) [2023-12-27 01:04:34,005][105620] Updated weights for policy 1, policy_version 1331905 (0.0009) [2023-12-27 01:04:34,068][105620] Updated weights for policy 1, policy_version 1331915 (0.0008) [2023-12-27 01:04:34,131][105620] Updated weights for policy 1, policy_version 1331925 (0.0009) [2023-12-27 01:04:34,193][105620] Updated weights for policy 1, policy_version 1331935 (0.0009) [2023-12-27 01:04:34,565][105692] Updated weights for policy 0, policy_version 1330007 (0.0009) [2023-12-27 01:04:34,622][105692] Updated weights for policy 0, policy_version 1330017 (0.0010) [2023-12-27 01:04:34,675][105692] Updated weights for policy 0, policy_version 1330027 (0.0010) [2023-12-27 01:04:34,897][105620] Updated weights for policy 1, policy_version 1331945 (0.0008) [2023-12-27 01:04:34,952][105620] Updated weights for policy 1, policy_version 1331955 (0.0009) [2023-12-27 01:04:35,007][105620] Updated weights for policy 1, policy_version 1331965 (0.0009) [2023-12-27 01:04:35,450][105692] Updated weights for policy 0, policy_version 1330037 (0.0008) [2023-12-27 01:04:35,506][105692] Updated weights for policy 0, policy_version 1330047 (0.0008) [2023-12-27 01:04:35,557][105692] Updated weights for policy 0, policy_version 1330057 (0.0009) [2023-12-27 01:04:35,762][105620] Updated weights for policy 1, policy_version 1331975 (0.0007) [2023-12-27 01:04:35,815][105620] Updated weights for policy 1, policy_version 1331985 (0.0005) [2023-12-27 01:04:35,874][105620] Updated weights for policy 1, policy_version 1331995 (0.0005) [2023-12-27 01:04:36,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 681582592. Throughput: 0: 9705.9, 1: 9613.5. Samples: 681570532. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:04:36,062][104569] Avg episode reward: [(0, '8639.233'), (1, '9267.633')] [2023-12-27 01:04:36,270][105692] Updated weights for policy 0, policy_version 1330067 (0.0008) [2023-12-27 01:04:36,336][105692] Updated weights for policy 0, policy_version 1330077 (0.0009) [2023-12-27 01:04:36,395][105692] Updated weights for policy 0, policy_version 1330087 (0.0010) [2023-12-27 01:04:36,500][105620] Updated weights for policy 1, policy_version 1332005 (0.0007) [2023-12-27 01:04:36,564][105620] Updated weights for policy 1, policy_version 1332015 (0.0008) [2023-12-27 01:04:36,627][105620] Updated weights for policy 1, policy_version 1332025 (0.0009) [2023-12-27 01:04:37,141][105692] Updated weights for policy 0, policy_version 1330097 (0.0009) [2023-12-27 01:04:37,199][105692] Updated weights for policy 0, policy_version 1330107 (0.0009) [2023-12-27 01:04:37,254][105692] Updated weights for policy 0, policy_version 1330117 (0.0009) [2023-12-27 01:04:37,302][105692] Updated weights for policy 0, policy_version 1330127 (0.0009) [2023-12-27 01:04:37,365][105620] Updated weights for policy 1, policy_version 1332035 (0.0009) [2023-12-27 01:04:37,426][105620] Updated weights for policy 1, policy_version 1332045 (0.0009) [2023-12-27 01:04:37,480][105620] Updated weights for policy 1, policy_version 1332055 (0.0008) [2023-12-27 01:04:38,087][105692] Updated weights for policy 0, policy_version 1330137 (0.0009) [2023-12-27 01:04:38,135][105692] Updated weights for policy 0, policy_version 1330147 (0.0009) [2023-12-27 01:04:38,182][105692] Updated weights for policy 0, policy_version 1330157 (0.0009) [2023-12-27 01:04:38,251][105620] Updated weights for policy 1, policy_version 1332065 (0.0009) [2023-12-27 01:04:38,301][105620] Updated weights for policy 1, policy_version 1332075 (0.0009) [2023-12-27 01:04:38,363][105620] Updated weights for policy 1, policy_version 1332085 (0.0009) [2023-12-27 01:04:38,420][105620] Updated weights for policy 1, policy_version 1332095 (0.0009) [2023-12-27 01:04:38,984][105692] Updated weights for policy 0, policy_version 1330167 (0.0008) [2023-12-27 01:04:39,046][105692] Updated weights for policy 0, policy_version 1330177 (0.0009) [2023-12-27 01:04:39,105][105692] Updated weights for policy 0, policy_version 1330187 (0.0008) [2023-12-27 01:04:39,151][105620] Updated weights for policy 1, policy_version 1332105 (0.0009) [2023-12-27 01:04:39,211][105620] Updated weights for policy 1, policy_version 1332115 (0.0009) [2023-12-27 01:04:39,273][105620] Updated weights for policy 1, policy_version 1332125 (0.0008) [2023-12-27 01:04:39,886][105692] Updated weights for policy 0, policy_version 1330197 (0.0008) [2023-12-27 01:04:39,943][105692] Updated weights for policy 0, policy_version 1330207 (0.0008) [2023-12-27 01:04:40,013][105692] Updated weights for policy 0, policy_version 1330217 (0.0008) [2023-12-27 01:04:40,043][105620] Updated weights for policy 1, policy_version 1332135 (0.0008) [2023-12-27 01:04:40,098][105620] Updated weights for policy 1, policy_version 1332145 (0.0010) [2023-12-27 01:04:40,158][105620] Updated weights for policy 1, policy_version 1332155 (0.0009) [2023-12-27 01:04:40,748][105692] Updated weights for policy 0, policy_version 1330227 (0.0007) [2023-12-27 01:04:40,798][105692] Updated weights for policy 0, policy_version 1330237 (0.0006) [2023-12-27 01:04:40,857][105692] Updated weights for policy 0, policy_version 1330247 (0.0007) [2023-12-27 01:04:40,880][105620] Updated weights for policy 1, policy_version 1332165 (0.0009) [2023-12-27 01:04:40,939][105620] Updated weights for policy 1, policy_version 1332175 (0.0007) [2023-12-27 01:04:41,000][105620] Updated weights for policy 1, policy_version 1332185 (0.0009) [2023-12-27 01:04:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 681680896. Throughput: 0: 9589.3, 1: 9615.9. Samples: 681682792. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:04:41,063][104569] Avg episode reward: [(0, '8817.664'), (1, '9177.080')] [2023-12-27 01:04:41,581][105692] Updated weights for policy 0, policy_version 1330257 (0.0009) [2023-12-27 01:04:41,647][105692] Updated weights for policy 0, policy_version 1330267 (0.0009) [2023-12-27 01:04:41,710][105692] Updated weights for policy 0, policy_version 1330277 (0.0009) [2023-12-27 01:04:41,777][105692] Updated weights for policy 0, policy_version 1330287 (0.0007) [2023-12-27 01:04:41,780][105620] Updated weights for policy 1, policy_version 1332195 (0.0009) [2023-12-27 01:04:41,846][105620] Updated weights for policy 1, policy_version 1332205 (0.0010) [2023-12-27 01:04:41,904][105620] Updated weights for policy 1, policy_version 1332215 (0.0009) [2023-12-27 01:04:42,467][105692] Updated weights for policy 0, policy_version 1330297 (0.0009) [2023-12-27 01:04:42,528][105692] Updated weights for policy 0, policy_version 1330307 (0.0008) [2023-12-27 01:04:42,585][105692] Updated weights for policy 0, policy_version 1330317 (0.0009) [2023-12-27 01:04:42,712][105620] Updated weights for policy 1, policy_version 1332225 (0.0009) [2023-12-27 01:04:42,771][105620] Updated weights for policy 1, policy_version 1332235 (0.0009) [2023-12-27 01:04:42,821][105620] Updated weights for policy 1, policy_version 1332245 (0.0008) [2023-12-27 01:04:42,875][105620] Updated weights for policy 1, policy_version 1332255 (0.0008) [2023-12-27 01:04:43,273][105692] Updated weights for policy 0, policy_version 1330327 (0.0009) [2023-12-27 01:04:43,327][105692] Updated weights for policy 0, policy_version 1330337 (0.0009) [2023-12-27 01:04:43,381][105692] Updated weights for policy 0, policy_version 1330347 (0.0009) [2023-12-27 01:04:43,663][105620] Updated weights for policy 1, policy_version 1332265 (0.0009) [2023-12-27 01:04:43,709][105620] Updated weights for policy 1, policy_version 1332275 (0.0009) [2023-12-27 01:04:43,756][105620] Updated weights for policy 1, policy_version 1332285 (0.0009) [2023-12-27 01:04:44,131][105692] Updated weights for policy 0, policy_version 1330357 (0.0009) [2023-12-27 01:04:44,185][105692] Updated weights for policy 0, policy_version 1330367 (0.0006) [2023-12-27 01:04:44,248][105692] Updated weights for policy 0, policy_version 1330377 (0.0005) [2023-12-27 01:04:44,559][105620] Updated weights for policy 1, policy_version 1332295 (0.0008) [2023-12-27 01:04:44,626][105620] Updated weights for policy 1, policy_version 1332305 (0.0010) [2023-12-27 01:04:44,683][105620] Updated weights for policy 1, policy_version 1332316 (0.0009) [2023-12-27 01:04:44,886][105692] Updated weights for policy 0, policy_version 1330387 (0.0006) [2023-12-27 01:04:44,948][105692] Updated weights for policy 0, policy_version 1330397 (0.0009) [2023-12-27 01:04:45,006][105692] Updated weights for policy 0, policy_version 1330407 (0.0009) [2023-12-27 01:04:45,509][105620] Updated weights for policy 1, policy_version 1332326 (0.0009) [2023-12-27 01:04:45,564][105620] Updated weights for policy 1, policy_version 1332336 (0.0009) [2023-12-27 01:04:45,625][105620] Updated weights for policy 1, policy_version 1332346 (0.0009) [2023-12-27 01:04:45,667][105692] Updated weights for policy 0, policy_version 1330417 (0.0007) [2023-12-27 01:04:45,725][105692] Updated weights for policy 0, policy_version 1330427 (0.0009) [2023-12-27 01:04:45,772][105692] Updated weights for policy 0, policy_version 1330437 (0.0009) [2023-12-27 01:04:45,828][105692] Updated weights for policy 0, policy_version 1330447 (0.0010) [2023-12-27 01:04:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 681771008. Throughput: 0: 9486.2, 1: 9485.6. Samples: 681738556. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:04:46,063][104569] Avg episode reward: [(0, '8994.680'), (1, '8911.215')] [2023-12-27 01:04:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001332352_341123072.pth... [2023-12-27 01:04:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001330448_340647936.pth... [2023-12-27 01:04:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001331264_340844544.pth [2023-12-27 01:04:46,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001329328_340361216.pth [2023-12-27 01:04:46,404][105620] Updated weights for policy 1, policy_version 1332356 (0.0007) [2023-12-27 01:04:46,464][105620] Updated weights for policy 1, policy_version 1332366 (0.0010) [2023-12-27 01:04:46,492][105692] Updated weights for policy 0, policy_version 1330457 (0.0006) [2023-12-27 01:04:46,513][105620] Updated weights for policy 1, policy_version 1332376 (0.0010) [2023-12-27 01:04:46,541][105692] Updated weights for policy 0, policy_version 1330467 (0.0005) [2023-12-27 01:04:46,596][105692] Updated weights for policy 0, policy_version 1330477 (0.0005) [2023-12-27 01:04:47,100][105620] Updated weights for policy 1, policy_version 1332386 (0.0011) [2023-12-27 01:04:47,152][105620] Updated weights for policy 1, policy_version 1332396 (0.0010) [2023-12-27 01:04:47,198][105620] Updated weights for policy 1, policy_version 1332406 (0.0010) [2023-12-27 01:04:47,244][105692] Updated weights for policy 0, policy_version 1330487 (0.0009) [2023-12-27 01:04:47,246][105620] Updated weights for policy 1, policy_version 1332416 (0.0010) [2023-12-27 01:04:47,303][105692] Updated weights for policy 0, policy_version 1330497 (0.0010) [2023-12-27 01:04:47,361][105692] Updated weights for policy 0, policy_version 1330507 (0.0010) [2023-12-27 01:04:47,963][105620] Updated weights for policy 1, policy_version 1332426 (0.0006) [2023-12-27 01:04:48,021][105620] Updated weights for policy 1, policy_version 1332436 (0.0005) [2023-12-27 01:04:48,051][105692] Updated weights for policy 0, policy_version 1330517 (0.0010) [2023-12-27 01:04:48,080][105620] Updated weights for policy 1, policy_version 1332446 (0.0005) [2023-12-27 01:04:48,114][105692] Updated weights for policy 0, policy_version 1330527 (0.0011) [2023-12-27 01:04:48,179][105692] Updated weights for policy 0, policy_version 1330537 (0.0010) [2023-12-27 01:04:48,781][105620] Updated weights for policy 1, policy_version 1332456 (0.0008) [2023-12-27 01:04:48,832][105620] Updated weights for policy 1, policy_version 1332466 (0.0009) [2023-12-27 01:04:48,851][105692] Updated weights for policy 0, policy_version 1330547 (0.0010) [2023-12-27 01:04:48,886][105620] Updated weights for policy 1, policy_version 1332476 (0.0008) [2023-12-27 01:04:48,900][105692] Updated weights for policy 0, policy_version 1330557 (0.0006) [2023-12-27 01:04:48,957][105692] Updated weights for policy 0, policy_version 1330567 (0.0009) [2023-12-27 01:04:49,583][105620] Updated weights for policy 1, policy_version 1332486 (0.0008) [2023-12-27 01:04:49,650][105620] Updated weights for policy 1, policy_version 1332496 (0.0008) [2023-12-27 01:04:49,709][105620] Updated weights for policy 1, policy_version 1332506 (0.0008) [2023-12-27 01:04:49,717][105692] Updated weights for policy 0, policy_version 1330577 (0.0010) [2023-12-27 01:04:49,776][105692] Updated weights for policy 0, policy_version 1330587 (0.0011) [2023-12-27 01:04:49,838][105692] Updated weights for policy 0, policy_version 1330597 (0.0010) [2023-12-27 01:04:49,900][105692] Updated weights for policy 0, policy_version 1330607 (0.0010) [2023-12-27 01:04:50,401][105620] Updated weights for policy 1, policy_version 1332516 (0.0007) [2023-12-27 01:04:50,455][105620] Updated weights for policy 1, policy_version 1332526 (0.0008) [2023-12-27 01:04:50,511][105620] Updated weights for policy 1, policy_version 1332536 (0.0008) [2023-12-27 01:04:50,644][105692] Updated weights for policy 0, policy_version 1330617 (0.0010) [2023-12-27 01:04:50,700][105692] Updated weights for policy 0, policy_version 1330627 (0.0011) [2023-12-27 01:04:50,760][105692] Updated weights for policy 0, policy_version 1330637 (0.0011) [2023-12-27 01:04:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 681869312. Throughput: 0: 9579.4, 1: 9512.8. Samples: 681857916. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:04:51,063][104569] Avg episode reward: [(0, '8723.525'), (1, '8912.829')] [2023-12-27 01:04:51,194][105620] Updated weights for policy 1, policy_version 1332546 (0.0008) [2023-12-27 01:04:51,250][105620] Updated weights for policy 1, policy_version 1332556 (0.0008) [2023-12-27 01:04:51,318][105620] Updated weights for policy 1, policy_version 1332566 (0.0009) [2023-12-27 01:04:51,391][105620] Updated weights for policy 1, policy_version 1332576 (0.0008) [2023-12-27 01:04:51,488][105692] Updated weights for policy 0, policy_version 1330647 (0.0011) [2023-12-27 01:04:51,542][105692] Updated weights for policy 0, policy_version 1330657 (0.0007) [2023-12-27 01:04:51,604][105692] Updated weights for policy 0, policy_version 1330667 (0.0011) [2023-12-27 01:04:52,137][105620] Updated weights for policy 1, policy_version 1332586 (0.0008) [2023-12-27 01:04:52,183][105620] Updated weights for policy 1, policy_version 1332596 (0.0008) [2023-12-27 01:04:52,238][105620] Updated weights for policy 1, policy_version 1332606 (0.0008) [2023-12-27 01:04:52,330][105692] Updated weights for policy 0, policy_version 1330677 (0.0009) [2023-12-27 01:04:52,403][105692] Updated weights for policy 0, policy_version 1330687 (0.0011) [2023-12-27 01:04:52,455][105692] Updated weights for policy 0, policy_version 1330697 (0.0011) [2023-12-27 01:04:53,009][105620] Updated weights for policy 1, policy_version 1332616 (0.0008) [2023-12-27 01:04:53,068][105620] Updated weights for policy 1, policy_version 1332626 (0.0008) [2023-12-27 01:04:53,113][105620] Updated weights for policy 1, policy_version 1332636 (0.0008) [2023-12-27 01:04:53,212][105692] Updated weights for policy 0, policy_version 1330707 (0.0009) [2023-12-27 01:04:53,278][105692] Updated weights for policy 0, policy_version 1330717 (0.0008) [2023-12-27 01:04:53,329][105692] Updated weights for policy 0, policy_version 1330727 (0.0006) [2023-12-27 01:04:53,882][105620] Updated weights for policy 1, policy_version 1332646 (0.0006) [2023-12-27 01:04:53,920][105692] Updated weights for policy 0, policy_version 1330737 (0.0006) [2023-12-27 01:04:53,942][105620] Updated weights for policy 1, policy_version 1332656 (0.0007) [2023-12-27 01:04:53,972][105692] Updated weights for policy 0, policy_version 1330747 (0.0010) [2023-12-27 01:04:54,005][105620] Updated weights for policy 1, policy_version 1332666 (0.0006) [2023-12-27 01:04:54,030][105692] Updated weights for policy 0, policy_version 1330757 (0.0011) [2023-12-27 01:04:54,092][105692] Updated weights for policy 0, policy_version 1330767 (0.0010) [2023-12-27 01:04:54,626][105620] Updated weights for policy 1, policy_version 1332676 (0.0007) [2023-12-27 01:04:54,693][105620] Updated weights for policy 1, policy_version 1332686 (0.0010) [2023-12-27 01:04:54,753][105620] Updated weights for policy 1, policy_version 1332696 (0.0009) [2023-12-27 01:04:54,771][105692] Updated weights for policy 0, policy_version 1330777 (0.0008) [2023-12-27 01:04:54,822][105692] Updated weights for policy 0, policy_version 1330787 (0.0008) [2023-12-27 01:04:54,871][105692] Updated weights for policy 0, policy_version 1330797 (0.0008) [2023-12-27 01:04:55,502][105620] Updated weights for policy 1, policy_version 1332706 (0.0007) [2023-12-27 01:04:55,562][105620] Updated weights for policy 1, policy_version 1332716 (0.0008) [2023-12-27 01:04:55,603][105692] Updated weights for policy 0, policy_version 1330807 (0.0009) [2023-12-27 01:04:55,621][105620] Updated weights for policy 1, policy_version 1332726 (0.0005) [2023-12-27 01:04:55,657][105692] Updated weights for policy 0, policy_version 1330817 (0.0011) [2023-12-27 01:04:55,679][105620] Updated weights for policy 1, policy_version 1332736 (0.0005) [2023-12-27 01:04:55,705][105692] Updated weights for policy 0, policy_version 1330827 (0.0010) [2023-12-27 01:04:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 681967616. Throughput: 0: 9673.6, 1: 9520.5. Samples: 681974808. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:04:56,063][104569] Avg episode reward: [(0, '8632.517'), (1, '9090.373')] [2023-12-27 01:04:56,291][105620] Updated weights for policy 1, policy_version 1332746 (0.0005) [2023-12-27 01:04:56,345][105620] Updated weights for policy 1, policy_version 1332756 (0.0005) [2023-12-27 01:04:56,398][105692] Updated weights for policy 0, policy_version 1330837 (0.0008) [2023-12-27 01:04:56,418][105620] Updated weights for policy 1, policy_version 1332766 (0.0006) [2023-12-27 01:04:56,465][105692] Updated weights for policy 0, policy_version 1330847 (0.0005) [2023-12-27 01:04:56,485][105585] KL-divergence is very high: 234.8759 [2023-12-27 01:04:56,523][105692] Updated weights for policy 0, policy_version 1330857 (0.0005) [2023-12-27 01:04:56,530][105585] KL-divergence is very high: 439.7298 [2023-12-27 01:04:57,092][105620] Updated weights for policy 1, policy_version 1332776 (0.0006) [2023-12-27 01:04:57,150][105620] Updated weights for policy 1, policy_version 1332786 (0.0005) [2023-12-27 01:04:57,159][105692] Updated weights for policy 0, policy_version 1330867 (0.0007) [2023-12-27 01:04:57,210][105692] Updated weights for policy 0, policy_version 1330877 (0.0010) [2023-12-27 01:04:57,219][105620] Updated weights for policy 1, policy_version 1332796 (0.0005) [2023-12-27 01:04:57,262][105692] Updated weights for policy 0, policy_version 1330887 (0.0010) [2023-12-27 01:04:57,810][105620] Updated weights for policy 1, policy_version 1332806 (0.0005) [2023-12-27 01:04:57,867][105620] Updated weights for policy 1, policy_version 1332816 (0.0005) [2023-12-27 01:04:57,925][105620] Updated weights for policy 1, policy_version 1332826 (0.0005) [2023-12-27 01:04:58,022][105692] Updated weights for policy 0, policy_version 1330897 (0.0010) [2023-12-27 01:04:58,083][105692] Updated weights for policy 0, policy_version 1330907 (0.0011) [2023-12-27 01:04:58,134][105692] Updated weights for policy 0, policy_version 1330917 (0.0010) [2023-12-27 01:04:58,196][105692] Updated weights for policy 0, policy_version 1330927 (0.0009) [2023-12-27 01:04:58,585][105620] Updated weights for policy 1, policy_version 1332836 (0.0009) [2023-12-27 01:04:58,648][105620] Updated weights for policy 1, policy_version 1332846 (0.0011) [2023-12-27 01:04:58,712][105620] Updated weights for policy 1, policy_version 1332856 (0.0010) [2023-12-27 01:04:59,027][105692] Updated weights for policy 0, policy_version 1330937 (0.0010) [2023-12-27 01:04:59,087][105692] Updated weights for policy 0, policy_version 1330947 (0.0011) [2023-12-27 01:04:59,145][105692] Updated weights for policy 0, policy_version 1330957 (0.0010) [2023-12-27 01:04:59,508][105620] Updated weights for policy 1, policy_version 1332866 (0.0009) [2023-12-27 01:04:59,564][105620] Updated weights for policy 1, policy_version 1332876 (0.0006) [2023-12-27 01:04:59,625][105620] Updated weights for policy 1, policy_version 1332886 (0.0008) [2023-12-27 01:04:59,683][105620] Updated weights for policy 1, policy_version 1332896 (0.0009) [2023-12-27 01:04:59,986][105692] Updated weights for policy 0, policy_version 1330967 (0.0007) [2023-12-27 01:05:00,048][105692] Updated weights for policy 0, policy_version 1330977 (0.0006) [2023-12-27 01:05:00,111][105692] Updated weights for policy 0, policy_version 1330987 (0.0006) [2023-12-27 01:05:00,359][105620] Updated weights for policy 1, policy_version 1332906 (0.0005) [2023-12-27 01:05:00,415][105620] Updated weights for policy 1, policy_version 1332916 (0.0006) [2023-12-27 01:05:00,461][105620] Updated weights for policy 1, policy_version 1332926 (0.0008) [2023-12-27 01:05:00,789][105692] Updated weights for policy 0, policy_version 1330997 (0.0007) [2023-12-27 01:05:00,847][105692] Updated weights for policy 0, policy_version 1331007 (0.0009) [2023-12-27 01:05:00,893][105692] Updated weights for policy 0, policy_version 1331017 (0.0009) [2023-12-27 01:05:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 682065920. Throughput: 0: 9707.2, 1: 9578.0. Samples: 682035540. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:05:01,062][104569] Avg episode reward: [(0, '8723.174'), (1, '9179.249')] [2023-12-27 01:05:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001331024_340795392.pth... [2023-12-27 01:05:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001329872_340500480.pth [2023-12-27 01:05:01,110][105620] Updated weights for policy 1, policy_version 1332936 (0.0008) [2023-12-27 01:05:01,168][105620] Updated weights for policy 1, policy_version 1332946 (0.0009) [2023-12-27 01:05:01,225][105620] Updated weights for policy 1, policy_version 1332956 (0.0009) [2023-12-27 01:05:01,249][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001332960_341278720.pth... [2023-12-27 01:05:01,253][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001331808_340983808.pth [2023-12-27 01:05:01,584][105692] Updated weights for policy 0, policy_version 1331027 (0.0009) [2023-12-27 01:05:01,647][105692] Updated weights for policy 0, policy_version 1331037 (0.0007) [2023-12-27 01:05:01,705][105692] Updated weights for policy 0, policy_version 1331047 (0.0007) [2023-12-27 01:05:01,957][105620] Updated weights for policy 1, policy_version 1332966 (0.0007) [2023-12-27 01:05:02,010][105620] Updated weights for policy 1, policy_version 1332976 (0.0010) [2023-12-27 01:05:02,074][105620] Updated weights for policy 1, policy_version 1332986 (0.0011) [2023-12-27 01:05:02,357][105692] Updated weights for policy 0, policy_version 1331057 (0.0006) [2023-12-27 01:05:02,412][105585] KL-divergence is very high: 151.2720 [2023-12-27 01:05:02,423][105692] Updated weights for policy 0, policy_version 1331067 (0.0007) [2023-12-27 01:05:02,461][105585] KL-divergence is very high: 258.1383 [2023-12-27 01:05:02,483][105692] Updated weights for policy 0, policy_version 1331077 (0.0005) [2023-12-27 01:05:02,510][105585] KL-divergence is very high: 266.8848 [2023-12-27 01:05:02,549][105692] Updated weights for policy 0, policy_version 1331087 (0.0007) [2023-12-27 01:05:02,749][105620] Updated weights for policy 1, policy_version 1332996 (0.0009) [2023-12-27 01:05:02,813][105620] Updated weights for policy 1, policy_version 1333006 (0.0010) [2023-12-27 01:05:02,873][105620] Updated weights for policy 1, policy_version 1333016 (0.0009) [2023-12-27 01:05:03,186][105692] Updated weights for policy 0, policy_version 1331097 (0.0005) [2023-12-27 01:05:03,232][105692] Updated weights for policy 0, policy_version 1331107 (0.0005) [2023-12-27 01:05:03,278][105692] Updated weights for policy 0, policy_version 1331117 (0.0005) [2023-12-27 01:05:03,431][105620] Updated weights for policy 1, policy_version 1333026 (0.0006) [2023-12-27 01:05:03,485][105620] Updated weights for policy 1, policy_version 1333036 (0.0010) [2023-12-27 01:05:03,537][105620] Updated weights for policy 1, policy_version 1333046 (0.0009) [2023-12-27 01:05:03,593][105620] Updated weights for policy 1, policy_version 1333056 (0.0005) [2023-12-27 01:05:04,024][105692] Updated weights for policy 0, policy_version 1331127 (0.0010) [2023-12-27 01:05:04,070][105692] Updated weights for policy 0, policy_version 1331137 (0.0008) [2023-12-27 01:05:04,126][105692] Updated weights for policy 0, policy_version 1331147 (0.0009) [2023-12-27 01:05:04,230][105620] Updated weights for policy 1, policy_version 1333066 (0.0009) [2023-12-27 01:05:04,294][105620] Updated weights for policy 1, policy_version 1333076 (0.0007) [2023-12-27 01:05:04,364][105620] Updated weights for policy 1, policy_version 1333086 (0.0008) [2023-12-27 01:05:04,910][105692] Updated weights for policy 0, policy_version 1331157 (0.0009) [2023-12-27 01:05:04,967][105692] Updated weights for policy 0, policy_version 1331167 (0.0009) [2023-12-27 01:05:05,025][105692] Updated weights for policy 0, policy_version 1331177 (0.0009) [2023-12-27 01:05:05,080][105620] Updated weights for policy 1, policy_version 1333096 (0.0008) [2023-12-27 01:05:05,133][105620] Updated weights for policy 1, policy_version 1333106 (0.0010) [2023-12-27 01:05:05,191][105620] Updated weights for policy 1, policy_version 1333116 (0.0008) [2023-12-27 01:05:05,696][105692] Updated weights for policy 0, policy_version 1331187 (0.0006) [2023-12-27 01:05:05,764][105692] Updated weights for policy 0, policy_version 1331197 (0.0005) [2023-12-27 01:05:05,824][105692] Updated weights for policy 0, policy_version 1331207 (0.0008) [2023-12-27 01:05:05,857][105620] Updated weights for policy 1, policy_version 1333126 (0.0008) [2023-12-27 01:05:05,916][105620] Updated weights for policy 1, policy_version 1333136 (0.0010) [2023-12-27 01:05:05,974][105620] Updated weights for policy 1, policy_version 1333146 (0.0010) [2023-12-27 01:05:06,062][104569] Fps is (10 sec: 20479.4, 60 sec: 19387.6, 300 sec: 19410.9). Total num frames: 682172416. Throughput: 0: 9624.8, 1: 9652.6. Samples: 682154996. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:05:06,064][104569] Avg episode reward: [(0, '8540.110'), (1, '9358.015')] [2023-12-27 01:05:06,536][105692] Updated weights for policy 0, policy_version 1331217 (0.0006) [2023-12-27 01:05:06,599][105692] Updated weights for policy 0, policy_version 1331227 (0.0008) [2023-12-27 01:05:06,667][105692] Updated weights for policy 0, policy_version 1331237 (0.0008) [2023-12-27 01:05:06,731][105692] Updated weights for policy 0, policy_version 1331247 (0.0008) [2023-12-27 01:05:06,742][105620] Updated weights for policy 1, policy_version 1333156 (0.0011) [2023-12-27 01:05:06,792][105620] Updated weights for policy 1, policy_version 1333166 (0.0011) [2023-12-27 01:05:06,838][105620] Updated weights for policy 1, policy_version 1333176 (0.0011) [2023-12-27 01:05:07,452][105692] Updated weights for policy 0, policy_version 1331257 (0.0009) [2023-12-27 01:05:07,512][105692] Updated weights for policy 0, policy_version 1331267 (0.0009) [2023-12-27 01:05:07,573][105692] Updated weights for policy 0, policy_version 1331277 (0.0008) [2023-12-27 01:05:07,592][105620] Updated weights for policy 1, policy_version 1333186 (0.0010) [2023-12-27 01:05:07,650][105620] Updated weights for policy 1, policy_version 1333196 (0.0005) [2023-12-27 01:05:07,715][105620] Updated weights for policy 1, policy_version 1333206 (0.0005) [2023-12-27 01:05:07,779][105620] Updated weights for policy 1, policy_version 1333216 (0.0006) [2023-12-27 01:05:08,344][105620] Updated weights for policy 1, policy_version 1333226 (0.0006) [2023-12-27 01:05:08,402][105620] Updated weights for policy 1, policy_version 1333236 (0.0007) [2023-12-27 01:05:08,425][105692] Updated weights for policy 0, policy_version 1331287 (0.0009) [2023-12-27 01:05:08,458][105620] Updated weights for policy 1, policy_version 1333246 (0.0008) [2023-12-27 01:05:08,482][105692] Updated weights for policy 0, policy_version 1331297 (0.0008) [2023-12-27 01:05:08,546][105692] Updated weights for policy 0, policy_version 1331307 (0.0009) [2023-12-27 01:05:09,128][105620] Updated weights for policy 1, policy_version 1333256 (0.0007) [2023-12-27 01:05:09,187][105620] Updated weights for policy 1, policy_version 1333266 (0.0010) [2023-12-27 01:05:09,251][105620] Updated weights for policy 1, policy_version 1333276 (0.0012) [2023-12-27 01:05:09,373][105692] Updated weights for policy 0, policy_version 1331317 (0.0009) [2023-12-27 01:05:09,437][105692] Updated weights for policy 0, policy_version 1331327 (0.0008) [2023-12-27 01:05:09,462][105585] KL-divergence is very high: 215.4145 [2023-12-27 01:05:09,490][105692] Updated weights for policy 0, policy_version 1331337 (0.0008) [2023-12-27 01:05:09,510][105585] KL-divergence is very high: 351.8460 [2023-12-27 01:05:09,954][105620] Updated weights for policy 1, policy_version 1333286 (0.0009) [2023-12-27 01:05:10,021][105620] Updated weights for policy 1, policy_version 1333296 (0.0010) [2023-12-27 01:05:10,081][105620] Updated weights for policy 1, policy_version 1333306 (0.0011) [2023-12-27 01:05:10,287][105692] Updated weights for policy 0, policy_version 1331347 (0.0008) [2023-12-27 01:05:10,342][105692] Updated weights for policy 0, policy_version 1331357 (0.0008) [2023-12-27 01:05:10,400][105692] Updated weights for policy 0, policy_version 1331367 (0.0007) [2023-12-27 01:05:10,756][105620] Updated weights for policy 1, policy_version 1333316 (0.0009) [2023-12-27 01:05:10,806][105620] Updated weights for policy 1, policy_version 1333326 (0.0009) [2023-12-27 01:05:10,864][105620] Updated weights for policy 1, policy_version 1333336 (0.0009) [2023-12-27 01:05:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 682262528. Throughput: 0: 9610.6, 1: 9726.7. Samples: 682270268. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:05:11,062][104569] Avg episode reward: [(0, '8272.719'), (1, '9087.871')] [2023-12-27 01:05:11,202][105692] Updated weights for policy 0, policy_version 1331377 (0.0007) [2023-12-27 01:05:11,265][105692] Updated weights for policy 0, policy_version 1331387 (0.0008) [2023-12-27 01:05:11,317][105692] Updated weights for policy 0, policy_version 1331397 (0.0008) [2023-12-27 01:05:11,378][105692] Updated weights for policy 0, policy_version 1331407 (0.0008) [2023-12-27 01:05:11,631][105620] Updated weights for policy 1, policy_version 1333346 (0.0006) [2023-12-27 01:05:11,686][105620] Updated weights for policy 1, policy_version 1333356 (0.0008) [2023-12-27 01:05:11,755][105620] Updated weights for policy 1, policy_version 1333366 (0.0010) [2023-12-27 01:05:11,816][105620] Updated weights for policy 1, policy_version 1333376 (0.0009) [2023-12-27 01:05:12,214][105692] Updated weights for policy 0, policy_version 1331417 (0.0009) [2023-12-27 01:05:12,277][105692] Updated weights for policy 0, policy_version 1331428 (0.0010) [2023-12-27 01:05:12,339][105692] Updated weights for policy 0, policy_version 1331438 (0.0007) [2023-12-27 01:05:12,520][105620] Updated weights for policy 1, policy_version 1333386 (0.0008) [2023-12-27 01:05:12,587][105620] Updated weights for policy 1, policy_version 1333396 (0.0008) [2023-12-27 01:05:12,646][105620] Updated weights for policy 1, policy_version 1333406 (0.0008) [2023-12-27 01:05:13,111][105692] Updated weights for policy 0, policy_version 1331448 (0.0006) [2023-12-27 01:05:13,180][105692] Updated weights for policy 0, policy_version 1331458 (0.0005) [2023-12-27 01:05:13,235][105692] Updated weights for policy 0, policy_version 1331468 (0.0005) [2023-12-27 01:05:13,476][105620] Updated weights for policy 1, policy_version 1333416 (0.0009) [2023-12-27 01:05:13,547][105620] Updated weights for policy 1, policy_version 1333426 (0.0010) [2023-12-27 01:05:13,615][105620] Updated weights for policy 1, policy_version 1333436 (0.0009) [2023-12-27 01:05:13,764][105692] Updated weights for policy 0, policy_version 1331478 (0.0008) [2023-12-27 01:05:13,812][105692] Updated weights for policy 0, policy_version 1331488 (0.0010) [2023-12-27 01:05:13,860][105692] Updated weights for policy 0, policy_version 1331498 (0.0010) [2023-12-27 01:05:14,330][105620] Updated weights for policy 1, policy_version 1333446 (0.0008) [2023-12-27 01:05:14,395][105620] Updated weights for policy 1, policy_version 1333456 (0.0009) [2023-12-27 01:05:14,460][105620] Updated weights for policy 1, policy_version 1333466 (0.0009) [2023-12-27 01:05:14,577][105692] Updated weights for policy 0, policy_version 1331508 (0.0010) [2023-12-27 01:05:14,636][105692] Updated weights for policy 0, policy_version 1331518 (0.0010) [2023-12-27 01:05:14,690][105692] Updated weights for policy 0, policy_version 1331528 (0.0009) [2023-12-27 01:05:15,188][105620] Updated weights for policy 1, policy_version 1333476 (0.0007) [2023-12-27 01:05:15,255][105620] Updated weights for policy 1, policy_version 1333486 (0.0005) [2023-12-27 01:05:15,309][105620] Updated weights for policy 1, policy_version 1333496 (0.0005) [2023-12-27 01:05:15,465][105692] Updated weights for policy 0, policy_version 1331538 (0.0011) [2023-12-27 01:05:15,513][105692] Updated weights for policy 0, policy_version 1331548 (0.0010) [2023-12-27 01:05:15,568][105692] Updated weights for policy 0, policy_version 1331558 (0.0010) [2023-12-27 01:05:15,619][105692] Updated weights for policy 0, policy_version 1331568 (0.0010) [2023-12-27 01:05:15,846][105620] Updated weights for policy 1, policy_version 1333506 (0.0008) [2023-12-27 01:05:15,899][105620] Updated weights for policy 1, policy_version 1333517 (0.0010) [2023-12-27 01:05:15,946][105620] Updated weights for policy 1, policy_version 1333527 (0.0007) [2023-12-27 01:05:16,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 682360832. Throughput: 0: 9604.0, 1: 9690.0. Samples: 682325688. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:05:16,062][104569] Avg episode reward: [(0, '8631.630'), (1, '9087.584')] [2023-12-27 01:05:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001333536_341426176.pth... [2023-12-27 01:05:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001331568_340934656.pth... [2023-12-27 01:05:16,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001330448_340647936.pth [2023-12-27 01:05:16,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001332352_341123072.pth [2023-12-27 01:05:16,235][105692] Updated weights for policy 0, policy_version 1331578 (0.0005) [2023-12-27 01:05:16,289][105692] Updated weights for policy 0, policy_version 1331588 (0.0005) [2023-12-27 01:05:16,343][105692] Updated weights for policy 0, policy_version 1331598 (0.0005) [2023-12-27 01:05:16,698][105620] Updated weights for policy 1, policy_version 1333537 (0.0006) [2023-12-27 01:05:16,759][105620] Updated weights for policy 1, policy_version 1333547 (0.0006) [2023-12-27 01:05:16,817][105620] Updated weights for policy 1, policy_version 1333557 (0.0006) [2023-12-27 01:05:16,874][105620] Updated weights for policy 1, policy_version 1333567 (0.0006) [2023-12-27 01:05:16,967][105692] Updated weights for policy 0, policy_version 1331608 (0.0009) [2023-12-27 01:05:17,023][105692] Updated weights for policy 0, policy_version 1331618 (0.0010) [2023-12-27 01:05:17,067][105692] Updated weights for policy 0, policy_version 1331628 (0.0010) [2023-12-27 01:05:17,478][105620] Updated weights for policy 1, policy_version 1333577 (0.0006) [2023-12-27 01:05:17,526][105620] Updated weights for policy 1, policy_version 1333587 (0.0010) [2023-12-27 01:05:17,571][105620] Updated weights for policy 1, policy_version 1333597 (0.0010) [2023-12-27 01:05:17,771][105692] Updated weights for policy 0, policy_version 1331638 (0.0008) [2023-12-27 01:05:17,817][105692] Updated weights for policy 0, policy_version 1331648 (0.0008) [2023-12-27 01:05:17,866][105692] Updated weights for policy 0, policy_version 1331658 (0.0008) [2023-12-27 01:05:18,285][105620] Updated weights for policy 1, policy_version 1333607 (0.0007) [2023-12-27 01:05:18,351][105620] Updated weights for policy 1, policy_version 1333617 (0.0007) [2023-12-27 01:05:18,414][105620] Updated weights for policy 1, policy_version 1333627 (0.0007) [2023-12-27 01:05:18,576][105692] Updated weights for policy 0, policy_version 1331668 (0.0008) [2023-12-27 01:05:18,630][105692] Updated weights for policy 0, policy_version 1331678 (0.0010) [2023-12-27 01:05:18,691][105692] Updated weights for policy 0, policy_version 1331689 (0.0010) [2023-12-27 01:05:18,984][105620] Updated weights for policy 1, policy_version 1333637 (0.0008) [2023-12-27 01:05:19,042][105620] Updated weights for policy 1, policy_version 1333647 (0.0005) [2023-12-27 01:05:19,097][105620] Updated weights for policy 1, policy_version 1333657 (0.0005) [2023-12-27 01:05:19,427][105692] Updated weights for policy 0, policy_version 1331699 (0.0009) [2023-12-27 01:05:19,493][105692] Updated weights for policy 0, policy_version 1331709 (0.0008) [2023-12-27 01:05:19,558][105692] Updated weights for policy 0, policy_version 1331719 (0.0009) [2023-12-27 01:05:19,854][105620] Updated weights for policy 1, policy_version 1333667 (0.0006) [2023-12-27 01:05:19,912][105620] Updated weights for policy 1, policy_version 1333677 (0.0008) [2023-12-27 01:05:19,972][105620] Updated weights for policy 1, policy_version 1333687 (0.0008) [2023-12-27 01:05:20,345][105692] Updated weights for policy 0, policy_version 1331729 (0.0009) [2023-12-27 01:05:20,404][105692] Updated weights for policy 0, policy_version 1331739 (0.0011) [2023-12-27 01:05:20,454][105692] Updated weights for policy 0, policy_version 1331749 (0.0011) [2023-12-27 01:05:20,517][105692] Updated weights for policy 0, policy_version 1331759 (0.0011) [2023-12-27 01:05:20,782][105620] Updated weights for policy 1, policy_version 1333697 (0.0008) [2023-12-27 01:05:20,843][105620] Updated weights for policy 1, policy_version 1333707 (0.0008) [2023-12-27 01:05:20,903][105620] Updated weights for policy 1, policy_version 1333717 (0.0008) [2023-12-27 01:05:20,964][105620] Updated weights for policy 1, policy_version 1333727 (0.0008) [2023-12-27 01:05:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 682459136. Throughput: 0: 9674.2, 1: 9835.7. Samples: 682448480. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:05:21,062][104569] Avg episode reward: [(0, '8720.826'), (1, '9356.895')] [2023-12-27 01:05:21,291][105692] Updated weights for policy 0, policy_version 1331769 (0.0011) [2023-12-27 01:05:21,359][105692] Updated weights for policy 0, policy_version 1331779 (0.0010) [2023-12-27 01:05:21,424][105692] Updated weights for policy 0, policy_version 1331789 (0.0011) [2023-12-27 01:05:21,743][105620] Updated weights for policy 1, policy_version 1333737 (0.0008) [2023-12-27 01:05:21,810][105620] Updated weights for policy 1, policy_version 1333747 (0.0008) [2023-12-27 01:05:21,880][105620] Updated weights for policy 1, policy_version 1333757 (0.0008) [2023-12-27 01:05:22,214][105692] Updated weights for policy 0, policy_version 1331799 (0.0011) [2023-12-27 01:05:22,281][105692] Updated weights for policy 0, policy_version 1331809 (0.0011) [2023-12-27 01:05:22,337][105692] Updated weights for policy 0, policy_version 1331819 (0.0011) [2023-12-27 01:05:22,545][105620] Updated weights for policy 1, policy_version 1333767 (0.0008) [2023-12-27 01:05:22,605][105620] Updated weights for policy 1, policy_version 1333777 (0.0008) [2023-12-27 01:05:22,665][105620] Updated weights for policy 1, policy_version 1333787 (0.0008) [2023-12-27 01:05:23,113][105692] Updated weights for policy 0, policy_version 1331829 (0.0010) [2023-12-27 01:05:23,167][105692] Updated weights for policy 0, policy_version 1331839 (0.0010) [2023-12-27 01:05:23,215][105692] Updated weights for policy 0, policy_version 1331849 (0.0009) [2023-12-27 01:05:23,353][105620] Updated weights for policy 1, policy_version 1333797 (0.0006) [2023-12-27 01:05:23,404][105620] Updated weights for policy 1, policy_version 1333807 (0.0005) [2023-12-27 01:05:23,447][105620] Updated weights for policy 1, policy_version 1333817 (0.0005) [2023-12-27 01:05:23,994][105692] Updated weights for policy 0, policy_version 1331859 (0.0009) [2023-12-27 01:05:24,041][105692] Updated weights for policy 0, policy_version 1331869 (0.0009) [2023-12-27 01:05:24,075][105620] Updated weights for policy 1, policy_version 1333827 (0.0007) [2023-12-27 01:05:24,089][105692] Updated weights for policy 0, policy_version 1331879 (0.0007) [2023-12-27 01:05:24,127][105620] Updated weights for policy 1, policy_version 1333837 (0.0010) [2023-12-27 01:05:24,187][105620] Updated weights for policy 1, policy_version 1333847 (0.0008) [2023-12-27 01:05:24,800][105692] Updated weights for policy 0, policy_version 1331889 (0.0006) [2023-12-27 01:05:24,807][105620] Updated weights for policy 1, policy_version 1333857 (0.0010) [2023-12-27 01:05:24,854][105692] Updated weights for policy 0, policy_version 1331899 (0.0005) [2023-12-27 01:05:24,866][105620] Updated weights for policy 1, policy_version 1333867 (0.0010) [2023-12-27 01:05:24,881][105585] KL-divergence is very high: 119.5116 [2023-12-27 01:05:24,907][105692] Updated weights for policy 0, policy_version 1331909 (0.0005) [2023-12-27 01:05:24,922][105585] KL-divergence is very high: 235.3808 [2023-12-27 01:05:24,925][105620] Updated weights for policy 1, policy_version 1333877 (0.0010) [2023-12-27 01:05:24,956][105692] Updated weights for policy 0, policy_version 1331919 (0.0005) [2023-12-27 01:05:24,977][105620] Updated weights for policy 1, policy_version 1333887 (0.0010) [2023-12-27 01:05:25,638][105620] Updated weights for policy 1, policy_version 1333897 (0.0011) [2023-12-27 01:05:25,697][105620] Updated weights for policy 1, policy_version 1333907 (0.0011) [2023-12-27 01:05:25,708][105692] Updated weights for policy 0, policy_version 1331929 (0.0006) [2023-12-27 01:05:25,758][105620] Updated weights for policy 1, policy_version 1333917 (0.0011) [2023-12-27 01:05:25,767][105692] Updated weights for policy 0, policy_version 1331939 (0.0006) [2023-12-27 01:05:25,819][105692] Updated weights for policy 0, policy_version 1331949 (0.0008) [2023-12-27 01:05:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 682557440. Throughput: 0: 9684.7, 1: 9877.0. Samples: 682563068. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:05:26,062][104569] Avg episode reward: [(0, '8626.483'), (1, '9268.245')] [2023-12-27 01:05:26,471][105620] Updated weights for policy 1, policy_version 1333927 (0.0010) [2023-12-27 01:05:26,486][105692] Updated weights for policy 0, policy_version 1331959 (0.0007) [2023-12-27 01:05:26,520][105620] Updated weights for policy 1, policy_version 1333937 (0.0009) [2023-12-27 01:05:26,538][105692] Updated weights for policy 0, policy_version 1331969 (0.0005) [2023-12-27 01:05:26,569][105620] Updated weights for policy 1, policy_version 1333947 (0.0010) [2023-12-27 01:05:26,595][105692] Updated weights for policy 0, policy_version 1331979 (0.0005) [2023-12-27 01:05:27,230][105692] Updated weights for policy 0, policy_version 1331989 (0.0005) [2023-12-27 01:05:27,285][105692] Updated weights for policy 0, policy_version 1331999 (0.0005) [2023-12-27 01:05:27,322][105620] Updated weights for policy 1, policy_version 1333957 (0.0010) [2023-12-27 01:05:27,343][105692] Updated weights for policy 0, policy_version 1332009 (0.0006) [2023-12-27 01:05:27,380][105620] Updated weights for policy 1, policy_version 1333967 (0.0010) [2023-12-27 01:05:27,434][105620] Updated weights for policy 1, policy_version 1333977 (0.0010) [2023-12-27 01:05:27,988][105692] Updated weights for policy 0, policy_version 1332019 (0.0006) [2023-12-27 01:05:28,036][105692] Updated weights for policy 0, policy_version 1332029 (0.0008) [2023-12-27 01:05:28,080][105692] Updated weights for policy 0, policy_version 1332039 (0.0008) [2023-12-27 01:05:28,167][105620] Updated weights for policy 1, policy_version 1333987 (0.0010) [2023-12-27 01:05:28,231][105620] Updated weights for policy 1, policy_version 1333997 (0.0010) [2023-12-27 01:05:28,295][105620] Updated weights for policy 1, policy_version 1334007 (0.0010) [2023-12-27 01:05:28,912][105692] Updated weights for policy 0, policy_version 1332049 (0.0008) [2023-12-27 01:05:28,914][105620] Updated weights for policy 1, policy_version 1334017 (0.0010) [2023-12-27 01:05:28,965][105692] Updated weights for policy 0, policy_version 1332059 (0.0006) [2023-12-27 01:05:28,968][105620] Updated weights for policy 1, policy_version 1334027 (0.0010) [2023-12-27 01:05:29,015][105620] Updated weights for policy 1, policy_version 1334037 (0.0010) [2023-12-27 01:05:29,029][105692] Updated weights for policy 0, policy_version 1332069 (0.0006) [2023-12-27 01:05:29,063][105620] Updated weights for policy 1, policy_version 1334047 (0.0010) [2023-12-27 01:05:29,096][105692] Updated weights for policy 0, policy_version 1332079 (0.0006) [2023-12-27 01:05:29,770][105692] Updated weights for policy 0, policy_version 1332089 (0.0009) [2023-12-27 01:05:29,827][105692] Updated weights for policy 0, policy_version 1332099 (0.0009) [2023-12-27 01:05:29,862][105620] Updated weights for policy 1, policy_version 1334057 (0.0008) [2023-12-27 01:05:29,891][105692] Updated weights for policy 0, policy_version 1332109 (0.0009) [2023-12-27 01:05:29,914][105620] Updated weights for policy 1, policy_version 1334067 (0.0007) [2023-12-27 01:05:29,973][105620] Updated weights for policy 1, policy_version 1334077 (0.0009) [2023-12-27 01:05:30,614][105692] Updated weights for policy 0, policy_version 1332119 (0.0008) [2023-12-27 01:05:30,671][105692] Updated weights for policy 0, policy_version 1332129 (0.0006) [2023-12-27 01:05:30,738][105692] Updated weights for policy 0, policy_version 1332139 (0.0005) [2023-12-27 01:05:30,767][105620] Updated weights for policy 1, policy_version 1334087 (0.0009) [2023-12-27 01:05:30,833][105620] Updated weights for policy 1, policy_version 1334098 (0.0010) [2023-12-27 01:05:30,903][105620] Updated weights for policy 1, policy_version 1334108 (0.0010) [2023-12-27 01:05:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 682655744. Throughput: 0: 9720.9, 1: 9950.3. Samples: 682623760. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:05:31,062][104569] Avg episode reward: [(0, '8536.753'), (1, '9268.139')] [2023-12-27 01:05:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001332144_341082112.pth... [2023-12-27 01:05:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001334112_341573632.pth... [2023-12-27 01:05:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001331024_340795392.pth [2023-12-27 01:05:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001332960_341278720.pth [2023-12-27 01:05:31,356][105692] Updated weights for policy 0, policy_version 1332149 (0.0007) [2023-12-27 01:05:31,425][105692] Updated weights for policy 0, policy_version 1332159 (0.0008) [2023-12-27 01:05:31,433][105585] KL-divergence is very high: 124.2266 [2023-12-27 01:05:31,474][105692] Updated weights for policy 0, policy_version 1332169 (0.0008) [2023-12-27 01:05:31,474][105585] KL-divergence is very high: 223.1005 [2023-12-27 01:05:31,728][105620] Updated weights for policy 1, policy_version 1334118 (0.0009) [2023-12-27 01:05:31,791][105620] Updated weights for policy 1, policy_version 1334128 (0.0009) [2023-12-27 01:05:31,849][105620] Updated weights for policy 1, policy_version 1334138 (0.0009) [2023-12-27 01:05:32,194][105692] Updated weights for policy 0, policy_version 1332179 (0.0009) [2023-12-27 01:05:32,250][105692] Updated weights for policy 0, policy_version 1332189 (0.0009) [2023-12-27 01:05:32,302][105692] Updated weights for policy 0, policy_version 1332199 (0.0009) [2023-12-27 01:05:32,583][105620] Updated weights for policy 1, policy_version 1334148 (0.0009) [2023-12-27 01:05:32,638][105620] Updated weights for policy 1, policy_version 1334158 (0.0007) [2023-12-27 01:05:32,688][105620] Updated weights for policy 1, policy_version 1334168 (0.0008) [2023-12-27 01:05:32,979][105692] Updated weights for policy 0, policy_version 1332209 (0.0009) [2023-12-27 01:05:33,032][105692] Updated weights for policy 0, policy_version 1332219 (0.0005) [2023-12-27 01:05:33,091][105692] Updated weights for policy 0, policy_version 1332229 (0.0008) [2023-12-27 01:05:33,148][105692] Updated weights for policy 0, policy_version 1332239 (0.0006) [2023-12-27 01:05:33,505][105620] Updated weights for policy 1, policy_version 1334179 (0.0010) [2023-12-27 01:05:33,555][105620] Updated weights for policy 1, policy_version 1334189 (0.0009) [2023-12-27 01:05:33,612][105620] Updated weights for policy 1, policy_version 1334199 (0.0009) [2023-12-27 01:05:33,800][105692] Updated weights for policy 0, policy_version 1332249 (0.0006) [2023-12-27 01:05:33,857][105692] Updated weights for policy 0, policy_version 1332259 (0.0005) [2023-12-27 01:05:33,913][105692] Updated weights for policy 0, policy_version 1332269 (0.0005) [2023-12-27 01:05:34,416][105620] Updated weights for policy 1, policy_version 1334209 (0.0009) [2023-12-27 01:05:34,488][105620] Updated weights for policy 1, policy_version 1334219 (0.0008) [2023-12-27 01:05:34,555][105620] Updated weights for policy 1, policy_version 1334229 (0.0009) [2023-12-27 01:05:34,565][105692] Updated weights for policy 0, policy_version 1332279 (0.0007) [2023-12-27 01:05:34,618][105692] Updated weights for policy 0, policy_version 1332289 (0.0006) [2023-12-27 01:05:34,619][105620] Updated weights for policy 1, policy_version 1334239 (0.0009) [2023-12-27 01:05:34,672][105692] Updated weights for policy 0, policy_version 1332299 (0.0008) [2023-12-27 01:05:35,334][105620] Updated weights for policy 1, policy_version 1334249 (0.0009) [2023-12-27 01:05:35,387][105620] Updated weights for policy 1, policy_version 1334259 (0.0008) [2023-12-27 01:05:35,425][105692] Updated weights for policy 0, policy_version 1332309 (0.0010) [2023-12-27 01:05:35,439][105620] Updated weights for policy 1, policy_version 1334269 (0.0008) [2023-12-27 01:05:35,476][105692] Updated weights for policy 0, policy_version 1332319 (0.0007) [2023-12-27 01:05:35,526][105692] Updated weights for policy 0, policy_version 1332329 (0.0008) [2023-12-27 01:05:36,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 682745856. Throughput: 0: 9701.8, 1: 9859.5. Samples: 682738172. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:05:36,063][104569] Avg episode reward: [(0, '8807.965'), (1, '9268.105')] [2023-12-27 01:05:36,231][105620] Updated weights for policy 1, policy_version 1334279 (0.0007) [2023-12-27 01:05:36,273][105692] Updated weights for policy 0, policy_version 1332339 (0.0006) [2023-12-27 01:05:36,297][105620] Updated weights for policy 1, policy_version 1334289 (0.0006) [2023-12-27 01:05:36,336][105692] Updated weights for policy 0, policy_version 1332349 (0.0008) [2023-12-27 01:05:36,362][105620] Updated weights for policy 1, policy_version 1334299 (0.0005) [2023-12-27 01:05:36,391][105692] Updated weights for policy 0, policy_version 1332359 (0.0008) [2023-12-27 01:05:37,079][105620] Updated weights for policy 1, policy_version 1334309 (0.0007) [2023-12-27 01:05:37,082][105692] Updated weights for policy 0, policy_version 1332369 (0.0008) [2023-12-27 01:05:37,135][105692] Updated weights for policy 0, policy_version 1332379 (0.0006) [2023-12-27 01:05:37,137][105620] Updated weights for policy 1, policy_version 1334319 (0.0008) [2023-12-27 01:05:37,188][105692] Updated weights for policy 0, policy_version 1332389 (0.0007) [2023-12-27 01:05:37,197][105620] Updated weights for policy 1, policy_version 1334329 (0.0006) [2023-12-27 01:05:37,248][105692] Updated weights for policy 0, policy_version 1332399 (0.0008) [2023-12-27 01:05:37,957][105620] Updated weights for policy 1, policy_version 1334339 (0.0006) [2023-12-27 01:05:38,014][105620] Updated weights for policy 1, policy_version 1334349 (0.0007) [2023-12-27 01:05:38,016][105692] Updated weights for policy 0, policy_version 1332409 (0.0006) [2023-12-27 01:05:38,069][105620] Updated weights for policy 1, policy_version 1334359 (0.0009) [2023-12-27 01:05:38,075][105692] Updated weights for policy 0, policy_version 1332419 (0.0005) [2023-12-27 01:05:38,135][105692] Updated weights for policy 0, policy_version 1332429 (0.0007) [2023-12-27 01:05:38,859][105620] Updated weights for policy 1, policy_version 1334369 (0.0008) [2023-12-27 01:05:38,864][105692] Updated weights for policy 0, policy_version 1332439 (0.0008) [2023-12-27 01:05:38,918][105692] Updated weights for policy 0, policy_version 1332449 (0.0008) [2023-12-27 01:05:38,920][105620] Updated weights for policy 1, policy_version 1334379 (0.0011) [2023-12-27 01:05:38,975][105692] Updated weights for policy 0, policy_version 1332459 (0.0006) [2023-12-27 01:05:38,980][105620] Updated weights for policy 1, policy_version 1334389 (0.0010) [2023-12-27 01:05:39,040][105620] Updated weights for policy 1, policy_version 1334399 (0.0011) [2023-12-27 01:05:39,711][105620] Updated weights for policy 1, policy_version 1334409 (0.0008) [2023-12-27 01:05:39,765][105620] Updated weights for policy 1, policy_version 1334419 (0.0011) [2023-12-27 01:05:39,767][105692] Updated weights for policy 0, policy_version 1332469 (0.0007) [2023-12-27 01:05:39,819][105620] Updated weights for policy 1, policy_version 1334429 (0.0011) [2023-12-27 01:05:39,833][105692] Updated weights for policy 0, policy_version 1332479 (0.0007) [2023-12-27 01:05:39,897][105692] Updated weights for policy 0, policy_version 1332489 (0.0010) [2023-12-27 01:05:40,419][105620] Updated weights for policy 1, policy_version 1334439 (0.0009) [2023-12-27 01:05:40,483][105620] Updated weights for policy 1, policy_version 1334449 (0.0011) [2023-12-27 01:05:40,547][105620] Updated weights for policy 1, policy_version 1334459 (0.0011) [2023-12-27 01:05:40,743][105692] Updated weights for policy 0, policy_version 1332499 (0.0009) [2023-12-27 01:05:40,812][105692] Updated weights for policy 0, policy_version 1332509 (0.0009) [2023-12-27 01:05:40,861][105692] Updated weights for policy 0, policy_version 1332519 (0.0010) [2023-12-27 01:05:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 682844160. Throughput: 0: 9627.7, 1: 9844.3. Samples: 682851044. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:05:41,062][104569] Avg episode reward: [(0, '8902.688'), (1, '9266.294')] [2023-12-27 01:05:41,297][105620] Updated weights for policy 1, policy_version 1334469 (0.0008) [2023-12-27 01:05:41,365][105620] Updated weights for policy 1, policy_version 1334479 (0.0007) [2023-12-27 01:05:41,425][105620] Updated weights for policy 1, policy_version 1334489 (0.0008) [2023-12-27 01:05:41,662][105692] Updated weights for policy 0, policy_version 1332529 (0.0010) [2023-12-27 01:05:41,739][105692] Updated weights for policy 0, policy_version 1332539 (0.0007) [2023-12-27 01:05:41,805][105692] Updated weights for policy 0, policy_version 1332549 (0.0006) [2023-12-27 01:05:41,877][105692] Updated weights for policy 0, policy_version 1332559 (0.0006) [2023-12-27 01:05:42,082][105620] Updated weights for policy 1, policy_version 1334499 (0.0006) [2023-12-27 01:05:42,138][105620] Updated weights for policy 1, policy_version 1334509 (0.0009) [2023-12-27 01:05:42,193][105620] Updated weights for policy 1, policy_version 1334519 (0.0009) [2023-12-27 01:05:42,544][105692] Updated weights for policy 0, policy_version 1332569 (0.0010) [2023-12-27 01:05:42,607][105692] Updated weights for policy 0, policy_version 1332579 (0.0011) [2023-12-27 01:05:42,667][105692] Updated weights for policy 0, policy_version 1332589 (0.0011) [2023-12-27 01:05:42,902][105620] Updated weights for policy 1, policy_version 1334529 (0.0008) [2023-12-27 01:05:42,954][105620] Updated weights for policy 1, policy_version 1334539 (0.0005) [2023-12-27 01:05:43,021][105620] Updated weights for policy 1, policy_version 1334549 (0.0006) [2023-12-27 01:05:43,082][105620] Updated weights for policy 1, policy_version 1334559 (0.0008) [2023-12-27 01:05:43,393][105692] Updated weights for policy 0, policy_version 1332599 (0.0011) [2023-12-27 01:05:43,446][105692] Updated weights for policy 0, policy_version 1332609 (0.0010) [2023-12-27 01:05:43,498][105692] Updated weights for policy 0, policy_version 1332619 (0.0009) [2023-12-27 01:05:43,711][105620] Updated weights for policy 1, policy_version 1334569 (0.0010) [2023-12-27 01:05:43,762][105620] Updated weights for policy 1, policy_version 1334579 (0.0008) [2023-12-27 01:05:43,819][105620] Updated weights for policy 1, policy_version 1334589 (0.0010) [2023-12-27 01:05:44,206][105692] Updated weights for policy 0, policy_version 1332629 (0.0009) [2023-12-27 01:05:44,264][105692] Updated weights for policy 0, policy_version 1332639 (0.0009) [2023-12-27 01:05:44,314][105692] Updated weights for policy 0, policy_version 1332649 (0.0008) [2023-12-27 01:05:44,503][105620] Updated weights for policy 1, policy_version 1334599 (0.0008) [2023-12-27 01:05:44,549][105620] Updated weights for policy 1, policy_version 1334609 (0.0009) [2023-12-27 01:05:44,610][105620] Updated weights for policy 1, policy_version 1334619 (0.0008) [2023-12-27 01:05:45,087][105692] Updated weights for policy 0, policy_version 1332659 (0.0009) [2023-12-27 01:05:45,141][105692] Updated weights for policy 0, policy_version 1332669 (0.0009) [2023-12-27 01:05:45,196][105692] Updated weights for policy 0, policy_version 1332679 (0.0009) [2023-12-27 01:05:45,403][105620] Updated weights for policy 1, policy_version 1334629 (0.0009) [2023-12-27 01:05:45,466][105620] Updated weights for policy 1, policy_version 1334639 (0.0009) [2023-12-27 01:05:45,525][105620] Updated weights for policy 1, policy_version 1334649 (0.0009) [2023-12-27 01:05:45,943][105692] Updated weights for policy 0, policy_version 1332689 (0.0009) [2023-12-27 01:05:45,990][105692] Updated weights for policy 0, policy_version 1332699 (0.0008) [2023-12-27 01:05:46,052][105692] Updated weights for policy 0, policy_version 1332709 (0.0008) [2023-12-27 01:05:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 682934272. Throughput: 0: 9587.0, 1: 9809.6. Samples: 682908396. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:05:46,063][104569] Avg episode reward: [(0, '8994.231'), (1, '9178.187')] [2023-12-27 01:05:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001334656_341712896.pth... [2023-12-27 01:05:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001333536_341426176.pth [2023-12-27 01:05:46,104][105692] Updated weights for policy 0, policy_version 1332719 (0.0008) [2023-12-27 01:05:46,107][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001332720_341229568.pth... [2023-12-27 01:05:46,110][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001331568_340934656.pth [2023-12-27 01:05:46,291][105620] Updated weights for policy 1, policy_version 1334659 (0.0010) [2023-12-27 01:05:46,342][105620] Updated weights for policy 1, policy_version 1334669 (0.0010) [2023-12-27 01:05:46,403][105620] Updated weights for policy 1, policy_version 1334679 (0.0005) [2023-12-27 01:05:46,795][105692] Updated weights for policy 0, policy_version 1332729 (0.0005) [2023-12-27 01:05:46,852][105692] Updated weights for policy 0, policy_version 1332739 (0.0009) [2023-12-27 01:05:46,897][105692] Updated weights for policy 0, policy_version 1332749 (0.0010) [2023-12-27 01:05:47,155][105620] Updated weights for policy 1, policy_version 1334689 (0.0006) [2023-12-27 01:05:47,213][105620] Updated weights for policy 1, policy_version 1334699 (0.0008) [2023-12-27 01:05:47,280][105620] Updated weights for policy 1, policy_version 1334709 (0.0007) [2023-12-27 01:05:47,330][105620] Updated weights for policy 1, policy_version 1334719 (0.0009) [2023-12-27 01:05:47,498][105692] Updated weights for policy 0, policy_version 1332759 (0.0007) [2023-12-27 01:05:47,551][105692] Updated weights for policy 0, policy_version 1332769 (0.0006) [2023-12-27 01:05:47,615][105692] Updated weights for policy 0, policy_version 1332779 (0.0005) [2023-12-27 01:05:48,082][105620] Updated weights for policy 1, policy_version 1334729 (0.0005) [2023-12-27 01:05:48,143][105620] Updated weights for policy 1, policy_version 1334739 (0.0008) [2023-12-27 01:05:48,201][105620] Updated weights for policy 1, policy_version 1334750 (0.0010) [2023-12-27 01:05:48,248][105692] Updated weights for policy 0, policy_version 1332789 (0.0005) [2023-12-27 01:05:48,301][105692] Updated weights for policy 0, policy_version 1332799 (0.0005) [2023-12-27 01:05:48,361][105692] Updated weights for policy 0, policy_version 1332809 (0.0009) [2023-12-27 01:05:48,860][105620] Updated weights for policy 1, policy_version 1334760 (0.0008) [2023-12-27 01:05:48,928][105620] Updated weights for policy 1, policy_version 1334770 (0.0009) [2023-12-27 01:05:48,997][105620] Updated weights for policy 1, policy_version 1334780 (0.0008) [2023-12-27 01:05:49,017][105692] Updated weights for policy 0, policy_version 1332819 (0.0008) [2023-12-27 01:05:49,080][105692] Updated weights for policy 0, policy_version 1332829 (0.0010) [2023-12-27 01:05:49,142][105692] Updated weights for policy 0, policy_version 1332839 (0.0009) [2023-12-27 01:05:49,680][105620] Updated weights for policy 1, policy_version 1334790 (0.0009) [2023-12-27 01:05:49,739][105620] Updated weights for policy 1, policy_version 1334800 (0.0010) [2023-12-27 01:05:49,799][105620] Updated weights for policy 1, policy_version 1334810 (0.0010) [2023-12-27 01:05:49,882][105692] Updated weights for policy 0, policy_version 1332849 (0.0006) [2023-12-27 01:05:49,947][105692] Updated weights for policy 0, policy_version 1332859 (0.0008) [2023-12-27 01:05:50,006][105692] Updated weights for policy 0, policy_version 1332869 (0.0008) [2023-12-27 01:05:50,058][105692] Updated weights for policy 0, policy_version 1332879 (0.0008) [2023-12-27 01:05:50,514][105620] Updated weights for policy 1, policy_version 1334820 (0.0009) [2023-12-27 01:05:50,571][105620] Updated weights for policy 1, policy_version 1334830 (0.0010) [2023-12-27 01:05:50,631][105620] Updated weights for policy 1, policy_version 1334840 (0.0009) [2023-12-27 01:05:50,724][105585] KL-divergence is very high: 101.9770 [2023-12-27 01:05:50,729][105692] Updated weights for policy 0, policy_version 1332889 (0.0008) [2023-12-27 01:05:50,771][105585] KL-divergence is very high: 110.3467 [2023-12-27 01:05:50,791][105692] Updated weights for policy 0, policy_version 1332899 (0.0009) [2023-12-27 01:05:50,825][105585] KL-divergence is very high: 111.6794 [2023-12-27 01:05:50,856][105692] Updated weights for policy 0, policy_version 1332909 (0.0007) [2023-12-27 01:05:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 683040768. Throughput: 0: 9655.2, 1: 9723.8. Samples: 683027044. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:05:51,063][104569] Avg episode reward: [(0, '8644.105'), (1, '8999.752')] [2023-12-27 01:05:51,382][105620] Updated weights for policy 1, policy_version 1334850 (0.0010) [2023-12-27 01:05:51,435][105620] Updated weights for policy 1, policy_version 1334860 (0.0010) [2023-12-27 01:05:51,488][105620] Updated weights for policy 1, policy_version 1334870 (0.0010) [2023-12-27 01:05:51,544][105620] Updated weights for policy 1, policy_version 1334880 (0.0010) [2023-12-27 01:05:51,554][105692] Updated weights for policy 0, policy_version 1332919 (0.0006) [2023-12-27 01:05:51,615][105692] Updated weights for policy 0, policy_version 1332929 (0.0008) [2023-12-27 01:05:51,690][105692] Updated weights for policy 0, policy_version 1332939 (0.0007) [2023-12-27 01:05:52,331][105620] Updated weights for policy 1, policy_version 1334890 (0.0010) [2023-12-27 01:05:52,396][105620] Updated weights for policy 1, policy_version 1334900 (0.0009) [2023-12-27 01:05:52,413][105692] Updated weights for policy 0, policy_version 1332949 (0.0008) [2023-12-27 01:05:52,455][105620] Updated weights for policy 1, policy_version 1334910 (0.0006) [2023-12-27 01:05:52,476][105692] Updated weights for policy 0, policy_version 1332959 (0.0008) [2023-12-27 01:05:52,536][105692] Updated weights for policy 0, policy_version 1332969 (0.0007) [2023-12-27 01:05:53,188][105620] Updated weights for policy 1, policy_version 1334920 (0.0009) [2023-12-27 01:05:53,243][105620] Updated weights for policy 1, policy_version 1334930 (0.0009) [2023-12-27 01:05:53,248][105692] Updated weights for policy 0, policy_version 1332979 (0.0008) [2023-12-27 01:05:53,291][105585] KL-divergence is very high: 109.7307 [2023-12-27 01:05:53,295][105620] Updated weights for policy 1, policy_version 1334940 (0.0009) [2023-12-27 01:05:53,314][105692] Updated weights for policy 0, policy_version 1332989 (0.0006) [2023-12-27 01:05:53,337][105585] KL-divergence is very high: 184.5953 [2023-12-27 01:05:53,368][105692] Updated weights for policy 0, policy_version 1333001 (0.0010) [2023-12-27 01:05:53,372][105585] KL-divergence is very high: 200.6483 [2023-12-27 01:05:53,901][105620] Updated weights for policy 1, policy_version 1334950 (0.0008) [2023-12-27 01:05:53,954][105620] Updated weights for policy 1, policy_version 1334960 (0.0010) [2023-12-27 01:05:53,995][105692] Updated weights for policy 0, policy_version 1333012 (0.0008) [2023-12-27 01:05:54,010][105620] Updated weights for policy 1, policy_version 1334970 (0.0010) [2023-12-27 01:05:54,060][105692] Updated weights for policy 0, policy_version 1333022 (0.0008) [2023-12-27 01:05:54,108][105692] Updated weights for policy 0, policy_version 1333032 (0.0006) [2023-12-27 01:05:54,642][105620] Updated weights for policy 1, policy_version 1334980 (0.0006) [2023-12-27 01:05:54,687][105620] Updated weights for policy 1, policy_version 1334990 (0.0008) [2023-12-27 01:05:54,735][105620] Updated weights for policy 1, policy_version 1335000 (0.0010) [2023-12-27 01:05:54,757][105692] Updated weights for policy 0, policy_version 1333042 (0.0005) [2023-12-27 01:05:54,815][105692] Updated weights for policy 0, policy_version 1333052 (0.0007) [2023-12-27 01:05:54,863][105692] Updated weights for policy 0, policy_version 1333062 (0.0008) [2023-12-27 01:05:54,921][105692] Updated weights for policy 0, policy_version 1333072 (0.0008) [2023-12-27 01:05:55,444][105620] Updated weights for policy 1, policy_version 1335010 (0.0009) [2023-12-27 01:05:55,500][105620] Updated weights for policy 1, policy_version 1335020 (0.0009) [2023-12-27 01:05:55,558][105620] Updated weights for policy 1, policy_version 1335030 (0.0010) [2023-12-27 01:05:55,621][105620] Updated weights for policy 1, policy_version 1335040 (0.0010) [2023-12-27 01:05:55,631][105692] Updated weights for policy 0, policy_version 1333082 (0.0008) [2023-12-27 01:05:55,688][105692] Updated weights for policy 0, policy_version 1333092 (0.0008) [2023-12-27 01:05:55,749][105692] Updated weights for policy 0, policy_version 1333102 (0.0007) [2023-12-27 01:05:56,062][104569] Fps is (10 sec: 20480.6, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 683139072. Throughput: 0: 9771.0, 1: 9699.6. Samples: 683146448. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:05:56,062][104569] Avg episode reward: [(0, '8552.722'), (1, '8996.253')] [2023-12-27 01:05:56,383][105620] Updated weights for policy 1, policy_version 1335050 (0.0010) [2023-12-27 01:05:56,446][105620] Updated weights for policy 1, policy_version 1335060 (0.0010) [2023-12-27 01:05:56,508][105620] Updated weights for policy 1, policy_version 1335070 (0.0010) [2023-12-27 01:05:56,510][105692] Updated weights for policy 0, policy_version 1333112 (0.0006) [2023-12-27 01:05:56,565][105692] Updated weights for policy 0, policy_version 1333122 (0.0008) [2023-12-27 01:05:56,621][105692] Updated weights for policy 0, policy_version 1333132 (0.0008) [2023-12-27 01:05:57,231][105620] Updated weights for policy 1, policy_version 1335080 (0.0011) [2023-12-27 01:05:57,299][105620] Updated weights for policy 1, policy_version 1335090 (0.0010) [2023-12-27 01:05:57,328][105692] Updated weights for policy 0, policy_version 1333142 (0.0007) [2023-12-27 01:05:57,360][105620] Updated weights for policy 1, policy_version 1335100 (0.0010) [2023-12-27 01:05:57,376][105692] Updated weights for policy 0, policy_version 1333152 (0.0010) [2023-12-27 01:05:57,424][105692] Updated weights for policy 0, policy_version 1333162 (0.0006) [2023-12-27 01:05:57,970][105620] Updated weights for policy 1, policy_version 1335110 (0.0007) [2023-12-27 01:05:58,014][105692] Updated weights for policy 0, policy_version 1333172 (0.0007) [2023-12-27 01:05:58,026][105620] Updated weights for policy 1, policy_version 1335120 (0.0010) [2023-12-27 01:05:58,075][105692] Updated weights for policy 0, policy_version 1333182 (0.0008) [2023-12-27 01:05:58,081][105620] Updated weights for policy 1, policy_version 1335130 (0.0010) [2023-12-27 01:05:58,138][105692] Updated weights for policy 0, policy_version 1333192 (0.0007) [2023-12-27 01:05:58,861][105620] Updated weights for policy 1, policy_version 1335140 (0.0009) [2023-12-27 01:05:58,881][105692] Updated weights for policy 0, policy_version 1333202 (0.0011) [2023-12-27 01:05:58,934][105620] Updated weights for policy 1, policy_version 1335150 (0.0008) [2023-12-27 01:05:58,951][105692] Updated weights for policy 0, policy_version 1333212 (0.0011) [2023-12-27 01:05:58,985][105620] Updated weights for policy 1, policy_version 1335160 (0.0006) [2023-12-27 01:05:59,013][105692] Updated weights for policy 0, policy_version 1333222 (0.0010) [2023-12-27 01:05:59,068][105692] Updated weights for policy 0, policy_version 1333232 (0.0010) [2023-12-27 01:05:59,781][105620] Updated weights for policy 1, policy_version 1335170 (0.0008) [2023-12-27 01:05:59,852][105620] Updated weights for policy 1, policy_version 1335180 (0.0007) [2023-12-27 01:05:59,862][105692] Updated weights for policy 0, policy_version 1333242 (0.0008) [2023-12-27 01:05:59,909][105620] Updated weights for policy 1, policy_version 1335190 (0.0007) [2023-12-27 01:05:59,916][105692] Updated weights for policy 0, policy_version 1333252 (0.0007) [2023-12-27 01:05:59,977][105620] Updated weights for policy 1, policy_version 1335200 (0.0006) [2023-12-27 01:05:59,977][105692] Updated weights for policy 0, policy_version 1333262 (0.0010) [2023-12-27 01:06:00,642][105620] Updated weights for policy 1, policy_version 1335210 (0.0008) [2023-12-27 01:06:00,698][105620] Updated weights for policy 1, policy_version 1335220 (0.0007) [2023-12-27 01:06:00,700][105692] Updated weights for policy 0, policy_version 1333272 (0.0009) [2023-12-27 01:06:00,754][105620] Updated weights for policy 1, policy_version 1335230 (0.0007) [2023-12-27 01:06:00,756][105692] Updated weights for policy 0, policy_version 1333282 (0.0006) [2023-12-27 01:06:00,815][105692] Updated weights for policy 0, policy_version 1333292 (0.0009) [2023-12-27 01:06:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 683237376. Throughput: 0: 9805.0, 1: 9742.9. Samples: 683205340. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:06:01,062][104569] Avg episode reward: [(0, '8812.053'), (1, '9096.020')] [2023-12-27 01:06:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001333296_341377024.pth... [2023-12-27 01:06:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001335232_341860352.pth... [2023-12-27 01:06:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001332144_341082112.pth [2023-12-27 01:06:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001334112_341573632.pth [2023-12-27 01:06:01,524][105620] Updated weights for policy 1, policy_version 1335240 (0.0009) [2023-12-27 01:06:01,574][105620] Updated weights for policy 1, policy_version 1335250 (0.0007) [2023-12-27 01:06:01,584][105692] Updated weights for policy 0, policy_version 1333302 (0.0009) [2023-12-27 01:06:01,631][105620] Updated weights for policy 1, policy_version 1335260 (0.0006) [2023-12-27 01:06:01,643][105692] Updated weights for policy 0, policy_version 1333312 (0.0008) [2023-12-27 01:06:01,708][105692] Updated weights for policy 0, policy_version 1333322 (0.0007) [2023-12-27 01:06:02,322][105620] Updated weights for policy 1, policy_version 1335270 (0.0008) [2023-12-27 01:06:02,388][105620] Updated weights for policy 1, policy_version 1335280 (0.0009) [2023-12-27 01:06:02,445][105620] Updated weights for policy 1, policy_version 1335290 (0.0009) [2023-12-27 01:06:02,452][105692] Updated weights for policy 0, policy_version 1333332 (0.0007) [2023-12-27 01:06:02,499][105692] Updated weights for policy 0, policy_version 1333342 (0.0008) [2023-12-27 01:06:02,549][105692] Updated weights for policy 0, policy_version 1333352 (0.0009) [2023-12-27 01:06:03,120][105620] Updated weights for policy 1, policy_version 1335300 (0.0009) [2023-12-27 01:06:03,167][105620] Updated weights for policy 1, policy_version 1335310 (0.0008) [2023-12-27 01:06:03,218][105620] Updated weights for policy 1, policy_version 1335320 (0.0008) [2023-12-27 01:06:03,255][105692] Updated weights for policy 0, policy_version 1333362 (0.0007) [2023-12-27 01:06:03,303][105692] Updated weights for policy 0, policy_version 1333372 (0.0010) [2023-12-27 01:06:03,354][105692] Updated weights for policy 0, policy_version 1333382 (0.0010) [2023-12-27 01:06:03,401][105692] Updated weights for policy 0, policy_version 1333392 (0.0010) [2023-12-27 01:06:03,827][105620] Updated weights for policy 1, policy_version 1335330 (0.0008) [2023-12-27 01:06:03,886][105620] Updated weights for policy 1, policy_version 1335340 (0.0008) [2023-12-27 01:06:03,939][105620] Updated weights for policy 1, policy_version 1335350 (0.0007) [2023-12-27 01:06:03,990][105620] Updated weights for policy 1, policy_version 1335360 (0.0005) [2023-12-27 01:06:04,163][105692] Updated weights for policy 0, policy_version 1333402 (0.0011) [2023-12-27 01:06:04,234][105692] Updated weights for policy 0, policy_version 1333412 (0.0011) [2023-12-27 01:06:04,295][105692] Updated weights for policy 0, policy_version 1333422 (0.0011) [2023-12-27 01:06:04,598][105620] Updated weights for policy 1, policy_version 1335370 (0.0008) [2023-12-27 01:06:04,647][105620] Updated weights for policy 1, policy_version 1335380 (0.0008) [2023-12-27 01:06:04,708][105620] Updated weights for policy 1, policy_version 1335390 (0.0008) [2023-12-27 01:06:05,040][105692] Updated weights for policy 0, policy_version 1333432 (0.0011) [2023-12-27 01:06:05,093][105692] Updated weights for policy 0, policy_version 1333442 (0.0008) [2023-12-27 01:06:05,160][105692] Updated weights for policy 0, policy_version 1333452 (0.0006) [2023-12-27 01:06:05,474][105620] Updated weights for policy 1, policy_version 1335400 (0.0007) [2023-12-27 01:06:05,525][105620] Updated weights for policy 1, policy_version 1335411 (0.0010) [2023-12-27 01:06:05,577][105620] Updated weights for policy 1, policy_version 1335422 (0.0009) [2023-12-27 01:06:05,752][105692] Updated weights for policy 0, policy_version 1333462 (0.0005) [2023-12-27 01:06:05,809][105692] Updated weights for policy 0, policy_version 1333472 (0.0008) [2023-12-27 01:06:05,861][105692] Updated weights for policy 0, policy_version 1333482 (0.0010) [2023-12-27 01:06:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 683335680. Throughput: 0: 9706.8, 1: 9705.7. Samples: 683322048. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:06:06,063][104569] Avg episode reward: [(0, '8812.997'), (1, '8525.785')] [2023-12-27 01:06:06,346][105620] Updated weights for policy 1, policy_version 1335432 (0.0010) [2023-12-27 01:06:06,417][105620] Updated weights for policy 1, policy_version 1335442 (0.0009) [2023-12-27 01:06:06,476][105620] Updated weights for policy 1, policy_version 1335452 (0.0009) [2023-12-27 01:06:06,530][105692] Updated weights for policy 0, policy_version 1333492 (0.0010) [2023-12-27 01:06:06,583][105692] Updated weights for policy 0, policy_version 1333502 (0.0006) [2023-12-27 01:06:06,637][105692] Updated weights for policy 0, policy_version 1333512 (0.0005) [2023-12-27 01:06:07,289][105692] Updated weights for policy 0, policy_version 1333522 (0.0008) [2023-12-27 01:06:07,298][105620] Updated weights for policy 1, policy_version 1335462 (0.0009) [2023-12-27 01:06:07,337][105692] Updated weights for policy 0, policy_version 1333532 (0.0007) [2023-12-27 01:06:07,361][105620] Updated weights for policy 1, policy_version 1335472 (0.0009) [2023-12-27 01:06:07,383][105692] Updated weights for policy 0, policy_version 1333542 (0.0007) [2023-12-27 01:06:07,413][105620] Updated weights for policy 1, policy_version 1335482 (0.0006) [2023-12-27 01:06:07,436][105692] Updated weights for policy 0, policy_version 1333552 (0.0010) [2023-12-27 01:06:08,094][105692] Updated weights for policy 0, policy_version 1333562 (0.0005) [2023-12-27 01:06:08,155][105692] Updated weights for policy 0, policy_version 1333572 (0.0006) [2023-12-27 01:06:08,205][105692] Updated weights for policy 0, policy_version 1333582 (0.0009) [2023-12-27 01:06:08,223][105620] Updated weights for policy 1, policy_version 1335492 (0.0006) [2023-12-27 01:06:08,281][105620] Updated weights for policy 1, policy_version 1335502 (0.0008) [2023-12-27 01:06:08,345][105620] Updated weights for policy 1, policy_version 1335512 (0.0007) [2023-12-27 01:06:08,937][105692] Updated weights for policy 0, policy_version 1333592 (0.0010) [2023-12-27 01:06:08,995][105692] Updated weights for policy 0, policy_version 1333602 (0.0010) [2023-12-27 01:06:09,057][105692] Updated weights for policy 0, policy_version 1333612 (0.0010) [2023-12-27 01:06:09,085][105620] Updated weights for policy 1, policy_version 1335522 (0.0009) [2023-12-27 01:06:09,154][105620] Updated weights for policy 1, policy_version 1335532 (0.0010) [2023-12-27 01:06:09,222][105620] Updated weights for policy 1, policy_version 1335542 (0.0010) [2023-12-27 01:06:09,283][105620] Updated weights for policy 1, policy_version 1335552 (0.0010) [2023-12-27 01:06:09,817][105692] Updated weights for policy 0, policy_version 1333622 (0.0010) [2023-12-27 01:06:09,882][105692] Updated weights for policy 0, policy_version 1333632 (0.0009) [2023-12-27 01:06:09,947][105692] Updated weights for policy 0, policy_version 1333642 (0.0011) [2023-12-27 01:06:10,050][105620] Updated weights for policy 1, policy_version 1335562 (0.0008) [2023-12-27 01:06:10,113][105620] Updated weights for policy 1, policy_version 1335572 (0.0008) [2023-12-27 01:06:10,178][105620] Updated weights for policy 1, policy_version 1335582 (0.0008) [2023-12-27 01:06:10,735][105692] Updated weights for policy 0, policy_version 1333652 (0.0010) [2023-12-27 01:06:10,798][105692] Updated weights for policy 0, policy_version 1333662 (0.0009) [2023-12-27 01:06:10,861][105692] Updated weights for policy 0, policy_version 1333672 (0.0006) [2023-12-27 01:06:10,985][105620] Updated weights for policy 1, policy_version 1335592 (0.0008) [2023-12-27 01:06:11,059][105620] Updated weights for policy 1, policy_version 1335602 (0.0008) [2023-12-27 01:06:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 683425792. Throughput: 0: 9822.8, 1: 9588.0. Samples: 683436552. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:06:11,063][104569] Avg episode reward: [(0, '8104.187'), (1, '8303.350')] [2023-12-27 01:06:11,126][105620] Updated weights for policy 1, policy_version 1335612 (0.0010) [2023-12-27 01:06:11,538][105692] Updated weights for policy 0, policy_version 1333682 (0.0007) [2023-12-27 01:06:11,602][105692] Updated weights for policy 0, policy_version 1333693 (0.0011) [2023-12-27 01:06:11,668][105692] Updated weights for policy 0, policy_version 1333703 (0.0009) [2023-12-27 01:06:11,818][105620] Updated weights for policy 1, policy_version 1335622 (0.0008) [2023-12-27 01:06:11,873][105620] Updated weights for policy 1, policy_version 1335632 (0.0011) [2023-12-27 01:06:11,940][105620] Updated weights for policy 1, policy_version 1335642 (0.0011) [2023-12-27 01:06:12,336][105692] Updated weights for policy 0, policy_version 1333713 (0.0006) [2023-12-27 01:06:12,405][105692] Updated weights for policy 0, policy_version 1333723 (0.0008) [2023-12-27 01:06:12,477][105692] Updated weights for policy 0, policy_version 1333733 (0.0010) [2023-12-27 01:06:12,530][105620] Updated weights for policy 1, policy_version 1335652 (0.0008) [2023-12-27 01:06:12,541][105692] Updated weights for policy 0, policy_version 1333743 (0.0009) [2023-12-27 01:06:12,600][105620] Updated weights for policy 1, policy_version 1335662 (0.0006) [2023-12-27 01:06:12,661][105620] Updated weights for policy 1, policy_version 1335672 (0.0006) [2023-12-27 01:06:13,237][105620] Updated weights for policy 1, policy_version 1335682 (0.0009) [2023-12-27 01:06:13,294][105620] Updated weights for policy 1, policy_version 1335692 (0.0006) [2023-12-27 01:06:13,321][105692] Updated weights for policy 0, policy_version 1333753 (0.0007) [2023-12-27 01:06:13,355][105620] Updated weights for policy 1, policy_version 1335702 (0.0006) [2023-12-27 01:06:13,376][105692] Updated weights for policy 0, policy_version 1333763 (0.0007) [2023-12-27 01:06:13,413][105620] Updated weights for policy 1, policy_version 1335712 (0.0006) [2023-12-27 01:06:13,435][105692] Updated weights for policy 0, policy_version 1333773 (0.0008) [2023-12-27 01:06:14,087][105692] Updated weights for policy 0, policy_version 1333783 (0.0008) [2023-12-27 01:06:14,101][105620] Updated weights for policy 1, policy_version 1335722 (0.0007) [2023-12-27 01:06:14,149][105692] Updated weights for policy 0, policy_version 1333793 (0.0006) [2023-12-27 01:06:14,167][105620] Updated weights for policy 1, policy_version 1335732 (0.0008) [2023-12-27 01:06:14,206][105692] Updated weights for policy 0, policy_version 1333803 (0.0006) [2023-12-27 01:06:14,230][105620] Updated weights for policy 1, policy_version 1335742 (0.0008) [2023-12-27 01:06:14,834][105692] Updated weights for policy 0, policy_version 1333813 (0.0007) [2023-12-27 01:06:14,891][105692] Updated weights for policy 0, policy_version 1333823 (0.0009) [2023-12-27 01:06:14,960][105692] Updated weights for policy 0, policy_version 1333833 (0.0009) [2023-12-27 01:06:14,975][105620] Updated weights for policy 1, policy_version 1335752 (0.0008) [2023-12-27 01:06:15,039][105620] Updated weights for policy 1, policy_version 1335762 (0.0006) [2023-12-27 01:06:15,102][105620] Updated weights for policy 1, policy_version 1335772 (0.0008) [2023-12-27 01:06:15,647][105692] Updated weights for policy 0, policy_version 1333843 (0.0008) [2023-12-27 01:06:15,714][105620] Updated weights for policy 1, policy_version 1335782 (0.0008) [2023-12-27 01:06:15,714][105692] Updated weights for policy 0, policy_version 1333853 (0.0005) [2023-12-27 01:06:15,767][105620] Updated weights for policy 1, policy_version 1335792 (0.0008) [2023-12-27 01:06:15,779][105692] Updated weights for policy 0, policy_version 1333863 (0.0005) [2023-12-27 01:06:15,827][105620] Updated weights for policy 1, policy_version 1335802 (0.0009) [2023-12-27 01:06:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 683532288. Throughput: 0: 9760.7, 1: 9617.6. Samples: 683495784. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:06:16,063][104569] Avg episode reward: [(0, '7825.768'), (1, '8544.992')] [2023-12-27 01:06:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001333872_341524480.pth... [2023-12-27 01:06:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001335808_342007808.pth... [2023-12-27 01:06:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001332720_341229568.pth [2023-12-27 01:06:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001334656_341712896.pth [2023-12-27 01:06:16,385][105692] Updated weights for policy 0, policy_version 1333873 (0.0006) [2023-12-27 01:06:16,418][105620] Updated weights for policy 1, policy_version 1335812 (0.0008) [2023-12-27 01:06:16,444][105692] Updated weights for policy 0, policy_version 1333883 (0.0008) [2023-12-27 01:06:16,466][105620] Updated weights for policy 1, policy_version 1335822 (0.0010) [2023-12-27 01:06:16,468][105585] KL-divergence is very high: 157.1969 [2023-12-27 01:06:16,500][105692] Updated weights for policy 0, policy_version 1333893 (0.0007) [2023-12-27 01:06:16,507][105585] KL-divergence is very high: 288.8992 [2023-12-27 01:06:16,520][105620] Updated weights for policy 1, policy_version 1335832 (0.0005) [2023-12-27 01:06:16,552][105585] KL-divergence is very high: 315.9757 [2023-12-27 01:06:16,552][105692] Updated weights for policy 0, policy_version 1333903 (0.0010) [2023-12-27 01:06:17,076][105620] Updated weights for policy 1, policy_version 1335842 (0.0008) [2023-12-27 01:06:17,140][105620] Updated weights for policy 1, policy_version 1335852 (0.0010) [2023-12-27 01:06:17,170][105692] Updated weights for policy 0, policy_version 1333913 (0.0010) [2023-12-27 01:06:17,175][105585] KL-divergence is very high: 119.2286 [2023-12-27 01:06:17,194][105620] Updated weights for policy 1, policy_version 1335862 (0.0007) [2023-12-27 01:06:17,218][105585] KL-divergence is very high: 153.0761 [2023-12-27 01:06:17,225][105692] Updated weights for policy 0, policy_version 1333923 (0.0010) [2023-12-27 01:06:17,250][105620] Updated weights for policy 1, policy_version 1335872 (0.0005) [2023-12-27 01:06:17,261][105585] KL-divergence is very high: 160.7125 [2023-12-27 01:06:17,278][105692] Updated weights for policy 0, policy_version 1333933 (0.0009) [2023-12-27 01:06:17,871][105620] Updated weights for policy 1, policy_version 1335882 (0.0010) [2023-12-27 01:06:17,931][105692] Updated weights for policy 0, policy_version 1333943 (0.0007) [2023-12-27 01:06:17,935][105620] Updated weights for policy 1, policy_version 1335892 (0.0010) [2023-12-27 01:06:17,991][105692] Updated weights for policy 0, policy_version 1333953 (0.0006) [2023-12-27 01:06:17,997][105620] Updated weights for policy 1, policy_version 1335902 (0.0010) [2023-12-27 01:06:18,046][105692] Updated weights for policy 0, policy_version 1333963 (0.0008) [2023-12-27 01:06:18,656][105620] Updated weights for policy 1, policy_version 1335912 (0.0009) [2023-12-27 01:06:18,717][105620] Updated weights for policy 1, policy_version 1335922 (0.0008) [2023-12-27 01:06:18,772][105620] Updated weights for policy 1, policy_version 1335932 (0.0009) [2023-12-27 01:06:18,827][105692] Updated weights for policy 0, policy_version 1333973 (0.0009) [2023-12-27 01:06:18,880][105692] Updated weights for policy 0, policy_version 1333983 (0.0009) [2023-12-27 01:06:18,936][105692] Updated weights for policy 0, policy_version 1333993 (0.0009) [2023-12-27 01:06:19,485][105620] Updated weights for policy 1, policy_version 1335942 (0.0010) [2023-12-27 01:06:19,551][105620] Updated weights for policy 1, policy_version 1335952 (0.0009) [2023-12-27 01:06:19,612][105620] Updated weights for policy 1, policy_version 1335962 (0.0008) [2023-12-27 01:06:19,783][105692] Updated weights for policy 0, policy_version 1334003 (0.0010) [2023-12-27 01:06:19,848][105692] Updated weights for policy 0, policy_version 1334013 (0.0009) [2023-12-27 01:06:19,919][105692] Updated weights for policy 0, policy_version 1334023 (0.0010) [2023-12-27 01:06:20,380][105620] Updated weights for policy 1, policy_version 1335972 (0.0010) [2023-12-27 01:06:20,439][105620] Updated weights for policy 1, policy_version 1335982 (0.0010) [2023-12-27 01:06:20,492][105620] Updated weights for policy 1, policy_version 1335992 (0.0010) [2023-12-27 01:06:20,618][105692] Updated weights for policy 0, policy_version 1334033 (0.0009) [2023-12-27 01:06:20,684][105692] Updated weights for policy 0, policy_version 1334043 (0.0009) [2023-12-27 01:06:20,741][105692] Updated weights for policy 0, policy_version 1334053 (0.0008) [2023-12-27 01:06:20,805][105692] Updated weights for policy 0, policy_version 1334063 (0.0008) [2023-12-27 01:06:21,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 683630592. Throughput: 0: 9785.2, 1: 9820.8. Samples: 683620440. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 01:06:21,062][104569] Avg episode reward: [(0, '8183.101'), (1, '9005.562')] [2023-12-27 01:06:21,241][105620] Updated weights for policy 1, policy_version 1336002 (0.0010) [2023-12-27 01:06:21,299][105620] Updated weights for policy 1, policy_version 1336012 (0.0009) [2023-12-27 01:06:21,367][105620] Updated weights for policy 1, policy_version 1336022 (0.0009) [2023-12-27 01:06:21,426][105620] Updated weights for policy 1, policy_version 1336032 (0.0008) [2023-12-27 01:06:21,596][105692] Updated weights for policy 0, policy_version 1334073 (0.0010) [2023-12-27 01:06:21,658][105692] Updated weights for policy 0, policy_version 1334083 (0.0009) [2023-12-27 01:06:21,725][105692] Updated weights for policy 0, policy_version 1334093 (0.0009) [2023-12-27 01:06:22,097][105620] Updated weights for policy 1, policy_version 1336042 (0.0008) [2023-12-27 01:06:22,163][105620] Updated weights for policy 1, policy_version 1336052 (0.0008) [2023-12-27 01:06:22,231][105620] Updated weights for policy 1, policy_version 1336062 (0.0008) [2023-12-27 01:06:22,537][105692] Updated weights for policy 0, policy_version 1334103 (0.0008) [2023-12-27 01:06:22,591][105692] Updated weights for policy 0, policy_version 1334113 (0.0009) [2023-12-27 01:06:22,646][105692] Updated weights for policy 0, policy_version 1334123 (0.0009) [2023-12-27 01:06:22,931][105620] Updated weights for policy 1, policy_version 1336072 (0.0010) [2023-12-27 01:06:22,984][105620] Updated weights for policy 1, policy_version 1336082 (0.0009) [2023-12-27 01:06:23,037][105620] Updated weights for policy 1, policy_version 1336092 (0.0010) [2023-12-27 01:06:23,459][105692] Updated weights for policy 0, policy_version 1334133 (0.0010) [2023-12-27 01:06:23,507][105692] Updated weights for policy 0, policy_version 1334143 (0.0008) [2023-12-27 01:06:23,553][105692] Updated weights for policy 0, policy_version 1334153 (0.0009) [2023-12-27 01:06:23,717][105620] Updated weights for policy 1, policy_version 1336102 (0.0008) [2023-12-27 01:06:23,779][105620] Updated weights for policy 1, policy_version 1336112 (0.0011) [2023-12-27 01:06:23,829][105620] Updated weights for policy 1, policy_version 1336122 (0.0010) [2023-12-27 01:06:24,417][105692] Updated weights for policy 0, policy_version 1334163 (0.0009) [2023-12-27 01:06:24,438][105620] Updated weights for policy 1, policy_version 1336132 (0.0008) [2023-12-27 01:06:24,470][105692] Updated weights for policy 0, policy_version 1334173 (0.0010) [2023-12-27 01:06:24,494][105620] Updated weights for policy 1, policy_version 1336142 (0.0005) [2023-12-27 01:06:24,520][105692] Updated weights for policy 0, policy_version 1334183 (0.0008) [2023-12-27 01:06:24,543][105620] Updated weights for policy 1, policy_version 1336152 (0.0005) [2023-12-27 01:06:25,174][105620] Updated weights for policy 1, policy_version 1336162 (0.0007) [2023-12-27 01:06:25,235][105620] Updated weights for policy 1, policy_version 1336172 (0.0006) [2023-12-27 01:06:25,306][105620] Updated weights for policy 1, policy_version 1336182 (0.0010) [2023-12-27 01:06:25,326][105692] Updated weights for policy 0, policy_version 1334193 (0.0008) [2023-12-27 01:06:25,368][105620] Updated weights for policy 1, policy_version 1336192 (0.0010) [2023-12-27 01:06:25,388][105692] Updated weights for policy 0, policy_version 1334203 (0.0006) [2023-12-27 01:06:25,447][105692] Updated weights for policy 0, policy_version 1334213 (0.0008) [2023-12-27 01:06:25,509][105692] Updated weights for policy 0, policy_version 1334223 (0.0008) [2023-12-27 01:06:25,982][105620] Updated weights for policy 1, policy_version 1336202 (0.0007) [2023-12-27 01:06:26,034][105620] Updated weights for policy 1, policy_version 1336212 (0.0009) [2023-12-27 01:06:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 683720704. Throughput: 0: 9728.0, 1: 9904.3. Samples: 683734496. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:06:26,062][104569] Avg episode reward: [(0, '8466.667'), (1, '8672.979')] [2023-12-27 01:06:26,099][105620] Updated weights for policy 1, policy_version 1336222 (0.0006) [2023-12-27 01:06:26,281][105692] Updated weights for policy 0, policy_version 1334233 (0.0009) [2023-12-27 01:06:26,336][105692] Updated weights for policy 0, policy_version 1334243 (0.0009) [2023-12-27 01:06:26,397][105692] Updated weights for policy 0, policy_version 1334253 (0.0010) [2023-12-27 01:06:26,685][105620] Updated weights for policy 1, policy_version 1336232 (0.0007) [2023-12-27 01:06:26,753][105620] Updated weights for policy 1, policy_version 1336242 (0.0005) [2023-12-27 01:06:26,813][105620] Updated weights for policy 1, policy_version 1336252 (0.0006) [2023-12-27 01:06:27,131][105692] Updated weights for policy 0, policy_version 1334263 (0.0007) [2023-12-27 01:06:27,189][105692] Updated weights for policy 0, policy_version 1334273 (0.0006) [2023-12-27 01:06:27,240][105692] Updated weights for policy 0, policy_version 1334283 (0.0005) [2023-12-27 01:06:27,384][105620] Updated weights for policy 1, policy_version 1336262 (0.0006) [2023-12-27 01:06:27,450][105620] Updated weights for policy 1, policy_version 1336272 (0.0006) [2023-12-27 01:06:27,501][105620] Updated weights for policy 1, policy_version 1336282 (0.0005) [2023-12-27 01:06:27,819][105692] Updated weights for policy 0, policy_version 1334293 (0.0007) [2023-12-27 01:06:27,869][105692] Updated weights for policy 0, policy_version 1334303 (0.0007) [2023-12-27 01:06:27,917][105692] Updated weights for policy 0, policy_version 1334313 (0.0005) [2023-12-27 01:06:28,043][105620] Updated weights for policy 1, policy_version 1336292 (0.0005) [2023-12-27 01:06:28,093][105620] Updated weights for policy 1, policy_version 1336302 (0.0005) [2023-12-27 01:06:28,154][105620] Updated weights for policy 1, policy_version 1336312 (0.0005) [2023-12-27 01:06:28,589][105692] Updated weights for policy 0, policy_version 1334323 (0.0006) [2023-12-27 01:06:28,636][105692] Updated weights for policy 0, policy_version 1334333 (0.0008) [2023-12-27 01:06:28,687][105692] Updated weights for policy 0, policy_version 1334343 (0.0007) [2023-12-27 01:06:28,800][105620] Updated weights for policy 1, policy_version 1336322 (0.0006) [2023-12-27 01:06:28,868][105620] Updated weights for policy 1, policy_version 1336332 (0.0009) [2023-12-27 01:06:28,927][105620] Updated weights for policy 1, policy_version 1336342 (0.0008) [2023-12-27 01:06:28,978][105620] Updated weights for policy 1, policy_version 1336352 (0.0009) [2023-12-27 01:06:29,435][105692] Updated weights for policy 0, policy_version 1334353 (0.0009) [2023-12-27 01:06:29,496][105692] Updated weights for policy 0, policy_version 1334363 (0.0010) [2023-12-27 01:06:29,558][105692] Updated weights for policy 0, policy_version 1334373 (0.0011) [2023-12-27 01:06:29,616][105692] Updated weights for policy 0, policy_version 1334383 (0.0010) [2023-12-27 01:06:29,692][105620] Updated weights for policy 1, policy_version 1336362 (0.0006) [2023-12-27 01:06:29,754][105620] Updated weights for policy 1, policy_version 1336372 (0.0008) [2023-12-27 01:06:29,811][105620] Updated weights for policy 1, policy_version 1336382 (0.0007) [2023-12-27 01:06:30,357][105692] Updated weights for policy 0, policy_version 1334393 (0.0011) [2023-12-27 01:06:30,420][105692] Updated weights for policy 0, policy_version 1334403 (0.0011) [2023-12-27 01:06:30,476][105692] Updated weights for policy 0, policy_version 1334413 (0.0007) [2023-12-27 01:06:30,548][105620] Updated weights for policy 1, policy_version 1336392 (0.0010) [2023-12-27 01:06:30,602][105620] Updated weights for policy 1, policy_version 1336402 (0.0010) [2023-12-27 01:06:30,656][105620] Updated weights for policy 1, policy_version 1336412 (0.0010) [2023-12-27 01:06:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 683827200. Throughput: 0: 9792.1, 1: 9997.3. Samples: 683798916. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:06:31,063][104569] Avg episode reward: [(0, '8463.888'), (1, '8861.482')] [2023-12-27 01:06:31,066][105692] Updated weights for policy 0, policy_version 1334423 (0.0007) [2023-12-27 01:06:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001336416_342163456.pth... [2023-12-27 01:06:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001335232_341860352.pth [2023-12-27 01:06:31,121][105692] Updated weights for policy 0, policy_version 1334433 (0.0007) [2023-12-27 01:06:31,178][105692] Updated weights for policy 0, policy_version 1334443 (0.0007) [2023-12-27 01:06:31,204][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001334448_341671936.pth... [2023-12-27 01:06:31,207][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001333296_341377024.pth [2023-12-27 01:06:31,353][105620] Updated weights for policy 1, policy_version 1336422 (0.0009) [2023-12-27 01:06:31,417][105620] Updated weights for policy 1, policy_version 1336432 (0.0010) [2023-12-27 01:06:31,484][105620] Updated weights for policy 1, policy_version 1336442 (0.0011) [2023-12-27 01:06:31,818][105692] Updated weights for policy 0, policy_version 1334453 (0.0007) [2023-12-27 01:06:31,864][105692] Updated weights for policy 0, policy_version 1334463 (0.0006) [2023-12-27 01:06:31,911][105692] Updated weights for policy 0, policy_version 1334473 (0.0006) [2023-12-27 01:06:32,175][105620] Updated weights for policy 1, policy_version 1336452 (0.0010) [2023-12-27 01:06:32,234][105620] Updated weights for policy 1, policy_version 1336462 (0.0011) [2023-12-27 01:06:32,297][105620] Updated weights for policy 1, policy_version 1336472 (0.0011) [2023-12-27 01:06:32,555][105692] Updated weights for policy 0, policy_version 1334483 (0.0006) [2023-12-27 01:06:32,618][105692] Updated weights for policy 0, policy_version 1334493 (0.0007) [2023-12-27 01:06:32,683][105692] Updated weights for policy 0, policy_version 1334503 (0.0006) [2023-12-27 01:06:32,976][105620] Updated weights for policy 1, policy_version 1336482 (0.0010) [2023-12-27 01:06:33,022][105620] Updated weights for policy 1, policy_version 1336492 (0.0010) [2023-12-27 01:06:33,087][105620] Updated weights for policy 1, policy_version 1336502 (0.0010) [2023-12-27 01:06:33,139][105620] Updated weights for policy 1, policy_version 1336512 (0.0010) [2023-12-27 01:06:33,313][105692] Updated weights for policy 0, policy_version 1334513 (0.0009) [2023-12-27 01:06:33,364][105692] Updated weights for policy 0, policy_version 1334523 (0.0005) [2023-12-27 01:06:33,411][105692] Updated weights for policy 0, policy_version 1334533 (0.0005) [2023-12-27 01:06:33,465][105692] Updated weights for policy 0, policy_version 1334543 (0.0005) [2023-12-27 01:06:33,890][105620] Updated weights for policy 1, policy_version 1336522 (0.0010) [2023-12-27 01:06:33,949][105620] Updated weights for policy 1, policy_version 1336532 (0.0010) [2023-12-27 01:06:33,972][105692] Updated weights for policy 0, policy_version 1334553 (0.0005) [2023-12-27 01:06:34,013][105620] Updated weights for policy 1, policy_version 1336542 (0.0011) [2023-12-27 01:06:34,032][105692] Updated weights for policy 0, policy_version 1334563 (0.0005) [2023-12-27 01:06:34,078][105692] Updated weights for policy 0, policy_version 1334573 (0.0005) [2023-12-27 01:06:34,687][105620] Updated weights for policy 1, policy_version 1336552 (0.0010) [2023-12-27 01:06:34,748][105620] Updated weights for policy 1, policy_version 1336562 (0.0008) [2023-12-27 01:06:34,788][105692] Updated weights for policy 0, policy_version 1334583 (0.0007) [2023-12-27 01:06:34,810][105620] Updated weights for policy 1, policy_version 1336572 (0.0009) [2023-12-27 01:06:34,845][105692] Updated weights for policy 0, policy_version 1334593 (0.0007) [2023-12-27 01:06:34,905][105692] Updated weights for policy 0, policy_version 1334603 (0.0009) [2023-12-27 01:06:35,438][105620] Updated weights for policy 1, policy_version 1336582 (0.0008) [2023-12-27 01:06:35,493][105620] Updated weights for policy 1, policy_version 1336592 (0.0009) [2023-12-27 01:06:35,545][105620] Updated weights for policy 1, policy_version 1336602 (0.0009) [2023-12-27 01:06:35,703][105692] Updated weights for policy 0, policy_version 1334613 (0.0010) [2023-12-27 01:06:35,753][105692] Updated weights for policy 0, policy_version 1334623 (0.0007) [2023-12-27 01:06:35,816][105692] Updated weights for policy 0, policy_version 1334633 (0.0005) [2023-12-27 01:06:36,062][104569] Fps is (10 sec: 21299.3, 60 sec: 19797.4, 300 sec: 19494.2). Total num frames: 683933696. Throughput: 0: 9860.5, 1: 10020.0. Samples: 683921668. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:06:36,062][104569] Avg episode reward: [(0, '5318.319'), (1, '9012.363')] [2023-12-27 01:06:36,435][105620] Updated weights for policy 1, policy_version 1336612 (0.0008) [2023-12-27 01:06:36,445][105692] Updated weights for policy 0, policy_version 1334643 (0.0006) [2023-12-27 01:06:36,501][105620] Updated weights for policy 1, policy_version 1336622 (0.0007) [2023-12-27 01:06:36,503][105692] Updated weights for policy 0, policy_version 1334653 (0.0006) [2023-12-27 01:06:36,559][105620] Updated weights for policy 1, policy_version 1336632 (0.0007) [2023-12-27 01:06:36,561][105692] Updated weights for policy 0, policy_version 1334663 (0.0006) [2023-12-27 01:06:37,295][105692] Updated weights for policy 0, policy_version 1334673 (0.0007) [2023-12-27 01:06:37,340][105692] Updated weights for policy 0, policy_version 1334683 (0.0008) [2023-12-27 01:06:37,345][105620] Updated weights for policy 1, policy_version 1336642 (0.0007) [2023-12-27 01:06:37,393][105692] Updated weights for policy 0, policy_version 1334693 (0.0006) [2023-12-27 01:06:37,403][105620] Updated weights for policy 1, policy_version 1336652 (0.0009) [2023-12-27 01:06:37,454][105692] Updated weights for policy 0, policy_version 1334703 (0.0007) [2023-12-27 01:06:37,464][105620] Updated weights for policy 1, policy_version 1336662 (0.0006) [2023-12-27 01:06:37,540][105620] Updated weights for policy 1, policy_version 1336672 (0.0006) [2023-12-27 01:06:38,048][105692] Updated weights for policy 0, policy_version 1334713 (0.0009) [2023-12-27 01:06:38,107][105692] Updated weights for policy 0, policy_version 1334723 (0.0009) [2023-12-27 01:06:38,155][105692] Updated weights for policy 0, policy_version 1334733 (0.0008) [2023-12-27 01:06:38,308][105620] Updated weights for policy 1, policy_version 1336682 (0.0009) [2023-12-27 01:06:38,372][105620] Updated weights for policy 1, policy_version 1336692 (0.0009) [2023-12-27 01:06:38,434][105620] Updated weights for policy 1, policy_version 1336702 (0.0009) [2023-12-27 01:06:38,849][105692] Updated weights for policy 0, policy_version 1334743 (0.0006) [2023-12-27 01:06:38,905][105692] Updated weights for policy 0, policy_version 1334753 (0.0008) [2023-12-27 01:06:38,974][105692] Updated weights for policy 0, policy_version 1334763 (0.0006) [2023-12-27 01:06:39,263][105620] Updated weights for policy 1, policy_version 1336712 (0.0008) [2023-12-27 01:06:39,331][105620] Updated weights for policy 1, policy_version 1336722 (0.0007) [2023-12-27 01:06:39,412][105620] Updated weights for policy 1, policy_version 1336732 (0.0008) [2023-12-27 01:06:39,689][105692] Updated weights for policy 0, policy_version 1334773 (0.0007) [2023-12-27 01:06:39,748][105692] Updated weights for policy 0, policy_version 1334783 (0.0006) [2023-12-27 01:06:39,801][105692] Updated weights for policy 0, policy_version 1334793 (0.0009) [2023-12-27 01:06:40,156][105620] Updated weights for policy 1, policy_version 1336742 (0.0009) [2023-12-27 01:06:40,213][105620] Updated weights for policy 1, policy_version 1336752 (0.0005) [2023-12-27 01:06:40,274][105620] Updated weights for policy 1, policy_version 1336762 (0.0007) [2023-12-27 01:06:40,580][105692] Updated weights for policy 0, policy_version 1334803 (0.0008) [2023-12-27 01:06:40,638][105692] Updated weights for policy 0, policy_version 1334813 (0.0010) [2023-12-27 01:06:40,696][105692] Updated weights for policy 0, policy_version 1334823 (0.0010) [2023-12-27 01:06:40,824][105620] Updated weights for policy 1, policy_version 1336772 (0.0009) [2023-12-27 01:06:40,879][105620] Updated weights for policy 1, policy_version 1336782 (0.0009) [2023-12-27 01:06:40,946][105620] Updated weights for policy 1, policy_version 1336792 (0.0009) [2023-12-27 01:06:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 684032000. Throughput: 0: 9836.3, 1: 9951.4. Samples: 684036892. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:06:41,062][104569] Avg episode reward: [(0, '4064.152'), (1, '8996.516')] [2023-12-27 01:06:41,503][105692] Updated weights for policy 0, policy_version 1334833 (0.0009) [2023-12-27 01:06:41,558][105692] Updated weights for policy 0, policy_version 1334843 (0.0009) [2023-12-27 01:06:41,622][105692] Updated weights for policy 0, policy_version 1334853 (0.0009) [2023-12-27 01:06:41,688][105692] Updated weights for policy 0, policy_version 1334863 (0.0010) [2023-12-27 01:06:41,771][105620] Updated weights for policy 1, policy_version 1336802 (0.0008) [2023-12-27 01:06:41,834][105620] Updated weights for policy 1, policy_version 1336812 (0.0009) [2023-12-27 01:06:41,900][105620] Updated weights for policy 1, policy_version 1336822 (0.0009) [2023-12-27 01:06:41,963][105620] Updated weights for policy 1, policy_version 1336832 (0.0009) [2023-12-27 01:06:42,427][105692] Updated weights for policy 0, policy_version 1334873 (0.0005) [2023-12-27 01:06:42,484][105692] Updated weights for policy 0, policy_version 1334883 (0.0006) [2023-12-27 01:06:42,535][105692] Updated weights for policy 0, policy_version 1334893 (0.0009) [2023-12-27 01:06:42,759][105620] Updated weights for policy 1, policy_version 1336842 (0.0009) [2023-12-27 01:06:42,824][105620] Updated weights for policy 1, policy_version 1336852 (0.0009) [2023-12-27 01:06:42,886][105620] Updated weights for policy 1, policy_version 1336862 (0.0006) [2023-12-27 01:06:43,301][105692] Updated weights for policy 0, policy_version 1334903 (0.0009) [2023-12-27 01:06:43,347][105692] Updated weights for policy 0, policy_version 1334913 (0.0008) [2023-12-27 01:06:43,395][105692] Updated weights for policy 0, policy_version 1334923 (0.0009) [2023-12-27 01:06:43,550][105620] Updated weights for policy 1, policy_version 1336872 (0.0008) [2023-12-27 01:06:43,601][105620] Updated weights for policy 1, policy_version 1336882 (0.0010) [2023-12-27 01:06:43,651][105620] Updated weights for policy 1, policy_version 1336892 (0.0008) [2023-12-27 01:06:44,168][105692] Updated weights for policy 0, policy_version 1334933 (0.0009) [2023-12-27 01:06:44,218][105692] Updated weights for policy 0, policy_version 1334943 (0.0009) [2023-12-27 01:06:44,265][105692] Updated weights for policy 0, policy_version 1334954 (0.0009) [2023-12-27 01:06:44,375][105620] Updated weights for policy 1, policy_version 1336902 (0.0008) [2023-12-27 01:06:44,428][105620] Updated weights for policy 1, policy_version 1336912 (0.0009) [2023-12-27 01:06:44,493][105620] Updated weights for policy 1, policy_version 1336922 (0.0009) [2023-12-27 01:06:45,027][105692] Updated weights for policy 0, policy_version 1334964 (0.0008) [2023-12-27 01:06:45,094][105692] Updated weights for policy 0, policy_version 1334974 (0.0009) [2023-12-27 01:06:45,154][105692] Updated weights for policy 0, policy_version 1334984 (0.0008) [2023-12-27 01:06:45,309][105620] Updated weights for policy 1, policy_version 1336932 (0.0009) [2023-12-27 01:06:45,372][105620] Updated weights for policy 1, policy_version 1336942 (0.0008) [2023-12-27 01:06:45,439][105620] Updated weights for policy 1, policy_version 1336952 (0.0008) [2023-12-27 01:06:45,792][105692] Updated weights for policy 0, policy_version 1334994 (0.0008) [2023-12-27 01:06:45,852][105692] Updated weights for policy 0, policy_version 1335004 (0.0005) [2023-12-27 01:06:45,906][105692] Updated weights for policy 0, policy_version 1335014 (0.0009) [2023-12-27 01:06:46,030][105620] Updated weights for policy 1, policy_version 1336962 (0.0008) [2023-12-27 01:06:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19797.4, 300 sec: 19494.2). Total num frames: 684122112. Throughput: 0: 9772.7, 1: 9906.8. Samples: 684090920. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:06:46,062][104569] Avg episode reward: [(0, '6810.389'), (1, '8821.779')] [2023-12-27 01:06:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001335024_341819392.pth... [2023-12-27 01:06:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001333872_341524480.pth [2023-12-27 01:06:46,077][105620] Updated weights for policy 1, policy_version 1336972 (0.0005) [2023-12-27 01:06:46,131][105620] Updated weights for policy 1, policy_version 1336982 (0.0005) [2023-12-27 01:06:46,183][105620] Updated weights for policy 1, policy_version 1336992 (0.0005) [2023-12-27 01:06:46,183][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001336992_342310912.pth... [2023-12-27 01:06:46,188][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001335808_342007808.pth [2023-12-27 01:06:46,635][105692] Updated weights for policy 0, policy_version 1335025 (0.0009) [2023-12-27 01:06:46,700][105692] Updated weights for policy 0, policy_version 1335035 (0.0005) [2023-12-27 01:06:46,714][105620] Updated weights for policy 1, policy_version 1337002 (0.0006) [2023-12-27 01:06:46,749][105692] Updated weights for policy 0, policy_version 1335045 (0.0005) [2023-12-27 01:06:46,777][105620] Updated weights for policy 1, policy_version 1337012 (0.0005) [2023-12-27 01:06:46,802][105692] Updated weights for policy 0, policy_version 1335055 (0.0008) [2023-12-27 01:06:46,826][105620] Updated weights for policy 1, policy_version 1337022 (0.0005) [2023-12-27 01:06:47,332][105620] Updated weights for policy 1, policy_version 1337032 (0.0005) [2023-12-27 01:06:47,384][105620] Updated weights for policy 1, policy_version 1337042 (0.0009) [2023-12-27 01:06:47,449][105620] Updated weights for policy 1, policy_version 1337052 (0.0010) [2023-12-27 01:06:47,565][105692] Updated weights for policy 0, policy_version 1335065 (0.0008) [2023-12-27 01:06:47,620][105692] Updated weights for policy 0, policy_version 1335075 (0.0009) [2023-12-27 01:06:47,667][105692] Updated weights for policy 0, policy_version 1335085 (0.0009) [2023-12-27 01:06:48,117][105620] Updated weights for policy 1, policy_version 1337062 (0.0007) [2023-12-27 01:06:48,174][105620] Updated weights for policy 1, policy_version 1337072 (0.0005) [2023-12-27 01:06:48,227][105620] Updated weights for policy 1, policy_version 1337082 (0.0005) [2023-12-27 01:06:48,418][105692] Updated weights for policy 0, policy_version 1335095 (0.0009) [2023-12-27 01:06:48,471][105692] Updated weights for policy 0, policy_version 1335105 (0.0010) [2023-12-27 01:06:48,523][105692] Updated weights for policy 0, policy_version 1335115 (0.0007) [2023-12-27 01:06:48,791][105620] Updated weights for policy 1, policy_version 1337092 (0.0005) [2023-12-27 01:06:48,844][105620] Updated weights for policy 1, policy_version 1337102 (0.0006) [2023-12-27 01:06:48,902][105620] Updated weights for policy 1, policy_version 1337112 (0.0006) [2023-12-27 01:06:49,356][105692] Updated weights for policy 0, policy_version 1335125 (0.0009) [2023-12-27 01:06:49,424][105692] Updated weights for policy 0, policy_version 1335135 (0.0008) [2023-12-27 01:06:49,485][105692] Updated weights for policy 0, policy_version 1335145 (0.0007) [2023-12-27 01:06:49,582][105620] Updated weights for policy 1, policy_version 1337122 (0.0007) [2023-12-27 01:06:49,649][105620] Updated weights for policy 1, policy_version 1337132 (0.0008) [2023-12-27 01:06:49,700][105620] Updated weights for policy 1, policy_version 1337142 (0.0009) [2023-12-27 01:06:49,751][105620] Updated weights for policy 1, policy_version 1337152 (0.0007) [2023-12-27 01:06:50,168][105692] Updated weights for policy 0, policy_version 1335155 (0.0006) [2023-12-27 01:06:50,226][105692] Updated weights for policy 0, policy_version 1335165 (0.0010) [2023-12-27 01:06:50,295][105692] Updated weights for policy 0, policy_version 1335176 (0.0011) [2023-12-27 01:06:50,443][105620] Updated weights for policy 1, policy_version 1337162 (0.0009) [2023-12-27 01:06:50,499][105620] Updated weights for policy 1, policy_version 1337172 (0.0009) [2023-12-27 01:06:50,555][105620] Updated weights for policy 1, policy_version 1337182 (0.0009) [2023-12-27 01:06:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 684220416. Throughput: 0: 9789.0, 1: 10009.4. Samples: 684212976. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:06:51,063][104569] Avg episode reward: [(0, '8558.100'), (1, '8728.082')] [2023-12-27 01:06:51,094][105692] Updated weights for policy 0, policy_version 1335186 (0.0009) [2023-12-27 01:06:51,157][105692] Updated weights for policy 0, policy_version 1335196 (0.0008) [2023-12-27 01:06:51,206][105692] Updated weights for policy 0, policy_version 1335206 (0.0009) [2023-12-27 01:06:51,270][105692] Updated weights for policy 0, policy_version 1335216 (0.0008) [2023-12-27 01:06:51,294][105620] Updated weights for policy 1, policy_version 1337192 (0.0009) [2023-12-27 01:06:51,358][105620] Updated weights for policy 1, policy_version 1337202 (0.0009) [2023-12-27 01:06:51,429][105620] Updated weights for policy 1, policy_version 1337212 (0.0007) [2023-12-27 01:06:52,096][105692] Updated weights for policy 0, policy_version 1335226 (0.0008) [2023-12-27 01:06:52,099][105620] Updated weights for policy 1, policy_version 1337222 (0.0008) [2023-12-27 01:06:52,120][105585] KL-divergence is very high: 135.4558 [2023-12-27 01:06:52,158][105692] Updated weights for policy 0, policy_version 1335237 (0.0009) [2023-12-27 01:06:52,159][105620] Updated weights for policy 1, policy_version 1337232 (0.0006) [2023-12-27 01:06:52,162][105585] KL-divergence is very high: 222.2928 [2023-12-27 01:06:52,213][105585] KL-divergence is very high: 191.9586 [2023-12-27 01:06:52,218][105692] Updated weights for policy 0, policy_version 1335247 (0.0009) [2023-12-27 01:06:52,220][105620] Updated weights for policy 1, policy_version 1337242 (0.0010) [2023-12-27 01:06:52,925][105692] Updated weights for policy 0, policy_version 1335257 (0.0009) [2023-12-27 01:06:52,957][105620] Updated weights for policy 1, policy_version 1337252 (0.0007) [2023-12-27 01:06:52,979][105692] Updated weights for policy 0, policy_version 1335267 (0.0008) [2023-12-27 01:06:53,006][105620] Updated weights for policy 1, policy_version 1337262 (0.0006) [2023-12-27 01:06:53,038][105692] Updated weights for policy 0, policy_version 1335277 (0.0006) [2023-12-27 01:06:53,062][105620] Updated weights for policy 1, policy_version 1337272 (0.0008) [2023-12-27 01:06:53,754][105692] Updated weights for policy 0, policy_version 1335287 (0.0008) [2023-12-27 01:06:53,820][105692] Updated weights for policy 0, policy_version 1335297 (0.0009) [2023-12-27 01:06:53,834][105620] Updated weights for policy 1, policy_version 1337282 (0.0010) [2023-12-27 01:06:53,876][105692] Updated weights for policy 0, policy_version 1335307 (0.0006) [2023-12-27 01:06:53,886][105620] Updated weights for policy 1, policy_version 1337292 (0.0007) [2023-12-27 01:06:53,943][105620] Updated weights for policy 1, policy_version 1337302 (0.0008) [2023-12-27 01:06:53,998][105620] Updated weights for policy 1, policy_version 1337312 (0.0006) [2023-12-27 01:06:54,576][105692] Updated weights for policy 0, policy_version 1335317 (0.0009) [2023-12-27 01:06:54,625][105692] Updated weights for policy 0, policy_version 1335327 (0.0010) [2023-12-27 01:06:54,677][105692] Updated weights for policy 0, policy_version 1335337 (0.0010) [2023-12-27 01:06:54,700][105620] Updated weights for policy 1, policy_version 1337322 (0.0007) [2023-12-27 01:06:54,753][105620] Updated weights for policy 1, policy_version 1337332 (0.0008) [2023-12-27 01:06:54,812][105620] Updated weights for policy 1, policy_version 1337342 (0.0008) [2023-12-27 01:06:55,437][105692] Updated weights for policy 0, policy_version 1335347 (0.0010) [2023-12-27 01:06:55,492][105692] Updated weights for policy 0, policy_version 1335357 (0.0009) [2023-12-27 01:06:55,557][105692] Updated weights for policy 0, policy_version 1335367 (0.0009) [2023-12-27 01:06:55,583][105620] Updated weights for policy 1, policy_version 1337352 (0.0008) [2023-12-27 01:06:55,637][105620] Updated weights for policy 1, policy_version 1337362 (0.0009) [2023-12-27 01:06:55,690][105620] Updated weights for policy 1, policy_version 1337372 (0.0005) [2023-12-27 01:06:56,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 684318720. Throughput: 0: 9696.4, 1: 10110.6. Samples: 684327872. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:06:56,063][104569] Avg episode reward: [(0, '8282.880'), (1, '8831.291')] [2023-12-27 01:06:56,156][105692] Updated weights for policy 0, policy_version 1335377 (0.0007) [2023-12-27 01:06:56,218][105692] Updated weights for policy 0, policy_version 1335387 (0.0008) [2023-12-27 01:06:56,267][105692] Updated weights for policy 0, policy_version 1335397 (0.0007) [2023-12-27 01:06:56,285][105620] Updated weights for policy 1, policy_version 1337382 (0.0006) [2023-12-27 01:06:56,322][105692] Updated weights for policy 0, policy_version 1335407 (0.0005) [2023-12-27 01:06:56,342][105620] Updated weights for policy 1, policy_version 1337392 (0.0008) [2023-12-27 01:06:56,399][105620] Updated weights for policy 1, policy_version 1337403 (0.0011) [2023-12-27 01:06:57,009][105692] Updated weights for policy 0, policy_version 1335417 (0.0010) [2023-12-27 01:06:57,060][105692] Updated weights for policy 0, policy_version 1335427 (0.0010) [2023-12-27 01:06:57,115][105692] Updated weights for policy 0, policy_version 1335437 (0.0010) [2023-12-27 01:06:57,149][105620] Updated weights for policy 1, policy_version 1337413 (0.0010) [2023-12-27 01:06:57,207][105620] Updated weights for policy 1, policy_version 1337423 (0.0006) [2023-12-27 01:06:57,258][105620] Updated weights for policy 1, policy_version 1337433 (0.0005) [2023-12-27 01:06:57,754][105692] Updated weights for policy 0, policy_version 1335447 (0.0010) [2023-12-27 01:06:57,802][105692] Updated weights for policy 0, policy_version 1335457 (0.0010) [2023-12-27 01:06:57,846][105692] Updated weights for policy 0, policy_version 1335467 (0.0010) [2023-12-27 01:06:57,908][105620] Updated weights for policy 1, policy_version 1337443 (0.0007) [2023-12-27 01:06:57,960][105620] Updated weights for policy 1, policy_version 1337453 (0.0008) [2023-12-27 01:06:58,011][105620] Updated weights for policy 1, policy_version 1337463 (0.0009) [2023-12-27 01:06:58,557][105692] Updated weights for policy 0, policy_version 1335477 (0.0010) [2023-12-27 01:06:58,623][105692] Updated weights for policy 0, policy_version 1335487 (0.0009) [2023-12-27 01:06:58,680][105692] Updated weights for policy 0, policy_version 1335497 (0.0011) [2023-12-27 01:06:58,845][105620] Updated weights for policy 1, policy_version 1337473 (0.0009) [2023-12-27 01:06:58,926][105620] Updated weights for policy 1, policy_version 1337484 (0.0009) [2023-12-27 01:06:58,979][105620] Updated weights for policy 1, policy_version 1337494 (0.0009) [2023-12-27 01:06:59,035][105620] Updated weights for policy 1, policy_version 1337504 (0.0009) [2023-12-27 01:06:59,504][105692] Updated weights for policy 0, policy_version 1335507 (0.0010) [2023-12-27 01:06:59,568][105692] Updated weights for policy 0, policy_version 1335517 (0.0009) [2023-12-27 01:06:59,621][105692] Updated weights for policy 0, policy_version 1335527 (0.0011) [2023-12-27 01:06:59,764][105620] Updated weights for policy 1, policy_version 1337514 (0.0008) [2023-12-27 01:06:59,812][105620] Updated weights for policy 1, policy_version 1337524 (0.0007) [2023-12-27 01:06:59,872][105620] Updated weights for policy 1, policy_version 1337534 (0.0008) [2023-12-27 01:07:00,371][105692] Updated weights for policy 0, policy_version 1335537 (0.0010) [2023-12-27 01:07:00,432][105692] Updated weights for policy 0, policy_version 1335547 (0.0010) [2023-12-27 01:07:00,486][105692] Updated weights for policy 0, policy_version 1335557 (0.0008) [2023-12-27 01:07:00,537][105692] Updated weights for policy 0, policy_version 1335567 (0.0006) [2023-12-27 01:07:00,549][105620] Updated weights for policy 1, policy_version 1337544 (0.0010) [2023-12-27 01:07:00,614][105620] Updated weights for policy 1, policy_version 1337554 (0.0006) [2023-12-27 01:07:00,675][105620] Updated weights for policy 1, policy_version 1337564 (0.0008) [2023-12-27 01:07:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 684417024. Throughput: 0: 9786.6, 1: 10055.5. Samples: 684388676. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:07:01,062][104569] Avg episode reward: [(0, '8367.970'), (1, '9102.315')] [2023-12-27 01:07:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001335568_341958656.pth... [2023-12-27 01:07:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001337568_342458368.pth... [2023-12-27 01:07:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001334448_341671936.pth [2023-12-27 01:07:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001336416_342163456.pth [2023-12-27 01:07:01,320][105692] Updated weights for policy 0, policy_version 1335577 (0.0008) [2023-12-27 01:07:01,365][105620] Updated weights for policy 1, policy_version 1337574 (0.0009) [2023-12-27 01:07:01,379][105692] Updated weights for policy 0, policy_version 1335587 (0.0008) [2023-12-27 01:07:01,423][105620] Updated weights for policy 1, policy_version 1337584 (0.0007) [2023-12-27 01:07:01,436][105692] Updated weights for policy 0, policy_version 1335597 (0.0009) [2023-12-27 01:07:01,481][105620] Updated weights for policy 1, policy_version 1337594 (0.0008) [2023-12-27 01:07:02,119][105692] Updated weights for policy 0, policy_version 1335607 (0.0008) [2023-12-27 01:07:02,149][105620] Updated weights for policy 1, policy_version 1337604 (0.0008) [2023-12-27 01:07:02,175][105692] Updated weights for policy 0, policy_version 1335617 (0.0007) [2023-12-27 01:07:02,213][105620] Updated weights for policy 1, policy_version 1337614 (0.0009) [2023-12-27 01:07:02,224][105692] Updated weights for policy 0, policy_version 1335627 (0.0007) [2023-12-27 01:07:02,276][105620] Updated weights for policy 1, policy_version 1337624 (0.0008) [2023-12-27 01:07:02,957][105692] Updated weights for policy 0, policy_version 1335637 (0.0006) [2023-12-27 01:07:03,022][105692] Updated weights for policy 0, policy_version 1335647 (0.0005) [2023-12-27 01:07:03,037][105620] Updated weights for policy 1, policy_version 1337634 (0.0009) [2023-12-27 01:07:03,073][105692] Updated weights for policy 0, policy_version 1335657 (0.0005) [2023-12-27 01:07:03,085][105620] Updated weights for policy 1, policy_version 1337644 (0.0008) [2023-12-27 01:07:03,139][105620] Updated weights for policy 1, policy_version 1337654 (0.0009) [2023-12-27 01:07:03,197][105620] Updated weights for policy 1, policy_version 1337664 (0.0010) [2023-12-27 01:07:03,665][105692] Updated weights for policy 0, policy_version 1335667 (0.0005) [2023-12-27 01:07:03,715][105692] Updated weights for policy 0, policy_version 1335677 (0.0005) [2023-12-27 01:07:03,769][105692] Updated weights for policy 0, policy_version 1335687 (0.0005) [2023-12-27 01:07:04,063][105620] Updated weights for policy 1, policy_version 1337674 (0.0008) [2023-12-27 01:07:04,134][105620] Updated weights for policy 1, policy_version 1337684 (0.0008) [2023-12-27 01:07:04,197][105620] Updated weights for policy 1, policy_version 1337694 (0.0008) [2023-12-27 01:07:04,442][105692] Updated weights for policy 0, policy_version 1335697 (0.0006) [2023-12-27 01:07:04,497][105692] Updated weights for policy 0, policy_version 1335707 (0.0011) [2023-12-27 01:07:04,542][105692] Updated weights for policy 0, policy_version 1335717 (0.0010) [2023-12-27 01:07:04,591][105692] Updated weights for policy 0, policy_version 1335727 (0.0010) [2023-12-27 01:07:04,925][105620] Updated weights for policy 1, policy_version 1337704 (0.0008) [2023-12-27 01:07:04,974][105620] Updated weights for policy 1, policy_version 1337714 (0.0006) [2023-12-27 01:07:05,030][105620] Updated weights for policy 1, policy_version 1337725 (0.0007) [2023-12-27 01:07:05,247][105692] Updated weights for policy 0, policy_version 1335737 (0.0006) [2023-12-27 01:07:05,298][105692] Updated weights for policy 0, policy_version 1335747 (0.0007) [2023-12-27 01:07:05,345][105692] Updated weights for policy 0, policy_version 1335757 (0.0007) [2023-12-27 01:07:05,892][105620] Updated weights for policy 1, policy_version 1337735 (0.0009) [2023-12-27 01:07:05,909][105692] Updated weights for policy 0, policy_version 1335767 (0.0005) [2023-12-27 01:07:05,941][105620] Updated weights for policy 1, policy_version 1337745 (0.0009) [2023-12-27 01:07:05,971][105692] Updated weights for policy 0, policy_version 1335777 (0.0006) [2023-12-27 01:07:06,004][105620] Updated weights for policy 1, policy_version 1337755 (0.0008) [2023-12-27 01:07:06,028][105692] Updated weights for policy 0, policy_version 1335787 (0.0005) [2023-12-27 01:07:06,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 684523520. Throughput: 0: 9708.8, 1: 9930.5. Samples: 684504208. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:07:06,062][104569] Avg episode reward: [(0, '8276.572'), (1, '8831.437')] [2023-12-27 01:07:06,597][105692] Updated weights for policy 0, policy_version 1335797 (0.0005) [2023-12-27 01:07:06,664][105692] Updated weights for policy 0, policy_version 1335807 (0.0009) [2023-12-27 01:07:06,729][105692] Updated weights for policy 0, policy_version 1335817 (0.0010) [2023-12-27 01:07:06,885][105620] Updated weights for policy 1, policy_version 1337765 (0.0009) [2023-12-27 01:07:06,945][105620] Updated weights for policy 1, policy_version 1337775 (0.0008) [2023-12-27 01:07:07,004][105620] Updated weights for policy 1, policy_version 1337785 (0.0007) [2023-12-27 01:07:07,435][105692] Updated weights for policy 0, policy_version 1335827 (0.0011) [2023-12-27 01:07:07,493][105692] Updated weights for policy 0, policy_version 1335837 (0.0010) [2023-12-27 01:07:07,551][105692] Updated weights for policy 0, policy_version 1335847 (0.0007) [2023-12-27 01:07:07,720][105620] Updated weights for policy 1, policy_version 1337795 (0.0007) [2023-12-27 01:07:07,780][105620] Updated weights for policy 1, policy_version 1337805 (0.0005) [2023-12-27 01:07:07,798][105586] KL-divergence is very high: 148.7866 [2023-12-27 01:07:07,836][105620] Updated weights for policy 1, policy_version 1337815 (0.0005) [2023-12-27 01:07:07,866][105586] KL-divergence is very high: 172.4242 [2023-12-27 01:07:08,261][105692] Updated weights for policy 0, policy_version 1335857 (0.0008) [2023-12-27 01:07:08,327][105692] Updated weights for policy 0, policy_version 1335867 (0.0010) [2023-12-27 01:07:08,372][105620] Updated weights for policy 1, policy_version 1337825 (0.0006) [2023-12-27 01:07:08,394][105692] Updated weights for policy 0, policy_version 1335877 (0.0011) [2023-12-27 01:07:08,420][105620] Updated weights for policy 1, policy_version 1337835 (0.0008) [2023-12-27 01:07:08,449][105692] Updated weights for policy 0, policy_version 1335887 (0.0010) [2023-12-27 01:07:08,472][105620] Updated weights for policy 1, policy_version 1337845 (0.0007) [2023-12-27 01:07:08,540][105620] Updated weights for policy 1, policy_version 1337855 (0.0006) [2023-12-27 01:07:09,125][105620] Updated weights for policy 1, policy_version 1337865 (0.0010) [2023-12-27 01:07:09,176][105692] Updated weights for policy 0, policy_version 1335897 (0.0006) [2023-12-27 01:07:09,178][105620] Updated weights for policy 1, policy_version 1337875 (0.0011) [2023-12-27 01:07:09,234][105620] Updated weights for policy 1, policy_version 1337885 (0.0011) [2023-12-27 01:07:09,235][105692] Updated weights for policy 0, policy_version 1335907 (0.0007) [2023-12-27 01:07:09,301][105692] Updated weights for policy 0, policy_version 1335917 (0.0006) [2023-12-27 01:07:09,990][105620] Updated weights for policy 1, policy_version 1337895 (0.0011) [2023-12-27 01:07:10,019][105692] Updated weights for policy 0, policy_version 1335927 (0.0007) [2023-12-27 01:07:10,051][105620] Updated weights for policy 1, policy_version 1337905 (0.0009) [2023-12-27 01:07:10,081][105692] Updated weights for policy 0, policy_version 1335937 (0.0005) [2023-12-27 01:07:10,120][105620] Updated weights for policy 1, policy_version 1337915 (0.0009) [2023-12-27 01:07:10,135][105692] Updated weights for policy 0, policy_version 1335947 (0.0006) [2023-12-27 01:07:10,820][105692] Updated weights for policy 0, policy_version 1335957 (0.0009) [2023-12-27 01:07:10,885][105692] Updated weights for policy 0, policy_version 1335967 (0.0010) [2023-12-27 01:07:10,888][105620] Updated weights for policy 1, policy_version 1337925 (0.0006) [2023-12-27 01:07:10,938][105692] Updated weights for policy 0, policy_version 1335977 (0.0007) [2023-12-27 01:07:10,941][105620] Updated weights for policy 1, policy_version 1337935 (0.0007) [2023-12-27 01:07:11,000][105620] Updated weights for policy 1, policy_version 1337945 (0.0007) [2023-12-27 01:07:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19933.9, 300 sec: 19549.7). Total num frames: 684621824. Throughput: 0: 9916.1, 1: 9852.9. Samples: 684624100. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:07:11,062][104569] Avg episode reward: [(0, '8270.971'), (1, '7152.340')] [2023-12-27 01:07:11,757][105692] Updated weights for policy 0, policy_version 1335987 (0.0008) [2023-12-27 01:07:11,787][105620] Updated weights for policy 1, policy_version 1337955 (0.0008) [2023-12-27 01:07:11,820][105692] Updated weights for policy 0, policy_version 1335997 (0.0010) [2023-12-27 01:07:11,847][105620] Updated weights for policy 1, policy_version 1337965 (0.0006) [2023-12-27 01:07:11,883][105692] Updated weights for policy 0, policy_version 1336007 (0.0009) [2023-12-27 01:07:11,910][105620] Updated weights for policy 1, policy_version 1337975 (0.0010) [2023-12-27 01:07:12,582][105692] Updated weights for policy 0, policy_version 1336017 (0.0011) [2023-12-27 01:07:12,651][105692] Updated weights for policy 0, policy_version 1336027 (0.0011) [2023-12-27 01:07:12,667][105620] Updated weights for policy 1, policy_version 1337985 (0.0010) [2023-12-27 01:07:12,714][105692] Updated weights for policy 0, policy_version 1336037 (0.0011) [2023-12-27 01:07:12,733][105620] Updated weights for policy 1, policy_version 1337995 (0.0010) [2023-12-27 01:07:12,773][105692] Updated weights for policy 0, policy_version 1336047 (0.0011) [2023-12-27 01:07:12,795][105620] Updated weights for policy 1, policy_version 1338005 (0.0010) [2023-12-27 01:07:12,865][105620] Updated weights for policy 1, policy_version 1338015 (0.0011) [2023-12-27 01:07:13,438][105692] Updated weights for policy 0, policy_version 1336057 (0.0006) [2023-12-27 01:07:13,488][105620] Updated weights for policy 1, policy_version 1338025 (0.0006) [2023-12-27 01:07:13,496][105692] Updated weights for policy 0, policy_version 1336067 (0.0005) [2023-12-27 01:07:13,543][105692] Updated weights for policy 0, policy_version 1336077 (0.0007) [2023-12-27 01:07:13,560][105620] Updated weights for policy 1, policy_version 1338035 (0.0009) [2023-12-27 01:07:13,619][105620] Updated weights for policy 1, policy_version 1338045 (0.0010) [2023-12-27 01:07:14,183][105692] Updated weights for policy 0, policy_version 1336087 (0.0006) [2023-12-27 01:07:14,229][105692] Updated weights for policy 0, policy_version 1336097 (0.0007) [2023-12-27 01:07:14,276][105620] Updated weights for policy 1, policy_version 1338055 (0.0007) [2023-12-27 01:07:14,276][105692] Updated weights for policy 0, policy_version 1336107 (0.0008) [2023-12-27 01:07:14,334][105620] Updated weights for policy 1, policy_version 1338065 (0.0005) [2023-12-27 01:07:14,386][105620] Updated weights for policy 1, policy_version 1338075 (0.0006) [2023-12-27 01:07:14,986][105692] Updated weights for policy 0, policy_version 1336117 (0.0008) [2023-12-27 01:07:15,049][105692] Updated weights for policy 0, policy_version 1336127 (0.0011) [2023-12-27 01:07:15,063][105620] Updated weights for policy 1, policy_version 1338085 (0.0007) [2023-12-27 01:07:15,104][105692] Updated weights for policy 0, policy_version 1336137 (0.0008) [2023-12-27 01:07:15,119][105620] Updated weights for policy 1, policy_version 1338095 (0.0009) [2023-12-27 01:07:15,179][105620] Updated weights for policy 1, policy_version 1338105 (0.0009) [2023-12-27 01:07:15,780][105692] Updated weights for policy 0, policy_version 1336147 (0.0007) [2023-12-27 01:07:15,843][105692] Updated weights for policy 0, policy_version 1336157 (0.0010) [2023-12-27 01:07:15,906][105692] Updated weights for policy 0, policy_version 1336167 (0.0011) [2023-12-27 01:07:15,917][105620] Updated weights for policy 1, policy_version 1338115 (0.0009) [2023-12-27 01:07:15,975][105620] Updated weights for policy 1, policy_version 1338125 (0.0006) [2023-12-27 01:07:16,023][105620] Updated weights for policy 1, policy_version 1338135 (0.0008) [2023-12-27 01:07:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 684711936. Throughput: 0: 9866.3, 1: 9746.7. Samples: 684681500. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:07:16,063][104569] Avg episode reward: [(0, '8902.490'), (1, '7252.501')] [2023-12-27 01:07:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001336176_342114304.pth... [2023-12-27 01:07:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001338144_342605824.pth... [2023-12-27 01:07:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001335024_341819392.pth [2023-12-27 01:07:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001336992_342310912.pth [2023-12-27 01:07:16,644][105692] Updated weights for policy 0, policy_version 1336177 (0.0010) [2023-12-27 01:07:16,648][105620] Updated weights for policy 1, policy_version 1338145 (0.0007) [2023-12-27 01:07:16,702][105692] Updated weights for policy 0, policy_version 1336187 (0.0010) [2023-12-27 01:07:16,708][105620] Updated weights for policy 1, policy_version 1338155 (0.0005) [2023-12-27 01:07:16,757][105692] Updated weights for policy 0, policy_version 1336197 (0.0010) [2023-12-27 01:07:16,771][105620] Updated weights for policy 1, policy_version 1338165 (0.0005) [2023-12-27 01:07:16,815][105692] Updated weights for policy 0, policy_version 1336207 (0.0007) [2023-12-27 01:07:16,835][105620] Updated weights for policy 1, policy_version 1338175 (0.0008) [2023-12-27 01:07:17,359][105692] Updated weights for policy 0, policy_version 1336217 (0.0005) [2023-12-27 01:07:17,413][105692] Updated weights for policy 0, policy_version 1336227 (0.0005) [2023-12-27 01:07:17,468][105692] Updated weights for policy 0, policy_version 1336237 (0.0005) [2023-12-27 01:07:17,559][105620] Updated weights for policy 1, policy_version 1338185 (0.0006) [2023-12-27 01:07:17,626][105620] Updated weights for policy 1, policy_version 1338195 (0.0008) [2023-12-27 01:07:17,694][105620] Updated weights for policy 1, policy_version 1338205 (0.0006) [2023-12-27 01:07:18,036][105692] Updated weights for policy 0, policy_version 1336247 (0.0005) [2023-12-27 01:07:18,095][105692] Updated weights for policy 0, policy_version 1336257 (0.0005) [2023-12-27 01:07:18,153][105692] Updated weights for policy 0, policy_version 1336267 (0.0005) [2023-12-27 01:07:18,363][105620] Updated weights for policy 1, policy_version 1338215 (0.0009) [2023-12-27 01:07:18,418][105620] Updated weights for policy 1, policy_version 1338225 (0.0005) [2023-12-27 01:07:18,464][105620] Updated weights for policy 1, policy_version 1338235 (0.0005) [2023-12-27 01:07:18,841][105692] Updated weights for policy 0, policy_version 1336277 (0.0007) [2023-12-27 01:07:18,893][105692] Updated weights for policy 0, policy_version 1336287 (0.0009) [2023-12-27 01:07:18,946][105692] Updated weights for policy 0, policy_version 1336297 (0.0009) [2023-12-27 01:07:19,137][105620] Updated weights for policy 1, policy_version 1338245 (0.0008) [2023-12-27 01:07:19,205][105620] Updated weights for policy 1, policy_version 1338255 (0.0009) [2023-12-27 01:07:19,271][105620] Updated weights for policy 1, policy_version 1338265 (0.0009) [2023-12-27 01:07:19,662][105692] Updated weights for policy 0, policy_version 1336307 (0.0009) [2023-12-27 01:07:19,728][105692] Updated weights for policy 0, policy_version 1336317 (0.0006) [2023-12-27 01:07:19,798][105692] Updated weights for policy 0, policy_version 1336327 (0.0007) [2023-12-27 01:07:20,102][105620] Updated weights for policy 1, policy_version 1338275 (0.0009) [2023-12-27 01:07:20,153][105620] Updated weights for policy 1, policy_version 1338285 (0.0010) [2023-12-27 01:07:20,209][105620] Updated weights for policy 1, policy_version 1338295 (0.0009) [2023-12-27 01:07:20,446][105692] Updated weights for policy 0, policy_version 1336337 (0.0008) [2023-12-27 01:07:20,502][105692] Updated weights for policy 0, policy_version 1336347 (0.0009) [2023-12-27 01:07:20,565][105692] Updated weights for policy 0, policy_version 1336357 (0.0009) [2023-12-27 01:07:20,625][105692] Updated weights for policy 0, policy_version 1336367 (0.0009) [2023-12-27 01:07:21,028][105620] Updated weights for policy 1, policy_version 1338305 (0.0009) [2023-12-27 01:07:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 684810240. Throughput: 0: 9839.2, 1: 9751.3. Samples: 684803240. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:07:21,063][104569] Avg episode reward: [(0, '8545.188'), (1, '7855.172')] [2023-12-27 01:07:21,091][105620] Updated weights for policy 1, policy_version 1338315 (0.0009) [2023-12-27 01:07:21,155][105620] Updated weights for policy 1, policy_version 1338325 (0.0009) [2023-12-27 01:07:21,222][105620] Updated weights for policy 1, policy_version 1338335 (0.0009) [2023-12-27 01:07:21,346][105692] Updated weights for policy 0, policy_version 1336377 (0.0008) [2023-12-27 01:07:21,417][105692] Updated weights for policy 0, policy_version 1336387 (0.0009) [2023-12-27 01:07:21,475][105692] Updated weights for policy 0, policy_version 1336397 (0.0009) [2023-12-27 01:07:21,995][105620] Updated weights for policy 1, policy_version 1338345 (0.0010) [2023-12-27 01:07:22,060][105620] Updated weights for policy 1, policy_version 1338355 (0.0011) [2023-12-27 01:07:22,114][105620] Updated weights for policy 1, policy_version 1338365 (0.0011) [2023-12-27 01:07:22,117][105586] KL-divergence is very high: 219.4384 [2023-12-27 01:07:22,289][105692] Updated weights for policy 0, policy_version 1336407 (0.0010) [2023-12-27 01:07:22,364][105692] Updated weights for policy 0, policy_version 1336417 (0.0011) [2023-12-27 01:07:22,426][105692] Updated weights for policy 0, policy_version 1336427 (0.0011) [2023-12-27 01:07:22,823][105620] Updated weights for policy 1, policy_version 1338375 (0.0011) [2023-12-27 01:07:22,880][105620] Updated weights for policy 1, policy_version 1338385 (0.0006) [2023-12-27 01:07:22,938][105620] Updated weights for policy 1, policy_version 1338395 (0.0005) [2023-12-27 01:07:23,162][105692] Updated weights for policy 0, policy_version 1336437 (0.0011) [2023-12-27 01:07:23,217][105692] Updated weights for policy 0, policy_version 1336447 (0.0010) [2023-12-27 01:07:23,266][105692] Updated weights for policy 0, policy_version 1336457 (0.0010) [2023-12-27 01:07:23,530][105620] Updated weights for policy 1, policy_version 1338405 (0.0008) [2023-12-27 01:07:23,586][105620] Updated weights for policy 1, policy_version 1338415 (0.0011) [2023-12-27 01:07:23,635][105620] Updated weights for policy 1, policy_version 1338425 (0.0010) [2023-12-27 01:07:24,007][105692] Updated weights for policy 0, policy_version 1336467 (0.0010) [2023-12-27 01:07:24,069][105692] Updated weights for policy 0, policy_version 1336477 (0.0011) [2023-12-27 01:07:24,135][105692] Updated weights for policy 0, policy_version 1336487 (0.0009) [2023-12-27 01:07:24,232][105620] Updated weights for policy 1, policy_version 1338435 (0.0009) [2023-12-27 01:07:24,290][105620] Updated weights for policy 1, policy_version 1338445 (0.0007) [2023-12-27 01:07:24,342][105620] Updated weights for policy 1, policy_version 1338455 (0.0010) [2023-12-27 01:07:24,719][105692] Updated weights for policy 0, policy_version 1336497 (0.0009) [2023-12-27 01:07:24,767][105692] Updated weights for policy 0, policy_version 1336507 (0.0005) [2023-12-27 01:07:24,817][105692] Updated weights for policy 0, policy_version 1336517 (0.0008) [2023-12-27 01:07:24,864][105692] Updated weights for policy 0, policy_version 1336527 (0.0008) [2023-12-27 01:07:25,063][105620] Updated weights for policy 1, policy_version 1338465 (0.0010) [2023-12-27 01:07:25,121][105620] Updated weights for policy 1, policy_version 1338475 (0.0009) [2023-12-27 01:07:25,175][105620] Updated weights for policy 1, policy_version 1338485 (0.0006) [2023-12-27 01:07:25,226][105620] Updated weights for policy 1, policy_version 1338495 (0.0005) [2023-12-27 01:07:25,668][105692] Updated weights for policy 0, policy_version 1336537 (0.0006) [2023-12-27 01:07:25,720][105692] Updated weights for policy 0, policy_version 1336547 (0.0005) [2023-12-27 01:07:25,777][105692] Updated weights for policy 0, policy_version 1336557 (0.0007) [2023-12-27 01:07:25,917][105620] Updated weights for policy 1, policy_version 1338505 (0.0009) [2023-12-27 01:07:25,982][105620] Updated weights for policy 1, policy_version 1338515 (0.0009) [2023-12-27 01:07:26,041][105620] Updated weights for policy 1, policy_version 1338525 (0.0009) [2023-12-27 01:07:26,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19933.9, 300 sec: 19549.7). Total num frames: 684916736. Throughput: 0: 9815.8, 1: 9815.0. Samples: 684920280. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:07:26,062][104569] Avg episode reward: [(0, '8457.792'), (1, '7406.197')] [2023-12-27 01:07:26,462][105692] Updated weights for policy 0, policy_version 1336567 (0.0008) [2023-12-27 01:07:26,514][105692] Updated weights for policy 0, policy_version 1336577 (0.0009) [2023-12-27 01:07:26,569][105692] Updated weights for policy 0, policy_version 1336587 (0.0009) [2023-12-27 01:07:26,817][105620] Updated weights for policy 1, policy_version 1338535 (0.0009) [2023-12-27 01:07:26,870][105620] Updated weights for policy 1, policy_version 1338545 (0.0009) [2023-12-27 01:07:26,927][105620] Updated weights for policy 1, policy_version 1338555 (0.0009) [2023-12-27 01:07:27,295][105692] Updated weights for policy 0, policy_version 1336597 (0.0007) [2023-12-27 01:07:27,357][105692] Updated weights for policy 0, policy_version 1336607 (0.0005) [2023-12-27 01:07:27,414][105692] Updated weights for policy 0, policy_version 1336617 (0.0005) [2023-12-27 01:07:27,810][105620] Updated weights for policy 1, policy_version 1338565 (0.0009) [2023-12-27 01:07:27,862][105620] Updated weights for policy 1, policy_version 1338575 (0.0009) [2023-12-27 01:07:27,897][105692] Updated weights for policy 0, policy_version 1336627 (0.0005) [2023-12-27 01:07:27,918][105620] Updated weights for policy 1, policy_version 1338585 (0.0008) [2023-12-27 01:07:27,940][105692] Updated weights for policy 0, policy_version 1336637 (0.0005) [2023-12-27 01:07:28,005][105692] Updated weights for policy 0, policy_version 1336647 (0.0005) [2023-12-27 01:07:28,661][105692] Updated weights for policy 0, policy_version 1336657 (0.0005) [2023-12-27 01:07:28,696][105620] Updated weights for policy 1, policy_version 1338595 (0.0009) [2023-12-27 01:07:28,716][105692] Updated weights for policy 0, policy_version 1336667 (0.0005) [2023-12-27 01:07:28,750][105620] Updated weights for policy 1, policy_version 1338605 (0.0009) [2023-12-27 01:07:28,775][105692] Updated weights for policy 0, policy_version 1336677 (0.0006) [2023-12-27 01:07:28,813][105620] Updated weights for policy 1, policy_version 1338615 (0.0007) [2023-12-27 01:07:28,835][105692] Updated weights for policy 0, policy_version 1336687 (0.0007) [2023-12-27 01:07:29,481][105692] Updated weights for policy 0, policy_version 1336697 (0.0008) [2023-12-27 01:07:29,538][105692] Updated weights for policy 0, policy_version 1336707 (0.0005) [2023-12-27 01:07:29,601][105692] Updated weights for policy 0, policy_version 1336717 (0.0006) [2023-12-27 01:07:29,645][105620] Updated weights for policy 1, policy_version 1338625 (0.0007) [2023-12-27 01:07:29,693][105620] Updated weights for policy 1, policy_version 1338635 (0.0008) [2023-12-27 01:07:29,741][105620] Updated weights for policy 1, policy_version 1338645 (0.0007) [2023-12-27 01:07:29,797][105620] Updated weights for policy 1, policy_version 1338655 (0.0008) [2023-12-27 01:07:30,233][105692] Updated weights for policy 0, policy_version 1336727 (0.0008) [2023-12-27 01:07:30,290][105692] Updated weights for policy 0, policy_version 1336737 (0.0006) [2023-12-27 01:07:30,340][105692] Updated weights for policy 0, policy_version 1336747 (0.0007) [2023-12-27 01:07:30,461][105620] Updated weights for policy 1, policy_version 1338665 (0.0009) [2023-12-27 01:07:30,516][105620] Updated weights for policy 1, policy_version 1338675 (0.0009) [2023-12-27 01:07:30,573][105620] Updated weights for policy 1, policy_version 1338685 (0.0009) [2023-12-27 01:07:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 685006848. Throughput: 0: 9935.5, 1: 9792.3. Samples: 684978668. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:07:31,062][104569] Avg episode reward: [(0, '8366.019'), (1, '8071.060')] [2023-12-27 01:07:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001338688_342745088.pth... [2023-12-27 01:07:31,069][105692] Updated weights for policy 0, policy_version 1336757 (0.0009) [2023-12-27 01:07:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001337568_342458368.pth [2023-12-27 01:07:31,130][105692] Updated weights for policy 0, policy_version 1336767 (0.0009) [2023-12-27 01:07:31,192][105692] Updated weights for policy 0, policy_version 1336777 (0.0009) [2023-12-27 01:07:31,234][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001336784_342269952.pth... [2023-12-27 01:07:31,238][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001335568_341958656.pth [2023-12-27 01:07:31,281][105620] Updated weights for policy 1, policy_version 1338695 (0.0008) [2023-12-27 01:07:31,343][105620] Updated weights for policy 1, policy_version 1338705 (0.0009) [2023-12-27 01:07:31,405][105620] Updated weights for policy 1, policy_version 1338715 (0.0007) [2023-12-27 01:07:31,412][105586] KL-divergence is very high: 110.9038 [2023-12-27 01:07:31,907][105692] Updated weights for policy 0, policy_version 1336787 (0.0009) [2023-12-27 01:07:31,960][105692] Updated weights for policy 0, policy_version 1336797 (0.0009) [2023-12-27 01:07:32,018][105692] Updated weights for policy 0, policy_version 1336807 (0.0009) [2023-12-27 01:07:32,105][105620] Updated weights for policy 1, policy_version 1338725 (0.0006) [2023-12-27 01:07:32,163][105620] Updated weights for policy 1, policy_version 1338735 (0.0008) [2023-12-27 01:07:32,220][105620] Updated weights for policy 1, policy_version 1338745 (0.0008) [2023-12-27 01:07:32,730][105692] Updated weights for policy 0, policy_version 1336817 (0.0009) [2023-12-27 01:07:32,798][105692] Updated weights for policy 0, policy_version 1336827 (0.0005) [2023-12-27 01:07:32,858][105692] Updated weights for policy 0, policy_version 1336837 (0.0007) [2023-12-27 01:07:32,919][105692] Updated weights for policy 0, policy_version 1336847 (0.0009) [2023-12-27 01:07:33,014][105620] Updated weights for policy 1, policy_version 1338755 (0.0009) [2023-12-27 01:07:33,068][105620] Updated weights for policy 1, policy_version 1338765 (0.0009) [2023-12-27 01:07:33,125][105620] Updated weights for policy 1, policy_version 1338775 (0.0009) [2023-12-27 01:07:33,616][105692] Updated weights for policy 0, policy_version 1336857 (0.0009) [2023-12-27 01:07:33,676][105692] Updated weights for policy 0, policy_version 1336867 (0.0009) [2023-12-27 01:07:33,723][105692] Updated weights for policy 0, policy_version 1336877 (0.0009) [2023-12-27 01:07:33,856][105620] Updated weights for policy 1, policy_version 1338785 (0.0008) [2023-12-27 01:07:33,917][105620] Updated weights for policy 1, policy_version 1338795 (0.0008) [2023-12-27 01:07:33,963][105620] Updated weights for policy 1, policy_version 1338805 (0.0006) [2023-12-27 01:07:34,030][105620] Updated weights for policy 1, policy_version 1338815 (0.0005) [2023-12-27 01:07:34,489][105692] Updated weights for policy 0, policy_version 1336887 (0.0009) [2023-12-27 01:07:34,553][105692] Updated weights for policy 0, policy_version 1336897 (0.0007) [2023-12-27 01:07:34,614][105692] Updated weights for policy 0, policy_version 1336907 (0.0008) [2023-12-27 01:07:34,726][105620] Updated weights for policy 1, policy_version 1338825 (0.0006) [2023-12-27 01:07:34,783][105620] Updated weights for policy 1, policy_version 1338835 (0.0005) [2023-12-27 01:07:34,837][105620] Updated weights for policy 1, policy_version 1338845 (0.0005) [2023-12-27 01:07:35,401][105692] Updated weights for policy 0, policy_version 1336917 (0.0009) [2023-12-27 01:07:35,458][105620] Updated weights for policy 1, policy_version 1338855 (0.0007) [2023-12-27 01:07:35,459][105692] Updated weights for policy 0, policy_version 1336927 (0.0007) [2023-12-27 01:07:35,508][105620] Updated weights for policy 1, policy_version 1338865 (0.0007) [2023-12-27 01:07:35,509][105692] Updated weights for policy 0, policy_version 1336937 (0.0007) [2023-12-27 01:07:35,555][105620] Updated weights for policy 1, policy_version 1338875 (0.0007) [2023-12-27 01:07:36,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 685105152. Throughput: 0: 9985.3, 1: 9636.2. Samples: 685095944. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:07:36,062][104569] Avg episode reward: [(0, '8547.074'), (1, '8823.461')] [2023-12-27 01:07:36,256][105620] Updated weights for policy 1, policy_version 1338885 (0.0009) [2023-12-27 01:07:36,275][105692] Updated weights for policy 0, policy_version 1336947 (0.0009) [2023-12-27 01:07:36,309][105620] Updated weights for policy 1, policy_version 1338895 (0.0009) [2023-12-27 01:07:36,337][105692] Updated weights for policy 0, policy_version 1336957 (0.0007) [2023-12-27 01:07:36,367][105620] Updated weights for policy 1, policy_version 1338905 (0.0006) [2023-12-27 01:07:36,395][105692] Updated weights for policy 0, policy_version 1336967 (0.0008) [2023-12-27 01:07:37,029][105620] Updated weights for policy 1, policy_version 1338915 (0.0007) [2023-12-27 01:07:37,088][105620] Updated weights for policy 1, policy_version 1338925 (0.0008) [2023-12-27 01:07:37,139][105620] Updated weights for policy 1, policy_version 1338935 (0.0009) [2023-12-27 01:07:37,202][105692] Updated weights for policy 0, policy_version 1336977 (0.0009) [2023-12-27 01:07:37,253][105692] Updated weights for policy 0, policy_version 1336987 (0.0009) [2023-12-27 01:07:37,316][105692] Updated weights for policy 0, policy_version 1336997 (0.0009) [2023-12-27 01:07:37,374][105692] Updated weights for policy 0, policy_version 1337007 (0.0009) [2023-12-27 01:07:37,810][105620] Updated weights for policy 1, policy_version 1338945 (0.0008) [2023-12-27 01:07:37,862][105620] Updated weights for policy 1, policy_version 1338955 (0.0010) [2023-12-27 01:07:37,912][105620] Updated weights for policy 1, policy_version 1338965 (0.0010) [2023-12-27 01:07:37,965][105620] Updated weights for policy 1, policy_version 1338975 (0.0010) [2023-12-27 01:07:38,190][105692] Updated weights for policy 0, policy_version 1337017 (0.0010) [2023-12-27 01:07:38,243][105692] Updated weights for policy 0, policy_version 1337028 (0.0009) [2023-12-27 01:07:38,305][105692] Updated weights for policy 0, policy_version 1337038 (0.0010) [2023-12-27 01:07:38,591][105620] Updated weights for policy 1, policy_version 1338985 (0.0009) [2023-12-27 01:07:38,646][105620] Updated weights for policy 1, policy_version 1338995 (0.0008) [2023-12-27 01:07:38,706][105620] Updated weights for policy 1, policy_version 1339005 (0.0006) [2023-12-27 01:07:39,126][105692] Updated weights for policy 0, policy_version 1337048 (0.0011) [2023-12-27 01:07:39,181][105692] Updated weights for policy 0, policy_version 1337058 (0.0011) [2023-12-27 01:07:39,241][105692] Updated weights for policy 0, policy_version 1337068 (0.0009) [2023-12-27 01:07:39,436][105620] Updated weights for policy 1, policy_version 1339015 (0.0009) [2023-12-27 01:07:39,491][105620] Updated weights for policy 1, policy_version 1339025 (0.0010) [2023-12-27 01:07:39,550][105620] Updated weights for policy 1, policy_version 1339035 (0.0009) [2023-12-27 01:07:40,013][105692] Updated weights for policy 0, policy_version 1337078 (0.0009) [2023-12-27 01:07:40,065][105692] Updated weights for policy 0, policy_version 1337088 (0.0009) [2023-12-27 01:07:40,127][105692] Updated weights for policy 0, policy_version 1337098 (0.0009) [2023-12-27 01:07:40,330][105620] Updated weights for policy 1, policy_version 1339045 (0.0009) [2023-12-27 01:07:40,385][105620] Updated weights for policy 1, policy_version 1339055 (0.0009) [2023-12-27 01:07:40,436][105620] Updated weights for policy 1, policy_version 1339065 (0.0009) [2023-12-27 01:07:40,896][105692] Updated weights for policy 0, policy_version 1337108 (0.0008) [2023-12-27 01:07:40,962][105692] Updated weights for policy 0, policy_version 1337118 (0.0009) [2023-12-27 01:07:41,017][105692] Updated weights for policy 0, policy_version 1337128 (0.0009) [2023-12-27 01:07:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 685195264. Throughput: 0: 9917.3, 1: 9669.8. Samples: 685209288. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:07:41,062][104569] Avg episode reward: [(0, '8547.775'), (1, '9177.051')] [2023-12-27 01:07:41,204][105620] Updated weights for policy 1, policy_version 1339075 (0.0008) [2023-12-27 01:07:41,268][105620] Updated weights for policy 1, policy_version 1339085 (0.0007) [2023-12-27 01:07:41,322][105620] Updated weights for policy 1, policy_version 1339095 (0.0008) [2023-12-27 01:07:41,799][105692] Updated weights for policy 0, policy_version 1337138 (0.0010) [2023-12-27 01:07:41,859][105692] Updated weights for policy 0, policy_version 1337148 (0.0006) [2023-12-27 01:07:41,922][105692] Updated weights for policy 0, policy_version 1337158 (0.0005) [2023-12-27 01:07:41,986][105692] Updated weights for policy 0, policy_version 1337168 (0.0006) [2023-12-27 01:07:42,162][105620] Updated weights for policy 1, policy_version 1339105 (0.0009) [2023-12-27 01:07:42,217][105620] Updated weights for policy 1, policy_version 1339115 (0.0008) [2023-12-27 01:07:42,281][105620] Updated weights for policy 1, policy_version 1339125 (0.0008) [2023-12-27 01:07:42,347][105620] Updated weights for policy 1, policy_version 1339135 (0.0008) [2023-12-27 01:07:42,647][105692] Updated weights for policy 0, policy_version 1337178 (0.0009) [2023-12-27 01:07:42,709][105692] Updated weights for policy 0, policy_version 1337188 (0.0009) [2023-12-27 01:07:42,760][105692] Updated weights for policy 0, policy_version 1337198 (0.0006) [2023-12-27 01:07:43,113][105620] Updated weights for policy 1, policy_version 1339145 (0.0009) [2023-12-27 01:07:43,170][105620] Updated weights for policy 1, policy_version 1339155 (0.0009) [2023-12-27 01:07:43,223][105620] Updated weights for policy 1, policy_version 1339165 (0.0009) [2023-12-27 01:07:43,444][105692] Updated weights for policy 0, policy_version 1337208 (0.0007) [2023-12-27 01:07:43,508][105692] Updated weights for policy 0, policy_version 1337218 (0.0005) [2023-12-27 01:07:43,563][105692] Updated weights for policy 0, policy_version 1337228 (0.0005) [2023-12-27 01:07:44,086][105620] Updated weights for policy 1, policy_version 1339175 (0.0008) [2023-12-27 01:07:44,112][105692] Updated weights for policy 0, policy_version 1337238 (0.0006) [2023-12-27 01:07:44,142][105620] Updated weights for policy 1, policy_version 1339185 (0.0007) [2023-12-27 01:07:44,169][105692] Updated weights for policy 0, policy_version 1337248 (0.0006) [2023-12-27 01:07:44,198][105620] Updated weights for policy 1, policy_version 1339195 (0.0008) [2023-12-27 01:07:44,228][105692] Updated weights for policy 0, policy_version 1337258 (0.0007) [2023-12-27 01:07:44,964][105620] Updated weights for policy 1, policy_version 1339205 (0.0008) [2023-12-27 01:07:44,995][105692] Updated weights for policy 0, policy_version 1337268 (0.0009) [2023-12-27 01:07:45,018][105620] Updated weights for policy 1, policy_version 1339215 (0.0011) [2023-12-27 01:07:45,053][105692] Updated weights for policy 0, policy_version 1337278 (0.0009) [2023-12-27 01:07:45,075][105620] Updated weights for policy 1, policy_version 1339225 (0.0011) [2023-12-27 01:07:45,118][105692] Updated weights for policy 0, policy_version 1337288 (0.0006) [2023-12-27 01:07:45,812][105692] Updated weights for policy 0, policy_version 1337298 (0.0009) [2023-12-27 01:07:45,815][105620] Updated weights for policy 1, policy_version 1339235 (0.0009) [2023-12-27 01:07:45,863][105692] Updated weights for policy 0, policy_version 1337308 (0.0005) [2023-12-27 01:07:45,874][105620] Updated weights for policy 1, policy_version 1339245 (0.0005) [2023-12-27 01:07:45,912][105692] Updated weights for policy 0, policy_version 1337318 (0.0005) [2023-12-27 01:07:45,939][105620] Updated weights for policy 1, policy_version 1339255 (0.0008) [2023-12-27 01:07:45,968][105692] Updated weights for policy 0, policy_version 1337328 (0.0006) [2023-12-27 01:07:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 685301760. Throughput: 0: 9861.5, 1: 9604.7. Samples: 685264660. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:07:46,063][104569] Avg episode reward: [(0, '8461.461'), (1, '9268.399')] [2023-12-27 01:07:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001339264_342892544.pth... [2023-12-27 01:07:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001337328_342409216.pth... [2023-12-27 01:07:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001338144_342605824.pth [2023-12-27 01:07:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001336176_342114304.pth [2023-12-27 01:07:46,608][105620] Updated weights for policy 1, policy_version 1339265 (0.0008) [2023-12-27 01:07:46,672][105620] Updated weights for policy 1, policy_version 1339275 (0.0008) [2023-12-27 01:07:46,725][105620] Updated weights for policy 1, policy_version 1339285 (0.0007) [2023-12-27 01:07:46,730][105692] Updated weights for policy 0, policy_version 1337338 (0.0009) [2023-12-27 01:07:46,780][105692] Updated weights for policy 0, policy_version 1337348 (0.0006) [2023-12-27 01:07:46,782][105620] Updated weights for policy 1, policy_version 1339295 (0.0007) [2023-12-27 01:07:46,833][105692] Updated weights for policy 0, policy_version 1337358 (0.0008) [2023-12-27 01:07:47,470][105692] Updated weights for policy 0, policy_version 1337368 (0.0006) [2023-12-27 01:07:47,532][105692] Updated weights for policy 0, policy_version 1337378 (0.0005) [2023-12-27 01:07:47,585][105692] Updated weights for policy 0, policy_version 1337388 (0.0006) [2023-12-27 01:07:47,588][105620] Updated weights for policy 1, policy_version 1339305 (0.0006) [2023-12-27 01:07:47,639][105620] Updated weights for policy 1, policy_version 1339315 (0.0005) [2023-12-27 01:07:47,690][105620] Updated weights for policy 1, policy_version 1339325 (0.0005) [2023-12-27 01:07:48,254][105692] Updated weights for policy 0, policy_version 1337398 (0.0008) [2023-12-27 01:07:48,256][105620] Updated weights for policy 1, policy_version 1339335 (0.0006) [2023-12-27 01:07:48,307][105620] Updated weights for policy 1, policy_version 1339345 (0.0007) [2023-12-27 01:07:48,309][105692] Updated weights for policy 0, policy_version 1337408 (0.0010) [2023-12-27 01:07:48,371][105620] Updated weights for policy 1, policy_version 1339355 (0.0008) [2023-12-27 01:07:48,378][105692] Updated weights for policy 0, policy_version 1337418 (0.0009) [2023-12-27 01:07:49,057][105692] Updated weights for policy 0, policy_version 1337428 (0.0011) [2023-12-27 01:07:49,115][105692] Updated weights for policy 0, policy_version 1337438 (0.0009) [2023-12-27 01:07:49,122][105620] Updated weights for policy 1, policy_version 1339365 (0.0008) [2023-12-27 01:07:49,175][105692] Updated weights for policy 0, policy_version 1337448 (0.0009) [2023-12-27 01:07:49,177][105620] Updated weights for policy 1, policy_version 1339375 (0.0007) [2023-12-27 01:07:49,233][105620] Updated weights for policy 1, policy_version 1339385 (0.0007) [2023-12-27 01:07:49,849][105692] Updated weights for policy 0, policy_version 1337458 (0.0010) [2023-12-27 01:07:49,911][105692] Updated weights for policy 0, policy_version 1337468 (0.0007) [2023-12-27 01:07:49,967][105692] Updated weights for policy 0, policy_version 1337478 (0.0011) [2023-12-27 01:07:50,007][105620] Updated weights for policy 1, policy_version 1339395 (0.0009) [2023-12-27 01:07:50,028][105692] Updated weights for policy 0, policy_version 1337488 (0.0008) [2023-12-27 01:07:50,060][105620] Updated weights for policy 1, policy_version 1339405 (0.0011) [2023-12-27 01:07:50,114][105620] Updated weights for policy 1, policy_version 1339415 (0.0011) [2023-12-27 01:07:50,653][105692] Updated weights for policy 0, policy_version 1337498 (0.0011) [2023-12-27 01:07:50,713][105692] Updated weights for policy 0, policy_version 1337508 (0.0011) [2023-12-27 01:07:50,773][105692] Updated weights for policy 0, policy_version 1337518 (0.0011) [2023-12-27 01:07:50,889][105620] Updated weights for policy 1, policy_version 1339425 (0.0011) [2023-12-27 01:07:50,955][105620] Updated weights for policy 1, policy_version 1339435 (0.0011) [2023-12-27 01:07:51,007][105620] Updated weights for policy 1, policy_version 1339445 (0.0010) [2023-12-27 01:07:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 685391872. Throughput: 0: 9909.4, 1: 9619.8. Samples: 685383024. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:07:51,063][104569] Avg episode reward: [(0, '7451.964'), (1, '9267.555')] [2023-12-27 01:07:51,069][105620] Updated weights for policy 1, policy_version 1339455 (0.0009) [2023-12-27 01:07:51,468][105692] Updated weights for policy 0, policy_version 1337528 (0.0010) [2023-12-27 01:07:51,526][105692] Updated weights for policy 0, policy_version 1337538 (0.0010) [2023-12-27 01:07:51,578][105692] Updated weights for policy 0, policy_version 1337548 (0.0010) [2023-12-27 01:07:51,838][105620] Updated weights for policy 1, policy_version 1339465 (0.0008) [2023-12-27 01:07:51,903][105620] Updated weights for policy 1, policy_version 1339475 (0.0008) [2023-12-27 01:07:51,971][105620] Updated weights for policy 1, policy_version 1339485 (0.0008) [2023-12-27 01:07:52,367][105692] Updated weights for policy 0, policy_version 1337558 (0.0009) [2023-12-27 01:07:52,415][105692] Updated weights for policy 0, policy_version 1337568 (0.0009) [2023-12-27 01:07:52,481][105692] Updated weights for policy 0, policy_version 1337578 (0.0011) [2023-12-27 01:07:52,661][105620] Updated weights for policy 1, policy_version 1339495 (0.0008) [2023-12-27 01:07:52,716][105620] Updated weights for policy 1, policy_version 1339505 (0.0008) [2023-12-27 01:07:52,764][105620] Updated weights for policy 1, policy_version 1339515 (0.0009) [2023-12-27 01:07:53,154][105692] Updated weights for policy 0, policy_version 1337588 (0.0008) [2023-12-27 01:07:53,212][105692] Updated weights for policy 0, policy_version 1337598 (0.0007) [2023-12-27 01:07:53,275][105692] Updated weights for policy 0, policy_version 1337608 (0.0008) [2023-12-27 01:07:53,538][105620] Updated weights for policy 1, policy_version 1339525 (0.0011) [2023-12-27 01:07:53,602][105620] Updated weights for policy 1, policy_version 1339535 (0.0009) [2023-12-27 01:07:53,650][105620] Updated weights for policy 1, policy_version 1339545 (0.0006) [2023-12-27 01:07:53,975][105692] Updated weights for policy 0, policy_version 1337618 (0.0006) [2023-12-27 01:07:54,039][105692] Updated weights for policy 0, policy_version 1337628 (0.0007) [2023-12-27 01:07:54,102][105692] Updated weights for policy 0, policy_version 1337638 (0.0010) [2023-12-27 01:07:54,162][105692] Updated weights for policy 0, policy_version 1337648 (0.0008) [2023-12-27 01:07:54,310][105620] Updated weights for policy 1, policy_version 1339555 (0.0008) [2023-12-27 01:07:54,369][105620] Updated weights for policy 1, policy_version 1339565 (0.0006) [2023-12-27 01:07:54,438][105620] Updated weights for policy 1, policy_version 1339575 (0.0005) [2023-12-27 01:07:54,765][105692] Updated weights for policy 0, policy_version 1337658 (0.0005) [2023-12-27 01:07:54,833][105692] Updated weights for policy 0, policy_version 1337668 (0.0008) [2023-12-27 01:07:54,889][105692] Updated weights for policy 0, policy_version 1337678 (0.0007) [2023-12-27 01:07:55,259][105620] Updated weights for policy 1, policy_version 1339585 (0.0009) [2023-12-27 01:07:55,315][105620] Updated weights for policy 1, policy_version 1339595 (0.0009) [2023-12-27 01:07:55,371][105620] Updated weights for policy 1, policy_version 1339605 (0.0009) [2023-12-27 01:07:55,406][105692] Updated weights for policy 0, policy_version 1337688 (0.0008) [2023-12-27 01:07:55,428][105620] Updated weights for policy 1, policy_version 1339615 (0.0008) [2023-12-27 01:07:55,461][105692] Updated weights for policy 0, policy_version 1337698 (0.0011) [2023-12-27 01:07:55,522][105692] Updated weights for policy 0, policy_version 1337708 (0.0011) [2023-12-27 01:07:56,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 685490176. Throughput: 0: 9914.0, 1: 9582.8. Samples: 685501456. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:07:56,062][104569] Avg episode reward: [(0, '8101.499'), (1, '9088.323')] [2023-12-27 01:07:56,099][105620] Updated weights for policy 1, policy_version 1339625 (0.0006) [2023-12-27 01:07:56,152][105620] Updated weights for policy 1, policy_version 1339635 (0.0009) [2023-12-27 01:07:56,170][105692] Updated weights for policy 0, policy_version 1337718 (0.0008) [2023-12-27 01:07:56,209][105620] Updated weights for policy 1, policy_version 1339645 (0.0008) [2023-12-27 01:07:56,227][105692] Updated weights for policy 0, policy_version 1337728 (0.0006) [2023-12-27 01:07:56,282][105692] Updated weights for policy 0, policy_version 1337738 (0.0009) [2023-12-27 01:07:56,981][105620] Updated weights for policy 1, policy_version 1339655 (0.0005) [2023-12-27 01:07:56,996][105692] Updated weights for policy 0, policy_version 1337748 (0.0007) [2023-12-27 01:07:57,034][105620] Updated weights for policy 1, policy_version 1339665 (0.0008) [2023-12-27 01:07:57,056][105692] Updated weights for policy 0, policy_version 1337758 (0.0007) [2023-12-27 01:07:57,087][105620] Updated weights for policy 1, policy_version 1339675 (0.0006) [2023-12-27 01:07:57,111][105692] Updated weights for policy 0, policy_version 1337768 (0.0007) [2023-12-27 01:07:57,703][105620] Updated weights for policy 1, policy_version 1339685 (0.0009) [2023-12-27 01:07:57,769][105620] Updated weights for policy 1, policy_version 1339695 (0.0010) [2023-12-27 01:07:57,816][105620] Updated weights for policy 1, policy_version 1339705 (0.0009) [2023-12-27 01:07:57,825][105692] Updated weights for policy 0, policy_version 1337778 (0.0009) [2023-12-27 01:07:57,871][105692] Updated weights for policy 0, policy_version 1337788 (0.0007) [2023-12-27 01:07:57,918][105692] Updated weights for policy 0, policy_version 1337798 (0.0008) [2023-12-27 01:07:57,976][105692] Updated weights for policy 0, policy_version 1337808 (0.0008) [2023-12-27 01:07:58,676][105620] Updated weights for policy 1, policy_version 1339715 (0.0009) [2023-12-27 01:07:58,708][105692] Updated weights for policy 0, policy_version 1337818 (0.0011) [2023-12-27 01:07:58,739][105620] Updated weights for policy 1, policy_version 1339725 (0.0010) [2023-12-27 01:07:58,772][105692] Updated weights for policy 0, policy_version 1337828 (0.0011) [2023-12-27 01:07:58,804][105620] Updated weights for policy 1, policy_version 1339735 (0.0008) [2023-12-27 01:07:58,838][105692] Updated weights for policy 0, policy_version 1337838 (0.0011) [2023-12-27 01:07:59,553][105620] Updated weights for policy 1, policy_version 1339745 (0.0008) [2023-12-27 01:07:59,557][105692] Updated weights for policy 0, policy_version 1337848 (0.0010) [2023-12-27 01:07:59,613][105620] Updated weights for policy 1, policy_version 1339755 (0.0011) [2023-12-27 01:07:59,619][105692] Updated weights for policy 0, policy_version 1337858 (0.0011) [2023-12-27 01:07:59,665][105620] Updated weights for policy 1, policy_version 1339765 (0.0011) [2023-12-27 01:07:59,666][105692] Updated weights for policy 0, policy_version 1337868 (0.0010) [2023-12-27 01:07:59,721][105620] Updated weights for policy 1, policy_version 1339775 (0.0011) [2023-12-27 01:08:00,360][105620] Updated weights for policy 1, policy_version 1339785 (0.0008) [2023-12-27 01:08:00,418][105620] Updated weights for policy 1, policy_version 1339795 (0.0007) [2023-12-27 01:08:00,424][105692] Updated weights for policy 0, policy_version 1337878 (0.0010) [2023-12-27 01:08:00,486][105620] Updated weights for policy 1, policy_version 1339805 (0.0005) [2023-12-27 01:08:00,492][105692] Updated weights for policy 0, policy_version 1337888 (0.0011) [2023-12-27 01:08:00,552][105692] Updated weights for policy 0, policy_version 1337898 (0.0010) [2023-12-27 01:08:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 685588480. Throughput: 0: 9958.1, 1: 9565.9. Samples: 685560076. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:08:01,063][104569] Avg episode reward: [(0, '8456.953'), (1, '9000.374')] [2023-12-27 01:08:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001337904_342556672.pth... [2023-12-27 01:08:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001339808_343031808.pth... [2023-12-27 01:08:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001336784_342269952.pth [2023-12-27 01:08:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001338688_342745088.pth [2023-12-27 01:08:01,140][105692] Updated weights for policy 0, policy_version 1337908 (0.0006) [2023-12-27 01:08:01,172][105620] Updated weights for policy 1, policy_version 1339815 (0.0007) [2023-12-27 01:08:01,204][105692] Updated weights for policy 0, policy_version 1337918 (0.0008) [2023-12-27 01:08:01,228][105620] Updated weights for policy 1, policy_version 1339825 (0.0005) [2023-12-27 01:08:01,259][105692] Updated weights for policy 0, policy_version 1337928 (0.0008) [2023-12-27 01:08:01,289][105620] Updated weights for policy 1, policy_version 1339835 (0.0007) [2023-12-27 01:08:01,975][105692] Updated weights for policy 0, policy_version 1337938 (0.0007) [2023-12-27 01:08:02,041][105692] Updated weights for policy 0, policy_version 1337948 (0.0008) [2023-12-27 01:08:02,046][105620] Updated weights for policy 1, policy_version 1339845 (0.0006) [2023-12-27 01:08:02,106][105692] Updated weights for policy 0, policy_version 1337958 (0.0006) [2023-12-27 01:08:02,107][105620] Updated weights for policy 1, policy_version 1339855 (0.0007) [2023-12-27 01:08:02,166][105620] Updated weights for policy 1, policy_version 1339865 (0.0008) [2023-12-27 01:08:02,170][105692] Updated weights for policy 0, policy_version 1337968 (0.0006) [2023-12-27 01:08:02,823][105620] Updated weights for policy 1, policy_version 1339875 (0.0006) [2023-12-27 01:08:02,845][105692] Updated weights for policy 0, policy_version 1337978 (0.0008) [2023-12-27 01:08:02,877][105620] Updated weights for policy 1, policy_version 1339885 (0.0005) [2023-12-27 01:08:02,903][105692] Updated weights for policy 0, policy_version 1337988 (0.0009) [2023-12-27 01:08:02,929][105620] Updated weights for policy 1, policy_version 1339895 (0.0005) [2023-12-27 01:08:02,960][105692] Updated weights for policy 0, policy_version 1337998 (0.0008) [2023-12-27 01:08:03,495][105620] Updated weights for policy 1, policy_version 1339905 (0.0006) [2023-12-27 01:08:03,540][105620] Updated weights for policy 1, policy_version 1339915 (0.0005) [2023-12-27 01:08:03,592][105620] Updated weights for policy 1, policy_version 1339925 (0.0005) [2023-12-27 01:08:03,651][105620] Updated weights for policy 1, policy_version 1339935 (0.0006) [2023-12-27 01:08:03,779][105692] Updated weights for policy 0, policy_version 1338008 (0.0006) [2023-12-27 01:08:03,846][105692] Updated weights for policy 0, policy_version 1338018 (0.0007) [2023-12-27 01:08:03,896][105692] Updated weights for policy 0, policy_version 1338028 (0.0008) [2023-12-27 01:08:04,226][105620] Updated weights for policy 1, policy_version 1339945 (0.0010) [2023-12-27 01:08:04,285][105620] Updated weights for policy 1, policy_version 1339955 (0.0010) [2023-12-27 01:08:04,343][105620] Updated weights for policy 1, policy_version 1339965 (0.0009) [2023-12-27 01:08:04,553][105692] Updated weights for policy 0, policy_version 1338038 (0.0006) [2023-12-27 01:08:04,612][105692] Updated weights for policy 0, policy_version 1338048 (0.0006) [2023-12-27 01:08:04,682][105692] Updated weights for policy 0, policy_version 1338058 (0.0009) [2023-12-27 01:08:05,005][105620] Updated weights for policy 1, policy_version 1339975 (0.0009) [2023-12-27 01:08:05,056][105620] Updated weights for policy 1, policy_version 1339985 (0.0009) [2023-12-27 01:08:05,118][105620] Updated weights for policy 1, policy_version 1339995 (0.0008) [2023-12-27 01:08:05,328][105692] Updated weights for policy 0, policy_version 1338068 (0.0008) [2023-12-27 01:08:05,382][105692] Updated weights for policy 0, policy_version 1338078 (0.0006) [2023-12-27 01:08:05,427][105692] Updated weights for policy 0, policy_version 1338088 (0.0006) [2023-12-27 01:08:05,940][105620] Updated weights for policy 1, policy_version 1340005 (0.0007) [2023-12-27 01:08:06,001][105620] Updated weights for policy 1, policy_version 1340015 (0.0007) [2023-12-27 01:08:06,055][105692] Updated weights for policy 0, policy_version 1338098 (0.0008) [2023-12-27 01:08:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 685686784. Throughput: 0: 9868.5, 1: 9655.9. Samples: 685681836. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:08:06,062][104569] Avg episode reward: [(0, '8637.185'), (1, '8915.505')] [2023-12-27 01:08:06,065][105620] Updated weights for policy 1, policy_version 1340025 (0.0009) [2023-12-27 01:08:06,113][105692] Updated weights for policy 0, policy_version 1338108 (0.0007) [2023-12-27 01:08:06,163][105585] KL-divergence is very high: 146.4473 [2023-12-27 01:08:06,175][105692] Updated weights for policy 0, policy_version 1338118 (0.0009) [2023-12-27 01:08:06,208][105585] KL-divergence is very high: 248.3905 [2023-12-27 01:08:06,229][105692] Updated weights for policy 0, policy_version 1338128 (0.0008) [2023-12-27 01:08:06,838][105692] Updated weights for policy 0, policy_version 1338138 (0.0009) [2023-12-27 01:08:06,869][105620] Updated weights for policy 1, policy_version 1340035 (0.0008) [2023-12-27 01:08:06,892][105692] Updated weights for policy 0, policy_version 1338148 (0.0007) [2023-12-27 01:08:06,928][105620] Updated weights for policy 1, policy_version 1340045 (0.0008) [2023-12-27 01:08:06,951][105692] Updated weights for policy 0, policy_version 1338158 (0.0006) [2023-12-27 01:08:06,991][105620] Updated weights for policy 1, policy_version 1340055 (0.0008) [2023-12-27 01:08:07,693][105692] Updated weights for policy 0, policy_version 1338168 (0.0009) [2023-12-27 01:08:07,745][105620] Updated weights for policy 1, policy_version 1340065 (0.0008) [2023-12-27 01:08:07,753][105692] Updated weights for policy 0, policy_version 1338178 (0.0009) [2023-12-27 01:08:07,804][105620] Updated weights for policy 1, policy_version 1340075 (0.0006) [2023-12-27 01:08:07,814][105692] Updated weights for policy 0, policy_version 1338188 (0.0007) [2023-12-27 01:08:07,868][105620] Updated weights for policy 1, policy_version 1340085 (0.0005) [2023-12-27 01:08:07,918][105620] Updated weights for policy 1, policy_version 1340095 (0.0005) [2023-12-27 01:08:08,589][105692] Updated weights for policy 0, policy_version 1338198 (0.0008) [2023-12-27 01:08:08,600][105620] Updated weights for policy 1, policy_version 1340105 (0.0008) [2023-12-27 01:08:08,652][105620] Updated weights for policy 1, policy_version 1340115 (0.0005) [2023-12-27 01:08:08,654][105692] Updated weights for policy 0, policy_version 1338208 (0.0008) [2023-12-27 01:08:08,711][105620] Updated weights for policy 1, policy_version 1340125 (0.0009) [2023-12-27 01:08:08,717][105692] Updated weights for policy 0, policy_version 1338218 (0.0009) [2023-12-27 01:08:09,413][105620] Updated weights for policy 1, policy_version 1340135 (0.0009) [2023-12-27 01:08:09,470][105620] Updated weights for policy 1, policy_version 1340145 (0.0009) [2023-12-27 01:08:09,530][105620] Updated weights for policy 1, policy_version 1340155 (0.0009) [2023-12-27 01:08:09,541][105692] Updated weights for policy 0, policy_version 1338228 (0.0009) [2023-12-27 01:08:09,597][105692] Updated weights for policy 0, policy_version 1338238 (0.0008) [2023-12-27 01:08:09,663][105692] Updated weights for policy 0, policy_version 1338248 (0.0008) [2023-12-27 01:08:10,324][105620] Updated weights for policy 1, policy_version 1340165 (0.0008) [2023-12-27 01:08:10,372][105692] Updated weights for policy 0, policy_version 1338258 (0.0006) [2023-12-27 01:08:10,386][105620] Updated weights for policy 1, policy_version 1340175 (0.0008) [2023-12-27 01:08:10,435][105692] Updated weights for policy 0, policy_version 1338268 (0.0008) [2023-12-27 01:08:10,443][105620] Updated weights for policy 1, policy_version 1340185 (0.0006) [2023-12-27 01:08:10,494][105692] Updated weights for policy 0, policy_version 1338278 (0.0008) [2023-12-27 01:08:10,553][105692] Updated weights for policy 0, policy_version 1338288 (0.0008) [2023-12-27 01:08:11,014][105620] Updated weights for policy 1, policy_version 1340195 (0.0006) [2023-12-27 01:08:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 685785088. Throughput: 0: 9872.5, 1: 9611.8. Samples: 685797076. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:08:11,062][104569] Avg episode reward: [(0, '8359.088'), (1, '8999.822')] [2023-12-27 01:08:11,080][105620] Updated weights for policy 1, policy_version 1340205 (0.0008) [2023-12-27 01:08:11,155][105620] Updated weights for policy 1, policy_version 1340215 (0.0009) [2023-12-27 01:08:11,382][105692] Updated weights for policy 0, policy_version 1338298 (0.0008) [2023-12-27 01:08:11,437][105692] Updated weights for policy 0, policy_version 1338308 (0.0008) [2023-12-27 01:08:11,491][105692] Updated weights for policy 0, policy_version 1338318 (0.0006) [2023-12-27 01:08:11,977][105620] Updated weights for policy 1, policy_version 1340225 (0.0008) [2023-12-27 01:08:12,046][105620] Updated weights for policy 1, policy_version 1340235 (0.0009) [2023-12-27 01:08:12,108][105620] Updated weights for policy 1, policy_version 1340245 (0.0009) [2023-12-27 01:08:12,171][105620] Updated weights for policy 1, policy_version 1340255 (0.0009) [2023-12-27 01:08:12,177][105692] Updated weights for policy 0, policy_version 1338328 (0.0007) [2023-12-27 01:08:12,235][105692] Updated weights for policy 0, policy_version 1338338 (0.0009) [2023-12-27 01:08:12,293][105692] Updated weights for policy 0, policy_version 1338348 (0.0009) [2023-12-27 01:08:12,867][105620] Updated weights for policy 1, policy_version 1340265 (0.0006) [2023-12-27 01:08:12,929][105620] Updated weights for policy 1, policy_version 1340275 (0.0005) [2023-12-27 01:08:13,006][105620] Updated weights for policy 1, policy_version 1340285 (0.0007) [2023-12-27 01:08:13,150][105692] Updated weights for policy 0, policy_version 1338358 (0.0009) [2023-12-27 01:08:13,204][105692] Updated weights for policy 0, policy_version 1338368 (0.0010) [2023-12-27 01:08:13,262][105692] Updated weights for policy 0, policy_version 1338378 (0.0010) [2023-12-27 01:08:13,577][105620] Updated weights for policy 1, policy_version 1340295 (0.0010) [2023-12-27 01:08:13,624][105620] Updated weights for policy 1, policy_version 1340305 (0.0006) [2023-12-27 01:08:13,673][105620] Updated weights for policy 1, policy_version 1340315 (0.0005) [2023-12-27 01:08:14,165][105692] Updated weights for policy 0, policy_version 1338389 (0.0010) [2023-12-27 01:08:14,219][105620] Updated weights for policy 1, policy_version 1340325 (0.0006) [2023-12-27 01:08:14,219][105692] Updated weights for policy 0, policy_version 1338399 (0.0008) [2023-12-27 01:08:14,272][105692] Updated weights for policy 0, policy_version 1338409 (0.0008) [2023-12-27 01:08:14,286][105620] Updated weights for policy 1, policy_version 1340335 (0.0008) [2023-12-27 01:08:14,348][105620] Updated weights for policy 1, policy_version 1340345 (0.0008) [2023-12-27 01:08:15,007][105620] Updated weights for policy 1, policy_version 1340355 (0.0008) [2023-12-27 01:08:15,058][105620] Updated weights for policy 1, policy_version 1340365 (0.0008) [2023-12-27 01:08:15,094][105692] Updated weights for policy 0, policy_version 1338419 (0.0007) [2023-12-27 01:08:15,117][105620] Updated weights for policy 1, policy_version 1340375 (0.0007) [2023-12-27 01:08:15,156][105692] Updated weights for policy 0, policy_version 1338429 (0.0008) [2023-12-27 01:08:15,218][105692] Updated weights for policy 0, policy_version 1338439 (0.0009) [2023-12-27 01:08:15,847][105620] Updated weights for policy 1, policy_version 1340385 (0.0007) [2023-12-27 01:08:15,912][105620] Updated weights for policy 1, policy_version 1340395 (0.0010) [2023-12-27 01:08:15,942][105692] Updated weights for policy 0, policy_version 1338449 (0.0009) [2023-12-27 01:08:15,967][105620] Updated weights for policy 1, policy_version 1340405 (0.0006) [2023-12-27 01:08:16,002][105692] Updated weights for policy 0, policy_version 1338459 (0.0009) [2023-12-27 01:08:16,026][105620] Updated weights for policy 1, policy_version 1340415 (0.0007) [2023-12-27 01:08:16,056][105692] Updated weights for policy 0, policy_version 1338469 (0.0009) [2023-12-27 01:08:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 685883392. Throughput: 0: 9741.1, 1: 9709.2. Samples: 685853932. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:08:16,062][104569] Avg episode reward: [(0, '8100.673'), (1, '9007.657')] [2023-12-27 01:08:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001340416_343187456.pth... [2023-12-27 01:08:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001339264_342892544.pth [2023-12-27 01:08:16,115][105692] Updated weights for policy 0, policy_version 1338479 (0.0009) [2023-12-27 01:08:16,120][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001338480_342704128.pth... [2023-12-27 01:08:16,125][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001337328_342409216.pth [2023-12-27 01:08:16,745][105620] Updated weights for policy 1, policy_version 1340425 (0.0006) [2023-12-27 01:08:16,794][105620] Updated weights for policy 1, policy_version 1340435 (0.0009) [2023-12-27 01:08:16,846][105620] Updated weights for policy 1, policy_version 1340445 (0.0009) [2023-12-27 01:08:16,906][105692] Updated weights for policy 0, policy_version 1338489 (0.0008) [2023-12-27 01:08:16,953][105692] Updated weights for policy 0, policy_version 1338499 (0.0009) [2023-12-27 01:08:17,000][105692] Updated weights for policy 0, policy_version 1338509 (0.0009) [2023-12-27 01:08:17,578][105620] Updated weights for policy 1, policy_version 1340455 (0.0009) [2023-12-27 01:08:17,624][105620] Updated weights for policy 1, policy_version 1340465 (0.0008) [2023-12-27 01:08:17,681][105620] Updated weights for policy 1, policy_version 1340475 (0.0009) [2023-12-27 01:08:17,799][105692] Updated weights for policy 0, policy_version 1338519 (0.0009) [2023-12-27 01:08:17,859][105692] Updated weights for policy 0, policy_version 1338529 (0.0009) [2023-12-27 01:08:17,921][105692] Updated weights for policy 0, policy_version 1338539 (0.0009) [2023-12-27 01:08:18,338][105620] Updated weights for policy 1, policy_version 1340485 (0.0008) [2023-12-27 01:08:18,406][105620] Updated weights for policy 1, policy_version 1340495 (0.0008) [2023-12-27 01:08:18,475][105620] Updated weights for policy 1, policy_version 1340505 (0.0009) [2023-12-27 01:08:18,756][105692] Updated weights for policy 0, policy_version 1338549 (0.0009) [2023-12-27 01:08:18,807][105692] Updated weights for policy 0, policy_version 1338559 (0.0009) [2023-12-27 01:08:18,863][105692] Updated weights for policy 0, policy_version 1338569 (0.0010) [2023-12-27 01:08:19,071][105620] Updated weights for policy 1, policy_version 1340515 (0.0007) [2023-12-27 01:08:19,116][105620] Updated weights for policy 1, policy_version 1340525 (0.0006) [2023-12-27 01:08:19,173][105620] Updated weights for policy 1, policy_version 1340535 (0.0005) [2023-12-27 01:08:19,692][105692] Updated weights for policy 0, policy_version 1338579 (0.0009) [2023-12-27 01:08:19,760][105692] Updated weights for policy 0, policy_version 1338589 (0.0009) [2023-12-27 01:08:19,825][105692] Updated weights for policy 0, policy_version 1338599 (0.0009) [2023-12-27 01:08:19,923][105620] Updated weights for policy 1, policy_version 1340545 (0.0007) [2023-12-27 01:08:19,991][105620] Updated weights for policy 1, policy_version 1340555 (0.0010) [2023-12-27 01:08:20,051][105620] Updated weights for policy 1, policy_version 1340565 (0.0009) [2023-12-27 01:08:20,120][105620] Updated weights for policy 1, policy_version 1340575 (0.0009) [2023-12-27 01:08:20,632][105692] Updated weights for policy 0, policy_version 1338609 (0.0007) [2023-12-27 01:08:20,695][105692] Updated weights for policy 0, policy_version 1338619 (0.0008) [2023-12-27 01:08:20,763][105692] Updated weights for policy 0, policy_version 1338629 (0.0008) [2023-12-27 01:08:20,819][105620] Updated weights for policy 1, policy_version 1340585 (0.0006) [2023-12-27 01:08:20,820][105692] Updated weights for policy 0, policy_version 1338639 (0.0008) [2023-12-27 01:08:20,875][105620] Updated weights for policy 1, policy_version 1340595 (0.0006) [2023-12-27 01:08:20,935][105620] Updated weights for policy 1, policy_version 1340605 (0.0006) [2023-12-27 01:08:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 685981696. Throughput: 0: 9600.3, 1: 9772.3. Samples: 685967708. Policy #0 lag: (min: 29.0, avg: 37.5, max: 61.0) [2023-12-27 01:08:21,063][104569] Avg episode reward: [(0, '8279.975'), (1, '9189.896')] [2023-12-27 01:08:21,578][105692] Updated weights for policy 0, policy_version 1338649 (0.0006) [2023-12-27 01:08:21,640][105692] Updated weights for policy 0, policy_version 1338659 (0.0007) [2023-12-27 01:08:21,680][105620] Updated weights for policy 1, policy_version 1340615 (0.0009) [2023-12-27 01:08:21,699][105692] Updated weights for policy 0, policy_version 1338669 (0.0007) [2023-12-27 01:08:21,747][105620] Updated weights for policy 1, policy_version 1340625 (0.0011) [2023-12-27 01:08:21,811][105620] Updated weights for policy 1, policy_version 1340635 (0.0011) [2023-12-27 01:08:22,478][105692] Updated weights for policy 0, policy_version 1338679 (0.0008) [2023-12-27 01:08:22,530][105692] Updated weights for policy 0, policy_version 1338689 (0.0008) [2023-12-27 01:08:22,584][105620] Updated weights for policy 1, policy_version 1340645 (0.0011) [2023-12-27 01:08:22,586][105692] Updated weights for policy 0, policy_version 1338699 (0.0007) [2023-12-27 01:08:22,643][105620] Updated weights for policy 1, policy_version 1340655 (0.0011) [2023-12-27 01:08:22,703][105620] Updated weights for policy 1, policy_version 1340665 (0.0011) [2023-12-27 01:08:23,365][105692] Updated weights for policy 0, policy_version 1338709 (0.0008) [2023-12-27 01:08:23,420][105692] Updated weights for policy 0, policy_version 1338719 (0.0010) [2023-12-27 01:08:23,463][105620] Updated weights for policy 1, policy_version 1340675 (0.0010) [2023-12-27 01:08:23,478][105692] Updated weights for policy 0, policy_version 1338729 (0.0010) [2023-12-27 01:08:23,521][105620] Updated weights for policy 1, policy_version 1340685 (0.0010) [2023-12-27 01:08:23,576][105620] Updated weights for policy 1, policy_version 1340695 (0.0010) [2023-12-27 01:08:24,123][105692] Updated weights for policy 0, policy_version 1338739 (0.0009) [2023-12-27 01:08:24,153][105620] Updated weights for policy 1, policy_version 1340705 (0.0010) [2023-12-27 01:08:24,183][105692] Updated weights for policy 0, policy_version 1338749 (0.0009) [2023-12-27 01:08:24,214][105620] Updated weights for policy 1, policy_version 1340715 (0.0006) [2023-12-27 01:08:24,246][105692] Updated weights for policy 0, policy_version 1338759 (0.0011) [2023-12-27 01:08:24,279][105620] Updated weights for policy 1, policy_version 1340725 (0.0007) [2023-12-27 01:08:24,343][105620] Updated weights for policy 1, policy_version 1340735 (0.0011) [2023-12-27 01:08:25,007][105692] Updated weights for policy 0, policy_version 1338769 (0.0010) [2023-12-27 01:08:25,051][105620] Updated weights for policy 1, policy_version 1340745 (0.0009) [2023-12-27 01:08:25,069][105692] Updated weights for policy 0, policy_version 1338779 (0.0009) [2023-12-27 01:08:25,110][105620] Updated weights for policy 1, policy_version 1340755 (0.0007) [2023-12-27 01:08:25,126][105692] Updated weights for policy 0, policy_version 1338789 (0.0008) [2023-12-27 01:08:25,171][105620] Updated weights for policy 1, policy_version 1340765 (0.0008) [2023-12-27 01:08:25,173][105692] Updated weights for policy 0, policy_version 1338799 (0.0007) [2023-12-27 01:08:25,808][105692] Updated weights for policy 0, policy_version 1338809 (0.0010) [2023-12-27 01:08:25,862][105620] Updated weights for policy 1, policy_version 1340775 (0.0010) [2023-12-27 01:08:25,869][105692] Updated weights for policy 0, policy_version 1338819 (0.0010) [2023-12-27 01:08:25,920][105620] Updated weights for policy 1, policy_version 1340785 (0.0009) [2023-12-27 01:08:25,930][105692] Updated weights for policy 0, policy_version 1338829 (0.0010) [2023-12-27 01:08:25,970][105620] Updated weights for policy 1, policy_version 1340795 (0.0008) [2023-12-27 01:08:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 686080000. Throughput: 0: 9663.9, 1: 9732.6. Samples: 686082132. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:08:26,062][104569] Avg episode reward: [(0, '8725.921'), (1, '9085.318')] [2023-12-27 01:08:26,491][105692] Updated weights for policy 0, policy_version 1338839 (0.0006) [2023-12-27 01:08:26,541][105692] Updated weights for policy 0, policy_version 1338849 (0.0010) [2023-12-27 01:08:26,589][105692] Updated weights for policy 0, policy_version 1338859 (0.0010) [2023-12-27 01:08:26,771][105620] Updated weights for policy 1, policy_version 1340806 (0.0008) [2023-12-27 01:08:26,819][105620] Updated weights for policy 1, policy_version 1340816 (0.0009) [2023-12-27 01:08:26,869][105620] Updated weights for policy 1, policy_version 1340827 (0.0006) [2023-12-27 01:08:27,306][105692] Updated weights for policy 0, policy_version 1338869 (0.0010) [2023-12-27 01:08:27,391][105692] Updated weights for policy 0, policy_version 1338879 (0.0010) [2023-12-27 01:08:27,445][105692] Updated weights for policy 0, policy_version 1338889 (0.0010) [2023-12-27 01:08:27,483][105620] Updated weights for policy 1, policy_version 1340837 (0.0005) [2023-12-27 01:08:27,545][105620] Updated weights for policy 1, policy_version 1340847 (0.0006) [2023-12-27 01:08:27,607][105620] Updated weights for policy 1, policy_version 1340857 (0.0008) [2023-12-27 01:08:28,143][105692] Updated weights for policy 0, policy_version 1338899 (0.0009) [2023-12-27 01:08:28,201][105692] Updated weights for policy 0, policy_version 1338909 (0.0005) [2023-12-27 01:08:28,226][105620] Updated weights for policy 1, policy_version 1340867 (0.0007) [2023-12-27 01:08:28,255][105692] Updated weights for policy 0, policy_version 1338919 (0.0009) [2023-12-27 01:08:28,274][105620] Updated weights for policy 1, policy_version 1340877 (0.0006) [2023-12-27 01:08:28,322][105620] Updated weights for policy 1, policy_version 1340887 (0.0006) [2023-12-27 01:08:28,858][105692] Updated weights for policy 0, policy_version 1338929 (0.0007) [2023-12-27 01:08:28,919][105692] Updated weights for policy 0, policy_version 1338939 (0.0010) [2023-12-27 01:08:28,977][105692] Updated weights for policy 0, policy_version 1338949 (0.0010) [2023-12-27 01:08:29,032][105692] Updated weights for policy 0, policy_version 1338959 (0.0010) [2023-12-27 01:08:29,076][105620] Updated weights for policy 1, policy_version 1340897 (0.0008) [2023-12-27 01:08:29,126][105620] Updated weights for policy 1, policy_version 1340907 (0.0008) [2023-12-27 01:08:29,178][105620] Updated weights for policy 1, policy_version 1340917 (0.0008) [2023-12-27 01:08:29,239][105620] Updated weights for policy 1, policy_version 1340927 (0.0007) [2023-12-27 01:08:29,735][105692] Updated weights for policy 0, policy_version 1338969 (0.0009) [2023-12-27 01:08:29,789][105692] Updated weights for policy 0, policy_version 1338980 (0.0010) [2023-12-27 01:08:29,845][105692] Updated weights for policy 0, policy_version 1338990 (0.0009) [2023-12-27 01:08:29,943][105620] Updated weights for policy 1, policy_version 1340937 (0.0008) [2023-12-27 01:08:30,004][105620] Updated weights for policy 1, policy_version 1340947 (0.0009) [2023-12-27 01:08:30,070][105620] Updated weights for policy 1, policy_version 1340957 (0.0009) [2023-12-27 01:08:30,576][105692] Updated weights for policy 0, policy_version 1339000 (0.0006) [2023-12-27 01:08:30,632][105692] Updated weights for policy 0, policy_version 1339010 (0.0005) [2023-12-27 01:08:30,687][105692] Updated weights for policy 0, policy_version 1339020 (0.0008) [2023-12-27 01:08:30,856][105620] Updated weights for policy 1, policy_version 1340967 (0.0009) [2023-12-27 01:08:30,902][105620] Updated weights for policy 1, policy_version 1340977 (0.0008) [2023-12-27 01:08:30,957][105620] Updated weights for policy 1, policy_version 1340988 (0.0009) [2023-12-27 01:08:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 686178304. Throughput: 0: 9710.5, 1: 9833.6. Samples: 686144144. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:08:31,063][104569] Avg episode reward: [(0, '8639.619'), (1, '8993.090')] [2023-12-27 01:08:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001339024_342843392.pth... [2023-12-27 01:08:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001340992_343334912.pth... [2023-12-27 01:08:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001339808_343031808.pth [2023-12-27 01:08:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001337904_342556672.pth [2023-12-27 01:08:31,384][105692] Updated weights for policy 0, policy_version 1339030 (0.0010) [2023-12-27 01:08:31,447][105692] Updated weights for policy 0, policy_version 1339040 (0.0007) [2023-12-27 01:08:31,506][105692] Updated weights for policy 0, policy_version 1339050 (0.0007) [2023-12-27 01:08:31,849][105620] Updated weights for policy 1, policy_version 1340998 (0.0009) [2023-12-27 01:08:31,902][105620] Updated weights for policy 1, policy_version 1341008 (0.0009) [2023-12-27 01:08:31,961][105620] Updated weights for policy 1, policy_version 1341019 (0.0010) [2023-12-27 01:08:32,071][105692] Updated weights for policy 0, policy_version 1339060 (0.0010) [2023-12-27 01:08:32,116][105692] Updated weights for policy 0, policy_version 1339070 (0.0010) [2023-12-27 01:08:32,160][105692] Updated weights for policy 0, policy_version 1339080 (0.0010) [2023-12-27 01:08:32,716][105620] Updated weights for policy 1, policy_version 1341030 (0.0010) [2023-12-27 01:08:32,768][105620] Updated weights for policy 1, policy_version 1341040 (0.0010) [2023-12-27 01:08:32,824][105620] Updated weights for policy 1, policy_version 1341050 (0.0008) [2023-12-27 01:08:32,842][105692] Updated weights for policy 0, policy_version 1339090 (0.0009) [2023-12-27 01:08:32,887][105692] Updated weights for policy 0, policy_version 1339100 (0.0005) [2023-12-27 01:08:32,934][105692] Updated weights for policy 0, policy_version 1339110 (0.0006) [2023-12-27 01:08:32,997][105692] Updated weights for policy 0, policy_version 1339120 (0.0008) [2023-12-27 01:08:33,497][105620] Updated weights for policy 1, policy_version 1341060 (0.0010) [2023-12-27 01:08:33,541][105692] Updated weights for policy 0, policy_version 1339130 (0.0007) [2023-12-27 01:08:33,563][105620] Updated weights for policy 1, policy_version 1341070 (0.0005) [2023-12-27 01:08:33,593][105692] Updated weights for policy 0, policy_version 1339140 (0.0009) [2023-12-27 01:08:33,620][105620] Updated weights for policy 1, policy_version 1341080 (0.0007) [2023-12-27 01:08:33,639][105692] Updated weights for policy 0, policy_version 1339150 (0.0007) [2023-12-27 01:08:34,282][105620] Updated weights for policy 1, policy_version 1341090 (0.0010) [2023-12-27 01:08:34,352][105620] Updated weights for policy 1, policy_version 1341100 (0.0009) [2023-12-27 01:08:34,402][105692] Updated weights for policy 0, policy_version 1339160 (0.0007) [2023-12-27 01:08:34,422][105620] Updated weights for policy 1, policy_version 1341110 (0.0007) [2023-12-27 01:08:34,463][105692] Updated weights for policy 0, policy_version 1339170 (0.0008) [2023-12-27 01:08:34,486][105620] Updated weights for policy 1, policy_version 1341120 (0.0006) [2023-12-27 01:08:34,527][105692] Updated weights for policy 0, policy_version 1339180 (0.0008) [2023-12-27 01:08:35,139][105620] Updated weights for policy 1, policy_version 1341130 (0.0011) [2023-12-27 01:08:35,193][105620] Updated weights for policy 1, policy_version 1341140 (0.0010) [2023-12-27 01:08:35,252][105620] Updated weights for policy 1, policy_version 1341150 (0.0010) [2023-12-27 01:08:35,323][105692] Updated weights for policy 0, policy_version 1339190 (0.0008) [2023-12-27 01:08:35,375][105692] Updated weights for policy 0, policy_version 1339200 (0.0010) [2023-12-27 01:08:35,379][105585] KL-divergence is very high: 100.0449 [2023-12-27 01:08:35,426][105585] KL-divergence is very high: 135.0243 [2023-12-27 01:08:35,432][105692] Updated weights for policy 0, policy_version 1339210 (0.0009) [2023-12-27 01:08:35,872][105620] Updated weights for policy 1, policy_version 1341160 (0.0006) [2023-12-27 01:08:35,941][105620] Updated weights for policy 1, policy_version 1341170 (0.0005) [2023-12-27 01:08:36,006][105620] Updated weights for policy 1, policy_version 1341180 (0.0006) [2023-12-27 01:08:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 686276608. Throughput: 0: 9751.1, 1: 9812.1. Samples: 686263368. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:08:36,062][104569] Avg episode reward: [(0, '8462.184'), (1, '8992.572')] [2023-12-27 01:08:36,305][105692] Updated weights for policy 0, policy_version 1339220 (0.0009) [2023-12-27 01:08:36,364][105692] Updated weights for policy 0, policy_version 1339230 (0.0009) [2023-12-27 01:08:36,426][105692] Updated weights for policy 0, policy_version 1339240 (0.0009) [2023-12-27 01:08:36,602][105620] Updated weights for policy 1, policy_version 1341190 (0.0008) [2023-12-27 01:08:36,656][105620] Updated weights for policy 1, policy_version 1341200 (0.0009) [2023-12-27 01:08:36,709][105620] Updated weights for policy 1, policy_version 1341210 (0.0009) [2023-12-27 01:08:37,127][105692] Updated weights for policy 0, policy_version 1339250 (0.0009) [2023-12-27 01:08:37,190][105692] Updated weights for policy 0, policy_version 1339260 (0.0010) [2023-12-27 01:08:37,242][105692] Updated weights for policy 0, policy_version 1339270 (0.0009) [2023-12-27 01:08:37,294][105692] Updated weights for policy 0, policy_version 1339280 (0.0009) [2023-12-27 01:08:37,431][105620] Updated weights for policy 1, policy_version 1341220 (0.0010) [2023-12-27 01:08:37,493][105620] Updated weights for policy 1, policy_version 1341230 (0.0008) [2023-12-27 01:08:37,564][105620] Updated weights for policy 1, policy_version 1341240 (0.0008) [2023-12-27 01:08:38,106][105692] Updated weights for policy 0, policy_version 1339290 (0.0008) [2023-12-27 01:08:38,175][105692] Updated weights for policy 0, policy_version 1339300 (0.0008) [2023-12-27 01:08:38,222][105692] Updated weights for policy 0, policy_version 1339310 (0.0008) [2023-12-27 01:08:38,309][105620] Updated weights for policy 1, policy_version 1341250 (0.0011) [2023-12-27 01:08:38,379][105620] Updated weights for policy 1, policy_version 1341260 (0.0009) [2023-12-27 01:08:38,440][105620] Updated weights for policy 1, policy_version 1341270 (0.0009) [2023-12-27 01:08:38,503][105620] Updated weights for policy 1, policy_version 1341280 (0.0009) [2023-12-27 01:08:38,959][105692] Updated weights for policy 0, policy_version 1339320 (0.0008) [2023-12-27 01:08:39,023][105692] Updated weights for policy 0, policy_version 1339330 (0.0010) [2023-12-27 01:08:39,089][105692] Updated weights for policy 0, policy_version 1339340 (0.0011) [2023-12-27 01:08:39,288][105620] Updated weights for policy 1, policy_version 1341290 (0.0011) [2023-12-27 01:08:39,356][105620] Updated weights for policy 1, policy_version 1341300 (0.0009) [2023-12-27 01:08:39,421][105620] Updated weights for policy 1, policy_version 1341310 (0.0009) [2023-12-27 01:08:39,801][105692] Updated weights for policy 0, policy_version 1339350 (0.0010) [2023-12-27 01:08:39,871][105692] Updated weights for policy 0, policy_version 1339360 (0.0009) [2023-12-27 01:08:39,934][105692] Updated weights for policy 0, policy_version 1339370 (0.0009) [2023-12-27 01:08:40,178][105620] Updated weights for policy 1, policy_version 1341320 (0.0010) [2023-12-27 01:08:40,242][105620] Updated weights for policy 1, policy_version 1341330 (0.0009) [2023-12-27 01:08:40,304][105620] Updated weights for policy 1, policy_version 1341340 (0.0009) [2023-12-27 01:08:40,705][105692] Updated weights for policy 0, policy_version 1339380 (0.0009) [2023-12-27 01:08:40,767][105692] Updated weights for policy 0, policy_version 1339390 (0.0010) [2023-12-27 01:08:40,830][105692] Updated weights for policy 0, policy_version 1339400 (0.0008) [2023-12-27 01:08:41,008][105620] Updated weights for policy 1, policy_version 1341350 (0.0007) [2023-12-27 01:08:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 686366720. Throughput: 0: 9572.1, 1: 9876.4. Samples: 686376640. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:08:41,062][104569] Avg episode reward: [(0, '8910.869'), (1, '8725.994')] [2023-12-27 01:08:41,076][105620] Updated weights for policy 1, policy_version 1341360 (0.0008) [2023-12-27 01:08:41,142][105620] Updated weights for policy 1, policy_version 1341370 (0.0006) [2023-12-27 01:08:41,681][105692] Updated weights for policy 0, policy_version 1339410 (0.0008) [2023-12-27 01:08:41,742][105692] Updated weights for policy 0, policy_version 1339420 (0.0009) [2023-12-27 01:08:41,798][105692] Updated weights for policy 0, policy_version 1339430 (0.0009) [2023-12-27 01:08:41,851][105692] Updated weights for policy 0, policy_version 1339440 (0.0008) [2023-12-27 01:08:41,900][105620] Updated weights for policy 1, policy_version 1341380 (0.0008) [2023-12-27 01:08:41,952][105620] Updated weights for policy 1, policy_version 1341390 (0.0008) [2023-12-27 01:08:42,013][105620] Updated weights for policy 1, policy_version 1341400 (0.0008) [2023-12-27 01:08:42,588][105692] Updated weights for policy 0, policy_version 1339450 (0.0009) [2023-12-27 01:08:42,648][105692] Updated weights for policy 0, policy_version 1339460 (0.0008) [2023-12-27 01:08:42,715][105692] Updated weights for policy 0, policy_version 1339470 (0.0008) [2023-12-27 01:08:42,800][105620] Updated weights for policy 1, policy_version 1341410 (0.0008) [2023-12-27 01:08:42,858][105620] Updated weights for policy 1, policy_version 1341420 (0.0009) [2023-12-27 01:08:42,920][105620] Updated weights for policy 1, policy_version 1341430 (0.0009) [2023-12-27 01:08:42,979][105620] Updated weights for policy 1, policy_version 1341440 (0.0008) [2023-12-27 01:08:43,425][105692] Updated weights for policy 0, policy_version 1339480 (0.0009) [2023-12-27 01:08:43,479][105692] Updated weights for policy 0, policy_version 1339490 (0.0008) [2023-12-27 01:08:43,540][105692] Updated weights for policy 0, policy_version 1339500 (0.0009) [2023-12-27 01:08:43,746][105620] Updated weights for policy 1, policy_version 1341450 (0.0009) [2023-12-27 01:08:43,798][105620] Updated weights for policy 1, policy_version 1341460 (0.0009) [2023-12-27 01:08:43,849][105620] Updated weights for policy 1, policy_version 1341471 (0.0010) [2023-12-27 01:08:44,148][105692] Updated weights for policy 0, policy_version 1339510 (0.0007) [2023-12-27 01:08:44,210][105692] Updated weights for policy 0, policy_version 1339520 (0.0005) [2023-12-27 01:08:44,273][105692] Updated weights for policy 0, policy_version 1339530 (0.0006) [2023-12-27 01:08:44,567][105620] Updated weights for policy 1, policy_version 1341481 (0.0010) [2023-12-27 01:08:44,622][105620] Updated weights for policy 1, policy_version 1341491 (0.0010) [2023-12-27 01:08:44,683][105620] Updated weights for policy 1, policy_version 1341501 (0.0010) [2023-12-27 01:08:44,923][105692] Updated weights for policy 0, policy_version 1339540 (0.0006) [2023-12-27 01:08:44,989][105692] Updated weights for policy 0, policy_version 1339550 (0.0005) [2023-12-27 01:08:45,051][105692] Updated weights for policy 0, policy_version 1339560 (0.0006) [2023-12-27 01:08:45,403][105620] Updated weights for policy 1, policy_version 1341511 (0.0009) [2023-12-27 01:08:45,465][105620] Updated weights for policy 1, policy_version 1341521 (0.0009) [2023-12-27 01:08:45,521][105620] Updated weights for policy 1, policy_version 1341531 (0.0009) [2023-12-27 01:08:45,697][105692] Updated weights for policy 0, policy_version 1339570 (0.0006) [2023-12-27 01:08:45,754][105692] Updated weights for policy 0, policy_version 1339580 (0.0008) [2023-12-27 01:08:45,808][105692] Updated weights for policy 0, policy_version 1339590 (0.0005) [2023-12-27 01:08:45,854][105692] Updated weights for policy 0, policy_version 1339600 (0.0005) [2023-12-27 01:08:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 686465024. Throughput: 0: 9501.8, 1: 9847.6. Samples: 686430800. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:08:46,062][104569] Avg episode reward: [(0, '8908.062'), (1, '8908.028')] [2023-12-27 01:08:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001339600_342990848.pth... [2023-12-27 01:08:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001341536_343474176.pth... [2023-12-27 01:08:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001338480_342704128.pth [2023-12-27 01:08:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001340416_343187456.pth [2023-12-27 01:08:46,339][105620] Updated weights for policy 1, policy_version 1341541 (0.0009) [2023-12-27 01:08:46,400][105620] Updated weights for policy 1, policy_version 1341551 (0.0009) [2023-12-27 01:08:46,459][105620] Updated weights for policy 1, policy_version 1341561 (0.0009) [2023-12-27 01:08:46,540][105692] Updated weights for policy 0, policy_version 1339610 (0.0008) [2023-12-27 01:08:46,601][105692] Updated weights for policy 0, policy_version 1339620 (0.0009) [2023-12-27 01:08:46,664][105692] Updated weights for policy 0, policy_version 1339630 (0.0009) [2023-12-27 01:08:47,146][105620] Updated weights for policy 1, policy_version 1341571 (0.0007) [2023-12-27 01:08:47,210][105620] Updated weights for policy 1, policy_version 1341581 (0.0005) [2023-12-27 01:08:47,265][105620] Updated weights for policy 1, policy_version 1341591 (0.0006) [2023-12-27 01:08:47,502][105692] Updated weights for policy 0, policy_version 1339640 (0.0010) [2023-12-27 01:08:47,556][105692] Updated weights for policy 0, policy_version 1339651 (0.0010) [2023-12-27 01:08:47,615][105692] Updated weights for policy 0, policy_version 1339661 (0.0010) [2023-12-27 01:08:47,829][105620] Updated weights for policy 1, policy_version 1341601 (0.0006) [2023-12-27 01:08:47,896][105620] Updated weights for policy 1, policy_version 1341611 (0.0011) [2023-12-27 01:08:47,957][105620] Updated weights for policy 1, policy_version 1341621 (0.0011) [2023-12-27 01:08:48,021][105620] Updated weights for policy 1, policy_version 1341631 (0.0011) [2023-12-27 01:08:48,426][105692] Updated weights for policy 0, policy_version 1339671 (0.0009) [2023-12-27 01:08:48,477][105692] Updated weights for policy 0, policy_version 1339681 (0.0009) [2023-12-27 01:08:48,527][105692] Updated weights for policy 0, policy_version 1339691 (0.0008) [2023-12-27 01:08:48,696][105620] Updated weights for policy 1, policy_version 1341641 (0.0006) [2023-12-27 01:08:48,761][105620] Updated weights for policy 1, policy_version 1341651 (0.0008) [2023-12-27 01:08:48,824][105620] Updated weights for policy 1, policy_version 1341661 (0.0009) [2023-12-27 01:08:49,330][105692] Updated weights for policy 0, policy_version 1339701 (0.0009) [2023-12-27 01:08:49,401][105692] Updated weights for policy 0, policy_version 1339711 (0.0007) [2023-12-27 01:08:49,468][105692] Updated weights for policy 0, policy_version 1339721 (0.0008) [2023-12-27 01:08:49,494][105620] Updated weights for policy 1, policy_version 1341671 (0.0010) [2023-12-27 01:08:49,540][105620] Updated weights for policy 1, policy_version 1341681 (0.0011) [2023-12-27 01:08:49,597][105620] Updated weights for policy 1, policy_version 1341691 (0.0006) [2023-12-27 01:08:50,277][105692] Updated weights for policy 0, policy_version 1339731 (0.0006) [2023-12-27 01:08:50,279][105620] Updated weights for policy 1, policy_version 1341701 (0.0008) [2023-12-27 01:08:50,342][105620] Updated weights for policy 1, policy_version 1341711 (0.0011) [2023-12-27 01:08:50,344][105692] Updated weights for policy 0, policy_version 1339741 (0.0005) [2023-12-27 01:08:50,400][105692] Updated weights for policy 0, policy_version 1339751 (0.0005) [2023-12-27 01:08:50,402][105620] Updated weights for policy 1, policy_version 1341721 (0.0011) [2023-12-27 01:08:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 686555136. Throughput: 0: 9489.7, 1: 9793.3. Samples: 686549572. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:08:51,062][104569] Avg episode reward: [(0, '8819.229'), (1, '9176.738')] [2023-12-27 01:08:51,158][105620] Updated weights for policy 1, policy_version 1341731 (0.0011) [2023-12-27 01:08:51,169][105692] Updated weights for policy 0, policy_version 1339761 (0.0005) [2023-12-27 01:08:51,219][105620] Updated weights for policy 1, policy_version 1341741 (0.0011) [2023-12-27 01:08:51,221][105692] Updated weights for policy 0, policy_version 1339771 (0.0006) [2023-12-27 01:08:51,276][105620] Updated weights for policy 1, policy_version 1341751 (0.0011) [2023-12-27 01:08:51,286][105692] Updated weights for policy 0, policy_version 1339781 (0.0006) [2023-12-27 01:08:51,335][105692] Updated weights for policy 0, policy_version 1339791 (0.0006) [2023-12-27 01:08:52,024][105620] Updated weights for policy 1, policy_version 1341761 (0.0010) [2023-12-27 01:08:52,089][105620] Updated weights for policy 1, policy_version 1341771 (0.0008) [2023-12-27 01:08:52,149][105620] Updated weights for policy 1, policy_version 1341781 (0.0007) [2023-12-27 01:08:52,155][105692] Updated weights for policy 0, policy_version 1339801 (0.0008) [2023-12-27 01:08:52,202][105620] Updated weights for policy 1, policy_version 1341791 (0.0009) [2023-12-27 01:08:52,204][105692] Updated weights for policy 0, policy_version 1339811 (0.0006) [2023-12-27 01:08:52,257][105692] Updated weights for policy 0, policy_version 1339821 (0.0008) [2023-12-27 01:08:52,819][105620] Updated weights for policy 1, policy_version 1341801 (0.0009) [2023-12-27 01:08:52,884][105620] Updated weights for policy 1, policy_version 1341811 (0.0009) [2023-12-27 01:08:52,946][105620] Updated weights for policy 1, policy_version 1341821 (0.0009) [2023-12-27 01:08:53,040][105692] Updated weights for policy 0, policy_version 1339831 (0.0010) [2023-12-27 01:08:53,093][105692] Updated weights for policy 0, policy_version 1339841 (0.0009) [2023-12-27 01:08:53,156][105692] Updated weights for policy 0, policy_version 1339852 (0.0011) [2023-12-27 01:08:53,554][105620] Updated weights for policy 1, policy_version 1341831 (0.0010) [2023-12-27 01:08:53,609][105620] Updated weights for policy 1, policy_version 1341841 (0.0010) [2023-12-27 01:08:53,660][105620] Updated weights for policy 1, policy_version 1341851 (0.0010) [2023-12-27 01:08:53,878][105692] Updated weights for policy 0, policy_version 1339862 (0.0007) [2023-12-27 01:08:53,929][105692] Updated weights for policy 0, policy_version 1339872 (0.0005) [2023-12-27 01:08:53,978][105692] Updated weights for policy 0, policy_version 1339882 (0.0005) [2023-12-27 01:08:54,282][105620] Updated weights for policy 1, policy_version 1341861 (0.0008) [2023-12-27 01:08:54,330][105620] Updated weights for policy 1, policy_version 1341871 (0.0005) [2023-12-27 01:08:54,393][105620] Updated weights for policy 1, policy_version 1341881 (0.0007) [2023-12-27 01:08:54,614][105692] Updated weights for policy 0, policy_version 1339892 (0.0005) [2023-12-27 01:08:54,662][105692] Updated weights for policy 0, policy_version 1339902 (0.0005) [2023-12-27 01:08:54,716][105692] Updated weights for policy 0, policy_version 1339912 (0.0005) [2023-12-27 01:08:55,005][105620] Updated weights for policy 1, policy_version 1341891 (0.0009) [2023-12-27 01:08:55,056][105620] Updated weights for policy 1, policy_version 1341901 (0.0005) [2023-12-27 01:08:55,118][105620] Updated weights for policy 1, policy_version 1341911 (0.0005) [2023-12-27 01:08:55,254][105692] Updated weights for policy 0, policy_version 1339922 (0.0006) [2023-12-27 01:08:55,317][105692] Updated weights for policy 0, policy_version 1339932 (0.0005) [2023-12-27 01:08:55,376][105692] Updated weights for policy 0, policy_version 1339942 (0.0005) [2023-12-27 01:08:55,438][105692] Updated weights for policy 0, policy_version 1339952 (0.0007) [2023-12-27 01:08:55,733][105620] Updated weights for policy 1, policy_version 1341921 (0.0006) [2023-12-27 01:08:55,790][105620] Updated weights for policy 1, policy_version 1341931 (0.0009) [2023-12-27 01:08:55,855][105620] Updated weights for policy 1, policy_version 1341941 (0.0006) [2023-12-27 01:08:55,913][105620] Updated weights for policy 1, policy_version 1341951 (0.0005) [2023-12-27 01:08:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 686661632. Throughput: 0: 9506.3, 1: 9913.4. Samples: 686670964. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:08:56,062][104569] Avg episode reward: [(0, '8911.138'), (1, '8996.767')] [2023-12-27 01:08:56,084][105692] Updated weights for policy 0, policy_version 1339962 (0.0009) [2023-12-27 01:08:56,132][105692] Updated weights for policy 0, policy_version 1339972 (0.0010) [2023-12-27 01:08:56,183][105692] Updated weights for policy 0, policy_version 1339982 (0.0010) [2023-12-27 01:08:56,640][105620] Updated weights for policy 1, policy_version 1341961 (0.0009) [2023-12-27 01:08:56,692][105620] Updated weights for policy 1, policy_version 1341971 (0.0008) [2023-12-27 01:08:56,744][105620] Updated weights for policy 1, policy_version 1341981 (0.0009) [2023-12-27 01:08:56,903][105692] Updated weights for policy 0, policy_version 1339992 (0.0010) [2023-12-27 01:08:56,954][105692] Updated weights for policy 0, policy_version 1340002 (0.0009) [2023-12-27 01:08:57,016][105692] Updated weights for policy 0, policy_version 1340012 (0.0005) [2023-12-27 01:08:57,378][105620] Updated weights for policy 1, policy_version 1341991 (0.0010) [2023-12-27 01:08:57,446][105620] Updated weights for policy 1, policy_version 1342001 (0.0010) [2023-12-27 01:08:57,507][105620] Updated weights for policy 1, policy_version 1342011 (0.0008) [2023-12-27 01:08:57,640][105692] Updated weights for policy 0, policy_version 1340022 (0.0005) [2023-12-27 01:08:57,698][105692] Updated weights for policy 0, policy_version 1340032 (0.0005) [2023-12-27 01:08:57,750][105692] Updated weights for policy 0, policy_version 1340042 (0.0007) [2023-12-27 01:08:58,095][105620] Updated weights for policy 1, policy_version 1342021 (0.0005) [2023-12-27 01:08:58,141][105620] Updated weights for policy 1, policy_version 1342031 (0.0005) [2023-12-27 01:08:58,212][105620] Updated weights for policy 1, policy_version 1342041 (0.0006) [2023-12-27 01:08:58,348][105692] Updated weights for policy 0, policy_version 1340052 (0.0007) [2023-12-27 01:08:58,408][105692] Updated weights for policy 0, policy_version 1340062 (0.0008) [2023-12-27 01:08:58,473][105692] Updated weights for policy 0, policy_version 1340072 (0.0008) [2023-12-27 01:08:59,005][105620] Updated weights for policy 1, policy_version 1342051 (0.0008) [2023-12-27 01:08:59,058][105620] Updated weights for policy 1, policy_version 1342061 (0.0008) [2023-12-27 01:08:59,118][105620] Updated weights for policy 1, policy_version 1342071 (0.0005) [2023-12-27 01:08:59,213][105692] Updated weights for policy 0, policy_version 1340082 (0.0008) [2023-12-27 01:08:59,285][105692] Updated weights for policy 0, policy_version 1340092 (0.0007) [2023-12-27 01:08:59,343][105692] Updated weights for policy 0, policy_version 1340102 (0.0009) [2023-12-27 01:08:59,414][105692] Updated weights for policy 0, policy_version 1340112 (0.0013) [2023-12-27 01:08:59,875][105620] Updated weights for policy 1, policy_version 1342081 (0.0006) [2023-12-27 01:08:59,935][105620] Updated weights for policy 1, policy_version 1342091 (0.0007) [2023-12-27 01:08:59,999][105620] Updated weights for policy 1, policy_version 1342101 (0.0009) [2023-12-27 01:09:00,050][105620] Updated weights for policy 1, policy_version 1342111 (0.0008) [2023-12-27 01:09:00,110][105692] Updated weights for policy 0, policy_version 1340122 (0.0007) [2023-12-27 01:09:00,167][105692] Updated weights for policy 0, policy_version 1340132 (0.0005) [2023-12-27 01:09:00,225][105692] Updated weights for policy 0, policy_version 1340142 (0.0006) [2023-12-27 01:09:00,780][105620] Updated weights for policy 1, policy_version 1342121 (0.0006) [2023-12-27 01:09:00,848][105692] Updated weights for policy 0, policy_version 1340152 (0.0006) [2023-12-27 01:09:00,849][105620] Updated weights for policy 1, policy_version 1342131 (0.0005) [2023-12-27 01:09:00,894][105692] Updated weights for policy 0, policy_version 1340162 (0.0005) [2023-12-27 01:09:00,905][105620] Updated weights for policy 1, policy_version 1342141 (0.0005) [2023-12-27 01:09:00,943][105692] Updated weights for policy 0, policy_version 1340172 (0.0007) [2023-12-27 01:09:01,062][104569] Fps is (10 sec: 21298.8, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 686768128. Throughput: 0: 9606.8, 1: 9912.8. Samples: 686732316. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:09:01,062][104569] Avg episode reward: [(0, '8633.284'), (1, '9265.231')] [2023-12-27 01:09:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001340176_343138304.pth... [2023-12-27 01:09:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001342144_343629824.pth... [2023-12-27 01:09:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001339024_342843392.pth [2023-12-27 01:09:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001340992_343334912.pth [2023-12-27 01:09:01,531][105620] Updated weights for policy 1, policy_version 1342151 (0.0007) [2023-12-27 01:09:01,579][105620] Updated weights for policy 1, policy_version 1342161 (0.0007) [2023-12-27 01:09:01,643][105620] Updated weights for policy 1, policy_version 1342171 (0.0007) [2023-12-27 01:09:01,700][105692] Updated weights for policy 0, policy_version 1340182 (0.0011) [2023-12-27 01:09:01,761][105692] Updated weights for policy 0, policy_version 1340192 (0.0009) [2023-12-27 01:09:01,816][105692] Updated weights for policy 0, policy_version 1340202 (0.0010) [2023-12-27 01:09:02,429][105620] Updated weights for policy 1, policy_version 1342181 (0.0008) [2023-12-27 01:09:02,488][105620] Updated weights for policy 1, policy_version 1342191 (0.0008) [2023-12-27 01:09:02,539][105620] Updated weights for policy 1, policy_version 1342201 (0.0008) [2023-12-27 01:09:02,552][105585] KL-divergence is very high: 482.6410 [2023-12-27 01:09:02,559][105692] Updated weights for policy 0, policy_version 1340212 (0.0009) [2023-12-27 01:09:02,592][105585] KL-divergence is very high: 781.8248 [2023-12-27 01:09:02,605][105692] Updated weights for policy 0, policy_version 1340222 (0.0009) [2023-12-27 01:09:02,636][105585] KL-divergence is very high: 807.1563 [2023-12-27 01:09:02,673][105692] Updated weights for policy 0, policy_version 1340232 (0.0009) [2023-12-27 01:09:02,693][105585] KL-divergence is very high: 732.8556 [2023-12-27 01:09:03,156][105620] Updated weights for policy 1, policy_version 1342211 (0.0009) [2023-12-27 01:09:03,205][105620] Updated weights for policy 1, policy_version 1342221 (0.0008) [2023-12-27 01:09:03,256][105620] Updated weights for policy 1, policy_version 1342231 (0.0008) [2023-12-27 01:09:03,409][105692] Updated weights for policy 0, policy_version 1340242 (0.0010) [2023-12-27 01:09:03,469][105692] Updated weights for policy 0, policy_version 1340253 (0.0007) [2023-12-27 01:09:03,523][105692] Updated weights for policy 0, policy_version 1340263 (0.0008) [2023-12-27 01:09:03,980][105620] Updated weights for policy 1, policy_version 1342241 (0.0007) [2023-12-27 01:09:04,035][105620] Updated weights for policy 1, policy_version 1342251 (0.0010) [2023-12-27 01:09:04,089][105620] Updated weights for policy 1, policy_version 1342261 (0.0008) [2023-12-27 01:09:04,113][105692] Updated weights for policy 0, policy_version 1340273 (0.0008) [2023-12-27 01:09:04,137][105620] Updated weights for policy 1, policy_version 1342271 (0.0010) [2023-12-27 01:09:04,159][105692] Updated weights for policy 0, policy_version 1340283 (0.0008) [2023-12-27 01:09:04,215][105692] Updated weights for policy 0, policy_version 1340293 (0.0005) [2023-12-27 01:09:04,281][105692] Updated weights for policy 0, policy_version 1340303 (0.0006) [2023-12-27 01:09:04,890][105620] Updated weights for policy 1, policy_version 1342281 (0.0010) [2023-12-27 01:09:04,945][105620] Updated weights for policy 1, policy_version 1342291 (0.0010) [2023-12-27 01:09:04,949][105692] Updated weights for policy 0, policy_version 1340313 (0.0009) [2023-12-27 01:09:05,000][105692] Updated weights for policy 0, policy_version 1340323 (0.0005) [2023-12-27 01:09:05,007][105620] Updated weights for policy 1, policy_version 1342301 (0.0010) [2023-12-27 01:09:05,055][105692] Updated weights for policy 0, policy_version 1340333 (0.0010) [2023-12-27 01:09:05,756][105620] Updated weights for policy 1, policy_version 1342311 (0.0011) [2023-12-27 01:09:05,777][105692] Updated weights for policy 0, policy_version 1340343 (0.0006) [2023-12-27 01:09:05,812][105620] Updated weights for policy 1, policy_version 1342321 (0.0010) [2023-12-27 01:09:05,824][105692] Updated weights for policy 0, policy_version 1340353 (0.0005) [2023-12-27 01:09:05,873][105620] Updated weights for policy 1, policy_version 1342331 (0.0010) [2023-12-27 01:09:05,875][105692] Updated weights for policy 0, policy_version 1340363 (0.0005) [2023-12-27 01:09:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 686866432. Throughput: 0: 9770.3, 1: 9855.8. Samples: 686850880. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:09:06,062][104569] Avg episode reward: [(0, '8634.716'), (1, '9357.074')] [2023-12-27 01:09:06,558][105692] Updated weights for policy 0, policy_version 1340373 (0.0009) [2023-12-27 01:09:06,610][105620] Updated weights for policy 1, policy_version 1342341 (0.0010) [2023-12-27 01:09:06,618][105692] Updated weights for policy 0, policy_version 1340383 (0.0011) [2023-12-27 01:09:06,673][105692] Updated weights for policy 0, policy_version 1340393 (0.0010) [2023-12-27 01:09:06,676][105620] Updated weights for policy 1, policy_version 1342351 (0.0010) [2023-12-27 01:09:06,738][105620] Updated weights for policy 1, policy_version 1342361 (0.0010) [2023-12-27 01:09:07,379][105692] Updated weights for policy 0, policy_version 1340403 (0.0009) [2023-12-27 01:09:07,426][105692] Updated weights for policy 0, policy_version 1340413 (0.0005) [2023-12-27 01:09:07,469][105620] Updated weights for policy 1, policy_version 1342371 (0.0010) [2023-12-27 01:09:07,481][105692] Updated weights for policy 0, policy_version 1340423 (0.0006) [2023-12-27 01:09:07,527][105620] Updated weights for policy 1, policy_version 1342381 (0.0010) [2023-12-27 01:09:07,578][105620] Updated weights for policy 1, policy_version 1342391 (0.0009) [2023-12-27 01:09:08,060][105692] Updated weights for policy 0, policy_version 1340433 (0.0008) [2023-12-27 01:09:08,124][105692] Updated weights for policy 0, policy_version 1340443 (0.0010) [2023-12-27 01:09:08,125][105620] Updated weights for policy 1, policy_version 1342401 (0.0005) [2023-12-27 01:09:08,173][105692] Updated weights for policy 0, policy_version 1340454 (0.0009) [2023-12-27 01:09:08,179][105620] Updated weights for policy 1, policy_version 1342411 (0.0006) [2023-12-27 01:09:08,220][105692] Updated weights for policy 0, policy_version 1340464 (0.0009) [2023-12-27 01:09:08,227][105620] Updated weights for policy 1, policy_version 1342421 (0.0005) [2023-12-27 01:09:08,272][105620] Updated weights for policy 1, policy_version 1342431 (0.0005) [2023-12-27 01:09:08,984][105620] Updated weights for policy 1, policy_version 1342441 (0.0009) [2023-12-27 01:09:09,038][105620] Updated weights for policy 1, policy_version 1342451 (0.0007) [2023-12-27 01:09:09,060][105692] Updated weights for policy 0, policy_version 1340474 (0.0008) [2023-12-27 01:09:09,100][105620] Updated weights for policy 1, policy_version 1342461 (0.0008) [2023-12-27 01:09:09,114][105692] Updated weights for policy 0, policy_version 1340484 (0.0008) [2023-12-27 01:09:09,170][105692] Updated weights for policy 0, policy_version 1340494 (0.0009) [2023-12-27 01:09:09,850][105692] Updated weights for policy 0, policy_version 1340504 (0.0009) [2023-12-27 01:09:09,904][105692] Updated weights for policy 0, policy_version 1340514 (0.0008) [2023-12-27 01:09:09,933][105620] Updated weights for policy 1, policy_version 1342471 (0.0007) [2023-12-27 01:09:09,967][105692] Updated weights for policy 0, policy_version 1340524 (0.0008) [2023-12-27 01:09:09,986][105620] Updated weights for policy 1, policy_version 1342481 (0.0008) [2023-12-27 01:09:10,048][105620] Updated weights for policy 1, policy_version 1342491 (0.0008) [2023-12-27 01:09:10,687][105620] Updated weights for policy 1, policy_version 1342501 (0.0007) [2023-12-27 01:09:10,724][105692] Updated weights for policy 0, policy_version 1340534 (0.0008) [2023-12-27 01:09:10,738][105620] Updated weights for policy 1, policy_version 1342511 (0.0007) [2023-12-27 01:09:10,787][105692] Updated weights for policy 0, policy_version 1340544 (0.0009) [2023-12-27 01:09:10,792][105620] Updated weights for policy 1, policy_version 1342521 (0.0007) [2023-12-27 01:09:10,845][105692] Updated weights for policy 0, policy_version 1340554 (0.0007) [2023-12-27 01:09:11,063][104569] Fps is (10 sec: 19658.1, 60 sec: 19660.3, 300 sec: 19521.9). Total num frames: 686964736. Throughput: 0: 9842.9, 1: 9869.5. Samples: 686969224. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:09:11,064][104569] Avg episode reward: [(0, '8636.629'), (1, '9267.077')] [2023-12-27 01:09:11,584][105620] Updated weights for policy 1, policy_version 1342531 (0.0007) [2023-12-27 01:09:11,630][105692] Updated weights for policy 0, policy_version 1340564 (0.0007) [2023-12-27 01:09:11,656][105620] Updated weights for policy 1, policy_version 1342541 (0.0007) [2023-12-27 01:09:11,692][105692] Updated weights for policy 0, policy_version 1340574 (0.0007) [2023-12-27 01:09:11,714][105620] Updated weights for policy 1, policy_version 1342551 (0.0009) [2023-12-27 01:09:11,750][105692] Updated weights for policy 0, policy_version 1340584 (0.0009) [2023-12-27 01:09:12,357][105620] Updated weights for policy 1, policy_version 1342561 (0.0008) [2023-12-27 01:09:12,423][105620] Updated weights for policy 1, policy_version 1342571 (0.0007) [2023-12-27 01:09:12,484][105620] Updated weights for policy 1, policy_version 1342581 (0.0005) [2023-12-27 01:09:12,548][105620] Updated weights for policy 1, policy_version 1342591 (0.0008) [2023-12-27 01:09:12,600][105692] Updated weights for policy 0, policy_version 1340594 (0.0008) [2023-12-27 01:09:12,663][105692] Updated weights for policy 0, policy_version 1340604 (0.0009) [2023-12-27 01:09:12,727][105692] Updated weights for policy 0, policy_version 1340614 (0.0009) [2023-12-27 01:09:12,790][105692] Updated weights for policy 0, policy_version 1340624 (0.0009) [2023-12-27 01:09:13,277][105620] Updated weights for policy 1, policy_version 1342601 (0.0010) [2023-12-27 01:09:13,336][105620] Updated weights for policy 1, policy_version 1342611 (0.0010) [2023-12-27 01:09:13,401][105620] Updated weights for policy 1, policy_version 1342621 (0.0010) [2023-12-27 01:09:13,545][105692] Updated weights for policy 0, policy_version 1340634 (0.0010) [2023-12-27 01:09:13,600][105692] Updated weights for policy 0, policy_version 1340644 (0.0010) [2023-12-27 01:09:13,654][105692] Updated weights for policy 0, policy_version 1340654 (0.0006) [2023-12-27 01:09:14,102][105620] Updated weights for policy 1, policy_version 1342631 (0.0006) [2023-12-27 01:09:14,153][105620] Updated weights for policy 1, policy_version 1342641 (0.0009) [2023-12-27 01:09:14,203][105620] Updated weights for policy 1, policy_version 1342651 (0.0007) [2023-12-27 01:09:14,322][105692] Updated weights for policy 0, policy_version 1340664 (0.0009) [2023-12-27 01:09:14,375][105692] Updated weights for policy 0, policy_version 1340674 (0.0005) [2023-12-27 01:09:14,380][105585] KL-divergence is very high: 130.2749 [2023-12-27 01:09:14,425][105585] KL-divergence is very high: 115.8986 [2023-12-27 01:09:14,431][105692] Updated weights for policy 0, policy_version 1340684 (0.0008) [2023-12-27 01:09:14,954][105620] Updated weights for policy 1, policy_version 1342661 (0.0009) [2023-12-27 01:09:15,015][105620] Updated weights for policy 1, policy_version 1342671 (0.0011) [2023-12-27 01:09:15,073][105586] KL-divergence is very high: 234.6361 [2023-12-27 01:09:15,078][105620] Updated weights for policy 1, policy_version 1342681 (0.0010) [2023-12-27 01:09:15,079][105586] KL-divergence is very high: 267.9065 [2023-12-27 01:09:15,189][105692] Updated weights for policy 0, policy_version 1340694 (0.0009) [2023-12-27 01:09:15,253][105692] Updated weights for policy 0, policy_version 1340704 (0.0008) [2023-12-27 01:09:15,318][105692] Updated weights for policy 0, policy_version 1340714 (0.0009) [2023-12-27 01:09:15,773][105620] Updated weights for policy 1, policy_version 1342691 (0.0009) [2023-12-27 01:09:15,824][105620] Updated weights for policy 1, policy_version 1342701 (0.0005) [2023-12-27 01:09:15,872][105620] Updated weights for policy 1, policy_version 1342711 (0.0005) [2023-12-27 01:09:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 687054848. Throughput: 0: 9728.5, 1: 9845.2. Samples: 687024956. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:09:16,062][104569] Avg episode reward: [(0, '8547.549'), (1, '8909.698')] [2023-12-27 01:09:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001340720_343277568.pth... [2023-12-27 01:09:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001342720_343777280.pth... [2023-12-27 01:09:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001339600_342990848.pth [2023-12-27 01:09:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001341536_343474176.pth [2023-12-27 01:09:16,117][105692] Updated weights for policy 0, policy_version 1340724 (0.0009) [2023-12-27 01:09:16,175][105692] Updated weights for policy 0, policy_version 1340734 (0.0010) [2023-12-27 01:09:16,234][105692] Updated weights for policy 0, policy_version 1340744 (0.0011) [2023-12-27 01:09:16,595][105620] Updated weights for policy 1, policy_version 1342721 (0.0009) [2023-12-27 01:09:16,653][105620] Updated weights for policy 1, policy_version 1342731 (0.0008) [2023-12-27 01:09:16,712][105620] Updated weights for policy 1, policy_version 1342741 (0.0008) [2023-12-27 01:09:16,767][105620] Updated weights for policy 1, policy_version 1342751 (0.0008) [2023-12-27 01:09:16,992][105692] Updated weights for policy 0, policy_version 1340754 (0.0010) [2023-12-27 01:09:17,050][105692] Updated weights for policy 0, policy_version 1340764 (0.0011) [2023-12-27 01:09:17,103][105692] Updated weights for policy 0, policy_version 1340774 (0.0011) [2023-12-27 01:09:17,167][105692] Updated weights for policy 0, policy_version 1340784 (0.0011) [2023-12-27 01:09:17,526][105620] Updated weights for policy 1, policy_version 1342761 (0.0008) [2023-12-27 01:09:17,577][105620] Updated weights for policy 1, policy_version 1342771 (0.0008) [2023-12-27 01:09:17,627][105620] Updated weights for policy 1, policy_version 1342781 (0.0008) [2023-12-27 01:09:17,897][105692] Updated weights for policy 0, policy_version 1340794 (0.0009) [2023-12-27 01:09:17,952][105692] Updated weights for policy 0, policy_version 1340804 (0.0010) [2023-12-27 01:09:18,011][105692] Updated weights for policy 0, policy_version 1340814 (0.0008) [2023-12-27 01:09:18,346][105620] Updated weights for policy 1, policy_version 1342791 (0.0007) [2023-12-27 01:09:18,411][105620] Updated weights for policy 1, policy_version 1342801 (0.0008) [2023-12-27 01:09:18,471][105620] Updated weights for policy 1, policy_version 1342811 (0.0008) [2023-12-27 01:09:18,737][105692] Updated weights for policy 0, policy_version 1340824 (0.0005) [2023-12-27 01:09:18,799][105692] Updated weights for policy 0, policy_version 1340834 (0.0006) [2023-12-27 01:09:18,862][105692] Updated weights for policy 0, policy_version 1340844 (0.0005) [2023-12-27 01:09:19,307][105620] Updated weights for policy 1, policy_version 1342821 (0.0009) [2023-12-27 01:09:19,376][105620] Updated weights for policy 1, policy_version 1342831 (0.0008) [2023-12-27 01:09:19,440][105620] Updated weights for policy 1, policy_version 1342841 (0.0009) [2023-12-27 01:09:19,560][105692] Updated weights for policy 0, policy_version 1340854 (0.0009) [2023-12-27 01:09:19,628][105692] Updated weights for policy 0, policy_version 1340864 (0.0009) [2023-12-27 01:09:19,687][105692] Updated weights for policy 0, policy_version 1340874 (0.0009) [2023-12-27 01:09:20,193][105620] Updated weights for policy 1, policy_version 1342851 (0.0008) [2023-12-27 01:09:20,258][105620] Updated weights for policy 1, policy_version 1342861 (0.0008) [2023-12-27 01:09:20,323][105620] Updated weights for policy 1, policy_version 1342871 (0.0008) [2023-12-27 01:09:20,454][105692] Updated weights for policy 0, policy_version 1340884 (0.0010) [2023-12-27 01:09:20,514][105692] Updated weights for policy 0, policy_version 1340894 (0.0011) [2023-12-27 01:09:20,580][105692] Updated weights for policy 0, policy_version 1340904 (0.0011) [2023-12-27 01:09:20,962][105620] Updated weights for policy 1, policy_version 1342881 (0.0009) [2023-12-27 01:09:21,026][105620] Updated weights for policy 1, policy_version 1342891 (0.0010) [2023-12-27 01:09:21,062][104569] Fps is (10 sec: 18025.1, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 687144960. Throughput: 0: 9617.1, 1: 9817.0. Samples: 687137908. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:09:21,062][104569] Avg episode reward: [(0, '8551.685'), (1, '8909.456')] [2023-12-27 01:09:21,095][105620] Updated weights for policy 1, policy_version 1342901 (0.0008) [2023-12-27 01:09:21,154][105620] Updated weights for policy 1, policy_version 1342911 (0.0008) [2023-12-27 01:09:21,374][105692] Updated weights for policy 0, policy_version 1340914 (0.0011) [2023-12-27 01:09:21,443][105692] Updated weights for policy 0, policy_version 1340924 (0.0009) [2023-12-27 01:09:21,514][105692] Updated weights for policy 0, policy_version 1340934 (0.0011) [2023-12-27 01:09:21,587][105692] Updated weights for policy 0, policy_version 1340944 (0.0011) [2023-12-27 01:09:21,859][105620] Updated weights for policy 1, policy_version 1342921 (0.0010) [2023-12-27 01:09:21,919][105620] Updated weights for policy 1, policy_version 1342931 (0.0009) [2023-12-27 01:09:21,969][105620] Updated weights for policy 1, policy_version 1342941 (0.0009) [2023-12-27 01:09:22,324][105692] Updated weights for policy 0, policy_version 1340954 (0.0008) [2023-12-27 01:09:22,390][105692] Updated weights for policy 0, policy_version 1340964 (0.0008) [2023-12-27 01:09:22,460][105692] Updated weights for policy 0, policy_version 1340974 (0.0007) [2023-12-27 01:09:22,715][105620] Updated weights for policy 1, policy_version 1342951 (0.0009) [2023-12-27 01:09:22,775][105620] Updated weights for policy 1, policy_version 1342961 (0.0007) [2023-12-27 01:09:22,837][105620] Updated weights for policy 1, policy_version 1342971 (0.0005) [2023-12-27 01:09:23,168][105692] Updated weights for policy 0, policy_version 1340984 (0.0010) [2023-12-27 01:09:23,228][105692] Updated weights for policy 0, policy_version 1340994 (0.0010) [2023-12-27 01:09:23,281][105692] Updated weights for policy 0, policy_version 1341004 (0.0010) [2023-12-27 01:09:23,476][105620] Updated weights for policy 1, policy_version 1342981 (0.0008) [2023-12-27 01:09:23,531][105620] Updated weights for policy 1, policy_version 1342992 (0.0011) [2023-12-27 01:09:23,585][105620] Updated weights for policy 1, policy_version 1343002 (0.0010) [2023-12-27 01:09:23,851][105692] Updated weights for policy 0, policy_version 1341014 (0.0007) [2023-12-27 01:09:23,915][105692] Updated weights for policy 0, policy_version 1341024 (0.0007) [2023-12-27 01:09:23,979][105692] Updated weights for policy 0, policy_version 1341034 (0.0010) [2023-12-27 01:09:24,275][105620] Updated weights for policy 1, policy_version 1343012 (0.0007) [2023-12-27 01:09:24,334][105620] Updated weights for policy 1, policy_version 1343022 (0.0011) [2023-12-27 01:09:24,389][105620] Updated weights for policy 1, policy_version 1343032 (0.0010) [2023-12-27 01:09:24,538][105692] Updated weights for policy 0, policy_version 1341044 (0.0008) [2023-12-27 01:09:24,589][105692] Updated weights for policy 0, policy_version 1341054 (0.0005) [2023-12-27 01:09:24,643][105692] Updated weights for policy 0, policy_version 1341064 (0.0005) [2023-12-27 01:09:25,124][105620] Updated weights for policy 1, policy_version 1343042 (0.0010) [2023-12-27 01:09:25,191][105620] Updated weights for policy 1, policy_version 1343052 (0.0007) [2023-12-27 01:09:25,242][105620] Updated weights for policy 1, policy_version 1343062 (0.0005) [2023-12-27 01:09:25,277][105692] Updated weights for policy 0, policy_version 1341074 (0.0005) [2023-12-27 01:09:25,303][105620] Updated weights for policy 1, policy_version 1343072 (0.0005) [2023-12-27 01:09:25,333][105692] Updated weights for policy 0, policy_version 1341084 (0.0005) [2023-12-27 01:09:25,394][105692] Updated weights for policy 0, policy_version 1341094 (0.0005) [2023-12-27 01:09:25,445][105692] Updated weights for policy 0, policy_version 1341104 (0.0005) [2023-12-27 01:09:25,999][105692] Updated weights for policy 0, policy_version 1341114 (0.0007) [2023-12-27 01:09:26,004][105620] Updated weights for policy 1, policy_version 1343082 (0.0009) [2023-12-27 01:09:26,056][105692] Updated weights for policy 0, policy_version 1341124 (0.0006) [2023-12-27 01:09:26,061][105620] Updated weights for policy 1, policy_version 1343092 (0.0008) [2023-12-27 01:09:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 687243264. Throughput: 0: 9755.9, 1: 9836.8. Samples: 687258312. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:09:26,062][104569] Avg episode reward: [(0, '8190.218'), (1, '8995.613')] [2023-12-27 01:09:26,118][105692] Updated weights for policy 0, policy_version 1341134 (0.0007) [2023-12-27 01:09:26,120][105620] Updated weights for policy 1, policy_version 1343102 (0.0007) [2023-12-27 01:09:26,744][105620] Updated weights for policy 1, policy_version 1343112 (0.0005) [2023-12-27 01:09:26,795][105620] Updated weights for policy 1, policy_version 1343122 (0.0006) [2023-12-27 01:09:26,814][105692] Updated weights for policy 0, policy_version 1341144 (0.0010) [2023-12-27 01:09:26,854][105620] Updated weights for policy 1, policy_version 1343132 (0.0007) [2023-12-27 01:09:26,873][105692] Updated weights for policy 0, policy_version 1341154 (0.0010) [2023-12-27 01:09:26,930][105692] Updated weights for policy 0, policy_version 1341164 (0.0010) [2023-12-27 01:09:27,553][105620] Updated weights for policy 1, policy_version 1343142 (0.0010) [2023-12-27 01:09:27,603][105620] Updated weights for policy 1, policy_version 1343152 (0.0009) [2023-12-27 01:09:27,651][105692] Updated weights for policy 0, policy_version 1341174 (0.0010) [2023-12-27 01:09:27,662][105620] Updated weights for policy 1, policy_version 1343162 (0.0007) [2023-12-27 01:09:27,702][105692] Updated weights for policy 0, policy_version 1341184 (0.0010) [2023-12-27 01:09:27,754][105692] Updated weights for policy 0, policy_version 1341194 (0.0010) [2023-12-27 01:09:28,419][105692] Updated weights for policy 0, policy_version 1341204 (0.0008) [2023-12-27 01:09:28,448][105620] Updated weights for policy 1, policy_version 1343172 (0.0010) [2023-12-27 01:09:28,484][105692] Updated weights for policy 0, policy_version 1341214 (0.0008) [2023-12-27 01:09:28,503][105620] Updated weights for policy 1, policy_version 1343182 (0.0010) [2023-12-27 01:09:28,541][105692] Updated weights for policy 0, policy_version 1341224 (0.0008) [2023-12-27 01:09:28,564][105620] Updated weights for policy 1, policy_version 1343192 (0.0011) [2023-12-27 01:09:29,203][105692] Updated weights for policy 0, policy_version 1341234 (0.0010) [2023-12-27 01:09:29,276][105692] Updated weights for policy 0, policy_version 1341244 (0.0010) [2023-12-27 01:09:29,281][105620] Updated weights for policy 1, policy_version 1343202 (0.0009) [2023-12-27 01:09:29,336][105692] Updated weights for policy 0, policy_version 1341254 (0.0010) [2023-12-27 01:09:29,342][105620] Updated weights for policy 1, policy_version 1343212 (0.0009) [2023-12-27 01:09:29,396][105692] Updated weights for policy 0, policy_version 1341264 (0.0010) [2023-12-27 01:09:29,399][105620] Updated weights for policy 1, policy_version 1343222 (0.0010) [2023-12-27 01:09:29,443][105620] Updated weights for policy 1, policy_version 1343232 (0.0010) [2023-12-27 01:09:30,061][105692] Updated weights for policy 0, policy_version 1341274 (0.0011) [2023-12-27 01:09:30,123][105692] Updated weights for policy 0, policy_version 1341284 (0.0010) [2023-12-27 01:09:30,136][105620] Updated weights for policy 1, policy_version 1343242 (0.0008) [2023-12-27 01:09:30,175][105692] Updated weights for policy 0, policy_version 1341294 (0.0010) [2023-12-27 01:09:30,189][105620] Updated weights for policy 1, policy_version 1343252 (0.0005) [2023-12-27 01:09:30,248][105620] Updated weights for policy 1, policy_version 1343262 (0.0008) [2023-12-27 01:09:30,876][105692] Updated weights for policy 0, policy_version 1341304 (0.0009) [2023-12-27 01:09:30,925][105620] Updated weights for policy 1, policy_version 1343272 (0.0009) [2023-12-27 01:09:30,931][105692] Updated weights for policy 0, policy_version 1341314 (0.0006) [2023-12-27 01:09:30,981][105620] Updated weights for policy 1, policy_version 1343282 (0.0007) [2023-12-27 01:09:30,990][105692] Updated weights for policy 0, policy_version 1341324 (0.0006) [2023-12-27 01:09:31,033][105620] Updated weights for policy 1, policy_version 1343292 (0.0007) [2023-12-27 01:09:31,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 687357952. Throughput: 0: 9836.4, 1: 9884.5. Samples: 687318244. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:09:31,063][104569] Avg episode reward: [(0, '8087.996'), (1, '8994.775')] [2023-12-27 01:09:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001341328_343433216.pth... [2023-12-27 01:09:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001343296_343924736.pth... [2023-12-27 01:09:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001342144_343629824.pth [2023-12-27 01:09:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001340176_343138304.pth [2023-12-27 01:09:31,736][105620] Updated weights for policy 1, policy_version 1343302 (0.0009) [2023-12-27 01:09:31,783][105692] Updated weights for policy 0, policy_version 1341334 (0.0008) [2023-12-27 01:09:31,799][105620] Updated weights for policy 1, policy_version 1343312 (0.0005) [2023-12-27 01:09:31,844][105692] Updated weights for policy 0, policy_version 1341344 (0.0009) [2023-12-27 01:09:31,860][105620] Updated weights for policy 1, policy_version 1343322 (0.0006) [2023-12-27 01:09:31,896][105692] Updated weights for policy 0, policy_version 1341354 (0.0008) [2023-12-27 01:09:32,564][105620] Updated weights for policy 1, policy_version 1343332 (0.0007) [2023-12-27 01:09:32,625][105620] Updated weights for policy 1, policy_version 1343342 (0.0008) [2023-12-27 01:09:32,684][105620] Updated weights for policy 1, policy_version 1343352 (0.0009) [2023-12-27 01:09:32,725][105692] Updated weights for policy 0, policy_version 1341364 (0.0009) [2023-12-27 01:09:32,784][105692] Updated weights for policy 0, policy_version 1341374 (0.0008) [2023-12-27 01:09:32,845][105692] Updated weights for policy 0, policy_version 1341384 (0.0009) [2023-12-27 01:09:33,445][105620] Updated weights for policy 1, policy_version 1343362 (0.0008) [2023-12-27 01:09:33,495][105620] Updated weights for policy 1, policy_version 1343372 (0.0009) [2023-12-27 01:09:33,542][105620] Updated weights for policy 1, policy_version 1343382 (0.0009) [2023-12-27 01:09:33,576][105692] Updated weights for policy 0, policy_version 1341394 (0.0008) [2023-12-27 01:09:33,587][105620] Updated weights for policy 1, policy_version 1343392 (0.0007) [2023-12-27 01:09:33,634][105692] Updated weights for policy 0, policy_version 1341404 (0.0009) [2023-12-27 01:09:33,684][105692] Updated weights for policy 0, policy_version 1341414 (0.0009) [2023-12-27 01:09:33,734][105692] Updated weights for policy 0, policy_version 1341424 (0.0009) [2023-12-27 01:09:34,271][105620] Updated weights for policy 1, policy_version 1343402 (0.0009) [2023-12-27 01:09:34,325][105620] Updated weights for policy 1, policy_version 1343412 (0.0009) [2023-12-27 01:09:34,386][105620] Updated weights for policy 1, policy_version 1343422 (0.0008) [2023-12-27 01:09:34,569][105692] Updated weights for policy 0, policy_version 1341434 (0.0009) [2023-12-27 01:09:34,628][105692] Updated weights for policy 0, policy_version 1341444 (0.0009) [2023-12-27 01:09:34,686][105692] Updated weights for policy 0, policy_version 1341454 (0.0009) [2023-12-27 01:09:35,111][105620] Updated weights for policy 1, policy_version 1343432 (0.0009) [2023-12-27 01:09:35,165][105620] Updated weights for policy 1, policy_version 1343442 (0.0008) [2023-12-27 01:09:35,224][105620] Updated weights for policy 1, policy_version 1343452 (0.0008) [2023-12-27 01:09:35,436][105692] Updated weights for policy 0, policy_version 1341464 (0.0009) [2023-12-27 01:09:35,486][105692] Updated weights for policy 0, policy_version 1341474 (0.0009) [2023-12-27 01:09:35,540][105692] Updated weights for policy 0, policy_version 1341484 (0.0009) [2023-12-27 01:09:35,963][105620] Updated weights for policy 1, policy_version 1343462 (0.0009) [2023-12-27 01:09:36,010][105620] Updated weights for policy 1, policy_version 1343472 (0.0009) [2023-12-27 01:09:36,061][105620] Updated weights for policy 1, policy_version 1343482 (0.0009) [2023-12-27 01:09:36,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 687439872. Throughput: 0: 9798.4, 1: 9866.2. Samples: 687434480. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:09:36,063][104569] Avg episode reward: [(0, '8545.810'), (1, '9082.438')] [2023-12-27 01:09:36,306][105692] Updated weights for policy 0, policy_version 1341494 (0.0007) [2023-12-27 01:09:36,362][105692] Updated weights for policy 0, policy_version 1341504 (0.0009) [2023-12-27 01:09:36,420][105692] Updated weights for policy 0, policy_version 1341514 (0.0010) [2023-12-27 01:09:36,774][105620] Updated weights for policy 1, policy_version 1343492 (0.0008) [2023-12-27 01:09:36,827][105620] Updated weights for policy 1, policy_version 1343502 (0.0008) [2023-12-27 01:09:36,886][105620] Updated weights for policy 1, policy_version 1343512 (0.0009) [2023-12-27 01:09:37,170][105692] Updated weights for policy 0, policy_version 1341524 (0.0008) [2023-12-27 01:09:37,231][105692] Updated weights for policy 0, policy_version 1341534 (0.0010) [2023-12-27 01:09:37,289][105692] Updated weights for policy 0, policy_version 1341544 (0.0009) [2023-12-27 01:09:37,558][105620] Updated weights for policy 1, policy_version 1343522 (0.0009) [2023-12-27 01:09:37,608][105620] Updated weights for policy 1, policy_version 1343532 (0.0008) [2023-12-27 01:09:37,660][105620] Updated weights for policy 1, policy_version 1343542 (0.0009) [2023-12-27 01:09:37,717][105620] Updated weights for policy 1, policy_version 1343552 (0.0009) [2023-12-27 01:09:37,994][105692] Updated weights for policy 0, policy_version 1341554 (0.0007) [2023-12-27 01:09:38,059][105692] Updated weights for policy 0, policy_version 1341564 (0.0009) [2023-12-27 01:09:38,114][105692] Updated weights for policy 0, policy_version 1341574 (0.0005) [2023-12-27 01:09:38,170][105692] Updated weights for policy 0, policy_version 1341584 (0.0005) [2023-12-27 01:09:38,518][105620] Updated weights for policy 1, policy_version 1343562 (0.0008) [2023-12-27 01:09:38,569][105620] Updated weights for policy 1, policy_version 1343572 (0.0008) [2023-12-27 01:09:38,617][105620] Updated weights for policy 1, policy_version 1343582 (0.0008) [2023-12-27 01:09:38,758][105692] Updated weights for policy 0, policy_version 1341594 (0.0011) [2023-12-27 01:09:38,813][105692] Updated weights for policy 0, policy_version 1341604 (0.0011) [2023-12-27 01:09:38,876][105692] Updated weights for policy 0, policy_version 1341614 (0.0011) [2023-12-27 01:09:39,427][105620] Updated weights for policy 1, policy_version 1343592 (0.0007) [2023-12-27 01:09:39,493][105620] Updated weights for policy 1, policy_version 1343602 (0.0008) [2023-12-27 01:09:39,549][105620] Updated weights for policy 1, policy_version 1343612 (0.0008) [2023-12-27 01:09:39,627][105692] Updated weights for policy 0, policy_version 1341624 (0.0011) [2023-12-27 01:09:39,690][105692] Updated weights for policy 0, policy_version 1341634 (0.0011) [2023-12-27 01:09:39,750][105692] Updated weights for policy 0, policy_version 1341644 (0.0011) [2023-12-27 01:09:40,322][105620] Updated weights for policy 1, policy_version 1343622 (0.0009) [2023-12-27 01:09:40,388][105620] Updated weights for policy 1, policy_version 1343632 (0.0010) [2023-12-27 01:09:40,431][105692] Updated weights for policy 0, policy_version 1341654 (0.0007) [2023-12-27 01:09:40,442][105620] Updated weights for policy 1, policy_version 1343642 (0.0009) [2023-12-27 01:09:40,495][105692] Updated weights for policy 0, policy_version 1341664 (0.0006) [2023-12-27 01:09:40,555][105692] Updated weights for policy 0, policy_version 1341674 (0.0006) [2023-12-27 01:09:41,062][104569] Fps is (10 sec: 18022.1, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 687538176. Throughput: 0: 9788.7, 1: 9716.5. Samples: 687548704. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:09:41,063][104569] Avg episode reward: [(0, '8458.769'), (1, '8814.316')] [2023-12-27 01:09:41,246][105692] Updated weights for policy 0, policy_version 1341684 (0.0007) [2023-12-27 01:09:41,249][105620] Updated weights for policy 1, policy_version 1343652 (0.0008) [2023-12-27 01:09:41,310][105692] Updated weights for policy 0, policy_version 1341694 (0.0008) [2023-12-27 01:09:41,312][105620] Updated weights for policy 1, policy_version 1343662 (0.0007) [2023-12-27 01:09:41,375][105692] Updated weights for policy 0, policy_version 1341704 (0.0010) [2023-12-27 01:09:41,377][105620] Updated weights for policy 1, policy_version 1343672 (0.0008) [2023-12-27 01:09:42,091][105692] Updated weights for policy 0, policy_version 1341714 (0.0007) [2023-12-27 01:09:42,118][105620] Updated weights for policy 1, policy_version 1343682 (0.0008) [2023-12-27 01:09:42,149][105692] Updated weights for policy 0, policy_version 1341724 (0.0006) [2023-12-27 01:09:42,176][105620] Updated weights for policy 1, policy_version 1343692 (0.0008) [2023-12-27 01:09:42,199][105692] Updated weights for policy 0, policy_version 1341734 (0.0006) [2023-12-27 01:09:42,233][105620] Updated weights for policy 1, policy_version 1343702 (0.0009) [2023-12-27 01:09:42,253][105692] Updated weights for policy 0, policy_version 1341744 (0.0007) [2023-12-27 01:09:42,292][105620] Updated weights for policy 1, policy_version 1343712 (0.0008) [2023-12-27 01:09:43,049][105692] Updated weights for policy 0, policy_version 1341754 (0.0008) [2023-12-27 01:09:43,071][105620] Updated weights for policy 1, policy_version 1343722 (0.0007) [2023-12-27 01:09:43,110][105692] Updated weights for policy 0, policy_version 1341764 (0.0007) [2023-12-27 01:09:43,128][105620] Updated weights for policy 1, policy_version 1343732 (0.0006) [2023-12-27 01:09:43,167][105692] Updated weights for policy 0, policy_version 1341774 (0.0007) [2023-12-27 01:09:43,185][105620] Updated weights for policy 1, policy_version 1343742 (0.0006) [2023-12-27 01:09:43,834][105692] Updated weights for policy 0, policy_version 1341784 (0.0008) [2023-12-27 01:09:43,897][105692] Updated weights for policy 0, policy_version 1341794 (0.0006) [2023-12-27 01:09:43,926][105620] Updated weights for policy 1, policy_version 1343752 (0.0009) [2023-12-27 01:09:43,955][105692] Updated weights for policy 0, policy_version 1341804 (0.0006) [2023-12-27 01:09:43,999][105620] Updated weights for policy 1, policy_version 1343762 (0.0007) [2023-12-27 01:09:44,052][105620] Updated weights for policy 1, policy_version 1343773 (0.0010) [2023-12-27 01:09:44,576][105692] Updated weights for policy 0, policy_version 1341814 (0.0007) [2023-12-27 01:09:44,635][105692] Updated weights for policy 0, policy_version 1341824 (0.0005) [2023-12-27 01:09:44,684][105692] Updated weights for policy 0, policy_version 1341834 (0.0005) [2023-12-27 01:09:44,891][105620] Updated weights for policy 1, policy_version 1343783 (0.0009) [2023-12-27 01:09:44,947][105620] Updated weights for policy 1, policy_version 1343793 (0.0009) [2023-12-27 01:09:44,998][105620] Updated weights for policy 1, policy_version 1343803 (0.0009) [2023-12-27 01:09:45,363][105692] Updated weights for policy 0, policy_version 1341844 (0.0007) [2023-12-27 01:09:45,425][105692] Updated weights for policy 0, policy_version 1341854 (0.0009) [2023-12-27 01:09:45,487][105692] Updated weights for policy 0, policy_version 1341864 (0.0006) [2023-12-27 01:09:45,734][105620] Updated weights for policy 1, policy_version 1343813 (0.0007) [2023-12-27 01:09:45,788][105620] Updated weights for policy 1, policy_version 1343823 (0.0005) [2023-12-27 01:09:45,845][105620] Updated weights for policy 1, policy_version 1343833 (0.0005) [2023-12-27 01:09:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 687636480. Throughput: 0: 9723.3, 1: 9653.6. Samples: 687604280. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:09:46,063][104569] Avg episode reward: [(0, '8455.340'), (1, '8816.961')] [2023-12-27 01:09:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001341872_343572480.pth... [2023-12-27 01:09:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001343840_344064000.pth... [2023-12-27 01:09:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001340720_343277568.pth [2023-12-27 01:09:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001342720_343777280.pth [2023-12-27 01:09:46,342][105692] Updated weights for policy 0, policy_version 1341874 (0.0010) [2023-12-27 01:09:46,390][105620] Updated weights for policy 1, policy_version 1343843 (0.0006) [2023-12-27 01:09:46,395][105692] Updated weights for policy 0, policy_version 1341884 (0.0008) [2023-12-27 01:09:46,442][105620] Updated weights for policy 1, policy_version 1343853 (0.0006) [2023-12-27 01:09:46,452][105692] Updated weights for policy 0, policy_version 1341894 (0.0007) [2023-12-27 01:09:46,500][105692] Updated weights for policy 0, policy_version 1341904 (0.0005) [2023-12-27 01:09:46,505][105620] Updated weights for policy 1, policy_version 1343863 (0.0008) [2023-12-27 01:09:47,165][105620] Updated weights for policy 1, policy_version 1343873 (0.0005) [2023-12-27 01:09:47,224][105620] Updated weights for policy 1, policy_version 1343883 (0.0007) [2023-12-27 01:09:47,270][105692] Updated weights for policy 0, policy_version 1341914 (0.0007) [2023-12-27 01:09:47,282][105620] Updated weights for policy 1, policy_version 1343893 (0.0007) [2023-12-27 01:09:47,329][105692] Updated weights for policy 0, policy_version 1341924 (0.0009) [2023-12-27 01:09:47,338][105620] Updated weights for policy 1, policy_version 1343903 (0.0007) [2023-12-27 01:09:47,397][105692] Updated weights for policy 0, policy_version 1341934 (0.0005) [2023-12-27 01:09:48,039][105620] Updated weights for policy 1, policy_version 1343913 (0.0008) [2023-12-27 01:09:48,051][105692] Updated weights for policy 0, policy_version 1341944 (0.0006) [2023-12-27 01:09:48,103][105620] Updated weights for policy 1, policy_version 1343923 (0.0009) [2023-12-27 01:09:48,110][105692] Updated weights for policy 0, policy_version 1341954 (0.0005) [2023-12-27 01:09:48,160][105620] Updated weights for policy 1, policy_version 1343933 (0.0009) [2023-12-27 01:09:48,166][105692] Updated weights for policy 0, policy_version 1341964 (0.0005) [2023-12-27 01:09:48,845][105692] Updated weights for policy 0, policy_version 1341974 (0.0010) [2023-12-27 01:09:48,903][105620] Updated weights for policy 1, policy_version 1343943 (0.0006) [2023-12-27 01:09:48,903][105692] Updated weights for policy 0, policy_version 1341984 (0.0009) [2023-12-27 01:09:48,955][105692] Updated weights for policy 0, policy_version 1341994 (0.0007) [2023-12-27 01:09:48,967][105620] Updated weights for policy 1, policy_version 1343953 (0.0008) [2023-12-27 01:09:49,023][105620] Updated weights for policy 1, policy_version 1343963 (0.0008) [2023-12-27 01:09:49,720][105620] Updated weights for policy 1, policy_version 1343973 (0.0009) [2023-12-27 01:09:49,766][105692] Updated weights for policy 0, policy_version 1342004 (0.0009) [2023-12-27 01:09:49,783][105620] Updated weights for policy 1, policy_version 1343983 (0.0008) [2023-12-27 01:09:49,827][105692] Updated weights for policy 0, policy_version 1342014 (0.0008) [2023-12-27 01:09:49,846][105620] Updated weights for policy 1, policy_version 1343993 (0.0007) [2023-12-27 01:09:49,892][105692] Updated weights for policy 0, policy_version 1342024 (0.0007) [2023-12-27 01:09:50,578][105692] Updated weights for policy 0, policy_version 1342034 (0.0007) [2023-12-27 01:09:50,581][105620] Updated weights for policy 1, policy_version 1344003 (0.0008) [2023-12-27 01:09:50,634][105692] Updated weights for policy 0, policy_version 1342044 (0.0008) [2023-12-27 01:09:50,647][105620] Updated weights for policy 1, policy_version 1344013 (0.0008) [2023-12-27 01:09:50,702][105692] Updated weights for policy 0, policy_version 1342054 (0.0008) [2023-12-27 01:09:50,709][105620] Updated weights for policy 1, policy_version 1344023 (0.0007) [2023-12-27 01:09:50,766][105692] Updated weights for policy 0, policy_version 1342064 (0.0008) [2023-12-27 01:09:51,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 687734784. Throughput: 0: 9686.2, 1: 9654.7. Samples: 687721220. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:09:51,062][104569] Avg episode reward: [(0, '8457.283'), (1, '8819.753')] [2023-12-27 01:09:51,424][105620] Updated weights for policy 1, policy_version 1344033 (0.0008) [2023-12-27 01:09:51,482][105620] Updated weights for policy 1, policy_version 1344043 (0.0008) [2023-12-27 01:09:51,544][105620] Updated weights for policy 1, policy_version 1344053 (0.0008) [2023-12-27 01:09:51,548][105692] Updated weights for policy 0, policy_version 1342074 (0.0009) [2023-12-27 01:09:51,599][105692] Updated weights for policy 0, policy_version 1342084 (0.0007) [2023-12-27 01:09:51,604][105620] Updated weights for policy 1, policy_version 1344063 (0.0008) [2023-12-27 01:09:51,667][105692] Updated weights for policy 0, policy_version 1342094 (0.0009) [2023-12-27 01:09:52,298][105620] Updated weights for policy 1, policy_version 1344073 (0.0009) [2023-12-27 01:09:52,354][105620] Updated weights for policy 1, policy_version 1344083 (0.0008) [2023-12-27 01:09:52,410][105620] Updated weights for policy 1, policy_version 1344093 (0.0009) [2023-12-27 01:09:52,438][105692] Updated weights for policy 0, policy_version 1342104 (0.0009) [2023-12-27 01:09:52,501][105692] Updated weights for policy 0, policy_version 1342114 (0.0009) [2023-12-27 01:09:52,554][105692] Updated weights for policy 0, policy_version 1342124 (0.0009) [2023-12-27 01:09:53,168][105620] Updated weights for policy 1, policy_version 1344103 (0.0009) [2023-12-27 01:09:53,226][105620] Updated weights for policy 1, policy_version 1344114 (0.0010) [2023-12-27 01:09:53,284][105620] Updated weights for policy 1, policy_version 1344124 (0.0009) [2023-12-27 01:09:53,287][105692] Updated weights for policy 0, policy_version 1342134 (0.0007) [2023-12-27 01:09:53,351][105692] Updated weights for policy 0, policy_version 1342144 (0.0005) [2023-12-27 01:09:53,399][105692] Updated weights for policy 0, policy_version 1342154 (0.0005) [2023-12-27 01:09:53,998][105620] Updated weights for policy 1, policy_version 1344134 (0.0009) [2023-12-27 01:09:54,049][105620] Updated weights for policy 1, policy_version 1344144 (0.0008) [2023-12-27 01:09:54,100][105620] Updated weights for policy 1, policy_version 1344154 (0.0007) [2023-12-27 01:09:54,102][105692] Updated weights for policy 0, policy_version 1342164 (0.0008) [2023-12-27 01:09:54,154][105692] Updated weights for policy 0, policy_version 1342174 (0.0008) [2023-12-27 01:09:54,220][105692] Updated weights for policy 0, policy_version 1342184 (0.0010) [2023-12-27 01:09:54,749][105620] Updated weights for policy 1, policy_version 1344164 (0.0005) [2023-12-27 01:09:54,802][105620] Updated weights for policy 1, policy_version 1344174 (0.0005) [2023-12-27 01:09:54,855][105620] Updated weights for policy 1, policy_version 1344184 (0.0006) [2023-12-27 01:09:54,945][105692] Updated weights for policy 0, policy_version 1342194 (0.0010) [2023-12-27 01:09:55,009][105692] Updated weights for policy 0, policy_version 1342204 (0.0011) [2023-12-27 01:09:55,070][105692] Updated weights for policy 0, policy_version 1342214 (0.0011) [2023-12-27 01:09:55,130][105692] Updated weights for policy 0, policy_version 1342224 (0.0011) [2023-12-27 01:09:55,470][105620] Updated weights for policy 1, policy_version 1344194 (0.0005) [2023-12-27 01:09:55,539][105620] Updated weights for policy 1, policy_version 1344204 (0.0005) [2023-12-27 01:09:55,594][105620] Updated weights for policy 1, policy_version 1344214 (0.0005) [2023-12-27 01:09:55,655][105620] Updated weights for policy 1, policy_version 1344224 (0.0005) [2023-12-27 01:09:55,831][105692] Updated weights for policy 0, policy_version 1342234 (0.0008) [2023-12-27 01:09:55,895][105692] Updated weights for policy 0, policy_version 1342244 (0.0006) [2023-12-27 01:09:55,964][105692] Updated weights for policy 0, policy_version 1342254 (0.0005) [2023-12-27 01:09:56,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 687833088. Throughput: 0: 9640.4, 1: 9696.1. Samples: 687839340. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:09:56,063][104569] Avg episode reward: [(0, '7645.771'), (1, '8913.801')] [2023-12-27 01:09:56,230][105620] Updated weights for policy 1, policy_version 1344234 (0.0011) [2023-12-27 01:09:56,283][105620] Updated weights for policy 1, policy_version 1344244 (0.0010) [2023-12-27 01:09:56,338][105620] Updated weights for policy 1, policy_version 1344254 (0.0010) [2023-12-27 01:09:56,567][105692] Updated weights for policy 0, policy_version 1342264 (0.0009) [2023-12-27 01:09:56,611][105692] Updated weights for policy 0, policy_version 1342274 (0.0010) [2023-12-27 01:09:56,665][105692] Updated weights for policy 0, policy_version 1342284 (0.0006) [2023-12-27 01:09:57,062][105620] Updated weights for policy 1, policy_version 1344264 (0.0010) [2023-12-27 01:09:57,117][105620] Updated weights for policy 1, policy_version 1344274 (0.0006) [2023-12-27 01:09:57,172][105620] Updated weights for policy 1, policy_version 1344284 (0.0006) [2023-12-27 01:09:57,226][105692] Updated weights for policy 0, policy_version 1342294 (0.0008) [2023-12-27 01:09:57,273][105692] Updated weights for policy 0, policy_version 1342304 (0.0010) [2023-12-27 01:09:57,335][105692] Updated weights for policy 0, policy_version 1342314 (0.0009) [2023-12-27 01:09:57,845][105620] Updated weights for policy 1, policy_version 1344294 (0.0007) [2023-12-27 01:09:57,904][105620] Updated weights for policy 1, policy_version 1344304 (0.0006) [2023-12-27 01:09:57,921][105692] Updated weights for policy 0, policy_version 1342324 (0.0006) [2023-12-27 01:09:57,970][105620] Updated weights for policy 1, policy_version 1344314 (0.0007) [2023-12-27 01:09:57,979][105692] Updated weights for policy 0, policy_version 1342334 (0.0006) [2023-12-27 01:09:58,032][105692] Updated weights for policy 0, policy_version 1342344 (0.0007) [2023-12-27 01:09:58,654][105620] Updated weights for policy 1, policy_version 1344324 (0.0007) [2023-12-27 01:09:58,717][105620] Updated weights for policy 1, policy_version 1344334 (0.0008) [2023-12-27 01:09:58,784][105620] Updated weights for policy 1, policy_version 1344344 (0.0008) [2023-12-27 01:09:58,814][105692] Updated weights for policy 0, policy_version 1342354 (0.0008) [2023-12-27 01:09:58,883][105692] Updated weights for policy 0, policy_version 1342364 (0.0008) [2023-12-27 01:09:58,945][105692] Updated weights for policy 0, policy_version 1342374 (0.0006) [2023-12-27 01:09:58,999][105692] Updated weights for policy 0, policy_version 1342384 (0.0005) [2023-12-27 01:09:59,585][105620] Updated weights for policy 1, policy_version 1344354 (0.0008) [2023-12-27 01:09:59,658][105620] Updated weights for policy 1, policy_version 1344364 (0.0007) [2023-12-27 01:09:59,691][105692] Updated weights for policy 0, policy_version 1342394 (0.0010) [2023-12-27 01:09:59,718][105620] Updated weights for policy 1, policy_version 1344374 (0.0006) [2023-12-27 01:09:59,754][105692] Updated weights for policy 0, policy_version 1342404 (0.0011) [2023-12-27 01:09:59,777][105620] Updated weights for policy 1, policy_version 1344384 (0.0006) [2023-12-27 01:09:59,820][105692] Updated weights for policy 0, policy_version 1342414 (0.0009) [2023-12-27 01:10:00,388][105620] Updated weights for policy 1, policy_version 1344394 (0.0006) [2023-12-27 01:10:00,444][105620] Updated weights for policy 1, policy_version 1344404 (0.0005) [2023-12-27 01:10:00,461][105692] Updated weights for policy 0, policy_version 1342424 (0.0010) [2023-12-27 01:10:00,504][105620] Updated weights for policy 1, policy_version 1344414 (0.0005) [2023-12-27 01:10:00,519][105692] Updated weights for policy 0, policy_version 1342434 (0.0010) [2023-12-27 01:10:00,576][105692] Updated weights for policy 0, policy_version 1342444 (0.0010) [2023-12-27 01:10:01,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 687931392. Throughput: 0: 9775.7, 1: 9703.3. Samples: 687901516. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:10:01,062][104569] Avg episode reward: [(0, '7466.094'), (1, '9093.515')] [2023-12-27 01:10:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001342448_343719936.pth... [2023-12-27 01:10:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001344416_344211456.pth... [2023-12-27 01:10:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001341328_343433216.pth [2023-12-27 01:10:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001343296_343924736.pth [2023-12-27 01:10:01,185][105620] Updated weights for policy 1, policy_version 1344424 (0.0008) [2023-12-27 01:10:01,238][105620] Updated weights for policy 1, policy_version 1344434 (0.0008) [2023-12-27 01:10:01,301][105620] Updated weights for policy 1, policy_version 1344444 (0.0008) [2023-12-27 01:10:01,331][105692] Updated weights for policy 0, policy_version 1342454 (0.0009) [2023-12-27 01:10:01,398][105692] Updated weights for policy 0, policy_version 1342464 (0.0009) [2023-12-27 01:10:01,435][105585] KL-divergence is very high: 132.9260 [2023-12-27 01:10:01,452][105692] Updated weights for policy 0, policy_version 1342474 (0.0009) [2023-12-27 01:10:01,475][105585] KL-divergence is very high: 119.9533 [2023-12-27 01:10:02,082][105620] Updated weights for policy 1, policy_version 1344454 (0.0008) [2023-12-27 01:10:02,134][105620] Updated weights for policy 1, policy_version 1344464 (0.0008) [2023-12-27 01:10:02,186][105620] Updated weights for policy 1, policy_version 1344474 (0.0008) [2023-12-27 01:10:02,214][105692] Updated weights for policy 0, policy_version 1342484 (0.0007) [2023-12-27 01:10:02,283][105692] Updated weights for policy 0, policy_version 1342494 (0.0008) [2023-12-27 01:10:02,343][105692] Updated weights for policy 0, policy_version 1342504 (0.0010) [2023-12-27 01:10:02,892][105620] Updated weights for policy 1, policy_version 1344484 (0.0008) [2023-12-27 01:10:02,954][105620] Updated weights for policy 1, policy_version 1344494 (0.0008) [2023-12-27 01:10:03,004][105620] Updated weights for policy 1, policy_version 1344504 (0.0008) [2023-12-27 01:10:03,015][105692] Updated weights for policy 0, policy_version 1342514 (0.0010) [2023-12-27 01:10:03,071][105692] Updated weights for policy 0, policy_version 1342524 (0.0011) [2023-12-27 01:10:03,136][105692] Updated weights for policy 0, policy_version 1342534 (0.0010) [2023-12-27 01:10:03,202][105692] Updated weights for policy 0, policy_version 1342544 (0.0009) [2023-12-27 01:10:03,745][105620] Updated weights for policy 1, policy_version 1344514 (0.0007) [2023-12-27 01:10:03,793][105620] Updated weights for policy 1, policy_version 1344524 (0.0010) [2023-12-27 01:10:03,845][105620] Updated weights for policy 1, policy_version 1344534 (0.0010) [2023-12-27 01:10:03,880][105692] Updated weights for policy 0, policy_version 1342554 (0.0008) [2023-12-27 01:10:03,901][105620] Updated weights for policy 1, policy_version 1344544 (0.0010) [2023-12-27 01:10:03,945][105692] Updated weights for policy 0, policy_version 1342564 (0.0006) [2023-12-27 01:10:04,004][105692] Updated weights for policy 0, policy_version 1342574 (0.0011) [2023-12-27 01:10:04,656][105620] Updated weights for policy 1, policy_version 1344554 (0.0010) [2023-12-27 01:10:04,703][105620] Updated weights for policy 1, policy_version 1344564 (0.0009) [2023-12-27 01:10:04,723][105692] Updated weights for policy 0, policy_version 1342584 (0.0006) [2023-12-27 01:10:04,767][105620] Updated weights for policy 1, policy_version 1344574 (0.0006) [2023-12-27 01:10:04,781][105692] Updated weights for policy 0, policy_version 1342594 (0.0006) [2023-12-27 01:10:04,830][105692] Updated weights for policy 0, policy_version 1342604 (0.0005) [2023-12-27 01:10:05,341][105620] Updated weights for policy 1, policy_version 1344584 (0.0010) [2023-12-27 01:10:05,394][105620] Updated weights for policy 1, policy_version 1344594 (0.0010) [2023-12-27 01:10:05,443][105620] Updated weights for policy 1, policy_version 1344604 (0.0010) [2023-12-27 01:10:05,572][105692] Updated weights for policy 0, policy_version 1342614 (0.0009) [2023-12-27 01:10:05,631][105692] Updated weights for policy 0, policy_version 1342624 (0.0010) [2023-12-27 01:10:05,692][105692] Updated weights for policy 0, policy_version 1342634 (0.0010) [2023-12-27 01:10:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 688029696. Throughput: 0: 9809.7, 1: 9764.9. Samples: 688018764. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:10:06,063][104569] Avg episode reward: [(0, '7550.108'), (1, '9091.441')] [2023-12-27 01:10:06,128][105620] Updated weights for policy 1, policy_version 1344614 (0.0009) [2023-12-27 01:10:06,198][105620] Updated weights for policy 1, policy_version 1344624 (0.0010) [2023-12-27 01:10:06,255][105620] Updated weights for policy 1, policy_version 1344634 (0.0011) [2023-12-27 01:10:06,321][105692] Updated weights for policy 0, policy_version 1342644 (0.0008) [2023-12-27 01:10:06,381][105692] Updated weights for policy 0, policy_version 1342654 (0.0005) [2023-12-27 01:10:06,442][105692] Updated weights for policy 0, policy_version 1342664 (0.0007) [2023-12-27 01:10:06,923][105620] Updated weights for policy 1, policy_version 1344644 (0.0011) [2023-12-27 01:10:06,986][105620] Updated weights for policy 1, policy_version 1344654 (0.0011) [2023-12-27 01:10:07,039][105620] Updated weights for policy 1, policy_version 1344664 (0.0010) [2023-12-27 01:10:07,048][105692] Updated weights for policy 0, policy_version 1342674 (0.0010) [2023-12-27 01:10:07,102][105692] Updated weights for policy 0, policy_version 1342684 (0.0007) [2023-12-27 01:10:07,162][105692] Updated weights for policy 0, policy_version 1342694 (0.0008) [2023-12-27 01:10:07,224][105692] Updated weights for policy 0, policy_version 1342704 (0.0010) [2023-12-27 01:10:07,806][105620] Updated weights for policy 1, policy_version 1344674 (0.0011) [2023-12-27 01:10:07,869][105620] Updated weights for policy 1, policy_version 1344684 (0.0010) [2023-12-27 01:10:07,892][105692] Updated weights for policy 0, policy_version 1342714 (0.0007) [2023-12-27 01:10:07,928][105620] Updated weights for policy 1, policy_version 1344694 (0.0010) [2023-12-27 01:10:07,939][105692] Updated weights for policy 0, policy_version 1342724 (0.0007) [2023-12-27 01:10:07,981][105620] Updated weights for policy 1, policy_version 1344704 (0.0010) [2023-12-27 01:10:07,984][105692] Updated weights for policy 0, policy_version 1342734 (0.0007) [2023-12-27 01:10:08,637][105620] Updated weights for policy 1, policy_version 1344714 (0.0011) [2023-12-27 01:10:08,692][105692] Updated weights for policy 0, policy_version 1342744 (0.0007) [2023-12-27 01:10:08,697][105620] Updated weights for policy 1, policy_version 1344724 (0.0011) [2023-12-27 01:10:08,747][105692] Updated weights for policy 0, policy_version 1342754 (0.0005) [2023-12-27 01:10:08,750][105620] Updated weights for policy 1, policy_version 1344734 (0.0011) [2023-12-27 01:10:08,796][105692] Updated weights for policy 0, policy_version 1342764 (0.0005) [2023-12-27 01:10:09,438][105692] Updated weights for policy 0, policy_version 1342774 (0.0007) [2023-12-27 01:10:09,501][105692] Updated weights for policy 0, policy_version 1342784 (0.0007) [2023-12-27 01:10:09,564][105692] Updated weights for policy 0, policy_version 1342794 (0.0010) [2023-12-27 01:10:09,572][105620] Updated weights for policy 1, policy_version 1344744 (0.0011) [2023-12-27 01:10:09,636][105620] Updated weights for policy 1, policy_version 1344754 (0.0011) [2023-12-27 01:10:09,702][105620] Updated weights for policy 1, policy_version 1344764 (0.0011) [2023-12-27 01:10:10,306][105692] Updated weights for policy 0, policy_version 1342804 (0.0011) [2023-12-27 01:10:10,365][105692] Updated weights for policy 0, policy_version 1342814 (0.0011) [2023-12-27 01:10:10,396][105620] Updated weights for policy 1, policy_version 1344774 (0.0008) [2023-12-27 01:10:10,418][105692] Updated weights for policy 0, policy_version 1342824 (0.0011) [2023-12-27 01:10:10,449][105620] Updated weights for policy 1, policy_version 1344784 (0.0006) [2023-12-27 01:10:10,497][105620] Updated weights for policy 1, policy_version 1344794 (0.0008) [2023-12-27 01:10:11,053][105692] Updated weights for policy 0, policy_version 1342834 (0.0010) [2023-12-27 01:10:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19388.2, 300 sec: 19549.7). Total num frames: 688128000. Throughput: 0: 9813.7, 1: 9755.4. Samples: 688138924. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:10:11,063][104569] Avg episode reward: [(0, '8087.244'), (1, '9175.184')] [2023-12-27 01:10:11,126][105692] Updated weights for policy 0, policy_version 1342844 (0.0008) [2023-12-27 01:10:11,192][105692] Updated weights for policy 0, policy_version 1342854 (0.0008) [2023-12-27 01:10:11,258][105692] Updated weights for policy 0, policy_version 1342864 (0.0008) [2023-12-27 01:10:11,303][105620] Updated weights for policy 1, policy_version 1344804 (0.0008) [2023-12-27 01:10:11,367][105620] Updated weights for policy 1, policy_version 1344814 (0.0008) [2023-12-27 01:10:11,443][105620] Updated weights for policy 1, policy_version 1344824 (0.0010) [2023-12-27 01:10:11,981][105692] Updated weights for policy 0, policy_version 1342874 (0.0006) [2023-12-27 01:10:12,031][105692] Updated weights for policy 0, policy_version 1342884 (0.0005) [2023-12-27 01:10:12,086][105692] Updated weights for policy 0, policy_version 1342894 (0.0006) [2023-12-27 01:10:12,271][105620] Updated weights for policy 1, policy_version 1344834 (0.0009) [2023-12-27 01:10:12,336][105620] Updated weights for policy 1, policy_version 1344844 (0.0009) [2023-12-27 01:10:12,412][105620] Updated weights for policy 1, policy_version 1344854 (0.0009) [2023-12-27 01:10:12,480][105620] Updated weights for policy 1, policy_version 1344864 (0.0009) [2023-12-27 01:10:12,773][105692] Updated weights for policy 0, policy_version 1342904 (0.0010) [2023-12-27 01:10:12,825][105692] Updated weights for policy 0, policy_version 1342914 (0.0010) [2023-12-27 01:10:12,886][105692] Updated weights for policy 0, policy_version 1342924 (0.0011) [2023-12-27 01:10:13,232][105620] Updated weights for policy 1, policy_version 1344874 (0.0007) [2023-12-27 01:10:13,292][105620] Updated weights for policy 1, policy_version 1344884 (0.0008) [2023-12-27 01:10:13,348][105620] Updated weights for policy 1, policy_version 1344894 (0.0008) [2023-12-27 01:10:13,585][105692] Updated weights for policy 0, policy_version 1342934 (0.0009) [2023-12-27 01:10:13,633][105692] Updated weights for policy 0, policy_version 1342944 (0.0010) [2023-12-27 01:10:13,688][105692] Updated weights for policy 0, policy_version 1342954 (0.0010) [2023-12-27 01:10:13,984][105620] Updated weights for policy 1, policy_version 1344904 (0.0009) [2023-12-27 01:10:14,053][105620] Updated weights for policy 1, policy_version 1344914 (0.0007) [2023-12-27 01:10:14,107][105620] Updated weights for policy 1, policy_version 1344924 (0.0008) [2023-12-27 01:10:14,297][105692] Updated weights for policy 0, policy_version 1342964 (0.0010) [2023-12-27 01:10:14,348][105692] Updated weights for policy 0, policy_version 1342974 (0.0010) [2023-12-27 01:10:14,406][105692] Updated weights for policy 0, policy_version 1342984 (0.0009) [2023-12-27 01:10:14,841][105620] Updated weights for policy 1, policy_version 1344934 (0.0007) [2023-12-27 01:10:14,900][105620] Updated weights for policy 1, policy_version 1344944 (0.0006) [2023-12-27 01:10:14,963][105620] Updated weights for policy 1, policy_version 1344954 (0.0006) [2023-12-27 01:10:15,097][105692] Updated weights for policy 0, policy_version 1342994 (0.0006) [2023-12-27 01:10:15,160][105692] Updated weights for policy 0, policy_version 1343004 (0.0011) [2023-12-27 01:10:15,227][105692] Updated weights for policy 0, policy_version 1343014 (0.0011) [2023-12-27 01:10:15,294][105692] Updated weights for policy 0, policy_version 1343024 (0.0011) [2023-12-27 01:10:15,677][105620] Updated weights for policy 1, policy_version 1344964 (0.0007) [2023-12-27 01:10:15,736][105620] Updated weights for policy 1, policy_version 1344974 (0.0008) [2023-12-27 01:10:15,790][105620] Updated weights for policy 1, policy_version 1344984 (0.0008) [2023-12-27 01:10:16,015][105692] Updated weights for policy 0, policy_version 1343034 (0.0005) [2023-12-27 01:10:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 688226304. Throughput: 0: 9806.0, 1: 9724.0. Samples: 688197096. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:10:16,062][104569] Avg episode reward: [(0, '8372.223'), (1, '9171.940')] [2023-12-27 01:10:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001344992_344358912.pth... [2023-12-27 01:10:16,068][105692] Updated weights for policy 0, policy_version 1343044 (0.0005) [2023-12-27 01:10:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001343840_344064000.pth [2023-12-27 01:10:16,123][105692] Updated weights for policy 0, policy_version 1343054 (0.0006) [2023-12-27 01:10:16,136][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001343056_343875584.pth... [2023-12-27 01:10:16,139][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001341872_343572480.pth [2023-12-27 01:10:16,352][105620] Updated weights for policy 1, policy_version 1344994 (0.0007) [2023-12-27 01:10:16,418][105620] Updated weights for policy 1, policy_version 1345004 (0.0006) [2023-12-27 01:10:16,489][105620] Updated weights for policy 1, policy_version 1345014 (0.0005) [2023-12-27 01:10:16,551][105620] Updated weights for policy 1, policy_version 1345024 (0.0009) [2023-12-27 01:10:16,824][105692] Updated weights for policy 0, policy_version 1343064 (0.0006) [2023-12-27 01:10:16,873][105692] Updated weights for policy 0, policy_version 1343074 (0.0005) [2023-12-27 01:10:16,924][105692] Updated weights for policy 0, policy_version 1343084 (0.0005) [2023-12-27 01:10:17,277][105620] Updated weights for policy 1, policy_version 1345034 (0.0008) [2023-12-27 01:10:17,324][105620] Updated weights for policy 1, policy_version 1345044 (0.0009) [2023-12-27 01:10:17,371][105620] Updated weights for policy 1, policy_version 1345054 (0.0008) [2023-12-27 01:10:17,542][105692] Updated weights for policy 0, policy_version 1343094 (0.0007) [2023-12-27 01:10:17,601][105692] Updated weights for policy 0, policy_version 1343104 (0.0009) [2023-12-27 01:10:17,652][105692] Updated weights for policy 0, policy_version 1343114 (0.0009) [2023-12-27 01:10:18,115][105620] Updated weights for policy 1, policy_version 1345064 (0.0007) [2023-12-27 01:10:18,178][105620] Updated weights for policy 1, policy_version 1345074 (0.0006) [2023-12-27 01:10:18,224][105620] Updated weights for policy 1, policy_version 1345084 (0.0005) [2023-12-27 01:10:18,446][105692] Updated weights for policy 0, policy_version 1343124 (0.0010) [2023-12-27 01:10:18,506][105692] Updated weights for policy 0, policy_version 1343134 (0.0008) [2023-12-27 01:10:18,574][105692] Updated weights for policy 0, policy_version 1343144 (0.0009) [2023-12-27 01:10:18,920][105620] Updated weights for policy 1, policy_version 1345094 (0.0008) [2023-12-27 01:10:18,972][105620] Updated weights for policy 1, policy_version 1345104 (0.0010) [2023-12-27 01:10:19,040][105620] Updated weights for policy 1, policy_version 1345114 (0.0010) [2023-12-27 01:10:19,261][105692] Updated weights for policy 0, policy_version 1343154 (0.0006) [2023-12-27 01:10:19,327][105692] Updated weights for policy 0, policy_version 1343164 (0.0008) [2023-12-27 01:10:19,395][105692] Updated weights for policy 0, policy_version 1343174 (0.0008) [2023-12-27 01:10:19,443][105692] Updated weights for policy 0, policy_version 1343184 (0.0005) [2023-12-27 01:10:19,814][105620] Updated weights for policy 1, policy_version 1345124 (0.0008) [2023-12-27 01:10:19,880][105620] Updated weights for policy 1, policy_version 1345134 (0.0009) [2023-12-27 01:10:19,945][105620] Updated weights for policy 1, policy_version 1345144 (0.0008) [2023-12-27 01:10:20,227][105692] Updated weights for policy 0, policy_version 1343194 (0.0009) [2023-12-27 01:10:20,274][105692] Updated weights for policy 0, policy_version 1343204 (0.0009) [2023-12-27 01:10:20,337][105692] Updated weights for policy 0, policy_version 1343214 (0.0009) [2023-12-27 01:10:20,713][105620] Updated weights for policy 1, policy_version 1345154 (0.0010) [2023-12-27 01:10:20,775][105620] Updated weights for policy 1, policy_version 1345164 (0.0009) [2023-12-27 01:10:20,841][105620] Updated weights for policy 1, policy_version 1345174 (0.0010) [2023-12-27 01:10:20,910][105620] Updated weights for policy 1, policy_version 1345184 (0.0009) [2023-12-27 01:10:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 688324608. Throughput: 0: 9891.7, 1: 9688.6. Samples: 688315588. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:10:21,063][104569] Avg episode reward: [(0, '8370.650'), (1, '8999.244')] [2023-12-27 01:10:21,070][105692] Updated weights for policy 0, policy_version 1343224 (0.0009) [2023-12-27 01:10:21,132][105692] Updated weights for policy 0, policy_version 1343234 (0.0008) [2023-12-27 01:10:21,191][105692] Updated weights for policy 0, policy_version 1343244 (0.0010) [2023-12-27 01:10:21,712][105620] Updated weights for policy 1, policy_version 1345194 (0.0008) [2023-12-27 01:10:21,786][105620] Updated weights for policy 1, policy_version 1345204 (0.0008) [2023-12-27 01:10:21,852][105620] Updated weights for policy 1, policy_version 1345214 (0.0008) [2023-12-27 01:10:21,989][105692] Updated weights for policy 0, policy_version 1343254 (0.0008) [2023-12-27 01:10:22,053][105692] Updated weights for policy 0, policy_version 1343264 (0.0007) [2023-12-27 01:10:22,117][105692] Updated weights for policy 0, policy_version 1343274 (0.0008) [2023-12-27 01:10:22,577][105620] Updated weights for policy 1, policy_version 1345224 (0.0008) [2023-12-27 01:10:22,638][105620] Updated weights for policy 1, policy_version 1345234 (0.0008) [2023-12-27 01:10:22,696][105620] Updated weights for policy 1, policy_version 1345244 (0.0009) [2023-12-27 01:10:22,892][105692] Updated weights for policy 0, policy_version 1343284 (0.0007) [2023-12-27 01:10:22,959][105692] Updated weights for policy 0, policy_version 1343294 (0.0007) [2023-12-27 01:10:23,022][105692] Updated weights for policy 0, policy_version 1343304 (0.0009) [2023-12-27 01:10:23,405][105620] Updated weights for policy 1, policy_version 1345254 (0.0007) [2023-12-27 01:10:23,462][105620] Updated weights for policy 1, policy_version 1345264 (0.0006) [2023-12-27 01:10:23,517][105620] Updated weights for policy 1, policy_version 1345274 (0.0006) [2023-12-27 01:10:23,700][105692] Updated weights for policy 0, policy_version 1343314 (0.0009) [2023-12-27 01:10:23,758][105692] Updated weights for policy 0, policy_version 1343324 (0.0010) [2023-12-27 01:10:23,812][105692] Updated weights for policy 0, policy_version 1343334 (0.0010) [2023-12-27 01:10:23,860][105692] Updated weights for policy 0, policy_version 1343344 (0.0010) [2023-12-27 01:10:24,042][105620] Updated weights for policy 1, policy_version 1345284 (0.0005) [2023-12-27 01:10:24,091][105620] Updated weights for policy 1, policy_version 1345294 (0.0005) [2023-12-27 01:10:24,136][105620] Updated weights for policy 1, policy_version 1345304 (0.0008) [2023-12-27 01:10:24,491][105692] Updated weights for policy 0, policy_version 1343354 (0.0008) [2023-12-27 01:10:24,550][105692] Updated weights for policy 0, policy_version 1343364 (0.0011) [2023-12-27 01:10:24,606][105692] Updated weights for policy 0, policy_version 1343374 (0.0011) [2023-12-27 01:10:24,795][105620] Updated weights for policy 1, policy_version 1345314 (0.0010) [2023-12-27 01:10:24,843][105620] Updated weights for policy 1, policy_version 1345324 (0.0010) [2023-12-27 01:10:24,895][105620] Updated weights for policy 1, policy_version 1345334 (0.0010) [2023-12-27 01:10:24,943][105620] Updated weights for policy 1, policy_version 1345344 (0.0010) [2023-12-27 01:10:25,214][105692] Updated weights for policy 0, policy_version 1343384 (0.0009) [2023-12-27 01:10:25,258][105692] Updated weights for policy 0, policy_version 1343394 (0.0007) [2023-12-27 01:10:25,309][105692] Updated weights for policy 0, policy_version 1343404 (0.0008) [2023-12-27 01:10:25,689][105620] Updated weights for policy 1, policy_version 1345354 (0.0009) [2023-12-27 01:10:25,753][105620] Updated weights for policy 1, policy_version 1345364 (0.0010) [2023-12-27 01:10:25,807][105620] Updated weights for policy 1, policy_version 1345374 (0.0010) [2023-12-27 01:10:25,964][105692] Updated weights for policy 0, policy_version 1343414 (0.0009) [2023-12-27 01:10:26,022][105692] Updated weights for policy 0, policy_version 1343424 (0.0010) [2023-12-27 01:10:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 688422912. Throughput: 0: 9886.8, 1: 9771.8. Samples: 688433336. Policy #0 lag: (min: 23.0, avg: 23.2, max: 35.0) [2023-12-27 01:10:26,062][104569] Avg episode reward: [(0, '8095.879'), (1, '8725.239')] [2023-12-27 01:10:26,068][105692] Updated weights for policy 0, policy_version 1343434 (0.0008) [2023-12-27 01:10:26,407][105620] Updated weights for policy 1, policy_version 1345384 (0.0005) [2023-12-27 01:10:26,469][105620] Updated weights for policy 1, policy_version 1345394 (0.0008) [2023-12-27 01:10:26,527][105620] Updated weights for policy 1, policy_version 1345404 (0.0010) [2023-12-27 01:10:26,940][105692] Updated weights for policy 0, policy_version 1343445 (0.0009) [2023-12-27 01:10:26,994][105692] Updated weights for policy 0, policy_version 1343456 (0.0010) [2023-12-27 01:10:27,008][105585] KL-divergence is very high: 133.0772 [2023-12-27 01:10:27,014][105585] KL-divergence is very high: 162.7701 [2023-12-27 01:10:27,059][105692] Updated weights for policy 0, policy_version 1343467 (0.0010) [2023-12-27 01:10:27,059][105585] KL-divergence is very high: 112.3147 [2023-12-27 01:10:27,066][105585] KL-divergence is very high: 129.5475 [2023-12-27 01:10:27,084][105620] Updated weights for policy 1, policy_version 1345414 (0.0007) [2023-12-27 01:10:27,135][105620] Updated weights for policy 1, policy_version 1345424 (0.0010) [2023-12-27 01:10:27,191][105620] Updated weights for policy 1, policy_version 1345434 (0.0009) [2023-12-27 01:10:27,799][105620] Updated weights for policy 1, policy_version 1345444 (0.0007) [2023-12-27 01:10:27,850][105620] Updated weights for policy 1, policy_version 1345454 (0.0006) [2023-12-27 01:10:27,897][105692] Updated weights for policy 0, policy_version 1343477 (0.0007) [2023-12-27 01:10:27,916][105620] Updated weights for policy 1, policy_version 1345464 (0.0005) [2023-12-27 01:10:27,955][105692] Updated weights for policy 0, policy_version 1343487 (0.0005) [2023-12-27 01:10:27,995][105585] KL-divergence is very high: 115.1034 [2023-12-27 01:10:28,014][105585] KL-divergence is very high: 129.7262 [2023-12-27 01:10:28,015][105692] Updated weights for policy 0, policy_version 1343497 (0.0008) [2023-12-27 01:10:28,029][105585] KL-divergence is very high: 170.3508 [2023-12-27 01:10:28,035][105585] KL-divergence is very high: 241.3558 [2023-12-27 01:10:28,041][105585] KL-divergence is very high: 282.2912 [2023-12-27 01:10:28,560][105585] KL-divergence is very high: 129.1165 [2023-12-27 01:10:28,574][105692] Updated weights for policy 0, policy_version 1343507 (0.0006) [2023-12-27 01:10:28,589][105620] Updated weights for policy 1, policy_version 1345474 (0.0005) [2023-12-27 01:10:28,598][105585] KL-divergence is very high: 229.9634 [2023-12-27 01:10:28,610][105585] KL-divergence is very high: 104.6698 [2023-12-27 01:10:28,629][105692] Updated weights for policy 0, policy_version 1343517 (0.0009) [2023-12-27 01:10:28,640][105585] KL-divergence is very high: 213.9824 [2023-12-27 01:10:28,646][105620] Updated weights for policy 1, policy_version 1345484 (0.0008) [2023-12-27 01:10:28,684][105585] KL-divergence is very high: 193.7334 [2023-12-27 01:10:28,685][105692] Updated weights for policy 0, policy_version 1343527 (0.0006) [2023-12-27 01:10:28,704][105620] Updated weights for policy 1, policy_version 1345494 (0.0005) [2023-12-27 01:10:28,730][105585] KL-divergence is very high: 167.1550 [2023-12-27 01:10:28,762][105620] Updated weights for policy 1, policy_version 1345504 (0.0008) [2023-12-27 01:10:29,382][105620] Updated weights for policy 1, policy_version 1345514 (0.0011) [2023-12-27 01:10:29,433][105692] Updated weights for policy 0, policy_version 1343537 (0.0008) [2023-12-27 01:10:29,439][105620] Updated weights for policy 1, policy_version 1345524 (0.0010) [2023-12-27 01:10:29,482][105692] Updated weights for policy 0, policy_version 1343547 (0.0005) [2023-12-27 01:10:29,498][105620] Updated weights for policy 1, policy_version 1345534 (0.0010) [2023-12-27 01:10:29,531][105692] Updated weights for policy 0, policy_version 1343557 (0.0008) [2023-12-27 01:10:29,576][105692] Updated weights for policy 0, policy_version 1343567 (0.0008) [2023-12-27 01:10:30,202][105620] Updated weights for policy 1, policy_version 1345544 (0.0011) [2023-12-27 01:10:30,265][105620] Updated weights for policy 1, policy_version 1345554 (0.0009) [2023-12-27 01:10:30,325][105620] Updated weights for policy 1, policy_version 1345564 (0.0011) [2023-12-27 01:10:30,397][105692] Updated weights for policy 0, policy_version 1343577 (0.0008) [2023-12-27 01:10:30,463][105692] Updated weights for policy 0, policy_version 1343587 (0.0007) [2023-12-27 01:10:30,523][105692] Updated weights for policy 0, policy_version 1343597 (0.0008) [2023-12-27 01:10:30,979][105620] Updated weights for policy 1, policy_version 1345574 (0.0008) [2023-12-27 01:10:31,024][105620] Updated weights for policy 1, policy_version 1345584 (0.0006) [2023-12-27 01:10:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 688521216. Throughput: 0: 9906.2, 1: 9903.2. Samples: 688495696. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:10:31,063][104569] Avg episode reward: [(0, '7287.007'), (1, '8819.301')] [2023-12-27 01:10:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001343600_344014848.pth... [2023-12-27 01:10:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001342448_343719936.pth [2023-12-27 01:10:31,088][105620] Updated weights for policy 1, policy_version 1345594 (0.0009) [2023-12-27 01:10:31,113][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001345600_344514560.pth... [2023-12-27 01:10:31,118][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001344416_344211456.pth [2023-12-27 01:10:31,293][105692] Updated weights for policy 0, policy_version 1343607 (0.0008) [2023-12-27 01:10:31,348][105692] Updated weights for policy 0, policy_version 1343617 (0.0009) [2023-12-27 01:10:31,411][105692] Updated weights for policy 0, policy_version 1343627 (0.0009) [2023-12-27 01:10:31,827][105620] Updated weights for policy 1, policy_version 1345604 (0.0008) [2023-12-27 01:10:31,895][105620] Updated weights for policy 1, policy_version 1345614 (0.0005) [2023-12-27 01:10:31,959][105620] Updated weights for policy 1, policy_version 1345624 (0.0005) [2023-12-27 01:10:32,210][105692] Updated weights for policy 0, policy_version 1343637 (0.0008) [2023-12-27 01:10:32,279][105692] Updated weights for policy 0, policy_version 1343647 (0.0008) [2023-12-27 01:10:32,330][105692] Updated weights for policy 0, policy_version 1343657 (0.0005) [2023-12-27 01:10:32,614][105620] Updated weights for policy 1, policy_version 1345634 (0.0005) [2023-12-27 01:10:32,678][105620] Updated weights for policy 1, policy_version 1345644 (0.0005) [2023-12-27 01:10:32,745][105620] Updated weights for policy 1, policy_version 1345654 (0.0006) [2023-12-27 01:10:32,805][105620] Updated weights for policy 1, policy_version 1345664 (0.0010) [2023-12-27 01:10:33,023][105692] Updated weights for policy 0, policy_version 1343667 (0.0007) [2023-12-27 01:10:33,076][105692] Updated weights for policy 0, policy_version 1343677 (0.0008) [2023-12-27 01:10:33,130][105692] Updated weights for policy 0, policy_version 1343687 (0.0009) [2023-12-27 01:10:33,413][105620] Updated weights for policy 1, policy_version 1345675 (0.0007) [2023-12-27 01:10:33,464][105620] Updated weights for policy 1, policy_version 1345685 (0.0005) [2023-12-27 01:10:33,515][105620] Updated weights for policy 1, policy_version 1345695 (0.0005) [2023-12-27 01:10:33,933][105692] Updated weights for policy 0, policy_version 1343699 (0.0009) [2023-12-27 01:10:33,995][105692] Updated weights for policy 0, policy_version 1343709 (0.0005) [2023-12-27 01:10:34,046][105692] Updated weights for policy 0, policy_version 1343719 (0.0005) [2023-12-27 01:10:34,095][105620] Updated weights for policy 1, policy_version 1345705 (0.0010) [2023-12-27 01:10:34,152][105620] Updated weights for policy 1, policy_version 1345715 (0.0010) [2023-12-27 01:10:34,215][105620] Updated weights for policy 1, policy_version 1345725 (0.0008) [2023-12-27 01:10:34,632][105692] Updated weights for policy 0, policy_version 1343729 (0.0006) [2023-12-27 01:10:34,690][105692] Updated weights for policy 0, policy_version 1343739 (0.0009) [2023-12-27 01:10:34,742][105692] Updated weights for policy 0, policy_version 1343749 (0.0010) [2023-12-27 01:10:34,796][105692] Updated weights for policy 0, policy_version 1343759 (0.0006) [2023-12-27 01:10:34,917][105620] Updated weights for policy 1, policy_version 1345735 (0.0006) [2023-12-27 01:10:34,966][105620] Updated weights for policy 1, policy_version 1345745 (0.0010) [2023-12-27 01:10:35,028][105620] Updated weights for policy 1, policy_version 1345755 (0.0011) [2023-12-27 01:10:35,533][105692] Updated weights for policy 0, policy_version 1343769 (0.0007) [2023-12-27 01:10:35,595][105692] Updated weights for policy 0, policy_version 1343779 (0.0008) [2023-12-27 01:10:35,657][105692] Updated weights for policy 0, policy_version 1343789 (0.0008) [2023-12-27 01:10:35,751][105620] Updated weights for policy 1, policy_version 1345765 (0.0010) [2023-12-27 01:10:35,805][105620] Updated weights for policy 1, policy_version 1345775 (0.0010) [2023-12-27 01:10:35,860][105620] Updated weights for policy 1, policy_version 1345785 (0.0010) [2023-12-27 01:10:36,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19797.3, 300 sec: 19605.2). Total num frames: 688627712. Throughput: 0: 9869.7, 1: 9993.1. Samples: 688615052. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:10:36,063][104569] Avg episode reward: [(0, '7520.456'), (1, '9089.901')] [2023-12-27 01:10:36,372][105692] Updated weights for policy 0, policy_version 1343799 (0.0006) [2023-12-27 01:10:36,424][105692] Updated weights for policy 0, policy_version 1343809 (0.0006) [2023-12-27 01:10:36,484][105692] Updated weights for policy 0, policy_version 1343819 (0.0009) [2023-12-27 01:10:36,541][105620] Updated weights for policy 1, policy_version 1345795 (0.0009) [2023-12-27 01:10:36,605][105620] Updated weights for policy 1, policy_version 1345805 (0.0009) [2023-12-27 01:10:36,671][105620] Updated weights for policy 1, policy_version 1345815 (0.0011) [2023-12-27 01:10:37,189][105692] Updated weights for policy 0, policy_version 1343829 (0.0009) [2023-12-27 01:10:37,238][105692] Updated weights for policy 0, policy_version 1343839 (0.0008) [2023-12-27 01:10:37,291][105692] Updated weights for policy 0, policy_version 1343849 (0.0008) [2023-12-27 01:10:37,379][105620] Updated weights for policy 1, policy_version 1345825 (0.0011) [2023-12-27 01:10:37,438][105620] Updated weights for policy 1, policy_version 1345835 (0.0006) [2023-12-27 01:10:37,488][105620] Updated weights for policy 1, policy_version 1345845 (0.0009) [2023-12-27 01:10:37,551][105620] Updated weights for policy 1, policy_version 1345855 (0.0011) [2023-12-27 01:10:38,097][105692] Updated weights for policy 0, policy_version 1343859 (0.0008) [2023-12-27 01:10:38,149][105692] Updated weights for policy 0, policy_version 1343869 (0.0008) [2023-12-27 01:10:38,201][105692] Updated weights for policy 0, policy_version 1343879 (0.0008) [2023-12-27 01:10:38,276][105620] Updated weights for policy 1, policy_version 1345865 (0.0007) [2023-12-27 01:10:38,339][105620] Updated weights for policy 1, policy_version 1345875 (0.0006) [2023-12-27 01:10:38,407][105620] Updated weights for policy 1, policy_version 1345885 (0.0006) [2023-12-27 01:10:38,958][105692] Updated weights for policy 0, policy_version 1343889 (0.0008) [2023-12-27 01:10:39,021][105692] Updated weights for policy 0, policy_version 1343899 (0.0008) [2023-12-27 01:10:39,080][105692] Updated weights for policy 0, policy_version 1343909 (0.0007) [2023-12-27 01:10:39,101][105620] Updated weights for policy 1, policy_version 1345895 (0.0010) [2023-12-27 01:10:39,132][105692] Updated weights for policy 0, policy_version 1343919 (0.0009) [2023-12-27 01:10:39,163][105620] Updated weights for policy 1, policy_version 1345905 (0.0010) [2023-12-27 01:10:39,227][105620] Updated weights for policy 1, policy_version 1345915 (0.0012) [2023-12-27 01:10:39,928][105585] KL-divergence is very high: 247.1054 [2023-12-27 01:10:39,934][105585] KL-divergence is very high: 232.8855 [2023-12-27 01:10:39,935][105692] Updated weights for policy 0, policy_version 1343929 (0.0008) [2023-12-27 01:10:39,947][105585] KL-divergence is very high: 349.5480 [2023-12-27 01:10:39,964][105620] Updated weights for policy 1, policy_version 1345925 (0.0010) [2023-12-27 01:10:39,979][105585] KL-divergence is very high: 671.8763 [2023-12-27 01:10:39,987][105585] KL-divergence is very high: 533.9384 [2023-12-27 01:10:39,999][105692] Updated weights for policy 0, policy_version 1343939 (0.0006) [2023-12-27 01:10:40,000][105585] KL-divergence is very high: 628.6163 [2023-12-27 01:10:40,020][105620] Updated weights for policy 1, policy_version 1345935 (0.0007) [2023-12-27 01:10:40,029][105585] KL-divergence is very high: 775.0559 [2023-12-27 01:10:40,035][105585] KL-divergence is very high: 578.4766 [2023-12-27 01:10:40,047][105585] KL-divergence is very high: 650.2834 [2023-12-27 01:10:40,059][105692] Updated weights for policy 0, policy_version 1343949 (0.0009) [2023-12-27 01:10:40,079][105620] Updated weights for policy 1, policy_version 1345945 (0.0006) [2023-12-27 01:10:40,655][105692] Updated weights for policy 0, policy_version 1343959 (0.0006) [2023-12-27 01:10:40,715][105692] Updated weights for policy 0, policy_version 1343969 (0.0005) [2023-12-27 01:10:40,766][105692] Updated weights for policy 0, policy_version 1343979 (0.0009) [2023-12-27 01:10:40,926][105620] Updated weights for policy 1, policy_version 1345955 (0.0007) [2023-12-27 01:10:40,986][105620] Updated weights for policy 1, policy_version 1345965 (0.0005) [2023-12-27 01:10:41,059][105620] Updated weights for policy 1, policy_version 1345975 (0.0008) [2023-12-27 01:10:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 688717824. Throughput: 0: 9872.7, 1: 9909.9. Samples: 688729556. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:10:41,062][104569] Avg episode reward: [(0, '8052.776'), (1, '8909.296')] [2023-12-27 01:10:41,483][105692] Updated weights for policy 0, policy_version 1343989 (0.0007) [2023-12-27 01:10:41,540][105692] Updated weights for policy 0, policy_version 1343999 (0.0005) [2023-12-27 01:10:41,600][105692] Updated weights for policy 0, policy_version 1344009 (0.0006) [2023-12-27 01:10:41,850][105620] Updated weights for policy 1, policy_version 1345985 (0.0009) [2023-12-27 01:10:41,913][105620] Updated weights for policy 1, policy_version 1345995 (0.0010) [2023-12-27 01:10:41,985][105620] Updated weights for policy 1, policy_version 1346005 (0.0010) [2023-12-27 01:10:42,058][105620] Updated weights for policy 1, policy_version 1346015 (0.0009) [2023-12-27 01:10:42,238][105692] Updated weights for policy 0, policy_version 1344019 (0.0009) [2023-12-27 01:10:42,301][105692] Updated weights for policy 0, policy_version 1344029 (0.0010) [2023-12-27 01:10:42,366][105692] Updated weights for policy 0, policy_version 1344039 (0.0008) [2023-12-27 01:10:42,766][105620] Updated weights for policy 1, policy_version 1346025 (0.0009) [2023-12-27 01:10:42,813][105620] Updated weights for policy 1, policy_version 1346035 (0.0008) [2023-12-27 01:10:42,859][105620] Updated weights for policy 1, policy_version 1346045 (0.0008) [2023-12-27 01:10:43,096][105692] Updated weights for policy 0, policy_version 1344049 (0.0010) [2023-12-27 01:10:43,159][105692] Updated weights for policy 0, policy_version 1344059 (0.0009) [2023-12-27 01:10:43,219][105692] Updated weights for policy 0, policy_version 1344069 (0.0009) [2023-12-27 01:10:43,270][105692] Updated weights for policy 0, policy_version 1344079 (0.0009) [2023-12-27 01:10:43,645][105620] Updated weights for policy 1, policy_version 1346055 (0.0008) [2023-12-27 01:10:43,698][105620] Updated weights for policy 1, policy_version 1346065 (0.0008) [2023-12-27 01:10:43,748][105620] Updated weights for policy 1, policy_version 1346075 (0.0008) [2023-12-27 01:10:44,027][105692] Updated weights for policy 0, policy_version 1344089 (0.0009) [2023-12-27 01:10:44,082][105692] Updated weights for policy 0, policy_version 1344099 (0.0009) [2023-12-27 01:10:44,141][105692] Updated weights for policy 0, policy_version 1344109 (0.0009) [2023-12-27 01:10:44,529][105620] Updated weights for policy 1, policy_version 1346085 (0.0007) [2023-12-27 01:10:44,593][105620] Updated weights for policy 1, policy_version 1346095 (0.0009) [2023-12-27 01:10:44,646][105620] Updated weights for policy 1, policy_version 1346105 (0.0009) [2023-12-27 01:10:44,791][105692] Updated weights for policy 0, policy_version 1344119 (0.0009) [2023-12-27 01:10:44,853][105692] Updated weights for policy 0, policy_version 1344129 (0.0009) [2023-12-27 01:10:44,912][105692] Updated weights for policy 0, policy_version 1344139 (0.0009) [2023-12-27 01:10:45,407][105620] Updated weights for policy 1, policy_version 1346115 (0.0009) [2023-12-27 01:10:45,465][105620] Updated weights for policy 1, policy_version 1346125 (0.0009) [2023-12-27 01:10:45,530][105620] Updated weights for policy 1, policy_version 1346135 (0.0008) [2023-12-27 01:10:45,670][105692] Updated weights for policy 0, policy_version 1344149 (0.0007) [2023-12-27 01:10:45,720][105692] Updated weights for policy 0, policy_version 1344159 (0.0005) [2023-12-27 01:10:45,781][105692] Updated weights for policy 0, policy_version 1344169 (0.0006) [2023-12-27 01:10:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 688816128. Throughput: 0: 9797.7, 1: 9846.3. Samples: 688785496. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:10:46,063][104569] Avg episode reward: [(0, '8282.653'), (1, '8817.839')] [2023-12-27 01:10:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001346144_344653824.pth... [2023-12-27 01:10:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001344176_344162304.pth... [2023-12-27 01:10:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001344992_344358912.pth [2023-12-27 01:10:46,081][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001343056_343875584.pth [2023-12-27 01:10:46,198][105620] Updated weights for policy 1, policy_version 1346145 (0.0009) [2023-12-27 01:10:46,248][105620] Updated weights for policy 1, policy_version 1346155 (0.0007) [2023-12-27 01:10:46,296][105620] Updated weights for policy 1, policy_version 1346165 (0.0006) [2023-12-27 01:10:46,349][105620] Updated weights for policy 1, policy_version 1346175 (0.0008) [2023-12-27 01:10:46,439][105692] Updated weights for policy 0, policy_version 1344179 (0.0007) [2023-12-27 01:10:46,504][105692] Updated weights for policy 0, policy_version 1344189 (0.0010) [2023-12-27 01:10:46,566][105692] Updated weights for policy 0, policy_version 1344199 (0.0009) [2023-12-27 01:10:46,935][105620] Updated weights for policy 1, policy_version 1346185 (0.0007) [2023-12-27 01:10:46,995][105620] Updated weights for policy 1, policy_version 1346195 (0.0008) [2023-12-27 01:10:47,049][105620] Updated weights for policy 1, policy_version 1346205 (0.0009) [2023-12-27 01:10:47,391][105692] Updated weights for policy 0, policy_version 1344209 (0.0010) [2023-12-27 01:10:47,463][105692] Updated weights for policy 0, policy_version 1344219 (0.0010) [2023-12-27 01:10:47,532][105692] Updated weights for policy 0, policy_version 1344229 (0.0010) [2023-12-27 01:10:47,585][105692] Updated weights for policy 0, policy_version 1344239 (0.0008) [2023-12-27 01:10:47,621][105620] Updated weights for policy 1, policy_version 1346215 (0.0007) [2023-12-27 01:10:47,682][105620] Updated weights for policy 1, policy_version 1346225 (0.0006) [2023-12-27 01:10:47,738][105620] Updated weights for policy 1, policy_version 1346235 (0.0009) [2023-12-27 01:10:48,315][105620] Updated weights for policy 1, policy_version 1346245 (0.0010) [2023-12-27 01:10:48,379][105620] Updated weights for policy 1, policy_version 1346255 (0.0010) [2023-12-27 01:10:48,442][105620] Updated weights for policy 1, policy_version 1346265 (0.0010) [2023-12-27 01:10:48,453][105692] Updated weights for policy 0, policy_version 1344249 (0.0007) [2023-12-27 01:10:48,515][105692] Updated weights for policy 0, policy_version 1344259 (0.0006) [2023-12-27 01:10:48,569][105692] Updated weights for policy 0, policy_version 1344269 (0.0008) [2023-12-27 01:10:49,095][105620] Updated weights for policy 1, policy_version 1346275 (0.0009) [2023-12-27 01:10:49,158][105620] Updated weights for policy 1, policy_version 1346285 (0.0005) [2023-12-27 01:10:49,229][105620] Updated weights for policy 1, policy_version 1346295 (0.0007) [2023-12-27 01:10:49,390][105692] Updated weights for policy 0, policy_version 1344279 (0.0010) [2023-12-27 01:10:49,455][105692] Updated weights for policy 0, policy_version 1344289 (0.0009) [2023-12-27 01:10:49,516][105692] Updated weights for policy 0, policy_version 1344299 (0.0009) [2023-12-27 01:10:49,931][105620] Updated weights for policy 1, policy_version 1346305 (0.0007) [2023-12-27 01:10:49,998][105620] Updated weights for policy 1, policy_version 1346315 (0.0006) [2023-12-27 01:10:50,065][105620] Updated weights for policy 1, policy_version 1346325 (0.0011) [2023-12-27 01:10:50,127][105620] Updated weights for policy 1, policy_version 1346335 (0.0010) [2023-12-27 01:10:50,319][105692] Updated weights for policy 0, policy_version 1344309 (0.0009) [2023-12-27 01:10:50,375][105692] Updated weights for policy 0, policy_version 1344319 (0.0008) [2023-12-27 01:10:50,426][105692] Updated weights for policy 0, policy_version 1344329 (0.0009) [2023-12-27 01:10:50,856][105620] Updated weights for policy 1, policy_version 1346345 (0.0011) [2023-12-27 01:10:50,920][105620] Updated weights for policy 1, policy_version 1346355 (0.0011) [2023-12-27 01:10:50,976][105620] Updated weights for policy 1, policy_version 1346365 (0.0011) [2023-12-27 01:10:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 688914432. Throughput: 0: 9721.1, 1: 9944.5. Samples: 688903716. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:10:51,062][104569] Avg episode reward: [(0, '8186.057'), (1, '9086.123')] [2023-12-27 01:10:51,216][105692] Updated weights for policy 0, policy_version 1344339 (0.0007) [2023-12-27 01:10:51,281][105692] Updated weights for policy 0, policy_version 1344349 (0.0009) [2023-12-27 01:10:51,340][105692] Updated weights for policy 0, policy_version 1344359 (0.0010) [2023-12-27 01:10:51,670][105620] Updated weights for policy 1, policy_version 1346375 (0.0009) [2023-12-27 01:10:51,741][105620] Updated weights for policy 1, policy_version 1346385 (0.0008) [2023-12-27 01:10:51,777][105586] KL-divergence is very high: 163.3895 [2023-12-27 01:10:51,808][105620] Updated weights for policy 1, policy_version 1346395 (0.0008) [2023-12-27 01:10:51,826][105586] KL-divergence is very high: 181.8412 [2023-12-27 01:10:52,158][105692] Updated weights for policy 0, policy_version 1344369 (0.0009) [2023-12-27 01:10:52,214][105692] Updated weights for policy 0, policy_version 1344379 (0.0008) [2023-12-27 01:10:52,276][105692] Updated weights for policy 0, policy_version 1344389 (0.0007) [2023-12-27 01:10:52,335][105692] Updated weights for policy 0, policy_version 1344399 (0.0006) [2023-12-27 01:10:52,534][105620] Updated weights for policy 1, policy_version 1346405 (0.0009) [2023-12-27 01:10:52,595][105620] Updated weights for policy 1, policy_version 1346415 (0.0009) [2023-12-27 01:10:52,650][105620] Updated weights for policy 1, policy_version 1346425 (0.0008) [2023-12-27 01:10:53,068][105692] Updated weights for policy 0, policy_version 1344409 (0.0009) [2023-12-27 01:10:53,119][105692] Updated weights for policy 0, policy_version 1344419 (0.0007) [2023-12-27 01:10:53,183][105692] Updated weights for policy 0, policy_version 1344429 (0.0006) [2023-12-27 01:10:53,428][105620] Updated weights for policy 1, policy_version 1346435 (0.0010) [2023-12-27 01:10:53,491][105620] Updated weights for policy 1, policy_version 1346445 (0.0006) [2023-12-27 01:10:53,558][105620] Updated weights for policy 1, policy_version 1346455 (0.0006) [2023-12-27 01:10:53,944][105692] Updated weights for policy 0, policy_version 1344439 (0.0009) [2023-12-27 01:10:53,952][105585] KL-divergence is very high: 183.1754 [2023-12-27 01:10:54,004][105585] KL-divergence is very high: 409.7197 [2023-12-27 01:10:54,010][105692] Updated weights for policy 0, policy_version 1344449 (0.0009) [2023-12-27 01:10:54,043][105585] KL-divergence is very high: 472.3612 [2023-12-27 01:10:54,060][105692] Updated weights for policy 0, policy_version 1344459 (0.0009) [2023-12-27 01:10:54,174][105620] Updated weights for policy 1, policy_version 1346465 (0.0007) [2023-12-27 01:10:54,240][105620] Updated weights for policy 1, policy_version 1346475 (0.0009) [2023-12-27 01:10:54,305][105620] Updated weights for policy 1, policy_version 1346485 (0.0009) [2023-12-27 01:10:54,359][105620] Updated weights for policy 1, policy_version 1346495 (0.0010) [2023-12-27 01:10:54,721][105692] Updated weights for policy 0, policy_version 1344470 (0.0010) [2023-12-27 01:10:54,777][105692] Updated weights for policy 0, policy_version 1344480 (0.0011) [2023-12-27 01:10:54,833][105692] Updated weights for policy 0, policy_version 1344490 (0.0011) [2023-12-27 01:10:55,097][105620] Updated weights for policy 1, policy_version 1346505 (0.0007) [2023-12-27 01:10:55,159][105620] Updated weights for policy 1, policy_version 1346515 (0.0008) [2023-12-27 01:10:55,219][105620] Updated weights for policy 1, policy_version 1346525 (0.0007) [2023-12-27 01:10:55,515][105692] Updated weights for policy 0, policy_version 1344500 (0.0009) [2023-12-27 01:10:55,573][105692] Updated weights for policy 0, policy_version 1344510 (0.0010) [2023-12-27 01:10:55,640][105692] Updated weights for policy 0, policy_version 1344520 (0.0010) [2023-12-27 01:10:55,904][105620] Updated weights for policy 1, policy_version 1346535 (0.0005) [2023-12-27 01:10:55,959][105620] Updated weights for policy 1, policy_version 1346545 (0.0005) [2023-12-27 01:10:56,011][105620] Updated weights for policy 1, policy_version 1346555 (0.0008) [2023-12-27 01:10:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 689012736. Throughput: 0: 9620.5, 1: 9915.1. Samples: 689018028. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:10:56,063][104569] Avg episode reward: [(0, '8000.625'), (1, '8908.452')] [2023-12-27 01:10:56,234][105692] Updated weights for policy 0, policy_version 1344530 (0.0009) [2023-12-27 01:10:56,284][105692] Updated weights for policy 0, policy_version 1344540 (0.0006) [2023-12-27 01:10:56,334][105692] Updated weights for policy 0, policy_version 1344550 (0.0005) [2023-12-27 01:10:56,402][105692] Updated weights for policy 0, policy_version 1344560 (0.0005) [2023-12-27 01:10:56,844][105620] Updated weights for policy 1, policy_version 1346565 (0.0010) [2023-12-27 01:10:56,897][105620] Updated weights for policy 1, policy_version 1346575 (0.0008) [2023-12-27 01:10:56,918][105692] Updated weights for policy 0, policy_version 1344570 (0.0005) [2023-12-27 01:10:56,965][105620] Updated weights for policy 1, policy_version 1346585 (0.0008) [2023-12-27 01:10:56,986][105692] Updated weights for policy 0, policy_version 1344580 (0.0006) [2023-12-27 01:10:57,041][105692] Updated weights for policy 0, policy_version 1344590 (0.0009) [2023-12-27 01:10:57,603][105692] Updated weights for policy 0, policy_version 1344600 (0.0006) [2023-12-27 01:10:57,649][105692] Updated weights for policy 0, policy_version 1344610 (0.0005) [2023-12-27 01:10:57,695][105692] Updated weights for policy 0, policy_version 1344620 (0.0005) [2023-12-27 01:10:57,744][105620] Updated weights for policy 1, policy_version 1346595 (0.0008) [2023-12-27 01:10:57,806][105620] Updated weights for policy 1, policy_version 1346605 (0.0010) [2023-12-27 01:10:57,872][105620] Updated weights for policy 1, policy_version 1346615 (0.0010) [2023-12-27 01:10:58,369][105692] Updated weights for policy 0, policy_version 1344630 (0.0008) [2023-12-27 01:10:58,441][105692] Updated weights for policy 0, policy_version 1344640 (0.0008) [2023-12-27 01:10:58,504][105692] Updated weights for policy 0, policy_version 1344650 (0.0009) [2023-12-27 01:10:58,639][105620] Updated weights for policy 1, policy_version 1346625 (0.0009) [2023-12-27 01:10:58,702][105620] Updated weights for policy 1, policy_version 1346635 (0.0009) [2023-12-27 01:10:58,769][105620] Updated weights for policy 1, policy_version 1346645 (0.0008) [2023-12-27 01:10:58,834][105620] Updated weights for policy 1, policy_version 1346655 (0.0010) [2023-12-27 01:10:59,339][105692] Updated weights for policy 0, policy_version 1344660 (0.0009) [2023-12-27 01:10:59,401][105692] Updated weights for policy 0, policy_version 1344670 (0.0009) [2023-12-27 01:10:59,454][105692] Updated weights for policy 0, policy_version 1344681 (0.0009) [2023-12-27 01:10:59,597][105620] Updated weights for policy 1, policy_version 1346665 (0.0007) [2023-12-27 01:10:59,660][105620] Updated weights for policy 1, policy_version 1346675 (0.0009) [2023-12-27 01:10:59,709][105620] Updated weights for policy 1, policy_version 1346685 (0.0008) [2023-12-27 01:11:00,224][105692] Updated weights for policy 0, policy_version 1344691 (0.0010) [2023-12-27 01:11:00,286][105692] Updated weights for policy 0, policy_version 1344701 (0.0010) [2023-12-27 01:11:00,345][105692] Updated weights for policy 0, policy_version 1344711 (0.0010) [2023-12-27 01:11:00,416][105620] Updated weights for policy 1, policy_version 1346695 (0.0007) [2023-12-27 01:11:00,462][105620] Updated weights for policy 1, policy_version 1346705 (0.0005) [2023-12-27 01:11:00,510][105620] Updated weights for policy 1, policy_version 1346715 (0.0006) [2023-12-27 01:11:01,051][105692] Updated weights for policy 0, policy_version 1344721 (0.0010) [2023-12-27 01:11:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 689102848. Throughput: 0: 9686.5, 1: 9873.0. Samples: 689077272. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:11:01,063][104569] Avg episode reward: [(0, '8185.965'), (1, '8911.244')] [2023-12-27 01:11:01,097][105620] Updated weights for policy 1, policy_version 1346725 (0.0008) [2023-12-27 01:11:01,118][105692] Updated weights for policy 0, policy_version 1344731 (0.0008) [2023-12-27 01:11:01,166][105620] Updated weights for policy 1, policy_version 1346735 (0.0010) [2023-12-27 01:11:01,184][105692] Updated weights for policy 0, policy_version 1344741 (0.0010) [2023-12-27 01:11:01,225][105620] Updated weights for policy 1, policy_version 1346745 (0.0010) [2023-12-27 01:11:01,240][105692] Updated weights for policy 0, policy_version 1344751 (0.0011) [2023-12-27 01:11:01,242][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001344752_344309760.pth... [2023-12-27 01:11:01,247][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001343600_344014848.pth [2023-12-27 01:11:01,265][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001346752_344809472.pth... [2023-12-27 01:11:01,269][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001345600_344514560.pth [2023-12-27 01:11:01,910][105620] Updated weights for policy 1, policy_version 1346755 (0.0009) [2023-12-27 01:11:01,970][105620] Updated weights for policy 1, policy_version 1346765 (0.0005) [2023-12-27 01:11:02,011][105692] Updated weights for policy 0, policy_version 1344761 (0.0009) [2023-12-27 01:11:02,023][105620] Updated weights for policy 1, policy_version 1346775 (0.0005) [2023-12-27 01:11:02,076][105692] Updated weights for policy 0, policy_version 1344771 (0.0008) [2023-12-27 01:11:02,133][105692] Updated weights for policy 0, policy_version 1344781 (0.0007) [2023-12-27 01:11:02,711][105620] Updated weights for policy 1, policy_version 1346785 (0.0007) [2023-12-27 01:11:02,773][105620] Updated weights for policy 1, policy_version 1346795 (0.0010) [2023-12-27 01:11:02,795][105692] Updated weights for policy 0, policy_version 1344791 (0.0005) [2023-12-27 01:11:02,828][105620] Updated weights for policy 1, policy_version 1346805 (0.0010) [2023-12-27 01:11:02,847][105692] Updated weights for policy 0, policy_version 1344801 (0.0005) [2023-12-27 01:11:02,879][105620] Updated weights for policy 1, policy_version 1346815 (0.0010) [2023-12-27 01:11:02,902][105692] Updated weights for policy 0, policy_version 1344811 (0.0006) [2023-12-27 01:11:03,601][105586] KL-divergence is very high: 115.4718 [2023-12-27 01:11:03,609][105620] Updated weights for policy 1, policy_version 1346825 (0.0011) [2023-12-27 01:11:03,644][105586] KL-divergence is very high: 142.6799 [2023-12-27 01:11:03,660][105620] Updated weights for policy 1, policy_version 1346835 (0.0010) [2023-12-27 01:11:03,671][105692] Updated weights for policy 0, policy_version 1344821 (0.0008) [2023-12-27 01:11:03,687][105586] KL-divergence is very high: 124.3803 [2023-12-27 01:11:03,718][105620] Updated weights for policy 1, policy_version 1346845 (0.0011) [2023-12-27 01:11:03,729][105692] Updated weights for policy 0, policy_version 1344831 (0.0006) [2023-12-27 01:11:03,785][105692] Updated weights for policy 0, policy_version 1344841 (0.0008) [2023-12-27 01:11:04,485][105620] Updated weights for policy 1, policy_version 1346855 (0.0009) [2023-12-27 01:11:04,525][105692] Updated weights for policy 0, policy_version 1344851 (0.0009) [2023-12-27 01:11:04,549][105620] Updated weights for policy 1, policy_version 1346865 (0.0009) [2023-12-27 01:11:04,589][105692] Updated weights for policy 0, policy_version 1344861 (0.0007) [2023-12-27 01:11:04,603][105620] Updated weights for policy 1, policy_version 1346875 (0.0010) [2023-12-27 01:11:04,647][105692] Updated weights for policy 0, policy_version 1344871 (0.0007) [2023-12-27 01:11:05,271][105620] Updated weights for policy 1, policy_version 1346885 (0.0008) [2023-12-27 01:11:05,317][105620] Updated weights for policy 1, policy_version 1346895 (0.0008) [2023-12-27 01:11:05,363][105620] Updated weights for policy 1, policy_version 1346905 (0.0009) [2023-12-27 01:11:05,445][105692] Updated weights for policy 0, policy_version 1344881 (0.0009) [2023-12-27 01:11:05,510][105692] Updated weights for policy 0, policy_version 1344891 (0.0009) [2023-12-27 01:11:05,563][105692] Updated weights for policy 0, policy_version 1344901 (0.0006) [2023-12-27 01:11:05,626][105692] Updated weights for policy 0, policy_version 1344911 (0.0005) [2023-12-27 01:11:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 689201152. Throughput: 0: 9590.9, 1: 9905.4. Samples: 689192924. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:11:06,062][104569] Avg episode reward: [(0, '8185.786'), (1, '8552.820')] [2023-12-27 01:11:06,125][105620] Updated weights for policy 1, policy_version 1346915 (0.0008) [2023-12-27 01:11:06,183][105620] Updated weights for policy 1, policy_version 1346925 (0.0006) [2023-12-27 01:11:06,250][105620] Updated weights for policy 1, policy_version 1346935 (0.0005) [2023-12-27 01:11:06,353][105692] Updated weights for policy 0, policy_version 1344921 (0.0009) [2023-12-27 01:11:06,419][105692] Updated weights for policy 0, policy_version 1344931 (0.0009) [2023-12-27 01:11:06,485][105692] Updated weights for policy 0, policy_version 1344941 (0.0009) [2023-12-27 01:11:06,892][105620] Updated weights for policy 1, policy_version 1346945 (0.0008) [2023-12-27 01:11:06,955][105620] Updated weights for policy 1, policy_version 1346955 (0.0005) [2023-12-27 01:11:07,011][105620] Updated weights for policy 1, policy_version 1346965 (0.0006) [2023-12-27 01:11:07,057][105620] Updated weights for policy 1, policy_version 1346975 (0.0007) [2023-12-27 01:11:07,167][105692] Updated weights for policy 0, policy_version 1344951 (0.0010) [2023-12-27 01:11:07,222][105692] Updated weights for policy 0, policy_version 1344961 (0.0009) [2023-12-27 01:11:07,269][105692] Updated weights for policy 0, policy_version 1344971 (0.0008) [2023-12-27 01:11:07,695][105620] Updated weights for policy 1, policy_version 1346985 (0.0009) [2023-12-27 01:11:07,743][105620] Updated weights for policy 1, policy_version 1346995 (0.0010) [2023-12-27 01:11:07,792][105620] Updated weights for policy 1, policy_version 1347005 (0.0010) [2023-12-27 01:11:08,066][105692] Updated weights for policy 0, policy_version 1344981 (0.0009) [2023-12-27 01:11:08,126][105692] Updated weights for policy 0, policy_version 1344991 (0.0009) [2023-12-27 01:11:08,191][105692] Updated weights for policy 0, policy_version 1345001 (0.0009) [2023-12-27 01:11:08,552][105620] Updated weights for policy 1, policy_version 1347015 (0.0010) [2023-12-27 01:11:08,617][105620] Updated weights for policy 1, policy_version 1347025 (0.0009) [2023-12-27 01:11:08,675][105620] Updated weights for policy 1, policy_version 1347035 (0.0008) [2023-12-27 01:11:08,957][105692] Updated weights for policy 0, policy_version 1345011 (0.0009) [2023-12-27 01:11:09,020][105692] Updated weights for policy 0, policy_version 1345021 (0.0009) [2023-12-27 01:11:09,071][105692] Updated weights for policy 0, policy_version 1345031 (0.0009) [2023-12-27 01:11:09,446][105620] Updated weights for policy 1, policy_version 1347045 (0.0009) [2023-12-27 01:11:09,510][105620] Updated weights for policy 1, policy_version 1347055 (0.0009) [2023-12-27 01:11:09,576][105620] Updated weights for policy 1, policy_version 1347065 (0.0007) [2023-12-27 01:11:09,884][105692] Updated weights for policy 0, policy_version 1345041 (0.0009) [2023-12-27 01:11:09,952][105692] Updated weights for policy 0, policy_version 1345051 (0.0009) [2023-12-27 01:11:10,011][105692] Updated weights for policy 0, policy_version 1345061 (0.0008) [2023-12-27 01:11:10,069][105692] Updated weights for policy 0, policy_version 1345071 (0.0010) [2023-12-27 01:11:10,337][105620] Updated weights for policy 1, policy_version 1347075 (0.0009) [2023-12-27 01:11:10,396][105620] Updated weights for policy 1, policy_version 1347085 (0.0010) [2023-12-27 01:11:10,451][105620] Updated weights for policy 1, policy_version 1347095 (0.0010) [2023-12-27 01:11:10,739][105692] Updated weights for policy 0, policy_version 1345081 (0.0007) [2023-12-27 01:11:10,799][105692] Updated weights for policy 0, policy_version 1345091 (0.0010) [2023-12-27 01:11:10,867][105692] Updated weights for policy 0, policy_version 1345101 (0.0007) [2023-12-27 01:11:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 689299456. Throughput: 0: 9529.4, 1: 9887.8. Samples: 689307112. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:11:11,062][104569] Avg episode reward: [(0, '8182.462'), (1, '8551.851')] [2023-12-27 01:11:11,184][105620] Updated weights for policy 1, policy_version 1347105 (0.0009) [2023-12-27 01:11:11,241][105620] Updated weights for policy 1, policy_version 1347115 (0.0009) [2023-12-27 01:11:11,306][105620] Updated weights for policy 1, policy_version 1347125 (0.0012) [2023-12-27 01:11:11,365][105620] Updated weights for policy 1, policy_version 1347135 (0.0012) [2023-12-27 01:11:11,560][105692] Updated weights for policy 0, policy_version 1345111 (0.0009) [2023-12-27 01:11:11,625][105692] Updated weights for policy 0, policy_version 1345121 (0.0008) [2023-12-27 01:11:11,689][105692] Updated weights for policy 0, policy_version 1345131 (0.0008) [2023-12-27 01:11:12,117][105620] Updated weights for policy 1, policy_version 1347145 (0.0007) [2023-12-27 01:11:12,184][105620] Updated weights for policy 1, policy_version 1347155 (0.0006) [2023-12-27 01:11:12,250][105620] Updated weights for policy 1, policy_version 1347165 (0.0006) [2023-12-27 01:11:12,454][105692] Updated weights for policy 0, policy_version 1345141 (0.0007) [2023-12-27 01:11:12,501][105692] Updated weights for policy 0, policy_version 1345151 (0.0005) [2023-12-27 01:11:12,550][105692] Updated weights for policy 0, policy_version 1345161 (0.0007) [2023-12-27 01:11:12,804][105620] Updated weights for policy 1, policy_version 1347175 (0.0006) [2023-12-27 01:11:12,868][105620] Updated weights for policy 1, policy_version 1347185 (0.0007) [2023-12-27 01:11:12,937][105620] Updated weights for policy 1, policy_version 1347195 (0.0009) [2023-12-27 01:11:13,245][105692] Updated weights for policy 0, policy_version 1345171 (0.0011) [2023-12-27 01:11:13,302][105692] Updated weights for policy 0, policy_version 1345181 (0.0011) [2023-12-27 01:11:13,356][105692] Updated weights for policy 0, policy_version 1345191 (0.0010) [2023-12-27 01:11:13,632][105620] Updated weights for policy 1, policy_version 1347205 (0.0005) [2023-12-27 01:11:13,688][105620] Updated weights for policy 1, policy_version 1347215 (0.0007) [2023-12-27 01:11:13,736][105620] Updated weights for policy 1, policy_version 1347225 (0.0008) [2023-12-27 01:11:14,081][105692] Updated weights for policy 0, policy_version 1345201 (0.0010) [2023-12-27 01:11:14,137][105692] Updated weights for policy 0, policy_version 1345211 (0.0005) [2023-12-27 01:11:14,197][105692] Updated weights for policy 0, policy_version 1345221 (0.0006) [2023-12-27 01:11:14,253][105692] Updated weights for policy 0, policy_version 1345231 (0.0006) [2023-12-27 01:11:14,519][105620] Updated weights for policy 1, policy_version 1347235 (0.0008) [2023-12-27 01:11:14,570][105620] Updated weights for policy 1, policy_version 1347245 (0.0006) [2023-12-27 01:11:14,636][105620] Updated weights for policy 1, policy_version 1347255 (0.0005) [2023-12-27 01:11:14,957][105692] Updated weights for policy 0, policy_version 1345241 (0.0008) [2023-12-27 01:11:15,020][105692] Updated weights for policy 0, policy_version 1345251 (0.0006) [2023-12-27 01:11:15,078][105692] Updated weights for policy 0, policy_version 1345261 (0.0007) [2023-12-27 01:11:15,304][105620] Updated weights for policy 1, policy_version 1347265 (0.0007) [2023-12-27 01:11:15,364][105620] Updated weights for policy 1, policy_version 1347275 (0.0011) [2023-12-27 01:11:15,423][105620] Updated weights for policy 1, policy_version 1347285 (0.0010) [2023-12-27 01:11:15,480][105620] Updated weights for policy 1, policy_version 1347295 (0.0010) [2023-12-27 01:11:15,715][105692] Updated weights for policy 0, policy_version 1345271 (0.0008) [2023-12-27 01:11:15,773][105692] Updated weights for policy 0, policy_version 1345281 (0.0008) [2023-12-27 01:11:15,818][105692] Updated weights for policy 0, policy_version 1345291 (0.0007) [2023-12-27 01:11:16,062][104569] Fps is (10 sec: 19659.9, 60 sec: 19524.1, 300 sec: 19549.7). Total num frames: 689397760. Throughput: 0: 9525.6, 1: 9806.1. Samples: 689365632. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:11:16,064][104569] Avg episode reward: [(0, '7504.106'), (1, '8735.175')] [2023-12-27 01:11:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001345296_344449024.pth... [2023-12-27 01:11:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001347296_344948736.pth... [2023-12-27 01:11:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001346144_344653824.pth [2023-12-27 01:11:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001344176_344162304.pth [2023-12-27 01:11:16,148][105620] Updated weights for policy 1, policy_version 1347305 (0.0006) [2023-12-27 01:11:16,199][105620] Updated weights for policy 1, policy_version 1347315 (0.0005) [2023-12-27 01:11:16,246][105620] Updated weights for policy 1, policy_version 1347325 (0.0010) [2023-12-27 01:11:16,610][105692] Updated weights for policy 0, policy_version 1345301 (0.0008) [2023-12-27 01:11:16,658][105692] Updated weights for policy 0, policy_version 1345311 (0.0007) [2023-12-27 01:11:16,709][105692] Updated weights for policy 0, policy_version 1345321 (0.0008) [2023-12-27 01:11:16,948][105620] Updated weights for policy 1, policy_version 1347335 (0.0010) [2023-12-27 01:11:17,012][105620] Updated weights for policy 1, policy_version 1347345 (0.0010) [2023-12-27 01:11:17,061][105620] Updated weights for policy 1, policy_version 1347355 (0.0008) [2023-12-27 01:11:17,546][105692] Updated weights for policy 0, policy_version 1345331 (0.0008) [2023-12-27 01:11:17,598][105692] Updated weights for policy 0, policy_version 1345341 (0.0008) [2023-12-27 01:11:17,638][105620] Updated weights for policy 1, policy_version 1347365 (0.0008) [2023-12-27 01:11:17,648][105692] Updated weights for policy 0, policy_version 1345351 (0.0006) [2023-12-27 01:11:17,696][105620] Updated weights for policy 1, policy_version 1347375 (0.0010) [2023-12-27 01:11:17,751][105620] Updated weights for policy 1, policy_version 1347385 (0.0006) [2023-12-27 01:11:18,435][105692] Updated weights for policy 0, policy_version 1345361 (0.0006) [2023-12-27 01:11:18,458][105620] Updated weights for policy 1, policy_version 1347395 (0.0008) [2023-12-27 01:11:18,486][105692] Updated weights for policy 0, policy_version 1345371 (0.0007) [2023-12-27 01:11:18,522][105620] Updated weights for policy 1, policy_version 1347405 (0.0010) [2023-12-27 01:11:18,540][105692] Updated weights for policy 0, policy_version 1345381 (0.0008) [2023-12-27 01:11:18,584][105620] Updated weights for policy 1, policy_version 1347415 (0.0010) [2023-12-27 01:11:18,599][105692] Updated weights for policy 0, policy_version 1345391 (0.0008) [2023-12-27 01:11:19,229][105620] Updated weights for policy 1, policy_version 1347425 (0.0010) [2023-12-27 01:11:19,293][105620] Updated weights for policy 1, policy_version 1347435 (0.0009) [2023-12-27 01:11:19,358][105620] Updated weights for policy 1, policy_version 1347445 (0.0009) [2023-12-27 01:11:19,421][105620] Updated weights for policy 1, policy_version 1347455 (0.0007) [2023-12-27 01:11:19,445][105692] Updated weights for policy 0, policy_version 1345401 (0.0010) [2023-12-27 01:11:19,510][105692] Updated weights for policy 0, policy_version 1345411 (0.0008) [2023-12-27 01:11:19,571][105692] Updated weights for policy 0, policy_version 1345421 (0.0009) [2023-12-27 01:11:20,234][105620] Updated weights for policy 1, policy_version 1347465 (0.0008) [2023-12-27 01:11:20,293][105620] Updated weights for policy 1, policy_version 1347475 (0.0006) [2023-12-27 01:11:20,342][105620] Updated weights for policy 1, policy_version 1347485 (0.0008) [2023-12-27 01:11:20,360][105692] Updated weights for policy 0, policy_version 1345431 (0.0008) [2023-12-27 01:11:20,423][105692] Updated weights for policy 0, policy_version 1345441 (0.0010) [2023-12-27 01:11:20,489][105692] Updated weights for policy 0, policy_version 1345451 (0.0010) [2023-12-27 01:11:21,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 689487872. Throughput: 0: 9495.4, 1: 9768.3. Samples: 689481916. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:11:21,063][104569] Avg episode reward: [(0, '8138.361'), (1, '8716.817')] [2023-12-27 01:11:21,069][105620] Updated weights for policy 1, policy_version 1347495 (0.0007) [2023-12-27 01:11:21,133][105620] Updated weights for policy 1, policy_version 1347505 (0.0009) [2023-12-27 01:11:21,199][105620] Updated weights for policy 1, policy_version 1347515 (0.0009) [2023-12-27 01:11:21,280][105692] Updated weights for policy 0, policy_version 1345461 (0.0009) [2023-12-27 01:11:21,344][105692] Updated weights for policy 0, policy_version 1345471 (0.0008) [2023-12-27 01:11:21,412][105692] Updated weights for policy 0, policy_version 1345481 (0.0008) [2023-12-27 01:11:22,033][105620] Updated weights for policy 1, policy_version 1347525 (0.0009) [2023-12-27 01:11:22,096][105620] Updated weights for policy 1, policy_version 1347535 (0.0007) [2023-12-27 01:11:22,110][105692] Updated weights for policy 0, policy_version 1345491 (0.0008) [2023-12-27 01:11:22,161][105620] Updated weights for policy 1, policy_version 1347545 (0.0006) [2023-12-27 01:11:22,163][105692] Updated weights for policy 0, policy_version 1345501 (0.0007) [2023-12-27 01:11:22,225][105692] Updated weights for policy 0, policy_version 1345511 (0.0006) [2023-12-27 01:11:22,879][105692] Updated weights for policy 0, policy_version 1345521 (0.0009) [2023-12-27 01:11:22,940][105692] Updated weights for policy 0, policy_version 1345531 (0.0009) [2023-12-27 01:11:22,984][105620] Updated weights for policy 1, policy_version 1347555 (0.0009) [2023-12-27 01:11:22,999][105692] Updated weights for policy 0, policy_version 1345541 (0.0007) [2023-12-27 01:11:23,039][105620] Updated weights for policy 1, policy_version 1347565 (0.0007) [2023-12-27 01:11:23,049][105692] Updated weights for policy 0, policy_version 1345551 (0.0006) [2023-12-27 01:11:23,089][105620] Updated weights for policy 1, policy_version 1347575 (0.0008) [2023-12-27 01:11:23,657][105692] Updated weights for policy 0, policy_version 1345561 (0.0006) [2023-12-27 01:11:23,705][105692] Updated weights for policy 0, policy_version 1345571 (0.0007) [2023-12-27 01:11:23,739][105620] Updated weights for policy 1, policy_version 1347585 (0.0009) [2023-12-27 01:11:23,758][105692] Updated weights for policy 0, policy_version 1345581 (0.0007) [2023-12-27 01:11:23,793][105620] Updated weights for policy 1, policy_version 1347595 (0.0007) [2023-12-27 01:11:23,843][105620] Updated weights for policy 1, policy_version 1347605 (0.0008) [2023-12-27 01:11:23,894][105620] Updated weights for policy 1, policy_version 1347615 (0.0009) [2023-12-27 01:11:24,530][105692] Updated weights for policy 0, policy_version 1345591 (0.0009) [2023-12-27 01:11:24,587][105692] Updated weights for policy 0, policy_version 1345601 (0.0006) [2023-12-27 01:11:24,606][105620] Updated weights for policy 1, policy_version 1347625 (0.0009) [2023-12-27 01:11:24,641][105692] Updated weights for policy 0, policy_version 1345611 (0.0006) [2023-12-27 01:11:24,664][105620] Updated weights for policy 1, policy_version 1347635 (0.0008) [2023-12-27 01:11:24,717][105620] Updated weights for policy 1, policy_version 1347645 (0.0008) [2023-12-27 01:11:25,326][105692] Updated weights for policy 0, policy_version 1345621 (0.0008) [2023-12-27 01:11:25,370][105692] Updated weights for policy 0, policy_version 1345631 (0.0010) [2023-12-27 01:11:25,418][105692] Updated weights for policy 0, policy_version 1345641 (0.0010) [2023-12-27 01:11:25,531][105620] Updated weights for policy 1, policy_version 1347655 (0.0008) [2023-12-27 01:11:25,591][105620] Updated weights for policy 1, policy_version 1347665 (0.0008) [2023-12-27 01:11:25,642][105620] Updated weights for policy 1, policy_version 1347676 (0.0009) [2023-12-27 01:11:26,062][104569] Fps is (10 sec: 18842.6, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 689586176. Throughput: 0: 9531.8, 1: 9736.1. Samples: 689596608. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:11:26,062][104569] Avg episode reward: [(0, '8662.587'), (1, '8375.432')] [2023-12-27 01:11:26,144][105692] Updated weights for policy 0, policy_version 1345651 (0.0010) [2023-12-27 01:11:26,202][105692] Updated weights for policy 0, policy_version 1345661 (0.0011) [2023-12-27 01:11:26,255][105692] Updated weights for policy 0, policy_version 1345671 (0.0011) [2023-12-27 01:11:26,306][105620] Updated weights for policy 1, policy_version 1347686 (0.0009) [2023-12-27 01:11:26,360][105620] Updated weights for policy 1, policy_version 1347696 (0.0010) [2023-12-27 01:11:26,407][105620] Updated weights for policy 1, policy_version 1347706 (0.0008) [2023-12-27 01:11:26,834][105692] Updated weights for policy 0, policy_version 1345681 (0.0010) [2023-12-27 01:11:26,878][105692] Updated weights for policy 0, policy_version 1345691 (0.0010) [2023-12-27 01:11:26,938][105692] Updated weights for policy 0, policy_version 1345701 (0.0010) [2023-12-27 01:11:26,989][105692] Updated weights for policy 0, policy_version 1345711 (0.0010) [2023-12-27 01:11:27,129][105620] Updated weights for policy 1, policy_version 1347716 (0.0008) [2023-12-27 01:11:27,189][105620] Updated weights for policy 1, policy_version 1347726 (0.0006) [2023-12-27 01:11:27,239][105620] Updated weights for policy 1, policy_version 1347736 (0.0008) [2023-12-27 01:11:27,657][105692] Updated weights for policy 0, policy_version 1345721 (0.0006) [2023-12-27 01:11:27,708][105692] Updated weights for policy 0, policy_version 1345731 (0.0006) [2023-12-27 01:11:27,761][105692] Updated weights for policy 0, policy_version 1345741 (0.0005) [2023-12-27 01:11:27,788][105620] Updated weights for policy 1, policy_version 1347746 (0.0008) [2023-12-27 01:11:27,835][105620] Updated weights for policy 1, policy_version 1347756 (0.0009) [2023-12-27 01:11:27,884][105620] Updated weights for policy 1, policy_version 1347766 (0.0006) [2023-12-27 01:11:27,935][105620] Updated weights for policy 1, policy_version 1347776 (0.0005) [2023-12-27 01:11:28,344][105692] Updated weights for policy 0, policy_version 1345751 (0.0010) [2023-12-27 01:11:28,409][105692] Updated weights for policy 0, policy_version 1345761 (0.0010) [2023-12-27 01:11:28,467][105692] Updated weights for policy 0, policy_version 1345771 (0.0010) [2023-12-27 01:11:28,582][105620] Updated weights for policy 1, policy_version 1347786 (0.0007) [2023-12-27 01:11:28,632][105620] Updated weights for policy 1, policy_version 1347796 (0.0005) [2023-12-27 01:11:28,691][105620] Updated weights for policy 1, policy_version 1347806 (0.0008) [2023-12-27 01:11:29,067][105692] Updated weights for policy 0, policy_version 1345781 (0.0008) [2023-12-27 01:11:29,122][105692] Updated weights for policy 0, policy_version 1345791 (0.0008) [2023-12-27 01:11:29,181][105692] Updated weights for policy 0, policy_version 1345801 (0.0011) [2023-12-27 01:11:29,456][105620] Updated weights for policy 1, policy_version 1347816 (0.0010) [2023-12-27 01:11:29,512][105620] Updated weights for policy 1, policy_version 1347826 (0.0010) [2023-12-27 01:11:29,564][105620] Updated weights for policy 1, policy_version 1347836 (0.0008) [2023-12-27 01:11:29,774][105692] Updated weights for policy 0, policy_version 1345811 (0.0007) [2023-12-27 01:11:29,835][105692] Updated weights for policy 0, policy_version 1345821 (0.0009) [2023-12-27 01:11:29,900][105692] Updated weights for policy 0, policy_version 1345831 (0.0011) [2023-12-27 01:11:30,344][105620] Updated weights for policy 1, policy_version 1347846 (0.0006) [2023-12-27 01:11:30,392][105620] Updated weights for policy 1, policy_version 1347856 (0.0005) [2023-12-27 01:11:30,441][105620] Updated weights for policy 1, policy_version 1347866 (0.0007) [2023-12-27 01:11:30,617][105692] Updated weights for policy 0, policy_version 1345841 (0.0011) [2023-12-27 01:11:30,684][105692] Updated weights for policy 0, policy_version 1345851 (0.0010) [2023-12-27 01:11:30,746][105692] Updated weights for policy 0, policy_version 1345861 (0.0010) [2023-12-27 01:11:30,805][105692] Updated weights for policy 0, policy_version 1345871 (0.0010) [2023-12-27 01:11:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 689692672. Throughput: 0: 9604.8, 1: 9854.0. Samples: 689661140. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:11:31,063][104569] Avg episode reward: [(0, '8631.697'), (1, '8559.983')] [2023-12-27 01:11:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001345872_344596480.pth... [2023-12-27 01:11:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001344752_344309760.pth [2023-12-27 01:11:31,088][105620] Updated weights for policy 1, policy_version 1347876 (0.0007) [2023-12-27 01:11:31,153][105620] Updated weights for policy 1, policy_version 1347886 (0.0009) [2023-12-27 01:11:31,215][105620] Updated weights for policy 1, policy_version 1347896 (0.0010) [2023-12-27 01:11:31,261][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001347904_345104384.pth... [2023-12-27 01:11:31,266][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001346752_344809472.pth [2023-12-27 01:11:31,496][105692] Updated weights for policy 0, policy_version 1345881 (0.0011) [2023-12-27 01:11:31,552][105692] Updated weights for policy 0, policy_version 1345891 (0.0010) [2023-12-27 01:11:31,618][105692] Updated weights for policy 0, policy_version 1345901 (0.0010) [2023-12-27 01:11:31,963][105620] Updated weights for policy 1, policy_version 1347906 (0.0010) [2023-12-27 01:11:32,017][105620] Updated weights for policy 1, policy_version 1347916 (0.0005) [2023-12-27 01:11:32,079][105620] Updated weights for policy 1, policy_version 1347926 (0.0009) [2023-12-27 01:11:32,135][105620] Updated weights for policy 1, policy_version 1347936 (0.0011) [2023-12-27 01:11:32,251][105692] Updated weights for policy 0, policy_version 1345911 (0.0010) [2023-12-27 01:11:32,318][105692] Updated weights for policy 0, policy_version 1345921 (0.0011) [2023-12-27 01:11:32,384][105692] Updated weights for policy 0, policy_version 1345931 (0.0011) [2023-12-27 01:11:32,802][105620] Updated weights for policy 1, policy_version 1347946 (0.0010) [2023-12-27 01:11:32,862][105620] Updated weights for policy 1, policy_version 1347956 (0.0010) [2023-12-27 01:11:32,920][105620] Updated weights for policy 1, policy_version 1347966 (0.0010) [2023-12-27 01:11:33,103][105692] Updated weights for policy 0, policy_version 1345941 (0.0009) [2023-12-27 01:11:33,157][105692] Updated weights for policy 0, policy_version 1345951 (0.0007) [2023-12-27 01:11:33,210][105692] Updated weights for policy 0, policy_version 1345961 (0.0005) [2023-12-27 01:11:33,645][105620] Updated weights for policy 1, policy_version 1347976 (0.0010) [2023-12-27 01:11:33,702][105620] Updated weights for policy 1, policy_version 1347986 (0.0010) [2023-12-27 01:11:33,766][105620] Updated weights for policy 1, policy_version 1347996 (0.0010) [2023-12-27 01:11:33,813][105692] Updated weights for policy 0, policy_version 1345971 (0.0005) [2023-12-27 01:11:33,869][105692] Updated weights for policy 0, policy_version 1345981 (0.0005) [2023-12-27 01:11:33,924][105692] Updated weights for policy 0, policy_version 1345991 (0.0006) [2023-12-27 01:11:34,419][105620] Updated weights for policy 1, policy_version 1348006 (0.0009) [2023-12-27 01:11:34,482][105620] Updated weights for policy 1, policy_version 1348016 (0.0009) [2023-12-27 01:11:34,541][105620] Updated weights for policy 1, policy_version 1348026 (0.0009) [2023-12-27 01:11:34,694][105692] Updated weights for policy 0, policy_version 1346001 (0.0010) [2023-12-27 01:11:34,758][105692] Updated weights for policy 0, policy_version 1346011 (0.0008) [2023-12-27 01:11:34,816][105692] Updated weights for policy 0, policy_version 1346021 (0.0010) [2023-12-27 01:11:34,867][105692] Updated weights for policy 0, policy_version 1346031 (0.0009) [2023-12-27 01:11:35,285][105620] Updated weights for policy 1, policy_version 1348036 (0.0008) [2023-12-27 01:11:35,336][105620] Updated weights for policy 1, policy_version 1348046 (0.0005) [2023-12-27 01:11:35,392][105620] Updated weights for policy 1, policy_version 1348056 (0.0005) [2023-12-27 01:11:35,472][105692] Updated weights for policy 0, policy_version 1346041 (0.0009) [2023-12-27 01:11:35,534][105692] Updated weights for policy 0, policy_version 1346051 (0.0010) [2023-12-27 01:11:35,593][105692] Updated weights for policy 0, policy_version 1346061 (0.0011) [2023-12-27 01:11:35,985][105620] Updated weights for policy 1, policy_version 1348066 (0.0005) [2023-12-27 01:11:36,034][105620] Updated weights for policy 1, policy_version 1348076 (0.0005) [2023-12-27 01:11:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 689790976. Throughput: 0: 9752.3, 1: 9762.6. Samples: 689781884. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:11:36,062][104569] Avg episode reward: [(0, '8723.962'), (1, '8544.044')] [2023-12-27 01:11:36,083][105620] Updated weights for policy 1, policy_version 1348086 (0.0008) [2023-12-27 01:11:36,145][105620] Updated weights for policy 1, policy_version 1348096 (0.0009) [2023-12-27 01:11:36,253][105692] Updated weights for policy 0, policy_version 1346071 (0.0009) [2023-12-27 01:11:36,317][105692] Updated weights for policy 0, policy_version 1346081 (0.0007) [2023-12-27 01:11:36,380][105692] Updated weights for policy 0, policy_version 1346091 (0.0010) [2023-12-27 01:11:36,898][105620] Updated weights for policy 1, policy_version 1348106 (0.0008) [2023-12-27 01:11:36,966][105620] Updated weights for policy 1, policy_version 1348116 (0.0009) [2023-12-27 01:11:37,027][105620] Updated weights for policy 1, policy_version 1348126 (0.0009) [2023-12-27 01:11:37,098][105692] Updated weights for policy 0, policy_version 1346101 (0.0009) [2023-12-27 01:11:37,158][105692] Updated weights for policy 0, policy_version 1346111 (0.0009) [2023-12-27 01:11:37,212][105692] Updated weights for policy 0, policy_version 1346121 (0.0009) [2023-12-27 01:11:37,671][105620] Updated weights for policy 1, policy_version 1348136 (0.0010) [2023-12-27 01:11:37,730][105620] Updated weights for policy 1, policy_version 1348146 (0.0009) [2023-12-27 01:11:37,786][105620] Updated weights for policy 1, policy_version 1348156 (0.0010) [2023-12-27 01:11:37,961][105692] Updated weights for policy 0, policy_version 1346131 (0.0010) [2023-12-27 01:11:38,010][105692] Updated weights for policy 0, policy_version 1346141 (0.0008) [2023-12-27 01:11:38,069][105692] Updated weights for policy 0, policy_version 1346151 (0.0008) [2023-12-27 01:11:38,450][105620] Updated weights for policy 1, policy_version 1348166 (0.0009) [2023-12-27 01:11:38,512][105620] Updated weights for policy 1, policy_version 1348176 (0.0009) [2023-12-27 01:11:38,561][105620] Updated weights for policy 1, policy_version 1348186 (0.0009) [2023-12-27 01:11:38,851][105692] Updated weights for policy 0, policy_version 1346161 (0.0008) [2023-12-27 01:11:38,908][105692] Updated weights for policy 0, policy_version 1346171 (0.0007) [2023-12-27 01:11:38,960][105692] Updated weights for policy 0, policy_version 1346181 (0.0008) [2023-12-27 01:11:39,025][105692] Updated weights for policy 0, policy_version 1346191 (0.0007) [2023-12-27 01:11:39,246][105620] Updated weights for policy 1, policy_version 1348196 (0.0008) [2023-12-27 01:11:39,302][105620] Updated weights for policy 1, policy_version 1348206 (0.0010) [2023-12-27 01:11:39,350][105620] Updated weights for policy 1, policy_version 1348216 (0.0009) [2023-12-27 01:11:39,723][105692] Updated weights for policy 0, policy_version 1346201 (0.0008) [2023-12-27 01:11:39,770][105692] Updated weights for policy 0, policy_version 1346211 (0.0009) [2023-12-27 01:11:39,828][105692] Updated weights for policy 0, policy_version 1346221 (0.0009) [2023-12-27 01:11:40,121][105620] Updated weights for policy 1, policy_version 1348226 (0.0008) [2023-12-27 01:11:40,177][105620] Updated weights for policy 1, policy_version 1348236 (0.0009) [2023-12-27 01:11:40,232][105620] Updated weights for policy 1, policy_version 1348246 (0.0009) [2023-12-27 01:11:40,289][105620] Updated weights for policy 1, policy_version 1348256 (0.0008) [2023-12-27 01:11:40,621][105692] Updated weights for policy 0, policy_version 1346231 (0.0008) [2023-12-27 01:11:40,668][105692] Updated weights for policy 0, policy_version 1346241 (0.0008) [2023-12-27 01:11:40,718][105692] Updated weights for policy 0, policy_version 1346251 (0.0009) [2023-12-27 01:11:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 689889280. Throughput: 0: 9797.4, 1: 9788.1. Samples: 689899372. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:11:41,062][104569] Avg episode reward: [(0, '8452.736'), (1, '8727.501')] [2023-12-27 01:11:41,081][105620] Updated weights for policy 1, policy_version 1348266 (0.0009) [2023-12-27 01:11:41,149][105620] Updated weights for policy 1, policy_version 1348276 (0.0010) [2023-12-27 01:11:41,209][105620] Updated weights for policy 1, policy_version 1348286 (0.0009) [2023-12-27 01:11:41,437][105692] Updated weights for policy 0, policy_version 1346261 (0.0008) [2023-12-27 01:11:41,485][105692] Updated weights for policy 0, policy_version 1346271 (0.0009) [2023-12-27 01:11:41,544][105692] Updated weights for policy 0, policy_version 1346281 (0.0009) [2023-12-27 01:11:41,964][105620] Updated weights for policy 1, policy_version 1348296 (0.0008) [2023-12-27 01:11:42,029][105620] Updated weights for policy 1, policy_version 1348306 (0.0009) [2023-12-27 01:11:42,091][105620] Updated weights for policy 1, policy_version 1348316 (0.0009) [2023-12-27 01:11:42,345][105692] Updated weights for policy 0, policy_version 1346291 (0.0010) [2023-12-27 01:11:42,405][105692] Updated weights for policy 0, policy_version 1346301 (0.0008) [2023-12-27 01:11:42,458][105692] Updated weights for policy 0, policy_version 1346311 (0.0008) [2023-12-27 01:11:42,867][105620] Updated weights for policy 1, policy_version 1348326 (0.0008) [2023-12-27 01:11:42,926][105620] Updated weights for policy 1, policy_version 1348336 (0.0008) [2023-12-27 01:11:42,991][105620] Updated weights for policy 1, policy_version 1348346 (0.0008) [2023-12-27 01:11:43,153][105692] Updated weights for policy 0, policy_version 1346321 (0.0011) [2023-12-27 01:11:43,211][105692] Updated weights for policy 0, policy_version 1346331 (0.0010) [2023-12-27 01:11:43,262][105692] Updated weights for policy 0, policy_version 1346341 (0.0010) [2023-12-27 01:11:43,319][105692] Updated weights for policy 0, policy_version 1346351 (0.0007) [2023-12-27 01:11:43,671][105620] Updated weights for policy 1, policy_version 1348356 (0.0009) [2023-12-27 01:11:43,729][105620] Updated weights for policy 1, policy_version 1348366 (0.0009) [2023-12-27 01:11:43,783][105620] Updated weights for policy 1, policy_version 1348376 (0.0009) [2023-12-27 01:11:43,953][105692] Updated weights for policy 0, policy_version 1346361 (0.0008) [2023-12-27 01:11:44,011][105692] Updated weights for policy 0, policy_version 1346371 (0.0009) [2023-12-27 01:11:44,077][105692] Updated weights for policy 0, policy_version 1346381 (0.0009) [2023-12-27 01:11:44,647][105620] Updated weights for policy 1, policy_version 1348386 (0.0009) [2023-12-27 01:11:44,657][105692] Updated weights for policy 0, policy_version 1346391 (0.0007) [2023-12-27 01:11:44,684][105585] KL-divergence is very high: 111.2450 [2023-12-27 01:11:44,708][105692] Updated weights for policy 0, policy_version 1346401 (0.0007) [2023-12-27 01:11:44,709][105620] Updated weights for policy 1, policy_version 1348396 (0.0008) [2023-12-27 01:11:44,723][105585] KL-divergence is very high: 193.7534 [2023-12-27 01:11:44,755][105692] Updated weights for policy 0, policy_version 1346411 (0.0008) [2023-12-27 01:11:44,763][105585] KL-divergence is very high: 206.8540 [2023-12-27 01:11:44,771][105620] Updated weights for policy 1, policy_version 1348406 (0.0007) [2023-12-27 01:11:45,460][105692] Updated weights for policy 0, policy_version 1346421 (0.0008) [2023-12-27 01:11:45,505][105692] Updated weights for policy 0, policy_version 1346431 (0.0008) [2023-12-27 01:11:45,556][105620] Updated weights for policy 1, policy_version 1348417 (0.0010) [2023-12-27 01:11:45,564][105692] Updated weights for policy 0, policy_version 1346441 (0.0007) [2023-12-27 01:11:45,612][105620] Updated weights for policy 1, policy_version 1348427 (0.0009) [2023-12-27 01:11:45,676][105620] Updated weights for policy 1, policy_version 1348437 (0.0010) [2023-12-27 01:11:45,720][105620] Updated weights for policy 1, policy_version 1348447 (0.0005) [2023-12-27 01:11:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 689987584. Throughput: 0: 9704.8, 1: 9830.4. Samples: 689956356. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:11:46,063][104569] Avg episode reward: [(0, '8544.771'), (1, '8998.001')] [2023-12-27 01:11:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001346448_344743936.pth... [2023-12-27 01:11:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001348448_345243648.pth... [2023-12-27 01:11:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001345296_344449024.pth [2023-12-27 01:11:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001347296_344948736.pth [2023-12-27 01:11:46,338][105620] Updated weights for policy 1, policy_version 1348457 (0.0009) [2023-12-27 01:11:46,389][105620] Updated weights for policy 1, policy_version 1348467 (0.0009) [2023-12-27 01:11:46,391][105692] Updated weights for policy 0, policy_version 1346451 (0.0008) [2023-12-27 01:11:46,438][105620] Updated weights for policy 1, policy_version 1348477 (0.0006) [2023-12-27 01:11:46,444][105692] Updated weights for policy 0, policy_version 1346461 (0.0006) [2023-12-27 01:11:46,489][105692] Updated weights for policy 0, policy_version 1346471 (0.0008) [2023-12-27 01:11:47,133][105692] Updated weights for policy 0, policy_version 1346481 (0.0008) [2023-12-27 01:11:47,190][105692] Updated weights for policy 0, policy_version 1346491 (0.0005) [2023-12-27 01:11:47,246][105692] Updated weights for policy 0, policy_version 1346501 (0.0005) [2023-12-27 01:11:47,293][105620] Updated weights for policy 1, policy_version 1348487 (0.0009) [2023-12-27 01:11:47,297][105692] Updated weights for policy 0, policy_version 1346511 (0.0005) [2023-12-27 01:11:47,347][105620] Updated weights for policy 1, policy_version 1348497 (0.0008) [2023-12-27 01:11:47,393][105620] Updated weights for policy 1, policy_version 1348507 (0.0009) [2023-12-27 01:11:47,928][105692] Updated weights for policy 0, policy_version 1346521 (0.0005) [2023-12-27 01:11:47,984][105692] Updated weights for policy 0, policy_version 1346531 (0.0005) [2023-12-27 01:11:48,045][105692] Updated weights for policy 0, policy_version 1346541 (0.0005) [2023-12-27 01:11:48,211][105620] Updated weights for policy 1, policy_version 1348517 (0.0008) [2023-12-27 01:11:48,267][105620] Updated weights for policy 1, policy_version 1348527 (0.0010) [2023-12-27 01:11:48,327][105620] Updated weights for policy 1, policy_version 1348537 (0.0010) [2023-12-27 01:11:48,639][105692] Updated weights for policy 0, policy_version 1346551 (0.0008) [2023-12-27 01:11:48,693][105692] Updated weights for policy 0, policy_version 1346561 (0.0009) [2023-12-27 01:11:48,745][105692] Updated weights for policy 0, policy_version 1346571 (0.0009) [2023-12-27 01:11:49,038][105620] Updated weights for policy 1, policy_version 1348547 (0.0008) [2023-12-27 01:11:49,092][105620] Updated weights for policy 1, policy_version 1348557 (0.0005) [2023-12-27 01:11:49,143][105620] Updated weights for policy 1, policy_version 1348567 (0.0006) [2023-12-27 01:11:49,495][105692] Updated weights for policy 0, policy_version 1346581 (0.0007) [2023-12-27 01:11:49,553][105692] Updated weights for policy 0, policy_version 1346591 (0.0005) [2023-12-27 01:11:49,610][105692] Updated weights for policy 0, policy_version 1346601 (0.0006) [2023-12-27 01:11:49,917][105620] Updated weights for policy 1, policy_version 1348577 (0.0005) [2023-12-27 01:11:49,979][105620] Updated weights for policy 1, policy_version 1348587 (0.0008) [2023-12-27 01:11:50,029][105620] Updated weights for policy 1, policy_version 1348597 (0.0006) [2023-12-27 01:11:50,077][105620] Updated weights for policy 1, policy_version 1348607 (0.0005) [2023-12-27 01:11:50,300][105692] Updated weights for policy 0, policy_version 1346611 (0.0007) [2023-12-27 01:11:50,371][105692] Updated weights for policy 0, policy_version 1346621 (0.0010) [2023-12-27 01:11:50,428][105692] Updated weights for policy 0, policy_version 1346631 (0.0009) [2023-12-27 01:11:50,762][105620] Updated weights for policy 1, policy_version 1348617 (0.0009) [2023-12-27 01:11:50,826][105620] Updated weights for policy 1, policy_version 1348627 (0.0009) [2023-12-27 01:11:50,892][105620] Updated weights for policy 1, policy_version 1348637 (0.0009) [2023-12-27 01:11:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 690085888. Throughput: 0: 9837.6, 1: 9736.3. Samples: 690073748. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:11:51,062][104569] Avg episode reward: [(0, '8724.007'), (1, '9088.932')] [2023-12-27 01:11:51,195][105692] Updated weights for policy 0, policy_version 1346641 (0.0009) [2023-12-27 01:11:51,256][105692] Updated weights for policy 0, policy_version 1346651 (0.0007) [2023-12-27 01:11:51,319][105692] Updated weights for policy 0, policy_version 1346661 (0.0009) [2023-12-27 01:11:51,381][105692] Updated weights for policy 0, policy_version 1346671 (0.0010) [2023-12-27 01:11:51,646][105620] Updated weights for policy 1, policy_version 1348647 (0.0010) [2023-12-27 01:11:51,712][105620] Updated weights for policy 1, policy_version 1348657 (0.0009) [2023-12-27 01:11:51,785][105620] Updated weights for policy 1, policy_version 1348667 (0.0009) [2023-12-27 01:11:52,201][105692] Updated weights for policy 0, policy_version 1346681 (0.0008) [2023-12-27 01:11:52,252][105692] Updated weights for policy 0, policy_version 1346691 (0.0008) [2023-12-27 01:11:52,315][105692] Updated weights for policy 0, policy_version 1346701 (0.0010) [2023-12-27 01:11:52,437][105620] Updated weights for policy 1, policy_version 1348677 (0.0007) [2023-12-27 01:11:52,496][105620] Updated weights for policy 1, policy_version 1348687 (0.0008) [2023-12-27 01:11:52,551][105620] Updated weights for policy 1, policy_version 1348697 (0.0007) [2023-12-27 01:11:53,061][105692] Updated weights for policy 0, policy_version 1346711 (0.0007) [2023-12-27 01:11:53,122][105692] Updated weights for policy 0, policy_version 1346721 (0.0005) [2023-12-27 01:11:53,175][105692] Updated weights for policy 0, policy_version 1346731 (0.0006) [2023-12-27 01:11:53,355][105620] Updated weights for policy 1, policy_version 1348707 (0.0009) [2023-12-27 01:11:53,412][105620] Updated weights for policy 1, policy_version 1348717 (0.0008) [2023-12-27 01:11:53,477][105620] Updated weights for policy 1, policy_version 1348727 (0.0009) [2023-12-27 01:11:53,719][105692] Updated weights for policy 0, policy_version 1346741 (0.0005) [2023-12-27 01:11:53,774][105692] Updated weights for policy 0, policy_version 1346751 (0.0008) [2023-12-27 01:11:53,823][105692] Updated weights for policy 0, policy_version 1346761 (0.0007) [2023-12-27 01:11:53,835][105585] KL-divergence is very high: 111.0118 [2023-12-27 01:11:54,255][105620] Updated weights for policy 1, policy_version 1348737 (0.0010) [2023-12-27 01:11:54,303][105620] Updated weights for policy 1, policy_version 1348747 (0.0008) [2023-12-27 01:11:54,352][105620] Updated weights for policy 1, policy_version 1348757 (0.0008) [2023-12-27 01:11:54,410][105620] Updated weights for policy 1, policy_version 1348767 (0.0006) [2023-12-27 01:11:54,454][105692] Updated weights for policy 0, policy_version 1346771 (0.0007) [2023-12-27 01:11:54,502][105692] Updated weights for policy 0, policy_version 1346781 (0.0010) [2023-12-27 01:11:54,542][105585] KL-divergence is very high: 105.7408 [2023-12-27 01:11:54,553][105692] Updated weights for policy 0, policy_version 1346791 (0.0010) [2023-12-27 01:11:54,586][105585] KL-divergence is very high: 119.3686 [2023-12-27 01:11:55,120][105620] Updated weights for policy 1, policy_version 1348777 (0.0005) [2023-12-27 01:11:55,168][105620] Updated weights for policy 1, policy_version 1348787 (0.0005) [2023-12-27 01:11:55,212][105620] Updated weights for policy 1, policy_version 1348797 (0.0005) [2023-12-27 01:11:55,344][105692] Updated weights for policy 0, policy_version 1346801 (0.0010) [2023-12-27 01:11:55,409][105692] Updated weights for policy 0, policy_version 1346811 (0.0007) [2023-12-27 01:11:55,473][105692] Updated weights for policy 0, policy_version 1346821 (0.0006) [2023-12-27 01:11:55,533][105692] Updated weights for policy 0, policy_version 1346831 (0.0006) [2023-12-27 01:11:55,930][105620] Updated weights for policy 1, policy_version 1348807 (0.0007) [2023-12-27 01:11:55,981][105620] Updated weights for policy 1, policy_version 1348817 (0.0008) [2023-12-27 01:11:56,039][105620] Updated weights for policy 1, policy_version 1348827 (0.0008) [2023-12-27 01:11:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 690176000. Throughput: 0: 9898.2, 1: 9716.7. Samples: 690189788. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:11:56,063][104569] Avg episode reward: [(0, '8269.593'), (1, '9000.466')] [2023-12-27 01:11:56,189][105692] Updated weights for policy 0, policy_version 1346841 (0.0010) [2023-12-27 01:11:56,247][105692] Updated weights for policy 0, policy_version 1346851 (0.0010) [2023-12-27 01:11:56,296][105692] Updated weights for policy 0, policy_version 1346861 (0.0010) [2023-12-27 01:11:56,800][105620] Updated weights for policy 1, policy_version 1348837 (0.0008) [2023-12-27 01:11:56,849][105620] Updated weights for policy 1, policy_version 1348847 (0.0008) [2023-12-27 01:11:56,900][105620] Updated weights for policy 1, policy_version 1348857 (0.0007) [2023-12-27 01:11:57,047][105692] Updated weights for policy 0, policy_version 1346871 (0.0010) [2023-12-27 01:11:57,095][105692] Updated weights for policy 0, policy_version 1346881 (0.0010) [2023-12-27 01:11:57,153][105692] Updated weights for policy 0, policy_version 1346891 (0.0010) [2023-12-27 01:11:57,677][105620] Updated weights for policy 1, policy_version 1348867 (0.0008) [2023-12-27 01:11:57,733][105620] Updated weights for policy 1, policy_version 1348877 (0.0008) [2023-12-27 01:11:57,792][105620] Updated weights for policy 1, policy_version 1348887 (0.0008) [2023-12-27 01:11:57,910][105692] Updated weights for policy 0, policy_version 1346901 (0.0011) [2023-12-27 01:11:57,967][105692] Updated weights for policy 0, policy_version 1346911 (0.0010) [2023-12-27 01:11:58,024][105692] Updated weights for policy 0, policy_version 1346921 (0.0010) [2023-12-27 01:11:58,589][105620] Updated weights for policy 1, policy_version 1348897 (0.0008) [2023-12-27 01:11:58,645][105620] Updated weights for policy 1, policy_version 1348907 (0.0008) [2023-12-27 01:11:58,704][105620] Updated weights for policy 1, policy_version 1348917 (0.0008) [2023-12-27 01:11:58,769][105620] Updated weights for policy 1, policy_version 1348927 (0.0008) [2023-12-27 01:11:58,872][105692] Updated weights for policy 0, policy_version 1346931 (0.0010) [2023-12-27 01:11:58,938][105692] Updated weights for policy 0, policy_version 1346941 (0.0011) [2023-12-27 01:11:58,991][105692] Updated weights for policy 0, policy_version 1346951 (0.0011) [2023-12-27 01:11:59,546][105620] Updated weights for policy 1, policy_version 1348937 (0.0008) [2023-12-27 01:11:59,601][105620] Updated weights for policy 1, policy_version 1348947 (0.0008) [2023-12-27 01:11:59,662][105620] Updated weights for policy 1, policy_version 1348957 (0.0008) [2023-12-27 01:11:59,752][105692] Updated weights for policy 0, policy_version 1346961 (0.0010) [2023-12-27 01:11:59,817][105692] Updated weights for policy 0, policy_version 1346971 (0.0011) [2023-12-27 01:11:59,884][105692] Updated weights for policy 0, policy_version 1346981 (0.0011) [2023-12-27 01:11:59,948][105692] Updated weights for policy 0, policy_version 1346991 (0.0011) [2023-12-27 01:12:00,431][105620] Updated weights for policy 1, policy_version 1348967 (0.0008) [2023-12-27 01:12:00,482][105620] Updated weights for policy 1, policy_version 1348977 (0.0008) [2023-12-27 01:12:00,533][105620] Updated weights for policy 1, policy_version 1348987 (0.0008) [2023-12-27 01:12:00,680][105692] Updated weights for policy 0, policy_version 1347001 (0.0010) [2023-12-27 01:12:00,724][105692] Updated weights for policy 0, policy_version 1347011 (0.0010) [2023-12-27 01:12:00,778][105692] Updated weights for policy 0, policy_version 1347021 (0.0010) [2023-12-27 01:12:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 690274304. Throughput: 0: 9887.6, 1: 9656.5. Samples: 690245108. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:12:01,063][104569] Avg episode reward: [(0, '8362.805'), (1, '8909.427')] [2023-12-27 01:12:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001347024_344891392.pth... [2023-12-27 01:12:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001348992_345382912.pth... [2023-12-27 01:12:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001345872_344596480.pth [2023-12-27 01:12:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001347904_345104384.pth [2023-12-27 01:12:01,253][105620] Updated weights for policy 1, policy_version 1348997 (0.0007) [2023-12-27 01:12:01,317][105620] Updated weights for policy 1, policy_version 1349007 (0.0006) [2023-12-27 01:12:01,379][105620] Updated weights for policy 1, policy_version 1349017 (0.0008) [2023-12-27 01:12:01,555][105692] Updated weights for policy 0, policy_version 1347031 (0.0011) [2023-12-27 01:12:01,611][105692] Updated weights for policy 0, policy_version 1347041 (0.0010) [2023-12-27 01:12:01,682][105692] Updated weights for policy 0, policy_version 1347051 (0.0011) [2023-12-27 01:12:02,011][105620] Updated weights for policy 1, policy_version 1349027 (0.0007) [2023-12-27 01:12:02,068][105620] Updated weights for policy 1, policy_version 1349037 (0.0006) [2023-12-27 01:12:02,125][105620] Updated weights for policy 1, policy_version 1349047 (0.0006) [2023-12-27 01:12:02,492][105692] Updated weights for policy 0, policy_version 1347061 (0.0010) [2023-12-27 01:12:02,542][105692] Updated weights for policy 0, policy_version 1347071 (0.0008) [2023-12-27 01:12:02,598][105692] Updated weights for policy 0, policy_version 1347081 (0.0006) [2023-12-27 01:12:02,776][105620] Updated weights for policy 1, policy_version 1349057 (0.0006) [2023-12-27 01:12:02,828][105620] Updated weights for policy 1, policy_version 1349067 (0.0006) [2023-12-27 01:12:02,892][105620] Updated weights for policy 1, policy_version 1349077 (0.0009) [2023-12-27 01:12:02,948][105620] Updated weights for policy 1, policy_version 1349087 (0.0008) [2023-12-27 01:12:03,226][105692] Updated weights for policy 0, policy_version 1347091 (0.0006) [2023-12-27 01:12:03,284][105692] Updated weights for policy 0, policy_version 1347101 (0.0005) [2023-12-27 01:12:03,338][105692] Updated weights for policy 0, policy_version 1347111 (0.0006) [2023-12-27 01:12:03,661][105620] Updated weights for policy 1, policy_version 1349097 (0.0007) [2023-12-27 01:12:03,710][105620] Updated weights for policy 1, policy_version 1349107 (0.0005) [2023-12-27 01:12:03,763][105620] Updated weights for policy 1, policy_version 1349117 (0.0005) [2023-12-27 01:12:04,103][105692] Updated weights for policy 0, policy_version 1347121 (0.0009) [2023-12-27 01:12:04,164][105692] Updated weights for policy 0, policy_version 1347131 (0.0008) [2023-12-27 01:12:04,232][105692] Updated weights for policy 0, policy_version 1347141 (0.0007) [2023-12-27 01:12:04,288][105692] Updated weights for policy 0, policy_version 1347151 (0.0009) [2023-12-27 01:12:04,404][105620] Updated weights for policy 1, policy_version 1349127 (0.0007) [2023-12-27 01:12:04,468][105620] Updated weights for policy 1, policy_version 1349137 (0.0008) [2023-12-27 01:12:04,525][105620] Updated weights for policy 1, policy_version 1349147 (0.0005) [2023-12-27 01:12:05,037][105692] Updated weights for policy 0, policy_version 1347161 (0.0007) [2023-12-27 01:12:05,098][105692] Updated weights for policy 0, policy_version 1347171 (0.0008) [2023-12-27 01:12:05,145][105692] Updated weights for policy 0, policy_version 1347181 (0.0009) [2023-12-27 01:12:05,201][105620] Updated weights for policy 1, policy_version 1349157 (0.0007) [2023-12-27 01:12:05,259][105620] Updated weights for policy 1, policy_version 1349167 (0.0009) [2023-12-27 01:12:05,328][105620] Updated weights for policy 1, policy_version 1349177 (0.0009) [2023-12-27 01:12:05,836][105692] Updated weights for policy 0, policy_version 1347191 (0.0009) [2023-12-27 01:12:05,892][105692] Updated weights for policy 0, policy_version 1347201 (0.0009) [2023-12-27 01:12:05,952][105692] Updated weights for policy 0, policy_version 1347211 (0.0009) [2023-12-27 01:12:06,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 690372608. Throughput: 0: 9896.4, 1: 9645.0. Samples: 690361276. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:12:06,062][104569] Avg episode reward: [(0, '8813.926'), (1, '8556.861')] [2023-12-27 01:12:06,129][105620] Updated weights for policy 1, policy_version 1349187 (0.0009) [2023-12-27 01:12:06,196][105620] Updated weights for policy 1, policy_version 1349197 (0.0008) [2023-12-27 01:12:06,259][105620] Updated weights for policy 1, policy_version 1349207 (0.0009) [2023-12-27 01:12:06,751][105692] Updated weights for policy 0, policy_version 1347221 (0.0007) [2023-12-27 01:12:06,811][105692] Updated weights for policy 0, policy_version 1347231 (0.0005) [2023-12-27 01:12:06,856][105620] Updated weights for policy 1, policy_version 1349217 (0.0007) [2023-12-27 01:12:06,873][105692] Updated weights for policy 0, policy_version 1347241 (0.0006) [2023-12-27 01:12:06,914][105620] Updated weights for policy 1, policy_version 1349227 (0.0010) [2023-12-27 01:12:06,969][105620] Updated weights for policy 1, policy_version 1349237 (0.0009) [2023-12-27 01:12:07,019][105620] Updated weights for policy 1, policy_version 1349247 (0.0009) [2023-12-27 01:12:07,533][105692] Updated weights for policy 0, policy_version 1347251 (0.0005) [2023-12-27 01:12:07,595][105692] Updated weights for policy 0, policy_version 1347261 (0.0006) [2023-12-27 01:12:07,653][105692] Updated weights for policy 0, policy_version 1347271 (0.0009) [2023-12-27 01:12:07,820][105620] Updated weights for policy 1, policy_version 1349258 (0.0010) [2023-12-27 01:12:07,877][105620] Updated weights for policy 1, policy_version 1349268 (0.0011) [2023-12-27 01:12:07,936][105620] Updated weights for policy 1, policy_version 1349280 (0.0010) [2023-12-27 01:12:08,284][105692] Updated weights for policy 0, policy_version 1347281 (0.0009) [2023-12-27 01:12:08,351][105692] Updated weights for policy 0, policy_version 1347291 (0.0008) [2023-12-27 01:12:08,418][105692] Updated weights for policy 0, policy_version 1347301 (0.0008) [2023-12-27 01:12:08,480][105692] Updated weights for policy 0, policy_version 1347311 (0.0006) [2023-12-27 01:12:08,770][105620] Updated weights for policy 1, policy_version 1349290 (0.0008) [2023-12-27 01:12:08,817][105620] Updated weights for policy 1, policy_version 1349300 (0.0009) [2023-12-27 01:12:08,875][105620] Updated weights for policy 1, policy_version 1349310 (0.0008) [2023-12-27 01:12:09,118][105692] Updated weights for policy 0, policy_version 1347321 (0.0008) [2023-12-27 01:12:09,184][105692] Updated weights for policy 0, policy_version 1347331 (0.0007) [2023-12-27 01:12:09,248][105692] Updated weights for policy 0, policy_version 1347341 (0.0008) [2023-12-27 01:12:09,642][105620] Updated weights for policy 1, policy_version 1349320 (0.0008) [2023-12-27 01:12:09,707][105620] Updated weights for policy 1, policy_version 1349330 (0.0006) [2023-12-27 01:12:09,762][105620] Updated weights for policy 1, policy_version 1349340 (0.0009) [2023-12-27 01:12:10,010][105692] Updated weights for policy 0, policy_version 1347351 (0.0009) [2023-12-27 01:12:10,068][105692] Updated weights for policy 0, policy_version 1347361 (0.0007) [2023-12-27 01:12:10,129][105692] Updated weights for policy 0, policy_version 1347371 (0.0007) [2023-12-27 01:12:10,460][105620] Updated weights for policy 1, policy_version 1349350 (0.0009) [2023-12-27 01:12:10,523][105620] Updated weights for policy 1, policy_version 1349360 (0.0009) [2023-12-27 01:12:10,573][105620] Updated weights for policy 1, policy_version 1349370 (0.0007) [2023-12-27 01:12:10,803][105692] Updated weights for policy 0, policy_version 1347381 (0.0008) [2023-12-27 01:12:10,861][105692] Updated weights for policy 0, policy_version 1347391 (0.0009) [2023-12-27 01:12:10,916][105692] Updated weights for policy 0, policy_version 1347401 (0.0009) [2023-12-27 01:12:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 690470912. Throughput: 0: 9886.0, 1: 9657.2. Samples: 690476056. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:12:11,063][104569] Avg episode reward: [(0, '8996.925'), (1, '8283.673')] [2023-12-27 01:12:11,308][105620] Updated weights for policy 1, policy_version 1349380 (0.0007) [2023-12-27 01:12:11,368][105620] Updated weights for policy 1, policy_version 1349390 (0.0009) [2023-12-27 01:12:11,434][105620] Updated weights for policy 1, policy_version 1349400 (0.0009) [2023-12-27 01:12:11,719][105692] Updated weights for policy 0, policy_version 1347411 (0.0009) [2023-12-27 01:12:11,780][105692] Updated weights for policy 0, policy_version 1347421 (0.0008) [2023-12-27 01:12:11,844][105692] Updated weights for policy 0, policy_version 1347431 (0.0009) [2023-12-27 01:12:12,313][105620] Updated weights for policy 1, policy_version 1349410 (0.0009) [2023-12-27 01:12:12,378][105620] Updated weights for policy 1, policy_version 1349420 (0.0009) [2023-12-27 01:12:12,442][105620] Updated weights for policy 1, policy_version 1349430 (0.0009) [2023-12-27 01:12:12,503][105620] Updated weights for policy 1, policy_version 1349440 (0.0009) [2023-12-27 01:12:12,517][105692] Updated weights for policy 0, policy_version 1347441 (0.0009) [2023-12-27 01:12:12,569][105692] Updated weights for policy 0, policy_version 1347451 (0.0005) [2023-12-27 01:12:12,632][105692] Updated weights for policy 0, policy_version 1347461 (0.0005) [2023-12-27 01:12:12,697][105692] Updated weights for policy 0, policy_version 1347471 (0.0009) [2023-12-27 01:12:13,201][105620] Updated weights for policy 1, policy_version 1349450 (0.0005) [2023-12-27 01:12:13,259][105620] Updated weights for policy 1, policy_version 1349460 (0.0010) [2023-12-27 01:12:13,321][105620] Updated weights for policy 1, policy_version 1349470 (0.0010) [2023-12-27 01:12:13,419][105692] Updated weights for policy 0, policy_version 1347481 (0.0006) [2023-12-27 01:12:13,465][105692] Updated weights for policy 0, policy_version 1347491 (0.0006) [2023-12-27 01:12:13,519][105692] Updated weights for policy 0, policy_version 1347501 (0.0005) [2023-12-27 01:12:14,012][105620] Updated weights for policy 1, policy_version 1349480 (0.0008) [2023-12-27 01:12:14,047][105692] Updated weights for policy 0, policy_version 1347511 (0.0009) [2023-12-27 01:12:14,072][105620] Updated weights for policy 1, policy_version 1349490 (0.0005) [2023-12-27 01:12:14,113][105692] Updated weights for policy 0, policy_version 1347521 (0.0011) [2023-12-27 01:12:14,135][105620] Updated weights for policy 1, policy_version 1349500 (0.0006) [2023-12-27 01:12:14,172][105692] Updated weights for policy 0, policy_version 1347531 (0.0011) [2023-12-27 01:12:14,882][105620] Updated weights for policy 1, policy_version 1349510 (0.0007) [2023-12-27 01:12:14,884][105692] Updated weights for policy 0, policy_version 1347541 (0.0007) [2023-12-27 01:12:14,949][105620] Updated weights for policy 1, policy_version 1349520 (0.0007) [2023-12-27 01:12:14,949][105692] Updated weights for policy 0, policy_version 1347551 (0.0008) [2023-12-27 01:12:15,008][105620] Updated weights for policy 1, policy_version 1349530 (0.0008) [2023-12-27 01:12:15,010][105692] Updated weights for policy 0, policy_version 1347561 (0.0009) [2023-12-27 01:12:15,753][105620] Updated weights for policy 1, policy_version 1349540 (0.0007) [2023-12-27 01:12:15,759][105692] Updated weights for policy 0, policy_version 1347571 (0.0008) [2023-12-27 01:12:15,811][105692] Updated weights for policy 0, policy_version 1347581 (0.0006) [2023-12-27 01:12:15,812][105620] Updated weights for policy 1, policy_version 1349550 (0.0008) [2023-12-27 01:12:15,860][105692] Updated weights for policy 0, policy_version 1347591 (0.0008) [2023-12-27 01:12:15,863][105620] Updated weights for policy 1, policy_version 1349560 (0.0007) [2023-12-27 01:12:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.4, 300 sec: 19521.9). Total num frames: 690569216. Throughput: 0: 9819.5, 1: 9571.3. Samples: 690533728. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:12:16,063][104569] Avg episode reward: [(0, '8545.487'), (1, '8285.982')] [2023-12-27 01:12:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001349568_345530368.pth... [2023-12-27 01:12:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001347600_345038848.pth... [2023-12-27 01:12:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001348448_345243648.pth [2023-12-27 01:12:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001346448_344743936.pth [2023-12-27 01:12:16,600][105620] Updated weights for policy 1, policy_version 1349570 (0.0009) [2023-12-27 01:12:16,626][105692] Updated weights for policy 0, policy_version 1347601 (0.0005) [2023-12-27 01:12:16,651][105620] Updated weights for policy 1, policy_version 1349580 (0.0009) [2023-12-27 01:12:16,684][105692] Updated weights for policy 0, policy_version 1347611 (0.0007) [2023-12-27 01:12:16,698][105620] Updated weights for policy 1, policy_version 1349590 (0.0006) [2023-12-27 01:12:16,736][105692] Updated weights for policy 0, policy_version 1347621 (0.0006) [2023-12-27 01:12:16,754][105620] Updated weights for policy 1, policy_version 1349600 (0.0008) [2023-12-27 01:12:16,789][105692] Updated weights for policy 0, policy_version 1347631 (0.0008) [2023-12-27 01:12:17,503][105620] Updated weights for policy 1, policy_version 1349610 (0.0009) [2023-12-27 01:12:17,543][105692] Updated weights for policy 0, policy_version 1347641 (0.0007) [2023-12-27 01:12:17,558][105620] Updated weights for policy 1, policy_version 1349620 (0.0008) [2023-12-27 01:12:17,595][105692] Updated weights for policy 0, policy_version 1347651 (0.0005) [2023-12-27 01:12:17,618][105620] Updated weights for policy 1, policy_version 1349630 (0.0009) [2023-12-27 01:12:17,647][105692] Updated weights for policy 0, policy_version 1347661 (0.0005) [2023-12-27 01:12:18,334][105692] Updated weights for policy 0, policy_version 1347671 (0.0007) [2023-12-27 01:12:18,394][105620] Updated weights for policy 1, policy_version 1349640 (0.0007) [2023-12-27 01:12:18,398][105692] Updated weights for policy 0, policy_version 1347681 (0.0009) [2023-12-27 01:12:18,453][105692] Updated weights for policy 0, policy_version 1347691 (0.0009) [2023-12-27 01:12:18,462][105620] Updated weights for policy 1, policy_version 1349650 (0.0005) [2023-12-27 01:12:18,506][105586] KL-divergence is very high: 109.5665 [2023-12-27 01:12:18,533][105620] Updated weights for policy 1, policy_version 1349660 (0.0006) [2023-12-27 01:12:19,119][105620] Updated weights for policy 1, policy_version 1349670 (0.0007) [2023-12-27 01:12:19,174][105620] Updated weights for policy 1, policy_version 1349680 (0.0009) [2023-12-27 01:12:19,237][105620] Updated weights for policy 1, policy_version 1349690 (0.0008) [2023-12-27 01:12:19,300][105692] Updated weights for policy 0, policy_version 1347701 (0.0007) [2023-12-27 01:12:19,367][105692] Updated weights for policy 0, policy_version 1347711 (0.0009) [2023-12-27 01:12:19,425][105692] Updated weights for policy 0, policy_version 1347721 (0.0009) [2023-12-27 01:12:19,999][105620] Updated weights for policy 1, policy_version 1349700 (0.0007) [2023-12-27 01:12:20,068][105620] Updated weights for policy 1, policy_version 1349710 (0.0007) [2023-12-27 01:12:20,126][105620] Updated weights for policy 1, policy_version 1349720 (0.0009) [2023-12-27 01:12:20,233][105692] Updated weights for policy 0, policy_version 1347731 (0.0008) [2023-12-27 01:12:20,285][105692] Updated weights for policy 0, policy_version 1347741 (0.0009) [2023-12-27 01:12:20,340][105692] Updated weights for policy 0, policy_version 1347751 (0.0009) [2023-12-27 01:12:20,824][105620] Updated weights for policy 1, policy_version 1349730 (0.0008) [2023-12-27 01:12:20,873][105620] Updated weights for policy 1, policy_version 1349740 (0.0008) [2023-12-27 01:12:20,926][105620] Updated weights for policy 1, policy_version 1349750 (0.0008) [2023-12-27 01:12:20,978][105620] Updated weights for policy 1, policy_version 1349760 (0.0008) [2023-12-27 01:12:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 690659328. Throughput: 0: 9725.9, 1: 9534.0. Samples: 690648580. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:12:21,062][104569] Avg episode reward: [(0, '8452.542'), (1, '8563.394')] [2023-12-27 01:12:21,165][105692] Updated weights for policy 0, policy_version 1347761 (0.0009) [2023-12-27 01:12:21,231][105692] Updated weights for policy 0, policy_version 1347771 (0.0011) [2023-12-27 01:12:21,292][105692] Updated weights for policy 0, policy_version 1347781 (0.0011) [2023-12-27 01:12:21,355][105692] Updated weights for policy 0, policy_version 1347791 (0.0011) [2023-12-27 01:12:21,712][105620] Updated weights for policy 1, policy_version 1349770 (0.0006) [2023-12-27 01:12:21,780][105620] Updated weights for policy 1, policy_version 1349780 (0.0008) [2023-12-27 01:12:21,839][105620] Updated weights for policy 1, policy_version 1349790 (0.0008) [2023-12-27 01:12:22,182][105692] Updated weights for policy 0, policy_version 1347801 (0.0010) [2023-12-27 01:12:22,253][105692] Updated weights for policy 0, policy_version 1347811 (0.0010) [2023-12-27 01:12:22,320][105692] Updated weights for policy 0, policy_version 1347821 (0.0008) [2023-12-27 01:12:22,526][105620] Updated weights for policy 1, policy_version 1349800 (0.0009) [2023-12-27 01:12:22,591][105620] Updated weights for policy 1, policy_version 1349810 (0.0009) [2023-12-27 01:12:22,656][105620] Updated weights for policy 1, policy_version 1349820 (0.0009) [2023-12-27 01:12:23,070][105692] Updated weights for policy 0, policy_version 1347831 (0.0007) [2023-12-27 01:12:23,144][105692] Updated weights for policy 0, policy_version 1347841 (0.0010) [2023-12-27 01:12:23,204][105692] Updated weights for policy 0, policy_version 1347851 (0.0009) [2023-12-27 01:12:23,395][105620] Updated weights for policy 1, policy_version 1349830 (0.0009) [2023-12-27 01:12:23,444][105620] Updated weights for policy 1, policy_version 1349840 (0.0009) [2023-12-27 01:12:23,488][105620] Updated weights for policy 1, policy_version 1349850 (0.0009) [2023-12-27 01:12:23,898][105692] Updated weights for policy 0, policy_version 1347861 (0.0008) [2023-12-27 01:12:23,945][105692] Updated weights for policy 0, policy_version 1347871 (0.0009) [2023-12-27 01:12:24,000][105692] Updated weights for policy 0, policy_version 1347881 (0.0009) [2023-12-27 01:12:24,192][105620] Updated weights for policy 1, policy_version 1349860 (0.0008) [2023-12-27 01:12:24,250][105620] Updated weights for policy 1, policy_version 1349870 (0.0009) [2023-12-27 01:12:24,299][105620] Updated weights for policy 1, policy_version 1349880 (0.0009) [2023-12-27 01:12:24,773][105692] Updated weights for policy 0, policy_version 1347891 (0.0009) [2023-12-27 01:12:24,820][105692] Updated weights for policy 0, policy_version 1347901 (0.0009) [2023-12-27 01:12:24,868][105692] Updated weights for policy 0, policy_version 1347911 (0.0009) [2023-12-27 01:12:25,056][105620] Updated weights for policy 1, policy_version 1349890 (0.0008) [2023-12-27 01:12:25,105][105620] Updated weights for policy 1, policy_version 1349900 (0.0008) [2023-12-27 01:12:25,155][105620] Updated weights for policy 1, policy_version 1349910 (0.0009) [2023-12-27 01:12:25,201][105620] Updated weights for policy 1, policy_version 1349920 (0.0008) [2023-12-27 01:12:25,652][105692] Updated weights for policy 0, policy_version 1347921 (0.0009) [2023-12-27 01:12:25,699][105692] Updated weights for policy 0, policy_version 1347931 (0.0009) [2023-12-27 01:12:25,745][105692] Updated weights for policy 0, policy_version 1347941 (0.0008) [2023-12-27 01:12:25,792][105692] Updated weights for policy 0, policy_version 1347951 (0.0006) [2023-12-27 01:12:25,966][105620] Updated weights for policy 1, policy_version 1349930 (0.0008) [2023-12-27 01:12:26,012][105620] Updated weights for policy 1, policy_version 1349940 (0.0009) [2023-12-27 01:12:26,062][105620] Updated weights for policy 1, policy_version 1349950 (0.0008) [2023-12-27 01:12:26,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 690749440. Throughput: 0: 9641.1, 1: 9498.0. Samples: 690760632. Policy #0 lag: (min: 31.0, avg: 31.1, max: 42.0) [2023-12-27 01:12:26,062][104569] Avg episode reward: [(0, '8717.781'), (1, '8299.696')] [2023-12-27 01:12:26,569][105692] Updated weights for policy 0, policy_version 1347961 (0.0009) [2023-12-27 01:12:26,643][105692] Updated weights for policy 0, policy_version 1347971 (0.0009) [2023-12-27 01:12:26,706][105692] Updated weights for policy 0, policy_version 1347981 (0.0010) [2023-12-27 01:12:26,762][105620] Updated weights for policy 1, policy_version 1349960 (0.0008) [2023-12-27 01:12:26,816][105620] Updated weights for policy 1, policy_version 1349970 (0.0009) [2023-12-27 01:12:26,871][105620] Updated weights for policy 1, policy_version 1349980 (0.0009) [2023-12-27 01:12:27,518][105620] Updated weights for policy 1, policy_version 1349990 (0.0010) [2023-12-27 01:12:27,522][105692] Updated weights for policy 0, policy_version 1347991 (0.0006) [2023-12-27 01:12:27,563][105620] Updated weights for policy 1, policy_version 1350000 (0.0010) [2023-12-27 01:12:27,576][105692] Updated weights for policy 0, policy_version 1348001 (0.0005) [2023-12-27 01:12:27,607][105620] Updated weights for policy 1, policy_version 1350010 (0.0010) [2023-12-27 01:12:27,623][105692] Updated weights for policy 0, policy_version 1348011 (0.0005) [2023-12-27 01:12:28,187][105692] Updated weights for policy 0, policy_version 1348021 (0.0008) [2023-12-27 01:12:28,247][105692] Updated weights for policy 0, policy_version 1348031 (0.0009) [2023-12-27 01:12:28,306][105620] Updated weights for policy 1, policy_version 1350020 (0.0010) [2023-12-27 01:12:28,308][105692] Updated weights for policy 0, policy_version 1348041 (0.0005) [2023-12-27 01:12:28,370][105620] Updated weights for policy 1, policy_version 1350030 (0.0011) [2023-12-27 01:12:28,426][105620] Updated weights for policy 1, policy_version 1350040 (0.0011) [2023-12-27 01:12:29,011][105692] Updated weights for policy 0, policy_version 1348051 (0.0007) [2023-12-27 01:12:29,075][105692] Updated weights for policy 0, policy_version 1348061 (0.0006) [2023-12-27 01:12:29,140][105692] Updated weights for policy 0, policy_version 1348071 (0.0011) [2023-12-27 01:12:29,175][105620] Updated weights for policy 1, policy_version 1350050 (0.0010) [2023-12-27 01:12:29,236][105620] Updated weights for policy 1, policy_version 1350060 (0.0011) [2023-12-27 01:12:29,292][105620] Updated weights for policy 1, policy_version 1350070 (0.0010) [2023-12-27 01:12:29,349][105620] Updated weights for policy 1, policy_version 1350080 (0.0011) [2023-12-27 01:12:29,791][105692] Updated weights for policy 0, policy_version 1348081 (0.0010) [2023-12-27 01:12:29,859][105692] Updated weights for policy 0, policy_version 1348091 (0.0009) [2023-12-27 01:12:29,915][105692] Updated weights for policy 0, policy_version 1348101 (0.0010) [2023-12-27 01:12:29,977][105692] Updated weights for policy 0, policy_version 1348111 (0.0009) [2023-12-27 01:12:30,137][105620] Updated weights for policy 1, policy_version 1350090 (0.0011) [2023-12-27 01:12:30,195][105620] Updated weights for policy 1, policy_version 1350100 (0.0010) [2023-12-27 01:12:30,243][105620] Updated weights for policy 1, policy_version 1350110 (0.0010) [2023-12-27 01:12:30,640][105692] Updated weights for policy 0, policy_version 1348121 (0.0005) [2023-12-27 01:12:30,691][105692] Updated weights for policy 0, policy_version 1348131 (0.0006) [2023-12-27 01:12:30,692][105585] KL-divergence is very high: 178.0898 [2023-12-27 01:12:30,730][105585] KL-divergence is very high: 338.1863 [2023-12-27 01:12:30,753][105692] Updated weights for policy 0, policy_version 1348141 (0.0008) [2023-12-27 01:12:31,004][105620] Updated weights for policy 1, policy_version 1350120 (0.0010) [2023-12-27 01:12:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 690847744. Throughput: 0: 9635.8, 1: 9559.3. Samples: 690820132. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:12:31,062][104569] Avg episode reward: [(0, '8262.815'), (1, '8464.117')] [2023-12-27 01:12:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001348144_345178112.pth... [2023-12-27 01:12:31,069][105620] Updated weights for policy 1, policy_version 1350130 (0.0010) [2023-12-27 01:12:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001347024_344891392.pth [2023-12-27 01:12:31,127][105620] Updated weights for policy 1, policy_version 1350140 (0.0010) [2023-12-27 01:12:31,153][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001350144_345677824.pth... [2023-12-27 01:12:31,158][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001348992_345382912.pth [2023-12-27 01:12:31,402][105692] Updated weights for policy 0, policy_version 1348151 (0.0007) [2023-12-27 01:12:31,461][105692] Updated weights for policy 0, policy_version 1348161 (0.0005) [2023-12-27 01:12:31,523][105692] Updated weights for policy 0, policy_version 1348171 (0.0007) [2023-12-27 01:12:31,901][105620] Updated weights for policy 1, policy_version 1350150 (0.0010) [2023-12-27 01:12:31,959][105620] Updated weights for policy 1, policy_version 1350160 (0.0009) [2023-12-27 01:12:32,010][105620] Updated weights for policy 1, policy_version 1350170 (0.0009) [2023-12-27 01:12:32,174][105692] Updated weights for policy 0, policy_version 1348181 (0.0008) [2023-12-27 01:12:32,228][105692] Updated weights for policy 0, policy_version 1348191 (0.0008) [2023-12-27 01:12:32,285][105692] Updated weights for policy 0, policy_version 1348201 (0.0009) [2023-12-27 01:12:32,646][105620] Updated weights for policy 1, policy_version 1350180 (0.0007) [2023-12-27 01:12:32,695][105620] Updated weights for policy 1, policy_version 1350190 (0.0005) [2023-12-27 01:12:32,755][105620] Updated weights for policy 1, policy_version 1350200 (0.0008) [2023-12-27 01:12:33,090][105692] Updated weights for policy 0, policy_version 1348211 (0.0008) [2023-12-27 01:12:33,155][105692] Updated weights for policy 0, policy_version 1348221 (0.0009) [2023-12-27 01:12:33,210][105692] Updated weights for policy 0, policy_version 1348231 (0.0009) [2023-12-27 01:12:33,454][105620] Updated weights for policy 1, policy_version 1350210 (0.0009) [2023-12-27 01:12:33,516][105620] Updated weights for policy 1, policy_version 1350220 (0.0010) [2023-12-27 01:12:33,571][105620] Updated weights for policy 1, policy_version 1350230 (0.0010) [2023-12-27 01:12:33,623][105620] Updated weights for policy 1, policy_version 1350240 (0.0009) [2023-12-27 01:12:33,817][105692] Updated weights for policy 0, policy_version 1348241 (0.0008) [2023-12-27 01:12:33,877][105692] Updated weights for policy 0, policy_version 1348251 (0.0005) [2023-12-27 01:12:33,930][105692] Updated weights for policy 0, policy_version 1348262 (0.0010) [2023-12-27 01:12:34,335][105620] Updated weights for policy 1, policy_version 1350250 (0.0009) [2023-12-27 01:12:34,401][105620] Updated weights for policy 1, policy_version 1350260 (0.0008) [2023-12-27 01:12:34,464][105620] Updated weights for policy 1, policy_version 1350270 (0.0009) [2023-12-27 01:12:34,713][105692] Updated weights for policy 0, policy_version 1348273 (0.0010) [2023-12-27 01:12:34,766][105692] Updated weights for policy 0, policy_version 1348283 (0.0010) [2023-12-27 01:12:34,824][105692] Updated weights for policy 0, policy_version 1348293 (0.0010) [2023-12-27 01:12:34,878][105692] Updated weights for policy 0, policy_version 1348303 (0.0010) [2023-12-27 01:12:35,118][105620] Updated weights for policy 1, policy_version 1350280 (0.0009) [2023-12-27 01:12:35,180][105620] Updated weights for policy 1, policy_version 1350290 (0.0009) [2023-12-27 01:12:35,244][105620] Updated weights for policy 1, policy_version 1350300 (0.0009) [2023-12-27 01:12:35,625][105692] Updated weights for policy 0, policy_version 1348313 (0.0011) [2023-12-27 01:12:35,676][105692] Updated weights for policy 0, policy_version 1348323 (0.0009) [2023-12-27 01:12:35,733][105692] Updated weights for policy 0, policy_version 1348333 (0.0005) [2023-12-27 01:12:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 690946048. Throughput: 0: 9595.7, 1: 9615.4. Samples: 690938244. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:12:36,062][104569] Avg episode reward: [(0, '8360.376'), (1, '8368.388')] [2023-12-27 01:12:36,067][105620] Updated weights for policy 1, policy_version 1350310 (0.0008) [2023-12-27 01:12:36,136][105620] Updated weights for policy 1, policy_version 1350320 (0.0008) [2023-12-27 01:12:36,194][105620] Updated weights for policy 1, policy_version 1350330 (0.0006) [2023-12-27 01:12:36,392][105692] Updated weights for policy 0, policy_version 1348343 (0.0008) [2023-12-27 01:12:36,447][105692] Updated weights for policy 0, policy_version 1348353 (0.0009) [2023-12-27 01:12:36,501][105692] Updated weights for policy 0, policy_version 1348363 (0.0009) [2023-12-27 01:12:36,875][105620] Updated weights for policy 1, policy_version 1350340 (0.0007) [2023-12-27 01:12:36,933][105620] Updated weights for policy 1, policy_version 1350350 (0.0008) [2023-12-27 01:12:36,989][105620] Updated weights for policy 1, policy_version 1350360 (0.0008) [2023-12-27 01:12:37,320][105692] Updated weights for policy 0, policy_version 1348373 (0.0009) [2023-12-27 01:12:37,379][105692] Updated weights for policy 0, policy_version 1348383 (0.0009) [2023-12-27 01:12:37,440][105692] Updated weights for policy 0, policy_version 1348393 (0.0010) [2023-12-27 01:12:37,718][105620] Updated weights for policy 1, policy_version 1350370 (0.0008) [2023-12-27 01:12:37,779][105620] Updated weights for policy 1, policy_version 1350380 (0.0005) [2023-12-27 01:12:37,830][105620] Updated weights for policy 1, policy_version 1350390 (0.0005) [2023-12-27 01:12:37,887][105620] Updated weights for policy 1, policy_version 1350400 (0.0008) [2023-12-27 01:12:38,054][105692] Updated weights for policy 0, policy_version 1348403 (0.0008) [2023-12-27 01:12:38,115][105692] Updated weights for policy 0, policy_version 1348413 (0.0005) [2023-12-27 01:12:38,168][105692] Updated weights for policy 0, policy_version 1348423 (0.0005) [2023-12-27 01:12:38,667][105620] Updated weights for policy 1, policy_version 1350410 (0.0010) [2023-12-27 01:12:38,727][105620] Updated weights for policy 1, policy_version 1350420 (0.0009) [2023-12-27 01:12:38,731][105692] Updated weights for policy 0, policy_version 1348433 (0.0005) [2023-12-27 01:12:38,789][105692] Updated weights for policy 0, policy_version 1348443 (0.0007) [2023-12-27 01:12:38,791][105620] Updated weights for policy 1, policy_version 1350430 (0.0008) [2023-12-27 01:12:38,851][105692] Updated weights for policy 0, policy_version 1348453 (0.0009) [2023-12-27 01:12:38,910][105692] Updated weights for policy 0, policy_version 1348463 (0.0009) [2023-12-27 01:12:39,548][105620] Updated weights for policy 1, policy_version 1350440 (0.0009) [2023-12-27 01:12:39,610][105620] Updated weights for policy 1, policy_version 1350450 (0.0009) [2023-12-27 01:12:39,673][105620] Updated weights for policy 1, policy_version 1350460 (0.0008) [2023-12-27 01:12:39,717][105692] Updated weights for policy 0, policy_version 1348473 (0.0008) [2023-12-27 01:12:39,775][105692] Updated weights for policy 0, policy_version 1348483 (0.0010) [2023-12-27 01:12:39,836][105692] Updated weights for policy 0, policy_version 1348493 (0.0009) [2023-12-27 01:12:40,371][105620] Updated weights for policy 1, policy_version 1350470 (0.0008) [2023-12-27 01:12:40,436][105620] Updated weights for policy 1, policy_version 1350480 (0.0007) [2023-12-27 01:12:40,484][105620] Updated weights for policy 1, policy_version 1350490 (0.0007) [2023-12-27 01:12:40,647][105692] Updated weights for policy 0, policy_version 1348503 (0.0009) [2023-12-27 01:12:40,695][105692] Updated weights for policy 0, policy_version 1348513 (0.0009) [2023-12-27 01:12:40,746][105692] Updated weights for policy 0, policy_version 1348523 (0.0009) [2023-12-27 01:12:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 691044352. Throughput: 0: 9598.5, 1: 9605.5. Samples: 691053968. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:12:41,063][104569] Avg episode reward: [(0, '8361.746'), (1, '8367.387')] [2023-12-27 01:12:41,182][105620] Updated weights for policy 1, policy_version 1350500 (0.0007) [2023-12-27 01:12:41,250][105620] Updated weights for policy 1, policy_version 1350510 (0.0008) [2023-12-27 01:12:41,312][105620] Updated weights for policy 1, policy_version 1350520 (0.0009) [2023-12-27 01:12:41,511][105692] Updated weights for policy 0, policy_version 1348533 (0.0009) [2023-12-27 01:12:41,574][105692] Updated weights for policy 0, policy_version 1348543 (0.0009) [2023-12-27 01:12:41,641][105692] Updated weights for policy 0, policy_version 1348553 (0.0008) [2023-12-27 01:12:42,050][105620] Updated weights for policy 1, policy_version 1350530 (0.0009) [2023-12-27 01:12:42,114][105620] Updated weights for policy 1, policy_version 1350540 (0.0010) [2023-12-27 01:12:42,168][105620] Updated weights for policy 1, policy_version 1350550 (0.0008) [2023-12-27 01:12:42,237][105620] Updated weights for policy 1, policy_version 1350560 (0.0009) [2023-12-27 01:12:42,364][105692] Updated weights for policy 0, policy_version 1348563 (0.0010) [2023-12-27 01:12:42,432][105692] Updated weights for policy 0, policy_version 1348573 (0.0006) [2023-12-27 01:12:42,493][105692] Updated weights for policy 0, policy_version 1348583 (0.0009) [2023-12-27 01:12:43,040][105620] Updated weights for policy 1, policy_version 1350570 (0.0009) [2023-12-27 01:12:43,100][105620] Updated weights for policy 1, policy_version 1350580 (0.0009) [2023-12-27 01:12:43,161][105620] Updated weights for policy 1, policy_version 1350590 (0.0009) [2023-12-27 01:12:43,171][105692] Updated weights for policy 0, policy_version 1348593 (0.0010) [2023-12-27 01:12:43,219][105692] Updated weights for policy 0, policy_version 1348603 (0.0008) [2023-12-27 01:12:43,262][105692] Updated weights for policy 0, policy_version 1348613 (0.0007) [2023-12-27 01:12:43,310][105692] Updated weights for policy 0, policy_version 1348623 (0.0008) [2023-12-27 01:12:43,817][105620] Updated weights for policy 1, policy_version 1350600 (0.0006) [2023-12-27 01:12:43,865][105620] Updated weights for policy 1, policy_version 1350610 (0.0005) [2023-12-27 01:12:43,917][105620] Updated weights for policy 1, policy_version 1350620 (0.0005) [2023-12-27 01:12:43,954][105692] Updated weights for policy 0, policy_version 1348633 (0.0009) [2023-12-27 01:12:44,012][105692] Updated weights for policy 0, policy_version 1348645 (0.0010) [2023-12-27 01:12:44,075][105692] Updated weights for policy 0, policy_version 1348655 (0.0010) [2023-12-27 01:12:44,464][105620] Updated weights for policy 1, policy_version 1350630 (0.0006) [2023-12-27 01:12:44,525][105620] Updated weights for policy 1, policy_version 1350640 (0.0006) [2023-12-27 01:12:44,585][105620] Updated weights for policy 1, policy_version 1350650 (0.0010) [2023-12-27 01:12:44,951][105692] Updated weights for policy 0, policy_version 1348665 (0.0008) [2023-12-27 01:12:45,011][105692] Updated weights for policy 0, policy_version 1348675 (0.0009) [2023-12-27 01:12:45,078][105692] Updated weights for policy 0, policy_version 1348685 (0.0008) [2023-12-27 01:12:45,304][105620] Updated weights for policy 1, policy_version 1350660 (0.0010) [2023-12-27 01:12:45,362][105620] Updated weights for policy 1, policy_version 1350670 (0.0010) [2023-12-27 01:12:45,421][105620] Updated weights for policy 1, policy_version 1350680 (0.0011) [2023-12-27 01:12:45,785][105692] Updated weights for policy 0, policy_version 1348695 (0.0008) [2023-12-27 01:12:45,851][105692] Updated weights for policy 0, policy_version 1348705 (0.0009) [2023-12-27 01:12:45,919][105692] Updated weights for policy 0, policy_version 1348715 (0.0009) [2023-12-27 01:12:46,062][104569] Fps is (10 sec: 19660.0, 60 sec: 19251.1, 300 sec: 19494.2). Total num frames: 691142656. Throughput: 0: 9614.1, 1: 9642.2. Samples: 691111648. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:12:46,063][104569] Avg episode reward: [(0, '8632.446'), (1, '8283.339')] [2023-12-27 01:12:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001350688_345817088.pth... [2023-12-27 01:12:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001348720_345325568.pth... [2023-12-27 01:12:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001349568_345530368.pth [2023-12-27 01:12:46,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001347600_345038848.pth [2023-12-27 01:12:46,151][105620] Updated weights for policy 1, policy_version 1350690 (0.0009) [2023-12-27 01:12:46,203][105620] Updated weights for policy 1, policy_version 1350700 (0.0007) [2023-12-27 01:12:46,254][105620] Updated weights for policy 1, policy_version 1350710 (0.0007) [2023-12-27 01:12:46,312][105620] Updated weights for policy 1, policy_version 1350720 (0.0005) [2023-12-27 01:12:46,636][105692] Updated weights for policy 0, policy_version 1348725 (0.0007) [2023-12-27 01:12:46,686][105692] Updated weights for policy 0, policy_version 1348735 (0.0005) [2023-12-27 01:12:46,737][105692] Updated weights for policy 0, policy_version 1348745 (0.0005) [2023-12-27 01:12:47,114][105620] Updated weights for policy 1, policy_version 1350730 (0.0009) [2023-12-27 01:12:47,168][105620] Updated weights for policy 1, policy_version 1350741 (0.0010) [2023-12-27 01:12:47,225][105620] Updated weights for policy 1, policy_version 1350752 (0.0009) [2023-12-27 01:12:47,262][105692] Updated weights for policy 0, policy_version 1348755 (0.0007) [2023-12-27 01:12:47,320][105692] Updated weights for policy 0, policy_version 1348765 (0.0010) [2023-12-27 01:12:47,374][105692] Updated weights for policy 0, policy_version 1348775 (0.0010) [2023-12-27 01:12:48,024][105692] Updated weights for policy 0, policy_version 1348785 (0.0010) [2023-12-27 01:12:48,052][105620] Updated weights for policy 1, policy_version 1350762 (0.0008) [2023-12-27 01:12:48,075][105692] Updated weights for policy 0, policy_version 1348795 (0.0008) [2023-12-27 01:12:48,101][105620] Updated weights for policy 1, policy_version 1350772 (0.0005) [2023-12-27 01:12:48,121][105692] Updated weights for policy 0, policy_version 1348805 (0.0008) [2023-12-27 01:12:48,155][105620] Updated weights for policy 1, policy_version 1350782 (0.0006) [2023-12-27 01:12:48,179][105692] Updated weights for policy 0, policy_version 1348815 (0.0009) [2023-12-27 01:12:48,902][105692] Updated weights for policy 0, policy_version 1348825 (0.0009) [2023-12-27 01:12:48,962][105620] Updated weights for policy 1, policy_version 1350792 (0.0009) [2023-12-27 01:12:48,967][105692] Updated weights for policy 0, policy_version 1348835 (0.0008) [2023-12-27 01:12:49,012][105620] Updated weights for policy 1, policy_version 1350802 (0.0007) [2023-12-27 01:12:49,034][105692] Updated weights for policy 0, policy_version 1348845 (0.0008) [2023-12-27 01:12:49,077][105620] Updated weights for policy 1, policy_version 1350812 (0.0008) [2023-12-27 01:12:49,734][105692] Updated weights for policy 0, policy_version 1348855 (0.0011) [2023-12-27 01:12:49,772][105620] Updated weights for policy 1, policy_version 1350822 (0.0007) [2023-12-27 01:12:49,797][105692] Updated weights for policy 0, policy_version 1348865 (0.0011) [2023-12-27 01:12:49,833][105620] Updated weights for policy 1, policy_version 1350832 (0.0007) [2023-12-27 01:12:49,855][105692] Updated weights for policy 0, policy_version 1348875 (0.0009) [2023-12-27 01:12:49,892][105620] Updated weights for policy 1, policy_version 1350842 (0.0007) [2023-12-27 01:12:50,478][105692] Updated weights for policy 0, policy_version 1348885 (0.0007) [2023-12-27 01:12:50,539][105692] Updated weights for policy 0, policy_version 1348895 (0.0006) [2023-12-27 01:12:50,604][105692] Updated weights for policy 0, policy_version 1348905 (0.0008) [2023-12-27 01:12:50,714][105620] Updated weights for policy 1, policy_version 1350852 (0.0008) [2023-12-27 01:12:50,773][105620] Updated weights for policy 1, policy_version 1350862 (0.0007) [2023-12-27 01:12:50,830][105620] Updated weights for policy 1, policy_version 1350872 (0.0008) [2023-12-27 01:12:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 691240960. Throughput: 0: 9717.5, 1: 9572.9. Samples: 691229344. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:12:51,063][104569] Avg episode reward: [(0, '8720.424'), (1, '8103.083')] [2023-12-27 01:12:51,319][105692] Updated weights for policy 0, policy_version 1348915 (0.0009) [2023-12-27 01:12:51,384][105692] Updated weights for policy 0, policy_version 1348925 (0.0008) [2023-12-27 01:12:51,450][105692] Updated weights for policy 0, policy_version 1348935 (0.0010) [2023-12-27 01:12:51,627][105620] Updated weights for policy 1, policy_version 1350882 (0.0007) [2023-12-27 01:12:51,682][105620] Updated weights for policy 1, policy_version 1350892 (0.0007) [2023-12-27 01:12:51,733][105620] Updated weights for policy 1, policy_version 1350902 (0.0006) [2023-12-27 01:12:51,811][105620] Updated weights for policy 1, policy_version 1350912 (0.0009) [2023-12-27 01:12:52,206][105692] Updated weights for policy 0, policy_version 1348945 (0.0009) [2023-12-27 01:12:52,272][105692] Updated weights for policy 0, policy_version 1348955 (0.0009) [2023-12-27 01:12:52,333][105692] Updated weights for policy 0, policy_version 1348965 (0.0009) [2023-12-27 01:12:52,399][105692] Updated weights for policy 0, policy_version 1348975 (0.0008) [2023-12-27 01:12:52,535][105620] Updated weights for policy 1, policy_version 1350922 (0.0007) [2023-12-27 01:12:52,588][105620] Updated weights for policy 1, policy_version 1350932 (0.0008) [2023-12-27 01:12:52,654][105620] Updated weights for policy 1, policy_version 1350942 (0.0008) [2023-12-27 01:12:53,182][105692] Updated weights for policy 0, policy_version 1348985 (0.0009) [2023-12-27 01:12:53,244][105692] Updated weights for policy 0, policy_version 1348995 (0.0009) [2023-12-27 01:12:53,302][105692] Updated weights for policy 0, policy_version 1349005 (0.0009) [2023-12-27 01:12:53,416][105620] Updated weights for policy 1, policy_version 1350952 (0.0007) [2023-12-27 01:12:53,466][105620] Updated weights for policy 1, policy_version 1350962 (0.0009) [2023-12-27 01:12:53,516][105620] Updated weights for policy 1, policy_version 1350972 (0.0008) [2023-12-27 01:12:54,094][105692] Updated weights for policy 0, policy_version 1349015 (0.0010) [2023-12-27 01:12:54,151][105692] Updated weights for policy 0, policy_version 1349025 (0.0009) [2023-12-27 01:12:54,188][105620] Updated weights for policy 1, policy_version 1350982 (0.0008) [2023-12-27 01:12:54,210][105692] Updated weights for policy 0, policy_version 1349035 (0.0008) [2023-12-27 01:12:54,242][105620] Updated weights for policy 1, policy_version 1350992 (0.0005) [2023-12-27 01:12:54,291][105620] Updated weights for policy 1, policy_version 1351002 (0.0006) [2023-12-27 01:12:55,000][105692] Updated weights for policy 0, policy_version 1349045 (0.0009) [2023-12-27 01:12:55,008][105620] Updated weights for policy 1, policy_version 1351012 (0.0008) [2023-12-27 01:12:55,054][105692] Updated weights for policy 0, policy_version 1349055 (0.0010) [2023-12-27 01:12:55,068][105620] Updated weights for policy 1, policy_version 1351022 (0.0006) [2023-12-27 01:12:55,106][105692] Updated weights for policy 0, policy_version 1349065 (0.0011) [2023-12-27 01:12:55,128][105620] Updated weights for policy 1, policy_version 1351032 (0.0005) [2023-12-27 01:12:55,775][105692] Updated weights for policy 0, policy_version 1349075 (0.0009) [2023-12-27 01:12:55,830][105692] Updated weights for policy 0, policy_version 1349085 (0.0008) [2023-12-27 01:12:55,889][105692] Updated weights for policy 0, policy_version 1349095 (0.0008) [2023-12-27 01:12:55,908][105620] Updated weights for policy 1, policy_version 1351042 (0.0008) [2023-12-27 01:12:55,958][105620] Updated weights for policy 1, policy_version 1351052 (0.0010) [2023-12-27 01:12:56,013][105620] Updated weights for policy 1, policy_version 1351062 (0.0010) [2023-12-27 01:12:56,062][104569] Fps is (10 sec: 18842.4, 60 sec: 19251.3, 300 sec: 19466.4). Total num frames: 691331072. Throughput: 0: 9655.7, 1: 9564.5. Samples: 691340960. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:12:56,062][104569] Avg episode reward: [(0, '8539.678'), (1, '8365.730')] [2023-12-27 01:12:56,065][105620] Updated weights for policy 1, policy_version 1351072 (0.0010) [2023-12-27 01:12:56,593][105692] Updated weights for policy 0, policy_version 1349105 (0.0008) [2023-12-27 01:12:56,661][105692] Updated weights for policy 0, policy_version 1349115 (0.0007) [2023-12-27 01:12:56,732][105692] Updated weights for policy 0, policy_version 1349125 (0.0005) [2023-12-27 01:12:56,794][105692] Updated weights for policy 0, policy_version 1349135 (0.0010) [2023-12-27 01:12:56,802][105620] Updated weights for policy 1, policy_version 1351082 (0.0011) [2023-12-27 01:12:56,850][105620] Updated weights for policy 1, policy_version 1351092 (0.0010) [2023-12-27 01:12:56,894][105620] Updated weights for policy 1, policy_version 1351102 (0.0010) [2023-12-27 01:12:57,386][105692] Updated weights for policy 0, policy_version 1349145 (0.0008) [2023-12-27 01:12:57,435][105692] Updated weights for policy 0, policy_version 1349155 (0.0005) [2023-12-27 01:12:57,492][105692] Updated weights for policy 0, policy_version 1349165 (0.0006) [2023-12-27 01:12:57,676][105620] Updated weights for policy 1, policy_version 1351112 (0.0010) [2023-12-27 01:12:57,729][105620] Updated weights for policy 1, policy_version 1351122 (0.0010) [2023-12-27 01:12:57,791][105620] Updated weights for policy 1, policy_version 1351132 (0.0010) [2023-12-27 01:12:58,161][105692] Updated weights for policy 0, policy_version 1349175 (0.0007) [2023-12-27 01:12:58,223][105692] Updated weights for policy 0, policy_version 1349185 (0.0008) [2023-12-27 01:12:58,278][105692] Updated weights for policy 0, policy_version 1349195 (0.0008) [2023-12-27 01:12:58,565][105620] Updated weights for policy 1, policy_version 1351142 (0.0010) [2023-12-27 01:12:58,638][105620] Updated weights for policy 1, policy_version 1351152 (0.0011) [2023-12-27 01:12:58,702][105620] Updated weights for policy 1, policy_version 1351162 (0.0008) [2023-12-27 01:12:59,036][105692] Updated weights for policy 0, policy_version 1349205 (0.0009) [2023-12-27 01:12:59,084][105692] Updated weights for policy 0, policy_version 1349215 (0.0010) [2023-12-27 01:12:59,139][105692] Updated weights for policy 0, policy_version 1349225 (0.0010) [2023-12-27 01:12:59,415][105620] Updated weights for policy 1, policy_version 1351172 (0.0010) [2023-12-27 01:12:59,478][105620] Updated weights for policy 1, policy_version 1351182 (0.0007) [2023-12-27 01:12:59,534][105620] Updated weights for policy 1, policy_version 1351192 (0.0007) [2023-12-27 01:13:00,008][105692] Updated weights for policy 0, policy_version 1349235 (0.0010) [2023-12-27 01:13:00,069][105692] Updated weights for policy 0, policy_version 1349245 (0.0009) [2023-12-27 01:13:00,129][105692] Updated weights for policy 0, policy_version 1349255 (0.0006) [2023-12-27 01:13:00,181][105620] Updated weights for policy 1, policy_version 1351202 (0.0006) [2023-12-27 01:13:00,242][105620] Updated weights for policy 1, policy_version 1351212 (0.0009) [2023-12-27 01:13:00,312][105620] Updated weights for policy 1, policy_version 1351222 (0.0009) [2023-12-27 01:13:00,371][105620] Updated weights for policy 1, policy_version 1351232 (0.0009) [2023-12-27 01:13:00,822][105692] Updated weights for policy 0, policy_version 1349265 (0.0009) [2023-12-27 01:13:00,872][105692] Updated weights for policy 0, policy_version 1349275 (0.0008) [2023-12-27 01:13:00,924][105692] Updated weights for policy 0, policy_version 1349285 (0.0008) [2023-12-27 01:13:00,962][105620] Updated weights for policy 1, policy_version 1351242 (0.0011) [2023-12-27 01:13:00,973][105692] Updated weights for policy 0, policy_version 1349295 (0.0005) [2023-12-27 01:13:01,015][105620] Updated weights for policy 1, policy_version 1351252 (0.0010) [2023-12-27 01:13:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 691429376. Throughput: 0: 9698.3, 1: 9554.2. Samples: 691400088. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:13:01,062][104569] Avg episode reward: [(0, '8356.515'), (1, '8300.474')] [2023-12-27 01:13:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001349296_345473024.pth... [2023-12-27 01:13:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001348144_345178112.pth [2023-12-27 01:13:01,078][105620] Updated weights for policy 1, policy_version 1351262 (0.0011) [2023-12-27 01:13:01,088][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001351264_345964544.pth... [2023-12-27 01:13:01,092][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001350144_345677824.pth [2023-12-27 01:13:01,710][105692] Updated weights for policy 0, policy_version 1349305 (0.0008) [2023-12-27 01:13:01,767][105692] Updated weights for policy 0, policy_version 1349315 (0.0005) [2023-12-27 01:13:01,775][105585] KL-divergence is very high: 103.6174 [2023-12-27 01:13:01,792][105585] KL-divergence is very high: 100.6414 [2023-12-27 01:13:01,827][105692] Updated weights for policy 0, policy_version 1349325 (0.0006) [2023-12-27 01:13:01,860][105620] Updated weights for policy 1, policy_version 1351272 (0.0011) [2023-12-27 01:13:01,917][105620] Updated weights for policy 1, policy_version 1351282 (0.0010) [2023-12-27 01:13:01,975][105620] Updated weights for policy 1, policy_version 1351292 (0.0010) [2023-12-27 01:13:02,476][105692] Updated weights for policy 0, policy_version 1349335 (0.0006) [2023-12-27 01:13:02,533][105692] Updated weights for policy 0, policy_version 1349345 (0.0006) [2023-12-27 01:13:02,591][105692] Updated weights for policy 0, policy_version 1349355 (0.0005) [2023-12-27 01:13:02,710][105620] Updated weights for policy 1, policy_version 1351302 (0.0011) [2023-12-27 01:13:02,769][105620] Updated weights for policy 1, policy_version 1351312 (0.0009) [2023-12-27 01:13:02,815][105620] Updated weights for policy 1, policy_version 1351322 (0.0005) [2023-12-27 01:13:03,256][105692] Updated weights for policy 0, policy_version 1349365 (0.0006) [2023-12-27 01:13:03,328][105692] Updated weights for policy 0, policy_version 1349375 (0.0010) [2023-12-27 01:13:03,397][105692] Updated weights for policy 0, policy_version 1349385 (0.0010) [2023-12-27 01:13:03,457][105620] Updated weights for policy 1, policy_version 1351332 (0.0009) [2023-12-27 01:13:03,506][105620] Updated weights for policy 1, policy_version 1351342 (0.0009) [2023-12-27 01:13:03,558][105620] Updated weights for policy 1, policy_version 1351352 (0.0005) [2023-12-27 01:13:04,056][105692] Updated weights for policy 0, policy_version 1349395 (0.0008) [2023-12-27 01:13:04,117][105692] Updated weights for policy 0, policy_version 1349405 (0.0006) [2023-12-27 01:13:04,186][105692] Updated weights for policy 0, policy_version 1349415 (0.0006) [2023-12-27 01:13:04,205][105620] Updated weights for policy 1, policy_version 1351362 (0.0006) [2023-12-27 01:13:04,269][105620] Updated weights for policy 1, policy_version 1351372 (0.0011) [2023-12-27 01:13:04,340][105620] Updated weights for policy 1, policy_version 1351382 (0.0010) [2023-12-27 01:13:04,402][105620] Updated weights for policy 1, policy_version 1351392 (0.0009) [2023-12-27 01:13:04,871][105692] Updated weights for policy 0, policy_version 1349425 (0.0006) [2023-12-27 01:13:04,933][105692] Updated weights for policy 0, policy_version 1349435 (0.0008) [2023-12-27 01:13:04,992][105692] Updated weights for policy 0, policy_version 1349445 (0.0009) [2023-12-27 01:13:05,050][105692] Updated weights for policy 0, policy_version 1349455 (0.0009) [2023-12-27 01:13:05,125][105620] Updated weights for policy 1, policy_version 1351402 (0.0009) [2023-12-27 01:13:05,198][105620] Updated weights for policy 1, policy_version 1351412 (0.0010) [2023-12-27 01:13:05,262][105620] Updated weights for policy 1, policy_version 1351422 (0.0009) [2023-12-27 01:13:05,662][105692] Updated weights for policy 0, policy_version 1349465 (0.0006) [2023-12-27 01:13:05,707][105692] Updated weights for policy 0, policy_version 1349475 (0.0008) [2023-12-27 01:13:05,753][105692] Updated weights for policy 0, policy_version 1349485 (0.0005) [2023-12-27 01:13:06,004][105620] Updated weights for policy 1, policy_version 1351432 (0.0010) [2023-12-27 01:13:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 691527680. Throughput: 0: 9732.2, 1: 9624.7. Samples: 691519640. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:13:06,063][104569] Avg episode reward: [(0, '8263.487'), (1, '8473.645')] [2023-12-27 01:13:06,065][105620] Updated weights for policy 1, policy_version 1351442 (0.0010) [2023-12-27 01:13:06,127][105620] Updated weights for policy 1, policy_version 1351452 (0.0010) [2023-12-27 01:13:06,451][105692] Updated weights for policy 0, policy_version 1349495 (0.0008) [2023-12-27 01:13:06,521][105692] Updated weights for policy 0, policy_version 1349505 (0.0008) [2023-12-27 01:13:06,574][105692] Updated weights for policy 0, policy_version 1349515 (0.0008) [2023-12-27 01:13:06,889][105620] Updated weights for policy 1, policy_version 1351462 (0.0011) [2023-12-27 01:13:06,944][105620] Updated weights for policy 1, policy_version 1351472 (0.0010) [2023-12-27 01:13:07,009][105620] Updated weights for policy 1, policy_version 1351482 (0.0010) [2023-12-27 01:13:07,353][105692] Updated weights for policy 0, policy_version 1349525 (0.0007) [2023-12-27 01:13:07,401][105692] Updated weights for policy 0, policy_version 1349535 (0.0006) [2023-12-27 01:13:07,450][105692] Updated weights for policy 0, policy_version 1349545 (0.0010) [2023-12-27 01:13:07,636][105620] Updated weights for policy 1, policy_version 1351492 (0.0008) [2023-12-27 01:13:07,683][105620] Updated weights for policy 1, policy_version 1351502 (0.0009) [2023-12-27 01:13:07,731][105620] Updated weights for policy 1, policy_version 1351512 (0.0009) [2023-12-27 01:13:08,165][105692] Updated weights for policy 0, policy_version 1349555 (0.0009) [2023-12-27 01:13:08,221][105692] Updated weights for policy 0, policy_version 1349565 (0.0011) [2023-12-27 01:13:08,283][105692] Updated weights for policy 0, policy_version 1349575 (0.0010) [2023-12-27 01:13:08,393][105620] Updated weights for policy 1, policy_version 1351522 (0.0009) [2023-12-27 01:13:08,452][105620] Updated weights for policy 1, policy_version 1351532 (0.0011) [2023-12-27 01:13:08,508][105620] Updated weights for policy 1, policy_version 1351542 (0.0010) [2023-12-27 01:13:08,568][105620] Updated weights for policy 1, policy_version 1351552 (0.0011) [2023-12-27 01:13:09,046][105692] Updated weights for policy 0, policy_version 1349585 (0.0011) [2023-12-27 01:13:09,098][105692] Updated weights for policy 0, policy_version 1349595 (0.0011) [2023-12-27 01:13:09,150][105692] Updated weights for policy 0, policy_version 1349605 (0.0011) [2023-12-27 01:13:09,205][105692] Updated weights for policy 0, policy_version 1349615 (0.0011) [2023-12-27 01:13:09,229][105620] Updated weights for policy 1, policy_version 1351562 (0.0010) [2023-12-27 01:13:09,288][105620] Updated weights for policy 1, policy_version 1351572 (0.0008) [2023-12-27 01:13:09,347][105620] Updated weights for policy 1, policy_version 1351582 (0.0006) [2023-12-27 01:13:10,006][105620] Updated weights for policy 1, policy_version 1351592 (0.0006) [2023-12-27 01:13:10,064][105620] Updated weights for policy 1, policy_version 1351602 (0.0010) [2023-12-27 01:13:10,100][105692] Updated weights for policy 0, policy_version 1349625 (0.0009) [2023-12-27 01:13:10,126][105620] Updated weights for policy 1, policy_version 1351612 (0.0011) [2023-12-27 01:13:10,161][105692] Updated weights for policy 0, policy_version 1349635 (0.0007) [2023-12-27 01:13:10,219][105692] Updated weights for policy 0, policy_version 1349645 (0.0010) [2023-12-27 01:13:10,822][105620] Updated weights for policy 1, policy_version 1351622 (0.0010) [2023-12-27 01:13:10,884][105620] Updated weights for policy 1, policy_version 1351632 (0.0011) [2023-12-27 01:13:10,919][105692] Updated weights for policy 0, policy_version 1349655 (0.0008) [2023-12-27 01:13:10,930][105620] Updated weights for policy 1, policy_version 1351642 (0.0010) [2023-12-27 01:13:10,981][105692] Updated weights for policy 0, policy_version 1349665 (0.0006) [2023-12-27 01:13:11,039][105692] Updated weights for policy 0, policy_version 1349675 (0.0007) [2023-12-27 01:13:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 691625984. Throughput: 0: 9784.1, 1: 9674.1. Samples: 691636248. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:13:11,062][104569] Avg episode reward: [(0, '8628.868'), (1, '8543.313')] [2023-12-27 01:13:11,719][105620] Updated weights for policy 1, policy_version 1351652 (0.0010) [2023-12-27 01:13:11,747][105692] Updated weights for policy 0, policy_version 1349685 (0.0007) [2023-12-27 01:13:11,786][105620] Updated weights for policy 1, policy_version 1351662 (0.0011) [2023-12-27 01:13:11,808][105692] Updated weights for policy 0, policy_version 1349695 (0.0006) [2023-12-27 01:13:11,846][105620] Updated weights for policy 1, policy_version 1351672 (0.0011) [2023-12-27 01:13:11,861][105692] Updated weights for policy 0, policy_version 1349705 (0.0006) [2023-12-27 01:13:12,556][105620] Updated weights for policy 1, policy_version 1351682 (0.0011) [2023-12-27 01:13:12,585][105692] Updated weights for policy 0, policy_version 1349715 (0.0005) [2023-12-27 01:13:12,616][105620] Updated weights for policy 1, policy_version 1351692 (0.0010) [2023-12-27 01:13:12,633][105692] Updated weights for policy 0, policy_version 1349725 (0.0008) [2023-12-27 01:13:12,679][105620] Updated weights for policy 1, policy_version 1351702 (0.0006) [2023-12-27 01:13:12,697][105692] Updated weights for policy 0, policy_version 1349735 (0.0007) [2023-12-27 01:13:12,747][105620] Updated weights for policy 1, policy_version 1351712 (0.0006) [2023-12-27 01:13:13,284][105620] Updated weights for policy 1, policy_version 1351722 (0.0005) [2023-12-27 01:13:13,341][105620] Updated weights for policy 1, policy_version 1351732 (0.0006) [2023-12-27 01:13:13,401][105692] Updated weights for policy 0, policy_version 1349745 (0.0006) [2023-12-27 01:13:13,401][105620] Updated weights for policy 1, policy_version 1351742 (0.0005) [2023-12-27 01:13:13,458][105692] Updated weights for policy 0, policy_version 1349755 (0.0006) [2023-12-27 01:13:13,515][105692] Updated weights for policy 0, policy_version 1349765 (0.0006) [2023-12-27 01:13:13,566][105692] Updated weights for policy 0, policy_version 1349775 (0.0008) [2023-12-27 01:13:14,054][105620] Updated weights for policy 1, policy_version 1351752 (0.0006) [2023-12-27 01:13:14,111][105620] Updated weights for policy 1, policy_version 1351762 (0.0007) [2023-12-27 01:13:14,167][105620] Updated weights for policy 1, policy_version 1351772 (0.0011) [2023-12-27 01:13:14,201][105692] Updated weights for policy 0, policy_version 1349785 (0.0007) [2023-12-27 01:13:14,252][105692] Updated weights for policy 0, policy_version 1349795 (0.0007) [2023-12-27 01:13:14,301][105692] Updated weights for policy 0, policy_version 1349805 (0.0008) [2023-12-27 01:13:14,922][105620] Updated weights for policy 1, policy_version 1351782 (0.0008) [2023-12-27 01:13:14,987][105620] Updated weights for policy 1, policy_version 1351792 (0.0007) [2023-12-27 01:13:15,014][105692] Updated weights for policy 0, policy_version 1349815 (0.0007) [2023-12-27 01:13:15,050][105620] Updated weights for policy 1, policy_version 1351802 (0.0009) [2023-12-27 01:13:15,072][105692] Updated weights for policy 0, policy_version 1349825 (0.0007) [2023-12-27 01:13:15,134][105692] Updated weights for policy 0, policy_version 1349835 (0.0008) [2023-12-27 01:13:15,736][105620] Updated weights for policy 1, policy_version 1351812 (0.0007) [2023-12-27 01:13:15,801][105620] Updated weights for policy 1, policy_version 1351822 (0.0006) [2023-12-27 01:13:15,845][105692] Updated weights for policy 0, policy_version 1349845 (0.0009) [2023-12-27 01:13:15,855][105620] Updated weights for policy 1, policy_version 1351832 (0.0008) [2023-12-27 01:13:15,905][105692] Updated weights for policy 0, policy_version 1349855 (0.0011) [2023-12-27 01:13:15,961][105692] Updated weights for policy 0, policy_version 1349865 (0.0010) [2023-12-27 01:13:16,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 691732480. Throughput: 0: 9793.1, 1: 9675.4. Samples: 691696216. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:13:16,062][104569] Avg episode reward: [(0, '9084.158'), (1, '8455.725')] [2023-12-27 01:13:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001349872_345620480.pth... [2023-12-27 01:13:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001351840_346112000.pth... [2023-12-27 01:13:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001350688_345817088.pth [2023-12-27 01:13:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001348720_345325568.pth [2023-12-27 01:13:16,488][105620] Updated weights for policy 1, policy_version 1351842 (0.0008) [2023-12-27 01:13:16,547][105620] Updated weights for policy 1, policy_version 1351852 (0.0011) [2023-12-27 01:13:16,567][105692] Updated weights for policy 0, policy_version 1349875 (0.0009) [2023-12-27 01:13:16,608][105620] Updated weights for policy 1, policy_version 1351862 (0.0006) [2023-12-27 01:13:16,629][105692] Updated weights for policy 0, policy_version 1349885 (0.0005) [2023-12-27 01:13:16,668][105620] Updated weights for policy 1, policy_version 1351872 (0.0006) [2023-12-27 01:13:16,692][105692] Updated weights for policy 0, policy_version 1349895 (0.0010) [2023-12-27 01:13:17,293][105620] Updated weights for policy 1, policy_version 1351882 (0.0011) [2023-12-27 01:13:17,315][105692] Updated weights for policy 0, policy_version 1349905 (0.0010) [2023-12-27 01:13:17,355][105620] Updated weights for policy 1, policy_version 1351892 (0.0010) [2023-12-27 01:13:17,369][105692] Updated weights for policy 0, policy_version 1349915 (0.0010) [2023-12-27 01:13:17,413][105620] Updated weights for policy 1, policy_version 1351902 (0.0010) [2023-12-27 01:13:17,415][105692] Updated weights for policy 0, policy_version 1349925 (0.0007) [2023-12-27 01:13:17,461][105692] Updated weights for policy 0, policy_version 1349935 (0.0005) [2023-12-27 01:13:18,021][105692] Updated weights for policy 0, policy_version 1349945 (0.0005) [2023-12-27 01:13:18,080][105692] Updated weights for policy 0, policy_version 1349955 (0.0005) [2023-12-27 01:13:18,143][105692] Updated weights for policy 0, policy_version 1349965 (0.0005) [2023-12-27 01:13:18,148][105620] Updated weights for policy 1, policy_version 1351912 (0.0010) [2023-12-27 01:13:18,218][105620] Updated weights for policy 1, policy_version 1351922 (0.0010) [2023-12-27 01:13:18,273][105620] Updated weights for policy 1, policy_version 1351932 (0.0010) [2023-12-27 01:13:18,723][105692] Updated weights for policy 0, policy_version 1349975 (0.0008) [2023-12-27 01:13:18,780][105692] Updated weights for policy 0, policy_version 1349985 (0.0008) [2023-12-27 01:13:18,826][105692] Updated weights for policy 0, policy_version 1349995 (0.0008) [2023-12-27 01:13:19,054][105620] Updated weights for policy 1, policy_version 1351942 (0.0010) [2023-12-27 01:13:19,116][105620] Updated weights for policy 1, policy_version 1351952 (0.0010) [2023-12-27 01:13:19,160][105620] Updated weights for policy 1, policy_version 1351962 (0.0010) [2023-12-27 01:13:19,639][105692] Updated weights for policy 0, policy_version 1350005 (0.0008) [2023-12-27 01:13:19,705][105692] Updated weights for policy 0, policy_version 1350015 (0.0008) [2023-12-27 01:13:19,772][105692] Updated weights for policy 0, policy_version 1350025 (0.0008) [2023-12-27 01:13:19,918][105620] Updated weights for policy 1, policy_version 1351972 (0.0010) [2023-12-27 01:13:19,979][105620] Updated weights for policy 1, policy_version 1351982 (0.0011) [2023-12-27 01:13:20,036][105620] Updated weights for policy 1, policy_version 1351992 (0.0010) [2023-12-27 01:13:20,492][105692] Updated weights for policy 0, policy_version 1350035 (0.0007) [2023-12-27 01:13:20,561][105692] Updated weights for policy 0, policy_version 1350045 (0.0006) [2023-12-27 01:13:20,624][105692] Updated weights for policy 0, policy_version 1350055 (0.0008) [2023-12-27 01:13:20,805][105620] Updated weights for policy 1, policy_version 1352002 (0.0010) [2023-12-27 01:13:20,864][105620] Updated weights for policy 1, policy_version 1352012 (0.0010) [2023-12-27 01:13:20,925][105620] Updated weights for policy 1, policy_version 1352022 (0.0011) [2023-12-27 01:13:20,988][105620] Updated weights for policy 1, policy_version 1352032 (0.0011) [2023-12-27 01:13:21,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 691830784. Throughput: 0: 9871.6, 1: 9688.2. Samples: 691818436. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:13:21,063][104569] Avg episode reward: [(0, '8351.581'), (1, '8095.029')] [2023-12-27 01:13:21,298][105692] Updated weights for policy 0, policy_version 1350065 (0.0009) [2023-12-27 01:13:21,370][105692] Updated weights for policy 0, policy_version 1350075 (0.0010) [2023-12-27 01:13:21,430][105692] Updated weights for policy 0, policy_version 1350085 (0.0008) [2023-12-27 01:13:21,481][105692] Updated weights for policy 0, policy_version 1350095 (0.0008) [2023-12-27 01:13:21,835][105620] Updated weights for policy 1, policy_version 1352042 (0.0009) [2023-12-27 01:13:21,897][105620] Updated weights for policy 1, policy_version 1352052 (0.0009) [2023-12-27 01:13:21,962][105620] Updated weights for policy 1, policy_version 1352062 (0.0009) [2023-12-27 01:13:22,249][105692] Updated weights for policy 0, policy_version 1350105 (0.0009) [2023-12-27 01:13:22,311][105692] Updated weights for policy 0, policy_version 1350115 (0.0010) [2023-12-27 01:13:22,375][105692] Updated weights for policy 0, policy_version 1350125 (0.0007) [2023-12-27 01:13:22,762][105620] Updated weights for policy 1, policy_version 1352072 (0.0009) [2023-12-27 01:13:22,825][105620] Updated weights for policy 1, policy_version 1352082 (0.0008) [2023-12-27 01:13:22,877][105620] Updated weights for policy 1, policy_version 1352092 (0.0005) [2023-12-27 01:13:23,061][105692] Updated weights for policy 0, policy_version 1350135 (0.0008) [2023-12-27 01:13:23,117][105692] Updated weights for policy 0, policy_version 1350145 (0.0010) [2023-12-27 01:13:23,175][105692] Updated weights for policy 0, policy_version 1350155 (0.0010) [2023-12-27 01:13:23,524][105620] Updated weights for policy 1, policy_version 1352102 (0.0005) [2023-12-27 01:13:23,594][105620] Updated weights for policy 1, policy_version 1352112 (0.0006) [2023-12-27 01:13:23,644][105620] Updated weights for policy 1, policy_version 1352122 (0.0005) [2023-12-27 01:13:23,958][105692] Updated weights for policy 0, policy_version 1350165 (0.0009) [2023-12-27 01:13:24,020][105692] Updated weights for policy 0, policy_version 1350175 (0.0009) [2023-12-27 01:13:24,078][105692] Updated weights for policy 0, policy_version 1350185 (0.0009) [2023-12-27 01:13:24,271][105620] Updated weights for policy 1, policy_version 1352132 (0.0007) [2023-12-27 01:13:24,332][105620] Updated weights for policy 1, policy_version 1352142 (0.0009) [2023-12-27 01:13:24,397][105620] Updated weights for policy 1, policy_version 1352152 (0.0009) [2023-12-27 01:13:24,774][105692] Updated weights for policy 0, policy_version 1350195 (0.0009) [2023-12-27 01:13:24,831][105692] Updated weights for policy 0, policy_version 1350205 (0.0008) [2023-12-27 01:13:24,875][105692] Updated weights for policy 0, policy_version 1350215 (0.0008) [2023-12-27 01:13:25,148][105620] Updated weights for policy 1, policy_version 1352162 (0.0009) [2023-12-27 01:13:25,199][105620] Updated weights for policy 1, policy_version 1352172 (0.0010) [2023-12-27 01:13:25,247][105620] Updated weights for policy 1, policy_version 1352182 (0.0010) [2023-12-27 01:13:25,292][105620] Updated weights for policy 1, policy_version 1352192 (0.0010) [2023-12-27 01:13:25,575][105692] Updated weights for policy 0, policy_version 1350225 (0.0008) [2023-12-27 01:13:25,631][105692] Updated weights for policy 0, policy_version 1350235 (0.0008) [2023-12-27 01:13:25,679][105692] Updated weights for policy 0, policy_version 1350246 (0.0008) [2023-12-27 01:13:25,733][105692] Updated weights for policy 0, policy_version 1350256 (0.0009) [2023-12-27 01:13:25,985][105620] Updated weights for policy 1, policy_version 1352202 (0.0010) [2023-12-27 01:13:26,039][105620] Updated weights for policy 1, policy_version 1352212 (0.0010) [2023-12-27 01:13:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 691920896. Throughput: 0: 9855.6, 1: 9697.8. Samples: 691933868. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:13:26,062][104569] Avg episode reward: [(0, '7900.193'), (1, '7919.290')] [2023-12-27 01:13:26,091][105620] Updated weights for policy 1, policy_version 1352222 (0.0010) [2023-12-27 01:13:26,480][105692] Updated weights for policy 0, policy_version 1350266 (0.0006) [2023-12-27 01:13:26,540][105692] Updated weights for policy 0, policy_version 1350276 (0.0006) [2023-12-27 01:13:26,606][105692] Updated weights for policy 0, policy_version 1350286 (0.0006) [2023-12-27 01:13:26,786][105620] Updated weights for policy 1, policy_version 1352232 (0.0010) [2023-12-27 01:13:26,837][105620] Updated weights for policy 1, policy_version 1352242 (0.0010) [2023-12-27 01:13:26,887][105620] Updated weights for policy 1, policy_version 1352252 (0.0010) [2023-12-27 01:13:27,182][105692] Updated weights for policy 0, policy_version 1350296 (0.0009) [2023-12-27 01:13:27,236][105692] Updated weights for policy 0, policy_version 1350306 (0.0010) [2023-12-27 01:13:27,300][105692] Updated weights for policy 0, policy_version 1350316 (0.0010) [2023-12-27 01:13:27,627][105620] Updated weights for policy 1, policy_version 1352262 (0.0010) [2023-12-27 01:13:27,675][105620] Updated weights for policy 1, policy_version 1352272 (0.0010) [2023-12-27 01:13:27,722][105620] Updated weights for policy 1, policy_version 1352282 (0.0010) [2023-12-27 01:13:27,894][105692] Updated weights for policy 0, policy_version 1350326 (0.0007) [2023-12-27 01:13:27,961][105692] Updated weights for policy 0, policy_version 1350336 (0.0005) [2023-12-27 01:13:28,009][105692] Updated weights for policy 0, policy_version 1350346 (0.0005) [2023-12-27 01:13:28,365][105620] Updated weights for policy 1, policy_version 1352292 (0.0008) [2023-12-27 01:13:28,425][105620] Updated weights for policy 1, policy_version 1352302 (0.0007) [2023-12-27 01:13:28,483][105620] Updated weights for policy 1, policy_version 1352312 (0.0010) [2023-12-27 01:13:28,676][105692] Updated weights for policy 0, policy_version 1350356 (0.0007) [2023-12-27 01:13:28,727][105692] Updated weights for policy 0, policy_version 1350366 (0.0010) [2023-12-27 01:13:28,788][105692] Updated weights for policy 0, policy_version 1350376 (0.0010) [2023-12-27 01:13:29,196][105620] Updated weights for policy 1, policy_version 1352322 (0.0010) [2023-12-27 01:13:29,256][105620] Updated weights for policy 1, policy_version 1352332 (0.0010) [2023-12-27 01:13:29,303][105620] Updated weights for policy 1, policy_version 1352342 (0.0010) [2023-12-27 01:13:29,362][105620] Updated weights for policy 1, policy_version 1352352 (0.0011) [2023-12-27 01:13:29,509][105692] Updated weights for policy 0, policy_version 1350386 (0.0010) [2023-12-27 01:13:29,570][105692] Updated weights for policy 0, policy_version 1350396 (0.0010) [2023-12-27 01:13:29,635][105692] Updated weights for policy 0, policy_version 1350406 (0.0010) [2023-12-27 01:13:29,696][105692] Updated weights for policy 0, policy_version 1350416 (0.0010) [2023-12-27 01:13:30,124][105620] Updated weights for policy 1, policy_version 1352362 (0.0009) [2023-12-27 01:13:30,188][105620] Updated weights for policy 1, policy_version 1352372 (0.0008) [2023-12-27 01:13:30,235][105620] Updated weights for policy 1, policy_version 1352382 (0.0009) [2023-12-27 01:13:30,380][105692] Updated weights for policy 0, policy_version 1350426 (0.0010) [2023-12-27 01:13:30,428][105692] Updated weights for policy 0, policy_version 1350436 (0.0010) [2023-12-27 01:13:30,486][105692] Updated weights for policy 0, policy_version 1350446 (0.0010) [2023-12-27 01:13:31,023][105620] Updated weights for policy 1, policy_version 1352392 (0.0006) [2023-12-27 01:13:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 692019200. Throughput: 0: 9921.9, 1: 9736.3. Samples: 691996260. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:13:31,062][104569] Avg episode reward: [(0, '8357.383'), (1, '8278.456')] [2023-12-27 01:13:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001350448_345767936.pth... [2023-12-27 01:13:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001349296_345473024.pth [2023-12-27 01:13:31,087][105620] Updated weights for policy 1, policy_version 1352402 (0.0008) [2023-12-27 01:13:31,150][105620] Updated weights for policy 1, policy_version 1352412 (0.0008) [2023-12-27 01:13:31,170][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001352416_346259456.pth... [2023-12-27 01:13:31,175][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001351264_345964544.pth [2023-12-27 01:13:31,191][105692] Updated weights for policy 0, policy_version 1350456 (0.0008) [2023-12-27 01:13:31,249][105692] Updated weights for policy 0, policy_version 1350466 (0.0006) [2023-12-27 01:13:31,305][105692] Updated weights for policy 0, policy_version 1350476 (0.0006) [2023-12-27 01:13:31,848][105620] Updated weights for policy 1, policy_version 1352422 (0.0008) [2023-12-27 01:13:31,901][105620] Updated weights for policy 1, policy_version 1352432 (0.0009) [2023-12-27 01:13:31,958][105620] Updated weights for policy 1, policy_version 1352442 (0.0008) [2023-12-27 01:13:32,026][105692] Updated weights for policy 0, policy_version 1350486 (0.0007) [2023-12-27 01:13:32,088][105692] Updated weights for policy 0, policy_version 1350496 (0.0007) [2023-12-27 01:13:32,156][105692] Updated weights for policy 0, policy_version 1350506 (0.0005) [2023-12-27 01:13:32,736][105620] Updated weights for policy 1, policy_version 1352452 (0.0009) [2023-12-27 01:13:32,798][105620] Updated weights for policy 1, policy_version 1352462 (0.0008) [2023-12-27 01:13:32,858][105620] Updated weights for policy 1, policy_version 1352472 (0.0006) [2023-12-27 01:13:32,877][105692] Updated weights for policy 0, policy_version 1350516 (0.0011) [2023-12-27 01:13:32,929][105692] Updated weights for policy 0, policy_version 1350526 (0.0010) [2023-12-27 01:13:32,984][105692] Updated weights for policy 0, policy_version 1350536 (0.0010) [2023-12-27 01:13:33,448][105620] Updated weights for policy 1, policy_version 1352482 (0.0005) [2023-12-27 01:13:33,498][105620] Updated weights for policy 1, policy_version 1352492 (0.0005) [2023-12-27 01:13:33,546][105620] Updated weights for policy 1, policy_version 1352502 (0.0005) [2023-12-27 01:13:33,592][105692] Updated weights for policy 0, policy_version 1350546 (0.0011) [2023-12-27 01:13:33,599][105620] Updated weights for policy 1, policy_version 1352512 (0.0005) [2023-12-27 01:13:33,663][105692] Updated weights for policy 0, policy_version 1350556 (0.0011) [2023-12-27 01:13:33,731][105692] Updated weights for policy 0, policy_version 1350566 (0.0010) [2023-12-27 01:13:33,802][105692] Updated weights for policy 0, policy_version 1350576 (0.0011) [2023-12-27 01:13:34,133][105620] Updated weights for policy 1, policy_version 1352522 (0.0005) [2023-12-27 01:13:34,197][105620] Updated weights for policy 1, policy_version 1352532 (0.0010) [2023-12-27 01:13:34,258][105620] Updated weights for policy 1, policy_version 1352542 (0.0008) [2023-12-27 01:13:34,534][105692] Updated weights for policy 0, policy_version 1350586 (0.0009) [2023-12-27 01:13:34,597][105692] Updated weights for policy 0, policy_version 1350596 (0.0009) [2023-12-27 01:13:34,653][105692] Updated weights for policy 0, policy_version 1350606 (0.0009) [2023-12-27 01:13:34,947][105620] Updated weights for policy 1, policy_version 1352552 (0.0007) [2023-12-27 01:13:35,008][105620] Updated weights for policy 1, policy_version 1352562 (0.0010) [2023-12-27 01:13:35,066][105620] Updated weights for policy 1, policy_version 1352572 (0.0010) [2023-12-27 01:13:35,430][105692] Updated weights for policy 0, policy_version 1350616 (0.0010) [2023-12-27 01:13:35,482][105692] Updated weights for policy 0, policy_version 1350626 (0.0011) [2023-12-27 01:13:35,537][105692] Updated weights for policy 0, policy_version 1350636 (0.0011) [2023-12-27 01:13:35,681][105620] Updated weights for policy 1, policy_version 1352582 (0.0007) [2023-12-27 01:13:35,753][105620] Updated weights for policy 1, policy_version 1352592 (0.0005) [2023-12-27 01:13:35,812][105620] Updated weights for policy 1, policy_version 1352602 (0.0005) [2023-12-27 01:13:36,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 692125696. Throughput: 0: 9884.1, 1: 9796.9. Samples: 692114992. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:13:36,063][104569] Avg episode reward: [(0, '8262.200'), (1, '8548.475')] [2023-12-27 01:13:36,266][105692] Updated weights for policy 0, policy_version 1350646 (0.0008) [2023-12-27 01:13:36,328][105692] Updated weights for policy 0, policy_version 1350656 (0.0005) [2023-12-27 01:13:36,387][105692] Updated weights for policy 0, policy_version 1350666 (0.0011) [2023-12-27 01:13:36,428][105620] Updated weights for policy 1, policy_version 1352612 (0.0007) [2023-12-27 01:13:36,489][105620] Updated weights for policy 1, policy_version 1352622 (0.0011) [2023-12-27 01:13:36,553][105620] Updated weights for policy 1, policy_version 1352632 (0.0011) [2023-12-27 01:13:36,993][105692] Updated weights for policy 0, policy_version 1350676 (0.0011) [2023-12-27 01:13:37,053][105692] Updated weights for policy 0, policy_version 1350686 (0.0011) [2023-12-27 01:13:37,114][105692] Updated weights for policy 0, policy_version 1350696 (0.0011) [2023-12-27 01:13:37,284][105620] Updated weights for policy 1, policy_version 1352642 (0.0011) [2023-12-27 01:13:37,341][105620] Updated weights for policy 1, policy_version 1352652 (0.0011) [2023-12-27 01:13:37,399][105620] Updated weights for policy 1, policy_version 1352662 (0.0009) [2023-12-27 01:13:37,455][105620] Updated weights for policy 1, policy_version 1352672 (0.0006) [2023-12-27 01:13:37,794][105692] Updated weights for policy 0, policy_version 1350706 (0.0010) [2023-12-27 01:13:37,850][105692] Updated weights for policy 0, policy_version 1350716 (0.0005) [2023-12-27 01:13:37,903][105692] Updated weights for policy 0, policy_version 1350726 (0.0005) [2023-12-27 01:13:37,962][105692] Updated weights for policy 0, policy_version 1350736 (0.0005) [2023-12-27 01:13:38,216][105620] Updated weights for policy 1, policy_version 1352682 (0.0008) [2023-12-27 01:13:38,265][105620] Updated weights for policy 1, policy_version 1352692 (0.0010) [2023-12-27 01:13:38,317][105620] Updated weights for policy 1, policy_version 1352702 (0.0010) [2023-12-27 01:13:38,498][105692] Updated weights for policy 0, policy_version 1350746 (0.0005) [2023-12-27 01:13:38,568][105692] Updated weights for policy 0, policy_version 1350756 (0.0009) [2023-12-27 01:13:38,647][105692] Updated weights for policy 0, policy_version 1350766 (0.0010) [2023-12-27 01:13:38,974][105620] Updated weights for policy 1, policy_version 1352712 (0.0010) [2023-12-27 01:13:39,026][105620] Updated weights for policy 1, policy_version 1352722 (0.0010) [2023-12-27 01:13:39,074][105620] Updated weights for policy 1, policy_version 1352732 (0.0010) [2023-12-27 01:13:39,280][105692] Updated weights for policy 0, policy_version 1350776 (0.0010) [2023-12-27 01:13:39,349][105692] Updated weights for policy 0, policy_version 1350786 (0.0011) [2023-12-27 01:13:39,423][105692] Updated weights for policy 0, policy_version 1350796 (0.0010) [2023-12-27 01:13:39,763][105620] Updated weights for policy 1, policy_version 1352742 (0.0010) [2023-12-27 01:13:39,834][105620] Updated weights for policy 1, policy_version 1352752 (0.0011) [2023-12-27 01:13:39,898][105620] Updated weights for policy 1, policy_version 1352762 (0.0011) [2023-12-27 01:13:40,172][105692] Updated weights for policy 0, policy_version 1350806 (0.0010) [2023-12-27 01:13:40,221][105692] Updated weights for policy 0, policy_version 1350816 (0.0010) [2023-12-27 01:13:40,277][105692] Updated weights for policy 0, policy_version 1350826 (0.0010) [2023-12-27 01:13:40,558][105620] Updated weights for policy 1, policy_version 1352772 (0.0008) [2023-12-27 01:13:40,609][105620] Updated weights for policy 1, policy_version 1352782 (0.0008) [2023-12-27 01:13:40,666][105620] Updated weights for policy 1, policy_version 1352792 (0.0011) [2023-12-27 01:13:40,938][105692] Updated weights for policy 0, policy_version 1350836 (0.0011) [2023-12-27 01:13:40,990][105692] Updated weights for policy 0, policy_version 1350846 (0.0011) [2023-12-27 01:13:41,051][105692] Updated weights for policy 0, policy_version 1350856 (0.0010) [2023-12-27 01:13:41,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 692224000. Throughput: 0: 9983.2, 1: 9915.6. Samples: 692236408. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:13:41,063][104569] Avg episode reward: [(0, '8167.118'), (1, '8366.108')] [2023-12-27 01:13:41,349][105620] Updated weights for policy 1, policy_version 1352802 (0.0008) [2023-12-27 01:13:41,419][105620] Updated weights for policy 1, policy_version 1352812 (0.0008) [2023-12-27 01:13:41,485][105620] Updated weights for policy 1, policy_version 1352822 (0.0006) [2023-12-27 01:13:41,550][105620] Updated weights for policy 1, policy_version 1352832 (0.0008) [2023-12-27 01:13:41,820][105692] Updated weights for policy 0, policy_version 1350866 (0.0009) [2023-12-27 01:13:41,882][105692] Updated weights for policy 0, policy_version 1350876 (0.0006) [2023-12-27 01:13:41,946][105692] Updated weights for policy 0, policy_version 1350886 (0.0007) [2023-12-27 01:13:42,022][105692] Updated weights for policy 0, policy_version 1350896 (0.0010) [2023-12-27 01:13:42,216][105620] Updated weights for policy 1, policy_version 1352842 (0.0008) [2023-12-27 01:13:42,284][105620] Updated weights for policy 1, policy_version 1352852 (0.0008) [2023-12-27 01:13:42,349][105620] Updated weights for policy 1, policy_version 1352862 (0.0008) [2023-12-27 01:13:42,753][105692] Updated weights for policy 0, policy_version 1350906 (0.0008) [2023-12-27 01:13:42,809][105692] Updated weights for policy 0, policy_version 1350916 (0.0009) [2023-12-27 01:13:42,862][105692] Updated weights for policy 0, policy_version 1350926 (0.0009) [2023-12-27 01:13:43,049][105620] Updated weights for policy 1, policy_version 1352872 (0.0006) [2023-12-27 01:13:43,094][105620] Updated weights for policy 1, policy_version 1352882 (0.0009) [2023-12-27 01:13:43,140][105620] Updated weights for policy 1, policy_version 1352892 (0.0010) [2023-12-27 01:13:43,717][105692] Updated weights for policy 0, policy_version 1350936 (0.0007) [2023-12-27 01:13:43,722][105620] Updated weights for policy 1, policy_version 1352902 (0.0010) [2023-12-27 01:13:43,771][105620] Updated weights for policy 1, policy_version 1352912 (0.0010) [2023-12-27 01:13:43,773][105692] Updated weights for policy 0, policy_version 1350946 (0.0006) [2023-12-27 01:13:43,825][105692] Updated weights for policy 0, policy_version 1350956 (0.0005) [2023-12-27 01:13:43,827][105620] Updated weights for policy 1, policy_version 1352922 (0.0010) [2023-12-27 01:13:44,436][105620] Updated weights for policy 1, policy_version 1352932 (0.0009) [2023-12-27 01:13:44,495][105620] Updated weights for policy 1, policy_version 1352942 (0.0008) [2023-12-27 01:13:44,561][105620] Updated weights for policy 1, policy_version 1352952 (0.0010) [2023-12-27 01:13:44,664][105692] Updated weights for policy 0, policy_version 1350966 (0.0006) [2023-12-27 01:13:44,726][105692] Updated weights for policy 0, policy_version 1350976 (0.0008) [2023-12-27 01:13:44,791][105692] Updated weights for policy 0, policy_version 1350986 (0.0009) [2023-12-27 01:13:45,190][105620] Updated weights for policy 1, policy_version 1352962 (0.0010) [2023-12-27 01:13:45,258][105620] Updated weights for policy 1, policy_version 1352972 (0.0009) [2023-12-27 01:13:45,318][105620] Updated weights for policy 1, policy_version 1352982 (0.0011) [2023-12-27 01:13:45,374][105620] Updated weights for policy 1, policy_version 1352992 (0.0011) [2023-12-27 01:13:45,605][105692] Updated weights for policy 0, policy_version 1350996 (0.0008) [2023-12-27 01:13:45,662][105692] Updated weights for policy 0, policy_version 1351006 (0.0005) [2023-12-27 01:13:45,712][105692] Updated weights for policy 0, policy_version 1351016 (0.0005) [2023-12-27 01:13:45,929][105620] Updated weights for policy 1, policy_version 1353002 (0.0006) [2023-12-27 01:13:45,984][105620] Updated weights for policy 1, policy_version 1353012 (0.0005) [2023-12-27 01:13:46,038][105620] Updated weights for policy 1, policy_version 1353022 (0.0005) [2023-12-27 01:13:46,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 692330496. Throughput: 0: 9905.0, 1: 9979.5. Samples: 692294896. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:13:46,063][104569] Avg episode reward: [(0, '8536.475'), (1, '8628.618')] [2023-12-27 01:13:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001351024_345915392.pth... [2023-12-27 01:13:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001353024_346415104.pth... [2023-12-27 01:13:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001351840_346112000.pth [2023-12-27 01:13:46,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001349872_345620480.pth [2023-12-27 01:13:46,283][105692] Updated weights for policy 0, policy_version 1351026 (0.0007) [2023-12-27 01:13:46,331][105692] Updated weights for policy 0, policy_version 1351036 (0.0005) [2023-12-27 01:13:46,377][105692] Updated weights for policy 0, policy_version 1351046 (0.0005) [2023-12-27 01:13:46,434][105692] Updated weights for policy 0, policy_version 1351056 (0.0009) [2023-12-27 01:13:46,711][105620] Updated weights for policy 1, policy_version 1353032 (0.0007) [2023-12-27 01:13:46,760][105620] Updated weights for policy 1, policy_version 1353042 (0.0008) [2023-12-27 01:13:46,803][105620] Updated weights for policy 1, policy_version 1353052 (0.0007) [2023-12-27 01:13:47,110][105692] Updated weights for policy 0, policy_version 1351066 (0.0010) [2023-12-27 01:13:47,164][105692] Updated weights for policy 0, policy_version 1351076 (0.0010) [2023-12-27 01:13:47,222][105692] Updated weights for policy 0, policy_version 1351086 (0.0010) [2023-12-27 01:13:47,572][105620] Updated weights for policy 1, policy_version 1353062 (0.0009) [2023-12-27 01:13:47,632][105620] Updated weights for policy 1, policy_version 1353072 (0.0011) [2023-12-27 01:13:47,695][105620] Updated weights for policy 1, policy_version 1353082 (0.0011) [2023-12-27 01:13:47,852][105692] Updated weights for policy 0, policy_version 1351096 (0.0010) [2023-12-27 01:13:47,897][105692] Updated weights for policy 0, policy_version 1351106 (0.0007) [2023-12-27 01:13:47,943][105692] Updated weights for policy 0, policy_version 1351116 (0.0005) [2023-12-27 01:13:48,414][105620] Updated weights for policy 1, policy_version 1353092 (0.0011) [2023-12-27 01:13:48,478][105620] Updated weights for policy 1, policy_version 1353102 (0.0011) [2023-12-27 01:13:48,530][105620] Updated weights for policy 1, policy_version 1353112 (0.0011) [2023-12-27 01:13:48,647][105692] Updated weights for policy 0, policy_version 1351126 (0.0010) [2023-12-27 01:13:48,710][105692] Updated weights for policy 0, policy_version 1351136 (0.0011) [2023-12-27 01:13:48,772][105692] Updated weights for policy 0, policy_version 1351146 (0.0007) [2023-12-27 01:13:49,277][105620] Updated weights for policy 1, policy_version 1353122 (0.0009) [2023-12-27 01:13:49,348][105620] Updated weights for policy 1, policy_version 1353132 (0.0011) [2023-12-27 01:13:49,368][105692] Updated weights for policy 0, policy_version 1351156 (0.0008) [2023-12-27 01:13:49,414][105620] Updated weights for policy 1, policy_version 1353142 (0.0010) [2023-12-27 01:13:49,433][105692] Updated weights for policy 0, policy_version 1351166 (0.0007) [2023-12-27 01:13:49,475][105620] Updated weights for policy 1, policy_version 1353152 (0.0009) [2023-12-27 01:13:49,486][105692] Updated weights for policy 0, policy_version 1351176 (0.0007) [2023-12-27 01:13:50,186][105692] Updated weights for policy 0, policy_version 1351186 (0.0008) [2023-12-27 01:13:50,241][105620] Updated weights for policy 1, policy_version 1353162 (0.0010) [2023-12-27 01:13:50,249][105692] Updated weights for policy 0, policy_version 1351196 (0.0006) [2023-12-27 01:13:50,305][105620] Updated weights for policy 1, policy_version 1353172 (0.0009) [2023-12-27 01:13:50,307][105692] Updated weights for policy 0, policy_version 1351206 (0.0007) [2023-12-27 01:13:50,367][105620] Updated weights for policy 1, policy_version 1353182 (0.0010) [2023-12-27 01:13:50,370][105692] Updated weights for policy 0, policy_version 1351216 (0.0006) [2023-12-27 01:13:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 692420608. Throughput: 0: 9925.7, 1: 9996.4. Samples: 692416136. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:13:51,062][104569] Avg episode reward: [(0, '8536.390'), (1, '8812.706')] [2023-12-27 01:13:51,078][105692] Updated weights for policy 0, policy_version 1351226 (0.0008) [2023-12-27 01:13:51,088][105620] Updated weights for policy 1, policy_version 1353192 (0.0009) [2023-12-27 01:13:51,143][105692] Updated weights for policy 0, policy_version 1351236 (0.0007) [2023-12-27 01:13:51,146][105620] Updated weights for policy 1, policy_version 1353202 (0.0007) [2023-12-27 01:13:51,202][105692] Updated weights for policy 0, policy_version 1351246 (0.0006) [2023-12-27 01:13:51,208][105620] Updated weights for policy 1, policy_version 1353212 (0.0007) [2023-12-27 01:13:51,886][105692] Updated weights for policy 0, policy_version 1351256 (0.0009) [2023-12-27 01:13:51,938][105692] Updated weights for policy 0, policy_version 1351266 (0.0009) [2023-12-27 01:13:51,988][105620] Updated weights for policy 1, policy_version 1353222 (0.0007) [2023-12-27 01:13:51,997][105692] Updated weights for policy 0, policy_version 1351276 (0.0009) [2023-12-27 01:13:52,039][105620] Updated weights for policy 1, policy_version 1353232 (0.0007) [2023-12-27 01:13:52,096][105620] Updated weights for policy 1, policy_version 1353242 (0.0009) [2023-12-27 01:13:52,739][105692] Updated weights for policy 0, policy_version 1351286 (0.0007) [2023-12-27 01:13:52,812][105692] Updated weights for policy 0, policy_version 1351296 (0.0007) [2023-12-27 01:13:52,821][105620] Updated weights for policy 1, policy_version 1353252 (0.0010) [2023-12-27 01:13:52,877][105692] Updated weights for policy 0, policy_version 1351306 (0.0009) [2023-12-27 01:13:52,883][105620] Updated weights for policy 1, policy_version 1353262 (0.0011) [2023-12-27 01:13:52,945][105620] Updated weights for policy 1, policy_version 1353272 (0.0010) [2023-12-27 01:13:53,522][105620] Updated weights for policy 1, policy_version 1353282 (0.0009) [2023-12-27 01:13:53,587][105620] Updated weights for policy 1, policy_version 1353292 (0.0009) [2023-12-27 01:13:53,627][105692] Updated weights for policy 0, policy_version 1351316 (0.0007) [2023-12-27 01:13:53,653][105620] Updated weights for policy 1, policy_version 1353302 (0.0010) [2023-12-27 01:13:53,687][105692] Updated weights for policy 0, policy_version 1351326 (0.0011) [2023-12-27 01:13:53,707][105620] Updated weights for policy 1, policy_version 1353312 (0.0010) [2023-12-27 01:13:53,745][105692] Updated weights for policy 0, policy_version 1351336 (0.0010) [2023-12-27 01:13:54,403][105620] Updated weights for policy 1, policy_version 1353322 (0.0005) [2023-12-27 01:13:54,470][105620] Updated weights for policy 1, policy_version 1353332 (0.0009) [2023-12-27 01:13:54,470][105692] Updated weights for policy 0, policy_version 1351346 (0.0010) [2023-12-27 01:13:54,527][105692] Updated weights for policy 0, policy_version 1351356 (0.0009) [2023-12-27 01:13:54,528][105620] Updated weights for policy 1, policy_version 1353342 (0.0010) [2023-12-27 01:13:54,587][105692] Updated weights for policy 0, policy_version 1351366 (0.0006) [2023-12-27 01:13:54,633][105692] Updated weights for policy 0, policy_version 1351376 (0.0008) [2023-12-27 01:13:55,233][105620] Updated weights for policy 1, policy_version 1353352 (0.0010) [2023-12-27 01:13:55,291][105620] Updated weights for policy 1, policy_version 1353362 (0.0010) [2023-12-27 01:13:55,338][105620] Updated weights for policy 1, policy_version 1353372 (0.0010) [2023-12-27 01:13:55,341][105692] Updated weights for policy 0, policy_version 1351386 (0.0006) [2023-12-27 01:13:55,389][105692] Updated weights for policy 0, policy_version 1351396 (0.0005) [2023-12-27 01:13:55,443][105692] Updated weights for policy 0, policy_version 1351406 (0.0007) [2023-12-27 01:13:56,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 692518912. Throughput: 0: 9959.3, 1: 9967.6. Samples: 692532964. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:13:56,062][104569] Avg episode reward: [(0, '8170.513'), (1, '8994.729')] [2023-12-27 01:13:56,068][105692] Updated weights for policy 0, policy_version 1351416 (0.0006) [2023-12-27 01:13:56,106][105620] Updated weights for policy 1, policy_version 1353382 (0.0010) [2023-12-27 01:13:56,130][105692] Updated weights for policy 0, policy_version 1351426 (0.0005) [2023-12-27 01:13:56,169][105620] Updated weights for policy 1, policy_version 1353392 (0.0011) [2023-12-27 01:13:56,192][105692] Updated weights for policy 0, policy_version 1351436 (0.0007) [2023-12-27 01:13:56,222][105620] Updated weights for policy 1, policy_version 1353402 (0.0011) [2023-12-27 01:13:56,859][105692] Updated weights for policy 0, policy_version 1351446 (0.0010) [2023-12-27 01:13:56,919][105692] Updated weights for policy 0, policy_version 1351456 (0.0010) [2023-12-27 01:13:56,962][105620] Updated weights for policy 1, policy_version 1353412 (0.0010) [2023-12-27 01:13:56,973][105692] Updated weights for policy 0, policy_version 1351466 (0.0010) [2023-12-27 01:13:57,010][105620] Updated weights for policy 1, policy_version 1353422 (0.0010) [2023-12-27 01:13:57,058][105620] Updated weights for policy 1, policy_version 1353432 (0.0010) [2023-12-27 01:13:57,606][105692] Updated weights for policy 0, policy_version 1351476 (0.0008) [2023-12-27 01:13:57,666][105692] Updated weights for policy 0, policy_version 1351486 (0.0005) [2023-12-27 01:13:57,718][105692] Updated weights for policy 0, policy_version 1351496 (0.0006) [2023-12-27 01:13:57,820][105620] Updated weights for policy 1, policy_version 1353442 (0.0010) [2023-12-27 01:13:57,868][105620] Updated weights for policy 1, policy_version 1353452 (0.0010) [2023-12-27 01:13:57,919][105620] Updated weights for policy 1, policy_version 1353462 (0.0010) [2023-12-27 01:13:57,967][105620] Updated weights for policy 1, policy_version 1353472 (0.0010) [2023-12-27 01:13:58,305][105692] Updated weights for policy 0, policy_version 1351506 (0.0009) [2023-12-27 01:13:58,390][105692] Updated weights for policy 0, policy_version 1351516 (0.0008) [2023-12-27 01:13:58,446][105692] Updated weights for policy 0, policy_version 1351526 (0.0008) [2023-12-27 01:13:58,502][105692] Updated weights for policy 0, policy_version 1351536 (0.0009) [2023-12-27 01:13:58,772][105620] Updated weights for policy 1, policy_version 1353482 (0.0008) [2023-12-27 01:13:58,835][105620] Updated weights for policy 1, policy_version 1353492 (0.0009) [2023-12-27 01:13:58,899][105620] Updated weights for policy 1, policy_version 1353502 (0.0009) [2023-12-27 01:13:59,360][105692] Updated weights for policy 0, policy_version 1351546 (0.0008) [2023-12-27 01:13:59,421][105692] Updated weights for policy 0, policy_version 1351556 (0.0009) [2023-12-27 01:13:59,476][105692] Updated weights for policy 0, policy_version 1351566 (0.0009) [2023-12-27 01:13:59,697][105620] Updated weights for policy 1, policy_version 1353512 (0.0008) [2023-12-27 01:13:59,751][105620] Updated weights for policy 1, policy_version 1353522 (0.0009) [2023-12-27 01:13:59,804][105620] Updated weights for policy 1, policy_version 1353532 (0.0008) [2023-12-27 01:14:00,251][105692] Updated weights for policy 0, policy_version 1351576 (0.0009) [2023-12-27 01:14:00,312][105692] Updated weights for policy 0, policy_version 1351586 (0.0010) [2023-12-27 01:14:00,360][105692] Updated weights for policy 0, policy_version 1351596 (0.0009) [2023-12-27 01:14:00,457][105620] Updated weights for policy 1, policy_version 1353542 (0.0008) [2023-12-27 01:14:00,507][105620] Updated weights for policy 1, policy_version 1353552 (0.0008) [2023-12-27 01:14:00,558][105620] Updated weights for policy 1, policy_version 1353562 (0.0009) [2023-12-27 01:14:00,993][105692] Updated weights for policy 0, policy_version 1351607 (0.0007) [2023-12-27 01:14:01,055][105692] Updated weights for policy 0, policy_version 1351617 (0.0007) [2023-12-27 01:14:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 692617216. Throughput: 0: 10016.4, 1: 9901.1. Samples: 692592504. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:14:01,062][104569] Avg episode reward: [(0, '8356.431'), (1, '8719.685')] [2023-12-27 01:14:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001353568_346554368.pth... [2023-12-27 01:14:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001352416_346259456.pth [2023-12-27 01:14:01,121][105692] Updated weights for policy 0, policy_version 1351627 (0.0008) [2023-12-27 01:14:01,152][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001351632_346071040.pth... [2023-12-27 01:14:01,156][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001350448_345767936.pth [2023-12-27 01:14:01,306][105620] Updated weights for policy 1, policy_version 1353572 (0.0008) [2023-12-27 01:14:01,370][105620] Updated weights for policy 1, policy_version 1353582 (0.0009) [2023-12-27 01:14:01,437][105620] Updated weights for policy 1, policy_version 1353592 (0.0006) [2023-12-27 01:14:01,759][105692] Updated weights for policy 0, policy_version 1351637 (0.0008) [2023-12-27 01:14:01,829][105692] Updated weights for policy 0, policy_version 1351647 (0.0005) [2023-12-27 01:14:01,900][105692] Updated weights for policy 0, policy_version 1351657 (0.0005) [2023-12-27 01:14:02,105][105620] Updated weights for policy 1, policy_version 1353602 (0.0010) [2023-12-27 01:14:02,165][105620] Updated weights for policy 1, policy_version 1353612 (0.0009) [2023-12-27 01:14:02,218][105620] Updated weights for policy 1, policy_version 1353622 (0.0010) [2023-12-27 01:14:02,271][105620] Updated weights for policy 1, policy_version 1353632 (0.0009) [2023-12-27 01:14:02,442][105692] Updated weights for policy 0, policy_version 1351667 (0.0008) [2023-12-27 01:14:02,499][105692] Updated weights for policy 0, policy_version 1351677 (0.0008) [2023-12-27 01:14:02,557][105692] Updated weights for policy 0, policy_version 1351687 (0.0009) [2023-12-27 01:14:03,055][105620] Updated weights for policy 1, policy_version 1353642 (0.0008) [2023-12-27 01:14:03,117][105620] Updated weights for policy 1, policy_version 1353652 (0.0006) [2023-12-27 01:14:03,184][105620] Updated weights for policy 1, policy_version 1353662 (0.0005) [2023-12-27 01:14:03,190][105692] Updated weights for policy 0, policy_version 1351697 (0.0009) [2023-12-27 01:14:03,240][105692] Updated weights for policy 0, policy_version 1351707 (0.0005) [2023-12-27 01:14:03,301][105692] Updated weights for policy 0, policy_version 1351717 (0.0005) [2023-12-27 01:14:03,356][105692] Updated weights for policy 0, policy_version 1351727 (0.0009) [2023-12-27 01:14:03,730][105620] Updated weights for policy 1, policy_version 1353672 (0.0006) [2023-12-27 01:14:03,786][105620] Updated weights for policy 1, policy_version 1353682 (0.0005) [2023-12-27 01:14:03,863][105620] Updated weights for policy 1, policy_version 1353692 (0.0006) [2023-12-27 01:14:04,136][105692] Updated weights for policy 0, policy_version 1351737 (0.0010) [2023-12-27 01:14:04,202][105692] Updated weights for policy 0, policy_version 1351747 (0.0008) [2023-12-27 01:14:04,261][105692] Updated weights for policy 0, policy_version 1351757 (0.0009) [2023-12-27 01:14:04,529][105620] Updated weights for policy 1, policy_version 1353702 (0.0009) [2023-12-27 01:14:04,584][105620] Updated weights for policy 1, policy_version 1353712 (0.0007) [2023-12-27 01:14:04,642][105620] Updated weights for policy 1, policy_version 1353722 (0.0005) [2023-12-27 01:14:05,066][105692] Updated weights for policy 0, policy_version 1351767 (0.0007) [2023-12-27 01:14:05,131][105692] Updated weights for policy 0, policy_version 1351777 (0.0005) [2023-12-27 01:14:05,197][105692] Updated weights for policy 0, policy_version 1351787 (0.0005) [2023-12-27 01:14:05,336][105620] Updated weights for policy 1, policy_version 1353732 (0.0010) [2023-12-27 01:14:05,391][105620] Updated weights for policy 1, policy_version 1353743 (0.0010) [2023-12-27 01:14:05,447][105620] Updated weights for policy 1, policy_version 1353754 (0.0010) [2023-12-27 01:14:05,707][105692] Updated weights for policy 0, policy_version 1351797 (0.0008) [2023-12-27 01:14:05,755][105692] Updated weights for policy 0, policy_version 1351807 (0.0008) [2023-12-27 01:14:05,803][105692] Updated weights for policy 0, policy_version 1351817 (0.0008) [2023-12-27 01:14:06,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19933.8, 300 sec: 19522.0). Total num frames: 692723712. Throughput: 0: 9930.0, 1: 9946.4. Samples: 692712876. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:14:06,063][104569] Avg episode reward: [(0, '8541.456'), (1, '8449.711')] [2023-12-27 01:14:06,179][105620] Updated weights for policy 1, policy_version 1353764 (0.0009) [2023-12-27 01:14:06,233][105620] Updated weights for policy 1, policy_version 1353774 (0.0008) [2023-12-27 01:14:06,298][105620] Updated weights for policy 1, policy_version 1353784 (0.0008) [2023-12-27 01:14:06,601][105692] Updated weights for policy 0, policy_version 1351827 (0.0009) [2023-12-27 01:14:06,668][105692] Updated weights for policy 0, policy_version 1351837 (0.0010) [2023-12-27 01:14:06,722][105692] Updated weights for policy 0, policy_version 1351847 (0.0010) [2023-12-27 01:14:06,962][105620] Updated weights for policy 1, policy_version 1353794 (0.0006) [2023-12-27 01:14:07,030][105620] Updated weights for policy 1, policy_version 1353804 (0.0008) [2023-12-27 01:14:07,088][105620] Updated weights for policy 1, policy_version 1353814 (0.0009) [2023-12-27 01:14:07,140][105620] Updated weights for policy 1, policy_version 1353824 (0.0009) [2023-12-27 01:14:07,377][105692] Updated weights for policy 0, policy_version 1351857 (0.0009) [2023-12-27 01:14:07,449][105692] Updated weights for policy 0, policy_version 1351867 (0.0005) [2023-12-27 01:14:07,507][105692] Updated weights for policy 0, policy_version 1351877 (0.0009) [2023-12-27 01:14:07,562][105692] Updated weights for policy 0, policy_version 1351887 (0.0010) [2023-12-27 01:14:07,868][105620] Updated weights for policy 1, policy_version 1353834 (0.0009) [2023-12-27 01:14:07,924][105620] Updated weights for policy 1, policy_version 1353844 (0.0009) [2023-12-27 01:14:07,989][105620] Updated weights for policy 1, policy_version 1353854 (0.0009) [2023-12-27 01:14:08,209][105692] Updated weights for policy 0, policy_version 1351897 (0.0006) [2023-12-27 01:14:08,265][105692] Updated weights for policy 0, policy_version 1351907 (0.0007) [2023-12-27 01:14:08,320][105692] Updated weights for policy 0, policy_version 1351917 (0.0009) [2023-12-27 01:14:08,665][105620] Updated weights for policy 1, policy_version 1353864 (0.0008) [2023-12-27 01:14:08,717][105620] Updated weights for policy 1, policy_version 1353874 (0.0009) [2023-12-27 01:14:08,776][105620] Updated weights for policy 1, policy_version 1353884 (0.0009) [2023-12-27 01:14:09,077][105692] Updated weights for policy 0, policy_version 1351927 (0.0009) [2023-12-27 01:14:09,134][105692] Updated weights for policy 0, policy_version 1351937 (0.0009) [2023-12-27 01:14:09,192][105692] Updated weights for policy 0, policy_version 1351947 (0.0009) [2023-12-27 01:14:09,514][105620] Updated weights for policy 1, policy_version 1353894 (0.0009) [2023-12-27 01:14:09,573][105620] Updated weights for policy 1, policy_version 1353904 (0.0008) [2023-12-27 01:14:09,639][105620] Updated weights for policy 1, policy_version 1353914 (0.0005) [2023-12-27 01:14:09,990][105692] Updated weights for policy 0, policy_version 1351957 (0.0008) [2023-12-27 01:14:10,051][105692] Updated weights for policy 0, policy_version 1351967 (0.0008) [2023-12-27 01:14:10,105][105692] Updated weights for policy 0, policy_version 1351977 (0.0009) [2023-12-27 01:14:10,246][105620] Updated weights for policy 1, policy_version 1353924 (0.0005) [2023-12-27 01:14:10,307][105620] Updated weights for policy 1, policy_version 1353934 (0.0006) [2023-12-27 01:14:10,372][105620] Updated weights for policy 1, policy_version 1353944 (0.0009) [2023-12-27 01:14:10,843][105692] Updated weights for policy 0, policy_version 1351987 (0.0009) [2023-12-27 01:14:10,891][105692] Updated weights for policy 0, policy_version 1351997 (0.0009) [2023-12-27 01:14:10,939][105692] Updated weights for policy 0, policy_version 1352007 (0.0009) [2023-12-27 01:14:11,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19933.8, 300 sec: 19549.7). Total num frames: 692822016. Throughput: 0: 9948.0, 1: 9976.8. Samples: 692830488. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:14:11,063][104569] Avg episode reward: [(0, '8260.668'), (1, '8364.112')] [2023-12-27 01:14:11,085][105620] Updated weights for policy 1, policy_version 1353954 (0.0009) [2023-12-27 01:14:11,155][105620] Updated weights for policy 1, policy_version 1353964 (0.0009) [2023-12-27 01:14:11,217][105620] Updated weights for policy 1, policy_version 1353974 (0.0009) [2023-12-27 01:14:11,282][105620] Updated weights for policy 1, policy_version 1353984 (0.0009) [2023-12-27 01:14:11,765][105692] Updated weights for policy 0, policy_version 1352018 (0.0009) [2023-12-27 01:14:11,823][105692] Updated weights for policy 0, policy_version 1352028 (0.0007) [2023-12-27 01:14:11,875][105692] Updated weights for policy 0, policy_version 1352038 (0.0006) [2023-12-27 01:14:11,924][105692] Updated weights for policy 0, policy_version 1352048 (0.0005) [2023-12-27 01:14:12,107][105620] Updated weights for policy 1, policy_version 1353994 (0.0009) [2023-12-27 01:14:12,172][105620] Updated weights for policy 1, policy_version 1354004 (0.0009) [2023-12-27 01:14:12,232][105620] Updated weights for policy 1, policy_version 1354014 (0.0010) [2023-12-27 01:14:12,604][105692] Updated weights for policy 0, policy_version 1352058 (0.0008) [2023-12-27 01:14:12,658][105692] Updated weights for policy 0, policy_version 1352068 (0.0007) [2023-12-27 01:14:12,716][105692] Updated weights for policy 0, policy_version 1352078 (0.0008) [2023-12-27 01:14:12,980][105620] Updated weights for policy 1, policy_version 1354024 (0.0010) [2023-12-27 01:14:13,040][105620] Updated weights for policy 1, policy_version 1354035 (0.0008) [2023-12-27 01:14:13,093][105586] KL-divergence is very high: 156.9570 [2023-12-27 01:14:13,106][105620] Updated weights for policy 1, policy_version 1354045 (0.0006) [2023-12-27 01:14:13,107][105586] KL-divergence is very high: 175.2054 [2023-12-27 01:14:13,444][105692] Updated weights for policy 0, policy_version 1352088 (0.0008) [2023-12-27 01:14:13,490][105692] Updated weights for policy 0, policy_version 1352098 (0.0010) [2023-12-27 01:14:13,542][105692] Updated weights for policy 0, policy_version 1352108 (0.0010) [2023-12-27 01:14:13,658][105620] Updated weights for policy 1, policy_version 1354055 (0.0007) [2023-12-27 01:14:13,702][105620] Updated weights for policy 1, policy_version 1354065 (0.0007) [2023-12-27 01:14:13,751][105620] Updated weights for policy 1, policy_version 1354075 (0.0008) [2023-12-27 01:14:14,183][105692] Updated weights for policy 0, policy_version 1352118 (0.0010) [2023-12-27 01:14:14,238][105692] Updated weights for policy 0, policy_version 1352128 (0.0011) [2023-12-27 01:14:14,290][105692] Updated weights for policy 0, policy_version 1352138 (0.0010) [2023-12-27 01:14:14,475][105620] Updated weights for policy 1, policy_version 1354085 (0.0005) [2023-12-27 01:14:14,538][105620] Updated weights for policy 1, policy_version 1354095 (0.0008) [2023-12-27 01:14:14,591][105620] Updated weights for policy 1, policy_version 1354105 (0.0010) [2023-12-27 01:14:14,921][105692] Updated weights for policy 0, policy_version 1352148 (0.0008) [2023-12-27 01:14:14,988][105692] Updated weights for policy 0, policy_version 1352158 (0.0008) [2023-12-27 01:14:15,054][105692] Updated weights for policy 0, policy_version 1352168 (0.0008) [2023-12-27 01:14:15,319][105620] Updated weights for policy 1, policy_version 1354115 (0.0010) [2023-12-27 01:14:15,383][105620] Updated weights for policy 1, policy_version 1354125 (0.0011) [2023-12-27 01:14:15,446][105620] Updated weights for policy 1, policy_version 1354135 (0.0011) [2023-12-27 01:14:15,676][105692] Updated weights for policy 0, policy_version 1352178 (0.0006) [2023-12-27 01:14:15,731][105692] Updated weights for policy 0, policy_version 1352188 (0.0005) [2023-12-27 01:14:15,787][105692] Updated weights for policy 0, policy_version 1352198 (0.0005) [2023-12-27 01:14:15,847][105692] Updated weights for policy 0, policy_version 1352208 (0.0006) [2023-12-27 01:14:16,042][105620] Updated weights for policy 1, policy_version 1354145 (0.0011) [2023-12-27 01:14:16,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 692920320. Throughput: 0: 9877.5, 1: 9939.1. Samples: 692888008. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:14:16,062][104569] Avg episode reward: [(0, '8439.777'), (1, '8546.334')] [2023-12-27 01:14:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001352208_346218496.pth... [2023-12-27 01:14:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001351024_345915392.pth [2023-12-27 01:14:16,106][105620] Updated weights for policy 1, policy_version 1354155 (0.0009) [2023-12-27 01:14:16,168][105620] Updated weights for policy 1, policy_version 1354165 (0.0011) [2023-12-27 01:14:16,220][105620] Updated weights for policy 1, policy_version 1354175 (0.0010) [2023-12-27 01:14:16,223][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001354176_346710016.pth... [2023-12-27 01:14:16,226][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001353024_346415104.pth [2023-12-27 01:14:16,392][105692] Updated weights for policy 0, policy_version 1352218 (0.0005) [2023-12-27 01:14:16,456][105692] Updated weights for policy 0, policy_version 1352228 (0.0006) [2023-12-27 01:14:16,519][105692] Updated weights for policy 0, policy_version 1352238 (0.0005) [2023-12-27 01:14:16,872][105620] Updated weights for policy 1, policy_version 1354185 (0.0010) [2023-12-27 01:14:16,934][105620] Updated weights for policy 1, policy_version 1354195 (0.0010) [2023-12-27 01:14:16,992][105620] Updated weights for policy 1, policy_version 1354205 (0.0010) [2023-12-27 01:14:17,075][105692] Updated weights for policy 0, policy_version 1352248 (0.0007) [2023-12-27 01:14:17,127][105692] Updated weights for policy 0, policy_version 1352258 (0.0010) [2023-12-27 01:14:17,140][105585] KL-divergence is very high: 163.7894 [2023-12-27 01:14:17,179][105692] Updated weights for policy 0, policy_version 1352268 (0.0010) [2023-12-27 01:14:17,182][105585] KL-divergence is very high: 165.2540 [2023-12-27 01:14:17,654][105620] Updated weights for policy 1, policy_version 1354215 (0.0010) [2023-12-27 01:14:17,705][105620] Updated weights for policy 1, policy_version 1354225 (0.0010) [2023-12-27 01:14:17,756][105620] Updated weights for policy 1, policy_version 1354235 (0.0009) [2023-12-27 01:14:17,903][105692] Updated weights for policy 0, policy_version 1352278 (0.0010) [2023-12-27 01:14:17,958][105692] Updated weights for policy 0, policy_version 1352288 (0.0008) [2023-12-27 01:14:18,021][105692] Updated weights for policy 0, policy_version 1352298 (0.0005) [2023-12-27 01:14:18,420][105620] Updated weights for policy 1, policy_version 1354245 (0.0008) [2023-12-27 01:14:18,483][105620] Updated weights for policy 1, policy_version 1354255 (0.0008) [2023-12-27 01:14:18,544][105620] Updated weights for policy 1, policy_version 1354265 (0.0006) [2023-12-27 01:14:18,668][105692] Updated weights for policy 0, policy_version 1352308 (0.0007) [2023-12-27 01:14:18,725][105692] Updated weights for policy 0, policy_version 1352318 (0.0005) [2023-12-27 01:14:18,786][105692] Updated weights for policy 0, policy_version 1352328 (0.0006) [2023-12-27 01:14:19,207][105620] Updated weights for policy 1, policy_version 1354275 (0.0009) [2023-12-27 01:14:19,268][105620] Updated weights for policy 1, policy_version 1354285 (0.0008) [2023-12-27 01:14:19,328][105620] Updated weights for policy 1, policy_version 1354295 (0.0008) [2023-12-27 01:14:19,401][105692] Updated weights for policy 0, policy_version 1352338 (0.0006) [2023-12-27 01:14:19,464][105692] Updated weights for policy 0, policy_version 1352348 (0.0006) [2023-12-27 01:14:19,528][105692] Updated weights for policy 0, policy_version 1352358 (0.0009) [2023-12-27 01:14:19,588][105692] Updated weights for policy 0, policy_version 1352368 (0.0010) [2023-12-27 01:14:20,068][105620] Updated weights for policy 1, policy_version 1354305 (0.0009) [2023-12-27 01:14:20,132][105620] Updated weights for policy 1, policy_version 1354315 (0.0009) [2023-12-27 01:14:20,196][105620] Updated weights for policy 1, policy_version 1354325 (0.0008) [2023-12-27 01:14:20,255][105620] Updated weights for policy 1, policy_version 1354335 (0.0009) [2023-12-27 01:14:20,306][105692] Updated weights for policy 0, policy_version 1352378 (0.0008) [2023-12-27 01:14:20,371][105692] Updated weights for policy 0, policy_version 1352388 (0.0010) [2023-12-27 01:14:20,430][105692] Updated weights for policy 0, policy_version 1352398 (0.0010) [2023-12-27 01:14:20,988][105620] Updated weights for policy 1, policy_version 1354345 (0.0008) [2023-12-27 01:14:21,056][105620] Updated weights for policy 1, policy_version 1354355 (0.0008) [2023-12-27 01:14:21,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 693018624. Throughput: 0: 10026.4, 1: 9962.6. Samples: 693014496. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:14:21,062][104569] Avg episode reward: [(0, '8258.460'), (1, '8631.412')] [2023-12-27 01:14:21,123][105620] Updated weights for policy 1, policy_version 1354365 (0.0007) [2023-12-27 01:14:21,136][105692] Updated weights for policy 0, policy_version 1352408 (0.0011) [2023-12-27 01:14:21,206][105692] Updated weights for policy 0, policy_version 1352418 (0.0011) [2023-12-27 01:14:21,275][105692] Updated weights for policy 0, policy_version 1352428 (0.0009) [2023-12-27 01:14:21,858][105620] Updated weights for policy 1, policy_version 1354375 (0.0009) [2023-12-27 01:14:21,917][105620] Updated weights for policy 1, policy_version 1354385 (0.0007) [2023-12-27 01:14:21,979][105620] Updated weights for policy 1, policy_version 1354395 (0.0008) [2023-12-27 01:14:21,998][105692] Updated weights for policy 0, policy_version 1352438 (0.0007) [2023-12-27 01:14:22,057][105692] Updated weights for policy 0, policy_version 1352448 (0.0008) [2023-12-27 01:14:22,117][105692] Updated weights for policy 0, policy_version 1352458 (0.0009) [2023-12-27 01:14:22,702][105620] Updated weights for policy 1, policy_version 1354405 (0.0007) [2023-12-27 01:14:22,769][105620] Updated weights for policy 1, policy_version 1354415 (0.0007) [2023-12-27 01:14:22,829][105620] Updated weights for policy 1, policy_version 1354425 (0.0007) [2023-12-27 01:14:22,960][105692] Updated weights for policy 0, policy_version 1352468 (0.0009) [2023-12-27 01:14:23,024][105692] Updated weights for policy 0, policy_version 1352478 (0.0009) [2023-12-27 01:14:23,080][105692] Updated weights for policy 0, policy_version 1352488 (0.0008) [2023-12-27 01:14:23,481][105620] Updated weights for policy 1, policy_version 1354435 (0.0007) [2023-12-27 01:14:23,533][105620] Updated weights for policy 1, policy_version 1354445 (0.0009) [2023-12-27 01:14:23,591][105620] Updated weights for policy 1, policy_version 1354455 (0.0009) [2023-12-27 01:14:23,848][105692] Updated weights for policy 0, policy_version 1352498 (0.0009) [2023-12-27 01:14:23,902][105692] Updated weights for policy 0, policy_version 1352509 (0.0009) [2023-12-27 01:14:23,953][105692] Updated weights for policy 0, policy_version 1352520 (0.0010) [2023-12-27 01:14:24,281][105620] Updated weights for policy 1, policy_version 1354465 (0.0009) [2023-12-27 01:14:24,345][105620] Updated weights for policy 1, policy_version 1354475 (0.0009) [2023-12-27 01:14:24,402][105620] Updated weights for policy 1, policy_version 1354485 (0.0008) [2023-12-27 01:14:24,465][105620] Updated weights for policy 1, policy_version 1354495 (0.0008) [2023-12-27 01:14:24,706][105692] Updated weights for policy 0, policy_version 1352530 (0.0010) [2023-12-27 01:14:24,760][105692] Updated weights for policy 0, policy_version 1352540 (0.0011) [2023-12-27 01:14:24,805][105692] Updated weights for policy 0, policy_version 1352550 (0.0010) [2023-12-27 01:14:24,858][105692] Updated weights for policy 0, policy_version 1352560 (0.0011) [2023-12-27 01:14:25,156][105620] Updated weights for policy 1, policy_version 1354505 (0.0009) [2023-12-27 01:14:25,206][105620] Updated weights for policy 1, policy_version 1354515 (0.0008) [2023-12-27 01:14:25,255][105620] Updated weights for policy 1, policy_version 1354525 (0.0008) [2023-12-27 01:14:25,630][105692] Updated weights for policy 0, policy_version 1352570 (0.0007) [2023-12-27 01:14:25,691][105692] Updated weights for policy 0, policy_version 1352580 (0.0005) [2023-12-27 01:14:25,743][105692] Updated weights for policy 0, policy_version 1352590 (0.0007) [2023-12-27 01:14:26,011][105620] Updated weights for policy 1, policy_version 1354535 (0.0009) [2023-12-27 01:14:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19933.8, 300 sec: 19521.9). Total num frames: 693116928. Throughput: 0: 9920.9, 1: 9904.0. Samples: 693128528. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-12-27 01:14:26,062][105620] Updated weights for policy 1, policy_version 1354545 (0.0010) [2023-12-27 01:14:26,063][104569] Avg episode reward: [(0, '7710.901'), (1, '8631.624')] [2023-12-27 01:14:26,114][105620] Updated weights for policy 1, policy_version 1354555 (0.0010) [2023-12-27 01:14:26,364][105692] Updated weights for policy 0, policy_version 1352600 (0.0008) [2023-12-27 01:14:26,415][105692] Updated weights for policy 0, policy_version 1352610 (0.0009) [2023-12-27 01:14:26,463][105692] Updated weights for policy 0, policy_version 1352620 (0.0008) [2023-12-27 01:14:26,863][105620] Updated weights for policy 1, policy_version 1354565 (0.0007) [2023-12-27 01:14:26,909][105620] Updated weights for policy 1, policy_version 1354575 (0.0005) [2023-12-27 01:14:26,972][105620] Updated weights for policy 1, policy_version 1354585 (0.0005) [2023-12-27 01:14:27,049][105692] Updated weights for policy 0, policy_version 1352630 (0.0007) [2023-12-27 01:14:27,103][105692] Updated weights for policy 0, policy_version 1352640 (0.0005) [2023-12-27 01:14:27,156][105692] Updated weights for policy 0, policy_version 1352650 (0.0005) [2023-12-27 01:14:27,596][105620] Updated weights for policy 1, policy_version 1354595 (0.0007) [2023-12-27 01:14:27,654][105620] Updated weights for policy 1, policy_version 1354605 (0.0010) [2023-12-27 01:14:27,718][105620] Updated weights for policy 1, policy_version 1354615 (0.0010) [2023-12-27 01:14:27,763][105692] Updated weights for policy 0, policy_version 1352660 (0.0006) [2023-12-27 01:14:27,834][105692] Updated weights for policy 0, policy_version 1352670 (0.0005) [2023-12-27 01:14:27,892][105692] Updated weights for policy 0, policy_version 1352680 (0.0005) [2023-12-27 01:14:28,466][105620] Updated weights for policy 1, policy_version 1354625 (0.0010) [2023-12-27 01:14:28,521][105620] Updated weights for policy 1, policy_version 1354635 (0.0010) [2023-12-27 01:14:28,534][105692] Updated weights for policy 0, policy_version 1352690 (0.0005) [2023-12-27 01:14:28,573][105620] Updated weights for policy 1, policy_version 1354645 (0.0010) [2023-12-27 01:14:28,590][105692] Updated weights for policy 0, policy_version 1352700 (0.0005) [2023-12-27 01:14:28,625][105620] Updated weights for policy 1, policy_version 1354655 (0.0010) [2023-12-27 01:14:28,653][105692] Updated weights for policy 0, policy_version 1352710 (0.0005) [2023-12-27 01:14:28,715][105692] Updated weights for policy 0, policy_version 1352720 (0.0007) [2023-12-27 01:14:29,378][105620] Updated weights for policy 1, policy_version 1354665 (0.0008) [2023-12-27 01:14:29,429][105620] Updated weights for policy 1, policy_version 1354675 (0.0010) [2023-12-27 01:14:29,443][105692] Updated weights for policy 0, policy_version 1352730 (0.0005) [2023-12-27 01:14:29,487][105620] Updated weights for policy 1, policy_version 1354685 (0.0010) [2023-12-27 01:14:29,505][105692] Updated weights for policy 0, policy_version 1352740 (0.0005) [2023-12-27 01:14:29,568][105692] Updated weights for policy 0, policy_version 1352750 (0.0008) [2023-12-27 01:14:30,228][105620] Updated weights for policy 1, policy_version 1354695 (0.0007) [2023-12-27 01:14:30,287][105692] Updated weights for policy 0, policy_version 1352760 (0.0009) [2023-12-27 01:14:30,289][105620] Updated weights for policy 1, policy_version 1354705 (0.0005) [2023-12-27 01:14:30,348][105692] Updated weights for policy 0, policy_version 1352770 (0.0007) [2023-12-27 01:14:30,351][105620] Updated weights for policy 1, policy_version 1354715 (0.0006) [2023-12-27 01:14:30,410][105692] Updated weights for policy 0, policy_version 1352780 (0.0010) [2023-12-27 01:14:30,881][105620] Updated weights for policy 1, policy_version 1354725 (0.0007) [2023-12-27 01:14:30,928][105620] Updated weights for policy 1, policy_version 1354735 (0.0005) [2023-12-27 01:14:30,976][105620] Updated weights for policy 1, policy_version 1354745 (0.0007) [2023-12-27 01:14:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 20070.4, 300 sec: 19605.3). Total num frames: 693223424. Throughput: 0: 10040.8, 1: 9879.3. Samples: 693191292. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:14:31,062][104569] Avg episode reward: [(0, '7900.071'), (1, '8814.934')] [2023-12-27 01:14:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001352784_346365952.pth... [2023-12-27 01:14:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001354752_346857472.pth... [2023-12-27 01:14:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001351632_346071040.pth [2023-12-27 01:14:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001353568_346554368.pth [2023-12-27 01:14:31,251][105692] Updated weights for policy 0, policy_version 1352790 (0.0009) [2023-12-27 01:14:31,309][105692] Updated weights for policy 0, policy_version 1352800 (0.0009) [2023-12-27 01:14:31,376][105692] Updated weights for policy 0, policy_version 1352810 (0.0010) [2023-12-27 01:14:31,659][105620] Updated weights for policy 1, policy_version 1354755 (0.0009) [2023-12-27 01:14:31,725][105620] Updated weights for policy 1, policy_version 1354765 (0.0008) [2023-12-27 01:14:31,784][105620] Updated weights for policy 1, policy_version 1354775 (0.0009) [2023-12-27 01:14:31,814][105586] KL-divergence is very high: 128.0358 [2023-12-27 01:14:32,059][105692] Updated weights for policy 0, policy_version 1352820 (0.0009) [2023-12-27 01:14:32,112][105692] Updated weights for policy 0, policy_version 1352830 (0.0009) [2023-12-27 01:14:32,170][105692] Updated weights for policy 0, policy_version 1352840 (0.0009) [2023-12-27 01:14:32,496][105620] Updated weights for policy 1, policy_version 1354785 (0.0009) [2023-12-27 01:14:32,550][105620] Updated weights for policy 1, policy_version 1354795 (0.0009) [2023-12-27 01:14:32,601][105620] Updated weights for policy 1, policy_version 1354805 (0.0009) [2023-12-27 01:14:32,670][105620] Updated weights for policy 1, policy_version 1354815 (0.0010) [2023-12-27 01:14:32,874][105692] Updated weights for policy 0, policy_version 1352850 (0.0010) [2023-12-27 01:14:32,921][105692] Updated weights for policy 0, policy_version 1352860 (0.0009) [2023-12-27 01:14:32,980][105692] Updated weights for policy 0, policy_version 1352870 (0.0006) [2023-12-27 01:14:33,045][105692] Updated weights for policy 0, policy_version 1352880 (0.0005) [2023-12-27 01:14:33,354][105620] Updated weights for policy 1, policy_version 1354825 (0.0009) [2023-12-27 01:14:33,421][105620] Updated weights for policy 1, policy_version 1354835 (0.0010) [2023-12-27 01:14:33,465][105620] Updated weights for policy 1, policy_version 1354845 (0.0010) [2023-12-27 01:14:33,677][105692] Updated weights for policy 0, policy_version 1352890 (0.0008) [2023-12-27 01:14:33,731][105692] Updated weights for policy 0, policy_version 1352900 (0.0008) [2023-12-27 01:14:33,779][105692] Updated weights for policy 0, policy_version 1352910 (0.0008) [2023-12-27 01:14:34,203][105620] Updated weights for policy 1, policy_version 1354855 (0.0010) [2023-12-27 01:14:34,266][105620] Updated weights for policy 1, policy_version 1354865 (0.0007) [2023-12-27 01:14:34,322][105620] Updated weights for policy 1, policy_version 1354875 (0.0011) [2023-12-27 01:14:34,604][105692] Updated weights for policy 0, policy_version 1352920 (0.0008) [2023-12-27 01:14:34,667][105692] Updated weights for policy 0, policy_version 1352930 (0.0008) [2023-12-27 01:14:34,726][105692] Updated weights for policy 0, policy_version 1352940 (0.0008) [2023-12-27 01:14:35,059][105620] Updated weights for policy 1, policy_version 1354885 (0.0010) [2023-12-27 01:14:35,108][105620] Updated weights for policy 1, policy_version 1354895 (0.0010) [2023-12-27 01:14:35,170][105620] Updated weights for policy 1, policy_version 1354905 (0.0011) [2023-12-27 01:14:35,468][105692] Updated weights for policy 0, policy_version 1352950 (0.0008) [2023-12-27 01:14:35,514][105692] Updated weights for policy 0, policy_version 1352960 (0.0008) [2023-12-27 01:14:35,566][105692] Updated weights for policy 0, policy_version 1352970 (0.0008) [2023-12-27 01:14:35,956][105620] Updated weights for policy 1, policy_version 1354915 (0.0010) [2023-12-27 01:14:36,014][105620] Updated weights for policy 1, policy_version 1354925 (0.0009) [2023-12-27 01:14:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 693313536. Throughput: 0: 9981.4, 1: 9861.0. Samples: 693309044. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:14:36,062][104569] Avg episode reward: [(0, '7440.814'), (1, '8632.771')] [2023-12-27 01:14:36,075][105620] Updated weights for policy 1, policy_version 1354935 (0.0008) [2023-12-27 01:14:36,313][105692] Updated weights for policy 0, policy_version 1352980 (0.0009) [2023-12-27 01:14:36,375][105692] Updated weights for policy 0, policy_version 1352990 (0.0009) [2023-12-27 01:14:36,430][105692] Updated weights for policy 0, policy_version 1353000 (0.0009) [2023-12-27 01:14:36,762][105620] Updated weights for policy 1, policy_version 1354945 (0.0007) [2023-12-27 01:14:36,836][105620] Updated weights for policy 1, policy_version 1354955 (0.0006) [2023-12-27 01:14:36,898][105620] Updated weights for policy 1, policy_version 1354965 (0.0009) [2023-12-27 01:14:36,963][105620] Updated weights for policy 1, policy_version 1354975 (0.0009) [2023-12-27 01:14:37,125][105692] Updated weights for policy 0, policy_version 1353010 (0.0009) [2023-12-27 01:14:37,187][105692] Updated weights for policy 0, policy_version 1353020 (0.0010) [2023-12-27 01:14:37,250][105692] Updated weights for policy 0, policy_version 1353030 (0.0011) [2023-12-27 01:14:37,306][105692] Updated weights for policy 0, policy_version 1353040 (0.0011) [2023-12-27 01:14:37,676][105620] Updated weights for policy 1, policy_version 1354985 (0.0008) [2023-12-27 01:14:37,738][105620] Updated weights for policy 1, policy_version 1354995 (0.0008) [2023-12-27 01:14:37,802][105620] Updated weights for policy 1, policy_version 1355005 (0.0008) [2023-12-27 01:14:38,063][105692] Updated weights for policy 0, policy_version 1353050 (0.0010) [2023-12-27 01:14:38,119][105692] Updated weights for policy 0, policy_version 1353060 (0.0010) [2023-12-27 01:14:38,167][105692] Updated weights for policy 0, policy_version 1353070 (0.0010) [2023-12-27 01:14:38,562][105620] Updated weights for policy 1, policy_version 1355015 (0.0008) [2023-12-27 01:14:38,628][105620] Updated weights for policy 1, policy_version 1355025 (0.0009) [2023-12-27 01:14:38,694][105620] Updated weights for policy 1, policy_version 1355035 (0.0008) [2023-12-27 01:14:38,884][105692] Updated weights for policy 0, policy_version 1353080 (0.0006) [2023-12-27 01:14:38,922][105585] KL-divergence is very high: 102.3585 [2023-12-27 01:14:38,945][105692] Updated weights for policy 0, policy_version 1353090 (0.0010) [2023-12-27 01:14:38,966][105585] KL-divergence is very high: 193.5857 [2023-12-27 01:14:38,999][105692] Updated weights for policy 0, policy_version 1353100 (0.0010) [2023-12-27 01:14:39,013][105585] KL-divergence is very high: 217.7467 [2023-12-27 01:14:39,380][105620] Updated weights for policy 1, policy_version 1355045 (0.0008) [2023-12-27 01:14:39,451][105620] Updated weights for policy 1, policy_version 1355055 (0.0008) [2023-12-27 01:14:39,510][105620] Updated weights for policy 1, policy_version 1355065 (0.0011) [2023-12-27 01:14:39,672][105692] Updated weights for policy 0, policy_version 1353110 (0.0008) [2023-12-27 01:14:39,740][105692] Updated weights for policy 0, policy_version 1353120 (0.0006) [2023-12-27 01:14:39,790][105692] Updated weights for policy 0, policy_version 1353130 (0.0006) [2023-12-27 01:14:40,255][105620] Updated weights for policy 1, policy_version 1355075 (0.0010) [2023-12-27 01:14:40,314][105620] Updated weights for policy 1, policy_version 1355085 (0.0010) [2023-12-27 01:14:40,366][105620] Updated weights for policy 1, policy_version 1355095 (0.0010) [2023-12-27 01:14:40,487][105692] Updated weights for policy 0, policy_version 1353140 (0.0008) [2023-12-27 01:14:40,547][105692] Updated weights for policy 0, policy_version 1353150 (0.0009) [2023-12-27 01:14:40,609][105692] Updated weights for policy 0, policy_version 1353160 (0.0008) [2023-12-27 01:14:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 693411840. Throughput: 0: 9969.0, 1: 9804.3. Samples: 693422760. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:14:41,062][104569] Avg episode reward: [(0, '7897.418'), (1, '8274.780')] [2023-12-27 01:14:41,126][105620] Updated weights for policy 1, policy_version 1355105 (0.0010) [2023-12-27 01:14:41,197][105620] Updated weights for policy 1, policy_version 1355115 (0.0011) [2023-12-27 01:14:41,250][105620] Updated weights for policy 1, policy_version 1355125 (0.0011) [2023-12-27 01:14:41,316][105620] Updated weights for policy 1, policy_version 1355135 (0.0011) [2023-12-27 01:14:41,337][105692] Updated weights for policy 0, policy_version 1353170 (0.0006) [2023-12-27 01:14:41,403][105692] Updated weights for policy 0, policy_version 1353180 (0.0008) [2023-12-27 01:14:41,463][105692] Updated weights for policy 0, policy_version 1353190 (0.0008) [2023-12-27 01:14:41,524][105692] Updated weights for policy 0, policy_version 1353200 (0.0009) [2023-12-27 01:14:42,099][105620] Updated weights for policy 1, policy_version 1355145 (0.0011) [2023-12-27 01:14:42,162][105620] Updated weights for policy 1, policy_version 1355155 (0.0011) [2023-12-27 01:14:42,217][105620] Updated weights for policy 1, policy_version 1355166 (0.0009) [2023-12-27 01:14:42,316][105692] Updated weights for policy 0, policy_version 1353210 (0.0009) [2023-12-27 01:14:42,376][105692] Updated weights for policy 0, policy_version 1353220 (0.0009) [2023-12-27 01:14:42,434][105692] Updated weights for policy 0, policy_version 1353230 (0.0010) [2023-12-27 01:14:42,891][105620] Updated weights for policy 1, policy_version 1355176 (0.0008) [2023-12-27 01:14:42,961][105620] Updated weights for policy 1, policy_version 1355186 (0.0006) [2023-12-27 01:14:43,023][105620] Updated weights for policy 1, policy_version 1355196 (0.0006) [2023-12-27 01:14:43,200][105692] Updated weights for policy 0, policy_version 1353240 (0.0010) [2023-12-27 01:14:43,265][105692] Updated weights for policy 0, policy_version 1353250 (0.0010) [2023-12-27 01:14:43,312][105692] Updated weights for policy 0, policy_version 1353260 (0.0010) [2023-12-27 01:14:43,707][105620] Updated weights for policy 1, policy_version 1355206 (0.0007) [2023-12-27 01:14:43,764][105620] Updated weights for policy 1, policy_version 1355216 (0.0005) [2023-12-27 01:14:43,827][105620] Updated weights for policy 1, policy_version 1355226 (0.0006) [2023-12-27 01:14:44,008][105692] Updated weights for policy 0, policy_version 1353270 (0.0007) [2023-12-27 01:14:44,063][105692] Updated weights for policy 0, policy_version 1353280 (0.0006) [2023-12-27 01:14:44,132][105692] Updated weights for policy 0, policy_version 1353290 (0.0010) [2023-12-27 01:14:44,341][105620] Updated weights for policy 1, policy_version 1355236 (0.0005) [2023-12-27 01:14:44,399][105620] Updated weights for policy 1, policy_version 1355246 (0.0006) [2023-12-27 01:14:44,456][105620] Updated weights for policy 1, policy_version 1355256 (0.0005) [2023-12-27 01:14:44,824][105692] Updated weights for policy 0, policy_version 1353300 (0.0010) [2023-12-27 01:14:44,883][105692] Updated weights for policy 0, policy_version 1353310 (0.0010) [2023-12-27 01:14:44,943][105692] Updated weights for policy 0, policy_version 1353320 (0.0011) [2023-12-27 01:14:45,123][105620] Updated weights for policy 1, policy_version 1355266 (0.0006) [2023-12-27 01:14:45,172][105620] Updated weights for policy 1, policy_version 1355276 (0.0008) [2023-12-27 01:14:45,228][105620] Updated weights for policy 1, policy_version 1355286 (0.0008) [2023-12-27 01:14:45,300][105620] Updated weights for policy 1, policy_version 1355296 (0.0009) [2023-12-27 01:14:45,679][105692] Updated weights for policy 0, policy_version 1353330 (0.0011) [2023-12-27 01:14:45,737][105692] Updated weights for policy 0, policy_version 1353340 (0.0010) [2023-12-27 01:14:45,789][105692] Updated weights for policy 0, policy_version 1353350 (0.0010) [2023-12-27 01:14:45,838][105692] Updated weights for policy 0, policy_version 1353360 (0.0010) [2023-12-27 01:14:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 693510144. Throughput: 0: 9874.5, 1: 9842.4. Samples: 693479768. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:14:46,063][104569] Avg episode reward: [(0, '8081.847'), (1, '8373.484')] [2023-12-27 01:14:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001353360_346513408.pth... [2023-12-27 01:14:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001352208_346218496.pth [2023-12-27 01:14:46,101][105620] Updated weights for policy 1, policy_version 1355306 (0.0008) [2023-12-27 01:14:46,156][105620] Updated weights for policy 1, policy_version 1355316 (0.0008) [2023-12-27 01:14:46,206][105620] Updated weights for policy 1, policy_version 1355326 (0.0009) [2023-12-27 01:14:46,216][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001355328_347004928.pth... [2023-12-27 01:14:46,220][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001354176_346710016.pth [2023-12-27 01:14:46,517][105692] Updated weights for policy 0, policy_version 1353370 (0.0008) [2023-12-27 01:14:46,569][105692] Updated weights for policy 0, policy_version 1353380 (0.0010) [2023-12-27 01:14:46,623][105692] Updated weights for policy 0, policy_version 1353390 (0.0010) [2023-12-27 01:14:46,948][105620] Updated weights for policy 1, policy_version 1355336 (0.0007) [2023-12-27 01:14:47,000][105620] Updated weights for policy 1, policy_version 1355346 (0.0008) [2023-12-27 01:14:47,051][105620] Updated weights for policy 1, policy_version 1355356 (0.0008) [2023-12-27 01:14:47,344][105692] Updated weights for policy 0, policy_version 1353400 (0.0010) [2023-12-27 01:14:47,395][105692] Updated weights for policy 0, policy_version 1353410 (0.0010) [2023-12-27 01:14:47,452][105692] Updated weights for policy 0, policy_version 1353420 (0.0010) [2023-12-27 01:14:47,812][105620] Updated weights for policy 1, policy_version 1355366 (0.0008) [2023-12-27 01:14:47,871][105620] Updated weights for policy 1, policy_version 1355376 (0.0008) [2023-12-27 01:14:47,923][105620] Updated weights for policy 1, policy_version 1355386 (0.0010) [2023-12-27 01:14:48,196][105692] Updated weights for policy 0, policy_version 1353430 (0.0010) [2023-12-27 01:14:48,264][105692] Updated weights for policy 0, policy_version 1353440 (0.0010) [2023-12-27 01:14:48,319][105692] Updated weights for policy 0, policy_version 1353450 (0.0010) [2023-12-27 01:14:48,549][105620] Updated weights for policy 1, policy_version 1355396 (0.0008) [2023-12-27 01:14:48,611][105620] Updated weights for policy 1, policy_version 1355406 (0.0005) [2023-12-27 01:14:48,670][105620] Updated weights for policy 1, policy_version 1355416 (0.0006) [2023-12-27 01:14:49,077][105692] Updated weights for policy 0, policy_version 1353460 (0.0011) [2023-12-27 01:14:49,142][105692] Updated weights for policy 0, policy_version 1353470 (0.0011) [2023-12-27 01:14:49,204][105692] Updated weights for policy 0, policy_version 1353480 (0.0011) [2023-12-27 01:14:49,304][105620] Updated weights for policy 1, policy_version 1355426 (0.0006) [2023-12-27 01:14:49,372][105620] Updated weights for policy 1, policy_version 1355436 (0.0008) [2023-12-27 01:14:49,423][105620] Updated weights for policy 1, policy_version 1355446 (0.0007) [2023-12-27 01:14:49,472][105620] Updated weights for policy 1, policy_version 1355456 (0.0010) [2023-12-27 01:14:49,952][105692] Updated weights for policy 0, policy_version 1353490 (0.0010) [2023-12-27 01:14:50,003][105692] Updated weights for policy 0, policy_version 1353500 (0.0010) [2023-12-27 01:14:50,072][105692] Updated weights for policy 0, policy_version 1353510 (0.0010) [2023-12-27 01:14:50,134][105692] Updated weights for policy 0, policy_version 1353520 (0.0010) [2023-12-27 01:14:50,182][105620] Updated weights for policy 1, policy_version 1355466 (0.0010) [2023-12-27 01:14:50,240][105620] Updated weights for policy 1, policy_version 1355476 (0.0010) [2023-12-27 01:14:50,299][105620] Updated weights for policy 1, policy_version 1355486 (0.0010) [2023-12-27 01:14:50,768][105692] Updated weights for policy 0, policy_version 1353530 (0.0005) [2023-12-27 01:14:50,833][105692] Updated weights for policy 0, policy_version 1353540 (0.0010) [2023-12-27 01:14:50,891][105692] Updated weights for policy 0, policy_version 1353550 (0.0010) [2023-12-27 01:14:51,060][105620] Updated weights for policy 1, policy_version 1355496 (0.0010) [2023-12-27 01:14:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 693608448. Throughput: 0: 9841.9, 1: 9841.0. Samples: 693598604. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:14:51,062][104569] Avg episode reward: [(0, '8263.403'), (1, '8369.265')] [2023-12-27 01:14:51,118][105620] Updated weights for policy 1, policy_version 1355506 (0.0010) [2023-12-27 01:14:51,174][105620] Updated weights for policy 1, policy_version 1355516 (0.0011) [2023-12-27 01:14:51,663][105692] Updated weights for policy 0, policy_version 1353560 (0.0010) [2023-12-27 01:14:51,731][105692] Updated weights for policy 0, policy_version 1353570 (0.0010) [2023-12-27 01:14:51,785][105692] Updated weights for policy 0, policy_version 1353580 (0.0010) [2023-12-27 01:14:51,903][105620] Updated weights for policy 1, policy_version 1355526 (0.0011) [2023-12-27 01:14:51,952][105620] Updated weights for policy 1, policy_version 1355536 (0.0011) [2023-12-27 01:14:52,005][105620] Updated weights for policy 1, policy_version 1355546 (0.0011) [2023-12-27 01:14:52,574][105692] Updated weights for policy 0, policy_version 1353590 (0.0010) [2023-12-27 01:14:52,635][105692] Updated weights for policy 0, policy_version 1353600 (0.0007) [2023-12-27 01:14:52,706][105692] Updated weights for policy 0, policy_version 1353610 (0.0005) [2023-12-27 01:14:52,731][105620] Updated weights for policy 1, policy_version 1355556 (0.0009) [2023-12-27 01:14:52,783][105620] Updated weights for policy 1, policy_version 1355566 (0.0010) [2023-12-27 01:14:52,834][105620] Updated weights for policy 1, policy_version 1355576 (0.0009) [2023-12-27 01:14:53,335][105692] Updated weights for policy 0, policy_version 1353620 (0.0006) [2023-12-27 01:14:53,394][105692] Updated weights for policy 0, policy_version 1353630 (0.0006) [2023-12-27 01:14:53,449][105692] Updated weights for policy 0, policy_version 1353640 (0.0006) [2023-12-27 01:14:53,515][105620] Updated weights for policy 1, policy_version 1355586 (0.0011) [2023-12-27 01:14:53,573][105620] Updated weights for policy 1, policy_version 1355596 (0.0010) [2023-12-27 01:14:53,632][105620] Updated weights for policy 1, policy_version 1355606 (0.0010) [2023-12-27 01:14:53,688][105620] Updated weights for policy 1, policy_version 1355616 (0.0010) [2023-12-27 01:14:53,954][105692] Updated weights for policy 0, policy_version 1353650 (0.0005) [2023-12-27 01:14:54,013][105692] Updated weights for policy 0, policy_version 1353660 (0.0005) [2023-12-27 01:14:54,072][105692] Updated weights for policy 0, policy_version 1353670 (0.0005) [2023-12-27 01:14:54,135][105692] Updated weights for policy 0, policy_version 1353680 (0.0006) [2023-12-27 01:14:54,334][105620] Updated weights for policy 1, policy_version 1355626 (0.0005) [2023-12-27 01:14:54,418][105620] Updated weights for policy 1, policy_version 1355636 (0.0005) [2023-12-27 01:14:54,476][105620] Updated weights for policy 1, policy_version 1355646 (0.0006) [2023-12-27 01:14:54,710][105692] Updated weights for policy 0, policy_version 1353690 (0.0009) [2023-12-27 01:14:54,758][105692] Updated weights for policy 0, policy_version 1353700 (0.0010) [2023-12-27 01:14:54,803][105692] Updated weights for policy 0, policy_version 1353710 (0.0005) [2023-12-27 01:14:55,017][105620] Updated weights for policy 1, policy_version 1355656 (0.0010) [2023-12-27 01:14:55,083][105620] Updated weights for policy 1, policy_version 1355666 (0.0011) [2023-12-27 01:14:55,143][105620] Updated weights for policy 1, policy_version 1355676 (0.0011) [2023-12-27 01:14:55,443][105692] Updated weights for policy 0, policy_version 1353720 (0.0006) [2023-12-27 01:14:55,500][105692] Updated weights for policy 0, policy_version 1353730 (0.0010) [2023-12-27 01:14:55,554][105692] Updated weights for policy 0, policy_version 1353740 (0.0010) [2023-12-27 01:14:55,804][105620] Updated weights for policy 1, policy_version 1355686 (0.0007) [2023-12-27 01:14:55,852][105620] Updated weights for policy 1, policy_version 1355696 (0.0007) [2023-12-27 01:14:55,916][105620] Updated weights for policy 1, policy_version 1355706 (0.0010) [2023-12-27 01:14:56,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 693714944. Throughput: 0: 9935.4, 1: 9879.8. Samples: 693722168. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:14:56,062][104569] Avg episode reward: [(0, '8624.935'), (1, '8369.400')] [2023-12-27 01:14:56,286][105692] Updated weights for policy 0, policy_version 1353750 (0.0010) [2023-12-27 01:14:56,334][105692] Updated weights for policy 0, policy_version 1353760 (0.0010) [2023-12-27 01:14:56,385][105692] Updated weights for policy 0, policy_version 1353770 (0.0010) [2023-12-27 01:14:56,572][105620] Updated weights for policy 1, policy_version 1355716 (0.0008) [2023-12-27 01:14:56,625][105620] Updated weights for policy 1, policy_version 1355726 (0.0005) [2023-12-27 01:14:56,678][105620] Updated weights for policy 1, policy_version 1355736 (0.0009) [2023-12-27 01:14:57,072][105692] Updated weights for policy 0, policy_version 1353780 (0.0009) [2023-12-27 01:14:57,133][105692] Updated weights for policy 0, policy_version 1353790 (0.0005) [2023-12-27 01:14:57,186][105692] Updated weights for policy 0, policy_version 1353800 (0.0006) [2023-12-27 01:14:57,216][105585] KL-divergence is very high: 101.2225 [2023-12-27 01:14:57,399][105620] Updated weights for policy 1, policy_version 1355746 (0.0009) [2023-12-27 01:14:57,452][105620] Updated weights for policy 1, policy_version 1355756 (0.0005) [2023-12-27 01:14:57,503][105620] Updated weights for policy 1, policy_version 1355766 (0.0005) [2023-12-27 01:14:57,558][105620] Updated weights for policy 1, policy_version 1355776 (0.0005) [2023-12-27 01:14:57,892][105692] Updated weights for policy 0, policy_version 1353810 (0.0010) [2023-12-27 01:14:57,949][105692] Updated weights for policy 0, policy_version 1353820 (0.0010) [2023-12-27 01:14:58,013][105692] Updated weights for policy 0, policy_version 1353830 (0.0010) [2023-12-27 01:14:58,071][105692] Updated weights for policy 0, policy_version 1353840 (0.0010) [2023-12-27 01:14:58,095][105620] Updated weights for policy 1, policy_version 1355786 (0.0006) [2023-12-27 01:14:58,160][105620] Updated weights for policy 1, policy_version 1355796 (0.0007) [2023-12-27 01:14:58,220][105620] Updated weights for policy 1, policy_version 1355806 (0.0008) [2023-12-27 01:14:58,868][105692] Updated weights for policy 0, policy_version 1353850 (0.0009) [2023-12-27 01:14:58,942][105692] Updated weights for policy 0, policy_version 1353860 (0.0008) [2023-12-27 01:14:59,000][105692] Updated weights for policy 0, policy_version 1353870 (0.0009) [2023-12-27 01:14:59,052][105620] Updated weights for policy 1, policy_version 1355816 (0.0007) [2023-12-27 01:14:59,109][105620] Updated weights for policy 1, policy_version 1355826 (0.0007) [2023-12-27 01:14:59,162][105620] Updated weights for policy 1, policy_version 1355836 (0.0005) [2023-12-27 01:14:59,841][105620] Updated weights for policy 1, policy_version 1355846 (0.0007) [2023-12-27 01:14:59,850][105692] Updated weights for policy 0, policy_version 1353880 (0.0009) [2023-12-27 01:14:59,896][105620] Updated weights for policy 1, policy_version 1355856 (0.0008) [2023-12-27 01:14:59,914][105692] Updated weights for policy 0, policy_version 1353890 (0.0007) [2023-12-27 01:14:59,951][105620] Updated weights for policy 1, policy_version 1355866 (0.0007) [2023-12-27 01:14:59,978][105692] Updated weights for policy 0, policy_version 1353900 (0.0007) [2023-12-27 01:15:00,604][105620] Updated weights for policy 1, policy_version 1355876 (0.0007) [2023-12-27 01:15:00,663][105620] Updated weights for policy 1, policy_version 1355886 (0.0009) [2023-12-27 01:15:00,700][105692] Updated weights for policy 0, policy_version 1353910 (0.0007) [2023-12-27 01:15:00,715][105620] Updated weights for policy 1, policy_version 1355896 (0.0010) [2023-12-27 01:15:00,764][105692] Updated weights for policy 0, policy_version 1353920 (0.0006) [2023-12-27 01:15:00,823][105692] Updated weights for policy 0, policy_version 1353930 (0.0008) [2023-12-27 01:15:01,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 693813248. Throughput: 0: 9945.5, 1: 9929.2. Samples: 693782372. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:15:01,062][104569] Avg episode reward: [(0, '7891.022'), (1, '8638.990')] [2023-12-27 01:15:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001353936_346660864.pth... [2023-12-27 01:15:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001355904_347152384.pth... [2023-12-27 01:15:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001352784_346365952.pth [2023-12-27 01:15:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001354752_346857472.pth [2023-12-27 01:15:01,426][105620] Updated weights for policy 1, policy_version 1355906 (0.0010) [2023-12-27 01:15:01,489][105620] Updated weights for policy 1, policy_version 1355916 (0.0009) [2023-12-27 01:15:01,540][105692] Updated weights for policy 0, policy_version 1353940 (0.0009) [2023-12-27 01:15:01,545][105620] Updated weights for policy 1, policy_version 1355926 (0.0007) [2023-12-27 01:15:01,596][105692] Updated weights for policy 0, policy_version 1353950 (0.0007) [2023-12-27 01:15:01,602][105620] Updated weights for policy 1, policy_version 1355936 (0.0008) [2023-12-27 01:15:01,675][105692] Updated weights for policy 0, policy_version 1353960 (0.0008) [2023-12-27 01:15:01,695][105585] KL-divergence is very high: 123.2604 [2023-12-27 01:15:02,329][105620] Updated weights for policy 1, policy_version 1355946 (0.0009) [2023-12-27 01:15:02,393][105620] Updated weights for policy 1, policy_version 1355956 (0.0009) [2023-12-27 01:15:02,419][105692] Updated weights for policy 0, policy_version 1353970 (0.0007) [2023-12-27 01:15:02,447][105620] Updated weights for policy 1, policy_version 1355966 (0.0009) [2023-12-27 01:15:02,478][105692] Updated weights for policy 0, policy_version 1353980 (0.0009) [2023-12-27 01:15:02,537][105692] Updated weights for policy 0, policy_version 1353990 (0.0011) [2023-12-27 01:15:02,599][105692] Updated weights for policy 0, policy_version 1354000 (0.0010) [2023-12-27 01:15:03,146][105620] Updated weights for policy 1, policy_version 1355976 (0.0008) [2023-12-27 01:15:03,203][105620] Updated weights for policy 1, policy_version 1355986 (0.0009) [2023-12-27 01:15:03,255][105620] Updated weights for policy 1, policy_version 1355996 (0.0010) [2023-12-27 01:15:03,296][105692] Updated weights for policy 0, policy_version 1354010 (0.0005) [2023-12-27 01:15:03,344][105692] Updated weights for policy 0, policy_version 1354020 (0.0005) [2023-12-27 01:15:03,392][105692] Updated weights for policy 0, policy_version 1354030 (0.0005) [2023-12-27 01:15:03,988][105620] Updated weights for policy 1, policy_version 1356006 (0.0008) [2023-12-27 01:15:03,999][105692] Updated weights for policy 0, policy_version 1354040 (0.0009) [2023-12-27 01:15:04,048][105620] Updated weights for policy 1, policy_version 1356016 (0.0005) [2023-12-27 01:15:04,048][105692] Updated weights for policy 0, policy_version 1354050 (0.0010) [2023-12-27 01:15:04,107][105692] Updated weights for policy 0, policy_version 1354060 (0.0011) [2023-12-27 01:15:04,109][105620] Updated weights for policy 1, policy_version 1356026 (0.0006) [2023-12-27 01:15:04,783][105620] Updated weights for policy 1, policy_version 1356036 (0.0006) [2023-12-27 01:15:04,833][105620] Updated weights for policy 1, policy_version 1356046 (0.0008) [2023-12-27 01:15:04,848][105692] Updated weights for policy 0, policy_version 1354070 (0.0007) [2023-12-27 01:15:04,885][105620] Updated weights for policy 1, policy_version 1356056 (0.0007) [2023-12-27 01:15:04,914][105692] Updated weights for policy 0, policy_version 1354080 (0.0007) [2023-12-27 01:15:04,962][105692] Updated weights for policy 0, policy_version 1354090 (0.0008) [2023-12-27 01:15:05,450][105620] Updated weights for policy 1, policy_version 1356066 (0.0006) [2023-12-27 01:15:05,513][105620] Updated weights for policy 1, policy_version 1356076 (0.0005) [2023-12-27 01:15:05,567][105620] Updated weights for policy 1, policy_version 1356086 (0.0006) [2023-12-27 01:15:05,613][105620] Updated weights for policy 1, policy_version 1356096 (0.0005) [2023-12-27 01:15:05,691][105692] Updated weights for policy 0, policy_version 1354100 (0.0007) [2023-12-27 01:15:05,750][105692] Updated weights for policy 0, policy_version 1354110 (0.0008) [2023-12-27 01:15:05,804][105692] Updated weights for policy 0, policy_version 1354120 (0.0010) [2023-12-27 01:15:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 693911552. Throughput: 0: 9742.5, 1: 9897.9. Samples: 693898312. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:15:06,062][104569] Avg episode reward: [(0, '7989.311'), (1, '8817.320')] [2023-12-27 01:15:06,173][105620] Updated weights for policy 1, policy_version 1356106 (0.0008) [2023-12-27 01:15:06,221][105620] Updated weights for policy 1, policy_version 1356116 (0.0008) [2023-12-27 01:15:06,277][105620] Updated weights for policy 1, policy_version 1356126 (0.0008) [2023-12-27 01:15:06,513][105692] Updated weights for policy 0, policy_version 1354131 (0.0009) [2023-12-27 01:15:06,580][105692] Updated weights for policy 0, policy_version 1354141 (0.0006) [2023-12-27 01:15:06,644][105692] Updated weights for policy 0, policy_version 1354151 (0.0006) [2023-12-27 01:15:06,997][105620] Updated weights for policy 1, policy_version 1356136 (0.0010) [2023-12-27 01:15:07,055][105620] Updated weights for policy 1, policy_version 1356146 (0.0011) [2023-12-27 01:15:07,117][105620] Updated weights for policy 1, policy_version 1356156 (0.0011) [2023-12-27 01:15:07,328][105692] Updated weights for policy 0, policy_version 1354161 (0.0006) [2023-12-27 01:15:07,386][105692] Updated weights for policy 0, policy_version 1354171 (0.0009) [2023-12-27 01:15:07,439][105692] Updated weights for policy 0, policy_version 1354181 (0.0008) [2023-12-27 01:15:07,492][105692] Updated weights for policy 0, policy_version 1354191 (0.0008) [2023-12-27 01:15:07,873][105620] Updated weights for policy 1, policy_version 1356166 (0.0010) [2023-12-27 01:15:07,925][105620] Updated weights for policy 1, policy_version 1356176 (0.0010) [2023-12-27 01:15:07,984][105620] Updated weights for policy 1, policy_version 1356186 (0.0010) [2023-12-27 01:15:08,281][105692] Updated weights for policy 0, policy_version 1354201 (0.0010) [2023-12-27 01:15:08,349][105692] Updated weights for policy 0, policy_version 1354211 (0.0009) [2023-12-27 01:15:08,409][105692] Updated weights for policy 0, policy_version 1354221 (0.0008) [2023-12-27 01:15:08,674][105620] Updated weights for policy 1, policy_version 1356196 (0.0011) [2023-12-27 01:15:08,727][105620] Updated weights for policy 1, policy_version 1356206 (0.0011) [2023-12-27 01:15:08,790][105620] Updated weights for policy 1, policy_version 1356216 (0.0011) [2023-12-27 01:15:09,175][105692] Updated weights for policy 0, policy_version 1354231 (0.0006) [2023-12-27 01:15:09,238][105692] Updated weights for policy 0, policy_version 1354241 (0.0007) [2023-12-27 01:15:09,258][105585] KL-divergence is very high: 148.0995 [2023-12-27 01:15:09,301][105692] Updated weights for policy 0, policy_version 1354251 (0.0008) [2023-12-27 01:15:09,311][105585] KL-divergence is very high: 145.6050 [2023-12-27 01:15:09,482][105620] Updated weights for policy 1, policy_version 1356226 (0.0009) [2023-12-27 01:15:09,536][105620] Updated weights for policy 1, policy_version 1356236 (0.0005) [2023-12-27 01:15:09,597][105620] Updated weights for policy 1, policy_version 1356246 (0.0007) [2023-12-27 01:15:09,657][105620] Updated weights for policy 1, policy_version 1356256 (0.0005) [2023-12-27 01:15:10,095][105692] Updated weights for policy 0, policy_version 1354261 (0.0009) [2023-12-27 01:15:10,158][105692] Updated weights for policy 0, policy_version 1354271 (0.0009) [2023-12-27 01:15:10,217][105585] KL-divergence is very high: 101.8629 [2023-12-27 01:15:10,217][105692] Updated weights for policy 0, policy_version 1354281 (0.0009) [2023-12-27 01:15:10,314][105620] Updated weights for policy 1, policy_version 1356266 (0.0009) [2023-12-27 01:15:10,366][105620] Updated weights for policy 1, policy_version 1356276 (0.0009) [2023-12-27 01:15:10,421][105620] Updated weights for policy 1, policy_version 1356286 (0.0009) [2023-12-27 01:15:10,935][105692] Updated weights for policy 0, policy_version 1354291 (0.0008) [2023-12-27 01:15:10,983][105692] Updated weights for policy 0, policy_version 1354301 (0.0005) [2023-12-27 01:15:11,052][105692] Updated weights for policy 0, policy_version 1354311 (0.0008) [2023-12-27 01:15:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 694001664. Throughput: 0: 9750.8, 1: 9966.3. Samples: 694015796. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:15:11,063][104569] Avg episode reward: [(0, '8169.891'), (1, '8635.840')] [2023-12-27 01:15:11,240][105620] Updated weights for policy 1, policy_version 1356297 (0.0009) [2023-12-27 01:15:11,303][105620] Updated weights for policy 1, policy_version 1356307 (0.0007) [2023-12-27 01:15:11,369][105620] Updated weights for policy 1, policy_version 1356317 (0.0008) [2023-12-27 01:15:11,824][105692] Updated weights for policy 0, policy_version 1354321 (0.0009) [2023-12-27 01:15:11,884][105692] Updated weights for policy 0, policy_version 1354331 (0.0005) [2023-12-27 01:15:11,933][105692] Updated weights for policy 0, policy_version 1354341 (0.0006) [2023-12-27 01:15:11,995][105692] Updated weights for policy 0, policy_version 1354351 (0.0006) [2023-12-27 01:15:12,184][105620] Updated weights for policy 1, policy_version 1356327 (0.0009) [2023-12-27 01:15:12,248][105620] Updated weights for policy 1, policy_version 1356337 (0.0009) [2023-12-27 01:15:12,320][105620] Updated weights for policy 1, policy_version 1356347 (0.0010) [2023-12-27 01:15:12,681][105692] Updated weights for policy 0, policy_version 1354361 (0.0008) [2023-12-27 01:15:12,734][105692] Updated weights for policy 0, policy_version 1354371 (0.0009) [2023-12-27 01:15:12,788][105692] Updated weights for policy 0, policy_version 1354381 (0.0009) [2023-12-27 01:15:13,074][105620] Updated weights for policy 1, policy_version 1356357 (0.0009) [2023-12-27 01:15:13,132][105620] Updated weights for policy 1, policy_version 1356367 (0.0009) [2023-12-27 01:15:13,185][105620] Updated weights for policy 1, policy_version 1356377 (0.0008) [2023-12-27 01:15:13,577][105692] Updated weights for policy 0, policy_version 1354391 (0.0009) [2023-12-27 01:15:13,632][105692] Updated weights for policy 0, policy_version 1354401 (0.0009) [2023-12-27 01:15:13,693][105692] Updated weights for policy 0, policy_version 1354411 (0.0009) [2023-12-27 01:15:13,883][105620] Updated weights for policy 1, policy_version 1356387 (0.0009) [2023-12-27 01:15:13,944][105620] Updated weights for policy 1, policy_version 1356397 (0.0008) [2023-12-27 01:15:13,997][105620] Updated weights for policy 1, policy_version 1356407 (0.0010) [2023-12-27 01:15:14,418][105692] Updated weights for policy 0, policy_version 1354421 (0.0009) [2023-12-27 01:15:14,483][105692] Updated weights for policy 0, policy_version 1354431 (0.0009) [2023-12-27 01:15:14,541][105692] Updated weights for policy 0, policy_version 1354441 (0.0009) [2023-12-27 01:15:14,739][105620] Updated weights for policy 1, policy_version 1356417 (0.0009) [2023-12-27 01:15:14,800][105620] Updated weights for policy 1, policy_version 1356427 (0.0009) [2023-12-27 01:15:14,857][105620] Updated weights for policy 1, policy_version 1356437 (0.0010) [2023-12-27 01:15:14,907][105620] Updated weights for policy 1, policy_version 1356447 (0.0011) [2023-12-27 01:15:15,326][105692] Updated weights for policy 0, policy_version 1354451 (0.0009) [2023-12-27 01:15:15,377][105692] Updated weights for policy 0, policy_version 1354461 (0.0009) [2023-12-27 01:15:15,443][105692] Updated weights for policy 0, policy_version 1354471 (0.0009) [2023-12-27 01:15:15,653][105620] Updated weights for policy 1, policy_version 1356457 (0.0009) [2023-12-27 01:15:15,707][105620] Updated weights for policy 1, policy_version 1356467 (0.0008) [2023-12-27 01:15:15,761][105620] Updated weights for policy 1, policy_version 1356477 (0.0008) [2023-12-27 01:15:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 694099968. Throughput: 0: 9646.3, 1: 9929.2. Samples: 694072192. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:15:16,062][104569] Avg episode reward: [(0, '7721.567'), (1, '8366.776')] [2023-12-27 01:15:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001354480_346800128.pth... [2023-12-27 01:15:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001356480_347299840.pth... [2023-12-27 01:15:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001353360_346513408.pth [2023-12-27 01:15:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001355328_347004928.pth [2023-12-27 01:15:16,264][105692] Updated weights for policy 0, policy_version 1354481 (0.0009) [2023-12-27 01:15:16,318][105692] Updated weights for policy 0, policy_version 1354492 (0.0010) [2023-12-27 01:15:16,365][105692] Updated weights for policy 0, policy_version 1354503 (0.0009) [2023-12-27 01:15:16,375][105620] Updated weights for policy 1, policy_version 1356487 (0.0009) [2023-12-27 01:15:16,430][105620] Updated weights for policy 1, policy_version 1356497 (0.0008) [2023-12-27 01:15:16,481][105620] Updated weights for policy 1, policy_version 1356507 (0.0009) [2023-12-27 01:15:17,047][105620] Updated weights for policy 1, policy_version 1356517 (0.0006) [2023-12-27 01:15:17,106][105620] Updated weights for policy 1, policy_version 1356527 (0.0005) [2023-12-27 01:15:17,162][105620] Updated weights for policy 1, policy_version 1356537 (0.0005) [2023-12-27 01:15:17,274][105692] Updated weights for policy 0, policy_version 1354513 (0.0006) [2023-12-27 01:15:17,340][105692] Updated weights for policy 0, policy_version 1354523 (0.0010) [2023-12-27 01:15:17,398][105692] Updated weights for policy 0, policy_version 1354533 (0.0010) [2023-12-27 01:15:17,462][105692] Updated weights for policy 0, policy_version 1354543 (0.0010) [2023-12-27 01:15:17,694][105620] Updated weights for policy 1, policy_version 1356547 (0.0005) [2023-12-27 01:15:17,747][105620] Updated weights for policy 1, policy_version 1356557 (0.0006) [2023-12-27 01:15:17,794][105620] Updated weights for policy 1, policy_version 1356567 (0.0005) [2023-12-27 01:15:18,182][105692] Updated weights for policy 0, policy_version 1354553 (0.0011) [2023-12-27 01:15:18,247][105692] Updated weights for policy 0, policy_version 1354563 (0.0011) [2023-12-27 01:15:18,319][105692] Updated weights for policy 0, policy_version 1354573 (0.0006) [2023-12-27 01:15:18,428][105620] Updated weights for policy 1, policy_version 1356577 (0.0008) [2023-12-27 01:15:18,484][105620] Updated weights for policy 1, policy_version 1356587 (0.0009) [2023-12-27 01:15:18,537][105620] Updated weights for policy 1, policy_version 1356597 (0.0008) [2023-12-27 01:15:18,585][105620] Updated weights for policy 1, policy_version 1356607 (0.0008) [2023-12-27 01:15:19,017][105692] Updated weights for policy 0, policy_version 1354583 (0.0010) [2023-12-27 01:15:19,083][105692] Updated weights for policy 0, policy_version 1354593 (0.0009) [2023-12-27 01:15:19,152][105692] Updated weights for policy 0, policy_version 1354603 (0.0010) [2023-12-27 01:15:19,333][105620] Updated weights for policy 1, policy_version 1356617 (0.0007) [2023-12-27 01:15:19,399][105620] Updated weights for policy 1, policy_version 1356627 (0.0009) [2023-12-27 01:15:19,451][105620] Updated weights for policy 1, policy_version 1356637 (0.0010) [2023-12-27 01:15:19,821][105692] Updated weights for policy 0, policy_version 1354613 (0.0008) [2023-12-27 01:15:19,884][105692] Updated weights for policy 0, policy_version 1354623 (0.0008) [2023-12-27 01:15:19,940][105692] Updated weights for policy 0, policy_version 1354633 (0.0008) [2023-12-27 01:15:20,234][105620] Updated weights for policy 1, policy_version 1356647 (0.0011) [2023-12-27 01:15:20,300][105620] Updated weights for policy 1, policy_version 1356657 (0.0011) [2023-12-27 01:15:20,360][105620] Updated weights for policy 1, policy_version 1356667 (0.0008) [2023-12-27 01:15:20,561][105692] Updated weights for policy 0, policy_version 1354643 (0.0006) [2023-12-27 01:15:20,635][105692] Updated weights for policy 0, policy_version 1354653 (0.0008) [2023-12-27 01:15:20,696][105692] Updated weights for policy 0, policy_version 1354663 (0.0008) [2023-12-27 01:15:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 694198272. Throughput: 0: 9584.1, 1: 9970.6. Samples: 694189004. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:15:21,062][104569] Avg episode reward: [(0, '7814.945'), (1, '8360.356')] [2023-12-27 01:15:21,148][105620] Updated weights for policy 1, policy_version 1356677 (0.0010) [2023-12-27 01:15:21,202][105620] Updated weights for policy 1, policy_version 1356687 (0.0011) [2023-12-27 01:15:21,260][105620] Updated weights for policy 1, policy_version 1356697 (0.0011) [2023-12-27 01:15:21,439][105692] Updated weights for policy 0, policy_version 1354673 (0.0007) [2023-12-27 01:15:21,493][105692] Updated weights for policy 0, policy_version 1354683 (0.0007) [2023-12-27 01:15:21,552][105692] Updated weights for policy 0, policy_version 1354693 (0.0008) [2023-12-27 01:15:21,616][105692] Updated weights for policy 0, policy_version 1354703 (0.0008) [2023-12-27 01:15:22,015][105620] Updated weights for policy 1, policy_version 1356707 (0.0009) [2023-12-27 01:15:22,085][105620] Updated weights for policy 1, policy_version 1356717 (0.0006) [2023-12-27 01:15:22,155][105620] Updated weights for policy 1, policy_version 1356727 (0.0006) [2023-12-27 01:15:22,396][105692] Updated weights for policy 0, policy_version 1354713 (0.0009) [2023-12-27 01:15:22,460][105692] Updated weights for policy 0, policy_version 1354723 (0.0009) [2023-12-27 01:15:22,513][105692] Updated weights for policy 0, policy_version 1354733 (0.0010) [2023-12-27 01:15:22,812][105620] Updated weights for policy 1, policy_version 1356737 (0.0007) [2023-12-27 01:15:22,869][105620] Updated weights for policy 1, policy_version 1356747 (0.0008) [2023-12-27 01:15:22,931][105620] Updated weights for policy 1, policy_version 1356757 (0.0011) [2023-12-27 01:15:22,982][105620] Updated weights for policy 1, policy_version 1356767 (0.0011) [2023-12-27 01:15:23,287][105692] Updated weights for policy 0, policy_version 1354743 (0.0009) [2023-12-27 01:15:23,342][105692] Updated weights for policy 0, policy_version 1354753 (0.0008) [2023-12-27 01:15:23,394][105692] Updated weights for policy 0, policy_version 1354763 (0.0009) [2023-12-27 01:15:23,677][105620] Updated weights for policy 1, policy_version 1356777 (0.0009) [2023-12-27 01:15:23,732][105620] Updated weights for policy 1, policy_version 1356787 (0.0009) [2023-12-27 01:15:23,782][105620] Updated weights for policy 1, policy_version 1356797 (0.0009) [2023-12-27 01:15:24,093][105692] Updated weights for policy 0, policy_version 1354773 (0.0008) [2023-12-27 01:15:24,152][105692] Updated weights for policy 0, policy_version 1354783 (0.0005) [2023-12-27 01:15:24,204][105692] Updated weights for policy 0, policy_version 1354793 (0.0006) [2023-12-27 01:15:24,572][105620] Updated weights for policy 1, policy_version 1356807 (0.0008) [2023-12-27 01:15:24,635][105620] Updated weights for policy 1, policy_version 1356817 (0.0009) [2023-12-27 01:15:24,698][105620] Updated weights for policy 1, policy_version 1356827 (0.0009) [2023-12-27 01:15:24,800][105692] Updated weights for policy 0, policy_version 1354803 (0.0007) [2023-12-27 01:15:24,852][105692] Updated weights for policy 0, policy_version 1354813 (0.0009) [2023-12-27 01:15:24,912][105692] Updated weights for policy 0, policy_version 1354823 (0.0009) [2023-12-27 01:15:25,417][105620] Updated weights for policy 1, policy_version 1356837 (0.0009) [2023-12-27 01:15:25,482][105620] Updated weights for policy 1, policy_version 1356847 (0.0009) [2023-12-27 01:15:25,543][105620] Updated weights for policy 1, policy_version 1356857 (0.0009) [2023-12-27 01:15:25,643][105692] Updated weights for policy 0, policy_version 1354833 (0.0009) [2023-12-27 01:15:25,691][105692] Updated weights for policy 0, policy_version 1354843 (0.0008) [2023-12-27 01:15:25,747][105692] Updated weights for policy 0, policy_version 1354853 (0.0005) [2023-12-27 01:15:25,794][105692] Updated weights for policy 0, policy_version 1354863 (0.0009) [2023-12-27 01:15:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 694296576. Throughput: 0: 9615.6, 1: 9986.2. Samples: 694304836. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:15:26,062][104569] Avg episode reward: [(0, '7986.619'), (1, '8722.260')] [2023-12-27 01:15:26,287][105620] Updated weights for policy 1, policy_version 1356867 (0.0010) [2023-12-27 01:15:26,345][105620] Updated weights for policy 1, policy_version 1356877 (0.0009) [2023-12-27 01:15:26,404][105620] Updated weights for policy 1, policy_version 1356887 (0.0009) [2023-12-27 01:15:26,554][105692] Updated weights for policy 0, policy_version 1354873 (0.0008) [2023-12-27 01:15:26,612][105692] Updated weights for policy 0, policy_version 1354883 (0.0009) [2023-12-27 01:15:26,673][105692] Updated weights for policy 0, policy_version 1354893 (0.0009) [2023-12-27 01:15:27,124][105620] Updated weights for policy 1, policy_version 1356897 (0.0009) [2023-12-27 01:15:27,192][105620] Updated weights for policy 1, policy_version 1356907 (0.0009) [2023-12-27 01:15:27,249][105620] Updated weights for policy 1, policy_version 1356917 (0.0009) [2023-12-27 01:15:27,307][105620] Updated weights for policy 1, policy_version 1356927 (0.0009) [2023-12-27 01:15:27,405][105692] Updated weights for policy 0, policy_version 1354903 (0.0009) [2023-12-27 01:15:27,452][105692] Updated weights for policy 0, policy_version 1354913 (0.0009) [2023-12-27 01:15:27,506][105692] Updated weights for policy 0, policy_version 1354923 (0.0009) [2023-12-27 01:15:28,032][105620] Updated weights for policy 1, policy_version 1356937 (0.0009) [2023-12-27 01:15:28,094][105620] Updated weights for policy 1, policy_version 1356947 (0.0009) [2023-12-27 01:15:28,152][105620] Updated weights for policy 1, policy_version 1356957 (0.0009) [2023-12-27 01:15:28,272][105692] Updated weights for policy 0, policy_version 1354933 (0.0008) [2023-12-27 01:15:28,325][105692] Updated weights for policy 0, policy_version 1354943 (0.0008) [2023-12-27 01:15:28,382][105692] Updated weights for policy 0, policy_version 1354953 (0.0007) [2023-12-27 01:15:28,905][105620] Updated weights for policy 1, policy_version 1356967 (0.0009) [2023-12-27 01:15:28,962][105620] Updated weights for policy 1, policy_version 1356978 (0.0009) [2023-12-27 01:15:29,019][105620] Updated weights for policy 1, policy_version 1356988 (0.0010) [2023-12-27 01:15:29,112][105692] Updated weights for policy 0, policy_version 1354963 (0.0008) [2023-12-27 01:15:29,163][105692] Updated weights for policy 0, policy_version 1354973 (0.0005) [2023-12-27 01:15:29,215][105692] Updated weights for policy 0, policy_version 1354983 (0.0006) [2023-12-27 01:15:29,833][105692] Updated weights for policy 0, policy_version 1354994 (0.0008) [2023-12-27 01:15:29,870][105620] Updated weights for policy 1, policy_version 1356998 (0.0007) [2023-12-27 01:15:29,896][105692] Updated weights for policy 0, policy_version 1355004 (0.0011) [2023-12-27 01:15:29,931][105620] Updated weights for policy 1, policy_version 1357008 (0.0006) [2023-12-27 01:15:29,960][105692] Updated weights for policy 0, policy_version 1355014 (0.0008) [2023-12-27 01:15:29,992][105620] Updated weights for policy 1, policy_version 1357018 (0.0008) [2023-12-27 01:15:30,023][105692] Updated weights for policy 0, policy_version 1355024 (0.0006) [2023-12-27 01:15:30,552][105692] Updated weights for policy 0, policy_version 1355034 (0.0006) [2023-12-27 01:15:30,611][105692] Updated weights for policy 0, policy_version 1355044 (0.0005) [2023-12-27 01:15:30,668][105585] KL-divergence is very high: 108.4387 [2023-12-27 01:15:30,675][105692] Updated weights for policy 0, policy_version 1355054 (0.0008) [2023-12-27 01:15:30,836][105620] Updated weights for policy 1, policy_version 1357028 (0.0008) [2023-12-27 01:15:30,899][105620] Updated weights for policy 1, policy_version 1357038 (0.0005) [2023-12-27 01:15:30,957][105620] Updated weights for policy 1, policy_version 1357048 (0.0005) [2023-12-27 01:15:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 694394880. Throughput: 0: 9620.4, 1: 9967.8. Samples: 694361232. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:15:31,063][104569] Avg episode reward: [(0, '7979.192'), (1, '8996.126')] [2023-12-27 01:15:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001355056_346947584.pth... [2023-12-27 01:15:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001357056_347447296.pth... [2023-12-27 01:15:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001353936_346660864.pth [2023-12-27 01:15:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001355904_347152384.pth [2023-12-27 01:15:31,386][105692] Updated weights for policy 0, policy_version 1355064 (0.0008) [2023-12-27 01:15:31,456][105692] Updated weights for policy 0, policy_version 1355074 (0.0006) [2023-12-27 01:15:31,528][105692] Updated weights for policy 0, policy_version 1355084 (0.0006) [2023-12-27 01:15:31,595][105620] Updated weights for policy 1, policy_version 1357058 (0.0005) [2023-12-27 01:15:31,655][105620] Updated weights for policy 1, policy_version 1357068 (0.0006) [2023-12-27 01:15:31,713][105620] Updated weights for policy 1, policy_version 1357078 (0.0007) [2023-12-27 01:15:31,778][105620] Updated weights for policy 1, policy_version 1357088 (0.0009) [2023-12-27 01:15:32,244][105692] Updated weights for policy 0, policy_version 1355094 (0.0010) [2023-12-27 01:15:32,309][105692] Updated weights for policy 0, policy_version 1355104 (0.0009) [2023-12-27 01:15:32,368][105692] Updated weights for policy 0, policy_version 1355114 (0.0010) [2023-12-27 01:15:32,485][105620] Updated weights for policy 1, policy_version 1357098 (0.0008) [2023-12-27 01:15:32,537][105620] Updated weights for policy 1, policy_version 1357108 (0.0008) [2023-12-27 01:15:32,585][105620] Updated weights for policy 1, policy_version 1357118 (0.0005) [2023-12-27 01:15:33,088][105692] Updated weights for policy 0, policy_version 1355124 (0.0007) [2023-12-27 01:15:33,132][105620] Updated weights for policy 1, policy_version 1357128 (0.0005) [2023-12-27 01:15:33,144][105692] Updated weights for policy 0, policy_version 1355134 (0.0005) [2023-12-27 01:15:33,189][105620] Updated weights for policy 1, policy_version 1357138 (0.0005) [2023-12-27 01:15:33,208][105692] Updated weights for policy 0, policy_version 1355144 (0.0007) [2023-12-27 01:15:33,241][105620] Updated weights for policy 1, policy_version 1357148 (0.0005) [2023-12-27 01:15:33,906][105620] Updated weights for policy 1, policy_version 1357158 (0.0007) [2023-12-27 01:15:33,927][105692] Updated weights for policy 0, policy_version 1355154 (0.0009) [2023-12-27 01:15:33,961][105620] Updated weights for policy 1, policy_version 1357168 (0.0008) [2023-12-27 01:15:33,972][105692] Updated weights for policy 0, policy_version 1355164 (0.0008) [2023-12-27 01:15:34,006][105620] Updated weights for policy 1, policy_version 1357178 (0.0006) [2023-12-27 01:15:34,016][105692] Updated weights for policy 0, policy_version 1355174 (0.0010) [2023-12-27 01:15:34,059][105692] Updated weights for policy 0, policy_version 1355184 (0.0010) [2023-12-27 01:15:34,789][105620] Updated weights for policy 1, policy_version 1357188 (0.0007) [2023-12-27 01:15:34,844][105620] Updated weights for policy 1, policy_version 1357198 (0.0007) [2023-12-27 01:15:34,849][105692] Updated weights for policy 0, policy_version 1355194 (0.0010) [2023-12-27 01:15:34,897][105620] Updated weights for policy 1, policy_version 1357208 (0.0008) [2023-12-27 01:15:34,911][105692] Updated weights for policy 0, policy_version 1355204 (0.0010) [2023-12-27 01:15:34,960][105692] Updated weights for policy 0, policy_version 1355214 (0.0010) [2023-12-27 01:15:35,656][105620] Updated weights for policy 1, policy_version 1357218 (0.0009) [2023-12-27 01:15:35,696][105692] Updated weights for policy 0, policy_version 1355224 (0.0007) [2023-12-27 01:15:35,719][105620] Updated weights for policy 1, policy_version 1357228 (0.0009) [2023-12-27 01:15:35,753][105692] Updated weights for policy 0, policy_version 1355234 (0.0009) [2023-12-27 01:15:35,779][105620] Updated weights for policy 1, policy_version 1357238 (0.0009) [2023-12-27 01:15:35,811][105692] Updated weights for policy 0, policy_version 1355244 (0.0006) [2023-12-27 01:15:35,844][105620] Updated weights for policy 1, policy_version 1357248 (0.0009) [2023-12-27 01:15:36,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 694493184. Throughput: 0: 9669.9, 1: 9908.2. Samples: 694479620. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:15:36,063][104569] Avg episode reward: [(0, '7795.899'), (1, '8903.472')] [2023-12-27 01:15:36,554][105692] Updated weights for policy 0, policy_version 1355254 (0.0009) [2023-12-27 01:15:36,599][105620] Updated weights for policy 1, policy_version 1357258 (0.0007) [2023-12-27 01:15:36,619][105692] Updated weights for policy 0, policy_version 1355264 (0.0010) [2023-12-27 01:15:36,663][105620] Updated weights for policy 1, policy_version 1357268 (0.0007) [2023-12-27 01:15:36,684][105692] Updated weights for policy 0, policy_version 1355274 (0.0010) [2023-12-27 01:15:36,716][105620] Updated weights for policy 1, policy_version 1357278 (0.0007) [2023-12-27 01:15:37,419][105620] Updated weights for policy 1, policy_version 1357288 (0.0008) [2023-12-27 01:15:37,450][105692] Updated weights for policy 0, policy_version 1355284 (0.0007) [2023-12-27 01:15:37,481][105620] Updated weights for policy 1, policy_version 1357298 (0.0007) [2023-12-27 01:15:37,511][105692] Updated weights for policy 0, policy_version 1355294 (0.0007) [2023-12-27 01:15:37,538][105620] Updated weights for policy 1, policy_version 1357308 (0.0007) [2023-12-27 01:15:37,574][105692] Updated weights for policy 0, policy_version 1355304 (0.0007) [2023-12-27 01:15:38,297][105692] Updated weights for policy 0, policy_version 1355314 (0.0008) [2023-12-27 01:15:38,322][105620] Updated weights for policy 1, policy_version 1357318 (0.0009) [2023-12-27 01:15:38,367][105692] Updated weights for policy 0, policy_version 1355324 (0.0008) [2023-12-27 01:15:38,392][105620] Updated weights for policy 1, policy_version 1357328 (0.0008) [2023-12-27 01:15:38,438][105692] Updated weights for policy 0, policy_version 1355334 (0.0006) [2023-12-27 01:15:38,459][105620] Updated weights for policy 1, policy_version 1357338 (0.0008) [2023-12-27 01:15:38,495][105692] Updated weights for policy 0, policy_version 1355344 (0.0007) [2023-12-27 01:15:39,134][105620] Updated weights for policy 1, policy_version 1357348 (0.0009) [2023-12-27 01:15:39,201][105620] Updated weights for policy 1, policy_version 1357358 (0.0008) [2023-12-27 01:15:39,261][105692] Updated weights for policy 0, policy_version 1355354 (0.0010) [2023-12-27 01:15:39,262][105620] Updated weights for policy 1, policy_version 1357368 (0.0008) [2023-12-27 01:15:39,320][105692] Updated weights for policy 0, policy_version 1355364 (0.0007) [2023-12-27 01:15:39,382][105692] Updated weights for policy 0, policy_version 1355374 (0.0009) [2023-12-27 01:15:40,056][105620] Updated weights for policy 1, policy_version 1357378 (0.0007) [2023-12-27 01:15:40,125][105620] Updated weights for policy 1, policy_version 1357388 (0.0009) [2023-12-27 01:15:40,184][105692] Updated weights for policy 0, policy_version 1355384 (0.0007) [2023-12-27 01:15:40,193][105620] Updated weights for policy 1, policy_version 1357398 (0.0009) [2023-12-27 01:15:40,245][105692] Updated weights for policy 0, policy_version 1355394 (0.0008) [2023-12-27 01:15:40,257][105620] Updated weights for policy 1, policy_version 1357408 (0.0008) [2023-12-27 01:15:40,307][105692] Updated weights for policy 0, policy_version 1355404 (0.0006) [2023-12-27 01:15:40,929][105620] Updated weights for policy 1, policy_version 1357418 (0.0007) [2023-12-27 01:15:40,993][105620] Updated weights for policy 1, policy_version 1357428 (0.0007) [2023-12-27 01:15:41,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 694575104. Throughput: 0: 9510.8, 1: 9814.0. Samples: 694591788. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:15:41,062][104569] Avg episode reward: [(0, '7795.947'), (1, '8812.562')] [2023-12-27 01:15:41,063][105620] Updated weights for policy 1, policy_version 1357438 (0.0009) [2023-12-27 01:15:41,094][105692] Updated weights for policy 0, policy_version 1355414 (0.0007) [2023-12-27 01:15:41,152][105692] Updated weights for policy 0, policy_version 1355424 (0.0008) [2023-12-27 01:15:41,223][105692] Updated weights for policy 0, policy_version 1355434 (0.0007) [2023-12-27 01:15:41,781][105620] Updated weights for policy 1, policy_version 1357448 (0.0008) [2023-12-27 01:15:41,839][105620] Updated weights for policy 1, policy_version 1357458 (0.0005) [2023-12-27 01:15:41,895][105620] Updated weights for policy 1, policy_version 1357468 (0.0008) [2023-12-27 01:15:41,953][105692] Updated weights for policy 0, policy_version 1355444 (0.0009) [2023-12-27 01:15:42,021][105692] Updated weights for policy 0, policy_version 1355454 (0.0010) [2023-12-27 01:15:42,087][105692] Updated weights for policy 0, policy_version 1355464 (0.0009) [2023-12-27 01:15:42,504][105620] Updated weights for policy 1, policy_version 1357478 (0.0007) [2023-12-27 01:15:42,551][105620] Updated weights for policy 1, policy_version 1357488 (0.0005) [2023-12-27 01:15:42,607][105620] Updated weights for policy 1, policy_version 1357498 (0.0006) [2023-12-27 01:15:42,917][105692] Updated weights for policy 0, policy_version 1355474 (0.0010) [2023-12-27 01:15:42,974][105692] Updated weights for policy 0, policy_version 1355484 (0.0010) [2023-12-27 01:15:43,026][105692] Updated weights for policy 0, policy_version 1355494 (0.0009) [2023-12-27 01:15:43,083][105692] Updated weights for policy 0, policy_version 1355504 (0.0009) [2023-12-27 01:15:43,142][105620] Updated weights for policy 1, policy_version 1357508 (0.0006) [2023-12-27 01:15:43,193][105620] Updated weights for policy 1, policy_version 1357518 (0.0005) [2023-12-27 01:15:43,239][105620] Updated weights for policy 1, policy_version 1357528 (0.0005) [2023-12-27 01:15:43,866][105692] Updated weights for policy 0, policy_version 1355514 (0.0009) [2023-12-27 01:15:43,866][105585] KL-divergence is very high: 100.4417 [2023-12-27 01:15:43,872][105585] KL-divergence is very high: 189.3540 [2023-12-27 01:15:43,913][105585] KL-divergence is very high: 105.7719 [2023-12-27 01:15:43,919][105585] KL-divergence is very high: 168.4294 [2023-12-27 01:15:43,923][105692] Updated weights for policy 0, policy_version 1355524 (0.0009) [2023-12-27 01:15:43,962][105620] Updated weights for policy 1, policy_version 1357538 (0.0006) [2023-12-27 01:15:43,981][105692] Updated weights for policy 0, policy_version 1355534 (0.0008) [2023-12-27 01:15:44,025][105620] Updated weights for policy 1, policy_version 1357548 (0.0008) [2023-12-27 01:15:44,086][105620] Updated weights for policy 1, policy_version 1357558 (0.0009) [2023-12-27 01:15:44,148][105620] Updated weights for policy 1, policy_version 1357568 (0.0009) [2023-12-27 01:15:44,743][105692] Updated weights for policy 0, policy_version 1355544 (0.0008) [2023-12-27 01:15:44,793][105620] Updated weights for policy 1, policy_version 1357578 (0.0008) [2023-12-27 01:15:44,804][105692] Updated weights for policy 0, policy_version 1355554 (0.0008) [2023-12-27 01:15:44,850][105620] Updated weights for policy 1, policy_version 1357588 (0.0008) [2023-12-27 01:15:44,852][105692] Updated weights for policy 0, policy_version 1355564 (0.0005) [2023-12-27 01:15:44,910][105620] Updated weights for policy 1, policy_version 1357598 (0.0008) [2023-12-27 01:15:45,635][105692] Updated weights for policy 0, policy_version 1355574 (0.0006) [2023-12-27 01:15:45,640][105620] Updated weights for policy 1, policy_version 1357608 (0.0007) [2023-12-27 01:15:45,691][105692] Updated weights for policy 0, policy_version 1355584 (0.0005) [2023-12-27 01:15:45,699][105620] Updated weights for policy 1, policy_version 1357618 (0.0006) [2023-12-27 01:15:45,748][105692] Updated weights for policy 0, policy_version 1355594 (0.0006) [2023-12-27 01:15:45,769][105620] Updated weights for policy 1, policy_version 1357628 (0.0007) [2023-12-27 01:15:46,062][104569] Fps is (10 sec: 18840.8, 60 sec: 19524.1, 300 sec: 19549.7). Total num frames: 694681600. Throughput: 0: 9441.5, 1: 9841.0. Samples: 694650096. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:15:46,064][104569] Avg episode reward: [(0, '8356.920'), (1, '8722.200')] [2023-12-27 01:15:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001355600_347086848.pth... [2023-12-27 01:15:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001357632_347594752.pth... [2023-12-27 01:15:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001356480_347299840.pth [2023-12-27 01:15:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001354480_346800128.pth [2023-12-27 01:15:46,471][105692] Updated weights for policy 0, policy_version 1355604 (0.0007) [2023-12-27 01:15:46,473][105620] Updated weights for policy 1, policy_version 1357638 (0.0008) [2023-12-27 01:15:46,519][105692] Updated weights for policy 0, policy_version 1355614 (0.0005) [2023-12-27 01:15:46,536][105620] Updated weights for policy 1, policy_version 1357648 (0.0009) [2023-12-27 01:15:46,575][105692] Updated weights for policy 0, policy_version 1355624 (0.0007) [2023-12-27 01:15:46,586][105620] Updated weights for policy 1, policy_version 1357658 (0.0008) [2023-12-27 01:15:47,148][105620] Updated weights for policy 1, policy_version 1357668 (0.0006) [2023-12-27 01:15:47,196][105620] Updated weights for policy 1, policy_version 1357678 (0.0009) [2023-12-27 01:15:47,242][105620] Updated weights for policy 1, policy_version 1357688 (0.0008) [2023-12-27 01:15:47,415][105692] Updated weights for policy 0, policy_version 1355634 (0.0009) [2023-12-27 01:15:47,468][105692] Updated weights for policy 0, policy_version 1355644 (0.0009) [2023-12-27 01:15:47,520][105692] Updated weights for policy 0, policy_version 1355654 (0.0009) [2023-12-27 01:15:47,574][105692] Updated weights for policy 0, policy_version 1355664 (0.0010) [2023-12-27 01:15:47,894][105620] Updated weights for policy 1, policy_version 1357698 (0.0009) [2023-12-27 01:15:47,942][105620] Updated weights for policy 1, policy_version 1357708 (0.0010) [2023-12-27 01:15:48,007][105620] Updated weights for policy 1, policy_version 1357718 (0.0010) [2023-12-27 01:15:48,055][105620] Updated weights for policy 1, policy_version 1357728 (0.0010) [2023-12-27 01:15:48,407][105692] Updated weights for policy 0, policy_version 1355674 (0.0007) [2023-12-27 01:15:48,474][105692] Updated weights for policy 0, policy_version 1355684 (0.0007) [2023-12-27 01:15:48,535][105692] Updated weights for policy 0, policy_version 1355694 (0.0008) [2023-12-27 01:15:48,823][105620] Updated weights for policy 1, policy_version 1357738 (0.0010) [2023-12-27 01:15:48,885][105620] Updated weights for policy 1, policy_version 1357748 (0.0010) [2023-12-27 01:15:48,937][105620] Updated weights for policy 1, policy_version 1357758 (0.0010) [2023-12-27 01:15:49,275][105692] Updated weights for policy 0, policy_version 1355704 (0.0009) [2023-12-27 01:15:49,340][105692] Updated weights for policy 0, policy_version 1355714 (0.0008) [2023-12-27 01:15:49,406][105692] Updated weights for policy 0, policy_version 1355724 (0.0008) [2023-12-27 01:15:49,606][105620] Updated weights for policy 1, policy_version 1357768 (0.0007) [2023-12-27 01:15:49,658][105620] Updated weights for policy 1, policy_version 1357778 (0.0011) [2023-12-27 01:15:49,684][105586] KL-divergence is very high: 116.3712 [2023-12-27 01:15:49,711][105620] Updated weights for policy 1, policy_version 1357788 (0.0007) [2023-12-27 01:15:49,725][105586] KL-divergence is very high: 121.0818 [2023-12-27 01:15:50,198][105692] Updated weights for policy 0, policy_version 1355734 (0.0008) [2023-12-27 01:15:50,253][105692] Updated weights for policy 0, policy_version 1355744 (0.0008) [2023-12-27 01:15:50,312][105692] Updated weights for policy 0, policy_version 1355754 (0.0008) [2023-12-27 01:15:50,404][105620] Updated weights for policy 1, policy_version 1357798 (0.0008) [2023-12-27 01:15:50,448][105620] Updated weights for policy 1, policy_version 1357808 (0.0010) [2023-12-27 01:15:50,496][105620] Updated weights for policy 1, policy_version 1357818 (0.0010) [2023-12-27 01:15:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 694771712. Throughput: 0: 9389.7, 1: 9889.1. Samples: 694765856. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:15:51,062][104569] Avg episode reward: [(0, '7988.309'), (1, '8994.918')] [2023-12-27 01:15:51,098][105692] Updated weights for policy 0, policy_version 1355764 (0.0008) [2023-12-27 01:15:51,160][105692] Updated weights for policy 0, policy_version 1355774 (0.0008) [2023-12-27 01:15:51,212][105692] Updated weights for policy 0, policy_version 1355784 (0.0008) [2023-12-27 01:15:51,229][105620] Updated weights for policy 1, policy_version 1357828 (0.0010) [2023-12-27 01:15:51,292][105620] Updated weights for policy 1, policy_version 1357838 (0.0011) [2023-12-27 01:15:51,356][105620] Updated weights for policy 1, policy_version 1357848 (0.0012) [2023-12-27 01:15:52,004][105692] Updated weights for policy 0, policy_version 1355794 (0.0009) [2023-12-27 01:15:52,077][105692] Updated weights for policy 0, policy_version 1355804 (0.0009) [2023-12-27 01:15:52,080][105620] Updated weights for policy 1, policy_version 1357858 (0.0008) [2023-12-27 01:15:52,141][105620] Updated weights for policy 1, policy_version 1357868 (0.0008) [2023-12-27 01:15:52,145][105692] Updated weights for policy 0, policy_version 1355814 (0.0008) [2023-12-27 01:15:52,192][105692] Updated weights for policy 0, policy_version 1355824 (0.0008) [2023-12-27 01:15:52,200][105620] Updated weights for policy 1, policy_version 1357878 (0.0008) [2023-12-27 01:15:52,262][105620] Updated weights for policy 1, policy_version 1357888 (0.0010) [2023-12-27 01:15:52,816][105692] Updated weights for policy 0, policy_version 1355834 (0.0009) [2023-12-27 01:15:52,882][105692] Updated weights for policy 0, policy_version 1355844 (0.0010) [2023-12-27 01:15:52,943][105692] Updated weights for policy 0, policy_version 1355854 (0.0009) [2023-12-27 01:15:53,019][105620] Updated weights for policy 1, policy_version 1357898 (0.0005) [2023-12-27 01:15:53,072][105620] Updated weights for policy 1, policy_version 1357908 (0.0006) [2023-12-27 01:15:53,122][105620] Updated weights for policy 1, policy_version 1357918 (0.0005) [2023-12-27 01:15:53,663][105620] Updated weights for policy 1, policy_version 1357928 (0.0008) [2023-12-27 01:15:53,664][105692] Updated weights for policy 0, policy_version 1355864 (0.0006) [2023-12-27 01:15:53,720][105692] Updated weights for policy 0, policy_version 1355874 (0.0006) [2023-12-27 01:15:53,724][105620] Updated weights for policy 1, policy_version 1357938 (0.0005) [2023-12-27 01:15:53,782][105620] Updated weights for policy 1, policy_version 1357948 (0.0009) [2023-12-27 01:15:53,787][105692] Updated weights for policy 0, policy_version 1355884 (0.0005) [2023-12-27 01:15:54,455][105692] Updated weights for policy 0, policy_version 1355894 (0.0006) [2023-12-27 01:15:54,460][105620] Updated weights for policy 1, policy_version 1357959 (0.0008) [2023-12-27 01:15:54,508][105692] Updated weights for policy 0, policy_version 1355904 (0.0007) [2023-12-27 01:15:54,517][105620] Updated weights for policy 1, policy_version 1357969 (0.0008) [2023-12-27 01:15:54,565][105692] Updated weights for policy 0, policy_version 1355914 (0.0006) [2023-12-27 01:15:54,580][105620] Updated weights for policy 1, policy_version 1357979 (0.0007) [2023-12-27 01:15:55,203][105620] Updated weights for policy 1, policy_version 1357989 (0.0008) [2023-12-27 01:15:55,262][105620] Updated weights for policy 1, policy_version 1357999 (0.0009) [2023-12-27 01:15:55,325][105620] Updated weights for policy 1, policy_version 1358009 (0.0009) [2023-12-27 01:15:55,398][105692] Updated weights for policy 0, policy_version 1355924 (0.0007) [2023-12-27 01:15:55,471][105692] Updated weights for policy 0, policy_version 1355934 (0.0009) [2023-12-27 01:15:55,539][105692] Updated weights for policy 0, policy_version 1355944 (0.0009) [2023-12-27 01:15:55,947][105620] Updated weights for policy 1, policy_version 1358019 (0.0008) [2023-12-27 01:15:55,993][105620] Updated weights for policy 1, policy_version 1358029 (0.0005) [2023-12-27 01:15:56,052][105620] Updated weights for policy 1, policy_version 1358039 (0.0008) [2023-12-27 01:15:56,062][104569] Fps is (10 sec: 18842.8, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 694870016. Throughput: 0: 9397.1, 1: 9893.9. Samples: 694883888. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:15:56,062][104569] Avg episode reward: [(0, '8167.139'), (1, '9174.495')] [2023-12-27 01:15:56,359][105692] Updated weights for policy 0, policy_version 1355954 (0.0009) [2023-12-27 01:15:56,415][105692] Updated weights for policy 0, policy_version 1355964 (0.0008) [2023-12-27 01:15:56,475][105692] Updated weights for policy 0, policy_version 1355974 (0.0009) [2023-12-27 01:15:56,527][105692] Updated weights for policy 0, policy_version 1355984 (0.0010) [2023-12-27 01:15:56,699][105620] Updated weights for policy 1, policy_version 1358049 (0.0008) [2023-12-27 01:15:56,747][105620] Updated weights for policy 1, policy_version 1358059 (0.0005) [2023-12-27 01:15:56,798][105620] Updated weights for policy 1, policy_version 1358069 (0.0009) [2023-12-27 01:15:56,858][105620] Updated weights for policy 1, policy_version 1358079 (0.0008) [2023-12-27 01:15:57,284][105692] Updated weights for policy 0, policy_version 1355994 (0.0009) [2023-12-27 01:15:57,336][105692] Updated weights for policy 0, policy_version 1356004 (0.0010) [2023-12-27 01:15:57,397][105692] Updated weights for policy 0, policy_version 1356014 (0.0010) [2023-12-27 01:15:57,605][105620] Updated weights for policy 1, policy_version 1358089 (0.0008) [2023-12-27 01:15:57,663][105620] Updated weights for policy 1, policy_version 1358099 (0.0008) [2023-12-27 01:15:57,718][105620] Updated weights for policy 1, policy_version 1358109 (0.0007) [2023-12-27 01:15:58,143][105692] Updated weights for policy 0, policy_version 1356024 (0.0011) [2023-12-27 01:15:58,215][105692] Updated weights for policy 0, policy_version 1356034 (0.0010) [2023-12-27 01:15:58,282][105692] Updated weights for policy 0, policy_version 1356044 (0.0011) [2023-12-27 01:15:58,502][105620] Updated weights for policy 1, policy_version 1358119 (0.0007) [2023-12-27 01:15:58,568][105620] Updated weights for policy 1, policy_version 1358129 (0.0007) [2023-12-27 01:15:58,641][105620] Updated weights for policy 1, policy_version 1358139 (0.0007) [2023-12-27 01:15:59,154][105692] Updated weights for policy 0, policy_version 1356054 (0.0009) [2023-12-27 01:15:59,223][105692] Updated weights for policy 0, policy_version 1356064 (0.0009) [2023-12-27 01:15:59,298][105692] Updated weights for policy 0, policy_version 1356074 (0.0009) [2023-12-27 01:15:59,471][105620] Updated weights for policy 1, policy_version 1358149 (0.0007) [2023-12-27 01:15:59,535][105620] Updated weights for policy 1, policy_version 1358159 (0.0006) [2023-12-27 01:15:59,598][105620] Updated weights for policy 1, policy_version 1358169 (0.0008) [2023-12-27 01:16:00,193][105692] Updated weights for policy 0, policy_version 1356084 (0.0009) [2023-12-27 01:16:00,221][105620] Updated weights for policy 1, policy_version 1358179 (0.0007) [2023-12-27 01:16:00,257][105692] Updated weights for policy 0, policy_version 1356094 (0.0009) [2023-12-27 01:16:00,280][105620] Updated weights for policy 1, policy_version 1358189 (0.0007) [2023-12-27 01:16:00,322][105692] Updated weights for policy 0, policy_version 1356104 (0.0007) [2023-12-27 01:16:00,342][105620] Updated weights for policy 1, policy_version 1358199 (0.0007) [2023-12-27 01:16:00,975][105692] Updated weights for policy 0, policy_version 1356114 (0.0007) [2023-12-27 01:16:01,034][105692] Updated weights for policy 0, policy_version 1356124 (0.0010) [2023-12-27 01:16:01,042][105620] Updated weights for policy 1, policy_version 1358209 (0.0007) [2023-12-27 01:16:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.7, 300 sec: 19522.0). Total num frames: 694960128. Throughput: 0: 9363.8, 1: 9909.3. Samples: 694939484. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:16:01,062][104569] Avg episode reward: [(0, '8078.700'), (1, '9266.985')] [2023-12-27 01:16:01,092][105692] Updated weights for policy 0, policy_version 1356134 (0.0010) [2023-12-27 01:16:01,097][105620] Updated weights for policy 1, policy_version 1358219 (0.0008) [2023-12-27 01:16:01,146][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001356144_347226112.pth... [2023-12-27 01:16:01,148][105692] Updated weights for policy 0, policy_version 1356144 (0.0008) [2023-12-27 01:16:01,150][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001355056_346947584.pth [2023-12-27 01:16:01,154][105620] Updated weights for policy 1, policy_version 1358229 (0.0009) [2023-12-27 01:16:01,207][105620] Updated weights for policy 1, policy_version 1358239 (0.0009) [2023-12-27 01:16:01,211][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001358240_347750400.pth... [2023-12-27 01:16:01,215][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001357056_347447296.pth [2023-12-27 01:16:01,898][105692] Updated weights for policy 0, policy_version 1356154 (0.0007) [2023-12-27 01:16:01,957][105692] Updated weights for policy 0, policy_version 1356164 (0.0006) [2023-12-27 01:16:01,972][105620] Updated weights for policy 1, policy_version 1358249 (0.0008) [2023-12-27 01:16:02,018][105692] Updated weights for policy 0, policy_version 1356174 (0.0007) [2023-12-27 01:16:02,026][105620] Updated weights for policy 1, policy_version 1358259 (0.0008) [2023-12-27 01:16:02,080][105620] Updated weights for policy 1, policy_version 1358269 (0.0009) [2023-12-27 01:16:02,705][105692] Updated weights for policy 0, policy_version 1356184 (0.0010) [2023-12-27 01:16:02,768][105692] Updated weights for policy 0, policy_version 1356194 (0.0008) [2023-12-27 01:16:02,830][105692] Updated weights for policy 0, policy_version 1356204 (0.0006) [2023-12-27 01:16:02,876][105620] Updated weights for policy 1, policy_version 1358279 (0.0010) [2023-12-27 01:16:02,941][105620] Updated weights for policy 1, policy_version 1358289 (0.0010) [2023-12-27 01:16:02,994][105620] Updated weights for policy 1, policy_version 1358299 (0.0008) [2023-12-27 01:16:03,500][105692] Updated weights for policy 0, policy_version 1356214 (0.0008) [2023-12-27 01:16:03,544][105692] Updated weights for policy 0, policy_version 1356224 (0.0010) [2023-12-27 01:16:03,596][105692] Updated weights for policy 0, policy_version 1356234 (0.0010) [2023-12-27 01:16:03,788][105620] Updated weights for policy 1, policy_version 1358309 (0.0009) [2023-12-27 01:16:03,848][105620] Updated weights for policy 1, policy_version 1358319 (0.0008) [2023-12-27 01:16:03,915][105620] Updated weights for policy 1, policy_version 1358329 (0.0008) [2023-12-27 01:16:04,375][105692] Updated weights for policy 0, policy_version 1356244 (0.0010) [2023-12-27 01:16:04,435][105692] Updated weights for policy 0, policy_version 1356254 (0.0011) [2023-12-27 01:16:04,494][105692] Updated weights for policy 0, policy_version 1356264 (0.0010) [2023-12-27 01:16:04,703][105620] Updated weights for policy 1, policy_version 1358339 (0.0009) [2023-12-27 01:16:04,755][105620] Updated weights for policy 1, policy_version 1358349 (0.0008) [2023-12-27 01:16:04,804][105620] Updated weights for policy 1, policy_version 1358359 (0.0008) [2023-12-27 01:16:05,251][105692] Updated weights for policy 0, policy_version 1356274 (0.0010) [2023-12-27 01:16:05,317][105692] Updated weights for policy 0, policy_version 1356284 (0.0011) [2023-12-27 01:16:05,372][105692] Updated weights for policy 0, policy_version 1356294 (0.0010) [2023-12-27 01:16:05,403][105620] Updated weights for policy 1, policy_version 1358369 (0.0007) [2023-12-27 01:16:05,427][105692] Updated weights for policy 0, policy_version 1356304 (0.0010) [2023-12-27 01:16:05,449][105620] Updated weights for policy 1, policy_version 1358379 (0.0005) [2023-12-27 01:16:05,510][105620] Updated weights for policy 1, policy_version 1358389 (0.0005) [2023-12-27 01:16:05,560][105620] Updated weights for policy 1, policy_version 1358399 (0.0005) [2023-12-27 01:16:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.6, 300 sec: 19521.9). Total num frames: 695058432. Throughput: 0: 9375.2, 1: 9772.5. Samples: 695050652. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:16:06,062][104569] Avg episode reward: [(0, '8076.242'), (1, '9177.814')] [2023-12-27 01:16:06,099][105620] Updated weights for policy 1, policy_version 1358409 (0.0009) [2023-12-27 01:16:06,160][105620] Updated weights for policy 1, policy_version 1358419 (0.0008) [2023-12-27 01:16:06,175][105692] Updated weights for policy 0, policy_version 1356314 (0.0010) [2023-12-27 01:16:06,221][105620] Updated weights for policy 1, policy_version 1358429 (0.0011) [2023-12-27 01:16:06,239][105692] Updated weights for policy 0, policy_version 1356324 (0.0010) [2023-12-27 01:16:06,301][105692] Updated weights for policy 0, policy_version 1356334 (0.0011) [2023-12-27 01:16:06,952][105692] Updated weights for policy 0, policy_version 1356344 (0.0009) [2023-12-27 01:16:06,982][105620] Updated weights for policy 1, policy_version 1358439 (0.0011) [2023-12-27 01:16:07,012][105692] Updated weights for policy 0, policy_version 1356354 (0.0011) [2023-12-27 01:16:07,036][105586] KL-divergence is very high: 119.5900 [2023-12-27 01:16:07,037][105620] Updated weights for policy 1, policy_version 1358449 (0.0010) [2023-12-27 01:16:07,065][105692] Updated weights for policy 0, policy_version 1356364 (0.0010) [2023-12-27 01:16:07,084][105586] KL-divergence is very high: 120.9083 [2023-12-27 01:16:07,096][105620] Updated weights for policy 1, policy_version 1358459 (0.0010) [2023-12-27 01:16:07,756][105692] Updated weights for policy 0, policy_version 1356374 (0.0007) [2023-12-27 01:16:07,817][105692] Updated weights for policy 0, policy_version 1356384 (0.0005) [2023-12-27 01:16:07,868][105620] Updated weights for policy 1, policy_version 1358469 (0.0010) [2023-12-27 01:16:07,878][105692] Updated weights for policy 0, policy_version 1356394 (0.0005) [2023-12-27 01:16:07,926][105620] Updated weights for policy 1, policy_version 1358479 (0.0009) [2023-12-27 01:16:07,979][105620] Updated weights for policy 1, policy_version 1358489 (0.0009) [2023-12-27 01:16:08,511][105692] Updated weights for policy 0, policy_version 1356404 (0.0007) [2023-12-27 01:16:08,567][105692] Updated weights for policy 0, policy_version 1356414 (0.0011) [2023-12-27 01:16:08,603][105620] Updated weights for policy 1, policy_version 1358499 (0.0009) [2023-12-27 01:16:08,623][105692] Updated weights for policy 0, policy_version 1356424 (0.0011) [2023-12-27 01:16:08,671][105620] Updated weights for policy 1, policy_version 1358509 (0.0008) [2023-12-27 01:16:08,730][105620] Updated weights for policy 1, policy_version 1358519 (0.0008) [2023-12-27 01:16:09,323][105692] Updated weights for policy 0, policy_version 1356434 (0.0011) [2023-12-27 01:16:09,399][105692] Updated weights for policy 0, policy_version 1356444 (0.0011) [2023-12-27 01:16:09,464][105692] Updated weights for policy 0, policy_version 1356454 (0.0012) [2023-12-27 01:16:09,488][105620] Updated weights for policy 1, policy_version 1358529 (0.0008) [2023-12-27 01:16:09,523][105692] Updated weights for policy 0, policy_version 1356464 (0.0008) [2023-12-27 01:16:09,544][105620] Updated weights for policy 1, policy_version 1358539 (0.0006) [2023-12-27 01:16:09,610][105620] Updated weights for policy 1, policy_version 1358549 (0.0008) [2023-12-27 01:16:09,676][105620] Updated weights for policy 1, policy_version 1358559 (0.0007) [2023-12-27 01:16:10,225][105692] Updated weights for policy 0, policy_version 1356474 (0.0006) [2023-12-27 01:16:10,291][105692] Updated weights for policy 0, policy_version 1356484 (0.0008) [2023-12-27 01:16:10,355][105692] Updated weights for policy 0, policy_version 1356494 (0.0012) [2023-12-27 01:16:10,439][105620] Updated weights for policy 1, policy_version 1358569 (0.0009) [2023-12-27 01:16:10,516][105620] Updated weights for policy 1, policy_version 1358579 (0.0009) [2023-12-27 01:16:10,583][105620] Updated weights for policy 1, policy_version 1358589 (0.0009) [2023-12-27 01:16:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 695156736. Throughput: 0: 9370.4, 1: 9852.1. Samples: 695169848. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:16:11,063][104569] Avg episode reward: [(0, '8351.896'), (1, '8998.485')] [2023-12-27 01:16:11,075][105692] Updated weights for policy 0, policy_version 1356504 (0.0009) [2023-12-27 01:16:11,147][105692] Updated weights for policy 0, policy_version 1356514 (0.0008) [2023-12-27 01:16:11,210][105692] Updated weights for policy 0, policy_version 1356524 (0.0009) [2023-12-27 01:16:11,349][105620] Updated weights for policy 1, policy_version 1358599 (0.0008) [2023-12-27 01:16:11,412][105620] Updated weights for policy 1, policy_version 1358609 (0.0008) [2023-12-27 01:16:11,475][105620] Updated weights for policy 1, policy_version 1358619 (0.0008) [2023-12-27 01:16:11,971][105692] Updated weights for policy 0, policy_version 1356534 (0.0009) [2023-12-27 01:16:12,024][105692] Updated weights for policy 0, policy_version 1356544 (0.0008) [2023-12-27 01:16:12,080][105692] Updated weights for policy 0, policy_version 1356554 (0.0008) [2023-12-27 01:16:12,271][105620] Updated weights for policy 1, policy_version 1358629 (0.0008) [2023-12-27 01:16:12,344][105620] Updated weights for policy 1, policy_version 1358639 (0.0008) [2023-12-27 01:16:12,418][105620] Updated weights for policy 1, policy_version 1358649 (0.0008) [2023-12-27 01:16:12,889][105692] Updated weights for policy 0, policy_version 1356564 (0.0009) [2023-12-27 01:16:12,955][105692] Updated weights for policy 0, policy_version 1356574 (0.0009) [2023-12-27 01:16:13,010][105692] Updated weights for policy 0, policy_version 1356584 (0.0010) [2023-12-27 01:16:13,130][105620] Updated weights for policy 1, policy_version 1358659 (0.0008) [2023-12-27 01:16:13,195][105620] Updated weights for policy 1, policy_version 1358669 (0.0006) [2023-12-27 01:16:13,268][105620] Updated weights for policy 1, policy_version 1358679 (0.0005) [2023-12-27 01:16:13,748][105692] Updated weights for policy 0, policy_version 1356594 (0.0009) [2023-12-27 01:16:13,805][105692] Updated weights for policy 0, policy_version 1356604 (0.0008) [2023-12-27 01:16:13,862][105692] Updated weights for policy 0, policy_version 1356614 (0.0005) [2023-12-27 01:16:13,922][105692] Updated weights for policy 0, policy_version 1356624 (0.0006) [2023-12-27 01:16:13,972][105620] Updated weights for policy 1, policy_version 1358689 (0.0007) [2023-12-27 01:16:14,025][105620] Updated weights for policy 1, policy_version 1358699 (0.0009) [2023-12-27 01:16:14,071][105620] Updated weights for policy 1, policy_version 1358709 (0.0008) [2023-12-27 01:16:14,121][105620] Updated weights for policy 1, policy_version 1358719 (0.0009) [2023-12-27 01:16:14,623][105692] Updated weights for policy 0, policy_version 1356634 (0.0007) [2023-12-27 01:16:14,674][105692] Updated weights for policy 0, policy_version 1356644 (0.0006) [2023-12-27 01:16:14,723][105692] Updated weights for policy 0, policy_version 1356654 (0.0005) [2023-12-27 01:16:14,947][105620] Updated weights for policy 1, policy_version 1358729 (0.0009) [2023-12-27 01:16:15,015][105620] Updated weights for policy 1, policy_version 1358739 (0.0008) [2023-12-27 01:16:15,082][105620] Updated weights for policy 1, policy_version 1358749 (0.0008) [2023-12-27 01:16:15,377][105692] Updated weights for policy 0, policy_version 1356664 (0.0007) [2023-12-27 01:16:15,441][105692] Updated weights for policy 0, policy_version 1356674 (0.0009) [2023-12-27 01:16:15,500][105692] Updated weights for policy 0, policy_version 1356684 (0.0010) [2023-12-27 01:16:15,934][105620] Updated weights for policy 1, policy_version 1358759 (0.0008) [2023-12-27 01:16:16,003][105620] Updated weights for policy 1, policy_version 1358769 (0.0009) [2023-12-27 01:16:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.7, 300 sec: 19522.0). Total num frames: 695246848. Throughput: 0: 9362.6, 1: 9834.7. Samples: 695225108. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:16:16,063][104569] Avg episode reward: [(0, '8449.417'), (1, '8995.062')] [2023-12-27 01:16:16,068][105620] Updated weights for policy 1, policy_version 1358779 (0.0007) [2023-12-27 01:16:16,078][105692] Updated weights for policy 0, policy_version 1356694 (0.0010) [2023-12-27 01:16:16,092][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001358784_347889664.pth... [2023-12-27 01:16:16,096][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001357632_347594752.pth [2023-12-27 01:16:16,170][105692] Updated weights for policy 0, policy_version 1356704 (0.0011) [2023-12-27 01:16:16,207][105585] KL-divergence is very high: 106.4479 [2023-12-27 01:16:16,236][105692] Updated weights for policy 0, policy_version 1356714 (0.0010) [2023-12-27 01:16:16,268][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001356720_347373568.pth... [2023-12-27 01:16:16,271][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001355600_347086848.pth [2023-12-27 01:16:16,759][105620] Updated weights for policy 1, policy_version 1358789 (0.0006) [2023-12-27 01:16:16,807][105620] Updated weights for policy 1, policy_version 1358799 (0.0005) [2023-12-27 01:16:16,854][105620] Updated weights for policy 1, policy_version 1358809 (0.0005) [2023-12-27 01:16:16,939][105692] Updated weights for policy 0, policy_version 1356724 (0.0010) [2023-12-27 01:16:16,998][105692] Updated weights for policy 0, policy_version 1356734 (0.0010) [2023-12-27 01:16:17,046][105692] Updated weights for policy 0, policy_version 1356744 (0.0010) [2023-12-27 01:16:17,384][105620] Updated weights for policy 1, policy_version 1358819 (0.0005) [2023-12-27 01:16:17,431][105620] Updated weights for policy 1, policy_version 1358829 (0.0005) [2023-12-27 01:16:17,487][105620] Updated weights for policy 1, policy_version 1358839 (0.0005) [2023-12-27 01:16:17,761][105692] Updated weights for policy 0, policy_version 1356754 (0.0010) [2023-12-27 01:16:17,812][105692] Updated weights for policy 0, policy_version 1356764 (0.0010) [2023-12-27 01:16:17,823][105585] KL-divergence is very high: 158.6021 [2023-12-27 01:16:17,846][105585] KL-divergence is very high: 191.0899 [2023-12-27 01:16:17,863][105585] KL-divergence is very high: 100.1939 [2023-12-27 01:16:17,870][105692] Updated weights for policy 0, policy_version 1356774 (0.0010) [2023-12-27 01:16:17,870][105585] KL-divergence is very high: 280.1606 [2023-12-27 01:16:17,894][105585] KL-divergence is very high: 241.6045 [2023-12-27 01:16:17,916][105585] KL-divergence is very high: 101.8216 [2023-12-27 01:16:17,921][105585] KL-divergence is very high: 279.8959 [2023-12-27 01:16:17,931][105692] Updated weights for policy 0, policy_version 1356784 (0.0005) [2023-12-27 01:16:18,020][105620] Updated weights for policy 1, policy_version 1358849 (0.0006) [2023-12-27 01:16:18,088][105620] Updated weights for policy 1, policy_version 1358859 (0.0010) [2023-12-27 01:16:18,135][105620] Updated weights for policy 1, policy_version 1358869 (0.0010) [2023-12-27 01:16:18,193][105620] Updated weights for policy 1, policy_version 1358879 (0.0010) [2023-12-27 01:16:18,602][105692] Updated weights for policy 0, policy_version 1356794 (0.0006) [2023-12-27 01:16:18,669][105692] Updated weights for policy 0, policy_version 1356804 (0.0009) [2023-12-27 01:16:18,725][105692] Updated weights for policy 0, policy_version 1356814 (0.0008) [2023-12-27 01:16:18,900][105620] Updated weights for policy 1, policy_version 1358889 (0.0011) [2023-12-27 01:16:18,967][105620] Updated weights for policy 1, policy_version 1358899 (0.0011) [2023-12-27 01:16:19,031][105620] Updated weights for policy 1, policy_version 1358909 (0.0009) [2023-12-27 01:16:19,315][105692] Updated weights for policy 0, policy_version 1356824 (0.0008) [2023-12-27 01:16:19,389][105692] Updated weights for policy 0, policy_version 1356834 (0.0008) [2023-12-27 01:16:19,453][105692] Updated weights for policy 0, policy_version 1356844 (0.0008) [2023-12-27 01:16:19,744][105620] Updated weights for policy 1, policy_version 1358919 (0.0011) [2023-12-27 01:16:19,808][105620] Updated weights for policy 1, policy_version 1358929 (0.0008) [2023-12-27 01:16:19,878][105620] Updated weights for policy 1, policy_version 1358939 (0.0011) [2023-12-27 01:16:20,101][105692] Updated weights for policy 0, policy_version 1356854 (0.0009) [2023-12-27 01:16:20,154][105692] Updated weights for policy 0, policy_version 1356864 (0.0011) [2023-12-27 01:16:20,210][105692] Updated weights for policy 0, policy_version 1356874 (0.0011) [2023-12-27 01:16:20,589][105620] Updated weights for policy 1, policy_version 1358949 (0.0010) [2023-12-27 01:16:20,649][105620] Updated weights for policy 1, policy_version 1358959 (0.0009) [2023-12-27 01:16:20,705][105620] Updated weights for policy 1, policy_version 1358969 (0.0010) [2023-12-27 01:16:20,944][105692] Updated weights for policy 0, policy_version 1356884 (0.0011) [2023-12-27 01:16:21,007][105692] Updated weights for policy 0, policy_version 1356894 (0.0008) [2023-12-27 01:16:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 695353344. Throughput: 0: 9394.5, 1: 9867.9. Samples: 695346428. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:16:21,063][104569] Avg episode reward: [(0, '8266.308'), (1, '9082.312')] [2023-12-27 01:16:21,072][105692] Updated weights for policy 0, policy_version 1356904 (0.0008) [2023-12-27 01:16:21,509][105620] Updated weights for policy 1, policy_version 1358979 (0.0010) [2023-12-27 01:16:21,575][105620] Updated weights for policy 1, policy_version 1358989 (0.0008) [2023-12-27 01:16:21,638][105620] Updated weights for policy 1, policy_version 1358999 (0.0008) [2023-12-27 01:16:21,857][105692] Updated weights for policy 0, policy_version 1356914 (0.0006) [2023-12-27 01:16:21,914][105692] Updated weights for policy 0, policy_version 1356924 (0.0006) [2023-12-27 01:16:21,974][105692] Updated weights for policy 0, policy_version 1356934 (0.0009) [2023-12-27 01:16:22,029][105692] Updated weights for policy 0, policy_version 1356944 (0.0009) [2023-12-27 01:16:22,392][105620] Updated weights for policy 1, policy_version 1359009 (0.0009) [2023-12-27 01:16:22,456][105620] Updated weights for policy 1, policy_version 1359019 (0.0006) [2023-12-27 01:16:22,516][105620] Updated weights for policy 1, policy_version 1359029 (0.0008) [2023-12-27 01:16:22,569][105620] Updated weights for policy 1, policy_version 1359039 (0.0008) [2023-12-27 01:16:22,780][105692] Updated weights for policy 0, policy_version 1356954 (0.0008) [2023-12-27 01:16:22,846][105692] Updated weights for policy 0, policy_version 1356964 (0.0006) [2023-12-27 01:16:22,916][105692] Updated weights for policy 0, policy_version 1356974 (0.0005) [2023-12-27 01:16:23,363][105620] Updated weights for policy 1, policy_version 1359049 (0.0009) [2023-12-27 01:16:23,415][105620] Updated weights for policy 1, policy_version 1359059 (0.0009) [2023-12-27 01:16:23,461][105620] Updated weights for policy 1, policy_version 1359069 (0.0009) [2023-12-27 01:16:23,506][105692] Updated weights for policy 0, policy_version 1356984 (0.0009) [2023-12-27 01:16:23,564][105692] Updated weights for policy 0, policy_version 1356994 (0.0009) [2023-12-27 01:16:23,623][105692] Updated weights for policy 0, policy_version 1357004 (0.0008) [2023-12-27 01:16:24,234][105620] Updated weights for policy 1, policy_version 1359079 (0.0008) [2023-12-27 01:16:24,293][105620] Updated weights for policy 1, policy_version 1359089 (0.0009) [2023-12-27 01:16:24,343][105620] Updated weights for policy 1, policy_version 1359099 (0.0009) [2023-12-27 01:16:24,380][105692] Updated weights for policy 0, policy_version 1357014 (0.0007) [2023-12-27 01:16:24,431][105692] Updated weights for policy 0, policy_version 1357024 (0.0008) [2023-12-27 01:16:24,482][105692] Updated weights for policy 0, policy_version 1357034 (0.0009) [2023-12-27 01:16:25,061][105620] Updated weights for policy 1, policy_version 1359109 (0.0008) [2023-12-27 01:16:25,108][105620] Updated weights for policy 1, policy_version 1359119 (0.0009) [2023-12-27 01:16:25,154][105620] Updated weights for policy 1, policy_version 1359129 (0.0008) [2023-12-27 01:16:25,274][105692] Updated weights for policy 0, policy_version 1357044 (0.0008) [2023-12-27 01:16:25,331][105692] Updated weights for policy 0, policy_version 1357054 (0.0009) [2023-12-27 01:16:25,386][105692] Updated weights for policy 0, policy_version 1357064 (0.0009) [2023-12-27 01:16:25,916][105620] Updated weights for policy 1, policy_version 1359139 (0.0009) [2023-12-27 01:16:25,963][105620] Updated weights for policy 1, policy_version 1359149 (0.0008) [2023-12-27 01:16:26,009][105620] Updated weights for policy 1, policy_version 1359159 (0.0008) [2023-12-27 01:16:26,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 695451648. Throughput: 0: 9429.3, 1: 9847.7. Samples: 695459252. Policy #0 lag: (min: 18.0, avg: 40.6, max: 50.0) [2023-12-27 01:16:26,062][104569] Avg episode reward: [(0, '7987.863'), (1, '9084.379')] [2023-12-27 01:16:26,143][105692] Updated weights for policy 0, policy_version 1357074 (0.0009) [2023-12-27 01:16:26,211][105692] Updated weights for policy 0, policy_version 1357084 (0.0010) [2023-12-27 01:16:26,274][105692] Updated weights for policy 0, policy_version 1357094 (0.0009) [2023-12-27 01:16:26,342][105692] Updated weights for policy 0, policy_version 1357104 (0.0010) [2023-12-27 01:16:26,692][105620] Updated weights for policy 1, policy_version 1359169 (0.0010) [2023-12-27 01:16:26,752][105620] Updated weights for policy 1, policy_version 1359179 (0.0009) [2023-12-27 01:16:26,799][105620] Updated weights for policy 1, policy_version 1359189 (0.0009) [2023-12-27 01:16:26,846][105620] Updated weights for policy 1, policy_version 1359199 (0.0008) [2023-12-27 01:16:27,109][105692] Updated weights for policy 0, policy_version 1357114 (0.0009) [2023-12-27 01:16:27,170][105692] Updated weights for policy 0, policy_version 1357124 (0.0009) [2023-12-27 01:16:27,220][105692] Updated weights for policy 0, policy_version 1357134 (0.0009) [2023-12-27 01:16:27,609][105620] Updated weights for policy 1, policy_version 1359209 (0.0009) [2023-12-27 01:16:27,661][105620] Updated weights for policy 1, policy_version 1359219 (0.0009) [2023-12-27 01:16:27,716][105620] Updated weights for policy 1, policy_version 1359231 (0.0011) [2023-12-27 01:16:27,911][105692] Updated weights for policy 0, policy_version 1357144 (0.0009) [2023-12-27 01:16:27,961][105692] Updated weights for policy 0, policy_version 1357154 (0.0009) [2023-12-27 01:16:28,014][105692] Updated weights for policy 0, policy_version 1357164 (0.0008) [2023-12-27 01:16:28,524][105620] Updated weights for policy 1, policy_version 1359241 (0.0009) [2023-12-27 01:16:28,596][105620] Updated weights for policy 1, policy_version 1359251 (0.0009) [2023-12-27 01:16:28,659][105620] Updated weights for policy 1, policy_version 1359261 (0.0009) [2023-12-27 01:16:28,769][105692] Updated weights for policy 0, policy_version 1357174 (0.0008) [2023-12-27 01:16:28,834][105692] Updated weights for policy 0, policy_version 1357184 (0.0009) [2023-12-27 01:16:28,894][105692] Updated weights for policy 0, policy_version 1357194 (0.0009) [2023-12-27 01:16:29,408][105620] Updated weights for policy 1, policy_version 1359271 (0.0009) [2023-12-27 01:16:29,457][105620] Updated weights for policy 1, policy_version 1359281 (0.0010) [2023-12-27 01:16:29,518][105620] Updated weights for policy 1, policy_version 1359292 (0.0010) [2023-12-27 01:16:29,610][105692] Updated weights for policy 0, policy_version 1357204 (0.0008) [2023-12-27 01:16:29,668][105692] Updated weights for policy 0, policy_version 1357214 (0.0006) [2023-12-27 01:16:29,728][105692] Updated weights for policy 0, policy_version 1357224 (0.0005) [2023-12-27 01:16:30,311][105692] Updated weights for policy 0, policy_version 1357234 (0.0006) [2023-12-27 01:16:30,360][105692] Updated weights for policy 0, policy_version 1357244 (0.0006) [2023-12-27 01:16:30,384][105620] Updated weights for policy 1, policy_version 1359302 (0.0010) [2023-12-27 01:16:30,410][105692] Updated weights for policy 0, policy_version 1357254 (0.0009) [2023-12-27 01:16:30,432][105620] Updated weights for policy 1, policy_version 1359312 (0.0005) [2023-12-27 01:16:30,455][105692] Updated weights for policy 0, policy_version 1357264 (0.0010) [2023-12-27 01:16:30,479][105620] Updated weights for policy 1, policy_version 1359322 (0.0007) [2023-12-27 01:16:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19114.6, 300 sec: 19494.2). Total num frames: 695541760. Throughput: 0: 9477.5, 1: 9764.3. Samples: 695515968. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:16:31,063][104569] Avg episode reward: [(0, '7993.866'), (1, '9265.970')] [2023-12-27 01:16:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001359328_348028928.pth... [2023-12-27 01:16:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001358240_347750400.pth [2023-12-27 01:16:31,132][105692] Updated weights for policy 0, policy_version 1357274 (0.0009) [2023-12-27 01:16:31,194][105692] Updated weights for policy 0, policy_version 1357284 (0.0009) [2023-12-27 01:16:31,257][105692] Updated weights for policy 0, policy_version 1357294 (0.0010) [2023-12-27 01:16:31,265][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001357296_347521024.pth... [2023-12-27 01:16:31,270][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001356144_347226112.pth [2023-12-27 01:16:31,298][105620] Updated weights for policy 1, policy_version 1359332 (0.0009) [2023-12-27 01:16:31,364][105620] Updated weights for policy 1, policy_version 1359342 (0.0008) [2023-12-27 01:16:31,422][105620] Updated weights for policy 1, policy_version 1359352 (0.0008) [2023-12-27 01:16:31,914][105692] Updated weights for policy 0, policy_version 1357304 (0.0006) [2023-12-27 01:16:31,988][105692] Updated weights for policy 0, policy_version 1357314 (0.0007) [2023-12-27 01:16:32,040][105692] Updated weights for policy 0, policy_version 1357324 (0.0006) [2023-12-27 01:16:32,096][105620] Updated weights for policy 1, policy_version 1359362 (0.0008) [2023-12-27 01:16:32,158][105620] Updated weights for policy 1, policy_version 1359372 (0.0005) [2023-12-27 01:16:32,209][105620] Updated weights for policy 1, policy_version 1359382 (0.0005) [2023-12-27 01:16:32,260][105620] Updated weights for policy 1, policy_version 1359392 (0.0010) [2023-12-27 01:16:32,611][105692] Updated weights for policy 0, policy_version 1357334 (0.0005) [2023-12-27 01:16:32,670][105692] Updated weights for policy 0, policy_version 1357344 (0.0006) [2023-12-27 01:16:32,724][105692] Updated weights for policy 0, policy_version 1357354 (0.0005) [2023-12-27 01:16:32,984][105620] Updated weights for policy 1, policy_version 1359402 (0.0005) [2023-12-27 01:16:33,045][105620] Updated weights for policy 1, policy_version 1359412 (0.0005) [2023-12-27 01:16:33,088][105620] Updated weights for policy 1, policy_version 1359422 (0.0005) [2023-12-27 01:16:33,237][105692] Updated weights for policy 0, policy_version 1357364 (0.0005) [2023-12-27 01:16:33,293][105692] Updated weights for policy 0, policy_version 1357374 (0.0005) [2023-12-27 01:16:33,349][105692] Updated weights for policy 0, policy_version 1357384 (0.0005) [2023-12-27 01:16:33,609][105620] Updated weights for policy 1, policy_version 1359432 (0.0005) [2023-12-27 01:16:33,669][105620] Updated weights for policy 1, policy_version 1359442 (0.0009) [2023-12-27 01:16:33,723][105620] Updated weights for policy 1, policy_version 1359452 (0.0009) [2023-12-27 01:16:33,908][105692] Updated weights for policy 0, policy_version 1357394 (0.0005) [2023-12-27 01:16:33,966][105692] Updated weights for policy 0, policy_version 1357404 (0.0005) [2023-12-27 01:16:34,015][105692] Updated weights for policy 0, policy_version 1357414 (0.0005) [2023-12-27 01:16:34,070][105692] Updated weights for policy 0, policy_version 1357424 (0.0005) [2023-12-27 01:16:34,461][105620] Updated weights for policy 1, policy_version 1359462 (0.0008) [2023-12-27 01:16:34,523][105620] Updated weights for policy 1, policy_version 1359472 (0.0007) [2023-12-27 01:16:34,588][105620] Updated weights for policy 1, policy_version 1359482 (0.0010) [2023-12-27 01:16:34,694][105692] Updated weights for policy 0, policy_version 1357434 (0.0008) [2023-12-27 01:16:34,750][105692] Updated weights for policy 0, policy_version 1357444 (0.0008) [2023-12-27 01:16:34,805][105692] Updated weights for policy 0, policy_version 1357454 (0.0006) [2023-12-27 01:16:35,359][105692] Updated weights for policy 0, policy_version 1357464 (0.0005) [2023-12-27 01:16:35,428][105692] Updated weights for policy 0, policy_version 1357474 (0.0005) [2023-12-27 01:16:35,446][105620] Updated weights for policy 1, policy_version 1359492 (0.0010) [2023-12-27 01:16:35,494][105692] Updated weights for policy 0, policy_version 1357484 (0.0005) [2023-12-27 01:16:35,502][105620] Updated weights for policy 1, policy_version 1359502 (0.0008) [2023-12-27 01:16:35,562][105620] Updated weights for policy 1, policy_version 1359512 (0.0009) [2023-12-27 01:16:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.3, 300 sec: 19522.0). Total num frames: 695648256. Throughput: 0: 9758.1, 1: 9674.6. Samples: 695640328. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:16:36,062][104569] Avg episode reward: [(0, '8358.582'), (1, '9084.807')] [2023-12-27 01:16:36,148][105692] Updated weights for policy 0, policy_version 1357494 (0.0007) [2023-12-27 01:16:36,164][105620] Updated weights for policy 1, policy_version 1359522 (0.0006) [2023-12-27 01:16:36,205][105692] Updated weights for policy 0, policy_version 1357504 (0.0007) [2023-12-27 01:16:36,225][105620] Updated weights for policy 1, policy_version 1359532 (0.0008) [2023-12-27 01:16:36,241][105585] KL-divergence is very high: 172.0002 [2023-12-27 01:16:36,262][105692] Updated weights for policy 0, policy_version 1357514 (0.0007) [2023-12-27 01:16:36,286][105585] KL-divergence is very high: 194.2944 [2023-12-27 01:16:36,291][105620] Updated weights for policy 1, policy_version 1359542 (0.0011) [2023-12-27 01:16:36,355][105620] Updated weights for policy 1, policy_version 1359552 (0.0011) [2023-12-27 01:16:36,996][105692] Updated weights for policy 0, policy_version 1357524 (0.0008) [2023-12-27 01:16:37,053][105692] Updated weights for policy 0, policy_version 1357534 (0.0011) [2023-12-27 01:16:37,065][105620] Updated weights for policy 1, policy_version 1359562 (0.0005) [2023-12-27 01:16:37,070][105585] KL-divergence is very high: 103.6628 [2023-12-27 01:16:37,082][105585] KL-divergence is very high: 115.1768 [2023-12-27 01:16:37,105][105692] Updated weights for policy 0, policy_version 1357544 (0.0011) [2023-12-27 01:16:37,125][105620] Updated weights for policy 1, policy_version 1359572 (0.0005) [2023-12-27 01:16:37,187][105620] Updated weights for policy 1, policy_version 1359582 (0.0005) [2023-12-27 01:16:37,804][105620] Updated weights for policy 1, policy_version 1359592 (0.0005) [2023-12-27 01:16:37,838][105692] Updated weights for policy 0, policy_version 1357554 (0.0011) [2023-12-27 01:16:37,859][105620] Updated weights for policy 1, policy_version 1359602 (0.0005) [2023-12-27 01:16:37,895][105692] Updated weights for policy 0, policy_version 1357564 (0.0010) [2023-12-27 01:16:37,911][105620] Updated weights for policy 1, policy_version 1359612 (0.0006) [2023-12-27 01:16:37,949][105692] Updated weights for policy 0, policy_version 1357574 (0.0008) [2023-12-27 01:16:37,998][105692] Updated weights for policy 0, policy_version 1357584 (0.0008) [2023-12-27 01:16:38,657][105620] Updated weights for policy 1, policy_version 1359622 (0.0010) [2023-12-27 01:16:38,716][105620] Updated weights for policy 1, policy_version 1359632 (0.0008) [2023-12-27 01:16:38,731][105692] Updated weights for policy 0, policy_version 1357594 (0.0006) [2023-12-27 01:16:38,774][105620] Updated weights for policy 1, policy_version 1359642 (0.0007) [2023-12-27 01:16:38,788][105692] Updated weights for policy 0, policy_version 1357604 (0.0008) [2023-12-27 01:16:38,850][105692] Updated weights for policy 0, policy_version 1357614 (0.0008) [2023-12-27 01:16:39,488][105620] Updated weights for policy 1, policy_version 1359652 (0.0009) [2023-12-27 01:16:39,549][105620] Updated weights for policy 1, policy_version 1359662 (0.0010) [2023-12-27 01:16:39,590][105692] Updated weights for policy 0, policy_version 1357624 (0.0005) [2023-12-27 01:16:39,606][105620] Updated weights for policy 1, policy_version 1359672 (0.0008) [2023-12-27 01:16:39,643][105692] Updated weights for policy 0, policy_version 1357634 (0.0009) [2023-12-27 01:16:39,696][105692] Updated weights for policy 0, policy_version 1357644 (0.0008) [2023-12-27 01:16:40,377][105692] Updated weights for policy 0, policy_version 1357654 (0.0009) [2023-12-27 01:16:40,421][105620] Updated weights for policy 1, policy_version 1359682 (0.0005) [2023-12-27 01:16:40,443][105692] Updated weights for policy 0, policy_version 1357664 (0.0009) [2023-12-27 01:16:40,479][105620] Updated weights for policy 1, policy_version 1359692 (0.0006) [2023-12-27 01:16:40,505][105692] Updated weights for policy 0, policy_version 1357674 (0.0008) [2023-12-27 01:16:40,537][105620] Updated weights for policy 1, policy_version 1359702 (0.0006) [2023-12-27 01:16:40,591][105620] Updated weights for policy 1, policy_version 1359712 (0.0008) [2023-12-27 01:16:41,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 695746560. Throughput: 0: 9836.4, 1: 9592.5. Samples: 695758192. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:16:41,063][104569] Avg episode reward: [(0, '8072.949'), (1, '9084.375')] [2023-12-27 01:16:41,296][105692] Updated weights for policy 0, policy_version 1357684 (0.0008) [2023-12-27 01:16:41,344][105620] Updated weights for policy 1, policy_version 1359722 (0.0009) [2023-12-27 01:16:41,371][105692] Updated weights for policy 0, policy_version 1357694 (0.0007) [2023-12-27 01:16:41,418][105620] Updated weights for policy 1, policy_version 1359732 (0.0010) [2023-12-27 01:16:41,443][105692] Updated weights for policy 0, policy_version 1357704 (0.0009) [2023-12-27 01:16:41,482][105620] Updated weights for policy 1, policy_version 1359742 (0.0007) [2023-12-27 01:16:42,232][105620] Updated weights for policy 1, policy_version 1359752 (0.0008) [2023-12-27 01:16:42,245][105692] Updated weights for policy 0, policy_version 1357714 (0.0007) [2023-12-27 01:16:42,305][105620] Updated weights for policy 1, policy_version 1359762 (0.0008) [2023-12-27 01:16:42,313][105692] Updated weights for policy 0, policy_version 1357724 (0.0007) [2023-12-27 01:16:42,372][105620] Updated weights for policy 1, policy_version 1359772 (0.0008) [2023-12-27 01:16:42,383][105692] Updated weights for policy 0, policy_version 1357734 (0.0009) [2023-12-27 01:16:42,441][105692] Updated weights for policy 0, policy_version 1357744 (0.0009) [2023-12-27 01:16:43,094][105620] Updated weights for policy 1, policy_version 1359782 (0.0008) [2023-12-27 01:16:43,149][105620] Updated weights for policy 1, policy_version 1359792 (0.0009) [2023-12-27 01:16:43,177][105692] Updated weights for policy 0, policy_version 1357754 (0.0008) [2023-12-27 01:16:43,209][105620] Updated weights for policy 1, policy_version 1359802 (0.0007) [2023-12-27 01:16:43,230][105692] Updated weights for policy 0, policy_version 1357764 (0.0009) [2023-12-27 01:16:43,278][105692] Updated weights for policy 0, policy_version 1357774 (0.0009) [2023-12-27 01:16:43,876][105692] Updated weights for policy 0, policy_version 1357784 (0.0005) [2023-12-27 01:16:43,903][105620] Updated weights for policy 1, policy_version 1359812 (0.0007) [2023-12-27 01:16:43,931][105692] Updated weights for policy 0, policy_version 1357794 (0.0005) [2023-12-27 01:16:43,954][105620] Updated weights for policy 1, policy_version 1359822 (0.0006) [2023-12-27 01:16:43,987][105692] Updated weights for policy 0, policy_version 1357804 (0.0006) [2023-12-27 01:16:44,006][105620] Updated weights for policy 1, policy_version 1359832 (0.0005) [2023-12-27 01:16:44,558][105692] Updated weights for policy 0, policy_version 1357814 (0.0009) [2023-12-27 01:16:44,607][105692] Updated weights for policy 0, policy_version 1357824 (0.0008) [2023-12-27 01:16:44,650][105692] Updated weights for policy 0, policy_version 1357834 (0.0007) [2023-12-27 01:16:44,732][105620] Updated weights for policy 1, policy_version 1359842 (0.0006) [2023-12-27 01:16:44,792][105620] Updated weights for policy 1, policy_version 1359852 (0.0009) [2023-12-27 01:16:44,851][105620] Updated weights for policy 1, policy_version 1359862 (0.0009) [2023-12-27 01:16:44,917][105620] Updated weights for policy 1, policy_version 1359872 (0.0011) [2023-12-27 01:16:45,467][105692] Updated weights for policy 0, policy_version 1357844 (0.0007) [2023-12-27 01:16:45,519][105692] Updated weights for policy 0, policy_version 1357854 (0.0008) [2023-12-27 01:16:45,526][105620] Updated weights for policy 1, policy_version 1359882 (0.0006) [2023-12-27 01:16:45,580][105692] Updated weights for policy 0, policy_version 1357864 (0.0008) [2023-12-27 01:16:45,593][105620] Updated weights for policy 1, policy_version 1359892 (0.0009) [2023-12-27 01:16:45,652][105620] Updated weights for policy 1, policy_version 1359902 (0.0010) [2023-12-27 01:16:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.9, 300 sec: 19521.9). Total num frames: 695844864. Throughput: 0: 9847.1, 1: 9580.9. Samples: 695813744. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:16:46,063][104569] Avg episode reward: [(0, '7905.646'), (1, '8995.693')] [2023-12-27 01:16:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001357872_347668480.pth... [2023-12-27 01:16:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001359904_348176384.pth... [2023-12-27 01:16:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001358784_347889664.pth [2023-12-27 01:16:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001356720_347373568.pth [2023-12-27 01:16:46,235][105692] Updated weights for policy 0, policy_version 1357874 (0.0007) [2023-12-27 01:16:46,298][105692] Updated weights for policy 0, policy_version 1357884 (0.0009) [2023-12-27 01:16:46,354][105692] Updated weights for policy 0, policy_version 1357894 (0.0008) [2023-12-27 01:16:46,359][105620] Updated weights for policy 1, policy_version 1359912 (0.0009) [2023-12-27 01:16:46,407][105692] Updated weights for policy 0, policy_version 1357904 (0.0006) [2023-12-27 01:16:46,408][105620] Updated weights for policy 1, policy_version 1359922 (0.0006) [2023-12-27 01:16:46,463][105620] Updated weights for policy 1, policy_version 1359932 (0.0008) [2023-12-27 01:16:47,099][105692] Updated weights for policy 0, policy_version 1357914 (0.0006) [2023-12-27 01:16:47,146][105692] Updated weights for policy 0, policy_version 1357924 (0.0009) [2023-12-27 01:16:47,209][105692] Updated weights for policy 0, policy_version 1357934 (0.0009) [2023-12-27 01:16:47,264][105620] Updated weights for policy 1, policy_version 1359942 (0.0009) [2023-12-27 01:16:47,325][105620] Updated weights for policy 1, policy_version 1359952 (0.0009) [2023-12-27 01:16:47,376][105620] Updated weights for policy 1, policy_version 1359962 (0.0009) [2023-12-27 01:16:47,878][105692] Updated weights for policy 0, policy_version 1357944 (0.0009) [2023-12-27 01:16:47,937][105692] Updated weights for policy 0, policy_version 1357954 (0.0010) [2023-12-27 01:16:47,999][105692] Updated weights for policy 0, policy_version 1357964 (0.0010) [2023-12-27 01:16:48,050][105620] Updated weights for policy 1, policy_version 1359972 (0.0008) [2023-12-27 01:16:48,099][105620] Updated weights for policy 1, policy_version 1359982 (0.0006) [2023-12-27 01:16:48,145][105620] Updated weights for policy 1, policy_version 1359992 (0.0008) [2023-12-27 01:16:48,675][105692] Updated weights for policy 0, policy_version 1357974 (0.0007) [2023-12-27 01:16:48,735][105692] Updated weights for policy 0, policy_version 1357984 (0.0005) [2023-12-27 01:16:48,801][105692] Updated weights for policy 0, policy_version 1357994 (0.0009) [2023-12-27 01:16:48,922][105620] Updated weights for policy 1, policy_version 1360002 (0.0008) [2023-12-27 01:16:48,979][105620] Updated weights for policy 1, policy_version 1360012 (0.0007) [2023-12-27 01:16:49,024][105620] Updated weights for policy 1, policy_version 1360022 (0.0008) [2023-12-27 01:16:49,069][105620] Updated weights for policy 1, policy_version 1360032 (0.0007) [2023-12-27 01:16:49,544][105692] Updated weights for policy 0, policy_version 1358004 (0.0010) [2023-12-27 01:16:49,600][105692] Updated weights for policy 0, policy_version 1358014 (0.0009) [2023-12-27 01:16:49,646][105692] Updated weights for policy 0, policy_version 1358024 (0.0009) [2023-12-27 01:16:49,826][105620] Updated weights for policy 1, policy_version 1360042 (0.0007) [2023-12-27 01:16:49,888][105620] Updated weights for policy 1, policy_version 1360052 (0.0008) [2023-12-27 01:16:49,950][105620] Updated weights for policy 1, policy_version 1360062 (0.0009) [2023-12-27 01:16:50,459][105692] Updated weights for policy 0, policy_version 1358034 (0.0009) [2023-12-27 01:16:50,510][105692] Updated weights for policy 0, policy_version 1358044 (0.0009) [2023-12-27 01:16:50,565][105692] Updated weights for policy 0, policy_version 1358054 (0.0007) [2023-12-27 01:16:50,626][105692] Updated weights for policy 0, policy_version 1358064 (0.0007) [2023-12-27 01:16:50,641][105620] Updated weights for policy 1, policy_version 1360072 (0.0009) [2023-12-27 01:16:50,700][105620] Updated weights for policy 1, policy_version 1360082 (0.0010) [2023-12-27 01:16:50,764][105620] Updated weights for policy 1, policy_version 1360092 (0.0009) [2023-12-27 01:16:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 695943168. Throughput: 0: 9989.7, 1: 9630.7. Samples: 695933568. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:16:51,062][104569] Avg episode reward: [(0, '7722.836'), (1, '8816.244')] [2023-12-27 01:16:51,367][105692] Updated weights for policy 0, policy_version 1358074 (0.0008) [2023-12-27 01:16:51,422][105692] Updated weights for policy 0, policy_version 1358084 (0.0008) [2023-12-27 01:16:51,477][105692] Updated weights for policy 0, policy_version 1358094 (0.0008) [2023-12-27 01:16:51,516][105620] Updated weights for policy 1, policy_version 1360102 (0.0007) [2023-12-27 01:16:51,577][105620] Updated weights for policy 1, policy_version 1360112 (0.0009) [2023-12-27 01:16:51,639][105620] Updated weights for policy 1, policy_version 1360122 (0.0008) [2023-12-27 01:16:52,246][105692] Updated weights for policy 0, policy_version 1358104 (0.0010) [2023-12-27 01:16:52,309][105692] Updated weights for policy 0, policy_version 1358114 (0.0011) [2023-12-27 01:16:52,319][105620] Updated weights for policy 1, policy_version 1360132 (0.0008) [2023-12-27 01:16:52,368][105692] Updated weights for policy 0, policy_version 1358124 (0.0011) [2023-12-27 01:16:52,387][105620] Updated weights for policy 1, policy_version 1360142 (0.0007) [2023-12-27 01:16:52,435][105620] Updated weights for policy 1, policy_version 1360152 (0.0008) [2023-12-27 01:16:52,991][105692] Updated weights for policy 0, policy_version 1358134 (0.0007) [2023-12-27 01:16:53,048][105692] Updated weights for policy 0, policy_version 1358144 (0.0006) [2023-12-27 01:16:53,106][105692] Updated weights for policy 0, policy_version 1358154 (0.0005) [2023-12-27 01:16:53,126][105585] KL-divergence is very high: 131.4267 [2023-12-27 01:16:53,210][105620] Updated weights for policy 1, policy_version 1360162 (0.0006) [2023-12-27 01:16:53,271][105620] Updated weights for policy 1, policy_version 1360172 (0.0007) [2023-12-27 01:16:53,323][105620] Updated weights for policy 1, policy_version 1360182 (0.0008) [2023-12-27 01:16:53,371][105620] Updated weights for policy 1, policy_version 1360192 (0.0008) [2023-12-27 01:16:53,709][105692] Updated weights for policy 0, policy_version 1358164 (0.0006) [2023-12-27 01:16:53,758][105692] Updated weights for policy 0, policy_version 1358174 (0.0006) [2023-12-27 01:16:53,819][105692] Updated weights for policy 0, policy_version 1358184 (0.0005) [2023-12-27 01:16:54,113][105620] Updated weights for policy 1, policy_version 1360202 (0.0005) [2023-12-27 01:16:54,172][105620] Updated weights for policy 1, policy_version 1360212 (0.0008) [2023-12-27 01:16:54,232][105620] Updated weights for policy 1, policy_version 1360222 (0.0011) [2023-12-27 01:16:54,447][105692] Updated weights for policy 0, policy_version 1358194 (0.0006) [2023-12-27 01:16:54,507][105692] Updated weights for policy 0, policy_version 1358204 (0.0010) [2023-12-27 01:16:54,564][105692] Updated weights for policy 0, policy_version 1358215 (0.0010) [2023-12-27 01:16:54,827][105620] Updated weights for policy 1, policy_version 1360232 (0.0006) [2023-12-27 01:16:54,891][105620] Updated weights for policy 1, policy_version 1360242 (0.0005) [2023-12-27 01:16:54,957][105620] Updated weights for policy 1, policy_version 1360252 (0.0007) [2023-12-27 01:16:55,378][105692] Updated weights for policy 0, policy_version 1358225 (0.0011) [2023-12-27 01:16:55,435][105692] Updated weights for policy 0, policy_version 1358235 (0.0010) [2023-12-27 01:16:55,492][105692] Updated weights for policy 0, policy_version 1358245 (0.0009) [2023-12-27 01:16:55,551][105692] Updated weights for policy 0, policy_version 1358255 (0.0008) [2023-12-27 01:16:55,628][105620] Updated weights for policy 1, policy_version 1360262 (0.0010) [2023-12-27 01:16:55,689][105620] Updated weights for policy 1, policy_version 1360272 (0.0005) [2023-12-27 01:16:55,746][105620] Updated weights for policy 1, policy_version 1360282 (0.0008) [2023-12-27 01:16:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 696041472. Throughput: 0: 9978.8, 1: 9605.2. Samples: 696051132. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:16:56,063][104569] Avg episode reward: [(0, '8079.835'), (1, '8996.288')] [2023-12-27 01:16:56,280][105692] Updated weights for policy 0, policy_version 1358265 (0.0011) [2023-12-27 01:16:56,324][105620] Updated weights for policy 1, policy_version 1360292 (0.0011) [2023-12-27 01:16:56,336][105692] Updated weights for policy 0, policy_version 1358275 (0.0011) [2023-12-27 01:16:56,376][105620] Updated weights for policy 1, policy_version 1360302 (0.0010) [2023-12-27 01:16:56,388][105692] Updated weights for policy 0, policy_version 1358285 (0.0011) [2023-12-27 01:16:56,435][105620] Updated weights for policy 1, policy_version 1360312 (0.0010) [2023-12-27 01:16:56,967][105692] Updated weights for policy 0, policy_version 1358295 (0.0007) [2023-12-27 01:16:57,030][105692] Updated weights for policy 0, policy_version 1358305 (0.0005) [2023-12-27 01:16:57,095][105692] Updated weights for policy 0, policy_version 1358315 (0.0005) [2023-12-27 01:16:57,168][105620] Updated weights for policy 1, policy_version 1360322 (0.0009) [2023-12-27 01:16:57,225][105620] Updated weights for policy 1, policy_version 1360332 (0.0006) [2023-12-27 01:16:57,273][105620] Updated weights for policy 1, policy_version 1360342 (0.0005) [2023-12-27 01:16:57,335][105620] Updated weights for policy 1, policy_version 1360352 (0.0006) [2023-12-27 01:16:57,641][105692] Updated weights for policy 0, policy_version 1358325 (0.0008) [2023-12-27 01:16:57,688][105692] Updated weights for policy 0, policy_version 1358335 (0.0010) [2023-12-27 01:16:57,737][105692] Updated weights for policy 0, policy_version 1358345 (0.0007) [2023-12-27 01:16:57,868][105620] Updated weights for policy 1, policy_version 1360362 (0.0005) [2023-12-27 01:16:57,917][105620] Updated weights for policy 1, policy_version 1360372 (0.0005) [2023-12-27 01:16:57,976][105620] Updated weights for policy 1, policy_version 1360382 (0.0006) [2023-12-27 01:16:58,428][105692] Updated weights for policy 0, policy_version 1358355 (0.0007) [2023-12-27 01:16:58,497][105692] Updated weights for policy 0, policy_version 1358365 (0.0009) [2023-12-27 01:16:58,566][105692] Updated weights for policy 0, policy_version 1358375 (0.0006) [2023-12-27 01:16:58,684][105620] Updated weights for policy 1, policy_version 1360392 (0.0008) [2023-12-27 01:16:58,746][105620] Updated weights for policy 1, policy_version 1360402 (0.0009) [2023-12-27 01:16:58,801][105620] Updated weights for policy 1, policy_version 1360412 (0.0009) [2023-12-27 01:16:59,294][105692] Updated weights for policy 0, policy_version 1358385 (0.0007) [2023-12-27 01:16:59,357][105692] Updated weights for policy 0, policy_version 1358395 (0.0009) [2023-12-27 01:16:59,422][105692] Updated weights for policy 0, policy_version 1358405 (0.0008) [2023-12-27 01:16:59,486][105692] Updated weights for policy 0, policy_version 1358415 (0.0008) [2023-12-27 01:16:59,652][105620] Updated weights for policy 1, policy_version 1360422 (0.0007) [2023-12-27 01:16:59,715][105620] Updated weights for policy 1, policy_version 1360432 (0.0008) [2023-12-27 01:16:59,769][105620] Updated weights for policy 1, policy_version 1360442 (0.0009) [2023-12-27 01:17:00,166][105692] Updated weights for policy 0, policy_version 1358425 (0.0008) [2023-12-27 01:17:00,217][105692] Updated weights for policy 0, policy_version 1358435 (0.0009) [2023-12-27 01:17:00,266][105692] Updated weights for policy 0, policy_version 1358445 (0.0008) [2023-12-27 01:17:00,494][105620] Updated weights for policy 1, policy_version 1360452 (0.0009) [2023-12-27 01:17:00,546][105620] Updated weights for policy 1, policy_version 1360463 (0.0010) [2023-12-27 01:17:00,597][105620] Updated weights for policy 1, policy_version 1360473 (0.0007) [2023-12-27 01:17:01,047][105692] Updated weights for policy 0, policy_version 1358455 (0.0008) [2023-12-27 01:17:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 696139776. Throughput: 0: 10077.6, 1: 9687.9. Samples: 696114556. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:17:01,062][104569] Avg episode reward: [(0, '8351.377'), (1, '9086.874')] [2023-12-27 01:17:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001360480_348323840.pth... [2023-12-27 01:17:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001359328_348028928.pth [2023-12-27 01:17:01,104][105692] Updated weights for policy 0, policy_version 1358465 (0.0008) [2023-12-27 01:17:01,163][105692] Updated weights for policy 0, policy_version 1358475 (0.0010) [2023-12-27 01:17:01,190][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001358480_347824128.pth... [2023-12-27 01:17:01,195][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001357296_347521024.pth [2023-12-27 01:17:01,223][105620] Updated weights for policy 1, policy_version 1360483 (0.0007) [2023-12-27 01:17:01,284][105620] Updated weights for policy 1, policy_version 1360493 (0.0010) [2023-12-27 01:17:01,350][105620] Updated weights for policy 1, policy_version 1360503 (0.0010) [2023-12-27 01:17:01,939][105692] Updated weights for policy 0, policy_version 1358485 (0.0008) [2023-12-27 01:17:01,994][105692] Updated weights for policy 0, policy_version 1358495 (0.0008) [2023-12-27 01:17:02,052][105692] Updated weights for policy 0, policy_version 1358505 (0.0008) [2023-12-27 01:17:02,162][105620] Updated weights for policy 1, policy_version 1360513 (0.0009) [2023-12-27 01:17:02,217][105620] Updated weights for policy 1, policy_version 1360523 (0.0010) [2023-12-27 01:17:02,280][105620] Updated weights for policy 1, policy_version 1360533 (0.0011) [2023-12-27 01:17:02,342][105620] Updated weights for policy 1, policy_version 1360543 (0.0011) [2023-12-27 01:17:02,794][105692] Updated weights for policy 0, policy_version 1358515 (0.0008) [2023-12-27 01:17:02,845][105692] Updated weights for policy 0, policy_version 1358525 (0.0008) [2023-12-27 01:17:02,892][105692] Updated weights for policy 0, policy_version 1358535 (0.0007) [2023-12-27 01:17:03,097][105620] Updated weights for policy 1, policy_version 1360553 (0.0011) [2023-12-27 01:17:03,157][105620] Updated weights for policy 1, policy_version 1360563 (0.0011) [2023-12-27 01:17:03,206][105620] Updated weights for policy 1, policy_version 1360573 (0.0011) [2023-12-27 01:17:03,677][105692] Updated weights for policy 0, policy_version 1358545 (0.0008) [2023-12-27 01:17:03,721][105692] Updated weights for policy 0, policy_version 1358555 (0.0007) [2023-12-27 01:17:03,768][105692] Updated weights for policy 0, policy_version 1358565 (0.0008) [2023-12-27 01:17:03,816][105692] Updated weights for policy 0, policy_version 1358575 (0.0008) [2023-12-27 01:17:03,964][105620] Updated weights for policy 1, policy_version 1360583 (0.0011) [2023-12-27 01:17:04,024][105620] Updated weights for policy 1, policy_version 1360593 (0.0011) [2023-12-27 01:17:04,069][105620] Updated weights for policy 1, policy_version 1360603 (0.0010) [2023-12-27 01:17:04,612][105692] Updated weights for policy 0, policy_version 1358585 (0.0008) [2023-12-27 01:17:04,663][105692] Updated weights for policy 0, policy_version 1358595 (0.0008) [2023-12-27 01:17:04,714][105692] Updated weights for policy 0, policy_version 1358605 (0.0008) [2023-12-27 01:17:04,836][105620] Updated weights for policy 1, policy_version 1360613 (0.0010) [2023-12-27 01:17:04,887][105620] Updated weights for policy 1, policy_version 1360623 (0.0010) [2023-12-27 01:17:04,936][105620] Updated weights for policy 1, policy_version 1360633 (0.0010) [2023-12-27 01:17:05,480][105692] Updated weights for policy 0, policy_version 1358615 (0.0008) [2023-12-27 01:17:05,528][105692] Updated weights for policy 0, policy_version 1358625 (0.0008) [2023-12-27 01:17:05,572][105692] Updated weights for policy 0, policy_version 1358635 (0.0008) [2023-12-27 01:17:05,707][105620] Updated weights for policy 1, policy_version 1360643 (0.0010) [2023-12-27 01:17:05,768][105620] Updated weights for policy 1, policy_version 1360653 (0.0010) [2023-12-27 01:17:05,833][105620] Updated weights for policy 1, policy_version 1360663 (0.0010) [2023-12-27 01:17:06,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 696238080. Throughput: 0: 9939.7, 1: 9622.8. Samples: 696226736. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:17:06,062][104569] Avg episode reward: [(0, '8178.478'), (1, '9266.747')] [2023-12-27 01:17:06,349][105692] Updated weights for policy 0, policy_version 1358645 (0.0008) [2023-12-27 01:17:06,398][105692] Updated weights for policy 0, policy_version 1358655 (0.0008) [2023-12-27 01:17:06,453][105692] Updated weights for policy 0, policy_version 1358665 (0.0007) [2023-12-27 01:17:06,572][105620] Updated weights for policy 1, policy_version 1360673 (0.0010) [2023-12-27 01:17:06,643][105620] Updated weights for policy 1, policy_version 1360683 (0.0011) [2023-12-27 01:17:06,706][105620] Updated weights for policy 1, policy_version 1360693 (0.0011) [2023-12-27 01:17:06,766][105620] Updated weights for policy 1, policy_version 1360703 (0.0011) [2023-12-27 01:17:07,236][105692] Updated weights for policy 0, policy_version 1358675 (0.0007) [2023-12-27 01:17:07,293][105692] Updated weights for policy 0, policy_version 1358685 (0.0008) [2023-12-27 01:17:07,352][105692] Updated weights for policy 0, policy_version 1358695 (0.0009) [2023-12-27 01:17:07,504][105620] Updated weights for policy 1, policy_version 1360713 (0.0006) [2023-12-27 01:17:07,569][105620] Updated weights for policy 1, policy_version 1360723 (0.0005) [2023-12-27 01:17:07,636][105620] Updated weights for policy 1, policy_version 1360733 (0.0006) [2023-12-27 01:17:08,162][105692] Updated weights for policy 0, policy_version 1358705 (0.0010) [2023-12-27 01:17:08,208][105620] Updated weights for policy 1, policy_version 1360743 (0.0005) [2023-12-27 01:17:08,220][105692] Updated weights for policy 0, policy_version 1358715 (0.0009) [2023-12-27 01:17:08,275][105620] Updated weights for policy 1, policy_version 1360753 (0.0005) [2023-12-27 01:17:08,282][105692] Updated weights for policy 0, policy_version 1358725 (0.0008) [2023-12-27 01:17:08,341][105620] Updated weights for policy 1, policy_version 1360763 (0.0006) [2023-12-27 01:17:08,341][105692] Updated weights for policy 0, policy_version 1358735 (0.0008) [2023-12-27 01:17:09,037][105620] Updated weights for policy 1, policy_version 1360773 (0.0008) [2023-12-27 01:17:09,095][105620] Updated weights for policy 1, policy_version 1360783 (0.0007) [2023-12-27 01:17:09,117][105692] Updated weights for policy 0, policy_version 1358745 (0.0009) [2023-12-27 01:17:09,144][105620] Updated weights for policy 1, policy_version 1360793 (0.0006) [2023-12-27 01:17:09,180][105692] Updated weights for policy 0, policy_version 1358755 (0.0007) [2023-12-27 01:17:09,250][105692] Updated weights for policy 0, policy_version 1358765 (0.0009) [2023-12-27 01:17:09,929][105692] Updated weights for policy 0, policy_version 1358775 (0.0008) [2023-12-27 01:17:09,971][105620] Updated weights for policy 1, policy_version 1360803 (0.0009) [2023-12-27 01:17:09,993][105692] Updated weights for policy 0, policy_version 1358785 (0.0006) [2023-12-27 01:17:10,024][105620] Updated weights for policy 1, policy_version 1360813 (0.0010) [2023-12-27 01:17:10,055][105692] Updated weights for policy 0, policy_version 1358795 (0.0005) [2023-12-27 01:17:10,077][105620] Updated weights for policy 1, policy_version 1360823 (0.0010) [2023-12-27 01:17:10,719][105692] Updated weights for policy 0, policy_version 1358805 (0.0006) [2023-12-27 01:17:10,783][105692] Updated weights for policy 0, policy_version 1358815 (0.0005) [2023-12-27 01:17:10,844][105692] Updated weights for policy 0, policy_version 1358825 (0.0006) [2023-12-27 01:17:10,923][105620] Updated weights for policy 1, policy_version 1360833 (0.0011) [2023-12-27 01:17:10,989][105620] Updated weights for policy 1, policy_version 1360843 (0.0011) [2023-12-27 01:17:11,051][105620] Updated weights for policy 1, policy_version 1360853 (0.0009) [2023-12-27 01:17:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 696328192. Throughput: 0: 9915.4, 1: 9638.8. Samples: 696339196. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:17:11,063][104569] Avg episode reward: [(0, '8089.722'), (1, '9265.865')] [2023-12-27 01:17:11,114][105620] Updated weights for policy 1, policy_version 1360863 (0.0011) [2023-12-27 01:17:11,539][105692] Updated weights for policy 0, policy_version 1358835 (0.0006) [2023-12-27 01:17:11,599][105692] Updated weights for policy 0, policy_version 1358845 (0.0008) [2023-12-27 01:17:11,664][105692] Updated weights for policy 0, policy_version 1358855 (0.0008) [2023-12-27 01:17:11,898][105620] Updated weights for policy 1, policy_version 1360873 (0.0010) [2023-12-27 01:17:11,951][105620] Updated weights for policy 1, policy_version 1360883 (0.0010) [2023-12-27 01:17:12,004][105620] Updated weights for policy 1, policy_version 1360893 (0.0010) [2023-12-27 01:17:12,449][105692] Updated weights for policy 0, policy_version 1358865 (0.0008) [2023-12-27 01:17:12,513][105692] Updated weights for policy 0, policy_version 1358875 (0.0007) [2023-12-27 01:17:12,578][105692] Updated weights for policy 0, policy_version 1358885 (0.0006) [2023-12-27 01:17:12,640][105692] Updated weights for policy 0, policy_version 1358895 (0.0008) [2023-12-27 01:17:12,765][105620] Updated weights for policy 1, policy_version 1360903 (0.0008) [2023-12-27 01:17:12,834][105620] Updated weights for policy 1, policy_version 1360913 (0.0005) [2023-12-27 01:17:12,895][105620] Updated weights for policy 1, policy_version 1360923 (0.0006) [2023-12-27 01:17:13,431][105692] Updated weights for policy 0, policy_version 1358905 (0.0009) [2023-12-27 01:17:13,493][105692] Updated weights for policy 0, policy_version 1358915 (0.0009) [2023-12-27 01:17:13,514][105620] Updated weights for policy 1, policy_version 1360933 (0.0005) [2023-12-27 01:17:13,546][105692] Updated weights for policy 0, policy_version 1358925 (0.0008) [2023-12-27 01:17:13,578][105620] Updated weights for policy 1, policy_version 1360943 (0.0005) [2023-12-27 01:17:13,639][105620] Updated weights for policy 1, policy_version 1360953 (0.0005) [2023-12-27 01:17:14,317][105620] Updated weights for policy 1, policy_version 1360963 (0.0007) [2023-12-27 01:17:14,334][105692] Updated weights for policy 0, policy_version 1358935 (0.0007) [2023-12-27 01:17:14,378][105620] Updated weights for policy 1, policy_version 1360973 (0.0007) [2023-12-27 01:17:14,395][105692] Updated weights for policy 0, policy_version 1358945 (0.0009) [2023-12-27 01:17:14,445][105620] Updated weights for policy 1, policy_version 1360983 (0.0006) [2023-12-27 01:17:14,459][105692] Updated weights for policy 0, policy_version 1358955 (0.0008) [2023-12-27 01:17:15,128][105620] Updated weights for policy 1, policy_version 1360993 (0.0008) [2023-12-27 01:17:15,200][105620] Updated weights for policy 1, policy_version 1361003 (0.0011) [2023-12-27 01:17:15,244][105692] Updated weights for policy 0, policy_version 1358965 (0.0010) [2023-12-27 01:17:15,263][105620] Updated weights for policy 1, policy_version 1361013 (0.0011) [2023-12-27 01:17:15,302][105692] Updated weights for policy 0, policy_version 1358975 (0.0008) [2023-12-27 01:17:15,325][105620] Updated weights for policy 1, policy_version 1361023 (0.0008) [2023-12-27 01:17:15,360][105692] Updated weights for policy 0, policy_version 1358985 (0.0009) [2023-12-27 01:17:15,973][105620] Updated weights for policy 1, policy_version 1361033 (0.0010) [2023-12-27 01:17:16,024][105620] Updated weights for policy 1, policy_version 1361043 (0.0010) [2023-12-27 01:17:16,062][104569] Fps is (10 sec: 18021.9, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 696418304. Throughput: 0: 9888.2, 1: 9659.1. Samples: 696395596. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:17:16,063][104569] Avg episode reward: [(0, '8358.440'), (1, '8992.584')] [2023-12-27 01:17:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001358992_347955200.pth... [2023-12-27 01:17:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001357872_347668480.pth [2023-12-27 01:17:16,079][105620] Updated weights for policy 1, policy_version 1361053 (0.0010) [2023-12-27 01:17:16,093][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001361056_348471296.pth... [2023-12-27 01:17:16,098][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001359904_348176384.pth [2023-12-27 01:17:16,151][105692] Updated weights for policy 0, policy_version 1358995 (0.0008) [2023-12-27 01:17:16,201][105692] Updated weights for policy 0, policy_version 1359005 (0.0007) [2023-12-27 01:17:16,250][105692] Updated weights for policy 0, policy_version 1359015 (0.0005) [2023-12-27 01:17:16,825][105620] Updated weights for policy 1, policy_version 1361063 (0.0010) [2023-12-27 01:17:16,883][105692] Updated weights for policy 0, policy_version 1359025 (0.0005) [2023-12-27 01:17:16,888][105620] Updated weights for policy 1, policy_version 1361073 (0.0010) [2023-12-27 01:17:16,943][105620] Updated weights for policy 1, policy_version 1361083 (0.0010) [2023-12-27 01:17:16,943][105692] Updated weights for policy 0, policy_version 1359035 (0.0006) [2023-12-27 01:17:16,998][105585] KL-divergence is very high: 105.9794 [2023-12-27 01:17:17,001][105692] Updated weights for policy 0, policy_version 1359045 (0.0010) [2023-12-27 01:17:17,037][105585] KL-divergence is very high: 135.5104 [2023-12-27 01:17:17,054][105692] Updated weights for policy 0, policy_version 1359056 (0.0010) [2023-12-27 01:17:17,601][105620] Updated weights for policy 1, policy_version 1361093 (0.0006) [2023-12-27 01:17:17,671][105620] Updated weights for policy 1, policy_version 1361103 (0.0005) [2023-12-27 01:17:17,733][105620] Updated weights for policy 1, policy_version 1361113 (0.0006) [2023-12-27 01:17:17,896][105692] Updated weights for policy 0, policy_version 1359066 (0.0010) [2023-12-27 01:17:17,951][105692] Updated weights for policy 0, policy_version 1359077 (0.0011) [2023-12-27 01:17:18,007][105692] Updated weights for policy 0, policy_version 1359088 (0.0010) [2023-12-27 01:17:18,296][105620] Updated weights for policy 1, policy_version 1361123 (0.0006) [2023-12-27 01:17:18,356][105620] Updated weights for policy 1, policy_version 1361133 (0.0009) [2023-12-27 01:17:18,412][105620] Updated weights for policy 1, policy_version 1361143 (0.0009) [2023-12-27 01:17:18,825][105692] Updated weights for policy 0, policy_version 1359098 (0.0008) [2023-12-27 01:17:18,878][105692] Updated weights for policy 0, policy_version 1359108 (0.0006) [2023-12-27 01:17:18,931][105692] Updated weights for policy 0, policy_version 1359118 (0.0006) [2023-12-27 01:17:19,262][105620] Updated weights for policy 1, policy_version 1361153 (0.0008) [2023-12-27 01:17:19,320][105620] Updated weights for policy 1, policy_version 1361163 (0.0008) [2023-12-27 01:17:19,388][105620] Updated weights for policy 1, policy_version 1361173 (0.0008) [2023-12-27 01:17:19,442][105620] Updated weights for policy 1, policy_version 1361183 (0.0009) [2023-12-27 01:17:19,516][105692] Updated weights for policy 0, policy_version 1359128 (0.0008) [2023-12-27 01:17:19,573][105692] Updated weights for policy 0, policy_version 1359138 (0.0006) [2023-12-27 01:17:19,627][105692] Updated weights for policy 0, policy_version 1359148 (0.0006) [2023-12-27 01:17:20,242][105692] Updated weights for policy 0, policy_version 1359158 (0.0009) [2023-12-27 01:17:20,299][105692] Updated weights for policy 0, policy_version 1359168 (0.0006) [2023-12-27 01:17:20,314][105620] Updated weights for policy 1, policy_version 1361193 (0.0009) [2023-12-27 01:17:20,355][105692] Updated weights for policy 0, policy_version 1359178 (0.0007) [2023-12-27 01:17:20,370][105620] Updated weights for policy 1, policy_version 1361203 (0.0009) [2023-12-27 01:17:20,423][105620] Updated weights for policy 1, policy_version 1361213 (0.0009) [2023-12-27 01:17:20,958][105692] Updated weights for policy 0, policy_version 1359188 (0.0008) [2023-12-27 01:17:21,005][105692] Updated weights for policy 0, policy_version 1359198 (0.0009) [2023-12-27 01:17:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 696516608. Throughput: 0: 9673.6, 1: 9664.5. Samples: 696510544. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:17:21,063][104569] Avg episode reward: [(0, '8077.449'), (1, '8990.737')] [2023-12-27 01:17:21,063][105692] Updated weights for policy 0, policy_version 1359208 (0.0009) [2023-12-27 01:17:21,359][105620] Updated weights for policy 1, policy_version 1361223 (0.0008) [2023-12-27 01:17:21,428][105620] Updated weights for policy 1, policy_version 1361233 (0.0010) [2023-12-27 01:17:21,486][105620] Updated weights for policy 1, policy_version 1361243 (0.0009) [2023-12-27 01:17:21,764][105692] Updated weights for policy 0, policy_version 1359218 (0.0006) [2023-12-27 01:17:21,833][105692] Updated weights for policy 0, policy_version 1359228 (0.0009) [2023-12-27 01:17:21,884][105692] Updated weights for policy 0, policy_version 1359238 (0.0011) [2023-12-27 01:17:21,947][105692] Updated weights for policy 0, policy_version 1359248 (0.0011) [2023-12-27 01:17:22,284][105620] Updated weights for policy 1, policy_version 1361253 (0.0009) [2023-12-27 01:17:22,349][105620] Updated weights for policy 1, policy_version 1361263 (0.0009) [2023-12-27 01:17:22,410][105620] Updated weights for policy 1, policy_version 1361273 (0.0008) [2023-12-27 01:17:22,681][105692] Updated weights for policy 0, policy_version 1359258 (0.0011) [2023-12-27 01:17:22,737][105692] Updated weights for policy 0, policy_version 1359268 (0.0011) [2023-12-27 01:17:22,790][105692] Updated weights for policy 0, policy_version 1359278 (0.0011) [2023-12-27 01:17:23,168][105620] Updated weights for policy 1, policy_version 1361283 (0.0008) [2023-12-27 01:17:23,219][105620] Updated weights for policy 1, policy_version 1361293 (0.0007) [2023-12-27 01:17:23,269][105620] Updated weights for policy 1, policy_version 1361303 (0.0008) [2023-12-27 01:17:23,553][105692] Updated weights for policy 0, policy_version 1359288 (0.0010) [2023-12-27 01:17:23,602][105692] Updated weights for policy 0, policy_version 1359298 (0.0010) [2023-12-27 01:17:23,650][105692] Updated weights for policy 0, policy_version 1359308 (0.0010) [2023-12-27 01:17:23,953][105620] Updated weights for policy 1, policy_version 1361313 (0.0008) [2023-12-27 01:17:24,008][105620] Updated weights for policy 1, policy_version 1361323 (0.0008) [2023-12-27 01:17:24,063][105620] Updated weights for policy 1, policy_version 1361333 (0.0008) [2023-12-27 01:17:24,110][105620] Updated weights for policy 1, policy_version 1361343 (0.0008) [2023-12-27 01:17:24,406][105692] Updated weights for policy 0, policy_version 1359318 (0.0011) [2023-12-27 01:17:24,461][105692] Updated weights for policy 0, policy_version 1359328 (0.0011) [2023-12-27 01:17:24,517][105692] Updated weights for policy 0, policy_version 1359338 (0.0011) [2023-12-27 01:17:24,912][105620] Updated weights for policy 1, policy_version 1361354 (0.0010) [2023-12-27 01:17:24,966][105620] Updated weights for policy 1, policy_version 1361364 (0.0009) [2023-12-27 01:17:25,022][105620] Updated weights for policy 1, policy_version 1361374 (0.0008) [2023-12-27 01:17:25,129][105692] Updated weights for policy 0, policy_version 1359348 (0.0008) [2023-12-27 01:17:25,180][105692] Updated weights for policy 0, policy_version 1359358 (0.0005) [2023-12-27 01:17:25,239][105692] Updated weights for policy 0, policy_version 1359368 (0.0007) [2023-12-27 01:17:25,781][105692] Updated weights for policy 0, policy_version 1359378 (0.0005) [2023-12-27 01:17:25,828][105692] Updated weights for policy 0, policy_version 1359388 (0.0006) [2023-12-27 01:17:25,875][105692] Updated weights for policy 0, policy_version 1359398 (0.0007) [2023-12-27 01:17:25,919][105620] Updated weights for policy 1, policy_version 1361384 (0.0008) [2023-12-27 01:17:25,926][105692] Updated weights for policy 0, policy_version 1359408 (0.0006) [2023-12-27 01:17:25,975][105620] Updated weights for policy 1, policy_version 1361394 (0.0008) [2023-12-27 01:17:26,039][105620] Updated weights for policy 1, policy_version 1361404 (0.0009) [2023-12-27 01:17:26,062][104569] Fps is (10 sec: 20480.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 696623104. Throughput: 0: 9720.3, 1: 9537.8. Samples: 696624804. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:17:26,062][104569] Avg episode reward: [(0, '7987.587'), (1, '9173.497')] [2023-12-27 01:17:26,573][105692] Updated weights for policy 0, policy_version 1359418 (0.0005) [2023-12-27 01:17:26,623][105692] Updated weights for policy 0, policy_version 1359428 (0.0008) [2023-12-27 01:17:26,675][105692] Updated weights for policy 0, policy_version 1359438 (0.0007) [2023-12-27 01:17:26,860][105620] Updated weights for policy 1, policy_version 1361414 (0.0008) [2023-12-27 01:17:26,906][105620] Updated weights for policy 1, policy_version 1361424 (0.0008) [2023-12-27 01:17:26,963][105620] Updated weights for policy 1, policy_version 1361434 (0.0009) [2023-12-27 01:17:27,330][105692] Updated weights for policy 0, policy_version 1359448 (0.0008) [2023-12-27 01:17:27,380][105692] Updated weights for policy 0, policy_version 1359458 (0.0009) [2023-12-27 01:17:27,427][105692] Updated weights for policy 0, policy_version 1359468 (0.0009) [2023-12-27 01:17:27,717][105620] Updated weights for policy 1, policy_version 1361444 (0.0009) [2023-12-27 01:17:27,767][105620] Updated weights for policy 1, policy_version 1361454 (0.0009) [2023-12-27 01:17:27,824][105620] Updated weights for policy 1, policy_version 1361464 (0.0009) [2023-12-27 01:17:28,126][105692] Updated weights for policy 0, policy_version 1359478 (0.0007) [2023-12-27 01:17:28,173][105692] Updated weights for policy 0, policy_version 1359488 (0.0005) [2023-12-27 01:17:28,218][105692] Updated weights for policy 0, policy_version 1359498 (0.0005) [2023-12-27 01:17:28,660][105620] Updated weights for policy 1, policy_version 1361474 (0.0008) [2023-12-27 01:17:28,713][105620] Updated weights for policy 1, policy_version 1361484 (0.0008) [2023-12-27 01:17:28,770][105620] Updated weights for policy 1, policy_version 1361494 (0.0008) [2023-12-27 01:17:28,823][105620] Updated weights for policy 1, policy_version 1361504 (0.0009) [2023-12-27 01:17:28,894][105692] Updated weights for policy 0, policy_version 1359508 (0.0007) [2023-12-27 01:17:28,944][105692] Updated weights for policy 0, policy_version 1359518 (0.0009) [2023-12-27 01:17:28,994][105692] Updated weights for policy 0, policy_version 1359528 (0.0009) [2023-12-27 01:17:29,657][105620] Updated weights for policy 1, policy_version 1361514 (0.0008) [2023-12-27 01:17:29,718][105620] Updated weights for policy 1, policy_version 1361524 (0.0008) [2023-12-27 01:17:29,767][105692] Updated weights for policy 0, policy_version 1359538 (0.0009) [2023-12-27 01:17:29,777][105620] Updated weights for policy 1, policy_version 1361534 (0.0006) [2023-12-27 01:17:29,834][105692] Updated weights for policy 0, policy_version 1359548 (0.0007) [2023-12-27 01:17:29,899][105692] Updated weights for policy 0, policy_version 1359558 (0.0008) [2023-12-27 01:17:29,959][105692] Updated weights for policy 0, policy_version 1359568 (0.0009) [2023-12-27 01:17:30,522][105620] Updated weights for policy 1, policy_version 1361544 (0.0009) [2023-12-27 01:17:30,583][105620] Updated weights for policy 1, policy_version 1361554 (0.0008) [2023-12-27 01:17:30,615][105692] Updated weights for policy 0, policy_version 1359578 (0.0006) [2023-12-27 01:17:30,633][105620] Updated weights for policy 1, policy_version 1361564 (0.0008) [2023-12-27 01:17:30,665][105692] Updated weights for policy 0, policy_version 1359588 (0.0005) [2023-12-27 01:17:30,726][105692] Updated weights for policy 0, policy_version 1359598 (0.0006) [2023-12-27 01:17:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 696713216. Throughput: 0: 9825.0, 1: 9499.6. Samples: 696683352. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:17:31,063][104569] Avg episode reward: [(0, '7535.038'), (1, '8992.631')] [2023-12-27 01:17:31,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001359600_348110848.pth... [2023-12-27 01:17:31,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001361568_348602368.pth... [2023-12-27 01:17:31,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001360480_348323840.pth [2023-12-27 01:17:31,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001358480_347824128.pth [2023-12-27 01:17:31,319][105620] Updated weights for policy 1, policy_version 1361574 (0.0009) [2023-12-27 01:17:31,382][105620] Updated weights for policy 1, policy_version 1361584 (0.0008) [2023-12-27 01:17:31,439][105620] Updated weights for policy 1, policy_version 1361594 (0.0009) [2023-12-27 01:17:31,502][105692] Updated weights for policy 0, policy_version 1359608 (0.0010) [2023-12-27 01:17:31,563][105692] Updated weights for policy 0, policy_version 1359618 (0.0009) [2023-12-27 01:17:31,627][105692] Updated weights for policy 0, policy_version 1359628 (0.0007) [2023-12-27 01:17:32,205][105692] Updated weights for policy 0, policy_version 1359638 (0.0008) [2023-12-27 01:17:32,223][105620] Updated weights for policy 1, policy_version 1361604 (0.0009) [2023-12-27 01:17:32,266][105692] Updated weights for policy 0, policy_version 1359648 (0.0006) [2023-12-27 01:17:32,286][105620] Updated weights for policy 1, policy_version 1361614 (0.0008) [2023-12-27 01:17:32,318][105692] Updated weights for policy 0, policy_version 1359658 (0.0007) [2023-12-27 01:17:32,344][105620] Updated weights for policy 1, policy_version 1361624 (0.0008) [2023-12-27 01:17:32,971][105692] Updated weights for policy 0, policy_version 1359668 (0.0007) [2023-12-27 01:17:33,019][105692] Updated weights for policy 0, policy_version 1359678 (0.0010) [2023-12-27 01:17:33,079][105692] Updated weights for policy 0, policy_version 1359688 (0.0007) [2023-12-27 01:17:33,145][105620] Updated weights for policy 1, policy_version 1361634 (0.0009) [2023-12-27 01:17:33,199][105620] Updated weights for policy 1, policy_version 1361644 (0.0006) [2023-12-27 01:17:33,254][105620] Updated weights for policy 1, policy_version 1361654 (0.0005) [2023-12-27 01:17:33,318][105620] Updated weights for policy 1, policy_version 1361664 (0.0006) [2023-12-27 01:17:33,641][105692] Updated weights for policy 0, policy_version 1359698 (0.0005) [2023-12-27 01:17:33,706][105692] Updated weights for policy 0, policy_version 1359708 (0.0005) [2023-12-27 01:17:33,769][105692] Updated weights for policy 0, policy_version 1359718 (0.0005) [2023-12-27 01:17:33,820][105692] Updated weights for policy 0, policy_version 1359728 (0.0005) [2023-12-27 01:17:33,858][105620] Updated weights for policy 1, policy_version 1361674 (0.0005) [2023-12-27 01:17:33,908][105620] Updated weights for policy 1, policy_version 1361684 (0.0007) [2023-12-27 01:17:33,965][105620] Updated weights for policy 1, policy_version 1361694 (0.0010) [2023-12-27 01:17:34,367][105692] Updated weights for policy 0, policy_version 1359738 (0.0005) [2023-12-27 01:17:34,437][105692] Updated weights for policy 0, policy_version 1359748 (0.0008) [2023-12-27 01:17:34,507][105692] Updated weights for policy 0, policy_version 1359758 (0.0009) [2023-12-27 01:17:34,579][105620] Updated weights for policy 1, policy_version 1361704 (0.0009) [2023-12-27 01:17:34,645][105620] Updated weights for policy 1, policy_version 1361714 (0.0009) [2023-12-27 01:17:34,707][105620] Updated weights for policy 1, policy_version 1361724 (0.0009) [2023-12-27 01:17:35,215][105692] Updated weights for policy 0, policy_version 1359768 (0.0008) [2023-12-27 01:17:35,272][105692] Updated weights for policy 0, policy_version 1359778 (0.0008) [2023-12-27 01:17:35,325][105692] Updated weights for policy 0, policy_version 1359788 (0.0009) [2023-12-27 01:17:35,427][105620] Updated weights for policy 1, policy_version 1361734 (0.0009) [2023-12-27 01:17:35,483][105620] Updated weights for policy 1, policy_version 1361744 (0.0009) [2023-12-27 01:17:35,539][105620] Updated weights for policy 1, policy_version 1361754 (0.0010) [2023-12-27 01:17:36,033][105692] Updated weights for policy 0, policy_version 1359798 (0.0009) [2023-12-27 01:17:36,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 696811520. Throughput: 0: 9859.4, 1: 9507.4. Samples: 696805080. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:17:36,063][104569] Avg episode reward: [(0, '8177.562'), (1, '8542.898')] [2023-12-27 01:17:36,094][105692] Updated weights for policy 0, policy_version 1359808 (0.0009) [2023-12-27 01:17:36,158][105692] Updated weights for policy 0, policy_version 1359818 (0.0008) [2023-12-27 01:17:36,308][105620] Updated weights for policy 1, policy_version 1361764 (0.0009) [2023-12-27 01:17:36,370][105620] Updated weights for policy 1, policy_version 1361774 (0.0006) [2023-12-27 01:17:36,424][105620] Updated weights for policy 1, policy_version 1361784 (0.0005) [2023-12-27 01:17:36,973][105620] Updated weights for policy 1, policy_version 1361794 (0.0006) [2023-12-27 01:17:37,023][105692] Updated weights for policy 0, policy_version 1359828 (0.0009) [2023-12-27 01:17:37,030][105620] Updated weights for policy 1, policy_version 1361804 (0.0006) [2023-12-27 01:17:37,081][105692] Updated weights for policy 0, policy_version 1359838 (0.0009) [2023-12-27 01:17:37,087][105620] Updated weights for policy 1, policy_version 1361814 (0.0006) [2023-12-27 01:17:37,141][105692] Updated weights for policy 0, policy_version 1359848 (0.0008) [2023-12-27 01:17:37,151][105620] Updated weights for policy 1, policy_version 1361824 (0.0006) [2023-12-27 01:17:37,783][105620] Updated weights for policy 1, policy_version 1361834 (0.0005) [2023-12-27 01:17:37,833][105620] Updated weights for policy 1, policy_version 1361844 (0.0008) [2023-12-27 01:17:37,885][105620] Updated weights for policy 1, policy_version 1361854 (0.0009) [2023-12-27 01:17:37,957][105692] Updated weights for policy 0, policy_version 1359858 (0.0009) [2023-12-27 01:17:38,024][105692] Updated weights for policy 0, policy_version 1359868 (0.0008) [2023-12-27 01:17:38,084][105692] Updated weights for policy 0, policy_version 1359878 (0.0007) [2023-12-27 01:17:38,139][105692] Updated weights for policy 0, policy_version 1359888 (0.0005) [2023-12-27 01:17:38,661][105620] Updated weights for policy 1, policy_version 1361864 (0.0009) [2023-12-27 01:17:38,725][105620] Updated weights for policy 1, policy_version 1361874 (0.0009) [2023-12-27 01:17:38,784][105620] Updated weights for policy 1, policy_version 1361884 (0.0007) [2023-12-27 01:17:38,814][105692] Updated weights for policy 0, policy_version 1359898 (0.0008) [2023-12-27 01:17:38,875][105692] Updated weights for policy 0, policy_version 1359908 (0.0009) [2023-12-27 01:17:38,936][105692] Updated weights for policy 0, policy_version 1359918 (0.0009) [2023-12-27 01:17:39,524][105620] Updated weights for policy 1, policy_version 1361894 (0.0007) [2023-12-27 01:17:39,590][105620] Updated weights for policy 1, policy_version 1361904 (0.0009) [2023-12-27 01:17:39,645][105620] Updated weights for policy 1, policy_version 1361914 (0.0009) [2023-12-27 01:17:39,694][105692] Updated weights for policy 0, policy_version 1359928 (0.0008) [2023-12-27 01:17:39,754][105692] Updated weights for policy 0, policy_version 1359938 (0.0009) [2023-12-27 01:17:39,827][105692] Updated weights for policy 0, policy_version 1359948 (0.0010) [2023-12-27 01:17:40,369][105620] Updated weights for policy 1, policy_version 1361924 (0.0009) [2023-12-27 01:17:40,426][105620] Updated weights for policy 1, policy_version 1361934 (0.0009) [2023-12-27 01:17:40,488][105620] Updated weights for policy 1, policy_version 1361944 (0.0008) [2023-12-27 01:17:40,632][105692] Updated weights for policy 0, policy_version 1359958 (0.0009) [2023-12-27 01:17:40,702][105692] Updated weights for policy 0, policy_version 1359968 (0.0009) [2023-12-27 01:17:40,764][105692] Updated weights for policy 0, policy_version 1359978 (0.0009) [2023-12-27 01:17:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 696909824. Throughput: 0: 9774.5, 1: 9504.7. Samples: 696918696. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:17:41,063][104569] Avg episode reward: [(0, '8442.919'), (1, '8719.114')] [2023-12-27 01:17:41,228][105620] Updated weights for policy 1, policy_version 1361954 (0.0009) [2023-12-27 01:17:41,293][105620] Updated weights for policy 1, policy_version 1361964 (0.0010) [2023-12-27 01:17:41,359][105620] Updated weights for policy 1, policy_version 1361974 (0.0009) [2023-12-27 01:17:41,425][105620] Updated weights for policy 1, policy_version 1361984 (0.0006) [2023-12-27 01:17:41,594][105692] Updated weights for policy 0, policy_version 1359988 (0.0010) [2023-12-27 01:17:41,660][105692] Updated weights for policy 0, policy_version 1359998 (0.0009) [2023-12-27 01:17:41,744][105692] Updated weights for policy 0, policy_version 1360008 (0.0010) [2023-12-27 01:17:42,143][105620] Updated weights for policy 1, policy_version 1361994 (0.0009) [2023-12-27 01:17:42,200][105620] Updated weights for policy 1, policy_version 1362004 (0.0007) [2023-12-27 01:17:42,264][105620] Updated weights for policy 1, policy_version 1362014 (0.0008) [2023-12-27 01:17:42,452][105692] Updated weights for policy 0, policy_version 1360018 (0.0010) [2023-12-27 01:17:42,500][105692] Updated weights for policy 0, policy_version 1360028 (0.0009) [2023-12-27 01:17:42,551][105692] Updated weights for policy 0, policy_version 1360038 (0.0009) [2023-12-27 01:17:42,612][105692] Updated weights for policy 0, policy_version 1360048 (0.0009) [2023-12-27 01:17:42,915][105620] Updated weights for policy 1, policy_version 1362024 (0.0006) [2023-12-27 01:17:42,984][105620] Updated weights for policy 1, policy_version 1362034 (0.0005) [2023-12-27 01:17:43,051][105620] Updated weights for policy 1, policy_version 1362044 (0.0006) [2023-12-27 01:17:43,372][105692] Updated weights for policy 0, policy_version 1360058 (0.0008) [2023-12-27 01:17:43,436][105692] Updated weights for policy 0, policy_version 1360068 (0.0008) [2023-12-27 01:17:43,498][105692] Updated weights for policy 0, policy_version 1360078 (0.0009) [2023-12-27 01:17:43,673][105620] Updated weights for policy 1, policy_version 1362054 (0.0007) [2023-12-27 01:17:43,725][105620] Updated weights for policy 1, policy_version 1362064 (0.0009) [2023-12-27 01:17:43,780][105620] Updated weights for policy 1, policy_version 1362074 (0.0009) [2023-12-27 01:17:44,204][105692] Updated weights for policy 0, policy_version 1360088 (0.0009) [2023-12-27 01:17:44,255][105692] Updated weights for policy 0, policy_version 1360098 (0.0009) [2023-12-27 01:17:44,310][105692] Updated weights for policy 0, policy_version 1360108 (0.0009) [2023-12-27 01:17:44,567][105620] Updated weights for policy 1, policy_version 1362084 (0.0009) [2023-12-27 01:17:44,619][105620] Updated weights for policy 1, policy_version 1362094 (0.0009) [2023-12-27 01:17:44,680][105620] Updated weights for policy 1, policy_version 1362104 (0.0009) [2023-12-27 01:17:45,052][105692] Updated weights for policy 0, policy_version 1360118 (0.0009) [2023-12-27 01:17:45,101][105692] Updated weights for policy 0, policy_version 1360128 (0.0009) [2023-12-27 01:17:45,159][105692] Updated weights for policy 0, policy_version 1360138 (0.0008) [2023-12-27 01:17:45,447][105620] Updated weights for policy 1, policy_version 1362114 (0.0009) [2023-12-27 01:17:45,510][105620] Updated weights for policy 1, policy_version 1362124 (0.0008) [2023-12-27 01:17:45,568][105620] Updated weights for policy 1, policy_version 1362134 (0.0009) [2023-12-27 01:17:45,627][105620] Updated weights for policy 1, policy_version 1362144 (0.0009) [2023-12-27 01:17:45,984][105692] Updated weights for policy 0, policy_version 1360148 (0.0010) [2023-12-27 01:17:46,054][105692] Updated weights for policy 0, policy_version 1360158 (0.0009) [2023-12-27 01:17:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 696999936. Throughput: 0: 9655.6, 1: 9471.7. Samples: 696975288. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:17:46,062][104569] Avg episode reward: [(0, '8439.709'), (1, '8903.129')] [2023-12-27 01:17:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001362144_348749824.pth... [2023-12-27 01:17:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001361056_348471296.pth [2023-12-27 01:17:46,113][105692] Updated weights for policy 0, policy_version 1360170 (0.0010) [2023-12-27 01:17:46,142][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001360176_348258304.pth... [2023-12-27 01:17:46,145][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001358992_347955200.pth [2023-12-27 01:17:46,235][105620] Updated weights for policy 1, policy_version 1362154 (0.0005) [2023-12-27 01:17:46,296][105620] Updated weights for policy 1, policy_version 1362164 (0.0005) [2023-12-27 01:17:46,345][105620] Updated weights for policy 1, policy_version 1362174 (0.0005) [2023-12-27 01:17:46,863][105692] Updated weights for policy 0, policy_version 1360180 (0.0009) [2023-12-27 01:17:46,911][105692] Updated weights for policy 0, policy_version 1360190 (0.0008) [2023-12-27 01:17:46,925][105620] Updated weights for policy 1, policy_version 1362184 (0.0006) [2023-12-27 01:17:46,966][105692] Updated weights for policy 0, policy_version 1360200 (0.0008) [2023-12-27 01:17:46,988][105620] Updated weights for policy 1, policy_version 1362194 (0.0006) [2023-12-27 01:17:47,052][105620] Updated weights for policy 1, policy_version 1362204 (0.0007) [2023-12-27 01:17:47,739][105692] Updated weights for policy 0, policy_version 1360210 (0.0008) [2023-12-27 01:17:47,794][105692] Updated weights for policy 0, policy_version 1360220 (0.0009) [2023-12-27 01:17:47,796][105620] Updated weights for policy 1, policy_version 1362214 (0.0008) [2023-12-27 01:17:47,857][105692] Updated weights for policy 0, policy_version 1360230 (0.0007) [2023-12-27 01:17:47,859][105620] Updated weights for policy 1, policy_version 1362224 (0.0006) [2023-12-27 01:17:47,917][105692] Updated weights for policy 0, policy_version 1360240 (0.0007) [2023-12-27 01:17:47,919][105620] Updated weights for policy 1, policy_version 1362234 (0.0006) [2023-12-27 01:17:48,668][105620] Updated weights for policy 1, policy_version 1362244 (0.0007) [2023-12-27 01:17:48,673][105692] Updated weights for policy 0, policy_version 1360250 (0.0011) [2023-12-27 01:17:48,728][105620] Updated weights for policy 1, policy_version 1362254 (0.0005) [2023-12-27 01:17:48,733][105692] Updated weights for policy 0, policy_version 1360260 (0.0011) [2023-12-27 01:17:48,789][105620] Updated weights for policy 1, policy_version 1362264 (0.0006) [2023-12-27 01:17:48,793][105692] Updated weights for policy 0, policy_version 1360270 (0.0011) [2023-12-27 01:17:49,527][105692] Updated weights for policy 0, policy_version 1360280 (0.0009) [2023-12-27 01:17:49,537][105620] Updated weights for policy 1, policy_version 1362274 (0.0008) [2023-12-27 01:17:49,588][105692] Updated weights for policy 0, policy_version 1360290 (0.0006) [2023-12-27 01:17:49,601][105620] Updated weights for policy 1, policy_version 1362284 (0.0008) [2023-12-27 01:17:49,643][105692] Updated weights for policy 0, policy_version 1360300 (0.0005) [2023-12-27 01:17:49,660][105620] Updated weights for policy 1, policy_version 1362294 (0.0009) [2023-12-27 01:17:49,715][105620] Updated weights for policy 1, policy_version 1362304 (0.0008) [2023-12-27 01:17:50,349][105692] Updated weights for policy 0, policy_version 1360310 (0.0005) [2023-12-27 01:17:50,415][105692] Updated weights for policy 0, policy_version 1360320 (0.0007) [2023-12-27 01:17:50,476][105692] Updated weights for policy 0, policy_version 1360330 (0.0008) [2023-12-27 01:17:50,521][105620] Updated weights for policy 1, policy_version 1362314 (0.0008) [2023-12-27 01:17:50,593][105620] Updated weights for policy 1, policy_version 1362324 (0.0009) [2023-12-27 01:17:50,659][105620] Updated weights for policy 1, policy_version 1362334 (0.0008) [2023-12-27 01:17:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 697098240. Throughput: 0: 9663.7, 1: 9513.3. Samples: 697089704. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:17:51,063][104569] Avg episode reward: [(0, '8359.806'), (1, '9086.237')] [2023-12-27 01:17:51,118][105692] Updated weights for policy 0, policy_version 1360340 (0.0007) [2023-12-27 01:17:51,183][105692] Updated weights for policy 0, policy_version 1360350 (0.0008) [2023-12-27 01:17:51,247][105692] Updated weights for policy 0, policy_version 1360360 (0.0008) [2023-12-27 01:17:51,471][105620] Updated weights for policy 1, policy_version 1362344 (0.0008) [2023-12-27 01:17:51,522][105620] Updated weights for policy 1, policy_version 1362354 (0.0009) [2023-12-27 01:17:51,571][105620] Updated weights for policy 1, policy_version 1362364 (0.0008) [2023-12-27 01:17:51,942][105692] Updated weights for policy 0, policy_version 1360370 (0.0008) [2023-12-27 01:17:52,005][105692] Updated weights for policy 0, policy_version 1360380 (0.0008) [2023-12-27 01:17:52,067][105692] Updated weights for policy 0, policy_version 1360390 (0.0005) [2023-12-27 01:17:52,126][105692] Updated weights for policy 0, policy_version 1360400 (0.0007) [2023-12-27 01:17:52,427][105620] Updated weights for policy 1, policy_version 1362374 (0.0008) [2023-12-27 01:17:52,488][105620] Updated weights for policy 1, policy_version 1362384 (0.0008) [2023-12-27 01:17:52,551][105620] Updated weights for policy 1, policy_version 1362394 (0.0006) [2023-12-27 01:17:52,860][105692] Updated weights for policy 0, policy_version 1360410 (0.0010) [2023-12-27 01:17:52,908][105692] Updated weights for policy 0, policy_version 1360420 (0.0010) [2023-12-27 01:17:52,963][105692] Updated weights for policy 0, policy_version 1360430 (0.0010) [2023-12-27 01:17:53,301][105620] Updated weights for policy 1, policy_version 1362404 (0.0008) [2023-12-27 01:17:53,360][105620] Updated weights for policy 1, policy_version 1362414 (0.0008) [2023-12-27 01:17:53,405][105620] Updated weights for policy 1, policy_version 1362424 (0.0008) [2023-12-27 01:17:53,715][105692] Updated weights for policy 0, policy_version 1360440 (0.0010) [2023-12-27 01:17:53,763][105692] Updated weights for policy 0, policy_version 1360450 (0.0010) [2023-12-27 01:17:53,814][105692] Updated weights for policy 0, policy_version 1360460 (0.0010) [2023-12-27 01:17:54,170][105620] Updated weights for policy 1, policy_version 1362434 (0.0008) [2023-12-27 01:17:54,226][105620] Updated weights for policy 1, policy_version 1362444 (0.0008) [2023-12-27 01:17:54,277][105620] Updated weights for policy 1, policy_version 1362454 (0.0008) [2023-12-27 01:17:54,329][105620] Updated weights for policy 1, policy_version 1362464 (0.0008) [2023-12-27 01:17:54,573][105692] Updated weights for policy 0, policy_version 1360470 (0.0009) [2023-12-27 01:17:54,631][105692] Updated weights for policy 0, policy_version 1360480 (0.0009) [2023-12-27 01:17:54,690][105692] Updated weights for policy 0, policy_version 1360490 (0.0009) [2023-12-27 01:17:55,057][105620] Updated weights for policy 1, policy_version 1362474 (0.0006) [2023-12-27 01:17:55,123][105620] Updated weights for policy 1, policy_version 1362484 (0.0005) [2023-12-27 01:17:55,182][105620] Updated weights for policy 1, policy_version 1362494 (0.0005) [2023-12-27 01:17:55,357][105692] Updated weights for policy 0, policy_version 1360500 (0.0005) [2023-12-27 01:17:55,413][105692] Updated weights for policy 0, policy_version 1360510 (0.0005) [2023-12-27 01:17:55,466][105692] Updated weights for policy 0, policy_version 1360520 (0.0005) [2023-12-27 01:17:55,703][105620] Updated weights for policy 1, policy_version 1362504 (0.0005) [2023-12-27 01:17:55,771][105620] Updated weights for policy 1, policy_version 1362514 (0.0005) [2023-12-27 01:17:55,835][105620] Updated weights for policy 1, policy_version 1362524 (0.0006) [2023-12-27 01:17:56,001][105692] Updated weights for policy 0, policy_version 1360530 (0.0006) [2023-12-27 01:17:56,056][105692] Updated weights for policy 0, policy_version 1360540 (0.0010) [2023-12-27 01:17:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.3, 300 sec: 19549.7). Total num frames: 697196544. Throughput: 0: 9738.7, 1: 9538.2. Samples: 697206656. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:17:56,062][104569] Avg episode reward: [(0, '8628.145'), (1, '9085.697')] [2023-12-27 01:17:56,112][105692] Updated weights for policy 0, policy_version 1360550 (0.0009) [2023-12-27 01:17:56,165][105692] Updated weights for policy 0, policy_version 1360560 (0.0007) [2023-12-27 01:17:56,426][105620] Updated weights for policy 1, policy_version 1362534 (0.0008) [2023-12-27 01:17:56,476][105620] Updated weights for policy 1, policy_version 1362544 (0.0008) [2023-12-27 01:17:56,523][105620] Updated weights for policy 1, policy_version 1362554 (0.0009) [2023-12-27 01:17:56,803][105692] Updated weights for policy 0, policy_version 1360570 (0.0008) [2023-12-27 01:17:56,850][105692] Updated weights for policy 0, policy_version 1360580 (0.0009) [2023-12-27 01:17:56,904][105692] Updated weights for policy 0, policy_version 1360590 (0.0009) [2023-12-27 01:17:57,362][105620] Updated weights for policy 1, policy_version 1362564 (0.0009) [2023-12-27 01:17:57,423][105620] Updated weights for policy 1, policy_version 1362574 (0.0009) [2023-12-27 01:17:57,482][105620] Updated weights for policy 1, policy_version 1362584 (0.0009) [2023-12-27 01:17:57,548][105692] Updated weights for policy 0, policy_version 1360600 (0.0009) [2023-12-27 01:17:57,602][105692] Updated weights for policy 0, policy_version 1360610 (0.0009) [2023-12-27 01:17:57,662][105692] Updated weights for policy 0, policy_version 1360620 (0.0009) [2023-12-27 01:17:58,255][105620] Updated weights for policy 1, policy_version 1362594 (0.0006) [2023-12-27 01:17:58,320][105620] Updated weights for policy 1, policy_version 1362604 (0.0007) [2023-12-27 01:17:58,360][105692] Updated weights for policy 0, policy_version 1360630 (0.0008) [2023-12-27 01:17:58,388][105620] Updated weights for policy 1, policy_version 1362614 (0.0008) [2023-12-27 01:17:58,422][105692] Updated weights for policy 0, policy_version 1360640 (0.0008) [2023-12-27 01:17:58,456][105620] Updated weights for policy 1, policy_version 1362624 (0.0008) [2023-12-27 01:17:58,484][105692] Updated weights for policy 0, policy_version 1360650 (0.0007) [2023-12-27 01:17:59,196][105692] Updated weights for policy 0, policy_version 1360660 (0.0009) [2023-12-27 01:17:59,260][105692] Updated weights for policy 0, policy_version 1360670 (0.0008) [2023-12-27 01:17:59,327][105692] Updated weights for policy 0, policy_version 1360680 (0.0008) [2023-12-27 01:17:59,350][105620] Updated weights for policy 1, policy_version 1362634 (0.0008) [2023-12-27 01:17:59,417][105620] Updated weights for policy 1, policy_version 1362644 (0.0008) [2023-12-27 01:17:59,475][105620] Updated weights for policy 1, policy_version 1362654 (0.0006) [2023-12-27 01:18:00,118][105692] Updated weights for policy 0, policy_version 1360690 (0.0007) [2023-12-27 01:18:00,120][105620] Updated weights for policy 1, policy_version 1362664 (0.0009) [2023-12-27 01:18:00,173][105692] Updated weights for policy 0, policy_version 1360700 (0.0006) [2023-12-27 01:18:00,182][105620] Updated weights for policy 1, policy_version 1362674 (0.0008) [2023-12-27 01:18:00,229][105692] Updated weights for policy 0, policy_version 1360710 (0.0007) [2023-12-27 01:18:00,239][105620] Updated weights for policy 1, policy_version 1362684 (0.0006) [2023-12-27 01:18:00,292][105692] Updated weights for policy 0, policy_version 1360720 (0.0007) [2023-12-27 01:18:00,895][105620] Updated weights for policy 1, policy_version 1362694 (0.0006) [2023-12-27 01:18:00,952][105620] Updated weights for policy 1, policy_version 1362704 (0.0006) [2023-12-27 01:18:01,005][105620] Updated weights for policy 1, policy_version 1362714 (0.0005) [2023-12-27 01:18:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 697294848. Throughput: 0: 9845.4, 1: 9476.3. Samples: 697265068. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:18:01,062][104569] Avg episode reward: [(0, '8717.068'), (1, '9174.720')] [2023-12-27 01:18:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001362720_348897280.pth... [2023-12-27 01:18:01,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001361568_348602368.pth [2023-12-27 01:18:01,114][105692] Updated weights for policy 0, policy_version 1360730 (0.0010) [2023-12-27 01:18:01,177][105692] Updated weights for policy 0, policy_version 1360740 (0.0008) [2023-12-27 01:18:01,238][105692] Updated weights for policy 0, policy_version 1360750 (0.0009) [2023-12-27 01:18:01,250][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001360752_348405760.pth... [2023-12-27 01:18:01,255][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001359600_348110848.pth [2023-12-27 01:18:01,682][105620] Updated weights for policy 1, policy_version 1362724 (0.0008) [2023-12-27 01:18:01,740][105620] Updated weights for policy 1, policy_version 1362734 (0.0008) [2023-12-27 01:18:01,792][105620] Updated weights for policy 1, policy_version 1362744 (0.0008) [2023-12-27 01:18:02,040][105692] Updated weights for policy 0, policy_version 1360760 (0.0009) [2023-12-27 01:18:02,100][105692] Updated weights for policy 0, policy_version 1360770 (0.0009) [2023-12-27 01:18:02,151][105692] Updated weights for policy 0, policy_version 1360780 (0.0008) [2023-12-27 01:18:02,566][105620] Updated weights for policy 1, policy_version 1362754 (0.0009) [2023-12-27 01:18:02,632][105620] Updated weights for policy 1, policy_version 1362764 (0.0011) [2023-12-27 01:18:02,701][105620] Updated weights for policy 1, policy_version 1362774 (0.0011) [2023-12-27 01:18:02,756][105620] Updated weights for policy 1, policy_version 1362784 (0.0010) [2023-12-27 01:18:02,832][105692] Updated weights for policy 0, policy_version 1360790 (0.0007) [2023-12-27 01:18:02,899][105692] Updated weights for policy 0, policy_version 1360800 (0.0008) [2023-12-27 01:18:02,965][105692] Updated weights for policy 0, policy_version 1360810 (0.0009) [2023-12-27 01:18:03,470][105620] Updated weights for policy 1, policy_version 1362794 (0.0010) [2023-12-27 01:18:03,519][105620] Updated weights for policy 1, policy_version 1362804 (0.0008) [2023-12-27 01:18:03,582][105620] Updated weights for policy 1, policy_version 1362814 (0.0005) [2023-12-27 01:18:03,583][105692] Updated weights for policy 0, policy_version 1360820 (0.0009) [2023-12-27 01:18:03,638][105692] Updated weights for policy 0, policy_version 1360830 (0.0009) [2023-12-27 01:18:03,702][105692] Updated weights for policy 0, policy_version 1360840 (0.0005) [2023-12-27 01:18:04,248][105620] Updated weights for policy 1, policy_version 1362824 (0.0007) [2023-12-27 01:18:04,309][105620] Updated weights for policy 1, policy_version 1362834 (0.0008) [2023-12-27 01:18:04,374][105692] Updated weights for policy 0, policy_version 1360850 (0.0005) [2023-12-27 01:18:04,375][105620] Updated weights for policy 1, policy_version 1362844 (0.0011) [2023-12-27 01:18:04,441][105692] Updated weights for policy 0, policy_version 1360860 (0.0006) [2023-12-27 01:18:04,507][105692] Updated weights for policy 0, policy_version 1360870 (0.0007) [2023-12-27 01:18:04,575][105692] Updated weights for policy 0, policy_version 1360880 (0.0008) [2023-12-27 01:18:05,103][105620] Updated weights for policy 1, policy_version 1362854 (0.0010) [2023-12-27 01:18:05,154][105620] Updated weights for policy 1, policy_version 1362864 (0.0010) [2023-12-27 01:18:05,202][105620] Updated weights for policy 1, policy_version 1362874 (0.0010) [2023-12-27 01:18:05,239][105692] Updated weights for policy 0, policy_version 1360890 (0.0005) [2023-12-27 01:18:05,284][105692] Updated weights for policy 0, policy_version 1360900 (0.0008) [2023-12-27 01:18:05,332][105692] Updated weights for policy 0, policy_version 1360910 (0.0008) [2023-12-27 01:18:05,958][105620] Updated weights for policy 1, policy_version 1362884 (0.0010) [2023-12-27 01:18:05,971][105692] Updated weights for policy 0, policy_version 1360920 (0.0010) [2023-12-27 01:18:06,019][105620] Updated weights for policy 1, policy_version 1362894 (0.0010) [2023-12-27 01:18:06,025][105692] Updated weights for policy 0, policy_version 1360930 (0.0010) [2023-12-27 01:18:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19522.0). Total num frames: 697384960. Throughput: 0: 9847.8, 1: 9499.0. Samples: 697381144. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:18:06,062][104569] Avg episode reward: [(0, '8534.866'), (1, '9263.941')] [2023-12-27 01:18:06,074][105620] Updated weights for policy 1, policy_version 1362904 (0.0010) [2023-12-27 01:18:06,081][105692] Updated weights for policy 0, policy_version 1360940 (0.0010) [2023-12-27 01:18:06,796][105620] Updated weights for policy 1, policy_version 1362914 (0.0008) [2023-12-27 01:18:06,843][105692] Updated weights for policy 0, policy_version 1360950 (0.0011) [2023-12-27 01:18:06,859][105620] Updated weights for policy 1, policy_version 1362924 (0.0007) [2023-12-27 01:18:06,895][105692] Updated weights for policy 0, policy_version 1360960 (0.0010) [2023-12-27 01:18:06,918][105620] Updated weights for policy 1, policy_version 1362934 (0.0006) [2023-12-27 01:18:06,954][105692] Updated weights for policy 0, policy_version 1360970 (0.0010) [2023-12-27 01:18:06,971][105620] Updated weights for policy 1, policy_version 1362944 (0.0005) [2023-12-27 01:18:07,627][105620] Updated weights for policy 1, policy_version 1362954 (0.0010) [2023-12-27 01:18:07,679][105620] Updated weights for policy 1, policy_version 1362964 (0.0010) [2023-12-27 01:18:07,718][105692] Updated weights for policy 0, policy_version 1360980 (0.0008) [2023-12-27 01:18:07,732][105620] Updated weights for policy 1, policy_version 1362974 (0.0010) [2023-12-27 01:18:07,775][105692] Updated weights for policy 0, policy_version 1360990 (0.0007) [2023-12-27 01:18:07,834][105692] Updated weights for policy 0, policy_version 1361000 (0.0008) [2023-12-27 01:18:08,367][105620] Updated weights for policy 1, policy_version 1362984 (0.0010) [2023-12-27 01:18:08,432][105620] Updated weights for policy 1, policy_version 1362994 (0.0008) [2023-12-27 01:18:08,492][105620] Updated weights for policy 1, policy_version 1363004 (0.0011) [2023-12-27 01:18:08,652][105692] Updated weights for policy 0, policy_version 1361010 (0.0007) [2023-12-27 01:18:08,712][105692] Updated weights for policy 0, policy_version 1361022 (0.0009) [2023-12-27 01:18:08,766][105692] Updated weights for policy 0, policy_version 1361033 (0.0010) [2023-12-27 01:18:09,168][105620] Updated weights for policy 1, policy_version 1363014 (0.0008) [2023-12-27 01:18:09,223][105620] Updated weights for policy 1, policy_version 1363024 (0.0009) [2023-12-27 01:18:09,286][105620] Updated weights for policy 1, policy_version 1363034 (0.0011) [2023-12-27 01:18:09,576][105692] Updated weights for policy 0, policy_version 1361044 (0.0008) [2023-12-27 01:18:09,631][105692] Updated weights for policy 0, policy_version 1361054 (0.0006) [2023-12-27 01:18:09,690][105692] Updated weights for policy 0, policy_version 1361064 (0.0006) [2023-12-27 01:18:09,998][105620] Updated weights for policy 1, policy_version 1363044 (0.0011) [2023-12-27 01:18:10,061][105620] Updated weights for policy 1, policy_version 1363054 (0.0010) [2023-12-27 01:18:10,119][105620] Updated weights for policy 1, policy_version 1363064 (0.0009) [2023-12-27 01:18:10,312][105692] Updated weights for policy 0, policy_version 1361074 (0.0006) [2023-12-27 01:18:10,372][105692] Updated weights for policy 0, policy_version 1361084 (0.0005) [2023-12-27 01:18:10,437][105692] Updated weights for policy 0, policy_version 1361094 (0.0007) [2023-12-27 01:18:10,503][105692] Updated weights for policy 0, policy_version 1361104 (0.0006) [2023-12-27 01:18:10,869][105620] Updated weights for policy 1, policy_version 1363074 (0.0011) [2023-12-27 01:18:10,931][105620] Updated weights for policy 1, policy_version 1363085 (0.0010) [2023-12-27 01:18:10,989][105620] Updated weights for policy 1, policy_version 1363095 (0.0010) [2023-12-27 01:18:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 697491456. Throughput: 0: 9767.6, 1: 9652.5. Samples: 697498708. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:18:11,062][104569] Avg episode reward: [(0, '8263.548'), (1, '8993.899')] [2023-12-27 01:18:11,170][105692] Updated weights for policy 0, policy_version 1361114 (0.0008) [2023-12-27 01:18:11,235][105692] Updated weights for policy 0, policy_version 1361124 (0.0009) [2023-12-27 01:18:11,299][105692] Updated weights for policy 0, policy_version 1361134 (0.0009) [2023-12-27 01:18:11,755][105620] Updated weights for policy 1, policy_version 1363105 (0.0010) [2023-12-27 01:18:11,818][105620] Updated weights for policy 1, policy_version 1363115 (0.0010) [2023-12-27 01:18:11,871][105620] Updated weights for policy 1, policy_version 1363125 (0.0010) [2023-12-27 01:18:11,924][105620] Updated weights for policy 1, policy_version 1363135 (0.0010) [2023-12-27 01:18:12,114][105692] Updated weights for policy 0, policy_version 1361144 (0.0007) [2023-12-27 01:18:12,176][105692] Updated weights for policy 0, policy_version 1361154 (0.0007) [2023-12-27 01:18:12,243][105692] Updated weights for policy 0, policy_version 1361164 (0.0006) [2023-12-27 01:18:12,708][105620] Updated weights for policy 1, policy_version 1363145 (0.0009) [2023-12-27 01:18:12,767][105620] Updated weights for policy 1, policy_version 1363155 (0.0008) [2023-12-27 01:18:12,824][105620] Updated weights for policy 1, policy_version 1363165 (0.0009) [2023-12-27 01:18:12,919][105692] Updated weights for policy 0, policy_version 1361174 (0.0009) [2023-12-27 01:18:12,974][105692] Updated weights for policy 0, policy_version 1361184 (0.0009) [2023-12-27 01:18:13,022][105692] Updated weights for policy 0, policy_version 1361194 (0.0009) [2023-12-27 01:18:13,459][105620] Updated weights for policy 1, policy_version 1363175 (0.0008) [2023-12-27 01:18:13,514][105620] Updated weights for policy 1, policy_version 1363185 (0.0009) [2023-12-27 01:18:13,563][105620] Updated weights for policy 1, policy_version 1363195 (0.0007) [2023-12-27 01:18:13,727][105692] Updated weights for policy 0, policy_version 1361204 (0.0008) [2023-12-27 01:18:13,776][105692] Updated weights for policy 0, policy_version 1361214 (0.0005) [2023-12-27 01:18:13,837][105692] Updated weights for policy 0, policy_version 1361224 (0.0006) [2023-12-27 01:18:14,172][105620] Updated weights for policy 1, policy_version 1363205 (0.0006) [2023-12-27 01:18:14,220][105620] Updated weights for policy 1, policy_version 1363215 (0.0010) [2023-12-27 01:18:14,267][105620] Updated weights for policy 1, policy_version 1363225 (0.0010) [2023-12-27 01:18:14,502][105692] Updated weights for policy 0, policy_version 1361234 (0.0007) [2023-12-27 01:18:14,559][105692] Updated weights for policy 0, policy_version 1361244 (0.0009) [2023-12-27 01:18:14,615][105692] Updated weights for policy 0, policy_version 1361255 (0.0010) [2023-12-27 01:18:14,926][105620] Updated weights for policy 1, policy_version 1363235 (0.0009) [2023-12-27 01:18:14,989][105620] Updated weights for policy 1, policy_version 1363245 (0.0008) [2023-12-27 01:18:15,056][105620] Updated weights for policy 1, policy_version 1363255 (0.0008) [2023-12-27 01:18:15,339][105692] Updated weights for policy 0, policy_version 1361265 (0.0006) [2023-12-27 01:18:15,400][105692] Updated weights for policy 0, policy_version 1361275 (0.0008) [2023-12-27 01:18:15,460][105692] Updated weights for policy 0, policy_version 1361285 (0.0008) [2023-12-27 01:18:15,529][105692] Updated weights for policy 0, policy_version 1361295 (0.0008) [2023-12-27 01:18:15,809][105620] Updated weights for policy 1, policy_version 1363265 (0.0008) [2023-12-27 01:18:15,860][105620] Updated weights for policy 1, policy_version 1363275 (0.0010) [2023-12-27 01:18:15,908][105620] Updated weights for policy 1, policy_version 1363285 (0.0010) [2023-12-27 01:18:15,963][105620] Updated weights for policy 1, policy_version 1363295 (0.0010) [2023-12-27 01:18:16,062][104569] Fps is (10 sec: 20479.2, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 697589760. Throughput: 0: 9671.1, 1: 9729.9. Samples: 697556400. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:18:16,063][104569] Avg episode reward: [(0, '7893.728'), (1, '8903.697')] [2023-12-27 01:18:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001361296_348545024.pth... [2023-12-27 01:18:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001363296_349044736.pth... [2023-12-27 01:18:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001360176_348258304.pth [2023-12-27 01:18:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001362144_348749824.pth [2023-12-27 01:18:16,283][105692] Updated weights for policy 0, policy_version 1361305 (0.0007) [2023-12-27 01:18:16,341][105692] Updated weights for policy 0, policy_version 1361316 (0.0009) [2023-12-27 01:18:16,387][105692] Updated weights for policy 0, policy_version 1361326 (0.0008) [2023-12-27 01:18:16,708][105620] Updated weights for policy 1, policy_version 1363305 (0.0010) [2023-12-27 01:18:16,768][105620] Updated weights for policy 1, policy_version 1363315 (0.0009) [2023-12-27 01:18:16,823][105620] Updated weights for policy 1, policy_version 1363325 (0.0010) [2023-12-27 01:18:17,175][105692] Updated weights for policy 0, policy_version 1361336 (0.0009) [2023-12-27 01:18:17,236][105692] Updated weights for policy 0, policy_version 1361346 (0.0009) [2023-12-27 01:18:17,296][105692] Updated weights for policy 0, policy_version 1361356 (0.0009) [2023-12-27 01:18:17,508][105620] Updated weights for policy 1, policy_version 1363335 (0.0010) [2023-12-27 01:18:17,567][105620] Updated weights for policy 1, policy_version 1363345 (0.0010) [2023-12-27 01:18:17,625][105620] Updated weights for policy 1, policy_version 1363355 (0.0010) [2023-12-27 01:18:18,090][105692] Updated weights for policy 0, policy_version 1361366 (0.0010) [2023-12-27 01:18:18,141][105692] Updated weights for policy 0, policy_version 1361376 (0.0010) [2023-12-27 01:18:18,189][105692] Updated weights for policy 0, policy_version 1361386 (0.0010) [2023-12-27 01:18:18,278][105620] Updated weights for policy 1, policy_version 1363365 (0.0010) [2023-12-27 01:18:18,326][105620] Updated weights for policy 1, policy_version 1363375 (0.0007) [2023-12-27 01:18:18,389][105620] Updated weights for policy 1, policy_version 1363385 (0.0009) [2023-12-27 01:18:18,808][105692] Updated weights for policy 0, policy_version 1361396 (0.0008) [2023-12-27 01:18:18,870][105692] Updated weights for policy 0, policy_version 1361406 (0.0011) [2023-12-27 01:18:18,928][105692] Updated weights for policy 0, policy_version 1361416 (0.0010) [2023-12-27 01:18:19,228][105620] Updated weights for policy 1, policy_version 1363395 (0.0009) [2023-12-27 01:18:19,291][105620] Updated weights for policy 1, policy_version 1363405 (0.0007) [2023-12-27 01:18:19,357][105620] Updated weights for policy 1, policy_version 1363415 (0.0007) [2023-12-27 01:18:19,618][105692] Updated weights for policy 0, policy_version 1361426 (0.0006) [2023-12-27 01:18:19,675][105692] Updated weights for policy 0, policy_version 1361437 (0.0010) [2023-12-27 01:18:19,735][105692] Updated weights for policy 0, policy_version 1361447 (0.0006) [2023-12-27 01:18:20,052][105620] Updated weights for policy 1, policy_version 1363425 (0.0010) [2023-12-27 01:18:20,118][105620] Updated weights for policy 1, policy_version 1363435 (0.0006) [2023-12-27 01:18:20,184][105620] Updated weights for policy 1, policy_version 1363445 (0.0006) [2023-12-27 01:18:20,253][105620] Updated weights for policy 1, policy_version 1363455 (0.0009) [2023-12-27 01:18:20,415][105692] Updated weights for policy 0, policy_version 1361457 (0.0007) [2023-12-27 01:18:20,476][105692] Updated weights for policy 0, policy_version 1361467 (0.0007) [2023-12-27 01:18:20,532][105692] Updated weights for policy 0, policy_version 1361477 (0.0005) [2023-12-27 01:18:20,598][105692] Updated weights for policy 0, policy_version 1361487 (0.0009) [2023-12-27 01:18:20,924][105620] Updated weights for policy 1, policy_version 1363465 (0.0010) [2023-12-27 01:18:20,997][105620] Updated weights for policy 1, policy_version 1363475 (0.0007) [2023-12-27 01:18:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 697679872. Throughput: 0: 9579.0, 1: 9724.5. Samples: 697673740. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:18:21,063][104569] Avg episode reward: [(0, '8262.864'), (1, '9080.729')] [2023-12-27 01:18:21,064][105620] Updated weights for policy 1, policy_version 1363485 (0.0010) [2023-12-27 01:18:21,225][105692] Updated weights for policy 0, policy_version 1361497 (0.0009) [2023-12-27 01:18:21,284][105692] Updated weights for policy 0, policy_version 1361507 (0.0009) [2023-12-27 01:18:21,334][105692] Updated weights for policy 0, policy_version 1361517 (0.0009) [2023-12-27 01:18:21,873][105620] Updated weights for policy 1, policy_version 1363495 (0.0009) [2023-12-27 01:18:21,924][105620] Updated weights for policy 1, policy_version 1363505 (0.0008) [2023-12-27 01:18:21,976][105620] Updated weights for policy 1, policy_version 1363515 (0.0010) [2023-12-27 01:18:22,084][105692] Updated weights for policy 0, policy_version 1361527 (0.0008) [2023-12-27 01:18:22,144][105692] Updated weights for policy 0, policy_version 1361537 (0.0005) [2023-12-27 01:18:22,183][105585] KL-divergence is very high: 174.9197 [2023-12-27 01:18:22,199][105692] Updated weights for policy 0, policy_version 1361547 (0.0009) [2023-12-27 01:18:22,838][105620] Updated weights for policy 1, policy_version 1363525 (0.0010) [2023-12-27 01:18:22,883][105692] Updated weights for policy 0, policy_version 1361557 (0.0008) [2023-12-27 01:18:22,897][105620] Updated weights for policy 1, policy_version 1363535 (0.0009) [2023-12-27 01:18:22,942][105692] Updated weights for policy 0, policy_version 1361567 (0.0008) [2023-12-27 01:18:22,952][105620] Updated weights for policy 1, policy_version 1363545 (0.0008) [2023-12-27 01:18:22,999][105692] Updated weights for policy 0, policy_version 1361577 (0.0007) [2023-12-27 01:18:23,656][105620] Updated weights for policy 1, policy_version 1363555 (0.0006) [2023-12-27 01:18:23,708][105620] Updated weights for policy 1, policy_version 1363565 (0.0008) [2023-12-27 01:18:23,744][105692] Updated weights for policy 0, policy_version 1361587 (0.0009) [2023-12-27 01:18:23,762][105620] Updated weights for policy 1, policy_version 1363575 (0.0009) [2023-12-27 01:18:23,792][105692] Updated weights for policy 0, policy_version 1361597 (0.0010) [2023-12-27 01:18:23,847][105692] Updated weights for policy 0, policy_version 1361607 (0.0010) [2023-12-27 01:18:24,538][105620] Updated weights for policy 1, policy_version 1363585 (0.0006) [2023-12-27 01:18:24,587][105620] Updated weights for policy 1, policy_version 1363595 (0.0008) [2023-12-27 01:18:24,608][105692] Updated weights for policy 0, policy_version 1361617 (0.0010) [2023-12-27 01:18:24,634][105620] Updated weights for policy 1, policy_version 1363605 (0.0007) [2023-12-27 01:18:24,663][105692] Updated weights for policy 0, policy_version 1361627 (0.0010) [2023-12-27 01:18:24,686][105620] Updated weights for policy 1, policy_version 1363615 (0.0006) [2023-12-27 01:18:24,721][105692] Updated weights for policy 0, policy_version 1361637 (0.0010) [2023-12-27 01:18:24,779][105692] Updated weights for policy 0, policy_version 1361647 (0.0010) [2023-12-27 01:18:25,317][105620] Updated weights for policy 1, policy_version 1363625 (0.0007) [2023-12-27 01:18:25,376][105620] Updated weights for policy 1, policy_version 1363635 (0.0006) [2023-12-27 01:18:25,431][105692] Updated weights for policy 0, policy_version 1361657 (0.0006) [2023-12-27 01:18:25,433][105620] Updated weights for policy 1, policy_version 1363645 (0.0006) [2023-12-27 01:18:25,487][105692] Updated weights for policy 0, policy_version 1361667 (0.0005) [2023-12-27 01:18:25,550][105692] Updated weights for policy 0, policy_version 1361677 (0.0005) [2023-12-27 01:18:26,062][104569] Fps is (10 sec: 18842.3, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 697778176. Throughput: 0: 9698.7, 1: 9676.3. Samples: 697790568. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 01:18:26,062][104569] Avg episode reward: [(0, '8265.411'), (1, '9082.172')] [2023-12-27 01:18:26,071][105692] Updated weights for policy 0, policy_version 1361687 (0.0006) [2023-12-27 01:18:26,097][105620] Updated weights for policy 1, policy_version 1363655 (0.0008) [2023-12-27 01:18:26,127][105692] Updated weights for policy 0, policy_version 1361697 (0.0005) [2023-12-27 01:18:26,158][105620] Updated weights for policy 1, policy_version 1363665 (0.0010) [2023-12-27 01:18:26,185][105692] Updated weights for policy 0, policy_version 1361707 (0.0005) [2023-12-27 01:18:26,221][105620] Updated weights for policy 1, policy_version 1363675 (0.0009) [2023-12-27 01:18:26,695][105692] Updated weights for policy 0, policy_version 1361717 (0.0005) [2023-12-27 01:18:26,746][105692] Updated weights for policy 0, policy_version 1361727 (0.0005) [2023-12-27 01:18:26,796][105692] Updated weights for policy 0, policy_version 1361737 (0.0005) [2023-12-27 01:18:27,124][105620] Updated weights for policy 1, policy_version 1363685 (0.0008) [2023-12-27 01:18:27,180][105620] Updated weights for policy 1, policy_version 1363695 (0.0009) [2023-12-27 01:18:27,240][105620] Updated weights for policy 1, policy_version 1363705 (0.0009) [2023-12-27 01:18:27,306][105692] Updated weights for policy 0, policy_version 1361747 (0.0006) [2023-12-27 01:18:27,360][105692] Updated weights for policy 0, policy_version 1361757 (0.0006) [2023-12-27 01:18:27,418][105692] Updated weights for policy 0, policy_version 1361767 (0.0010) [2023-12-27 01:18:27,948][105620] Updated weights for policy 1, policy_version 1363715 (0.0008) [2023-12-27 01:18:28,007][105620] Updated weights for policy 1, policy_version 1363725 (0.0006) [2023-12-27 01:18:28,064][105620] Updated weights for policy 1, policy_version 1363735 (0.0010) [2023-12-27 01:18:28,113][105692] Updated weights for policy 0, policy_version 1361777 (0.0010) [2023-12-27 01:18:28,164][105692] Updated weights for policy 0, policy_version 1361787 (0.0009) [2023-12-27 01:18:28,234][105692] Updated weights for policy 0, policy_version 1361797 (0.0010) [2023-12-27 01:18:28,296][105692] Updated weights for policy 0, policy_version 1361807 (0.0010) [2023-12-27 01:18:28,782][105620] Updated weights for policy 1, policy_version 1363745 (0.0010) [2023-12-27 01:18:28,844][105620] Updated weights for policy 1, policy_version 1363755 (0.0011) [2023-12-27 01:18:28,902][105620] Updated weights for policy 1, policy_version 1363765 (0.0010) [2023-12-27 01:18:28,926][105692] Updated weights for policy 0, policy_version 1361817 (0.0006) [2023-12-27 01:18:28,969][105620] Updated weights for policy 1, policy_version 1363775 (0.0010) [2023-12-27 01:18:28,973][105692] Updated weights for policy 0, policy_version 1361827 (0.0005) [2023-12-27 01:18:29,020][105692] Updated weights for policy 0, policy_version 1361837 (0.0008) [2023-12-27 01:18:29,685][105620] Updated weights for policy 1, policy_version 1363785 (0.0011) [2023-12-27 01:18:29,740][105692] Updated weights for policy 0, policy_version 1361847 (0.0011) [2023-12-27 01:18:29,745][105620] Updated weights for policy 1, policy_version 1363795 (0.0011) [2023-12-27 01:18:29,798][105692] Updated weights for policy 0, policy_version 1361857 (0.0008) [2023-12-27 01:18:29,798][105620] Updated weights for policy 1, policy_version 1363805 (0.0011) [2023-12-27 01:18:29,864][105692] Updated weights for policy 0, policy_version 1361867 (0.0008) [2023-12-27 01:18:30,604][105692] Updated weights for policy 0, policy_version 1361877 (0.0010) [2023-12-27 01:18:30,648][105692] Updated weights for policy 0, policy_version 1361887 (0.0010) [2023-12-27 01:18:30,649][105620] Updated weights for policy 1, policy_version 1363815 (0.0009) [2023-12-27 01:18:30,698][105620] Updated weights for policy 1, policy_version 1363825 (0.0007) [2023-12-27 01:18:30,699][105692] Updated weights for policy 0, policy_version 1361897 (0.0010) [2023-12-27 01:18:30,756][105620] Updated weights for policy 1, policy_version 1363835 (0.0008) [2023-12-27 01:18:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 697884672. Throughput: 0: 9884.6, 1: 9627.6. Samples: 697853340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:18:31,063][104569] Avg episode reward: [(0, '8901.518'), (1, '9084.659')] [2023-12-27 01:18:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001361904_348700672.pth... [2023-12-27 01:18:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001363840_349184000.pth... [2023-12-27 01:18:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001360752_348405760.pth [2023-12-27 01:18:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001362720_348897280.pth [2023-12-27 01:18:31,415][105692] Updated weights for policy 0, policy_version 1361907 (0.0008) [2023-12-27 01:18:31,477][105620] Updated weights for policy 1, policy_version 1363845 (0.0008) [2023-12-27 01:18:31,479][105692] Updated weights for policy 0, policy_version 1361917 (0.0010) [2023-12-27 01:18:31,535][105620] Updated weights for policy 1, policy_version 1363855 (0.0005) [2023-12-27 01:18:31,537][105692] Updated weights for policy 0, policy_version 1361927 (0.0011) [2023-12-27 01:18:31,591][105620] Updated weights for policy 1, policy_version 1363865 (0.0005) [2023-12-27 01:18:32,263][105692] Updated weights for policy 0, policy_version 1361937 (0.0010) [2023-12-27 01:18:32,265][105620] Updated weights for policy 1, policy_version 1363875 (0.0009) [2023-12-27 01:18:32,316][105620] Updated weights for policy 1, policy_version 1363885 (0.0010) [2023-12-27 01:18:32,328][105692] Updated weights for policy 0, policy_version 1361947 (0.0008) [2023-12-27 01:18:32,371][105620] Updated weights for policy 1, policy_version 1363895 (0.0011) [2023-12-27 01:18:32,394][105692] Updated weights for policy 0, policy_version 1361957 (0.0009) [2023-12-27 01:18:32,457][105692] Updated weights for policy 0, policy_version 1361967 (0.0008) [2023-12-27 01:18:33,045][105620] Updated weights for policy 1, policy_version 1363905 (0.0010) [2023-12-27 01:18:33,096][105620] Updated weights for policy 1, policy_version 1363915 (0.0005) [2023-12-27 01:18:33,154][105620] Updated weights for policy 1, policy_version 1363925 (0.0010) [2023-12-27 01:18:33,191][105692] Updated weights for policy 0, policy_version 1361977 (0.0006) [2023-12-27 01:18:33,212][105620] Updated weights for policy 1, policy_version 1363935 (0.0010) [2023-12-27 01:18:33,244][105692] Updated weights for policy 0, policy_version 1361987 (0.0007) [2023-12-27 01:18:33,288][105692] Updated weights for policy 0, policy_version 1361997 (0.0008) [2023-12-27 01:18:33,861][105692] Updated weights for policy 0, policy_version 1362007 (0.0008) [2023-12-27 01:18:33,875][105620] Updated weights for policy 1, policy_version 1363945 (0.0006) [2023-12-27 01:18:33,908][105692] Updated weights for policy 0, policy_version 1362017 (0.0008) [2023-12-27 01:18:33,918][105620] Updated weights for policy 1, policy_version 1363955 (0.0005) [2023-12-27 01:18:33,964][105692] Updated weights for policy 0, policy_version 1362027 (0.0009) [2023-12-27 01:18:33,979][105620] Updated weights for policy 1, policy_version 1363965 (0.0005) [2023-12-27 01:18:34,616][105620] Updated weights for policy 1, policy_version 1363975 (0.0009) [2023-12-27 01:18:34,633][105692] Updated weights for policy 0, policy_version 1362037 (0.0007) [2023-12-27 01:18:34,673][105620] Updated weights for policy 1, policy_version 1363985 (0.0011) [2023-12-27 01:18:34,696][105692] Updated weights for policy 0, policy_version 1362047 (0.0006) [2023-12-27 01:18:34,734][105620] Updated weights for policy 1, policy_version 1363995 (0.0006) [2023-12-27 01:18:34,766][105692] Updated weights for policy 0, policy_version 1362057 (0.0008) [2023-12-27 01:18:35,404][105620] Updated weights for policy 1, policy_version 1364005 (0.0005) [2023-12-27 01:18:35,439][105692] Updated weights for policy 0, policy_version 1362067 (0.0008) [2023-12-27 01:18:35,471][105620] Updated weights for policy 1, policy_version 1364015 (0.0009) [2023-12-27 01:18:35,498][105692] Updated weights for policy 0, policy_version 1362077 (0.0008) [2023-12-27 01:18:35,550][105620] Updated weights for policy 1, policy_version 1364025 (0.0011) [2023-12-27 01:18:35,557][105692] Updated weights for policy 0, policy_version 1362087 (0.0010) [2023-12-27 01:18:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 697982976. Throughput: 0: 9989.7, 1: 9640.2. Samples: 697973048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:18:36,063][104569] Avg episode reward: [(0, '8174.853'), (1, '9175.740')] [2023-12-27 01:18:36,123][105620] Updated weights for policy 1, policy_version 1364035 (0.0010) [2023-12-27 01:18:36,185][105620] Updated weights for policy 1, policy_version 1364045 (0.0006) [2023-12-27 01:18:36,251][105620] Updated weights for policy 1, policy_version 1364055 (0.0005) [2023-12-27 01:18:36,284][105692] Updated weights for policy 0, policy_version 1362097 (0.0010) [2023-12-27 01:18:36,343][105692] Updated weights for policy 0, policy_version 1362107 (0.0010) [2023-12-27 01:18:36,402][105692] Updated weights for policy 0, policy_version 1362117 (0.0010) [2023-12-27 01:18:36,468][105692] Updated weights for policy 0, policy_version 1362127 (0.0011) [2023-12-27 01:18:36,906][105620] Updated weights for policy 1, policy_version 1364065 (0.0009) [2023-12-27 01:18:36,971][105620] Updated weights for policy 1, policy_version 1364075 (0.0010) [2023-12-27 01:18:37,026][105620] Updated weights for policy 1, policy_version 1364085 (0.0011) [2023-12-27 01:18:37,078][105620] Updated weights for policy 1, policy_version 1364095 (0.0010) [2023-12-27 01:18:37,200][105692] Updated weights for policy 0, policy_version 1362137 (0.0010) [2023-12-27 01:18:37,272][105692] Updated weights for policy 0, policy_version 1362147 (0.0010) [2023-12-27 01:18:37,336][105692] Updated weights for policy 0, policy_version 1362157 (0.0010) [2023-12-27 01:18:37,805][105620] Updated weights for policy 1, policy_version 1364105 (0.0006) [2023-12-27 01:18:37,871][105620] Updated weights for policy 1, policy_version 1364115 (0.0009) [2023-12-27 01:18:37,927][105620] Updated weights for policy 1, policy_version 1364125 (0.0010) [2023-12-27 01:18:37,962][105692] Updated weights for policy 0, policy_version 1362167 (0.0009) [2023-12-27 01:18:38,025][105692] Updated weights for policy 0, policy_version 1362177 (0.0007) [2023-12-27 01:18:38,079][105692] Updated weights for policy 0, policy_version 1362187 (0.0006) [2023-12-27 01:18:38,579][105620] Updated weights for policy 1, policy_version 1364135 (0.0009) [2023-12-27 01:18:38,651][105620] Updated weights for policy 1, policy_version 1364145 (0.0010) [2023-12-27 01:18:38,710][105620] Updated weights for policy 1, policy_version 1364155 (0.0011) [2023-12-27 01:18:38,733][105692] Updated weights for policy 0, policy_version 1362197 (0.0007) [2023-12-27 01:18:38,794][105692] Updated weights for policy 0, policy_version 1362207 (0.0008) [2023-12-27 01:18:38,861][105692] Updated weights for policy 0, policy_version 1362217 (0.0007) [2023-12-27 01:18:39,441][105692] Updated weights for policy 0, policy_version 1362227 (0.0006) [2023-12-27 01:18:39,450][105620] Updated weights for policy 1, policy_version 1364165 (0.0011) [2023-12-27 01:18:39,497][105692] Updated weights for policy 0, policy_version 1362237 (0.0008) [2023-12-27 01:18:39,502][105620] Updated weights for policy 1, policy_version 1364175 (0.0011) [2023-12-27 01:18:39,553][105692] Updated weights for policy 0, policy_version 1362247 (0.0008) [2023-12-27 01:18:39,554][105620] Updated weights for policy 1, policy_version 1364185 (0.0011) [2023-12-27 01:18:40,272][105620] Updated weights for policy 1, policy_version 1364195 (0.0011) [2023-12-27 01:18:40,321][105620] Updated weights for policy 1, policy_version 1364205 (0.0010) [2023-12-27 01:18:40,350][105692] Updated weights for policy 0, policy_version 1362257 (0.0007) [2023-12-27 01:18:40,374][105620] Updated weights for policy 1, policy_version 1364215 (0.0010) [2023-12-27 01:18:40,403][105692] Updated weights for policy 0, policy_version 1362267 (0.0005) [2023-12-27 01:18:40,419][105585] KL-divergence is very high: 225.4049 [2023-12-27 01:18:40,455][105692] Updated weights for policy 0, policy_version 1362277 (0.0005) [2023-12-27 01:18:40,463][105585] KL-divergence is very high: 348.3344 [2023-12-27 01:18:40,503][105585] KL-divergence is very high: 271.1567 [2023-12-27 01:18:40,507][105692] Updated weights for policy 0, policy_version 1362287 (0.0005) [2023-12-27 01:18:41,010][105620] Updated weights for policy 1, policy_version 1364225 (0.0011) [2023-12-27 01:18:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 698081280. Throughput: 0: 10003.8, 1: 9716.2. Samples: 698094056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:18:41,062][104569] Avg episode reward: [(0, '8173.957'), (1, '9021.408')] [2023-12-27 01:18:41,077][105620] Updated weights for policy 1, policy_version 1364235 (0.0011) [2023-12-27 01:18:41,144][105620] Updated weights for policy 1, policy_version 1364245 (0.0011) [2023-12-27 01:18:41,214][105620] Updated weights for policy 1, policy_version 1364255 (0.0011) [2023-12-27 01:18:41,264][105692] Updated weights for policy 0, policy_version 1362297 (0.0007) [2023-12-27 01:18:41,326][105692] Updated weights for policy 0, policy_version 1362307 (0.0008) [2023-12-27 01:18:41,398][105692] Updated weights for policy 0, policy_version 1362317 (0.0009) [2023-12-27 01:18:41,964][105620] Updated weights for policy 1, policy_version 1364265 (0.0009) [2023-12-27 01:18:42,026][105620] Updated weights for policy 1, policy_version 1364275 (0.0008) [2023-12-27 01:18:42,098][105620] Updated weights for policy 1, policy_version 1364285 (0.0010) [2023-12-27 01:18:42,157][105692] Updated weights for policy 0, policy_version 1362327 (0.0006) [2023-12-27 01:18:42,227][105692] Updated weights for policy 0, policy_version 1362337 (0.0008) [2023-12-27 01:18:42,296][105692] Updated weights for policy 0, policy_version 1362347 (0.0009) [2023-12-27 01:18:42,876][105692] Updated weights for policy 0, policy_version 1362357 (0.0007) [2023-12-27 01:18:42,926][105620] Updated weights for policy 1, policy_version 1364295 (0.0007) [2023-12-27 01:18:42,937][105692] Updated weights for policy 0, policy_version 1362367 (0.0007) [2023-12-27 01:18:42,983][105620] Updated weights for policy 1, policy_version 1364305 (0.0007) [2023-12-27 01:18:42,997][105692] Updated weights for policy 0, policy_version 1362377 (0.0006) [2023-12-27 01:18:43,043][105620] Updated weights for policy 1, policy_version 1364315 (0.0006) [2023-12-27 01:18:43,696][105620] Updated weights for policy 1, policy_version 1364325 (0.0008) [2023-12-27 01:18:43,730][105692] Updated weights for policy 0, policy_version 1362387 (0.0007) [2023-12-27 01:18:43,746][105620] Updated weights for policy 1, policy_version 1364335 (0.0008) [2023-12-27 01:18:43,784][105692] Updated weights for policy 0, policy_version 1362397 (0.0007) [2023-12-27 01:18:43,795][105620] Updated weights for policy 1, policy_version 1364345 (0.0009) [2023-12-27 01:18:43,838][105692] Updated weights for policy 0, policy_version 1362407 (0.0008) [2023-12-27 01:18:44,458][105620] Updated weights for policy 1, policy_version 1364355 (0.0005) [2023-12-27 01:18:44,510][105620] Updated weights for policy 1, policy_version 1364365 (0.0006) [2023-12-27 01:18:44,566][105620] Updated weights for policy 1, policy_version 1364375 (0.0008) [2023-12-27 01:18:44,647][105692] Updated weights for policy 0, policy_version 1362417 (0.0009) [2023-12-27 01:18:44,716][105692] Updated weights for policy 0, policy_version 1362427 (0.0008) [2023-12-27 01:18:44,771][105692] Updated weights for policy 0, policy_version 1362437 (0.0009) [2023-12-27 01:18:44,835][105692] Updated weights for policy 0, policy_version 1362447 (0.0008) [2023-12-27 01:18:45,201][105620] Updated weights for policy 1, policy_version 1364385 (0.0010) [2023-12-27 01:18:45,273][105620] Updated weights for policy 1, policy_version 1364395 (0.0007) [2023-12-27 01:18:45,338][105620] Updated weights for policy 1, policy_version 1364405 (0.0005) [2023-12-27 01:18:45,398][105620] Updated weights for policy 1, policy_version 1364415 (0.0007) [2023-12-27 01:18:45,527][105692] Updated weights for policy 0, policy_version 1362457 (0.0006) [2023-12-27 01:18:45,593][105692] Updated weights for policy 0, policy_version 1362467 (0.0005) [2023-12-27 01:18:45,670][105692] Updated weights for policy 0, policy_version 1362477 (0.0005) [2023-12-27 01:18:45,956][105620] Updated weights for policy 1, policy_version 1364425 (0.0006) [2023-12-27 01:18:46,024][105620] Updated weights for policy 1, policy_version 1364435 (0.0005) [2023-12-27 01:18:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 698179584. Throughput: 0: 9935.3, 1: 9741.0. Samples: 698150508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:18:46,063][104569] Avg episode reward: [(0, '8531.368'), (1, '8871.593')] [2023-12-27 01:18:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001362480_348848128.pth... [2023-12-27 01:18:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001361296_348545024.pth [2023-12-27 01:18:46,086][105620] Updated weights for policy 1, policy_version 1364445 (0.0006) [2023-12-27 01:18:46,103][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001364448_349339648.pth... [2023-12-27 01:18:46,107][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001363296_349044736.pth [2023-12-27 01:18:46,307][105692] Updated weights for policy 0, policy_version 1362487 (0.0008) [2023-12-27 01:18:46,358][105692] Updated weights for policy 0, policy_version 1362497 (0.0009) [2023-12-27 01:18:46,408][105692] Updated weights for policy 0, policy_version 1362507 (0.0009) [2023-12-27 01:18:46,717][105620] Updated weights for policy 1, policy_version 1364455 (0.0009) [2023-12-27 01:18:46,767][105620] Updated weights for policy 1, policy_version 1364465 (0.0009) [2023-12-27 01:18:46,813][105620] Updated weights for policy 1, policy_version 1364475 (0.0008) [2023-12-27 01:18:47,194][105692] Updated weights for policy 0, policy_version 1362517 (0.0007) [2023-12-27 01:18:47,262][105692] Updated weights for policy 0, policy_version 1362527 (0.0008) [2023-12-27 01:18:47,321][105692] Updated weights for policy 0, policy_version 1362537 (0.0006) [2023-12-27 01:18:47,613][105620] Updated weights for policy 1, policy_version 1364485 (0.0009) [2023-12-27 01:18:47,675][105620] Updated weights for policy 1, policy_version 1364495 (0.0010) [2023-12-27 01:18:47,732][105620] Updated weights for policy 1, policy_version 1364505 (0.0010) [2023-12-27 01:18:48,044][105692] Updated weights for policy 0, policy_version 1362547 (0.0008) [2023-12-27 01:18:48,109][105692] Updated weights for policy 0, policy_version 1362557 (0.0008) [2023-12-27 01:18:48,173][105692] Updated weights for policy 0, policy_version 1362567 (0.0008) [2023-12-27 01:18:48,472][105620] Updated weights for policy 1, policy_version 1364515 (0.0010) [2023-12-27 01:18:48,534][105620] Updated weights for policy 1, policy_version 1364525 (0.0010) [2023-12-27 01:18:48,589][105620] Updated weights for policy 1, policy_version 1364535 (0.0010) [2023-12-27 01:18:48,886][105692] Updated weights for policy 0, policy_version 1362577 (0.0008) [2023-12-27 01:18:48,951][105692] Updated weights for policy 0, policy_version 1362587 (0.0009) [2023-12-27 01:18:49,007][105692] Updated weights for policy 0, policy_version 1362597 (0.0009) [2023-12-27 01:18:49,055][105692] Updated weights for policy 0, policy_version 1362607 (0.0009) [2023-12-27 01:18:49,332][105620] Updated weights for policy 1, policy_version 1364545 (0.0010) [2023-12-27 01:18:49,397][105620] Updated weights for policy 1, policy_version 1364555 (0.0008) [2023-12-27 01:18:49,457][105620] Updated weights for policy 1, policy_version 1364565 (0.0009) [2023-12-27 01:18:49,514][105620] Updated weights for policy 1, policy_version 1364575 (0.0009) [2023-12-27 01:18:49,806][105692] Updated weights for policy 0, policy_version 1362617 (0.0009) [2023-12-27 01:18:49,866][105692] Updated weights for policy 0, policy_version 1362627 (0.0009) [2023-12-27 01:18:49,924][105692] Updated weights for policy 0, policy_version 1362637 (0.0009) [2023-12-27 01:18:50,265][105620] Updated weights for policy 1, policy_version 1364585 (0.0006) [2023-12-27 01:18:50,325][105620] Updated weights for policy 1, policy_version 1364595 (0.0006) [2023-12-27 01:18:50,388][105620] Updated weights for policy 1, policy_version 1364605 (0.0006) [2023-12-27 01:18:50,597][105692] Updated weights for policy 0, policy_version 1362647 (0.0007) [2023-12-27 01:18:50,667][105692] Updated weights for policy 0, policy_version 1362657 (0.0006) [2023-12-27 01:18:50,738][105692] Updated weights for policy 0, policy_version 1362667 (0.0009) [2023-12-27 01:18:51,015][105620] Updated weights for policy 1, policy_version 1364615 (0.0007) [2023-12-27 01:18:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 698277888. Throughput: 0: 9944.6, 1: 9801.9. Samples: 698269736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:18:51,062][104569] Avg episode reward: [(0, '8532.501'), (1, '8951.492')] [2023-12-27 01:18:51,077][105620] Updated weights for policy 1, policy_version 1364625 (0.0009) [2023-12-27 01:18:51,141][105620] Updated weights for policy 1, policy_version 1364635 (0.0008) [2023-12-27 01:18:51,524][105692] Updated weights for policy 0, policy_version 1362677 (0.0009) [2023-12-27 01:18:51,584][105692] Updated weights for policy 0, policy_version 1362687 (0.0009) [2023-12-27 01:18:51,647][105692] Updated weights for policy 0, policy_version 1362697 (0.0009) [2023-12-27 01:18:51,873][105620] Updated weights for policy 1, policy_version 1364645 (0.0010) [2023-12-27 01:18:51,935][105620] Updated weights for policy 1, policy_version 1364655 (0.0009) [2023-12-27 01:18:51,999][105620] Updated weights for policy 1, policy_version 1364665 (0.0009) [2023-12-27 01:18:52,391][105692] Updated weights for policy 0, policy_version 1362707 (0.0009) [2023-12-27 01:18:52,454][105692] Updated weights for policy 0, policy_version 1362717 (0.0008) [2023-12-27 01:18:52,510][105692] Updated weights for policy 0, policy_version 1362727 (0.0009) [2023-12-27 01:18:52,755][105620] Updated weights for policy 1, policy_version 1364675 (0.0008) [2023-12-27 01:18:52,821][105620] Updated weights for policy 1, policy_version 1364685 (0.0009) [2023-12-27 01:18:52,879][105620] Updated weights for policy 1, policy_version 1364695 (0.0010) [2023-12-27 01:18:53,209][105692] Updated weights for policy 0, policy_version 1362737 (0.0009) [2023-12-27 01:18:53,259][105692] Updated weights for policy 0, policy_version 1362747 (0.0006) [2023-12-27 01:18:53,316][105692] Updated weights for policy 0, policy_version 1362757 (0.0006) [2023-12-27 01:18:53,369][105692] Updated weights for policy 0, policy_version 1362767 (0.0008) [2023-12-27 01:18:53,720][105620] Updated weights for policy 1, policy_version 1364705 (0.0008) [2023-12-27 01:18:53,768][105620] Updated weights for policy 1, policy_version 1364715 (0.0008) [2023-12-27 01:18:53,814][105620] Updated weights for policy 1, policy_version 1364725 (0.0008) [2023-12-27 01:18:53,858][105620] Updated weights for policy 1, policy_version 1364735 (0.0007) [2023-12-27 01:18:53,951][105692] Updated weights for policy 0, policy_version 1362777 (0.0010) [2023-12-27 01:18:54,007][105692] Updated weights for policy 0, policy_version 1362787 (0.0011) [2023-12-27 01:18:54,069][105692] Updated weights for policy 0, policy_version 1362797 (0.0011) [2023-12-27 01:18:54,689][105620] Updated weights for policy 1, policy_version 1364745 (0.0008) [2023-12-27 01:18:54,745][105692] Updated weights for policy 0, policy_version 1362807 (0.0010) [2023-12-27 01:18:54,751][105620] Updated weights for policy 1, policy_version 1364755 (0.0007) [2023-12-27 01:18:54,797][105692] Updated weights for policy 0, policy_version 1362817 (0.0009) [2023-12-27 01:18:54,803][105620] Updated weights for policy 1, policy_version 1364765 (0.0006) [2023-12-27 01:18:54,851][105585] KL-divergence is very high: 226.6607 [2023-12-27 01:18:54,852][105692] Updated weights for policy 0, policy_version 1362827 (0.0010) [2023-12-27 01:18:55,517][105620] Updated weights for policy 1, policy_version 1364775 (0.0007) [2023-12-27 01:18:55,571][105620] Updated weights for policy 1, policy_version 1364785 (0.0008) [2023-12-27 01:18:55,583][105692] Updated weights for policy 0, policy_version 1362837 (0.0010) [2023-12-27 01:18:55,631][105620] Updated weights for policy 1, policy_version 1364795 (0.0006) [2023-12-27 01:18:55,641][105692] Updated weights for policy 0, policy_version 1362847 (0.0010) [2023-12-27 01:18:55,699][105692] Updated weights for policy 0, policy_version 1362857 (0.0010) [2023-12-27 01:18:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 698376192. Throughput: 0: 9967.9, 1: 9719.6. Samples: 698384644. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:18:56,062][104569] Avg episode reward: [(0, '8347.587'), (1, '9281.826')] [2023-12-27 01:18:56,295][105620] Updated weights for policy 1, policy_version 1364805 (0.0007) [2023-12-27 01:18:56,355][105620] Updated weights for policy 1, policy_version 1364815 (0.0008) [2023-12-27 01:18:56,412][105620] Updated weights for policy 1, policy_version 1364825 (0.0010) [2023-12-27 01:18:56,429][105692] Updated weights for policy 0, policy_version 1362867 (0.0009) [2023-12-27 01:18:56,480][105692] Updated weights for policy 0, policy_version 1362877 (0.0005) [2023-12-27 01:18:56,522][105692] Updated weights for policy 0, policy_version 1362887 (0.0005) [2023-12-27 01:18:57,041][105692] Updated weights for policy 0, policy_version 1362897 (0.0005) [2023-12-27 01:18:57,095][105692] Updated weights for policy 0, policy_version 1362907 (0.0005) [2023-12-27 01:18:57,108][105620] Updated weights for policy 1, policy_version 1364835 (0.0008) [2023-12-27 01:18:57,152][105692] Updated weights for policy 0, policy_version 1362917 (0.0009) [2023-12-27 01:18:57,168][105620] Updated weights for policy 1, policy_version 1364845 (0.0006) [2023-12-27 01:18:57,207][105692] Updated weights for policy 0, policy_version 1362927 (0.0010) [2023-12-27 01:18:57,223][105620] Updated weights for policy 1, policy_version 1364855 (0.0005) [2023-12-27 01:18:57,756][105692] Updated weights for policy 0, policy_version 1362937 (0.0010) [2023-12-27 01:18:57,813][105692] Updated weights for policy 0, policy_version 1362947 (0.0010) [2023-12-27 01:18:57,870][105692] Updated weights for policy 0, policy_version 1362957 (0.0009) [2023-12-27 01:18:57,941][105620] Updated weights for policy 1, policy_version 1364865 (0.0005) [2023-12-27 01:18:57,987][105620] Updated weights for policy 1, policy_version 1364875 (0.0008) [2023-12-27 01:18:58,034][105620] Updated weights for policy 1, policy_version 1364885 (0.0009) [2023-12-27 01:18:58,080][105620] Updated weights for policy 1, policy_version 1364895 (0.0008) [2023-12-27 01:18:58,687][105692] Updated weights for policy 0, policy_version 1362967 (0.0009) [2023-12-27 01:18:58,747][105692] Updated weights for policy 0, policy_version 1362977 (0.0008) [2023-12-27 01:18:58,810][105692] Updated weights for policy 0, policy_version 1362987 (0.0009) [2023-12-27 01:18:58,880][105620] Updated weights for policy 1, policy_version 1364905 (0.0009) [2023-12-27 01:18:58,954][105620] Updated weights for policy 1, policy_version 1364915 (0.0009) [2023-12-27 01:18:59,018][105620] Updated weights for policy 1, policy_version 1364925 (0.0009) [2023-12-27 01:18:59,667][105692] Updated weights for policy 0, policy_version 1362997 (0.0010) [2023-12-27 01:18:59,693][105620] Updated weights for policy 1, policy_version 1364935 (0.0006) [2023-12-27 01:18:59,723][105692] Updated weights for policy 0, policy_version 1363007 (0.0009) [2023-12-27 01:18:59,739][105620] Updated weights for policy 1, policy_version 1364945 (0.0005) [2023-12-27 01:18:59,772][105692] Updated weights for policy 0, policy_version 1363017 (0.0008) [2023-12-27 01:18:59,790][105620] Updated weights for policy 1, policy_version 1364955 (0.0009) [2023-12-27 01:19:00,520][105620] Updated weights for policy 1, policy_version 1364965 (0.0008) [2023-12-27 01:19:00,543][105692] Updated weights for policy 0, policy_version 1363027 (0.0006) [2023-12-27 01:19:00,575][105620] Updated weights for policy 1, policy_version 1364975 (0.0009) [2023-12-27 01:19:00,596][105692] Updated weights for policy 0, policy_version 1363037 (0.0005) [2023-12-27 01:19:00,635][105620] Updated weights for policy 1, policy_version 1364985 (0.0009) [2023-12-27 01:19:00,648][105692] Updated weights for policy 0, policy_version 1363047 (0.0005) [2023-12-27 01:19:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.7, 300 sec: 19494.2). Total num frames: 698474496. Throughput: 0: 10058.2, 1: 9709.0. Samples: 698445920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:19:01,063][104569] Avg episode reward: [(0, '7890.011'), (1, '9354.534')] [2023-12-27 01:19:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001363056_348995584.pth... [2023-12-27 01:19:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001364992_349478912.pth... [2023-12-27 01:19:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001361904_348700672.pth [2023-12-27 01:19:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001363840_349184000.pth [2023-12-27 01:19:01,261][105692] Updated weights for policy 0, policy_version 1363057 (0.0008) [2023-12-27 01:19:01,323][105692] Updated weights for policy 0, policy_version 1363067 (0.0009) [2023-12-27 01:19:01,388][105692] Updated weights for policy 0, policy_version 1363077 (0.0008) [2023-12-27 01:19:01,457][105692] Updated weights for policy 0, policy_version 1363087 (0.0009) [2023-12-27 01:19:01,480][105620] Updated weights for policy 1, policy_version 1364995 (0.0007) [2023-12-27 01:19:01,538][105620] Updated weights for policy 1, policy_version 1365005 (0.0008) [2023-12-27 01:19:01,610][105620] Updated weights for policy 1, policy_version 1365015 (0.0008) [2023-12-27 01:19:02,194][105692] Updated weights for policy 0, policy_version 1363097 (0.0009) [2023-12-27 01:19:02,252][105692] Updated weights for policy 0, policy_version 1363107 (0.0009) [2023-12-27 01:19:02,309][105692] Updated weights for policy 0, policy_version 1363117 (0.0009) [2023-12-27 01:19:02,371][105620] Updated weights for policy 1, policy_version 1365025 (0.0009) [2023-12-27 01:19:02,438][105620] Updated weights for policy 1, policy_version 1365035 (0.0007) [2023-12-27 01:19:02,496][105620] Updated weights for policy 1, policy_version 1365045 (0.0009) [2023-12-27 01:19:02,557][105620] Updated weights for policy 1, policy_version 1365055 (0.0009) [2023-12-27 01:19:03,138][105692] Updated weights for policy 0, policy_version 1363127 (0.0008) [2023-12-27 01:19:03,189][105620] Updated weights for policy 1, policy_version 1365065 (0.0006) [2023-12-27 01:19:03,196][105692] Updated weights for policy 0, policy_version 1363137 (0.0009) [2023-12-27 01:19:03,239][105620] Updated weights for policy 1, policy_version 1365075 (0.0009) [2023-12-27 01:19:03,249][105692] Updated weights for policy 0, policy_version 1363147 (0.0007) [2023-12-27 01:19:03,283][105620] Updated weights for policy 1, policy_version 1365085 (0.0010) [2023-12-27 01:19:03,915][105620] Updated weights for policy 1, policy_version 1365095 (0.0010) [2023-12-27 01:19:03,978][105620] Updated weights for policy 1, policy_version 1365105 (0.0011) [2023-12-27 01:19:04,040][105692] Updated weights for policy 0, policy_version 1363157 (0.0006) [2023-12-27 01:19:04,040][105620] Updated weights for policy 1, policy_version 1365115 (0.0010) [2023-12-27 01:19:04,101][105692] Updated weights for policy 0, policy_version 1363167 (0.0008) [2023-12-27 01:19:04,159][105692] Updated weights for policy 0, policy_version 1363177 (0.0010) [2023-12-27 01:19:04,616][105620] Updated weights for policy 1, policy_version 1365125 (0.0008) [2023-12-27 01:19:04,687][105620] Updated weights for policy 1, policy_version 1365135 (0.0006) [2023-12-27 01:19:04,754][105620] Updated weights for policy 1, policy_version 1365145 (0.0007) [2023-12-27 01:19:05,023][105692] Updated weights for policy 0, policy_version 1363187 (0.0008) [2023-12-27 01:19:05,074][105692] Updated weights for policy 0, policy_version 1363197 (0.0008) [2023-12-27 01:19:05,144][105692] Updated weights for policy 0, policy_version 1363207 (0.0008) [2023-12-27 01:19:05,416][105620] Updated weights for policy 1, policy_version 1365155 (0.0007) [2023-12-27 01:19:05,478][105620] Updated weights for policy 1, policy_version 1365165 (0.0010) [2023-12-27 01:19:05,537][105620] Updated weights for policy 1, policy_version 1365175 (0.0010) [2023-12-27 01:19:05,882][105692] Updated weights for policy 0, policy_version 1363217 (0.0009) [2023-12-27 01:19:05,940][105692] Updated weights for policy 0, policy_version 1363227 (0.0008) [2023-12-27 01:19:05,988][105692] Updated weights for policy 0, policy_version 1363237 (0.0009) [2023-12-27 01:19:06,045][105692] Updated weights for policy 0, policy_version 1363247 (0.0009) [2023-12-27 01:19:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 698572800. Throughput: 0: 9944.6, 1: 9748.0. Samples: 698559904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:19:06,062][104569] Avg episode reward: [(0, '7981.430'), (1, '9079.563')] [2023-12-27 01:19:06,276][105620] Updated weights for policy 1, policy_version 1365185 (0.0010) [2023-12-27 01:19:06,348][105620] Updated weights for policy 1, policy_version 1365195 (0.0005) [2023-12-27 01:19:06,413][105620] Updated weights for policy 1, policy_version 1365205 (0.0009) [2023-12-27 01:19:06,476][105620] Updated weights for policy 1, policy_version 1365215 (0.0009) [2023-12-27 01:19:06,871][105692] Updated weights for policy 0, policy_version 1363257 (0.0006) [2023-12-27 01:19:06,935][105692] Updated weights for policy 0, policy_version 1363267 (0.0006) [2023-12-27 01:19:07,005][105692] Updated weights for policy 0, policy_version 1363277 (0.0007) [2023-12-27 01:19:07,124][105620] Updated weights for policy 1, policy_version 1365225 (0.0006) [2023-12-27 01:19:07,178][105620] Updated weights for policy 1, policy_version 1365235 (0.0009) [2023-12-27 01:19:07,233][105620] Updated weights for policy 1, policy_version 1365245 (0.0009) [2023-12-27 01:19:07,633][105692] Updated weights for policy 0, policy_version 1363287 (0.0009) [2023-12-27 01:19:07,687][105692] Updated weights for policy 0, policy_version 1363297 (0.0009) [2023-12-27 01:19:07,744][105692] Updated weights for policy 0, policy_version 1363307 (0.0009) [2023-12-27 01:19:07,918][105620] Updated weights for policy 1, policy_version 1365255 (0.0007) [2023-12-27 01:19:07,968][105620] Updated weights for policy 1, policy_version 1365265 (0.0009) [2023-12-27 01:19:08,022][105620] Updated weights for policy 1, policy_version 1365275 (0.0009) [2023-12-27 01:19:08,511][105692] Updated weights for policy 0, policy_version 1363318 (0.0009) [2023-12-27 01:19:08,576][105692] Updated weights for policy 0, policy_version 1363328 (0.0009) [2023-12-27 01:19:08,641][105692] Updated weights for policy 0, policy_version 1363338 (0.0009) [2023-12-27 01:19:08,798][105620] Updated weights for policy 1, policy_version 1365285 (0.0009) [2023-12-27 01:19:08,859][105620] Updated weights for policy 1, policy_version 1365295 (0.0009) [2023-12-27 01:19:08,921][105620] Updated weights for policy 1, policy_version 1365305 (0.0009) [2023-12-27 01:19:09,387][105692] Updated weights for policy 0, policy_version 1363348 (0.0009) [2023-12-27 01:19:09,453][105692] Updated weights for policy 0, policy_version 1363358 (0.0009) [2023-12-27 01:19:09,505][105692] Updated weights for policy 0, policy_version 1363368 (0.0009) [2023-12-27 01:19:09,704][105620] Updated weights for policy 1, policy_version 1365315 (0.0009) [2023-12-27 01:19:09,761][105620] Updated weights for policy 1, policy_version 1365325 (0.0010) [2023-12-27 01:19:09,813][105620] Updated weights for policy 1, policy_version 1365335 (0.0009) [2023-12-27 01:19:10,163][105692] Updated weights for policy 0, policy_version 1363378 (0.0008) [2023-12-27 01:19:10,226][105692] Updated weights for policy 0, policy_version 1363388 (0.0009) [2023-12-27 01:19:10,285][105692] Updated weights for policy 0, policy_version 1363398 (0.0009) [2023-12-27 01:19:10,344][105692] Updated weights for policy 0, policy_version 1363408 (0.0009) [2023-12-27 01:19:10,648][105620] Updated weights for policy 1, policy_version 1365345 (0.0007) [2023-12-27 01:19:10,697][105620] Updated weights for policy 1, policy_version 1365355 (0.0006) [2023-12-27 01:19:10,755][105620] Updated weights for policy 1, policy_version 1365365 (0.0009) [2023-12-27 01:19:10,811][105620] Updated weights for policy 1, policy_version 1365375 (0.0009) [2023-12-27 01:19:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 698662912. Throughput: 0: 9863.7, 1: 9740.9. Samples: 698672776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:19:11,063][104569] Avg episode reward: [(0, '8073.266'), (1, '8989.905')] [2023-12-27 01:19:11,081][105692] Updated weights for policy 0, policy_version 1363418 (0.0006) [2023-12-27 01:19:11,140][105692] Updated weights for policy 0, policy_version 1363428 (0.0007) [2023-12-27 01:19:11,208][105692] Updated weights for policy 0, policy_version 1363438 (0.0007) [2023-12-27 01:19:11,633][105620] Updated weights for policy 1, policy_version 1365385 (0.0010) [2023-12-27 01:19:11,696][105620] Updated weights for policy 1, policy_version 1365395 (0.0009) [2023-12-27 01:19:11,771][105620] Updated weights for policy 1, policy_version 1365405 (0.0009) [2023-12-27 01:19:11,924][105692] Updated weights for policy 0, policy_version 1363448 (0.0007) [2023-12-27 01:19:11,982][105692] Updated weights for policy 0, policy_version 1363458 (0.0010) [2023-12-27 01:19:12,042][105692] Updated weights for policy 0, policy_version 1363468 (0.0009) [2023-12-27 01:19:12,489][105620] Updated weights for policy 1, policy_version 1365415 (0.0011) [2023-12-27 01:19:12,544][105620] Updated weights for policy 1, policy_version 1365425 (0.0010) [2023-12-27 01:19:12,606][105620] Updated weights for policy 1, policy_version 1365435 (0.0011) [2023-12-27 01:19:12,821][105692] Updated weights for policy 0, policy_version 1363478 (0.0009) [2023-12-27 01:19:12,883][105692] Updated weights for policy 0, policy_version 1363488 (0.0010) [2023-12-27 01:19:12,948][105692] Updated weights for policy 0, policy_version 1363498 (0.0010) [2023-12-27 01:19:13,359][105620] Updated weights for policy 1, policy_version 1365445 (0.0011) [2023-12-27 01:19:13,417][105620] Updated weights for policy 1, policy_version 1365455 (0.0010) [2023-12-27 01:19:13,472][105620] Updated weights for policy 1, policy_version 1365465 (0.0010) [2023-12-27 01:19:13,620][105692] Updated weights for policy 0, policy_version 1363508 (0.0009) [2023-12-27 01:19:13,681][105692] Updated weights for policy 0, policy_version 1363518 (0.0007) [2023-12-27 01:19:13,742][105692] Updated weights for policy 0, policy_version 1363528 (0.0010) [2023-12-27 01:19:14,194][105620] Updated weights for policy 1, policy_version 1365475 (0.0009) [2023-12-27 01:19:14,254][105620] Updated weights for policy 1, policy_version 1365485 (0.0006) [2023-12-27 01:19:14,311][105620] Updated weights for policy 1, policy_version 1365495 (0.0008) [2023-12-27 01:19:14,403][105692] Updated weights for policy 0, policy_version 1363538 (0.0009) [2023-12-27 01:19:14,451][105692] Updated weights for policy 0, policy_version 1363548 (0.0008) [2023-12-27 01:19:14,500][105692] Updated weights for policy 0, policy_version 1363558 (0.0007) [2023-12-27 01:19:14,548][105692] Updated weights for policy 0, policy_version 1363568 (0.0008) [2023-12-27 01:19:15,014][105620] Updated weights for policy 1, policy_version 1365505 (0.0010) [2023-12-27 01:19:15,090][105620] Updated weights for policy 1, policy_version 1365515 (0.0008) [2023-12-27 01:19:15,156][105620] Updated weights for policy 1, policy_version 1365525 (0.0008) [2023-12-27 01:19:15,230][105620] Updated weights for policy 1, policy_version 1365535 (0.0009) [2023-12-27 01:19:15,380][105692] Updated weights for policy 0, policy_version 1363578 (0.0009) [2023-12-27 01:19:15,429][105692] Updated weights for policy 0, policy_version 1363588 (0.0009) [2023-12-27 01:19:15,478][105692] Updated weights for policy 0, policy_version 1363598 (0.0008) [2023-12-27 01:19:15,897][105620] Updated weights for policy 1, policy_version 1365545 (0.0011) [2023-12-27 01:19:15,941][105620] Updated weights for policy 1, policy_version 1365555 (0.0010) [2023-12-27 01:19:15,989][105620] Updated weights for policy 1, policy_version 1365565 (0.0010) [2023-12-27 01:19:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 698761216. Throughput: 0: 9729.3, 1: 9745.3. Samples: 698729700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:19:16,062][104569] Avg episode reward: [(0, '8072.651'), (1, '9081.747')] [2023-12-27 01:19:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001365568_349626368.pth... [2023-12-27 01:19:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001363600_349134848.pth... [2023-12-27 01:19:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001364448_349339648.pth [2023-12-27 01:19:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001362480_348848128.pth [2023-12-27 01:19:16,185][105692] Updated weights for policy 0, policy_version 1363608 (0.0010) [2023-12-27 01:19:16,250][105692] Updated weights for policy 0, policy_version 1363618 (0.0011) [2023-12-27 01:19:16,307][105692] Updated weights for policy 0, policy_version 1363628 (0.0009) [2023-12-27 01:19:16,768][105620] Updated weights for policy 1, policy_version 1365575 (0.0010) [2023-12-27 01:19:16,820][105620] Updated weights for policy 1, policy_version 1365585 (0.0010) [2023-12-27 01:19:16,875][105620] Updated weights for policy 1, policy_version 1365595 (0.0010) [2023-12-27 01:19:17,087][105692] Updated weights for policy 0, policy_version 1363638 (0.0009) [2023-12-27 01:19:17,147][105692] Updated weights for policy 0, policy_version 1363648 (0.0008) [2023-12-27 01:19:17,206][105692] Updated weights for policy 0, policy_version 1363658 (0.0008) [2023-12-27 01:19:17,636][105620] Updated weights for policy 1, policy_version 1365605 (0.0010) [2023-12-27 01:19:17,691][105620] Updated weights for policy 1, policy_version 1365615 (0.0010) [2023-12-27 01:19:17,743][105620] Updated weights for policy 1, policy_version 1365625 (0.0010) [2023-12-27 01:19:17,975][105692] Updated weights for policy 0, policy_version 1363668 (0.0007) [2023-12-27 01:19:18,035][105692] Updated weights for policy 0, policy_version 1363678 (0.0008) [2023-12-27 01:19:18,095][105692] Updated weights for policy 0, policy_version 1363688 (0.0008) [2023-12-27 01:19:18,504][105620] Updated weights for policy 1, policy_version 1365635 (0.0010) [2023-12-27 01:19:18,563][105620] Updated weights for policy 1, policy_version 1365645 (0.0010) [2023-12-27 01:19:18,621][105620] Updated weights for policy 1, policy_version 1365655 (0.0010) [2023-12-27 01:19:18,822][105692] Updated weights for policy 0, policy_version 1363698 (0.0008) [2023-12-27 01:19:18,881][105692] Updated weights for policy 0, policy_version 1363708 (0.0011) [2023-12-27 01:19:18,943][105692] Updated weights for policy 0, policy_version 1363718 (0.0010) [2023-12-27 01:19:18,995][105692] Updated weights for policy 0, policy_version 1363728 (0.0011) [2023-12-27 01:19:19,388][105620] Updated weights for policy 1, policy_version 1365665 (0.0010) [2023-12-27 01:19:19,456][105620] Updated weights for policy 1, policy_version 1365675 (0.0009) [2023-12-27 01:19:19,523][105620] Updated weights for policy 1, policy_version 1365685 (0.0007) [2023-12-27 01:19:19,585][105620] Updated weights for policy 1, policy_version 1365695 (0.0008) [2023-12-27 01:19:19,794][105692] Updated weights for policy 0, policy_version 1363738 (0.0007) [2023-12-27 01:19:19,861][105692] Updated weights for policy 0, policy_version 1363748 (0.0008) [2023-12-27 01:19:19,922][105692] Updated weights for policy 0, policy_version 1363758 (0.0010) [2023-12-27 01:19:20,346][105620] Updated weights for policy 1, policy_version 1365705 (0.0011) [2023-12-27 01:19:20,408][105620] Updated weights for policy 1, policy_version 1365715 (0.0010) [2023-12-27 01:19:20,467][105620] Updated weights for policy 1, policy_version 1365725 (0.0011) [2023-12-27 01:19:20,656][105692] Updated weights for policy 0, policy_version 1363768 (0.0009) [2023-12-27 01:19:20,722][105692] Updated weights for policy 0, policy_version 1363778 (0.0007) [2023-12-27 01:19:20,788][105692] Updated weights for policy 0, policy_version 1363788 (0.0008) [2023-12-27 01:19:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 698851328. Throughput: 0: 9623.7, 1: 9686.9. Samples: 698842020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:19:21,062][104569] Avg episode reward: [(0, '8622.080'), (1, '9079.354')] [2023-12-27 01:19:21,204][105620] Updated weights for policy 1, policy_version 1365735 (0.0011) [2023-12-27 01:19:21,272][105620] Updated weights for policy 1, policy_version 1365745 (0.0009) [2023-12-27 01:19:21,332][105620] Updated weights for policy 1, policy_version 1365755 (0.0008) [2023-12-27 01:19:21,499][105692] Updated weights for policy 0, policy_version 1363798 (0.0009) [2023-12-27 01:19:21,562][105692] Updated weights for policy 0, policy_version 1363808 (0.0010) [2023-12-27 01:19:21,630][105692] Updated weights for policy 0, policy_version 1363818 (0.0011) [2023-12-27 01:19:22,111][105620] Updated weights for policy 1, policy_version 1365765 (0.0010) [2023-12-27 01:19:22,163][105620] Updated weights for policy 1, policy_version 1365775 (0.0010) [2023-12-27 01:19:22,223][105620] Updated weights for policy 1, policy_version 1365785 (0.0011) [2023-12-27 01:19:22,414][105692] Updated weights for policy 0, policy_version 1363828 (0.0011) [2023-12-27 01:19:22,470][105692] Updated weights for policy 0, policy_version 1363838 (0.0011) [2023-12-27 01:19:22,524][105692] Updated weights for policy 0, policy_version 1363848 (0.0010) [2023-12-27 01:19:22,991][105620] Updated weights for policy 1, policy_version 1365795 (0.0009) [2023-12-27 01:19:23,056][105620] Updated weights for policy 1, policy_version 1365805 (0.0008) [2023-12-27 01:19:23,114][105620] Updated weights for policy 1, policy_version 1365815 (0.0006) [2023-12-27 01:19:23,259][105692] Updated weights for policy 0, policy_version 1363858 (0.0010) [2023-12-27 01:19:23,318][105692] Updated weights for policy 0, policy_version 1363868 (0.0011) [2023-12-27 01:19:23,374][105692] Updated weights for policy 0, policy_version 1363878 (0.0011) [2023-12-27 01:19:23,434][105692] Updated weights for policy 0, policy_version 1363888 (0.0011) [2023-12-27 01:19:23,728][105620] Updated weights for policy 1, policy_version 1365825 (0.0011) [2023-12-27 01:19:23,780][105620] Updated weights for policy 1, policy_version 1365835 (0.0008) [2023-12-27 01:19:23,843][105620] Updated weights for policy 1, policy_version 1365845 (0.0005) [2023-12-27 01:19:23,903][105620] Updated weights for policy 1, policy_version 1365855 (0.0005) [2023-12-27 01:19:24,126][105692] Updated weights for policy 0, policy_version 1363898 (0.0011) [2023-12-27 01:19:24,174][105692] Updated weights for policy 0, policy_version 1363908 (0.0006) [2023-12-27 01:19:24,227][105692] Updated weights for policy 0, policy_version 1363918 (0.0005) [2023-12-27 01:19:24,507][105620] Updated weights for policy 1, policy_version 1365865 (0.0010) [2023-12-27 01:19:24,565][105620] Updated weights for policy 1, policy_version 1365875 (0.0011) [2023-12-27 01:19:24,623][105620] Updated weights for policy 1, policy_version 1365885 (0.0010) [2023-12-27 01:19:24,785][105692] Updated weights for policy 0, policy_version 1363928 (0.0005) [2023-12-27 01:19:24,831][105692] Updated weights for policy 0, policy_version 1363938 (0.0005) [2023-12-27 01:19:24,879][105692] Updated weights for policy 0, policy_version 1363948 (0.0005) [2023-12-27 01:19:25,364][105620] Updated weights for policy 1, policy_version 1365895 (0.0010) [2023-12-27 01:19:25,421][105620] Updated weights for policy 1, policy_version 1365905 (0.0010) [2023-12-27 01:19:25,487][105620] Updated weights for policy 1, policy_version 1365915 (0.0010) [2023-12-27 01:19:25,488][105692] Updated weights for policy 0, policy_version 1363958 (0.0006) [2023-12-27 01:19:25,542][105692] Updated weights for policy 0, policy_version 1363968 (0.0005) [2023-12-27 01:19:25,606][105692] Updated weights for policy 0, policy_version 1363978 (0.0005) [2023-12-27 01:19:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 698949632. Throughput: 0: 9623.3, 1: 9630.3. Samples: 698960468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:19:26,063][104569] Avg episode reward: [(0, '8812.073'), (1, '9080.078')] [2023-12-27 01:19:26,226][105620] Updated weights for policy 1, policy_version 1365925 (0.0010) [2023-12-27 01:19:26,265][105692] Updated weights for policy 0, policy_version 1363988 (0.0005) [2023-12-27 01:19:26,285][105620] Updated weights for policy 1, policy_version 1365935 (0.0010) [2023-12-27 01:19:26,320][105692] Updated weights for policy 0, policy_version 1363998 (0.0006) [2023-12-27 01:19:26,343][105620] Updated weights for policy 1, policy_version 1365945 (0.0010) [2023-12-27 01:19:26,377][105692] Updated weights for policy 0, policy_version 1364008 (0.0010) [2023-12-27 01:19:27,013][105692] Updated weights for policy 0, policy_version 1364018 (0.0009) [2023-12-27 01:19:27,076][105692] Updated weights for policy 0, policy_version 1364028 (0.0006) [2023-12-27 01:19:27,077][105620] Updated weights for policy 1, policy_version 1365955 (0.0010) [2023-12-27 01:19:27,142][105692] Updated weights for policy 0, policy_version 1364038 (0.0005) [2023-12-27 01:19:27,145][105620] Updated weights for policy 1, policy_version 1365965 (0.0005) [2023-12-27 01:19:27,202][105620] Updated weights for policy 1, policy_version 1365975 (0.0005) [2023-12-27 01:19:27,205][105692] Updated weights for policy 0, policy_version 1364048 (0.0008) [2023-12-27 01:19:27,779][105692] Updated weights for policy 0, policy_version 1364058 (0.0008) [2023-12-27 01:19:27,830][105692] Updated weights for policy 0, policy_version 1364068 (0.0005) [2023-12-27 01:19:27,882][105620] Updated weights for policy 1, policy_version 1365985 (0.0006) [2023-12-27 01:19:27,892][105692] Updated weights for policy 0, policy_version 1364078 (0.0005) [2023-12-27 01:19:27,933][105620] Updated weights for policy 1, policy_version 1365995 (0.0009) [2023-12-27 01:19:27,979][105620] Updated weights for policy 1, policy_version 1366005 (0.0008) [2023-12-27 01:19:28,032][105620] Updated weights for policy 1, policy_version 1366015 (0.0006) [2023-12-27 01:19:28,469][105692] Updated weights for policy 0, policy_version 1364088 (0.0006) [2023-12-27 01:19:28,526][105692] Updated weights for policy 0, policy_version 1364098 (0.0006) [2023-12-27 01:19:28,586][105692] Updated weights for policy 0, policy_version 1364108 (0.0008) [2023-12-27 01:19:28,630][105620] Updated weights for policy 1, policy_version 1366025 (0.0010) [2023-12-27 01:19:28,678][105620] Updated weights for policy 1, policy_version 1366035 (0.0010) [2023-12-27 01:19:28,742][105620] Updated weights for policy 1, policy_version 1366045 (0.0009) [2023-12-27 01:19:29,365][105620] Updated weights for policy 1, policy_version 1366055 (0.0008) [2023-12-27 01:19:29,383][105692] Updated weights for policy 0, policy_version 1364118 (0.0008) [2023-12-27 01:19:29,428][105620] Updated weights for policy 1, policy_version 1366065 (0.0007) [2023-12-27 01:19:29,436][105692] Updated weights for policy 0, policy_version 1364128 (0.0008) [2023-12-27 01:19:29,482][105620] Updated weights for policy 1, policy_version 1366075 (0.0009) [2023-12-27 01:19:29,495][105692] Updated weights for policy 0, policy_version 1364138 (0.0009) [2023-12-27 01:19:30,141][105620] Updated weights for policy 1, policy_version 1366085 (0.0009) [2023-12-27 01:19:30,199][105620] Updated weights for policy 1, policy_version 1366095 (0.0006) [2023-12-27 01:19:30,247][105620] Updated weights for policy 1, policy_version 1366105 (0.0005) [2023-12-27 01:19:30,293][105692] Updated weights for policy 0, policy_version 1364148 (0.0007) [2023-12-27 01:19:30,351][105692] Updated weights for policy 0, policy_version 1364158 (0.0009) [2023-12-27 01:19:30,406][105692] Updated weights for policy 0, policy_version 1364168 (0.0009) [2023-12-27 01:19:30,948][105620] Updated weights for policy 1, policy_version 1366115 (0.0007) [2023-12-27 01:19:31,000][105620] Updated weights for policy 1, policy_version 1366125 (0.0006) [2023-12-27 01:19:31,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 699047936. Throughput: 0: 9721.1, 1: 9687.7. Samples: 699023904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:19:31,062][105620] Updated weights for policy 1, policy_version 1366135 (0.0007) [2023-12-27 01:19:31,063][104569] Avg episode reward: [(0, '8536.126'), (1, '9082.579')] [2023-12-27 01:19:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001364176_349282304.pth... [2023-12-27 01:19:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001363056_348995584.pth [2023-12-27 01:19:31,114][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001366144_349773824.pth... [2023-12-27 01:19:31,119][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001364992_349478912.pth [2023-12-27 01:19:31,190][105692] Updated weights for policy 0, policy_version 1364178 (0.0011) [2023-12-27 01:19:31,248][105692] Updated weights for policy 0, policy_version 1364188 (0.0009) [2023-12-27 01:19:31,311][105692] Updated weights for policy 0, policy_version 1364198 (0.0009) [2023-12-27 01:19:31,378][105692] Updated weights for policy 0, policy_version 1364208 (0.0009) [2023-12-27 01:19:31,776][105620] Updated weights for policy 1, policy_version 1366145 (0.0009) [2023-12-27 01:19:31,822][105620] Updated weights for policy 1, policy_version 1366155 (0.0009) [2023-12-27 01:19:31,879][105620] Updated weights for policy 1, policy_version 1366165 (0.0009) [2023-12-27 01:19:31,940][105620] Updated weights for policy 1, policy_version 1366175 (0.0009) [2023-12-27 01:19:32,157][105692] Updated weights for policy 0, policy_version 1364218 (0.0009) [2023-12-27 01:19:32,217][105692] Updated weights for policy 0, policy_version 1364228 (0.0007) [2023-12-27 01:19:32,280][105692] Updated weights for policy 0, policy_version 1364238 (0.0006) [2023-12-27 01:19:32,760][105620] Updated weights for policy 1, policy_version 1366185 (0.0009) [2023-12-27 01:19:32,813][105620] Updated weights for policy 1, policy_version 1366195 (0.0009) [2023-12-27 01:19:32,865][105620] Updated weights for policy 1, policy_version 1366205 (0.0009) [2023-12-27 01:19:32,947][105692] Updated weights for policy 0, policy_version 1364248 (0.0006) [2023-12-27 01:19:33,009][105692] Updated weights for policy 0, policy_version 1364259 (0.0006) [2023-12-27 01:19:33,058][105692] Updated weights for policy 0, policy_version 1364269 (0.0008) [2023-12-27 01:19:33,639][105620] Updated weights for policy 1, policy_version 1366216 (0.0010) [2023-12-27 01:19:33,683][105620] Updated weights for policy 1, policy_version 1366226 (0.0010) [2023-12-27 01:19:33,721][105692] Updated weights for policy 0, policy_version 1364279 (0.0006) [2023-12-27 01:19:33,734][105620] Updated weights for policy 1, policy_version 1366236 (0.0010) [2023-12-27 01:19:33,780][105692] Updated weights for policy 0, policy_version 1364289 (0.0006) [2023-12-27 01:19:33,808][105585] KL-divergence is very high: 124.0488 [2023-12-27 01:19:33,840][105692] Updated weights for policy 0, policy_version 1364299 (0.0005) [2023-12-27 01:19:33,852][105585] KL-divergence is very high: 231.4726 [2023-12-27 01:19:34,340][105620] Updated weights for policy 1, policy_version 1366246 (0.0007) [2023-12-27 01:19:34,361][105692] Updated weights for policy 0, policy_version 1364309 (0.0007) [2023-12-27 01:19:34,406][105620] Updated weights for policy 1, policy_version 1366256 (0.0006) [2023-12-27 01:19:34,417][105692] Updated weights for policy 0, policy_version 1364319 (0.0011) [2023-12-27 01:19:34,469][105620] Updated weights for policy 1, policy_version 1366266 (0.0008) [2023-12-27 01:19:34,480][105692] Updated weights for policy 0, policy_version 1364329 (0.0011) [2023-12-27 01:19:35,129][105620] Updated weights for policy 1, policy_version 1366276 (0.0011) [2023-12-27 01:19:35,141][105692] Updated weights for policy 0, policy_version 1364339 (0.0011) [2023-12-27 01:19:35,184][105620] Updated weights for policy 1, policy_version 1366286 (0.0010) [2023-12-27 01:19:35,185][105692] Updated weights for policy 0, policy_version 1364349 (0.0010) [2023-12-27 01:19:35,233][105692] Updated weights for policy 0, policy_version 1364359 (0.0010) [2023-12-27 01:19:35,238][105620] Updated weights for policy 1, policy_version 1366296 (0.0010) [2023-12-27 01:19:35,899][105620] Updated weights for policy 1, policy_version 1366306 (0.0007) [2023-12-27 01:19:35,936][105692] Updated weights for policy 0, policy_version 1364369 (0.0010) [2023-12-27 01:19:35,962][105620] Updated weights for policy 1, policy_version 1366316 (0.0007) [2023-12-27 01:19:35,992][105692] Updated weights for policy 0, policy_version 1364379 (0.0010) [2023-12-27 01:19:36,022][105620] Updated weights for policy 1, policy_version 1366326 (0.0005) [2023-12-27 01:19:36,052][105692] Updated weights for policy 0, policy_version 1364389 (0.0010) [2023-12-27 01:19:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 699146240. Throughput: 0: 9727.9, 1: 9662.1. Samples: 699142284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:19:36,062][104569] Avg episode reward: [(0, '8346.841'), (1, '8994.872')] [2023-12-27 01:19:36,076][105620] Updated weights for policy 1, policy_version 1366336 (0.0007) [2023-12-27 01:19:36,112][105692] Updated weights for policy 0, policy_version 1364399 (0.0008) [2023-12-27 01:19:36,767][105692] Updated weights for policy 0, policy_version 1364409 (0.0008) [2023-12-27 01:19:36,820][105692] Updated weights for policy 0, policy_version 1364419 (0.0010) [2023-12-27 01:19:36,879][105692] Updated weights for policy 0, policy_version 1364429 (0.0006) [2023-12-27 01:19:36,921][105620] Updated weights for policy 1, policy_version 1366346 (0.0008) [2023-12-27 01:19:36,986][105620] Updated weights for policy 1, policy_version 1366356 (0.0008) [2023-12-27 01:19:37,053][105620] Updated weights for policy 1, policy_version 1366366 (0.0010) [2023-12-27 01:19:37,546][105692] Updated weights for policy 0, policy_version 1364439 (0.0008) [2023-12-27 01:19:37,603][105692] Updated weights for policy 0, policy_version 1364449 (0.0009) [2023-12-27 01:19:37,657][105692] Updated weights for policy 0, policy_version 1364459 (0.0007) [2023-12-27 01:19:37,830][105620] Updated weights for policy 1, policy_version 1366376 (0.0006) [2023-12-27 01:19:37,879][105620] Updated weights for policy 1, policy_version 1366386 (0.0005) [2023-12-27 01:19:37,923][105620] Updated weights for policy 1, policy_version 1366396 (0.0005) [2023-12-27 01:19:38,400][105692] Updated weights for policy 0, policy_version 1364469 (0.0007) [2023-12-27 01:19:38,459][105692] Updated weights for policy 0, policy_version 1364479 (0.0006) [2023-12-27 01:19:38,519][105692] Updated weights for policy 0, policy_version 1364489 (0.0007) [2023-12-27 01:19:38,551][105620] Updated weights for policy 1, policy_version 1366406 (0.0009) [2023-12-27 01:19:38,614][105620] Updated weights for policy 1, policy_version 1366416 (0.0011) [2023-12-27 01:19:38,683][105620] Updated weights for policy 1, policy_version 1366426 (0.0011) [2023-12-27 01:19:39,161][105692] Updated weights for policy 0, policy_version 1364499 (0.0006) [2023-12-27 01:19:39,225][105692] Updated weights for policy 0, policy_version 1364509 (0.0008) [2023-12-27 01:19:39,287][105692] Updated weights for policy 0, policy_version 1364519 (0.0009) [2023-12-27 01:19:39,378][105620] Updated weights for policy 1, policy_version 1366436 (0.0010) [2023-12-27 01:19:39,441][105620] Updated weights for policy 1, policy_version 1366446 (0.0011) [2023-12-27 01:19:39,505][105620] Updated weights for policy 1, policy_version 1366456 (0.0009) [2023-12-27 01:19:40,128][105692] Updated weights for policy 0, policy_version 1364529 (0.0008) [2023-12-27 01:19:40,130][105620] Updated weights for policy 1, policy_version 1366466 (0.0006) [2023-12-27 01:19:40,184][105692] Updated weights for policy 0, policy_version 1364539 (0.0007) [2023-12-27 01:19:40,192][105620] Updated weights for policy 1, policy_version 1366476 (0.0011) [2023-12-27 01:19:40,243][105692] Updated weights for policy 0, policy_version 1364549 (0.0008) [2023-12-27 01:19:40,256][105620] Updated weights for policy 1, policy_version 1366486 (0.0011) [2023-12-27 01:19:40,296][105692] Updated weights for policy 0, policy_version 1364559 (0.0007) [2023-12-27 01:19:40,319][105620] Updated weights for policy 1, policy_version 1366496 (0.0011) [2023-12-27 01:19:40,973][105620] Updated weights for policy 1, policy_version 1366506 (0.0008) [2023-12-27 01:19:41,007][105692] Updated weights for policy 0, policy_version 1364569 (0.0006) [2023-12-27 01:19:41,018][105620] Updated weights for policy 1, policy_version 1366516 (0.0010) [2023-12-27 01:19:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 699244544. Throughput: 0: 9730.6, 1: 9743.1. Samples: 699260960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:19:41,063][104569] Avg episode reward: [(0, '8163.459'), (1, '8808.720')] [2023-12-27 01:19:41,075][105692] Updated weights for policy 0, policy_version 1364579 (0.0008) [2023-12-27 01:19:41,085][105620] Updated weights for policy 1, policy_version 1366526 (0.0011) [2023-12-27 01:19:41,146][105692] Updated weights for policy 0, policy_version 1364589 (0.0008) [2023-12-27 01:19:41,842][105692] Updated weights for policy 0, policy_version 1364599 (0.0007) [2023-12-27 01:19:41,885][105620] Updated weights for policy 1, policy_version 1366536 (0.0011) [2023-12-27 01:19:41,897][105692] Updated weights for policy 0, policy_version 1364609 (0.0007) [2023-12-27 01:19:41,942][105620] Updated weights for policy 1, policy_version 1366546 (0.0011) [2023-12-27 01:19:41,956][105692] Updated weights for policy 0, policy_version 1364619 (0.0005) [2023-12-27 01:19:41,998][105620] Updated weights for policy 1, policy_version 1366556 (0.0011) [2023-12-27 01:19:42,637][105692] Updated weights for policy 0, policy_version 1364629 (0.0007) [2023-12-27 01:19:42,701][105692] Updated weights for policy 0, policy_version 1364639 (0.0008) [2023-12-27 01:19:42,765][105692] Updated weights for policy 0, policy_version 1364649 (0.0009) [2023-12-27 01:19:42,780][105620] Updated weights for policy 1, policy_version 1366566 (0.0011) [2023-12-27 01:19:42,847][105620] Updated weights for policy 1, policy_version 1366576 (0.0011) [2023-12-27 01:19:42,912][105620] Updated weights for policy 1, policy_version 1366586 (0.0009) [2023-12-27 01:19:43,467][105692] Updated weights for policy 0, policy_version 1364659 (0.0008) [2023-12-27 01:19:43,524][105692] Updated weights for policy 0, policy_version 1364669 (0.0009) [2023-12-27 01:19:43,588][105692] Updated weights for policy 0, policy_version 1364679 (0.0009) [2023-12-27 01:19:43,619][105620] Updated weights for policy 1, policy_version 1366596 (0.0009) [2023-12-27 01:19:43,671][105620] Updated weights for policy 1, policy_version 1366606 (0.0009) [2023-12-27 01:19:43,730][105620] Updated weights for policy 1, policy_version 1366617 (0.0010) [2023-12-27 01:19:44,160][105692] Updated weights for policy 0, policy_version 1364689 (0.0006) [2023-12-27 01:19:44,223][105692] Updated weights for policy 0, policy_version 1364699 (0.0008) [2023-12-27 01:19:44,286][105692] Updated weights for policy 0, policy_version 1364709 (0.0010) [2023-12-27 01:19:44,346][105692] Updated weights for policy 0, policy_version 1364719 (0.0005) [2023-12-27 01:19:44,644][105620] Updated weights for policy 1, policy_version 1366627 (0.0009) [2023-12-27 01:19:44,701][105620] Updated weights for policy 1, policy_version 1366637 (0.0008) [2023-12-27 01:19:44,765][105620] Updated weights for policy 1, policy_version 1366647 (0.0009) [2023-12-27 01:19:44,944][105692] Updated weights for policy 0, policy_version 1364729 (0.0008) [2023-12-27 01:19:45,000][105692] Updated weights for policy 0, policy_version 1364739 (0.0009) [2023-12-27 01:19:45,057][105692] Updated weights for policy 0, policy_version 1364749 (0.0010) [2023-12-27 01:19:45,517][105620] Updated weights for policy 1, policy_version 1366657 (0.0009) [2023-12-27 01:19:45,568][105620] Updated weights for policy 1, policy_version 1366667 (0.0009) [2023-12-27 01:19:45,615][105620] Updated weights for policy 1, policy_version 1366677 (0.0008) [2023-12-27 01:19:45,669][105620] Updated weights for policy 1, policy_version 1366687 (0.0008) [2023-12-27 01:19:45,822][105692] Updated weights for policy 0, policy_version 1364759 (0.0009) [2023-12-27 01:19:45,884][105692] Updated weights for policy 0, policy_version 1364769 (0.0008) [2023-12-27 01:19:45,931][105692] Updated weights for policy 0, policy_version 1364779 (0.0009) [2023-12-27 01:19:46,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 699351040. Throughput: 0: 9678.8, 1: 9699.1. Samples: 699317928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:19:46,063][104569] Avg episode reward: [(0, '8533.154'), (1, '9078.579')] [2023-12-27 01:19:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001364784_349437952.pth... [2023-12-27 01:19:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001366688_349913088.pth... [2023-12-27 01:19:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001363600_349134848.pth [2023-12-27 01:19:46,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001365568_349626368.pth [2023-12-27 01:19:46,456][105620] Updated weights for policy 1, policy_version 1366697 (0.0009) [2023-12-27 01:19:46,512][105620] Updated weights for policy 1, policy_version 1366707 (0.0009) [2023-12-27 01:19:46,570][105620] Updated weights for policy 1, policy_version 1366717 (0.0007) [2023-12-27 01:19:46,655][105692] Updated weights for policy 0, policy_version 1364789 (0.0009) [2023-12-27 01:19:46,713][105692] Updated weights for policy 0, policy_version 1364799 (0.0005) [2023-12-27 01:19:46,772][105692] Updated weights for policy 0, policy_version 1364809 (0.0005) [2023-12-27 01:19:47,165][105620] Updated weights for policy 1, policy_version 1366727 (0.0005) [2023-12-27 01:19:47,220][105620] Updated weights for policy 1, policy_version 1366737 (0.0006) [2023-12-27 01:19:47,267][105620] Updated weights for policy 1, policy_version 1366747 (0.0005) [2023-12-27 01:19:47,521][105692] Updated weights for policy 0, policy_version 1364819 (0.0006) [2023-12-27 01:19:47,582][105692] Updated weights for policy 0, policy_version 1364829 (0.0009) [2023-12-27 01:19:47,641][105692] Updated weights for policy 0, policy_version 1364839 (0.0006) [2023-12-27 01:19:47,873][105620] Updated weights for policy 1, policy_version 1366757 (0.0007) [2023-12-27 01:19:47,922][105620] Updated weights for policy 1, policy_version 1366767 (0.0008) [2023-12-27 01:19:47,975][105620] Updated weights for policy 1, policy_version 1366777 (0.0008) [2023-12-27 01:19:48,327][105692] Updated weights for policy 0, policy_version 1364849 (0.0006) [2023-12-27 01:19:48,399][105692] Updated weights for policy 0, policy_version 1364859 (0.0009) [2023-12-27 01:19:48,459][105692] Updated weights for policy 0, policy_version 1364869 (0.0010) [2023-12-27 01:19:48,508][105692] Updated weights for policy 0, policy_version 1364879 (0.0010) [2023-12-27 01:19:48,731][105620] Updated weights for policy 1, policy_version 1366787 (0.0008) [2023-12-27 01:19:48,784][105620] Updated weights for policy 1, policy_version 1366797 (0.0010) [2023-12-27 01:19:48,840][105620] Updated weights for policy 1, policy_version 1366807 (0.0009) [2023-12-27 01:19:49,088][105692] Updated weights for policy 0, policy_version 1364889 (0.0006) [2023-12-27 01:19:49,151][105692] Updated weights for policy 0, policy_version 1364899 (0.0005) [2023-12-27 01:19:49,198][105692] Updated weights for policy 0, policy_version 1364909 (0.0005) [2023-12-27 01:19:49,670][105620] Updated weights for policy 1, policy_version 1366817 (0.0009) [2023-12-27 01:19:49,723][105620] Updated weights for policy 1, policy_version 1366827 (0.0006) [2023-12-27 01:19:49,779][105620] Updated weights for policy 1, policy_version 1366837 (0.0009) [2023-12-27 01:19:49,813][105692] Updated weights for policy 0, policy_version 1364919 (0.0006) [2023-12-27 01:19:49,834][105620] Updated weights for policy 1, policy_version 1366847 (0.0008) [2023-12-27 01:19:49,876][105692] Updated weights for policy 0, policy_version 1364929 (0.0008) [2023-12-27 01:19:49,934][105692] Updated weights for policy 0, policy_version 1364939 (0.0007) [2023-12-27 01:19:50,593][105620] Updated weights for policy 1, policy_version 1366857 (0.0010) [2023-12-27 01:19:50,639][105692] Updated weights for policy 0, policy_version 1364949 (0.0007) [2023-12-27 01:19:50,659][105620] Updated weights for policy 1, policy_version 1366867 (0.0008) [2023-12-27 01:19:50,702][105692] Updated weights for policy 0, policy_version 1364959 (0.0006) [2023-12-27 01:19:50,725][105620] Updated weights for policy 1, policy_version 1366877 (0.0010) [2023-12-27 01:19:50,763][105692] Updated weights for policy 0, policy_version 1364969 (0.0009) [2023-12-27 01:19:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 699449344. Throughput: 0: 9866.4, 1: 9629.1. Samples: 699437204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:19:51,062][104569] Avg episode reward: [(0, '8439.802'), (1, '8994.700')] [2023-12-27 01:19:51,441][105620] Updated weights for policy 1, policy_version 1366887 (0.0009) [2023-12-27 01:19:51,493][105620] Updated weights for policy 1, policy_version 1366897 (0.0010) [2023-12-27 01:19:51,544][105692] Updated weights for policy 0, policy_version 1364979 (0.0010) [2023-12-27 01:19:51,553][105620] Updated weights for policy 1, policy_version 1366907 (0.0010) [2023-12-27 01:19:51,605][105692] Updated weights for policy 0, policy_version 1364989 (0.0009) [2023-12-27 01:19:51,672][105692] Updated weights for policy 0, policy_version 1364999 (0.0009) [2023-12-27 01:19:52,323][105620] Updated weights for policy 1, policy_version 1366917 (0.0010) [2023-12-27 01:19:52,387][105692] Updated weights for policy 0, policy_version 1365009 (0.0009) [2023-12-27 01:19:52,389][105620] Updated weights for policy 1, policy_version 1366927 (0.0009) [2023-12-27 01:19:52,447][105692] Updated weights for policy 0, policy_version 1365019 (0.0011) [2023-12-27 01:19:52,453][105620] Updated weights for policy 1, policy_version 1366937 (0.0010) [2023-12-27 01:19:52,509][105692] Updated weights for policy 0, policy_version 1365029 (0.0011) [2023-12-27 01:19:52,576][105692] Updated weights for policy 0, policy_version 1365039 (0.0010) [2023-12-27 01:19:53,176][105620] Updated weights for policy 1, policy_version 1366947 (0.0010) [2023-12-27 01:19:53,235][105620] Updated weights for policy 1, policy_version 1366957 (0.0009) [2023-12-27 01:19:53,299][105620] Updated weights for policy 1, policy_version 1366967 (0.0010) [2023-12-27 01:19:53,301][105692] Updated weights for policy 0, policy_version 1365049 (0.0007) [2023-12-27 01:19:53,361][105692] Updated weights for policy 0, policy_version 1365059 (0.0007) [2023-12-27 01:19:53,427][105692] Updated weights for policy 0, policy_version 1365069 (0.0005) [2023-12-27 01:19:53,957][105692] Updated weights for policy 0, policy_version 1365079 (0.0005) [2023-12-27 01:19:54,010][105620] Updated weights for policy 1, policy_version 1366977 (0.0007) [2023-12-27 01:19:54,019][105692] Updated weights for policy 0, policy_version 1365089 (0.0005) [2023-12-27 01:19:54,065][105620] Updated weights for policy 1, policy_version 1366987 (0.0010) [2023-12-27 01:19:54,075][105692] Updated weights for policy 0, policy_version 1365099 (0.0005) [2023-12-27 01:19:54,127][105620] Updated weights for policy 1, policy_version 1366997 (0.0010) [2023-12-27 01:19:54,183][105620] Updated weights for policy 1, policy_version 1367007 (0.0010) [2023-12-27 01:19:54,739][105692] Updated weights for policy 0, policy_version 1365109 (0.0006) [2023-12-27 01:19:54,796][105692] Updated weights for policy 0, policy_version 1365119 (0.0005) [2023-12-27 01:19:54,820][105620] Updated weights for policy 1, policy_version 1367017 (0.0006) [2023-12-27 01:19:54,848][105692] Updated weights for policy 0, policy_version 1365129 (0.0008) [2023-12-27 01:19:54,877][105620] Updated weights for policy 1, policy_version 1367027 (0.0005) [2023-12-27 01:19:54,929][105620] Updated weights for policy 1, policy_version 1367037 (0.0005) [2023-12-27 01:19:55,419][105692] Updated weights for policy 0, policy_version 1365139 (0.0006) [2023-12-27 01:19:55,485][105692] Updated weights for policy 0, policy_version 1365149 (0.0006) [2023-12-27 01:19:55,515][105620] Updated weights for policy 1, policy_version 1367047 (0.0005) [2023-12-27 01:19:55,538][105692] Updated weights for policy 0, policy_version 1365159 (0.0008) [2023-12-27 01:19:55,571][105620] Updated weights for policy 1, policy_version 1367057 (0.0005) [2023-12-27 01:19:55,621][105620] Updated weights for policy 1, policy_version 1367067 (0.0007) [2023-12-27 01:19:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 699547648. Throughput: 0: 9980.0, 1: 9702.4. Samples: 699558484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:19:56,063][104569] Avg episode reward: [(0, '8439.941'), (1, '8994.933')] [2023-12-27 01:19:56,189][105692] Updated weights for policy 0, policy_version 1365169 (0.0009) [2023-12-27 01:19:56,245][105692] Updated weights for policy 0, policy_version 1365179 (0.0011) [2023-12-27 01:19:56,290][105692] Updated weights for policy 0, policy_version 1365189 (0.0010) [2023-12-27 01:19:56,330][105620] Updated weights for policy 1, policy_version 1367077 (0.0009) [2023-12-27 01:19:56,350][105692] Updated weights for policy 0, policy_version 1365199 (0.0010) [2023-12-27 01:19:56,389][105620] Updated weights for policy 1, policy_version 1367087 (0.0010) [2023-12-27 01:19:56,447][105620] Updated weights for policy 1, policy_version 1367097 (0.0010) [2023-12-27 01:19:56,971][105692] Updated weights for policy 0, policy_version 1365209 (0.0010) [2023-12-27 01:19:57,022][105692] Updated weights for policy 0, policy_version 1365219 (0.0008) [2023-12-27 01:19:57,046][105620] Updated weights for policy 1, policy_version 1367107 (0.0009) [2023-12-27 01:19:57,070][105692] Updated weights for policy 0, policy_version 1365229 (0.0009) [2023-12-27 01:19:57,092][105620] Updated weights for policy 1, policy_version 1367117 (0.0005) [2023-12-27 01:19:57,145][105620] Updated weights for policy 1, policy_version 1367127 (0.0005) [2023-12-27 01:19:57,666][105620] Updated weights for policy 1, policy_version 1367137 (0.0005) [2023-12-27 01:19:57,729][105620] Updated weights for policy 1, policy_version 1367147 (0.0006) [2023-12-27 01:19:57,780][105620] Updated weights for policy 1, policy_version 1367157 (0.0005) [2023-12-27 01:19:57,831][105620] Updated weights for policy 1, policy_version 1367167 (0.0005) [2023-12-27 01:19:57,973][105692] Updated weights for policy 0, policy_version 1365239 (0.0006) [2023-12-27 01:19:58,038][105692] Updated weights for policy 0, policy_version 1365249 (0.0006) [2023-12-27 01:19:58,092][105692] Updated weights for policy 0, policy_version 1365259 (0.0010) [2023-12-27 01:19:58,426][105620] Updated weights for policy 1, policy_version 1367177 (0.0009) [2023-12-27 01:19:58,490][105620] Updated weights for policy 1, policy_version 1367187 (0.0010) [2023-12-27 01:19:58,552][105620] Updated weights for policy 1, policy_version 1367197 (0.0010) [2023-12-27 01:19:58,818][105692] Updated weights for policy 0, policy_version 1365269 (0.0010) [2023-12-27 01:19:58,882][105692] Updated weights for policy 0, policy_version 1365279 (0.0010) [2023-12-27 01:19:58,949][105692] Updated weights for policy 0, policy_version 1365289 (0.0009) [2023-12-27 01:19:59,390][105620] Updated weights for policy 1, policy_version 1367207 (0.0009) [2023-12-27 01:19:59,447][105620] Updated weights for policy 1, policy_version 1367218 (0.0010) [2023-12-27 01:19:59,512][105620] Updated weights for policy 1, policy_version 1367228 (0.0010) [2023-12-27 01:19:59,680][105692] Updated weights for policy 0, policy_version 1365299 (0.0007) [2023-12-27 01:19:59,745][105692] Updated weights for policy 0, policy_version 1365309 (0.0005) [2023-12-27 01:19:59,803][105692] Updated weights for policy 0, policy_version 1365319 (0.0006) [2023-12-27 01:20:00,276][105620] Updated weights for policy 1, policy_version 1367238 (0.0006) [2023-12-27 01:20:00,326][105620] Updated weights for policy 1, policy_version 1367248 (0.0005) [2023-12-27 01:20:00,370][105692] Updated weights for policy 0, policy_version 1365329 (0.0008) [2023-12-27 01:20:00,389][105620] Updated weights for policy 1, policy_version 1367258 (0.0006) [2023-12-27 01:20:00,424][105692] Updated weights for policy 0, policy_version 1365339 (0.0007) [2023-12-27 01:20:00,477][105692] Updated weights for policy 0, policy_version 1365349 (0.0009) [2023-12-27 01:20:00,530][105692] Updated weights for policy 0, policy_version 1365359 (0.0009) [2023-12-27 01:20:01,001][105620] Updated weights for policy 1, policy_version 1367268 (0.0007) [2023-12-27 01:20:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 699645952. Throughput: 0: 9980.3, 1: 9804.7. Samples: 699620020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:20:01,062][104569] Avg episode reward: [(0, '8438.051'), (1, '9354.735')] [2023-12-27 01:20:01,064][105620] Updated weights for policy 1, policy_version 1367278 (0.0010) [2023-12-27 01:20:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001365360_349585408.pth... [2023-12-27 01:20:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001364176_349282304.pth [2023-12-27 01:20:01,120][105620] Updated weights for policy 1, policy_version 1367288 (0.0009) [2023-12-27 01:20:01,164][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001367296_350068736.pth... [2023-12-27 01:20:01,167][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001366144_349773824.pth [2023-12-27 01:20:01,331][105692] Updated weights for policy 0, policy_version 1365369 (0.0010) [2023-12-27 01:20:01,399][105692] Updated weights for policy 0, policy_version 1365379 (0.0009) [2023-12-27 01:20:01,463][105692] Updated weights for policy 0, policy_version 1365389 (0.0009) [2023-12-27 01:20:01,848][105620] Updated weights for policy 1, policy_version 1367298 (0.0009) [2023-12-27 01:20:01,897][105620] Updated weights for policy 1, policy_version 1367308 (0.0005) [2023-12-27 01:20:01,943][105620] Updated weights for policy 1, policy_version 1367318 (0.0008) [2023-12-27 01:20:01,995][105586] KL-divergence is very high: 116.1348 [2023-12-27 01:20:02,002][105620] Updated weights for policy 1, policy_version 1367328 (0.0009) [2023-12-27 01:20:02,248][105692] Updated weights for policy 0, policy_version 1365399 (0.0008) [2023-12-27 01:20:02,301][105692] Updated weights for policy 0, policy_version 1365409 (0.0008) [2023-12-27 01:20:02,359][105692] Updated weights for policy 0, policy_version 1365419 (0.0009) [2023-12-27 01:20:02,721][105620] Updated weights for policy 1, policy_version 1367338 (0.0005) [2023-12-27 01:20:02,775][105620] Updated weights for policy 1, policy_version 1367348 (0.0005) [2023-12-27 01:20:02,848][105620] Updated weights for policy 1, policy_version 1367358 (0.0008) [2023-12-27 01:20:03,191][105692] Updated weights for policy 0, policy_version 1365429 (0.0009) [2023-12-27 01:20:03,250][105692] Updated weights for policy 0, policy_version 1365439 (0.0009) [2023-12-27 01:20:03,313][105692] Updated weights for policy 0, policy_version 1365449 (0.0009) [2023-12-27 01:20:03,553][105620] Updated weights for policy 1, policy_version 1367368 (0.0008) [2023-12-27 01:20:03,607][105620] Updated weights for policy 1, policy_version 1367378 (0.0008) [2023-12-27 01:20:03,653][105620] Updated weights for policy 1, policy_version 1367388 (0.0008) [2023-12-27 01:20:03,974][105692] Updated weights for policy 0, policy_version 1365459 (0.0009) [2023-12-27 01:20:04,027][105692] Updated weights for policy 0, policy_version 1365469 (0.0008) [2023-12-27 01:20:04,076][105692] Updated weights for policy 0, policy_version 1365479 (0.0007) [2023-12-27 01:20:04,400][105620] Updated weights for policy 1, policy_version 1367398 (0.0009) [2023-12-27 01:20:04,444][105620] Updated weights for policy 1, policy_version 1367408 (0.0007) [2023-12-27 01:20:04,498][105620] Updated weights for policy 1, policy_version 1367418 (0.0008) [2023-12-27 01:20:04,768][105692] Updated weights for policy 0, policy_version 1365489 (0.0008) [2023-12-27 01:20:04,828][105692] Updated weights for policy 0, policy_version 1365499 (0.0005) [2023-12-27 01:20:04,882][105692] Updated weights for policy 0, policy_version 1365509 (0.0009) [2023-12-27 01:20:04,930][105692] Updated weights for policy 0, policy_version 1365519 (0.0010) [2023-12-27 01:20:05,265][105620] Updated weights for policy 1, policy_version 1367428 (0.0010) [2023-12-27 01:20:05,320][105620] Updated weights for policy 1, policy_version 1367438 (0.0007) [2023-12-27 01:20:05,371][105620] Updated weights for policy 1, policy_version 1367448 (0.0008) [2023-12-27 01:20:05,654][105692] Updated weights for policy 0, policy_version 1365529 (0.0010) [2023-12-27 01:20:05,715][105692] Updated weights for policy 0, policy_version 1365539 (0.0010) [2023-12-27 01:20:05,779][105692] Updated weights for policy 0, policy_version 1365549 (0.0010) [2023-12-27 01:20:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 699744256. Throughput: 0: 9990.6, 1: 9844.8. Samples: 699734612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:20:06,063][104569] Avg episode reward: [(0, '8256.903'), (1, '9086.052')] [2023-12-27 01:20:06,073][105620] Updated weights for policy 1, policy_version 1367458 (0.0008) [2023-12-27 01:20:06,129][105620] Updated weights for policy 1, policy_version 1367468 (0.0008) [2023-12-27 01:20:06,191][105620] Updated weights for policy 1, policy_version 1367478 (0.0005) [2023-12-27 01:20:06,238][105620] Updated weights for policy 1, policy_version 1367488 (0.0005) [2023-12-27 01:20:06,516][105692] Updated weights for policy 0, policy_version 1365559 (0.0010) [2023-12-27 01:20:06,575][105692] Updated weights for policy 0, policy_version 1365569 (0.0010) [2023-12-27 01:20:06,630][105692] Updated weights for policy 0, policy_version 1365579 (0.0010) [2023-12-27 01:20:06,881][105620] Updated weights for policy 1, policy_version 1367498 (0.0006) [2023-12-27 01:20:06,935][105620] Updated weights for policy 1, policy_version 1367508 (0.0005) [2023-12-27 01:20:06,995][105620] Updated weights for policy 1, policy_version 1367518 (0.0005) [2023-12-27 01:20:07,400][105692] Updated weights for policy 0, policy_version 1365589 (0.0009) [2023-12-27 01:20:07,465][105692] Updated weights for policy 0, policy_version 1365599 (0.0008) [2023-12-27 01:20:07,521][105692] Updated weights for policy 0, policy_version 1365609 (0.0006) [2023-12-27 01:20:07,536][105620] Updated weights for policy 1, policy_version 1367528 (0.0010) [2023-12-27 01:20:07,596][105620] Updated weights for policy 1, policy_version 1367538 (0.0005) [2023-12-27 01:20:07,651][105620] Updated weights for policy 1, policy_version 1367548 (0.0005) [2023-12-27 01:20:08,269][105692] Updated weights for policy 0, policy_version 1365619 (0.0007) [2023-12-27 01:20:08,284][105620] Updated weights for policy 1, policy_version 1367558 (0.0005) [2023-12-27 01:20:08,334][105692] Updated weights for policy 0, policy_version 1365629 (0.0009) [2023-12-27 01:20:08,347][105620] Updated weights for policy 1, policy_version 1367568 (0.0007) [2023-12-27 01:20:08,401][105692] Updated weights for policy 0, policy_version 1365639 (0.0008) [2023-12-27 01:20:08,410][105620] Updated weights for policy 1, policy_version 1367578 (0.0008) [2023-12-27 01:20:09,125][105692] Updated weights for policy 0, policy_version 1365649 (0.0007) [2023-12-27 01:20:09,165][105620] Updated weights for policy 1, policy_version 1367588 (0.0007) [2023-12-27 01:20:09,182][105692] Updated weights for policy 0, policy_version 1365659 (0.0007) [2023-12-27 01:20:09,223][105620] Updated weights for policy 1, policy_version 1367598 (0.0009) [2023-12-27 01:20:09,245][105692] Updated weights for policy 0, policy_version 1365669 (0.0008) [2023-12-27 01:20:09,277][105620] Updated weights for policy 1, policy_version 1367608 (0.0006) [2023-12-27 01:20:09,300][105692] Updated weights for policy 0, policy_version 1365679 (0.0008) [2023-12-27 01:20:10,017][105620] Updated weights for policy 1, policy_version 1367618 (0.0007) [2023-12-27 01:20:10,068][105620] Updated weights for policy 1, policy_version 1367628 (0.0005) [2023-12-27 01:20:10,110][105692] Updated weights for policy 0, policy_version 1365689 (0.0008) [2023-12-27 01:20:10,128][105620] Updated weights for policy 1, policy_version 1367638 (0.0005) [2023-12-27 01:20:10,160][105692] Updated weights for policy 0, policy_version 1365699 (0.0009) [2023-12-27 01:20:10,196][105620] Updated weights for policy 1, policy_version 1367648 (0.0006) [2023-12-27 01:20:10,212][105692] Updated weights for policy 0, policy_version 1365709 (0.0007) [2023-12-27 01:20:10,822][105620] Updated weights for policy 1, policy_version 1367658 (0.0009) [2023-12-27 01:20:10,877][105620] Updated weights for policy 1, policy_version 1367668 (0.0007) [2023-12-27 01:20:10,929][105620] Updated weights for policy 1, policy_version 1367678 (0.0007) [2023-12-27 01:20:11,032][105692] Updated weights for policy 0, policy_version 1365719 (0.0007) [2023-12-27 01:20:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 699842560. Throughput: 0: 9893.8, 1: 9909.3. Samples: 699851604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:20:11,062][104569] Avg episode reward: [(0, '8167.752'), (1, '8903.642')] [2023-12-27 01:20:11,099][105692] Updated weights for policy 0, policy_version 1365729 (0.0007) [2023-12-27 01:20:11,167][105692] Updated weights for policy 0, policy_version 1365739 (0.0008) [2023-12-27 01:20:11,656][105620] Updated weights for policy 1, policy_version 1367688 (0.0009) [2023-12-27 01:20:11,720][105620] Updated weights for policy 1, policy_version 1367698 (0.0009) [2023-12-27 01:20:11,787][105620] Updated weights for policy 1, policy_version 1367708 (0.0007) [2023-12-27 01:20:11,964][105692] Updated weights for policy 0, policy_version 1365749 (0.0008) [2023-12-27 01:20:12,029][105692] Updated weights for policy 0, policy_version 1365759 (0.0009) [2023-12-27 01:20:12,095][105692] Updated weights for policy 0, policy_version 1365769 (0.0008) [2023-12-27 01:20:12,461][105620] Updated weights for policy 1, policy_version 1367718 (0.0008) [2023-12-27 01:20:12,512][105620] Updated weights for policy 1, policy_version 1367728 (0.0008) [2023-12-27 01:20:12,573][105620] Updated weights for policy 1, policy_version 1367738 (0.0008) [2023-12-27 01:20:12,819][105692] Updated weights for policy 0, policy_version 1365779 (0.0007) [2023-12-27 01:20:12,888][105692] Updated weights for policy 0, policy_version 1365789 (0.0006) [2023-12-27 01:20:12,944][105692] Updated weights for policy 0, policy_version 1365799 (0.0009) [2023-12-27 01:20:13,360][105620] Updated weights for policy 1, policy_version 1367748 (0.0009) [2023-12-27 01:20:13,415][105620] Updated weights for policy 1, policy_version 1367758 (0.0005) [2023-12-27 01:20:13,483][105620] Updated weights for policy 1, policy_version 1367768 (0.0008) [2023-12-27 01:20:13,675][105692] Updated weights for policy 0, policy_version 1365809 (0.0009) [2023-12-27 01:20:13,733][105692] Updated weights for policy 0, policy_version 1365819 (0.0009) [2023-12-27 01:20:13,786][105692] Updated weights for policy 0, policy_version 1365829 (0.0009) [2023-12-27 01:20:13,838][105692] Updated weights for policy 0, policy_version 1365839 (0.0009) [2023-12-27 01:20:14,192][105620] Updated weights for policy 1, policy_version 1367778 (0.0009) [2023-12-27 01:20:14,247][105620] Updated weights for policy 1, policy_version 1367788 (0.0010) [2023-12-27 01:20:14,303][105620] Updated weights for policy 1, policy_version 1367798 (0.0008) [2023-12-27 01:20:14,359][105620] Updated weights for policy 1, policy_version 1367808 (0.0010) [2023-12-27 01:20:14,477][105692] Updated weights for policy 0, policy_version 1365849 (0.0009) [2023-12-27 01:20:14,531][105692] Updated weights for policy 0, policy_version 1365859 (0.0009) [2023-12-27 01:20:14,595][105692] Updated weights for policy 0, policy_version 1365869 (0.0009) [2023-12-27 01:20:15,183][105620] Updated weights for policy 1, policy_version 1367818 (0.0009) [2023-12-27 01:20:15,237][105620] Updated weights for policy 1, policy_version 1367828 (0.0009) [2023-12-27 01:20:15,295][105692] Updated weights for policy 0, policy_version 1365879 (0.0006) [2023-12-27 01:20:15,297][105620] Updated weights for policy 1, policy_version 1367838 (0.0009) [2023-12-27 01:20:15,345][105692] Updated weights for policy 0, policy_version 1365889 (0.0008) [2023-12-27 01:20:15,405][105692] Updated weights for policy 0, policy_version 1365899 (0.0009) [2023-12-27 01:20:16,029][105620] Updated weights for policy 1, policy_version 1367848 (0.0006) [2023-12-27 01:20:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 699932672. Throughput: 0: 9771.1, 1: 9884.0. Samples: 699908384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:20:16,063][104569] Avg episode reward: [(0, '8257.385'), (1, '8993.780')] [2023-12-27 01:20:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001365904_349724672.pth... [2023-12-27 01:20:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001364784_349437952.pth [2023-12-27 01:20:16,080][105620] Updated weights for policy 1, policy_version 1367858 (0.0005) [2023-12-27 01:20:16,132][105620] Updated weights for policy 1, policy_version 1367868 (0.0006) [2023-12-27 01:20:16,154][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001367872_350216192.pth... [2023-12-27 01:20:16,157][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001366688_349913088.pth [2023-12-27 01:20:16,175][105692] Updated weights for policy 0, policy_version 1365909 (0.0009) [2023-12-27 01:20:16,236][105692] Updated weights for policy 0, policy_version 1365919 (0.0010) [2023-12-27 01:20:16,294][105692] Updated weights for policy 0, policy_version 1365929 (0.0010) [2023-12-27 01:20:16,745][105620] Updated weights for policy 1, policy_version 1367878 (0.0007) [2023-12-27 01:20:16,808][105620] Updated weights for policy 1, policy_version 1367888 (0.0005) [2023-12-27 01:20:16,868][105620] Updated weights for policy 1, policy_version 1367898 (0.0005) [2023-12-27 01:20:17,124][105692] Updated weights for policy 0, policy_version 1365941 (0.0010) [2023-12-27 01:20:17,171][105692] Updated weights for policy 0, policy_version 1365951 (0.0009) [2023-12-27 01:20:17,219][105692] Updated weights for policy 0, policy_version 1365961 (0.0008) [2023-12-27 01:20:17,471][105620] Updated weights for policy 1, policy_version 1367908 (0.0005) [2023-12-27 01:20:17,519][105620] Updated weights for policy 1, policy_version 1367918 (0.0005) [2023-12-27 01:20:17,566][105620] Updated weights for policy 1, policy_version 1367928 (0.0009) [2023-12-27 01:20:18,021][105692] Updated weights for policy 0, policy_version 1365971 (0.0009) [2023-12-27 01:20:18,085][105692] Updated weights for policy 0, policy_version 1365981 (0.0010) [2023-12-27 01:20:18,142][105692] Updated weights for policy 0, policy_version 1365991 (0.0009) [2023-12-27 01:20:18,310][105620] Updated weights for policy 1, policy_version 1367938 (0.0009) [2023-12-27 01:20:18,372][105620] Updated weights for policy 1, policy_version 1367948 (0.0009) [2023-12-27 01:20:18,373][105586] KL-divergence is very high: 134.2985 [2023-12-27 01:20:18,420][105586] KL-divergence is very high: 241.7376 [2023-12-27 01:20:18,434][105620] Updated weights for policy 1, policy_version 1367958 (0.0009) [2023-12-27 01:20:18,470][105586] KL-divergence is very high: 249.8436 [2023-12-27 01:20:18,491][105620] Updated weights for policy 1, policy_version 1367968 (0.0009) [2023-12-27 01:20:18,901][105692] Updated weights for policy 0, policy_version 1366001 (0.0009) [2023-12-27 01:20:18,960][105692] Updated weights for policy 0, policy_version 1366011 (0.0006) [2023-12-27 01:20:19,021][105692] Updated weights for policy 0, policy_version 1366021 (0.0007) [2023-12-27 01:20:19,082][105692] Updated weights for policy 0, policy_version 1366031 (0.0008) [2023-12-27 01:20:19,222][105620] Updated weights for policy 1, policy_version 1367978 (0.0011) [2023-12-27 01:20:19,292][105620] Updated weights for policy 1, policy_version 1367988 (0.0011) [2023-12-27 01:20:19,360][105620] Updated weights for policy 1, policy_version 1367998 (0.0007) [2023-12-27 01:20:19,805][105692] Updated weights for policy 0, policy_version 1366041 (0.0006) [2023-12-27 01:20:19,871][105692] Updated weights for policy 0, policy_version 1366051 (0.0008) [2023-12-27 01:20:19,940][105692] Updated weights for policy 0, policy_version 1366061 (0.0008) [2023-12-27 01:20:20,129][105620] Updated weights for policy 1, policy_version 1368008 (0.0009) [2023-12-27 01:20:20,195][105620] Updated weights for policy 1, policy_version 1368018 (0.0009) [2023-12-27 01:20:20,258][105620] Updated weights for policy 1, policy_version 1368028 (0.0009) [2023-12-27 01:20:20,716][105692] Updated weights for policy 0, policy_version 1366071 (0.0010) [2023-12-27 01:20:20,778][105692] Updated weights for policy 0, policy_version 1366081 (0.0010) [2023-12-27 01:20:20,833][105692] Updated weights for policy 0, policy_version 1366091 (0.0009) [2023-12-27 01:20:20,908][105620] Updated weights for policy 1, policy_version 1368038 (0.0009) [2023-12-27 01:20:20,957][105620] Updated weights for policy 1, policy_version 1368048 (0.0008) [2023-12-27 01:20:21,023][105620] Updated weights for policy 1, policy_version 1368058 (0.0008) [2023-12-27 01:20:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 700039168. Throughput: 0: 9744.4, 1: 9836.3. Samples: 700023420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:20:21,063][104569] Avg episode reward: [(0, '8623.997'), (1, '9085.220')] [2023-12-27 01:20:21,666][105692] Updated weights for policy 0, policy_version 1366101 (0.0009) [2023-12-27 01:20:21,730][105692] Updated weights for policy 0, policy_version 1366111 (0.0009) [2023-12-27 01:20:21,763][105620] Updated weights for policy 1, policy_version 1368068 (0.0008) [2023-12-27 01:20:21,794][105692] Updated weights for policy 0, policy_version 1366121 (0.0009) [2023-12-27 01:20:21,828][105620] Updated weights for policy 1, policy_version 1368078 (0.0008) [2023-12-27 01:20:21,901][105620] Updated weights for policy 1, policy_version 1368088 (0.0009) [2023-12-27 01:20:22,573][105692] Updated weights for policy 0, policy_version 1366131 (0.0007) [2023-12-27 01:20:22,644][105692] Updated weights for policy 0, policy_version 1366141 (0.0010) [2023-12-27 01:20:22,675][105620] Updated weights for policy 1, policy_version 1368098 (0.0009) [2023-12-27 01:20:22,710][105692] Updated weights for policy 0, policy_version 1366151 (0.0007) [2023-12-27 01:20:22,733][105620] Updated weights for policy 1, policy_version 1368108 (0.0008) [2023-12-27 01:20:22,794][105620] Updated weights for policy 1, policy_version 1368118 (0.0010) [2023-12-27 01:20:22,860][105620] Updated weights for policy 1, policy_version 1368128 (0.0008) [2023-12-27 01:20:23,452][105692] Updated weights for policy 0, policy_version 1366161 (0.0006) [2023-12-27 01:20:23,501][105692] Updated weights for policy 0, policy_version 1366171 (0.0009) [2023-12-27 01:20:23,554][105692] Updated weights for policy 0, policy_version 1366181 (0.0009) [2023-12-27 01:20:23,606][105692] Updated weights for policy 0, policy_version 1366191 (0.0009) [2023-12-27 01:20:23,625][105620] Updated weights for policy 1, policy_version 1368138 (0.0008) [2023-12-27 01:20:23,678][105620] Updated weights for policy 1, policy_version 1368148 (0.0008) [2023-12-27 01:20:23,735][105620] Updated weights for policy 1, policy_version 1368158 (0.0009) [2023-12-27 01:20:24,410][105692] Updated weights for policy 0, policy_version 1366201 (0.0009) [2023-12-27 01:20:24,475][105692] Updated weights for policy 0, policy_version 1366211 (0.0008) [2023-12-27 01:20:24,495][105620] Updated weights for policy 1, policy_version 1368168 (0.0007) [2023-12-27 01:20:24,525][105692] Updated weights for policy 0, policy_version 1366221 (0.0007) [2023-12-27 01:20:24,555][105620] Updated weights for policy 1, policy_version 1368178 (0.0008) [2023-12-27 01:20:24,618][105620] Updated weights for policy 1, policy_version 1368188 (0.0008) [2023-12-27 01:20:25,217][105692] Updated weights for policy 0, policy_version 1366231 (0.0008) [2023-12-27 01:20:25,264][105692] Updated weights for policy 0, policy_version 1366241 (0.0007) [2023-12-27 01:20:25,318][105692] Updated weights for policy 0, policy_version 1366251 (0.0005) [2023-12-27 01:20:25,376][105620] Updated weights for policy 1, policy_version 1368198 (0.0009) [2023-12-27 01:20:25,432][105620] Updated weights for policy 1, policy_version 1368208 (0.0009) [2023-12-27 01:20:25,485][105620] Updated weights for policy 1, policy_version 1368219 (0.0010) [2023-12-27 01:20:25,882][105692] Updated weights for policy 0, policy_version 1366261 (0.0005) [2023-12-27 01:20:25,936][105692] Updated weights for policy 0, policy_version 1366271 (0.0008) [2023-12-27 01:20:25,990][105692] Updated weights for policy 0, policy_version 1366281 (0.0007) [2023-12-27 01:20:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 700129280. Throughput: 0: 9652.0, 1: 9744.4. Samples: 700133796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:20:26,063][104569] Avg episode reward: [(0, '8266.404'), (1, '8905.765')] [2023-12-27 01:20:26,317][105620] Updated weights for policy 1, policy_version 1368230 (0.0009) [2023-12-27 01:20:26,368][105620] Updated weights for policy 1, policy_version 1368240 (0.0009) [2023-12-27 01:20:26,422][105620] Updated weights for policy 1, policy_version 1368250 (0.0009) [2023-12-27 01:20:26,693][105692] Updated weights for policy 0, policy_version 1366291 (0.0009) [2023-12-27 01:20:26,741][105692] Updated weights for policy 0, policy_version 1366301 (0.0009) [2023-12-27 01:20:26,788][105692] Updated weights for policy 0, policy_version 1366311 (0.0009) [2023-12-27 01:20:27,153][105620] Updated weights for policy 1, policy_version 1368260 (0.0009) [2023-12-27 01:20:27,207][105620] Updated weights for policy 1, policy_version 1368270 (0.0009) [2023-12-27 01:20:27,256][105620] Updated weights for policy 1, policy_version 1368280 (0.0009) [2023-12-27 01:20:27,518][105692] Updated weights for policy 0, policy_version 1366321 (0.0009) [2023-12-27 01:20:27,569][105692] Updated weights for policy 0, policy_version 1366331 (0.0009) [2023-12-27 01:20:27,620][105692] Updated weights for policy 0, policy_version 1366341 (0.0009) [2023-12-27 01:20:27,671][105692] Updated weights for policy 0, policy_version 1366352 (0.0009) [2023-12-27 01:20:27,932][105620] Updated weights for policy 1, policy_version 1368290 (0.0008) [2023-12-27 01:20:27,987][105620] Updated weights for policy 1, policy_version 1368300 (0.0005) [2023-12-27 01:20:28,045][105620] Updated weights for policy 1, policy_version 1368310 (0.0005) [2023-12-27 01:20:28,097][105620] Updated weights for policy 1, policy_version 1368320 (0.0005) [2023-12-27 01:20:28,255][105692] Updated weights for policy 0, policy_version 1366362 (0.0005) [2023-12-27 01:20:28,300][105692] Updated weights for policy 0, policy_version 1366372 (0.0005) [2023-12-27 01:20:28,365][105692] Updated weights for policy 0, policy_version 1366382 (0.0005) [2023-12-27 01:20:28,657][105620] Updated weights for policy 1, policy_version 1368330 (0.0005) [2023-12-27 01:20:28,730][105620] Updated weights for policy 1, policy_version 1368340 (0.0005) [2023-12-27 01:20:28,797][105620] Updated weights for policy 1, policy_version 1368350 (0.0005) [2023-12-27 01:20:29,007][105692] Updated weights for policy 0, policy_version 1366392 (0.0006) [2023-12-27 01:20:29,068][105692] Updated weights for policy 0, policy_version 1366402 (0.0005) [2023-12-27 01:20:29,125][105692] Updated weights for policy 0, policy_version 1366412 (0.0005) [2023-12-27 01:20:29,487][105620] Updated weights for policy 1, policy_version 1368360 (0.0005) [2023-12-27 01:20:29,545][105620] Updated weights for policy 1, policy_version 1368370 (0.0006) [2023-12-27 01:20:29,612][105620] Updated weights for policy 1, policy_version 1368380 (0.0009) [2023-12-27 01:20:29,691][105692] Updated weights for policy 0, policy_version 1366422 (0.0008) [2023-12-27 01:20:29,748][105692] Updated weights for policy 0, policy_version 1366432 (0.0008) [2023-12-27 01:20:29,804][105692] Updated weights for policy 0, policy_version 1366442 (0.0007) [2023-12-27 01:20:30,247][105620] Updated weights for policy 1, policy_version 1368390 (0.0010) [2023-12-27 01:20:30,308][105620] Updated weights for policy 1, policy_version 1368400 (0.0010) [2023-12-27 01:20:30,362][105620] Updated weights for policy 1, policy_version 1368410 (0.0010) [2023-12-27 01:20:30,555][105692] Updated weights for policy 0, policy_version 1366452 (0.0009) [2023-12-27 01:20:30,607][105692] Updated weights for policy 0, policy_version 1366462 (0.0008) [2023-12-27 01:20:30,659][105692] Updated weights for policy 0, policy_version 1366472 (0.0005) [2023-12-27 01:20:30,953][105620] Updated weights for policy 1, policy_version 1368420 (0.0008) [2023-12-27 01:20:31,007][105620] Updated weights for policy 1, policy_version 1368430 (0.0005) [2023-12-27 01:20:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19438.7). Total num frames: 700227584. Throughput: 0: 9716.4, 1: 9846.8. Samples: 700198268. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:20:31,062][104569] Avg episode reward: [(0, '8184.246'), (1, '8902.767')] [2023-12-27 01:20:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001366480_349872128.pth... [2023-12-27 01:20:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001365360_349585408.pth [2023-12-27 01:20:31,083][105620] Updated weights for policy 1, policy_version 1368440 (0.0006) [2023-12-27 01:20:31,130][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001368448_350363648.pth... [2023-12-27 01:20:31,135][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001367296_350068736.pth [2023-12-27 01:20:31,262][105692] Updated weights for policy 0, policy_version 1366482 (0.0006) [2023-12-27 01:20:31,320][105692] Updated weights for policy 0, policy_version 1366492 (0.0009) [2023-12-27 01:20:31,385][105692] Updated weights for policy 0, policy_version 1366502 (0.0010) [2023-12-27 01:20:31,436][105692] Updated weights for policy 0, policy_version 1366512 (0.0009) [2023-12-27 01:20:31,738][105620] Updated weights for policy 1, policy_version 1368450 (0.0008) [2023-12-27 01:20:31,793][105620] Updated weights for policy 1, policy_version 1368460 (0.0009) [2023-12-27 01:20:31,851][105620] Updated weights for policy 1, policy_version 1368470 (0.0009) [2023-12-27 01:20:31,900][105620] Updated weights for policy 1, policy_version 1368480 (0.0006) [2023-12-27 01:20:32,255][105692] Updated weights for policy 0, policy_version 1366522 (0.0010) [2023-12-27 01:20:32,311][105692] Updated weights for policy 0, policy_version 1366532 (0.0010) [2023-12-27 01:20:32,371][105692] Updated weights for policy 0, policy_version 1366542 (0.0010) [2023-12-27 01:20:32,576][105620] Updated weights for policy 1, policy_version 1368490 (0.0011) [2023-12-27 01:20:32,635][105620] Updated weights for policy 1, policy_version 1368500 (0.0010) [2023-12-27 01:20:32,687][105620] Updated weights for policy 1, policy_version 1368510 (0.0010) [2023-12-27 01:20:33,103][105692] Updated weights for policy 0, policy_version 1366552 (0.0007) [2023-12-27 01:20:33,165][105692] Updated weights for policy 0, policy_version 1366562 (0.0009) [2023-12-27 01:20:33,232][105692] Updated weights for policy 0, policy_version 1366572 (0.0009) [2023-12-27 01:20:33,395][105620] Updated weights for policy 1, policy_version 1368520 (0.0006) [2023-12-27 01:20:33,444][105620] Updated weights for policy 1, policy_version 1368530 (0.0008) [2023-12-27 01:20:33,507][105620] Updated weights for policy 1, policy_version 1368540 (0.0008) [2023-12-27 01:20:33,937][105692] Updated weights for policy 0, policy_version 1366582 (0.0007) [2023-12-27 01:20:33,989][105692] Updated weights for policy 0, policy_version 1366592 (0.0007) [2023-12-27 01:20:34,047][105692] Updated weights for policy 0, policy_version 1366602 (0.0010) [2023-12-27 01:20:34,209][105620] Updated weights for policy 1, policy_version 1368550 (0.0008) [2023-12-27 01:20:34,269][105620] Updated weights for policy 1, policy_version 1368560 (0.0009) [2023-12-27 01:20:34,330][105620] Updated weights for policy 1, policy_version 1368570 (0.0009) [2023-12-27 01:20:34,780][105692] Updated weights for policy 0, policy_version 1366612 (0.0009) [2023-12-27 01:20:34,847][105692] Updated weights for policy 0, policy_version 1366622 (0.0010) [2023-12-27 01:20:34,906][105692] Updated weights for policy 0, policy_version 1366632 (0.0009) [2023-12-27 01:20:35,045][105620] Updated weights for policy 1, policy_version 1368580 (0.0009) [2023-12-27 01:20:35,094][105620] Updated weights for policy 1, policy_version 1368590 (0.0008) [2023-12-27 01:20:35,152][105620] Updated weights for policy 1, policy_version 1368600 (0.0005) [2023-12-27 01:20:35,708][105692] Updated weights for policy 0, policy_version 1366642 (0.0008) [2023-12-27 01:20:35,756][105692] Updated weights for policy 0, policy_version 1366652 (0.0008) [2023-12-27 01:20:35,803][105692] Updated weights for policy 0, policy_version 1366662 (0.0008) [2023-12-27 01:20:35,812][105620] Updated weights for policy 1, policy_version 1368610 (0.0007) [2023-12-27 01:20:35,849][105692] Updated weights for policy 0, policy_version 1366672 (0.0008) [2023-12-27 01:20:35,863][105620] Updated weights for policy 1, policy_version 1368620 (0.0005) [2023-12-27 01:20:35,926][105620] Updated weights for policy 1, policy_version 1368630 (0.0006) [2023-12-27 01:20:35,983][105620] Updated weights for policy 1, policy_version 1368640 (0.0005) [2023-12-27 01:20:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19521.9). Total num frames: 700334080. Throughput: 0: 9658.3, 1: 9932.0. Samples: 700318768. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:20:36,063][104569] Avg episode reward: [(0, '8453.479'), (1, '8805.179')] [2023-12-27 01:20:36,560][105620] Updated weights for policy 1, policy_version 1368650 (0.0010) [2023-12-27 01:20:36,614][105620] Updated weights for policy 1, policy_version 1368660 (0.0010) [2023-12-27 01:20:36,677][105620] Updated weights for policy 1, policy_version 1368670 (0.0010) [2023-12-27 01:20:36,705][105692] Updated weights for policy 0, policy_version 1366682 (0.0006) [2023-12-27 01:20:36,761][105692] Updated weights for policy 0, policy_version 1366692 (0.0008) [2023-12-27 01:20:36,813][105692] Updated weights for policy 0, policy_version 1366702 (0.0008) [2023-12-27 01:20:37,424][105620] Updated weights for policy 1, policy_version 1368680 (0.0010) [2023-12-27 01:20:37,486][105620] Updated weights for policy 1, policy_version 1368690 (0.0011) [2023-12-27 01:20:37,546][105620] Updated weights for policy 1, policy_version 1368700 (0.0011) [2023-12-27 01:20:37,575][105692] Updated weights for policy 0, policy_version 1366712 (0.0009) [2023-12-27 01:20:37,626][105692] Updated weights for policy 0, policy_version 1366722 (0.0008) [2023-12-27 01:20:37,684][105692] Updated weights for policy 0, policy_version 1366732 (0.0008) [2023-12-27 01:20:38,247][105620] Updated weights for policy 1, policy_version 1368710 (0.0009) [2023-12-27 01:20:38,315][105620] Updated weights for policy 1, policy_version 1368720 (0.0009) [2023-12-27 01:20:38,378][105620] Updated weights for policy 1, policy_version 1368730 (0.0011) [2023-12-27 01:20:38,486][105692] Updated weights for policy 0, policy_version 1366742 (0.0009) [2023-12-27 01:20:38,543][105692] Updated weights for policy 0, policy_version 1366753 (0.0010) [2023-12-27 01:20:38,610][105692] Updated weights for policy 0, policy_version 1366763 (0.0010) [2023-12-27 01:20:39,001][105620] Updated weights for policy 1, policy_version 1368740 (0.0007) [2023-12-27 01:20:39,057][105620] Updated weights for policy 1, policy_version 1368750 (0.0009) [2023-12-27 01:20:39,126][105620] Updated weights for policy 1, policy_version 1368760 (0.0005) [2023-12-27 01:20:39,457][105692] Updated weights for policy 0, policy_version 1366773 (0.0010) [2023-12-27 01:20:39,519][105692] Updated weights for policy 0, policy_version 1366783 (0.0010) [2023-12-27 01:20:39,578][105692] Updated weights for policy 0, policy_version 1366793 (0.0008) [2023-12-27 01:20:39,864][105620] Updated weights for policy 1, policy_version 1368770 (0.0006) [2023-12-27 01:20:39,937][105620] Updated weights for policy 1, policy_version 1368780 (0.0006) [2023-12-27 01:20:40,008][105620] Updated weights for policy 1, policy_version 1368790 (0.0008) [2023-12-27 01:20:40,067][105620] Updated weights for policy 1, policy_version 1368800 (0.0008) [2023-12-27 01:20:40,309][105692] Updated weights for policy 0, policy_version 1366803 (0.0009) [2023-12-27 01:20:40,374][105692] Updated weights for policy 0, policy_version 1366813 (0.0011) [2023-12-27 01:20:40,430][105692] Updated weights for policy 0, policy_version 1366823 (0.0011) [2023-12-27 01:20:40,668][105620] Updated weights for policy 1, policy_version 1368810 (0.0005) [2023-12-27 01:20:40,714][105620] Updated weights for policy 1, policy_version 1368820 (0.0005) [2023-12-27 01:20:40,760][105620] Updated weights for policy 1, policy_version 1368830 (0.0005) [2023-12-27 01:20:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19466.5). Total num frames: 700424192. Throughput: 0: 9474.0, 1: 9979.4. Samples: 700433884. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:20:41,063][104569] Avg episode reward: [(0, '8540.449'), (1, '8896.879')] [2023-12-27 01:20:41,179][105692] Updated weights for policy 0, policy_version 1366833 (0.0010) [2023-12-27 01:20:41,236][105692] Updated weights for policy 0, policy_version 1366843 (0.0008) [2023-12-27 01:20:41,299][105692] Updated weights for policy 0, policy_version 1366853 (0.0008) [2023-12-27 01:20:41,356][105692] Updated weights for policy 0, policy_version 1366863 (0.0008) [2023-12-27 01:20:41,453][105620] Updated weights for policy 1, policy_version 1368840 (0.0009) [2023-12-27 01:20:41,511][105620] Updated weights for policy 1, policy_version 1368850 (0.0010) [2023-12-27 01:20:41,570][105620] Updated weights for policy 1, policy_version 1368860 (0.0009) [2023-12-27 01:20:42,161][105692] Updated weights for policy 0, policy_version 1366873 (0.0010) [2023-12-27 01:20:42,227][105692] Updated weights for policy 0, policy_version 1366883 (0.0010) [2023-12-27 01:20:42,287][105692] Updated weights for policy 0, policy_version 1366893 (0.0011) [2023-12-27 01:20:42,305][105620] Updated weights for policy 1, policy_version 1368870 (0.0008) [2023-12-27 01:20:42,374][105620] Updated weights for policy 1, policy_version 1368880 (0.0009) [2023-12-27 01:20:42,435][105620] Updated weights for policy 1, policy_version 1368890 (0.0006) [2023-12-27 01:20:42,969][105692] Updated weights for policy 0, policy_version 1366903 (0.0007) [2023-12-27 01:20:43,016][105692] Updated weights for policy 0, policy_version 1366913 (0.0006) [2023-12-27 01:20:43,066][105692] Updated weights for policy 0, policy_version 1366923 (0.0007) [2023-12-27 01:20:43,186][105620] Updated weights for policy 1, policy_version 1368900 (0.0006) [2023-12-27 01:20:43,251][105620] Updated weights for policy 1, policy_version 1368910 (0.0008) [2023-12-27 01:20:43,319][105620] Updated weights for policy 1, policy_version 1368920 (0.0008) [2023-12-27 01:20:43,724][105692] Updated weights for policy 0, policy_version 1366933 (0.0008) [2023-12-27 01:20:43,786][105692] Updated weights for policy 0, policy_version 1366943 (0.0008) [2023-12-27 01:20:43,844][105692] Updated weights for policy 0, policy_version 1366953 (0.0005) [2023-12-27 01:20:44,090][105620] Updated weights for policy 1, policy_version 1368930 (0.0008) [2023-12-27 01:20:44,160][105620] Updated weights for policy 1, policy_version 1368940 (0.0008) [2023-12-27 01:20:44,214][105620] Updated weights for policy 1, policy_version 1368950 (0.0007) [2023-12-27 01:20:44,274][105620] Updated weights for policy 1, policy_version 1368960 (0.0008) [2023-12-27 01:20:44,493][105692] Updated weights for policy 0, policy_version 1366963 (0.0008) [2023-12-27 01:20:44,541][105692] Updated weights for policy 0, policy_version 1366973 (0.0010) [2023-12-27 01:20:44,596][105692] Updated weights for policy 0, policy_version 1366983 (0.0010) [2023-12-27 01:20:45,105][105620] Updated weights for policy 1, policy_version 1368970 (0.0007) [2023-12-27 01:20:45,171][105620] Updated weights for policy 1, policy_version 1368980 (0.0007) [2023-12-27 01:20:45,228][105692] Updated weights for policy 0, policy_version 1366993 (0.0007) [2023-12-27 01:20:45,239][105620] Updated weights for policy 1, policy_version 1368990 (0.0008) [2023-12-27 01:20:45,290][105692] Updated weights for policy 0, policy_version 1367003 (0.0007) [2023-12-27 01:20:45,349][105692] Updated weights for policy 0, policy_version 1367013 (0.0005) [2023-12-27 01:20:45,397][105692] Updated weights for policy 0, policy_version 1367023 (0.0009) [2023-12-27 01:20:46,041][105620] Updated weights for policy 1, policy_version 1369000 (0.0007) [2023-12-27 01:20:46,047][105692] Updated weights for policy 0, policy_version 1367033 (0.0006) [2023-12-27 01:20:46,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 700514304. Throughput: 0: 9473.1, 1: 9871.1. Samples: 700490512. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:20:46,063][104569] Avg episode reward: [(0, '8539.002'), (1, '8991.121')] [2023-12-27 01:20:46,105][105620] Updated weights for policy 1, policy_version 1369010 (0.0006) [2023-12-27 01:20:46,106][105692] Updated weights for policy 0, policy_version 1367043 (0.0007) [2023-12-27 01:20:46,162][105620] Updated weights for policy 1, policy_version 1369020 (0.0005) [2023-12-27 01:20:46,163][105692] Updated weights for policy 0, policy_version 1367053 (0.0009) [2023-12-27 01:20:46,179][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001367056_350019584.pth... [2023-12-27 01:20:46,183][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001365904_349724672.pth [2023-12-27 01:20:46,185][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001369024_350511104.pth... [2023-12-27 01:20:46,188][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001367872_350216192.pth [2023-12-27 01:20:46,737][105620] Updated weights for policy 1, policy_version 1369030 (0.0008) [2023-12-27 01:20:46,758][105692] Updated weights for policy 0, policy_version 1367063 (0.0008) [2023-12-27 01:20:46,782][105585] KL-divergence is very high: 151.0810 [2023-12-27 01:20:46,798][105620] Updated weights for policy 1, policy_version 1369040 (0.0010) [2023-12-27 01:20:46,799][105585] KL-divergence is very high: 165.0906 [2023-12-27 01:20:46,816][105692] Updated weights for policy 0, policy_version 1367073 (0.0006) [2023-12-27 01:20:46,826][105585] KL-divergence is very high: 180.2709 [2023-12-27 01:20:46,841][105585] KL-divergence is very high: 139.4600 [2023-12-27 01:20:46,856][105620] Updated weights for policy 1, policy_version 1369050 (0.0010) [2023-12-27 01:20:46,866][105692] Updated weights for policy 0, policy_version 1367083 (0.0007) [2023-12-27 01:20:47,516][105620] Updated weights for policy 1, policy_version 1369060 (0.0009) [2023-12-27 01:20:47,542][105692] Updated weights for policy 0, policy_version 1367093 (0.0007) [2023-12-27 01:20:47,586][105620] Updated weights for policy 1, policy_version 1369070 (0.0006) [2023-12-27 01:20:47,596][105692] Updated weights for policy 0, policy_version 1367103 (0.0007) [2023-12-27 01:20:47,649][105620] Updated weights for policy 1, policy_version 1369080 (0.0011) [2023-12-27 01:20:47,655][105692] Updated weights for policy 0, policy_version 1367113 (0.0005) [2023-12-27 01:20:48,178][105620] Updated weights for policy 1, policy_version 1369090 (0.0009) [2023-12-27 01:20:48,238][105620] Updated weights for policy 1, policy_version 1369100 (0.0006) [2023-12-27 01:20:48,297][105620] Updated weights for policy 1, policy_version 1369110 (0.0007) [2023-12-27 01:20:48,365][105620] Updated weights for policy 1, policy_version 1369120 (0.0008) [2023-12-27 01:20:48,522][105692] Updated weights for policy 0, policy_version 1367123 (0.0006) [2023-12-27 01:20:48,583][105692] Updated weights for policy 0, policy_version 1367134 (0.0009) [2023-12-27 01:20:48,641][105692] Updated weights for policy 0, policy_version 1367144 (0.0009) [2023-12-27 01:20:48,901][105620] Updated weights for policy 1, policy_version 1369130 (0.0005) [2023-12-27 01:20:48,970][105620] Updated weights for policy 1, policy_version 1369140 (0.0009) [2023-12-27 01:20:49,024][105620] Updated weights for policy 1, policy_version 1369150 (0.0006) [2023-12-27 01:20:49,516][105692] Updated weights for policy 0, policy_version 1367154 (0.0009) [2023-12-27 01:20:49,571][105692] Updated weights for policy 0, policy_version 1367164 (0.0008) [2023-12-27 01:20:49,628][105692] Updated weights for policy 0, policy_version 1367174 (0.0008) [2023-12-27 01:20:49,678][105692] Updated weights for policy 0, policy_version 1367184 (0.0006) [2023-12-27 01:20:49,686][105620] Updated weights for policy 1, policy_version 1369160 (0.0010) [2023-12-27 01:20:49,740][105620] Updated weights for policy 1, policy_version 1369170 (0.0011) [2023-12-27 01:20:49,797][105620] Updated weights for policy 1, policy_version 1369180 (0.0010) [2023-12-27 01:20:50,514][105620] Updated weights for policy 1, policy_version 1369190 (0.0009) [2023-12-27 01:20:50,516][105692] Updated weights for policy 0, policy_version 1367194 (0.0008) [2023-12-27 01:20:50,570][105620] Updated weights for policy 1, policy_version 1369200 (0.0007) [2023-12-27 01:20:50,579][105692] Updated weights for policy 0, policy_version 1367204 (0.0007) [2023-12-27 01:20:50,630][105620] Updated weights for policy 1, policy_version 1369210 (0.0008) [2023-12-27 01:20:50,647][105692] Updated weights for policy 0, policy_version 1367214 (0.0008) [2023-12-27 01:20:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 700620800. Throughput: 0: 9529.0, 1: 9938.1. Samples: 700610628. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:20:51,063][104569] Avg episode reward: [(0, '8451.684'), (1, '8900.267')] [2023-12-27 01:20:51,309][105620] Updated weights for policy 1, policy_version 1369220 (0.0008) [2023-12-27 01:20:51,375][105620] Updated weights for policy 1, policy_version 1369230 (0.0009) [2023-12-27 01:20:51,438][105692] Updated weights for policy 0, policy_version 1367224 (0.0007) [2023-12-27 01:20:51,442][105620] Updated weights for policy 1, policy_version 1369240 (0.0008) [2023-12-27 01:20:51,493][105692] Updated weights for policy 0, policy_version 1367234 (0.0007) [2023-12-27 01:20:51,541][105692] Updated weights for policy 0, policy_version 1367244 (0.0009) [2023-12-27 01:20:52,237][105620] Updated weights for policy 1, policy_version 1369250 (0.0007) [2023-12-27 01:20:52,260][105692] Updated weights for policy 0, policy_version 1367254 (0.0009) [2023-12-27 01:20:52,301][105620] Updated weights for policy 1, policy_version 1369260 (0.0007) [2023-12-27 01:20:52,315][105692] Updated weights for policy 0, policy_version 1367264 (0.0008) [2023-12-27 01:20:52,357][105620] Updated weights for policy 1, policy_version 1369270 (0.0007) [2023-12-27 01:20:52,376][105692] Updated weights for policy 0, policy_version 1367274 (0.0008) [2023-12-27 01:20:52,420][105620] Updated weights for policy 1, policy_version 1369280 (0.0008) [2023-12-27 01:20:53,065][105692] Updated weights for policy 0, policy_version 1367284 (0.0007) [2023-12-27 01:20:53,095][105620] Updated weights for policy 1, policy_version 1369290 (0.0006) [2023-12-27 01:20:53,130][105692] Updated weights for policy 0, policy_version 1367294 (0.0006) [2023-12-27 01:20:53,152][105620] Updated weights for policy 1, policy_version 1369300 (0.0011) [2023-12-27 01:20:53,185][105692] Updated weights for policy 0, policy_version 1367304 (0.0005) [2023-12-27 01:20:53,208][105620] Updated weights for policy 1, policy_version 1369310 (0.0010) [2023-12-27 01:20:53,717][105692] Updated weights for policy 0, policy_version 1367314 (0.0006) [2023-12-27 01:20:53,773][105692] Updated weights for policy 0, policy_version 1367324 (0.0010) [2023-12-27 01:20:53,782][105620] Updated weights for policy 1, policy_version 1369320 (0.0011) [2023-12-27 01:20:53,828][105692] Updated weights for policy 0, policy_version 1367334 (0.0010) [2023-12-27 01:20:53,838][105620] Updated weights for policy 1, policy_version 1369330 (0.0010) [2023-12-27 01:20:53,879][105692] Updated weights for policy 0, policy_version 1367344 (0.0010) [2023-12-27 01:20:53,892][105620] Updated weights for policy 1, policy_version 1369340 (0.0010) [2023-12-27 01:20:54,595][105620] Updated weights for policy 1, policy_version 1369350 (0.0011) [2023-12-27 01:20:54,634][105692] Updated weights for policy 0, policy_version 1367354 (0.0011) [2023-12-27 01:20:54,645][105620] Updated weights for policy 1, policy_version 1369360 (0.0011) [2023-12-27 01:20:54,684][105692] Updated weights for policy 0, policy_version 1367364 (0.0010) [2023-12-27 01:20:54,704][105620] Updated weights for policy 1, policy_version 1369370 (0.0011) [2023-12-27 01:20:54,743][105692] Updated weights for policy 0, policy_version 1367374 (0.0010) [2023-12-27 01:20:55,340][105620] Updated weights for policy 1, policy_version 1369380 (0.0008) [2023-12-27 01:20:55,389][105620] Updated weights for policy 1, policy_version 1369390 (0.0005) [2023-12-27 01:20:55,440][105620] Updated weights for policy 1, policy_version 1369400 (0.0005) [2023-12-27 01:20:55,531][105692] Updated weights for policy 0, policy_version 1367384 (0.0009) [2023-12-27 01:20:55,590][105692] Updated weights for policy 0, policy_version 1367394 (0.0008) [2023-12-27 01:20:55,641][105692] Updated weights for policy 0, policy_version 1367405 (0.0009) [2023-12-27 01:20:56,012][105620] Updated weights for policy 1, policy_version 1369410 (0.0008) [2023-12-27 01:20:56,062][104569] Fps is (10 sec: 20480.6, 60 sec: 19524.4, 300 sec: 19522.0). Total num frames: 700719104. Throughput: 0: 9569.6, 1: 9963.0. Samples: 700730572. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:20:56,062][104569] Avg episode reward: [(0, '8538.843'), (1, '9081.830')] [2023-12-27 01:20:56,071][105620] Updated weights for policy 1, policy_version 1369420 (0.0006) [2023-12-27 01:20:56,134][105620] Updated weights for policy 1, policy_version 1369430 (0.0006) [2023-12-27 01:20:56,196][105620] Updated weights for policy 1, policy_version 1369440 (0.0006) [2023-12-27 01:20:56,350][105692] Updated weights for policy 0, policy_version 1367415 (0.0007) [2023-12-27 01:20:56,420][105692] Updated weights for policy 0, policy_version 1367425 (0.0005) [2023-12-27 01:20:56,480][105692] Updated weights for policy 0, policy_version 1367435 (0.0007) [2023-12-27 01:20:56,827][105620] Updated weights for policy 1, policy_version 1369450 (0.0008) [2023-12-27 01:20:56,881][105620] Updated weights for policy 1, policy_version 1369460 (0.0007) [2023-12-27 01:20:56,934][105620] Updated weights for policy 1, policy_version 1369470 (0.0005) [2023-12-27 01:20:57,080][105692] Updated weights for policy 0, policy_version 1367445 (0.0008) [2023-12-27 01:20:57,135][105692] Updated weights for policy 0, policy_version 1367455 (0.0010) [2023-12-27 01:20:57,184][105692] Updated weights for policy 0, policy_version 1367466 (0.0008) [2023-12-27 01:20:57,547][105620] Updated weights for policy 1, policy_version 1369480 (0.0005) [2023-12-27 01:20:57,605][105620] Updated weights for policy 1, policy_version 1369490 (0.0006) [2023-12-27 01:20:57,664][105620] Updated weights for policy 1, policy_version 1369500 (0.0009) [2023-12-27 01:20:57,781][105692] Updated weights for policy 0, policy_version 1367477 (0.0007) [2023-12-27 01:20:57,833][105692] Updated weights for policy 0, policy_version 1367487 (0.0005) [2023-12-27 01:20:57,881][105692] Updated weights for policy 0, policy_version 1367497 (0.0006) [2023-12-27 01:20:58,425][105620] Updated weights for policy 1, policy_version 1369510 (0.0008) [2023-12-27 01:20:58,490][105620] Updated weights for policy 1, policy_version 1369520 (0.0008) [2023-12-27 01:20:58,558][105620] Updated weights for policy 1, policy_version 1369530 (0.0008) [2023-12-27 01:20:58,666][105692] Updated weights for policy 0, policy_version 1367507 (0.0009) [2023-12-27 01:20:58,732][105692] Updated weights for policy 0, policy_version 1367517 (0.0008) [2023-12-27 01:20:58,806][105692] Updated weights for policy 0, policy_version 1367527 (0.0007) [2023-12-27 01:20:59,300][105620] Updated weights for policy 1, policy_version 1369540 (0.0006) [2023-12-27 01:20:59,364][105620] Updated weights for policy 1, policy_version 1369550 (0.0009) [2023-12-27 01:20:59,432][105620] Updated weights for policy 1, policy_version 1369560 (0.0008) [2023-12-27 01:20:59,559][105692] Updated weights for policy 0, policy_version 1367537 (0.0008) [2023-12-27 01:20:59,619][105692] Updated weights for policy 0, policy_version 1367547 (0.0006) [2023-12-27 01:20:59,682][105692] Updated weights for policy 0, policy_version 1367557 (0.0005) [2023-12-27 01:20:59,734][105692] Updated weights for policy 0, policy_version 1367567 (0.0009) [2023-12-27 01:21:00,105][105620] Updated weights for policy 1, policy_version 1369570 (0.0007) [2023-12-27 01:21:00,163][105620] Updated weights for policy 1, policy_version 1369580 (0.0008) [2023-12-27 01:21:00,229][105620] Updated weights for policy 1, policy_version 1369590 (0.0008) [2023-12-27 01:21:00,285][105620] Updated weights for policy 1, policy_version 1369600 (0.0008) [2023-12-27 01:21:00,364][105692] Updated weights for policy 0, policy_version 1367577 (0.0009) [2023-12-27 01:21:00,426][105692] Updated weights for policy 0, policy_version 1367587 (0.0009) [2023-12-27 01:21:00,476][105692] Updated weights for policy 0, policy_version 1367597 (0.0008) [2023-12-27 01:21:00,921][105620] Updated weights for policy 1, policy_version 1369610 (0.0009) [2023-12-27 01:21:00,967][105620] Updated weights for policy 1, policy_version 1369620 (0.0009) [2023-12-27 01:21:01,017][105620] Updated weights for policy 1, policy_version 1369630 (0.0009) [2023-12-27 01:21:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 700825600. Throughput: 0: 9655.8, 1: 9984.9. Samples: 700792216. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:21:01,063][104569] Avg episode reward: [(0, '7990.044'), (1, '9175.622')] [2023-12-27 01:21:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001367600_350158848.pth... [2023-12-27 01:21:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001369632_350666752.pth... [2023-12-27 01:21:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001366480_349872128.pth [2023-12-27 01:21:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001368448_350363648.pth [2023-12-27 01:21:01,192][105692] Updated weights for policy 0, policy_version 1367607 (0.0008) [2023-12-27 01:21:01,252][105692] Updated weights for policy 0, policy_version 1367617 (0.0009) [2023-12-27 01:21:01,316][105692] Updated weights for policy 0, policy_version 1367627 (0.0008) [2023-12-27 01:21:01,742][105620] Updated weights for policy 1, policy_version 1369640 (0.0007) [2023-12-27 01:21:01,800][105620] Updated weights for policy 1, policy_version 1369650 (0.0006) [2023-12-27 01:21:01,850][105620] Updated weights for policy 1, policy_version 1369661 (0.0007) [2023-12-27 01:21:02,118][105692] Updated weights for policy 0, policy_version 1367637 (0.0009) [2023-12-27 01:21:02,169][105692] Updated weights for policy 0, policy_version 1367647 (0.0009) [2023-12-27 01:21:02,215][105692] Updated weights for policy 0, policy_version 1367657 (0.0008) [2023-12-27 01:21:02,519][105620] Updated weights for policy 1, policy_version 1369671 (0.0006) [2023-12-27 01:21:02,575][105620] Updated weights for policy 1, policy_version 1369681 (0.0005) [2023-12-27 01:21:02,634][105620] Updated weights for policy 1, policy_version 1369691 (0.0005) [2023-12-27 01:21:03,030][105692] Updated weights for policy 0, policy_version 1367667 (0.0009) [2023-12-27 01:21:03,088][105692] Updated weights for policy 0, policy_version 1367677 (0.0009) [2023-12-27 01:21:03,155][105692] Updated weights for policy 0, policy_version 1367687 (0.0006) [2023-12-27 01:21:03,234][105620] Updated weights for policy 1, policy_version 1369701 (0.0005) [2023-12-27 01:21:03,286][105620] Updated weights for policy 1, policy_version 1369711 (0.0006) [2023-12-27 01:21:03,336][105620] Updated weights for policy 1, policy_version 1369721 (0.0006) [2023-12-27 01:21:03,817][105692] Updated weights for policy 0, policy_version 1367697 (0.0006) [2023-12-27 01:21:03,878][105692] Updated weights for policy 0, policy_version 1367707 (0.0009) [2023-12-27 01:21:03,935][105692] Updated weights for policy 0, policy_version 1367717 (0.0008) [2023-12-27 01:21:03,986][105620] Updated weights for policy 1, policy_version 1369731 (0.0008) [2023-12-27 01:21:04,003][105692] Updated weights for policy 0, policy_version 1367727 (0.0009) [2023-12-27 01:21:04,038][105620] Updated weights for policy 1, policy_version 1369741 (0.0009) [2023-12-27 01:21:04,090][105620] Updated weights for policy 1, policy_version 1369751 (0.0009) [2023-12-27 01:21:04,757][105692] Updated weights for policy 0, policy_version 1367737 (0.0009) [2023-12-27 01:21:04,813][105692] Updated weights for policy 0, policy_version 1367747 (0.0008) [2023-12-27 01:21:04,871][105692] Updated weights for policy 0, policy_version 1367757 (0.0008) [2023-12-27 01:21:04,898][105620] Updated weights for policy 1, policy_version 1369761 (0.0009) [2023-12-27 01:21:04,957][105620] Updated weights for policy 1, policy_version 1369771 (0.0005) [2023-12-27 01:21:05,017][105620] Updated weights for policy 1, policy_version 1369781 (0.0005) [2023-12-27 01:21:05,084][105620] Updated weights for policy 1, policy_version 1369791 (0.0005) [2023-12-27 01:21:05,517][105692] Updated weights for policy 0, policy_version 1367767 (0.0006) [2023-12-27 01:21:05,579][105692] Updated weights for policy 0, policy_version 1367777 (0.0008) [2023-12-27 01:21:05,631][105692] Updated weights for policy 0, policy_version 1367787 (0.0008) [2023-12-27 01:21:05,778][105620] Updated weights for policy 1, policy_version 1369801 (0.0011) [2023-12-27 01:21:05,826][105620] Updated weights for policy 1, policy_version 1369811 (0.0010) [2023-12-27 01:21:05,874][105620] Updated weights for policy 1, policy_version 1369821 (0.0005) [2023-12-27 01:21:06,062][104569] Fps is (10 sec: 20479.1, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 700923904. Throughput: 0: 9656.5, 1: 10035.8. Samples: 700909576. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:21:06,064][104569] Avg episode reward: [(0, '8085.701'), (1, '9265.896')] [2023-12-27 01:21:06,396][105692] Updated weights for policy 0, policy_version 1367797 (0.0008) [2023-12-27 01:21:06,447][105692] Updated weights for policy 0, policy_version 1367807 (0.0009) [2023-12-27 01:21:06,511][105692] Updated weights for policy 0, policy_version 1367817 (0.0010) [2023-12-27 01:21:06,606][105620] Updated weights for policy 1, policy_version 1369831 (0.0008) [2023-12-27 01:21:06,662][105620] Updated weights for policy 1, policy_version 1369841 (0.0009) [2023-12-27 01:21:06,713][105620] Updated weights for policy 1, policy_version 1369851 (0.0009) [2023-12-27 01:21:07,294][105692] Updated weights for policy 0, policy_version 1367827 (0.0009) [2023-12-27 01:21:07,342][105692] Updated weights for policy 0, policy_version 1367837 (0.0009) [2023-12-27 01:21:07,389][105692] Updated weights for policy 0, policy_version 1367847 (0.0009) [2023-12-27 01:21:07,434][105620] Updated weights for policy 1, policy_version 1369861 (0.0009) [2023-12-27 01:21:07,494][105620] Updated weights for policy 1, policy_version 1369871 (0.0009) [2023-12-27 01:21:07,542][105620] Updated weights for policy 1, policy_version 1369881 (0.0009) [2023-12-27 01:21:08,189][105620] Updated weights for policy 1, policy_version 1369891 (0.0007) [2023-12-27 01:21:08,243][105692] Updated weights for policy 0, policy_version 1367857 (0.0009) [2023-12-27 01:21:08,249][105620] Updated weights for policy 1, policy_version 1369901 (0.0008) [2023-12-27 01:21:08,309][105692] Updated weights for policy 0, policy_version 1367867 (0.0007) [2023-12-27 01:21:08,315][105620] Updated weights for policy 1, policy_version 1369911 (0.0007) [2023-12-27 01:21:08,368][105692] Updated weights for policy 0, policy_version 1367877 (0.0009) [2023-12-27 01:21:08,430][105692] Updated weights for policy 0, policy_version 1367887 (0.0010) [2023-12-27 01:21:08,950][105620] Updated weights for policy 1, policy_version 1369921 (0.0008) [2023-12-27 01:21:09,001][105620] Updated weights for policy 1, policy_version 1369931 (0.0010) [2023-12-27 01:21:09,059][105620] Updated weights for policy 1, policy_version 1369941 (0.0010) [2023-12-27 01:21:09,114][105620] Updated weights for policy 1, policy_version 1369951 (0.0010) [2023-12-27 01:21:09,173][105692] Updated weights for policy 0, policy_version 1367897 (0.0006) [2023-12-27 01:21:09,228][105692] Updated weights for policy 0, policy_version 1367907 (0.0007) [2023-12-27 01:21:09,292][105692] Updated weights for policy 0, policy_version 1367917 (0.0008) [2023-12-27 01:21:09,862][105620] Updated weights for policy 1, policy_version 1369961 (0.0007) [2023-12-27 01:21:09,946][105620] Updated weights for policy 1, policy_version 1369971 (0.0007) [2023-12-27 01:21:10,009][105620] Updated weights for policy 1, policy_version 1369981 (0.0008) [2023-12-27 01:21:10,068][105692] Updated weights for policy 0, policy_version 1367927 (0.0009) [2023-12-27 01:21:10,127][105692] Updated weights for policy 0, policy_version 1367937 (0.0008) [2023-12-27 01:21:10,187][105692] Updated weights for policy 0, policy_version 1367947 (0.0011) [2023-12-27 01:21:10,630][105620] Updated weights for policy 1, policy_version 1369991 (0.0008) [2023-12-27 01:21:10,699][105620] Updated weights for policy 1, policy_version 1370001 (0.0009) [2023-12-27 01:21:10,759][105620] Updated weights for policy 1, policy_version 1370011 (0.0009) [2023-12-27 01:21:10,892][105692] Updated weights for policy 0, policy_version 1367957 (0.0008) [2023-12-27 01:21:10,953][105692] Updated weights for policy 0, policy_version 1367967 (0.0009) [2023-12-27 01:21:11,022][105692] Updated weights for policy 0, policy_version 1367977 (0.0010) [2023-12-27 01:21:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 701014016. Throughput: 0: 9655.6, 1: 10149.3. Samples: 701025016. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:21:11,063][104569] Avg episode reward: [(0, '7662.787'), (1, '9176.114')] [2023-12-27 01:21:11,456][105620] Updated weights for policy 1, policy_version 1370021 (0.0008) [2023-12-27 01:21:11,516][105620] Updated weights for policy 1, policy_version 1370031 (0.0006) [2023-12-27 01:21:11,579][105620] Updated weights for policy 1, policy_version 1370041 (0.0006) [2023-12-27 01:21:11,782][105692] Updated weights for policy 0, policy_version 1367987 (0.0010) [2023-12-27 01:21:11,841][105692] Updated weights for policy 0, policy_version 1367997 (0.0009) [2023-12-27 01:21:11,905][105692] Updated weights for policy 0, policy_version 1368007 (0.0008) [2023-12-27 01:21:12,317][105620] Updated weights for policy 1, policy_version 1370051 (0.0011) [2023-12-27 01:21:12,384][105620] Updated weights for policy 1, policy_version 1370061 (0.0010) [2023-12-27 01:21:12,441][105620] Updated weights for policy 1, policy_version 1370071 (0.0010) [2023-12-27 01:21:12,620][105692] Updated weights for policy 0, policy_version 1368017 (0.0008) [2023-12-27 01:21:12,668][105692] Updated weights for policy 0, policy_version 1368027 (0.0008) [2023-12-27 01:21:12,721][105692] Updated weights for policy 0, policy_version 1368037 (0.0008) [2023-12-27 01:21:12,780][105692] Updated weights for policy 0, policy_version 1368047 (0.0008) [2023-12-27 01:21:13,114][105620] Updated weights for policy 1, policy_version 1370081 (0.0007) [2023-12-27 01:21:13,172][105620] Updated weights for policy 1, policy_version 1370091 (0.0010) [2023-12-27 01:21:13,227][105620] Updated weights for policy 1, policy_version 1370101 (0.0010) [2023-12-27 01:21:13,291][105620] Updated weights for policy 1, policy_version 1370111 (0.0010) [2023-12-27 01:21:13,542][105692] Updated weights for policy 0, policy_version 1368057 (0.0009) [2023-12-27 01:21:13,588][105692] Updated weights for policy 0, policy_version 1368067 (0.0008) [2023-12-27 01:21:13,637][105692] Updated weights for policy 0, policy_version 1368077 (0.0008) [2023-12-27 01:21:14,025][105620] Updated weights for policy 1, policy_version 1370121 (0.0009) [2023-12-27 01:21:14,075][105620] Updated weights for policy 1, policy_version 1370131 (0.0005) [2023-12-27 01:21:14,121][105620] Updated weights for policy 1, policy_version 1370141 (0.0009) [2023-12-27 01:21:14,440][105692] Updated weights for policy 0, policy_version 1368087 (0.0009) [2023-12-27 01:21:14,505][105692] Updated weights for policy 0, policy_version 1368097 (0.0008) [2023-12-27 01:21:14,564][105692] Updated weights for policy 0, policy_version 1368107 (0.0008) [2023-12-27 01:21:14,857][105620] Updated weights for policy 1, policy_version 1370151 (0.0007) [2023-12-27 01:21:14,923][105620] Updated weights for policy 1, policy_version 1370161 (0.0006) [2023-12-27 01:21:14,992][105620] Updated weights for policy 1, policy_version 1370171 (0.0009) [2023-12-27 01:21:15,365][105692] Updated weights for policy 0, policy_version 1368117 (0.0008) [2023-12-27 01:21:15,423][105692] Updated weights for policy 0, policy_version 1368127 (0.0010) [2023-12-27 01:21:15,491][105692] Updated weights for policy 0, policy_version 1368137 (0.0010) [2023-12-27 01:21:15,592][105620] Updated weights for policy 1, policy_version 1370181 (0.0011) [2023-12-27 01:21:15,644][105620] Updated weights for policy 1, policy_version 1370191 (0.0010) [2023-12-27 01:21:15,698][105620] Updated weights for policy 1, policy_version 1370201 (0.0010) [2023-12-27 01:21:16,062][104569] Fps is (10 sec: 18842.3, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 701112320. Throughput: 0: 9569.3, 1: 10091.5. Samples: 701083004. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:21:16,062][104569] Avg episode reward: [(0, '7733.406'), (1, '9083.545')] [2023-12-27 01:21:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001368144_350298112.pth... [2023-12-27 01:21:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001370208_350814208.pth... [2023-12-27 01:21:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001369024_350511104.pth [2023-12-27 01:21:16,096][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001367056_350019584.pth [2023-12-27 01:21:16,246][105692] Updated weights for policy 0, policy_version 1368147 (0.0008) [2023-12-27 01:21:16,304][105692] Updated weights for policy 0, policy_version 1368157 (0.0008) [2023-12-27 01:21:16,356][105692] Updated weights for policy 0, policy_version 1368167 (0.0008) [2023-12-27 01:21:16,450][105620] Updated weights for policy 1, policy_version 1370211 (0.0010) [2023-12-27 01:21:16,500][105620] Updated weights for policy 1, policy_version 1370221 (0.0010) [2023-12-27 01:21:16,555][105620] Updated weights for policy 1, policy_version 1370231 (0.0010) [2023-12-27 01:21:17,170][105692] Updated weights for policy 0, policy_version 1368177 (0.0008) [2023-12-27 01:21:17,218][105620] Updated weights for policy 1, policy_version 1370241 (0.0010) [2023-12-27 01:21:17,233][105692] Updated weights for policy 0, policy_version 1368187 (0.0005) [2023-12-27 01:21:17,269][105620] Updated weights for policy 1, policy_version 1370251 (0.0010) [2023-12-27 01:21:17,301][105692] Updated weights for policy 0, policy_version 1368197 (0.0005) [2023-12-27 01:21:17,321][105620] Updated weights for policy 1, policy_version 1370261 (0.0010) [2023-12-27 01:21:17,372][105692] Updated weights for policy 0, policy_version 1368207 (0.0005) [2023-12-27 01:21:17,376][105620] Updated weights for policy 1, policy_version 1370271 (0.0010) [2023-12-27 01:21:17,924][105692] Updated weights for policy 0, policy_version 1368217 (0.0008) [2023-12-27 01:21:17,986][105692] Updated weights for policy 0, policy_version 1368227 (0.0008) [2023-12-27 01:21:18,045][105692] Updated weights for policy 0, policy_version 1368237 (0.0008) [2023-12-27 01:21:18,054][105620] Updated weights for policy 1, policy_version 1370281 (0.0010) [2023-12-27 01:21:18,103][105620] Updated weights for policy 1, policy_version 1370291 (0.0010) [2023-12-27 01:21:18,158][105620] Updated weights for policy 1, policy_version 1370301 (0.0011) [2023-12-27 01:21:18,850][105692] Updated weights for policy 0, policy_version 1368247 (0.0008) [2023-12-27 01:21:18,872][105620] Updated weights for policy 1, policy_version 1370311 (0.0010) [2023-12-27 01:21:18,910][105692] Updated weights for policy 0, policy_version 1368257 (0.0006) [2023-12-27 01:21:18,924][105620] Updated weights for policy 1, policy_version 1370321 (0.0010) [2023-12-27 01:21:18,967][105692] Updated weights for policy 0, policy_version 1368267 (0.0005) [2023-12-27 01:21:18,979][105620] Updated weights for policy 1, policy_version 1370331 (0.0010) [2023-12-27 01:21:19,642][105620] Updated weights for policy 1, policy_version 1370341 (0.0008) [2023-12-27 01:21:19,689][105620] Updated weights for policy 1, policy_version 1370351 (0.0005) [2023-12-27 01:21:19,755][105620] Updated weights for policy 1, policy_version 1370361 (0.0005) [2023-12-27 01:21:19,770][105692] Updated weights for policy 0, policy_version 1368277 (0.0009) [2023-12-27 01:21:19,835][105692] Updated weights for policy 0, policy_version 1368287 (0.0009) [2023-12-27 01:21:19,896][105692] Updated weights for policy 0, policy_version 1368297 (0.0008) [2023-12-27 01:21:20,466][105620] Updated weights for policy 1, policy_version 1370371 (0.0006) [2023-12-27 01:21:20,520][105620] Updated weights for policy 1, policy_version 1370381 (0.0005) [2023-12-27 01:21:20,583][105620] Updated weights for policy 1, policy_version 1370391 (0.0010) [2023-12-27 01:21:20,641][105692] Updated weights for policy 0, policy_version 1368307 (0.0008) [2023-12-27 01:21:20,702][105692] Updated weights for policy 0, policy_version 1368317 (0.0006) [2023-12-27 01:21:20,763][105692] Updated weights for policy 0, policy_version 1368327 (0.0008) [2023-12-27 01:21:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 701210624. Throughput: 0: 9475.4, 1: 10099.0. Samples: 701199616. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:21:21,063][104569] Avg episode reward: [(0, '8267.813'), (1, '9262.869')] [2023-12-27 01:21:21,345][105620] Updated weights for policy 1, policy_version 1370401 (0.0009) [2023-12-27 01:21:21,418][105620] Updated weights for policy 1, policy_version 1370411 (0.0008) [2023-12-27 01:21:21,486][105620] Updated weights for policy 1, policy_version 1370421 (0.0007) [2023-12-27 01:21:21,547][105620] Updated weights for policy 1, policy_version 1370431 (0.0008) [2023-12-27 01:21:21,565][105692] Updated weights for policy 0, policy_version 1368337 (0.0008) [2023-12-27 01:21:21,628][105692] Updated weights for policy 0, policy_version 1368347 (0.0007) [2023-12-27 01:21:21,692][105692] Updated weights for policy 0, policy_version 1368357 (0.0008) [2023-12-27 01:21:21,758][105692] Updated weights for policy 0, policy_version 1368367 (0.0008) [2023-12-27 01:21:22,314][105620] Updated weights for policy 1, policy_version 1370441 (0.0009) [2023-12-27 01:21:22,381][105620] Updated weights for policy 1, policy_version 1370451 (0.0010) [2023-12-27 01:21:22,447][105620] Updated weights for policy 1, policy_version 1370461 (0.0009) [2023-12-27 01:21:22,473][105692] Updated weights for policy 0, policy_version 1368377 (0.0008) [2023-12-27 01:21:22,528][105692] Updated weights for policy 0, policy_version 1368387 (0.0009) [2023-12-27 01:21:22,582][105692] Updated weights for policy 0, policy_version 1368397 (0.0009) [2023-12-27 01:21:23,171][105620] Updated weights for policy 1, policy_version 1370471 (0.0007) [2023-12-27 01:21:23,230][105620] Updated weights for policy 1, policy_version 1370481 (0.0008) [2023-12-27 01:21:23,289][105620] Updated weights for policy 1, policy_version 1370491 (0.0008) [2023-12-27 01:21:23,355][105692] Updated weights for policy 0, policy_version 1368407 (0.0010) [2023-12-27 01:21:23,406][105692] Updated weights for policy 0, policy_version 1368417 (0.0010) [2023-12-27 01:21:23,451][105692] Updated weights for policy 0, policy_version 1368427 (0.0010) [2023-12-27 01:21:24,080][105692] Updated weights for policy 0, policy_version 1368437 (0.0007) [2023-12-27 01:21:24,094][105620] Updated weights for policy 1, policy_version 1370501 (0.0007) [2023-12-27 01:21:24,148][105692] Updated weights for policy 0, policy_version 1368447 (0.0006) [2023-12-27 01:21:24,156][105620] Updated weights for policy 1, policy_version 1370511 (0.0007) [2023-12-27 01:21:24,196][105692] Updated weights for policy 0, policy_version 1368457 (0.0008) [2023-12-27 01:21:24,214][105620] Updated weights for policy 1, policy_version 1370521 (0.0006) [2023-12-27 01:21:24,863][105620] Updated weights for policy 1, policy_version 1370531 (0.0007) [2023-12-27 01:21:24,896][105692] Updated weights for policy 0, policy_version 1368467 (0.0009) [2023-12-27 01:21:24,919][105620] Updated weights for policy 1, policy_version 1370541 (0.0005) [2023-12-27 01:21:24,956][105692] Updated weights for policy 0, policy_version 1368477 (0.0005) [2023-12-27 01:21:24,975][105620] Updated weights for policy 1, policy_version 1370551 (0.0005) [2023-12-27 01:21:25,021][105692] Updated weights for policy 0, policy_version 1368487 (0.0005) [2023-12-27 01:21:25,643][105692] Updated weights for policy 0, policy_version 1368497 (0.0006) [2023-12-27 01:21:25,706][105620] Updated weights for policy 1, policy_version 1370561 (0.0007) [2023-12-27 01:21:25,712][105692] Updated weights for policy 0, policy_version 1368507 (0.0010) [2023-12-27 01:21:25,769][105620] Updated weights for policy 1, policy_version 1370571 (0.0006) [2023-12-27 01:21:25,771][105692] Updated weights for policy 0, policy_version 1368517 (0.0010) [2023-12-27 01:21:25,824][105692] Updated weights for policy 0, policy_version 1368527 (0.0008) [2023-12-27 01:21:25,827][105620] Updated weights for policy 1, policy_version 1370581 (0.0006) [2023-12-27 01:21:25,894][105620] Updated weights for policy 1, policy_version 1370591 (0.0010) [2023-12-27 01:21:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 701308928. Throughput: 0: 9595.1, 1: 9956.1. Samples: 701313688. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:21:26,062][104569] Avg episode reward: [(0, '8356.326'), (1, '9262.308')] [2023-12-27 01:21:26,399][105692] Updated weights for policy 0, policy_version 1368537 (0.0010) [2023-12-27 01:21:26,461][105692] Updated weights for policy 0, policy_version 1368547 (0.0010) [2023-12-27 01:21:26,511][105692] Updated weights for policy 0, policy_version 1368557 (0.0010) [2023-12-27 01:21:26,656][105620] Updated weights for policy 1, policy_version 1370601 (0.0006) [2023-12-27 01:21:26,708][105620] Updated weights for policy 1, policy_version 1370611 (0.0005) [2023-12-27 01:21:26,758][105620] Updated weights for policy 1, policy_version 1370621 (0.0005) [2023-12-27 01:21:27,251][105692] Updated weights for policy 0, policy_version 1368567 (0.0007) [2023-12-27 01:21:27,300][105692] Updated weights for policy 0, policy_version 1368577 (0.0006) [2023-12-27 01:21:27,356][105692] Updated weights for policy 0, policy_version 1368587 (0.0009) [2023-12-27 01:21:27,404][105620] Updated weights for policy 1, policy_version 1370631 (0.0007) [2023-12-27 01:21:27,451][105620] Updated weights for policy 1, policy_version 1370641 (0.0008) [2023-12-27 01:21:27,498][105620] Updated weights for policy 1, policy_version 1370651 (0.0009) [2023-12-27 01:21:28,049][105692] Updated weights for policy 0, policy_version 1368597 (0.0009) [2023-12-27 01:21:28,100][105692] Updated weights for policy 0, policy_version 1368607 (0.0010) [2023-12-27 01:21:28,165][105692] Updated weights for policy 0, policy_version 1368617 (0.0010) [2023-12-27 01:21:28,308][105620] Updated weights for policy 1, policy_version 1370661 (0.0008) [2023-12-27 01:21:28,372][105620] Updated weights for policy 1, policy_version 1370671 (0.0006) [2023-12-27 01:21:28,438][105620] Updated weights for policy 1, policy_version 1370681 (0.0006) [2023-12-27 01:21:28,871][105692] Updated weights for policy 0, policy_version 1368627 (0.0009) [2023-12-27 01:21:28,945][105692] Updated weights for policy 0, policy_version 1368637 (0.0010) [2023-12-27 01:21:29,012][105692] Updated weights for policy 0, policy_version 1368647 (0.0010) [2023-12-27 01:21:29,028][105620] Updated weights for policy 1, policy_version 1370691 (0.0006) [2023-12-27 01:21:29,082][105620] Updated weights for policy 1, policy_version 1370701 (0.0005) [2023-12-27 01:21:29,140][105620] Updated weights for policy 1, policy_version 1370711 (0.0006) [2023-12-27 01:21:29,645][105692] Updated weights for policy 0, policy_version 1368657 (0.0010) [2023-12-27 01:21:29,709][105692] Updated weights for policy 0, policy_version 1368667 (0.0007) [2023-12-27 01:21:29,763][105692] Updated weights for policy 0, policy_version 1368677 (0.0008) [2023-12-27 01:21:29,818][105620] Updated weights for policy 1, policy_version 1370721 (0.0006) [2023-12-27 01:21:29,825][105692] Updated weights for policy 0, policy_version 1368687 (0.0008) [2023-12-27 01:21:29,880][105620] Updated weights for policy 1, policy_version 1370731 (0.0008) [2023-12-27 01:21:29,943][105620] Updated weights for policy 1, policy_version 1370741 (0.0007) [2023-12-27 01:21:29,994][105620] Updated weights for policy 1, policy_version 1370751 (0.0005) [2023-12-27 01:21:30,600][105692] Updated weights for policy 0, policy_version 1368697 (0.0007) [2023-12-27 01:21:30,664][105692] Updated weights for policy 0, policy_version 1368707 (0.0006) [2023-12-27 01:21:30,705][105620] Updated weights for policy 1, policy_version 1370761 (0.0010) [2023-12-27 01:21:30,721][105692] Updated weights for policy 0, policy_version 1368717 (0.0005) [2023-12-27 01:21:30,762][105620] Updated weights for policy 1, policy_version 1370771 (0.0010) [2023-12-27 01:21:30,809][105620] Updated weights for policy 1, policy_version 1370781 (0.0010) [2023-12-27 01:21:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 701407232. Throughput: 0: 9623.5, 1: 10009.6. Samples: 701374000. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:21:31,063][104569] Avg episode reward: [(0, '8266.459'), (1, '9083.734')] [2023-12-27 01:21:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001368720_350445568.pth... [2023-12-27 01:21:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001370784_350961664.pth... [2023-12-27 01:21:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001367600_350158848.pth [2023-12-27 01:21:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001369632_350666752.pth [2023-12-27 01:21:31,472][105692] Updated weights for policy 0, policy_version 1368727 (0.0009) [2023-12-27 01:21:31,513][105620] Updated weights for policy 1, policy_version 1370791 (0.0010) [2023-12-27 01:21:31,524][105692] Updated weights for policy 0, policy_version 1368737 (0.0010) [2023-12-27 01:21:31,571][105620] Updated weights for policy 1, policy_version 1370801 (0.0010) [2023-12-27 01:21:31,584][105692] Updated weights for policy 0, policy_version 1368747 (0.0010) [2023-12-27 01:21:31,627][105620] Updated weights for policy 1, policy_version 1370811 (0.0010) [2023-12-27 01:21:32,321][105692] Updated weights for policy 0, policy_version 1368757 (0.0009) [2023-12-27 01:21:32,380][105620] Updated weights for policy 1, policy_version 1370821 (0.0010) [2023-12-27 01:21:32,385][105692] Updated weights for policy 0, policy_version 1368767 (0.0011) [2023-12-27 01:21:32,435][105620] Updated weights for policy 1, policy_version 1370831 (0.0010) [2023-12-27 01:21:32,440][105692] Updated weights for policy 0, policy_version 1368777 (0.0008) [2023-12-27 01:21:32,487][105620] Updated weights for policy 1, policy_version 1370841 (0.0010) [2023-12-27 01:21:33,173][105692] Updated weights for policy 0, policy_version 1368787 (0.0009) [2023-12-27 01:21:33,235][105692] Updated weights for policy 0, policy_version 1368797 (0.0010) [2023-12-27 01:21:33,235][105620] Updated weights for policy 1, policy_version 1370851 (0.0010) [2023-12-27 01:21:33,293][105620] Updated weights for policy 1, policy_version 1370861 (0.0010) [2023-12-27 01:21:33,293][105692] Updated weights for policy 0, policy_version 1368807 (0.0010) [2023-12-27 01:21:33,344][105620] Updated weights for policy 1, policy_version 1370871 (0.0010) [2023-12-27 01:21:33,956][105620] Updated weights for policy 1, policy_version 1370881 (0.0010) [2023-12-27 01:21:34,014][105620] Updated weights for policy 1, policy_version 1370891 (0.0008) [2023-12-27 01:21:34,037][105692] Updated weights for policy 0, policy_version 1368817 (0.0010) [2023-12-27 01:21:34,069][105620] Updated weights for policy 1, policy_version 1370901 (0.0010) [2023-12-27 01:21:34,095][105692] Updated weights for policy 0, policy_version 1368827 (0.0010) [2023-12-27 01:21:34,123][105620] Updated weights for policy 1, policy_version 1370911 (0.0009) [2023-12-27 01:21:34,155][105692] Updated weights for policy 0, policy_version 1368837 (0.0011) [2023-12-27 01:21:34,208][105692] Updated weights for policy 0, policy_version 1368847 (0.0010) [2023-12-27 01:21:34,776][105620] Updated weights for policy 1, policy_version 1370921 (0.0010) [2023-12-27 01:21:34,831][105620] Updated weights for policy 1, policy_version 1370931 (0.0010) [2023-12-27 01:21:34,879][105620] Updated weights for policy 1, policy_version 1370941 (0.0010) [2023-12-27 01:21:34,969][105692] Updated weights for policy 0, policy_version 1368857 (0.0010) [2023-12-27 01:21:35,021][105692] Updated weights for policy 0, policy_version 1368867 (0.0010) [2023-12-27 01:21:35,069][105692] Updated weights for policy 0, policy_version 1368877 (0.0010) [2023-12-27 01:21:35,605][105620] Updated weights for policy 1, policy_version 1370951 (0.0009) [2023-12-27 01:21:35,658][105620] Updated weights for policy 1, policy_version 1370961 (0.0010) [2023-12-27 01:21:35,712][105620] Updated weights for policy 1, policy_version 1370971 (0.0010) [2023-12-27 01:21:35,827][105692] Updated weights for policy 0, policy_version 1368887 (0.0010) [2023-12-27 01:21:35,834][105585] KL-divergence is very high: 177.2336 [2023-12-27 01:21:35,877][105585] KL-divergence is very high: 326.3239 [2023-12-27 01:21:35,882][105692] Updated weights for policy 0, policy_version 1368897 (0.0010) [2023-12-27 01:21:35,914][105585] KL-divergence is very high: 362.9907 [2023-12-27 01:21:35,929][105692] Updated weights for policy 0, policy_version 1368907 (0.0010) [2023-12-27 01:21:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 701505536. Throughput: 0: 9578.3, 1: 9991.6. Samples: 701491276. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:21:36,063][104569] Avg episode reward: [(0, '8450.190'), (1, '8905.485')] [2023-12-27 01:21:36,397][105620] Updated weights for policy 1, policy_version 1370981 (0.0009) [2023-12-27 01:21:36,470][105620] Updated weights for policy 1, policy_version 1370991 (0.0008) [2023-12-27 01:21:36,536][105620] Updated weights for policy 1, policy_version 1371001 (0.0010) [2023-12-27 01:21:36,607][105692] Updated weights for policy 0, policy_version 1368917 (0.0010) [2023-12-27 01:21:36,671][105692] Updated weights for policy 0, policy_version 1368927 (0.0011) [2023-12-27 01:21:36,730][105692] Updated weights for policy 0, policy_version 1368937 (0.0011) [2023-12-27 01:21:37,148][105620] Updated weights for policy 1, policy_version 1371011 (0.0009) [2023-12-27 01:21:37,214][105620] Updated weights for policy 1, policy_version 1371021 (0.0006) [2023-12-27 01:21:37,265][105620] Updated weights for policy 1, policy_version 1371031 (0.0005) [2023-12-27 01:21:37,420][105692] Updated weights for policy 0, policy_version 1368947 (0.0009) [2023-12-27 01:21:37,483][105692] Updated weights for policy 0, policy_version 1368957 (0.0007) [2023-12-27 01:21:37,538][105692] Updated weights for policy 0, policy_version 1368967 (0.0010) [2023-12-27 01:21:37,799][105620] Updated weights for policy 1, policy_version 1371041 (0.0005) [2023-12-27 01:21:37,861][105620] Updated weights for policy 1, policy_version 1371051 (0.0005) [2023-12-27 01:21:37,928][105620] Updated weights for policy 1, policy_version 1371061 (0.0007) [2023-12-27 01:21:37,988][105620] Updated weights for policy 1, policy_version 1371071 (0.0011) [2023-12-27 01:21:38,268][105692] Updated weights for policy 0, policy_version 1368977 (0.0010) [2023-12-27 01:21:38,332][105692] Updated weights for policy 0, policy_version 1368987 (0.0011) [2023-12-27 01:21:38,400][105692] Updated weights for policy 0, policy_version 1368997 (0.0012) [2023-12-27 01:21:38,456][105692] Updated weights for policy 0, policy_version 1369007 (0.0006) [2023-12-27 01:21:38,590][105620] Updated weights for policy 1, policy_version 1371081 (0.0008) [2023-12-27 01:21:38,653][105620] Updated weights for policy 1, policy_version 1371091 (0.0007) [2023-12-27 01:21:38,717][105620] Updated weights for policy 1, policy_version 1371101 (0.0009) [2023-12-27 01:21:39,039][105692] Updated weights for policy 0, policy_version 1369017 (0.0008) [2023-12-27 01:21:39,088][105692] Updated weights for policy 0, policy_version 1369027 (0.0008) [2023-12-27 01:21:39,133][105692] Updated weights for policy 0, policy_version 1369037 (0.0008) [2023-12-27 01:21:39,471][105620] Updated weights for policy 1, policy_version 1371111 (0.0010) [2023-12-27 01:21:39,533][105620] Updated weights for policy 1, policy_version 1371121 (0.0011) [2023-12-27 01:21:39,593][105620] Updated weights for policy 1, policy_version 1371131 (0.0011) [2023-12-27 01:21:39,915][105692] Updated weights for policy 0, policy_version 1369047 (0.0008) [2023-12-27 01:21:39,974][105692] Updated weights for policy 0, policy_version 1369057 (0.0009) [2023-12-27 01:21:40,027][105692] Updated weights for policy 0, policy_version 1369067 (0.0007) [2023-12-27 01:21:40,312][105620] Updated weights for policy 1, policy_version 1371141 (0.0011) [2023-12-27 01:21:40,371][105620] Updated weights for policy 1, policy_version 1371151 (0.0010) [2023-12-27 01:21:40,429][105620] Updated weights for policy 1, policy_version 1371161 (0.0009) [2023-12-27 01:21:40,711][105692] Updated weights for policy 0, policy_version 1369077 (0.0008) [2023-12-27 01:21:40,767][105692] Updated weights for policy 0, policy_version 1369087 (0.0008) [2023-12-27 01:21:40,831][105692] Updated weights for policy 0, policy_version 1369097 (0.0008) [2023-12-27 01:21:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 701603840. Throughput: 0: 9620.8, 1: 9969.8. Samples: 701612148. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:21:41,063][104569] Avg episode reward: [(0, '8533.137'), (1, '8814.132')] [2023-12-27 01:21:41,207][105620] Updated weights for policy 1, policy_version 1371171 (0.0010) [2023-12-27 01:21:41,266][105620] Updated weights for policy 1, policy_version 1371181 (0.0007) [2023-12-27 01:21:41,332][105620] Updated weights for policy 1, policy_version 1371191 (0.0007) [2023-12-27 01:21:41,760][105692] Updated weights for policy 0, policy_version 1369107 (0.0008) [2023-12-27 01:21:41,827][105692] Updated weights for policy 0, policy_version 1369117 (0.0008) [2023-12-27 01:21:41,890][105692] Updated weights for policy 0, policy_version 1369127 (0.0008) [2023-12-27 01:21:42,014][105620] Updated weights for policy 1, policy_version 1371201 (0.0011) [2023-12-27 01:21:42,073][105620] Updated weights for policy 1, policy_version 1371211 (0.0006) [2023-12-27 01:21:42,138][105620] Updated weights for policy 1, policy_version 1371221 (0.0009) [2023-12-27 01:21:42,204][105620] Updated weights for policy 1, policy_version 1371231 (0.0005) [2023-12-27 01:21:42,629][105692] Updated weights for policy 0, policy_version 1369137 (0.0008) [2023-12-27 01:21:42,692][105692] Updated weights for policy 0, policy_version 1369147 (0.0008) [2023-12-27 01:21:42,756][105692] Updated weights for policy 0, policy_version 1369157 (0.0008) [2023-12-27 01:21:42,811][105692] Updated weights for policy 0, policy_version 1369167 (0.0006) [2023-12-27 01:21:42,907][105620] Updated weights for policy 1, policy_version 1371241 (0.0010) [2023-12-27 01:21:42,955][105620] Updated weights for policy 1, policy_version 1371251 (0.0010) [2023-12-27 01:21:43,000][105620] Updated weights for policy 1, policy_version 1371261 (0.0010) [2023-12-27 01:21:43,357][105692] Updated weights for policy 0, policy_version 1369177 (0.0009) [2023-12-27 01:21:43,412][105692] Updated weights for policy 0, policy_version 1369187 (0.0010) [2023-12-27 01:21:43,469][105692] Updated weights for policy 0, policy_version 1369197 (0.0009) [2023-12-27 01:21:43,614][105620] Updated weights for policy 1, policy_version 1371271 (0.0011) [2023-12-27 01:21:43,669][105620] Updated weights for policy 1, policy_version 1371281 (0.0010) [2023-12-27 01:21:43,720][105620] Updated weights for policy 1, policy_version 1371291 (0.0010) [2023-12-27 01:21:44,248][105692] Updated weights for policy 0, policy_version 1369207 (0.0010) [2023-12-27 01:21:44,309][105692] Updated weights for policy 0, policy_version 1369217 (0.0010) [2023-12-27 01:21:44,360][105692] Updated weights for policy 0, policy_version 1369228 (0.0009) [2023-12-27 01:21:44,370][105620] Updated weights for policy 1, policy_version 1371301 (0.0009) [2023-12-27 01:21:44,429][105620] Updated weights for policy 1, policy_version 1371311 (0.0010) [2023-12-27 01:21:44,490][105620] Updated weights for policy 1, policy_version 1371321 (0.0010) [2023-12-27 01:21:45,170][105692] Updated weights for policy 0, policy_version 1369238 (0.0007) [2023-12-27 01:21:45,239][105692] Updated weights for policy 0, policy_version 1369248 (0.0008) [2023-12-27 01:21:45,251][105620] Updated weights for policy 1, policy_version 1371331 (0.0010) [2023-12-27 01:21:45,298][105692] Updated weights for policy 0, policy_version 1369258 (0.0006) [2023-12-27 01:21:45,304][105620] Updated weights for policy 1, policy_version 1371341 (0.0010) [2023-12-27 01:21:45,364][105620] Updated weights for policy 1, policy_version 1371351 (0.0011) [2023-12-27 01:21:46,054][105692] Updated weights for policy 0, policy_version 1369268 (0.0007) [2023-12-27 01:21:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.9, 300 sec: 19494.2). Total num frames: 701693952. Throughput: 0: 9531.0, 1: 9988.1. Samples: 701670576. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:21:46,063][104569] Avg episode reward: [(0, '8449.265'), (1, '8906.544')] [2023-12-27 01:21:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001371360_351109120.pth... [2023-12-27 01:21:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001370208_350814208.pth [2023-12-27 01:21:46,072][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_001371360_351109120.pth [2023-12-27 01:21:46,111][105620] Updated weights for policy 1, policy_version 1371361 (0.0011) [2023-12-27 01:21:46,120][105692] Updated weights for policy 0, policy_version 1369278 (0.0008) [2023-12-27 01:21:46,172][105620] Updated weights for policy 1, policy_version 1371371 (0.0010) [2023-12-27 01:21:46,178][105692] Updated weights for policy 0, policy_version 1369288 (0.0005) [2023-12-27 01:21:46,214][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001369296_350593024.pth... [2023-12-27 01:21:46,217][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001368144_350298112.pth [2023-12-27 01:21:46,218][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_001369296_350593024.pth [2023-12-27 01:21:46,227][105620] Updated weights for policy 1, policy_version 1371381 (0.0010) [2023-12-27 01:21:46,295][105620] Updated weights for policy 1, policy_version 1371391 (0.0010) [2023-12-27 01:21:46,930][105692] Updated weights for policy 0, policy_version 1369298 (0.0009) [2023-12-27 01:21:46,992][105692] Updated weights for policy 0, policy_version 1369308 (0.0008) [2023-12-27 01:21:47,028][105620] Updated weights for policy 1, policy_version 1371401 (0.0010) [2023-12-27 01:21:47,040][105692] Updated weights for policy 0, policy_version 1369318 (0.0007) [2023-12-27 01:21:47,086][105620] Updated weights for policy 1, policy_version 1371411 (0.0010) [2023-12-27 01:21:47,089][105692] Updated weights for policy 0, policy_version 1369328 (0.0007) [2023-12-27 01:21:47,148][105620] Updated weights for policy 1, policy_version 1371421 (0.0010) [2023-12-27 01:21:47,792][105692] Updated weights for policy 0, policy_version 1369338 (0.0005) [2023-12-27 01:21:47,850][105692] Updated weights for policy 0, policy_version 1369348 (0.0005) [2023-12-27 01:21:47,872][105620] Updated weights for policy 1, policy_version 1371431 (0.0010) [2023-12-27 01:21:47,910][105692] Updated weights for policy 0, policy_version 1369358 (0.0005) [2023-12-27 01:21:47,934][105620] Updated weights for policy 1, policy_version 1371441 (0.0011) [2023-12-27 01:21:47,995][105620] Updated weights for policy 1, policy_version 1371451 (0.0010) [2023-12-27 01:21:48,490][105692] Updated weights for policy 0, policy_version 1369368 (0.0010) [2023-12-27 01:21:48,539][105692] Updated weights for policy 0, policy_version 1369378 (0.0011) [2023-12-27 01:21:48,598][105692] Updated weights for policy 0, policy_version 1369388 (0.0005) [2023-12-27 01:21:48,717][105620] Updated weights for policy 1, policy_version 1371461 (0.0010) [2023-12-27 01:21:48,786][105620] Updated weights for policy 1, policy_version 1371471 (0.0011) [2023-12-27 01:21:48,849][105620] Updated weights for policy 1, policy_version 1371481 (0.0011) [2023-12-27 01:21:49,299][105692] Updated weights for policy 0, policy_version 1369398 (0.0008) [2023-12-27 01:21:49,359][105692] Updated weights for policy 0, policy_version 1369408 (0.0009) [2023-12-27 01:21:49,414][105692] Updated weights for policy 0, policy_version 1369418 (0.0008) [2023-12-27 01:21:49,580][105620] Updated weights for policy 1, policy_version 1371491 (0.0009) [2023-12-27 01:21:49,639][105620] Updated weights for policy 1, policy_version 1371501 (0.0005) [2023-12-27 01:21:49,698][105620] Updated weights for policy 1, policy_version 1371511 (0.0005) [2023-12-27 01:21:50,067][105692] Updated weights for policy 0, policy_version 1369428 (0.0009) [2023-12-27 01:21:50,128][105692] Updated weights for policy 0, policy_version 1369438 (0.0009) [2023-12-27 01:21:50,180][105692] Updated weights for policy 0, policy_version 1369448 (0.0009) [2023-12-27 01:21:50,347][105620] Updated weights for policy 1, policy_version 1371521 (0.0006) [2023-12-27 01:21:50,399][105620] Updated weights for policy 1, policy_version 1371531 (0.0006) [2023-12-27 01:21:50,458][105620] Updated weights for policy 1, policy_version 1371541 (0.0005) [2023-12-27 01:21:50,512][105620] Updated weights for policy 1, policy_version 1371551 (0.0006) [2023-12-27 01:21:50,919][105692] Updated weights for policy 0, policy_version 1369458 (0.0009) [2023-12-27 01:21:50,986][105692] Updated weights for policy 0, policy_version 1369468 (0.0009) [2023-12-27 01:21:51,050][105692] Updated weights for policy 0, policy_version 1369478 (0.0009) [2023-12-27 01:21:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 701792256. Throughput: 0: 9560.6, 1: 9928.8. Samples: 701786592. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:21:51,062][104569] Avg episode reward: [(0, '8722.084'), (1, '8718.649')] [2023-12-27 01:21:51,115][105692] Updated weights for policy 0, policy_version 1369488 (0.0008) [2023-12-27 01:21:51,262][105620] Updated weights for policy 1, policy_version 1371561 (0.0009) [2023-12-27 01:21:51,322][105620] Updated weights for policy 1, policy_version 1371571 (0.0009) [2023-12-27 01:21:51,389][105620] Updated weights for policy 1, policy_version 1371581 (0.0009) [2023-12-27 01:21:51,835][105692] Updated weights for policy 0, policy_version 1369498 (0.0009) [2023-12-27 01:21:51,894][105692] Updated weights for policy 0, policy_version 1369508 (0.0009) [2023-12-27 01:21:51,949][105692] Updated weights for policy 0, policy_version 1369518 (0.0009) [2023-12-27 01:21:52,168][105620] Updated weights for policy 1, policy_version 1371591 (0.0006) [2023-12-27 01:21:52,217][105620] Updated weights for policy 1, policy_version 1371601 (0.0005) [2023-12-27 01:21:52,275][105620] Updated weights for policy 1, policy_version 1371611 (0.0008) [2023-12-27 01:21:52,768][105692] Updated weights for policy 0, policy_version 1369528 (0.0008) [2023-12-27 01:21:52,828][105692] Updated weights for policy 0, policy_version 1369538 (0.0008) [2023-12-27 01:21:52,888][105692] Updated weights for policy 0, policy_version 1369548 (0.0008) [2023-12-27 01:21:52,962][105620] Updated weights for policy 1, policy_version 1371621 (0.0008) [2023-12-27 01:21:53,012][105620] Updated weights for policy 1, policy_version 1371631 (0.0009) [2023-12-27 01:21:53,073][105620] Updated weights for policy 1, policy_version 1371641 (0.0008) [2023-12-27 01:21:53,604][105692] Updated weights for policy 0, policy_version 1369558 (0.0009) [2023-12-27 01:21:53,678][105692] Updated weights for policy 0, policy_version 1369568 (0.0010) [2023-12-27 01:21:53,745][105692] Updated weights for policy 0, policy_version 1369578 (0.0010) [2023-12-27 01:21:53,782][105620] Updated weights for policy 1, policy_version 1371651 (0.0008) [2023-12-27 01:21:53,837][105620] Updated weights for policy 1, policy_version 1371661 (0.0010) [2023-12-27 01:21:53,891][105620] Updated weights for policy 1, policy_version 1371671 (0.0008) [2023-12-27 01:21:54,541][105692] Updated weights for policy 0, policy_version 1369588 (0.0009) [2023-12-27 01:21:54,579][105620] Updated weights for policy 1, policy_version 1371681 (0.0008) [2023-12-27 01:21:54,597][105692] Updated weights for policy 0, policy_version 1369598 (0.0009) [2023-12-27 01:21:54,639][105620] Updated weights for policy 1, policy_version 1371691 (0.0007) [2023-12-27 01:21:54,649][105692] Updated weights for policy 0, policy_version 1369608 (0.0006) [2023-12-27 01:21:54,702][105620] Updated weights for policy 1, policy_version 1371701 (0.0009) [2023-12-27 01:21:54,762][105620] Updated weights for policy 1, policy_version 1371711 (0.0008) [2023-12-27 01:21:55,387][105692] Updated weights for policy 0, policy_version 1369618 (0.0008) [2023-12-27 01:21:55,439][105692] Updated weights for policy 0, policy_version 1369628 (0.0008) [2023-12-27 01:21:55,490][105620] Updated weights for policy 1, policy_version 1371721 (0.0007) [2023-12-27 01:21:55,495][105692] Updated weights for policy 0, policy_version 1369638 (0.0008) [2023-12-27 01:21:55,550][105620] Updated weights for policy 1, policy_version 1371731 (0.0006) [2023-12-27 01:21:55,552][105692] Updated weights for policy 0, policy_version 1369648 (0.0007) [2023-12-27 01:21:55,608][105620] Updated weights for policy 1, policy_version 1371741 (0.0008) [2023-12-27 01:21:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 701890560. Throughput: 0: 9575.8, 1: 9878.4. Samples: 701900456. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:21:56,062][104569] Avg episode reward: [(0, '8902.473'), (1, '8804.441')] [2023-12-27 01:21:56,297][105692] Updated weights for policy 0, policy_version 1369658 (0.0008) [2023-12-27 01:21:56,348][105692] Updated weights for policy 0, policy_version 1369668 (0.0007) [2023-12-27 01:21:56,357][105620] Updated weights for policy 1, policy_version 1371751 (0.0008) [2023-12-27 01:21:56,396][105692] Updated weights for policy 0, policy_version 1369678 (0.0006) [2023-12-27 01:21:56,418][105620] Updated weights for policy 1, policy_version 1371761 (0.0008) [2023-12-27 01:21:56,473][105620] Updated weights for policy 1, policy_version 1371771 (0.0009) [2023-12-27 01:21:57,162][105692] Updated weights for policy 0, policy_version 1369688 (0.0006) [2023-12-27 01:21:57,224][105692] Updated weights for policy 0, policy_version 1369698 (0.0008) [2023-12-27 01:21:57,229][105620] Updated weights for policy 1, policy_version 1371781 (0.0007) [2023-12-27 01:21:57,279][105692] Updated weights for policy 0, policy_version 1369708 (0.0010) [2023-12-27 01:21:57,289][105620] Updated weights for policy 1, policy_version 1371791 (0.0007) [2023-12-27 01:21:57,307][105586] KL-divergence is very high: 147.3684 [2023-12-27 01:21:57,352][105620] Updated weights for policy 1, policy_version 1371801 (0.0008) [2023-12-27 01:21:57,359][105586] KL-divergence is very high: 129.5340 [2023-12-27 01:21:57,846][105692] Updated weights for policy 0, policy_version 1369718 (0.0006) [2023-12-27 01:21:57,904][105692] Updated weights for policy 0, policy_version 1369728 (0.0005) [2023-12-27 01:21:57,915][105620] Updated weights for policy 1, policy_version 1371811 (0.0007) [2023-12-27 01:21:57,949][105692] Updated weights for policy 0, policy_version 1369738 (0.0005) [2023-12-27 01:21:57,960][105620] Updated weights for policy 1, policy_version 1371821 (0.0005) [2023-12-27 01:21:58,007][105620] Updated weights for policy 1, policy_version 1371831 (0.0005) [2023-12-27 01:21:58,635][105692] Updated weights for policy 0, policy_version 1369748 (0.0009) [2023-12-27 01:21:58,705][105692] Updated weights for policy 0, policy_version 1369758 (0.0011) [2023-12-27 01:21:58,753][105620] Updated weights for policy 1, policy_version 1371841 (0.0005) [2023-12-27 01:21:58,768][105692] Updated weights for policy 0, policy_version 1369768 (0.0009) [2023-12-27 01:21:58,825][105620] Updated weights for policy 1, policy_version 1371851 (0.0008) [2023-12-27 01:21:58,898][105620] Updated weights for policy 1, policy_version 1371861 (0.0008) [2023-12-27 01:21:58,956][105620] Updated weights for policy 1, policy_version 1371871 (0.0008) [2023-12-27 01:21:59,567][105692] Updated weights for policy 0, policy_version 1369778 (0.0009) [2023-12-27 01:21:59,628][105692] Updated weights for policy 0, policy_version 1369788 (0.0008) [2023-12-27 01:21:59,689][105692] Updated weights for policy 0, policy_version 1369798 (0.0009) [2023-12-27 01:21:59,729][105620] Updated weights for policy 1, policy_version 1371881 (0.0007) [2023-12-27 01:21:59,747][105692] Updated weights for policy 0, policy_version 1369808 (0.0009) [2023-12-27 01:21:59,788][105620] Updated weights for policy 1, policy_version 1371891 (0.0007) [2023-12-27 01:21:59,853][105620] Updated weights for policy 1, policy_version 1371901 (0.0009) [2023-12-27 01:22:00,478][105692] Updated weights for policy 0, policy_version 1369818 (0.0008) [2023-12-27 01:22:00,524][105692] Updated weights for policy 0, policy_version 1369828 (0.0008) [2023-12-27 01:22:00,584][105692] Updated weights for policy 0, policy_version 1369838 (0.0008) [2023-12-27 01:22:00,630][105620] Updated weights for policy 1, policy_version 1371911 (0.0008) [2023-12-27 01:22:00,687][105620] Updated weights for policy 1, policy_version 1371921 (0.0009) [2023-12-27 01:22:00,748][105620] Updated weights for policy 1, policy_version 1371931 (0.0008) [2023-12-27 01:22:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 701988864. Throughput: 0: 9622.6, 1: 9889.0. Samples: 701961032. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:22:01,062][104569] Avg episode reward: [(0, '8273.992'), (1, '9083.371')] [2023-12-27 01:22:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001369840_350732288.pth... [2023-12-27 01:22:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001371936_351256576.pth... [2023-12-27 01:22:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001368720_350445568.pth [2023-12-27 01:22:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001370784_350961664.pth [2023-12-27 01:22:01,320][105692] Updated weights for policy 0, policy_version 1369848 (0.0006) [2023-12-27 01:22:01,386][105692] Updated weights for policy 0, policy_version 1369858 (0.0008) [2023-12-27 01:22:01,440][105692] Updated weights for policy 0, policy_version 1369868 (0.0008) [2023-12-27 01:22:01,509][105620] Updated weights for policy 1, policy_version 1371941 (0.0008) [2023-12-27 01:22:01,564][105620] Updated weights for policy 1, policy_version 1371951 (0.0009) [2023-12-27 01:22:01,632][105620] Updated weights for policy 1, policy_version 1371961 (0.0009) [2023-12-27 01:22:02,078][105692] Updated weights for policy 0, policy_version 1369878 (0.0008) [2023-12-27 01:22:02,132][105692] Updated weights for policy 0, policy_version 1369888 (0.0005) [2023-12-27 01:22:02,194][105692] Updated weights for policy 0, policy_version 1369898 (0.0007) [2023-12-27 01:22:02,444][105620] Updated weights for policy 1, policy_version 1371971 (0.0009) [2023-12-27 01:22:02,498][105620] Updated weights for policy 1, policy_version 1371981 (0.0010) [2023-12-27 01:22:02,556][105620] Updated weights for policy 1, policy_version 1371991 (0.0010) [2023-12-27 01:22:02,782][105692] Updated weights for policy 0, policy_version 1369908 (0.0007) [2023-12-27 01:22:02,836][105692] Updated weights for policy 0, policy_version 1369918 (0.0005) [2023-12-27 01:22:02,906][105692] Updated weights for policy 0, policy_version 1369928 (0.0006) [2023-12-27 01:22:03,400][105620] Updated weights for policy 1, policy_version 1372001 (0.0010) [2023-12-27 01:22:03,452][105620] Updated weights for policy 1, policy_version 1372011 (0.0009) [2023-12-27 01:22:03,503][105620] Updated weights for policy 1, policy_version 1372021 (0.0009) [2023-12-27 01:22:03,513][105692] Updated weights for policy 0, policy_version 1369938 (0.0008) [2023-12-27 01:22:03,552][105620] Updated weights for policy 1, policy_version 1372032 (0.0009) [2023-12-27 01:22:03,566][105692] Updated weights for policy 0, policy_version 1369948 (0.0005) [2023-12-27 01:22:03,611][105692] Updated weights for policy 0, policy_version 1369958 (0.0005) [2023-12-27 01:22:03,670][105692] Updated weights for policy 0, policy_version 1369968 (0.0005) [2023-12-27 01:22:04,289][105692] Updated weights for policy 0, policy_version 1369978 (0.0009) [2023-12-27 01:22:04,345][105692] Updated weights for policy 0, policy_version 1369988 (0.0009) [2023-12-27 01:22:04,395][105692] Updated weights for policy 0, policy_version 1369998 (0.0008) [2023-12-27 01:22:04,444][105620] Updated weights for policy 1, policy_version 1372042 (0.0008) [2023-12-27 01:22:04,507][105620] Updated weights for policy 1, policy_version 1372052 (0.0006) [2023-12-27 01:22:04,572][105620] Updated weights for policy 1, policy_version 1372062 (0.0006) [2023-12-27 01:22:05,200][105692] Updated weights for policy 0, policy_version 1370008 (0.0008) [2023-12-27 01:22:05,225][105620] Updated weights for policy 1, policy_version 1372072 (0.0006) [2023-12-27 01:22:05,248][105692] Updated weights for policy 0, policy_version 1370018 (0.0007) [2023-12-27 01:22:05,270][105620] Updated weights for policy 1, policy_version 1372082 (0.0006) [2023-12-27 01:22:05,300][105692] Updated weights for policy 0, policy_version 1370028 (0.0006) [2023-12-27 01:22:05,330][105620] Updated weights for policy 1, policy_version 1372092 (0.0008) [2023-12-27 01:22:06,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.3, 300 sec: 19494.2). Total num frames: 702078976. Throughput: 0: 9745.6, 1: 9708.2. Samples: 702075036. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:22:06,063][104569] Avg episode reward: [(0, '7903.219'), (1, '9083.769')] [2023-12-27 01:22:06,075][105692] Updated weights for policy 0, policy_version 1370038 (0.0007) [2023-12-27 01:22:06,080][105620] Updated weights for policy 1, policy_version 1372102 (0.0010) [2023-12-27 01:22:06,136][105692] Updated weights for policy 0, policy_version 1370048 (0.0009) [2023-12-27 01:22:06,146][105620] Updated weights for policy 1, policy_version 1372112 (0.0009) [2023-12-27 01:22:06,203][105692] Updated weights for policy 0, policy_version 1370058 (0.0009) [2023-12-27 01:22:06,209][105620] Updated weights for policy 1, policy_version 1372122 (0.0009) [2023-12-27 01:22:06,924][105692] Updated weights for policy 0, policy_version 1370068 (0.0008) [2023-12-27 01:22:06,981][105620] Updated weights for policy 1, policy_version 1372132 (0.0008) [2023-12-27 01:22:06,983][105692] Updated weights for policy 0, policy_version 1370078 (0.0008) [2023-12-27 01:22:07,037][105692] Updated weights for policy 0, policy_version 1370088 (0.0006) [2023-12-27 01:22:07,038][105620] Updated weights for policy 1, policy_version 1372142 (0.0008) [2023-12-27 01:22:07,103][105620] Updated weights for policy 1, policy_version 1372152 (0.0009) [2023-12-27 01:22:07,770][105692] Updated weights for policy 0, policy_version 1370098 (0.0008) [2023-12-27 01:22:07,820][105620] Updated weights for policy 1, policy_version 1372162 (0.0008) [2023-12-27 01:22:07,825][105692] Updated weights for policy 0, policy_version 1370108 (0.0009) [2023-12-27 01:22:07,882][105692] Updated weights for policy 0, policy_version 1370118 (0.0007) [2023-12-27 01:22:07,884][105620] Updated weights for policy 1, policy_version 1372172 (0.0006) [2023-12-27 01:22:07,938][105620] Updated weights for policy 1, policy_version 1372182 (0.0007) [2023-12-27 01:22:07,944][105692] Updated weights for policy 0, policy_version 1370128 (0.0007) [2023-12-27 01:22:07,996][105620] Updated weights for policy 1, policy_version 1372192 (0.0008) [2023-12-27 01:22:08,718][105692] Updated weights for policy 0, policy_version 1370138 (0.0008) [2023-12-27 01:22:08,757][105620] Updated weights for policy 1, policy_version 1372202 (0.0008) [2023-12-27 01:22:08,775][105692] Updated weights for policy 0, policy_version 1370148 (0.0007) [2023-12-27 01:22:08,810][105620] Updated weights for policy 1, policy_version 1372212 (0.0006) [2023-12-27 01:22:08,828][105692] Updated weights for policy 0, policy_version 1370158 (0.0007) [2023-12-27 01:22:08,865][105620] Updated weights for policy 1, policy_version 1372222 (0.0007) [2023-12-27 01:22:09,593][105692] Updated weights for policy 0, policy_version 1370168 (0.0009) [2023-12-27 01:22:09,644][105620] Updated weights for policy 1, policy_version 1372232 (0.0007) [2023-12-27 01:22:09,654][105692] Updated weights for policy 0, policy_version 1370178 (0.0007) [2023-12-27 01:22:09,706][105620] Updated weights for policy 1, policy_version 1372242 (0.0007) [2023-12-27 01:22:09,708][105692] Updated weights for policy 0, policy_version 1370188 (0.0006) [2023-12-27 01:22:09,771][105620] Updated weights for policy 1, policy_version 1372252 (0.0008) [2023-12-27 01:22:10,509][105692] Updated weights for policy 0, policy_version 1370198 (0.0009) [2023-12-27 01:22:10,512][105620] Updated weights for policy 1, policy_version 1372262 (0.0008) [2023-12-27 01:22:10,566][105692] Updated weights for policy 0, policy_version 1370208 (0.0010) [2023-12-27 01:22:10,576][105620] Updated weights for policy 1, policy_version 1372272 (0.0007) [2023-12-27 01:22:10,627][105692] Updated weights for policy 0, policy_version 1370218 (0.0010) [2023-12-27 01:22:10,631][105620] Updated weights for policy 1, policy_version 1372282 (0.0007) [2023-12-27 01:22:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 702177280. Throughput: 0: 9670.2, 1: 9714.6. Samples: 702186004. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:22:11,063][104569] Avg episode reward: [(0, '8445.725'), (1, '9077.477')] [2023-12-27 01:22:11,412][105692] Updated weights for policy 0, policy_version 1370228 (0.0011) [2023-12-27 01:22:11,423][105620] Updated weights for policy 1, policy_version 1372292 (0.0009) [2023-12-27 01:22:11,469][105692] Updated weights for policy 0, policy_version 1370238 (0.0011) [2023-12-27 01:22:11,483][105620] Updated weights for policy 1, policy_version 1372302 (0.0007) [2023-12-27 01:22:11,525][105692] Updated weights for policy 0, policy_version 1370248 (0.0011) [2023-12-27 01:22:11,540][105620] Updated weights for policy 1, policy_version 1372312 (0.0006) [2023-12-27 01:22:12,276][105692] Updated weights for policy 0, policy_version 1370258 (0.0010) [2023-12-27 01:22:12,307][105620] Updated weights for policy 1, policy_version 1372322 (0.0008) [2023-12-27 01:22:12,334][105692] Updated weights for policy 0, policy_version 1370268 (0.0008) [2023-12-27 01:22:12,378][105620] Updated weights for policy 1, policy_version 1372332 (0.0009) [2023-12-27 01:22:12,404][105692] Updated weights for policy 0, policy_version 1370278 (0.0008) [2023-12-27 01:22:12,431][105620] Updated weights for policy 1, policy_version 1372342 (0.0006) [2023-12-27 01:22:12,464][105692] Updated weights for policy 0, policy_version 1370288 (0.0008) [2023-12-27 01:22:12,492][105620] Updated weights for policy 1, policy_version 1372352 (0.0009) [2023-12-27 01:22:13,168][105692] Updated weights for policy 0, policy_version 1370298 (0.0006) [2023-12-27 01:22:13,226][105692] Updated weights for policy 0, policy_version 1370308 (0.0005) [2023-12-27 01:22:13,253][105620] Updated weights for policy 1, policy_version 1372362 (0.0010) [2023-12-27 01:22:13,283][105692] Updated weights for policy 0, policy_version 1370318 (0.0008) [2023-12-27 01:22:13,318][105620] Updated weights for policy 1, policy_version 1372372 (0.0009) [2023-12-27 01:22:13,376][105620] Updated weights for policy 1, policy_version 1372382 (0.0008) [2023-12-27 01:22:13,920][105692] Updated weights for policy 0, policy_version 1370328 (0.0005) [2023-12-27 01:22:13,982][105692] Updated weights for policy 0, policy_version 1370338 (0.0005) [2023-12-27 01:22:14,037][105692] Updated weights for policy 0, policy_version 1370348 (0.0005) [2023-12-27 01:22:14,048][105620] Updated weights for policy 1, policy_version 1372392 (0.0005) [2023-12-27 01:22:14,102][105620] Updated weights for policy 1, policy_version 1372402 (0.0005) [2023-12-27 01:22:14,164][105620] Updated weights for policy 1, policy_version 1372412 (0.0005) [2023-12-27 01:22:14,661][105692] Updated weights for policy 0, policy_version 1370358 (0.0005) [2023-12-27 01:22:14,721][105692] Updated weights for policy 0, policy_version 1370368 (0.0005) [2023-12-27 01:22:14,783][105692] Updated weights for policy 0, policy_version 1370378 (0.0006) [2023-12-27 01:22:14,875][105620] Updated weights for policy 1, policy_version 1372422 (0.0008) [2023-12-27 01:22:14,938][105620] Updated weights for policy 1, policy_version 1372432 (0.0010) [2023-12-27 01:22:15,001][105620] Updated weights for policy 1, policy_version 1372442 (0.0009) [2023-12-27 01:22:15,382][105692] Updated weights for policy 0, policy_version 1370388 (0.0008) [2023-12-27 01:22:15,437][105692] Updated weights for policy 0, policy_version 1370398 (0.0008) [2023-12-27 01:22:15,501][105692] Updated weights for policy 0, policy_version 1370408 (0.0008) [2023-12-27 01:22:15,755][105620] Updated weights for policy 1, policy_version 1372452 (0.0009) [2023-12-27 01:22:15,812][105620] Updated weights for policy 1, policy_version 1372462 (0.0009) [2023-12-27 01:22:15,862][105620] Updated weights for policy 1, policy_version 1372472 (0.0008) [2023-12-27 01:22:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 702275584. Throughput: 0: 9634.9, 1: 9662.4. Samples: 702242376. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:22:16,062][104569] Avg episode reward: [(0, '8542.019'), (1, '8895.667')] [2023-12-27 01:22:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001370416_350879744.pth... [2023-12-27 01:22:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001372480_351395840.pth... [2023-12-27 01:22:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001369296_350593024.pth [2023-12-27 01:22:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001371360_351109120.pth [2023-12-27 01:22:16,213][105692] Updated weights for policy 0, policy_version 1370418 (0.0009) [2023-12-27 01:22:16,271][105692] Updated weights for policy 0, policy_version 1370429 (0.0010) [2023-12-27 01:22:16,322][105692] Updated weights for policy 0, policy_version 1370439 (0.0009) [2023-12-27 01:22:16,584][105620] Updated weights for policy 1, policy_version 1372482 (0.0009) [2023-12-27 01:22:16,643][105620] Updated weights for policy 1, policy_version 1372492 (0.0009) [2023-12-27 01:22:16,695][105620] Updated weights for policy 1, policy_version 1372502 (0.0009) [2023-12-27 01:22:16,748][105620] Updated weights for policy 1, policy_version 1372512 (0.0010) [2023-12-27 01:22:16,994][105692] Updated weights for policy 0, policy_version 1370451 (0.0010) [2023-12-27 01:22:17,043][105692] Updated weights for policy 0, policy_version 1370461 (0.0009) [2023-12-27 01:22:17,093][105692] Updated weights for policy 0, policy_version 1370471 (0.0008) [2023-12-27 01:22:17,555][105620] Updated weights for policy 1, policy_version 1372522 (0.0009) [2023-12-27 01:22:17,617][105620] Updated weights for policy 1, policy_version 1372532 (0.0010) [2023-12-27 01:22:17,675][105620] Updated weights for policy 1, policy_version 1372542 (0.0010) [2023-12-27 01:22:17,809][105692] Updated weights for policy 0, policy_version 1370481 (0.0007) [2023-12-27 01:22:17,872][105692] Updated weights for policy 0, policy_version 1370491 (0.0009) [2023-12-27 01:22:17,935][105692] Updated weights for policy 0, policy_version 1370501 (0.0011) [2023-12-27 01:22:18,001][105692] Updated weights for policy 0, policy_version 1370511 (0.0011) [2023-12-27 01:22:18,236][105620] Updated weights for policy 1, policy_version 1372552 (0.0007) [2023-12-27 01:22:18,286][105620] Updated weights for policy 1, policy_version 1372562 (0.0007) [2023-12-27 01:22:18,337][105620] Updated weights for policy 1, policy_version 1372572 (0.0007) [2023-12-27 01:22:18,713][105692] Updated weights for policy 0, policy_version 1370521 (0.0011) [2023-12-27 01:22:18,758][105585] KL-divergence is very high: 115.0607 [2023-12-27 01:22:18,769][105692] Updated weights for policy 0, policy_version 1370531 (0.0011) [2023-12-27 01:22:18,799][105585] KL-divergence is very high: 226.8164 [2023-12-27 01:22:18,822][105692] Updated weights for policy 0, policy_version 1370541 (0.0011) [2023-12-27 01:22:18,944][105620] Updated weights for policy 1, policy_version 1372582 (0.0008) [2023-12-27 01:22:19,004][105620] Updated weights for policy 1, policy_version 1372592 (0.0007) [2023-12-27 01:22:19,061][105620] Updated weights for policy 1, policy_version 1372602 (0.0008) [2023-12-27 01:22:19,610][105692] Updated weights for policy 0, policy_version 1370551 (0.0011) [2023-12-27 01:22:19,672][105692] Updated weights for policy 0, policy_version 1370561 (0.0010) [2023-12-27 01:22:19,736][105692] Updated weights for policy 0, policy_version 1370571 (0.0010) [2023-12-27 01:22:19,832][105620] Updated weights for policy 1, policy_version 1372612 (0.0008) [2023-12-27 01:22:19,896][105620] Updated weights for policy 1, policy_version 1372622 (0.0008) [2023-12-27 01:22:19,955][105620] Updated weights for policy 1, policy_version 1372632 (0.0008) [2023-12-27 01:22:20,491][105692] Updated weights for policy 0, policy_version 1370581 (0.0011) [2023-12-27 01:22:20,554][105692] Updated weights for policy 0, policy_version 1370591 (0.0011) [2023-12-27 01:22:20,619][105692] Updated weights for policy 0, policy_version 1370601 (0.0010) [2023-12-27 01:22:20,689][105620] Updated weights for policy 1, policy_version 1372642 (0.0008) [2023-12-27 01:22:20,753][105620] Updated weights for policy 1, policy_version 1372652 (0.0011) [2023-12-27 01:22:20,814][105620] Updated weights for policy 1, policy_version 1372662 (0.0011) [2023-12-27 01:22:20,878][105620] Updated weights for policy 1, policy_version 1372672 (0.0011) [2023-12-27 01:22:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 702373888. Throughput: 0: 9724.4, 1: 9636.0. Samples: 702362492. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:22:21,063][104569] Avg episode reward: [(0, '7810.728'), (1, '8716.561')] [2023-12-27 01:22:21,385][105692] Updated weights for policy 0, policy_version 1370611 (0.0010) [2023-12-27 01:22:21,444][105692] Updated weights for policy 0, policy_version 1370621 (0.0008) [2023-12-27 01:22:21,491][105692] Updated weights for policy 0, policy_version 1370631 (0.0008) [2023-12-27 01:22:21,648][105620] Updated weights for policy 1, policy_version 1372682 (0.0009) [2023-12-27 01:22:21,716][105620] Updated weights for policy 1, policy_version 1372692 (0.0009) [2023-12-27 01:22:21,783][105620] Updated weights for policy 1, policy_version 1372702 (0.0009) [2023-12-27 01:22:22,235][105692] Updated weights for policy 0, policy_version 1370641 (0.0006) [2023-12-27 01:22:22,295][105692] Updated weights for policy 0, policy_version 1370651 (0.0010) [2023-12-27 01:22:22,377][105692] Updated weights for policy 0, policy_version 1370661 (0.0008) [2023-12-27 01:22:22,425][105692] Updated weights for policy 0, policy_version 1370671 (0.0009) [2023-12-27 01:22:22,534][105620] Updated weights for policy 1, policy_version 1372712 (0.0009) [2023-12-27 01:22:22,588][105620] Updated weights for policy 1, policy_version 1372722 (0.0008) [2023-12-27 01:22:22,651][105620] Updated weights for policy 1, policy_version 1372732 (0.0009) [2023-12-27 01:22:23,199][105692] Updated weights for policy 0, policy_version 1370681 (0.0009) [2023-12-27 01:22:23,251][105692] Updated weights for policy 0, policy_version 1370691 (0.0009) [2023-12-27 01:22:23,301][105692] Updated weights for policy 0, policy_version 1370701 (0.0009) [2023-12-27 01:22:23,402][105620] Updated weights for policy 1, policy_version 1372742 (0.0009) [2023-12-27 01:22:23,455][105620] Updated weights for policy 1, policy_version 1372752 (0.0009) [2023-12-27 01:22:23,508][105620] Updated weights for policy 1, policy_version 1372763 (0.0009) [2023-12-27 01:22:23,993][105692] Updated weights for policy 0, policy_version 1370711 (0.0008) [2023-12-27 01:22:24,051][105692] Updated weights for policy 0, policy_version 1370721 (0.0010) [2023-12-27 01:22:24,113][105692] Updated weights for policy 0, policy_version 1370731 (0.0010) [2023-12-27 01:22:24,374][105620] Updated weights for policy 1, policy_version 1372773 (0.0008) [2023-12-27 01:22:24,436][105620] Updated weights for policy 1, policy_version 1372783 (0.0008) [2023-12-27 01:22:24,493][105620] Updated weights for policy 1, policy_version 1372793 (0.0008) [2023-12-27 01:22:24,769][105692] Updated weights for policy 0, policy_version 1370741 (0.0009) [2023-12-27 01:22:24,819][105692] Updated weights for policy 0, policy_version 1370751 (0.0009) [2023-12-27 01:22:24,867][105692] Updated weights for policy 0, policy_version 1370761 (0.0009) [2023-12-27 01:22:25,249][105620] Updated weights for policy 1, policy_version 1372803 (0.0008) [2023-12-27 01:22:25,306][105620] Updated weights for policy 1, policy_version 1372813 (0.0009) [2023-12-27 01:22:25,356][105620] Updated weights for policy 1, policy_version 1372823 (0.0009) [2023-12-27 01:22:25,633][105692] Updated weights for policy 0, policy_version 1370771 (0.0009) [2023-12-27 01:22:25,680][105692] Updated weights for policy 0, policy_version 1370781 (0.0009) [2023-12-27 01:22:25,727][105692] Updated weights for policy 0, policy_version 1370791 (0.0008) [2023-12-27 01:22:26,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 702464000. Throughput: 0: 9678.7, 1: 9492.7. Samples: 702474864. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 01:22:26,063][104569] Avg episode reward: [(0, '7987.844'), (1, '8898.477')] [2023-12-27 01:22:26,120][105620] Updated weights for policy 1, policy_version 1372833 (0.0009) [2023-12-27 01:22:26,173][105620] Updated weights for policy 1, policy_version 1372843 (0.0008) [2023-12-27 01:22:26,226][105620] Updated weights for policy 1, policy_version 1372853 (0.0009) [2023-12-27 01:22:26,277][105620] Updated weights for policy 1, policy_version 1372863 (0.0009) [2023-12-27 01:22:26,506][105692] Updated weights for policy 0, policy_version 1370801 (0.0009) [2023-12-27 01:22:26,567][105692] Updated weights for policy 0, policy_version 1370811 (0.0009) [2023-12-27 01:22:26,628][105692] Updated weights for policy 0, policy_version 1370821 (0.0009) [2023-12-27 01:22:26,690][105692] Updated weights for policy 0, policy_version 1370831 (0.0009) [2023-12-27 01:22:27,049][105620] Updated weights for policy 1, policy_version 1372873 (0.0007) [2023-12-27 01:22:27,105][105620] Updated weights for policy 1, policy_version 1372883 (0.0007) [2023-12-27 01:22:27,162][105620] Updated weights for policy 1, policy_version 1372893 (0.0009) [2023-12-27 01:22:27,441][105692] Updated weights for policy 0, policy_version 1370841 (0.0009) [2023-12-27 01:22:27,491][105692] Updated weights for policy 0, policy_version 1370851 (0.0009) [2023-12-27 01:22:27,537][105692] Updated weights for policy 0, policy_version 1370861 (0.0008) [2023-12-27 01:22:27,850][105620] Updated weights for policy 1, policy_version 1372903 (0.0009) [2023-12-27 01:22:27,908][105620] Updated weights for policy 1, policy_version 1372913 (0.0009) [2023-12-27 01:22:27,970][105620] Updated weights for policy 1, policy_version 1372923 (0.0009) [2023-12-27 01:22:28,336][105692] Updated weights for policy 0, policy_version 1370871 (0.0009) [2023-12-27 01:22:28,404][105692] Updated weights for policy 0, policy_version 1370881 (0.0009) [2023-12-27 01:22:28,471][105692] Updated weights for policy 0, policy_version 1370891 (0.0009) [2023-12-27 01:22:28,659][105620] Updated weights for policy 1, policy_version 1372933 (0.0007) [2023-12-27 01:22:28,719][105620] Updated weights for policy 1, policy_version 1372943 (0.0009) [2023-12-27 01:22:28,781][105620] Updated weights for policy 1, policy_version 1372953 (0.0009) [2023-12-27 01:22:29,275][105692] Updated weights for policy 0, policy_version 1370901 (0.0010) [2023-12-27 01:22:29,338][105692] Updated weights for policy 0, policy_version 1370911 (0.0011) [2023-12-27 01:22:29,405][105692] Updated weights for policy 0, policy_version 1370921 (0.0010) [2023-12-27 01:22:29,454][105620] Updated weights for policy 1, policy_version 1372963 (0.0008) [2023-12-27 01:22:29,499][105620] Updated weights for policy 1, policy_version 1372973 (0.0008) [2023-12-27 01:22:29,547][105620] Updated weights for policy 1, policy_version 1372983 (0.0008) [2023-12-27 01:22:30,152][105692] Updated weights for policy 0, policy_version 1370931 (0.0010) [2023-12-27 01:22:30,211][105692] Updated weights for policy 0, policy_version 1370941 (0.0010) [2023-12-27 01:22:30,272][105692] Updated weights for policy 0, policy_version 1370951 (0.0010) [2023-12-27 01:22:30,366][105620] Updated weights for policy 1, policy_version 1372993 (0.0008) [2023-12-27 01:22:30,421][105620] Updated weights for policy 1, policy_version 1373003 (0.0008) [2023-12-27 01:22:30,472][105620] Updated weights for policy 1, policy_version 1373013 (0.0008) [2023-12-27 01:22:30,519][105620] Updated weights for policy 1, policy_version 1373023 (0.0008) [2023-12-27 01:22:30,902][105692] Updated weights for policy 0, policy_version 1370961 (0.0010) [2023-12-27 01:22:30,970][105692] Updated weights for policy 0, policy_version 1370971 (0.0006) [2023-12-27 01:22:31,038][105692] Updated weights for policy 0, policy_version 1370981 (0.0006) [2023-12-27 01:22:31,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 702554112. Throughput: 0: 9668.3, 1: 9453.1. Samples: 702531044. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:22:31,063][104569] Avg episode reward: [(0, '8353.345'), (1, '9173.695')] [2023-12-27 01:22:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001373024_351535104.pth... [2023-12-27 01:22:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001371936_351256576.pth [2023-12-27 01:22:31,103][105692] Updated weights for policy 0, policy_version 1370991 (0.0006) [2023-12-27 01:22:31,108][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001370992_351027200.pth... [2023-12-27 01:22:31,111][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001369840_350732288.pth [2023-12-27 01:22:31,370][105620] Updated weights for policy 1, policy_version 1373033 (0.0008) [2023-12-27 01:22:31,442][105620] Updated weights for policy 1, policy_version 1373043 (0.0006) [2023-12-27 01:22:31,508][105620] Updated weights for policy 1, policy_version 1373053 (0.0005) [2023-12-27 01:22:31,803][105692] Updated weights for policy 0, policy_version 1371001 (0.0007) [2023-12-27 01:22:31,856][105692] Updated weights for policy 0, policy_version 1371011 (0.0005) [2023-12-27 01:22:31,857][105585] KL-divergence is very high: 145.4498 [2023-12-27 01:22:31,899][105585] KL-divergence is very high: 125.5750 [2023-12-27 01:22:31,911][105692] Updated weights for policy 0, policy_version 1371021 (0.0006) [2023-12-27 01:22:32,146][105620] Updated weights for policy 1, policy_version 1373063 (0.0007) [2023-12-27 01:22:32,210][105620] Updated weights for policy 1, policy_version 1373073 (0.0006) [2023-12-27 01:22:32,282][105620] Updated weights for policy 1, policy_version 1373083 (0.0007) [2023-12-27 01:22:32,537][105692] Updated weights for policy 0, policy_version 1371031 (0.0009) [2023-12-27 01:22:32,596][105692] Updated weights for policy 0, policy_version 1371041 (0.0009) [2023-12-27 01:22:32,650][105692] Updated weights for policy 0, policy_version 1371051 (0.0008) [2023-12-27 01:22:32,986][105620] Updated weights for policy 1, policy_version 1373093 (0.0009) [2023-12-27 01:22:33,041][105620] Updated weights for policy 1, policy_version 1373103 (0.0010) [2023-12-27 01:22:33,106][105620] Updated weights for policy 1, policy_version 1373113 (0.0008) [2023-12-27 01:22:33,402][105692] Updated weights for policy 0, policy_version 1371061 (0.0009) [2023-12-27 01:22:33,455][105692] Updated weights for policy 0, policy_version 1371071 (0.0010) [2023-12-27 01:22:33,507][105692] Updated weights for policy 0, policy_version 1371082 (0.0009) [2023-12-27 01:22:33,713][105620] Updated weights for policy 1, policy_version 1373123 (0.0006) [2023-12-27 01:22:33,763][105620] Updated weights for policy 1, policy_version 1373133 (0.0005) [2023-12-27 01:22:33,819][105620] Updated weights for policy 1, policy_version 1373143 (0.0005) [2023-12-27 01:22:34,352][105692] Updated weights for policy 0, policy_version 1371092 (0.0009) [2023-12-27 01:22:34,418][105692] Updated weights for policy 0, policy_version 1371102 (0.0009) [2023-12-27 01:22:34,481][105692] Updated weights for policy 0, policy_version 1371112 (0.0009) [2023-12-27 01:22:34,502][105620] Updated weights for policy 1, policy_version 1373153 (0.0006) [2023-12-27 01:22:34,568][105620] Updated weights for policy 1, policy_version 1373163 (0.0007) [2023-12-27 01:22:34,629][105620] Updated weights for policy 1, policy_version 1373173 (0.0008) [2023-12-27 01:22:34,692][105620] Updated weights for policy 1, policy_version 1373183 (0.0009) [2023-12-27 01:22:35,186][105692] Updated weights for policy 0, policy_version 1371122 (0.0009) [2023-12-27 01:22:35,241][105692] Updated weights for policy 0, policy_version 1371132 (0.0009) [2023-12-27 01:22:35,292][105692] Updated weights for policy 0, policy_version 1371143 (0.0009) [2023-12-27 01:22:35,427][105620] Updated weights for policy 1, policy_version 1373193 (0.0009) [2023-12-27 01:22:35,474][105620] Updated weights for policy 1, policy_version 1373203 (0.0009) [2023-12-27 01:22:35,521][105620] Updated weights for policy 1, policy_version 1373213 (0.0008) [2023-12-27 01:22:36,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 702652416. Throughput: 0: 9646.7, 1: 9461.8. Samples: 702646476. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:22:36,062][104569] Avg episode reward: [(0, '8627.848'), (1, '9173.744')] [2023-12-27 01:22:36,127][105692] Updated weights for policy 0, policy_version 1371154 (0.0010) [2023-12-27 01:22:36,189][105692] Updated weights for policy 0, policy_version 1371164 (0.0009) [2023-12-27 01:22:36,225][105620] Updated weights for policy 1, policy_version 1373223 (0.0008) [2023-12-27 01:22:36,249][105692] Updated weights for policy 0, policy_version 1371174 (0.0005) [2023-12-27 01:22:36,285][105620] Updated weights for policy 1, policy_version 1373233 (0.0009) [2023-12-27 01:22:36,307][105692] Updated weights for policy 0, policy_version 1371184 (0.0006) [2023-12-27 01:22:36,349][105620] Updated weights for policy 1, policy_version 1373243 (0.0009) [2023-12-27 01:22:36,965][105692] Updated weights for policy 0, policy_version 1371194 (0.0006) [2023-12-27 01:22:37,031][105692] Updated weights for policy 0, policy_version 1371204 (0.0007) [2023-12-27 01:22:37,098][105692] Updated weights for policy 0, policy_version 1371214 (0.0008) [2023-12-27 01:22:37,195][105620] Updated weights for policy 1, policy_version 1373253 (0.0009) [2023-12-27 01:22:37,252][105620] Updated weights for policy 1, policy_version 1373263 (0.0010) [2023-12-27 01:22:37,316][105620] Updated weights for policy 1, policy_version 1373273 (0.0006) [2023-12-27 01:22:37,676][105692] Updated weights for policy 0, policy_version 1371224 (0.0006) [2023-12-27 01:22:37,743][105692] Updated weights for policy 0, policy_version 1371234 (0.0006) [2023-12-27 01:22:37,805][105692] Updated weights for policy 0, policy_version 1371244 (0.0006) [2023-12-27 01:22:37,992][105620] Updated weights for policy 1, policy_version 1373283 (0.0007) [2023-12-27 01:22:38,046][105620] Updated weights for policy 1, policy_version 1373293 (0.0009) [2023-12-27 01:22:38,105][105620] Updated weights for policy 1, policy_version 1373303 (0.0010) [2023-12-27 01:22:38,425][105692] Updated weights for policy 0, policy_version 1371254 (0.0007) [2023-12-27 01:22:38,476][105692] Updated weights for policy 0, policy_version 1371264 (0.0009) [2023-12-27 01:22:38,537][105692] Updated weights for policy 0, policy_version 1371274 (0.0009) [2023-12-27 01:22:38,872][105620] Updated weights for policy 1, policy_version 1373313 (0.0009) [2023-12-27 01:22:38,926][105620] Updated weights for policy 1, policy_version 1373323 (0.0009) [2023-12-27 01:22:38,980][105620] Updated weights for policy 1, policy_version 1373333 (0.0009) [2023-12-27 01:22:39,039][105620] Updated weights for policy 1, policy_version 1373343 (0.0009) [2023-12-27 01:22:39,268][105692] Updated weights for policy 0, policy_version 1371284 (0.0007) [2023-12-27 01:22:39,334][105692] Updated weights for policy 0, policy_version 1371294 (0.0008) [2023-12-27 01:22:39,405][105692] Updated weights for policy 0, policy_version 1371304 (0.0009) [2023-12-27 01:22:39,810][105620] Updated weights for policy 1, policy_version 1373353 (0.0009) [2023-12-27 01:22:39,873][105620] Updated weights for policy 1, policy_version 1373363 (0.0009) [2023-12-27 01:22:39,929][105620] Updated weights for policy 1, policy_version 1373373 (0.0009) [2023-12-27 01:22:40,171][105692] Updated weights for policy 0, policy_version 1371314 (0.0008) [2023-12-27 01:22:40,234][105692] Updated weights for policy 0, policy_version 1371324 (0.0006) [2023-12-27 01:22:40,301][105692] Updated weights for policy 0, policy_version 1371334 (0.0006) [2023-12-27 01:22:40,365][105692] Updated weights for policy 0, policy_version 1371344 (0.0008) [2023-12-27 01:22:40,615][105620] Updated weights for policy 1, policy_version 1373383 (0.0007) [2023-12-27 01:22:40,675][105620] Updated weights for policy 1, policy_version 1373393 (0.0008) [2023-12-27 01:22:40,733][105620] Updated weights for policy 1, policy_version 1373403 (0.0008) [2023-12-27 01:22:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19114.6, 300 sec: 19494.2). Total num frames: 702750720. Throughput: 0: 9720.6, 1: 9433.3. Samples: 702762384. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:22:41,063][104569] Avg episode reward: [(0, '8722.175'), (1, '9174.616')] [2023-12-27 01:22:41,094][105692] Updated weights for policy 0, policy_version 1371354 (0.0011) [2023-12-27 01:22:41,168][105692] Updated weights for policy 0, policy_version 1371364 (0.0011) [2023-12-27 01:22:41,229][105692] Updated weights for policy 0, policy_version 1371374 (0.0011) [2023-12-27 01:22:41,528][105620] Updated weights for policy 1, policy_version 1373413 (0.0008) [2023-12-27 01:22:41,593][105620] Updated weights for policy 1, policy_version 1373423 (0.0008) [2023-12-27 01:22:41,664][105620] Updated weights for policy 1, policy_version 1373433 (0.0009) [2023-12-27 01:22:42,017][105692] Updated weights for policy 0, policy_version 1371384 (0.0011) [2023-12-27 01:22:42,080][105692] Updated weights for policy 0, policy_version 1371394 (0.0011) [2023-12-27 01:22:42,137][105692] Updated weights for policy 0, policy_version 1371404 (0.0011) [2023-12-27 01:22:42,453][105620] Updated weights for policy 1, policy_version 1373443 (0.0008) [2023-12-27 01:22:42,515][105620] Updated weights for policy 1, policy_version 1373453 (0.0009) [2023-12-27 01:22:42,583][105620] Updated weights for policy 1, policy_version 1373463 (0.0006) [2023-12-27 01:22:42,905][105692] Updated weights for policy 0, policy_version 1371414 (0.0011) [2023-12-27 01:22:42,970][105692] Updated weights for policy 0, policy_version 1371424 (0.0011) [2023-12-27 01:22:43,029][105692] Updated weights for policy 0, policy_version 1371434 (0.0011) [2023-12-27 01:22:43,177][105620] Updated weights for policy 1, policy_version 1373473 (0.0007) [2023-12-27 01:22:43,241][105620] Updated weights for policy 1, policy_version 1373483 (0.0006) [2023-12-27 01:22:43,301][105620] Updated weights for policy 1, policy_version 1373493 (0.0006) [2023-12-27 01:22:43,360][105620] Updated weights for policy 1, policy_version 1373503 (0.0010) [2023-12-27 01:22:43,766][105692] Updated weights for policy 0, policy_version 1371444 (0.0010) [2023-12-27 01:22:43,829][105692] Updated weights for policy 0, policy_version 1371454 (0.0010) [2023-12-27 01:22:43,891][105692] Updated weights for policy 0, policy_version 1371464 (0.0008) [2023-12-27 01:22:43,921][105620] Updated weights for policy 1, policy_version 1373513 (0.0006) [2023-12-27 01:22:43,973][105620] Updated weights for policy 1, policy_version 1373523 (0.0008) [2023-12-27 01:22:44,019][105620] Updated weights for policy 1, policy_version 1373533 (0.0009) [2023-12-27 01:22:44,648][105692] Updated weights for policy 0, policy_version 1371474 (0.0008) [2023-12-27 01:22:44,704][105692] Updated weights for policy 0, policy_version 1371484 (0.0008) [2023-12-27 01:22:44,730][105620] Updated weights for policy 1, policy_version 1373544 (0.0010) [2023-12-27 01:22:44,760][105692] Updated weights for policy 0, policy_version 1371494 (0.0006) [2023-12-27 01:22:44,794][105620] Updated weights for policy 1, policy_version 1373554 (0.0008) [2023-12-27 01:22:44,816][105692] Updated weights for policy 0, policy_version 1371504 (0.0007) [2023-12-27 01:22:44,862][105620] Updated weights for policy 1, policy_version 1373564 (0.0009) [2023-12-27 01:22:45,429][105692] Updated weights for policy 0, policy_version 1371514 (0.0005) [2023-12-27 01:22:45,484][105692] Updated weights for policy 0, policy_version 1371524 (0.0005) [2023-12-27 01:22:45,547][105692] Updated weights for policy 0, policy_version 1371534 (0.0007) [2023-12-27 01:22:45,716][105620] Updated weights for policy 1, policy_version 1373574 (0.0007) [2023-12-27 01:22:45,773][105620] Updated weights for policy 1, policy_version 1373584 (0.0009) [2023-12-27 01:22:45,829][105620] Updated weights for policy 1, policy_version 1373594 (0.0009) [2023-12-27 01:22:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 702849024. Throughput: 0: 9645.7, 1: 9444.1. Samples: 702820072. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:22:46,062][104569] Avg episode reward: [(0, '8447.316'), (1, '8903.712')] [2023-12-27 01:22:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001371536_351166464.pth... [2023-12-27 01:22:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001373600_351682560.pth... [2023-12-27 01:22:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001370416_350879744.pth [2023-12-27 01:22:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001372480_351395840.pth [2023-12-27 01:22:46,208][105692] Updated weights for policy 0, policy_version 1371544 (0.0009) [2023-12-27 01:22:46,270][105692] Updated weights for policy 0, policy_version 1371554 (0.0008) [2023-12-27 01:22:46,327][105692] Updated weights for policy 0, policy_version 1371564 (0.0008) [2023-12-27 01:22:46,541][105620] Updated weights for policy 1, policy_version 1373604 (0.0008) [2023-12-27 01:22:46,586][105620] Updated weights for policy 1, policy_version 1373614 (0.0010) [2023-12-27 01:22:46,634][105620] Updated weights for policy 1, policy_version 1373624 (0.0010) [2023-12-27 01:22:47,062][105692] Updated weights for policy 0, policy_version 1371574 (0.0007) [2023-12-27 01:22:47,114][105692] Updated weights for policy 0, policy_version 1371584 (0.0008) [2023-12-27 01:22:47,167][105692] Updated weights for policy 0, policy_version 1371594 (0.0008) [2023-12-27 01:22:47,401][105620] Updated weights for policy 1, policy_version 1373634 (0.0010) [2023-12-27 01:22:47,463][105620] Updated weights for policy 1, policy_version 1373644 (0.0010) [2023-12-27 01:22:47,518][105620] Updated weights for policy 1, policy_version 1373654 (0.0010) [2023-12-27 01:22:47,566][105620] Updated weights for policy 1, policy_version 1373664 (0.0010) [2023-12-27 01:22:47,978][105692] Updated weights for policy 0, policy_version 1371604 (0.0009) [2023-12-27 01:22:48,041][105692] Updated weights for policy 0, policy_version 1371614 (0.0010) [2023-12-27 01:22:48,098][105692] Updated weights for policy 0, policy_version 1371624 (0.0010) [2023-12-27 01:22:48,195][105620] Updated weights for policy 1, policy_version 1373674 (0.0006) [2023-12-27 01:22:48,249][105620] Updated weights for policy 1, policy_version 1373684 (0.0009) [2023-12-27 01:22:48,304][105620] Updated weights for policy 1, policy_version 1373694 (0.0009) [2023-12-27 01:22:48,830][105692] Updated weights for policy 0, policy_version 1371635 (0.0011) [2023-12-27 01:22:48,890][105692] Updated weights for policy 0, policy_version 1371645 (0.0010) [2023-12-27 01:22:48,948][105692] Updated weights for policy 0, policy_version 1371655 (0.0011) [2023-12-27 01:22:49,076][105620] Updated weights for policy 1, policy_version 1373704 (0.0010) [2023-12-27 01:22:49,131][105620] Updated weights for policy 1, policy_version 1373714 (0.0010) [2023-12-27 01:22:49,183][105620] Updated weights for policy 1, policy_version 1373724 (0.0011) [2023-12-27 01:22:49,719][105692] Updated weights for policy 0, policy_version 1371665 (0.0010) [2023-12-27 01:22:49,781][105692] Updated weights for policy 0, policy_version 1371675 (0.0005) [2023-12-27 01:22:49,853][105692] Updated weights for policy 0, policy_version 1371685 (0.0008) [2023-12-27 01:22:49,903][105620] Updated weights for policy 1, policy_version 1373734 (0.0010) [2023-12-27 01:22:49,916][105692] Updated weights for policy 0, policy_version 1371695 (0.0008) [2023-12-27 01:22:49,975][105620] Updated weights for policy 1, policy_version 1373744 (0.0013) [2023-12-27 01:22:50,029][105620] Updated weights for policy 1, policy_version 1373754 (0.0009) [2023-12-27 01:22:50,625][105692] Updated weights for policy 0, policy_version 1371705 (0.0009) [2023-12-27 01:22:50,681][105692] Updated weights for policy 0, policy_version 1371715 (0.0008) [2023-12-27 01:22:50,743][105692] Updated weights for policy 0, policy_version 1371725 (0.0009) [2023-12-27 01:22:50,808][105620] Updated weights for policy 1, policy_version 1373764 (0.0009) [2023-12-27 01:22:50,871][105620] Updated weights for policy 1, policy_version 1373774 (0.0009) [2023-12-27 01:22:50,936][105620] Updated weights for policy 1, policy_version 1373784 (0.0008) [2023-12-27 01:22:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 702947328. Throughput: 0: 9564.4, 1: 9535.3. Samples: 702934520. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:22:51,063][104569] Avg episode reward: [(0, '8171.762'), (1, '8905.970')] [2023-12-27 01:22:51,530][105692] Updated weights for policy 0, policy_version 1371735 (0.0007) [2023-12-27 01:22:51,584][105692] Updated weights for policy 0, policy_version 1371745 (0.0006) [2023-12-27 01:22:51,651][105692] Updated weights for policy 0, policy_version 1371755 (0.0009) [2023-12-27 01:22:51,707][105620] Updated weights for policy 1, policy_version 1373794 (0.0010) [2023-12-27 01:22:51,773][105620] Updated weights for policy 1, policy_version 1373804 (0.0008) [2023-12-27 01:22:51,827][105620] Updated weights for policy 1, policy_version 1373814 (0.0009) [2023-12-27 01:22:51,874][105620] Updated weights for policy 1, policy_version 1373824 (0.0009) [2023-12-27 01:22:52,385][105692] Updated weights for policy 0, policy_version 1371765 (0.0009) [2023-12-27 01:22:52,444][105692] Updated weights for policy 0, policy_version 1371775 (0.0010) [2023-12-27 01:22:52,508][105692] Updated weights for policy 0, policy_version 1371785 (0.0010) [2023-12-27 01:22:52,625][105620] Updated weights for policy 1, policy_version 1373834 (0.0009) [2023-12-27 01:22:52,677][105620] Updated weights for policy 1, policy_version 1373844 (0.0009) [2023-12-27 01:22:52,725][105620] Updated weights for policy 1, policy_version 1373854 (0.0009) [2023-12-27 01:22:53,279][105692] Updated weights for policy 0, policy_version 1371795 (0.0009) [2023-12-27 01:22:53,334][105692] Updated weights for policy 0, policy_version 1371805 (0.0009) [2023-12-27 01:22:53,390][105692] Updated weights for policy 0, policy_version 1371815 (0.0009) [2023-12-27 01:22:53,499][105620] Updated weights for policy 1, policy_version 1373864 (0.0009) [2023-12-27 01:22:53,546][105620] Updated weights for policy 1, policy_version 1373874 (0.0009) [2023-12-27 01:22:53,593][105620] Updated weights for policy 1, policy_version 1373884 (0.0009) [2023-12-27 01:22:54,103][105692] Updated weights for policy 0, policy_version 1371825 (0.0008) [2023-12-27 01:22:54,154][105692] Updated weights for policy 0, policy_version 1371835 (0.0006) [2023-12-27 01:22:54,199][105692] Updated weights for policy 0, policy_version 1371845 (0.0005) [2023-12-27 01:22:54,246][105692] Updated weights for policy 0, policy_version 1371855 (0.0005) [2023-12-27 01:22:54,334][105620] Updated weights for policy 1, policy_version 1373894 (0.0010) [2023-12-27 01:22:54,392][105620] Updated weights for policy 1, policy_version 1373904 (0.0010) [2023-12-27 01:22:54,451][105620] Updated weights for policy 1, policy_version 1373914 (0.0011) [2023-12-27 01:22:54,872][105692] Updated weights for policy 0, policy_version 1371865 (0.0005) [2023-12-27 01:22:54,926][105692] Updated weights for policy 0, policy_version 1371875 (0.0005) [2023-12-27 01:22:54,984][105692] Updated weights for policy 0, policy_version 1371885 (0.0007) [2023-12-27 01:22:55,210][105620] Updated weights for policy 1, policy_version 1373924 (0.0011) [2023-12-27 01:22:55,275][105620] Updated weights for policy 1, policy_version 1373934 (0.0010) [2023-12-27 01:22:55,323][105620] Updated weights for policy 1, policy_version 1373944 (0.0010) [2023-12-27 01:22:55,610][105692] Updated weights for policy 0, policy_version 1371895 (0.0008) [2023-12-27 01:22:55,664][105692] Updated weights for policy 0, policy_version 1371905 (0.0008) [2023-12-27 01:22:55,715][105692] Updated weights for policy 0, policy_version 1371915 (0.0008) [2023-12-27 01:22:56,013][105620] Updated weights for policy 1, policy_version 1373954 (0.0009) [2023-12-27 01:22:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 703037440. Throughput: 0: 9655.0, 1: 9541.5. Samples: 703049848. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:22:56,062][104569] Avg episode reward: [(0, '8258.646'), (1, '8995.304')] [2023-12-27 01:22:56,063][105620] Updated weights for policy 1, policy_version 1373964 (0.0006) [2023-12-27 01:22:56,110][105620] Updated weights for policy 1, policy_version 1373974 (0.0007) [2023-12-27 01:22:56,155][105620] Updated weights for policy 1, policy_version 1373984 (0.0007) [2023-12-27 01:22:56,479][105692] Updated weights for policy 0, policy_version 1371925 (0.0008) [2023-12-27 01:22:56,536][105692] Updated weights for policy 0, policy_version 1371935 (0.0005) [2023-12-27 01:22:56,602][105692] Updated weights for policy 0, policy_version 1371945 (0.0005) [2023-12-27 01:22:56,877][105620] Updated weights for policy 1, policy_version 1373994 (0.0005) [2023-12-27 01:22:56,926][105620] Updated weights for policy 1, policy_version 1374004 (0.0005) [2023-12-27 01:22:56,977][105620] Updated weights for policy 1, policy_version 1374014 (0.0007) [2023-12-27 01:22:57,109][105692] Updated weights for policy 0, policy_version 1371955 (0.0005) [2023-12-27 01:22:57,159][105692] Updated weights for policy 0, policy_version 1371965 (0.0005) [2023-12-27 01:22:57,213][105692] Updated weights for policy 0, policy_version 1371975 (0.0005) [2023-12-27 01:22:57,607][105620] Updated weights for policy 1, policy_version 1374024 (0.0009) [2023-12-27 01:22:57,667][105620] Updated weights for policy 1, policy_version 1374034 (0.0008) [2023-12-27 01:22:57,730][105620] Updated weights for policy 1, policy_version 1374044 (0.0006) [2023-12-27 01:22:57,818][105692] Updated weights for policy 0, policy_version 1371985 (0.0006) [2023-12-27 01:22:57,883][105692] Updated weights for policy 0, policy_version 1371995 (0.0005) [2023-12-27 01:22:57,942][105692] Updated weights for policy 0, policy_version 1372005 (0.0005) [2023-12-27 01:22:57,993][105692] Updated weights for policy 0, policy_version 1372015 (0.0005) [2023-12-27 01:22:58,467][105620] Updated weights for policy 1, policy_version 1374054 (0.0008) [2023-12-27 01:22:58,534][105620] Updated weights for policy 1, policy_version 1374064 (0.0010) [2023-12-27 01:22:58,582][105692] Updated weights for policy 0, policy_version 1372025 (0.0008) [2023-12-27 01:22:58,602][105620] Updated weights for policy 1, policy_version 1374074 (0.0006) [2023-12-27 01:22:58,644][105692] Updated weights for policy 0, policy_version 1372035 (0.0009) [2023-12-27 01:22:58,703][105692] Updated weights for policy 0, policy_version 1372045 (0.0009) [2023-12-27 01:22:59,300][105620] Updated weights for policy 1, policy_version 1374084 (0.0009) [2023-12-27 01:22:59,370][105620] Updated weights for policy 1, policy_version 1374094 (0.0009) [2023-12-27 01:22:59,433][105692] Updated weights for policy 0, policy_version 1372055 (0.0009) [2023-12-27 01:22:59,436][105620] Updated weights for policy 1, policy_version 1374104 (0.0008) [2023-12-27 01:22:59,490][105692] Updated weights for policy 0, policy_version 1372065 (0.0006) [2023-12-27 01:22:59,549][105692] Updated weights for policy 0, policy_version 1372075 (0.0010) [2023-12-27 01:23:00,063][105620] Updated weights for policy 1, policy_version 1374114 (0.0006) [2023-12-27 01:23:00,119][105620] Updated weights for policy 1, policy_version 1374124 (0.0006) [2023-12-27 01:23:00,166][105620] Updated weights for policy 1, policy_version 1374134 (0.0006) [2023-12-27 01:23:00,235][105620] Updated weights for policy 1, policy_version 1374144 (0.0005) [2023-12-27 01:23:00,428][105692] Updated weights for policy 0, policy_version 1372085 (0.0009) [2023-12-27 01:23:00,489][105692] Updated weights for policy 0, policy_version 1372095 (0.0009) [2023-12-27 01:23:00,548][105692] Updated weights for policy 0, policy_version 1372105 (0.0009) [2023-12-27 01:23:00,814][105620] Updated weights for policy 1, policy_version 1374154 (0.0006) [2023-12-27 01:23:00,871][105620] Updated weights for policy 1, policy_version 1374164 (0.0007) [2023-12-27 01:23:00,934][105620] Updated weights for policy 1, policy_version 1374174 (0.0009) [2023-12-27 01:23:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 703143936. Throughput: 0: 9750.9, 1: 9591.0. Samples: 703112764. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:23:01,063][104569] Avg episode reward: [(0, '8441.483'), (1, '9173.711')] [2023-12-27 01:23:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001372112_351313920.pth... [2023-12-27 01:23:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001374176_351830016.pth... [2023-12-27 01:23:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001370992_351027200.pth [2023-12-27 01:23:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001373024_351535104.pth [2023-12-27 01:23:01,273][105692] Updated weights for policy 0, policy_version 1372115 (0.0008) [2023-12-27 01:23:01,328][105692] Updated weights for policy 0, policy_version 1372125 (0.0007) [2023-12-27 01:23:01,391][105692] Updated weights for policy 0, policy_version 1372135 (0.0008) [2023-12-27 01:23:01,630][105620] Updated weights for policy 1, policy_version 1374184 (0.0011) [2023-12-27 01:23:01,689][105620] Updated weights for policy 1, policy_version 1374194 (0.0010) [2023-12-27 01:23:01,756][105620] Updated weights for policy 1, policy_version 1374204 (0.0011) [2023-12-27 01:23:02,056][105692] Updated weights for policy 0, policy_version 1372145 (0.0008) [2023-12-27 01:23:02,109][105692] Updated weights for policy 0, policy_version 1372155 (0.0008) [2023-12-27 01:23:02,168][105692] Updated weights for policy 0, policy_version 1372165 (0.0008) [2023-12-27 01:23:02,228][105692] Updated weights for policy 0, policy_version 1372175 (0.0008) [2023-12-27 01:23:02,486][105620] Updated weights for policy 1, policy_version 1374214 (0.0011) [2023-12-27 01:23:02,544][105620] Updated weights for policy 1, policy_version 1374224 (0.0010) [2023-12-27 01:23:02,604][105620] Updated weights for policy 1, policy_version 1374234 (0.0008) [2023-12-27 01:23:03,020][105692] Updated weights for policy 0, policy_version 1372185 (0.0008) [2023-12-27 01:23:03,065][105692] Updated weights for policy 0, policy_version 1372195 (0.0008) [2023-12-27 01:23:03,113][105692] Updated weights for policy 0, policy_version 1372205 (0.0008) [2023-12-27 01:23:03,298][105620] Updated weights for policy 1, policy_version 1374244 (0.0007) [2023-12-27 01:23:03,359][105620] Updated weights for policy 1, policy_version 1374254 (0.0010) [2023-12-27 01:23:03,407][105620] Updated weights for policy 1, policy_version 1374264 (0.0010) [2023-12-27 01:23:03,711][105692] Updated weights for policy 0, policy_version 1372215 (0.0006) [2023-12-27 01:23:03,767][105692] Updated weights for policy 0, policy_version 1372225 (0.0008) [2023-12-27 01:23:03,812][105692] Updated weights for policy 0, policy_version 1372235 (0.0008) [2023-12-27 01:23:04,147][105620] Updated weights for policy 1, policy_version 1374274 (0.0010) [2023-12-27 01:23:04,207][105620] Updated weights for policy 1, policy_version 1374284 (0.0010) [2023-12-27 01:23:04,262][105620] Updated weights for policy 1, policy_version 1374294 (0.0011) [2023-12-27 01:23:04,321][105620] Updated weights for policy 1, policy_version 1374304 (0.0010) [2023-12-27 01:23:04,606][105692] Updated weights for policy 0, policy_version 1372245 (0.0009) [2023-12-27 01:23:04,665][105692] Updated weights for policy 0, policy_version 1372255 (0.0010) [2023-12-27 01:23:04,718][105692] Updated weights for policy 0, policy_version 1372265 (0.0009) [2023-12-27 01:23:04,944][105620] Updated weights for policy 1, policy_version 1374314 (0.0009) [2023-12-27 01:23:04,998][105620] Updated weights for policy 1, policy_version 1374324 (0.0010) [2023-12-27 01:23:05,053][105620] Updated weights for policy 1, policy_version 1374334 (0.0010) [2023-12-27 01:23:05,547][105692] Updated weights for policy 0, policy_version 1372275 (0.0008) [2023-12-27 01:23:05,605][105692] Updated weights for policy 0, policy_version 1372285 (0.0007) [2023-12-27 01:23:05,662][105692] Updated weights for policy 0, policy_version 1372295 (0.0009) [2023-12-27 01:23:05,742][105620] Updated weights for policy 1, policy_version 1374344 (0.0008) [2023-12-27 01:23:05,779][105586] KL-divergence is very high: 142.3300 [2023-12-27 01:23:05,796][105620] Updated weights for policy 1, policy_version 1374354 (0.0008) [2023-12-27 01:23:05,799][105586] KL-divergence is very high: 100.0159 [2023-12-27 01:23:05,819][105586] KL-divergence is very high: 240.4892 [2023-12-27 01:23:05,838][105586] KL-divergence is very high: 114.6658 [2023-12-27 01:23:05,842][105620] Updated weights for policy 1, policy_version 1374364 (0.0009) [2023-12-27 01:23:05,856][105586] KL-divergence is very high: 251.1897 [2023-12-27 01:23:06,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 703242240. Throughput: 0: 9646.6, 1: 9636.5. Samples: 703230236. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:23:06,063][104569] Avg episode reward: [(0, '8446.457'), (1, '9084.642')] [2023-12-27 01:23:06,387][105692] Updated weights for policy 0, policy_version 1372305 (0.0008) [2023-12-27 01:23:06,445][105692] Updated weights for policy 0, policy_version 1372315 (0.0010) [2023-12-27 01:23:06,506][105692] Updated weights for policy 0, policy_version 1372325 (0.0009) [2023-12-27 01:23:06,521][105620] Updated weights for policy 1, policy_version 1374374 (0.0007) [2023-12-27 01:23:06,565][105692] Updated weights for policy 0, policy_version 1372335 (0.0010) [2023-12-27 01:23:06,576][105620] Updated weights for policy 1, policy_version 1374384 (0.0007) [2023-12-27 01:23:06,627][105620] Updated weights for policy 1, policy_version 1374394 (0.0009) [2023-12-27 01:23:07,339][105620] Updated weights for policy 1, policy_version 1374404 (0.0009) [2023-12-27 01:23:07,379][105692] Updated weights for policy 0, policy_version 1372345 (0.0007) [2023-12-27 01:23:07,399][105620] Updated weights for policy 1, policy_version 1374414 (0.0006) [2023-12-27 01:23:07,446][105692] Updated weights for policy 0, policy_version 1372355 (0.0008) [2023-12-27 01:23:07,457][105620] Updated weights for policy 1, policy_version 1374424 (0.0006) [2023-12-27 01:23:07,508][105692] Updated weights for policy 0, policy_version 1372365 (0.0007) [2023-12-27 01:23:08,134][105620] Updated weights for policy 1, policy_version 1374434 (0.0006) [2023-12-27 01:23:08,196][105620] Updated weights for policy 1, policy_version 1374444 (0.0006) [2023-12-27 01:23:08,257][105620] Updated weights for policy 1, policy_version 1374454 (0.0011) [2023-12-27 01:23:08,317][105620] Updated weights for policy 1, policy_version 1374464 (0.0010) [2023-12-27 01:23:08,348][105692] Updated weights for policy 0, policy_version 1372375 (0.0009) [2023-12-27 01:23:08,399][105692] Updated weights for policy 0, policy_version 1372386 (0.0009) [2023-12-27 01:23:08,453][105692] Updated weights for policy 0, policy_version 1372396 (0.0009) [2023-12-27 01:23:08,963][105620] Updated weights for policy 1, policy_version 1374474 (0.0009) [2023-12-27 01:23:09,022][105620] Updated weights for policy 1, policy_version 1374484 (0.0006) [2023-12-27 01:23:09,071][105620] Updated weights for policy 1, policy_version 1374494 (0.0005) [2023-12-27 01:23:09,309][105692] Updated weights for policy 0, policy_version 1372406 (0.0008) [2023-12-27 01:23:09,379][105692] Updated weights for policy 0, policy_version 1372416 (0.0009) [2023-12-27 01:23:09,448][105692] Updated weights for policy 0, policy_version 1372426 (0.0007) [2023-12-27 01:23:09,734][105620] Updated weights for policy 1, policy_version 1374504 (0.0009) [2023-12-27 01:23:09,796][105620] Updated weights for policy 1, policy_version 1374514 (0.0010) [2023-12-27 01:23:09,857][105620] Updated weights for policy 1, policy_version 1374524 (0.0008) [2023-12-27 01:23:10,135][105692] Updated weights for policy 0, policy_version 1372436 (0.0007) [2023-12-27 01:23:10,204][105692] Updated weights for policy 0, policy_version 1372446 (0.0006) [2023-12-27 01:23:10,274][105692] Updated weights for policy 0, policy_version 1372456 (0.0006) [2023-12-27 01:23:10,595][105620] Updated weights for policy 1, policy_version 1374534 (0.0007) [2023-12-27 01:23:10,648][105620] Updated weights for policy 1, policy_version 1374544 (0.0005) [2023-12-27 01:23:10,706][105620] Updated weights for policy 1, policy_version 1374554 (0.0005) [2023-12-27 01:23:10,947][105692] Updated weights for policy 0, policy_version 1372466 (0.0006) [2023-12-27 01:23:11,009][105692] Updated weights for policy 0, policy_version 1372476 (0.0010) [2023-12-27 01:23:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 703332352. Throughput: 0: 9568.5, 1: 9774.7. Samples: 703345304. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:23:11,062][104569] Avg episode reward: [(0, '8353.513'), (1, '8719.486')] [2023-12-27 01:23:11,071][105692] Updated weights for policy 0, policy_version 1372486 (0.0008) [2023-12-27 01:23:11,135][105692] Updated weights for policy 0, policy_version 1372496 (0.0008) [2023-12-27 01:23:11,319][105620] Updated weights for policy 1, policy_version 1374564 (0.0008) [2023-12-27 01:23:11,386][105620] Updated weights for policy 1, policy_version 1374574 (0.0009) [2023-12-27 01:23:11,453][105620] Updated weights for policy 1, policy_version 1374584 (0.0009) [2023-12-27 01:23:11,844][105692] Updated weights for policy 0, policy_version 1372506 (0.0009) [2023-12-27 01:23:11,904][105585] KL-divergence is very high: 162.5219 [2023-12-27 01:23:11,906][105692] Updated weights for policy 0, policy_version 1372516 (0.0009) [2023-12-27 01:23:11,950][105585] KL-divergence is very high: 190.6788 [2023-12-27 01:23:11,962][105692] Updated weights for policy 0, policy_version 1372526 (0.0009) [2023-12-27 01:23:12,256][105620] Updated weights for policy 1, policy_version 1374594 (0.0010) [2023-12-27 01:23:12,312][105620] Updated weights for policy 1, policy_version 1374604 (0.0009) [2023-12-27 01:23:12,377][105620] Updated weights for policy 1, policy_version 1374614 (0.0009) [2023-12-27 01:23:12,441][105620] Updated weights for policy 1, policy_version 1374624 (0.0009) [2023-12-27 01:23:12,701][105692] Updated weights for policy 0, policy_version 1372536 (0.0009) [2023-12-27 01:23:12,762][105692] Updated weights for policy 0, policy_version 1372546 (0.0010) [2023-12-27 01:23:12,821][105692] Updated weights for policy 0, policy_version 1372556 (0.0006) [2023-12-27 01:23:13,214][105620] Updated weights for policy 1, policy_version 1374634 (0.0005) [2023-12-27 01:23:13,277][105620] Updated weights for policy 1, policy_version 1374644 (0.0011) [2023-12-27 01:23:13,339][105620] Updated weights for policy 1, policy_version 1374654 (0.0010) [2023-12-27 01:23:13,455][105692] Updated weights for policy 0, policy_version 1372566 (0.0008) [2023-12-27 01:23:13,515][105692] Updated weights for policy 0, policy_version 1372576 (0.0007) [2023-12-27 01:23:13,579][105692] Updated weights for policy 0, policy_version 1372586 (0.0005) [2023-12-27 01:23:14,035][105620] Updated weights for policy 1, policy_version 1374664 (0.0006) [2023-12-27 01:23:14,084][105620] Updated weights for policy 1, policy_version 1374674 (0.0005) [2023-12-27 01:23:14,096][105692] Updated weights for policy 0, policy_version 1372596 (0.0007) [2023-12-27 01:23:14,139][105620] Updated weights for policy 1, policy_version 1374684 (0.0008) [2023-12-27 01:23:14,152][105692] Updated weights for policy 0, policy_version 1372606 (0.0010) [2023-12-27 01:23:14,210][105692] Updated weights for policy 0, policy_version 1372616 (0.0008) [2023-12-27 01:23:14,872][105620] Updated weights for policy 1, policy_version 1374694 (0.0009) [2023-12-27 01:23:14,935][105620] Updated weights for policy 1, policy_version 1374704 (0.0008) [2023-12-27 01:23:14,949][105692] Updated weights for policy 0, policy_version 1372626 (0.0009) [2023-12-27 01:23:14,991][105620] Updated weights for policy 1, policy_version 1374714 (0.0008) [2023-12-27 01:23:15,008][105692] Updated weights for policy 0, policy_version 1372636 (0.0007) [2023-12-27 01:23:15,068][105692] Updated weights for policy 0, policy_version 1372646 (0.0011) [2023-12-27 01:23:15,137][105692] Updated weights for policy 0, policy_version 1372656 (0.0007) [2023-12-27 01:23:15,765][105620] Updated weights for policy 1, policy_version 1374724 (0.0008) [2023-12-27 01:23:15,819][105692] Updated weights for policy 0, policy_version 1372666 (0.0008) [2023-12-27 01:23:15,824][105620] Updated weights for policy 1, policy_version 1374734 (0.0010) [2023-12-27 01:23:15,870][105692] Updated weights for policy 0, policy_version 1372676 (0.0005) [2023-12-27 01:23:15,882][105620] Updated weights for policy 1, policy_version 1374744 (0.0010) [2023-12-27 01:23:15,937][105692] Updated weights for policy 0, policy_version 1372686 (0.0006) [2023-12-27 01:23:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 703438848. Throughput: 0: 9613.6, 1: 9744.6. Samples: 703402164. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:23:16,063][104569] Avg episode reward: [(0, '8529.220'), (1, '8897.989')] [2023-12-27 01:23:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001374752_351977472.pth... [2023-12-27 01:23:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001372688_351461376.pth... [2023-12-27 01:23:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001371536_351166464.pth [2023-12-27 01:23:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001373600_351682560.pth [2023-12-27 01:23:16,606][105620] Updated weights for policy 1, policy_version 1374754 (0.0010) [2023-12-27 01:23:16,608][105692] Updated weights for policy 0, policy_version 1372696 (0.0010) [2023-12-27 01:23:16,667][105620] Updated weights for policy 1, policy_version 1374764 (0.0009) [2023-12-27 01:23:16,673][105692] Updated weights for policy 0, policy_version 1372706 (0.0010) [2023-12-27 01:23:16,732][105620] Updated weights for policy 1, policy_version 1374774 (0.0008) [2023-12-27 01:23:16,739][105692] Updated weights for policy 0, policy_version 1372716 (0.0010) [2023-12-27 01:23:16,798][105620] Updated weights for policy 1, policy_version 1374784 (0.0008) [2023-12-27 01:23:17,325][105692] Updated weights for policy 0, policy_version 1372726 (0.0007) [2023-12-27 01:23:17,386][105692] Updated weights for policy 0, policy_version 1372736 (0.0008) [2023-12-27 01:23:17,439][105692] Updated weights for policy 0, policy_version 1372746 (0.0005) [2023-12-27 01:23:17,448][105620] Updated weights for policy 1, policy_version 1374794 (0.0006) [2023-12-27 01:23:17,507][105620] Updated weights for policy 1, policy_version 1374804 (0.0010) [2023-12-27 01:23:17,565][105620] Updated weights for policy 1, policy_version 1374814 (0.0010) [2023-12-27 01:23:18,008][105692] Updated weights for policy 0, policy_version 1372756 (0.0006) [2023-12-27 01:23:18,064][105692] Updated weights for policy 0, policy_version 1372766 (0.0005) [2023-12-27 01:23:18,125][105692] Updated weights for policy 0, policy_version 1372776 (0.0010) [2023-12-27 01:23:18,232][105620] Updated weights for policy 1, policy_version 1374824 (0.0010) [2023-12-27 01:23:18,280][105620] Updated weights for policy 1, policy_version 1374834 (0.0010) [2023-12-27 01:23:18,339][105620] Updated weights for policy 1, policy_version 1374844 (0.0010) [2023-12-27 01:23:18,772][105692] Updated weights for policy 0, policy_version 1372786 (0.0011) [2023-12-27 01:23:18,838][105692] Updated weights for policy 0, policy_version 1372796 (0.0010) [2023-12-27 01:23:18,904][105692] Updated weights for policy 0, policy_version 1372806 (0.0011) [2023-12-27 01:23:18,963][105692] Updated weights for policy 0, policy_version 1372816 (0.0011) [2023-12-27 01:23:19,100][105620] Updated weights for policy 1, policy_version 1374854 (0.0008) [2023-12-27 01:23:19,149][105620] Updated weights for policy 1, policy_version 1374864 (0.0005) [2023-12-27 01:23:19,206][105620] Updated weights for policy 1, policy_version 1374874 (0.0005) [2023-12-27 01:23:19,682][105692] Updated weights for policy 0, policy_version 1372826 (0.0011) [2023-12-27 01:23:19,742][105692] Updated weights for policy 0, policy_version 1372836 (0.0011) [2023-12-27 01:23:19,805][105692] Updated weights for policy 0, policy_version 1372846 (0.0011) [2023-12-27 01:23:19,882][105620] Updated weights for policy 1, policy_version 1374884 (0.0008) [2023-12-27 01:23:19,947][105620] Updated weights for policy 1, policy_version 1374894 (0.0008) [2023-12-27 01:23:20,016][105620] Updated weights for policy 1, policy_version 1374904 (0.0009) [2023-12-27 01:23:20,528][105692] Updated weights for policy 0, policy_version 1372856 (0.0010) [2023-12-27 01:23:20,590][105692] Updated weights for policy 0, policy_version 1372866 (0.0009) [2023-12-27 01:23:20,645][105692] Updated weights for policy 0, policy_version 1372876 (0.0008) [2023-12-27 01:23:20,860][105620] Updated weights for policy 1, policy_version 1374914 (0.0008) [2023-12-27 01:23:20,924][105620] Updated weights for policy 1, policy_version 1374924 (0.0006) [2023-12-27 01:23:20,987][105620] Updated weights for policy 1, policy_version 1374934 (0.0009) [2023-12-27 01:23:21,053][105620] Updated weights for policy 1, policy_version 1374944 (0.0008) [2023-12-27 01:23:21,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 703537152. Throughput: 0: 9757.6, 1: 9742.9. Samples: 703524000. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:23:21,062][104569] Avg episode reward: [(0, '8536.548'), (1, '9173.652')] [2023-12-27 01:23:21,361][105692] Updated weights for policy 0, policy_version 1372886 (0.0008) [2023-12-27 01:23:21,428][105692] Updated weights for policy 0, policy_version 1372896 (0.0009) [2023-12-27 01:23:21,488][105692] Updated weights for policy 0, policy_version 1372906 (0.0009) [2023-12-27 01:23:21,800][105620] Updated weights for policy 1, policy_version 1374954 (0.0011) [2023-12-27 01:23:21,856][105620] Updated weights for policy 1, policy_version 1374964 (0.0010) [2023-12-27 01:23:21,920][105620] Updated weights for policy 1, policy_version 1374974 (0.0011) [2023-12-27 01:23:22,269][105692] Updated weights for policy 0, policy_version 1372916 (0.0008) [2023-12-27 01:23:22,333][105692] Updated weights for policy 0, policy_version 1372926 (0.0008) [2023-12-27 01:23:22,407][105692] Updated weights for policy 0, policy_version 1372936 (0.0009) [2023-12-27 01:23:22,693][105620] Updated weights for policy 1, policy_version 1374984 (0.0011) [2023-12-27 01:23:22,759][105620] Updated weights for policy 1, policy_version 1374994 (0.0010) [2023-12-27 01:23:22,814][105620] Updated weights for policy 1, policy_version 1375004 (0.0009) [2023-12-27 01:23:23,223][105692] Updated weights for policy 0, policy_version 1372946 (0.0009) [2023-12-27 01:23:23,274][105692] Updated weights for policy 0, policy_version 1372956 (0.0010) [2023-12-27 01:23:23,326][105692] Updated weights for policy 0, policy_version 1372966 (0.0010) [2023-12-27 01:23:23,374][105692] Updated weights for policy 0, policy_version 1372976 (0.0010) [2023-12-27 01:23:23,536][105620] Updated weights for policy 1, policy_version 1375014 (0.0010) [2023-12-27 01:23:23,595][105620] Updated weights for policy 1, policy_version 1375025 (0.0011) [2023-12-27 01:23:23,664][105620] Updated weights for policy 1, policy_version 1375035 (0.0010) [2023-12-27 01:23:23,998][105692] Updated weights for policy 0, policy_version 1372986 (0.0011) [2023-12-27 01:23:24,059][105692] Updated weights for policy 0, policy_version 1372996 (0.0010) [2023-12-27 01:23:24,112][105692] Updated weights for policy 0, policy_version 1373006 (0.0009) [2023-12-27 01:23:24,437][105620] Updated weights for policy 1, policy_version 1375045 (0.0010) [2023-12-27 01:23:24,495][105620] Updated weights for policy 1, policy_version 1375055 (0.0010) [2023-12-27 01:23:24,547][105620] Updated weights for policy 1, policy_version 1375065 (0.0010) [2023-12-27 01:23:24,818][105692] Updated weights for policy 0, policy_version 1373016 (0.0009) [2023-12-27 01:23:24,878][105692] Updated weights for policy 0, policy_version 1373026 (0.0008) [2023-12-27 01:23:24,934][105692] Updated weights for policy 0, policy_version 1373036 (0.0008) [2023-12-27 01:23:25,299][105620] Updated weights for policy 1, policy_version 1375075 (0.0009) [2023-12-27 01:23:25,360][105620] Updated weights for policy 1, policy_version 1375085 (0.0006) [2023-12-27 01:23:25,425][105620] Updated weights for policy 1, policy_version 1375095 (0.0009) [2023-12-27 01:23:25,671][105692] Updated weights for policy 0, policy_version 1373046 (0.0007) [2023-12-27 01:23:25,729][105692] Updated weights for policy 0, policy_version 1373056 (0.0007) [2023-12-27 01:23:25,794][105692] Updated weights for policy 0, policy_version 1373066 (0.0009) [2023-12-27 01:23:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 703627264. Throughput: 0: 9704.6, 1: 9705.5. Samples: 703635836. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:23:26,063][104569] Avg episode reward: [(0, '8903.540'), (1, '8995.227')] [2023-12-27 01:23:26,107][105620] Updated weights for policy 1, policy_version 1375105 (0.0007) [2023-12-27 01:23:26,167][105620] Updated weights for policy 1, policy_version 1375115 (0.0009) [2023-12-27 01:23:26,230][105620] Updated weights for policy 1, policy_version 1375125 (0.0009) [2023-12-27 01:23:26,283][105620] Updated weights for policy 1, policy_version 1375135 (0.0009) [2023-12-27 01:23:26,437][105692] Updated weights for policy 0, policy_version 1373076 (0.0008) [2023-12-27 01:23:26,481][105692] Updated weights for policy 0, policy_version 1373086 (0.0005) [2023-12-27 01:23:26,546][105692] Updated weights for policy 0, policy_version 1373096 (0.0005) [2023-12-27 01:23:27,088][105620] Updated weights for policy 1, policy_version 1375145 (0.0009) [2023-12-27 01:23:27,134][105620] Updated weights for policy 1, policy_version 1375155 (0.0008) [2023-12-27 01:23:27,190][105620] Updated weights for policy 1, policy_version 1375165 (0.0008) [2023-12-27 01:23:27,202][105692] Updated weights for policy 0, policy_version 1373106 (0.0005) [2023-12-27 01:23:27,252][105692] Updated weights for policy 0, policy_version 1373116 (0.0009) [2023-12-27 01:23:27,303][105692] Updated weights for policy 0, policy_version 1373126 (0.0009) [2023-12-27 01:23:27,356][105692] Updated weights for policy 0, policy_version 1373136 (0.0009) [2023-12-27 01:23:27,778][105620] Updated weights for policy 1, policy_version 1375175 (0.0006) [2023-12-27 01:23:27,829][105620] Updated weights for policy 1, policy_version 1375185 (0.0005) [2023-12-27 01:23:27,886][105620] Updated weights for policy 1, policy_version 1375195 (0.0005) [2023-12-27 01:23:28,238][105692] Updated weights for policy 0, policy_version 1373146 (0.0008) [2023-12-27 01:23:28,285][105692] Updated weights for policy 0, policy_version 1373156 (0.0008) [2023-12-27 01:23:28,333][105692] Updated weights for policy 0, policy_version 1373166 (0.0009) [2023-12-27 01:23:28,494][105620] Updated weights for policy 1, policy_version 1375205 (0.0006) [2023-12-27 01:23:28,550][105620] Updated weights for policy 1, policy_version 1375215 (0.0006) [2023-12-27 01:23:28,608][105620] Updated weights for policy 1, policy_version 1375225 (0.0009) [2023-12-27 01:23:29,169][105692] Updated weights for policy 0, policy_version 1373176 (0.0010) [2023-12-27 01:23:29,193][105620] Updated weights for policy 1, policy_version 1375235 (0.0007) [2023-12-27 01:23:29,220][105692] Updated weights for policy 0, policy_version 1373186 (0.0007) [2023-12-27 01:23:29,256][105620] Updated weights for policy 1, policy_version 1375245 (0.0009) [2023-12-27 01:23:29,279][105692] Updated weights for policy 0, policy_version 1373196 (0.0008) [2023-12-27 01:23:29,315][105620] Updated weights for policy 1, policy_version 1375255 (0.0009) [2023-12-27 01:23:29,900][105692] Updated weights for policy 0, policy_version 1373206 (0.0006) [2023-12-27 01:23:29,971][105692] Updated weights for policy 0, policy_version 1373216 (0.0007) [2023-12-27 01:23:30,036][105620] Updated weights for policy 1, policy_version 1375265 (0.0010) [2023-12-27 01:23:30,040][105692] Updated weights for policy 0, policy_version 1373226 (0.0006) [2023-12-27 01:23:30,100][105620] Updated weights for policy 1, policy_version 1375275 (0.0008) [2023-12-27 01:23:30,159][105620] Updated weights for policy 1, policy_version 1375285 (0.0010) [2023-12-27 01:23:30,219][105620] Updated weights for policy 1, policy_version 1375295 (0.0005) [2023-12-27 01:23:30,729][105692] Updated weights for policy 0, policy_version 1373236 (0.0006) [2023-12-27 01:23:30,774][105692] Updated weights for policy 0, policy_version 1373246 (0.0008) [2023-12-27 01:23:30,821][105692] Updated weights for policy 0, policy_version 1373256 (0.0008) [2023-12-27 01:23:30,921][105620] Updated weights for policy 1, policy_version 1375305 (0.0005) [2023-12-27 01:23:30,969][105620] Updated weights for policy 1, policy_version 1375315 (0.0005) [2023-12-27 01:23:31,031][105620] Updated weights for policy 1, policy_version 1375325 (0.0007) [2023-12-27 01:23:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 703733760. Throughput: 0: 9727.1, 1: 9737.3. Samples: 703695972. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:23:31,063][104569] Avg episode reward: [(0, '8717.640'), (1, '9084.673')] [2023-12-27 01:23:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001373264_351608832.pth... [2023-12-27 01:23:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001375328_352124928.pth... [2023-12-27 01:23:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001374176_351830016.pth [2023-12-27 01:23:31,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001372112_351313920.pth [2023-12-27 01:23:31,677][105620] Updated weights for policy 1, policy_version 1375335 (0.0009) [2023-12-27 01:23:31,707][105692] Updated weights for policy 0, policy_version 1373266 (0.0009) [2023-12-27 01:23:31,747][105620] Updated weights for policy 1, policy_version 1375345 (0.0011) [2023-12-27 01:23:31,769][105692] Updated weights for policy 0, policy_version 1373276 (0.0006) [2023-12-27 01:23:31,806][105620] Updated weights for policy 1, policy_version 1375355 (0.0011) [2023-12-27 01:23:31,830][105692] Updated weights for policy 0, policy_version 1373286 (0.0006) [2023-12-27 01:23:31,893][105692] Updated weights for policy 0, policy_version 1373296 (0.0008) [2023-12-27 01:23:32,560][105620] Updated weights for policy 1, policy_version 1375365 (0.0009) [2023-12-27 01:23:32,612][105620] Updated weights for policy 1, policy_version 1375375 (0.0009) [2023-12-27 01:23:32,630][105692] Updated weights for policy 0, policy_version 1373306 (0.0006) [2023-12-27 01:23:32,667][105620] Updated weights for policy 1, policy_version 1375385 (0.0010) [2023-12-27 01:23:32,685][105692] Updated weights for policy 0, policy_version 1373316 (0.0006) [2023-12-27 01:23:32,734][105692] Updated weights for policy 0, policy_version 1373326 (0.0007) [2023-12-27 01:23:33,359][105620] Updated weights for policy 1, policy_version 1375395 (0.0010) [2023-12-27 01:23:33,405][105620] Updated weights for policy 1, policy_version 1375405 (0.0009) [2023-12-27 01:23:33,450][105620] Updated weights for policy 1, policy_version 1375415 (0.0005) [2023-12-27 01:23:33,494][105692] Updated weights for policy 0, policy_version 1373336 (0.0007) [2023-12-27 01:23:33,543][105692] Updated weights for policy 0, policy_version 1373346 (0.0008) [2023-12-27 01:23:33,589][105692] Updated weights for policy 0, policy_version 1373356 (0.0008) [2023-12-27 01:23:34,046][105620] Updated weights for policy 1, policy_version 1375425 (0.0007) [2023-12-27 01:23:34,103][105620] Updated weights for policy 1, policy_version 1375435 (0.0010) [2023-12-27 01:23:34,168][105620] Updated weights for policy 1, policy_version 1375445 (0.0009) [2023-12-27 01:23:34,239][105620] Updated weights for policy 1, policy_version 1375455 (0.0011) [2023-12-27 01:23:34,424][105692] Updated weights for policy 0, policy_version 1373367 (0.0009) [2023-12-27 01:23:34,492][105692] Updated weights for policy 0, policy_version 1373377 (0.0008) [2023-12-27 01:23:34,558][105692] Updated weights for policy 0, policy_version 1373387 (0.0008) [2023-12-27 01:23:35,026][105620] Updated weights for policy 1, policy_version 1375465 (0.0011) [2023-12-27 01:23:35,087][105620] Updated weights for policy 1, policy_version 1375475 (0.0011) [2023-12-27 01:23:35,160][105620] Updated weights for policy 1, policy_version 1375485 (0.0011) [2023-12-27 01:23:35,317][105692] Updated weights for policy 0, policy_version 1373397 (0.0008) [2023-12-27 01:23:35,375][105692] Updated weights for policy 0, policy_version 1373407 (0.0009) [2023-12-27 01:23:35,433][105692] Updated weights for policy 0, policy_version 1373417 (0.0009) [2023-12-27 01:23:35,801][105620] Updated weights for policy 1, policy_version 1375495 (0.0007) [2023-12-27 01:23:35,855][105620] Updated weights for policy 1, policy_version 1375505 (0.0006) [2023-12-27 01:23:35,916][105620] Updated weights for policy 1, policy_version 1375515 (0.0005) [2023-12-27 01:23:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 703823872. Throughput: 0: 9696.4, 1: 9789.6. Samples: 703811392. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:23:36,062][104569] Avg episode reward: [(0, '8084.814'), (1, '9356.082')] [2023-12-27 01:23:36,272][105692] Updated weights for policy 0, policy_version 1373427 (0.0009) [2023-12-27 01:23:36,339][105692] Updated weights for policy 0, policy_version 1373437 (0.0010) [2023-12-27 01:23:36,407][105692] Updated weights for policy 0, policy_version 1373447 (0.0010) [2023-12-27 01:23:36,554][105620] Updated weights for policy 1, policy_version 1375525 (0.0005) [2023-12-27 01:23:36,609][105620] Updated weights for policy 1, policy_version 1375535 (0.0006) [2023-12-27 01:23:36,664][105620] Updated weights for policy 1, policy_version 1375545 (0.0006) [2023-12-27 01:23:37,242][105692] Updated weights for policy 0, policy_version 1373457 (0.0010) [2023-12-27 01:23:37,295][105692] Updated weights for policy 0, policy_version 1373467 (0.0009) [2023-12-27 01:23:37,304][105620] Updated weights for policy 1, policy_version 1375555 (0.0006) [2023-12-27 01:23:37,348][105692] Updated weights for policy 0, policy_version 1373477 (0.0010) [2023-12-27 01:23:37,363][105620] Updated weights for policy 1, policy_version 1375565 (0.0005) [2023-12-27 01:23:37,404][105692] Updated weights for policy 0, policy_version 1373487 (0.0009) [2023-12-27 01:23:37,411][105620] Updated weights for policy 1, policy_version 1375575 (0.0005) [2023-12-27 01:23:38,128][105620] Updated weights for policy 1, policy_version 1375585 (0.0010) [2023-12-27 01:23:38,184][105620] Updated weights for policy 1, policy_version 1375595 (0.0008) [2023-12-27 01:23:38,210][105692] Updated weights for policy 0, policy_version 1373497 (0.0008) [2023-12-27 01:23:38,235][105620] Updated weights for policy 1, policy_version 1375605 (0.0007) [2023-12-27 01:23:38,257][105692] Updated weights for policy 0, policy_version 1373507 (0.0008) [2023-12-27 01:23:38,287][105620] Updated weights for policy 1, policy_version 1375615 (0.0007) [2023-12-27 01:23:38,314][105692] Updated weights for policy 0, policy_version 1373517 (0.0006) [2023-12-27 01:23:39,070][105620] Updated weights for policy 1, policy_version 1375625 (0.0009) [2023-12-27 01:23:39,105][105692] Updated weights for policy 0, policy_version 1373527 (0.0006) [2023-12-27 01:23:39,130][105620] Updated weights for policy 1, policy_version 1375635 (0.0007) [2023-12-27 01:23:39,161][105692] Updated weights for policy 0, policy_version 1373537 (0.0006) [2023-12-27 01:23:39,190][105620] Updated weights for policy 1, policy_version 1375645 (0.0009) [2023-12-27 01:23:39,210][105692] Updated weights for policy 0, policy_version 1373547 (0.0007) [2023-12-27 01:23:39,943][105620] Updated weights for policy 1, policy_version 1375655 (0.0009) [2023-12-27 01:23:40,002][105620] Updated weights for policy 1, policy_version 1375665 (0.0009) [2023-12-27 01:23:40,034][105692] Updated weights for policy 0, policy_version 1373557 (0.0008) [2023-12-27 01:23:40,061][105620] Updated weights for policy 1, policy_version 1375675 (0.0009) [2023-12-27 01:23:40,095][105692] Updated weights for policy 0, policy_version 1373567 (0.0005) [2023-12-27 01:23:40,157][105692] Updated weights for policy 0, policy_version 1373577 (0.0009) [2023-12-27 01:23:40,829][105620] Updated weights for policy 1, policy_version 1375685 (0.0009) [2023-12-27 01:23:40,895][105620] Updated weights for policy 1, policy_version 1375695 (0.0009) [2023-12-27 01:23:40,909][105692] Updated weights for policy 0, policy_version 1373587 (0.0009) [2023-12-27 01:23:40,955][105692] Updated weights for policy 0, policy_version 1373597 (0.0008) [2023-12-27 01:23:40,957][105620] Updated weights for policy 1, policy_version 1375705 (0.0006) [2023-12-27 01:23:41,008][105692] Updated weights for policy 0, policy_version 1373607 (0.0006) [2023-12-27 01:23:41,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 703913984. Throughput: 0: 9543.7, 1: 9853.2. Samples: 703922712. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:23:41,063][104569] Avg episode reward: [(0, '7996.388'), (1, '9265.376')] [2023-12-27 01:23:41,731][105620] Updated weights for policy 1, policy_version 1375715 (0.0008) [2023-12-27 01:23:41,781][105692] Updated weights for policy 0, policy_version 1373617 (0.0009) [2023-12-27 01:23:41,796][105620] Updated weights for policy 1, policy_version 1375725 (0.0008) [2023-12-27 01:23:41,842][105692] Updated weights for policy 0, policy_version 1373627 (0.0006) [2023-12-27 01:23:41,858][105620] Updated weights for policy 1, policy_version 1375735 (0.0008) [2023-12-27 01:23:41,905][105692] Updated weights for policy 0, policy_version 1373637 (0.0010) [2023-12-27 01:23:41,967][105692] Updated weights for policy 0, policy_version 1373647 (0.0009) [2023-12-27 01:23:42,591][105620] Updated weights for policy 1, policy_version 1375745 (0.0008) [2023-12-27 01:23:42,653][105620] Updated weights for policy 1, policy_version 1375755 (0.0010) [2023-12-27 01:23:42,716][105620] Updated weights for policy 1, policy_version 1375765 (0.0010) [2023-12-27 01:23:42,750][105692] Updated weights for policy 0, policy_version 1373657 (0.0006) [2023-12-27 01:23:42,775][105620] Updated weights for policy 1, policy_version 1375775 (0.0010) [2023-12-27 01:23:42,815][105692] Updated weights for policy 0, policy_version 1373667 (0.0007) [2023-12-27 01:23:42,878][105692] Updated weights for policy 0, policy_version 1373677 (0.0008) [2023-12-27 01:23:43,440][105620] Updated weights for policy 1, policy_version 1375785 (0.0007) [2023-12-27 01:23:43,508][105620] Updated weights for policy 1, policy_version 1375795 (0.0010) [2023-12-27 01:23:43,577][105620] Updated weights for policy 1, policy_version 1375805 (0.0010) [2023-12-27 01:23:43,615][105692] Updated weights for policy 0, policy_version 1373687 (0.0008) [2023-12-27 01:23:43,668][105585] KL-divergence is very high: 112.0166 [2023-12-27 01:23:43,674][105692] Updated weights for policy 0, policy_version 1373697 (0.0008) [2023-12-27 01:23:43,711][105585] KL-divergence is very high: 114.7957 [2023-12-27 01:23:43,728][105692] Updated weights for policy 0, policy_version 1373707 (0.0009) [2023-12-27 01:23:44,283][105620] Updated weights for policy 1, policy_version 1375815 (0.0009) [2023-12-27 01:23:44,340][105620] Updated weights for policy 1, policy_version 1375825 (0.0010) [2023-12-27 01:23:44,391][105620] Updated weights for policy 1, policy_version 1375835 (0.0010) [2023-12-27 01:23:44,425][105692] Updated weights for policy 0, policy_version 1373717 (0.0008) [2023-12-27 01:23:44,483][105692] Updated weights for policy 0, policy_version 1373727 (0.0009) [2023-12-27 01:23:44,548][105692] Updated weights for policy 0, policy_version 1373737 (0.0007) [2023-12-27 01:23:45,130][105620] Updated weights for policy 1, policy_version 1375845 (0.0008) [2023-12-27 01:23:45,192][105620] Updated weights for policy 1, policy_version 1375855 (0.0010) [2023-12-27 01:23:45,260][105620] Updated weights for policy 1, policy_version 1375865 (0.0009) [2023-12-27 01:23:45,300][105692] Updated weights for policy 0, policy_version 1373747 (0.0008) [2023-12-27 01:23:45,362][105692] Updated weights for policy 0, policy_version 1373757 (0.0009) [2023-12-27 01:23:45,423][105692] Updated weights for policy 0, policy_version 1373767 (0.0008) [2023-12-27 01:23:46,031][105620] Updated weights for policy 1, policy_version 1375875 (0.0009) [2023-12-27 01:23:46,062][104569] Fps is (10 sec: 18022.0, 60 sec: 19251.1, 300 sec: 19410.9). Total num frames: 704004096. Throughput: 0: 9421.9, 1: 9822.5. Samples: 703978768. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:23:46,063][104569] Avg episode reward: [(0, '8351.839'), (1, '8993.195')] [2023-12-27 01:23:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001373776_351739904.pth... [2023-12-27 01:23:46,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001372688_351461376.pth [2023-12-27 01:23:46,097][105620] Updated weights for policy 1, policy_version 1375885 (0.0009) [2023-12-27 01:23:46,128][105692] Updated weights for policy 0, policy_version 1373777 (0.0009) [2023-12-27 01:23:46,159][105620] Updated weights for policy 1, policy_version 1375895 (0.0007) [2023-12-27 01:23:46,189][105692] Updated weights for policy 0, policy_version 1373787 (0.0009) [2023-12-27 01:23:46,211][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001375904_352272384.pth... [2023-12-27 01:23:46,215][105585] KL-divergence is very high: 237.1654 [2023-12-27 01:23:46,215][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001374752_351977472.pth [2023-12-27 01:23:46,249][105692] Updated weights for policy 0, policy_version 1373797 (0.0008) [2023-12-27 01:23:46,257][105585] KL-divergence is very high: 402.4454 [2023-12-27 01:23:46,296][105585] KL-divergence is very high: 418.1762 [2023-12-27 01:23:46,304][105692] Updated weights for policy 0, policy_version 1373808 (0.0010) [2023-12-27 01:23:46,819][105620] Updated weights for policy 1, policy_version 1375905 (0.0006) [2023-12-27 01:23:46,867][105620] Updated weights for policy 1, policy_version 1375915 (0.0005) [2023-12-27 01:23:46,916][105620] Updated weights for policy 1, policy_version 1375925 (0.0005) [2023-12-27 01:23:46,964][105620] Updated weights for policy 1, policy_version 1375935 (0.0005) [2023-12-27 01:23:47,167][105692] Updated weights for policy 0, policy_version 1373818 (0.0008) [2023-12-27 01:23:47,219][105692] Updated weights for policy 0, policy_version 1373828 (0.0009) [2023-12-27 01:23:47,266][105692] Updated weights for policy 0, policy_version 1373838 (0.0009) [2023-12-27 01:23:47,600][105620] Updated weights for policy 1, policy_version 1375945 (0.0009) [2023-12-27 01:23:47,653][105620] Updated weights for policy 1, policy_version 1375955 (0.0009) [2023-12-27 01:23:47,706][105620] Updated weights for policy 1, policy_version 1375965 (0.0009) [2023-12-27 01:23:47,932][105692] Updated weights for policy 0, policy_version 1373848 (0.0007) [2023-12-27 01:23:47,996][105692] Updated weights for policy 0, policy_version 1373858 (0.0006) [2023-12-27 01:23:48,057][105692] Updated weights for policy 0, policy_version 1373868 (0.0005) [2023-12-27 01:23:48,595][105692] Updated weights for policy 0, policy_version 1373878 (0.0008) [2023-12-27 01:23:48,598][105620] Updated weights for policy 1, policy_version 1375975 (0.0008) [2023-12-27 01:23:48,652][105620] Updated weights for policy 1, policy_version 1375985 (0.0007) [2023-12-27 01:23:48,657][105692] Updated weights for policy 0, policy_version 1373888 (0.0011) [2023-12-27 01:23:48,714][105620] Updated weights for policy 1, policy_version 1375995 (0.0007) [2023-12-27 01:23:48,715][105692] Updated weights for policy 0, policy_version 1373898 (0.0007) [2023-12-27 01:23:49,328][105692] Updated weights for policy 0, policy_version 1373908 (0.0010) [2023-12-27 01:23:49,396][105692] Updated weights for policy 0, policy_version 1373918 (0.0009) [2023-12-27 01:23:49,444][105692] Updated weights for policy 0, policy_version 1373928 (0.0009) [2023-12-27 01:23:49,574][105620] Updated weights for policy 1, policy_version 1376005 (0.0008) [2023-12-27 01:23:49,634][105620] Updated weights for policy 1, policy_version 1376015 (0.0009) [2023-12-27 01:23:49,684][105620] Updated weights for policy 1, policy_version 1376025 (0.0009) [2023-12-27 01:23:50,188][105692] Updated weights for policy 0, policy_version 1373938 (0.0009) [2023-12-27 01:23:50,241][105692] Updated weights for policy 0, policy_version 1373948 (0.0009) [2023-12-27 01:23:50,299][105692] Updated weights for policy 0, policy_version 1373958 (0.0008) [2023-12-27 01:23:50,358][105692] Updated weights for policy 0, policy_version 1373968 (0.0009) [2023-12-27 01:23:50,452][105620] Updated weights for policy 1, policy_version 1376035 (0.0009) [2023-12-27 01:23:50,523][105620] Updated weights for policy 1, policy_version 1376045 (0.0010) [2023-12-27 01:23:50,591][105620] Updated weights for policy 1, policy_version 1376055 (0.0008) [2023-12-27 01:23:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 704102400. Throughput: 0: 9490.8, 1: 9691.3. Samples: 704093428. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:23:51,062][104569] Avg episode reward: [(0, '8530.019'), (1, '8992.115')] [2023-12-27 01:23:51,122][105692] Updated weights for policy 0, policy_version 1373978 (0.0007) [2023-12-27 01:23:51,187][105692] Updated weights for policy 0, policy_version 1373988 (0.0007) [2023-12-27 01:23:51,238][105692] Updated weights for policy 0, policy_version 1373998 (0.0008) [2023-12-27 01:23:51,381][105620] Updated weights for policy 1, policy_version 1376065 (0.0009) [2023-12-27 01:23:51,440][105620] Updated weights for policy 1, policy_version 1376075 (0.0007) [2023-12-27 01:23:51,492][105620] Updated weights for policy 1, policy_version 1376085 (0.0010) [2023-12-27 01:23:51,558][105620] Updated weights for policy 1, policy_version 1376095 (0.0011) [2023-12-27 01:23:51,937][105692] Updated weights for policy 0, policy_version 1374008 (0.0008) [2023-12-27 01:23:51,997][105692] Updated weights for policy 0, policy_version 1374018 (0.0007) [2023-12-27 01:23:52,059][105692] Updated weights for policy 0, policy_version 1374028 (0.0005) [2023-12-27 01:23:52,296][105620] Updated weights for policy 1, policy_version 1376105 (0.0011) [2023-12-27 01:23:52,367][105620] Updated weights for policy 1, policy_version 1376115 (0.0011) [2023-12-27 01:23:52,432][105620] Updated weights for policy 1, policy_version 1376125 (0.0011) [2023-12-27 01:23:52,656][105692] Updated weights for policy 0, policy_version 1374038 (0.0006) [2023-12-27 01:23:52,712][105692] Updated weights for policy 0, policy_version 1374048 (0.0008) [2023-12-27 01:23:52,760][105692] Updated weights for policy 0, policy_version 1374058 (0.0008) [2023-12-27 01:23:53,171][105620] Updated weights for policy 1, policy_version 1376135 (0.0010) [2023-12-27 01:23:53,222][105620] Updated weights for policy 1, policy_version 1376145 (0.0009) [2023-12-27 01:23:53,285][105620] Updated weights for policy 1, policy_version 1376155 (0.0009) [2023-12-27 01:23:53,504][105692] Updated weights for policy 0, policy_version 1374068 (0.0009) [2023-12-27 01:23:53,565][105692] Updated weights for policy 0, policy_version 1374078 (0.0009) [2023-12-27 01:23:53,623][105692] Updated weights for policy 0, policy_version 1374088 (0.0009) [2023-12-27 01:23:54,047][105620] Updated weights for policy 1, policy_version 1376165 (0.0009) [2023-12-27 01:23:54,104][105620] Updated weights for policy 1, policy_version 1376175 (0.0006) [2023-12-27 01:23:54,159][105620] Updated weights for policy 1, policy_version 1376185 (0.0005) [2023-12-27 01:23:54,393][105692] Updated weights for policy 0, policy_version 1374098 (0.0009) [2023-12-27 01:23:54,455][105692] Updated weights for policy 0, policy_version 1374108 (0.0010) [2023-12-27 01:23:54,479][105585] KL-divergence is very high: 125.4119 [2023-12-27 01:23:54,513][105692] Updated weights for policy 0, policy_version 1374118 (0.0010) [2023-12-27 01:23:54,523][105585] KL-divergence is very high: 131.0080 [2023-12-27 01:23:54,572][105692] Updated weights for policy 0, policy_version 1374128 (0.0009) [2023-12-27 01:23:54,749][105620] Updated weights for policy 1, policy_version 1376195 (0.0007) [2023-12-27 01:23:54,813][105620] Updated weights for policy 1, policy_version 1376205 (0.0005) [2023-12-27 01:23:54,878][105620] Updated weights for policy 1, policy_version 1376215 (0.0005) [2023-12-27 01:23:55,297][105692] Updated weights for policy 0, policy_version 1374138 (0.0009) [2023-12-27 01:23:55,362][105692] Updated weights for policy 0, policy_version 1374148 (0.0009) [2023-12-27 01:23:55,420][105692] Updated weights for policy 0, policy_version 1374158 (0.0009) [2023-12-27 01:23:55,477][105620] Updated weights for policy 1, policy_version 1376225 (0.0006) [2023-12-27 01:23:55,541][105620] Updated weights for policy 1, policy_version 1376235 (0.0009) [2023-12-27 01:23:55,602][105620] Updated weights for policy 1, policy_version 1376245 (0.0008) [2023-12-27 01:23:55,661][105620] Updated weights for policy 1, policy_version 1376255 (0.0009) [2023-12-27 01:23:56,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 704200704. Throughput: 0: 9585.2, 1: 9627.3. Samples: 704209864. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:23:56,062][104569] Avg episode reward: [(0, '8352.331'), (1, '9173.395')] [2023-12-27 01:23:56,147][105692] Updated weights for policy 0, policy_version 1374168 (0.0009) [2023-12-27 01:23:56,204][105692] Updated weights for policy 0, policy_version 1374178 (0.0009) [2023-12-27 01:23:56,264][105692] Updated weights for policy 0, policy_version 1374188 (0.0009) [2023-12-27 01:23:56,401][105620] Updated weights for policy 1, policy_version 1376265 (0.0009) [2023-12-27 01:23:56,453][105620] Updated weights for policy 1, policy_version 1376275 (0.0009) [2023-12-27 01:23:56,514][105620] Updated weights for policy 1, policy_version 1376285 (0.0009) [2023-12-27 01:23:56,983][105692] Updated weights for policy 0, policy_version 1374198 (0.0008) [2023-12-27 01:23:57,027][105692] Updated weights for policy 0, policy_version 1374208 (0.0007) [2023-12-27 01:23:57,079][105692] Updated weights for policy 0, policy_version 1374218 (0.0005) [2023-12-27 01:23:57,251][105620] Updated weights for policy 1, policy_version 1376295 (0.0008) [2023-12-27 01:23:57,323][105620] Updated weights for policy 1, policy_version 1376305 (0.0008) [2023-12-27 01:23:57,390][105620] Updated weights for policy 1, policy_version 1376315 (0.0008) [2023-12-27 01:23:57,663][105692] Updated weights for policy 0, policy_version 1374228 (0.0007) [2023-12-27 01:23:57,721][105692] Updated weights for policy 0, policy_version 1374238 (0.0009) [2023-12-27 01:23:57,775][105692] Updated weights for policy 0, policy_version 1374250 (0.0010) [2023-12-27 01:23:57,924][105620] Updated weights for policy 1, policy_version 1376325 (0.0006) [2023-12-27 01:23:57,969][105620] Updated weights for policy 1, policy_version 1376335 (0.0005) [2023-12-27 01:23:58,019][105620] Updated weights for policy 1, policy_version 1376345 (0.0007) [2023-12-27 01:23:58,576][105692] Updated weights for policy 0, policy_version 1374261 (0.0010) [2023-12-27 01:23:58,633][105692] Updated weights for policy 0, policy_version 1374271 (0.0010) [2023-12-27 01:23:58,693][105692] Updated weights for policy 0, policy_version 1374281 (0.0011) [2023-12-27 01:23:58,782][105620] Updated weights for policy 1, policy_version 1376355 (0.0008) [2023-12-27 01:23:58,845][105620] Updated weights for policy 1, policy_version 1376365 (0.0008) [2023-12-27 01:23:58,907][105620] Updated weights for policy 1, policy_version 1376375 (0.0008) [2023-12-27 01:23:59,386][105692] Updated weights for policy 0, policy_version 1374291 (0.0011) [2023-12-27 01:23:59,453][105692] Updated weights for policy 0, policy_version 1374301 (0.0007) [2023-12-27 01:23:59,483][105620] Updated weights for policy 1, policy_version 1376385 (0.0006) [2023-12-27 01:23:59,508][105692] Updated weights for policy 0, policy_version 1374311 (0.0010) [2023-12-27 01:23:59,538][105620] Updated weights for policy 1, policy_version 1376395 (0.0006) [2023-12-27 01:23:59,604][105620] Updated weights for policy 1, policy_version 1376405 (0.0007) [2023-12-27 01:23:59,658][105620] Updated weights for policy 1, policy_version 1376415 (0.0008) [2023-12-27 01:24:00,222][105692] Updated weights for policy 0, policy_version 1374321 (0.0010) [2023-12-27 01:24:00,276][105692] Updated weights for policy 0, policy_version 1374331 (0.0010) [2023-12-27 01:24:00,316][105620] Updated weights for policy 1, policy_version 1376425 (0.0006) [2023-12-27 01:24:00,335][105692] Updated weights for policy 0, policy_version 1374341 (0.0010) [2023-12-27 01:24:00,376][105620] Updated weights for policy 1, policy_version 1376435 (0.0005) [2023-12-27 01:24:00,391][105692] Updated weights for policy 0, policy_version 1374351 (0.0007) [2023-12-27 01:24:00,435][105620] Updated weights for policy 1, policy_version 1376445 (0.0006) [2023-12-27 01:24:00,975][105692] Updated weights for policy 0, policy_version 1374361 (0.0005) [2023-12-27 01:24:01,019][105620] Updated weights for policy 1, policy_version 1376455 (0.0008) [2023-12-27 01:24:01,033][105692] Updated weights for policy 0, policy_version 1374371 (0.0006) [2023-12-27 01:24:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.3, 300 sec: 19410.9). Total num frames: 704299008. Throughput: 0: 9605.4, 1: 9688.2. Samples: 704270376. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:24:01,062][104569] Avg episode reward: [(0, '8622.715'), (1, '9173.991')] [2023-12-27 01:24:01,084][105620] Updated weights for policy 1, policy_version 1376465 (0.0008) [2023-12-27 01:24:01,094][105692] Updated weights for policy 0, policy_version 1374381 (0.0007) [2023-12-27 01:24:01,112][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001374384_351895552.pth... [2023-12-27 01:24:01,116][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001373264_351608832.pth [2023-12-27 01:24:01,145][105620] Updated weights for policy 1, policy_version 1376475 (0.0009) [2023-12-27 01:24:01,174][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001376480_352419840.pth... [2023-12-27 01:24:01,179][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001375328_352124928.pth [2023-12-27 01:24:01,805][105692] Updated weights for policy 0, policy_version 1374391 (0.0008) [2023-12-27 01:24:01,869][105692] Updated weights for policy 0, policy_version 1374401 (0.0009) [2023-12-27 01:24:01,913][105620] Updated weights for policy 1, policy_version 1376485 (0.0010) [2023-12-27 01:24:01,929][105692] Updated weights for policy 0, policy_version 1374411 (0.0008) [2023-12-27 01:24:01,974][105620] Updated weights for policy 1, policy_version 1376495 (0.0008) [2023-12-27 01:24:02,032][105620] Updated weights for policy 1, policy_version 1376505 (0.0009) [2023-12-27 01:24:02,706][105620] Updated weights for policy 1, policy_version 1376515 (0.0008) [2023-12-27 01:24:02,712][105692] Updated weights for policy 0, policy_version 1374421 (0.0010) [2023-12-27 01:24:02,762][105620] Updated weights for policy 1, policy_version 1376525 (0.0005) [2023-12-27 01:24:02,771][105692] Updated weights for policy 0, policy_version 1374431 (0.0009) [2023-12-27 01:24:02,820][105620] Updated weights for policy 1, policy_version 1376535 (0.0006) [2023-12-27 01:24:02,835][105692] Updated weights for policy 0, policy_version 1374441 (0.0009) [2023-12-27 01:24:03,368][105620] Updated weights for policy 1, policy_version 1376545 (0.0006) [2023-12-27 01:24:03,419][105620] Updated weights for policy 1, policy_version 1376555 (0.0010) [2023-12-27 01:24:03,464][105620] Updated weights for policy 1, policy_version 1376565 (0.0010) [2023-12-27 01:24:03,511][105620] Updated weights for policy 1, policy_version 1376575 (0.0009) [2023-12-27 01:24:03,544][105692] Updated weights for policy 0, policy_version 1374451 (0.0009) [2023-12-27 01:24:03,593][105692] Updated weights for policy 0, policy_version 1374461 (0.0007) [2023-12-27 01:24:03,641][105692] Updated weights for policy 0, policy_version 1374471 (0.0008) [2023-12-27 01:24:04,284][105620] Updated weights for policy 1, policy_version 1376585 (0.0009) [2023-12-27 01:24:04,335][105692] Updated weights for policy 0, policy_version 1374481 (0.0008) [2023-12-27 01:24:04,346][105620] Updated weights for policy 1, policy_version 1376595 (0.0008) [2023-12-27 01:24:04,398][105692] Updated weights for policy 0, policy_version 1374491 (0.0008) [2023-12-27 01:24:04,406][105620] Updated weights for policy 1, policy_version 1376605 (0.0008) [2023-12-27 01:24:04,457][105692] Updated weights for policy 0, policy_version 1374501 (0.0008) [2023-12-27 01:24:04,516][105692] Updated weights for policy 0, policy_version 1374511 (0.0009) [2023-12-27 01:24:05,085][105620] Updated weights for policy 1, policy_version 1376615 (0.0008) [2023-12-27 01:24:05,142][105620] Updated weights for policy 1, policy_version 1376625 (0.0009) [2023-12-27 01:24:05,199][105620] Updated weights for policy 1, policy_version 1376635 (0.0009) [2023-12-27 01:24:05,363][105692] Updated weights for policy 0, policy_version 1374521 (0.0009) [2023-12-27 01:24:05,434][105692] Updated weights for policy 0, policy_version 1374531 (0.0010) [2023-12-27 01:24:05,495][105692] Updated weights for policy 0, policy_version 1374541 (0.0008) [2023-12-27 01:24:05,861][105620] Updated weights for policy 1, policy_version 1376645 (0.0007) [2023-12-27 01:24:05,919][105620] Updated weights for policy 1, policy_version 1376655 (0.0009) [2023-12-27 01:24:05,972][105620] Updated weights for policy 1, policy_version 1376665 (0.0009) [2023-12-27 01:24:06,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 704405504. Throughput: 0: 9507.8, 1: 9795.3. Samples: 704392636. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:24:06,062][104569] Avg episode reward: [(0, '8164.351'), (1, '8988.981')] [2023-12-27 01:24:06,268][105692] Updated weights for policy 0, policy_version 1374551 (0.0009) [2023-12-27 01:24:06,322][105692] Updated weights for policy 0, policy_version 1374561 (0.0010) [2023-12-27 01:24:06,378][105692] Updated weights for policy 0, policy_version 1374571 (0.0009) [2023-12-27 01:24:06,655][105620] Updated weights for policy 1, policy_version 1376675 (0.0009) [2023-12-27 01:24:06,714][105620] Updated weights for policy 1, policy_version 1376685 (0.0009) [2023-12-27 01:24:06,778][105620] Updated weights for policy 1, policy_version 1376695 (0.0009) [2023-12-27 01:24:07,288][105692] Updated weights for policy 0, policy_version 1374581 (0.0009) [2023-12-27 01:24:07,349][105692] Updated weights for policy 0, policy_version 1374591 (0.0010) [2023-12-27 01:24:07,370][105620] Updated weights for policy 1, policy_version 1376705 (0.0008) [2023-12-27 01:24:07,414][105692] Updated weights for policy 0, policy_version 1374601 (0.0008) [2023-12-27 01:24:07,420][105620] Updated weights for policy 1, policy_version 1376715 (0.0005) [2023-12-27 01:24:07,470][105620] Updated weights for policy 1, policy_version 1376725 (0.0005) [2023-12-27 01:24:07,526][105620] Updated weights for policy 1, policy_version 1376735 (0.0008) [2023-12-27 01:24:08,140][105620] Updated weights for policy 1, policy_version 1376745 (0.0009) [2023-12-27 01:24:08,196][105620] Updated weights for policy 1, policy_version 1376755 (0.0009) [2023-12-27 01:24:08,236][105692] Updated weights for policy 0, policy_version 1374611 (0.0009) [2023-12-27 01:24:08,250][105620] Updated weights for policy 1, policy_version 1376765 (0.0007) [2023-12-27 01:24:08,291][105692] Updated weights for policy 0, policy_version 1374621 (0.0008) [2023-12-27 01:24:08,356][105692] Updated weights for policy 0, policy_version 1374631 (0.0009) [2023-12-27 01:24:09,054][105692] Updated weights for policy 0, policy_version 1374641 (0.0006) [2023-12-27 01:24:09,089][105620] Updated weights for policy 1, policy_version 1376775 (0.0006) [2023-12-27 01:24:09,113][105692] Updated weights for policy 0, policy_version 1374651 (0.0008) [2023-12-27 01:24:09,135][105620] Updated weights for policy 1, policy_version 1376785 (0.0005) [2023-12-27 01:24:09,161][105692] Updated weights for policy 0, policy_version 1374661 (0.0009) [2023-12-27 01:24:09,179][105620] Updated weights for policy 1, policy_version 1376795 (0.0005) [2023-12-27 01:24:09,213][105692] Updated weights for policy 0, policy_version 1374671 (0.0008) [2023-12-27 01:24:09,898][105620] Updated weights for policy 1, policy_version 1376805 (0.0007) [2023-12-27 01:24:09,963][105620] Updated weights for policy 1, policy_version 1376815 (0.0008) [2023-12-27 01:24:10,023][105692] Updated weights for policy 0, policy_version 1374681 (0.0006) [2023-12-27 01:24:10,025][105620] Updated weights for policy 1, policy_version 1376825 (0.0008) [2023-12-27 01:24:10,080][105692] Updated weights for policy 0, policy_version 1374691 (0.0008) [2023-12-27 01:24:10,145][105692] Updated weights for policy 0, policy_version 1374701 (0.0009) [2023-12-27 01:24:10,739][105620] Updated weights for policy 1, policy_version 1376835 (0.0008) [2023-12-27 01:24:10,801][105620] Updated weights for policy 1, policy_version 1376845 (0.0009) [2023-12-27 01:24:10,860][105620] Updated weights for policy 1, policy_version 1376855 (0.0008) [2023-12-27 01:24:10,882][105692] Updated weights for policy 0, policy_version 1374712 (0.0009) [2023-12-27 01:24:10,932][105692] Updated weights for policy 0, policy_version 1374722 (0.0007) [2023-12-27 01:24:10,995][105692] Updated weights for policy 0, policy_version 1374732 (0.0009) [2023-12-27 01:24:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 704503808. Throughput: 0: 9402.9, 1: 9910.9. Samples: 704504960. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:24:11,062][104569] Avg episode reward: [(0, '7892.588'), (1, '8896.742')] [2023-12-27 01:24:11,604][105620] Updated weights for policy 1, policy_version 1376865 (0.0007) [2023-12-27 01:24:11,668][105620] Updated weights for policy 1, policy_version 1376875 (0.0009) [2023-12-27 01:24:11,740][105620] Updated weights for policy 1, policy_version 1376885 (0.0010) [2023-12-27 01:24:11,804][105620] Updated weights for policy 1, policy_version 1376895 (0.0008) [2023-12-27 01:24:11,830][105692] Updated weights for policy 0, policy_version 1374742 (0.0007) [2023-12-27 01:24:11,888][105692] Updated weights for policy 0, policy_version 1374752 (0.0006) [2023-12-27 01:24:11,951][105692] Updated weights for policy 0, policy_version 1374762 (0.0006) [2023-12-27 01:24:12,624][105620] Updated weights for policy 1, policy_version 1376905 (0.0007) [2023-12-27 01:24:12,638][105692] Updated weights for policy 0, policy_version 1374772 (0.0009) [2023-12-27 01:24:12,685][105620] Updated weights for policy 1, policy_version 1376915 (0.0007) [2023-12-27 01:24:12,688][105692] Updated weights for policy 0, policy_version 1374782 (0.0006) [2023-12-27 01:24:12,737][105692] Updated weights for policy 0, policy_version 1374792 (0.0006) [2023-12-27 01:24:12,742][105620] Updated weights for policy 1, policy_version 1376925 (0.0008) [2023-12-27 01:24:13,353][105620] Updated weights for policy 1, policy_version 1376935 (0.0009) [2023-12-27 01:24:13,415][105620] Updated weights for policy 1, policy_version 1376945 (0.0009) [2023-12-27 01:24:13,473][105620] Updated weights for policy 1, policy_version 1376955 (0.0009) [2023-12-27 01:24:13,518][105692] Updated weights for policy 0, policy_version 1374802 (0.0008) [2023-12-27 01:24:13,574][105692] Updated weights for policy 0, policy_version 1374812 (0.0010) [2023-12-27 01:24:13,629][105692] Updated weights for policy 0, policy_version 1374823 (0.0010) [2023-12-27 01:24:14,044][105620] Updated weights for policy 1, policy_version 1376965 (0.0007) [2023-12-27 01:24:14,111][105620] Updated weights for policy 1, policy_version 1376975 (0.0005) [2023-12-27 01:24:14,167][105620] Updated weights for policy 1, policy_version 1376985 (0.0005) [2023-12-27 01:24:14,499][105692] Updated weights for policy 0, policy_version 1374833 (0.0010) [2023-12-27 01:24:14,564][105692] Updated weights for policy 0, policy_version 1374843 (0.0009) [2023-12-27 01:24:14,633][105692] Updated weights for policy 0, policy_version 1374853 (0.0010) [2023-12-27 01:24:14,699][105692] Updated weights for policy 0, policy_version 1374863 (0.0009) [2023-12-27 01:24:14,779][105620] Updated weights for policy 1, policy_version 1376995 (0.0006) [2023-12-27 01:24:14,834][105620] Updated weights for policy 1, policy_version 1377005 (0.0006) [2023-12-27 01:24:14,889][105620] Updated weights for policy 1, policy_version 1377015 (0.0005) [2023-12-27 01:24:15,463][105692] Updated weights for policy 0, policy_version 1374873 (0.0009) [2023-12-27 01:24:15,524][105692] Updated weights for policy 0, policy_version 1374883 (0.0009) [2023-12-27 01:24:15,572][105692] Updated weights for policy 0, policy_version 1374893 (0.0009) [2023-12-27 01:24:15,608][105620] Updated weights for policy 1, policy_version 1377025 (0.0008) [2023-12-27 01:24:15,662][105620] Updated weights for policy 1, policy_version 1377035 (0.0008) [2023-12-27 01:24:15,710][105620] Updated weights for policy 1, policy_version 1377045 (0.0008) [2023-12-27 01:24:15,774][105620] Updated weights for policy 1, policy_version 1377055 (0.0009) [2023-12-27 01:24:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 704593920. Throughput: 0: 9399.7, 1: 9882.1. Samples: 704563648. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:24:16,062][104569] Avg episode reward: [(0, '8166.265'), (1, '8990.733')] [2023-12-27 01:24:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001374896_352026624.pth... [2023-12-27 01:24:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001377056_352567296.pth... [2023-12-27 01:24:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001373776_351739904.pth [2023-12-27 01:24:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001375904_352272384.pth [2023-12-27 01:24:16,344][105692] Updated weights for policy 0, policy_version 1374903 (0.0009) [2023-12-27 01:24:16,395][105692] Updated weights for policy 0, policy_version 1374913 (0.0009) [2023-12-27 01:24:16,441][105692] Updated weights for policy 0, policy_version 1374923 (0.0009) [2023-12-27 01:24:16,477][105620] Updated weights for policy 1, policy_version 1377065 (0.0007) [2023-12-27 01:24:16,534][105620] Updated weights for policy 1, policy_version 1377075 (0.0009) [2023-12-27 01:24:16,598][105620] Updated weights for policy 1, policy_version 1377085 (0.0009) [2023-12-27 01:24:17,245][105620] Updated weights for policy 1, policy_version 1377095 (0.0008) [2023-12-27 01:24:17,277][105692] Updated weights for policy 0, policy_version 1374933 (0.0007) [2023-12-27 01:24:17,292][105620] Updated weights for policy 1, policy_version 1377105 (0.0009) [2023-12-27 01:24:17,334][105692] Updated weights for policy 0, policy_version 1374943 (0.0008) [2023-12-27 01:24:17,340][105620] Updated weights for policy 1, policy_version 1377115 (0.0006) [2023-12-27 01:24:17,392][105692] Updated weights for policy 0, policy_version 1374953 (0.0007) [2023-12-27 01:24:18,078][105692] Updated weights for policy 0, policy_version 1374963 (0.0009) [2023-12-27 01:24:18,124][105620] Updated weights for policy 1, policy_version 1377125 (0.0007) [2023-12-27 01:24:18,138][105692] Updated weights for policy 0, policy_version 1374973 (0.0007) [2023-12-27 01:24:18,181][105620] Updated weights for policy 1, policy_version 1377135 (0.0007) [2023-12-27 01:24:18,194][105692] Updated weights for policy 0, policy_version 1374983 (0.0009) [2023-12-27 01:24:18,233][105620] Updated weights for policy 1, policy_version 1377145 (0.0006) [2023-12-27 01:24:18,975][105692] Updated weights for policy 0, policy_version 1374993 (0.0007) [2023-12-27 01:24:19,016][105620] Updated weights for policy 1, policy_version 1377155 (0.0007) [2023-12-27 01:24:19,035][105692] Updated weights for policy 0, policy_version 1375003 (0.0008) [2023-12-27 01:24:19,069][105620] Updated weights for policy 1, policy_version 1377165 (0.0007) [2023-12-27 01:24:19,094][105692] Updated weights for policy 0, policy_version 1375013 (0.0008) [2023-12-27 01:24:19,128][105620] Updated weights for policy 1, policy_version 1377175 (0.0008) [2023-12-27 01:24:19,142][105692] Updated weights for policy 0, policy_version 1375023 (0.0006) [2023-12-27 01:24:19,805][105620] Updated weights for policy 1, policy_version 1377185 (0.0008) [2023-12-27 01:24:19,871][105620] Updated weights for policy 1, policy_version 1377195 (0.0009) [2023-12-27 01:24:19,932][105620] Updated weights for policy 1, policy_version 1377205 (0.0009) [2023-12-27 01:24:19,980][105692] Updated weights for policy 0, policy_version 1375033 (0.0008) [2023-12-27 01:24:19,990][105620] Updated weights for policy 1, policy_version 1377215 (0.0007) [2023-12-27 01:24:20,038][105692] Updated weights for policy 0, policy_version 1375043 (0.0008) [2023-12-27 01:24:20,103][105692] Updated weights for policy 0, policy_version 1375053 (0.0009) [2023-12-27 01:24:20,709][105620] Updated weights for policy 1, policy_version 1377225 (0.0006) [2023-12-27 01:24:20,769][105620] Updated weights for policy 1, policy_version 1377235 (0.0005) [2023-12-27 01:24:20,833][105620] Updated weights for policy 1, policy_version 1377245 (0.0005) [2023-12-27 01:24:20,953][105692] Updated weights for policy 0, policy_version 1375063 (0.0009) [2023-12-27 01:24:21,015][105692] Updated weights for policy 0, policy_version 1375073 (0.0009) [2023-12-27 01:24:21,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19114.7, 300 sec: 19438.6). Total num frames: 704684032. Throughput: 0: 9364.9, 1: 9874.7. Samples: 704677172. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:24:21,063][104569] Avg episode reward: [(0, '8442.607'), (1, '8989.574')] [2023-12-27 01:24:21,083][105692] Updated weights for policy 0, policy_version 1375083 (0.0008) [2023-12-27 01:24:21,469][105620] Updated weights for policy 1, policy_version 1377255 (0.0009) [2023-12-27 01:24:21,529][105620] Updated weights for policy 1, policy_version 1377265 (0.0010) [2023-12-27 01:24:21,592][105620] Updated weights for policy 1, policy_version 1377276 (0.0009) [2023-12-27 01:24:21,828][105692] Updated weights for policy 0, policy_version 1375093 (0.0009) [2023-12-27 01:24:21,884][105692] Updated weights for policy 0, policy_version 1375103 (0.0009) [2023-12-27 01:24:21,938][105692] Updated weights for policy 0, policy_version 1375113 (0.0007) [2023-12-27 01:24:22,340][105620] Updated weights for policy 1, policy_version 1377286 (0.0009) [2023-12-27 01:24:22,407][105620] Updated weights for policy 1, policy_version 1377296 (0.0008) [2023-12-27 01:24:22,460][105620] Updated weights for policy 1, policy_version 1377306 (0.0010) [2023-12-27 01:24:22,693][105692] Updated weights for policy 0, policy_version 1375123 (0.0009) [2023-12-27 01:24:22,753][105692] Updated weights for policy 0, policy_version 1375133 (0.0011) [2023-12-27 01:24:22,813][105692] Updated weights for policy 0, policy_version 1375143 (0.0008) [2023-12-27 01:24:23,237][105620] Updated weights for policy 1, policy_version 1377316 (0.0008) [2023-12-27 01:24:23,295][105620] Updated weights for policy 1, policy_version 1377326 (0.0009) [2023-12-27 01:24:23,363][105620] Updated weights for policy 1, policy_version 1377336 (0.0011) [2023-12-27 01:24:23,408][105692] Updated weights for policy 0, policy_version 1375153 (0.0006) [2023-12-27 01:24:23,472][105692] Updated weights for policy 0, policy_version 1375163 (0.0008) [2023-12-27 01:24:23,529][105692] Updated weights for policy 0, policy_version 1375173 (0.0008) [2023-12-27 01:24:23,588][105692] Updated weights for policy 0, policy_version 1375183 (0.0005) [2023-12-27 01:24:24,085][105620] Updated weights for policy 1, policy_version 1377346 (0.0010) [2023-12-27 01:24:24,139][105620] Updated weights for policy 1, policy_version 1377356 (0.0010) [2023-12-27 01:24:24,184][105620] Updated weights for policy 1, policy_version 1377366 (0.0010) [2023-12-27 01:24:24,235][105620] Updated weights for policy 1, policy_version 1377376 (0.0010) [2023-12-27 01:24:24,254][105692] Updated weights for policy 0, policy_version 1375193 (0.0010) [2023-12-27 01:24:24,298][105692] Updated weights for policy 0, policy_version 1375203 (0.0010) [2023-12-27 01:24:24,350][105692] Updated weights for policy 0, policy_version 1375213 (0.0011) [2023-12-27 01:24:24,923][105620] Updated weights for policy 1, policy_version 1377386 (0.0010) [2023-12-27 01:24:24,978][105620] Updated weights for policy 1, policy_version 1377396 (0.0009) [2023-12-27 01:24:24,983][105692] Updated weights for policy 0, policy_version 1375223 (0.0007) [2023-12-27 01:24:25,024][105620] Updated weights for policy 1, policy_version 1377406 (0.0005) [2023-12-27 01:24:25,038][105692] Updated weights for policy 0, policy_version 1375233 (0.0006) [2023-12-27 01:24:25,095][105692] Updated weights for policy 0, policy_version 1375243 (0.0005) [2023-12-27 01:24:25,571][105620] Updated weights for policy 1, policy_version 1377416 (0.0005) [2023-12-27 01:24:25,619][105620] Updated weights for policy 1, policy_version 1377426 (0.0005) [2023-12-27 01:24:25,678][105620] Updated weights for policy 1, policy_version 1377436 (0.0005) [2023-12-27 01:24:25,773][105692] Updated weights for policy 0, policy_version 1375253 (0.0008) [2023-12-27 01:24:25,837][105692] Updated weights for policy 0, policy_version 1375263 (0.0010) [2023-12-27 01:24:25,898][105692] Updated weights for policy 0, policy_version 1375273 (0.0009) [2023-12-27 01:24:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 704790528. Throughput: 0: 9506.8, 1: 9905.0. Samples: 704796240. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 01:24:26,063][104569] Avg episode reward: [(0, '8539.685'), (1, '8802.769')] [2023-12-27 01:24:26,374][105620] Updated weights for policy 1, policy_version 1377446 (0.0007) [2023-12-27 01:24:26,430][105620] Updated weights for policy 1, policy_version 1377456 (0.0006) [2023-12-27 01:24:26,492][105620] Updated weights for policy 1, policy_version 1377466 (0.0006) [2023-12-27 01:24:26,634][105692] Updated weights for policy 0, policy_version 1375283 (0.0010) [2023-12-27 01:24:26,685][105692] Updated weights for policy 0, policy_version 1375293 (0.0010) [2023-12-27 01:24:26,738][105692] Updated weights for policy 0, policy_version 1375303 (0.0010) [2023-12-27 01:24:27,089][105620] Updated weights for policy 1, policy_version 1377476 (0.0005) [2023-12-27 01:24:27,141][105620] Updated weights for policy 1, policy_version 1377486 (0.0005) [2023-12-27 01:24:27,201][105620] Updated weights for policy 1, policy_version 1377496 (0.0006) [2023-12-27 01:24:27,363][105692] Updated weights for policy 0, policy_version 1375313 (0.0010) [2023-12-27 01:24:27,414][105692] Updated weights for policy 0, policy_version 1375323 (0.0005) [2023-12-27 01:24:27,463][105692] Updated weights for policy 0, policy_version 1375333 (0.0008) [2023-12-27 01:24:27,524][105692] Updated weights for policy 0, policy_version 1375343 (0.0010) [2023-12-27 01:24:27,789][105620] Updated weights for policy 1, policy_version 1377506 (0.0005) [2023-12-27 01:24:27,842][105620] Updated weights for policy 1, policy_version 1377516 (0.0005) [2023-12-27 01:24:27,903][105620] Updated weights for policy 1, policy_version 1377526 (0.0005) [2023-12-27 01:24:27,970][105620] Updated weights for policy 1, policy_version 1377536 (0.0007) [2023-12-27 01:24:28,226][105692] Updated weights for policy 0, policy_version 1375353 (0.0010) [2023-12-27 01:24:28,279][105692] Updated weights for policy 0, policy_version 1375363 (0.0010) [2023-12-27 01:24:28,339][105692] Updated weights for policy 0, policy_version 1375373 (0.0010) [2023-12-27 01:24:28,663][105620] Updated weights for policy 1, policy_version 1377546 (0.0008) [2023-12-27 01:24:28,718][105620] Updated weights for policy 1, policy_version 1377556 (0.0008) [2023-12-27 01:24:28,769][105620] Updated weights for policy 1, policy_version 1377566 (0.0008) [2023-12-27 01:24:29,080][105692] Updated weights for policy 0, policy_version 1375383 (0.0011) [2023-12-27 01:24:29,148][105692] Updated weights for policy 0, policy_version 1375393 (0.0010) [2023-12-27 01:24:29,199][105692] Updated weights for policy 0, policy_version 1375403 (0.0010) [2023-12-27 01:24:29,469][105620] Updated weights for policy 1, policy_version 1377576 (0.0006) [2023-12-27 01:24:29,526][105620] Updated weights for policy 1, policy_version 1377586 (0.0006) [2023-12-27 01:24:29,583][105620] Updated weights for policy 1, policy_version 1377596 (0.0005) [2023-12-27 01:24:29,902][105692] Updated weights for policy 0, policy_version 1375413 (0.0011) [2023-12-27 01:24:29,969][105692] Updated weights for policy 0, policy_version 1375423 (0.0010) [2023-12-27 01:24:30,031][105692] Updated weights for policy 0, policy_version 1375433 (0.0008) [2023-12-27 01:24:30,131][105620] Updated weights for policy 1, policy_version 1377606 (0.0006) [2023-12-27 01:24:30,196][105620] Updated weights for policy 1, policy_version 1377616 (0.0008) [2023-12-27 01:24:30,240][105620] Updated weights for policy 1, policy_version 1377626 (0.0008) [2023-12-27 01:24:30,753][105692] Updated weights for policy 0, policy_version 1375443 (0.0010) [2023-12-27 01:24:30,804][105692] Updated weights for policy 0, policy_version 1375453 (0.0010) [2023-12-27 01:24:30,851][105692] Updated weights for policy 0, policy_version 1375463 (0.0010) [2023-12-27 01:24:30,949][105620] Updated weights for policy 1, policy_version 1377636 (0.0009) [2023-12-27 01:24:30,996][105620] Updated weights for policy 1, policy_version 1377646 (0.0007) [2023-12-27 01:24:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 704888832. Throughput: 0: 9562.9, 1: 9962.6. Samples: 704857408. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:24:31,062][104569] Avg episode reward: [(0, '8630.035'), (1, '8987.132')] [2023-12-27 01:24:31,064][105620] Updated weights for policy 1, policy_version 1377656 (0.0009) [2023-12-27 01:24:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001375472_352174080.pth... [2023-12-27 01:24:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001374384_351895552.pth [2023-12-27 01:24:31,118][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001377664_352722944.pth... [2023-12-27 01:24:31,123][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001376480_352419840.pth [2023-12-27 01:24:31,618][105692] Updated weights for policy 0, policy_version 1375473 (0.0011) [2023-12-27 01:24:31,680][105692] Updated weights for policy 0, policy_version 1375483 (0.0010) [2023-12-27 01:24:31,750][105692] Updated weights for policy 0, policy_version 1375493 (0.0010) [2023-12-27 01:24:31,805][105692] Updated weights for policy 0, policy_version 1375503 (0.0009) [2023-12-27 01:24:31,814][105620] Updated weights for policy 1, policy_version 1377666 (0.0008) [2023-12-27 01:24:31,871][105620] Updated weights for policy 1, policy_version 1377676 (0.0005) [2023-12-27 01:24:31,936][105620] Updated weights for policy 1, policy_version 1377686 (0.0006) [2023-12-27 01:24:31,995][105620] Updated weights for policy 1, policy_version 1377696 (0.0008) [2023-12-27 01:24:32,500][105692] Updated weights for policy 0, policy_version 1375513 (0.0009) [2023-12-27 01:24:32,564][105692] Updated weights for policy 0, policy_version 1375523 (0.0009) [2023-12-27 01:24:32,629][105692] Updated weights for policy 0, policy_version 1375533 (0.0010) [2023-12-27 01:24:32,710][105620] Updated weights for policy 1, policy_version 1377706 (0.0008) [2023-12-27 01:24:32,761][105620] Updated weights for policy 1, policy_version 1377716 (0.0009) [2023-12-27 01:24:32,809][105620] Updated weights for policy 1, policy_version 1377726 (0.0009) [2023-12-27 01:24:33,381][105692] Updated weights for policy 0, policy_version 1375543 (0.0009) [2023-12-27 01:24:33,433][105692] Updated weights for policy 0, policy_version 1375553 (0.0008) [2023-12-27 01:24:33,498][105692] Updated weights for policy 0, policy_version 1375563 (0.0007) [2023-12-27 01:24:33,594][105620] Updated weights for policy 1, policy_version 1377737 (0.0010) [2023-12-27 01:24:33,652][105620] Updated weights for policy 1, policy_version 1377747 (0.0010) [2023-12-27 01:24:33,720][105620] Updated weights for policy 1, policy_version 1377757 (0.0009) [2023-12-27 01:24:34,055][105692] Updated weights for policy 0, policy_version 1375573 (0.0007) [2023-12-27 01:24:34,116][105692] Updated weights for policy 0, policy_version 1375583 (0.0010) [2023-12-27 01:24:34,180][105692] Updated weights for policy 0, policy_version 1375593 (0.0008) [2023-12-27 01:24:34,382][105620] Updated weights for policy 1, policy_version 1377767 (0.0009) [2023-12-27 01:24:34,446][105620] Updated weights for policy 1, policy_version 1377777 (0.0009) [2023-12-27 01:24:34,512][105620] Updated weights for policy 1, policy_version 1377787 (0.0010) [2023-12-27 01:24:34,826][105692] Updated weights for policy 0, policy_version 1375603 (0.0009) [2023-12-27 01:24:34,884][105692] Updated weights for policy 0, policy_version 1375613 (0.0009) [2023-12-27 01:24:34,940][105692] Updated weights for policy 0, policy_version 1375623 (0.0008) [2023-12-27 01:24:35,326][105620] Updated weights for policy 1, policy_version 1377797 (0.0009) [2023-12-27 01:24:35,395][105620] Updated weights for policy 1, policy_version 1377807 (0.0008) [2023-12-27 01:24:35,453][105620] Updated weights for policy 1, policy_version 1377817 (0.0009) [2023-12-27 01:24:35,523][105692] Updated weights for policy 0, policy_version 1375633 (0.0006) [2023-12-27 01:24:35,577][105692] Updated weights for policy 0, policy_version 1375643 (0.0009) [2023-12-27 01:24:35,634][105692] Updated weights for policy 0, policy_version 1375653 (0.0010) [2023-12-27 01:24:35,684][105692] Updated weights for policy 0, policy_version 1375663 (0.0011) [2023-12-27 01:24:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 704987136. Throughput: 0: 9561.1, 1: 10066.3. Samples: 704976660. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:24:36,062][104569] Avg episode reward: [(0, '8532.696'), (1, '9174.186')] [2023-12-27 01:24:36,259][105620] Updated weights for policy 1, policy_version 1377827 (0.0007) [2023-12-27 01:24:36,323][105692] Updated weights for policy 0, policy_version 1375673 (0.0008) [2023-12-27 01:24:36,329][105620] Updated weights for policy 1, policy_version 1377837 (0.0006) [2023-12-27 01:24:36,362][105585] KL-divergence is very high: 108.0467 [2023-12-27 01:24:36,388][105692] Updated weights for policy 0, policy_version 1375683 (0.0010) [2023-12-27 01:24:36,391][105620] Updated weights for policy 1, policy_version 1377847 (0.0006) [2023-12-27 01:24:36,414][105585] KL-divergence is very high: 208.2246 [2023-12-27 01:24:36,451][105692] Updated weights for policy 0, policy_version 1375693 (0.0010) [2023-12-27 01:24:36,461][105585] KL-divergence is very high: 223.8181 [2023-12-27 01:24:37,109][105620] Updated weights for policy 1, policy_version 1377857 (0.0008) [2023-12-27 01:24:37,168][105620] Updated weights for policy 1, policy_version 1377867 (0.0008) [2023-12-27 01:24:37,213][105692] Updated weights for policy 0, policy_version 1375703 (0.0010) [2023-12-27 01:24:37,224][105620] Updated weights for policy 1, policy_version 1377877 (0.0006) [2023-12-27 01:24:37,258][105692] Updated weights for policy 0, policy_version 1375713 (0.0010) [2023-12-27 01:24:37,291][105620] Updated weights for policy 1, policy_version 1377887 (0.0006) [2023-12-27 01:24:37,320][105692] Updated weights for policy 0, policy_version 1375723 (0.0010) [2023-12-27 01:24:38,052][105620] Updated weights for policy 1, policy_version 1377897 (0.0008) [2023-12-27 01:24:38,066][105692] Updated weights for policy 0, policy_version 1375733 (0.0010) [2023-12-27 01:24:38,104][105620] Updated weights for policy 1, policy_version 1377907 (0.0005) [2023-12-27 01:24:38,118][105692] Updated weights for policy 0, policy_version 1375743 (0.0010) [2023-12-27 01:24:38,163][105620] Updated weights for policy 1, policy_version 1377917 (0.0005) [2023-12-27 01:24:38,180][105692] Updated weights for policy 0, policy_version 1375753 (0.0010) [2023-12-27 01:24:38,935][105692] Updated weights for policy 0, policy_version 1375763 (0.0010) [2023-12-27 01:24:38,936][105620] Updated weights for policy 1, policy_version 1377927 (0.0008) [2023-12-27 01:24:38,985][105620] Updated weights for policy 1, policy_version 1377937 (0.0007) [2023-12-27 01:24:38,994][105692] Updated weights for policy 0, policy_version 1375773 (0.0010) [2023-12-27 01:24:39,035][105620] Updated weights for policy 1, policy_version 1377947 (0.0009) [2023-12-27 01:24:39,049][105692] Updated weights for policy 0, policy_version 1375783 (0.0010) [2023-12-27 01:24:39,806][105692] Updated weights for policy 0, policy_version 1375793 (0.0010) [2023-12-27 01:24:39,875][105692] Updated weights for policy 0, policy_version 1375803 (0.0007) [2023-12-27 01:24:39,906][105620] Updated weights for policy 1, policy_version 1377957 (0.0007) [2023-12-27 01:24:39,939][105692] Updated weights for policy 0, policy_version 1375813 (0.0007) [2023-12-27 01:24:39,966][105620] Updated weights for policy 1, policy_version 1377967 (0.0008) [2023-12-27 01:24:40,006][105692] Updated weights for policy 0, policy_version 1375823 (0.0007) [2023-12-27 01:24:40,026][105620] Updated weights for policy 1, policy_version 1377977 (0.0008) [2023-12-27 01:24:40,652][105692] Updated weights for policy 0, policy_version 1375833 (0.0007) [2023-12-27 01:24:40,710][105692] Updated weights for policy 0, policy_version 1375843 (0.0006) [2023-12-27 01:24:40,772][105692] Updated weights for policy 0, policy_version 1375853 (0.0006) [2023-12-27 01:24:40,830][105620] Updated weights for policy 1, policy_version 1377987 (0.0008) [2023-12-27 01:24:40,879][105620] Updated weights for policy 1, policy_version 1377997 (0.0008) [2023-12-27 01:24:40,931][105620] Updated weights for policy 1, policy_version 1378008 (0.0010) [2023-12-27 01:24:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 705085440. Throughput: 0: 9591.4, 1: 9957.6. Samples: 705089568. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:24:41,063][104569] Avg episode reward: [(0, '7980.469'), (1, '9081.892')] [2023-12-27 01:24:41,463][105692] Updated weights for policy 0, policy_version 1375863 (0.0007) [2023-12-27 01:24:41,521][105692] Updated weights for policy 0, policy_version 1375873 (0.0009) [2023-12-27 01:24:41,575][105692] Updated weights for policy 0, policy_version 1375883 (0.0010) [2023-12-27 01:24:41,647][105620] Updated weights for policy 1, policy_version 1378018 (0.0006) [2023-12-27 01:24:41,716][105620] Updated weights for policy 1, policy_version 1378028 (0.0007) [2023-12-27 01:24:41,783][105620] Updated weights for policy 1, policy_version 1378038 (0.0011) [2023-12-27 01:24:41,844][105620] Updated weights for policy 1, policy_version 1378048 (0.0011) [2023-12-27 01:24:42,368][105692] Updated weights for policy 0, policy_version 1375893 (0.0010) [2023-12-27 01:24:42,426][105692] Updated weights for policy 0, policy_version 1375903 (0.0008) [2023-12-27 01:24:42,492][105692] Updated weights for policy 0, policy_version 1375913 (0.0010) [2023-12-27 01:24:42,551][105620] Updated weights for policy 1, policy_version 1378058 (0.0010) [2023-12-27 01:24:42,601][105620] Updated weights for policy 1, policy_version 1378068 (0.0010) [2023-12-27 01:24:42,657][105620] Updated weights for policy 1, policy_version 1378078 (0.0010) [2023-12-27 01:24:43,121][105692] Updated weights for policy 0, policy_version 1375923 (0.0009) [2023-12-27 01:24:43,187][105692] Updated weights for policy 0, policy_version 1375933 (0.0006) [2023-12-27 01:24:43,244][105692] Updated weights for policy 0, policy_version 1375943 (0.0005) [2023-12-27 01:24:43,433][105620] Updated weights for policy 1, policy_version 1378088 (0.0010) [2023-12-27 01:24:43,480][105620] Updated weights for policy 1, policy_version 1378098 (0.0009) [2023-12-27 01:24:43,535][105620] Updated weights for policy 1, policy_version 1378108 (0.0005) [2023-12-27 01:24:43,814][105692] Updated weights for policy 0, policy_version 1375953 (0.0005) [2023-12-27 01:24:43,877][105692] Updated weights for policy 0, policy_version 1375963 (0.0005) [2023-12-27 01:24:43,922][105692] Updated weights for policy 0, policy_version 1375973 (0.0005) [2023-12-27 01:24:43,968][105692] Updated weights for policy 0, policy_version 1375983 (0.0005) [2023-12-27 01:24:44,236][105620] Updated weights for policy 1, policy_version 1378118 (0.0010) [2023-12-27 01:24:44,289][105620] Updated weights for policy 1, policy_version 1378128 (0.0010) [2023-12-27 01:24:44,343][105620] Updated weights for policy 1, policy_version 1378138 (0.0010) [2023-12-27 01:24:44,563][105692] Updated weights for policy 0, policy_version 1375993 (0.0005) [2023-12-27 01:24:44,618][105692] Updated weights for policy 0, policy_version 1376003 (0.0010) [2023-12-27 01:24:44,665][105692] Updated weights for policy 0, policy_version 1376013 (0.0009) [2023-12-27 01:24:45,101][105620] Updated weights for policy 1, policy_version 1378148 (0.0009) [2023-12-27 01:24:45,164][105620] Updated weights for policy 1, policy_version 1378158 (0.0006) [2023-12-27 01:24:45,234][105620] Updated weights for policy 1, policy_version 1378168 (0.0007) [2023-12-27 01:24:45,376][105692] Updated weights for policy 0, policy_version 1376023 (0.0009) [2023-12-27 01:24:45,431][105692] Updated weights for policy 0, policy_version 1376033 (0.0010) [2023-12-27 01:24:45,494][105692] Updated weights for policy 0, policy_version 1376043 (0.0010) [2023-12-27 01:24:45,772][105620] Updated weights for policy 1, policy_version 1378178 (0.0008) [2023-12-27 01:24:45,820][105620] Updated weights for policy 1, policy_version 1378188 (0.0005) [2023-12-27 01:24:45,866][105620] Updated weights for policy 1, policy_version 1378198 (0.0005) [2023-12-27 01:24:45,923][105620] Updated weights for policy 1, policy_version 1378208 (0.0005) [2023-12-27 01:24:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.9, 300 sec: 19438.6). Total num frames: 705183744. Throughput: 0: 9601.3, 1: 9919.5. Samples: 705148816. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:24:46,063][104569] Avg episode reward: [(0, '8164.685'), (1, '8775.192')] [2023-12-27 01:24:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001376048_352321536.pth... [2023-12-27 01:24:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001378208_352862208.pth... [2023-12-27 01:24:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001374896_352026624.pth [2023-12-27 01:24:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001377056_352567296.pth [2023-12-27 01:24:46,433][105692] Updated weights for policy 0, policy_version 1376053 (0.0009) [2023-12-27 01:24:46,461][105620] Updated weights for policy 1, policy_version 1378218 (0.0008) [2023-12-27 01:24:46,490][105692] Updated weights for policy 0, policy_version 1376063 (0.0006) [2023-12-27 01:24:46,512][105620] Updated weights for policy 1, policy_version 1378228 (0.0010) [2023-12-27 01:24:46,541][105692] Updated weights for policy 0, policy_version 1376073 (0.0005) [2023-12-27 01:24:46,570][105620] Updated weights for policy 1, policy_version 1378238 (0.0010) [2023-12-27 01:24:47,115][105620] Updated weights for policy 1, policy_version 1378248 (0.0007) [2023-12-27 01:24:47,172][105620] Updated weights for policy 1, policy_version 1378258 (0.0005) [2023-12-27 01:24:47,222][105692] Updated weights for policy 0, policy_version 1376083 (0.0008) [2023-12-27 01:24:47,228][105620] Updated weights for policy 1, policy_version 1378268 (0.0005) [2023-12-27 01:24:47,276][105692] Updated weights for policy 0, policy_version 1376093 (0.0009) [2023-12-27 01:24:47,333][105692] Updated weights for policy 0, policy_version 1376103 (0.0013) [2023-12-27 01:24:47,775][105620] Updated weights for policy 1, policy_version 1378278 (0.0005) [2023-12-27 01:24:47,838][105620] Updated weights for policy 1, policy_version 1378288 (0.0006) [2023-12-27 01:24:47,886][105620] Updated weights for policy 1, policy_version 1378298 (0.0009) [2023-12-27 01:24:48,159][105692] Updated weights for policy 0, policy_version 1376114 (0.0010) [2023-12-27 01:24:48,220][105692] Updated weights for policy 0, policy_version 1376124 (0.0010) [2023-12-27 01:24:48,275][105692] Updated weights for policy 0, policy_version 1376134 (0.0010) [2023-12-27 01:24:48,330][105692] Updated weights for policy 0, policy_version 1376144 (0.0009) [2023-12-27 01:24:48,535][105620] Updated weights for policy 1, policy_version 1378308 (0.0007) [2023-12-27 01:24:48,603][105620] Updated weights for policy 1, policy_version 1378318 (0.0005) [2023-12-27 01:24:48,661][105620] Updated weights for policy 1, policy_version 1378328 (0.0005) [2023-12-27 01:24:49,178][105692] Updated weights for policy 0, policy_version 1376154 (0.0009) [2023-12-27 01:24:49,251][105692] Updated weights for policy 0, policy_version 1376164 (0.0008) [2023-12-27 01:24:49,309][105620] Updated weights for policy 1, policy_version 1378338 (0.0006) [2023-12-27 01:24:49,312][105692] Updated weights for policy 0, policy_version 1376174 (0.0006) [2023-12-27 01:24:49,375][105620] Updated weights for policy 1, policy_version 1378348 (0.0011) [2023-12-27 01:24:49,434][105620] Updated weights for policy 1, policy_version 1378358 (0.0011) [2023-12-27 01:24:49,483][105620] Updated weights for policy 1, policy_version 1378368 (0.0010) [2023-12-27 01:24:49,953][105692] Updated weights for policy 0, policy_version 1376184 (0.0008) [2023-12-27 01:24:50,009][105692] Updated weights for policy 0, policy_version 1376194 (0.0009) [2023-12-27 01:24:50,064][105692] Updated weights for policy 0, policy_version 1376204 (0.0008) [2023-12-27 01:24:50,256][105620] Updated weights for policy 1, policy_version 1378378 (0.0006) [2023-12-27 01:24:50,315][105620] Updated weights for policy 1, policy_version 1378388 (0.0007) [2023-12-27 01:24:50,379][105620] Updated weights for policy 1, policy_version 1378398 (0.0010) [2023-12-27 01:24:50,865][105692] Updated weights for policy 0, policy_version 1376214 (0.0007) [2023-12-27 01:24:50,913][105692] Updated weights for policy 0, policy_version 1376224 (0.0005) [2023-12-27 01:24:50,961][105692] Updated weights for policy 0, policy_version 1376234 (0.0006) [2023-12-27 01:24:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19438.7). Total num frames: 705282048. Throughput: 0: 9549.3, 1: 9983.5. Samples: 705271612. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:24:51,062][104569] Avg episode reward: [(0, '8352.805'), (1, '8719.646')] [2023-12-27 01:24:51,102][105620] Updated weights for policy 1, policy_version 1378408 (0.0007) [2023-12-27 01:24:51,164][105620] Updated weights for policy 1, policy_version 1378418 (0.0008) [2023-12-27 01:24:51,226][105620] Updated weights for policy 1, policy_version 1378428 (0.0008) [2023-12-27 01:24:51,714][105692] Updated weights for policy 0, policy_version 1376244 (0.0007) [2023-12-27 01:24:51,778][105692] Updated weights for policy 0, policy_version 1376254 (0.0009) [2023-12-27 01:24:51,838][105692] Updated weights for policy 0, policy_version 1376264 (0.0008) [2023-12-27 01:24:51,914][105620] Updated weights for policy 1, policy_version 1378438 (0.0009) [2023-12-27 01:24:51,962][105620] Updated weights for policy 1, policy_version 1378448 (0.0008) [2023-12-27 01:24:52,014][105620] Updated weights for policy 1, policy_version 1378458 (0.0008) [2023-12-27 01:24:52,507][105692] Updated weights for policy 0, policy_version 1376274 (0.0008) [2023-12-27 01:24:52,567][105692] Updated weights for policy 0, policy_version 1376284 (0.0009) [2023-12-27 01:24:52,622][105692] Updated weights for policy 0, policy_version 1376294 (0.0009) [2023-12-27 01:24:52,693][105692] Updated weights for policy 0, policy_version 1376304 (0.0010) [2023-12-27 01:24:52,783][105620] Updated weights for policy 1, policy_version 1378468 (0.0007) [2023-12-27 01:24:52,841][105620] Updated weights for policy 1, policy_version 1378478 (0.0009) [2023-12-27 01:24:52,899][105620] Updated weights for policy 1, policy_version 1378488 (0.0009) [2023-12-27 01:24:53,415][105692] Updated weights for policy 0, policy_version 1376314 (0.0005) [2023-12-27 01:24:53,468][105692] Updated weights for policy 0, policy_version 1376324 (0.0008) [2023-12-27 01:24:53,515][105692] Updated weights for policy 0, policy_version 1376334 (0.0008) [2023-12-27 01:24:53,696][105620] Updated weights for policy 1, policy_version 1378498 (0.0009) [2023-12-27 01:24:53,750][105620] Updated weights for policy 1, policy_version 1378508 (0.0008) [2023-12-27 01:24:53,800][105620] Updated weights for policy 1, policy_version 1378518 (0.0009) [2023-12-27 01:24:53,856][105620] Updated weights for policy 1, policy_version 1378528 (0.0010) [2023-12-27 01:24:54,172][105692] Updated weights for policy 0, policy_version 1376344 (0.0009) [2023-12-27 01:24:54,233][105692] Updated weights for policy 0, policy_version 1376354 (0.0006) [2023-12-27 01:24:54,294][105692] Updated weights for policy 0, policy_version 1376364 (0.0010) [2023-12-27 01:24:54,612][105620] Updated weights for policy 1, policy_version 1378538 (0.0010) [2023-12-27 01:24:54,673][105620] Updated weights for policy 1, policy_version 1378548 (0.0010) [2023-12-27 01:24:54,740][105620] Updated weights for policy 1, policy_version 1378558 (0.0010) [2023-12-27 01:24:54,870][105692] Updated weights for policy 0, policy_version 1376374 (0.0007) [2023-12-27 01:24:54,917][105692] Updated weights for policy 0, policy_version 1376384 (0.0009) [2023-12-27 01:24:54,968][105692] Updated weights for policy 0, policy_version 1376394 (0.0007) [2023-12-27 01:24:55,351][105620] Updated weights for policy 1, policy_version 1378568 (0.0006) [2023-12-27 01:24:55,407][105620] Updated weights for policy 1, policy_version 1378578 (0.0009) [2023-12-27 01:24:55,454][105620] Updated weights for policy 1, policy_version 1378588 (0.0009) [2023-12-27 01:24:55,707][105692] Updated weights for policy 0, policy_version 1376404 (0.0008) [2023-12-27 01:24:55,762][105692] Updated weights for policy 0, policy_version 1376414 (0.0011) [2023-12-27 01:24:55,810][105692] Updated weights for policy 0, policy_version 1376424 (0.0010) [2023-12-27 01:24:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 705380352. Throughput: 0: 9717.2, 1: 9933.5. Samples: 705389244. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:24:56,062][104569] Avg episode reward: [(0, '8446.010'), (1, '9081.438')] [2023-12-27 01:24:56,171][105620] Updated weights for policy 1, policy_version 1378598 (0.0008) [2023-12-27 01:24:56,222][105620] Updated weights for policy 1, policy_version 1378608 (0.0010) [2023-12-27 01:24:56,269][105620] Updated weights for policy 1, policy_version 1378618 (0.0010) [2023-12-27 01:24:56,491][105692] Updated weights for policy 0, policy_version 1376434 (0.0009) [2023-12-27 01:24:56,556][105692] Updated weights for policy 0, policy_version 1376444 (0.0006) [2023-12-27 01:24:56,614][105692] Updated weights for policy 0, policy_version 1376454 (0.0009) [2023-12-27 01:24:56,672][105692] Updated weights for policy 0, policy_version 1376464 (0.0010) [2023-12-27 01:24:56,996][105620] Updated weights for policy 1, policy_version 1378628 (0.0009) [2023-12-27 01:24:57,053][105620] Updated weights for policy 1, policy_version 1378638 (0.0005) [2023-12-27 01:24:57,100][105620] Updated weights for policy 1, policy_version 1378648 (0.0006) [2023-12-27 01:24:57,349][105692] Updated weights for policy 0, policy_version 1376474 (0.0010) [2023-12-27 01:24:57,400][105692] Updated weights for policy 0, policy_version 1376484 (0.0010) [2023-12-27 01:24:57,448][105692] Updated weights for policy 0, policy_version 1376494 (0.0010) [2023-12-27 01:24:57,656][105620] Updated weights for policy 1, policy_version 1378658 (0.0005) [2023-12-27 01:24:57,709][105620] Updated weights for policy 1, policy_version 1378668 (0.0005) [2023-12-27 01:24:57,762][105620] Updated weights for policy 1, policy_version 1378678 (0.0005) [2023-12-27 01:24:57,827][105620] Updated weights for policy 1, policy_version 1378688 (0.0005) [2023-12-27 01:24:58,213][105692] Updated weights for policy 0, policy_version 1376504 (0.0011) [2023-12-27 01:24:58,276][105692] Updated weights for policy 0, policy_version 1376514 (0.0010) [2023-12-27 01:24:58,349][105692] Updated weights for policy 0, policy_version 1376524 (0.0011) [2023-12-27 01:24:58,528][105620] Updated weights for policy 1, policy_version 1378698 (0.0011) [2023-12-27 01:24:58,595][105620] Updated weights for policy 1, policy_version 1378708 (0.0011) [2023-12-27 01:24:58,664][105620] Updated weights for policy 1, policy_version 1378718 (0.0011) [2023-12-27 01:24:59,147][105692] Updated weights for policy 0, policy_version 1376534 (0.0011) [2023-12-27 01:24:59,210][105692] Updated weights for policy 0, policy_version 1376544 (0.0011) [2023-12-27 01:24:59,292][105692] Updated weights for policy 0, policy_version 1376554 (0.0010) [2023-12-27 01:24:59,504][105620] Updated weights for policy 1, policy_version 1378728 (0.0008) [2023-12-27 01:24:59,563][105620] Updated weights for policy 1, policy_version 1378738 (0.0008) [2023-12-27 01:24:59,615][105620] Updated weights for policy 1, policy_version 1378748 (0.0010) [2023-12-27 01:24:59,986][105692] Updated weights for policy 0, policy_version 1376564 (0.0009) [2023-12-27 01:25:00,035][105692] Updated weights for policy 0, policy_version 1376574 (0.0006) [2023-12-27 01:25:00,079][105692] Updated weights for policy 0, policy_version 1376584 (0.0005) [2023-12-27 01:25:00,462][105620] Updated weights for policy 1, policy_version 1378758 (0.0009) [2023-12-27 01:25:00,515][105620] Updated weights for policy 1, policy_version 1378768 (0.0010) [2023-12-27 01:25:00,566][105620] Updated weights for policy 1, policy_version 1378778 (0.0008) [2023-12-27 01:25:00,651][105692] Updated weights for policy 0, policy_version 1376594 (0.0005) [2023-12-27 01:25:00,714][105692] Updated weights for policy 0, policy_version 1376604 (0.0005) [2023-12-27 01:25:00,769][105692] Updated weights for policy 0, policy_version 1376614 (0.0005) [2023-12-27 01:25:00,830][105692] Updated weights for policy 0, policy_version 1376624 (0.0005) [2023-12-27 01:25:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 705478656. Throughput: 0: 9734.7, 1: 9929.4. Samples: 705448536. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:25:01,063][104569] Avg episode reward: [(0, '8625.111'), (1, '8991.565')] [2023-12-27 01:25:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001376624_352468992.pth... [2023-12-27 01:25:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001378784_353009664.pth... [2023-12-27 01:25:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001377664_352722944.pth [2023-12-27 01:25:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001375472_352174080.pth [2023-12-27 01:25:01,327][105620] Updated weights for policy 1, policy_version 1378788 (0.0008) [2023-12-27 01:25:01,389][105620] Updated weights for policy 1, policy_version 1378798 (0.0007) [2023-12-27 01:25:01,454][105620] Updated weights for policy 1, policy_version 1378808 (0.0010) [2023-12-27 01:25:01,490][105692] Updated weights for policy 0, policy_version 1376634 (0.0009) [2023-12-27 01:25:01,538][105692] Updated weights for policy 0, policy_version 1376644 (0.0010) [2023-12-27 01:25:01,586][105692] Updated weights for policy 0, policy_version 1376654 (0.0009) [2023-12-27 01:25:02,114][105620] Updated weights for policy 1, policy_version 1378818 (0.0011) [2023-12-27 01:25:02,172][105620] Updated weights for policy 1, policy_version 1378828 (0.0010) [2023-12-27 01:25:02,231][105620] Updated weights for policy 1, policy_version 1378838 (0.0010) [2023-12-27 01:25:02,290][105620] Updated weights for policy 1, policy_version 1378848 (0.0008) [2023-12-27 01:25:02,322][105692] Updated weights for policy 0, policy_version 1376664 (0.0009) [2023-12-27 01:25:02,379][105692] Updated weights for policy 0, policy_version 1376674 (0.0011) [2023-12-27 01:25:02,437][105692] Updated weights for policy 0, policy_version 1376684 (0.0010) [2023-12-27 01:25:02,924][105620] Updated weights for policy 1, policy_version 1378858 (0.0006) [2023-12-27 01:25:02,973][105620] Updated weights for policy 1, policy_version 1378868 (0.0005) [2023-12-27 01:25:03,027][105620] Updated weights for policy 1, policy_version 1378878 (0.0010) [2023-12-27 01:25:03,045][105692] Updated weights for policy 0, policy_version 1376694 (0.0008) [2023-12-27 01:25:03,096][105692] Updated weights for policy 0, policy_version 1376704 (0.0008) [2023-12-27 01:25:03,141][105692] Updated weights for policy 0, policy_version 1376714 (0.0008) [2023-12-27 01:25:03,733][105620] Updated weights for policy 1, policy_version 1378888 (0.0010) [2023-12-27 01:25:03,782][105692] Updated weights for policy 0, policy_version 1376724 (0.0008) [2023-12-27 01:25:03,795][105620] Updated weights for policy 1, policy_version 1378898 (0.0006) [2023-12-27 01:25:03,840][105692] Updated weights for policy 0, policy_version 1376734 (0.0008) [2023-12-27 01:25:03,856][105620] Updated weights for policy 1, policy_version 1378908 (0.0007) [2023-12-27 01:25:03,904][105692] Updated weights for policy 0, policy_version 1376744 (0.0008) [2023-12-27 01:25:04,586][105620] Updated weights for policy 1, policy_version 1378918 (0.0010) [2023-12-27 01:25:04,624][105692] Updated weights for policy 0, policy_version 1376754 (0.0008) [2023-12-27 01:25:04,639][105620] Updated weights for policy 1, policy_version 1378928 (0.0011) [2023-12-27 01:25:04,680][105692] Updated weights for policy 0, policy_version 1376764 (0.0005) [2023-12-27 01:25:04,697][105620] Updated weights for policy 1, policy_version 1378938 (0.0011) [2023-12-27 01:25:04,737][105692] Updated weights for policy 0, policy_version 1376774 (0.0005) [2023-12-27 01:25:04,786][105692] Updated weights for policy 0, policy_version 1376784 (0.0006) [2023-12-27 01:25:05,348][105620] Updated weights for policy 1, policy_version 1378948 (0.0010) [2023-12-27 01:25:05,406][105620] Updated weights for policy 1, policy_version 1378958 (0.0010) [2023-12-27 01:25:05,461][105620] Updated weights for policy 1, policy_version 1378968 (0.0010) [2023-12-27 01:25:05,567][105692] Updated weights for policy 0, policy_version 1376794 (0.0008) [2023-12-27 01:25:05,635][105692] Updated weights for policy 0, policy_version 1376804 (0.0008) [2023-12-27 01:25:05,693][105692] Updated weights for policy 0, policy_version 1376814 (0.0010) [2023-12-27 01:25:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 705576960. Throughput: 0: 9902.3, 1: 9881.7. Samples: 705567452. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:25:06,063][104569] Avg episode reward: [(0, '8069.885'), (1, '8866.062')] [2023-12-27 01:25:06,111][105620] Updated weights for policy 1, policy_version 1378978 (0.0010) [2023-12-27 01:25:06,176][105620] Updated weights for policy 1, policy_version 1378988 (0.0011) [2023-12-27 01:25:06,235][105620] Updated weights for policy 1, policy_version 1378998 (0.0011) [2023-12-27 01:25:06,298][105620] Updated weights for policy 1, policy_version 1379008 (0.0011) [2023-12-27 01:25:06,437][105692] Updated weights for policy 0, policy_version 1376824 (0.0009) [2023-12-27 01:25:06,499][105692] Updated weights for policy 0, policy_version 1376834 (0.0009) [2023-12-27 01:25:06,564][105692] Updated weights for policy 0, policy_version 1376844 (0.0008) [2023-12-27 01:25:07,047][105620] Updated weights for policy 1, policy_version 1379018 (0.0011) [2023-12-27 01:25:07,113][105620] Updated weights for policy 1, policy_version 1379028 (0.0011) [2023-12-27 01:25:07,169][105620] Updated weights for policy 1, policy_version 1379038 (0.0010) [2023-12-27 01:25:07,266][105692] Updated weights for policy 0, policy_version 1376854 (0.0008) [2023-12-27 01:25:07,329][105585] KL-divergence is very high: 106.0017 [2023-12-27 01:25:07,330][105692] Updated weights for policy 0, policy_version 1376864 (0.0006) [2023-12-27 01:25:07,374][105585] KL-divergence is very high: 107.0575 [2023-12-27 01:25:07,386][105692] Updated weights for policy 0, policy_version 1376874 (0.0008) [2023-12-27 01:25:07,809][105620] Updated weights for policy 1, policy_version 1379048 (0.0006) [2023-12-27 01:25:07,855][105620] Updated weights for policy 1, policy_version 1379058 (0.0005) [2023-12-27 01:25:07,906][105620] Updated weights for policy 1, policy_version 1379068 (0.0005) [2023-12-27 01:25:08,198][105692] Updated weights for policy 0, policy_version 1376884 (0.0007) [2023-12-27 01:25:08,244][105692] Updated weights for policy 0, policy_version 1376894 (0.0005) [2023-12-27 01:25:08,277][105585] KL-divergence is very high: 117.6760 [2023-12-27 01:25:08,293][105692] Updated weights for policy 0, policy_version 1376904 (0.0005) [2023-12-27 01:25:08,321][105585] KL-divergence is very high: 118.0610 [2023-12-27 01:25:08,605][105620] Updated weights for policy 1, policy_version 1379078 (0.0007) [2023-12-27 01:25:08,657][105620] Updated weights for policy 1, policy_version 1379088 (0.0009) [2023-12-27 01:25:08,718][105620] Updated weights for policy 1, policy_version 1379098 (0.0010) [2023-12-27 01:25:08,914][105692] Updated weights for policy 0, policy_version 1376914 (0.0006) [2023-12-27 01:25:08,975][105692] Updated weights for policy 0, policy_version 1376924 (0.0006) [2023-12-27 01:25:09,027][105692] Updated weights for policy 0, policy_version 1376934 (0.0005) [2023-12-27 01:25:09,086][105692] Updated weights for policy 0, policy_version 1376944 (0.0007) [2023-12-27 01:25:09,510][105620] Updated weights for policy 1, policy_version 1379108 (0.0010) [2023-12-27 01:25:09,564][105620] Updated weights for policy 1, policy_version 1379118 (0.0008) [2023-12-27 01:25:09,617][105620] Updated weights for policy 1, policy_version 1379128 (0.0008) [2023-12-27 01:25:09,738][105692] Updated weights for policy 0, policy_version 1376954 (0.0010) [2023-12-27 01:25:09,798][105692] Updated weights for policy 0, policy_version 1376964 (0.0010) [2023-12-27 01:25:09,866][105692] Updated weights for policy 0, policy_version 1376974 (0.0011) [2023-12-27 01:25:10,420][105620] Updated weights for policy 1, policy_version 1379138 (0.0008) [2023-12-27 01:25:10,481][105620] Updated weights for policy 1, policy_version 1379148 (0.0008) [2023-12-27 01:25:10,545][105620] Updated weights for policy 1, policy_version 1379158 (0.0008) [2023-12-27 01:25:10,610][105620] Updated weights for policy 1, policy_version 1379168 (0.0008) [2023-12-27 01:25:10,650][105692] Updated weights for policy 0, policy_version 1376984 (0.0010) [2023-12-27 01:25:10,716][105692] Updated weights for policy 0, policy_version 1376994 (0.0010) [2023-12-27 01:25:10,774][105692] Updated weights for policy 0, policy_version 1377004 (0.0010) [2023-12-27 01:25:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 705675264. Throughput: 0: 9872.5, 1: 9833.1. Samples: 705682988. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:25:11,062][104569] Avg episode reward: [(0, '7977.649'), (1, '9101.961')] [2023-12-27 01:25:11,308][105620] Updated weights for policy 1, policy_version 1379178 (0.0009) [2023-12-27 01:25:11,373][105620] Updated weights for policy 1, policy_version 1379188 (0.0009) [2023-12-27 01:25:11,435][105620] Updated weights for policy 1, policy_version 1379198 (0.0006) [2023-12-27 01:25:11,515][105692] Updated weights for policy 0, policy_version 1377014 (0.0010) [2023-12-27 01:25:11,581][105692] Updated weights for policy 0, policy_version 1377024 (0.0010) [2023-12-27 01:25:11,644][105692] Updated weights for policy 0, policy_version 1377034 (0.0009) [2023-12-27 01:25:12,122][105620] Updated weights for policy 1, policy_version 1379208 (0.0007) [2023-12-27 01:25:12,180][105620] Updated weights for policy 1, policy_version 1379218 (0.0007) [2023-12-27 01:25:12,242][105620] Updated weights for policy 1, policy_version 1379228 (0.0008) [2023-12-27 01:25:12,460][105692] Updated weights for policy 0, policy_version 1377044 (0.0008) [2023-12-27 01:25:12,515][105692] Updated weights for policy 0, policy_version 1377054 (0.0005) [2023-12-27 01:25:12,574][105692] Updated weights for policy 0, policy_version 1377064 (0.0007) [2023-12-27 01:25:13,033][105620] Updated weights for policy 1, policy_version 1379238 (0.0010) [2023-12-27 01:25:13,085][105620] Updated weights for policy 1, policy_version 1379248 (0.0009) [2023-12-27 01:25:13,136][105620] Updated weights for policy 1, policy_version 1379258 (0.0009) [2023-12-27 01:25:13,289][105692] Updated weights for policy 0, policy_version 1377074 (0.0009) [2023-12-27 01:25:13,345][105692] Updated weights for policy 0, policy_version 1377084 (0.0009) [2023-12-27 01:25:13,392][105692] Updated weights for policy 0, policy_version 1377094 (0.0009) [2023-12-27 01:25:13,440][105692] Updated weights for policy 0, policy_version 1377104 (0.0009) [2023-12-27 01:25:13,940][105620] Updated weights for policy 1, policy_version 1379268 (0.0009) [2023-12-27 01:25:14,006][105620] Updated weights for policy 1, policy_version 1379278 (0.0011) [2023-12-27 01:25:14,071][105620] Updated weights for policy 1, policy_version 1379288 (0.0010) [2023-12-27 01:25:14,093][105692] Updated weights for policy 0, policy_version 1377114 (0.0006) [2023-12-27 01:25:14,150][105692] Updated weights for policy 0, policy_version 1377124 (0.0008) [2023-12-27 01:25:14,219][105692] Updated weights for policy 0, policy_version 1377134 (0.0009) [2023-12-27 01:25:14,776][105620] Updated weights for policy 1, policy_version 1379298 (0.0010) [2023-12-27 01:25:14,836][105620] Updated weights for policy 1, policy_version 1379308 (0.0010) [2023-12-27 01:25:14,885][105620] Updated weights for policy 1, policy_version 1379318 (0.0011) [2023-12-27 01:25:14,943][105620] Updated weights for policy 1, policy_version 1379328 (0.0011) [2023-12-27 01:25:14,971][105692] Updated weights for policy 0, policy_version 1377144 (0.0009) [2023-12-27 01:25:15,025][105692] Updated weights for policy 0, policy_version 1377154 (0.0008) [2023-12-27 01:25:15,081][105692] Updated weights for policy 0, policy_version 1377164 (0.0008) [2023-12-27 01:25:15,717][105620] Updated weights for policy 1, policy_version 1379338 (0.0011) [2023-12-27 01:25:15,777][105620] Updated weights for policy 1, policy_version 1379348 (0.0011) [2023-12-27 01:25:15,802][105692] Updated weights for policy 0, policy_version 1377174 (0.0009) [2023-12-27 01:25:15,833][105620] Updated weights for policy 1, policy_version 1379358 (0.0010) [2023-12-27 01:25:15,858][105692] Updated weights for policy 0, policy_version 1377184 (0.0006) [2023-12-27 01:25:15,910][105692] Updated weights for policy 0, policy_version 1377194 (0.0008) [2023-12-27 01:25:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 705773568. Throughput: 0: 9819.8, 1: 9775.0. Samples: 705739176. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:25:16,062][104569] Avg episode reward: [(0, '8622.176'), (1, '9171.068')] [2023-12-27 01:25:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001379360_353157120.pth... [2023-12-27 01:25:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001377200_352616448.pth... [2023-12-27 01:25:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001376048_352321536.pth [2023-12-27 01:25:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001378208_352862208.pth [2023-12-27 01:25:16,516][105620] Updated weights for policy 1, policy_version 1379368 (0.0009) [2023-12-27 01:25:16,577][105620] Updated weights for policy 1, policy_version 1379378 (0.0009) [2023-12-27 01:25:16,639][105620] Updated weights for policy 1, policy_version 1379388 (0.0009) [2023-12-27 01:25:16,690][105692] Updated weights for policy 0, policy_version 1377204 (0.0010) [2023-12-27 01:25:16,741][105692] Updated weights for policy 0, policy_version 1377214 (0.0006) [2023-12-27 01:25:16,794][105692] Updated weights for policy 0, policy_version 1377224 (0.0008) [2023-12-27 01:25:17,329][105620] Updated weights for policy 1, policy_version 1379398 (0.0008) [2023-12-27 01:25:17,376][105620] Updated weights for policy 1, policy_version 1379408 (0.0008) [2023-12-27 01:25:17,423][105620] Updated weights for policy 1, policy_version 1379418 (0.0009) [2023-12-27 01:25:17,577][105692] Updated weights for policy 0, policy_version 1377234 (0.0009) [2023-12-27 01:25:17,624][105692] Updated weights for policy 0, policy_version 1377244 (0.0009) [2023-12-27 01:25:17,673][105692] Updated weights for policy 0, policy_version 1377254 (0.0008) [2023-12-27 01:25:17,731][105692] Updated weights for policy 0, policy_version 1377264 (0.0009) [2023-12-27 01:25:18,215][105620] Updated weights for policy 1, policy_version 1379428 (0.0009) [2023-12-27 01:25:18,272][105620] Updated weights for policy 1, policy_version 1379438 (0.0009) [2023-12-27 01:25:18,331][105620] Updated weights for policy 1, policy_version 1379448 (0.0008) [2023-12-27 01:25:18,405][105692] Updated weights for policy 0, policy_version 1377274 (0.0009) [2023-12-27 01:25:18,475][105692] Updated weights for policy 0, policy_version 1377284 (0.0008) [2023-12-27 01:25:18,536][105692] Updated weights for policy 0, policy_version 1377294 (0.0008) [2023-12-27 01:25:19,129][105620] Updated weights for policy 1, policy_version 1379458 (0.0010) [2023-12-27 01:25:19,188][105620] Updated weights for policy 1, policy_version 1379468 (0.0009) [2023-12-27 01:25:19,214][105692] Updated weights for policy 0, policy_version 1377304 (0.0006) [2023-12-27 01:25:19,256][105620] Updated weights for policy 1, policy_version 1379478 (0.0008) [2023-12-27 01:25:19,274][105692] Updated weights for policy 0, policy_version 1377314 (0.0007) [2023-12-27 01:25:19,326][105620] Updated weights for policy 1, policy_version 1379488 (0.0006) [2023-12-27 01:25:19,344][105692] Updated weights for policy 0, policy_version 1377324 (0.0007) [2023-12-27 01:25:20,035][105620] Updated weights for policy 1, policy_version 1379498 (0.0009) [2023-12-27 01:25:20,093][105620] Updated weights for policy 1, policy_version 1379508 (0.0009) [2023-12-27 01:25:20,100][105692] Updated weights for policy 0, policy_version 1377334 (0.0008) [2023-12-27 01:25:20,151][105692] Updated weights for policy 0, policy_version 1377344 (0.0009) [2023-12-27 01:25:20,155][105620] Updated weights for policy 1, policy_version 1379518 (0.0009) [2023-12-27 01:25:20,209][105692] Updated weights for policy 0, policy_version 1377354 (0.0009) [2023-12-27 01:25:20,953][105692] Updated weights for policy 0, policy_version 1377364 (0.0007) [2023-12-27 01:25:20,971][105620] Updated weights for policy 1, policy_version 1379528 (0.0007) [2023-12-27 01:25:21,008][105692] Updated weights for policy 0, policy_version 1377374 (0.0008) [2023-12-27 01:25:21,036][105620] Updated weights for policy 1, policy_version 1379538 (0.0009) [2023-12-27 01:25:21,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 705855488. Throughput: 0: 9794.7, 1: 9711.3. Samples: 705854432. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:25:21,062][104569] Avg episode reward: [(0, '8257.025'), (1, '8988.910')] [2023-12-27 01:25:21,074][105692] Updated weights for policy 0, policy_version 1377384 (0.0006) [2023-12-27 01:25:21,101][105620] Updated weights for policy 1, policy_version 1379548 (0.0010) [2023-12-27 01:25:21,864][105620] Updated weights for policy 1, policy_version 1379558 (0.0010) [2023-12-27 01:25:21,883][105692] Updated weights for policy 0, policy_version 1377394 (0.0007) [2023-12-27 01:25:21,929][105620] Updated weights for policy 1, policy_version 1379568 (0.0010) [2023-12-27 01:25:21,946][105692] Updated weights for policy 0, policy_version 1377404 (0.0006) [2023-12-27 01:25:21,988][105620] Updated weights for policy 1, policy_version 1379578 (0.0009) [2023-12-27 01:25:22,011][105692] Updated weights for policy 0, policy_version 1377414 (0.0006) [2023-12-27 01:25:22,073][105692] Updated weights for policy 0, policy_version 1377424 (0.0008) [2023-12-27 01:25:22,640][105620] Updated weights for policy 1, policy_version 1379588 (0.0011) [2023-12-27 01:25:22,702][105620] Updated weights for policy 1, policy_version 1379598 (0.0006) [2023-12-27 01:25:22,758][105620] Updated weights for policy 1, policy_version 1379608 (0.0006) [2023-12-27 01:25:22,843][105692] Updated weights for policy 0, policy_version 1377434 (0.0009) [2023-12-27 01:25:22,906][105692] Updated weights for policy 0, policy_version 1377444 (0.0008) [2023-12-27 01:25:22,966][105692] Updated weights for policy 0, policy_version 1377454 (0.0009) [2023-12-27 01:25:23,375][105620] Updated weights for policy 1, policy_version 1379618 (0.0007) [2023-12-27 01:25:23,426][105620] Updated weights for policy 1, policy_version 1379628 (0.0008) [2023-12-27 01:25:23,473][105620] Updated weights for policy 1, policy_version 1379638 (0.0009) [2023-12-27 01:25:23,709][105692] Updated weights for policy 0, policy_version 1377464 (0.0007) [2023-12-27 01:25:23,764][105692] Updated weights for policy 0, policy_version 1377474 (0.0005) [2023-12-27 01:25:23,823][105692] Updated weights for policy 0, policy_version 1377484 (0.0006) [2023-12-27 01:25:24,170][105620] Updated weights for policy 1, policy_version 1379649 (0.0010) [2023-12-27 01:25:24,222][105620] Updated weights for policy 1, policy_version 1379659 (0.0008) [2023-12-27 01:25:24,269][105620] Updated weights for policy 1, policy_version 1379669 (0.0008) [2023-12-27 01:25:24,316][105620] Updated weights for policy 1, policy_version 1379679 (0.0008) [2023-12-27 01:25:24,539][105692] Updated weights for policy 0, policy_version 1377494 (0.0011) [2023-12-27 01:25:24,587][105692] Updated weights for policy 0, policy_version 1377504 (0.0010) [2023-12-27 01:25:24,639][105692] Updated weights for policy 0, policy_version 1377514 (0.0010) [2023-12-27 01:25:25,164][105620] Updated weights for policy 1, policy_version 1379689 (0.0008) [2023-12-27 01:25:25,215][105620] Updated weights for policy 1, policy_version 1379699 (0.0009) [2023-12-27 01:25:25,238][105692] Updated weights for policy 0, policy_version 1377524 (0.0008) [2023-12-27 01:25:25,263][105620] Updated weights for policy 1, policy_version 1379709 (0.0008) [2023-12-27 01:25:25,305][105692] Updated weights for policy 0, policy_version 1377534 (0.0006) [2023-12-27 01:25:25,325][105585] KL-divergence is very high: 124.7625 [2023-12-27 01:25:25,367][105692] Updated weights for policy 0, policy_version 1377544 (0.0007) [2023-12-27 01:25:25,372][105585] KL-divergence is very high: 131.8936 [2023-12-27 01:25:26,027][105620] Updated weights for policy 1, policy_version 1379719 (0.0009) [2023-12-27 01:25:26,041][105692] Updated weights for policy 0, policy_version 1377554 (0.0011) [2023-12-27 01:25:26,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 705953792. Throughput: 0: 9740.7, 1: 9796.1. Samples: 705968724. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:25:26,062][104569] Avg episode reward: [(0, '8168.071'), (1, '8988.751')] [2023-12-27 01:25:26,079][105620] Updated weights for policy 1, policy_version 1379729 (0.0005) [2023-12-27 01:25:26,100][105692] Updated weights for policy 0, policy_version 1377564 (0.0010) [2023-12-27 01:25:26,141][105620] Updated weights for policy 1, policy_version 1379739 (0.0005) [2023-12-27 01:25:26,151][105692] Updated weights for policy 0, policy_version 1377574 (0.0010) [2023-12-27 01:25:26,216][105692] Updated weights for policy 0, policy_version 1377584 (0.0011) [2023-12-27 01:25:26,815][105620] Updated weights for policy 1, policy_version 1379749 (0.0007) [2023-12-27 01:25:26,866][105620] Updated weights for policy 1, policy_version 1379759 (0.0005) [2023-12-27 01:25:26,870][105692] Updated weights for policy 0, policy_version 1377594 (0.0011) [2023-12-27 01:25:26,919][105620] Updated weights for policy 1, policy_version 1379769 (0.0005) [2023-12-27 01:25:26,919][105692] Updated weights for policy 0, policy_version 1377604 (0.0007) [2023-12-27 01:25:26,975][105692] Updated weights for policy 0, policy_version 1377615 (0.0009) [2023-12-27 01:25:27,551][105620] Updated weights for policy 1, policy_version 1379779 (0.0007) [2023-12-27 01:25:27,608][105620] Updated weights for policy 1, policy_version 1379789 (0.0009) [2023-12-27 01:25:27,656][105620] Updated weights for policy 1, policy_version 1379799 (0.0005) [2023-12-27 01:25:27,670][105692] Updated weights for policy 0, policy_version 1377625 (0.0007) [2023-12-27 01:25:27,730][105692] Updated weights for policy 0, policy_version 1377635 (0.0005) [2023-12-27 01:25:27,792][105692] Updated weights for policy 0, policy_version 1377645 (0.0007) [2023-12-27 01:25:28,313][105620] Updated weights for policy 1, policy_version 1379809 (0.0005) [2023-12-27 01:25:28,376][105620] Updated weights for policy 1, policy_version 1379819 (0.0007) [2023-12-27 01:25:28,427][105692] Updated weights for policy 0, policy_version 1377655 (0.0007) [2023-12-27 01:25:28,429][105620] Updated weights for policy 1, policy_version 1379829 (0.0005) [2023-12-27 01:25:28,481][105620] Updated weights for policy 1, policy_version 1379839 (0.0005) [2023-12-27 01:25:28,485][105692] Updated weights for policy 0, policy_version 1377665 (0.0011) [2023-12-27 01:25:28,541][105692] Updated weights for policy 0, policy_version 1377675 (0.0010) [2023-12-27 01:25:29,046][105620] Updated weights for policy 1, policy_version 1379849 (0.0005) [2023-12-27 01:25:29,091][105620] Updated weights for policy 1, policy_version 1379859 (0.0005) [2023-12-27 01:25:29,145][105620] Updated weights for policy 1, policy_version 1379869 (0.0006) [2023-12-27 01:25:29,223][105692] Updated weights for policy 0, policy_version 1377685 (0.0010) [2023-12-27 01:25:29,285][105692] Updated weights for policy 0, policy_version 1377695 (0.0011) [2023-12-27 01:25:29,350][105692] Updated weights for policy 0, policy_version 1377705 (0.0011) [2023-12-27 01:25:29,840][105620] Updated weights for policy 1, policy_version 1379879 (0.0007) [2023-12-27 01:25:29,901][105620] Updated weights for policy 1, policy_version 1379889 (0.0007) [2023-12-27 01:25:29,971][105620] Updated weights for policy 1, policy_version 1379899 (0.0008) [2023-12-27 01:25:30,065][105692] Updated weights for policy 0, policy_version 1377715 (0.0011) [2023-12-27 01:25:30,120][105692] Updated weights for policy 0, policy_version 1377725 (0.0010) [2023-12-27 01:25:30,176][105692] Updated weights for policy 0, policy_version 1377735 (0.0010) [2023-12-27 01:25:30,660][105620] Updated weights for policy 1, policy_version 1379909 (0.0008) [2023-12-27 01:25:30,689][105586] KL-divergence is very high: 148.4965 [2023-12-27 01:25:30,713][105620] Updated weights for policy 1, policy_version 1379919 (0.0008) [2023-12-27 01:25:30,728][105586] KL-divergence is very high: 220.5643 [2023-12-27 01:25:30,772][105620] Updated weights for policy 1, policy_version 1379929 (0.0008) [2023-12-27 01:25:30,778][105586] KL-divergence is very high: 230.6434 [2023-12-27 01:25:30,941][105692] Updated weights for policy 0, policy_version 1377745 (0.0011) [2023-12-27 01:25:30,999][105692] Updated weights for policy 0, policy_version 1377755 (0.0010) [2023-12-27 01:25:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 706060288. Throughput: 0: 9748.7, 1: 9885.2. Samples: 706032344. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:25:31,063][104569] Avg episode reward: [(0, '8348.905'), (1, '8381.680')] [2023-12-27 01:25:31,065][105692] Updated weights for policy 0, policy_version 1377765 (0.0010) [2023-12-27 01:25:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001379936_353304576.pth... [2023-12-27 01:25:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001378784_353009664.pth [2023-12-27 01:25:31,122][105692] Updated weights for policy 0, policy_version 1377775 (0.0011) [2023-12-27 01:25:31,128][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001377776_352763904.pth... [2023-12-27 01:25:31,133][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001376624_352468992.pth [2023-12-27 01:25:31,505][105620] Updated weights for policy 1, policy_version 1379939 (0.0008) [2023-12-27 01:25:31,556][105620] Updated weights for policy 1, policy_version 1379949 (0.0008) [2023-12-27 01:25:31,600][105620] Updated weights for policy 1, policy_version 1379959 (0.0008) [2023-12-27 01:25:31,869][105692] Updated weights for policy 0, policy_version 1377785 (0.0010) [2023-12-27 01:25:31,931][105692] Updated weights for policy 0, policy_version 1377795 (0.0011) [2023-12-27 01:25:31,989][105692] Updated weights for policy 0, policy_version 1377805 (0.0011) [2023-12-27 01:25:32,368][105620] Updated weights for policy 1, policy_version 1379969 (0.0008) [2023-12-27 01:25:32,433][105620] Updated weights for policy 1, policy_version 1379979 (0.0008) [2023-12-27 01:25:32,498][105620] Updated weights for policy 1, policy_version 1379989 (0.0008) [2023-12-27 01:25:32,564][105620] Updated weights for policy 1, policy_version 1379999 (0.0008) [2023-12-27 01:25:32,744][105692] Updated weights for policy 0, policy_version 1377815 (0.0011) [2023-12-27 01:25:32,809][105692] Updated weights for policy 0, policy_version 1377825 (0.0009) [2023-12-27 01:25:32,860][105692] Updated weights for policy 0, policy_version 1377835 (0.0005) [2023-12-27 01:25:33,160][105620] Updated weights for policy 1, policy_version 1380009 (0.0006) [2023-12-27 01:25:33,210][105620] Updated weights for policy 1, policy_version 1380019 (0.0005) [2023-12-27 01:25:33,259][105620] Updated weights for policy 1, policy_version 1380029 (0.0005) [2023-12-27 01:25:33,474][105692] Updated weights for policy 0, policy_version 1377845 (0.0008) [2023-12-27 01:25:33,522][105692] Updated weights for policy 0, policy_version 1377855 (0.0009) [2023-12-27 01:25:33,572][105692] Updated weights for policy 0, policy_version 1377865 (0.0009) [2023-12-27 01:25:33,973][105620] Updated weights for policy 1, policy_version 1380039 (0.0009) [2023-12-27 01:25:34,031][105620] Updated weights for policy 1, policy_version 1380049 (0.0010) [2023-12-27 01:25:34,086][105620] Updated weights for policy 1, policy_version 1380059 (0.0008) [2023-12-27 01:25:34,160][105692] Updated weights for policy 0, policy_version 1377875 (0.0010) [2023-12-27 01:25:34,213][105692] Updated weights for policy 0, policy_version 1377885 (0.0009) [2023-12-27 01:25:34,269][105692] Updated weights for policy 0, policy_version 1377895 (0.0009) [2023-12-27 01:25:34,879][105620] Updated weights for policy 1, policy_version 1380069 (0.0009) [2023-12-27 01:25:34,930][105620] Updated weights for policy 1, policy_version 1380080 (0.0009) [2023-12-27 01:25:34,979][105620] Updated weights for policy 1, policy_version 1380090 (0.0008) [2023-12-27 01:25:35,050][105692] Updated weights for policy 0, policy_version 1377905 (0.0009) [2023-12-27 01:25:35,102][105692] Updated weights for policy 0, policy_version 1377915 (0.0009) [2023-12-27 01:25:35,164][105692] Updated weights for policy 0, policy_version 1377925 (0.0010) [2023-12-27 01:25:35,219][105692] Updated weights for policy 0, policy_version 1377936 (0.0010) [2023-12-27 01:25:35,717][105620] Updated weights for policy 1, policy_version 1380100 (0.0007) [2023-12-27 01:25:35,775][105620] Updated weights for policy 1, policy_version 1380110 (0.0010) [2023-12-27 01:25:35,828][105620] Updated weights for policy 1, policy_version 1380120 (0.0008) [2023-12-27 01:25:35,996][105692] Updated weights for policy 0, policy_version 1377946 (0.0011) [2023-12-27 01:25:36,058][105692] Updated weights for policy 0, policy_version 1377956 (0.0011) [2023-12-27 01:25:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 706158592. Throughput: 0: 9796.5, 1: 9717.4. Samples: 706149740. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:25:36,063][104569] Avg episode reward: [(0, '8436.312'), (1, '7947.668')] [2023-12-27 01:25:36,121][105692] Updated weights for policy 0, policy_version 1377966 (0.0008) [2023-12-27 01:25:36,583][105620] Updated weights for policy 1, policy_version 1380130 (0.0007) [2023-12-27 01:25:36,650][105620] Updated weights for policy 1, policy_version 1380140 (0.0008) [2023-12-27 01:25:36,694][105692] Updated weights for policy 0, policy_version 1377976 (0.0006) [2023-12-27 01:25:36,710][105620] Updated weights for policy 1, policy_version 1380150 (0.0009) [2023-12-27 01:25:36,751][105692] Updated weights for policy 0, policy_version 1377986 (0.0005) [2023-12-27 01:25:36,766][105620] Updated weights for policy 1, policy_version 1380160 (0.0009) [2023-12-27 01:25:36,807][105692] Updated weights for policy 0, policy_version 1377996 (0.0005) [2023-12-27 01:25:37,485][105620] Updated weights for policy 1, policy_version 1380170 (0.0008) [2023-12-27 01:25:37,507][105692] Updated weights for policy 0, policy_version 1378006 (0.0009) [2023-12-27 01:25:37,549][105620] Updated weights for policy 1, policy_version 1380180 (0.0007) [2023-12-27 01:25:37,558][105692] Updated weights for policy 0, policy_version 1378016 (0.0010) [2023-12-27 01:25:37,565][105585] KL-divergence is very high: 124.0527 [2023-12-27 01:25:37,598][105620] Updated weights for policy 1, policy_version 1380190 (0.0009) [2023-12-27 01:25:37,614][105585] KL-divergence is very high: 139.8112 [2023-12-27 01:25:37,621][105692] Updated weights for policy 0, policy_version 1378026 (0.0011) [2023-12-27 01:25:38,300][105620] Updated weights for policy 1, policy_version 1380200 (0.0008) [2023-12-27 01:25:38,340][105692] Updated weights for policy 0, policy_version 1378036 (0.0010) [2023-12-27 01:25:38,358][105620] Updated weights for policy 1, policy_version 1380210 (0.0008) [2023-12-27 01:25:38,401][105692] Updated weights for policy 0, policy_version 1378046 (0.0008) [2023-12-27 01:25:38,416][105620] Updated weights for policy 1, policy_version 1380220 (0.0007) [2023-12-27 01:25:38,457][105692] Updated weights for policy 0, policy_version 1378056 (0.0008) [2023-12-27 01:25:39,113][105620] Updated weights for policy 1, policy_version 1380230 (0.0006) [2023-12-27 01:25:39,177][105620] Updated weights for policy 1, policy_version 1380240 (0.0007) [2023-12-27 01:25:39,248][105620] Updated weights for policy 1, policy_version 1380250 (0.0009) [2023-12-27 01:25:39,265][105692] Updated weights for policy 0, policy_version 1378066 (0.0008) [2023-12-27 01:25:39,328][105692] Updated weights for policy 0, policy_version 1378076 (0.0007) [2023-12-27 01:25:39,396][105692] Updated weights for policy 0, policy_version 1378086 (0.0008) [2023-12-27 01:25:39,460][105692] Updated weights for policy 0, policy_version 1378096 (0.0008) [2023-12-27 01:25:39,948][105620] Updated weights for policy 1, policy_version 1380260 (0.0010) [2023-12-27 01:25:40,012][105620] Updated weights for policy 1, policy_version 1380270 (0.0011) [2023-12-27 01:25:40,070][105620] Updated weights for policy 1, policy_version 1380280 (0.0010) [2023-12-27 01:25:40,217][105692] Updated weights for policy 0, policy_version 1378106 (0.0007) [2023-12-27 01:25:40,278][105692] Updated weights for policy 0, policy_version 1378116 (0.0009) [2023-12-27 01:25:40,338][105692] Updated weights for policy 0, policy_version 1378126 (0.0008) [2023-12-27 01:25:40,821][105620] Updated weights for policy 1, policy_version 1380290 (0.0011) [2023-12-27 01:25:40,887][105620] Updated weights for policy 1, policy_version 1380300 (0.0011) [2023-12-27 01:25:40,948][105620] Updated weights for policy 1, policy_version 1380310 (0.0010) [2023-12-27 01:25:41,013][105620] Updated weights for policy 1, policy_version 1380320 (0.0010) [2023-12-27 01:25:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 706256896. Throughput: 0: 9740.3, 1: 9707.5. Samples: 706264392. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:25:41,062][104569] Avg episode reward: [(0, '8621.144'), (1, '6996.612')] [2023-12-27 01:25:41,079][105692] Updated weights for policy 0, policy_version 1378136 (0.0009) [2023-12-27 01:25:41,144][105692] Updated weights for policy 0, policy_version 1378146 (0.0010) [2023-12-27 01:25:41,205][105692] Updated weights for policy 0, policy_version 1378156 (0.0011) [2023-12-27 01:25:41,767][105620] Updated weights for policy 1, policy_version 1380330 (0.0011) [2023-12-27 01:25:41,833][105620] Updated weights for policy 1, policy_version 1380340 (0.0011) [2023-12-27 01:25:41,893][105620] Updated weights for policy 1, policy_version 1380350 (0.0011) [2023-12-27 01:25:41,956][105692] Updated weights for policy 0, policy_version 1378166 (0.0010) [2023-12-27 01:25:42,017][105692] Updated weights for policy 0, policy_version 1378176 (0.0011) [2023-12-27 01:25:42,081][105692] Updated weights for policy 0, policy_version 1378186 (0.0009) [2023-12-27 01:25:42,641][105620] Updated weights for policy 1, policy_version 1380360 (0.0010) [2023-12-27 01:25:42,701][105620] Updated weights for policy 1, policy_version 1380370 (0.0009) [2023-12-27 01:25:42,757][105620] Updated weights for policy 1, policy_version 1380380 (0.0009) [2023-12-27 01:25:42,790][105692] Updated weights for policy 0, policy_version 1378196 (0.0008) [2023-12-27 01:25:42,837][105692] Updated weights for policy 0, policy_version 1378206 (0.0009) [2023-12-27 01:25:42,890][105692] Updated weights for policy 0, policy_version 1378216 (0.0005) [2023-12-27 01:25:43,488][105692] Updated weights for policy 0, policy_version 1378226 (0.0006) [2023-12-27 01:25:43,539][105692] Updated weights for policy 0, policy_version 1378237 (0.0009) [2023-12-27 01:25:43,581][105620] Updated weights for policy 1, policy_version 1380390 (0.0008) [2023-12-27 01:25:43,585][105692] Updated weights for policy 0, policy_version 1378247 (0.0005) [2023-12-27 01:25:43,634][105620] Updated weights for policy 1, policy_version 1380400 (0.0009) [2023-12-27 01:25:43,687][105620] Updated weights for policy 1, policy_version 1380410 (0.0009) [2023-12-27 01:25:44,225][105692] Updated weights for policy 0, policy_version 1378257 (0.0006) [2023-12-27 01:25:44,274][105692] Updated weights for policy 0, policy_version 1378267 (0.0010) [2023-12-27 01:25:44,330][105692] Updated weights for policy 0, policy_version 1378277 (0.0009) [2023-12-27 01:25:44,386][105692] Updated weights for policy 0, policy_version 1378287 (0.0005) [2023-12-27 01:25:44,549][105620] Updated weights for policy 1, policy_version 1380420 (0.0010) [2023-12-27 01:25:44,601][105620] Updated weights for policy 1, policy_version 1380430 (0.0009) [2023-12-27 01:25:44,655][105620] Updated weights for policy 1, policy_version 1380440 (0.0010) [2023-12-27 01:25:45,035][105692] Updated weights for policy 0, policy_version 1378297 (0.0009) [2023-12-27 01:25:45,101][105692] Updated weights for policy 0, policy_version 1378307 (0.0009) [2023-12-27 01:25:45,156][105692] Updated weights for policy 0, policy_version 1378317 (0.0006) [2023-12-27 01:25:45,463][105620] Updated weights for policy 1, policy_version 1380451 (0.0009) [2023-12-27 01:25:45,512][105620] Updated weights for policy 1, policy_version 1380461 (0.0009) [2023-12-27 01:25:45,562][105620] Updated weights for policy 1, policy_version 1380471 (0.0009) [2023-12-27 01:25:45,884][105692] Updated weights for policy 0, policy_version 1378327 (0.0008) [2023-12-27 01:25:45,930][105692] Updated weights for policy 0, policy_version 1378337 (0.0006) [2023-12-27 01:25:45,984][105692] Updated weights for policy 0, policy_version 1378347 (0.0008) [2023-12-27 01:25:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 706355200. Throughput: 0: 9756.2, 1: 9620.7. Samples: 706320504. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:25:46,063][104569] Avg episode reward: [(0, '8528.951'), (1, '6162.783')] [2023-12-27 01:25:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001378352_352911360.pth... [2023-12-27 01:25:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001380480_353443840.pth... [2023-12-27 01:25:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001377200_352616448.pth [2023-12-27 01:25:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001379360_353157120.pth [2023-12-27 01:25:46,257][105620] Updated weights for policy 1, policy_version 1380481 (0.0008) [2023-12-27 01:25:46,311][105620] Updated weights for policy 1, policy_version 1380491 (0.0010) [2023-12-27 01:25:46,368][105620] Updated weights for policy 1, policy_version 1380501 (0.0010) [2023-12-27 01:25:46,421][105620] Updated weights for policy 1, policy_version 1380511 (0.0009) [2023-12-27 01:25:46,616][105692] Updated weights for policy 0, policy_version 1378357 (0.0009) [2023-12-27 01:25:46,664][105692] Updated weights for policy 0, policy_version 1378367 (0.0009) [2023-12-27 01:25:46,711][105692] Updated weights for policy 0, policy_version 1378377 (0.0009) [2023-12-27 01:25:47,242][105620] Updated weights for policy 1, policy_version 1380521 (0.0008) [2023-12-27 01:25:47,296][105620] Updated weights for policy 1, policy_version 1380531 (0.0008) [2023-12-27 01:25:47,358][105620] Updated weights for policy 1, policy_version 1380541 (0.0008) [2023-12-27 01:25:47,455][105692] Updated weights for policy 0, policy_version 1378387 (0.0009) [2023-12-27 01:25:47,499][105692] Updated weights for policy 0, policy_version 1378397 (0.0010) [2023-12-27 01:25:47,544][105692] Updated weights for policy 0, policy_version 1378407 (0.0010) [2023-12-27 01:25:48,104][105620] Updated weights for policy 1, policy_version 1380551 (0.0006) [2023-12-27 01:25:48,161][105620] Updated weights for policy 1, policy_version 1380561 (0.0006) [2023-12-27 01:25:48,221][105620] Updated weights for policy 1, policy_version 1380571 (0.0008) [2023-12-27 01:25:48,249][105692] Updated weights for policy 0, policy_version 1378417 (0.0009) [2023-12-27 01:25:48,307][105692] Updated weights for policy 0, policy_version 1378427 (0.0010) [2023-12-27 01:25:48,362][105692] Updated weights for policy 0, policy_version 1378437 (0.0007) [2023-12-27 01:25:48,422][105692] Updated weights for policy 0, policy_version 1378447 (0.0010) [2023-12-27 01:25:48,948][105620] Updated weights for policy 1, policy_version 1380581 (0.0008) [2023-12-27 01:25:48,996][105620] Updated weights for policy 1, policy_version 1380591 (0.0008) [2023-12-27 01:25:49,051][105620] Updated weights for policy 1, policy_version 1380601 (0.0007) [2023-12-27 01:25:49,158][105692] Updated weights for policy 0, policy_version 1378457 (0.0011) [2023-12-27 01:25:49,214][105692] Updated weights for policy 0, policy_version 1378467 (0.0010) [2023-12-27 01:25:49,282][105692] Updated weights for policy 0, policy_version 1378477 (0.0008) [2023-12-27 01:25:49,795][105620] Updated weights for policy 1, policy_version 1380611 (0.0009) [2023-12-27 01:25:49,857][105620] Updated weights for policy 1, policy_version 1380621 (0.0009) [2023-12-27 01:25:49,923][105620] Updated weights for policy 1, policy_version 1380631 (0.0009) [2023-12-27 01:25:50,119][105692] Updated weights for policy 0, policy_version 1378487 (0.0010) [2023-12-27 01:25:50,180][105692] Updated weights for policy 0, policy_version 1378497 (0.0011) [2023-12-27 01:25:50,243][105692] Updated weights for policy 0, policy_version 1378507 (0.0011) [2023-12-27 01:25:50,679][105620] Updated weights for policy 1, policy_version 1380641 (0.0008) [2023-12-27 01:25:50,730][105620] Updated weights for policy 1, policy_version 1380651 (0.0008) [2023-12-27 01:25:50,782][105620] Updated weights for policy 1, policy_version 1380661 (0.0008) [2023-12-27 01:25:50,843][105620] Updated weights for policy 1, policy_version 1380671 (0.0008) [2023-12-27 01:25:51,011][105692] Updated weights for policy 0, policy_version 1378517 (0.0010) [2023-12-27 01:25:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 706445312. Throughput: 0: 9731.8, 1: 9588.0. Samples: 706436844. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:25:51,063][104569] Avg episode reward: [(0, '8160.723'), (1, '3775.134')] [2023-12-27 01:25:51,073][105692] Updated weights for policy 0, policy_version 1378527 (0.0009) [2023-12-27 01:25:51,132][105692] Updated weights for policy 0, policy_version 1378537 (0.0010) [2023-12-27 01:25:51,588][105620] Updated weights for policy 1, policy_version 1380681 (0.0009) [2023-12-27 01:25:51,650][105620] Updated weights for policy 1, policy_version 1380691 (0.0007) [2023-12-27 01:25:51,709][105620] Updated weights for policy 1, policy_version 1380701 (0.0007) [2023-12-27 01:25:51,913][105692] Updated weights for policy 0, policy_version 1378547 (0.0010) [2023-12-27 01:25:51,977][105692] Updated weights for policy 0, policy_version 1378557 (0.0009) [2023-12-27 01:25:52,044][105692] Updated weights for policy 0, policy_version 1378567 (0.0010) [2023-12-27 01:25:52,519][105620] Updated weights for policy 1, policy_version 1380711 (0.0009) [2023-12-27 01:25:52,570][105620] Updated weights for policy 1, policy_version 1380721 (0.0008) [2023-12-27 01:25:52,627][105620] Updated weights for policy 1, policy_version 1380731 (0.0009) [2023-12-27 01:25:52,726][105692] Updated weights for policy 0, policy_version 1378577 (0.0009) [2023-12-27 01:25:52,779][105692] Updated weights for policy 0, policy_version 1378587 (0.0009) [2023-12-27 01:25:52,840][105692] Updated weights for policy 0, policy_version 1378597 (0.0009) [2023-12-27 01:25:52,902][105692] Updated weights for policy 0, policy_version 1378607 (0.0009) [2023-12-27 01:25:53,370][105620] Updated weights for policy 1, policy_version 1380741 (0.0009) [2023-12-27 01:25:53,417][105620] Updated weights for policy 1, policy_version 1380751 (0.0009) [2023-12-27 01:25:53,463][105586] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000003 [2023-12-27 01:25:53,680][105692] Updated weights for policy 0, policy_version 1378617 (0.0008) [2023-12-27 01:25:53,726][105692] Updated weights for policy 0, policy_version 1378627 (0.0008) [2023-12-27 01:25:53,781][105692] Updated weights for policy 0, policy_version 1378637 (0.0009) [2023-12-27 01:25:54,202][105620] Updated weights for policy 1, policy_version 1380761 (0.0009) [2023-12-27 01:25:54,267][105620] Updated weights for policy 1, policy_version 1380771 (0.0009) [2023-12-27 01:25:54,328][105620] Updated weights for policy 1, policy_version 1380781 (0.0008) [2023-12-27 01:25:54,390][105620] Updated weights for policy 1, policy_version 1380791 (0.0008) [2023-12-27 01:25:54,592][105692] Updated weights for policy 0, policy_version 1378647 (0.0009) [2023-12-27 01:25:54,652][105692] Updated weights for policy 0, policy_version 1378657 (0.0005) [2023-12-27 01:25:54,707][105692] Updated weights for policy 0, policy_version 1378667 (0.0005) [2023-12-27 01:25:55,101][105620] Updated weights for policy 1, policy_version 1380801 (0.0008) [2023-12-27 01:25:55,157][105620] Updated weights for policy 1, policy_version 1380811 (0.0008) [2023-12-27 01:25:55,204][105620] Updated weights for policy 1, policy_version 1380821 (0.0008) [2023-12-27 01:25:55,370][105692] Updated weights for policy 0, policy_version 1378677 (0.0007) [2023-12-27 01:25:55,424][105692] Updated weights for policy 0, policy_version 1378687 (0.0009) [2023-12-27 01:25:55,475][105692] Updated weights for policy 0, policy_version 1378697 (0.0009) [2023-12-27 01:25:55,955][105620] Updated weights for policy 1, policy_version 1380831 (0.0009) [2023-12-27 01:25:56,007][105620] Updated weights for policy 1, policy_version 1380841 (0.0010) [2023-12-27 01:25:56,062][104569] Fps is (10 sec: 18023.0, 60 sec: 19251.2, 300 sec: 19355.3). Total num frames: 706535424. Throughput: 0: 9685.1, 1: 9564.3. Samples: 706549208. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:25:56,062][104569] Avg episode reward: [(0, '7886.791'), (1, '4703.611')] [2023-12-27 01:25:56,064][105620] Updated weights for policy 1, policy_version 1380851 (0.0010) [2023-12-27 01:25:56,252][105692] Updated weights for policy 0, policy_version 1378707 (0.0009) [2023-12-27 01:25:56,299][105692] Updated weights for policy 0, policy_version 1378717 (0.0007) [2023-12-27 01:25:56,360][105692] Updated weights for policy 0, policy_version 1378727 (0.0008) [2023-12-27 01:25:56,827][105620] Updated weights for policy 1, policy_version 1380861 (0.0008) [2023-12-27 01:25:56,881][105620] Updated weights for policy 1, policy_version 1380871 (0.0005) [2023-12-27 01:25:56,934][105620] Updated weights for policy 1, policy_version 1380881 (0.0008) [2023-12-27 01:25:57,139][105692] Updated weights for policy 0, policy_version 1378737 (0.0008) [2023-12-27 01:25:57,195][105692] Updated weights for policy 0, policy_version 1378747 (0.0010) [2023-12-27 01:25:57,252][105692] Updated weights for policy 0, policy_version 1378757 (0.0007) [2023-12-27 01:25:57,305][105692] Updated weights for policy 0, policy_version 1378767 (0.0007) [2023-12-27 01:25:57,557][105620] Updated weights for policy 1, policy_version 1380891 (0.0010) [2023-12-27 01:25:57,610][105620] Updated weights for policy 1, policy_version 1380901 (0.0010) [2023-12-27 01:25:57,667][105620] Updated weights for policy 1, policy_version 1380911 (0.0010) [2023-12-27 01:25:58,094][105692] Updated weights for policy 0, policy_version 1378777 (0.0005) [2023-12-27 01:25:58,158][105692] Updated weights for policy 0, policy_version 1378787 (0.0007) [2023-12-27 01:25:58,221][105692] Updated weights for policy 0, policy_version 1378797 (0.0008) [2023-12-27 01:25:58,413][105620] Updated weights for policy 1, policy_version 1380921 (0.0010) [2023-12-27 01:25:58,476][105620] Updated weights for policy 1, policy_version 1380931 (0.0007) [2023-12-27 01:25:58,542][105620] Updated weights for policy 1, policy_version 1380941 (0.0007) [2023-12-27 01:25:58,602][105620] Updated weights for policy 1, policy_version 1380951 (0.0009) [2023-12-27 01:25:59,040][105692] Updated weights for policy 0, policy_version 1378807 (0.0010) [2023-12-27 01:25:59,094][105692] Updated weights for policy 0, policy_version 1378817 (0.0010) [2023-12-27 01:25:59,142][105692] Updated weights for policy 0, policy_version 1378827 (0.0011) [2023-12-27 01:25:59,419][105620] Updated weights for policy 1, policy_version 1380961 (0.0010) [2023-12-27 01:25:59,481][105620] Updated weights for policy 1, policy_version 1380971 (0.0010) [2023-12-27 01:25:59,536][105620] Updated weights for policy 1, policy_version 1380981 (0.0010) [2023-12-27 01:25:59,861][105692] Updated weights for policy 0, policy_version 1378837 (0.0009) [2023-12-27 01:25:59,914][105692] Updated weights for policy 0, policy_version 1378847 (0.0007) [2023-12-27 01:25:59,979][105692] Updated weights for policy 0, policy_version 1378857 (0.0009) [2023-12-27 01:26:00,208][105620] Updated weights for policy 1, policy_version 1380991 (0.0010) [2023-12-27 01:26:00,266][105620] Updated weights for policy 1, policy_version 1381001 (0.0010) [2023-12-27 01:26:00,324][105620] Updated weights for policy 1, policy_version 1381011 (0.0010) [2023-12-27 01:26:00,591][105692] Updated weights for policy 0, policy_version 1378867 (0.0007) [2023-12-27 01:26:00,648][105692] Updated weights for policy 0, policy_version 1378877 (0.0007) [2023-12-27 01:26:00,674][105585] KL-divergence is very high: 125.7395 [2023-12-27 01:26:00,709][105692] Updated weights for policy 0, policy_version 1378887 (0.0010) [2023-12-27 01:26:00,722][105585] KL-divergence is very high: 131.0287 [2023-12-27 01:26:01,054][105620] Updated weights for policy 1, policy_version 1381021 (0.0010) [2023-12-27 01:26:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19355.4). Total num frames: 706633728. Throughput: 0: 9683.5, 1: 9573.4. Samples: 706605736. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:26:01,063][104569] Avg episode reward: [(0, '7982.178'), (1, '6951.470')] [2023-12-27 01:26:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001378896_353050624.pth... [2023-12-27 01:26:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001377776_352763904.pth [2023-12-27 01:26:01,106][105620] Updated weights for policy 1, policy_version 1381031 (0.0011) [2023-12-27 01:26:01,170][105620] Updated weights for policy 1, policy_version 1381041 (0.0010) [2023-12-27 01:26:01,211][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001381048_353591296.pth... [2023-12-27 01:26:01,215][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001379936_353304576.pth [2023-12-27 01:26:01,370][105692] Updated weights for policy 0, policy_version 1378897 (0.0006) [2023-12-27 01:26:01,439][105692] Updated weights for policy 0, policy_version 1378907 (0.0008) [2023-12-27 01:26:01,508][105692] Updated weights for policy 0, policy_version 1378917 (0.0008) [2023-12-27 01:26:01,560][105692] Updated weights for policy 0, policy_version 1378927 (0.0008) [2023-12-27 01:26:01,797][105620] Updated weights for policy 1, policy_version 1381051 (0.0010) [2023-12-27 01:26:01,849][105620] Updated weights for policy 1, policy_version 1381061 (0.0006) [2023-12-27 01:26:01,907][105620] Updated weights for policy 1, policy_version 1381071 (0.0005) [2023-12-27 01:26:02,315][105692] Updated weights for policy 0, policy_version 1378937 (0.0011) [2023-12-27 01:26:02,388][105692] Updated weights for policy 0, policy_version 1378947 (0.0009) [2023-12-27 01:26:02,448][105692] Updated weights for policy 0, policy_version 1378957 (0.0010) [2023-12-27 01:26:02,552][105620] Updated weights for policy 1, policy_version 1381081 (0.0006) [2023-12-27 01:26:02,607][105620] Updated weights for policy 1, policy_version 1381091 (0.0010) [2023-12-27 01:26:02,662][105620] Updated weights for policy 1, policy_version 1381101 (0.0010) [2023-12-27 01:26:02,710][105620] Updated weights for policy 1, policy_version 1381111 (0.0010) [2023-12-27 01:26:03,198][105692] Updated weights for policy 0, policy_version 1378967 (0.0011) [2023-12-27 01:26:03,246][105692] Updated weights for policy 0, policy_version 1378977 (0.0010) [2023-12-27 01:26:03,295][105692] Updated weights for policy 0, policy_version 1378987 (0.0009) [2023-12-27 01:26:03,453][105620] Updated weights for policy 1, policy_version 1381121 (0.0010) [2023-12-27 01:26:03,507][105620] Updated weights for policy 1, policy_version 1381131 (0.0010) [2023-12-27 01:26:03,567][105620] Updated weights for policy 1, policy_version 1381141 (0.0010) [2023-12-27 01:26:04,066][105692] Updated weights for policy 0, policy_version 1378997 (0.0009) [2023-12-27 01:26:04,119][105692] Updated weights for policy 0, policy_version 1379007 (0.0011) [2023-12-27 01:26:04,175][105692] Updated weights for policy 0, policy_version 1379017 (0.0009) [2023-12-27 01:26:04,311][105620] Updated weights for policy 1, policy_version 1381151 (0.0010) [2023-12-27 01:26:04,367][105620] Updated weights for policy 1, policy_version 1381161 (0.0011) [2023-12-27 01:26:04,430][105620] Updated weights for policy 1, policy_version 1381171 (0.0011) [2023-12-27 01:26:04,884][105692] Updated weights for policy 0, policy_version 1379027 (0.0009) [2023-12-27 01:26:04,946][105692] Updated weights for policy 0, policy_version 1379037 (0.0006) [2023-12-27 01:26:05,001][105692] Updated weights for policy 0, policy_version 1379047 (0.0010) [2023-12-27 01:26:05,180][105620] Updated weights for policy 1, policy_version 1381181 (0.0010) [2023-12-27 01:26:05,231][105620] Updated weights for policy 1, policy_version 1381191 (0.0010) [2023-12-27 01:26:05,286][105620] Updated weights for policy 1, policy_version 1381201 (0.0010) [2023-12-27 01:26:05,517][105692] Updated weights for policy 0, policy_version 1379057 (0.0005) [2023-12-27 01:26:05,577][105692] Updated weights for policy 0, policy_version 1379067 (0.0006) [2023-12-27 01:26:05,635][105692] Updated weights for policy 0, policy_version 1379077 (0.0006) [2023-12-27 01:26:05,698][105692] Updated weights for policy 0, policy_version 1379087 (0.0008) [2023-12-27 01:26:06,028][105620] Updated weights for policy 1, policy_version 1381211 (0.0010) [2023-12-27 01:26:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 706732032. Throughput: 0: 9665.4, 1: 9627.4. Samples: 706722608. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:26:06,062][104569] Avg episode reward: [(0, '7524.094'), (1, '7421.629')] [2023-12-27 01:26:06,097][105620] Updated weights for policy 1, policy_version 1381221 (0.0010) [2023-12-27 01:26:06,168][105620] Updated weights for policy 1, policy_version 1381231 (0.0010) [2023-12-27 01:26:06,332][105692] Updated weights for policy 0, policy_version 1379097 (0.0010) [2023-12-27 01:26:06,391][105692] Updated weights for policy 0, policy_version 1379107 (0.0011) [2023-12-27 01:26:06,453][105692] Updated weights for policy 0, policy_version 1379117 (0.0010) [2023-12-27 01:26:06,934][105620] Updated weights for policy 1, policy_version 1381241 (0.0009) [2023-12-27 01:26:06,981][105620] Updated weights for policy 1, policy_version 1381251 (0.0008) [2023-12-27 01:26:07,036][105620] Updated weights for policy 1, policy_version 1381261 (0.0009) [2023-12-27 01:26:07,087][105620] Updated weights for policy 1, policy_version 1381271 (0.0009) [2023-12-27 01:26:07,195][105692] Updated weights for policy 0, policy_version 1379127 (0.0010) [2023-12-27 01:26:07,248][105692] Updated weights for policy 0, policy_version 1379137 (0.0009) [2023-12-27 01:26:07,300][105692] Updated weights for policy 0, policy_version 1379147 (0.0009) [2023-12-27 01:26:07,832][105620] Updated weights for policy 1, policy_version 1381281 (0.0009) [2023-12-27 01:26:07,889][105620] Updated weights for policy 1, policy_version 1381291 (0.0009) [2023-12-27 01:26:07,937][105692] Updated weights for policy 0, policy_version 1379157 (0.0007) [2023-12-27 01:26:07,942][105620] Updated weights for policy 1, policy_version 1381301 (0.0008) [2023-12-27 01:26:07,987][105692] Updated weights for policy 0, policy_version 1379167 (0.0005) [2023-12-27 01:26:08,039][105692] Updated weights for policy 0, policy_version 1379177 (0.0007) [2023-12-27 01:26:08,656][105620] Updated weights for policy 1, policy_version 1381311 (0.0010) [2023-12-27 01:26:08,703][105692] Updated weights for policy 0, policy_version 1379187 (0.0007) [2023-12-27 01:26:08,713][105620] Updated weights for policy 1, policy_version 1381321 (0.0010) [2023-12-27 01:26:08,764][105692] Updated weights for policy 0, policy_version 1379197 (0.0008) [2023-12-27 01:26:08,767][105620] Updated weights for policy 1, policy_version 1381331 (0.0006) [2023-12-27 01:26:08,818][105692] Updated weights for policy 0, policy_version 1379207 (0.0009) [2023-12-27 01:26:09,504][105620] Updated weights for policy 1, policy_version 1381341 (0.0007) [2023-12-27 01:26:09,567][105620] Updated weights for policy 1, policy_version 1381351 (0.0009) [2023-12-27 01:26:09,613][105692] Updated weights for policy 0, policy_version 1379217 (0.0009) [2023-12-27 01:26:09,623][105620] Updated weights for policy 1, policy_version 1381361 (0.0008) [2023-12-27 01:26:09,675][105692] Updated weights for policy 0, policy_version 1379227 (0.0008) [2023-12-27 01:26:09,738][105692] Updated weights for policy 0, policy_version 1379237 (0.0009) [2023-12-27 01:26:09,801][105692] Updated weights for policy 0, policy_version 1379247 (0.0009) [2023-12-27 01:26:10,398][105620] Updated weights for policy 1, policy_version 1381371 (0.0007) [2023-12-27 01:26:10,460][105620] Updated weights for policy 1, policy_version 1381381 (0.0008) [2023-12-27 01:26:10,526][105620] Updated weights for policy 1, policy_version 1381391 (0.0007) [2023-12-27 01:26:10,586][105692] Updated weights for policy 0, policy_version 1379257 (0.0007) [2023-12-27 01:26:10,638][105692] Updated weights for policy 0, policy_version 1379267 (0.0008) [2023-12-27 01:26:10,690][105692] Updated weights for policy 0, policy_version 1379277 (0.0008) [2023-12-27 01:26:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 706830336. Throughput: 0: 9765.8, 1: 9605.5. Samples: 706840432. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:26:11,063][104569] Avg episode reward: [(0, '7615.530'), (1, '6796.423')] [2023-12-27 01:26:11,233][105620] Updated weights for policy 1, policy_version 1381401 (0.0010) [2023-12-27 01:26:11,298][105620] Updated weights for policy 1, policy_version 1381411 (0.0011) [2023-12-27 01:26:11,359][105620] Updated weights for policy 1, policy_version 1381421 (0.0011) [2023-12-27 01:26:11,435][105620] Updated weights for policy 1, policy_version 1381431 (0.0009) [2023-12-27 01:26:11,541][105692] Updated weights for policy 0, policy_version 1379287 (0.0007) [2023-12-27 01:26:11,613][105692] Updated weights for policy 0, policy_version 1379297 (0.0006) [2023-12-27 01:26:11,673][105692] Updated weights for policy 0, policy_version 1379307 (0.0010) [2023-12-27 01:26:12,193][105620] Updated weights for policy 1, policy_version 1381441 (0.0010) [2023-12-27 01:26:12,260][105620] Updated weights for policy 1, policy_version 1381451 (0.0011) [2023-12-27 01:26:12,328][105620] Updated weights for policy 1, policy_version 1381461 (0.0011) [2023-12-27 01:26:12,407][105692] Updated weights for policy 0, policy_version 1379317 (0.0009) [2023-12-27 01:26:12,479][105692] Updated weights for policy 0, policy_version 1379327 (0.0008) [2023-12-27 01:26:12,547][105692] Updated weights for policy 0, policy_version 1379337 (0.0007) [2023-12-27 01:26:13,018][105620] Updated weights for policy 1, policy_version 1381471 (0.0009) [2023-12-27 01:26:13,072][105620] Updated weights for policy 1, policy_version 1381481 (0.0010) [2023-12-27 01:26:13,124][105586] KL-divergence is very high: 180.7815 [2023-12-27 01:26:13,134][105620] Updated weights for policy 1, policy_version 1381491 (0.0010) [2023-12-27 01:26:13,204][105692] Updated weights for policy 0, policy_version 1379347 (0.0007) [2023-12-27 01:26:13,261][105692] Updated weights for policy 0, policy_version 1379357 (0.0013) [2023-12-27 01:26:13,318][105692] Updated weights for policy 0, policy_version 1379368 (0.0008) [2023-12-27 01:26:13,728][105620] Updated weights for policy 1, policy_version 1381501 (0.0008) [2023-12-27 01:26:13,777][105620] Updated weights for policy 1, policy_version 1381511 (0.0010) [2023-12-27 01:26:13,834][105620] Updated weights for policy 1, policy_version 1381521 (0.0008) [2023-12-27 01:26:13,948][105692] Updated weights for policy 0, policy_version 1379378 (0.0005) [2023-12-27 01:26:13,997][105692] Updated weights for policy 0, policy_version 1379388 (0.0005) [2023-12-27 01:26:14,056][105692] Updated weights for policy 0, policy_version 1379398 (0.0005) [2023-12-27 01:26:14,126][105692] Updated weights for policy 0, policy_version 1379408 (0.0006) [2023-12-27 01:26:14,498][105620] Updated weights for policy 1, policy_version 1381531 (0.0007) [2023-12-27 01:26:14,551][105620] Updated weights for policy 1, policy_version 1381541 (0.0009) [2023-12-27 01:26:14,596][105620] Updated weights for policy 1, policy_version 1381551 (0.0008) [2023-12-27 01:26:14,736][105692] Updated weights for policy 0, policy_version 1379418 (0.0009) [2023-12-27 01:26:14,799][105692] Updated weights for policy 0, policy_version 1379428 (0.0007) [2023-12-27 01:26:14,853][105692] Updated weights for policy 0, policy_version 1379438 (0.0008) [2023-12-27 01:26:15,333][105620] Updated weights for policy 1, policy_version 1381561 (0.0006) [2023-12-27 01:26:15,384][105620] Updated weights for policy 1, policy_version 1381571 (0.0009) [2023-12-27 01:26:15,449][105620] Updated weights for policy 1, policy_version 1381581 (0.0010) [2023-12-27 01:26:15,504][105620] Updated weights for policy 1, policy_version 1381591 (0.0010) [2023-12-27 01:26:15,628][105692] Updated weights for policy 0, policy_version 1379448 (0.0009) [2023-12-27 01:26:15,684][105692] Updated weights for policy 0, policy_version 1379458 (0.0008) [2023-12-27 01:26:15,729][105692] Updated weights for policy 0, policy_version 1379468 (0.0007) [2023-12-27 01:26:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 706928640. Throughput: 0: 9702.9, 1: 9535.3. Samples: 706898064. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:26:16,063][104569] Avg episode reward: [(0, '7614.664'), (1, '7545.126')] [2023-12-27 01:26:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001379472_353198080.pth... [2023-12-27 01:26:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001381592_353730560.pth... [2023-12-27 01:26:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001378352_352911360.pth [2023-12-27 01:26:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001380480_353443840.pth [2023-12-27 01:26:16,255][105620] Updated weights for policy 1, policy_version 1381601 (0.0008) [2023-12-27 01:26:16,306][105620] Updated weights for policy 1, policy_version 1381611 (0.0005) [2023-12-27 01:26:16,362][105620] Updated weights for policy 1, policy_version 1381621 (0.0007) [2023-12-27 01:26:16,469][105692] Updated weights for policy 0, policy_version 1379478 (0.0006) [2023-12-27 01:26:16,532][105692] Updated weights for policy 0, policy_version 1379488 (0.0009) [2023-12-27 01:26:16,588][105692] Updated weights for policy 0, policy_version 1379498 (0.0010) [2023-12-27 01:26:17,114][105620] Updated weights for policy 1, policy_version 1381631 (0.0005) [2023-12-27 01:26:17,164][105620] Updated weights for policy 1, policy_version 1381641 (0.0005) [2023-12-27 01:26:17,186][105692] Updated weights for policy 0, policy_version 1379508 (0.0009) [2023-12-27 01:26:17,220][105620] Updated weights for policy 1, policy_version 1381651 (0.0005) [2023-12-27 01:26:17,246][105692] Updated weights for policy 0, policy_version 1379518 (0.0005) [2023-12-27 01:26:17,310][105692] Updated weights for policy 0, policy_version 1379528 (0.0005) [2023-12-27 01:26:17,741][105620] Updated weights for policy 1, policy_version 1381661 (0.0005) [2023-12-27 01:26:17,802][105620] Updated weights for policy 1, policy_version 1381671 (0.0006) [2023-12-27 01:26:17,863][105620] Updated weights for policy 1, policy_version 1381681 (0.0010) [2023-12-27 01:26:17,888][105692] Updated weights for policy 0, policy_version 1379538 (0.0008) [2023-12-27 01:26:17,949][105692] Updated weights for policy 0, policy_version 1379548 (0.0010) [2023-12-27 01:26:18,010][105692] Updated weights for policy 0, policy_version 1379558 (0.0010) [2023-12-27 01:26:18,068][105692] Updated weights for policy 0, policy_version 1379568 (0.0009) [2023-12-27 01:26:18,464][105620] Updated weights for policy 1, policy_version 1381691 (0.0010) [2023-12-27 01:26:18,524][105620] Updated weights for policy 1, policy_version 1381701 (0.0011) [2023-12-27 01:26:18,591][105620] Updated weights for policy 1, policy_version 1381711 (0.0011) [2023-12-27 01:26:18,798][105692] Updated weights for policy 0, policy_version 1379578 (0.0010) [2023-12-27 01:26:18,847][105692] Updated weights for policy 0, policy_version 1379588 (0.0010) [2023-12-27 01:26:18,902][105692] Updated weights for policy 0, policy_version 1379598 (0.0010) [2023-12-27 01:26:19,342][105620] Updated weights for policy 1, policy_version 1381721 (0.0011) [2023-12-27 01:26:19,407][105620] Updated weights for policy 1, policy_version 1381731 (0.0011) [2023-12-27 01:26:19,455][105620] Updated weights for policy 1, policy_version 1381741 (0.0011) [2023-12-27 01:26:19,520][105620] Updated weights for policy 1, policy_version 1381751 (0.0007) [2023-12-27 01:26:19,660][105692] Updated weights for policy 0, policy_version 1379608 (0.0011) [2023-12-27 01:26:19,718][105692] Updated weights for policy 0, policy_version 1379618 (0.0011) [2023-12-27 01:26:19,777][105692] Updated weights for policy 0, policy_version 1379628 (0.0011) [2023-12-27 01:26:20,254][105620] Updated weights for policy 1, policy_version 1381761 (0.0006) [2023-12-27 01:26:20,309][105620] Updated weights for policy 1, policy_version 1381771 (0.0006) [2023-12-27 01:26:20,377][105620] Updated weights for policy 1, policy_version 1381781 (0.0008) [2023-12-27 01:26:20,520][105692] Updated weights for policy 0, policy_version 1379638 (0.0008) [2023-12-27 01:26:20,589][105692] Updated weights for policy 0, policy_version 1379648 (0.0006) [2023-12-27 01:26:20,659][105692] Updated weights for policy 0, policy_version 1379658 (0.0007) [2023-12-27 01:26:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 707026944. Throughput: 0: 9754.5, 1: 9581.2. Samples: 707019844. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:26:21,063][104569] Avg episode reward: [(0, '8252.699'), (1, '8635.510')] [2023-12-27 01:26:21,086][105620] Updated weights for policy 1, policy_version 1381791 (0.0009) [2023-12-27 01:26:21,156][105620] Updated weights for policy 1, policy_version 1381801 (0.0010) [2023-12-27 01:26:21,221][105620] Updated weights for policy 1, policy_version 1381811 (0.0011) [2023-12-27 01:26:21,383][105692] Updated weights for policy 0, policy_version 1379668 (0.0008) [2023-12-27 01:26:21,444][105692] Updated weights for policy 0, policy_version 1379678 (0.0011) [2023-12-27 01:26:21,490][105692] Updated weights for policy 0, policy_version 1379688 (0.0011) [2023-12-27 01:26:21,985][105620] Updated weights for policy 1, policy_version 1381821 (0.0011) [2023-12-27 01:26:22,043][105620] Updated weights for policy 1, policy_version 1381831 (0.0009) [2023-12-27 01:26:22,109][105620] Updated weights for policy 1, policy_version 1381841 (0.0006) [2023-12-27 01:26:22,177][105692] Updated weights for policy 0, policy_version 1379698 (0.0011) [2023-12-27 01:26:22,241][105692] Updated weights for policy 0, policy_version 1379708 (0.0011) [2023-12-27 01:26:22,306][105692] Updated weights for policy 0, policy_version 1379718 (0.0011) [2023-12-27 01:26:22,367][105692] Updated weights for policy 0, policy_version 1379728 (0.0011) [2023-12-27 01:26:22,838][105620] Updated weights for policy 1, policy_version 1381851 (0.0006) [2023-12-27 01:26:22,903][105620] Updated weights for policy 1, policy_version 1381861 (0.0007) [2023-12-27 01:26:22,961][105620] Updated weights for policy 1, policy_version 1381871 (0.0010) [2023-12-27 01:26:23,009][105692] Updated weights for policy 0, policy_version 1379738 (0.0005) [2023-12-27 01:26:23,061][105692] Updated weights for policy 0, policy_version 1379748 (0.0006) [2023-12-27 01:26:23,114][105692] Updated weights for policy 0, policy_version 1379758 (0.0011) [2023-12-27 01:26:23,706][105620] Updated weights for policy 1, policy_version 1381881 (0.0007) [2023-12-27 01:26:23,757][105620] Updated weights for policy 1, policy_version 1381891 (0.0008) [2023-12-27 01:26:23,807][105620] Updated weights for policy 1, policy_version 1381901 (0.0008) [2023-12-27 01:26:23,846][105692] Updated weights for policy 0, policy_version 1379768 (0.0010) [2023-12-27 01:26:23,868][105620] Updated weights for policy 1, policy_version 1381911 (0.0009) [2023-12-27 01:26:23,908][105692] Updated weights for policy 0, policy_version 1379778 (0.0010) [2023-12-27 01:26:23,969][105692] Updated weights for policy 0, policy_version 1379788 (0.0010) [2023-12-27 01:26:24,500][105620] Updated weights for policy 1, policy_version 1381921 (0.0007) [2023-12-27 01:26:24,549][105620] Updated weights for policy 1, policy_version 1381931 (0.0010) [2023-12-27 01:26:24,599][105620] Updated weights for policy 1, policy_version 1381941 (0.0010) [2023-12-27 01:26:24,720][105692] Updated weights for policy 0, policy_version 1379798 (0.0009) [2023-12-27 01:26:24,775][105692] Updated weights for policy 0, policy_version 1379808 (0.0010) [2023-12-27 01:26:24,830][105692] Updated weights for policy 0, policy_version 1379818 (0.0011) [2023-12-27 01:26:25,333][105620] Updated weights for policy 1, policy_version 1381951 (0.0010) [2023-12-27 01:26:25,384][105620] Updated weights for policy 1, policy_version 1381961 (0.0010) [2023-12-27 01:26:25,445][105620] Updated weights for policy 1, policy_version 1381971 (0.0010) [2023-12-27 01:26:25,460][105692] Updated weights for policy 0, policy_version 1379828 (0.0008) [2023-12-27 01:26:25,511][105692] Updated weights for policy 0, policy_version 1379838 (0.0005) [2023-12-27 01:26:25,572][105692] Updated weights for policy 0, policy_version 1379848 (0.0005) [2023-12-27 01:26:25,595][105585] KL-divergence is very high: 119.7022 [2023-12-27 01:26:26,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 707125248. Throughput: 0: 9806.7, 1: 9593.3. Samples: 707137388. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-12-27 01:26:26,063][104569] Avg episode reward: [(0, '8621.210'), (1, '8717.720')] [2023-12-27 01:26:26,103][105692] Updated weights for policy 0, policy_version 1379858 (0.0008) [2023-12-27 01:26:26,117][105620] Updated weights for policy 1, policy_version 1381981 (0.0008) [2023-12-27 01:26:26,166][105692] Updated weights for policy 0, policy_version 1379868 (0.0005) [2023-12-27 01:26:26,180][105620] Updated weights for policy 1, policy_version 1381991 (0.0007) [2023-12-27 01:26:26,234][105620] Updated weights for policy 1, policy_version 1382001 (0.0006) [2023-12-27 01:26:26,237][105692] Updated weights for policy 0, policy_version 1379878 (0.0005) [2023-12-27 01:26:26,295][105692] Updated weights for policy 0, policy_version 1379888 (0.0005) [2023-12-27 01:26:26,820][105692] Updated weights for policy 0, policy_version 1379898 (0.0005) [2023-12-27 01:26:26,870][105692] Updated weights for policy 0, policy_version 1379908 (0.0005) [2023-12-27 01:26:26,920][105692] Updated weights for policy 0, policy_version 1379918 (0.0005) [2023-12-27 01:26:26,975][105620] Updated weights for policy 1, policy_version 1382011 (0.0008) [2023-12-27 01:26:27,022][105620] Updated weights for policy 1, policy_version 1382021 (0.0008) [2023-12-27 01:26:27,075][105620] Updated weights for policy 1, policy_version 1382031 (0.0008) [2023-12-27 01:26:27,451][105692] Updated weights for policy 0, policy_version 1379928 (0.0009) [2023-12-27 01:26:27,509][105692] Updated weights for policy 0, policy_version 1379938 (0.0010) [2023-12-27 01:26:27,562][105692] Updated weights for policy 0, policy_version 1379948 (0.0010) [2023-12-27 01:26:27,799][105620] Updated weights for policy 1, policy_version 1382042 (0.0010) [2023-12-27 01:26:27,864][105620] Updated weights for policy 1, policy_version 1382052 (0.0005) [2023-12-27 01:26:27,918][105620] Updated weights for policy 1, policy_version 1382062 (0.0008) [2023-12-27 01:26:27,962][105620] Updated weights for policy 1, policy_version 1382072 (0.0008) [2023-12-27 01:26:28,321][105692] Updated weights for policy 0, policy_version 1379958 (0.0010) [2023-12-27 01:26:28,386][105692] Updated weights for policy 0, policy_version 1379968 (0.0010) [2023-12-27 01:26:28,448][105692] Updated weights for policy 0, policy_version 1379978 (0.0010) [2023-12-27 01:26:28,668][105620] Updated weights for policy 1, policy_version 1382082 (0.0008) [2023-12-27 01:26:28,720][105620] Updated weights for policy 1, policy_version 1382092 (0.0008) [2023-12-27 01:26:28,780][105620] Updated weights for policy 1, policy_version 1382102 (0.0009) [2023-12-27 01:26:29,164][105692] Updated weights for policy 0, policy_version 1379988 (0.0008) [2023-12-27 01:26:29,218][105692] Updated weights for policy 0, policy_version 1379998 (0.0010) [2023-12-27 01:26:29,283][105692] Updated weights for policy 0, policy_version 1380008 (0.0010) [2023-12-27 01:26:29,542][105620] Updated weights for policy 1, policy_version 1382112 (0.0010) [2023-12-27 01:26:29,600][105620] Updated weights for policy 1, policy_version 1382122 (0.0010) [2023-12-27 01:26:29,659][105620] Updated weights for policy 1, policy_version 1382132 (0.0010) [2023-12-27 01:26:30,011][105692] Updated weights for policy 0, policy_version 1380018 (0.0011) [2023-12-27 01:26:30,067][105692] Updated weights for policy 0, policy_version 1380028 (0.0009) [2023-12-27 01:26:30,122][105692] Updated weights for policy 0, policy_version 1380038 (0.0006) [2023-12-27 01:26:30,218][105692] Updated weights for policy 0, policy_version 1380048 (0.0010) [2023-12-27 01:26:30,348][105620] Updated weights for policy 1, policy_version 1382142 (0.0008) [2023-12-27 01:26:30,410][105620] Updated weights for policy 1, policy_version 1382152 (0.0005) [2023-12-27 01:26:30,471][105620] Updated weights for policy 1, policy_version 1382162 (0.0005) [2023-12-27 01:26:30,894][105692] Updated weights for policy 0, policy_version 1380058 (0.0008) [2023-12-27 01:26:30,957][105692] Updated weights for policy 0, policy_version 1380068 (0.0005) [2023-12-27 01:26:31,008][105692] Updated weights for policy 0, policy_version 1380078 (0.0005) [2023-12-27 01:26:31,027][105620] Updated weights for policy 1, policy_version 1382172 (0.0007) [2023-12-27 01:26:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 707231744. Throughput: 0: 9891.9, 1: 9658.6. Samples: 707200272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:26:31,063][104569] Avg episode reward: [(0, '8531.309'), (1, '9013.751')] [2023-12-27 01:26:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001380080_353353728.pth... [2023-12-27 01:26:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001378896_353050624.pth [2023-12-27 01:26:31,086][105620] Updated weights for policy 1, policy_version 1382182 (0.0007) [2023-12-27 01:26:31,153][105620] Updated weights for policy 1, policy_version 1382192 (0.0010) [2023-12-27 01:26:31,194][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001382200_353886208.pth... [2023-12-27 01:26:31,197][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001381048_353591296.pth [2023-12-27 01:26:31,745][105692] Updated weights for policy 0, policy_version 1380088 (0.0008) [2023-12-27 01:26:31,800][105692] Updated weights for policy 0, policy_version 1380099 (0.0009) [2023-12-27 01:26:31,847][105692] Updated weights for policy 0, policy_version 1380109 (0.0009) [2023-12-27 01:26:31,856][105620] Updated weights for policy 1, policy_version 1382202 (0.0010) [2023-12-27 01:26:31,905][105620] Updated weights for policy 1, policy_version 1382212 (0.0008) [2023-12-27 01:26:31,960][105620] Updated weights for policy 1, policy_version 1382222 (0.0009) [2023-12-27 01:26:32,015][105620] Updated weights for policy 1, policy_version 1382232 (0.0009) [2023-12-27 01:26:32,527][105692] Updated weights for policy 0, policy_version 1380119 (0.0008) [2023-12-27 01:26:32,597][105692] Updated weights for policy 0, policy_version 1380129 (0.0011) [2023-12-27 01:26:32,663][105692] Updated weights for policy 0, policy_version 1380139 (0.0007) [2023-12-27 01:26:32,789][105620] Updated weights for policy 1, policy_version 1382242 (0.0010) [2023-12-27 01:26:32,859][105620] Updated weights for policy 1, policy_version 1382252 (0.0006) [2023-12-27 01:26:32,930][105620] Updated weights for policy 1, policy_version 1382262 (0.0008) [2023-12-27 01:26:33,192][105692] Updated weights for policy 0, policy_version 1380149 (0.0006) [2023-12-27 01:26:33,243][105692] Updated weights for policy 0, policy_version 1380159 (0.0007) [2023-12-27 01:26:33,301][105692] Updated weights for policy 0, policy_version 1380169 (0.0007) [2023-12-27 01:26:33,607][105620] Updated weights for policy 1, policy_version 1382272 (0.0010) [2023-12-27 01:26:33,658][105620] Updated weights for policy 1, policy_version 1382282 (0.0007) [2023-12-27 01:26:33,710][105620] Updated weights for policy 1, policy_version 1382292 (0.0005) [2023-12-27 01:26:33,972][105692] Updated weights for policy 0, policy_version 1380179 (0.0010) [2023-12-27 01:26:34,028][105692] Updated weights for policy 0, policy_version 1380189 (0.0011) [2023-12-27 01:26:34,083][105692] Updated weights for policy 0, policy_version 1380199 (0.0011) [2023-12-27 01:26:34,279][105620] Updated weights for policy 1, policy_version 1382302 (0.0008) [2023-12-27 01:26:34,348][105620] Updated weights for policy 1, policy_version 1382312 (0.0011) [2023-12-27 01:26:34,413][105620] Updated weights for policy 1, policy_version 1382322 (0.0008) [2023-12-27 01:26:34,824][105692] Updated weights for policy 0, policy_version 1380209 (0.0010) [2023-12-27 01:26:34,872][105692] Updated weights for policy 0, policy_version 1380219 (0.0010) [2023-12-27 01:26:34,926][105692] Updated weights for policy 0, policy_version 1380229 (0.0010) [2023-12-27 01:26:34,983][105692] Updated weights for policy 0, policy_version 1380239 (0.0010) [2023-12-27 01:26:35,050][105620] Updated weights for policy 1, policy_version 1382332 (0.0007) [2023-12-27 01:26:35,108][105620] Updated weights for policy 1, policy_version 1382342 (0.0010) [2023-12-27 01:26:35,166][105620] Updated weights for policy 1, policy_version 1382352 (0.0010) [2023-12-27 01:26:35,634][105692] Updated weights for policy 0, policy_version 1380249 (0.0006) [2023-12-27 01:26:35,678][105692] Updated weights for policy 0, policy_version 1380259 (0.0005) [2023-12-27 01:26:35,726][105692] Updated weights for policy 0, policy_version 1380269 (0.0009) [2023-12-27 01:26:35,862][105620] Updated weights for policy 1, policy_version 1382362 (0.0011) [2023-12-27 01:26:35,911][105620] Updated weights for policy 1, policy_version 1382372 (0.0008) [2023-12-27 01:26:35,971][105620] Updated weights for policy 1, policy_version 1382382 (0.0005) [2023-12-27 01:26:36,035][105620] Updated weights for policy 1, policy_version 1382392 (0.0010) [2023-12-27 01:26:36,062][104569] Fps is (10 sec: 21299.1, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 707338240. Throughput: 0: 9886.1, 1: 9795.8. Samples: 707322532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:26:36,063][104569] Avg episode reward: [(0, '8257.001'), (1, '8459.240')] [2023-12-27 01:26:36,341][105692] Updated weights for policy 0, policy_version 1380279 (0.0009) [2023-12-27 01:26:36,408][105692] Updated weights for policy 0, policy_version 1380289 (0.0006) [2023-12-27 01:26:36,469][105692] Updated weights for policy 0, policy_version 1380299 (0.0006) [2023-12-27 01:26:36,781][105620] Updated weights for policy 1, policy_version 1382402 (0.0011) [2023-12-27 01:26:36,858][105620] Updated weights for policy 1, policy_version 1382412 (0.0011) [2023-12-27 01:26:36,926][105620] Updated weights for policy 1, policy_version 1382422 (0.0008) [2023-12-27 01:26:37,051][105692] Updated weights for policy 0, policy_version 1380309 (0.0006) [2023-12-27 01:26:37,107][105692] Updated weights for policy 0, policy_version 1380319 (0.0009) [2023-12-27 01:26:37,177][105692] Updated weights for policy 0, policy_version 1380329 (0.0005) [2023-12-27 01:26:37,611][105620] Updated weights for policy 1, policy_version 1382432 (0.0007) [2023-12-27 01:26:37,671][105620] Updated weights for policy 1, policy_version 1382442 (0.0005) [2023-12-27 01:26:37,731][105620] Updated weights for policy 1, policy_version 1382452 (0.0009) [2023-12-27 01:26:37,827][105692] Updated weights for policy 0, policy_version 1380339 (0.0006) [2023-12-27 01:26:37,890][105692] Updated weights for policy 0, policy_version 1380349 (0.0010) [2023-12-27 01:26:37,948][105692] Updated weights for policy 0, policy_version 1380359 (0.0010) [2023-12-27 01:26:38,415][105620] Updated weights for policy 1, policy_version 1382462 (0.0008) [2023-12-27 01:26:38,474][105620] Updated weights for policy 1, policy_version 1382472 (0.0007) [2023-12-27 01:26:38,533][105620] Updated weights for policy 1, policy_version 1382482 (0.0006) [2023-12-27 01:26:38,637][105692] Updated weights for policy 0, policy_version 1380369 (0.0010) [2023-12-27 01:26:38,703][105692] Updated weights for policy 0, policy_version 1380379 (0.0006) [2023-12-27 01:26:38,773][105692] Updated weights for policy 0, policy_version 1380389 (0.0006) [2023-12-27 01:26:38,828][105692] Updated weights for policy 0, policy_version 1380399 (0.0008) [2023-12-27 01:26:39,244][105620] Updated weights for policy 1, policy_version 1382492 (0.0008) [2023-12-27 01:26:39,310][105620] Updated weights for policy 1, policy_version 1382502 (0.0011) [2023-12-27 01:26:39,379][105620] Updated weights for policy 1, policy_version 1382512 (0.0008) [2023-12-27 01:26:39,535][105692] Updated weights for policy 0, policy_version 1380409 (0.0008) [2023-12-27 01:26:39,600][105692] Updated weights for policy 0, policy_version 1380419 (0.0008) [2023-12-27 01:26:39,669][105692] Updated weights for policy 0, policy_version 1380429 (0.0008) [2023-12-27 01:26:40,144][105620] Updated weights for policy 1, policy_version 1382522 (0.0009) [2023-12-27 01:26:40,216][105620] Updated weights for policy 1, policy_version 1382532 (0.0008) [2023-12-27 01:26:40,285][105620] Updated weights for policy 1, policy_version 1382542 (0.0006) [2023-12-27 01:26:40,349][105620] Updated weights for policy 1, policy_version 1382552 (0.0010) [2023-12-27 01:26:40,442][105692] Updated weights for policy 0, policy_version 1380439 (0.0009) [2023-12-27 01:26:40,513][105692] Updated weights for policy 0, policy_version 1380449 (0.0008) [2023-12-27 01:26:40,572][105692] Updated weights for policy 0, policy_version 1380459 (0.0008) [2023-12-27 01:26:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 707428352. Throughput: 0: 10013.2, 1: 9820.3. Samples: 707441720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:26:41,063][104569] Avg episode reward: [(0, '8261.800'), (1, '7583.097')] [2023-12-27 01:26:41,068][105620] Updated weights for policy 1, policy_version 1382562 (0.0011) [2023-12-27 01:26:41,129][105620] Updated weights for policy 1, policy_version 1382572 (0.0010) [2023-12-27 01:26:41,205][105620] Updated weights for policy 1, policy_version 1382582 (0.0013) [2023-12-27 01:26:41,304][105692] Updated weights for policy 0, policy_version 1380469 (0.0008) [2023-12-27 01:26:41,377][105692] Updated weights for policy 0, policy_version 1380479 (0.0009) [2023-12-27 01:26:41,390][105585] KL-divergence is very high: 114.7188 [2023-12-27 01:26:41,437][105585] KL-divergence is very high: 106.1952 [2023-12-27 01:26:41,438][105692] Updated weights for policy 0, policy_version 1380489 (0.0009) [2023-12-27 01:26:41,911][105620] Updated weights for policy 1, policy_version 1382592 (0.0009) [2023-12-27 01:26:41,959][105620] Updated weights for policy 1, policy_version 1382602 (0.0009) [2023-12-27 01:26:42,019][105620] Updated weights for policy 1, policy_version 1382612 (0.0009) [2023-12-27 01:26:42,222][105692] Updated weights for policy 0, policy_version 1380499 (0.0009) [2023-12-27 01:26:42,286][105692] Updated weights for policy 0, policy_version 1380509 (0.0009) [2023-12-27 01:26:42,355][105692] Updated weights for policy 0, policy_version 1380519 (0.0009) [2023-12-27 01:26:42,885][105620] Updated weights for policy 1, policy_version 1382622 (0.0009) [2023-12-27 01:26:42,940][105620] Updated weights for policy 1, policy_version 1382633 (0.0011) [2023-12-27 01:26:42,965][105692] Updated weights for policy 0, policy_version 1380529 (0.0007) [2023-12-27 01:26:42,999][105620] Updated weights for policy 1, policy_version 1382643 (0.0009) [2023-12-27 01:26:43,017][105692] Updated weights for policy 0, policy_version 1380539 (0.0005) [2023-12-27 01:26:43,062][105692] Updated weights for policy 0, policy_version 1380549 (0.0005) [2023-12-27 01:26:43,109][105692] Updated weights for policy 0, policy_version 1380559 (0.0009) [2023-12-27 01:26:43,732][105620] Updated weights for policy 1, policy_version 1382653 (0.0008) [2023-12-27 01:26:43,794][105620] Updated weights for policy 1, policy_version 1382663 (0.0008) [2023-12-27 01:26:43,857][105620] Updated weights for policy 1, policy_version 1382673 (0.0007) [2023-12-27 01:26:43,888][105692] Updated weights for policy 0, policy_version 1380569 (0.0007) [2023-12-27 01:26:43,939][105692] Updated weights for policy 0, policy_version 1380579 (0.0010) [2023-12-27 01:26:44,000][105692] Updated weights for policy 0, policy_version 1380589 (0.0010) [2023-12-27 01:26:44,559][105692] Updated weights for policy 0, policy_version 1380599 (0.0009) [2023-12-27 01:26:44,615][105692] Updated weights for policy 0, policy_version 1380609 (0.0010) [2023-12-27 01:26:44,674][105692] Updated weights for policy 0, policy_version 1380619 (0.0010) [2023-12-27 01:26:44,680][105620] Updated weights for policy 1, policy_version 1382683 (0.0008) [2023-12-27 01:26:44,747][105620] Updated weights for policy 1, policy_version 1382693 (0.0007) [2023-12-27 01:26:44,810][105620] Updated weights for policy 1, policy_version 1382703 (0.0008) [2023-12-27 01:26:45,358][105692] Updated weights for policy 0, policy_version 1380629 (0.0008) [2023-12-27 01:26:45,421][105692] Updated weights for policy 0, policy_version 1380639 (0.0008) [2023-12-27 01:26:45,480][105692] Updated weights for policy 0, policy_version 1380649 (0.0010) [2023-12-27 01:26:45,627][105620] Updated weights for policy 1, policy_version 1382713 (0.0009) [2023-12-27 01:26:45,695][105620] Updated weights for policy 1, policy_version 1382723 (0.0010) [2023-12-27 01:26:45,748][105620] Updated weights for policy 1, policy_version 1382733 (0.0008) [2023-12-27 01:26:45,800][105620] Updated weights for policy 1, policy_version 1382743 (0.0008) [2023-12-27 01:26:46,058][105692] Updated weights for policy 0, policy_version 1380659 (0.0007) [2023-12-27 01:26:46,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 707526656. Throughput: 0: 10043.1, 1: 9772.2. Samples: 707497432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:26:46,063][104569] Avg episode reward: [(0, '8444.158'), (1, '7914.049')] [2023-12-27 01:26:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001382744_354025472.pth... [2023-12-27 01:26:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001381592_353730560.pth [2023-12-27 01:26:46,116][105692] Updated weights for policy 0, policy_version 1380669 (0.0010) [2023-12-27 01:26:46,171][105692] Updated weights for policy 0, policy_version 1380679 (0.0010) [2023-12-27 01:26:46,228][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001380688_353509376.pth... [2023-12-27 01:26:46,233][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001379472_353198080.pth [2023-12-27 01:26:46,528][105620] Updated weights for policy 1, policy_version 1382753 (0.0008) [2023-12-27 01:26:46,583][105620] Updated weights for policy 1, policy_version 1382763 (0.0007) [2023-12-27 01:26:46,641][105620] Updated weights for policy 1, policy_version 1382773 (0.0008) [2023-12-27 01:26:46,895][105692] Updated weights for policy 0, policy_version 1380689 (0.0010) [2023-12-27 01:26:46,949][105692] Updated weights for policy 0, policy_version 1380699 (0.0005) [2023-12-27 01:26:47,001][105692] Updated weights for policy 0, policy_version 1380709 (0.0005) [2023-12-27 01:26:47,057][105692] Updated weights for policy 0, policy_version 1380719 (0.0008) [2023-12-27 01:26:47,395][105620] Updated weights for policy 1, policy_version 1382783 (0.0010) [2023-12-27 01:26:47,447][105620] Updated weights for policy 1, policy_version 1382793 (0.0007) [2023-12-27 01:26:47,503][105620] Updated weights for policy 1, policy_version 1382803 (0.0005) [2023-12-27 01:26:47,590][105692] Updated weights for policy 0, policy_version 1380729 (0.0010) [2023-12-27 01:26:47,645][105692] Updated weights for policy 0, policy_version 1380739 (0.0010) [2023-12-27 01:26:47,708][105692] Updated weights for policy 0, policy_version 1380749 (0.0010) [2023-12-27 01:26:48,164][105620] Updated weights for policy 1, policy_version 1382813 (0.0007) [2023-12-27 01:26:48,214][105620] Updated weights for policy 1, policy_version 1382823 (0.0008) [2023-12-27 01:26:48,274][105620] Updated weights for policy 1, policy_version 1382833 (0.0008) [2023-12-27 01:26:48,396][105692] Updated weights for policy 0, policy_version 1380759 (0.0010) [2023-12-27 01:26:48,458][105692] Updated weights for policy 0, policy_version 1380769 (0.0010) [2023-12-27 01:26:48,530][105692] Updated weights for policy 0, policy_version 1380779 (0.0010) [2023-12-27 01:26:49,077][105620] Updated weights for policy 1, policy_version 1382843 (0.0008) [2023-12-27 01:26:49,140][105620] Updated weights for policy 1, policy_version 1382853 (0.0008) [2023-12-27 01:26:49,200][105620] Updated weights for policy 1, policy_version 1382863 (0.0008) [2023-12-27 01:26:49,254][105692] Updated weights for policy 0, policy_version 1380789 (0.0010) [2023-12-27 01:26:49,307][105692] Updated weights for policy 0, policy_version 1380799 (0.0010) [2023-12-27 01:26:49,375][105692] Updated weights for policy 0, policy_version 1380809 (0.0012) [2023-12-27 01:26:50,002][105620] Updated weights for policy 1, policy_version 1382873 (0.0007) [2023-12-27 01:26:50,064][105620] Updated weights for policy 1, policy_version 1382883 (0.0008) [2023-12-27 01:26:50,117][105620] Updated weights for policy 1, policy_version 1382893 (0.0008) [2023-12-27 01:26:50,155][105692] Updated weights for policy 0, policy_version 1380819 (0.0010) [2023-12-27 01:26:50,165][105620] Updated weights for policy 1, policy_version 1382903 (0.0007) [2023-12-27 01:26:50,207][105692] Updated weights for policy 0, policy_version 1380829 (0.0010) [2023-12-27 01:26:50,264][105692] Updated weights for policy 0, policy_version 1380839 (0.0010) [2023-12-27 01:26:50,973][105620] Updated weights for policy 1, policy_version 1382913 (0.0008) [2023-12-27 01:26:51,026][105620] Updated weights for policy 1, policy_version 1382923 (0.0007) [2023-12-27 01:26:51,033][105692] Updated weights for policy 0, policy_version 1380849 (0.0010) [2023-12-27 01:26:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 707616768. Throughput: 0: 10161.9, 1: 9679.4. Samples: 707615468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:26:51,063][104569] Avg episode reward: [(0, '8170.190'), (1, '8903.816')] [2023-12-27 01:26:51,091][105620] Updated weights for policy 1, policy_version 1382933 (0.0008) [2023-12-27 01:26:51,095][105692] Updated weights for policy 0, policy_version 1380859 (0.0007) [2023-12-27 01:26:51,161][105692] Updated weights for policy 0, policy_version 1380869 (0.0011) [2023-12-27 01:26:51,214][105692] Updated weights for policy 0, policy_version 1380879 (0.0011) [2023-12-27 01:26:51,898][105620] Updated weights for policy 1, policy_version 1382943 (0.0009) [2023-12-27 01:26:51,949][105692] Updated weights for policy 0, policy_version 1380889 (0.0010) [2023-12-27 01:26:51,960][105620] Updated weights for policy 1, policy_version 1382953 (0.0007) [2023-12-27 01:26:52,006][105692] Updated weights for policy 0, policy_version 1380899 (0.0006) [2023-12-27 01:26:52,017][105620] Updated weights for policy 1, policy_version 1382963 (0.0009) [2023-12-27 01:26:52,072][105692] Updated weights for policy 0, policy_version 1380909 (0.0006) [2023-12-27 01:26:52,716][105692] Updated weights for policy 0, policy_version 1380919 (0.0011) [2023-12-27 01:26:52,778][105692] Updated weights for policy 0, policy_version 1380929 (0.0007) [2023-12-27 01:26:52,791][105620] Updated weights for policy 1, policy_version 1382974 (0.0008) [2023-12-27 01:26:52,829][105692] Updated weights for policy 0, policy_version 1380939 (0.0005) [2023-12-27 01:26:52,844][105620] Updated weights for policy 1, policy_version 1382984 (0.0009) [2023-12-27 01:26:52,906][105620] Updated weights for policy 1, policy_version 1382994 (0.0010) [2023-12-27 01:26:53,472][105692] Updated weights for policy 0, policy_version 1380949 (0.0005) [2023-12-27 01:26:53,525][105692] Updated weights for policy 0, policy_version 1380959 (0.0005) [2023-12-27 01:26:53,571][105692] Updated weights for policy 0, policy_version 1380969 (0.0005) [2023-12-27 01:26:53,605][105620] Updated weights for policy 1, policy_version 1383004 (0.0010) [2023-12-27 01:26:53,654][105620] Updated weights for policy 1, policy_version 1383014 (0.0010) [2023-12-27 01:26:53,702][105620] Updated weights for policy 1, policy_version 1383024 (0.0009) [2023-12-27 01:26:54,188][105692] Updated weights for policy 0, policy_version 1380979 (0.0005) [2023-12-27 01:26:54,239][105692] Updated weights for policy 0, policy_version 1380989 (0.0005) [2023-12-27 01:26:54,298][105692] Updated weights for policy 0, policy_version 1380999 (0.0005) [2023-12-27 01:26:54,410][105620] Updated weights for policy 1, policy_version 1383034 (0.0009) [2023-12-27 01:26:54,462][105620] Updated weights for policy 1, policy_version 1383044 (0.0005) [2023-12-27 01:26:54,508][105620] Updated weights for policy 1, policy_version 1383054 (0.0009) [2023-12-27 01:26:54,564][105620] Updated weights for policy 1, policy_version 1383064 (0.0007) [2023-12-27 01:26:54,957][105692] Updated weights for policy 0, policy_version 1381009 (0.0010) [2023-12-27 01:26:55,009][105692] Updated weights for policy 0, policy_version 1381019 (0.0009) [2023-12-27 01:26:55,058][105692] Updated weights for policy 0, policy_version 1381029 (0.0010) [2023-12-27 01:26:55,114][105692] Updated weights for policy 0, policy_version 1381039 (0.0011) [2023-12-27 01:26:55,211][105620] Updated weights for policy 1, policy_version 1383074 (0.0010) [2023-12-27 01:26:55,271][105620] Updated weights for policy 1, policy_version 1383084 (0.0009) [2023-12-27 01:26:55,331][105620] Updated weights for policy 1, policy_version 1383094 (0.0010) [2023-12-27 01:26:55,814][105692] Updated weights for policy 0, policy_version 1381049 (0.0006) [2023-12-27 01:26:55,865][105692] Updated weights for policy 0, policy_version 1381059 (0.0005) [2023-12-27 01:26:55,918][105620] Updated weights for policy 1, policy_version 1383104 (0.0007) [2023-12-27 01:26:55,928][105692] Updated weights for policy 0, policy_version 1381069 (0.0009) [2023-12-27 01:26:55,962][105620] Updated weights for policy 1, policy_version 1383114 (0.0010) [2023-12-27 01:26:56,019][105620] Updated weights for policy 1, policy_version 1383124 (0.0008) [2023-12-27 01:26:56,062][104569] Fps is (10 sec: 20480.8, 60 sec: 19933.9, 300 sec: 19466.4). Total num frames: 707731456. Throughput: 0: 10132.0, 1: 9736.7. Samples: 707734524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:26:56,062][104569] Avg episode reward: [(0, '8442.051'), (1, '9083.681')] [2023-12-27 01:26:56,580][105620] Updated weights for policy 1, policy_version 1383134 (0.0007) [2023-12-27 01:26:56,590][105692] Updated weights for policy 0, policy_version 1381079 (0.0007) [2023-12-27 01:26:56,632][105620] Updated weights for policy 1, policy_version 1383144 (0.0005) [2023-12-27 01:26:56,652][105692] Updated weights for policy 0, policy_version 1381089 (0.0005) [2023-12-27 01:26:56,686][105620] Updated weights for policy 1, policy_version 1383154 (0.0005) [2023-12-27 01:26:56,713][105692] Updated weights for policy 0, policy_version 1381099 (0.0009) [2023-12-27 01:26:57,200][105620] Updated weights for policy 1, policy_version 1383164 (0.0005) [2023-12-27 01:26:57,245][105620] Updated weights for policy 1, policy_version 1383174 (0.0006) [2023-12-27 01:26:57,289][105620] Updated weights for policy 1, policy_version 1383184 (0.0006) [2023-12-27 01:26:57,388][105692] Updated weights for policy 0, policy_version 1381109 (0.0010) [2023-12-27 01:26:57,439][105692] Updated weights for policy 0, policy_version 1381119 (0.0010) [2023-12-27 01:26:57,497][105692] Updated weights for policy 0, policy_version 1381129 (0.0010) [2023-12-27 01:26:57,860][105620] Updated weights for policy 1, policy_version 1383194 (0.0007) [2023-12-27 01:26:57,927][105620] Updated weights for policy 1, policy_version 1383204 (0.0005) [2023-12-27 01:26:57,981][105620] Updated weights for policy 1, policy_version 1383214 (0.0005) [2023-12-27 01:26:58,044][105620] Updated weights for policy 1, policy_version 1383224 (0.0008) [2023-12-27 01:26:58,236][105692] Updated weights for policy 0, policy_version 1381139 (0.0010) [2023-12-27 01:26:58,296][105692] Updated weights for policy 0, policy_version 1381149 (0.0011) [2023-12-27 01:26:58,356][105692] Updated weights for policy 0, policy_version 1381159 (0.0011) [2023-12-27 01:26:58,703][105620] Updated weights for policy 1, policy_version 1383234 (0.0006) [2023-12-27 01:26:58,762][105620] Updated weights for policy 1, policy_version 1383244 (0.0010) [2023-12-27 01:26:58,833][105620] Updated weights for policy 1, policy_version 1383254 (0.0006) [2023-12-27 01:26:59,135][105692] Updated weights for policy 0, policy_version 1381169 (0.0010) [2023-12-27 01:26:59,183][105692] Updated weights for policy 0, policy_version 1381179 (0.0010) [2023-12-27 01:26:59,244][105692] Updated weights for policy 0, policy_version 1381189 (0.0009) [2023-12-27 01:26:59,299][105692] Updated weights for policy 0, policy_version 1381199 (0.0008) [2023-12-27 01:26:59,429][105620] Updated weights for policy 1, policy_version 1383264 (0.0010) [2023-12-27 01:26:59,491][105620] Updated weights for policy 1, policy_version 1383274 (0.0009) [2023-12-27 01:26:59,546][105620] Updated weights for policy 1, policy_version 1383284 (0.0009) [2023-12-27 01:26:59,990][105692] Updated weights for policy 0, policy_version 1381209 (0.0008) [2023-12-27 01:27:00,042][105692] Updated weights for policy 0, policy_version 1381219 (0.0007) [2023-12-27 01:27:00,101][105692] Updated weights for policy 0, policy_version 1381229 (0.0008) [2023-12-27 01:27:00,326][105620] Updated weights for policy 1, policy_version 1383294 (0.0010) [2023-12-27 01:27:00,382][105620] Updated weights for policy 1, policy_version 1383304 (0.0009) [2023-12-27 01:27:00,434][105620] Updated weights for policy 1, policy_version 1383315 (0.0009) [2023-12-27 01:27:00,771][105692] Updated weights for policy 0, policy_version 1381239 (0.0009) [2023-12-27 01:27:00,819][105692] Updated weights for policy 0, policy_version 1381249 (0.0008) [2023-12-27 01:27:00,865][105692] Updated weights for policy 0, policy_version 1381259 (0.0009) [2023-12-27 01:27:01,062][104569] Fps is (10 sec: 21298.9, 60 sec: 19933.8, 300 sec: 19494.2). Total num frames: 707829760. Throughput: 0: 10173.3, 1: 9850.7. Samples: 707799140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:27:01,063][104569] Avg episode reward: [(0, '8164.839'), (1, '8991.611')] [2023-12-27 01:27:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001381264_353656832.pth... [2023-12-27 01:27:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001383320_354172928.pth... [2023-12-27 01:27:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001380080_353353728.pth [2023-12-27 01:27:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001382200_353886208.pth [2023-12-27 01:27:01,181][105620] Updated weights for policy 1, policy_version 1383325 (0.0006) [2023-12-27 01:27:01,239][105620] Updated weights for policy 1, policy_version 1383335 (0.0009) [2023-12-27 01:27:01,295][105620] Updated weights for policy 1, policy_version 1383345 (0.0006) [2023-12-27 01:27:01,680][105692] Updated weights for policy 0, policy_version 1381269 (0.0009) [2023-12-27 01:27:01,744][105692] Updated weights for policy 0, policy_version 1381279 (0.0009) [2023-12-27 01:27:01,792][105692] Updated weights for policy 0, policy_version 1381289 (0.0009) [2023-12-27 01:27:01,915][105620] Updated weights for policy 1, policy_version 1383355 (0.0006) [2023-12-27 01:27:01,964][105620] Updated weights for policy 1, policy_version 1383365 (0.0010) [2023-12-27 01:27:02,019][105620] Updated weights for policy 1, policy_version 1383375 (0.0010) [2023-12-27 01:27:02,596][105692] Updated weights for policy 0, policy_version 1381299 (0.0009) [2023-12-27 01:27:02,643][105692] Updated weights for policy 0, policy_version 1381309 (0.0008) [2023-12-27 01:27:02,691][105692] Updated weights for policy 0, policy_version 1381319 (0.0008) [2023-12-27 01:27:02,732][105620] Updated weights for policy 1, policy_version 1383385 (0.0011) [2023-12-27 01:27:02,793][105620] Updated weights for policy 1, policy_version 1383395 (0.0010) [2023-12-27 01:27:02,851][105620] Updated weights for policy 1, policy_version 1383405 (0.0010) [2023-12-27 01:27:02,910][105620] Updated weights for policy 1, policy_version 1383415 (0.0010) [2023-12-27 01:27:03,504][105692] Updated weights for policy 0, policy_version 1381329 (0.0008) [2023-12-27 01:27:03,550][105620] Updated weights for policy 1, policy_version 1383425 (0.0006) [2023-12-27 01:27:03,563][105692] Updated weights for policy 0, policy_version 1381339 (0.0009) [2023-12-27 01:27:03,603][105620] Updated weights for policy 1, policy_version 1383435 (0.0006) [2023-12-27 01:27:03,623][105692] Updated weights for policy 0, policy_version 1381349 (0.0008) [2023-12-27 01:27:03,665][105620] Updated weights for policy 1, policy_version 1383445 (0.0005) [2023-12-27 01:27:03,686][105692] Updated weights for policy 0, policy_version 1381359 (0.0009) [2023-12-27 01:27:04,345][105620] Updated weights for policy 1, policy_version 1383455 (0.0008) [2023-12-27 01:27:04,403][105620] Updated weights for policy 1, policy_version 1383465 (0.0008) [2023-12-27 01:27:04,456][105692] Updated weights for policy 0, policy_version 1381369 (0.0006) [2023-12-27 01:27:04,462][105620] Updated weights for policy 1, policy_version 1383475 (0.0007) [2023-12-27 01:27:04,518][105692] Updated weights for policy 0, policy_version 1381379 (0.0009) [2023-12-27 01:27:04,582][105692] Updated weights for policy 0, policy_version 1381389 (0.0009) [2023-12-27 01:27:05,218][105620] Updated weights for policy 1, policy_version 1383485 (0.0006) [2023-12-27 01:27:05,271][105620] Updated weights for policy 1, policy_version 1383495 (0.0005) [2023-12-27 01:27:05,324][105620] Updated weights for policy 1, policy_version 1383505 (0.0006) [2023-12-27 01:27:05,328][105692] Updated weights for policy 0, policy_version 1381399 (0.0009) [2023-12-27 01:27:05,377][105692] Updated weights for policy 0, policy_version 1381409 (0.0006) [2023-12-27 01:27:05,433][105692] Updated weights for policy 0, policy_version 1381419 (0.0008) [2023-12-27 01:27:06,040][105620] Updated weights for policy 1, policy_version 1383515 (0.0008) [2023-12-27 01:27:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 707919872. Throughput: 0: 10047.2, 1: 9835.6. Samples: 707914568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:27:06,062][104569] Avg episode reward: [(0, '8075.668'), (1, '8444.553')] [2023-12-27 01:27:06,088][105620] Updated weights for policy 1, policy_version 1383525 (0.0009) [2023-12-27 01:27:06,148][105620] Updated weights for policy 1, policy_version 1383535 (0.0009) [2023-12-27 01:27:06,202][105692] Updated weights for policy 0, policy_version 1381429 (0.0008) [2023-12-27 01:27:06,255][105692] Updated weights for policy 0, policy_version 1381439 (0.0006) [2023-12-27 01:27:06,316][105692] Updated weights for policy 0, policy_version 1381449 (0.0006) [2023-12-27 01:27:06,952][105620] Updated weights for policy 1, policy_version 1383545 (0.0009) [2023-12-27 01:27:06,969][105692] Updated weights for policy 0, policy_version 1381459 (0.0008) [2023-12-27 01:27:07,003][105620] Updated weights for policy 1, policy_version 1383555 (0.0008) [2023-12-27 01:27:07,025][105692] Updated weights for policy 0, policy_version 1381469 (0.0008) [2023-12-27 01:27:07,052][105620] Updated weights for policy 1, policy_version 1383565 (0.0007) [2023-12-27 01:27:07,079][105692] Updated weights for policy 0, policy_version 1381479 (0.0007) [2023-12-27 01:27:07,098][105620] Updated weights for policy 1, policy_version 1383575 (0.0006) [2023-12-27 01:27:07,759][105620] Updated weights for policy 1, policy_version 1383585 (0.0009) [2023-12-27 01:27:07,813][105620] Updated weights for policy 1, policy_version 1383595 (0.0009) [2023-12-27 01:27:07,864][105620] Updated weights for policy 1, policy_version 1383605 (0.0009) [2023-12-27 01:27:07,866][105692] Updated weights for policy 0, policy_version 1381489 (0.0008) [2023-12-27 01:27:07,921][105692] Updated weights for policy 0, policy_version 1381499 (0.0008) [2023-12-27 01:27:07,990][105692] Updated weights for policy 0, policy_version 1381509 (0.0009) [2023-12-27 01:27:08,052][105692] Updated weights for policy 0, policy_version 1381519 (0.0009) [2023-12-27 01:27:08,629][105620] Updated weights for policy 1, policy_version 1383615 (0.0008) [2023-12-27 01:27:08,688][105620] Updated weights for policy 1, policy_version 1383625 (0.0009) [2023-12-27 01:27:08,746][105620] Updated weights for policy 1, policy_version 1383635 (0.0009) [2023-12-27 01:27:08,802][105692] Updated weights for policy 0, policy_version 1381529 (0.0008) [2023-12-27 01:27:08,868][105692] Updated weights for policy 0, policy_version 1381539 (0.0009) [2023-12-27 01:27:08,926][105692] Updated weights for policy 0, policy_version 1381549 (0.0009) [2023-12-27 01:27:09,430][105620] Updated weights for policy 1, policy_version 1383645 (0.0007) [2023-12-27 01:27:09,491][105620] Updated weights for policy 1, policy_version 1383655 (0.0008) [2023-12-27 01:27:09,554][105620] Updated weights for policy 1, policy_version 1383665 (0.0009) [2023-12-27 01:27:09,693][105692] Updated weights for policy 0, policy_version 1381559 (0.0009) [2023-12-27 01:27:09,755][105692] Updated weights for policy 0, policy_version 1381569 (0.0009) [2023-12-27 01:27:09,810][105692] Updated weights for policy 0, policy_version 1381579 (0.0008) [2023-12-27 01:27:10,327][105620] Updated weights for policy 1, policy_version 1383675 (0.0009) [2023-12-27 01:27:10,397][105620] Updated weights for policy 1, policy_version 1383685 (0.0009) [2023-12-27 01:27:10,456][105620] Updated weights for policy 1, policy_version 1383695 (0.0009) [2023-12-27 01:27:10,573][105692] Updated weights for policy 0, policy_version 1381589 (0.0009) [2023-12-27 01:27:10,622][105692] Updated weights for policy 0, policy_version 1381599 (0.0009) [2023-12-27 01:27:10,687][105692] Updated weights for policy 0, policy_version 1381609 (0.0010) [2023-12-27 01:27:11,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19797.4, 300 sec: 19466.4). Total num frames: 708018176. Throughput: 0: 9966.5, 1: 9818.5. Samples: 708027712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:27:11,062][104569] Avg episode reward: [(0, '8073.306'), (1, '8899.841')] [2023-12-27 01:27:11,200][105620] Updated weights for policy 1, policy_version 1383705 (0.0009) [2023-12-27 01:27:11,266][105620] Updated weights for policy 1, policy_version 1383715 (0.0008) [2023-12-27 01:27:11,334][105620] Updated weights for policy 1, policy_version 1383725 (0.0009) [2023-12-27 01:27:11,406][105620] Updated weights for policy 1, policy_version 1383735 (0.0008) [2023-12-27 01:27:11,495][105692] Updated weights for policy 0, policy_version 1381619 (0.0008) [2023-12-27 01:27:11,546][105692] Updated weights for policy 0, policy_version 1381629 (0.0008) [2023-12-27 01:27:11,605][105692] Updated weights for policy 0, policy_version 1381639 (0.0008) [2023-12-27 01:27:11,664][105585] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000007 [2023-12-27 01:27:12,181][105620] Updated weights for policy 1, policy_version 1383745 (0.0010) [2023-12-27 01:27:12,241][105620] Updated weights for policy 1, policy_version 1383755 (0.0010) [2023-12-27 01:27:12,307][105620] Updated weights for policy 1, policy_version 1383765 (0.0011) [2023-12-27 01:27:12,438][105692] Updated weights for policy 0, policy_version 1381649 (0.0009) [2023-12-27 01:27:12,495][105692] Updated weights for policy 0, policy_version 1381659 (0.0008) [2023-12-27 01:27:12,544][105692] Updated weights for policy 0, policy_version 1381669 (0.0008) [2023-12-27 01:27:12,602][105692] Updated weights for policy 0, policy_version 1381679 (0.0008) [2023-12-27 01:27:13,053][105620] Updated weights for policy 1, policy_version 1383775 (0.0010) [2023-12-27 01:27:13,113][105620] Updated weights for policy 1, policy_version 1383785 (0.0010) [2023-12-27 01:27:13,178][105620] Updated weights for policy 1, policy_version 1383795 (0.0010) [2023-12-27 01:27:13,411][105692] Updated weights for policy 0, policy_version 1381689 (0.0008) [2023-12-27 01:27:13,473][105692] Updated weights for policy 0, policy_version 1381699 (0.0008) [2023-12-27 01:27:13,531][105692] Updated weights for policy 0, policy_version 1381709 (0.0008) [2023-12-27 01:27:13,918][105620] Updated weights for policy 1, policy_version 1383805 (0.0010) [2023-12-27 01:27:13,962][105620] Updated weights for policy 1, policy_version 1383815 (0.0010) [2023-12-27 01:27:14,013][105620] Updated weights for policy 1, policy_version 1383825 (0.0010) [2023-12-27 01:27:14,292][105692] Updated weights for policy 0, policy_version 1381719 (0.0008) [2023-12-27 01:27:14,344][105692] Updated weights for policy 0, policy_version 1381729 (0.0008) [2023-12-27 01:27:14,393][105692] Updated weights for policy 0, policy_version 1381739 (0.0008) [2023-12-27 01:27:14,780][105620] Updated weights for policy 1, policy_version 1383835 (0.0010) [2023-12-27 01:27:14,846][105620] Updated weights for policy 1, policy_version 1383845 (0.0010) [2023-12-27 01:27:14,901][105620] Updated weights for policy 1, policy_version 1383855 (0.0010) [2023-12-27 01:27:15,190][105692] Updated weights for policy 0, policy_version 1381749 (0.0008) [2023-12-27 01:27:15,258][105692] Updated weights for policy 0, policy_version 1381759 (0.0009) [2023-12-27 01:27:15,323][105692] Updated weights for policy 0, policy_version 1381769 (0.0008) [2023-12-27 01:27:15,617][105620] Updated weights for policy 1, policy_version 1383865 (0.0010) [2023-12-27 01:27:15,664][105620] Updated weights for policy 1, policy_version 1383875 (0.0010) [2023-12-27 01:27:15,709][105620] Updated weights for policy 1, policy_version 1383885 (0.0010) [2023-12-27 01:27:15,756][105620] Updated weights for policy 1, policy_version 1383895 (0.0010) [2023-12-27 01:27:15,986][105692] Updated weights for policy 0, policy_version 1381779 (0.0008) [2023-12-27 01:27:16,041][105692] Updated weights for policy 0, policy_version 1381789 (0.0008) [2023-12-27 01:27:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.9, 300 sec: 19438.6). Total num frames: 708108288. Throughput: 0: 9796.0, 1: 9781.1. Samples: 708081240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:27:16,062][104569] Avg episode reward: [(0, '8067.313'), (1, '8810.073')] [2023-12-27 01:27:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001383896_354320384.pth... [2023-12-27 01:27:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001382744_354025472.pth [2023-12-27 01:27:16,100][105692] Updated weights for policy 0, policy_version 1381799 (0.0008) [2023-12-27 01:27:16,155][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001381808_353796096.pth... [2023-12-27 01:27:16,160][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001380688_353509376.pth [2023-12-27 01:27:16,532][105620] Updated weights for policy 1, policy_version 1383905 (0.0010) [2023-12-27 01:27:16,580][105620] Updated weights for policy 1, policy_version 1383915 (0.0010) [2023-12-27 01:27:16,639][105620] Updated weights for policy 1, policy_version 1383925 (0.0010) [2023-12-27 01:27:16,873][105692] Updated weights for policy 0, policy_version 1381809 (0.0009) [2023-12-27 01:27:16,917][105692] Updated weights for policy 0, policy_version 1381819 (0.0007) [2023-12-27 01:27:16,966][105692] Updated weights for policy 0, policy_version 1381829 (0.0008) [2023-12-27 01:27:17,025][105692] Updated weights for policy 0, policy_version 1381839 (0.0008) [2023-12-27 01:27:17,377][105620] Updated weights for policy 1, policy_version 1383935 (0.0010) [2023-12-27 01:27:17,442][105620] Updated weights for policy 1, policy_version 1383945 (0.0010) [2023-12-27 01:27:17,505][105620] Updated weights for policy 1, policy_version 1383955 (0.0010) [2023-12-27 01:27:17,812][105692] Updated weights for policy 0, policy_version 1381849 (0.0008) [2023-12-27 01:27:17,874][105692] Updated weights for policy 0, policy_version 1381859 (0.0008) [2023-12-27 01:27:17,929][105692] Updated weights for policy 0, policy_version 1381869 (0.0008) [2023-12-27 01:27:18,250][105620] Updated weights for policy 1, policy_version 1383965 (0.0010) [2023-12-27 01:27:18,300][105620] Updated weights for policy 1, policy_version 1383975 (0.0010) [2023-12-27 01:27:18,370][105620] Updated weights for policy 1, policy_version 1383985 (0.0011) [2023-12-27 01:27:18,692][105692] Updated weights for policy 0, policy_version 1381879 (0.0010) [2023-12-27 01:27:18,741][105692] Updated weights for policy 0, policy_version 1381889 (0.0010) [2023-12-27 01:27:18,789][105692] Updated weights for policy 0, policy_version 1381899 (0.0010) [2023-12-27 01:27:19,098][105620] Updated weights for policy 1, policy_version 1383995 (0.0011) [2023-12-27 01:27:19,156][105620] Updated weights for policy 1, policy_version 1384005 (0.0010) [2023-12-27 01:27:19,208][105620] Updated weights for policy 1, policy_version 1384015 (0.0010) [2023-12-27 01:27:19,437][105692] Updated weights for policy 0, policy_version 1381909 (0.0009) [2023-12-27 01:27:19,488][105692] Updated weights for policy 0, policy_version 1381919 (0.0008) [2023-12-27 01:27:19,543][105692] Updated weights for policy 0, policy_version 1381929 (0.0008) [2023-12-27 01:27:19,956][105620] Updated weights for policy 1, policy_version 1384025 (0.0010) [2023-12-27 01:27:20,021][105620] Updated weights for policy 1, policy_version 1384035 (0.0009) [2023-12-27 01:27:20,085][105620] Updated weights for policy 1, policy_version 1384045 (0.0010) [2023-12-27 01:27:20,154][105620] Updated weights for policy 1, policy_version 1384055 (0.0010) [2023-12-27 01:27:20,271][105692] Updated weights for policy 0, policy_version 1381939 (0.0007) [2023-12-27 01:27:20,326][105692] Updated weights for policy 0, policy_version 1381949 (0.0008) [2023-12-27 01:27:20,389][105692] Updated weights for policy 0, policy_version 1381959 (0.0008) [2023-12-27 01:27:20,946][105620] Updated weights for policy 1, policy_version 1384065 (0.0009) [2023-12-27 01:27:21,005][105620] Updated weights for policy 1, policy_version 1384075 (0.0008) [2023-12-27 01:27:21,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 708198400. Throughput: 0: 9711.7, 1: 9672.5. Samples: 708194816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:27:21,062][104569] Avg episode reward: [(0, '8156.889'), (1, '8446.388')] [2023-12-27 01:27:21,084][105620] Updated weights for policy 1, policy_version 1384085 (0.0009) [2023-12-27 01:27:21,153][105692] Updated weights for policy 0, policy_version 1381969 (0.0008) [2023-12-27 01:27:21,220][105692] Updated weights for policy 0, policy_version 1381979 (0.0008) [2023-12-27 01:27:21,284][105692] Updated weights for policy 0, policy_version 1381989 (0.0009) [2023-12-27 01:27:21,348][105692] Updated weights for policy 0, policy_version 1381999 (0.0008) [2023-12-27 01:27:21,882][105620] Updated weights for policy 1, policy_version 1384095 (0.0009) [2023-12-27 01:27:21,941][105620] Updated weights for policy 1, policy_version 1384105 (0.0009) [2023-12-27 01:27:22,010][105620] Updated weights for policy 1, policy_version 1384115 (0.0009) [2023-12-27 01:27:22,128][105692] Updated weights for policy 0, policy_version 1382009 (0.0009) [2023-12-27 01:27:22,193][105692] Updated weights for policy 0, policy_version 1382019 (0.0009) [2023-12-27 01:27:22,255][105692] Updated weights for policy 0, policy_version 1382029 (0.0009) [2023-12-27 01:27:22,778][105620] Updated weights for policy 1, policy_version 1384125 (0.0009) [2023-12-27 01:27:22,841][105620] Updated weights for policy 1, policy_version 1384135 (0.0009) [2023-12-27 01:27:22,904][105620] Updated weights for policy 1, policy_version 1384145 (0.0009) [2023-12-27 01:27:22,994][105692] Updated weights for policy 0, policy_version 1382039 (0.0009) [2023-12-27 01:27:23,056][105692] Updated weights for policy 0, policy_version 1382049 (0.0008) [2023-12-27 01:27:23,117][105692] Updated weights for policy 0, policy_version 1382059 (0.0009) [2023-12-27 01:27:23,683][105620] Updated weights for policy 1, policy_version 1384155 (0.0009) [2023-12-27 01:27:23,730][105620] Updated weights for policy 1, policy_version 1384165 (0.0009) [2023-12-27 01:27:23,780][105620] Updated weights for policy 1, policy_version 1384175 (0.0008) [2023-12-27 01:27:23,826][105692] Updated weights for policy 0, policy_version 1382069 (0.0008) [2023-12-27 01:27:23,892][105692] Updated weights for policy 0, policy_version 1382079 (0.0010) [2023-12-27 01:27:23,941][105692] Updated weights for policy 0, policy_version 1382089 (0.0008) [2023-12-27 01:27:24,409][105620] Updated weights for policy 1, policy_version 1384185 (0.0006) [2023-12-27 01:27:24,463][105620] Updated weights for policy 1, policy_version 1384195 (0.0009) [2023-12-27 01:27:24,521][105620] Updated weights for policy 1, policy_version 1384205 (0.0009) [2023-12-27 01:27:24,586][105620] Updated weights for policy 1, policy_version 1384215 (0.0009) [2023-12-27 01:27:24,798][105692] Updated weights for policy 0, policy_version 1382099 (0.0009) [2023-12-27 01:27:24,862][105692] Updated weights for policy 0, policy_version 1382109 (0.0008) [2023-12-27 01:27:24,930][105692] Updated weights for policy 0, policy_version 1382119 (0.0010) [2023-12-27 01:27:25,204][105620] Updated weights for policy 1, policy_version 1384225 (0.0006) [2023-12-27 01:27:25,260][105620] Updated weights for policy 1, policy_version 1384235 (0.0005) [2023-12-27 01:27:25,323][105620] Updated weights for policy 1, policy_version 1384245 (0.0007) [2023-12-27 01:27:25,664][105692] Updated weights for policy 0, policy_version 1382129 (0.0009) [2023-12-27 01:27:25,714][105692] Updated weights for policy 0, policy_version 1382139 (0.0008) [2023-12-27 01:27:25,764][105692] Updated weights for policy 0, policy_version 1382149 (0.0008) [2023-12-27 01:27:25,814][105692] Updated weights for policy 0, policy_version 1382159 (0.0008) [2023-12-27 01:27:26,005][105620] Updated weights for policy 1, policy_version 1384255 (0.0009) [2023-12-27 01:27:26,059][105620] Updated weights for policy 1, policy_version 1384265 (0.0009) [2023-12-27 01:27:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 708296704. Throughput: 0: 9589.9, 1: 9651.7. Samples: 708307592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:27:26,062][104569] Avg episode reward: [(0, '8345.213'), (1, '8900.861')] [2023-12-27 01:27:26,122][105620] Updated weights for policy 1, policy_version 1384275 (0.0009) [2023-12-27 01:27:26,577][105692] Updated weights for policy 0, policy_version 1382169 (0.0005) [2023-12-27 01:27:26,635][105692] Updated weights for policy 0, policy_version 1382179 (0.0006) [2023-12-27 01:27:26,682][105692] Updated weights for policy 0, policy_version 1382189 (0.0005) [2023-12-27 01:27:26,838][105620] Updated weights for policy 1, policy_version 1384285 (0.0008) [2023-12-27 01:27:26,894][105620] Updated weights for policy 1, policy_version 1384295 (0.0009) [2023-12-27 01:27:26,950][105620] Updated weights for policy 1, policy_version 1384305 (0.0009) [2023-12-27 01:27:27,294][105692] Updated weights for policy 0, policy_version 1382199 (0.0009) [2023-12-27 01:27:27,344][105692] Updated weights for policy 0, policy_version 1382209 (0.0008) [2023-12-27 01:27:27,398][105692] Updated weights for policy 0, policy_version 1382219 (0.0009) [2023-12-27 01:27:27,657][105620] Updated weights for policy 1, policy_version 1384315 (0.0009) [2023-12-27 01:27:27,706][105620] Updated weights for policy 1, policy_version 1384325 (0.0008) [2023-12-27 01:27:27,760][105620] Updated weights for policy 1, policy_version 1384335 (0.0008) [2023-12-27 01:27:28,143][105692] Updated weights for policy 0, policy_version 1382229 (0.0009) [2023-12-27 01:27:28,196][105692] Updated weights for policy 0, policy_version 1382239 (0.0008) [2023-12-27 01:27:28,245][105692] Updated weights for policy 0, policy_version 1382249 (0.0005) [2023-12-27 01:27:28,503][105620] Updated weights for policy 1, policy_version 1384345 (0.0009) [2023-12-27 01:27:28,563][105620] Updated weights for policy 1, policy_version 1384355 (0.0008) [2023-12-27 01:27:28,610][105620] Updated weights for policy 1, policy_version 1384365 (0.0009) [2023-12-27 01:27:28,663][105620] Updated weights for policy 1, policy_version 1384376 (0.0010) [2023-12-27 01:27:28,938][105692] Updated weights for policy 0, policy_version 1382259 (0.0008) [2023-12-27 01:27:28,991][105692] Updated weights for policy 0, policy_version 1382269 (0.0008) [2023-12-27 01:27:29,036][105692] Updated weights for policy 0, policy_version 1382279 (0.0008) [2023-12-27 01:27:29,445][105620] Updated weights for policy 1, policy_version 1384386 (0.0008) [2023-12-27 01:27:29,498][105620] Updated weights for policy 1, policy_version 1384396 (0.0008) [2023-12-27 01:27:29,545][105620] Updated weights for policy 1, policy_version 1384406 (0.0008) [2023-12-27 01:27:29,771][105692] Updated weights for policy 0, policy_version 1382289 (0.0008) [2023-12-27 01:27:29,825][105692] Updated weights for policy 0, policy_version 1382299 (0.0008) [2023-12-27 01:27:29,880][105692] Updated weights for policy 0, policy_version 1382309 (0.0007) [2023-12-27 01:27:29,937][105692] Updated weights for policy 0, policy_version 1382319 (0.0006) [2023-12-27 01:27:30,342][105620] Updated weights for policy 1, policy_version 1384416 (0.0009) [2023-12-27 01:27:30,395][105620] Updated weights for policy 1, policy_version 1384426 (0.0008) [2023-12-27 01:27:30,445][105620] Updated weights for policy 1, policy_version 1384436 (0.0009) [2023-12-27 01:27:30,635][105692] Updated weights for policy 0, policy_version 1382329 (0.0009) [2023-12-27 01:27:30,689][105692] Updated weights for policy 0, policy_version 1382339 (0.0009) [2023-12-27 01:27:30,742][105692] Updated weights for policy 0, policy_version 1382349 (0.0010) [2023-12-27 01:27:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 708395008. Throughput: 0: 9598.6, 1: 9688.3. Samples: 708365332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:27:31,063][104569] Avg episode reward: [(0, '8343.846'), (1, '9269.669')] [2023-12-27 01:27:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001382352_353935360.pth... [2023-12-27 01:27:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001384440_354459648.pth... [2023-12-27 01:27:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001381264_353656832.pth [2023-12-27 01:27:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001383320_354172928.pth [2023-12-27 01:27:31,113][105620] Updated weights for policy 1, policy_version 1384446 (0.0009) [2023-12-27 01:27:31,175][105620] Updated weights for policy 1, policy_version 1384456 (0.0009) [2023-12-27 01:27:31,229][105620] Updated weights for policy 1, policy_version 1384466 (0.0010) [2023-12-27 01:27:31,502][105692] Updated weights for policy 0, policy_version 1382359 (0.0008) [2023-12-27 01:27:31,561][105692] Updated weights for policy 0, policy_version 1382369 (0.0009) [2023-12-27 01:27:31,619][105692] Updated weights for policy 0, policy_version 1382379 (0.0009) [2023-12-27 01:27:32,005][105620] Updated weights for policy 1, policy_version 1384476 (0.0009) [2023-12-27 01:27:32,062][105620] Updated weights for policy 1, policy_version 1384487 (0.0010) [2023-12-27 01:27:32,114][105620] Updated weights for policy 1, policy_version 1384497 (0.0009) [2023-12-27 01:27:32,338][105692] Updated weights for policy 0, policy_version 1382389 (0.0009) [2023-12-27 01:27:32,407][105692] Updated weights for policy 0, policy_version 1382399 (0.0009) [2023-12-27 01:27:32,473][105692] Updated weights for policy 0, policy_version 1382409 (0.0009) [2023-12-27 01:27:32,853][105620] Updated weights for policy 1, policy_version 1384507 (0.0009) [2023-12-27 01:27:32,899][105620] Updated weights for policy 1, policy_version 1384517 (0.0009) [2023-12-27 01:27:32,947][105620] Updated weights for policy 1, policy_version 1384527 (0.0009) [2023-12-27 01:27:33,254][105692] Updated weights for policy 0, policy_version 1382419 (0.0007) [2023-12-27 01:27:33,306][105692] Updated weights for policy 0, policy_version 1382430 (0.0010) [2023-12-27 01:27:33,358][105692] Updated weights for policy 0, policy_version 1382440 (0.0008) [2023-12-27 01:27:33,527][105620] Updated weights for policy 1, policy_version 1384537 (0.0009) [2023-12-27 01:27:33,592][105620] Updated weights for policy 1, policy_version 1384547 (0.0005) [2023-12-27 01:27:33,636][105620] Updated weights for policy 1, policy_version 1384557 (0.0005) [2023-12-27 01:27:33,684][105620] Updated weights for policy 1, policy_version 1384567 (0.0005) [2023-12-27 01:27:34,241][105692] Updated weights for policy 0, policy_version 1382450 (0.0007) [2023-12-27 01:27:34,274][105620] Updated weights for policy 1, policy_version 1384577 (0.0009) [2023-12-27 01:27:34,297][105692] Updated weights for policy 0, policy_version 1382460 (0.0006) [2023-12-27 01:27:34,338][105620] Updated weights for policy 1, policy_version 1384587 (0.0009) [2023-12-27 01:27:34,353][105692] Updated weights for policy 0, policy_version 1382470 (0.0008) [2023-12-27 01:27:34,401][105620] Updated weights for policy 1, policy_version 1384597 (0.0007) [2023-12-27 01:27:34,407][105692] Updated weights for policy 0, policy_version 1382480 (0.0009) [2023-12-27 01:27:35,151][105620] Updated weights for policy 1, policy_version 1384607 (0.0007) [2023-12-27 01:27:35,185][105692] Updated weights for policy 0, policy_version 1382490 (0.0008) [2023-12-27 01:27:35,206][105620] Updated weights for policy 1, policy_version 1384617 (0.0005) [2023-12-27 01:27:35,251][105692] Updated weights for policy 0, policy_version 1382500 (0.0007) [2023-12-27 01:27:35,261][105620] Updated weights for policy 1, policy_version 1384627 (0.0005) [2023-12-27 01:27:35,308][105692] Updated weights for policy 0, policy_version 1382510 (0.0009) [2023-12-27 01:27:35,960][105620] Updated weights for policy 1, policy_version 1384637 (0.0005) [2023-12-27 01:27:36,018][105620] Updated weights for policy 1, policy_version 1384647 (0.0005) [2023-12-27 01:27:36,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19438.7). Total num frames: 708485120. Throughput: 0: 9455.2, 1: 9799.2. Samples: 708481916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:27:36,062][104569] Avg episode reward: [(0, '8160.951'), (1, '9080.421')] [2023-12-27 01:27:36,080][105620] Updated weights for policy 1, policy_version 1384657 (0.0005) [2023-12-27 01:27:36,082][105692] Updated weights for policy 0, policy_version 1382520 (0.0007) [2023-12-27 01:27:36,145][105692] Updated weights for policy 0, policy_version 1382530 (0.0009) [2023-12-27 01:27:36,209][105692] Updated weights for policy 0, policy_version 1382540 (0.0008) [2023-12-27 01:27:36,676][105620] Updated weights for policy 1, policy_version 1384667 (0.0008) [2023-12-27 01:27:36,741][105620] Updated weights for policy 1, policy_version 1384677 (0.0006) [2023-12-27 01:27:36,797][105620] Updated weights for policy 1, policy_version 1384687 (0.0009) [2023-12-27 01:27:37,023][105692] Updated weights for policy 0, policy_version 1382550 (0.0009) [2023-12-27 01:27:37,086][105692] Updated weights for policy 0, policy_version 1382560 (0.0009) [2023-12-27 01:27:37,153][105692] Updated weights for policy 0, policy_version 1382570 (0.0007) [2023-12-27 01:27:37,433][105620] Updated weights for policy 1, policy_version 1384697 (0.0008) [2023-12-27 01:27:37,494][105620] Updated weights for policy 1, policy_version 1384707 (0.0009) [2023-12-27 01:27:37,552][105620] Updated weights for policy 1, policy_version 1384717 (0.0008) [2023-12-27 01:27:37,617][105620] Updated weights for policy 1, policy_version 1384727 (0.0009) [2023-12-27 01:27:37,888][105692] Updated weights for policy 0, policy_version 1382580 (0.0009) [2023-12-27 01:27:37,945][105692] Updated weights for policy 0, policy_version 1382590 (0.0008) [2023-12-27 01:27:37,996][105692] Updated weights for policy 0, policy_version 1382600 (0.0009) [2023-12-27 01:27:38,333][105620] Updated weights for policy 1, policy_version 1384737 (0.0008) [2023-12-27 01:27:38,402][105620] Updated weights for policy 1, policy_version 1384747 (0.0007) [2023-12-27 01:27:38,466][105620] Updated weights for policy 1, policy_version 1384757 (0.0005) [2023-12-27 01:27:38,880][105692] Updated weights for policy 0, policy_version 1382610 (0.0009) [2023-12-27 01:27:38,942][105692] Updated weights for policy 0, policy_version 1382620 (0.0010) [2023-12-27 01:27:38,985][105620] Updated weights for policy 1, policy_version 1384767 (0.0006) [2023-12-27 01:27:39,002][105692] Updated weights for policy 0, policy_version 1382630 (0.0010) [2023-12-27 01:27:39,044][105620] Updated weights for policy 1, policy_version 1384777 (0.0009) [2023-12-27 01:27:39,059][105692] Updated weights for policy 0, policy_version 1382640 (0.0006) [2023-12-27 01:27:39,096][105620] Updated weights for policy 1, policy_version 1384787 (0.0008) [2023-12-27 01:27:39,774][105620] Updated weights for policy 1, policy_version 1384797 (0.0010) [2023-12-27 01:27:39,837][105620] Updated weights for policy 1, policy_version 1384807 (0.0009) [2023-12-27 01:27:39,898][105620] Updated weights for policy 1, policy_version 1384817 (0.0008) [2023-12-27 01:27:39,933][105692] Updated weights for policy 0, policy_version 1382650 (0.0008) [2023-12-27 01:27:40,001][105692] Updated weights for policy 0, policy_version 1382660 (0.0009) [2023-12-27 01:27:40,069][105692] Updated weights for policy 0, policy_version 1382670 (0.0008) [2023-12-27 01:27:40,687][105620] Updated weights for policy 1, policy_version 1384827 (0.0008) [2023-12-27 01:27:40,749][105620] Updated weights for policy 1, policy_version 1384837 (0.0008) [2023-12-27 01:27:40,783][105692] Updated weights for policy 0, policy_version 1382680 (0.0008) [2023-12-27 01:27:40,803][105620] Updated weights for policy 1, policy_version 1384847 (0.0006) [2023-12-27 01:27:40,841][105692] Updated weights for policy 0, policy_version 1382690 (0.0008) [2023-12-27 01:27:40,892][105692] Updated weights for policy 0, policy_version 1382700 (0.0007) [2023-12-27 01:27:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 708591616. Throughput: 0: 9297.6, 1: 9857.9. Samples: 708596520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:27:41,062][104569] Avg episode reward: [(0, '7979.995'), (1, '9080.438')] [2023-12-27 01:27:41,553][105620] Updated weights for policy 1, policy_version 1384857 (0.0008) [2023-12-27 01:27:41,606][105620] Updated weights for policy 1, policy_version 1384867 (0.0009) [2023-12-27 01:27:41,674][105620] Updated weights for policy 1, policy_version 1384877 (0.0010) [2023-12-27 01:27:41,686][105692] Updated weights for policy 0, policy_version 1382710 (0.0008) [2023-12-27 01:27:41,741][105620] Updated weights for policy 1, policy_version 1384887 (0.0009) [2023-12-27 01:27:41,758][105692] Updated weights for policy 0, policy_version 1382720 (0.0008) [2023-12-27 01:27:41,832][105692] Updated weights for policy 0, policy_version 1382730 (0.0007) [2023-12-27 01:27:42,406][105620] Updated weights for policy 1, policy_version 1384897 (0.0007) [2023-12-27 01:27:42,464][105620] Updated weights for policy 1, policy_version 1384907 (0.0009) [2023-12-27 01:27:42,518][105620] Updated weights for policy 1, policy_version 1384917 (0.0008) [2023-12-27 01:27:42,593][105692] Updated weights for policy 0, policy_version 1382740 (0.0010) [2023-12-27 01:27:42,645][105692] Updated weights for policy 0, policy_version 1382750 (0.0009) [2023-12-27 01:27:42,704][105692] Updated weights for policy 0, policy_version 1382760 (0.0009) [2023-12-27 01:27:43,279][105620] Updated weights for policy 1, policy_version 1384927 (0.0009) [2023-12-27 01:27:43,325][105620] Updated weights for policy 1, policy_version 1384937 (0.0008) [2023-12-27 01:27:43,368][105620] Updated weights for policy 1, policy_version 1384947 (0.0007) [2023-12-27 01:27:43,455][105692] Updated weights for policy 0, policy_version 1382770 (0.0009) [2023-12-27 01:27:43,502][105692] Updated weights for policy 0, policy_version 1382780 (0.0009) [2023-12-27 01:27:43,550][105692] Updated weights for policy 0, policy_version 1382790 (0.0008) [2023-12-27 01:27:44,133][105620] Updated weights for policy 1, policy_version 1384957 (0.0008) [2023-12-27 01:27:44,188][105620] Updated weights for policy 1, policy_version 1384967 (0.0008) [2023-12-27 01:27:44,251][105620] Updated weights for policy 1, policy_version 1384977 (0.0009) [2023-12-27 01:27:44,321][105692] Updated weights for policy 0, policy_version 1382801 (0.0009) [2023-12-27 01:27:44,374][105692] Updated weights for policy 0, policy_version 1382811 (0.0009) [2023-12-27 01:27:44,422][105692] Updated weights for policy 0, policy_version 1382821 (0.0009) [2023-12-27 01:27:44,476][105692] Updated weights for policy 0, policy_version 1382831 (0.0008) [2023-12-27 01:27:45,088][105620] Updated weights for policy 1, policy_version 1384987 (0.0009) [2023-12-27 01:27:45,146][105620] Updated weights for policy 1, policy_version 1384997 (0.0009) [2023-12-27 01:27:45,184][105692] Updated weights for policy 0, policy_version 1382841 (0.0006) [2023-12-27 01:27:45,203][105620] Updated weights for policy 1, policy_version 1385007 (0.0007) [2023-12-27 01:27:45,246][105692] Updated weights for policy 0, policy_version 1382851 (0.0007) [2023-12-27 01:27:45,297][105692] Updated weights for policy 0, policy_version 1382861 (0.0009) [2023-12-27 01:27:45,957][105692] Updated weights for policy 0, policy_version 1382871 (0.0007) [2023-12-27 01:27:45,966][105620] Updated weights for policy 1, policy_version 1385017 (0.0007) [2023-12-27 01:27:46,005][105585] KL-divergence is very high: 114.5734 [2023-12-27 01:27:46,007][105692] Updated weights for policy 0, policy_version 1382881 (0.0009) [2023-12-27 01:27:46,010][105585] KL-divergence is very high: 127.1324 [2023-12-27 01:27:46,029][105620] Updated weights for policy 1, policy_version 1385027 (0.0005) [2023-12-27 01:27:46,044][105585] KL-divergence is very high: 101.5722 [2023-12-27 01:27:46,049][105585] KL-divergence is very high: 115.6121 [2023-12-27 01:27:46,055][105692] Updated weights for policy 0, policy_version 1382891 (0.0009) [2023-12-27 01:27:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.8, 300 sec: 19410.9). Total num frames: 708673536. Throughput: 0: 9229.0, 1: 9720.7. Samples: 708651876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:27:46,062][104569] Avg episode reward: [(0, '8067.400'), (1, '9266.548')] [2023-12-27 01:27:46,078][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001382896_354074624.pth... [2023-12-27 01:27:46,082][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001381808_353796096.pth [2023-12-27 01:27:46,087][105620] Updated weights for policy 1, policy_version 1385037 (0.0007) [2023-12-27 01:27:46,140][105620] Updated weights for policy 1, policy_version 1385047 (0.0010) [2023-12-27 01:27:46,141][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001385048_354615296.pth... [2023-12-27 01:27:46,148][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001383896_354320384.pth [2023-12-27 01:27:46,699][105692] Updated weights for policy 0, policy_version 1382901 (0.0007) [2023-12-27 01:27:46,719][105620] Updated weights for policy 1, policy_version 1385057 (0.0006) [2023-12-27 01:27:46,751][105692] Updated weights for policy 0, policy_version 1382911 (0.0008) [2023-12-27 01:27:46,773][105620] Updated weights for policy 1, policy_version 1385067 (0.0007) [2023-12-27 01:27:46,804][105692] Updated weights for policy 0, policy_version 1382921 (0.0007) [2023-12-27 01:27:46,833][105620] Updated weights for policy 1, policy_version 1385077 (0.0008) [2023-12-27 01:27:47,422][105620] Updated weights for policy 1, policy_version 1385087 (0.0006) [2023-12-27 01:27:47,477][105620] Updated weights for policy 1, policy_version 1385097 (0.0005) [2023-12-27 01:27:47,483][105692] Updated weights for policy 0, policy_version 1382931 (0.0007) [2023-12-27 01:27:47,526][105620] Updated weights for policy 1, policy_version 1385107 (0.0007) [2023-12-27 01:27:47,534][105692] Updated weights for policy 0, policy_version 1382941 (0.0009) [2023-12-27 01:27:47,589][105692] Updated weights for policy 0, policy_version 1382951 (0.0006) [2023-12-27 01:27:48,232][105620] Updated weights for policy 1, policy_version 1385117 (0.0008) [2023-12-27 01:27:48,285][105620] Updated weights for policy 1, policy_version 1385127 (0.0011) [2023-12-27 01:27:48,303][105692] Updated weights for policy 0, policy_version 1382961 (0.0006) [2023-12-27 01:27:48,339][105620] Updated weights for policy 1, policy_version 1385137 (0.0011) [2023-12-27 01:27:48,373][105692] Updated weights for policy 0, policy_version 1382971 (0.0010) [2023-12-27 01:27:48,426][105692] Updated weights for policy 0, policy_version 1382981 (0.0007) [2023-12-27 01:27:48,485][105692] Updated weights for policy 0, policy_version 1382991 (0.0007) [2023-12-27 01:27:49,144][105620] Updated weights for policy 1, policy_version 1385147 (0.0007) [2023-12-27 01:27:49,171][105692] Updated weights for policy 0, policy_version 1383001 (0.0007) [2023-12-27 01:27:49,195][105620] Updated weights for policy 1, policy_version 1385157 (0.0007) [2023-12-27 01:27:49,230][105692] Updated weights for policy 0, policy_version 1383011 (0.0007) [2023-12-27 01:27:49,256][105620] Updated weights for policy 1, policy_version 1385167 (0.0008) [2023-12-27 01:27:49,291][105692] Updated weights for policy 0, policy_version 1383021 (0.0007) [2023-12-27 01:27:49,980][105620] Updated weights for policy 1, policy_version 1385177 (0.0008) [2023-12-27 01:27:50,042][105620] Updated weights for policy 1, policy_version 1385187 (0.0009) [2023-12-27 01:27:50,095][105620] Updated weights for policy 1, policy_version 1385197 (0.0011) [2023-12-27 01:27:50,118][105692] Updated weights for policy 0, policy_version 1383031 (0.0007) [2023-12-27 01:27:50,148][105620] Updated weights for policy 1, policy_version 1385207 (0.0011) [2023-12-27 01:27:50,168][105692] Updated weights for policy 0, policy_version 1383041 (0.0006) [2023-12-27 01:27:50,217][105692] Updated weights for policy 0, policy_version 1383051 (0.0008) [2023-12-27 01:27:50,898][105620] Updated weights for policy 1, policy_version 1385217 (0.0010) [2023-12-27 01:27:50,949][105620] Updated weights for policy 1, policy_version 1385227 (0.0008) [2023-12-27 01:27:50,997][105620] Updated weights for policy 1, policy_version 1385237 (0.0008) [2023-12-27 01:27:51,012][105692] Updated weights for policy 0, policy_version 1383061 (0.0008) [2023-12-27 01:27:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 708780032. Throughput: 0: 9328.6, 1: 9695.7. Samples: 708770664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:27:51,062][104569] Avg episode reward: [(0, '7883.350'), (1, '8995.601')] [2023-12-27 01:27:51,073][105692] Updated weights for policy 0, policy_version 1383071 (0.0010) [2023-12-27 01:27:51,134][105692] Updated weights for policy 0, policy_version 1383081 (0.0010) [2023-12-27 01:27:51,679][105620] Updated weights for policy 1, policy_version 1385247 (0.0006) [2023-12-27 01:27:51,728][105620] Updated weights for policy 1, policy_version 1385257 (0.0006) [2023-12-27 01:27:51,800][105620] Updated weights for policy 1, policy_version 1385267 (0.0007) [2023-12-27 01:27:51,959][105692] Updated weights for policy 0, policy_version 1383091 (0.0009) [2023-12-27 01:27:52,025][105692] Updated weights for policy 0, policy_version 1383101 (0.0007) [2023-12-27 01:27:52,085][105692] Updated weights for policy 0, policy_version 1383111 (0.0008) [2023-12-27 01:27:52,516][105620] Updated weights for policy 1, policy_version 1385277 (0.0007) [2023-12-27 01:27:52,567][105620] Updated weights for policy 1, policy_version 1385288 (0.0009) [2023-12-27 01:27:52,617][105620] Updated weights for policy 1, policy_version 1385298 (0.0009) [2023-12-27 01:27:52,884][105692] Updated weights for policy 0, policy_version 1383121 (0.0009) [2023-12-27 01:27:52,940][105692] Updated weights for policy 0, policy_version 1383131 (0.0010) [2023-12-27 01:27:52,986][105585] KL-divergence is very high: 101.6935 [2023-12-27 01:27:52,990][105692] Updated weights for policy 0, policy_version 1383142 (0.0007) [2023-12-27 01:27:53,025][105585] KL-divergence is very high: 147.9388 [2023-12-27 01:27:53,036][105585] KL-divergence is very high: 108.3715 [2023-12-27 01:27:53,043][105692] Updated weights for policy 0, policy_version 1383152 (0.0005) [2023-12-27 01:27:53,282][105620] Updated weights for policy 1, policy_version 1385308 (0.0007) [2023-12-27 01:27:53,349][105620] Updated weights for policy 1, policy_version 1385318 (0.0007) [2023-12-27 01:27:53,406][105620] Updated weights for policy 1, policy_version 1385328 (0.0008) [2023-12-27 01:27:53,646][105692] Updated weights for policy 0, policy_version 1383162 (0.0011) [2023-12-27 01:27:53,702][105692] Updated weights for policy 0, policy_version 1383172 (0.0011) [2023-12-27 01:27:53,751][105692] Updated weights for policy 0, policy_version 1383182 (0.0010) [2023-12-27 01:27:54,077][105620] Updated weights for policy 1, policy_version 1385338 (0.0010) [2023-12-27 01:27:54,131][105620] Updated weights for policy 1, policy_version 1385348 (0.0008) [2023-12-27 01:27:54,186][105620] Updated weights for policy 1, policy_version 1385358 (0.0005) [2023-12-27 01:27:54,242][105620] Updated weights for policy 1, policy_version 1385368 (0.0005) [2023-12-27 01:27:54,507][105692] Updated weights for policy 0, policy_version 1383192 (0.0011) [2023-12-27 01:27:54,568][105692] Updated weights for policy 0, policy_version 1383202 (0.0011) [2023-12-27 01:27:54,628][105692] Updated weights for policy 0, policy_version 1383212 (0.0011) [2023-12-27 01:27:54,804][105620] Updated weights for policy 1, policy_version 1385378 (0.0005) [2023-12-27 01:27:54,851][105620] Updated weights for policy 1, policy_version 1385388 (0.0005) [2023-12-27 01:27:54,897][105620] Updated weights for policy 1, policy_version 1385398 (0.0005) [2023-12-27 01:27:55,315][105692] Updated weights for policy 0, policy_version 1383222 (0.0007) [2023-12-27 01:27:55,361][105692] Updated weights for policy 0, policy_version 1383232 (0.0005) [2023-12-27 01:27:55,414][105692] Updated weights for policy 0, policy_version 1383242 (0.0005) [2023-12-27 01:27:55,558][105620] Updated weights for policy 1, policy_version 1385408 (0.0010) [2023-12-27 01:27:55,612][105620] Updated weights for policy 1, policy_version 1385418 (0.0010) [2023-12-27 01:27:55,663][105620] Updated weights for policy 1, policy_version 1385428 (0.0010) [2023-12-27 01:27:56,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19114.7, 300 sec: 19438.7). Total num frames: 708878336. Throughput: 0: 9350.8, 1: 9792.1. Samples: 708889140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:27:56,062][104569] Avg episode reward: [(0, '7705.778'), (1, '8996.117')] [2023-12-27 01:27:56,119][105692] Updated weights for policy 0, policy_version 1383252 (0.0008) [2023-12-27 01:27:56,167][105692] Updated weights for policy 0, policy_version 1383262 (0.0010) [2023-12-27 01:27:56,214][105692] Updated weights for policy 0, policy_version 1383272 (0.0009) [2023-12-27 01:27:56,411][105620] Updated weights for policy 1, policy_version 1385438 (0.0010) [2023-12-27 01:27:56,473][105620] Updated weights for policy 1, policy_version 1385448 (0.0010) [2023-12-27 01:27:56,522][105620] Updated weights for policy 1, policy_version 1385458 (0.0010) [2023-12-27 01:27:56,861][105692] Updated weights for policy 0, policy_version 1383282 (0.0008) [2023-12-27 01:27:56,927][105692] Updated weights for policy 0, policy_version 1383292 (0.0008) [2023-12-27 01:27:56,984][105692] Updated weights for policy 0, policy_version 1383302 (0.0007) [2023-12-27 01:27:57,041][105692] Updated weights for policy 0, policy_version 1383312 (0.0005) [2023-12-27 01:27:57,275][105620] Updated weights for policy 1, policy_version 1385468 (0.0010) [2023-12-27 01:27:57,328][105620] Updated weights for policy 1, policy_version 1385478 (0.0005) [2023-12-27 01:27:57,387][105620] Updated weights for policy 1, policy_version 1385488 (0.0005) [2023-12-27 01:27:57,809][105692] Updated weights for policy 0, policy_version 1383323 (0.0010) [2023-12-27 01:27:57,863][105692] Updated weights for policy 0, policy_version 1383335 (0.0010) [2023-12-27 01:27:57,896][105620] Updated weights for policy 1, policy_version 1385498 (0.0005) [2023-12-27 01:27:57,942][105620] Updated weights for policy 1, policy_version 1385508 (0.0005) [2023-12-27 01:27:57,996][105620] Updated weights for policy 1, policy_version 1385518 (0.0005) [2023-12-27 01:27:58,050][105620] Updated weights for policy 1, policy_version 1385528 (0.0005) [2023-12-27 01:27:58,763][105620] Updated weights for policy 1, policy_version 1385538 (0.0008) [2023-12-27 01:27:58,776][105692] Updated weights for policy 0, policy_version 1383346 (0.0009) [2023-12-27 01:27:58,834][105620] Updated weights for policy 1, policy_version 1385548 (0.0009) [2023-12-27 01:27:58,858][105692] Updated weights for policy 0, policy_version 1383356 (0.0009) [2023-12-27 01:27:58,906][105620] Updated weights for policy 1, policy_version 1385558 (0.0009) [2023-12-27 01:27:58,924][105692] Updated weights for policy 0, policy_version 1383366 (0.0008) [2023-12-27 01:27:58,981][105692] Updated weights for policy 0, policy_version 1383376 (0.0009) [2023-12-27 01:27:59,674][105620] Updated weights for policy 1, policy_version 1385568 (0.0009) [2023-12-27 01:27:59,729][105620] Updated weights for policy 1, policy_version 1385578 (0.0009) [2023-12-27 01:27:59,786][105620] Updated weights for policy 1, policy_version 1385588 (0.0007) [2023-12-27 01:27:59,804][105692] Updated weights for policy 0, policy_version 1383386 (0.0008) [2023-12-27 01:27:59,863][105692] Updated weights for policy 0, policy_version 1383396 (0.0009) [2023-12-27 01:27:59,911][105692] Updated weights for policy 0, policy_version 1383406 (0.0009) [2023-12-27 01:28:00,504][105620] Updated weights for policy 1, policy_version 1385598 (0.0006) [2023-12-27 01:28:00,555][105620] Updated weights for policy 1, policy_version 1385608 (0.0005) [2023-12-27 01:28:00,620][105620] Updated weights for policy 1, policy_version 1385618 (0.0006) [2023-12-27 01:28:00,722][105692] Updated weights for policy 0, policy_version 1383416 (0.0006) [2023-12-27 01:28:00,767][105692] Updated weights for policy 0, policy_version 1383426 (0.0008) [2023-12-27 01:28:00,811][105692] Updated weights for policy 0, policy_version 1383436 (0.0007) [2023-12-27 01:28:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19114.7, 300 sec: 19438.7). Total num frames: 708976640. Throughput: 0: 9400.0, 1: 9859.1. Samples: 708947900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:28:01,062][104569] Avg episode reward: [(0, '7984.547'), (1, '8993.169')] [2023-12-27 01:28:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001383440_354213888.pth... [2023-12-27 01:28:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001385624_354762752.pth... [2023-12-27 01:28:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001382352_353935360.pth [2023-12-27 01:28:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001384440_354459648.pth [2023-12-27 01:28:01,314][105620] Updated weights for policy 1, policy_version 1385628 (0.0006) [2023-12-27 01:28:01,375][105620] Updated weights for policy 1, policy_version 1385638 (0.0008) [2023-12-27 01:28:01,436][105620] Updated weights for policy 1, policy_version 1385648 (0.0008) [2023-12-27 01:28:01,579][105692] Updated weights for policy 0, policy_version 1383446 (0.0008) [2023-12-27 01:28:01,632][105692] Updated weights for policy 0, policy_version 1383456 (0.0010) [2023-12-27 01:28:01,691][105692] Updated weights for policy 0, policy_version 1383466 (0.0006) [2023-12-27 01:28:02,180][105620] Updated weights for policy 1, policy_version 1385658 (0.0009) [2023-12-27 01:28:02,248][105620] Updated weights for policy 1, policy_version 1385668 (0.0010) [2023-12-27 01:28:02,272][105692] Updated weights for policy 0, policy_version 1383476 (0.0008) [2023-12-27 01:28:02,308][105620] Updated weights for policy 1, policy_version 1385678 (0.0010) [2023-12-27 01:28:02,330][105692] Updated weights for policy 0, policy_version 1383486 (0.0007) [2023-12-27 01:28:02,371][105620] Updated weights for policy 1, policy_version 1385688 (0.0010) [2023-12-27 01:28:02,392][105692] Updated weights for policy 0, policy_version 1383496 (0.0007) [2023-12-27 01:28:02,980][105692] Updated weights for policy 0, policy_version 1383506 (0.0006) [2023-12-27 01:28:02,997][105620] Updated weights for policy 1, policy_version 1385698 (0.0009) [2023-12-27 01:28:03,029][105692] Updated weights for policy 0, policy_version 1383516 (0.0008) [2023-12-27 01:28:03,058][105620] Updated weights for policy 1, policy_version 1385708 (0.0005) [2023-12-27 01:28:03,088][105692] Updated weights for policy 0, policy_version 1383526 (0.0010) [2023-12-27 01:28:03,115][105620] Updated weights for policy 1, policy_version 1385718 (0.0005) [2023-12-27 01:28:03,136][105692] Updated weights for policy 0, policy_version 1383536 (0.0009) [2023-12-27 01:28:03,765][105692] Updated weights for policy 0, policy_version 1383546 (0.0005) [2023-12-27 01:28:03,767][105620] Updated weights for policy 1, policy_version 1385728 (0.0010) [2023-12-27 01:28:03,811][105692] Updated weights for policy 0, policy_version 1383556 (0.0005) [2023-12-27 01:28:03,825][105620] Updated weights for policy 1, policy_version 1385738 (0.0010) [2023-12-27 01:28:03,870][105692] Updated weights for policy 0, policy_version 1383566 (0.0008) [2023-12-27 01:28:03,888][105620] Updated weights for policy 1, policy_version 1385748 (0.0010) [2023-12-27 01:28:04,562][105692] Updated weights for policy 0, policy_version 1383576 (0.0006) [2023-12-27 01:28:04,619][105692] Updated weights for policy 0, policy_version 1383586 (0.0005) [2023-12-27 01:28:04,650][105620] Updated weights for policy 1, policy_version 1385758 (0.0011) [2023-12-27 01:28:04,674][105692] Updated weights for policy 0, policy_version 1383596 (0.0005) [2023-12-27 01:28:04,705][105620] Updated weights for policy 1, policy_version 1385768 (0.0010) [2023-12-27 01:28:04,767][105620] Updated weights for policy 1, policy_version 1385778 (0.0010) [2023-12-27 01:28:05,351][105692] Updated weights for policy 0, policy_version 1383606 (0.0005) [2023-12-27 01:28:05,422][105692] Updated weights for policy 0, policy_version 1383616 (0.0006) [2023-12-27 01:28:05,445][105620] Updated weights for policy 1, policy_version 1385788 (0.0008) [2023-12-27 01:28:05,478][105692] Updated weights for policy 0, policy_version 1383626 (0.0009) [2023-12-27 01:28:05,510][105620] Updated weights for policy 1, policy_version 1385798 (0.0008) [2023-12-27 01:28:05,563][105620] Updated weights for policy 1, policy_version 1385808 (0.0007) [2023-12-27 01:28:06,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19251.1, 300 sec: 19466.4). Total num frames: 709074944. Throughput: 0: 9459.0, 1: 9896.8. Samples: 709065832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:28:06,063][104569] Avg episode reward: [(0, '8161.888'), (1, '8990.338')] [2023-12-27 01:28:06,075][105692] Updated weights for policy 0, policy_version 1383636 (0.0010) [2023-12-27 01:28:06,142][105692] Updated weights for policy 0, policy_version 1383646 (0.0009) [2023-12-27 01:28:06,203][105692] Updated weights for policy 0, policy_version 1383656 (0.0010) [2023-12-27 01:28:06,229][105620] Updated weights for policy 1, policy_version 1385818 (0.0006) [2023-12-27 01:28:06,287][105620] Updated weights for policy 1, policy_version 1385828 (0.0010) [2023-12-27 01:28:06,347][105620] Updated weights for policy 1, policy_version 1385838 (0.0011) [2023-12-27 01:28:06,409][105620] Updated weights for policy 1, policy_version 1385848 (0.0011) [2023-12-27 01:28:06,790][105692] Updated weights for policy 0, policy_version 1383666 (0.0006) [2023-12-27 01:28:06,861][105692] Updated weights for policy 0, policy_version 1383676 (0.0007) [2023-12-27 01:28:06,922][105692] Updated weights for policy 0, policy_version 1383686 (0.0006) [2023-12-27 01:28:06,988][105692] Updated weights for policy 0, policy_version 1383696 (0.0006) [2023-12-27 01:28:07,110][105620] Updated weights for policy 1, policy_version 1385858 (0.0011) [2023-12-27 01:28:07,167][105620] Updated weights for policy 1, policy_version 1385868 (0.0011) [2023-12-27 01:28:07,234][105620] Updated weights for policy 1, policy_version 1385878 (0.0011) [2023-12-27 01:28:07,588][105692] Updated weights for policy 0, policy_version 1383706 (0.0006) [2023-12-27 01:28:07,647][105692] Updated weights for policy 0, policy_version 1383716 (0.0006) [2023-12-27 01:28:07,704][105692] Updated weights for policy 0, policy_version 1383726 (0.0006) [2023-12-27 01:28:07,864][105620] Updated weights for policy 1, policy_version 1385888 (0.0006) [2023-12-27 01:28:07,927][105620] Updated weights for policy 1, policy_version 1385898 (0.0005) [2023-12-27 01:28:07,982][105620] Updated weights for policy 1, policy_version 1385908 (0.0005) [2023-12-27 01:28:08,277][105692] Updated weights for policy 0, policy_version 1383736 (0.0009) [2023-12-27 01:28:08,333][105692] Updated weights for policy 0, policy_version 1383746 (0.0010) [2023-12-27 01:28:08,401][105692] Updated weights for policy 0, policy_version 1383756 (0.0011) [2023-12-27 01:28:08,505][105620] Updated weights for policy 1, policy_version 1385918 (0.0009) [2023-12-27 01:28:08,565][105620] Updated weights for policy 1, policy_version 1385928 (0.0006) [2023-12-27 01:28:08,632][105620] Updated weights for policy 1, policy_version 1385938 (0.0005) [2023-12-27 01:28:09,078][105692] Updated weights for policy 0, policy_version 1383766 (0.0011) [2023-12-27 01:28:09,137][105692] Updated weights for policy 0, policy_version 1383776 (0.0011) [2023-12-27 01:28:09,186][105692] Updated weights for policy 0, policy_version 1383786 (0.0011) [2023-12-27 01:28:09,308][105620] Updated weights for policy 1, policy_version 1385948 (0.0007) [2023-12-27 01:28:09,369][105620] Updated weights for policy 1, policy_version 1385958 (0.0011) [2023-12-27 01:28:09,443][105620] Updated weights for policy 1, policy_version 1385968 (0.0007) [2023-12-27 01:28:09,937][105692] Updated weights for policy 0, policy_version 1383796 (0.0009) [2023-12-27 01:28:10,009][105692] Updated weights for policy 0, policy_version 1383806 (0.0009) [2023-12-27 01:28:10,070][105692] Updated weights for policy 0, policy_version 1383816 (0.0011) [2023-12-27 01:28:10,211][105620] Updated weights for policy 1, policy_version 1385978 (0.0009) [2023-12-27 01:28:10,280][105620] Updated weights for policy 1, policy_version 1385988 (0.0010) [2023-12-27 01:28:10,351][105620] Updated weights for policy 1, policy_version 1385998 (0.0005) [2023-12-27 01:28:10,408][105620] Updated weights for policy 1, policy_version 1386008 (0.0009) [2023-12-27 01:28:10,790][105692] Updated weights for policy 0, policy_version 1383826 (0.0011) [2023-12-27 01:28:10,851][105692] Updated weights for policy 0, policy_version 1383836 (0.0010) [2023-12-27 01:28:10,913][105692] Updated weights for policy 0, policy_version 1383846 (0.0010) [2023-12-27 01:28:10,976][105692] Updated weights for policy 0, policy_version 1383856 (0.0010) [2023-12-27 01:28:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 709181440. Throughput: 0: 9631.5, 1: 9993.9. Samples: 709190732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:28:11,062][104569] Avg episode reward: [(0, '7789.532'), (1, '8900.854')] [2023-12-27 01:28:11,114][105620] Updated weights for policy 1, policy_version 1386018 (0.0009) [2023-12-27 01:28:11,186][105620] Updated weights for policy 1, policy_version 1386028 (0.0009) [2023-12-27 01:28:11,254][105620] Updated weights for policy 1, policy_version 1386038 (0.0009) [2023-12-27 01:28:11,696][105692] Updated weights for policy 0, policy_version 1383866 (0.0009) [2023-12-27 01:28:11,762][105692] Updated weights for policy 0, policy_version 1383876 (0.0010) [2023-12-27 01:28:11,829][105692] Updated weights for policy 0, policy_version 1383886 (0.0011) [2023-12-27 01:28:12,038][105620] Updated weights for policy 1, policy_version 1386048 (0.0008) [2023-12-27 01:28:12,095][105620] Updated weights for policy 1, policy_version 1386058 (0.0010) [2023-12-27 01:28:12,149][105620] Updated weights for policy 1, policy_version 1386068 (0.0008) [2023-12-27 01:28:12,559][105692] Updated weights for policy 0, policy_version 1383896 (0.0008) [2023-12-27 01:28:12,622][105692] Updated weights for policy 0, policy_version 1383906 (0.0010) [2023-12-27 01:28:12,675][105692] Updated weights for policy 0, policy_version 1383916 (0.0011) [2023-12-27 01:28:12,908][105620] Updated weights for policy 1, policy_version 1386078 (0.0007) [2023-12-27 01:28:12,963][105620] Updated weights for policy 1, policy_version 1386088 (0.0007) [2023-12-27 01:28:13,012][105620] Updated weights for policy 1, policy_version 1386098 (0.0008) [2023-12-27 01:28:13,409][105692] Updated weights for policy 0, policy_version 1383927 (0.0010) [2023-12-27 01:28:13,457][105692] Updated weights for policy 0, policy_version 1383937 (0.0010) [2023-12-27 01:28:13,508][105692] Updated weights for policy 0, policy_version 1383947 (0.0010) [2023-12-27 01:28:13,771][105620] Updated weights for policy 1, policy_version 1386108 (0.0008) [2023-12-27 01:28:13,815][105620] Updated weights for policy 1, policy_version 1386118 (0.0008) [2023-12-27 01:28:13,859][105620] Updated weights for policy 1, policy_version 1386128 (0.0007) [2023-12-27 01:28:14,252][105692] Updated weights for policy 0, policy_version 1383957 (0.0009) [2023-12-27 01:28:14,297][105692] Updated weights for policy 0, policy_version 1383967 (0.0010) [2023-12-27 01:28:14,343][105692] Updated weights for policy 0, policy_version 1383977 (0.0010) [2023-12-27 01:28:14,640][105620] Updated weights for policy 1, policy_version 1386138 (0.0008) [2023-12-27 01:28:14,696][105620] Updated weights for policy 1, policy_version 1386148 (0.0008) [2023-12-27 01:28:14,750][105620] Updated weights for policy 1, policy_version 1386158 (0.0008) [2023-12-27 01:28:14,814][105620] Updated weights for policy 1, policy_version 1386168 (0.0008) [2023-12-27 01:28:15,093][105692] Updated weights for policy 0, policy_version 1383987 (0.0010) [2023-12-27 01:28:15,157][105692] Updated weights for policy 0, policy_version 1383997 (0.0011) [2023-12-27 01:28:15,218][105692] Updated weights for policy 0, policy_version 1384007 (0.0011) [2023-12-27 01:28:15,645][105620] Updated weights for policy 1, policy_version 1386178 (0.0008) [2023-12-27 01:28:15,705][105620] Updated weights for policy 1, policy_version 1386188 (0.0009) [2023-12-27 01:28:15,769][105620] Updated weights for policy 1, policy_version 1386198 (0.0008) [2023-12-27 01:28:15,906][105692] Updated weights for policy 0, policy_version 1384017 (0.0011) [2023-12-27 01:28:15,963][105692] Updated weights for policy 0, policy_version 1384027 (0.0010) [2023-12-27 01:28:16,015][105692] Updated weights for policy 0, policy_version 1384037 (0.0006) [2023-12-27 01:28:16,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 709271552. Throughput: 0: 9601.2, 1: 9972.8. Samples: 709246164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:28:16,062][104569] Avg episode reward: [(0, '6961.345'), (1, '8811.965')] [2023-12-27 01:28:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001386200_354910208.pth... [2023-12-27 01:28:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001385048_354615296.pth [2023-12-27 01:28:16,088][105692] Updated weights for policy 0, policy_version 1384047 (0.0005) [2023-12-27 01:28:16,095][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001384048_354369536.pth... [2023-12-27 01:28:16,100][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001382896_354074624.pth [2023-12-27 01:28:16,638][105692] Updated weights for policy 0, policy_version 1384057 (0.0006) [2023-12-27 01:28:16,638][105620] Updated weights for policy 1, policy_version 1386208 (0.0010) [2023-12-27 01:28:16,690][105620] Updated weights for policy 1, policy_version 1386218 (0.0007) [2023-12-27 01:28:16,696][105692] Updated weights for policy 0, policy_version 1384067 (0.0006) [2023-12-27 01:28:16,745][105620] Updated weights for policy 1, policy_version 1386228 (0.0008) [2023-12-27 01:28:16,760][105692] Updated weights for policy 0, policy_version 1384077 (0.0006) [2023-12-27 01:28:17,334][105692] Updated weights for policy 0, policy_version 1384087 (0.0005) [2023-12-27 01:28:17,388][105692] Updated weights for policy 0, policy_version 1384097 (0.0006) [2023-12-27 01:28:17,446][105692] Updated weights for policy 0, policy_version 1384107 (0.0005) [2023-12-27 01:28:17,636][105620] Updated weights for policy 1, policy_version 1386238 (0.0009) [2023-12-27 01:28:17,689][105620] Updated weights for policy 1, policy_version 1386248 (0.0009) [2023-12-27 01:28:17,750][105620] Updated weights for policy 1, policy_version 1386258 (0.0009) [2023-12-27 01:28:17,945][105692] Updated weights for policy 0, policy_version 1384117 (0.0007) [2023-12-27 01:28:18,008][105692] Updated weights for policy 0, policy_version 1384127 (0.0009) [2023-12-27 01:28:18,058][105692] Updated weights for policy 0, policy_version 1384137 (0.0009) [2023-12-27 01:28:18,587][105620] Updated weights for policy 1, policy_version 1386268 (0.0008) [2023-12-27 01:28:18,642][105620] Updated weights for policy 1, policy_version 1386278 (0.0008) [2023-12-27 01:28:18,690][105620] Updated weights for policy 1, policy_version 1386288 (0.0009) [2023-12-27 01:28:18,770][105692] Updated weights for policy 0, policy_version 1384147 (0.0009) [2023-12-27 01:28:18,828][105692] Updated weights for policy 0, policy_version 1384157 (0.0009) [2023-12-27 01:28:18,885][105692] Updated weights for policy 0, policy_version 1384167 (0.0010) [2023-12-27 01:28:19,361][105620] Updated weights for policy 1, policy_version 1386298 (0.0008) [2023-12-27 01:28:19,426][105620] Updated weights for policy 1, policy_version 1386308 (0.0007) [2023-12-27 01:28:19,491][105620] Updated weights for policy 1, policy_version 1386318 (0.0006) [2023-12-27 01:28:19,544][105620] Updated weights for policy 1, policy_version 1386328 (0.0005) [2023-12-27 01:28:19,697][105692] Updated weights for policy 0, policy_version 1384177 (0.0011) [2023-12-27 01:28:19,760][105692] Updated weights for policy 0, policy_version 1384187 (0.0010) [2023-12-27 01:28:19,815][105692] Updated weights for policy 0, policy_version 1384197 (0.0010) [2023-12-27 01:28:19,877][105692] Updated weights for policy 0, policy_version 1384207 (0.0011) [2023-12-27 01:28:20,190][105620] Updated weights for policy 1, policy_version 1386338 (0.0009) [2023-12-27 01:28:20,246][105620] Updated weights for policy 1, policy_version 1386348 (0.0008) [2023-12-27 01:28:20,306][105620] Updated weights for policy 1, policy_version 1386358 (0.0008) [2023-12-27 01:28:20,652][105692] Updated weights for policy 0, policy_version 1384217 (0.0008) [2023-12-27 01:28:20,711][105692] Updated weights for policy 0, policy_version 1384227 (0.0008) [2023-12-27 01:28:20,778][105692] Updated weights for policy 0, policy_version 1384237 (0.0008) [2023-12-27 01:28:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 709369856. Throughput: 0: 9749.4, 1: 9841.9. Samples: 709363524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:28:21,062][104569] Avg episode reward: [(0, '7607.836'), (1, '8632.271')] [2023-12-27 01:28:21,064][105620] Updated weights for policy 1, policy_version 1386368 (0.0008) [2023-12-27 01:28:21,132][105620] Updated weights for policy 1, policy_version 1386378 (0.0010) [2023-12-27 01:28:21,208][105620] Updated weights for policy 1, policy_version 1386388 (0.0009) [2023-12-27 01:28:21,573][105692] Updated weights for policy 0, policy_version 1384247 (0.0010) [2023-12-27 01:28:21,642][105692] Updated weights for policy 0, policy_version 1384257 (0.0010) [2023-12-27 01:28:21,706][105692] Updated weights for policy 0, policy_version 1384267 (0.0010) [2023-12-27 01:28:21,857][105620] Updated weights for policy 1, policy_version 1386398 (0.0006) [2023-12-27 01:28:21,922][105620] Updated weights for policy 1, policy_version 1386408 (0.0009) [2023-12-27 01:28:21,987][105620] Updated weights for policy 1, policy_version 1386418 (0.0011) [2023-12-27 01:28:22,380][105692] Updated weights for policy 0, policy_version 1384277 (0.0009) [2023-12-27 01:28:22,442][105692] Updated weights for policy 0, policy_version 1384287 (0.0006) [2023-12-27 01:28:22,455][105585] KL-divergence is very high: 153.3517 [2023-12-27 01:28:22,473][105585] KL-divergence is very high: 135.4067 [2023-12-27 01:28:22,505][105585] KL-divergence is very high: 251.3661 [2023-12-27 01:28:22,506][105692] Updated weights for policy 0, policy_version 1384297 (0.0008) [2023-12-27 01:28:22,523][105585] KL-divergence is very high: 161.1705 [2023-12-27 01:28:22,775][105620] Updated weights for policy 1, policy_version 1386428 (0.0010) [2023-12-27 01:28:22,824][105620] Updated weights for policy 1, policy_version 1386438 (0.0010) [2023-12-27 01:28:22,879][105620] Updated weights for policy 1, policy_version 1386448 (0.0007) [2023-12-27 01:28:23,146][105692] Updated weights for policy 0, policy_version 1384307 (0.0008) [2023-12-27 01:28:23,218][105692] Updated weights for policy 0, policy_version 1384317 (0.0006) [2023-12-27 01:28:23,281][105692] Updated weights for policy 0, policy_version 1384327 (0.0007) [2023-12-27 01:28:23,642][105620] Updated weights for policy 1, policy_version 1386458 (0.0008) [2023-12-27 01:28:23,692][105620] Updated weights for policy 1, policy_version 1386468 (0.0009) [2023-12-27 01:28:23,746][105620] Updated weights for policy 1, policy_version 1386478 (0.0009) [2023-12-27 01:28:23,796][105620] Updated weights for policy 1, policy_version 1386488 (0.0009) [2023-12-27 01:28:23,958][105692] Updated weights for policy 0, policy_version 1384337 (0.0009) [2023-12-27 01:28:24,017][105692] Updated weights for policy 0, policy_version 1384347 (0.0009) [2023-12-27 01:28:24,079][105692] Updated weights for policy 0, policy_version 1384357 (0.0009) [2023-12-27 01:28:24,127][105692] Updated weights for policy 0, policy_version 1384367 (0.0009) [2023-12-27 01:28:24,511][105620] Updated weights for policy 1, policy_version 1386498 (0.0005) [2023-12-27 01:28:24,569][105620] Updated weights for policy 1, policy_version 1386508 (0.0006) [2023-12-27 01:28:24,624][105620] Updated weights for policy 1, policy_version 1386518 (0.0006) [2023-12-27 01:28:24,899][105692] Updated weights for policy 0, policy_version 1384377 (0.0009) [2023-12-27 01:28:24,950][105692] Updated weights for policy 0, policy_version 1384387 (0.0009) [2023-12-27 01:28:24,998][105692] Updated weights for policy 0, policy_version 1384397 (0.0009) [2023-12-27 01:28:25,337][105620] Updated weights for policy 1, policy_version 1386528 (0.0008) [2023-12-27 01:28:25,396][105620] Updated weights for policy 1, policy_version 1386538 (0.0009) [2023-12-27 01:28:25,456][105620] Updated weights for policy 1, policy_version 1386548 (0.0008) [2023-12-27 01:28:25,744][105692] Updated weights for policy 0, policy_version 1384407 (0.0008) [2023-12-27 01:28:25,795][105692] Updated weights for policy 0, policy_version 1384417 (0.0009) [2023-12-27 01:28:25,843][105692] Updated weights for policy 0, policy_version 1384427 (0.0009) [2023-12-27 01:28:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 709468160. Throughput: 0: 9834.0, 1: 9734.0. Samples: 709477084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:28:26,063][104569] Avg episode reward: [(0, '7520.120'), (1, '8493.525')] [2023-12-27 01:28:26,249][105620] Updated weights for policy 1, policy_version 1386558 (0.0009) [2023-12-27 01:28:26,317][105620] Updated weights for policy 1, policy_version 1386568 (0.0009) [2023-12-27 01:28:26,365][105620] Updated weights for policy 1, policy_version 1386578 (0.0008) [2023-12-27 01:28:26,481][105692] Updated weights for policy 0, policy_version 1384437 (0.0010) [2023-12-27 01:28:26,535][105692] Updated weights for policy 0, policy_version 1384447 (0.0010) [2023-12-27 01:28:26,588][105692] Updated weights for policy 0, policy_version 1384457 (0.0005) [2023-12-27 01:28:27,157][105692] Updated weights for policy 0, policy_version 1384467 (0.0006) [2023-12-27 01:28:27,175][105620] Updated weights for policy 1, policy_version 1386588 (0.0008) [2023-12-27 01:28:27,218][105692] Updated weights for policy 0, policy_version 1384477 (0.0007) [2023-12-27 01:28:27,232][105620] Updated weights for policy 1, policy_version 1386598 (0.0007) [2023-12-27 01:28:27,273][105692] Updated weights for policy 0, policy_version 1384487 (0.0010) [2023-12-27 01:28:27,299][105620] Updated weights for policy 1, policy_version 1386608 (0.0005) [2023-12-27 01:28:27,996][105692] Updated weights for policy 0, policy_version 1384497 (0.0010) [2023-12-27 01:28:28,033][105620] Updated weights for policy 1, policy_version 1386618 (0.0007) [2023-12-27 01:28:28,046][105692] Updated weights for policy 0, policy_version 1384507 (0.0010) [2023-12-27 01:28:28,091][105620] Updated weights for policy 1, policy_version 1386628 (0.0006) [2023-12-27 01:28:28,104][105692] Updated weights for policy 0, policy_version 1384517 (0.0010) [2023-12-27 01:28:28,152][105620] Updated weights for policy 1, policy_version 1386638 (0.0006) [2023-12-27 01:28:28,162][105692] Updated weights for policy 0, policy_version 1384527 (0.0010) [2023-12-27 01:28:28,209][105620] Updated weights for policy 1, policy_version 1386648 (0.0007) [2023-12-27 01:28:28,926][105692] Updated weights for policy 0, policy_version 1384537 (0.0010) [2023-12-27 01:28:28,952][105620] Updated weights for policy 1, policy_version 1386658 (0.0006) [2023-12-27 01:28:28,978][105692] Updated weights for policy 0, policy_version 1384547 (0.0010) [2023-12-27 01:28:29,007][105620] Updated weights for policy 1, policy_version 1386668 (0.0005) [2023-12-27 01:28:29,029][105692] Updated weights for policy 0, policy_version 1384557 (0.0010) [2023-12-27 01:28:29,062][105620] Updated weights for policy 1, policy_version 1386678 (0.0005) [2023-12-27 01:28:29,755][105692] Updated weights for policy 0, policy_version 1384567 (0.0009) [2023-12-27 01:28:29,793][105620] Updated weights for policy 1, policy_version 1386688 (0.0006) [2023-12-27 01:28:29,817][105692] Updated weights for policy 0, policy_version 1384577 (0.0006) [2023-12-27 01:28:29,856][105620] Updated weights for policy 1, policy_version 1386698 (0.0007) [2023-12-27 01:28:29,878][105692] Updated weights for policy 0, policy_version 1384587 (0.0006) [2023-12-27 01:28:29,916][105620] Updated weights for policy 1, policy_version 1386708 (0.0008) [2023-12-27 01:28:30,491][105692] Updated weights for policy 0, policy_version 1384597 (0.0006) [2023-12-27 01:28:30,548][105692] Updated weights for policy 0, policy_version 1384607 (0.0007) [2023-12-27 01:28:30,608][105692] Updated weights for policy 0, policy_version 1384617 (0.0005) [2023-12-27 01:28:30,759][105620] Updated weights for policy 1, policy_version 1386718 (0.0009) [2023-12-27 01:28:30,822][105620] Updated weights for policy 1, policy_version 1386728 (0.0009) [2023-12-27 01:28:30,887][105620] Updated weights for policy 1, policy_version 1386738 (0.0007) [2023-12-27 01:28:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 709566464. Throughput: 0: 9932.8, 1: 9701.2. Samples: 709535408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:28:31,063][104569] Avg episode reward: [(0, '7611.122'), (1, '8942.894')] [2023-12-27 01:28:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001384624_354516992.pth... [2023-12-27 01:28:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001386744_355049472.pth... [2023-12-27 01:28:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001385624_354762752.pth [2023-12-27 01:28:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001383440_354213888.pth [2023-12-27 01:28:31,162][105692] Updated weights for policy 0, policy_version 1384627 (0.0006) [2023-12-27 01:28:31,216][105692] Updated weights for policy 0, policy_version 1384637 (0.0007) [2023-12-27 01:28:31,277][105692] Updated weights for policy 0, policy_version 1384647 (0.0010) [2023-12-27 01:28:31,612][105620] Updated weights for policy 1, policy_version 1386748 (0.0006) [2023-12-27 01:28:31,678][105620] Updated weights for policy 1, policy_version 1386758 (0.0008) [2023-12-27 01:28:31,744][105620] Updated weights for policy 1, policy_version 1386768 (0.0010) [2023-12-27 01:28:32,054][105692] Updated weights for policy 0, policy_version 1384657 (0.0007) [2023-12-27 01:28:32,107][105692] Updated weights for policy 0, policy_version 1384667 (0.0009) [2023-12-27 01:28:32,161][105692] Updated weights for policy 0, policy_version 1384678 (0.0010) [2023-12-27 01:28:32,217][105692] Updated weights for policy 0, policy_version 1384688 (0.0007) [2023-12-27 01:28:32,376][105620] Updated weights for policy 1, policy_version 1386778 (0.0006) [2023-12-27 01:28:32,442][105620] Updated weights for policy 1, policy_version 1386788 (0.0005) [2023-12-27 01:28:32,505][105620] Updated weights for policy 1, policy_version 1386798 (0.0005) [2023-12-27 01:28:32,573][105620] Updated weights for policy 1, policy_version 1386808 (0.0005) [2023-12-27 01:28:33,054][105692] Updated weights for policy 0, policy_version 1384698 (0.0009) [2023-12-27 01:28:33,089][105620] Updated weights for policy 1, policy_version 1386818 (0.0008) [2023-12-27 01:28:33,107][105692] Updated weights for policy 0, policy_version 1384708 (0.0008) [2023-12-27 01:28:33,146][105620] Updated weights for policy 1, policy_version 1386828 (0.0007) [2023-12-27 01:28:33,156][105692] Updated weights for policy 0, policy_version 1384718 (0.0008) [2023-12-27 01:28:33,203][105620] Updated weights for policy 1, policy_version 1386838 (0.0008) [2023-12-27 01:28:33,910][105692] Updated weights for policy 0, policy_version 1384728 (0.0007) [2023-12-27 01:28:33,931][105620] Updated weights for policy 1, policy_version 1386848 (0.0007) [2023-12-27 01:28:33,964][105692] Updated weights for policy 0, policy_version 1384738 (0.0007) [2023-12-27 01:28:33,982][105620] Updated weights for policy 1, policy_version 1386858 (0.0007) [2023-12-27 01:28:34,023][105692] Updated weights for policy 0, policy_version 1384748 (0.0007) [2023-12-27 01:28:34,033][105620] Updated weights for policy 1, policy_version 1386868 (0.0006) [2023-12-27 01:28:34,664][105692] Updated weights for policy 0, policy_version 1384758 (0.0007) [2023-12-27 01:28:34,712][105620] Updated weights for policy 1, policy_version 1386878 (0.0006) [2023-12-27 01:28:34,727][105692] Updated weights for policy 0, policy_version 1384768 (0.0007) [2023-12-27 01:28:34,769][105620] Updated weights for policy 1, policy_version 1386888 (0.0007) [2023-12-27 01:28:34,781][105692] Updated weights for policy 0, policy_version 1384778 (0.0008) [2023-12-27 01:28:34,830][105620] Updated weights for policy 1, policy_version 1386899 (0.0009) [2023-12-27 01:28:35,510][105692] Updated weights for policy 0, policy_version 1384788 (0.0007) [2023-12-27 01:28:35,560][105692] Updated weights for policy 0, policy_version 1384798 (0.0007) [2023-12-27 01:28:35,566][105620] Updated weights for policy 1, policy_version 1386909 (0.0009) [2023-12-27 01:28:35,605][105692] Updated weights for policy 0, policy_version 1384808 (0.0005) [2023-12-27 01:28:35,623][105620] Updated weights for policy 1, policy_version 1386919 (0.0008) [2023-12-27 01:28:35,677][105620] Updated weights for policy 1, policy_version 1386929 (0.0009) [2023-12-27 01:28:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.7, 300 sec: 19494.2). Total num frames: 709664768. Throughput: 0: 9919.8, 1: 9704.0. Samples: 709653736. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:28:36,063][104569] Avg episode reward: [(0, '7889.092'), (1, '9172.925')] [2023-12-27 01:28:36,292][105692] Updated weights for policy 0, policy_version 1384818 (0.0006) [2023-12-27 01:28:36,351][105692] Updated weights for policy 0, policy_version 1384828 (0.0011) [2023-12-27 01:28:36,417][105692] Updated weights for policy 0, policy_version 1384838 (0.0011) [2023-12-27 01:28:36,437][105620] Updated weights for policy 1, policy_version 1386939 (0.0009) [2023-12-27 01:28:36,477][105692] Updated weights for policy 0, policy_version 1384848 (0.0011) [2023-12-27 01:28:36,497][105620] Updated weights for policy 1, policy_version 1386949 (0.0006) [2023-12-27 01:28:36,560][105620] Updated weights for policy 1, policy_version 1386959 (0.0008) [2023-12-27 01:28:37,182][105692] Updated weights for policy 0, policy_version 1384858 (0.0008) [2023-12-27 01:28:37,246][105692] Updated weights for policy 0, policy_version 1384868 (0.0010) [2023-12-27 01:28:37,321][105692] Updated weights for policy 0, policy_version 1384878 (0.0009) [2023-12-27 01:28:37,324][105620] Updated weights for policy 1, policy_version 1386969 (0.0008) [2023-12-27 01:28:37,376][105620] Updated weights for policy 1, policy_version 1386979 (0.0008) [2023-12-27 01:28:37,439][105620] Updated weights for policy 1, policy_version 1386989 (0.0008) [2023-12-27 01:28:37,499][105620] Updated weights for policy 1, policy_version 1386999 (0.0008) [2023-12-27 01:28:38,040][105692] Updated weights for policy 0, policy_version 1384888 (0.0011) [2023-12-27 01:28:38,091][105692] Updated weights for policy 0, policy_version 1384898 (0.0010) [2023-12-27 01:28:38,147][105692] Updated weights for policy 0, policy_version 1384908 (0.0011) [2023-12-27 01:28:38,281][105620] Updated weights for policy 1, policy_version 1387009 (0.0008) [2023-12-27 01:28:38,355][105620] Updated weights for policy 1, policy_version 1387019 (0.0009) [2023-12-27 01:28:38,411][105620] Updated weights for policy 1, policy_version 1387029 (0.0008) [2023-12-27 01:28:38,885][105692] Updated weights for policy 0, policy_version 1384918 (0.0010) [2023-12-27 01:28:38,936][105692] Updated weights for policy 0, policy_version 1384928 (0.0010) [2023-12-27 01:28:38,991][105692] Updated weights for policy 0, policy_version 1384938 (0.0010) [2023-12-27 01:28:39,165][105620] Updated weights for policy 1, policy_version 1387039 (0.0009) [2023-12-27 01:28:39,225][105620] Updated weights for policy 1, policy_version 1387049 (0.0009) [2023-12-27 01:28:39,281][105620] Updated weights for policy 1, policy_version 1387059 (0.0008) [2023-12-27 01:28:39,683][105692] Updated weights for policy 0, policy_version 1384948 (0.0010) [2023-12-27 01:28:39,741][105692] Updated weights for policy 0, policy_version 1384958 (0.0010) [2023-12-27 01:28:39,797][105692] Updated weights for policy 0, policy_version 1384968 (0.0009) [2023-12-27 01:28:40,087][105620] Updated weights for policy 1, policy_version 1387069 (0.0008) [2023-12-27 01:28:40,158][105620] Updated weights for policy 1, policy_version 1387079 (0.0006) [2023-12-27 01:28:40,222][105620] Updated weights for policy 1, policy_version 1387089 (0.0008) [2023-12-27 01:28:40,620][105692] Updated weights for policy 0, policy_version 1384978 (0.0009) [2023-12-27 01:28:40,679][105692] Updated weights for policy 0, policy_version 1384988 (0.0009) [2023-12-27 01:28:40,742][105692] Updated weights for policy 0, policy_version 1384998 (0.0009) [2023-12-27 01:28:40,800][105692] Updated weights for policy 0, policy_version 1385008 (0.0008) [2023-12-27 01:28:40,940][105620] Updated weights for policy 1, policy_version 1387099 (0.0009) [2023-12-27 01:28:41,001][105620] Updated weights for policy 1, policy_version 1387109 (0.0009) [2023-12-27 01:28:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 709754880. Throughput: 0: 9939.9, 1: 9575.3. Samples: 709767324. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:28:41,062][104569] Avg episode reward: [(0, '8621.787'), (1, '9175.168')] [2023-12-27 01:28:41,074][105620] Updated weights for policy 1, policy_version 1387119 (0.0008) [2023-12-27 01:28:41,551][105692] Updated weights for policy 0, policy_version 1385018 (0.0006) [2023-12-27 01:28:41,609][105692] Updated weights for policy 0, policy_version 1385028 (0.0006) [2023-12-27 01:28:41,668][105692] Updated weights for policy 0, policy_version 1385038 (0.0007) [2023-12-27 01:28:41,852][105620] Updated weights for policy 1, policy_version 1387129 (0.0008) [2023-12-27 01:28:41,909][105620] Updated weights for policy 1, policy_version 1387139 (0.0005) [2023-12-27 01:28:41,966][105620] Updated weights for policy 1, policy_version 1387149 (0.0007) [2023-12-27 01:28:42,027][105620] Updated weights for policy 1, policy_version 1387159 (0.0008) [2023-12-27 01:28:42,420][105692] Updated weights for policy 0, policy_version 1385048 (0.0006) [2023-12-27 01:28:42,480][105692] Updated weights for policy 0, policy_version 1385058 (0.0007) [2023-12-27 01:28:42,538][105692] Updated weights for policy 0, policy_version 1385068 (0.0008) [2023-12-27 01:28:42,669][105620] Updated weights for policy 1, policy_version 1387169 (0.0008) [2023-12-27 01:28:42,731][105620] Updated weights for policy 1, policy_version 1387179 (0.0005) [2023-12-27 01:28:42,800][105620] Updated weights for policy 1, policy_version 1387189 (0.0008) [2023-12-27 01:28:43,155][105692] Updated weights for policy 0, policy_version 1385078 (0.0009) [2023-12-27 01:28:43,204][105692] Updated weights for policy 0, policy_version 1385088 (0.0007) [2023-12-27 01:28:43,258][105692] Updated weights for policy 0, policy_version 1385098 (0.0009) [2023-12-27 01:28:43,528][105620] Updated weights for policy 1, policy_version 1387199 (0.0009) [2023-12-27 01:28:43,588][105620] Updated weights for policy 1, policy_version 1387209 (0.0008) [2023-12-27 01:28:43,638][105620] Updated weights for policy 1, policy_version 1387219 (0.0007) [2023-12-27 01:28:44,037][105692] Updated weights for policy 0, policy_version 1385108 (0.0009) [2023-12-27 01:28:44,093][105692] Updated weights for policy 0, policy_version 1385118 (0.0006) [2023-12-27 01:28:44,151][105692] Updated weights for policy 0, policy_version 1385128 (0.0005) [2023-12-27 01:28:44,407][105620] Updated weights for policy 1, policy_version 1387229 (0.0007) [2023-12-27 01:28:44,463][105620] Updated weights for policy 1, policy_version 1387239 (0.0005) [2023-12-27 01:28:44,511][105620] Updated weights for policy 1, policy_version 1387249 (0.0006) [2023-12-27 01:28:44,752][105692] Updated weights for policy 0, policy_version 1385138 (0.0006) [2023-12-27 01:28:44,822][105692] Updated weights for policy 0, policy_version 1385148 (0.0007) [2023-12-27 01:28:44,883][105692] Updated weights for policy 0, policy_version 1385158 (0.0008) [2023-12-27 01:28:44,939][105692] Updated weights for policy 0, policy_version 1385168 (0.0009) [2023-12-27 01:28:45,315][105620] Updated weights for policy 1, policy_version 1387260 (0.0010) [2023-12-27 01:28:45,372][105620] Updated weights for policy 1, policy_version 1387270 (0.0009) [2023-12-27 01:28:45,440][105620] Updated weights for policy 1, policy_version 1387280 (0.0009) [2023-12-27 01:28:45,553][105692] Updated weights for policy 0, policy_version 1385178 (0.0009) [2023-12-27 01:28:45,618][105692] Updated weights for policy 0, policy_version 1385188 (0.0006) [2023-12-27 01:28:45,678][105692] Updated weights for policy 0, policy_version 1385198 (0.0005) [2023-12-27 01:28:46,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 709853184. Throughput: 0: 9951.1, 1: 9528.7. Samples: 709824492. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:28:46,062][104569] Avg episode reward: [(0, '8713.574'), (1, '9085.250')] [2023-12-27 01:28:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001387288_355188736.pth... [2023-12-27 01:28:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001385200_354664448.pth... [2023-12-27 01:28:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001386200_354910208.pth [2023-12-27 01:28:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001384048_354369536.pth [2023-12-27 01:28:46,244][105692] Updated weights for policy 0, policy_version 1385208 (0.0006) [2023-12-27 01:28:46,287][105692] Updated weights for policy 0, policy_version 1385218 (0.0009) [2023-12-27 01:28:46,288][105620] Updated weights for policy 1, policy_version 1387290 (0.0008) [2023-12-27 01:28:46,346][105692] Updated weights for policy 0, policy_version 1385228 (0.0005) [2023-12-27 01:28:46,352][105620] Updated weights for policy 1, policy_version 1387300 (0.0008) [2023-12-27 01:28:46,415][105620] Updated weights for policy 1, policy_version 1387310 (0.0008) [2023-12-27 01:28:46,468][105620] Updated weights for policy 1, policy_version 1387320 (0.0009) [2023-12-27 01:28:46,974][105692] Updated weights for policy 0, policy_version 1385238 (0.0005) [2023-12-27 01:28:47,040][105692] Updated weights for policy 0, policy_version 1385248 (0.0007) [2023-12-27 01:28:47,079][105620] Updated weights for policy 1, policy_version 1387330 (0.0007) [2023-12-27 01:28:47,103][105692] Updated weights for policy 0, policy_version 1385258 (0.0007) [2023-12-27 01:28:47,136][105620] Updated weights for policy 1, policy_version 1387340 (0.0008) [2023-12-27 01:28:47,201][105620] Updated weights for policy 1, policy_version 1387350 (0.0009) [2023-12-27 01:28:47,707][105692] Updated weights for policy 0, policy_version 1385268 (0.0007) [2023-12-27 01:28:47,768][105692] Updated weights for policy 0, policy_version 1385278 (0.0007) [2023-12-27 01:28:47,770][105620] Updated weights for policy 1, policy_version 1387360 (0.0006) [2023-12-27 01:28:47,824][105692] Updated weights for policy 0, policy_version 1385288 (0.0008) [2023-12-27 01:28:47,827][105620] Updated weights for policy 1, policy_version 1387370 (0.0007) [2023-12-27 01:28:47,885][105620] Updated weights for policy 1, policy_version 1387380 (0.0009) [2023-12-27 01:28:48,466][105692] Updated weights for policy 0, policy_version 1385298 (0.0006) [2023-12-27 01:28:48,528][105692] Updated weights for policy 0, policy_version 1385308 (0.0006) [2023-12-27 01:28:48,593][105692] Updated weights for policy 0, policy_version 1385318 (0.0007) [2023-12-27 01:28:48,604][105620] Updated weights for policy 1, policy_version 1387390 (0.0008) [2023-12-27 01:28:48,646][105692] Updated weights for policy 0, policy_version 1385328 (0.0008) [2023-12-27 01:28:48,665][105620] Updated weights for policy 1, policy_version 1387400 (0.0007) [2023-12-27 01:28:48,729][105620] Updated weights for policy 1, policy_version 1387410 (0.0009) [2023-12-27 01:28:49,225][105692] Updated weights for policy 0, policy_version 1385338 (0.0010) [2023-12-27 01:28:49,283][105692] Updated weights for policy 0, policy_version 1385348 (0.0009) [2023-12-27 01:28:49,344][105692] Updated weights for policy 0, policy_version 1385358 (0.0011) [2023-12-27 01:28:49,550][105620] Updated weights for policy 1, policy_version 1387420 (0.0010) [2023-12-27 01:28:49,604][105620] Updated weights for policy 1, policy_version 1387430 (0.0009) [2023-12-27 01:28:49,660][105620] Updated weights for policy 1, policy_version 1387440 (0.0009) [2023-12-27 01:28:50,044][105692] Updated weights for policy 0, policy_version 1385368 (0.0007) [2023-12-27 01:28:50,106][105692] Updated weights for policy 0, policy_version 1385378 (0.0005) [2023-12-27 01:28:50,161][105692] Updated weights for policy 0, policy_version 1385388 (0.0005) [2023-12-27 01:28:50,519][105620] Updated weights for policy 1, policy_version 1387450 (0.0009) [2023-12-27 01:28:50,575][105620] Updated weights for policy 1, policy_version 1387460 (0.0008) [2023-12-27 01:28:50,638][105620] Updated weights for policy 1, policy_version 1387470 (0.0010) [2023-12-27 01:28:50,696][105620] Updated weights for policy 1, policy_version 1387480 (0.0009) [2023-12-27 01:28:50,757][105692] Updated weights for policy 0, policy_version 1385398 (0.0008) [2023-12-27 01:28:50,812][105692] Updated weights for policy 0, policy_version 1385408 (0.0009) [2023-12-27 01:28:50,872][105692] Updated weights for policy 0, policy_version 1385418 (0.0009) [2023-12-27 01:28:51,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 709959680. Throughput: 0: 10080.7, 1: 9486.1. Samples: 709946332. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:28:51,063][104569] Avg episode reward: [(0, '8530.528'), (1, '9174.819')] [2023-12-27 01:28:51,469][105620] Updated weights for policy 1, policy_version 1387490 (0.0009) [2023-12-27 01:28:51,527][105620] Updated weights for policy 1, policy_version 1387500 (0.0009) [2023-12-27 01:28:51,589][105620] Updated weights for policy 1, policy_version 1387510 (0.0009) [2023-12-27 01:28:51,665][105692] Updated weights for policy 0, policy_version 1385428 (0.0008) [2023-12-27 01:28:51,717][105692] Updated weights for policy 0, policy_version 1385438 (0.0009) [2023-12-27 01:28:51,786][105692] Updated weights for policy 0, policy_version 1385448 (0.0008) [2023-12-27 01:28:52,371][105620] Updated weights for policy 1, policy_version 1387520 (0.0009) [2023-12-27 01:28:52,437][105620] Updated weights for policy 1, policy_version 1387530 (0.0009) [2023-12-27 01:28:52,498][105620] Updated weights for policy 1, policy_version 1387540 (0.0008) [2023-12-27 01:28:52,564][105692] Updated weights for policy 0, policy_version 1385458 (0.0008) [2023-12-27 01:28:52,628][105692] Updated weights for policy 0, policy_version 1385468 (0.0010) [2023-12-27 01:28:52,688][105692] Updated weights for policy 0, policy_version 1385478 (0.0007) [2023-12-27 01:28:52,758][105692] Updated weights for policy 0, policy_version 1385488 (0.0007) [2023-12-27 01:28:53,267][105620] Updated weights for policy 1, policy_version 1387550 (0.0009) [2023-12-27 01:28:53,332][105620] Updated weights for policy 1, policy_version 1387560 (0.0010) [2023-12-27 01:28:53,388][105620] Updated weights for policy 1, policy_version 1387570 (0.0009) [2023-12-27 01:28:53,443][105692] Updated weights for policy 0, policy_version 1385498 (0.0005) [2023-12-27 01:28:53,490][105692] Updated weights for policy 0, policy_version 1385508 (0.0006) [2023-12-27 01:28:53,537][105692] Updated weights for policy 0, policy_version 1385518 (0.0008) [2023-12-27 01:28:54,193][105620] Updated weights for policy 1, policy_version 1387580 (0.0009) [2023-12-27 01:28:54,211][105692] Updated weights for policy 0, policy_version 1385528 (0.0009) [2023-12-27 01:28:54,253][105620] Updated weights for policy 1, policy_version 1387590 (0.0007) [2023-12-27 01:28:54,267][105692] Updated weights for policy 0, policy_version 1385538 (0.0006) [2023-12-27 01:28:54,314][105620] Updated weights for policy 1, policy_version 1387600 (0.0007) [2023-12-27 01:28:54,324][105692] Updated weights for policy 0, policy_version 1385548 (0.0007) [2023-12-27 01:28:55,056][105620] Updated weights for policy 1, policy_version 1387610 (0.0007) [2023-12-27 01:28:55,059][105692] Updated weights for policy 0, policy_version 1385558 (0.0007) [2023-12-27 01:28:55,109][105692] Updated weights for policy 0, policy_version 1385568 (0.0007) [2023-12-27 01:28:55,118][105620] Updated weights for policy 1, policy_version 1387620 (0.0009) [2023-12-27 01:28:55,167][105692] Updated weights for policy 0, policy_version 1385578 (0.0005) [2023-12-27 01:28:55,174][105620] Updated weights for policy 1, policy_version 1387630 (0.0009) [2023-12-27 01:28:55,229][105620] Updated weights for policy 1, policy_version 1387640 (0.0010) [2023-12-27 01:28:55,725][105692] Updated weights for policy 0, policy_version 1385588 (0.0005) [2023-12-27 01:28:55,784][105692] Updated weights for policy 0, policy_version 1385598 (0.0005) [2023-12-27 01:28:55,838][105692] Updated weights for policy 0, policy_version 1385608 (0.0005) [2023-12-27 01:28:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 710049792. Throughput: 0: 10013.7, 1: 9297.4. Samples: 710059732. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:28:56,063][104569] Avg episode reward: [(0, '8256.844'), (1, '9081.185')] [2023-12-27 01:28:56,103][105620] Updated weights for policy 1, policy_version 1387650 (0.0009) [2023-12-27 01:28:56,155][105620] Updated weights for policy 1, policy_version 1387660 (0.0008) [2023-12-27 01:28:56,204][105620] Updated weights for policy 1, policy_version 1387670 (0.0008) [2023-12-27 01:28:56,411][105692] Updated weights for policy 0, policy_version 1385618 (0.0006) [2023-12-27 01:28:56,463][105692] Updated weights for policy 0, policy_version 1385628 (0.0009) [2023-12-27 01:28:56,525][105692] Updated weights for policy 0, policy_version 1385638 (0.0010) [2023-12-27 01:28:56,579][105692] Updated weights for policy 0, policy_version 1385648 (0.0010) [2023-12-27 01:28:57,013][105620] Updated weights for policy 1, policy_version 1387680 (0.0008) [2023-12-27 01:28:57,065][105620] Updated weights for policy 1, policy_version 1387690 (0.0008) [2023-12-27 01:28:57,113][105620] Updated weights for policy 1, policy_version 1387700 (0.0008) [2023-12-27 01:28:57,303][105692] Updated weights for policy 0, policy_version 1385658 (0.0009) [2023-12-27 01:28:57,355][105692] Updated weights for policy 0, policy_version 1385668 (0.0010) [2023-12-27 01:28:57,406][105692] Updated weights for policy 0, policy_version 1385678 (0.0010) [2023-12-27 01:28:57,868][105620] Updated weights for policy 1, policy_version 1387710 (0.0008) [2023-12-27 01:28:57,919][105620] Updated weights for policy 1, policy_version 1387720 (0.0009) [2023-12-27 01:28:57,970][105620] Updated weights for policy 1, policy_version 1387730 (0.0008) [2023-12-27 01:28:58,126][105692] Updated weights for policy 0, policy_version 1385688 (0.0010) [2023-12-27 01:28:58,188][105692] Updated weights for policy 0, policy_version 1385698 (0.0010) [2023-12-27 01:28:58,246][105692] Updated weights for policy 0, policy_version 1385708 (0.0011) [2023-12-27 01:28:58,808][105620] Updated weights for policy 1, policy_version 1387740 (0.0008) [2023-12-27 01:28:58,877][105620] Updated weights for policy 1, policy_version 1387750 (0.0007) [2023-12-27 01:28:58,945][105620] Updated weights for policy 1, policy_version 1387760 (0.0007) [2023-12-27 01:28:59,112][105692] Updated weights for policy 0, policy_version 1385718 (0.0009) [2023-12-27 01:28:59,179][105692] Updated weights for policy 0, policy_version 1385728 (0.0008) [2023-12-27 01:28:59,253][105692] Updated weights for policy 0, policy_version 1385738 (0.0008) [2023-12-27 01:28:59,645][105620] Updated weights for policy 1, policy_version 1387770 (0.0006) [2023-12-27 01:28:59,709][105620] Updated weights for policy 1, policy_version 1387780 (0.0006) [2023-12-27 01:28:59,785][105620] Updated weights for policy 1, policy_version 1387790 (0.0009) [2023-12-27 01:28:59,842][105620] Updated weights for policy 1, policy_version 1387800 (0.0009) [2023-12-27 01:28:59,924][105692] Updated weights for policy 0, policy_version 1385748 (0.0008) [2023-12-27 01:28:59,982][105692] Updated weights for policy 0, policy_version 1385758 (0.0009) [2023-12-27 01:29:00,038][105692] Updated weights for policy 0, policy_version 1385768 (0.0009) [2023-12-27 01:29:00,595][105620] Updated weights for policy 1, policy_version 1387810 (0.0009) [2023-12-27 01:29:00,654][105620] Updated weights for policy 1, policy_version 1387820 (0.0009) [2023-12-27 01:29:00,689][105692] Updated weights for policy 0, policy_version 1385778 (0.0007) [2023-12-27 01:29:00,701][105620] Updated weights for policy 1, policy_version 1387830 (0.0009) [2023-12-27 01:29:00,755][105692] Updated weights for policy 0, policy_version 1385788 (0.0007) [2023-12-27 01:29:00,808][105585] KL-divergence is very high: 100.4149 [2023-12-27 01:29:00,818][105692] Updated weights for policy 0, policy_version 1385798 (0.0009) [2023-12-27 01:29:00,856][105585] KL-divergence is very high: 107.2960 [2023-12-27 01:29:00,879][105692] Updated weights for policy 0, policy_version 1385808 (0.0010) [2023-12-27 01:29:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 710148096. Throughput: 0: 10064.1, 1: 9280.1. Samples: 710116656. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:29:01,063][104569] Avg episode reward: [(0, '8164.444'), (1, '8989.270')] [2023-12-27 01:29:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001385808_354820096.pth... [2023-12-27 01:29:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001387832_355328000.pth... [2023-12-27 01:29:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001384624_354516992.pth [2023-12-27 01:29:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001386744_355049472.pth [2023-12-27 01:29:01,485][105620] Updated weights for policy 1, policy_version 1387840 (0.0008) [2023-12-27 01:29:01,539][105620] Updated weights for policy 1, policy_version 1387850 (0.0008) [2023-12-27 01:29:01,600][105620] Updated weights for policy 1, policy_version 1387860 (0.0007) [2023-12-27 01:29:01,601][105692] Updated weights for policy 0, policy_version 1385818 (0.0010) [2023-12-27 01:29:01,664][105692] Updated weights for policy 0, policy_version 1385828 (0.0010) [2023-12-27 01:29:01,730][105692] Updated weights for policy 0, policy_version 1385838 (0.0011) [2023-12-27 01:29:02,376][105620] Updated weights for policy 1, policy_version 1387870 (0.0007) [2023-12-27 01:29:02,443][105620] Updated weights for policy 1, policy_version 1387880 (0.0009) [2023-12-27 01:29:02,469][105692] Updated weights for policy 0, policy_version 1385848 (0.0007) [2023-12-27 01:29:02,506][105620] Updated weights for policy 1, policy_version 1387890 (0.0008) [2023-12-27 01:29:02,523][105692] Updated weights for policy 0, policy_version 1385858 (0.0006) [2023-12-27 01:29:02,578][105692] Updated weights for policy 0, policy_version 1385868 (0.0006) [2023-12-27 01:29:03,222][105620] Updated weights for policy 1, policy_version 1387900 (0.0009) [2023-12-27 01:29:03,247][105692] Updated weights for policy 0, policy_version 1385878 (0.0009) [2023-12-27 01:29:03,281][105620] Updated weights for policy 1, policy_version 1387910 (0.0008) [2023-12-27 01:29:03,303][105692] Updated weights for policy 0, policy_version 1385888 (0.0006) [2023-12-27 01:29:03,339][105620] Updated weights for policy 1, policy_version 1387920 (0.0007) [2023-12-27 01:29:03,361][105692] Updated weights for policy 0, policy_version 1385898 (0.0006) [2023-12-27 01:29:04,043][105692] Updated weights for policy 0, policy_version 1385908 (0.0008) [2023-12-27 01:29:04,090][105692] Updated weights for policy 0, policy_version 1385918 (0.0009) [2023-12-27 01:29:04,105][105620] Updated weights for policy 1, policy_version 1387930 (0.0007) [2023-12-27 01:29:04,152][105692] Updated weights for policy 0, policy_version 1385928 (0.0007) [2023-12-27 01:29:04,159][105620] Updated weights for policy 1, policy_version 1387940 (0.0006) [2023-12-27 01:29:04,210][105620] Updated weights for policy 1, policy_version 1387950 (0.0007) [2023-12-27 01:29:04,267][105620] Updated weights for policy 1, policy_version 1387960 (0.0009) [2023-12-27 01:29:04,759][105692] Updated weights for policy 0, policy_version 1385938 (0.0006) [2023-12-27 01:29:04,811][105692] Updated weights for policy 0, policy_version 1385948 (0.0006) [2023-12-27 01:29:04,862][105692] Updated weights for policy 0, policy_version 1385958 (0.0010) [2023-12-27 01:29:04,926][105692] Updated weights for policy 0, policy_version 1385968 (0.0010) [2023-12-27 01:29:05,115][105620] Updated weights for policy 1, policy_version 1387970 (0.0007) [2023-12-27 01:29:05,168][105620] Updated weights for policy 1, policy_version 1387980 (0.0008) [2023-12-27 01:29:05,212][105620] Updated weights for policy 1, policy_version 1387990 (0.0008) [2023-12-27 01:29:05,630][105692] Updated weights for policy 0, policy_version 1385978 (0.0010) [2023-12-27 01:29:05,692][105692] Updated weights for policy 0, policy_version 1385988 (0.0009) [2023-12-27 01:29:05,746][105692] Updated weights for policy 0, policy_version 1385998 (0.0006) [2023-12-27 01:29:05,980][105620] Updated weights for policy 1, policy_version 1388000 (0.0006) [2023-12-27 01:29:06,037][105620] Updated weights for policy 1, policy_version 1388010 (0.0008) [2023-12-27 01:29:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 710238208. Throughput: 0: 9973.6, 1: 9285.0. Samples: 710230160. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:29:06,062][104569] Avg episode reward: [(0, '8350.896'), (1, '8898.953')] [2023-12-27 01:29:06,094][105620] Updated weights for policy 1, policy_version 1388020 (0.0006) [2023-12-27 01:29:06,394][105692] Updated weights for policy 0, policy_version 1386008 (0.0010) [2023-12-27 01:29:06,456][105692] Updated weights for policy 0, policy_version 1386018 (0.0010) [2023-12-27 01:29:06,529][105692] Updated weights for policy 0, policy_version 1386028 (0.0008) [2023-12-27 01:29:06,651][105620] Updated weights for policy 1, policy_version 1388030 (0.0009) [2023-12-27 01:29:06,717][105620] Updated weights for policy 1, policy_version 1388040 (0.0008) [2023-12-27 01:29:06,778][105620] Updated weights for policy 1, policy_version 1388050 (0.0010) [2023-12-27 01:29:07,211][105692] Updated weights for policy 0, policy_version 1386038 (0.0011) [2023-12-27 01:29:07,270][105692] Updated weights for policy 0, policy_version 1386048 (0.0011) [2023-12-27 01:29:07,325][105585] KL-divergence is very high: 143.4655 [2023-12-27 01:29:07,338][105692] Updated weights for policy 0, policy_version 1386058 (0.0007) [2023-12-27 01:29:07,541][105620] Updated weights for policy 1, policy_version 1388060 (0.0010) [2023-12-27 01:29:07,600][105620] Updated weights for policy 1, policy_version 1388070 (0.0010) [2023-12-27 01:29:07,668][105620] Updated weights for policy 1, policy_version 1388080 (0.0009) [2023-12-27 01:29:08,047][105692] Updated weights for policy 0, policy_version 1386068 (0.0009) [2023-12-27 01:29:08,095][105692] Updated weights for policy 0, policy_version 1386078 (0.0009) [2023-12-27 01:29:08,146][105692] Updated weights for policy 0, policy_version 1386088 (0.0009) [2023-12-27 01:29:08,331][105620] Updated weights for policy 1, policy_version 1388090 (0.0008) [2023-12-27 01:29:08,399][105620] Updated weights for policy 1, policy_version 1388100 (0.0006) [2023-12-27 01:29:08,467][105620] Updated weights for policy 1, policy_version 1388110 (0.0006) [2023-12-27 01:29:08,537][105620] Updated weights for policy 1, policy_version 1388120 (0.0006) [2023-12-27 01:29:08,897][105692] Updated weights for policy 0, policy_version 1386098 (0.0010) [2023-12-27 01:29:08,954][105692] Updated weights for policy 0, policy_version 1386108 (0.0009) [2023-12-27 01:29:09,002][105692] Updated weights for policy 0, policy_version 1386118 (0.0009) [2023-12-27 01:29:09,050][105692] Updated weights for policy 0, policy_version 1386128 (0.0009) [2023-12-27 01:29:09,219][105620] Updated weights for policy 1, policy_version 1388130 (0.0009) [2023-12-27 01:29:09,283][105620] Updated weights for policy 1, policy_version 1388140 (0.0009) [2023-12-27 01:29:09,344][105620] Updated weights for policy 1, policy_version 1388150 (0.0008) [2023-12-27 01:29:09,904][105692] Updated weights for policy 0, policy_version 1386138 (0.0010) [2023-12-27 01:29:09,972][105692] Updated weights for policy 0, policy_version 1386148 (0.0010) [2023-12-27 01:29:10,008][105620] Updated weights for policy 1, policy_version 1388160 (0.0008) [2023-12-27 01:29:10,027][105692] Updated weights for policy 0, policy_version 1386158 (0.0007) [2023-12-27 01:29:10,070][105620] Updated weights for policy 1, policy_version 1388170 (0.0007) [2023-12-27 01:29:10,137][105620] Updated weights for policy 1, policy_version 1388180 (0.0008) [2023-12-27 01:29:10,823][105692] Updated weights for policy 0, policy_version 1386168 (0.0008) [2023-12-27 01:29:10,854][105620] Updated weights for policy 1, policy_version 1388190 (0.0008) [2023-12-27 01:29:10,883][105692] Updated weights for policy 0, policy_version 1386178 (0.0008) [2023-12-27 01:29:10,906][105620] Updated weights for policy 1, policy_version 1388200 (0.0006) [2023-12-27 01:29:10,933][105692] Updated weights for policy 0, policy_version 1386188 (0.0007) [2023-12-27 01:29:10,959][105620] Updated weights for policy 1, policy_version 1388210 (0.0006) [2023-12-27 01:29:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 710344704. Throughput: 0: 10000.6, 1: 9344.7. Samples: 710347620. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:29:11,062][104569] Avg episode reward: [(0, '8441.574'), (1, '9083.339')] [2023-12-27 01:29:11,689][105692] Updated weights for policy 0, policy_version 1386198 (0.0009) [2023-12-27 01:29:11,761][105620] Updated weights for policy 1, policy_version 1388220 (0.0009) [2023-12-27 01:29:11,762][105692] Updated weights for policy 0, policy_version 1386208 (0.0008) [2023-12-27 01:29:11,817][105620] Updated weights for policy 1, policy_version 1388230 (0.0006) [2023-12-27 01:29:11,819][105692] Updated weights for policy 0, policy_version 1386218 (0.0008) [2023-12-27 01:29:11,872][105620] Updated weights for policy 1, policy_version 1388240 (0.0007) [2023-12-27 01:29:12,570][105692] Updated weights for policy 0, policy_version 1386228 (0.0008) [2023-12-27 01:29:12,634][105692] Updated weights for policy 0, policy_version 1386238 (0.0008) [2023-12-27 01:29:12,673][105620] Updated weights for policy 1, policy_version 1388250 (0.0008) [2023-12-27 01:29:12,687][105692] Updated weights for policy 0, policy_version 1386248 (0.0008) [2023-12-27 01:29:12,729][105620] Updated weights for policy 1, policy_version 1388260 (0.0009) [2023-12-27 01:29:12,776][105620] Updated weights for policy 1, policy_version 1388270 (0.0008) [2023-12-27 01:29:12,826][105620] Updated weights for policy 1, policy_version 1388280 (0.0009) [2023-12-27 01:29:13,472][105692] Updated weights for policy 0, policy_version 1386258 (0.0007) [2023-12-27 01:29:13,478][105620] Updated weights for policy 1, policy_version 1388290 (0.0007) [2023-12-27 01:29:13,531][105692] Updated weights for policy 0, policy_version 1386268 (0.0006) [2023-12-27 01:29:13,540][105620] Updated weights for policy 1, policy_version 1388300 (0.0009) [2023-12-27 01:29:13,585][105692] Updated weights for policy 0, policy_version 1386278 (0.0007) [2023-12-27 01:29:13,597][105620] Updated weights for policy 1, policy_version 1388310 (0.0008) [2023-12-27 01:29:13,640][105692] Updated weights for policy 0, policy_version 1386288 (0.0005) [2023-12-27 01:29:14,191][105692] Updated weights for policy 0, policy_version 1386298 (0.0010) [2023-12-27 01:29:14,243][105620] Updated weights for policy 1, policy_version 1388320 (0.0007) [2023-12-27 01:29:14,245][105692] Updated weights for policy 0, policy_version 1386308 (0.0007) [2023-12-27 01:29:14,287][105620] Updated weights for policy 1, policy_version 1388330 (0.0007) [2023-12-27 01:29:14,305][105692] Updated weights for policy 0, policy_version 1386318 (0.0007) [2023-12-27 01:29:14,344][105620] Updated weights for policy 1, policy_version 1388340 (0.0007) [2023-12-27 01:29:15,039][105692] Updated weights for policy 0, policy_version 1386328 (0.0009) [2023-12-27 01:29:15,091][105692] Updated weights for policy 0, policy_version 1386338 (0.0009) [2023-12-27 01:29:15,135][105620] Updated weights for policy 1, policy_version 1388350 (0.0009) [2023-12-27 01:29:15,149][105692] Updated weights for policy 0, policy_version 1386348 (0.0007) [2023-12-27 01:29:15,197][105620] Updated weights for policy 1, policy_version 1388360 (0.0008) [2023-12-27 01:29:15,259][105620] Updated weights for policy 1, policy_version 1388370 (0.0009) [2023-12-27 01:29:15,892][105692] Updated weights for policy 0, policy_version 1386358 (0.0008) [2023-12-27 01:29:15,953][105692] Updated weights for policy 0, policy_version 1386368 (0.0009) [2023-12-27 01:29:15,999][105692] Updated weights for policy 0, policy_version 1386378 (0.0008) [2023-12-27 01:29:16,012][105620] Updated weights for policy 1, policy_version 1388380 (0.0009) [2023-12-27 01:29:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 710434816. Throughput: 0: 9913.2, 1: 9364.8. Samples: 710402916. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:29:16,062][104569] Avg episode reward: [(0, '7979.228'), (1, '8990.307')] [2023-12-27 01:29:16,064][105620] Updated weights for policy 1, policy_version 1388390 (0.0009) [2023-12-27 01:29:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001386384_354967552.pth... [2023-12-27 01:29:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001385200_354664448.pth [2023-12-27 01:29:16,130][105620] Updated weights for policy 1, policy_version 1388400 (0.0010) [2023-12-27 01:29:16,184][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001388408_355475456.pth... [2023-12-27 01:29:16,189][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001387288_355188736.pth [2023-12-27 01:29:16,706][105692] Updated weights for policy 0, policy_version 1386388 (0.0008) [2023-12-27 01:29:16,761][105692] Updated weights for policy 0, policy_version 1386398 (0.0009) [2023-12-27 01:29:16,811][105692] Updated weights for policy 0, policy_version 1386408 (0.0009) [2023-12-27 01:29:16,903][105620] Updated weights for policy 1, policy_version 1388410 (0.0009) [2023-12-27 01:29:16,962][105620] Updated weights for policy 1, policy_version 1388420 (0.0005) [2023-12-27 01:29:17,022][105620] Updated weights for policy 1, policy_version 1388430 (0.0005) [2023-12-27 01:29:17,084][105620] Updated weights for policy 1, policy_version 1388440 (0.0005) [2023-12-27 01:29:17,583][105620] Updated weights for policy 1, policy_version 1388450 (0.0005) [2023-12-27 01:29:17,645][105620] Updated weights for policy 1, policy_version 1388460 (0.0005) [2023-12-27 01:29:17,705][105620] Updated weights for policy 1, policy_version 1388470 (0.0009) [2023-12-27 01:29:17,708][105692] Updated weights for policy 0, policy_version 1386418 (0.0008) [2023-12-27 01:29:17,755][105692] Updated weights for policy 0, policy_version 1386428 (0.0009) [2023-12-27 01:29:17,803][105692] Updated weights for policy 0, policy_version 1386438 (0.0009) [2023-12-27 01:29:17,857][105692] Updated weights for policy 0, policy_version 1386448 (0.0009) [2023-12-27 01:29:18,354][105620] Updated weights for policy 1, policy_version 1388480 (0.0009) [2023-12-27 01:29:18,411][105620] Updated weights for policy 1, policy_version 1388490 (0.0009) [2023-12-27 01:29:18,469][105620] Updated weights for policy 1, policy_version 1388500 (0.0009) [2023-12-27 01:29:18,658][105692] Updated weights for policy 0, policy_version 1386458 (0.0005) [2023-12-27 01:29:18,717][105692] Updated weights for policy 0, policy_version 1386468 (0.0005) [2023-12-27 01:29:18,766][105692] Updated weights for policy 0, policy_version 1386478 (0.0006) [2023-12-27 01:29:19,333][105620] Updated weights for policy 1, policy_version 1388510 (0.0009) [2023-12-27 01:29:19,393][105620] Updated weights for policy 1, policy_version 1388520 (0.0009) [2023-12-27 01:29:19,412][105692] Updated weights for policy 0, policy_version 1386488 (0.0009) [2023-12-27 01:29:19,450][105620] Updated weights for policy 1, policy_version 1388530 (0.0006) [2023-12-27 01:29:19,476][105692] Updated weights for policy 0, policy_version 1386498 (0.0009) [2023-12-27 01:29:19,541][105692] Updated weights for policy 0, policy_version 1386508 (0.0009) [2023-12-27 01:29:20,194][105620] Updated weights for policy 1, policy_version 1388540 (0.0007) [2023-12-27 01:29:20,252][105620] Updated weights for policy 1, policy_version 1388550 (0.0009) [2023-12-27 01:29:20,316][105620] Updated weights for policy 1, policy_version 1388560 (0.0008) [2023-12-27 01:29:20,334][105692] Updated weights for policy 0, policy_version 1386518 (0.0008) [2023-12-27 01:29:20,398][105692] Updated weights for policy 0, policy_version 1386528 (0.0008) [2023-12-27 01:29:20,453][105692] Updated weights for policy 0, policy_version 1386538 (0.0009) [2023-12-27 01:29:21,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 710524928. Throughput: 0: 9869.6, 1: 9367.9. Samples: 710519424. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:29:21,063][104569] Avg episode reward: [(0, '8068.150'), (1, '8718.304')] [2023-12-27 01:29:21,106][105620] Updated weights for policy 1, policy_version 1388570 (0.0007) [2023-12-27 01:29:21,173][105620] Updated weights for policy 1, policy_version 1388580 (0.0008) [2023-12-27 01:29:21,223][105692] Updated weights for policy 0, policy_version 1386548 (0.0009) [2023-12-27 01:29:21,236][105620] Updated weights for policy 1, policy_version 1388590 (0.0009) [2023-12-27 01:29:21,291][105692] Updated weights for policy 0, policy_version 1386558 (0.0008) [2023-12-27 01:29:21,305][105620] Updated weights for policy 1, policy_version 1388600 (0.0007) [2023-12-27 01:29:21,357][105692] Updated weights for policy 0, policy_version 1386568 (0.0008) [2023-12-27 01:29:22,027][105620] Updated weights for policy 1, policy_version 1388610 (0.0010) [2023-12-27 01:29:22,085][105620] Updated weights for policy 1, policy_version 1388620 (0.0008) [2023-12-27 01:29:22,094][105692] Updated weights for policy 0, policy_version 1386578 (0.0010) [2023-12-27 01:29:22,147][105620] Updated weights for policy 1, policy_version 1388630 (0.0008) [2023-12-27 01:29:22,151][105692] Updated weights for policy 0, policy_version 1386588 (0.0006) [2023-12-27 01:29:22,207][105692] Updated weights for policy 0, policy_version 1386598 (0.0006) [2023-12-27 01:29:22,278][105692] Updated weights for policy 0, policy_version 1386608 (0.0008) [2023-12-27 01:29:22,952][105620] Updated weights for policy 1, policy_version 1388640 (0.0008) [2023-12-27 01:29:22,992][105692] Updated weights for policy 0, policy_version 1386618 (0.0007) [2023-12-27 01:29:23,012][105620] Updated weights for policy 1, policy_version 1388650 (0.0008) [2023-12-27 01:29:23,048][105692] Updated weights for policy 0, policy_version 1386628 (0.0007) [2023-12-27 01:29:23,066][105620] Updated weights for policy 1, policy_version 1388660 (0.0009) [2023-12-27 01:29:23,102][105692] Updated weights for policy 0, policy_version 1386638 (0.0007) [2023-12-27 01:29:23,840][105620] Updated weights for policy 1, policy_version 1388670 (0.0008) [2023-12-27 01:29:23,890][105692] Updated weights for policy 0, policy_version 1386648 (0.0007) [2023-12-27 01:29:23,892][105620] Updated weights for policy 1, policy_version 1388680 (0.0007) [2023-12-27 01:29:23,943][105692] Updated weights for policy 0, policy_version 1386658 (0.0006) [2023-12-27 01:29:23,951][105620] Updated weights for policy 1, policy_version 1388690 (0.0008) [2023-12-27 01:29:24,000][105692] Updated weights for policy 0, policy_version 1386668 (0.0010) [2023-12-27 01:29:24,612][105620] Updated weights for policy 1, policy_version 1388700 (0.0006) [2023-12-27 01:29:24,667][105620] Updated weights for policy 1, policy_version 1388710 (0.0008) [2023-12-27 01:29:24,725][105620] Updated weights for policy 1, policy_version 1388720 (0.0008) [2023-12-27 01:29:24,729][105692] Updated weights for policy 0, policy_version 1386678 (0.0010) [2023-12-27 01:29:24,779][105585] KL-divergence is very high: 123.0905 [2023-12-27 01:29:24,779][105692] Updated weights for policy 0, policy_version 1386688 (0.0010) [2023-12-27 01:29:24,819][105585] KL-divergence is very high: 117.1581 [2023-12-27 01:29:24,828][105692] Updated weights for policy 0, policy_version 1386698 (0.0005) [2023-12-27 01:29:25,477][105620] Updated weights for policy 1, policy_version 1388730 (0.0007) [2023-12-27 01:29:25,541][105692] Updated weights for policy 0, policy_version 1386708 (0.0005) [2023-12-27 01:29:25,546][105620] Updated weights for policy 1, policy_version 1388740 (0.0010) [2023-12-27 01:29:25,593][105692] Updated weights for policy 0, policy_version 1386718 (0.0009) [2023-12-27 01:29:25,607][105620] Updated weights for policy 1, policy_version 1388750 (0.0008) [2023-12-27 01:29:25,642][105692] Updated weights for policy 0, policy_version 1386728 (0.0010) [2023-12-27 01:29:25,667][105620] Updated weights for policy 1, policy_version 1388760 (0.0006) [2023-12-27 01:29:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.3, 300 sec: 19438.6). Total num frames: 710623232. Throughput: 0: 9843.1, 1: 9354.9. Samples: 710631236. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:29:26,062][104569] Avg episode reward: [(0, '8249.590'), (1, '9084.857')] [2023-12-27 01:29:26,392][105692] Updated weights for policy 0, policy_version 1386738 (0.0010) [2023-12-27 01:29:26,410][105620] Updated weights for policy 1, policy_version 1388770 (0.0007) [2023-12-27 01:29:26,450][105692] Updated weights for policy 0, policy_version 1386748 (0.0010) [2023-12-27 01:29:26,472][105620] Updated weights for policy 1, policy_version 1388780 (0.0005) [2023-12-27 01:29:26,505][105692] Updated weights for policy 0, policy_version 1386758 (0.0010) [2023-12-27 01:29:26,519][105620] Updated weights for policy 1, policy_version 1388790 (0.0005) [2023-12-27 01:29:26,559][105692] Updated weights for policy 0, policy_version 1386768 (0.0010) [2023-12-27 01:29:27,126][105692] Updated weights for policy 0, policy_version 1386778 (0.0005) [2023-12-27 01:29:27,171][105692] Updated weights for policy 0, policy_version 1386788 (0.0005) [2023-12-27 01:29:27,216][105692] Updated weights for policy 0, policy_version 1386798 (0.0005) [2023-12-27 01:29:27,352][105620] Updated weights for policy 1, policy_version 1388800 (0.0008) [2023-12-27 01:29:27,417][105620] Updated weights for policy 1, policy_version 1388810 (0.0009) [2023-12-27 01:29:27,487][105620] Updated weights for policy 1, policy_version 1388820 (0.0009) [2023-12-27 01:29:27,755][105692] Updated weights for policy 0, policy_version 1386808 (0.0009) [2023-12-27 01:29:27,806][105692] Updated weights for policy 0, policy_version 1386818 (0.0010) [2023-12-27 01:29:27,864][105692] Updated weights for policy 0, policy_version 1386828 (0.0010) [2023-12-27 01:29:28,316][105620] Updated weights for policy 1, policy_version 1388830 (0.0009) [2023-12-27 01:29:28,384][105620] Updated weights for policy 1, policy_version 1388840 (0.0009) [2023-12-27 01:29:28,445][105620] Updated weights for policy 1, policy_version 1388850 (0.0009) [2023-12-27 01:29:28,552][105692] Updated weights for policy 0, policy_version 1386838 (0.0008) [2023-12-27 01:29:28,601][105692] Updated weights for policy 0, policy_version 1386848 (0.0010) [2023-12-27 01:29:28,654][105692] Updated weights for policy 0, policy_version 1386858 (0.0010) [2023-12-27 01:29:29,212][105620] Updated weights for policy 1, policy_version 1388860 (0.0009) [2023-12-27 01:29:29,272][105620] Updated weights for policy 1, policy_version 1388870 (0.0008) [2023-12-27 01:29:29,340][105620] Updated weights for policy 1, policy_version 1388880 (0.0006) [2023-12-27 01:29:29,402][105692] Updated weights for policy 0, policy_version 1386868 (0.0008) [2023-12-27 01:29:29,468][105692] Updated weights for policy 0, policy_version 1386878 (0.0005) [2023-12-27 01:29:29,522][105692] Updated weights for policy 0, policy_version 1386888 (0.0005) [2023-12-27 01:29:29,933][105620] Updated weights for policy 1, policy_version 1388890 (0.0007) [2023-12-27 01:29:29,993][105620] Updated weights for policy 1, policy_version 1388900 (0.0008) [2023-12-27 01:29:30,054][105620] Updated weights for policy 1, policy_version 1388910 (0.0008) [2023-12-27 01:29:30,109][105620] Updated weights for policy 1, policy_version 1388920 (0.0008) [2023-12-27 01:29:30,117][105692] Updated weights for policy 0, policy_version 1386898 (0.0006) [2023-12-27 01:29:30,172][105692] Updated weights for policy 0, policy_version 1386908 (0.0010) [2023-12-27 01:29:30,223][105692] Updated weights for policy 0, policy_version 1386918 (0.0010) [2023-12-27 01:29:30,275][105692] Updated weights for policy 0, policy_version 1386928 (0.0010) [2023-12-27 01:29:30,739][105620] Updated weights for policy 1, policy_version 1388930 (0.0005) [2023-12-27 01:29:30,786][105620] Updated weights for policy 1, policy_version 1388940 (0.0006) [2023-12-27 01:29:30,846][105620] Updated weights for policy 1, policy_version 1388950 (0.0005) [2023-12-27 01:29:31,053][105692] Updated weights for policy 0, policy_version 1386938 (0.0011) [2023-12-27 01:29:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 710721536. Throughput: 0: 9935.0, 1: 9307.3. Samples: 710690396. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:29:31,062][104569] Avg episode reward: [(0, '7884.417'), (1, '9356.798')] [2023-12-27 01:29:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001388952_355614720.pth... [2023-12-27 01:29:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001387832_355328000.pth [2023-12-27 01:29:31,113][105692] Updated weights for policy 0, policy_version 1386948 (0.0010) [2023-12-27 01:29:31,178][105692] Updated weights for policy 0, policy_version 1386958 (0.0011) [2023-12-27 01:29:31,188][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001386960_355115008.pth... [2023-12-27 01:29:31,193][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001385808_354820096.pth [2023-12-27 01:29:31,565][105620] Updated weights for policy 1, policy_version 1388960 (0.0007) [2023-12-27 01:29:31,609][105620] Updated weights for policy 1, policy_version 1388970 (0.0008) [2023-12-27 01:29:31,663][105620] Updated weights for policy 1, policy_version 1388980 (0.0008) [2023-12-27 01:29:31,919][105692] Updated weights for policy 0, policy_version 1386968 (0.0010) [2023-12-27 01:29:31,978][105692] Updated weights for policy 0, policy_version 1386978 (0.0010) [2023-12-27 01:29:32,033][105692] Updated weights for policy 0, policy_version 1386988 (0.0010) [2023-12-27 01:29:32,309][105620] Updated weights for policy 1, policy_version 1388990 (0.0008) [2023-12-27 01:29:32,362][105620] Updated weights for policy 1, policy_version 1389000 (0.0008) [2023-12-27 01:29:32,425][105620] Updated weights for policy 1, policy_version 1389010 (0.0008) [2023-12-27 01:29:32,724][105692] Updated weights for policy 0, policy_version 1386998 (0.0007) [2023-12-27 01:29:32,778][105692] Updated weights for policy 0, policy_version 1387008 (0.0005) [2023-12-27 01:29:32,824][105692] Updated weights for policy 0, policy_version 1387018 (0.0005) [2023-12-27 01:29:33,263][105620] Updated weights for policy 1, policy_version 1389020 (0.0009) [2023-12-27 01:29:33,318][105620] Updated weights for policy 1, policy_version 1389030 (0.0009) [2023-12-27 01:29:33,364][105620] Updated weights for policy 1, policy_version 1389040 (0.0008) [2023-12-27 01:29:33,396][105692] Updated weights for policy 0, policy_version 1387028 (0.0006) [2023-12-27 01:29:33,455][105692] Updated weights for policy 0, policy_version 1387038 (0.0008) [2023-12-27 01:29:33,512][105692] Updated weights for policy 0, policy_version 1387048 (0.0009) [2023-12-27 01:29:34,117][105620] Updated weights for policy 1, policy_version 1389050 (0.0009) [2023-12-27 01:29:34,180][105620] Updated weights for policy 1, policy_version 1389060 (0.0008) [2023-12-27 01:29:34,230][105620] Updated weights for policy 1, policy_version 1389070 (0.0005) [2023-12-27 01:29:34,276][105692] Updated weights for policy 0, policy_version 1387058 (0.0009) [2023-12-27 01:29:34,294][105620] Updated weights for policy 1, policy_version 1389080 (0.0006) [2023-12-27 01:29:34,330][105692] Updated weights for policy 0, policy_version 1387068 (0.0010) [2023-12-27 01:29:34,383][105692] Updated weights for policy 0, policy_version 1387078 (0.0009) [2023-12-27 01:29:34,431][105692] Updated weights for policy 0, policy_version 1387088 (0.0009) [2023-12-27 01:29:34,982][105620] Updated weights for policy 1, policy_version 1389090 (0.0006) [2023-12-27 01:29:35,038][105620] Updated weights for policy 1, policy_version 1389100 (0.0007) [2023-12-27 01:29:35,088][105620] Updated weights for policy 1, policy_version 1389110 (0.0008) [2023-12-27 01:29:35,243][105692] Updated weights for policy 0, policy_version 1387098 (0.0009) [2023-12-27 01:29:35,289][105692] Updated weights for policy 0, policy_version 1387108 (0.0009) [2023-12-27 01:29:35,347][105692] Updated weights for policy 0, policy_version 1387118 (0.0009) [2023-12-27 01:29:35,839][105620] Updated weights for policy 1, policy_version 1389120 (0.0009) [2023-12-27 01:29:35,889][105620] Updated weights for policy 1, policy_version 1389130 (0.0009) [2023-12-27 01:29:35,935][105620] Updated weights for policy 1, policy_version 1389140 (0.0009) [2023-12-27 01:29:36,052][105692] Updated weights for policy 0, policy_version 1387128 (0.0009) [2023-12-27 01:29:36,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 710819840. Throughput: 0: 9789.8, 1: 9380.7. Samples: 710809004. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:29:36,063][104569] Avg episode reward: [(0, '8070.694'), (1, '9356.974')] [2023-12-27 01:29:36,113][105692] Updated weights for policy 0, policy_version 1387138 (0.0008) [2023-12-27 01:29:36,170][105692] Updated weights for policy 0, policy_version 1387148 (0.0008) [2023-12-27 01:29:36,726][105620] Updated weights for policy 1, policy_version 1389150 (0.0009) [2023-12-27 01:29:36,787][105620] Updated weights for policy 1, policy_version 1389160 (0.0009) [2023-12-27 01:29:36,848][105620] Updated weights for policy 1, policy_version 1389170 (0.0010) [2023-12-27 01:29:36,960][105692] Updated weights for policy 0, policy_version 1387158 (0.0008) [2023-12-27 01:29:37,028][105692] Updated weights for policy 0, policy_version 1387168 (0.0006) [2023-12-27 01:29:37,094][105692] Updated weights for policy 0, policy_version 1387178 (0.0008) [2023-12-27 01:29:37,647][105620] Updated weights for policy 1, policy_version 1389180 (0.0008) [2023-12-27 01:29:37,704][105620] Updated weights for policy 1, policy_version 1389190 (0.0008) [2023-12-27 01:29:37,723][105692] Updated weights for policy 0, policy_version 1387188 (0.0006) [2023-12-27 01:29:37,764][105620] Updated weights for policy 1, policy_version 1389200 (0.0009) [2023-12-27 01:29:37,779][105692] Updated weights for policy 0, policy_version 1387198 (0.0006) [2023-12-27 01:29:37,838][105692] Updated weights for policy 0, policy_version 1387208 (0.0007) [2023-12-27 01:29:38,527][105620] Updated weights for policy 1, policy_version 1389210 (0.0008) [2023-12-27 01:29:38,594][105620] Updated weights for policy 1, policy_version 1389220 (0.0009) [2023-12-27 01:29:38,628][105692] Updated weights for policy 0, policy_version 1387218 (0.0009) [2023-12-27 01:29:38,661][105620] Updated weights for policy 1, policy_version 1389230 (0.0009) [2023-12-27 01:29:38,674][105692] Updated weights for policy 0, policy_version 1387228 (0.0006) [2023-12-27 01:29:38,728][105620] Updated weights for policy 1, policy_version 1389240 (0.0010) [2023-12-27 01:29:38,736][105692] Updated weights for policy 0, policy_version 1387238 (0.0008) [2023-12-27 01:29:38,795][105692] Updated weights for policy 0, policy_version 1387248 (0.0006) [2023-12-27 01:29:39,415][105692] Updated weights for policy 0, policy_version 1387258 (0.0008) [2023-12-27 01:29:39,478][105692] Updated weights for policy 0, policy_version 1387268 (0.0009) [2023-12-27 01:29:39,537][105692] Updated weights for policy 0, policy_version 1387278 (0.0010) [2023-12-27 01:29:39,540][105620] Updated weights for policy 1, policy_version 1389250 (0.0009) [2023-12-27 01:29:39,601][105620] Updated weights for policy 1, policy_version 1389260 (0.0009) [2023-12-27 01:29:39,662][105620] Updated weights for policy 1, policy_version 1389270 (0.0009) [2023-12-27 01:29:40,289][105692] Updated weights for policy 0, policy_version 1387288 (0.0009) [2023-12-27 01:29:40,345][105692] Updated weights for policy 0, policy_version 1387298 (0.0009) [2023-12-27 01:29:40,403][105692] Updated weights for policy 0, policy_version 1387308 (0.0008) [2023-12-27 01:29:40,438][105620] Updated weights for policy 1, policy_version 1389280 (0.0008) [2023-12-27 01:29:40,492][105620] Updated weights for policy 1, policy_version 1389290 (0.0009) [2023-12-27 01:29:40,546][105620] Updated weights for policy 1, policy_version 1389300 (0.0009) [2023-12-27 01:29:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 710909952. Throughput: 0: 9734.0, 1: 9386.9. Samples: 710920168. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:29:41,063][104569] Avg episode reward: [(0, '8618.119'), (1, '9357.053')] [2023-12-27 01:29:41,151][105692] Updated weights for policy 0, policy_version 1387318 (0.0008) [2023-12-27 01:29:41,214][105692] Updated weights for policy 0, policy_version 1387328 (0.0008) [2023-12-27 01:29:41,287][105692] Updated weights for policy 0, policy_version 1387338 (0.0007) [2023-12-27 01:29:41,432][105620] Updated weights for policy 1, policy_version 1389310 (0.0008) [2023-12-27 01:29:41,498][105620] Updated weights for policy 1, policy_version 1389320 (0.0008) [2023-12-27 01:29:41,568][105620] Updated weights for policy 1, policy_version 1389330 (0.0009) [2023-12-27 01:29:42,032][105692] Updated weights for policy 0, policy_version 1387348 (0.0007) [2023-12-27 01:29:42,090][105692] Updated weights for policy 0, policy_version 1387358 (0.0009) [2023-12-27 01:29:42,146][105692] Updated weights for policy 0, policy_version 1387368 (0.0009) [2023-12-27 01:29:42,167][105620] Updated weights for policy 1, policy_version 1389340 (0.0009) [2023-12-27 01:29:42,224][105620] Updated weights for policy 1, policy_version 1389350 (0.0006) [2023-12-27 01:29:42,288][105620] Updated weights for policy 1, policy_version 1389360 (0.0006) [2023-12-27 01:29:42,871][105692] Updated weights for policy 0, policy_version 1387378 (0.0007) [2023-12-27 01:29:42,931][105692] Updated weights for policy 0, policy_version 1387388 (0.0007) [2023-12-27 01:29:42,993][105692] Updated weights for policy 0, policy_version 1387398 (0.0009) [2023-12-27 01:29:43,001][105620] Updated weights for policy 1, policy_version 1389370 (0.0006) [2023-12-27 01:29:43,049][105692] Updated weights for policy 0, policy_version 1387408 (0.0007) [2023-12-27 01:29:43,056][105620] Updated weights for policy 1, policy_version 1389380 (0.0007) [2023-12-27 01:29:43,118][105620] Updated weights for policy 1, policy_version 1389390 (0.0009) [2023-12-27 01:29:43,177][105620] Updated weights for policy 1, policy_version 1389400 (0.0009) [2023-12-27 01:29:43,805][105692] Updated weights for policy 0, policy_version 1387418 (0.0007) [2023-12-27 01:29:43,851][105620] Updated weights for policy 1, policy_version 1389410 (0.0009) [2023-12-27 01:29:43,856][105692] Updated weights for policy 0, policy_version 1387428 (0.0006) [2023-12-27 01:29:43,902][105620] Updated weights for policy 1, policy_version 1389420 (0.0005) [2023-12-27 01:29:43,913][105692] Updated weights for policy 0, policy_version 1387438 (0.0008) [2023-12-27 01:29:43,962][105620] Updated weights for policy 1, policy_version 1389430 (0.0008) [2023-12-27 01:29:44,598][105620] Updated weights for policy 1, policy_version 1389440 (0.0009) [2023-12-27 01:29:44,654][105692] Updated weights for policy 0, policy_version 1387448 (0.0006) [2023-12-27 01:29:44,655][105620] Updated weights for policy 1, policy_version 1389450 (0.0008) [2023-12-27 01:29:44,703][105692] Updated weights for policy 0, policy_version 1387458 (0.0009) [2023-12-27 01:29:44,708][105620] Updated weights for policy 1, policy_version 1389460 (0.0005) [2023-12-27 01:29:44,752][105692] Updated weights for policy 0, policy_version 1387468 (0.0009) [2023-12-27 01:29:45,316][105620] Updated weights for policy 1, policy_version 1389470 (0.0006) [2023-12-27 01:29:45,359][105692] Updated weights for policy 0, policy_version 1387478 (0.0006) [2023-12-27 01:29:45,387][105620] Updated weights for policy 1, policy_version 1389480 (0.0011) [2023-12-27 01:29:45,412][105692] Updated weights for policy 0, policy_version 1387488 (0.0005) [2023-12-27 01:29:45,454][105620] Updated weights for policy 1, policy_version 1389490 (0.0011) [2023-12-27 01:29:45,471][105692] Updated weights for policy 0, policy_version 1387498 (0.0006) [2023-12-27 01:29:46,062][104569] Fps is (10 sec: 18840.8, 60 sec: 19251.0, 300 sec: 19410.8). Total num frames: 711008256. Throughput: 0: 9692.3, 1: 9455.9. Samples: 710978332. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:29:46,064][104569] Avg episode reward: [(0, '8529.847'), (1, '8355.050')] [2023-12-27 01:29:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001387504_355254272.pth... [2023-12-27 01:29:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001386384_354967552.pth [2023-12-27 01:29:46,102][105620] Updated weights for policy 1, policy_version 1389500 (0.0011) [2023-12-27 01:29:46,160][105620] Updated weights for policy 1, policy_version 1389510 (0.0010) [2023-12-27 01:29:46,214][105620] Updated weights for policy 1, policy_version 1389520 (0.0010) [2023-12-27 01:29:46,221][105692] Updated weights for policy 0, policy_version 1387508 (0.0008) [2023-12-27 01:29:46,254][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001389528_355762176.pth... [2023-12-27 01:29:46,258][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001388408_355475456.pth [2023-12-27 01:29:46,280][105692] Updated weights for policy 0, policy_version 1387518 (0.0007) [2023-12-27 01:29:46,340][105692] Updated weights for policy 0, policy_version 1387528 (0.0008) [2023-12-27 01:29:46,939][105620] Updated weights for policy 1, policy_version 1389530 (0.0010) [2023-12-27 01:29:46,988][105620] Updated weights for policy 1, policy_version 1389540 (0.0010) [2023-12-27 01:29:47,018][105692] Updated weights for policy 0, policy_version 1387538 (0.0008) [2023-12-27 01:29:47,037][105620] Updated weights for policy 1, policy_version 1389550 (0.0010) [2023-12-27 01:29:47,071][105692] Updated weights for policy 0, policy_version 1387548 (0.0006) [2023-12-27 01:29:47,081][105620] Updated weights for policy 1, policy_version 1389560 (0.0010) [2023-12-27 01:29:47,125][105692] Updated weights for policy 0, policy_version 1387558 (0.0007) [2023-12-27 01:29:47,169][105692] Updated weights for policy 0, policy_version 1387568 (0.0008) [2023-12-27 01:29:47,726][105620] Updated weights for policy 1, policy_version 1389570 (0.0008) [2023-12-27 01:29:47,781][105620] Updated weights for policy 1, policy_version 1389580 (0.0010) [2023-12-27 01:29:47,847][105620] Updated weights for policy 1, policy_version 1389590 (0.0010) [2023-12-27 01:29:48,008][105692] Updated weights for policy 0, policy_version 1387578 (0.0008) [2023-12-27 01:29:48,055][105692] Updated weights for policy 0, policy_version 1387588 (0.0005) [2023-12-27 01:29:48,103][105692] Updated weights for policy 0, policy_version 1387598 (0.0005) [2023-12-27 01:29:48,574][105620] Updated weights for policy 1, policy_version 1389600 (0.0011) [2023-12-27 01:29:48,633][105620] Updated weights for policy 1, policy_version 1389610 (0.0011) [2023-12-27 01:29:48,696][105620] Updated weights for policy 1, policy_version 1389620 (0.0011) [2023-12-27 01:29:48,782][105692] Updated weights for policy 0, policy_version 1387608 (0.0008) [2023-12-27 01:29:48,839][105692] Updated weights for policy 0, policy_version 1387618 (0.0008) [2023-12-27 01:29:48,895][105692] Updated weights for policy 0, policy_version 1387628 (0.0008) [2023-12-27 01:29:49,469][105620] Updated weights for policy 1, policy_version 1389630 (0.0011) [2023-12-27 01:29:49,535][105620] Updated weights for policy 1, policy_version 1389640 (0.0010) [2023-12-27 01:29:49,591][105620] Updated weights for policy 1, policy_version 1389650 (0.0009) [2023-12-27 01:29:49,606][105692] Updated weights for policy 0, policy_version 1387638 (0.0007) [2023-12-27 01:29:49,656][105692] Updated weights for policy 0, policy_version 1387648 (0.0007) [2023-12-27 01:29:49,710][105692] Updated weights for policy 0, policy_version 1387658 (0.0008) [2023-12-27 01:29:50,308][105620] Updated weights for policy 1, policy_version 1389660 (0.0011) [2023-12-27 01:29:50,361][105620] Updated weights for policy 1, policy_version 1389670 (0.0011) [2023-12-27 01:29:50,417][105620] Updated weights for policy 1, policy_version 1389680 (0.0005) [2023-12-27 01:29:50,491][105692] Updated weights for policy 0, policy_version 1387668 (0.0009) [2023-12-27 01:29:50,543][105692] Updated weights for policy 0, policy_version 1387678 (0.0010) [2023-12-27 01:29:50,613][105692] Updated weights for policy 0, policy_version 1387688 (0.0009) [2023-12-27 01:29:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19114.7, 300 sec: 19410.9). Total num frames: 711106560. Throughput: 0: 9697.1, 1: 9594.3. Samples: 711098272. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:29:51,062][104569] Avg episode reward: [(0, '8437.299'), (1, '8042.312')] [2023-12-27 01:29:51,088][105620] Updated weights for policy 1, policy_version 1389690 (0.0006) [2023-12-27 01:29:51,150][105620] Updated weights for policy 1, policy_version 1389700 (0.0009) [2023-12-27 01:29:51,200][105620] Updated weights for policy 1, policy_version 1389710 (0.0011) [2023-12-27 01:29:51,271][105620] Updated weights for policy 1, policy_version 1389720 (0.0007) [2023-12-27 01:29:51,461][105692] Updated weights for policy 0, policy_version 1387698 (0.0009) [2023-12-27 01:29:51,521][105692] Updated weights for policy 0, policy_version 1387708 (0.0009) [2023-12-27 01:29:51,582][105692] Updated weights for policy 0, policy_version 1387718 (0.0010) [2023-12-27 01:29:51,648][105692] Updated weights for policy 0, policy_version 1387728 (0.0009) [2023-12-27 01:29:52,034][105620] Updated weights for policy 1, policy_version 1389730 (0.0010) [2023-12-27 01:29:52,086][105620] Updated weights for policy 1, policy_version 1389740 (0.0011) [2023-12-27 01:29:52,145][105620] Updated weights for policy 1, policy_version 1389750 (0.0011) [2023-12-27 01:29:52,332][105692] Updated weights for policy 0, policy_version 1387738 (0.0010) [2023-12-27 01:29:52,392][105692] Updated weights for policy 0, policy_version 1387748 (0.0011) [2023-12-27 01:29:52,447][105692] Updated weights for policy 0, policy_version 1387758 (0.0011) [2023-12-27 01:29:52,886][105620] Updated weights for policy 1, policy_version 1389760 (0.0008) [2023-12-27 01:29:52,945][105620] Updated weights for policy 1, policy_version 1389770 (0.0007) [2023-12-27 01:29:53,007][105620] Updated weights for policy 1, policy_version 1389780 (0.0011) [2023-12-27 01:29:53,216][105692] Updated weights for policy 0, policy_version 1387768 (0.0010) [2023-12-27 01:29:53,268][105692] Updated weights for policy 0, policy_version 1387778 (0.0010) [2023-12-27 01:29:53,316][105692] Updated weights for policy 0, policy_version 1387788 (0.0010) [2023-12-27 01:29:53,549][105620] Updated weights for policy 1, policy_version 1389790 (0.0007) [2023-12-27 01:29:53,602][105620] Updated weights for policy 1, policy_version 1389800 (0.0005) [2023-12-27 01:29:53,655][105620] Updated weights for policy 1, policy_version 1389810 (0.0005) [2023-12-27 01:29:53,956][105692] Updated weights for policy 0, policy_version 1387798 (0.0007) [2023-12-27 01:29:54,023][105692] Updated weights for policy 0, policy_version 1387808 (0.0006) [2023-12-27 01:29:54,087][105692] Updated weights for policy 0, policy_version 1387818 (0.0005) [2023-12-27 01:29:54,348][105620] Updated weights for policy 1, policy_version 1389820 (0.0010) [2023-12-27 01:29:54,404][105620] Updated weights for policy 1, policy_version 1389830 (0.0011) [2023-12-27 01:29:54,467][105620] Updated weights for policy 1, policy_version 1389840 (0.0011) [2023-12-27 01:29:54,630][105692] Updated weights for policy 0, policy_version 1387828 (0.0008) [2023-12-27 01:29:54,689][105692] Updated weights for policy 0, policy_version 1387838 (0.0010) [2023-12-27 01:29:54,748][105692] Updated weights for policy 0, policy_version 1387848 (0.0010) [2023-12-27 01:29:55,216][105620] Updated weights for policy 1, policy_version 1389850 (0.0010) [2023-12-27 01:29:55,261][105620] Updated weights for policy 1, policy_version 1389860 (0.0010) [2023-12-27 01:29:55,313][105620] Updated weights for policy 1, policy_version 1389870 (0.0010) [2023-12-27 01:29:55,368][105620] Updated weights for policy 1, policy_version 1389880 (0.0010) [2023-12-27 01:29:55,485][105692] Updated weights for policy 0, policy_version 1387858 (0.0011) [2023-12-27 01:29:55,533][105692] Updated weights for policy 0, policy_version 1387868 (0.0010) [2023-12-27 01:29:55,597][105692] Updated weights for policy 0, policy_version 1387878 (0.0010) [2023-12-27 01:29:55,661][105692] Updated weights for policy 0, policy_version 1387888 (0.0010) [2023-12-27 01:29:56,062][104569] Fps is (10 sec: 19661.8, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 711204864. Throughput: 0: 9701.9, 1: 9592.2. Samples: 711215852. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:29:56,063][104569] Avg episode reward: [(0, '8528.126'), (1, '8652.411')] [2023-12-27 01:29:56,138][105620] Updated weights for policy 1, policy_version 1389890 (0.0010) [2023-12-27 01:29:56,183][105620] Updated weights for policy 1, policy_version 1389900 (0.0010) [2023-12-27 01:29:56,232][105620] Updated weights for policy 1, policy_version 1389910 (0.0010) [2023-12-27 01:29:56,375][105692] Updated weights for policy 0, policy_version 1387898 (0.0010) [2023-12-27 01:29:56,435][105692] Updated weights for policy 0, policy_version 1387908 (0.0010) [2023-12-27 01:29:56,502][105692] Updated weights for policy 0, policy_version 1387918 (0.0010) [2023-12-27 01:29:56,976][105620] Updated weights for policy 1, policy_version 1389920 (0.0006) [2023-12-27 01:29:57,021][105620] Updated weights for policy 1, policy_version 1389930 (0.0005) [2023-12-27 01:29:57,077][105620] Updated weights for policy 1, policy_version 1389940 (0.0005) [2023-12-27 01:29:57,234][105692] Updated weights for policy 0, policy_version 1387928 (0.0011) [2023-12-27 01:29:57,281][105692] Updated weights for policy 0, policy_version 1387938 (0.0010) [2023-12-27 01:29:57,330][105692] Updated weights for policy 0, policy_version 1387948 (0.0009) [2023-12-27 01:29:57,642][105620] Updated weights for policy 1, policy_version 1389950 (0.0008) [2023-12-27 01:29:57,686][105620] Updated weights for policy 1, policy_version 1389960 (0.0010) [2023-12-27 01:29:57,730][105620] Updated weights for policy 1, policy_version 1389970 (0.0010) [2023-12-27 01:29:58,060][105692] Updated weights for policy 0, policy_version 1387958 (0.0010) [2023-12-27 01:29:58,114][105692] Updated weights for policy 0, policy_version 1387968 (0.0010) [2023-12-27 01:29:58,167][105692] Updated weights for policy 0, policy_version 1387978 (0.0010) [2023-12-27 01:29:58,468][105620] Updated weights for policy 1, policy_version 1389980 (0.0010) [2023-12-27 01:29:58,534][105620] Updated weights for policy 1, policy_version 1389990 (0.0007) [2023-12-27 01:29:58,607][105620] Updated weights for policy 1, policy_version 1390000 (0.0009) [2023-12-27 01:29:58,948][105692] Updated weights for policy 0, policy_version 1387988 (0.0009) [2023-12-27 01:29:59,008][105692] Updated weights for policy 0, policy_version 1387998 (0.0008) [2023-12-27 01:29:59,072][105692] Updated weights for policy 0, policy_version 1388008 (0.0008) [2023-12-27 01:29:59,454][105620] Updated weights for policy 1, policy_version 1390010 (0.0008) [2023-12-27 01:29:59,514][105620] Updated weights for policy 1, policy_version 1390020 (0.0008) [2023-12-27 01:29:59,570][105620] Updated weights for policy 1, policy_version 1390030 (0.0008) [2023-12-27 01:29:59,629][105620] Updated weights for policy 1, policy_version 1390040 (0.0010) [2023-12-27 01:29:59,858][105692] Updated weights for policy 0, policy_version 1388018 (0.0008) [2023-12-27 01:29:59,907][105692] Updated weights for policy 0, policy_version 1388028 (0.0006) [2023-12-27 01:29:59,971][105692] Updated weights for policy 0, policy_version 1388038 (0.0010) [2023-12-27 01:30:00,025][105692] Updated weights for policy 0, policy_version 1388048 (0.0008) [2023-12-27 01:30:00,349][105620] Updated weights for policy 1, policy_version 1390050 (0.0010) [2023-12-27 01:30:00,401][105620] Updated weights for policy 1, policy_version 1390060 (0.0008) [2023-12-27 01:30:00,457][105620] Updated weights for policy 1, policy_version 1390070 (0.0008) [2023-12-27 01:30:00,639][105692] Updated weights for policy 0, policy_version 1388058 (0.0010) [2023-12-27 01:30:00,691][105692] Updated weights for policy 0, policy_version 1388068 (0.0009) [2023-12-27 01:30:00,752][105692] Updated weights for policy 0, policy_version 1388078 (0.0009) [2023-12-27 01:30:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 711303168. Throughput: 0: 9730.7, 1: 9628.5. Samples: 711274080. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:30:01,062][104569] Avg episode reward: [(0, '8169.677'), (1, '9085.168')] [2023-12-27 01:30:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001388080_355401728.pth... [2023-12-27 01:30:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001390072_355901440.pth... [2023-12-27 01:30:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001386960_355115008.pth [2023-12-27 01:30:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001388952_355614720.pth [2023-12-27 01:30:01,275][105620] Updated weights for policy 1, policy_version 1390080 (0.0008) [2023-12-27 01:30:01,333][105620] Updated weights for policy 1, policy_version 1390090 (0.0009) [2023-12-27 01:30:01,402][105620] Updated weights for policy 1, policy_version 1390100 (0.0009) [2023-12-27 01:30:01,428][105692] Updated weights for policy 0, policy_version 1388088 (0.0007) [2023-12-27 01:30:01,482][105692] Updated weights for policy 0, policy_version 1388098 (0.0007) [2023-12-27 01:30:01,540][105692] Updated weights for policy 0, policy_version 1388108 (0.0010) [2023-12-27 01:30:02,138][105620] Updated weights for policy 1, policy_version 1390110 (0.0009) [2023-12-27 01:30:02,198][105620] Updated weights for policy 1, policy_version 1390120 (0.0009) [2023-12-27 01:30:02,226][105692] Updated weights for policy 0, policy_version 1388118 (0.0007) [2023-12-27 01:30:02,258][105620] Updated weights for policy 1, policy_version 1390130 (0.0009) [2023-12-27 01:30:02,287][105692] Updated weights for policy 0, policy_version 1388128 (0.0008) [2023-12-27 01:30:02,343][105692] Updated weights for policy 0, policy_version 1388138 (0.0010) [2023-12-27 01:30:02,910][105692] Updated weights for policy 0, policy_version 1388148 (0.0007) [2023-12-27 01:30:02,955][105692] Updated weights for policy 0, policy_version 1388158 (0.0005) [2023-12-27 01:30:03,011][105692] Updated weights for policy 0, policy_version 1388168 (0.0005) [2023-12-27 01:30:03,142][105620] Updated weights for policy 1, policy_version 1390140 (0.0007) [2023-12-27 01:30:03,196][105620] Updated weights for policy 1, policy_version 1390150 (0.0007) [2023-12-27 01:30:03,247][105620] Updated weights for policy 1, policy_version 1390160 (0.0009) [2023-12-27 01:30:03,540][105692] Updated weights for policy 0, policy_version 1388178 (0.0005) [2023-12-27 01:30:03,605][105692] Updated weights for policy 0, policy_version 1388188 (0.0005) [2023-12-27 01:30:03,672][105692] Updated weights for policy 0, policy_version 1388198 (0.0007) [2023-12-27 01:30:03,732][105692] Updated weights for policy 0, policy_version 1388208 (0.0008) [2023-12-27 01:30:03,847][105620] Updated weights for policy 1, policy_version 1390170 (0.0010) [2023-12-27 01:30:03,901][105620] Updated weights for policy 1, policy_version 1390180 (0.0009) [2023-12-27 01:30:03,963][105620] Updated weights for policy 1, policy_version 1390190 (0.0006) [2023-12-27 01:30:04,022][105620] Updated weights for policy 1, policy_version 1390200 (0.0007) [2023-12-27 01:30:04,402][105692] Updated weights for policy 0, policy_version 1388218 (0.0008) [2023-12-27 01:30:04,463][105692] Updated weights for policy 0, policy_version 1388228 (0.0006) [2023-12-27 01:30:04,535][105692] Updated weights for policy 0, policy_version 1388238 (0.0008) [2023-12-27 01:30:04,608][105620] Updated weights for policy 1, policy_version 1390210 (0.0005) [2023-12-27 01:30:04,668][105620] Updated weights for policy 1, policy_version 1390220 (0.0007) [2023-12-27 01:30:04,720][105620] Updated weights for policy 1, policy_version 1390230 (0.0010) [2023-12-27 01:30:05,142][105692] Updated weights for policy 0, policy_version 1388248 (0.0005) [2023-12-27 01:30:05,199][105692] Updated weights for policy 0, policy_version 1388258 (0.0006) [2023-12-27 01:30:05,255][105692] Updated weights for policy 0, policy_version 1388268 (0.0005) [2023-12-27 01:30:05,351][105620] Updated weights for policy 1, policy_version 1390240 (0.0010) [2023-12-27 01:30:05,409][105620] Updated weights for policy 1, policy_version 1390250 (0.0010) [2023-12-27 01:30:05,464][105620] Updated weights for policy 1, policy_version 1390260 (0.0009) [2023-12-27 01:30:05,943][105692] Updated weights for policy 0, policy_version 1388278 (0.0009) [2023-12-27 01:30:05,990][105692] Updated weights for policy 0, policy_version 1388288 (0.0009) [2023-12-27 01:30:06,057][105692] Updated weights for policy 0, policy_version 1388298 (0.0009) [2023-12-27 01:30:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 711401472. Throughput: 0: 9854.2, 1: 9597.2. Samples: 711394740. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:30:06,063][104569] Avg episode reward: [(0, '8166.084'), (1, '5842.976')] [2023-12-27 01:30:06,156][105620] Updated weights for policy 1, policy_version 1390270 (0.0008) [2023-12-27 01:30:06,223][105620] Updated weights for policy 1, policy_version 1390280 (0.0010) [2023-12-27 01:30:06,285][105620] Updated weights for policy 1, policy_version 1390290 (0.0009) [2023-12-27 01:30:06,843][105692] Updated weights for policy 0, policy_version 1388308 (0.0009) [2023-12-27 01:30:06,897][105692] Updated weights for policy 0, policy_version 1388318 (0.0009) [2023-12-27 01:30:06,961][105692] Updated weights for policy 0, policy_version 1388328 (0.0008) [2023-12-27 01:30:06,966][105620] Updated weights for policy 1, policy_version 1390300 (0.0009) [2023-12-27 01:30:07,028][105620] Updated weights for policy 1, policy_version 1390310 (0.0006) [2023-12-27 01:30:07,094][105620] Updated weights for policy 1, policy_version 1390320 (0.0009) [2023-12-27 01:30:07,540][105692] Updated weights for policy 0, policy_version 1388338 (0.0007) [2023-12-27 01:30:07,599][105692] Updated weights for policy 0, policy_version 1388348 (0.0005) [2023-12-27 01:30:07,658][105692] Updated weights for policy 0, policy_version 1388358 (0.0005) [2023-12-27 01:30:07,716][105692] Updated weights for policy 0, policy_version 1388368 (0.0005) [2023-12-27 01:30:07,973][105620] Updated weights for policy 1, policy_version 1390330 (0.0009) [2023-12-27 01:30:08,037][105620] Updated weights for policy 1, policy_version 1390340 (0.0008) [2023-12-27 01:30:08,097][105620] Updated weights for policy 1, policy_version 1390350 (0.0008) [2023-12-27 01:30:08,157][105620] Updated weights for policy 1, policy_version 1390360 (0.0008) [2023-12-27 01:30:08,377][105692] Updated weights for policy 0, policy_version 1388378 (0.0009) [2023-12-27 01:30:08,438][105692] Updated weights for policy 0, policy_version 1388388 (0.0008) [2023-12-27 01:30:08,500][105692] Updated weights for policy 0, policy_version 1388398 (0.0009) [2023-12-27 01:30:08,915][105620] Updated weights for policy 1, policy_version 1390370 (0.0009) [2023-12-27 01:30:08,963][105620] Updated weights for policy 1, policy_version 1390380 (0.0009) [2023-12-27 01:30:09,024][105620] Updated weights for policy 1, policy_version 1390390 (0.0009) [2023-12-27 01:30:09,222][105692] Updated weights for policy 0, policy_version 1388408 (0.0010) [2023-12-27 01:30:09,279][105692] Updated weights for policy 0, policy_version 1388418 (0.0008) [2023-12-27 01:30:09,331][105692] Updated weights for policy 0, policy_version 1388428 (0.0008) [2023-12-27 01:30:09,805][105620] Updated weights for policy 1, policy_version 1390400 (0.0009) [2023-12-27 01:30:09,869][105620] Updated weights for policy 1, policy_version 1390410 (0.0009) [2023-12-27 01:30:09,929][105620] Updated weights for policy 1, policy_version 1390420 (0.0009) [2023-12-27 01:30:10,183][105692] Updated weights for policy 0, policy_version 1388438 (0.0008) [2023-12-27 01:30:10,249][105692] Updated weights for policy 0, policy_version 1388448 (0.0007) [2023-12-27 01:30:10,310][105692] Updated weights for policy 0, policy_version 1388458 (0.0009) [2023-12-27 01:30:10,766][105620] Updated weights for policy 1, policy_version 1390430 (0.0009) [2023-12-27 01:30:10,826][105620] Updated weights for policy 1, policy_version 1390440 (0.0009) [2023-12-27 01:30:10,882][105620] Updated weights for policy 1, policy_version 1390450 (0.0009) [2023-12-27 01:30:10,917][105692] Updated weights for policy 0, policy_version 1388468 (0.0008) [2023-12-27 01:30:10,975][105692] Updated weights for policy 0, policy_version 1388478 (0.0007) [2023-12-27 01:30:11,033][105692] Updated weights for policy 0, policy_version 1388488 (0.0007) [2023-12-27 01:30:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 711499776. Throughput: 0: 9922.9, 1: 9587.5. Samples: 711509208. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:30:11,062][104569] Avg episode reward: [(0, '8065.426'), (1, '3862.571')] [2023-12-27 01:30:11,754][105620] Updated weights for policy 1, policy_version 1390460 (0.0008) [2023-12-27 01:30:11,805][105620] Updated weights for policy 1, policy_version 1390470 (0.0009) [2023-12-27 01:30:11,819][105692] Updated weights for policy 0, policy_version 1388498 (0.0010) [2023-12-27 01:30:11,863][105620] Updated weights for policy 1, policy_version 1390480 (0.0007) [2023-12-27 01:30:11,881][105692] Updated weights for policy 0, policy_version 1388508 (0.0007) [2023-12-27 01:30:11,930][105692] Updated weights for policy 0, policy_version 1388518 (0.0008) [2023-12-27 01:30:11,983][105692] Updated weights for policy 0, policy_version 1388528 (0.0009) [2023-12-27 01:30:12,642][105620] Updated weights for policy 1, policy_version 1390490 (0.0008) [2023-12-27 01:30:12,689][105620] Updated weights for policy 1, policy_version 1390500 (0.0008) [2023-12-27 01:30:12,742][105620] Updated weights for policy 1, policy_version 1390510 (0.0008) [2023-12-27 01:30:12,781][105692] Updated weights for policy 0, policy_version 1388538 (0.0008) [2023-12-27 01:30:12,802][105620] Updated weights for policy 1, policy_version 1390520 (0.0007) [2023-12-27 01:30:12,834][105692] Updated weights for policy 0, policy_version 1388548 (0.0009) [2023-12-27 01:30:12,888][105692] Updated weights for policy 0, policy_version 1388558 (0.0008) [2023-12-27 01:30:13,566][105620] Updated weights for policy 1, policy_version 1390530 (0.0009) [2023-12-27 01:30:13,627][105620] Updated weights for policy 1, policy_version 1390540 (0.0009) [2023-12-27 01:30:13,661][105692] Updated weights for policy 0, policy_version 1388568 (0.0008) [2023-12-27 01:30:13,684][105620] Updated weights for policy 1, policy_version 1390550 (0.0007) [2023-12-27 01:30:13,723][105692] Updated weights for policy 0, policy_version 1388578 (0.0005) [2023-12-27 01:30:13,787][105692] Updated weights for policy 0, policy_version 1388588 (0.0008) [2023-12-27 01:30:14,449][105692] Updated weights for policy 0, policy_version 1388598 (0.0009) [2023-12-27 01:30:14,480][105620] Updated weights for policy 1, policy_version 1390560 (0.0008) [2023-12-27 01:30:14,502][105692] Updated weights for policy 0, policy_version 1388608 (0.0006) [2023-12-27 01:30:14,533][105620] Updated weights for policy 1, policy_version 1390570 (0.0009) [2023-12-27 01:30:14,561][105692] Updated weights for policy 0, policy_version 1388618 (0.0005) [2023-12-27 01:30:14,582][105620] Updated weights for policy 1, policy_version 1390580 (0.0008) [2023-12-27 01:30:15,177][105692] Updated weights for policy 0, policy_version 1388628 (0.0007) [2023-12-27 01:30:15,229][105692] Updated weights for policy 0, policy_version 1388638 (0.0009) [2023-12-27 01:30:15,292][105692] Updated weights for policy 0, policy_version 1388648 (0.0009) [2023-12-27 01:30:15,414][105620] Updated weights for policy 1, policy_version 1390590 (0.0009) [2023-12-27 01:30:15,463][105620] Updated weights for policy 1, policy_version 1390600 (0.0009) [2023-12-27 01:30:15,510][105620] Updated weights for policy 1, policy_version 1390610 (0.0009) [2023-12-27 01:30:16,047][105692] Updated weights for policy 0, policy_version 1388658 (0.0009) [2023-12-27 01:30:16,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 711589888. Throughput: 0: 9819.7, 1: 9601.7. Samples: 711564356. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:30:16,062][104569] Avg episode reward: [(0, '8253.813'), (1, '6110.980')] [2023-12-27 01:30:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001390616_356040704.pth... [2023-12-27 01:30:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001389528_355762176.pth [2023-12-27 01:30:16,118][105692] Updated weights for policy 0, policy_version 1388668 (0.0009) [2023-12-27 01:30:16,174][105692] Updated weights for policy 0, policy_version 1388678 (0.0008) [2023-12-27 01:30:16,227][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001388688_355557376.pth... [2023-12-27 01:30:16,230][105692] Updated weights for policy 0, policy_version 1388688 (0.0006) [2023-12-27 01:30:16,231][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001387504_355254272.pth [2023-12-27 01:30:16,295][105620] Updated weights for policy 1, policy_version 1390620 (0.0008) [2023-12-27 01:30:16,346][105620] Updated weights for policy 1, policy_version 1390630 (0.0009) [2023-12-27 01:30:16,393][105620] Updated weights for policy 1, policy_version 1390640 (0.0009) [2023-12-27 01:30:16,964][105692] Updated weights for policy 0, policy_version 1388698 (0.0010) [2023-12-27 01:30:17,016][105692] Updated weights for policy 0, policy_version 1388708 (0.0010) [2023-12-27 01:30:17,064][105692] Updated weights for policy 0, policy_version 1388718 (0.0010) [2023-12-27 01:30:17,193][105620] Updated weights for policy 1, policy_version 1390650 (0.0009) [2023-12-27 01:30:17,255][105620] Updated weights for policy 1, policy_version 1390660 (0.0010) [2023-12-27 01:30:17,322][105620] Updated weights for policy 1, policy_version 1390670 (0.0010) [2023-12-27 01:30:17,382][105620] Updated weights for policy 1, policy_version 1390680 (0.0009) [2023-12-27 01:30:17,694][105692] Updated weights for policy 0, policy_version 1388728 (0.0006) [2023-12-27 01:30:17,757][105692] Updated weights for policy 0, policy_version 1388738 (0.0005) [2023-12-27 01:30:17,814][105692] Updated weights for policy 0, policy_version 1388748 (0.0007) [2023-12-27 01:30:18,247][105620] Updated weights for policy 1, policy_version 1390690 (0.0010) [2023-12-27 01:30:18,301][105620] Updated weights for policy 1, policy_version 1390701 (0.0009) [2023-12-27 01:30:18,342][105692] Updated weights for policy 0, policy_version 1388758 (0.0007) [2023-12-27 01:30:18,364][105620] Updated weights for policy 1, policy_version 1390711 (0.0007) [2023-12-27 01:30:18,403][105692] Updated weights for policy 0, policy_version 1388768 (0.0007) [2023-12-27 01:30:18,456][105692] Updated weights for policy 0, policy_version 1388778 (0.0007) [2023-12-27 01:30:19,063][105620] Updated weights for policy 1, policy_version 1390721 (0.0006) [2023-12-27 01:30:19,132][105620] Updated weights for policy 1, policy_version 1390731 (0.0006) [2023-12-27 01:30:19,189][105620] Updated weights for policy 1, policy_version 1390741 (0.0006) [2023-12-27 01:30:19,260][105692] Updated weights for policy 0, policy_version 1388788 (0.0008) [2023-12-27 01:30:19,318][105692] Updated weights for policy 0, policy_version 1388798 (0.0009) [2023-12-27 01:30:19,392][105692] Updated weights for policy 0, policy_version 1388808 (0.0009) [2023-12-27 01:30:19,898][105620] Updated weights for policy 1, policy_version 1390751 (0.0008) [2023-12-27 01:30:19,956][105620] Updated weights for policy 1, policy_version 1390761 (0.0009) [2023-12-27 01:30:20,012][105620] Updated weights for policy 1, policy_version 1390771 (0.0008) [2023-12-27 01:30:20,157][105692] Updated weights for policy 0, policy_version 1388818 (0.0008) [2023-12-27 01:30:20,221][105692] Updated weights for policy 0, policy_version 1388828 (0.0006) [2023-12-27 01:30:20,284][105692] Updated weights for policy 0, policy_version 1388838 (0.0006) [2023-12-27 01:30:20,336][105692] Updated weights for policy 0, policy_version 1388848 (0.0006) [2023-12-27 01:30:20,807][105620] Updated weights for policy 1, policy_version 1390781 (0.0009) [2023-12-27 01:30:20,875][105620] Updated weights for policy 1, policy_version 1390791 (0.0009) [2023-12-27 01:30:20,934][105620] Updated weights for policy 1, policy_version 1390801 (0.0008) [2023-12-27 01:30:20,980][105692] Updated weights for policy 0, policy_version 1388858 (0.0009) [2023-12-27 01:30:21,043][105692] Updated weights for policy 0, policy_version 1388868 (0.0009) [2023-12-27 01:30:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 711688192. Throughput: 0: 9853.5, 1: 9471.9. Samples: 711678648. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:30:21,062][104569] Avg episode reward: [(0, '8895.075'), (1, '8188.044')] [2023-12-27 01:30:21,110][105692] Updated weights for policy 0, policy_version 1388878 (0.0007) [2023-12-27 01:30:21,672][105620] Updated weights for policy 1, policy_version 1390811 (0.0007) [2023-12-27 01:30:21,742][105620] Updated weights for policy 1, policy_version 1390821 (0.0009) [2023-12-27 01:30:21,806][105620] Updated weights for policy 1, policy_version 1390831 (0.0010) [2023-12-27 01:30:21,894][105692] Updated weights for policy 0, policy_version 1388888 (0.0009) [2023-12-27 01:30:21,959][105692] Updated weights for policy 0, policy_version 1388898 (0.0009) [2023-12-27 01:30:22,023][105692] Updated weights for policy 0, policy_version 1388908 (0.0010) [2023-12-27 01:30:22,551][105620] Updated weights for policy 1, policy_version 1390841 (0.0008) [2023-12-27 01:30:22,604][105620] Updated weights for policy 1, policy_version 1390851 (0.0009) [2023-12-27 01:30:22,659][105620] Updated weights for policy 1, policy_version 1390861 (0.0008) [2023-12-27 01:30:22,714][105620] Updated weights for policy 1, policy_version 1390871 (0.0009) [2023-12-27 01:30:22,753][105692] Updated weights for policy 0, policy_version 1388918 (0.0009) [2023-12-27 01:30:22,819][105692] Updated weights for policy 0, policy_version 1388928 (0.0010) [2023-12-27 01:30:22,891][105692] Updated weights for policy 0, policy_version 1388938 (0.0010) [2023-12-27 01:30:23,444][105620] Updated weights for policy 1, policy_version 1390881 (0.0009) [2023-12-27 01:30:23,511][105620] Updated weights for policy 1, policy_version 1390891 (0.0009) [2023-12-27 01:30:23,570][105620] Updated weights for policy 1, policy_version 1390901 (0.0011) [2023-12-27 01:30:23,642][105692] Updated weights for policy 0, policy_version 1388948 (0.0009) [2023-12-27 01:30:23,687][105692] Updated weights for policy 0, policy_version 1388958 (0.0008) [2023-12-27 01:30:23,736][105692] Updated weights for policy 0, policy_version 1388969 (0.0009) [2023-12-27 01:30:24,259][105620] Updated weights for policy 1, policy_version 1390911 (0.0007) [2023-12-27 01:30:24,315][105620] Updated weights for policy 1, policy_version 1390921 (0.0005) [2023-12-27 01:30:24,359][105620] Updated weights for policy 1, policy_version 1390931 (0.0009) [2023-12-27 01:30:24,530][105692] Updated weights for policy 0, policy_version 1388979 (0.0008) [2023-12-27 01:30:24,587][105692] Updated weights for policy 0, policy_version 1388989 (0.0009) [2023-12-27 01:30:24,647][105692] Updated weights for policy 0, policy_version 1388999 (0.0007) [2023-12-27 01:30:24,959][105620] Updated weights for policy 1, policy_version 1390941 (0.0008) [2023-12-27 01:30:25,016][105620] Updated weights for policy 1, policy_version 1390951 (0.0005) [2023-12-27 01:30:25,073][105620] Updated weights for policy 1, policy_version 1390961 (0.0005) [2023-12-27 01:30:25,378][105692] Updated weights for policy 0, policy_version 1389009 (0.0009) [2023-12-27 01:30:25,434][105692] Updated weights for policy 0, policy_version 1389019 (0.0010) [2023-12-27 01:30:25,488][105692] Updated weights for policy 0, policy_version 1389029 (0.0010) [2023-12-27 01:30:25,543][105692] Updated weights for policy 0, policy_version 1389039 (0.0009) [2023-12-27 01:30:25,667][105620] Updated weights for policy 1, policy_version 1390971 (0.0007) [2023-12-27 01:30:25,733][105620] Updated weights for policy 1, policy_version 1390981 (0.0011) [2023-12-27 01:30:25,799][105620] Updated weights for policy 1, policy_version 1390991 (0.0011) [2023-12-27 01:30:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 711786496. Throughput: 0: 9805.9, 1: 9597.6. Samples: 711793324. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:30:26,062][104569] Avg episode reward: [(0, '8619.426'), (1, '9000.366')] [2023-12-27 01:30:26,178][105692] Updated weights for policy 0, policy_version 1389049 (0.0007) [2023-12-27 01:30:26,230][105692] Updated weights for policy 0, policy_version 1389059 (0.0008) [2023-12-27 01:30:26,283][105692] Updated weights for policy 0, policy_version 1389069 (0.0008) [2023-12-27 01:30:26,533][105620] Updated weights for policy 1, policy_version 1391001 (0.0010) [2023-12-27 01:30:26,581][105620] Updated weights for policy 1, policy_version 1391011 (0.0010) [2023-12-27 01:30:26,632][105620] Updated weights for policy 1, policy_version 1391021 (0.0010) [2023-12-27 01:30:26,695][105620] Updated weights for policy 1, policy_version 1391031 (0.0011) [2023-12-27 01:30:26,958][105692] Updated weights for policy 0, policy_version 1389079 (0.0010) [2023-12-27 01:30:27,016][105692] Updated weights for policy 0, policy_version 1389089 (0.0010) [2023-12-27 01:30:27,075][105692] Updated weights for policy 0, policy_version 1389099 (0.0010) [2023-12-27 01:30:27,450][105620] Updated weights for policy 1, policy_version 1391041 (0.0011) [2023-12-27 01:30:27,507][105620] Updated weights for policy 1, policy_version 1391051 (0.0010) [2023-12-27 01:30:27,561][105620] Updated weights for policy 1, policy_version 1391061 (0.0010) [2023-12-27 01:30:27,749][105692] Updated weights for policy 0, policy_version 1389109 (0.0009) [2023-12-27 01:30:27,818][105692] Updated weights for policy 0, policy_version 1389119 (0.0009) [2023-12-27 01:30:27,884][105692] Updated weights for policy 0, policy_version 1389129 (0.0010) [2023-12-27 01:30:28,106][105620] Updated weights for policy 1, policy_version 1391071 (0.0005) [2023-12-27 01:30:28,152][105620] Updated weights for policy 1, policy_version 1391081 (0.0005) [2023-12-27 01:30:28,197][105620] Updated weights for policy 1, policy_version 1391091 (0.0005) [2023-12-27 01:30:28,742][105620] Updated weights for policy 1, policy_version 1391101 (0.0008) [2023-12-27 01:30:28,760][105692] Updated weights for policy 0, policy_version 1389139 (0.0008) [2023-12-27 01:30:28,802][105620] Updated weights for policy 1, policy_version 1391111 (0.0012) [2023-12-27 01:30:28,812][105692] Updated weights for policy 0, policy_version 1389149 (0.0008) [2023-12-27 01:30:28,854][105620] Updated weights for policy 1, policy_version 1391121 (0.0011) [2023-12-27 01:30:28,857][105692] Updated weights for policy 0, policy_version 1389159 (0.0005) [2023-12-27 01:30:29,560][105620] Updated weights for policy 1, policy_version 1391131 (0.0009) [2023-12-27 01:30:29,621][105692] Updated weights for policy 0, policy_version 1389169 (0.0008) [2023-12-27 01:30:29,621][105620] Updated weights for policy 1, policy_version 1391141 (0.0006) [2023-12-27 01:30:29,676][105620] Updated weights for policy 1, policy_version 1391151 (0.0011) [2023-12-27 01:30:29,680][105692] Updated weights for policy 0, policy_version 1389179 (0.0011) [2023-12-27 01:30:29,733][105692] Updated weights for policy 0, policy_version 1389189 (0.0010) [2023-12-27 01:30:29,786][105692] Updated weights for policy 0, policy_version 1389199 (0.0011) [2023-12-27 01:30:30,382][105620] Updated weights for policy 1, policy_version 1391161 (0.0009) [2023-12-27 01:30:30,429][105620] Updated weights for policy 1, policy_version 1391171 (0.0009) [2023-12-27 01:30:30,481][105620] Updated weights for policy 1, policy_version 1391181 (0.0009) [2023-12-27 01:30:30,531][105620] Updated weights for policy 1, policy_version 1391191 (0.0008) [2023-12-27 01:30:30,556][105692] Updated weights for policy 0, policy_version 1389209 (0.0006) [2023-12-27 01:30:30,618][105692] Updated weights for policy 0, policy_version 1389219 (0.0005) [2023-12-27 01:30:30,673][105692] Updated weights for policy 0, policy_version 1389229 (0.0005) [2023-12-27 01:30:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 711884800. Throughput: 0: 9842.2, 1: 9650.1. Samples: 711855472. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-12-27 01:30:31,062][104569] Avg episode reward: [(0, '8529.687'), (1, '8999.012')] [2023-12-27 01:30:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001389232_355696640.pth... [2023-12-27 01:30:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001391192_356188160.pth... [2023-12-27 01:30:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001388080_355401728.pth [2023-12-27 01:30:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001390072_355901440.pth [2023-12-27 01:30:31,378][105620] Updated weights for policy 1, policy_version 1391201 (0.0009) [2023-12-27 01:30:31,382][105692] Updated weights for policy 0, policy_version 1389239 (0.0007) [2023-12-27 01:30:31,429][105620] Updated weights for policy 1, policy_version 1391211 (0.0009) [2023-12-27 01:30:31,459][105692] Updated weights for policy 0, policy_version 1389249 (0.0007) [2023-12-27 01:30:31,481][105620] Updated weights for policy 1, policy_version 1391221 (0.0008) [2023-12-27 01:30:31,512][105692] Updated weights for policy 0, policy_version 1389259 (0.0008) [2023-12-27 01:30:32,223][105692] Updated weights for policy 0, policy_version 1389269 (0.0009) [2023-12-27 01:30:32,279][105692] Updated weights for policy 0, policy_version 1389279 (0.0009) [2023-12-27 01:30:32,289][105620] Updated weights for policy 1, policy_version 1391231 (0.0007) [2023-12-27 01:30:32,336][105692] Updated weights for policy 0, policy_version 1389289 (0.0007) [2023-12-27 01:30:32,355][105620] Updated weights for policy 1, policy_version 1391241 (0.0009) [2023-12-27 01:30:32,416][105620] Updated weights for policy 1, policy_version 1391251 (0.0008) [2023-12-27 01:30:33,099][105692] Updated weights for policy 0, policy_version 1389299 (0.0008) [2023-12-27 01:30:33,154][105692] Updated weights for policy 0, policy_version 1389309 (0.0005) [2023-12-27 01:30:33,195][105620] Updated weights for policy 1, policy_version 1391261 (0.0008) [2023-12-27 01:30:33,214][105692] Updated weights for policy 0, policy_version 1389319 (0.0005) [2023-12-27 01:30:33,265][105620] Updated weights for policy 1, policy_version 1391271 (0.0008) [2023-12-27 01:30:33,321][105620] Updated weights for policy 1, policy_version 1391281 (0.0010) [2023-12-27 01:30:33,749][105692] Updated weights for policy 0, policy_version 1389329 (0.0005) [2023-12-27 01:30:33,814][105692] Updated weights for policy 0, policy_version 1389339 (0.0005) [2023-12-27 01:30:33,879][105692] Updated weights for policy 0, policy_version 1389349 (0.0005) [2023-12-27 01:30:33,914][105620] Updated weights for policy 1, policy_version 1391292 (0.0008) [2023-12-27 01:30:33,932][105692] Updated weights for policy 0, policy_version 1389359 (0.0005) [2023-12-27 01:30:33,964][105620] Updated weights for policy 1, policy_version 1391302 (0.0005) [2023-12-27 01:30:34,015][105620] Updated weights for policy 1, policy_version 1391312 (0.0005) [2023-12-27 01:30:34,550][105692] Updated weights for policy 0, policy_version 1389369 (0.0010) [2023-12-27 01:30:34,582][105620] Updated weights for policy 1, policy_version 1391322 (0.0006) [2023-12-27 01:30:34,602][105692] Updated weights for policy 0, policy_version 1389379 (0.0010) [2023-12-27 01:30:34,637][105620] Updated weights for policy 1, policy_version 1391332 (0.0006) [2023-12-27 01:30:34,654][105692] Updated weights for policy 0, policy_version 1389389 (0.0010) [2023-12-27 01:30:34,693][105620] Updated weights for policy 1, policy_version 1391342 (0.0007) [2023-12-27 01:30:34,753][105620] Updated weights for policy 1, policy_version 1391352 (0.0008) [2023-12-27 01:30:35,414][105692] Updated weights for policy 0, policy_version 1389399 (0.0010) [2023-12-27 01:30:35,461][105620] Updated weights for policy 1, policy_version 1391362 (0.0005) [2023-12-27 01:30:35,475][105692] Updated weights for policy 0, policy_version 1389409 (0.0010) [2023-12-27 01:30:35,524][105620] Updated weights for policy 1, policy_version 1391372 (0.0005) [2023-12-27 01:30:35,532][105692] Updated weights for policy 0, policy_version 1389419 (0.0008) [2023-12-27 01:30:35,582][105620] Updated weights for policy 1, policy_version 1391382 (0.0006) [2023-12-27 01:30:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 711983104. Throughput: 0: 9854.9, 1: 9607.0. Samples: 711974060. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:30:36,062][104569] Avg episode reward: [(0, '8070.041'), (1, '8899.350')] [2023-12-27 01:30:36,134][105692] Updated weights for policy 0, policy_version 1389429 (0.0008) [2023-12-27 01:30:36,147][105620] Updated weights for policy 1, policy_version 1391392 (0.0007) [2023-12-27 01:30:36,188][105692] Updated weights for policy 0, policy_version 1389439 (0.0009) [2023-12-27 01:30:36,210][105620] Updated weights for policy 1, policy_version 1391402 (0.0006) [2023-12-27 01:30:36,245][105692] Updated weights for policy 0, policy_version 1389449 (0.0008) [2023-12-27 01:30:36,273][105620] Updated weights for policy 1, policy_version 1391412 (0.0010) [2023-12-27 01:30:36,891][105620] Updated weights for policy 1, policy_version 1391422 (0.0008) [2023-12-27 01:30:36,950][105620] Updated weights for policy 1, policy_version 1391432 (0.0011) [2023-12-27 01:30:37,009][105620] Updated weights for policy 1, policy_version 1391442 (0.0011) [2023-12-27 01:30:37,068][105692] Updated weights for policy 0, policy_version 1389459 (0.0008) [2023-12-27 01:30:37,129][105692] Updated weights for policy 0, policy_version 1389469 (0.0008) [2023-12-27 01:30:37,186][105692] Updated weights for policy 0, policy_version 1389479 (0.0008) [2023-12-27 01:30:37,752][105620] Updated weights for policy 1, policy_version 1391452 (0.0010) [2023-12-27 01:30:37,801][105620] Updated weights for policy 1, policy_version 1391462 (0.0010) [2023-12-27 01:30:37,849][105620] Updated weights for policy 1, policy_version 1391472 (0.0010) [2023-12-27 01:30:37,974][105692] Updated weights for policy 0, policy_version 1389489 (0.0008) [2023-12-27 01:30:38,027][105692] Updated weights for policy 0, policy_version 1389499 (0.0008) [2023-12-27 01:30:38,079][105692] Updated weights for policy 0, policy_version 1389509 (0.0008) [2023-12-27 01:30:38,123][105692] Updated weights for policy 0, policy_version 1389519 (0.0008) [2023-12-27 01:30:38,625][105620] Updated weights for policy 1, policy_version 1391482 (0.0010) [2023-12-27 01:30:38,674][105620] Updated weights for policy 1, policy_version 1391492 (0.0010) [2023-12-27 01:30:38,718][105620] Updated weights for policy 1, policy_version 1391502 (0.0010) [2023-12-27 01:30:38,766][105620] Updated weights for policy 1, policy_version 1391512 (0.0010) [2023-12-27 01:30:38,921][105692] Updated weights for policy 0, policy_version 1389529 (0.0008) [2023-12-27 01:30:38,965][105692] Updated weights for policy 0, policy_version 1389539 (0.0007) [2023-12-27 01:30:39,021][105692] Updated weights for policy 0, policy_version 1389549 (0.0008) [2023-12-27 01:30:39,524][105620] Updated weights for policy 1, policy_version 1391522 (0.0006) [2023-12-27 01:30:39,571][105620] Updated weights for policy 1, policy_version 1391532 (0.0005) [2023-12-27 01:30:39,632][105620] Updated weights for policy 1, policy_version 1391542 (0.0007) [2023-12-27 01:30:39,835][105692] Updated weights for policy 0, policy_version 1389559 (0.0009) [2023-12-27 01:30:39,899][105692] Updated weights for policy 0, policy_version 1389569 (0.0010) [2023-12-27 01:30:39,967][105692] Updated weights for policy 0, policy_version 1389579 (0.0009) [2023-12-27 01:30:40,353][105620] Updated weights for policy 1, policy_version 1391552 (0.0009) [2023-12-27 01:30:40,410][105620] Updated weights for policy 1, policy_version 1391562 (0.0010) [2023-12-27 01:30:40,474][105620] Updated weights for policy 1, policy_version 1391572 (0.0007) [2023-12-27 01:30:40,707][105692] Updated weights for policy 0, policy_version 1389589 (0.0007) [2023-12-27 01:30:40,772][105692] Updated weights for policy 0, policy_version 1389599 (0.0007) [2023-12-27 01:30:40,832][105692] Updated weights for policy 0, policy_version 1389609 (0.0006) [2023-12-27 01:30:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 712081408. Throughput: 0: 9792.8, 1: 9641.2. Samples: 712090380. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:30:41,062][104569] Avg episode reward: [(0, '7884.841'), (1, '8898.233')] [2023-12-27 01:30:41,092][105620] Updated weights for policy 1, policy_version 1391582 (0.0006) [2023-12-27 01:30:41,160][105620] Updated weights for policy 1, policy_version 1391592 (0.0009) [2023-12-27 01:30:41,219][105620] Updated weights for policy 1, policy_version 1391602 (0.0010) [2023-12-27 01:30:41,561][105692] Updated weights for policy 0, policy_version 1389619 (0.0008) [2023-12-27 01:30:41,627][105692] Updated weights for policy 0, policy_version 1389629 (0.0008) [2023-12-27 01:30:41,685][105692] Updated weights for policy 0, policy_version 1389639 (0.0006) [2023-12-27 01:30:42,006][105620] Updated weights for policy 1, policy_version 1391612 (0.0010) [2023-12-27 01:30:42,060][105620] Updated weights for policy 1, policy_version 1391622 (0.0006) [2023-12-27 01:30:42,111][105620] Updated weights for policy 1, policy_version 1391632 (0.0005) [2023-12-27 01:30:42,446][105692] Updated weights for policy 0, policy_version 1389649 (0.0008) [2023-12-27 01:30:42,497][105692] Updated weights for policy 0, policy_version 1389659 (0.0009) [2023-12-27 01:30:42,558][105692] Updated weights for policy 0, policy_version 1389669 (0.0009) [2023-12-27 01:30:42,618][105692] Updated weights for policy 0, policy_version 1389679 (0.0008) [2023-12-27 01:30:42,826][105620] Updated weights for policy 1, policy_version 1391642 (0.0006) [2023-12-27 01:30:42,885][105620] Updated weights for policy 1, policy_version 1391652 (0.0010) [2023-12-27 01:30:42,939][105620] Updated weights for policy 1, policy_version 1391663 (0.0010) [2023-12-27 01:30:43,179][105692] Updated weights for policy 0, policy_version 1389689 (0.0005) [2023-12-27 01:30:43,234][105692] Updated weights for policy 0, policy_version 1389699 (0.0005) [2023-12-27 01:30:43,290][105692] Updated weights for policy 0, policy_version 1389709 (0.0006) [2023-12-27 01:30:43,654][105620] Updated weights for policy 1, policy_version 1391673 (0.0010) [2023-12-27 01:30:43,722][105620] Updated weights for policy 1, policy_version 1391683 (0.0006) [2023-12-27 01:30:43,774][105620] Updated weights for policy 1, policy_version 1391693 (0.0008) [2023-12-27 01:30:43,819][105620] Updated weights for policy 1, policy_version 1391703 (0.0008) [2023-12-27 01:30:44,000][105692] Updated weights for policy 0, policy_version 1389719 (0.0009) [2023-12-27 01:30:44,062][105692] Updated weights for policy 0, policy_version 1389729 (0.0010) [2023-12-27 01:30:44,123][105692] Updated weights for policy 0, policy_version 1389739 (0.0010) [2023-12-27 01:30:44,483][105620] Updated weights for policy 1, policy_version 1391713 (0.0007) [2023-12-27 01:30:44,531][105620] Updated weights for policy 1, policy_version 1391723 (0.0005) [2023-12-27 01:30:44,576][105620] Updated weights for policy 1, policy_version 1391733 (0.0005) [2023-12-27 01:30:44,844][105692] Updated weights for policy 0, policy_version 1389749 (0.0009) [2023-12-27 01:30:44,910][105692] Updated weights for policy 0, policy_version 1389759 (0.0011) [2023-12-27 01:30:44,965][105692] Updated weights for policy 0, policy_version 1389769 (0.0010) [2023-12-27 01:30:45,261][105620] Updated weights for policy 1, policy_version 1391743 (0.0009) [2023-12-27 01:30:45,318][105620] Updated weights for policy 1, policy_version 1391753 (0.0011) [2023-12-27 01:30:45,374][105620] Updated weights for policy 1, policy_version 1391763 (0.0011) [2023-12-27 01:30:45,716][105692] Updated weights for policy 0, policy_version 1389779 (0.0008) [2023-12-27 01:30:45,768][105692] Updated weights for policy 0, policy_version 1389789 (0.0010) [2023-12-27 01:30:45,823][105692] Updated weights for policy 0, policy_version 1389799 (0.0010) [2023-12-27 01:30:46,062][104569] Fps is (10 sec: 19659.8, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 712179712. Throughput: 0: 9821.5, 1: 9616.2. Samples: 712148784. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:30:46,064][104569] Avg episode reward: [(0, '7793.295'), (1, '8713.203')] [2023-12-27 01:30:46,074][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001389808_355844096.pth... [2023-12-27 01:30:46,074][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001391768_356335616.pth... [2023-12-27 01:30:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001388688_355557376.pth [2023-12-27 01:30:46,085][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001390616_356040704.pth [2023-12-27 01:30:46,143][105620] Updated weights for policy 1, policy_version 1391773 (0.0010) [2023-12-27 01:30:46,202][105620] Updated weights for policy 1, policy_version 1391783 (0.0010) [2023-12-27 01:30:46,264][105620] Updated weights for policy 1, policy_version 1391793 (0.0010) [2023-12-27 01:30:46,582][105692] Updated weights for policy 0, policy_version 1389809 (0.0010) [2023-12-27 01:30:46,629][105692] Updated weights for policy 0, policy_version 1389819 (0.0010) [2023-12-27 01:30:46,681][105692] Updated weights for policy 0, policy_version 1389829 (0.0010) [2023-12-27 01:30:46,695][105585] KL-divergence is very high: 112.6245 [2023-12-27 01:30:46,736][105692] Updated weights for policy 0, policy_version 1389839 (0.0010) [2023-12-27 01:30:46,872][105620] Updated weights for policy 1, policy_version 1391803 (0.0009) [2023-12-27 01:30:46,932][105620] Updated weights for policy 1, policy_version 1391813 (0.0008) [2023-12-27 01:30:46,984][105620] Updated weights for policy 1, policy_version 1391823 (0.0007) [2023-12-27 01:30:47,440][105692] Updated weights for policy 0, policy_version 1389849 (0.0010) [2023-12-27 01:30:47,492][105692] Updated weights for policy 0, policy_version 1389859 (0.0010) [2023-12-27 01:30:47,544][105692] Updated weights for policy 0, policy_version 1389869 (0.0010) [2023-12-27 01:30:47,566][105620] Updated weights for policy 1, policy_version 1391833 (0.0005) [2023-12-27 01:30:47,633][105620] Updated weights for policy 1, policy_version 1391843 (0.0008) [2023-12-27 01:30:47,700][105620] Updated weights for policy 1, policy_version 1391853 (0.0008) [2023-12-27 01:30:47,764][105620] Updated weights for policy 1, policy_version 1391863 (0.0008) [2023-12-27 01:30:48,304][105692] Updated weights for policy 0, policy_version 1389879 (0.0010) [2023-12-27 01:30:48,374][105692] Updated weights for policy 0, policy_version 1389889 (0.0011) [2023-12-27 01:30:48,427][105692] Updated weights for policy 0, policy_version 1389899 (0.0008) [2023-12-27 01:30:48,429][105620] Updated weights for policy 1, policy_version 1391873 (0.0007) [2023-12-27 01:30:48,486][105620] Updated weights for policy 1, policy_version 1391883 (0.0008) [2023-12-27 01:30:48,545][105620] Updated weights for policy 1, policy_version 1391893 (0.0008) [2023-12-27 01:30:49,173][105692] Updated weights for policy 0, policy_version 1389909 (0.0009) [2023-12-27 01:30:49,230][105692] Updated weights for policy 0, policy_version 1389919 (0.0009) [2023-12-27 01:30:49,293][105692] Updated weights for policy 0, policy_version 1389929 (0.0007) [2023-12-27 01:30:49,295][105620] Updated weights for policy 1, policy_version 1391903 (0.0008) [2023-12-27 01:30:49,357][105620] Updated weights for policy 1, policy_version 1391913 (0.0006) [2023-12-27 01:30:49,423][105620] Updated weights for policy 1, policy_version 1391923 (0.0008) [2023-12-27 01:30:50,011][105692] Updated weights for policy 0, policy_version 1389939 (0.0007) [2023-12-27 01:30:50,078][105692] Updated weights for policy 0, policy_version 1389949 (0.0008) [2023-12-27 01:30:50,152][105692] Updated weights for policy 0, policy_version 1389959 (0.0008) [2023-12-27 01:30:50,193][105620] Updated weights for policy 1, policy_version 1391933 (0.0009) [2023-12-27 01:30:50,257][105620] Updated weights for policy 1, policy_version 1391943 (0.0009) [2023-12-27 01:30:50,316][105620] Updated weights for policy 1, policy_version 1391953 (0.0009) [2023-12-27 01:30:50,900][105692] Updated weights for policy 0, policy_version 1389969 (0.0008) [2023-12-27 01:30:50,948][105692] Updated weights for policy 0, policy_version 1389979 (0.0008) [2023-12-27 01:30:50,996][105692] Updated weights for policy 0, policy_version 1389989 (0.0008) [2023-12-27 01:30:51,024][105620] Updated weights for policy 1, policy_version 1391963 (0.0010) [2023-12-27 01:30:51,060][105692] Updated weights for policy 0, policy_version 1389999 (0.0008) [2023-12-27 01:30:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 712269824. Throughput: 0: 9678.9, 1: 9687.0. Samples: 712266208. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:30:51,063][104569] Avg episode reward: [(0, '8164.541'), (1, '8808.465')] [2023-12-27 01:30:51,084][105620] Updated weights for policy 1, policy_version 1391973 (0.0010) [2023-12-27 01:30:51,147][105620] Updated weights for policy 1, policy_version 1391983 (0.0009) [2023-12-27 01:30:51,863][105692] Updated weights for policy 0, policy_version 1390009 (0.0008) [2023-12-27 01:30:51,912][105692] Updated weights for policy 0, policy_version 1390019 (0.0009) [2023-12-27 01:30:51,933][105620] Updated weights for policy 1, policy_version 1391993 (0.0008) [2023-12-27 01:30:51,960][105692] Updated weights for policy 0, policy_version 1390029 (0.0008) [2023-12-27 01:30:51,996][105620] Updated weights for policy 1, policy_version 1392003 (0.0009) [2023-12-27 01:30:52,061][105620] Updated weights for policy 1, policy_version 1392013 (0.0009) [2023-12-27 01:30:52,123][105620] Updated weights for policy 1, policy_version 1392023 (0.0010) [2023-12-27 01:30:52,734][105692] Updated weights for policy 0, policy_version 1390039 (0.0008) [2023-12-27 01:30:52,788][105692] Updated weights for policy 0, policy_version 1390049 (0.0008) [2023-12-27 01:30:52,842][105692] Updated weights for policy 0, policy_version 1390059 (0.0008) [2023-12-27 01:30:52,855][105620] Updated weights for policy 1, policy_version 1392033 (0.0006) [2023-12-27 01:30:52,919][105620] Updated weights for policy 1, policy_version 1392043 (0.0008) [2023-12-27 01:30:52,974][105620] Updated weights for policy 1, policy_version 1392053 (0.0010) [2023-12-27 01:30:53,609][105620] Updated weights for policy 1, policy_version 1392063 (0.0006) [2023-12-27 01:30:53,678][105620] Updated weights for policy 1, policy_version 1392073 (0.0007) [2023-12-27 01:30:53,684][105692] Updated weights for policy 0, policy_version 1390069 (0.0007) [2023-12-27 01:30:53,734][105620] Updated weights for policy 1, policy_version 1392083 (0.0008) [2023-12-27 01:30:53,744][105692] Updated weights for policy 0, policy_version 1390079 (0.0006) [2023-12-27 01:30:53,799][105692] Updated weights for policy 0, policy_version 1390089 (0.0008) [2023-12-27 01:30:54,337][105620] Updated weights for policy 1, policy_version 1392093 (0.0008) [2023-12-27 01:30:54,410][105620] Updated weights for policy 1, policy_version 1392103 (0.0010) [2023-12-27 01:30:54,427][105692] Updated weights for policy 0, policy_version 1390099 (0.0007) [2023-12-27 01:30:54,459][105620] Updated weights for policy 1, policy_version 1392113 (0.0010) [2023-12-27 01:30:54,491][105692] Updated weights for policy 0, policy_version 1390109 (0.0005) [2023-12-27 01:30:54,552][105692] Updated weights for policy 0, policy_version 1390119 (0.0006) [2023-12-27 01:30:55,199][105620] Updated weights for policy 1, policy_version 1392123 (0.0009) [2023-12-27 01:30:55,252][105692] Updated weights for policy 0, policy_version 1390129 (0.0007) [2023-12-27 01:30:55,264][105620] Updated weights for policy 1, policy_version 1392133 (0.0006) [2023-12-27 01:30:55,312][105692] Updated weights for policy 0, policy_version 1390139 (0.0011) [2023-12-27 01:30:55,321][105620] Updated weights for policy 1, policy_version 1392143 (0.0005) [2023-12-27 01:30:55,358][105692] Updated weights for policy 0, policy_version 1390149 (0.0010) [2023-12-27 01:30:55,407][105692] Updated weights for policy 0, policy_version 1390159 (0.0006) [2023-12-27 01:30:55,907][105620] Updated weights for policy 1, policy_version 1392153 (0.0006) [2023-12-27 01:30:55,964][105620] Updated weights for policy 1, policy_version 1392163 (0.0010) [2023-12-27 01:30:56,022][105620] Updated weights for policy 1, policy_version 1392173 (0.0010) [2023-12-27 01:30:56,033][105692] Updated weights for policy 0, policy_version 1390169 (0.0010) [2023-12-27 01:30:56,062][104569] Fps is (10 sec: 18842.6, 60 sec: 19387.8, 300 sec: 19438.7). Total num frames: 712368128. Throughput: 0: 9609.8, 1: 9796.4. Samples: 712382484. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:30:56,062][104569] Avg episode reward: [(0, '8811.988'), (1, '9082.909')] [2023-12-27 01:30:56,081][105620] Updated weights for policy 1, policy_version 1392183 (0.0011) [2023-12-27 01:30:56,087][105692] Updated weights for policy 0, policy_version 1390179 (0.0011) [2023-12-27 01:30:56,147][105692] Updated weights for policy 0, policy_version 1390189 (0.0011) [2023-12-27 01:30:56,708][105692] Updated weights for policy 0, policy_version 1390199 (0.0006) [2023-12-27 01:30:56,774][105692] Updated weights for policy 0, policy_version 1390209 (0.0006) [2023-12-27 01:30:56,836][105692] Updated weights for policy 0, policy_version 1390219 (0.0008) [2023-12-27 01:30:56,844][105620] Updated weights for policy 1, policy_version 1392193 (0.0009) [2023-12-27 01:30:56,898][105620] Updated weights for policy 1, policy_version 1392203 (0.0009) [2023-12-27 01:30:56,955][105620] Updated weights for policy 1, policy_version 1392213 (0.0011) [2023-12-27 01:30:57,462][105692] Updated weights for policy 0, policy_version 1390229 (0.0009) [2023-12-27 01:30:57,521][105692] Updated weights for policy 0, policy_version 1390239 (0.0011) [2023-12-27 01:30:57,579][105692] Updated weights for policy 0, policy_version 1390249 (0.0010) [2023-12-27 01:30:57,662][105620] Updated weights for policy 1, policy_version 1392223 (0.0008) [2023-12-27 01:30:57,714][105620] Updated weights for policy 1, policy_version 1392233 (0.0008) [2023-12-27 01:30:57,773][105620] Updated weights for policy 1, policy_version 1392243 (0.0007) [2023-12-27 01:30:58,299][105692] Updated weights for policy 0, policy_version 1390259 (0.0010) [2023-12-27 01:30:58,370][105692] Updated weights for policy 0, policy_version 1390269 (0.0011) [2023-12-27 01:30:58,425][105692] Updated weights for policy 0, policy_version 1390279 (0.0010) [2023-12-27 01:30:58,567][105620] Updated weights for policy 1, policy_version 1392253 (0.0007) [2023-12-27 01:30:58,632][105620] Updated weights for policy 1, policy_version 1392263 (0.0008) [2023-12-27 01:30:58,701][105620] Updated weights for policy 1, policy_version 1392273 (0.0008) [2023-12-27 01:30:59,434][105692] Updated weights for policy 0, policy_version 1390289 (0.0008) [2023-12-27 01:30:59,496][105692] Updated weights for policy 0, policy_version 1390299 (0.0009) [2023-12-27 01:30:59,530][105620] Updated weights for policy 1, policy_version 1392283 (0.0007) [2023-12-27 01:30:59,559][105692] Updated weights for policy 0, policy_version 1390309 (0.0008) [2023-12-27 01:30:59,595][105620] Updated weights for policy 1, policy_version 1392293 (0.0009) [2023-12-27 01:30:59,617][105692] Updated weights for policy 0, policy_version 1390319 (0.0007) [2023-12-27 01:30:59,656][105620] Updated weights for policy 1, policy_version 1392303 (0.0009) [2023-12-27 01:31:00,315][105692] Updated weights for policy 0, policy_version 1390329 (0.0008) [2023-12-27 01:31:00,382][105692] Updated weights for policy 0, policy_version 1390339 (0.0009) [2023-12-27 01:31:00,394][105620] Updated weights for policy 1, policy_version 1392313 (0.0010) [2023-12-27 01:31:00,450][105692] Updated weights for policy 0, policy_version 1390349 (0.0009) [2023-12-27 01:31:00,455][105620] Updated weights for policy 1, policy_version 1392323 (0.0011) [2023-12-27 01:31:00,508][105620] Updated weights for policy 1, policy_version 1392333 (0.0011) [2023-12-27 01:31:00,570][105620] Updated weights for policy 1, policy_version 1392343 (0.0010) [2023-12-27 01:31:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 712466432. Throughput: 0: 9686.0, 1: 9802.5. Samples: 712441336. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:31:01,062][104569] Avg episode reward: [(0, '8258.496'), (1, '9356.079')] [2023-12-27 01:31:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001392344_356483072.pth... [2023-12-27 01:31:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001391192_356188160.pth [2023-12-27 01:31:01,120][105692] Updated weights for policy 0, policy_version 1390359 (0.0007) [2023-12-27 01:31:01,183][105585] KL-divergence is very high: 125.1717 [2023-12-27 01:31:01,184][105692] Updated weights for policy 0, policy_version 1390369 (0.0008) [2023-12-27 01:31:01,227][105585] KL-divergence is very high: 119.0536 [2023-12-27 01:31:01,241][105692] Updated weights for policy 0, policy_version 1390379 (0.0006) [2023-12-27 01:31:01,270][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001390384_355991552.pth... [2023-12-27 01:31:01,274][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001389232_355696640.pth [2023-12-27 01:31:01,378][105620] Updated weights for policy 1, policy_version 1392353 (0.0009) [2023-12-27 01:31:01,441][105620] Updated weights for policy 1, policy_version 1392363 (0.0007) [2023-12-27 01:31:01,504][105620] Updated weights for policy 1, policy_version 1392373 (0.0008) [2023-12-27 01:31:01,955][105585] KL-divergence is very high: 106.6965 [2023-12-27 01:31:01,973][105692] Updated weights for policy 0, policy_version 1390389 (0.0008) [2023-12-27 01:31:02,022][105692] Updated weights for policy 0, policy_version 1390399 (0.0008) [2023-12-27 01:31:02,074][105692] Updated weights for policy 0, policy_version 1390409 (0.0008) [2023-12-27 01:31:02,261][105620] Updated weights for policy 1, policy_version 1392383 (0.0009) [2023-12-27 01:31:02,326][105620] Updated weights for policy 1, policy_version 1392393 (0.0008) [2023-12-27 01:31:02,384][105620] Updated weights for policy 1, policy_version 1392403 (0.0008) [2023-12-27 01:31:02,797][105692] Updated weights for policy 0, policy_version 1390419 (0.0008) [2023-12-27 01:31:02,857][105692] Updated weights for policy 0, policy_version 1390429 (0.0008) [2023-12-27 01:31:02,906][105692] Updated weights for policy 0, policy_version 1390439 (0.0008) [2023-12-27 01:31:03,098][105620] Updated weights for policy 1, policy_version 1392413 (0.0010) [2023-12-27 01:31:03,147][105620] Updated weights for policy 1, policy_version 1392423 (0.0011) [2023-12-27 01:31:03,206][105620] Updated weights for policy 1, policy_version 1392433 (0.0010) [2023-12-27 01:31:03,703][105692] Updated weights for policy 0, policy_version 1390449 (0.0008) [2023-12-27 01:31:03,756][105692] Updated weights for policy 0, policy_version 1390459 (0.0008) [2023-12-27 01:31:03,811][105692] Updated weights for policy 0, policy_version 1390469 (0.0009) [2023-12-27 01:31:03,874][105692] Updated weights for policy 0, policy_version 1390479 (0.0009) [2023-12-27 01:31:03,950][105620] Updated weights for policy 1, policy_version 1392443 (0.0011) [2023-12-27 01:31:04,012][105620] Updated weights for policy 1, policy_version 1392453 (0.0009) [2023-12-27 01:31:04,077][105620] Updated weights for policy 1, policy_version 1392463 (0.0008) [2023-12-27 01:31:04,686][105692] Updated weights for policy 0, policy_version 1390489 (0.0009) [2023-12-27 01:31:04,736][105692] Updated weights for policy 0, policy_version 1390499 (0.0008) [2023-12-27 01:31:04,797][105692] Updated weights for policy 0, policy_version 1390509 (0.0009) [2023-12-27 01:31:04,848][105620] Updated weights for policy 1, policy_version 1392473 (0.0009) [2023-12-27 01:31:04,902][105620] Updated weights for policy 1, policy_version 1392483 (0.0011) [2023-12-27 01:31:04,947][105620] Updated weights for policy 1, policy_version 1392493 (0.0010) [2023-12-27 01:31:04,997][105620] Updated weights for policy 1, policy_version 1392503 (0.0010) [2023-12-27 01:31:05,398][105692] Updated weights for policy 0, policy_version 1390519 (0.0006) [2023-12-27 01:31:05,458][105692] Updated weights for policy 0, policy_version 1390529 (0.0005) [2023-12-27 01:31:05,525][105692] Updated weights for policy 0, policy_version 1390539 (0.0007) [2023-12-27 01:31:05,682][105620] Updated weights for policy 1, policy_version 1392513 (0.0010) [2023-12-27 01:31:05,738][105620] Updated weights for policy 1, policy_version 1392523 (0.0010) [2023-12-27 01:31:05,793][105620] Updated weights for policy 1, policy_version 1392533 (0.0009) [2023-12-27 01:31:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 712564736. Throughput: 0: 9570.8, 1: 9830.5. Samples: 712551704. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:31:06,062][104569] Avg episode reward: [(0, '7979.254'), (1, '9085.204')] [2023-12-27 01:31:06,111][105692] Updated weights for policy 0, policy_version 1390549 (0.0008) [2023-12-27 01:31:06,175][105692] Updated weights for policy 0, policy_version 1390559 (0.0007) [2023-12-27 01:31:06,241][105692] Updated weights for policy 0, policy_version 1390569 (0.0008) [2023-12-27 01:31:06,428][105620] Updated weights for policy 1, policy_version 1392543 (0.0009) [2023-12-27 01:31:06,483][105620] Updated weights for policy 1, policy_version 1392553 (0.0011) [2023-12-27 01:31:06,543][105620] Updated weights for policy 1, policy_version 1392563 (0.0010) [2023-12-27 01:31:06,996][105692] Updated weights for policy 0, policy_version 1390579 (0.0008) [2023-12-27 01:31:07,048][105692] Updated weights for policy 0, policy_version 1390589 (0.0009) [2023-12-27 01:31:07,107][105692] Updated weights for policy 0, policy_version 1390599 (0.0009) [2023-12-27 01:31:07,271][105620] Updated weights for policy 1, policy_version 1392573 (0.0010) [2023-12-27 01:31:07,326][105620] Updated weights for policy 1, policy_version 1392583 (0.0009) [2023-12-27 01:31:07,382][105620] Updated weights for policy 1, policy_version 1392593 (0.0009) [2023-12-27 01:31:07,870][105692] Updated weights for policy 0, policy_version 1390609 (0.0009) [2023-12-27 01:31:07,928][105692] Updated weights for policy 0, policy_version 1390619 (0.0009) [2023-12-27 01:31:07,975][105692] Updated weights for policy 0, policy_version 1390629 (0.0009) [2023-12-27 01:31:08,033][105692] Updated weights for policy 0, policy_version 1390639 (0.0009) [2023-12-27 01:31:08,121][105620] Updated weights for policy 1, policy_version 1392603 (0.0010) [2023-12-27 01:31:08,171][105620] Updated weights for policy 1, policy_version 1392613 (0.0009) [2023-12-27 01:31:08,218][105620] Updated weights for policy 1, policy_version 1392623 (0.0009) [2023-12-27 01:31:08,850][105620] Updated weights for policy 1, policy_version 1392633 (0.0008) [2023-12-27 01:31:08,875][105692] Updated weights for policy 0, policy_version 1390649 (0.0009) [2023-12-27 01:31:08,905][105620] Updated weights for policy 1, policy_version 1392643 (0.0007) [2023-12-27 01:31:08,927][105692] Updated weights for policy 0, policy_version 1390659 (0.0010) [2023-12-27 01:31:08,961][105620] Updated weights for policy 1, policy_version 1392653 (0.0008) [2023-12-27 01:31:08,976][105692] Updated weights for policy 0, policy_version 1390669 (0.0007) [2023-12-27 01:31:09,012][105620] Updated weights for policy 1, policy_version 1392663 (0.0008) [2023-12-27 01:31:09,770][105620] Updated weights for policy 1, policy_version 1392674 (0.0010) [2023-12-27 01:31:09,789][105692] Updated weights for policy 0, policy_version 1390679 (0.0006) [2023-12-27 01:31:09,837][105620] Updated weights for policy 1, policy_version 1392684 (0.0009) [2023-12-27 01:31:09,857][105692] Updated weights for policy 0, policy_version 1390689 (0.0008) [2023-12-27 01:31:09,896][105620] Updated weights for policy 1, policy_version 1392694 (0.0009) [2023-12-27 01:31:09,919][105692] Updated weights for policy 0, policy_version 1390699 (0.0008) [2023-12-27 01:31:10,589][105692] Updated weights for policy 0, policy_version 1390709 (0.0009) [2023-12-27 01:31:10,639][105692] Updated weights for policy 0, policy_version 1390719 (0.0008) [2023-12-27 01:31:10,680][105620] Updated weights for policy 1, policy_version 1392704 (0.0007) [2023-12-27 01:31:10,686][105692] Updated weights for policy 0, policy_version 1390729 (0.0007) [2023-12-27 01:31:10,733][105620] Updated weights for policy 1, policy_version 1392714 (0.0008) [2023-12-27 01:31:10,780][105620] Updated weights for policy 1, policy_version 1392724 (0.0008) [2023-12-27 01:31:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19438.7). Total num frames: 712663040. Throughput: 0: 9618.8, 1: 9839.9. Samples: 712668968. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:31:11,063][104569] Avg episode reward: [(0, '8075.064'), (1, '8808.359')] [2023-12-27 01:31:11,509][105692] Updated weights for policy 0, policy_version 1390739 (0.0007) [2023-12-27 01:31:11,540][105620] Updated weights for policy 1, policy_version 1392734 (0.0007) [2023-12-27 01:31:11,564][105692] Updated weights for policy 0, policy_version 1390749 (0.0007) [2023-12-27 01:31:11,598][105620] Updated weights for policy 1, policy_version 1392744 (0.0008) [2023-12-27 01:31:11,622][105692] Updated weights for policy 0, policy_version 1390759 (0.0006) [2023-12-27 01:31:11,668][105620] Updated weights for policy 1, policy_version 1392754 (0.0008) [2023-12-27 01:31:12,398][105692] Updated weights for policy 0, policy_version 1390769 (0.0008) [2023-12-27 01:31:12,455][105692] Updated weights for policy 0, policy_version 1390779 (0.0009) [2023-12-27 01:31:12,458][105620] Updated weights for policy 1, policy_version 1392764 (0.0008) [2023-12-27 01:31:12,513][105692] Updated weights for policy 0, policy_version 1390789 (0.0009) [2023-12-27 01:31:12,523][105620] Updated weights for policy 1, policy_version 1392774 (0.0009) [2023-12-27 01:31:12,566][105692] Updated weights for policy 0, policy_version 1390799 (0.0009) [2023-12-27 01:31:12,584][105620] Updated weights for policy 1, policy_version 1392784 (0.0009) [2023-12-27 01:31:13,323][105692] Updated weights for policy 0, policy_version 1390809 (0.0007) [2023-12-27 01:31:13,354][105620] Updated weights for policy 1, policy_version 1392794 (0.0009) [2023-12-27 01:31:13,390][105692] Updated weights for policy 0, policy_version 1390819 (0.0008) [2023-12-27 01:31:13,410][105620] Updated weights for policy 1, policy_version 1392804 (0.0009) [2023-12-27 01:31:13,451][105692] Updated weights for policy 0, policy_version 1390829 (0.0010) [2023-12-27 01:31:13,465][105620] Updated weights for policy 1, policy_version 1392814 (0.0006) [2023-12-27 01:31:13,520][105620] Updated weights for policy 1, policy_version 1392824 (0.0008) [2023-12-27 01:31:14,138][105692] Updated weights for policy 0, policy_version 1390839 (0.0009) [2023-12-27 01:31:14,199][105692] Updated weights for policy 0, policy_version 1390849 (0.0009) [2023-12-27 01:31:14,259][105692] Updated weights for policy 0, policy_version 1390859 (0.0006) [2023-12-27 01:31:14,270][105620] Updated weights for policy 1, policy_version 1392834 (0.0009) [2023-12-27 01:31:14,329][105620] Updated weights for policy 1, policy_version 1392844 (0.0007) [2023-12-27 01:31:14,396][105620] Updated weights for policy 1, policy_version 1392854 (0.0009) [2023-12-27 01:31:14,984][105692] Updated weights for policy 0, policy_version 1390869 (0.0007) [2023-12-27 01:31:15,051][105692] Updated weights for policy 0, policy_version 1390879 (0.0009) [2023-12-27 01:31:15,053][105620] Updated weights for policy 1, policy_version 1392864 (0.0007) [2023-12-27 01:31:15,106][105620] Updated weights for policy 1, policy_version 1392874 (0.0008) [2023-12-27 01:31:15,108][105692] Updated weights for policy 0, policy_version 1390889 (0.0006) [2023-12-27 01:31:15,165][105620] Updated weights for policy 1, policy_version 1392884 (0.0007) [2023-12-27 01:31:15,866][105620] Updated weights for policy 1, policy_version 1392894 (0.0008) [2023-12-27 01:31:15,889][105692] Updated weights for policy 0, policy_version 1390899 (0.0006) [2023-12-27 01:31:15,920][105620] Updated weights for policy 1, policy_version 1392904 (0.0008) [2023-12-27 01:31:15,939][105692] Updated weights for policy 0, policy_version 1390909 (0.0006) [2023-12-27 01:31:15,979][105620] Updated weights for policy 1, policy_version 1392914 (0.0008) [2023-12-27 01:31:15,990][105692] Updated weights for policy 0, policy_version 1390919 (0.0007) [2023-12-27 01:31:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 712761344. Throughput: 0: 9557.1, 1: 9738.0. Samples: 712723752. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:31:16,062][104569] Avg episode reward: [(0, '8531.186'), (1, '8652.204')] [2023-12-27 01:31:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001390928_356130816.pth... [2023-12-27 01:31:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001392920_356630528.pth... [2023-12-27 01:31:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001391768_356335616.pth [2023-12-27 01:31:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001389808_355844096.pth [2023-12-27 01:31:16,687][105692] Updated weights for policy 0, policy_version 1390929 (0.0009) [2023-12-27 01:31:16,746][105692] Updated weights for policy 0, policy_version 1390939 (0.0009) [2023-12-27 01:31:16,765][105620] Updated weights for policy 1, policy_version 1392924 (0.0007) [2023-12-27 01:31:16,803][105692] Updated weights for policy 0, policy_version 1390949 (0.0008) [2023-12-27 01:31:16,810][105620] Updated weights for policy 1, policy_version 1392934 (0.0006) [2023-12-27 01:31:16,858][105620] Updated weights for policy 1, policy_version 1392944 (0.0005) [2023-12-27 01:31:16,860][105692] Updated weights for policy 0, policy_version 1390959 (0.0007) [2023-12-27 01:31:17,497][105692] Updated weights for policy 0, policy_version 1390969 (0.0006) [2023-12-27 01:31:17,555][105692] Updated weights for policy 0, policy_version 1390979 (0.0005) [2023-12-27 01:31:17,615][105692] Updated weights for policy 0, policy_version 1390989 (0.0008) [2023-12-27 01:31:17,721][105620] Updated weights for policy 1, policy_version 1392954 (0.0008) [2023-12-27 01:31:17,780][105620] Updated weights for policy 1, policy_version 1392964 (0.0009) [2023-12-27 01:31:17,832][105620] Updated weights for policy 1, policy_version 1392974 (0.0005) [2023-12-27 01:31:17,897][105620] Updated weights for policy 1, policy_version 1392984 (0.0005) [2023-12-27 01:31:18,213][105692] Updated weights for policy 0, policy_version 1390999 (0.0007) [2023-12-27 01:31:18,259][105692] Updated weights for policy 0, policy_version 1391009 (0.0005) [2023-12-27 01:31:18,305][105692] Updated weights for policy 0, policy_version 1391019 (0.0005) [2023-12-27 01:31:18,478][105620] Updated weights for policy 1, policy_version 1392994 (0.0009) [2023-12-27 01:31:18,547][105620] Updated weights for policy 1, policy_version 1393004 (0.0009) [2023-12-27 01:31:18,572][105586] KL-divergence is very high: 106.2153 [2023-12-27 01:31:18,603][105620] Updated weights for policy 1, policy_version 1393014 (0.0009) [2023-12-27 01:31:19,021][105692] Updated weights for policy 0, policy_version 1391029 (0.0010) [2023-12-27 01:31:19,076][105692] Updated weights for policy 0, policy_version 1391039 (0.0010) [2023-12-27 01:31:19,134][105692] Updated weights for policy 0, policy_version 1391049 (0.0011) [2023-12-27 01:31:19,302][105586] KL-divergence is very high: 104.8633 [2023-12-27 01:31:19,302][105620] Updated weights for policy 1, policy_version 1393024 (0.0010) [2023-12-27 01:31:19,355][105586] KL-divergence is very high: 171.4556 [2023-12-27 01:31:19,369][105620] Updated weights for policy 1, policy_version 1393034 (0.0010) [2023-12-27 01:31:19,382][105586] KL-divergence is very high: 110.7831 [2023-12-27 01:31:19,405][105586] KL-divergence is very high: 186.8034 [2023-12-27 01:31:19,428][105620] Updated weights for policy 1, policy_version 1393044 (0.0010) [2023-12-27 01:31:19,430][105586] KL-divergence is very high: 108.2575 [2023-12-27 01:31:19,815][105692] Updated weights for policy 0, policy_version 1391059 (0.0009) [2023-12-27 01:31:19,881][105692] Updated weights for policy 0, policy_version 1391069 (0.0008) [2023-12-27 01:31:19,945][105692] Updated weights for policy 0, policy_version 1391079 (0.0008) [2023-12-27 01:31:20,198][105620] Updated weights for policy 1, policy_version 1393054 (0.0009) [2023-12-27 01:31:20,251][105620] Updated weights for policy 1, policy_version 1393064 (0.0008) [2023-12-27 01:31:20,302][105620] Updated weights for policy 1, policy_version 1393074 (0.0008) [2023-12-27 01:31:20,697][105692] Updated weights for policy 0, policy_version 1391089 (0.0007) [2023-12-27 01:31:20,758][105692] Updated weights for policy 0, policy_version 1391099 (0.0009) [2023-12-27 01:31:20,810][105692] Updated weights for policy 0, policy_version 1391109 (0.0009) [2023-12-27 01:31:20,872][105692] Updated weights for policy 0, policy_version 1391119 (0.0008) [2023-12-27 01:31:20,968][105620] Updated weights for policy 1, policy_version 1393084 (0.0005) [2023-12-27 01:31:21,017][105620] Updated weights for policy 1, policy_version 1393094 (0.0008) [2023-12-27 01:31:21,062][104569] Fps is (10 sec: 18840.8, 60 sec: 19387.6, 300 sec: 19410.8). Total num frames: 712851456. Throughput: 0: 9565.8, 1: 9713.9. Samples: 712841656. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:31:21,063][104569] Avg episode reward: [(0, '8346.822'), (1, '7178.257')] [2023-12-27 01:31:21,075][105620] Updated weights for policy 1, policy_version 1393104 (0.0008) [2023-12-27 01:31:21,709][105692] Updated weights for policy 0, policy_version 1391129 (0.0008) [2023-12-27 01:31:21,777][105692] Updated weights for policy 0, policy_version 1391139 (0.0009) [2023-12-27 01:31:21,831][105620] Updated weights for policy 1, policy_version 1393114 (0.0007) [2023-12-27 01:31:21,847][105692] Updated weights for policy 0, policy_version 1391149 (0.0009) [2023-12-27 01:31:21,887][105620] Updated weights for policy 1, policy_version 1393124 (0.0010) [2023-12-27 01:31:21,940][105620] Updated weights for policy 1, policy_version 1393134 (0.0010) [2023-12-27 01:31:22,002][105620] Updated weights for policy 1, policy_version 1393144 (0.0010) [2023-12-27 01:31:22,545][105692] Updated weights for policy 0, policy_version 1391159 (0.0007) [2023-12-27 01:31:22,609][105692] Updated weights for policy 0, policy_version 1391169 (0.0006) [2023-12-27 01:31:22,666][105692] Updated weights for policy 0, policy_version 1391179 (0.0006) [2023-12-27 01:31:22,738][105620] Updated weights for policy 1, policy_version 1393154 (0.0010) [2023-12-27 01:31:22,808][105620] Updated weights for policy 1, policy_version 1393164 (0.0009) [2023-12-27 01:31:22,871][105620] Updated weights for policy 1, policy_version 1393174 (0.0009) [2023-12-27 01:31:23,313][105692] Updated weights for policy 0, policy_version 1391189 (0.0007) [2023-12-27 01:31:23,367][105692] Updated weights for policy 0, policy_version 1391199 (0.0010) [2023-12-27 01:31:23,421][105692] Updated weights for policy 0, policy_version 1391209 (0.0010) [2023-12-27 01:31:23,576][105620] Updated weights for policy 1, policy_version 1393184 (0.0008) [2023-12-27 01:31:23,634][105620] Updated weights for policy 1, policy_version 1393194 (0.0009) [2023-12-27 01:31:23,696][105620] Updated weights for policy 1, policy_version 1393204 (0.0009) [2023-12-27 01:31:24,189][105692] Updated weights for policy 0, policy_version 1391219 (0.0009) [2023-12-27 01:31:24,246][105692] Updated weights for policy 0, policy_version 1391229 (0.0007) [2023-12-27 01:31:24,291][105692] Updated weights for policy 0, policy_version 1391239 (0.0008) [2023-12-27 01:31:24,479][105620] Updated weights for policy 1, policy_version 1393214 (0.0007) [2023-12-27 01:31:24,545][105620] Updated weights for policy 1, policy_version 1393224 (0.0005) [2023-12-27 01:31:24,597][105620] Updated weights for policy 1, policy_version 1393234 (0.0005) [2023-12-27 01:31:24,963][105692] Updated weights for policy 0, policy_version 1391249 (0.0008) [2023-12-27 01:31:25,028][105692] Updated weights for policy 0, policy_version 1391259 (0.0006) [2023-12-27 01:31:25,087][105692] Updated weights for policy 0, policy_version 1391269 (0.0005) [2023-12-27 01:31:25,147][105692] Updated weights for policy 0, policy_version 1391279 (0.0007) [2023-12-27 01:31:25,233][105620] Updated weights for policy 1, policy_version 1393244 (0.0006) [2023-12-27 01:31:25,283][105620] Updated weights for policy 1, policy_version 1393254 (0.0008) [2023-12-27 01:31:25,330][105620] Updated weights for policy 1, policy_version 1393264 (0.0007) [2023-12-27 01:31:25,805][105692] Updated weights for policy 0, policy_version 1391289 (0.0010) [2023-12-27 01:31:25,854][105692] Updated weights for policy 0, policy_version 1391299 (0.0011) [2023-12-27 01:31:25,910][105692] Updated weights for policy 0, policy_version 1391309 (0.0011) [2023-12-27 01:31:26,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 712949760. Throughput: 0: 9628.2, 1: 9656.3. Samples: 712958188. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:31:26,063][104569] Avg episode reward: [(0, '8159.346'), (1, '7859.170')] [2023-12-27 01:31:26,077][105620] Updated weights for policy 1, policy_version 1393274 (0.0008) [2023-12-27 01:31:26,134][105620] Updated weights for policy 1, policy_version 1393284 (0.0008) [2023-12-27 01:31:26,186][105620] Updated weights for policy 1, policy_version 1393294 (0.0008) [2023-12-27 01:31:26,231][105620] Updated weights for policy 1, policy_version 1393304 (0.0008) [2023-12-27 01:31:26,677][105692] Updated weights for policy 0, policy_version 1391319 (0.0010) [2023-12-27 01:31:26,734][105692] Updated weights for policy 0, policy_version 1391329 (0.0010) [2023-12-27 01:31:26,778][105585] KL-divergence is very high: 116.9752 [2023-12-27 01:31:26,795][105692] Updated weights for policy 0, policy_version 1391339 (0.0011) [2023-12-27 01:31:26,986][105620] Updated weights for policy 1, policy_version 1393314 (0.0008) [2023-12-27 01:31:27,032][105620] Updated weights for policy 1, policy_version 1393324 (0.0007) [2023-12-27 01:31:27,079][105620] Updated weights for policy 1, policy_version 1393334 (0.0008) [2023-12-27 01:31:27,509][105692] Updated weights for policy 0, policy_version 1391349 (0.0008) [2023-12-27 01:31:27,565][105692] Updated weights for policy 0, policy_version 1391359 (0.0008) [2023-12-27 01:31:27,612][105692] Updated weights for policy 0, policy_version 1391369 (0.0009) [2023-12-27 01:31:27,822][105620] Updated weights for policy 1, policy_version 1393344 (0.0009) [2023-12-27 01:31:27,885][105620] Updated weights for policy 1, policy_version 1393354 (0.0010) [2023-12-27 01:31:27,948][105620] Updated weights for policy 1, policy_version 1393364 (0.0010) [2023-12-27 01:31:28,242][105692] Updated weights for policy 0, policy_version 1391379 (0.0010) [2023-12-27 01:31:28,303][105692] Updated weights for policy 0, policy_version 1391389 (0.0010) [2023-12-27 01:31:28,366][105692] Updated weights for policy 0, policy_version 1391399 (0.0011) [2023-12-27 01:31:28,722][105620] Updated weights for policy 1, policy_version 1393375 (0.0009) [2023-12-27 01:31:28,789][105620] Updated weights for policy 1, policy_version 1393385 (0.0008) [2023-12-27 01:31:28,850][105620] Updated weights for policy 1, policy_version 1393395 (0.0009) [2023-12-27 01:31:29,023][105692] Updated weights for policy 0, policy_version 1391409 (0.0010) [2023-12-27 01:31:29,080][105692] Updated weights for policy 0, policy_version 1391419 (0.0005) [2023-12-27 01:31:29,133][105692] Updated weights for policy 0, policy_version 1391429 (0.0006) [2023-12-27 01:31:29,189][105692] Updated weights for policy 0, policy_version 1391440 (0.0010) [2023-12-27 01:31:29,615][105620] Updated weights for policy 1, policy_version 1393405 (0.0009) [2023-12-27 01:31:29,672][105620] Updated weights for policy 1, policy_version 1393415 (0.0008) [2023-12-27 01:31:29,722][105620] Updated weights for policy 1, policy_version 1393425 (0.0009) [2023-12-27 01:31:29,863][105692] Updated weights for policy 0, policy_version 1391450 (0.0009) [2023-12-27 01:31:29,910][105692] Updated weights for policy 0, policy_version 1391460 (0.0009) [2023-12-27 01:31:29,971][105692] Updated weights for policy 0, policy_version 1391470 (0.0008) [2023-12-27 01:31:30,476][105620] Updated weights for policy 1, policy_version 1393435 (0.0009) [2023-12-27 01:31:30,539][105620] Updated weights for policy 1, policy_version 1393445 (0.0010) [2023-12-27 01:31:30,594][105620] Updated weights for policy 1, policy_version 1393455 (0.0010) [2023-12-27 01:31:30,720][105692] Updated weights for policy 0, policy_version 1391480 (0.0006) [2023-12-27 01:31:30,777][105692] Updated weights for policy 0, policy_version 1391490 (0.0005) [2023-12-27 01:31:30,831][105692] Updated weights for policy 0, policy_version 1391500 (0.0006) [2023-12-27 01:31:31,062][104569] Fps is (10 sec: 19661.6, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 713048064. Throughput: 0: 9620.1, 1: 9649.7. Samples: 713015916. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:31:31,062][104569] Avg episode reward: [(0, '7887.497'), (1, '8895.275')] [2023-12-27 01:31:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001391504_356278272.pth... [2023-12-27 01:31:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001393464_356769792.pth... [2023-12-27 01:31:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001392344_356483072.pth [2023-12-27 01:31:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001390384_355991552.pth [2023-12-27 01:31:31,216][105620] Updated weights for policy 1, policy_version 1393465 (0.0010) [2023-12-27 01:31:31,273][105620] Updated weights for policy 1, policy_version 1393475 (0.0011) [2023-12-27 01:31:31,336][105620] Updated weights for policy 1, policy_version 1393485 (0.0010) [2023-12-27 01:31:31,400][105620] Updated weights for policy 1, policy_version 1393495 (0.0011) [2023-12-27 01:31:31,506][105692] Updated weights for policy 0, policy_version 1391510 (0.0005) [2023-12-27 01:31:31,563][105692] Updated weights for policy 0, policy_version 1391520 (0.0006) [2023-12-27 01:31:31,614][105692] Updated weights for policy 0, policy_version 1391530 (0.0008) [2023-12-27 01:31:32,049][105620] Updated weights for policy 1, policy_version 1393505 (0.0006) [2023-12-27 01:31:32,115][105620] Updated weights for policy 1, policy_version 1393515 (0.0005) [2023-12-27 01:31:32,182][105620] Updated weights for policy 1, policy_version 1393525 (0.0005) [2023-12-27 01:31:32,246][105692] Updated weights for policy 0, policy_version 1391540 (0.0011) [2023-12-27 01:31:32,309][105692] Updated weights for policy 0, policy_version 1391550 (0.0007) [2023-12-27 01:31:32,373][105692] Updated weights for policy 0, policy_version 1391560 (0.0009) [2023-12-27 01:31:32,827][105620] Updated weights for policy 1, policy_version 1393535 (0.0009) [2023-12-27 01:31:32,878][105620] Updated weights for policy 1, policy_version 1393545 (0.0010) [2023-12-27 01:31:32,924][105620] Updated weights for policy 1, policy_version 1393555 (0.0010) [2023-12-27 01:31:33,085][105692] Updated weights for policy 0, policy_version 1391570 (0.0010) [2023-12-27 01:31:33,139][105692] Updated weights for policy 0, policy_version 1391580 (0.0010) [2023-12-27 01:31:33,184][105692] Updated weights for policy 0, policy_version 1391590 (0.0010) [2023-12-27 01:31:33,228][105692] Updated weights for policy 0, policy_version 1391600 (0.0010) [2023-12-27 01:31:33,659][105620] Updated weights for policy 1, policy_version 1393565 (0.0008) [2023-12-27 01:31:33,722][105620] Updated weights for policy 1, policy_version 1393575 (0.0005) [2023-12-27 01:31:33,778][105620] Updated weights for policy 1, policy_version 1393585 (0.0009) [2023-12-27 01:31:33,976][105692] Updated weights for policy 0, policy_version 1391610 (0.0005) [2023-12-27 01:31:34,025][105692] Updated weights for policy 0, policy_version 1391620 (0.0005) [2023-12-27 01:31:34,078][105692] Updated weights for policy 0, policy_version 1391630 (0.0005) [2023-12-27 01:31:34,459][105620] Updated weights for policy 1, policy_version 1393595 (0.0010) [2023-12-27 01:31:34,521][105620] Updated weights for policy 1, policy_version 1393605 (0.0011) [2023-12-27 01:31:34,597][105620] Updated weights for policy 1, policy_version 1393615 (0.0011) [2023-12-27 01:31:34,694][105692] Updated weights for policy 0, policy_version 1391640 (0.0006) [2023-12-27 01:31:34,754][105692] Updated weights for policy 0, policy_version 1391650 (0.0010) [2023-12-27 01:31:34,806][105692] Updated weights for policy 0, policy_version 1391660 (0.0011) [2023-12-27 01:31:35,326][105620] Updated weights for policy 1, policy_version 1393625 (0.0010) [2023-12-27 01:31:35,377][105620] Updated weights for policy 1, policy_version 1393635 (0.0010) [2023-12-27 01:31:35,425][105620] Updated weights for policy 1, policy_version 1393645 (0.0010) [2023-12-27 01:31:35,479][105620] Updated weights for policy 1, policy_version 1393655 (0.0010) [2023-12-27 01:31:35,519][105692] Updated weights for policy 0, policy_version 1391670 (0.0009) [2023-12-27 01:31:35,572][105692] Updated weights for policy 0, policy_version 1391680 (0.0008) [2023-12-27 01:31:35,624][105692] Updated weights for policy 0, policy_version 1391690 (0.0008) [2023-12-27 01:31:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 713146368. Throughput: 0: 9732.5, 1: 9625.9. Samples: 713137332. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:31:36,062][104569] Avg episode reward: [(0, '8163.864'), (1, '8624.518')] [2023-12-27 01:31:36,221][105620] Updated weights for policy 1, policy_version 1393665 (0.0008) [2023-12-27 01:31:36,277][105620] Updated weights for policy 1, policy_version 1393675 (0.0008) [2023-12-27 01:31:36,335][105620] Updated weights for policy 1, policy_version 1393685 (0.0008) [2023-12-27 01:31:36,412][105692] Updated weights for policy 0, policy_version 1391700 (0.0008) [2023-12-27 01:31:36,475][105692] Updated weights for policy 0, policy_version 1391710 (0.0009) [2023-12-27 01:31:36,534][105692] Updated weights for policy 0, policy_version 1391720 (0.0009) [2023-12-27 01:31:37,153][105692] Updated weights for policy 0, policy_version 1391730 (0.0007) [2023-12-27 01:31:37,156][105620] Updated weights for policy 1, policy_version 1393695 (0.0007) [2023-12-27 01:31:37,203][105692] Updated weights for policy 0, policy_version 1391740 (0.0008) [2023-12-27 01:31:37,205][105620] Updated weights for policy 1, policy_version 1393705 (0.0006) [2023-12-27 01:31:37,254][105692] Updated weights for policy 0, policy_version 1391750 (0.0007) [2023-12-27 01:31:37,256][105620] Updated weights for policy 1, policy_version 1393715 (0.0006) [2023-12-27 01:31:37,306][105692] Updated weights for policy 0, policy_version 1391760 (0.0009) [2023-12-27 01:31:37,888][105620] Updated weights for policy 1, policy_version 1393725 (0.0007) [2023-12-27 01:31:37,941][105620] Updated weights for policy 1, policy_version 1393735 (0.0005) [2023-12-27 01:31:38,004][105620] Updated weights for policy 1, policy_version 1393745 (0.0006) [2023-12-27 01:31:38,125][105692] Updated weights for policy 0, policy_version 1391770 (0.0005) [2023-12-27 01:31:38,176][105692] Updated weights for policy 0, policy_version 1391780 (0.0005) [2023-12-27 01:31:38,224][105692] Updated weights for policy 0, policy_version 1391790 (0.0005) [2023-12-27 01:31:38,722][105620] Updated weights for policy 1, policy_version 1393755 (0.0010) [2023-12-27 01:31:38,788][105620] Updated weights for policy 1, policy_version 1393765 (0.0010) [2023-12-27 01:31:38,816][105692] Updated weights for policy 0, policy_version 1391800 (0.0005) [2023-12-27 01:31:38,847][105620] Updated weights for policy 1, policy_version 1393775 (0.0008) [2023-12-27 01:31:38,862][105692] Updated weights for policy 0, policy_version 1391810 (0.0005) [2023-12-27 01:31:38,924][105692] Updated weights for policy 0, policy_version 1391820 (0.0007) [2023-12-27 01:31:39,645][105620] Updated weights for policy 1, policy_version 1393785 (0.0008) [2023-12-27 01:31:39,666][105692] Updated weights for policy 0, policy_version 1391830 (0.0009) [2023-12-27 01:31:39,708][105620] Updated weights for policy 1, policy_version 1393795 (0.0006) [2023-12-27 01:31:39,726][105692] Updated weights for policy 0, policy_version 1391840 (0.0008) [2023-12-27 01:31:39,756][105620] Updated weights for policy 1, policy_version 1393805 (0.0008) [2023-12-27 01:31:39,790][105692] Updated weights for policy 0, policy_version 1391850 (0.0007) [2023-12-27 01:31:39,811][105620] Updated weights for policy 1, policy_version 1393815 (0.0006) [2023-12-27 01:31:40,473][105692] Updated weights for policy 0, policy_version 1391860 (0.0006) [2023-12-27 01:31:40,531][105692] Updated weights for policy 0, policy_version 1391870 (0.0007) [2023-12-27 01:31:40,576][105620] Updated weights for policy 1, policy_version 1393825 (0.0006) [2023-12-27 01:31:40,589][105692] Updated weights for policy 0, policy_version 1391880 (0.0009) [2023-12-27 01:31:40,638][105620] Updated weights for policy 1, policy_version 1393835 (0.0005) [2023-12-27 01:31:40,697][105620] Updated weights for policy 1, policy_version 1393845 (0.0005) [2023-12-27 01:31:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 713244672. Throughput: 0: 9792.2, 1: 9564.4. Samples: 713253532. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:31:41,062][104569] Avg episode reward: [(0, '8271.968'), (1, '8808.364')] [2023-12-27 01:31:41,366][105692] Updated weights for policy 0, policy_version 1391890 (0.0009) [2023-12-27 01:31:41,428][105692] Updated weights for policy 0, policy_version 1391900 (0.0008) [2023-12-27 01:31:41,439][105620] Updated weights for policy 1, policy_version 1393855 (0.0008) [2023-12-27 01:31:41,482][105692] Updated weights for policy 0, policy_version 1391910 (0.0007) [2023-12-27 01:31:41,504][105620] Updated weights for policy 1, policy_version 1393865 (0.0006) [2023-12-27 01:31:41,533][105692] Updated weights for policy 0, policy_version 1391920 (0.0006) [2023-12-27 01:31:41,571][105620] Updated weights for policy 1, policy_version 1393875 (0.0009) [2023-12-27 01:31:42,177][105692] Updated weights for policy 0, policy_version 1391930 (0.0008) [2023-12-27 01:31:42,240][105692] Updated weights for policy 0, policy_version 1391940 (0.0007) [2023-12-27 01:31:42,300][105692] Updated weights for policy 0, policy_version 1391950 (0.0008) [2023-12-27 01:31:42,470][105620] Updated weights for policy 1, policy_version 1393885 (0.0008) [2023-12-27 01:31:42,532][105620] Updated weights for policy 1, policy_version 1393895 (0.0010) [2023-12-27 01:31:42,582][105620] Updated weights for policy 1, policy_version 1393905 (0.0010) [2023-12-27 01:31:43,043][105692] Updated weights for policy 0, policy_version 1391960 (0.0008) [2023-12-27 01:31:43,096][105692] Updated weights for policy 0, policy_version 1391970 (0.0008) [2023-12-27 01:31:43,152][105692] Updated weights for policy 0, policy_version 1391980 (0.0008) [2023-12-27 01:31:43,336][105620] Updated weights for policy 1, policy_version 1393915 (0.0008) [2023-12-27 01:31:43,395][105620] Updated weights for policy 1, policy_version 1393925 (0.0011) [2023-12-27 01:31:43,454][105620] Updated weights for policy 1, policy_version 1393935 (0.0011) [2023-12-27 01:31:43,930][105692] Updated weights for policy 0, policy_version 1391990 (0.0008) [2023-12-27 01:31:43,985][105692] Updated weights for policy 0, policy_version 1392000 (0.0008) [2023-12-27 01:31:44,040][105692] Updated weights for policy 0, policy_version 1392010 (0.0008) [2023-12-27 01:31:44,195][105620] Updated weights for policy 1, policy_version 1393945 (0.0010) [2023-12-27 01:31:44,256][105620] Updated weights for policy 1, policy_version 1393955 (0.0010) [2023-12-27 01:31:44,317][105620] Updated weights for policy 1, policy_version 1393965 (0.0011) [2023-12-27 01:31:44,376][105620] Updated weights for policy 1, policy_version 1393975 (0.0010) [2023-12-27 01:31:44,802][105692] Updated weights for policy 0, policy_version 1392020 (0.0008) [2023-12-27 01:31:44,862][105692] Updated weights for policy 0, policy_version 1392030 (0.0008) [2023-12-27 01:31:44,919][105692] Updated weights for policy 0, policy_version 1392040 (0.0008) [2023-12-27 01:31:45,131][105620] Updated weights for policy 1, policy_version 1393985 (0.0011) [2023-12-27 01:31:45,191][105620] Updated weights for policy 1, policy_version 1393995 (0.0011) [2023-12-27 01:31:45,251][105620] Updated weights for policy 1, policy_version 1394005 (0.0011) [2023-12-27 01:31:45,697][105692] Updated weights for policy 0, policy_version 1392050 (0.0008) [2023-12-27 01:31:45,760][105692] Updated weights for policy 0, policy_version 1392060 (0.0008) [2023-12-27 01:31:45,819][105692] Updated weights for policy 0, policy_version 1392070 (0.0008) [2023-12-27 01:31:45,863][105692] Updated weights for policy 0, policy_version 1392080 (0.0008) [2023-12-27 01:31:46,002][105620] Updated weights for policy 1, policy_version 1394015 (0.0011) [2023-12-27 01:31:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.4, 300 sec: 19383.1). Total num frames: 713334784. Throughput: 0: 9727.2, 1: 9551.4. Samples: 713308872. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:31:46,062][104569] Avg episode reward: [(0, '8363.348'), (1, '8808.883')] [2023-12-27 01:31:46,063][105620] Updated weights for policy 1, policy_version 1394025 (0.0010) [2023-12-27 01:31:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001392080_356425728.pth... [2023-12-27 01:31:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001390928_356130816.pth [2023-12-27 01:31:46,129][105620] Updated weights for policy 1, policy_version 1394035 (0.0011) [2023-12-27 01:31:46,159][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001394040_356917248.pth... [2023-12-27 01:31:46,163][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001392920_356630528.pth [2023-12-27 01:31:46,638][105692] Updated weights for policy 0, policy_version 1392090 (0.0008) [2023-12-27 01:31:46,683][105692] Updated weights for policy 0, policy_version 1392100 (0.0008) [2023-12-27 01:31:46,731][105692] Updated weights for policy 0, policy_version 1392110 (0.0007) [2023-12-27 01:31:46,851][105620] Updated weights for policy 1, policy_version 1394045 (0.0011) [2023-12-27 01:31:46,899][105620] Updated weights for policy 1, policy_version 1394055 (0.0010) [2023-12-27 01:31:46,951][105620] Updated weights for policy 1, policy_version 1394065 (0.0010) [2023-12-27 01:31:47,506][105692] Updated weights for policy 0, policy_version 1392120 (0.0008) [2023-12-27 01:31:47,555][105692] Updated weights for policy 0, policy_version 1392130 (0.0007) [2023-12-27 01:31:47,600][105692] Updated weights for policy 0, policy_version 1392140 (0.0008) [2023-12-27 01:31:47,733][105620] Updated weights for policy 1, policy_version 1394075 (0.0010) [2023-12-27 01:31:47,798][105620] Updated weights for policy 1, policy_version 1394085 (0.0010) [2023-12-27 01:31:47,862][105620] Updated weights for policy 1, policy_version 1394095 (0.0010) [2023-12-27 01:31:48,395][105692] Updated weights for policy 0, policy_version 1392150 (0.0008) [2023-12-27 01:31:48,459][105692] Updated weights for policy 0, policy_version 1392160 (0.0008) [2023-12-27 01:31:48,511][105692] Updated weights for policy 0, policy_version 1392170 (0.0008) [2023-12-27 01:31:48,583][105620] Updated weights for policy 1, policy_version 1394105 (0.0010) [2023-12-27 01:31:48,642][105620] Updated weights for policy 1, policy_version 1394115 (0.0010) [2023-12-27 01:31:48,694][105620] Updated weights for policy 1, policy_version 1394125 (0.0010) [2023-12-27 01:31:48,746][105620] Updated weights for policy 1, policy_version 1394135 (0.0010) [2023-12-27 01:31:49,275][105692] Updated weights for policy 0, policy_version 1392180 (0.0008) [2023-12-27 01:31:49,339][105692] Updated weights for policy 0, policy_version 1392190 (0.0008) [2023-12-27 01:31:49,396][105692] Updated weights for policy 0, policy_version 1392200 (0.0008) [2023-12-27 01:31:49,548][105620] Updated weights for policy 1, policy_version 1394145 (0.0010) [2023-12-27 01:31:49,611][105620] Updated weights for policy 1, policy_version 1394155 (0.0010) [2023-12-27 01:31:49,663][105620] Updated weights for policy 1, policy_version 1394165 (0.0010) [2023-12-27 01:31:50,175][105692] Updated weights for policy 0, policy_version 1392210 (0.0008) [2023-12-27 01:31:50,228][105692] Updated weights for policy 0, policy_version 1392220 (0.0009) [2023-12-27 01:31:50,280][105692] Updated weights for policy 0, policy_version 1392230 (0.0008) [2023-12-27 01:31:50,329][105692] Updated weights for policy 0, policy_version 1392240 (0.0008) [2023-12-27 01:31:50,406][105620] Updated weights for policy 1, policy_version 1394175 (0.0010) [2023-12-27 01:31:50,457][105620] Updated weights for policy 1, policy_version 1394185 (0.0010) [2023-12-27 01:31:50,506][105620] Updated weights for policy 1, policy_version 1394195 (0.0010) [2023-12-27 01:31:51,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19251.2, 300 sec: 19299.8). Total num frames: 713424896. Throughput: 0: 9719.1, 1: 9556.9. Samples: 713419124. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:31:51,062][104569] Avg episode reward: [(0, '8433.048'), (1, '8899.005')] [2023-12-27 01:31:51,130][105692] Updated weights for policy 0, policy_version 1392250 (0.0008) [2023-12-27 01:31:51,191][105692] Updated weights for policy 0, policy_version 1392260 (0.0008) [2023-12-27 01:31:51,236][105692] Updated weights for policy 0, policy_version 1392270 (0.0007) [2023-12-27 01:31:51,294][105620] Updated weights for policy 1, policy_version 1394205 (0.0011) [2023-12-27 01:31:51,359][105620] Updated weights for policy 1, policy_version 1394215 (0.0011) [2023-12-27 01:31:51,423][105620] Updated weights for policy 1, policy_version 1394225 (0.0011) [2023-12-27 01:31:52,086][105692] Updated weights for policy 0, policy_version 1392280 (0.0009) [2023-12-27 01:31:52,088][105620] Updated weights for policy 1, policy_version 1394235 (0.0011) [2023-12-27 01:31:52,141][105692] Updated weights for policy 0, policy_version 1392290 (0.0006) [2023-12-27 01:31:52,154][105620] Updated weights for policy 1, policy_version 1394245 (0.0011) [2023-12-27 01:31:52,188][105692] Updated weights for policy 0, policy_version 1392300 (0.0008) [2023-12-27 01:31:52,217][105620] Updated weights for policy 1, policy_version 1394255 (0.0010) [2023-12-27 01:31:52,913][105692] Updated weights for policy 0, policy_version 1392310 (0.0007) [2023-12-27 01:31:52,964][105620] Updated weights for policy 1, policy_version 1394265 (0.0011) [2023-12-27 01:31:52,967][105692] Updated weights for policy 0, policy_version 1392320 (0.0006) [2023-12-27 01:31:53,020][105620] Updated weights for policy 1, policy_version 1394275 (0.0010) [2023-12-27 01:31:53,024][105692] Updated weights for policy 0, policy_version 1392330 (0.0006) [2023-12-27 01:31:53,082][105620] Updated weights for policy 1, policy_version 1394285 (0.0011) [2023-12-27 01:31:53,150][105620] Updated weights for policy 1, policy_version 1394295 (0.0010) [2023-12-27 01:31:53,752][105692] Updated weights for policy 0, policy_version 1392340 (0.0006) [2023-12-27 01:31:53,765][105620] Updated weights for policy 1, policy_version 1394305 (0.0010) [2023-12-27 01:31:53,812][105692] Updated weights for policy 0, policy_version 1392350 (0.0006) [2023-12-27 01:31:53,818][105620] Updated weights for policy 1, policy_version 1394315 (0.0011) [2023-12-27 01:31:53,867][105620] Updated weights for policy 1, policy_version 1394325 (0.0010) [2023-12-27 01:31:53,869][105692] Updated weights for policy 0, policy_version 1392360 (0.0006) [2023-12-27 01:31:54,588][105620] Updated weights for policy 1, policy_version 1394335 (0.0010) [2023-12-27 01:31:54,651][105620] Updated weights for policy 1, policy_version 1394345 (0.0007) [2023-12-27 01:31:54,676][105692] Updated weights for policy 0, policy_version 1392370 (0.0008) [2023-12-27 01:31:54,713][105620] Updated weights for policy 1, policy_version 1394355 (0.0007) [2023-12-27 01:31:54,727][105692] Updated weights for policy 0, policy_version 1392380 (0.0007) [2023-12-27 01:31:54,779][105692] Updated weights for policy 0, policy_version 1392390 (0.0005) [2023-12-27 01:31:54,835][105692] Updated weights for policy 0, policy_version 1392400 (0.0005) [2023-12-27 01:31:55,329][105620] Updated weights for policy 1, policy_version 1394365 (0.0006) [2023-12-27 01:31:55,394][105620] Updated weights for policy 1, policy_version 1394375 (0.0005) [2023-12-27 01:31:55,453][105620] Updated weights for policy 1, policy_version 1394385 (0.0005) [2023-12-27 01:31:55,556][105692] Updated weights for policy 0, policy_version 1392410 (0.0005) [2023-12-27 01:31:55,610][105692] Updated weights for policy 0, policy_version 1392420 (0.0005) [2023-12-27 01:31:55,667][105692] Updated weights for policy 0, policy_version 1392430 (0.0007) [2023-12-27 01:31:56,045][105620] Updated weights for policy 1, policy_version 1394395 (0.0008) [2023-12-27 01:31:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19299.8). Total num frames: 713523200. Throughput: 0: 9675.5, 1: 9583.0. Samples: 713535600. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:31:56,063][104569] Avg episode reward: [(0, '8435.616'), (1, '9080.271')] [2023-12-27 01:31:56,100][105620] Updated weights for policy 1, policy_version 1394405 (0.0009) [2023-12-27 01:31:56,156][105620] Updated weights for policy 1, policy_version 1394415 (0.0009) [2023-12-27 01:31:56,397][105692] Updated weights for policy 0, policy_version 1392440 (0.0010) [2023-12-27 01:31:56,450][105692] Updated weights for policy 0, policy_version 1392450 (0.0010) [2023-12-27 01:31:56,507][105692] Updated weights for policy 0, policy_version 1392460 (0.0013) [2023-12-27 01:31:56,761][105620] Updated weights for policy 1, policy_version 1394425 (0.0007) [2023-12-27 01:31:56,819][105620] Updated weights for policy 1, policy_version 1394435 (0.0010) [2023-12-27 01:31:56,875][105620] Updated weights for policy 1, policy_version 1394445 (0.0007) [2023-12-27 01:31:56,925][105620] Updated weights for policy 1, policy_version 1394455 (0.0007) [2023-12-27 01:31:57,357][105692] Updated weights for policy 0, policy_version 1392470 (0.0007) [2023-12-27 01:31:57,423][105692] Updated weights for policy 0, policy_version 1392480 (0.0006) [2023-12-27 01:31:57,490][105692] Updated weights for policy 0, policy_version 1392490 (0.0005) [2023-12-27 01:31:57,577][105620] Updated weights for policy 1, policy_version 1394465 (0.0010) [2023-12-27 01:31:57,646][105620] Updated weights for policy 1, policy_version 1394475 (0.0010) [2023-12-27 01:31:57,710][105620] Updated weights for policy 1, policy_version 1394485 (0.0010) [2023-12-27 01:31:58,074][105692] Updated weights for policy 0, policy_version 1392500 (0.0006) [2023-12-27 01:31:58,124][105692] Updated weights for policy 0, policy_version 1392510 (0.0005) [2023-12-27 01:31:58,182][105692] Updated weights for policy 0, policy_version 1392520 (0.0007) [2023-12-27 01:31:58,461][105620] Updated weights for policy 1, policy_version 1394495 (0.0008) [2023-12-27 01:31:58,524][105620] Updated weights for policy 1, policy_version 1394505 (0.0009) [2023-12-27 01:31:58,591][105620] Updated weights for policy 1, policy_version 1394515 (0.0009) [2023-12-27 01:31:58,915][105692] Updated weights for policy 0, policy_version 1392530 (0.0008) [2023-12-27 01:31:58,975][105692] Updated weights for policy 0, policy_version 1392540 (0.0009) [2023-12-27 01:31:59,039][105692] Updated weights for policy 0, policy_version 1392550 (0.0009) [2023-12-27 01:31:59,087][105692] Updated weights for policy 0, policy_version 1392560 (0.0009) [2023-12-27 01:31:59,364][105620] Updated weights for policy 1, policy_version 1394525 (0.0008) [2023-12-27 01:31:59,425][105620] Updated weights for policy 1, policy_version 1394535 (0.0006) [2023-12-27 01:31:59,486][105620] Updated weights for policy 1, policy_version 1394545 (0.0007) [2023-12-27 01:31:59,801][105692] Updated weights for policy 0, policy_version 1392570 (0.0009) [2023-12-27 01:31:59,863][105692] Updated weights for policy 0, policy_version 1392580 (0.0009) [2023-12-27 01:31:59,918][105692] Updated weights for policy 0, policy_version 1392590 (0.0009) [2023-12-27 01:32:00,089][105620] Updated weights for policy 1, policy_version 1394555 (0.0008) [2023-12-27 01:32:00,143][105620] Updated weights for policy 1, policy_version 1394565 (0.0009) [2023-12-27 01:32:00,205][105620] Updated weights for policy 1, policy_version 1394575 (0.0010) [2023-12-27 01:32:00,597][105692] Updated weights for policy 0, policy_version 1392600 (0.0006) [2023-12-27 01:32:00,659][105692] Updated weights for policy 0, policy_version 1392610 (0.0007) [2023-12-27 01:32:00,712][105692] Updated weights for policy 0, policy_version 1392620 (0.0009) [2023-12-27 01:32:00,926][105620] Updated weights for policy 1, policy_version 1394585 (0.0008) [2023-12-27 01:32:00,977][105620] Updated weights for policy 1, policy_version 1394595 (0.0010) [2023-12-27 01:32:01,037][105620] Updated weights for policy 1, policy_version 1394605 (0.0010) [2023-12-27 01:32:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19327.6). Total num frames: 713621504. Throughput: 0: 9703.9, 1: 9626.3. Samples: 713593612. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:32:01,063][104569] Avg episode reward: [(0, '8438.189'), (1, '9082.466')] [2023-12-27 01:32:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001392624_356564992.pth... [2023-12-27 01:32:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001391504_356278272.pth [2023-12-27 01:32:01,100][105620] Updated weights for policy 1, policy_version 1394615 (0.0010) [2023-12-27 01:32:01,105][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001394616_357064704.pth... [2023-12-27 01:32:01,109][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001393464_356769792.pth [2023-12-27 01:32:01,421][105692] Updated weights for policy 0, policy_version 1392630 (0.0010) [2023-12-27 01:32:01,485][105692] Updated weights for policy 0, policy_version 1392640 (0.0010) [2023-12-27 01:32:01,543][105692] Updated weights for policy 0, policy_version 1392650 (0.0009) [2023-12-27 01:32:01,797][105620] Updated weights for policy 1, policy_version 1394625 (0.0008) [2023-12-27 01:32:01,860][105620] Updated weights for policy 1, policy_version 1394635 (0.0006) [2023-12-27 01:32:01,926][105620] Updated weights for policy 1, policy_version 1394645 (0.0007) [2023-12-27 01:32:02,331][105692] Updated weights for policy 0, policy_version 1392660 (0.0009) [2023-12-27 01:32:02,396][105692] Updated weights for policy 0, policy_version 1392670 (0.0009) [2023-12-27 01:32:02,454][105692] Updated weights for policy 0, policy_version 1392680 (0.0009) [2023-12-27 01:32:02,594][105620] Updated weights for policy 1, policy_version 1394655 (0.0007) [2023-12-27 01:32:02,656][105620] Updated weights for policy 1, policy_version 1394665 (0.0005) [2023-12-27 01:32:02,718][105620] Updated weights for policy 1, policy_version 1394675 (0.0007) [2023-12-27 01:32:03,174][105692] Updated weights for policy 0, policy_version 1392690 (0.0009) [2023-12-27 01:32:03,238][105692] Updated weights for policy 0, policy_version 1392700 (0.0006) [2023-12-27 01:32:03,293][105692] Updated weights for policy 0, policy_version 1392710 (0.0011) [2023-12-27 01:32:03,313][105620] Updated weights for policy 1, policy_version 1394685 (0.0009) [2023-12-27 01:32:03,351][105692] Updated weights for policy 0, policy_version 1392720 (0.0010) [2023-12-27 01:32:03,375][105620] Updated weights for policy 1, policy_version 1394695 (0.0007) [2023-12-27 01:32:03,435][105620] Updated weights for policy 1, policy_version 1394705 (0.0007) [2023-12-27 01:32:03,980][105692] Updated weights for policy 0, policy_version 1392730 (0.0006) [2023-12-27 01:32:04,040][105692] Updated weights for policy 0, policy_version 1392740 (0.0005) [2023-12-27 01:32:04,047][105620] Updated weights for policy 1, policy_version 1394715 (0.0009) [2023-12-27 01:32:04,107][105692] Updated weights for policy 0, policy_version 1392750 (0.0006) [2023-12-27 01:32:04,110][105620] Updated weights for policy 1, policy_version 1394725 (0.0009) [2023-12-27 01:32:04,172][105620] Updated weights for policy 1, policy_version 1394735 (0.0010) [2023-12-27 01:32:04,822][105692] Updated weights for policy 0, policy_version 1392760 (0.0009) [2023-12-27 01:32:04,854][105620] Updated weights for policy 1, policy_version 1394745 (0.0010) [2023-12-27 01:32:04,877][105692] Updated weights for policy 0, policy_version 1392770 (0.0008) [2023-12-27 01:32:04,899][105620] Updated weights for policy 1, policy_version 1394755 (0.0010) [2023-12-27 01:32:04,935][105692] Updated weights for policy 0, policy_version 1392780 (0.0006) [2023-12-27 01:32:04,944][105620] Updated weights for policy 1, policy_version 1394765 (0.0010) [2023-12-27 01:32:04,999][105620] Updated weights for policy 1, policy_version 1394775 (0.0010) [2023-12-27 01:32:05,610][105620] Updated weights for policy 1, policy_version 1394785 (0.0006) [2023-12-27 01:32:05,670][105620] Updated weights for policy 1, policy_version 1394795 (0.0005) [2023-12-27 01:32:05,731][105620] Updated weights for policy 1, policy_version 1394805 (0.0005) [2023-12-27 01:32:05,789][105692] Updated weights for policy 0, policy_version 1392790 (0.0008) [2023-12-27 01:32:05,846][105692] Updated weights for policy 0, policy_version 1392801 (0.0010) [2023-12-27 01:32:05,899][105692] Updated weights for policy 0, policy_version 1392812 (0.0009) [2023-12-27 01:32:06,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19387.6, 300 sec: 19355.3). Total num frames: 713728000. Throughput: 0: 9673.9, 1: 9717.7. Samples: 713714272. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:32:06,063][104569] Avg episode reward: [(0, '8441.336'), (1, '9081.329')] [2023-12-27 01:32:06,353][105620] Updated weights for policy 1, policy_version 1394815 (0.0008) [2023-12-27 01:32:06,400][105620] Updated weights for policy 1, policy_version 1394825 (0.0008) [2023-12-27 01:32:06,452][105620] Updated weights for policy 1, policy_version 1394835 (0.0009) [2023-12-27 01:32:06,721][105692] Updated weights for policy 0, policy_version 1392822 (0.0009) [2023-12-27 01:32:06,781][105692] Updated weights for policy 0, policy_version 1392832 (0.0009) [2023-12-27 01:32:06,832][105692] Updated weights for policy 0, policy_version 1392842 (0.0009) [2023-12-27 01:32:07,207][105620] Updated weights for policy 1, policy_version 1394845 (0.0009) [2023-12-27 01:32:07,265][105620] Updated weights for policy 1, policy_version 1394855 (0.0009) [2023-12-27 01:32:07,326][105620] Updated weights for policy 1, policy_version 1394865 (0.0008) [2023-12-27 01:32:07,619][105692] Updated weights for policy 0, policy_version 1392852 (0.0010) [2023-12-27 01:32:07,675][105692] Updated weights for policy 0, policy_version 1392862 (0.0009) [2023-12-27 01:32:07,738][105692] Updated weights for policy 0, policy_version 1392872 (0.0009) [2023-12-27 01:32:08,046][105620] Updated weights for policy 1, policy_version 1394875 (0.0008) [2023-12-27 01:32:08,097][105620] Updated weights for policy 1, policy_version 1394885 (0.0005) [2023-12-27 01:32:08,146][105620] Updated weights for policy 1, policy_version 1394895 (0.0005) [2023-12-27 01:32:08,556][105692] Updated weights for policy 0, policy_version 1392882 (0.0009) [2023-12-27 01:32:08,614][105692] Updated weights for policy 0, policy_version 1392892 (0.0009) [2023-12-27 01:32:08,663][105692] Updated weights for policy 0, policy_version 1392902 (0.0009) [2023-12-27 01:32:08,721][105692] Updated weights for policy 0, policy_version 1392912 (0.0007) [2023-12-27 01:32:08,791][105620] Updated weights for policy 1, policy_version 1394905 (0.0006) [2023-12-27 01:32:08,855][105620] Updated weights for policy 1, policy_version 1394915 (0.0008) [2023-12-27 01:32:08,919][105620] Updated weights for policy 1, policy_version 1394925 (0.0006) [2023-12-27 01:32:08,986][105620] Updated weights for policy 1, policy_version 1394935 (0.0006) [2023-12-27 01:32:09,527][105692] Updated weights for policy 0, policy_version 1392922 (0.0010) [2023-12-27 01:32:09,586][105692] Updated weights for policy 0, policy_version 1392932 (0.0009) [2023-12-27 01:32:09,644][105692] Updated weights for policy 0, policy_version 1392942 (0.0006) [2023-12-27 01:32:09,658][105620] Updated weights for policy 1, policy_version 1394945 (0.0009) [2023-12-27 01:32:09,721][105620] Updated weights for policy 1, policy_version 1394955 (0.0009) [2023-12-27 01:32:09,783][105620] Updated weights for policy 1, policy_version 1394965 (0.0009) [2023-12-27 01:32:10,453][105692] Updated weights for policy 0, policy_version 1392952 (0.0007) [2023-12-27 01:32:10,475][105620] Updated weights for policy 1, policy_version 1394975 (0.0009) [2023-12-27 01:32:10,502][105692] Updated weights for policy 0, policy_version 1392962 (0.0005) [2023-12-27 01:32:10,532][105620] Updated weights for policy 1, policy_version 1394985 (0.0008) [2023-12-27 01:32:10,558][105692] Updated weights for policy 0, policy_version 1392972 (0.0006) [2023-12-27 01:32:10,582][105620] Updated weights for policy 1, policy_version 1394995 (0.0006) [2023-12-27 01:32:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19355.3). Total num frames: 713818112. Throughput: 0: 9539.6, 1: 9783.0. Samples: 713827708. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:32:11,063][104569] Avg episode reward: [(0, '8621.896'), (1, '8895.435')] [2023-12-27 01:32:11,297][105692] Updated weights for policy 0, policy_version 1392982 (0.0008) [2023-12-27 01:32:11,354][105692] Updated weights for policy 0, policy_version 1392992 (0.0011) [2023-12-27 01:32:11,357][105620] Updated weights for policy 1, policy_version 1395005 (0.0008) [2023-12-27 01:32:11,418][105620] Updated weights for policy 1, policy_version 1395015 (0.0007) [2023-12-27 01:32:11,421][105692] Updated weights for policy 0, policy_version 1393002 (0.0010) [2023-12-27 01:32:11,469][105620] Updated weights for policy 1, policy_version 1395025 (0.0007) [2023-12-27 01:32:12,189][105620] Updated weights for policy 1, policy_version 1395035 (0.0008) [2023-12-27 01:32:12,243][105692] Updated weights for policy 0, policy_version 1393012 (0.0009) [2023-12-27 01:32:12,250][105620] Updated weights for policy 1, policy_version 1395045 (0.0008) [2023-12-27 01:32:12,304][105692] Updated weights for policy 0, policy_version 1393022 (0.0007) [2023-12-27 01:32:12,318][105620] Updated weights for policy 1, policy_version 1395055 (0.0008) [2023-12-27 01:32:12,365][105692] Updated weights for policy 0, policy_version 1393032 (0.0007) [2023-12-27 01:32:13,017][105692] Updated weights for policy 0, policy_version 1393042 (0.0005) [2023-12-27 01:32:13,073][105692] Updated weights for policy 0, policy_version 1393052 (0.0005) [2023-12-27 01:32:13,131][105692] Updated weights for policy 0, policy_version 1393062 (0.0007) [2023-12-27 01:32:13,157][105620] Updated weights for policy 1, policy_version 1395065 (0.0007) [2023-12-27 01:32:13,179][105692] Updated weights for policy 0, policy_version 1393072 (0.0008) [2023-12-27 01:32:13,220][105620] Updated weights for policy 1, policy_version 1395075 (0.0009) [2023-12-27 01:32:13,290][105620] Updated weights for policy 1, policy_version 1395085 (0.0010) [2023-12-27 01:32:13,351][105620] Updated weights for policy 1, policy_version 1395095 (0.0009) [2023-12-27 01:32:13,832][105692] Updated weights for policy 0, policy_version 1393082 (0.0009) [2023-12-27 01:32:13,879][105692] Updated weights for policy 0, policy_version 1393092 (0.0008) [2023-12-27 01:32:13,932][105692] Updated weights for policy 0, policy_version 1393102 (0.0008) [2023-12-27 01:32:14,100][105620] Updated weights for policy 1, policy_version 1395105 (0.0009) [2023-12-27 01:32:14,151][105620] Updated weights for policy 1, policy_version 1395115 (0.0009) [2023-12-27 01:32:14,211][105620] Updated weights for policy 1, policy_version 1395125 (0.0009) [2023-12-27 01:32:14,721][105692] Updated weights for policy 0, policy_version 1393112 (0.0008) [2023-12-27 01:32:14,773][105692] Updated weights for policy 0, policy_version 1393122 (0.0008) [2023-12-27 01:32:14,835][105692] Updated weights for policy 0, policy_version 1393132 (0.0008) [2023-12-27 01:32:14,978][105620] Updated weights for policy 1, policy_version 1395135 (0.0008) [2023-12-27 01:32:15,037][105620] Updated weights for policy 1, policy_version 1395145 (0.0009) [2023-12-27 01:32:15,099][105620] Updated weights for policy 1, policy_version 1395155 (0.0009) [2023-12-27 01:32:15,589][105692] Updated weights for policy 0, policy_version 1393142 (0.0009) [2023-12-27 01:32:15,640][105692] Updated weights for policy 0, policy_version 1393152 (0.0009) [2023-12-27 01:32:15,691][105692] Updated weights for policy 0, policy_version 1393162 (0.0009) [2023-12-27 01:32:15,856][105620] Updated weights for policy 1, policy_version 1395165 (0.0009) [2023-12-27 01:32:15,917][105620] Updated weights for policy 1, policy_version 1395175 (0.0009) [2023-12-27 01:32:15,963][105620] Updated weights for policy 1, policy_version 1395185 (0.0008) [2023-12-27 01:32:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.1, 300 sec: 19383.1). Total num frames: 713916416. Throughput: 0: 9534.8, 1: 9758.0. Samples: 713884096. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:32:16,063][104569] Avg episode reward: [(0, '8799.893'), (1, '8802.786')] [2023-12-27 01:32:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001393168_356704256.pth... [2023-12-27 01:32:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001395192_357212160.pth... [2023-12-27 01:32:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001392080_356425728.pth [2023-12-27 01:32:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001394040_356917248.pth [2023-12-27 01:32:16,422][105692] Updated weights for policy 0, policy_version 1393172 (0.0007) [2023-12-27 01:32:16,489][105692] Updated weights for policy 0, policy_version 1393182 (0.0005) [2023-12-27 01:32:16,544][105692] Updated weights for policy 0, policy_version 1393192 (0.0005) [2023-12-27 01:32:16,769][105620] Updated weights for policy 1, policy_version 1395195 (0.0009) [2023-12-27 01:32:16,827][105620] Updated weights for policy 1, policy_version 1395205 (0.0009) [2023-12-27 01:32:16,877][105620] Updated weights for policy 1, policy_version 1395215 (0.0009) [2023-12-27 01:32:17,219][105692] Updated weights for policy 0, policy_version 1393202 (0.0008) [2023-12-27 01:32:17,269][105692] Updated weights for policy 0, policy_version 1393212 (0.0008) [2023-12-27 01:32:17,316][105692] Updated weights for policy 0, policy_version 1393222 (0.0009) [2023-12-27 01:32:17,379][105692] Updated weights for policy 0, policy_version 1393232 (0.0009) [2023-12-27 01:32:17,639][105620] Updated weights for policy 1, policy_version 1395225 (0.0009) [2023-12-27 01:32:17,687][105620] Updated weights for policy 1, policy_version 1395235 (0.0009) [2023-12-27 01:32:17,739][105620] Updated weights for policy 1, policy_version 1395245 (0.0009) [2023-12-27 01:32:17,786][105620] Updated weights for policy 1, policy_version 1395255 (0.0008) [2023-12-27 01:32:18,148][105692] Updated weights for policy 0, policy_version 1393242 (0.0009) [2023-12-27 01:32:18,210][105692] Updated weights for policy 0, policy_version 1393252 (0.0009) [2023-12-27 01:32:18,272][105692] Updated weights for policy 0, policy_version 1393262 (0.0009) [2023-12-27 01:32:18,571][105620] Updated weights for policy 1, policy_version 1395265 (0.0010) [2023-12-27 01:32:18,634][105620] Updated weights for policy 1, policy_version 1395275 (0.0010) [2023-12-27 01:32:18,695][105620] Updated weights for policy 1, policy_version 1395285 (0.0010) [2023-12-27 01:32:18,991][105692] Updated weights for policy 0, policy_version 1393272 (0.0009) [2023-12-27 01:32:19,039][105692] Updated weights for policy 0, policy_version 1393282 (0.0009) [2023-12-27 01:32:19,090][105692] Updated weights for policy 0, policy_version 1393292 (0.0009) [2023-12-27 01:32:19,459][105620] Updated weights for policy 1, policy_version 1395295 (0.0009) [2023-12-27 01:32:19,526][105620] Updated weights for policy 1, policy_version 1395305 (0.0008) [2023-12-27 01:32:19,574][105620] Updated weights for policy 1, policy_version 1395315 (0.0008) [2023-12-27 01:32:19,902][105692] Updated weights for policy 0, policy_version 1393302 (0.0009) [2023-12-27 01:32:19,973][105692] Updated weights for policy 0, policy_version 1393312 (0.0009) [2023-12-27 01:32:20,034][105692] Updated weights for policy 0, policy_version 1393322 (0.0009) [2023-12-27 01:32:20,347][105620] Updated weights for policy 1, policy_version 1395325 (0.0007) [2023-12-27 01:32:20,399][105620] Updated weights for policy 1, policy_version 1395335 (0.0009) [2023-12-27 01:32:20,454][105620] Updated weights for policy 1, policy_version 1395345 (0.0009) [2023-12-27 01:32:20,812][105692] Updated weights for policy 0, policy_version 1393332 (0.0009) [2023-12-27 01:32:20,875][105692] Updated weights for policy 0, policy_version 1393342 (0.0008) [2023-12-27 01:32:20,925][105692] Updated weights for policy 0, policy_version 1393352 (0.0008) [2023-12-27 01:32:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.3, 300 sec: 19355.3). Total num frames: 714006528. Throughput: 0: 9418.5, 1: 9652.7. Samples: 713995536. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:32:21,063][104569] Avg episode reward: [(0, '8982.356'), (1, '8897.055')] [2023-12-27 01:32:21,239][105620] Updated weights for policy 1, policy_version 1395355 (0.0009) [2023-12-27 01:32:21,305][105620] Updated weights for policy 1, policy_version 1395365 (0.0009) [2023-12-27 01:32:21,376][105620] Updated weights for policy 1, policy_version 1395375 (0.0007) [2023-12-27 01:32:21,728][105692] Updated weights for policy 0, policy_version 1393362 (0.0009) [2023-12-27 01:32:21,799][105692] Updated weights for policy 0, policy_version 1393372 (0.0009) [2023-12-27 01:32:21,858][105692] Updated weights for policy 0, policy_version 1393382 (0.0009) [2023-12-27 01:32:21,909][105692] Updated weights for policy 0, policy_version 1393392 (0.0009) [2023-12-27 01:32:22,053][105620] Updated weights for policy 1, policy_version 1395385 (0.0009) [2023-12-27 01:32:22,112][105620] Updated weights for policy 1, policy_version 1395395 (0.0008) [2023-12-27 01:32:22,176][105620] Updated weights for policy 1, policy_version 1395405 (0.0008) [2023-12-27 01:32:22,242][105620] Updated weights for policy 1, policy_version 1395415 (0.0008) [2023-12-27 01:32:22,676][105692] Updated weights for policy 0, policy_version 1393402 (0.0011) [2023-12-27 01:32:22,729][105692] Updated weights for policy 0, policy_version 1393412 (0.0010) [2023-12-27 01:32:22,785][105692] Updated weights for policy 0, policy_version 1393422 (0.0011) [2023-12-27 01:32:22,954][105620] Updated weights for policy 1, policy_version 1395425 (0.0006) [2023-12-27 01:32:23,022][105620] Updated weights for policy 1, policy_version 1395435 (0.0005) [2023-12-27 01:32:23,074][105620] Updated weights for policy 1, policy_version 1395445 (0.0005) [2023-12-27 01:32:23,557][105692] Updated weights for policy 0, policy_version 1393432 (0.0011) [2023-12-27 01:32:23,610][105692] Updated weights for policy 0, policy_version 1393442 (0.0011) [2023-12-27 01:32:23,659][105620] Updated weights for policy 1, policy_version 1395455 (0.0007) [2023-12-27 01:32:23,664][105692] Updated weights for policy 0, policy_version 1393452 (0.0010) [2023-12-27 01:32:23,712][105620] Updated weights for policy 1, policy_version 1395465 (0.0008) [2023-12-27 01:32:23,774][105620] Updated weights for policy 1, policy_version 1395475 (0.0009) [2023-12-27 01:32:24,362][105692] Updated weights for policy 0, policy_version 1393462 (0.0009) [2023-12-27 01:32:24,398][105585] KL-divergence is very high: 139.7167 [2023-12-27 01:32:24,424][105692] Updated weights for policy 0, policy_version 1393472 (0.0011) [2023-12-27 01:32:24,447][105585] KL-divergence is very high: 254.1958 [2023-12-27 01:32:24,483][105692] Updated weights for policy 0, policy_version 1393482 (0.0010) [2023-12-27 01:32:24,496][105585] KL-divergence is very high: 268.5989 [2023-12-27 01:32:24,541][105620] Updated weights for policy 1, policy_version 1395485 (0.0008) [2023-12-27 01:32:24,607][105620] Updated weights for policy 1, policy_version 1395495 (0.0006) [2023-12-27 01:32:24,672][105620] Updated weights for policy 1, policy_version 1395505 (0.0005) [2023-12-27 01:32:25,181][105692] Updated weights for policy 0, policy_version 1393492 (0.0011) [2023-12-27 01:32:25,239][105692] Updated weights for policy 0, policy_version 1393502 (0.0010) [2023-12-27 01:32:25,296][105692] Updated weights for policy 0, policy_version 1393512 (0.0010) [2023-12-27 01:32:25,372][105620] Updated weights for policy 1, policy_version 1395515 (0.0006) [2023-12-27 01:32:25,430][105620] Updated weights for policy 1, policy_version 1395525 (0.0005) [2023-12-27 01:32:25,485][105620] Updated weights for policy 1, policy_version 1395535 (0.0005) [2023-12-27 01:32:25,926][105692] Updated weights for policy 0, policy_version 1393522 (0.0009) [2023-12-27 01:32:25,974][105692] Updated weights for policy 0, policy_version 1393532 (0.0005) [2023-12-27 01:32:26,026][105692] Updated weights for policy 0, policy_version 1393542 (0.0007) [2023-12-27 01:32:26,062][104569] Fps is (10 sec: 18022.7, 60 sec: 19114.7, 300 sec: 19327.6). Total num frames: 714096640. Throughput: 0: 9334.7, 1: 9684.0. Samples: 714109376. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:32:26,063][104569] Avg episode reward: [(0, '8891.395'), (1, '8898.012')] [2023-12-27 01:32:26,074][105692] Updated weights for policy 0, policy_version 1393552 (0.0010) [2023-12-27 01:32:26,158][105620] Updated weights for policy 1, policy_version 1395545 (0.0008) [2023-12-27 01:32:26,216][105620] Updated weights for policy 1, policy_version 1395555 (0.0008) [2023-12-27 01:32:26,267][105620] Updated weights for policy 1, policy_version 1395565 (0.0008) [2023-12-27 01:32:26,319][105620] Updated weights for policy 1, policy_version 1395575 (0.0008) [2023-12-27 01:32:26,803][105692] Updated weights for policy 0, policy_version 1393562 (0.0011) [2023-12-27 01:32:26,864][105692] Updated weights for policy 0, policy_version 1393572 (0.0010) [2023-12-27 01:32:26,905][105620] Updated weights for policy 1, policy_version 1395585 (0.0005) [2023-12-27 01:32:26,916][105692] Updated weights for policy 0, policy_version 1393582 (0.0010) [2023-12-27 01:32:26,956][105620] Updated weights for policy 1, policy_version 1395595 (0.0005) [2023-12-27 01:32:27,012][105620] Updated weights for policy 1, policy_version 1395605 (0.0005) [2023-12-27 01:32:27,561][105620] Updated weights for policy 1, policy_version 1395615 (0.0005) [2023-12-27 01:32:27,612][105620] Updated weights for policy 1, policy_version 1395625 (0.0005) [2023-12-27 01:32:27,650][105692] Updated weights for policy 0, policy_version 1393592 (0.0010) [2023-12-27 01:32:27,669][105620] Updated weights for policy 1, policy_version 1395635 (0.0005) [2023-12-27 01:32:27,704][105692] Updated weights for policy 0, policy_version 1393602 (0.0010) [2023-12-27 01:32:27,765][105692] Updated weights for policy 0, policy_version 1393612 (0.0010) [2023-12-27 01:32:28,252][105620] Updated weights for policy 1, policy_version 1395645 (0.0008) [2023-12-27 01:32:28,304][105620] Updated weights for policy 1, policy_version 1395655 (0.0006) [2023-12-27 01:32:28,366][105620] Updated weights for policy 1, policy_version 1395665 (0.0007) [2023-12-27 01:32:28,511][105692] Updated weights for policy 0, policy_version 1393622 (0.0010) [2023-12-27 01:32:28,563][105692] Updated weights for policy 0, policy_version 1393632 (0.0010) [2023-12-27 01:32:28,615][105692] Updated weights for policy 0, policy_version 1393642 (0.0010) [2023-12-27 01:32:29,047][105620] Updated weights for policy 1, policy_version 1395675 (0.0010) [2023-12-27 01:32:29,105][105620] Updated weights for policy 1, policy_version 1395685 (0.0010) [2023-12-27 01:32:29,170][105620] Updated weights for policy 1, policy_version 1395695 (0.0006) [2023-12-27 01:32:29,372][105692] Updated weights for policy 0, policy_version 1393652 (0.0010) [2023-12-27 01:32:29,426][105692] Updated weights for policy 0, policy_version 1393662 (0.0008) [2023-12-27 01:32:29,487][105692] Updated weights for policy 0, policy_version 1393672 (0.0007) [2023-12-27 01:32:29,874][105620] Updated weights for policy 1, policy_version 1395705 (0.0008) [2023-12-27 01:32:29,941][105620] Updated weights for policy 1, policy_version 1395715 (0.0008) [2023-12-27 01:32:30,004][105620] Updated weights for policy 1, policy_version 1395725 (0.0008) [2023-12-27 01:32:30,073][105620] Updated weights for policy 1, policy_version 1395735 (0.0009) [2023-12-27 01:32:30,231][105692] Updated weights for policy 0, policy_version 1393682 (0.0006) [2023-12-27 01:32:30,285][105692] Updated weights for policy 0, policy_version 1393692 (0.0008) [2023-12-27 01:32:30,349][105692] Updated weights for policy 0, policy_version 1393702 (0.0007) [2023-12-27 01:32:30,415][105692] Updated weights for policy 0, policy_version 1393712 (0.0005) [2023-12-27 01:32:30,881][105620] Updated weights for policy 1, policy_version 1395745 (0.0010) [2023-12-27 01:32:30,937][105620] Updated weights for policy 1, policy_version 1395755 (0.0009) [2023-12-27 01:32:30,951][105692] Updated weights for policy 0, policy_version 1393722 (0.0005) [2023-12-27 01:32:30,985][105620] Updated weights for policy 1, policy_version 1395765 (0.0006) [2023-12-27 01:32:31,013][105692] Updated weights for policy 0, policy_version 1393732 (0.0006) [2023-12-27 01:32:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 714203136. Throughput: 0: 9360.8, 1: 9841.0. Samples: 714172952. Policy #0 lag: (min: 11.0, avg: 27.8, max: 43.0) [2023-12-27 01:32:31,063][104569] Avg episode reward: [(0, '8712.046'), (1, '8716.324')] [2023-12-27 01:32:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001395768_357359616.pth... [2023-12-27 01:32:31,068][105692] Updated weights for policy 0, policy_version 1393742 (0.0009) [2023-12-27 01:32:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001394616_357064704.pth [2023-12-27 01:32:31,079][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001393744_356851712.pth... [2023-12-27 01:32:31,082][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001392624_356564992.pth [2023-12-27 01:32:31,783][105692] Updated weights for policy 0, policy_version 1393752 (0.0009) [2023-12-27 01:32:31,793][105620] Updated weights for policy 1, policy_version 1395775 (0.0006) [2023-12-27 01:32:31,843][105692] Updated weights for policy 0, policy_version 1393762 (0.0008) [2023-12-27 01:32:31,857][105620] Updated weights for policy 1, policy_version 1395785 (0.0007) [2023-12-27 01:32:31,900][105692] Updated weights for policy 0, policy_version 1393772 (0.0007) [2023-12-27 01:32:31,914][105620] Updated weights for policy 1, policy_version 1395795 (0.0007) [2023-12-27 01:32:32,547][105692] Updated weights for policy 0, policy_version 1393782 (0.0008) [2023-12-27 01:32:32,609][105692] Updated weights for policy 0, policy_version 1393792 (0.0010) [2023-12-27 01:32:32,665][105620] Updated weights for policy 1, policy_version 1395805 (0.0008) [2023-12-27 01:32:32,670][105692] Updated weights for policy 0, policy_version 1393802 (0.0010) [2023-12-27 01:32:32,722][105620] Updated weights for policy 1, policy_version 1395815 (0.0006) [2023-12-27 01:32:32,774][105620] Updated weights for policy 1, policy_version 1395825 (0.0008) [2023-12-27 01:32:33,310][105692] Updated weights for policy 0, policy_version 1393812 (0.0009) [2023-12-27 01:32:33,362][105692] Updated weights for policy 0, policy_version 1393822 (0.0005) [2023-12-27 01:32:33,419][105692] Updated weights for policy 0, policy_version 1393832 (0.0005) [2023-12-27 01:32:33,637][105620] Updated weights for policy 1, policy_version 1395835 (0.0008) [2023-12-27 01:32:33,684][105620] Updated weights for policy 1, policy_version 1395845 (0.0008) [2023-12-27 01:32:33,731][105620] Updated weights for policy 1, policy_version 1395855 (0.0008) [2023-12-27 01:32:34,043][105692] Updated weights for policy 0, policy_version 1393842 (0.0006) [2023-12-27 01:32:34,102][105692] Updated weights for policy 0, policy_version 1393852 (0.0011) [2023-12-27 01:32:34,173][105692] Updated weights for policy 0, policy_version 1393862 (0.0007) [2023-12-27 01:32:34,249][105692] Updated weights for policy 0, policy_version 1393872 (0.0006) [2023-12-27 01:32:34,494][105620] Updated weights for policy 1, policy_version 1395865 (0.0008) [2023-12-27 01:32:34,558][105620] Updated weights for policy 1, policy_version 1395875 (0.0010) [2023-12-27 01:32:34,624][105620] Updated weights for policy 1, policy_version 1395885 (0.0010) [2023-12-27 01:32:34,681][105620] Updated weights for policy 1, policy_version 1395895 (0.0009) [2023-12-27 01:32:34,892][105692] Updated weights for policy 0, policy_version 1393882 (0.0006) [2023-12-27 01:32:34,956][105692] Updated weights for policy 0, policy_version 1393892 (0.0005) [2023-12-27 01:32:35,006][105692] Updated weights for policy 0, policy_version 1393902 (0.0005) [2023-12-27 01:32:35,520][105620] Updated weights for policy 1, policy_version 1395905 (0.0009) [2023-12-27 01:32:35,575][105620] Updated weights for policy 1, policy_version 1395915 (0.0009) [2023-12-27 01:32:35,580][105692] Updated weights for policy 0, policy_version 1393912 (0.0007) [2023-12-27 01:32:35,620][105620] Updated weights for policy 1, policy_version 1395925 (0.0006) [2023-12-27 01:32:35,626][105692] Updated weights for policy 0, policy_version 1393922 (0.0006) [2023-12-27 01:32:35,635][105585] KL-divergence is very high: 148.9293 [2023-12-27 01:32:35,678][105692] Updated weights for policy 0, policy_version 1393932 (0.0008) [2023-12-27 01:32:35,678][105585] KL-divergence is very high: 157.2400 [2023-12-27 01:32:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19251.2, 300 sec: 19355.3). Total num frames: 714301440. Throughput: 0: 9516.2, 1: 9803.5. Samples: 714288508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:32:36,063][104569] Avg episode reward: [(0, '8351.775'), (1, '8810.537')] [2023-12-27 01:32:36,289][105620] Updated weights for policy 1, policy_version 1395935 (0.0008) [2023-12-27 01:32:36,356][105620] Updated weights for policy 1, policy_version 1395945 (0.0009) [2023-12-27 01:32:36,419][105620] Updated weights for policy 1, policy_version 1395955 (0.0009) [2023-12-27 01:32:36,473][105692] Updated weights for policy 0, policy_version 1393942 (0.0007) [2023-12-27 01:32:36,528][105692] Updated weights for policy 0, policy_version 1393952 (0.0006) [2023-12-27 01:32:36,584][105692] Updated weights for policy 0, policy_version 1393962 (0.0009) [2023-12-27 01:32:37,052][105620] Updated weights for policy 1, policy_version 1395965 (0.0010) [2023-12-27 01:32:37,108][105620] Updated weights for policy 1, policy_version 1395975 (0.0009) [2023-12-27 01:32:37,159][105620] Updated weights for policy 1, policy_version 1395985 (0.0009) [2023-12-27 01:32:37,344][105692] Updated weights for policy 0, policy_version 1393972 (0.0008) [2023-12-27 01:32:37,399][105692] Updated weights for policy 0, policy_version 1393982 (0.0007) [2023-12-27 01:32:37,453][105692] Updated weights for policy 0, policy_version 1393992 (0.0008) [2023-12-27 01:32:37,942][105620] Updated weights for policy 1, policy_version 1395995 (0.0009) [2023-12-27 01:32:38,001][105620] Updated weights for policy 1, policy_version 1396005 (0.0008) [2023-12-27 01:32:38,066][105620] Updated weights for policy 1, policy_version 1396015 (0.0008) [2023-12-27 01:32:38,134][105692] Updated weights for policy 0, policy_version 1394002 (0.0007) [2023-12-27 01:32:38,191][105692] Updated weights for policy 0, policy_version 1394012 (0.0008) [2023-12-27 01:32:38,244][105692] Updated weights for policy 0, policy_version 1394022 (0.0007) [2023-12-27 01:32:38,296][105692] Updated weights for policy 0, policy_version 1394032 (0.0009) [2023-12-27 01:32:38,721][105620] Updated weights for policy 1, policy_version 1396025 (0.0007) [2023-12-27 01:32:38,783][105620] Updated weights for policy 1, policy_version 1396035 (0.0009) [2023-12-27 01:32:38,845][105620] Updated weights for policy 1, policy_version 1396045 (0.0009) [2023-12-27 01:32:38,906][105620] Updated weights for policy 1, policy_version 1396055 (0.0009) [2023-12-27 01:32:39,094][105692] Updated weights for policy 0, policy_version 1394042 (0.0009) [2023-12-27 01:32:39,158][105692] Updated weights for policy 0, policy_version 1394052 (0.0009) [2023-12-27 01:32:39,243][105692] Updated weights for policy 0, policy_version 1394062 (0.0010) [2023-12-27 01:32:39,615][105620] Updated weights for policy 1, policy_version 1396065 (0.0006) [2023-12-27 01:32:39,668][105620] Updated weights for policy 1, policy_version 1396075 (0.0007) [2023-12-27 01:32:39,716][105620] Updated weights for policy 1, policy_version 1396085 (0.0009) [2023-12-27 01:32:40,070][105692] Updated weights for policy 0, policy_version 1394072 (0.0008) [2023-12-27 01:32:40,130][105692] Updated weights for policy 0, policy_version 1394082 (0.0008) [2023-12-27 01:32:40,188][105692] Updated weights for policy 0, policy_version 1394092 (0.0008) [2023-12-27 01:32:40,430][105620] Updated weights for policy 1, policy_version 1396095 (0.0010) [2023-12-27 01:32:40,479][105620] Updated weights for policy 1, policy_version 1396105 (0.0010) [2023-12-27 01:32:40,525][105620] Updated weights for policy 1, policy_version 1396115 (0.0010) [2023-12-27 01:32:40,976][105692] Updated weights for policy 0, policy_version 1394102 (0.0008) [2023-12-27 01:32:41,040][105692] Updated weights for policy 0, policy_version 1394112 (0.0009) [2023-12-27 01:32:41,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19114.7, 300 sec: 19383.1). Total num frames: 714391552. Throughput: 0: 9540.6, 1: 9752.8. Samples: 714403804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:32:41,062][104569] Avg episode reward: [(0, '8448.318'), (1, '8713.575')] [2023-12-27 01:32:41,110][105692] Updated weights for policy 0, policy_version 1394122 (0.0010) [2023-12-27 01:32:41,256][105620] Updated weights for policy 1, policy_version 1396125 (0.0009) [2023-12-27 01:32:41,322][105620] Updated weights for policy 1, policy_version 1396135 (0.0007) [2023-12-27 01:32:41,387][105620] Updated weights for policy 1, policy_version 1396145 (0.0010) [2023-12-27 01:32:41,861][105692] Updated weights for policy 0, policy_version 1394132 (0.0009) [2023-12-27 01:32:41,919][105692] Updated weights for policy 0, policy_version 1394142 (0.0009) [2023-12-27 01:32:41,980][105692] Updated weights for policy 0, policy_version 1394152 (0.0008) [2023-12-27 01:32:42,116][105620] Updated weights for policy 1, policy_version 1396155 (0.0009) [2023-12-27 01:32:42,177][105620] Updated weights for policy 1, policy_version 1396165 (0.0006) [2023-12-27 01:32:42,236][105620] Updated weights for policy 1, policy_version 1396175 (0.0007) [2023-12-27 01:32:42,735][105692] Updated weights for policy 0, policy_version 1394162 (0.0009) [2023-12-27 01:32:42,806][105692] Updated weights for policy 0, policy_version 1394172 (0.0008) [2023-12-27 01:32:42,859][105620] Updated weights for policy 1, policy_version 1396185 (0.0007) [2023-12-27 01:32:42,875][105692] Updated weights for policy 0, policy_version 1394182 (0.0008) [2023-12-27 01:32:42,920][105620] Updated weights for policy 1, policy_version 1396195 (0.0006) [2023-12-27 01:32:42,940][105692] Updated weights for policy 0, policy_version 1394192 (0.0008) [2023-12-27 01:32:42,981][105620] Updated weights for policy 1, policy_version 1396205 (0.0006) [2023-12-27 01:32:43,041][105620] Updated weights for policy 1, policy_version 1396215 (0.0006) [2023-12-27 01:32:43,578][105620] Updated weights for policy 1, policy_version 1396225 (0.0009) [2023-12-27 01:32:43,627][105620] Updated weights for policy 1, policy_version 1396235 (0.0008) [2023-12-27 01:32:43,647][105692] Updated weights for policy 0, policy_version 1394202 (0.0005) [2023-12-27 01:32:43,672][105620] Updated weights for policy 1, policy_version 1396245 (0.0009) [2023-12-27 01:32:43,695][105692] Updated weights for policy 0, policy_version 1394212 (0.0007) [2023-12-27 01:32:43,749][105692] Updated weights for policy 0, policy_version 1394222 (0.0009) [2023-12-27 01:32:44,441][105692] Updated weights for policy 0, policy_version 1394232 (0.0008) [2023-12-27 01:32:44,454][105620] Updated weights for policy 1, policy_version 1396255 (0.0006) [2023-12-27 01:32:44,490][105692] Updated weights for policy 0, policy_version 1394242 (0.0009) [2023-12-27 01:32:44,514][105620] Updated weights for policy 1, policy_version 1396265 (0.0006) [2023-12-27 01:32:44,536][105692] Updated weights for policy 0, policy_version 1394252 (0.0008) [2023-12-27 01:32:44,573][105620] Updated weights for policy 1, policy_version 1396275 (0.0007) [2023-12-27 01:32:45,240][105620] Updated weights for policy 1, policy_version 1396285 (0.0006) [2023-12-27 01:32:45,302][105620] Updated weights for policy 1, policy_version 1396295 (0.0007) [2023-12-27 01:32:45,320][105692] Updated weights for policy 0, policy_version 1394262 (0.0009) [2023-12-27 01:32:45,364][105620] Updated weights for policy 1, policy_version 1396305 (0.0007) [2023-12-27 01:32:45,382][105692] Updated weights for policy 0, policy_version 1394272 (0.0007) [2023-12-27 01:32:45,450][105692] Updated weights for policy 0, policy_version 1394282 (0.0007) [2023-12-27 01:32:45,940][105620] Updated weights for policy 1, policy_version 1396315 (0.0006) [2023-12-27 01:32:45,993][105620] Updated weights for policy 1, policy_version 1396325 (0.0008) [2023-12-27 01:32:46,041][105620] Updated weights for policy 1, policy_version 1396335 (0.0006) [2023-12-27 01:32:46,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19251.1, 300 sec: 19355.3). Total num frames: 714489856. Throughput: 0: 9520.1, 1: 9791.8. Samples: 714462656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:32:46,063][104569] Avg episode reward: [(0, '8545.008'), (1, '8478.888')] [2023-12-27 01:32:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001394288_356990976.pth... [2023-12-27 01:32:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001393168_356704256.pth [2023-12-27 01:32:46,089][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001396344_357507072.pth... [2023-12-27 01:32:46,093][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001395192_357212160.pth [2023-12-27 01:32:46,204][105692] Updated weights for policy 0, policy_version 1394292 (0.0007) [2023-12-27 01:32:46,258][105692] Updated weights for policy 0, policy_version 1394302 (0.0010) [2023-12-27 01:32:46,312][105692] Updated weights for policy 0, policy_version 1394312 (0.0009) [2023-12-27 01:32:46,637][105620] Updated weights for policy 1, policy_version 1396345 (0.0008) [2023-12-27 01:32:46,703][105620] Updated weights for policy 1, policy_version 1396355 (0.0006) [2023-12-27 01:32:46,755][105620] Updated weights for policy 1, policy_version 1396365 (0.0005) [2023-12-27 01:32:46,817][105620] Updated weights for policy 1, policy_version 1396375 (0.0007) [2023-12-27 01:32:47,161][105692] Updated weights for policy 0, policy_version 1394322 (0.0009) [2023-12-27 01:32:47,218][105692] Updated weights for policy 0, policy_version 1394332 (0.0008) [2023-12-27 01:32:47,281][105692] Updated weights for policy 0, policy_version 1394342 (0.0006) [2023-12-27 01:32:47,352][105692] Updated weights for policy 0, policy_version 1394352 (0.0006) [2023-12-27 01:32:47,430][105620] Updated weights for policy 1, policy_version 1396385 (0.0006) [2023-12-27 01:32:47,481][105620] Updated weights for policy 1, policy_version 1396395 (0.0005) [2023-12-27 01:32:47,548][105620] Updated weights for policy 1, policy_version 1396405 (0.0005) [2023-12-27 01:32:47,890][105692] Updated weights for policy 0, policy_version 1394362 (0.0009) [2023-12-27 01:32:47,942][105692] Updated weights for policy 0, policy_version 1394372 (0.0010) [2023-12-27 01:32:48,003][105692] Updated weights for policy 0, policy_version 1394382 (0.0010) [2023-12-27 01:32:48,110][105620] Updated weights for policy 1, policy_version 1396415 (0.0007) [2023-12-27 01:32:48,160][105620] Updated weights for policy 1, policy_version 1396425 (0.0005) [2023-12-27 01:32:48,221][105620] Updated weights for policy 1, policy_version 1396435 (0.0006) [2023-12-27 01:32:48,667][105692] Updated weights for policy 0, policy_version 1394392 (0.0006) [2023-12-27 01:32:48,722][105692] Updated weights for policy 0, policy_version 1394402 (0.0006) [2023-12-27 01:32:48,777][105692] Updated weights for policy 0, policy_version 1394412 (0.0009) [2023-12-27 01:32:49,002][105620] Updated weights for policy 1, policy_version 1396445 (0.0007) [2023-12-27 01:32:49,056][105620] Updated weights for policy 1, policy_version 1396455 (0.0008) [2023-12-27 01:32:49,114][105620] Updated weights for policy 1, policy_version 1396465 (0.0008) [2023-12-27 01:32:49,453][105692] Updated weights for policy 0, policy_version 1394422 (0.0008) [2023-12-27 01:32:49,512][105692] Updated weights for policy 0, policy_version 1394432 (0.0009) [2023-12-27 01:32:49,564][105692] Updated weights for policy 0, policy_version 1394442 (0.0007) [2023-12-27 01:32:49,951][105620] Updated weights for policy 1, policy_version 1396475 (0.0008) [2023-12-27 01:32:50,006][105620] Updated weights for policy 1, policy_version 1396485 (0.0007) [2023-12-27 01:32:50,072][105620] Updated weights for policy 1, policy_version 1396495 (0.0006) [2023-12-27 01:32:50,314][105692] Updated weights for policy 0, policy_version 1394452 (0.0007) [2023-12-27 01:32:50,370][105692] Updated weights for policy 0, policy_version 1394462 (0.0010) [2023-12-27 01:32:50,433][105692] Updated weights for policy 0, policy_version 1394473 (0.0010) [2023-12-27 01:32:50,669][105620] Updated weights for policy 1, policy_version 1396505 (0.0006) [2023-12-27 01:32:50,726][105620] Updated weights for policy 1, policy_version 1396515 (0.0011) [2023-12-27 01:32:50,784][105620] Updated weights for policy 1, policy_version 1396525 (0.0007) [2023-12-27 01:32:50,842][105620] Updated weights for policy 1, policy_version 1396535 (0.0011) [2023-12-27 01:32:51,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19524.2, 300 sec: 19383.1). Total num frames: 714596352. Throughput: 0: 9519.4, 1: 9795.6. Samples: 714583444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:32:51,063][104569] Avg episode reward: [(0, '8348.519'), (1, '8850.209')] [2023-12-27 01:32:51,312][105692] Updated weights for policy 0, policy_version 1394484 (0.0010) [2023-12-27 01:32:51,380][105692] Updated weights for policy 0, policy_version 1394494 (0.0008) [2023-12-27 01:32:51,428][105692] Updated weights for policy 0, policy_version 1394504 (0.0007) [2023-12-27 01:32:51,566][105620] Updated weights for policy 1, policy_version 1396545 (0.0010) [2023-12-27 01:32:51,632][105620] Updated weights for policy 1, policy_version 1396555 (0.0008) [2023-12-27 01:32:51,697][105620] Updated weights for policy 1, policy_version 1396565 (0.0008) [2023-12-27 01:32:52,274][105692] Updated weights for policy 0, policy_version 1394514 (0.0008) [2023-12-27 01:32:52,288][105620] Updated weights for policy 1, policy_version 1396575 (0.0009) [2023-12-27 01:32:52,328][105692] Updated weights for policy 0, policy_version 1394524 (0.0006) [2023-12-27 01:32:52,346][105620] Updated weights for policy 1, policy_version 1396585 (0.0008) [2023-12-27 01:32:52,393][105692] Updated weights for policy 0, policy_version 1394534 (0.0008) [2023-12-27 01:32:52,407][105620] Updated weights for policy 1, policy_version 1396595 (0.0008) [2023-12-27 01:32:52,443][105692] Updated weights for policy 0, policy_version 1394544 (0.0007) [2023-12-27 01:32:53,101][105620] Updated weights for policy 1, policy_version 1396605 (0.0008) [2023-12-27 01:32:53,161][105620] Updated weights for policy 1, policy_version 1396615 (0.0009) [2023-12-27 01:32:53,216][105620] Updated weights for policy 1, policy_version 1396625 (0.0006) [2023-12-27 01:32:53,251][105692] Updated weights for policy 0, policy_version 1394554 (0.0010) [2023-12-27 01:32:53,306][105692] Updated weights for policy 0, policy_version 1394564 (0.0009) [2023-12-27 01:32:53,360][105692] Updated weights for policy 0, policy_version 1394574 (0.0009) [2023-12-27 01:32:53,815][105620] Updated weights for policy 1, policy_version 1396635 (0.0005) [2023-12-27 01:32:53,871][105620] Updated weights for policy 1, policy_version 1396645 (0.0008) [2023-12-27 01:32:53,925][105620] Updated weights for policy 1, policy_version 1396655 (0.0009) [2023-12-27 01:32:54,129][105692] Updated weights for policy 0, policy_version 1394584 (0.0008) [2023-12-27 01:32:54,180][105692] Updated weights for policy 0, policy_version 1394594 (0.0009) [2023-12-27 01:32:54,233][105692] Updated weights for policy 0, policy_version 1394604 (0.0009) [2023-12-27 01:32:54,666][105620] Updated weights for policy 1, policy_version 1396665 (0.0009) [2023-12-27 01:32:54,723][105620] Updated weights for policy 1, policy_version 1396675 (0.0009) [2023-12-27 01:32:54,789][105620] Updated weights for policy 1, policy_version 1396685 (0.0007) [2023-12-27 01:32:54,844][105620] Updated weights for policy 1, policy_version 1396695 (0.0006) [2023-12-27 01:32:55,059][105692] Updated weights for policy 0, policy_version 1394614 (0.0009) [2023-12-27 01:32:55,099][105585] KL-divergence is very high: 123.9360 [2023-12-27 01:32:55,108][105692] Updated weights for policy 0, policy_version 1394624 (0.0007) [2023-12-27 01:32:55,138][105585] KL-divergence is very high: 205.9268 [2023-12-27 01:32:55,159][105692] Updated weights for policy 0, policy_version 1394634 (0.0008) [2023-12-27 01:32:55,187][105585] KL-divergence is very high: 193.7084 [2023-12-27 01:32:55,534][105620] Updated weights for policy 1, policy_version 1396705 (0.0010) [2023-12-27 01:32:55,582][105620] Updated weights for policy 1, policy_version 1396715 (0.0010) [2023-12-27 01:32:55,626][105620] Updated weights for policy 1, policy_version 1396725 (0.0010) [2023-12-27 01:32:55,967][105692] Updated weights for policy 0, policy_version 1394644 (0.0008) [2023-12-27 01:32:56,030][105692] Updated weights for policy 0, policy_version 1394654 (0.0006) [2023-12-27 01:32:56,062][104569] Fps is (10 sec: 19661.6, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 714686464. Throughput: 0: 9530.2, 1: 9807.3. Samples: 714697896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:32:56,062][104569] Avg episode reward: [(0, '8529.414'), (1, '9088.566')] [2023-12-27 01:32:56,090][105692] Updated weights for policy 0, policy_version 1394664 (0.0008) [2023-12-27 01:32:56,385][105620] Updated weights for policy 1, policy_version 1396735 (0.0010) [2023-12-27 01:32:56,429][105620] Updated weights for policy 1, policy_version 1396745 (0.0010) [2023-12-27 01:32:56,481][105620] Updated weights for policy 1, policy_version 1396755 (0.0010) [2023-12-27 01:32:56,803][105692] Updated weights for policy 0, policy_version 1394674 (0.0008) [2023-12-27 01:32:56,861][105692] Updated weights for policy 0, policy_version 1394684 (0.0009) [2023-12-27 01:32:56,913][105692] Updated weights for policy 0, policy_version 1394694 (0.0010) [2023-12-27 01:32:56,964][105692] Updated weights for policy 0, policy_version 1394704 (0.0010) [2023-12-27 01:32:57,183][105620] Updated weights for policy 1, policy_version 1396765 (0.0008) [2023-12-27 01:32:57,241][105620] Updated weights for policy 1, policy_version 1396775 (0.0005) [2023-12-27 01:32:57,283][105620] Updated weights for policy 1, policy_version 1396785 (0.0005) [2023-12-27 01:32:57,602][105692] Updated weights for policy 0, policy_version 1394714 (0.0010) [2023-12-27 01:32:57,667][105692] Updated weights for policy 0, policy_version 1394724 (0.0010) [2023-12-27 01:32:57,716][105692] Updated weights for policy 0, policy_version 1394734 (0.0008) [2023-12-27 01:32:57,843][105620] Updated weights for policy 1, policy_version 1396795 (0.0006) [2023-12-27 01:32:57,893][105620] Updated weights for policy 1, policy_version 1396805 (0.0005) [2023-12-27 01:32:57,944][105620] Updated weights for policy 1, policy_version 1396815 (0.0005) [2023-12-27 01:32:58,441][105692] Updated weights for policy 0, policy_version 1394744 (0.0009) [2023-12-27 01:32:58,500][105692] Updated weights for policy 0, policy_version 1394754 (0.0009) [2023-12-27 01:32:58,560][105692] Updated weights for policy 0, policy_version 1394764 (0.0009) [2023-12-27 01:32:58,591][105620] Updated weights for policy 1, policy_version 1396825 (0.0006) [2023-12-27 01:32:58,659][105620] Updated weights for policy 1, policy_version 1396835 (0.0009) [2023-12-27 01:32:58,725][105620] Updated weights for policy 1, policy_version 1396845 (0.0009) [2023-12-27 01:32:58,790][105620] Updated weights for policy 1, policy_version 1396855 (0.0008) [2023-12-27 01:32:59,394][105692] Updated weights for policy 0, policy_version 1394774 (0.0008) [2023-12-27 01:32:59,457][105692] Updated weights for policy 0, policy_version 1394784 (0.0009) [2023-12-27 01:32:59,508][105692] Updated weights for policy 0, policy_version 1394794 (0.0008) [2023-12-27 01:32:59,540][105620] Updated weights for policy 1, policy_version 1396865 (0.0008) [2023-12-27 01:32:59,603][105620] Updated weights for policy 1, policy_version 1396875 (0.0009) [2023-12-27 01:32:59,671][105620] Updated weights for policy 1, policy_version 1396885 (0.0009) [2023-12-27 01:33:00,304][105620] Updated weights for policy 1, policy_version 1396895 (0.0006) [2023-12-27 01:33:00,349][105692] Updated weights for policy 0, policy_version 1394804 (0.0007) [2023-12-27 01:33:00,371][105620] Updated weights for policy 1, policy_version 1396905 (0.0007) [2023-12-27 01:33:00,411][105692] Updated weights for policy 0, policy_version 1394814 (0.0007) [2023-12-27 01:33:00,430][105620] Updated weights for policy 1, policy_version 1396915 (0.0008) [2023-12-27 01:33:00,472][105692] Updated weights for policy 0, policy_version 1394824 (0.0007) [2023-12-27 01:33:01,040][105620] Updated weights for policy 1, policy_version 1396925 (0.0007) [2023-12-27 01:33:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 714784768. Throughput: 0: 9535.8, 1: 9896.4. Samples: 714758540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:33:01,063][104569] Avg episode reward: [(0, '8714.901'), (1, '7115.157')] [2023-12-27 01:33:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001394832_357130240.pth... [2023-12-27 01:33:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001393744_356851712.pth [2023-12-27 01:33:01,095][105620] Updated weights for policy 1, policy_version 1396936 (0.0007) [2023-12-27 01:33:01,162][105620] Updated weights for policy 1, policy_version 1396946 (0.0011) [2023-12-27 01:33:01,202][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001396952_357662720.pth... [2023-12-27 01:33:01,207][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001395768_357359616.pth [2023-12-27 01:33:01,268][105692] Updated weights for policy 0, policy_version 1394834 (0.0009) [2023-12-27 01:33:01,320][105692] Updated weights for policy 0, policy_version 1394844 (0.0006) [2023-12-27 01:33:01,387][105692] Updated weights for policy 0, policy_version 1394854 (0.0009) [2023-12-27 01:33:01,445][105692] Updated weights for policy 0, policy_version 1394864 (0.0005) [2023-12-27 01:33:01,909][105620] Updated weights for policy 1, policy_version 1396956 (0.0009) [2023-12-27 01:33:01,973][105620] Updated weights for policy 1, policy_version 1396966 (0.0005) [2023-12-27 01:33:02,024][105620] Updated weights for policy 1, policy_version 1396976 (0.0007) [2023-12-27 01:33:02,045][105692] Updated weights for policy 0, policy_version 1394874 (0.0008) [2023-12-27 01:33:02,096][105692] Updated weights for policy 0, policy_version 1394884 (0.0009) [2023-12-27 01:33:02,148][105692] Updated weights for policy 0, policy_version 1394894 (0.0009) [2023-12-27 01:33:02,641][105620] Updated weights for policy 1, policy_version 1396986 (0.0006) [2023-12-27 01:33:02,695][105620] Updated weights for policy 1, policy_version 1396996 (0.0005) [2023-12-27 01:33:02,763][105620] Updated weights for policy 1, policy_version 1397006 (0.0005) [2023-12-27 01:33:02,829][105620] Updated weights for policy 1, policy_version 1397016 (0.0009) [2023-12-27 01:33:02,830][105692] Updated weights for policy 0, policy_version 1394904 (0.0006) [2023-12-27 01:33:02,879][105692] Updated weights for policy 0, policy_version 1394914 (0.0006) [2023-12-27 01:33:02,933][105692] Updated weights for policy 0, policy_version 1394924 (0.0005) [2023-12-27 01:33:03,496][105620] Updated weights for policy 1, policy_version 1397026 (0.0005) [2023-12-27 01:33:03,545][105620] Updated weights for policy 1, policy_version 1397036 (0.0005) [2023-12-27 01:33:03,554][105586] KL-divergence is very high: 135.4465 [2023-12-27 01:33:03,583][105586] KL-divergence is very high: 133.5423 [2023-12-27 01:33:03,597][105620] Updated weights for policy 1, policy_version 1397046 (0.0006) [2023-12-27 01:33:03,598][105586] KL-divergence is very high: 191.2634 [2023-12-27 01:33:03,661][105692] Updated weights for policy 0, policy_version 1394934 (0.0007) [2023-12-27 01:33:03,718][105692] Updated weights for policy 0, policy_version 1394945 (0.0010) [2023-12-27 01:33:03,770][105692] Updated weights for policy 0, policy_version 1394955 (0.0009) [2023-12-27 01:33:04,221][105620] Updated weights for policy 1, policy_version 1397056 (0.0008) [2023-12-27 01:33:04,282][105620] Updated weights for policy 1, policy_version 1397066 (0.0009) [2023-12-27 01:33:04,339][105620] Updated weights for policy 1, policy_version 1397076 (0.0005) [2023-12-27 01:33:04,598][105692] Updated weights for policy 0, policy_version 1394965 (0.0007) [2023-12-27 01:33:04,661][105692] Updated weights for policy 0, policy_version 1394975 (0.0006) [2023-12-27 01:33:04,723][105692] Updated weights for policy 0, policy_version 1394985 (0.0008) [2023-12-27 01:33:04,975][105620] Updated weights for policy 1, policy_version 1397086 (0.0007) [2023-12-27 01:33:05,036][105620] Updated weights for policy 1, policy_version 1397096 (0.0009) [2023-12-27 01:33:05,091][105620] Updated weights for policy 1, policy_version 1397106 (0.0009) [2023-12-27 01:33:05,470][105692] Updated weights for policy 0, policy_version 1394995 (0.0008) [2023-12-27 01:33:05,523][105692] Updated weights for policy 0, policy_version 1395005 (0.0005) [2023-12-27 01:33:05,593][105692] Updated weights for policy 0, policy_version 1395015 (0.0005) [2023-12-27 01:33:05,885][105620] Updated weights for policy 1, policy_version 1397116 (0.0009) [2023-12-27 01:33:05,946][105620] Updated weights for policy 1, policy_version 1397126 (0.0008) [2023-12-27 01:33:06,005][105620] Updated weights for policy 1, policy_version 1397136 (0.0009) [2023-12-27 01:33:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.8, 300 sec: 19355.3). Total num frames: 714891264. Throughput: 0: 9520.0, 1: 10073.5. Samples: 714877244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:33:06,062][104569] Avg episode reward: [(0, '8257.488'), (1, '5690.541')] [2023-12-27 01:33:06,154][105692] Updated weights for policy 0, policy_version 1395025 (0.0006) [2023-12-27 01:33:06,215][105692] Updated weights for policy 0, policy_version 1395035 (0.0008) [2023-12-27 01:33:06,276][105692] Updated weights for policy 0, policy_version 1395045 (0.0006) [2023-12-27 01:33:06,341][105692] Updated weights for policy 0, policy_version 1395055 (0.0006) [2023-12-27 01:33:06,774][105620] Updated weights for policy 1, policy_version 1397146 (0.0009) [2023-12-27 01:33:06,837][105620] Updated weights for policy 1, policy_version 1397156 (0.0009) [2023-12-27 01:33:06,888][105620] Updated weights for policy 1, policy_version 1397166 (0.0007) [2023-12-27 01:33:06,937][105620] Updated weights for policy 1, policy_version 1397176 (0.0005) [2023-12-27 01:33:07,070][105692] Updated weights for policy 0, policy_version 1395065 (0.0009) [2023-12-27 01:33:07,117][105692] Updated weights for policy 0, policy_version 1395075 (0.0008) [2023-12-27 01:33:07,171][105692] Updated weights for policy 0, policy_version 1395085 (0.0009) [2023-12-27 01:33:07,600][105620] Updated weights for policy 1, policy_version 1397186 (0.0009) [2023-12-27 01:33:07,651][105620] Updated weights for policy 1, policy_version 1397196 (0.0009) [2023-12-27 01:33:07,703][105620] Updated weights for policy 1, policy_version 1397206 (0.0009) [2023-12-27 01:33:07,950][105692] Updated weights for policy 0, policy_version 1395095 (0.0009) [2023-12-27 01:33:08,000][105692] Updated weights for policy 0, policy_version 1395105 (0.0009) [2023-12-27 01:33:08,051][105692] Updated weights for policy 0, policy_version 1395115 (0.0009) [2023-12-27 01:33:08,483][105620] Updated weights for policy 1, policy_version 1397216 (0.0007) [2023-12-27 01:33:08,538][105620] Updated weights for policy 1, policy_version 1397226 (0.0005) [2023-12-27 01:33:08,600][105620] Updated weights for policy 1, policy_version 1397236 (0.0009) [2023-12-27 01:33:08,843][105692] Updated weights for policy 0, policy_version 1395125 (0.0009) [2023-12-27 01:33:08,904][105692] Updated weights for policy 0, policy_version 1395135 (0.0008) [2023-12-27 01:33:08,962][105692] Updated weights for policy 0, policy_version 1395145 (0.0009) [2023-12-27 01:33:09,300][105620] Updated weights for policy 1, policy_version 1397246 (0.0009) [2023-12-27 01:33:09,375][105620] Updated weights for policy 1, policy_version 1397256 (0.0009) [2023-12-27 01:33:09,443][105620] Updated weights for policy 1, policy_version 1397266 (0.0009) [2023-12-27 01:33:09,768][105692] Updated weights for policy 0, policy_version 1395155 (0.0010) [2023-12-27 01:33:09,832][105692] Updated weights for policy 0, policy_version 1395165 (0.0009) [2023-12-27 01:33:09,887][105692] Updated weights for policy 0, policy_version 1395175 (0.0009) [2023-12-27 01:33:10,179][105620] Updated weights for policy 1, policy_version 1397276 (0.0007) [2023-12-27 01:33:10,241][105620] Updated weights for policy 1, policy_version 1397286 (0.0005) [2023-12-27 01:33:10,297][105620] Updated weights for policy 1, policy_version 1397296 (0.0007) [2023-12-27 01:33:10,698][105692] Updated weights for policy 0, policy_version 1395185 (0.0009) [2023-12-27 01:33:10,775][105692] Updated weights for policy 0, policy_version 1395195 (0.0009) [2023-12-27 01:33:10,825][105692] Updated weights for policy 0, policy_version 1395205 (0.0008) [2023-12-27 01:33:10,883][105692] Updated weights for policy 0, policy_version 1395215 (0.0009) [2023-12-27 01:33:10,987][105620] Updated weights for policy 1, policy_version 1397306 (0.0009) [2023-12-27 01:33:11,050][105620] Updated weights for policy 1, policy_version 1397316 (0.0008) [2023-12-27 01:33:11,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.8, 300 sec: 19355.3). Total num frames: 714981376. Throughput: 0: 9536.5, 1: 10047.8. Samples: 714990664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:33:11,062][104569] Avg episode reward: [(0, '8442.780'), (1, '6879.127')] [2023-12-27 01:33:11,119][105620] Updated weights for policy 1, policy_version 1397326 (0.0009) [2023-12-27 01:33:11,185][105620] Updated weights for policy 1, policy_version 1397336 (0.0009) [2023-12-27 01:33:11,653][105692] Updated weights for policy 0, policy_version 1395225 (0.0007) [2023-12-27 01:33:11,720][105692] Updated weights for policy 0, policy_version 1395235 (0.0008) [2023-12-27 01:33:11,784][105692] Updated weights for policy 0, policy_version 1395245 (0.0007) [2023-12-27 01:33:12,034][105620] Updated weights for policy 1, policy_version 1397346 (0.0009) [2023-12-27 01:33:12,096][105620] Updated weights for policy 1, policy_version 1397356 (0.0009) [2023-12-27 01:33:12,157][105620] Updated weights for policy 1, policy_version 1397366 (0.0008) [2023-12-27 01:33:12,491][105692] Updated weights for policy 0, policy_version 1395255 (0.0009) [2023-12-27 01:33:12,549][105692] Updated weights for policy 0, policy_version 1395265 (0.0008) [2023-12-27 01:33:12,606][105692] Updated weights for policy 0, policy_version 1395275 (0.0005) [2023-12-27 01:33:12,938][105620] Updated weights for policy 1, policy_version 1397376 (0.0009) [2023-12-27 01:33:12,992][105620] Updated weights for policy 1, policy_version 1397386 (0.0007) [2023-12-27 01:33:13,053][105620] Updated weights for policy 1, policy_version 1397396 (0.0009) [2023-12-27 01:33:13,311][105692] Updated weights for policy 0, policy_version 1395285 (0.0007) [2023-12-27 01:33:13,365][105692] Updated weights for policy 0, policy_version 1395295 (0.0010) [2023-12-27 01:33:13,424][105692] Updated weights for policy 0, policy_version 1395305 (0.0010) [2023-12-27 01:33:13,675][105620] Updated weights for policy 1, policy_version 1397406 (0.0007) [2023-12-27 01:33:13,729][105620] Updated weights for policy 1, policy_version 1397416 (0.0006) [2023-12-27 01:33:13,795][105620] Updated weights for policy 1, policy_version 1397426 (0.0008) [2023-12-27 01:33:14,186][105692] Updated weights for policy 0, policy_version 1395315 (0.0009) [2023-12-27 01:33:14,240][105692] Updated weights for policy 0, policy_version 1395325 (0.0008) [2023-12-27 01:33:14,300][105692] Updated weights for policy 0, policy_version 1395335 (0.0009) [2023-12-27 01:33:14,386][105620] Updated weights for policy 1, policy_version 1397436 (0.0006) [2023-12-27 01:33:14,436][105620] Updated weights for policy 1, policy_version 1397446 (0.0005) [2023-12-27 01:33:14,499][105620] Updated weights for policy 1, policy_version 1397456 (0.0007) [2023-12-27 01:33:15,117][105692] Updated weights for policy 0, policy_version 1395345 (0.0009) [2023-12-27 01:33:15,167][105692] Updated weights for policy 0, policy_version 1395355 (0.0008) [2023-12-27 01:33:15,198][105620] Updated weights for policy 1, policy_version 1397466 (0.0010) [2023-12-27 01:33:15,218][105692] Updated weights for policy 0, policy_version 1395365 (0.0009) [2023-12-27 01:33:15,253][105620] Updated weights for policy 1, policy_version 1397476 (0.0011) [2023-12-27 01:33:15,275][105692] Updated weights for policy 0, policy_version 1395375 (0.0006) [2023-12-27 01:33:15,312][105620] Updated weights for policy 1, policy_version 1397486 (0.0010) [2023-12-27 01:33:15,370][105620] Updated weights for policy 1, policy_version 1397496 (0.0010) [2023-12-27 01:33:16,055][105692] Updated weights for policy 0, policy_version 1395385 (0.0009) [2023-12-27 01:33:16,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19251.3, 300 sec: 19327.6). Total num frames: 715071488. Throughput: 0: 9489.6, 1: 9928.2. Samples: 715046752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:33:16,062][104569] Avg episode reward: [(0, '8627.162'), (1, '8567.977')] [2023-12-27 01:33:16,104][105692] Updated weights for policy 0, policy_version 1395395 (0.0008) [2023-12-27 01:33:16,115][105620] Updated weights for policy 1, policy_version 1397506 (0.0006) [2023-12-27 01:33:16,163][105692] Updated weights for policy 0, policy_version 1395405 (0.0007) [2023-12-27 01:33:16,165][105620] Updated weights for policy 1, policy_version 1397516 (0.0006) [2023-12-27 01:33:16,180][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001395408_357277696.pth... [2023-12-27 01:33:16,184][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001394288_356990976.pth [2023-12-27 01:33:16,218][105620] Updated weights for policy 1, policy_version 1397526 (0.0006) [2023-12-27 01:33:16,227][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001397528_357810176.pth... [2023-12-27 01:33:16,230][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001396344_357507072.pth [2023-12-27 01:33:16,928][105620] Updated weights for policy 1, policy_version 1397536 (0.0009) [2023-12-27 01:33:16,940][105692] Updated weights for policy 0, policy_version 1395415 (0.0007) [2023-12-27 01:33:16,978][105620] Updated weights for policy 1, policy_version 1397546 (0.0008) [2023-12-27 01:33:16,996][105692] Updated weights for policy 0, policy_version 1395425 (0.0007) [2023-12-27 01:33:17,026][105620] Updated weights for policy 1, policy_version 1397556 (0.0007) [2023-12-27 01:33:17,048][105692] Updated weights for policy 0, policy_version 1395435 (0.0007) [2023-12-27 01:33:17,798][105692] Updated weights for policy 0, policy_version 1395445 (0.0008) [2023-12-27 01:33:17,806][105620] Updated weights for policy 1, policy_version 1397566 (0.0006) [2023-12-27 01:33:17,862][105692] Updated weights for policy 0, policy_version 1395455 (0.0005) [2023-12-27 01:33:17,864][105620] Updated weights for policy 1, policy_version 1397576 (0.0008) [2023-12-27 01:33:17,924][105620] Updated weights for policy 1, policy_version 1397586 (0.0009) [2023-12-27 01:33:17,929][105692] Updated weights for policy 0, policy_version 1395465 (0.0006) [2023-12-27 01:33:18,571][105692] Updated weights for policy 0, policy_version 1395475 (0.0008) [2023-12-27 01:33:18,632][105692] Updated weights for policy 0, policy_version 1395485 (0.0009) [2023-12-27 01:33:18,657][105620] Updated weights for policy 1, policy_version 1397596 (0.0009) [2023-12-27 01:33:18,684][105692] Updated weights for policy 0, policy_version 1395495 (0.0006) [2023-12-27 01:33:18,714][105620] Updated weights for policy 1, policy_version 1397606 (0.0011) [2023-12-27 01:33:18,772][105620] Updated weights for policy 1, policy_version 1397616 (0.0008) [2023-12-27 01:33:19,478][105692] Updated weights for policy 0, policy_version 1395505 (0.0006) [2023-12-27 01:33:19,543][105620] Updated weights for policy 1, policy_version 1397626 (0.0008) [2023-12-27 01:33:19,544][105692] Updated weights for policy 0, policy_version 1395515 (0.0007) [2023-12-27 01:33:19,600][105692] Updated weights for policy 0, policy_version 1395525 (0.0008) [2023-12-27 01:33:19,606][105620] Updated weights for policy 1, policy_version 1397636 (0.0011) [2023-12-27 01:33:19,665][105692] Updated weights for policy 0, policy_version 1395535 (0.0005) [2023-12-27 01:33:19,666][105620] Updated weights for policy 1, policy_version 1397646 (0.0011) [2023-12-27 01:33:19,726][105620] Updated weights for policy 1, policy_version 1397656 (0.0011) [2023-12-27 01:33:20,470][105692] Updated weights for policy 0, policy_version 1395545 (0.0008) [2023-12-27 01:33:20,520][105620] Updated weights for policy 1, policy_version 1397666 (0.0011) [2023-12-27 01:33:20,523][105692] Updated weights for policy 0, policy_version 1395555 (0.0006) [2023-12-27 01:33:20,583][105692] Updated weights for policy 0, policy_version 1395565 (0.0008) [2023-12-27 01:33:20,584][105620] Updated weights for policy 1, policy_version 1397676 (0.0010) [2023-12-27 01:33:20,650][105620] Updated weights for policy 1, policy_version 1397686 (0.0011) [2023-12-27 01:33:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19327.6). Total num frames: 715169792. Throughput: 0: 9364.6, 1: 10016.3. Samples: 715160648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:33:21,063][104569] Avg episode reward: [(0, '8444.448'), (1, '9077.666')] [2023-12-27 01:33:21,389][105692] Updated weights for policy 0, policy_version 1395575 (0.0008) [2023-12-27 01:33:21,422][105620] Updated weights for policy 1, policy_version 1397696 (0.0008) [2023-12-27 01:33:21,454][105692] Updated weights for policy 0, policy_version 1395585 (0.0008) [2023-12-27 01:33:21,492][105620] Updated weights for policy 1, policy_version 1397706 (0.0007) [2023-12-27 01:33:21,505][105692] Updated weights for policy 0, policy_version 1395595 (0.0008) [2023-12-27 01:33:21,558][105620] Updated weights for policy 1, policy_version 1397716 (0.0009) [2023-12-27 01:33:22,231][105620] Updated weights for policy 1, policy_version 1397726 (0.0007) [2023-12-27 01:33:22,302][105620] Updated weights for policy 1, policy_version 1397736 (0.0008) [2023-12-27 01:33:22,347][105692] Updated weights for policy 0, policy_version 1395605 (0.0006) [2023-12-27 01:33:22,366][105620] Updated weights for policy 1, policy_version 1397746 (0.0008) [2023-12-27 01:33:22,416][105692] Updated weights for policy 0, policy_version 1395615 (0.0009) [2023-12-27 01:33:22,477][105692] Updated weights for policy 0, policy_version 1395625 (0.0008) [2023-12-27 01:33:23,072][105620] Updated weights for policy 1, policy_version 1397756 (0.0007) [2023-12-27 01:33:23,138][105620] Updated weights for policy 1, policy_version 1397766 (0.0009) [2023-12-27 01:33:23,176][105692] Updated weights for policy 0, policy_version 1395635 (0.0007) [2023-12-27 01:33:23,194][105620] Updated weights for policy 1, policy_version 1397776 (0.0008) [2023-12-27 01:33:23,237][105692] Updated weights for policy 0, policy_version 1395645 (0.0007) [2023-12-27 01:33:23,298][105692] Updated weights for policy 0, policy_version 1395655 (0.0009) [2023-12-27 01:33:23,942][105620] Updated weights for policy 1, policy_version 1397786 (0.0006) [2023-12-27 01:33:23,998][105620] Updated weights for policy 1, policy_version 1397796 (0.0008) [2023-12-27 01:33:24,056][105620] Updated weights for policy 1, policy_version 1397806 (0.0005) [2023-12-27 01:33:24,058][105692] Updated weights for policy 0, policy_version 1395665 (0.0009) [2023-12-27 01:33:24,110][105692] Updated weights for policy 0, policy_version 1395675 (0.0008) [2023-12-27 01:33:24,115][105620] Updated weights for policy 1, policy_version 1397816 (0.0006) [2023-12-27 01:33:24,170][105692] Updated weights for policy 0, policy_version 1395685 (0.0008) [2023-12-27 01:33:24,231][105692] Updated weights for policy 0, policy_version 1395695 (0.0008) [2023-12-27 01:33:24,807][105620] Updated weights for policy 1, policy_version 1397826 (0.0009) [2023-12-27 01:33:24,860][105620] Updated weights for policy 1, policy_version 1397836 (0.0010) [2023-12-27 01:33:24,914][105620] Updated weights for policy 1, policy_version 1397846 (0.0008) [2023-12-27 01:33:24,940][105692] Updated weights for policy 0, policy_version 1395705 (0.0006) [2023-12-27 01:33:24,998][105692] Updated weights for policy 0, policy_version 1395715 (0.0006) [2023-12-27 01:33:25,047][105692] Updated weights for policy 0, policy_version 1395725 (0.0011) [2023-12-27 01:33:25,581][105692] Updated weights for policy 0, policy_version 1395735 (0.0009) [2023-12-27 01:33:25,632][105692] Updated weights for policy 0, policy_version 1395745 (0.0010) [2023-12-27 01:33:25,690][105692] Updated weights for policy 0, policy_version 1395755 (0.0010) [2023-12-27 01:33:25,709][105620] Updated weights for policy 1, policy_version 1397856 (0.0010) [2023-12-27 01:33:25,763][105620] Updated weights for policy 1, policy_version 1397866 (0.0010) [2023-12-27 01:33:25,821][105620] Updated weights for policy 1, policy_version 1397876 (0.0010) [2023-12-27 01:33:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19327.6). Total num frames: 715268096. Throughput: 0: 9362.0, 1: 9969.7. Samples: 715273732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:33:26,063][104569] Avg episode reward: [(0, '7900.080'), (1, '9263.715')] [2023-12-27 01:33:26,367][105692] Updated weights for policy 0, policy_version 1395765 (0.0008) [2023-12-27 01:33:26,412][105692] Updated weights for policy 0, policy_version 1395775 (0.0006) [2023-12-27 01:33:26,457][105692] Updated weights for policy 0, policy_version 1395785 (0.0009) [2023-12-27 01:33:26,503][105620] Updated weights for policy 1, policy_version 1397886 (0.0009) [2023-12-27 01:33:26,549][105620] Updated weights for policy 1, policy_version 1397896 (0.0005) [2023-12-27 01:33:26,596][105620] Updated weights for policy 1, policy_version 1397906 (0.0005) [2023-12-27 01:33:27,201][105692] Updated weights for policy 0, policy_version 1395795 (0.0010) [2023-12-27 01:33:27,259][105692] Updated weights for policy 0, policy_version 1395805 (0.0010) [2023-12-27 01:33:27,318][105692] Updated weights for policy 0, policy_version 1395815 (0.0010) [2023-12-27 01:33:27,322][105620] Updated weights for policy 1, policy_version 1397916 (0.0007) [2023-12-27 01:33:27,370][105620] Updated weights for policy 1, policy_version 1397926 (0.0010) [2023-12-27 01:33:27,420][105620] Updated weights for policy 1, policy_version 1397936 (0.0010) [2023-12-27 01:33:27,948][105692] Updated weights for policy 0, policy_version 1395825 (0.0010) [2023-12-27 01:33:28,000][105692] Updated weights for policy 0, policy_version 1395835 (0.0008) [2023-12-27 01:33:28,051][105692] Updated weights for policy 0, policy_version 1395845 (0.0008) [2023-12-27 01:33:28,099][105692] Updated weights for policy 0, policy_version 1395855 (0.0007) [2023-12-27 01:33:28,178][105620] Updated weights for policy 1, policy_version 1397946 (0.0010) [2023-12-27 01:33:28,228][105620] Updated weights for policy 1, policy_version 1397956 (0.0010) [2023-12-27 01:33:28,276][105620] Updated weights for policy 1, policy_version 1397966 (0.0010) [2023-12-27 01:33:28,326][105620] Updated weights for policy 1, policy_version 1397976 (0.0010) [2023-12-27 01:33:28,857][105692] Updated weights for policy 0, policy_version 1395865 (0.0005) [2023-12-27 01:33:28,904][105692] Updated weights for policy 0, policy_version 1395875 (0.0005) [2023-12-27 01:33:28,950][105692] Updated weights for policy 0, policy_version 1395885 (0.0005) [2023-12-27 01:33:29,091][105620] Updated weights for policy 1, policy_version 1397986 (0.0010) [2023-12-27 01:33:29,150][105620] Updated weights for policy 1, policy_version 1397996 (0.0010) [2023-12-27 01:33:29,208][105620] Updated weights for policy 1, policy_version 1398006 (0.0010) [2023-12-27 01:33:29,657][105692] Updated weights for policy 0, policy_version 1395895 (0.0009) [2023-12-27 01:33:29,715][105692] Updated weights for policy 0, policy_version 1395905 (0.0010) [2023-12-27 01:33:29,762][105692] Updated weights for policy 0, policy_version 1395915 (0.0010) [2023-12-27 01:33:29,955][105620] Updated weights for policy 1, policy_version 1398016 (0.0009) [2023-12-27 01:33:30,021][105620] Updated weights for policy 1, policy_version 1398026 (0.0009) [2023-12-27 01:33:30,077][105620] Updated weights for policy 1, policy_version 1398036 (0.0008) [2023-12-27 01:33:30,542][105692] Updated weights for policy 0, policy_version 1395925 (0.0008) [2023-12-27 01:33:30,600][105692] Updated weights for policy 0, policy_version 1395935 (0.0011) [2023-12-27 01:33:30,648][105692] Updated weights for policy 0, policy_version 1395945 (0.0010) [2023-12-27 01:33:30,716][105620] Updated weights for policy 1, policy_version 1398046 (0.0008) [2023-12-27 01:33:30,764][105620] Updated weights for policy 1, policy_version 1398056 (0.0008) [2023-12-27 01:33:30,808][105620] Updated weights for policy 1, policy_version 1398066 (0.0007) [2023-12-27 01:33:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19327.6). Total num frames: 715366400. Throughput: 0: 9418.1, 1: 9930.8. Samples: 715333352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:33:31,062][104569] Avg episode reward: [(0, '7631.312'), (1, '9174.278')] [2023-12-27 01:33:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001395952_357416960.pth... [2023-12-27 01:33:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001398072_357949440.pth... [2023-12-27 01:33:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001394832_357130240.pth [2023-12-27 01:33:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001396952_357662720.pth [2023-12-27 01:33:31,396][105692] Updated weights for policy 0, policy_version 1395955 (0.0010) [2023-12-27 01:33:31,459][105692] Updated weights for policy 0, policy_version 1395965 (0.0009) [2023-12-27 01:33:31,515][105692] Updated weights for policy 0, policy_version 1395975 (0.0008) [2023-12-27 01:33:31,580][105620] Updated weights for policy 1, policy_version 1398076 (0.0009) [2023-12-27 01:33:31,644][105620] Updated weights for policy 1, policy_version 1398086 (0.0009) [2023-12-27 01:33:31,711][105620] Updated weights for policy 1, policy_version 1398096 (0.0009) [2023-12-27 01:33:32,283][105692] Updated weights for policy 0, policy_version 1395985 (0.0007) [2023-12-27 01:33:32,343][105692] Updated weights for policy 0, policy_version 1395995 (0.0011) [2023-12-27 01:33:32,410][105692] Updated weights for policy 0, policy_version 1396005 (0.0010) [2023-12-27 01:33:32,469][105692] Updated weights for policy 0, policy_version 1396015 (0.0011) [2023-12-27 01:33:32,490][105620] Updated weights for policy 1, policy_version 1398106 (0.0008) [2023-12-27 01:33:32,545][105620] Updated weights for policy 1, policy_version 1398116 (0.0010) [2023-12-27 01:33:32,589][105620] Updated weights for policy 1, policy_version 1398126 (0.0010) [2023-12-27 01:33:32,637][105620] Updated weights for policy 1, policy_version 1398136 (0.0010) [2023-12-27 01:33:33,103][105692] Updated weights for policy 0, policy_version 1396025 (0.0009) [2023-12-27 01:33:33,152][105692] Updated weights for policy 0, policy_version 1396035 (0.0005) [2023-12-27 01:33:33,215][105692] Updated weights for policy 0, policy_version 1396045 (0.0005) [2023-12-27 01:33:33,360][105620] Updated weights for policy 1, policy_version 1398146 (0.0008) [2023-12-27 01:33:33,403][105620] Updated weights for policy 1, policy_version 1398156 (0.0007) [2023-12-27 01:33:33,449][105620] Updated weights for policy 1, policy_version 1398166 (0.0007) [2023-12-27 01:33:33,887][105692] Updated weights for policy 0, policy_version 1396055 (0.0009) [2023-12-27 01:33:33,941][105692] Updated weights for policy 0, policy_version 1396065 (0.0010) [2023-12-27 01:33:34,000][105692] Updated weights for policy 0, policy_version 1396075 (0.0010) [2023-12-27 01:33:34,226][105620] Updated weights for policy 1, policy_version 1398176 (0.0008) [2023-12-27 01:33:34,281][105620] Updated weights for policy 1, policy_version 1398186 (0.0008) [2023-12-27 01:33:34,347][105620] Updated weights for policy 1, policy_version 1398196 (0.0008) [2023-12-27 01:33:34,750][105692] Updated weights for policy 0, policy_version 1396085 (0.0011) [2023-12-27 01:33:34,798][105692] Updated weights for policy 0, policy_version 1396095 (0.0010) [2023-12-27 01:33:34,806][105585] KL-divergence is very high: 114.1175 [2023-12-27 01:33:34,857][105585] KL-divergence is very high: 127.3865 [2023-12-27 01:33:34,858][105692] Updated weights for policy 0, policy_version 1396105 (0.0011) [2023-12-27 01:33:35,070][105620] Updated weights for policy 1, policy_version 1398206 (0.0009) [2023-12-27 01:33:35,119][105620] Updated weights for policy 1, policy_version 1398216 (0.0010) [2023-12-27 01:33:35,163][105620] Updated weights for policy 1, policy_version 1398226 (0.0010) [2023-12-27 01:33:35,626][105692] Updated weights for policy 0, policy_version 1396115 (0.0011) [2023-12-27 01:33:35,671][105692] Updated weights for policy 0, policy_version 1396125 (0.0010) [2023-12-27 01:33:35,715][105692] Updated weights for policy 0, policy_version 1396135 (0.0010) [2023-12-27 01:33:35,931][105620] Updated weights for policy 1, policy_version 1398236 (0.0008) [2023-12-27 01:33:35,990][105620] Updated weights for policy 1, policy_version 1398246 (0.0005) [2023-12-27 01:33:36,055][105620] Updated weights for policy 1, policy_version 1398256 (0.0006) [2023-12-27 01:33:36,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19327.6). Total num frames: 715456512. Throughput: 0: 9405.4, 1: 9822.9. Samples: 715448712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:33:36,062][104569] Avg episode reward: [(0, '8162.922'), (1, '8648.463')] [2023-12-27 01:33:36,492][105692] Updated weights for policy 0, policy_version 1396145 (0.0010) [2023-12-27 01:33:36,544][105692] Updated weights for policy 0, policy_version 1396155 (0.0010) [2023-12-27 01:33:36,613][105692] Updated weights for policy 0, policy_version 1396165 (0.0011) [2023-12-27 01:33:36,682][105692] Updated weights for policy 0, policy_version 1396175 (0.0011) [2023-12-27 01:33:36,685][105620] Updated weights for policy 1, policy_version 1398266 (0.0006) [2023-12-27 01:33:36,733][105620] Updated weights for policy 1, policy_version 1398276 (0.0008) [2023-12-27 01:33:36,782][105620] Updated weights for policy 1, policy_version 1398286 (0.0008) [2023-12-27 01:33:36,835][105620] Updated weights for policy 1, policy_version 1398296 (0.0008) [2023-12-27 01:33:37,419][105692] Updated weights for policy 0, policy_version 1396185 (0.0010) [2023-12-27 01:33:37,476][105692] Updated weights for policy 0, policy_version 1396195 (0.0011) [2023-12-27 01:33:37,527][105620] Updated weights for policy 1, policy_version 1398306 (0.0006) [2023-12-27 01:33:37,533][105692] Updated weights for policy 0, policy_version 1396205 (0.0011) [2023-12-27 01:33:37,590][105620] Updated weights for policy 1, policy_version 1398316 (0.0007) [2023-12-27 01:33:37,643][105620] Updated weights for policy 1, policy_version 1398326 (0.0008) [2023-12-27 01:33:38,237][105620] Updated weights for policy 1, policy_version 1398336 (0.0009) [2023-12-27 01:33:38,286][105620] Updated weights for policy 1, policy_version 1398346 (0.0010) [2023-12-27 01:33:38,291][105692] Updated weights for policy 0, policy_version 1396215 (0.0010) [2023-12-27 01:33:38,345][105620] Updated weights for policy 1, policy_version 1398356 (0.0010) [2023-12-27 01:33:38,360][105692] Updated weights for policy 0, policy_version 1396225 (0.0011) [2023-12-27 01:33:38,418][105692] Updated weights for policy 0, policy_version 1396235 (0.0011) [2023-12-27 01:33:39,063][105620] Updated weights for policy 1, policy_version 1398366 (0.0009) [2023-12-27 01:33:39,125][105620] Updated weights for policy 1, policy_version 1398376 (0.0007) [2023-12-27 01:33:39,155][105692] Updated weights for policy 0, policy_version 1396245 (0.0009) [2023-12-27 01:33:39,187][105620] Updated weights for policy 1, policy_version 1398386 (0.0006) [2023-12-27 01:33:39,221][105692] Updated weights for policy 0, policy_version 1396255 (0.0008) [2023-12-27 01:33:39,280][105692] Updated weights for policy 0, policy_version 1396265 (0.0008) [2023-12-27 01:33:39,985][105620] Updated weights for policy 1, policy_version 1398396 (0.0007) [2023-12-27 01:33:40,049][105620] Updated weights for policy 1, policy_version 1398406 (0.0008) [2023-12-27 01:33:40,060][105692] Updated weights for policy 0, policy_version 1396275 (0.0008) [2023-12-27 01:33:40,104][105620] Updated weights for policy 1, policy_version 1398416 (0.0006) [2023-12-27 01:33:40,119][105692] Updated weights for policy 0, policy_version 1396285 (0.0008) [2023-12-27 01:33:40,120][105585] KL-divergence is very high: 289.9589 [2023-12-27 01:33:40,168][105585] KL-divergence is very high: 564.0902 [2023-12-27 01:33:40,181][105692] Updated weights for policy 0, policy_version 1396295 (0.0008) [2023-12-27 01:33:40,208][105585] KL-divergence is very high: 667.2690 [2023-12-27 01:33:40,817][105620] Updated weights for policy 1, policy_version 1398426 (0.0007) [2023-12-27 01:33:40,879][105620] Updated weights for policy 1, policy_version 1398436 (0.0007) [2023-12-27 01:33:40,942][105620] Updated weights for policy 1, policy_version 1398446 (0.0005) [2023-12-27 01:33:40,972][105692] Updated weights for policy 0, policy_version 1396305 (0.0008) [2023-12-27 01:33:41,009][105620] Updated weights for policy 1, policy_version 1398456 (0.0008) [2023-12-27 01:33:41,023][105692] Updated weights for policy 0, policy_version 1396315 (0.0006) [2023-12-27 01:33:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19327.6). Total num frames: 715554816. Throughput: 0: 9461.1, 1: 9778.1. Samples: 715563660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:33:41,062][104569] Avg episode reward: [(0, '8164.217'), (1, '8073.587')] [2023-12-27 01:33:41,089][105692] Updated weights for policy 0, policy_version 1396325 (0.0008) [2023-12-27 01:33:41,158][105692] Updated weights for policy 0, policy_version 1396335 (0.0008) [2023-12-27 01:33:41,681][105620] Updated weights for policy 1, policy_version 1398466 (0.0008) [2023-12-27 01:33:41,743][105620] Updated weights for policy 1, policy_version 1398476 (0.0007) [2023-12-27 01:33:41,794][105620] Updated weights for policy 1, policy_version 1398486 (0.0005) [2023-12-27 01:33:41,970][105692] Updated weights for policy 0, policy_version 1396345 (0.0008) [2023-12-27 01:33:42,033][105692] Updated weights for policy 0, policy_version 1396355 (0.0008) [2023-12-27 01:33:42,100][105692] Updated weights for policy 0, policy_version 1396365 (0.0008) [2023-12-27 01:33:42,394][105620] Updated weights for policy 1, policy_version 1398496 (0.0006) [2023-12-27 01:33:42,459][105620] Updated weights for policy 1, policy_version 1398506 (0.0008) [2023-12-27 01:33:42,520][105620] Updated weights for policy 1, policy_version 1398516 (0.0006) [2023-12-27 01:33:42,896][105692] Updated weights for policy 0, policy_version 1396375 (0.0006) [2023-12-27 01:33:42,959][105692] Updated weights for policy 0, policy_version 1396385 (0.0008) [2023-12-27 01:33:43,022][105692] Updated weights for policy 0, policy_version 1396395 (0.0008) [2023-12-27 01:33:43,182][105620] Updated weights for policy 1, policy_version 1398526 (0.0009) [2023-12-27 01:33:43,238][105620] Updated weights for policy 1, policy_version 1398536 (0.0011) [2023-12-27 01:33:43,308][105620] Updated weights for policy 1, policy_version 1398546 (0.0010) [2023-12-27 01:33:43,603][105692] Updated weights for policy 0, policy_version 1396405 (0.0007) [2023-12-27 01:33:43,658][105692] Updated weights for policy 0, policy_version 1396415 (0.0005) [2023-12-27 01:33:43,704][105692] Updated weights for policy 0, policy_version 1396425 (0.0005) [2023-12-27 01:33:44,050][105620] Updated weights for policy 1, policy_version 1398556 (0.0011) [2023-12-27 01:33:44,112][105620] Updated weights for policy 1, policy_version 1398566 (0.0009) [2023-12-27 01:33:44,167][105620] Updated weights for policy 1, policy_version 1398576 (0.0010) [2023-12-27 01:33:44,423][105692] Updated weights for policy 0, policy_version 1396435 (0.0007) [2023-12-27 01:33:44,477][105692] Updated weights for policy 0, policy_version 1396445 (0.0008) [2023-12-27 01:33:44,522][105692] Updated weights for policy 0, policy_version 1396455 (0.0008) [2023-12-27 01:33:44,792][105620] Updated weights for policy 1, policy_version 1398586 (0.0006) [2023-12-27 01:33:44,852][105620] Updated weights for policy 1, policy_version 1398596 (0.0008) [2023-12-27 01:33:44,912][105620] Updated weights for policy 1, policy_version 1398606 (0.0009) [2023-12-27 01:33:44,975][105620] Updated weights for policy 1, policy_version 1398616 (0.0010) [2023-12-27 01:33:45,211][105692] Updated weights for policy 0, policy_version 1396465 (0.0008) [2023-12-27 01:33:45,267][105692] Updated weights for policy 0, policy_version 1396475 (0.0009) [2023-12-27 01:33:45,319][105692] Updated weights for policy 0, policy_version 1396485 (0.0009) [2023-12-27 01:33:45,374][105692] Updated weights for policy 0, policy_version 1396495 (0.0009) [2023-12-27 01:33:45,735][105620] Updated weights for policy 1, policy_version 1398626 (0.0009) [2023-12-27 01:33:45,780][105620] Updated weights for policy 1, policy_version 1398636 (0.0008) [2023-12-27 01:33:45,826][105620] Updated weights for policy 1, policy_version 1398646 (0.0008) [2023-12-27 01:33:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.8, 300 sec: 19299.8). Total num frames: 715653120. Throughput: 0: 9435.9, 1: 9766.7. Samples: 715622656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:33:46,063][104569] Avg episode reward: [(0, '8719.173'), (1, '7791.562')] [2023-12-27 01:33:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001396496_357556224.pth... [2023-12-27 01:33:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001398648_358096896.pth... [2023-12-27 01:33:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001395408_357277696.pth [2023-12-27 01:33:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001397528_357810176.pth [2023-12-27 01:33:46,135][105692] Updated weights for policy 0, policy_version 1396505 (0.0009) [2023-12-27 01:33:46,187][105692] Updated weights for policy 0, policy_version 1396515 (0.0009) [2023-12-27 01:33:46,233][105692] Updated weights for policy 0, policy_version 1396525 (0.0008) [2023-12-27 01:33:46,617][105620] Updated weights for policy 1, policy_version 1398656 (0.0009) [2023-12-27 01:33:46,669][105620] Updated weights for policy 1, policy_version 1398666 (0.0009) [2023-12-27 01:33:46,719][105620] Updated weights for policy 1, policy_version 1398677 (0.0010) [2023-12-27 01:33:46,922][105692] Updated weights for policy 0, policy_version 1396535 (0.0006) [2023-12-27 01:33:46,980][105692] Updated weights for policy 0, policy_version 1396545 (0.0005) [2023-12-27 01:33:47,035][105692] Updated weights for policy 0, policy_version 1396555 (0.0005) [2023-12-27 01:33:47,592][105692] Updated weights for policy 0, policy_version 1396565 (0.0009) [2023-12-27 01:33:47,612][105620] Updated weights for policy 1, policy_version 1398687 (0.0010) [2023-12-27 01:33:47,649][105692] Updated weights for policy 0, policy_version 1396575 (0.0005) [2023-12-27 01:33:47,673][105620] Updated weights for policy 1, policy_version 1398697 (0.0008) [2023-12-27 01:33:47,697][105692] Updated weights for policy 0, policy_version 1396585 (0.0005) [2023-12-27 01:33:47,726][105620] Updated weights for policy 1, policy_version 1398707 (0.0008) [2023-12-27 01:33:48,389][105692] Updated weights for policy 0, policy_version 1396595 (0.0007) [2023-12-27 01:33:48,449][105692] Updated weights for policy 0, policy_version 1396605 (0.0007) [2023-12-27 01:33:48,507][105692] Updated weights for policy 0, policy_version 1396615 (0.0007) [2023-12-27 01:33:48,521][105620] Updated weights for policy 1, policy_version 1398717 (0.0007) [2023-12-27 01:33:48,587][105620] Updated weights for policy 1, policy_version 1398727 (0.0007) [2023-12-27 01:33:48,651][105620] Updated weights for policy 1, policy_version 1398737 (0.0007) [2023-12-27 01:33:49,228][105692] Updated weights for policy 0, policy_version 1396625 (0.0007) [2023-12-27 01:33:49,295][105692] Updated weights for policy 0, policy_version 1396635 (0.0009) [2023-12-27 01:33:49,362][105692] Updated weights for policy 0, policy_version 1396645 (0.0008) [2023-12-27 01:33:49,423][105620] Updated weights for policy 1, policy_version 1398747 (0.0008) [2023-12-27 01:33:49,425][105692] Updated weights for policy 0, policy_version 1396655 (0.0007) [2023-12-27 01:33:49,481][105620] Updated weights for policy 1, policy_version 1398757 (0.0008) [2023-12-27 01:33:49,543][105620] Updated weights for policy 1, policy_version 1398767 (0.0008) [2023-12-27 01:33:50,172][105692] Updated weights for policy 0, policy_version 1396665 (0.0010) [2023-12-27 01:33:50,231][105692] Updated weights for policy 0, policy_version 1396675 (0.0010) [2023-12-27 01:33:50,288][105692] Updated weights for policy 0, policy_version 1396685 (0.0011) [2023-12-27 01:33:50,294][105620] Updated weights for policy 1, policy_version 1398777 (0.0006) [2023-12-27 01:33:50,360][105620] Updated weights for policy 1, policy_version 1398787 (0.0008) [2023-12-27 01:33:50,421][105620] Updated weights for policy 1, policy_version 1398797 (0.0007) [2023-12-27 01:33:50,483][105620] Updated weights for policy 1, policy_version 1398807 (0.0008) [2023-12-27 01:33:51,055][105692] Updated weights for policy 0, policy_version 1396695 (0.0009) [2023-12-27 01:33:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19299.8). Total num frames: 715743232. Throughput: 0: 9533.2, 1: 9583.3. Samples: 715737488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:33:51,062][104569] Avg episode reward: [(0, '8540.396'), (1, '8388.765')] [2023-12-27 01:33:51,118][105692] Updated weights for policy 0, policy_version 1396705 (0.0006) [2023-12-27 01:33:51,174][105692] Updated weights for policy 0, policy_version 1396715 (0.0010) [2023-12-27 01:33:51,257][105620] Updated weights for policy 1, policy_version 1398817 (0.0011) [2023-12-27 01:33:51,309][105620] Updated weights for policy 1, policy_version 1398827 (0.0010) [2023-12-27 01:33:51,389][105620] Updated weights for policy 1, policy_version 1398837 (0.0011) [2023-12-27 01:33:51,919][105692] Updated weights for policy 0, policy_version 1396725 (0.0010) [2023-12-27 01:33:51,987][105692] Updated weights for policy 0, policy_version 1396735 (0.0006) [2023-12-27 01:33:52,052][105692] Updated weights for policy 0, policy_version 1396745 (0.0006) [2023-12-27 01:33:52,137][105620] Updated weights for policy 1, policy_version 1398847 (0.0007) [2023-12-27 01:33:52,189][105620] Updated weights for policy 1, policy_version 1398857 (0.0005) [2023-12-27 01:33:52,248][105620] Updated weights for policy 1, policy_version 1398867 (0.0007) [2023-12-27 01:33:52,617][105692] Updated weights for policy 0, policy_version 1396755 (0.0008) [2023-12-27 01:33:52,677][105692] Updated weights for policy 0, policy_version 1396765 (0.0009) [2023-12-27 01:33:52,737][105692] Updated weights for policy 0, policy_version 1396775 (0.0008) [2023-12-27 01:33:52,994][105620] Updated weights for policy 1, policy_version 1398877 (0.0011) [2023-12-27 01:33:53,050][105620] Updated weights for policy 1, policy_version 1398887 (0.0011) [2023-12-27 01:33:53,099][105620] Updated weights for policy 1, policy_version 1398897 (0.0010) [2023-12-27 01:33:53,334][105692] Updated weights for policy 0, policy_version 1396785 (0.0008) [2023-12-27 01:33:53,391][105692] Updated weights for policy 0, policy_version 1396795 (0.0007) [2023-12-27 01:33:53,445][105692] Updated weights for policy 0, policy_version 1396805 (0.0005) [2023-12-27 01:33:53,503][105692] Updated weights for policy 0, policy_version 1396815 (0.0005) [2023-12-27 01:33:53,845][105620] Updated weights for policy 1, policy_version 1398907 (0.0009) [2023-12-27 01:33:53,896][105620] Updated weights for policy 1, policy_version 1398917 (0.0005) [2023-12-27 01:33:53,955][105620] Updated weights for policy 1, policy_version 1398927 (0.0005) [2023-12-27 01:33:54,157][105692] Updated weights for policy 0, policy_version 1396825 (0.0009) [2023-12-27 01:33:54,204][105692] Updated weights for policy 0, policy_version 1396835 (0.0009) [2023-12-27 01:33:54,255][105692] Updated weights for policy 0, policy_version 1396845 (0.0009) [2023-12-27 01:33:54,713][105620] Updated weights for policy 1, policy_version 1398937 (0.0007) [2023-12-27 01:33:54,776][105620] Updated weights for policy 1, policy_version 1398947 (0.0007) [2023-12-27 01:33:54,837][105620] Updated weights for policy 1, policy_version 1398957 (0.0008) [2023-12-27 01:33:54,893][105620] Updated weights for policy 1, policy_version 1398967 (0.0008) [2023-12-27 01:33:54,903][105692] Updated weights for policy 0, policy_version 1396855 (0.0009) [2023-12-27 01:33:54,954][105692] Updated weights for policy 0, policy_version 1396865 (0.0009) [2023-12-27 01:33:55,017][105692] Updated weights for policy 0, policy_version 1396875 (0.0009) [2023-12-27 01:33:55,633][105620] Updated weights for policy 1, policy_version 1398977 (0.0009) [2023-12-27 01:33:55,687][105620] Updated weights for policy 1, policy_version 1398987 (0.0009) [2023-12-27 01:33:55,739][105692] Updated weights for policy 0, policy_version 1396885 (0.0009) [2023-12-27 01:33:55,740][105620] Updated weights for policy 1, policy_version 1398997 (0.0009) [2023-12-27 01:33:55,798][105692] Updated weights for policy 0, policy_version 1396895 (0.0009) [2023-12-27 01:33:55,855][105692] Updated weights for policy 0, policy_version 1396905 (0.0008) [2023-12-27 01:33:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.8, 300 sec: 19327.6). Total num frames: 715849728. Throughput: 0: 9636.7, 1: 9546.5. Samples: 715853908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:33:56,062][104569] Avg episode reward: [(0, '8541.765'), (1, '9080.422')] [2023-12-27 01:33:56,390][105620] Updated weights for policy 1, policy_version 1399007 (0.0007) [2023-12-27 01:33:56,455][105620] Updated weights for policy 1, policy_version 1399017 (0.0006) [2023-12-27 01:33:56,508][105620] Updated weights for policy 1, policy_version 1399027 (0.0009) [2023-12-27 01:33:56,628][105692] Updated weights for policy 0, policy_version 1396915 (0.0008) [2023-12-27 01:33:56,685][105692] Updated weights for policy 0, policy_version 1396925 (0.0005) [2023-12-27 01:33:56,740][105692] Updated weights for policy 0, policy_version 1396935 (0.0010) [2023-12-27 01:33:57,302][105620] Updated weights for policy 1, policy_version 1399037 (0.0010) [2023-12-27 01:33:57,330][105692] Updated weights for policy 0, policy_version 1396945 (0.0009) [2023-12-27 01:33:57,362][105620] Updated weights for policy 1, policy_version 1399047 (0.0008) [2023-12-27 01:33:57,393][105692] Updated weights for policy 0, policy_version 1396955 (0.0007) [2023-12-27 01:33:57,419][105620] Updated weights for policy 1, policy_version 1399057 (0.0007) [2023-12-27 01:33:57,454][105692] Updated weights for policy 0, policy_version 1396965 (0.0006) [2023-12-27 01:33:57,499][105692] Updated weights for policy 0, policy_version 1396975 (0.0008) [2023-12-27 01:33:58,158][105692] Updated weights for policy 0, policy_version 1396985 (0.0009) [2023-12-27 01:33:58,203][105620] Updated weights for policy 1, policy_version 1399067 (0.0007) [2023-12-27 01:33:58,218][105692] Updated weights for policy 0, policy_version 1396995 (0.0007) [2023-12-27 01:33:58,262][105620] Updated weights for policy 1, policy_version 1399077 (0.0008) [2023-12-27 01:33:58,268][105692] Updated weights for policy 0, policy_version 1397005 (0.0008) [2023-12-27 01:33:58,337][105620] Updated weights for policy 1, policy_version 1399087 (0.0008) [2023-12-27 01:33:59,089][105692] Updated weights for policy 0, policy_version 1397015 (0.0008) [2023-12-27 01:33:59,155][105692] Updated weights for policy 0, policy_version 1397025 (0.0008) [2023-12-27 01:33:59,168][105620] Updated weights for policy 1, policy_version 1399097 (0.0008) [2023-12-27 01:33:59,222][105692] Updated weights for policy 0, policy_version 1397035 (0.0011) [2023-12-27 01:33:59,230][105620] Updated weights for policy 1, policy_version 1399107 (0.0008) [2023-12-27 01:33:59,294][105620] Updated weights for policy 1, policy_version 1399117 (0.0010) [2023-12-27 01:33:59,364][105620] Updated weights for policy 1, policy_version 1399127 (0.0007) [2023-12-27 01:33:59,929][105692] Updated weights for policy 0, policy_version 1397045 (0.0009) [2023-12-27 01:33:59,994][105692] Updated weights for policy 0, policy_version 1397055 (0.0010) [2023-12-27 01:34:00,056][105692] Updated weights for policy 0, policy_version 1397065 (0.0010) [2023-12-27 01:34:00,086][105620] Updated weights for policy 1, policy_version 1399137 (0.0005) [2023-12-27 01:34:00,140][105620] Updated weights for policy 1, policy_version 1399147 (0.0008) [2023-12-27 01:34:00,193][105620] Updated weights for policy 1, policy_version 1399157 (0.0008) [2023-12-27 01:34:00,770][105692] Updated weights for policy 0, policy_version 1397075 (0.0009) [2023-12-27 01:34:00,824][105692] Updated weights for policy 0, policy_version 1397085 (0.0005) [2023-12-27 01:34:00,876][105692] Updated weights for policy 0, policy_version 1397095 (0.0005) [2023-12-27 01:34:00,992][105620] Updated weights for policy 1, policy_version 1399167 (0.0009) [2023-12-27 01:34:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19327.6). Total num frames: 715939840. Throughput: 0: 9692.5, 1: 9536.9. Samples: 715912080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:34:01,063][105620] Updated weights for policy 1, policy_version 1399177 (0.0008) [2023-12-27 01:34:01,063][104569] Avg episode reward: [(0, '8444.836'), (1, '9172.675')] [2023-12-27 01:34:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001397104_357711872.pth... [2023-12-27 01:34:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001395952_357416960.pth [2023-12-27 01:34:01,123][105620] Updated weights for policy 1, policy_version 1399187 (0.0008) [2023-12-27 01:34:01,156][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001399192_358236160.pth... [2023-12-27 01:34:01,161][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001398072_357949440.pth [2023-12-27 01:34:01,481][105692] Updated weights for policy 0, policy_version 1397105 (0.0005) [2023-12-27 01:34:01,541][105692] Updated weights for policy 0, policy_version 1397115 (0.0005) [2023-12-27 01:34:01,608][105692] Updated weights for policy 0, policy_version 1397125 (0.0006) [2023-12-27 01:34:01,691][105692] Updated weights for policy 0, policy_version 1397135 (0.0008) [2023-12-27 01:34:01,929][105620] Updated weights for policy 1, policy_version 1399197 (0.0008) [2023-12-27 01:34:01,983][105620] Updated weights for policy 1, policy_version 1399207 (0.0009) [2023-12-27 01:34:02,040][105620] Updated weights for policy 1, policy_version 1399217 (0.0009) [2023-12-27 01:34:02,285][105692] Updated weights for policy 0, policy_version 1397145 (0.0010) [2023-12-27 01:34:02,344][105692] Updated weights for policy 0, policy_version 1397155 (0.0010) [2023-12-27 01:34:02,409][105692] Updated weights for policy 0, policy_version 1397165 (0.0009) [2023-12-27 01:34:02,807][105620] Updated weights for policy 1, policy_version 1399227 (0.0009) [2023-12-27 01:34:02,862][105620] Updated weights for policy 1, policy_version 1399237 (0.0010) [2023-12-27 01:34:02,915][105620] Updated weights for policy 1, policy_version 1399247 (0.0009) [2023-12-27 01:34:02,996][105692] Updated weights for policy 0, policy_version 1397175 (0.0010) [2023-12-27 01:34:03,055][105692] Updated weights for policy 0, policy_version 1397185 (0.0010) [2023-12-27 01:34:03,122][105692] Updated weights for policy 0, policy_version 1397195 (0.0011) [2023-12-27 01:34:03,629][105620] Updated weights for policy 1, policy_version 1399258 (0.0009) [2023-12-27 01:34:03,681][105620] Updated weights for policy 1, policy_version 1399268 (0.0010) [2023-12-27 01:34:03,743][105620] Updated weights for policy 1, policy_version 1399278 (0.0010) [2023-12-27 01:34:03,798][105620] Updated weights for policy 1, policy_version 1399288 (0.0007) [2023-12-27 01:34:03,863][105692] Updated weights for policy 0, policy_version 1397205 (0.0011) [2023-12-27 01:34:03,920][105692] Updated weights for policy 0, policy_version 1397215 (0.0011) [2023-12-27 01:34:03,973][105692] Updated weights for policy 0, policy_version 1397225 (0.0011) [2023-12-27 01:34:04,551][105620] Updated weights for policy 1, policy_version 1399298 (0.0011) [2023-12-27 01:34:04,613][105620] Updated weights for policy 1, policy_version 1399308 (0.0010) [2023-12-27 01:34:04,679][105620] Updated weights for policy 1, policy_version 1399318 (0.0007) [2023-12-27 01:34:04,765][105692] Updated weights for policy 0, policy_version 1397235 (0.0011) [2023-12-27 01:34:04,823][105692] Updated weights for policy 0, policy_version 1397245 (0.0010) [2023-12-27 01:34:04,877][105692] Updated weights for policy 0, policy_version 1397255 (0.0010) [2023-12-27 01:34:05,400][105620] Updated weights for policy 1, policy_version 1399328 (0.0006) [2023-12-27 01:34:05,447][105620] Updated weights for policy 1, policy_version 1399338 (0.0009) [2023-12-27 01:34:05,494][105620] Updated weights for policy 1, policy_version 1399348 (0.0008) [2023-12-27 01:34:05,622][105692] Updated weights for policy 0, policy_version 1397265 (0.0010) [2023-12-27 01:34:05,680][105692] Updated weights for policy 0, policy_version 1397275 (0.0009) [2023-12-27 01:34:05,735][105692] Updated weights for policy 0, policy_version 1397285 (0.0009) [2023-12-27 01:34:05,786][105692] Updated weights for policy 0, policy_version 1397295 (0.0009) [2023-12-27 01:34:06,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19114.6, 300 sec: 19299.8). Total num frames: 716038144. Throughput: 0: 9787.2, 1: 9478.7. Samples: 716027616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:34:06,063][104569] Avg episode reward: [(0, '8533.226'), (1, '9356.874')] [2023-12-27 01:34:06,156][105620] Updated weights for policy 1, policy_version 1399358 (0.0008) [2023-12-27 01:34:06,214][105620] Updated weights for policy 1, policy_version 1399368 (0.0006) [2023-12-27 01:34:06,283][105620] Updated weights for policy 1, policy_version 1399378 (0.0006) [2023-12-27 01:34:06,643][105692] Updated weights for policy 0, policy_version 1397305 (0.0011) [2023-12-27 01:34:06,706][105692] Updated weights for policy 0, policy_version 1397315 (0.0008) [2023-12-27 01:34:06,765][105692] Updated weights for policy 0, policy_version 1397325 (0.0007) [2023-12-27 01:34:06,922][105620] Updated weights for policy 1, policy_version 1399388 (0.0008) [2023-12-27 01:34:06,973][105620] Updated weights for policy 1, policy_version 1399398 (0.0008) [2023-12-27 01:34:07,025][105620] Updated weights for policy 1, policy_version 1399408 (0.0008) [2023-12-27 01:34:07,494][105692] Updated weights for policy 0, policy_version 1397335 (0.0011) [2023-12-27 01:34:07,557][105692] Updated weights for policy 0, policy_version 1397345 (0.0007) [2023-12-27 01:34:07,611][105692] Updated weights for policy 0, policy_version 1397355 (0.0006) [2023-12-27 01:34:07,741][105620] Updated weights for policy 1, policy_version 1399418 (0.0008) [2023-12-27 01:34:07,787][105620] Updated weights for policy 1, policy_version 1399428 (0.0005) [2023-12-27 01:34:07,836][105620] Updated weights for policy 1, policy_version 1399438 (0.0005) [2023-12-27 01:34:07,884][105620] Updated weights for policy 1, policy_version 1399448 (0.0005) [2023-12-27 01:34:08,151][105692] Updated weights for policy 0, policy_version 1397365 (0.0007) [2023-12-27 01:34:08,199][105692] Updated weights for policy 0, policy_version 1397375 (0.0006) [2023-12-27 01:34:08,258][105692] Updated weights for policy 0, policy_version 1397385 (0.0005) [2023-12-27 01:34:08,450][105620] Updated weights for policy 1, policy_version 1399458 (0.0005) [2023-12-27 01:34:08,509][105620] Updated weights for policy 1, policy_version 1399468 (0.0006) [2023-12-27 01:34:08,560][105620] Updated weights for policy 1, policy_version 1399478 (0.0007) [2023-12-27 01:34:08,966][105692] Updated weights for policy 0, policy_version 1397395 (0.0008) [2023-12-27 01:34:09,029][105692] Updated weights for policy 0, policy_version 1397405 (0.0011) [2023-12-27 01:34:09,086][105692] Updated weights for policy 0, policy_version 1397415 (0.0011) [2023-12-27 01:34:09,267][105620] Updated weights for policy 1, policy_version 1399488 (0.0009) [2023-12-27 01:34:09,330][105620] Updated weights for policy 1, policy_version 1399498 (0.0008) [2023-12-27 01:34:09,389][105620] Updated weights for policy 1, policy_version 1399508 (0.0008) [2023-12-27 01:34:09,852][105692] Updated weights for policy 0, policy_version 1397425 (0.0010) [2023-12-27 01:34:09,914][105692] Updated weights for policy 0, policy_version 1397435 (0.0010) [2023-12-27 01:34:09,980][105692] Updated weights for policy 0, policy_version 1397445 (0.0008) [2023-12-27 01:34:10,044][105692] Updated weights for policy 0, policy_version 1397455 (0.0008) [2023-12-27 01:34:10,165][105620] Updated weights for policy 1, policy_version 1399518 (0.0009) [2023-12-27 01:34:10,232][105620] Updated weights for policy 1, policy_version 1399528 (0.0011) [2023-12-27 01:34:10,299][105620] Updated weights for policy 1, policy_version 1399538 (0.0011) [2023-12-27 01:34:10,832][105692] Updated weights for policy 0, policy_version 1397465 (0.0008) [2023-12-27 01:34:10,889][105692] Updated weights for policy 0, policy_version 1397475 (0.0007) [2023-12-27 01:34:10,939][105692] Updated weights for policy 0, policy_version 1397485 (0.0007) [2023-12-27 01:34:10,965][105620] Updated weights for policy 1, policy_version 1399548 (0.0010) [2023-12-27 01:34:11,027][105620] Updated weights for policy 1, policy_version 1399558 (0.0010) [2023-12-27 01:34:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19327.6). Total num frames: 716136448. Throughput: 0: 9789.5, 1: 9589.1. Samples: 716145772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:34:11,063][104569] Avg episode reward: [(0, '8724.762'), (1, '9357.095')] [2023-12-27 01:34:11,087][105620] Updated weights for policy 1, policy_version 1399568 (0.0011) [2023-12-27 01:34:11,717][105692] Updated weights for policy 0, policy_version 1397495 (0.0008) [2023-12-27 01:34:11,784][105692] Updated weights for policy 0, policy_version 1397505 (0.0008) [2023-12-27 01:34:11,837][105692] Updated weights for policy 0, policy_version 1397515 (0.0008) [2023-12-27 01:34:11,853][105620] Updated weights for policy 1, policy_version 1399578 (0.0010) [2023-12-27 01:34:11,920][105620] Updated weights for policy 1, policy_version 1399588 (0.0010) [2023-12-27 01:34:11,979][105620] Updated weights for policy 1, policy_version 1399598 (0.0010) [2023-12-27 01:34:12,042][105620] Updated weights for policy 1, policy_version 1399608 (0.0010) [2023-12-27 01:34:12,573][105692] Updated weights for policy 0, policy_version 1397525 (0.0008) [2023-12-27 01:34:12,637][105692] Updated weights for policy 0, policy_version 1397535 (0.0010) [2023-12-27 01:34:12,697][105692] Updated weights for policy 0, policy_version 1397545 (0.0011) [2023-12-27 01:34:12,726][105620] Updated weights for policy 1, policy_version 1399618 (0.0010) [2023-12-27 01:34:12,785][105620] Updated weights for policy 1, policy_version 1399628 (0.0011) [2023-12-27 01:34:12,843][105620] Updated weights for policy 1, policy_version 1399638 (0.0010) [2023-12-27 01:34:13,449][105692] Updated weights for policy 0, policy_version 1397555 (0.0010) [2023-12-27 01:34:13,464][105620] Updated weights for policy 1, policy_version 1399648 (0.0010) [2023-12-27 01:34:13,504][105692] Updated weights for policy 0, policy_version 1397565 (0.0010) [2023-12-27 01:34:13,512][105620] Updated weights for policy 1, policy_version 1399658 (0.0007) [2023-12-27 01:34:13,564][105692] Updated weights for policy 0, policy_version 1397575 (0.0010) [2023-12-27 01:34:13,592][105620] Updated weights for policy 1, policy_version 1399668 (0.0009) [2023-12-27 01:34:14,162][105620] Updated weights for policy 1, policy_version 1399678 (0.0011) [2023-12-27 01:34:14,220][105620] Updated weights for policy 1, policy_version 1399688 (0.0010) [2023-12-27 01:34:14,265][105620] Updated weights for policy 1, policy_version 1399698 (0.0010) [2023-12-27 01:34:14,292][105692] Updated weights for policy 0, policy_version 1397585 (0.0007) [2023-12-27 01:34:14,355][105692] Updated weights for policy 0, policy_version 1397595 (0.0011) [2023-12-27 01:34:14,407][105692] Updated weights for policy 0, policy_version 1397605 (0.0011) [2023-12-27 01:34:14,456][105692] Updated weights for policy 0, policy_version 1397615 (0.0010) [2023-12-27 01:34:15,055][105620] Updated weights for policy 1, policy_version 1399708 (0.0010) [2023-12-27 01:34:15,119][105620] Updated weights for policy 1, policy_version 1399718 (0.0005) [2023-12-27 01:34:15,142][105692] Updated weights for policy 0, policy_version 1397625 (0.0011) [2023-12-27 01:34:15,182][105620] Updated weights for policy 1, policy_version 1399728 (0.0006) [2023-12-27 01:34:15,199][105692] Updated weights for policy 0, policy_version 1397635 (0.0011) [2023-12-27 01:34:15,252][105692] Updated weights for policy 0, policy_version 1397645 (0.0011) [2023-12-27 01:34:15,825][105620] Updated weights for policy 1, policy_version 1399738 (0.0010) [2023-12-27 01:34:15,874][105620] Updated weights for policy 1, policy_version 1399748 (0.0010) [2023-12-27 01:34:15,922][105692] Updated weights for policy 0, policy_version 1397655 (0.0011) [2023-12-27 01:34:15,923][105620] Updated weights for policy 1, policy_version 1399758 (0.0008) [2023-12-27 01:34:15,971][105692] Updated weights for policy 0, policy_version 1397665 (0.0010) [2023-12-27 01:34:15,973][105620] Updated weights for policy 1, policy_version 1399768 (0.0005) [2023-12-27 01:34:16,019][105692] Updated weights for policy 0, policy_version 1397675 (0.0010) [2023-12-27 01:34:16,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.2, 300 sec: 19383.1). Total num frames: 716242944. Throughput: 0: 9726.7, 1: 9611.7. Samples: 716203584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:34:16,063][104569] Avg episode reward: [(0, '8632.722'), (1, '9174.848')] [2023-12-27 01:34:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001397680_357859328.pth... [2023-12-27 01:34:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001399768_358383616.pth... [2023-12-27 01:34:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001398648_358096896.pth [2023-12-27 01:34:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001396496_357556224.pth [2023-12-27 01:34:16,691][105620] Updated weights for policy 1, policy_version 1399778 (0.0006) [2023-12-27 01:34:16,742][105620] Updated weights for policy 1, policy_version 1399788 (0.0007) [2023-12-27 01:34:16,751][105692] Updated weights for policy 0, policy_version 1397685 (0.0010) [2023-12-27 01:34:16,797][105620] Updated weights for policy 1, policy_version 1399798 (0.0006) [2023-12-27 01:34:16,799][105692] Updated weights for policy 0, policy_version 1397695 (0.0010) [2023-12-27 01:34:16,867][105692] Updated weights for policy 0, policy_version 1397705 (0.0010) [2023-12-27 01:34:17,366][105620] Updated weights for policy 1, policy_version 1399808 (0.0005) [2023-12-27 01:34:17,434][105620] Updated weights for policy 1, policy_version 1399818 (0.0005) [2023-12-27 01:34:17,498][105620] Updated weights for policy 1, policy_version 1399828 (0.0006) [2023-12-27 01:34:17,593][105692] Updated weights for policy 0, policy_version 1397715 (0.0010) [2023-12-27 01:34:17,666][105692] Updated weights for policy 0, policy_version 1397725 (0.0011) [2023-12-27 01:34:17,725][105692] Updated weights for policy 0, policy_version 1397735 (0.0010) [2023-12-27 01:34:18,100][105620] Updated weights for policy 1, policy_version 1399838 (0.0006) [2023-12-27 01:34:18,162][105620] Updated weights for policy 1, policy_version 1399848 (0.0006) [2023-12-27 01:34:18,227][105620] Updated weights for policy 1, policy_version 1399858 (0.0010) [2023-12-27 01:34:18,358][105692] Updated weights for policy 0, policy_version 1397745 (0.0009) [2023-12-27 01:34:18,414][105692] Updated weights for policy 0, policy_version 1397755 (0.0011) [2023-12-27 01:34:18,463][105692] Updated weights for policy 0, policy_version 1397765 (0.0011) [2023-12-27 01:34:18,518][105692] Updated weights for policy 0, policy_version 1397775 (0.0011) [2023-12-27 01:34:18,833][105620] Updated weights for policy 1, policy_version 1399868 (0.0011) [2023-12-27 01:34:18,886][105620] Updated weights for policy 1, policy_version 1399878 (0.0011) [2023-12-27 01:34:18,946][105620] Updated weights for policy 1, policy_version 1399888 (0.0011) [2023-12-27 01:34:19,297][105692] Updated weights for policy 0, policy_version 1397785 (0.0011) [2023-12-27 01:34:19,362][105692] Updated weights for policy 0, policy_version 1397795 (0.0011) [2023-12-27 01:34:19,417][105692] Updated weights for policy 0, policy_version 1397805 (0.0010) [2023-12-27 01:34:19,665][105620] Updated weights for policy 1, policy_version 1399898 (0.0011) [2023-12-27 01:34:19,734][105620] Updated weights for policy 1, policy_version 1399908 (0.0011) [2023-12-27 01:34:19,795][105620] Updated weights for policy 1, policy_version 1399918 (0.0011) [2023-12-27 01:34:19,866][105620] Updated weights for policy 1, policy_version 1399928 (0.0010) [2023-12-27 01:34:20,204][105692] Updated weights for policy 0, policy_version 1397815 (0.0010) [2023-12-27 01:34:20,267][105692] Updated weights for policy 0, policy_version 1397825 (0.0011) [2023-12-27 01:34:20,331][105692] Updated weights for policy 0, policy_version 1397835 (0.0010) [2023-12-27 01:34:20,569][105620] Updated weights for policy 1, policy_version 1399938 (0.0010) [2023-12-27 01:34:20,636][105620] Updated weights for policy 1, policy_version 1399948 (0.0010) [2023-12-27 01:34:20,691][105620] Updated weights for policy 1, policy_version 1399958 (0.0008) [2023-12-27 01:34:21,043][105692] Updated weights for policy 0, policy_version 1397845 (0.0007) [2023-12-27 01:34:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19355.3). Total num frames: 716333056. Throughput: 0: 9747.5, 1: 9727.9. Samples: 716325104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:34:21,062][104569] Avg episode reward: [(0, '8349.889'), (1, '9174.877')] [2023-12-27 01:34:21,107][105692] Updated weights for policy 0, policy_version 1397855 (0.0009) [2023-12-27 01:34:21,170][105692] Updated weights for policy 0, policy_version 1397865 (0.0009) [2023-12-27 01:34:21,400][105620] Updated weights for policy 1, policy_version 1399968 (0.0010) [2023-12-27 01:34:21,460][105620] Updated weights for policy 1, policy_version 1399978 (0.0010) [2023-12-27 01:34:21,516][105620] Updated weights for policy 1, policy_version 1399988 (0.0010) [2023-12-27 01:34:21,941][105692] Updated weights for policy 0, policy_version 1397875 (0.0010) [2023-12-27 01:34:21,998][105692] Updated weights for policy 0, policy_version 1397885 (0.0008) [2023-12-27 01:34:22,065][105692] Updated weights for policy 0, policy_version 1397895 (0.0008) [2023-12-27 01:34:22,299][105620] Updated weights for policy 1, policy_version 1399998 (0.0010) [2023-12-27 01:34:22,367][105620] Updated weights for policy 1, policy_version 1400008 (0.0009) [2023-12-27 01:34:22,431][105620] Updated weights for policy 1, policy_version 1400018 (0.0010) [2023-12-27 01:34:22,804][105692] Updated weights for policy 0, policy_version 1397905 (0.0009) [2023-12-27 01:34:22,867][105692] Updated weights for policy 0, policy_version 1397915 (0.0005) [2023-12-27 01:34:22,928][105692] Updated weights for policy 0, policy_version 1397925 (0.0010) [2023-12-27 01:34:22,992][105692] Updated weights for policy 0, policy_version 1397935 (0.0011) [2023-12-27 01:34:23,200][105620] Updated weights for policy 1, policy_version 1400028 (0.0011) [2023-12-27 01:34:23,255][105620] Updated weights for policy 1, policy_version 1400038 (0.0010) [2023-12-27 01:34:23,316][105620] Updated weights for policy 1, policy_version 1400048 (0.0010) [2023-12-27 01:34:23,635][105692] Updated weights for policy 0, policy_version 1397945 (0.0008) [2023-12-27 01:34:23,699][105692] Updated weights for policy 0, policy_version 1397955 (0.0007) [2023-12-27 01:34:23,767][105692] Updated weights for policy 0, policy_version 1397965 (0.0008) [2023-12-27 01:34:23,893][105620] Updated weights for policy 1, policy_version 1400058 (0.0010) [2023-12-27 01:34:23,954][105620] Updated weights for policy 1, policy_version 1400068 (0.0006) [2023-12-27 01:34:24,015][105620] Updated weights for policy 1, policy_version 1400078 (0.0006) [2023-12-27 01:34:24,081][105620] Updated weights for policy 1, policy_version 1400088 (0.0009) [2023-12-27 01:34:24,369][105692] Updated weights for policy 0, policy_version 1397975 (0.0006) [2023-12-27 01:34:24,423][105692] Updated weights for policy 0, policy_version 1397985 (0.0005) [2023-12-27 01:34:24,477][105692] Updated weights for policy 0, policy_version 1397995 (0.0005) [2023-12-27 01:34:24,744][105620] Updated weights for policy 1, policy_version 1400098 (0.0008) [2023-12-27 01:34:24,792][105620] Updated weights for policy 1, policy_version 1400108 (0.0006) [2023-12-27 01:34:24,853][105620] Updated weights for policy 1, policy_version 1400118 (0.0007) [2023-12-27 01:34:25,127][105692] Updated weights for policy 0, policy_version 1398005 (0.0006) [2023-12-27 01:34:25,190][105692] Updated weights for policy 0, policy_version 1398015 (0.0009) [2023-12-27 01:34:25,253][105692] Updated weights for policy 0, policy_version 1398025 (0.0009) [2023-12-27 01:34:25,562][105620] Updated weights for policy 1, policy_version 1400128 (0.0009) [2023-12-27 01:34:25,622][105620] Updated weights for policy 1, policy_version 1400138 (0.0007) [2023-12-27 01:34:25,685][105620] Updated weights for policy 1, policy_version 1400148 (0.0011) [2023-12-27 01:34:26,011][105692] Updated weights for policy 0, policy_version 1398035 (0.0009) [2023-12-27 01:34:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 716431360. Throughput: 0: 9822.2, 1: 9711.2. Samples: 716442668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:34:26,063][104569] Avg episode reward: [(0, '8440.822'), (1, '9085.247')] [2023-12-27 01:34:26,066][105692] Updated weights for policy 0, policy_version 1398045 (0.0008) [2023-12-27 01:34:26,118][105692] Updated weights for policy 0, policy_version 1398055 (0.0009) [2023-12-27 01:34:26,370][105620] Updated weights for policy 1, policy_version 1400158 (0.0011) [2023-12-27 01:34:26,426][105620] Updated weights for policy 1, policy_version 1400168 (0.0011) [2023-12-27 01:34:26,490][105620] Updated weights for policy 1, policy_version 1400178 (0.0010) [2023-12-27 01:34:26,920][105692] Updated weights for policy 0, policy_version 1398065 (0.0009) [2023-12-27 01:34:26,970][105692] Updated weights for policy 0, policy_version 1398075 (0.0009) [2023-12-27 01:34:27,017][105692] Updated weights for policy 0, policy_version 1398085 (0.0009) [2023-12-27 01:34:27,071][105692] Updated weights for policy 0, policy_version 1398095 (0.0009) [2023-12-27 01:34:27,171][105620] Updated weights for policy 1, policy_version 1400188 (0.0010) [2023-12-27 01:34:27,222][105620] Updated weights for policy 1, policy_version 1400198 (0.0009) [2023-12-27 01:34:27,273][105620] Updated weights for policy 1, policy_version 1400208 (0.0009) [2023-12-27 01:34:27,790][105692] Updated weights for policy 0, policy_version 1398105 (0.0009) [2023-12-27 01:34:27,844][105692] Updated weights for policy 0, policy_version 1398115 (0.0009) [2023-12-27 01:34:27,892][105692] Updated weights for policy 0, policy_version 1398125 (0.0006) [2023-12-27 01:34:28,029][105620] Updated weights for policy 1, policy_version 1400218 (0.0008) [2023-12-27 01:34:28,080][105620] Updated weights for policy 1, policy_version 1400228 (0.0005) [2023-12-27 01:34:28,138][105620] Updated weights for policy 1, policy_version 1400239 (0.0010) [2023-12-27 01:34:28,499][105692] Updated weights for policy 0, policy_version 1398135 (0.0008) [2023-12-27 01:34:28,547][105692] Updated weights for policy 0, policy_version 1398145 (0.0009) [2023-12-27 01:34:28,605][105692] Updated weights for policy 0, policy_version 1398156 (0.0010) [2023-12-27 01:34:28,897][105620] Updated weights for policy 1, policy_version 1400249 (0.0009) [2023-12-27 01:34:28,950][105620] Updated weights for policy 1, policy_version 1400259 (0.0009) [2023-12-27 01:34:29,004][105620] Updated weights for policy 1, policy_version 1400269 (0.0009) [2023-12-27 01:34:29,011][105586] KL-divergence is very high: 174.2148 [2023-12-27 01:34:29,063][105586] KL-divergence is very high: 191.7244 [2023-12-27 01:34:29,069][105620] Updated weights for policy 1, policy_version 1400279 (0.0009) [2023-12-27 01:34:29,408][105692] Updated weights for policy 0, policy_version 1398167 (0.0011) [2023-12-27 01:34:29,463][105692] Updated weights for policy 0, policy_version 1398177 (0.0009) [2023-12-27 01:34:29,517][105692] Updated weights for policy 0, policy_version 1398187 (0.0009) [2023-12-27 01:34:29,777][105620] Updated weights for policy 1, policy_version 1400290 (0.0009) [2023-12-27 01:34:29,824][105620] Updated weights for policy 1, policy_version 1400300 (0.0009) [2023-12-27 01:34:29,889][105620] Updated weights for policy 1, policy_version 1400310 (0.0009) [2023-12-27 01:34:30,210][105692] Updated weights for policy 0, policy_version 1398198 (0.0008) [2023-12-27 01:34:30,279][105692] Updated weights for policy 0, policy_version 1398208 (0.0006) [2023-12-27 01:34:30,333][105692] Updated weights for policy 0, policy_version 1398218 (0.0007) [2023-12-27 01:34:30,673][105620] Updated weights for policy 1, policy_version 1400320 (0.0009) [2023-12-27 01:34:30,722][105620] Updated weights for policy 1, policy_version 1400330 (0.0008) [2023-12-27 01:34:30,769][105620] Updated weights for policy 1, policy_version 1400340 (0.0009) [2023-12-27 01:34:30,968][105692] Updated weights for policy 0, policy_version 1398228 (0.0007) [2023-12-27 01:34:31,029][105692] Updated weights for policy 0, policy_version 1398238 (0.0009) [2023-12-27 01:34:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 716529664. Throughput: 0: 9852.5, 1: 9675.1. Samples: 716501396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:34:31,062][104569] Avg episode reward: [(0, '8356.321'), (1, '8904.722')] [2023-12-27 01:34:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001400344_358531072.pth... [2023-12-27 01:34:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001399192_358236160.pth [2023-12-27 01:34:31,087][105692] Updated weights for policy 0, policy_version 1398248 (0.0009) [2023-12-27 01:34:31,134][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001398256_358006784.pth... [2023-12-27 01:34:31,139][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001397104_357711872.pth [2023-12-27 01:34:31,495][105620] Updated weights for policy 1, policy_version 1400350 (0.0007) [2023-12-27 01:34:31,548][105620] Updated weights for policy 1, policy_version 1400360 (0.0005) [2023-12-27 01:34:31,598][105620] Updated weights for policy 1, policy_version 1400370 (0.0005) [2023-12-27 01:34:31,936][105692] Updated weights for policy 0, policy_version 1398258 (0.0009) [2023-12-27 01:34:31,991][105692] Updated weights for policy 0, policy_version 1398269 (0.0010) [2023-12-27 01:34:32,054][105692] Updated weights for policy 0, policy_version 1398279 (0.0009) [2023-12-27 01:34:32,202][105620] Updated weights for policy 1, policy_version 1400380 (0.0008) [2023-12-27 01:34:32,265][105620] Updated weights for policy 1, policy_version 1400390 (0.0008) [2023-12-27 01:34:32,323][105620] Updated weights for policy 1, policy_version 1400400 (0.0009) [2023-12-27 01:34:32,824][105692] Updated weights for policy 0, policy_version 1398289 (0.0009) [2023-12-27 01:34:32,884][105692] Updated weights for policy 0, policy_version 1398299 (0.0008) [2023-12-27 01:34:32,937][105692] Updated weights for policy 0, policy_version 1398309 (0.0008) [2023-12-27 01:34:32,986][105620] Updated weights for policy 1, policy_version 1400410 (0.0009) [2023-12-27 01:34:32,996][105692] Updated weights for policy 0, policy_version 1398319 (0.0007) [2023-12-27 01:34:33,037][105620] Updated weights for policy 1, policy_version 1400420 (0.0010) [2023-12-27 01:34:33,104][105620] Updated weights for policy 1, policy_version 1400430 (0.0008) [2023-12-27 01:34:33,155][105620] Updated weights for policy 1, policy_version 1400440 (0.0010) [2023-12-27 01:34:33,731][105692] Updated weights for policy 0, policy_version 1398329 (0.0007) [2023-12-27 01:34:33,785][105692] Updated weights for policy 0, policy_version 1398339 (0.0005) [2023-12-27 01:34:33,796][105620] Updated weights for policy 1, policy_version 1400450 (0.0008) [2023-12-27 01:34:33,831][105692] Updated weights for policy 0, policy_version 1398349 (0.0005) [2023-12-27 01:34:33,846][105620] Updated weights for policy 1, policy_version 1400460 (0.0008) [2023-12-27 01:34:33,901][105620] Updated weights for policy 1, policy_version 1400470 (0.0009) [2023-12-27 01:34:34,560][105692] Updated weights for policy 0, policy_version 1398359 (0.0005) [2023-12-27 01:34:34,623][105692] Updated weights for policy 0, policy_version 1398369 (0.0005) [2023-12-27 01:34:34,681][105692] Updated weights for policy 0, policy_version 1398379 (0.0006) [2023-12-27 01:34:34,683][105620] Updated weights for policy 1, policy_version 1400480 (0.0008) [2023-12-27 01:34:34,741][105620] Updated weights for policy 1, policy_version 1400490 (0.0009) [2023-12-27 01:34:34,801][105620] Updated weights for policy 1, policy_version 1400500 (0.0009) [2023-12-27 01:34:35,322][105692] Updated weights for policy 0, policy_version 1398389 (0.0006) [2023-12-27 01:34:35,380][105692] Updated weights for policy 0, policy_version 1398399 (0.0008) [2023-12-27 01:34:35,435][105692] Updated weights for policy 0, policy_version 1398409 (0.0009) [2023-12-27 01:34:35,598][105620] Updated weights for policy 1, policy_version 1400510 (0.0009) [2023-12-27 01:34:35,648][105620] Updated weights for policy 1, policy_version 1400520 (0.0008) [2023-12-27 01:34:35,701][105620] Updated weights for policy 1, policy_version 1400530 (0.0009) [2023-12-27 01:34:36,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 716627968. Throughput: 0: 9785.0, 1: 9779.5. Samples: 716617888. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:34:36,062][104569] Avg episode reward: [(0, '8537.238'), (1, '8574.176')] [2023-12-27 01:34:36,155][105692] Updated weights for policy 0, policy_version 1398419 (0.0009) [2023-12-27 01:34:36,213][105692] Updated weights for policy 0, policy_version 1398429 (0.0009) [2023-12-27 01:34:36,275][105692] Updated weights for policy 0, policy_version 1398439 (0.0008) [2023-12-27 01:34:36,473][105620] Updated weights for policy 1, policy_version 1400540 (0.0010) [2023-12-27 01:34:36,529][105620] Updated weights for policy 1, policy_version 1400550 (0.0009) [2023-12-27 01:34:36,584][105620] Updated weights for policy 1, policy_version 1400560 (0.0008) [2023-12-27 01:34:37,042][105692] Updated weights for policy 0, policy_version 1398449 (0.0009) [2023-12-27 01:34:37,104][105692] Updated weights for policy 0, policy_version 1398459 (0.0009) [2023-12-27 01:34:37,161][105692] Updated weights for policy 0, policy_version 1398469 (0.0010) [2023-12-27 01:34:37,218][105692] Updated weights for policy 0, policy_version 1398479 (0.0009) [2023-12-27 01:34:37,333][105620] Updated weights for policy 1, policy_version 1400570 (0.0009) [2023-12-27 01:34:37,394][105620] Updated weights for policy 1, policy_version 1400580 (0.0009) [2023-12-27 01:34:37,452][105620] Updated weights for policy 1, policy_version 1400590 (0.0009) [2023-12-27 01:34:37,513][105620] Updated weights for policy 1, policy_version 1400600 (0.0009) [2023-12-27 01:34:37,910][105692] Updated weights for policy 0, policy_version 1398489 (0.0008) [2023-12-27 01:34:37,965][105692] Updated weights for policy 0, policy_version 1398499 (0.0009) [2023-12-27 01:34:38,011][105692] Updated weights for policy 0, policy_version 1398509 (0.0008) [2023-12-27 01:34:38,312][105620] Updated weights for policy 1, policy_version 1400610 (0.0006) [2023-12-27 01:34:38,375][105620] Updated weights for policy 1, policy_version 1400620 (0.0007) [2023-12-27 01:34:38,442][105620] Updated weights for policy 1, policy_version 1400630 (0.0008) [2023-12-27 01:34:38,849][105692] Updated weights for policy 0, policy_version 1398519 (0.0009) [2023-12-27 01:34:38,897][105692] Updated weights for policy 0, policy_version 1398529 (0.0009) [2023-12-27 01:34:38,950][105692] Updated weights for policy 0, policy_version 1398539 (0.0010) [2023-12-27 01:34:39,036][105620] Updated weights for policy 1, policy_version 1400640 (0.0006) [2023-12-27 01:34:39,099][105620] Updated weights for policy 1, policy_version 1400650 (0.0006) [2023-12-27 01:34:39,164][105620] Updated weights for policy 1, policy_version 1400660 (0.0006) [2023-12-27 01:34:39,794][105692] Updated weights for policy 0, policy_version 1398549 (0.0008) [2023-12-27 01:34:39,799][105620] Updated weights for policy 1, policy_version 1400670 (0.0009) [2023-12-27 01:34:39,859][105620] Updated weights for policy 1, policy_version 1400680 (0.0008) [2023-12-27 01:34:39,860][105692] Updated weights for policy 0, policy_version 1398559 (0.0007) [2023-12-27 01:34:39,918][105620] Updated weights for policy 1, policy_version 1400690 (0.0007) [2023-12-27 01:34:39,922][105692] Updated weights for policy 0, policy_version 1398569 (0.0008) [2023-12-27 01:34:40,657][105620] Updated weights for policy 1, policy_version 1400700 (0.0008) [2023-12-27 01:34:40,704][105620] Updated weights for policy 1, policy_version 1400710 (0.0005) [2023-12-27 01:34:40,740][105692] Updated weights for policy 0, policy_version 1398579 (0.0008) [2023-12-27 01:34:40,756][105620] Updated weights for policy 1, policy_version 1400720 (0.0006) [2023-12-27 01:34:40,800][105692] Updated weights for policy 0, policy_version 1398589 (0.0008) [2023-12-27 01:34:40,861][105692] Updated weights for policy 0, policy_version 1398599 (0.0009) [2023-12-27 01:34:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 716726272. Throughput: 0: 9664.3, 1: 9840.8. Samples: 716731636. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:34:41,062][104569] Avg episode reward: [(0, '8538.590'), (1, '8231.191')] [2023-12-27 01:34:41,471][105620] Updated weights for policy 1, policy_version 1400730 (0.0007) [2023-12-27 01:34:41,535][105620] Updated weights for policy 1, policy_version 1400740 (0.0008) [2023-12-27 01:34:41,582][105620] Updated weights for policy 1, policy_version 1400750 (0.0009) [2023-12-27 01:34:41,639][105620] Updated weights for policy 1, policy_version 1400760 (0.0009) [2023-12-27 01:34:41,698][105692] Updated weights for policy 0, policy_version 1398610 (0.0011) [2023-12-27 01:34:41,756][105692] Updated weights for policy 0, policy_version 1398620 (0.0010) [2023-12-27 01:34:41,818][105692] Updated weights for policy 0, policy_version 1398631 (0.0013) [2023-12-27 01:34:42,272][105620] Updated weights for policy 1, policy_version 1400770 (0.0006) [2023-12-27 01:34:42,337][105620] Updated weights for policy 1, policy_version 1400780 (0.0010) [2023-12-27 01:34:42,406][105620] Updated weights for policy 1, policy_version 1400790 (0.0009) [2023-12-27 01:34:42,625][105692] Updated weights for policy 0, policy_version 1398641 (0.0009) [2023-12-27 01:34:42,679][105692] Updated weights for policy 0, policy_version 1398651 (0.0008) [2023-12-27 01:34:42,741][105692] Updated weights for policy 0, policy_version 1398661 (0.0006) [2023-12-27 01:34:42,794][105692] Updated weights for policy 0, policy_version 1398671 (0.0008) [2023-12-27 01:34:43,016][105620] Updated weights for policy 1, policy_version 1400800 (0.0006) [2023-12-27 01:34:43,071][105620] Updated weights for policy 1, policy_version 1400810 (0.0005) [2023-12-27 01:34:43,128][105620] Updated weights for policy 1, policy_version 1400820 (0.0005) [2023-12-27 01:34:43,583][105692] Updated weights for policy 0, policy_version 1398681 (0.0006) [2023-12-27 01:34:43,635][105692] Updated weights for policy 0, policy_version 1398691 (0.0005) [2023-12-27 01:34:43,685][105692] Updated weights for policy 0, policy_version 1398701 (0.0005) [2023-12-27 01:34:43,746][105620] Updated weights for policy 1, policy_version 1400830 (0.0007) [2023-12-27 01:34:43,802][105620] Updated weights for policy 1, policy_version 1400840 (0.0006) [2023-12-27 01:34:43,858][105620] Updated weights for policy 1, policy_version 1400850 (0.0005) [2023-12-27 01:34:44,306][105692] Updated weights for policy 0, policy_version 1398711 (0.0009) [2023-12-27 01:34:44,358][105692] Updated weights for policy 0, policy_version 1398721 (0.0008) [2023-12-27 01:34:44,406][105692] Updated weights for policy 0, policy_version 1398731 (0.0010) [2023-12-27 01:34:44,483][105620] Updated weights for policy 1, policy_version 1400860 (0.0008) [2023-12-27 01:34:44,541][105620] Updated weights for policy 1, policy_version 1400870 (0.0009) [2023-12-27 01:34:44,593][105620] Updated weights for policy 1, policy_version 1400880 (0.0008) [2023-12-27 01:34:45,145][105692] Updated weights for policy 0, policy_version 1398741 (0.0009) [2023-12-27 01:34:45,208][105692] Updated weights for policy 0, policy_version 1398751 (0.0007) [2023-12-27 01:34:45,271][105692] Updated weights for policy 0, policy_version 1398761 (0.0009) [2023-12-27 01:34:45,327][105620] Updated weights for policy 1, policy_version 1400890 (0.0009) [2023-12-27 01:34:45,394][105620] Updated weights for policy 1, policy_version 1400900 (0.0010) [2023-12-27 01:34:45,455][105620] Updated weights for policy 1, policy_version 1400910 (0.0010) [2023-12-27 01:34:45,521][105620] Updated weights for policy 1, policy_version 1400920 (0.0010) [2023-12-27 01:34:45,989][105692] Updated weights for policy 0, policy_version 1398771 (0.0008) [2023-12-27 01:34:46,051][105692] Updated weights for policy 0, policy_version 1398781 (0.0006) [2023-12-27 01:34:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 716816384. Throughput: 0: 9583.8, 1: 9935.9. Samples: 716790468. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:34:46,062][104569] Avg episode reward: [(0, '8624.890'), (1, '6591.241')] [2023-12-27 01:34:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001400920_358678528.pth... [2023-12-27 01:34:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001399768_358383616.pth [2023-12-27 01:34:46,113][105692] Updated weights for policy 0, policy_version 1398791 (0.0005) [2023-12-27 01:34:46,151][105620] Updated weights for policy 1, policy_version 1400930 (0.0009) [2023-12-27 01:34:46,161][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001398800_358146048.pth... [2023-12-27 01:34:46,166][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001397680_357859328.pth [2023-12-27 01:34:46,208][105620] Updated weights for policy 1, policy_version 1400940 (0.0007) [2023-12-27 01:34:46,259][105620] Updated weights for policy 1, policy_version 1400950 (0.0007) [2023-12-27 01:34:46,679][105692] Updated weights for policy 0, policy_version 1398801 (0.0007) [2023-12-27 01:34:46,736][105692] Updated weights for policy 0, policy_version 1398811 (0.0008) [2023-12-27 01:34:46,792][105692] Updated weights for policy 0, policy_version 1398821 (0.0008) [2023-12-27 01:34:46,840][105692] Updated weights for policy 0, policy_version 1398831 (0.0009) [2023-12-27 01:34:46,940][105620] Updated weights for policy 1, policy_version 1400960 (0.0008) [2023-12-27 01:34:46,995][105620] Updated weights for policy 1, policy_version 1400970 (0.0009) [2023-12-27 01:34:47,037][105586] KL-divergence is very high: 152.7845 [2023-12-27 01:34:47,042][105620] Updated weights for policy 1, policy_version 1400980 (0.0009) [2023-12-27 01:34:47,046][105586] KL-divergence is very high: 161.5202 [2023-12-27 01:34:47,612][105692] Updated weights for policy 0, policy_version 1398841 (0.0009) [2023-12-27 01:34:47,670][105692] Updated weights for policy 0, policy_version 1398851 (0.0009) [2023-12-27 01:34:47,718][105692] Updated weights for policy 0, policy_version 1398861 (0.0009) [2023-12-27 01:34:47,814][105620] Updated weights for policy 1, policy_version 1400990 (0.0009) [2023-12-27 01:34:47,875][105620] Updated weights for policy 1, policy_version 1401000 (0.0009) [2023-12-27 01:34:47,928][105620] Updated weights for policy 1, policy_version 1401010 (0.0009) [2023-12-27 01:34:48,390][105692] Updated weights for policy 0, policy_version 1398871 (0.0008) [2023-12-27 01:34:48,451][105692] Updated weights for policy 0, policy_version 1398881 (0.0010) [2023-12-27 01:34:48,513][105692] Updated weights for policy 0, policy_version 1398891 (0.0009) [2023-12-27 01:34:48,649][105620] Updated weights for policy 1, policy_version 1401020 (0.0008) [2023-12-27 01:34:48,707][105620] Updated weights for policy 1, policy_version 1401030 (0.0009) [2023-12-27 01:34:48,755][105620] Updated weights for policy 1, policy_version 1401040 (0.0009) [2023-12-27 01:34:49,187][105692] Updated weights for policy 0, policy_version 1398901 (0.0008) [2023-12-27 01:34:49,250][105692] Updated weights for policy 0, policy_version 1398911 (0.0008) [2023-12-27 01:34:49,306][105692] Updated weights for policy 0, policy_version 1398921 (0.0009) [2023-12-27 01:34:49,507][105620] Updated weights for policy 1, policy_version 1401050 (0.0009) [2023-12-27 01:34:49,559][105620] Updated weights for policy 1, policy_version 1401060 (0.0007) [2023-12-27 01:34:49,619][105620] Updated weights for policy 1, policy_version 1401070 (0.0007) [2023-12-27 01:34:50,196][105692] Updated weights for policy 0, policy_version 1398931 (0.0009) [2023-12-27 01:34:50,255][105692] Updated weights for policy 0, policy_version 1398941 (0.0010) [2023-12-27 01:34:50,315][105692] Updated weights for policy 0, policy_version 1398951 (0.0006) [2023-12-27 01:34:50,320][105620] Updated weights for policy 1, policy_version 1401081 (0.0007) [2023-12-27 01:34:50,373][105620] Updated weights for policy 1, policy_version 1401091 (0.0010) [2023-12-27 01:34:50,435][105620] Updated weights for policy 1, policy_version 1401101 (0.0008) [2023-12-27 01:34:50,499][105620] Updated weights for policy 1, policy_version 1401111 (0.0006) [2023-12-27 01:34:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19355.3). Total num frames: 716914688. Throughput: 0: 9573.6, 1: 10031.6. Samples: 716909844. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:34:51,062][104569] Avg episode reward: [(0, '8259.805'), (1, '4091.412')] [2023-12-27 01:34:51,078][105692] Updated weights for policy 0, policy_version 1398961 (0.0006) [2023-12-27 01:34:51,147][105692] Updated weights for policy 0, policy_version 1398971 (0.0008) [2023-12-27 01:34:51,172][105620] Updated weights for policy 1, policy_version 1401121 (0.0010) [2023-12-27 01:34:51,202][105692] Updated weights for policy 0, policy_version 1398981 (0.0007) [2023-12-27 01:34:51,231][105620] Updated weights for policy 1, policy_version 1401131 (0.0010) [2023-12-27 01:34:51,264][105692] Updated weights for policy 0, policy_version 1398991 (0.0010) [2023-12-27 01:34:51,298][105620] Updated weights for policy 1, policy_version 1401141 (0.0011) [2023-12-27 01:34:52,050][105620] Updated weights for policy 1, policy_version 1401151 (0.0011) [2023-12-27 01:34:52,050][105692] Updated weights for policy 0, policy_version 1399001 (0.0008) [2023-12-27 01:34:52,102][105692] Updated weights for policy 0, policy_version 1399011 (0.0009) [2023-12-27 01:34:52,102][105620] Updated weights for policy 1, policy_version 1401161 (0.0010) [2023-12-27 01:34:52,152][105692] Updated weights for policy 0, policy_version 1399021 (0.0008) [2023-12-27 01:34:52,162][105620] Updated weights for policy 1, policy_version 1401171 (0.0011) [2023-12-27 01:34:52,877][105692] Updated weights for policy 0, policy_version 1399031 (0.0006) [2023-12-27 01:34:52,930][105620] Updated weights for policy 1, policy_version 1401181 (0.0010) [2023-12-27 01:34:52,936][105692] Updated weights for policy 0, policy_version 1399041 (0.0005) [2023-12-27 01:34:52,993][105620] Updated weights for policy 1, policy_version 1401191 (0.0011) [2023-12-27 01:34:52,998][105692] Updated weights for policy 0, policy_version 1399051 (0.0005) [2023-12-27 01:34:53,054][105620] Updated weights for policy 1, policy_version 1401201 (0.0011) [2023-12-27 01:34:53,537][105692] Updated weights for policy 0, policy_version 1399061 (0.0006) [2023-12-27 01:34:53,582][105692] Updated weights for policy 0, policy_version 1399071 (0.0006) [2023-12-27 01:34:53,641][105692] Updated weights for policy 0, policy_version 1399081 (0.0005) [2023-12-27 01:34:53,753][105620] Updated weights for policy 1, policy_version 1401211 (0.0010) [2023-12-27 01:34:53,815][105620] Updated weights for policy 1, policy_version 1401221 (0.0011) [2023-12-27 01:34:53,882][105620] Updated weights for policy 1, policy_version 1401231 (0.0010) [2023-12-27 01:34:54,326][105692] Updated weights for policy 0, policy_version 1399091 (0.0007) [2023-12-27 01:34:54,375][105692] Updated weights for policy 0, policy_version 1399101 (0.0010) [2023-12-27 01:34:54,423][105692] Updated weights for policy 0, policy_version 1399111 (0.0010) [2023-12-27 01:34:54,581][105620] Updated weights for policy 1, policy_version 1401241 (0.0008) [2023-12-27 01:34:54,636][105620] Updated weights for policy 1, policy_version 1401251 (0.0005) [2023-12-27 01:34:54,699][105620] Updated weights for policy 1, policy_version 1401261 (0.0006) [2023-12-27 01:34:54,756][105620] Updated weights for policy 1, policy_version 1401271 (0.0005) [2023-12-27 01:34:55,115][105692] Updated weights for policy 0, policy_version 1399121 (0.0010) [2023-12-27 01:34:55,174][105692] Updated weights for policy 0, policy_version 1399131 (0.0005) [2023-12-27 01:34:55,236][105692] Updated weights for policy 0, policy_version 1399141 (0.0006) [2023-12-27 01:34:55,299][105692] Updated weights for policy 0, policy_version 1399151 (0.0010) [2023-12-27 01:34:55,363][105620] Updated weights for policy 1, policy_version 1401281 (0.0008) [2023-12-27 01:34:55,416][105620] Updated weights for policy 1, policy_version 1401291 (0.0008) [2023-12-27 01:34:55,469][105620] Updated weights for policy 1, policy_version 1401301 (0.0008) [2023-12-27 01:34:55,970][105692] Updated weights for policy 0, policy_version 1399161 (0.0006) [2023-12-27 01:34:56,036][105692] Updated weights for policy 0, policy_version 1399171 (0.0005) [2023-12-27 01:34:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 717012992. Throughput: 0: 9609.9, 1: 9958.5. Samples: 717026348. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:34:56,062][104569] Avg episode reward: [(0, '8454.390'), (1, '5475.716')] [2023-12-27 01:34:56,097][105692] Updated weights for policy 0, policy_version 1399181 (0.0010) [2023-12-27 01:34:56,289][105620] Updated weights for policy 1, policy_version 1401311 (0.0009) [2023-12-27 01:34:56,347][105620] Updated weights for policy 1, policy_version 1401321 (0.0008) [2023-12-27 01:34:56,417][105620] Updated weights for policy 1, policy_version 1401331 (0.0005) [2023-12-27 01:34:56,646][105692] Updated weights for policy 0, policy_version 1399191 (0.0007) [2023-12-27 01:34:56,702][105692] Updated weights for policy 0, policy_version 1399201 (0.0005) [2023-12-27 01:34:56,759][105692] Updated weights for policy 0, policy_version 1399211 (0.0005) [2023-12-27 01:34:57,121][105620] Updated weights for policy 1, policy_version 1401341 (0.0006) [2023-12-27 01:34:57,168][105620] Updated weights for policy 1, policy_version 1401351 (0.0008) [2023-12-27 01:34:57,220][105620] Updated weights for policy 1, policy_version 1401361 (0.0008) [2023-12-27 01:34:57,431][105692] Updated weights for policy 0, policy_version 1399221 (0.0009) [2023-12-27 01:34:57,478][105692] Updated weights for policy 0, policy_version 1399231 (0.0010) [2023-12-27 01:34:57,526][105692] Updated weights for policy 0, policy_version 1399241 (0.0010) [2023-12-27 01:34:57,983][105620] Updated weights for policy 1, policy_version 1401371 (0.0007) [2023-12-27 01:34:58,044][105620] Updated weights for policy 1, policy_version 1401381 (0.0005) [2023-12-27 01:34:58,105][105620] Updated weights for policy 1, policy_version 1401391 (0.0005) [2023-12-27 01:34:58,267][105692] Updated weights for policy 0, policy_version 1399251 (0.0010) [2023-12-27 01:34:58,327][105692] Updated weights for policy 0, policy_version 1399261 (0.0011) [2023-12-27 01:34:58,392][105692] Updated weights for policy 0, policy_version 1399271 (0.0009) [2023-12-27 01:34:58,856][105620] Updated weights for policy 1, policy_version 1401401 (0.0006) [2023-12-27 01:34:58,927][105620] Updated weights for policy 1, policy_version 1401411 (0.0009) [2023-12-27 01:34:58,988][105620] Updated weights for policy 1, policy_version 1401421 (0.0009) [2023-12-27 01:34:59,054][105620] Updated weights for policy 1, policy_version 1401431 (0.0010) [2023-12-27 01:34:59,211][105692] Updated weights for policy 0, policy_version 1399281 (0.0010) [2023-12-27 01:34:59,281][105692] Updated weights for policy 0, policy_version 1399291 (0.0008) [2023-12-27 01:34:59,354][105692] Updated weights for policy 0, policy_version 1399301 (0.0009) [2023-12-27 01:34:59,413][105692] Updated weights for policy 0, policy_version 1399311 (0.0008) [2023-12-27 01:34:59,767][105620] Updated weights for policy 1, policy_version 1401441 (0.0010) [2023-12-27 01:34:59,844][105620] Updated weights for policy 1, policy_version 1401451 (0.0009) [2023-12-27 01:34:59,907][105620] Updated weights for policy 1, policy_version 1401461 (0.0008) [2023-12-27 01:35:00,009][105692] Updated weights for policy 0, policy_version 1399321 (0.0008) [2023-12-27 01:35:00,074][105692] Updated weights for policy 0, policy_version 1399331 (0.0009) [2023-12-27 01:35:00,139][105692] Updated weights for policy 0, policy_version 1399341 (0.0009) [2023-12-27 01:35:00,646][105620] Updated weights for policy 1, policy_version 1401471 (0.0009) [2023-12-27 01:35:00,697][105620] Updated weights for policy 1, policy_version 1401482 (0.0009) [2023-12-27 01:35:00,715][105692] Updated weights for policy 0, policy_version 1399351 (0.0007) [2023-12-27 01:35:00,756][105620] Updated weights for policy 1, policy_version 1401492 (0.0008) [2023-12-27 01:35:00,770][105692] Updated weights for policy 0, policy_version 1399361 (0.0006) [2023-12-27 01:35:00,828][105692] Updated weights for policy 0, policy_version 1399371 (0.0006) [2023-12-27 01:35:01,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19660.8, 300 sec: 19383.1). Total num frames: 717119488. Throughput: 0: 9691.5, 1: 9903.1. Samples: 717085340. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:35:01,062][104569] Avg episode reward: [(0, '8276.687'), (1, '7936.861')] [2023-12-27 01:35:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001399376_358293504.pth... [2023-12-27 01:35:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001401496_358825984.pth... [2023-12-27 01:35:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001400344_358531072.pth [2023-12-27 01:35:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001398256_358006784.pth [2023-12-27 01:35:01,540][105620] Updated weights for policy 1, policy_version 1401502 (0.0008) [2023-12-27 01:35:01,572][105692] Updated weights for policy 0, policy_version 1399381 (0.0005) [2023-12-27 01:35:01,593][105620] Updated weights for policy 1, policy_version 1401512 (0.0009) [2023-12-27 01:35:01,634][105692] Updated weights for policy 0, policy_version 1399391 (0.0006) [2023-12-27 01:35:01,650][105620] Updated weights for policy 1, policy_version 1401522 (0.0008) [2023-12-27 01:35:01,697][105692] Updated weights for policy 0, policy_version 1399401 (0.0007) [2023-12-27 01:35:02,426][105692] Updated weights for policy 0, policy_version 1399411 (0.0009) [2023-12-27 01:35:02,466][105620] Updated weights for policy 1, policy_version 1401532 (0.0006) [2023-12-27 01:35:02,484][105692] Updated weights for policy 0, policy_version 1399421 (0.0008) [2023-12-27 01:35:02,516][105620] Updated weights for policy 1, policy_version 1401542 (0.0007) [2023-12-27 01:35:02,535][105692] Updated weights for policy 0, policy_version 1399431 (0.0008) [2023-12-27 01:35:02,575][105620] Updated weights for policy 1, policy_version 1401552 (0.0006) [2023-12-27 01:35:03,204][105692] Updated weights for policy 0, policy_version 1399441 (0.0008) [2023-12-27 01:35:03,263][105692] Updated weights for policy 0, policy_version 1399451 (0.0009) [2023-12-27 01:35:03,330][105692] Updated weights for policy 0, policy_version 1399461 (0.0009) [2023-12-27 01:35:03,370][105620] Updated weights for policy 1, policy_version 1401562 (0.0007) [2023-12-27 01:35:03,385][105692] Updated weights for policy 0, policy_version 1399471 (0.0007) [2023-12-27 01:35:03,422][105620] Updated weights for policy 1, policy_version 1401572 (0.0007) [2023-12-27 01:35:03,476][105620] Updated weights for policy 1, policy_version 1401582 (0.0009) [2023-12-27 01:35:03,524][105620] Updated weights for policy 1, policy_version 1401592 (0.0008) [2023-12-27 01:35:04,133][105692] Updated weights for policy 0, policy_version 1399481 (0.0009) [2023-12-27 01:35:04,191][105692] Updated weights for policy 0, policy_version 1399491 (0.0009) [2023-12-27 01:35:04,255][105692] Updated weights for policy 0, policy_version 1399501 (0.0009) [2023-12-27 01:35:04,304][105620] Updated weights for policy 1, policy_version 1401602 (0.0008) [2023-12-27 01:35:04,361][105620] Updated weights for policy 1, policy_version 1401612 (0.0007) [2023-12-27 01:35:04,412][105620] Updated weights for policy 1, policy_version 1401622 (0.0005) [2023-12-27 01:35:05,025][105692] Updated weights for policy 0, policy_version 1399511 (0.0006) [2023-12-27 01:35:05,092][105692] Updated weights for policy 0, policy_version 1399521 (0.0005) [2023-12-27 01:35:05,151][105620] Updated weights for policy 1, policy_version 1401632 (0.0007) [2023-12-27 01:35:05,153][105692] Updated weights for policy 0, policy_version 1399531 (0.0007) [2023-12-27 01:35:05,201][105620] Updated weights for policy 1, policy_version 1401642 (0.0007) [2023-12-27 01:35:05,264][105620] Updated weights for policy 1, policy_version 1401652 (0.0008) [2023-12-27 01:35:05,765][105692] Updated weights for policy 0, policy_version 1399541 (0.0007) [2023-12-27 01:35:05,818][105692] Updated weights for policy 0, policy_version 1399551 (0.0005) [2023-12-27 01:35:05,868][105692] Updated weights for policy 0, policy_version 1399561 (0.0007) [2023-12-27 01:35:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19355.3). Total num frames: 717209600. Throughput: 0: 9682.6, 1: 9750.6. Samples: 717199596. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:35:06,063][104569] Avg episode reward: [(0, '7989.152'), (1, '8158.490')] [2023-12-27 01:35:06,083][105620] Updated weights for policy 1, policy_version 1401662 (0.0008) [2023-12-27 01:35:06,150][105620] Updated weights for policy 1, policy_version 1401672 (0.0008) [2023-12-27 01:35:06,206][105620] Updated weights for policy 1, policy_version 1401682 (0.0006) [2023-12-27 01:35:06,646][105692] Updated weights for policy 0, policy_version 1399571 (0.0010) [2023-12-27 01:35:06,705][105692] Updated weights for policy 0, policy_version 1399581 (0.0011) [2023-12-27 01:35:06,765][105692] Updated weights for policy 0, policy_version 1399591 (0.0010) [2023-12-27 01:35:06,983][105620] Updated weights for policy 1, policy_version 1401692 (0.0008) [2023-12-27 01:35:07,032][105620] Updated weights for policy 1, policy_version 1401702 (0.0008) [2023-12-27 01:35:07,088][105620] Updated weights for policy 1, policy_version 1401712 (0.0008) [2023-12-27 01:35:07,526][105692] Updated weights for policy 0, policy_version 1399601 (0.0010) [2023-12-27 01:35:07,581][105692] Updated weights for policy 0, policy_version 1399611 (0.0010) [2023-12-27 01:35:07,644][105692] Updated weights for policy 0, policy_version 1399621 (0.0008) [2023-12-27 01:35:07,691][105692] Updated weights for policy 0, policy_version 1399631 (0.0009) [2023-12-27 01:35:07,886][105620] Updated weights for policy 1, policy_version 1401722 (0.0008) [2023-12-27 01:35:07,939][105620] Updated weights for policy 1, policy_version 1401732 (0.0008) [2023-12-27 01:35:07,983][105620] Updated weights for policy 1, policy_version 1401742 (0.0008) [2023-12-27 01:35:08,039][105620] Updated weights for policy 1, policy_version 1401752 (0.0008) [2023-12-27 01:35:08,411][105692] Updated weights for policy 0, policy_version 1399641 (0.0010) [2023-12-27 01:35:08,474][105692] Updated weights for policy 0, policy_version 1399651 (0.0011) [2023-12-27 01:35:08,526][105692] Updated weights for policy 0, policy_version 1399661 (0.0010) [2023-12-27 01:35:08,802][105620] Updated weights for policy 1, policy_version 1401762 (0.0010) [2023-12-27 01:35:08,861][105620] Updated weights for policy 1, policy_version 1401772 (0.0010) [2023-12-27 01:35:08,919][105620] Updated weights for policy 1, policy_version 1401782 (0.0010) [2023-12-27 01:35:09,277][105692] Updated weights for policy 0, policy_version 1399671 (0.0010) [2023-12-27 01:35:09,334][105692] Updated weights for policy 0, policy_version 1399681 (0.0010) [2023-12-27 01:35:09,407][105692] Updated weights for policy 0, policy_version 1399691 (0.0013) [2023-12-27 01:35:09,578][105620] Updated weights for policy 1, policy_version 1401792 (0.0010) [2023-12-27 01:35:09,638][105620] Updated weights for policy 1, policy_version 1401802 (0.0009) [2023-12-27 01:35:09,697][105620] Updated weights for policy 1, policy_version 1401812 (0.0009) [2023-12-27 01:35:10,195][105692] Updated weights for policy 0, policy_version 1399701 (0.0007) [2023-12-27 01:35:10,264][105692] Updated weights for policy 0, policy_version 1399711 (0.0008) [2023-12-27 01:35:10,323][105692] Updated weights for policy 0, policy_version 1399721 (0.0011) [2023-12-27 01:35:10,408][105620] Updated weights for policy 1, policy_version 1401822 (0.0011) [2023-12-27 01:35:10,475][105620] Updated weights for policy 1, policy_version 1401832 (0.0011) [2023-12-27 01:35:10,542][105620] Updated weights for policy 1, policy_version 1401842 (0.0011) [2023-12-27 01:35:11,035][105692] Updated weights for policy 0, policy_version 1399731 (0.0009) [2023-12-27 01:35:11,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 717299712. Throughput: 0: 9651.4, 1: 9675.3. Samples: 717312368. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:35:11,063][104569] Avg episode reward: [(0, '8170.589'), (1, '8546.006')] [2023-12-27 01:35:11,103][105692] Updated weights for policy 0, policy_version 1399741 (0.0010) [2023-12-27 01:35:11,168][105692] Updated weights for policy 0, policy_version 1399751 (0.0011) [2023-12-27 01:35:11,287][105620] Updated weights for policy 1, policy_version 1401852 (0.0010) [2023-12-27 01:35:11,343][105620] Updated weights for policy 1, policy_version 1401862 (0.0008) [2023-12-27 01:35:11,417][105620] Updated weights for policy 1, policy_version 1401872 (0.0009) [2023-12-27 01:35:11,868][105692] Updated weights for policy 0, policy_version 1399761 (0.0010) [2023-12-27 01:35:11,925][105692] Updated weights for policy 0, policy_version 1399771 (0.0008) [2023-12-27 01:35:11,982][105692] Updated weights for policy 0, policy_version 1399781 (0.0011) [2023-12-27 01:35:12,034][105692] Updated weights for policy 0, policy_version 1399791 (0.0011) [2023-12-27 01:35:12,172][105620] Updated weights for policy 1, policy_version 1401882 (0.0006) [2023-12-27 01:35:12,224][105620] Updated weights for policy 1, policy_version 1401892 (0.0008) [2023-12-27 01:35:12,284][105620] Updated weights for policy 1, policy_version 1401902 (0.0008) [2023-12-27 01:35:12,345][105620] Updated weights for policy 1, policy_version 1401912 (0.0010) [2023-12-27 01:35:12,715][105692] Updated weights for policy 0, policy_version 1399801 (0.0006) [2023-12-27 01:35:12,772][105692] Updated weights for policy 0, policy_version 1399811 (0.0011) [2023-12-27 01:35:12,833][105692] Updated weights for policy 0, policy_version 1399821 (0.0008) [2023-12-27 01:35:13,204][105620] Updated weights for policy 1, policy_version 1401922 (0.0008) [2023-12-27 01:35:13,259][105620] Updated weights for policy 1, policy_version 1401932 (0.0008) [2023-12-27 01:35:13,317][105620] Updated weights for policy 1, policy_version 1401942 (0.0008) [2023-12-27 01:35:13,460][105692] Updated weights for policy 0, policy_version 1399831 (0.0009) [2023-12-27 01:35:13,525][105692] Updated weights for policy 0, policy_version 1399841 (0.0011) [2023-12-27 01:35:13,583][105692] Updated weights for policy 0, policy_version 1399851 (0.0010) [2023-12-27 01:35:14,102][105620] Updated weights for policy 1, policy_version 1401952 (0.0009) [2023-12-27 01:35:14,167][105620] Updated weights for policy 1, policy_version 1401962 (0.0009) [2023-12-27 01:35:14,220][105620] Updated weights for policy 1, policy_version 1401972 (0.0008) [2023-12-27 01:35:14,226][105692] Updated weights for policy 0, policy_version 1399861 (0.0008) [2023-12-27 01:35:14,283][105692] Updated weights for policy 0, policy_version 1399871 (0.0005) [2023-12-27 01:35:14,339][105692] Updated weights for policy 0, policy_version 1399881 (0.0005) [2023-12-27 01:35:14,952][105692] Updated weights for policy 0, policy_version 1399891 (0.0007) [2023-12-27 01:35:14,991][105620] Updated weights for policy 1, policy_version 1401982 (0.0007) [2023-12-27 01:35:15,016][105692] Updated weights for policy 0, policy_version 1399901 (0.0011) [2023-12-27 01:35:15,057][105620] Updated weights for policy 1, policy_version 1401992 (0.0007) [2023-12-27 01:35:15,079][105692] Updated weights for policy 0, policy_version 1399911 (0.0011) [2023-12-27 01:35:15,114][105620] Updated weights for policy 1, policy_version 1402002 (0.0007) [2023-12-27 01:35:15,716][105692] Updated weights for policy 0, policy_version 1399921 (0.0006) [2023-12-27 01:35:15,766][105692] Updated weights for policy 0, policy_version 1399931 (0.0009) [2023-12-27 01:35:15,817][105692] Updated weights for policy 0, policy_version 1399941 (0.0006) [2023-12-27 01:35:15,862][105620] Updated weights for policy 1, policy_version 1402012 (0.0010) [2023-12-27 01:35:15,878][105692] Updated weights for policy 0, policy_version 1399951 (0.0007) [2023-12-27 01:35:15,913][105620] Updated weights for policy 1, policy_version 1402022 (0.0009) [2023-12-27 01:35:15,968][105620] Updated weights for policy 1, policy_version 1402032 (0.0008) [2023-12-27 01:35:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 717406208. Throughput: 0: 9661.6, 1: 9624.4. Samples: 717369268. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:35:16,063][104569] Avg episode reward: [(0, '8449.570'), (1, '8719.693')] [2023-12-27 01:35:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001399952_358440960.pth... [2023-12-27 01:35:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001402040_358965248.pth... [2023-12-27 01:35:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001398800_358146048.pth [2023-12-27 01:35:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001400920_358678528.pth [2023-12-27 01:35:16,534][105692] Updated weights for policy 0, policy_version 1399961 (0.0011) [2023-12-27 01:35:16,582][105692] Updated weights for policy 0, policy_version 1399971 (0.0010) [2023-12-27 01:35:16,645][105692] Updated weights for policy 0, policy_version 1399981 (0.0010) [2023-12-27 01:35:16,652][105620] Updated weights for policy 1, policy_version 1402042 (0.0008) [2023-12-27 01:35:16,700][105620] Updated weights for policy 1, policy_version 1402052 (0.0010) [2023-12-27 01:35:16,745][105620] Updated weights for policy 1, policy_version 1402062 (0.0008) [2023-12-27 01:35:16,795][105620] Updated weights for policy 1, policy_version 1402072 (0.0005) [2023-12-27 01:35:17,333][105692] Updated weights for policy 0, policy_version 1399991 (0.0010) [2023-12-27 01:35:17,345][105620] Updated weights for policy 1, policy_version 1402082 (0.0010) [2023-12-27 01:35:17,378][105692] Updated weights for policy 0, policy_version 1400001 (0.0010) [2023-12-27 01:35:17,393][105620] Updated weights for policy 1, policy_version 1402092 (0.0010) [2023-12-27 01:35:17,426][105692] Updated weights for policy 0, policy_version 1400011 (0.0010) [2023-12-27 01:35:17,441][105620] Updated weights for policy 1, policy_version 1402102 (0.0010) [2023-12-27 01:35:18,157][105692] Updated weights for policy 0, policy_version 1400021 (0.0011) [2023-12-27 01:35:18,178][105620] Updated weights for policy 1, policy_version 1402112 (0.0007) [2023-12-27 01:35:18,213][105692] Updated weights for policy 0, policy_version 1400031 (0.0010) [2023-12-27 01:35:18,224][105620] Updated weights for policy 1, policy_version 1402122 (0.0005) [2023-12-27 01:35:18,265][105692] Updated weights for policy 0, policy_version 1400041 (0.0010) [2023-12-27 01:35:18,270][105620] Updated weights for policy 1, policy_version 1402132 (0.0005) [2023-12-27 01:35:18,991][105620] Updated weights for policy 1, policy_version 1402142 (0.0010) [2023-12-27 01:35:19,022][105692] Updated weights for policy 0, policy_version 1400051 (0.0010) [2023-12-27 01:35:19,052][105620] Updated weights for policy 1, policy_version 1402152 (0.0010) [2023-12-27 01:35:19,074][105692] Updated weights for policy 0, policy_version 1400061 (0.0010) [2023-12-27 01:35:19,110][105620] Updated weights for policy 1, policy_version 1402162 (0.0010) [2023-12-27 01:35:19,125][105692] Updated weights for policy 0, policy_version 1400071 (0.0010) [2023-12-27 01:35:19,902][105620] Updated weights for policy 1, policy_version 1402172 (0.0010) [2023-12-27 01:35:19,920][105692] Updated weights for policy 0, policy_version 1400081 (0.0010) [2023-12-27 01:35:19,971][105620] Updated weights for policy 1, policy_version 1402182 (0.0007) [2023-12-27 01:35:19,986][105692] Updated weights for policy 0, policy_version 1400091 (0.0009) [2023-12-27 01:35:20,030][105620] Updated weights for policy 1, policy_version 1402192 (0.0006) [2023-12-27 01:35:20,052][105692] Updated weights for policy 0, policy_version 1400101 (0.0009) [2023-12-27 01:35:20,115][105692] Updated weights for policy 0, policy_version 1400111 (0.0009) [2023-12-27 01:35:20,846][105620] Updated weights for policy 1, policy_version 1402202 (0.0008) [2023-12-27 01:35:20,855][105692] Updated weights for policy 0, policy_version 1400121 (0.0006) [2023-12-27 01:35:20,907][105620] Updated weights for policy 1, policy_version 1402212 (0.0007) [2023-12-27 01:35:20,913][105692] Updated weights for policy 0, policy_version 1400131 (0.0011) [2023-12-27 01:35:20,973][105620] Updated weights for policy 1, policy_version 1402222 (0.0010) [2023-12-27 01:35:20,977][105692] Updated weights for policy 0, policy_version 1400141 (0.0011) [2023-12-27 01:35:21,040][105620] Updated weights for policy 1, policy_version 1402232 (0.0010) [2023-12-27 01:35:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.2, 300 sec: 19383.1). Total num frames: 717504512. Throughput: 0: 9746.4, 1: 9608.2. Samples: 717488844. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:35:21,063][104569] Avg episode reward: [(0, '8446.570'), (1, '8739.953')] [2023-12-27 01:35:21,716][105620] Updated weights for policy 1, policy_version 1402242 (0.0011) [2023-12-27 01:35:21,750][105692] Updated weights for policy 0, policy_version 1400151 (0.0010) [2023-12-27 01:35:21,780][105620] Updated weights for policy 1, policy_version 1402252 (0.0009) [2023-12-27 01:35:21,813][105692] Updated weights for policy 0, policy_version 1400161 (0.0008) [2023-12-27 01:35:21,836][105620] Updated weights for policy 1, policy_version 1402262 (0.0006) [2023-12-27 01:35:21,876][105692] Updated weights for policy 0, policy_version 1400171 (0.0008) [2023-12-27 01:35:22,588][105620] Updated weights for policy 1, policy_version 1402272 (0.0007) [2023-12-27 01:35:22,631][105692] Updated weights for policy 0, policy_version 1400181 (0.0010) [2023-12-27 01:35:22,648][105620] Updated weights for policy 1, policy_version 1402282 (0.0009) [2023-12-27 01:35:22,693][105692] Updated weights for policy 0, policy_version 1400191 (0.0008) [2023-12-27 01:35:22,708][105620] Updated weights for policy 1, policy_version 1402292 (0.0006) [2023-12-27 01:35:22,763][105692] Updated weights for policy 0, policy_version 1400201 (0.0009) [2023-12-27 01:35:23,382][105620] Updated weights for policy 1, policy_version 1402302 (0.0006) [2023-12-27 01:35:23,449][105620] Updated weights for policy 1, policy_version 1402312 (0.0011) [2023-12-27 01:35:23,512][105620] Updated weights for policy 1, policy_version 1402322 (0.0011) [2023-12-27 01:35:23,550][105692] Updated weights for policy 0, policy_version 1400211 (0.0008) [2023-12-27 01:35:23,610][105692] Updated weights for policy 0, policy_version 1400221 (0.0008) [2023-12-27 01:35:23,666][105692] Updated weights for policy 0, policy_version 1400231 (0.0007) [2023-12-27 01:35:24,131][105620] Updated weights for policy 1, policy_version 1402332 (0.0009) [2023-12-27 01:35:24,204][105620] Updated weights for policy 1, policy_version 1402342 (0.0005) [2023-12-27 01:35:24,270][105620] Updated weights for policy 1, policy_version 1402352 (0.0005) [2023-12-27 01:35:24,538][105692] Updated weights for policy 0, policy_version 1400241 (0.0008) [2023-12-27 01:35:24,589][105692] Updated weights for policy 0, policy_version 1400251 (0.0008) [2023-12-27 01:35:24,640][105692] Updated weights for policy 0, policy_version 1400261 (0.0007) [2023-12-27 01:35:24,699][105692] Updated weights for policy 0, policy_version 1400271 (0.0008) [2023-12-27 01:35:24,813][105620] Updated weights for policy 1, policy_version 1402362 (0.0005) [2023-12-27 01:35:24,876][105620] Updated weights for policy 1, policy_version 1402372 (0.0005) [2023-12-27 01:35:24,941][105620] Updated weights for policy 1, policy_version 1402382 (0.0006) [2023-12-27 01:35:24,999][105620] Updated weights for policy 1, policy_version 1402392 (0.0011) [2023-12-27 01:35:25,513][105620] Updated weights for policy 1, policy_version 1402402 (0.0011) [2023-12-27 01:35:25,568][105620] Updated weights for policy 1, policy_version 1402412 (0.0009) [2023-12-27 01:35:25,578][105692] Updated weights for policy 0, policy_version 1400281 (0.0007) [2023-12-27 01:35:25,630][105620] Updated weights for policy 1, policy_version 1402422 (0.0009) [2023-12-27 01:35:25,641][105692] Updated weights for policy 0, policy_version 1400291 (0.0006) [2023-12-27 01:35:25,700][105692] Updated weights for policy 0, policy_version 1400301 (0.0005) [2023-12-27 01:35:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.8, 300 sec: 19355.3). Total num frames: 717594624. Throughput: 0: 9683.7, 1: 9679.4. Samples: 717602976. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:35:26,062][104569] Avg episode reward: [(0, '8445.458'), (1, '8493.699')] [2023-12-27 01:35:26,270][105620] Updated weights for policy 1, policy_version 1402432 (0.0006) [2023-12-27 01:35:26,341][105620] Updated weights for policy 1, policy_version 1402442 (0.0005) [2023-12-27 01:35:26,411][105620] Updated weights for policy 1, policy_version 1402452 (0.0005) [2023-12-27 01:35:26,449][105692] Updated weights for policy 0, policy_version 1400311 (0.0008) [2023-12-27 01:35:26,505][105692] Updated weights for policy 0, policy_version 1400321 (0.0009) [2023-12-27 01:35:26,557][105692] Updated weights for policy 0, policy_version 1400331 (0.0009) [2023-12-27 01:35:26,954][105620] Updated weights for policy 1, policy_version 1402462 (0.0008) [2023-12-27 01:35:27,006][105620] Updated weights for policy 1, policy_version 1402472 (0.0010) [2023-12-27 01:35:27,053][105620] Updated weights for policy 1, policy_version 1402482 (0.0010) [2023-12-27 01:35:27,369][105692] Updated weights for policy 0, policy_version 1400341 (0.0010) [2023-12-27 01:35:27,430][105692] Updated weights for policy 0, policy_version 1400351 (0.0010) [2023-12-27 01:35:27,493][105692] Updated weights for policy 0, policy_version 1400361 (0.0010) [2023-12-27 01:35:27,656][105620] Updated weights for policy 1, policy_version 1402492 (0.0009) [2023-12-27 01:35:27,717][105620] Updated weights for policy 1, policy_version 1402502 (0.0010) [2023-12-27 01:35:27,778][105620] Updated weights for policy 1, policy_version 1402512 (0.0010) [2023-12-27 01:35:28,219][105692] Updated weights for policy 0, policy_version 1400372 (0.0009) [2023-12-27 01:35:28,270][105692] Updated weights for policy 0, policy_version 1400382 (0.0009) [2023-12-27 01:35:28,327][105692] Updated weights for policy 0, policy_version 1400392 (0.0008) [2023-12-27 01:35:28,357][105620] Updated weights for policy 1, policy_version 1402522 (0.0010) [2023-12-27 01:35:28,420][105620] Updated weights for policy 1, policy_version 1402532 (0.0006) [2023-12-27 01:35:28,484][105620] Updated weights for policy 1, policy_version 1402542 (0.0005) [2023-12-27 01:35:28,540][105620] Updated weights for policy 1, policy_version 1402552 (0.0005) [2023-12-27 01:35:29,114][105692] Updated weights for policy 0, policy_version 1400402 (0.0008) [2023-12-27 01:35:29,163][105620] Updated weights for policy 1, policy_version 1402562 (0.0006) [2023-12-27 01:35:29,176][105692] Updated weights for policy 0, policy_version 1400412 (0.0008) [2023-12-27 01:35:29,224][105620] Updated weights for policy 1, policy_version 1402572 (0.0006) [2023-12-27 01:35:29,228][105692] Updated weights for policy 0, policy_version 1400422 (0.0008) [2023-12-27 01:35:29,282][105620] Updated weights for policy 1, policy_version 1402582 (0.0008) [2023-12-27 01:35:29,291][105692] Updated weights for policy 0, policy_version 1400432 (0.0008) [2023-12-27 01:35:29,886][105620] Updated weights for policy 1, policy_version 1402592 (0.0010) [2023-12-27 01:35:29,950][105620] Updated weights for policy 1, policy_version 1402602 (0.0012) [2023-12-27 01:35:30,009][105620] Updated weights for policy 1, policy_version 1402612 (0.0010) [2023-12-27 01:35:30,127][105692] Updated weights for policy 0, policy_version 1400442 (0.0008) [2023-12-27 01:35:30,186][105692] Updated weights for policy 0, policy_version 1400452 (0.0008) [2023-12-27 01:35:30,241][105692] Updated weights for policy 0, policy_version 1400462 (0.0008) [2023-12-27 01:35:30,737][105620] Updated weights for policy 1, policy_version 1402622 (0.0007) [2023-12-27 01:35:30,786][105620] Updated weights for policy 1, policy_version 1402632 (0.0005) [2023-12-27 01:35:30,832][105620] Updated weights for policy 1, policy_version 1402642 (0.0005) [2023-12-27 01:35:30,924][105692] Updated weights for policy 0, policy_version 1400472 (0.0005) [2023-12-27 01:35:30,977][105692] Updated weights for policy 0, policy_version 1400482 (0.0005) [2023-12-27 01:35:31,026][105692] Updated weights for policy 0, policy_version 1400492 (0.0006) [2023-12-27 01:35:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 717701120. Throughput: 0: 9698.0, 1: 9731.9. Samples: 717664812. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:35:31,062][104569] Avg episode reward: [(0, '8633.354'), (1, '8655.539')] [2023-12-27 01:35:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001400496_358580224.pth... [2023-12-27 01:35:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001402648_359120896.pth... [2023-12-27 01:35:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001399376_358293504.pth [2023-12-27 01:35:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001401496_358825984.pth [2023-12-27 01:35:31,520][105620] Updated weights for policy 1, policy_version 1402652 (0.0007) [2023-12-27 01:35:31,577][105620] Updated weights for policy 1, policy_version 1402662 (0.0008) [2023-12-27 01:35:31,640][105620] Updated weights for policy 1, policy_version 1402672 (0.0008) [2023-12-27 01:35:31,755][105692] Updated weights for policy 0, policy_version 1400502 (0.0009) [2023-12-27 01:35:31,817][105692] Updated weights for policy 0, policy_version 1400512 (0.0009) [2023-12-27 01:35:31,882][105692] Updated weights for policy 0, policy_version 1400522 (0.0009) [2023-12-27 01:35:32,410][105620] Updated weights for policy 1, policy_version 1402682 (0.0009) [2023-12-27 01:35:32,460][105620] Updated weights for policy 1, policy_version 1402692 (0.0009) [2023-12-27 01:35:32,510][105620] Updated weights for policy 1, policy_version 1402702 (0.0008) [2023-12-27 01:35:32,571][105620] Updated weights for policy 1, policy_version 1402712 (0.0009) [2023-12-27 01:35:32,624][105692] Updated weights for policy 0, policy_version 1400532 (0.0009) [2023-12-27 01:35:32,686][105692] Updated weights for policy 0, policy_version 1400542 (0.0009) [2023-12-27 01:35:32,744][105692] Updated weights for policy 0, policy_version 1400552 (0.0009) [2023-12-27 01:35:33,306][105620] Updated weights for policy 1, policy_version 1402722 (0.0009) [2023-12-27 01:35:33,358][105620] Updated weights for policy 1, policy_version 1402732 (0.0008) [2023-12-27 01:35:33,426][105620] Updated weights for policy 1, policy_version 1402742 (0.0005) [2023-12-27 01:35:33,469][105692] Updated weights for policy 0, policy_version 1400562 (0.0010) [2023-12-27 01:35:33,529][105692] Updated weights for policy 0, policy_version 1400572 (0.0009) [2023-12-27 01:35:33,585][105692] Updated weights for policy 0, policy_version 1400582 (0.0009) [2023-12-27 01:35:33,644][105692] Updated weights for policy 0, policy_version 1400592 (0.0006) [2023-12-27 01:35:34,001][105620] Updated weights for policy 1, policy_version 1402752 (0.0009) [2023-12-27 01:35:34,061][105620] Updated weights for policy 1, policy_version 1402762 (0.0007) [2023-12-27 01:35:34,114][105620] Updated weights for policy 1, policy_version 1402772 (0.0010) [2023-12-27 01:35:34,470][105692] Updated weights for policy 0, policy_version 1400602 (0.0009) [2023-12-27 01:35:34,527][105692] Updated weights for policy 0, policy_version 1400612 (0.0010) [2023-12-27 01:35:34,583][105692] Updated weights for policy 0, policy_version 1400622 (0.0009) [2023-12-27 01:35:34,738][105620] Updated weights for policy 1, policy_version 1402782 (0.0007) [2023-12-27 01:35:34,792][105620] Updated weights for policy 1, policy_version 1402792 (0.0005) [2023-12-27 01:35:34,849][105620] Updated weights for policy 1, policy_version 1402802 (0.0007) [2023-12-27 01:35:35,441][105620] Updated weights for policy 1, policy_version 1402812 (0.0005) [2023-12-27 01:35:35,452][105692] Updated weights for policy 0, policy_version 1400632 (0.0009) [2023-12-27 01:35:35,489][105620] Updated weights for policy 1, policy_version 1402822 (0.0005) [2023-12-27 01:35:35,514][105692] Updated weights for policy 0, policy_version 1400642 (0.0009) [2023-12-27 01:35:35,537][105620] Updated weights for policy 1, policy_version 1402832 (0.0006) [2023-12-27 01:35:35,572][105692] Updated weights for policy 0, policy_version 1400652 (0.0007) [2023-12-27 01:35:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 717791232. Throughput: 0: 9592.3, 1: 9785.4. Samples: 717781840. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:35:36,062][104569] Avg episode reward: [(0, '8181.937'), (1, '8233.483')] [2023-12-27 01:35:36,213][105620] Updated weights for policy 1, policy_version 1402842 (0.0007) [2023-12-27 01:35:36,270][105620] Updated weights for policy 1, policy_version 1402852 (0.0008) [2023-12-27 01:35:36,323][105620] Updated weights for policy 1, policy_version 1402862 (0.0008) [2023-12-27 01:35:36,353][105692] Updated weights for policy 0, policy_version 1400662 (0.0007) [2023-12-27 01:35:36,371][105620] Updated weights for policy 1, policy_version 1402872 (0.0008) [2023-12-27 01:35:36,411][105692] Updated weights for policy 0, policy_version 1400672 (0.0007) [2023-12-27 01:35:36,485][105692] Updated weights for policy 0, policy_version 1400682 (0.0008) [2023-12-27 01:35:37,158][105620] Updated weights for policy 1, policy_version 1402882 (0.0010) [2023-12-27 01:35:37,168][105692] Updated weights for policy 0, policy_version 1400692 (0.0008) [2023-12-27 01:35:37,214][105620] Updated weights for policy 1, policy_version 1402892 (0.0010) [2023-12-27 01:35:37,222][105692] Updated weights for policy 0, policy_version 1400702 (0.0008) [2023-12-27 01:35:37,276][105620] Updated weights for policy 1, policy_version 1402902 (0.0010) [2023-12-27 01:35:37,279][105692] Updated weights for policy 0, policy_version 1400712 (0.0006) [2023-12-27 01:35:37,898][105692] Updated weights for policy 0, policy_version 1400722 (0.0007) [2023-12-27 01:35:37,951][105692] Updated weights for policy 0, policy_version 1400732 (0.0005) [2023-12-27 01:35:38,006][105692] Updated weights for policy 0, policy_version 1400742 (0.0010) [2023-12-27 01:35:38,016][105620] Updated weights for policy 1, policy_version 1402912 (0.0011) [2023-12-27 01:35:38,066][105692] Updated weights for policy 0, policy_version 1400752 (0.0011) [2023-12-27 01:35:38,068][105620] Updated weights for policy 1, policy_version 1402922 (0.0010) [2023-12-27 01:35:38,117][105620] Updated weights for policy 1, policy_version 1402932 (0.0010) [2023-12-27 01:35:38,702][105692] Updated weights for policy 0, policy_version 1400762 (0.0011) [2023-12-27 01:35:38,756][105692] Updated weights for policy 0, policy_version 1400772 (0.0010) [2023-12-27 01:35:38,818][105692] Updated weights for policy 0, policy_version 1400782 (0.0010) [2023-12-27 01:35:38,899][105620] Updated weights for policy 1, policy_version 1402942 (0.0010) [2023-12-27 01:35:38,962][105620] Updated weights for policy 1, policy_version 1402952 (0.0007) [2023-12-27 01:35:39,014][105620] Updated weights for policy 1, policy_version 1402962 (0.0009) [2023-12-27 01:35:39,549][105692] Updated weights for policy 0, policy_version 1400792 (0.0010) [2023-12-27 01:35:39,611][105692] Updated weights for policy 0, policy_version 1400802 (0.0010) [2023-12-27 01:35:39,646][105620] Updated weights for policy 1, policy_version 1402972 (0.0008) [2023-12-27 01:35:39,673][105692] Updated weights for policy 0, policy_version 1400812 (0.0008) [2023-12-27 01:35:39,695][105620] Updated weights for policy 1, policy_version 1402982 (0.0006) [2023-12-27 01:35:39,742][105620] Updated weights for policy 1, policy_version 1402992 (0.0009) [2023-12-27 01:35:40,407][105692] Updated weights for policy 0, policy_version 1400822 (0.0007) [2023-12-27 01:35:40,468][105692] Updated weights for policy 0, policy_version 1400832 (0.0007) [2023-12-27 01:35:40,485][105620] Updated weights for policy 1, policy_version 1403002 (0.0009) [2023-12-27 01:35:40,526][105692] Updated weights for policy 0, policy_version 1400842 (0.0010) [2023-12-27 01:35:40,553][105620] Updated weights for policy 1, policy_version 1403012 (0.0011) [2023-12-27 01:35:40,616][105620] Updated weights for policy 1, policy_version 1403022 (0.0011) [2023-12-27 01:35:40,680][105620] Updated weights for policy 1, policy_version 1403032 (0.0008) [2023-12-27 01:35:41,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19355.4). Total num frames: 717889536. Throughput: 0: 9582.0, 1: 9803.4. Samples: 717898692. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:35:41,063][104569] Avg episode reward: [(0, '7990.604'), (1, '8119.917')] [2023-12-27 01:35:41,293][105692] Updated weights for policy 0, policy_version 1400852 (0.0007) [2023-12-27 01:35:41,343][105692] Updated weights for policy 0, policy_version 1400862 (0.0009) [2023-12-27 01:35:41,410][105692] Updated weights for policy 0, policy_version 1400872 (0.0007) [2023-12-27 01:35:41,425][105620] Updated weights for policy 1, policy_version 1403042 (0.0006) [2023-12-27 01:35:41,483][105620] Updated weights for policy 1, policy_version 1403052 (0.0007) [2023-12-27 01:35:41,545][105620] Updated weights for policy 1, policy_version 1403062 (0.0008) [2023-12-27 01:35:42,197][105692] Updated weights for policy 0, policy_version 1400882 (0.0009) [2023-12-27 01:35:42,245][105692] Updated weights for policy 0, policy_version 1400892 (0.0009) [2023-12-27 01:35:42,306][105692] Updated weights for policy 0, policy_version 1400902 (0.0008) [2023-12-27 01:35:42,313][105620] Updated weights for policy 1, policy_version 1403072 (0.0008) [2023-12-27 01:35:42,367][105692] Updated weights for policy 0, policy_version 1400912 (0.0008) [2023-12-27 01:35:42,371][105620] Updated weights for policy 1, policy_version 1403082 (0.0007) [2023-12-27 01:35:42,431][105620] Updated weights for policy 1, policy_version 1403092 (0.0009) [2023-12-27 01:35:43,122][105692] Updated weights for policy 0, policy_version 1400922 (0.0009) [2023-12-27 01:35:43,173][105692] Updated weights for policy 0, policy_version 1400932 (0.0010) [2023-12-27 01:35:43,197][105620] Updated weights for policy 1, policy_version 1403103 (0.0007) [2023-12-27 01:35:43,238][105692] Updated weights for policy 0, policy_version 1400942 (0.0008) [2023-12-27 01:35:43,243][105620] Updated weights for policy 1, policy_version 1403113 (0.0006) [2023-12-27 01:35:43,289][105620] Updated weights for policy 1, policy_version 1403123 (0.0005) [2023-12-27 01:35:43,843][105620] Updated weights for policy 1, policy_version 1403133 (0.0005) [2023-12-27 01:35:43,891][105620] Updated weights for policy 1, policy_version 1403143 (0.0005) [2023-12-27 01:35:43,943][105620] Updated weights for policy 1, policy_version 1403153 (0.0005) [2023-12-27 01:35:44,132][105692] Updated weights for policy 0, policy_version 1400952 (0.0008) [2023-12-27 01:35:44,177][105692] Updated weights for policy 0, policy_version 1400962 (0.0008) [2023-12-27 01:35:44,229][105692] Updated weights for policy 0, policy_version 1400972 (0.0008) [2023-12-27 01:35:44,613][105620] Updated weights for policy 1, policy_version 1403163 (0.0007) [2023-12-27 01:35:44,674][105620] Updated weights for policy 1, policy_version 1403173 (0.0010) [2023-12-27 01:35:44,726][105620] Updated weights for policy 1, policy_version 1403183 (0.0010) [2023-12-27 01:35:44,987][105692] Updated weights for policy 0, policy_version 1400982 (0.0007) [2023-12-27 01:35:45,054][105692] Updated weights for policy 0, policy_version 1400992 (0.0007) [2023-12-27 01:35:45,109][105692] Updated weights for policy 0, policy_version 1401002 (0.0006) [2023-12-27 01:35:45,495][105620] Updated weights for policy 1, policy_version 1403193 (0.0011) [2023-12-27 01:35:45,550][105620] Updated weights for policy 1, policy_version 1403203 (0.0010) [2023-12-27 01:35:45,615][105620] Updated weights for policy 1, policy_version 1403213 (0.0010) [2023-12-27 01:35:45,665][105692] Updated weights for policy 0, policy_version 1401012 (0.0008) [2023-12-27 01:35:45,669][105620] Updated weights for policy 1, policy_version 1403223 (0.0010) [2023-12-27 01:35:45,727][105692] Updated weights for policy 0, policy_version 1401022 (0.0011) [2023-12-27 01:35:45,785][105692] Updated weights for policy 0, policy_version 1401032 (0.0010) [2023-12-27 01:35:46,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19524.2, 300 sec: 19383.1). Total num frames: 717987840. Throughput: 0: 9501.7, 1: 9843.2. Samples: 717955860. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:35:46,063][104569] Avg episode reward: [(0, '7805.686'), (1, '8319.944')] [2023-12-27 01:35:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001401040_358719488.pth... [2023-12-27 01:35:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001403224_359268352.pth... [2023-12-27 01:35:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001399952_358440960.pth [2023-12-27 01:35:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001402040_358965248.pth [2023-12-27 01:35:46,366][105620] Updated weights for policy 1, policy_version 1403233 (0.0010) [2023-12-27 01:35:46,414][105620] Updated weights for policy 1, policy_version 1403243 (0.0005) [2023-12-27 01:35:46,471][105620] Updated weights for policy 1, policy_version 1403253 (0.0005) [2023-12-27 01:35:46,508][105692] Updated weights for policy 0, policy_version 1401042 (0.0008) [2023-12-27 01:35:46,570][105692] Updated weights for policy 0, policy_version 1401052 (0.0010) [2023-12-27 01:35:46,618][105692] Updated weights for policy 0, policy_version 1401062 (0.0010) [2023-12-27 01:35:46,662][105692] Updated weights for policy 0, policy_version 1401072 (0.0010) [2023-12-27 01:35:47,117][105620] Updated weights for policy 1, policy_version 1403263 (0.0007) [2023-12-27 01:35:47,166][105620] Updated weights for policy 1, policy_version 1403273 (0.0005) [2023-12-27 01:35:47,223][105620] Updated weights for policy 1, policy_version 1403283 (0.0008) [2023-12-27 01:35:47,412][105692] Updated weights for policy 0, policy_version 1401082 (0.0010) [2023-12-27 01:35:47,459][105692] Updated weights for policy 0, policy_version 1401092 (0.0010) [2023-12-27 01:35:47,504][105692] Updated weights for policy 0, policy_version 1401102 (0.0010) [2023-12-27 01:35:47,889][105620] Updated weights for policy 1, policy_version 1403293 (0.0007) [2023-12-27 01:35:47,952][105620] Updated weights for policy 1, policy_version 1403303 (0.0005) [2023-12-27 01:35:48,021][105620] Updated weights for policy 1, policy_version 1403313 (0.0005) [2023-12-27 01:35:48,291][105692] Updated weights for policy 0, policy_version 1401112 (0.0010) [2023-12-27 01:35:48,360][105692] Updated weights for policy 0, policy_version 1401122 (0.0011) [2023-12-27 01:35:48,430][105692] Updated weights for policy 0, policy_version 1401132 (0.0010) [2023-12-27 01:35:48,621][105620] Updated weights for policy 1, policy_version 1403323 (0.0007) [2023-12-27 01:35:48,677][105620] Updated weights for policy 1, policy_version 1403333 (0.0008) [2023-12-27 01:35:48,736][105620] Updated weights for policy 1, policy_version 1403343 (0.0008) [2023-12-27 01:35:49,095][105692] Updated weights for policy 0, policy_version 1401142 (0.0008) [2023-12-27 01:35:49,153][105692] Updated weights for policy 0, policy_version 1401152 (0.0009) [2023-12-27 01:35:49,209][105692] Updated weights for policy 0, policy_version 1401162 (0.0005) [2023-12-27 01:35:49,544][105620] Updated weights for policy 1, policy_version 1403353 (0.0008) [2023-12-27 01:35:49,599][105620] Updated weights for policy 1, policy_version 1403363 (0.0007) [2023-12-27 01:35:49,666][105620] Updated weights for policy 1, policy_version 1403373 (0.0006) [2023-12-27 01:35:49,728][105620] Updated weights for policy 1, policy_version 1403383 (0.0008) [2023-12-27 01:35:49,965][105692] Updated weights for policy 0, policy_version 1401172 (0.0008) [2023-12-27 01:35:50,029][105692] Updated weights for policy 0, policy_version 1401182 (0.0010) [2023-12-27 01:35:50,088][105692] Updated weights for policy 0, policy_version 1401192 (0.0010) [2023-12-27 01:35:50,444][105620] Updated weights for policy 1, policy_version 1403393 (0.0007) [2023-12-27 01:35:50,503][105620] Updated weights for policy 1, policy_version 1403403 (0.0005) [2023-12-27 01:35:50,555][105620] Updated weights for policy 1, policy_version 1403413 (0.0005) [2023-12-27 01:35:50,844][105692] Updated weights for policy 0, policy_version 1401202 (0.0011) [2023-12-27 01:35:50,904][105692] Updated weights for policy 0, policy_version 1401212 (0.0010) [2023-12-27 01:35:50,966][105692] Updated weights for policy 0, policy_version 1401222 (0.0010) [2023-12-27 01:35:51,025][105692] Updated weights for policy 0, policy_version 1401232 (0.0010) [2023-12-27 01:35:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19383.1). Total num frames: 718086144. Throughput: 0: 9490.8, 1: 9928.9. Samples: 718073484. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:35:51,063][104569] Avg episode reward: [(0, '7902.517'), (1, '8494.116')] [2023-12-27 01:35:51,217][105620] Updated weights for policy 1, policy_version 1403423 (0.0008) [2023-12-27 01:35:51,272][105620] Updated weights for policy 1, policy_version 1403433 (0.0009) [2023-12-27 01:35:51,331][105620] Updated weights for policy 1, policy_version 1403443 (0.0008) [2023-12-27 01:35:51,845][105692] Updated weights for policy 0, policy_version 1401242 (0.0010) [2023-12-27 01:35:51,894][105692] Updated weights for policy 0, policy_version 1401252 (0.0010) [2023-12-27 01:35:51,951][105692] Updated weights for policy 0, policy_version 1401262 (0.0011) [2023-12-27 01:35:52,102][105620] Updated weights for policy 1, policy_version 1403453 (0.0008) [2023-12-27 01:35:52,159][105620] Updated weights for policy 1, policy_version 1403463 (0.0006) [2023-12-27 01:35:52,224][105620] Updated weights for policy 1, policy_version 1403473 (0.0007) [2023-12-27 01:35:52,700][105692] Updated weights for policy 0, policy_version 1401272 (0.0009) [2023-12-27 01:35:52,770][105692] Updated weights for policy 0, policy_version 1401282 (0.0008) [2023-12-27 01:35:52,835][105692] Updated weights for policy 0, policy_version 1401292 (0.0009) [2023-12-27 01:35:52,876][105620] Updated weights for policy 1, policy_version 1403483 (0.0007) [2023-12-27 01:35:52,935][105620] Updated weights for policy 1, policy_version 1403493 (0.0006) [2023-12-27 01:35:52,991][105620] Updated weights for policy 1, policy_version 1403503 (0.0006) [2023-12-27 01:35:53,523][105692] Updated weights for policy 0, policy_version 1401302 (0.0007) [2023-12-27 01:35:53,568][105692] Updated weights for policy 0, policy_version 1401312 (0.0007) [2023-12-27 01:35:53,608][105620] Updated weights for policy 1, policy_version 1403513 (0.0011) [2023-12-27 01:35:53,616][105692] Updated weights for policy 0, policy_version 1401322 (0.0010) [2023-12-27 01:35:53,667][105620] Updated weights for policy 1, policy_version 1403523 (0.0010) [2023-12-27 01:35:53,711][105620] Updated weights for policy 1, policy_version 1403533 (0.0008) [2023-12-27 01:35:53,762][105620] Updated weights for policy 1, policy_version 1403543 (0.0010) [2023-12-27 01:35:54,277][105692] Updated weights for policy 0, policy_version 1401332 (0.0008) [2023-12-27 01:35:54,328][105692] Updated weights for policy 0, policy_version 1401342 (0.0005) [2023-12-27 01:35:54,399][105692] Updated weights for policy 0, policy_version 1401352 (0.0008) [2023-12-27 01:35:54,460][105620] Updated weights for policy 1, policy_version 1403553 (0.0006) [2023-12-27 01:35:54,520][105620] Updated weights for policy 1, policy_version 1403563 (0.0008) [2023-12-27 01:35:54,578][105620] Updated weights for policy 1, policy_version 1403573 (0.0008) [2023-12-27 01:35:54,997][105692] Updated weights for policy 0, policy_version 1401362 (0.0010) [2023-12-27 01:35:55,061][105692] Updated weights for policy 0, policy_version 1401372 (0.0010) [2023-12-27 01:35:55,119][105692] Updated weights for policy 0, policy_version 1401382 (0.0010) [2023-12-27 01:35:55,176][105692] Updated weights for policy 0, policy_version 1401392 (0.0010) [2023-12-27 01:35:55,315][105620] Updated weights for policy 1, policy_version 1403583 (0.0010) [2023-12-27 01:35:55,363][105620] Updated weights for policy 1, policy_version 1403593 (0.0010) [2023-12-27 01:35:55,416][105620] Updated weights for policy 1, policy_version 1403603 (0.0007) [2023-12-27 01:35:55,908][105692] Updated weights for policy 0, policy_version 1401402 (0.0010) [2023-12-27 01:35:55,970][105692] Updated weights for policy 0, policy_version 1401412 (0.0010) [2023-12-27 01:35:56,033][105692] Updated weights for policy 0, policy_version 1401422 (0.0009) [2023-12-27 01:35:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.2, 300 sec: 19383.1). Total num frames: 718184448. Throughput: 0: 9515.1, 1: 10032.5. Samples: 718192012. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:35:56,063][104569] Avg episode reward: [(0, '8264.124'), (1, '8915.289')] [2023-12-27 01:35:56,189][105620] Updated weights for policy 1, policy_version 1403613 (0.0005) [2023-12-27 01:35:56,249][105620] Updated weights for policy 1, policy_version 1403623 (0.0006) [2023-12-27 01:35:56,308][105620] Updated weights for policy 1, policy_version 1403633 (0.0006) [2023-12-27 01:35:56,763][105692] Updated weights for policy 0, policy_version 1401432 (0.0006) [2023-12-27 01:35:56,824][105692] Updated weights for policy 0, policy_version 1401442 (0.0010) [2023-12-27 01:35:56,875][105692] Updated weights for policy 0, policy_version 1401452 (0.0010) [2023-12-27 01:35:56,950][105620] Updated weights for policy 1, policy_version 1403643 (0.0006) [2023-12-27 01:35:57,007][105620] Updated weights for policy 1, policy_version 1403653 (0.0006) [2023-12-27 01:35:57,071][105620] Updated weights for policy 1, policy_version 1403663 (0.0005) [2023-12-27 01:35:57,445][105692] Updated weights for policy 0, policy_version 1401462 (0.0007) [2023-12-27 01:35:57,476][105585] KL-divergence is very high: 233.9782 [2023-12-27 01:35:57,496][105692] Updated weights for policy 0, policy_version 1401472 (0.0005) [2023-12-27 01:35:57,525][105585] KL-divergence is very high: 406.4243 [2023-12-27 01:35:57,557][105692] Updated weights for policy 0, policy_version 1401482 (0.0006) [2023-12-27 01:35:57,570][105585] KL-divergence is very high: 408.9515 [2023-12-27 01:35:57,738][105620] Updated weights for policy 1, policy_version 1403673 (0.0010) [2023-12-27 01:35:57,788][105620] Updated weights for policy 1, policy_version 1403683 (0.0010) [2023-12-27 01:35:57,842][105620] Updated weights for policy 1, policy_version 1403693 (0.0010) [2023-12-27 01:35:57,886][105620] Updated weights for policy 1, policy_version 1403703 (0.0010) [2023-12-27 01:35:58,048][105585] KL-divergence is very high: 124.2676 [2023-12-27 01:35:58,072][105692] Updated weights for policy 0, policy_version 1401492 (0.0005) [2023-12-27 01:35:58,091][105585] KL-divergence is very high: 104.7276 [2023-12-27 01:35:58,122][105692] Updated weights for policy 0, policy_version 1401502 (0.0005) [2023-12-27 01:35:58,180][105692] Updated weights for policy 0, policy_version 1401512 (0.0007) [2023-12-27 01:35:58,675][105620] Updated weights for policy 1, policy_version 1403713 (0.0008) [2023-12-27 01:35:58,740][105620] Updated weights for policy 1, policy_version 1403723 (0.0008) [2023-12-27 01:35:58,805][105620] Updated weights for policy 1, policy_version 1403733 (0.0008) [2023-12-27 01:35:58,969][105692] Updated weights for policy 0, policy_version 1401522 (0.0008) [2023-12-27 01:35:59,029][105692] Updated weights for policy 0, policy_version 1401532 (0.0008) [2023-12-27 01:35:59,088][105692] Updated weights for policy 0, policy_version 1401542 (0.0008) [2023-12-27 01:35:59,154][105692] Updated weights for policy 0, policy_version 1401552 (0.0008) [2023-12-27 01:35:59,563][105620] Updated weights for policy 1, policy_version 1403743 (0.0009) [2023-12-27 01:35:59,623][105620] Updated weights for policy 1, policy_version 1403753 (0.0011) [2023-12-27 01:35:59,685][105620] Updated weights for policy 1, policy_version 1403763 (0.0011) [2023-12-27 01:35:59,890][105692] Updated weights for policy 0, policy_version 1401562 (0.0005) [2023-12-27 01:35:59,959][105692] Updated weights for policy 0, policy_version 1401572 (0.0007) [2023-12-27 01:36:00,016][105692] Updated weights for policy 0, policy_version 1401582 (0.0009) [2023-12-27 01:36:00,355][105620] Updated weights for policy 1, policy_version 1403773 (0.0008) [2023-12-27 01:36:00,412][105620] Updated weights for policy 1, policy_version 1403783 (0.0005) [2023-12-27 01:36:00,466][105620] Updated weights for policy 1, policy_version 1403793 (0.0007) [2023-12-27 01:36:00,700][105692] Updated weights for policy 0, policy_version 1401592 (0.0006) [2023-12-27 01:36:00,750][105692] Updated weights for policy 0, policy_version 1401602 (0.0005) [2023-12-27 01:36:00,806][105692] Updated weights for policy 0, policy_version 1401612 (0.0007) [2023-12-27 01:36:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 718282752. Throughput: 0: 9558.9, 1: 10088.5. Samples: 718253400. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:36:01,062][104569] Avg episode reward: [(0, '8080.817'), (1, '8737.289')] [2023-12-27 01:36:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001401616_358866944.pth... [2023-12-27 01:36:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001403800_359415808.pth... [2023-12-27 01:36:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001400496_358580224.pth [2023-12-27 01:36:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001402648_359120896.pth [2023-12-27 01:36:01,152][105620] Updated weights for policy 1, policy_version 1403803 (0.0008) [2023-12-27 01:36:01,209][105620] Updated weights for policy 1, policy_version 1403813 (0.0005) [2023-12-27 01:36:01,262][105620] Updated weights for policy 1, policy_version 1403823 (0.0005) [2023-12-27 01:36:01,530][105692] Updated weights for policy 0, policy_version 1401622 (0.0006) [2023-12-27 01:36:01,580][105692] Updated weights for policy 0, policy_version 1401632 (0.0008) [2023-12-27 01:36:01,633][105692] Updated weights for policy 0, policy_version 1401642 (0.0008) [2023-12-27 01:36:02,001][105620] Updated weights for policy 1, policy_version 1403833 (0.0007) [2023-12-27 01:36:02,068][105620] Updated weights for policy 1, policy_version 1403843 (0.0010) [2023-12-27 01:36:02,135][105620] Updated weights for policy 1, policy_version 1403853 (0.0010) [2023-12-27 01:36:02,197][105620] Updated weights for policy 1, policy_version 1403863 (0.0010) [2023-12-27 01:36:02,288][105692] Updated weights for policy 0, policy_version 1401652 (0.0008) [2023-12-27 01:36:02,334][105692] Updated weights for policy 0, policy_version 1401662 (0.0008) [2023-12-27 01:36:02,397][105692] Updated weights for policy 0, policy_version 1401672 (0.0009) [2023-12-27 01:36:02,962][105692] Updated weights for policy 0, policy_version 1401682 (0.0008) [2023-12-27 01:36:03,015][105692] Updated weights for policy 0, policy_version 1401692 (0.0005) [2023-12-27 01:36:03,067][105692] Updated weights for policy 0, policy_version 1401702 (0.0005) [2023-12-27 01:36:03,088][105620] Updated weights for policy 1, policy_version 1403873 (0.0009) [2023-12-27 01:36:03,141][105620] Updated weights for policy 1, policy_version 1403883 (0.0008) [2023-12-27 01:36:03,191][105620] Updated weights for policy 1, policy_version 1403893 (0.0010) [2023-12-27 01:36:03,654][105692] Updated weights for policy 0, policy_version 1401713 (0.0009) [2023-12-27 01:36:03,722][105692] Updated weights for policy 0, policy_version 1401723 (0.0005) [2023-12-27 01:36:03,788][105692] Updated weights for policy 0, policy_version 1401733 (0.0005) [2023-12-27 01:36:03,856][105692] Updated weights for policy 0, policy_version 1401743 (0.0007) [2023-12-27 01:36:03,993][105620] Updated weights for policy 1, policy_version 1403903 (0.0009) [2023-12-27 01:36:04,056][105620] Updated weights for policy 1, policy_version 1403913 (0.0009) [2023-12-27 01:36:04,112][105620] Updated weights for policy 1, policy_version 1403923 (0.0009) [2023-12-27 01:36:04,510][105692] Updated weights for policy 0, policy_version 1401753 (0.0007) [2023-12-27 01:36:04,565][105692] Updated weights for policy 0, policy_version 1401763 (0.0005) [2023-12-27 01:36:04,623][105692] Updated weights for policy 0, policy_version 1401773 (0.0008) [2023-12-27 01:36:04,928][105620] Updated weights for policy 1, policy_version 1403933 (0.0009) [2023-12-27 01:36:04,987][105620] Updated weights for policy 1, policy_version 1403943 (0.0008) [2023-12-27 01:36:05,050][105620] Updated weights for policy 1, policy_version 1403953 (0.0006) [2023-12-27 01:36:05,322][105692] Updated weights for policy 0, policy_version 1401783 (0.0009) [2023-12-27 01:36:05,369][105692] Updated weights for policy 0, policy_version 1401793 (0.0008) [2023-12-27 01:36:05,420][105692] Updated weights for policy 0, policy_version 1401803 (0.0009) [2023-12-27 01:36:05,813][105620] Updated weights for policy 1, policy_version 1403963 (0.0008) [2023-12-27 01:36:05,858][105620] Updated weights for policy 1, policy_version 1403973 (0.0008) [2023-12-27 01:36:05,902][105620] Updated weights for policy 1, policy_version 1403983 (0.0005) [2023-12-27 01:36:06,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 718381056. Throughput: 0: 9578.1, 1: 10012.6. Samples: 718370420. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:36:06,062][104569] Avg episode reward: [(0, '8261.197'), (1, '8680.662')] [2023-12-27 01:36:06,122][105692] Updated weights for policy 0, policy_version 1401813 (0.0009) [2023-12-27 01:36:06,173][105692] Updated weights for policy 0, policy_version 1401823 (0.0009) [2023-12-27 01:36:06,230][105692] Updated weights for policy 0, policy_version 1401833 (0.0010) [2023-12-27 01:36:06,648][105620] Updated weights for policy 1, policy_version 1403993 (0.0007) [2023-12-27 01:36:06,710][105620] Updated weights for policy 1, policy_version 1404003 (0.0008) [2023-12-27 01:36:06,768][105620] Updated weights for policy 1, policy_version 1404013 (0.0009) [2023-12-27 01:36:06,830][105620] Updated weights for policy 1, policy_version 1404023 (0.0009) [2023-12-27 01:36:06,999][105692] Updated weights for policy 0, policy_version 1401843 (0.0009) [2023-12-27 01:36:07,050][105692] Updated weights for policy 0, policy_version 1401853 (0.0008) [2023-12-27 01:36:07,102][105692] Updated weights for policy 0, policy_version 1401863 (0.0009) [2023-12-27 01:36:07,485][105620] Updated weights for policy 1, policy_version 1404033 (0.0008) [2023-12-27 01:36:07,547][105620] Updated weights for policy 1, policy_version 1404043 (0.0009) [2023-12-27 01:36:07,597][105620] Updated weights for policy 1, policy_version 1404053 (0.0008) [2023-12-27 01:36:07,902][105692] Updated weights for policy 0, policy_version 1401873 (0.0009) [2023-12-27 01:36:07,959][105692] Updated weights for policy 0, policy_version 1401883 (0.0006) [2023-12-27 01:36:08,012][105692] Updated weights for policy 0, policy_version 1401893 (0.0006) [2023-12-27 01:36:08,066][105692] Updated weights for policy 0, policy_version 1401903 (0.0006) [2023-12-27 01:36:08,410][105620] Updated weights for policy 1, policy_version 1404063 (0.0009) [2023-12-27 01:36:08,473][105620] Updated weights for policy 1, policy_version 1404073 (0.0008) [2023-12-27 01:36:08,537][105620] Updated weights for policy 1, policy_version 1404083 (0.0009) [2023-12-27 01:36:08,720][105692] Updated weights for policy 0, policy_version 1401913 (0.0009) [2023-12-27 01:36:08,745][105585] KL-divergence is very high: 101.0386 [2023-12-27 01:36:08,775][105692] Updated weights for policy 0, policy_version 1401923 (0.0008) [2023-12-27 01:36:08,797][105585] KL-divergence is very high: 192.4620 [2023-12-27 01:36:08,839][105692] Updated weights for policy 0, policy_version 1401933 (0.0007) [2023-12-27 01:36:08,842][105585] KL-divergence is very high: 233.6602 [2023-12-27 01:36:09,338][105620] Updated weights for policy 1, policy_version 1404093 (0.0009) [2023-12-27 01:36:09,407][105620] Updated weights for policy 1, policy_version 1404103 (0.0009) [2023-12-27 01:36:09,473][105620] Updated weights for policy 1, policy_version 1404113 (0.0008) [2023-12-27 01:36:09,533][105692] Updated weights for policy 0, policy_version 1401943 (0.0010) [2023-12-27 01:36:09,594][105692] Updated weights for policy 0, policy_version 1401953 (0.0011) [2023-12-27 01:36:09,647][105692] Updated weights for policy 0, policy_version 1401963 (0.0010) [2023-12-27 01:36:10,256][105620] Updated weights for policy 1, policy_version 1404123 (0.0009) [2023-12-27 01:36:10,320][105620] Updated weights for policy 1, policy_version 1404133 (0.0008) [2023-12-27 01:36:10,381][105620] Updated weights for policy 1, policy_version 1404143 (0.0008) [2023-12-27 01:36:10,430][105692] Updated weights for policy 0, policy_version 1401973 (0.0009) [2023-12-27 01:36:10,495][105692] Updated weights for policy 0, policy_version 1401983 (0.0009) [2023-12-27 01:36:10,555][105692] Updated weights for policy 0, policy_version 1401993 (0.0008) [2023-12-27 01:36:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19355.3). Total num frames: 718471168. Throughput: 0: 9685.1, 1: 9864.3. Samples: 718482700. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:36:11,063][104569] Avg episode reward: [(0, '8714.455'), (1, '2933.137')] [2023-12-27 01:36:11,191][105620] Updated weights for policy 1, policy_version 1404153 (0.0007) [2023-12-27 01:36:11,253][105620] Updated weights for policy 1, policy_version 1404163 (0.0006) [2023-12-27 01:36:11,274][105692] Updated weights for policy 0, policy_version 1402003 (0.0009) [2023-12-27 01:36:11,316][105620] Updated weights for policy 1, policy_version 1404173 (0.0008) [2023-12-27 01:36:11,334][105692] Updated weights for policy 0, policy_version 1402013 (0.0008) [2023-12-27 01:36:11,385][105620] Updated weights for policy 1, policy_version 1404183 (0.0009) [2023-12-27 01:36:11,405][105692] Updated weights for policy 0, policy_version 1402023 (0.0008) [2023-12-27 01:36:12,064][105620] Updated weights for policy 1, policy_version 1404193 (0.0008) [2023-12-27 01:36:12,129][105620] Updated weights for policy 1, policy_version 1404203 (0.0009) [2023-12-27 01:36:12,191][105620] Updated weights for policy 1, policy_version 1404213 (0.0009) [2023-12-27 01:36:12,219][105692] Updated weights for policy 0, policy_version 1402033 (0.0009) [2023-12-27 01:36:12,290][105692] Updated weights for policy 0, policy_version 1402043 (0.0009) [2023-12-27 01:36:12,359][105692] Updated weights for policy 0, policy_version 1402053 (0.0009) [2023-12-27 01:36:12,426][105692] Updated weights for policy 0, policy_version 1402063 (0.0008) [2023-12-27 01:36:12,809][105620] Updated weights for policy 1, policy_version 1404223 (0.0007) [2023-12-27 01:36:12,863][105620] Updated weights for policy 1, policy_version 1404233 (0.0005) [2023-12-27 01:36:12,923][105620] Updated weights for policy 1, policy_version 1404243 (0.0006) [2023-12-27 01:36:13,242][105692] Updated weights for policy 0, policy_version 1402074 (0.0010) [2023-12-27 01:36:13,311][105692] Updated weights for policy 0, policy_version 1402085 (0.0009) [2023-12-27 01:36:13,377][105692] Updated weights for policy 0, policy_version 1402095 (0.0010) [2023-12-27 01:36:13,446][105620] Updated weights for policy 1, policy_version 1404253 (0.0005) [2023-12-27 01:36:13,491][105620] Updated weights for policy 1, policy_version 1404263 (0.0007) [2023-12-27 01:36:13,549][105620] Updated weights for policy 1, policy_version 1404273 (0.0010) [2023-12-27 01:36:14,176][105620] Updated weights for policy 1, policy_version 1404283 (0.0010) [2023-12-27 01:36:14,211][105692] Updated weights for policy 0, policy_version 1402105 (0.0006) [2023-12-27 01:36:14,228][105620] Updated weights for policy 1, policy_version 1404293 (0.0010) [2023-12-27 01:36:14,264][105692] Updated weights for policy 0, policy_version 1402115 (0.0008) [2023-12-27 01:36:14,277][105620] Updated weights for policy 1, policy_version 1404303 (0.0010) [2023-12-27 01:36:14,323][105692] Updated weights for policy 0, policy_version 1402125 (0.0008) [2023-12-27 01:36:14,922][105620] Updated weights for policy 1, policy_version 1404313 (0.0006) [2023-12-27 01:36:14,996][105620] Updated weights for policy 1, policy_version 1404323 (0.0005) [2023-12-27 01:36:15,062][105620] Updated weights for policy 1, policy_version 1404333 (0.0006) [2023-12-27 01:36:15,122][105620] Updated weights for policy 1, policy_version 1404343 (0.0006) [2023-12-27 01:36:15,158][105692] Updated weights for policy 0, policy_version 1402135 (0.0007) [2023-12-27 01:36:15,230][105692] Updated weights for policy 0, policy_version 1402145 (0.0010) [2023-12-27 01:36:15,298][105692] Updated weights for policy 0, policy_version 1402155 (0.0010) [2023-12-27 01:36:15,694][105620] Updated weights for policy 1, policy_version 1404353 (0.0008) [2023-12-27 01:36:15,749][105620] Updated weights for policy 1, policy_version 1404363 (0.0009) [2023-12-27 01:36:15,800][105620] Updated weights for policy 1, policy_version 1404373 (0.0008) [2023-12-27 01:36:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 718569472. Throughput: 0: 9664.4, 1: 9811.6. Samples: 718541228. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:36:16,062][104569] Avg episode reward: [(0, '8809.562'), (1, '3641.493')] [2023-12-27 01:36:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001404376_359563264.pth... [2023-12-27 01:36:16,069][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001403224_359268352.pth [2023-12-27 01:36:16,088][105692] Updated weights for policy 0, policy_version 1402165 (0.0010) [2023-12-27 01:36:16,143][105692] Updated weights for policy 0, policy_version 1402176 (0.0010) [2023-12-27 01:36:16,199][105692] Updated weights for policy 0, policy_version 1402186 (0.0006) [2023-12-27 01:36:16,228][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001402192_359014400.pth... [2023-12-27 01:36:16,231][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001401040_358719488.pth [2023-12-27 01:36:16,504][105620] Updated weights for policy 1, policy_version 1404383 (0.0009) [2023-12-27 01:36:16,550][105620] Updated weights for policy 1, policy_version 1404393 (0.0008) [2023-12-27 01:36:16,600][105620] Updated weights for policy 1, policy_version 1404404 (0.0009) [2023-12-27 01:36:16,921][105692] Updated weights for policy 0, policy_version 1402196 (0.0007) [2023-12-27 01:36:16,971][105692] Updated weights for policy 0, policy_version 1402206 (0.0009) [2023-12-27 01:36:17,019][105692] Updated weights for policy 0, policy_version 1402216 (0.0009) [2023-12-27 01:36:17,319][105620] Updated weights for policy 1, policy_version 1404414 (0.0007) [2023-12-27 01:36:17,365][105620] Updated weights for policy 1, policy_version 1404424 (0.0007) [2023-12-27 01:36:17,419][105620] Updated weights for policy 1, policy_version 1404434 (0.0009) [2023-12-27 01:36:17,829][105692] Updated weights for policy 0, policy_version 1402226 (0.0009) [2023-12-27 01:36:17,887][105692] Updated weights for policy 0, policy_version 1402236 (0.0009) [2023-12-27 01:36:17,935][105692] Updated weights for policy 0, policy_version 1402246 (0.0008) [2023-12-27 01:36:17,982][105692] Updated weights for policy 0, policy_version 1402256 (0.0009) [2023-12-27 01:36:18,123][105620] Updated weights for policy 1, policy_version 1404444 (0.0007) [2023-12-27 01:36:18,170][105620] Updated weights for policy 1, policy_version 1404454 (0.0008) [2023-12-27 01:36:18,224][105620] Updated weights for policy 1, policy_version 1404464 (0.0010) [2023-12-27 01:36:18,780][105692] Updated weights for policy 0, policy_version 1402266 (0.0008) [2023-12-27 01:36:18,833][105692] Updated weights for policy 0, policy_version 1402276 (0.0008) [2023-12-27 01:36:18,886][105692] Updated weights for policy 0, policy_version 1402286 (0.0008) [2023-12-27 01:36:18,968][105620] Updated weights for policy 1, policy_version 1404474 (0.0010) [2023-12-27 01:36:19,016][105620] Updated weights for policy 1, policy_version 1404484 (0.0010) [2023-12-27 01:36:19,064][105620] Updated weights for policy 1, policy_version 1404494 (0.0010) [2023-12-27 01:36:19,119][105620] Updated weights for policy 1, policy_version 1404504 (0.0010) [2023-12-27 01:36:19,689][105692] Updated weights for policy 0, policy_version 1402296 (0.0009) [2023-12-27 01:36:19,753][105692] Updated weights for policy 0, policy_version 1402306 (0.0008) [2023-12-27 01:36:19,818][105692] Updated weights for policy 0, policy_version 1402316 (0.0008) [2023-12-27 01:36:19,917][105620] Updated weights for policy 1, policy_version 1404514 (0.0011) [2023-12-27 01:36:19,973][105620] Updated weights for policy 1, policy_version 1404524 (0.0010) [2023-12-27 01:36:20,030][105620] Updated weights for policy 1, policy_version 1404534 (0.0011) [2023-12-27 01:36:20,621][105692] Updated weights for policy 0, policy_version 1402326 (0.0008) [2023-12-27 01:36:20,683][105692] Updated weights for policy 0, policy_version 1402336 (0.0008) [2023-12-27 01:36:20,745][105692] Updated weights for policy 0, policy_version 1402346 (0.0008) [2023-12-27 01:36:20,826][105620] Updated weights for policy 1, policy_version 1404544 (0.0011) [2023-12-27 01:36:20,890][105620] Updated weights for policy 1, policy_version 1404554 (0.0010) [2023-12-27 01:36:20,950][105620] Updated weights for policy 1, policy_version 1404564 (0.0011) [2023-12-27 01:36:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 718667776. Throughput: 0: 9635.6, 1: 9785.1. Samples: 718655768. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:36:21,062][104569] Avg episode reward: [(0, '8625.016'), (1, '6382.734')] [2023-12-27 01:36:21,558][105692] Updated weights for policy 0, policy_version 1402356 (0.0008) [2023-12-27 01:36:21,628][105692] Updated weights for policy 0, policy_version 1402366 (0.0008) [2023-12-27 01:36:21,689][105692] Updated weights for policy 0, policy_version 1402376 (0.0008) [2023-12-27 01:36:21,725][105620] Updated weights for policy 1, policy_version 1404574 (0.0011) [2023-12-27 01:36:21,790][105620] Updated weights for policy 1, policy_version 1404584 (0.0009) [2023-12-27 01:36:21,849][105620] Updated weights for policy 1, policy_version 1404594 (0.0009) [2023-12-27 01:36:22,500][105692] Updated weights for policy 0, policy_version 1402386 (0.0007) [2023-12-27 01:36:22,554][105692] Updated weights for policy 0, policy_version 1402396 (0.0008) [2023-12-27 01:36:22,595][105620] Updated weights for policy 1, policy_version 1404604 (0.0010) [2023-12-27 01:36:22,614][105692] Updated weights for policy 0, policy_version 1402406 (0.0008) [2023-12-27 01:36:22,655][105620] Updated weights for policy 1, policy_version 1404614 (0.0011) [2023-12-27 01:36:22,670][105692] Updated weights for policy 0, policy_version 1402416 (0.0007) [2023-12-27 01:36:22,721][105620] Updated weights for policy 1, policy_version 1404624 (0.0011) [2023-12-27 01:36:23,446][105692] Updated weights for policy 0, policy_version 1402426 (0.0008) [2023-12-27 01:36:23,475][105620] Updated weights for policy 1, policy_version 1404634 (0.0010) [2023-12-27 01:36:23,493][105692] Updated weights for policy 0, policy_version 1402436 (0.0008) [2023-12-27 01:36:23,530][105620] Updated weights for policy 1, policy_version 1404644 (0.0010) [2023-12-27 01:36:23,537][105692] Updated weights for policy 0, policy_version 1402446 (0.0007) [2023-12-27 01:36:23,576][105620] Updated weights for policy 1, policy_version 1404654 (0.0010) [2023-12-27 01:36:23,623][105620] Updated weights for policy 1, policy_version 1404664 (0.0009) [2023-12-27 01:36:24,316][105692] Updated weights for policy 0, policy_version 1402456 (0.0009) [2023-12-27 01:36:24,364][105692] Updated weights for policy 0, policy_version 1402466 (0.0007) [2023-12-27 01:36:24,380][105620] Updated weights for policy 1, policy_version 1404674 (0.0008) [2023-12-27 01:36:24,411][105692] Updated weights for policy 0, policy_version 1402476 (0.0009) [2023-12-27 01:36:24,433][105620] Updated weights for policy 1, policy_version 1404684 (0.0008) [2023-12-27 01:36:24,496][105620] Updated weights for policy 1, policy_version 1404694 (0.0009) [2023-12-27 01:36:25,159][105692] Updated weights for policy 0, policy_version 1402486 (0.0008) [2023-12-27 01:36:25,216][105692] Updated weights for policy 0, policy_version 1402496 (0.0008) [2023-12-27 01:36:25,224][105620] Updated weights for policy 1, policy_version 1404704 (0.0007) [2023-12-27 01:36:25,280][105692] Updated weights for policy 0, policy_version 1402506 (0.0006) [2023-12-27 01:36:25,281][105620] Updated weights for policy 1, policy_version 1404714 (0.0007) [2023-12-27 01:36:25,336][105620] Updated weights for policy 1, policy_version 1404724 (0.0007) [2023-12-27 01:36:26,037][105692] Updated weights for policy 0, policy_version 1402516 (0.0009) [2023-12-27 01:36:26,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19251.2, 300 sec: 19327.6). Total num frames: 718749696. Throughput: 0: 9545.1, 1: 9716.4. Samples: 718765456. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:36:26,063][104569] Avg episode reward: [(0, '8349.209'), (1, '8616.490')] [2023-12-27 01:36:26,070][105620] Updated weights for policy 1, policy_version 1404734 (0.0008) [2023-12-27 01:36:26,103][105692] Updated weights for policy 0, policy_version 1402526 (0.0008) [2023-12-27 01:36:26,125][105620] Updated weights for policy 1, policy_version 1404744 (0.0007) [2023-12-27 01:36:26,166][105692] Updated weights for policy 0, policy_version 1402536 (0.0008) [2023-12-27 01:36:26,182][105620] Updated weights for policy 1, policy_version 1404754 (0.0006) [2023-12-27 01:36:26,874][105620] Updated weights for policy 1, policy_version 1404764 (0.0007) [2023-12-27 01:36:26,925][105620] Updated weights for policy 1, policy_version 1404774 (0.0009) [2023-12-27 01:36:26,937][105692] Updated weights for policy 0, policy_version 1402546 (0.0007) [2023-12-27 01:36:26,969][105620] Updated weights for policy 1, policy_version 1404784 (0.0007) [2023-12-27 01:36:26,994][105692] Updated weights for policy 0, policy_version 1402556 (0.0009) [2023-12-27 01:36:27,054][105692] Updated weights for policy 0, policy_version 1402566 (0.0009) [2023-12-27 01:36:27,116][105692] Updated weights for policy 0, policy_version 1402576 (0.0009) [2023-12-27 01:36:27,700][105620] Updated weights for policy 1, policy_version 1404794 (0.0006) [2023-12-27 01:36:27,761][105620] Updated weights for policy 1, policy_version 1404804 (0.0009) [2023-12-27 01:36:27,822][105620] Updated weights for policy 1, policy_version 1404814 (0.0006) [2023-12-27 01:36:27,874][105692] Updated weights for policy 0, policy_version 1402586 (0.0008) [2023-12-27 01:36:27,937][105692] Updated weights for policy 0, policy_version 1402596 (0.0010) [2023-12-27 01:36:28,006][105692] Updated weights for policy 0, policy_version 1402606 (0.0010) [2023-12-27 01:36:28,443][105620] Updated weights for policy 1, policy_version 1404825 (0.0007) [2023-12-27 01:36:28,490][105620] Updated weights for policy 1, policy_version 1404835 (0.0009) [2023-12-27 01:36:28,542][105620] Updated weights for policy 1, policy_version 1404845 (0.0008) [2023-12-27 01:36:28,610][105620] Updated weights for policy 1, policy_version 1404855 (0.0005) [2023-12-27 01:36:28,761][105692] Updated weights for policy 0, policy_version 1402616 (0.0009) [2023-12-27 01:36:28,815][105692] Updated weights for policy 0, policy_version 1402626 (0.0008) [2023-12-27 01:36:28,881][105692] Updated weights for policy 0, policy_version 1402636 (0.0009) [2023-12-27 01:36:29,179][105620] Updated weights for policy 1, policy_version 1404865 (0.0006) [2023-12-27 01:36:29,240][105620] Updated weights for policy 1, policy_version 1404875 (0.0009) [2023-12-27 01:36:29,299][105620] Updated weights for policy 1, policy_version 1404885 (0.0009) [2023-12-27 01:36:29,699][105692] Updated weights for policy 0, policy_version 1402646 (0.0009) [2023-12-27 01:36:29,746][105692] Updated weights for policy 0, policy_version 1402656 (0.0009) [2023-12-27 01:36:29,801][105692] Updated weights for policy 0, policy_version 1402666 (0.0008) [2023-12-27 01:36:30,026][105620] Updated weights for policy 1, policy_version 1404895 (0.0009) [2023-12-27 01:36:30,078][105620] Updated weights for policy 1, policy_version 1404905 (0.0009) [2023-12-27 01:36:30,139][105620] Updated weights for policy 1, policy_version 1404915 (0.0010) [2023-12-27 01:36:30,448][105692] Updated weights for policy 0, policy_version 1402676 (0.0007) [2023-12-27 01:36:30,501][105692] Updated weights for policy 0, policy_version 1402686 (0.0007) [2023-12-27 01:36:30,551][105692] Updated weights for policy 0, policy_version 1402696 (0.0006) [2023-12-27 01:36:30,774][105620] Updated weights for policy 1, policy_version 1404925 (0.0010) [2023-12-27 01:36:30,820][105620] Updated weights for policy 1, policy_version 1404935 (0.0009) [2023-12-27 01:36:30,866][105620] Updated weights for policy 1, policy_version 1404945 (0.0008) [2023-12-27 01:36:31,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.2, 300 sec: 19355.3). Total num frames: 718856192. Throughput: 0: 9550.5, 1: 9746.4. Samples: 718824216. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 01:36:31,063][104569] Avg episode reward: [(0, '7801.256'), (1, '8901.645')] [2023-12-27 01:36:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001402704_359145472.pth... [2023-12-27 01:36:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001404952_359710720.pth... [2023-12-27 01:36:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001401616_358866944.pth [2023-12-27 01:36:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001403800_359415808.pth [2023-12-27 01:36:31,228][105692] Updated weights for policy 0, policy_version 1402706 (0.0008) [2023-12-27 01:36:31,285][105692] Updated weights for policy 0, policy_version 1402716 (0.0005) [2023-12-27 01:36:31,347][105692] Updated weights for policy 0, policy_version 1402726 (0.0008) [2023-12-27 01:36:31,409][105692] Updated weights for policy 0, policy_version 1402736 (0.0009) [2023-12-27 01:36:31,684][105620] Updated weights for policy 1, policy_version 1404955 (0.0009) [2023-12-27 01:36:31,752][105620] Updated weights for policy 1, policy_version 1404965 (0.0008) [2023-12-27 01:36:31,814][105620] Updated weights for policy 1, policy_version 1404975 (0.0008) [2023-12-27 01:36:32,185][105692] Updated weights for policy 0, policy_version 1402746 (0.0009) [2023-12-27 01:36:32,235][105692] Updated weights for policy 0, policy_version 1402756 (0.0009) [2023-12-27 01:36:32,295][105692] Updated weights for policy 0, policy_version 1402766 (0.0008) [2023-12-27 01:36:32,470][105620] Updated weights for policy 1, policy_version 1404985 (0.0008) [2023-12-27 01:36:32,525][105620] Updated weights for policy 1, policy_version 1404995 (0.0009) [2023-12-27 01:36:32,587][105620] Updated weights for policy 1, policy_version 1405005 (0.0010) [2023-12-27 01:36:32,639][105620] Updated weights for policy 1, policy_version 1405015 (0.0010) [2023-12-27 01:36:33,080][105692] Updated weights for policy 0, policy_version 1402776 (0.0008) [2023-12-27 01:36:33,131][105692] Updated weights for policy 0, policy_version 1402786 (0.0008) [2023-12-27 01:36:33,183][105692] Updated weights for policy 0, policy_version 1402796 (0.0008) [2023-12-27 01:36:33,381][105620] Updated weights for policy 1, policy_version 1405025 (0.0006) [2023-12-27 01:36:33,431][105620] Updated weights for policy 1, policy_version 1405035 (0.0005) [2023-12-27 01:36:33,491][105620] Updated weights for policy 1, policy_version 1405045 (0.0006) [2023-12-27 01:36:33,994][105620] Updated weights for policy 1, policy_version 1405055 (0.0005) [2023-12-27 01:36:34,052][105620] Updated weights for policy 1, policy_version 1405065 (0.0008) [2023-12-27 01:36:34,057][105692] Updated weights for policy 0, policy_version 1402806 (0.0007) [2023-12-27 01:36:34,103][105620] Updated weights for policy 1, policy_version 1405075 (0.0010) [2023-12-27 01:36:34,105][105692] Updated weights for policy 0, policy_version 1402816 (0.0005) [2023-12-27 01:36:34,159][105692] Updated weights for policy 0, policy_version 1402826 (0.0007) [2023-12-27 01:36:34,866][105620] Updated weights for policy 1, policy_version 1405085 (0.0010) [2023-12-27 01:36:34,892][105692] Updated weights for policy 0, policy_version 1402836 (0.0007) [2023-12-27 01:36:34,930][105620] Updated weights for policy 1, policy_version 1405095 (0.0011) [2023-12-27 01:36:34,944][105692] Updated weights for policy 0, policy_version 1402846 (0.0007) [2023-12-27 01:36:34,990][105692] Updated weights for policy 0, policy_version 1402856 (0.0008) [2023-12-27 01:36:34,995][105620] Updated weights for policy 1, policy_version 1405105 (0.0010) [2023-12-27 01:36:35,691][105620] Updated weights for policy 1, policy_version 1405115 (0.0011) [2023-12-27 01:36:35,735][105620] Updated weights for policy 1, policy_version 1405125 (0.0010) [2023-12-27 01:36:35,766][105692] Updated weights for policy 0, policy_version 1402866 (0.0006) [2023-12-27 01:36:35,784][105620] Updated weights for policy 1, policy_version 1405135 (0.0010) [2023-12-27 01:36:35,824][105692] Updated weights for policy 0, policy_version 1402876 (0.0005) [2023-12-27 01:36:35,881][105692] Updated weights for policy 0, policy_version 1402886 (0.0007) [2023-12-27 01:36:35,940][105692] Updated weights for policy 0, policy_version 1402896 (0.0008) [2023-12-27 01:36:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 718954496. Throughput: 0: 9488.7, 1: 9778.7. Samples: 718940516. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:36:36,062][104569] Avg episode reward: [(0, '7982.005'), (1, '8812.836')] [2023-12-27 01:36:36,574][105620] Updated weights for policy 1, policy_version 1405145 (0.0011) [2023-12-27 01:36:36,640][105620] Updated weights for policy 1, policy_version 1405155 (0.0011) [2023-12-27 01:36:36,703][105620] Updated weights for policy 1, policy_version 1405165 (0.0011) [2023-12-27 01:36:36,729][105692] Updated weights for policy 0, policy_version 1402906 (0.0009) [2023-12-27 01:36:36,752][105620] Updated weights for policy 1, policy_version 1405175 (0.0010) [2023-12-27 01:36:36,783][105692] Updated weights for policy 0, policy_version 1402916 (0.0007) [2023-12-27 01:36:36,836][105692] Updated weights for policy 0, policy_version 1402926 (0.0008) [2023-12-27 01:36:37,391][105620] Updated weights for policy 1, policy_version 1405185 (0.0010) [2023-12-27 01:36:37,436][105620] Updated weights for policy 1, policy_version 1405195 (0.0010) [2023-12-27 01:36:37,492][105620] Updated weights for policy 1, policy_version 1405205 (0.0011) [2023-12-27 01:36:37,670][105692] Updated weights for policy 0, policy_version 1402936 (0.0008) [2023-12-27 01:36:37,727][105692] Updated weights for policy 0, policy_version 1402946 (0.0008) [2023-12-27 01:36:37,785][105692] Updated weights for policy 0, policy_version 1402956 (0.0010) [2023-12-27 01:36:38,155][105620] Updated weights for policy 1, policy_version 1405215 (0.0010) [2023-12-27 01:36:38,207][105620] Updated weights for policy 1, policy_version 1405225 (0.0011) [2023-12-27 01:36:38,255][105620] Updated weights for policy 1, policy_version 1405235 (0.0010) [2023-12-27 01:36:38,546][105692] Updated weights for policy 0, policy_version 1402967 (0.0007) [2023-12-27 01:36:38,599][105692] Updated weights for policy 0, policy_version 1402977 (0.0005) [2023-12-27 01:36:38,649][105692] Updated weights for policy 0, policy_version 1402987 (0.0010) [2023-12-27 01:36:38,958][105620] Updated weights for policy 1, policy_version 1405245 (0.0010) [2023-12-27 01:36:39,014][105620] Updated weights for policy 1, policy_version 1405255 (0.0011) [2023-12-27 01:36:39,081][105620] Updated weights for policy 1, policy_version 1405265 (0.0011) [2023-12-27 01:36:39,416][105692] Updated weights for policy 0, policy_version 1402997 (0.0010) [2023-12-27 01:36:39,475][105692] Updated weights for policy 0, policy_version 1403007 (0.0009) [2023-12-27 01:36:39,531][105692] Updated weights for policy 0, policy_version 1403017 (0.0010) [2023-12-27 01:36:39,845][105620] Updated weights for policy 1, policy_version 1405275 (0.0009) [2023-12-27 01:36:39,907][105620] Updated weights for policy 1, policy_version 1405285 (0.0008) [2023-12-27 01:36:39,964][105620] Updated weights for policy 1, policy_version 1405295 (0.0009) [2023-12-27 01:36:40,315][105692] Updated weights for policy 0, policy_version 1403027 (0.0010) [2023-12-27 01:36:40,378][105692] Updated weights for policy 0, policy_version 1403037 (0.0010) [2023-12-27 01:36:40,444][105692] Updated weights for policy 0, policy_version 1403047 (0.0008) [2023-12-27 01:36:40,598][105620] Updated weights for policy 1, policy_version 1405305 (0.0006) [2023-12-27 01:36:40,660][105620] Updated weights for policy 1, policy_version 1405315 (0.0011) [2023-12-27 01:36:40,722][105620] Updated weights for policy 1, policy_version 1405325 (0.0010) [2023-12-27 01:36:40,773][105620] Updated weights for policy 1, policy_version 1405335 (0.0008) [2023-12-27 01:36:41,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.3, 300 sec: 19355.3). Total num frames: 719044608. Throughput: 0: 9396.0, 1: 9760.3. Samples: 719054044. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:36:41,062][104569] Avg episode reward: [(0, '8165.135'), (1, '8902.517')] [2023-12-27 01:36:41,124][105692] Updated weights for policy 0, policy_version 1403057 (0.0009) [2023-12-27 01:36:41,210][105692] Updated weights for policy 0, policy_version 1403067 (0.0009) [2023-12-27 01:36:41,275][105692] Updated weights for policy 0, policy_version 1403077 (0.0007) [2023-12-27 01:36:41,335][105692] Updated weights for policy 0, policy_version 1403087 (0.0012) [2023-12-27 01:36:41,557][105620] Updated weights for policy 1, policy_version 1405345 (0.0009) [2023-12-27 01:36:41,621][105620] Updated weights for policy 1, policy_version 1405355 (0.0009) [2023-12-27 01:36:41,680][105620] Updated weights for policy 1, policy_version 1405365 (0.0009) [2023-12-27 01:36:42,062][105692] Updated weights for policy 0, policy_version 1403097 (0.0006) [2023-12-27 01:36:42,127][105692] Updated weights for policy 0, policy_version 1403107 (0.0006) [2023-12-27 01:36:42,190][105692] Updated weights for policy 0, policy_version 1403117 (0.0010) [2023-12-27 01:36:42,494][105620] Updated weights for policy 1, policy_version 1405375 (0.0008) [2023-12-27 01:36:42,559][105620] Updated weights for policy 1, policy_version 1405385 (0.0009) [2023-12-27 01:36:42,621][105620] Updated weights for policy 1, policy_version 1405395 (0.0009) [2023-12-27 01:36:42,906][105692] Updated weights for policy 0, policy_version 1403127 (0.0010) [2023-12-27 01:36:42,969][105692] Updated weights for policy 0, policy_version 1403137 (0.0010) [2023-12-27 01:36:43,032][105692] Updated weights for policy 0, policy_version 1403147 (0.0011) [2023-12-27 01:36:43,354][105620] Updated weights for policy 1, policy_version 1405405 (0.0010) [2023-12-27 01:36:43,415][105620] Updated weights for policy 1, policy_version 1405416 (0.0009) [2023-12-27 01:36:43,461][105620] Updated weights for policy 1, policy_version 1405426 (0.0010) [2023-12-27 01:36:43,735][105692] Updated weights for policy 0, policy_version 1403157 (0.0009) [2023-12-27 01:36:43,792][105692] Updated weights for policy 0, policy_version 1403167 (0.0010) [2023-12-27 01:36:43,856][105692] Updated weights for policy 0, policy_version 1403177 (0.0010) [2023-12-27 01:36:44,253][105620] Updated weights for policy 1, policy_version 1405436 (0.0008) [2023-12-27 01:36:44,300][105620] Updated weights for policy 1, policy_version 1405446 (0.0008) [2023-12-27 01:36:44,347][105620] Updated weights for policy 1, policy_version 1405456 (0.0007) [2023-12-27 01:36:44,558][105692] Updated weights for policy 0, policy_version 1403187 (0.0010) [2023-12-27 01:36:44,622][105692] Updated weights for policy 0, policy_version 1403197 (0.0010) [2023-12-27 01:36:44,635][105585] KL-divergence is very high: 115.7395 [2023-12-27 01:36:44,670][105692] Updated weights for policy 0, policy_version 1403207 (0.0010) [2023-12-27 01:36:44,675][105585] KL-divergence is very high: 127.8600 [2023-12-27 01:36:45,147][105620] Updated weights for policy 1, policy_version 1405466 (0.0008) [2023-12-27 01:36:45,200][105620] Updated weights for policy 1, policy_version 1405476 (0.0008) [2023-12-27 01:36:45,261][105620] Updated weights for policy 1, policy_version 1405486 (0.0008) [2023-12-27 01:36:45,305][105620] Updated weights for policy 1, policy_version 1405496 (0.0008) [2023-12-27 01:36:45,400][105692] Updated weights for policy 0, policy_version 1403217 (0.0010) [2023-12-27 01:36:45,463][105692] Updated weights for policy 0, policy_version 1403227 (0.0011) [2023-12-27 01:36:45,522][105692] Updated weights for policy 0, policy_version 1403237 (0.0011) [2023-12-27 01:36:45,585][105692] Updated weights for policy 0, policy_version 1403247 (0.0011) [2023-12-27 01:36:46,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19114.8, 300 sec: 19355.3). Total num frames: 719134720. Throughput: 0: 9328.4, 1: 9710.9. Samples: 719110172. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:36:46,062][104569] Avg episode reward: [(0, '8350.374'), (1, '8812.253')] [2023-12-27 01:36:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001403248_359284736.pth... [2023-12-27 01:36:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001402192_359014400.pth [2023-12-27 01:36:46,086][105620] Updated weights for policy 1, policy_version 1405506 (0.0008) [2023-12-27 01:36:46,137][105620] Updated weights for policy 1, policy_version 1405516 (0.0008) [2023-12-27 01:36:46,189][105620] Updated weights for policy 1, policy_version 1405526 (0.0008) [2023-12-27 01:36:46,196][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001405528_359858176.pth... [2023-12-27 01:36:46,199][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001404376_359563264.pth [2023-12-27 01:36:46,325][105692] Updated weights for policy 0, policy_version 1403257 (0.0006) [2023-12-27 01:36:46,386][105692] Updated weights for policy 0, policy_version 1403267 (0.0005) [2023-12-27 01:36:46,434][105692] Updated weights for policy 0, policy_version 1403277 (0.0007) [2023-12-27 01:36:47,005][105620] Updated weights for policy 1, policy_version 1405536 (0.0009) [2023-12-27 01:36:47,057][105620] Updated weights for policy 1, policy_version 1405546 (0.0009) [2023-12-27 01:36:47,069][105692] Updated weights for policy 0, policy_version 1403287 (0.0010) [2023-12-27 01:36:47,108][105620] Updated weights for policy 1, policy_version 1405556 (0.0006) [2023-12-27 01:36:47,124][105692] Updated weights for policy 0, policy_version 1403297 (0.0007) [2023-12-27 01:36:47,178][105692] Updated weights for policy 0, policy_version 1403307 (0.0007) [2023-12-27 01:36:47,760][105692] Updated weights for policy 0, policy_version 1403317 (0.0008) [2023-12-27 01:36:47,811][105692] Updated weights for policy 0, policy_version 1403327 (0.0006) [2023-12-27 01:36:47,857][105692] Updated weights for policy 0, policy_version 1403337 (0.0005) [2023-12-27 01:36:47,958][105620] Updated weights for policy 1, policy_version 1405566 (0.0008) [2023-12-27 01:36:48,004][105620] Updated weights for policy 1, policy_version 1405576 (0.0008) [2023-12-27 01:36:48,055][105620] Updated weights for policy 1, policy_version 1405586 (0.0008) [2023-12-27 01:36:48,518][105692] Updated weights for policy 0, policy_version 1403347 (0.0007) [2023-12-27 01:36:48,577][105692] Updated weights for policy 0, policy_version 1403357 (0.0011) [2023-12-27 01:36:48,643][105692] Updated weights for policy 0, policy_version 1403367 (0.0010) [2023-12-27 01:36:48,867][105620] Updated weights for policy 1, policy_version 1405596 (0.0008) [2023-12-27 01:36:48,929][105620] Updated weights for policy 1, policy_version 1405606 (0.0009) [2023-12-27 01:36:48,988][105620] Updated weights for policy 1, policy_version 1405616 (0.0008) [2023-12-27 01:36:49,439][105692] Updated weights for policy 0, policy_version 1403377 (0.0010) [2023-12-27 01:36:49,491][105692] Updated weights for policy 0, policy_version 1403387 (0.0010) [2023-12-27 01:36:49,542][105692] Updated weights for policy 0, policy_version 1403397 (0.0010) [2023-12-27 01:36:49,596][105692] Updated weights for policy 0, policy_version 1403407 (0.0010) [2023-12-27 01:36:49,821][105620] Updated weights for policy 1, policy_version 1405626 (0.0009) [2023-12-27 01:36:49,887][105620] Updated weights for policy 1, policy_version 1405636 (0.0008) [2023-12-27 01:36:49,950][105620] Updated weights for policy 1, policy_version 1405646 (0.0008) [2023-12-27 01:36:50,012][105620] Updated weights for policy 1, policy_version 1405656 (0.0008) [2023-12-27 01:36:50,399][105692] Updated weights for policy 0, policy_version 1403417 (0.0011) [2023-12-27 01:36:50,457][105692] Updated weights for policy 0, policy_version 1403427 (0.0009) [2023-12-27 01:36:50,505][105692] Updated weights for policy 0, policy_version 1403437 (0.0006) [2023-12-27 01:36:50,813][105620] Updated weights for policy 1, policy_version 1405666 (0.0008) [2023-12-27 01:36:50,870][105620] Updated weights for policy 1, policy_version 1405676 (0.0009) [2023-12-27 01:36:50,937][105620] Updated weights for policy 1, policy_version 1405686 (0.0009) [2023-12-27 01:36:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.7, 300 sec: 19355.3). Total num frames: 719233024. Throughput: 0: 9286.6, 1: 9659.8. Samples: 719223012. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:36:51,063][104569] Avg episode reward: [(0, '8257.253'), (1, '8645.815')] [2023-12-27 01:36:51,195][105692] Updated weights for policy 0, policy_version 1403447 (0.0005) [2023-12-27 01:36:51,258][105692] Updated weights for policy 0, policy_version 1403457 (0.0008) [2023-12-27 01:36:51,315][105692] Updated weights for policy 0, policy_version 1403467 (0.0011) [2023-12-27 01:36:51,658][105620] Updated weights for policy 1, policy_version 1405696 (0.0008) [2023-12-27 01:36:51,726][105620] Updated weights for policy 1, policy_version 1405706 (0.0008) [2023-12-27 01:36:51,793][105620] Updated weights for policy 1, policy_version 1405716 (0.0010) [2023-12-27 01:36:52,024][105692] Updated weights for policy 0, policy_version 1403477 (0.0011) [2023-12-27 01:36:52,088][105692] Updated weights for policy 0, policy_version 1403487 (0.0011) [2023-12-27 01:36:52,144][105692] Updated weights for policy 0, policy_version 1403497 (0.0011) [2023-12-27 01:36:52,522][105620] Updated weights for policy 1, policy_version 1405726 (0.0008) [2023-12-27 01:36:52,570][105620] Updated weights for policy 1, policy_version 1405736 (0.0008) [2023-12-27 01:36:52,615][105620] Updated weights for policy 1, policy_version 1405746 (0.0008) [2023-12-27 01:36:52,871][105692] Updated weights for policy 0, policy_version 1403507 (0.0011) [2023-12-27 01:36:52,920][105692] Updated weights for policy 0, policy_version 1403517 (0.0011) [2023-12-27 01:36:52,980][105692] Updated weights for policy 0, policy_version 1403527 (0.0010) [2023-12-27 01:36:53,333][105620] Updated weights for policy 1, policy_version 1405756 (0.0007) [2023-12-27 01:36:53,400][105620] Updated weights for policy 1, policy_version 1405766 (0.0005) [2023-12-27 01:36:53,466][105620] Updated weights for policy 1, policy_version 1405776 (0.0005) [2023-12-27 01:36:53,666][105692] Updated weights for policy 0, policy_version 1403537 (0.0010) [2023-12-27 01:36:53,713][105692] Updated weights for policy 0, policy_version 1403547 (0.0005) [2023-12-27 01:36:53,763][105692] Updated weights for policy 0, policy_version 1403557 (0.0005) [2023-12-27 01:36:53,815][105692] Updated weights for policy 0, policy_version 1403567 (0.0009) [2023-12-27 01:36:53,964][105620] Updated weights for policy 1, policy_version 1405786 (0.0006) [2023-12-27 01:36:54,030][105620] Updated weights for policy 1, policy_version 1405796 (0.0006) [2023-12-27 01:36:54,088][105620] Updated weights for policy 1, policy_version 1405806 (0.0006) [2023-12-27 01:36:54,148][105620] Updated weights for policy 1, policy_version 1405816 (0.0006) [2023-12-27 01:36:54,617][105692] Updated weights for policy 0, policy_version 1403577 (0.0009) [2023-12-27 01:36:54,665][105692] Updated weights for policy 0, policy_version 1403587 (0.0008) [2023-12-27 01:36:54,724][105692] Updated weights for policy 0, policy_version 1403597 (0.0008) [2023-12-27 01:36:54,774][105620] Updated weights for policy 1, policy_version 1405826 (0.0006) [2023-12-27 01:36:54,823][105620] Updated weights for policy 1, policy_version 1405836 (0.0007) [2023-12-27 01:36:54,878][105620] Updated weights for policy 1, policy_version 1405848 (0.0010) [2023-12-27 01:36:55,393][105692] Updated weights for policy 0, policy_version 1403607 (0.0006) [2023-12-27 01:36:55,463][105692] Updated weights for policy 0, policy_version 1403617 (0.0006) [2023-12-27 01:36:55,533][105692] Updated weights for policy 0, policy_version 1403627 (0.0005) [2023-12-27 01:36:55,634][105620] Updated weights for policy 1, policy_version 1405858 (0.0006) [2023-12-27 01:36:55,684][105620] Updated weights for policy 1, policy_version 1405868 (0.0008) [2023-12-27 01:36:55,737][105620] Updated weights for policy 1, policy_version 1405879 (0.0009) [2023-12-27 01:36:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19114.7, 300 sec: 19355.3). Total num frames: 719331328. Throughput: 0: 9317.1, 1: 9773.5. Samples: 719341776. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:36:56,063][104569] Avg episode reward: [(0, '8443.258'), (1, '8832.603')] [2023-12-27 01:36:56,077][105692] Updated weights for policy 0, policy_version 1403637 (0.0008) [2023-12-27 01:36:56,131][105692] Updated weights for policy 0, policy_version 1403647 (0.0008) [2023-12-27 01:36:56,195][105692] Updated weights for policy 0, policy_version 1403657 (0.0007) [2023-12-27 01:36:56,454][105620] Updated weights for policy 1, policy_version 1405889 (0.0006) [2023-12-27 01:36:56,500][105620] Updated weights for policy 1, policy_version 1405899 (0.0005) [2023-12-27 01:36:56,552][105620] Updated weights for policy 1, policy_version 1405909 (0.0005) [2023-12-27 01:36:56,940][105692] Updated weights for policy 0, policy_version 1403667 (0.0007) [2023-12-27 01:36:56,988][105692] Updated weights for policy 0, policy_version 1403677 (0.0008) [2023-12-27 01:36:57,039][105692] Updated weights for policy 0, policy_version 1403687 (0.0008) [2023-12-27 01:36:57,250][105620] Updated weights for policy 1, policy_version 1405919 (0.0009) [2023-12-27 01:36:57,309][105620] Updated weights for policy 1, policy_version 1405930 (0.0010) [2023-12-27 01:36:57,375][105620] Updated weights for policy 1, policy_version 1405940 (0.0009) [2023-12-27 01:36:57,750][105692] Updated weights for policy 0, policy_version 1403697 (0.0007) [2023-12-27 01:36:57,808][105692] Updated weights for policy 0, policy_version 1403707 (0.0005) [2023-12-27 01:36:57,858][105692] Updated weights for policy 0, policy_version 1403717 (0.0005) [2023-12-27 01:36:57,917][105692] Updated weights for policy 0, policy_version 1403727 (0.0005) [2023-12-27 01:36:58,060][105620] Updated weights for policy 1, policy_version 1405950 (0.0010) [2023-12-27 01:36:58,115][105620] Updated weights for policy 1, policy_version 1405960 (0.0010) [2023-12-27 01:36:58,183][105620] Updated weights for policy 1, policy_version 1405970 (0.0010) [2023-12-27 01:36:58,550][105692] Updated weights for policy 0, policy_version 1403737 (0.0008) [2023-12-27 01:36:58,617][105692] Updated weights for policy 0, policy_version 1403747 (0.0007) [2023-12-27 01:36:58,683][105692] Updated weights for policy 0, policy_version 1403757 (0.0008) [2023-12-27 01:36:58,966][105620] Updated weights for policy 1, policy_version 1405980 (0.0008) [2023-12-27 01:36:59,026][105620] Updated weights for policy 1, policy_version 1405990 (0.0006) [2023-12-27 01:36:59,085][105620] Updated weights for policy 1, policy_version 1406000 (0.0009) [2023-12-27 01:36:59,446][105692] Updated weights for policy 0, policy_version 1403767 (0.0008) [2023-12-27 01:36:59,512][105692] Updated weights for policy 0, policy_version 1403777 (0.0008) [2023-12-27 01:36:59,578][105692] Updated weights for policy 0, policy_version 1403787 (0.0008) [2023-12-27 01:36:59,787][105620] Updated weights for policy 1, policy_version 1406010 (0.0009) [2023-12-27 01:36:59,860][105620] Updated weights for policy 1, policy_version 1406020 (0.0008) [2023-12-27 01:36:59,927][105620] Updated weights for policy 1, policy_version 1406030 (0.0008) [2023-12-27 01:36:59,990][105620] Updated weights for policy 1, policy_version 1406040 (0.0009) [2023-12-27 01:37:00,219][105692] Updated weights for policy 0, policy_version 1403797 (0.0009) [2023-12-27 01:37:00,278][105692] Updated weights for policy 0, policy_version 1403807 (0.0009) [2023-12-27 01:37:00,325][105692] Updated weights for policy 0, policy_version 1403817 (0.0009) [2023-12-27 01:37:00,685][105620] Updated weights for policy 1, policy_version 1406050 (0.0008) [2023-12-27 01:37:00,739][105620] Updated weights for policy 1, policy_version 1406060 (0.0009) [2023-12-27 01:37:00,800][105620] Updated weights for policy 1, policy_version 1406070 (0.0007) [2023-12-27 01:37:01,054][105692] Updated weights for policy 0, policy_version 1403827 (0.0008) [2023-12-27 01:37:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19114.7, 300 sec: 19327.6). Total num frames: 719429632. Throughput: 0: 9423.6, 1: 9695.6. Samples: 719401596. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:37:01,062][104569] Avg episode reward: [(0, '8258.720'), (1, '8994.604')] [2023-12-27 01:37:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001406072_359997440.pth... [2023-12-27 01:37:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001404952_359710720.pth [2023-12-27 01:37:01,143][105692] Updated weights for policy 0, policy_version 1403838 (0.0009) [2023-12-27 01:37:01,199][105692] Updated weights for policy 0, policy_version 1403848 (0.0010) [2023-12-27 01:37:01,240][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001403856_359440384.pth... [2023-12-27 01:37:01,244][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001402704_359145472.pth [2023-12-27 01:37:01,389][105620] Updated weights for policy 1, policy_version 1406080 (0.0009) [2023-12-27 01:37:01,448][105620] Updated weights for policy 1, policy_version 1406090 (0.0010) [2023-12-27 01:37:01,506][105620] Updated weights for policy 1, policy_version 1406100 (0.0010) [2023-12-27 01:37:01,973][105692] Updated weights for policy 0, policy_version 1403858 (0.0009) [2023-12-27 01:37:02,023][105692] Updated weights for policy 0, policy_version 1403868 (0.0008) [2023-12-27 01:37:02,079][105692] Updated weights for policy 0, policy_version 1403878 (0.0007) [2023-12-27 01:37:02,135][105692] Updated weights for policy 0, policy_version 1403888 (0.0005) [2023-12-27 01:37:02,138][105620] Updated weights for policy 1, policy_version 1406110 (0.0007) [2023-12-27 01:37:02,191][105620] Updated weights for policy 1, policy_version 1406120 (0.0006) [2023-12-27 01:37:02,246][105620] Updated weights for policy 1, policy_version 1406130 (0.0006) [2023-12-27 01:37:02,827][105620] Updated weights for policy 1, policy_version 1406140 (0.0008) [2023-12-27 01:37:02,843][105692] Updated weights for policy 0, policy_version 1403898 (0.0010) [2023-12-27 01:37:02,872][105620] Updated weights for policy 1, policy_version 1406150 (0.0010) [2023-12-27 01:37:02,906][105692] Updated weights for policy 0, policy_version 1403908 (0.0006) [2023-12-27 01:37:02,931][105620] Updated weights for policy 1, policy_version 1406160 (0.0010) [2023-12-27 01:37:02,964][105692] Updated weights for policy 0, policy_version 1403918 (0.0005) [2023-12-27 01:37:03,606][105692] Updated weights for policy 0, policy_version 1403928 (0.0009) [2023-12-27 01:37:03,647][105620] Updated weights for policy 1, policy_version 1406170 (0.0010) [2023-12-27 01:37:03,662][105692] Updated weights for policy 0, policy_version 1403938 (0.0009) [2023-12-27 01:37:03,713][105620] Updated weights for policy 1, policy_version 1406180 (0.0005) [2023-12-27 01:37:03,726][105692] Updated weights for policy 0, policy_version 1403948 (0.0010) [2023-12-27 01:37:03,776][105620] Updated weights for policy 1, policy_version 1406190 (0.0006) [2023-12-27 01:37:03,830][105620] Updated weights for policy 1, policy_version 1406200 (0.0006) [2023-12-27 01:37:04,391][105692] Updated weights for policy 0, policy_version 1403958 (0.0007) [2023-12-27 01:37:04,438][105692] Updated weights for policy 0, policy_version 1403968 (0.0009) [2023-12-27 01:37:04,489][105620] Updated weights for policy 1, policy_version 1406210 (0.0006) [2023-12-27 01:37:04,499][105692] Updated weights for policy 0, policy_version 1403978 (0.0008) [2023-12-27 01:37:04,554][105620] Updated weights for policy 1, policy_version 1406220 (0.0007) [2023-12-27 01:37:04,612][105620] Updated weights for policy 1, policy_version 1406230 (0.0009) [2023-12-27 01:37:05,143][105692] Updated weights for policy 0, policy_version 1403988 (0.0008) [2023-12-27 01:37:05,191][105692] Updated weights for policy 0, policy_version 1403999 (0.0005) [2023-12-27 01:37:05,240][105692] Updated weights for policy 0, policy_version 1404009 (0.0005) [2023-12-27 01:37:05,454][105620] Updated weights for policy 1, policy_version 1406240 (0.0008) [2023-12-27 01:37:05,511][105620] Updated weights for policy 1, policy_version 1406250 (0.0009) [2023-12-27 01:37:05,564][105620] Updated weights for policy 1, policy_version 1406261 (0.0010) [2023-12-27 01:37:05,817][105692] Updated weights for policy 0, policy_version 1404019 (0.0007) [2023-12-27 01:37:05,873][105692] Updated weights for policy 0, policy_version 1404029 (0.0010) [2023-12-27 01:37:05,932][105692] Updated weights for policy 0, policy_version 1404039 (0.0010) [2023-12-27 01:37:06,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 719536128. Throughput: 0: 9522.2, 1: 9711.2. Samples: 719521272. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:37:06,062][104569] Avg episode reward: [(0, '7985.873'), (1, '9174.232')] [2023-12-27 01:37:06,348][105620] Updated weights for policy 1, policy_version 1406271 (0.0010) [2023-12-27 01:37:06,407][105620] Updated weights for policy 1, policy_version 1406281 (0.0010) [2023-12-27 01:37:06,457][105620] Updated weights for policy 1, policy_version 1406291 (0.0010) [2023-12-27 01:37:06,618][105692] Updated weights for policy 0, policy_version 1404049 (0.0010) [2023-12-27 01:37:06,678][105692] Updated weights for policy 0, policy_version 1404059 (0.0011) [2023-12-27 01:37:06,744][105692] Updated weights for policy 0, policy_version 1404069 (0.0010) [2023-12-27 01:37:06,807][105692] Updated weights for policy 0, policy_version 1404079 (0.0011) [2023-12-27 01:37:07,224][105620] Updated weights for policy 1, policy_version 1406301 (0.0010) [2023-12-27 01:37:07,285][105620] Updated weights for policy 1, policy_version 1406311 (0.0010) [2023-12-27 01:37:07,340][105620] Updated weights for policy 1, policy_version 1406321 (0.0010) [2023-12-27 01:37:07,478][105692] Updated weights for policy 0, policy_version 1404089 (0.0010) [2023-12-27 01:37:07,538][105692] Updated weights for policy 0, policy_version 1404099 (0.0006) [2023-12-27 01:37:07,591][105692] Updated weights for policy 0, policy_version 1404109 (0.0005) [2023-12-27 01:37:08,059][105620] Updated weights for policy 1, policy_version 1406331 (0.0010) [2023-12-27 01:37:08,124][105620] Updated weights for policy 1, policy_version 1406341 (0.0009) [2023-12-27 01:37:08,126][105692] Updated weights for policy 0, policy_version 1404119 (0.0005) [2023-12-27 01:37:08,177][105692] Updated weights for policy 0, policy_version 1404129 (0.0010) [2023-12-27 01:37:08,180][105620] Updated weights for policy 1, policy_version 1406351 (0.0010) [2023-12-27 01:37:08,223][105692] Updated weights for policy 0, policy_version 1404139 (0.0009) [2023-12-27 01:37:08,839][105620] Updated weights for policy 1, policy_version 1406361 (0.0011) [2023-12-27 01:37:08,900][105620] Updated weights for policy 1, policy_version 1406371 (0.0005) [2023-12-27 01:37:08,916][105692] Updated weights for policy 0, policy_version 1404149 (0.0007) [2023-12-27 01:37:08,961][105620] Updated weights for policy 1, policy_version 1406381 (0.0006) [2023-12-27 01:37:08,971][105692] Updated weights for policy 0, policy_version 1404159 (0.0009) [2023-12-27 01:37:09,024][105620] Updated weights for policy 1, policy_version 1406391 (0.0005) [2023-12-27 01:37:09,025][105692] Updated weights for policy 0, policy_version 1404169 (0.0005) [2023-12-27 01:37:09,722][105620] Updated weights for policy 1, policy_version 1406401 (0.0008) [2023-12-27 01:37:09,749][105692] Updated weights for policy 0, policy_version 1404179 (0.0006) [2023-12-27 01:37:09,783][105620] Updated weights for policy 1, policy_version 1406411 (0.0011) [2023-12-27 01:37:09,814][105692] Updated weights for policy 0, policy_version 1404189 (0.0006) [2023-12-27 01:37:09,851][105620] Updated weights for policy 1, policy_version 1406421 (0.0011) [2023-12-27 01:37:09,879][105692] Updated weights for policy 0, policy_version 1404199 (0.0007) [2023-12-27 01:37:10,548][105620] Updated weights for policy 1, policy_version 1406431 (0.0008) [2023-12-27 01:37:10,594][105620] Updated weights for policy 1, policy_version 1406441 (0.0010) [2023-12-27 01:37:10,653][105620] Updated weights for policy 1, policy_version 1406451 (0.0009) [2023-12-27 01:37:10,656][105692] Updated weights for policy 0, policy_version 1404209 (0.0009) [2023-12-27 01:37:10,725][105692] Updated weights for policy 0, policy_version 1404219 (0.0010) [2023-12-27 01:37:10,784][105692] Updated weights for policy 0, policy_version 1404230 (0.0010) [2023-12-27 01:37:10,833][105692] Updated weights for policy 0, policy_version 1404240 (0.0008) [2023-12-27 01:37:11,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 719634432. Throughput: 0: 9713.4, 1: 9768.3. Samples: 719642132. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:37:11,062][104569] Avg episode reward: [(0, '8536.113'), (1, '9086.342')] [2023-12-27 01:37:11,337][105620] Updated weights for policy 1, policy_version 1406461 (0.0006) [2023-12-27 01:37:11,416][105620] Updated weights for policy 1, policy_version 1406471 (0.0014) [2023-12-27 01:37:11,468][105620] Updated weights for policy 1, policy_version 1406481 (0.0006) [2023-12-27 01:37:11,631][105692] Updated weights for policy 0, policy_version 1404250 (0.0008) [2023-12-27 01:37:11,693][105692] Updated weights for policy 0, policy_version 1404260 (0.0008) [2023-12-27 01:37:11,756][105692] Updated weights for policy 0, policy_version 1404270 (0.0008) [2023-12-27 01:37:12,058][105620] Updated weights for policy 1, policy_version 1406491 (0.0007) [2023-12-27 01:37:12,121][105620] Updated weights for policy 1, policy_version 1406501 (0.0011) [2023-12-27 01:37:12,177][105620] Updated weights for policy 1, policy_version 1406511 (0.0011) [2023-12-27 01:37:12,413][105692] Updated weights for policy 0, policy_version 1404280 (0.0008) [2023-12-27 01:37:12,468][105692] Updated weights for policy 0, policy_version 1404290 (0.0008) [2023-12-27 01:37:12,526][105692] Updated weights for policy 0, policy_version 1404300 (0.0008) [2023-12-27 01:37:12,925][105620] Updated weights for policy 1, policy_version 1406521 (0.0010) [2023-12-27 01:37:12,985][105620] Updated weights for policy 1, policy_version 1406531 (0.0009) [2023-12-27 01:37:13,041][105620] Updated weights for policy 1, policy_version 1406541 (0.0005) [2023-12-27 01:37:13,117][105620] Updated weights for policy 1, policy_version 1406551 (0.0005) [2023-12-27 01:37:13,358][105692] Updated weights for policy 0, policy_version 1404310 (0.0008) [2023-12-27 01:37:13,419][105692] Updated weights for policy 0, policy_version 1404320 (0.0008) [2023-12-27 01:37:13,468][105692] Updated weights for policy 0, policy_version 1404330 (0.0008) [2023-12-27 01:37:13,697][105620] Updated weights for policy 1, policy_version 1406561 (0.0005) [2023-12-27 01:37:13,753][105620] Updated weights for policy 1, policy_version 1406571 (0.0006) [2023-12-27 01:37:13,805][105620] Updated weights for policy 1, policy_version 1406581 (0.0007) [2023-12-27 01:37:14,321][105692] Updated weights for policy 0, policy_version 1404340 (0.0008) [2023-12-27 01:37:14,372][105692] Updated weights for policy 0, policy_version 1404350 (0.0007) [2023-12-27 01:37:14,428][105692] Updated weights for policy 0, policy_version 1404360 (0.0008) [2023-12-27 01:37:14,468][105620] Updated weights for policy 1, policy_version 1406591 (0.0010) [2023-12-27 01:37:14,512][105620] Updated weights for policy 1, policy_version 1406601 (0.0010) [2023-12-27 01:37:14,560][105620] Updated weights for policy 1, policy_version 1406611 (0.0010) [2023-12-27 01:37:15,160][105692] Updated weights for policy 0, policy_version 1404370 (0.0010) [2023-12-27 01:37:15,213][105692] Updated weights for policy 0, policy_version 1404380 (0.0010) [2023-12-27 01:37:15,266][105692] Updated weights for policy 0, policy_version 1404390 (0.0010) [2023-12-27 01:37:15,318][105692] Updated weights for policy 0, policy_version 1404400 (0.0011) [2023-12-27 01:37:15,355][105620] Updated weights for policy 1, policy_version 1406621 (0.0010) [2023-12-27 01:37:15,421][105620] Updated weights for policy 1, policy_version 1406631 (0.0011) [2023-12-27 01:37:15,470][105620] Updated weights for policy 1, policy_version 1406641 (0.0010) [2023-12-27 01:37:16,042][105692] Updated weights for policy 0, policy_version 1404410 (0.0008) [2023-12-27 01:37:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 719724544. Throughput: 0: 9703.0, 1: 9768.6. Samples: 719700436. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:37:16,062][104569] Avg episode reward: [(0, '8348.973'), (1, '8814.120')] [2023-12-27 01:37:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001406648_360144896.pth... [2023-12-27 01:37:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001405528_359858176.pth [2023-12-27 01:37:16,111][105692] Updated weights for policy 0, policy_version 1404420 (0.0009) [2023-12-27 01:37:16,160][105620] Updated weights for policy 1, policy_version 1406651 (0.0009) [2023-12-27 01:37:16,174][105692] Updated weights for policy 0, policy_version 1404430 (0.0009) [2023-12-27 01:37:16,184][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001404432_359587840.pth... [2023-12-27 01:37:16,189][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001403248_359284736.pth [2023-12-27 01:37:16,212][105620] Updated weights for policy 1, policy_version 1406661 (0.0009) [2023-12-27 01:37:16,267][105620] Updated weights for policy 1, policy_version 1406671 (0.0005) [2023-12-27 01:37:16,777][105692] Updated weights for policy 0, policy_version 1404440 (0.0005) [2023-12-27 01:37:16,836][105692] Updated weights for policy 0, policy_version 1404450 (0.0005) [2023-12-27 01:37:16,869][105620] Updated weights for policy 1, policy_version 1406681 (0.0006) [2023-12-27 01:37:16,898][105692] Updated weights for policy 0, policy_version 1404460 (0.0006) [2023-12-27 01:37:16,931][105620] Updated weights for policy 1, policy_version 1406691 (0.0010) [2023-12-27 01:37:16,985][105620] Updated weights for policy 1, policy_version 1406701 (0.0009) [2023-12-27 01:37:17,053][105620] Updated weights for policy 1, policy_version 1406711 (0.0005) [2023-12-27 01:37:17,495][105692] Updated weights for policy 0, policy_version 1404470 (0.0008) [2023-12-27 01:37:17,547][105692] Updated weights for policy 0, policy_version 1404481 (0.0010) [2023-12-27 01:37:17,575][105620] Updated weights for policy 1, policy_version 1406721 (0.0005) [2023-12-27 01:37:17,595][105692] Updated weights for policy 0, policy_version 1404492 (0.0008) [2023-12-27 01:37:17,619][105620] Updated weights for policy 1, policy_version 1406731 (0.0007) [2023-12-27 01:37:17,673][105620] Updated weights for policy 1, policy_version 1406741 (0.0006) [2023-12-27 01:37:18,365][105620] Updated weights for policy 1, policy_version 1406751 (0.0007) [2023-12-27 01:37:18,421][105692] Updated weights for policy 0, policy_version 1404502 (0.0006) [2023-12-27 01:37:18,430][105620] Updated weights for policy 1, policy_version 1406761 (0.0009) [2023-12-27 01:37:18,482][105692] Updated weights for policy 0, policy_version 1404512 (0.0006) [2023-12-27 01:37:18,489][105620] Updated weights for policy 1, policy_version 1406771 (0.0009) [2023-12-27 01:37:18,543][105692] Updated weights for policy 0, policy_version 1404522 (0.0005) [2023-12-27 01:37:19,189][105692] Updated weights for policy 0, policy_version 1404532 (0.0008) [2023-12-27 01:37:19,250][105692] Updated weights for policy 0, policy_version 1404542 (0.0011) [2023-12-27 01:37:19,270][105620] Updated weights for policy 1, policy_version 1406781 (0.0007) [2023-12-27 01:37:19,312][105692] Updated weights for policy 0, policy_version 1404552 (0.0011) [2023-12-27 01:37:19,340][105620] Updated weights for policy 1, policy_version 1406791 (0.0007) [2023-12-27 01:37:19,410][105620] Updated weights for policy 1, policy_version 1406801 (0.0008) [2023-12-27 01:37:20,078][105692] Updated weights for policy 0, policy_version 1404562 (0.0009) [2023-12-27 01:37:20,138][105692] Updated weights for policy 0, policy_version 1404572 (0.0010) [2023-12-27 01:37:20,199][105692] Updated weights for policy 0, policy_version 1404582 (0.0010) [2023-12-27 01:37:20,200][105620] Updated weights for policy 1, policy_version 1406811 (0.0008) [2023-12-27 01:37:20,250][105620] Updated weights for policy 1, policy_version 1406821 (0.0008) [2023-12-27 01:37:20,252][105692] Updated weights for policy 0, policy_version 1404592 (0.0007) [2023-12-27 01:37:20,308][105620] Updated weights for policy 1, policy_version 1406831 (0.0009) [2023-12-27 01:37:20,998][105692] Updated weights for policy 0, policy_version 1404602 (0.0011) [2023-12-27 01:37:21,046][105620] Updated weights for policy 1, policy_version 1406841 (0.0009) [2023-12-27 01:37:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 719822848. Throughput: 0: 9774.3, 1: 9767.1. Samples: 719819880. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:37:21,062][104569] Avg episode reward: [(0, '7711.694'), (1, '8993.635')] [2023-12-27 01:37:21,067][105692] Updated weights for policy 0, policy_version 1404612 (0.0011) [2023-12-27 01:37:21,109][105620] Updated weights for policy 1, policy_version 1406851 (0.0009) [2023-12-27 01:37:21,128][105692] Updated weights for policy 0, policy_version 1404622 (0.0011) [2023-12-27 01:37:21,170][105620] Updated weights for policy 1, policy_version 1406861 (0.0010) [2023-12-27 01:37:21,234][105620] Updated weights for policy 1, policy_version 1406871 (0.0009) [2023-12-27 01:37:21,881][105692] Updated weights for policy 0, policy_version 1404632 (0.0009) [2023-12-27 01:37:21,949][105692] Updated weights for policy 0, policy_version 1404642 (0.0006) [2023-12-27 01:37:21,975][105620] Updated weights for policy 1, policy_version 1406881 (0.0007) [2023-12-27 01:37:22,017][105692] Updated weights for policy 0, policy_version 1404652 (0.0006) [2023-12-27 01:37:22,035][105620] Updated weights for policy 1, policy_version 1406891 (0.0008) [2023-12-27 01:37:22,097][105620] Updated weights for policy 1, policy_version 1406901 (0.0009) [2023-12-27 01:37:22,683][105692] Updated weights for policy 0, policy_version 1404662 (0.0006) [2023-12-27 01:37:22,751][105692] Updated weights for policy 0, policy_version 1404672 (0.0008) [2023-12-27 01:37:22,811][105620] Updated weights for policy 1, policy_version 1406911 (0.0008) [2023-12-27 01:37:22,814][105692] Updated weights for policy 0, policy_version 1404682 (0.0008) [2023-12-27 01:37:22,868][105620] Updated weights for policy 1, policy_version 1406921 (0.0007) [2023-12-27 01:37:22,918][105620] Updated weights for policy 1, policy_version 1406931 (0.0009) [2023-12-27 01:37:23,577][105620] Updated weights for policy 1, policy_version 1406941 (0.0009) [2023-12-27 01:37:23,609][105692] Updated weights for policy 0, policy_version 1404692 (0.0008) [2023-12-27 01:37:23,644][105620] Updated weights for policy 1, policy_version 1406951 (0.0007) [2023-12-27 01:37:23,662][105692] Updated weights for policy 0, policy_version 1404702 (0.0007) [2023-12-27 01:37:23,708][105620] Updated weights for policy 1, policy_version 1406961 (0.0008) [2023-12-27 01:37:23,718][105692] Updated weights for policy 0, policy_version 1404712 (0.0008) [2023-12-27 01:37:24,277][105620] Updated weights for policy 1, policy_version 1406971 (0.0008) [2023-12-27 01:37:24,346][105620] Updated weights for policy 1, policy_version 1406981 (0.0008) [2023-12-27 01:37:24,409][105620] Updated weights for policy 1, policy_version 1406991 (0.0008) [2023-12-27 01:37:24,493][105692] Updated weights for policy 0, policy_version 1404722 (0.0009) [2023-12-27 01:37:24,553][105692] Updated weights for policy 0, policy_version 1404732 (0.0006) [2023-12-27 01:37:24,610][105692] Updated weights for policy 0, policy_version 1404742 (0.0006) [2023-12-27 01:37:24,666][105692] Updated weights for policy 0, policy_version 1404752 (0.0011) [2023-12-27 01:37:25,221][105620] Updated weights for policy 1, policy_version 1407001 (0.0008) [2023-12-27 01:37:25,238][105692] Updated weights for policy 0, policy_version 1404762 (0.0008) [2023-12-27 01:37:25,269][105620] Updated weights for policy 1, policy_version 1407011 (0.0005) [2023-12-27 01:37:25,287][105692] Updated weights for policy 0, policy_version 1404772 (0.0010) [2023-12-27 01:37:25,318][105620] Updated weights for policy 1, policy_version 1407021 (0.0007) [2023-12-27 01:37:25,336][105692] Updated weights for policy 0, policy_version 1404782 (0.0005) [2023-12-27 01:37:25,377][105620] Updated weights for policy 1, policy_version 1407031 (0.0009) [2023-12-27 01:37:25,991][105692] Updated weights for policy 0, policy_version 1404792 (0.0009) [2023-12-27 01:37:26,049][105692] Updated weights for policy 0, policy_version 1404802 (0.0011) [2023-12-27 01:37:26,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19524.2, 300 sec: 19383.1). Total num frames: 719921152. Throughput: 0: 9878.7, 1: 9721.2. Samples: 719936044. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:37:26,063][104569] Avg episode reward: [(0, '6058.417'), (1, '8988.742')] [2023-12-27 01:37:26,108][105692] Updated weights for policy 0, policy_version 1404812 (0.0011) [2023-12-27 01:37:26,179][105620] Updated weights for policy 1, policy_version 1407041 (0.0006) [2023-12-27 01:37:26,235][105620] Updated weights for policy 1, policy_version 1407051 (0.0008) [2023-12-27 01:37:26,294][105620] Updated weights for policy 1, policy_version 1407061 (0.0008) [2023-12-27 01:37:26,858][105692] Updated weights for policy 0, policy_version 1404822 (0.0009) [2023-12-27 01:37:26,916][105692] Updated weights for policy 0, policy_version 1404832 (0.0009) [2023-12-27 01:37:26,977][105692] Updated weights for policy 0, policy_version 1404842 (0.0009) [2023-12-27 01:37:27,055][105620] Updated weights for policy 1, policy_version 1407071 (0.0009) [2023-12-27 01:37:27,106][105620] Updated weights for policy 1, policy_version 1407081 (0.0009) [2023-12-27 01:37:27,153][105620] Updated weights for policy 1, policy_version 1407091 (0.0009) [2023-12-27 01:37:27,635][105692] Updated weights for policy 0, policy_version 1404852 (0.0007) [2023-12-27 01:37:27,691][105692] Updated weights for policy 0, policy_version 1404862 (0.0008) [2023-12-27 01:37:27,749][105692] Updated weights for policy 0, policy_version 1404872 (0.0010) [2023-12-27 01:37:28,014][105620] Updated weights for policy 1, policy_version 1407101 (0.0009) [2023-12-27 01:37:28,070][105620] Updated weights for policy 1, policy_version 1407111 (0.0009) [2023-12-27 01:37:28,135][105620] Updated weights for policy 1, policy_version 1407121 (0.0009) [2023-12-27 01:37:28,295][105692] Updated weights for policy 0, policy_version 1404882 (0.0006) [2023-12-27 01:37:28,355][105692] Updated weights for policy 0, policy_version 1404892 (0.0007) [2023-12-27 01:37:28,412][105692] Updated weights for policy 0, policy_version 1404902 (0.0007) [2023-12-27 01:37:28,473][105692] Updated weights for policy 0, policy_version 1404912 (0.0011) [2023-12-27 01:37:28,945][105620] Updated weights for policy 1, policy_version 1407131 (0.0009) [2023-12-27 01:37:29,005][105620] Updated weights for policy 1, policy_version 1407141 (0.0008) [2023-12-27 01:37:29,060][105620] Updated weights for policy 1, policy_version 1407151 (0.0008) [2023-12-27 01:37:29,167][105692] Updated weights for policy 0, policy_version 1404922 (0.0010) [2023-12-27 01:37:29,233][105692] Updated weights for policy 0, policy_version 1404932 (0.0010) [2023-12-27 01:37:29,291][105692] Updated weights for policy 0, policy_version 1404942 (0.0008) [2023-12-27 01:37:29,860][105620] Updated weights for policy 1, policy_version 1407161 (0.0009) [2023-12-27 01:37:29,922][105620] Updated weights for policy 1, policy_version 1407171 (0.0009) [2023-12-27 01:37:29,989][105620] Updated weights for policy 1, policy_version 1407181 (0.0010) [2023-12-27 01:37:30,019][105692] Updated weights for policy 0, policy_version 1404952 (0.0007) [2023-12-27 01:37:30,052][105620] Updated weights for policy 1, policy_version 1407191 (0.0010) [2023-12-27 01:37:30,078][105692] Updated weights for policy 0, policy_version 1404962 (0.0005) [2023-12-27 01:37:30,143][105692] Updated weights for policy 0, policy_version 1404972 (0.0006) [2023-12-27 01:37:30,688][105692] Updated weights for policy 0, policy_version 1404982 (0.0005) [2023-12-27 01:37:30,744][105692] Updated weights for policy 0, policy_version 1404992 (0.0005) [2023-12-27 01:37:30,777][105620] Updated weights for policy 1, policy_version 1407201 (0.0010) [2023-12-27 01:37:30,800][105692] Updated weights for policy 0, policy_version 1405002 (0.0005) [2023-12-27 01:37:30,829][105620] Updated weights for policy 1, policy_version 1407211 (0.0010) [2023-12-27 01:37:30,891][105620] Updated weights for policy 1, policy_version 1407221 (0.0010) [2023-12-27 01:37:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 720027648. Throughput: 0: 9920.3, 1: 9719.2. Samples: 719993948. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:37:31,062][104569] Avg episode reward: [(0, '7425.744'), (1, '8988.960')] [2023-12-27 01:37:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001405008_359735296.pth... [2023-12-27 01:37:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001407224_360292352.pth... [2023-12-27 01:37:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001406072_359997440.pth [2023-12-27 01:37:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001403856_359440384.pth [2023-12-27 01:37:31,368][105692] Updated weights for policy 0, policy_version 1405012 (0.0006) [2023-12-27 01:37:31,430][105692] Updated weights for policy 0, policy_version 1405022 (0.0007) [2023-12-27 01:37:31,487][105692] Updated weights for policy 0, policy_version 1405032 (0.0008) [2023-12-27 01:37:31,651][105620] Updated weights for policy 1, policy_version 1407231 (0.0011) [2023-12-27 01:37:31,707][105620] Updated weights for policy 1, policy_version 1407241 (0.0010) [2023-12-27 01:37:31,772][105620] Updated weights for policy 1, policy_version 1407251 (0.0010) [2023-12-27 01:37:32,168][105692] Updated weights for policy 0, policy_version 1405042 (0.0008) [2023-12-27 01:37:32,228][105692] Updated weights for policy 0, policy_version 1405052 (0.0007) [2023-12-27 01:37:32,286][105692] Updated weights for policy 0, policy_version 1405062 (0.0006) [2023-12-27 01:37:32,349][105692] Updated weights for policy 0, policy_version 1405072 (0.0008) [2023-12-27 01:37:32,464][105620] Updated weights for policy 1, policy_version 1407261 (0.0010) [2023-12-27 01:37:32,520][105620] Updated weights for policy 1, policy_version 1407272 (0.0011) [2023-12-27 01:37:32,585][105620] Updated weights for policy 1, policy_version 1407282 (0.0006) [2023-12-27 01:37:32,944][105692] Updated weights for policy 0, policy_version 1405082 (0.0005) [2023-12-27 01:37:33,007][105692] Updated weights for policy 0, policy_version 1405092 (0.0007) [2023-12-27 01:37:33,064][105692] Updated weights for policy 0, policy_version 1405104 (0.0012) [2023-12-27 01:37:33,247][105620] Updated weights for policy 1, policy_version 1407292 (0.0009) [2023-12-27 01:37:33,318][105620] Updated weights for policy 1, policy_version 1407302 (0.0010) [2023-12-27 01:37:33,382][105620] Updated weights for policy 1, policy_version 1407312 (0.0010) [2023-12-27 01:37:33,647][105692] Updated weights for policy 0, policy_version 1405114 (0.0010) [2023-12-27 01:37:33,709][105692] Updated weights for policy 0, policy_version 1405124 (0.0010) [2023-12-27 01:37:33,779][105692] Updated weights for policy 0, policy_version 1405134 (0.0005) [2023-12-27 01:37:33,909][105620] Updated weights for policy 1, policy_version 1407322 (0.0007) [2023-12-27 01:37:33,961][105620] Updated weights for policy 1, policy_version 1407332 (0.0008) [2023-12-27 01:37:34,016][105620] Updated weights for policy 1, policy_version 1407342 (0.0008) [2023-12-27 01:37:34,074][105620] Updated weights for policy 1, policy_version 1407352 (0.0006) [2023-12-27 01:37:34,429][105692] Updated weights for policy 0, policy_version 1405144 (0.0010) [2023-12-27 01:37:34,492][105692] Updated weights for policy 0, policy_version 1405154 (0.0010) [2023-12-27 01:37:34,553][105692] Updated weights for policy 0, policy_version 1405164 (0.0010) [2023-12-27 01:37:34,797][105620] Updated weights for policy 1, policy_version 1407362 (0.0009) [2023-12-27 01:37:34,853][105620] Updated weights for policy 1, policy_version 1407372 (0.0010) [2023-12-27 01:37:34,905][105620] Updated weights for policy 1, policy_version 1407382 (0.0009) [2023-12-27 01:37:35,181][105692] Updated weights for policy 0, policy_version 1405174 (0.0007) [2023-12-27 01:37:35,242][105692] Updated weights for policy 0, policy_version 1405184 (0.0009) [2023-12-27 01:37:35,302][105692] Updated weights for policy 0, policy_version 1405194 (0.0009) [2023-12-27 01:37:35,675][105620] Updated weights for policy 1, policy_version 1407392 (0.0009) [2023-12-27 01:37:35,745][105620] Updated weights for policy 1, policy_version 1407402 (0.0008) [2023-12-27 01:37:35,806][105620] Updated weights for policy 1, policy_version 1407412 (0.0008) [2023-12-27 01:37:36,036][105692] Updated weights for policy 0, policy_version 1405204 (0.0009) [2023-12-27 01:37:36,062][104569] Fps is (10 sec: 20480.8, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 720125952. Throughput: 0: 10009.8, 1: 9845.1. Samples: 720116480. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:37:36,062][104569] Avg episode reward: [(0, '8164.034'), (1, '9176.261')] [2023-12-27 01:37:36,098][105692] Updated weights for policy 0, policy_version 1405214 (0.0010) [2023-12-27 01:37:36,164][105692] Updated weights for policy 0, policy_version 1405224 (0.0009) [2023-12-27 01:37:36,517][105620] Updated weights for policy 1, policy_version 1407422 (0.0009) [2023-12-27 01:37:36,576][105620] Updated weights for policy 1, policy_version 1407432 (0.0009) [2023-12-27 01:37:36,643][105620] Updated weights for policy 1, policy_version 1407442 (0.0009) [2023-12-27 01:37:36,971][105692] Updated weights for policy 0, policy_version 1405234 (0.0009) [2023-12-27 01:37:37,032][105692] Updated weights for policy 0, policy_version 1405244 (0.0009) [2023-12-27 01:37:37,090][105692] Updated weights for policy 0, policy_version 1405254 (0.0009) [2023-12-27 01:37:37,148][105692] Updated weights for policy 0, policy_version 1405264 (0.0009) [2023-12-27 01:37:37,303][105620] Updated weights for policy 1, policy_version 1407452 (0.0009) [2023-12-27 01:37:37,354][105620] Updated weights for policy 1, policy_version 1407462 (0.0009) [2023-12-27 01:37:37,406][105620] Updated weights for policy 1, policy_version 1407472 (0.0009) [2023-12-27 01:37:37,894][105692] Updated weights for policy 0, policy_version 1405274 (0.0008) [2023-12-27 01:37:37,961][105692] Updated weights for policy 0, policy_version 1405284 (0.0005) [2023-12-27 01:37:38,015][105692] Updated weights for policy 0, policy_version 1405294 (0.0005) [2023-12-27 01:37:38,161][105620] Updated weights for policy 1, policy_version 1407482 (0.0007) [2023-12-27 01:37:38,228][105620] Updated weights for policy 1, policy_version 1407492 (0.0005) [2023-12-27 01:37:38,291][105620] Updated weights for policy 1, policy_version 1407502 (0.0006) [2023-12-27 01:37:38,360][105620] Updated weights for policy 1, policy_version 1407512 (0.0008) [2023-12-27 01:37:38,615][105692] Updated weights for policy 0, policy_version 1405304 (0.0008) [2023-12-27 01:37:38,681][105692] Updated weights for policy 0, policy_version 1405314 (0.0008) [2023-12-27 01:37:38,736][105692] Updated weights for policy 0, policy_version 1405324 (0.0006) [2023-12-27 01:37:39,051][105620] Updated weights for policy 1, policy_version 1407522 (0.0009) [2023-12-27 01:37:39,110][105620] Updated weights for policy 1, policy_version 1407532 (0.0009) [2023-12-27 01:37:39,165][105620] Updated weights for policy 1, policy_version 1407542 (0.0007) [2023-12-27 01:37:39,522][105692] Updated weights for policy 0, policy_version 1405334 (0.0009) [2023-12-27 01:37:39,582][105692] Updated weights for policy 0, policy_version 1405344 (0.0009) [2023-12-27 01:37:39,641][105692] Updated weights for policy 0, policy_version 1405354 (0.0009) [2023-12-27 01:37:39,951][105620] Updated weights for policy 1, policy_version 1407552 (0.0010) [2023-12-27 01:37:40,007][105620] Updated weights for policy 1, policy_version 1407562 (0.0010) [2023-12-27 01:37:40,080][105620] Updated weights for policy 1, policy_version 1407572 (0.0006) [2023-12-27 01:37:40,291][105692] Updated weights for policy 0, policy_version 1405364 (0.0008) [2023-12-27 01:37:40,345][105692] Updated weights for policy 0, policy_version 1405374 (0.0009) [2023-12-27 01:37:40,401][105692] Updated weights for policy 0, policy_version 1405384 (0.0009) [2023-12-27 01:37:40,740][105620] Updated weights for policy 1, policy_version 1407582 (0.0007) [2023-12-27 01:37:40,795][105620] Updated weights for policy 1, policy_version 1407592 (0.0005) [2023-12-27 01:37:40,848][105620] Updated weights for policy 1, policy_version 1407602 (0.0010) [2023-12-27 01:37:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19438.7). Total num frames: 720224256. Throughput: 0: 9999.6, 1: 9802.7. Samples: 720232876. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:37:41,062][104569] Avg episode reward: [(0, '8165.107'), (1, '9173.024')] [2023-12-27 01:37:41,161][105692] Updated weights for policy 0, policy_version 1405394 (0.0009) [2023-12-27 01:37:41,222][105692] Updated weights for policy 0, policy_version 1405404 (0.0008) [2023-12-27 01:37:41,280][105692] Updated weights for policy 0, policy_version 1405414 (0.0008) [2023-12-27 01:37:41,340][105692] Updated weights for policy 0, policy_version 1405424 (0.0008) [2023-12-27 01:37:41,625][105620] Updated weights for policy 1, policy_version 1407612 (0.0011) [2023-12-27 01:37:41,685][105620] Updated weights for policy 1, policy_version 1407622 (0.0013) [2023-12-27 01:37:41,754][105620] Updated weights for policy 1, policy_version 1407632 (0.0012) [2023-12-27 01:37:42,139][105692] Updated weights for policy 0, policy_version 1405434 (0.0009) [2023-12-27 01:37:42,199][105692] Updated weights for policy 0, policy_version 1405444 (0.0008) [2023-12-27 01:37:42,253][105692] Updated weights for policy 0, policy_version 1405454 (0.0008) [2023-12-27 01:37:42,493][105620] Updated weights for policy 1, policy_version 1407642 (0.0011) [2023-12-27 01:37:42,550][105620] Updated weights for policy 1, policy_version 1407652 (0.0011) [2023-12-27 01:37:42,609][105620] Updated weights for policy 1, policy_version 1407662 (0.0010) [2023-12-27 01:37:42,669][105620] Updated weights for policy 1, policy_version 1407672 (0.0007) [2023-12-27 01:37:43,083][105692] Updated weights for policy 0, policy_version 1405464 (0.0009) [2023-12-27 01:37:43,150][105692] Updated weights for policy 0, policy_version 1405474 (0.0007) [2023-12-27 01:37:43,217][105692] Updated weights for policy 0, policy_version 1405484 (0.0008) [2023-12-27 01:37:43,258][105620] Updated weights for policy 1, policy_version 1407682 (0.0007) [2023-12-27 01:37:43,329][105620] Updated weights for policy 1, policy_version 1407692 (0.0010) [2023-12-27 01:37:43,397][105620] Updated weights for policy 1, policy_version 1407702 (0.0007) [2023-12-27 01:37:43,758][105692] Updated weights for policy 0, policy_version 1405494 (0.0007) [2023-12-27 01:37:43,821][105692] Updated weights for policy 0, policy_version 1405504 (0.0008) [2023-12-27 01:37:43,880][105692] Updated weights for policy 0, policy_version 1405514 (0.0009) [2023-12-27 01:37:44,115][105620] Updated weights for policy 1, policy_version 1407712 (0.0009) [2023-12-27 01:37:44,181][105620] Updated weights for policy 1, policy_version 1407722 (0.0009) [2023-12-27 01:37:44,235][105620] Updated weights for policy 1, policy_version 1407732 (0.0009) [2023-12-27 01:37:44,545][105692] Updated weights for policy 0, policy_version 1405524 (0.0009) [2023-12-27 01:37:44,596][105692] Updated weights for policy 0, policy_version 1405534 (0.0009) [2023-12-27 01:37:44,643][105692] Updated weights for policy 0, policy_version 1405544 (0.0008) [2023-12-27 01:37:45,002][105620] Updated weights for policy 1, policy_version 1407742 (0.0009) [2023-12-27 01:37:45,054][105620] Updated weights for policy 1, policy_version 1407752 (0.0009) [2023-12-27 01:37:45,107][105620] Updated weights for policy 1, policy_version 1407762 (0.0009) [2023-12-27 01:37:45,383][105692] Updated weights for policy 0, policy_version 1405554 (0.0009) [2023-12-27 01:37:45,445][105692] Updated weights for policy 0, policy_version 1405564 (0.0009) [2023-12-27 01:37:45,508][105692] Updated weights for policy 0, policy_version 1405574 (0.0010) [2023-12-27 01:37:45,569][105692] Updated weights for policy 0, policy_version 1405584 (0.0008) [2023-12-27 01:37:45,806][105620] Updated weights for policy 1, policy_version 1407772 (0.0009) [2023-12-27 01:37:45,854][105620] Updated weights for policy 1, policy_version 1407782 (0.0008) [2023-12-27 01:37:45,900][105620] Updated weights for policy 1, policy_version 1407792 (0.0008) [2023-12-27 01:37:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19410.9). Total num frames: 720322560. Throughput: 0: 9931.7, 1: 9815.6. Samples: 720290228. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:37:46,062][104569] Avg episode reward: [(0, '8713.952'), (1, '8989.640')] [2023-12-27 01:37:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001407800_360439808.pth... [2023-12-27 01:37:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001405584_359882752.pth... [2023-12-27 01:37:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001406648_360144896.pth [2023-12-27 01:37:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001404432_359587840.pth [2023-12-27 01:37:46,299][105692] Updated weights for policy 0, policy_version 1405594 (0.0010) [2023-12-27 01:37:46,349][105692] Updated weights for policy 0, policy_version 1405604 (0.0009) [2023-12-27 01:37:46,403][105692] Updated weights for policy 0, policy_version 1405614 (0.0010) [2023-12-27 01:37:46,541][105620] Updated weights for policy 1, policy_version 1407802 (0.0009) [2023-12-27 01:37:46,587][105620] Updated weights for policy 1, policy_version 1407812 (0.0008) [2023-12-27 01:37:46,636][105620] Updated weights for policy 1, policy_version 1407822 (0.0008) [2023-12-27 01:37:46,689][105620] Updated weights for policy 1, policy_version 1407832 (0.0005) [2023-12-27 01:37:47,280][105620] Updated weights for policy 1, policy_version 1407842 (0.0007) [2023-12-27 01:37:47,291][105692] Updated weights for policy 0, policy_version 1405624 (0.0007) [2023-12-27 01:37:47,337][105620] Updated weights for policy 1, policy_version 1407852 (0.0008) [2023-12-27 01:37:47,344][105692] Updated weights for policy 0, policy_version 1405634 (0.0005) [2023-12-27 01:37:47,393][105620] Updated weights for policy 1, policy_version 1407862 (0.0009) [2023-12-27 01:37:47,404][105692] Updated weights for policy 0, policy_version 1405644 (0.0006) [2023-12-27 01:37:47,956][105692] Updated weights for policy 0, policy_version 1405654 (0.0008) [2023-12-27 01:37:48,014][105692] Updated weights for policy 0, policy_version 1405664 (0.0010) [2023-12-27 01:37:48,068][105692] Updated weights for policy 0, policy_version 1405674 (0.0008) [2023-12-27 01:37:48,236][105620] Updated weights for policy 1, policy_version 1407872 (0.0008) [2023-12-27 01:37:48,284][105620] Updated weights for policy 1, policy_version 1407882 (0.0008) [2023-12-27 01:37:48,340][105620] Updated weights for policy 1, policy_version 1407892 (0.0010) [2023-12-27 01:37:48,822][105692] Updated weights for policy 0, policy_version 1405684 (0.0009) [2023-12-27 01:37:48,878][105692] Updated weights for policy 0, policy_version 1405694 (0.0008) [2023-12-27 01:37:48,933][105692] Updated weights for policy 0, policy_version 1405704 (0.0006) [2023-12-27 01:37:49,043][105620] Updated weights for policy 1, policy_version 1407902 (0.0009) [2023-12-27 01:37:49,092][105620] Updated weights for policy 1, policy_version 1407912 (0.0010) [2023-12-27 01:37:49,140][105620] Updated weights for policy 1, policy_version 1407922 (0.0010) [2023-12-27 01:37:49,672][105692] Updated weights for policy 0, policy_version 1405714 (0.0007) [2023-12-27 01:37:49,728][105692] Updated weights for policy 0, policy_version 1405724 (0.0006) [2023-12-27 01:37:49,793][105692] Updated weights for policy 0, policy_version 1405734 (0.0009) [2023-12-27 01:37:49,813][105620] Updated weights for policy 1, policy_version 1407932 (0.0009) [2023-12-27 01:37:49,855][105692] Updated weights for policy 0, policy_version 1405744 (0.0009) [2023-12-27 01:37:49,877][105620] Updated weights for policy 1, policy_version 1407942 (0.0009) [2023-12-27 01:37:49,938][105620] Updated weights for policy 1, policy_version 1407952 (0.0009) [2023-12-27 01:37:50,473][105692] Updated weights for policy 0, policy_version 1405754 (0.0009) [2023-12-27 01:37:50,518][105692] Updated weights for policy 0, policy_version 1405764 (0.0009) [2023-12-27 01:37:50,576][105692] Updated weights for policy 0, policy_version 1405774 (0.0008) [2023-12-27 01:37:50,592][105620] Updated weights for policy 1, policy_version 1407962 (0.0009) [2023-12-27 01:37:50,654][105620] Updated weights for policy 1, policy_version 1407972 (0.0009) [2023-12-27 01:37:50,704][105620] Updated weights for policy 1, policy_version 1407982 (0.0009) [2023-12-27 01:37:50,768][105620] Updated weights for policy 1, policy_version 1407992 (0.0010) [2023-12-27 01:37:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19438.6). Total num frames: 720420864. Throughput: 0: 9935.8, 1: 9774.9. Samples: 720408256. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:37:51,062][104569] Avg episode reward: [(0, '8626.618'), (1, '8807.409')] [2023-12-27 01:37:51,347][105692] Updated weights for policy 0, policy_version 1405784 (0.0009) [2023-12-27 01:37:51,419][105692] Updated weights for policy 0, policy_version 1405794 (0.0008) [2023-12-27 01:37:51,477][105692] Updated weights for policy 0, policy_version 1405804 (0.0009) [2023-12-27 01:37:51,550][105620] Updated weights for policy 1, policy_version 1408002 (0.0009) [2023-12-27 01:37:51,606][105620] Updated weights for policy 1, policy_version 1408012 (0.0008) [2023-12-27 01:37:51,679][105620] Updated weights for policy 1, policy_version 1408022 (0.0009) [2023-12-27 01:37:52,102][105692] Updated weights for policy 0, policy_version 1405814 (0.0007) [2023-12-27 01:37:52,152][105692] Updated weights for policy 0, policy_version 1405824 (0.0005) [2023-12-27 01:37:52,207][105692] Updated weights for policy 0, policy_version 1405834 (0.0006) [2023-12-27 01:37:52,375][105620] Updated weights for policy 1, policy_version 1408032 (0.0008) [2023-12-27 01:37:52,445][105620] Updated weights for policy 1, policy_version 1408042 (0.0009) [2023-12-27 01:37:52,510][105620] Updated weights for policy 1, policy_version 1408052 (0.0009) [2023-12-27 01:37:52,891][105692] Updated weights for policy 0, policy_version 1405844 (0.0007) [2023-12-27 01:37:52,942][105692] Updated weights for policy 0, policy_version 1405854 (0.0008) [2023-12-27 01:37:53,009][105692] Updated weights for policy 0, policy_version 1405864 (0.0006) [2023-12-27 01:37:53,214][105620] Updated weights for policy 1, policy_version 1408062 (0.0006) [2023-12-27 01:37:53,269][105620] Updated weights for policy 1, policy_version 1408072 (0.0005) [2023-12-27 01:37:53,330][105620] Updated weights for policy 1, policy_version 1408082 (0.0006) [2023-12-27 01:37:53,775][105692] Updated weights for policy 0, policy_version 1405874 (0.0010) [2023-12-27 01:37:53,833][105692] Updated weights for policy 0, policy_version 1405884 (0.0010) [2023-12-27 01:37:53,878][105620] Updated weights for policy 1, policy_version 1408092 (0.0005) [2023-12-27 01:37:53,885][105692] Updated weights for policy 0, policy_version 1405894 (0.0009) [2023-12-27 01:37:53,932][105620] Updated weights for policy 1, policy_version 1408102 (0.0008) [2023-12-27 01:37:53,935][105692] Updated weights for policy 0, policy_version 1405904 (0.0007) [2023-12-27 01:37:53,988][105620] Updated weights for policy 1, policy_version 1408112 (0.0006) [2023-12-27 01:37:54,574][105620] Updated weights for policy 1, policy_version 1408122 (0.0006) [2023-12-27 01:37:54,623][105620] Updated weights for policy 1, policy_version 1408132 (0.0010) [2023-12-27 01:37:54,674][105620] Updated weights for policy 1, policy_version 1408142 (0.0010) [2023-12-27 01:37:54,731][105692] Updated weights for policy 0, policy_version 1405914 (0.0006) [2023-12-27 01:37:54,736][105620] Updated weights for policy 1, policy_version 1408152 (0.0010) [2023-12-27 01:37:54,782][105692] Updated weights for policy 0, policy_version 1405924 (0.0008) [2023-12-27 01:37:54,834][105692] Updated weights for policy 0, policy_version 1405934 (0.0008) [2023-12-27 01:37:55,506][105620] Updated weights for policy 1, policy_version 1408162 (0.0010) [2023-12-27 01:37:55,554][105692] Updated weights for policy 0, policy_version 1405944 (0.0008) [2023-12-27 01:37:55,564][105620] Updated weights for policy 1, policy_version 1408172 (0.0010) [2023-12-27 01:37:55,609][105692] Updated weights for policy 0, policy_version 1405954 (0.0005) [2023-12-27 01:37:55,622][105620] Updated weights for policy 1, policy_version 1408182 (0.0010) [2023-12-27 01:37:55,661][105692] Updated weights for policy 0, policy_version 1405964 (0.0008) [2023-12-27 01:37:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19438.6). Total num frames: 720519168. Throughput: 0: 9846.7, 1: 9825.5. Samples: 720527384. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:37:56,063][104569] Avg episode reward: [(0, '7982.822'), (1, '8899.433')] [2023-12-27 01:37:56,305][105620] Updated weights for policy 1, policy_version 1408192 (0.0011) [2023-12-27 01:37:56,355][105620] Updated weights for policy 1, policy_version 1408202 (0.0010) [2023-12-27 01:37:56,417][105620] Updated weights for policy 1, policy_version 1408212 (0.0008) [2023-12-27 01:37:56,449][105692] Updated weights for policy 0, policy_version 1405974 (0.0007) [2023-12-27 01:37:56,513][105692] Updated weights for policy 0, policy_version 1405984 (0.0009) [2023-12-27 01:37:56,576][105692] Updated weights for policy 0, policy_version 1405994 (0.0010) [2023-12-27 01:37:57,106][105620] Updated weights for policy 1, policy_version 1408222 (0.0006) [2023-12-27 01:37:57,169][105620] Updated weights for policy 1, policy_version 1408232 (0.0009) [2023-12-27 01:37:57,231][105620] Updated weights for policy 1, policy_version 1408242 (0.0009) [2023-12-27 01:37:57,290][105692] Updated weights for policy 0, policy_version 1406005 (0.0009) [2023-12-27 01:37:57,348][105692] Updated weights for policy 0, policy_version 1406015 (0.0008) [2023-12-27 01:37:57,397][105692] Updated weights for policy 0, policy_version 1406025 (0.0008) [2023-12-27 01:37:57,926][105620] Updated weights for policy 1, policy_version 1408252 (0.0007) [2023-12-27 01:37:57,986][105620] Updated weights for policy 1, policy_version 1408262 (0.0005) [2023-12-27 01:37:58,038][105620] Updated weights for policy 1, policy_version 1408272 (0.0006) [2023-12-27 01:37:58,043][105586] KL-divergence is very high: 108.2909 [2023-12-27 01:37:58,204][105692] Updated weights for policy 0, policy_version 1406035 (0.0009) [2023-12-27 01:37:58,261][105692] Updated weights for policy 0, policy_version 1406045 (0.0008) [2023-12-27 01:37:58,326][105692] Updated weights for policy 0, policy_version 1406055 (0.0009) [2023-12-27 01:37:58,794][105620] Updated weights for policy 1, policy_version 1408282 (0.0009) [2023-12-27 01:37:58,859][105620] Updated weights for policy 1, policy_version 1408292 (0.0009) [2023-12-27 01:37:58,926][105620] Updated weights for policy 1, policy_version 1408302 (0.0009) [2023-12-27 01:37:58,983][105620] Updated weights for policy 1, policy_version 1408312 (0.0009) [2023-12-27 01:37:59,242][105692] Updated weights for policy 0, policy_version 1406065 (0.0008) [2023-12-27 01:37:59,306][105692] Updated weights for policy 0, policy_version 1406075 (0.0009) [2023-12-27 01:37:59,374][105692] Updated weights for policy 0, policy_version 1406085 (0.0009) [2023-12-27 01:37:59,427][105692] Updated weights for policy 0, policy_version 1406095 (0.0009) [2023-12-27 01:37:59,794][105620] Updated weights for policy 1, policy_version 1408322 (0.0009) [2023-12-27 01:37:59,855][105620] Updated weights for policy 1, policy_version 1408332 (0.0008) [2023-12-27 01:37:59,919][105620] Updated weights for policy 1, policy_version 1408342 (0.0009) [2023-12-27 01:38:00,180][105692] Updated weights for policy 0, policy_version 1406105 (0.0008) [2023-12-27 01:38:00,233][105692] Updated weights for policy 0, policy_version 1406115 (0.0008) [2023-12-27 01:38:00,281][105692] Updated weights for policy 0, policy_version 1406125 (0.0007) [2023-12-27 01:38:00,652][105620] Updated weights for policy 1, policy_version 1408352 (0.0006) [2023-12-27 01:38:00,712][105620] Updated weights for policy 1, policy_version 1408362 (0.0007) [2023-12-27 01:38:00,764][105620] Updated weights for policy 1, policy_version 1408372 (0.0009) [2023-12-27 01:38:00,911][105692] Updated weights for policy 0, policy_version 1406135 (0.0009) [2023-12-27 01:38:00,958][105692] Updated weights for policy 0, policy_version 1406145 (0.0006) [2023-12-27 01:38:01,006][105692] Updated weights for policy 0, policy_version 1406155 (0.0005) [2023-12-27 01:38:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19410.9). Total num frames: 720617472. Throughput: 0: 9848.4, 1: 9784.4. Samples: 720583916. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:38:01,063][104569] Avg episode reward: [(0, '7984.069'), (1, '8987.204')] [2023-12-27 01:38:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001406160_360030208.pth... [2023-12-27 01:38:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001408376_360587264.pth... [2023-12-27 01:38:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001405008_359735296.pth [2023-12-27 01:38:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001407224_360292352.pth [2023-12-27 01:38:01,489][105620] Updated weights for policy 1, policy_version 1408382 (0.0008) [2023-12-27 01:38:01,538][105620] Updated weights for policy 1, policy_version 1408392 (0.0009) [2023-12-27 01:38:01,586][105620] Updated weights for policy 1, policy_version 1408402 (0.0010) [2023-12-27 01:38:01,656][105692] Updated weights for policy 0, policy_version 1406165 (0.0008) [2023-12-27 01:38:01,709][105692] Updated weights for policy 0, policy_version 1406175 (0.0010) [2023-12-27 01:38:01,778][105692] Updated weights for policy 0, policy_version 1406185 (0.0007) [2023-12-27 01:38:02,308][105620] Updated weights for policy 1, policy_version 1408412 (0.0010) [2023-12-27 01:38:02,374][105620] Updated weights for policy 1, policy_version 1408422 (0.0008) [2023-12-27 01:38:02,418][105692] Updated weights for policy 0, policy_version 1406195 (0.0007) [2023-12-27 01:38:02,428][105620] Updated weights for policy 1, policy_version 1408432 (0.0008) [2023-12-27 01:38:02,480][105692] Updated weights for policy 0, policy_version 1406205 (0.0010) [2023-12-27 01:38:02,541][105692] Updated weights for policy 0, policy_version 1406215 (0.0010) [2023-12-27 01:38:03,140][105692] Updated weights for policy 0, policy_version 1406225 (0.0010) [2023-12-27 01:38:03,160][105620] Updated weights for policy 1, policy_version 1408442 (0.0008) [2023-12-27 01:38:03,200][105692] Updated weights for policy 0, policy_version 1406235 (0.0009) [2023-12-27 01:38:03,213][105620] Updated weights for policy 1, policy_version 1408452 (0.0006) [2023-12-27 01:38:03,260][105692] Updated weights for policy 0, policy_version 1406245 (0.0007) [2023-12-27 01:38:03,277][105620] Updated weights for policy 1, policy_version 1408462 (0.0008) [2023-12-27 01:38:03,315][105692] Updated weights for policy 0, policy_version 1406255 (0.0008) [2023-12-27 01:38:03,342][105620] Updated weights for policy 1, policy_version 1408472 (0.0007) [2023-12-27 01:38:03,869][105692] Updated weights for policy 0, policy_version 1406265 (0.0010) [2023-12-27 01:38:03,928][105692] Updated weights for policy 0, policy_version 1406275 (0.0009) [2023-12-27 01:38:03,994][105692] Updated weights for policy 0, policy_version 1406285 (0.0007) [2023-12-27 01:38:04,047][105620] Updated weights for policy 1, policy_version 1408482 (0.0008) [2023-12-27 01:38:04,114][105620] Updated weights for policy 1, policy_version 1408492 (0.0008) [2023-12-27 01:38:04,179][105620] Updated weights for policy 1, policy_version 1408502 (0.0009) [2023-12-27 01:38:04,608][105692] Updated weights for policy 0, policy_version 1406295 (0.0006) [2023-12-27 01:38:04,669][105692] Updated weights for policy 0, policy_version 1406305 (0.0005) [2023-12-27 01:38:04,736][105692] Updated weights for policy 0, policy_version 1406315 (0.0005) [2023-12-27 01:38:05,043][105620] Updated weights for policy 1, policy_version 1408512 (0.0006) [2023-12-27 01:38:05,098][105620] Updated weights for policy 1, policy_version 1408522 (0.0009) [2023-12-27 01:38:05,152][105620] Updated weights for policy 1, policy_version 1408532 (0.0008) [2023-12-27 01:38:05,304][105692] Updated weights for policy 0, policy_version 1406325 (0.0008) [2023-12-27 01:38:05,362][105692] Updated weights for policy 0, policy_version 1406335 (0.0011) [2023-12-27 01:38:05,416][105692] Updated weights for policy 0, policy_version 1406345 (0.0010) [2023-12-27 01:38:05,945][105620] Updated weights for policy 1, policy_version 1408542 (0.0008) [2023-12-27 01:38:05,995][105620] Updated weights for policy 1, policy_version 1408552 (0.0005) [2023-12-27 01:38:06,053][105692] Updated weights for policy 0, policy_version 1406355 (0.0010) [2023-12-27 01:38:06,058][105620] Updated weights for policy 1, policy_version 1408562 (0.0005) [2023-12-27 01:38:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 720707584. Throughput: 0: 9938.6, 1: 9668.6. Samples: 720702208. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:38:06,063][104569] Avg episode reward: [(0, '7994.243'), (1, '8438.629')] [2023-12-27 01:38:06,113][105692] Updated weights for policy 0, policy_version 1406365 (0.0008) [2023-12-27 01:38:06,175][105692] Updated weights for policy 0, policy_version 1406375 (0.0007) [2023-12-27 01:38:06,740][105620] Updated weights for policy 1, policy_version 1408572 (0.0007) [2023-12-27 01:38:06,801][105620] Updated weights for policy 1, policy_version 1408582 (0.0009) [2023-12-27 01:38:06,862][105620] Updated weights for policy 1, policy_version 1408592 (0.0010) [2023-12-27 01:38:06,942][105692] Updated weights for policy 0, policy_version 1406385 (0.0008) [2023-12-27 01:38:07,011][105692] Updated weights for policy 0, policy_version 1406395 (0.0007) [2023-12-27 01:38:07,075][105692] Updated weights for policy 0, policy_version 1406405 (0.0009) [2023-12-27 01:38:07,138][105692] Updated weights for policy 0, policy_version 1406415 (0.0009) [2023-12-27 01:38:07,694][105620] Updated weights for policy 1, policy_version 1408602 (0.0008) [2023-12-27 01:38:07,756][105620] Updated weights for policy 1, policy_version 1408612 (0.0007) [2023-12-27 01:38:07,766][105692] Updated weights for policy 0, policy_version 1406425 (0.0007) [2023-12-27 01:38:07,794][105585] KL-divergence is very high: 130.9805 [2023-12-27 01:38:07,808][105585] KL-divergence is very high: 184.6157 [2023-12-27 01:38:07,815][105692] Updated weights for policy 0, policy_version 1406435 (0.0005) [2023-12-27 01:38:07,817][105620] Updated weights for policy 1, policy_version 1408622 (0.0008) [2023-12-27 01:38:07,833][105585] KL-divergence is very high: 323.0544 [2023-12-27 01:38:07,849][105585] KL-divergence is very high: 318.6952 [2023-12-27 01:38:07,864][105692] Updated weights for policy 0, policy_version 1406445 (0.0005) [2023-12-27 01:38:07,876][105620] Updated weights for policy 1, policy_version 1408632 (0.0008) [2023-12-27 01:38:07,877][105585] KL-divergence is very high: 420.8770 [2023-12-27 01:38:08,502][105692] Updated weights for policy 0, policy_version 1406455 (0.0009) [2023-12-27 01:38:08,559][105692] Updated weights for policy 0, policy_version 1406465 (0.0009) [2023-12-27 01:38:08,615][105692] Updated weights for policy 0, policy_version 1406476 (0.0009) [2023-12-27 01:38:08,658][105620] Updated weights for policy 1, policy_version 1408642 (0.0009) [2023-12-27 01:38:08,719][105620] Updated weights for policy 1, policy_version 1408652 (0.0009) [2023-12-27 01:38:08,781][105620] Updated weights for policy 1, policy_version 1408662 (0.0009) [2023-12-27 01:38:09,440][105692] Updated weights for policy 0, policy_version 1406486 (0.0008) [2023-12-27 01:38:09,500][105692] Updated weights for policy 0, policy_version 1406496 (0.0008) [2023-12-27 01:38:09,530][105620] Updated weights for policy 1, policy_version 1408672 (0.0008) [2023-12-27 01:38:09,561][105692] Updated weights for policy 0, policy_version 1406506 (0.0008) [2023-12-27 01:38:09,596][105620] Updated weights for policy 1, policy_version 1408682 (0.0007) [2023-12-27 01:38:09,661][105620] Updated weights for policy 1, policy_version 1408692 (0.0008) [2023-12-27 01:38:10,320][105692] Updated weights for policy 0, policy_version 1406516 (0.0008) [2023-12-27 01:38:10,373][105692] Updated weights for policy 0, policy_version 1406526 (0.0009) [2023-12-27 01:38:10,426][105620] Updated weights for policy 1, policy_version 1408702 (0.0007) [2023-12-27 01:38:10,428][105692] Updated weights for policy 0, policy_version 1406536 (0.0008) [2023-12-27 01:38:10,485][105620] Updated weights for policy 1, policy_version 1408712 (0.0007) [2023-12-27 01:38:10,546][105620] Updated weights for policy 1, policy_version 1408722 (0.0009) [2023-12-27 01:38:11,052][105692] Updated weights for policy 0, policy_version 1406546 (0.0007) [2023-12-27 01:38:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 720805888. Throughput: 0: 9954.3, 1: 9596.9. Samples: 720815840. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:38:11,063][104569] Avg episode reward: [(0, '7795.059'), (1, '8257.954')] [2023-12-27 01:38:11,114][105692] Updated weights for policy 0, policy_version 1406556 (0.0007) [2023-12-27 01:38:11,181][105692] Updated weights for policy 0, policy_version 1406567 (0.0010) [2023-12-27 01:38:11,343][105620] Updated weights for policy 1, policy_version 1408732 (0.0008) [2023-12-27 01:38:11,415][105620] Updated weights for policy 1, policy_version 1408742 (0.0008) [2023-12-27 01:38:11,482][105620] Updated weights for policy 1, policy_version 1408752 (0.0010) [2023-12-27 01:38:11,981][105692] Updated weights for policy 0, policy_version 1406577 (0.0009) [2023-12-27 01:38:12,046][105692] Updated weights for policy 0, policy_version 1406587 (0.0009) [2023-12-27 01:38:12,114][105692] Updated weights for policy 0, policy_version 1406597 (0.0007) [2023-12-27 01:38:12,175][105692] Updated weights for policy 0, policy_version 1406607 (0.0009) [2023-12-27 01:38:12,212][105620] Updated weights for policy 1, policy_version 1408762 (0.0009) [2023-12-27 01:38:12,278][105620] Updated weights for policy 1, policy_version 1408772 (0.0009) [2023-12-27 01:38:12,337][105620] Updated weights for policy 1, policy_version 1408782 (0.0009) [2023-12-27 01:38:12,406][105620] Updated weights for policy 1, policy_version 1408792 (0.0008) [2023-12-27 01:38:12,924][105692] Updated weights for policy 0, policy_version 1406617 (0.0006) [2023-12-27 01:38:12,983][105692] Updated weights for policy 0, policy_version 1406627 (0.0008) [2023-12-27 01:38:13,045][105692] Updated weights for policy 0, policy_version 1406637 (0.0009) [2023-12-27 01:38:13,085][105620] Updated weights for policy 1, policy_version 1408802 (0.0007) [2023-12-27 01:38:13,132][105620] Updated weights for policy 1, policy_version 1408812 (0.0009) [2023-12-27 01:38:13,179][105620] Updated weights for policy 1, policy_version 1408822 (0.0009) [2023-12-27 01:38:13,786][105692] Updated weights for policy 0, policy_version 1406647 (0.0008) [2023-12-27 01:38:13,834][105692] Updated weights for policy 0, policy_version 1406657 (0.0009) [2023-12-27 01:38:13,880][105692] Updated weights for policy 0, policy_version 1406667 (0.0008) [2023-12-27 01:38:13,918][105620] Updated weights for policy 1, policy_version 1408832 (0.0008) [2023-12-27 01:38:13,968][105620] Updated weights for policy 1, policy_version 1408842 (0.0009) [2023-12-27 01:38:14,014][105620] Updated weights for policy 1, policy_version 1408852 (0.0008) [2023-12-27 01:38:14,646][105692] Updated weights for policy 0, policy_version 1406677 (0.0007) [2023-12-27 01:38:14,696][105620] Updated weights for policy 1, policy_version 1408862 (0.0010) [2023-12-27 01:38:14,707][105692] Updated weights for policy 0, policy_version 1406687 (0.0005) [2023-12-27 01:38:14,751][105620] Updated weights for policy 1, policy_version 1408872 (0.0011) [2023-12-27 01:38:14,759][105692] Updated weights for policy 0, policy_version 1406697 (0.0005) [2023-12-27 01:38:14,817][105620] Updated weights for policy 1, policy_version 1408882 (0.0010) [2023-12-27 01:38:15,339][105692] Updated weights for policy 0, policy_version 1406707 (0.0007) [2023-12-27 01:38:15,405][105692] Updated weights for policy 0, policy_version 1406717 (0.0007) [2023-12-27 01:38:15,471][105692] Updated weights for policy 0, policy_version 1406727 (0.0008) [2023-12-27 01:38:15,539][105620] Updated weights for policy 1, policy_version 1408892 (0.0011) [2023-12-27 01:38:15,603][105620] Updated weights for policy 1, policy_version 1408902 (0.0011) [2023-12-27 01:38:15,670][105620] Updated weights for policy 1, policy_version 1408912 (0.0011) [2023-12-27 01:38:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 720904192. Throughput: 0: 9907.6, 1: 9636.0. Samples: 720873412. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:38:16,063][104569] Avg episode reward: [(0, '8530.090'), (1, '8807.096')] [2023-12-27 01:38:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001408920_360726528.pth... [2023-12-27 01:38:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001406736_360177664.pth... [2023-12-27 01:38:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001407800_360439808.pth [2023-12-27 01:38:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001405584_359882752.pth [2023-12-27 01:38:16,110][105692] Updated weights for policy 0, policy_version 1406737 (0.0008) [2023-12-27 01:38:16,172][105692] Updated weights for policy 0, policy_version 1406747 (0.0010) [2023-12-27 01:38:16,220][105692] Updated weights for policy 0, policy_version 1406757 (0.0010) [2023-12-27 01:38:16,265][105692] Updated weights for policy 0, policy_version 1406767 (0.0010) [2023-12-27 01:38:16,425][105620] Updated weights for policy 1, policy_version 1408922 (0.0011) [2023-12-27 01:38:16,479][105620] Updated weights for policy 1, policy_version 1408932 (0.0011) [2023-12-27 01:38:16,528][105620] Updated weights for policy 1, policy_version 1408942 (0.0005) [2023-12-27 01:38:16,586][105620] Updated weights for policy 1, policy_version 1408952 (0.0006) [2023-12-27 01:38:16,927][105692] Updated weights for policy 0, policy_version 1406777 (0.0006) [2023-12-27 01:38:16,993][105692] Updated weights for policy 0, policy_version 1406787 (0.0007) [2023-12-27 01:38:17,045][105692] Updated weights for policy 0, policy_version 1406797 (0.0010) [2023-12-27 01:38:17,254][105620] Updated weights for policy 1, policy_version 1408962 (0.0005) [2023-12-27 01:38:17,308][105620] Updated weights for policy 1, policy_version 1408972 (0.0008) [2023-12-27 01:38:17,367][105620] Updated weights for policy 1, policy_version 1408982 (0.0010) [2023-12-27 01:38:17,664][105692] Updated weights for policy 0, policy_version 1406807 (0.0007) [2023-12-27 01:38:17,719][105692] Updated weights for policy 0, policy_version 1406817 (0.0005) [2023-12-27 01:38:17,774][105692] Updated weights for policy 0, policy_version 1406827 (0.0005) [2023-12-27 01:38:18,210][105620] Updated weights for policy 1, policy_version 1408992 (0.0009) [2023-12-27 01:38:18,262][105620] Updated weights for policy 1, policy_version 1409002 (0.0010) [2023-12-27 01:38:18,288][105692] Updated weights for policy 0, policy_version 1406837 (0.0006) [2023-12-27 01:38:18,318][105620] Updated weights for policy 1, policy_version 1409012 (0.0008) [2023-12-27 01:38:18,349][105692] Updated weights for policy 0, policy_version 1406847 (0.0007) [2023-12-27 01:38:18,415][105692] Updated weights for policy 0, policy_version 1406857 (0.0005) [2023-12-27 01:38:19,021][105620] Updated weights for policy 1, policy_version 1409022 (0.0007) [2023-12-27 01:38:19,059][105692] Updated weights for policy 0, policy_version 1406867 (0.0009) [2023-12-27 01:38:19,079][105620] Updated weights for policy 1, policy_version 1409032 (0.0006) [2023-12-27 01:38:19,116][105692] Updated weights for policy 0, policy_version 1406877 (0.0009) [2023-12-27 01:38:19,148][105620] Updated weights for policy 1, policy_version 1409042 (0.0005) [2023-12-27 01:38:19,172][105692] Updated weights for policy 0, policy_version 1406887 (0.0010) [2023-12-27 01:38:19,828][105692] Updated weights for policy 0, policy_version 1406897 (0.0010) [2023-12-27 01:38:19,894][105620] Updated weights for policy 1, policy_version 1409052 (0.0007) [2023-12-27 01:38:19,896][105692] Updated weights for policy 0, policy_version 1406907 (0.0007) [2023-12-27 01:38:19,955][105620] Updated weights for policy 1, policy_version 1409062 (0.0008) [2023-12-27 01:38:19,962][105692] Updated weights for policy 0, policy_version 1406917 (0.0007) [2023-12-27 01:38:20,019][105620] Updated weights for policy 1, policy_version 1409072 (0.0008) [2023-12-27 01:38:20,027][105692] Updated weights for policy 0, policy_version 1406927 (0.0007) [2023-12-27 01:38:20,677][105620] Updated weights for policy 1, policy_version 1409082 (0.0007) [2023-12-27 01:38:20,735][105620] Updated weights for policy 1, policy_version 1409092 (0.0010) [2023-12-27 01:38:20,777][105692] Updated weights for policy 0, policy_version 1406937 (0.0006) [2023-12-27 01:38:20,792][105620] Updated weights for policy 1, policy_version 1409102 (0.0009) [2023-12-27 01:38:20,844][105692] Updated weights for policy 0, policy_version 1406947 (0.0006) [2023-12-27 01:38:20,862][105620] Updated weights for policy 1, policy_version 1409112 (0.0009) [2023-12-27 01:38:20,898][105692] Updated weights for policy 0, policy_version 1406957 (0.0005) [2023-12-27 01:38:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 721010688. Throughput: 0: 9904.8, 1: 9615.5. Samples: 720994896. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:38:21,062][104569] Avg episode reward: [(0, '7972.531'), (1, '9084.139')] [2023-12-27 01:38:21,597][105692] Updated weights for policy 0, policy_version 1406967 (0.0006) [2023-12-27 01:38:21,600][105620] Updated weights for policy 1, policy_version 1409122 (0.0006) [2023-12-27 01:38:21,666][105692] Updated weights for policy 0, policy_version 1406977 (0.0008) [2023-12-27 01:38:21,669][105620] Updated weights for policy 1, policy_version 1409132 (0.0008) [2023-12-27 01:38:21,731][105620] Updated weights for policy 1, policy_version 1409142 (0.0008) [2023-12-27 01:38:21,733][105692] Updated weights for policy 0, policy_version 1406987 (0.0008) [2023-12-27 01:38:22,454][105692] Updated weights for policy 0, policy_version 1406997 (0.0008) [2023-12-27 01:38:22,503][105692] Updated weights for policy 0, policy_version 1407007 (0.0007) [2023-12-27 01:38:22,508][105620] Updated weights for policy 1, policy_version 1409152 (0.0007) [2023-12-27 01:38:22,567][105692] Updated weights for policy 0, policy_version 1407017 (0.0007) [2023-12-27 01:38:22,569][105620] Updated weights for policy 1, policy_version 1409162 (0.0007) [2023-12-27 01:38:22,623][105620] Updated weights for policy 1, policy_version 1409172 (0.0006) [2023-12-27 01:38:23,336][105692] Updated weights for policy 0, policy_version 1407027 (0.0009) [2023-12-27 01:38:23,376][105620] Updated weights for policy 1, policy_version 1409182 (0.0006) [2023-12-27 01:38:23,394][105692] Updated weights for policy 0, policy_version 1407037 (0.0008) [2023-12-27 01:38:23,428][105620] Updated weights for policy 1, policy_version 1409192 (0.0006) [2023-12-27 01:38:23,447][105692] Updated weights for policy 0, policy_version 1407047 (0.0006) [2023-12-27 01:38:23,485][105620] Updated weights for policy 1, policy_version 1409202 (0.0007) [2023-12-27 01:38:24,223][105692] Updated weights for policy 0, policy_version 1407057 (0.0007) [2023-12-27 01:38:24,260][105620] Updated weights for policy 1, policy_version 1409212 (0.0009) [2023-12-27 01:38:24,289][105692] Updated weights for policy 0, policy_version 1407067 (0.0006) [2023-12-27 01:38:24,322][105620] Updated weights for policy 1, policy_version 1409222 (0.0010) [2023-12-27 01:38:24,352][105692] Updated weights for policy 0, policy_version 1407077 (0.0007) [2023-12-27 01:38:24,385][105620] Updated weights for policy 1, policy_version 1409232 (0.0011) [2023-12-27 01:38:24,411][105692] Updated weights for policy 0, policy_version 1407087 (0.0005) [2023-12-27 01:38:25,048][105620] Updated weights for policy 1, policy_version 1409242 (0.0009) [2023-12-27 01:38:25,103][105620] Updated weights for policy 1, policy_version 1409252 (0.0005) [2023-12-27 01:38:25,161][105620] Updated weights for policy 1, policy_version 1409262 (0.0005) [2023-12-27 01:38:25,193][105692] Updated weights for policy 0, policy_version 1407097 (0.0009) [2023-12-27 01:38:25,216][105620] Updated weights for policy 1, policy_version 1409272 (0.0005) [2023-12-27 01:38:25,249][105692] Updated weights for policy 0, policy_version 1407108 (0.0010) [2023-12-27 01:38:25,304][105692] Updated weights for policy 0, policy_version 1407120 (0.0010) [2023-12-27 01:38:25,743][105620] Updated weights for policy 1, policy_version 1409282 (0.0008) [2023-12-27 01:38:25,799][105620] Updated weights for policy 1, policy_version 1409292 (0.0007) [2023-12-27 01:38:25,859][105620] Updated weights for policy 1, policy_version 1409302 (0.0010) [2023-12-27 01:38:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 721100800. Throughput: 0: 9837.7, 1: 9651.9. Samples: 721109912. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:38:26,063][104569] Avg episode reward: [(0, '7789.179'), (1, '8811.700')] [2023-12-27 01:38:26,152][105585] KL-divergence is very high: 136.9780 [2023-12-27 01:38:26,159][105585] KL-divergence is very high: 194.9541 [2023-12-27 01:38:26,172][105692] Updated weights for policy 0, policy_version 1407130 (0.0008) [2023-12-27 01:38:26,186][105585] KL-divergence is very high: 111.3391 [2023-12-27 01:38:26,206][105585] KL-divergence is very high: 272.1253 [2023-12-27 01:38:26,214][105585] KL-divergence is very high: 352.9018 [2023-12-27 01:38:26,240][105585] KL-divergence is very high: 131.0630 [2023-12-27 01:38:26,241][105692] Updated weights for policy 0, policy_version 1407140 (0.0009) [2023-12-27 01:38:26,260][105585] KL-divergence is very high: 290.8222 [2023-12-27 01:38:26,267][105585] KL-divergence is very high: 375.8643 [2023-12-27 01:38:26,296][105585] KL-divergence is very high: 112.1289 [2023-12-27 01:38:26,309][105692] Updated weights for policy 0, policy_version 1407150 (0.0010) [2023-12-27 01:38:26,316][105585] KL-divergence is very high: 252.2328 [2023-12-27 01:38:26,475][105620] Updated weights for policy 1, policy_version 1409312 (0.0007) [2023-12-27 01:38:26,528][105620] Updated weights for policy 1, policy_version 1409322 (0.0006) [2023-12-27 01:38:26,583][105620] Updated weights for policy 1, policy_version 1409332 (0.0005) [2023-12-27 01:38:26,918][105692] Updated weights for policy 0, policy_version 1407160 (0.0009) [2023-12-27 01:38:26,981][105692] Updated weights for policy 0, policy_version 1407170 (0.0007) [2023-12-27 01:38:27,047][105692] Updated weights for policy 0, policy_version 1407180 (0.0008) [2023-12-27 01:38:27,192][105620] Updated weights for policy 1, policy_version 1409342 (0.0010) [2023-12-27 01:38:27,249][105620] Updated weights for policy 1, policy_version 1409352 (0.0010) [2023-12-27 01:38:27,305][105620] Updated weights for policy 1, policy_version 1409362 (0.0010) [2023-12-27 01:38:27,757][105692] Updated weights for policy 0, policy_version 1407190 (0.0008) [2023-12-27 01:38:27,826][105692] Updated weights for policy 0, policy_version 1407200 (0.0009) [2023-12-27 01:38:27,893][105692] Updated weights for policy 0, policy_version 1407210 (0.0009) [2023-12-27 01:38:28,073][105620] Updated weights for policy 1, policy_version 1409372 (0.0010) [2023-12-27 01:38:28,126][105620] Updated weights for policy 1, policy_version 1409382 (0.0010) [2023-12-27 01:38:28,173][105620] Updated weights for policy 1, policy_version 1409392 (0.0010) [2023-12-27 01:38:28,648][105692] Updated weights for policy 0, policy_version 1407220 (0.0008) [2023-12-27 01:38:28,696][105692] Updated weights for policy 0, policy_version 1407230 (0.0008) [2023-12-27 01:38:28,756][105692] Updated weights for policy 0, policy_version 1407240 (0.0008) [2023-12-27 01:38:28,932][105620] Updated weights for policy 1, policy_version 1409402 (0.0010) [2023-12-27 01:38:28,975][105620] Updated weights for policy 1, policy_version 1409412 (0.0009) [2023-12-27 01:38:29,022][105620] Updated weights for policy 1, policy_version 1409422 (0.0009) [2023-12-27 01:38:29,066][105620] Updated weights for policy 1, policy_version 1409432 (0.0010) [2023-12-27 01:38:29,507][105692] Updated weights for policy 0, policy_version 1407250 (0.0008) [2023-12-27 01:38:29,567][105692] Updated weights for policy 0, policy_version 1407260 (0.0007) [2023-12-27 01:38:29,633][105692] Updated weights for policy 0, policy_version 1407270 (0.0007) [2023-12-27 01:38:29,684][105692] Updated weights for policy 0, policy_version 1407280 (0.0008) [2023-12-27 01:38:29,863][105620] Updated weights for policy 1, policy_version 1409442 (0.0011) [2023-12-27 01:38:29,925][105620] Updated weights for policy 1, policy_version 1409452 (0.0010) [2023-12-27 01:38:29,992][105620] Updated weights for policy 1, policy_version 1409462 (0.0010) [2023-12-27 01:38:30,369][105692] Updated weights for policy 0, policy_version 1407290 (0.0011) [2023-12-27 01:38:30,417][105692] Updated weights for policy 0, policy_version 1407300 (0.0010) [2023-12-27 01:38:30,465][105692] Updated weights for policy 0, policy_version 1407310 (0.0010) [2023-12-27 01:38:30,694][105620] Updated weights for policy 1, policy_version 1409472 (0.0006) [2023-12-27 01:38:30,760][105620] Updated weights for policy 1, policy_version 1409482 (0.0007) [2023-12-27 01:38:30,820][105620] Updated weights for policy 1, policy_version 1409492 (0.0008) [2023-12-27 01:38:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 721199104. Throughput: 0: 9851.5, 1: 9680.3. Samples: 721169156. Policy #0 lag: (min: 5.0, avg: 6.7, max: 24.0) [2023-12-27 01:38:31,062][104569] Avg episode reward: [(0, '8163.333'), (1, '8718.002')] [2023-12-27 01:38:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001407312_360325120.pth... [2023-12-27 01:38:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001409496_360873984.pth... [2023-12-27 01:38:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001406160_360030208.pth [2023-12-27 01:38:31,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001408376_360587264.pth [2023-12-27 01:38:31,230][105692] Updated weights for policy 0, policy_version 1407320 (0.0007) [2023-12-27 01:38:31,299][105692] Updated weights for policy 0, policy_version 1407330 (0.0006) [2023-12-27 01:38:31,373][105692] Updated weights for policy 0, policy_version 1407340 (0.0008) [2023-12-27 01:38:31,426][105620] Updated weights for policy 1, policy_version 1409502 (0.0006) [2023-12-27 01:38:31,473][105620] Updated weights for policy 1, policy_version 1409512 (0.0005) [2023-12-27 01:38:31,522][105620] Updated weights for policy 1, policy_version 1409522 (0.0006) [2023-12-27 01:38:32,013][105692] Updated weights for policy 0, policy_version 1407350 (0.0009) [2023-12-27 01:38:32,067][105692] Updated weights for policy 0, policy_version 1407362 (0.0010) [2023-12-27 01:38:32,129][105692] Updated weights for policy 0, policy_version 1407372 (0.0009) [2023-12-27 01:38:32,148][105620] Updated weights for policy 1, policy_version 1409532 (0.0008) [2023-12-27 01:38:32,193][105620] Updated weights for policy 1, policy_version 1409542 (0.0008) [2023-12-27 01:38:32,248][105620] Updated weights for policy 1, policy_version 1409552 (0.0006) [2023-12-27 01:38:32,902][105692] Updated weights for policy 0, policy_version 1407382 (0.0009) [2023-12-27 01:38:32,947][105620] Updated weights for policy 1, policy_version 1409562 (0.0006) [2023-12-27 01:38:32,957][105692] Updated weights for policy 0, policy_version 1407392 (0.0010) [2023-12-27 01:38:32,998][105620] Updated weights for policy 1, policy_version 1409572 (0.0005) [2023-12-27 01:38:33,015][105692] Updated weights for policy 0, policy_version 1407402 (0.0011) [2023-12-27 01:38:33,050][105620] Updated weights for policy 1, policy_version 1409582 (0.0005) [2023-12-27 01:38:33,100][105620] Updated weights for policy 1, policy_version 1409592 (0.0008) [2023-12-27 01:38:33,763][105692] Updated weights for policy 0, policy_version 1407412 (0.0011) [2023-12-27 01:38:33,764][105620] Updated weights for policy 1, policy_version 1409602 (0.0008) [2023-12-27 01:38:33,809][105620] Updated weights for policy 1, policy_version 1409612 (0.0006) [2023-12-27 01:38:33,814][105692] Updated weights for policy 0, policy_version 1407422 (0.0010) [2023-12-27 01:38:33,857][105620] Updated weights for policy 1, policy_version 1409622 (0.0006) [2023-12-27 01:38:33,862][105692] Updated weights for policy 0, policy_version 1407432 (0.0010) [2023-12-27 01:38:34,516][105620] Updated weights for policy 1, policy_version 1409632 (0.0007) [2023-12-27 01:38:34,577][105620] Updated weights for policy 1, policy_version 1409642 (0.0008) [2023-12-27 01:38:34,637][105620] Updated weights for policy 1, policy_version 1409652 (0.0009) [2023-12-27 01:38:34,699][105692] Updated weights for policy 0, policy_version 1407442 (0.0009) [2023-12-27 01:38:34,750][105692] Updated weights for policy 0, policy_version 1407452 (0.0005) [2023-12-27 01:38:34,797][105692] Updated weights for policy 0, policy_version 1407462 (0.0005) [2023-12-27 01:38:34,849][105692] Updated weights for policy 0, policy_version 1407472 (0.0006) [2023-12-27 01:38:35,445][105620] Updated weights for policy 1, policy_version 1409662 (0.0010) [2023-12-27 01:38:35,456][105692] Updated weights for policy 0, policy_version 1407482 (0.0005) [2023-12-27 01:38:35,490][105620] Updated weights for policy 1, policy_version 1409672 (0.0010) [2023-12-27 01:38:35,500][105692] Updated weights for policy 0, policy_version 1407492 (0.0006) [2023-12-27 01:38:35,542][105620] Updated weights for policy 1, policy_version 1409682 (0.0011) [2023-12-27 01:38:35,554][105586] KL-divergence is very high: 110.5294 [2023-12-27 01:38:35,556][105692] Updated weights for policy 0, policy_version 1407502 (0.0005) [2023-12-27 01:38:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 721297408. Throughput: 0: 9825.6, 1: 9714.7. Samples: 721287576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:38:36,063][104569] Avg episode reward: [(0, '8624.402'), (1, '8809.503')] [2023-12-27 01:38:36,318][105620] Updated weights for policy 1, policy_version 1409692 (0.0011) [2023-12-27 01:38:36,352][105692] Updated weights for policy 0, policy_version 1407512 (0.0005) [2023-12-27 01:38:36,384][105620] Updated weights for policy 1, policy_version 1409702 (0.0011) [2023-12-27 01:38:36,413][105692] Updated weights for policy 0, policy_version 1407522 (0.0009) [2023-12-27 01:38:36,441][105620] Updated weights for policy 1, policy_version 1409712 (0.0011) [2023-12-27 01:38:36,469][105692] Updated weights for policy 0, policy_version 1407532 (0.0010) [2023-12-27 01:38:37,123][105620] Updated weights for policy 1, policy_version 1409722 (0.0010) [2023-12-27 01:38:37,185][105620] Updated weights for policy 1, policy_version 1409732 (0.0010) [2023-12-27 01:38:37,227][105692] Updated weights for policy 0, policy_version 1407542 (0.0006) [2023-12-27 01:38:37,252][105620] Updated weights for policy 1, policy_version 1409742 (0.0007) [2023-12-27 01:38:37,279][105692] Updated weights for policy 0, policy_version 1407552 (0.0005) [2023-12-27 01:38:37,303][105620] Updated weights for policy 1, policy_version 1409752 (0.0008) [2023-12-27 01:38:37,352][105692] Updated weights for policy 0, policy_version 1407562 (0.0005) [2023-12-27 01:38:37,882][105620] Updated weights for policy 1, policy_version 1409762 (0.0010) [2023-12-27 01:38:37,912][105692] Updated weights for policy 0, policy_version 1407572 (0.0007) [2023-12-27 01:38:37,940][105620] Updated weights for policy 1, policy_version 1409772 (0.0010) [2023-12-27 01:38:37,963][105692] Updated weights for policy 0, policy_version 1407582 (0.0006) [2023-12-27 01:38:38,000][105620] Updated weights for policy 1, policy_version 1409782 (0.0009) [2023-12-27 01:38:38,028][105692] Updated weights for policy 0, policy_version 1407592 (0.0007) [2023-12-27 01:38:38,766][105692] Updated weights for policy 0, policy_version 1407602 (0.0007) [2023-12-27 01:38:38,786][105620] Updated weights for policy 1, policy_version 1409792 (0.0006) [2023-12-27 01:38:38,818][105692] Updated weights for policy 0, policy_version 1407612 (0.0009) [2023-12-27 01:38:38,845][105620] Updated weights for policy 1, policy_version 1409802 (0.0005) [2023-12-27 01:38:38,871][105692] Updated weights for policy 0, policy_version 1407622 (0.0008) [2023-12-27 01:38:38,898][105620] Updated weights for policy 1, policy_version 1409812 (0.0005) [2023-12-27 01:38:39,598][105620] Updated weights for policy 1, policy_version 1409822 (0.0009) [2023-12-27 01:38:39,614][105692] Updated weights for policy 0, policy_version 1407633 (0.0008) [2023-12-27 01:38:39,654][105620] Updated weights for policy 1, policy_version 1409832 (0.0010) [2023-12-27 01:38:39,668][105692] Updated weights for policy 0, policy_version 1407643 (0.0007) [2023-12-27 01:38:39,721][105620] Updated weights for policy 1, policy_version 1409842 (0.0011) [2023-12-27 01:38:39,724][105692] Updated weights for policy 0, policy_version 1407653 (0.0008) [2023-12-27 01:38:39,783][105692] Updated weights for policy 0, policy_version 1407663 (0.0009) [2023-12-27 01:38:40,449][105620] Updated weights for policy 1, policy_version 1409852 (0.0010) [2023-12-27 01:38:40,513][105620] Updated weights for policy 1, policy_version 1409862 (0.0009) [2023-12-27 01:38:40,554][105692] Updated weights for policy 0, policy_version 1407673 (0.0006) [2023-12-27 01:38:40,580][105620] Updated weights for policy 1, policy_version 1409872 (0.0009) [2023-12-27 01:38:40,611][105692] Updated weights for policy 0, policy_version 1407683 (0.0006) [2023-12-27 01:38:40,680][105692] Updated weights for policy 0, policy_version 1407693 (0.0009) [2023-12-27 01:38:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 721395712. Throughput: 0: 9830.3, 1: 9671.7. Samples: 721404972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:38:41,062][104569] Avg episode reward: [(0, '8715.851'), (1, '8899.379')] [2023-12-27 01:38:41,229][105620] Updated weights for policy 1, policy_version 1409882 (0.0005) [2023-12-27 01:38:41,298][105620] Updated weights for policy 1, policy_version 1409892 (0.0008) [2023-12-27 01:38:41,364][105620] Updated weights for policy 1, policy_version 1409902 (0.0009) [2023-12-27 01:38:41,431][105620] Updated weights for policy 1, policy_version 1409912 (0.0009) [2023-12-27 01:38:41,471][105692] Updated weights for policy 0, policy_version 1407703 (0.0008) [2023-12-27 01:38:41,534][105692] Updated weights for policy 0, policy_version 1407713 (0.0009) [2023-12-27 01:38:41,596][105692] Updated weights for policy 0, policy_version 1407723 (0.0009) [2023-12-27 01:38:42,138][105620] Updated weights for policy 1, policy_version 1409922 (0.0007) [2023-12-27 01:38:42,206][105620] Updated weights for policy 1, policy_version 1409932 (0.0008) [2023-12-27 01:38:42,270][105620] Updated weights for policy 1, policy_version 1409942 (0.0007) [2023-12-27 01:38:42,386][105692] Updated weights for policy 0, policy_version 1407733 (0.0009) [2023-12-27 01:38:42,448][105692] Updated weights for policy 0, policy_version 1407743 (0.0010) [2023-12-27 01:38:42,507][105692] Updated weights for policy 0, policy_version 1407753 (0.0009) [2023-12-27 01:38:42,940][105620] Updated weights for policy 1, policy_version 1409952 (0.0010) [2023-12-27 01:38:42,997][105620] Updated weights for policy 1, policy_version 1409962 (0.0009) [2023-12-27 01:38:43,060][105620] Updated weights for policy 1, policy_version 1409972 (0.0009) [2023-12-27 01:38:43,092][105692] Updated weights for policy 0, policy_version 1407763 (0.0005) [2023-12-27 01:38:43,156][105692] Updated weights for policy 0, policy_version 1407773 (0.0009) [2023-12-27 01:38:43,206][105692] Updated weights for policy 0, policy_version 1407783 (0.0009) [2023-12-27 01:38:43,735][105620] Updated weights for policy 1, policy_version 1409982 (0.0009) [2023-12-27 01:38:43,786][105620] Updated weights for policy 1, policy_version 1409992 (0.0009) [2023-12-27 01:38:43,833][105620] Updated weights for policy 1, policy_version 1410002 (0.0009) [2023-12-27 01:38:43,982][105692] Updated weights for policy 0, policy_version 1407793 (0.0010) [2023-12-27 01:38:44,044][105692] Updated weights for policy 0, policy_version 1407803 (0.0009) [2023-12-27 01:38:44,102][105692] Updated weights for policy 0, policy_version 1407813 (0.0010) [2023-12-27 01:38:44,159][105692] Updated weights for policy 0, policy_version 1407824 (0.0010) [2023-12-27 01:38:44,503][105620] Updated weights for policy 1, policy_version 1410012 (0.0008) [2023-12-27 01:38:44,554][105620] Updated weights for policy 1, policy_version 1410022 (0.0005) [2023-12-27 01:38:44,610][105620] Updated weights for policy 1, policy_version 1410032 (0.0005) [2023-12-27 01:38:44,979][105692] Updated weights for policy 0, policy_version 1407834 (0.0008) [2023-12-27 01:38:45,044][105692] Updated weights for policy 0, policy_version 1407844 (0.0007) [2023-12-27 01:38:45,100][105692] Updated weights for policy 0, policy_version 1407854 (0.0006) [2023-12-27 01:38:45,264][105620] Updated weights for policy 1, policy_version 1410042 (0.0006) [2023-12-27 01:38:45,326][105620] Updated weights for policy 1, policy_version 1410052 (0.0008) [2023-12-27 01:38:45,386][105620] Updated weights for policy 1, policy_version 1410062 (0.0008) [2023-12-27 01:38:45,446][105620] Updated weights for policy 1, policy_version 1410072 (0.0011) [2023-12-27 01:38:45,864][105692] Updated weights for policy 0, policy_version 1407864 (0.0009) [2023-12-27 01:38:45,921][105692] Updated weights for policy 0, policy_version 1407874 (0.0009) [2023-12-27 01:38:45,978][105692] Updated weights for policy 0, policy_version 1407884 (0.0006) [2023-12-27 01:38:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 721494016. Throughput: 0: 9849.0, 1: 9665.2. Samples: 721462060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:38:46,063][104569] Avg episode reward: [(0, '8621.583'), (1, '8714.924')] [2023-12-27 01:38:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001407888_360472576.pth... [2023-12-27 01:38:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001406736_360177664.pth [2023-12-27 01:38:46,113][105620] Updated weights for policy 1, policy_version 1410082 (0.0010) [2023-12-27 01:38:46,165][105620] Updated weights for policy 1, policy_version 1410092 (0.0009) [2023-12-27 01:38:46,222][105620] Updated weights for policy 1, policy_version 1410102 (0.0009) [2023-12-27 01:38:46,232][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001410104_361029632.pth... [2023-12-27 01:38:46,236][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001408920_360726528.pth [2023-12-27 01:38:46,610][105692] Updated weights for policy 0, policy_version 1407894 (0.0005) [2023-12-27 01:38:46,656][105692] Updated weights for policy 0, policy_version 1407904 (0.0006) [2023-12-27 01:38:46,706][105692] Updated weights for policy 0, policy_version 1407914 (0.0009) [2023-12-27 01:38:47,038][105620] Updated weights for policy 1, policy_version 1410112 (0.0009) [2023-12-27 01:38:47,099][105620] Updated weights for policy 1, policy_version 1410122 (0.0006) [2023-12-27 01:38:47,155][105620] Updated weights for policy 1, policy_version 1410132 (0.0005) [2023-12-27 01:38:47,449][105692] Updated weights for policy 0, policy_version 1407924 (0.0008) [2023-12-27 01:38:47,507][105692] Updated weights for policy 0, policy_version 1407934 (0.0005) [2023-12-27 01:38:47,576][105692] Updated weights for policy 0, policy_version 1407944 (0.0006) [2023-12-27 01:38:47,752][105620] Updated weights for policy 1, policy_version 1410142 (0.0006) [2023-12-27 01:38:47,822][105620] Updated weights for policy 1, policy_version 1410152 (0.0006) [2023-12-27 01:38:47,884][105620] Updated weights for policy 1, policy_version 1410162 (0.0008) [2023-12-27 01:38:48,196][105692] Updated weights for policy 0, policy_version 1407954 (0.0006) [2023-12-27 01:38:48,247][105692] Updated weights for policy 0, policy_version 1407964 (0.0008) [2023-12-27 01:38:48,312][105692] Updated weights for policy 0, policy_version 1407974 (0.0009) [2023-12-27 01:38:48,366][105692] Updated weights for policy 0, policy_version 1407984 (0.0008) [2023-12-27 01:38:48,510][105620] Updated weights for policy 1, policy_version 1410172 (0.0009) [2023-12-27 01:38:48,572][105620] Updated weights for policy 1, policy_version 1410182 (0.0008) [2023-12-27 01:38:48,639][105620] Updated weights for policy 1, policy_version 1410192 (0.0008) [2023-12-27 01:38:49,168][105692] Updated weights for policy 0, policy_version 1407994 (0.0005) [2023-12-27 01:38:49,222][105692] Updated weights for policy 0, policy_version 1408004 (0.0009) [2023-12-27 01:38:49,285][105692] Updated weights for policy 0, policy_version 1408014 (0.0007) [2023-12-27 01:38:49,382][105620] Updated weights for policy 1, policy_version 1410202 (0.0007) [2023-12-27 01:38:49,449][105620] Updated weights for policy 1, policy_version 1410212 (0.0007) [2023-12-27 01:38:49,514][105620] Updated weights for policy 1, policy_version 1410222 (0.0010) [2023-12-27 01:38:49,565][105620] Updated weights for policy 1, policy_version 1410232 (0.0009) [2023-12-27 01:38:49,959][105692] Updated weights for policy 0, policy_version 1408024 (0.0008) [2023-12-27 01:38:50,013][105692] Updated weights for policy 0, policy_version 1408034 (0.0009) [2023-12-27 01:38:50,071][105692] Updated weights for policy 0, policy_version 1408044 (0.0010) [2023-12-27 01:38:50,293][105620] Updated weights for policy 1, policy_version 1410242 (0.0006) [2023-12-27 01:38:50,349][105620] Updated weights for policy 1, policy_version 1410252 (0.0009) [2023-12-27 01:38:50,400][105620] Updated weights for policy 1, policy_version 1410262 (0.0009) [2023-12-27 01:38:50,863][105692] Updated weights for policy 0, policy_version 1408054 (0.0009) [2023-12-27 01:38:50,926][105692] Updated weights for policy 0, policy_version 1408064 (0.0010) [2023-12-27 01:38:50,980][105692] Updated weights for policy 0, policy_version 1408074 (0.0010) [2023-12-27 01:38:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 721592320. Throughput: 0: 9747.8, 1: 9775.3. Samples: 721580744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:38:51,062][104569] Avg episode reward: [(0, '8260.821'), (1, '8623.122')] [2023-12-27 01:38:51,088][105620] Updated weights for policy 1, policy_version 1410272 (0.0008) [2023-12-27 01:38:51,146][105620] Updated weights for policy 1, policy_version 1410282 (0.0009) [2023-12-27 01:38:51,209][105620] Updated weights for policy 1, policy_version 1410292 (0.0008) [2023-12-27 01:38:51,785][105692] Updated weights for policy 0, policy_version 1408084 (0.0007) [2023-12-27 01:38:51,849][105692] Updated weights for policy 0, policy_version 1408094 (0.0005) [2023-12-27 01:38:51,919][105692] Updated weights for policy 0, policy_version 1408104 (0.0005) [2023-12-27 01:38:51,998][105620] Updated weights for policy 1, policy_version 1410302 (0.0009) [2023-12-27 01:38:52,062][105620] Updated weights for policy 1, policy_version 1410312 (0.0010) [2023-12-27 01:38:52,126][105620] Updated weights for policy 1, policy_version 1410322 (0.0009) [2023-12-27 01:38:52,521][105692] Updated weights for policy 0, policy_version 1408114 (0.0007) [2023-12-27 01:38:52,585][105692] Updated weights for policy 0, policy_version 1408124 (0.0011) [2023-12-27 01:38:52,644][105692] Updated weights for policy 0, policy_version 1408134 (0.0011) [2023-12-27 01:38:52,713][105692] Updated weights for policy 0, policy_version 1408144 (0.0011) [2023-12-27 01:38:52,866][105620] Updated weights for policy 1, policy_version 1410332 (0.0010) [2023-12-27 01:38:52,915][105620] Updated weights for policy 1, policy_version 1410342 (0.0010) [2023-12-27 01:38:52,984][105620] Updated weights for policy 1, policy_version 1410352 (0.0010) [2023-12-27 01:38:53,361][105692] Updated weights for policy 0, policy_version 1408154 (0.0006) [2023-12-27 01:38:53,414][105692] Updated weights for policy 0, policy_version 1408164 (0.0006) [2023-12-27 01:38:53,467][105692] Updated weights for policy 0, policy_version 1408174 (0.0005) [2023-12-27 01:38:53,726][105620] Updated weights for policy 1, policy_version 1410362 (0.0010) [2023-12-27 01:38:53,794][105620] Updated weights for policy 1, policy_version 1410372 (0.0010) [2023-12-27 01:38:53,855][105620] Updated weights for policy 1, policy_version 1410382 (0.0010) [2023-12-27 01:38:53,906][105620] Updated weights for policy 1, policy_version 1410392 (0.0010) [2023-12-27 01:38:54,062][105692] Updated weights for policy 0, policy_version 1408184 (0.0005) [2023-12-27 01:38:54,124][105692] Updated weights for policy 0, policy_version 1408194 (0.0006) [2023-12-27 01:38:54,171][105692] Updated weights for policy 0, policy_version 1408204 (0.0005) [2023-12-27 01:38:54,632][105620] Updated weights for policy 1, policy_version 1410402 (0.0006) [2023-12-27 01:38:54,707][105620] Updated weights for policy 1, policy_version 1410412 (0.0007) [2023-12-27 01:38:54,766][105620] Updated weights for policy 1, policy_version 1410422 (0.0007) [2023-12-27 01:38:54,767][105692] Updated weights for policy 0, policy_version 1408214 (0.0010) [2023-12-27 01:38:54,829][105692] Updated weights for policy 0, policy_version 1408224 (0.0010) [2023-12-27 01:38:54,856][105585] KL-divergence is very high: 107.0297 [2023-12-27 01:38:54,888][105692] Updated weights for policy 0, policy_version 1408234 (0.0010) [2023-12-27 01:38:54,897][105585] KL-divergence is very high: 116.8319 [2023-12-27 01:38:55,323][105620] Updated weights for policy 1, policy_version 1410432 (0.0005) [2023-12-27 01:38:55,373][105620] Updated weights for policy 1, policy_version 1410442 (0.0008) [2023-12-27 01:38:55,417][105620] Updated weights for policy 1, policy_version 1410452 (0.0008) [2023-12-27 01:38:55,611][105692] Updated weights for policy 0, policy_version 1408244 (0.0010) [2023-12-27 01:38:55,655][105692] Updated weights for policy 0, policy_version 1408254 (0.0010) [2023-12-27 01:38:55,703][105692] Updated weights for policy 0, policy_version 1408264 (0.0010) [2023-12-27 01:38:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 721690624. Throughput: 0: 9765.5, 1: 9871.7. Samples: 721699516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:38:56,063][104569] Avg episode reward: [(0, '8087.808'), (1, '8896.764')] [2023-12-27 01:38:56,219][105620] Updated weights for policy 1, policy_version 1410462 (0.0008) [2023-12-27 01:38:56,282][105620] Updated weights for policy 1, policy_version 1410472 (0.0008) [2023-12-27 01:38:56,349][105620] Updated weights for policy 1, policy_version 1410482 (0.0008) [2023-12-27 01:38:56,404][105692] Updated weights for policy 0, policy_version 1408274 (0.0010) [2023-12-27 01:38:56,459][105692] Updated weights for policy 0, policy_version 1408284 (0.0010) [2023-12-27 01:38:56,518][105692] Updated weights for policy 0, policy_version 1408294 (0.0011) [2023-12-27 01:38:56,575][105692] Updated weights for policy 0, policy_version 1408304 (0.0010) [2023-12-27 01:38:57,086][105620] Updated weights for policy 1, policy_version 1410492 (0.0008) [2023-12-27 01:38:57,142][105620] Updated weights for policy 1, policy_version 1410502 (0.0008) [2023-12-27 01:38:57,199][105620] Updated weights for policy 1, policy_version 1410512 (0.0009) [2023-12-27 01:38:57,296][105692] Updated weights for policy 0, policy_version 1408314 (0.0009) [2023-12-27 01:38:57,361][105692] Updated weights for policy 0, policy_version 1408324 (0.0009) [2023-12-27 01:38:57,424][105692] Updated weights for policy 0, policy_version 1408334 (0.0009) [2023-12-27 01:38:57,927][105620] Updated weights for policy 1, policy_version 1410522 (0.0008) [2023-12-27 01:38:57,985][105620] Updated weights for policy 1, policy_version 1410532 (0.0005) [2023-12-27 01:38:58,043][105620] Updated weights for policy 1, policy_version 1410542 (0.0006) [2023-12-27 01:38:58,099][105620] Updated weights for policy 1, policy_version 1410552 (0.0007) [2023-12-27 01:38:58,181][105692] Updated weights for policy 0, policy_version 1408344 (0.0010) [2023-12-27 01:38:58,244][105692] Updated weights for policy 0, policy_version 1408354 (0.0011) [2023-12-27 01:38:58,304][105692] Updated weights for policy 0, policy_version 1408364 (0.0010) [2023-12-27 01:38:58,870][105620] Updated weights for policy 1, policy_version 1410562 (0.0007) [2023-12-27 01:38:58,941][105620] Updated weights for policy 1, policy_version 1410572 (0.0008) [2023-12-27 01:38:59,011][105620] Updated weights for policy 1, policy_version 1410582 (0.0007) [2023-12-27 01:38:59,202][105692] Updated weights for policy 0, policy_version 1408374 (0.0010) [2023-12-27 01:38:59,270][105692] Updated weights for policy 0, policy_version 1408384 (0.0009) [2023-12-27 01:38:59,332][105692] Updated weights for policy 0, policy_version 1408394 (0.0011) [2023-12-27 01:38:59,701][105620] Updated weights for policy 1, policy_version 1410592 (0.0008) [2023-12-27 01:38:59,765][105620] Updated weights for policy 1, policy_version 1410602 (0.0008) [2023-12-27 01:38:59,832][105620] Updated weights for policy 1, policy_version 1410612 (0.0008) [2023-12-27 01:39:00,070][105692] Updated weights for policy 0, policy_version 1408404 (0.0008) [2023-12-27 01:39:00,131][105692] Updated weights for policy 0, policy_version 1408414 (0.0007) [2023-12-27 01:39:00,193][105692] Updated weights for policy 0, policy_version 1408424 (0.0007) [2023-12-27 01:39:00,564][105620] Updated weights for policy 1, policy_version 1410622 (0.0010) [2023-12-27 01:39:00,621][105620] Updated weights for policy 1, policy_version 1410632 (0.0010) [2023-12-27 01:39:00,679][105620] Updated weights for policy 1, policy_version 1410642 (0.0010) [2023-12-27 01:39:00,847][105692] Updated weights for policy 0, policy_version 1408434 (0.0007) [2023-12-27 01:39:00,901][105692] Updated weights for policy 0, policy_version 1408444 (0.0005) [2023-12-27 01:39:00,950][105692] Updated weights for policy 0, policy_version 1408454 (0.0005) [2023-12-27 01:39:01,003][105692] Updated weights for policy 0, policy_version 1408464 (0.0005) [2023-12-27 01:39:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 721788928. Throughput: 0: 9749.5, 1: 9858.5. Samples: 721755772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:39:01,063][104569] Avg episode reward: [(0, '8265.869'), (1, '8899.604')] [2023-12-27 01:39:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001408464_360620032.pth... [2023-12-27 01:39:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001410648_361168896.pth... [2023-12-27 01:39:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001409496_360873984.pth [2023-12-27 01:39:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001407312_360325120.pth [2023-12-27 01:39:01,454][105620] Updated weights for policy 1, policy_version 1410652 (0.0010) [2023-12-27 01:39:01,499][105620] Updated weights for policy 1, policy_version 1410662 (0.0010) [2023-12-27 01:39:01,564][105620] Updated weights for policy 1, policy_version 1410672 (0.0010) [2023-12-27 01:39:01,658][105692] Updated weights for policy 0, policy_version 1408474 (0.0009) [2023-12-27 01:39:01,721][105692] Updated weights for policy 0, policy_version 1408484 (0.0010) [2023-12-27 01:39:01,775][105692] Updated weights for policy 0, policy_version 1408494 (0.0008) [2023-12-27 01:39:02,324][105620] Updated weights for policy 1, policy_version 1410682 (0.0009) [2023-12-27 01:39:02,391][105620] Updated weights for policy 1, policy_version 1410692 (0.0006) [2023-12-27 01:39:02,457][105620] Updated weights for policy 1, policy_version 1410702 (0.0006) [2023-12-27 01:39:02,525][105620] Updated weights for policy 1, policy_version 1410712 (0.0006) [2023-12-27 01:39:02,538][105692] Updated weights for policy 0, policy_version 1408504 (0.0007) [2023-12-27 01:39:02,591][105692] Updated weights for policy 0, policy_version 1408514 (0.0010) [2023-12-27 01:39:02,654][105692] Updated weights for policy 0, policy_version 1408524 (0.0010) [2023-12-27 01:39:03,161][105620] Updated weights for policy 1, policy_version 1410722 (0.0009) [2023-12-27 01:39:03,225][105620] Updated weights for policy 1, policy_version 1410732 (0.0008) [2023-12-27 01:39:03,282][105620] Updated weights for policy 1, policy_version 1410742 (0.0008) [2023-12-27 01:39:03,291][105692] Updated weights for policy 0, policy_version 1408534 (0.0010) [2023-12-27 01:39:03,339][105692] Updated weights for policy 0, policy_version 1408544 (0.0010) [2023-12-27 01:39:03,386][105692] Updated weights for policy 0, policy_version 1408554 (0.0010) [2023-12-27 01:39:04,050][105620] Updated weights for policy 1, policy_version 1410752 (0.0007) [2023-12-27 01:39:04,107][105692] Updated weights for policy 0, policy_version 1408564 (0.0010) [2023-12-27 01:39:04,109][105620] Updated weights for policy 1, policy_version 1410762 (0.0006) [2023-12-27 01:39:04,166][105692] Updated weights for policy 0, policy_version 1408574 (0.0008) [2023-12-27 01:39:04,168][105620] Updated weights for policy 1, policy_version 1410772 (0.0006) [2023-12-27 01:39:04,222][105692] Updated weights for policy 0, policy_version 1408584 (0.0009) [2023-12-27 01:39:04,768][105620] Updated weights for policy 1, policy_version 1410782 (0.0007) [2023-12-27 01:39:04,823][105620] Updated weights for policy 1, policy_version 1410792 (0.0009) [2023-12-27 01:39:04,885][105620] Updated weights for policy 1, policy_version 1410802 (0.0009) [2023-12-27 01:39:04,989][105692] Updated weights for policy 0, policy_version 1408594 (0.0009) [2023-12-27 01:39:05,050][105692] Updated weights for policy 0, policy_version 1408604 (0.0009) [2023-12-27 01:39:05,112][105692] Updated weights for policy 0, policy_version 1408614 (0.0009) [2023-12-27 01:39:05,167][105692] Updated weights for policy 0, policy_version 1408624 (0.0009) [2023-12-27 01:39:05,602][105620] Updated weights for policy 1, policy_version 1410812 (0.0008) [2023-12-27 01:39:05,649][105620] Updated weights for policy 1, policy_version 1410822 (0.0009) [2023-12-27 01:39:05,699][105620] Updated weights for policy 1, policy_version 1410832 (0.0007) [2023-12-27 01:39:05,935][105692] Updated weights for policy 0, policy_version 1408634 (0.0008) [2023-12-27 01:39:05,990][105692] Updated weights for policy 0, policy_version 1408644 (0.0008) [2023-12-27 01:39:06,046][105692] Updated weights for policy 0, policy_version 1408654 (0.0008) [2023-12-27 01:39:06,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.9, 300 sec: 19494.2). Total num frames: 721887232. Throughput: 0: 9625.3, 1: 9860.6. Samples: 721871764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:39:06,062][104569] Avg episode reward: [(0, '8068.636'), (1, '8713.629')] [2023-12-27 01:39:06,457][105620] Updated weights for policy 1, policy_version 1410842 (0.0009) [2023-12-27 01:39:06,520][105620] Updated weights for policy 1, policy_version 1410852 (0.0011) [2023-12-27 01:39:06,583][105620] Updated weights for policy 1, policy_version 1410862 (0.0007) [2023-12-27 01:39:06,644][105620] Updated weights for policy 1, policy_version 1410872 (0.0005) [2023-12-27 01:39:06,784][105692] Updated weights for policy 0, policy_version 1408664 (0.0008) [2023-12-27 01:39:06,840][105692] Updated weights for policy 0, policy_version 1408674 (0.0008) [2023-12-27 01:39:06,891][105692] Updated weights for policy 0, policy_version 1408684 (0.0008) [2023-12-27 01:39:07,254][105620] Updated weights for policy 1, policy_version 1410882 (0.0005) [2023-12-27 01:39:07,322][105620] Updated weights for policy 1, policy_version 1410892 (0.0005) [2023-12-27 01:39:07,377][105620] Updated weights for policy 1, policy_version 1410902 (0.0011) [2023-12-27 01:39:07,717][105692] Updated weights for policy 0, policy_version 1408694 (0.0009) [2023-12-27 01:39:07,779][105692] Updated weights for policy 0, policy_version 1408704 (0.0008) [2023-12-27 01:39:07,827][105692] Updated weights for policy 0, policy_version 1408714 (0.0008) [2023-12-27 01:39:08,066][105620] Updated weights for policy 1, policy_version 1410912 (0.0006) [2023-12-27 01:39:08,121][105620] Updated weights for policy 1, policy_version 1410922 (0.0005) [2023-12-27 01:39:08,171][105620] Updated weights for policy 1, policy_version 1410932 (0.0005) [2023-12-27 01:39:08,659][105692] Updated weights for policy 0, policy_version 1408724 (0.0008) [2023-12-27 01:39:08,715][105692] Updated weights for policy 0, policy_version 1408734 (0.0010) [2023-12-27 01:39:08,766][105692] Updated weights for policy 0, policy_version 1408744 (0.0009) [2023-12-27 01:39:08,789][105620] Updated weights for policy 1, policy_version 1410942 (0.0006) [2023-12-27 01:39:08,843][105620] Updated weights for policy 1, policy_version 1410952 (0.0009) [2023-12-27 01:39:08,901][105620] Updated weights for policy 1, policy_version 1410962 (0.0009) [2023-12-27 01:39:09,594][105692] Updated weights for policy 0, policy_version 1408754 (0.0007) [2023-12-27 01:39:09,654][105692] Updated weights for policy 0, policy_version 1408764 (0.0010) [2023-12-27 01:39:09,656][105620] Updated weights for policy 1, policy_version 1410972 (0.0007) [2023-12-27 01:39:09,709][105620] Updated weights for policy 1, policy_version 1410982 (0.0008) [2023-12-27 01:39:09,712][105692] Updated weights for policy 0, policy_version 1408774 (0.0007) [2023-12-27 01:39:09,768][105620] Updated weights for policy 1, policy_version 1410992 (0.0005) [2023-12-27 01:39:09,774][105692] Updated weights for policy 0, policy_version 1408784 (0.0009) [2023-12-27 01:39:10,479][105620] Updated weights for policy 1, policy_version 1411002 (0.0006) [2023-12-27 01:39:10,540][105620] Updated weights for policy 1, policy_version 1411012 (0.0008) [2023-12-27 01:39:10,568][105692] Updated weights for policy 0, policy_version 1408794 (0.0007) [2023-12-27 01:39:10,600][105620] Updated weights for policy 1, policy_version 1411022 (0.0008) [2023-12-27 01:39:10,618][105692] Updated weights for policy 0, policy_version 1408804 (0.0006) [2023-12-27 01:39:10,657][105620] Updated weights for policy 1, policy_version 1411032 (0.0007) [2023-12-27 01:39:10,671][105692] Updated weights for policy 0, policy_version 1408814 (0.0008) [2023-12-27 01:39:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 721977344. Throughput: 0: 9595.4, 1: 9868.1. Samples: 721985764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:39:11,062][104569] Avg episode reward: [(0, '7437.471'), (1, '8897.294')] [2023-12-27 01:39:11,420][105620] Updated weights for policy 1, policy_version 1411042 (0.0010) [2023-12-27 01:39:11,477][105692] Updated weights for policy 0, policy_version 1408824 (0.0006) [2023-12-27 01:39:11,484][105620] Updated weights for policy 1, policy_version 1411052 (0.0009) [2023-12-27 01:39:11,535][105692] Updated weights for policy 0, policy_version 1408834 (0.0006) [2023-12-27 01:39:11,552][105620] Updated weights for policy 1, policy_version 1411062 (0.0008) [2023-12-27 01:39:11,590][105692] Updated weights for policy 0, policy_version 1408844 (0.0008) [2023-12-27 01:39:12,312][105620] Updated weights for policy 1, policy_version 1411072 (0.0009) [2023-12-27 01:39:12,329][105692] Updated weights for policy 0, policy_version 1408854 (0.0009) [2023-12-27 01:39:12,369][105620] Updated weights for policy 1, policy_version 1411082 (0.0010) [2023-12-27 01:39:12,384][105692] Updated weights for policy 0, policy_version 1408864 (0.0007) [2023-12-27 01:39:12,429][105620] Updated weights for policy 1, policy_version 1411092 (0.0010) [2023-12-27 01:39:12,447][105692] Updated weights for policy 0, policy_version 1408874 (0.0009) [2023-12-27 01:39:13,050][105620] Updated weights for policy 1, policy_version 1411102 (0.0011) [2023-12-27 01:39:13,102][105620] Updated weights for policy 1, policy_version 1411112 (0.0011) [2023-12-27 01:39:13,153][105620] Updated weights for policy 1, policy_version 1411122 (0.0010) [2023-12-27 01:39:13,258][105692] Updated weights for policy 0, policy_version 1408884 (0.0009) [2023-12-27 01:39:13,315][105692] Updated weights for policy 0, policy_version 1408894 (0.0010) [2023-12-27 01:39:13,369][105692] Updated weights for policy 0, policy_version 1408906 (0.0011) [2023-12-27 01:39:13,744][105620] Updated weights for policy 1, policy_version 1411132 (0.0009) [2023-12-27 01:39:13,791][105620] Updated weights for policy 1, policy_version 1411142 (0.0010) [2023-12-27 01:39:13,835][105620] Updated weights for policy 1, policy_version 1411152 (0.0006) [2023-12-27 01:39:14,227][105692] Updated weights for policy 0, policy_version 1408917 (0.0010) [2023-12-27 01:39:14,293][105692] Updated weights for policy 0, policy_version 1408927 (0.0010) [2023-12-27 01:39:14,355][105692] Updated weights for policy 0, policy_version 1408937 (0.0009) [2023-12-27 01:39:14,381][105620] Updated weights for policy 1, policy_version 1411162 (0.0006) [2023-12-27 01:39:14,442][105620] Updated weights for policy 1, policy_version 1411172 (0.0007) [2023-12-27 01:39:14,494][105620] Updated weights for policy 1, policy_version 1411182 (0.0010) [2023-12-27 01:39:14,542][105620] Updated weights for policy 1, policy_version 1411192 (0.0010) [2023-12-27 01:39:15,135][105692] Updated weights for policy 0, policy_version 1408947 (0.0008) [2023-12-27 01:39:15,199][105692] Updated weights for policy 0, policy_version 1408957 (0.0008) [2023-12-27 01:39:15,262][105692] Updated weights for policy 0, policy_version 1408967 (0.0009) [2023-12-27 01:39:15,297][105620] Updated weights for policy 1, policy_version 1411202 (0.0011) [2023-12-27 01:39:15,362][105620] Updated weights for policy 1, policy_version 1411212 (0.0011) [2023-12-27 01:39:15,433][105620] Updated weights for policy 1, policy_version 1411222 (0.0011) [2023-12-27 01:39:15,954][105692] Updated weights for policy 0, policy_version 1408977 (0.0008) [2023-12-27 01:39:16,011][105692] Updated weights for policy 0, policy_version 1408987 (0.0006) [2023-12-27 01:39:16,060][105692] Updated weights for policy 0, policy_version 1408997 (0.0005) [2023-12-27 01:39:16,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 722067456. Throughput: 0: 9545.4, 1: 9874.2. Samples: 722043040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:39:16,062][104569] Avg episode reward: [(0, '8078.354'), (1, '8991.180')] [2023-12-27 01:39:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001411224_361316352.pth... [2023-12-27 01:39:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001410104_361029632.pth [2023-12-27 01:39:16,120][105692] Updated weights for policy 0, policy_version 1409007 (0.0005) [2023-12-27 01:39:16,124][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001409008_360759296.pth... [2023-12-27 01:39:16,127][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001407888_360472576.pth [2023-12-27 01:39:16,170][105620] Updated weights for policy 1, policy_version 1411232 (0.0010) [2023-12-27 01:39:16,218][105620] Updated weights for policy 1, policy_version 1411242 (0.0010) [2023-12-27 01:39:16,262][105620] Updated weights for policy 1, policy_version 1411252 (0.0010) [2023-12-27 01:39:16,724][105692] Updated weights for policy 0, policy_version 1409017 (0.0008) [2023-12-27 01:39:16,786][105692] Updated weights for policy 0, policy_version 1409027 (0.0008) [2023-12-27 01:39:16,853][105692] Updated weights for policy 0, policy_version 1409037 (0.0009) [2023-12-27 01:39:16,984][105620] Updated weights for policy 1, policy_version 1411262 (0.0007) [2023-12-27 01:39:17,033][105620] Updated weights for policy 1, policy_version 1411272 (0.0005) [2023-12-27 01:39:17,093][105620] Updated weights for policy 1, policy_version 1411282 (0.0005) [2023-12-27 01:39:17,643][105692] Updated weights for policy 0, policy_version 1409047 (0.0009) [2023-12-27 01:39:17,699][105692] Updated weights for policy 0, policy_version 1409057 (0.0008) [2023-12-27 01:39:17,739][105620] Updated weights for policy 1, policy_version 1411292 (0.0007) [2023-12-27 01:39:17,759][105692] Updated weights for policy 0, policy_version 1409067 (0.0008) [2023-12-27 01:39:17,801][105620] Updated weights for policy 1, policy_version 1411302 (0.0005) [2023-12-27 01:39:17,857][105620] Updated weights for policy 1, policy_version 1411312 (0.0006) [2023-12-27 01:39:18,505][105620] Updated weights for policy 1, policy_version 1411322 (0.0005) [2023-12-27 01:39:18,565][105620] Updated weights for policy 1, policy_version 1411332 (0.0009) [2023-12-27 01:39:18,607][105692] Updated weights for policy 0, policy_version 1409077 (0.0009) [2023-12-27 01:39:18,619][105620] Updated weights for policy 1, policy_version 1411342 (0.0005) [2023-12-27 01:39:18,662][105692] Updated weights for policy 0, policy_version 1409087 (0.0008) [2023-12-27 01:39:18,676][105620] Updated weights for policy 1, policy_version 1411352 (0.0008) [2023-12-27 01:39:18,717][105692] Updated weights for policy 0, policy_version 1409097 (0.0007) [2023-12-27 01:39:19,376][105620] Updated weights for policy 1, policy_version 1411362 (0.0010) [2023-12-27 01:39:19,437][105620] Updated weights for policy 1, policy_version 1411372 (0.0011) [2023-12-27 01:39:19,509][105620] Updated weights for policy 1, policy_version 1411382 (0.0010) [2023-12-27 01:39:19,542][105692] Updated weights for policy 0, policy_version 1409107 (0.0007) [2023-12-27 01:39:19,604][105692] Updated weights for policy 0, policy_version 1409117 (0.0006) [2023-12-27 01:39:19,661][105692] Updated weights for policy 0, policy_version 1409127 (0.0006) [2023-12-27 01:39:20,314][105620] Updated weights for policy 1, policy_version 1411392 (0.0009) [2023-12-27 01:39:20,334][105692] Updated weights for policy 0, policy_version 1409137 (0.0006) [2023-12-27 01:39:20,377][105620] Updated weights for policy 1, policy_version 1411402 (0.0008) [2023-12-27 01:39:20,385][105692] Updated weights for policy 0, policy_version 1409147 (0.0007) [2023-12-27 01:39:20,434][105692] Updated weights for policy 0, policy_version 1409157 (0.0006) [2023-12-27 01:39:20,444][105620] Updated weights for policy 1, policy_version 1411412 (0.0009) [2023-12-27 01:39:20,490][105692] Updated weights for policy 0, policy_version 1409167 (0.0007) [2023-12-27 01:39:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19438.7). Total num frames: 722165760. Throughput: 0: 9510.3, 1: 9854.8. Samples: 722159000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:39:21,062][104569] Avg episode reward: [(0, '7712.436'), (1, '8628.133')] [2023-12-27 01:39:21,165][105620] Updated weights for policy 1, policy_version 1411422 (0.0011) [2023-12-27 01:39:21,230][105620] Updated weights for policy 1, policy_version 1411432 (0.0005) [2023-12-27 01:39:21,293][105620] Updated weights for policy 1, policy_version 1411442 (0.0008) [2023-12-27 01:39:21,296][105692] Updated weights for policy 0, policy_version 1409177 (0.0007) [2023-12-27 01:39:21,369][105692] Updated weights for policy 0, policy_version 1409187 (0.0007) [2023-12-27 01:39:21,433][105692] Updated weights for policy 0, policy_version 1409197 (0.0006) [2023-12-27 01:39:21,985][105620] Updated weights for policy 1, policy_version 1411452 (0.0006) [2023-12-27 01:39:22,049][105620] Updated weights for policy 1, policy_version 1411462 (0.0005) [2023-12-27 01:39:22,119][105620] Updated weights for policy 1, policy_version 1411472 (0.0006) [2023-12-27 01:39:22,159][105692] Updated weights for policy 0, policy_version 1409207 (0.0007) [2023-12-27 01:39:22,216][105692] Updated weights for policy 0, policy_version 1409217 (0.0010) [2023-12-27 01:39:22,277][105692] Updated weights for policy 0, policy_version 1409227 (0.0008) [2023-12-27 01:39:22,712][105620] Updated weights for policy 1, policy_version 1411482 (0.0006) [2023-12-27 01:39:22,776][105620] Updated weights for policy 1, policy_version 1411492 (0.0008) [2023-12-27 01:39:22,836][105620] Updated weights for policy 1, policy_version 1411502 (0.0009) [2023-12-27 01:39:22,896][105620] Updated weights for policy 1, policy_version 1411512 (0.0007) [2023-12-27 01:39:23,059][105692] Updated weights for policy 0, policy_version 1409237 (0.0009) [2023-12-27 01:39:23,107][105692] Updated weights for policy 0, policy_version 1409247 (0.0008) [2023-12-27 01:39:23,158][105692] Updated weights for policy 0, policy_version 1409257 (0.0009) [2023-12-27 01:39:23,571][105620] Updated weights for policy 1, policy_version 1411522 (0.0009) [2023-12-27 01:39:23,626][105620] Updated weights for policy 1, policy_version 1411532 (0.0009) [2023-12-27 01:39:23,672][105620] Updated weights for policy 1, policy_version 1411542 (0.0008) [2023-12-27 01:39:23,948][105692] Updated weights for policy 0, policy_version 1409267 (0.0009) [2023-12-27 01:39:24,009][105692] Updated weights for policy 0, policy_version 1409277 (0.0009) [2023-12-27 01:39:24,060][105692] Updated weights for policy 0, policy_version 1409287 (0.0009) [2023-12-27 01:39:24,438][105620] Updated weights for policy 1, policy_version 1411552 (0.0009) [2023-12-27 01:39:24,505][105620] Updated weights for policy 1, policy_version 1411562 (0.0008) [2023-12-27 01:39:24,575][105620] Updated weights for policy 1, policy_version 1411572 (0.0007) [2023-12-27 01:39:24,847][105692] Updated weights for policy 0, policy_version 1409297 (0.0009) [2023-12-27 01:39:24,898][105692] Updated weights for policy 0, policy_version 1409307 (0.0009) [2023-12-27 01:39:24,942][105692] Updated weights for policy 0, policy_version 1409317 (0.0008) [2023-12-27 01:39:24,987][105692] Updated weights for policy 0, policy_version 1409327 (0.0008) [2023-12-27 01:39:25,301][105620] Updated weights for policy 1, policy_version 1411582 (0.0010) [2023-12-27 01:39:25,363][105620] Updated weights for policy 1, policy_version 1411592 (0.0010) [2023-12-27 01:39:25,422][105620] Updated weights for policy 1, policy_version 1411602 (0.0010) [2023-12-27 01:39:25,752][105692] Updated weights for policy 0, policy_version 1409337 (0.0008) [2023-12-27 01:39:25,759][105585] KL-divergence is very high: 310.6117 [2023-12-27 01:39:25,771][105585] KL-divergence is very high: 399.3728 [2023-12-27 01:39:25,805][105585] KL-divergence is very high: 581.7266 [2023-12-27 01:39:25,810][105692] Updated weights for policy 0, policy_version 1409347 (0.0009) [2023-12-27 01:39:25,817][105585] KL-divergence is very high: 617.3367 [2023-12-27 01:39:25,854][105585] KL-divergence is very high: 699.3481 [2023-12-27 01:39:25,864][105585] KL-divergence is very high: 715.3726 [2023-12-27 01:39:25,870][105692] Updated weights for policy 0, policy_version 1409357 (0.0007) [2023-12-27 01:39:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 722264064. Throughput: 0: 9439.5, 1: 9847.2. Samples: 722272872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:39:26,062][104569] Avg episode reward: [(0, '7342.531'), (1, '8810.413')] [2023-12-27 01:39:26,182][105620] Updated weights for policy 1, policy_version 1411612 (0.0011) [2023-12-27 01:39:26,237][105620] Updated weights for policy 1, policy_version 1411622 (0.0009) [2023-12-27 01:39:26,284][105620] Updated weights for policy 1, policy_version 1411632 (0.0005) [2023-12-27 01:39:26,447][105692] Updated weights for policy 0, policy_version 1409367 (0.0006) [2023-12-27 01:39:26,514][105692] Updated weights for policy 0, policy_version 1409377 (0.0005) [2023-12-27 01:39:26,574][105692] Updated weights for policy 0, policy_version 1409387 (0.0006) [2023-12-27 01:39:26,954][105620] Updated weights for policy 1, policy_version 1411642 (0.0006) [2023-12-27 01:39:27,012][105620] Updated weights for policy 1, policy_version 1411652 (0.0010) [2023-12-27 01:39:27,064][105620] Updated weights for policy 1, policy_version 1411662 (0.0010) [2023-12-27 01:39:27,086][105692] Updated weights for policy 0, policy_version 1409397 (0.0006) [2023-12-27 01:39:27,122][105620] Updated weights for policy 1, policy_version 1411672 (0.0010) [2023-12-27 01:39:27,151][105692] Updated weights for policy 0, policy_version 1409407 (0.0005) [2023-12-27 01:39:27,208][105692] Updated weights for policy 0, policy_version 1409417 (0.0005) [2023-12-27 01:39:27,770][105692] Updated weights for policy 0, policy_version 1409427 (0.0005) [2023-12-27 01:39:27,829][105692] Updated weights for policy 0, policy_version 1409438 (0.0006) [2023-12-27 01:39:27,864][105620] Updated weights for policy 1, policy_version 1411682 (0.0010) [2023-12-27 01:39:27,881][105692] Updated weights for policy 0, policy_version 1409448 (0.0005) [2023-12-27 01:39:27,920][105620] Updated weights for policy 1, policy_version 1411692 (0.0010) [2023-12-27 01:39:27,972][105620] Updated weights for policy 1, policy_version 1411702 (0.0010) [2023-12-27 01:39:28,477][105692] Updated weights for policy 0, policy_version 1409458 (0.0006) [2023-12-27 01:39:28,538][105692] Updated weights for policy 0, policy_version 1409468 (0.0010) [2023-12-27 01:39:28,601][105692] Updated weights for policy 0, policy_version 1409478 (0.0010) [2023-12-27 01:39:28,649][105692] Updated weights for policy 0, policy_version 1409488 (0.0010) [2023-12-27 01:39:28,745][105620] Updated weights for policy 1, policy_version 1411712 (0.0010) [2023-12-27 01:39:28,789][105620] Updated weights for policy 1, policy_version 1411722 (0.0010) [2023-12-27 01:39:28,844][105620] Updated weights for policy 1, policy_version 1411732 (0.0010) [2023-12-27 01:39:29,406][105692] Updated weights for policy 0, policy_version 1409498 (0.0009) [2023-12-27 01:39:29,467][105692] Updated weights for policy 0, policy_version 1409508 (0.0008) [2023-12-27 01:39:29,526][105692] Updated weights for policy 0, policy_version 1409518 (0.0010) [2023-12-27 01:39:29,609][105620] Updated weights for policy 1, policy_version 1411742 (0.0010) [2023-12-27 01:39:29,668][105620] Updated weights for policy 1, policy_version 1411752 (0.0009) [2023-12-27 01:39:29,727][105620] Updated weights for policy 1, policy_version 1411762 (0.0009) [2023-12-27 01:39:30,176][105692] Updated weights for policy 0, policy_version 1409528 (0.0007) [2023-12-27 01:39:30,229][105692] Updated weights for policy 0, policy_version 1409538 (0.0007) [2023-12-27 01:39:30,287][105692] Updated weights for policy 0, policy_version 1409548 (0.0010) [2023-12-27 01:39:30,551][105620] Updated weights for policy 1, policy_version 1411772 (0.0008) [2023-12-27 01:39:30,596][105620] Updated weights for policy 1, policy_version 1411782 (0.0008) [2023-12-27 01:39:30,642][105620] Updated weights for policy 1, policy_version 1411792 (0.0009) [2023-12-27 01:39:30,910][105692] Updated weights for policy 0, policy_version 1409558 (0.0007) [2023-12-27 01:39:30,970][105692] Updated weights for policy 0, policy_version 1409568 (0.0005) [2023-12-27 01:39:31,041][105692] Updated weights for policy 0, policy_version 1409578 (0.0007) [2023-12-27 01:39:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 722362368. Throughput: 0: 9594.8, 1: 9841.9. Samples: 722336708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:39:31,063][104569] Avg episode reward: [(0, '8074.181'), (1, '9092.016')] [2023-12-27 01:39:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001411800_361463808.pth... [2023-12-27 01:39:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001410648_361168896.pth [2023-12-27 01:39:31,080][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001409584_360906752.pth... [2023-12-27 01:39:31,083][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001408464_360620032.pth [2023-12-27 01:39:31,413][105620] Updated weights for policy 1, policy_version 1411802 (0.0009) [2023-12-27 01:39:31,465][105620] Updated weights for policy 1, policy_version 1411812 (0.0008) [2023-12-27 01:39:31,510][105620] Updated weights for policy 1, policy_version 1411822 (0.0008) [2023-12-27 01:39:31,565][105620] Updated weights for policy 1, policy_version 1411832 (0.0008) [2023-12-27 01:39:31,787][105692] Updated weights for policy 0, policy_version 1409588 (0.0008) [2023-12-27 01:39:31,841][105692] Updated weights for policy 0, policy_version 1409598 (0.0008) [2023-12-27 01:39:31,894][105692] Updated weights for policy 0, policy_version 1409608 (0.0009) [2023-12-27 01:39:32,316][105620] Updated weights for policy 1, policy_version 1411842 (0.0009) [2023-12-27 01:39:32,379][105620] Updated weights for policy 1, policy_version 1411852 (0.0009) [2023-12-27 01:39:32,438][105620] Updated weights for policy 1, policy_version 1411862 (0.0010) [2023-12-27 01:39:32,681][105692] Updated weights for policy 0, policy_version 1409618 (0.0008) [2023-12-27 01:39:32,730][105692] Updated weights for policy 0, policy_version 1409628 (0.0005) [2023-12-27 01:39:32,779][105692] Updated weights for policy 0, policy_version 1409638 (0.0005) [2023-12-27 01:39:32,836][105692] Updated weights for policy 0, policy_version 1409648 (0.0006) [2023-12-27 01:39:33,133][105620] Updated weights for policy 1, policy_version 1411872 (0.0009) [2023-12-27 01:39:33,193][105620] Updated weights for policy 1, policy_version 1411882 (0.0009) [2023-12-27 01:39:33,251][105620] Updated weights for policy 1, policy_version 1411894 (0.0010) [2023-12-27 01:39:33,396][105692] Updated weights for policy 0, policy_version 1409658 (0.0006) [2023-12-27 01:39:33,447][105692] Updated weights for policy 0, policy_version 1409668 (0.0008) [2023-12-27 01:39:33,497][105692] Updated weights for policy 0, policy_version 1409678 (0.0008) [2023-12-27 01:39:33,902][105620] Updated weights for policy 1, policy_version 1411904 (0.0007) [2023-12-27 01:39:33,958][105620] Updated weights for policy 1, policy_version 1411914 (0.0006) [2023-12-27 01:39:34,011][105620] Updated weights for policy 1, policy_version 1411924 (0.0006) [2023-12-27 01:39:34,118][105692] Updated weights for policy 0, policy_version 1409688 (0.0007) [2023-12-27 01:39:34,190][105692] Updated weights for policy 0, policy_version 1409698 (0.0008) [2023-12-27 01:39:34,253][105692] Updated weights for policy 0, policy_version 1409708 (0.0010) [2023-12-27 01:39:34,577][105620] Updated weights for policy 1, policy_version 1411934 (0.0005) [2023-12-27 01:39:34,638][105620] Updated weights for policy 1, policy_version 1411944 (0.0008) [2023-12-27 01:39:34,699][105620] Updated weights for policy 1, policy_version 1411954 (0.0007) [2023-12-27 01:39:34,901][105692] Updated weights for policy 0, policy_version 1409718 (0.0011) [2023-12-27 01:39:34,955][105692] Updated weights for policy 0, policy_version 1409728 (0.0010) [2023-12-27 01:39:35,001][105692] Updated weights for policy 0, policy_version 1409738 (0.0011) [2023-12-27 01:39:35,347][105620] Updated weights for policy 1, policy_version 1411964 (0.0007) [2023-12-27 01:39:35,399][105620] Updated weights for policy 1, policy_version 1411974 (0.0006) [2023-12-27 01:39:35,447][105620] Updated weights for policy 1, policy_version 1411984 (0.0008) [2023-12-27 01:39:35,750][105692] Updated weights for policy 0, policy_version 1409748 (0.0011) [2023-12-27 01:39:35,808][105692] Updated weights for policy 0, policy_version 1409758 (0.0010) [2023-12-27 01:39:35,870][105692] Updated weights for policy 0, policy_version 1409768 (0.0011) [2023-12-27 01:39:36,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 722468864. Throughput: 0: 9669.7, 1: 9809.4. Samples: 722457304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:39:36,062][104569] Avg episode reward: [(0, '7709.185'), (1, '8755.506')] [2023-12-27 01:39:36,134][105620] Updated weights for policy 1, policy_version 1411994 (0.0008) [2023-12-27 01:39:36,187][105620] Updated weights for policy 1, policy_version 1412004 (0.0008) [2023-12-27 01:39:36,249][105620] Updated weights for policy 1, policy_version 1412014 (0.0009) [2023-12-27 01:39:36,303][105620] Updated weights for policy 1, policy_version 1412024 (0.0010) [2023-12-27 01:39:36,514][105692] Updated weights for policy 0, policy_version 1409778 (0.0009) [2023-12-27 01:39:36,598][105692] Updated weights for policy 0, policy_version 1409788 (0.0010) [2023-12-27 01:39:36,659][105692] Updated weights for policy 0, policy_version 1409798 (0.0011) [2023-12-27 01:39:36,722][105692] Updated weights for policy 0, policy_version 1409808 (0.0010) [2023-12-27 01:39:37,151][105620] Updated weights for policy 1, policy_version 1412034 (0.0009) [2023-12-27 01:39:37,208][105620] Updated weights for policy 1, policy_version 1412045 (0.0010) [2023-12-27 01:39:37,264][105620] Updated weights for policy 1, policy_version 1412055 (0.0010) [2023-12-27 01:39:37,351][105692] Updated weights for policy 0, policy_version 1409818 (0.0007) [2023-12-27 01:39:37,417][105692] Updated weights for policy 0, policy_version 1409828 (0.0006) [2023-12-27 01:39:37,465][105692] Updated weights for policy 0, policy_version 1409838 (0.0006) [2023-12-27 01:39:37,961][105620] Updated weights for policy 1, policy_version 1412065 (0.0006) [2023-12-27 01:39:38,013][105620] Updated weights for policy 1, policy_version 1412075 (0.0007) [2023-12-27 01:39:38,069][105620] Updated weights for policy 1, policy_version 1412085 (0.0009) [2023-12-27 01:39:38,133][105692] Updated weights for policy 0, policy_version 1409848 (0.0006) [2023-12-27 01:39:38,180][105692] Updated weights for policy 0, policy_version 1409858 (0.0005) [2023-12-27 01:39:38,237][105692] Updated weights for policy 0, policy_version 1409868 (0.0007) [2023-12-27 01:39:38,751][105620] Updated weights for policy 1, policy_version 1412095 (0.0007) [2023-12-27 01:39:38,821][105620] Updated weights for policy 1, policy_version 1412105 (0.0006) [2023-12-27 01:39:38,887][105620] Updated weights for policy 1, policy_version 1412115 (0.0008) [2023-12-27 01:39:38,965][105692] Updated weights for policy 0, policy_version 1409878 (0.0011) [2023-12-27 01:39:39,017][105692] Updated weights for policy 0, policy_version 1409888 (0.0010) [2023-12-27 01:39:39,065][105692] Updated weights for policy 0, policy_version 1409898 (0.0010) [2023-12-27 01:39:39,550][105620] Updated weights for policy 1, policy_version 1412125 (0.0009) [2023-12-27 01:39:39,617][105620] Updated weights for policy 1, policy_version 1412135 (0.0010) [2023-12-27 01:39:39,683][105620] Updated weights for policy 1, policy_version 1412145 (0.0011) [2023-12-27 01:39:39,870][105692] Updated weights for policy 0, policy_version 1409908 (0.0008) [2023-12-27 01:39:39,935][105692] Updated weights for policy 0, policy_version 1409918 (0.0007) [2023-12-27 01:39:39,994][105692] Updated weights for policy 0, policy_version 1409928 (0.0009) [2023-12-27 01:39:40,426][105620] Updated weights for policy 1, policy_version 1412155 (0.0011) [2023-12-27 01:39:40,485][105620] Updated weights for policy 1, policy_version 1412165 (0.0011) [2023-12-27 01:39:40,552][105620] Updated weights for policy 1, policy_version 1412175 (0.0011) [2023-12-27 01:39:40,741][105692] Updated weights for policy 0, policy_version 1409938 (0.0008) [2023-12-27 01:39:40,805][105692] Updated weights for policy 0, policy_version 1409948 (0.0009) [2023-12-27 01:39:40,868][105692] Updated weights for policy 0, policy_version 1409958 (0.0008) [2023-12-27 01:39:40,931][105692] Updated weights for policy 0, policy_version 1409968 (0.0009) [2023-12-27 01:39:41,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 722567168. Throughput: 0: 9635.0, 1: 9805.5. Samples: 722574340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:39:41,063][104569] Avg episode reward: [(0, '7979.976'), (1, '8645.736')] [2023-12-27 01:39:41,297][105620] Updated weights for policy 1, policy_version 1412185 (0.0010) [2023-12-27 01:39:41,364][105620] Updated weights for policy 1, policy_version 1412195 (0.0012) [2023-12-27 01:39:41,419][105620] Updated weights for policy 1, policy_version 1412205 (0.0008) [2023-12-27 01:39:41,473][105620] Updated weights for policy 1, policy_version 1412215 (0.0008) [2023-12-27 01:39:41,664][105692] Updated weights for policy 0, policy_version 1409978 (0.0010) [2023-12-27 01:39:41,729][105692] Updated weights for policy 0, policy_version 1409988 (0.0012) [2023-12-27 01:39:41,792][105692] Updated weights for policy 0, policy_version 1409998 (0.0010) [2023-12-27 01:39:42,255][105620] Updated weights for policy 1, policy_version 1412225 (0.0008) [2023-12-27 01:39:42,320][105620] Updated weights for policy 1, policy_version 1412235 (0.0009) [2023-12-27 01:39:42,384][105620] Updated weights for policy 1, policy_version 1412245 (0.0007) [2023-12-27 01:39:42,562][105692] Updated weights for policy 0, policy_version 1410008 (0.0010) [2023-12-27 01:39:42,624][105692] Updated weights for policy 0, policy_version 1410018 (0.0010) [2023-12-27 01:39:42,690][105692] Updated weights for policy 0, policy_version 1410028 (0.0011) [2023-12-27 01:39:43,112][105620] Updated weights for policy 1, policy_version 1412255 (0.0009) [2023-12-27 01:39:43,159][105620] Updated weights for policy 1, policy_version 1412265 (0.0008) [2023-12-27 01:39:43,215][105620] Updated weights for policy 1, policy_version 1412275 (0.0008) [2023-12-27 01:39:43,361][105692] Updated weights for policy 0, policy_version 1410038 (0.0008) [2023-12-27 01:39:43,410][105692] Updated weights for policy 0, policy_version 1410048 (0.0005) [2023-12-27 01:39:43,460][105692] Updated weights for policy 0, policy_version 1410058 (0.0006) [2023-12-27 01:39:43,999][105692] Updated weights for policy 0, policy_version 1410068 (0.0006) [2023-12-27 01:39:44,015][105620] Updated weights for policy 1, policy_version 1412285 (0.0006) [2023-12-27 01:39:44,048][105692] Updated weights for policy 0, policy_version 1410078 (0.0007) [2023-12-27 01:39:44,067][105620] Updated weights for policy 1, policy_version 1412295 (0.0007) [2023-12-27 01:39:44,106][105692] Updated weights for policy 0, policy_version 1410088 (0.0005) [2023-12-27 01:39:44,123][105620] Updated weights for policy 1, policy_version 1412305 (0.0008) [2023-12-27 01:39:44,671][105692] Updated weights for policy 0, policy_version 1410098 (0.0007) [2023-12-27 01:39:44,729][105692] Updated weights for policy 0, policy_version 1410108 (0.0010) [2023-12-27 01:39:44,788][105692] Updated weights for policy 0, policy_version 1410118 (0.0009) [2023-12-27 01:39:44,844][105692] Updated weights for policy 0, policy_version 1410128 (0.0009) [2023-12-27 01:39:44,947][105620] Updated weights for policy 1, policy_version 1412315 (0.0009) [2023-12-27 01:39:45,015][105620] Updated weights for policy 1, policy_version 1412325 (0.0008) [2023-12-27 01:39:45,080][105620] Updated weights for policy 1, policy_version 1412335 (0.0009) [2023-12-27 01:39:45,567][105692] Updated weights for policy 0, policy_version 1410138 (0.0010) [2023-12-27 01:39:45,622][105692] Updated weights for policy 0, policy_version 1410148 (0.0011) [2023-12-27 01:39:45,676][105692] Updated weights for policy 0, policy_version 1410158 (0.0010) [2023-12-27 01:39:45,798][105620] Updated weights for policy 1, policy_version 1412345 (0.0009) [2023-12-27 01:39:45,849][105620] Updated weights for policy 1, policy_version 1412355 (0.0009) [2023-12-27 01:39:45,907][105620] Updated weights for policy 1, policy_version 1412365 (0.0009) [2023-12-27 01:39:45,966][105620] Updated weights for policy 1, policy_version 1412375 (0.0008) [2023-12-27 01:39:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 722665472. Throughput: 0: 9652.6, 1: 9789.4. Samples: 722630656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:39:46,062][104569] Avg episode reward: [(0, '7792.078'), (1, '8810.455')] [2023-12-27 01:39:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001410160_361054208.pth... [2023-12-27 01:39:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001412376_361611264.pth... [2023-12-27 01:39:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001409008_360759296.pth [2023-12-27 01:39:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001411224_361316352.pth [2023-12-27 01:39:46,400][105692] Updated weights for policy 0, policy_version 1410168 (0.0010) [2023-12-27 01:39:46,464][105692] Updated weights for policy 0, policy_version 1410178 (0.0010) [2023-12-27 01:39:46,518][105692] Updated weights for policy 0, policy_version 1410188 (0.0010) [2023-12-27 01:39:46,698][105620] Updated weights for policy 1, policy_version 1412385 (0.0005) [2023-12-27 01:39:46,758][105620] Updated weights for policy 1, policy_version 1412395 (0.0005) [2023-12-27 01:39:46,825][105620] Updated weights for policy 1, policy_version 1412405 (0.0006) [2023-12-27 01:39:47,258][105692] Updated weights for policy 0, policy_version 1410198 (0.0010) [2023-12-27 01:39:47,320][105692] Updated weights for policy 0, policy_version 1410208 (0.0010) [2023-12-27 01:39:47,358][105620] Updated weights for policy 1, policy_version 1412415 (0.0006) [2023-12-27 01:39:47,386][105692] Updated weights for policy 0, policy_version 1410218 (0.0007) [2023-12-27 01:39:47,424][105620] Updated weights for policy 1, policy_version 1412425 (0.0005) [2023-12-27 01:39:47,478][105620] Updated weights for policy 1, policy_version 1412435 (0.0005) [2023-12-27 01:39:47,972][105620] Updated weights for policy 1, policy_version 1412445 (0.0005) [2023-12-27 01:39:48,026][105620] Updated weights for policy 1, policy_version 1412455 (0.0005) [2023-12-27 01:39:48,059][105692] Updated weights for policy 0, policy_version 1410228 (0.0007) [2023-12-27 01:39:48,097][105620] Updated weights for policy 1, policy_version 1412465 (0.0005) [2023-12-27 01:39:48,117][105692] Updated weights for policy 0, policy_version 1410238 (0.0010) [2023-12-27 01:39:48,168][105692] Updated weights for policy 0, policy_version 1410248 (0.0010) [2023-12-27 01:39:48,651][105620] Updated weights for policy 1, policy_version 1412475 (0.0007) [2023-12-27 01:39:48,701][105620] Updated weights for policy 1, policy_version 1412485 (0.0011) [2023-12-27 01:39:48,758][105620] Updated weights for policy 1, policy_version 1412495 (0.0009) [2023-12-27 01:39:48,949][105692] Updated weights for policy 0, policy_version 1410258 (0.0010) [2023-12-27 01:39:49,005][105692] Updated weights for policy 0, policy_version 1410268 (0.0008) [2023-12-27 01:39:49,066][105692] Updated weights for policy 0, policy_version 1410278 (0.0008) [2023-12-27 01:39:49,131][105692] Updated weights for policy 0, policy_version 1410288 (0.0008) [2023-12-27 01:39:49,478][105620] Updated weights for policy 1, policy_version 1412505 (0.0010) [2023-12-27 01:39:49,541][105620] Updated weights for policy 1, policy_version 1412515 (0.0011) [2023-12-27 01:39:49,603][105620] Updated weights for policy 1, policy_version 1412525 (0.0011) [2023-12-27 01:39:49,662][105620] Updated weights for policy 1, policy_version 1412535 (0.0011) [2023-12-27 01:39:49,921][105692] Updated weights for policy 0, policy_version 1410298 (0.0008) [2023-12-27 01:39:49,986][105692] Updated weights for policy 0, policy_version 1410308 (0.0007) [2023-12-27 01:39:50,045][105692] Updated weights for policy 0, policy_version 1410318 (0.0008) [2023-12-27 01:39:50,329][105620] Updated weights for policy 1, policy_version 1412545 (0.0006) [2023-12-27 01:39:50,392][105620] Updated weights for policy 1, policy_version 1412555 (0.0009) [2023-12-27 01:39:50,454][105620] Updated weights for policy 1, policy_version 1412565 (0.0011) [2023-12-27 01:39:50,768][105692] Updated weights for policy 0, policy_version 1410328 (0.0006) [2023-12-27 01:39:50,826][105692] Updated weights for policy 0, policy_version 1410338 (0.0006) [2023-12-27 01:39:50,891][105692] Updated weights for policy 0, policy_version 1410348 (0.0008) [2023-12-27 01:39:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 722763776. Throughput: 0: 9688.4, 1: 9882.4. Samples: 722752456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:39:51,063][104569] Avg episode reward: [(0, '7247.561'), (1, '8717.127')] [2023-12-27 01:39:51,219][105620] Updated weights for policy 1, policy_version 1412575 (0.0010) [2023-12-27 01:39:51,283][105620] Updated weights for policy 1, policy_version 1412585 (0.0009) [2023-12-27 01:39:51,340][105620] Updated weights for policy 1, policy_version 1412595 (0.0008) [2023-12-27 01:39:51,603][105692] Updated weights for policy 0, policy_version 1410358 (0.0008) [2023-12-27 01:39:51,671][105692] Updated weights for policy 0, policy_version 1410368 (0.0007) [2023-12-27 01:39:51,743][105692] Updated weights for policy 0, policy_version 1410378 (0.0008) [2023-12-27 01:39:52,145][105620] Updated weights for policy 1, policy_version 1412605 (0.0007) [2023-12-27 01:39:52,212][105620] Updated weights for policy 1, policy_version 1412615 (0.0005) [2023-12-27 01:39:52,273][105620] Updated weights for policy 1, policy_version 1412625 (0.0009) [2023-12-27 01:39:52,427][105692] Updated weights for policy 0, policy_version 1410388 (0.0008) [2023-12-27 01:39:52,490][105692] Updated weights for policy 0, policy_version 1410398 (0.0009) [2023-12-27 01:39:52,553][105692] Updated weights for policy 0, policy_version 1410408 (0.0009) [2023-12-27 01:39:53,038][105620] Updated weights for policy 1, policy_version 1412635 (0.0008) [2023-12-27 01:39:53,092][105620] Updated weights for policy 1, policy_version 1412645 (0.0009) [2023-12-27 01:39:53,155][105620] Updated weights for policy 1, policy_version 1412655 (0.0007) [2023-12-27 01:39:53,244][105692] Updated weights for policy 0, policy_version 1410418 (0.0009) [2023-12-27 01:39:53,306][105692] Updated weights for policy 0, policy_version 1410428 (0.0010) [2023-12-27 01:39:53,359][105692] Updated weights for policy 0, policy_version 1410438 (0.0009) [2023-12-27 01:39:53,410][105692] Updated weights for policy 0, policy_version 1410448 (0.0009) [2023-12-27 01:39:53,784][105620] Updated weights for policy 1, policy_version 1412665 (0.0006) [2023-12-27 01:39:53,842][105620] Updated weights for policy 1, policy_version 1412675 (0.0009) [2023-12-27 01:39:53,888][105620] Updated weights for policy 1, policy_version 1412685 (0.0008) [2023-12-27 01:39:53,933][105620] Updated weights for policy 1, policy_version 1412695 (0.0005) [2023-12-27 01:39:54,246][105692] Updated weights for policy 0, policy_version 1410458 (0.0009) [2023-12-27 01:39:54,307][105692] Updated weights for policy 0, policy_version 1410468 (0.0009) [2023-12-27 01:39:54,369][105692] Updated weights for policy 0, policy_version 1410478 (0.0009) [2023-12-27 01:39:54,570][105620] Updated weights for policy 1, policy_version 1412705 (0.0009) [2023-12-27 01:39:54,611][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000004 [2023-12-27 01:39:55,161][105692] Updated weights for policy 0, policy_version 1410488 (0.0010) [2023-12-27 01:39:55,213][105692] Updated weights for policy 0, policy_version 1410498 (0.0009) [2023-12-27 01:39:55,263][105620] Updated weights for policy 1, policy_version 1412715 (0.0009) [2023-12-27 01:39:55,263][105692] Updated weights for policy 0, policy_version 1410508 (0.0008) [2023-12-27 01:39:55,314][105620] Updated weights for policy 1, policy_version 1412725 (0.0005) [2023-12-27 01:39:55,367][105620] Updated weights for policy 1, policy_version 1412735 (0.0006) [2023-12-27 01:39:56,027][105620] Updated weights for policy 1, policy_version 1412745 (0.0009) [2023-12-27 01:39:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19438.7). Total num frames: 722853888. Throughput: 0: 9720.4, 1: 9898.0. Samples: 722868592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:39:56,062][104569] Avg episode reward: [(0, '7612.854'), (1, '8897.806')] [2023-12-27 01:39:56,082][105620] Updated weights for policy 1, policy_version 1412755 (0.0008) [2023-12-27 01:39:56,087][105692] Updated weights for policy 0, policy_version 1410518 (0.0008) [2023-12-27 01:39:56,131][105620] Updated weights for policy 1, policy_version 1412765 (0.0006) [2023-12-27 01:39:56,140][105692] Updated weights for policy 0, policy_version 1410528 (0.0007) [2023-12-27 01:39:56,183][105620] Updated weights for policy 1, policy_version 1412775 (0.0008) [2023-12-27 01:39:56,190][105692] Updated weights for policy 0, policy_version 1410538 (0.0007) [2023-12-27 01:39:56,959][105620] Updated weights for policy 1, policy_version 1412785 (0.0007) [2023-12-27 01:39:56,961][105692] Updated weights for policy 0, policy_version 1410548 (0.0009) [2023-12-27 01:39:57,010][105620] Updated weights for policy 1, policy_version 1412795 (0.0006) [2023-12-27 01:39:57,017][105692] Updated weights for policy 0, policy_version 1410558 (0.0007) [2023-12-27 01:39:57,057][105620] Updated weights for policy 1, policy_version 1412805 (0.0007) [2023-12-27 01:39:57,059][105692] Updated weights for policy 0, policy_version 1410568 (0.0007) [2023-12-27 01:39:57,767][105620] Updated weights for policy 1, policy_version 1412815 (0.0008) [2023-12-27 01:39:57,813][105620] Updated weights for policy 1, policy_version 1412825 (0.0008) [2023-12-27 01:39:57,841][105692] Updated weights for policy 0, policy_version 1410578 (0.0008) [2023-12-27 01:39:57,864][105620] Updated weights for policy 1, policy_version 1412835 (0.0007) [2023-12-27 01:39:57,896][105692] Updated weights for policy 0, policy_version 1410588 (0.0007) [2023-12-27 01:39:57,946][105692] Updated weights for policy 0, policy_version 1410598 (0.0009) [2023-12-27 01:39:57,998][105692] Updated weights for policy 0, policy_version 1410608 (0.0010) [2023-12-27 01:39:58,647][105620] Updated weights for policy 1, policy_version 1412845 (0.0007) [2023-12-27 01:39:58,709][105620] Updated weights for policy 1, policy_version 1412855 (0.0008) [2023-12-27 01:39:58,773][105620] Updated weights for policy 1, policy_version 1412865 (0.0007) [2023-12-27 01:39:58,917][105692] Updated weights for policy 0, policy_version 1410618 (0.0008) [2023-12-27 01:39:58,982][105692] Updated weights for policy 0, policy_version 1410628 (0.0007) [2023-12-27 01:39:59,040][105692] Updated weights for policy 0, policy_version 1410638 (0.0007) [2023-12-27 01:39:59,526][105620] Updated weights for policy 1, policy_version 1412875 (0.0008) [2023-12-27 01:39:59,592][105620] Updated weights for policy 1, policy_version 1412885 (0.0010) [2023-12-27 01:39:59,659][105620] Updated weights for policy 1, policy_version 1412895 (0.0010) [2023-12-27 01:39:59,733][105692] Updated weights for policy 0, policy_version 1410648 (0.0006) [2023-12-27 01:39:59,797][105692] Updated weights for policy 0, policy_version 1410658 (0.0006) [2023-12-27 01:39:59,869][105692] Updated weights for policy 0, policy_version 1410668 (0.0008) [2023-12-27 01:40:00,461][105620] Updated weights for policy 1, policy_version 1412905 (0.0009) [2023-12-27 01:40:00,519][105620] Updated weights for policy 1, policy_version 1412915 (0.0009) [2023-12-27 01:40:00,557][105692] Updated weights for policy 0, policy_version 1410678 (0.0008) [2023-12-27 01:40:00,577][105620] Updated weights for policy 1, policy_version 1412925 (0.0006) [2023-12-27 01:40:00,618][105692] Updated weights for policy 0, policy_version 1410688 (0.0006) [2023-12-27 01:40:00,637][105620] Updated weights for policy 1, policy_version 1412935 (0.0009) [2023-12-27 01:40:00,669][105692] Updated weights for policy 0, policy_version 1410698 (0.0008) [2023-12-27 01:40:01,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 722952192. Throughput: 0: 9733.2, 1: 9842.7. Samples: 722923956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:40:01,062][104569] Avg episode reward: [(0, '8068.683'), (1, '8991.449')] [2023-12-27 01:40:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001410704_361193472.pth... [2023-12-27 01:40:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001412936_361758720.pth... [2023-12-27 01:40:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001411800_361463808.pth [2023-12-27 01:40:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001409584_360906752.pth [2023-12-27 01:40:01,392][105692] Updated weights for policy 0, policy_version 1410708 (0.0008) [2023-12-27 01:40:01,443][105620] Updated weights for policy 1, policy_version 1412945 (0.0007) [2023-12-27 01:40:01,454][105692] Updated weights for policy 0, policy_version 1410718 (0.0007) [2023-12-27 01:40:01,500][105620] Updated weights for policy 1, policy_version 1412955 (0.0006) [2023-12-27 01:40:01,513][105692] Updated weights for policy 0, policy_version 1410728 (0.0009) [2023-12-27 01:40:01,555][105620] Updated weights for policy 1, policy_version 1412965 (0.0005) [2023-12-27 01:40:02,227][105692] Updated weights for policy 0, policy_version 1410738 (0.0008) [2023-12-27 01:40:02,289][105692] Updated weights for policy 0, policy_version 1410748 (0.0010) [2023-12-27 01:40:02,341][105620] Updated weights for policy 1, policy_version 1412975 (0.0007) [2023-12-27 01:40:02,342][105692] Updated weights for policy 0, policy_version 1410758 (0.0007) [2023-12-27 01:40:02,404][105692] Updated weights for policy 0, policy_version 1410768 (0.0006) [2023-12-27 01:40:02,405][105620] Updated weights for policy 1, policy_version 1412985 (0.0008) [2023-12-27 01:40:02,454][105620] Updated weights for policy 1, policy_version 1412995 (0.0009) [2023-12-27 01:40:03,098][105692] Updated weights for policy 0, policy_version 1410778 (0.0005) [2023-12-27 01:40:03,154][105692] Updated weights for policy 0, policy_version 1410788 (0.0008) [2023-12-27 01:40:03,204][105692] Updated weights for policy 0, policy_version 1410798 (0.0006) [2023-12-27 01:40:03,246][105620] Updated weights for policy 1, policy_version 1413005 (0.0009) [2023-12-27 01:40:03,299][105620] Updated weights for policy 1, policy_version 1413016 (0.0010) [2023-12-27 01:40:03,352][105620] Updated weights for policy 1, policy_version 1413028 (0.0010) [2023-12-27 01:40:03,799][105692] Updated weights for policy 0, policy_version 1410808 (0.0005) [2023-12-27 01:40:03,861][105692] Updated weights for policy 0, policy_version 1410818 (0.0008) [2023-12-27 01:40:03,922][105692] Updated weights for policy 0, policy_version 1410828 (0.0008) [2023-12-27 01:40:04,189][105620] Updated weights for policy 1, policy_version 1413039 (0.0008) [2023-12-27 01:40:04,251][105620] Updated weights for policy 1, policy_version 1413049 (0.0009) [2023-12-27 01:40:04,316][105620] Updated weights for policy 1, policy_version 1413059 (0.0008) [2023-12-27 01:40:04,564][105692] Updated weights for policy 0, policy_version 1410838 (0.0009) [2023-12-27 01:40:04,625][105692] Updated weights for policy 0, policy_version 1410848 (0.0008) [2023-12-27 01:40:04,687][105692] Updated weights for policy 0, policy_version 1410858 (0.0009) [2023-12-27 01:40:05,125][105620] Updated weights for policy 1, policy_version 1413069 (0.0009) [2023-12-27 01:40:05,179][105620] Updated weights for policy 1, policy_version 1413079 (0.0009) [2023-12-27 01:40:05,237][105620] Updated weights for policy 1, policy_version 1413089 (0.0009) [2023-12-27 01:40:05,380][105692] Updated weights for policy 0, policy_version 1410868 (0.0009) [2023-12-27 01:40:05,444][105692] Updated weights for policy 0, policy_version 1410878 (0.0009) [2023-12-27 01:40:05,503][105692] Updated weights for policy 0, policy_version 1410888 (0.0009) [2023-12-27 01:40:05,998][105620] Updated weights for policy 1, policy_version 1413099 (0.0009) [2023-12-27 01:40:06,056][105620] Updated weights for policy 1, policy_version 1413109 (0.0009) [2023-12-27 01:40:06,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19251.1, 300 sec: 19466.4). Total num frames: 723042304. Throughput: 0: 9821.5, 1: 9683.8. Samples: 723036740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:40:06,063][104569] Avg episode reward: [(0, '7802.229'), (1, '8899.348')] [2023-12-27 01:40:06,121][105620] Updated weights for policy 1, policy_version 1413119 (0.0009) [2023-12-27 01:40:06,212][105692] Updated weights for policy 0, policy_version 1410898 (0.0010) [2023-12-27 01:40:06,268][105692] Updated weights for policy 0, policy_version 1410908 (0.0009) [2023-12-27 01:40:06,316][105692] Updated weights for policy 0, policy_version 1410918 (0.0009) [2023-12-27 01:40:06,365][105692] Updated weights for policy 0, policy_version 1410928 (0.0009) [2023-12-27 01:40:06,834][105620] Updated weights for policy 1, policy_version 1413129 (0.0009) [2023-12-27 01:40:06,892][105620] Updated weights for policy 1, policy_version 1413139 (0.0009) [2023-12-27 01:40:06,954][105620] Updated weights for policy 1, policy_version 1413149 (0.0009) [2023-12-27 01:40:07,016][105620] Updated weights for policy 1, policy_version 1413159 (0.0009) [2023-12-27 01:40:07,171][105692] Updated weights for policy 0, policy_version 1410938 (0.0006) [2023-12-27 01:40:07,236][105692] Updated weights for policy 0, policy_version 1410948 (0.0006) [2023-12-27 01:40:07,307][105692] Updated weights for policy 0, policy_version 1410958 (0.0005) [2023-12-27 01:40:07,797][105692] Updated weights for policy 0, policy_version 1410968 (0.0005) [2023-12-27 01:40:07,850][105692] Updated weights for policy 0, policy_version 1410978 (0.0005) [2023-12-27 01:40:07,900][105620] Updated weights for policy 1, policy_version 1413169 (0.0008) [2023-12-27 01:40:07,905][105692] Updated weights for policy 0, policy_version 1410988 (0.0005) [2023-12-27 01:40:07,961][105620] Updated weights for policy 1, policy_version 1413179 (0.0008) [2023-12-27 01:40:08,024][105620] Updated weights for policy 1, policy_version 1413189 (0.0009) [2023-12-27 01:40:08,587][105692] Updated weights for policy 0, policy_version 1410998 (0.0007) [2023-12-27 01:40:08,652][105692] Updated weights for policy 0, policy_version 1411008 (0.0007) [2023-12-27 01:40:08,718][105692] Updated weights for policy 0, policy_version 1411018 (0.0008) [2023-12-27 01:40:08,833][105620] Updated weights for policy 1, policy_version 1413199 (0.0008) [2023-12-27 01:40:08,891][105620] Updated weights for policy 1, policy_version 1413209 (0.0009) [2023-12-27 01:40:08,953][105620] Updated weights for policy 1, policy_version 1413219 (0.0009) [2023-12-27 01:40:09,419][105692] Updated weights for policy 0, policy_version 1411028 (0.0008) [2023-12-27 01:40:09,479][105692] Updated weights for policy 0, policy_version 1411038 (0.0008) [2023-12-27 01:40:09,533][105692] Updated weights for policy 0, policy_version 1411048 (0.0009) [2023-12-27 01:40:09,758][105620] Updated weights for policy 1, policy_version 1413229 (0.0009) [2023-12-27 01:40:09,825][105620] Updated weights for policy 1, policy_version 1413239 (0.0009) [2023-12-27 01:40:09,895][105620] Updated weights for policy 1, policy_version 1413249 (0.0009) [2023-12-27 01:40:10,254][105692] Updated weights for policy 0, policy_version 1411058 (0.0008) [2023-12-27 01:40:10,306][105692] Updated weights for policy 0, policy_version 1411068 (0.0009) [2023-12-27 01:40:10,373][105692] Updated weights for policy 0, policy_version 1411078 (0.0009) [2023-12-27 01:40:10,436][105692] Updated weights for policy 0, policy_version 1411088 (0.0009) [2023-12-27 01:40:10,668][105620] Updated weights for policy 1, policy_version 1413259 (0.0009) [2023-12-27 01:40:10,723][105620] Updated weights for policy 1, policy_version 1413269 (0.0007) [2023-12-27 01:40:10,787][105620] Updated weights for policy 1, policy_version 1413279 (0.0009) [2023-12-27 01:40:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19438.7). Total num frames: 723140608. Throughput: 0: 9912.5, 1: 9571.0. Samples: 723149628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:40:11,062][104569] Avg episode reward: [(0, '7981.423'), (1, '9083.553')] [2023-12-27 01:40:11,202][105692] Updated weights for policy 0, policy_version 1411098 (0.0008) [2023-12-27 01:40:11,262][105692] Updated weights for policy 0, policy_version 1411108 (0.0008) [2023-12-27 01:40:11,323][105692] Updated weights for policy 0, policy_version 1411118 (0.0008) [2023-12-27 01:40:11,569][105620] Updated weights for policy 1, policy_version 1413289 (0.0009) [2023-12-27 01:40:11,644][105620] Updated weights for policy 1, policy_version 1413299 (0.0009) [2023-12-27 01:40:11,701][105620] Updated weights for policy 1, policy_version 1413309 (0.0009) [2023-12-27 01:40:11,764][105620] Updated weights for policy 1, policy_version 1413319 (0.0009) [2023-12-27 01:40:12,082][105692] Updated weights for policy 0, policy_version 1411128 (0.0009) [2023-12-27 01:40:12,141][105692] Updated weights for policy 0, policy_version 1411138 (0.0009) [2023-12-27 01:40:12,202][105692] Updated weights for policy 0, policy_version 1411148 (0.0009) [2023-12-27 01:40:12,566][105620] Updated weights for policy 1, policy_version 1413329 (0.0009) [2023-12-27 01:40:12,625][105620] Updated weights for policy 1, policy_version 1413339 (0.0009) [2023-12-27 01:40:12,685][105620] Updated weights for policy 1, policy_version 1413349 (0.0009) [2023-12-27 01:40:12,948][105692] Updated weights for policy 0, policy_version 1411158 (0.0009) [2023-12-27 01:40:13,010][105692] Updated weights for policy 0, policy_version 1411168 (0.0009) [2023-12-27 01:40:13,057][105585] KL-divergence is very high: 157.5094 [2023-12-27 01:40:13,065][105692] Updated weights for policy 0, policy_version 1411178 (0.0008) [2023-12-27 01:40:13,455][105620] Updated weights for policy 1, policy_version 1413359 (0.0009) [2023-12-27 01:40:13,508][105620] Updated weights for policy 1, policy_version 1413369 (0.0009) [2023-12-27 01:40:13,567][105620] Updated weights for policy 1, policy_version 1413379 (0.0009) [2023-12-27 01:40:13,801][105692] Updated weights for policy 0, policy_version 1411188 (0.0009) [2023-12-27 01:40:13,859][105692] Updated weights for policy 0, policy_version 1411198 (0.0009) [2023-12-27 01:40:13,905][105692] Updated weights for policy 0, policy_version 1411208 (0.0008) [2023-12-27 01:40:14,330][105620] Updated weights for policy 1, policy_version 1413389 (0.0007) [2023-12-27 01:40:14,384][105620] Updated weights for policy 1, policy_version 1413399 (0.0006) [2023-12-27 01:40:14,438][105620] Updated weights for policy 1, policy_version 1413409 (0.0007) [2023-12-27 01:40:14,685][105692] Updated weights for policy 0, policy_version 1411218 (0.0008) [2023-12-27 01:40:14,747][105692] Updated weights for policy 0, policy_version 1411228 (0.0010) [2023-12-27 01:40:14,810][105692] Updated weights for policy 0, policy_version 1411238 (0.0008) [2023-12-27 01:40:14,861][105692] Updated weights for policy 0, policy_version 1411248 (0.0008) [2023-12-27 01:40:15,084][105620] Updated weights for policy 1, policy_version 1413419 (0.0009) [2023-12-27 01:40:15,144][105620] Updated weights for policy 1, policy_version 1413429 (0.0009) [2023-12-27 01:40:15,199][105620] Updated weights for policy 1, policy_version 1413439 (0.0009) [2023-12-27 01:40:15,631][105692] Updated weights for policy 0, policy_version 1411258 (0.0009) [2023-12-27 01:40:15,689][105692] Updated weights for policy 0, policy_version 1411268 (0.0009) [2023-12-27 01:40:15,743][105692] Updated weights for policy 0, policy_version 1411278 (0.0009) [2023-12-27 01:40:15,909][105620] Updated weights for policy 1, policy_version 1413449 (0.0009) [2023-12-27 01:40:15,964][105620] Updated weights for policy 1, policy_version 1413459 (0.0009) [2023-12-27 01:40:16,023][105620] Updated weights for policy 1, policy_version 1413469 (0.0008) [2023-12-27 01:40:16,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 723230720. Throughput: 0: 9755.1, 1: 9527.7. Samples: 723204432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:40:16,063][104569] Avg episode reward: [(0, '7884.702'), (1, '9358.140')] [2023-12-27 01:40:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001411280_361340928.pth... [2023-12-27 01:40:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001410160_361054208.pth [2023-12-27 01:40:16,087][105620] Updated weights for policy 1, policy_version 1413479 (0.0006) [2023-12-27 01:40:16,091][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001413480_361897984.pth... [2023-12-27 01:40:16,094][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001412376_361611264.pth [2023-12-27 01:40:16,503][105692] Updated weights for policy 0, policy_version 1411288 (0.0008) [2023-12-27 01:40:16,567][105692] Updated weights for policy 0, policy_version 1411298 (0.0010) [2023-12-27 01:40:16,632][105692] Updated weights for policy 0, policy_version 1411308 (0.0010) [2023-12-27 01:40:16,719][105620] Updated weights for policy 1, policy_version 1413489 (0.0007) [2023-12-27 01:40:16,777][105620] Updated weights for policy 1, policy_version 1413499 (0.0005) [2023-12-27 01:40:16,834][105620] Updated weights for policy 1, policy_version 1413509 (0.0008) [2023-12-27 01:40:17,283][105692] Updated weights for policy 0, policy_version 1411318 (0.0007) [2023-12-27 01:40:17,346][105692] Updated weights for policy 0, policy_version 1411328 (0.0005) [2023-12-27 01:40:17,409][105692] Updated weights for policy 0, policy_version 1411338 (0.0005) [2023-12-27 01:40:17,466][105620] Updated weights for policy 1, policy_version 1413519 (0.0008) [2023-12-27 01:40:17,529][105620] Updated weights for policy 1, policy_version 1413529 (0.0005) [2023-12-27 01:40:17,603][105620] Updated weights for policy 1, policy_version 1413539 (0.0006) [2023-12-27 01:40:17,905][105692] Updated weights for policy 0, policy_version 1411348 (0.0007) [2023-12-27 01:40:17,959][105692] Updated weights for policy 0, policy_version 1411358 (0.0010) [2023-12-27 01:40:18,006][105692] Updated weights for policy 0, policy_version 1411368 (0.0008) [2023-12-27 01:40:18,248][105620] Updated weights for policy 1, policy_version 1413549 (0.0009) [2023-12-27 01:40:18,299][105620] Updated weights for policy 1, policy_version 1413559 (0.0010) [2023-12-27 01:40:18,357][105620] Updated weights for policy 1, policy_version 1413569 (0.0009) [2023-12-27 01:40:18,739][105692] Updated weights for policy 0, policy_version 1411378 (0.0010) [2023-12-27 01:40:18,803][105692] Updated weights for policy 0, policy_version 1411388 (0.0011) [2023-12-27 01:40:18,862][105692] Updated weights for policy 0, policy_version 1411398 (0.0010) [2023-12-27 01:40:18,929][105692] Updated weights for policy 0, policy_version 1411408 (0.0010) [2023-12-27 01:40:19,062][105620] Updated weights for policy 1, policy_version 1413579 (0.0009) [2023-12-27 01:40:19,107][105620] Updated weights for policy 1, policy_version 1413589 (0.0010) [2023-12-27 01:40:19,159][105620] Updated weights for policy 1, policy_version 1413599 (0.0010) [2023-12-27 01:40:19,617][105692] Updated weights for policy 0, policy_version 1411418 (0.0010) [2023-12-27 01:40:19,680][105692] Updated weights for policy 0, policy_version 1411428 (0.0009) [2023-12-27 01:40:19,746][105692] Updated weights for policy 0, policy_version 1411438 (0.0008) [2023-12-27 01:40:19,846][105620] Updated weights for policy 1, policy_version 1413609 (0.0010) [2023-12-27 01:40:19,914][105620] Updated weights for policy 1, policy_version 1413619 (0.0009) [2023-12-27 01:40:19,982][105620] Updated weights for policy 1, policy_version 1413629 (0.0007) [2023-12-27 01:40:20,045][105620] Updated weights for policy 1, policy_version 1413639 (0.0009) [2023-12-27 01:40:20,606][105692] Updated weights for policy 0, policy_version 1411448 (0.0012) [2023-12-27 01:40:20,676][105692] Updated weights for policy 0, policy_version 1411458 (0.0009) [2023-12-27 01:40:20,736][105692] Updated weights for policy 0, policy_version 1411468 (0.0008) [2023-12-27 01:40:20,758][105620] Updated weights for policy 1, policy_version 1413649 (0.0009) [2023-12-27 01:40:20,818][105620] Updated weights for policy 1, policy_version 1413659 (0.0009) [2023-12-27 01:40:20,880][105620] Updated weights for policy 1, policy_version 1413669 (0.0009) [2023-12-27 01:40:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 723337216. Throughput: 0: 9704.6, 1: 9584.5. Samples: 723325316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:40:21,062][104569] Avg episode reward: [(0, '8346.099'), (1, '9265.895')] [2023-12-27 01:40:21,506][105692] Updated weights for policy 0, policy_version 1411478 (0.0008) [2023-12-27 01:40:21,571][105692] Updated weights for policy 0, policy_version 1411488 (0.0009) [2023-12-27 01:40:21,611][105620] Updated weights for policy 1, policy_version 1413679 (0.0008) [2023-12-27 01:40:21,640][105692] Updated weights for policy 0, policy_version 1411498 (0.0008) [2023-12-27 01:40:21,681][105620] Updated weights for policy 1, policy_version 1413689 (0.0008) [2023-12-27 01:40:21,751][105620] Updated weights for policy 1, policy_version 1413699 (0.0008) [2023-12-27 01:40:22,456][105692] Updated weights for policy 0, policy_version 1411508 (0.0009) [2023-12-27 01:40:22,464][105620] Updated weights for policy 1, policy_version 1413709 (0.0008) [2023-12-27 01:40:22,518][105620] Updated weights for policy 1, policy_version 1413719 (0.0008) [2023-12-27 01:40:22,520][105692] Updated weights for policy 0, policy_version 1411518 (0.0010) [2023-12-27 01:40:22,579][105620] Updated weights for policy 1, policy_version 1413729 (0.0006) [2023-12-27 01:40:22,581][105692] Updated weights for policy 0, policy_version 1411528 (0.0010) [2023-12-27 01:40:23,278][105620] Updated weights for policy 1, policy_version 1413739 (0.0006) [2023-12-27 01:40:23,315][105692] Updated weights for policy 0, policy_version 1411538 (0.0011) [2023-12-27 01:40:23,333][105620] Updated weights for policy 1, policy_version 1413749 (0.0011) [2023-12-27 01:40:23,368][105692] Updated weights for policy 0, policy_version 1411548 (0.0010) [2023-12-27 01:40:23,390][105620] Updated weights for policy 1, policy_version 1413759 (0.0011) [2023-12-27 01:40:23,417][105692] Updated weights for policy 0, policy_version 1411558 (0.0010) [2023-12-27 01:40:23,472][105692] Updated weights for policy 0, policy_version 1411568 (0.0010) [2023-12-27 01:40:23,976][105620] Updated weights for policy 1, policy_version 1413769 (0.0011) [2023-12-27 01:40:24,035][105620] Updated weights for policy 1, policy_version 1413779 (0.0009) [2023-12-27 01:40:24,093][105620] Updated weights for policy 1, policy_version 1413789 (0.0010) [2023-12-27 01:40:24,159][105620] Updated weights for policy 1, policy_version 1413799 (0.0011) [2023-12-27 01:40:24,246][105692] Updated weights for policy 0, policy_version 1411578 (0.0010) [2023-12-27 01:40:24,304][105692] Updated weights for policy 0, policy_version 1411588 (0.0010) [2023-12-27 01:40:24,366][105692] Updated weights for policy 0, policy_version 1411598 (0.0010) [2023-12-27 01:40:24,837][105620] Updated weights for policy 1, policy_version 1413809 (0.0009) [2023-12-27 01:40:24,899][105620] Updated weights for policy 1, policy_version 1413819 (0.0010) [2023-12-27 01:40:24,957][105620] Updated weights for policy 1, policy_version 1413829 (0.0011) [2023-12-27 01:40:25,001][105692] Updated weights for policy 0, policy_version 1411608 (0.0008) [2023-12-27 01:40:25,050][105692] Updated weights for policy 0, policy_version 1411618 (0.0008) [2023-12-27 01:40:25,098][105692] Updated weights for policy 0, policy_version 1411628 (0.0008) [2023-12-27 01:40:25,685][105620] Updated weights for policy 1, policy_version 1413839 (0.0010) [2023-12-27 01:40:25,694][105692] Updated weights for policy 0, policy_version 1411638 (0.0006) [2023-12-27 01:40:25,734][105620] Updated weights for policy 1, policy_version 1413849 (0.0010) [2023-12-27 01:40:25,741][105692] Updated weights for policy 0, policy_version 1411648 (0.0005) [2023-12-27 01:40:25,785][105620] Updated weights for policy 1, policy_version 1413859 (0.0010) [2023-12-27 01:40:25,791][105692] Updated weights for policy 0, policy_version 1411658 (0.0005) [2023-12-27 01:40:26,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 723435520. Throughput: 0: 9645.9, 1: 9607.9. Samples: 723440764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:40:26,063][104569] Avg episode reward: [(0, '8527.602'), (1, '8992.138')] [2023-12-27 01:40:26,325][105692] Updated weights for policy 0, policy_version 1411668 (0.0005) [2023-12-27 01:40:26,396][105692] Updated weights for policy 0, policy_version 1411678 (0.0005) [2023-12-27 01:40:26,453][105692] Updated weights for policy 0, policy_version 1411688 (0.0005) [2023-12-27 01:40:26,514][105620] Updated weights for policy 1, policy_version 1413869 (0.0009) [2023-12-27 01:40:26,569][105620] Updated weights for policy 1, policy_version 1413879 (0.0005) [2023-12-27 01:40:26,634][105620] Updated weights for policy 1, policy_version 1413889 (0.0005) [2023-12-27 01:40:27,107][105692] Updated weights for policy 0, policy_version 1411698 (0.0010) [2023-12-27 01:40:27,164][105692] Updated weights for policy 0, policy_version 1411708 (0.0010) [2023-12-27 01:40:27,186][105585] KL-divergence is very high: 104.0871 [2023-12-27 01:40:27,220][105692] Updated weights for policy 0, policy_version 1411718 (0.0010) [2023-12-27 01:40:27,278][105692] Updated weights for policy 0, policy_version 1411728 (0.0010) [2023-12-27 01:40:27,330][105620] Updated weights for policy 1, policy_version 1413899 (0.0009) [2023-12-27 01:40:27,377][105620] Updated weights for policy 1, policy_version 1413909 (0.0007) [2023-12-27 01:40:27,431][105620] Updated weights for policy 1, policy_version 1413919 (0.0009) [2023-12-27 01:40:27,832][105692] Updated weights for policy 0, policy_version 1411738 (0.0010) [2023-12-27 01:40:27,886][105692] Updated weights for policy 0, policy_version 1411748 (0.0007) [2023-12-27 01:40:27,932][105692] Updated weights for policy 0, policy_version 1411758 (0.0005) [2023-12-27 01:40:28,164][105620] Updated weights for policy 1, policy_version 1413929 (0.0010) [2023-12-27 01:40:28,216][105620] Updated weights for policy 1, policy_version 1413939 (0.0010) [2023-12-27 01:40:28,271][105620] Updated weights for policy 1, policy_version 1413949 (0.0010) [2023-12-27 01:40:28,322][105620] Updated weights for policy 1, policy_version 1413959 (0.0010) [2023-12-27 01:40:28,592][105692] Updated weights for policy 0, policy_version 1411768 (0.0005) [2023-12-27 01:40:28,646][105692] Updated weights for policy 0, policy_version 1411778 (0.0005) [2023-12-27 01:40:28,695][105692] Updated weights for policy 0, policy_version 1411788 (0.0008) [2023-12-27 01:40:29,084][105620] Updated weights for policy 1, policy_version 1413969 (0.0010) [2023-12-27 01:40:29,140][105620] Updated weights for policy 1, policy_version 1413979 (0.0010) [2023-12-27 01:40:29,191][105620] Updated weights for policy 1, policy_version 1413989 (0.0010) [2023-12-27 01:40:29,405][105692] Updated weights for policy 0, policy_version 1411798 (0.0008) [2023-12-27 01:40:29,471][105692] Updated weights for policy 0, policy_version 1411808 (0.0005) [2023-12-27 01:40:29,529][105692] Updated weights for policy 0, policy_version 1411818 (0.0005) [2023-12-27 01:40:29,978][105620] Updated weights for policy 1, policy_version 1413999 (0.0009) [2023-12-27 01:40:30,040][105620] Updated weights for policy 1, policy_version 1414009 (0.0008) [2023-12-27 01:40:30,102][105620] Updated weights for policy 1, policy_version 1414019 (0.0008) [2023-12-27 01:40:30,105][105692] Updated weights for policy 0, policy_version 1411828 (0.0007) [2023-12-27 01:40:30,156][105692] Updated weights for policy 0, policy_version 1411838 (0.0010) [2023-12-27 01:40:30,205][105692] Updated weights for policy 0, policy_version 1411848 (0.0010) [2023-12-27 01:40:30,831][105620] Updated weights for policy 1, policy_version 1414029 (0.0007) [2023-12-27 01:40:30,873][105692] Updated weights for policy 0, policy_version 1411858 (0.0010) [2023-12-27 01:40:30,880][105620] Updated weights for policy 1, policy_version 1414039 (0.0008) [2023-12-27 01:40:30,934][105692] Updated weights for policy 0, policy_version 1411868 (0.0005) [2023-12-27 01:40:30,936][105620] Updated weights for policy 1, policy_version 1414049 (0.0009) [2023-12-27 01:40:30,983][105692] Updated weights for policy 0, policy_version 1411878 (0.0008) [2023-12-27 01:40:31,041][105692] Updated weights for policy 0, policy_version 1411888 (0.0010) [2023-12-27 01:40:31,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 723542016. Throughput: 0: 9768.3, 1: 9646.3. Samples: 723504316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:40:31,063][104569] Avg episode reward: [(0, '6976.186'), (1, '8992.015')] [2023-12-27 01:40:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001411888_361496576.pth... [2023-12-27 01:40:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001414056_362045440.pth... [2023-12-27 01:40:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001410704_361193472.pth [2023-12-27 01:40:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001412936_361758720.pth [2023-12-27 01:40:31,730][105620] Updated weights for policy 1, policy_version 1414059 (0.0008) [2023-12-27 01:40:31,750][105692] Updated weights for policy 0, policy_version 1411898 (0.0011) [2023-12-27 01:40:31,788][105620] Updated weights for policy 1, policy_version 1414069 (0.0005) [2023-12-27 01:40:31,806][105692] Updated weights for policy 0, policy_version 1411908 (0.0011) [2023-12-27 01:40:31,841][105586] KL-divergence is very high: 111.3109 [2023-12-27 01:40:31,845][105620] Updated weights for policy 1, policy_version 1414079 (0.0005) [2023-12-27 01:40:31,865][105692] Updated weights for policy 0, policy_version 1411918 (0.0011) [2023-12-27 01:40:32,556][105692] Updated weights for policy 0, policy_version 1411928 (0.0010) [2023-12-27 01:40:32,576][105620] Updated weights for policy 1, policy_version 1414089 (0.0010) [2023-12-27 01:40:32,619][105692] Updated weights for policy 0, policy_version 1411938 (0.0008) [2023-12-27 01:40:32,639][105620] Updated weights for policy 1, policy_version 1414099 (0.0010) [2023-12-27 01:40:32,672][105692] Updated weights for policy 0, policy_version 1411948 (0.0010) [2023-12-27 01:40:32,700][105620] Updated weights for policy 1, policy_version 1414109 (0.0006) [2023-12-27 01:40:32,765][105620] Updated weights for policy 1, policy_version 1414119 (0.0010) [2023-12-27 01:40:33,329][105620] Updated weights for policy 1, policy_version 1414129 (0.0006) [2023-12-27 01:40:33,396][105620] Updated weights for policy 1, policy_version 1414139 (0.0008) [2023-12-27 01:40:33,396][105692] Updated weights for policy 0, policy_version 1411958 (0.0007) [2023-12-27 01:40:33,456][105692] Updated weights for policy 0, policy_version 1411968 (0.0007) [2023-12-27 01:40:33,457][105620] Updated weights for policy 1, policy_version 1414149 (0.0010) [2023-12-27 01:40:33,511][105692] Updated weights for policy 0, policy_version 1411978 (0.0005) [2023-12-27 01:40:34,004][105692] Updated weights for policy 0, policy_version 1411988 (0.0005) [2023-12-27 01:40:34,031][105620] Updated weights for policy 1, policy_version 1414159 (0.0007) [2023-12-27 01:40:34,057][105692] Updated weights for policy 0, policy_version 1411998 (0.0006) [2023-12-27 01:40:34,089][105620] Updated weights for policy 1, policy_version 1414169 (0.0005) [2023-12-27 01:40:34,109][105692] Updated weights for policy 0, policy_version 1412008 (0.0010) [2023-12-27 01:40:34,143][105620] Updated weights for policy 1, policy_version 1414179 (0.0010) [2023-12-27 01:40:34,751][105620] Updated weights for policy 1, policy_version 1414189 (0.0009) [2023-12-27 01:40:34,787][105692] Updated weights for policy 0, policy_version 1412018 (0.0010) [2023-12-27 01:40:34,799][105620] Updated weights for policy 1, policy_version 1414199 (0.0006) [2023-12-27 01:40:34,846][105692] Updated weights for policy 0, policy_version 1412028 (0.0011) [2023-12-27 01:40:34,849][105620] Updated weights for policy 1, policy_version 1414209 (0.0006) [2023-12-27 01:40:34,911][105692] Updated weights for policy 0, policy_version 1412038 (0.0010) [2023-12-27 01:40:34,979][105692] Updated weights for policy 0, policy_version 1412048 (0.0011) [2023-12-27 01:40:35,524][105620] Updated weights for policy 1, policy_version 1414219 (0.0005) [2023-12-27 01:40:35,583][105620] Updated weights for policy 1, policy_version 1414229 (0.0005) [2023-12-27 01:40:35,646][105620] Updated weights for policy 1, policy_version 1414239 (0.0008) [2023-12-27 01:40:35,700][105692] Updated weights for policy 0, policy_version 1412058 (0.0011) [2023-12-27 01:40:35,763][105692] Updated weights for policy 0, policy_version 1412068 (0.0011) [2023-12-27 01:40:35,828][105692] Updated weights for policy 0, policy_version 1412078 (0.0011) [2023-12-27 01:40:36,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 723640320. Throughput: 0: 9849.4, 1: 9614.6. Samples: 723628332. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:40:36,062][104569] Avg episode reward: [(0, '7893.660'), (1, '8808.593')] [2023-12-27 01:40:36,254][105620] Updated weights for policy 1, policy_version 1414249 (0.0006) [2023-12-27 01:40:36,315][105620] Updated weights for policy 1, policy_version 1414259 (0.0008) [2023-12-27 01:40:36,379][105620] Updated weights for policy 1, policy_version 1414269 (0.0008) [2023-12-27 01:40:36,438][105620] Updated weights for policy 1, policy_version 1414279 (0.0008) [2023-12-27 01:40:36,591][105692] Updated weights for policy 0, policy_version 1412088 (0.0011) [2023-12-27 01:40:36,648][105692] Updated weights for policy 0, policy_version 1412098 (0.0011) [2023-12-27 01:40:36,702][105692] Updated weights for policy 0, policy_version 1412108 (0.0009) [2023-12-27 01:40:37,151][105620] Updated weights for policy 1, policy_version 1414289 (0.0008) [2023-12-27 01:40:37,203][105620] Updated weights for policy 1, policy_version 1414299 (0.0008) [2023-12-27 01:40:37,258][105620] Updated weights for policy 1, policy_version 1414309 (0.0008) [2023-12-27 01:40:37,444][105692] Updated weights for policy 0, policy_version 1412118 (0.0008) [2023-12-27 01:40:37,502][105692] Updated weights for policy 0, policy_version 1412128 (0.0006) [2023-12-27 01:40:37,564][105692] Updated weights for policy 0, policy_version 1412138 (0.0005) [2023-12-27 01:40:37,927][105620] Updated weights for policy 1, policy_version 1414320 (0.0007) [2023-12-27 01:40:37,978][105620] Updated weights for policy 1, policy_version 1414330 (0.0006) [2023-12-27 01:40:38,044][105620] Updated weights for policy 1, policy_version 1414340 (0.0009) [2023-12-27 01:40:38,136][105692] Updated weights for policy 0, policy_version 1412148 (0.0007) [2023-12-27 01:40:38,201][105692] Updated weights for policy 0, policy_version 1412158 (0.0009) [2023-12-27 01:40:38,256][105692] Updated weights for policy 0, policy_version 1412168 (0.0009) [2023-12-27 01:40:38,710][105620] Updated weights for policy 1, policy_version 1414350 (0.0009) [2023-12-27 01:40:38,769][105620] Updated weights for policy 1, policy_version 1414360 (0.0009) [2023-12-27 01:40:38,827][105620] Updated weights for policy 1, policy_version 1414370 (0.0010) [2023-12-27 01:40:39,050][105692] Updated weights for policy 0, policy_version 1412178 (0.0009) [2023-12-27 01:40:39,109][105692] Updated weights for policy 0, policy_version 1412188 (0.0007) [2023-12-27 01:40:39,172][105692] Updated weights for policy 0, policy_version 1412198 (0.0011) [2023-12-27 01:40:39,240][105692] Updated weights for policy 0, policy_version 1412208 (0.0010) [2023-12-27 01:40:39,644][105620] Updated weights for policy 1, policy_version 1414380 (0.0009) [2023-12-27 01:40:39,708][105620] Updated weights for policy 1, policy_version 1414390 (0.0008) [2023-12-27 01:40:39,780][105620] Updated weights for policy 1, policy_version 1414400 (0.0010) [2023-12-27 01:40:39,945][105692] Updated weights for policy 0, policy_version 1412218 (0.0009) [2023-12-27 01:40:40,005][105692] Updated weights for policy 0, policy_version 1412228 (0.0010) [2023-12-27 01:40:40,061][105692] Updated weights for policy 0, policy_version 1412238 (0.0009) [2023-12-27 01:40:40,583][105620] Updated weights for policy 1, policy_version 1414410 (0.0008) [2023-12-27 01:40:40,641][105620] Updated weights for policy 1, policy_version 1414420 (0.0009) [2023-12-27 01:40:40,696][105620] Updated weights for policy 1, policy_version 1414430 (0.0009) [2023-12-27 01:40:40,722][105692] Updated weights for policy 0, policy_version 1412248 (0.0009) [2023-12-27 01:40:40,755][105620] Updated weights for policy 1, policy_version 1414440 (0.0009) [2023-12-27 01:40:40,786][105692] Updated weights for policy 0, policy_version 1412258 (0.0009) [2023-12-27 01:40:40,850][105692] Updated weights for policy 0, policy_version 1412268 (0.0010) [2023-12-27 01:40:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 723738624. Throughput: 0: 9899.9, 1: 9568.4. Samples: 723744668. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:40:41,062][104569] Avg episode reward: [(0, '8167.913'), (1, '8989.914')] [2023-12-27 01:40:41,565][105620] Updated weights for policy 1, policy_version 1414450 (0.0009) [2023-12-27 01:40:41,584][105692] Updated weights for policy 0, policy_version 1412278 (0.0008) [2023-12-27 01:40:41,626][105620] Updated weights for policy 1, policy_version 1414460 (0.0008) [2023-12-27 01:40:41,650][105692] Updated weights for policy 0, policy_version 1412288 (0.0007) [2023-12-27 01:40:41,692][105620] Updated weights for policy 1, policy_version 1414470 (0.0008) [2023-12-27 01:40:41,710][105692] Updated weights for policy 0, policy_version 1412298 (0.0008) [2023-12-27 01:40:42,373][105692] Updated weights for policy 0, policy_version 1412308 (0.0008) [2023-12-27 01:40:42,435][105692] Updated weights for policy 0, policy_version 1412318 (0.0007) [2023-12-27 01:40:42,495][105692] Updated weights for policy 0, policy_version 1412328 (0.0007) [2023-12-27 01:40:42,581][105620] Updated weights for policy 1, policy_version 1414480 (0.0009) [2023-12-27 01:40:42,639][105620] Updated weights for policy 1, policy_version 1414490 (0.0009) [2023-12-27 01:40:42,709][105620] Updated weights for policy 1, policy_version 1414500 (0.0009) [2023-12-27 01:40:43,197][105692] Updated weights for policy 0, policy_version 1412338 (0.0009) [2023-12-27 01:40:43,259][105692] Updated weights for policy 0, policy_version 1412348 (0.0009) [2023-12-27 01:40:43,318][105692] Updated weights for policy 0, policy_version 1412358 (0.0009) [2023-12-27 01:40:43,369][105692] Updated weights for policy 0, policy_version 1412368 (0.0006) [2023-12-27 01:40:43,442][105620] Updated weights for policy 1, policy_version 1414510 (0.0010) [2023-12-27 01:40:43,497][105620] Updated weights for policy 1, policy_version 1414520 (0.0010) [2023-12-27 01:40:43,564][105620] Updated weights for policy 1, policy_version 1414530 (0.0005) [2023-12-27 01:40:44,046][105692] Updated weights for policy 0, policy_version 1412378 (0.0009) [2023-12-27 01:40:44,091][105692] Updated weights for policy 0, policy_version 1412388 (0.0008) [2023-12-27 01:40:44,154][105692] Updated weights for policy 0, policy_version 1412398 (0.0005) [2023-12-27 01:40:44,229][105620] Updated weights for policy 1, policy_version 1414540 (0.0008) [2023-12-27 01:40:44,294][105620] Updated weights for policy 1, policy_version 1414550 (0.0008) [2023-12-27 01:40:44,357][105620] Updated weights for policy 1, policy_version 1414560 (0.0005) [2023-12-27 01:40:44,819][105692] Updated weights for policy 0, policy_version 1412408 (0.0008) [2023-12-27 01:40:44,877][105692] Updated weights for policy 0, policy_version 1412418 (0.0008) [2023-12-27 01:40:44,944][105692] Updated weights for policy 0, policy_version 1412428 (0.0006) [2023-12-27 01:40:45,061][105620] Updated weights for policy 1, policy_version 1414570 (0.0008) [2023-12-27 01:40:45,123][105620] Updated weights for policy 1, policy_version 1414580 (0.0010) [2023-12-27 01:40:45,187][105620] Updated weights for policy 1, policy_version 1414590 (0.0009) [2023-12-27 01:40:45,250][105620] Updated weights for policy 1, policy_version 1414600 (0.0008) [2023-12-27 01:40:45,638][105692] Updated weights for policy 0, policy_version 1412438 (0.0008) [2023-12-27 01:40:45,686][105692] Updated weights for policy 0, policy_version 1412448 (0.0007) [2023-12-27 01:40:45,734][105692] Updated weights for policy 0, policy_version 1412458 (0.0006) [2023-12-27 01:40:45,834][105620] Updated weights for policy 1, policy_version 1414610 (0.0006) [2023-12-27 01:40:45,905][105620] Updated weights for policy 1, policy_version 1414620 (0.0006) [2023-12-27 01:40:45,977][105620] Updated weights for policy 1, policy_version 1414630 (0.0005) [2023-12-27 01:40:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 723836928. Throughput: 0: 9960.2, 1: 9535.2. Samples: 723801248. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:40:46,062][104569] Avg episode reward: [(0, '7794.508'), (1, '9357.873')] [2023-12-27 01:40:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001414632_362192896.pth... [2023-12-27 01:40:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001412464_361644032.pth... [2023-12-27 01:40:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001413480_361897984.pth [2023-12-27 01:40:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001411280_361340928.pth [2023-12-27 01:40:46,347][105692] Updated weights for policy 0, policy_version 1412468 (0.0007) [2023-12-27 01:40:46,394][105692] Updated weights for policy 0, policy_version 1412478 (0.0008) [2023-12-27 01:40:46,442][105692] Updated weights for policy 0, policy_version 1412488 (0.0008) [2023-12-27 01:40:46,571][105620] Updated weights for policy 1, policy_version 1414640 (0.0008) [2023-12-27 01:40:46,623][105620] Updated weights for policy 1, policy_version 1414650 (0.0009) [2023-12-27 01:40:46,685][105620] Updated weights for policy 1, policy_version 1414660 (0.0009) [2023-12-27 01:40:47,037][105692] Updated weights for policy 0, policy_version 1412498 (0.0005) [2023-12-27 01:40:47,089][105692] Updated weights for policy 0, policy_version 1412508 (0.0008) [2023-12-27 01:40:47,150][105692] Updated weights for policy 0, policy_version 1412518 (0.0010) [2023-12-27 01:40:47,204][105692] Updated weights for policy 0, policy_version 1412528 (0.0010) [2023-12-27 01:40:47,459][105620] Updated weights for policy 1, policy_version 1414670 (0.0006) [2023-12-27 01:40:47,512][105620] Updated weights for policy 1, policy_version 1414680 (0.0005) [2023-12-27 01:40:47,562][105620] Updated weights for policy 1, policy_version 1414690 (0.0005) [2023-12-27 01:40:47,858][105692] Updated weights for policy 0, policy_version 1412538 (0.0011) [2023-12-27 01:40:47,916][105692] Updated weights for policy 0, policy_version 1412548 (0.0010) [2023-12-27 01:40:47,971][105692] Updated weights for policy 0, policy_version 1412558 (0.0010) [2023-12-27 01:40:48,172][105620] Updated weights for policy 1, policy_version 1414700 (0.0006) [2023-12-27 01:40:48,224][105620] Updated weights for policy 1, policy_version 1414710 (0.0006) [2023-12-27 01:40:48,274][105620] Updated weights for policy 1, policy_version 1414720 (0.0005) [2023-12-27 01:40:48,677][105692] Updated weights for policy 0, policy_version 1412568 (0.0009) [2023-12-27 01:40:48,731][105692] Updated weights for policy 0, policy_version 1412578 (0.0010) [2023-12-27 01:40:48,788][105692] Updated weights for policy 0, policy_version 1412588 (0.0009) [2023-12-27 01:40:48,905][105620] Updated weights for policy 1, policy_version 1414730 (0.0005) [2023-12-27 01:40:48,971][105620] Updated weights for policy 1, policy_version 1414740 (0.0008) [2023-12-27 01:40:49,030][105620] Updated weights for policy 1, policy_version 1414750 (0.0006) [2023-12-27 01:40:49,091][105620] Updated weights for policy 1, policy_version 1414760 (0.0009) [2023-12-27 01:40:49,580][105692] Updated weights for policy 0, policy_version 1412598 (0.0009) [2023-12-27 01:40:49,628][105692] Updated weights for policy 0, policy_version 1412608 (0.0008) [2023-12-27 01:40:49,676][105692] Updated weights for policy 0, policy_version 1412618 (0.0007) [2023-12-27 01:40:49,812][105620] Updated weights for policy 1, policy_version 1414770 (0.0010) [2023-12-27 01:40:49,876][105620] Updated weights for policy 1, policy_version 1414780 (0.0011) [2023-12-27 01:40:49,941][105620] Updated weights for policy 1, policy_version 1414790 (0.0008) [2023-12-27 01:40:50,525][105692] Updated weights for policy 0, policy_version 1412628 (0.0008) [2023-12-27 01:40:50,590][105692] Updated weights for policy 0, policy_version 1412638 (0.0008) [2023-12-27 01:40:50,602][105620] Updated weights for policy 1, policy_version 1414800 (0.0009) [2023-12-27 01:40:50,652][105692] Updated weights for policy 0, policy_version 1412648 (0.0009) [2023-12-27 01:40:50,666][105620] Updated weights for policy 1, policy_version 1414810 (0.0008) [2023-12-27 01:40:50,728][105620] Updated weights for policy 1, policy_version 1414820 (0.0008) [2023-12-27 01:40:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 723935232. Throughput: 0: 10016.5, 1: 9718.7. Samples: 723924820. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:40:51,063][104569] Avg episode reward: [(0, '7339.972'), (1, '8992.600')] [2023-12-27 01:40:51,310][105620] Updated weights for policy 1, policy_version 1414830 (0.0007) [2023-12-27 01:40:51,378][105620] Updated weights for policy 1, policy_version 1414840 (0.0008) [2023-12-27 01:40:51,432][105692] Updated weights for policy 0, policy_version 1412658 (0.0006) [2023-12-27 01:40:51,438][105620] Updated weights for policy 1, policy_version 1414850 (0.0008) [2023-12-27 01:40:51,496][105692] Updated weights for policy 0, policy_version 1412668 (0.0008) [2023-12-27 01:40:51,562][105692] Updated weights for policy 0, policy_version 1412678 (0.0008) [2023-12-27 01:40:51,617][105692] Updated weights for policy 0, policy_version 1412688 (0.0009) [2023-12-27 01:40:52,113][105620] Updated weights for policy 1, policy_version 1414860 (0.0006) [2023-12-27 01:40:52,169][105620] Updated weights for policy 1, policy_version 1414870 (0.0006) [2023-12-27 01:40:52,227][105620] Updated weights for policy 1, policy_version 1414880 (0.0006) [2023-12-27 01:40:52,366][105692] Updated weights for policy 0, policy_version 1412698 (0.0011) [2023-12-27 01:40:52,421][105692] Updated weights for policy 0, policy_version 1412708 (0.0010) [2023-12-27 01:40:52,473][105692] Updated weights for policy 0, policy_version 1412718 (0.0010) [2023-12-27 01:40:53,014][105620] Updated weights for policy 1, policy_version 1414890 (0.0008) [2023-12-27 01:40:53,047][105692] Updated weights for policy 0, policy_version 1412728 (0.0006) [2023-12-27 01:40:53,075][105620] Updated weights for policy 1, policy_version 1414900 (0.0008) [2023-12-27 01:40:53,108][105692] Updated weights for policy 0, policy_version 1412738 (0.0005) [2023-12-27 01:40:53,131][105620] Updated weights for policy 1, policy_version 1414910 (0.0008) [2023-12-27 01:40:53,174][105692] Updated weights for policy 0, policy_version 1412748 (0.0005) [2023-12-27 01:40:53,181][105620] Updated weights for policy 1, policy_version 1414920 (0.0009) [2023-12-27 01:40:53,700][105692] Updated weights for policy 0, policy_version 1412758 (0.0006) [2023-12-27 01:40:53,759][105692] Updated weights for policy 0, policy_version 1412768 (0.0005) [2023-12-27 01:40:53,817][105692] Updated weights for policy 0, policy_version 1412778 (0.0006) [2023-12-27 01:40:53,936][105620] Updated weights for policy 1, policy_version 1414930 (0.0007) [2023-12-27 01:40:54,004][105620] Updated weights for policy 1, policy_version 1414940 (0.0007) [2023-12-27 01:40:54,058][105620] Updated weights for policy 1, policy_version 1414950 (0.0008) [2023-12-27 01:40:54,481][105692] Updated weights for policy 0, policy_version 1412788 (0.0006) [2023-12-27 01:40:54,535][105692] Updated weights for policy 0, policy_version 1412798 (0.0008) [2023-12-27 01:40:54,592][105692] Updated weights for policy 0, policy_version 1412808 (0.0006) [2023-12-27 01:40:54,770][105620] Updated weights for policy 1, policy_version 1414960 (0.0010) [2023-12-27 01:40:54,821][105620] Updated weights for policy 1, policy_version 1414970 (0.0010) [2023-12-27 01:40:54,870][105620] Updated weights for policy 1, policy_version 1414980 (0.0010) [2023-12-27 01:40:55,169][105692] Updated weights for policy 0, policy_version 1412818 (0.0010) [2023-12-27 01:40:55,234][105692] Updated weights for policy 0, policy_version 1412828 (0.0010) [2023-12-27 01:40:55,299][105692] Updated weights for policy 0, policy_version 1412838 (0.0010) [2023-12-27 01:40:55,366][105692] Updated weights for policy 0, policy_version 1412848 (0.0010) [2023-12-27 01:40:55,634][105620] Updated weights for policy 1, policy_version 1414990 (0.0010) [2023-12-27 01:40:55,699][105620] Updated weights for policy 1, policy_version 1415000 (0.0010) [2023-12-27 01:40:55,758][105620] Updated weights for policy 1, policy_version 1415010 (0.0009) [2023-12-27 01:40:56,031][105692] Updated weights for policy 0, policy_version 1412858 (0.0008) [2023-12-27 01:40:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 724033536. Throughput: 0: 10048.2, 1: 9842.3. Samples: 724044700. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:40:56,062][104569] Avg episode reward: [(0, '7339.911'), (1, '8900.013')] [2023-12-27 01:40:56,090][105692] Updated weights for policy 0, policy_version 1412868 (0.0010) [2023-12-27 01:40:56,142][105692] Updated weights for policy 0, policy_version 1412878 (0.0008) [2023-12-27 01:40:56,493][105620] Updated weights for policy 1, policy_version 1415020 (0.0010) [2023-12-27 01:40:56,541][105620] Updated weights for policy 1, policy_version 1415030 (0.0010) [2023-12-27 01:40:56,592][105620] Updated weights for policy 1, policy_version 1415040 (0.0010) [2023-12-27 01:40:56,828][105692] Updated weights for policy 0, policy_version 1412888 (0.0006) [2023-12-27 01:40:56,886][105692] Updated weights for policy 0, policy_version 1412898 (0.0006) [2023-12-27 01:40:56,944][105692] Updated weights for policy 0, policy_version 1412908 (0.0005) [2023-12-27 01:40:57,319][105620] Updated weights for policy 1, policy_version 1415050 (0.0009) [2023-12-27 01:40:57,373][105620] Updated weights for policy 1, policy_version 1415060 (0.0009) [2023-12-27 01:40:57,434][105620] Updated weights for policy 1, policy_version 1415070 (0.0010) [2023-12-27 01:40:57,494][105620] Updated weights for policy 1, policy_version 1415080 (0.0010) [2023-12-27 01:40:57,542][105692] Updated weights for policy 0, policy_version 1412918 (0.0008) [2023-12-27 01:40:57,600][105692] Updated weights for policy 0, policy_version 1412928 (0.0010) [2023-12-27 01:40:57,657][105692] Updated weights for policy 0, policy_version 1412938 (0.0010) [2023-12-27 01:40:58,234][105620] Updated weights for policy 1, policy_version 1415090 (0.0011) [2023-12-27 01:40:58,295][105620] Updated weights for policy 1, policy_version 1415100 (0.0010) [2023-12-27 01:40:58,305][105692] Updated weights for policy 0, policy_version 1412948 (0.0010) [2023-12-27 01:40:58,373][105692] Updated weights for policy 0, policy_version 1412958 (0.0009) [2023-12-27 01:40:58,388][105620] Updated weights for policy 1, policy_version 1415110 (0.0009) [2023-12-27 01:40:58,438][105692] Updated weights for policy 0, policy_version 1412968 (0.0010) [2023-12-27 01:40:59,186][105620] Updated weights for policy 1, policy_version 1415120 (0.0010) [2023-12-27 01:40:59,248][105620] Updated weights for policy 1, policy_version 1415130 (0.0009) [2023-12-27 01:40:59,286][105692] Updated weights for policy 0, policy_version 1412978 (0.0007) [2023-12-27 01:40:59,307][105620] Updated weights for policy 1, policy_version 1415140 (0.0008) [2023-12-27 01:40:59,345][105692] Updated weights for policy 0, policy_version 1412988 (0.0008) [2023-12-27 01:40:59,416][105692] Updated weights for policy 0, policy_version 1412998 (0.0008) [2023-12-27 01:40:59,463][105692] Updated weights for policy 0, policy_version 1413008 (0.0008) [2023-12-27 01:41:00,094][105620] Updated weights for policy 1, policy_version 1415150 (0.0008) [2023-12-27 01:41:00,146][105692] Updated weights for policy 0, policy_version 1413018 (0.0011) [2023-12-27 01:41:00,147][105620] Updated weights for policy 1, policy_version 1415160 (0.0010) [2023-12-27 01:41:00,195][105692] Updated weights for policy 0, policy_version 1413028 (0.0010) [2023-12-27 01:41:00,202][105620] Updated weights for policy 1, policy_version 1415170 (0.0010) [2023-12-27 01:41:00,247][105692] Updated weights for policy 0, policy_version 1413038 (0.0011) [2023-12-27 01:41:00,924][105620] Updated weights for policy 1, policy_version 1415180 (0.0008) [2023-12-27 01:41:00,971][105692] Updated weights for policy 0, policy_version 1413048 (0.0011) [2023-12-27 01:41:00,977][105620] Updated weights for policy 1, policy_version 1415190 (0.0010) [2023-12-27 01:41:01,024][105692] Updated weights for policy 0, policy_version 1413058 (0.0010) [2023-12-27 01:41:01,032][105620] Updated weights for policy 1, policy_version 1415200 (0.0007) [2023-12-27 01:41:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 724123648. Throughput: 0: 10107.0, 1: 9873.3. Samples: 724103540. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:41:01,062][104569] Avg episode reward: [(0, '7611.816'), (1, '9083.016')] [2023-12-27 01:41:01,080][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001415208_362340352.pth... [2023-12-27 01:41:01,084][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001414056_362045440.pth [2023-12-27 01:41:01,092][105692] Updated weights for policy 0, policy_version 1413068 (0.0009) [2023-12-27 01:41:01,115][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001413072_361799680.pth... [2023-12-27 01:41:01,120][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001411888_361496576.pth [2023-12-27 01:41:01,833][105620] Updated weights for policy 1, policy_version 1415210 (0.0010) [2023-12-27 01:41:01,845][105692] Updated weights for policy 0, policy_version 1413078 (0.0010) [2023-12-27 01:41:01,892][105620] Updated weights for policy 1, policy_version 1415220 (0.0008) [2023-12-27 01:41:01,901][105692] Updated weights for policy 0, policy_version 1413088 (0.0010) [2023-12-27 01:41:01,948][105620] Updated weights for policy 1, policy_version 1415230 (0.0008) [2023-12-27 01:41:01,959][105692] Updated weights for policy 0, policy_version 1413098 (0.0010) [2023-12-27 01:41:02,004][105620] Updated weights for policy 1, policy_version 1415240 (0.0007) [2023-12-27 01:41:02,673][105692] Updated weights for policy 0, policy_version 1413108 (0.0010) [2023-12-27 01:41:02,729][105692] Updated weights for policy 0, policy_version 1413118 (0.0007) [2023-12-27 01:41:02,759][105620] Updated weights for policy 1, policy_version 1415250 (0.0007) [2023-12-27 01:41:02,790][105692] Updated weights for policy 0, policy_version 1413128 (0.0007) [2023-12-27 01:41:02,816][105620] Updated weights for policy 1, policy_version 1415260 (0.0008) [2023-12-27 01:41:02,876][105620] Updated weights for policy 1, policy_version 1415270 (0.0006) [2023-12-27 01:41:03,431][105692] Updated weights for policy 0, policy_version 1413138 (0.0007) [2023-12-27 01:41:03,477][105620] Updated weights for policy 1, policy_version 1415280 (0.0010) [2023-12-27 01:41:03,483][105692] Updated weights for policy 0, policy_version 1413148 (0.0005) [2023-12-27 01:41:03,525][105620] Updated weights for policy 1, policy_version 1415290 (0.0010) [2023-12-27 01:41:03,532][105692] Updated weights for policy 0, policy_version 1413158 (0.0008) [2023-12-27 01:41:03,577][105620] Updated weights for policy 1, policy_version 1415300 (0.0010) [2023-12-27 01:41:03,583][105692] Updated weights for policy 0, policy_version 1413168 (0.0010) [2023-12-27 01:41:04,319][105692] Updated weights for policy 0, policy_version 1413178 (0.0009) [2023-12-27 01:41:04,346][105620] Updated weights for policy 1, policy_version 1415310 (0.0009) [2023-12-27 01:41:04,377][105692] Updated weights for policy 0, policy_version 1413188 (0.0007) [2023-12-27 01:41:04,413][105620] Updated weights for policy 1, policy_version 1415320 (0.0011) [2023-12-27 01:41:04,439][105692] Updated weights for policy 0, policy_version 1413198 (0.0006) [2023-12-27 01:41:04,465][105620] Updated weights for policy 1, policy_version 1415330 (0.0008) [2023-12-27 01:41:05,151][105692] Updated weights for policy 0, policy_version 1413208 (0.0008) [2023-12-27 01:41:05,158][105620] Updated weights for policy 1, policy_version 1415340 (0.0008) [2023-12-27 01:41:05,201][105692] Updated weights for policy 0, policy_version 1413218 (0.0006) [2023-12-27 01:41:05,217][105620] Updated weights for policy 1, policy_version 1415350 (0.0010) [2023-12-27 01:41:05,252][105692] Updated weights for policy 0, policy_version 1413228 (0.0005) [2023-12-27 01:41:05,274][105620] Updated weights for policy 1, policy_version 1415360 (0.0010) [2023-12-27 01:41:05,774][105692] Updated weights for policy 0, policy_version 1413238 (0.0005) [2023-12-27 01:41:05,831][105692] Updated weights for policy 0, policy_version 1413248 (0.0005) [2023-12-27 01:41:05,879][105692] Updated weights for policy 0, policy_version 1413258 (0.0005) [2023-12-27 01:41:05,999][105620] Updated weights for policy 1, policy_version 1415370 (0.0009) [2023-12-27 01:41:06,060][105620] Updated weights for policy 1, policy_version 1415380 (0.0008) [2023-12-27 01:41:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19522.0). Total num frames: 724230144. Throughput: 0: 10072.3, 1: 9770.4. Samples: 724218236. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:41:06,062][104569] Avg episode reward: [(0, '7980.651'), (1, '8808.656')] [2023-12-27 01:41:06,126][105620] Updated weights for policy 1, policy_version 1415390 (0.0010) [2023-12-27 01:41:06,178][105620] Updated weights for policy 1, policy_version 1415400 (0.0010) [2023-12-27 01:41:06,545][105692] Updated weights for policy 0, policy_version 1413268 (0.0010) [2023-12-27 01:41:06,599][105692] Updated weights for policy 0, policy_version 1413278 (0.0006) [2023-12-27 01:41:06,666][105692] Updated weights for policy 0, policy_version 1413288 (0.0006) [2023-12-27 01:41:06,932][105620] Updated weights for policy 1, policy_version 1415410 (0.0011) [2023-12-27 01:41:06,988][105620] Updated weights for policy 1, policy_version 1415420 (0.0010) [2023-12-27 01:41:07,041][105620] Updated weights for policy 1, policy_version 1415430 (0.0010) [2023-12-27 01:41:07,348][105692] Updated weights for policy 0, policy_version 1413298 (0.0007) [2023-12-27 01:41:07,397][105692] Updated weights for policy 0, policy_version 1413308 (0.0011) [2023-12-27 01:41:07,456][105692] Updated weights for policy 0, policy_version 1413318 (0.0011) [2023-12-27 01:41:07,514][105692] Updated weights for policy 0, policy_version 1413328 (0.0010) [2023-12-27 01:41:07,828][105620] Updated weights for policy 1, policy_version 1415440 (0.0010) [2023-12-27 01:41:07,884][105620] Updated weights for policy 1, policy_version 1415451 (0.0010) [2023-12-27 01:41:07,937][105620] Updated weights for policy 1, policy_version 1415462 (0.0010) [2023-12-27 01:41:08,218][105692] Updated weights for policy 0, policy_version 1413338 (0.0011) [2023-12-27 01:41:08,271][105692] Updated weights for policy 0, policy_version 1413348 (0.0009) [2023-12-27 01:41:08,324][105692] Updated weights for policy 0, policy_version 1413358 (0.0006) [2023-12-27 01:41:08,609][105620] Updated weights for policy 1, policy_version 1415472 (0.0006) [2023-12-27 01:41:08,670][105620] Updated weights for policy 1, policy_version 1415482 (0.0005) [2023-12-27 01:41:08,735][105620] Updated weights for policy 1, policy_version 1415492 (0.0006) [2023-12-27 01:41:08,950][105692] Updated weights for policy 0, policy_version 1413368 (0.0009) [2023-12-27 01:41:09,005][105692] Updated weights for policy 0, policy_version 1413378 (0.0009) [2023-12-27 01:41:09,056][105692] Updated weights for policy 0, policy_version 1413388 (0.0008) [2023-12-27 01:41:09,347][105620] Updated weights for policy 1, policy_version 1415502 (0.0007) [2023-12-27 01:41:09,426][105620] Updated weights for policy 1, policy_version 1415512 (0.0009) [2023-12-27 01:41:09,475][105620] Updated weights for policy 1, policy_version 1415522 (0.0008) [2023-12-27 01:41:09,867][105692] Updated weights for policy 0, policy_version 1413398 (0.0009) [2023-12-27 01:41:09,931][105692] Updated weights for policy 0, policy_version 1413408 (0.0009) [2023-12-27 01:41:09,986][105692] Updated weights for policy 0, policy_version 1413418 (0.0010) [2023-12-27 01:41:10,188][105620] Updated weights for policy 1, policy_version 1415532 (0.0009) [2023-12-27 01:41:10,247][105620] Updated weights for policy 1, policy_version 1415542 (0.0009) [2023-12-27 01:41:10,294][105620] Updated weights for policy 1, policy_version 1415552 (0.0009) [2023-12-27 01:41:10,802][105692] Updated weights for policy 0, policy_version 1413428 (0.0009) [2023-12-27 01:41:10,867][105692] Updated weights for policy 0, policy_version 1413438 (0.0009) [2023-12-27 01:41:10,914][105692] Updated weights for policy 0, policy_version 1413448 (0.0008) [2023-12-27 01:41:10,969][105620] Updated weights for policy 1, policy_version 1415562 (0.0007) [2023-12-27 01:41:11,024][105620] Updated weights for policy 1, policy_version 1415572 (0.0008) [2023-12-27 01:41:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19521.9). Total num frames: 724328448. Throughput: 0: 10182.9, 1: 9766.4. Samples: 724338480. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:41:11,063][104569] Avg episode reward: [(0, '8343.707'), (1, '8896.650')] [2023-12-27 01:41:11,091][105620] Updated weights for policy 1, policy_version 1415582 (0.0009) [2023-12-27 01:41:11,152][105620] Updated weights for policy 1, policy_version 1415592 (0.0009) [2023-12-27 01:41:11,728][105692] Updated weights for policy 0, policy_version 1413458 (0.0009) [2023-12-27 01:41:11,796][105692] Updated weights for policy 0, policy_version 1413468 (0.0007) [2023-12-27 01:41:11,861][105692] Updated weights for policy 0, policy_version 1413478 (0.0006) [2023-12-27 01:41:11,917][105692] Updated weights for policy 0, policy_version 1413488 (0.0008) [2023-12-27 01:41:11,987][105620] Updated weights for policy 1, policy_version 1415602 (0.0009) [2023-12-27 01:41:12,050][105620] Updated weights for policy 1, policy_version 1415612 (0.0009) [2023-12-27 01:41:12,114][105620] Updated weights for policy 1, policy_version 1415622 (0.0007) [2023-12-27 01:41:12,620][105692] Updated weights for policy 0, policy_version 1413498 (0.0006) [2023-12-27 01:41:12,679][105692] Updated weights for policy 0, policy_version 1413508 (0.0008) [2023-12-27 01:41:12,741][105692] Updated weights for policy 0, policy_version 1413518 (0.0010) [2023-12-27 01:41:12,868][105620] Updated weights for policy 1, policy_version 1415632 (0.0006) [2023-12-27 01:41:12,923][105620] Updated weights for policy 1, policy_version 1415642 (0.0008) [2023-12-27 01:41:12,984][105620] Updated weights for policy 1, policy_version 1415652 (0.0005) [2023-12-27 01:41:13,506][105620] Updated weights for policy 1, policy_version 1415662 (0.0008) [2023-12-27 01:41:13,562][105620] Updated weights for policy 1, policy_version 1415672 (0.0007) [2023-12-27 01:41:13,621][105620] Updated weights for policy 1, policy_version 1415682 (0.0007) [2023-12-27 01:41:13,639][105692] Updated weights for policy 0, policy_version 1413528 (0.0007) [2023-12-27 01:41:13,705][105692] Updated weights for policy 0, policy_version 1413538 (0.0008) [2023-12-27 01:41:13,776][105692] Updated weights for policy 0, policy_version 1413548 (0.0009) [2023-12-27 01:41:14,346][105620] Updated weights for policy 1, policy_version 1415692 (0.0009) [2023-12-27 01:41:14,403][105620] Updated weights for policy 1, policy_version 1415702 (0.0007) [2023-12-27 01:41:14,462][105620] Updated weights for policy 1, policy_version 1415712 (0.0005) [2023-12-27 01:41:14,547][105692] Updated weights for policy 0, policy_version 1413558 (0.0009) [2023-12-27 01:41:14,608][105692] Updated weights for policy 0, policy_version 1413569 (0.0010) [2023-12-27 01:41:14,669][105692] Updated weights for policy 0, policy_version 1413579 (0.0008) [2023-12-27 01:41:15,165][105620] Updated weights for policy 1, policy_version 1415722 (0.0008) [2023-12-27 01:41:15,219][105620] Updated weights for policy 1, policy_version 1415732 (0.0011) [2023-12-27 01:41:15,264][105620] Updated weights for policy 1, policy_version 1415742 (0.0010) [2023-12-27 01:41:15,313][105620] Updated weights for policy 1, policy_version 1415752 (0.0011) [2023-12-27 01:41:15,426][105692] Updated weights for policy 0, policy_version 1413589 (0.0008) [2023-12-27 01:41:15,485][105692] Updated weights for policy 0, policy_version 1413599 (0.0008) [2023-12-27 01:41:15,535][105692] Updated weights for policy 0, policy_version 1413609 (0.0008) [2023-12-27 01:41:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 724418560. Throughput: 0: 10009.9, 1: 9776.1. Samples: 724394684. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:41:16,063][104569] Avg episode reward: [(0, '8708.806'), (1, '8988.435')] [2023-12-27 01:41:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001413616_361938944.pth... [2023-12-27 01:41:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001412464_361644032.pth [2023-12-27 01:41:16,101][105620] Updated weights for policy 1, policy_version 1415762 (0.0010) [2023-12-27 01:41:16,159][105620] Updated weights for policy 1, policy_version 1415772 (0.0010) [2023-12-27 01:41:16,217][105620] Updated weights for policy 1, policy_version 1415782 (0.0010) [2023-12-27 01:41:16,227][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001415784_362487808.pth... [2023-12-27 01:41:16,231][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001414632_362192896.pth [2023-12-27 01:41:16,257][105692] Updated weights for policy 0, policy_version 1413619 (0.0009) [2023-12-27 01:41:16,312][105692] Updated weights for policy 0, policy_version 1413631 (0.0011) [2023-12-27 01:41:16,369][105692] Updated weights for policy 0, policy_version 1413642 (0.0009) [2023-12-27 01:41:16,769][105620] Updated weights for policy 1, policy_version 1415792 (0.0006) [2023-12-27 01:41:16,820][105620] Updated weights for policy 1, policy_version 1415802 (0.0005) [2023-12-27 01:41:16,872][105620] Updated weights for policy 1, policy_version 1415812 (0.0009) [2023-12-27 01:41:17,246][105692] Updated weights for policy 0, policy_version 1413653 (0.0008) [2023-12-27 01:41:17,310][105692] Updated weights for policy 0, policy_version 1413663 (0.0005) [2023-12-27 01:41:17,365][105692] Updated weights for policy 0, policy_version 1413673 (0.0006) [2023-12-27 01:41:17,477][105620] Updated weights for policy 1, policy_version 1415822 (0.0007) [2023-12-27 01:41:17,536][105620] Updated weights for policy 1, policy_version 1415832 (0.0005) [2023-12-27 01:41:17,582][105620] Updated weights for policy 1, policy_version 1415842 (0.0005) [2023-12-27 01:41:18,120][105692] Updated weights for policy 0, policy_version 1413684 (0.0008) [2023-12-27 01:41:18,145][105620] Updated weights for policy 1, policy_version 1415852 (0.0007) [2023-12-27 01:41:18,174][105692] Updated weights for policy 0, policy_version 1413694 (0.0006) [2023-12-27 01:41:18,209][105620] Updated weights for policy 1, policy_version 1415862 (0.0007) [2023-12-27 01:41:18,233][105692] Updated weights for policy 0, policy_version 1413704 (0.0010) [2023-12-27 01:41:18,275][105620] Updated weights for policy 1, policy_version 1415872 (0.0006) [2023-12-27 01:41:18,829][105692] Updated weights for policy 0, policy_version 1413714 (0.0010) [2023-12-27 01:41:18,888][105692] Updated weights for policy 0, policy_version 1413724 (0.0010) [2023-12-27 01:41:18,951][105692] Updated weights for policy 0, policy_version 1413734 (0.0010) [2023-12-27 01:41:18,965][105620] Updated weights for policy 1, policy_version 1415882 (0.0007) [2023-12-27 01:41:19,010][105692] Updated weights for policy 0, policy_version 1413744 (0.0011) [2023-12-27 01:41:19,034][105620] Updated weights for policy 1, policy_version 1415892 (0.0005) [2023-12-27 01:41:19,095][105620] Updated weights for policy 1, policy_version 1415902 (0.0005) [2023-12-27 01:41:19,158][105620] Updated weights for policy 1, policy_version 1415912 (0.0005) [2023-12-27 01:41:19,733][105692] Updated weights for policy 0, policy_version 1413754 (0.0011) [2023-12-27 01:41:19,771][105620] Updated weights for policy 1, policy_version 1415922 (0.0006) [2023-12-27 01:41:19,799][105692] Updated weights for policy 0, policy_version 1413764 (0.0007) [2023-12-27 01:41:19,830][105620] Updated weights for policy 1, policy_version 1415932 (0.0007) [2023-12-27 01:41:19,867][105692] Updated weights for policy 0, policy_version 1413774 (0.0009) [2023-12-27 01:41:19,885][105620] Updated weights for policy 1, policy_version 1415942 (0.0006) [2023-12-27 01:41:20,596][105620] Updated weights for policy 1, policy_version 1415952 (0.0008) [2023-12-27 01:41:20,651][105692] Updated weights for policy 0, policy_version 1413784 (0.0011) [2023-12-27 01:41:20,657][105620] Updated weights for policy 1, policy_version 1415962 (0.0006) [2023-12-27 01:41:20,712][105692] Updated weights for policy 0, policy_version 1413794 (0.0011) [2023-12-27 01:41:20,719][105620] Updated weights for policy 1, policy_version 1415972 (0.0006) [2023-12-27 01:41:20,773][105692] Updated weights for policy 0, policy_version 1413804 (0.0011) [2023-12-27 01:41:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 724525056. Throughput: 0: 9852.8, 1: 9829.6. Samples: 724514040. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:41:21,063][104569] Avg episode reward: [(0, '7697.423'), (1, '9080.773')] [2023-12-27 01:41:21,471][105620] Updated weights for policy 1, policy_version 1415982 (0.0007) [2023-12-27 01:41:21,532][105620] Updated weights for policy 1, policy_version 1415992 (0.0007) [2023-12-27 01:41:21,545][105692] Updated weights for policy 0, policy_version 1413814 (0.0009) [2023-12-27 01:41:21,602][105620] Updated weights for policy 1, policy_version 1416002 (0.0006) [2023-12-27 01:41:21,607][105692] Updated weights for policy 0, policy_version 1413824 (0.0011) [2023-12-27 01:41:21,671][105692] Updated weights for policy 0, policy_version 1413834 (0.0008) [2023-12-27 01:41:22,425][105620] Updated weights for policy 1, policy_version 1416012 (0.0008) [2023-12-27 01:41:22,447][105692] Updated weights for policy 0, policy_version 1413844 (0.0010) [2023-12-27 01:41:22,487][105620] Updated weights for policy 1, policy_version 1416022 (0.0008) [2023-12-27 01:41:22,510][105692] Updated weights for policy 0, policy_version 1413854 (0.0007) [2023-12-27 01:41:22,545][105620] Updated weights for policy 1, policy_version 1416032 (0.0008) [2023-12-27 01:41:22,570][105692] Updated weights for policy 0, policy_version 1413864 (0.0007) [2023-12-27 01:41:23,221][105620] Updated weights for policy 1, policy_version 1416042 (0.0007) [2023-12-27 01:41:23,284][105620] Updated weights for policy 1, policy_version 1416052 (0.0008) [2023-12-27 01:41:23,344][105620] Updated weights for policy 1, policy_version 1416062 (0.0008) [2023-12-27 01:41:23,370][105692] Updated weights for policy 0, policy_version 1413874 (0.0008) [2023-12-27 01:41:23,408][105620] Updated weights for policy 1, policy_version 1416072 (0.0006) [2023-12-27 01:41:23,422][105692] Updated weights for policy 0, policy_version 1413884 (0.0006) [2023-12-27 01:41:23,475][105692] Updated weights for policy 0, policy_version 1413894 (0.0005) [2023-12-27 01:41:23,528][105692] Updated weights for policy 0, policy_version 1413904 (0.0005) [2023-12-27 01:41:24,140][105692] Updated weights for policy 0, policy_version 1413914 (0.0005) [2023-12-27 01:41:24,190][105692] Updated weights for policy 0, policy_version 1413924 (0.0005) [2023-12-27 01:41:24,224][105620] Updated weights for policy 1, policy_version 1416082 (0.0009) [2023-12-27 01:41:24,247][105692] Updated weights for policy 0, policy_version 1413934 (0.0006) [2023-12-27 01:41:24,283][105620] Updated weights for policy 1, policy_version 1416092 (0.0010) [2023-12-27 01:41:24,331][105620] Updated weights for policy 1, policy_version 1416102 (0.0009) [2023-12-27 01:41:24,920][105692] Updated weights for policy 0, policy_version 1413944 (0.0005) [2023-12-27 01:41:24,969][105692] Updated weights for policy 0, policy_version 1413954 (0.0005) [2023-12-27 01:41:25,017][105692] Updated weights for policy 0, policy_version 1413964 (0.0008) [2023-12-27 01:41:25,097][105620] Updated weights for policy 1, policy_version 1416112 (0.0006) [2023-12-27 01:41:25,163][105620] Updated weights for policy 1, policy_version 1416122 (0.0010) [2023-12-27 01:41:25,229][105620] Updated weights for policy 1, policy_version 1416132 (0.0007) [2023-12-27 01:41:25,604][105692] Updated weights for policy 0, policy_version 1413974 (0.0008) [2023-12-27 01:41:25,663][105692] Updated weights for policy 0, policy_version 1413984 (0.0010) [2023-12-27 01:41:25,723][105692] Updated weights for policy 0, policy_version 1413994 (0.0006) [2023-12-27 01:41:25,908][105620] Updated weights for policy 1, policy_version 1416142 (0.0008) [2023-12-27 01:41:25,972][105620] Updated weights for policy 1, policy_version 1416152 (0.0006) [2023-12-27 01:41:26,024][105620] Updated weights for policy 1, policy_version 1416162 (0.0005) [2023-12-27 01:41:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.9, 300 sec: 19522.0). Total num frames: 724615168. Throughput: 0: 9856.6, 1: 9774.6. Samples: 724628072. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:41:26,062][104569] Avg episode reward: [(0, '7426.199'), (1, '9172.631')] [2023-12-27 01:41:26,297][105692] Updated weights for policy 0, policy_version 1414004 (0.0007) [2023-12-27 01:41:26,360][105692] Updated weights for policy 0, policy_version 1414014 (0.0009) [2023-12-27 01:41:26,424][105692] Updated weights for policy 0, policy_version 1414024 (0.0008) [2023-12-27 01:41:26,649][105620] Updated weights for policy 1, policy_version 1416172 (0.0008) [2023-12-27 01:41:26,707][105620] Updated weights for policy 1, policy_version 1416182 (0.0010) [2023-12-27 01:41:26,760][105620] Updated weights for policy 1, policy_version 1416193 (0.0010) [2023-12-27 01:41:26,952][105692] Updated weights for policy 0, policy_version 1414034 (0.0005) [2023-12-27 01:41:27,023][105692] Updated weights for policy 0, policy_version 1414044 (0.0005) [2023-12-27 01:41:27,089][105692] Updated weights for policy 0, policy_version 1414054 (0.0005) [2023-12-27 01:41:27,149][105692] Updated weights for policy 0, policy_version 1414064 (0.0005) [2023-12-27 01:41:27,652][105620] Updated weights for policy 1, policy_version 1416203 (0.0009) [2023-12-27 01:41:27,680][105585] KL-divergence is very high: 355.0959 [2023-12-27 01:41:27,691][105692] Updated weights for policy 0, policy_version 1414074 (0.0005) [2023-12-27 01:41:27,702][105585] KL-divergence is very high: 209.9649 [2023-12-27 01:41:27,703][105620] Updated weights for policy 1, policy_version 1416213 (0.0008) [2023-12-27 01:41:27,725][105585] KL-divergence is very high: 614.7462 [2023-12-27 01:41:27,747][105692] Updated weights for policy 0, policy_version 1414084 (0.0005) [2023-12-27 01:41:27,749][105585] KL-divergence is very high: 252.5834 [2023-12-27 01:41:27,756][105620] Updated weights for policy 1, policy_version 1416224 (0.0010) [2023-12-27 01:41:27,771][105585] KL-divergence is very high: 685.2326 [2023-12-27 01:41:27,794][105585] KL-divergence is very high: 238.4428 [2023-12-27 01:41:27,806][105692] Updated weights for policy 0, policy_version 1414094 (0.0007) [2023-12-27 01:41:28,384][105692] Updated weights for policy 0, policy_version 1414104 (0.0008) [2023-12-27 01:41:28,446][105692] Updated weights for policy 0, policy_version 1414114 (0.0009) [2023-12-27 01:41:28,501][105692] Updated weights for policy 0, policy_version 1414124 (0.0010) [2023-12-27 01:41:28,631][105620] Updated weights for policy 1, policy_version 1416234 (0.0008) [2023-12-27 01:41:28,694][105620] Updated weights for policy 1, policy_version 1416244 (0.0008) [2023-12-27 01:41:28,746][105620] Updated weights for policy 1, policy_version 1416254 (0.0007) [2023-12-27 01:41:28,793][105620] Updated weights for policy 1, policy_version 1416264 (0.0008) [2023-12-27 01:41:29,188][105692] Updated weights for policy 0, policy_version 1414134 (0.0008) [2023-12-27 01:41:29,256][105692] Updated weights for policy 0, policy_version 1414144 (0.0008) [2023-12-27 01:41:29,305][105692] Updated weights for policy 0, policy_version 1414154 (0.0008) [2023-12-27 01:41:29,600][105620] Updated weights for policy 1, policy_version 1416274 (0.0011) [2023-12-27 01:41:29,655][105620] Updated weights for policy 1, policy_version 1416284 (0.0007) [2023-12-27 01:41:29,704][105620] Updated weights for policy 1, policy_version 1416294 (0.0010) [2023-12-27 01:41:30,085][105692] Updated weights for policy 0, policy_version 1414164 (0.0008) [2023-12-27 01:41:30,147][105692] Updated weights for policy 0, policy_version 1414174 (0.0008) [2023-12-27 01:41:30,202][105692] Updated weights for policy 0, policy_version 1414184 (0.0008) [2023-12-27 01:41:30,402][105620] Updated weights for policy 1, policy_version 1416304 (0.0008) [2023-12-27 01:41:30,450][105620] Updated weights for policy 1, policy_version 1416314 (0.0010) [2023-12-27 01:41:30,498][105620] Updated weights for policy 1, policy_version 1416324 (0.0010) [2023-12-27 01:41:30,937][105692] Updated weights for policy 0, policy_version 1414194 (0.0008) [2023-12-27 01:41:30,994][105692] Updated weights for policy 0, policy_version 1414204 (0.0010) [2023-12-27 01:41:31,052][105692] Updated weights for policy 0, policy_version 1414214 (0.0008) [2023-12-27 01:41:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 724713472. Throughput: 0: 9974.5, 1: 9772.1. Samples: 724689852. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:41:31,063][104569] Avg episode reward: [(0, '7975.774'), (1, '9172.641')] [2023-12-27 01:41:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001416328_362627072.pth... [2023-12-27 01:41:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001415208_362340352.pth [2023-12-27 01:41:31,106][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001414224_362094592.pth... [2023-12-27 01:41:31,107][105692] Updated weights for policy 0, policy_version 1414224 (0.0008) [2023-12-27 01:41:31,110][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001413072_361799680.pth [2023-12-27 01:41:31,166][105620] Updated weights for policy 1, policy_version 1416334 (0.0009) [2023-12-27 01:41:31,216][105620] Updated weights for policy 1, policy_version 1416344 (0.0008) [2023-12-27 01:41:31,268][105620] Updated weights for policy 1, policy_version 1416354 (0.0008) [2023-12-27 01:41:31,942][105692] Updated weights for policy 0, policy_version 1414234 (0.0009) [2023-12-27 01:41:32,002][105692] Updated weights for policy 0, policy_version 1414244 (0.0009) [2023-12-27 01:41:32,020][105620] Updated weights for policy 1, policy_version 1416364 (0.0007) [2023-12-27 01:41:32,060][105692] Updated weights for policy 0, policy_version 1414254 (0.0008) [2023-12-27 01:41:32,082][105620] Updated weights for policy 1, policy_version 1416374 (0.0007) [2023-12-27 01:41:32,132][105620] Updated weights for policy 1, policy_version 1416384 (0.0008) [2023-12-27 01:41:32,826][105692] Updated weights for policy 0, policy_version 1414264 (0.0005) [2023-12-27 01:41:32,884][105692] Updated weights for policy 0, policy_version 1414274 (0.0006) [2023-12-27 01:41:32,896][105620] Updated weights for policy 1, policy_version 1416394 (0.0010) [2023-12-27 01:41:32,937][105692] Updated weights for policy 0, policy_version 1414284 (0.0007) [2023-12-27 01:41:32,958][105620] Updated weights for policy 1, policy_version 1416404 (0.0011) [2023-12-27 01:41:33,016][105620] Updated weights for policy 1, policy_version 1416414 (0.0011) [2023-12-27 01:41:33,085][105620] Updated weights for policy 1, policy_version 1416424 (0.0010) [2023-12-27 01:41:33,579][105692] Updated weights for policy 0, policy_version 1414294 (0.0008) [2023-12-27 01:41:33,635][105692] Updated weights for policy 0, policy_version 1414304 (0.0006) [2023-12-27 01:41:33,693][105692] Updated weights for policy 0, policy_version 1414314 (0.0007) [2023-12-27 01:41:33,812][105620] Updated weights for policy 1, policy_version 1416434 (0.0010) [2023-12-27 01:41:33,876][105620] Updated weights for policy 1, policy_version 1416444 (0.0010) [2023-12-27 01:41:33,923][105620] Updated weights for policy 1, policy_version 1416454 (0.0010) [2023-12-27 01:41:34,297][105692] Updated weights for policy 0, policy_version 1414324 (0.0007) [2023-12-27 01:41:34,360][105692] Updated weights for policy 0, policy_version 1414334 (0.0008) [2023-12-27 01:41:34,426][105692] Updated weights for policy 0, policy_version 1414344 (0.0008) [2023-12-27 01:41:34,671][105620] Updated weights for policy 1, policy_version 1416464 (0.0011) [2023-12-27 01:41:34,730][105620] Updated weights for policy 1, policy_version 1416474 (0.0011) [2023-12-27 01:41:34,792][105620] Updated weights for policy 1, policy_version 1416484 (0.0011) [2023-12-27 01:41:35,169][105692] Updated weights for policy 0, policy_version 1414354 (0.0009) [2023-12-27 01:41:35,224][105692] Updated weights for policy 0, policy_version 1414364 (0.0010) [2023-12-27 01:41:35,278][105692] Updated weights for policy 0, policy_version 1414374 (0.0010) [2023-12-27 01:41:35,332][105692] Updated weights for policy 0, policy_version 1414384 (0.0010) [2023-12-27 01:41:35,468][105620] Updated weights for policy 1, policy_version 1416494 (0.0007) [2023-12-27 01:41:35,521][105620] Updated weights for policy 1, policy_version 1416504 (0.0005) [2023-12-27 01:41:35,572][105620] Updated weights for policy 1, policy_version 1416514 (0.0006) [2023-12-27 01:41:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 724811776. Throughput: 0: 9877.3, 1: 9679.5. Samples: 724804872. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:41:36,062][104569] Avg episode reward: [(0, '7973.998'), (1, '8898.196')] [2023-12-27 01:41:36,091][105692] Updated weights for policy 0, policy_version 1414394 (0.0010) [2023-12-27 01:41:36,155][105692] Updated weights for policy 0, policy_version 1414404 (0.0008) [2023-12-27 01:41:36,217][105692] Updated weights for policy 0, policy_version 1414414 (0.0007) [2023-12-27 01:41:36,242][105620] Updated weights for policy 1, policy_version 1416524 (0.0007) [2023-12-27 01:41:36,310][105620] Updated weights for policy 1, policy_version 1416534 (0.0006) [2023-12-27 01:41:36,372][105620] Updated weights for policy 1, policy_version 1416544 (0.0009) [2023-12-27 01:41:36,896][105692] Updated weights for policy 0, policy_version 1414424 (0.0010) [2023-12-27 01:41:36,952][105692] Updated weights for policy 0, policy_version 1414434 (0.0010) [2023-12-27 01:41:37,011][105692] Updated weights for policy 0, policy_version 1414444 (0.0006) [2023-12-27 01:41:37,051][105620] Updated weights for policy 1, policy_version 1416554 (0.0010) [2023-12-27 01:41:37,106][105620] Updated weights for policy 1, policy_version 1416564 (0.0010) [2023-12-27 01:41:37,168][105620] Updated weights for policy 1, policy_version 1416574 (0.0010) [2023-12-27 01:41:37,230][105620] Updated weights for policy 1, policy_version 1416584 (0.0010) [2023-12-27 01:41:37,706][105692] Updated weights for policy 0, policy_version 1414454 (0.0009) [2023-12-27 01:41:37,765][105692] Updated weights for policy 0, policy_version 1414464 (0.0011) [2023-12-27 01:41:37,827][105692] Updated weights for policy 0, policy_version 1414474 (0.0010) [2023-12-27 01:41:37,965][105620] Updated weights for policy 1, policy_version 1416594 (0.0011) [2023-12-27 01:41:38,017][105620] Updated weights for policy 1, policy_version 1416604 (0.0008) [2023-12-27 01:41:38,076][105620] Updated weights for policy 1, policy_version 1416614 (0.0007) [2023-12-27 01:41:38,579][105692] Updated weights for policy 0, policy_version 1414484 (0.0010) [2023-12-27 01:41:38,638][105692] Updated weights for policy 0, policy_version 1414494 (0.0011) [2023-12-27 01:41:38,696][105692] Updated weights for policy 0, policy_version 1414504 (0.0009) [2023-12-27 01:41:38,862][105620] Updated weights for policy 1, policy_version 1416624 (0.0008) [2023-12-27 01:41:38,923][105620] Updated weights for policy 1, policy_version 1416634 (0.0009) [2023-12-27 01:41:38,976][105620] Updated weights for policy 1, policy_version 1416644 (0.0010) [2023-12-27 01:41:39,442][105692] Updated weights for policy 0, policy_version 1414514 (0.0010) [2023-12-27 01:41:39,494][105692] Updated weights for policy 0, policy_version 1414524 (0.0008) [2023-12-27 01:41:39,558][105692] Updated weights for policy 0, policy_version 1414534 (0.0005) [2023-12-27 01:41:39,615][105692] Updated weights for policy 0, policy_version 1414544 (0.0005) [2023-12-27 01:41:39,740][105620] Updated weights for policy 1, policy_version 1416654 (0.0009) [2023-12-27 01:41:39,803][105620] Updated weights for policy 1, policy_version 1416664 (0.0008) [2023-12-27 01:41:39,868][105620] Updated weights for policy 1, policy_version 1416674 (0.0009) [2023-12-27 01:41:40,309][105692] Updated weights for policy 0, policy_version 1414554 (0.0008) [2023-12-27 01:41:40,376][105692] Updated weights for policy 0, policy_version 1414564 (0.0009) [2023-12-27 01:41:40,440][105692] Updated weights for policy 0, policy_version 1414574 (0.0008) [2023-12-27 01:41:40,630][105620] Updated weights for policy 1, policy_version 1416684 (0.0011) [2023-12-27 01:41:40,685][105620] Updated weights for policy 1, policy_version 1416694 (0.0010) [2023-12-27 01:41:40,743][105620] Updated weights for policy 1, policy_version 1416704 (0.0010) [2023-12-27 01:41:41,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 724910080. Throughput: 0: 9796.3, 1: 9657.7. Samples: 724920128. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:41:41,063][104569] Avg episode reward: [(0, '7514.065'), (1, '8898.135')] [2023-12-27 01:41:41,143][105692] Updated weights for policy 0, policy_version 1414584 (0.0008) [2023-12-27 01:41:41,203][105692] Updated weights for policy 0, policy_version 1414594 (0.0007) [2023-12-27 01:41:41,254][105692] Updated weights for policy 0, policy_version 1414604 (0.0009) [2023-12-27 01:41:41,438][105620] Updated weights for policy 1, policy_version 1416714 (0.0010) [2023-12-27 01:41:41,498][105620] Updated weights for policy 1, policy_version 1416724 (0.0011) [2023-12-27 01:41:41,558][105620] Updated weights for policy 1, policy_version 1416734 (0.0011) [2023-12-27 01:41:41,614][105620] Updated weights for policy 1, policy_version 1416744 (0.0010) [2023-12-27 01:41:42,030][105692] Updated weights for policy 0, policy_version 1414614 (0.0009) [2023-12-27 01:41:42,088][105692] Updated weights for policy 0, policy_version 1414624 (0.0011) [2023-12-27 01:41:42,148][105692] Updated weights for policy 0, policy_version 1414634 (0.0011) [2023-12-27 01:41:42,392][105620] Updated weights for policy 1, policy_version 1416754 (0.0010) [2023-12-27 01:41:42,456][105620] Updated weights for policy 1, policy_version 1416764 (0.0007) [2023-12-27 01:41:42,512][105620] Updated weights for policy 1, policy_version 1416774 (0.0006) [2023-12-27 01:41:42,912][105692] Updated weights for policy 0, policy_version 1414644 (0.0011) [2023-12-27 01:41:42,967][105692] Updated weights for policy 0, policy_version 1414654 (0.0010) [2023-12-27 01:41:43,025][105692] Updated weights for policy 0, policy_version 1414664 (0.0010) [2023-12-27 01:41:43,069][105620] Updated weights for policy 1, policy_version 1416784 (0.0008) [2023-12-27 01:41:43,121][105620] Updated weights for policy 1, policy_version 1416794 (0.0005) [2023-12-27 01:41:43,178][105620] Updated weights for policy 1, policy_version 1416804 (0.0005) [2023-12-27 01:41:43,709][105692] Updated weights for policy 0, policy_version 1414674 (0.0010) [2023-12-27 01:41:43,729][105620] Updated weights for policy 1, policy_version 1416814 (0.0009) [2023-12-27 01:41:43,773][105692] Updated weights for policy 0, policy_version 1414684 (0.0009) [2023-12-27 01:41:43,789][105620] Updated weights for policy 1, policy_version 1416824 (0.0010) [2023-12-27 01:41:43,833][105692] Updated weights for policy 0, policy_version 1414694 (0.0011) [2023-12-27 01:41:43,847][105620] Updated weights for policy 1, policy_version 1416834 (0.0009) [2023-12-27 01:41:43,886][105692] Updated weights for policy 0, policy_version 1414704 (0.0011) [2023-12-27 01:41:44,400][105620] Updated weights for policy 1, policy_version 1416844 (0.0006) [2023-12-27 01:41:44,459][105620] Updated weights for policy 1, policy_version 1416854 (0.0009) [2023-12-27 01:41:44,519][105620] Updated weights for policy 1, policy_version 1416864 (0.0010) [2023-12-27 01:41:44,616][105692] Updated weights for policy 0, policy_version 1414714 (0.0005) [2023-12-27 01:41:44,666][105692] Updated weights for policy 0, policy_version 1414724 (0.0009) [2023-12-27 01:41:44,719][105692] Updated weights for policy 0, policy_version 1414734 (0.0008) [2023-12-27 01:41:45,214][105620] Updated weights for policy 1, policy_version 1416874 (0.0010) [2023-12-27 01:41:45,283][105620] Updated weights for policy 1, policy_version 1416884 (0.0006) [2023-12-27 01:41:45,350][105620] Updated weights for policy 1, policy_version 1416894 (0.0006) [2023-12-27 01:41:45,369][105692] Updated weights for policy 0, policy_version 1414744 (0.0009) [2023-12-27 01:41:45,415][105620] Updated weights for policy 1, policy_version 1416904 (0.0006) [2023-12-27 01:41:45,425][105692] Updated weights for policy 0, policy_version 1414754 (0.0011) [2023-12-27 01:41:45,481][105692] Updated weights for policy 0, policy_version 1414764 (0.0011) [2023-12-27 01:41:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 725008384. Throughput: 0: 9753.8, 1: 9729.9. Samples: 724980312. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:41:46,063][104569] Avg episode reward: [(0, '7521.007'), (1, '9172.818')] [2023-12-27 01:41:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001414768_362233856.pth... [2023-12-27 01:41:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001413616_361938944.pth [2023-12-27 01:41:46,072][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_001414768_362233856.pth [2023-12-27 01:41:46,123][105620] Updated weights for policy 1, policy_version 1416914 (0.0005) [2023-12-27 01:41:46,168][105620] Updated weights for policy 1, policy_version 1416924 (0.0005) [2023-12-27 01:41:46,227][105620] Updated weights for policy 1, policy_version 1416934 (0.0006) [2023-12-27 01:41:46,229][105692] Updated weights for policy 0, policy_version 1414774 (0.0010) [2023-12-27 01:41:46,236][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001416936_362782720.pth... [2023-12-27 01:41:46,240][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001415784_362487808.pth [2023-12-27 01:41:46,241][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_001416936_362782720.pth [2023-12-27 01:41:46,277][105692] Updated weights for policy 0, policy_version 1414784 (0.0010) [2023-12-27 01:41:46,337][105692] Updated weights for policy 0, policy_version 1414794 (0.0008) [2023-12-27 01:41:46,844][105620] Updated weights for policy 1, policy_version 1416944 (0.0006) [2023-12-27 01:41:46,901][105620] Updated weights for policy 1, policy_version 1416954 (0.0008) [2023-12-27 01:41:46,960][105620] Updated weights for policy 1, policy_version 1416964 (0.0006) [2023-12-27 01:41:46,992][105692] Updated weights for policy 0, policy_version 1414804 (0.0006) [2023-12-27 01:41:47,041][105692] Updated weights for policy 0, policy_version 1414814 (0.0005) [2023-12-27 01:41:47,085][105692] Updated weights for policy 0, policy_version 1414824 (0.0005) [2023-12-27 01:41:47,632][105692] Updated weights for policy 0, policy_version 1414834 (0.0006) [2023-12-27 01:41:47,643][105620] Updated weights for policy 1, policy_version 1416974 (0.0007) [2023-12-27 01:41:47,695][105692] Updated weights for policy 0, policy_version 1414844 (0.0008) [2023-12-27 01:41:47,702][105620] Updated weights for policy 1, policy_version 1416984 (0.0006) [2023-12-27 01:41:47,745][105692] Updated weights for policy 0, policy_version 1414854 (0.0007) [2023-12-27 01:41:47,762][105620] Updated weights for policy 1, policy_version 1416994 (0.0008) [2023-12-27 01:41:47,796][105692] Updated weights for policy 0, policy_version 1414864 (0.0007) [2023-12-27 01:41:48,335][105620] Updated weights for policy 1, policy_version 1417004 (0.0008) [2023-12-27 01:41:48,395][105620] Updated weights for policy 1, policy_version 1417014 (0.0008) [2023-12-27 01:41:48,454][105620] Updated weights for policy 1, policy_version 1417024 (0.0008) [2023-12-27 01:41:48,665][105692] Updated weights for policy 0, policy_version 1414874 (0.0010) [2023-12-27 01:41:48,730][105692] Updated weights for policy 0, policy_version 1414884 (0.0009) [2023-12-27 01:41:48,797][105692] Updated weights for policy 0, policy_version 1414894 (0.0009) [2023-12-27 01:41:49,236][105620] Updated weights for policy 1, policy_version 1417034 (0.0010) [2023-12-27 01:41:49,296][105620] Updated weights for policy 1, policy_version 1417044 (0.0010) [2023-12-27 01:41:49,357][105620] Updated weights for policy 1, policy_version 1417054 (0.0010) [2023-12-27 01:41:49,413][105620] Updated weights for policy 1, policy_version 1417064 (0.0008) [2023-12-27 01:41:49,539][105692] Updated weights for policy 0, policy_version 1414904 (0.0009) [2023-12-27 01:41:49,600][105692] Updated weights for policy 0, policy_version 1414914 (0.0009) [2023-12-27 01:41:49,662][105692] Updated weights for policy 0, policy_version 1414924 (0.0009) [2023-12-27 01:41:50,162][105620] Updated weights for policy 1, policy_version 1417074 (0.0009) [2023-12-27 01:41:50,219][105620] Updated weights for policy 1, policy_version 1417084 (0.0007) [2023-12-27 01:41:50,283][105620] Updated weights for policy 1, policy_version 1417094 (0.0009) [2023-12-27 01:41:50,381][105692] Updated weights for policy 0, policy_version 1414934 (0.0009) [2023-12-27 01:41:50,446][105692] Updated weights for policy 0, policy_version 1414944 (0.0009) [2023-12-27 01:41:50,514][105692] Updated weights for policy 0, policy_version 1414954 (0.0008) [2023-12-27 01:41:51,027][105620] Updated weights for policy 1, policy_version 1417104 (0.0007) [2023-12-27 01:41:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 725106688. Throughput: 0: 9790.3, 1: 9836.0. Samples: 725101420. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:41:51,062][104569] Avg episode reward: [(0, '7331.477'), (1, '8803.807')] [2023-12-27 01:41:51,089][105620] Updated weights for policy 1, policy_version 1417114 (0.0008) [2023-12-27 01:41:51,151][105620] Updated weights for policy 1, policy_version 1417124 (0.0008) [2023-12-27 01:41:51,296][105692] Updated weights for policy 0, policy_version 1414964 (0.0009) [2023-12-27 01:41:51,354][105692] Updated weights for policy 0, policy_version 1414974 (0.0010) [2023-12-27 01:41:51,419][105692] Updated weights for policy 0, policy_version 1414984 (0.0012) [2023-12-27 01:41:51,873][105620] Updated weights for policy 1, policy_version 1417134 (0.0009) [2023-12-27 01:41:51,930][105620] Updated weights for policy 1, policy_version 1417144 (0.0009) [2023-12-27 01:41:51,983][105620] Updated weights for policy 1, policy_version 1417154 (0.0006) [2023-12-27 01:41:52,095][105692] Updated weights for policy 0, policy_version 1414995 (0.0009) [2023-12-27 01:41:52,157][105692] Updated weights for policy 0, policy_version 1415005 (0.0006) [2023-12-27 01:41:52,214][105692] Updated weights for policy 0, policy_version 1415015 (0.0006) [2023-12-27 01:41:52,677][105620] Updated weights for policy 1, policy_version 1417164 (0.0007) [2023-12-27 01:41:52,742][105620] Updated weights for policy 1, policy_version 1417174 (0.0011) [2023-12-27 01:41:52,810][105620] Updated weights for policy 1, policy_version 1417184 (0.0010) [2023-12-27 01:41:52,814][105692] Updated weights for policy 0, policy_version 1415025 (0.0008) [2023-12-27 01:41:52,870][105692] Updated weights for policy 0, policy_version 1415035 (0.0011) [2023-12-27 01:41:52,928][105692] Updated weights for policy 0, policy_version 1415045 (0.0010) [2023-12-27 01:41:52,980][105692] Updated weights for policy 0, policy_version 1415055 (0.0010) [2023-12-27 01:41:53,354][105620] Updated weights for policy 1, policy_version 1417194 (0.0008) [2023-12-27 01:41:53,416][105620] Updated weights for policy 1, policy_version 1417204 (0.0006) [2023-12-27 01:41:53,475][105620] Updated weights for policy 1, policy_version 1417214 (0.0005) [2023-12-27 01:41:53,549][105620] Updated weights for policy 1, policy_version 1417224 (0.0006) [2023-12-27 01:41:53,663][105692] Updated weights for policy 0, policy_version 1415065 (0.0007) [2023-12-27 01:41:53,720][105692] Updated weights for policy 0, policy_version 1415075 (0.0006) [2023-12-27 01:41:53,770][105692] Updated weights for policy 0, policy_version 1415085 (0.0006) [2023-12-27 01:41:54,054][105620] Updated weights for policy 1, policy_version 1417234 (0.0005) [2023-12-27 01:41:54,105][105620] Updated weights for policy 1, policy_version 1417244 (0.0005) [2023-12-27 01:41:54,165][105620] Updated weights for policy 1, policy_version 1417254 (0.0008) [2023-12-27 01:41:54,370][105692] Updated weights for policy 0, policy_version 1415095 (0.0008) [2023-12-27 01:41:54,432][105692] Updated weights for policy 0, policy_version 1415105 (0.0011) [2023-12-27 01:41:54,484][105692] Updated weights for policy 0, policy_version 1415115 (0.0010) [2023-12-27 01:41:54,836][105620] Updated weights for policy 1, policy_version 1417264 (0.0008) [2023-12-27 01:41:54,886][105620] Updated weights for policy 1, policy_version 1417274 (0.0008) [2023-12-27 01:41:54,946][105620] Updated weights for policy 1, policy_version 1417284 (0.0008) [2023-12-27 01:41:55,235][105692] Updated weights for policy 0, policy_version 1415125 (0.0010) [2023-12-27 01:41:55,297][105692] Updated weights for policy 0, policy_version 1415135 (0.0011) [2023-12-27 01:41:55,347][105692] Updated weights for policy 0, policy_version 1415145 (0.0010) [2023-12-27 01:41:55,623][105620] Updated weights for policy 1, policy_version 1417294 (0.0010) [2023-12-27 01:41:55,677][105620] Updated weights for policy 1, policy_version 1417304 (0.0010) [2023-12-27 01:41:55,735][105620] Updated weights for policy 1, policy_version 1417314 (0.0010) [2023-12-27 01:41:56,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.7, 300 sec: 19605.3). Total num frames: 725213184. Throughput: 0: 9762.5, 1: 9903.2. Samples: 725223436. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:41:56,063][104569] Avg episode reward: [(0, '7698.778'), (1, '8896.437')] [2023-12-27 01:41:56,094][105692] Updated weights for policy 0, policy_version 1415155 (0.0010) [2023-12-27 01:41:56,149][105692] Updated weights for policy 0, policy_version 1415165 (0.0007) [2023-12-27 01:41:56,201][105692] Updated weights for policy 0, policy_version 1415175 (0.0008) [2023-12-27 01:41:56,476][105620] Updated weights for policy 1, policy_version 1417324 (0.0010) [2023-12-27 01:41:56,524][105620] Updated weights for policy 1, policy_version 1417334 (0.0010) [2023-12-27 01:41:56,578][105620] Updated weights for policy 1, policy_version 1417344 (0.0010) [2023-12-27 01:41:56,889][105692] Updated weights for policy 0, policy_version 1415185 (0.0008) [2023-12-27 01:41:56,954][105692] Updated weights for policy 0, policy_version 1415195 (0.0005) [2023-12-27 01:41:57,024][105692] Updated weights for policy 0, policy_version 1415205 (0.0005) [2023-12-27 01:41:57,095][105692] Updated weights for policy 0, policy_version 1415215 (0.0008) [2023-12-27 01:41:57,233][105620] Updated weights for policy 1, policy_version 1417354 (0.0010) [2023-12-27 01:41:57,290][105620] Updated weights for policy 1, policy_version 1417364 (0.0010) [2023-12-27 01:41:57,349][105620] Updated weights for policy 1, policy_version 1417374 (0.0010) [2023-12-27 01:41:57,407][105620] Updated weights for policy 1, policy_version 1417384 (0.0010) [2023-12-27 01:41:57,808][105692] Updated weights for policy 0, policy_version 1415225 (0.0008) [2023-12-27 01:41:57,867][105692] Updated weights for policy 0, policy_version 1415235 (0.0008) [2023-12-27 01:41:57,924][105692] Updated weights for policy 0, policy_version 1415245 (0.0008) [2023-12-27 01:41:58,138][105620] Updated weights for policy 1, policy_version 1417394 (0.0010) [2023-12-27 01:41:58,205][105620] Updated weights for policy 1, policy_version 1417404 (0.0010) [2023-12-27 01:41:58,271][105620] Updated weights for policy 1, policy_version 1417414 (0.0010) [2023-12-27 01:41:58,679][105692] Updated weights for policy 0, policy_version 1415255 (0.0008) [2023-12-27 01:41:58,744][105692] Updated weights for policy 0, policy_version 1415265 (0.0008) [2023-12-27 01:41:58,812][105692] Updated weights for policy 0, policy_version 1415275 (0.0008) [2023-12-27 01:41:59,007][105620] Updated weights for policy 1, policy_version 1417424 (0.0008) [2023-12-27 01:41:59,066][105620] Updated weights for policy 1, policy_version 1417434 (0.0011) [2023-12-27 01:41:59,115][105620] Updated weights for policy 1, policy_version 1417444 (0.0010) [2023-12-27 01:41:59,583][105692] Updated weights for policy 0, policy_version 1415285 (0.0008) [2023-12-27 01:41:59,642][105692] Updated weights for policy 0, policy_version 1415295 (0.0010) [2023-12-27 01:41:59,703][105692] Updated weights for policy 0, policy_version 1415305 (0.0010) [2023-12-27 01:41:59,799][105620] Updated weights for policy 1, policy_version 1417454 (0.0010) [2023-12-27 01:41:59,862][105620] Updated weights for policy 1, policy_version 1417464 (0.0011) [2023-12-27 01:41:59,919][105620] Updated weights for policy 1, policy_version 1417474 (0.0011) [2023-12-27 01:42:00,458][105692] Updated weights for policy 0, policy_version 1415315 (0.0009) [2023-12-27 01:42:00,512][105692] Updated weights for policy 0, policy_version 1415325 (0.0009) [2023-12-27 01:42:00,571][105692] Updated weights for policy 0, policy_version 1415335 (0.0009) [2023-12-27 01:42:00,616][105620] Updated weights for policy 1, policy_version 1417484 (0.0009) [2023-12-27 01:42:00,674][105620] Updated weights for policy 1, policy_version 1417494 (0.0009) [2023-12-27 01:42:00,741][105620] Updated weights for policy 1, policy_version 1417504 (0.0010) [2023-12-27 01:42:01,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 725311488. Throughput: 0: 9812.6, 1: 9896.6. Samples: 725281600. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:42:01,063][104569] Avg episode reward: [(0, '8070.279'), (1, '9173.348')] [2023-12-27 01:42:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001415344_362381312.pth... [2023-12-27 01:42:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001417512_362930176.pth... [2023-12-27 01:42:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001416328_362627072.pth [2023-12-27 01:42:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001414224_362094592.pth [2023-12-27 01:42:01,241][105692] Updated weights for policy 0, policy_version 1415346 (0.0008) [2023-12-27 01:42:01,300][105692] Updated weights for policy 0, policy_version 1415356 (0.0008) [2023-12-27 01:42:01,379][105692] Updated weights for policy 0, policy_version 1415366 (0.0009) [2023-12-27 01:42:01,435][105692] Updated weights for policy 0, policy_version 1415376 (0.0008) [2023-12-27 01:42:01,463][105620] Updated weights for policy 1, policy_version 1417514 (0.0010) [2023-12-27 01:42:01,525][105620] Updated weights for policy 1, policy_version 1417524 (0.0008) [2023-12-27 01:42:01,577][105620] Updated weights for policy 1, policy_version 1417534 (0.0009) [2023-12-27 01:42:01,637][105620] Updated weights for policy 1, policy_version 1417544 (0.0009) [2023-12-27 01:42:02,188][105692] Updated weights for policy 0, policy_version 1415386 (0.0006) [2023-12-27 01:42:02,244][105692] Updated weights for policy 0, policy_version 1415396 (0.0008) [2023-12-27 01:42:02,303][105692] Updated weights for policy 0, policy_version 1415406 (0.0009) [2023-12-27 01:42:02,422][105620] Updated weights for policy 1, policy_version 1417554 (0.0009) [2023-12-27 01:42:02,487][105620] Updated weights for policy 1, policy_version 1417564 (0.0009) [2023-12-27 01:42:02,545][105620] Updated weights for policy 1, policy_version 1417574 (0.0009) [2023-12-27 01:42:03,053][105692] Updated weights for policy 0, policy_version 1415416 (0.0006) [2023-12-27 01:42:03,104][105692] Updated weights for policy 0, policy_version 1415426 (0.0009) [2023-12-27 01:42:03,151][105692] Updated weights for policy 0, policy_version 1415436 (0.0009) [2023-12-27 01:42:03,207][105620] Updated weights for policy 1, policy_version 1417584 (0.0009) [2023-12-27 01:42:03,254][105620] Updated weights for policy 1, policy_version 1417594 (0.0009) [2023-12-27 01:42:03,303][105620] Updated weights for policy 1, policy_version 1417604 (0.0010) [2023-12-27 01:42:03,775][105692] Updated weights for policy 0, policy_version 1415446 (0.0007) [2023-12-27 01:42:03,833][105692] Updated weights for policy 0, policy_version 1415456 (0.0010) [2023-12-27 01:42:03,902][105692] Updated weights for policy 0, policy_version 1415466 (0.0011) [2023-12-27 01:42:04,121][105620] Updated weights for policy 1, policy_version 1417614 (0.0009) [2023-12-27 01:42:04,175][105620] Updated weights for policy 1, policy_version 1417624 (0.0007) [2023-12-27 01:42:04,224][105620] Updated weights for policy 1, policy_version 1417634 (0.0008) [2023-12-27 01:42:04,635][105692] Updated weights for policy 0, policy_version 1415476 (0.0011) [2023-12-27 01:42:04,697][105692] Updated weights for policy 0, policy_version 1415486 (0.0009) [2023-12-27 01:42:04,755][105692] Updated weights for policy 0, policy_version 1415496 (0.0010) [2023-12-27 01:42:04,969][105620] Updated weights for policy 1, policy_version 1417644 (0.0008) [2023-12-27 01:42:05,016][105620] Updated weights for policy 1, policy_version 1417654 (0.0007) [2023-12-27 01:42:05,061][105620] Updated weights for policy 1, policy_version 1417664 (0.0008) [2023-12-27 01:42:05,479][105692] Updated weights for policy 0, policy_version 1415506 (0.0010) [2023-12-27 01:42:05,538][105692] Updated weights for policy 0, policy_version 1415516 (0.0011) [2023-12-27 01:42:05,590][105692] Updated weights for policy 0, policy_version 1415526 (0.0010) [2023-12-27 01:42:05,648][105692] Updated weights for policy 0, policy_version 1415536 (0.0010) [2023-12-27 01:42:05,691][105620] Updated weights for policy 1, policy_version 1417675 (0.0009) [2023-12-27 01:42:05,742][105620] Updated weights for policy 1, policy_version 1417685 (0.0007) [2023-12-27 01:42:05,797][105620] Updated weights for policy 1, policy_version 1417695 (0.0008) [2023-12-27 01:42:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 725409792. Throughput: 0: 9834.5, 1: 9765.9. Samples: 725396060. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:42:06,063][104569] Avg episode reward: [(0, '8067.967'), (1, '9081.458')] [2023-12-27 01:42:06,415][105692] Updated weights for policy 0, policy_version 1415546 (0.0006) [2023-12-27 01:42:06,483][105692] Updated weights for policy 0, policy_version 1415556 (0.0006) [2023-12-27 01:42:06,511][105620] Updated weights for policy 1, policy_version 1417705 (0.0008) [2023-12-27 01:42:06,544][105692] Updated weights for policy 0, policy_version 1415566 (0.0009) [2023-12-27 01:42:06,572][105620] Updated weights for policy 1, policy_version 1417715 (0.0006) [2023-12-27 01:42:06,630][105620] Updated weights for policy 1, policy_version 1417725 (0.0006) [2023-12-27 01:42:06,693][105620] Updated weights for policy 1, policy_version 1417735 (0.0005) [2023-12-27 01:42:07,265][105692] Updated weights for policy 0, policy_version 1415576 (0.0011) [2023-12-27 01:42:07,331][105692] Updated weights for policy 0, policy_version 1415586 (0.0010) [2023-12-27 01:42:07,369][105620] Updated weights for policy 1, policy_version 1417745 (0.0010) [2023-12-27 01:42:07,390][105692] Updated weights for policy 0, policy_version 1415596 (0.0010) [2023-12-27 01:42:07,423][105620] Updated weights for policy 1, policy_version 1417755 (0.0009) [2023-12-27 01:42:07,478][105620] Updated weights for policy 1, policy_version 1417765 (0.0010) [2023-12-27 01:42:08,074][105620] Updated weights for policy 1, policy_version 1417775 (0.0010) [2023-12-27 01:42:08,111][105692] Updated weights for policy 0, policy_version 1415606 (0.0010) [2023-12-27 01:42:08,129][105620] Updated weights for policy 1, policy_version 1417785 (0.0010) [2023-12-27 01:42:08,159][105692] Updated weights for policy 0, policy_version 1415616 (0.0010) [2023-12-27 01:42:08,191][105620] Updated weights for policy 1, policy_version 1417795 (0.0010) [2023-12-27 01:42:08,208][105692] Updated weights for policy 0, policy_version 1415626 (0.0008) [2023-12-27 01:42:08,909][105692] Updated weights for policy 0, policy_version 1415636 (0.0006) [2023-12-27 01:42:08,965][105692] Updated weights for policy 0, policy_version 1415646 (0.0005) [2023-12-27 01:42:08,992][105620] Updated weights for policy 1, policy_version 1417805 (0.0008) [2023-12-27 01:42:09,015][105692] Updated weights for policy 0, policy_version 1415656 (0.0005) [2023-12-27 01:42:09,050][105620] Updated weights for policy 1, policy_version 1417815 (0.0008) [2023-12-27 01:42:09,104][105620] Updated weights for policy 1, policy_version 1417825 (0.0008) [2023-12-27 01:42:09,704][105692] Updated weights for policy 0, policy_version 1415666 (0.0007) [2023-12-27 01:42:09,764][105692] Updated weights for policy 0, policy_version 1415676 (0.0008) [2023-12-27 01:42:09,822][105692] Updated weights for policy 0, policy_version 1415686 (0.0008) [2023-12-27 01:42:09,880][105620] Updated weights for policy 1, policy_version 1417835 (0.0008) [2023-12-27 01:42:09,891][105692] Updated weights for policy 0, policy_version 1415696 (0.0008) [2023-12-27 01:42:09,943][105620] Updated weights for policy 1, policy_version 1417845 (0.0008) [2023-12-27 01:42:10,002][105620] Updated weights for policy 1, policy_version 1417855 (0.0008) [2023-12-27 01:42:10,558][105692] Updated weights for policy 0, policy_version 1415706 (0.0005) [2023-12-27 01:42:10,618][105692] Updated weights for policy 0, policy_version 1415716 (0.0007) [2023-12-27 01:42:10,684][105692] Updated weights for policy 0, policy_version 1415726 (0.0011) [2023-12-27 01:42:10,744][105620] Updated weights for policy 1, policy_version 1417865 (0.0008) [2023-12-27 01:42:10,806][105620] Updated weights for policy 1, policy_version 1417875 (0.0009) [2023-12-27 01:42:10,852][105620] Updated weights for policy 1, policy_version 1417885 (0.0010) [2023-12-27 01:42:10,901][105620] Updated weights for policy 1, policy_version 1417895 (0.0008) [2023-12-27 01:42:11,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 725508096. Throughput: 0: 9847.5, 1: 9837.3. Samples: 725513884. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:42:11,062][104569] Avg episode reward: [(0, '7983.819'), (1, '8805.584')] [2023-12-27 01:42:11,445][105692] Updated weights for policy 0, policy_version 1415736 (0.0009) [2023-12-27 01:42:11,504][105692] Updated weights for policy 0, policy_version 1415746 (0.0010) [2023-12-27 01:42:11,560][105692] Updated weights for policy 0, policy_version 1415756 (0.0010) [2023-12-27 01:42:11,615][105620] Updated weights for policy 1, policy_version 1417905 (0.0007) [2023-12-27 01:42:11,683][105620] Updated weights for policy 1, policy_version 1417915 (0.0009) [2023-12-27 01:42:11,752][105620] Updated weights for policy 1, policy_version 1417925 (0.0008) [2023-12-27 01:42:12,292][105692] Updated weights for policy 0, policy_version 1415766 (0.0007) [2023-12-27 01:42:12,300][105585] KL-divergence is very high: 102.9361 [2023-12-27 01:42:12,315][105585] KL-divergence is very high: 144.9807 [2023-12-27 01:42:12,355][105585] KL-divergence is very high: 258.2391 [2023-12-27 01:42:12,362][105692] Updated weights for policy 0, policy_version 1415776 (0.0009) [2023-12-27 01:42:12,371][105585] KL-divergence is very high: 279.8554 [2023-12-27 01:42:12,412][105585] KL-divergence is very high: 329.9454 [2023-12-27 01:42:12,426][105585] KL-divergence is very high: 333.3351 [2023-12-27 01:42:12,433][105692] Updated weights for policy 0, policy_version 1415786 (0.0009) [2023-12-27 01:42:12,464][105585] KL-divergence is very high: 332.0995 [2023-12-27 01:42:12,506][105620] Updated weights for policy 1, policy_version 1417935 (0.0009) [2023-12-27 01:42:12,566][105620] Updated weights for policy 1, policy_version 1417945 (0.0009) [2023-12-27 01:42:12,620][105620] Updated weights for policy 1, policy_version 1417955 (0.0009) [2023-12-27 01:42:13,116][105692] Updated weights for policy 0, policy_version 1415796 (0.0009) [2023-12-27 01:42:13,178][105692] Updated weights for policy 0, policy_version 1415806 (0.0009) [2023-12-27 01:42:13,240][105692] Updated weights for policy 0, policy_version 1415816 (0.0009) [2023-12-27 01:42:13,367][105620] Updated weights for policy 1, policy_version 1417965 (0.0008) [2023-12-27 01:42:13,420][105620] Updated weights for policy 1, policy_version 1417975 (0.0009) [2023-12-27 01:42:13,481][105620] Updated weights for policy 1, policy_version 1417985 (0.0009) [2023-12-27 01:42:13,919][105692] Updated weights for policy 0, policy_version 1415826 (0.0009) [2023-12-27 01:42:13,970][105692] Updated weights for policy 0, policy_version 1415836 (0.0007) [2023-12-27 01:42:14,032][105692] Updated weights for policy 0, policy_version 1415846 (0.0010) [2023-12-27 01:42:14,091][105692] Updated weights for policy 0, policy_version 1415856 (0.0009) [2023-12-27 01:42:14,196][105620] Updated weights for policy 1, policy_version 1417995 (0.0009) [2023-12-27 01:42:14,267][105620] Updated weights for policy 1, policy_version 1418005 (0.0005) [2023-12-27 01:42:14,335][105620] Updated weights for policy 1, policy_version 1418015 (0.0005) [2023-12-27 01:42:14,734][105692] Updated weights for policy 0, policy_version 1415866 (0.0010) [2023-12-27 01:42:14,803][105692] Updated weights for policy 0, policy_version 1415876 (0.0011) [2023-12-27 01:42:14,865][105692] Updated weights for policy 0, policy_version 1415886 (0.0011) [2023-12-27 01:42:15,049][105620] Updated weights for policy 1, policy_version 1418025 (0.0008) [2023-12-27 01:42:15,103][105620] Updated weights for policy 1, policy_version 1418035 (0.0008) [2023-12-27 01:42:15,160][105620] Updated weights for policy 1, policy_version 1418045 (0.0008) [2023-12-27 01:42:15,220][105620] Updated weights for policy 1, policy_version 1418055 (0.0008) [2023-12-27 01:42:15,588][105692] Updated weights for policy 0, policy_version 1415896 (0.0006) [2023-12-27 01:42:15,638][105692] Updated weights for policy 0, policy_version 1415906 (0.0006) [2023-12-27 01:42:15,686][105692] Updated weights for policy 0, policy_version 1415916 (0.0005) [2023-12-27 01:42:15,815][105620] Updated weights for policy 1, policy_version 1418065 (0.0008) [2023-12-27 01:42:15,880][105620] Updated weights for policy 1, policy_version 1418075 (0.0008) [2023-12-27 01:42:15,933][105620] Updated weights for policy 1, policy_version 1418085 (0.0009) [2023-12-27 01:42:16,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 725606400. Throughput: 0: 9696.5, 1: 9869.6. Samples: 725570324. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:42:16,062][104569] Avg episode reward: [(0, '7433.873'), (1, '8163.585')] [2023-12-27 01:42:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001415920_362528768.pth... [2023-12-27 01:42:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001418088_363077632.pth... [2023-12-27 01:42:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001414768_362233856.pth [2023-12-27 01:42:16,082][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001416936_362782720.pth [2023-12-27 01:42:16,355][105692] Updated weights for policy 0, policy_version 1415926 (0.0008) [2023-12-27 01:42:16,409][105692] Updated weights for policy 0, policy_version 1415936 (0.0010) [2023-12-27 01:42:16,471][105692] Updated weights for policy 0, policy_version 1415946 (0.0010) [2023-12-27 01:42:16,496][105620] Updated weights for policy 1, policy_version 1418095 (0.0008) [2023-12-27 01:42:16,546][105620] Updated weights for policy 1, policy_version 1418105 (0.0008) [2023-12-27 01:42:16,598][105620] Updated weights for policy 1, policy_version 1418115 (0.0009) [2023-12-27 01:42:17,056][105692] Updated weights for policy 0, policy_version 1415956 (0.0009) [2023-12-27 01:42:17,102][105692] Updated weights for policy 0, policy_version 1415966 (0.0005) [2023-12-27 01:42:17,151][105692] Updated weights for policy 0, policy_version 1415976 (0.0006) [2023-12-27 01:42:17,368][105620] Updated weights for policy 1, policy_version 1418125 (0.0009) [2023-12-27 01:42:17,429][105620] Updated weights for policy 1, policy_version 1418135 (0.0009) [2023-12-27 01:42:17,489][105620] Updated weights for policy 1, policy_version 1418145 (0.0008) [2023-12-27 01:42:17,861][105692] Updated weights for policy 0, policy_version 1415986 (0.0009) [2023-12-27 01:42:17,933][105692] Updated weights for policy 0, policy_version 1415996 (0.0010) [2023-12-27 01:42:18,003][105692] Updated weights for policy 0, policy_version 1416006 (0.0009) [2023-12-27 01:42:18,069][105692] Updated weights for policy 0, policy_version 1416016 (0.0010) [2023-12-27 01:42:18,117][105620] Updated weights for policy 1, policy_version 1418155 (0.0009) [2023-12-27 01:42:18,174][105620] Updated weights for policy 1, policy_version 1418165 (0.0009) [2023-12-27 01:42:18,229][105620] Updated weights for policy 1, policy_version 1418175 (0.0009) [2023-12-27 01:42:18,775][105692] Updated weights for policy 0, policy_version 1416026 (0.0009) [2023-12-27 01:42:18,841][105692] Updated weights for policy 0, policy_version 1416036 (0.0011) [2023-12-27 01:42:18,896][105692] Updated weights for policy 0, policy_version 1416046 (0.0010) [2023-12-27 01:42:19,028][105620] Updated weights for policy 1, policy_version 1418185 (0.0010) [2023-12-27 01:42:19,085][105620] Updated weights for policy 1, policy_version 1418196 (0.0010) [2023-12-27 01:42:19,136][105620] Updated weights for policy 1, policy_version 1418207 (0.0010) [2023-12-27 01:42:19,579][105692] Updated weights for policy 0, policy_version 1416056 (0.0010) [2023-12-27 01:42:19,640][105692] Updated weights for policy 0, policy_version 1416066 (0.0009) [2023-12-27 01:42:19,707][105692] Updated weights for policy 0, policy_version 1416076 (0.0010) [2023-12-27 01:42:19,956][105620] Updated weights for policy 1, policy_version 1418217 (0.0009) [2023-12-27 01:42:20,016][105620] Updated weights for policy 1, policy_version 1418227 (0.0008) [2023-12-27 01:42:20,075][105620] Updated weights for policy 1, policy_version 1418237 (0.0007) [2023-12-27 01:42:20,136][105620] Updated weights for policy 1, policy_version 1418247 (0.0008) [2023-12-27 01:42:20,469][105692] Updated weights for policy 0, policy_version 1416086 (0.0010) [2023-12-27 01:42:20,521][105692] Updated weights for policy 0, policy_version 1416096 (0.0010) [2023-12-27 01:42:20,583][105692] Updated weights for policy 0, policy_version 1416106 (0.0011) [2023-12-27 01:42:20,923][105620] Updated weights for policy 1, policy_version 1418257 (0.0008) [2023-12-27 01:42:20,992][105620] Updated weights for policy 1, policy_version 1418267 (0.0008) [2023-12-27 01:42:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 725696512. Throughput: 0: 9780.2, 1: 9920.4. Samples: 725691400. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:42:21,062][104569] Avg episode reward: [(0, '7335.438'), (1, '7745.635')] [2023-12-27 01:42:21,065][105620] Updated weights for policy 1, policy_version 1418277 (0.0008) [2023-12-27 01:42:21,352][105692] Updated weights for policy 0, policy_version 1416116 (0.0010) [2023-12-27 01:42:21,419][105692] Updated weights for policy 0, policy_version 1416126 (0.0009) [2023-12-27 01:42:21,482][105692] Updated weights for policy 0, policy_version 1416136 (0.0010) [2023-12-27 01:42:21,847][105620] Updated weights for policy 1, policy_version 1418287 (0.0008) [2023-12-27 01:42:21,916][105620] Updated weights for policy 1, policy_version 1418297 (0.0007) [2023-12-27 01:42:21,986][105620] Updated weights for policy 1, policy_version 1418307 (0.0008) [2023-12-27 01:42:22,262][105692] Updated weights for policy 0, policy_version 1416146 (0.0010) [2023-12-27 01:42:22,314][105692] Updated weights for policy 0, policy_version 1416156 (0.0011) [2023-12-27 01:42:22,386][105692] Updated weights for policy 0, policy_version 1416166 (0.0010) [2023-12-27 01:42:22,436][105692] Updated weights for policy 0, policy_version 1416176 (0.0008) [2023-12-27 01:42:22,570][105620] Updated weights for policy 1, policy_version 1418317 (0.0007) [2023-12-27 01:42:22,630][105620] Updated weights for policy 1, policy_version 1418327 (0.0008) [2023-12-27 01:42:22,689][105620] Updated weights for policy 1, policy_version 1418337 (0.0009) [2023-12-27 01:42:23,207][105692] Updated weights for policy 0, policy_version 1416186 (0.0007) [2023-12-27 01:42:23,270][105692] Updated weights for policy 0, policy_version 1416196 (0.0008) [2023-12-27 01:42:23,325][105692] Updated weights for policy 0, policy_version 1416206 (0.0009) [2023-12-27 01:42:23,490][105620] Updated weights for policy 1, policy_version 1418347 (0.0009) [2023-12-27 01:42:23,541][105620] Updated weights for policy 1, policy_version 1418357 (0.0009) [2023-12-27 01:42:23,595][105620] Updated weights for policy 1, policy_version 1418367 (0.0008) [2023-12-27 01:42:23,916][105692] Updated weights for policy 0, policy_version 1416216 (0.0008) [2023-12-27 01:42:23,973][105692] Updated weights for policy 0, policy_version 1416226 (0.0009) [2023-12-27 01:42:24,039][105692] Updated weights for policy 0, policy_version 1416236 (0.0009) [2023-12-27 01:42:24,345][105620] Updated weights for policy 1, policy_version 1418377 (0.0009) [2023-12-27 01:42:24,405][105620] Updated weights for policy 1, policy_version 1418387 (0.0011) [2023-12-27 01:42:24,450][105620] Updated weights for policy 1, policy_version 1418397 (0.0011) [2023-12-27 01:42:24,502][105620] Updated weights for policy 1, policy_version 1418407 (0.0011) [2023-12-27 01:42:24,828][105692] Updated weights for policy 0, policy_version 1416246 (0.0008) [2023-12-27 01:42:24,885][105692] Updated weights for policy 0, policy_version 1416256 (0.0007) [2023-12-27 01:42:24,955][105692] Updated weights for policy 0, policy_version 1416266 (0.0005) [2023-12-27 01:42:25,247][105620] Updated weights for policy 1, policy_version 1418417 (0.0006) [2023-12-27 01:42:25,293][105620] Updated weights for policy 1, policy_version 1418427 (0.0005) [2023-12-27 01:42:25,351][105620] Updated weights for policy 1, policy_version 1418437 (0.0006) [2023-12-27 01:42:25,613][105692] Updated weights for policy 0, policy_version 1416276 (0.0007) [2023-12-27 01:42:25,667][105692] Updated weights for policy 0, policy_version 1416286 (0.0008) [2023-12-27 01:42:25,718][105692] Updated weights for policy 0, policy_version 1416296 (0.0006) [2023-12-27 01:42:26,000][105620] Updated weights for policy 1, policy_version 1418447 (0.0005) [2023-12-27 01:42:26,047][105620] Updated weights for policy 1, policy_version 1418457 (0.0005) [2023-12-27 01:42:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 725794816. Throughput: 0: 9765.5, 1: 9911.6. Samples: 725805600. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:42:26,063][104569] Avg episode reward: [(0, '6323.481'), (1, '8272.575')] [2023-12-27 01:42:26,091][105620] Updated weights for policy 1, policy_version 1418467 (0.0006) [2023-12-27 01:42:26,313][105692] Updated weights for policy 0, policy_version 1416306 (0.0006) [2023-12-27 01:42:26,362][105692] Updated weights for policy 0, policy_version 1416316 (0.0005) [2023-12-27 01:42:26,415][105692] Updated weights for policy 0, policy_version 1416326 (0.0005) [2023-12-27 01:42:26,473][105692] Updated weights for policy 0, policy_version 1416336 (0.0005) [2023-12-27 01:42:26,702][105620] Updated weights for policy 1, policy_version 1418477 (0.0008) [2023-12-27 01:42:26,760][105620] Updated weights for policy 1, policy_version 1418487 (0.0010) [2023-12-27 01:42:26,804][105620] Updated weights for policy 1, policy_version 1418497 (0.0010) [2023-12-27 01:42:27,015][105692] Updated weights for policy 0, policy_version 1416346 (0.0005) [2023-12-27 01:42:27,061][105692] Updated weights for policy 0, policy_version 1416356 (0.0005) [2023-12-27 01:42:27,065][105585] KL-divergence is very high: 153.7864 [2023-12-27 01:42:27,108][105585] KL-divergence is very high: 187.5589 [2023-12-27 01:42:27,114][105692] Updated weights for policy 0, policy_version 1416366 (0.0005) [2023-12-27 01:42:27,502][105620] Updated weights for policy 1, policy_version 1418507 (0.0009) [2023-12-27 01:42:27,552][105620] Updated weights for policy 1, policy_version 1418517 (0.0009) [2023-12-27 01:42:27,603][105620] Updated weights for policy 1, policy_version 1418527 (0.0010) [2023-12-27 01:42:27,651][105692] Updated weights for policy 0, policy_version 1416376 (0.0009) [2023-12-27 01:42:27,702][105692] Updated weights for policy 0, policy_version 1416386 (0.0010) [2023-12-27 01:42:27,750][105692] Updated weights for policy 0, policy_version 1416396 (0.0008) [2023-12-27 01:42:28,306][105620] Updated weights for policy 1, policy_version 1418537 (0.0007) [2023-12-27 01:42:28,333][105692] Updated weights for policy 0, policy_version 1416406 (0.0006) [2023-12-27 01:42:28,368][105620] Updated weights for policy 1, policy_version 1418547 (0.0010) [2023-12-27 01:42:28,391][105692] Updated weights for policy 0, policy_version 1416416 (0.0007) [2023-12-27 01:42:28,432][105620] Updated weights for policy 1, policy_version 1418557 (0.0007) [2023-12-27 01:42:28,441][105692] Updated weights for policy 0, policy_version 1416426 (0.0011) [2023-12-27 01:42:28,490][105620] Updated weights for policy 1, policy_version 1418567 (0.0011) [2023-12-27 01:42:29,189][105692] Updated weights for policy 0, policy_version 1416436 (0.0010) [2023-12-27 01:42:29,208][105620] Updated weights for policy 1, policy_version 1418577 (0.0007) [2023-12-27 01:42:29,256][105692] Updated weights for policy 0, policy_version 1416446 (0.0008) [2023-12-27 01:42:29,279][105620] Updated weights for policy 1, policy_version 1418587 (0.0007) [2023-12-27 01:42:29,325][105692] Updated weights for policy 0, policy_version 1416456 (0.0008) [2023-12-27 01:42:29,342][105620] Updated weights for policy 1, policy_version 1418597 (0.0008) [2023-12-27 01:42:29,961][105620] Updated weights for policy 1, policy_version 1418607 (0.0009) [2023-12-27 01:42:30,014][105620] Updated weights for policy 1, policy_version 1418617 (0.0010) [2023-12-27 01:42:30,075][105620] Updated weights for policy 1, policy_version 1418627 (0.0007) [2023-12-27 01:42:30,088][105692] Updated weights for policy 0, policy_version 1416466 (0.0005) [2023-12-27 01:42:30,144][105692] Updated weights for policy 0, policy_version 1416476 (0.0005) [2023-12-27 01:42:30,202][105692] Updated weights for policy 0, policy_version 1416486 (0.0005) [2023-12-27 01:42:30,269][105692] Updated weights for policy 0, policy_version 1416496 (0.0008) [2023-12-27 01:42:30,681][105620] Updated weights for policy 1, policy_version 1418637 (0.0007) [2023-12-27 01:42:30,735][105620] Updated weights for policy 1, policy_version 1418647 (0.0009) [2023-12-27 01:42:30,793][105620] Updated weights for policy 1, policy_version 1418657 (0.0009) [2023-12-27 01:42:30,976][105692] Updated weights for policy 0, policy_version 1416506 (0.0009) [2023-12-27 01:42:31,029][105692] Updated weights for policy 0, policy_version 1416516 (0.0009) [2023-12-27 01:42:31,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 725901312. Throughput: 0: 9918.6, 1: 9896.4. Samples: 725871984. Policy #0 lag: (min: 15.0, avg: 15.8, max: 37.0) [2023-12-27 01:42:31,063][104569] Avg episode reward: [(0, '6874.615'), (1, '8832.090')] [2023-12-27 01:42:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001418664_363225088.pth... [2023-12-27 01:42:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001417512_362930176.pth [2023-12-27 01:42:31,099][105692] Updated weights for policy 0, policy_version 1416526 (0.0010) [2023-12-27 01:42:31,108][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001416528_362684416.pth... [2023-12-27 01:42:31,112][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001415344_362381312.pth [2023-12-27 01:42:31,591][105620] Updated weights for policy 1, policy_version 1418667 (0.0009) [2023-12-27 01:42:31,647][105620] Updated weights for policy 1, policy_version 1418677 (0.0009) [2023-12-27 01:42:31,709][105620] Updated weights for policy 1, policy_version 1418687 (0.0008) [2023-12-27 01:42:31,899][105692] Updated weights for policy 0, policy_version 1416536 (0.0008) [2023-12-27 01:42:31,971][105692] Updated weights for policy 0, policy_version 1416546 (0.0009) [2023-12-27 01:42:32,037][105692] Updated weights for policy 0, policy_version 1416556 (0.0010) [2023-12-27 01:42:32,338][105620] Updated weights for policy 1, policy_version 1418697 (0.0008) [2023-12-27 01:42:32,403][105620] Updated weights for policy 1, policy_version 1418707 (0.0008) [2023-12-27 01:42:32,430][105586] KL-divergence is very high: 101.2507 [2023-12-27 01:42:32,454][105620] Updated weights for policy 1, policy_version 1418717 (0.0005) [2023-12-27 01:42:32,470][105586] KL-divergence is very high: 104.8029 [2023-12-27 01:42:32,505][105620] Updated weights for policy 1, policy_version 1418727 (0.0005) [2023-12-27 01:42:32,891][105692] Updated weights for policy 0, policy_version 1416566 (0.0009) [2023-12-27 01:42:32,949][105692] Updated weights for policy 0, policy_version 1416577 (0.0010) [2023-12-27 01:42:33,006][105692] Updated weights for policy 0, policy_version 1416587 (0.0009) [2023-12-27 01:42:33,106][105620] Updated weights for policy 1, policy_version 1418737 (0.0005) [2023-12-27 01:42:33,153][105620] Updated weights for policy 1, policy_version 1418747 (0.0005) [2023-12-27 01:42:33,208][105620] Updated weights for policy 1, policy_version 1418757 (0.0007) [2023-12-27 01:42:33,785][105620] Updated weights for policy 1, policy_version 1418767 (0.0009) [2023-12-27 01:42:33,835][105620] Updated weights for policy 1, policy_version 1418777 (0.0009) [2023-12-27 01:42:33,850][105692] Updated weights for policy 0, policy_version 1416597 (0.0007) [2023-12-27 01:42:33,884][105620] Updated weights for policy 1, policy_version 1418787 (0.0008) [2023-12-27 01:42:33,910][105692] Updated weights for policy 0, policy_version 1416607 (0.0008) [2023-12-27 01:42:33,963][105692] Updated weights for policy 0, policy_version 1416617 (0.0009) [2023-12-27 01:42:34,678][105620] Updated weights for policy 1, policy_version 1418797 (0.0006) [2023-12-27 01:42:34,691][105692] Updated weights for policy 0, policy_version 1416627 (0.0009) [2023-12-27 01:42:34,727][105620] Updated weights for policy 1, policy_version 1418807 (0.0005) [2023-12-27 01:42:34,748][105692] Updated weights for policy 0, policy_version 1416637 (0.0009) [2023-12-27 01:42:34,784][105620] Updated weights for policy 1, policy_version 1418817 (0.0007) [2023-12-27 01:42:34,808][105692] Updated weights for policy 0, policy_version 1416647 (0.0008) [2023-12-27 01:42:35,425][105692] Updated weights for policy 0, policy_version 1416657 (0.0007) [2023-12-27 01:42:35,472][105692] Updated weights for policy 0, policy_version 1416667 (0.0009) [2023-12-27 01:42:35,533][105692] Updated weights for policy 0, policy_version 1416677 (0.0009) [2023-12-27 01:42:35,573][105620] Updated weights for policy 1, policy_version 1418827 (0.0008) [2023-12-27 01:42:35,583][105692] Updated weights for policy 0, policy_version 1416687 (0.0007) [2023-12-27 01:42:35,628][105620] Updated weights for policy 1, policy_version 1418837 (0.0007) [2023-12-27 01:42:35,689][105620] Updated weights for policy 1, policy_version 1418847 (0.0009) [2023-12-27 01:42:36,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 725999616. Throughput: 0: 9798.2, 1: 9902.9. Samples: 725987964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:42:36,062][104569] Avg episode reward: [(0, '6965.781'), (1, '8913.438')] [2023-12-27 01:42:36,321][105692] Updated weights for policy 0, policy_version 1416697 (0.0009) [2023-12-27 01:42:36,391][105692] Updated weights for policy 0, policy_version 1416707 (0.0009) [2023-12-27 01:42:36,450][105620] Updated weights for policy 1, policy_version 1418857 (0.0009) [2023-12-27 01:42:36,452][105692] Updated weights for policy 0, policy_version 1416717 (0.0009) [2023-12-27 01:42:36,503][105620] Updated weights for policy 1, policy_version 1418867 (0.0007) [2023-12-27 01:42:36,562][105620] Updated weights for policy 1, policy_version 1418877 (0.0009) [2023-12-27 01:42:36,617][105620] Updated weights for policy 1, policy_version 1418887 (0.0009) [2023-12-27 01:42:37,154][105692] Updated weights for policy 0, policy_version 1416727 (0.0008) [2023-12-27 01:42:37,216][105692] Updated weights for policy 0, policy_version 1416737 (0.0009) [2023-12-27 01:42:37,277][105692] Updated weights for policy 0, policy_version 1416747 (0.0009) [2023-12-27 01:42:37,394][105620] Updated weights for policy 1, policy_version 1418897 (0.0009) [2023-12-27 01:42:37,455][105620] Updated weights for policy 1, policy_version 1418907 (0.0009) [2023-12-27 01:42:37,514][105620] Updated weights for policy 1, policy_version 1418917 (0.0009) [2023-12-27 01:42:38,007][105692] Updated weights for policy 0, policy_version 1416757 (0.0009) [2023-12-27 01:42:38,058][105692] Updated weights for policy 0, policy_version 1416767 (0.0009) [2023-12-27 01:42:38,116][105692] Updated weights for policy 0, policy_version 1416777 (0.0009) [2023-12-27 01:42:38,281][105620] Updated weights for policy 1, policy_version 1418927 (0.0009) [2023-12-27 01:42:38,327][105620] Updated weights for policy 1, policy_version 1418937 (0.0008) [2023-12-27 01:42:38,386][105620] Updated weights for policy 1, policy_version 1418947 (0.0009) [2023-12-27 01:42:38,918][105692] Updated weights for policy 0, policy_version 1416787 (0.0009) [2023-12-27 01:42:38,984][105692] Updated weights for policy 0, policy_version 1416797 (0.0010) [2023-12-27 01:42:39,036][105692] Updated weights for policy 0, policy_version 1416807 (0.0009) [2023-12-27 01:42:39,063][105620] Updated weights for policy 1, policy_version 1418957 (0.0007) [2023-12-27 01:42:39,105][105620] Updated weights for policy 1, policy_version 1418967 (0.0006) [2023-12-27 01:42:39,155][105620] Updated weights for policy 1, policy_version 1418978 (0.0008) [2023-12-27 01:42:39,710][105692] Updated weights for policy 0, policy_version 1416817 (0.0007) [2023-12-27 01:42:39,763][105692] Updated weights for policy 0, policy_version 1416827 (0.0009) [2023-12-27 01:42:39,826][105692] Updated weights for policy 0, policy_version 1416837 (0.0011) [2023-12-27 01:42:39,897][105692] Updated weights for policy 0, policy_version 1416847 (0.0011) [2023-12-27 01:42:39,916][105620] Updated weights for policy 1, policy_version 1418988 (0.0006) [2023-12-27 01:42:39,979][105620] Updated weights for policy 1, policy_version 1418998 (0.0008) [2023-12-27 01:42:40,035][105620] Updated weights for policy 1, policy_version 1419008 (0.0008) [2023-12-27 01:42:40,616][105620] Updated weights for policy 1, policy_version 1419018 (0.0007) [2023-12-27 01:42:40,660][105692] Updated weights for policy 0, policy_version 1416857 (0.0011) [2023-12-27 01:42:40,666][105620] Updated weights for policy 1, policy_version 1419028 (0.0005) [2023-12-27 01:42:40,709][105692] Updated weights for policy 0, policy_version 1416867 (0.0010) [2023-12-27 01:42:40,716][105620] Updated weights for policy 1, policy_version 1419038 (0.0005) [2023-12-27 01:42:40,765][105692] Updated weights for policy 0, policy_version 1416877 (0.0011) [2023-12-27 01:42:40,767][105620] Updated weights for policy 1, policy_version 1419048 (0.0006) [2023-12-27 01:42:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 726097920. Throughput: 0: 9757.2, 1: 9820.9. Samples: 726104448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:42:41,062][104569] Avg episode reward: [(0, '6684.314'), (1, '8462.642')] [2023-12-27 01:42:41,475][105620] Updated weights for policy 1, policy_version 1419058 (0.0008) [2023-12-27 01:42:41,533][105620] Updated weights for policy 1, policy_version 1419068 (0.0011) [2023-12-27 01:42:41,552][105692] Updated weights for policy 0, policy_version 1416887 (0.0009) [2023-12-27 01:42:41,589][105620] Updated weights for policy 1, policy_version 1419078 (0.0011) [2023-12-27 01:42:41,618][105692] Updated weights for policy 0, policy_version 1416897 (0.0008) [2023-12-27 01:42:41,688][105692] Updated weights for policy 0, policy_version 1416907 (0.0008) [2023-12-27 01:42:42,316][105620] Updated weights for policy 1, policy_version 1419088 (0.0010) [2023-12-27 01:42:42,386][105620] Updated weights for policy 1, policy_version 1419098 (0.0010) [2023-12-27 01:42:42,394][105692] Updated weights for policy 0, policy_version 1416917 (0.0008) [2023-12-27 01:42:42,457][105620] Updated weights for policy 1, policy_version 1419108 (0.0011) [2023-12-27 01:42:42,460][105692] Updated weights for policy 0, policy_version 1416927 (0.0007) [2023-12-27 01:42:42,514][105692] Updated weights for policy 0, policy_version 1416937 (0.0007) [2023-12-27 01:42:43,231][105620] Updated weights for policy 1, policy_version 1419118 (0.0011) [2023-12-27 01:42:43,268][105692] Updated weights for policy 0, policy_version 1416947 (0.0007) [2023-12-27 01:42:43,293][105620] Updated weights for policy 1, policy_version 1419128 (0.0010) [2023-12-27 01:42:43,319][105692] Updated weights for policy 0, policy_version 1416957 (0.0006) [2023-12-27 01:42:43,348][105620] Updated weights for policy 1, policy_version 1419138 (0.0010) [2023-12-27 01:42:43,363][105585] KL-divergence is very high: 108.1351 [2023-12-27 01:42:43,374][105692] Updated weights for policy 0, policy_version 1416967 (0.0005) [2023-12-27 01:42:43,406][105585] KL-divergence is very high: 128.1157 [2023-12-27 01:42:44,084][105620] Updated weights for policy 1, policy_version 1419148 (0.0010) [2023-12-27 01:42:44,132][105692] Updated weights for policy 0, policy_version 1416977 (0.0008) [2023-12-27 01:42:44,141][105620] Updated weights for policy 1, policy_version 1419158 (0.0009) [2023-12-27 01:42:44,188][105692] Updated weights for policy 0, policy_version 1416987 (0.0008) [2023-12-27 01:42:44,191][105620] Updated weights for policy 1, policy_version 1419168 (0.0005) [2023-12-27 01:42:44,241][105692] Updated weights for policy 0, policy_version 1416997 (0.0009) [2023-12-27 01:42:44,306][105692] Updated weights for policy 0, policy_version 1417007 (0.0009) [2023-12-27 01:42:44,945][105620] Updated weights for policy 1, policy_version 1419178 (0.0005) [2023-12-27 01:42:45,017][105620] Updated weights for policy 1, policy_version 1419188 (0.0005) [2023-12-27 01:42:45,078][105620] Updated weights for policy 1, policy_version 1419198 (0.0010) [2023-12-27 01:42:45,120][105692] Updated weights for policy 0, policy_version 1417017 (0.0008) [2023-12-27 01:42:45,138][105620] Updated weights for policy 1, policy_version 1419208 (0.0011) [2023-12-27 01:42:45,174][105692] Updated weights for policy 0, policy_version 1417027 (0.0007) [2023-12-27 01:42:45,233][105692] Updated weights for policy 0, policy_version 1417037 (0.0010) [2023-12-27 01:42:45,768][105620] Updated weights for policy 1, policy_version 1419218 (0.0005) [2023-12-27 01:42:45,819][105620] Updated weights for policy 1, policy_version 1419228 (0.0005) [2023-12-27 01:42:45,871][105620] Updated weights for policy 1, policy_version 1419238 (0.0010) [2023-12-27 01:42:45,949][105692] Updated weights for policy 0, policy_version 1417047 (0.0007) [2023-12-27 01:42:45,998][105692] Updated weights for policy 0, policy_version 1417057 (0.0005) [2023-12-27 01:42:46,049][105692] Updated weights for policy 0, policy_version 1417067 (0.0005) [2023-12-27 01:42:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 726188032. Throughput: 0: 9730.3, 1: 9789.4. Samples: 726159980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:42:46,062][104569] Avg episode reward: [(0, '6779.496'), (1, '7890.353')] [2023-12-27 01:42:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001419240_363372544.pth... [2023-12-27 01:42:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001418088_363077632.pth [2023-12-27 01:42:46,075][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001417072_362823680.pth... [2023-12-27 01:42:46,080][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001415920_362528768.pth [2023-12-27 01:42:46,576][105620] Updated weights for policy 1, policy_version 1419248 (0.0010) [2023-12-27 01:42:46,641][105620] Updated weights for policy 1, policy_version 1419258 (0.0011) [2023-12-27 01:42:46,703][105620] Updated weights for policy 1, policy_version 1419268 (0.0011) [2023-12-27 01:42:46,736][105692] Updated weights for policy 0, policy_version 1417077 (0.0007) [2023-12-27 01:42:46,789][105692] Updated weights for policy 0, policy_version 1417087 (0.0008) [2023-12-27 01:42:46,839][105692] Updated weights for policy 0, policy_version 1417097 (0.0010) [2023-12-27 01:42:47,373][105620] Updated weights for policy 1, policy_version 1419278 (0.0010) [2023-12-27 01:42:47,421][105620] Updated weights for policy 1, policy_version 1419288 (0.0010) [2023-12-27 01:42:47,465][105620] Updated weights for policy 1, policy_version 1419298 (0.0010) [2023-12-27 01:42:47,583][105692] Updated weights for policy 0, policy_version 1417107 (0.0010) [2023-12-27 01:42:47,643][105692] Updated weights for policy 0, policy_version 1417117 (0.0005) [2023-12-27 01:42:47,701][105692] Updated weights for policy 0, policy_version 1417127 (0.0005) [2023-12-27 01:42:48,210][105620] Updated weights for policy 1, policy_version 1419308 (0.0010) [2023-12-27 01:42:48,268][105620] Updated weights for policy 1, policy_version 1419318 (0.0010) [2023-12-27 01:42:48,315][105620] Updated weights for policy 1, policy_version 1419328 (0.0010) [2023-12-27 01:42:48,382][105692] Updated weights for policy 0, policy_version 1417137 (0.0010) [2023-12-27 01:42:48,441][105692] Updated weights for policy 0, policy_version 1417147 (0.0010) [2023-12-27 01:42:48,489][105692] Updated weights for policy 0, policy_version 1417157 (0.0010) [2023-12-27 01:42:48,553][105692] Updated weights for policy 0, policy_version 1417167 (0.0010) [2023-12-27 01:42:49,064][105620] Updated weights for policy 1, policy_version 1419338 (0.0011) [2023-12-27 01:42:49,118][105620] Updated weights for policy 1, policy_version 1419348 (0.0010) [2023-12-27 01:42:49,176][105620] Updated weights for policy 1, policy_version 1419358 (0.0010) [2023-12-27 01:42:49,228][105620] Updated weights for policy 1, policy_version 1419368 (0.0010) [2023-12-27 01:42:49,311][105692] Updated weights for policy 0, policy_version 1417177 (0.0010) [2023-12-27 01:42:49,378][105692] Updated weights for policy 0, policy_version 1417187 (0.0011) [2023-12-27 01:42:49,440][105692] Updated weights for policy 0, policy_version 1417197 (0.0010) [2023-12-27 01:42:50,001][105620] Updated weights for policy 1, policy_version 1419378 (0.0008) [2023-12-27 01:42:50,068][105620] Updated weights for policy 1, policy_version 1419388 (0.0007) [2023-12-27 01:42:50,128][105620] Updated weights for policy 1, policy_version 1419398 (0.0008) [2023-12-27 01:42:50,170][105692] Updated weights for policy 0, policy_version 1417207 (0.0011) [2023-12-27 01:42:50,231][105692] Updated weights for policy 0, policy_version 1417217 (0.0010) [2023-12-27 01:42:50,280][105692] Updated weights for policy 0, policy_version 1417227 (0.0010) [2023-12-27 01:42:50,868][105620] Updated weights for policy 1, policy_version 1419408 (0.0008) [2023-12-27 01:42:50,913][105620] Updated weights for policy 1, policy_version 1419418 (0.0008) [2023-12-27 01:42:50,962][105620] Updated weights for policy 1, policy_version 1419428 (0.0008) [2023-12-27 01:42:51,058][105692] Updated weights for policy 0, policy_version 1417237 (0.0011) [2023-12-27 01:42:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 726286336. Throughput: 0: 9725.4, 1: 9839.9. Samples: 726276492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:42:51,062][104569] Avg episode reward: [(0, '7432.909'), (1, '7633.838')] [2023-12-27 01:42:51,121][105692] Updated weights for policy 0, policy_version 1417247 (0.0011) [2023-12-27 01:42:51,180][105692] Updated weights for policy 0, policy_version 1417257 (0.0011) [2023-12-27 01:42:51,821][105620] Updated weights for policy 1, policy_version 1419438 (0.0008) [2023-12-27 01:42:51,850][105692] Updated weights for policy 0, policy_version 1417267 (0.0010) [2023-12-27 01:42:51,878][105620] Updated weights for policy 1, policy_version 1419448 (0.0007) [2023-12-27 01:42:51,914][105692] Updated weights for policy 0, policy_version 1417277 (0.0008) [2023-12-27 01:42:51,933][105620] Updated weights for policy 1, policy_version 1419458 (0.0006) [2023-12-27 01:42:51,970][105692] Updated weights for policy 0, policy_version 1417287 (0.0008) [2023-12-27 01:42:52,625][105620] Updated weights for policy 1, policy_version 1419468 (0.0010) [2023-12-27 01:42:52,690][105620] Updated weights for policy 1, policy_version 1419478 (0.0010) [2023-12-27 01:42:52,753][105620] Updated weights for policy 1, policy_version 1419488 (0.0011) [2023-12-27 01:42:52,762][105692] Updated weights for policy 0, policy_version 1417297 (0.0008) [2023-12-27 01:42:52,823][105692] Updated weights for policy 0, policy_version 1417307 (0.0009) [2023-12-27 01:42:52,887][105692] Updated weights for policy 0, policy_version 1417317 (0.0008) [2023-12-27 01:42:52,950][105692] Updated weights for policy 0, policy_version 1417327 (0.0008) [2023-12-27 01:42:53,410][105620] Updated weights for policy 1, policy_version 1419498 (0.0009) [2023-12-27 01:42:53,471][105620] Updated weights for policy 1, policy_version 1419508 (0.0005) [2023-12-27 01:42:53,529][105620] Updated weights for policy 1, policy_version 1419518 (0.0006) [2023-12-27 01:42:53,580][105620] Updated weights for policy 1, policy_version 1419528 (0.0005) [2023-12-27 01:42:53,704][105692] Updated weights for policy 0, policy_version 1417337 (0.0010) [2023-12-27 01:42:53,769][105692] Updated weights for policy 0, policy_version 1417347 (0.0010) [2023-12-27 01:42:53,836][105692] Updated weights for policy 0, policy_version 1417357 (0.0008) [2023-12-27 01:42:54,094][105620] Updated weights for policy 1, policy_version 1419538 (0.0005) [2023-12-27 01:42:54,155][105620] Updated weights for policy 1, policy_version 1419548 (0.0009) [2023-12-27 01:42:54,204][105620] Updated weights for policy 1, policy_version 1419558 (0.0010) [2023-12-27 01:42:54,387][105692] Updated weights for policy 0, policy_version 1417367 (0.0010) [2023-12-27 01:42:54,446][105692] Updated weights for policy 0, policy_version 1417377 (0.0010) [2023-12-27 01:42:54,514][105692] Updated weights for policy 0, policy_version 1417387 (0.0011) [2023-12-27 01:42:54,809][105620] Updated weights for policy 1, policy_version 1419568 (0.0006) [2023-12-27 01:42:54,864][105620] Updated weights for policy 1, policy_version 1419578 (0.0005) [2023-12-27 01:42:54,925][105620] Updated weights for policy 1, policy_version 1419588 (0.0005) [2023-12-27 01:42:55,220][105692] Updated weights for policy 0, policy_version 1417397 (0.0010) [2023-12-27 01:42:55,272][105692] Updated weights for policy 0, policy_version 1417407 (0.0010) [2023-12-27 01:42:55,323][105692] Updated weights for policy 0, policy_version 1417417 (0.0010) [2023-12-27 01:42:55,513][105620] Updated weights for policy 1, policy_version 1419598 (0.0009) [2023-12-27 01:42:55,572][105620] Updated weights for policy 1, policy_version 1419608 (0.0010) [2023-12-27 01:42:55,624][105620] Updated weights for policy 1, policy_version 1419618 (0.0010) [2023-12-27 01:42:55,926][105692] Updated weights for policy 0, policy_version 1417427 (0.0009) [2023-12-27 01:42:55,980][105692] Updated weights for policy 0, policy_version 1417437 (0.0006) [2023-12-27 01:42:56,047][105692] Updated weights for policy 0, policy_version 1417447 (0.0007) [2023-12-27 01:42:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 726384640. Throughput: 0: 9720.0, 1: 9890.7. Samples: 726396364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:42:56,062][104569] Avg episode reward: [(0, '7618.565'), (1, '8812.719')] [2023-12-27 01:42:56,303][105620] Updated weights for policy 1, policy_version 1419628 (0.0010) [2023-12-27 01:42:56,351][105620] Updated weights for policy 1, policy_version 1419638 (0.0010) [2023-12-27 01:42:56,403][105620] Updated weights for policy 1, policy_version 1419648 (0.0010) [2023-12-27 01:42:56,640][105692] Updated weights for policy 0, policy_version 1417457 (0.0008) [2023-12-27 01:42:56,697][105692] Updated weights for policy 0, policy_version 1417467 (0.0005) [2023-12-27 01:42:56,753][105692] Updated weights for policy 0, policy_version 1417477 (0.0005) [2023-12-27 01:42:56,799][105692] Updated weights for policy 0, policy_version 1417487 (0.0005) [2023-12-27 01:42:56,975][105620] Updated weights for policy 1, policy_version 1419658 (0.0007) [2023-12-27 01:42:57,039][105620] Updated weights for policy 1, policy_version 1419668 (0.0007) [2023-12-27 01:42:57,097][105620] Updated weights for policy 1, policy_version 1419678 (0.0010) [2023-12-27 01:42:57,166][105620] Updated weights for policy 1, policy_version 1419688 (0.0006) [2023-12-27 01:42:57,418][105692] Updated weights for policy 0, policy_version 1417497 (0.0010) [2023-12-27 01:42:57,462][105692] Updated weights for policy 0, policy_version 1417507 (0.0010) [2023-12-27 01:42:57,509][105692] Updated weights for policy 0, policy_version 1417517 (0.0010) [2023-12-27 01:42:57,720][105620] Updated weights for policy 1, policy_version 1419698 (0.0005) [2023-12-27 01:42:57,770][105620] Updated weights for policy 1, policy_version 1419708 (0.0005) [2023-12-27 01:42:57,820][105620] Updated weights for policy 1, policy_version 1419718 (0.0005) [2023-12-27 01:42:58,279][105692] Updated weights for policy 0, policy_version 1417527 (0.0011) [2023-12-27 01:42:58,345][105692] Updated weights for policy 0, policy_version 1417537 (0.0010) [2023-12-27 01:42:58,360][105585] KL-divergence is very high: 306.6651 [2023-12-27 01:42:58,404][105585] KL-divergence is very high: 505.4958 [2023-12-27 01:42:58,404][105692] Updated weights for policy 0, policy_version 1417547 (0.0009) [2023-12-27 01:42:58,470][105620] Updated weights for policy 1, policy_version 1419728 (0.0008) [2023-12-27 01:42:58,532][105620] Updated weights for policy 1, policy_version 1419738 (0.0009) [2023-12-27 01:42:58,589][105620] Updated weights for policy 1, policy_version 1419748 (0.0009) [2023-12-27 01:42:59,226][105692] Updated weights for policy 0, policy_version 1417557 (0.0009) [2023-12-27 01:42:59,295][105692] Updated weights for policy 0, policy_version 1417568 (0.0008) [2023-12-27 01:42:59,356][105692] Updated weights for policy 0, policy_version 1417578 (0.0008) [2023-12-27 01:42:59,368][105620] Updated weights for policy 1, policy_version 1419758 (0.0008) [2023-12-27 01:42:59,437][105620] Updated weights for policy 1, policy_version 1419768 (0.0010) [2023-12-27 01:42:59,498][105620] Updated weights for policy 1, policy_version 1419778 (0.0009) [2023-12-27 01:43:00,055][105692] Updated weights for policy 0, policy_version 1417588 (0.0008) [2023-12-27 01:43:00,114][105692] Updated weights for policy 0, policy_version 1417598 (0.0008) [2023-12-27 01:43:00,172][105692] Updated weights for policy 0, policy_version 1417608 (0.0009) [2023-12-27 01:43:00,259][105620] Updated weights for policy 1, policy_version 1419788 (0.0010) [2023-12-27 01:43:00,315][105620] Updated weights for policy 1, policy_version 1419798 (0.0009) [2023-12-27 01:43:00,369][105620] Updated weights for policy 1, policy_version 1419808 (0.0009) [2023-12-27 01:43:00,926][105692] Updated weights for policy 0, policy_version 1417618 (0.0008) [2023-12-27 01:43:00,977][105692] Updated weights for policy 0, policy_version 1417628 (0.0005) [2023-12-27 01:43:01,032][105692] Updated weights for policy 0, policy_version 1417638 (0.0006) [2023-12-27 01:43:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 726482944. Throughput: 0: 9793.6, 1: 9981.6. Samples: 726460208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:43:01,063][104569] Avg episode reward: [(0, '6420.614'), (1, '8994.274')] [2023-12-27 01:43:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001419816_363520000.pth... [2023-12-27 01:43:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001418664_363225088.pth [2023-12-27 01:43:01,093][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001417648_362971136.pth... [2023-12-27 01:43:01,095][105692] Updated weights for policy 0, policy_version 1417648 (0.0007) [2023-12-27 01:43:01,097][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001416528_362684416.pth [2023-12-27 01:43:01,120][105620] Updated weights for policy 1, policy_version 1419818 (0.0010) [2023-12-27 01:43:01,186][105620] Updated weights for policy 1, policy_version 1419828 (0.0010) [2023-12-27 01:43:01,239][105620] Updated weights for policy 1, policy_version 1419838 (0.0009) [2023-12-27 01:43:01,297][105620] Updated weights for policy 1, policy_version 1419848 (0.0009) [2023-12-27 01:43:01,705][105692] Updated weights for policy 0, policy_version 1417658 (0.0008) [2023-12-27 01:43:01,774][105692] Updated weights for policy 0, policy_version 1417668 (0.0008) [2023-12-27 01:43:01,838][105692] Updated weights for policy 0, policy_version 1417678 (0.0007) [2023-12-27 01:43:02,147][105620] Updated weights for policy 1, policy_version 1419858 (0.0009) [2023-12-27 01:43:02,193][105620] Updated weights for policy 1, policy_version 1419868 (0.0009) [2023-12-27 01:43:02,248][105620] Updated weights for policy 1, policy_version 1419878 (0.0008) [2023-12-27 01:43:02,559][105692] Updated weights for policy 0, policy_version 1417688 (0.0009) [2023-12-27 01:43:02,612][105692] Updated weights for policy 0, policy_version 1417698 (0.0007) [2023-12-27 01:43:02,674][105692] Updated weights for policy 0, policy_version 1417708 (0.0010) [2023-12-27 01:43:02,925][105620] Updated weights for policy 1, policy_version 1419888 (0.0009) [2023-12-27 01:43:02,974][105620] Updated weights for policy 1, policy_version 1419898 (0.0008) [2023-12-27 01:43:03,020][105620] Updated weights for policy 1, policy_version 1419908 (0.0008) [2023-12-27 01:43:03,401][105692] Updated weights for policy 0, policy_version 1417718 (0.0010) [2023-12-27 01:43:03,458][105692] Updated weights for policy 0, policy_version 1417728 (0.0010) [2023-12-27 01:43:03,512][105692] Updated weights for policy 0, policy_version 1417738 (0.0010) [2023-12-27 01:43:03,800][105620] Updated weights for policy 1, policy_version 1419918 (0.0009) [2023-12-27 01:43:03,861][105620] Updated weights for policy 1, policy_version 1419928 (0.0008) [2023-12-27 01:43:03,916][105620] Updated weights for policy 1, policy_version 1419938 (0.0008) [2023-12-27 01:43:04,239][105692] Updated weights for policy 0, policy_version 1417748 (0.0010) [2023-12-27 01:43:04,301][105692] Updated weights for policy 0, policy_version 1417758 (0.0009) [2023-12-27 01:43:04,355][105692] Updated weights for policy 0, policy_version 1417768 (0.0007) [2023-12-27 01:43:04,698][105620] Updated weights for policy 1, policy_version 1419948 (0.0009) [2023-12-27 01:43:04,757][105620] Updated weights for policy 1, policy_version 1419958 (0.0009) [2023-12-27 01:43:04,806][105620] Updated weights for policy 1, policy_version 1419968 (0.0008) [2023-12-27 01:43:05,035][105692] Updated weights for policy 0, policy_version 1417778 (0.0008) [2023-12-27 01:43:05,105][105692] Updated weights for policy 0, policy_version 1417788 (0.0009) [2023-12-27 01:43:05,171][105692] Updated weights for policy 0, policy_version 1417798 (0.0010) [2023-12-27 01:43:05,234][105692] Updated weights for policy 0, policy_version 1417808 (0.0009) [2023-12-27 01:43:05,450][105620] Updated weights for policy 1, policy_version 1419978 (0.0008) [2023-12-27 01:43:05,501][105620] Updated weights for policy 1, policy_version 1419988 (0.0009) [2023-12-27 01:43:05,559][105620] Updated weights for policy 1, policy_version 1419998 (0.0009) [2023-12-27 01:43:05,612][105620] Updated weights for policy 1, policy_version 1420008 (0.0009) [2023-12-27 01:43:05,957][105692] Updated weights for policy 0, policy_version 1417818 (0.0010) [2023-12-27 01:43:06,028][105692] Updated weights for policy 0, policy_version 1417828 (0.0005) [2023-12-27 01:43:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.4, 300 sec: 19577.5). Total num frames: 726581248. Throughput: 0: 9725.3, 1: 9886.3. Samples: 726573924. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:43:06,062][104569] Avg episode reward: [(0, '6955.543'), (1, '8991.290')] [2023-12-27 01:43:06,084][105692] Updated weights for policy 0, policy_version 1417838 (0.0005) [2023-12-27 01:43:06,423][105620] Updated weights for policy 1, policy_version 1420018 (0.0011) [2023-12-27 01:43:06,476][105620] Updated weights for policy 1, policy_version 1420028 (0.0009) [2023-12-27 01:43:06,534][105620] Updated weights for policy 1, policy_version 1420038 (0.0007) [2023-12-27 01:43:06,716][105692] Updated weights for policy 0, policy_version 1417848 (0.0008) [2023-12-27 01:43:06,769][105692] Updated weights for policy 0, policy_version 1417858 (0.0008) [2023-12-27 01:43:06,823][105692] Updated weights for policy 0, policy_version 1417868 (0.0010) [2023-12-27 01:43:07,188][105620] Updated weights for policy 1, policy_version 1420048 (0.0006) [2023-12-27 01:43:07,249][105620] Updated weights for policy 1, policy_version 1420058 (0.0006) [2023-12-27 01:43:07,312][105620] Updated weights for policy 1, policy_version 1420068 (0.0009) [2023-12-27 01:43:07,677][105692] Updated weights for policy 0, policy_version 1417878 (0.0009) [2023-12-27 01:43:07,739][105692] Updated weights for policy 0, policy_version 1417888 (0.0009) [2023-12-27 01:43:07,804][105692] Updated weights for policy 0, policy_version 1417898 (0.0008) [2023-12-27 01:43:07,984][105620] Updated weights for policy 1, policy_version 1420078 (0.0007) [2023-12-27 01:43:08,037][105620] Updated weights for policy 1, policy_version 1420088 (0.0005) [2023-12-27 01:43:08,089][105620] Updated weights for policy 1, policy_version 1420098 (0.0005) [2023-12-27 01:43:08,479][105692] Updated weights for policy 0, policy_version 1417908 (0.0009) [2023-12-27 01:43:08,527][105692] Updated weights for policy 0, policy_version 1417918 (0.0010) [2023-12-27 01:43:08,587][105692] Updated weights for policy 0, policy_version 1417928 (0.0010) [2023-12-27 01:43:08,643][105620] Updated weights for policy 1, policy_version 1420108 (0.0006) [2023-12-27 01:43:08,704][105620] Updated weights for policy 1, policy_version 1420118 (0.0009) [2023-12-27 01:43:08,760][105620] Updated weights for policy 1, policy_version 1420128 (0.0008) [2023-12-27 01:43:09,206][105692] Updated weights for policy 0, policy_version 1417938 (0.0009) [2023-12-27 01:43:09,265][105692] Updated weights for policy 0, policy_version 1417948 (0.0008) [2023-12-27 01:43:09,319][105692] Updated weights for policy 0, policy_version 1417958 (0.0006) [2023-12-27 01:43:09,386][105692] Updated weights for policy 0, policy_version 1417968 (0.0009) [2023-12-27 01:43:09,451][105620] Updated weights for policy 1, policy_version 1420138 (0.0009) [2023-12-27 01:43:09,516][105620] Updated weights for policy 1, policy_version 1420148 (0.0009) [2023-12-27 01:43:09,572][105620] Updated weights for policy 1, policy_version 1420158 (0.0009) [2023-12-27 01:43:09,628][105620] Updated weights for policy 1, policy_version 1420168 (0.0009) [2023-12-27 01:43:10,164][105692] Updated weights for policy 0, policy_version 1417978 (0.0009) [2023-12-27 01:43:10,213][105692] Updated weights for policy 0, policy_version 1417988 (0.0009) [2023-12-27 01:43:10,261][105692] Updated weights for policy 0, policy_version 1417998 (0.0009) [2023-12-27 01:43:10,373][105620] Updated weights for policy 1, policy_version 1420178 (0.0008) [2023-12-27 01:43:10,426][105620] Updated weights for policy 1, policy_version 1420188 (0.0008) [2023-12-27 01:43:10,474][105620] Updated weights for policy 1, policy_version 1420198 (0.0009) [2023-12-27 01:43:11,042][105692] Updated weights for policy 0, policy_version 1418008 (0.0009) [2023-12-27 01:43:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 726679552. Throughput: 0: 9745.2, 1: 9968.2. Samples: 726692700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:43:11,062][104569] Avg episode reward: [(0, '7420.174'), (1, '8624.742')] [2023-12-27 01:43:11,108][105692] Updated weights for policy 0, policy_version 1418018 (0.0008) [2023-12-27 01:43:11,177][105692] Updated weights for policy 0, policy_version 1418028 (0.0009) [2023-12-27 01:43:11,271][105620] Updated weights for policy 1, policy_version 1420208 (0.0009) [2023-12-27 01:43:11,329][105620] Updated weights for policy 1, policy_version 1420218 (0.0008) [2023-12-27 01:43:11,403][105620] Updated weights for policy 1, policy_version 1420228 (0.0009) [2023-12-27 01:43:11,897][105692] Updated weights for policy 0, policy_version 1418038 (0.0007) [2023-12-27 01:43:11,952][105692] Updated weights for policy 0, policy_version 1418048 (0.0006) [2023-12-27 01:43:12,005][105692] Updated weights for policy 0, policy_version 1418058 (0.0006) [2023-12-27 01:43:12,221][105620] Updated weights for policy 1, policy_version 1420238 (0.0009) [2023-12-27 01:43:12,289][105620] Updated weights for policy 1, policy_version 1420248 (0.0008) [2023-12-27 01:43:12,351][105620] Updated weights for policy 1, policy_version 1420258 (0.0008) [2023-12-27 01:43:12,712][105692] Updated weights for policy 0, policy_version 1418068 (0.0006) [2023-12-27 01:43:12,771][105692] Updated weights for policy 0, policy_version 1418078 (0.0005) [2023-12-27 01:43:12,831][105692] Updated weights for policy 0, policy_version 1418088 (0.0005) [2023-12-27 01:43:13,159][105620] Updated weights for policy 1, policy_version 1420268 (0.0008) [2023-12-27 01:43:13,210][105620] Updated weights for policy 1, policy_version 1420278 (0.0008) [2023-12-27 01:43:13,263][105620] Updated weights for policy 1, policy_version 1420289 (0.0010) [2023-12-27 01:43:13,385][105692] Updated weights for policy 0, policy_version 1418098 (0.0006) [2023-12-27 01:43:13,431][105692] Updated weights for policy 0, policy_version 1418108 (0.0008) [2023-12-27 01:43:13,482][105692] Updated weights for policy 0, policy_version 1418118 (0.0006) [2023-12-27 01:43:13,520][105585] KL-divergence is very high: 108.1288 [2023-12-27 01:43:13,537][105692] Updated weights for policy 0, policy_version 1418128 (0.0005) [2023-12-27 01:43:14,100][105620] Updated weights for policy 1, policy_version 1420300 (0.0010) [2023-12-27 01:43:14,152][105620] Updated weights for policy 1, policy_version 1420310 (0.0010) [2023-12-27 01:43:14,198][105692] Updated weights for policy 0, policy_version 1418138 (0.0006) [2023-12-27 01:43:14,211][105620] Updated weights for policy 1, policy_version 1420320 (0.0010) [2023-12-27 01:43:14,248][105692] Updated weights for policy 0, policy_version 1418148 (0.0008) [2023-12-27 01:43:14,295][105692] Updated weights for policy 0, policy_version 1418158 (0.0007) [2023-12-27 01:43:14,978][105620] Updated weights for policy 1, policy_version 1420330 (0.0010) [2023-12-27 01:43:15,040][105620] Updated weights for policy 1, policy_version 1420340 (0.0009) [2023-12-27 01:43:15,080][105692] Updated weights for policy 0, policy_version 1418168 (0.0008) [2023-12-27 01:43:15,095][105585] KL-divergence is very high: 129.6441 [2023-12-27 01:43:15,098][105620] Updated weights for policy 1, policy_version 1420350 (0.0007) [2023-12-27 01:43:15,133][105692] Updated weights for policy 0, policy_version 1418178 (0.0006) [2023-12-27 01:43:15,137][105585] KL-divergence is very high: 251.6388 [2023-12-27 01:43:15,159][105620] Updated weights for policy 1, policy_version 1420360 (0.0009) [2023-12-27 01:43:15,187][105585] KL-divergence is very high: 290.5686 [2023-12-27 01:43:15,193][105692] Updated weights for policy 0, policy_version 1418188 (0.0008) [2023-12-27 01:43:15,867][105620] Updated weights for policy 1, policy_version 1420370 (0.0008) [2023-12-27 01:43:15,921][105620] Updated weights for policy 1, policy_version 1420380 (0.0009) [2023-12-27 01:43:15,975][105620] Updated weights for policy 1, policy_version 1420390 (0.0007) [2023-12-27 01:43:15,984][105692] Updated weights for policy 0, policy_version 1418198 (0.0008) [2023-12-27 01:43:16,041][105692] Updated weights for policy 0, policy_version 1418208 (0.0009) [2023-12-27 01:43:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 726777856. Throughput: 0: 9632.2, 1: 9872.6. Samples: 726749700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:43:16,062][104569] Avg episode reward: [(0, '6870.564'), (1, '8344.882')] [2023-12-27 01:43:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001420392_363667456.pth... [2023-12-27 01:43:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001419240_363372544.pth [2023-12-27 01:43:16,107][105692] Updated weights for policy 0, policy_version 1418218 (0.0010) [2023-12-27 01:43:16,147][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001418224_363118592.pth... [2023-12-27 01:43:16,151][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001417072_362823680.pth [2023-12-27 01:43:16,550][105620] Updated weights for policy 1, policy_version 1420400 (0.0006) [2023-12-27 01:43:16,600][105620] Updated weights for policy 1, policy_version 1420410 (0.0009) [2023-12-27 01:43:16,663][105620] Updated weights for policy 1, policy_version 1420420 (0.0005) [2023-12-27 01:43:16,872][105692] Updated weights for policy 0, policy_version 1418228 (0.0008) [2023-12-27 01:43:16,944][105692] Updated weights for policy 0, policy_version 1418238 (0.0008) [2023-12-27 01:43:17,007][105692] Updated weights for policy 0, policy_version 1418248 (0.0011) [2023-12-27 01:43:17,240][105620] Updated weights for policy 1, policy_version 1420430 (0.0007) [2023-12-27 01:43:17,299][105620] Updated weights for policy 1, policy_version 1420440 (0.0007) [2023-12-27 01:43:17,347][105620] Updated weights for policy 1, policy_version 1420450 (0.0008) [2023-12-27 01:43:17,747][105692] Updated weights for policy 0, policy_version 1418258 (0.0011) [2023-12-27 01:43:17,800][105692] Updated weights for policy 0, policy_version 1418268 (0.0010) [2023-12-27 01:43:17,854][105692] Updated weights for policy 0, policy_version 1418279 (0.0010) [2023-12-27 01:43:17,976][105620] Updated weights for policy 1, policy_version 1420460 (0.0007) [2023-12-27 01:43:18,029][105620] Updated weights for policy 1, policy_version 1420470 (0.0005) [2023-12-27 01:43:18,086][105620] Updated weights for policy 1, policy_version 1420480 (0.0005) [2023-12-27 01:43:18,674][105692] Updated weights for policy 0, policy_version 1418289 (0.0010) [2023-12-27 01:43:18,737][105692] Updated weights for policy 0, policy_version 1418299 (0.0009) [2023-12-27 01:43:18,797][105692] Updated weights for policy 0, policy_version 1418309 (0.0008) [2023-12-27 01:43:18,825][105620] Updated weights for policy 1, policy_version 1420490 (0.0009) [2023-12-27 01:43:18,855][105692] Updated weights for policy 0, policy_version 1418319 (0.0007) [2023-12-27 01:43:18,880][105620] Updated weights for policy 1, policy_version 1420500 (0.0010) [2023-12-27 01:43:18,933][105620] Updated weights for policy 1, policy_version 1420510 (0.0010) [2023-12-27 01:43:18,985][105620] Updated weights for policy 1, policy_version 1420520 (0.0010) [2023-12-27 01:43:19,513][105692] Updated weights for policy 0, policy_version 1418329 (0.0007) [2023-12-27 01:43:19,576][105692] Updated weights for policy 0, policy_version 1418339 (0.0006) [2023-12-27 01:43:19,640][105692] Updated weights for policy 0, policy_version 1418349 (0.0005) [2023-12-27 01:43:19,766][105620] Updated weights for policy 1, policy_version 1420530 (0.0011) [2023-12-27 01:43:19,827][105620] Updated weights for policy 1, policy_version 1420540 (0.0011) [2023-12-27 01:43:19,884][105620] Updated weights for policy 1, policy_version 1420550 (0.0010) [2023-12-27 01:43:20,295][105692] Updated weights for policy 0, policy_version 1418359 (0.0009) [2023-12-27 01:43:20,347][105692] Updated weights for policy 0, policy_version 1418369 (0.0010) [2023-12-27 01:43:20,406][105692] Updated weights for policy 0, policy_version 1418379 (0.0011) [2023-12-27 01:43:20,660][105620] Updated weights for policy 1, policy_version 1420560 (0.0008) [2023-12-27 01:43:20,714][105620] Updated weights for policy 1, policy_version 1420570 (0.0008) [2023-12-27 01:43:20,778][105620] Updated weights for policy 1, policy_version 1420580 (0.0009) [2023-12-27 01:43:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 726876160. Throughput: 0: 9698.8, 1: 9832.1. Samples: 726866856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:43:21,063][104569] Avg episode reward: [(0, '6962.631'), (1, '8347.685')] [2023-12-27 01:43:21,165][105692] Updated weights for policy 0, policy_version 1418389 (0.0009) [2023-12-27 01:43:21,230][105692] Updated weights for policy 0, policy_version 1418399 (0.0008) [2023-12-27 01:43:21,300][105692] Updated weights for policy 0, policy_version 1418409 (0.0007) [2023-12-27 01:43:21,518][105620] Updated weights for policy 1, policy_version 1420590 (0.0011) [2023-12-27 01:43:21,577][105620] Updated weights for policy 1, policy_version 1420600 (0.0010) [2023-12-27 01:43:21,639][105620] Updated weights for policy 1, policy_version 1420610 (0.0009) [2023-12-27 01:43:21,970][105692] Updated weights for policy 0, policy_version 1418419 (0.0008) [2023-12-27 01:43:22,043][105692] Updated weights for policy 0, policy_version 1418429 (0.0009) [2023-12-27 01:43:22,095][105692] Updated weights for policy 0, policy_version 1418439 (0.0006) [2023-12-27 01:43:22,379][105620] Updated weights for policy 1, policy_version 1420620 (0.0008) [2023-12-27 01:43:22,445][105620] Updated weights for policy 1, policy_version 1420630 (0.0009) [2023-12-27 01:43:22,515][105620] Updated weights for policy 1, policy_version 1420640 (0.0011) [2023-12-27 01:43:22,749][105692] Updated weights for policy 0, policy_version 1418449 (0.0007) [2023-12-27 01:43:22,809][105692] Updated weights for policy 0, policy_version 1418459 (0.0005) [2023-12-27 01:43:22,878][105692] Updated weights for policy 0, policy_version 1418469 (0.0006) [2023-12-27 01:43:22,944][105692] Updated weights for policy 0, policy_version 1418479 (0.0008) [2023-12-27 01:43:23,285][105620] Updated weights for policy 1, policy_version 1420650 (0.0011) [2023-12-27 01:43:23,348][105620] Updated weights for policy 1, policy_version 1420660 (0.0009) [2023-12-27 01:43:23,413][105620] Updated weights for policy 1, policy_version 1420670 (0.0008) [2023-12-27 01:43:23,480][105620] Updated weights for policy 1, policy_version 1420680 (0.0011) [2023-12-27 01:43:23,616][105692] Updated weights for policy 0, policy_version 1418489 (0.0010) [2023-12-27 01:43:23,669][105692] Updated weights for policy 0, policy_version 1418499 (0.0010) [2023-12-27 01:43:23,716][105692] Updated weights for policy 0, policy_version 1418509 (0.0010) [2023-12-27 01:43:24,142][105620] Updated weights for policy 1, policy_version 1420690 (0.0009) [2023-12-27 01:43:24,205][105620] Updated weights for policy 1, policy_version 1420700 (0.0005) [2023-12-27 01:43:24,257][105620] Updated weights for policy 1, policy_version 1420710 (0.0008) [2023-12-27 01:43:24,353][105692] Updated weights for policy 0, policy_version 1418519 (0.0008) [2023-12-27 01:43:24,413][105692] Updated weights for policy 0, policy_version 1418529 (0.0009) [2023-12-27 01:43:24,467][105692] Updated weights for policy 0, policy_version 1418539 (0.0007) [2023-12-27 01:43:25,004][105620] Updated weights for policy 1, policy_version 1420720 (0.0010) [2023-12-27 01:43:25,074][105620] Updated weights for policy 1, policy_version 1420730 (0.0011) [2023-12-27 01:43:25,127][105620] Updated weights for policy 1, policy_version 1420740 (0.0007) [2023-12-27 01:43:25,128][105692] Updated weights for policy 0, policy_version 1418549 (0.0009) [2023-12-27 01:43:25,182][105692] Updated weights for policy 0, policy_version 1418559 (0.0010) [2023-12-27 01:43:25,240][105692] Updated weights for policy 0, policy_version 1418569 (0.0010) [2023-12-27 01:43:25,771][105620] Updated weights for policy 1, policy_version 1420750 (0.0005) [2023-12-27 01:43:25,831][105620] Updated weights for policy 1, policy_version 1420760 (0.0005) [2023-12-27 01:43:25,890][105620] Updated weights for policy 1, policy_version 1420770 (0.0005) [2023-12-27 01:43:25,968][105692] Updated weights for policy 0, policy_version 1418579 (0.0010) [2023-12-27 01:43:26,036][105692] Updated weights for policy 0, policy_version 1418589 (0.0011) [2023-12-27 01:43:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 726974464. Throughput: 0: 9757.2, 1: 9819.6. Samples: 726985408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:43:26,063][104569] Avg episode reward: [(0, '7612.767'), (1, '8532.439')] [2023-12-27 01:43:26,104][105692] Updated weights for policy 0, policy_version 1418599 (0.0011) [2023-12-27 01:43:26,413][105620] Updated weights for policy 1, policy_version 1420780 (0.0006) [2023-12-27 01:43:26,466][105620] Updated weights for policy 1, policy_version 1420790 (0.0006) [2023-12-27 01:43:26,515][105620] Updated weights for policy 1, policy_version 1420800 (0.0010) [2023-12-27 01:43:26,834][105692] Updated weights for policy 0, policy_version 1418609 (0.0010) [2023-12-27 01:43:26,895][105692] Updated weights for policy 0, policy_version 1418619 (0.0010) [2023-12-27 01:43:26,953][105692] Updated weights for policy 0, policy_version 1418629 (0.0010) [2023-12-27 01:43:27,004][105692] Updated weights for policy 0, policy_version 1418639 (0.0010) [2023-12-27 01:43:27,237][105620] Updated weights for policy 1, policy_version 1420810 (0.0010) [2023-12-27 01:43:27,296][105620] Updated weights for policy 1, policy_version 1420820 (0.0007) [2023-12-27 01:43:27,356][105620] Updated weights for policy 1, policy_version 1420830 (0.0007) [2023-12-27 01:43:27,406][105620] Updated weights for policy 1, policy_version 1420840 (0.0006) [2023-12-27 01:43:27,651][105692] Updated weights for policy 0, policy_version 1418649 (0.0010) [2023-12-27 01:43:27,709][105692] Updated weights for policy 0, policy_version 1418659 (0.0009) [2023-12-27 01:43:27,776][105692] Updated weights for policy 0, policy_version 1418669 (0.0007) [2023-12-27 01:43:28,103][105620] Updated weights for policy 1, policy_version 1420850 (0.0005) [2023-12-27 01:43:28,150][105620] Updated weights for policy 1, policy_version 1420860 (0.0005) [2023-12-27 01:43:28,194][105620] Updated weights for policy 1, policy_version 1420870 (0.0005) [2023-12-27 01:43:28,328][105692] Updated weights for policy 0, policy_version 1418679 (0.0006) [2023-12-27 01:43:28,388][105692] Updated weights for policy 0, policy_version 1418689 (0.0008) [2023-12-27 01:43:28,437][105692] Updated weights for policy 0, policy_version 1418699 (0.0008) [2023-12-27 01:43:28,762][105620] Updated weights for policy 1, policy_version 1420880 (0.0007) [2023-12-27 01:43:28,822][105620] Updated weights for policy 1, policy_version 1420890 (0.0008) [2023-12-27 01:43:28,881][105620] Updated weights for policy 1, policy_version 1420900 (0.0008) [2023-12-27 01:43:29,143][105692] Updated weights for policy 0, policy_version 1418709 (0.0007) [2023-12-27 01:43:29,211][105692] Updated weights for policy 0, policy_version 1418719 (0.0008) [2023-12-27 01:43:29,270][105692] Updated weights for policy 0, policy_version 1418729 (0.0010) [2023-12-27 01:43:29,561][105620] Updated weights for policy 1, policy_version 1420910 (0.0009) [2023-12-27 01:43:29,617][105620] Updated weights for policy 1, policy_version 1420920 (0.0009) [2023-12-27 01:43:29,679][105620] Updated weights for policy 1, policy_version 1420930 (0.0009) [2023-12-27 01:43:29,974][105692] Updated weights for policy 0, policy_version 1418739 (0.0007) [2023-12-27 01:43:30,022][105692] Updated weights for policy 0, policy_version 1418749 (0.0009) [2023-12-27 01:43:30,069][105692] Updated weights for policy 0, policy_version 1418759 (0.0008) [2023-12-27 01:43:30,440][105620] Updated weights for policy 1, policy_version 1420940 (0.0008) [2023-12-27 01:43:30,492][105620] Updated weights for policy 1, policy_version 1420950 (0.0009) [2023-12-27 01:43:30,540][105620] Updated weights for policy 1, policy_version 1420960 (0.0009) [2023-12-27 01:43:30,841][105692] Updated weights for policy 0, policy_version 1418769 (0.0009) [2023-12-27 01:43:30,898][105692] Updated weights for policy 0, policy_version 1418779 (0.0008) [2023-12-27 01:43:30,946][105692] Updated weights for policy 0, policy_version 1418789 (0.0006) [2023-12-27 01:43:30,997][105692] Updated weights for policy 0, policy_version 1418799 (0.0005) [2023-12-27 01:43:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 727080960. Throughput: 0: 9827.2, 1: 9916.8. Samples: 727048460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:43:31,063][104569] Avg episode reward: [(0, '8438.081'), (1, '8714.176')] [2023-12-27 01:43:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001418800_363266048.pth... [2023-12-27 01:43:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001420968_363814912.pth... [2023-12-27 01:43:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001419816_363520000.pth [2023-12-27 01:43:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001417648_362971136.pth [2023-12-27 01:43:31,343][105620] Updated weights for policy 1, policy_version 1420970 (0.0009) [2023-12-27 01:43:31,413][105620] Updated weights for policy 1, policy_version 1420980 (0.0008) [2023-12-27 01:43:31,477][105620] Updated weights for policy 1, policy_version 1420990 (0.0008) [2023-12-27 01:43:31,541][105620] Updated weights for policy 1, policy_version 1421000 (0.0009) [2023-12-27 01:43:31,709][105692] Updated weights for policy 0, policy_version 1418809 (0.0008) [2023-12-27 01:43:31,766][105692] Updated weights for policy 0, policy_version 1418819 (0.0011) [2023-12-27 01:43:31,815][105692] Updated weights for policy 0, policy_version 1418829 (0.0010) [2023-12-27 01:43:32,180][105620] Updated weights for policy 1, policy_version 1421010 (0.0006) [2023-12-27 01:43:32,235][105620] Updated weights for policy 1, policy_version 1421020 (0.0006) [2023-12-27 01:43:32,295][105620] Updated weights for policy 1, policy_version 1421030 (0.0008) [2023-12-27 01:43:32,546][105692] Updated weights for policy 0, policy_version 1418839 (0.0010) [2023-12-27 01:43:32,591][105692] Updated weights for policy 0, policy_version 1418849 (0.0010) [2023-12-27 01:43:32,640][105692] Updated weights for policy 0, policy_version 1418859 (0.0010) [2023-12-27 01:43:32,915][105620] Updated weights for policy 1, policy_version 1421040 (0.0006) [2023-12-27 01:43:32,966][105620] Updated weights for policy 1, policy_version 1421050 (0.0006) [2023-12-27 01:43:33,029][105620] Updated weights for policy 1, policy_version 1421060 (0.0006) [2023-12-27 01:43:33,284][105692] Updated weights for policy 0, policy_version 1418869 (0.0008) [2023-12-27 01:43:33,327][105692] Updated weights for policy 0, policy_version 1418879 (0.0010) [2023-12-27 01:43:33,378][105692] Updated weights for policy 0, policy_version 1418889 (0.0010) [2023-12-27 01:43:33,661][105620] Updated weights for policy 1, policy_version 1421070 (0.0005) [2023-12-27 01:43:33,713][105620] Updated weights for policy 1, policy_version 1421080 (0.0005) [2023-12-27 01:43:33,777][105620] Updated weights for policy 1, policy_version 1421090 (0.0007) [2023-12-27 01:43:34,052][105692] Updated weights for policy 0, policy_version 1418899 (0.0010) [2023-12-27 01:43:34,112][105692] Updated weights for policy 0, policy_version 1418910 (0.0010) [2023-12-27 01:43:34,174][105692] Updated weights for policy 0, policy_version 1418920 (0.0009) [2023-12-27 01:43:34,437][105620] Updated weights for policy 1, policy_version 1421100 (0.0007) [2023-12-27 01:43:34,486][105620] Updated weights for policy 1, policy_version 1421110 (0.0008) [2023-12-27 01:43:34,537][105620] Updated weights for policy 1, policy_version 1421120 (0.0008) [2023-12-27 01:43:34,931][105692] Updated weights for policy 0, policy_version 1418930 (0.0007) [2023-12-27 01:43:34,979][105692] Updated weights for policy 0, policy_version 1418940 (0.0005) [2023-12-27 01:43:35,032][105692] Updated weights for policy 0, policy_version 1418950 (0.0005) [2023-12-27 01:43:35,088][105692] Updated weights for policy 0, policy_version 1418960 (0.0005) [2023-12-27 01:43:35,342][105620] Updated weights for policy 1, policy_version 1421130 (0.0008) [2023-12-27 01:43:35,398][105620] Updated weights for policy 1, policy_version 1421140 (0.0005) [2023-12-27 01:43:35,460][105620] Updated weights for policy 1, policy_version 1421150 (0.0005) [2023-12-27 01:43:35,529][105620] Updated weights for policy 1, policy_version 1421160 (0.0005) [2023-12-27 01:43:35,606][105692] Updated weights for policy 0, policy_version 1418970 (0.0010) [2023-12-27 01:43:35,654][105692] Updated weights for policy 0, policy_version 1418980 (0.0010) [2023-12-27 01:43:35,708][105692] Updated weights for policy 0, policy_version 1418990 (0.0010) [2023-12-27 01:43:36,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 727179264. Throughput: 0: 9874.0, 1: 9933.5. Samples: 727167832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:43:36,062][104569] Avg episode reward: [(0, '8072.133'), (1, '8717.638')] [2023-12-27 01:43:36,128][105620] Updated weights for policy 1, policy_version 1421170 (0.0007) [2023-12-27 01:43:36,194][105620] Updated weights for policy 1, policy_version 1421180 (0.0008) [2023-12-27 01:43:36,258][105620] Updated weights for policy 1, policy_version 1421190 (0.0008) [2023-12-27 01:43:36,484][105692] Updated weights for policy 0, policy_version 1419000 (0.0007) [2023-12-27 01:43:36,548][105692] Updated weights for policy 0, policy_version 1419010 (0.0007) [2023-12-27 01:43:36,606][105692] Updated weights for policy 0, policy_version 1419020 (0.0005) [2023-12-27 01:43:37,061][105620] Updated weights for policy 1, policy_version 1421200 (0.0009) [2023-12-27 01:43:37,117][105620] Updated weights for policy 1, policy_version 1421210 (0.0008) [2023-12-27 01:43:37,153][105692] Updated weights for policy 0, policy_version 1419030 (0.0008) [2023-12-27 01:43:37,165][105620] Updated weights for policy 1, policy_version 1421220 (0.0010) [2023-12-27 01:43:37,209][105692] Updated weights for policy 0, policy_version 1419040 (0.0010) [2023-12-27 01:43:37,270][105692] Updated weights for policy 0, policy_version 1419050 (0.0006) [2023-12-27 01:43:37,959][105620] Updated weights for policy 1, policy_version 1421230 (0.0008) [2023-12-27 01:43:38,003][105692] Updated weights for policy 0, policy_version 1419060 (0.0008) [2023-12-27 01:43:38,013][105620] Updated weights for policy 1, policy_version 1421240 (0.0008) [2023-12-27 01:43:38,065][105692] Updated weights for policy 0, policy_version 1419070 (0.0010) [2023-12-27 01:43:38,071][105620] Updated weights for policy 1, policy_version 1421250 (0.0005) [2023-12-27 01:43:38,120][105692] Updated weights for policy 0, policy_version 1419080 (0.0010) [2023-12-27 01:43:38,841][105620] Updated weights for policy 1, policy_version 1421260 (0.0007) [2023-12-27 01:43:38,845][105692] Updated weights for policy 0, policy_version 1419090 (0.0010) [2023-12-27 01:43:38,895][105620] Updated weights for policy 1, policy_version 1421270 (0.0008) [2023-12-27 01:43:38,901][105692] Updated weights for policy 0, policy_version 1419100 (0.0010) [2023-12-27 01:43:38,944][105620] Updated weights for policy 1, policy_version 1421280 (0.0005) [2023-12-27 01:43:38,950][105692] Updated weights for policy 0, policy_version 1419110 (0.0010) [2023-12-27 01:43:39,005][105692] Updated weights for policy 0, policy_version 1419120 (0.0011) [2023-12-27 01:43:39,717][105692] Updated weights for policy 0, policy_version 1419130 (0.0009) [2023-12-27 01:43:39,766][105620] Updated weights for policy 1, policy_version 1421290 (0.0006) [2023-12-27 01:43:39,770][105692] Updated weights for policy 0, policy_version 1419140 (0.0011) [2023-12-27 01:43:39,826][105620] Updated weights for policy 1, policy_version 1421300 (0.0007) [2023-12-27 01:43:39,828][105692] Updated weights for policy 0, policy_version 1419150 (0.0011) [2023-12-27 01:43:39,884][105620] Updated weights for policy 1, policy_version 1421310 (0.0009) [2023-12-27 01:43:39,938][105620] Updated weights for policy 1, policy_version 1421320 (0.0008) [2023-12-27 01:43:40,602][105692] Updated weights for policy 0, policy_version 1419160 (0.0011) [2023-12-27 01:43:40,628][105620] Updated weights for policy 1, policy_version 1421330 (0.0006) [2023-12-27 01:43:40,661][105692] Updated weights for policy 0, policy_version 1419170 (0.0011) [2023-12-27 01:43:40,691][105620] Updated weights for policy 1, policy_version 1421340 (0.0006) [2023-12-27 01:43:40,721][105692] Updated weights for policy 0, policy_version 1419180 (0.0011) [2023-12-27 01:43:40,748][105620] Updated weights for policy 1, policy_version 1421350 (0.0005) [2023-12-27 01:43:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 727277568. Throughput: 0: 9931.6, 1: 9812.8. Samples: 727284860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:43:41,062][104569] Avg episode reward: [(0, '6690.321'), (1, '8715.872')] [2023-12-27 01:43:41,434][105620] Updated weights for policy 1, policy_version 1421360 (0.0007) [2023-12-27 01:43:41,497][105620] Updated weights for policy 1, policy_version 1421370 (0.0005) [2023-12-27 01:43:41,542][105692] Updated weights for policy 0, policy_version 1419190 (0.0009) [2023-12-27 01:43:41,558][105620] Updated weights for policy 1, policy_version 1421380 (0.0007) [2023-12-27 01:43:41,603][105692] Updated weights for policy 0, policy_version 1419200 (0.0008) [2023-12-27 01:43:41,628][105585] KL-divergence is very high: 108.9534 [2023-12-27 01:43:41,663][105692] Updated weights for policy 0, policy_version 1419210 (0.0009) [2023-12-27 01:43:41,675][105585] KL-divergence is very high: 133.4982 [2023-12-27 01:43:42,289][105620] Updated weights for policy 1, policy_version 1421390 (0.0009) [2023-12-27 01:43:42,354][105620] Updated weights for policy 1, policy_version 1421400 (0.0011) [2023-12-27 01:43:42,421][105620] Updated weights for policy 1, policy_version 1421410 (0.0009) [2023-12-27 01:43:42,434][105692] Updated weights for policy 0, policy_version 1419220 (0.0007) [2023-12-27 01:43:42,487][105692] Updated weights for policy 0, policy_version 1419230 (0.0009) [2023-12-27 01:43:42,541][105692] Updated weights for policy 0, policy_version 1419240 (0.0009) [2023-12-27 01:43:43,119][105620] Updated weights for policy 1, policy_version 1421420 (0.0007) [2023-12-27 01:43:43,165][105620] Updated weights for policy 1, policy_version 1421430 (0.0005) [2023-12-27 01:43:43,211][105620] Updated weights for policy 1, policy_version 1421440 (0.0005) [2023-12-27 01:43:43,275][105692] Updated weights for policy 0, policy_version 1419250 (0.0008) [2023-12-27 01:43:43,330][105692] Updated weights for policy 0, policy_version 1419260 (0.0005) [2023-12-27 01:43:43,389][105692] Updated weights for policy 0, policy_version 1419270 (0.0005) [2023-12-27 01:43:43,443][105692] Updated weights for policy 0, policy_version 1419280 (0.0005) [2023-12-27 01:43:43,966][105620] Updated weights for policy 1, policy_version 1421450 (0.0006) [2023-12-27 01:43:44,015][105692] Updated weights for policy 0, policy_version 1419290 (0.0006) [2023-12-27 01:43:44,034][105620] Updated weights for policy 1, policy_version 1421460 (0.0009) [2023-12-27 01:43:44,071][105692] Updated weights for policy 0, policy_version 1419300 (0.0006) [2023-12-27 01:43:44,095][105620] Updated weights for policy 1, policy_version 1421470 (0.0008) [2023-12-27 01:43:44,119][105692] Updated weights for policy 0, policy_version 1419310 (0.0007) [2023-12-27 01:43:44,149][105620] Updated weights for policy 1, policy_version 1421480 (0.0006) [2023-12-27 01:43:44,717][105620] Updated weights for policy 1, policy_version 1421490 (0.0008) [2023-12-27 01:43:44,788][105620] Updated weights for policy 1, policy_version 1421500 (0.0009) [2023-12-27 01:43:44,829][105692] Updated weights for policy 0, policy_version 1419320 (0.0010) [2023-12-27 01:43:44,853][105620] Updated weights for policy 1, policy_version 1421510 (0.0010) [2023-12-27 01:43:44,897][105692] Updated weights for policy 0, policy_version 1419330 (0.0011) [2023-12-27 01:43:44,962][105692] Updated weights for policy 0, policy_version 1419340 (0.0011) [2023-12-27 01:43:45,608][105620] Updated weights for policy 1, policy_version 1421520 (0.0008) [2023-12-27 01:43:45,655][105620] Updated weights for policy 1, policy_version 1421530 (0.0008) [2023-12-27 01:43:45,712][105620] Updated weights for policy 1, policy_version 1421540 (0.0008) [2023-12-27 01:43:45,712][105692] Updated weights for policy 0, policy_version 1419350 (0.0011) [2023-12-27 01:43:45,767][105692] Updated weights for policy 0, policy_version 1419360 (0.0010) [2023-12-27 01:43:45,812][105692] Updated weights for policy 0, policy_version 1419370 (0.0007) [2023-12-27 01:43:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 727375872. Throughput: 0: 9846.2, 1: 9741.5. Samples: 727341656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:43:46,062][104569] Avg episode reward: [(0, '7141.442'), (1, '8899.708')] [2023-12-27 01:43:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001421544_363962368.pth... [2023-12-27 01:43:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001419376_363413504.pth... [2023-12-27 01:43:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001420392_363667456.pth [2023-12-27 01:43:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001418224_363118592.pth [2023-12-27 01:43:46,373][105620] Updated weights for policy 1, policy_version 1421550 (0.0010) [2023-12-27 01:43:46,423][105620] Updated weights for policy 1, policy_version 1421560 (0.0010) [2023-12-27 01:43:46,471][105620] Updated weights for policy 1, policy_version 1421570 (0.0005) [2023-12-27 01:43:46,564][105692] Updated weights for policy 0, policy_version 1419380 (0.0008) [2023-12-27 01:43:46,622][105692] Updated weights for policy 0, policy_version 1419390 (0.0010) [2023-12-27 01:43:46,678][105692] Updated weights for policy 0, policy_version 1419400 (0.0011) [2023-12-27 01:43:47,097][105620] Updated weights for policy 1, policy_version 1421580 (0.0006) [2023-12-27 01:43:47,147][105620] Updated weights for policy 1, policy_version 1421590 (0.0005) [2023-12-27 01:43:47,199][105620] Updated weights for policy 1, policy_version 1421600 (0.0005) [2023-12-27 01:43:47,357][105692] Updated weights for policy 0, policy_version 1419410 (0.0009) [2023-12-27 01:43:47,416][105692] Updated weights for policy 0, policy_version 1419420 (0.0006) [2023-12-27 01:43:47,475][105692] Updated weights for policy 0, policy_version 1419430 (0.0009) [2023-12-27 01:43:47,519][105692] Updated weights for policy 0, policy_version 1419440 (0.0010) [2023-12-27 01:43:47,740][105620] Updated weights for policy 1, policy_version 1421610 (0.0005) [2023-12-27 01:43:47,792][105620] Updated weights for policy 1, policy_version 1421620 (0.0008) [2023-12-27 01:43:47,843][105620] Updated weights for policy 1, policy_version 1421630 (0.0007) [2023-12-27 01:43:47,889][105620] Updated weights for policy 1, policy_version 1421640 (0.0005) [2023-12-27 01:43:48,245][105692] Updated weights for policy 0, policy_version 1419450 (0.0009) [2023-12-27 01:43:48,296][105692] Updated weights for policy 0, policy_version 1419460 (0.0009) [2023-12-27 01:43:48,356][105692] Updated weights for policy 0, policy_version 1419470 (0.0008) [2023-12-27 01:43:48,533][105620] Updated weights for policy 1, policy_version 1421650 (0.0009) [2023-12-27 01:43:48,591][105620] Updated weights for policy 1, policy_version 1421660 (0.0009) [2023-12-27 01:43:48,640][105620] Updated weights for policy 1, policy_version 1421670 (0.0006) [2023-12-27 01:43:49,109][105692] Updated weights for policy 0, policy_version 1419480 (0.0006) [2023-12-27 01:43:49,180][105692] Updated weights for policy 0, policy_version 1419490 (0.0005) [2023-12-27 01:43:49,253][105692] Updated weights for policy 0, policy_version 1419500 (0.0007) [2023-12-27 01:43:49,270][105620] Updated weights for policy 1, policy_version 1421680 (0.0007) [2023-12-27 01:43:49,331][105620] Updated weights for policy 1, policy_version 1421690 (0.0007) [2023-12-27 01:43:49,408][105620] Updated weights for policy 1, policy_version 1421700 (0.0009) [2023-12-27 01:43:49,935][105692] Updated weights for policy 0, policy_version 1419510 (0.0010) [2023-12-27 01:43:49,998][105692] Updated weights for policy 0, policy_version 1419520 (0.0011) [2023-12-27 01:43:50,038][105620] Updated weights for policy 1, policy_version 1421710 (0.0010) [2023-12-27 01:43:50,059][105692] Updated weights for policy 0, policy_version 1419530 (0.0011) [2023-12-27 01:43:50,100][105620] Updated weights for policy 1, policy_version 1421720 (0.0010) [2023-12-27 01:43:50,160][105620] Updated weights for policy 1, policy_version 1421730 (0.0008) [2023-12-27 01:43:50,741][105692] Updated weights for policy 0, policy_version 1419540 (0.0009) [2023-12-27 01:43:50,798][105692] Updated weights for policy 0, policy_version 1419550 (0.0010) [2023-12-27 01:43:50,846][105692] Updated weights for policy 0, policy_version 1419560 (0.0010) [2023-12-27 01:43:50,968][105620] Updated weights for policy 1, policy_version 1421740 (0.0008) [2023-12-27 01:43:51,030][105620] Updated weights for policy 1, policy_version 1421750 (0.0009) [2023-12-27 01:43:51,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 727474176. Throughput: 0: 9849.4, 1: 9955.6. Samples: 727465152. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:43:51,063][104569] Avg episode reward: [(0, '7235.534'), (1, '8897.043')] [2023-12-27 01:43:51,096][105620] Updated weights for policy 1, policy_version 1421760 (0.0009) [2023-12-27 01:43:51,558][105692] Updated weights for policy 0, policy_version 1419570 (0.0010) [2023-12-27 01:43:51,626][105692] Updated weights for policy 0, policy_version 1419580 (0.0006) [2023-12-27 01:43:51,687][105692] Updated weights for policy 0, policy_version 1419590 (0.0009) [2023-12-27 01:43:51,754][105692] Updated weights for policy 0, policy_version 1419600 (0.0010) [2023-12-27 01:43:51,890][105620] Updated weights for policy 1, policy_version 1421770 (0.0008) [2023-12-27 01:43:51,949][105620] Updated weights for policy 1, policy_version 1421780 (0.0008) [2023-12-27 01:43:52,007][105620] Updated weights for policy 1, policy_version 1421790 (0.0008) [2023-12-27 01:43:52,063][105620] Updated weights for policy 1, policy_version 1421800 (0.0007) [2023-12-27 01:43:52,501][105692] Updated weights for policy 0, policy_version 1419610 (0.0010) [2023-12-27 01:43:52,554][105692] Updated weights for policy 0, policy_version 1419620 (0.0010) [2023-12-27 01:43:52,603][105585] KL-divergence is very high: 103.8351 [2023-12-27 01:43:52,617][105692] Updated weights for policy 0, policy_version 1419630 (0.0011) [2023-12-27 01:43:52,831][105620] Updated weights for policy 1, policy_version 1421810 (0.0008) [2023-12-27 01:43:52,882][105620] Updated weights for policy 1, policy_version 1421820 (0.0007) [2023-12-27 01:43:52,940][105620] Updated weights for policy 1, policy_version 1421830 (0.0008) [2023-12-27 01:43:53,353][105692] Updated weights for policy 0, policy_version 1419640 (0.0010) [2023-12-27 01:43:53,398][105692] Updated weights for policy 0, policy_version 1419650 (0.0010) [2023-12-27 01:43:53,459][105692] Updated weights for policy 0, policy_version 1419660 (0.0010) [2023-12-27 01:43:53,687][105620] Updated weights for policy 1, policy_version 1421840 (0.0010) [2023-12-27 01:43:53,739][105620] Updated weights for policy 1, policy_version 1421850 (0.0010) [2023-12-27 01:43:53,787][105620] Updated weights for policy 1, policy_version 1421860 (0.0010) [2023-12-27 01:43:54,212][105692] Updated weights for policy 0, policy_version 1419670 (0.0010) [2023-12-27 01:43:54,270][105692] Updated weights for policy 0, policy_version 1419680 (0.0011) [2023-12-27 01:43:54,322][105692] Updated weights for policy 0, policy_version 1419690 (0.0010) [2023-12-27 01:43:54,540][105620] Updated weights for policy 1, policy_version 1421870 (0.0010) [2023-12-27 01:43:54,598][105620] Updated weights for policy 1, policy_version 1421880 (0.0009) [2023-12-27 01:43:54,656][105620] Updated weights for policy 1, policy_version 1421890 (0.0010) [2023-12-27 01:43:55,072][105692] Updated weights for policy 0, policy_version 1419700 (0.0010) [2023-12-27 01:43:55,124][105692] Updated weights for policy 0, policy_version 1419710 (0.0010) [2023-12-27 01:43:55,182][105692] Updated weights for policy 0, policy_version 1419720 (0.0006) [2023-12-27 01:43:55,284][105620] Updated weights for policy 1, policy_version 1421900 (0.0010) [2023-12-27 01:43:55,347][105620] Updated weights for policy 1, policy_version 1421910 (0.0010) [2023-12-27 01:43:55,409][105620] Updated weights for policy 1, policy_version 1421920 (0.0010) [2023-12-27 01:43:55,908][105692] Updated weights for policy 0, policy_version 1419730 (0.0010) [2023-12-27 01:43:55,955][105692] Updated weights for policy 0, policy_version 1419740 (0.0006) [2023-12-27 01:43:55,979][105620] Updated weights for policy 1, policy_version 1421930 (0.0008) [2023-12-27 01:43:56,018][105692] Updated weights for policy 0, policy_version 1419750 (0.0006) [2023-12-27 01:43:56,036][105620] Updated weights for policy 1, policy_version 1421940 (0.0005) [2023-12-27 01:43:56,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 727564288. Throughput: 0: 9836.5, 1: 9873.5. Samples: 727579652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:43:56,062][104569] Avg episode reward: [(0, '7246.961'), (1, '8619.298')] [2023-12-27 01:43:56,076][105692] Updated weights for policy 0, policy_version 1419760 (0.0009) [2023-12-27 01:43:56,089][105620] Updated weights for policy 1, policy_version 1421950 (0.0006) [2023-12-27 01:43:56,158][105620] Updated weights for policy 1, policy_version 1421960 (0.0006) [2023-12-27 01:43:56,704][105692] Updated weights for policy 0, policy_version 1419770 (0.0008) [2023-12-27 01:43:56,729][105620] Updated weights for policy 1, policy_version 1421970 (0.0005) [2023-12-27 01:43:56,764][105692] Updated weights for policy 0, policy_version 1419780 (0.0005) [2023-12-27 01:43:56,777][105620] Updated weights for policy 1, policy_version 1421980 (0.0007) [2023-12-27 01:43:56,829][105692] Updated weights for policy 0, policy_version 1419790 (0.0005) [2023-12-27 01:43:56,832][105620] Updated weights for policy 1, policy_version 1421990 (0.0010) [2023-12-27 01:43:57,480][105620] Updated weights for policy 1, policy_version 1422000 (0.0007) [2023-12-27 01:43:57,491][105692] Updated weights for policy 0, policy_version 1419800 (0.0005) [2023-12-27 01:43:57,545][105620] Updated weights for policy 1, policy_version 1422010 (0.0006) [2023-12-27 01:43:57,549][105692] Updated weights for policy 0, policy_version 1419810 (0.0005) [2023-12-27 01:43:57,598][105692] Updated weights for policy 0, policy_version 1419820 (0.0005) [2023-12-27 01:43:57,604][105620] Updated weights for policy 1, policy_version 1422020 (0.0009) [2023-12-27 01:43:58,247][105692] Updated weights for policy 0, policy_version 1419830 (0.0007) [2023-12-27 01:43:58,265][105620] Updated weights for policy 1, policy_version 1422030 (0.0009) [2023-12-27 01:43:58,270][105585] KL-divergence is very high: 105.9904 [2023-12-27 01:43:58,292][105692] Updated weights for policy 0, policy_version 1419840 (0.0006) [2023-12-27 01:43:58,307][105585] KL-divergence is very high: 177.2825 [2023-12-27 01:43:58,321][105620] Updated weights for policy 1, policy_version 1422040 (0.0008) [2023-12-27 01:43:58,348][105692] Updated weights for policy 0, policy_version 1419850 (0.0011) [2023-12-27 01:43:58,354][105585] KL-divergence is very high: 194.6285 [2023-12-27 01:43:58,383][105620] Updated weights for policy 1, policy_version 1422050 (0.0008) [2023-12-27 01:43:59,177][105620] Updated weights for policy 1, policy_version 1422060 (0.0008) [2023-12-27 01:43:59,186][105692] Updated weights for policy 0, policy_version 1419860 (0.0007) [2023-12-27 01:43:59,233][105620] Updated weights for policy 1, policy_version 1422070 (0.0008) [2023-12-27 01:43:59,263][105692] Updated weights for policy 0, policy_version 1419870 (0.0008) [2023-12-27 01:43:59,298][105620] Updated weights for policy 1, policy_version 1422080 (0.0008) [2023-12-27 01:43:59,325][105692] Updated weights for policy 0, policy_version 1419880 (0.0009) [2023-12-27 01:43:59,913][105620] Updated weights for policy 1, policy_version 1422090 (0.0008) [2023-12-27 01:43:59,986][105620] Updated weights for policy 1, policy_version 1422100 (0.0009) [2023-12-27 01:44:00,054][105620] Updated weights for policy 1, policy_version 1422110 (0.0009) [2023-12-27 01:44:00,120][105620] Updated weights for policy 1, policy_version 1422120 (0.0010) [2023-12-27 01:44:00,137][105692] Updated weights for policy 0, policy_version 1419890 (0.0010) [2023-12-27 01:44:00,188][105692] Updated weights for policy 0, policy_version 1419900 (0.0007) [2023-12-27 01:44:00,250][105692] Updated weights for policy 0, policy_version 1419910 (0.0009) [2023-12-27 01:44:00,301][105692] Updated weights for policy 0, policy_version 1419920 (0.0009) [2023-12-27 01:44:00,881][105620] Updated weights for policy 1, policy_version 1422130 (0.0010) [2023-12-27 01:44:00,941][105620] Updated weights for policy 1, policy_version 1422140 (0.0010) [2023-12-27 01:44:00,999][105620] Updated weights for policy 1, policy_version 1422150 (0.0010) [2023-12-27 01:44:01,008][105692] Updated weights for policy 0, policy_version 1419930 (0.0005) [2023-12-27 01:44:01,057][105692] Updated weights for policy 0, policy_version 1419940 (0.0008) [2023-12-27 01:44:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19605.2). Total num frames: 727670784. Throughput: 0: 9839.2, 1: 9974.4. Samples: 727641312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:44:01,063][104569] Avg episode reward: [(0, '7609.052'), (1, '8160.929')] [2023-12-27 01:44:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001422152_364118016.pth... [2023-12-27 01:44:01,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001420968_363814912.pth [2023-12-27 01:44:01,106][105692] Updated weights for policy 0, policy_version 1419950 (0.0008) [2023-12-27 01:44:01,115][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001419952_363560960.pth... [2023-12-27 01:44:01,119][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001418800_363266048.pth [2023-12-27 01:44:01,773][105620] Updated weights for policy 1, policy_version 1422160 (0.0009) [2023-12-27 01:44:01,835][105620] Updated weights for policy 1, policy_version 1422170 (0.0009) [2023-12-27 01:44:01,895][105620] Updated weights for policy 1, policy_version 1422180 (0.0008) [2023-12-27 01:44:01,901][105692] Updated weights for policy 0, policy_version 1419960 (0.0008) [2023-12-27 01:44:01,955][105692] Updated weights for policy 0, policy_version 1419970 (0.0007) [2023-12-27 01:44:02,014][105692] Updated weights for policy 0, policy_version 1419980 (0.0008) [2023-12-27 01:44:02,616][105620] Updated weights for policy 1, policy_version 1422190 (0.0010) [2023-12-27 01:44:02,673][105620] Updated weights for policy 1, policy_version 1422200 (0.0005) [2023-12-27 01:44:02,691][105692] Updated weights for policy 0, policy_version 1419990 (0.0006) [2023-12-27 01:44:02,721][105585] KL-divergence is very high: 114.0243 [2023-12-27 01:44:02,733][105620] Updated weights for policy 1, policy_version 1422210 (0.0005) [2023-12-27 01:44:02,745][105692] Updated weights for policy 0, policy_version 1420000 (0.0005) [2023-12-27 01:44:02,765][105585] KL-divergence is very high: 104.2855 [2023-12-27 01:44:02,773][105585] KL-divergence is very high: 201.5184 [2023-12-27 01:44:02,806][105692] Updated weights for policy 0, policy_version 1420010 (0.0009) [2023-12-27 01:44:02,811][105585] KL-divergence is very high: 106.7271 [2023-12-27 01:44:02,816][105585] KL-divergence is very high: 217.6407 [2023-12-27 01:44:03,330][105620] Updated weights for policy 1, policy_version 1422220 (0.0008) [2023-12-27 01:44:03,388][105620] Updated weights for policy 1, policy_version 1422230 (0.0011) [2023-12-27 01:44:03,432][105620] Updated weights for policy 1, policy_version 1422240 (0.0010) [2023-12-27 01:44:03,530][105585] KL-divergence is very high: 105.2776 [2023-12-27 01:44:03,558][105692] Updated weights for policy 0, policy_version 1420020 (0.0009) [2023-12-27 01:44:03,618][105692] Updated weights for policy 0, policy_version 1420030 (0.0009) [2023-12-27 01:44:03,686][105692] Updated weights for policy 0, policy_version 1420040 (0.0008) [2023-12-27 01:44:04,009][105620] Updated weights for policy 1, policy_version 1422250 (0.0006) [2023-12-27 01:44:04,072][105620] Updated weights for policy 1, policy_version 1422260 (0.0006) [2023-12-27 01:44:04,140][105620] Updated weights for policy 1, policy_version 1422270 (0.0006) [2023-12-27 01:44:04,204][105620] Updated weights for policy 1, policy_version 1422280 (0.0011) [2023-12-27 01:44:04,440][105692] Updated weights for policy 0, policy_version 1420050 (0.0010) [2023-12-27 01:44:04,474][105585] KL-divergence is very high: 159.6944 [2023-12-27 01:44:04,496][105692] Updated weights for policy 0, policy_version 1420060 (0.0010) [2023-12-27 01:44:04,512][105585] KL-divergence is very high: 152.2518 [2023-12-27 01:44:04,518][105585] KL-divergence is very high: 309.9501 [2023-12-27 01:44:04,552][105692] Updated weights for policy 0, policy_version 1420070 (0.0011) [2023-12-27 01:44:04,554][105585] KL-divergence is very high: 178.2980 [2023-12-27 01:44:04,561][105585] KL-divergence is very high: 367.8164 [2023-12-27 01:44:04,606][105585] KL-divergence is very high: 173.1488 [2023-12-27 01:44:04,905][105620] Updated weights for policy 1, policy_version 1422290 (0.0010) [2023-12-27 01:44:04,964][105620] Updated weights for policy 1, policy_version 1422300 (0.0010) [2023-12-27 01:44:05,022][105620] Updated weights for policy 1, policy_version 1422310 (0.0010) [2023-12-27 01:44:05,226][105692] Updated weights for policy 0, policy_version 1420081 (0.0010) [2023-12-27 01:44:05,281][105692] Updated weights for policy 0, policy_version 1420091 (0.0008) [2023-12-27 01:44:05,327][105692] Updated weights for policy 0, policy_version 1420101 (0.0005) [2023-12-27 01:44:05,374][105692] Updated weights for policy 0, policy_version 1420111 (0.0005) [2023-12-27 01:44:05,707][105620] Updated weights for policy 1, policy_version 1422320 (0.0009) [2023-12-27 01:44:05,762][105620] Updated weights for policy 1, policy_version 1422330 (0.0009) [2023-12-27 01:44:05,821][105620] Updated weights for policy 1, policy_version 1422340 (0.0010) [2023-12-27 01:44:05,957][105692] Updated weights for policy 0, policy_version 1420121 (0.0006) [2023-12-27 01:44:06,005][105692] Updated weights for policy 0, policy_version 1420131 (0.0010) [2023-12-27 01:44:06,053][105692] Updated weights for policy 0, policy_version 1420141 (0.0010) [2023-12-27 01:44:06,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 727769088. Throughput: 0: 9800.3, 1: 9976.7. Samples: 727756820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:44:06,062][104569] Avg episode reward: [(0, '7327.136'), (1, '8344.887')] [2023-12-27 01:44:06,686][105692] Updated weights for policy 0, policy_version 1420151 (0.0007) [2023-12-27 01:44:06,687][105620] Updated weights for policy 1, policy_version 1422350 (0.0009) [2023-12-27 01:44:06,740][105620] Updated weights for policy 1, policy_version 1422360 (0.0008) [2023-12-27 01:44:06,742][105692] Updated weights for policy 0, policy_version 1420161 (0.0006) [2023-12-27 01:44:06,797][105620] Updated weights for policy 1, policy_version 1422370 (0.0009) [2023-12-27 01:44:06,803][105692] Updated weights for policy 0, policy_version 1420171 (0.0007) [2023-12-27 01:44:07,362][105692] Updated weights for policy 0, policy_version 1420181 (0.0009) [2023-12-27 01:44:07,420][105692] Updated weights for policy 0, policy_version 1420191 (0.0009) [2023-12-27 01:44:07,474][105692] Updated weights for policy 0, policy_version 1420201 (0.0007) [2023-12-27 01:44:07,601][105620] Updated weights for policy 1, policy_version 1422380 (0.0006) [2023-12-27 01:44:07,655][105620] Updated weights for policy 1, policy_version 1422390 (0.0005) [2023-12-27 01:44:07,713][105620] Updated weights for policy 1, policy_version 1422400 (0.0006) [2023-12-27 01:44:08,251][105620] Updated weights for policy 1, policy_version 1422410 (0.0005) [2023-12-27 01:44:08,306][105620] Updated weights for policy 1, policy_version 1422420 (0.0006) [2023-12-27 01:44:08,357][105692] Updated weights for policy 0, policy_version 1420211 (0.0009) [2023-12-27 01:44:08,369][105620] Updated weights for policy 1, policy_version 1422430 (0.0008) [2023-12-27 01:44:08,417][105692] Updated weights for policy 0, policy_version 1420221 (0.0008) [2023-12-27 01:44:08,428][105620] Updated weights for policy 1, policy_version 1422440 (0.0008) [2023-12-27 01:44:08,478][105692] Updated weights for policy 0, policy_version 1420231 (0.0009) [2023-12-27 01:44:09,158][105620] Updated weights for policy 1, policy_version 1422450 (0.0009) [2023-12-27 01:44:09,193][105692] Updated weights for policy 0, policy_version 1420241 (0.0008) [2023-12-27 01:44:09,212][105620] Updated weights for policy 1, policy_version 1422460 (0.0007) [2023-12-27 01:44:09,253][105692] Updated weights for policy 0, policy_version 1420251 (0.0008) [2023-12-27 01:44:09,272][105585] KL-divergence is very high: 117.3679 [2023-12-27 01:44:09,278][105620] Updated weights for policy 1, policy_version 1422470 (0.0008) [2023-12-27 01:44:09,289][105585] KL-divergence is very high: 158.3357 [2023-12-27 01:44:09,308][105585] KL-divergence is very high: 121.8420 [2023-12-27 01:44:09,315][105692] Updated weights for policy 0, policy_version 1420261 (0.0010) [2023-12-27 01:44:09,318][105585] KL-divergence is very high: 167.6697 [2023-12-27 01:44:09,335][105585] KL-divergence is very high: 185.5375 [2023-12-27 01:44:09,355][105585] KL-divergence is very high: 126.7001 [2023-12-27 01:44:09,369][105585] KL-divergence is very high: 158.2062 [2023-12-27 01:44:09,381][105692] Updated weights for policy 0, policy_version 1420272 (0.0011) [2023-12-27 01:44:10,093][105620] Updated weights for policy 1, policy_version 1422480 (0.0008) [2023-12-27 01:44:10,148][105620] Updated weights for policy 1, policy_version 1422490 (0.0009) [2023-12-27 01:44:10,167][105692] Updated weights for policy 0, policy_version 1420282 (0.0010) [2023-12-27 01:44:10,202][105620] Updated weights for policy 1, policy_version 1422500 (0.0006) [2023-12-27 01:44:10,216][105692] Updated weights for policy 0, policy_version 1420292 (0.0010) [2023-12-27 01:44:10,271][105692] Updated weights for policy 0, policy_version 1420302 (0.0010) [2023-12-27 01:44:10,924][105692] Updated weights for policy 0, policy_version 1420312 (0.0006) [2023-12-27 01:44:10,966][105620] Updated weights for policy 1, policy_version 1422510 (0.0006) [2023-12-27 01:44:10,980][105692] Updated weights for policy 0, policy_version 1420322 (0.0011) [2023-12-27 01:44:11,020][105620] Updated weights for policy 1, policy_version 1422520 (0.0006) [2023-12-27 01:44:11,039][105692] Updated weights for policy 0, policy_version 1420332 (0.0010) [2023-12-27 01:44:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 727859200. Throughput: 0: 9791.9, 1: 9950.3. Samples: 727873808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:44:11,063][104569] Avg episode reward: [(0, '7422.813'), (1, '8804.239')] [2023-12-27 01:44:11,089][105620] Updated weights for policy 1, policy_version 1422530 (0.0009) [2023-12-27 01:44:11,839][105692] Updated weights for policy 0, policy_version 1420342 (0.0011) [2023-12-27 01:44:11,861][105620] Updated weights for policy 1, policy_version 1422540 (0.0008) [2023-12-27 01:44:11,903][105692] Updated weights for policy 0, policy_version 1420352 (0.0011) [2023-12-27 01:44:11,917][105620] Updated weights for policy 1, policy_version 1422550 (0.0007) [2023-12-27 01:44:11,967][105692] Updated weights for policy 0, policy_version 1420362 (0.0007) [2023-12-27 01:44:11,983][105620] Updated weights for policy 1, policy_version 1422560 (0.0008) [2023-12-27 01:44:12,651][105692] Updated weights for policy 0, policy_version 1420372 (0.0007) [2023-12-27 01:44:12,717][105692] Updated weights for policy 0, policy_version 1420382 (0.0008) [2023-12-27 01:44:12,759][105620] Updated weights for policy 1, policy_version 1422570 (0.0010) [2023-12-27 01:44:12,779][105692] Updated weights for policy 0, policy_version 1420392 (0.0007) [2023-12-27 01:44:12,814][105620] Updated weights for policy 1, policy_version 1422580 (0.0008) [2023-12-27 01:44:12,872][105620] Updated weights for policy 1, policy_version 1422590 (0.0005) [2023-12-27 01:44:12,943][105620] Updated weights for policy 1, policy_version 1422600 (0.0010) [2023-12-27 01:44:13,361][105692] Updated weights for policy 0, policy_version 1420402 (0.0007) [2023-12-27 01:44:13,432][105692] Updated weights for policy 0, policy_version 1420412 (0.0006) [2023-12-27 01:44:13,499][105692] Updated weights for policy 0, policy_version 1420422 (0.0005) [2023-12-27 01:44:13,565][105692] Updated weights for policy 0, policy_version 1420432 (0.0006) [2023-12-27 01:44:13,638][105620] Updated weights for policy 1, policy_version 1422610 (0.0009) [2023-12-27 01:44:13,691][105620] Updated weights for policy 1, policy_version 1422621 (0.0010) [2023-12-27 01:44:13,738][105620] Updated weights for policy 1, policy_version 1422631 (0.0009) [2023-12-27 01:44:14,060][105692] Updated weights for policy 0, policy_version 1420442 (0.0005) [2023-12-27 01:44:14,115][105692] Updated weights for policy 0, policy_version 1420452 (0.0006) [2023-12-27 01:44:14,178][105692] Updated weights for policy 0, policy_version 1420462 (0.0008) [2023-12-27 01:44:14,588][105620] Updated weights for policy 1, policy_version 1422641 (0.0009) [2023-12-27 01:44:14,635][105620] Updated weights for policy 1, policy_version 1422651 (0.0008) [2023-12-27 01:44:14,680][105620] Updated weights for policy 1, policy_version 1422661 (0.0008) [2023-12-27 01:44:14,898][105692] Updated weights for policy 0, policy_version 1420472 (0.0010) [2023-12-27 01:44:14,954][105692] Updated weights for policy 0, policy_version 1420482 (0.0011) [2023-12-27 01:44:15,028][105692] Updated weights for policy 0, policy_version 1420493 (0.0008) [2023-12-27 01:44:15,475][105620] Updated weights for policy 1, policy_version 1422671 (0.0010) [2023-12-27 01:44:15,537][105620] Updated weights for policy 1, policy_version 1422681 (0.0010) [2023-12-27 01:44:15,586][105620] Updated weights for policy 1, policy_version 1422691 (0.0010) [2023-12-27 01:44:15,665][105692] Updated weights for policy 0, policy_version 1420503 (0.0009) [2023-12-27 01:44:15,723][105692] Updated weights for policy 0, policy_version 1420513 (0.0010) [2023-12-27 01:44:15,778][105692] Updated weights for policy 0, policy_version 1420523 (0.0010) [2023-12-27 01:44:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 727965696. Throughput: 0: 9804.5, 1: 9839.4. Samples: 727932436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:44:16,063][104569] Avg episode reward: [(0, '7333.408'), (1, '8715.595')] [2023-12-27 01:44:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001420528_363708416.pth... [2023-12-27 01:44:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001422696_364257280.pth... [2023-12-27 01:44:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001419376_363413504.pth [2023-12-27 01:44:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001421544_363962368.pth [2023-12-27 01:44:16,320][105620] Updated weights for policy 1, policy_version 1422701 (0.0009) [2023-12-27 01:44:16,377][105620] Updated weights for policy 1, policy_version 1422711 (0.0007) [2023-12-27 01:44:16,430][105620] Updated weights for policy 1, policy_version 1422721 (0.0008) [2023-12-27 01:44:16,478][105692] Updated weights for policy 0, policy_version 1420533 (0.0011) [2023-12-27 01:44:16,540][105692] Updated weights for policy 0, policy_version 1420543 (0.0010) [2023-12-27 01:44:16,599][105692] Updated weights for policy 0, policy_version 1420553 (0.0010) [2023-12-27 01:44:17,099][105620] Updated weights for policy 1, policy_version 1422731 (0.0006) [2023-12-27 01:44:17,160][105620] Updated weights for policy 1, policy_version 1422741 (0.0005) [2023-12-27 01:44:17,206][105692] Updated weights for policy 0, policy_version 1420563 (0.0008) [2023-12-27 01:44:17,211][105620] Updated weights for policy 1, policy_version 1422751 (0.0008) [2023-12-27 01:44:17,266][105692] Updated weights for policy 0, policy_version 1420573 (0.0005) [2023-12-27 01:44:17,326][105692] Updated weights for policy 0, policy_version 1420583 (0.0005) [2023-12-27 01:44:17,800][105620] Updated weights for policy 1, policy_version 1422761 (0.0010) [2023-12-27 01:44:17,868][105620] Updated weights for policy 1, policy_version 1422771 (0.0006) [2023-12-27 01:44:17,900][105692] Updated weights for policy 0, policy_version 1420593 (0.0006) [2023-12-27 01:44:17,935][105620] Updated weights for policy 1, policy_version 1422781 (0.0006) [2023-12-27 01:44:17,946][105692] Updated weights for policy 0, policy_version 1420603 (0.0008) [2023-12-27 01:44:17,988][105620] Updated weights for policy 1, policy_version 1422791 (0.0008) [2023-12-27 01:44:18,003][105692] Updated weights for policy 0, policy_version 1420613 (0.0007) [2023-12-27 01:44:18,053][105692] Updated weights for policy 0, policy_version 1420623 (0.0008) [2023-12-27 01:44:18,604][105620] Updated weights for policy 1, policy_version 1422801 (0.0009) [2023-12-27 01:44:18,658][105620] Updated weights for policy 1, policy_version 1422811 (0.0009) [2023-12-27 01:44:18,718][105620] Updated weights for policy 1, policy_version 1422821 (0.0008) [2023-12-27 01:44:18,834][105692] Updated weights for policy 0, policy_version 1420633 (0.0009) [2023-12-27 01:44:18,892][105692] Updated weights for policy 0, policy_version 1420643 (0.0008) [2023-12-27 01:44:18,953][105692] Updated weights for policy 0, policy_version 1420653 (0.0009) [2023-12-27 01:44:19,407][105620] Updated weights for policy 1, policy_version 1422831 (0.0008) [2023-12-27 01:44:19,474][105620] Updated weights for policy 1, policy_version 1422841 (0.0006) [2023-12-27 01:44:19,545][105620] Updated weights for policy 1, policy_version 1422851 (0.0008) [2023-12-27 01:44:19,611][105692] Updated weights for policy 0, policy_version 1420663 (0.0007) [2023-12-27 01:44:19,674][105692] Updated weights for policy 0, policy_version 1420673 (0.0006) [2023-12-27 01:44:19,728][105692] Updated weights for policy 0, policy_version 1420683 (0.0006) [2023-12-27 01:44:20,189][105620] Updated weights for policy 1, policy_version 1422861 (0.0008) [2023-12-27 01:44:20,250][105620] Updated weights for policy 1, policy_version 1422871 (0.0009) [2023-12-27 01:44:20,306][105620] Updated weights for policy 1, policy_version 1422881 (0.0009) [2023-12-27 01:44:20,345][105692] Updated weights for policy 0, policy_version 1420693 (0.0007) [2023-12-27 01:44:20,403][105692] Updated weights for policy 0, policy_version 1420703 (0.0010) [2023-12-27 01:44:20,465][105692] Updated weights for policy 0, policy_version 1420713 (0.0009) [2023-12-27 01:44:21,025][105620] Updated weights for policy 1, policy_version 1422891 (0.0006) [2023-12-27 01:44:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 728064000. Throughput: 0: 9887.5, 1: 9856.1. Samples: 728056296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:44:21,063][104569] Avg episode reward: [(0, '8064.179'), (1, '8902.051')] [2023-12-27 01:44:21,095][105620] Updated weights for policy 1, policy_version 1422901 (0.0008) [2023-12-27 01:44:21,160][105620] Updated weights for policy 1, policy_version 1422911 (0.0009) [2023-12-27 01:44:21,275][105692] Updated weights for policy 0, policy_version 1420723 (0.0009) [2023-12-27 01:44:21,338][105692] Updated weights for policy 0, policy_version 1420733 (0.0008) [2023-12-27 01:44:21,412][105692] Updated weights for policy 0, policy_version 1420743 (0.0007) [2023-12-27 01:44:21,884][105620] Updated weights for policy 1, policy_version 1422921 (0.0010) [2023-12-27 01:44:21,947][105620] Updated weights for policy 1, policy_version 1422931 (0.0009) [2023-12-27 01:44:21,998][105620] Updated weights for policy 1, policy_version 1422941 (0.0008) [2023-12-27 01:44:22,056][105620] Updated weights for policy 1, policy_version 1422951 (0.0009) [2023-12-27 01:44:22,193][105692] Updated weights for policy 0, policy_version 1420753 (0.0009) [2023-12-27 01:44:22,251][105692] Updated weights for policy 0, policy_version 1420763 (0.0009) [2023-12-27 01:44:22,314][105692] Updated weights for policy 0, policy_version 1420773 (0.0006) [2023-12-27 01:44:22,387][105692] Updated weights for policy 0, policy_version 1420783 (0.0008) [2023-12-27 01:44:22,847][105620] Updated weights for policy 1, policy_version 1422961 (0.0009) [2023-12-27 01:44:22,913][105620] Updated weights for policy 1, policy_version 1422971 (0.0009) [2023-12-27 01:44:22,964][105620] Updated weights for policy 1, policy_version 1422981 (0.0009) [2023-12-27 01:44:23,089][105692] Updated weights for policy 0, policy_version 1420793 (0.0009) [2023-12-27 01:44:23,144][105692] Updated weights for policy 0, policy_version 1420803 (0.0009) [2023-12-27 01:44:23,192][105692] Updated weights for policy 0, policy_version 1420813 (0.0009) [2023-12-27 01:44:23,713][105620] Updated weights for policy 1, policy_version 1422991 (0.0007) [2023-12-27 01:44:23,759][105620] Updated weights for policy 1, policy_version 1423001 (0.0005) [2023-12-27 01:44:23,811][105620] Updated weights for policy 1, policy_version 1423011 (0.0005) [2023-12-27 01:44:23,908][105692] Updated weights for policy 0, policy_version 1420823 (0.0006) [2023-12-27 01:44:23,959][105692] Updated weights for policy 0, policy_version 1420833 (0.0008) [2023-12-27 01:44:24,009][105692] Updated weights for policy 0, policy_version 1420843 (0.0008) [2023-12-27 01:44:24,395][105620] Updated weights for policy 1, policy_version 1423021 (0.0007) [2023-12-27 01:44:24,455][105620] Updated weights for policy 1, policy_version 1423031 (0.0009) [2023-12-27 01:44:24,513][105620] Updated weights for policy 1, policy_version 1423041 (0.0009) [2023-12-27 01:44:24,734][105692] Updated weights for policy 0, policy_version 1420853 (0.0010) [2023-12-27 01:44:24,794][105692] Updated weights for policy 0, policy_version 1420863 (0.0009) [2023-12-27 01:44:24,853][105692] Updated weights for policy 0, policy_version 1420873 (0.0009) [2023-12-27 01:44:25,187][105620] Updated weights for policy 1, policy_version 1423051 (0.0010) [2023-12-27 01:44:25,245][105620] Updated weights for policy 1, policy_version 1423061 (0.0009) [2023-12-27 01:44:25,292][105620] Updated weights for policy 1, policy_version 1423071 (0.0009) [2023-12-27 01:44:25,654][105692] Updated weights for policy 0, policy_version 1420883 (0.0010) [2023-12-27 01:44:25,702][105692] Updated weights for policy 0, policy_version 1420893 (0.0010) [2023-12-27 01:44:25,764][105692] Updated weights for policy 0, policy_version 1420903 (0.0010) [2023-12-27 01:44:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 728162304. Throughput: 0: 9796.3, 1: 9904.4. Samples: 728171396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:44:26,062][104569] Avg episode reward: [(0, '7883.969'), (1, '8901.343')] [2023-12-27 01:44:26,067][105620] Updated weights for policy 1, policy_version 1423081 (0.0009) [2023-12-27 01:44:26,117][105620] Updated weights for policy 1, policy_version 1423091 (0.0008) [2023-12-27 01:44:26,167][105620] Updated weights for policy 1, policy_version 1423101 (0.0009) [2023-12-27 01:44:26,219][105620] Updated weights for policy 1, policy_version 1423111 (0.0009) [2023-12-27 01:44:26,468][105692] Updated weights for policy 0, policy_version 1420913 (0.0009) [2023-12-27 01:44:26,527][105692] Updated weights for policy 0, policy_version 1420923 (0.0009) [2023-12-27 01:44:26,585][105692] Updated weights for policy 0, policy_version 1420933 (0.0009) [2023-12-27 01:44:26,644][105692] Updated weights for policy 0, policy_version 1420943 (0.0009) [2023-12-27 01:44:26,995][105620] Updated weights for policy 1, policy_version 1423121 (0.0010) [2023-12-27 01:44:27,053][105620] Updated weights for policy 1, policy_version 1423131 (0.0009) [2023-12-27 01:44:27,107][105620] Updated weights for policy 1, policy_version 1423141 (0.0009) [2023-12-27 01:44:27,347][105692] Updated weights for policy 0, policy_version 1420953 (0.0006) [2023-12-27 01:44:27,410][105692] Updated weights for policy 0, policy_version 1420963 (0.0005) [2023-12-27 01:44:27,480][105692] Updated weights for policy 0, policy_version 1420973 (0.0006) [2023-12-27 01:44:27,949][105620] Updated weights for policy 1, policy_version 1423151 (0.0008) [2023-12-27 01:44:28,008][105620] Updated weights for policy 1, policy_version 1423161 (0.0006) [2023-12-27 01:44:28,054][105692] Updated weights for policy 0, policy_version 1420983 (0.0008) [2023-12-27 01:44:28,061][105620] Updated weights for policy 1, policy_version 1423171 (0.0006) [2023-12-27 01:44:28,104][105692] Updated weights for policy 0, policy_version 1420993 (0.0007) [2023-12-27 01:44:28,149][105692] Updated weights for policy 0, policy_version 1421003 (0.0008) [2023-12-27 01:44:28,803][105620] Updated weights for policy 1, policy_version 1423181 (0.0007) [2023-12-27 01:44:28,855][105620] Updated weights for policy 1, policy_version 1423191 (0.0009) [2023-12-27 01:44:28,913][105692] Updated weights for policy 0, policy_version 1421014 (0.0007) [2023-12-27 01:44:28,916][105620] Updated weights for policy 1, policy_version 1423201 (0.0007) [2023-12-27 01:44:28,970][105692] Updated weights for policy 0, policy_version 1421024 (0.0007) [2023-12-27 01:44:29,024][105692] Updated weights for policy 0, policy_version 1421034 (0.0009) [2023-12-27 01:44:29,670][105620] Updated weights for policy 1, policy_version 1423211 (0.0007) [2023-12-27 01:44:29,734][105620] Updated weights for policy 1, policy_version 1423221 (0.0005) [2023-12-27 01:44:29,790][105692] Updated weights for policy 0, policy_version 1421044 (0.0009) [2023-12-27 01:44:29,791][105620] Updated weights for policy 1, policy_version 1423231 (0.0005) [2023-12-27 01:44:29,850][105692] Updated weights for policy 0, policy_version 1421054 (0.0008) [2023-12-27 01:44:29,902][105692] Updated weights for policy 0, policy_version 1421064 (0.0008) [2023-12-27 01:44:30,407][105620] Updated weights for policy 1, policy_version 1423241 (0.0006) [2023-12-27 01:44:30,460][105620] Updated weights for policy 1, policy_version 1423251 (0.0005) [2023-12-27 01:44:30,511][105620] Updated weights for policy 1, policy_version 1423261 (0.0005) [2023-12-27 01:44:30,562][105620] Updated weights for policy 1, policy_version 1423271 (0.0008) [2023-12-27 01:44:30,689][105692] Updated weights for policy 0, policy_version 1421074 (0.0007) [2023-12-27 01:44:30,735][105692] Updated weights for policy 0, policy_version 1421084 (0.0010) [2023-12-27 01:44:30,783][105692] Updated weights for policy 0, policy_version 1421094 (0.0010) [2023-12-27 01:44:30,836][105692] Updated weights for policy 0, policy_version 1421104 (0.0005) [2023-12-27 01:44:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 728260608. Throughput: 0: 9850.0, 1: 9859.4. Samples: 728228580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:44:31,062][104569] Avg episode reward: [(0, '7332.280'), (1, '7296.751')] [2023-12-27 01:44:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001421104_363855872.pth... [2023-12-27 01:44:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001423272_364404736.pth... [2023-12-27 01:44:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001422152_364118016.pth [2023-12-27 01:44:31,092][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001419952_363560960.pth [2023-12-27 01:44:31,387][105620] Updated weights for policy 1, policy_version 1423281 (0.0009) [2023-12-27 01:44:31,442][105620] Updated weights for policy 1, policy_version 1423291 (0.0009) [2023-12-27 01:44:31,464][105692] Updated weights for policy 0, policy_version 1421114 (0.0007) [2023-12-27 01:44:31,499][105620] Updated weights for policy 1, policy_version 1423301 (0.0006) [2023-12-27 01:44:31,527][105692] Updated weights for policy 0, policy_version 1421124 (0.0007) [2023-12-27 01:44:31,601][105692] Updated weights for policy 0, policy_version 1421134 (0.0009) [2023-12-27 01:44:32,178][105620] Updated weights for policy 1, policy_version 1423311 (0.0009) [2023-12-27 01:44:32,229][105620] Updated weights for policy 1, policy_version 1423321 (0.0009) [2023-12-27 01:44:32,293][105620] Updated weights for policy 1, policy_version 1423331 (0.0008) [2023-12-27 01:44:32,403][105692] Updated weights for policy 0, policy_version 1421144 (0.0009) [2023-12-27 01:44:32,455][105692] Updated weights for policy 0, policy_version 1421154 (0.0008) [2023-12-27 01:44:32,500][105692] Updated weights for policy 0, policy_version 1421164 (0.0008) [2023-12-27 01:44:33,052][105620] Updated weights for policy 1, policy_version 1423341 (0.0007) [2023-12-27 01:44:33,100][105620] Updated weights for policy 1, policy_version 1423351 (0.0009) [2023-12-27 01:44:33,150][105620] Updated weights for policy 1, policy_version 1423361 (0.0009) [2023-12-27 01:44:33,278][105692] Updated weights for policy 0, policy_version 1421174 (0.0009) [2023-12-27 01:44:33,326][105692] Updated weights for policy 0, policy_version 1421184 (0.0009) [2023-12-27 01:44:33,373][105692] Updated weights for policy 0, policy_version 1421194 (0.0009) [2023-12-27 01:44:33,837][105620] Updated weights for policy 1, policy_version 1423371 (0.0007) [2023-12-27 01:44:33,897][105620] Updated weights for policy 1, policy_version 1423381 (0.0005) [2023-12-27 01:44:33,960][105620] Updated weights for policy 1, policy_version 1423391 (0.0005) [2023-12-27 01:44:34,231][105692] Updated weights for policy 0, policy_version 1421204 (0.0009) [2023-12-27 01:44:34,278][105692] Updated weights for policy 0, policy_version 1421214 (0.0008) [2023-12-27 01:44:34,329][105692] Updated weights for policy 0, policy_version 1421224 (0.0009) [2023-12-27 01:44:34,556][105620] Updated weights for policy 1, policy_version 1423401 (0.0005) [2023-12-27 01:44:34,608][105620] Updated weights for policy 1, policy_version 1423411 (0.0006) [2023-12-27 01:44:34,665][105620] Updated weights for policy 1, policy_version 1423421 (0.0006) [2023-12-27 01:44:34,734][105620] Updated weights for policy 1, policy_version 1423431 (0.0008) [2023-12-27 01:44:35,140][105692] Updated weights for policy 0, policy_version 1421234 (0.0009) [2023-12-27 01:44:35,199][105692] Updated weights for policy 0, policy_version 1421244 (0.0010) [2023-12-27 01:44:35,265][105692] Updated weights for policy 0, policy_version 1421254 (0.0010) [2023-12-27 01:44:35,331][105692] Updated weights for policy 0, policy_version 1421264 (0.0010) [2023-12-27 01:44:35,499][105620] Updated weights for policy 1, policy_version 1423441 (0.0008) [2023-12-27 01:44:35,557][105620] Updated weights for policy 1, policy_version 1423451 (0.0007) [2023-12-27 01:44:35,609][105620] Updated weights for policy 1, policy_version 1423461 (0.0007) [2023-12-27 01:44:36,013][105692] Updated weights for policy 0, policy_version 1421274 (0.0005) [2023-12-27 01:44:36,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 728350720. Throughput: 0: 9778.2, 1: 9739.9. Samples: 728343468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:44:36,063][104569] Avg episode reward: [(0, '7052.688'), (1, '8013.699')] [2023-12-27 01:44:36,075][105692] Updated weights for policy 0, policy_version 1421284 (0.0009) [2023-12-27 01:44:36,139][105692] Updated weights for policy 0, policy_version 1421294 (0.0010) [2023-12-27 01:44:36,260][105620] Updated weights for policy 1, policy_version 1423471 (0.0007) [2023-12-27 01:44:36,307][105620] Updated weights for policy 1, policy_version 1423481 (0.0009) [2023-12-27 01:44:36,359][105620] Updated weights for policy 1, policy_version 1423491 (0.0008) [2023-12-27 01:44:36,858][105692] Updated weights for policy 0, policy_version 1421304 (0.0009) [2023-12-27 01:44:36,906][105692] Updated weights for policy 0, policy_version 1421314 (0.0009) [2023-12-27 01:44:36,962][105692] Updated weights for policy 0, policy_version 1421326 (0.0011) [2023-12-27 01:44:37,092][105620] Updated weights for policy 1, policy_version 1423501 (0.0007) [2023-12-27 01:44:37,147][105620] Updated weights for policy 1, policy_version 1423511 (0.0005) [2023-12-27 01:44:37,201][105620] Updated weights for policy 1, policy_version 1423521 (0.0006) [2023-12-27 01:44:37,701][105692] Updated weights for policy 0, policy_version 1421336 (0.0010) [2023-12-27 01:44:37,765][105692] Updated weights for policy 0, policy_version 1421346 (0.0011) [2023-12-27 01:44:37,825][105692] Updated weights for policy 0, policy_version 1421356 (0.0010) [2023-12-27 01:44:37,884][105620] Updated weights for policy 1, policy_version 1423531 (0.0007) [2023-12-27 01:44:37,944][105620] Updated weights for policy 1, policy_version 1423541 (0.0008) [2023-12-27 01:44:38,007][105620] Updated weights for policy 1, policy_version 1423551 (0.0008) [2023-12-27 01:44:38,471][105692] Updated weights for policy 0, policy_version 1421366 (0.0009) [2023-12-27 01:44:38,533][105692] Updated weights for policy 0, policy_version 1421376 (0.0008) [2023-12-27 01:44:38,594][105692] Updated weights for policy 0, policy_version 1421386 (0.0008) [2023-12-27 01:44:38,856][105620] Updated weights for policy 1, policy_version 1423561 (0.0009) [2023-12-27 01:44:38,912][105620] Updated weights for policy 1, policy_version 1423571 (0.0008) [2023-12-27 01:44:38,971][105620] Updated weights for policy 1, policy_version 1423581 (0.0008) [2023-12-27 01:44:39,016][105620] Updated weights for policy 1, policy_version 1423591 (0.0008) [2023-12-27 01:44:39,336][105692] Updated weights for policy 0, policy_version 1421396 (0.0012) [2023-12-27 01:44:39,398][105692] Updated weights for policy 0, policy_version 1421406 (0.0011) [2023-12-27 01:44:39,463][105692] Updated weights for policy 0, policy_version 1421416 (0.0009) [2023-12-27 01:44:39,795][105620] Updated weights for policy 1, policy_version 1423601 (0.0009) [2023-12-27 01:44:39,864][105620] Updated weights for policy 1, policy_version 1423611 (0.0010) [2023-12-27 01:44:39,925][105620] Updated weights for policy 1, policy_version 1423621 (0.0009) [2023-12-27 01:44:40,224][105692] Updated weights for policy 0, policy_version 1421426 (0.0008) [2023-12-27 01:44:40,289][105692] Updated weights for policy 0, policy_version 1421436 (0.0009) [2023-12-27 01:44:40,352][105692] Updated weights for policy 0, policy_version 1421446 (0.0009) [2023-12-27 01:44:40,410][105692] Updated weights for policy 0, policy_version 1421456 (0.0008) [2023-12-27 01:44:40,736][105620] Updated weights for policy 1, policy_version 1423631 (0.0009) [2023-12-27 01:44:40,788][105620] Updated weights for policy 1, policy_version 1423641 (0.0009) [2023-12-27 01:44:40,844][105620] Updated weights for policy 1, policy_version 1423651 (0.0009) [2023-12-27 01:44:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 728449024. Throughput: 0: 9794.8, 1: 9701.8. Samples: 728457000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:44:41,063][104569] Avg episode reward: [(0, '7603.545'), (1, '8437.575')] [2023-12-27 01:44:41,134][105692] Updated weights for policy 0, policy_version 1421466 (0.0009) [2023-12-27 01:44:41,199][105692] Updated weights for policy 0, policy_version 1421476 (0.0008) [2023-12-27 01:44:41,263][105692] Updated weights for policy 0, policy_version 1421486 (0.0008) [2023-12-27 01:44:41,588][105620] Updated weights for policy 1, policy_version 1423661 (0.0009) [2023-12-27 01:44:41,650][105620] Updated weights for policy 1, policy_version 1423671 (0.0008) [2023-12-27 01:44:41,717][105620] Updated weights for policy 1, policy_version 1423681 (0.0006) [2023-12-27 01:44:42,056][105692] Updated weights for policy 0, policy_version 1421496 (0.0008) [2023-12-27 01:44:42,121][105692] Updated weights for policy 0, policy_version 1421506 (0.0008) [2023-12-27 01:44:42,188][105692] Updated weights for policy 0, policy_version 1421516 (0.0008) [2023-12-27 01:44:42,375][105620] Updated weights for policy 1, policy_version 1423691 (0.0008) [2023-12-27 01:44:42,433][105620] Updated weights for policy 1, policy_version 1423701 (0.0009) [2023-12-27 01:44:42,484][105620] Updated weights for policy 1, policy_version 1423711 (0.0009) [2023-12-27 01:44:42,945][105692] Updated weights for policy 0, policy_version 1421526 (0.0009) [2023-12-27 01:44:43,009][105692] Updated weights for policy 0, policy_version 1421536 (0.0008) [2023-12-27 01:44:43,065][105692] Updated weights for policy 0, policy_version 1421546 (0.0005) [2023-12-27 01:44:43,202][105620] Updated weights for policy 1, policy_version 1423721 (0.0009) [2023-12-27 01:44:43,266][105620] Updated weights for policy 1, policy_version 1423731 (0.0009) [2023-12-27 01:44:43,326][105620] Updated weights for policy 1, policy_version 1423741 (0.0008) [2023-12-27 01:44:43,389][105620] Updated weights for policy 1, policy_version 1423751 (0.0005) [2023-12-27 01:44:43,771][105692] Updated weights for policy 0, policy_version 1421556 (0.0007) [2023-12-27 01:44:43,832][105692] Updated weights for policy 0, policy_version 1421566 (0.0005) [2023-12-27 01:44:43,894][105692] Updated weights for policy 0, policy_version 1421576 (0.0006) [2023-12-27 01:44:44,130][105620] Updated weights for policy 1, policy_version 1423761 (0.0009) [2023-12-27 01:44:44,204][105620] Updated weights for policy 1, policy_version 1423771 (0.0010) [2023-12-27 01:44:44,275][105620] Updated weights for policy 1, policy_version 1423781 (0.0009) [2023-12-27 01:44:44,438][105692] Updated weights for policy 0, policy_version 1421586 (0.0007) [2023-12-27 01:44:44,499][105692] Updated weights for policy 0, policy_version 1421596 (0.0009) [2023-12-27 01:44:44,547][105692] Updated weights for policy 0, policy_version 1421606 (0.0008) [2023-12-27 01:44:44,600][105692] Updated weights for policy 0, policy_version 1421616 (0.0007) [2023-12-27 01:44:45,055][105620] Updated weights for policy 1, policy_version 1423791 (0.0009) [2023-12-27 01:44:45,105][105620] Updated weights for policy 1, policy_version 1423801 (0.0009) [2023-12-27 01:44:45,166][105620] Updated weights for policy 1, policy_version 1423811 (0.0009) [2023-12-27 01:44:45,300][105692] Updated weights for policy 0, policy_version 1421626 (0.0008) [2023-12-27 01:44:45,358][105692] Updated weights for policy 0, policy_version 1421636 (0.0009) [2023-12-27 01:44:45,417][105692] Updated weights for policy 0, policy_version 1421646 (0.0009) [2023-12-27 01:44:45,991][105620] Updated weights for policy 1, policy_version 1423821 (0.0008) [2023-12-27 01:44:46,055][105620] Updated weights for policy 1, policy_version 1423831 (0.0009) [2023-12-27 01:44:46,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 728539136. Throughput: 0: 9723.3, 1: 9673.8. Samples: 728514180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:44:46,062][104569] Avg episode reward: [(0, '7610.071'), (1, '8342.710')] [2023-12-27 01:44:46,086][105692] Updated weights for policy 0, policy_version 1421656 (0.0010) [2023-12-27 01:44:46,113][105620] Updated weights for policy 1, policy_version 1423841 (0.0006) [2023-12-27 01:44:46,141][105692] Updated weights for policy 0, policy_version 1421666 (0.0007) [2023-12-27 01:44:46,151][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001423848_364552192.pth... [2023-12-27 01:44:46,156][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001422696_364257280.pth [2023-12-27 01:44:46,200][105692] Updated weights for policy 0, policy_version 1421676 (0.0008) [2023-12-27 01:44:46,224][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001421680_364003328.pth... [2023-12-27 01:44:46,228][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001420528_363708416.pth [2023-12-27 01:44:46,879][105692] Updated weights for policy 0, policy_version 1421686 (0.0007) [2023-12-27 01:44:46,881][105620] Updated weights for policy 1, policy_version 1423851 (0.0008) [2023-12-27 01:44:46,929][105620] Updated weights for policy 1, policy_version 1423861 (0.0009) [2023-12-27 01:44:46,937][105692] Updated weights for policy 0, policy_version 1421696 (0.0007) [2023-12-27 01:44:46,979][105620] Updated weights for policy 1, policy_version 1423871 (0.0008) [2023-12-27 01:44:46,982][105692] Updated weights for policy 0, policy_version 1421706 (0.0010) [2023-12-27 01:44:47,698][105692] Updated weights for policy 0, policy_version 1421716 (0.0010) [2023-12-27 01:44:47,750][105692] Updated weights for policy 0, policy_version 1421726 (0.0011) [2023-12-27 01:44:47,760][105620] Updated weights for policy 1, policy_version 1423881 (0.0008) [2023-12-27 01:44:47,801][105692] Updated weights for policy 0, policy_version 1421736 (0.0010) [2023-12-27 01:44:47,815][105620] Updated weights for policy 1, policy_version 1423891 (0.0005) [2023-12-27 01:44:47,872][105620] Updated weights for policy 1, policy_version 1423901 (0.0007) [2023-12-27 01:44:47,916][105620] Updated weights for policy 1, policy_version 1423911 (0.0008) [2023-12-27 01:44:48,564][105692] Updated weights for policy 0, policy_version 1421746 (0.0010) [2023-12-27 01:44:48,621][105692] Updated weights for policy 0, policy_version 1421756 (0.0010) [2023-12-27 01:44:48,670][105620] Updated weights for policy 1, policy_version 1423921 (0.0007) [2023-12-27 01:44:48,675][105692] Updated weights for policy 0, policy_version 1421766 (0.0010) [2023-12-27 01:44:48,731][105620] Updated weights for policy 1, policy_version 1423931 (0.0006) [2023-12-27 01:44:48,737][105692] Updated weights for policy 0, policy_version 1421776 (0.0008) [2023-12-27 01:44:48,793][105620] Updated weights for policy 1, policy_version 1423941 (0.0008) [2023-12-27 01:44:49,511][105692] Updated weights for policy 0, policy_version 1421786 (0.0009) [2023-12-27 01:44:49,551][105620] Updated weights for policy 1, policy_version 1423951 (0.0007) [2023-12-27 01:44:49,566][105692] Updated weights for policy 0, policy_version 1421796 (0.0008) [2023-12-27 01:44:49,601][105620] Updated weights for policy 1, policy_version 1423961 (0.0007) [2023-12-27 01:44:49,619][105692] Updated weights for policy 0, policy_version 1421806 (0.0007) [2023-12-27 01:44:49,657][105620] Updated weights for policy 1, policy_version 1423971 (0.0008) [2023-12-27 01:44:50,398][105620] Updated weights for policy 1, policy_version 1423981 (0.0007) [2023-12-27 01:44:50,403][105692] Updated weights for policy 0, policy_version 1421816 (0.0008) [2023-12-27 01:44:50,458][105620] Updated weights for policy 1, policy_version 1423991 (0.0006) [2023-12-27 01:44:50,470][105692] Updated weights for policy 0, policy_version 1421826 (0.0009) [2023-12-27 01:44:50,515][105620] Updated weights for policy 1, policy_version 1424001 (0.0009) [2023-12-27 01:44:50,528][105692] Updated weights for policy 0, policy_version 1421836 (0.0008) [2023-12-27 01:44:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 728637440. Throughput: 0: 9837.9, 1: 9534.5. Samples: 728628576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:44:51,062][104569] Avg episode reward: [(0, '7514.352'), (1, '8270.848')] [2023-12-27 01:44:51,235][105692] Updated weights for policy 0, policy_version 1421846 (0.0007) [2023-12-27 01:44:51,275][105620] Updated weights for policy 1, policy_version 1424011 (0.0008) [2023-12-27 01:44:51,294][105692] Updated weights for policy 0, policy_version 1421856 (0.0007) [2023-12-27 01:44:51,337][105620] Updated weights for policy 1, policy_version 1424021 (0.0007) [2023-12-27 01:44:51,351][105692] Updated weights for policy 0, policy_version 1421866 (0.0006) [2023-12-27 01:44:51,409][105620] Updated weights for policy 1, policy_version 1424031 (0.0010) [2023-12-27 01:44:52,094][105692] Updated weights for policy 0, policy_version 1421876 (0.0006) [2023-12-27 01:44:52,137][105620] Updated weights for policy 1, policy_version 1424041 (0.0010) [2023-12-27 01:44:52,153][105692] Updated weights for policy 0, policy_version 1421886 (0.0005) [2023-12-27 01:44:52,194][105620] Updated weights for policy 1, policy_version 1424051 (0.0008) [2023-12-27 01:44:52,215][105692] Updated weights for policy 0, policy_version 1421896 (0.0005) [2023-12-27 01:44:52,240][105620] Updated weights for policy 1, policy_version 1424061 (0.0008) [2023-12-27 01:44:52,300][105620] Updated weights for policy 1, policy_version 1424071 (0.0006) [2023-12-27 01:44:52,909][105692] Updated weights for policy 0, policy_version 1421906 (0.0008) [2023-12-27 01:44:52,933][105620] Updated weights for policy 1, policy_version 1424081 (0.0006) [2023-12-27 01:44:52,958][105692] Updated weights for policy 0, policy_version 1421916 (0.0007) [2023-12-27 01:44:52,997][105620] Updated weights for policy 1, policy_version 1424091 (0.0005) [2023-12-27 01:44:53,011][105692] Updated weights for policy 0, policy_version 1421926 (0.0005) [2023-12-27 01:44:53,059][105620] Updated weights for policy 1, policy_version 1424101 (0.0005) [2023-12-27 01:44:53,066][105692] Updated weights for policy 0, policy_version 1421936 (0.0005) [2023-12-27 01:44:53,595][105692] Updated weights for policy 0, policy_version 1421946 (0.0006) [2023-12-27 01:44:53,658][105692] Updated weights for policy 0, policy_version 1421956 (0.0011) [2023-12-27 01:44:53,664][105620] Updated weights for policy 1, policy_version 1424111 (0.0006) [2023-12-27 01:44:53,711][105692] Updated weights for policy 0, policy_version 1421966 (0.0011) [2023-12-27 01:44:53,717][105620] Updated weights for policy 1, policy_version 1424121 (0.0006) [2023-12-27 01:44:53,769][105620] Updated weights for policy 1, policy_version 1424131 (0.0008) [2023-12-27 01:44:54,389][105620] Updated weights for policy 1, policy_version 1424141 (0.0006) [2023-12-27 01:44:54,436][105692] Updated weights for policy 0, policy_version 1421976 (0.0010) [2023-12-27 01:44:54,449][105620] Updated weights for policy 1, policy_version 1424151 (0.0006) [2023-12-27 01:44:54,494][105692] Updated weights for policy 0, policy_version 1421986 (0.0008) [2023-12-27 01:44:54,506][105620] Updated weights for policy 1, policy_version 1424161 (0.0005) [2023-12-27 01:44:54,559][105692] Updated weights for policy 0, policy_version 1421996 (0.0009) [2023-12-27 01:44:55,059][105620] Updated weights for policy 1, policy_version 1424171 (0.0005) [2023-12-27 01:44:55,133][105620] Updated weights for policy 1, policy_version 1424181 (0.0007) [2023-12-27 01:44:55,203][105620] Updated weights for policy 1, policy_version 1424191 (0.0006) [2023-12-27 01:44:55,213][105692] Updated weights for policy 0, policy_version 1422006 (0.0008) [2023-12-27 01:44:55,270][105692] Updated weights for policy 0, policy_version 1422016 (0.0007) [2023-12-27 01:44:55,329][105692] Updated weights for policy 0, policy_version 1422026 (0.0007) [2023-12-27 01:44:55,851][105620] Updated weights for policy 1, policy_version 1424201 (0.0007) [2023-12-27 01:44:55,912][105620] Updated weights for policy 1, policy_version 1424211 (0.0010) [2023-12-27 01:44:55,961][105620] Updated weights for policy 1, policy_version 1424221 (0.0008) [2023-12-27 01:44:56,014][105620] Updated weights for policy 1, policy_version 1424231 (0.0005) [2023-12-27 01:44:56,033][105692] Updated weights for policy 0, policy_version 1422036 (0.0008) [2023-12-27 01:44:56,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 728743936. Throughput: 0: 9828.1, 1: 9665.9. Samples: 728751036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:44:56,062][104569] Avg episode reward: [(0, '7794.829'), (1, '8640.419')] [2023-12-27 01:44:56,084][105692] Updated weights for policy 0, policy_version 1422046 (0.0006) [2023-12-27 01:44:56,152][105692] Updated weights for policy 0, policy_version 1422056 (0.0005) [2023-12-27 01:44:56,700][105620] Updated weights for policy 1, policy_version 1424241 (0.0008) [2023-12-27 01:44:56,754][105620] Updated weights for policy 1, policy_version 1424251 (0.0010) [2023-12-27 01:44:56,804][105620] Updated weights for policy 1, policy_version 1424261 (0.0009) [2023-12-27 01:44:56,827][105692] Updated weights for policy 0, policy_version 1422066 (0.0008) [2023-12-27 01:44:56,878][105692] Updated weights for policy 0, policy_version 1422076 (0.0005) [2023-12-27 01:44:56,927][105692] Updated weights for policy 0, policy_version 1422086 (0.0005) [2023-12-27 01:44:56,976][105692] Updated weights for policy 0, policy_version 1422096 (0.0007) [2023-12-27 01:44:57,586][105620] Updated weights for policy 1, policy_version 1424271 (0.0008) [2023-12-27 01:44:57,631][105620] Updated weights for policy 1, policy_version 1424281 (0.0008) [2023-12-27 01:44:57,672][105692] Updated weights for policy 0, policy_version 1422106 (0.0007) [2023-12-27 01:44:57,689][105620] Updated weights for policy 1, policy_version 1424291 (0.0008) [2023-12-27 01:44:57,727][105692] Updated weights for policy 0, policy_version 1422116 (0.0006) [2023-12-27 01:44:57,777][105692] Updated weights for policy 0, policy_version 1422126 (0.0009) [2023-12-27 01:44:58,481][105692] Updated weights for policy 0, policy_version 1422136 (0.0009) [2023-12-27 01:44:58,500][105620] Updated weights for policy 1, policy_version 1424301 (0.0008) [2023-12-27 01:44:58,539][105692] Updated weights for policy 0, policy_version 1422146 (0.0008) [2023-12-27 01:44:58,566][105620] Updated weights for policy 1, policy_version 1424311 (0.0008) [2023-12-27 01:44:58,599][105692] Updated weights for policy 0, policy_version 1422156 (0.0008) [2023-12-27 01:44:58,627][105620] Updated weights for policy 1, policy_version 1424321 (0.0009) [2023-12-27 01:44:59,324][105620] Updated weights for policy 1, policy_version 1424331 (0.0008) [2023-12-27 01:44:59,387][105620] Updated weights for policy 1, policy_version 1424341 (0.0006) [2023-12-27 01:44:59,403][105692] Updated weights for policy 0, policy_version 1422166 (0.0008) [2023-12-27 01:44:59,445][105620] Updated weights for policy 1, policy_version 1424351 (0.0008) [2023-12-27 01:44:59,466][105692] Updated weights for policy 0, policy_version 1422176 (0.0006) [2023-12-27 01:44:59,480][105585] KL-divergence is very high: 127.6912 [2023-12-27 01:44:59,529][105692] Updated weights for policy 0, policy_version 1422186 (0.0007) [2023-12-27 01:44:59,530][105585] KL-divergence is very high: 155.6933 [2023-12-27 01:45:00,184][105620] Updated weights for policy 1, policy_version 1424361 (0.0007) [2023-12-27 01:45:00,232][105620] Updated weights for policy 1, policy_version 1424371 (0.0008) [2023-12-27 01:45:00,278][105620] Updated weights for policy 1, policy_version 1424381 (0.0009) [2023-12-27 01:45:00,305][105692] Updated weights for policy 0, policy_version 1422196 (0.0009) [2023-12-27 01:45:00,328][105620] Updated weights for policy 1, policy_version 1424391 (0.0007) [2023-12-27 01:45:00,360][105692] Updated weights for policy 0, policy_version 1422206 (0.0008) [2023-12-27 01:45:00,420][105692] Updated weights for policy 0, policy_version 1422216 (0.0009) [2023-12-27 01:45:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 728834048. Throughput: 0: 9803.9, 1: 9687.1. Samples: 728809532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:45:01,063][104569] Avg episode reward: [(0, '7794.805'), (1, '8718.621')] [2023-12-27 01:45:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001422224_364142592.pth... [2023-12-27 01:45:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001421104_363855872.pth [2023-12-27 01:45:01,103][105620] Updated weights for policy 1, policy_version 1424401 (0.0009) [2023-12-27 01:45:01,155][105620] Updated weights for policy 1, policy_version 1424411 (0.0008) [2023-12-27 01:45:01,206][105620] Updated weights for policy 1, policy_version 1424421 (0.0008) [2023-12-27 01:45:01,217][105692] Updated weights for policy 0, policy_version 1422226 (0.0009) [2023-12-27 01:45:01,222][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001424424_364699648.pth... [2023-12-27 01:45:01,225][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001423272_364404736.pth [2023-12-27 01:45:01,280][105692] Updated weights for policy 0, policy_version 1422236 (0.0011) [2023-12-27 01:45:01,339][105692] Updated weights for policy 0, policy_version 1422246 (0.0011) [2023-12-27 01:45:01,396][105692] Updated weights for policy 0, policy_version 1422256 (0.0009) [2023-12-27 01:45:01,972][105620] Updated weights for policy 1, policy_version 1424431 (0.0008) [2023-12-27 01:45:02,019][105620] Updated weights for policy 1, policy_version 1424441 (0.0008) [2023-12-27 01:45:02,070][105620] Updated weights for policy 1, policy_version 1424451 (0.0009) [2023-12-27 01:45:02,098][105692] Updated weights for policy 0, policy_version 1422266 (0.0011) [2023-12-27 01:45:02,150][105692] Updated weights for policy 0, policy_version 1422276 (0.0011) [2023-12-27 01:45:02,209][105692] Updated weights for policy 0, policy_version 1422286 (0.0010) [2023-12-27 01:45:02,855][105620] Updated weights for policy 1, policy_version 1424461 (0.0008) [2023-12-27 01:45:02,916][105620] Updated weights for policy 1, policy_version 1424471 (0.0008) [2023-12-27 01:45:02,964][105692] Updated weights for policy 0, policy_version 1422296 (0.0011) [2023-12-27 01:45:02,971][105620] Updated weights for policy 1, policy_version 1424481 (0.0006) [2023-12-27 01:45:03,016][105692] Updated weights for policy 0, policy_version 1422306 (0.0009) [2023-12-27 01:45:03,085][105692] Updated weights for policy 0, policy_version 1422316 (0.0007) [2023-12-27 01:45:03,708][105620] Updated weights for policy 1, policy_version 1424491 (0.0010) [2023-12-27 01:45:03,755][105620] Updated weights for policy 1, policy_version 1424501 (0.0010) [2023-12-27 01:45:03,786][105692] Updated weights for policy 0, policy_version 1422326 (0.0010) [2023-12-27 01:45:03,813][105620] Updated weights for policy 1, policy_version 1424511 (0.0010) [2023-12-27 01:45:03,839][105692] Updated weights for policy 0, policy_version 1422336 (0.0010) [2023-12-27 01:45:03,906][105692] Updated weights for policy 0, policy_version 1422346 (0.0009) [2023-12-27 01:45:04,498][105620] Updated weights for policy 1, policy_version 1424521 (0.0010) [2023-12-27 01:45:04,563][105620] Updated weights for policy 1, policy_version 1424531 (0.0006) [2023-12-27 01:45:04,621][105620] Updated weights for policy 1, policy_version 1424541 (0.0011) [2023-12-27 01:45:04,622][105692] Updated weights for policy 0, policy_version 1422356 (0.0009) [2023-12-27 01:45:04,675][105692] Updated weights for policy 0, policy_version 1422366 (0.0005) [2023-12-27 01:45:04,678][105620] Updated weights for policy 1, policy_version 1424551 (0.0011) [2023-12-27 01:45:04,725][105692] Updated weights for policy 0, policy_version 1422376 (0.0005) [2023-12-27 01:45:05,315][105620] Updated weights for policy 1, policy_version 1424561 (0.0006) [2023-12-27 01:45:05,327][105692] Updated weights for policy 0, policy_version 1422386 (0.0006) [2023-12-27 01:45:05,373][105620] Updated weights for policy 1, policy_version 1424571 (0.0005) [2023-12-27 01:45:05,390][105692] Updated weights for policy 0, policy_version 1422396 (0.0006) [2023-12-27 01:45:05,440][105620] Updated weights for policy 1, policy_version 1424581 (0.0009) [2023-12-27 01:45:05,440][105692] Updated weights for policy 0, policy_version 1422406 (0.0005) [2023-12-27 01:45:05,508][105692] Updated weights for policy 0, policy_version 1422416 (0.0006) [2023-12-27 01:45:06,046][105692] Updated weights for policy 0, policy_version 1422426 (0.0009) [2023-12-27 01:45:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 728932352. Throughput: 0: 9641.1, 1: 9618.6. Samples: 728922980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:45:06,062][104569] Avg episode reward: [(0, '7237.987'), (1, '8446.147')] [2023-12-27 01:45:06,096][105620] Updated weights for policy 1, policy_version 1424591 (0.0010) [2023-12-27 01:45:06,113][105692] Updated weights for policy 0, policy_version 1422436 (0.0010) [2023-12-27 01:45:06,163][105620] Updated weights for policy 1, policy_version 1424601 (0.0010) [2023-12-27 01:45:06,178][105692] Updated weights for policy 0, policy_version 1422446 (0.0007) [2023-12-27 01:45:06,223][105620] Updated weights for policy 1, policy_version 1424611 (0.0006) [2023-12-27 01:45:06,821][105692] Updated weights for policy 0, policy_version 1422456 (0.0010) [2023-12-27 01:45:06,873][105692] Updated weights for policy 0, policy_version 1422466 (0.0010) [2023-12-27 01:45:06,919][105620] Updated weights for policy 1, policy_version 1424621 (0.0006) [2023-12-27 01:45:06,936][105692] Updated weights for policy 0, policy_version 1422476 (0.0011) [2023-12-27 01:45:06,980][105620] Updated weights for policy 1, policy_version 1424631 (0.0008) [2023-12-27 01:45:07,036][105620] Updated weights for policy 1, policy_version 1424641 (0.0008) [2023-12-27 01:45:07,655][105620] Updated weights for policy 1, policy_version 1424651 (0.0007) [2023-12-27 01:45:07,700][105692] Updated weights for policy 0, policy_version 1422486 (0.0010) [2023-12-27 01:45:07,710][105620] Updated weights for policy 1, policy_version 1424661 (0.0007) [2023-12-27 01:45:07,752][105692] Updated weights for policy 0, policy_version 1422496 (0.0010) [2023-12-27 01:45:07,766][105620] Updated weights for policy 1, policy_version 1424671 (0.0010) [2023-12-27 01:45:07,807][105692] Updated weights for policy 0, policy_version 1422506 (0.0010) [2023-12-27 01:45:08,486][105692] Updated weights for policy 0, policy_version 1422516 (0.0010) [2023-12-27 01:45:08,504][105620] Updated weights for policy 1, policy_version 1424681 (0.0009) [2023-12-27 01:45:08,542][105692] Updated weights for policy 0, policy_version 1422526 (0.0011) [2023-12-27 01:45:08,563][105620] Updated weights for policy 1, policy_version 1424691 (0.0011) [2023-12-27 01:45:08,601][105692] Updated weights for policy 0, policy_version 1422536 (0.0011) [2023-12-27 01:45:08,622][105620] Updated weights for policy 1, policy_version 1424701 (0.0011) [2023-12-27 01:45:08,684][105620] Updated weights for policy 1, policy_version 1424711 (0.0010) [2023-12-27 01:45:09,340][105692] Updated weights for policy 0, policy_version 1422546 (0.0010) [2023-12-27 01:45:09,407][105692] Updated weights for policy 0, policy_version 1422556 (0.0010) [2023-12-27 01:45:09,432][105620] Updated weights for policy 1, policy_version 1424721 (0.0008) [2023-12-27 01:45:09,470][105692] Updated weights for policy 0, policy_version 1422566 (0.0010) [2023-12-27 01:45:09,499][105620] Updated weights for policy 1, policy_version 1424731 (0.0008) [2023-12-27 01:45:09,534][105692] Updated weights for policy 0, policy_version 1422576 (0.0011) [2023-12-27 01:45:09,558][105620] Updated weights for policy 1, policy_version 1424741 (0.0008) [2023-12-27 01:45:10,228][105620] Updated weights for policy 1, policy_version 1424751 (0.0010) [2023-12-27 01:45:10,282][105620] Updated weights for policy 1, policy_version 1424761 (0.0010) [2023-12-27 01:45:10,337][105620] Updated weights for policy 1, policy_version 1424771 (0.0007) [2023-12-27 01:45:10,351][105692] Updated weights for policy 0, policy_version 1422586 (0.0009) [2023-12-27 01:45:10,402][105692] Updated weights for policy 0, policy_version 1422596 (0.0009) [2023-12-27 01:45:10,461][105692] Updated weights for policy 0, policy_version 1422606 (0.0009) [2023-12-27 01:45:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 729030656. Throughput: 0: 9706.8, 1: 9642.0. Samples: 729042092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:45:11,062][104569] Avg episode reward: [(0, '7245.239'), (1, '8357.713')] [2023-12-27 01:45:11,113][105620] Updated weights for policy 1, policy_version 1424781 (0.0008) [2023-12-27 01:45:11,184][105620] Updated weights for policy 1, policy_version 1424791 (0.0009) [2023-12-27 01:45:11,193][105692] Updated weights for policy 0, policy_version 1422616 (0.0009) [2023-12-27 01:45:11,239][105620] Updated weights for policy 1, policy_version 1424801 (0.0006) [2023-12-27 01:45:11,254][105692] Updated weights for policy 0, policy_version 1422626 (0.0007) [2023-12-27 01:45:11,319][105692] Updated weights for policy 0, policy_version 1422636 (0.0009) [2023-12-27 01:45:12,073][105620] Updated weights for policy 1, policy_version 1424811 (0.0007) [2023-12-27 01:45:12,074][105692] Updated weights for policy 0, policy_version 1422646 (0.0009) [2023-12-27 01:45:12,127][105620] Updated weights for policy 1, policy_version 1424821 (0.0006) [2023-12-27 01:45:12,136][105692] Updated weights for policy 0, policy_version 1422656 (0.0009) [2023-12-27 01:45:12,186][105620] Updated weights for policy 1, policy_version 1424831 (0.0008) [2023-12-27 01:45:12,198][105692] Updated weights for policy 0, policy_version 1422666 (0.0007) [2023-12-27 01:45:12,954][105692] Updated weights for policy 0, policy_version 1422676 (0.0006) [2023-12-27 01:45:12,956][105620] Updated weights for policy 1, policy_version 1424841 (0.0007) [2023-12-27 01:45:13,012][105620] Updated weights for policy 1, policy_version 1424851 (0.0008) [2023-12-27 01:45:13,021][105692] Updated weights for policy 0, policy_version 1422686 (0.0005) [2023-12-27 01:45:13,064][105620] Updated weights for policy 1, policy_version 1424861 (0.0008) [2023-12-27 01:45:13,086][105692] Updated weights for policy 0, policy_version 1422696 (0.0005) [2023-12-27 01:45:13,117][105620] Updated weights for policy 1, policy_version 1424871 (0.0008) [2023-12-27 01:45:13,798][105620] Updated weights for policy 1, policy_version 1424881 (0.0005) [2023-12-27 01:45:13,852][105620] Updated weights for policy 1, policy_version 1424891 (0.0005) [2023-12-27 01:45:13,859][105692] Updated weights for policy 0, policy_version 1422706 (0.0007) [2023-12-27 01:45:13,909][105692] Updated weights for policy 0, policy_version 1422716 (0.0009) [2023-12-27 01:45:13,911][105620] Updated weights for policy 1, policy_version 1424901 (0.0005) [2023-12-27 01:45:13,966][105692] Updated weights for policy 0, policy_version 1422726 (0.0009) [2023-12-27 01:45:14,026][105692] Updated weights for policy 0, policy_version 1422736 (0.0008) [2023-12-27 01:45:14,509][105620] Updated weights for policy 1, policy_version 1424911 (0.0005) [2023-12-27 01:45:14,574][105620] Updated weights for policy 1, policy_version 1424921 (0.0009) [2023-12-27 01:45:14,630][105620] Updated weights for policy 1, policy_version 1424931 (0.0009) [2023-12-27 01:45:14,835][105692] Updated weights for policy 0, policy_version 1422746 (0.0009) [2023-12-27 01:45:14,899][105692] Updated weights for policy 0, policy_version 1422756 (0.0010) [2023-12-27 01:45:14,956][105692] Updated weights for policy 0, policy_version 1422766 (0.0009) [2023-12-27 01:45:15,329][105620] Updated weights for policy 1, policy_version 1424941 (0.0009) [2023-12-27 01:45:15,381][105620] Updated weights for policy 1, policy_version 1424951 (0.0009) [2023-12-27 01:45:15,429][105620] Updated weights for policy 1, policy_version 1424961 (0.0009) [2023-12-27 01:45:15,748][105692] Updated weights for policy 0, policy_version 1422776 (0.0010) [2023-12-27 01:45:15,807][105692] Updated weights for policy 0, policy_version 1422786 (0.0010) [2023-12-27 01:45:15,861][105692] Updated weights for policy 0, policy_version 1422796 (0.0010) [2023-12-27 01:45:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 729128960. Throughput: 0: 9676.8, 1: 9663.9. Samples: 729098908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:45:16,063][104569] Avg episode reward: [(0, '8079.840'), (1, '8502.569')] [2023-12-27 01:45:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001424968_364838912.pth... [2023-12-27 01:45:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001422800_364290048.pth... [2023-12-27 01:45:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001423848_364552192.pth [2023-12-27 01:45:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001421680_364003328.pth [2023-12-27 01:45:16,118][105620] Updated weights for policy 1, policy_version 1424971 (0.0009) [2023-12-27 01:45:16,175][105620] Updated weights for policy 1, policy_version 1424981 (0.0007) [2023-12-27 01:45:16,235][105620] Updated weights for policy 1, policy_version 1424991 (0.0008) [2023-12-27 01:45:16,576][105692] Updated weights for policy 0, policy_version 1422806 (0.0009) [2023-12-27 01:45:16,627][105692] Updated weights for policy 0, policy_version 1422816 (0.0009) [2023-12-27 01:45:16,679][105692] Updated weights for policy 0, policy_version 1422826 (0.0009) [2023-12-27 01:45:16,981][105620] Updated weights for policy 1, policy_version 1425001 (0.0009) [2023-12-27 01:45:17,031][105620] Updated weights for policy 1, policy_version 1425011 (0.0009) [2023-12-27 01:45:17,078][105620] Updated weights for policy 1, policy_version 1425021 (0.0009) [2023-12-27 01:45:17,158][105620] Updated weights for policy 1, policy_version 1425031 (0.0009) [2023-12-27 01:45:17,453][105692] Updated weights for policy 0, policy_version 1422836 (0.0008) [2023-12-27 01:45:17,504][105692] Updated weights for policy 0, policy_version 1422846 (0.0009) [2023-12-27 01:45:17,555][105692] Updated weights for policy 0, policy_version 1422856 (0.0009) [2023-12-27 01:45:17,909][105620] Updated weights for policy 1, policy_version 1425041 (0.0010) [2023-12-27 01:45:17,960][105620] Updated weights for policy 1, policy_version 1425051 (0.0009) [2023-12-27 01:45:18,006][105620] Updated weights for policy 1, policy_version 1425061 (0.0008) [2023-12-27 01:45:18,321][105692] Updated weights for policy 0, policy_version 1422866 (0.0009) [2023-12-27 01:45:18,389][105692] Updated weights for policy 0, policy_version 1422876 (0.0008) [2023-12-27 01:45:18,449][105692] Updated weights for policy 0, policy_version 1422886 (0.0009) [2023-12-27 01:45:18,509][105692] Updated weights for policy 0, policy_version 1422896 (0.0009) [2023-12-27 01:45:18,758][105620] Updated weights for policy 1, policy_version 1425071 (0.0007) [2023-12-27 01:45:18,818][105620] Updated weights for policy 1, policy_version 1425081 (0.0005) [2023-12-27 01:45:18,881][105620] Updated weights for policy 1, policy_version 1425091 (0.0008) [2023-12-27 01:45:19,290][105692] Updated weights for policy 0, policy_version 1422906 (0.0006) [2023-12-27 01:45:19,361][105692] Updated weights for policy 0, policy_version 1422916 (0.0006) [2023-12-27 01:45:19,429][105692] Updated weights for policy 0, policy_version 1422926 (0.0007) [2023-12-27 01:45:19,615][105620] Updated weights for policy 1, policy_version 1425101 (0.0009) [2023-12-27 01:45:19,670][105620] Updated weights for policy 1, policy_version 1425111 (0.0010) [2023-12-27 01:45:19,724][105620] Updated weights for policy 1, policy_version 1425121 (0.0008) [2023-12-27 01:45:20,083][105692] Updated weights for policy 0, policy_version 1422936 (0.0009) [2023-12-27 01:45:20,140][105692] Updated weights for policy 0, policy_version 1422946 (0.0009) [2023-12-27 01:45:20,196][105692] Updated weights for policy 0, policy_version 1422956 (0.0009) [2023-12-27 01:45:20,483][105620] Updated weights for policy 1, policy_version 1425131 (0.0009) [2023-12-27 01:45:20,541][105620] Updated weights for policy 1, policy_version 1425141 (0.0009) [2023-12-27 01:45:20,604][105620] Updated weights for policy 1, policy_version 1425151 (0.0008) [2023-12-27 01:45:20,995][105692] Updated weights for policy 0, policy_version 1422966 (0.0009) [2023-12-27 01:45:21,057][105692] Updated weights for policy 0, policy_version 1422976 (0.0008) [2023-12-27 01:45:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 729219072. Throughput: 0: 9656.2, 1: 9631.8. Samples: 729211424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:45:21,062][104569] Avg episode reward: [(0, '7986.147'), (1, '5360.655')] [2023-12-27 01:45:21,119][105692] Updated weights for policy 0, policy_version 1422986 (0.0010) [2023-12-27 01:45:21,342][105620] Updated weights for policy 1, policy_version 1425161 (0.0010) [2023-12-27 01:45:21,411][105620] Updated weights for policy 1, policy_version 1425171 (0.0009) [2023-12-27 01:45:21,444][105586] KL-divergence is very high: 108.9140 [2023-12-27 01:45:21,473][105620] Updated weights for policy 1, policy_version 1425181 (0.0006) [2023-12-27 01:45:21,491][105586] KL-divergence is very high: 106.1418 [2023-12-27 01:45:21,535][105620] Updated weights for policy 1, policy_version 1425191 (0.0006) [2023-12-27 01:45:21,964][105692] Updated weights for policy 0, policy_version 1422996 (0.0008) [2023-12-27 01:45:22,015][105692] Updated weights for policy 0, policy_version 1423006 (0.0008) [2023-12-27 01:45:22,071][105692] Updated weights for policy 0, policy_version 1423016 (0.0009) [2023-12-27 01:45:22,241][105620] Updated weights for policy 1, policy_version 1425201 (0.0009) [2023-12-27 01:45:22,299][105620] Updated weights for policy 1, policy_version 1425211 (0.0009) [2023-12-27 01:45:22,359][105620] Updated weights for policy 1, policy_version 1425221 (0.0008) [2023-12-27 01:45:22,868][105692] Updated weights for policy 0, policy_version 1423026 (0.0009) [2023-12-27 01:45:22,932][105692] Updated weights for policy 0, policy_version 1423036 (0.0009) [2023-12-27 01:45:22,987][105692] Updated weights for policy 0, policy_version 1423046 (0.0009) [2023-12-27 01:45:23,028][105620] Updated weights for policy 1, policy_version 1425231 (0.0006) [2023-12-27 01:45:23,042][105692] Updated weights for policy 0, policy_version 1423056 (0.0009) [2023-12-27 01:45:23,084][105620] Updated weights for policy 1, policy_version 1425241 (0.0008) [2023-12-27 01:45:23,145][105620] Updated weights for policy 1, policy_version 1425251 (0.0009) [2023-12-27 01:45:23,822][105692] Updated weights for policy 0, policy_version 1423066 (0.0009) [2023-12-27 01:45:23,865][105620] Updated weights for policy 1, policy_version 1425261 (0.0007) [2023-12-27 01:45:23,880][105692] Updated weights for policy 0, policy_version 1423077 (0.0009) [2023-12-27 01:45:23,920][105620] Updated weights for policy 1, policy_version 1425271 (0.0006) [2023-12-27 01:45:23,931][105692] Updated weights for policy 0, policy_version 1423087 (0.0009) [2023-12-27 01:45:23,978][105620] Updated weights for policy 1, policy_version 1425281 (0.0007) [2023-12-27 01:45:24,535][105620] Updated weights for policy 1, policy_version 1425291 (0.0009) [2023-12-27 01:45:24,603][105620] Updated weights for policy 1, policy_version 1425301 (0.0005) [2023-12-27 01:45:24,648][105620] Updated weights for policy 1, policy_version 1425311 (0.0005) [2023-12-27 01:45:24,826][105692] Updated weights for policy 0, policy_version 1423097 (0.0009) [2023-12-27 01:45:24,888][105692] Updated weights for policy 0, policy_version 1423107 (0.0008) [2023-12-27 01:45:24,947][105692] Updated weights for policy 0, policy_version 1423117 (0.0008) [2023-12-27 01:45:25,233][105620] Updated weights for policy 1, policy_version 1425321 (0.0006) [2023-12-27 01:45:25,300][105620] Updated weights for policy 1, policy_version 1425331 (0.0006) [2023-12-27 01:45:25,362][105620] Updated weights for policy 1, policy_version 1425341 (0.0005) [2023-12-27 01:45:25,423][105620] Updated weights for policy 1, policy_version 1425351 (0.0005) [2023-12-27 01:45:25,751][105692] Updated weights for policy 0, policy_version 1423127 (0.0009) [2023-12-27 01:45:25,805][105692] Updated weights for policy 0, policy_version 1423139 (0.0011) [2023-12-27 01:45:25,861][105692] Updated weights for policy 0, policy_version 1423149 (0.0009) [2023-12-27 01:45:25,934][105620] Updated weights for policy 1, policy_version 1425361 (0.0005) [2023-12-27 01:45:25,988][105620] Updated weights for policy 1, policy_version 1425371 (0.0005) [2023-12-27 01:45:26,049][105620] Updated weights for policy 1, policy_version 1425381 (0.0005) [2023-12-27 01:45:26,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 729317376. Throughput: 0: 9533.6, 1: 9806.5. Samples: 729327304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:45:26,063][104569] Avg episode reward: [(0, '7616.614'), (1, '3576.334')] [2023-12-27 01:45:26,570][105620] Updated weights for policy 1, policy_version 1425391 (0.0005) [2023-12-27 01:45:26,622][105620] Updated weights for policy 1, policy_version 1425401 (0.0005) [2023-12-27 01:45:26,634][105692] Updated weights for policy 0, policy_version 1423159 (0.0006) [2023-12-27 01:45:26,675][105620] Updated weights for policy 1, policy_version 1425411 (0.0005) [2023-12-27 01:45:26,680][105692] Updated weights for policy 0, policy_version 1423169 (0.0005) [2023-12-27 01:45:26,728][105692] Updated weights for policy 0, policy_version 1423179 (0.0005) [2023-12-27 01:45:27,280][105620] Updated weights for policy 1, policy_version 1425421 (0.0007) [2023-12-27 01:45:27,308][105692] Updated weights for policy 0, policy_version 1423189 (0.0006) [2023-12-27 01:45:27,340][105620] Updated weights for policy 1, policy_version 1425431 (0.0008) [2023-12-27 01:45:27,366][105692] Updated weights for policy 0, policy_version 1423199 (0.0009) [2023-12-27 01:45:27,391][105620] Updated weights for policy 1, policy_version 1425441 (0.0005) [2023-12-27 01:45:27,430][105692] Updated weights for policy 0, policy_version 1423209 (0.0009) [2023-12-27 01:45:27,942][105620] Updated weights for policy 1, policy_version 1425451 (0.0005) [2023-12-27 01:45:28,005][105620] Updated weights for policy 1, policy_version 1425461 (0.0005) [2023-12-27 01:45:28,059][105620] Updated weights for policy 1, policy_version 1425471 (0.0005) [2023-12-27 01:45:28,125][105692] Updated weights for policy 0, policy_version 1423219 (0.0009) [2023-12-27 01:45:28,186][105692] Updated weights for policy 0, policy_version 1423229 (0.0006) [2023-12-27 01:45:28,243][105692] Updated weights for policy 0, policy_version 1423239 (0.0008) [2023-12-27 01:45:28,728][105620] Updated weights for policy 1, policy_version 1425481 (0.0005) [2023-12-27 01:45:28,778][105620] Updated weights for policy 1, policy_version 1425491 (0.0007) [2023-12-27 01:45:28,825][105620] Updated weights for policy 1, policy_version 1425501 (0.0008) [2023-12-27 01:45:28,849][105692] Updated weights for policy 0, policy_version 1423249 (0.0008) [2023-12-27 01:45:28,882][105620] Updated weights for policy 1, policy_version 1425511 (0.0008) [2023-12-27 01:45:28,905][105692] Updated weights for policy 0, policy_version 1423259 (0.0007) [2023-12-27 01:45:28,952][105692] Updated weights for policy 0, policy_version 1423269 (0.0009) [2023-12-27 01:45:29,010][105692] Updated weights for policy 0, policy_version 1423279 (0.0009) [2023-12-27 01:45:29,688][105620] Updated weights for policy 1, policy_version 1425521 (0.0009) [2023-12-27 01:45:29,720][105692] Updated weights for policy 0, policy_version 1423289 (0.0008) [2023-12-27 01:45:29,751][105620] Updated weights for policy 1, policy_version 1425531 (0.0008) [2023-12-27 01:45:29,778][105692] Updated weights for policy 0, policy_version 1423299 (0.0006) [2023-12-27 01:45:29,813][105620] Updated weights for policy 1, policy_version 1425541 (0.0009) [2023-12-27 01:45:29,843][105692] Updated weights for policy 0, policy_version 1423309 (0.0007) [2023-12-27 01:45:30,438][105692] Updated weights for policy 0, policy_version 1423319 (0.0007) [2023-12-27 01:45:30,492][105692] Updated weights for policy 0, policy_version 1423329 (0.0009) [2023-12-27 01:45:30,543][105692] Updated weights for policy 0, policy_version 1423339 (0.0009) [2023-12-27 01:45:30,584][105620] Updated weights for policy 1, policy_version 1425551 (0.0008) [2023-12-27 01:45:30,647][105620] Updated weights for policy 1, policy_version 1425561 (0.0009) [2023-12-27 01:45:30,709][105620] Updated weights for policy 1, policy_version 1425571 (0.0009) [2023-12-27 01:45:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 729423872. Throughput: 0: 9627.3, 1: 9902.7. Samples: 729393028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:45:31,062][104569] Avg episode reward: [(0, '6965.866'), (1, '7167.275')] [2023-12-27 01:45:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001425576_364994560.pth... [2023-12-27 01:45:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001423344_364429312.pth... [2023-12-27 01:45:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001424424_364699648.pth [2023-12-27 01:45:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001422224_364142592.pth [2023-12-27 01:45:31,320][105692] Updated weights for policy 0, policy_version 1423349 (0.0008) [2023-12-27 01:45:31,364][105692] Updated weights for policy 0, policy_version 1423359 (0.0008) [2023-12-27 01:45:31,400][105620] Updated weights for policy 1, policy_version 1425581 (0.0007) [2023-12-27 01:45:31,423][105692] Updated weights for policy 0, policy_version 1423369 (0.0006) [2023-12-27 01:45:31,455][105620] Updated weights for policy 1, policy_version 1425591 (0.0009) [2023-12-27 01:45:31,505][105620] Updated weights for policy 1, policy_version 1425601 (0.0009) [2023-12-27 01:45:32,142][105692] Updated weights for policy 0, policy_version 1423379 (0.0006) [2023-12-27 01:45:32,193][105692] Updated weights for policy 0, policy_version 1423389 (0.0009) [2023-12-27 01:45:32,249][105692] Updated weights for policy 0, policy_version 1423399 (0.0009) [2023-12-27 01:45:32,282][105620] Updated weights for policy 1, policy_version 1425611 (0.0009) [2023-12-27 01:45:32,344][105620] Updated weights for policy 1, policy_version 1425621 (0.0008) [2023-12-27 01:45:32,409][105620] Updated weights for policy 1, policy_version 1425631 (0.0009) [2023-12-27 01:45:32,868][105692] Updated weights for policy 0, policy_version 1423409 (0.0007) [2023-12-27 01:45:32,918][105692] Updated weights for policy 0, policy_version 1423419 (0.0005) [2023-12-27 01:45:32,968][105692] Updated weights for policy 0, policy_version 1423429 (0.0005) [2023-12-27 01:45:33,017][105692] Updated weights for policy 0, policy_version 1423439 (0.0005) [2023-12-27 01:45:33,197][105620] Updated weights for policy 1, policy_version 1425641 (0.0009) [2023-12-27 01:45:33,254][105620] Updated weights for policy 1, policy_version 1425651 (0.0009) [2023-12-27 01:45:33,302][105620] Updated weights for policy 1, policy_version 1425661 (0.0008) [2023-12-27 01:45:33,352][105620] Updated weights for policy 1, policy_version 1425671 (0.0009) [2023-12-27 01:45:33,649][105692] Updated weights for policy 0, policy_version 1423449 (0.0008) [2023-12-27 01:45:33,695][105692] Updated weights for policy 0, policy_version 1423459 (0.0009) [2023-12-27 01:45:33,742][105692] Updated weights for policy 0, policy_version 1423469 (0.0008) [2023-12-27 01:45:34,077][105620] Updated weights for policy 1, policy_version 1425681 (0.0009) [2023-12-27 01:45:34,138][105620] Updated weights for policy 1, policy_version 1425691 (0.0009) [2023-12-27 01:45:34,196][105620] Updated weights for policy 1, policy_version 1425701 (0.0008) [2023-12-27 01:45:34,514][105692] Updated weights for policy 0, policy_version 1423479 (0.0010) [2023-12-27 01:45:34,574][105692] Updated weights for policy 0, policy_version 1423489 (0.0009) [2023-12-27 01:45:34,631][105692] Updated weights for policy 0, policy_version 1423499 (0.0008) [2023-12-27 01:45:35,036][105620] Updated weights for policy 1, policy_version 1425711 (0.0010) [2023-12-27 01:45:35,093][105620] Updated weights for policy 1, policy_version 1425721 (0.0009) [2023-12-27 01:45:35,147][105620] Updated weights for policy 1, policy_version 1425731 (0.0009) [2023-12-27 01:45:35,205][105692] Updated weights for policy 0, policy_version 1423509 (0.0007) [2023-12-27 01:45:35,252][105692] Updated weights for policy 0, policy_version 1423519 (0.0009) [2023-12-27 01:45:35,299][105692] Updated weights for policy 0, policy_version 1423529 (0.0008) [2023-12-27 01:45:35,904][105620] Updated weights for policy 1, policy_version 1425741 (0.0009) [2023-12-27 01:45:35,958][105620] Updated weights for policy 1, policy_version 1425751 (0.0008) [2023-12-27 01:45:36,010][105620] Updated weights for policy 1, policy_version 1425761 (0.0009) [2023-12-27 01:45:36,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.4, 300 sec: 19605.3). Total num frames: 729522176. Throughput: 0: 9648.9, 1: 9921.4. Samples: 729509240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:45:36,062][104569] Avg episode reward: [(0, '7332.586'), (1, '8166.221')] [2023-12-27 01:45:36,077][105692] Updated weights for policy 0, policy_version 1423539 (0.0009) [2023-12-27 01:45:36,142][105692] Updated weights for policy 0, policy_version 1423549 (0.0007) [2023-12-27 01:45:36,211][105692] Updated weights for policy 0, policy_version 1423559 (0.0006) [2023-12-27 01:45:36,812][105620] Updated weights for policy 1, policy_version 1425771 (0.0009) [2023-12-27 01:45:36,875][105620] Updated weights for policy 1, policy_version 1425781 (0.0009) [2023-12-27 01:45:36,934][105620] Updated weights for policy 1, policy_version 1425791 (0.0009) [2023-12-27 01:45:36,946][105692] Updated weights for policy 0, policy_version 1423569 (0.0007) [2023-12-27 01:45:37,005][105692] Updated weights for policy 0, policy_version 1423579 (0.0009) [2023-12-27 01:45:37,064][105692] Updated weights for policy 0, policy_version 1423589 (0.0009) [2023-12-27 01:45:37,119][105692] Updated weights for policy 0, policy_version 1423599 (0.0008) [2023-12-27 01:45:37,611][105620] Updated weights for policy 1, policy_version 1425801 (0.0008) [2023-12-27 01:45:37,666][105620] Updated weights for policy 1, policy_version 1425811 (0.0011) [2023-12-27 01:45:37,728][105620] Updated weights for policy 1, policy_version 1425821 (0.0010) [2023-12-27 01:45:37,784][105620] Updated weights for policy 1, policy_version 1425831 (0.0011) [2023-12-27 01:45:37,942][105692] Updated weights for policy 0, policy_version 1423609 (0.0008) [2023-12-27 01:45:37,986][105692] Updated weights for policy 0, policy_version 1423619 (0.0008) [2023-12-27 01:45:38,033][105692] Updated weights for policy 0, policy_version 1423629 (0.0006) [2023-12-27 01:45:38,497][105620] Updated weights for policy 1, policy_version 1425841 (0.0011) [2023-12-27 01:45:38,554][105620] Updated weights for policy 1, policy_version 1425851 (0.0011) [2023-12-27 01:45:38,619][105620] Updated weights for policy 1, policy_version 1425861 (0.0011) [2023-12-27 01:45:38,819][105692] Updated weights for policy 0, policy_version 1423639 (0.0008) [2023-12-27 01:45:38,875][105692] Updated weights for policy 0, policy_version 1423649 (0.0009) [2023-12-27 01:45:38,927][105692] Updated weights for policy 0, policy_version 1423659 (0.0009) [2023-12-27 01:45:39,313][105620] Updated weights for policy 1, policy_version 1425871 (0.0007) [2023-12-27 01:45:39,385][105620] Updated weights for policy 1, policy_version 1425881 (0.0008) [2023-12-27 01:45:39,457][105620] Updated weights for policy 1, policy_version 1425891 (0.0008) [2023-12-27 01:45:39,813][105692] Updated weights for policy 0, policy_version 1423669 (0.0009) [2023-12-27 01:45:39,874][105692] Updated weights for policy 0, policy_version 1423679 (0.0008) [2023-12-27 01:45:39,935][105692] Updated weights for policy 0, policy_version 1423689 (0.0009) [2023-12-27 01:45:40,096][105620] Updated weights for policy 1, policy_version 1425901 (0.0010) [2023-12-27 01:45:40,160][105620] Updated weights for policy 1, policy_version 1425911 (0.0009) [2023-12-27 01:45:40,223][105620] Updated weights for policy 1, policy_version 1425921 (0.0009) [2023-12-27 01:45:40,718][105692] Updated weights for policy 0, policy_version 1423699 (0.0009) [2023-12-27 01:45:40,776][105692] Updated weights for policy 0, policy_version 1423709 (0.0009) [2023-12-27 01:45:40,830][105692] Updated weights for policy 0, policy_version 1423719 (0.0008) [2023-12-27 01:45:41,004][105620] Updated weights for policy 1, policy_version 1425931 (0.0009) [2023-12-27 01:45:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 729612288. Throughput: 0: 9555.3, 1: 9809.2. Samples: 729622436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:45:41,062][104569] Avg episode reward: [(0, '7883.129'), (1, '8351.600')] [2023-12-27 01:45:41,074][105620] Updated weights for policy 1, policy_version 1425941 (0.0011) [2023-12-27 01:45:41,136][105620] Updated weights for policy 1, policy_version 1425951 (0.0011) [2023-12-27 01:45:41,639][105692] Updated weights for policy 0, policy_version 1423729 (0.0009) [2023-12-27 01:45:41,700][105692] Updated weights for policy 0, policy_version 1423739 (0.0008) [2023-12-27 01:45:41,763][105692] Updated weights for policy 0, policy_version 1423749 (0.0009) [2023-12-27 01:45:41,816][105692] Updated weights for policy 0, policy_version 1423759 (0.0008) [2023-12-27 01:45:41,940][105620] Updated weights for policy 1, policy_version 1425961 (0.0011) [2023-12-27 01:45:42,002][105620] Updated weights for policy 1, policy_version 1425971 (0.0010) [2023-12-27 01:45:42,078][105620] Updated weights for policy 1, policy_version 1425981 (0.0010) [2023-12-27 01:45:42,145][105620] Updated weights for policy 1, policy_version 1425991 (0.0007) [2023-12-27 01:45:42,577][105692] Updated weights for policy 0, policy_version 1423769 (0.0006) [2023-12-27 01:45:42,636][105692] Updated weights for policy 0, policy_version 1423779 (0.0008) [2023-12-27 01:45:42,692][105692] Updated weights for policy 0, policy_version 1423789 (0.0007) [2023-12-27 01:45:42,837][105620] Updated weights for policy 1, policy_version 1426001 (0.0010) [2023-12-27 01:45:42,903][105620] Updated weights for policy 1, policy_version 1426011 (0.0010) [2023-12-27 01:45:42,955][105620] Updated weights for policy 1, policy_version 1426021 (0.0010) [2023-12-27 01:45:43,290][105692] Updated weights for policy 0, policy_version 1423799 (0.0005) [2023-12-27 01:45:43,346][105692] Updated weights for policy 0, policy_version 1423809 (0.0006) [2023-12-27 01:45:43,399][105692] Updated weights for policy 0, policy_version 1423819 (0.0009) [2023-12-27 01:45:43,703][105620] Updated weights for policy 1, policy_version 1426031 (0.0007) [2023-12-27 01:45:43,757][105620] Updated weights for policy 1, policy_version 1426041 (0.0005) [2023-12-27 01:45:43,824][105620] Updated weights for policy 1, policy_version 1426051 (0.0005) [2023-12-27 01:45:44,223][105692] Updated weights for policy 0, policy_version 1423829 (0.0009) [2023-12-27 01:45:44,279][105692] Updated weights for policy 0, policy_version 1423839 (0.0009) [2023-12-27 01:45:44,330][105692] Updated weights for policy 0, policy_version 1423849 (0.0007) [2023-12-27 01:45:44,332][105620] Updated weights for policy 1, policy_version 1426061 (0.0006) [2023-12-27 01:45:44,385][105620] Updated weights for policy 1, policy_version 1426071 (0.0006) [2023-12-27 01:45:44,447][105620] Updated weights for policy 1, policy_version 1426081 (0.0009) [2023-12-27 01:45:45,093][105692] Updated weights for policy 0, policy_version 1423859 (0.0008) [2023-12-27 01:45:45,163][105692] Updated weights for policy 0, policy_version 1423869 (0.0011) [2023-12-27 01:45:45,173][105620] Updated weights for policy 1, policy_version 1426091 (0.0008) [2023-12-27 01:45:45,227][105692] Updated weights for policy 0, policy_version 1423879 (0.0011) [2023-12-27 01:45:45,238][105620] Updated weights for policy 1, policy_version 1426101 (0.0008) [2023-12-27 01:45:45,306][105620] Updated weights for policy 1, policy_version 1426111 (0.0008) [2023-12-27 01:45:45,858][105620] Updated weights for policy 1, policy_version 1426121 (0.0008) [2023-12-27 01:45:45,913][105620] Updated weights for policy 1, policy_version 1426131 (0.0011) [2023-12-27 01:45:45,968][105620] Updated weights for policy 1, policy_version 1426141 (0.0010) [2023-12-27 01:45:45,974][105692] Updated weights for policy 0, policy_version 1423889 (0.0011) [2023-12-27 01:45:46,018][105620] Updated weights for policy 1, policy_version 1426151 (0.0010) [2023-12-27 01:45:46,023][105692] Updated weights for policy 0, policy_version 1423899 (0.0011) [2023-12-27 01:45:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 729710592. Throughput: 0: 9524.8, 1: 9817.4. Samples: 729679932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:45:46,063][104569] Avg episode reward: [(0, '8158.357'), (1, '8350.320')] [2023-12-27 01:45:46,068][105692] Updated weights for policy 0, policy_version 1423909 (0.0010) [2023-12-27 01:45:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001426152_365142016.pth... [2023-12-27 01:45:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001424968_364838912.pth [2023-12-27 01:45:46,128][105692] Updated weights for policy 0, policy_version 1423919 (0.0011) [2023-12-27 01:45:46,132][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001423920_364576768.pth... [2023-12-27 01:45:46,135][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001422800_364290048.pth [2023-12-27 01:45:46,787][105620] Updated weights for policy 1, policy_version 1426161 (0.0008) [2023-12-27 01:45:46,842][105620] Updated weights for policy 1, policy_version 1426171 (0.0008) [2023-12-27 01:45:46,860][105692] Updated weights for policy 0, policy_version 1423929 (0.0006) [2023-12-27 01:45:46,902][105692] Updated weights for policy 0, policy_version 1423939 (0.0007) [2023-12-27 01:45:46,907][105620] Updated weights for policy 1, policy_version 1426181 (0.0009) [2023-12-27 01:45:46,951][105692] Updated weights for policy 0, policy_version 1423949 (0.0006) [2023-12-27 01:45:47,660][105620] Updated weights for policy 1, policy_version 1426191 (0.0010) [2023-12-27 01:45:47,662][105692] Updated weights for policy 0, policy_version 1423959 (0.0006) [2023-12-27 01:45:47,716][105692] Updated weights for policy 0, policy_version 1423969 (0.0006) [2023-12-27 01:45:47,722][105620] Updated weights for policy 1, policy_version 1426201 (0.0010) [2023-12-27 01:45:47,782][105692] Updated weights for policy 0, policy_version 1423979 (0.0005) [2023-12-27 01:45:47,788][105620] Updated weights for policy 1, policy_version 1426211 (0.0010) [2023-12-27 01:45:48,425][105620] Updated weights for policy 1, policy_version 1426221 (0.0008) [2023-12-27 01:45:48,479][105620] Updated weights for policy 1, policy_version 1426231 (0.0005) [2023-12-27 01:45:48,541][105620] Updated weights for policy 1, policy_version 1426241 (0.0010) [2023-12-27 01:45:48,587][105692] Updated weights for policy 0, policy_version 1423989 (0.0005) [2023-12-27 01:45:48,636][105692] Updated weights for policy 0, policy_version 1423999 (0.0008) [2023-12-27 01:45:48,680][105692] Updated weights for policy 0, policy_version 1424009 (0.0008) [2023-12-27 01:45:48,708][105585] KL-divergence is very high: 107.4228 [2023-12-27 01:45:49,150][105620] Updated weights for policy 1, policy_version 1426251 (0.0009) [2023-12-27 01:45:49,204][105620] Updated weights for policy 1, policy_version 1426261 (0.0005) [2023-12-27 01:45:49,272][105620] Updated weights for policy 1, policy_version 1426271 (0.0007) [2023-12-27 01:45:49,544][105692] Updated weights for policy 0, policy_version 1424019 (0.0008) [2023-12-27 01:45:49,598][105692] Updated weights for policy 0, policy_version 1424029 (0.0009) [2023-12-27 01:45:49,654][105692] Updated weights for policy 0, policy_version 1424039 (0.0009) [2023-12-27 01:45:49,690][105585] KL-divergence is very high: 104.4183 [2023-12-27 01:45:50,005][105620] Updated weights for policy 1, policy_version 1426281 (0.0009) [2023-12-27 01:45:50,056][105620] Updated weights for policy 1, policy_version 1426291 (0.0009) [2023-12-27 01:45:50,111][105620] Updated weights for policy 1, policy_version 1426301 (0.0009) [2023-12-27 01:45:50,169][105620] Updated weights for policy 1, policy_version 1426311 (0.0009) [2023-12-27 01:45:50,494][105692] Updated weights for policy 0, policy_version 1424049 (0.0009) [2023-12-27 01:45:50,548][105692] Updated weights for policy 0, policy_version 1424059 (0.0010) [2023-12-27 01:45:50,609][105692] Updated weights for policy 0, policy_version 1424069 (0.0009) [2023-12-27 01:45:50,671][105692] Updated weights for policy 0, policy_version 1424079 (0.0009) [2023-12-27 01:45:50,816][105620] Updated weights for policy 1, policy_version 1426321 (0.0007) [2023-12-27 01:45:50,883][105620] Updated weights for policy 1, policy_version 1426331 (0.0006) [2023-12-27 01:45:50,940][105620] Updated weights for policy 1, policy_version 1426341 (0.0005) [2023-12-27 01:45:51,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 729808896. Throughput: 0: 9499.6, 1: 9903.8. Samples: 729796140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:45:51,063][104569] Avg episode reward: [(0, '7430.433'), (1, '8364.125')] [2023-12-27 01:45:51,472][105692] Updated weights for policy 0, policy_version 1424089 (0.0010) [2023-12-27 01:45:51,528][105692] Updated weights for policy 0, policy_version 1424099 (0.0010) [2023-12-27 01:45:51,587][105692] Updated weights for policy 0, policy_version 1424109 (0.0011) [2023-12-27 01:45:51,636][105620] Updated weights for policy 1, policy_version 1426351 (0.0007) [2023-12-27 01:45:51,702][105620] Updated weights for policy 1, policy_version 1426361 (0.0006) [2023-12-27 01:45:51,774][105620] Updated weights for policy 1, policy_version 1426371 (0.0009) [2023-12-27 01:45:52,316][105692] Updated weights for policy 0, policy_version 1424119 (0.0010) [2023-12-27 01:45:52,380][105692] Updated weights for policy 0, policy_version 1424129 (0.0011) [2023-12-27 01:45:52,413][105620] Updated weights for policy 1, policy_version 1426381 (0.0009) [2023-12-27 01:45:52,440][105692] Updated weights for policy 0, policy_version 1424139 (0.0011) [2023-12-27 01:45:52,470][105620] Updated weights for policy 1, policy_version 1426391 (0.0006) [2023-12-27 01:45:52,537][105620] Updated weights for policy 1, policy_version 1426401 (0.0008) [2023-12-27 01:45:53,189][105692] Updated weights for policy 0, policy_version 1424149 (0.0010) [2023-12-27 01:45:53,246][105692] Updated weights for policy 0, policy_version 1424159 (0.0010) [2023-12-27 01:45:53,293][105620] Updated weights for policy 1, policy_version 1426411 (0.0008) [2023-12-27 01:45:53,304][105692] Updated weights for policy 0, policy_version 1424169 (0.0010) [2023-12-27 01:45:53,355][105620] Updated weights for policy 1, policy_version 1426421 (0.0010) [2023-12-27 01:45:53,410][105620] Updated weights for policy 1, policy_version 1426431 (0.0008) [2023-12-27 01:45:54,050][105692] Updated weights for policy 0, policy_version 1424179 (0.0011) [2023-12-27 01:45:54,101][105692] Updated weights for policy 0, policy_version 1424189 (0.0010) [2023-12-27 01:45:54,142][105620] Updated weights for policy 1, policy_version 1426441 (0.0007) [2023-12-27 01:45:54,166][105692] Updated weights for policy 0, policy_version 1424199 (0.0011) [2023-12-27 01:45:54,204][105620] Updated weights for policy 1, policy_version 1426451 (0.0005) [2023-12-27 01:45:54,269][105620] Updated weights for policy 1, policy_version 1426461 (0.0008) [2023-12-27 01:45:54,317][105620] Updated weights for policy 1, policy_version 1426471 (0.0008) [2023-12-27 01:45:54,878][105692] Updated weights for policy 0, policy_version 1424209 (0.0010) [2023-12-27 01:45:54,939][105692] Updated weights for policy 0, policy_version 1424219 (0.0008) [2023-12-27 01:45:54,983][105620] Updated weights for policy 1, policy_version 1426481 (0.0010) [2023-12-27 01:45:55,004][105692] Updated weights for policy 0, policy_version 1424229 (0.0008) [2023-12-27 01:45:55,032][105620] Updated weights for policy 1, policy_version 1426491 (0.0010) [2023-12-27 01:45:55,065][105692] Updated weights for policy 0, policy_version 1424239 (0.0006) [2023-12-27 01:45:55,088][105620] Updated weights for policy 1, policy_version 1426501 (0.0009) [2023-12-27 01:45:55,693][105692] Updated weights for policy 0, policy_version 1424249 (0.0006) [2023-12-27 01:45:55,757][105692] Updated weights for policy 0, policy_version 1424259 (0.0006) [2023-12-27 01:45:55,801][105620] Updated weights for policy 1, policy_version 1426511 (0.0007) [2023-12-27 01:45:55,823][105692] Updated weights for policy 0, policy_version 1424269 (0.0005) [2023-12-27 01:45:55,864][105620] Updated weights for policy 1, policy_version 1426521 (0.0010) [2023-12-27 01:45:55,931][105620] Updated weights for policy 1, policy_version 1426531 (0.0009) [2023-12-27 01:45:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 729907200. Throughput: 0: 9417.7, 1: 9908.4. Samples: 729911768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:45:56,062][104569] Avg episode reward: [(0, '7796.524'), (1, '9004.888')] [2023-12-27 01:45:56,469][105692] Updated weights for policy 0, policy_version 1424279 (0.0009) [2023-12-27 01:45:56,524][105692] Updated weights for policy 0, policy_version 1424289 (0.0010) [2023-12-27 01:45:56,577][105692] Updated weights for policy 0, policy_version 1424299 (0.0010) [2023-12-27 01:45:56,578][105620] Updated weights for policy 1, policy_version 1426541 (0.0010) [2023-12-27 01:45:56,637][105620] Updated weights for policy 1, policy_version 1426551 (0.0010) [2023-12-27 01:45:56,692][105620] Updated weights for policy 1, policy_version 1426561 (0.0010) [2023-12-27 01:45:57,328][105692] Updated weights for policy 0, policy_version 1424309 (0.0010) [2023-12-27 01:45:57,386][105692] Updated weights for policy 0, policy_version 1424319 (0.0009) [2023-12-27 01:45:57,386][105620] Updated weights for policy 1, policy_version 1426571 (0.0009) [2023-12-27 01:45:57,436][105692] Updated weights for policy 0, policy_version 1424329 (0.0008) [2023-12-27 01:45:57,446][105620] Updated weights for policy 1, policy_version 1426581 (0.0007) [2023-12-27 01:45:57,494][105620] Updated weights for policy 1, policy_version 1426591 (0.0008) [2023-12-27 01:45:58,106][105692] Updated weights for policy 0, policy_version 1424339 (0.0007) [2023-12-27 01:45:58,135][105620] Updated weights for policy 1, policy_version 1426601 (0.0006) [2023-12-27 01:45:58,171][105692] Updated weights for policy 0, policy_version 1424349 (0.0010) [2023-12-27 01:45:58,194][105620] Updated weights for policy 1, policy_version 1426611 (0.0009) [2023-12-27 01:45:58,238][105692] Updated weights for policy 0, policy_version 1424359 (0.0008) [2023-12-27 01:45:58,256][105620] Updated weights for policy 1, policy_version 1426621 (0.0006) [2023-12-27 01:45:58,322][105620] Updated weights for policy 1, policy_version 1426631 (0.0007) [2023-12-27 01:45:59,060][105692] Updated weights for policy 0, policy_version 1424369 (0.0008) [2023-12-27 01:45:59,119][105692] Updated weights for policy 0, policy_version 1424379 (0.0006) [2023-12-27 01:45:59,179][105620] Updated weights for policy 1, policy_version 1426641 (0.0011) [2023-12-27 01:45:59,186][105692] Updated weights for policy 0, policy_version 1424389 (0.0007) [2023-12-27 01:45:59,257][105620] Updated weights for policy 1, policy_version 1426651 (0.0008) [2023-12-27 01:45:59,260][105692] Updated weights for policy 0, policy_version 1424399 (0.0008) [2023-12-27 01:45:59,324][105620] Updated weights for policy 1, policy_version 1426661 (0.0008) [2023-12-27 01:46:00,057][105692] Updated weights for policy 0, policy_version 1424409 (0.0006) [2023-12-27 01:46:00,095][105620] Updated weights for policy 1, policy_version 1426671 (0.0007) [2023-12-27 01:46:00,121][105692] Updated weights for policy 0, policy_version 1424419 (0.0006) [2023-12-27 01:46:00,164][105620] Updated weights for policy 1, policy_version 1426681 (0.0005) [2023-12-27 01:46:00,181][105692] Updated weights for policy 0, policy_version 1424429 (0.0007) [2023-12-27 01:46:00,229][105620] Updated weights for policy 1, policy_version 1426691 (0.0007) [2023-12-27 01:46:00,807][105692] Updated weights for policy 0, policy_version 1424439 (0.0009) [2023-12-27 01:46:00,827][105620] Updated weights for policy 1, policy_version 1426701 (0.0007) [2023-12-27 01:46:00,858][105692] Updated weights for policy 0, policy_version 1424449 (0.0011) [2023-12-27 01:46:00,893][105620] Updated weights for policy 1, policy_version 1426711 (0.0008) [2023-12-27 01:46:00,927][105692] Updated weights for policy 0, policy_version 1424459 (0.0010) [2023-12-27 01:46:00,960][105620] Updated weights for policy 1, policy_version 1426721 (0.0008) [2023-12-27 01:46:01,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 730005504. Throughput: 0: 9428.2, 1: 9945.2. Samples: 729970712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:46:01,065][104569] Avg episode reward: [(0, '7879.182'), (1, '8350.790')] [2023-12-27 01:46:01,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001426728_365289472.pth... [2023-12-27 01:46:01,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001424464_364716032.pth... [2023-12-27 01:46:01,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001425576_364994560.pth [2023-12-27 01:46:01,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001423344_364429312.pth [2023-12-27 01:46:01,658][105620] Updated weights for policy 1, policy_version 1426731 (0.0008) [2023-12-27 01:46:01,712][105692] Updated weights for policy 0, policy_version 1424469 (0.0008) [2023-12-27 01:46:01,730][105620] Updated weights for policy 1, policy_version 1426741 (0.0008) [2023-12-27 01:46:01,779][105692] Updated weights for policy 0, policy_version 1424479 (0.0009) [2023-12-27 01:46:01,789][105620] Updated weights for policy 1, policy_version 1426751 (0.0008) [2023-12-27 01:46:01,840][105692] Updated weights for policy 0, policy_version 1424489 (0.0006) [2023-12-27 01:46:02,526][105620] Updated weights for policy 1, policy_version 1426761 (0.0009) [2023-12-27 01:46:02,601][105620] Updated weights for policy 1, policy_version 1426771 (0.0007) [2023-12-27 01:46:02,657][105620] Updated weights for policy 1, policy_version 1426781 (0.0008) [2023-12-27 01:46:02,658][105692] Updated weights for policy 0, policy_version 1424499 (0.0008) [2023-12-27 01:46:02,715][105620] Updated weights for policy 1, policy_version 1426791 (0.0007) [2023-12-27 01:46:02,717][105692] Updated weights for policy 0, policy_version 1424509 (0.0007) [2023-12-27 01:46:02,783][105692] Updated weights for policy 0, policy_version 1424519 (0.0009) [2023-12-27 01:46:03,425][105620] Updated weights for policy 1, policy_version 1426801 (0.0008) [2023-12-27 01:46:03,487][105620] Updated weights for policy 1, policy_version 1426811 (0.0009) [2023-12-27 01:46:03,530][105692] Updated weights for policy 0, policy_version 1424529 (0.0009) [2023-12-27 01:46:03,551][105620] Updated weights for policy 1, policy_version 1426821 (0.0009) [2023-12-27 01:46:03,592][105692] Updated weights for policy 0, policy_version 1424539 (0.0008) [2023-12-27 01:46:03,658][105692] Updated weights for policy 0, policy_version 1424549 (0.0008) [2023-12-27 01:46:03,716][105692] Updated weights for policy 0, policy_version 1424559 (0.0008) [2023-12-27 01:46:04,342][105620] Updated weights for policy 1, policy_version 1426831 (0.0008) [2023-12-27 01:46:04,401][105620] Updated weights for policy 1, policy_version 1426841 (0.0008) [2023-12-27 01:46:04,463][105692] Updated weights for policy 0, policy_version 1424569 (0.0007) [2023-12-27 01:46:04,464][105620] Updated weights for policy 1, policy_version 1426851 (0.0008) [2023-12-27 01:46:04,527][105692] Updated weights for policy 0, policy_version 1424579 (0.0010) [2023-12-27 01:46:04,591][105692] Updated weights for policy 0, policy_version 1424589 (0.0010) [2023-12-27 01:46:05,235][105620] Updated weights for policy 1, policy_version 1426861 (0.0008) [2023-12-27 01:46:05,287][105692] Updated weights for policy 0, policy_version 1424599 (0.0007) [2023-12-27 01:46:05,299][105620] Updated weights for policy 1, policy_version 1426871 (0.0009) [2023-12-27 01:46:05,353][105692] Updated weights for policy 0, policy_version 1424609 (0.0006) [2023-12-27 01:46:05,361][105620] Updated weights for policy 1, policy_version 1426881 (0.0008) [2023-12-27 01:46:05,420][105692] Updated weights for policy 0, policy_version 1424619 (0.0005) [2023-12-27 01:46:05,927][105692] Updated weights for policy 0, policy_version 1424629 (0.0006) [2023-12-27 01:46:05,975][105692] Updated weights for policy 0, policy_version 1424639 (0.0009) [2023-12-27 01:46:06,027][105692] Updated weights for policy 0, policy_version 1424649 (0.0009) [2023-12-27 01:46:06,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 730087424. Throughput: 0: 9421.9, 1: 9898.3. Samples: 730080832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:46:06,062][104569] Avg episode reward: [(0, '7790.288'), (1, '8259.294')] [2023-12-27 01:46:06,216][105620] Updated weights for policy 1, policy_version 1426891 (0.0008) [2023-12-27 01:46:06,281][105620] Updated weights for policy 1, policy_version 1426901 (0.0008) [2023-12-27 01:46:06,347][105620] Updated weights for policy 1, policy_version 1426911 (0.0009) [2023-12-27 01:46:06,766][105692] Updated weights for policy 0, policy_version 1424659 (0.0007) [2023-12-27 01:46:06,825][105692] Updated weights for policy 0, policy_version 1424669 (0.0006) [2023-12-27 01:46:06,888][105692] Updated weights for policy 0, policy_version 1424679 (0.0006) [2023-12-27 01:46:07,179][105620] Updated weights for policy 1, policy_version 1426921 (0.0008) [2023-12-27 01:46:07,240][105620] Updated weights for policy 1, policy_version 1426931 (0.0009) [2023-12-27 01:46:07,298][105620] Updated weights for policy 1, policy_version 1426941 (0.0010) [2023-12-27 01:46:07,357][105620] Updated weights for policy 1, policy_version 1426951 (0.0010) [2023-12-27 01:46:07,488][105692] Updated weights for policy 0, policy_version 1424689 (0.0006) [2023-12-27 01:46:07,544][105692] Updated weights for policy 0, policy_version 1424699 (0.0009) [2023-12-27 01:46:07,609][105692] Updated weights for policy 0, policy_version 1424710 (0.0009) [2023-12-27 01:46:07,659][105692] Updated weights for policy 0, policy_version 1424720 (0.0009) [2023-12-27 01:46:08,115][105620] Updated weights for policy 1, policy_version 1426961 (0.0008) [2023-12-27 01:46:08,163][105620] Updated weights for policy 1, policy_version 1426971 (0.0008) [2023-12-27 01:46:08,214][105620] Updated weights for policy 1, policy_version 1426981 (0.0008) [2023-12-27 01:46:08,449][105692] Updated weights for policy 0, policy_version 1424730 (0.0011) [2023-12-27 01:46:08,515][105692] Updated weights for policy 0, policy_version 1424740 (0.0010) [2023-12-27 01:46:08,575][105692] Updated weights for policy 0, policy_version 1424750 (0.0010) [2023-12-27 01:46:08,961][105620] Updated weights for policy 1, policy_version 1426991 (0.0009) [2023-12-27 01:46:09,021][105620] Updated weights for policy 1, policy_version 1427001 (0.0009) [2023-12-27 01:46:09,080][105620] Updated weights for policy 1, policy_version 1427011 (0.0008) [2023-12-27 01:46:09,267][105692] Updated weights for policy 0, policy_version 1424760 (0.0009) [2023-12-27 01:46:09,332][105692] Updated weights for policy 0, policy_version 1424770 (0.0008) [2023-12-27 01:46:09,396][105692] Updated weights for policy 0, policy_version 1424780 (0.0008) [2023-12-27 01:46:09,875][105620] Updated weights for policy 1, policy_version 1427021 (0.0009) [2023-12-27 01:46:09,942][105620] Updated weights for policy 1, policy_version 1427031 (0.0008) [2023-12-27 01:46:09,993][105620] Updated weights for policy 1, policy_version 1427041 (0.0006) [2023-12-27 01:46:10,144][105692] Updated weights for policy 0, policy_version 1424790 (0.0009) [2023-12-27 01:46:10,206][105692] Updated weights for policy 0, policy_version 1424800 (0.0010) [2023-12-27 01:46:10,266][105692] Updated weights for policy 0, policy_version 1424810 (0.0009) [2023-12-27 01:46:10,676][105620] Updated weights for policy 1, policy_version 1427051 (0.0008) [2023-12-27 01:46:10,724][105620] Updated weights for policy 1, policy_version 1427061 (0.0009) [2023-12-27 01:46:10,776][105620] Updated weights for policy 1, policy_version 1427071 (0.0009) [2023-12-27 01:46:10,979][105692] Updated weights for policy 0, policy_version 1424820 (0.0008) [2023-12-27 01:46:11,050][105692] Updated weights for policy 0, policy_version 1424830 (0.0009) [2023-12-27 01:46:11,062][104569] Fps is (10 sec: 18022.7, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 730185728. Throughput: 0: 9579.9, 1: 9676.7. Samples: 730193852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:46:11,062][104569] Avg episode reward: [(0, '8160.228'), (1, '8904.538')] [2023-12-27 01:46:11,116][105692] Updated weights for policy 0, policy_version 1424840 (0.0008) [2023-12-27 01:46:11,557][105620] Updated weights for policy 1, policy_version 1427081 (0.0010) [2023-12-27 01:46:11,628][105620] Updated weights for policy 1, policy_version 1427091 (0.0009) [2023-12-27 01:46:11,695][105620] Updated weights for policy 1, policy_version 1427101 (0.0008) [2023-12-27 01:46:11,761][105620] Updated weights for policy 1, policy_version 1427111 (0.0008) [2023-12-27 01:46:11,927][105692] Updated weights for policy 0, policy_version 1424850 (0.0008) [2023-12-27 01:46:11,986][105692] Updated weights for policy 0, policy_version 1424860 (0.0009) [2023-12-27 01:46:12,050][105692] Updated weights for policy 0, policy_version 1424870 (0.0009) [2023-12-27 01:46:12,110][105692] Updated weights for policy 0, policy_version 1424880 (0.0010) [2023-12-27 01:46:12,486][105620] Updated weights for policy 1, policy_version 1427121 (0.0008) [2023-12-27 01:46:12,532][105620] Updated weights for policy 1, policy_version 1427131 (0.0008) [2023-12-27 01:46:12,587][105620] Updated weights for policy 1, policy_version 1427141 (0.0008) [2023-12-27 01:46:12,852][105692] Updated weights for policy 0, policy_version 1424890 (0.0009) [2023-12-27 01:46:12,907][105692] Updated weights for policy 0, policy_version 1424900 (0.0008) [2023-12-27 01:46:12,959][105692] Updated weights for policy 0, policy_version 1424910 (0.0009) [2023-12-27 01:46:13,350][105620] Updated weights for policy 1, policy_version 1427151 (0.0010) [2023-12-27 01:46:13,406][105620] Updated weights for policy 1, policy_version 1427161 (0.0007) [2023-12-27 01:46:13,474][105620] Updated weights for policy 1, policy_version 1427171 (0.0011) [2023-12-27 01:46:13,567][105692] Updated weights for policy 0, policy_version 1424920 (0.0008) [2023-12-27 01:46:13,625][105692] Updated weights for policy 0, policy_version 1424930 (0.0008) [2023-12-27 01:46:13,681][105692] Updated weights for policy 0, policy_version 1424940 (0.0007) [2023-12-27 01:46:14,197][105620] Updated weights for policy 1, policy_version 1427181 (0.0011) [2023-12-27 01:46:14,251][105692] Updated weights for policy 0, policy_version 1424950 (0.0006) [2023-12-27 01:46:14,260][105620] Updated weights for policy 1, policy_version 1427191 (0.0010) [2023-12-27 01:46:14,317][105620] Updated weights for policy 1, policy_version 1427201 (0.0010) [2023-12-27 01:46:14,318][105692] Updated weights for policy 0, policy_version 1424960 (0.0006) [2023-12-27 01:46:14,371][105692] Updated weights for policy 0, policy_version 1424970 (0.0006) [2023-12-27 01:46:15,028][105620] Updated weights for policy 1, policy_version 1427211 (0.0011) [2023-12-27 01:46:15,076][105620] Updated weights for policy 1, policy_version 1427221 (0.0010) [2023-12-27 01:46:15,115][105692] Updated weights for policy 0, policy_version 1424980 (0.0007) [2023-12-27 01:46:15,133][105620] Updated weights for policy 1, policy_version 1427231 (0.0011) [2023-12-27 01:46:15,167][105692] Updated weights for policy 0, policy_version 1424990 (0.0006) [2023-12-27 01:46:15,220][105692] Updated weights for policy 0, policy_version 1425000 (0.0008) [2023-12-27 01:46:15,759][105620] Updated weights for policy 1, policy_version 1427241 (0.0011) [2023-12-27 01:46:15,814][105620] Updated weights for policy 1, policy_version 1427251 (0.0010) [2023-12-27 01:46:15,875][105620] Updated weights for policy 1, policy_version 1427261 (0.0010) [2023-12-27 01:46:15,929][105620] Updated weights for policy 1, policy_version 1427271 (0.0010) [2023-12-27 01:46:16,047][105692] Updated weights for policy 0, policy_version 1425010 (0.0009) [2023-12-27 01:46:16,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19251.1, 300 sec: 19521.9). Total num frames: 730284032. Throughput: 0: 9532.7, 1: 9548.5. Samples: 730251684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:46:16,063][104569] Avg episode reward: [(0, '7337.775'), (1, '8264.671')] [2023-12-27 01:46:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001427272_365428736.pth... [2023-12-27 01:46:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001426152_365142016.pth [2023-12-27 01:46:16,107][105692] Updated weights for policy 0, policy_version 1425020 (0.0010) [2023-12-27 01:46:16,136][105585] KL-divergence is very high: 117.0687 [2023-12-27 01:46:16,166][105692] Updated weights for policy 0, policy_version 1425030 (0.0008) [2023-12-27 01:46:16,182][105585] KL-divergence is very high: 109.5504 [2023-12-27 01:46:16,223][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001425040_364863488.pth... [2023-12-27 01:46:16,225][105692] Updated weights for policy 0, policy_version 1425040 (0.0010) [2023-12-27 01:46:16,227][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001423920_364576768.pth [2023-12-27 01:46:16,622][105620] Updated weights for policy 1, policy_version 1427281 (0.0008) [2023-12-27 01:46:16,685][105620] Updated weights for policy 1, policy_version 1427291 (0.0009) [2023-12-27 01:46:16,748][105620] Updated weights for policy 1, policy_version 1427301 (0.0009) [2023-12-27 01:46:16,998][105692] Updated weights for policy 0, policy_version 1425050 (0.0008) [2023-12-27 01:46:17,045][105692] Updated weights for policy 0, policy_version 1425060 (0.0008) [2023-12-27 01:46:17,095][105692] Updated weights for policy 0, policy_version 1425070 (0.0009) [2023-12-27 01:46:17,443][105620] Updated weights for policy 1, policy_version 1427311 (0.0009) [2023-12-27 01:46:17,490][105620] Updated weights for policy 1, policy_version 1427321 (0.0008) [2023-12-27 01:46:17,537][105620] Updated weights for policy 1, policy_version 1427331 (0.0009) [2023-12-27 01:46:17,886][105692] Updated weights for policy 0, policy_version 1425080 (0.0008) [2023-12-27 01:46:17,935][105692] Updated weights for policy 0, policy_version 1425090 (0.0008) [2023-12-27 01:46:17,981][105692] Updated weights for policy 0, policy_version 1425100 (0.0008) [2023-12-27 01:46:18,358][105620] Updated weights for policy 1, policy_version 1427341 (0.0010) [2023-12-27 01:46:18,414][105620] Updated weights for policy 1, policy_version 1427351 (0.0011) [2023-12-27 01:46:18,472][105620] Updated weights for policy 1, policy_version 1427361 (0.0011) [2023-12-27 01:46:18,775][105692] Updated weights for policy 0, policy_version 1425110 (0.0006) [2023-12-27 01:46:18,822][105692] Updated weights for policy 0, policy_version 1425120 (0.0005) [2023-12-27 01:46:18,888][105692] Updated weights for policy 0, policy_version 1425130 (0.0005) [2023-12-27 01:46:19,212][105620] Updated weights for policy 1, policy_version 1427371 (0.0009) [2023-12-27 01:46:19,276][105620] Updated weights for policy 1, policy_version 1427381 (0.0008) [2023-12-27 01:46:19,334][105620] Updated weights for policy 1, policy_version 1427391 (0.0008) [2023-12-27 01:46:19,560][105692] Updated weights for policy 0, policy_version 1425140 (0.0007) [2023-12-27 01:46:19,619][105692] Updated weights for policy 0, policy_version 1425150 (0.0010) [2023-12-27 01:46:19,675][105692] Updated weights for policy 0, policy_version 1425160 (0.0011) [2023-12-27 01:46:20,138][105620] Updated weights for policy 1, policy_version 1427401 (0.0008) [2023-12-27 01:46:20,201][105620] Updated weights for policy 1, policy_version 1427411 (0.0007) [2023-12-27 01:46:20,260][105620] Updated weights for policy 1, policy_version 1427421 (0.0011) [2023-12-27 01:46:20,321][105620] Updated weights for policy 1, policy_version 1427431 (0.0011) [2023-12-27 01:46:20,441][105692] Updated weights for policy 0, policy_version 1425170 (0.0011) [2023-12-27 01:46:20,505][105692] Updated weights for policy 0, policy_version 1425180 (0.0006) [2023-12-27 01:46:20,569][105692] Updated weights for policy 0, policy_version 1425190 (0.0008) [2023-12-27 01:46:20,634][105692] Updated weights for policy 0, policy_version 1425200 (0.0009) [2023-12-27 01:46:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 730374144. Throughput: 0: 9443.6, 1: 9619.9. Samples: 730367100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:46:21,062][104569] Avg episode reward: [(0, '7340.982'), (1, '8268.443')] [2023-12-27 01:46:21,104][105620] Updated weights for policy 1, policy_version 1427441 (0.0009) [2023-12-27 01:46:21,171][105620] Updated weights for policy 1, policy_version 1427451 (0.0009) [2023-12-27 01:46:21,229][105620] Updated weights for policy 1, policy_version 1427461 (0.0008) [2023-12-27 01:46:21,303][105692] Updated weights for policy 0, policy_version 1425210 (0.0009) [2023-12-27 01:46:21,370][105692] Updated weights for policy 0, policy_version 1425220 (0.0008) [2023-12-27 01:46:21,438][105692] Updated weights for policy 0, policy_version 1425230 (0.0008) [2023-12-27 01:46:21,956][105620] Updated weights for policy 1, policy_version 1427471 (0.0008) [2023-12-27 01:46:22,025][105620] Updated weights for policy 1, policy_version 1427481 (0.0008) [2023-12-27 01:46:22,093][105620] Updated weights for policy 1, policy_version 1427491 (0.0008) [2023-12-27 01:46:22,235][105692] Updated weights for policy 0, policy_version 1425240 (0.0011) [2023-12-27 01:46:22,298][105692] Updated weights for policy 0, policy_version 1425250 (0.0011) [2023-12-27 01:46:22,358][105692] Updated weights for policy 0, policy_version 1425260 (0.0011) [2023-12-27 01:46:22,755][105620] Updated weights for policy 1, policy_version 1427501 (0.0008) [2023-12-27 01:46:22,812][105620] Updated weights for policy 1, policy_version 1427511 (0.0008) [2023-12-27 01:46:22,872][105620] Updated weights for policy 1, policy_version 1427521 (0.0008) [2023-12-27 01:46:23,092][105692] Updated weights for policy 0, policy_version 1425270 (0.0011) [2023-12-27 01:46:23,158][105692] Updated weights for policy 0, policy_version 1425280 (0.0010) [2023-12-27 01:46:23,220][105692] Updated weights for policy 0, policy_version 1425290 (0.0011) [2023-12-27 01:46:23,657][105620] Updated weights for policy 1, policy_version 1427531 (0.0007) [2023-12-27 01:46:23,709][105620] Updated weights for policy 1, policy_version 1427541 (0.0008) [2023-12-27 01:46:23,756][105620] Updated weights for policy 1, policy_version 1427551 (0.0008) [2023-12-27 01:46:23,840][105692] Updated weights for policy 0, policy_version 1425300 (0.0009) [2023-12-27 01:46:23,901][105692] Updated weights for policy 0, policy_version 1425310 (0.0010) [2023-12-27 01:46:23,964][105692] Updated weights for policy 0, policy_version 1425320 (0.0007) [2023-12-27 01:46:24,476][105620] Updated weights for policy 1, policy_version 1427561 (0.0008) [2023-12-27 01:46:24,537][105620] Updated weights for policy 1, policy_version 1427571 (0.0007) [2023-12-27 01:46:24,598][105620] Updated weights for policy 1, policy_version 1427581 (0.0007) [2023-12-27 01:46:24,650][105692] Updated weights for policy 0, policy_version 1425330 (0.0008) [2023-12-27 01:46:24,652][105620] Updated weights for policy 1, policy_version 1427591 (0.0007) [2023-12-27 01:46:24,706][105692] Updated weights for policy 0, policy_version 1425340 (0.0009) [2023-12-27 01:46:24,758][105692] Updated weights for policy 0, policy_version 1425350 (0.0009) [2023-12-27 01:46:24,812][105692] Updated weights for policy 0, policy_version 1425360 (0.0009) [2023-12-27 01:46:25,368][105620] Updated weights for policy 1, policy_version 1427601 (0.0005) [2023-12-27 01:46:25,422][105620] Updated weights for policy 1, policy_version 1427611 (0.0009) [2023-12-27 01:46:25,468][105620] Updated weights for policy 1, policy_version 1427621 (0.0008) [2023-12-27 01:46:25,579][105692] Updated weights for policy 0, policy_version 1425370 (0.0009) [2023-12-27 01:46:25,636][105692] Updated weights for policy 0, policy_version 1425380 (0.0010) [2023-12-27 01:46:25,695][105692] Updated weights for policy 0, policy_version 1425391 (0.0010) [2023-12-27 01:46:26,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 730472448. Throughput: 0: 9482.7, 1: 9620.8. Samples: 730482092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:46:26,062][104569] Avg episode reward: [(0, '8080.111'), (1, '8451.726')] [2023-12-27 01:46:26,132][105620] Updated weights for policy 1, policy_version 1427631 (0.0009) [2023-12-27 01:46:26,188][105620] Updated weights for policy 1, policy_version 1427641 (0.0007) [2023-12-27 01:46:26,249][105620] Updated weights for policy 1, policy_version 1427651 (0.0009) [2023-12-27 01:46:26,466][105692] Updated weights for policy 0, policy_version 1425401 (0.0009) [2023-12-27 01:46:26,515][105692] Updated weights for policy 0, policy_version 1425411 (0.0009) [2023-12-27 01:46:26,563][105692] Updated weights for policy 0, policy_version 1425421 (0.0009) [2023-12-27 01:46:26,957][105620] Updated weights for policy 1, policy_version 1427661 (0.0009) [2023-12-27 01:46:27,016][105620] Updated weights for policy 1, policy_version 1427671 (0.0008) [2023-12-27 01:46:27,073][105620] Updated weights for policy 1, policy_version 1427681 (0.0009) [2023-12-27 01:46:27,358][105692] Updated weights for policy 0, policy_version 1425431 (0.0009) [2023-12-27 01:46:27,405][105692] Updated weights for policy 0, policy_version 1425441 (0.0009) [2023-12-27 01:46:27,463][105692] Updated weights for policy 0, policy_version 1425451 (0.0009) [2023-12-27 01:46:27,779][105620] Updated weights for policy 1, policy_version 1427691 (0.0007) [2023-12-27 01:46:27,833][105620] Updated weights for policy 1, policy_version 1427701 (0.0006) [2023-12-27 01:46:27,891][105620] Updated weights for policy 1, policy_version 1427711 (0.0009) [2023-12-27 01:46:28,190][105692] Updated weights for policy 0, policy_version 1425461 (0.0007) [2023-12-27 01:46:28,250][105692] Updated weights for policy 0, policy_version 1425471 (0.0005) [2023-12-27 01:46:28,318][105692] Updated weights for policy 0, policy_version 1425481 (0.0005) [2023-12-27 01:46:28,659][105620] Updated weights for policy 1, policy_version 1427721 (0.0009) [2023-12-27 01:46:28,715][105620] Updated weights for policy 1, policy_version 1427731 (0.0009) [2023-12-27 01:46:28,774][105620] Updated weights for policy 1, policy_version 1427741 (0.0009) [2023-12-27 01:46:28,834][105620] Updated weights for policy 1, policy_version 1427751 (0.0009) [2023-12-27 01:46:28,972][105692] Updated weights for policy 0, policy_version 1425491 (0.0008) [2023-12-27 01:46:29,025][105692] Updated weights for policy 0, policy_version 1425501 (0.0009) [2023-12-27 01:46:29,076][105692] Updated weights for policy 0, policy_version 1425512 (0.0009) [2023-12-27 01:46:29,509][105620] Updated weights for policy 1, policy_version 1427761 (0.0009) [2023-12-27 01:46:29,560][105620] Updated weights for policy 1, policy_version 1427771 (0.0008) [2023-12-27 01:46:29,619][105620] Updated weights for policy 1, policy_version 1427781 (0.0009) [2023-12-27 01:46:29,881][105692] Updated weights for policy 0, policy_version 1425523 (0.0010) [2023-12-27 01:46:29,946][105692] Updated weights for policy 0, policy_version 1425533 (0.0009) [2023-12-27 01:46:29,999][105692] Updated weights for policy 0, policy_version 1425543 (0.0009) [2023-12-27 01:46:30,273][105620] Updated weights for policy 1, policy_version 1427791 (0.0007) [2023-12-27 01:46:30,329][105620] Updated weights for policy 1, policy_version 1427801 (0.0005) [2023-12-27 01:46:30,383][105620] Updated weights for policy 1, policy_version 1427811 (0.0006) [2023-12-27 01:46:30,899][105692] Updated weights for policy 0, policy_version 1425553 (0.0009) [2023-12-27 01:46:30,942][105620] Updated weights for policy 1, policy_version 1427821 (0.0007) [2023-12-27 01:46:30,945][105692] Updated weights for policy 0, policy_version 1425563 (0.0008) [2023-12-27 01:46:30,990][105692] Updated weights for policy 0, policy_version 1425573 (0.0008) [2023-12-27 01:46:30,999][105620] Updated weights for policy 1, policy_version 1427831 (0.0009) [2023-12-27 01:46:31,043][105692] Updated weights for policy 0, policy_version 1425583 (0.0008) [2023-12-27 01:46:31,061][105620] Updated weights for policy 1, policy_version 1427841 (0.0008) [2023-12-27 01:46:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19114.7, 300 sec: 19521.9). Total num frames: 730570752. Throughput: 0: 9467.8, 1: 9616.5. Samples: 730538724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:46:31,062][104569] Avg episode reward: [(0, '7985.908'), (1, '7883.131')] [2023-12-27 01:46:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001425584_365002752.pth... [2023-12-27 01:46:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001424464_364716032.pth [2023-12-27 01:46:31,100][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001427848_365576192.pth... [2023-12-27 01:46:31,104][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001426728_365289472.pth [2023-12-27 01:46:31,729][105620] Updated weights for policy 1, policy_version 1427851 (0.0007) [2023-12-27 01:46:31,788][105620] Updated weights for policy 1, policy_version 1427861 (0.0010) [2023-12-27 01:46:31,846][105620] Updated weights for policy 1, policy_version 1427871 (0.0007) [2023-12-27 01:46:31,868][105692] Updated weights for policy 0, policy_version 1425593 (0.0008) [2023-12-27 01:46:31,915][105692] Updated weights for policy 0, policy_version 1425603 (0.0010) [2023-12-27 01:46:31,962][105692] Updated weights for policy 0, policy_version 1425613 (0.0009) [2023-12-27 01:46:32,502][105620] Updated weights for policy 1, policy_version 1427881 (0.0008) [2023-12-27 01:46:32,553][105620] Updated weights for policy 1, policy_version 1427891 (0.0010) [2023-12-27 01:46:32,601][105620] Updated weights for policy 1, policy_version 1427901 (0.0010) [2023-12-27 01:46:32,650][105620] Updated weights for policy 1, policy_version 1427911 (0.0010) [2023-12-27 01:46:32,796][105692] Updated weights for policy 0, policy_version 1425623 (0.0008) [2023-12-27 01:46:32,850][105692] Updated weights for policy 0, policy_version 1425633 (0.0010) [2023-12-27 01:46:32,904][105692] Updated weights for policy 0, policy_version 1425643 (0.0009) [2023-12-27 01:46:33,352][105620] Updated weights for policy 1, policy_version 1427921 (0.0006) [2023-12-27 01:46:33,412][105620] Updated weights for policy 1, policy_version 1427931 (0.0005) [2023-12-27 01:46:33,468][105620] Updated weights for policy 1, policy_version 1427941 (0.0005) [2023-12-27 01:46:33,620][105692] Updated weights for policy 0, policy_version 1425653 (0.0007) [2023-12-27 01:46:33,676][105692] Updated weights for policy 0, policy_version 1425663 (0.0010) [2023-12-27 01:46:33,724][105692] Updated weights for policy 0, policy_version 1425673 (0.0010) [2023-12-27 01:46:34,003][105620] Updated weights for policy 1, policy_version 1427951 (0.0005) [2023-12-27 01:46:34,053][105620] Updated weights for policy 1, policy_version 1427961 (0.0005) [2023-12-27 01:46:34,126][105620] Updated weights for policy 1, policy_version 1427971 (0.0006) [2023-12-27 01:46:34,403][105692] Updated weights for policy 0, policy_version 1425683 (0.0010) [2023-12-27 01:46:34,470][105692] Updated weights for policy 0, policy_version 1425693 (0.0011) [2023-12-27 01:46:34,540][105692] Updated weights for policy 0, policy_version 1425703 (0.0011) [2023-12-27 01:46:34,697][105620] Updated weights for policy 1, policy_version 1427981 (0.0009) [2023-12-27 01:46:34,750][105620] Updated weights for policy 1, policy_version 1427991 (0.0008) [2023-12-27 01:46:34,812][105620] Updated weights for policy 1, policy_version 1428001 (0.0008) [2023-12-27 01:46:35,234][105692] Updated weights for policy 0, policy_version 1425713 (0.0010) [2023-12-27 01:46:35,285][105692] Updated weights for policy 0, policy_version 1425723 (0.0005) [2023-12-27 01:46:35,338][105692] Updated weights for policy 0, policy_version 1425733 (0.0010) [2023-12-27 01:46:35,393][105692] Updated weights for policy 0, policy_version 1425743 (0.0010) [2023-12-27 01:46:35,610][105620] Updated weights for policy 1, policy_version 1428011 (0.0008) [2023-12-27 01:46:35,679][105620] Updated weights for policy 1, policy_version 1428021 (0.0009) [2023-12-27 01:46:35,755][105620] Updated weights for policy 1, policy_version 1428031 (0.0010) [2023-12-27 01:46:35,980][105692] Updated weights for policy 0, policy_version 1425753 (0.0006) [2023-12-27 01:46:36,033][105692] Updated weights for policy 0, policy_version 1425763 (0.0006) [2023-12-27 01:46:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19114.7, 300 sec: 19522.0). Total num frames: 730669056. Throughput: 0: 9464.2, 1: 9703.9. Samples: 730658700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:46:36,062][104569] Avg episode reward: [(0, '7709.566'), (1, '8041.396')] [2023-12-27 01:46:36,085][105692] Updated weights for policy 0, policy_version 1425773 (0.0010) [2023-12-27 01:46:36,528][105620] Updated weights for policy 1, policy_version 1428041 (0.0010) [2023-12-27 01:46:36,588][105620] Updated weights for policy 1, policy_version 1428051 (0.0009) [2023-12-27 01:46:36,649][105620] Updated weights for policy 1, policy_version 1428061 (0.0009) [2023-12-27 01:46:36,716][105620] Updated weights for policy 1, policy_version 1428071 (0.0009) [2023-12-27 01:46:36,746][105692] Updated weights for policy 0, policy_version 1425783 (0.0008) [2023-12-27 01:46:36,805][105692] Updated weights for policy 0, policy_version 1425793 (0.0011) [2023-12-27 01:46:36,865][105692] Updated weights for policy 0, policy_version 1425803 (0.0011) [2023-12-27 01:46:37,515][105620] Updated weights for policy 1, policy_version 1428081 (0.0005) [2023-12-27 01:46:37,579][105620] Updated weights for policy 1, policy_version 1428091 (0.0008) [2023-12-27 01:46:37,615][105692] Updated weights for policy 0, policy_version 1425813 (0.0011) [2023-12-27 01:46:37,629][105620] Updated weights for policy 1, policy_version 1428101 (0.0006) [2023-12-27 01:46:37,667][105692] Updated weights for policy 0, policy_version 1425823 (0.0011) [2023-12-27 01:46:37,722][105692] Updated weights for policy 0, policy_version 1425833 (0.0011) [2023-12-27 01:46:38,268][105620] Updated weights for policy 1, policy_version 1428111 (0.0007) [2023-12-27 01:46:38,334][105620] Updated weights for policy 1, policy_version 1428121 (0.0009) [2023-12-27 01:46:38,343][105692] Updated weights for policy 0, policy_version 1425843 (0.0010) [2023-12-27 01:46:38,401][105620] Updated weights for policy 1, policy_version 1428131 (0.0009) [2023-12-27 01:46:38,403][105692] Updated weights for policy 0, policy_version 1425853 (0.0011) [2023-12-27 01:46:38,430][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000002 [2023-12-27 01:46:38,458][105692] Updated weights for policy 0, policy_version 1425863 (0.0011) [2023-12-27 01:46:38,478][105585] KL-divergence is very high: 144.9469 [2023-12-27 01:46:38,491][105585] KL-divergence is very high: 130.8209 [2023-12-27 01:46:39,116][105620] Updated weights for policy 1, policy_version 1428141 (0.0009) [2023-12-27 01:46:39,175][105620] Updated weights for policy 1, policy_version 1428151 (0.0008) [2023-12-27 01:46:39,198][105692] Updated weights for policy 0, policy_version 1425873 (0.0011) [2023-12-27 01:46:39,245][105620] Updated weights for policy 1, policy_version 1428161 (0.0009) [2023-12-27 01:46:39,265][105692] Updated weights for policy 0, policy_version 1425883 (0.0009) [2023-12-27 01:46:39,332][105692] Updated weights for policy 0, policy_version 1425893 (0.0008) [2023-12-27 01:46:39,397][105692] Updated weights for policy 0, policy_version 1425903 (0.0008) [2023-12-27 01:46:40,038][105620] Updated weights for policy 1, policy_version 1428171 (0.0007) [2023-12-27 01:46:40,096][105620] Updated weights for policy 1, policy_version 1428181 (0.0006) [2023-12-27 01:46:40,161][105620] Updated weights for policy 1, policy_version 1428191 (0.0006) [2023-12-27 01:46:40,172][105692] Updated weights for policy 0, policy_version 1425913 (0.0008) [2023-12-27 01:46:40,224][105692] Updated weights for policy 0, policy_version 1425923 (0.0006) [2023-12-27 01:46:40,292][105692] Updated weights for policy 0, policy_version 1425933 (0.0006) [2023-12-27 01:46:40,777][105620] Updated weights for policy 1, policy_version 1428201 (0.0010) [2023-12-27 01:46:40,826][105620] Updated weights for policy 1, policy_version 1428211 (0.0011) [2023-12-27 01:46:40,868][105692] Updated weights for policy 0, policy_version 1425943 (0.0007) [2023-12-27 01:46:40,879][105620] Updated weights for policy 1, policy_version 1428221 (0.0009) [2023-12-27 01:46:40,922][105692] Updated weights for policy 0, policy_version 1425953 (0.0007) [2023-12-27 01:46:40,938][105620] Updated weights for policy 1, policy_version 1428231 (0.0008) [2023-12-27 01:46:40,969][105692] Updated weights for policy 0, policy_version 1425963 (0.0009) [2023-12-27 01:46:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 730775552. Throughput: 0: 9570.8, 1: 9643.6. Samples: 730776420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:46:41,062][104569] Avg episode reward: [(0, '7800.368'), (1, '8356.941')] [2023-12-27 01:46:41,760][105620] Updated weights for policy 1, policy_version 1428241 (0.0009) [2023-12-27 01:46:41,803][105692] Updated weights for policy 0, policy_version 1425973 (0.0010) [2023-12-27 01:46:41,815][105620] Updated weights for policy 1, policy_version 1428251 (0.0009) [2023-12-27 01:46:41,857][105692] Updated weights for policy 0, policy_version 1425983 (0.0010) [2023-12-27 01:46:41,871][105620] Updated weights for policy 1, policy_version 1428261 (0.0007) [2023-12-27 01:46:41,913][105692] Updated weights for policy 0, policy_version 1425993 (0.0010) [2023-12-27 01:46:42,643][105692] Updated weights for policy 0, policy_version 1426003 (0.0009) [2023-12-27 01:46:42,662][105620] Updated weights for policy 1, policy_version 1428271 (0.0008) [2023-12-27 01:46:42,702][105692] Updated weights for policy 0, policy_version 1426013 (0.0010) [2023-12-27 01:46:42,719][105620] Updated weights for policy 1, policy_version 1428281 (0.0006) [2023-12-27 01:46:42,759][105692] Updated weights for policy 0, policy_version 1426023 (0.0008) [2023-12-27 01:46:42,770][105620] Updated weights for policy 1, policy_version 1428291 (0.0007) [2023-12-27 01:46:43,415][105620] Updated weights for policy 1, policy_version 1428301 (0.0008) [2023-12-27 01:46:43,467][105620] Updated weights for policy 1, policy_version 1428311 (0.0008) [2023-12-27 01:46:43,516][105620] Updated weights for policy 1, policy_version 1428321 (0.0009) [2023-12-27 01:46:43,530][105692] Updated weights for policy 0, policy_version 1426033 (0.0009) [2023-12-27 01:46:43,578][105692] Updated weights for policy 0, policy_version 1426043 (0.0009) [2023-12-27 01:46:43,640][105692] Updated weights for policy 0, policy_version 1426053 (0.0010) [2023-12-27 01:46:43,691][105692] Updated weights for policy 0, policy_version 1426063 (0.0010) [2023-12-27 01:46:44,098][105620] Updated weights for policy 1, policy_version 1428331 (0.0009) [2023-12-27 01:46:44,146][105620] Updated weights for policy 1, policy_version 1428341 (0.0006) [2023-12-27 01:46:44,197][105620] Updated weights for policy 1, policy_version 1428351 (0.0005) [2023-12-27 01:46:44,340][105692] Updated weights for policy 0, policy_version 1426073 (0.0009) [2023-12-27 01:46:44,400][105692] Updated weights for policy 0, policy_version 1426083 (0.0010) [2023-12-27 01:46:44,453][105692] Updated weights for policy 0, policy_version 1426093 (0.0008) [2023-12-27 01:46:44,807][105620] Updated weights for policy 1, policy_version 1428361 (0.0006) [2023-12-27 01:46:44,869][105620] Updated weights for policy 1, policy_version 1428371 (0.0009) [2023-12-27 01:46:44,935][105620] Updated weights for policy 1, policy_version 1428381 (0.0008) [2023-12-27 01:46:44,992][105620] Updated weights for policy 1, policy_version 1428391 (0.0005) [2023-12-27 01:46:45,205][105692] Updated weights for policy 0, policy_version 1426103 (0.0007) [2023-12-27 01:46:45,268][105692] Updated weights for policy 0, policy_version 1426113 (0.0008) [2023-12-27 01:46:45,330][105692] Updated weights for policy 0, policy_version 1426123 (0.0008) [2023-12-27 01:46:45,721][105620] Updated weights for policy 1, policy_version 1428401 (0.0009) [2023-12-27 01:46:45,769][105620] Updated weights for policy 1, policy_version 1428411 (0.0009) [2023-12-27 01:46:45,820][105620] Updated weights for policy 1, policy_version 1428421 (0.0009) [2023-12-27 01:46:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 730865664. Throughput: 0: 9551.5, 1: 9635.8. Samples: 730834140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:46:46,062][104569] Avg episode reward: [(0, '7887.726'), (1, '8721.171')] [2023-12-27 01:46:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001428424_365723648.pth... [2023-12-27 01:46:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001427272_365428736.pth [2023-12-27 01:46:46,101][105692] Updated weights for policy 0, policy_version 1426133 (0.0009) [2023-12-27 01:46:46,156][105692] Updated weights for policy 0, policy_version 1426145 (0.0011) [2023-12-27 01:46:46,215][105692] Updated weights for policy 0, policy_version 1426156 (0.0009) [2023-12-27 01:46:46,241][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001426160_365150208.pth... [2023-12-27 01:46:46,246][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001425040_364863488.pth [2023-12-27 01:46:46,455][105620] Updated weights for policy 1, policy_version 1428431 (0.0008) [2023-12-27 01:46:46,515][105620] Updated weights for policy 1, policy_version 1428441 (0.0008) [2023-12-27 01:46:46,573][105620] Updated weights for policy 1, policy_version 1428451 (0.0007) [2023-12-27 01:46:46,994][105692] Updated weights for policy 0, policy_version 1426166 (0.0006) [2023-12-27 01:46:47,045][105692] Updated weights for policy 0, policy_version 1426176 (0.0008) [2023-12-27 01:46:47,103][105692] Updated weights for policy 0, policy_version 1426186 (0.0010) [2023-12-27 01:46:47,317][105620] Updated weights for policy 1, policy_version 1428461 (0.0005) [2023-12-27 01:46:47,366][105620] Updated weights for policy 1, policy_version 1428471 (0.0005) [2023-12-27 01:46:47,419][105620] Updated weights for policy 1, policy_version 1428481 (0.0009) [2023-12-27 01:46:47,669][105692] Updated weights for policy 0, policy_version 1426196 (0.0008) [2023-12-27 01:46:47,720][105692] Updated weights for policy 0, policy_version 1426206 (0.0010) [2023-12-27 01:46:47,771][105692] Updated weights for policy 0, policy_version 1426216 (0.0010) [2023-12-27 01:46:48,142][105620] Updated weights for policy 1, policy_version 1428491 (0.0010) [2023-12-27 01:46:48,190][105620] Updated weights for policy 1, policy_version 1428501 (0.0010) [2023-12-27 01:46:48,244][105620] Updated weights for policy 1, policy_version 1428511 (0.0010) [2023-12-27 01:46:48,355][105692] Updated weights for policy 0, policy_version 1426226 (0.0010) [2023-12-27 01:46:48,422][105692] Updated weights for policy 0, policy_version 1426236 (0.0011) [2023-12-27 01:46:48,489][105692] Updated weights for policy 0, policy_version 1426246 (0.0011) [2023-12-27 01:46:48,559][105692] Updated weights for policy 0, policy_version 1426256 (0.0011) [2023-12-27 01:46:48,981][105620] Updated weights for policy 1, policy_version 1428521 (0.0010) [2023-12-27 01:46:49,048][105620] Updated weights for policy 1, policy_version 1428531 (0.0008) [2023-12-27 01:46:49,114][105620] Updated weights for policy 1, policy_version 1428541 (0.0011) [2023-12-27 01:46:49,172][105620] Updated weights for policy 1, policy_version 1428551 (0.0010) [2023-12-27 01:46:49,304][105692] Updated weights for policy 0, policy_version 1426266 (0.0006) [2023-12-27 01:46:49,321][105585] KL-divergence is very high: 101.0714 [2023-12-27 01:46:49,371][105692] Updated weights for policy 0, policy_version 1426276 (0.0011) [2023-12-27 01:46:49,423][105692] Updated weights for policy 0, policy_version 1426286 (0.0010) [2023-12-27 01:46:49,792][105620] Updated weights for policy 1, policy_version 1428561 (0.0008) [2023-12-27 01:46:49,858][105620] Updated weights for policy 1, policy_version 1428571 (0.0010) [2023-12-27 01:46:49,914][105620] Updated weights for policy 1, policy_version 1428581 (0.0009) [2023-12-27 01:46:50,209][105692] Updated weights for policy 0, policy_version 1426296 (0.0009) [2023-12-27 01:46:50,258][105692] Updated weights for policy 0, policy_version 1426306 (0.0009) [2023-12-27 01:46:50,306][105692] Updated weights for policy 0, policy_version 1426316 (0.0009) [2023-12-27 01:46:50,624][105620] Updated weights for policy 1, policy_version 1428591 (0.0009) [2023-12-27 01:46:50,683][105620] Updated weights for policy 1, policy_version 1428601 (0.0009) [2023-12-27 01:46:50,743][105620] Updated weights for policy 1, policy_version 1428611 (0.0009) [2023-12-27 01:46:51,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 730963968. Throughput: 0: 9681.6, 1: 9745.3. Samples: 730955044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:46:51,063][104569] Avg episode reward: [(0, '7790.646'), (1, '6433.945')] [2023-12-27 01:46:51,138][105692] Updated weights for policy 0, policy_version 1426326 (0.0009) [2023-12-27 01:46:51,203][105692] Updated weights for policy 0, policy_version 1426336 (0.0009) [2023-12-27 01:46:51,262][105692] Updated weights for policy 0, policy_version 1426346 (0.0009) [2023-12-27 01:46:51,447][105620] Updated weights for policy 1, policy_version 1428621 (0.0007) [2023-12-27 01:46:51,499][105620] Updated weights for policy 1, policy_version 1428631 (0.0005) [2023-12-27 01:46:51,554][105620] Updated weights for policy 1, policy_version 1428641 (0.0006) [2023-12-27 01:46:52,072][105692] Updated weights for policy 0, policy_version 1426356 (0.0008) [2023-12-27 01:46:52,138][105692] Updated weights for policy 0, policy_version 1426366 (0.0006) [2023-12-27 01:46:52,210][105692] Updated weights for policy 0, policy_version 1426376 (0.0006) [2023-12-27 01:46:52,260][105620] Updated weights for policy 1, policy_version 1428651 (0.0008) [2023-12-27 01:46:52,328][105620] Updated weights for policy 1, policy_version 1428661 (0.0006) [2023-12-27 01:46:52,394][105620] Updated weights for policy 1, policy_version 1428671 (0.0006) [2023-12-27 01:46:52,776][105692] Updated weights for policy 0, policy_version 1426386 (0.0008) [2023-12-27 01:46:52,843][105692] Updated weights for policy 0, policy_version 1426396 (0.0007) [2023-12-27 01:46:52,910][105692] Updated weights for policy 0, policy_version 1426406 (0.0009) [2023-12-27 01:46:52,967][105620] Updated weights for policy 1, policy_version 1428681 (0.0007) [2023-12-27 01:46:52,969][105692] Updated weights for policy 0, policy_version 1426416 (0.0011) [2023-12-27 01:46:53,022][105620] Updated weights for policy 1, policy_version 1428691 (0.0008) [2023-12-27 01:46:53,076][105620] Updated weights for policy 1, policy_version 1428701 (0.0009) [2023-12-27 01:46:53,130][105620] Updated weights for policy 1, policy_version 1428711 (0.0007) [2023-12-27 01:46:53,610][105692] Updated weights for policy 0, policy_version 1426426 (0.0009) [2023-12-27 01:46:53,668][105692] Updated weights for policy 0, policy_version 1426436 (0.0009) [2023-12-27 01:46:53,732][105692] Updated weights for policy 0, policy_version 1426446 (0.0008) [2023-12-27 01:46:53,878][105620] Updated weights for policy 1, policy_version 1428721 (0.0009) [2023-12-27 01:46:53,932][105620] Updated weights for policy 1, policy_version 1428731 (0.0009) [2023-12-27 01:46:53,980][105620] Updated weights for policy 1, policy_version 1428741 (0.0007) [2023-12-27 01:46:54,360][105692] Updated weights for policy 0, policy_version 1426456 (0.0010) [2023-12-27 01:46:54,419][105692] Updated weights for policy 0, policy_version 1426466 (0.0010) [2023-12-27 01:46:54,481][105692] Updated weights for policy 0, policy_version 1426476 (0.0010) [2023-12-27 01:46:54,586][105620] Updated weights for policy 1, policy_version 1428751 (0.0005) [2023-12-27 01:46:54,637][105620] Updated weights for policy 1, policy_version 1428761 (0.0005) [2023-12-27 01:46:54,694][105620] Updated weights for policy 1, policy_version 1428771 (0.0009) [2023-12-27 01:46:55,146][105692] Updated weights for policy 0, policy_version 1426486 (0.0007) [2023-12-27 01:46:55,214][105692] Updated weights for policy 0, policy_version 1426496 (0.0006) [2023-12-27 01:46:55,277][105692] Updated weights for policy 0, policy_version 1426506 (0.0006) [2023-12-27 01:46:55,385][105620] Updated weights for policy 1, policy_version 1428781 (0.0009) [2023-12-27 01:46:55,435][105620] Updated weights for policy 1, policy_version 1428791 (0.0010) [2023-12-27 01:46:55,499][105620] Updated weights for policy 1, policy_version 1428801 (0.0010) [2023-12-27 01:46:55,819][105692] Updated weights for policy 0, policy_version 1426516 (0.0007) [2023-12-27 01:46:55,870][105692] Updated weights for policy 0, policy_version 1426526 (0.0010) [2023-12-27 01:46:55,929][105692] Updated weights for policy 0, policy_version 1426536 (0.0010) [2023-12-27 01:46:56,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 731070464. Throughput: 0: 9694.3, 1: 9943.8. Samples: 731077568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:46:56,062][104569] Avg episode reward: [(0, '7794.424'), (1, '6988.710')] [2023-12-27 01:46:56,076][105620] Updated weights for policy 1, policy_version 1428811 (0.0009) [2023-12-27 01:46:56,132][105620] Updated weights for policy 1, policy_version 1428821 (0.0005) [2023-12-27 01:46:56,190][105620] Updated weights for policy 1, policy_version 1428831 (0.0005) [2023-12-27 01:46:56,690][105620] Updated weights for policy 1, policy_version 1428841 (0.0005) [2023-12-27 01:46:56,702][105692] Updated weights for policy 0, policy_version 1426546 (0.0010) [2023-12-27 01:46:56,754][105620] Updated weights for policy 1, policy_version 1428851 (0.0007) [2023-12-27 01:46:56,762][105692] Updated weights for policy 0, policy_version 1426556 (0.0005) [2023-12-27 01:46:56,809][105692] Updated weights for policy 0, policy_version 1426566 (0.0005) [2023-12-27 01:46:56,812][105620] Updated weights for policy 1, policy_version 1428861 (0.0010) [2023-12-27 01:46:56,859][105692] Updated weights for policy 0, policy_version 1426576 (0.0005) [2023-12-27 01:46:56,870][105620] Updated weights for policy 1, policy_version 1428871 (0.0010) [2023-12-27 01:46:57,380][105692] Updated weights for policy 0, policy_version 1426586 (0.0005) [2023-12-27 01:46:57,398][105620] Updated weights for policy 1, policy_version 1428881 (0.0006) [2023-12-27 01:46:57,432][105692] Updated weights for policy 0, policy_version 1426596 (0.0005) [2023-12-27 01:46:57,450][105620] Updated weights for policy 1, policy_version 1428891 (0.0005) [2023-12-27 01:46:57,488][105692] Updated weights for policy 0, policy_version 1426606 (0.0009) [2023-12-27 01:46:57,504][105620] Updated weights for policy 1, policy_version 1428901 (0.0005) [2023-12-27 01:46:58,083][105692] Updated weights for policy 0, policy_version 1426616 (0.0009) [2023-12-27 01:46:58,098][105620] Updated weights for policy 1, policy_version 1428911 (0.0006) [2023-12-27 01:46:58,140][105692] Updated weights for policy 0, policy_version 1426626 (0.0006) [2023-12-27 01:46:58,152][105620] Updated weights for policy 1, policy_version 1428921 (0.0006) [2023-12-27 01:46:58,202][105692] Updated weights for policy 0, policy_version 1426636 (0.0009) [2023-12-27 01:46:58,215][105620] Updated weights for policy 1, policy_version 1428931 (0.0007) [2023-12-27 01:46:58,948][105620] Updated weights for policy 1, policy_version 1428941 (0.0008) [2023-12-27 01:46:58,976][105692] Updated weights for policy 0, policy_version 1426646 (0.0010) [2023-12-27 01:46:59,007][105620] Updated weights for policy 1, policy_version 1428951 (0.0006) [2023-12-27 01:46:59,036][105692] Updated weights for policy 0, policy_version 1426656 (0.0011) [2023-12-27 01:46:59,070][105620] Updated weights for policy 1, policy_version 1428961 (0.0006) [2023-12-27 01:46:59,098][105692] Updated weights for policy 0, policy_version 1426666 (0.0011) [2023-12-27 01:46:59,739][105692] Updated weights for policy 0, policy_version 1426676 (0.0008) [2023-12-27 01:46:59,799][105692] Updated weights for policy 0, policy_version 1426686 (0.0005) [2023-12-27 01:46:59,810][105620] Updated weights for policy 1, policy_version 1428971 (0.0007) [2023-12-27 01:46:59,862][105692] Updated weights for policy 0, policy_version 1426696 (0.0009) [2023-12-27 01:46:59,870][105620] Updated weights for policy 1, policy_version 1428981 (0.0009) [2023-12-27 01:46:59,932][105620] Updated weights for policy 1, policy_version 1428991 (0.0008) [2023-12-27 01:47:00,532][105692] Updated weights for policy 0, policy_version 1426706 (0.0010) [2023-12-27 01:47:00,590][105692] Updated weights for policy 0, policy_version 1426716 (0.0010) [2023-12-27 01:47:00,652][105692] Updated weights for policy 0, policy_version 1426726 (0.0010) [2023-12-27 01:47:00,708][105692] Updated weights for policy 0, policy_version 1426736 (0.0010) [2023-12-27 01:47:00,718][105620] Updated weights for policy 1, policy_version 1429001 (0.0008) [2023-12-27 01:47:00,771][105620] Updated weights for policy 1, policy_version 1429011 (0.0008) [2023-12-27 01:47:00,831][105620] Updated weights for policy 1, policy_version 1429021 (0.0008) [2023-12-27 01:47:00,882][105620] Updated weights for policy 1, policy_version 1429031 (0.0008) [2023-12-27 01:47:01,062][104569] Fps is (10 sec: 21299.4, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 731176960. Throughput: 0: 9749.3, 1: 10084.1. Samples: 731144184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:47:01,062][104569] Avg episode reward: [(0, '8251.286'), (1, '8141.208')] [2023-12-27 01:47:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001426736_365297664.pth... [2023-12-27 01:47:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001429032_365879296.pth... [2023-12-27 01:47:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001425584_365002752.pth [2023-12-27 01:47:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001427848_365576192.pth [2023-12-27 01:47:01,484][105692] Updated weights for policy 0, policy_version 1426746 (0.0010) [2023-12-27 01:47:01,553][105692] Updated weights for policy 0, policy_version 1426756 (0.0010) [2023-12-27 01:47:01,615][105692] Updated weights for policy 0, policy_version 1426766 (0.0010) [2023-12-27 01:47:01,679][105620] Updated weights for policy 1, policy_version 1429041 (0.0009) [2023-12-27 01:47:01,744][105620] Updated weights for policy 1, policy_version 1429051 (0.0008) [2023-12-27 01:47:01,796][105620] Updated weights for policy 1, policy_version 1429061 (0.0008) [2023-12-27 01:47:02,357][105692] Updated weights for policy 0, policy_version 1426776 (0.0008) [2023-12-27 01:47:02,423][105692] Updated weights for policy 0, policy_version 1426786 (0.0009) [2023-12-27 01:47:02,480][105620] Updated weights for policy 1, policy_version 1429071 (0.0007) [2023-12-27 01:47:02,494][105692] Updated weights for policy 0, policy_version 1426796 (0.0009) [2023-12-27 01:47:02,529][105620] Updated weights for policy 1, policy_version 1429081 (0.0005) [2023-12-27 01:47:02,530][105586] KL-divergence is very high: 115.5588 [2023-12-27 01:47:02,572][105586] KL-divergence is very high: 135.8781 [2023-12-27 01:47:02,583][105620] Updated weights for policy 1, policy_version 1429091 (0.0006) [2023-12-27 01:47:03,191][105692] Updated weights for policy 0, policy_version 1426806 (0.0008) [2023-12-27 01:47:03,200][105620] Updated weights for policy 1, policy_version 1429101 (0.0007) [2023-12-27 01:47:03,242][105692] Updated weights for policy 0, policy_version 1426816 (0.0008) [2023-12-27 01:47:03,256][105620] Updated weights for policy 1, policy_version 1429111 (0.0005) [2023-12-27 01:47:03,295][105692] Updated weights for policy 0, policy_version 1426826 (0.0005) [2023-12-27 01:47:03,300][105620] Updated weights for policy 1, policy_version 1429121 (0.0005) [2023-12-27 01:47:03,871][105620] Updated weights for policy 1, policy_version 1429131 (0.0006) [2023-12-27 01:47:03,937][105620] Updated weights for policy 1, policy_version 1429141 (0.0009) [2023-12-27 01:47:03,938][105692] Updated weights for policy 0, policy_version 1426836 (0.0006) [2023-12-27 01:47:03,960][105585] KL-divergence is very high: 108.7880 [2023-12-27 01:47:03,995][105692] Updated weights for policy 0, policy_version 1426846 (0.0007) [2023-12-27 01:47:04,003][105620] Updated weights for policy 1, policy_version 1429151 (0.0007) [2023-12-27 01:47:04,008][105585] KL-divergence is very high: 108.2995 [2023-12-27 01:47:04,052][105692] Updated weights for policy 0, policy_version 1426856 (0.0008) [2023-12-27 01:47:04,053][105585] KL-divergence is very high: 109.3497 [2023-12-27 01:47:04,748][105620] Updated weights for policy 1, policy_version 1429161 (0.0010) [2023-12-27 01:47:04,766][105692] Updated weights for policy 0, policy_version 1426866 (0.0008) [2023-12-27 01:47:04,803][105620] Updated weights for policy 1, policy_version 1429171 (0.0010) [2023-12-27 01:47:04,817][105692] Updated weights for policy 0, policy_version 1426876 (0.0005) [2023-12-27 01:47:04,853][105620] Updated weights for policy 1, policy_version 1429181 (0.0007) [2023-12-27 01:47:04,879][105692] Updated weights for policy 0, policy_version 1426886 (0.0006) [2023-12-27 01:47:04,913][105620] Updated weights for policy 1, policy_version 1429191 (0.0010) [2023-12-27 01:47:04,946][105692] Updated weights for policy 0, policy_version 1426896 (0.0007) [2023-12-27 01:47:05,549][105620] Updated weights for policy 1, policy_version 1429201 (0.0007) [2023-12-27 01:47:05,607][105620] Updated weights for policy 1, policy_version 1429211 (0.0009) [2023-12-27 01:47:05,664][105620] Updated weights for policy 1, policy_version 1429221 (0.0009) [2023-12-27 01:47:05,707][105692] Updated weights for policy 0, policy_version 1426906 (0.0008) [2023-12-27 01:47:05,756][105692] Updated weights for policy 0, policy_version 1426916 (0.0009) [2023-12-27 01:47:05,816][105692] Updated weights for policy 0, policy_version 1426926 (0.0008) [2023-12-27 01:47:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 731275264. Throughput: 0: 9781.4, 1: 10104.4. Samples: 731261964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:47:06,062][104569] Avg episode reward: [(0, '8257.008'), (1, '8229.204')] [2023-12-27 01:47:06,375][105620] Updated weights for policy 1, policy_version 1429231 (0.0009) [2023-12-27 01:47:06,430][105620] Updated weights for policy 1, policy_version 1429241 (0.0009) [2023-12-27 01:47:06,485][105620] Updated weights for policy 1, policy_version 1429251 (0.0009) [2023-12-27 01:47:06,541][105692] Updated weights for policy 0, policy_version 1426936 (0.0007) [2023-12-27 01:47:06,595][105692] Updated weights for policy 0, policy_version 1426946 (0.0008) [2023-12-27 01:47:06,651][105692] Updated weights for policy 0, policy_version 1426956 (0.0008) [2023-12-27 01:47:07,270][105692] Updated weights for policy 0, policy_version 1426966 (0.0007) [2023-12-27 01:47:07,286][105620] Updated weights for policy 1, policy_version 1429261 (0.0011) [2023-12-27 01:47:07,336][105692] Updated weights for policy 0, policy_version 1426976 (0.0006) [2023-12-27 01:47:07,342][105620] Updated weights for policy 1, policy_version 1429271 (0.0011) [2023-12-27 01:47:07,398][105692] Updated weights for policy 0, policy_version 1426986 (0.0007) [2023-12-27 01:47:07,402][105620] Updated weights for policy 1, policy_version 1429281 (0.0011) [2023-12-27 01:47:08,109][105692] Updated weights for policy 0, policy_version 1426996 (0.0006) [2023-12-27 01:47:08,147][105620] Updated weights for policy 1, policy_version 1429291 (0.0010) [2023-12-27 01:47:08,167][105692] Updated weights for policy 0, policy_version 1427006 (0.0005) [2023-12-27 01:47:08,195][105620] Updated weights for policy 1, policy_version 1429301 (0.0010) [2023-12-27 01:47:08,221][105692] Updated weights for policy 0, policy_version 1427016 (0.0005) [2023-12-27 01:47:08,244][105620] Updated weights for policy 1, policy_version 1429311 (0.0010) [2023-12-27 01:47:08,886][105692] Updated weights for policy 0, policy_version 1427026 (0.0006) [2023-12-27 01:47:08,941][105692] Updated weights for policy 0, policy_version 1427036 (0.0009) [2023-12-27 01:47:08,989][105620] Updated weights for policy 1, policy_version 1429321 (0.0010) [2023-12-27 01:47:09,000][105692] Updated weights for policy 0, policy_version 1427046 (0.0009) [2023-12-27 01:47:09,053][105620] Updated weights for policy 1, policy_version 1429331 (0.0006) [2023-12-27 01:47:09,080][105692] Updated weights for policy 0, policy_version 1427056 (0.0009) [2023-12-27 01:47:09,102][105620] Updated weights for policy 1, policy_version 1429341 (0.0005) [2023-12-27 01:47:09,161][105620] Updated weights for policy 1, policy_version 1429351 (0.0006) [2023-12-27 01:47:09,819][105692] Updated weights for policy 0, policy_version 1427066 (0.0009) [2023-12-27 01:47:09,874][105620] Updated weights for policy 1, policy_version 1429361 (0.0007) [2023-12-27 01:47:09,889][105692] Updated weights for policy 0, policy_version 1427076 (0.0008) [2023-12-27 01:47:09,945][105620] Updated weights for policy 1, policy_version 1429371 (0.0007) [2023-12-27 01:47:09,956][105692] Updated weights for policy 0, policy_version 1427086 (0.0009) [2023-12-27 01:47:10,009][105620] Updated weights for policy 1, policy_version 1429381 (0.0007) [2023-12-27 01:47:10,689][105692] Updated weights for policy 0, policy_version 1427096 (0.0008) [2023-12-27 01:47:10,719][105620] Updated weights for policy 1, policy_version 1429391 (0.0009) [2023-12-27 01:47:10,739][105692] Updated weights for policy 0, policy_version 1427106 (0.0010) [2023-12-27 01:47:10,777][105620] Updated weights for policy 1, policy_version 1429401 (0.0008) [2023-12-27 01:47:10,788][105692] Updated weights for policy 0, policy_version 1427116 (0.0010) [2023-12-27 01:47:10,831][105620] Updated weights for policy 1, policy_version 1429411 (0.0006) [2023-12-27 01:47:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 731373568. Throughput: 0: 9811.2, 1: 10109.7. Samples: 731378532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:47:11,062][104569] Avg episode reward: [(0, '7886.226'), (1, '8472.308')] [2023-12-27 01:47:11,590][105692] Updated weights for policy 0, policy_version 1427126 (0.0009) [2023-12-27 01:47:11,617][105620] Updated weights for policy 1, policy_version 1429421 (0.0008) [2023-12-27 01:47:11,656][105692] Updated weights for policy 0, policy_version 1427136 (0.0011) [2023-12-27 01:47:11,681][105620] Updated weights for policy 1, policy_version 1429431 (0.0007) [2023-12-27 01:47:11,715][105692] Updated weights for policy 0, policy_version 1427146 (0.0008) [2023-12-27 01:47:11,744][105620] Updated weights for policy 1, policy_version 1429441 (0.0007) [2023-12-27 01:47:12,495][105692] Updated weights for policy 0, policy_version 1427156 (0.0008) [2023-12-27 01:47:12,515][105620] Updated weights for policy 1, policy_version 1429451 (0.0009) [2023-12-27 01:47:12,558][105692] Updated weights for policy 0, policy_version 1427166 (0.0006) [2023-12-27 01:47:12,563][105620] Updated weights for policy 1, policy_version 1429461 (0.0008) [2023-12-27 01:47:12,621][105692] Updated weights for policy 0, policy_version 1427176 (0.0006) [2023-12-27 01:47:12,623][105620] Updated weights for policy 1, policy_version 1429471 (0.0008) [2023-12-27 01:47:13,322][105692] Updated weights for policy 0, policy_version 1427186 (0.0008) [2023-12-27 01:47:13,368][105620] Updated weights for policy 1, policy_version 1429481 (0.0007) [2023-12-27 01:47:13,371][105692] Updated weights for policy 0, policy_version 1427196 (0.0010) [2023-12-27 01:47:13,423][105620] Updated weights for policy 1, policy_version 1429491 (0.0007) [2023-12-27 01:47:13,425][105692] Updated weights for policy 0, policy_version 1427206 (0.0007) [2023-12-27 01:47:13,479][105620] Updated weights for policy 1, policy_version 1429501 (0.0007) [2023-12-27 01:47:13,482][105692] Updated weights for policy 0, policy_version 1427216 (0.0005) [2023-12-27 01:47:13,537][105620] Updated weights for policy 1, policy_version 1429511 (0.0009) [2023-12-27 01:47:14,190][105692] Updated weights for policy 0, policy_version 1427226 (0.0009) [2023-12-27 01:47:14,239][105692] Updated weights for policy 0, policy_version 1427236 (0.0008) [2023-12-27 01:47:14,275][105620] Updated weights for policy 1, policy_version 1429521 (0.0007) [2023-12-27 01:47:14,297][105692] Updated weights for policy 0, policy_version 1427246 (0.0009) [2023-12-27 01:47:14,321][105620] Updated weights for policy 1, policy_version 1429531 (0.0007) [2023-12-27 01:47:14,369][105620] Updated weights for policy 1, policy_version 1429541 (0.0009) [2023-12-27 01:47:15,074][105692] Updated weights for policy 0, policy_version 1427256 (0.0008) [2023-12-27 01:47:15,122][105620] Updated weights for policy 1, policy_version 1429551 (0.0009) [2023-12-27 01:47:15,135][105692] Updated weights for policy 0, policy_version 1427266 (0.0006) [2023-12-27 01:47:15,185][105620] Updated weights for policy 1, policy_version 1429561 (0.0008) [2023-12-27 01:47:15,205][105692] Updated weights for policy 0, policy_version 1427276 (0.0006) [2023-12-27 01:47:15,248][105620] Updated weights for policy 1, policy_version 1429571 (0.0008) [2023-12-27 01:47:15,797][105692] Updated weights for policy 0, policy_version 1427286 (0.0005) [2023-12-27 01:47:15,848][105692] Updated weights for policy 0, policy_version 1427296 (0.0005) [2023-12-27 01:47:15,893][105692] Updated weights for policy 0, policy_version 1427306 (0.0005) [2023-12-27 01:47:16,051][105620] Updated weights for policy 1, policy_version 1429581 (0.0010) [2023-12-27 01:47:16,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 731463680. Throughput: 0: 9788.5, 1: 10089.6. Samples: 731433244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:47:16,063][104569] Avg episode reward: [(0, '8803.951'), (1, '8388.004')] [2023-12-27 01:47:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001427312_365445120.pth... [2023-12-27 01:47:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001426160_365150208.pth [2023-12-27 01:47:16,097][105620] Updated weights for policy 1, policy_version 1429591 (0.0009) [2023-12-27 01:47:16,145][105620] Updated weights for policy 1, policy_version 1429601 (0.0009) [2023-12-27 01:47:16,184][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001429608_366026752.pth... [2023-12-27 01:47:16,187][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001428424_365723648.pth [2023-12-27 01:47:16,497][105692] Updated weights for policy 0, policy_version 1427316 (0.0008) [2023-12-27 01:47:16,563][105692] Updated weights for policy 0, policy_version 1427326 (0.0009) [2023-12-27 01:47:16,622][105692] Updated weights for policy 0, policy_version 1427336 (0.0010) [2023-12-27 01:47:17,007][105620] Updated weights for policy 1, policy_version 1429611 (0.0009) [2023-12-27 01:47:17,054][105620] Updated weights for policy 1, policy_version 1429621 (0.0006) [2023-12-27 01:47:17,107][105620] Updated weights for policy 1, policy_version 1429631 (0.0005) [2023-12-27 01:47:17,284][105692] Updated weights for policy 0, policy_version 1427346 (0.0009) [2023-12-27 01:47:17,336][105692] Updated weights for policy 0, policy_version 1427356 (0.0008) [2023-12-27 01:47:17,385][105692] Updated weights for policy 0, policy_version 1427366 (0.0008) [2023-12-27 01:47:17,430][105692] Updated weights for policy 0, policy_version 1427376 (0.0008) [2023-12-27 01:47:17,852][105620] Updated weights for policy 1, policy_version 1429641 (0.0008) [2023-12-27 01:47:17,910][105620] Updated weights for policy 1, policy_version 1429651 (0.0010) [2023-12-27 01:47:17,972][105620] Updated weights for policy 1, policy_version 1429661 (0.0010) [2023-12-27 01:47:18,029][105620] Updated weights for policy 1, policy_version 1429671 (0.0009) [2023-12-27 01:47:18,203][105692] Updated weights for policy 0, policy_version 1427386 (0.0007) [2023-12-27 01:47:18,261][105692] Updated weights for policy 0, policy_version 1427396 (0.0005) [2023-12-27 01:47:18,328][105692] Updated weights for policy 0, policy_version 1427406 (0.0006) [2023-12-27 01:47:18,737][105620] Updated weights for policy 1, policy_version 1429681 (0.0010) [2023-12-27 01:47:18,802][105620] Updated weights for policy 1, policy_version 1429691 (0.0010) [2023-12-27 01:47:18,874][105620] Updated weights for policy 1, policy_version 1429701 (0.0008) [2023-12-27 01:47:18,929][105692] Updated weights for policy 0, policy_version 1427416 (0.0007) [2023-12-27 01:47:18,988][105692] Updated weights for policy 0, policy_version 1427426 (0.0009) [2023-12-27 01:47:19,044][105692] Updated weights for policy 0, policy_version 1427436 (0.0008) [2023-12-27 01:47:19,648][105620] Updated weights for policy 1, policy_version 1429711 (0.0010) [2023-12-27 01:47:19,722][105620] Updated weights for policy 1, policy_version 1429721 (0.0010) [2023-12-27 01:47:19,770][105692] Updated weights for policy 0, policy_version 1427446 (0.0008) [2023-12-27 01:47:19,780][105620] Updated weights for policy 1, policy_version 1429731 (0.0009) [2023-12-27 01:47:19,833][105692] Updated weights for policy 0, policy_version 1427456 (0.0008) [2023-12-27 01:47:19,903][105692] Updated weights for policy 0, policy_version 1427466 (0.0009) [2023-12-27 01:47:20,472][105620] Updated weights for policy 1, policy_version 1429741 (0.0007) [2023-12-27 01:47:20,532][105620] Updated weights for policy 1, policy_version 1429751 (0.0009) [2023-12-27 01:47:20,595][105620] Updated weights for policy 1, policy_version 1429761 (0.0009) [2023-12-27 01:47:20,698][105692] Updated weights for policy 0, policy_version 1427476 (0.0010) [2023-12-27 01:47:20,755][105692] Updated weights for policy 0, policy_version 1427486 (0.0009) [2023-12-27 01:47:20,807][105692] Updated weights for policy 0, policy_version 1427497 (0.0011) [2023-12-27 01:47:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 731561984. Throughput: 0: 9950.8, 1: 9853.4. Samples: 731549892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:47:21,063][104569] Avg episode reward: [(0, '8626.713'), (1, '8669.262')] [2023-12-27 01:47:21,351][105620] Updated weights for policy 1, policy_version 1429771 (0.0008) [2023-12-27 01:47:21,426][105620] Updated weights for policy 1, policy_version 1429781 (0.0009) [2023-12-27 01:47:21,484][105620] Updated weights for policy 1, policy_version 1429791 (0.0010) [2023-12-27 01:47:21,541][105692] Updated weights for policy 0, policy_version 1427507 (0.0009) [2023-12-27 01:47:21,607][105692] Updated weights for policy 0, policy_version 1427517 (0.0008) [2023-12-27 01:47:21,670][105692] Updated weights for policy 0, policy_version 1427527 (0.0010) [2023-12-27 01:47:22,215][105620] Updated weights for policy 1, policy_version 1429801 (0.0008) [2023-12-27 01:47:22,269][105620] Updated weights for policy 1, policy_version 1429811 (0.0009) [2023-12-27 01:47:22,316][105620] Updated weights for policy 1, policy_version 1429821 (0.0008) [2023-12-27 01:47:22,381][105620] Updated weights for policy 1, policy_version 1429831 (0.0009) [2023-12-27 01:47:22,479][105692] Updated weights for policy 0, policy_version 1427537 (0.0009) [2023-12-27 01:47:22,538][105692] Updated weights for policy 0, policy_version 1427547 (0.0008) [2023-12-27 01:47:22,598][105692] Updated weights for policy 0, policy_version 1427557 (0.0009) [2023-12-27 01:47:22,657][105692] Updated weights for policy 0, policy_version 1427567 (0.0009) [2023-12-27 01:47:23,157][105620] Updated weights for policy 1, policy_version 1429841 (0.0009) [2023-12-27 01:47:23,219][105620] Updated weights for policy 1, policy_version 1429851 (0.0009) [2023-12-27 01:47:23,281][105620] Updated weights for policy 1, policy_version 1429861 (0.0009) [2023-12-27 01:47:23,426][105692] Updated weights for policy 0, policy_version 1427577 (0.0009) [2023-12-27 01:47:23,474][105692] Updated weights for policy 0, policy_version 1427587 (0.0008) [2023-12-27 01:47:23,520][105692] Updated weights for policy 0, policy_version 1427597 (0.0005) [2023-12-27 01:47:23,983][105620] Updated weights for policy 1, policy_version 1429871 (0.0009) [2023-12-27 01:47:24,049][105620] Updated weights for policy 1, policy_version 1429881 (0.0010) [2023-12-27 01:47:24,086][105692] Updated weights for policy 0, policy_version 1427607 (0.0005) [2023-12-27 01:47:24,109][105620] Updated weights for policy 1, policy_version 1429891 (0.0010) [2023-12-27 01:47:24,142][105692] Updated weights for policy 0, policy_version 1427617 (0.0007) [2023-12-27 01:47:24,207][105692] Updated weights for policy 0, policy_version 1427627 (0.0010) [2023-12-27 01:47:24,738][105692] Updated weights for policy 0, policy_version 1427637 (0.0007) [2023-12-27 01:47:24,804][105692] Updated weights for policy 0, policy_version 1427647 (0.0005) [2023-12-27 01:47:24,878][105692] Updated weights for policy 0, policy_version 1427657 (0.0007) [2023-12-27 01:47:24,890][105620] Updated weights for policy 1, policy_version 1429901 (0.0007) [2023-12-27 01:47:24,942][105620] Updated weights for policy 1, policy_version 1429911 (0.0005) [2023-12-27 01:47:24,992][105620] Updated weights for policy 1, policy_version 1429921 (0.0007) [2023-12-27 01:47:25,473][105692] Updated weights for policy 0, policy_version 1427667 (0.0010) [2023-12-27 01:47:25,535][105692] Updated weights for policy 0, policy_version 1427677 (0.0009) [2023-12-27 01:47:25,593][105692] Updated weights for policy 0, policy_version 1427687 (0.0005) [2023-12-27 01:47:25,636][105585] KL-divergence is very high: 113.1496 [2023-12-27 01:47:25,755][105620] Updated weights for policy 1, policy_version 1429931 (0.0009) [2023-12-27 01:47:25,800][105620] Updated weights for policy 1, policy_version 1429941 (0.0008) [2023-12-27 01:47:25,848][105620] Updated weights for policy 1, policy_version 1429951 (0.0007) [2023-12-27 01:47:26,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 731660288. Throughput: 0: 9935.2, 1: 9843.4. Samples: 731666456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:47:26,062][104569] Avg episode reward: [(0, '8170.169'), (1, '8725.841')] [2023-12-27 01:47:26,231][105692] Updated weights for policy 0, policy_version 1427697 (0.0005) [2023-12-27 01:47:26,300][105692] Updated weights for policy 0, policy_version 1427707 (0.0005) [2023-12-27 01:47:26,362][105692] Updated weights for policy 0, policy_version 1427717 (0.0005) [2023-12-27 01:47:26,425][105692] Updated weights for policy 0, policy_version 1427727 (0.0005) [2023-12-27 01:47:26,719][105620] Updated weights for policy 1, policy_version 1429961 (0.0008) [2023-12-27 01:47:26,771][105620] Updated weights for policy 1, policy_version 1429971 (0.0009) [2023-12-27 01:47:26,827][105620] Updated weights for policy 1, policy_version 1429982 (0.0009) [2023-12-27 01:47:26,884][105620] Updated weights for policy 1, policy_version 1429992 (0.0009) [2023-12-27 01:47:26,898][105692] Updated weights for policy 0, policy_version 1427737 (0.0005) [2023-12-27 01:47:26,956][105692] Updated weights for policy 0, policy_version 1427747 (0.0005) [2023-12-27 01:47:27,014][105692] Updated weights for policy 0, policy_version 1427757 (0.0005) [2023-12-27 01:47:27,604][105620] Updated weights for policy 1, policy_version 1430002 (0.0006) [2023-12-27 01:47:27,619][105692] Updated weights for policy 0, policy_version 1427767 (0.0005) [2023-12-27 01:47:27,666][105620] Updated weights for policy 1, policy_version 1430012 (0.0006) [2023-12-27 01:47:27,671][105692] Updated weights for policy 0, policy_version 1427777 (0.0005) [2023-12-27 01:47:27,722][105692] Updated weights for policy 0, policy_version 1427787 (0.0006) [2023-12-27 01:47:27,723][105620] Updated weights for policy 1, policy_version 1430022 (0.0010) [2023-12-27 01:47:28,249][105692] Updated weights for policy 0, policy_version 1427797 (0.0006) [2023-12-27 01:47:28,301][105692] Updated weights for policy 0, policy_version 1427807 (0.0008) [2023-12-27 01:47:28,366][105692] Updated weights for policy 0, policy_version 1427817 (0.0008) [2023-12-27 01:47:28,410][105620] Updated weights for policy 1, policy_version 1430032 (0.0008) [2023-12-27 01:47:28,472][105620] Updated weights for policy 1, policy_version 1430042 (0.0009) [2023-12-27 01:47:28,533][105620] Updated weights for policy 1, policy_version 1430052 (0.0008) [2023-12-27 01:47:29,117][105620] Updated weights for policy 1, policy_version 1430062 (0.0006) [2023-12-27 01:47:29,166][105692] Updated weights for policy 0, policy_version 1427827 (0.0009) [2023-12-27 01:47:29,167][105620] Updated weights for policy 1, policy_version 1430072 (0.0005) [2023-12-27 01:47:29,218][105692] Updated weights for policy 0, policy_version 1427837 (0.0006) [2023-12-27 01:47:29,219][105620] Updated weights for policy 1, policy_version 1430082 (0.0010) [2023-12-27 01:47:29,281][105692] Updated weights for policy 0, policy_version 1427847 (0.0007) [2023-12-27 01:47:29,908][105620] Updated weights for policy 1, policy_version 1430092 (0.0008) [2023-12-27 01:47:29,921][105692] Updated weights for policy 0, policy_version 1427857 (0.0009) [2023-12-27 01:47:29,971][105620] Updated weights for policy 1, policy_version 1430102 (0.0006) [2023-12-27 01:47:29,984][105692] Updated weights for policy 0, policy_version 1427867 (0.0008) [2023-12-27 01:47:30,034][105620] Updated weights for policy 1, policy_version 1430112 (0.0007) [2023-12-27 01:47:30,041][105692] Updated weights for policy 0, policy_version 1427877 (0.0007) [2023-12-27 01:47:30,105][105692] Updated weights for policy 0, policy_version 1427887 (0.0006) [2023-12-27 01:47:30,681][105620] Updated weights for policy 1, policy_version 1430122 (0.0009) [2023-12-27 01:47:30,712][105692] Updated weights for policy 0, policy_version 1427897 (0.0008) [2023-12-27 01:47:30,731][105620] Updated weights for policy 1, policy_version 1430132 (0.0005) [2023-12-27 01:47:30,761][105692] Updated weights for policy 0, policy_version 1427907 (0.0009) [2023-12-27 01:47:30,784][105620] Updated weights for policy 1, policy_version 1430142 (0.0005) [2023-12-27 01:47:30,809][105692] Updated weights for policy 0, policy_version 1427917 (0.0009) [2023-12-27 01:47:30,832][105620] Updated weights for policy 1, policy_version 1430152 (0.0005) [2023-12-27 01:47:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19933.9, 300 sec: 19549.7). Total num frames: 731766784. Throughput: 0: 10076.8, 1: 9842.4. Samples: 731730504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:47:31,062][104569] Avg episode reward: [(0, '8164.493'), (1, '8345.975')] [2023-12-27 01:47:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001427920_365600768.pth... [2023-12-27 01:47:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001430152_366166016.pth... [2023-12-27 01:47:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001429032_365879296.pth [2023-12-27 01:47:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001426736_365297664.pth [2023-12-27 01:47:31,404][105620] Updated weights for policy 1, policy_version 1430162 (0.0011) [2023-12-27 01:47:31,459][105620] Updated weights for policy 1, policy_version 1430172 (0.0010) [2023-12-27 01:47:31,517][105620] Updated weights for policy 1, policy_version 1430182 (0.0010) [2023-12-27 01:47:31,528][105692] Updated weights for policy 0, policy_version 1427927 (0.0010) [2023-12-27 01:47:31,591][105692] Updated weights for policy 0, policy_version 1427937 (0.0010) [2023-12-27 01:47:31,660][105692] Updated weights for policy 0, policy_version 1427947 (0.0009) [2023-12-27 01:47:32,282][105692] Updated weights for policy 0, policy_version 1427957 (0.0009) [2023-12-27 01:47:32,300][105620] Updated weights for policy 1, policy_version 1430192 (0.0008) [2023-12-27 01:47:32,338][105692] Updated weights for policy 0, policy_version 1427967 (0.0011) [2023-12-27 01:47:32,364][105620] Updated weights for policy 1, policy_version 1430202 (0.0006) [2023-12-27 01:47:32,401][105692] Updated weights for policy 0, policy_version 1427977 (0.0008) [2023-12-27 01:47:32,427][105620] Updated weights for policy 1, policy_version 1430212 (0.0008) [2023-12-27 01:47:33,097][105692] Updated weights for policy 0, policy_version 1427987 (0.0009) [2023-12-27 01:47:33,158][105692] Updated weights for policy 0, policy_version 1427997 (0.0010) [2023-12-27 01:47:33,175][105620] Updated weights for policy 1, policy_version 1430222 (0.0009) [2023-12-27 01:47:33,213][105692] Updated weights for policy 0, policy_version 1428007 (0.0010) [2023-12-27 01:47:33,223][105620] Updated weights for policy 1, policy_version 1430232 (0.0010) [2023-12-27 01:47:33,277][105620] Updated weights for policy 1, policy_version 1430242 (0.0006) [2023-12-27 01:47:33,857][105620] Updated weights for policy 1, policy_version 1430252 (0.0005) [2023-12-27 01:47:33,904][105692] Updated weights for policy 0, policy_version 1428017 (0.0010) [2023-12-27 01:47:33,917][105620] Updated weights for policy 1, policy_version 1430262 (0.0005) [2023-12-27 01:47:33,955][105692] Updated weights for policy 0, policy_version 1428027 (0.0008) [2023-12-27 01:47:33,962][105620] Updated weights for policy 1, policy_version 1430272 (0.0005) [2023-12-27 01:47:34,004][105692] Updated weights for policy 0, policy_version 1428037 (0.0006) [2023-12-27 01:47:34,072][105692] Updated weights for policy 0, policy_version 1428047 (0.0005) [2023-12-27 01:47:34,611][105620] Updated weights for policy 1, policy_version 1430282 (0.0006) [2023-12-27 01:47:34,662][105692] Updated weights for policy 0, policy_version 1428057 (0.0007) [2023-12-27 01:47:34,672][105620] Updated weights for policy 1, policy_version 1430292 (0.0006) [2023-12-27 01:47:34,697][105586] KL-divergence is very high: 118.5576 [2023-12-27 01:47:34,721][105620] Updated weights for policy 1, policy_version 1430302 (0.0005) [2023-12-27 01:47:34,723][105692] Updated weights for policy 0, policy_version 1428067 (0.0007) [2023-12-27 01:47:34,737][105586] KL-divergence is very high: 139.3129 [2023-12-27 01:47:34,781][105620] Updated weights for policy 1, policy_version 1430312 (0.0007) [2023-12-27 01:47:34,787][105692] Updated weights for policy 0, policy_version 1428077 (0.0006) [2023-12-27 01:47:35,467][105620] Updated weights for policy 1, policy_version 1430322 (0.0006) [2023-12-27 01:47:35,520][105620] Updated weights for policy 1, policy_version 1430332 (0.0010) [2023-12-27 01:47:35,559][105692] Updated weights for policy 0, policy_version 1428087 (0.0007) [2023-12-27 01:47:35,565][105620] Updated weights for policy 1, policy_version 1430342 (0.0010) [2023-12-27 01:47:35,616][105692] Updated weights for policy 0, policy_version 1428097 (0.0007) [2023-12-27 01:47:35,676][105692] Updated weights for policy 0, policy_version 1428107 (0.0007) [2023-12-27 01:47:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19933.8, 300 sec: 19549.7). Total num frames: 731865088. Throughput: 0: 10123.5, 1: 9868.0. Samples: 731854660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:47:36,063][104569] Avg episode reward: [(0, '8257.273'), (1, '8532.162')] [2023-12-27 01:47:36,212][105620] Updated weights for policy 1, policy_version 1430352 (0.0009) [2023-12-27 01:47:36,273][105620] Updated weights for policy 1, policy_version 1430362 (0.0009) [2023-12-27 01:47:36,319][105692] Updated weights for policy 0, policy_version 1428117 (0.0008) [2023-12-27 01:47:36,339][105620] Updated weights for policy 1, policy_version 1430372 (0.0008) [2023-12-27 01:47:36,371][105692] Updated weights for policy 0, policy_version 1428127 (0.0008) [2023-12-27 01:47:36,437][105692] Updated weights for policy 0, policy_version 1428137 (0.0010) [2023-12-27 01:47:36,985][105620] Updated weights for policy 1, policy_version 1430382 (0.0006) [2023-12-27 01:47:37,045][105620] Updated weights for policy 1, policy_version 1430392 (0.0009) [2023-12-27 01:47:37,101][105620] Updated weights for policy 1, policy_version 1430402 (0.0008) [2023-12-27 01:47:37,268][105692] Updated weights for policy 0, policy_version 1428147 (0.0009) [2023-12-27 01:47:37,322][105692] Updated weights for policy 0, policy_version 1428157 (0.0010) [2023-12-27 01:47:37,375][105692] Updated weights for policy 0, policy_version 1428167 (0.0009) [2023-12-27 01:47:37,681][105620] Updated weights for policy 1, policy_version 1430412 (0.0008) [2023-12-27 01:47:37,744][105620] Updated weights for policy 1, policy_version 1430422 (0.0010) [2023-12-27 01:47:37,814][105620] Updated weights for policy 1, policy_version 1430432 (0.0010) [2023-12-27 01:47:38,165][105692] Updated weights for policy 0, policy_version 1428177 (0.0009) [2023-12-27 01:47:38,227][105692] Updated weights for policy 0, policy_version 1428187 (0.0006) [2023-12-27 01:47:38,298][105692] Updated weights for policy 0, policy_version 1428197 (0.0010) [2023-12-27 01:47:38,366][105692] Updated weights for policy 0, policy_version 1428207 (0.0009) [2023-12-27 01:47:38,523][105620] Updated weights for policy 1, policy_version 1430442 (0.0010) [2023-12-27 01:47:38,574][105620] Updated weights for policy 1, policy_version 1430452 (0.0010) [2023-12-27 01:47:38,625][105620] Updated weights for policy 1, policy_version 1430462 (0.0010) [2023-12-27 01:47:38,687][105620] Updated weights for policy 1, policy_version 1430472 (0.0010) [2023-12-27 01:47:38,999][105692] Updated weights for policy 0, policy_version 1428217 (0.0008) [2023-12-27 01:47:39,058][105692] Updated weights for policy 0, policy_version 1428227 (0.0008) [2023-12-27 01:47:39,117][105692] Updated weights for policy 0, policy_version 1428237 (0.0007) [2023-12-27 01:47:39,445][105620] Updated weights for policy 1, policy_version 1430482 (0.0009) [2023-12-27 01:47:39,512][105620] Updated weights for policy 1, policy_version 1430492 (0.0009) [2023-12-27 01:47:39,575][105620] Updated weights for policy 1, policy_version 1430502 (0.0010) [2023-12-27 01:47:39,886][105692] Updated weights for policy 0, policy_version 1428247 (0.0008) [2023-12-27 01:47:39,948][105692] Updated weights for policy 0, policy_version 1428257 (0.0009) [2023-12-27 01:47:40,001][105692] Updated weights for policy 0, policy_version 1428267 (0.0008) [2023-12-27 01:47:40,339][105620] Updated weights for policy 1, policy_version 1430512 (0.0010) [2023-12-27 01:47:40,401][105620] Updated weights for policy 1, policy_version 1430522 (0.0011) [2023-12-27 01:47:40,473][105620] Updated weights for policy 1, policy_version 1430532 (0.0009) [2023-12-27 01:47:40,742][105692] Updated weights for policy 0, policy_version 1428277 (0.0008) [2023-12-27 01:47:40,801][105692] Updated weights for policy 0, policy_version 1428287 (0.0008) [2023-12-27 01:47:40,860][105692] Updated weights for policy 0, policy_version 1428297 (0.0008) [2023-12-27 01:47:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 731963392. Throughput: 0: 10033.0, 1: 9819.1. Samples: 731970916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:47:41,063][104569] Avg episode reward: [(0, '8443.480'), (1, '8719.543')] [2023-12-27 01:47:41,199][105620] Updated weights for policy 1, policy_version 1430542 (0.0009) [2023-12-27 01:47:41,268][105620] Updated weights for policy 1, policy_version 1430552 (0.0007) [2023-12-27 01:47:41,331][105620] Updated weights for policy 1, policy_version 1430562 (0.0006) [2023-12-27 01:47:41,693][105692] Updated weights for policy 0, policy_version 1428307 (0.0009) [2023-12-27 01:47:41,765][105692] Updated weights for policy 0, policy_version 1428317 (0.0010) [2023-12-27 01:47:41,831][105692] Updated weights for policy 0, policy_version 1428327 (0.0009) [2023-12-27 01:47:41,964][105620] Updated weights for policy 1, policy_version 1430572 (0.0009) [2023-12-27 01:47:42,028][105620] Updated weights for policy 1, policy_version 1430582 (0.0008) [2023-12-27 01:47:42,086][105620] Updated weights for policy 1, policy_version 1430592 (0.0007) [2023-12-27 01:47:42,640][105692] Updated weights for policy 0, policy_version 1428337 (0.0009) [2023-12-27 01:47:42,701][105692] Updated weights for policy 0, policy_version 1428347 (0.0008) [2023-12-27 01:47:42,766][105692] Updated weights for policy 0, policy_version 1428357 (0.0007) [2023-12-27 01:47:42,826][105620] Updated weights for policy 1, policy_version 1430602 (0.0006) [2023-12-27 01:47:42,830][105692] Updated weights for policy 0, policy_version 1428367 (0.0009) [2023-12-27 01:47:42,887][105620] Updated weights for policy 1, policy_version 1430612 (0.0006) [2023-12-27 01:47:42,941][105620] Updated weights for policy 1, policy_version 1430622 (0.0011) [2023-12-27 01:47:43,001][105620] Updated weights for policy 1, policy_version 1430632 (0.0011) [2023-12-27 01:47:43,588][105692] Updated weights for policy 0, policy_version 1428377 (0.0008) [2023-12-27 01:47:43,614][105620] Updated weights for policy 1, policy_version 1430642 (0.0005) [2023-12-27 01:47:43,640][105692] Updated weights for policy 0, policy_version 1428387 (0.0010) [2023-12-27 01:47:43,662][105620] Updated weights for policy 1, policy_version 1430652 (0.0005) [2023-12-27 01:47:43,688][105692] Updated weights for policy 0, policy_version 1428397 (0.0010) [2023-12-27 01:47:43,718][105620] Updated weights for policy 1, policy_version 1430662 (0.0005) [2023-12-27 01:47:44,209][105620] Updated weights for policy 1, policy_version 1430672 (0.0005) [2023-12-27 01:47:44,253][105620] Updated weights for policy 1, policy_version 1430682 (0.0006) [2023-12-27 01:47:44,302][105620] Updated weights for policy 1, policy_version 1430692 (0.0005) [2023-12-27 01:47:44,563][105692] Updated weights for policy 0, policy_version 1428407 (0.0010) [2023-12-27 01:47:44,619][105692] Updated weights for policy 0, policy_version 1428417 (0.0007) [2023-12-27 01:47:44,679][105692] Updated weights for policy 0, policy_version 1428427 (0.0009) [2023-12-27 01:47:44,958][105620] Updated weights for policy 1, policy_version 1430702 (0.0008) [2023-12-27 01:47:45,021][105620] Updated weights for policy 1, policy_version 1430712 (0.0011) [2023-12-27 01:47:45,083][105620] Updated weights for policy 1, policy_version 1430722 (0.0010) [2023-12-27 01:47:45,395][105692] Updated weights for policy 0, policy_version 1428437 (0.0007) [2023-12-27 01:47:45,456][105692] Updated weights for policy 0, policy_version 1428447 (0.0008) [2023-12-27 01:47:45,513][105692] Updated weights for policy 0, policy_version 1428457 (0.0010) [2023-12-27 01:47:45,720][105620] Updated weights for policy 1, policy_version 1430732 (0.0008) [2023-12-27 01:47:45,786][105620] Updated weights for policy 1, policy_version 1430742 (0.0010) [2023-12-27 01:47:45,847][105620] Updated weights for policy 1, policy_version 1430752 (0.0010) [2023-12-27 01:47:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19933.8, 300 sec: 19577.5). Total num frames: 732061696. Throughput: 0: 9935.3, 1: 9749.2. Samples: 732029992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:47:46,063][104569] Avg episode reward: [(0, '8443.858'), (1, '8715.872')] [2023-12-27 01:47:46,074][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001428464_365740032.pth... [2023-12-27 01:47:46,074][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001430760_366321664.pth... [2023-12-27 01:47:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001427312_365445120.pth [2023-12-27 01:47:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001429608_366026752.pth [2023-12-27 01:47:46,216][105692] Updated weights for policy 0, policy_version 1428467 (0.0009) [2023-12-27 01:47:46,262][105692] Updated weights for policy 0, policy_version 1428477 (0.0005) [2023-12-27 01:47:46,321][105692] Updated weights for policy 0, policy_version 1428487 (0.0010) [2023-12-27 01:47:46,471][105620] Updated weights for policy 1, policy_version 1430762 (0.0009) [2023-12-27 01:47:46,535][105620] Updated weights for policy 1, policy_version 1430772 (0.0005) [2023-12-27 01:47:46,597][105620] Updated weights for policy 1, policy_version 1430782 (0.0006) [2023-12-27 01:47:46,659][105620] Updated weights for policy 1, policy_version 1430792 (0.0005) [2023-12-27 01:47:46,989][105692] Updated weights for policy 0, policy_version 1428497 (0.0010) [2023-12-27 01:47:47,045][105692] Updated weights for policy 0, policy_version 1428507 (0.0006) [2023-12-27 01:47:47,106][105692] Updated weights for policy 0, policy_version 1428517 (0.0006) [2023-12-27 01:47:47,160][105692] Updated weights for policy 0, policy_version 1428527 (0.0005) [2023-12-27 01:47:47,270][105620] Updated weights for policy 1, policy_version 1430802 (0.0006) [2023-12-27 01:47:47,332][105620] Updated weights for policy 1, policy_version 1430812 (0.0008) [2023-12-27 01:47:47,404][105620] Updated weights for policy 1, policy_version 1430822 (0.0010) [2023-12-27 01:47:47,695][105692] Updated weights for policy 0, policy_version 1428537 (0.0005) [2023-12-27 01:47:47,748][105692] Updated weights for policy 0, policy_version 1428547 (0.0005) [2023-12-27 01:47:47,808][105692] Updated weights for policy 0, policy_version 1428557 (0.0005) [2023-12-27 01:47:47,934][105620] Updated weights for policy 1, policy_version 1430832 (0.0006) [2023-12-27 01:47:47,988][105620] Updated weights for policy 1, policy_version 1430842 (0.0005) [2023-12-27 01:47:48,038][105620] Updated weights for policy 1, policy_version 1430852 (0.0009) [2023-12-27 01:47:48,377][105692] Updated weights for policy 0, policy_version 1428567 (0.0008) [2023-12-27 01:47:48,438][105692] Updated weights for policy 0, policy_version 1428577 (0.0009) [2023-12-27 01:47:48,503][105692] Updated weights for policy 0, policy_version 1428587 (0.0009) [2023-12-27 01:47:48,655][105620] Updated weights for policy 1, policy_version 1430862 (0.0005) [2023-12-27 01:47:48,716][105620] Updated weights for policy 1, policy_version 1430872 (0.0007) [2023-12-27 01:47:48,772][105620] Updated weights for policy 1, policy_version 1430882 (0.0011) [2023-12-27 01:47:49,291][105692] Updated weights for policy 0, policy_version 1428597 (0.0008) [2023-12-27 01:47:49,364][105692] Updated weights for policy 0, policy_version 1428607 (0.0007) [2023-12-27 01:47:49,427][105692] Updated weights for policy 0, policy_version 1428617 (0.0009) [2023-12-27 01:47:49,509][105620] Updated weights for policy 1, policy_version 1430892 (0.0010) [2023-12-27 01:47:49,567][105620] Updated weights for policy 1, policy_version 1430902 (0.0009) [2023-12-27 01:47:49,631][105620] Updated weights for policy 1, policy_version 1430912 (0.0006) [2023-12-27 01:47:50,103][105692] Updated weights for policy 0, policy_version 1428627 (0.0008) [2023-12-27 01:47:50,170][105692] Updated weights for policy 0, policy_version 1428637 (0.0008) [2023-12-27 01:47:50,236][105692] Updated weights for policy 0, policy_version 1428647 (0.0009) [2023-12-27 01:47:50,311][105620] Updated weights for policy 1, policy_version 1430922 (0.0006) [2023-12-27 01:47:50,376][105620] Updated weights for policy 1, policy_version 1430932 (0.0010) [2023-12-27 01:47:50,442][105620] Updated weights for policy 1, policy_version 1430942 (0.0009) [2023-12-27 01:47:50,501][105620] Updated weights for policy 1, policy_version 1430952 (0.0010) [2023-12-27 01:47:50,979][105692] Updated weights for policy 0, policy_version 1428657 (0.0009) [2023-12-27 01:47:51,045][105692] Updated weights for policy 0, policy_version 1428667 (0.0007) [2023-12-27 01:47:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 732160000. Throughput: 0: 9953.5, 1: 9887.9. Samples: 732154828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:47:51,063][104569] Avg episode reward: [(0, '8352.971'), (1, '8715.919')] [2023-12-27 01:47:51,109][105692] Updated weights for policy 0, policy_version 1428677 (0.0008) [2023-12-27 01:47:51,172][105692] Updated weights for policy 0, policy_version 1428687 (0.0008) [2023-12-27 01:47:51,212][105620] Updated weights for policy 1, policy_version 1430962 (0.0008) [2023-12-27 01:47:51,262][105620] Updated weights for policy 1, policy_version 1430972 (0.0008) [2023-12-27 01:47:51,311][105620] Updated weights for policy 1, policy_version 1430982 (0.0006) [2023-12-27 01:47:51,927][105692] Updated weights for policy 0, policy_version 1428697 (0.0009) [2023-12-27 01:47:51,988][105692] Updated weights for policy 0, policy_version 1428707 (0.0010) [2023-12-27 01:47:52,044][105692] Updated weights for policy 0, policy_version 1428717 (0.0009) [2023-12-27 01:47:52,075][105620] Updated weights for policy 1, policy_version 1430992 (0.0009) [2023-12-27 01:47:52,135][105620] Updated weights for policy 1, policy_version 1431002 (0.0011) [2023-12-27 01:47:52,195][105620] Updated weights for policy 1, policy_version 1431012 (0.0011) [2023-12-27 01:47:52,697][105692] Updated weights for policy 0, policy_version 1428727 (0.0010) [2023-12-27 01:47:52,754][105692] Updated weights for policy 0, policy_version 1428737 (0.0011) [2023-12-27 01:47:52,806][105692] Updated weights for policy 0, policy_version 1428747 (0.0010) [2023-12-27 01:47:52,831][105620] Updated weights for policy 1, policy_version 1431022 (0.0010) [2023-12-27 01:47:52,898][105620] Updated weights for policy 1, policy_version 1431032 (0.0008) [2023-12-27 01:47:52,958][105620] Updated weights for policy 1, policy_version 1431042 (0.0005) [2023-12-27 01:47:53,538][105692] Updated weights for policy 0, policy_version 1428757 (0.0011) [2023-12-27 01:47:53,545][105620] Updated weights for policy 1, policy_version 1431052 (0.0005) [2023-12-27 01:47:53,597][105620] Updated weights for policy 1, policy_version 1431062 (0.0007) [2023-12-27 01:47:53,601][105692] Updated weights for policy 0, policy_version 1428767 (0.0010) [2023-12-27 01:47:53,654][105692] Updated weights for policy 0, policy_version 1428777 (0.0006) [2023-12-27 01:47:53,657][105620] Updated weights for policy 1, policy_version 1431072 (0.0008) [2023-12-27 01:47:54,197][105692] Updated weights for policy 0, policy_version 1428787 (0.0008) [2023-12-27 01:47:54,259][105692] Updated weights for policy 0, policy_version 1428797 (0.0005) [2023-12-27 01:47:54,321][105692] Updated weights for policy 0, policy_version 1428807 (0.0005) [2023-12-27 01:47:54,410][105620] Updated weights for policy 1, policy_version 1431082 (0.0007) [2023-12-27 01:47:54,466][105620] Updated weights for policy 1, policy_version 1431092 (0.0008) [2023-12-27 01:47:54,525][105620] Updated weights for policy 1, policy_version 1431102 (0.0008) [2023-12-27 01:47:54,579][105620] Updated weights for policy 1, policy_version 1431112 (0.0008) [2023-12-27 01:47:54,952][105692] Updated weights for policy 0, policy_version 1428817 (0.0006) [2023-12-27 01:47:55,010][105692] Updated weights for policy 0, policy_version 1428827 (0.0008) [2023-12-27 01:47:55,072][105692] Updated weights for policy 0, policy_version 1428837 (0.0005) [2023-12-27 01:47:55,134][105692] Updated weights for policy 0, policy_version 1428847 (0.0008) [2023-12-27 01:47:55,356][105620] Updated weights for policy 1, policy_version 1431122 (0.0008) [2023-12-27 01:47:55,407][105620] Updated weights for policy 1, policy_version 1431132 (0.0008) [2023-12-27 01:47:55,461][105620] Updated weights for policy 1, policy_version 1431142 (0.0007) [2023-12-27 01:47:55,765][105692] Updated weights for policy 0, policy_version 1428857 (0.0010) [2023-12-27 01:47:55,823][105692] Updated weights for policy 0, policy_version 1428867 (0.0010) [2023-12-27 01:47:55,887][105692] Updated weights for policy 0, policy_version 1428877 (0.0007) [2023-12-27 01:47:56,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19933.8, 300 sec: 19605.3). Total num frames: 732266496. Throughput: 0: 10006.2, 1: 9915.2. Samples: 732274996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:47:56,062][104569] Avg episode reward: [(0, '8439.222'), (1, '8898.436')] [2023-12-27 01:47:56,175][105620] Updated weights for policy 1, policy_version 1431152 (0.0010) [2023-12-27 01:47:56,229][105620] Updated weights for policy 1, policy_version 1431162 (0.0010) [2023-12-27 01:47:56,284][105620] Updated weights for policy 1, policy_version 1431172 (0.0010) [2023-12-27 01:47:56,405][105692] Updated weights for policy 0, policy_version 1428887 (0.0009) [2023-12-27 01:47:56,459][105692] Updated weights for policy 0, policy_version 1428897 (0.0010) [2023-12-27 01:47:56,506][105692] Updated weights for policy 0, policy_version 1428907 (0.0010) [2023-12-27 01:47:57,109][105620] Updated weights for policy 1, policy_version 1431182 (0.0007) [2023-12-27 01:47:57,168][105620] Updated weights for policy 1, policy_version 1431192 (0.0005) [2023-12-27 01:47:57,183][105692] Updated weights for policy 0, policy_version 1428917 (0.0008) [2023-12-27 01:47:57,227][105620] Updated weights for policy 1, policy_version 1431202 (0.0009) [2023-12-27 01:47:57,238][105692] Updated weights for policy 0, policy_version 1428927 (0.0007) [2023-12-27 01:47:57,299][105692] Updated weights for policy 0, policy_version 1428937 (0.0010) [2023-12-27 01:47:57,867][105620] Updated weights for policy 1, policy_version 1431212 (0.0007) [2023-12-27 01:47:57,904][105692] Updated weights for policy 0, policy_version 1428947 (0.0010) [2023-12-27 01:47:57,929][105620] Updated weights for policy 1, policy_version 1431222 (0.0005) [2023-12-27 01:47:57,968][105692] Updated weights for policy 0, policy_version 1428957 (0.0006) [2023-12-27 01:47:57,984][105620] Updated weights for policy 1, policy_version 1431232 (0.0005) [2023-12-27 01:47:58,030][105692] Updated weights for policy 0, policy_version 1428967 (0.0005) [2023-12-27 01:47:58,709][105620] Updated weights for policy 1, policy_version 1431242 (0.0006) [2023-12-27 01:47:58,734][105692] Updated weights for policy 0, policy_version 1428977 (0.0005) [2023-12-27 01:47:58,772][105620] Updated weights for policy 1, policy_version 1431252 (0.0009) [2023-12-27 01:47:58,805][105692] Updated weights for policy 0, policy_version 1428987 (0.0009) [2023-12-27 01:47:58,845][105620] Updated weights for policy 1, policy_version 1431262 (0.0008) [2023-12-27 01:47:58,871][105692] Updated weights for policy 0, policy_version 1428997 (0.0009) [2023-12-27 01:47:58,918][105620] Updated weights for policy 1, policy_version 1431272 (0.0007) [2023-12-27 01:47:58,938][105692] Updated weights for policy 0, policy_version 1429007 (0.0009) [2023-12-27 01:47:59,654][105620] Updated weights for policy 1, policy_version 1431282 (0.0008) [2023-12-27 01:47:59,716][105620] Updated weights for policy 1, policy_version 1431292 (0.0010) [2023-12-27 01:47:59,750][105692] Updated weights for policy 0, policy_version 1429017 (0.0008) [2023-12-27 01:47:59,771][105620] Updated weights for policy 1, policy_version 1431302 (0.0010) [2023-12-27 01:47:59,799][105692] Updated weights for policy 0, policy_version 1429027 (0.0006) [2023-12-27 01:47:59,866][105692] Updated weights for policy 0, policy_version 1429037 (0.0007) [2023-12-27 01:48:00,474][105620] Updated weights for policy 1, policy_version 1431312 (0.0007) [2023-12-27 01:48:00,534][105620] Updated weights for policy 1, policy_version 1431322 (0.0009) [2023-12-27 01:48:00,598][105620] Updated weights for policy 1, policy_version 1431332 (0.0009) [2023-12-27 01:48:00,640][105692] Updated weights for policy 0, policy_version 1429047 (0.0009) [2023-12-27 01:48:00,697][105692] Updated weights for policy 0, policy_version 1429057 (0.0009) [2023-12-27 01:48:00,751][105692] Updated weights for policy 0, policy_version 1429067 (0.0009) [2023-12-27 01:48:01,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.3, 300 sec: 19605.2). Total num frames: 732364800. Throughput: 0: 10130.7, 1: 9933.0. Samples: 732336108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:48:01,062][104569] Avg episode reward: [(0, '8528.792'), (1, '8805.645')] [2023-12-27 01:48:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001429072_365895680.pth... [2023-12-27 01:48:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001431336_366469120.pth... [2023-12-27 01:48:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001427920_365600768.pth [2023-12-27 01:48:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001430152_366166016.pth [2023-12-27 01:48:01,268][105620] Updated weights for policy 1, policy_version 1431342 (0.0008) [2023-12-27 01:48:01,331][105620] Updated weights for policy 1, policy_version 1431352 (0.0008) [2023-12-27 01:48:01,394][105620] Updated weights for policy 1, policy_version 1431362 (0.0008) [2023-12-27 01:48:01,497][105692] Updated weights for policy 0, policy_version 1429077 (0.0008) [2023-12-27 01:48:01,552][105692] Updated weights for policy 0, policy_version 1429087 (0.0006) [2023-12-27 01:48:01,614][105692] Updated weights for policy 0, policy_version 1429097 (0.0008) [2023-12-27 01:48:02,154][105620] Updated weights for policy 1, policy_version 1431372 (0.0008) [2023-12-27 01:48:02,204][105620] Updated weights for policy 1, policy_version 1431382 (0.0009) [2023-12-27 01:48:02,255][105620] Updated weights for policy 1, policy_version 1431392 (0.0009) [2023-12-27 01:48:02,326][105692] Updated weights for policy 0, policy_version 1429107 (0.0008) [2023-12-27 01:48:02,395][105692] Updated weights for policy 0, policy_version 1429117 (0.0008) [2023-12-27 01:48:02,460][105692] Updated weights for policy 0, policy_version 1429127 (0.0008) [2023-12-27 01:48:03,058][105620] Updated weights for policy 1, policy_version 1431402 (0.0008) [2023-12-27 01:48:03,076][105692] Updated weights for policy 0, policy_version 1429137 (0.0006) [2023-12-27 01:48:03,117][105620] Updated weights for policy 1, policy_version 1431412 (0.0010) [2023-12-27 01:48:03,141][105692] Updated weights for policy 0, policy_version 1429147 (0.0011) [2023-12-27 01:48:03,174][105620] Updated weights for policy 1, policy_version 1431422 (0.0008) [2023-12-27 01:48:03,199][105692] Updated weights for policy 0, policy_version 1429157 (0.0010) [2023-12-27 01:48:03,225][105620] Updated weights for policy 1, policy_version 1431432 (0.0007) [2023-12-27 01:48:03,251][105692] Updated weights for policy 0, policy_version 1429167 (0.0010) [2023-12-27 01:48:03,916][105620] Updated weights for policy 1, policy_version 1431442 (0.0008) [2023-12-27 01:48:03,929][105692] Updated weights for policy 0, policy_version 1429177 (0.0008) [2023-12-27 01:48:03,978][105620] Updated weights for policy 1, policy_version 1431452 (0.0006) [2023-12-27 01:48:03,993][105692] Updated weights for policy 0, policy_version 1429187 (0.0008) [2023-12-27 01:48:04,045][105620] Updated weights for policy 1, policy_version 1431462 (0.0008) [2023-12-27 01:48:04,056][105692] Updated weights for policy 0, policy_version 1429197 (0.0008) [2023-12-27 01:48:04,679][105692] Updated weights for policy 0, policy_version 1429207 (0.0007) [2023-12-27 01:48:04,698][105620] Updated weights for policy 1, policy_version 1431472 (0.0007) [2023-12-27 01:48:04,735][105692] Updated weights for policy 0, policy_version 1429217 (0.0009) [2023-12-27 01:48:04,753][105620] Updated weights for policy 1, policy_version 1431482 (0.0005) [2023-12-27 01:48:04,794][105692] Updated weights for policy 0, policy_version 1429227 (0.0007) [2023-12-27 01:48:04,815][105620] Updated weights for policy 1, policy_version 1431492 (0.0006) [2023-12-27 01:48:05,488][105620] Updated weights for policy 1, policy_version 1431502 (0.0007) [2023-12-27 01:48:05,489][105692] Updated weights for policy 0, policy_version 1429237 (0.0006) [2023-12-27 01:48:05,549][105692] Updated weights for policy 0, policy_version 1429247 (0.0007) [2023-12-27 01:48:05,550][105620] Updated weights for policy 1, policy_version 1431512 (0.0008) [2023-12-27 01:48:05,606][105692] Updated weights for policy 0, policy_version 1429257 (0.0007) [2023-12-27 01:48:05,613][105620] Updated weights for policy 1, policy_version 1431522 (0.0007) [2023-12-27 01:48:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 732463104. Throughput: 0: 10054.1, 1: 10018.8. Samples: 732453176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:48:06,063][104569] Avg episode reward: [(0, '8164.639'), (1, '8712.941')] [2023-12-27 01:48:06,172][105620] Updated weights for policy 1, policy_version 1431532 (0.0006) [2023-12-27 01:48:06,225][105620] Updated weights for policy 1, policy_version 1431542 (0.0008) [2023-12-27 01:48:06,280][105620] Updated weights for policy 1, policy_version 1431552 (0.0008) [2023-12-27 01:48:06,438][105692] Updated weights for policy 0, policy_version 1429267 (0.0009) [2023-12-27 01:48:06,500][105692] Updated weights for policy 0, policy_version 1429277 (0.0008) [2023-12-27 01:48:06,566][105692] Updated weights for policy 0, policy_version 1429287 (0.0009) [2023-12-27 01:48:07,045][105620] Updated weights for policy 1, policy_version 1431562 (0.0008) [2023-12-27 01:48:07,107][105620] Updated weights for policy 1, policy_version 1431572 (0.0007) [2023-12-27 01:48:07,167][105620] Updated weights for policy 1, policy_version 1431582 (0.0011) [2023-12-27 01:48:07,223][105620] Updated weights for policy 1, policy_version 1431592 (0.0010) [2023-12-27 01:48:07,312][105692] Updated weights for policy 0, policy_version 1429297 (0.0009) [2023-12-27 01:48:07,370][105692] Updated weights for policy 0, policy_version 1429307 (0.0010) [2023-12-27 01:48:07,431][105692] Updated weights for policy 0, policy_version 1429317 (0.0009) [2023-12-27 01:48:07,486][105692] Updated weights for policy 0, policy_version 1429327 (0.0009) [2023-12-27 01:48:07,882][105620] Updated weights for policy 1, policy_version 1431602 (0.0009) [2023-12-27 01:48:07,928][105620] Updated weights for policy 1, policy_version 1431612 (0.0006) [2023-12-27 01:48:07,973][105620] Updated weights for policy 1, policy_version 1431622 (0.0005) [2023-12-27 01:48:08,313][105692] Updated weights for policy 0, policy_version 1429337 (0.0009) [2023-12-27 01:48:08,378][105692] Updated weights for policy 0, policy_version 1429347 (0.0009) [2023-12-27 01:48:08,431][105692] Updated weights for policy 0, policy_version 1429357 (0.0008) [2023-12-27 01:48:08,605][105620] Updated weights for policy 1, policy_version 1431632 (0.0005) [2023-12-27 01:48:08,677][105620] Updated weights for policy 1, policy_version 1431642 (0.0006) [2023-12-27 01:48:08,736][105620] Updated weights for policy 1, policy_version 1431652 (0.0009) [2023-12-27 01:48:09,092][105692] Updated weights for policy 0, policy_version 1429367 (0.0009) [2023-12-27 01:48:09,146][105692] Updated weights for policy 0, policy_version 1429377 (0.0009) [2023-12-27 01:48:09,197][105692] Updated weights for policy 0, policy_version 1429387 (0.0009) [2023-12-27 01:48:09,468][105620] Updated weights for policy 1, policy_version 1431662 (0.0009) [2023-12-27 01:48:09,537][105620] Updated weights for policy 1, policy_version 1431672 (0.0008) [2023-12-27 01:48:09,597][105620] Updated weights for policy 1, policy_version 1431682 (0.0008) [2023-12-27 01:48:09,922][105692] Updated weights for policy 0, policy_version 1429397 (0.0009) [2023-12-27 01:48:09,991][105692] Updated weights for policy 0, policy_version 1429407 (0.0009) [2023-12-27 01:48:10,055][105692] Updated weights for policy 0, policy_version 1429417 (0.0010) [2023-12-27 01:48:10,391][105620] Updated weights for policy 1, policy_version 1431692 (0.0009) [2023-12-27 01:48:10,458][105620] Updated weights for policy 1, policy_version 1431702 (0.0009) [2023-12-27 01:48:10,530][105620] Updated weights for policy 1, policy_version 1431712 (0.0009) [2023-12-27 01:48:10,722][105692] Updated weights for policy 0, policy_version 1429427 (0.0008) [2023-12-27 01:48:10,779][105692] Updated weights for policy 0, policy_version 1429437 (0.0007) [2023-12-27 01:48:10,832][105692] Updated weights for policy 0, policy_version 1429447 (0.0008) [2023-12-27 01:48:11,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 732561408. Throughput: 0: 9964.8, 1: 10070.7. Samples: 732568052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:48:11,062][104569] Avg episode reward: [(0, '8169.587'), (1, '8804.880')] [2023-12-27 01:48:11,424][105620] Updated weights for policy 1, policy_version 1431722 (0.0010) [2023-12-27 01:48:11,478][105620] Updated weights for policy 1, policy_version 1431732 (0.0008) [2023-12-27 01:48:11,495][105692] Updated weights for policy 0, policy_version 1429457 (0.0009) [2023-12-27 01:48:11,531][105620] Updated weights for policy 1, policy_version 1431742 (0.0007) [2023-12-27 01:48:11,558][105692] Updated weights for policy 0, policy_version 1429467 (0.0008) [2023-12-27 01:48:11,581][105620] Updated weights for policy 1, policy_version 1431752 (0.0006) [2023-12-27 01:48:11,620][105692] Updated weights for policy 0, policy_version 1429477 (0.0008) [2023-12-27 01:48:11,683][105692] Updated weights for policy 0, policy_version 1429487 (0.0008) [2023-12-27 01:48:12,320][105620] Updated weights for policy 1, policy_version 1431762 (0.0008) [2023-12-27 01:48:12,386][105620] Updated weights for policy 1, policy_version 1431772 (0.0010) [2023-12-27 01:48:12,451][105620] Updated weights for policy 1, policy_version 1431782 (0.0011) [2023-12-27 01:48:12,482][105692] Updated weights for policy 0, policy_version 1429497 (0.0008) [2023-12-27 01:48:12,529][105692] Updated weights for policy 0, policy_version 1429507 (0.0008) [2023-12-27 01:48:12,591][105692] Updated weights for policy 0, policy_version 1429517 (0.0009) [2023-12-27 01:48:13,173][105620] Updated weights for policy 1, policy_version 1431792 (0.0007) [2023-12-27 01:48:13,228][105620] Updated weights for policy 1, policy_version 1431802 (0.0006) [2023-12-27 01:48:13,275][105620] Updated weights for policy 1, policy_version 1431812 (0.0009) [2023-12-27 01:48:13,365][105692] Updated weights for policy 0, policy_version 1429527 (0.0006) [2023-12-27 01:48:13,418][105692] Updated weights for policy 0, policy_version 1429537 (0.0005) [2023-12-27 01:48:13,466][105692] Updated weights for policy 0, policy_version 1429547 (0.0005) [2023-12-27 01:48:14,054][105692] Updated weights for policy 0, policy_version 1429557 (0.0008) [2023-12-27 01:48:14,096][105620] Updated weights for policy 1, policy_version 1431822 (0.0006) [2023-12-27 01:48:14,110][105692] Updated weights for policy 0, policy_version 1429567 (0.0010) [2023-12-27 01:48:14,149][105620] Updated weights for policy 1, policy_version 1431832 (0.0007) [2023-12-27 01:48:14,163][105692] Updated weights for policy 0, policy_version 1429577 (0.0006) [2023-12-27 01:48:14,208][105620] Updated weights for policy 1, policy_version 1431842 (0.0008) [2023-12-27 01:48:14,837][105692] Updated weights for policy 0, policy_version 1429587 (0.0007) [2023-12-27 01:48:14,895][105692] Updated weights for policy 0, policy_version 1429597 (0.0009) [2023-12-27 01:48:14,945][105692] Updated weights for policy 0, policy_version 1429607 (0.0009) [2023-12-27 01:48:14,957][105620] Updated weights for policy 1, policy_version 1431852 (0.0009) [2023-12-27 01:48:15,016][105620] Updated weights for policy 1, policy_version 1431862 (0.0009) [2023-12-27 01:48:15,085][105620] Updated weights for policy 1, policy_version 1431872 (0.0008) [2023-12-27 01:48:15,757][105692] Updated weights for policy 0, policy_version 1429617 (0.0008) [2023-12-27 01:48:15,764][105620] Updated weights for policy 1, policy_version 1431882 (0.0009) [2023-12-27 01:48:15,812][105620] Updated weights for policy 1, policy_version 1431892 (0.0007) [2023-12-27 01:48:15,814][105692] Updated weights for policy 0, policy_version 1429627 (0.0005) [2023-12-27 01:48:15,865][105692] Updated weights for policy 0, policy_version 1429637 (0.0008) [2023-12-27 01:48:15,866][105620] Updated weights for policy 1, policy_version 1431902 (0.0007) [2023-12-27 01:48:15,914][105692] Updated weights for policy 0, policy_version 1429647 (0.0007) [2023-12-27 01:48:15,916][105620] Updated weights for policy 1, policy_version 1431912 (0.0007) [2023-12-27 01:48:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 732659712. Throughput: 0: 9834.9, 1: 10021.5. Samples: 732624044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:48:16,063][104569] Avg episode reward: [(0, '8260.365'), (1, '8894.435')] [2023-12-27 01:48:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001429648_366043136.pth... [2023-12-27 01:48:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001431912_366616576.pth... [2023-12-27 01:48:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001428464_365740032.pth [2023-12-27 01:48:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001430760_366321664.pth [2023-12-27 01:48:16,572][105620] Updated weights for policy 1, policy_version 1431922 (0.0005) [2023-12-27 01:48:16,623][105620] Updated weights for policy 1, policy_version 1431932 (0.0009) [2023-12-27 01:48:16,681][105620] Updated weights for policy 1, policy_version 1431942 (0.0010) [2023-12-27 01:48:16,758][105692] Updated weights for policy 0, policy_version 1429657 (0.0009) [2023-12-27 01:48:16,825][105692] Updated weights for policy 0, policy_version 1429667 (0.0008) [2023-12-27 01:48:16,893][105692] Updated weights for policy 0, policy_version 1429677 (0.0008) [2023-12-27 01:48:17,354][105620] Updated weights for policy 1, policy_version 1431952 (0.0010) [2023-12-27 01:48:17,412][105620] Updated weights for policy 1, policy_version 1431962 (0.0008) [2023-12-27 01:48:17,476][105620] Updated weights for policy 1, policy_version 1431972 (0.0007) [2023-12-27 01:48:17,729][105692] Updated weights for policy 0, policy_version 1429687 (0.0009) [2023-12-27 01:48:17,783][105692] Updated weights for policy 0, policy_version 1429697 (0.0009) [2023-12-27 01:48:17,840][105692] Updated weights for policy 0, policy_version 1429707 (0.0010) [2023-12-27 01:48:18,208][105620] Updated weights for policy 1, policy_version 1431982 (0.0005) [2023-12-27 01:48:18,269][105620] Updated weights for policy 1, policy_version 1431992 (0.0005) [2023-12-27 01:48:18,328][105620] Updated weights for policy 1, policy_version 1432002 (0.0006) [2023-12-27 01:48:18,558][105692] Updated weights for policy 0, policy_version 1429717 (0.0010) [2023-12-27 01:48:18,625][105692] Updated weights for policy 0, policy_version 1429727 (0.0011) [2023-12-27 01:48:18,685][105692] Updated weights for policy 0, policy_version 1429737 (0.0011) [2023-12-27 01:48:18,999][105620] Updated weights for policy 1, policy_version 1432012 (0.0008) [2023-12-27 01:48:19,062][105620] Updated weights for policy 1, policy_version 1432022 (0.0008) [2023-12-27 01:48:19,121][105620] Updated weights for policy 1, policy_version 1432032 (0.0008) [2023-12-27 01:48:19,435][105692] Updated weights for policy 0, policy_version 1429747 (0.0011) [2023-12-27 01:48:19,483][105692] Updated weights for policy 0, policy_version 1429757 (0.0010) [2023-12-27 01:48:19,545][105692] Updated weights for policy 0, policy_version 1429767 (0.0010) [2023-12-27 01:48:19,895][105620] Updated weights for policy 1, policy_version 1432042 (0.0008) [2023-12-27 01:48:19,964][105620] Updated weights for policy 1, policy_version 1432052 (0.0009) [2023-12-27 01:48:20,026][105620] Updated weights for policy 1, policy_version 1432062 (0.0009) [2023-12-27 01:48:20,088][105620] Updated weights for policy 1, policy_version 1432072 (0.0009) [2023-12-27 01:48:20,259][105692] Updated weights for policy 0, policy_version 1429777 (0.0010) [2023-12-27 01:48:20,315][105692] Updated weights for policy 0, policy_version 1429787 (0.0007) [2023-12-27 01:48:20,379][105692] Updated weights for policy 0, policy_version 1429797 (0.0008) [2023-12-27 01:48:20,441][105692] Updated weights for policy 0, policy_version 1429807 (0.0007) [2023-12-27 01:48:20,912][105620] Updated weights for policy 1, policy_version 1432082 (0.0010) [2023-12-27 01:48:20,964][105620] Updated weights for policy 1, policy_version 1432092 (0.0010) [2023-12-27 01:48:21,018][105620] Updated weights for policy 1, policy_version 1432102 (0.0008) [2023-12-27 01:48:21,060][105692] Updated weights for policy 0, policy_version 1429817 (0.0007) [2023-12-27 01:48:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 732749824. Throughput: 0: 9701.9, 1: 9948.0. Samples: 732738904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:48:21,062][104569] Avg episode reward: [(0, '8257.469'), (1, '8894.108')] [2023-12-27 01:48:21,122][105692] Updated weights for policy 0, policy_version 1429827 (0.0007) [2023-12-27 01:48:21,185][105692] Updated weights for policy 0, policy_version 1429837 (0.0009) [2023-12-27 01:48:21,846][105620] Updated weights for policy 1, policy_version 1432112 (0.0007) [2023-12-27 01:48:21,916][105620] Updated weights for policy 1, policy_version 1432122 (0.0006) [2023-12-27 01:48:21,981][105692] Updated weights for policy 0, policy_version 1429847 (0.0008) [2023-12-27 01:48:21,984][105620] Updated weights for policy 1, policy_version 1432132 (0.0007) [2023-12-27 01:48:22,042][105692] Updated weights for policy 0, policy_version 1429857 (0.0009) [2023-12-27 01:48:22,106][105692] Updated weights for policy 0, policy_version 1429867 (0.0009) [2023-12-27 01:48:22,595][105620] Updated weights for policy 1, policy_version 1432142 (0.0008) [2023-12-27 01:48:22,668][105620] Updated weights for policy 1, policy_version 1432152 (0.0008) [2023-12-27 01:48:22,731][105620] Updated weights for policy 1, policy_version 1432162 (0.0007) [2023-12-27 01:48:22,870][105692] Updated weights for policy 0, policy_version 1429877 (0.0009) [2023-12-27 01:48:22,923][105692] Updated weights for policy 0, policy_version 1429887 (0.0007) [2023-12-27 01:48:22,975][105692] Updated weights for policy 0, policy_version 1429897 (0.0009) [2023-12-27 01:48:23,353][105620] Updated weights for policy 1, policy_version 1432172 (0.0006) [2023-12-27 01:48:23,416][105620] Updated weights for policy 1, policy_version 1432182 (0.0007) [2023-12-27 01:48:23,485][105620] Updated weights for policy 1, policy_version 1432192 (0.0006) [2023-12-27 01:48:23,873][105692] Updated weights for policy 0, policy_version 1429907 (0.0009) [2023-12-27 01:48:23,937][105692] Updated weights for policy 0, policy_version 1429917 (0.0009) [2023-12-27 01:48:23,998][105692] Updated weights for policy 0, policy_version 1429927 (0.0009) [2023-12-27 01:48:24,017][105620] Updated weights for policy 1, policy_version 1432202 (0.0005) [2023-12-27 01:48:24,066][105620] Updated weights for policy 1, policy_version 1432212 (0.0005) [2023-12-27 01:48:24,113][105620] Updated weights for policy 1, policy_version 1432222 (0.0008) [2023-12-27 01:48:24,168][105620] Updated weights for policy 1, policy_version 1432232 (0.0010) [2023-12-27 01:48:24,645][105692] Updated weights for policy 0, policy_version 1429937 (0.0009) [2023-12-27 01:48:24,711][105692] Updated weights for policy 0, policy_version 1429947 (0.0007) [2023-12-27 01:48:24,775][105692] Updated weights for policy 0, policy_version 1429957 (0.0009) [2023-12-27 01:48:24,789][105620] Updated weights for policy 1, policy_version 1432242 (0.0009) [2023-12-27 01:48:24,839][105692] Updated weights for policy 0, policy_version 1429967 (0.0009) [2023-12-27 01:48:24,843][105620] Updated weights for policy 1, policy_version 1432252 (0.0010) [2023-12-27 01:48:24,894][105620] Updated weights for policy 1, policy_version 1432262 (0.0010) [2023-12-27 01:48:25,542][105692] Updated weights for policy 0, policy_version 1429977 (0.0010) [2023-12-27 01:48:25,598][105692] Updated weights for policy 0, policy_version 1429987 (0.0010) [2023-12-27 01:48:25,633][105620] Updated weights for policy 1, policy_version 1432272 (0.0010) [2023-12-27 01:48:25,656][105692] Updated weights for policy 0, policy_version 1429997 (0.0010) [2023-12-27 01:48:25,690][105620] Updated weights for policy 1, policy_version 1432282 (0.0010) [2023-12-27 01:48:25,754][105620] Updated weights for policy 1, policy_version 1432292 (0.0010) [2023-12-27 01:48:26,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 732848128. Throughput: 0: 9716.0, 1: 9938.5. Samples: 732855368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:48:26,062][104569] Avg episode reward: [(0, '8071.502'), (1, '9171.145')] [2023-12-27 01:48:26,349][105620] Updated weights for policy 1, policy_version 1432302 (0.0007) [2023-12-27 01:48:26,401][105692] Updated weights for policy 0, policy_version 1430007 (0.0010) [2023-12-27 01:48:26,403][105620] Updated weights for policy 1, policy_version 1432312 (0.0005) [2023-12-27 01:48:26,458][105620] Updated weights for policy 1, policy_version 1432322 (0.0008) [2023-12-27 01:48:26,459][105692] Updated weights for policy 0, policy_version 1430017 (0.0010) [2023-12-27 01:48:26,517][105692] Updated weights for policy 0, policy_version 1430027 (0.0010) [2023-12-27 01:48:27,065][105620] Updated weights for policy 1, policy_version 1432332 (0.0011) [2023-12-27 01:48:27,109][105620] Updated weights for policy 1, policy_version 1432342 (0.0008) [2023-12-27 01:48:27,157][105620] Updated weights for policy 1, policy_version 1432352 (0.0006) [2023-12-27 01:48:27,180][105692] Updated weights for policy 0, policy_version 1430037 (0.0008) [2023-12-27 01:48:27,227][105692] Updated weights for policy 0, policy_version 1430048 (0.0008) [2023-12-27 01:48:27,276][105692] Updated weights for policy 0, policy_version 1430058 (0.0008) [2023-12-27 01:48:27,866][105692] Updated weights for policy 0, policy_version 1430068 (0.0005) [2023-12-27 01:48:27,918][105692] Updated weights for policy 0, policy_version 1430078 (0.0005) [2023-12-27 01:48:27,968][105620] Updated weights for policy 1, policy_version 1432362 (0.0008) [2023-12-27 01:48:27,977][105692] Updated weights for policy 0, policy_version 1430088 (0.0005) [2023-12-27 01:48:28,022][105620] Updated weights for policy 1, policy_version 1432372 (0.0008) [2023-12-27 01:48:28,081][105620] Updated weights for policy 1, policy_version 1432382 (0.0009) [2023-12-27 01:48:28,142][105620] Updated weights for policy 1, policy_version 1432392 (0.0009) [2023-12-27 01:48:28,599][105692] Updated weights for policy 0, policy_version 1430098 (0.0008) [2023-12-27 01:48:28,660][105692] Updated weights for policy 0, policy_version 1430108 (0.0010) [2023-12-27 01:48:28,725][105692] Updated weights for policy 0, policy_version 1430118 (0.0010) [2023-12-27 01:48:28,780][105692] Updated weights for policy 0, policy_version 1430128 (0.0010) [2023-12-27 01:48:28,948][105620] Updated weights for policy 1, policy_version 1432402 (0.0009) [2023-12-27 01:48:29,008][105620] Updated weights for policy 1, policy_version 1432412 (0.0009) [2023-12-27 01:48:29,069][105620] Updated weights for policy 1, policy_version 1432422 (0.0009) [2023-12-27 01:48:29,400][105692] Updated weights for policy 0, policy_version 1430138 (0.0010) [2023-12-27 01:48:29,458][105692] Updated weights for policy 0, policy_version 1430148 (0.0007) [2023-12-27 01:48:29,524][105692] Updated weights for policy 0, policy_version 1430158 (0.0005) [2023-12-27 01:48:29,886][105620] Updated weights for policy 1, policy_version 1432432 (0.0009) [2023-12-27 01:48:29,951][105620] Updated weights for policy 1, policy_version 1432442 (0.0008) [2023-12-27 01:48:30,017][105620] Updated weights for policy 1, policy_version 1432452 (0.0008) [2023-12-27 01:48:30,181][105692] Updated weights for policy 0, policy_version 1430168 (0.0006) [2023-12-27 01:48:30,236][105585] KL-divergence is very high: 117.6453 [2023-12-27 01:48:30,241][105692] Updated weights for policy 0, policy_version 1430178 (0.0005) [2023-12-27 01:48:30,274][105585] KL-divergence is very high: 137.6741 [2023-12-27 01:48:30,287][105692] Updated weights for policy 0, policy_version 1430188 (0.0005) [2023-12-27 01:48:30,862][105620] Updated weights for policy 1, policy_version 1432462 (0.0008) [2023-12-27 01:48:30,882][105692] Updated weights for policy 0, policy_version 1430198 (0.0005) [2023-12-27 01:48:30,920][105620] Updated weights for policy 1, policy_version 1432472 (0.0008) [2023-12-27 01:48:30,941][105692] Updated weights for policy 0, policy_version 1430208 (0.0005) [2023-12-27 01:48:30,975][105620] Updated weights for policy 1, policy_version 1432482 (0.0008) [2023-12-27 01:48:30,997][105692] Updated weights for policy 0, policy_version 1430218 (0.0005) [2023-12-27 01:48:31,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 732954624. Throughput: 0: 9813.2, 1: 9910.7. Samples: 732917564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:48:31,063][104569] Avg episode reward: [(0, '7980.409'), (1, '9173.416')] [2023-12-27 01:48:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001430224_366190592.pth... [2023-12-27 01:48:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001432488_366764032.pth... [2023-12-27 01:48:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001429072_365895680.pth [2023-12-27 01:48:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001431336_366469120.pth [2023-12-27 01:48:31,658][105692] Updated weights for policy 0, policy_version 1430228 (0.0009) [2023-12-27 01:48:31,723][105692] Updated weights for policy 0, policy_version 1430238 (0.0011) [2023-12-27 01:48:31,774][105692] Updated weights for policy 0, policy_version 1430248 (0.0009) [2023-12-27 01:48:31,784][105620] Updated weights for policy 1, policy_version 1432492 (0.0008) [2023-12-27 01:48:31,843][105620] Updated weights for policy 1, policy_version 1432502 (0.0008) [2023-12-27 01:48:31,900][105620] Updated weights for policy 1, policy_version 1432512 (0.0010) [2023-12-27 01:48:32,396][105692] Updated weights for policy 0, policy_version 1430258 (0.0008) [2023-12-27 01:48:32,454][105692] Updated weights for policy 0, policy_version 1430268 (0.0008) [2023-12-27 01:48:32,516][105692] Updated weights for policy 0, policy_version 1430278 (0.0009) [2023-12-27 01:48:32,571][105692] Updated weights for policy 0, policy_version 1430288 (0.0006) [2023-12-27 01:48:32,728][105620] Updated weights for policy 1, policy_version 1432522 (0.0009) [2023-12-27 01:48:32,779][105620] Updated weights for policy 1, policy_version 1432532 (0.0009) [2023-12-27 01:48:32,830][105620] Updated weights for policy 1, policy_version 1432542 (0.0009) [2023-12-27 01:48:32,881][105620] Updated weights for policy 1, policy_version 1432552 (0.0009) [2023-12-27 01:48:33,148][105692] Updated weights for policy 0, policy_version 1430298 (0.0008) [2023-12-27 01:48:33,195][105692] Updated weights for policy 0, policy_version 1430308 (0.0009) [2023-12-27 01:48:33,241][105692] Updated weights for policy 0, policy_version 1430318 (0.0009) [2023-12-27 01:48:33,686][105620] Updated weights for policy 1, policy_version 1432562 (0.0008) [2023-12-27 01:48:33,733][105620] Updated weights for policy 1, policy_version 1432572 (0.0009) [2023-12-27 01:48:33,787][105620] Updated weights for policy 1, policy_version 1432582 (0.0009) [2023-12-27 01:48:33,986][105692] Updated weights for policy 0, policy_version 1430328 (0.0009) [2023-12-27 01:48:34,040][105692] Updated weights for policy 0, policy_version 1430338 (0.0009) [2023-12-27 01:48:34,100][105692] Updated weights for policy 0, policy_version 1430348 (0.0009) [2023-12-27 01:48:34,554][105620] Updated weights for policy 1, policy_version 1432592 (0.0006) [2023-12-27 01:48:34,605][105620] Updated weights for policy 1, policy_version 1432602 (0.0005) [2023-12-27 01:48:34,668][105620] Updated weights for policy 1, policy_version 1432612 (0.0005) [2023-12-27 01:48:34,920][105692] Updated weights for policy 0, policy_version 1430358 (0.0008) [2023-12-27 01:48:34,978][105692] Updated weights for policy 0, policy_version 1430368 (0.0009) [2023-12-27 01:48:35,054][105692] Updated weights for policy 0, policy_version 1430378 (0.0009) [2023-12-27 01:48:35,318][105620] Updated weights for policy 1, policy_version 1432622 (0.0008) [2023-12-27 01:48:35,373][105620] Updated weights for policy 1, policy_version 1432633 (0.0009) [2023-12-27 01:48:35,426][105620] Updated weights for policy 1, policy_version 1432643 (0.0009) [2023-12-27 01:48:35,750][105692] Updated weights for policy 0, policy_version 1430388 (0.0009) [2023-12-27 01:48:35,809][105692] Updated weights for policy 0, policy_version 1430398 (0.0009) [2023-12-27 01:48:35,872][105692] Updated weights for policy 0, policy_version 1430408 (0.0009) [2023-12-27 01:48:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 733044736. Throughput: 0: 9879.8, 1: 9644.1. Samples: 733033404. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:48:36,063][104569] Avg episode reward: [(0, '7978.468'), (1, '8897.714')] [2023-12-27 01:48:36,168][105620] Updated weights for policy 1, policy_version 1432653 (0.0007) [2023-12-27 01:48:36,236][105620] Updated weights for policy 1, policy_version 1432663 (0.0008) [2023-12-27 01:48:36,300][105620] Updated weights for policy 1, policy_version 1432673 (0.0008) [2023-12-27 01:48:36,676][105692] Updated weights for policy 0, policy_version 1430418 (0.0009) [2023-12-27 01:48:36,725][105692] Updated weights for policy 0, policy_version 1430428 (0.0010) [2023-12-27 01:48:36,778][105692] Updated weights for policy 0, policy_version 1430438 (0.0010) [2023-12-27 01:48:36,834][105692] Updated weights for policy 0, policy_version 1430448 (0.0011) [2023-12-27 01:48:36,989][105620] Updated weights for policy 1, policy_version 1432683 (0.0007) [2023-12-27 01:48:37,049][105620] Updated weights for policy 1, policy_version 1432693 (0.0008) [2023-12-27 01:48:37,106][105620] Updated weights for policy 1, policy_version 1432703 (0.0008) [2023-12-27 01:48:37,601][105692] Updated weights for policy 0, policy_version 1430458 (0.0007) [2023-12-27 01:48:37,657][105692] Updated weights for policy 0, policy_version 1430468 (0.0007) [2023-12-27 01:48:37,714][105692] Updated weights for policy 0, policy_version 1430478 (0.0009) [2023-12-27 01:48:37,891][105620] Updated weights for policy 1, policy_version 1432713 (0.0008) [2023-12-27 01:48:37,943][105620] Updated weights for policy 1, policy_version 1432723 (0.0009) [2023-12-27 01:48:37,999][105620] Updated weights for policy 1, policy_version 1432733 (0.0009) [2023-12-27 01:48:38,052][105620] Updated weights for policy 1, policy_version 1432743 (0.0009) [2023-12-27 01:48:38,342][105692] Updated weights for policy 0, policy_version 1430488 (0.0008) [2023-12-27 01:48:38,403][105692] Updated weights for policy 0, policy_version 1430498 (0.0008) [2023-12-27 01:48:38,466][105692] Updated weights for policy 0, policy_version 1430508 (0.0008) [2023-12-27 01:48:38,773][105620] Updated weights for policy 1, policy_version 1432753 (0.0010) [2023-12-27 01:48:38,822][105620] Updated weights for policy 1, policy_version 1432763 (0.0010) [2023-12-27 01:48:38,874][105620] Updated weights for policy 1, policy_version 1432773 (0.0010) [2023-12-27 01:48:39,209][105692] Updated weights for policy 0, policy_version 1430518 (0.0009) [2023-12-27 01:48:39,275][105692] Updated weights for policy 0, policy_version 1430528 (0.0008) [2023-12-27 01:48:39,340][105692] Updated weights for policy 0, policy_version 1430538 (0.0008) [2023-12-27 01:48:39,634][105620] Updated weights for policy 1, policy_version 1432783 (0.0009) [2023-12-27 01:48:39,700][105620] Updated weights for policy 1, policy_version 1432793 (0.0010) [2023-12-27 01:48:39,763][105620] Updated weights for policy 1, policy_version 1432803 (0.0010) [2023-12-27 01:48:40,096][105692] Updated weights for policy 0, policy_version 1430548 (0.0007) [2023-12-27 01:48:40,157][105692] Updated weights for policy 0, policy_version 1430558 (0.0006) [2023-12-27 01:48:40,209][105692] Updated weights for policy 0, policy_version 1430568 (0.0006) [2023-12-27 01:48:40,452][105620] Updated weights for policy 1, policy_version 1432813 (0.0010) [2023-12-27 01:48:40,506][105620] Updated weights for policy 1, policy_version 1432823 (0.0010) [2023-12-27 01:48:40,561][105620] Updated weights for policy 1, policy_version 1432833 (0.0010) [2023-12-27 01:48:40,902][105692] Updated weights for policy 0, policy_version 1430578 (0.0007) [2023-12-27 01:48:40,968][105692] Updated weights for policy 0, policy_version 1430588 (0.0011) [2023-12-27 01:48:41,028][105692] Updated weights for policy 0, policy_version 1430598 (0.0011) [2023-12-27 01:48:41,062][104569] Fps is (10 sec: 18022.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 733134848. Throughput: 0: 9800.2, 1: 9621.9. Samples: 733148988. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:48:41,062][104569] Avg episode reward: [(0, '8164.770'), (1, '8988.075')] [2023-12-27 01:48:41,096][105692] Updated weights for policy 0, policy_version 1430608 (0.0012) [2023-12-27 01:48:41,238][105620] Updated weights for policy 1, policy_version 1432843 (0.0010) [2023-12-27 01:48:41,302][105620] Updated weights for policy 1, policy_version 1432853 (0.0007) [2023-12-27 01:48:41,372][105620] Updated weights for policy 1, policy_version 1432863 (0.0009) [2023-12-27 01:48:41,860][105692] Updated weights for policy 0, policy_version 1430618 (0.0011) [2023-12-27 01:48:41,922][105692] Updated weights for policy 0, policy_version 1430628 (0.0009) [2023-12-27 01:48:41,985][105692] Updated weights for policy 0, policy_version 1430638 (0.0011) [2023-12-27 01:48:42,027][105620] Updated weights for policy 1, policy_version 1432873 (0.0008) [2023-12-27 01:48:42,095][105620] Updated weights for policy 1, policy_version 1432883 (0.0005) [2023-12-27 01:48:42,164][105620] Updated weights for policy 1, policy_version 1432893 (0.0005) [2023-12-27 01:48:42,217][105620] Updated weights for policy 1, policy_version 1432903 (0.0010) [2023-12-27 01:48:42,681][105692] Updated weights for policy 0, policy_version 1430648 (0.0007) [2023-12-27 01:48:42,732][105692] Updated weights for policy 0, policy_version 1430658 (0.0006) [2023-12-27 01:48:42,796][105692] Updated weights for policy 0, policy_version 1430668 (0.0005) [2023-12-27 01:48:42,882][105620] Updated weights for policy 1, policy_version 1432913 (0.0009) [2023-12-27 01:48:42,941][105620] Updated weights for policy 1, policy_version 1432923 (0.0010) [2023-12-27 01:48:43,002][105620] Updated weights for policy 1, policy_version 1432933 (0.0010) [2023-12-27 01:48:43,507][105692] Updated weights for policy 0, policy_version 1430678 (0.0010) [2023-12-27 01:48:43,569][105692] Updated weights for policy 0, policy_version 1430688 (0.0010) [2023-12-27 01:48:43,627][105692] Updated weights for policy 0, policy_version 1430698 (0.0010) [2023-12-27 01:48:43,708][105620] Updated weights for policy 1, policy_version 1432943 (0.0010) [2023-12-27 01:48:43,757][105620] Updated weights for policy 1, policy_version 1432953 (0.0010) [2023-12-27 01:48:43,804][105620] Updated weights for policy 1, policy_version 1432963 (0.0010) [2023-12-27 01:48:44,189][105692] Updated weights for policy 0, policy_version 1430708 (0.0009) [2023-12-27 01:48:44,240][105692] Updated weights for policy 0, policy_version 1430718 (0.0005) [2023-12-27 01:48:44,297][105692] Updated weights for policy 0, policy_version 1430728 (0.0008) [2023-12-27 01:48:44,564][105620] Updated weights for policy 1, policy_version 1432973 (0.0008) [2023-12-27 01:48:44,618][105620] Updated weights for policy 1, policy_version 1432983 (0.0010) [2023-12-27 01:48:44,676][105620] Updated weights for policy 1, policy_version 1432993 (0.0010) [2023-12-27 01:48:45,016][105692] Updated weights for policy 0, policy_version 1430738 (0.0008) [2023-12-27 01:48:45,076][105692] Updated weights for policy 0, policy_version 1430748 (0.0009) [2023-12-27 01:48:45,137][105692] Updated weights for policy 0, policy_version 1430758 (0.0008) [2023-12-27 01:48:45,190][105692] Updated weights for policy 0, policy_version 1430768 (0.0008) [2023-12-27 01:48:45,419][105620] Updated weights for policy 1, policy_version 1433003 (0.0010) [2023-12-27 01:48:45,485][105620] Updated weights for policy 1, policy_version 1433013 (0.0011) [2023-12-27 01:48:45,562][105620] Updated weights for policy 1, policy_version 1433023 (0.0011) [2023-12-27 01:48:45,863][105692] Updated weights for policy 0, policy_version 1430778 (0.0011) [2023-12-27 01:48:45,921][105692] Updated weights for policy 0, policy_version 1430788 (0.0011) [2023-12-27 01:48:45,976][105692] Updated weights for policy 0, policy_version 1430798 (0.0011) [2023-12-27 01:48:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 733241344. Throughput: 0: 9691.6, 1: 9653.3. Samples: 733206628. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:48:46,062][104569] Avg episode reward: [(0, '8161.199'), (1, '9079.200')] [2023-12-27 01:48:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001433032_366903296.pth... [2023-12-27 01:48:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001430800_366338048.pth... [2023-12-27 01:48:46,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001431912_366616576.pth [2023-12-27 01:48:46,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001429648_366043136.pth [2023-12-27 01:48:46,198][105620] Updated weights for policy 1, policy_version 1433033 (0.0010) [2023-12-27 01:48:46,251][105620] Updated weights for policy 1, policy_version 1433043 (0.0007) [2023-12-27 01:48:46,299][105620] Updated weights for policy 1, policy_version 1433053 (0.0010) [2023-12-27 01:48:46,347][105620] Updated weights for policy 1, policy_version 1433063 (0.0010) [2023-12-27 01:48:46,639][105692] Updated weights for policy 0, policy_version 1430808 (0.0009) [2023-12-27 01:48:46,709][105692] Updated weights for policy 0, policy_version 1430818 (0.0008) [2023-12-27 01:48:46,771][105692] Updated weights for policy 0, policy_version 1430828 (0.0007) [2023-12-27 01:48:47,058][105620] Updated weights for policy 1, policy_version 1433073 (0.0010) [2023-12-27 01:48:47,107][105620] Updated weights for policy 1, policy_version 1433083 (0.0009) [2023-12-27 01:48:47,153][105620] Updated weights for policy 1, policy_version 1433093 (0.0005) [2023-12-27 01:48:47,384][105692] Updated weights for policy 0, policy_version 1430838 (0.0008) [2023-12-27 01:48:47,436][105692] Updated weights for policy 0, policy_version 1430848 (0.0011) [2023-12-27 01:48:47,486][105692] Updated weights for policy 0, policy_version 1430858 (0.0011) [2023-12-27 01:48:47,773][105620] Updated weights for policy 1, policy_version 1433103 (0.0005) [2023-12-27 01:48:47,834][105620] Updated weights for policy 1, policy_version 1433113 (0.0006) [2023-12-27 01:48:47,892][105620] Updated weights for policy 1, policy_version 1433123 (0.0006) [2023-12-27 01:48:48,226][105692] Updated weights for policy 0, policy_version 1430868 (0.0011) [2023-12-27 01:48:48,274][105692] Updated weights for policy 0, policy_version 1430878 (0.0010) [2023-12-27 01:48:48,319][105692] Updated weights for policy 0, policy_version 1430888 (0.0010) [2023-12-27 01:48:48,498][105620] Updated weights for policy 1, policy_version 1433133 (0.0008) [2023-12-27 01:48:48,560][105620] Updated weights for policy 1, policy_version 1433143 (0.0010) [2023-12-27 01:48:48,622][105620] Updated weights for policy 1, policy_version 1433153 (0.0010) [2023-12-27 01:48:48,964][105692] Updated weights for policy 0, policy_version 1430898 (0.0010) [2023-12-27 01:48:49,020][105692] Updated weights for policy 0, policy_version 1430908 (0.0007) [2023-12-27 01:48:49,074][105692] Updated weights for policy 0, policy_version 1430918 (0.0007) [2023-12-27 01:48:49,128][105692] Updated weights for policy 0, policy_version 1430928 (0.0008) [2023-12-27 01:48:49,305][105620] Updated weights for policy 1, policy_version 1433163 (0.0011) [2023-12-27 01:48:49,374][105620] Updated weights for policy 1, policy_version 1433173 (0.0009) [2023-12-27 01:48:49,427][105620] Updated weights for policy 1, policy_version 1433183 (0.0009) [2023-12-27 01:48:49,907][105692] Updated weights for policy 0, policy_version 1430938 (0.0008) [2023-12-27 01:48:49,977][105692] Updated weights for policy 0, policy_version 1430948 (0.0006) [2023-12-27 01:48:50,001][105585] KL-divergence is very high: 120.1616 [2023-12-27 01:48:50,009][105585] KL-divergence is very high: 155.9374 [2023-12-27 01:48:50,052][105692] Updated weights for policy 0, policy_version 1430958 (0.0006) [2023-12-27 01:48:50,063][105585] KL-divergence is very high: 229.1432 [2023-12-27 01:48:50,133][105620] Updated weights for policy 1, policy_version 1433193 (0.0009) [2023-12-27 01:48:50,184][105620] Updated weights for policy 1, policy_version 1433203 (0.0009) [2023-12-27 01:48:50,234][105620] Updated weights for policy 1, policy_version 1433213 (0.0009) [2023-12-27 01:48:50,291][105620] Updated weights for policy 1, policy_version 1433223 (0.0009) [2023-12-27 01:48:50,616][105692] Updated weights for policy 0, policy_version 1430968 (0.0009) [2023-12-27 01:48:50,676][105692] Updated weights for policy 0, policy_version 1430978 (0.0008) [2023-12-27 01:48:50,740][105692] Updated weights for policy 0, policy_version 1430988 (0.0011) [2023-12-27 01:48:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 733339648. Throughput: 0: 9788.0, 1: 9724.6. Samples: 733331244. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:48:51,062][104569] Avg episode reward: [(0, '8065.698'), (1, '8896.280')] [2023-12-27 01:48:51,111][105620] Updated weights for policy 1, policy_version 1433233 (0.0011) [2023-12-27 01:48:51,172][105620] Updated weights for policy 1, policy_version 1433243 (0.0011) [2023-12-27 01:48:51,232][105620] Updated weights for policy 1, policy_version 1433253 (0.0011) [2023-12-27 01:48:51,434][105692] Updated weights for policy 0, policy_version 1430998 (0.0009) [2023-12-27 01:48:51,485][105692] Updated weights for policy 0, policy_version 1431008 (0.0005) [2023-12-27 01:48:51,549][105692] Updated weights for policy 0, policy_version 1431018 (0.0008) [2023-12-27 01:48:51,996][105620] Updated weights for policy 1, policy_version 1433263 (0.0010) [2023-12-27 01:48:52,055][105620] Updated weights for policy 1, policy_version 1433273 (0.0010) [2023-12-27 01:48:52,114][105620] Updated weights for policy 1, policy_version 1433283 (0.0011) [2023-12-27 01:48:52,319][105692] Updated weights for policy 0, policy_version 1431028 (0.0008) [2023-12-27 01:48:52,373][105692] Updated weights for policy 0, policy_version 1431038 (0.0009) [2023-12-27 01:48:52,436][105692] Updated weights for policy 0, policy_version 1431048 (0.0008) [2023-12-27 01:48:52,871][105620] Updated weights for policy 1, policy_version 1433293 (0.0011) [2023-12-27 01:48:52,921][105620] Updated weights for policy 1, policy_version 1433303 (0.0010) [2023-12-27 01:48:52,973][105620] Updated weights for policy 1, policy_version 1433313 (0.0009) [2023-12-27 01:48:53,161][105692] Updated weights for policy 0, policy_version 1431058 (0.0005) [2023-12-27 01:48:53,225][105692] Updated weights for policy 0, policy_version 1431068 (0.0005) [2023-12-27 01:48:53,290][105692] Updated weights for policy 0, policy_version 1431078 (0.0006) [2023-12-27 01:48:53,356][105692] Updated weights for policy 0, policy_version 1431088 (0.0007) [2023-12-27 01:48:53,772][105620] Updated weights for policy 1, policy_version 1433323 (0.0009) [2023-12-27 01:48:53,840][105620] Updated weights for policy 1, policy_version 1433333 (0.0010) [2023-12-27 01:48:53,906][105620] Updated weights for policy 1, policy_version 1433343 (0.0007) [2023-12-27 01:48:53,967][105692] Updated weights for policy 0, policy_version 1431098 (0.0006) [2023-12-27 01:48:54,021][105692] Updated weights for policy 0, policy_version 1431108 (0.0010) [2023-12-27 01:48:54,075][105692] Updated weights for policy 0, policy_version 1431118 (0.0008) [2023-12-27 01:48:54,588][105620] Updated weights for policy 1, policy_version 1433353 (0.0010) [2023-12-27 01:48:54,650][105620] Updated weights for policy 1, policy_version 1433363 (0.0009) [2023-12-27 01:48:54,709][105620] Updated weights for policy 1, policy_version 1433373 (0.0008) [2023-12-27 01:48:54,770][105620] Updated weights for policy 1, policy_version 1433383 (0.0010) [2023-12-27 01:48:54,809][105692] Updated weights for policy 0, policy_version 1431128 (0.0006) [2023-12-27 01:48:54,862][105692] Updated weights for policy 0, policy_version 1431138 (0.0006) [2023-12-27 01:48:54,914][105692] Updated weights for policy 0, policy_version 1431148 (0.0005) [2023-12-27 01:48:55,505][105620] Updated weights for policy 1, policy_version 1433393 (0.0006) [2023-12-27 01:48:55,555][105620] Updated weights for policy 1, policy_version 1433403 (0.0005) [2023-12-27 01:48:55,580][105692] Updated weights for policy 0, policy_version 1431158 (0.0006) [2023-12-27 01:48:55,603][105620] Updated weights for policy 1, policy_version 1433413 (0.0007) [2023-12-27 01:48:55,630][105692] Updated weights for policy 0, policy_version 1431168 (0.0008) [2023-12-27 01:48:55,677][105692] Updated weights for policy 0, policy_version 1431178 (0.0008) [2023-12-27 01:48:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 733437952. Throughput: 0: 9862.6, 1: 9658.1. Samples: 733446484. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:48:56,062][104569] Avg episode reward: [(0, '7617.109'), (1, '8805.424')] [2023-12-27 01:48:56,284][105620] Updated weights for policy 1, policy_version 1433423 (0.0006) [2023-12-27 01:48:56,342][105620] Updated weights for policy 1, policy_version 1433433 (0.0008) [2023-12-27 01:48:56,399][105620] Updated weights for policy 1, policy_version 1433443 (0.0009) [2023-12-27 01:48:56,461][105692] Updated weights for policy 0, policy_version 1431188 (0.0010) [2023-12-27 01:48:56,512][105692] Updated weights for policy 0, policy_version 1431198 (0.0009) [2023-12-27 01:48:56,559][105692] Updated weights for policy 0, policy_version 1431208 (0.0009) [2023-12-27 01:48:56,975][105620] Updated weights for policy 1, policy_version 1433453 (0.0007) [2023-12-27 01:48:57,020][105620] Updated weights for policy 1, policy_version 1433463 (0.0009) [2023-12-27 01:48:57,068][105620] Updated weights for policy 1, policy_version 1433473 (0.0009) [2023-12-27 01:48:57,421][105692] Updated weights for policy 0, policy_version 1431219 (0.0008) [2023-12-27 01:48:57,472][105692] Updated weights for policy 0, policy_version 1431229 (0.0005) [2023-12-27 01:48:57,530][105692] Updated weights for policy 0, policy_version 1431239 (0.0005) [2023-12-27 01:48:57,737][105620] Updated weights for policy 1, policy_version 1433483 (0.0009) [2023-12-27 01:48:57,787][105620] Updated weights for policy 1, policy_version 1433494 (0.0008) [2023-12-27 01:48:57,833][105620] Updated weights for policy 1, policy_version 1433504 (0.0008) [2023-12-27 01:48:58,100][105692] Updated weights for policy 0, policy_version 1431249 (0.0005) [2023-12-27 01:48:58,161][105692] Updated weights for policy 0, policy_version 1431259 (0.0008) [2023-12-27 01:48:58,217][105692] Updated weights for policy 0, policy_version 1431269 (0.0008) [2023-12-27 01:48:58,269][105692] Updated weights for policy 0, policy_version 1431279 (0.0008) [2023-12-27 01:48:58,611][105620] Updated weights for policy 1, policy_version 1433514 (0.0008) [2023-12-27 01:48:58,662][105620] Updated weights for policy 1, policy_version 1433524 (0.0008) [2023-12-27 01:48:58,713][105620] Updated weights for policy 1, policy_version 1433534 (0.0009) [2023-12-27 01:48:58,774][105620] Updated weights for policy 1, policy_version 1433544 (0.0009) [2023-12-27 01:48:59,024][105692] Updated weights for policy 0, policy_version 1431289 (0.0008) [2023-12-27 01:48:59,070][105692] Updated weights for policy 0, policy_version 1431299 (0.0008) [2023-12-27 01:48:59,117][105692] Updated weights for policy 0, policy_version 1431309 (0.0009) [2023-12-27 01:48:59,525][105620] Updated weights for policy 1, policy_version 1433554 (0.0010) [2023-12-27 01:48:59,572][105620] Updated weights for policy 1, policy_version 1433564 (0.0009) [2023-12-27 01:48:59,629][105620] Updated weights for policy 1, policy_version 1433574 (0.0007) [2023-12-27 01:48:59,864][105692] Updated weights for policy 0, policy_version 1431319 (0.0008) [2023-12-27 01:48:59,927][105692] Updated weights for policy 0, policy_version 1431329 (0.0009) [2023-12-27 01:48:59,984][105692] Updated weights for policy 0, policy_version 1431339 (0.0009) [2023-12-27 01:49:00,274][105620] Updated weights for policy 1, policy_version 1433584 (0.0008) [2023-12-27 01:49:00,325][105620] Updated weights for policy 1, policy_version 1433594 (0.0009) [2023-12-27 01:49:00,383][105620] Updated weights for policy 1, policy_version 1433604 (0.0007) [2023-12-27 01:49:00,756][105692] Updated weights for policy 0, policy_version 1431349 (0.0007) [2023-12-27 01:49:00,807][105692] Updated weights for policy 0, policy_version 1431359 (0.0005) [2023-12-27 01:49:00,874][105692] Updated weights for policy 0, policy_version 1431369 (0.0005) [2023-12-27 01:49:00,999][105620] Updated weights for policy 1, policy_version 1433614 (0.0005) [2023-12-27 01:49:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 733536256. Throughput: 0: 9853.4, 1: 9727.4. Samples: 733505180. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:49:01,063][104569] Avg episode reward: [(0, '7982.391'), (1, '8988.619')] [2023-12-27 01:49:01,069][105620] Updated weights for policy 1, policy_version 1433624 (0.0007) [2023-12-27 01:49:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001431376_366485504.pth... [2023-12-27 01:49:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001430224_366190592.pth [2023-12-27 01:49:01,132][105620] Updated weights for policy 1, policy_version 1433634 (0.0006) [2023-12-27 01:49:01,169][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001433640_367058944.pth... [2023-12-27 01:49:01,175][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001432488_366764032.pth [2023-12-27 01:49:01,517][105692] Updated weights for policy 0, policy_version 1431379 (0.0007) [2023-12-27 01:49:01,569][105692] Updated weights for policy 0, policy_version 1431389 (0.0010) [2023-12-27 01:49:01,630][105692] Updated weights for policy 0, policy_version 1431399 (0.0009) [2023-12-27 01:49:01,871][105620] Updated weights for policy 1, policy_version 1433644 (0.0007) [2023-12-27 01:49:01,926][105620] Updated weights for policy 1, policy_version 1433654 (0.0008) [2023-12-27 01:49:01,986][105620] Updated weights for policy 1, policy_version 1433664 (0.0008) [2023-12-27 01:49:02,294][105692] Updated weights for policy 0, policy_version 1431409 (0.0006) [2023-12-27 01:49:02,357][105692] Updated weights for policy 0, policy_version 1431419 (0.0010) [2023-12-27 01:49:02,421][105692] Updated weights for policy 0, policy_version 1431429 (0.0010) [2023-12-27 01:49:02,481][105692] Updated weights for policy 0, policy_version 1431439 (0.0008) [2023-12-27 01:49:02,751][105620] Updated weights for policy 1, policy_version 1433674 (0.0009) [2023-12-27 01:49:02,809][105620] Updated weights for policy 1, policy_version 1433684 (0.0010) [2023-12-27 01:49:02,862][105620] Updated weights for policy 1, policy_version 1433694 (0.0009) [2023-12-27 01:49:02,919][105620] Updated weights for policy 1, policy_version 1433704 (0.0009) [2023-12-27 01:49:03,147][105692] Updated weights for policy 0, policy_version 1431449 (0.0006) [2023-12-27 01:49:03,204][105692] Updated weights for policy 0, policy_version 1431459 (0.0005) [2023-12-27 01:49:03,255][105692] Updated weights for policy 0, policy_version 1431469 (0.0006) [2023-12-27 01:49:03,598][105620] Updated weights for policy 1, policy_version 1433714 (0.0005) [2023-12-27 01:49:03,656][105620] Updated weights for policy 1, policy_version 1433724 (0.0005) [2023-12-27 01:49:03,698][105620] Updated weights for policy 1, policy_version 1433734 (0.0005) [2023-12-27 01:49:03,811][105692] Updated weights for policy 0, policy_version 1431479 (0.0005) [2023-12-27 01:49:03,875][105692] Updated weights for policy 0, policy_version 1431489 (0.0008) [2023-12-27 01:49:03,937][105692] Updated weights for policy 0, policy_version 1431499 (0.0007) [2023-12-27 01:49:04,364][105620] Updated weights for policy 1, policy_version 1433744 (0.0007) [2023-12-27 01:49:04,419][105620] Updated weights for policy 1, policy_version 1433754 (0.0008) [2023-12-27 01:49:04,473][105620] Updated weights for policy 1, policy_version 1433765 (0.0009) [2023-12-27 01:49:04,584][105692] Updated weights for policy 0, policy_version 1431509 (0.0005) [2023-12-27 01:49:04,631][105692] Updated weights for policy 0, policy_version 1431519 (0.0005) [2023-12-27 01:49:04,676][105692] Updated weights for policy 0, policy_version 1431529 (0.0005) [2023-12-27 01:49:05,302][105620] Updated weights for policy 1, policy_version 1433775 (0.0008) [2023-12-27 01:49:05,367][105692] Updated weights for policy 0, policy_version 1431539 (0.0009) [2023-12-27 01:49:05,371][105620] Updated weights for policy 1, policy_version 1433785 (0.0011) [2023-12-27 01:49:05,422][105692] Updated weights for policy 0, policy_version 1431549 (0.0011) [2023-12-27 01:49:05,427][105620] Updated weights for policy 1, policy_version 1433795 (0.0010) [2023-12-27 01:49:05,474][105692] Updated weights for policy 0, policy_version 1431559 (0.0010) [2023-12-27 01:49:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 733634560. Throughput: 0: 9973.0, 1: 9754.9. Samples: 733626660. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:49:06,062][104569] Avg episode reward: [(0, '8253.793'), (1, '9172.959')] [2023-12-27 01:49:06,153][105620] Updated weights for policy 1, policy_version 1433805 (0.0010) [2023-12-27 01:49:06,189][105692] Updated weights for policy 0, policy_version 1431569 (0.0009) [2023-12-27 01:49:06,205][105620] Updated weights for policy 1, policy_version 1433815 (0.0009) [2023-12-27 01:49:06,248][105692] Updated weights for policy 0, policy_version 1431579 (0.0011) [2023-12-27 01:49:06,257][105620] Updated weights for policy 1, policy_version 1433825 (0.0010) [2023-12-27 01:49:06,310][105692] Updated weights for policy 0, policy_version 1431589 (0.0011) [2023-12-27 01:49:06,373][105692] Updated weights for policy 0, policy_version 1431599 (0.0011) [2023-12-27 01:49:06,980][105620] Updated weights for policy 1, policy_version 1433835 (0.0009) [2023-12-27 01:49:07,037][105620] Updated weights for policy 1, policy_version 1433845 (0.0005) [2023-12-27 01:49:07,087][105620] Updated weights for policy 1, policy_version 1433855 (0.0005) [2023-12-27 01:49:07,153][105692] Updated weights for policy 0, policy_version 1431609 (0.0010) [2023-12-27 01:49:07,224][105692] Updated weights for policy 0, policy_version 1431619 (0.0006) [2023-12-27 01:49:07,296][105692] Updated weights for policy 0, policy_version 1431629 (0.0005) [2023-12-27 01:49:07,692][105620] Updated weights for policy 1, policy_version 1433865 (0.0006) [2023-12-27 01:49:07,745][105620] Updated weights for policy 1, policy_version 1433875 (0.0011) [2023-12-27 01:49:07,802][105620] Updated weights for policy 1, policy_version 1433885 (0.0011) [2023-12-27 01:49:07,849][105620] Updated weights for policy 1, policy_version 1433895 (0.0009) [2023-12-27 01:49:07,901][105692] Updated weights for policy 0, policy_version 1431639 (0.0006) [2023-12-27 01:49:07,959][105692] Updated weights for policy 0, policy_version 1431649 (0.0006) [2023-12-27 01:49:08,009][105692] Updated weights for policy 0, policy_version 1431659 (0.0005) [2023-12-27 01:49:08,492][105620] Updated weights for policy 1, policy_version 1433905 (0.0010) [2023-12-27 01:49:08,549][105620] Updated weights for policy 1, policy_version 1433915 (0.0011) [2023-12-27 01:49:08,557][105692] Updated weights for policy 0, policy_version 1431669 (0.0007) [2023-12-27 01:49:08,602][105620] Updated weights for policy 1, policy_version 1433925 (0.0011) [2023-12-27 01:49:08,627][105692] Updated weights for policy 0, policy_version 1431679 (0.0008) [2023-12-27 01:49:08,689][105692] Updated weights for policy 0, policy_version 1431689 (0.0008) [2023-12-27 01:49:09,385][105692] Updated weights for policy 0, policy_version 1431699 (0.0008) [2023-12-27 01:49:09,391][105620] Updated weights for policy 1, policy_version 1433935 (0.0009) [2023-12-27 01:49:09,450][105692] Updated weights for policy 0, policy_version 1431709 (0.0007) [2023-12-27 01:49:09,453][105620] Updated weights for policy 1, policy_version 1433945 (0.0007) [2023-12-27 01:49:09,507][105692] Updated weights for policy 0, policy_version 1431719 (0.0009) [2023-12-27 01:49:09,515][105620] Updated weights for policy 1, policy_version 1433955 (0.0007) [2023-12-27 01:49:10,238][105692] Updated weights for policy 0, policy_version 1431729 (0.0006) [2023-12-27 01:49:10,268][105620] Updated weights for policy 1, policy_version 1433965 (0.0008) [2023-12-27 01:49:10,292][105692] Updated weights for policy 0, policy_version 1431739 (0.0008) [2023-12-27 01:49:10,326][105620] Updated weights for policy 1, policy_version 1433975 (0.0008) [2023-12-27 01:49:10,341][105692] Updated weights for policy 0, policy_version 1431749 (0.0005) [2023-12-27 01:49:10,383][105620] Updated weights for policy 1, policy_version 1433985 (0.0007) [2023-12-27 01:49:10,393][105692] Updated weights for policy 0, policy_version 1431759 (0.0007) [2023-12-27 01:49:11,007][105692] Updated weights for policy 0, policy_version 1431769 (0.0007) [2023-12-27 01:49:11,021][105620] Updated weights for policy 1, policy_version 1433995 (0.0009) [2023-12-27 01:49:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 733732864. Throughput: 0: 10060.6, 1: 9753.6. Samples: 733747008. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:49:11,063][104569] Avg episode reward: [(0, '8620.782'), (1, '9357.418')] [2023-12-27 01:49:11,074][105692] Updated weights for policy 0, policy_version 1431779 (0.0009) [2023-12-27 01:49:11,084][105620] Updated weights for policy 1, policy_version 1434005 (0.0007) [2023-12-27 01:49:11,144][105692] Updated weights for policy 0, policy_version 1431789 (0.0010) [2023-12-27 01:49:11,146][105620] Updated weights for policy 1, policy_version 1434015 (0.0007) [2023-12-27 01:49:11,778][105692] Updated weights for policy 0, policy_version 1431799 (0.0007) [2023-12-27 01:49:11,834][105692] Updated weights for policy 0, policy_version 1431809 (0.0005) [2023-12-27 01:49:11,897][105692] Updated weights for policy 0, policy_version 1431819 (0.0008) [2023-12-27 01:49:11,956][105620] Updated weights for policy 1, policy_version 1434025 (0.0009) [2023-12-27 01:49:12,016][105620] Updated weights for policy 1, policy_version 1434035 (0.0008) [2023-12-27 01:49:12,075][105620] Updated weights for policy 1, policy_version 1434045 (0.0008) [2023-12-27 01:49:12,136][105620] Updated weights for policy 1, policy_version 1434055 (0.0008) [2023-12-27 01:49:12,578][105692] Updated weights for policy 0, policy_version 1431829 (0.0011) [2023-12-27 01:49:12,627][105692] Updated weights for policy 0, policy_version 1431839 (0.0006) [2023-12-27 01:49:12,690][105692] Updated weights for policy 0, policy_version 1431849 (0.0006) [2023-12-27 01:49:12,902][105620] Updated weights for policy 1, policy_version 1434065 (0.0011) [2023-12-27 01:49:12,965][105620] Updated weights for policy 1, policy_version 1434075 (0.0011) [2023-12-27 01:49:13,025][105620] Updated weights for policy 1, policy_version 1434085 (0.0010) [2023-12-27 01:49:13,230][105692] Updated weights for policy 0, policy_version 1431859 (0.0005) [2023-12-27 01:49:13,276][105692] Updated weights for policy 0, policy_version 1431869 (0.0005) [2023-12-27 01:49:13,324][105692] Updated weights for policy 0, policy_version 1431879 (0.0005) [2023-12-27 01:49:13,734][105620] Updated weights for policy 1, policy_version 1434095 (0.0010) [2023-12-27 01:49:13,786][105620] Updated weights for policy 1, policy_version 1434105 (0.0010) [2023-12-27 01:49:13,834][105620] Updated weights for policy 1, policy_version 1434115 (0.0010) [2023-12-27 01:49:13,842][105692] Updated weights for policy 0, policy_version 1431889 (0.0005) [2023-12-27 01:49:13,899][105692] Updated weights for policy 0, policy_version 1431899 (0.0005) [2023-12-27 01:49:13,956][105692] Updated weights for policy 0, policy_version 1431909 (0.0005) [2023-12-27 01:49:14,009][105692] Updated weights for policy 0, policy_version 1431919 (0.0010) [2023-12-27 01:49:14,601][105620] Updated weights for policy 1, policy_version 1434125 (0.0008) [2023-12-27 01:49:14,617][105692] Updated weights for policy 0, policy_version 1431929 (0.0009) [2023-12-27 01:49:14,657][105620] Updated weights for policy 1, policy_version 1434135 (0.0006) [2023-12-27 01:49:14,676][105692] Updated weights for policy 0, policy_version 1431939 (0.0009) [2023-12-27 01:49:14,712][105620] Updated weights for policy 1, policy_version 1434145 (0.0007) [2023-12-27 01:49:14,742][105692] Updated weights for policy 0, policy_version 1431949 (0.0007) [2023-12-27 01:49:15,412][105620] Updated weights for policy 1, policy_version 1434155 (0.0007) [2023-12-27 01:49:15,434][105692] Updated weights for policy 0, policy_version 1431959 (0.0008) [2023-12-27 01:49:15,467][105620] Updated weights for policy 1, policy_version 1434165 (0.0005) [2023-12-27 01:49:15,485][105692] Updated weights for policy 0, policy_version 1431969 (0.0009) [2023-12-27 01:49:15,521][105620] Updated weights for policy 1, policy_version 1434175 (0.0005) [2023-12-27 01:49:15,547][105692] Updated weights for policy 0, policy_version 1431979 (0.0007) [2023-12-27 01:49:16,062][104569] Fps is (10 sec: 20479.2, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 733839360. Throughput: 0: 10098.1, 1: 9690.4. Samples: 733808048. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:49:16,063][104569] Avg episode reward: [(0, '8621.013'), (1, '9080.708')] [2023-12-27 01:49:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001431984_366641152.pth... [2023-12-27 01:49:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001434184_367198208.pth... [2023-12-27 01:49:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001430800_366338048.pth [2023-12-27 01:49:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001433032_366903296.pth [2023-12-27 01:49:16,184][105620] Updated weights for policy 1, policy_version 1434185 (0.0009) [2023-12-27 01:49:16,237][105620] Updated weights for policy 1, policy_version 1434195 (0.0007) [2023-12-27 01:49:16,240][105692] Updated weights for policy 0, policy_version 1431989 (0.0007) [2023-12-27 01:49:16,295][105620] Updated weights for policy 1, policy_version 1434205 (0.0006) [2023-12-27 01:49:16,299][105692] Updated weights for policy 0, policy_version 1431999 (0.0008) [2023-12-27 01:49:16,355][105620] Updated weights for policy 1, policy_version 1434215 (0.0007) [2023-12-27 01:49:16,359][105692] Updated weights for policy 0, policy_version 1432009 (0.0005) [2023-12-27 01:49:17,058][105620] Updated weights for policy 1, policy_version 1434225 (0.0007) [2023-12-27 01:49:17,083][105692] Updated weights for policy 0, policy_version 1432019 (0.0007) [2023-12-27 01:49:17,119][105620] Updated weights for policy 1, policy_version 1434235 (0.0007) [2023-12-27 01:49:17,134][105692] Updated weights for policy 0, policy_version 1432029 (0.0007) [2023-12-27 01:49:17,169][105620] Updated weights for policy 1, policy_version 1434245 (0.0005) [2023-12-27 01:49:17,188][105692] Updated weights for policy 0, policy_version 1432039 (0.0008) [2023-12-27 01:49:17,771][105692] Updated weights for policy 0, policy_version 1432049 (0.0006) [2023-12-27 01:49:17,833][105692] Updated weights for policy 0, policy_version 1432059 (0.0007) [2023-12-27 01:49:17,835][105620] Updated weights for policy 1, policy_version 1434255 (0.0009) [2023-12-27 01:49:17,892][105620] Updated weights for policy 1, policy_version 1434265 (0.0011) [2023-12-27 01:49:17,901][105692] Updated weights for policy 0, policy_version 1432069 (0.0006) [2023-12-27 01:49:17,949][105620] Updated weights for policy 1, policy_version 1434275 (0.0011) [2023-12-27 01:49:17,966][105692] Updated weights for policy 0, policy_version 1432079 (0.0006) [2023-12-27 01:49:18,637][105620] Updated weights for policy 1, policy_version 1434285 (0.0008) [2023-12-27 01:49:18,646][105692] Updated weights for policy 0, policy_version 1432089 (0.0008) [2023-12-27 01:49:18,704][105620] Updated weights for policy 1, policy_version 1434295 (0.0006) [2023-12-27 01:49:18,706][105692] Updated weights for policy 0, policy_version 1432099 (0.0007) [2023-12-27 01:49:18,771][105620] Updated weights for policy 1, policy_version 1434305 (0.0009) [2023-12-27 01:49:18,772][105692] Updated weights for policy 0, policy_version 1432109 (0.0009) [2023-12-27 01:49:19,418][105692] Updated weights for policy 0, policy_version 1432119 (0.0007) [2023-12-27 01:49:19,476][105692] Updated weights for policy 0, policy_version 1432129 (0.0008) [2023-12-27 01:49:19,538][105692] Updated weights for policy 0, policy_version 1432139 (0.0009) [2023-12-27 01:49:19,629][105620] Updated weights for policy 1, policy_version 1434315 (0.0008) [2023-12-27 01:49:19,695][105620] Updated weights for policy 1, policy_version 1434325 (0.0006) [2023-12-27 01:49:19,767][105620] Updated weights for policy 1, policy_version 1434335 (0.0007) [2023-12-27 01:49:20,301][105692] Updated weights for policy 0, policy_version 1432149 (0.0007) [2023-12-27 01:49:20,368][105692] Updated weights for policy 0, policy_version 1432159 (0.0010) [2023-12-27 01:49:20,437][105692] Updated weights for policy 0, policy_version 1432169 (0.0011) [2023-12-27 01:49:20,516][105620] Updated weights for policy 1, policy_version 1434345 (0.0009) [2023-12-27 01:49:20,573][105620] Updated weights for policy 1, policy_version 1434355 (0.0008) [2023-12-27 01:49:20,641][105620] Updated weights for policy 1, policy_version 1434365 (0.0008) [2023-12-27 01:49:20,707][105620] Updated weights for policy 1, policy_version 1434375 (0.0008) [2023-12-27 01:49:21,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 733937664. Throughput: 0: 10098.9, 1: 9808.9. Samples: 733929256. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:49:21,062][104569] Avg episode reward: [(0, '8440.419'), (1, '8803.878')] [2023-12-27 01:49:21,157][105692] Updated weights for policy 0, policy_version 1432179 (0.0010) [2023-12-27 01:49:21,214][105692] Updated weights for policy 0, policy_version 1432189 (0.0008) [2023-12-27 01:49:21,279][105692] Updated weights for policy 0, policy_version 1432199 (0.0008) [2023-12-27 01:49:21,559][105620] Updated weights for policy 1, policy_version 1434385 (0.0009) [2023-12-27 01:49:21,626][105620] Updated weights for policy 1, policy_version 1434395 (0.0009) [2023-12-27 01:49:21,693][105620] Updated weights for policy 1, policy_version 1434405 (0.0009) [2023-12-27 01:49:22,063][105692] Updated weights for policy 0, policy_version 1432209 (0.0008) [2023-12-27 01:49:22,122][105692] Updated weights for policy 0, policy_version 1432219 (0.0009) [2023-12-27 01:49:22,188][105692] Updated weights for policy 0, policy_version 1432229 (0.0009) [2023-12-27 01:49:22,250][105692] Updated weights for policy 0, policy_version 1432239 (0.0009) [2023-12-27 01:49:22,453][105620] Updated weights for policy 1, policy_version 1434415 (0.0007) [2023-12-27 01:49:22,507][105620] Updated weights for policy 1, policy_version 1434425 (0.0005) [2023-12-27 01:49:22,568][105620] Updated weights for policy 1, policy_version 1434435 (0.0005) [2023-12-27 01:49:23,088][105692] Updated weights for policy 0, policy_version 1432249 (0.0010) [2023-12-27 01:49:23,137][105692] Updated weights for policy 0, policy_version 1432259 (0.0005) [2023-12-27 01:49:23,163][105620] Updated weights for policy 1, policy_version 1434445 (0.0008) [2023-12-27 01:49:23,188][105692] Updated weights for policy 0, policy_version 1432269 (0.0007) [2023-12-27 01:49:23,211][105620] Updated weights for policy 1, policy_version 1434455 (0.0008) [2023-12-27 01:49:23,258][105620] Updated weights for policy 1, policy_version 1434465 (0.0009) [2023-12-27 01:49:23,898][105692] Updated weights for policy 0, policy_version 1432279 (0.0008) [2023-12-27 01:49:23,957][105692] Updated weights for policy 0, policy_version 1432289 (0.0009) [2023-12-27 01:49:24,010][105692] Updated weights for policy 0, policy_version 1432299 (0.0010) [2023-12-27 01:49:24,039][105620] Updated weights for policy 1, policy_version 1434475 (0.0008) [2023-12-27 01:49:24,099][105620] Updated weights for policy 1, policy_version 1434485 (0.0008) [2023-12-27 01:49:24,155][105620] Updated weights for policy 1, policy_version 1434495 (0.0008) [2023-12-27 01:49:24,780][105692] Updated weights for policy 0, policy_version 1432309 (0.0009) [2023-12-27 01:49:24,838][105692] Updated weights for policy 0, policy_version 1432319 (0.0010) [2023-12-27 01:49:24,896][105692] Updated weights for policy 0, policy_version 1432329 (0.0010) [2023-12-27 01:49:24,913][105620] Updated weights for policy 1, policy_version 1434505 (0.0008) [2023-12-27 01:49:24,960][105620] Updated weights for policy 1, policy_version 1434515 (0.0007) [2023-12-27 01:49:25,004][105620] Updated weights for policy 1, policy_version 1434525 (0.0008) [2023-12-27 01:49:25,062][105620] Updated weights for policy 1, policy_version 1434535 (0.0008) [2023-12-27 01:49:25,535][105692] Updated weights for policy 0, policy_version 1432339 (0.0009) [2023-12-27 01:49:25,592][105692] Updated weights for policy 0, policy_version 1432349 (0.0008) [2023-12-27 01:49:25,655][105692] Updated weights for policy 0, policy_version 1432359 (0.0008) [2023-12-27 01:49:25,736][105620] Updated weights for policy 1, policy_version 1434545 (0.0006) [2023-12-27 01:49:25,789][105620] Updated weights for policy 1, policy_version 1434555 (0.0006) [2023-12-27 01:49:25,834][105620] Updated weights for policy 1, policy_version 1434565 (0.0008) [2023-12-27 01:49:26,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 734035968. Throughput: 0: 10079.9, 1: 9777.4. Samples: 734042568. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:49:26,062][104569] Avg episode reward: [(0, '8161.624'), (1, '8620.446')] [2023-12-27 01:49:26,386][105692] Updated weights for policy 0, policy_version 1432369 (0.0009) [2023-12-27 01:49:26,454][105692] Updated weights for policy 0, policy_version 1432379 (0.0008) [2023-12-27 01:49:26,457][105620] Updated weights for policy 1, policy_version 1434575 (0.0007) [2023-12-27 01:49:26,506][105620] Updated weights for policy 1, policy_version 1434585 (0.0005) [2023-12-27 01:49:26,516][105692] Updated weights for policy 0, policy_version 1432389 (0.0008) [2023-12-27 01:49:26,554][105620] Updated weights for policy 1, policy_version 1434595 (0.0005) [2023-12-27 01:49:26,565][105692] Updated weights for policy 0, policy_version 1432399 (0.0009) [2023-12-27 01:49:27,203][105620] Updated weights for policy 1, policy_version 1434605 (0.0006) [2023-12-27 01:49:27,212][105692] Updated weights for policy 0, policy_version 1432409 (0.0010) [2023-12-27 01:49:27,253][105620] Updated weights for policy 1, policy_version 1434615 (0.0009) [2023-12-27 01:49:27,269][105692] Updated weights for policy 0, policy_version 1432419 (0.0010) [2023-12-27 01:49:27,317][105620] Updated weights for policy 1, policy_version 1434625 (0.0010) [2023-12-27 01:49:27,327][105692] Updated weights for policy 0, policy_version 1432429 (0.0009) [2023-12-27 01:49:27,932][105620] Updated weights for policy 1, policy_version 1434635 (0.0009) [2023-12-27 01:49:27,941][105692] Updated weights for policy 0, policy_version 1432439 (0.0010) [2023-12-27 01:49:27,990][105620] Updated weights for policy 1, policy_version 1434645 (0.0005) [2023-12-27 01:49:27,992][105692] Updated weights for policy 0, policy_version 1432449 (0.0010) [2023-12-27 01:49:28,043][105692] Updated weights for policy 0, policy_version 1432459 (0.0010) [2023-12-27 01:49:28,047][105620] Updated weights for policy 1, policy_version 1434655 (0.0005) [2023-12-27 01:49:28,642][105620] Updated weights for policy 1, policy_version 1434665 (0.0007) [2023-12-27 01:49:28,696][105620] Updated weights for policy 1, policy_version 1434675 (0.0005) [2023-12-27 01:49:28,750][105620] Updated weights for policy 1, policy_version 1434685 (0.0005) [2023-12-27 01:49:28,809][105620] Updated weights for policy 1, policy_version 1434695 (0.0007) [2023-12-27 01:49:28,840][105692] Updated weights for policy 0, policy_version 1432469 (0.0010) [2023-12-27 01:49:28,896][105692] Updated weights for policy 0, policy_version 1432479 (0.0010) [2023-12-27 01:49:28,945][105692] Updated weights for policy 0, policy_version 1432489 (0.0010) [2023-12-27 01:49:29,556][105620] Updated weights for policy 1, policy_version 1434705 (0.0010) [2023-12-27 01:49:29,570][105692] Updated weights for policy 0, policy_version 1432499 (0.0010) [2023-12-27 01:49:29,616][105620] Updated weights for policy 1, policy_version 1434715 (0.0008) [2023-12-27 01:49:29,630][105692] Updated weights for policy 0, policy_version 1432509 (0.0007) [2023-12-27 01:49:29,671][105620] Updated weights for policy 1, policy_version 1434725 (0.0008) [2023-12-27 01:49:29,689][105692] Updated weights for policy 0, policy_version 1432519 (0.0007) [2023-12-27 01:49:30,388][105692] Updated weights for policy 0, policy_version 1432529 (0.0009) [2023-12-27 01:49:30,443][105692] Updated weights for policy 0, policy_version 1432539 (0.0010) [2023-12-27 01:49:30,456][105620] Updated weights for policy 1, policy_version 1434735 (0.0008) [2023-12-27 01:49:30,507][105692] Updated weights for policy 0, policy_version 1432549 (0.0010) [2023-12-27 01:49:30,520][105620] Updated weights for policy 1, policy_version 1434745 (0.0006) [2023-12-27 01:49:30,574][105692] Updated weights for policy 0, policy_version 1432559 (0.0006) [2023-12-27 01:49:30,575][105620] Updated weights for policy 1, policy_version 1434755 (0.0009) [2023-12-27 01:49:31,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 734134272. Throughput: 0: 10125.8, 1: 9862.2. Samples: 734106088. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:49:31,063][104569] Avg episode reward: [(0, '8248.416'), (1, '8804.919')] [2023-12-27 01:49:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001432560_366788608.pth... [2023-12-27 01:49:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001434760_367345664.pth... [2023-12-27 01:49:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001431376_366485504.pth [2023-12-27 01:49:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001433640_367058944.pth [2023-12-27 01:49:31,165][105692] Updated weights for policy 0, policy_version 1432569 (0.0012) [2023-12-27 01:49:31,220][105692] Updated weights for policy 0, policy_version 1432579 (0.0010) [2023-12-27 01:49:31,287][105692] Updated weights for policy 0, policy_version 1432589 (0.0011) [2023-12-27 01:49:31,395][105620] Updated weights for policy 1, policy_version 1434765 (0.0009) [2023-12-27 01:49:31,458][105620] Updated weights for policy 1, policy_version 1434775 (0.0008) [2023-12-27 01:49:31,517][105620] Updated weights for policy 1, policy_version 1434785 (0.0008) [2023-12-27 01:49:32,028][105692] Updated weights for policy 0, policy_version 1432599 (0.0010) [2023-12-27 01:49:32,093][105692] Updated weights for policy 0, policy_version 1432609 (0.0010) [2023-12-27 01:49:32,141][105692] Updated weights for policy 0, policy_version 1432619 (0.0010) [2023-12-27 01:49:32,279][105620] Updated weights for policy 1, policy_version 1434795 (0.0008) [2023-12-27 01:49:32,344][105620] Updated weights for policy 1, policy_version 1434805 (0.0006) [2023-12-27 01:49:32,405][105620] Updated weights for policy 1, policy_version 1434815 (0.0008) [2023-12-27 01:49:32,894][105692] Updated weights for policy 0, policy_version 1432629 (0.0010) [2023-12-27 01:49:32,948][105692] Updated weights for policy 0, policy_version 1432639 (0.0010) [2023-12-27 01:49:33,003][105692] Updated weights for policy 0, policy_version 1432649 (0.0010) [2023-12-27 01:49:33,012][105620] Updated weights for policy 1, policy_version 1434825 (0.0005) [2023-12-27 01:49:33,069][105620] Updated weights for policy 1, policy_version 1434835 (0.0007) [2023-12-27 01:49:33,130][105620] Updated weights for policy 1, policy_version 1434845 (0.0008) [2023-12-27 01:49:33,185][105620] Updated weights for policy 1, policy_version 1434855 (0.0008) [2023-12-27 01:49:33,755][105692] Updated weights for policy 0, policy_version 1432659 (0.0010) [2023-12-27 01:49:33,809][105692] Updated weights for policy 0, policy_version 1432669 (0.0010) [2023-12-27 01:49:33,856][105692] Updated weights for policy 0, policy_version 1432679 (0.0010) [2023-12-27 01:49:33,865][105620] Updated weights for policy 1, policy_version 1434865 (0.0006) [2023-12-27 01:49:33,912][105620] Updated weights for policy 1, policy_version 1434875 (0.0005) [2023-12-27 01:49:33,965][105620] Updated weights for policy 1, policy_version 1434885 (0.0005) [2023-12-27 01:49:34,611][105692] Updated weights for policy 0, policy_version 1432689 (0.0010) [2023-12-27 01:49:34,623][105620] Updated weights for policy 1, policy_version 1434895 (0.0005) [2023-12-27 01:49:34,675][105692] Updated weights for policy 0, policy_version 1432699 (0.0009) [2023-12-27 01:49:34,687][105620] Updated weights for policy 1, policy_version 1434905 (0.0006) [2023-12-27 01:49:34,738][105692] Updated weights for policy 0, policy_version 1432709 (0.0010) [2023-12-27 01:49:34,749][105620] Updated weights for policy 1, policy_version 1434915 (0.0006) [2023-12-27 01:49:34,797][105692] Updated weights for policy 0, policy_version 1432719 (0.0010) [2023-12-27 01:49:35,325][105620] Updated weights for policy 1, policy_version 1434925 (0.0006) [2023-12-27 01:49:35,387][105620] Updated weights for policy 1, policy_version 1434935 (0.0006) [2023-12-27 01:49:35,453][105620] Updated weights for policy 1, policy_version 1434945 (0.0005) [2023-12-27 01:49:35,499][105692] Updated weights for policy 0, policy_version 1432729 (0.0010) [2023-12-27 01:49:35,548][105692] Updated weights for policy 0, policy_version 1432739 (0.0010) [2023-12-27 01:49:35,596][105692] Updated weights for policy 0, policy_version 1432749 (0.0010) [2023-12-27 01:49:35,996][105620] Updated weights for policy 1, policy_version 1434955 (0.0007) [2023-12-27 01:49:36,058][105620] Updated weights for policy 1, policy_version 1434965 (0.0011) [2023-12-27 01:49:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 734232576. Throughput: 0: 10046.2, 1: 9769.0. Samples: 734222928. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:49:36,062][104569] Avg episode reward: [(0, '8341.650'), (1, '9080.693')] [2023-12-27 01:49:36,120][105620] Updated weights for policy 1, policy_version 1434975 (0.0011) [2023-12-27 01:49:36,338][105692] Updated weights for policy 0, policy_version 1432759 (0.0007) [2023-12-27 01:49:36,406][105692] Updated weights for policy 0, policy_version 1432769 (0.0006) [2023-12-27 01:49:36,471][105692] Updated weights for policy 0, policy_version 1432779 (0.0007) [2023-12-27 01:49:36,803][105620] Updated weights for policy 1, policy_version 1434985 (0.0010) [2023-12-27 01:49:36,864][105620] Updated weights for policy 1, policy_version 1434995 (0.0006) [2023-12-27 01:49:36,923][105620] Updated weights for policy 1, policy_version 1435005 (0.0006) [2023-12-27 01:49:36,980][105620] Updated weights for policy 1, policy_version 1435015 (0.0006) [2023-12-27 01:49:37,178][105692] Updated weights for policy 0, policy_version 1432789 (0.0008) [2023-12-27 01:49:37,232][105692] Updated weights for policy 0, policy_version 1432799 (0.0008) [2023-12-27 01:49:37,285][105692] Updated weights for policy 0, policy_version 1432809 (0.0007) [2023-12-27 01:49:37,614][105620] Updated weights for policy 1, policy_version 1435025 (0.0011) [2023-12-27 01:49:37,680][105620] Updated weights for policy 1, policy_version 1435035 (0.0011) [2023-12-27 01:49:37,745][105620] Updated weights for policy 1, policy_version 1435045 (0.0011) [2023-12-27 01:49:37,994][105692] Updated weights for policy 0, policy_version 1432819 (0.0007) [2023-12-27 01:49:38,063][105692] Updated weights for policy 0, policy_version 1432829 (0.0006) [2023-12-27 01:49:38,124][105692] Updated weights for policy 0, policy_version 1432839 (0.0006) [2023-12-27 01:49:38,376][105620] Updated weights for policy 1, policy_version 1435055 (0.0011) [2023-12-27 01:49:38,436][105620] Updated weights for policy 1, policy_version 1435065 (0.0011) [2023-12-27 01:49:38,492][105620] Updated weights for policy 1, policy_version 1435075 (0.0011) [2023-12-27 01:49:38,810][105692] Updated weights for policy 0, policy_version 1432849 (0.0006) [2023-12-27 01:49:38,860][105692] Updated weights for policy 0, policy_version 1432859 (0.0010) [2023-12-27 01:49:38,908][105692] Updated weights for policy 0, policy_version 1432869 (0.0010) [2023-12-27 01:49:38,957][105692] Updated weights for policy 0, policy_version 1432879 (0.0006) [2023-12-27 01:49:39,208][105620] Updated weights for policy 1, policy_version 1435085 (0.0009) [2023-12-27 01:49:39,275][105620] Updated weights for policy 1, policy_version 1435095 (0.0009) [2023-12-27 01:49:39,343][105620] Updated weights for policy 1, policy_version 1435105 (0.0010) [2023-12-27 01:49:39,685][105692] Updated weights for policy 0, policy_version 1432889 (0.0008) [2023-12-27 01:49:39,737][105692] Updated weights for policy 0, policy_version 1432899 (0.0009) [2023-12-27 01:49:39,797][105692] Updated weights for policy 0, policy_version 1432909 (0.0009) [2023-12-27 01:49:40,153][105620] Updated weights for policy 1, policy_version 1435115 (0.0008) [2023-12-27 01:49:40,215][105620] Updated weights for policy 1, policy_version 1435125 (0.0009) [2023-12-27 01:49:40,280][105620] Updated weights for policy 1, policy_version 1435135 (0.0008) [2023-12-27 01:49:40,530][105692] Updated weights for policy 0, policy_version 1432919 (0.0009) [2023-12-27 01:49:40,593][105692] Updated weights for policy 0, policy_version 1432929 (0.0007) [2023-12-27 01:49:40,662][105692] Updated weights for policy 0, policy_version 1432939 (0.0007) [2023-12-27 01:49:41,010][105620] Updated weights for policy 1, policy_version 1435145 (0.0009) [2023-12-27 01:49:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19933.8, 300 sec: 19633.0). Total num frames: 734330880. Throughput: 0: 10025.7, 1: 9882.5. Samples: 734342352. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:49:41,063][104569] Avg episode reward: [(0, '8064.942'), (1, '9172.892')] [2023-12-27 01:49:41,073][105620] Updated weights for policy 1, policy_version 1435155 (0.0009) [2023-12-27 01:49:41,151][105620] Updated weights for policy 1, policy_version 1435165 (0.0009) [2023-12-27 01:49:41,221][105620] Updated weights for policy 1, policy_version 1435175 (0.0009) [2023-12-27 01:49:41,296][105692] Updated weights for policy 0, policy_version 1432949 (0.0008) [2023-12-27 01:49:41,362][105692] Updated weights for policy 0, policy_version 1432959 (0.0007) [2023-12-27 01:49:41,423][105692] Updated weights for policy 0, policy_version 1432969 (0.0006) [2023-12-27 01:49:42,031][105620] Updated weights for policy 1, policy_version 1435185 (0.0008) [2023-12-27 01:49:42,084][105620] Updated weights for policy 1, policy_version 1435195 (0.0008) [2023-12-27 01:49:42,141][105620] Updated weights for policy 1, policy_version 1435205 (0.0008) [2023-12-27 01:49:42,147][105692] Updated weights for policy 0, policy_version 1432979 (0.0008) [2023-12-27 01:49:42,207][105692] Updated weights for policy 0, policy_version 1432989 (0.0011) [2023-12-27 01:49:42,276][105692] Updated weights for policy 0, policy_version 1432999 (0.0009) [2023-12-27 01:49:42,932][105692] Updated weights for policy 0, policy_version 1433009 (0.0008) [2023-12-27 01:49:42,966][105620] Updated weights for policy 1, policy_version 1435215 (0.0006) [2023-12-27 01:49:42,990][105692] Updated weights for policy 0, policy_version 1433019 (0.0010) [2023-12-27 01:49:43,024][105620] Updated weights for policy 1, policy_version 1435225 (0.0005) [2023-12-27 01:49:43,044][105692] Updated weights for policy 0, policy_version 1433029 (0.0009) [2023-12-27 01:49:43,075][105620] Updated weights for policy 1, policy_version 1435235 (0.0008) [2023-12-27 01:49:43,103][105692] Updated weights for policy 0, policy_version 1433039 (0.0005) [2023-12-27 01:49:43,727][105692] Updated weights for policy 0, policy_version 1433049 (0.0008) [2023-12-27 01:49:43,792][105692] Updated weights for policy 0, policy_version 1433059 (0.0006) [2023-12-27 01:49:43,816][105620] Updated weights for policy 1, policy_version 1435245 (0.0007) [2023-12-27 01:49:43,841][105692] Updated weights for policy 0, policy_version 1433069 (0.0005) [2023-12-27 01:49:43,870][105620] Updated weights for policy 1, policy_version 1435255 (0.0008) [2023-12-27 01:49:43,929][105620] Updated weights for policy 1, policy_version 1435266 (0.0010) [2023-12-27 01:49:44,432][105692] Updated weights for policy 0, policy_version 1433079 (0.0005) [2023-12-27 01:49:44,486][105692] Updated weights for policy 0, policy_version 1433089 (0.0005) [2023-12-27 01:49:44,542][105692] Updated weights for policy 0, policy_version 1433099 (0.0006) [2023-12-27 01:49:44,618][105620] Updated weights for policy 1, policy_version 1435276 (0.0007) [2023-12-27 01:49:44,678][105620] Updated weights for policy 1, policy_version 1435286 (0.0010) [2023-12-27 01:49:44,741][105620] Updated weights for policy 1, policy_version 1435296 (0.0011) [2023-12-27 01:49:45,181][105692] Updated weights for policy 0, policy_version 1433109 (0.0008) [2023-12-27 01:49:45,245][105692] Updated weights for policy 0, policy_version 1433119 (0.0010) [2023-12-27 01:49:45,317][105692] Updated weights for policy 0, policy_version 1433129 (0.0010) [2023-12-27 01:49:45,371][105620] Updated weights for policy 1, policy_version 1435306 (0.0010) [2023-12-27 01:49:45,425][105620] Updated weights for policy 1, policy_version 1435316 (0.0009) [2023-12-27 01:49:45,474][105620] Updated weights for policy 1, policy_version 1435326 (0.0009) [2023-12-27 01:49:45,520][105620] Updated weights for policy 1, policy_version 1435336 (0.0008) [2023-12-27 01:49:46,052][105692] Updated weights for policy 0, policy_version 1433139 (0.0009) [2023-12-27 01:49:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 734429184. Throughput: 0: 10079.8, 1: 9810.8. Samples: 734400260. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:49:46,063][104569] Avg episode reward: [(0, '8708.463'), (1, '9086.105')] [2023-12-27 01:49:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001435336_367493120.pth... [2023-12-27 01:49:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001434184_367198208.pth [2023-12-27 01:49:46,107][105692] Updated weights for policy 0, policy_version 1433149 (0.0009) [2023-12-27 01:49:46,163][105692] Updated weights for policy 0, policy_version 1433159 (0.0009) [2023-12-27 01:49:46,209][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001433168_366944256.pth... [2023-12-27 01:49:46,213][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001431984_366641152.pth [2023-12-27 01:49:46,257][105620] Updated weights for policy 1, policy_version 1435346 (0.0008) [2023-12-27 01:49:46,316][105620] Updated weights for policy 1, policy_version 1435356 (0.0008) [2023-12-27 01:49:46,374][105620] Updated weights for policy 1, policy_version 1435366 (0.0009) [2023-12-27 01:49:46,907][105692] Updated weights for policy 0, policy_version 1433169 (0.0010) [2023-12-27 01:49:46,958][105692] Updated weights for policy 0, policy_version 1433179 (0.0007) [2023-12-27 01:49:47,006][105692] Updated weights for policy 0, policy_version 1433189 (0.0009) [2023-12-27 01:49:47,056][105692] Updated weights for policy 0, policy_version 1433199 (0.0008) [2023-12-27 01:49:47,134][105620] Updated weights for policy 1, policy_version 1435376 (0.0009) [2023-12-27 01:49:47,195][105620] Updated weights for policy 1, policy_version 1435386 (0.0009) [2023-12-27 01:49:47,257][105620] Updated weights for policy 1, policy_version 1435396 (0.0009) [2023-12-27 01:49:47,813][105692] Updated weights for policy 0, policy_version 1433209 (0.0009) [2023-12-27 01:49:47,882][105692] Updated weights for policy 0, policy_version 1433219 (0.0007) [2023-12-27 01:49:47,942][105692] Updated weights for policy 0, policy_version 1433229 (0.0005) [2023-12-27 01:49:47,972][105620] Updated weights for policy 1, policy_version 1435406 (0.0009) [2023-12-27 01:49:48,025][105620] Updated weights for policy 1, policy_version 1435416 (0.0009) [2023-12-27 01:49:48,076][105620] Updated weights for policy 1, policy_version 1435426 (0.0009) [2023-12-27 01:49:48,614][105692] Updated weights for policy 0, policy_version 1433239 (0.0007) [2023-12-27 01:49:48,666][105692] Updated weights for policy 0, policy_version 1433249 (0.0008) [2023-12-27 01:49:48,714][105692] Updated weights for policy 0, policy_version 1433259 (0.0008) [2023-12-27 01:49:48,884][105620] Updated weights for policy 1, policy_version 1435437 (0.0010) [2023-12-27 01:49:48,944][105620] Updated weights for policy 1, policy_version 1435447 (0.0009) [2023-12-27 01:49:49,007][105620] Updated weights for policy 1, policy_version 1435457 (0.0006) [2023-12-27 01:49:49,482][105692] Updated weights for policy 0, policy_version 1433269 (0.0007) [2023-12-27 01:49:49,531][105692] Updated weights for policy 0, policy_version 1433279 (0.0008) [2023-12-27 01:49:49,584][105692] Updated weights for policy 0, policy_version 1433289 (0.0008) [2023-12-27 01:49:49,725][105620] Updated weights for policy 1, policy_version 1435467 (0.0007) [2023-12-27 01:49:49,784][105620] Updated weights for policy 1, policy_version 1435477 (0.0010) [2023-12-27 01:49:49,840][105620] Updated weights for policy 1, policy_version 1435487 (0.0009) [2023-12-27 01:49:50,404][105692] Updated weights for policy 0, policy_version 1433299 (0.0009) [2023-12-27 01:49:50,460][105692] Updated weights for policy 0, policy_version 1433309 (0.0006) [2023-12-27 01:49:50,468][105620] Updated weights for policy 1, policy_version 1435497 (0.0008) [2023-12-27 01:49:50,519][105692] Updated weights for policy 0, policy_version 1433319 (0.0006) [2023-12-27 01:49:50,532][105620] Updated weights for policy 1, policy_version 1435507 (0.0007) [2023-12-27 01:49:50,597][105620] Updated weights for policy 1, policy_version 1435517 (0.0008) [2023-12-27 01:49:50,653][105620] Updated weights for policy 1, policy_version 1435527 (0.0009) [2023-12-27 01:49:51,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 734527488. Throughput: 0: 10029.1, 1: 9764.0. Samples: 734517348. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:49:51,062][104569] Avg episode reward: [(0, '8616.748'), (1, '8904.775')] [2023-12-27 01:49:51,210][105692] Updated weights for policy 0, policy_version 1433329 (0.0006) [2023-12-27 01:49:51,267][105692] Updated weights for policy 0, policy_version 1433339 (0.0008) [2023-12-27 01:49:51,331][105692] Updated weights for policy 0, policy_version 1433349 (0.0009) [2023-12-27 01:49:51,397][105692] Updated weights for policy 0, policy_version 1433359 (0.0008) [2023-12-27 01:49:51,442][105620] Updated weights for policy 1, policy_version 1435537 (0.0009) [2023-12-27 01:49:51,501][105620] Updated weights for policy 1, policy_version 1435547 (0.0008) [2023-12-27 01:49:51,557][105620] Updated weights for policy 1, policy_version 1435557 (0.0005) [2023-12-27 01:49:52,232][105620] Updated weights for policy 1, policy_version 1435567 (0.0007) [2023-12-27 01:49:52,252][105692] Updated weights for policy 0, policy_version 1433369 (0.0006) [2023-12-27 01:49:52,290][105620] Updated weights for policy 1, policy_version 1435577 (0.0008) [2023-12-27 01:49:52,321][105692] Updated weights for policy 0, policy_version 1433379 (0.0009) [2023-12-27 01:49:52,345][105620] Updated weights for policy 1, policy_version 1435587 (0.0008) [2023-12-27 01:49:52,385][105692] Updated weights for policy 0, policy_version 1433389 (0.0008) [2023-12-27 01:49:53,036][105692] Updated weights for policy 0, policy_version 1433399 (0.0006) [2023-12-27 01:49:53,105][105692] Updated weights for policy 0, policy_version 1433409 (0.0005) [2023-12-27 01:49:53,140][105620] Updated weights for policy 1, policy_version 1435597 (0.0009) [2023-12-27 01:49:53,170][105692] Updated weights for policy 0, policy_version 1433419 (0.0006) [2023-12-27 01:49:53,194][105620] Updated weights for policy 1, policy_version 1435607 (0.0006) [2023-12-27 01:49:53,254][105620] Updated weights for policy 1, policy_version 1435617 (0.0005) [2023-12-27 01:49:53,684][105692] Updated weights for policy 0, policy_version 1433429 (0.0006) [2023-12-27 01:49:53,752][105692] Updated weights for policy 0, policy_version 1433439 (0.0005) [2023-12-27 01:49:53,792][105620] Updated weights for policy 1, policy_version 1435627 (0.0006) [2023-12-27 01:49:53,809][105692] Updated weights for policy 0, policy_version 1433449 (0.0005) [2023-12-27 01:49:53,848][105620] Updated weights for policy 1, policy_version 1435637 (0.0007) [2023-12-27 01:49:53,904][105620] Updated weights for policy 1, policy_version 1435647 (0.0010) [2023-12-27 01:49:54,379][105692] Updated weights for policy 0, policy_version 1433459 (0.0007) [2023-12-27 01:49:54,434][105692] Updated weights for policy 0, policy_version 1433469 (0.0008) [2023-12-27 01:49:54,485][105692] Updated weights for policy 0, policy_version 1433479 (0.0006) [2023-12-27 01:49:54,496][105620] Updated weights for policy 1, policy_version 1435657 (0.0007) [2023-12-27 01:49:54,549][105620] Updated weights for policy 1, policy_version 1435667 (0.0005) [2023-12-27 01:49:54,601][105620] Updated weights for policy 1, policy_version 1435677 (0.0005) [2023-12-27 01:49:54,659][105620] Updated weights for policy 1, policy_version 1435687 (0.0005) [2023-12-27 01:49:55,186][105620] Updated weights for policy 1, policy_version 1435697 (0.0006) [2023-12-27 01:49:55,249][105620] Updated weights for policy 1, policy_version 1435707 (0.0008) [2023-12-27 01:49:55,316][105620] Updated weights for policy 1, policy_version 1435717 (0.0005) [2023-12-27 01:49:55,356][105692] Updated weights for policy 0, policy_version 1433489 (0.0009) [2023-12-27 01:49:55,411][105692] Updated weights for policy 0, policy_version 1433499 (0.0009) [2023-12-27 01:49:55,474][105692] Updated weights for policy 0, policy_version 1433509 (0.0007) [2023-12-27 01:49:55,541][105692] Updated weights for policy 0, policy_version 1433519 (0.0008) [2023-12-27 01:49:55,886][105620] Updated weights for policy 1, policy_version 1435727 (0.0005) [2023-12-27 01:49:55,931][105620] Updated weights for policy 1, policy_version 1435737 (0.0005) [2023-12-27 01:49:55,982][105620] Updated weights for policy 1, policy_version 1435747 (0.0005) [2023-12-27 01:49:56,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19933.9, 300 sec: 19660.8). Total num frames: 734633984. Throughput: 0: 9973.5, 1: 9868.5. Samples: 734639896. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:49:56,062][104569] Avg episode reward: [(0, '8068.047'), (1, '9086.504')] [2023-12-27 01:49:56,364][105692] Updated weights for policy 0, policy_version 1433529 (0.0006) [2023-12-27 01:49:56,422][105692] Updated weights for policy 0, policy_version 1433539 (0.0006) [2023-12-27 01:49:56,473][105692] Updated weights for policy 0, policy_version 1433549 (0.0006) [2023-12-27 01:49:56,559][105620] Updated weights for policy 1, policy_version 1435757 (0.0008) [2023-12-27 01:49:56,607][105620] Updated weights for policy 1, policy_version 1435767 (0.0010) [2023-12-27 01:49:56,663][105620] Updated weights for policy 1, policy_version 1435777 (0.0011) [2023-12-27 01:49:57,085][105692] Updated weights for policy 0, policy_version 1433559 (0.0005) [2023-12-27 01:49:57,137][105692] Updated weights for policy 0, policy_version 1433569 (0.0006) [2023-12-27 01:49:57,185][105692] Updated weights for policy 0, policy_version 1433579 (0.0010) [2023-12-27 01:49:57,324][105620] Updated weights for policy 1, policy_version 1435787 (0.0011) [2023-12-27 01:49:57,389][105620] Updated weights for policy 1, policy_version 1435797 (0.0008) [2023-12-27 01:49:57,454][105620] Updated weights for policy 1, policy_version 1435807 (0.0008) [2023-12-27 01:49:57,798][105692] Updated weights for policy 0, policy_version 1433589 (0.0009) [2023-12-27 01:49:57,851][105692] Updated weights for policy 0, policy_version 1433599 (0.0006) [2023-12-27 01:49:57,918][105692] Updated weights for policy 0, policy_version 1433609 (0.0005) [2023-12-27 01:49:58,152][105620] Updated weights for policy 1, policy_version 1435817 (0.0009) [2023-12-27 01:49:58,222][105620] Updated weights for policy 1, policy_version 1435827 (0.0010) [2023-12-27 01:49:58,285][105620] Updated weights for policy 1, policy_version 1435837 (0.0011) [2023-12-27 01:49:58,349][105620] Updated weights for policy 1, policy_version 1435847 (0.0009) [2023-12-27 01:49:58,563][105692] Updated weights for policy 0, policy_version 1433619 (0.0007) [2023-12-27 01:49:58,624][105692] Updated weights for policy 0, policy_version 1433629 (0.0008) [2023-12-27 01:49:58,688][105692] Updated weights for policy 0, policy_version 1433639 (0.0008) [2023-12-27 01:49:59,102][105620] Updated weights for policy 1, policy_version 1435857 (0.0009) [2023-12-27 01:49:59,157][105620] Updated weights for policy 1, policy_version 1435867 (0.0011) [2023-12-27 01:49:59,222][105620] Updated weights for policy 1, policy_version 1435877 (0.0011) [2023-12-27 01:49:59,502][105692] Updated weights for policy 0, policy_version 1433649 (0.0009) [2023-12-27 01:49:59,565][105692] Updated weights for policy 0, policy_version 1433659 (0.0008) [2023-12-27 01:49:59,613][105692] Updated weights for policy 0, policy_version 1433669 (0.0008) [2023-12-27 01:49:59,657][105692] Updated weights for policy 0, policy_version 1433679 (0.0008) [2023-12-27 01:49:59,974][105620] Updated weights for policy 1, policy_version 1435887 (0.0010) [2023-12-27 01:50:00,037][105620] Updated weights for policy 1, policy_version 1435897 (0.0011) [2023-12-27 01:50:00,099][105620] Updated weights for policy 1, policy_version 1435907 (0.0010) [2023-12-27 01:50:00,386][105692] Updated weights for policy 0, policy_version 1433689 (0.0008) [2023-12-27 01:50:00,447][105692] Updated weights for policy 0, policy_version 1433699 (0.0008) [2023-12-27 01:50:00,519][105692] Updated weights for policy 0, policy_version 1433709 (0.0010) [2023-12-27 01:50:00,750][105620] Updated weights for policy 1, policy_version 1435917 (0.0008) [2023-12-27 01:50:00,805][105620] Updated weights for policy 1, policy_version 1435927 (0.0008) [2023-12-27 01:50:00,860][105620] Updated weights for policy 1, policy_version 1435937 (0.0010) [2023-12-27 01:50:01,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19933.9, 300 sec: 19660.8). Total num frames: 734732288. Throughput: 0: 9923.3, 1: 9939.4. Samples: 734701868. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:50:01,063][104569] Avg episode reward: [(0, '8442.086'), (1, '8990.270')] [2023-12-27 01:50:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001433712_367083520.pth... [2023-12-27 01:50:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001435944_367648768.pth... [2023-12-27 01:50:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001432560_366788608.pth [2023-12-27 01:50:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001434760_367345664.pth [2023-12-27 01:50:01,167][105692] Updated weights for policy 0, policy_version 1433720 (0.0010) [2023-12-27 01:50:01,219][105692] Updated weights for policy 0, policy_version 1433730 (0.0011) [2023-12-27 01:50:01,282][105692] Updated weights for policy 0, policy_version 1433740 (0.0011) [2023-12-27 01:50:01,566][105620] Updated weights for policy 1, policy_version 1435947 (0.0009) [2023-12-27 01:50:01,627][105620] Updated weights for policy 1, policy_version 1435957 (0.0010) [2023-12-27 01:50:01,690][105620] Updated weights for policy 1, policy_version 1435967 (0.0009) [2023-12-27 01:50:02,025][105692] Updated weights for policy 0, policy_version 1433750 (0.0010) [2023-12-27 01:50:02,088][105692] Updated weights for policy 0, policy_version 1433760 (0.0005) [2023-12-27 01:50:02,143][105692] Updated weights for policy 0, policy_version 1433770 (0.0005) [2023-12-27 01:50:02,469][105620] Updated weights for policy 1, policy_version 1435977 (0.0009) [2023-12-27 01:50:02,529][105620] Updated weights for policy 1, policy_version 1435987 (0.0007) [2023-12-27 01:50:02,585][105620] Updated weights for policy 1, policy_version 1435997 (0.0005) [2023-12-27 01:50:02,640][105620] Updated weights for policy 1, policy_version 1436007 (0.0008) [2023-12-27 01:50:02,704][105692] Updated weights for policy 0, policy_version 1433780 (0.0005) [2023-12-27 01:50:02,767][105692] Updated weights for policy 0, policy_version 1433790 (0.0007) [2023-12-27 01:50:02,830][105692] Updated weights for policy 0, policy_version 1433800 (0.0008) [2023-12-27 01:50:03,409][105620] Updated weights for policy 1, policy_version 1436017 (0.0006) [2023-12-27 01:50:03,424][105692] Updated weights for policy 0, policy_version 1433810 (0.0006) [2023-12-27 01:50:03,462][105620] Updated weights for policy 1, policy_version 1436027 (0.0005) [2023-12-27 01:50:03,486][105692] Updated weights for policy 0, policy_version 1433820 (0.0007) [2023-12-27 01:50:03,513][105620] Updated weights for policy 1, policy_version 1436037 (0.0005) [2023-12-27 01:50:03,547][105692] Updated weights for policy 0, policy_version 1433830 (0.0008) [2023-12-27 01:50:03,600][105692] Updated weights for policy 0, policy_version 1433840 (0.0009) [2023-12-27 01:50:04,236][105620] Updated weights for policy 1, policy_version 1436047 (0.0008) [2023-12-27 01:50:04,289][105620] Updated weights for policy 1, policy_version 1436057 (0.0009) [2023-12-27 01:50:04,350][105620] Updated weights for policy 1, policy_version 1436067 (0.0006) [2023-12-27 01:50:04,352][105692] Updated weights for policy 0, policy_version 1433850 (0.0008) [2023-12-27 01:50:04,412][105692] Updated weights for policy 0, policy_version 1433860 (0.0008) [2023-12-27 01:50:04,467][105692] Updated weights for policy 0, policy_version 1433870 (0.0008) [2023-12-27 01:50:05,027][105620] Updated weights for policy 1, policy_version 1436077 (0.0007) [2023-12-27 01:50:05,070][105620] Updated weights for policy 1, policy_version 1436087 (0.0005) [2023-12-27 01:50:05,123][105620] Updated weights for policy 1, policy_version 1436097 (0.0005) [2023-12-27 01:50:05,272][105692] Updated weights for policy 0, policy_version 1433880 (0.0007) [2023-12-27 01:50:05,318][105692] Updated weights for policy 0, policy_version 1433890 (0.0008) [2023-12-27 01:50:05,380][105692] Updated weights for policy 0, policy_version 1433900 (0.0009) [2023-12-27 01:50:05,826][105620] Updated weights for policy 1, policy_version 1436107 (0.0007) [2023-12-27 01:50:05,892][105620] Updated weights for policy 1, policy_version 1436117 (0.0005) [2023-12-27 01:50:05,951][105620] Updated weights for policy 1, policy_version 1436127 (0.0005) [2023-12-27 01:50:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19660.8). Total num frames: 734830592. Throughput: 0: 9860.0, 1: 9916.4. Samples: 734819196. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:50:06,062][104569] Avg episode reward: [(0, '8714.061'), (1, '8808.193')] [2023-12-27 01:50:06,192][105692] Updated weights for policy 0, policy_version 1433910 (0.0008) [2023-12-27 01:50:06,255][105692] Updated weights for policy 0, policy_version 1433920 (0.0008) [2023-12-27 01:50:06,322][105692] Updated weights for policy 0, policy_version 1433930 (0.0008) [2023-12-27 01:50:06,604][105620] Updated weights for policy 1, policy_version 1436137 (0.0006) [2023-12-27 01:50:06,660][105620] Updated weights for policy 1, policy_version 1436147 (0.0011) [2023-12-27 01:50:06,713][105620] Updated weights for policy 1, policy_version 1436157 (0.0010) [2023-12-27 01:50:06,771][105620] Updated weights for policy 1, policy_version 1436167 (0.0010) [2023-12-27 01:50:07,091][105692] Updated weights for policy 0, policy_version 1433940 (0.0009) [2023-12-27 01:50:07,142][105692] Updated weights for policy 0, policy_version 1433950 (0.0008) [2023-12-27 01:50:07,197][105692] Updated weights for policy 0, policy_version 1433960 (0.0006) [2023-12-27 01:50:07,418][105620] Updated weights for policy 1, policy_version 1436177 (0.0008) [2023-12-27 01:50:07,479][105620] Updated weights for policy 1, policy_version 1436187 (0.0008) [2023-12-27 01:50:07,539][105620] Updated weights for policy 1, policy_version 1436197 (0.0008) [2023-12-27 01:50:07,554][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000009 [2023-12-27 01:50:07,888][105692] Updated weights for policy 0, policy_version 1433970 (0.0005) [2023-12-27 01:50:07,934][105692] Updated weights for policy 0, policy_version 1433980 (0.0009) [2023-12-27 01:50:07,989][105692] Updated weights for policy 0, policy_version 1433990 (0.0010) [2023-12-27 01:50:08,040][105692] Updated weights for policy 0, policy_version 1434000 (0.0010) [2023-12-27 01:50:08,284][105620] Updated weights for policy 1, policy_version 1436207 (0.0009) [2023-12-27 01:50:08,345][105620] Updated weights for policy 1, policy_version 1436217 (0.0009) [2023-12-27 01:50:08,412][105620] Updated weights for policy 1, policy_version 1436227 (0.0008) [2023-12-27 01:50:08,810][105692] Updated weights for policy 0, policy_version 1434010 (0.0008) [2023-12-27 01:50:08,869][105692] Updated weights for policy 0, policy_version 1434020 (0.0008) [2023-12-27 01:50:08,921][105692] Updated weights for policy 0, policy_version 1434030 (0.0008) [2023-12-27 01:50:09,158][105620] Updated weights for policy 1, policy_version 1436237 (0.0011) [2023-12-27 01:50:09,224][105620] Updated weights for policy 1, policy_version 1436247 (0.0010) [2023-12-27 01:50:09,290][105620] Updated weights for policy 1, policy_version 1436257 (0.0010) [2023-12-27 01:50:09,672][105692] Updated weights for policy 0, policy_version 1434040 (0.0009) [2023-12-27 01:50:09,739][105692] Updated weights for policy 0, policy_version 1434050 (0.0009) [2023-12-27 01:50:09,802][105692] Updated weights for policy 0, policy_version 1434060 (0.0010) [2023-12-27 01:50:09,982][105620] Updated weights for policy 1, policy_version 1436267 (0.0010) [2023-12-27 01:50:10,045][105620] Updated weights for policy 1, policy_version 1436277 (0.0007) [2023-12-27 01:50:10,110][105620] Updated weights for policy 1, policy_version 1436287 (0.0008) [2023-12-27 01:50:10,590][105692] Updated weights for policy 0, policy_version 1434070 (0.0007) [2023-12-27 01:50:10,652][105692] Updated weights for policy 0, policy_version 1434080 (0.0006) [2023-12-27 01:50:10,708][105692] Updated weights for policy 0, policy_version 1434090 (0.0006) [2023-12-27 01:50:10,779][105620] Updated weights for policy 1, policy_version 1436297 (0.0007) [2023-12-27 01:50:10,833][105620] Updated weights for policy 1, policy_version 1436307 (0.0006) [2023-12-27 01:50:10,888][105620] Updated weights for policy 1, policy_version 1436317 (0.0007) [2023-12-27 01:50:10,948][105620] Updated weights for policy 1, policy_version 1436327 (0.0007) [2023-12-27 01:50:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.8, 300 sec: 19660.8). Total num frames: 734928896. Throughput: 0: 9819.9, 1: 9979.6. Samples: 734933548. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:50:11,063][104569] Avg episode reward: [(0, '8619.343'), (1, '8714.310')] [2023-12-27 01:50:11,480][105692] Updated weights for policy 0, policy_version 1434100 (0.0009) [2023-12-27 01:50:11,544][105692] Updated weights for policy 0, policy_version 1434110 (0.0009) [2023-12-27 01:50:11,607][105692] Updated weights for policy 0, policy_version 1434120 (0.0009) [2023-12-27 01:50:11,734][105620] Updated weights for policy 1, policy_version 1436337 (0.0009) [2023-12-27 01:50:11,800][105620] Updated weights for policy 1, policy_version 1436347 (0.0009) [2023-12-27 01:50:11,863][105620] Updated weights for policy 1, policy_version 1436357 (0.0008) [2023-12-27 01:50:12,263][105692] Updated weights for policy 0, policy_version 1434130 (0.0009) [2023-12-27 01:50:12,327][105692] Updated weights for policy 0, policy_version 1434140 (0.0009) [2023-12-27 01:50:12,389][105692] Updated weights for policy 0, policy_version 1434150 (0.0010) [2023-12-27 01:50:12,450][105692] Updated weights for policy 0, policy_version 1434160 (0.0009) [2023-12-27 01:50:12,627][105620] Updated weights for policy 1, policy_version 1436367 (0.0008) [2023-12-27 01:50:12,678][105620] Updated weights for policy 1, policy_version 1436377 (0.0009) [2023-12-27 01:50:12,726][105620] Updated weights for policy 1, policy_version 1436387 (0.0009) [2023-12-27 01:50:13,230][105692] Updated weights for policy 0, policy_version 1434170 (0.0009) [2023-12-27 01:50:13,291][105692] Updated weights for policy 0, policy_version 1434180 (0.0009) [2023-12-27 01:50:13,346][105692] Updated weights for policy 0, policy_version 1434190 (0.0009) [2023-12-27 01:50:13,440][105620] Updated weights for policy 1, policy_version 1436397 (0.0007) [2023-12-27 01:50:13,496][105620] Updated weights for policy 1, policy_version 1436407 (0.0005) [2023-12-27 01:50:13,550][105620] Updated weights for policy 1, policy_version 1436417 (0.0009) [2023-12-27 01:50:14,109][105620] Updated weights for policy 1, policy_version 1436427 (0.0008) [2023-12-27 01:50:14,166][105620] Updated weights for policy 1, policy_version 1436437 (0.0008) [2023-12-27 01:50:14,192][105692] Updated weights for policy 0, policy_version 1434200 (0.0008) [2023-12-27 01:50:14,219][105620] Updated weights for policy 1, policy_version 1436447 (0.0007) [2023-12-27 01:50:14,249][105692] Updated weights for policy 0, policy_version 1434210 (0.0008) [2023-12-27 01:50:14,298][105692] Updated weights for policy 0, policy_version 1434220 (0.0008) [2023-12-27 01:50:14,992][105620] Updated weights for policy 1, policy_version 1436457 (0.0007) [2023-12-27 01:50:15,061][105620] Updated weights for policy 1, policy_version 1436467 (0.0005) [2023-12-27 01:50:15,069][105692] Updated weights for policy 0, policy_version 1434230 (0.0009) [2023-12-27 01:50:15,123][105692] Updated weights for policy 0, policy_version 1434240 (0.0009) [2023-12-27 01:50:15,124][105620] Updated weights for policy 1, policy_version 1436477 (0.0005) [2023-12-27 01:50:15,189][105620] Updated weights for policy 1, policy_version 1436487 (0.0007) [2023-12-27 01:50:15,191][105692] Updated weights for policy 0, policy_version 1434250 (0.0009) [2023-12-27 01:50:15,863][105620] Updated weights for policy 1, policy_version 1436497 (0.0008) [2023-12-27 01:50:15,926][105620] Updated weights for policy 1, policy_version 1436507 (0.0009) [2023-12-27 01:50:15,955][105692] Updated weights for policy 0, policy_version 1434260 (0.0008) [2023-12-27 01:50:15,977][105620] Updated weights for policy 1, policy_version 1436517 (0.0008) [2023-12-27 01:50:16,007][105692] Updated weights for policy 0, policy_version 1434270 (0.0011) [2023-12-27 01:50:16,059][105692] Updated weights for policy 0, policy_version 1434280 (0.0010) [2023-12-27 01:50:16,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 735019008. Throughput: 0: 9763.3, 1: 9883.5. Samples: 734990196. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:50:16,063][104569] Avg episode reward: [(0, '8357.252'), (1, '8803.551')] [2023-12-27 01:50:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001436520_367796224.pth... [2023-12-27 01:50:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001435336_367493120.pth [2023-12-27 01:50:16,101][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001434288_367230976.pth... [2023-12-27 01:50:16,104][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001433168_366944256.pth [2023-12-27 01:50:16,692][105692] Updated weights for policy 0, policy_version 1434290 (0.0008) [2023-12-27 01:50:16,743][105692] Updated weights for policy 0, policy_version 1434300 (0.0010) [2023-12-27 01:50:16,769][105620] Updated weights for policy 1, policy_version 1436527 (0.0007) [2023-12-27 01:50:16,791][105692] Updated weights for policy 0, policy_version 1434310 (0.0007) [2023-12-27 01:50:16,816][105620] Updated weights for policy 1, policy_version 1436537 (0.0007) [2023-12-27 01:50:16,846][105692] Updated weights for policy 0, policy_version 1434320 (0.0006) [2023-12-27 01:50:16,870][105620] Updated weights for policy 1, policy_version 1436547 (0.0009) [2023-12-27 01:50:17,528][105692] Updated weights for policy 0, policy_version 1434330 (0.0005) [2023-12-27 01:50:17,588][105620] Updated weights for policy 1, policy_version 1436557 (0.0008) [2023-12-27 01:50:17,594][105692] Updated weights for policy 0, policy_version 1434340 (0.0005) [2023-12-27 01:50:17,644][105620] Updated weights for policy 1, policy_version 1436567 (0.0009) [2023-12-27 01:50:17,645][105692] Updated weights for policy 0, policy_version 1434350 (0.0005) [2023-12-27 01:50:17,700][105620] Updated weights for policy 1, policy_version 1436577 (0.0009) [2023-12-27 01:50:18,280][105692] Updated weights for policy 0, policy_version 1434360 (0.0008) [2023-12-27 01:50:18,346][105692] Updated weights for policy 0, policy_version 1434370 (0.0008) [2023-12-27 01:50:18,413][105692] Updated weights for policy 0, policy_version 1434380 (0.0008) [2023-12-27 01:50:18,454][105620] Updated weights for policy 1, policy_version 1436587 (0.0008) [2023-12-27 01:50:18,514][105620] Updated weights for policy 1, policy_version 1436597 (0.0005) [2023-12-27 01:50:18,586][105620] Updated weights for policy 1, policy_version 1436607 (0.0005) [2023-12-27 01:50:19,087][105620] Updated weights for policy 1, policy_version 1436617 (0.0005) [2023-12-27 01:50:19,135][105620] Updated weights for policy 1, policy_version 1436627 (0.0005) [2023-12-27 01:50:19,183][105620] Updated weights for policy 1, policy_version 1436637 (0.0005) [2023-12-27 01:50:19,238][105620] Updated weights for policy 1, policy_version 1436647 (0.0006) [2023-12-27 01:50:19,298][105692] Updated weights for policy 0, policy_version 1434390 (0.0009) [2023-12-27 01:50:19,363][105692] Updated weights for policy 0, policy_version 1434400 (0.0009) [2023-12-27 01:50:19,424][105692] Updated weights for policy 0, policy_version 1434410 (0.0009) [2023-12-27 01:50:19,881][105620] Updated weights for policy 1, policy_version 1436657 (0.0007) [2023-12-27 01:50:19,946][105620] Updated weights for policy 1, policy_version 1436667 (0.0007) [2023-12-27 01:50:20,004][105620] Updated weights for policy 1, policy_version 1436677 (0.0006) [2023-12-27 01:50:20,261][105692] Updated weights for policy 0, policy_version 1434420 (0.0010) [2023-12-27 01:50:20,318][105692] Updated weights for policy 0, policy_version 1434430 (0.0009) [2023-12-27 01:50:20,379][105692] Updated weights for policy 0, policy_version 1434440 (0.0009) [2023-12-27 01:50:20,782][105620] Updated weights for policy 1, policy_version 1436687 (0.0008) [2023-12-27 01:50:20,840][105620] Updated weights for policy 1, policy_version 1436697 (0.0009) [2023-12-27 01:50:20,892][105620] Updated weights for policy 1, policy_version 1436707 (0.0009) [2023-12-27 01:50:21,054][105692] Updated weights for policy 0, policy_version 1434450 (0.0008) [2023-12-27 01:50:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 735117312. Throughput: 0: 9701.4, 1: 9947.9. Samples: 735107148. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:50:21,062][104569] Avg episode reward: [(0, '8446.216'), (1, '8621.822')] [2023-12-27 01:50:21,120][105692] Updated weights for policy 0, policy_version 1434460 (0.0009) [2023-12-27 01:50:21,184][105692] Updated weights for policy 0, policy_version 1434470 (0.0009) [2023-12-27 01:50:21,262][105692] Updated weights for policy 0, policy_version 1434480 (0.0009) [2023-12-27 01:50:21,685][105620] Updated weights for policy 1, policy_version 1436717 (0.0009) [2023-12-27 01:50:21,753][105620] Updated weights for policy 1, policy_version 1436727 (0.0009) [2023-12-27 01:50:21,814][105620] Updated weights for policy 1, policy_version 1436737 (0.0007) [2023-12-27 01:50:22,103][105692] Updated weights for policy 0, policy_version 1434490 (0.0009) [2023-12-27 01:50:22,163][105692] Updated weights for policy 0, policy_version 1434500 (0.0009) [2023-12-27 01:50:22,226][105692] Updated weights for policy 0, policy_version 1434510 (0.0009) [2023-12-27 01:50:22,617][105620] Updated weights for policy 1, policy_version 1436747 (0.0009) [2023-12-27 01:50:22,681][105620] Updated weights for policy 1, policy_version 1436757 (0.0009) [2023-12-27 01:50:22,749][105620] Updated weights for policy 1, policy_version 1436767 (0.0009) [2023-12-27 01:50:23,055][105692] Updated weights for policy 0, policy_version 1434520 (0.0010) [2023-12-27 01:50:23,122][105692] Updated weights for policy 0, policy_version 1434530 (0.0008) [2023-12-27 01:50:23,192][105692] Updated weights for policy 0, policy_version 1434540 (0.0010) [2023-12-27 01:50:23,425][105620] Updated weights for policy 1, policy_version 1436777 (0.0009) [2023-12-27 01:50:23,480][105620] Updated weights for policy 1, policy_version 1436787 (0.0009) [2023-12-27 01:50:23,535][105620] Updated weights for policy 1, policy_version 1436797 (0.0009) [2023-12-27 01:50:23,599][105620] Updated weights for policy 1, policy_version 1436807 (0.0008) [2023-12-27 01:50:23,977][105692] Updated weights for policy 0, policy_version 1434550 (0.0009) [2023-12-27 01:50:24,035][105692] Updated weights for policy 0, policy_version 1434560 (0.0009) [2023-12-27 01:50:24,082][105692] Updated weights for policy 0, policy_version 1434570 (0.0008) [2023-12-27 01:50:24,290][105620] Updated weights for policy 1, policy_version 1436817 (0.0010) [2023-12-27 01:50:24,348][105620] Updated weights for policy 1, policy_version 1436827 (0.0010) [2023-12-27 01:50:24,404][105620] Updated weights for policy 1, policy_version 1436837 (0.0010) [2023-12-27 01:50:24,707][105692] Updated weights for policy 0, policy_version 1434580 (0.0010) [2023-12-27 01:50:24,770][105692] Updated weights for policy 0, policy_version 1434590 (0.0011) [2023-12-27 01:50:24,835][105692] Updated weights for policy 0, policy_version 1434600 (0.0006) [2023-12-27 01:50:25,116][105620] Updated weights for policy 1, policy_version 1436848 (0.0009) [2023-12-27 01:50:25,168][105620] Updated weights for policy 1, policy_version 1436858 (0.0008) [2023-12-27 01:50:25,229][105620] Updated weights for policy 1, policy_version 1436868 (0.0009) [2023-12-27 01:50:25,447][105692] Updated weights for policy 0, policy_version 1434610 (0.0005) [2023-12-27 01:50:25,513][105692] Updated weights for policy 0, policy_version 1434620 (0.0006) [2023-12-27 01:50:25,583][105692] Updated weights for policy 0, policy_version 1434630 (0.0009) [2023-12-27 01:50:25,649][105692] Updated weights for policy 0, policy_version 1434640 (0.0005) [2023-12-27 01:50:25,803][105620] Updated weights for policy 1, policy_version 1436878 (0.0006) [2023-12-27 01:50:25,852][105620] Updated weights for policy 1, policy_version 1436888 (0.0005) [2023-12-27 01:50:25,901][105620] Updated weights for policy 1, policy_version 1436898 (0.0005) [2023-12-27 01:50:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.7, 300 sec: 19633.0). Total num frames: 735215616. Throughput: 0: 9665.1, 1: 9894.7. Samples: 735222548. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:50:26,063][104569] Avg episode reward: [(0, '8527.894'), (1, '8348.771')] [2023-12-27 01:50:26,143][105692] Updated weights for policy 0, policy_version 1434650 (0.0010) [2023-12-27 01:50:26,204][105692] Updated weights for policy 0, policy_version 1434660 (0.0009) [2023-12-27 01:50:26,265][105692] Updated weights for policy 0, policy_version 1434670 (0.0010) [2023-12-27 01:50:26,548][105620] Updated weights for policy 1, policy_version 1436908 (0.0006) [2023-12-27 01:50:26,595][105620] Updated weights for policy 1, policy_version 1436918 (0.0008) [2023-12-27 01:50:26,651][105620] Updated weights for policy 1, policy_version 1436928 (0.0008) [2023-12-27 01:50:26,988][105692] Updated weights for policy 0, policy_version 1434680 (0.0011) [2023-12-27 01:50:27,046][105692] Updated weights for policy 0, policy_version 1434690 (0.0010) [2023-12-27 01:50:27,100][105692] Updated weights for policy 0, policy_version 1434700 (0.0010) [2023-12-27 01:50:27,424][105620] Updated weights for policy 1, policy_version 1436938 (0.0007) [2023-12-27 01:50:27,481][105620] Updated weights for policy 1, policy_version 1436948 (0.0010) [2023-12-27 01:50:27,532][105620] Updated weights for policy 1, policy_version 1436958 (0.0010) [2023-12-27 01:50:27,576][105620] Updated weights for policy 1, policy_version 1436968 (0.0010) [2023-12-27 01:50:27,858][105692] Updated weights for policy 0, policy_version 1434710 (0.0010) [2023-12-27 01:50:27,922][105692] Updated weights for policy 0, policy_version 1434720 (0.0010) [2023-12-27 01:50:27,983][105692] Updated weights for policy 0, policy_version 1434730 (0.0010) [2023-12-27 01:50:28,206][105620] Updated weights for policy 1, policy_version 1436978 (0.0010) [2023-12-27 01:50:28,260][105620] Updated weights for policy 1, policy_version 1436988 (0.0010) [2023-12-27 01:50:28,308][105620] Updated weights for policy 1, policy_version 1436998 (0.0010) [2023-12-27 01:50:28,588][105692] Updated weights for policy 0, policy_version 1434740 (0.0008) [2023-12-27 01:50:28,637][105692] Updated weights for policy 0, policy_version 1434750 (0.0007) [2023-12-27 01:50:28,699][105692] Updated weights for policy 0, policy_version 1434760 (0.0005) [2023-12-27 01:50:28,991][105620] Updated weights for policy 1, policy_version 1437008 (0.0010) [2023-12-27 01:50:29,046][105620] Updated weights for policy 1, policy_version 1437018 (0.0010) [2023-12-27 01:50:29,104][105620] Updated weights for policy 1, policy_version 1437028 (0.0010) [2023-12-27 01:50:29,350][105692] Updated weights for policy 0, policy_version 1434770 (0.0006) [2023-12-27 01:50:29,404][105692] Updated weights for policy 0, policy_version 1434780 (0.0008) [2023-12-27 01:50:29,458][105692] Updated weights for policy 0, policy_version 1434790 (0.0010) [2023-12-27 01:50:29,512][105692] Updated weights for policy 0, policy_version 1434800 (0.0010) [2023-12-27 01:50:29,816][105620] Updated weights for policy 1, policy_version 1437038 (0.0010) [2023-12-27 01:50:29,878][105620] Updated weights for policy 1, policy_version 1437048 (0.0011) [2023-12-27 01:50:29,942][105620] Updated weights for policy 1, policy_version 1437058 (0.0011) [2023-12-27 01:50:30,345][105692] Updated weights for policy 0, policy_version 1434810 (0.0009) [2023-12-27 01:50:30,404][105692] Updated weights for policy 0, policy_version 1434820 (0.0008) [2023-12-27 01:50:30,458][105692] Updated weights for policy 0, policy_version 1434830 (0.0009) [2023-12-27 01:50:30,608][105620] Updated weights for policy 1, policy_version 1437068 (0.0010) [2023-12-27 01:50:30,668][105620] Updated weights for policy 1, policy_version 1437078 (0.0008) [2023-12-27 01:50:30,729][105620] Updated weights for policy 1, policy_version 1437088 (0.0009) [2023-12-27 01:50:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 735313920. Throughput: 0: 9674.3, 1: 9977.6. Samples: 735284596. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) [2023-12-27 01:50:31,063][104569] Avg episode reward: [(0, '8441.164'), (1, '8714.447')] [2023-12-27 01:50:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001434832_367370240.pth... [2023-12-27 01:50:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001437096_367943680.pth... [2023-12-27 01:50:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001435944_367648768.pth [2023-12-27 01:50:31,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001433712_367083520.pth [2023-12-27 01:50:31,242][105692] Updated weights for policy 0, policy_version 1434840 (0.0008) [2023-12-27 01:50:31,305][105692] Updated weights for policy 0, policy_version 1434850 (0.0009) [2023-12-27 01:50:31,368][105692] Updated weights for policy 0, policy_version 1434860 (0.0009) [2023-12-27 01:50:31,417][105620] Updated weights for policy 1, policy_version 1437098 (0.0010) [2023-12-27 01:50:31,464][105620] Updated weights for policy 1, policy_version 1437108 (0.0009) [2023-12-27 01:50:31,514][105620] Updated weights for policy 1, policy_version 1437118 (0.0008) [2023-12-27 01:50:31,572][105620] Updated weights for policy 1, policy_version 1437128 (0.0009) [2023-12-27 01:50:32,119][105692] Updated weights for policy 0, policy_version 1434870 (0.0009) [2023-12-27 01:50:32,172][105692] Updated weights for policy 0, policy_version 1434880 (0.0008) [2023-12-27 01:50:32,216][105692] Updated weights for policy 0, policy_version 1434890 (0.0008) [2023-12-27 01:50:32,316][105620] Updated weights for policy 1, policy_version 1437138 (0.0011) [2023-12-27 01:50:32,381][105620] Updated weights for policy 1, policy_version 1437148 (0.0011) [2023-12-27 01:50:32,441][105620] Updated weights for policy 1, policy_version 1437158 (0.0010) [2023-12-27 01:50:33,017][105692] Updated weights for policy 0, policy_version 1434900 (0.0008) [2023-12-27 01:50:33,070][105692] Updated weights for policy 0, policy_version 1434910 (0.0009) [2023-12-27 01:50:33,135][105692] Updated weights for policy 0, policy_version 1434920 (0.0008) [2023-12-27 01:50:33,140][105620] Updated weights for policy 1, policy_version 1437168 (0.0007) [2023-12-27 01:50:33,185][105620] Updated weights for policy 1, policy_version 1437178 (0.0010) [2023-12-27 01:50:33,237][105620] Updated weights for policy 1, policy_version 1437188 (0.0010) [2023-12-27 01:50:33,864][105620] Updated weights for policy 1, policy_version 1437198 (0.0007) [2023-12-27 01:50:33,917][105620] Updated weights for policy 1, policy_version 1437208 (0.0005) [2023-12-27 01:50:33,972][105620] Updated weights for policy 1, policy_version 1437218 (0.0005) [2023-12-27 01:50:33,980][105692] Updated weights for policy 0, policy_version 1434930 (0.0006) [2023-12-27 01:50:34,043][105692] Updated weights for policy 0, policy_version 1434940 (0.0009) [2023-12-27 01:50:34,097][105692] Updated weights for policy 0, policy_version 1434951 (0.0010) [2023-12-27 01:50:34,615][105620] Updated weights for policy 1, policy_version 1437228 (0.0007) [2023-12-27 01:50:34,676][105620] Updated weights for policy 1, policy_version 1437238 (0.0009) [2023-12-27 01:50:34,742][105620] Updated weights for policy 1, policy_version 1437248 (0.0011) [2023-12-27 01:50:34,904][105692] Updated weights for policy 0, policy_version 1434961 (0.0009) [2023-12-27 01:50:34,965][105692] Updated weights for policy 0, policy_version 1434971 (0.0005) [2023-12-27 01:50:35,021][105692] Updated weights for policy 0, policy_version 1434981 (0.0008) [2023-12-27 01:50:35,066][105692] Updated weights for policy 0, policy_version 1434991 (0.0008) [2023-12-27 01:50:35,454][105620] Updated weights for policy 1, policy_version 1437258 (0.0011) [2023-12-27 01:50:35,500][105620] Updated weights for policy 1, policy_version 1437268 (0.0009) [2023-12-27 01:50:35,550][105620] Updated weights for policy 1, policy_version 1437278 (0.0006) [2023-12-27 01:50:35,604][105620] Updated weights for policy 1, policy_version 1437288 (0.0005) [2023-12-27 01:50:35,859][105692] Updated weights for policy 0, policy_version 1435001 (0.0009) [2023-12-27 01:50:35,912][105692] Updated weights for policy 0, policy_version 1435011 (0.0009) [2023-12-27 01:50:35,965][105692] Updated weights for policy 0, policy_version 1435021 (0.0009) [2023-12-27 01:50:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.7, 300 sec: 19660.8). Total num frames: 735412224. Throughput: 0: 9559.6, 1: 10042.5. Samples: 735399448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:50:36,063][104569] Avg episode reward: [(0, '8626.041'), (1, '8721.931')] [2023-12-27 01:50:36,206][105620] Updated weights for policy 1, policy_version 1437298 (0.0011) [2023-12-27 01:50:36,265][105620] Updated weights for policy 1, policy_version 1437308 (0.0011) [2023-12-27 01:50:36,324][105620] Updated weights for policy 1, policy_version 1437318 (0.0010) [2023-12-27 01:50:36,888][105692] Updated weights for policy 0, policy_version 1435031 (0.0010) [2023-12-27 01:50:36,933][105620] Updated weights for policy 1, policy_version 1437328 (0.0007) [2023-12-27 01:50:36,940][105692] Updated weights for policy 0, policy_version 1435041 (0.0007) [2023-12-27 01:50:36,990][105620] Updated weights for policy 1, policy_version 1437338 (0.0011) [2023-12-27 01:50:36,992][105692] Updated weights for policy 0, policy_version 1435051 (0.0007) [2023-12-27 01:50:37,037][105620] Updated weights for policy 1, policy_version 1437348 (0.0006) [2023-12-27 01:50:37,668][105620] Updated weights for policy 1, policy_version 1437358 (0.0009) [2023-12-27 01:50:37,733][105620] Updated weights for policy 1, policy_version 1437368 (0.0011) [2023-12-27 01:50:37,801][105620] Updated weights for policy 1, policy_version 1437378 (0.0011) [2023-12-27 01:50:37,812][105692] Updated weights for policy 0, policy_version 1435061 (0.0009) [2023-12-27 01:50:37,866][105692] Updated weights for policy 0, policy_version 1435071 (0.0008) [2023-12-27 01:50:37,924][105692] Updated weights for policy 0, policy_version 1435081 (0.0008) [2023-12-27 01:50:38,450][105620] Updated weights for policy 1, policy_version 1437388 (0.0011) [2023-12-27 01:50:38,505][105620] Updated weights for policy 1, policy_version 1437398 (0.0011) [2023-12-27 01:50:38,540][105692] Updated weights for policy 0, policy_version 1435091 (0.0006) [2023-12-27 01:50:38,562][105620] Updated weights for policy 1, policy_version 1437408 (0.0011) [2023-12-27 01:50:38,593][105692] Updated weights for policy 0, policy_version 1435101 (0.0007) [2023-12-27 01:50:38,649][105692] Updated weights for policy 0, policy_version 1435111 (0.0008) [2023-12-27 01:50:39,300][105620] Updated weights for policy 1, policy_version 1437418 (0.0011) [2023-12-27 01:50:39,363][105620] Updated weights for policy 1, policy_version 1437428 (0.0009) [2023-12-27 01:50:39,429][105620] Updated weights for policy 1, policy_version 1437438 (0.0010) [2023-12-27 01:50:39,454][105692] Updated weights for policy 0, policy_version 1435121 (0.0008) [2023-12-27 01:50:39,484][105620] Updated weights for policy 1, policy_version 1437448 (0.0007) [2023-12-27 01:50:39,510][105692] Updated weights for policy 0, policy_version 1435131 (0.0009) [2023-12-27 01:50:39,557][105692] Updated weights for policy 0, policy_version 1435141 (0.0009) [2023-12-27 01:50:39,619][105692] Updated weights for policy 0, policy_version 1435151 (0.0009) [2023-12-27 01:50:40,225][105620] Updated weights for policy 1, policy_version 1437458 (0.0007) [2023-12-27 01:50:40,279][105620] Updated weights for policy 1, policy_version 1437468 (0.0005) [2023-12-27 01:50:40,333][105620] Updated weights for policy 1, policy_version 1437478 (0.0008) [2023-12-27 01:50:40,439][105692] Updated weights for policy 0, policy_version 1435161 (0.0009) [2023-12-27 01:50:40,489][105692] Updated weights for policy 0, policy_version 1435172 (0.0008) [2023-12-27 01:50:40,551][105692] Updated weights for policy 0, policy_version 1435182 (0.0006) [2023-12-27 01:50:40,919][105620] Updated weights for policy 1, policy_version 1437488 (0.0006) [2023-12-27 01:50:40,973][105620] Updated weights for policy 1, policy_version 1437498 (0.0006) [2023-12-27 01:50:41,024][105620] Updated weights for policy 1, policy_version 1437508 (0.0009) [2023-12-27 01:50:41,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 735510528. Throughput: 0: 9468.3, 1: 10017.6. Samples: 735516760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:50:41,062][104569] Avg episode reward: [(0, '8625.230'), (1, '8721.548')] [2023-12-27 01:50:41,254][105692] Updated weights for policy 0, policy_version 1435192 (0.0008) [2023-12-27 01:50:41,328][105692] Updated weights for policy 0, policy_version 1435202 (0.0009) [2023-12-27 01:50:41,403][105692] Updated weights for policy 0, policy_version 1435212 (0.0008) [2023-12-27 01:50:41,764][105620] Updated weights for policy 1, policy_version 1437518 (0.0009) [2023-12-27 01:50:41,813][105620] Updated weights for policy 1, policy_version 1437528 (0.0008) [2023-12-27 01:50:41,861][105620] Updated weights for policy 1, policy_version 1437538 (0.0005) [2023-12-27 01:50:42,214][105692] Updated weights for policy 0, policy_version 1435222 (0.0009) [2023-12-27 01:50:42,270][105692] Updated weights for policy 0, policy_version 1435232 (0.0009) [2023-12-27 01:50:42,325][105692] Updated weights for policy 0, policy_version 1435242 (0.0009) [2023-12-27 01:50:42,511][105620] Updated weights for policy 1, policy_version 1437548 (0.0009) [2023-12-27 01:50:42,562][105620] Updated weights for policy 1, policy_version 1437558 (0.0009) [2023-12-27 01:50:42,624][105620] Updated weights for policy 1, policy_version 1437568 (0.0009) [2023-12-27 01:50:43,134][105692] Updated weights for policy 0, policy_version 1435252 (0.0008) [2023-12-27 01:50:43,184][105692] Updated weights for policy 0, policy_version 1435262 (0.0008) [2023-12-27 01:50:43,244][105692] Updated weights for policy 0, policy_version 1435272 (0.0008) [2023-12-27 01:50:43,323][105620] Updated weights for policy 1, policy_version 1437578 (0.0008) [2023-12-27 01:50:43,376][105620] Updated weights for policy 1, policy_version 1437588 (0.0008) [2023-12-27 01:50:43,424][105620] Updated weights for policy 1, policy_version 1437598 (0.0008) [2023-12-27 01:50:43,479][105620] Updated weights for policy 1, policy_version 1437608 (0.0008) [2023-12-27 01:50:43,914][105692] Updated weights for policy 0, policy_version 1435282 (0.0009) [2023-12-27 01:50:43,978][105692] Updated weights for policy 0, policy_version 1435292 (0.0009) [2023-12-27 01:50:44,043][105692] Updated weights for policy 0, policy_version 1435302 (0.0009) [2023-12-27 01:50:44,102][105692] Updated weights for policy 0, policy_version 1435312 (0.0009) [2023-12-27 01:50:44,269][105620] Updated weights for policy 1, policy_version 1437618 (0.0009) [2023-12-27 01:50:44,320][105620] Updated weights for policy 1, policy_version 1437628 (0.0009) [2023-12-27 01:50:44,379][105620] Updated weights for policy 1, policy_version 1437638 (0.0009) [2023-12-27 01:50:44,863][105692] Updated weights for policy 0, policy_version 1435322 (0.0008) [2023-12-27 01:50:44,930][105692] Updated weights for policy 0, policy_version 1435332 (0.0008) [2023-12-27 01:50:44,990][105692] Updated weights for policy 0, policy_version 1435342 (0.0008) [2023-12-27 01:50:45,116][105620] Updated weights for policy 1, policy_version 1437648 (0.0009) [2023-12-27 01:50:45,164][105620] Updated weights for policy 1, policy_version 1437658 (0.0008) [2023-12-27 01:50:45,224][105620] Updated weights for policy 1, policy_version 1437668 (0.0009) [2023-12-27 01:50:45,747][105692] Updated weights for policy 0, policy_version 1435352 (0.0008) [2023-12-27 01:50:45,795][105692] Updated weights for policy 0, policy_version 1435362 (0.0009) [2023-12-27 01:50:45,842][105692] Updated weights for policy 0, policy_version 1435373 (0.0009) [2023-12-27 01:50:45,991][105620] Updated weights for policy 1, policy_version 1437678 (0.0009) [2023-12-27 01:50:46,040][105620] Updated weights for policy 1, policy_version 1437688 (0.0010) [2023-12-27 01:50:46,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 735600640. Throughput: 0: 9381.3, 1: 9983.8. Samples: 735573292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:50:46,062][104569] Avg episode reward: [(0, '8435.873'), (1, '8989.462')] [2023-12-27 01:50:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001435376_367509504.pth... [2023-12-27 01:50:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001434288_367230976.pth [2023-12-27 01:50:46,094][105620] Updated weights for policy 1, policy_version 1437698 (0.0010) [2023-12-27 01:50:46,127][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001437704_368099328.pth... [2023-12-27 01:50:46,130][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001436520_367796224.pth [2023-12-27 01:50:46,640][105692] Updated weights for policy 0, policy_version 1435384 (0.0008) [2023-12-27 01:50:46,699][105692] Updated weights for policy 0, policy_version 1435394 (0.0008) [2023-12-27 01:50:46,757][105692] Updated weights for policy 0, policy_version 1435404 (0.0008) [2023-12-27 01:50:46,859][105620] Updated weights for policy 1, policy_version 1437708 (0.0010) [2023-12-27 01:50:46,907][105620] Updated weights for policy 1, policy_version 1437718 (0.0010) [2023-12-27 01:50:46,962][105620] Updated weights for policy 1, policy_version 1437728 (0.0010) [2023-12-27 01:50:47,494][105692] Updated weights for policy 0, policy_version 1435414 (0.0009) [2023-12-27 01:50:47,566][105692] Updated weights for policy 0, policy_version 1435424 (0.0010) [2023-12-27 01:50:47,628][105692] Updated weights for policy 0, policy_version 1435434 (0.0009) [2023-12-27 01:50:47,637][105620] Updated weights for policy 1, policy_version 1437738 (0.0009) [2023-12-27 01:50:47,693][105620] Updated weights for policy 1, policy_version 1437748 (0.0005) [2023-12-27 01:50:47,755][105620] Updated weights for policy 1, policy_version 1437758 (0.0005) [2023-12-27 01:50:47,801][105620] Updated weights for policy 1, policy_version 1437768 (0.0005) [2023-12-27 01:50:48,273][105692] Updated weights for policy 0, policy_version 1435444 (0.0008) [2023-12-27 01:50:48,331][105692] Updated weights for policy 0, policy_version 1435454 (0.0009) [2023-12-27 01:50:48,378][105620] Updated weights for policy 1, policy_version 1437778 (0.0008) [2023-12-27 01:50:48,391][105692] Updated weights for policy 0, policy_version 1435464 (0.0008) [2023-12-27 01:50:48,432][105620] Updated weights for policy 1, policy_version 1437788 (0.0006) [2023-12-27 01:50:48,485][105620] Updated weights for policy 1, policy_version 1437798 (0.0005) [2023-12-27 01:50:49,122][105620] Updated weights for policy 1, policy_version 1437808 (0.0010) [2023-12-27 01:50:49,158][105692] Updated weights for policy 0, policy_version 1435474 (0.0009) [2023-12-27 01:50:49,171][105620] Updated weights for policy 1, policy_version 1437818 (0.0010) [2023-12-27 01:50:49,218][105692] Updated weights for policy 0, policy_version 1435484 (0.0011) [2023-12-27 01:50:49,225][105620] Updated weights for policy 1, policy_version 1437828 (0.0010) [2023-12-27 01:50:49,283][105692] Updated weights for policy 0, policy_version 1435494 (0.0009) [2023-12-27 01:50:49,345][105692] Updated weights for policy 0, policy_version 1435504 (0.0008) [2023-12-27 01:50:49,986][105620] Updated weights for policy 1, policy_version 1437838 (0.0008) [2023-12-27 01:50:50,041][105620] Updated weights for policy 1, policy_version 1437848 (0.0010) [2023-12-27 01:50:50,096][105620] Updated weights for policy 1, policy_version 1437858 (0.0010) [2023-12-27 01:50:50,112][105692] Updated weights for policy 0, policy_version 1435514 (0.0011) [2023-12-27 01:50:50,164][105692] Updated weights for policy 0, policy_version 1435524 (0.0010) [2023-12-27 01:50:50,216][105692] Updated weights for policy 0, policy_version 1435534 (0.0009) [2023-12-27 01:50:50,771][105620] Updated weights for policy 1, policy_version 1437868 (0.0011) [2023-12-27 01:50:50,834][105620] Updated weights for policy 1, policy_version 1437878 (0.0007) [2023-12-27 01:50:50,893][105620] Updated weights for policy 1, policy_version 1437888 (0.0006) [2023-12-27 01:50:50,972][105692] Updated weights for policy 0, policy_version 1435544 (0.0009) [2023-12-27 01:50:51,042][105692] Updated weights for policy 0, policy_version 1435554 (0.0008) [2023-12-27 01:50:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 735698944. Throughput: 0: 9307.0, 1: 10035.8. Samples: 735689624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:50:51,063][104569] Avg episode reward: [(0, '8432.893'), (1, '8807.244')] [2023-12-27 01:50:51,101][105692] Updated weights for policy 0, policy_version 1435564 (0.0009) [2023-12-27 01:50:51,555][105620] Updated weights for policy 1, policy_version 1437898 (0.0009) [2023-12-27 01:50:51,619][105620] Updated weights for policy 1, policy_version 1437908 (0.0006) [2023-12-27 01:50:51,685][105620] Updated weights for policy 1, policy_version 1437918 (0.0008) [2023-12-27 01:50:51,752][105620] Updated weights for policy 1, policy_version 1437928 (0.0009) [2023-12-27 01:50:51,833][105692] Updated weights for policy 0, policy_version 1435574 (0.0009) [2023-12-27 01:50:51,892][105692] Updated weights for policy 0, policy_version 1435584 (0.0010) [2023-12-27 01:50:51,954][105692] Updated weights for policy 0, policy_version 1435594 (0.0009) [2023-12-27 01:50:52,442][105620] Updated weights for policy 1, policy_version 1437938 (0.0009) [2023-12-27 01:50:52,510][105620] Updated weights for policy 1, policy_version 1437948 (0.0009) [2023-12-27 01:50:52,571][105620] Updated weights for policy 1, policy_version 1437958 (0.0009) [2023-12-27 01:50:52,753][105692] Updated weights for policy 0, policy_version 1435604 (0.0010) [2023-12-27 01:50:52,804][105692] Updated weights for policy 0, policy_version 1435614 (0.0009) [2023-12-27 01:50:52,863][105692] Updated weights for policy 0, policy_version 1435624 (0.0006) [2023-12-27 01:50:53,330][105620] Updated weights for policy 1, policy_version 1437968 (0.0009) [2023-12-27 01:50:53,397][105620] Updated weights for policy 1, policy_version 1437978 (0.0009) [2023-12-27 01:50:53,462][105620] Updated weights for policy 1, policy_version 1437988 (0.0009) [2023-12-27 01:50:53,513][105692] Updated weights for policy 0, policy_version 1435634 (0.0006) [2023-12-27 01:50:53,562][105692] Updated weights for policy 0, policy_version 1435644 (0.0008) [2023-12-27 01:50:53,606][105692] Updated weights for policy 0, policy_version 1435654 (0.0006) [2023-12-27 01:50:53,652][105692] Updated weights for policy 0, policy_version 1435664 (0.0005) [2023-12-27 01:50:54,273][105620] Updated weights for policy 1, policy_version 1437998 (0.0009) [2023-12-27 01:50:54,330][105620] Updated weights for policy 1, policy_version 1438008 (0.0009) [2023-12-27 01:50:54,356][105692] Updated weights for policy 0, policy_version 1435674 (0.0008) [2023-12-27 01:50:54,391][105620] Updated weights for policy 1, policy_version 1438018 (0.0008) [2023-12-27 01:50:54,418][105692] Updated weights for policy 0, policy_version 1435684 (0.0006) [2023-12-27 01:50:54,475][105692] Updated weights for policy 0, policy_version 1435694 (0.0008) [2023-12-27 01:50:55,122][105620] Updated weights for policy 1, policy_version 1438028 (0.0006) [2023-12-27 01:50:55,185][105692] Updated weights for policy 0, policy_version 1435704 (0.0007) [2023-12-27 01:50:55,190][105620] Updated weights for policy 1, policy_version 1438038 (0.0008) [2023-12-27 01:50:55,248][105620] Updated weights for policy 1, policy_version 1438048 (0.0007) [2023-12-27 01:50:55,249][105692] Updated weights for policy 0, policy_version 1435714 (0.0011) [2023-12-27 01:50:55,302][105692] Updated weights for policy 0, policy_version 1435724 (0.0010) [2023-12-27 01:50:55,870][105620] Updated weights for policy 1, policy_version 1438058 (0.0006) [2023-12-27 01:50:55,927][105620] Updated weights for policy 1, policy_version 1438069 (0.0009) [2023-12-27 01:50:55,967][105692] Updated weights for policy 0, policy_version 1435734 (0.0008) [2023-12-27 01:50:55,988][105620] Updated weights for policy 1, policy_version 1438079 (0.0005) [2023-12-27 01:50:56,026][105692] Updated weights for policy 0, policy_version 1435744 (0.0008) [2023-12-27 01:50:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 735797248. Throughput: 0: 9388.4, 1: 9996.6. Samples: 735805872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:50:56,063][104569] Avg episode reward: [(0, '8437.060'), (1, '8807.193')] [2023-12-27 01:50:56,073][105692] Updated weights for policy 0, policy_version 1435754 (0.0006) [2023-12-27 01:50:56,684][105620] Updated weights for policy 1, policy_version 1438089 (0.0006) [2023-12-27 01:50:56,730][105620] Updated weights for policy 1, policy_version 1438099 (0.0009) [2023-12-27 01:50:56,762][105692] Updated weights for policy 0, policy_version 1435764 (0.0007) [2023-12-27 01:50:56,780][105620] Updated weights for policy 1, policy_version 1438109 (0.0007) [2023-12-27 01:50:56,814][105692] Updated weights for policy 0, policy_version 1435774 (0.0007) [2023-12-27 01:50:56,837][105620] Updated weights for policy 1, policy_version 1438119 (0.0008) [2023-12-27 01:50:56,871][105692] Updated weights for policy 0, policy_version 1435784 (0.0010) [2023-12-27 01:50:57,561][105692] Updated weights for policy 0, policy_version 1435794 (0.0008) [2023-12-27 01:50:57,600][105620] Updated weights for policy 1, policy_version 1438129 (0.0006) [2023-12-27 01:50:57,610][105692] Updated weights for policy 0, policy_version 1435804 (0.0006) [2023-12-27 01:50:57,660][105620] Updated weights for policy 1, policy_version 1438139 (0.0009) [2023-12-27 01:50:57,665][105692] Updated weights for policy 0, policy_version 1435814 (0.0006) [2023-12-27 01:50:57,712][105692] Updated weights for policy 0, policy_version 1435824 (0.0005) [2023-12-27 01:50:57,716][105620] Updated weights for policy 1, policy_version 1438149 (0.0010) [2023-12-27 01:50:58,270][105692] Updated weights for policy 0, policy_version 1435834 (0.0008) [2023-12-27 01:50:58,339][105692] Updated weights for policy 0, policy_version 1435844 (0.0008) [2023-12-27 01:50:58,383][105620] Updated weights for policy 1, policy_version 1438159 (0.0010) [2023-12-27 01:50:58,405][105692] Updated weights for policy 0, policy_version 1435854 (0.0008) [2023-12-27 01:50:58,448][105620] Updated weights for policy 1, policy_version 1438169 (0.0011) [2023-12-27 01:50:58,500][105620] Updated weights for policy 1, policy_version 1438179 (0.0010) [2023-12-27 01:50:59,226][105692] Updated weights for policy 0, policy_version 1435864 (0.0009) [2023-12-27 01:50:59,294][105692] Updated weights for policy 0, policy_version 1435874 (0.0008) [2023-12-27 01:50:59,357][105692] Updated weights for policy 0, policy_version 1435884 (0.0008) [2023-12-27 01:50:59,365][105620] Updated weights for policy 1, policy_version 1438189 (0.0010) [2023-12-27 01:50:59,430][105620] Updated weights for policy 1, policy_version 1438199 (0.0008) [2023-12-27 01:50:59,491][105620] Updated weights for policy 1, policy_version 1438209 (0.0008) [2023-12-27 01:51:00,077][105692] Updated weights for policy 0, policy_version 1435894 (0.0008) [2023-12-27 01:51:00,133][105692] Updated weights for policy 0, policy_version 1435904 (0.0005) [2023-12-27 01:51:00,191][105692] Updated weights for policy 0, policy_version 1435914 (0.0006) [2023-12-27 01:51:00,260][105620] Updated weights for policy 1, policy_version 1438219 (0.0008) [2023-12-27 01:51:00,313][105620] Updated weights for policy 1, policy_version 1438229 (0.0006) [2023-12-27 01:51:00,360][105620] Updated weights for policy 1, policy_version 1438239 (0.0008) [2023-12-27 01:51:00,978][105692] Updated weights for policy 0, policy_version 1435924 (0.0009) [2023-12-27 01:51:00,997][105620] Updated weights for policy 1, policy_version 1438249 (0.0007) [2023-12-27 01:51:01,044][105692] Updated weights for policy 0, policy_version 1435934 (0.0008) [2023-12-27 01:51:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19660.8). Total num frames: 735887360. Throughput: 0: 9472.6, 1: 9989.0. Samples: 735865964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:51:01,063][104569] Avg episode reward: [(0, '8161.541'), (1, '8896.746')] [2023-12-27 01:51:01,065][105620] Updated weights for policy 1, policy_version 1438259 (0.0006) [2023-12-27 01:51:01,113][105692] Updated weights for policy 0, policy_version 1435944 (0.0006) [2023-12-27 01:51:01,131][105620] Updated weights for policy 1, policy_version 1438269 (0.0007) [2023-12-27 01:51:01,163][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001435952_367656960.pth... [2023-12-27 01:51:01,167][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001434832_367370240.pth [2023-12-27 01:51:01,193][105620] Updated weights for policy 1, policy_version 1438279 (0.0008) [2023-12-27 01:51:01,196][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001438280_368246784.pth... [2023-12-27 01:51:01,199][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001437096_367943680.pth [2023-12-27 01:51:01,849][105692] Updated weights for policy 0, policy_version 1435954 (0.0008) [2023-12-27 01:51:01,906][105692] Updated weights for policy 0, policy_version 1435964 (0.0006) [2023-12-27 01:51:01,945][105620] Updated weights for policy 1, policy_version 1438289 (0.0009) [2023-12-27 01:51:01,961][105692] Updated weights for policy 0, policy_version 1435974 (0.0007) [2023-12-27 01:51:02,009][105620] Updated weights for policy 1, policy_version 1438299 (0.0008) [2023-12-27 01:51:02,015][105692] Updated weights for policy 0, policy_version 1435984 (0.0007) [2023-12-27 01:51:02,070][105620] Updated weights for policy 1, policy_version 1438309 (0.0008) [2023-12-27 01:51:02,673][105692] Updated weights for policy 0, policy_version 1435994 (0.0008) [2023-12-27 01:51:02,682][105620] Updated weights for policy 1, policy_version 1438319 (0.0005) [2023-12-27 01:51:02,735][105692] Updated weights for policy 0, policy_version 1436004 (0.0009) [2023-12-27 01:51:02,740][105620] Updated weights for policy 1, policy_version 1438329 (0.0005) [2023-12-27 01:51:02,784][105692] Updated weights for policy 0, policy_version 1436014 (0.0009) [2023-12-27 01:51:02,804][105620] Updated weights for policy 1, policy_version 1438339 (0.0005) [2023-12-27 01:51:03,419][105620] Updated weights for policy 1, policy_version 1438349 (0.0006) [2023-12-27 01:51:03,466][105620] Updated weights for policy 1, policy_version 1438359 (0.0005) [2023-12-27 01:51:03,518][105620] Updated weights for policy 1, policy_version 1438369 (0.0009) [2023-12-27 01:51:03,566][105692] Updated weights for policy 0, policy_version 1436024 (0.0009) [2023-12-27 01:51:03,612][105692] Updated weights for policy 0, policy_version 1436034 (0.0009) [2023-12-27 01:51:03,662][105692] Updated weights for policy 0, policy_version 1436044 (0.0008) [2023-12-27 01:51:04,227][105620] Updated weights for policy 1, policy_version 1438379 (0.0010) [2023-12-27 01:51:04,289][105620] Updated weights for policy 1, policy_version 1438389 (0.0009) [2023-12-27 01:51:04,352][105620] Updated weights for policy 1, policy_version 1438399 (0.0007) [2023-12-27 01:51:04,433][105692] Updated weights for policy 0, policy_version 1436054 (0.0008) [2023-12-27 01:51:04,484][105692] Updated weights for policy 0, policy_version 1436064 (0.0009) [2023-12-27 01:51:04,541][105692] Updated weights for policy 0, policy_version 1436074 (0.0008) [2023-12-27 01:51:05,065][105620] Updated weights for policy 1, policy_version 1438409 (0.0006) [2023-12-27 01:51:05,119][105620] Updated weights for policy 1, policy_version 1438419 (0.0010) [2023-12-27 01:51:05,184][105620] Updated weights for policy 1, policy_version 1438429 (0.0010) [2023-12-27 01:51:05,198][105692] Updated weights for policy 0, policy_version 1436084 (0.0008) [2023-12-27 01:51:05,249][105620] Updated weights for policy 1, policy_version 1438439 (0.0010) [2023-12-27 01:51:05,257][105692] Updated weights for policy 0, policy_version 1436094 (0.0005) [2023-12-27 01:51:05,320][105692] Updated weights for policy 0, policy_version 1436104 (0.0005) [2023-12-27 01:51:05,819][105692] Updated weights for policy 0, policy_version 1436114 (0.0005) [2023-12-27 01:51:05,884][105692] Updated weights for policy 0, policy_version 1436124 (0.0005) [2023-12-27 01:51:05,934][105692] Updated weights for policy 0, policy_version 1436134 (0.0005) [2023-12-27 01:51:05,959][105620] Updated weights for policy 1, policy_version 1438449 (0.0010) [2023-12-27 01:51:05,990][105692] Updated weights for policy 0, policy_version 1436144 (0.0005) [2023-12-27 01:51:06,003][105620] Updated weights for policy 1, policy_version 1438459 (0.0010) [2023-12-27 01:51:06,048][105620] Updated weights for policy 1, policy_version 1438469 (0.0010) [2023-12-27 01:51:06,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19524.3, 300 sec: 19716.3). Total num frames: 736002048. Throughput: 0: 9457.8, 1: 9971.8. Samples: 735981480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:51:06,062][104569] Avg episode reward: [(0, '8160.361'), (1, '8807.728')] [2023-12-27 01:51:06,677][105692] Updated weights for policy 0, policy_version 1436154 (0.0011) [2023-12-27 01:51:06,741][105692] Updated weights for policy 0, policy_version 1436164 (0.0011) [2023-12-27 01:51:06,799][105692] Updated weights for policy 0, policy_version 1436174 (0.0009) [2023-12-27 01:51:06,842][105620] Updated weights for policy 1, policy_version 1438479 (0.0010) [2023-12-27 01:51:06,911][105620] Updated weights for policy 1, policy_version 1438489 (0.0010) [2023-12-27 01:51:06,982][105620] Updated weights for policy 1, policy_version 1438499 (0.0010) [2023-12-27 01:51:07,437][105692] Updated weights for policy 0, policy_version 1436184 (0.0005) [2023-12-27 01:51:07,495][105692] Updated weights for policy 0, policy_version 1436194 (0.0006) [2023-12-27 01:51:07,553][105692] Updated weights for policy 0, policy_version 1436204 (0.0006) [2023-12-27 01:51:07,680][105620] Updated weights for policy 1, policy_version 1438509 (0.0009) [2023-12-27 01:51:07,738][105620] Updated weights for policy 1, policy_version 1438519 (0.0010) [2023-12-27 01:51:07,799][105620] Updated weights for policy 1, policy_version 1438529 (0.0010) [2023-12-27 01:51:08,167][105692] Updated weights for policy 0, policy_version 1436214 (0.0005) [2023-12-27 01:51:08,215][105692] Updated weights for policy 0, policy_version 1436224 (0.0005) [2023-12-27 01:51:08,269][105692] Updated weights for policy 0, policy_version 1436234 (0.0005) [2023-12-27 01:51:08,513][105620] Updated weights for policy 1, policy_version 1438539 (0.0010) [2023-12-27 01:51:08,574][105620] Updated weights for policy 1, policy_version 1438549 (0.0010) [2023-12-27 01:51:08,642][105620] Updated weights for policy 1, policy_version 1438559 (0.0007) [2023-12-27 01:51:08,941][105692] Updated weights for policy 0, policy_version 1436244 (0.0007) [2023-12-27 01:51:08,993][105692] Updated weights for policy 0, policy_version 1436254 (0.0010) [2023-12-27 01:51:09,045][105692] Updated weights for policy 0, policy_version 1436264 (0.0010) [2023-12-27 01:51:09,326][105620] Updated weights for policy 1, policy_version 1438569 (0.0008) [2023-12-27 01:51:09,398][105620] Updated weights for policy 1, policy_version 1438579 (0.0008) [2023-12-27 01:51:09,467][105620] Updated weights for policy 1, policy_version 1438589 (0.0008) [2023-12-27 01:51:09,525][105620] Updated weights for policy 1, policy_version 1438599 (0.0010) [2023-12-27 01:51:09,795][105692] Updated weights for policy 0, policy_version 1436274 (0.0010) [2023-12-27 01:51:09,873][105692] Updated weights for policy 0, policy_version 1436284 (0.0009) [2023-12-27 01:51:09,944][105692] Updated weights for policy 0, policy_version 1436294 (0.0009) [2023-12-27 01:51:09,995][105692] Updated weights for policy 0, policy_version 1436304 (0.0009) [2023-12-27 01:51:10,301][105620] Updated weights for policy 1, policy_version 1438609 (0.0009) [2023-12-27 01:51:10,363][105620] Updated weights for policy 1, policy_version 1438619 (0.0008) [2023-12-27 01:51:10,425][105620] Updated weights for policy 1, policy_version 1438629 (0.0009) [2023-12-27 01:51:10,682][105692] Updated weights for policy 0, policy_version 1436314 (0.0005) [2023-12-27 01:51:10,762][105692] Updated weights for policy 0, policy_version 1436324 (0.0007) [2023-12-27 01:51:10,822][105692] Updated weights for policy 0, policy_version 1436334 (0.0009) [2023-12-27 01:51:11,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.8, 300 sec: 19688.6). Total num frames: 736092160. Throughput: 0: 9609.7, 1: 9924.6. Samples: 736101584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:51:11,062][104569] Avg episode reward: [(0, '8708.113'), (1, '8626.287')] [2023-12-27 01:51:11,256][105620] Updated weights for policy 1, policy_version 1438639 (0.0009) [2023-12-27 01:51:11,315][105620] Updated weights for policy 1, policy_version 1438649 (0.0010) [2023-12-27 01:51:11,370][105620] Updated weights for policy 1, policy_version 1438659 (0.0008) [2023-12-27 01:51:11,567][105692] Updated weights for policy 0, policy_version 1436344 (0.0008) [2023-12-27 01:51:11,633][105692] Updated weights for policy 0, policy_version 1436354 (0.0009) [2023-12-27 01:51:11,695][105692] Updated weights for policy 0, policy_version 1436364 (0.0010) [2023-12-27 01:51:12,181][105620] Updated weights for policy 1, policy_version 1438669 (0.0009) [2023-12-27 01:51:12,241][105620] Updated weights for policy 1, policy_version 1438679 (0.0011) [2023-12-27 01:51:12,340][105620] Updated weights for policy 1, policy_version 1438689 (0.0008) [2023-12-27 01:51:12,510][105692] Updated weights for policy 0, policy_version 1436374 (0.0010) [2023-12-27 01:51:12,575][105692] Updated weights for policy 0, policy_version 1436384 (0.0010) [2023-12-27 01:51:12,641][105692] Updated weights for policy 0, policy_version 1436394 (0.0009) [2023-12-27 01:51:12,933][105620] Updated weights for policy 1, policy_version 1438699 (0.0008) [2023-12-27 01:51:12,996][105620] Updated weights for policy 1, policy_version 1438709 (0.0008) [2023-12-27 01:51:13,063][105620] Updated weights for policy 1, policy_version 1438719 (0.0009) [2023-12-27 01:51:13,408][105692] Updated weights for policy 0, policy_version 1436404 (0.0007) [2023-12-27 01:51:13,458][105692] Updated weights for policy 0, policy_version 1436414 (0.0005) [2023-12-27 01:51:13,504][105692] Updated weights for policy 0, policy_version 1436424 (0.0005) [2023-12-27 01:51:13,673][105620] Updated weights for policy 1, policy_version 1438729 (0.0008) [2023-12-27 01:51:13,720][105620] Updated weights for policy 1, policy_version 1438739 (0.0007) [2023-12-27 01:51:13,766][105620] Updated weights for policy 1, policy_version 1438749 (0.0005) [2023-12-27 01:51:13,815][105620] Updated weights for policy 1, policy_version 1438759 (0.0005) [2023-12-27 01:51:14,108][105692] Updated weights for policy 0, policy_version 1436434 (0.0005) [2023-12-27 01:51:14,169][105692] Updated weights for policy 0, policy_version 1436444 (0.0005) [2023-12-27 01:51:14,216][105692] Updated weights for policy 0, policy_version 1436454 (0.0005) [2023-12-27 01:51:14,269][105692] Updated weights for policy 0, policy_version 1436464 (0.0008) [2023-12-27 01:51:14,427][105620] Updated weights for policy 1, policy_version 1438769 (0.0005) [2023-12-27 01:51:14,480][105620] Updated weights for policy 1, policy_version 1438779 (0.0005) [2023-12-27 01:51:14,529][105620] Updated weights for policy 1, policy_version 1438789 (0.0008) [2023-12-27 01:51:15,015][105692] Updated weights for policy 0, policy_version 1436474 (0.0006) [2023-12-27 01:51:15,074][105692] Updated weights for policy 0, policy_version 1436484 (0.0005) [2023-12-27 01:51:15,135][105692] Updated weights for policy 0, policy_version 1436494 (0.0006) [2023-12-27 01:51:15,299][105620] Updated weights for policy 1, policy_version 1438799 (0.0008) [2023-12-27 01:51:15,360][105620] Updated weights for policy 1, policy_version 1438809 (0.0008) [2023-12-27 01:51:15,421][105620] Updated weights for policy 1, policy_version 1438819 (0.0009) [2023-12-27 01:51:15,770][105692] Updated weights for policy 0, policy_version 1436504 (0.0010) [2023-12-27 01:51:15,827][105692] Updated weights for policy 0, policy_version 1436514 (0.0006) [2023-12-27 01:51:15,886][105692] Updated weights for policy 0, policy_version 1436524 (0.0006) [2023-12-27 01:51:16,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19524.3, 300 sec: 19716.3). Total num frames: 736190464. Throughput: 0: 9520.9, 1: 9905.9. Samples: 736158804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:51:16,063][104569] Avg episode reward: [(0, '8893.651'), (1, '8899.781')] [2023-12-27 01:51:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001436528_367804416.pth... [2023-12-27 01:51:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001438824_368386048.pth... [2023-12-27 01:51:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001437704_368099328.pth [2023-12-27 01:51:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001435376_367509504.pth [2023-12-27 01:51:16,225][105620] Updated weights for policy 1, policy_version 1438829 (0.0009) [2023-12-27 01:51:16,279][105620] Updated weights for policy 1, policy_version 1438839 (0.0009) [2023-12-27 01:51:16,329][105620] Updated weights for policy 1, policy_version 1438849 (0.0009) [2023-12-27 01:51:16,437][105692] Updated weights for policy 0, policy_version 1436534 (0.0006) [2023-12-27 01:51:16,493][105692] Updated weights for policy 0, policy_version 1436544 (0.0005) [2023-12-27 01:51:16,539][105692] Updated weights for policy 0, policy_version 1436554 (0.0005) [2023-12-27 01:51:17,056][105620] Updated weights for policy 1, policy_version 1438859 (0.0009) [2023-12-27 01:51:17,114][105620] Updated weights for policy 1, policy_version 1438869 (0.0005) [2023-12-27 01:51:17,188][105620] Updated weights for policy 1, policy_version 1438879 (0.0006) [2023-12-27 01:51:17,216][105692] Updated weights for policy 0, policy_version 1436564 (0.0006) [2023-12-27 01:51:17,271][105692] Updated weights for policy 0, policy_version 1436574 (0.0008) [2023-12-27 01:51:17,329][105692] Updated weights for policy 0, policy_version 1436584 (0.0009) [2023-12-27 01:51:17,845][105620] Updated weights for policy 1, policy_version 1438889 (0.0006) [2023-12-27 01:51:17,895][105620] Updated weights for policy 1, policy_version 1438899 (0.0008) [2023-12-27 01:51:17,949][105620] Updated weights for policy 1, policy_version 1438909 (0.0007) [2023-12-27 01:51:18,005][105620] Updated weights for policy 1, policy_version 1438919 (0.0006) [2023-12-27 01:51:18,054][105692] Updated weights for policy 0, policy_version 1436594 (0.0009) [2023-12-27 01:51:18,108][105692] Updated weights for policy 0, policy_version 1436604 (0.0006) [2023-12-27 01:51:18,162][105692] Updated weights for policy 0, policy_version 1436614 (0.0010) [2023-12-27 01:51:18,215][105692] Updated weights for policy 0, policy_version 1436624 (0.0005) [2023-12-27 01:51:18,688][105620] Updated weights for policy 1, policy_version 1438929 (0.0010) [2023-12-27 01:51:18,747][105620] Updated weights for policy 1, policy_version 1438939 (0.0008) [2023-12-27 01:51:18,802][105620] Updated weights for policy 1, policy_version 1438949 (0.0008) [2023-12-27 01:51:18,854][105692] Updated weights for policy 0, policy_version 1436634 (0.0011) [2023-12-27 01:51:18,916][105692] Updated weights for policy 0, policy_version 1436644 (0.0010) [2023-12-27 01:51:18,974][105692] Updated weights for policy 0, policy_version 1436654 (0.0010) [2023-12-27 01:51:19,515][105620] Updated weights for policy 1, policy_version 1438959 (0.0008) [2023-12-27 01:51:19,571][105620] Updated weights for policy 1, policy_version 1438969 (0.0009) [2023-12-27 01:51:19,626][105620] Updated weights for policy 1, policy_version 1438979 (0.0009) [2023-12-27 01:51:19,668][105692] Updated weights for policy 0, policy_version 1436664 (0.0007) [2023-12-27 01:51:19,732][105692] Updated weights for policy 0, policy_version 1436674 (0.0006) [2023-12-27 01:51:19,798][105692] Updated weights for policy 0, policy_version 1436684 (0.0009) [2023-12-27 01:51:20,395][105620] Updated weights for policy 1, policy_version 1438989 (0.0010) [2023-12-27 01:51:20,444][105620] Updated weights for policy 1, policy_version 1438999 (0.0010) [2023-12-27 01:51:20,511][105620] Updated weights for policy 1, policy_version 1439009 (0.0011) [2023-12-27 01:51:20,517][105692] Updated weights for policy 0, policy_version 1436694 (0.0011) [2023-12-27 01:51:20,576][105692] Updated weights for policy 0, policy_version 1436704 (0.0011) [2023-12-27 01:51:20,641][105692] Updated weights for policy 0, policy_version 1436714 (0.0011) [2023-12-27 01:51:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19716.3). Total num frames: 736288768. Throughput: 0: 9709.3, 1: 9851.5. Samples: 736279680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:51:21,062][104569] Avg episode reward: [(0, '8894.523'), (1, '8991.761')] [2023-12-27 01:51:21,277][105620] Updated weights for policy 1, policy_version 1439019 (0.0010) [2023-12-27 01:51:21,345][105620] Updated weights for policy 1, policy_version 1439029 (0.0010) [2023-12-27 01:51:21,412][105692] Updated weights for policy 0, policy_version 1436724 (0.0011) [2023-12-27 01:51:21,415][105620] Updated weights for policy 1, policy_version 1439039 (0.0008) [2023-12-27 01:51:21,472][105692] Updated weights for policy 0, policy_version 1436734 (0.0011) [2023-12-27 01:51:21,532][105692] Updated weights for policy 0, policy_version 1436744 (0.0011) [2023-12-27 01:51:22,181][105620] Updated weights for policy 1, policy_version 1439049 (0.0006) [2023-12-27 01:51:22,238][105620] Updated weights for policy 1, policy_version 1439059 (0.0008) [2023-12-27 01:51:22,302][105620] Updated weights for policy 1, policy_version 1439069 (0.0008) [2023-12-27 01:51:22,327][105692] Updated weights for policy 0, policy_version 1436754 (0.0011) [2023-12-27 01:51:22,370][105620] Updated weights for policy 1, policy_version 1439079 (0.0008) [2023-12-27 01:51:22,394][105692] Updated weights for policy 0, policy_version 1436764 (0.0010) [2023-12-27 01:51:22,454][105692] Updated weights for policy 0, policy_version 1436774 (0.0011) [2023-12-27 01:51:22,513][105692] Updated weights for policy 0, policy_version 1436784 (0.0011) [2023-12-27 01:51:23,041][105620] Updated weights for policy 1, policy_version 1439089 (0.0008) [2023-12-27 01:51:23,089][105620] Updated weights for policy 1, policy_version 1439099 (0.0009) [2023-12-27 01:51:23,137][105620] Updated weights for policy 1, policy_version 1439109 (0.0009) [2023-12-27 01:51:23,252][105692] Updated weights for policy 0, policy_version 1436794 (0.0005) [2023-12-27 01:51:23,300][105692] Updated weights for policy 0, policy_version 1436804 (0.0008) [2023-12-27 01:51:23,357][105692] Updated weights for policy 0, policy_version 1436814 (0.0009) [2023-12-27 01:51:23,785][105620] Updated weights for policy 1, policy_version 1439119 (0.0009) [2023-12-27 01:51:23,844][105620] Updated weights for policy 1, policy_version 1439129 (0.0010) [2023-12-27 01:51:23,892][105620] Updated weights for policy 1, policy_version 1439139 (0.0010) [2023-12-27 01:51:23,934][105692] Updated weights for policy 0, policy_version 1436824 (0.0008) [2023-12-27 01:51:23,988][105692] Updated weights for policy 0, policy_version 1436834 (0.0009) [2023-12-27 01:51:24,040][105692] Updated weights for policy 0, policy_version 1436844 (0.0009) [2023-12-27 01:51:24,577][105620] Updated weights for policy 1, policy_version 1439149 (0.0009) [2023-12-27 01:51:24,639][105620] Updated weights for policy 1, policy_version 1439159 (0.0009) [2023-12-27 01:51:24,693][105620] Updated weights for policy 1, policy_version 1439169 (0.0005) [2023-12-27 01:51:24,774][105692] Updated weights for policy 0, policy_version 1436854 (0.0006) [2023-12-27 01:51:24,837][105692] Updated weights for policy 0, policy_version 1436864 (0.0005) [2023-12-27 01:51:24,896][105692] Updated weights for policy 0, policy_version 1436874 (0.0007) [2023-12-27 01:51:25,323][105620] Updated weights for policy 1, policy_version 1439179 (0.0009) [2023-12-27 01:51:25,391][105620] Updated weights for policy 1, policy_version 1439189 (0.0007) [2023-12-27 01:51:25,458][105620] Updated weights for policy 1, policy_version 1439199 (0.0009) [2023-12-27 01:51:25,659][105692] Updated weights for policy 0, policy_version 1436885 (0.0009) [2023-12-27 01:51:25,718][105692] Updated weights for policy 0, policy_version 1436895 (0.0009) [2023-12-27 01:51:25,774][105692] Updated weights for policy 0, policy_version 1436905 (0.0008) [2023-12-27 01:51:26,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19716.3). Total num frames: 736387072. Throughput: 0: 9787.3, 1: 9753.6. Samples: 736396100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:51:26,063][104569] Avg episode reward: [(0, '8527.854'), (1, '8627.479')] [2023-12-27 01:51:26,177][105620] Updated weights for policy 1, policy_version 1439209 (0.0010) [2023-12-27 01:51:26,232][105620] Updated weights for policy 1, policy_version 1439219 (0.0010) [2023-12-27 01:51:26,280][105620] Updated weights for policy 1, policy_version 1439229 (0.0010) [2023-12-27 01:51:26,329][105620] Updated weights for policy 1, policy_version 1439239 (0.0010) [2023-12-27 01:51:26,489][105692] Updated weights for policy 0, policy_version 1436915 (0.0008) [2023-12-27 01:51:26,540][105692] Updated weights for policy 0, policy_version 1436925 (0.0008) [2023-12-27 01:51:26,587][105692] Updated weights for policy 0, policy_version 1436935 (0.0008) [2023-12-27 01:51:27,094][105620] Updated weights for policy 1, policy_version 1439249 (0.0010) [2023-12-27 01:51:27,138][105620] Updated weights for policy 1, policy_version 1439259 (0.0010) [2023-12-27 01:51:27,194][105620] Updated weights for policy 1, policy_version 1439269 (0.0009) [2023-12-27 01:51:27,323][105692] Updated weights for policy 0, policy_version 1436945 (0.0008) [2023-12-27 01:51:27,372][105692] Updated weights for policy 0, policy_version 1436955 (0.0007) [2023-12-27 01:51:27,418][105692] Updated weights for policy 0, policy_version 1436965 (0.0008) [2023-12-27 01:51:27,466][105692] Updated weights for policy 0, policy_version 1436975 (0.0008) [2023-12-27 01:51:27,950][105620] Updated weights for policy 1, policy_version 1439279 (0.0010) [2023-12-27 01:51:27,997][105620] Updated weights for policy 1, policy_version 1439289 (0.0010) [2023-12-27 01:51:28,056][105620] Updated weights for policy 1, policy_version 1439299 (0.0010) [2023-12-27 01:51:28,077][105692] Updated weights for policy 0, policy_version 1436985 (0.0010) [2023-12-27 01:51:28,124][105692] Updated weights for policy 0, policy_version 1436995 (0.0010) [2023-12-27 01:51:28,175][105692] Updated weights for policy 0, policy_version 1437005 (0.0010) [2023-12-27 01:51:28,737][105620] Updated weights for policy 1, policy_version 1439309 (0.0010) [2023-12-27 01:51:28,800][105620] Updated weights for policy 1, policy_version 1439319 (0.0011) [2023-12-27 01:51:28,826][105692] Updated weights for policy 0, policy_version 1437015 (0.0008) [2023-12-27 01:51:28,863][105620] Updated weights for policy 1, policy_version 1439329 (0.0007) [2023-12-27 01:51:28,889][105692] Updated weights for policy 0, policy_version 1437025 (0.0011) [2023-12-27 01:51:28,949][105692] Updated weights for policy 0, policy_version 1437035 (0.0007) [2023-12-27 01:51:29,529][105620] Updated weights for policy 1, policy_version 1439339 (0.0008) [2023-12-27 01:51:29,561][105692] Updated weights for policy 0, policy_version 1437045 (0.0006) [2023-12-27 01:51:29,589][105620] Updated weights for policy 1, policy_version 1439349 (0.0008) [2023-12-27 01:51:29,592][105585] KL-divergence is very high: 127.6568 [2023-12-27 01:51:29,605][105692] Updated weights for policy 0, policy_version 1437055 (0.0007) [2023-12-27 01:51:29,611][105585] KL-divergence is very high: 294.0743 [2023-12-27 01:51:29,629][105585] KL-divergence is very high: 481.9341 [2023-12-27 01:51:29,635][105620] Updated weights for policy 1, policy_version 1439360 (0.0006) [2023-12-27 01:51:29,651][105585] KL-divergence is very high: 525.7931 [2023-12-27 01:51:29,657][105692] Updated weights for policy 0, policy_version 1437065 (0.0010) [2023-12-27 01:51:29,671][105585] KL-divergence is very high: 697.2659 [2023-12-27 01:51:30,290][105692] Updated weights for policy 0, policy_version 1437075 (0.0009) [2023-12-27 01:51:30,341][105692] Updated weights for policy 0, policy_version 1437085 (0.0006) [2023-12-27 01:51:30,389][105692] Updated weights for policy 0, policy_version 1437095 (0.0009) [2023-12-27 01:51:30,492][105620] Updated weights for policy 1, policy_version 1439370 (0.0006) [2023-12-27 01:51:30,553][105620] Updated weights for policy 1, policy_version 1439380 (0.0010) [2023-12-27 01:51:30,608][105620] Updated weights for policy 1, policy_version 1439390 (0.0009) [2023-12-27 01:51:30,654][105620] Updated weights for policy 1, policy_version 1439400 (0.0009) [2023-12-27 01:51:30,966][105692] Updated weights for policy 0, policy_version 1437105 (0.0006) [2023-12-27 01:51:31,027][105692] Updated weights for policy 0, policy_version 1437115 (0.0007) [2023-12-27 01:51:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19716.3). Total num frames: 736485376. Throughput: 0: 9857.4, 1: 9752.0. Samples: 736455716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:51:31,062][104569] Avg episode reward: [(0, '8250.789'), (1, '8719.637')] [2023-12-27 01:51:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001439400_368533504.pth... [2023-12-27 01:51:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001438280_368246784.pth [2023-12-27 01:51:31,085][105692] Updated weights for policy 0, policy_version 1437125 (0.0009) [2023-12-27 01:51:31,148][105692] Updated weights for policy 0, policy_version 1437135 (0.0009) [2023-12-27 01:51:31,155][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001437136_367960064.pth... [2023-12-27 01:51:31,159][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001435952_367656960.pth [2023-12-27 01:51:31,558][105620] Updated weights for policy 1, policy_version 1439410 (0.0009) [2023-12-27 01:51:31,623][105620] Updated weights for policy 1, policy_version 1439420 (0.0009) [2023-12-27 01:51:31,683][105620] Updated weights for policy 1, policy_version 1439430 (0.0009) [2023-12-27 01:51:31,786][105692] Updated weights for policy 0, policy_version 1437145 (0.0009) [2023-12-27 01:51:31,842][105692] Updated weights for policy 0, policy_version 1437155 (0.0009) [2023-12-27 01:51:31,889][105692] Updated weights for policy 0, policy_version 1437165 (0.0009) [2023-12-27 01:51:32,407][105620] Updated weights for policy 1, policy_version 1439440 (0.0007) [2023-12-27 01:51:32,462][105620] Updated weights for policy 1, policy_version 1439450 (0.0005) [2023-12-27 01:51:32,516][105620] Updated weights for policy 1, policy_version 1439460 (0.0005) [2023-12-27 01:51:32,683][105692] Updated weights for policy 0, policy_version 1437175 (0.0007) [2023-12-27 01:51:32,729][105692] Updated weights for policy 0, policy_version 1437185 (0.0005) [2023-12-27 01:51:32,787][105692] Updated weights for policy 0, policy_version 1437195 (0.0006) [2023-12-27 01:51:33,114][105620] Updated weights for policy 1, policy_version 1439470 (0.0005) [2023-12-27 01:51:33,168][105620] Updated weights for policy 1, policy_version 1439480 (0.0005) [2023-12-27 01:51:33,232][105620] Updated weights for policy 1, policy_version 1439490 (0.0006) [2023-12-27 01:51:33,386][105692] Updated weights for policy 0, policy_version 1437205 (0.0008) [2023-12-27 01:51:33,440][105692] Updated weights for policy 0, policy_version 1437215 (0.0010) [2023-12-27 01:51:33,498][105692] Updated weights for policy 0, policy_version 1437225 (0.0010) [2023-12-27 01:51:33,926][105620] Updated weights for policy 1, policy_version 1439500 (0.0009) [2023-12-27 01:51:33,992][105620] Updated weights for policy 1, policy_version 1439510 (0.0010) [2023-12-27 01:51:34,056][105692] Updated weights for policy 0, policy_version 1437235 (0.0009) [2023-12-27 01:51:34,056][105620] Updated weights for policy 1, policy_version 1439520 (0.0009) [2023-12-27 01:51:34,110][105692] Updated weights for policy 0, policy_version 1437245 (0.0005) [2023-12-27 01:51:34,171][105692] Updated weights for policy 0, policy_version 1437255 (0.0006) [2023-12-27 01:51:34,757][105620] Updated weights for policy 1, policy_version 1439530 (0.0008) [2023-12-27 01:51:34,813][105620] Updated weights for policy 1, policy_version 1439540 (0.0008) [2023-12-27 01:51:34,861][105620] Updated weights for policy 1, policy_version 1439550 (0.0008) [2023-12-27 01:51:34,883][105692] Updated weights for policy 0, policy_version 1437265 (0.0006) [2023-12-27 01:51:34,919][105620] Updated weights for policy 1, policy_version 1439560 (0.0007) [2023-12-27 01:51:34,939][105692] Updated weights for policy 0, policy_version 1437275 (0.0009) [2023-12-27 01:51:34,990][105692] Updated weights for policy 0, policy_version 1437285 (0.0010) [2023-12-27 01:51:35,042][105692] Updated weights for policy 0, policy_version 1437295 (0.0010) [2023-12-27 01:51:35,549][105620] Updated weights for policy 1, policy_version 1439570 (0.0008) [2023-12-27 01:51:35,598][105620] Updated weights for policy 1, policy_version 1439580 (0.0008) [2023-12-27 01:51:35,646][105620] Updated weights for policy 1, policy_version 1439590 (0.0008) [2023-12-27 01:51:35,803][105692] Updated weights for policy 0, policy_version 1437305 (0.0007) [2023-12-27 01:51:35,866][105692] Updated weights for policy 0, policy_version 1437315 (0.0009) [2023-12-27 01:51:35,925][105692] Updated weights for policy 0, policy_version 1437325 (0.0011) [2023-12-27 01:51:36,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.9, 300 sec: 19716.3). Total num frames: 736591872. Throughput: 0: 10055.1, 1: 9685.6. Samples: 736577956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:51:36,062][104569] Avg episode reward: [(0, '8800.051'), (1, '8992.594')] [2023-12-27 01:51:36,449][105620] Updated weights for policy 1, policy_version 1439600 (0.0009) [2023-12-27 01:51:36,515][105620] Updated weights for policy 1, policy_version 1439610 (0.0009) [2023-12-27 01:51:36,577][105620] Updated weights for policy 1, policy_version 1439620 (0.0009) [2023-12-27 01:51:36,580][105692] Updated weights for policy 0, policy_version 1437335 (0.0011) [2023-12-27 01:51:36,643][105692] Updated weights for policy 0, policy_version 1437345 (0.0011) [2023-12-27 01:51:36,709][105692] Updated weights for policy 0, policy_version 1437355 (0.0011) [2023-12-27 01:51:37,259][105620] Updated weights for policy 1, policy_version 1439630 (0.0009) [2023-12-27 01:51:37,316][105620] Updated weights for policy 1, policy_version 1439640 (0.0009) [2023-12-27 01:51:37,370][105620] Updated weights for policy 1, policy_version 1439650 (0.0008) [2023-12-27 01:51:37,458][105692] Updated weights for policy 0, policy_version 1437365 (0.0008) [2023-12-27 01:51:37,514][105692] Updated weights for policy 0, policy_version 1437375 (0.0008) [2023-12-27 01:51:37,566][105692] Updated weights for policy 0, policy_version 1437385 (0.0009) [2023-12-27 01:51:38,166][105620] Updated weights for policy 1, policy_version 1439660 (0.0008) [2023-12-27 01:51:38,221][105620] Updated weights for policy 1, policy_version 1439670 (0.0008) [2023-12-27 01:51:38,283][105620] Updated weights for policy 1, policy_version 1439680 (0.0008) [2023-12-27 01:51:38,285][105692] Updated weights for policy 0, policy_version 1437395 (0.0009) [2023-12-27 01:51:38,352][105692] Updated weights for policy 0, policy_version 1437405 (0.0010) [2023-12-27 01:51:38,419][105692] Updated weights for policy 0, policy_version 1437416 (0.0007) [2023-12-27 01:51:38,963][105620] Updated weights for policy 1, policy_version 1439690 (0.0006) [2023-12-27 01:51:39,018][105620] Updated weights for policy 1, policy_version 1439700 (0.0008) [2023-12-27 01:51:39,063][105692] Updated weights for policy 0, policy_version 1437426 (0.0006) [2023-12-27 01:51:39,076][105620] Updated weights for policy 1, policy_version 1439710 (0.0008) [2023-12-27 01:51:39,107][105692] Updated weights for policy 0, policy_version 1437436 (0.0010) [2023-12-27 01:51:39,134][105620] Updated weights for policy 1, policy_version 1439720 (0.0009) [2023-12-27 01:51:39,157][105692] Updated weights for policy 0, policy_version 1437446 (0.0008) [2023-12-27 01:51:39,203][105692] Updated weights for policy 0, policy_version 1437456 (0.0007) [2023-12-27 01:51:39,888][105620] Updated weights for policy 1, policy_version 1439730 (0.0008) [2023-12-27 01:51:39,894][105692] Updated weights for policy 0, policy_version 1437466 (0.0006) [2023-12-27 01:51:39,954][105620] Updated weights for policy 1, policy_version 1439740 (0.0007) [2023-12-27 01:51:39,955][105692] Updated weights for policy 0, policy_version 1437476 (0.0009) [2023-12-27 01:51:40,012][105692] Updated weights for policy 0, policy_version 1437486 (0.0008) [2023-12-27 01:51:40,016][105620] Updated weights for policy 1, policy_version 1439750 (0.0006) [2023-12-27 01:51:40,727][105692] Updated weights for policy 0, policy_version 1437496 (0.0008) [2023-12-27 01:51:40,774][105620] Updated weights for policy 1, policy_version 1439760 (0.0007) [2023-12-27 01:51:40,781][105692] Updated weights for policy 0, policy_version 1437506 (0.0006) [2023-12-27 01:51:40,832][105692] Updated weights for policy 0, policy_version 1437516 (0.0008) [2023-12-27 01:51:40,835][105620] Updated weights for policy 1, policy_version 1439770 (0.0009) [2023-12-27 01:51:40,894][105620] Updated weights for policy 1, policy_version 1439780 (0.0009) [2023-12-27 01:51:41,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19660.7, 300 sec: 19744.1). Total num frames: 736690176. Throughput: 0: 10077.7, 1: 9677.8. Samples: 736694868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:51:41,063][104569] Avg episode reward: [(0, '8800.499'), (1, '8900.513')] [2023-12-27 01:51:41,677][105620] Updated weights for policy 1, policy_version 1439790 (0.0008) [2023-12-27 01:51:41,705][105692] Updated weights for policy 0, policy_version 1437526 (0.0011) [2023-12-27 01:51:41,742][105620] Updated weights for policy 1, policy_version 1439800 (0.0007) [2023-12-27 01:51:41,773][105692] Updated weights for policy 0, policy_version 1437536 (0.0007) [2023-12-27 01:51:41,801][105620] Updated weights for policy 1, policy_version 1439810 (0.0006) [2023-12-27 01:51:41,832][105692] Updated weights for policy 0, policy_version 1437546 (0.0007) [2023-12-27 01:51:42,556][105620] Updated weights for policy 1, policy_version 1439820 (0.0008) [2023-12-27 01:51:42,562][105692] Updated weights for policy 0, policy_version 1437556 (0.0006) [2023-12-27 01:51:42,621][105620] Updated weights for policy 1, policy_version 1439830 (0.0009) [2023-12-27 01:51:42,628][105692] Updated weights for policy 0, policy_version 1437566 (0.0006) [2023-12-27 01:51:42,679][105620] Updated weights for policy 1, policy_version 1439840 (0.0007) [2023-12-27 01:51:42,689][105692] Updated weights for policy 0, policy_version 1437576 (0.0006) [2023-12-27 01:51:43,301][105620] Updated weights for policy 1, policy_version 1439850 (0.0007) [2023-12-27 01:51:43,346][105692] Updated weights for policy 0, policy_version 1437586 (0.0006) [2023-12-27 01:51:43,367][105620] Updated weights for policy 1, policy_version 1439860 (0.0009) [2023-12-27 01:51:43,396][105692] Updated weights for policy 0, policy_version 1437596 (0.0007) [2023-12-27 01:51:43,432][105620] Updated weights for policy 1, policy_version 1439870 (0.0006) [2023-12-27 01:51:43,451][105692] Updated weights for policy 0, policy_version 1437606 (0.0007) [2023-12-27 01:51:43,483][105620] Updated weights for policy 1, policy_version 1439880 (0.0006) [2023-12-27 01:51:43,522][105692] Updated weights for policy 0, policy_version 1437616 (0.0009) [2023-12-27 01:51:44,128][105620] Updated weights for policy 1, policy_version 1439890 (0.0009) [2023-12-27 01:51:44,192][105620] Updated weights for policy 1, policy_version 1439900 (0.0007) [2023-12-27 01:51:44,253][105620] Updated weights for policy 1, policy_version 1439910 (0.0008) [2023-12-27 01:51:44,294][105692] Updated weights for policy 0, policy_version 1437626 (0.0007) [2023-12-27 01:51:44,361][105692] Updated weights for policy 0, policy_version 1437636 (0.0009) [2023-12-27 01:51:44,420][105692] Updated weights for policy 0, policy_version 1437646 (0.0010) [2023-12-27 01:51:44,811][105620] Updated weights for policy 1, policy_version 1439920 (0.0009) [2023-12-27 01:51:44,872][105620] Updated weights for policy 1, policy_version 1439930 (0.0010) [2023-12-27 01:51:44,935][105620] Updated weights for policy 1, policy_version 1439940 (0.0011) [2023-12-27 01:51:45,329][105692] Updated weights for policy 0, policy_version 1437656 (0.0009) [2023-12-27 01:51:45,394][105692] Updated weights for policy 0, policy_version 1437666 (0.0008) [2023-12-27 01:51:45,465][105692] Updated weights for policy 0, policy_version 1437676 (0.0009) [2023-12-27 01:51:45,583][105620] Updated weights for policy 1, policy_version 1439950 (0.0010) [2023-12-27 01:51:45,634][105620] Updated weights for policy 1, policy_version 1439960 (0.0010) [2023-12-27 01:51:45,686][105620] Updated weights for policy 1, policy_version 1439970 (0.0010) [2023-12-27 01:51:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19660.7, 300 sec: 19716.3). Total num frames: 736780288. Throughput: 0: 9995.8, 1: 9697.0. Samples: 736752140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:51:46,062][104569] Avg episode reward: [(0, '8705.645'), (1, '8807.762')] [2023-12-27 01:51:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001437680_368099328.pth... [2023-12-27 01:51:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001439976_368680960.pth... [2023-12-27 01:51:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001436528_367804416.pth [2023-12-27 01:51:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001438824_368386048.pth [2023-12-27 01:51:46,210][105692] Updated weights for policy 0, policy_version 1437686 (0.0008) [2023-12-27 01:51:46,259][105692] Updated weights for policy 0, policy_version 1437696 (0.0008) [2023-12-27 01:51:46,317][105692] Updated weights for policy 0, policy_version 1437706 (0.0008) [2023-12-27 01:51:46,436][105620] Updated weights for policy 1, policy_version 1439980 (0.0010) [2023-12-27 01:51:46,495][105620] Updated weights for policy 1, policy_version 1439990 (0.0010) [2023-12-27 01:51:46,556][105620] Updated weights for policy 1, policy_version 1440000 (0.0010) [2023-12-27 01:51:47,149][105692] Updated weights for policy 0, policy_version 1437716 (0.0008) [2023-12-27 01:51:47,174][105620] Updated weights for policy 1, policy_version 1440010 (0.0010) [2023-12-27 01:51:47,211][105692] Updated weights for policy 0, policy_version 1437726 (0.0006) [2023-12-27 01:51:47,221][105620] Updated weights for policy 1, policy_version 1440020 (0.0010) [2023-12-27 01:51:47,267][105692] Updated weights for policy 0, policy_version 1437736 (0.0005) [2023-12-27 01:51:47,269][105620] Updated weights for policy 1, policy_version 1440030 (0.0010) [2023-12-27 01:51:47,320][105620] Updated weights for policy 1, policy_version 1440040 (0.0010) [2023-12-27 01:51:47,830][105692] Updated weights for policy 0, policy_version 1437746 (0.0006) [2023-12-27 01:51:47,881][105692] Updated weights for policy 0, policy_version 1437756 (0.0006) [2023-12-27 01:51:47,940][105692] Updated weights for policy 0, policy_version 1437766 (0.0005) [2023-12-27 01:51:47,993][105692] Updated weights for policy 0, policy_version 1437776 (0.0005) [2023-12-27 01:51:48,069][105620] Updated weights for policy 1, policy_version 1440050 (0.0010) [2023-12-27 01:51:48,127][105620] Updated weights for policy 1, policy_version 1440060 (0.0010) [2023-12-27 01:51:48,181][105620] Updated weights for policy 1, policy_version 1440070 (0.0010) [2023-12-27 01:51:48,701][105692] Updated weights for policy 0, policy_version 1437786 (0.0010) [2023-12-27 01:51:48,750][105692] Updated weights for policy 0, policy_version 1437796 (0.0010) [2023-12-27 01:51:48,798][105692] Updated weights for policy 0, policy_version 1437806 (0.0010) [2023-12-27 01:51:48,926][105620] Updated weights for policy 1, policy_version 1440080 (0.0009) [2023-12-27 01:51:48,974][105620] Updated weights for policy 1, policy_version 1440090 (0.0008) [2023-12-27 01:51:49,023][105620] Updated weights for policy 1, policy_version 1440100 (0.0009) [2023-12-27 01:51:49,566][105692] Updated weights for policy 0, policy_version 1437816 (0.0009) [2023-12-27 01:51:49,633][105692] Updated weights for policy 0, policy_version 1437826 (0.0008) [2023-12-27 01:51:49,700][105692] Updated weights for policy 0, policy_version 1437836 (0.0009) [2023-12-27 01:51:49,783][105620] Updated weights for policy 1, policy_version 1440110 (0.0009) [2023-12-27 01:51:49,836][105620] Updated weights for policy 1, policy_version 1440120 (0.0008) [2023-12-27 01:51:49,901][105620] Updated weights for policy 1, policy_version 1440130 (0.0008) [2023-12-27 01:51:50,441][105692] Updated weights for policy 0, policy_version 1437846 (0.0009) [2023-12-27 01:51:50,488][105692] Updated weights for policy 0, policy_version 1437856 (0.0009) [2023-12-27 01:51:50,544][105692] Updated weights for policy 0, policy_version 1437866 (0.0009) [2023-12-27 01:51:50,652][105620] Updated weights for policy 1, policy_version 1440140 (0.0008) [2023-12-27 01:51:50,703][105620] Updated weights for policy 1, policy_version 1440150 (0.0009) [2023-12-27 01:51:50,756][105620] Updated weights for policy 1, policy_version 1440160 (0.0009) [2023-12-27 01:51:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 736878592. Throughput: 0: 10006.7, 1: 9697.5. Samples: 736868172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:51:51,063][104569] Avg episode reward: [(0, '8803.238'), (1, '8807.767')] [2023-12-27 01:51:51,362][105692] Updated weights for policy 0, policy_version 1437876 (0.0009) [2023-12-27 01:51:51,425][105692] Updated weights for policy 0, policy_version 1437886 (0.0006) [2023-12-27 01:51:51,479][105692] Updated weights for policy 0, policy_version 1437896 (0.0005) [2023-12-27 01:51:51,522][105620] Updated weights for policy 1, policy_version 1440170 (0.0009) [2023-12-27 01:51:51,581][105620] Updated weights for policy 1, policy_version 1440180 (0.0010) [2023-12-27 01:51:51,643][105620] Updated weights for policy 1, policy_version 1440190 (0.0009) [2023-12-27 01:51:51,711][105620] Updated weights for policy 1, policy_version 1440200 (0.0008) [2023-12-27 01:51:52,187][105692] Updated weights for policy 0, policy_version 1437906 (0.0006) [2023-12-27 01:51:52,249][105692] Updated weights for policy 0, policy_version 1437916 (0.0009) [2023-12-27 01:51:52,307][105692] Updated weights for policy 0, policy_version 1437926 (0.0009) [2023-12-27 01:51:52,370][105692] Updated weights for policy 0, policy_version 1437936 (0.0007) [2023-12-27 01:51:52,465][105620] Updated weights for policy 1, policy_version 1440210 (0.0009) [2023-12-27 01:51:52,526][105620] Updated weights for policy 1, policy_version 1440220 (0.0009) [2023-12-27 01:51:52,590][105620] Updated weights for policy 1, policy_version 1440230 (0.0008) [2023-12-27 01:51:53,134][105692] Updated weights for policy 0, policy_version 1437946 (0.0009) [2023-12-27 01:51:53,182][105692] Updated weights for policy 0, policy_version 1437956 (0.0009) [2023-12-27 01:51:53,240][105692] Updated weights for policy 0, policy_version 1437966 (0.0009) [2023-12-27 01:51:53,347][105620] Updated weights for policy 1, policy_version 1440240 (0.0009) [2023-12-27 01:51:53,405][105620] Updated weights for policy 1, policy_version 1440250 (0.0010) [2023-12-27 01:51:53,467][105620] Updated weights for policy 1, policy_version 1440260 (0.0010) [2023-12-27 01:51:54,062][105620] Updated weights for policy 1, policy_version 1440270 (0.0008) [2023-12-27 01:51:54,087][105692] Updated weights for policy 0, policy_version 1437976 (0.0008) [2023-12-27 01:51:54,118][105620] Updated weights for policy 1, policy_version 1440280 (0.0005) [2023-12-27 01:51:54,139][105692] Updated weights for policy 0, policy_version 1437986 (0.0009) [2023-12-27 01:51:54,183][105620] Updated weights for policy 1, policy_version 1440290 (0.0008) [2023-12-27 01:51:54,189][105692] Updated weights for policy 0, policy_version 1437996 (0.0008) [2023-12-27 01:51:54,729][105620] Updated weights for policy 1, policy_version 1440300 (0.0006) [2023-12-27 01:51:54,779][105620] Updated weights for policy 1, policy_version 1440310 (0.0006) [2023-12-27 01:51:54,842][105620] Updated weights for policy 1, policy_version 1440320 (0.0006) [2023-12-27 01:51:55,054][105692] Updated weights for policy 0, policy_version 1438006 (0.0008) [2023-12-27 01:51:55,112][105692] Updated weights for policy 0, policy_version 1438016 (0.0010) [2023-12-27 01:51:55,165][105692] Updated weights for policy 0, policy_version 1438026 (0.0009) [2023-12-27 01:51:55,368][105620] Updated weights for policy 1, policy_version 1440330 (0.0005) [2023-12-27 01:51:55,417][105620] Updated weights for policy 1, policy_version 1440340 (0.0005) [2023-12-27 01:51:55,473][105620] Updated weights for policy 1, policy_version 1440350 (0.0005) [2023-12-27 01:51:55,531][105620] Updated weights for policy 1, policy_version 1440360 (0.0006) [2023-12-27 01:51:55,841][105692] Updated weights for policy 0, policy_version 1438036 (0.0008) [2023-12-27 01:51:55,888][105692] Updated weights for policy 0, policy_version 1438046 (0.0006) [2023-12-27 01:51:55,941][105692] Updated weights for policy 0, policy_version 1438056 (0.0009) [2023-12-27 01:51:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 736976896. Throughput: 0: 9784.4, 1: 9818.6. Samples: 736983724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:51:56,063][104569] Avg episode reward: [(0, '8531.059'), (1, '8901.068')] [2023-12-27 01:51:56,225][105620] Updated weights for policy 1, policy_version 1440370 (0.0008) [2023-12-27 01:51:56,275][105620] Updated weights for policy 1, policy_version 1440380 (0.0009) [2023-12-27 01:51:56,333][105620] Updated weights for policy 1, policy_version 1440390 (0.0008) [2023-12-27 01:51:56,681][105692] Updated weights for policy 0, policy_version 1438066 (0.0009) [2023-12-27 01:51:56,743][105692] Updated weights for policy 0, policy_version 1438076 (0.0009) [2023-12-27 01:51:56,808][105692] Updated weights for policy 0, policy_version 1438086 (0.0009) [2023-12-27 01:51:56,858][105692] Updated weights for policy 0, policy_version 1438096 (0.0009) [2023-12-27 01:51:57,094][105620] Updated weights for policy 1, policy_version 1440400 (0.0010) [2023-12-27 01:51:57,152][105620] Updated weights for policy 1, policy_version 1440410 (0.0009) [2023-12-27 01:51:57,216][105620] Updated weights for policy 1, policy_version 1440420 (0.0008) [2023-12-27 01:51:57,605][105692] Updated weights for policy 0, policy_version 1438106 (0.0005) [2023-12-27 01:51:57,660][105692] Updated weights for policy 0, policy_version 1438116 (0.0006) [2023-12-27 01:51:57,704][105692] Updated weights for policy 0, policy_version 1438126 (0.0007) [2023-12-27 01:51:57,925][105620] Updated weights for policy 1, policy_version 1440430 (0.0010) [2023-12-27 01:51:57,983][105620] Updated weights for policy 1, policy_version 1440440 (0.0010) [2023-12-27 01:51:58,042][105620] Updated weights for policy 1, policy_version 1440450 (0.0011) [2023-12-27 01:51:58,374][105692] Updated weights for policy 0, policy_version 1438136 (0.0008) [2023-12-27 01:51:58,437][105692] Updated weights for policy 0, policy_version 1438146 (0.0008) [2023-12-27 01:51:58,498][105692] Updated weights for policy 0, policy_version 1438156 (0.0008) [2023-12-27 01:51:58,817][105620] Updated weights for policy 1, policy_version 1440460 (0.0010) [2023-12-27 01:51:58,884][105620] Updated weights for policy 1, policy_version 1440470 (0.0011) [2023-12-27 01:51:58,948][105620] Updated weights for policy 1, policy_version 1440480 (0.0010) [2023-12-27 01:51:59,241][105692] Updated weights for policy 0, policy_version 1438166 (0.0007) [2023-12-27 01:51:59,303][105692] Updated weights for policy 0, policy_version 1438176 (0.0008) [2023-12-27 01:51:59,369][105692] Updated weights for policy 0, policy_version 1438186 (0.0008) [2023-12-27 01:51:59,597][105620] Updated weights for policy 1, policy_version 1440490 (0.0010) [2023-12-27 01:51:59,659][105620] Updated weights for policy 1, policy_version 1440500 (0.0010) [2023-12-27 01:51:59,708][105620] Updated weights for policy 1, policy_version 1440510 (0.0010) [2023-12-27 01:51:59,764][105620] Updated weights for policy 1, policy_version 1440520 (0.0010) [2023-12-27 01:52:00,027][105692] Updated weights for policy 0, policy_version 1438196 (0.0006) [2023-12-27 01:52:00,081][105692] Updated weights for policy 0, policy_version 1438206 (0.0008) [2023-12-27 01:52:00,144][105692] Updated weights for policy 0, policy_version 1438216 (0.0008) [2023-12-27 01:52:00,494][105620] Updated weights for policy 1, policy_version 1440530 (0.0009) [2023-12-27 01:52:00,555][105620] Updated weights for policy 1, policy_version 1440540 (0.0009) [2023-12-27 01:52:00,617][105620] Updated weights for policy 1, policy_version 1440550 (0.0007) [2023-12-27 01:52:00,908][105692] Updated weights for policy 0, policy_version 1438226 (0.0008) [2023-12-27 01:52:00,982][105692] Updated weights for policy 0, policy_version 1438236 (0.0010) [2023-12-27 01:52:01,034][105692] Updated weights for policy 0, policy_version 1438246 (0.0008) [2023-12-27 01:52:01,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 737067008. Throughput: 0: 9838.5, 1: 9776.3. Samples: 737041464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:52:01,062][104569] Avg episode reward: [(0, '8528.244'), (1, '9175.503')] [2023-12-27 01:52:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001440552_368828416.pth... [2023-12-27 01:52:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001439400_368533504.pth [2023-12-27 01:52:01,096][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001438256_368246784.pth... [2023-12-27 01:52:01,098][105692] Updated weights for policy 0, policy_version 1438256 (0.0007) [2023-12-27 01:52:01,101][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001437136_367960064.pth [2023-12-27 01:52:01,248][105620] Updated weights for policy 1, policy_version 1440560 (0.0009) [2023-12-27 01:52:01,300][105620] Updated weights for policy 1, policy_version 1440570 (0.0009) [2023-12-27 01:52:01,362][105620] Updated weights for policy 1, policy_version 1440580 (0.0009) [2023-12-27 01:52:01,859][105692] Updated weights for policy 0, policy_version 1438266 (0.0009) [2023-12-27 01:52:01,915][105692] Updated weights for policy 0, policy_version 1438276 (0.0009) [2023-12-27 01:52:01,977][105692] Updated weights for policy 0, policy_version 1438286 (0.0009) [2023-12-27 01:52:02,109][105620] Updated weights for policy 1, policy_version 1440590 (0.0009) [2023-12-27 01:52:02,166][105620] Updated weights for policy 1, policy_version 1440600 (0.0011) [2023-12-27 01:52:02,223][105620] Updated weights for policy 1, policy_version 1440611 (0.0010) [2023-12-27 01:52:02,726][105692] Updated weights for policy 0, policy_version 1438296 (0.0009) [2023-12-27 01:52:02,778][105692] Updated weights for policy 0, policy_version 1438306 (0.0005) [2023-12-27 01:52:02,831][105692] Updated weights for policy 0, policy_version 1438316 (0.0008) [2023-12-27 01:52:02,917][105620] Updated weights for policy 1, policy_version 1440621 (0.0010) [2023-12-27 01:52:02,962][105620] Updated weights for policy 1, policy_version 1440631 (0.0010) [2023-12-27 01:52:03,010][105620] Updated weights for policy 1, policy_version 1440641 (0.0010) [2023-12-27 01:52:03,585][105620] Updated weights for policy 1, policy_version 1440651 (0.0009) [2023-12-27 01:52:03,632][105620] Updated weights for policy 1, policy_version 1440661 (0.0006) [2023-12-27 01:52:03,673][105692] Updated weights for policy 0, policy_version 1438326 (0.0007) [2023-12-27 01:52:03,675][105620] Updated weights for policy 1, policy_version 1440671 (0.0010) [2023-12-27 01:52:03,725][105692] Updated weights for policy 0, policy_version 1438336 (0.0006) [2023-12-27 01:52:03,789][105692] Updated weights for policy 0, policy_version 1438346 (0.0008) [2023-12-27 01:52:04,351][105620] Updated weights for policy 1, policy_version 1440681 (0.0010) [2023-12-27 01:52:04,408][105620] Updated weights for policy 1, policy_version 1440691 (0.0005) [2023-12-27 01:52:04,465][105620] Updated weights for policy 1, policy_version 1440701 (0.0005) [2023-12-27 01:52:04,519][105620] Updated weights for policy 1, policy_version 1440711 (0.0005) [2023-12-27 01:52:04,619][105692] Updated weights for policy 0, policy_version 1438356 (0.0007) [2023-12-27 01:52:04,679][105692] Updated weights for policy 0, policy_version 1438366 (0.0008) [2023-12-27 01:52:04,741][105692] Updated weights for policy 0, policy_version 1438376 (0.0008) [2023-12-27 01:52:05,189][105620] Updated weights for policy 1, policy_version 1440721 (0.0010) [2023-12-27 01:52:05,247][105620] Updated weights for policy 1, policy_version 1440731 (0.0010) [2023-12-27 01:52:05,315][105620] Updated weights for policy 1, policy_version 1440741 (0.0010) [2023-12-27 01:52:05,509][105692] Updated weights for policy 0, policy_version 1438386 (0.0008) [2023-12-27 01:52:05,557][105692] Updated weights for policy 0, policy_version 1438396 (0.0008) [2023-12-27 01:52:05,601][105692] Updated weights for policy 0, policy_version 1438406 (0.0008) [2023-12-27 01:52:05,656][105692] Updated weights for policy 0, policy_version 1438416 (0.0008) [2023-12-27 01:52:06,034][105620] Updated weights for policy 1, policy_version 1440751 (0.0010) [2023-12-27 01:52:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 737165312. Throughput: 0: 9669.8, 1: 9864.8. Samples: 737158736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:52:06,062][104569] Avg episode reward: [(0, '8711.582'), (1, '9078.481')] [2023-12-27 01:52:06,090][105620] Updated weights for policy 1, policy_version 1440761 (0.0009) [2023-12-27 01:52:06,161][105620] Updated weights for policy 1, policy_version 1440771 (0.0011) [2023-12-27 01:52:06,462][105692] Updated weights for policy 0, policy_version 1438426 (0.0008) [2023-12-27 01:52:06,522][105692] Updated weights for policy 0, policy_version 1438436 (0.0008) [2023-12-27 01:52:06,578][105692] Updated weights for policy 0, policy_version 1438446 (0.0008) [2023-12-27 01:52:06,895][105620] Updated weights for policy 1, policy_version 1440781 (0.0011) [2023-12-27 01:52:06,944][105620] Updated weights for policy 1, policy_version 1440791 (0.0011) [2023-12-27 01:52:07,000][105620] Updated weights for policy 1, policy_version 1440801 (0.0011) [2023-12-27 01:52:07,352][105692] Updated weights for policy 0, policy_version 1438456 (0.0009) [2023-12-27 01:52:07,399][105692] Updated weights for policy 0, policy_version 1438466 (0.0008) [2023-12-27 01:52:07,456][105692] Updated weights for policy 0, policy_version 1438476 (0.0009) [2023-12-27 01:52:07,687][105620] Updated weights for policy 1, policy_version 1440811 (0.0009) [2023-12-27 01:52:07,743][105620] Updated weights for policy 1, policy_version 1440821 (0.0005) [2023-12-27 01:52:07,805][105620] Updated weights for policy 1, policy_version 1440831 (0.0005) [2023-12-27 01:52:08,327][105620] Updated weights for policy 1, policy_version 1440841 (0.0006) [2023-12-27 01:52:08,380][105692] Updated weights for policy 0, policy_version 1438486 (0.0007) [2023-12-27 01:52:08,389][105620] Updated weights for policy 1, policy_version 1440851 (0.0011) [2023-12-27 01:52:08,434][105692] Updated weights for policy 0, policy_version 1438496 (0.0009) [2023-12-27 01:52:08,448][105620] Updated weights for policy 1, policy_version 1440861 (0.0007) [2023-12-27 01:52:08,485][105692] Updated weights for policy 0, policy_version 1438506 (0.0009) [2023-12-27 01:52:08,498][105620] Updated weights for policy 1, policy_version 1440871 (0.0005) [2023-12-27 01:52:09,200][105620] Updated weights for policy 1, policy_version 1440881 (0.0010) [2023-12-27 01:52:09,265][105620] Updated weights for policy 1, policy_version 1440891 (0.0011) [2023-12-27 01:52:09,276][105692] Updated weights for policy 0, policy_version 1438516 (0.0007) [2023-12-27 01:52:09,318][105620] Updated weights for policy 1, policy_version 1440901 (0.0010) [2023-12-27 01:52:09,334][105692] Updated weights for policy 0, policy_version 1438526 (0.0006) [2023-12-27 01:52:09,396][105692] Updated weights for policy 0, policy_version 1438536 (0.0009) [2023-12-27 01:52:10,139][105620] Updated weights for policy 1, policy_version 1440911 (0.0009) [2023-12-27 01:52:10,203][105620] Updated weights for policy 1, policy_version 1440921 (0.0006) [2023-12-27 01:52:10,210][105692] Updated weights for policy 0, policy_version 1438546 (0.0008) [2023-12-27 01:52:10,259][105620] Updated weights for policy 1, policy_version 1440931 (0.0007) [2023-12-27 01:52:10,278][105692] Updated weights for policy 0, policy_version 1438556 (0.0009) [2023-12-27 01:52:10,343][105692] Updated weights for policy 0, policy_version 1438566 (0.0009) [2023-12-27 01:52:10,403][105692] Updated weights for policy 0, policy_version 1438576 (0.0009) [2023-12-27 01:52:10,918][105620] Updated weights for policy 1, policy_version 1440941 (0.0009) [2023-12-27 01:52:10,974][105620] Updated weights for policy 1, policy_version 1440951 (0.0011) [2023-12-27 01:52:11,034][105620] Updated weights for policy 1, policy_version 1440961 (0.0010) [2023-12-27 01:52:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 737255424. Throughput: 0: 9558.6, 1: 9890.8. Samples: 737271320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:52:11,063][104569] Avg episode reward: [(0, '8529.041'), (1, '8986.689')] [2023-12-27 01:52:11,191][105692] Updated weights for policy 0, policy_version 1438586 (0.0007) [2023-12-27 01:52:11,261][105692] Updated weights for policy 0, policy_version 1438596 (0.0007) [2023-12-27 01:52:11,326][105692] Updated weights for policy 0, policy_version 1438606 (0.0008) [2023-12-27 01:52:11,851][105620] Updated weights for policy 1, policy_version 1440971 (0.0010) [2023-12-27 01:52:11,915][105620] Updated weights for policy 1, policy_version 1440981 (0.0010) [2023-12-27 01:52:11,980][105620] Updated weights for policy 1, policy_version 1440991 (0.0009) [2023-12-27 01:52:12,096][105692] Updated weights for policy 0, policy_version 1438616 (0.0009) [2023-12-27 01:52:12,151][105692] Updated weights for policy 0, policy_version 1438626 (0.0008) [2023-12-27 01:52:12,211][105692] Updated weights for policy 0, policy_version 1438636 (0.0008) [2023-12-27 01:52:12,759][105620] Updated weights for policy 1, policy_version 1441001 (0.0008) [2023-12-27 01:52:12,810][105620] Updated weights for policy 1, policy_version 1441011 (0.0009) [2023-12-27 01:52:12,867][105620] Updated weights for policy 1, policy_version 1441021 (0.0009) [2023-12-27 01:52:12,923][105620] Updated weights for policy 1, policy_version 1441031 (0.0010) [2023-12-27 01:52:12,978][105692] Updated weights for policy 0, policy_version 1438646 (0.0009) [2023-12-27 01:52:13,029][105692] Updated weights for policy 0, policy_version 1438656 (0.0008) [2023-12-27 01:52:13,085][105692] Updated weights for policy 0, policy_version 1438666 (0.0009) [2023-12-27 01:52:13,545][105620] Updated weights for policy 1, policy_version 1441041 (0.0007) [2023-12-27 01:52:13,595][105620] Updated weights for policy 1, policy_version 1441051 (0.0008) [2023-12-27 01:52:13,658][105620] Updated weights for policy 1, policy_version 1441061 (0.0008) [2023-12-27 01:52:13,943][105692] Updated weights for policy 0, policy_version 1438676 (0.0009) [2023-12-27 01:52:13,993][105692] Updated weights for policy 0, policy_version 1438686 (0.0008) [2023-12-27 01:52:14,050][105692] Updated weights for policy 0, policy_version 1438696 (0.0009) [2023-12-27 01:52:14,325][105620] Updated weights for policy 1, policy_version 1441071 (0.0009) [2023-12-27 01:52:14,379][105620] Updated weights for policy 1, policy_version 1441081 (0.0009) [2023-12-27 01:52:14,433][105620] Updated weights for policy 1, policy_version 1441091 (0.0009) [2023-12-27 01:52:14,783][105692] Updated weights for policy 0, policy_version 1438706 (0.0009) [2023-12-27 01:52:14,851][105692] Updated weights for policy 0, policy_version 1438716 (0.0008) [2023-12-27 01:52:14,916][105692] Updated weights for policy 0, policy_version 1438726 (0.0008) [2023-12-27 01:52:14,981][105692] Updated weights for policy 0, policy_version 1438736 (0.0008) [2023-12-27 01:52:15,180][105620] Updated weights for policy 1, policy_version 1441101 (0.0007) [2023-12-27 01:52:15,243][105620] Updated weights for policy 1, policy_version 1441111 (0.0007) [2023-12-27 01:52:15,306][105620] Updated weights for policy 1, policy_version 1441121 (0.0011) [2023-12-27 01:52:15,762][105692] Updated weights for policy 0, policy_version 1438746 (0.0007) [2023-12-27 01:52:15,815][105692] Updated weights for policy 0, policy_version 1438756 (0.0007) [2023-12-27 01:52:15,860][105692] Updated weights for policy 0, policy_version 1438766 (0.0007) [2023-12-27 01:52:16,006][105620] Updated weights for policy 1, policy_version 1441131 (0.0009) [2023-12-27 01:52:16,053][105620] Updated weights for policy 1, policy_version 1441141 (0.0005) [2023-12-27 01:52:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 737353728. Throughput: 0: 9474.5, 1: 9884.0. Samples: 737326848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:52:16,063][104569] Avg episode reward: [(0, '8528.693'), (1, '8809.163')] [2023-12-27 01:52:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001438768_368377856.pth... [2023-12-27 01:52:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001437680_368099328.pth [2023-12-27 01:52:16,120][105620] Updated weights for policy 1, policy_version 1441151 (0.0009) [2023-12-27 01:52:16,169][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001441160_368984064.pth... [2023-12-27 01:52:16,172][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001439976_368680960.pth [2023-12-27 01:52:16,564][105692] Updated weights for policy 0, policy_version 1438776 (0.0006) [2023-12-27 01:52:16,613][105692] Updated weights for policy 0, policy_version 1438786 (0.0005) [2023-12-27 01:52:16,666][105692] Updated weights for policy 0, policy_version 1438796 (0.0005) [2023-12-27 01:52:16,817][105620] Updated weights for policy 1, policy_version 1441161 (0.0010) [2023-12-27 01:52:16,885][105620] Updated weights for policy 1, policy_version 1441171 (0.0010) [2023-12-27 01:52:16,941][105620] Updated weights for policy 1, policy_version 1441181 (0.0010) [2023-12-27 01:52:16,999][105620] Updated weights for policy 1, policy_version 1441191 (0.0010) [2023-12-27 01:52:17,290][105692] Updated weights for policy 0, policy_version 1438806 (0.0007) [2023-12-27 01:52:17,341][105692] Updated weights for policy 0, policy_version 1438816 (0.0007) [2023-12-27 01:52:17,396][105692] Updated weights for policy 0, policy_version 1438826 (0.0008) [2023-12-27 01:52:17,727][105620] Updated weights for policy 1, policy_version 1441201 (0.0010) [2023-12-27 01:52:17,775][105620] Updated weights for policy 1, policy_version 1441211 (0.0010) [2023-12-27 01:52:17,827][105620] Updated weights for policy 1, policy_version 1441221 (0.0010) [2023-12-27 01:52:18,057][105692] Updated weights for policy 0, policy_version 1438836 (0.0009) [2023-12-27 01:52:18,110][105692] Updated weights for policy 0, policy_version 1438846 (0.0008) [2023-12-27 01:52:18,159][105692] Updated weights for policy 0, policy_version 1438856 (0.0008) [2023-12-27 01:52:18,552][105620] Updated weights for policy 1, policy_version 1441231 (0.0010) [2023-12-27 01:52:18,608][105620] Updated weights for policy 1, policy_version 1441241 (0.0011) [2023-12-27 01:52:18,663][105620] Updated weights for policy 1, policy_version 1441251 (0.0010) [2023-12-27 01:52:18,940][105692] Updated weights for policy 0, policy_version 1438866 (0.0008) [2023-12-27 01:52:19,004][105692] Updated weights for policy 0, policy_version 1438876 (0.0009) [2023-12-27 01:52:19,067][105692] Updated weights for policy 0, policy_version 1438886 (0.0008) [2023-12-27 01:52:19,130][105692] Updated weights for policy 0, policy_version 1438896 (0.0009) [2023-12-27 01:52:19,385][105620] Updated weights for policy 1, policy_version 1441261 (0.0009) [2023-12-27 01:52:19,443][105620] Updated weights for policy 1, policy_version 1441271 (0.0005) [2023-12-27 01:52:19,506][105620] Updated weights for policy 1, policy_version 1441281 (0.0006) [2023-12-27 01:52:19,944][105692] Updated weights for policy 0, policy_version 1438906 (0.0009) [2023-12-27 01:52:19,999][105692] Updated weights for policy 0, policy_version 1438916 (0.0009) [2023-12-27 01:52:20,062][105692] Updated weights for policy 0, policy_version 1438926 (0.0006) [2023-12-27 01:52:20,245][105620] Updated weights for policy 1, policy_version 1441291 (0.0007) [2023-12-27 01:52:20,308][105620] Updated weights for policy 1, policy_version 1441301 (0.0009) [2023-12-27 01:52:20,366][105586] KL-divergence is very high: 111.8044 [2023-12-27 01:52:20,370][105620] Updated weights for policy 1, policy_version 1441311 (0.0006) [2023-12-27 01:52:20,413][105586] KL-divergence is very high: 130.8134 [2023-12-27 01:52:20,796][105692] Updated weights for policy 0, policy_version 1438936 (0.0008) [2023-12-27 01:52:20,859][105692] Updated weights for policy 0, policy_version 1438946 (0.0009) [2023-12-27 01:52:20,914][105692] Updated weights for policy 0, policy_version 1438956 (0.0009) [2023-12-27 01:52:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 737452032. Throughput: 0: 9306.1, 1: 9921.9. Samples: 737443220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:52:21,063][104569] Avg episode reward: [(0, '8710.107'), (1, '8353.578')] [2023-12-27 01:52:21,136][105620] Updated weights for policy 1, policy_version 1441321 (0.0006) [2023-12-27 01:52:21,197][105620] Updated weights for policy 1, policy_version 1441331 (0.0007) [2023-12-27 01:52:21,270][105620] Updated weights for policy 1, policy_version 1441341 (0.0009) [2023-12-27 01:52:21,335][105620] Updated weights for policy 1, policy_version 1441351 (0.0009) [2023-12-27 01:52:21,733][105692] Updated weights for policy 0, policy_version 1438966 (0.0009) [2023-12-27 01:52:21,793][105692] Updated weights for policy 0, policy_version 1438976 (0.0009) [2023-12-27 01:52:21,845][105692] Updated weights for policy 0, policy_version 1438986 (0.0009) [2023-12-27 01:52:22,051][105620] Updated weights for policy 1, policy_version 1441361 (0.0009) [2023-12-27 01:52:22,113][105620] Updated weights for policy 1, policy_version 1441371 (0.0009) [2023-12-27 01:52:22,175][105620] Updated weights for policy 1, policy_version 1441381 (0.0009) [2023-12-27 01:52:22,649][105692] Updated weights for policy 0, policy_version 1438996 (0.0008) [2023-12-27 01:52:22,707][105692] Updated weights for policy 0, policy_version 1439006 (0.0010) [2023-12-27 01:52:22,756][105692] Updated weights for policy 0, policy_version 1439016 (0.0009) [2023-12-27 01:52:22,889][105620] Updated weights for policy 1, policy_version 1441391 (0.0009) [2023-12-27 01:52:22,945][105620] Updated weights for policy 1, policy_version 1441401 (0.0008) [2023-12-27 01:52:23,000][105620] Updated weights for policy 1, policy_version 1441411 (0.0008) [2023-12-27 01:52:23,506][105692] Updated weights for policy 0, policy_version 1439026 (0.0008) [2023-12-27 01:52:23,564][105692] Updated weights for policy 0, policy_version 1439036 (0.0005) [2023-12-27 01:52:23,610][105692] Updated weights for policy 0, policy_version 1439046 (0.0005) [2023-12-27 01:52:23,660][105692] Updated weights for policy 0, policy_version 1439056 (0.0005) [2023-12-27 01:52:23,823][105620] Updated weights for policy 1, policy_version 1441421 (0.0009) [2023-12-27 01:52:23,875][105620] Updated weights for policy 1, policy_version 1441431 (0.0009) [2023-12-27 01:52:23,940][105620] Updated weights for policy 1, policy_version 1441441 (0.0010) [2023-12-27 01:52:24,285][105692] Updated weights for policy 0, policy_version 1439066 (0.0008) [2023-12-27 01:52:24,340][105692] Updated weights for policy 0, policy_version 1439076 (0.0008) [2023-12-27 01:52:24,395][105692] Updated weights for policy 0, policy_version 1439086 (0.0008) [2023-12-27 01:52:24,697][105620] Updated weights for policy 1, policy_version 1441451 (0.0010) [2023-12-27 01:52:24,758][105620] Updated weights for policy 1, policy_version 1441461 (0.0010) [2023-12-27 01:52:24,819][105620] Updated weights for policy 1, policy_version 1441471 (0.0010) [2023-12-27 01:52:25,161][105692] Updated weights for policy 0, policy_version 1439096 (0.0008) [2023-12-27 01:52:25,223][105692] Updated weights for policy 0, policy_version 1439106 (0.0008) [2023-12-27 01:52:25,278][105692] Updated weights for policy 0, policy_version 1439116 (0.0008) [2023-12-27 01:52:25,544][105620] Updated weights for policy 1, policy_version 1441481 (0.0010) [2023-12-27 01:52:25,610][105620] Updated weights for policy 1, policy_version 1441491 (0.0009) [2023-12-27 01:52:25,668][105620] Updated weights for policy 1, policy_version 1441501 (0.0009) [2023-12-27 01:52:25,724][105620] Updated weights for policy 1, policy_version 1441511 (0.0008) [2023-12-27 01:52:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 737542144. Throughput: 0: 9216.8, 1: 9892.8. Samples: 737554800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:52:26,062][104569] Avg episode reward: [(0, '8800.145'), (1, '8165.836')] [2023-12-27 01:52:26,066][105692] Updated weights for policy 0, policy_version 1439126 (0.0009) [2023-12-27 01:52:26,119][105692] Updated weights for policy 0, policy_version 1439136 (0.0010) [2023-12-27 01:52:26,176][105692] Updated weights for policy 0, policy_version 1439147 (0.0009) [2023-12-27 01:52:26,333][105620] Updated weights for policy 1, policy_version 1441521 (0.0007) [2023-12-27 01:52:26,395][105620] Updated weights for policy 1, policy_version 1441531 (0.0006) [2023-12-27 01:52:26,443][105620] Updated weights for policy 1, policy_version 1441541 (0.0005) [2023-12-27 01:52:27,040][105620] Updated weights for policy 1, policy_version 1441551 (0.0009) [2023-12-27 01:52:27,042][105692] Updated weights for policy 0, policy_version 1439157 (0.0007) [2023-12-27 01:52:27,098][105620] Updated weights for policy 1, policy_version 1441561 (0.0010) [2023-12-27 01:52:27,100][105692] Updated weights for policy 0, policy_version 1439167 (0.0005) [2023-12-27 01:52:27,149][105620] Updated weights for policy 1, policy_version 1441571 (0.0010) [2023-12-27 01:52:27,162][105692] Updated weights for policy 0, policy_version 1439177 (0.0006) [2023-12-27 01:52:27,863][105620] Updated weights for policy 1, policy_version 1441581 (0.0010) [2023-12-27 01:52:27,912][105692] Updated weights for policy 0, policy_version 1439187 (0.0008) [2023-12-27 01:52:27,921][105620] Updated weights for policy 1, policy_version 1441591 (0.0008) [2023-12-27 01:52:27,964][105692] Updated weights for policy 0, policy_version 1439197 (0.0008) [2023-12-27 01:52:27,978][105620] Updated weights for policy 1, policy_version 1441601 (0.0007) [2023-12-27 01:52:28,028][105692] Updated weights for policy 0, policy_version 1439207 (0.0007) [2023-12-27 01:52:28,683][105620] Updated weights for policy 1, policy_version 1441611 (0.0007) [2023-12-27 01:52:28,748][105620] Updated weights for policy 1, policy_version 1441621 (0.0008) [2023-12-27 01:52:28,812][105620] Updated weights for policy 1, policy_version 1441631 (0.0007) [2023-12-27 01:52:28,814][105692] Updated weights for policy 0, policy_version 1439217 (0.0009) [2023-12-27 01:52:28,872][105692] Updated weights for policy 0, policy_version 1439227 (0.0008) [2023-12-27 01:52:28,922][105692] Updated weights for policy 0, policy_version 1439237 (0.0008) [2023-12-27 01:52:28,972][105692] Updated weights for policy 0, policy_version 1439247 (0.0009) [2023-12-27 01:52:29,518][105620] Updated weights for policy 1, policy_version 1441641 (0.0006) [2023-12-27 01:52:29,579][105620] Updated weights for policy 1, policy_version 1441651 (0.0008) [2023-12-27 01:52:29,640][105620] Updated weights for policy 1, policy_version 1441661 (0.0009) [2023-12-27 01:52:29,694][105620] Updated weights for policy 1, policy_version 1441671 (0.0009) [2023-12-27 01:52:29,765][105692] Updated weights for policy 0, policy_version 1439257 (0.0009) [2023-12-27 01:52:29,830][105692] Updated weights for policy 0, policy_version 1439267 (0.0008) [2023-12-27 01:52:29,892][105692] Updated weights for policy 0, policy_version 1439277 (0.0008) [2023-12-27 01:52:30,445][105620] Updated weights for policy 1, policy_version 1441681 (0.0009) [2023-12-27 01:52:30,495][105620] Updated weights for policy 1, policy_version 1441691 (0.0009) [2023-12-27 01:52:30,553][105620] Updated weights for policy 1, policy_version 1441701 (0.0009) [2023-12-27 01:52:30,691][105692] Updated weights for policy 0, policy_version 1439287 (0.0009) [2023-12-27 01:52:30,744][105692] Updated weights for policy 0, policy_version 1439297 (0.0010) [2023-12-27 01:52:30,810][105692] Updated weights for policy 0, policy_version 1439307 (0.0009) [2023-12-27 01:52:31,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 737640448. Throughput: 0: 9190.9, 1: 9928.1. Samples: 737612492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:52:31,062][104569] Avg episode reward: [(0, '9075.315'), (1, '8712.820')] [2023-12-27 01:52:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001439312_368517120.pth... [2023-12-27 01:52:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001441704_369123328.pth... [2023-12-27 01:52:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001438256_368246784.pth [2023-12-27 01:52:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001440552_368828416.pth [2023-12-27 01:52:31,158][105620] Updated weights for policy 1, policy_version 1441711 (0.0008) [2023-12-27 01:52:31,225][105620] Updated weights for policy 1, policy_version 1441721 (0.0007) [2023-12-27 01:52:31,291][105620] Updated weights for policy 1, policy_version 1441731 (0.0008) [2023-12-27 01:52:31,657][105692] Updated weights for policy 0, policy_version 1439317 (0.0008) [2023-12-27 01:52:31,722][105692] Updated weights for policy 0, policy_version 1439327 (0.0009) [2023-12-27 01:52:31,790][105692] Updated weights for policy 0, policy_version 1439337 (0.0008) [2023-12-27 01:52:31,960][105620] Updated weights for policy 1, policy_version 1441741 (0.0009) [2023-12-27 01:52:32,006][105620] Updated weights for policy 1, policy_version 1441751 (0.0007) [2023-12-27 01:52:32,064][105620] Updated weights for policy 1, policy_version 1441761 (0.0009) [2023-12-27 01:52:32,581][105692] Updated weights for policy 0, policy_version 1439347 (0.0009) [2023-12-27 01:52:32,629][105692] Updated weights for policy 0, policy_version 1439357 (0.0009) [2023-12-27 01:52:32,675][105692] Updated weights for policy 0, policy_version 1439367 (0.0008) [2023-12-27 01:52:32,683][105620] Updated weights for policy 1, policy_version 1441771 (0.0008) [2023-12-27 01:52:32,731][105620] Updated weights for policy 1, policy_version 1441781 (0.0005) [2023-12-27 01:52:32,779][105620] Updated weights for policy 1, policy_version 1441791 (0.0005) [2023-12-27 01:52:33,393][105620] Updated weights for policy 1, policy_version 1441801 (0.0005) [2023-12-27 01:52:33,447][105620] Updated weights for policy 1, policy_version 1441811 (0.0005) [2023-12-27 01:52:33,503][105620] Updated weights for policy 1, policy_version 1441821 (0.0008) [2023-12-27 01:52:33,527][105692] Updated weights for policy 0, policy_version 1439377 (0.0008) [2023-12-27 01:52:33,559][105620] Updated weights for policy 1, policy_version 1441831 (0.0006) [2023-12-27 01:52:33,584][105692] Updated weights for policy 0, policy_version 1439387 (0.0009) [2023-12-27 01:52:33,643][105692] Updated weights for policy 0, policy_version 1439397 (0.0009) [2023-12-27 01:52:33,696][105692] Updated weights for policy 0, policy_version 1439407 (0.0009) [2023-12-27 01:52:34,169][105620] Updated weights for policy 1, policy_version 1441841 (0.0007) [2023-12-27 01:52:34,230][105620] Updated weights for policy 1, policy_version 1441851 (0.0007) [2023-12-27 01:52:34,292][105620] Updated weights for policy 1, policy_version 1441861 (0.0006) [2023-12-27 01:52:34,557][105692] Updated weights for policy 0, policy_version 1439417 (0.0009) [2023-12-27 01:52:34,609][105692] Updated weights for policy 0, policy_version 1439427 (0.0008) [2023-12-27 01:52:34,674][105692] Updated weights for policy 0, policy_version 1439437 (0.0005) [2023-12-27 01:52:34,969][105620] Updated weights for policy 1, policy_version 1441871 (0.0009) [2023-12-27 01:52:35,032][105620] Updated weights for policy 1, policy_version 1441881 (0.0008) [2023-12-27 01:52:35,075][105620] Updated weights for policy 1, policy_version 1441891 (0.0010) [2023-12-27 01:52:35,412][105692] Updated weights for policy 0, policy_version 1439447 (0.0008) [2023-12-27 01:52:35,467][105692] Updated weights for policy 0, policy_version 1439457 (0.0008) [2023-12-27 01:52:35,511][105692] Updated weights for policy 0, policy_version 1439467 (0.0008) [2023-12-27 01:52:35,805][105620] Updated weights for policy 1, policy_version 1441901 (0.0008) [2023-12-27 01:52:35,877][105620] Updated weights for policy 1, policy_version 1441911 (0.0010) [2023-12-27 01:52:35,922][105620] Updated weights for policy 1, policy_version 1441921 (0.0010) [2023-12-27 01:52:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19114.7, 300 sec: 19577.5). Total num frames: 737738752. Throughput: 0: 9102.1, 1: 10004.4. Samples: 737727960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 01:52:36,062][104569] Avg episode reward: [(0, '8987.932'), (1, '8991.255')] [2023-12-27 01:52:36,280][105692] Updated weights for policy 0, policy_version 1439477 (0.0008) [2023-12-27 01:52:36,327][105692] Updated weights for policy 0, policy_version 1439487 (0.0009) [2023-12-27 01:52:36,376][105692] Updated weights for policy 0, policy_version 1439497 (0.0009) [2023-12-27 01:52:36,650][105620] Updated weights for policy 1, policy_version 1441931 (0.0010) [2023-12-27 01:52:36,708][105620] Updated weights for policy 1, policy_version 1441941 (0.0010) [2023-12-27 01:52:36,774][105620] Updated weights for policy 1, policy_version 1441951 (0.0010) [2023-12-27 01:52:37,080][105692] Updated weights for policy 0, policy_version 1439507 (0.0009) [2023-12-27 01:52:37,142][105692] Updated weights for policy 0, policy_version 1439517 (0.0009) [2023-12-27 01:52:37,194][105692] Updated weights for policy 0, policy_version 1439527 (0.0009) [2023-12-27 01:52:37,553][105620] Updated weights for policy 1, policy_version 1441961 (0.0009) [2023-12-27 01:52:37,619][105620] Updated weights for policy 1, policy_version 1441971 (0.0009) [2023-12-27 01:52:37,685][105620] Updated weights for policy 1, policy_version 1441981 (0.0009) [2023-12-27 01:52:37,745][105620] Updated weights for policy 1, policy_version 1441991 (0.0009) [2023-12-27 01:52:37,928][105692] Updated weights for policy 0, policy_version 1439537 (0.0009) [2023-12-27 01:52:37,979][105692] Updated weights for policy 0, policy_version 1439547 (0.0009) [2023-12-27 01:52:38,037][105692] Updated weights for policy 0, policy_version 1439557 (0.0009) [2023-12-27 01:52:38,091][105692] Updated weights for policy 0, policy_version 1439567 (0.0009) [2023-12-27 01:52:38,486][105620] Updated weights for policy 1, policy_version 1442001 (0.0009) [2023-12-27 01:52:38,548][105620] Updated weights for policy 1, policy_version 1442011 (0.0009) [2023-12-27 01:52:38,607][105620] Updated weights for policy 1, policy_version 1442021 (0.0009) [2023-12-27 01:52:38,814][105692] Updated weights for policy 0, policy_version 1439577 (0.0009) [2023-12-27 01:52:38,861][105692] Updated weights for policy 0, policy_version 1439587 (0.0009) [2023-12-27 01:52:38,913][105692] Updated weights for policy 0, policy_version 1439597 (0.0009) [2023-12-27 01:52:39,379][105620] Updated weights for policy 1, policy_version 1442031 (0.0010) [2023-12-27 01:52:39,453][105620] Updated weights for policy 1, policy_version 1442041 (0.0008) [2023-12-27 01:52:39,525][105620] Updated weights for policy 1, policy_version 1442051 (0.0006) [2023-12-27 01:52:39,745][105692] Updated weights for policy 0, policy_version 1439607 (0.0008) [2023-12-27 01:52:39,797][105692] Updated weights for policy 0, policy_version 1439617 (0.0008) [2023-12-27 01:52:39,856][105692] Updated weights for policy 0, policy_version 1439627 (0.0008) [2023-12-27 01:52:40,158][105620] Updated weights for policy 1, policy_version 1442061 (0.0007) [2023-12-27 01:52:40,224][105620] Updated weights for policy 1, policy_version 1442071 (0.0008) [2023-12-27 01:52:40,288][105620] Updated weights for policy 1, policy_version 1442081 (0.0008) [2023-12-27 01:52:40,614][105692] Updated weights for policy 0, policy_version 1439637 (0.0008) [2023-12-27 01:52:40,669][105692] Updated weights for policy 0, policy_version 1439647 (0.0006) [2023-12-27 01:52:40,717][105692] Updated weights for policy 0, policy_version 1439657 (0.0009) [2023-12-27 01:52:41,042][105620] Updated weights for policy 1, policy_version 1442091 (0.0008) [2023-12-27 01:52:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 18978.2, 300 sec: 19549.7). Total num frames: 737828864. Throughput: 0: 9167.1, 1: 9881.5. Samples: 737840912. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:52:41,062][104569] Avg episode reward: [(0, '8717.702'), (1, '8166.903')] [2023-12-27 01:52:41,105][105620] Updated weights for policy 1, policy_version 1442102 (0.0009) [2023-12-27 01:52:41,165][105620] Updated weights for policy 1, policy_version 1442112 (0.0008) [2023-12-27 01:52:41,401][105692] Updated weights for policy 0, policy_version 1439667 (0.0010) [2023-12-27 01:52:41,467][105692] Updated weights for policy 0, policy_version 1439677 (0.0010) [2023-12-27 01:52:41,526][105692] Updated weights for policy 0, policy_version 1439687 (0.0009) [2023-12-27 01:52:41,927][105620] Updated weights for policy 1, policy_version 1442122 (0.0009) [2023-12-27 01:52:41,989][105620] Updated weights for policy 1, policy_version 1442132 (0.0008) [2023-12-27 01:52:42,050][105620] Updated weights for policy 1, policy_version 1442142 (0.0008) [2023-12-27 01:52:42,107][105620] Updated weights for policy 1, policy_version 1442152 (0.0008) [2023-12-27 01:52:42,238][105692] Updated weights for policy 0, policy_version 1439697 (0.0006) [2023-12-27 01:52:42,301][105692] Updated weights for policy 0, policy_version 1439707 (0.0009) [2023-12-27 01:52:42,366][105692] Updated weights for policy 0, policy_version 1439717 (0.0009) [2023-12-27 01:52:42,418][105692] Updated weights for policy 0, policy_version 1439727 (0.0010) [2023-12-27 01:52:42,823][105620] Updated weights for policy 1, policy_version 1442162 (0.0007) [2023-12-27 01:52:42,884][105620] Updated weights for policy 1, policy_version 1442172 (0.0006) [2023-12-27 01:52:42,947][105620] Updated weights for policy 1, policy_version 1442182 (0.0009) [2023-12-27 01:52:43,092][105692] Updated weights for policy 0, policy_version 1439737 (0.0010) [2023-12-27 01:52:43,148][105692] Updated weights for policy 0, policy_version 1439747 (0.0011) [2023-12-27 01:52:43,194][105692] Updated weights for policy 0, policy_version 1439757 (0.0007) [2023-12-27 01:52:43,684][105620] Updated weights for policy 1, policy_version 1442192 (0.0009) [2023-12-27 01:52:43,745][105620] Updated weights for policy 1, policy_version 1442202 (0.0010) [2023-12-27 01:52:43,808][105620] Updated weights for policy 1, policy_version 1442212 (0.0010) [2023-12-27 01:52:43,872][105692] Updated weights for policy 0, policy_version 1439767 (0.0005) [2023-12-27 01:52:43,930][105692] Updated weights for policy 0, policy_version 1439777 (0.0005) [2023-12-27 01:52:43,985][105692] Updated weights for policy 0, policy_version 1439787 (0.0005) [2023-12-27 01:52:44,606][105620] Updated weights for policy 1, policy_version 1442222 (0.0008) [2023-12-27 01:52:44,669][105620] Updated weights for policy 1, policy_version 1442232 (0.0008) [2023-12-27 01:52:44,674][105692] Updated weights for policy 0, policy_version 1439797 (0.0008) [2023-12-27 01:52:44,730][105620] Updated weights for policy 1, policy_version 1442242 (0.0007) [2023-12-27 01:52:44,732][105692] Updated weights for policy 0, policy_version 1439807 (0.0006) [2023-12-27 01:52:44,790][105692] Updated weights for policy 0, policy_version 1439817 (0.0007) [2023-12-27 01:52:45,497][105692] Updated weights for policy 0, policy_version 1439827 (0.0007) [2023-12-27 01:52:45,511][105620] Updated weights for policy 1, policy_version 1442252 (0.0007) [2023-12-27 01:52:45,550][105692] Updated weights for policy 0, policy_version 1439837 (0.0011) [2023-12-27 01:52:45,565][105620] Updated weights for policy 1, policy_version 1442262 (0.0005) [2023-12-27 01:52:45,599][105692] Updated weights for policy 0, policy_version 1439847 (0.0010) [2023-12-27 01:52:45,625][105620] Updated weights for policy 1, policy_version 1442272 (0.0006) [2023-12-27 01:52:46,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19114.6, 300 sec: 19549.7). Total num frames: 737927168. Throughput: 0: 9157.8, 1: 9869.9. Samples: 737897716. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:52:46,063][104569] Avg episode reward: [(0, '8712.325'), (1, '8166.879')] [2023-12-27 01:52:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001442280_369270784.pth... [2023-12-27 01:52:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001439856_368656384.pth... [2023-12-27 01:52:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001441160_368984064.pth [2023-12-27 01:52:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001438768_368377856.pth [2023-12-27 01:52:46,187][105692] Updated weights for policy 0, policy_version 1439857 (0.0008) [2023-12-27 01:52:46,247][105692] Updated weights for policy 0, policy_version 1439867 (0.0005) [2023-12-27 01:52:46,296][105692] Updated weights for policy 0, policy_version 1439877 (0.0005) [2023-12-27 01:52:46,345][105692] Updated weights for policy 0, policy_version 1439887 (0.0005) [2023-12-27 01:52:46,488][105620] Updated weights for policy 1, policy_version 1442282 (0.0009) [2023-12-27 01:52:46,552][105620] Updated weights for policy 1, policy_version 1442292 (0.0009) [2023-12-27 01:52:46,616][105620] Updated weights for policy 1, policy_version 1442302 (0.0007) [2023-12-27 01:52:46,676][105620] Updated weights for policy 1, policy_version 1442312 (0.0008) [2023-12-27 01:52:46,919][105692] Updated weights for policy 0, policy_version 1439897 (0.0009) [2023-12-27 01:52:46,970][105692] Updated weights for policy 0, policy_version 1439907 (0.0010) [2023-12-27 01:52:47,024][105692] Updated weights for policy 0, policy_version 1439917 (0.0009) [2023-12-27 01:52:47,392][105620] Updated weights for policy 1, policy_version 1442322 (0.0010) [2023-12-27 01:52:47,440][105620] Updated weights for policy 1, policy_version 1442332 (0.0010) [2023-12-27 01:52:47,490][105620] Updated weights for policy 1, policy_version 1442342 (0.0008) [2023-12-27 01:52:47,648][105692] Updated weights for policy 0, policy_version 1439927 (0.0006) [2023-12-27 01:52:47,707][105692] Updated weights for policy 0, policy_version 1439937 (0.0006) [2023-12-27 01:52:47,758][105692] Updated weights for policy 0, policy_version 1439947 (0.0010) [2023-12-27 01:52:48,275][105620] Updated weights for policy 1, policy_version 1442352 (0.0007) [2023-12-27 01:52:48,333][105620] Updated weights for policy 1, policy_version 1442362 (0.0008) [2023-12-27 01:52:48,400][105620] Updated weights for policy 1, policy_version 1442372 (0.0009) [2023-12-27 01:52:48,417][105692] Updated weights for policy 0, policy_version 1439957 (0.0008) [2023-12-27 01:52:48,480][105692] Updated weights for policy 0, policy_version 1439967 (0.0010) [2023-12-27 01:52:48,541][105692] Updated weights for policy 0, policy_version 1439977 (0.0010) [2023-12-27 01:52:49,174][105620] Updated weights for policy 1, policy_version 1442382 (0.0008) [2023-12-27 01:52:49,239][105620] Updated weights for policy 1, policy_version 1442392 (0.0008) [2023-12-27 01:52:49,271][105692] Updated weights for policy 0, policy_version 1439987 (0.0010) [2023-12-27 01:52:49,309][105620] Updated weights for policy 1, policy_version 1442402 (0.0008) [2023-12-27 01:52:49,336][105692] Updated weights for policy 0, policy_version 1439997 (0.0010) [2023-12-27 01:52:49,395][105692] Updated weights for policy 0, policy_version 1440007 (0.0010) [2023-12-27 01:52:49,959][105620] Updated weights for policy 1, policy_version 1442412 (0.0008) [2023-12-27 01:52:50,029][105620] Updated weights for policy 1, policy_version 1442422 (0.0008) [2023-12-27 01:52:50,096][105620] Updated weights for policy 1, policy_version 1442432 (0.0007) [2023-12-27 01:52:50,144][105692] Updated weights for policy 0, policy_version 1440017 (0.0010) [2023-12-27 01:52:50,206][105692] Updated weights for policy 0, policy_version 1440027 (0.0009) [2023-12-27 01:52:50,268][105692] Updated weights for policy 0, policy_version 1440037 (0.0009) [2023-12-27 01:52:50,332][105692] Updated weights for policy 0, policy_version 1440047 (0.0009) [2023-12-27 01:52:50,852][105620] Updated weights for policy 1, policy_version 1442442 (0.0008) [2023-12-27 01:52:50,914][105620] Updated weights for policy 1, policy_version 1442452 (0.0008) [2023-12-27 01:52:50,975][105620] Updated weights for policy 1, policy_version 1442462 (0.0009) [2023-12-27 01:52:51,038][105620] Updated weights for policy 1, policy_version 1442472 (0.0008) [2023-12-27 01:52:51,040][105692] Updated weights for policy 0, policy_version 1440057 (0.0006) [2023-12-27 01:52:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19114.7, 300 sec: 19522.0). Total num frames: 738025472. Throughput: 0: 9341.3, 1: 9706.2. Samples: 738015876. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:52:51,062][104569] Avg episode reward: [(0, '8895.521'), (1, '8719.105')] [2023-12-27 01:52:51,090][105692] Updated weights for policy 0, policy_version 1440067 (0.0008) [2023-12-27 01:52:51,151][105692] Updated weights for policy 0, policy_version 1440077 (0.0010) [2023-12-27 01:52:51,785][105620] Updated weights for policy 1, policy_version 1442482 (0.0009) [2023-12-27 01:52:51,843][105620] Updated weights for policy 1, policy_version 1442492 (0.0009) [2023-12-27 01:52:51,893][105620] Updated weights for policy 1, policy_version 1442502 (0.0008) [2023-12-27 01:52:51,928][105692] Updated weights for policy 0, policy_version 1440087 (0.0009) [2023-12-27 01:52:51,989][105692] Updated weights for policy 0, policy_version 1440097 (0.0008) [2023-12-27 01:52:52,048][105692] Updated weights for policy 0, policy_version 1440107 (0.0009) [2023-12-27 01:52:52,664][105620] Updated weights for policy 1, policy_version 1442512 (0.0009) [2023-12-27 01:52:52,730][105620] Updated weights for policy 1, policy_version 1442522 (0.0009) [2023-12-27 01:52:52,794][105620] Updated weights for policy 1, policy_version 1442532 (0.0009) [2023-12-27 01:52:52,817][105692] Updated weights for policy 0, policy_version 1440117 (0.0008) [2023-12-27 01:52:52,871][105692] Updated weights for policy 0, policy_version 1440128 (0.0010) [2023-12-27 01:52:52,930][105692] Updated weights for policy 0, policy_version 1440138 (0.0010) [2023-12-27 01:52:53,383][105620] Updated weights for policy 1, policy_version 1442542 (0.0008) [2023-12-27 01:52:53,430][105620] Updated weights for policy 1, policy_version 1442552 (0.0009) [2023-12-27 01:52:53,480][105620] Updated weights for policy 1, policy_version 1442563 (0.0009) [2023-12-27 01:52:53,685][105692] Updated weights for policy 0, policy_version 1440148 (0.0008) [2023-12-27 01:52:53,736][105692] Updated weights for policy 0, policy_version 1440158 (0.0009) [2023-12-27 01:52:53,783][105692] Updated weights for policy 0, policy_version 1440168 (0.0009) [2023-12-27 01:52:54,197][105620] Updated weights for policy 1, policy_version 1442573 (0.0008) [2023-12-27 01:52:54,260][105620] Updated weights for policy 1, policy_version 1442583 (0.0006) [2023-12-27 01:52:54,316][105620] Updated weights for policy 1, policy_version 1442593 (0.0010) [2023-12-27 01:52:54,606][105692] Updated weights for policy 0, policy_version 1440178 (0.0009) [2023-12-27 01:52:54,649][105692] Updated weights for policy 0, policy_version 1440188 (0.0006) [2023-12-27 01:52:54,699][105692] Updated weights for policy 0, policy_version 1440198 (0.0007) [2023-12-27 01:52:54,747][105692] Updated weights for policy 0, policy_version 1440208 (0.0008) [2023-12-27 01:52:55,022][105620] Updated weights for policy 1, policy_version 1442603 (0.0010) [2023-12-27 01:52:55,084][105620] Updated weights for policy 1, policy_version 1442613 (0.0010) [2023-12-27 01:52:55,143][105620] Updated weights for policy 1, policy_version 1442623 (0.0011) [2023-12-27 01:52:55,538][105692] Updated weights for policy 0, policy_version 1440218 (0.0008) [2023-12-27 01:52:55,593][105692] Updated weights for policy 0, policy_version 1440228 (0.0007) [2023-12-27 01:52:55,640][105692] Updated weights for policy 0, policy_version 1440238 (0.0008) [2023-12-27 01:52:55,867][105620] Updated weights for policy 1, policy_version 1442633 (0.0007) [2023-12-27 01:52:55,915][105620] Updated weights for policy 1, policy_version 1442643 (0.0010) [2023-12-27 01:52:55,966][105620] Updated weights for policy 1, policy_version 1442653 (0.0010) [2023-12-27 01:52:56,018][105620] Updated weights for policy 1, policy_version 1442663 (0.0010) [2023-12-27 01:52:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19114.6, 300 sec: 19522.0). Total num frames: 738123776. Throughput: 0: 9404.2, 1: 9655.4. Samples: 738129000. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:52:56,063][104569] Avg episode reward: [(0, '8620.184'), (1, '8627.065')] [2023-12-27 01:52:56,402][105692] Updated weights for policy 0, policy_version 1440248 (0.0008) [2023-12-27 01:52:56,458][105692] Updated weights for policy 0, policy_version 1440258 (0.0008) [2023-12-27 01:52:56,510][105692] Updated weights for policy 0, policy_version 1440268 (0.0009) [2023-12-27 01:52:56,772][105620] Updated weights for policy 1, policy_version 1442673 (0.0010) [2023-12-27 01:52:56,823][105620] Updated weights for policy 1, policy_version 1442683 (0.0010) [2023-12-27 01:52:56,868][105620] Updated weights for policy 1, policy_version 1442693 (0.0010) [2023-12-27 01:52:57,271][105692] Updated weights for policy 0, policy_version 1440278 (0.0008) [2023-12-27 01:52:57,336][105692] Updated weights for policy 0, policy_version 1440288 (0.0008) [2023-12-27 01:52:57,384][105692] Updated weights for policy 0, policy_version 1440298 (0.0008) [2023-12-27 01:52:57,629][105620] Updated weights for policy 1, policy_version 1442703 (0.0010) [2023-12-27 01:52:57,687][105620] Updated weights for policy 1, policy_version 1442713 (0.0010) [2023-12-27 01:52:57,748][105620] Updated weights for policy 1, policy_version 1442723 (0.0010) [2023-12-27 01:52:58,138][105692] Updated weights for policy 0, policy_version 1440308 (0.0008) [2023-12-27 01:52:58,206][105692] Updated weights for policy 0, policy_version 1440318 (0.0008) [2023-12-27 01:52:58,261][105692] Updated weights for policy 0, policy_version 1440328 (0.0008) [2023-12-27 01:52:58,506][105620] Updated weights for policy 1, policy_version 1442733 (0.0009) [2023-12-27 01:52:58,572][105620] Updated weights for policy 1, policy_version 1442743 (0.0007) [2023-12-27 01:52:58,639][105620] Updated weights for policy 1, policy_version 1442753 (0.0007) [2023-12-27 01:52:59,077][105692] Updated weights for policy 0, policy_version 1440338 (0.0008) [2023-12-27 01:52:59,134][105692] Updated weights for policy 0, policy_version 1440348 (0.0005) [2023-12-27 01:52:59,191][105692] Updated weights for policy 0, policy_version 1440358 (0.0008) [2023-12-27 01:52:59,267][105692] Updated weights for policy 0, policy_version 1440368 (0.0008) [2023-12-27 01:52:59,410][105620] Updated weights for policy 1, policy_version 1442763 (0.0009) [2023-12-27 01:52:59,461][105620] Updated weights for policy 1, policy_version 1442773 (0.0010) [2023-12-27 01:52:59,512][105620] Updated weights for policy 1, policy_version 1442783 (0.0010) [2023-12-27 01:53:00,044][105692] Updated weights for policy 0, policy_version 1440378 (0.0011) [2023-12-27 01:53:00,113][105692] Updated weights for policy 0, policy_version 1440388 (0.0011) [2023-12-27 01:53:00,179][105692] Updated weights for policy 0, policy_version 1440398 (0.0010) [2023-12-27 01:53:00,256][105620] Updated weights for policy 1, policy_version 1442793 (0.0010) [2023-12-27 01:53:00,318][105620] Updated weights for policy 1, policy_version 1442803 (0.0007) [2023-12-27 01:53:00,376][105620] Updated weights for policy 1, policy_version 1442813 (0.0010) [2023-12-27 01:53:00,434][105620] Updated weights for policy 1, policy_version 1442823 (0.0010) [2023-12-27 01:53:00,852][105692] Updated weights for policy 0, policy_version 1440408 (0.0010) [2023-12-27 01:53:00,914][105692] Updated weights for policy 0, policy_version 1440418 (0.0007) [2023-12-27 01:53:00,964][105692] Updated weights for policy 0, policy_version 1440428 (0.0008) [2023-12-27 01:53:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.7, 300 sec: 19494.2). Total num frames: 738213888. Throughput: 0: 9417.2, 1: 9631.6. Samples: 738184040. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:53:01,062][104569] Avg episode reward: [(0, '8432.999'), (1, '8898.802')] [2023-12-27 01:53:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001440432_368803840.pth... [2023-12-27 01:53:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001439312_368517120.pth [2023-12-27 01:53:01,083][105620] Updated weights for policy 1, policy_version 1442833 (0.0010) [2023-12-27 01:53:01,148][105620] Updated weights for policy 1, policy_version 1442843 (0.0013) [2023-12-27 01:53:01,209][105620] Updated weights for policy 1, policy_version 1442853 (0.0009) [2023-12-27 01:53:01,227][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001442856_369418240.pth... [2023-12-27 01:53:01,231][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001441704_369123328.pth [2023-12-27 01:53:01,689][105692] Updated weights for policy 0, policy_version 1440438 (0.0007) [2023-12-27 01:53:01,749][105692] Updated weights for policy 0, policy_version 1440448 (0.0008) [2023-12-27 01:53:01,809][105692] Updated weights for policy 0, policy_version 1440458 (0.0011) [2023-12-27 01:53:01,924][105620] Updated weights for policy 1, policy_version 1442863 (0.0008) [2023-12-27 01:53:01,998][105620] Updated weights for policy 1, policy_version 1442873 (0.0009) [2023-12-27 01:53:02,058][105620] Updated weights for policy 1, policy_version 1442883 (0.0011) [2023-12-27 01:53:02,539][105692] Updated weights for policy 0, policy_version 1440468 (0.0008) [2023-12-27 01:53:02,589][105692] Updated weights for policy 0, policy_version 1440478 (0.0005) [2023-12-27 01:53:02,646][105692] Updated weights for policy 0, policy_version 1440488 (0.0005) [2023-12-27 01:53:02,793][105620] Updated weights for policy 1, policy_version 1442893 (0.0010) [2023-12-27 01:53:02,858][105620] Updated weights for policy 1, policy_version 1442903 (0.0010) [2023-12-27 01:53:02,920][105620] Updated weights for policy 1, policy_version 1442913 (0.0010) [2023-12-27 01:53:03,350][105692] Updated weights for policy 0, policy_version 1440498 (0.0007) [2023-12-27 01:53:03,411][105692] Updated weights for policy 0, policy_version 1440508 (0.0010) [2023-12-27 01:53:03,478][105692] Updated weights for policy 0, policy_version 1440518 (0.0010) [2023-12-27 01:53:03,530][105620] Updated weights for policy 1, policy_version 1442923 (0.0007) [2023-12-27 01:53:03,538][105692] Updated weights for policy 0, policy_version 1440528 (0.0010) [2023-12-27 01:53:03,573][105620] Updated weights for policy 1, policy_version 1442933 (0.0007) [2023-12-27 01:53:03,619][105620] Updated weights for policy 1, policy_version 1442943 (0.0005) [2023-12-27 01:53:04,299][105620] Updated weights for policy 1, policy_version 1442953 (0.0006) [2023-12-27 01:53:04,311][105692] Updated weights for policy 0, policy_version 1440538 (0.0009) [2023-12-27 01:53:04,358][105620] Updated weights for policy 1, policy_version 1442963 (0.0007) [2023-12-27 01:53:04,369][105692] Updated weights for policy 0, policy_version 1440548 (0.0008) [2023-12-27 01:53:04,415][105620] Updated weights for policy 1, policy_version 1442973 (0.0009) [2023-12-27 01:53:04,440][105692] Updated weights for policy 0, policy_version 1440558 (0.0007) [2023-12-27 01:53:04,472][105620] Updated weights for policy 1, policy_version 1442983 (0.0008) [2023-12-27 01:53:05,199][105620] Updated weights for policy 1, policy_version 1442993 (0.0006) [2023-12-27 01:53:05,212][105692] Updated weights for policy 0, policy_version 1440568 (0.0011) [2023-12-27 01:53:05,254][105620] Updated weights for policy 1, policy_version 1443003 (0.0006) [2023-12-27 01:53:05,267][105692] Updated weights for policy 0, policy_version 1440578 (0.0011) [2023-12-27 01:53:05,305][105620] Updated weights for policy 1, policy_version 1443013 (0.0007) [2023-12-27 01:53:05,322][105692] Updated weights for policy 0, policy_version 1440588 (0.0011) [2023-12-27 01:53:06,062][104569] Fps is (10 sec: 18022.6, 60 sec: 18978.1, 300 sec: 19466.4). Total num frames: 738304000. Throughput: 0: 9390.6, 1: 9655.1. Samples: 738300276. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:53:06,062][104569] Avg episode reward: [(0, '8797.702'), (1, '9265.653')] [2023-12-27 01:53:06,078][105692] Updated weights for policy 0, policy_version 1440598 (0.0011) [2023-12-27 01:53:06,084][105620] Updated weights for policy 1, policy_version 1443023 (0.0007) [2023-12-27 01:53:06,135][105692] Updated weights for policy 0, policy_version 1440608 (0.0010) [2023-12-27 01:53:06,144][105620] Updated weights for policy 1, policy_version 1443033 (0.0007) [2023-12-27 01:53:06,197][105620] Updated weights for policy 1, policy_version 1443043 (0.0005) [2023-12-27 01:53:06,198][105692] Updated weights for policy 0, policy_version 1440618 (0.0011) [2023-12-27 01:53:06,873][105692] Updated weights for policy 0, policy_version 1440628 (0.0009) [2023-12-27 01:53:06,929][105692] Updated weights for policy 0, policy_version 1440638 (0.0007) [2023-12-27 01:53:06,943][105620] Updated weights for policy 1, policy_version 1443053 (0.0005) [2023-12-27 01:53:06,989][105692] Updated weights for policy 0, policy_version 1440648 (0.0009) [2023-12-27 01:53:06,995][105620] Updated weights for policy 1, policy_version 1443063 (0.0008) [2023-12-27 01:53:07,057][105620] Updated weights for policy 1, policy_version 1443073 (0.0011) [2023-12-27 01:53:07,633][105620] Updated weights for policy 1, policy_version 1443083 (0.0010) [2023-12-27 01:53:07,691][105620] Updated weights for policy 1, policy_version 1443093 (0.0009) [2023-12-27 01:53:07,713][105692] Updated weights for policy 0, policy_version 1440658 (0.0010) [2023-12-27 01:53:07,748][105620] Updated weights for policy 1, policy_version 1443103 (0.0009) [2023-12-27 01:53:07,775][105692] Updated weights for policy 0, policy_version 1440668 (0.0010) [2023-12-27 01:53:07,832][105692] Updated weights for policy 0, policy_version 1440678 (0.0008) [2023-12-27 01:53:07,886][105692] Updated weights for policy 0, policy_version 1440688 (0.0009) [2023-12-27 01:53:08,393][105620] Updated weights for policy 1, policy_version 1443113 (0.0010) [2023-12-27 01:53:08,449][105620] Updated weights for policy 1, policy_version 1443123 (0.0010) [2023-12-27 01:53:08,507][105620] Updated weights for policy 1, policy_version 1443133 (0.0010) [2023-12-27 01:53:08,568][105620] Updated weights for policy 1, policy_version 1443143 (0.0010) [2023-12-27 01:53:08,680][105692] Updated weights for policy 0, policy_version 1440698 (0.0011) [2023-12-27 01:53:08,739][105692] Updated weights for policy 0, policy_version 1440708 (0.0011) [2023-12-27 01:53:08,805][105692] Updated weights for policy 0, policy_version 1440718 (0.0011) [2023-12-27 01:53:09,244][105620] Updated weights for policy 1, policy_version 1443153 (0.0008) [2023-12-27 01:53:09,310][105620] Updated weights for policy 1, policy_version 1443163 (0.0007) [2023-12-27 01:53:09,378][105620] Updated weights for policy 1, policy_version 1443173 (0.0008) [2023-12-27 01:53:09,543][105692] Updated weights for policy 0, policy_version 1440728 (0.0009) [2023-12-27 01:53:09,605][105692] Updated weights for policy 0, policy_version 1440738 (0.0008) [2023-12-27 01:53:09,664][105692] Updated weights for policy 0, policy_version 1440748 (0.0008) [2023-12-27 01:53:10,095][105620] Updated weights for policy 1, policy_version 1443183 (0.0008) [2023-12-27 01:53:10,153][105620] Updated weights for policy 1, policy_version 1443193 (0.0007) [2023-12-27 01:53:10,213][105620] Updated weights for policy 1, policy_version 1443203 (0.0008) [2023-12-27 01:53:10,422][105692] Updated weights for policy 0, policy_version 1440758 (0.0010) [2023-12-27 01:53:10,475][105692] Updated weights for policy 0, policy_version 1440768 (0.0010) [2023-12-27 01:53:10,534][105692] Updated weights for policy 0, policy_version 1440778 (0.0011) [2023-12-27 01:53:11,018][105620] Updated weights for policy 1, policy_version 1443213 (0.0007) [2023-12-27 01:53:11,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 738402304. Throughput: 0: 9404.9, 1: 9717.8. Samples: 738415320. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:53:11,063][104569] Avg episode reward: [(0, '8983.606'), (1, '8901.265')] [2023-12-27 01:53:11,086][105620] Updated weights for policy 1, policy_version 1443223 (0.0006) [2023-12-27 01:53:11,159][105620] Updated weights for policy 1, policy_version 1443233 (0.0007) [2023-12-27 01:53:11,214][105692] Updated weights for policy 0, policy_version 1440788 (0.0009) [2023-12-27 01:53:11,282][105692] Updated weights for policy 0, policy_version 1440798 (0.0008) [2023-12-27 01:53:11,349][105692] Updated weights for policy 0, policy_version 1440808 (0.0008) [2023-12-27 01:53:11,879][105620] Updated weights for policy 1, policy_version 1443243 (0.0009) [2023-12-27 01:53:11,935][105620] Updated weights for policy 1, policy_version 1443253 (0.0008) [2023-12-27 01:53:11,991][105620] Updated weights for policy 1, policy_version 1443263 (0.0008) [2023-12-27 01:53:12,104][105692] Updated weights for policy 0, policy_version 1440818 (0.0008) [2023-12-27 01:53:12,160][105692] Updated weights for policy 0, policy_version 1440828 (0.0010) [2023-12-27 01:53:12,209][105692] Updated weights for policy 0, policy_version 1440838 (0.0010) [2023-12-27 01:53:12,266][105692] Updated weights for policy 0, policy_version 1440848 (0.0011) [2023-12-27 01:53:12,702][105620] Updated weights for policy 1, policy_version 1443273 (0.0008) [2023-12-27 01:53:12,746][105620] Updated weights for policy 1, policy_version 1443283 (0.0008) [2023-12-27 01:53:12,796][105620] Updated weights for policy 1, policy_version 1443293 (0.0008) [2023-12-27 01:53:12,849][105620] Updated weights for policy 1, policy_version 1443304 (0.0010) [2023-12-27 01:53:13,017][105692] Updated weights for policy 0, policy_version 1440858 (0.0006) [2023-12-27 01:53:13,064][105692] Updated weights for policy 0, policy_version 1440868 (0.0009) [2023-12-27 01:53:13,112][105692] Updated weights for policy 0, policy_version 1440878 (0.0009) [2023-12-27 01:53:13,668][105620] Updated weights for policy 1, policy_version 1443314 (0.0009) [2023-12-27 01:53:13,720][105620] Updated weights for policy 1, policy_version 1443324 (0.0009) [2023-12-27 01:53:13,777][105620] Updated weights for policy 1, policy_version 1443334 (0.0009) [2023-12-27 01:53:13,778][105692] Updated weights for policy 0, policy_version 1440888 (0.0006) [2023-12-27 01:53:13,824][105692] Updated weights for policy 0, policy_version 1440898 (0.0005) [2023-12-27 01:53:13,880][105692] Updated weights for policy 0, policy_version 1440908 (0.0005) [2023-12-27 01:53:14,494][105692] Updated weights for policy 0, policy_version 1440918 (0.0008) [2023-12-27 01:53:14,547][105692] Updated weights for policy 0, policy_version 1440928 (0.0008) [2023-12-27 01:53:14,602][105692] Updated weights for policy 0, policy_version 1440938 (0.0009) [2023-12-27 01:53:14,606][105620] Updated weights for policy 1, policy_version 1443344 (0.0006) [2023-12-27 01:53:14,664][105620] Updated weights for policy 1, policy_version 1443354 (0.0007) [2023-12-27 01:53:14,724][105620] Updated weights for policy 1, policy_version 1443364 (0.0008) [2023-12-27 01:53:15,346][105692] Updated weights for policy 0, policy_version 1440948 (0.0009) [2023-12-27 01:53:15,412][105692] Updated weights for policy 0, policy_version 1440958 (0.0008) [2023-12-27 01:53:15,476][105692] Updated weights for policy 0, policy_version 1440968 (0.0006) [2023-12-27 01:53:15,528][105620] Updated weights for policy 1, policy_version 1443374 (0.0009) [2023-12-27 01:53:15,591][105620] Updated weights for policy 1, policy_version 1443384 (0.0008) [2023-12-27 01:53:15,655][105620] Updated weights for policy 1, policy_version 1443394 (0.0008) [2023-12-27 01:53:16,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19114.6, 300 sec: 19494.2). Total num frames: 738500608. Throughput: 0: 9463.2, 1: 9636.2. Samples: 738471972. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:53:16,063][104569] Avg episode reward: [(0, '8710.852'), (1, '8626.497')] [2023-12-27 01:53:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001443400_369557504.pth... [2023-12-27 01:53:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001440976_368943104.pth... [2023-12-27 01:53:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001442280_369270784.pth [2023-12-27 01:53:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001439856_368656384.pth [2023-12-27 01:53:16,147][105692] Updated weights for policy 0, policy_version 1440978 (0.0008) [2023-12-27 01:53:16,201][105692] Updated weights for policy 0, policy_version 1440988 (0.0009) [2023-12-27 01:53:16,249][105692] Updated weights for policy 0, policy_version 1440998 (0.0009) [2023-12-27 01:53:16,307][105692] Updated weights for policy 0, policy_version 1441008 (0.0009) [2023-12-27 01:53:16,402][105620] Updated weights for policy 1, policy_version 1443404 (0.0009) [2023-12-27 01:53:16,455][105620] Updated weights for policy 1, policy_version 1443414 (0.0009) [2023-12-27 01:53:16,510][105620] Updated weights for policy 1, policy_version 1443424 (0.0009) [2023-12-27 01:53:17,079][105692] Updated weights for policy 0, policy_version 1441018 (0.0005) [2023-12-27 01:53:17,139][105692] Updated weights for policy 0, policy_version 1441028 (0.0005) [2023-12-27 01:53:17,206][105692] Updated weights for policy 0, policy_version 1441038 (0.0005) [2023-12-27 01:53:17,323][105620] Updated weights for policy 1, policy_version 1443434 (0.0009) [2023-12-27 01:53:17,378][105620] Updated weights for policy 1, policy_version 1443444 (0.0009) [2023-12-27 01:53:17,446][105620] Updated weights for policy 1, policy_version 1443454 (0.0009) [2023-12-27 01:53:17,514][105620] Updated weights for policy 1, policy_version 1443464 (0.0009) [2023-12-27 01:53:17,815][105692] Updated weights for policy 0, policy_version 1441048 (0.0007) [2023-12-27 01:53:17,878][105692] Updated weights for policy 0, policy_version 1441058 (0.0006) [2023-12-27 01:53:17,940][105692] Updated weights for policy 0, policy_version 1441068 (0.0006) [2023-12-27 01:53:18,220][105620] Updated weights for policy 1, policy_version 1443474 (0.0008) [2023-12-27 01:53:18,280][105620] Updated weights for policy 1, policy_version 1443484 (0.0008) [2023-12-27 01:53:18,326][105620] Updated weights for policy 1, policy_version 1443494 (0.0009) [2023-12-27 01:53:18,621][105692] Updated weights for policy 0, policy_version 1441078 (0.0010) [2023-12-27 01:53:18,673][105692] Updated weights for policy 0, policy_version 1441088 (0.0005) [2023-12-27 01:53:18,724][105692] Updated weights for policy 0, policy_version 1441098 (0.0009) [2023-12-27 01:53:19,110][105620] Updated weights for policy 1, policy_version 1443504 (0.0009) [2023-12-27 01:53:19,166][105620] Updated weights for policy 1, policy_version 1443515 (0.0009) [2023-12-27 01:53:19,222][105620] Updated weights for policy 1, policy_version 1443525 (0.0010) [2023-12-27 01:53:19,392][105692] Updated weights for policy 0, policy_version 1441108 (0.0008) [2023-12-27 01:53:19,457][105692] Updated weights for policy 0, policy_version 1441118 (0.0007) [2023-12-27 01:53:19,521][105692] Updated weights for policy 0, policy_version 1441128 (0.0008) [2023-12-27 01:53:20,066][105620] Updated weights for policy 1, policy_version 1443535 (0.0009) [2023-12-27 01:53:20,128][105620] Updated weights for policy 1, policy_version 1443545 (0.0009) [2023-12-27 01:53:20,194][105620] Updated weights for policy 1, policy_version 1443555 (0.0009) [2023-12-27 01:53:20,201][105692] Updated weights for policy 0, policy_version 1441138 (0.0008) [2023-12-27 01:53:20,257][105692] Updated weights for policy 0, policy_version 1441148 (0.0007) [2023-12-27 01:53:20,328][105692] Updated weights for policy 0, policy_version 1441158 (0.0010) [2023-12-27 01:53:20,390][105692] Updated weights for policy 0, policy_version 1441168 (0.0009) [2023-12-27 01:53:20,922][105620] Updated weights for policy 1, policy_version 1443565 (0.0007) [2023-12-27 01:53:20,985][105620] Updated weights for policy 1, policy_version 1443575 (0.0007) [2023-12-27 01:53:21,049][105620] Updated weights for policy 1, policy_version 1443585 (0.0009) [2023-12-27 01:53:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 18978.2, 300 sec: 19466.4). Total num frames: 738590720. Throughput: 0: 9662.0, 1: 9434.7. Samples: 738587312. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:53:21,062][104569] Avg episode reward: [(0, '8248.841'), (1, '8717.883')] [2023-12-27 01:53:21,198][105692] Updated weights for policy 0, policy_version 1441178 (0.0007) [2023-12-27 01:53:21,259][105692] Updated weights for policy 0, policy_version 1441188 (0.0009) [2023-12-27 01:53:21,324][105692] Updated weights for policy 0, policy_version 1441198 (0.0009) [2023-12-27 01:53:21,779][105620] Updated weights for policy 1, policy_version 1443595 (0.0010) [2023-12-27 01:53:21,847][105620] Updated weights for policy 1, policy_version 1443605 (0.0011) [2023-12-27 01:53:21,910][105620] Updated weights for policy 1, policy_version 1443615 (0.0011) [2023-12-27 01:53:22,082][105692] Updated weights for policy 0, policy_version 1441208 (0.0006) [2023-12-27 01:53:22,132][105692] Updated weights for policy 0, policy_version 1441218 (0.0005) [2023-12-27 01:53:22,192][105692] Updated weights for policy 0, policy_version 1441228 (0.0006) [2023-12-27 01:53:22,599][105620] Updated weights for policy 1, policy_version 1443625 (0.0010) [2023-12-27 01:53:22,658][105620] Updated weights for policy 1, policy_version 1443635 (0.0007) [2023-12-27 01:53:22,728][105620] Updated weights for policy 1, policy_version 1443645 (0.0011) [2023-12-27 01:53:22,784][105620] Updated weights for policy 1, policy_version 1443655 (0.0010) [2023-12-27 01:53:22,894][105692] Updated weights for policy 0, policy_version 1441238 (0.0009) [2023-12-27 01:53:22,956][105692] Updated weights for policy 0, policy_version 1441248 (0.0010) [2023-12-27 01:53:23,017][105692] Updated weights for policy 0, policy_version 1441258 (0.0010) [2023-12-27 01:53:23,503][105620] Updated weights for policy 1, policy_version 1443665 (0.0006) [2023-12-27 01:53:23,560][105620] Updated weights for policy 1, policy_version 1443675 (0.0007) [2023-12-27 01:53:23,619][105620] Updated weights for policy 1, policy_version 1443685 (0.0011) [2023-12-27 01:53:23,722][105692] Updated weights for policy 0, policy_version 1441268 (0.0010) [2023-12-27 01:53:23,787][105692] Updated weights for policy 0, policy_version 1441278 (0.0010) [2023-12-27 01:53:23,851][105692] Updated weights for policy 0, policy_version 1441288 (0.0010) [2023-12-27 01:53:24,254][105620] Updated weights for policy 1, policy_version 1443695 (0.0008) [2023-12-27 01:53:24,309][105620] Updated weights for policy 1, policy_version 1443705 (0.0011) [2023-12-27 01:53:24,371][105620] Updated weights for policy 1, policy_version 1443715 (0.0010) [2023-12-27 01:53:24,486][105692] Updated weights for policy 0, policy_version 1441298 (0.0006) [2023-12-27 01:53:24,547][105692] Updated weights for policy 0, policy_version 1441308 (0.0010) [2023-12-27 01:53:24,608][105692] Updated weights for policy 0, policy_version 1441318 (0.0010) [2023-12-27 01:53:24,676][105692] Updated weights for policy 0, policy_version 1441328 (0.0006) [2023-12-27 01:53:25,080][105620] Updated weights for policy 1, policy_version 1443725 (0.0008) [2023-12-27 01:53:25,146][105620] Updated weights for policy 1, policy_version 1443735 (0.0005) [2023-12-27 01:53:25,204][105620] Updated weights for policy 1, policy_version 1443745 (0.0006) [2023-12-27 01:53:25,242][105692] Updated weights for policy 0, policy_version 1441338 (0.0010) [2023-12-27 01:53:25,306][105692] Updated weights for policy 0, policy_version 1441348 (0.0010) [2023-12-27 01:53:25,366][105692] Updated weights for policy 0, policy_version 1441358 (0.0010) [2023-12-27 01:53:25,768][105620] Updated weights for policy 1, policy_version 1443755 (0.0007) [2023-12-27 01:53:25,812][105620] Updated weights for policy 1, policy_version 1443765 (0.0010) [2023-12-27 01:53:25,864][105620] Updated weights for policy 1, policy_version 1443775 (0.0010) [2023-12-27 01:53:26,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 738697216. Throughput: 0: 9721.5, 1: 9507.2. Samples: 738706204. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:53:26,062][104569] Avg episode reward: [(0, '8613.858'), (1, '8626.279')] [2023-12-27 01:53:26,087][105692] Updated weights for policy 0, policy_version 1441368 (0.0010) [2023-12-27 01:53:26,142][105692] Updated weights for policy 0, policy_version 1441378 (0.0007) [2023-12-27 01:53:26,205][105692] Updated weights for policy 0, policy_version 1441388 (0.0008) [2023-12-27 01:53:26,552][105620] Updated weights for policy 1, policy_version 1443785 (0.0010) [2023-12-27 01:53:26,603][105620] Updated weights for policy 1, policy_version 1443795 (0.0010) [2023-12-27 01:53:26,649][105620] Updated weights for policy 1, policy_version 1443805 (0.0010) [2023-12-27 01:53:26,700][105620] Updated weights for policy 1, policy_version 1443815 (0.0010) [2023-12-27 01:53:26,890][105692] Updated weights for policy 0, policy_version 1441398 (0.0009) [2023-12-27 01:53:26,948][105692] Updated weights for policy 0, policy_version 1441408 (0.0010) [2023-12-27 01:53:27,014][105692] Updated weights for policy 0, policy_version 1441418 (0.0011) [2023-12-27 01:53:27,326][105620] Updated weights for policy 1, policy_version 1443825 (0.0010) [2023-12-27 01:53:27,394][105620] Updated weights for policy 1, policy_version 1443835 (0.0006) [2023-12-27 01:53:27,465][105620] Updated weights for policy 1, policy_version 1443845 (0.0006) [2023-12-27 01:53:27,782][105692] Updated weights for policy 0, policy_version 1441428 (0.0011) [2023-12-27 01:53:27,826][105692] Updated weights for policy 0, policy_version 1441438 (0.0010) [2023-12-27 01:53:27,874][105692] Updated weights for policy 0, policy_version 1441448 (0.0010) [2023-12-27 01:53:27,954][105620] Updated weights for policy 1, policy_version 1443855 (0.0005) [2023-12-27 01:53:28,004][105620] Updated weights for policy 1, policy_version 1443865 (0.0006) [2023-12-27 01:53:28,067][105620] Updated weights for policy 1, policy_version 1443875 (0.0005) [2023-12-27 01:53:28,641][105692] Updated weights for policy 0, policy_version 1441458 (0.0010) [2023-12-27 01:53:28,656][105620] Updated weights for policy 1, policy_version 1443885 (0.0008) [2023-12-27 01:53:28,694][105692] Updated weights for policy 0, policy_version 1441468 (0.0010) [2023-12-27 01:53:28,712][105620] Updated weights for policy 1, policy_version 1443895 (0.0011) [2023-12-27 01:53:28,742][105692] Updated weights for policy 0, policy_version 1441478 (0.0010) [2023-12-27 01:53:28,771][105620] Updated weights for policy 1, policy_version 1443905 (0.0009) [2023-12-27 01:53:28,801][105692] Updated weights for policy 0, policy_version 1441488 (0.0010) [2023-12-27 01:53:29,348][105620] Updated weights for policy 1, policy_version 1443915 (0.0006) [2023-12-27 01:53:29,412][105620] Updated weights for policy 1, policy_version 1443925 (0.0008) [2023-12-27 01:53:29,469][105620] Updated weights for policy 1, policy_version 1443935 (0.0006) [2023-12-27 01:53:29,562][105692] Updated weights for policy 0, policy_version 1441498 (0.0010) [2023-12-27 01:53:29,621][105692] Updated weights for policy 0, policy_version 1441508 (0.0008) [2023-12-27 01:53:29,672][105692] Updated weights for policy 0, policy_version 1441518 (0.0008) [2023-12-27 01:53:30,186][105620] Updated weights for policy 1, policy_version 1443945 (0.0006) [2023-12-27 01:53:30,237][105620] Updated weights for policy 1, policy_version 1443955 (0.0008) [2023-12-27 01:53:30,285][105620] Updated weights for policy 1, policy_version 1443965 (0.0008) [2023-12-27 01:53:30,344][105620] Updated weights for policy 1, policy_version 1443975 (0.0008) [2023-12-27 01:53:30,442][105692] Updated weights for policy 0, policy_version 1441528 (0.0010) [2023-12-27 01:53:30,487][105692] Updated weights for policy 0, policy_version 1441538 (0.0010) [2023-12-27 01:53:30,534][105692] Updated weights for policy 0, policy_version 1441548 (0.0010) [2023-12-27 01:53:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 738795520. Throughput: 0: 9702.8, 1: 9661.7. Samples: 738769116. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:53:31,063][104569] Avg episode reward: [(0, '8889.964'), (1, '8900.546')] [2023-12-27 01:53:31,067][105620] Updated weights for policy 1, policy_version 1443985 (0.0008) [2023-12-27 01:53:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001441552_369090560.pth... [2023-12-27 01:53:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001440432_368803840.pth [2023-12-27 01:53:31,121][105620] Updated weights for policy 1, policy_version 1443995 (0.0008) [2023-12-27 01:53:31,185][105620] Updated weights for policy 1, policy_version 1444005 (0.0009) [2023-12-27 01:53:31,202][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001444008_369713152.pth... [2023-12-27 01:53:31,207][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001442856_369418240.pth [2023-12-27 01:53:31,290][105692] Updated weights for policy 0, policy_version 1441558 (0.0008) [2023-12-27 01:53:31,351][105692] Updated weights for policy 0, policy_version 1441568 (0.0007) [2023-12-27 01:53:31,413][105692] Updated weights for policy 0, policy_version 1441578 (0.0008) [2023-12-27 01:53:32,014][105620] Updated weights for policy 1, policy_version 1444015 (0.0007) [2023-12-27 01:53:32,067][105620] Updated weights for policy 1, policy_version 1444025 (0.0009) [2023-12-27 01:53:32,129][105620] Updated weights for policy 1, policy_version 1444035 (0.0009) [2023-12-27 01:53:32,152][105692] Updated weights for policy 0, policy_version 1441588 (0.0008) [2023-12-27 01:53:32,207][105692] Updated weights for policy 0, policy_version 1441598 (0.0009) [2023-12-27 01:53:32,258][105692] Updated weights for policy 0, policy_version 1441608 (0.0009) [2023-12-27 01:53:32,857][105620] Updated weights for policy 1, policy_version 1444045 (0.0007) [2023-12-27 01:53:32,917][105620] Updated weights for policy 1, policy_version 1444055 (0.0008) [2023-12-27 01:53:32,980][105620] Updated weights for policy 1, policy_version 1444065 (0.0009) [2023-12-27 01:53:33,029][105692] Updated weights for policy 0, policy_version 1441618 (0.0006) [2023-12-27 01:53:33,087][105692] Updated weights for policy 0, policy_version 1441628 (0.0007) [2023-12-27 01:53:33,134][105692] Updated weights for policy 0, policy_version 1441638 (0.0009) [2023-12-27 01:53:33,187][105692] Updated weights for policy 0, policy_version 1441648 (0.0008) [2023-12-27 01:53:33,667][105620] Updated weights for policy 1, policy_version 1444075 (0.0008) [2023-12-27 01:53:33,710][105620] Updated weights for policy 1, policy_version 1444085 (0.0009) [2023-12-27 01:53:33,767][105620] Updated weights for policy 1, policy_version 1444095 (0.0009) [2023-12-27 01:53:33,866][105692] Updated weights for policy 0, policy_version 1441658 (0.0005) [2023-12-27 01:53:33,925][105692] Updated weights for policy 0, policy_version 1441668 (0.0005) [2023-12-27 01:53:33,977][105692] Updated weights for policy 0, policy_version 1441678 (0.0005) [2023-12-27 01:53:34,553][105620] Updated weights for policy 1, policy_version 1444105 (0.0010) [2023-12-27 01:53:34,553][105692] Updated weights for policy 0, policy_version 1441688 (0.0008) [2023-12-27 01:53:34,616][105620] Updated weights for policy 1, policy_version 1444115 (0.0008) [2023-12-27 01:53:34,621][105692] Updated weights for policy 0, policy_version 1441698 (0.0007) [2023-12-27 01:53:34,680][105692] Updated weights for policy 0, policy_version 1441708 (0.0008) [2023-12-27 01:53:34,681][105620] Updated weights for policy 1, policy_version 1444125 (0.0006) [2023-12-27 01:53:34,743][105620] Updated weights for policy 1, policy_version 1444135 (0.0006) [2023-12-27 01:53:35,337][105692] Updated weights for policy 0, policy_version 1441718 (0.0008) [2023-12-27 01:53:35,399][105692] Updated weights for policy 0, policy_version 1441728 (0.0008) [2023-12-27 01:53:35,444][105620] Updated weights for policy 1, policy_version 1444145 (0.0008) [2023-12-27 01:53:35,458][105692] Updated weights for policy 0, policy_version 1441738 (0.0005) [2023-12-27 01:53:35,512][105620] Updated weights for policy 1, policy_version 1444155 (0.0008) [2023-12-27 01:53:35,571][105620] Updated weights for policy 1, policy_version 1444165 (0.0008) [2023-12-27 01:53:36,052][105692] Updated weights for policy 0, policy_version 1441748 (0.0009) [2023-12-27 01:53:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.1, 300 sec: 19521.9). Total num frames: 738893824. Throughput: 0: 9596.8, 1: 9740.6. Samples: 738886060. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:53:36,063][104569] Avg episode reward: [(0, '8982.867'), (1, '9174.224')] [2023-12-27 01:53:36,115][105692] Updated weights for policy 0, policy_version 1441758 (0.0009) [2023-12-27 01:53:36,180][105692] Updated weights for policy 0, policy_version 1441768 (0.0010) [2023-12-27 01:53:36,371][105620] Updated weights for policy 1, policy_version 1444175 (0.0008) [2023-12-27 01:53:36,431][105620] Updated weights for policy 1, policy_version 1444185 (0.0008) [2023-12-27 01:53:36,494][105620] Updated weights for policy 1, policy_version 1444195 (0.0008) [2023-12-27 01:53:36,863][105692] Updated weights for policy 0, policy_version 1441778 (0.0009) [2023-12-27 01:53:36,918][105692] Updated weights for policy 0, policy_version 1441788 (0.0010) [2023-12-27 01:53:36,976][105692] Updated weights for policy 0, policy_version 1441798 (0.0010) [2023-12-27 01:53:37,041][105692] Updated weights for policy 0, policy_version 1441808 (0.0010) [2023-12-27 01:53:37,247][105620] Updated weights for policy 1, policy_version 1444205 (0.0008) [2023-12-27 01:53:37,304][105620] Updated weights for policy 1, policy_version 1444215 (0.0008) [2023-12-27 01:53:37,369][105620] Updated weights for policy 1, policy_version 1444225 (0.0009) [2023-12-27 01:53:37,696][105692] Updated weights for policy 0, policy_version 1441818 (0.0011) [2023-12-27 01:53:37,764][105692] Updated weights for policy 0, policy_version 1441828 (0.0011) [2023-12-27 01:53:37,825][105692] Updated weights for policy 0, policy_version 1441838 (0.0006) [2023-12-27 01:53:38,160][105620] Updated weights for policy 1, policy_version 1444235 (0.0009) [2023-12-27 01:53:38,209][105620] Updated weights for policy 1, policy_version 1444245 (0.0008) [2023-12-27 01:53:38,256][105620] Updated weights for policy 1, policy_version 1444255 (0.0008) [2023-12-27 01:53:38,522][105692] Updated weights for policy 0, policy_version 1441848 (0.0009) [2023-12-27 01:53:38,573][105692] Updated weights for policy 0, policy_version 1441858 (0.0010) [2023-12-27 01:53:38,619][105692] Updated weights for policy 0, policy_version 1441868 (0.0005) [2023-12-27 01:53:38,919][105620] Updated weights for policy 1, policy_version 1444265 (0.0007) [2023-12-27 01:53:38,987][105620] Updated weights for policy 1, policy_version 1444275 (0.0005) [2023-12-27 01:53:39,051][105620] Updated weights for policy 1, policy_version 1444285 (0.0005) [2023-12-27 01:53:39,113][105620] Updated weights for policy 1, policy_version 1444295 (0.0005) [2023-12-27 01:53:39,342][105692] Updated weights for policy 0, policy_version 1441878 (0.0009) [2023-12-27 01:53:39,414][105692] Updated weights for policy 0, policy_version 1441888 (0.0008) [2023-12-27 01:53:39,478][105692] Updated weights for policy 0, policy_version 1441898 (0.0007) [2023-12-27 01:53:39,791][105620] Updated weights for policy 1, policy_version 1444305 (0.0008) [2023-12-27 01:53:39,860][105620] Updated weights for policy 1, policy_version 1444315 (0.0009) [2023-12-27 01:53:39,917][105620] Updated weights for policy 1, policy_version 1444325 (0.0005) [2023-12-27 01:53:40,203][105692] Updated weights for policy 0, policy_version 1441908 (0.0009) [2023-12-27 01:53:40,262][105692] Updated weights for policy 0, policy_version 1441918 (0.0009) [2023-12-27 01:53:40,325][105692] Updated weights for policy 0, policy_version 1441928 (0.0011) [2023-12-27 01:53:40,676][105620] Updated weights for policy 1, policy_version 1444335 (0.0008) [2023-12-27 01:53:40,739][105620] Updated weights for policy 1, policy_version 1444345 (0.0008) [2023-12-27 01:53:40,788][105620] Updated weights for policy 1, policy_version 1444355 (0.0008) [2023-12-27 01:53:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 738992128. Throughput: 0: 9708.9, 1: 9697.0. Samples: 739002264. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:53:41,063][104569] Avg episode reward: [(0, '8893.678'), (1, '9174.189')] [2023-12-27 01:53:41,067][105692] Updated weights for policy 0, policy_version 1441938 (0.0011) [2023-12-27 01:53:41,135][105692] Updated weights for policy 0, policy_version 1441948 (0.0011) [2023-12-27 01:53:41,198][105692] Updated weights for policy 0, policy_version 1441958 (0.0011) [2023-12-27 01:53:41,265][105692] Updated weights for policy 0, policy_version 1441968 (0.0011) [2023-12-27 01:53:41,571][105620] Updated weights for policy 1, policy_version 1444365 (0.0008) [2023-12-27 01:53:41,639][105620] Updated weights for policy 1, policy_version 1444375 (0.0008) [2023-12-27 01:53:41,699][105620] Updated weights for policy 1, policy_version 1444385 (0.0009) [2023-12-27 01:53:42,033][105692] Updated weights for policy 0, policy_version 1441978 (0.0011) [2023-12-27 01:53:42,093][105692] Updated weights for policy 0, policy_version 1441988 (0.0010) [2023-12-27 01:53:42,149][105692] Updated weights for policy 0, policy_version 1441998 (0.0010) [2023-12-27 01:53:42,488][105620] Updated weights for policy 1, policy_version 1444395 (0.0008) [2023-12-27 01:53:42,544][105620] Updated weights for policy 1, policy_version 1444405 (0.0008) [2023-12-27 01:53:42,597][105620] Updated weights for policy 1, policy_version 1444415 (0.0008) [2023-12-27 01:53:42,909][105692] Updated weights for policy 0, policy_version 1442008 (0.0011) [2023-12-27 01:53:42,973][105692] Updated weights for policy 0, policy_version 1442018 (0.0011) [2023-12-27 01:53:43,033][105692] Updated weights for policy 0, policy_version 1442028 (0.0010) [2023-12-27 01:53:43,333][105620] Updated weights for policy 1, policy_version 1444425 (0.0008) [2023-12-27 01:53:43,390][105620] Updated weights for policy 1, policy_version 1444435 (0.0005) [2023-12-27 01:53:43,447][105620] Updated weights for policy 1, policy_version 1444445 (0.0006) [2023-12-27 01:53:43,509][105620] Updated weights for policy 1, policy_version 1444455 (0.0010) [2023-12-27 01:53:43,770][105692] Updated weights for policy 0, policy_version 1442038 (0.0010) [2023-12-27 01:53:43,827][105692] Updated weights for policy 0, policy_version 1442048 (0.0010) [2023-12-27 01:53:43,884][105692] Updated weights for policy 0, policy_version 1442058 (0.0010) [2023-12-27 01:53:44,074][105620] Updated weights for policy 1, policy_version 1444465 (0.0006) [2023-12-27 01:53:44,137][105620] Updated weights for policy 1, policy_version 1444475 (0.0008) [2023-12-27 01:53:44,199][105620] Updated weights for policy 1, policy_version 1444485 (0.0010) [2023-12-27 01:53:44,474][105692] Updated weights for policy 0, policy_version 1442068 (0.0010) [2023-12-27 01:53:44,521][105692] Updated weights for policy 0, policy_version 1442078 (0.0010) [2023-12-27 01:53:44,568][105692] Updated weights for policy 0, policy_version 1442088 (0.0010) [2023-12-27 01:53:44,938][105620] Updated weights for policy 1, policy_version 1444495 (0.0010) [2023-12-27 01:53:45,004][105620] Updated weights for policy 1, policy_version 1444505 (0.0007) [2023-12-27 01:53:45,073][105620] Updated weights for policy 1, policy_version 1444515 (0.0007) [2023-12-27 01:53:45,305][105692] Updated weights for policy 0, policy_version 1442098 (0.0010) [2023-12-27 01:53:45,375][105692] Updated weights for policy 0, policy_version 1442108 (0.0009) [2023-12-27 01:53:45,443][105692] Updated weights for policy 0, policy_version 1442118 (0.0010) [2023-12-27 01:53:45,513][105692] Updated weights for policy 0, policy_version 1442128 (0.0010) [2023-12-27 01:53:45,736][105620] Updated weights for policy 1, policy_version 1444525 (0.0010) [2023-12-27 01:53:45,787][105620] Updated weights for policy 1, policy_version 1444536 (0.0009) [2023-12-27 01:53:45,837][105620] Updated weights for policy 1, policy_version 1444546 (0.0008) [2023-12-27 01:53:46,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 739090432. Throughput: 0: 9713.2, 1: 9723.9. Samples: 739058708. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:53:46,062][104569] Avg episode reward: [(0, '8712.885'), (1, '8814.655')] [2023-12-27 01:53:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001442128_369238016.pth... [2023-12-27 01:53:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001444552_369852416.pth... [2023-12-27 01:53:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001440976_368943104.pth [2023-12-27 01:53:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001443400_369557504.pth [2023-12-27 01:53:46,159][105692] Updated weights for policy 0, policy_version 1442138 (0.0007) [2023-12-27 01:53:46,206][105692] Updated weights for policy 0, policy_version 1442148 (0.0009) [2023-12-27 01:53:46,254][105692] Updated weights for policy 0, policy_version 1442158 (0.0009) [2023-12-27 01:53:46,636][105620] Updated weights for policy 1, policy_version 1444556 (0.0009) [2023-12-27 01:53:46,682][105620] Updated weights for policy 1, policy_version 1444566 (0.0009) [2023-12-27 01:53:46,728][105620] Updated weights for policy 1, policy_version 1444576 (0.0008) [2023-12-27 01:53:47,007][105692] Updated weights for policy 0, policy_version 1442168 (0.0006) [2023-12-27 01:53:47,071][105692] Updated weights for policy 0, policy_version 1442178 (0.0005) [2023-12-27 01:53:47,133][105692] Updated weights for policy 0, policy_version 1442188 (0.0008) [2023-12-27 01:53:47,512][105620] Updated weights for policy 1, policy_version 1444586 (0.0008) [2023-12-27 01:53:47,568][105620] Updated weights for policy 1, policy_version 1444596 (0.0005) [2023-12-27 01:53:47,618][105620] Updated weights for policy 1, policy_version 1444606 (0.0005) [2023-12-27 01:53:47,648][105692] Updated weights for policy 0, policy_version 1442198 (0.0009) [2023-12-27 01:53:47,675][105620] Updated weights for policy 1, policy_version 1444616 (0.0006) [2023-12-27 01:53:47,706][105692] Updated weights for policy 0, policy_version 1442208 (0.0010) [2023-12-27 01:53:47,774][105692] Updated weights for policy 0, policy_version 1442218 (0.0010) [2023-12-27 01:53:48,396][105620] Updated weights for policy 1, policy_version 1444626 (0.0008) [2023-12-27 01:53:48,463][105620] Updated weights for policy 1, policy_version 1444636 (0.0008) [2023-12-27 01:53:48,501][105692] Updated weights for policy 0, policy_version 1442228 (0.0010) [2023-12-27 01:53:48,523][105620] Updated weights for policy 1, policy_version 1444646 (0.0007) [2023-12-27 01:53:48,554][105692] Updated weights for policy 0, policy_version 1442238 (0.0011) [2023-12-27 01:53:48,606][105692] Updated weights for policy 0, policy_version 1442248 (0.0010) [2023-12-27 01:53:49,242][105620] Updated weights for policy 1, policy_version 1444656 (0.0008) [2023-12-27 01:53:49,290][105620] Updated weights for policy 1, policy_version 1444666 (0.0008) [2023-12-27 01:53:49,348][105620] Updated weights for policy 1, policy_version 1444676 (0.0008) [2023-12-27 01:53:49,391][105692] Updated weights for policy 0, policy_version 1442258 (0.0010) [2023-12-27 01:53:49,478][105692] Updated weights for policy 0, policy_version 1442268 (0.0011) [2023-12-27 01:53:49,529][105692] Updated weights for policy 0, policy_version 1442278 (0.0010) [2023-12-27 01:53:49,582][105692] Updated weights for policy 0, policy_version 1442288 (0.0010) [2023-12-27 01:53:50,133][105620] Updated weights for policy 1, policy_version 1444686 (0.0009) [2023-12-27 01:53:50,189][105620] Updated weights for policy 1, policy_version 1444696 (0.0008) [2023-12-27 01:53:50,251][105620] Updated weights for policy 1, policy_version 1444706 (0.0008) [2023-12-27 01:53:50,324][105692] Updated weights for policy 0, policy_version 1442298 (0.0011) [2023-12-27 01:53:50,390][105692] Updated weights for policy 0, policy_version 1442308 (0.0010) [2023-12-27 01:53:50,452][105692] Updated weights for policy 0, policy_version 1442318 (0.0010) [2023-12-27 01:53:51,019][105620] Updated weights for policy 1, policy_version 1444716 (0.0007) [2023-12-27 01:53:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 739180544. Throughput: 0: 9802.8, 1: 9660.8. Samples: 739176136. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:53:51,062][104569] Avg episode reward: [(0, '8804.511'), (1, '8634.352')] [2023-12-27 01:53:51,089][105620] Updated weights for policy 1, policy_version 1444726 (0.0008) [2023-12-27 01:53:51,156][105620] Updated weights for policy 1, policy_version 1444736 (0.0008) [2023-12-27 01:53:51,196][105692] Updated weights for policy 0, policy_version 1442328 (0.0010) [2023-12-27 01:53:51,248][105692] Updated weights for policy 0, policy_version 1442338 (0.0010) [2023-12-27 01:53:51,314][105692] Updated weights for policy 0, policy_version 1442348 (0.0011) [2023-12-27 01:53:51,939][105620] Updated weights for policy 1, policy_version 1444746 (0.0006) [2023-12-27 01:53:52,001][105620] Updated weights for policy 1, policy_version 1444756 (0.0008) [2023-12-27 01:53:52,064][105620] Updated weights for policy 1, policy_version 1444766 (0.0009) [2023-12-27 01:53:52,092][105692] Updated weights for policy 0, policy_version 1442358 (0.0011) [2023-12-27 01:53:52,123][105620] Updated weights for policy 1, policy_version 1444776 (0.0006) [2023-12-27 01:53:52,145][105692] Updated weights for policy 0, policy_version 1442368 (0.0010) [2023-12-27 01:53:52,201][105692] Updated weights for policy 0, policy_version 1442378 (0.0010) [2023-12-27 01:53:52,879][105620] Updated weights for policy 1, policy_version 1444786 (0.0008) [2023-12-27 01:53:52,941][105620] Updated weights for policy 1, policy_version 1444796 (0.0008) [2023-12-27 01:53:52,965][105692] Updated weights for policy 0, policy_version 1442388 (0.0010) [2023-12-27 01:53:53,002][105620] Updated weights for policy 1, policy_version 1444806 (0.0009) [2023-12-27 01:53:53,016][105692] Updated weights for policy 0, policy_version 1442398 (0.0010) [2023-12-27 01:53:53,067][105692] Updated weights for policy 0, policy_version 1442408 (0.0010) [2023-12-27 01:53:53,691][105620] Updated weights for policy 1, policy_version 1444816 (0.0006) [2023-12-27 01:53:53,704][105692] Updated weights for policy 0, policy_version 1442418 (0.0009) [2023-12-27 01:53:53,752][105620] Updated weights for policy 1, policy_version 1444826 (0.0005) [2023-12-27 01:53:53,753][105692] Updated weights for policy 0, policy_version 1442428 (0.0005) [2023-12-27 01:53:53,808][105692] Updated weights for policy 0, policy_version 1442438 (0.0006) [2023-12-27 01:53:53,810][105620] Updated weights for policy 1, policy_version 1444836 (0.0006) [2023-12-27 01:53:53,863][105692] Updated weights for policy 0, policy_version 1442448 (0.0006) [2023-12-27 01:53:54,430][105692] Updated weights for policy 0, policy_version 1442458 (0.0008) [2023-12-27 01:53:54,495][105620] Updated weights for policy 1, policy_version 1444846 (0.0011) [2023-12-27 01:53:54,498][105692] Updated weights for policy 0, policy_version 1442468 (0.0010) [2023-12-27 01:53:54,547][105620] Updated weights for policy 1, policy_version 1444856 (0.0011) [2023-12-27 01:53:54,560][105692] Updated weights for policy 0, policy_version 1442478 (0.0010) [2023-12-27 01:53:54,608][105620] Updated weights for policy 1, policy_version 1444866 (0.0007) [2023-12-27 01:53:55,203][105620] Updated weights for policy 1, policy_version 1444876 (0.0008) [2023-12-27 01:53:55,238][105692] Updated weights for policy 0, policy_version 1442488 (0.0010) [2023-12-27 01:53:55,262][105620] Updated weights for policy 1, policy_version 1444886 (0.0011) [2023-12-27 01:53:55,296][105692] Updated weights for policy 0, policy_version 1442498 (0.0010) [2023-12-27 01:53:55,320][105620] Updated weights for policy 1, policy_version 1444896 (0.0010) [2023-12-27 01:53:55,350][105692] Updated weights for policy 0, policy_version 1442508 (0.0010) [2023-12-27 01:53:55,976][105620] Updated weights for policy 1, policy_version 1444906 (0.0010) [2023-12-27 01:53:55,991][105692] Updated weights for policy 0, policy_version 1442518 (0.0009) [2023-12-27 01:53:56,036][105620] Updated weights for policy 1, policy_version 1444916 (0.0006) [2023-12-27 01:53:56,053][105692] Updated weights for policy 0, policy_version 1442528 (0.0011) [2023-12-27 01:53:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.3, 300 sec: 19466.4). Total num frames: 739278848. Throughput: 0: 9882.3, 1: 9651.7. Samples: 739294348. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:53:56,062][104569] Avg episode reward: [(0, '8714.414'), (1, '8810.378')] [2023-12-27 01:53:56,091][105620] Updated weights for policy 1, policy_version 1444926 (0.0006) [2023-12-27 01:53:56,116][105692] Updated weights for policy 0, policy_version 1442538 (0.0011) [2023-12-27 01:53:56,144][105620] Updated weights for policy 1, policy_version 1444936 (0.0005) [2023-12-27 01:53:56,745][105620] Updated weights for policy 1, policy_version 1444946 (0.0005) [2023-12-27 01:53:56,798][105620] Updated weights for policy 1, policy_version 1444956 (0.0005) [2023-12-27 01:53:56,838][105692] Updated weights for policy 0, policy_version 1442548 (0.0010) [2023-12-27 01:53:56,852][105620] Updated weights for policy 1, policy_version 1444966 (0.0006) [2023-12-27 01:53:56,886][105692] Updated weights for policy 0, policy_version 1442558 (0.0010) [2023-12-27 01:53:56,934][105692] Updated weights for policy 0, policy_version 1442568 (0.0010) [2023-12-27 01:53:57,423][105620] Updated weights for policy 1, policy_version 1444976 (0.0005) [2023-12-27 01:53:57,476][105620] Updated weights for policy 1, policy_version 1444986 (0.0005) [2023-12-27 01:53:57,529][105620] Updated weights for policy 1, policy_version 1444996 (0.0005) [2023-12-27 01:53:57,624][105692] Updated weights for policy 0, policy_version 1442578 (0.0007) [2023-12-27 01:53:57,680][105692] Updated weights for policy 0, policy_version 1442588 (0.0009) [2023-12-27 01:53:57,744][105692] Updated weights for policy 0, policy_version 1442598 (0.0005) [2023-12-27 01:53:57,798][105692] Updated weights for policy 0, policy_version 1442608 (0.0005) [2023-12-27 01:53:58,104][105620] Updated weights for policy 1, policy_version 1445006 (0.0006) [2023-12-27 01:53:58,155][105620] Updated weights for policy 1, policy_version 1445016 (0.0006) [2023-12-27 01:53:58,223][105620] Updated weights for policy 1, policy_version 1445026 (0.0008) [2023-12-27 01:53:58,401][105692] Updated weights for policy 0, policy_version 1442618 (0.0010) [2023-12-27 01:53:58,468][105692] Updated weights for policy 0, policy_version 1442628 (0.0011) [2023-12-27 01:53:58,523][105692] Updated weights for policy 0, policy_version 1442638 (0.0010) [2023-12-27 01:53:58,915][105620] Updated weights for policy 1, policy_version 1445036 (0.0008) [2023-12-27 01:53:58,980][105620] Updated weights for policy 1, policy_version 1445046 (0.0008) [2023-12-27 01:53:59,044][105620] Updated weights for policy 1, policy_version 1445056 (0.0008) [2023-12-27 01:53:59,265][105692] Updated weights for policy 0, policy_version 1442648 (0.0010) [2023-12-27 01:53:59,324][105692] Updated weights for policy 0, policy_version 1442658 (0.0010) [2023-12-27 01:53:59,386][105692] Updated weights for policy 0, policy_version 1442668 (0.0011) [2023-12-27 01:53:59,754][105620] Updated weights for policy 1, policy_version 1445066 (0.0008) [2023-12-27 01:53:59,816][105620] Updated weights for policy 1, policy_version 1445076 (0.0010) [2023-12-27 01:53:59,873][105620] Updated weights for policy 1, policy_version 1445087 (0.0009) [2023-12-27 01:54:00,118][105692] Updated weights for policy 0, policy_version 1442678 (0.0010) [2023-12-27 01:54:00,179][105692] Updated weights for policy 0, policy_version 1442688 (0.0010) [2023-12-27 01:54:00,237][105692] Updated weights for policy 0, policy_version 1442698 (0.0010) [2023-12-27 01:54:00,619][105620] Updated weights for policy 1, policy_version 1445097 (0.0008) [2023-12-27 01:54:00,684][105620] Updated weights for policy 1, policy_version 1445107 (0.0005) [2023-12-27 01:54:00,733][105620] Updated weights for policy 1, policy_version 1445117 (0.0005) [2023-12-27 01:54:00,792][105620] Updated weights for policy 1, policy_version 1445127 (0.0007) [2023-12-27 01:54:00,914][105692] Updated weights for policy 0, policy_version 1442708 (0.0008) [2023-12-27 01:54:00,973][105692] Updated weights for policy 0, policy_version 1442718 (0.0006) [2023-12-27 01:54:01,031][105692] Updated weights for policy 0, policy_version 1442728 (0.0006) [2023-12-27 01:54:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 739385344. Throughput: 0: 9914.4, 1: 9781.4. Samples: 739358276. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:54:01,062][104569] Avg episode reward: [(0, '8805.172'), (1, '8713.981')] [2023-12-27 01:54:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001445128_369999872.pth... [2023-12-27 01:54:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001444008_369713152.pth [2023-12-27 01:54:01,081][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001442736_369393664.pth... [2023-12-27 01:54:01,086][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001441552_369090560.pth [2023-12-27 01:54:01,428][105620] Updated weights for policy 1, policy_version 1445137 (0.0009) [2023-12-27 01:54:01,485][105620] Updated weights for policy 1, policy_version 1445147 (0.0009) [2023-12-27 01:54:01,537][105620] Updated weights for policy 1, policy_version 1445157 (0.0009) [2023-12-27 01:54:01,691][105692] Updated weights for policy 0, policy_version 1442738 (0.0008) [2023-12-27 01:54:01,756][105692] Updated weights for policy 0, policy_version 1442748 (0.0009) [2023-12-27 01:54:01,806][105692] Updated weights for policy 0, policy_version 1442758 (0.0008) [2023-12-27 01:54:01,853][105692] Updated weights for policy 0, policy_version 1442768 (0.0009) [2023-12-27 01:54:02,357][105620] Updated weights for policy 1, policy_version 1445167 (0.0009) [2023-12-27 01:54:02,424][105620] Updated weights for policy 1, policy_version 1445177 (0.0009) [2023-12-27 01:54:02,482][105620] Updated weights for policy 1, policy_version 1445187 (0.0008) [2023-12-27 01:54:02,571][105692] Updated weights for policy 0, policy_version 1442778 (0.0010) [2023-12-27 01:54:02,622][105692] Updated weights for policy 0, policy_version 1442788 (0.0007) [2023-12-27 01:54:02,672][105692] Updated weights for policy 0, policy_version 1442798 (0.0006) [2023-12-27 01:54:03,080][105620] Updated weights for policy 1, policy_version 1445197 (0.0006) [2023-12-27 01:54:03,131][105620] Updated weights for policy 1, policy_version 1445207 (0.0005) [2023-12-27 01:54:03,198][105620] Updated weights for policy 1, policy_version 1445217 (0.0009) [2023-12-27 01:54:03,469][105692] Updated weights for policy 0, policy_version 1442808 (0.0010) [2023-12-27 01:54:03,520][105692] Updated weights for policy 0, policy_version 1442818 (0.0010) [2023-12-27 01:54:03,568][105692] Updated weights for policy 0, policy_version 1442828 (0.0010) [2023-12-27 01:54:03,860][105620] Updated weights for policy 1, policy_version 1445227 (0.0010) [2023-12-27 01:54:03,928][105620] Updated weights for policy 1, policy_version 1445237 (0.0009) [2023-12-27 01:54:03,978][105620] Updated weights for policy 1, policy_version 1445247 (0.0008) [2023-12-27 01:54:04,373][105692] Updated weights for policy 0, policy_version 1442838 (0.0010) [2023-12-27 01:54:04,432][105692] Updated weights for policy 0, policy_version 1442848 (0.0010) [2023-12-27 01:54:04,495][105692] Updated weights for policy 0, policy_version 1442858 (0.0010) [2023-12-27 01:54:04,762][105620] Updated weights for policy 1, policy_version 1445257 (0.0008) [2023-12-27 01:54:04,813][105620] Updated weights for policy 1, policy_version 1445267 (0.0008) [2023-12-27 01:54:04,875][105620] Updated weights for policy 1, policy_version 1445277 (0.0008) [2023-12-27 01:54:04,927][105620] Updated weights for policy 1, policy_version 1445287 (0.0010) [2023-12-27 01:54:05,260][105692] Updated weights for policy 0, policy_version 1442868 (0.0009) [2023-12-27 01:54:05,326][105692] Updated weights for policy 0, policy_version 1442878 (0.0006) [2023-12-27 01:54:05,394][105692] Updated weights for policy 0, policy_version 1442888 (0.0008) [2023-12-27 01:54:05,722][105620] Updated weights for policy 1, policy_version 1445297 (0.0008) [2023-12-27 01:54:05,780][105620] Updated weights for policy 1, policy_version 1445307 (0.0009) [2023-12-27 01:54:05,837][105620] Updated weights for policy 1, policy_version 1445317 (0.0009) [2023-12-27 01:54:05,986][105692] Updated weights for policy 0, policy_version 1442898 (0.0006) [2023-12-27 01:54:06,059][105692] Updated weights for policy 0, policy_version 1442908 (0.0006) [2023-12-27 01:54:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 739483648. Throughput: 0: 9840.9, 1: 9867.5. Samples: 739474188. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:54:06,063][104569] Avg episode reward: [(0, '8802.368'), (1, '8990.275')] [2023-12-27 01:54:06,131][105692] Updated weights for policy 0, policy_version 1442918 (0.0007) [2023-12-27 01:54:06,197][105692] Updated weights for policy 0, policy_version 1442928 (0.0009) [2023-12-27 01:54:06,654][105620] Updated weights for policy 1, policy_version 1445327 (0.0009) [2023-12-27 01:54:06,716][105620] Updated weights for policy 1, policy_version 1445337 (0.0009) [2023-12-27 01:54:06,779][105620] Updated weights for policy 1, policy_version 1445347 (0.0009) [2023-12-27 01:54:06,847][105692] Updated weights for policy 0, policy_version 1442938 (0.0009) [2023-12-27 01:54:06,906][105692] Updated weights for policy 0, policy_version 1442948 (0.0009) [2023-12-27 01:54:06,939][105585] KL-divergence is very high: 127.3201 [2023-12-27 01:54:06,970][105692] Updated weights for policy 0, policy_version 1442958 (0.0008) [2023-12-27 01:54:07,563][105620] Updated weights for policy 1, policy_version 1445357 (0.0010) [2023-12-27 01:54:07,619][105620] Updated weights for policy 1, policy_version 1445367 (0.0009) [2023-12-27 01:54:07,673][105620] Updated weights for policy 1, policy_version 1445377 (0.0009) [2023-12-27 01:54:07,684][105692] Updated weights for policy 0, policy_version 1442968 (0.0007) [2023-12-27 01:54:07,731][105692] Updated weights for policy 0, policy_version 1442978 (0.0007) [2023-12-27 01:54:07,781][105692] Updated weights for policy 0, policy_version 1442988 (0.0009) [2023-12-27 01:54:08,443][105692] Updated weights for policy 0, policy_version 1442998 (0.0007) [2023-12-27 01:54:08,489][105620] Updated weights for policy 1, policy_version 1445387 (0.0007) [2023-12-27 01:54:08,502][105692] Updated weights for policy 0, policy_version 1443008 (0.0009) [2023-12-27 01:54:08,542][105620] Updated weights for policy 1, policy_version 1445397 (0.0009) [2023-12-27 01:54:08,560][105692] Updated weights for policy 0, policy_version 1443018 (0.0010) [2023-12-27 01:54:08,598][105620] Updated weights for policy 1, policy_version 1445407 (0.0005) [2023-12-27 01:54:09,316][105692] Updated weights for policy 0, policy_version 1443028 (0.0010) [2023-12-27 01:54:09,383][105620] Updated weights for policy 1, policy_version 1445417 (0.0009) [2023-12-27 01:54:09,387][105692] Updated weights for policy 0, policy_version 1443038 (0.0009) [2023-12-27 01:54:09,447][105692] Updated weights for policy 0, policy_version 1443048 (0.0009) [2023-12-27 01:54:09,450][105620] Updated weights for policy 1, policy_version 1445427 (0.0008) [2023-12-27 01:54:09,510][105620] Updated weights for policy 1, policy_version 1445437 (0.0008) [2023-12-27 01:54:09,570][105620] Updated weights for policy 1, policy_version 1445447 (0.0008) [2023-12-27 01:54:10,187][105692] Updated weights for policy 0, policy_version 1443058 (0.0006) [2023-12-27 01:54:10,252][105692] Updated weights for policy 0, policy_version 1443068 (0.0010) [2023-12-27 01:54:10,314][105692] Updated weights for policy 0, policy_version 1443078 (0.0009) [2023-12-27 01:54:10,351][105620] Updated weights for policy 1, policy_version 1445457 (0.0006) [2023-12-27 01:54:10,376][105692] Updated weights for policy 0, policy_version 1443088 (0.0008) [2023-12-27 01:54:10,414][105620] Updated weights for policy 1, policy_version 1445467 (0.0010) [2023-12-27 01:54:10,473][105620] Updated weights for policy 1, policy_version 1445477 (0.0011) [2023-12-27 01:54:11,037][105692] Updated weights for policy 0, policy_version 1443098 (0.0007) [2023-12-27 01:54:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 739573760. Throughput: 0: 9832.2, 1: 9742.0. Samples: 739587040. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:54:11,063][104569] Avg episode reward: [(0, '9076.506'), (1, '7285.328')] [2023-12-27 01:54:11,095][105692] Updated weights for policy 0, policy_version 1443108 (0.0011) [2023-12-27 01:54:11,163][105692] Updated weights for policy 0, policy_version 1443118 (0.0011) [2023-12-27 01:54:11,200][105620] Updated weights for policy 1, policy_version 1445487 (0.0011) [2023-12-27 01:54:11,268][105620] Updated weights for policy 1, policy_version 1445497 (0.0011) [2023-12-27 01:54:11,318][105620] Updated weights for policy 1, policy_version 1445507 (0.0010) [2023-12-27 01:54:11,938][105692] Updated weights for policy 0, policy_version 1443128 (0.0011) [2023-12-27 01:54:11,991][105692] Updated weights for policy 0, policy_version 1443138 (0.0011) [2023-12-27 01:54:12,023][105620] Updated weights for policy 1, policy_version 1445517 (0.0010) [2023-12-27 01:54:12,051][105692] Updated weights for policy 0, policy_version 1443148 (0.0011) [2023-12-27 01:54:12,085][105620] Updated weights for policy 1, policy_version 1445527 (0.0010) [2023-12-27 01:54:12,147][105620] Updated weights for policy 1, policy_version 1445537 (0.0010) [2023-12-27 01:54:12,765][105692] Updated weights for policy 0, policy_version 1443158 (0.0009) [2023-12-27 01:54:12,817][105692] Updated weights for policy 0, policy_version 1443168 (0.0010) [2023-12-27 01:54:12,868][105692] Updated weights for policy 0, policy_version 1443178 (0.0010) [2023-12-27 01:54:12,912][105620] Updated weights for policy 1, policy_version 1445548 (0.0010) [2023-12-27 01:54:12,960][105620] Updated weights for policy 1, policy_version 1445558 (0.0010) [2023-12-27 01:54:13,006][105620] Updated weights for policy 1, policy_version 1445568 (0.0009) [2023-12-27 01:54:13,614][105620] Updated weights for policy 1, policy_version 1445578 (0.0009) [2023-12-27 01:54:13,622][105692] Updated weights for policy 0, policy_version 1443188 (0.0011) [2023-12-27 01:54:13,667][105692] Updated weights for policy 0, policy_version 1443198 (0.0010) [2023-12-27 01:54:13,679][105620] Updated weights for policy 1, policy_version 1445588 (0.0005) [2023-12-27 01:54:13,717][105692] Updated weights for policy 0, policy_version 1443208 (0.0009) [2023-12-27 01:54:13,738][105620] Updated weights for policy 1, policy_version 1445598 (0.0006) [2023-12-27 01:54:13,786][105620] Updated weights for policy 1, policy_version 1445608 (0.0007) [2023-12-27 01:54:14,370][105620] Updated weights for policy 1, policy_version 1445618 (0.0008) [2023-12-27 01:54:14,415][105620] Updated weights for policy 1, policy_version 1445628 (0.0008) [2023-12-27 01:54:14,467][105620] Updated weights for policy 1, policy_version 1445638 (0.0008) [2023-12-27 01:54:14,477][105692] Updated weights for policy 0, policy_version 1443218 (0.0010) [2023-12-27 01:54:14,525][105692] Updated weights for policy 0, policy_version 1443228 (0.0008) [2023-12-27 01:54:14,578][105692] Updated weights for policy 0, policy_version 1443238 (0.0008) [2023-12-27 01:54:14,636][105692] Updated weights for policy 0, policy_version 1443248 (0.0008) [2023-12-27 01:54:15,230][105620] Updated weights for policy 1, policy_version 1445648 (0.0010) [2023-12-27 01:54:15,293][105620] Updated weights for policy 1, policy_version 1445658 (0.0011) [2023-12-27 01:54:15,359][105620] Updated weights for policy 1, policy_version 1445668 (0.0011) [2023-12-27 01:54:15,428][105692] Updated weights for policy 0, policy_version 1443258 (0.0011) [2023-12-27 01:54:15,489][105692] Updated weights for policy 0, policy_version 1443268 (0.0010) [2023-12-27 01:54:15,550][105692] Updated weights for policy 0, policy_version 1443278 (0.0010) [2023-12-27 01:54:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 739672064. Throughput: 0: 9848.3, 1: 9644.3. Samples: 739646284. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:54:16,063][104569] Avg episode reward: [(0, '8891.452'), (1, '7564.159')] [2023-12-27 01:54:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001443280_369532928.pth... [2023-12-27 01:54:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001442128_369238016.pth [2023-12-27 01:54:16,084][105620] Updated weights for policy 1, policy_version 1445678 (0.0008) [2023-12-27 01:54:16,132][105620] Updated weights for policy 1, policy_version 1445688 (0.0010) [2023-12-27 01:54:16,178][105620] Updated weights for policy 1, policy_version 1445698 (0.0010) [2023-12-27 01:54:16,209][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001445704_370147328.pth... [2023-12-27 01:54:16,212][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001444552_369852416.pth [2023-12-27 01:54:16,281][105692] Updated weights for policy 0, policy_version 1443288 (0.0010) [2023-12-27 01:54:16,337][105692] Updated weights for policy 0, policy_version 1443298 (0.0010) [2023-12-27 01:54:16,401][105692] Updated weights for policy 0, policy_version 1443308 (0.0011) [2023-12-27 01:54:16,957][105620] Updated weights for policy 1, policy_version 1445708 (0.0011) [2023-12-27 01:54:17,005][105620] Updated weights for policy 1, policy_version 1445718 (0.0010) [2023-12-27 01:54:17,049][105620] Updated weights for policy 1, policy_version 1445728 (0.0010) [2023-12-27 01:54:17,114][105692] Updated weights for policy 0, policy_version 1443318 (0.0011) [2023-12-27 01:54:17,165][105692] Updated weights for policy 0, policy_version 1443328 (0.0010) [2023-12-27 01:54:17,212][105692] Updated weights for policy 0, policy_version 1443338 (0.0010) [2023-12-27 01:54:17,773][105620] Updated weights for policy 1, policy_version 1445738 (0.0010) [2023-12-27 01:54:17,835][105620] Updated weights for policy 1, policy_version 1445748 (0.0005) [2023-12-27 01:54:17,878][105692] Updated weights for policy 0, policy_version 1443348 (0.0010) [2023-12-27 01:54:17,901][105620] Updated weights for policy 1, policy_version 1445758 (0.0010) [2023-12-27 01:54:17,939][105692] Updated weights for policy 0, policy_version 1443358 (0.0010) [2023-12-27 01:54:17,952][105620] Updated weights for policy 1, policy_version 1445768 (0.0010) [2023-12-27 01:54:18,000][105692] Updated weights for policy 0, policy_version 1443368 (0.0010) [2023-12-27 01:54:18,594][105620] Updated weights for policy 1, policy_version 1445778 (0.0007) [2023-12-27 01:54:18,645][105620] Updated weights for policy 1, policy_version 1445788 (0.0007) [2023-12-27 01:54:18,694][105620] Updated weights for policy 1, policy_version 1445798 (0.0005) [2023-12-27 01:54:18,779][105692] Updated weights for policy 0, policy_version 1443379 (0.0011) [2023-12-27 01:54:18,832][105692] Updated weights for policy 0, policy_version 1443389 (0.0010) [2023-12-27 01:54:18,886][105692] Updated weights for policy 0, policy_version 1443400 (0.0010) [2023-12-27 01:54:19,274][105620] Updated weights for policy 1, policy_version 1445808 (0.0007) [2023-12-27 01:54:19,340][105620] Updated weights for policy 1, policy_version 1445818 (0.0007) [2023-12-27 01:54:19,407][105620] Updated weights for policy 1, policy_version 1445828 (0.0008) [2023-12-27 01:54:19,685][105692] Updated weights for policy 0, policy_version 1443410 (0.0009) [2023-12-27 01:54:19,756][105692] Updated weights for policy 0, policy_version 1443420 (0.0008) [2023-12-27 01:54:19,824][105692] Updated weights for policy 0, policy_version 1443430 (0.0009) [2023-12-27 01:54:19,892][105692] Updated weights for policy 0, policy_version 1443440 (0.0010) [2023-12-27 01:54:20,095][105620] Updated weights for policy 1, policy_version 1445838 (0.0009) [2023-12-27 01:54:20,157][105620] Updated weights for policy 1, policy_version 1445848 (0.0009) [2023-12-27 01:54:20,211][105620] Updated weights for policy 1, policy_version 1445858 (0.0011) [2023-12-27 01:54:20,608][105692] Updated weights for policy 0, policy_version 1443450 (0.0009) [2023-12-27 01:54:20,676][105692] Updated weights for policy 0, policy_version 1443460 (0.0008) [2023-12-27 01:54:20,740][105692] Updated weights for policy 0, policy_version 1443470 (0.0009) [2023-12-27 01:54:20,999][105620] Updated weights for policy 1, policy_version 1445868 (0.0010) [2023-12-27 01:54:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 739770368. Throughput: 0: 9807.3, 1: 9698.0. Samples: 739763796. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:54:21,063][104569] Avg episode reward: [(0, '8615.532'), (1, '8782.922')] [2023-12-27 01:54:21,065][105620] Updated weights for policy 1, policy_version 1445878 (0.0007) [2023-12-27 01:54:21,129][105620] Updated weights for policy 1, policy_version 1445888 (0.0008) [2023-12-27 01:54:21,568][105692] Updated weights for policy 0, policy_version 1443480 (0.0010) [2023-12-27 01:54:21,623][105692] Updated weights for policy 0, policy_version 1443490 (0.0009) [2023-12-27 01:54:21,691][105692] Updated weights for policy 0, policy_version 1443500 (0.0006) [2023-12-27 01:54:21,821][105620] Updated weights for policy 1, policy_version 1445898 (0.0006) [2023-12-27 01:54:21,884][105620] Updated weights for policy 1, policy_version 1445908 (0.0009) [2023-12-27 01:54:21,946][105620] Updated weights for policy 1, policy_version 1445918 (0.0008) [2023-12-27 01:54:22,013][105620] Updated weights for policy 1, policy_version 1445928 (0.0008) [2023-12-27 01:54:22,436][105692] Updated weights for policy 0, policy_version 1443510 (0.0008) [2023-12-27 01:54:22,495][105692] Updated weights for policy 0, policy_version 1443520 (0.0009) [2023-12-27 01:54:22,543][105692] Updated weights for policy 0, policy_version 1443530 (0.0009) [2023-12-27 01:54:22,773][105620] Updated weights for policy 1, policy_version 1445938 (0.0009) [2023-12-27 01:54:22,836][105620] Updated weights for policy 1, policy_version 1445948 (0.0008) [2023-12-27 01:54:22,897][105620] Updated weights for policy 1, policy_version 1445958 (0.0009) [2023-12-27 01:54:23,290][105692] Updated weights for policy 0, policy_version 1443540 (0.0010) [2023-12-27 01:54:23,344][105692] Updated weights for policy 0, policy_version 1443550 (0.0010) [2023-12-27 01:54:23,405][105692] Updated weights for policy 0, policy_version 1443560 (0.0008) [2023-12-27 01:54:23,614][105620] Updated weights for policy 1, policy_version 1445968 (0.0007) [2023-12-27 01:54:23,677][105620] Updated weights for policy 1, policy_version 1445978 (0.0005) [2023-12-27 01:54:23,730][105620] Updated weights for policy 1, policy_version 1445988 (0.0005) [2023-12-27 01:54:24,089][105692] Updated weights for policy 0, policy_version 1443570 (0.0009) [2023-12-27 01:54:24,150][105692] Updated weights for policy 0, policy_version 1443580 (0.0010) [2023-12-27 01:54:24,216][105692] Updated weights for policy 0, policy_version 1443590 (0.0010) [2023-12-27 01:54:24,264][105692] Updated weights for policy 0, policy_version 1443600 (0.0010) [2023-12-27 01:54:24,294][105620] Updated weights for policy 1, policy_version 1445998 (0.0008) [2023-12-27 01:54:24,361][105620] Updated weights for policy 1, policy_version 1446008 (0.0007) [2023-12-27 01:54:24,413][105620] Updated weights for policy 1, policy_version 1446018 (0.0008) [2023-12-27 01:54:25,017][105692] Updated weights for policy 0, policy_version 1443610 (0.0009) [2023-12-27 01:54:25,051][105620] Updated weights for policy 1, policy_version 1446028 (0.0009) [2023-12-27 01:54:25,074][105692] Updated weights for policy 0, policy_version 1443620 (0.0006) [2023-12-27 01:54:25,105][105620] Updated weights for policy 1, policy_version 1446038 (0.0007) [2023-12-27 01:54:25,128][105692] Updated weights for policy 0, policy_version 1443630 (0.0007) [2023-12-27 01:54:25,157][105620] Updated weights for policy 1, policy_version 1446048 (0.0007) [2023-12-27 01:54:25,809][105692] Updated weights for policy 0, policy_version 1443640 (0.0006) [2023-12-27 01:54:25,868][105692] Updated weights for policy 0, policy_version 1443650 (0.0005) [2023-12-27 01:54:25,893][105620] Updated weights for policy 1, policy_version 1446058 (0.0006) [2023-12-27 01:54:25,924][105692] Updated weights for policy 0, policy_version 1443660 (0.0005) [2023-12-27 01:54:25,951][105620] Updated weights for policy 1, policy_version 1446068 (0.0009) [2023-12-27 01:54:26,016][105620] Updated weights for policy 1, policy_version 1446078 (0.0008) [2023-12-27 01:54:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 739868672. Throughput: 0: 9706.9, 1: 9765.7. Samples: 739878536. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:54:26,063][104569] Avg episode reward: [(0, '8984.608'), (1, '9175.831')] [2023-12-27 01:54:26,083][105620] Updated weights for policy 1, policy_version 1446088 (0.0007) [2023-12-27 01:54:26,448][105692] Updated weights for policy 0, policy_version 1443670 (0.0005) [2023-12-27 01:54:26,509][105692] Updated weights for policy 0, policy_version 1443680 (0.0005) [2023-12-27 01:54:26,577][105692] Updated weights for policy 0, policy_version 1443690 (0.0005) [2023-12-27 01:54:26,774][105620] Updated weights for policy 1, policy_version 1446098 (0.0005) [2023-12-27 01:54:26,829][105620] Updated weights for policy 1, policy_version 1446108 (0.0005) [2023-12-27 01:54:26,891][105620] Updated weights for policy 1, policy_version 1446118 (0.0005) [2023-12-27 01:54:27,154][105692] Updated weights for policy 0, policy_version 1443700 (0.0005) [2023-12-27 01:54:27,200][105692] Updated weights for policy 0, policy_version 1443710 (0.0006) [2023-12-27 01:54:27,248][105692] Updated weights for policy 0, policy_version 1443720 (0.0006) [2023-12-27 01:54:27,470][105620] Updated weights for policy 1, policy_version 1446128 (0.0007) [2023-12-27 01:54:27,523][105620] Updated weights for policy 1, policy_version 1446138 (0.0009) [2023-12-27 01:54:27,577][105620] Updated weights for policy 1, policy_version 1446148 (0.0010) [2023-12-27 01:54:27,834][105692] Updated weights for policy 0, policy_version 1443730 (0.0009) [2023-12-27 01:54:27,895][105692] Updated weights for policy 0, policy_version 1443740 (0.0007) [2023-12-27 01:54:27,959][105692] Updated weights for policy 0, policy_version 1443750 (0.0010) [2023-12-27 01:54:28,016][105692] Updated weights for policy 0, policy_version 1443760 (0.0010) [2023-12-27 01:54:28,276][105620] Updated weights for policy 1, policy_version 1446158 (0.0010) [2023-12-27 01:54:28,328][105620] Updated weights for policy 1, policy_version 1446168 (0.0010) [2023-12-27 01:54:28,355][105586] KL-divergence is very high: 105.4024 [2023-12-27 01:54:28,392][105620] Updated weights for policy 1, policy_version 1446178 (0.0011) [2023-12-27 01:54:28,406][105586] KL-divergence is very high: 118.7452 [2023-12-27 01:54:28,614][105692] Updated weights for policy 0, policy_version 1443770 (0.0008) [2023-12-27 01:54:28,676][105692] Updated weights for policy 0, policy_version 1443780 (0.0008) [2023-12-27 01:54:28,733][105692] Updated weights for policy 0, policy_version 1443790 (0.0008) [2023-12-27 01:54:29,104][105620] Updated weights for policy 1, policy_version 1446188 (0.0009) [2023-12-27 01:54:29,161][105620] Updated weights for policy 1, policy_version 1446198 (0.0006) [2023-12-27 01:54:29,214][105620] Updated weights for policy 1, policy_version 1446208 (0.0005) [2023-12-27 01:54:29,497][105692] Updated weights for policy 0, policy_version 1443800 (0.0007) [2023-12-27 01:54:29,547][105692] Updated weights for policy 0, policy_version 1443810 (0.0008) [2023-12-27 01:54:29,599][105692] Updated weights for policy 0, policy_version 1443820 (0.0008) [2023-12-27 01:54:29,921][105620] Updated weights for policy 1, policy_version 1446218 (0.0010) [2023-12-27 01:54:29,981][105620] Updated weights for policy 1, policy_version 1446228 (0.0011) [2023-12-27 01:54:30,036][105620] Updated weights for policy 1, policy_version 1446238 (0.0011) [2023-12-27 01:54:30,092][105620] Updated weights for policy 1, policy_version 1446248 (0.0010) [2023-12-27 01:54:30,394][105692] Updated weights for policy 0, policy_version 1443830 (0.0008) [2023-12-27 01:54:30,466][105692] Updated weights for policy 0, policy_version 1443840 (0.0008) [2023-12-27 01:54:30,518][105692] Updated weights for policy 0, policy_version 1443850 (0.0009) [2023-12-27 01:54:30,775][105620] Updated weights for policy 1, policy_version 1446258 (0.0010) [2023-12-27 01:54:30,823][105620] Updated weights for policy 1, policy_version 1446268 (0.0010) [2023-12-27 01:54:30,865][105620] Updated weights for policy 1, policy_version 1446278 (0.0008) [2023-12-27 01:54:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 739975168. Throughput: 0: 9881.1, 1: 9801.8. Samples: 739944436. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:54:31,062][104569] Avg episode reward: [(0, '9077.453'), (1, '8901.443')] [2023-12-27 01:54:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001443856_369680384.pth... [2023-12-27 01:54:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001446280_370294784.pth... [2023-12-27 01:54:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001442736_369393664.pth [2023-12-27 01:54:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001445128_369999872.pth [2023-12-27 01:54:31,224][105692] Updated weights for policy 0, policy_version 1443861 (0.0010) [2023-12-27 01:54:31,277][105692] Updated weights for policy 0, policy_version 1443871 (0.0010) [2023-12-27 01:54:31,336][105692] Updated weights for policy 0, policy_version 1443881 (0.0010) [2023-12-27 01:54:31,592][105620] Updated weights for policy 1, policy_version 1446288 (0.0007) [2023-12-27 01:54:31,664][105620] Updated weights for policy 1, policy_version 1446298 (0.0008) [2023-12-27 01:54:31,734][105620] Updated weights for policy 1, policy_version 1446309 (0.0011) [2023-12-27 01:54:32,031][105692] Updated weights for policy 0, policy_version 1443891 (0.0010) [2023-12-27 01:54:32,093][105692] Updated weights for policy 0, policy_version 1443901 (0.0009) [2023-12-27 01:54:32,158][105692] Updated weights for policy 0, policy_version 1443911 (0.0009) [2023-12-27 01:54:32,494][105620] Updated weights for policy 1, policy_version 1446319 (0.0009) [2023-12-27 01:54:32,549][105620] Updated weights for policy 1, policy_version 1446329 (0.0009) [2023-12-27 01:54:32,608][105620] Updated weights for policy 1, policy_version 1446339 (0.0008) [2023-12-27 01:54:32,824][105692] Updated weights for policy 0, policy_version 1443921 (0.0009) [2023-12-27 01:54:32,878][105692] Updated weights for policy 0, policy_version 1443931 (0.0005) [2023-12-27 01:54:32,931][105692] Updated weights for policy 0, policy_version 1443941 (0.0005) [2023-12-27 01:54:32,985][105692] Updated weights for policy 0, policy_version 1443951 (0.0005) [2023-12-27 01:54:33,401][105620] Updated weights for policy 1, policy_version 1446349 (0.0008) [2023-12-27 01:54:33,452][105620] Updated weights for policy 1, policy_version 1446360 (0.0010) [2023-12-27 01:54:33,490][105692] Updated weights for policy 0, policy_version 1443961 (0.0006) [2023-12-27 01:54:33,503][105620] Updated weights for policy 1, policy_version 1446370 (0.0009) [2023-12-27 01:54:33,548][105692] Updated weights for policy 0, policy_version 1443971 (0.0007) [2023-12-27 01:54:33,602][105692] Updated weights for policy 0, policy_version 1443981 (0.0009) [2023-12-27 01:54:34,086][105620] Updated weights for policy 1, policy_version 1446380 (0.0008) [2023-12-27 01:54:34,150][105620] Updated weights for policy 1, policy_version 1446390 (0.0009) [2023-12-27 01:54:34,213][105620] Updated weights for policy 1, policy_version 1446400 (0.0011) [2023-12-27 01:54:34,419][105692] Updated weights for policy 0, policy_version 1443991 (0.0009) [2023-12-27 01:54:34,467][105692] Updated weights for policy 0, policy_version 1444001 (0.0005) [2023-12-27 01:54:34,528][105692] Updated weights for policy 0, policy_version 1444011 (0.0007) [2023-12-27 01:54:34,974][105620] Updated weights for policy 1, policy_version 1446410 (0.0011) [2023-12-27 01:54:35,029][105620] Updated weights for policy 1, policy_version 1446420 (0.0010) [2023-12-27 01:54:35,107][105620] Updated weights for policy 1, policy_version 1446430 (0.0009) [2023-12-27 01:54:35,170][105620] Updated weights for policy 1, policy_version 1446440 (0.0010) [2023-12-27 01:54:35,323][105692] Updated weights for policy 0, policy_version 1444021 (0.0007) [2023-12-27 01:54:35,388][105692] Updated weights for policy 0, policy_version 1444031 (0.0005) [2023-12-27 01:54:35,458][105692] Updated weights for policy 0, policy_version 1444041 (0.0008) [2023-12-27 01:54:35,907][105620] Updated weights for policy 1, policy_version 1446450 (0.0010) [2023-12-27 01:54:35,955][105620] Updated weights for policy 1, policy_version 1446460 (0.0010) [2023-12-27 01:54:36,012][105620] Updated weights for policy 1, policy_version 1446470 (0.0010) [2023-12-27 01:54:36,062][104569] Fps is (10 sec: 20480.6, 60 sec: 19660.9, 300 sec: 19466.4). Total num frames: 740073472. Throughput: 0: 9843.2, 1: 9851.1. Samples: 740062380. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 01:54:36,062][104569] Avg episode reward: [(0, '8987.903'), (1, '9082.876')] [2023-12-27 01:54:36,116][105692] Updated weights for policy 0, policy_version 1444051 (0.0007) [2023-12-27 01:54:36,176][105692] Updated weights for policy 0, policy_version 1444061 (0.0011) [2023-12-27 01:54:36,235][105692] Updated weights for policy 0, policy_version 1444071 (0.0010) [2023-12-27 01:54:36,763][105620] Updated weights for policy 1, policy_version 1446480 (0.0010) [2023-12-27 01:54:36,821][105620] Updated weights for policy 1, policy_version 1446490 (0.0010) [2023-12-27 01:54:36,882][105620] Updated weights for policy 1, policy_version 1446501 (0.0012) [2023-12-27 01:54:36,906][105692] Updated weights for policy 0, policy_version 1444081 (0.0008) [2023-12-27 01:54:36,975][105692] Updated weights for policy 0, policy_version 1444091 (0.0009) [2023-12-27 01:54:37,041][105692] Updated weights for policy 0, policy_version 1444101 (0.0008) [2023-12-27 01:54:37,108][105692] Updated weights for policy 0, policy_version 1444111 (0.0006) [2023-12-27 01:54:37,660][105692] Updated weights for policy 0, policy_version 1444121 (0.0007) [2023-12-27 01:54:37,689][105620] Updated weights for policy 1, policy_version 1446511 (0.0010) [2023-12-27 01:54:37,708][105692] Updated weights for policy 0, policy_version 1444131 (0.0009) [2023-12-27 01:54:37,745][105620] Updated weights for policy 1, policy_version 1446521 (0.0010) [2023-12-27 01:54:37,765][105692] Updated weights for policy 0, policy_version 1444141 (0.0010) [2023-12-27 01:54:37,801][105620] Updated weights for policy 1, policy_version 1446531 (0.0010) [2023-12-27 01:54:38,488][105692] Updated weights for policy 0, policy_version 1444151 (0.0007) [2023-12-27 01:54:38,552][105692] Updated weights for policy 0, policy_version 1444161 (0.0007) [2023-12-27 01:54:38,561][105620] Updated weights for policy 1, policy_version 1446541 (0.0010) [2023-12-27 01:54:38,606][105620] Updated weights for policy 1, policy_version 1446551 (0.0010) [2023-12-27 01:54:38,611][105692] Updated weights for policy 0, policy_version 1444171 (0.0007) [2023-12-27 01:54:38,655][105620] Updated weights for policy 1, policy_version 1446561 (0.0010) [2023-12-27 01:54:39,237][105692] Updated weights for policy 0, policy_version 1444181 (0.0008) [2023-12-27 01:54:39,292][105692] Updated weights for policy 0, policy_version 1444191 (0.0007) [2023-12-27 01:54:39,364][105692] Updated weights for policy 0, policy_version 1444201 (0.0008) [2023-12-27 01:54:39,429][105620] Updated weights for policy 1, policy_version 1446571 (0.0010) [2023-12-27 01:54:39,486][105620] Updated weights for policy 1, policy_version 1446581 (0.0007) [2023-12-27 01:54:39,541][105620] Updated weights for policy 1, policy_version 1446591 (0.0005) [2023-12-27 01:54:40,248][105692] Updated weights for policy 0, policy_version 1444211 (0.0008) [2023-12-27 01:54:40,263][105620] Updated weights for policy 1, policy_version 1446601 (0.0008) [2023-12-27 01:54:40,305][105692] Updated weights for policy 0, policy_version 1444221 (0.0005) [2023-12-27 01:54:40,324][105620] Updated weights for policy 1, policy_version 1446611 (0.0010) [2023-12-27 01:54:40,366][105692] Updated weights for policy 0, policy_version 1444231 (0.0006) [2023-12-27 01:54:40,384][105620] Updated weights for policy 1, policy_version 1446621 (0.0010) [2023-12-27 01:54:40,441][105620] Updated weights for policy 1, policy_version 1446631 (0.0009) [2023-12-27 01:54:41,028][105692] Updated weights for policy 0, policy_version 1444241 (0.0010) [2023-12-27 01:54:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 740163584. Throughput: 0: 9828.1, 1: 9786.3. Samples: 740176996. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:54:41,062][104569] Avg episode reward: [(0, '8898.030'), (1, '8718.912')] [2023-12-27 01:54:41,089][105692] Updated weights for policy 0, policy_version 1444251 (0.0007) [2023-12-27 01:54:41,159][105692] Updated weights for policy 0, policy_version 1444261 (0.0009) [2023-12-27 01:54:41,222][105692] Updated weights for policy 0, policy_version 1444271 (0.0006) [2023-12-27 01:54:41,223][105620] Updated weights for policy 1, policy_version 1446641 (0.0011) [2023-12-27 01:54:41,286][105620] Updated weights for policy 1, policy_version 1446651 (0.0011) [2023-12-27 01:54:41,353][105620] Updated weights for policy 1, policy_version 1446661 (0.0011) [2023-12-27 01:54:42,011][105692] Updated weights for policy 0, policy_version 1444281 (0.0009) [2023-12-27 01:54:42,076][105692] Updated weights for policy 0, policy_version 1444291 (0.0007) [2023-12-27 01:54:42,089][105620] Updated weights for policy 1, policy_version 1446671 (0.0010) [2023-12-27 01:54:42,132][105692] Updated weights for policy 0, policy_version 1444301 (0.0006) [2023-12-27 01:54:42,150][105620] Updated weights for policy 1, policy_version 1446681 (0.0009) [2023-12-27 01:54:42,220][105620] Updated weights for policy 1, policy_version 1446691 (0.0005) [2023-12-27 01:54:42,854][105620] Updated weights for policy 1, policy_version 1446701 (0.0008) [2023-12-27 01:54:42,902][105620] Updated weights for policy 1, policy_version 1446711 (0.0009) [2023-12-27 01:54:42,956][105692] Updated weights for policy 0, policy_version 1444311 (0.0008) [2023-12-27 01:54:42,958][105620] Updated weights for policy 1, policy_version 1446721 (0.0008) [2023-12-27 01:54:43,008][105692] Updated weights for policy 0, policy_version 1444321 (0.0007) [2023-12-27 01:54:43,063][105692] Updated weights for policy 0, policy_version 1444331 (0.0008) [2023-12-27 01:54:43,610][105620] Updated weights for policy 1, policy_version 1446731 (0.0009) [2023-12-27 01:54:43,676][105620] Updated weights for policy 1, policy_version 1446741 (0.0008) [2023-12-27 01:54:43,734][105620] Updated weights for policy 1, policy_version 1446751 (0.0007) [2023-12-27 01:54:43,842][105692] Updated weights for policy 0, policy_version 1444341 (0.0008) [2023-12-27 01:54:43,909][105692] Updated weights for policy 0, policy_version 1444351 (0.0009) [2023-12-27 01:54:43,971][105692] Updated weights for policy 0, policy_version 1444361 (0.0011) [2023-12-27 01:54:44,363][105620] Updated weights for policy 1, policy_version 1446761 (0.0006) [2023-12-27 01:54:44,415][105620] Updated weights for policy 1, policy_version 1446771 (0.0009) [2023-12-27 01:54:44,467][105620] Updated weights for policy 1, policy_version 1446781 (0.0008) [2023-12-27 01:54:44,537][105620] Updated weights for policy 1, policy_version 1446791 (0.0010) [2023-12-27 01:54:44,694][105692] Updated weights for policy 0, policy_version 1444371 (0.0010) [2023-12-27 01:54:44,753][105692] Updated weights for policy 0, policy_version 1444381 (0.0010) [2023-12-27 01:54:44,816][105692] Updated weights for policy 0, policy_version 1444391 (0.0009) [2023-12-27 01:54:45,333][105620] Updated weights for policy 1, policy_version 1446801 (0.0011) [2023-12-27 01:54:45,400][105620] Updated weights for policy 1, policy_version 1446811 (0.0011) [2023-12-27 01:54:45,451][105620] Updated weights for policy 1, policy_version 1446821 (0.0010) [2023-12-27 01:54:45,518][105692] Updated weights for policy 0, policy_version 1444401 (0.0007) [2023-12-27 01:54:45,578][105692] Updated weights for policy 0, policy_version 1444411 (0.0007) [2023-12-27 01:54:45,636][105692] Updated weights for policy 0, policy_version 1444421 (0.0010) [2023-12-27 01:54:45,695][105692] Updated weights for policy 0, policy_version 1444431 (0.0010) [2023-12-27 01:54:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 740261888. Throughput: 0: 9740.3, 1: 9721.6. Samples: 740234064. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:54:46,062][104569] Avg episode reward: [(0, '8893.267'), (1, '8718.828')] [2023-12-27 01:54:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001444432_369827840.pth... [2023-12-27 01:54:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001446824_370434048.pth... [2023-12-27 01:54:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001443280_369532928.pth [2023-12-27 01:54:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001445704_370147328.pth [2023-12-27 01:54:46,198][105620] Updated weights for policy 1, policy_version 1446831 (0.0010) [2023-12-27 01:54:46,256][105620] Updated weights for policy 1, policy_version 1446841 (0.0010) [2023-12-27 01:54:46,314][105620] Updated weights for policy 1, policy_version 1446851 (0.0010) [2023-12-27 01:54:46,375][105692] Updated weights for policy 0, policy_version 1444441 (0.0008) [2023-12-27 01:54:46,436][105692] Updated weights for policy 0, policy_version 1444451 (0.0009) [2023-12-27 01:54:46,492][105692] Updated weights for policy 0, policy_version 1444461 (0.0008) [2023-12-27 01:54:47,059][105620] Updated weights for policy 1, policy_version 1446861 (0.0011) [2023-12-27 01:54:47,118][105620] Updated weights for policy 1, policy_version 1446871 (0.0010) [2023-12-27 01:54:47,157][105692] Updated weights for policy 0, policy_version 1444471 (0.0008) [2023-12-27 01:54:47,176][105620] Updated weights for policy 1, policy_version 1446881 (0.0010) [2023-12-27 01:54:47,217][105692] Updated weights for policy 0, policy_version 1444481 (0.0005) [2023-12-27 01:54:47,280][105692] Updated weights for policy 0, policy_version 1444491 (0.0009) [2023-12-27 01:54:47,898][105692] Updated weights for policy 0, policy_version 1444501 (0.0008) [2023-12-27 01:54:47,904][105620] Updated weights for policy 1, policy_version 1446891 (0.0010) [2023-12-27 01:54:47,950][105692] Updated weights for policy 0, policy_version 1444511 (0.0005) [2023-12-27 01:54:47,966][105620] Updated weights for policy 1, policy_version 1446901 (0.0009) [2023-12-27 01:54:47,997][105692] Updated weights for policy 0, policy_version 1444521 (0.0007) [2023-12-27 01:54:48,019][105620] Updated weights for policy 1, policy_version 1446911 (0.0007) [2023-12-27 01:54:48,662][105692] Updated weights for policy 0, policy_version 1444531 (0.0007) [2023-12-27 01:54:48,732][105692] Updated weights for policy 0, policy_version 1444541 (0.0009) [2023-12-27 01:54:48,780][105620] Updated weights for policy 1, policy_version 1446921 (0.0008) [2023-12-27 01:54:48,796][105692] Updated weights for policy 0, policy_version 1444551 (0.0009) [2023-12-27 01:54:48,843][105620] Updated weights for policy 1, policy_version 1446931 (0.0007) [2023-12-27 01:54:48,904][105620] Updated weights for policy 1, policy_version 1446941 (0.0010) [2023-12-27 01:54:48,966][105620] Updated weights for policy 1, policy_version 1446951 (0.0009) [2023-12-27 01:54:49,595][105692] Updated weights for policy 0, policy_version 1444561 (0.0009) [2023-12-27 01:54:49,647][105620] Updated weights for policy 1, policy_version 1446961 (0.0008) [2023-12-27 01:54:49,658][105692] Updated weights for policy 0, policy_version 1444571 (0.0005) [2023-12-27 01:54:49,705][105620] Updated weights for policy 1, policy_version 1446971 (0.0008) [2023-12-27 01:54:49,710][105692] Updated weights for policy 0, policy_version 1444581 (0.0005) [2023-12-27 01:54:49,764][105620] Updated weights for policy 1, policy_version 1446981 (0.0008) [2023-12-27 01:54:49,772][105692] Updated weights for policy 0, policy_version 1444591 (0.0005) [2023-12-27 01:54:50,456][105692] Updated weights for policy 0, policy_version 1444601 (0.0006) [2023-12-27 01:54:50,521][105692] Updated weights for policy 0, policy_version 1444611 (0.0005) [2023-12-27 01:54:50,588][105692] Updated weights for policy 0, policy_version 1444621 (0.0008) [2023-12-27 01:54:50,618][105620] Updated weights for policy 1, policy_version 1446991 (0.0009) [2023-12-27 01:54:50,687][105620] Updated weights for policy 1, policy_version 1447001 (0.0009) [2023-12-27 01:54:50,735][105586] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000009 [2023-12-27 01:54:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 740360192. Throughput: 0: 9789.6, 1: 9681.9. Samples: 740350404. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:54:51,063][104569] Avg episode reward: [(0, '8710.547'), (1, '8987.220')] [2023-12-27 01:54:51,260][105692] Updated weights for policy 0, policy_version 1444631 (0.0008) [2023-12-27 01:54:51,324][105692] Updated weights for policy 0, policy_version 1444641 (0.0008) [2023-12-27 01:54:51,395][105692] Updated weights for policy 0, policy_version 1444651 (0.0009) [2023-12-27 01:54:51,487][105620] Updated weights for policy 1, policy_version 1447011 (0.0009) [2023-12-27 01:54:51,549][105620] Updated weights for policy 1, policy_version 1447021 (0.0010) [2023-12-27 01:54:51,619][105620] Updated weights for policy 1, policy_version 1447031 (0.0010) [2023-12-27 01:54:52,051][105692] Updated weights for policy 0, policy_version 1444661 (0.0007) [2023-12-27 01:54:52,115][105692] Updated weights for policy 0, policy_version 1444671 (0.0008) [2023-12-27 01:54:52,166][105692] Updated weights for policy 0, policy_version 1444681 (0.0007) [2023-12-27 01:54:52,235][105620] Updated weights for policy 1, policy_version 1447041 (0.0008) [2023-12-27 01:54:52,303][105620] Updated weights for policy 1, policy_version 1447051 (0.0006) [2023-12-27 01:54:52,365][105620] Updated weights for policy 1, policy_version 1447061 (0.0008) [2023-12-27 01:54:52,428][105620] Updated weights for policy 1, policy_version 1447071 (0.0009) [2023-12-27 01:54:52,873][105692] Updated weights for policy 0, policy_version 1444691 (0.0008) [2023-12-27 01:54:52,936][105692] Updated weights for policy 0, policy_version 1444701 (0.0005) [2023-12-27 01:54:53,000][105692] Updated weights for policy 0, policy_version 1444711 (0.0007) [2023-12-27 01:54:53,070][105620] Updated weights for policy 1, policy_version 1447081 (0.0009) [2023-12-27 01:54:53,118][105620] Updated weights for policy 1, policy_version 1447091 (0.0009) [2023-12-27 01:54:53,163][105620] Updated weights for policy 1, policy_version 1447101 (0.0007) [2023-12-27 01:54:53,633][105692] Updated weights for policy 0, policy_version 1444721 (0.0008) [2023-12-27 01:54:53,702][105692] Updated weights for policy 0, policy_version 1444731 (0.0005) [2023-12-27 01:54:53,768][105620] Updated weights for policy 1, policy_version 1447111 (0.0006) [2023-12-27 01:54:53,770][105692] Updated weights for policy 0, policy_version 1444741 (0.0006) [2023-12-27 01:54:53,818][105620] Updated weights for policy 1, policy_version 1447121 (0.0006) [2023-12-27 01:54:53,838][105692] Updated weights for policy 0, policy_version 1444751 (0.0006) [2023-12-27 01:54:53,871][105620] Updated weights for policy 1, policy_version 1447131 (0.0009) [2023-12-27 01:54:54,410][105692] Updated weights for policy 0, policy_version 1444761 (0.0007) [2023-12-27 01:54:54,463][105692] Updated weights for policy 0, policy_version 1444771 (0.0010) [2023-12-27 01:54:54,499][105620] Updated weights for policy 1, policy_version 1447141 (0.0008) [2023-12-27 01:54:54,527][105692] Updated weights for policy 0, policy_version 1444781 (0.0009) [2023-12-27 01:54:54,546][105620] Updated weights for policy 1, policy_version 1447151 (0.0005) [2023-12-27 01:54:54,598][105620] Updated weights for policy 1, policy_version 1447161 (0.0005) [2023-12-27 01:54:55,179][105620] Updated weights for policy 1, policy_version 1447171 (0.0007) [2023-12-27 01:54:55,238][105620] Updated weights for policy 1, policy_version 1447181 (0.0011) [2023-12-27 01:54:55,274][105692] Updated weights for policy 0, policy_version 1444791 (0.0006) [2023-12-27 01:54:55,288][105620] Updated weights for policy 1, policy_version 1447191 (0.0009) [2023-12-27 01:54:55,335][105692] Updated weights for policy 0, policy_version 1444801 (0.0005) [2023-12-27 01:54:55,398][105692] Updated weights for policy 0, policy_version 1444811 (0.0005) [2023-12-27 01:54:55,899][105692] Updated weights for policy 0, policy_version 1444821 (0.0006) [2023-12-27 01:54:55,956][105692] Updated weights for policy 0, policy_version 1444831 (0.0005) [2023-12-27 01:54:56,001][105620] Updated weights for policy 1, policy_version 1447201 (0.0007) [2023-12-27 01:54:56,019][105692] Updated weights for policy 0, policy_version 1444841 (0.0006) [2023-12-27 01:54:56,050][105620] Updated weights for policy 1, policy_version 1447211 (0.0006) [2023-12-27 01:54:56,062][104569] Fps is (10 sec: 20479.4, 60 sec: 19797.2, 300 sec: 19438.6). Total num frames: 740466688. Throughput: 0: 9844.5, 1: 9873.4. Samples: 740474348. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:54:56,063][104569] Avg episode reward: [(0, '8436.622'), (1, '8533.199')] [2023-12-27 01:54:56,113][105620] Updated weights for policy 1, policy_version 1447221 (0.0005) [2023-12-27 01:54:56,179][105620] Updated weights for policy 1, policy_version 1447231 (0.0006) [2023-12-27 01:54:56,731][105692] Updated weights for policy 0, policy_version 1444851 (0.0008) [2023-12-27 01:54:56,747][105620] Updated weights for policy 1, policy_version 1447241 (0.0005) [2023-12-27 01:54:56,781][105692] Updated weights for policy 0, policy_version 1444861 (0.0007) [2023-12-27 01:54:56,799][105620] Updated weights for policy 1, policy_version 1447251 (0.0005) [2023-12-27 01:54:56,828][105692] Updated weights for policy 0, policy_version 1444871 (0.0007) [2023-12-27 01:54:56,842][105620] Updated weights for policy 1, policy_version 1447261 (0.0005) [2023-12-27 01:54:57,464][105692] Updated weights for policy 0, policy_version 1444881 (0.0008) [2023-12-27 01:54:57,520][105692] Updated weights for policy 0, policy_version 1444891 (0.0009) [2023-12-27 01:54:57,574][105620] Updated weights for policy 1, policy_version 1447271 (0.0006) [2023-12-27 01:54:57,586][105692] Updated weights for policy 0, policy_version 1444901 (0.0009) [2023-12-27 01:54:57,619][105620] Updated weights for policy 1, policy_version 1447281 (0.0006) [2023-12-27 01:54:57,641][105692] Updated weights for policy 0, policy_version 1444911 (0.0010) [2023-12-27 01:54:57,662][105620] Updated weights for policy 1, policy_version 1447291 (0.0005) [2023-12-27 01:54:58,234][105692] Updated weights for policy 0, policy_version 1444921 (0.0011) [2023-12-27 01:54:58,286][105692] Updated weights for policy 0, policy_version 1444931 (0.0010) [2023-12-27 01:54:58,349][105692] Updated weights for policy 0, policy_version 1444941 (0.0009) [2023-12-27 01:54:58,448][105620] Updated weights for policy 1, policy_version 1447301 (0.0007) [2023-12-27 01:54:58,522][105620] Updated weights for policy 1, policy_version 1447311 (0.0009) [2023-12-27 01:54:58,591][105620] Updated weights for policy 1, policy_version 1447321 (0.0008) [2023-12-27 01:54:59,128][105692] Updated weights for policy 0, policy_version 1444951 (0.0011) [2023-12-27 01:54:59,183][105692] Updated weights for policy 0, policy_version 1444961 (0.0010) [2023-12-27 01:54:59,256][105692] Updated weights for policy 0, policy_version 1444971 (0.0010) [2023-12-27 01:54:59,430][105620] Updated weights for policy 1, policy_version 1447331 (0.0008) [2023-12-27 01:54:59,491][105620] Updated weights for policy 1, policy_version 1447341 (0.0009) [2023-12-27 01:54:59,552][105620] Updated weights for policy 1, policy_version 1447351 (0.0009) [2023-12-27 01:54:59,978][105692] Updated weights for policy 0, policy_version 1444981 (0.0007) [2023-12-27 01:55:00,037][105692] Updated weights for policy 0, policy_version 1444991 (0.0006) [2023-12-27 01:55:00,091][105692] Updated weights for policy 0, policy_version 1445001 (0.0006) [2023-12-27 01:55:00,346][105620] Updated weights for policy 1, policy_version 1447361 (0.0009) [2023-12-27 01:55:00,408][105620] Updated weights for policy 1, policy_version 1447371 (0.0008) [2023-12-27 01:55:00,460][105620] Updated weights for policy 1, policy_version 1447381 (0.0008) [2023-12-27 01:55:00,507][105620] Updated weights for policy 1, policy_version 1447391 (0.0008) [2023-12-27 01:55:00,767][105692] Updated weights for policy 0, policy_version 1445011 (0.0008) [2023-12-27 01:55:00,811][105692] Updated weights for policy 0, policy_version 1445021 (0.0010) [2023-12-27 01:55:00,856][105692] Updated weights for policy 0, policy_version 1445031 (0.0005) [2023-12-27 01:55:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 740564992. Throughput: 0: 9906.3, 1: 9850.8. Samples: 740535348. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:55:01,062][104569] Avg episode reward: [(0, '8526.299'), (1, '8624.148')] [2023-12-27 01:55:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001445040_369983488.pth... [2023-12-27 01:55:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001443856_369680384.pth [2023-12-27 01:55:01,093][105620] Updated weights for policy 1, policy_version 1447401 (0.0008) [2023-12-27 01:55:01,153][105620] Updated weights for policy 1, policy_version 1447411 (0.0009) [2023-12-27 01:55:01,212][105620] Updated weights for policy 1, policy_version 1447421 (0.0010) [2023-12-27 01:55:01,225][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001447424_370589696.pth... [2023-12-27 01:55:01,230][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001446280_370294784.pth [2023-12-27 01:55:01,597][105692] Updated weights for policy 0, policy_version 1445041 (0.0008) [2023-12-27 01:55:01,664][105692] Updated weights for policy 0, policy_version 1445051 (0.0009) [2023-12-27 01:55:01,726][105692] Updated weights for policy 0, policy_version 1445061 (0.0010) [2023-12-27 01:55:01,790][105692] Updated weights for policy 0, policy_version 1445071 (0.0010) [2023-12-27 01:55:01,977][105620] Updated weights for policy 1, policy_version 1447431 (0.0008) [2023-12-27 01:55:02,036][105620] Updated weights for policy 1, policy_version 1447441 (0.0010) [2023-12-27 01:55:02,098][105620] Updated weights for policy 1, policy_version 1447451 (0.0011) [2023-12-27 01:55:02,538][105692] Updated weights for policy 0, policy_version 1445081 (0.0006) [2023-12-27 01:55:02,593][105692] Updated weights for policy 0, policy_version 1445091 (0.0009) [2023-12-27 01:55:02,646][105692] Updated weights for policy 0, policy_version 1445101 (0.0010) [2023-12-27 01:55:02,742][105620] Updated weights for policy 1, policy_version 1447461 (0.0008) [2023-12-27 01:55:02,805][105620] Updated weights for policy 1, policy_version 1447471 (0.0009) [2023-12-27 01:55:02,866][105620] Updated weights for policy 1, policy_version 1447481 (0.0006) [2023-12-27 01:55:03,398][105620] Updated weights for policy 1, policy_version 1447491 (0.0006) [2023-12-27 01:55:03,402][105692] Updated weights for policy 0, policy_version 1445111 (0.0008) [2023-12-27 01:55:03,453][105692] Updated weights for policy 0, policy_version 1445121 (0.0006) [2023-12-27 01:55:03,463][105620] Updated weights for policy 1, policy_version 1447501 (0.0008) [2023-12-27 01:55:03,512][105692] Updated weights for policy 0, policy_version 1445131 (0.0008) [2023-12-27 01:55:03,524][105620] Updated weights for policy 1, policy_version 1447511 (0.0008) [2023-12-27 01:55:04,162][105620] Updated weights for policy 1, policy_version 1447521 (0.0005) [2023-12-27 01:55:04,216][105692] Updated weights for policy 0, policy_version 1445141 (0.0009) [2023-12-27 01:55:04,225][105620] Updated weights for policy 1, policy_version 1447531 (0.0007) [2023-12-27 01:55:04,279][105692] Updated weights for policy 0, policy_version 1445151 (0.0011) [2023-12-27 01:55:04,284][105620] Updated weights for policy 1, policy_version 1447541 (0.0007) [2023-12-27 01:55:04,338][105692] Updated weights for policy 0, policy_version 1445161 (0.0011) [2023-12-27 01:55:04,349][105620] Updated weights for policy 1, policy_version 1447551 (0.0006) [2023-12-27 01:55:04,933][105692] Updated weights for policy 0, policy_version 1445171 (0.0009) [2023-12-27 01:55:04,988][105692] Updated weights for policy 0, policy_version 1445181 (0.0005) [2023-12-27 01:55:05,043][105692] Updated weights for policy 0, policy_version 1445191 (0.0006) [2023-12-27 01:55:05,165][105620] Updated weights for policy 1, policy_version 1447561 (0.0008) [2023-12-27 01:55:05,225][105620] Updated weights for policy 1, policy_version 1447571 (0.0007) [2023-12-27 01:55:05,277][105620] Updated weights for policy 1, policy_version 1447581 (0.0005) [2023-12-27 01:55:05,614][105692] Updated weights for policy 0, policy_version 1445201 (0.0006) [2023-12-27 01:55:05,666][105692] Updated weights for policy 0, policy_version 1445211 (0.0005) [2023-12-27 01:55:05,721][105692] Updated weights for policy 0, policy_version 1445221 (0.0006) [2023-12-27 01:55:05,775][105692] Updated weights for policy 0, policy_version 1445231 (0.0005) [2023-12-27 01:55:05,993][105620] Updated weights for policy 1, policy_version 1447591 (0.0009) [2023-12-27 01:55:06,054][105620] Updated weights for policy 1, policy_version 1447601 (0.0011) [2023-12-27 01:55:06,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 740663296. Throughput: 0: 9943.5, 1: 9828.8. Samples: 740653552. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:55:06,063][104569] Avg episode reward: [(0, '8798.397'), (1, '8785.957')] [2023-12-27 01:55:06,119][105620] Updated weights for policy 1, policy_version 1447611 (0.0011) [2023-12-27 01:55:06,375][105692] Updated weights for policy 0, policy_version 1445241 (0.0008) [2023-12-27 01:55:06,442][105692] Updated weights for policy 0, policy_version 1445251 (0.0010) [2023-12-27 01:55:06,501][105692] Updated weights for policy 0, policy_version 1445261 (0.0006) [2023-12-27 01:55:06,797][105620] Updated weights for policy 1, policy_version 1447621 (0.0009) [2023-12-27 01:55:06,859][105620] Updated weights for policy 1, policy_version 1447631 (0.0010) [2023-12-27 01:55:06,925][105620] Updated weights for policy 1, policy_version 1447641 (0.0010) [2023-12-27 01:55:07,216][105692] Updated weights for policy 0, policy_version 1445271 (0.0011) [2023-12-27 01:55:07,276][105692] Updated weights for policy 0, policy_version 1445281 (0.0011) [2023-12-27 01:55:07,336][105692] Updated weights for policy 0, policy_version 1445291 (0.0011) [2023-12-27 01:55:07,641][105620] Updated weights for policy 1, policy_version 1447651 (0.0008) [2023-12-27 01:55:07,703][105620] Updated weights for policy 1, policy_version 1447661 (0.0005) [2023-12-27 01:55:07,759][105620] Updated weights for policy 1, policy_version 1447671 (0.0007) [2023-12-27 01:55:07,978][105692] Updated weights for policy 0, policy_version 1445301 (0.0008) [2023-12-27 01:55:08,042][105692] Updated weights for policy 0, policy_version 1445311 (0.0007) [2023-12-27 01:55:08,104][105692] Updated weights for policy 0, policy_version 1445321 (0.0005) [2023-12-27 01:55:08,384][105620] Updated weights for policy 1, policy_version 1447681 (0.0008) [2023-12-27 01:55:08,445][105620] Updated weights for policy 1, policy_version 1447691 (0.0009) [2023-12-27 01:55:08,504][105620] Updated weights for policy 1, policy_version 1447701 (0.0010) [2023-12-27 01:55:08,563][105620] Updated weights for policy 1, policy_version 1447711 (0.0010) [2023-12-27 01:55:08,737][105692] Updated weights for policy 0, policy_version 1445331 (0.0008) [2023-12-27 01:55:08,784][105692] Updated weights for policy 0, policy_version 1445341 (0.0009) [2023-12-27 01:55:08,834][105692] Updated weights for policy 0, policy_version 1445351 (0.0009) [2023-12-27 01:55:09,303][105620] Updated weights for policy 1, policy_version 1447721 (0.0008) [2023-12-27 01:55:09,365][105620] Updated weights for policy 1, policy_version 1447731 (0.0008) [2023-12-27 01:55:09,432][105620] Updated weights for policy 1, policy_version 1447741 (0.0008) [2023-12-27 01:55:09,664][105692] Updated weights for policy 0, policy_version 1445361 (0.0008) [2023-12-27 01:55:09,726][105692] Updated weights for policy 0, policy_version 1445371 (0.0005) [2023-12-27 01:55:09,785][105692] Updated weights for policy 0, policy_version 1445381 (0.0005) [2023-12-27 01:55:09,845][105692] Updated weights for policy 0, policy_version 1445391 (0.0007) [2023-12-27 01:55:10,228][105620] Updated weights for policy 1, policy_version 1447751 (0.0006) [2023-12-27 01:55:10,287][105620] Updated weights for policy 1, policy_version 1447761 (0.0006) [2023-12-27 01:55:10,352][105620] Updated weights for policy 1, policy_version 1447771 (0.0008) [2023-12-27 01:55:10,586][105692] Updated weights for policy 0, policy_version 1445401 (0.0009) [2023-12-27 01:55:10,639][105692] Updated weights for policy 0, policy_version 1445411 (0.0009) [2023-12-27 01:55:10,690][105692] Updated weights for policy 0, policy_version 1445422 (0.0008) [2023-12-27 01:55:11,013][105620] Updated weights for policy 1, policy_version 1447781 (0.0010) [2023-12-27 01:55:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 740761600. Throughput: 0: 10085.1, 1: 9815.3. Samples: 740774052. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:55:11,063][104569] Avg episode reward: [(0, '8712.102'), (1, '8999.219')] [2023-12-27 01:55:11,073][105620] Updated weights for policy 1, policy_version 1447791 (0.0009) [2023-12-27 01:55:11,132][105620] Updated weights for policy 1, policy_version 1447801 (0.0006) [2023-12-27 01:55:11,373][105692] Updated weights for policy 0, policy_version 1445432 (0.0008) [2023-12-27 01:55:11,434][105692] Updated weights for policy 0, policy_version 1445442 (0.0011) [2023-12-27 01:55:11,498][105692] Updated weights for policy 0, policy_version 1445452 (0.0011) [2023-12-27 01:55:11,956][105620] Updated weights for policy 1, policy_version 1447811 (0.0007) [2023-12-27 01:55:12,018][105620] Updated weights for policy 1, policy_version 1447821 (0.0006) [2023-12-27 01:55:12,070][105620] Updated weights for policy 1, policy_version 1447831 (0.0006) [2023-12-27 01:55:12,182][105692] Updated weights for policy 0, policy_version 1445462 (0.0011) [2023-12-27 01:55:12,234][105692] Updated weights for policy 0, policy_version 1445472 (0.0010) [2023-12-27 01:55:12,286][105692] Updated weights for policy 0, policy_version 1445482 (0.0011) [2023-12-27 01:55:12,686][105620] Updated weights for policy 1, policy_version 1447841 (0.0005) [2023-12-27 01:55:12,757][105620] Updated weights for policy 1, policy_version 1447851 (0.0006) [2023-12-27 01:55:12,827][105620] Updated weights for policy 1, policy_version 1447861 (0.0007) [2023-12-27 01:55:12,891][105620] Updated weights for policy 1, policy_version 1447871 (0.0008) [2023-12-27 01:55:13,043][105692] Updated weights for policy 0, policy_version 1445492 (0.0008) [2023-12-27 01:55:13,095][105692] Updated weights for policy 0, policy_version 1445502 (0.0008) [2023-12-27 01:55:13,146][105692] Updated weights for policy 0, policy_version 1445512 (0.0010) [2023-12-27 01:55:13,565][105620] Updated weights for policy 1, policy_version 1447881 (0.0008) [2023-12-27 01:55:13,621][105620] Updated weights for policy 1, policy_version 1447891 (0.0009) [2023-12-27 01:55:13,681][105620] Updated weights for policy 1, policy_version 1447901 (0.0009) [2023-12-27 01:55:13,748][105692] Updated weights for policy 0, policy_version 1445522 (0.0011) [2023-12-27 01:55:13,810][105692] Updated weights for policy 0, policy_version 1445532 (0.0011) [2023-12-27 01:55:13,862][105692] Updated weights for policy 0, policy_version 1445542 (0.0011) [2023-12-27 01:55:13,922][105692] Updated weights for policy 0, policy_version 1445552 (0.0011) [2023-12-27 01:55:14,400][105620] Updated weights for policy 1, policy_version 1447911 (0.0009) [2023-12-27 01:55:14,460][105620] Updated weights for policy 1, policy_version 1447921 (0.0008) [2023-12-27 01:55:14,515][105620] Updated weights for policy 1, policy_version 1447931 (0.0008) [2023-12-27 01:55:14,660][105692] Updated weights for policy 0, policy_version 1445562 (0.0011) [2023-12-27 01:55:14,704][105692] Updated weights for policy 0, policy_version 1445572 (0.0010) [2023-12-27 01:55:14,748][105692] Updated weights for policy 0, policy_version 1445582 (0.0010) [2023-12-27 01:55:15,249][105620] Updated weights for policy 1, policy_version 1447941 (0.0009) [2023-12-27 01:55:15,301][105620] Updated weights for policy 1, policy_version 1447951 (0.0008) [2023-12-27 01:55:15,355][105620] Updated weights for policy 1, policy_version 1447961 (0.0007) [2023-12-27 01:55:15,531][105692] Updated weights for policy 0, policy_version 1445592 (0.0011) [2023-12-27 01:55:15,584][105692] Updated weights for policy 0, policy_version 1445602 (0.0011) [2023-12-27 01:55:15,637][105692] Updated weights for policy 0, policy_version 1445612 (0.0009) [2023-12-27 01:55:15,948][105620] Updated weights for policy 1, policy_version 1447971 (0.0006) [2023-12-27 01:55:16,000][105620] Updated weights for policy 1, policy_version 1447981 (0.0005) [2023-12-27 01:55:16,045][105620] Updated weights for policy 1, policy_version 1447991 (0.0005) [2023-12-27 01:55:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 740859904. Throughput: 0: 9965.7, 1: 9784.1. Samples: 740833184. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:55:16,063][104569] Avg episode reward: [(0, '9080.210'), (1, '8842.526')] [2023-12-27 01:55:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001445616_370130944.pth... [2023-12-27 01:55:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001444432_369827840.pth [2023-12-27 01:55:16,088][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001448000_370737152.pth... [2023-12-27 01:55:16,091][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001446824_370434048.pth [2023-12-27 01:55:16,239][105692] Updated weights for policy 0, policy_version 1445622 (0.0005) [2023-12-27 01:55:16,286][105692] Updated weights for policy 0, policy_version 1445632 (0.0006) [2023-12-27 01:55:16,340][105692] Updated weights for policy 0, policy_version 1445642 (0.0006) [2023-12-27 01:55:16,672][105620] Updated weights for policy 1, policy_version 1448001 (0.0006) [2023-12-27 01:55:16,719][105620] Updated weights for policy 1, policy_version 1448011 (0.0008) [2023-12-27 01:55:16,767][105620] Updated weights for policy 1, policy_version 1448021 (0.0008) [2023-12-27 01:55:16,821][105620] Updated weights for policy 1, policy_version 1448031 (0.0008) [2023-12-27 01:55:17,018][105692] Updated weights for policy 0, policy_version 1445652 (0.0009) [2023-12-27 01:55:17,064][105692] Updated weights for policy 0, policy_version 1445662 (0.0005) [2023-12-27 01:55:17,117][105692] Updated weights for policy 0, policy_version 1445672 (0.0005) [2023-12-27 01:55:17,667][105620] Updated weights for policy 1, policy_version 1448041 (0.0009) [2023-12-27 01:55:17,726][105620] Updated weights for policy 1, policy_version 1448051 (0.0008) [2023-12-27 01:55:17,784][105620] Updated weights for policy 1, policy_version 1448061 (0.0007) [2023-12-27 01:55:17,788][105692] Updated weights for policy 0, policy_version 1445682 (0.0006) [2023-12-27 01:55:17,848][105692] Updated weights for policy 0, policy_version 1445692 (0.0010) [2023-12-27 01:55:17,911][105692] Updated weights for policy 0, policy_version 1445702 (0.0010) [2023-12-27 01:55:17,959][105692] Updated weights for policy 0, policy_version 1445712 (0.0010) [2023-12-27 01:55:18,489][105620] Updated weights for policy 1, policy_version 1448071 (0.0010) [2023-12-27 01:55:18,552][105620] Updated weights for policy 1, policy_version 1448081 (0.0009) [2023-12-27 01:55:18,612][105620] Updated weights for policy 1, policy_version 1448091 (0.0008) [2023-12-27 01:55:18,733][105692] Updated weights for policy 0, policy_version 1445722 (0.0011) [2023-12-27 01:55:18,792][105692] Updated weights for policy 0, policy_version 1445732 (0.0011) [2023-12-27 01:55:18,847][105692] Updated weights for policy 0, policy_version 1445742 (0.0011) [2023-12-27 01:55:19,241][105620] Updated weights for policy 1, policy_version 1448101 (0.0010) [2023-12-27 01:55:19,303][105620] Updated weights for policy 1, policy_version 1448111 (0.0009) [2023-12-27 01:55:19,371][105620] Updated weights for policy 1, policy_version 1448121 (0.0009) [2023-12-27 01:55:19,608][105692] Updated weights for policy 0, policy_version 1445752 (0.0010) [2023-12-27 01:55:19,662][105692] Updated weights for policy 0, policy_version 1445762 (0.0010) [2023-12-27 01:55:19,713][105692] Updated weights for policy 0, policy_version 1445772 (0.0010) [2023-12-27 01:55:20,055][105620] Updated weights for policy 1, policy_version 1448131 (0.0009) [2023-12-27 01:55:20,120][105620] Updated weights for policy 1, policy_version 1448141 (0.0008) [2023-12-27 01:55:20,179][105620] Updated weights for policy 1, policy_version 1448151 (0.0010) [2023-12-27 01:55:20,585][105692] Updated weights for policy 0, policy_version 1445782 (0.0009) [2023-12-27 01:55:20,645][105692] Updated weights for policy 0, policy_version 1445792 (0.0007) [2023-12-27 01:55:20,711][105692] Updated weights for policy 0, policy_version 1445802 (0.0008) [2023-12-27 01:55:20,872][105620] Updated weights for policy 1, policy_version 1448161 (0.0008) [2023-12-27 01:55:20,939][105620] Updated weights for policy 1, policy_version 1448171 (0.0011) [2023-12-27 01:55:21,006][105620] Updated weights for policy 1, policy_version 1448181 (0.0011) [2023-12-27 01:55:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19466.4). Total num frames: 740958208. Throughput: 0: 9987.8, 1: 9808.0. Samples: 740953192. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:55:21,062][104569] Avg episode reward: [(0, '9170.069'), (1, '8806.944')] [2023-12-27 01:55:21,072][105620] Updated weights for policy 1, policy_version 1448191 (0.0010) [2023-12-27 01:55:21,514][105692] Updated weights for policy 0, policy_version 1445812 (0.0009) [2023-12-27 01:55:21,580][105692] Updated weights for policy 0, policy_version 1445822 (0.0010) [2023-12-27 01:55:21,642][105692] Updated weights for policy 0, policy_version 1445832 (0.0009) [2023-12-27 01:55:21,809][105620] Updated weights for policy 1, policy_version 1448201 (0.0009) [2023-12-27 01:55:21,874][105620] Updated weights for policy 1, policy_version 1448211 (0.0009) [2023-12-27 01:55:21,941][105620] Updated weights for policy 1, policy_version 1448221 (0.0006) [2023-12-27 01:55:22,402][105692] Updated weights for policy 0, policy_version 1445842 (0.0007) [2023-12-27 01:55:22,460][105692] Updated weights for policy 0, policy_version 1445852 (0.0010) [2023-12-27 01:55:22,518][105692] Updated weights for policy 0, policy_version 1445862 (0.0010) [2023-12-27 01:55:22,579][105692] Updated weights for policy 0, policy_version 1445872 (0.0008) [2023-12-27 01:55:22,612][105620] Updated weights for policy 1, policy_version 1448231 (0.0006) [2023-12-27 01:55:22,663][105620] Updated weights for policy 1, policy_version 1448241 (0.0008) [2023-12-27 01:55:22,718][105620] Updated weights for policy 1, policy_version 1448251 (0.0010) [2023-12-27 01:55:23,384][105692] Updated weights for policy 0, policy_version 1445882 (0.0010) [2023-12-27 01:55:23,445][105692] Updated weights for policy 0, policy_version 1445892 (0.0010) [2023-12-27 01:55:23,454][105620] Updated weights for policy 1, policy_version 1448261 (0.0008) [2023-12-27 01:55:23,507][105692] Updated weights for policy 0, policy_version 1445902 (0.0008) [2023-12-27 01:55:23,507][105620] Updated weights for policy 1, policy_version 1448271 (0.0006) [2023-12-27 01:55:23,563][105620] Updated weights for policy 1, policy_version 1448281 (0.0011) [2023-12-27 01:55:24,176][105620] Updated weights for policy 1, policy_version 1448291 (0.0009) [2023-12-27 01:55:24,228][105620] Updated weights for policy 1, policy_version 1448301 (0.0008) [2023-12-27 01:55:24,284][105620] Updated weights for policy 1, policy_version 1448311 (0.0010) [2023-12-27 01:55:24,328][105692] Updated weights for policy 0, policy_version 1445912 (0.0008) [2023-12-27 01:55:24,386][105692] Updated weights for policy 0, policy_version 1445922 (0.0009) [2023-12-27 01:55:24,445][105692] Updated weights for policy 0, policy_version 1445932 (0.0009) [2023-12-27 01:55:24,892][105620] Updated weights for policy 1, policy_version 1448321 (0.0010) [2023-12-27 01:55:24,957][105620] Updated weights for policy 1, policy_version 1448331 (0.0008) [2023-12-27 01:55:25,022][105620] Updated weights for policy 1, policy_version 1448341 (0.0008) [2023-12-27 01:55:25,095][105620] Updated weights for policy 1, policy_version 1448351 (0.0007) [2023-12-27 01:55:25,196][105692] Updated weights for policy 0, policy_version 1445942 (0.0007) [2023-12-27 01:55:25,259][105692] Updated weights for policy 0, policy_version 1445952 (0.0005) [2023-12-27 01:55:25,326][105692] Updated weights for policy 0, policy_version 1445962 (0.0005) [2023-12-27 01:55:25,636][105620] Updated weights for policy 1, policy_version 1448361 (0.0008) [2023-12-27 01:55:25,698][105620] Updated weights for policy 1, policy_version 1448371 (0.0009) [2023-12-27 01:55:25,768][105620] Updated weights for policy 1, policy_version 1448381 (0.0009) [2023-12-27 01:55:25,823][105692] Updated weights for policy 0, policy_version 1445972 (0.0006) [2023-12-27 01:55:25,889][105692] Updated weights for policy 0, policy_version 1445982 (0.0009) [2023-12-27 01:55:25,941][105692] Updated weights for policy 0, policy_version 1445992 (0.0007) [2023-12-27 01:55:26,062][104569] Fps is (10 sec: 20480.7, 60 sec: 19934.0, 300 sec: 19494.2). Total num frames: 741064704. Throughput: 0: 9865.3, 1: 9943.1. Samples: 741068372. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:55:26,062][104569] Avg episode reward: [(0, '8810.082'), (1, '8805.820')] [2023-12-27 01:55:26,502][105620] Updated weights for policy 1, policy_version 1448391 (0.0009) [2023-12-27 01:55:26,548][105620] Updated weights for policy 1, policy_version 1448401 (0.0008) [2023-12-27 01:55:26,602][105620] Updated weights for policy 1, policy_version 1448411 (0.0009) [2023-12-27 01:55:26,664][105692] Updated weights for policy 0, policy_version 1446002 (0.0009) [2023-12-27 01:55:26,725][105692] Updated weights for policy 0, policy_version 1446012 (0.0009) [2023-12-27 01:55:26,779][105692] Updated weights for policy 0, policy_version 1446022 (0.0009) [2023-12-27 01:55:26,838][105692] Updated weights for policy 0, policy_version 1446032 (0.0008) [2023-12-27 01:55:27,337][105620] Updated weights for policy 1, policy_version 1448421 (0.0009) [2023-12-27 01:55:27,388][105620] Updated weights for policy 1, policy_version 1448431 (0.0009) [2023-12-27 01:55:27,437][105620] Updated weights for policy 1, policy_version 1448441 (0.0008) [2023-12-27 01:55:27,586][105692] Updated weights for policy 0, policy_version 1446042 (0.0009) [2023-12-27 01:55:27,640][105692] Updated weights for policy 0, policy_version 1446052 (0.0009) [2023-12-27 01:55:27,691][105692] Updated weights for policy 0, policy_version 1446063 (0.0009) [2023-12-27 01:55:28,022][105620] Updated weights for policy 1, policy_version 1448451 (0.0007) [2023-12-27 01:55:28,072][105620] Updated weights for policy 1, policy_version 1448461 (0.0005) [2023-12-27 01:55:28,118][105620] Updated weights for policy 1, policy_version 1448471 (0.0005) [2023-12-27 01:55:28,596][105692] Updated weights for policy 0, policy_version 1446073 (0.0009) [2023-12-27 01:55:28,664][105692] Updated weights for policy 0, policy_version 1446083 (0.0008) [2023-12-27 01:55:28,725][105692] Updated weights for policy 0, policy_version 1446093 (0.0009) [2023-12-27 01:55:28,753][105620] Updated weights for policy 1, policy_version 1448481 (0.0006) [2023-12-27 01:55:28,808][105620] Updated weights for policy 1, policy_version 1448491 (0.0008) [2023-12-27 01:55:28,870][105620] Updated weights for policy 1, policy_version 1448501 (0.0009) [2023-12-27 01:55:28,929][105620] Updated weights for policy 1, policy_version 1448511 (0.0009) [2023-12-27 01:55:29,406][105692] Updated weights for policy 0, policy_version 1446103 (0.0008) [2023-12-27 01:55:29,467][105692] Updated weights for policy 0, policy_version 1446113 (0.0010) [2023-12-27 01:55:29,524][105692] Updated weights for policy 0, policy_version 1446123 (0.0009) [2023-12-27 01:55:29,671][105620] Updated weights for policy 1, policy_version 1448521 (0.0009) [2023-12-27 01:55:29,742][105620] Updated weights for policy 1, policy_version 1448531 (0.0010) [2023-12-27 01:55:29,808][105620] Updated weights for policy 1, policy_version 1448541 (0.0008) [2023-12-27 01:55:30,229][105692] Updated weights for policy 0, policy_version 1446133 (0.0008) [2023-12-27 01:55:30,276][105692] Updated weights for policy 0, policy_version 1446143 (0.0005) [2023-12-27 01:55:30,321][105692] Updated weights for policy 0, policy_version 1446153 (0.0005) [2023-12-27 01:55:30,584][105620] Updated weights for policy 1, policy_version 1448551 (0.0006) [2023-12-27 01:55:30,634][105620] Updated weights for policy 1, policy_version 1448561 (0.0005) [2023-12-27 01:55:30,678][105620] Updated weights for policy 1, policy_version 1448571 (0.0009) [2023-12-27 01:55:30,970][105692] Updated weights for policy 0, policy_version 1446163 (0.0007) [2023-12-27 01:55:31,024][105692] Updated weights for policy 0, policy_version 1446173 (0.0010) [2023-12-27 01:55:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 741154816. Throughput: 0: 9895.0, 1: 9961.7. Samples: 741127616. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:55:31,063][104569] Avg episode reward: [(0, '8806.648'), (1, '8805.309')] [2023-12-27 01:55:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001448576_370884608.pth... [2023-12-27 01:55:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001447424_370589696.pth [2023-12-27 01:55:31,079][105692] Updated weights for policy 0, policy_version 1446183 (0.0010) [2023-12-27 01:55:31,132][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001446192_370278400.pth... [2023-12-27 01:55:31,136][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001445040_369983488.pth [2023-12-27 01:55:31,364][105620] Updated weights for policy 1, policy_version 1448581 (0.0010) [2023-12-27 01:55:31,432][105620] Updated weights for policy 1, policy_version 1448591 (0.0011) [2023-12-27 01:55:31,496][105620] Updated weights for policy 1, policy_version 1448601 (0.0010) [2023-12-27 01:55:31,862][105692] Updated weights for policy 0, policy_version 1446193 (0.0010) [2023-12-27 01:55:31,917][105692] Updated weights for policy 0, policy_version 1446203 (0.0010) [2023-12-27 01:55:31,979][105692] Updated weights for policy 0, policy_version 1446213 (0.0010) [2023-12-27 01:55:32,038][105692] Updated weights for policy 0, policy_version 1446223 (0.0010) [2023-12-27 01:55:32,200][105620] Updated weights for policy 1, policy_version 1448611 (0.0011) [2023-12-27 01:55:32,262][105620] Updated weights for policy 1, policy_version 1448621 (0.0010) [2023-12-27 01:55:32,321][105620] Updated weights for policy 1, policy_version 1448631 (0.0011) [2023-12-27 01:55:32,716][105692] Updated weights for policy 0, policy_version 1446233 (0.0010) [2023-12-27 01:55:32,777][105692] Updated weights for policy 0, policy_version 1446243 (0.0010) [2023-12-27 01:55:32,829][105692] Updated weights for policy 0, policy_version 1446253 (0.0007) [2023-12-27 01:55:33,031][105620] Updated weights for policy 1, policy_version 1448641 (0.0011) [2023-12-27 01:55:33,085][105620] Updated weights for policy 1, policy_version 1448651 (0.0010) [2023-12-27 01:55:33,133][105620] Updated weights for policy 1, policy_version 1448661 (0.0010) [2023-12-27 01:55:33,187][105620] Updated weights for policy 1, policy_version 1448671 (0.0010) [2023-12-27 01:55:33,538][105692] Updated weights for policy 0, policy_version 1446263 (0.0010) [2023-12-27 01:55:33,585][105692] Updated weights for policy 0, policy_version 1446273 (0.0010) [2023-12-27 01:55:33,638][105692] Updated weights for policy 0, policy_version 1446283 (0.0010) [2023-12-27 01:55:33,948][105620] Updated weights for policy 1, policy_version 1448681 (0.0009) [2023-12-27 01:55:34,004][105620] Updated weights for policy 1, policy_version 1448691 (0.0009) [2023-12-27 01:55:34,065][105620] Updated weights for policy 1, policy_version 1448701 (0.0008) [2023-12-27 01:55:34,378][105692] Updated weights for policy 0, policy_version 1446293 (0.0007) [2023-12-27 01:55:34,429][105692] Updated weights for policy 0, policy_version 1446303 (0.0006) [2023-12-27 01:55:34,489][105692] Updated weights for policy 0, policy_version 1446313 (0.0007) [2023-12-27 01:55:34,778][105620] Updated weights for policy 1, policy_version 1448711 (0.0006) [2023-12-27 01:55:34,825][105620] Updated weights for policy 1, policy_version 1448721 (0.0005) [2023-12-27 01:55:34,874][105620] Updated weights for policy 1, policy_version 1448731 (0.0005) [2023-12-27 01:55:35,255][105692] Updated weights for policy 0, policy_version 1446323 (0.0007) [2023-12-27 01:55:35,308][105692] Updated weights for policy 0, policy_version 1446334 (0.0010) [2023-12-27 01:55:35,360][105692] Updated weights for policy 0, policy_version 1446345 (0.0010) [2023-12-27 01:55:35,405][105620] Updated weights for policy 1, policy_version 1448741 (0.0005) [2023-12-27 01:55:35,459][105620] Updated weights for policy 1, policy_version 1448751 (0.0005) [2023-12-27 01:55:35,513][105620] Updated weights for policy 1, policy_version 1448761 (0.0009) [2023-12-27 01:55:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 741253120. Throughput: 0: 9880.5, 1: 10000.2. Samples: 741245036. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:55:36,062][104569] Avg episode reward: [(0, '9166.107'), (1, '8896.698')] [2023-12-27 01:55:36,192][105692] Updated weights for policy 0, policy_version 1446355 (0.0009) [2023-12-27 01:55:36,236][105620] Updated weights for policy 1, policy_version 1448771 (0.0011) [2023-12-27 01:55:36,255][105692] Updated weights for policy 0, policy_version 1446365 (0.0009) [2023-12-27 01:55:36,299][105620] Updated weights for policy 1, policy_version 1448781 (0.0011) [2023-12-27 01:55:36,304][105692] Updated weights for policy 0, policy_version 1446375 (0.0010) [2023-12-27 01:55:36,358][105620] Updated weights for policy 1, policy_version 1448791 (0.0010) [2023-12-27 01:55:36,986][105692] Updated weights for policy 0, policy_version 1446385 (0.0010) [2023-12-27 01:55:37,056][105692] Updated weights for policy 0, policy_version 1446395 (0.0007) [2023-12-27 01:55:37,098][105620] Updated weights for policy 1, policy_version 1448801 (0.0010) [2023-12-27 01:55:37,120][105692] Updated weights for policy 0, policy_version 1446405 (0.0005) [2023-12-27 01:55:37,158][105620] Updated weights for policy 1, policy_version 1448811 (0.0007) [2023-12-27 01:55:37,185][105692] Updated weights for policy 0, policy_version 1446415 (0.0009) [2023-12-27 01:55:37,207][105620] Updated weights for policy 1, policy_version 1448821 (0.0010) [2023-12-27 01:55:37,262][105620] Updated weights for policy 1, policy_version 1448831 (0.0010) [2023-12-27 01:55:37,865][105692] Updated weights for policy 0, policy_version 1446425 (0.0011) [2023-12-27 01:55:37,920][105692] Updated weights for policy 0, policy_version 1446435 (0.0010) [2023-12-27 01:55:37,947][105620] Updated weights for policy 1, policy_version 1448841 (0.0007) [2023-12-27 01:55:37,972][105692] Updated weights for policy 0, policy_version 1446445 (0.0010) [2023-12-27 01:55:38,000][105620] Updated weights for policy 1, policy_version 1448851 (0.0006) [2023-12-27 01:55:38,059][105620] Updated weights for policy 1, policy_version 1448861 (0.0010) [2023-12-27 01:55:38,737][105692] Updated weights for policy 0, policy_version 1446455 (0.0011) [2023-12-27 01:55:38,797][105692] Updated weights for policy 0, policy_version 1446465 (0.0010) [2023-12-27 01:55:38,812][105620] Updated weights for policy 1, policy_version 1448871 (0.0010) [2023-12-27 01:55:38,859][105692] Updated weights for policy 0, policy_version 1446475 (0.0010) [2023-12-27 01:55:38,871][105620] Updated weights for policy 1, policy_version 1448881 (0.0010) [2023-12-27 01:55:38,922][105620] Updated weights for policy 1, policy_version 1448891 (0.0010) [2023-12-27 01:55:39,590][105692] Updated weights for policy 0, policy_version 1446485 (0.0008) [2023-12-27 01:55:39,655][105692] Updated weights for policy 0, policy_version 1446495 (0.0007) [2023-12-27 01:55:39,720][105692] Updated weights for policy 0, policy_version 1446505 (0.0011) [2023-12-27 01:55:39,763][105620] Updated weights for policy 1, policy_version 1448901 (0.0008) [2023-12-27 01:55:39,833][105620] Updated weights for policy 1, policy_version 1448911 (0.0010) [2023-12-27 01:55:39,893][105620] Updated weights for policy 1, policy_version 1448921 (0.0008) [2023-12-27 01:55:40,406][105692] Updated weights for policy 0, policy_version 1446515 (0.0011) [2023-12-27 01:55:40,468][105692] Updated weights for policy 0, policy_version 1446525 (0.0010) [2023-12-27 01:55:40,523][105692] Updated weights for policy 0, policy_version 1446535 (0.0010) [2023-12-27 01:55:40,667][105620] Updated weights for policy 1, policy_version 1448931 (0.0008) [2023-12-27 01:55:40,715][105620] Updated weights for policy 1, policy_version 1448941 (0.0008) [2023-12-27 01:55:40,767][105620] Updated weights for policy 1, policy_version 1448951 (0.0008) [2023-12-27 01:55:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19494.2). Total num frames: 741351424. Throughput: 0: 9776.7, 1: 9900.2. Samples: 741359804. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:55:41,062][104569] Avg episode reward: [(0, '8985.267'), (1, '8623.418')] [2023-12-27 01:55:41,262][105692] Updated weights for policy 0, policy_version 1446545 (0.0010) [2023-12-27 01:55:41,331][105692] Updated weights for policy 0, policy_version 1446555 (0.0006) [2023-12-27 01:55:41,399][105692] Updated weights for policy 0, policy_version 1446565 (0.0009) [2023-12-27 01:55:41,458][105692] Updated weights for policy 0, policy_version 1446575 (0.0007) [2023-12-27 01:55:41,482][105620] Updated weights for policy 1, policy_version 1448961 (0.0006) [2023-12-27 01:55:41,536][105620] Updated weights for policy 1, policy_version 1448971 (0.0009) [2023-12-27 01:55:41,598][105620] Updated weights for policy 1, policy_version 1448981 (0.0009) [2023-12-27 01:55:41,668][105620] Updated weights for policy 1, policy_version 1448991 (0.0008) [2023-12-27 01:55:42,234][105692] Updated weights for policy 0, policy_version 1446585 (0.0008) [2023-12-27 01:55:42,294][105692] Updated weights for policy 0, policy_version 1446595 (0.0009) [2023-12-27 01:55:42,316][105620] Updated weights for policy 1, policy_version 1449001 (0.0007) [2023-12-27 01:55:42,350][105692] Updated weights for policy 0, policy_version 1446605 (0.0008) [2023-12-27 01:55:42,377][105620] Updated weights for policy 1, policy_version 1449011 (0.0009) [2023-12-27 01:55:42,444][105620] Updated weights for policy 1, policy_version 1449021 (0.0009) [2023-12-27 01:55:43,110][105692] Updated weights for policy 0, policy_version 1446615 (0.0008) [2023-12-27 01:55:43,163][105692] Updated weights for policy 0, policy_version 1446625 (0.0008) [2023-12-27 01:55:43,194][105620] Updated weights for policy 1, policy_version 1449031 (0.0007) [2023-12-27 01:55:43,216][105692] Updated weights for policy 0, policy_version 1446635 (0.0006) [2023-12-27 01:55:43,246][105620] Updated weights for policy 1, policy_version 1449041 (0.0008) [2023-12-27 01:55:43,295][105620] Updated weights for policy 1, policy_version 1449051 (0.0009) [2023-12-27 01:55:43,879][105692] Updated weights for policy 0, policy_version 1446645 (0.0006) [2023-12-27 01:55:43,929][105692] Updated weights for policy 0, policy_version 1446655 (0.0005) [2023-12-27 01:55:43,985][105692] Updated weights for policy 0, policy_version 1446665 (0.0008) [2023-12-27 01:55:44,091][105620] Updated weights for policy 1, policy_version 1449061 (0.0009) [2023-12-27 01:55:44,152][105620] Updated weights for policy 1, policy_version 1449071 (0.0009) [2023-12-27 01:55:44,209][105620] Updated weights for policy 1, policy_version 1449081 (0.0008) [2023-12-27 01:55:44,644][105692] Updated weights for policy 0, policy_version 1446675 (0.0008) [2023-12-27 01:55:44,705][105692] Updated weights for policy 0, policy_version 1446685 (0.0008) [2023-12-27 01:55:44,766][105692] Updated weights for policy 0, policy_version 1446695 (0.0006) [2023-12-27 01:55:44,994][105620] Updated weights for policy 1, policy_version 1449091 (0.0008) [2023-12-27 01:55:45,059][105620] Updated weights for policy 1, policy_version 1449101 (0.0009) [2023-12-27 01:55:45,132][105620] Updated weights for policy 1, policy_version 1449111 (0.0009) [2023-12-27 01:55:45,492][105692] Updated weights for policy 0, policy_version 1446705 (0.0009) [2023-12-27 01:55:45,555][105692] Updated weights for policy 0, policy_version 1446715 (0.0009) [2023-12-27 01:55:45,616][105692] Updated weights for policy 0, policy_version 1446725 (0.0009) [2023-12-27 01:55:45,669][105692] Updated weights for policy 0, policy_version 1446735 (0.0009) [2023-12-27 01:55:45,903][105620] Updated weights for policy 1, policy_version 1449121 (0.0008) [2023-12-27 01:55:45,961][105620] Updated weights for policy 1, policy_version 1449131 (0.0010) [2023-12-27 01:55:46,017][105620] Updated weights for policy 1, policy_version 1449141 (0.0008) [2023-12-27 01:55:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 741441536. Throughput: 0: 9681.9, 1: 9895.2. Samples: 741416320. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:55:46,063][104569] Avg episode reward: [(0, '8714.582'), (1, '8621.659')] [2023-12-27 01:55:46,063][105620] Updated weights for policy 1, policy_version 1449151 (0.0009) [2023-12-27 01:55:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001449152_371032064.pth... [2023-12-27 01:55:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001446736_370417664.pth... [2023-12-27 01:55:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001448000_370737152.pth [2023-12-27 01:55:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001445616_370130944.pth [2023-12-27 01:55:46,291][105692] Updated weights for policy 0, policy_version 1446745 (0.0010) [2023-12-27 01:55:46,352][105692] Updated weights for policy 0, policy_version 1446755 (0.0010) [2023-12-27 01:55:46,417][105692] Updated weights for policy 0, policy_version 1446765 (0.0011) [2023-12-27 01:55:46,867][105620] Updated weights for policy 1, policy_version 1449161 (0.0008) [2023-12-27 01:55:46,915][105620] Updated weights for policy 1, policy_version 1449171 (0.0008) [2023-12-27 01:55:46,970][105620] Updated weights for policy 1, policy_version 1449181 (0.0008) [2023-12-27 01:55:47,067][105692] Updated weights for policy 0, policy_version 1446775 (0.0011) [2023-12-27 01:55:47,125][105692] Updated weights for policy 0, policy_version 1446785 (0.0010) [2023-12-27 01:55:47,182][105692] Updated weights for policy 0, policy_version 1446795 (0.0010) [2023-12-27 01:55:47,778][105692] Updated weights for policy 0, policy_version 1446805 (0.0010) [2023-12-27 01:55:47,798][105620] Updated weights for policy 1, policy_version 1449191 (0.0007) [2023-12-27 01:55:47,829][105692] Updated weights for policy 0, policy_version 1446815 (0.0006) [2023-12-27 01:55:47,856][105620] Updated weights for policy 1, policy_version 1449201 (0.0009) [2023-12-27 01:55:47,884][105692] Updated weights for policy 0, policy_version 1446825 (0.0005) [2023-12-27 01:55:47,910][105620] Updated weights for policy 1, policy_version 1449211 (0.0007) [2023-12-27 01:55:48,566][105692] Updated weights for policy 0, policy_version 1446835 (0.0008) [2023-12-27 01:55:48,619][105692] Updated weights for policy 0, policy_version 1446845 (0.0009) [2023-12-27 01:55:48,677][105692] Updated weights for policy 0, policy_version 1446855 (0.0009) [2023-12-27 01:55:48,718][105620] Updated weights for policy 1, policy_version 1449221 (0.0007) [2023-12-27 01:55:48,772][105620] Updated weights for policy 1, policy_version 1449231 (0.0007) [2023-12-27 01:55:48,834][105620] Updated weights for policy 1, policy_version 1449241 (0.0009) [2023-12-27 01:55:49,450][105692] Updated weights for policy 0, policy_version 1446865 (0.0007) [2023-12-27 01:55:49,501][105692] Updated weights for policy 0, policy_version 1446875 (0.0009) [2023-12-27 01:55:49,549][105692] Updated weights for policy 0, policy_version 1446885 (0.0009) [2023-12-27 01:55:49,600][105620] Updated weights for policy 1, policy_version 1449251 (0.0008) [2023-12-27 01:55:49,607][105692] Updated weights for policy 0, policy_version 1446895 (0.0008) [2023-12-27 01:55:49,660][105620] Updated weights for policy 1, policy_version 1449261 (0.0006) [2023-12-27 01:55:49,725][105620] Updated weights for policy 1, policy_version 1449271 (0.0010) [2023-12-27 01:55:50,329][105692] Updated weights for policy 0, policy_version 1446905 (0.0008) [2023-12-27 01:55:50,391][105692] Updated weights for policy 0, policy_version 1446915 (0.0010) [2023-12-27 01:55:50,446][105692] Updated weights for policy 0, policy_version 1446925 (0.0010) [2023-12-27 01:55:50,563][105620] Updated weights for policy 1, policy_version 1449281 (0.0009) [2023-12-27 01:55:50,619][105620] Updated weights for policy 1, policy_version 1449291 (0.0009) [2023-12-27 01:55:50,670][105620] Updated weights for policy 1, policy_version 1449301 (0.0009) [2023-12-27 01:55:50,733][105620] Updated weights for policy 1, policy_version 1449311 (0.0009) [2023-12-27 01:55:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 741539840. Throughput: 0: 9765.2, 1: 9743.9. Samples: 741531464. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:55:51,063][104569] Avg episode reward: [(0, '8619.610'), (1, '8714.211')] [2023-12-27 01:55:51,206][105692] Updated weights for policy 0, policy_version 1446935 (0.0007) [2023-12-27 01:55:51,265][105692] Updated weights for policy 0, policy_version 1446945 (0.0006) [2023-12-27 01:55:51,334][105692] Updated weights for policy 0, policy_version 1446955 (0.0006) [2023-12-27 01:55:51,414][105620] Updated weights for policy 1, policy_version 1449321 (0.0008) [2023-12-27 01:55:51,470][105620] Updated weights for policy 1, policy_version 1449331 (0.0008) [2023-12-27 01:55:51,529][105620] Updated weights for policy 1, policy_version 1449341 (0.0008) [2023-12-27 01:55:52,011][105692] Updated weights for policy 0, policy_version 1446965 (0.0010) [2023-12-27 01:55:52,070][105692] Updated weights for policy 0, policy_version 1446975 (0.0011) [2023-12-27 01:55:52,120][105692] Updated weights for policy 0, policy_version 1446985 (0.0011) [2023-12-27 01:55:52,315][105620] Updated weights for policy 1, policy_version 1449351 (0.0009) [2023-12-27 01:55:52,376][105620] Updated weights for policy 1, policy_version 1449361 (0.0008) [2023-12-27 01:55:52,441][105620] Updated weights for policy 1, policy_version 1449371 (0.0008) [2023-12-27 01:55:52,786][105692] Updated weights for policy 0, policy_version 1446995 (0.0010) [2023-12-27 01:55:52,844][105692] Updated weights for policy 0, policy_version 1447005 (0.0009) [2023-12-27 01:55:52,906][105692] Updated weights for policy 0, policy_version 1447015 (0.0006) [2023-12-27 01:55:53,315][105620] Updated weights for policy 1, policy_version 1449381 (0.0009) [2023-12-27 01:55:53,384][105620] Updated weights for policy 1, policy_version 1449391 (0.0009) [2023-12-27 01:55:53,445][105620] Updated weights for policy 1, policy_version 1449401 (0.0008) [2023-12-27 01:55:53,497][105692] Updated weights for policy 0, policy_version 1447025 (0.0006) [2023-12-27 01:55:53,553][105692] Updated weights for policy 0, policy_version 1447035 (0.0005) [2023-12-27 01:55:53,607][105692] Updated weights for policy 0, policy_version 1447045 (0.0005) [2023-12-27 01:55:53,652][105692] Updated weights for policy 0, policy_version 1447055 (0.0005) [2023-12-27 01:55:54,173][105692] Updated weights for policy 0, policy_version 1447065 (0.0010) [2023-12-27 01:55:54,178][105620] Updated weights for policy 1, policy_version 1449411 (0.0009) [2023-12-27 01:55:54,224][105692] Updated weights for policy 0, policy_version 1447075 (0.0010) [2023-12-27 01:55:54,235][105620] Updated weights for policy 1, policy_version 1449421 (0.0008) [2023-12-27 01:55:54,279][105692] Updated weights for policy 0, policy_version 1447085 (0.0010) [2023-12-27 01:55:54,285][105585] KL-divergence is very high: 111.0561 [2023-12-27 01:55:54,294][105620] Updated weights for policy 1, policy_version 1449431 (0.0007) [2023-12-27 01:55:54,963][105692] Updated weights for policy 0, policy_version 1447095 (0.0011) [2023-12-27 01:55:54,995][105620] Updated weights for policy 1, policy_version 1449441 (0.0008) [2023-12-27 01:55:55,025][105692] Updated weights for policy 0, policy_version 1447105 (0.0011) [2023-12-27 01:55:55,062][105620] Updated weights for policy 1, policy_version 1449451 (0.0008) [2023-12-27 01:55:55,082][105692] Updated weights for policy 0, policy_version 1447115 (0.0011) [2023-12-27 01:55:55,131][105620] Updated weights for policy 1, policy_version 1449461 (0.0009) [2023-12-27 01:55:55,201][105620] Updated weights for policy 1, policy_version 1449471 (0.0007) [2023-12-27 01:55:55,701][105692] Updated weights for policy 0, policy_version 1447125 (0.0008) [2023-12-27 01:55:55,755][105620] Updated weights for policy 1, policy_version 1449481 (0.0006) [2023-12-27 01:55:55,758][105692] Updated weights for policy 0, policy_version 1447135 (0.0005) [2023-12-27 01:55:55,810][105692] Updated weights for policy 0, policy_version 1447145 (0.0005) [2023-12-27 01:55:55,820][105620] Updated weights for policy 1, policy_version 1449491 (0.0006) [2023-12-27 01:55:55,874][105620] Updated weights for policy 1, policy_version 1449501 (0.0010) [2023-12-27 01:55:56,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.9, 300 sec: 19522.0). Total num frames: 741646336. Throughput: 0: 9779.4, 1: 9717.1. Samples: 741651392. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:55:56,062][104569] Avg episode reward: [(0, '8797.751'), (1, '8993.892')] [2023-12-27 01:55:56,426][105692] Updated weights for policy 0, policy_version 1447155 (0.0006) [2023-12-27 01:55:56,493][105692] Updated weights for policy 0, policy_version 1447165 (0.0010) [2023-12-27 01:55:56,494][105620] Updated weights for policy 1, policy_version 1449511 (0.0007) [2023-12-27 01:55:56,545][105620] Updated weights for policy 1, policy_version 1449521 (0.0009) [2023-12-27 01:55:56,547][105692] Updated weights for policy 0, policy_version 1447175 (0.0009) [2023-12-27 01:55:56,592][105620] Updated weights for policy 1, policy_version 1449531 (0.0005) [2023-12-27 01:55:57,174][105620] Updated weights for policy 1, policy_version 1449541 (0.0005) [2023-12-27 01:55:57,217][105620] Updated weights for policy 1, policy_version 1449551 (0.0005) [2023-12-27 01:55:57,246][105692] Updated weights for policy 0, policy_version 1447185 (0.0008) [2023-12-27 01:55:57,275][105620] Updated weights for policy 1, policy_version 1449561 (0.0005) [2023-12-27 01:55:57,307][105692] Updated weights for policy 0, policy_version 1447195 (0.0005) [2023-12-27 01:55:57,363][105692] Updated weights for policy 0, policy_version 1447205 (0.0008) [2023-12-27 01:55:57,427][105692] Updated weights for policy 0, policy_version 1447215 (0.0010) [2023-12-27 01:55:57,863][105620] Updated weights for policy 1, policy_version 1449571 (0.0005) [2023-12-27 01:55:57,925][105620] Updated weights for policy 1, policy_version 1449581 (0.0005) [2023-12-27 01:55:57,979][105620] Updated weights for policy 1, policy_version 1449591 (0.0005) [2023-12-27 01:55:58,092][105692] Updated weights for policy 0, policy_version 1447225 (0.0006) [2023-12-27 01:55:58,159][105692] Updated weights for policy 0, policy_version 1447235 (0.0006) [2023-12-27 01:55:58,216][105692] Updated weights for policy 0, policy_version 1447245 (0.0008) [2023-12-27 01:55:58,665][105620] Updated weights for policy 1, policy_version 1449601 (0.0007) [2023-12-27 01:55:58,732][105620] Updated weights for policy 1, policy_version 1449611 (0.0009) [2023-12-27 01:55:58,808][105620] Updated weights for policy 1, policy_version 1449621 (0.0010) [2023-12-27 01:55:58,865][105620] Updated weights for policy 1, policy_version 1449631 (0.0007) [2023-12-27 01:55:59,042][105692] Updated weights for policy 0, policy_version 1447255 (0.0010) [2023-12-27 01:55:59,106][105692] Updated weights for policy 0, policy_version 1447265 (0.0009) [2023-12-27 01:55:59,166][105692] Updated weights for policy 0, policy_version 1447275 (0.0008) [2023-12-27 01:55:59,544][105620] Updated weights for policy 1, policy_version 1449641 (0.0010) [2023-12-27 01:55:59,594][105620] Updated weights for policy 1, policy_version 1449651 (0.0011) [2023-12-27 01:55:59,639][105620] Updated weights for policy 1, policy_version 1449661 (0.0011) [2023-12-27 01:55:59,944][105692] Updated weights for policy 0, policy_version 1447285 (0.0008) [2023-12-27 01:55:59,998][105692] Updated weights for policy 0, policy_version 1447295 (0.0008) [2023-12-27 01:56:00,051][105692] Updated weights for policy 0, policy_version 1447305 (0.0008) [2023-12-27 01:56:00,379][105620] Updated weights for policy 1, policy_version 1449671 (0.0006) [2023-12-27 01:56:00,437][105620] Updated weights for policy 1, policy_version 1449681 (0.0006) [2023-12-27 01:56:00,499][105620] Updated weights for policy 1, policy_version 1449691 (0.0009) [2023-12-27 01:56:00,832][105692] Updated weights for policy 0, policy_version 1447315 (0.0009) [2023-12-27 01:56:00,886][105692] Updated weights for policy 0, policy_version 1447325 (0.0006) [2023-12-27 01:56:00,955][105692] Updated weights for policy 0, policy_version 1447335 (0.0010) [2023-12-27 01:56:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 741744640. Throughput: 0: 9804.8, 1: 9799.5. Samples: 741715372. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:56:01,063][104569] Avg episode reward: [(0, '8890.150'), (1, '9082.803')] [2023-12-27 01:56:01,063][105620] Updated weights for policy 1, policy_version 1449701 (0.0010) [2023-12-27 01:56:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001447344_370573312.pth... [2023-12-27 01:56:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001446192_370278400.pth [2023-12-27 01:56:01,133][105620] Updated weights for policy 1, policy_version 1449711 (0.0010) [2023-12-27 01:56:01,195][105620] Updated weights for policy 1, policy_version 1449721 (0.0010) [2023-12-27 01:56:01,242][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001449728_371179520.pth... [2023-12-27 01:56:01,248][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001448576_370884608.pth [2023-12-27 01:56:01,601][105692] Updated weights for policy 0, policy_version 1447345 (0.0010) [2023-12-27 01:56:01,668][105692] Updated weights for policy 0, policy_version 1447355 (0.0008) [2023-12-27 01:56:01,733][105692] Updated weights for policy 0, policy_version 1447365 (0.0007) [2023-12-27 01:56:01,789][105692] Updated weights for policy 0, policy_version 1447375 (0.0010) [2023-12-27 01:56:01,915][105620] Updated weights for policy 1, policy_version 1449731 (0.0007) [2023-12-27 01:56:01,988][105620] Updated weights for policy 1, policy_version 1449741 (0.0011) [2023-12-27 01:56:02,047][105620] Updated weights for policy 1, policy_version 1449751 (0.0011) [2023-12-27 01:56:02,367][105692] Updated weights for policy 0, policy_version 1447385 (0.0008) [2023-12-27 01:56:02,419][105692] Updated weights for policy 0, policy_version 1447395 (0.0010) [2023-12-27 01:56:02,475][105692] Updated weights for policy 0, policy_version 1447405 (0.0010) [2023-12-27 01:56:02,789][105620] Updated weights for policy 1, policy_version 1449761 (0.0010) [2023-12-27 01:56:02,848][105620] Updated weights for policy 1, policy_version 1449771 (0.0011) [2023-12-27 01:56:02,899][105620] Updated weights for policy 1, policy_version 1449781 (0.0010) [2023-12-27 01:56:02,951][105620] Updated weights for policy 1, policy_version 1449791 (0.0010) [2023-12-27 01:56:03,139][105692] Updated weights for policy 0, policy_version 1447415 (0.0010) [2023-12-27 01:56:03,194][105692] Updated weights for policy 0, policy_version 1447425 (0.0010) [2023-12-27 01:56:03,244][105692] Updated weights for policy 0, policy_version 1447435 (0.0010) [2023-12-27 01:56:03,683][105620] Updated weights for policy 1, policy_version 1449801 (0.0006) [2023-12-27 01:56:03,731][105620] Updated weights for policy 1, policy_version 1449811 (0.0005) [2023-12-27 01:56:03,787][105620] Updated weights for policy 1, policy_version 1449821 (0.0009) [2023-12-27 01:56:04,002][105692] Updated weights for policy 0, policy_version 1447445 (0.0010) [2023-12-27 01:56:04,057][105692] Updated weights for policy 0, policy_version 1447455 (0.0008) [2023-12-27 01:56:04,114][105692] Updated weights for policy 0, policy_version 1447465 (0.0008) [2023-12-27 01:56:04,535][105620] Updated weights for policy 1, policy_version 1449831 (0.0009) [2023-12-27 01:56:04,597][105620] Updated weights for policy 1, policy_version 1449841 (0.0009) [2023-12-27 01:56:04,654][105620] Updated weights for policy 1, policy_version 1449851 (0.0008) [2023-12-27 01:56:04,861][105692] Updated weights for policy 0, policy_version 1447475 (0.0007) [2023-12-27 01:56:04,918][105692] Updated weights for policy 0, policy_version 1447485 (0.0007) [2023-12-27 01:56:04,973][105692] Updated weights for policy 0, policy_version 1447495 (0.0009) [2023-12-27 01:56:05,367][105620] Updated weights for policy 1, policy_version 1449861 (0.0009) [2023-12-27 01:56:05,423][105620] Updated weights for policy 1, policy_version 1449871 (0.0008) [2023-12-27 01:56:05,469][105620] Updated weights for policy 1, policy_version 1449881 (0.0008) [2023-12-27 01:56:05,718][105692] Updated weights for policy 0, policy_version 1447505 (0.0010) [2023-12-27 01:56:05,780][105692] Updated weights for policy 0, policy_version 1447515 (0.0009) [2023-12-27 01:56:05,842][105692] Updated weights for policy 0, policy_version 1447525 (0.0010) [2023-12-27 01:56:05,902][105692] Updated weights for policy 0, policy_version 1447535 (0.0009) [2023-12-27 01:56:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 741842944. Throughput: 0: 9755.2, 1: 9773.0. Samples: 741831964. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:56:06,062][104569] Avg episode reward: [(0, '8709.499'), (1, '9172.047')] [2023-12-27 01:56:06,222][105620] Updated weights for policy 1, policy_version 1449891 (0.0009) [2023-12-27 01:56:06,284][105620] Updated weights for policy 1, policy_version 1449901 (0.0009) [2023-12-27 01:56:06,339][105620] Updated weights for policy 1, policy_version 1449911 (0.0008) [2023-12-27 01:56:06,680][105692] Updated weights for policy 0, policy_version 1447545 (0.0009) [2023-12-27 01:56:06,742][105692] Updated weights for policy 0, policy_version 1447555 (0.0008) [2023-12-27 01:56:06,799][105692] Updated weights for policy 0, policy_version 1447565 (0.0008) [2023-12-27 01:56:07,117][105620] Updated weights for policy 1, policy_version 1449921 (0.0009) [2023-12-27 01:56:07,172][105620] Updated weights for policy 1, policy_version 1449931 (0.0010) [2023-12-27 01:56:07,238][105620] Updated weights for policy 1, policy_version 1449941 (0.0010) [2023-12-27 01:56:07,290][105620] Updated weights for policy 1, policy_version 1449951 (0.0010) [2023-12-27 01:56:07,586][105692] Updated weights for policy 0, policy_version 1447575 (0.0008) [2023-12-27 01:56:07,631][105692] Updated weights for policy 0, policy_version 1447585 (0.0008) [2023-12-27 01:56:07,679][105692] Updated weights for policy 0, policy_version 1447595 (0.0008) [2023-12-27 01:56:08,038][105620] Updated weights for policy 1, policy_version 1449961 (0.0010) [2023-12-27 01:56:08,096][105620] Updated weights for policy 1, policy_version 1449971 (0.0010) [2023-12-27 01:56:08,153][105620] Updated weights for policy 1, policy_version 1449981 (0.0010) [2023-12-27 01:56:08,489][105692] Updated weights for policy 0, policy_version 1447605 (0.0008) [2023-12-27 01:56:08,553][105692] Updated weights for policy 0, policy_version 1447615 (0.0008) [2023-12-27 01:56:08,613][105692] Updated weights for policy 0, policy_version 1447625 (0.0008) [2023-12-27 01:56:08,913][105620] Updated weights for policy 1, policy_version 1449991 (0.0011) [2023-12-27 01:56:08,975][105620] Updated weights for policy 1, policy_version 1450001 (0.0011) [2023-12-27 01:56:09,028][105620] Updated weights for policy 1, policy_version 1450011 (0.0010) [2023-12-27 01:56:09,382][105692] Updated weights for policy 0, policy_version 1447635 (0.0009) [2023-12-27 01:56:09,444][105692] Updated weights for policy 0, policy_version 1447645 (0.0008) [2023-12-27 01:56:09,502][105692] Updated weights for policy 0, policy_version 1447655 (0.0008) [2023-12-27 01:56:09,813][105620] Updated weights for policy 1, policy_version 1450021 (0.0010) [2023-12-27 01:56:09,879][105620] Updated weights for policy 1, policy_version 1450031 (0.0011) [2023-12-27 01:56:09,945][105620] Updated weights for policy 1, policy_version 1450041 (0.0011) [2023-12-27 01:56:10,341][105692] Updated weights for policy 0, policy_version 1447665 (0.0008) [2023-12-27 01:56:10,402][105692] Updated weights for policy 0, policy_version 1447675 (0.0008) [2023-12-27 01:56:10,454][105692] Updated weights for policy 0, policy_version 1447685 (0.0007) [2023-12-27 01:56:10,503][105692] Updated weights for policy 0, policy_version 1447695 (0.0008) [2023-12-27 01:56:10,700][105620] Updated weights for policy 1, policy_version 1450051 (0.0010) [2023-12-27 01:56:10,760][105620] Updated weights for policy 1, policy_version 1450061 (0.0011) [2023-12-27 01:56:10,813][105620] Updated weights for policy 1, policy_version 1450071 (0.0011) [2023-12-27 01:56:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 741933056. Throughput: 0: 9755.1, 1: 9641.7. Samples: 741941228. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:56:11,062][104569] Avg episode reward: [(0, '8710.396'), (1, '9172.276')] [2023-12-27 01:56:11,333][105692] Updated weights for policy 0, policy_version 1447705 (0.0009) [2023-12-27 01:56:11,402][105692] Updated weights for policy 0, policy_version 1447715 (0.0008) [2023-12-27 01:56:11,448][105692] Updated weights for policy 0, policy_version 1447725 (0.0008) [2023-12-27 01:56:11,600][105620] Updated weights for policy 1, policy_version 1450081 (0.0011) [2023-12-27 01:56:11,665][105620] Updated weights for policy 1, policy_version 1450091 (0.0011) [2023-12-27 01:56:11,715][105620] Updated weights for policy 1, policy_version 1450101 (0.0011) [2023-12-27 01:56:11,780][105620] Updated weights for policy 1, policy_version 1450111 (0.0010) [2023-12-27 01:56:12,265][105692] Updated weights for policy 0, policy_version 1447735 (0.0009) [2023-12-27 01:56:12,331][105692] Updated weights for policy 0, policy_version 1447745 (0.0010) [2023-12-27 01:56:12,401][105692] Updated weights for policy 0, policy_version 1447755 (0.0011) [2023-12-27 01:56:12,547][105620] Updated weights for policy 1, policy_version 1450121 (0.0008) [2023-12-27 01:56:12,612][105620] Updated weights for policy 1, policy_version 1450131 (0.0005) [2023-12-27 01:56:12,679][105620] Updated weights for policy 1, policy_version 1450141 (0.0006) [2023-12-27 01:56:13,048][105692] Updated weights for policy 0, policy_version 1447765 (0.0010) [2023-12-27 01:56:13,099][105692] Updated weights for policy 0, policy_version 1447775 (0.0008) [2023-12-27 01:56:13,151][105692] Updated weights for policy 0, policy_version 1447785 (0.0006) [2023-12-27 01:56:13,355][105620] Updated weights for policy 1, policy_version 1450151 (0.0008) [2023-12-27 01:56:13,412][105620] Updated weights for policy 1, policy_version 1450161 (0.0009) [2023-12-27 01:56:13,465][105620] Updated weights for policy 1, policy_version 1450171 (0.0009) [2023-12-27 01:56:13,779][105692] Updated weights for policy 0, policy_version 1447795 (0.0005) [2023-12-27 01:56:13,831][105692] Updated weights for policy 0, policy_version 1447805 (0.0005) [2023-12-27 01:56:13,878][105692] Updated weights for policy 0, policy_version 1447815 (0.0005) [2023-12-27 01:56:14,093][105620] Updated weights for policy 1, policy_version 1450181 (0.0006) [2023-12-27 01:56:14,159][105620] Updated weights for policy 1, policy_version 1450191 (0.0005) [2023-12-27 01:56:14,211][105620] Updated weights for policy 1, policy_version 1450201 (0.0007) [2023-12-27 01:56:14,436][105692] Updated weights for policy 0, policy_version 1447825 (0.0005) [2023-12-27 01:56:14,489][105692] Updated weights for policy 0, policy_version 1447835 (0.0009) [2023-12-27 01:56:14,546][105692] Updated weights for policy 0, policy_version 1447845 (0.0006) [2023-12-27 01:56:14,605][105692] Updated weights for policy 0, policy_version 1447855 (0.0008) [2023-12-27 01:56:14,936][105620] Updated weights for policy 1, policy_version 1450212 (0.0010) [2023-12-27 01:56:15,002][105620] Updated weights for policy 1, policy_version 1450222 (0.0009) [2023-12-27 01:56:15,065][105620] Updated weights for policy 1, policy_version 1450232 (0.0009) [2023-12-27 01:56:15,294][105692] Updated weights for policy 0, policy_version 1447865 (0.0008) [2023-12-27 01:56:15,349][105692] Updated weights for policy 0, policy_version 1447875 (0.0009) [2023-12-27 01:56:15,416][105692] Updated weights for policy 0, policy_version 1447885 (0.0009) [2023-12-27 01:56:15,740][105620] Updated weights for policy 1, policy_version 1450242 (0.0008) [2023-12-27 01:56:15,795][105620] Updated weights for policy 1, policy_version 1450252 (0.0005) [2023-12-27 01:56:15,857][105620] Updated weights for policy 1, policy_version 1450262 (0.0006) [2023-12-27 01:56:15,915][105620] Updated weights for policy 1, policy_version 1450272 (0.0006) [2023-12-27 01:56:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.4, 300 sec: 19466.4). Total num frames: 742031360. Throughput: 0: 9766.8, 1: 9587.9. Samples: 741998576. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:56:16,063][104569] Avg episode reward: [(0, '8709.770'), (1, '8894.972')] [2023-12-27 01:56:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001450272_371318784.pth... [2023-12-27 01:56:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001447888_370712576.pth... [2023-12-27 01:56:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001449152_371032064.pth [2023-12-27 01:56:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001446736_370417664.pth [2023-12-27 01:56:16,166][105692] Updated weights for policy 0, policy_version 1447895 (0.0007) [2023-12-27 01:56:16,225][105692] Updated weights for policy 0, policy_version 1447905 (0.0011) [2023-12-27 01:56:16,281][105692] Updated weights for policy 0, policy_version 1447915 (0.0011) [2023-12-27 01:56:16,572][105620] Updated weights for policy 1, policy_version 1450282 (0.0010) [2023-12-27 01:56:16,627][105620] Updated weights for policy 1, policy_version 1450292 (0.0010) [2023-12-27 01:56:16,676][105620] Updated weights for policy 1, policy_version 1450302 (0.0010) [2023-12-27 01:56:16,841][105692] Updated weights for policy 0, policy_version 1447925 (0.0006) [2023-12-27 01:56:16,892][105692] Updated weights for policy 0, policy_version 1447935 (0.0005) [2023-12-27 01:56:16,948][105692] Updated weights for policy 0, policy_version 1447945 (0.0005) [2023-12-27 01:56:17,342][105620] Updated weights for policy 1, policy_version 1450312 (0.0006) [2023-12-27 01:56:17,404][105620] Updated weights for policy 1, policy_version 1450322 (0.0005) [2023-12-27 01:56:17,476][105620] Updated weights for policy 1, policy_version 1450332 (0.0009) [2023-12-27 01:56:17,498][105692] Updated weights for policy 0, policy_version 1447955 (0.0005) [2023-12-27 01:56:17,549][105692] Updated weights for policy 0, policy_version 1447965 (0.0005) [2023-12-27 01:56:17,607][105692] Updated weights for policy 0, policy_version 1447975 (0.0005) [2023-12-27 01:56:18,042][105620] Updated weights for policy 1, policy_version 1450342 (0.0007) [2023-12-27 01:56:18,093][105620] Updated weights for policy 1, policy_version 1450352 (0.0005) [2023-12-27 01:56:18,147][105620] Updated weights for policy 1, policy_version 1450362 (0.0005) [2023-12-27 01:56:18,258][105692] Updated weights for policy 0, policy_version 1447985 (0.0005) [2023-12-27 01:56:18,326][105692] Updated weights for policy 0, policy_version 1447995 (0.0006) [2023-12-27 01:56:18,392][105692] Updated weights for policy 0, policy_version 1448005 (0.0008) [2023-12-27 01:56:18,457][105692] Updated weights for policy 0, policy_version 1448015 (0.0006) [2023-12-27 01:56:18,803][105620] Updated weights for policy 1, policy_version 1450372 (0.0007) [2023-12-27 01:56:18,870][105620] Updated weights for policy 1, policy_version 1450382 (0.0011) [2023-12-27 01:56:18,931][105620] Updated weights for policy 1, policy_version 1450392 (0.0011) [2023-12-27 01:56:19,080][105692] Updated weights for policy 0, policy_version 1448025 (0.0010) [2023-12-27 01:56:19,143][105692] Updated weights for policy 0, policy_version 1448035 (0.0009) [2023-12-27 01:56:19,203][105692] Updated weights for policy 0, policy_version 1448045 (0.0008) [2023-12-27 01:56:19,695][105620] Updated weights for policy 1, policy_version 1450402 (0.0011) [2023-12-27 01:56:19,747][105620] Updated weights for policy 1, policy_version 1450412 (0.0010) [2023-12-27 01:56:19,811][105620] Updated weights for policy 1, policy_version 1450422 (0.0011) [2023-12-27 01:56:19,875][105620] Updated weights for policy 1, policy_version 1450432 (0.0010) [2023-12-27 01:56:19,962][105692] Updated weights for policy 0, policy_version 1448055 (0.0011) [2023-12-27 01:56:20,022][105692] Updated weights for policy 0, policy_version 1448065 (0.0011) [2023-12-27 01:56:20,082][105692] Updated weights for policy 0, policy_version 1448075 (0.0011) [2023-12-27 01:56:20,644][105620] Updated weights for policy 1, policy_version 1450442 (0.0008) [2023-12-27 01:56:20,701][105620] Updated weights for policy 1, policy_version 1450452 (0.0008) [2023-12-27 01:56:20,762][105620] Updated weights for policy 1, policy_version 1450462 (0.0008) [2023-12-27 01:56:20,850][105692] Updated weights for policy 0, policy_version 1448085 (0.0011) [2023-12-27 01:56:20,917][105692] Updated weights for policy 0, policy_version 1448095 (0.0011) [2023-12-27 01:56:20,989][105692] Updated weights for policy 0, policy_version 1448105 (0.0010) [2023-12-27 01:56:21,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 742137856. Throughput: 0: 9883.1, 1: 9650.4. Samples: 742124044. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:56:21,063][104569] Avg episode reward: [(0, '8712.053'), (1, '8801.332')] [2023-12-27 01:56:21,534][105620] Updated weights for policy 1, policy_version 1450472 (0.0008) [2023-12-27 01:56:21,594][105620] Updated weights for policy 1, policy_version 1450482 (0.0009) [2023-12-27 01:56:21,659][105620] Updated weights for policy 1, policy_version 1450492 (0.0008) [2023-12-27 01:56:21,734][105692] Updated weights for policy 0, policy_version 1448115 (0.0009) [2023-12-27 01:56:21,794][105692] Updated weights for policy 0, policy_version 1448125 (0.0011) [2023-12-27 01:56:21,851][105692] Updated weights for policy 0, policy_version 1448135 (0.0011) [2023-12-27 01:56:22,427][105620] Updated weights for policy 1, policy_version 1450502 (0.0007) [2023-12-27 01:56:22,494][105620] Updated weights for policy 1, policy_version 1450512 (0.0006) [2023-12-27 01:56:22,557][105620] Updated weights for policy 1, policy_version 1450522 (0.0008) [2023-12-27 01:56:22,595][105692] Updated weights for policy 0, policy_version 1448145 (0.0011) [2023-12-27 01:56:22,663][105692] Updated weights for policy 0, policy_version 1448155 (0.0010) [2023-12-27 01:56:22,734][105692] Updated weights for policy 0, policy_version 1448165 (0.0005) [2023-12-27 01:56:22,790][105692] Updated weights for policy 0, policy_version 1448175 (0.0006) [2023-12-27 01:56:23,137][105620] Updated weights for policy 1, policy_version 1450532 (0.0007) [2023-12-27 01:56:23,200][105620] Updated weights for policy 1, policy_version 1450542 (0.0010) [2023-12-27 01:56:23,253][105620] Updated weights for policy 1, policy_version 1450552 (0.0010) [2023-12-27 01:56:23,344][105692] Updated weights for policy 0, policy_version 1448185 (0.0010) [2023-12-27 01:56:23,397][105692] Updated weights for policy 0, policy_version 1448195 (0.0010) [2023-12-27 01:56:23,458][105692] Updated weights for policy 0, policy_version 1448205 (0.0010) [2023-12-27 01:56:23,949][105620] Updated weights for policy 1, policy_version 1450562 (0.0008) [2023-12-27 01:56:24,044][105620] Updated weights for policy 1, policy_version 1450572 (0.0007) [2023-12-27 01:56:24,088][105692] Updated weights for policy 0, policy_version 1448215 (0.0010) [2023-12-27 01:56:24,106][105620] Updated weights for policy 1, policy_version 1450582 (0.0008) [2023-12-27 01:56:24,147][105692] Updated weights for policy 0, policy_version 1448225 (0.0010) [2023-12-27 01:56:24,161][105620] Updated weights for policy 1, policy_version 1450592 (0.0010) [2023-12-27 01:56:24,203][105692] Updated weights for policy 0, policy_version 1448235 (0.0010) [2023-12-27 01:56:24,761][105620] Updated weights for policy 1, policy_version 1450602 (0.0009) [2023-12-27 01:56:24,817][105620] Updated weights for policy 1, policy_version 1450612 (0.0005) [2023-12-27 01:56:24,879][105620] Updated weights for policy 1, policy_version 1450622 (0.0005) [2023-12-27 01:56:24,953][105692] Updated weights for policy 0, policy_version 1448245 (0.0010) [2023-12-27 01:56:25,008][105692] Updated weights for policy 0, policy_version 1448255 (0.0010) [2023-12-27 01:56:25,060][105692] Updated weights for policy 0, policy_version 1448265 (0.0010) [2023-12-27 01:56:25,518][105620] Updated weights for policy 1, policy_version 1450632 (0.0005) [2023-12-27 01:56:25,584][105620] Updated weights for policy 1, policy_version 1450642 (0.0005) [2023-12-27 01:56:25,647][105620] Updated weights for policy 1, policy_version 1450652 (0.0005) [2023-12-27 01:56:25,826][105692] Updated weights for policy 0, policy_version 1448275 (0.0010) [2023-12-27 01:56:25,880][105692] Updated weights for policy 0, policy_version 1448285 (0.0010) [2023-12-27 01:56:25,928][105692] Updated weights for policy 0, policy_version 1448295 (0.0010) [2023-12-27 01:56:26,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 742236160. Throughput: 0: 9916.9, 1: 9717.3. Samples: 742243344. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:56:26,062][104569] Avg episode reward: [(0, '8985.262'), (1, '9082.055')] [2023-12-27 01:56:26,176][105620] Updated weights for policy 1, policy_version 1450662 (0.0005) [2023-12-27 01:56:26,232][105620] Updated weights for policy 1, policy_version 1450672 (0.0010) [2023-12-27 01:56:26,288][105620] Updated weights for policy 1, policy_version 1450682 (0.0009) [2023-12-27 01:56:26,547][105692] Updated weights for policy 0, policy_version 1448305 (0.0009) [2023-12-27 01:56:26,606][105692] Updated weights for policy 0, policy_version 1448315 (0.0010) [2023-12-27 01:56:26,672][105692] Updated weights for policy 0, policy_version 1448325 (0.0011) [2023-12-27 01:56:26,735][105692] Updated weights for policy 0, policy_version 1448335 (0.0011) [2023-12-27 01:56:26,883][105620] Updated weights for policy 1, policy_version 1450692 (0.0007) [2023-12-27 01:56:26,935][105620] Updated weights for policy 1, policy_version 1450702 (0.0006) [2023-12-27 01:56:26,982][105620] Updated weights for policy 1, policy_version 1450712 (0.0009) [2023-12-27 01:56:27,473][105692] Updated weights for policy 0, policy_version 1448345 (0.0010) [2023-12-27 01:56:27,521][105692] Updated weights for policy 0, policy_version 1448355 (0.0010) [2023-12-27 01:56:27,575][105692] Updated weights for policy 0, policy_version 1448365 (0.0010) [2023-12-27 01:56:27,592][105620] Updated weights for policy 1, policy_version 1450722 (0.0009) [2023-12-27 01:56:27,639][105620] Updated weights for policy 1, policy_version 1450732 (0.0008) [2023-12-27 01:56:27,697][105620] Updated weights for policy 1, policy_version 1450742 (0.0008) [2023-12-27 01:56:27,759][105620] Updated weights for policy 1, policy_version 1450752 (0.0008) [2023-12-27 01:56:28,266][105692] Updated weights for policy 0, policy_version 1448375 (0.0011) [2023-12-27 01:56:28,315][105692] Updated weights for policy 0, policy_version 1448385 (0.0011) [2023-12-27 01:56:28,374][105692] Updated weights for policy 0, policy_version 1448395 (0.0008) [2023-12-27 01:56:28,478][105620] Updated weights for policy 1, policy_version 1450762 (0.0007) [2023-12-27 01:56:28,526][105620] Updated weights for policy 1, policy_version 1450772 (0.0008) [2023-12-27 01:56:28,581][105620] Updated weights for policy 1, policy_version 1450782 (0.0008) [2023-12-27 01:56:29,093][105692] Updated weights for policy 0, policy_version 1448405 (0.0007) [2023-12-27 01:56:29,157][105692] Updated weights for policy 0, policy_version 1448415 (0.0005) [2023-12-27 01:56:29,209][105692] Updated weights for policy 0, policy_version 1448425 (0.0007) [2023-12-27 01:56:29,408][105620] Updated weights for policy 1, policy_version 1450792 (0.0010) [2023-12-27 01:56:29,463][105620] Updated weights for policy 1, policy_version 1450802 (0.0010) [2023-12-27 01:56:29,521][105620] Updated weights for policy 1, policy_version 1450812 (0.0009) [2023-12-27 01:56:29,789][105692] Updated weights for policy 0, policy_version 1448435 (0.0007) [2023-12-27 01:56:29,847][105692] Updated weights for policy 0, policy_version 1448445 (0.0006) [2023-12-27 01:56:29,897][105692] Updated weights for policy 0, policy_version 1448455 (0.0006) [2023-12-27 01:56:30,328][105620] Updated weights for policy 1, policy_version 1450822 (0.0007) [2023-12-27 01:56:30,390][105620] Updated weights for policy 1, policy_version 1450832 (0.0005) [2023-12-27 01:56:30,441][105620] Updated weights for policy 1, policy_version 1450842 (0.0005) [2023-12-27 01:56:30,596][105692] Updated weights for policy 0, policy_version 1448465 (0.0008) [2023-12-27 01:56:30,652][105692] Updated weights for policy 0, policy_version 1448475 (0.0010) [2023-12-27 01:56:30,703][105692] Updated weights for policy 0, policy_version 1448485 (0.0009) [2023-12-27 01:56:30,761][105692] Updated weights for policy 0, policy_version 1448495 (0.0010) [2023-12-27 01:56:30,960][105620] Updated weights for policy 1, policy_version 1450852 (0.0006) [2023-12-27 01:56:31,009][105620] Updated weights for policy 1, policy_version 1450862 (0.0005) [2023-12-27 01:56:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 742334464. Throughput: 0: 9960.0, 1: 9782.3. Samples: 742304720. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:56:31,062][104569] Avg episode reward: [(0, '9074.762'), (1, '9082.067')] [2023-12-27 01:56:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001448496_370868224.pth... [2023-12-27 01:56:31,070][105620] Updated weights for policy 1, policy_version 1450872 (0.0008) [2023-12-27 01:56:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001447344_370573312.pth [2023-12-27 01:56:31,130][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001450880_371474432.pth... [2023-12-27 01:56:31,135][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001449728_371179520.pth [2023-12-27 01:56:31,564][105692] Updated weights for policy 0, policy_version 1448505 (0.0008) [2023-12-27 01:56:31,619][105692] Updated weights for policy 0, policy_version 1448515 (0.0009) [2023-12-27 01:56:31,675][105692] Updated weights for policy 0, policy_version 1448525 (0.0008) [2023-12-27 01:56:31,800][105620] Updated weights for policy 1, policy_version 1450882 (0.0008) [2023-12-27 01:56:31,858][105620] Updated weights for policy 1, policy_version 1450892 (0.0009) [2023-12-27 01:56:31,912][105620] Updated weights for policy 1, policy_version 1450902 (0.0008) [2023-12-27 01:56:31,974][105620] Updated weights for policy 1, policy_version 1450912 (0.0009) [2023-12-27 01:56:32,424][105692] Updated weights for policy 0, policy_version 1448535 (0.0009) [2023-12-27 01:56:32,475][105692] Updated weights for policy 0, policy_version 1448545 (0.0009) [2023-12-27 01:56:32,530][105692] Updated weights for policy 0, policy_version 1448555 (0.0008) [2023-12-27 01:56:32,734][105620] Updated weights for policy 1, policy_version 1450922 (0.0011) [2023-12-27 01:56:32,796][105620] Updated weights for policy 1, policy_version 1450932 (0.0011) [2023-12-27 01:56:32,832][105586] KL-divergence is very high: 133.9985 [2023-12-27 01:56:32,852][105620] Updated weights for policy 1, policy_version 1450942 (0.0010) [2023-12-27 01:56:33,281][105692] Updated weights for policy 0, policy_version 1448565 (0.0007) [2023-12-27 01:56:33,342][105692] Updated weights for policy 0, policy_version 1448575 (0.0005) [2023-12-27 01:56:33,407][105692] Updated weights for policy 0, policy_version 1448585 (0.0005) [2023-12-27 01:56:33,629][105620] Updated weights for policy 1, policy_version 1450952 (0.0010) [2023-12-27 01:56:33,677][105620] Updated weights for policy 1, policy_version 1450962 (0.0008) [2023-12-27 01:56:33,737][105620] Updated weights for policy 1, policy_version 1450972 (0.0009) [2023-12-27 01:56:33,908][105692] Updated weights for policy 0, policy_version 1448595 (0.0007) [2023-12-27 01:56:33,955][105692] Updated weights for policy 0, policy_version 1448605 (0.0010) [2023-12-27 01:56:34,006][105692] Updated weights for policy 0, policy_version 1448615 (0.0010) [2023-12-27 01:56:34,519][105620] Updated weights for policy 1, policy_version 1450982 (0.0008) [2023-12-27 01:56:34,583][105620] Updated weights for policy 1, policy_version 1450992 (0.0008) [2023-12-27 01:56:34,644][105620] Updated weights for policy 1, policy_version 1451002 (0.0007) [2023-12-27 01:56:34,709][105692] Updated weights for policy 0, policy_version 1448625 (0.0005) [2023-12-27 01:56:34,765][105692] Updated weights for policy 0, policy_version 1448635 (0.0010) [2023-12-27 01:56:34,812][105692] Updated weights for policy 0, policy_version 1448645 (0.0009) [2023-12-27 01:56:34,860][105692] Updated weights for policy 0, policy_version 1448655 (0.0005) [2023-12-27 01:56:35,388][105620] Updated weights for policy 1, policy_version 1451012 (0.0007) [2023-12-27 01:56:35,418][105692] Updated weights for policy 0, policy_version 1448665 (0.0006) [2023-12-27 01:56:35,438][105620] Updated weights for policy 1, policy_version 1451022 (0.0005) [2023-12-27 01:56:35,479][105692] Updated weights for policy 0, policy_version 1448675 (0.0006) [2023-12-27 01:56:35,495][105620] Updated weights for policy 1, policy_version 1451032 (0.0005) [2023-12-27 01:56:35,544][105692] Updated weights for policy 0, policy_version 1448685 (0.0008) [2023-12-27 01:56:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 742432768. Throughput: 0: 9943.0, 1: 9865.2. Samples: 742422832. Policy #0 lag: (min: 29.0, avg: 38.3, max: 61.0) [2023-12-27 01:56:36,062][104569] Avg episode reward: [(0, '8890.867'), (1, '8899.891')] [2023-12-27 01:56:36,115][105692] Updated weights for policy 0, policy_version 1448695 (0.0010) [2023-12-27 01:56:36,133][105620] Updated weights for policy 1, policy_version 1451042 (0.0006) [2023-12-27 01:56:36,173][105692] Updated weights for policy 0, policy_version 1448705 (0.0011) [2023-12-27 01:56:36,196][105620] Updated weights for policy 1, policy_version 1451052 (0.0007) [2023-12-27 01:56:36,230][105692] Updated weights for policy 0, policy_version 1448715 (0.0011) [2023-12-27 01:56:36,261][105620] Updated weights for policy 1, policy_version 1451062 (0.0011) [2023-12-27 01:56:36,331][105620] Updated weights for policy 1, policy_version 1451072 (0.0011) [2023-12-27 01:56:37,001][105692] Updated weights for policy 0, policy_version 1448725 (0.0007) [2023-12-27 01:56:37,043][105620] Updated weights for policy 1, policy_version 1451082 (0.0008) [2023-12-27 01:56:37,060][105692] Updated weights for policy 0, policy_version 1448735 (0.0008) [2023-12-27 01:56:37,094][105620] Updated weights for policy 1, policy_version 1451092 (0.0005) [2023-12-27 01:56:37,117][105692] Updated weights for policy 0, policy_version 1448745 (0.0009) [2023-12-27 01:56:37,149][105620] Updated weights for policy 1, policy_version 1451102 (0.0006) [2023-12-27 01:56:37,696][105620] Updated weights for policy 1, policy_version 1451112 (0.0005) [2023-12-27 01:56:37,767][105620] Updated weights for policy 1, policy_version 1451122 (0.0006) [2023-12-27 01:56:37,829][105620] Updated weights for policy 1, policy_version 1451132 (0.0005) [2023-12-27 01:56:37,959][105692] Updated weights for policy 0, policy_version 1448755 (0.0009) [2023-12-27 01:56:38,025][105692] Updated weights for policy 0, policy_version 1448765 (0.0008) [2023-12-27 01:56:38,081][105692] Updated weights for policy 0, policy_version 1448775 (0.0005) [2023-12-27 01:56:38,382][105620] Updated weights for policy 1, policy_version 1451142 (0.0009) [2023-12-27 01:56:38,442][105620] Updated weights for policy 1, policy_version 1451152 (0.0011) [2023-12-27 01:56:38,504][105620] Updated weights for policy 1, policy_version 1451162 (0.0010) [2023-12-27 01:56:38,775][105692] Updated weights for policy 0, policy_version 1448785 (0.0007) [2023-12-27 01:56:38,825][105692] Updated weights for policy 0, policy_version 1448795 (0.0008) [2023-12-27 01:56:38,873][105692] Updated weights for policy 0, policy_version 1448805 (0.0008) [2023-12-27 01:56:38,925][105692] Updated weights for policy 0, policy_version 1448815 (0.0007) [2023-12-27 01:56:39,241][105620] Updated weights for policy 1, policy_version 1451172 (0.0010) [2023-12-27 01:56:39,296][105620] Updated weights for policy 1, policy_version 1451182 (0.0010) [2023-12-27 01:56:39,368][105620] Updated weights for policy 1, policy_version 1451192 (0.0011) [2023-12-27 01:56:39,718][105692] Updated weights for policy 0, policy_version 1448825 (0.0009) [2023-12-27 01:56:39,783][105692] Updated weights for policy 0, policy_version 1448835 (0.0009) [2023-12-27 01:56:39,850][105692] Updated weights for policy 0, policy_version 1448845 (0.0009) [2023-12-27 01:56:40,118][105620] Updated weights for policy 1, policy_version 1451202 (0.0010) [2023-12-27 01:56:40,166][105620] Updated weights for policy 1, policy_version 1451212 (0.0008) [2023-12-27 01:56:40,228][105620] Updated weights for policy 1, policy_version 1451222 (0.0009) [2023-12-27 01:56:40,293][105620] Updated weights for policy 1, policy_version 1451232 (0.0009) [2023-12-27 01:56:40,621][105692] Updated weights for policy 0, policy_version 1448855 (0.0008) [2023-12-27 01:56:40,679][105692] Updated weights for policy 0, policy_version 1448865 (0.0010) [2023-12-27 01:56:40,729][105692] Updated weights for policy 0, policy_version 1448875 (0.0009) [2023-12-27 01:56:40,993][105620] Updated weights for policy 1, policy_version 1451242 (0.0009) [2023-12-27 01:56:41,057][105620] Updated weights for policy 1, policy_version 1451252 (0.0008) [2023-12-27 01:56:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 742531072. Throughput: 0: 9838.4, 1: 9960.1. Samples: 742542324. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:56:41,063][104569] Avg episode reward: [(0, '8704.932'), (1, '8809.565')] [2023-12-27 01:56:41,122][105620] Updated weights for policy 1, policy_version 1451262 (0.0008) [2023-12-27 01:56:41,547][105692] Updated weights for policy 0, policy_version 1448885 (0.0010) [2023-12-27 01:56:41,608][105692] Updated weights for policy 0, policy_version 1448895 (0.0010) [2023-12-27 01:56:41,677][105692] Updated weights for policy 0, policy_version 1448905 (0.0009) [2023-12-27 01:56:42,003][105620] Updated weights for policy 1, policy_version 1451272 (0.0008) [2023-12-27 01:56:42,072][105620] Updated weights for policy 1, policy_version 1451282 (0.0008) [2023-12-27 01:56:42,131][105620] Updated weights for policy 1, policy_version 1451292 (0.0007) [2023-12-27 01:56:42,439][105692] Updated weights for policy 0, policy_version 1448915 (0.0011) [2023-12-27 01:56:42,500][105692] Updated weights for policy 0, policy_version 1448925 (0.0011) [2023-12-27 01:56:42,557][105692] Updated weights for policy 0, policy_version 1448935 (0.0010) [2023-12-27 01:56:42,850][105620] Updated weights for policy 1, policy_version 1451302 (0.0008) [2023-12-27 01:56:42,918][105620] Updated weights for policy 1, policy_version 1451312 (0.0010) [2023-12-27 01:56:42,985][105620] Updated weights for policy 1, policy_version 1451322 (0.0010) [2023-12-27 01:56:43,173][105692] Updated weights for policy 0, policy_version 1448945 (0.0010) [2023-12-27 01:56:43,226][105692] Updated weights for policy 0, policy_version 1448955 (0.0005) [2023-12-27 01:56:43,271][105692] Updated weights for policy 0, policy_version 1448965 (0.0009) [2023-12-27 01:56:43,326][105692] Updated weights for policy 0, policy_version 1448975 (0.0010) [2023-12-27 01:56:43,805][105620] Updated weights for policy 1, policy_version 1451332 (0.0010) [2023-12-27 01:56:43,850][105620] Updated weights for policy 1, policy_version 1451342 (0.0010) [2023-12-27 01:56:43,896][105620] Updated weights for policy 1, policy_version 1451352 (0.0010) [2023-12-27 01:56:43,901][105692] Updated weights for policy 0, policy_version 1448985 (0.0006) [2023-12-27 01:56:43,955][105692] Updated weights for policy 0, policy_version 1448995 (0.0010) [2023-12-27 01:56:44,003][105692] Updated weights for policy 0, policy_version 1449005 (0.0010) [2023-12-27 01:56:44,509][105620] Updated weights for policy 1, policy_version 1451362 (0.0009) [2023-12-27 01:56:44,574][105620] Updated weights for policy 1, policy_version 1451372 (0.0006) [2023-12-27 01:56:44,622][105620] Updated weights for policy 1, policy_version 1451382 (0.0005) [2023-12-27 01:56:44,672][105620] Updated weights for policy 1, policy_version 1451392 (0.0005) [2023-12-27 01:56:44,743][105692] Updated weights for policy 0, policy_version 1449015 (0.0010) [2023-12-27 01:56:44,807][105692] Updated weights for policy 0, policy_version 1449025 (0.0009) [2023-12-27 01:56:44,877][105692] Updated weights for policy 0, policy_version 1449035 (0.0007) [2023-12-27 01:56:45,329][105620] Updated weights for policy 1, policy_version 1451402 (0.0010) [2023-12-27 01:56:45,386][105620] Updated weights for policy 1, policy_version 1451412 (0.0011) [2023-12-27 01:56:45,442][105620] Updated weights for policy 1, policy_version 1451422 (0.0011) [2023-12-27 01:56:45,578][105692] Updated weights for policy 0, policy_version 1449045 (0.0009) [2023-12-27 01:56:45,640][105692] Updated weights for policy 0, policy_version 1449055 (0.0010) [2023-12-27 01:56:45,694][105692] Updated weights for policy 0, policy_version 1449065 (0.0010) [2023-12-27 01:56:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 742629376. Throughput: 0: 9789.6, 1: 9810.9. Samples: 742597396. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:56:46,063][104569] Avg episode reward: [(0, '8520.684'), (1, '8621.518')] [2023-12-27 01:56:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001449072_371015680.pth... [2023-12-27 01:56:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001447888_370712576.pth [2023-12-27 01:56:46,106][105620] Updated weights for policy 1, policy_version 1451432 (0.0006) [2023-12-27 01:56:46,172][105620] Updated weights for policy 1, policy_version 1451442 (0.0010) [2023-12-27 01:56:46,237][105620] Updated weights for policy 1, policy_version 1451452 (0.0010) [2023-12-27 01:56:46,258][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001451456_371621888.pth... [2023-12-27 01:56:46,263][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001450272_371318784.pth [2023-12-27 01:56:46,435][105692] Updated weights for policy 0, policy_version 1449075 (0.0010) [2023-12-27 01:56:46,492][105692] Updated weights for policy 0, policy_version 1449085 (0.0010) [2023-12-27 01:56:46,551][105692] Updated weights for policy 0, policy_version 1449095 (0.0010) [2023-12-27 01:56:46,930][105620] Updated weights for policy 1, policy_version 1451462 (0.0009) [2023-12-27 01:56:46,991][105620] Updated weights for policy 1, policy_version 1451472 (0.0010) [2023-12-27 01:56:47,053][105620] Updated weights for policy 1, policy_version 1451482 (0.0010) [2023-12-27 01:56:47,221][105692] Updated weights for policy 0, policy_version 1449105 (0.0010) [2023-12-27 01:56:47,285][105692] Updated weights for policy 0, policy_version 1449115 (0.0010) [2023-12-27 01:56:47,349][105692] Updated weights for policy 0, policy_version 1449125 (0.0010) [2023-12-27 01:56:47,416][105692] Updated weights for policy 0, policy_version 1449135 (0.0010) [2023-12-27 01:56:47,789][105620] Updated weights for policy 1, policy_version 1451492 (0.0010) [2023-12-27 01:56:47,838][105620] Updated weights for policy 1, policy_version 1451502 (0.0010) [2023-12-27 01:56:47,893][105620] Updated weights for policy 1, policy_version 1451512 (0.0010) [2023-12-27 01:56:48,126][105692] Updated weights for policy 0, policy_version 1449145 (0.0010) [2023-12-27 01:56:48,176][105692] Updated weights for policy 0, policy_version 1449155 (0.0010) [2023-12-27 01:56:48,237][105692] Updated weights for policy 0, policy_version 1449165 (0.0010) [2023-12-27 01:56:48,626][105620] Updated weights for policy 1, policy_version 1451522 (0.0009) [2023-12-27 01:56:48,687][105620] Updated weights for policy 1, policy_version 1451532 (0.0007) [2023-12-27 01:56:48,742][105620] Updated weights for policy 1, policy_version 1451542 (0.0010) [2023-12-27 01:56:48,802][105620] Updated weights for policy 1, policy_version 1451552 (0.0009) [2023-12-27 01:56:48,947][105692] Updated weights for policy 0, policy_version 1449175 (0.0007) [2023-12-27 01:56:49,001][105692] Updated weights for policy 0, policy_version 1449185 (0.0005) [2023-12-27 01:56:49,060][105692] Updated weights for policy 0, policy_version 1449195 (0.0008) [2023-12-27 01:56:49,507][105620] Updated weights for policy 1, policy_version 1451562 (0.0009) [2023-12-27 01:56:49,564][105620] Updated weights for policy 1, policy_version 1451572 (0.0010) [2023-12-27 01:56:49,625][105620] Updated weights for policy 1, policy_version 1451582 (0.0009) [2023-12-27 01:56:49,698][105692] Updated weights for policy 0, policy_version 1449205 (0.0010) [2023-12-27 01:56:49,760][105692] Updated weights for policy 0, policy_version 1449215 (0.0010) [2023-12-27 01:56:49,827][105692] Updated weights for policy 0, policy_version 1449225 (0.0011) [2023-12-27 01:56:50,405][105620] Updated weights for policy 1, policy_version 1451592 (0.0009) [2023-12-27 01:56:50,457][105620] Updated weights for policy 1, policy_version 1451602 (0.0009) [2023-12-27 01:56:50,467][105692] Updated weights for policy 0, policy_version 1449235 (0.0007) [2023-12-27 01:56:50,513][105620] Updated weights for policy 1, policy_version 1451612 (0.0006) [2023-12-27 01:56:50,527][105692] Updated weights for policy 0, policy_version 1449245 (0.0007) [2023-12-27 01:56:50,596][105692] Updated weights for policy 0, policy_version 1449255 (0.0009) [2023-12-27 01:56:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 742727680. Throughput: 0: 9846.4, 1: 9833.9. Samples: 742717580. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:56:51,063][104569] Avg episode reward: [(0, '8248.200'), (1, '8527.528')] [2023-12-27 01:56:51,240][105692] Updated weights for policy 0, policy_version 1449265 (0.0009) [2023-12-27 01:56:51,302][105692] Updated weights for policy 0, policy_version 1449275 (0.0009) [2023-12-27 01:56:51,330][105620] Updated weights for policy 1, policy_version 1451622 (0.0007) [2023-12-27 01:56:51,369][105692] Updated weights for policy 0, policy_version 1449285 (0.0008) [2023-12-27 01:56:51,393][105620] Updated weights for policy 1, policy_version 1451632 (0.0007) [2023-12-27 01:56:51,436][105692] Updated weights for policy 0, policy_version 1449295 (0.0008) [2023-12-27 01:56:51,448][105620] Updated weights for policy 1, policy_version 1451642 (0.0005) [2023-12-27 01:56:52,134][105620] Updated weights for policy 1, policy_version 1451652 (0.0007) [2023-12-27 01:56:52,170][105692] Updated weights for policy 0, policy_version 1449305 (0.0007) [2023-12-27 01:56:52,188][105620] Updated weights for policy 1, policy_version 1451662 (0.0007) [2023-12-27 01:56:52,230][105692] Updated weights for policy 0, policy_version 1449315 (0.0008) [2023-12-27 01:56:52,245][105620] Updated weights for policy 1, policy_version 1451672 (0.0005) [2023-12-27 01:56:52,292][105692] Updated weights for policy 0, policy_version 1449325 (0.0008) [2023-12-27 01:56:52,864][105620] Updated weights for policy 1, policy_version 1451682 (0.0007) [2023-12-27 01:56:52,917][105620] Updated weights for policy 1, policy_version 1451692 (0.0005) [2023-12-27 01:56:52,980][105620] Updated weights for policy 1, policy_version 1451702 (0.0005) [2023-12-27 01:56:53,041][105620] Updated weights for policy 1, policy_version 1451712 (0.0006) [2023-12-27 01:56:53,122][105692] Updated weights for policy 0, policy_version 1449335 (0.0010) [2023-12-27 01:56:53,184][105692] Updated weights for policy 0, policy_version 1449345 (0.0010) [2023-12-27 01:56:53,249][105692] Updated weights for policy 0, policy_version 1449355 (0.0010) [2023-12-27 01:56:53,561][105620] Updated weights for policy 1, policy_version 1451722 (0.0005) [2023-12-27 01:56:53,614][105620] Updated weights for policy 1, policy_version 1451732 (0.0007) [2023-12-27 01:56:53,668][105620] Updated weights for policy 1, policy_version 1451742 (0.0009) [2023-12-27 01:56:53,845][105692] Updated weights for policy 0, policy_version 1449365 (0.0008) [2023-12-27 01:56:53,916][105692] Updated weights for policy 0, policy_version 1449375 (0.0007) [2023-12-27 01:56:53,972][105692] Updated weights for policy 0, policy_version 1449385 (0.0008) [2023-12-27 01:56:54,267][105620] Updated weights for policy 1, policy_version 1451752 (0.0010) [2023-12-27 01:56:54,315][105620] Updated weights for policy 1, policy_version 1451762 (0.0010) [2023-12-27 01:56:54,359][105620] Updated weights for policy 1, policy_version 1451772 (0.0010) [2023-12-27 01:56:54,630][105692] Updated weights for policy 0, policy_version 1449395 (0.0009) [2023-12-27 01:56:54,690][105692] Updated weights for policy 0, policy_version 1449405 (0.0008) [2023-12-27 01:56:54,746][105692] Updated weights for policy 0, policy_version 1449415 (0.0008) [2023-12-27 01:56:55,111][105620] Updated weights for policy 1, policy_version 1451782 (0.0010) [2023-12-27 01:56:55,174][105620] Updated weights for policy 1, policy_version 1451792 (0.0011) [2023-12-27 01:56:55,236][105620] Updated weights for policy 1, policy_version 1451802 (0.0010) [2023-12-27 01:56:55,395][105692] Updated weights for policy 0, policy_version 1449425 (0.0008) [2023-12-27 01:56:55,453][105692] Updated weights for policy 0, policy_version 1449435 (0.0009) [2023-12-27 01:56:55,507][105692] Updated weights for policy 0, policy_version 1449445 (0.0006) [2023-12-27 01:56:55,564][105692] Updated weights for policy 0, policy_version 1449455 (0.0007) [2023-12-27 01:56:55,783][105620] Updated weights for policy 1, policy_version 1451812 (0.0006) [2023-12-27 01:56:55,836][105620] Updated weights for policy 1, policy_version 1451822 (0.0010) [2023-12-27 01:56:55,887][105620] Updated weights for policy 1, policy_version 1451832 (0.0010) [2023-12-27 01:56:56,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 742834176. Throughput: 0: 9999.3, 1: 9993.0. Samples: 742840880. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:56:56,062][104569] Avg episode reward: [(0, '8157.127'), (1, '8705.858')] [2023-12-27 01:56:56,146][105692] Updated weights for policy 0, policy_version 1449465 (0.0010) [2023-12-27 01:56:56,202][105692] Updated weights for policy 0, policy_version 1449475 (0.0010) [2023-12-27 01:56:56,253][105692] Updated weights for policy 0, policy_version 1449485 (0.0010) [2023-12-27 01:56:56,555][105620] Updated weights for policy 1, policy_version 1451842 (0.0009) [2023-12-27 01:56:56,602][105620] Updated weights for policy 1, policy_version 1451852 (0.0005) [2023-12-27 01:56:56,647][105620] Updated weights for policy 1, policy_version 1451862 (0.0005) [2023-12-27 01:56:56,701][105620] Updated weights for policy 1, policy_version 1451872 (0.0005) [2023-12-27 01:56:57,000][105692] Updated weights for policy 0, policy_version 1449495 (0.0010) [2023-12-27 01:56:57,060][105692] Updated weights for policy 0, policy_version 1449505 (0.0010) [2023-12-27 01:56:57,104][105692] Updated weights for policy 0, policy_version 1449515 (0.0010) [2023-12-27 01:56:57,356][105620] Updated weights for policy 1, policy_version 1451882 (0.0007) [2023-12-27 01:56:57,413][105620] Updated weights for policy 1, policy_version 1451892 (0.0008) [2023-12-27 01:56:57,460][105620] Updated weights for policy 1, policy_version 1451902 (0.0005) [2023-12-27 01:56:57,813][105692] Updated weights for policy 0, policy_version 1449525 (0.0008) [2023-12-27 01:56:57,875][105692] Updated weights for policy 0, policy_version 1449535 (0.0005) [2023-12-27 01:56:57,920][105692] Updated weights for policy 0, policy_version 1449545 (0.0005) [2023-12-27 01:56:58,118][105620] Updated weights for policy 1, policy_version 1451912 (0.0009) [2023-12-27 01:56:58,176][105620] Updated weights for policy 1, policy_version 1451922 (0.0010) [2023-12-27 01:56:58,241][105620] Updated weights for policy 1, policy_version 1451932 (0.0010) [2023-12-27 01:56:58,647][105692] Updated weights for policy 0, policy_version 1449555 (0.0006) [2023-12-27 01:56:58,706][105692] Updated weights for policy 0, policy_version 1449565 (0.0009) [2023-12-27 01:56:58,778][105692] Updated weights for policy 0, policy_version 1449575 (0.0008) [2023-12-27 01:56:59,081][105620] Updated weights for policy 1, policy_version 1451942 (0.0009) [2023-12-27 01:56:59,137][105620] Updated weights for policy 1, policy_version 1451952 (0.0008) [2023-12-27 01:56:59,190][105620] Updated weights for policy 1, policy_version 1451962 (0.0007) [2023-12-27 01:56:59,518][105692] Updated weights for policy 0, policy_version 1449585 (0.0007) [2023-12-27 01:56:59,578][105692] Updated weights for policy 0, policy_version 1449595 (0.0006) [2023-12-27 01:56:59,644][105692] Updated weights for policy 0, policy_version 1449605 (0.0010) [2023-12-27 01:56:59,701][105692] Updated weights for policy 0, policy_version 1449615 (0.0010) [2023-12-27 01:56:59,960][105620] Updated weights for policy 1, policy_version 1451972 (0.0008) [2023-12-27 01:57:00,020][105620] Updated weights for policy 1, policy_version 1451982 (0.0008) [2023-12-27 01:57:00,083][105620] Updated weights for policy 1, policy_version 1451992 (0.0008) [2023-12-27 01:57:00,412][105692] Updated weights for policy 0, policy_version 1449625 (0.0008) [2023-12-27 01:57:00,468][105692] Updated weights for policy 0, policy_version 1449635 (0.0006) [2023-12-27 01:57:00,520][105692] Updated weights for policy 0, policy_version 1449645 (0.0006) [2023-12-27 01:57:00,880][105620] Updated weights for policy 1, policy_version 1452002 (0.0007) [2023-12-27 01:57:00,934][105620] Updated weights for policy 1, policy_version 1452012 (0.0005) [2023-12-27 01:57:00,989][105620] Updated weights for policy 1, policy_version 1452022 (0.0008) [2023-12-27 01:57:01,054][105620] Updated weights for policy 1, policy_version 1452032 (0.0007) [2023-12-27 01:57:01,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 742932480. Throughput: 0: 10051.0, 1: 10018.4. Samples: 742901700. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:57:01,062][104569] Avg episode reward: [(0, '8341.227'), (1, '9074.572')] [2023-12-27 01:57:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001449648_371163136.pth... [2023-12-27 01:57:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001452032_371769344.pth... [2023-12-27 01:57:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001448496_370868224.pth [2023-12-27 01:57:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001450880_371474432.pth [2023-12-27 01:57:01,164][105692] Updated weights for policy 0, policy_version 1449655 (0.0008) [2023-12-27 01:57:01,224][105692] Updated weights for policy 0, policy_version 1449665 (0.0011) [2023-12-27 01:57:01,286][105692] Updated weights for policy 0, policy_version 1449675 (0.0011) [2023-12-27 01:57:01,805][105620] Updated weights for policy 1, policy_version 1452042 (0.0006) [2023-12-27 01:57:01,854][105620] Updated weights for policy 1, policy_version 1452052 (0.0008) [2023-12-27 01:57:01,900][105620] Updated weights for policy 1, policy_version 1452062 (0.0006) [2023-12-27 01:57:01,953][105692] Updated weights for policy 0, policy_version 1449685 (0.0011) [2023-12-27 01:57:02,021][105692] Updated weights for policy 0, policy_version 1449695 (0.0010) [2023-12-27 01:57:02,080][105692] Updated weights for policy 0, policy_version 1449705 (0.0010) [2023-12-27 01:57:02,625][105620] Updated weights for policy 1, policy_version 1452072 (0.0007) [2023-12-27 01:57:02,683][105620] Updated weights for policy 1, policy_version 1452082 (0.0008) [2023-12-27 01:57:02,745][105620] Updated weights for policy 1, policy_version 1452092 (0.0008) [2023-12-27 01:57:02,793][105692] Updated weights for policy 0, policy_version 1449715 (0.0009) [2023-12-27 01:57:02,858][105692] Updated weights for policy 0, policy_version 1449725 (0.0006) [2023-12-27 01:57:02,910][105692] Updated weights for policy 0, policy_version 1449735 (0.0007) [2023-12-27 01:57:03,538][105620] Updated weights for policy 1, policy_version 1452102 (0.0011) [2023-12-27 01:57:03,582][105692] Updated weights for policy 0, policy_version 1449745 (0.0007) [2023-12-27 01:57:03,597][105620] Updated weights for policy 1, policy_version 1452112 (0.0010) [2023-12-27 01:57:03,633][105692] Updated weights for policy 0, policy_version 1449755 (0.0009) [2023-12-27 01:57:03,653][105620] Updated weights for policy 1, policy_version 1452122 (0.0010) [2023-12-27 01:57:03,697][105692] Updated weights for policy 0, policy_version 1449765 (0.0005) [2023-12-27 01:57:03,760][105692] Updated weights for policy 0, policy_version 1449775 (0.0009) [2023-12-27 01:57:04,393][105620] Updated weights for policy 1, policy_version 1452132 (0.0009) [2023-12-27 01:57:04,428][105692] Updated weights for policy 0, policy_version 1449785 (0.0009) [2023-12-27 01:57:04,450][105620] Updated weights for policy 1, policy_version 1452142 (0.0009) [2023-12-27 01:57:04,487][105692] Updated weights for policy 0, policy_version 1449795 (0.0006) [2023-12-27 01:57:04,512][105620] Updated weights for policy 1, policy_version 1452152 (0.0009) [2023-12-27 01:57:04,552][105692] Updated weights for policy 0, policy_version 1449805 (0.0008) [2023-12-27 01:57:05,187][105620] Updated weights for policy 1, policy_version 1452162 (0.0007) [2023-12-27 01:57:05,239][105692] Updated weights for policy 0, policy_version 1449815 (0.0010) [2023-12-27 01:57:05,242][105620] Updated weights for policy 1, policy_version 1452172 (0.0006) [2023-12-27 01:57:05,287][105692] Updated weights for policy 0, policy_version 1449825 (0.0010) [2023-12-27 01:57:05,294][105620] Updated weights for policy 1, policy_version 1452182 (0.0005) [2023-12-27 01:57:05,340][105692] Updated weights for policy 0, policy_version 1449835 (0.0011) [2023-12-27 01:57:05,353][105620] Updated weights for policy 1, policy_version 1452192 (0.0005) [2023-12-27 01:57:05,987][105692] Updated weights for policy 0, policy_version 1449845 (0.0008) [2023-12-27 01:57:06,006][105620] Updated weights for policy 1, policy_version 1452202 (0.0008) [2023-12-27 01:57:06,033][105692] Updated weights for policy 0, policy_version 1449855 (0.0005) [2023-12-27 01:57:06,062][105620] Updated weights for policy 1, policy_version 1452213 (0.0009) [2023-12-27 01:57:06,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 743022592. Throughput: 0: 9936.6, 1: 9891.8. Samples: 743016324. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:57:06,063][104569] Avg episode reward: [(0, '8154.219'), (1, '9078.142')] [2023-12-27 01:57:06,091][105692] Updated weights for policy 0, policy_version 1449865 (0.0006) [2023-12-27 01:57:06,114][105620] Updated weights for policy 1, policy_version 1452223 (0.0009) [2023-12-27 01:57:06,829][105692] Updated weights for policy 0, policy_version 1449875 (0.0011) [2023-12-27 01:57:06,894][105692] Updated weights for policy 0, policy_version 1449885 (0.0008) [2023-12-27 01:57:06,919][105620] Updated weights for policy 1, policy_version 1452233 (0.0006) [2023-12-27 01:57:06,964][105692] Updated weights for policy 0, policy_version 1449895 (0.0007) [2023-12-27 01:57:06,982][105620] Updated weights for policy 1, policy_version 1452243 (0.0008) [2023-12-27 01:57:07,044][105620] Updated weights for policy 1, policy_version 1452253 (0.0008) [2023-12-27 01:57:07,614][105692] Updated weights for policy 0, policy_version 1449905 (0.0006) [2023-12-27 01:57:07,662][105620] Updated weights for policy 1, policy_version 1452263 (0.0006) [2023-12-27 01:57:07,666][105692] Updated weights for policy 0, policy_version 1449915 (0.0010) [2023-12-27 01:57:07,710][105620] Updated weights for policy 1, policy_version 1452273 (0.0006) [2023-12-27 01:57:07,718][105692] Updated weights for policy 0, policy_version 1449925 (0.0010) [2023-12-27 01:57:07,760][105620] Updated weights for policy 1, policy_version 1452283 (0.0005) [2023-12-27 01:57:07,772][105692] Updated weights for policy 0, policy_version 1449935 (0.0010) [2023-12-27 01:57:08,432][105620] Updated weights for policy 1, policy_version 1452293 (0.0006) [2023-12-27 01:57:08,501][105620] Updated weights for policy 1, policy_version 1452303 (0.0007) [2023-12-27 01:57:08,545][105692] Updated weights for policy 0, policy_version 1449945 (0.0008) [2023-12-27 01:57:08,567][105620] Updated weights for policy 1, policy_version 1452313 (0.0006) [2023-12-27 01:57:08,608][105692] Updated weights for policy 0, policy_version 1449955 (0.0010) [2023-12-27 01:57:08,667][105692] Updated weights for policy 0, policy_version 1449965 (0.0010) [2023-12-27 01:57:09,280][105620] Updated weights for policy 1, policy_version 1452323 (0.0008) [2023-12-27 01:57:09,344][105620] Updated weights for policy 1, policy_version 1452333 (0.0009) [2023-12-27 01:57:09,390][105692] Updated weights for policy 0, policy_version 1449975 (0.0008) [2023-12-27 01:57:09,404][105620] Updated weights for policy 1, policy_version 1452343 (0.0008) [2023-12-27 01:57:09,457][105692] Updated weights for policy 0, policy_version 1449985 (0.0006) [2023-12-27 01:57:09,515][105692] Updated weights for policy 0, policy_version 1449995 (0.0006) [2023-12-27 01:57:10,133][105620] Updated weights for policy 1, policy_version 1452353 (0.0007) [2023-12-27 01:57:10,206][105620] Updated weights for policy 1, policy_version 1452363 (0.0009) [2023-12-27 01:57:10,235][105692] Updated weights for policy 0, policy_version 1450005 (0.0006) [2023-12-27 01:57:10,269][105620] Updated weights for policy 1, policy_version 1452373 (0.0008) [2023-12-27 01:57:10,289][105692] Updated weights for policy 0, policy_version 1450015 (0.0008) [2023-12-27 01:57:10,334][105620] Updated weights for policy 1, policy_version 1452383 (0.0006) [2023-12-27 01:57:10,349][105692] Updated weights for policy 0, policy_version 1450025 (0.0007) [2023-12-27 01:57:11,038][105620] Updated weights for policy 1, policy_version 1452393 (0.0008) [2023-12-27 01:57:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 743120896. Throughput: 0: 9956.9, 1: 9871.4. Samples: 743135616. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:57:11,062][104569] Avg episode reward: [(0, '8611.592'), (1, '9170.152')] [2023-12-27 01:57:11,083][105692] Updated weights for policy 0, policy_version 1450035 (0.0009) [2023-12-27 01:57:11,100][105620] Updated weights for policy 1, policy_version 1452403 (0.0008) [2023-12-27 01:57:11,141][105692] Updated weights for policy 0, policy_version 1450045 (0.0007) [2023-12-27 01:57:11,159][105620] Updated weights for policy 1, policy_version 1452413 (0.0008) [2023-12-27 01:57:11,193][105692] Updated weights for policy 0, policy_version 1450055 (0.0009) [2023-12-27 01:57:11,932][105620] Updated weights for policy 1, policy_version 1452423 (0.0008) [2023-12-27 01:57:11,990][105620] Updated weights for policy 1, policy_version 1452433 (0.0008) [2023-12-27 01:57:12,014][105692] Updated weights for policy 0, policy_version 1450065 (0.0009) [2023-12-27 01:57:12,047][105620] Updated weights for policy 1, policy_version 1452443 (0.0007) [2023-12-27 01:57:12,081][105692] Updated weights for policy 0, policy_version 1450075 (0.0007) [2023-12-27 01:57:12,146][105692] Updated weights for policy 0, policy_version 1450085 (0.0010) [2023-12-27 01:57:12,196][105692] Updated weights for policy 0, policy_version 1450095 (0.0008) [2023-12-27 01:57:12,776][105620] Updated weights for policy 1, policy_version 1452453 (0.0008) [2023-12-27 01:57:12,837][105620] Updated weights for policy 1, policy_version 1452463 (0.0005) [2023-12-27 01:57:12,900][105620] Updated weights for policy 1, policy_version 1452473 (0.0006) [2023-12-27 01:57:12,986][105692] Updated weights for policy 0, policy_version 1450105 (0.0009) [2023-12-27 01:57:13,037][105692] Updated weights for policy 0, policy_version 1450115 (0.0009) [2023-12-27 01:57:13,092][105692] Updated weights for policy 0, policy_version 1450125 (0.0009) [2023-12-27 01:57:13,581][105620] Updated weights for policy 1, policy_version 1452483 (0.0007) [2023-12-27 01:57:13,649][105620] Updated weights for policy 1, policy_version 1452493 (0.0010) [2023-12-27 01:57:13,703][105620] Updated weights for policy 1, policy_version 1452503 (0.0010) [2023-12-27 01:57:13,756][105692] Updated weights for policy 0, policy_version 1450135 (0.0006) [2023-12-27 01:57:13,814][105692] Updated weights for policy 0, policy_version 1450145 (0.0005) [2023-12-27 01:57:13,872][105692] Updated weights for policy 0, policy_version 1450155 (0.0005) [2023-12-27 01:57:14,380][105692] Updated weights for policy 0, policy_version 1450165 (0.0005) [2023-12-27 01:57:14,436][105692] Updated weights for policy 0, policy_version 1450175 (0.0005) [2023-12-27 01:57:14,491][105692] Updated weights for policy 0, policy_version 1450185 (0.0005) [2023-12-27 01:57:14,561][105620] Updated weights for policy 1, policy_version 1452513 (0.0008) [2023-12-27 01:57:14,622][105620] Updated weights for policy 1, policy_version 1452523 (0.0006) [2023-12-27 01:57:14,689][105620] Updated weights for policy 1, policy_version 1452533 (0.0005) [2023-12-27 01:57:14,758][105620] Updated weights for policy 1, policy_version 1452543 (0.0005) [2023-12-27 01:57:15,109][105692] Updated weights for policy 0, policy_version 1450195 (0.0007) [2023-12-27 01:57:15,176][105692] Updated weights for policy 0, policy_version 1450205 (0.0011) [2023-12-27 01:57:15,239][105692] Updated weights for policy 0, policy_version 1450215 (0.0010) [2023-12-27 01:57:15,304][105620] Updated weights for policy 1, policy_version 1452553 (0.0010) [2023-12-27 01:57:15,357][105620] Updated weights for policy 1, policy_version 1452563 (0.0011) [2023-12-27 01:57:15,417][105620] Updated weights for policy 1, policy_version 1452573 (0.0011) [2023-12-27 01:57:15,983][105692] Updated weights for policy 0, policy_version 1450225 (0.0011) [2023-12-27 01:57:16,045][105692] Updated weights for policy 0, policy_version 1450235 (0.0010) [2023-12-27 01:57:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 743219200. Throughput: 0: 9906.0, 1: 9779.5. Samples: 743190564. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:57:16,062][104569] Avg episode reward: [(0, '8981.008'), (1, '8988.535')] [2023-12-27 01:57:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001452576_371908608.pth... [2023-12-27 01:57:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001451456_371621888.pth [2023-12-27 01:57:16,103][105692] Updated weights for policy 0, policy_version 1450245 (0.0010) [2023-12-27 01:57:16,165][105692] Updated weights for policy 0, policy_version 1450255 (0.0010) [2023-12-27 01:57:16,168][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001450256_371318784.pth... [2023-12-27 01:57:16,172][105620] Updated weights for policy 1, policy_version 1452583 (0.0007) [2023-12-27 01:57:16,172][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001449072_371015680.pth [2023-12-27 01:57:16,222][105620] Updated weights for policy 1, policy_version 1452593 (0.0005) [2023-12-27 01:57:16,276][105620] Updated weights for policy 1, policy_version 1452603 (0.0006) [2023-12-27 01:57:16,849][105692] Updated weights for policy 0, policy_version 1450265 (0.0010) [2023-12-27 01:57:16,910][105692] Updated weights for policy 0, policy_version 1450275 (0.0010) [2023-12-27 01:57:16,913][105620] Updated weights for policy 1, policy_version 1452613 (0.0007) [2023-12-27 01:57:16,970][105620] Updated weights for policy 1, policy_version 1452623 (0.0009) [2023-12-27 01:57:16,975][105692] Updated weights for policy 0, policy_version 1450285 (0.0008) [2023-12-27 01:57:17,027][105620] Updated weights for policy 1, policy_version 1452633 (0.0009) [2023-12-27 01:57:17,596][105692] Updated weights for policy 0, policy_version 1450295 (0.0005) [2023-12-27 01:57:17,655][105692] Updated weights for policy 0, policy_version 1450305 (0.0005) [2023-12-27 01:57:17,712][105692] Updated weights for policy 0, policy_version 1450315 (0.0006) [2023-12-27 01:57:17,819][105620] Updated weights for policy 1, policy_version 1452643 (0.0008) [2023-12-27 01:57:17,870][105620] Updated weights for policy 1, policy_version 1452653 (0.0009) [2023-12-27 01:57:17,938][105620] Updated weights for policy 1, policy_version 1452663 (0.0005) [2023-12-27 01:57:18,246][105692] Updated weights for policy 0, policy_version 1450325 (0.0005) [2023-12-27 01:57:18,299][105692] Updated weights for policy 0, policy_version 1450335 (0.0005) [2023-12-27 01:57:18,378][105692] Updated weights for policy 0, policy_version 1450345 (0.0008) [2023-12-27 01:57:18,710][105620] Updated weights for policy 1, policy_version 1452673 (0.0006) [2023-12-27 01:57:18,757][105620] Updated weights for policy 1, policy_version 1452683 (0.0008) [2023-12-27 01:57:18,817][105620] Updated weights for policy 1, policy_version 1452693 (0.0008) [2023-12-27 01:57:18,879][105620] Updated weights for policy 1, policy_version 1452703 (0.0008) [2023-12-27 01:57:19,035][105692] Updated weights for policy 0, policy_version 1450355 (0.0007) [2023-12-27 01:57:19,088][105692] Updated weights for policy 0, policy_version 1450365 (0.0010) [2023-12-27 01:57:19,143][105692] Updated weights for policy 0, policy_version 1450375 (0.0010) [2023-12-27 01:57:19,638][105620] Updated weights for policy 1, policy_version 1452713 (0.0008) [2023-12-27 01:57:19,702][105620] Updated weights for policy 1, policy_version 1452723 (0.0008) [2023-12-27 01:57:19,757][105620] Updated weights for policy 1, policy_version 1452733 (0.0008) [2023-12-27 01:57:19,935][105692] Updated weights for policy 0, policy_version 1450385 (0.0010) [2023-12-27 01:57:19,996][105692] Updated weights for policy 0, policy_version 1450395 (0.0011) [2023-12-27 01:57:20,057][105692] Updated weights for policy 0, policy_version 1450405 (0.0011) [2023-12-27 01:57:20,118][105692] Updated weights for policy 0, policy_version 1450415 (0.0011) [2023-12-27 01:57:20,633][105620] Updated weights for policy 1, policy_version 1452743 (0.0009) [2023-12-27 01:57:20,697][105620] Updated weights for policy 1, policy_version 1452753 (0.0009) [2023-12-27 01:57:20,749][105692] Updated weights for policy 0, policy_version 1450425 (0.0008) [2023-12-27 01:57:20,756][105620] Updated weights for policy 1, policy_version 1452763 (0.0009) [2023-12-27 01:57:20,810][105692] Updated weights for policy 0, policy_version 1450435 (0.0009) [2023-12-27 01:57:20,862][105692] Updated weights for policy 0, policy_version 1450445 (0.0009) [2023-12-27 01:57:21,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 743325696. Throughput: 0: 9971.5, 1: 9790.6. Samples: 743312132. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:57:21,063][104569] Avg episode reward: [(0, '8797.504'), (1, '8895.795')] [2023-12-27 01:57:21,528][105620] Updated weights for policy 1, policy_version 1452773 (0.0008) [2023-12-27 01:57:21,585][105620] Updated weights for policy 1, policy_version 1452783 (0.0006) [2023-12-27 01:57:21,654][105620] Updated weights for policy 1, policy_version 1452793 (0.0006) [2023-12-27 01:57:21,673][105692] Updated weights for policy 0, policy_version 1450455 (0.0009) [2023-12-27 01:57:21,737][105692] Updated weights for policy 0, policy_version 1450465 (0.0008) [2023-12-27 01:57:21,803][105692] Updated weights for policy 0, policy_version 1450475 (0.0010) [2023-12-27 01:57:22,322][105620] Updated weights for policy 1, policy_version 1452803 (0.0007) [2023-12-27 01:57:22,389][105620] Updated weights for policy 1, policy_version 1452813 (0.0007) [2023-12-27 01:57:22,455][105620] Updated weights for policy 1, policy_version 1452823 (0.0008) [2023-12-27 01:57:22,558][105692] Updated weights for policy 0, policy_version 1450485 (0.0009) [2023-12-27 01:57:22,615][105692] Updated weights for policy 0, policy_version 1450495 (0.0007) [2023-12-27 01:57:22,676][105692] Updated weights for policy 0, policy_version 1450505 (0.0010) [2023-12-27 01:57:23,028][105620] Updated weights for policy 1, policy_version 1452833 (0.0006) [2023-12-27 01:57:23,078][105620] Updated weights for policy 1, policy_version 1452843 (0.0005) [2023-12-27 01:57:23,130][105620] Updated weights for policy 1, policy_version 1452853 (0.0008) [2023-12-27 01:57:23,181][105620] Updated weights for policy 1, policy_version 1452863 (0.0010) [2023-12-27 01:57:23,476][105692] Updated weights for policy 0, policy_version 1450515 (0.0008) [2023-12-27 01:57:23,539][105585] KL-divergence is very high: 197.6595 [2023-12-27 01:57:23,540][105692] Updated weights for policy 0, policy_version 1450525 (0.0008) [2023-12-27 01:57:23,582][105585] KL-divergence is very high: 370.7715 [2023-12-27 01:57:23,591][105692] Updated weights for policy 0, policy_version 1450535 (0.0009) [2023-12-27 01:57:23,621][105585] KL-divergence is very high: 413.0571 [2023-12-27 01:57:23,868][105620] Updated weights for policy 1, policy_version 1452873 (0.0007) [2023-12-27 01:57:23,912][105620] Updated weights for policy 1, policy_version 1452883 (0.0005) [2023-12-27 01:57:23,964][105620] Updated weights for policy 1, policy_version 1452893 (0.0006) [2023-12-27 01:57:24,427][105692] Updated weights for policy 0, policy_version 1450545 (0.0010) [2023-12-27 01:57:24,481][105692] Updated weights for policy 0, policy_version 1450556 (0.0010) [2023-12-27 01:57:24,540][105692] Updated weights for policy 0, policy_version 1450567 (0.0010) [2023-12-27 01:57:24,543][105620] Updated weights for policy 1, policy_version 1452903 (0.0008) [2023-12-27 01:57:24,597][105620] Updated weights for policy 1, policy_version 1452913 (0.0008) [2023-12-27 01:57:24,659][105620] Updated weights for policy 1, policy_version 1452923 (0.0007) [2023-12-27 01:57:25,222][105692] Updated weights for policy 0, policy_version 1450577 (0.0009) [2023-12-27 01:57:25,273][105692] Updated weights for policy 0, policy_version 1450587 (0.0005) [2023-12-27 01:57:25,277][105620] Updated weights for policy 1, policy_version 1452933 (0.0009) [2023-12-27 01:57:25,320][105692] Updated weights for policy 0, policy_version 1450597 (0.0005) [2023-12-27 01:57:25,323][105620] Updated weights for policy 1, policy_version 1452943 (0.0005) [2023-12-27 01:57:25,371][105692] Updated weights for policy 0, policy_version 1450607 (0.0005) [2023-12-27 01:57:25,387][105620] Updated weights for policy 1, policy_version 1452953 (0.0005) [2023-12-27 01:57:25,978][105692] Updated weights for policy 0, policy_version 1450618 (0.0009) [2023-12-27 01:57:25,984][105620] Updated weights for policy 1, policy_version 1452963 (0.0006) [2023-12-27 01:57:26,033][105692] Updated weights for policy 0, policy_version 1450628 (0.0007) [2023-12-27 01:57:26,042][105620] Updated weights for policy 1, policy_version 1452973 (0.0007) [2023-12-27 01:57:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 743415808. Throughput: 0: 9943.5, 1: 9807.8. Samples: 743431128. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:57:26,062][104569] Avg episode reward: [(0, '8800.528'), (1, '8985.497')] [2023-12-27 01:57:26,085][105692] Updated weights for policy 0, policy_version 1450638 (0.0008) [2023-12-27 01:57:26,099][105620] Updated weights for policy 1, policy_version 1452983 (0.0006) [2023-12-27 01:57:26,699][105620] Updated weights for policy 1, policy_version 1452993 (0.0009) [2023-12-27 01:57:26,745][105620] Updated weights for policy 1, policy_version 1453003 (0.0008) [2023-12-27 01:57:26,791][105620] Updated weights for policy 1, policy_version 1453013 (0.0008) [2023-12-27 01:57:26,840][105620] Updated weights for policy 1, policy_version 1453023 (0.0008) [2023-12-27 01:57:26,898][105692] Updated weights for policy 0, policy_version 1450648 (0.0008) [2023-12-27 01:57:26,945][105692] Updated weights for policy 0, policy_version 1450659 (0.0008) [2023-12-27 01:57:26,991][105692] Updated weights for policy 0, policy_version 1450669 (0.0009) [2023-12-27 01:57:27,544][105620] Updated weights for policy 1, policy_version 1453033 (0.0008) [2023-12-27 01:57:27,596][105620] Updated weights for policy 1, policy_version 1453043 (0.0009) [2023-12-27 01:57:27,651][105620] Updated weights for policy 1, policy_version 1453053 (0.0010) [2023-12-27 01:57:27,785][105692] Updated weights for policy 0, policy_version 1450679 (0.0006) [2023-12-27 01:57:27,841][105692] Updated weights for policy 0, policy_version 1450689 (0.0008) [2023-12-27 01:57:27,893][105692] Updated weights for policy 0, policy_version 1450699 (0.0009) [2023-12-27 01:57:28,372][105620] Updated weights for policy 1, policy_version 1453063 (0.0010) [2023-12-27 01:57:28,431][105620] Updated weights for policy 1, policy_version 1453073 (0.0010) [2023-12-27 01:57:28,496][105620] Updated weights for policy 1, policy_version 1453083 (0.0010) [2023-12-27 01:57:28,543][105692] Updated weights for policy 0, policy_version 1450709 (0.0008) [2023-12-27 01:57:28,588][105692] Updated weights for policy 0, policy_version 1450719 (0.0010) [2023-12-27 01:57:28,640][105692] Updated weights for policy 0, policy_version 1450729 (0.0010) [2023-12-27 01:57:29,250][105620] Updated weights for policy 1, policy_version 1453093 (0.0011) [2023-12-27 01:57:29,313][105620] Updated weights for policy 1, policy_version 1453103 (0.0011) [2023-12-27 01:57:29,374][105692] Updated weights for policy 0, policy_version 1450739 (0.0010) [2023-12-27 01:57:29,384][105620] Updated weights for policy 1, policy_version 1453113 (0.0012) [2023-12-27 01:57:29,434][105692] Updated weights for policy 0, policy_version 1450749 (0.0007) [2023-12-27 01:57:29,495][105692] Updated weights for policy 0, policy_version 1450759 (0.0005) [2023-12-27 01:57:30,102][105620] Updated weights for policy 1, policy_version 1453123 (0.0009) [2023-12-27 01:57:30,154][105620] Updated weights for policy 1, policy_version 1453133 (0.0007) [2023-12-27 01:57:30,165][105692] Updated weights for policy 0, policy_version 1450769 (0.0008) [2023-12-27 01:57:30,201][105620] Updated weights for policy 1, policy_version 1453143 (0.0008) [2023-12-27 01:57:30,229][105692] Updated weights for policy 0, policy_version 1450779 (0.0006) [2023-12-27 01:57:30,292][105692] Updated weights for policy 0, policy_version 1450789 (0.0010) [2023-12-27 01:57:30,358][105692] Updated weights for policy 0, policy_version 1450799 (0.0011) [2023-12-27 01:57:30,942][105692] Updated weights for policy 0, policy_version 1450809 (0.0011) [2023-12-27 01:57:30,996][105692] Updated weights for policy 0, policy_version 1450819 (0.0010) [2023-12-27 01:57:31,010][105620] Updated weights for policy 1, policy_version 1453153 (0.0008) [2023-12-27 01:57:31,055][105692] Updated weights for policy 0, policy_version 1450829 (0.0010) [2023-12-27 01:57:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 743514112. Throughput: 0: 9940.6, 1: 9907.3. Samples: 743490552. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:57:31,063][104569] Avg episode reward: [(0, '8527.805'), (1, '9170.142')] [2023-12-27 01:57:31,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001450832_371466240.pth... [2023-12-27 01:57:31,073][105620] Updated weights for policy 1, policy_version 1453163 (0.0006) [2023-12-27 01:57:31,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001449648_371163136.pth [2023-12-27 01:57:31,134][105620] Updated weights for policy 1, policy_version 1453173 (0.0008) [2023-12-27 01:57:31,196][105620] Updated weights for policy 1, policy_version 1453183 (0.0008) [2023-12-27 01:57:31,200][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001453184_372064256.pth... [2023-12-27 01:57:31,205][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001452032_371769344.pth [2023-12-27 01:57:31,829][105692] Updated weights for policy 0, policy_version 1450839 (0.0009) [2023-12-27 01:57:31,851][105620] Updated weights for policy 1, policy_version 1453193 (0.0005) [2023-12-27 01:57:31,886][105692] Updated weights for policy 0, policy_version 1450849 (0.0009) [2023-12-27 01:57:31,908][105620] Updated weights for policy 1, policy_version 1453203 (0.0005) [2023-12-27 01:57:31,917][105585] KL-divergence is very high: 119.3236 [2023-12-27 01:57:31,947][105692] Updated weights for policy 0, policy_version 1450859 (0.0007) [2023-12-27 01:57:31,963][105585] KL-divergence is very high: 141.2826 [2023-12-27 01:57:31,968][105620] Updated weights for policy 1, policy_version 1453213 (0.0008) [2023-12-27 01:57:32,665][105620] Updated weights for policy 1, policy_version 1453223 (0.0008) [2023-12-27 01:57:32,675][105692] Updated weights for policy 0, policy_version 1450869 (0.0008) [2023-12-27 01:57:32,714][105620] Updated weights for policy 1, policy_version 1453233 (0.0010) [2023-12-27 01:57:32,722][105692] Updated weights for policy 0, policy_version 1450879 (0.0006) [2023-12-27 01:57:32,765][105620] Updated weights for policy 1, policy_version 1453243 (0.0010) [2023-12-27 01:57:32,777][105692] Updated weights for policy 0, policy_version 1450889 (0.0006) [2023-12-27 01:57:33,385][105620] Updated weights for policy 1, policy_version 1453253 (0.0008) [2023-12-27 01:57:33,436][105620] Updated weights for policy 1, policy_version 1453263 (0.0010) [2023-12-27 01:57:33,486][105620] Updated weights for policy 1, policy_version 1453273 (0.0010) [2023-12-27 01:57:33,587][105692] Updated weights for policy 0, policy_version 1450899 (0.0007) [2023-12-27 01:57:33,629][105692] Updated weights for policy 0, policy_version 1450909 (0.0006) [2023-12-27 01:57:33,675][105692] Updated weights for policy 0, policy_version 1450919 (0.0008) [2023-12-27 01:57:34,125][105620] Updated weights for policy 1, policy_version 1453283 (0.0010) [2023-12-27 01:57:34,182][105620] Updated weights for policy 1, policy_version 1453293 (0.0009) [2023-12-27 01:57:34,229][105620] Updated weights for policy 1, policy_version 1453303 (0.0008) [2023-12-27 01:57:34,526][105692] Updated weights for policy 0, policy_version 1450929 (0.0008) [2023-12-27 01:57:34,584][105692] Updated weights for policy 0, policy_version 1450939 (0.0009) [2023-12-27 01:57:34,641][105692] Updated weights for policy 0, policy_version 1450949 (0.0010) [2023-12-27 01:57:34,694][105692] Updated weights for policy 0, policy_version 1450959 (0.0009) [2023-12-27 01:57:34,909][105620] Updated weights for policy 1, policy_version 1453313 (0.0009) [2023-12-27 01:57:34,965][105620] Updated weights for policy 1, policy_version 1453323 (0.0008) [2023-12-27 01:57:35,017][105620] Updated weights for policy 1, policy_version 1453333 (0.0005) [2023-12-27 01:57:35,075][105620] Updated weights for policy 1, policy_version 1453343 (0.0005) [2023-12-27 01:57:35,553][105692] Updated weights for policy 0, policy_version 1450969 (0.0008) [2023-12-27 01:57:35,613][105692] Updated weights for policy 0, policy_version 1450979 (0.0009) [2023-12-27 01:57:35,665][105692] Updated weights for policy 0, policy_version 1450989 (0.0005) [2023-12-27 01:57:35,743][105620] Updated weights for policy 1, policy_version 1453353 (0.0010) [2023-12-27 01:57:35,791][105620] Updated weights for policy 1, policy_version 1453363 (0.0010) [2023-12-27 01:57:35,844][105620] Updated weights for policy 1, policy_version 1453373 (0.0010) [2023-12-27 01:57:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 743620608. Throughput: 0: 9872.3, 1: 9911.9. Samples: 743607868. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:57:36,062][104569] Avg episode reward: [(0, '8529.540'), (1, '9077.642')] [2023-12-27 01:57:36,265][105692] Updated weights for policy 0, policy_version 1450999 (0.0009) [2023-12-27 01:57:36,325][105692] Updated weights for policy 0, policy_version 1451009 (0.0011) [2023-12-27 01:57:36,377][105692] Updated weights for policy 0, policy_version 1451019 (0.0011) [2023-12-27 01:57:36,572][105620] Updated weights for policy 1, policy_version 1453383 (0.0010) [2023-12-27 01:57:36,646][105620] Updated weights for policy 1, policy_version 1453393 (0.0010) [2023-12-27 01:57:36,709][105620] Updated weights for policy 1, policy_version 1453403 (0.0006) [2023-12-27 01:57:37,110][105692] Updated weights for policy 0, policy_version 1451029 (0.0011) [2023-12-27 01:57:37,173][105692] Updated weights for policy 0, policy_version 1451039 (0.0011) [2023-12-27 01:57:37,226][105692] Updated weights for policy 0, policy_version 1451049 (0.0011) [2023-12-27 01:57:37,435][105620] Updated weights for policy 1, policy_version 1453413 (0.0009) [2023-12-27 01:57:37,497][105620] Updated weights for policy 1, policy_version 1453423 (0.0008) [2023-12-27 01:57:37,553][105620] Updated weights for policy 1, policy_version 1453433 (0.0008) [2023-12-27 01:57:37,932][105692] Updated weights for policy 0, policy_version 1451059 (0.0009) [2023-12-27 01:57:37,989][105692] Updated weights for policy 0, policy_version 1451069 (0.0005) [2023-12-27 01:57:38,039][105692] Updated weights for policy 0, policy_version 1451079 (0.0008) [2023-12-27 01:57:38,201][105620] Updated weights for policy 1, policy_version 1453443 (0.0007) [2023-12-27 01:57:38,246][105620] Updated weights for policy 1, policy_version 1453453 (0.0008) [2023-12-27 01:57:38,293][105620] Updated weights for policy 1, policy_version 1453463 (0.0008) [2023-12-27 01:57:38,771][105692] Updated weights for policy 0, policy_version 1451089 (0.0009) [2023-12-27 01:57:38,834][105692] Updated weights for policy 0, policy_version 1451099 (0.0010) [2023-12-27 01:57:38,896][105692] Updated weights for policy 0, policy_version 1451109 (0.0009) [2023-12-27 01:57:38,952][105692] Updated weights for policy 0, policy_version 1451119 (0.0009) [2023-12-27 01:57:39,097][105620] Updated weights for policy 1, policy_version 1453473 (0.0009) [2023-12-27 01:57:39,158][105620] Updated weights for policy 1, policy_version 1453483 (0.0009) [2023-12-27 01:57:39,215][105620] Updated weights for policy 1, policy_version 1453493 (0.0009) [2023-12-27 01:57:39,283][105620] Updated weights for policy 1, policy_version 1453503 (0.0006) [2023-12-27 01:57:39,723][105692] Updated weights for policy 0, policy_version 1451129 (0.0009) [2023-12-27 01:57:39,776][105692] Updated weights for policy 0, policy_version 1451139 (0.0009) [2023-12-27 01:57:39,837][105692] Updated weights for policy 0, policy_version 1451149 (0.0009) [2023-12-27 01:57:40,048][105620] Updated weights for policy 1, policy_version 1453513 (0.0010) [2023-12-27 01:57:40,105][105620] Updated weights for policy 1, policy_version 1453523 (0.0008) [2023-12-27 01:57:40,167][105620] Updated weights for policy 1, policy_version 1453533 (0.0009) [2023-12-27 01:57:40,559][105692] Updated weights for policy 0, policy_version 1451159 (0.0008) [2023-12-27 01:57:40,620][105692] Updated weights for policy 0, policy_version 1451169 (0.0005) [2023-12-27 01:57:40,685][105692] Updated weights for policy 0, policy_version 1451179 (0.0009) [2023-12-27 01:57:41,008][105620] Updated weights for policy 1, policy_version 1453543 (0.0009) [2023-12-27 01:57:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 743710720. Throughput: 0: 9811.2, 1: 9785.1. Samples: 743722712. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:57:41,062][104569] Avg episode reward: [(0, '8525.535'), (1, '8899.526')] [2023-12-27 01:57:41,065][105620] Updated weights for policy 1, policy_version 1453553 (0.0009) [2023-12-27 01:57:41,123][105620] Updated weights for policy 1, policy_version 1453563 (0.0010) [2023-12-27 01:57:41,327][105692] Updated weights for policy 0, policy_version 1451189 (0.0009) [2023-12-27 01:57:41,392][105692] Updated weights for policy 0, policy_version 1451199 (0.0009) [2023-12-27 01:57:41,455][105692] Updated weights for policy 0, policy_version 1451209 (0.0008) [2023-12-27 01:57:42,009][105620] Updated weights for policy 1, policy_version 1453573 (0.0010) [2023-12-27 01:57:42,069][105620] Updated weights for policy 1, policy_version 1453583 (0.0011) [2023-12-27 01:57:42,100][105692] Updated weights for policy 0, policy_version 1451219 (0.0006) [2023-12-27 01:57:42,126][105620] Updated weights for policy 1, policy_version 1453593 (0.0011) [2023-12-27 01:57:42,152][105692] Updated weights for policy 0, policy_version 1451229 (0.0005) [2023-12-27 01:57:42,200][105692] Updated weights for policy 0, policy_version 1451239 (0.0008) [2023-12-27 01:57:42,887][105620] Updated weights for policy 1, policy_version 1453603 (0.0010) [2023-12-27 01:57:42,934][105620] Updated weights for policy 1, policy_version 1453613 (0.0009) [2023-12-27 01:57:42,979][105692] Updated weights for policy 0, policy_version 1451249 (0.0008) [2023-12-27 01:57:42,993][105620] Updated weights for policy 1, policy_version 1453623 (0.0008) [2023-12-27 01:57:43,039][105692] Updated weights for policy 0, policy_version 1451259 (0.0008) [2023-12-27 01:57:43,097][105692] Updated weights for policy 0, policy_version 1451269 (0.0006) [2023-12-27 01:57:43,155][105692] Updated weights for policy 0, policy_version 1451279 (0.0005) [2023-12-27 01:57:43,680][105620] Updated weights for policy 1, policy_version 1453633 (0.0007) [2023-12-27 01:57:43,737][105620] Updated weights for policy 1, policy_version 1453643 (0.0009) [2023-12-27 01:57:43,800][105620] Updated weights for policy 1, policy_version 1453653 (0.0006) [2023-12-27 01:57:43,849][105620] Updated weights for policy 1, policy_version 1453663 (0.0010) [2023-12-27 01:57:43,878][105692] Updated weights for policy 0, policy_version 1451289 (0.0008) [2023-12-27 01:57:43,927][105692] Updated weights for policy 0, policy_version 1451300 (0.0010) [2023-12-27 01:57:43,982][105692] Updated weights for policy 0, policy_version 1451312 (0.0011) [2023-12-27 01:57:44,494][105620] Updated weights for policy 1, policy_version 1453673 (0.0008) [2023-12-27 01:57:44,558][105620] Updated weights for policy 1, policy_version 1453683 (0.0008) [2023-12-27 01:57:44,626][105620] Updated weights for policy 1, policy_version 1453693 (0.0008) [2023-12-27 01:57:44,713][105692] Updated weights for policy 0, policy_version 1451322 (0.0010) [2023-12-27 01:57:44,777][105692] Updated weights for policy 0, policy_version 1451332 (0.0008) [2023-12-27 01:57:44,838][105692] Updated weights for policy 0, policy_version 1451342 (0.0010) [2023-12-27 01:57:45,218][105620] Updated weights for policy 1, policy_version 1453703 (0.0010) [2023-12-27 01:57:45,282][105620] Updated weights for policy 1, policy_version 1453713 (0.0011) [2023-12-27 01:57:45,346][105620] Updated weights for policy 1, policy_version 1453723 (0.0011) [2023-12-27 01:57:45,683][105692] Updated weights for policy 0, policy_version 1451352 (0.0009) [2023-12-27 01:57:45,750][105692] Updated weights for policy 0, policy_version 1451362 (0.0009) [2023-12-27 01:57:45,812][105692] Updated weights for policy 0, policy_version 1451372 (0.0009) [2023-12-27 01:57:46,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19660.7, 300 sec: 19605.2). Total num frames: 743809024. Throughput: 0: 9791.9, 1: 9722.7. Samples: 743779860. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:57:46,063][104569] Avg episode reward: [(0, '8525.176'), (1, '8623.899')] [2023-12-27 01:57:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001451376_371605504.pth... [2023-12-27 01:57:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001450256_371318784.pth [2023-12-27 01:57:46,095][105620] Updated weights for policy 1, policy_version 1453733 (0.0010) [2023-12-27 01:57:46,146][105620] Updated weights for policy 1, policy_version 1453743 (0.0008) [2023-12-27 01:57:46,193][105620] Updated weights for policy 1, policy_version 1453753 (0.0009) [2023-12-27 01:57:46,223][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001453760_372211712.pth... [2023-12-27 01:57:46,226][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001452576_371908608.pth [2023-12-27 01:57:46,574][105692] Updated weights for policy 0, policy_version 1451382 (0.0008) [2023-12-27 01:57:46,621][105692] Updated weights for policy 0, policy_version 1451392 (0.0009) [2023-12-27 01:57:46,670][105692] Updated weights for policy 0, policy_version 1451403 (0.0009) [2023-12-27 01:57:46,895][105620] Updated weights for policy 1, policy_version 1453763 (0.0008) [2023-12-27 01:57:46,950][105620] Updated weights for policy 1, policy_version 1453773 (0.0006) [2023-12-27 01:57:47,016][105620] Updated weights for policy 1, policy_version 1453783 (0.0009) [2023-12-27 01:57:47,485][105692] Updated weights for policy 0, policy_version 1451413 (0.0009) [2023-12-27 01:57:47,549][105692] Updated weights for policy 0, policy_version 1451423 (0.0009) [2023-12-27 01:57:47,609][105692] Updated weights for policy 0, policy_version 1451433 (0.0009) [2023-12-27 01:57:47,677][105620] Updated weights for policy 1, policy_version 1453793 (0.0008) [2023-12-27 01:57:47,750][105620] Updated weights for policy 1, policy_version 1453803 (0.0006) [2023-12-27 01:57:47,810][105620] Updated weights for policy 1, policy_version 1453813 (0.0009) [2023-12-27 01:57:47,866][105620] Updated weights for policy 1, policy_version 1453823 (0.0010) [2023-12-27 01:57:48,285][105692] Updated weights for policy 0, policy_version 1451443 (0.0008) [2023-12-27 01:57:48,345][105692] Updated weights for policy 0, policy_version 1451453 (0.0008) [2023-12-27 01:57:48,407][105692] Updated weights for policy 0, policy_version 1451463 (0.0008) [2023-12-27 01:57:48,543][105620] Updated weights for policy 1, policy_version 1453833 (0.0010) [2023-12-27 01:57:48,608][105620] Updated weights for policy 1, policy_version 1453843 (0.0010) [2023-12-27 01:57:48,674][105620] Updated weights for policy 1, policy_version 1453853 (0.0010) [2023-12-27 01:57:49,190][105692] Updated weights for policy 0, policy_version 1451473 (0.0008) [2023-12-27 01:57:49,257][105692] Updated weights for policy 0, policy_version 1451483 (0.0008) [2023-12-27 01:57:49,321][105692] Updated weights for policy 0, policy_version 1451493 (0.0009) [2023-12-27 01:57:49,386][105692] Updated weights for policy 0, policy_version 1451503 (0.0008) [2023-12-27 01:57:49,412][105620] Updated weights for policy 1, policy_version 1453863 (0.0010) [2023-12-27 01:57:49,469][105620] Updated weights for policy 1, policy_version 1453873 (0.0010) [2023-12-27 01:57:49,530][105620] Updated weights for policy 1, policy_version 1453883 (0.0009) [2023-12-27 01:57:50,177][105692] Updated weights for policy 0, policy_version 1451513 (0.0006) [2023-12-27 01:57:50,202][105620] Updated weights for policy 1, policy_version 1453893 (0.0006) [2023-12-27 01:57:50,240][105692] Updated weights for policy 0, policy_version 1451523 (0.0007) [2023-12-27 01:57:50,266][105620] Updated weights for policy 1, policy_version 1453903 (0.0005) [2023-12-27 01:57:50,301][105692] Updated weights for policy 0, policy_version 1451533 (0.0007) [2023-12-27 01:57:50,329][105620] Updated weights for policy 1, policy_version 1453913 (0.0007) [2023-12-27 01:57:50,995][105692] Updated weights for policy 0, policy_version 1451543 (0.0009) [2023-12-27 01:57:51,027][105620] Updated weights for policy 1, policy_version 1453923 (0.0009) [2023-12-27 01:57:51,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 743899136. Throughput: 0: 9686.8, 1: 9843.4. Samples: 743895184. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:57:51,063][104569] Avg episode reward: [(0, '9075.425'), (1, '8986.794')] [2023-12-27 01:57:51,069][105692] Updated weights for policy 0, policy_version 1451553 (0.0011) [2023-12-27 01:57:51,087][105620] Updated weights for policy 1, policy_version 1453933 (0.0007) [2023-12-27 01:57:51,133][105692] Updated weights for policy 0, policy_version 1451563 (0.0010) [2023-12-27 01:57:51,156][105620] Updated weights for policy 1, policy_version 1453943 (0.0006) [2023-12-27 01:57:51,895][105620] Updated weights for policy 1, policy_version 1453953 (0.0006) [2023-12-27 01:57:51,917][105692] Updated weights for policy 0, policy_version 1451573 (0.0010) [2023-12-27 01:57:51,966][105620] Updated weights for policy 1, policy_version 1453963 (0.0006) [2023-12-27 01:57:51,983][105692] Updated weights for policy 0, policy_version 1451583 (0.0009) [2023-12-27 01:57:52,031][105620] Updated weights for policy 1, policy_version 1453973 (0.0006) [2023-12-27 01:57:52,047][105692] Updated weights for policy 0, policy_version 1451593 (0.0008) [2023-12-27 01:57:52,100][105620] Updated weights for policy 1, policy_version 1453983 (0.0005) [2023-12-27 01:57:52,737][105692] Updated weights for policy 0, policy_version 1451603 (0.0009) [2023-12-27 01:57:52,763][105620] Updated weights for policy 1, policy_version 1453993 (0.0007) [2023-12-27 01:57:52,796][105692] Updated weights for policy 0, policy_version 1451613 (0.0009) [2023-12-27 01:57:52,822][105620] Updated weights for policy 1, policy_version 1454003 (0.0006) [2023-12-27 01:57:52,846][105692] Updated weights for policy 0, policy_version 1451623 (0.0008) [2023-12-27 01:57:52,881][105620] Updated weights for policy 1, policy_version 1454013 (0.0006) [2023-12-27 01:57:53,553][105620] Updated weights for policy 1, policy_version 1454023 (0.0008) [2023-12-27 01:57:53,603][105620] Updated weights for policy 1, policy_version 1454033 (0.0008) [2023-12-27 01:57:53,631][105692] Updated weights for policy 0, policy_version 1451633 (0.0008) [2023-12-27 01:57:53,660][105620] Updated weights for policy 1, policy_version 1454043 (0.0006) [2023-12-27 01:57:53,682][105692] Updated weights for policy 0, policy_version 1451643 (0.0010) [2023-12-27 01:57:53,739][105692] Updated weights for policy 0, policy_version 1451653 (0.0010) [2023-12-27 01:57:53,811][105692] Updated weights for policy 0, policy_version 1451663 (0.0005) [2023-12-27 01:57:54,413][105620] Updated weights for policy 1, policy_version 1454053 (0.0007) [2023-12-27 01:57:54,413][105692] Updated weights for policy 0, policy_version 1451673 (0.0007) [2023-12-27 01:57:54,458][105620] Updated weights for policy 1, policy_version 1454063 (0.0007) [2023-12-27 01:57:54,475][105692] Updated weights for policy 0, policy_version 1451683 (0.0010) [2023-12-27 01:57:54,517][105620] Updated weights for policy 1, policy_version 1454073 (0.0007) [2023-12-27 01:57:54,534][105692] Updated weights for policy 0, policy_version 1451693 (0.0010) [2023-12-27 01:57:55,199][105620] Updated weights for policy 1, policy_version 1454083 (0.0005) [2023-12-27 01:57:55,207][105692] Updated weights for policy 0, policy_version 1451703 (0.0008) [2023-12-27 01:57:55,265][105620] Updated weights for policy 1, policy_version 1454093 (0.0008) [2023-12-27 01:57:55,267][105692] Updated weights for policy 0, policy_version 1451713 (0.0007) [2023-12-27 01:57:55,322][105692] Updated weights for policy 0, policy_version 1451723 (0.0008) [2023-12-27 01:57:55,327][105620] Updated weights for policy 1, policy_version 1454103 (0.0010) [2023-12-27 01:57:55,903][105692] Updated weights for policy 0, policy_version 1451733 (0.0010) [2023-12-27 01:57:55,948][105620] Updated weights for policy 1, policy_version 1454113 (0.0010) [2023-12-27 01:57:55,955][105692] Updated weights for policy 0, policy_version 1451743 (0.0006) [2023-12-27 01:57:56,007][105620] Updated weights for policy 1, policy_version 1454123 (0.0005) [2023-12-27 01:57:56,011][105692] Updated weights for policy 0, policy_version 1451753 (0.0005) [2023-12-27 01:57:56,058][105620] Updated weights for policy 1, policy_version 1454133 (0.0005) [2023-12-27 01:57:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 744005632. Throughput: 0: 9666.8, 1: 9837.5. Samples: 744013312. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:57:56,063][104569] Avg episode reward: [(0, '8803.222'), (1, '8988.507')] [2023-12-27 01:57:56,104][105620] Updated weights for policy 1, policy_version 1454143 (0.0007) [2023-12-27 01:57:56,568][105692] Updated weights for policy 0, policy_version 1451763 (0.0008) [2023-12-27 01:57:56,628][105692] Updated weights for policy 0, policy_version 1451773 (0.0009) [2023-12-27 01:57:56,682][105692] Updated weights for policy 0, policy_version 1451783 (0.0008) [2023-12-27 01:57:56,747][105620] Updated weights for policy 1, policy_version 1454153 (0.0006) [2023-12-27 01:57:56,794][105620] Updated weights for policy 1, policy_version 1454163 (0.0006) [2023-12-27 01:57:56,841][105620] Updated weights for policy 1, policy_version 1454173 (0.0009) [2023-12-27 01:57:57,425][105692] Updated weights for policy 0, policy_version 1451793 (0.0008) [2023-12-27 01:57:57,483][105692] Updated weights for policy 0, policy_version 1451803 (0.0009) [2023-12-27 01:57:57,501][105620] Updated weights for policy 1, policy_version 1454183 (0.0006) [2023-12-27 01:57:57,534][105692] Updated weights for policy 0, policy_version 1451813 (0.0010) [2023-12-27 01:57:57,556][105620] Updated weights for policy 1, policy_version 1454193 (0.0006) [2023-12-27 01:57:57,583][105692] Updated weights for policy 0, policy_version 1451823 (0.0010) [2023-12-27 01:57:57,612][105620] Updated weights for policy 1, policy_version 1454203 (0.0009) [2023-12-27 01:57:58,239][105692] Updated weights for policy 0, policy_version 1451833 (0.0011) [2023-12-27 01:57:58,298][105692] Updated weights for policy 0, policy_version 1451843 (0.0011) [2023-12-27 01:57:58,338][105620] Updated weights for policy 1, policy_version 1454213 (0.0010) [2023-12-27 01:57:58,365][105692] Updated weights for policy 0, policy_version 1451853 (0.0010) [2023-12-27 01:57:58,405][105620] Updated weights for policy 1, policy_version 1454223 (0.0009) [2023-12-27 01:57:58,469][105620] Updated weights for policy 1, policy_version 1454233 (0.0008) [2023-12-27 01:57:59,051][105692] Updated weights for policy 0, policy_version 1451863 (0.0010) [2023-12-27 01:57:59,113][105692] Updated weights for policy 0, policy_version 1451873 (0.0010) [2023-12-27 01:57:59,168][105692] Updated weights for policy 0, policy_version 1451883 (0.0010) [2023-12-27 01:57:59,278][105620] Updated weights for policy 1, policy_version 1454243 (0.0008) [2023-12-27 01:57:59,330][105620] Updated weights for policy 1, policy_version 1454253 (0.0008) [2023-12-27 01:57:59,395][105620] Updated weights for policy 1, policy_version 1454263 (0.0009) [2023-12-27 01:57:59,867][105692] Updated weights for policy 0, policy_version 1451893 (0.0009) [2023-12-27 01:57:59,923][105692] Updated weights for policy 0, policy_version 1451903 (0.0009) [2023-12-27 01:57:59,991][105692] Updated weights for policy 0, policy_version 1451913 (0.0008) [2023-12-27 01:58:00,235][105620] Updated weights for policy 1, policy_version 1454273 (0.0010) [2023-12-27 01:58:00,291][105620] Updated weights for policy 1, policy_version 1454283 (0.0008) [2023-12-27 01:58:00,354][105620] Updated weights for policy 1, policy_version 1454293 (0.0005) [2023-12-27 01:58:00,410][105620] Updated weights for policy 1, policy_version 1454303 (0.0005) [2023-12-27 01:58:00,692][105692] Updated weights for policy 0, policy_version 1451923 (0.0007) [2023-12-27 01:58:00,759][105692] Updated weights for policy 0, policy_version 1451933 (0.0007) [2023-12-27 01:58:00,831][105692] Updated weights for policy 0, policy_version 1451943 (0.0009) [2023-12-27 01:58:01,015][105620] Updated weights for policy 1, policy_version 1454313 (0.0006) [2023-12-27 01:58:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.2, 300 sec: 19660.8). Total num frames: 744103936. Throughput: 0: 9768.6, 1: 9885.1. Samples: 744074984. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:58:01,063][104569] Avg episode reward: [(0, '8803.709'), (1, '9168.642')] [2023-12-27 01:58:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001451952_371752960.pth... [2023-12-27 01:58:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001450832_371466240.pth [2023-12-27 01:58:01,082][105620] Updated weights for policy 1, policy_version 1454323 (0.0009) [2023-12-27 01:58:01,140][105620] Updated weights for policy 1, policy_version 1454333 (0.0009) [2023-12-27 01:58:01,158][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001454336_372359168.pth... [2023-12-27 01:58:01,163][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001453184_372064256.pth [2023-12-27 01:58:01,496][105692] Updated weights for policy 0, policy_version 1451953 (0.0007) [2023-12-27 01:58:01,554][105692] Updated weights for policy 0, policy_version 1451963 (0.0010) [2023-12-27 01:58:01,618][105692] Updated weights for policy 0, policy_version 1451973 (0.0010) [2023-12-27 01:58:01,680][105692] Updated weights for policy 0, policy_version 1451983 (0.0008) [2023-12-27 01:58:01,878][105620] Updated weights for policy 1, policy_version 1454343 (0.0008) [2023-12-27 01:58:01,939][105620] Updated weights for policy 1, policy_version 1454353 (0.0006) [2023-12-27 01:58:02,008][105620] Updated weights for policy 1, policy_version 1454363 (0.0007) [2023-12-27 01:58:02,473][105692] Updated weights for policy 0, policy_version 1451993 (0.0006) [2023-12-27 01:58:02,526][105692] Updated weights for policy 0, policy_version 1452003 (0.0006) [2023-12-27 01:58:02,578][105692] Updated weights for policy 0, policy_version 1452013 (0.0005) [2023-12-27 01:58:02,749][105620] Updated weights for policy 1, policy_version 1454373 (0.0007) [2023-12-27 01:58:02,798][105620] Updated weights for policy 1, policy_version 1454383 (0.0008) [2023-12-27 01:58:02,851][105620] Updated weights for policy 1, policy_version 1454393 (0.0008) [2023-12-27 01:58:03,253][105692] Updated weights for policy 0, policy_version 1452023 (0.0007) [2023-12-27 01:58:03,297][105692] Updated weights for policy 0, policy_version 1452033 (0.0008) [2023-12-27 01:58:03,342][105692] Updated weights for policy 0, policy_version 1452043 (0.0008) [2023-12-27 01:58:03,575][105620] Updated weights for policy 1, policy_version 1454403 (0.0009) [2023-12-27 01:58:03,629][105620] Updated weights for policy 1, policy_version 1454413 (0.0009) [2023-12-27 01:58:03,679][105620] Updated weights for policy 1, policy_version 1454423 (0.0009) [2023-12-27 01:58:04,176][105692] Updated weights for policy 0, policy_version 1452054 (0.0009) [2023-12-27 01:58:04,230][105692] Updated weights for policy 0, policy_version 1452064 (0.0009) [2023-12-27 01:58:04,280][105692] Updated weights for policy 0, policy_version 1452074 (0.0008) [2023-12-27 01:58:04,428][105620] Updated weights for policy 1, policy_version 1454433 (0.0010) [2023-12-27 01:58:04,495][105620] Updated weights for policy 1, policy_version 1454443 (0.0010) [2023-12-27 01:58:04,561][105620] Updated weights for policy 1, policy_version 1454453 (0.0010) [2023-12-27 01:58:04,616][105620] Updated weights for policy 1, policy_version 1454463 (0.0010) [2023-12-27 01:58:05,040][105692] Updated weights for policy 0, policy_version 1452084 (0.0007) [2023-12-27 01:58:05,092][105692] Updated weights for policy 0, policy_version 1452094 (0.0005) [2023-12-27 01:58:05,145][105692] Updated weights for policy 0, policy_version 1452104 (0.0005) [2023-12-27 01:58:05,243][105620] Updated weights for policy 1, policy_version 1454473 (0.0007) [2023-12-27 01:58:05,301][105620] Updated weights for policy 1, policy_version 1454483 (0.0010) [2023-12-27 01:58:05,353][105620] Updated weights for policy 1, policy_version 1454494 (0.0009) [2023-12-27 01:58:05,759][105692] Updated weights for policy 0, policy_version 1452114 (0.0007) [2023-12-27 01:58:05,813][105692] Updated weights for policy 0, policy_version 1452126 (0.0010) [2023-12-27 01:58:05,867][105692] Updated weights for policy 0, policy_version 1452137 (0.0010) [2023-12-27 01:58:06,032][105620] Updated weights for policy 1, policy_version 1454504 (0.0006) [2023-12-27 01:58:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 744202240. Throughput: 0: 9634.5, 1: 9870.8. Samples: 744189868. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:58:06,062][104569] Avg episode reward: [(0, '8615.625'), (1, '9078.621')] [2023-12-27 01:58:06,097][105620] Updated weights for policy 1, policy_version 1454514 (0.0010) [2023-12-27 01:58:06,162][105620] Updated weights for policy 1, policy_version 1454524 (0.0009) [2023-12-27 01:58:06,548][105692] Updated weights for policy 0, policy_version 1452148 (0.0008) [2023-12-27 01:58:06,610][105692] Updated weights for policy 0, policy_version 1452158 (0.0009) [2023-12-27 01:58:06,665][105692] Updated weights for policy 0, policy_version 1452168 (0.0006) [2023-12-27 01:58:06,859][105620] Updated weights for policy 1, policy_version 1454534 (0.0011) [2023-12-27 01:58:06,926][105620] Updated weights for policy 1, policy_version 1454544 (0.0011) [2023-12-27 01:58:06,992][105620] Updated weights for policy 1, policy_version 1454554 (0.0011) [2023-12-27 01:58:07,379][105692] Updated weights for policy 0, policy_version 1452178 (0.0007) [2023-12-27 01:58:07,438][105692] Updated weights for policy 0, policy_version 1452188 (0.0007) [2023-12-27 01:58:07,491][105692] Updated weights for policy 0, policy_version 1452198 (0.0009) [2023-12-27 01:58:07,543][105692] Updated weights for policy 0, policy_version 1452208 (0.0008) [2023-12-27 01:58:07,826][105620] Updated weights for policy 1, policy_version 1454564 (0.0009) [2023-12-27 01:58:07,887][105620] Updated weights for policy 1, policy_version 1454574 (0.0009) [2023-12-27 01:58:07,938][105620] Updated weights for policy 1, policy_version 1454584 (0.0010) [2023-12-27 01:58:08,115][105692] Updated weights for policy 0, policy_version 1452218 (0.0006) [2023-12-27 01:58:08,174][105692] Updated weights for policy 0, policy_version 1452229 (0.0010) [2023-12-27 01:58:08,234][105692] Updated weights for policy 0, policy_version 1452240 (0.0011) [2023-12-27 01:58:08,580][105620] Updated weights for policy 1, policy_version 1454594 (0.0010) [2023-12-27 01:58:08,628][105620] Updated weights for policy 1, policy_version 1454604 (0.0010) [2023-12-27 01:58:08,676][105620] Updated weights for policy 1, policy_version 1454614 (0.0010) [2023-12-27 01:58:08,732][105620] Updated weights for policy 1, policy_version 1454624 (0.0010) [2023-12-27 01:58:09,051][105692] Updated weights for policy 0, policy_version 1452250 (0.0008) [2023-12-27 01:58:09,112][105692] Updated weights for policy 0, policy_version 1452260 (0.0008) [2023-12-27 01:58:09,171][105692] Updated weights for policy 0, policy_version 1452270 (0.0008) [2023-12-27 01:58:09,451][105620] Updated weights for policy 1, policy_version 1454634 (0.0011) [2023-12-27 01:58:09,511][105620] Updated weights for policy 1, policy_version 1454644 (0.0009) [2023-12-27 01:58:09,581][105620] Updated weights for policy 1, policy_version 1454654 (0.0011) [2023-12-27 01:58:09,946][105692] Updated weights for policy 0, policy_version 1452280 (0.0008) [2023-12-27 01:58:10,003][105692] Updated weights for policy 0, policy_version 1452290 (0.0006) [2023-12-27 01:58:10,056][105692] Updated weights for policy 0, policy_version 1452300 (0.0005) [2023-12-27 01:58:10,337][105620] Updated weights for policy 1, policy_version 1454664 (0.0007) [2023-12-27 01:58:10,392][105620] Updated weights for policy 1, policy_version 1454674 (0.0010) [2023-12-27 01:58:10,439][105620] Updated weights for policy 1, policy_version 1454684 (0.0009) [2023-12-27 01:58:10,821][105692] Updated weights for policy 0, policy_version 1452310 (0.0007) [2023-12-27 01:58:10,878][105692] Updated weights for policy 0, policy_version 1452320 (0.0008) [2023-12-27 01:58:10,930][105692] Updated weights for policy 0, policy_version 1452330 (0.0008) [2023-12-27 01:58:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 744300544. Throughput: 0: 9695.5, 1: 9813.2. Samples: 744309024. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:58:11,063][104569] Avg episode reward: [(0, '8527.289'), (1, '8895.495')] [2023-12-27 01:58:11,130][105620] Updated weights for policy 1, policy_version 1454694 (0.0010) [2023-12-27 01:58:11,197][105620] Updated weights for policy 1, policy_version 1454704 (0.0011) [2023-12-27 01:58:11,264][105620] Updated weights for policy 1, policy_version 1454714 (0.0011) [2023-12-27 01:58:11,703][105692] Updated weights for policy 0, policy_version 1452340 (0.0008) [2023-12-27 01:58:11,767][105692] Updated weights for policy 0, policy_version 1452350 (0.0008) [2023-12-27 01:58:11,823][105692] Updated weights for policy 0, policy_version 1452360 (0.0006) [2023-12-27 01:58:12,052][105620] Updated weights for policy 1, policy_version 1454724 (0.0011) [2023-12-27 01:58:12,104][105620] Updated weights for policy 1, policy_version 1454734 (0.0010) [2023-12-27 01:58:12,158][105620] Updated weights for policy 1, policy_version 1454744 (0.0010) [2023-12-27 01:58:12,532][105692] Updated weights for policy 0, policy_version 1452370 (0.0005) [2023-12-27 01:58:12,596][105692] Updated weights for policy 0, policy_version 1452380 (0.0006) [2023-12-27 01:58:12,655][105692] Updated weights for policy 0, policy_version 1452390 (0.0008) [2023-12-27 01:58:12,715][105692] Updated weights for policy 0, policy_version 1452400 (0.0009) [2023-12-27 01:58:12,901][105620] Updated weights for policy 1, policy_version 1454754 (0.0007) [2023-12-27 01:58:12,963][105620] Updated weights for policy 1, policy_version 1454764 (0.0010) [2023-12-27 01:58:13,014][105620] Updated weights for policy 1, policy_version 1454774 (0.0009) [2023-12-27 01:58:13,060][105620] Updated weights for policy 1, policy_version 1454784 (0.0008) [2023-12-27 01:58:13,399][105692] Updated weights for policy 0, policy_version 1452410 (0.0010) [2023-12-27 01:58:13,457][105692] Updated weights for policy 0, policy_version 1452420 (0.0010) [2023-12-27 01:58:13,518][105692] Updated weights for policy 0, policy_version 1452430 (0.0010) [2023-12-27 01:58:13,784][105620] Updated weights for policy 1, policy_version 1454794 (0.0008) [2023-12-27 01:58:13,836][105620] Updated weights for policy 1, policy_version 1454804 (0.0007) [2023-12-27 01:58:13,893][105620] Updated weights for policy 1, policy_version 1454814 (0.0008) [2023-12-27 01:58:14,206][105692] Updated weights for policy 0, policy_version 1452440 (0.0007) [2023-12-27 01:58:14,258][105692] Updated weights for policy 0, policy_version 1452450 (0.0005) [2023-12-27 01:58:14,301][105692] Updated weights for policy 0, policy_version 1452460 (0.0005) [2023-12-27 01:58:14,637][105620] Updated weights for policy 1, policy_version 1454824 (0.0008) [2023-12-27 01:58:14,690][105620] Updated weights for policy 1, policy_version 1454834 (0.0008) [2023-12-27 01:58:14,751][105620] Updated weights for policy 1, policy_version 1454844 (0.0009) [2023-12-27 01:58:15,012][105692] Updated weights for policy 0, policy_version 1452470 (0.0007) [2023-12-27 01:58:15,075][105692] Updated weights for policy 0, policy_version 1452480 (0.0007) [2023-12-27 01:58:15,143][105692] Updated weights for policy 0, policy_version 1452490 (0.0008) [2023-12-27 01:58:15,510][105620] Updated weights for policy 1, policy_version 1454854 (0.0010) [2023-12-27 01:58:15,570][105620] Updated weights for policy 1, policy_version 1454864 (0.0011) [2023-12-27 01:58:15,630][105620] Updated weights for policy 1, policy_version 1454874 (0.0011) [2023-12-27 01:58:15,763][105692] Updated weights for policy 0, policy_version 1452500 (0.0006) [2023-12-27 01:58:15,821][105692] Updated weights for policy 0, policy_version 1452510 (0.0005) [2023-12-27 01:58:15,873][105692] Updated weights for policy 0, policy_version 1452520 (0.0005) [2023-12-27 01:58:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 744398848. Throughput: 0: 9681.7, 1: 9752.4. Samples: 744365084. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:58:16,062][104569] Avg episode reward: [(0, '8437.030'), (1, '8895.459')] [2023-12-27 01:58:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001454880_372498432.pth... [2023-12-27 01:58:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001452528_371900416.pth... [2023-12-27 01:58:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001453760_372211712.pth [2023-12-27 01:58:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001451376_371605504.pth [2023-12-27 01:58:16,255][105620] Updated weights for policy 1, policy_version 1454884 (0.0011) [2023-12-27 01:58:16,307][105620] Updated weights for policy 1, policy_version 1454894 (0.0010) [2023-12-27 01:58:16,356][105620] Updated weights for policy 1, policy_version 1454904 (0.0008) [2023-12-27 01:58:16,642][105692] Updated weights for policy 0, policy_version 1452530 (0.0006) [2023-12-27 01:58:16,706][105692] Updated weights for policy 0, policy_version 1452540 (0.0009) [2023-12-27 01:58:16,766][105692] Updated weights for policy 0, policy_version 1452550 (0.0008) [2023-12-27 01:58:16,827][105692] Updated weights for policy 0, policy_version 1452560 (0.0008) [2023-12-27 01:58:17,011][105620] Updated weights for policy 1, policy_version 1454914 (0.0010) [2023-12-27 01:58:17,077][105620] Updated weights for policy 1, policy_version 1454924 (0.0011) [2023-12-27 01:58:17,135][105620] Updated weights for policy 1, policy_version 1454934 (0.0010) [2023-12-27 01:58:17,180][105620] Updated weights for policy 1, policy_version 1454944 (0.0010) [2023-12-27 01:58:17,525][105692] Updated weights for policy 0, policy_version 1452570 (0.0011) [2023-12-27 01:58:17,585][105692] Updated weights for policy 0, policy_version 1452580 (0.0011) [2023-12-27 01:58:17,638][105692] Updated weights for policy 0, policy_version 1452590 (0.0011) [2023-12-27 01:58:17,927][105620] Updated weights for policy 1, policy_version 1454954 (0.0009) [2023-12-27 01:58:17,989][105620] Updated weights for policy 1, policy_version 1454964 (0.0008) [2023-12-27 01:58:18,051][105620] Updated weights for policy 1, policy_version 1454974 (0.0008) [2023-12-27 01:58:18,341][105692] Updated weights for policy 0, policy_version 1452600 (0.0010) [2023-12-27 01:58:18,393][105692] Updated weights for policy 0, policy_version 1452610 (0.0008) [2023-12-27 01:58:18,451][105692] Updated weights for policy 0, policy_version 1452620 (0.0007) [2023-12-27 01:58:18,818][105620] Updated weights for policy 1, policy_version 1454984 (0.0008) [2023-12-27 01:58:18,875][105620] Updated weights for policy 1, policy_version 1454994 (0.0009) [2023-12-27 01:58:18,937][105620] Updated weights for policy 1, policy_version 1455004 (0.0009) [2023-12-27 01:58:19,154][105692] Updated weights for policy 0, policy_version 1452630 (0.0005) [2023-12-27 01:58:19,213][105692] Updated weights for policy 0, policy_version 1452640 (0.0009) [2023-12-27 01:58:19,275][105692] Updated weights for policy 0, policy_version 1452650 (0.0009) [2023-12-27 01:58:19,751][105620] Updated weights for policy 1, policy_version 1455014 (0.0009) [2023-12-27 01:58:19,812][105620] Updated weights for policy 1, policy_version 1455024 (0.0009) [2023-12-27 01:58:19,875][105620] Updated weights for policy 1, policy_version 1455034 (0.0008) [2023-12-27 01:58:19,964][105692] Updated weights for policy 0, policy_version 1452660 (0.0009) [2023-12-27 01:58:20,015][105692] Updated weights for policy 0, policy_version 1452670 (0.0009) [2023-12-27 01:58:20,079][105692] Updated weights for policy 0, policy_version 1452680 (0.0007) [2023-12-27 01:58:20,670][105620] Updated weights for policy 1, policy_version 1455044 (0.0009) [2023-12-27 01:58:20,722][105620] Updated weights for policy 1, policy_version 1455054 (0.0008) [2023-12-27 01:58:20,778][105620] Updated weights for policy 1, policy_version 1455064 (0.0009) [2023-12-27 01:58:20,815][105692] Updated weights for policy 0, policy_version 1452690 (0.0007) [2023-12-27 01:58:20,871][105692] Updated weights for policy 0, policy_version 1452700 (0.0009) [2023-12-27 01:58:20,939][105692] Updated weights for policy 0, policy_version 1452710 (0.0009) [2023-12-27 01:58:21,003][105692] Updated weights for policy 0, policy_version 1452720 (0.0008) [2023-12-27 01:58:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 744497152. Throughput: 0: 9737.6, 1: 9680.0. Samples: 744481660. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:58:21,063][104569] Avg episode reward: [(0, '8073.920'), (1, '8621.989')] [2023-12-27 01:58:21,538][105620] Updated weights for policy 1, policy_version 1455074 (0.0008) [2023-12-27 01:58:21,597][105620] Updated weights for policy 1, policy_version 1455084 (0.0007) [2023-12-27 01:58:21,667][105620] Updated weights for policy 1, policy_version 1455094 (0.0010) [2023-12-27 01:58:21,731][105620] Updated weights for policy 1, policy_version 1455104 (0.0009) [2023-12-27 01:58:21,949][105692] Updated weights for policy 0, policy_version 1452730 (0.0009) [2023-12-27 01:58:22,017][105692] Updated weights for policy 0, policy_version 1452740 (0.0008) [2023-12-27 01:58:22,082][105692] Updated weights for policy 0, policy_version 1452750 (0.0009) [2023-12-27 01:58:22,449][105620] Updated weights for policy 1, policy_version 1455114 (0.0009) [2023-12-27 01:58:22,503][105620] Updated weights for policy 1, policy_version 1455124 (0.0009) [2023-12-27 01:58:22,559][105620] Updated weights for policy 1, policy_version 1455134 (0.0010) [2023-12-27 01:58:22,866][105692] Updated weights for policy 0, policy_version 1452760 (0.0007) [2023-12-27 01:58:22,931][105692] Updated weights for policy 0, policy_version 1452770 (0.0011) [2023-12-27 01:58:22,999][105692] Updated weights for policy 0, policy_version 1452780 (0.0011) [2023-12-27 01:58:23,327][105620] Updated weights for policy 1, policy_version 1455144 (0.0008) [2023-12-27 01:58:23,374][105620] Updated weights for policy 1, policy_version 1455154 (0.0010) [2023-12-27 01:58:23,418][105620] Updated weights for policy 1, policy_version 1455164 (0.0010) [2023-12-27 01:58:23,666][105692] Updated weights for policy 0, policy_version 1452790 (0.0011) [2023-12-27 01:58:23,728][105692] Updated weights for policy 0, policy_version 1452800 (0.0010) [2023-12-27 01:58:23,779][105692] Updated weights for policy 0, policy_version 1452810 (0.0010) [2023-12-27 01:58:24,175][105620] Updated weights for policy 1, policy_version 1455174 (0.0010) [2023-12-27 01:58:24,236][105620] Updated weights for policy 1, policy_version 1455184 (0.0010) [2023-12-27 01:58:24,280][105620] Updated weights for policy 1, policy_version 1455194 (0.0010) [2023-12-27 01:58:24,506][105692] Updated weights for policy 0, policy_version 1452820 (0.0010) [2023-12-27 01:58:24,563][105692] Updated weights for policy 0, policy_version 1452830 (0.0010) [2023-12-27 01:58:24,624][105692] Updated weights for policy 0, policy_version 1452840 (0.0010) [2023-12-27 01:58:25,031][105620] Updated weights for policy 1, policy_version 1455204 (0.0010) [2023-12-27 01:58:25,079][105620] Updated weights for policy 1, policy_version 1455214 (0.0010) [2023-12-27 01:58:25,138][105620] Updated weights for policy 1, policy_version 1455224 (0.0010) [2023-12-27 01:58:25,255][105692] Updated weights for policy 0, policy_version 1452850 (0.0009) [2023-12-27 01:58:25,322][105692] Updated weights for policy 0, policy_version 1452860 (0.0010) [2023-12-27 01:58:25,384][105692] Updated weights for policy 0, policy_version 1452870 (0.0010) [2023-12-27 01:58:25,446][105692] Updated weights for policy 0, policy_version 1452880 (0.0010) [2023-12-27 01:58:25,896][105620] Updated weights for policy 1, policy_version 1455234 (0.0010) [2023-12-27 01:58:25,951][105620] Updated weights for policy 1, policy_version 1455244 (0.0010) [2023-12-27 01:58:26,009][105620] Updated weights for policy 1, policy_version 1455254 (0.0010) [2023-12-27 01:58:26,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 744579072. Throughput: 0: 9706.3, 1: 9666.9. Samples: 744594508. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:58:26,062][104569] Avg episode reward: [(0, '8897.313'), (1, '8349.041')] [2023-12-27 01:58:26,073][105620] Updated weights for policy 1, policy_version 1455264 (0.0010) [2023-12-27 01:58:26,155][105692] Updated weights for policy 0, policy_version 1452890 (0.0010) [2023-12-27 01:58:26,200][105692] Updated weights for policy 0, policy_version 1452900 (0.0010) [2023-12-27 01:58:26,251][105692] Updated weights for policy 0, policy_version 1452910 (0.0010) [2023-12-27 01:58:26,815][105620] Updated weights for policy 1, policy_version 1455274 (0.0010) [2023-12-27 01:58:26,869][105620] Updated weights for policy 1, policy_version 1455284 (0.0010) [2023-12-27 01:58:26,916][105620] Updated weights for policy 1, policy_version 1455294 (0.0010) [2023-12-27 01:58:27,010][105692] Updated weights for policy 0, policy_version 1452920 (0.0010) [2023-12-27 01:58:27,057][105692] Updated weights for policy 0, policy_version 1452930 (0.0010) [2023-12-27 01:58:27,104][105692] Updated weights for policy 0, policy_version 1452940 (0.0010) [2023-12-27 01:58:27,615][105620] Updated weights for policy 1, policy_version 1455304 (0.0010) [2023-12-27 01:58:27,671][105620] Updated weights for policy 1, policy_version 1455314 (0.0005) [2023-12-27 01:58:27,722][105620] Updated weights for policy 1, policy_version 1455324 (0.0005) [2023-12-27 01:58:27,781][105692] Updated weights for policy 0, policy_version 1452950 (0.0008) [2023-12-27 01:58:27,833][105692] Updated weights for policy 0, policy_version 1452960 (0.0005) [2023-12-27 01:58:27,897][105692] Updated weights for policy 0, policy_version 1452970 (0.0005) [2023-12-27 01:58:28,354][105620] Updated weights for policy 1, policy_version 1455334 (0.0008) [2023-12-27 01:58:28,412][105620] Updated weights for policy 1, policy_version 1455344 (0.0010) [2023-12-27 01:58:28,471][105620] Updated weights for policy 1, policy_version 1455354 (0.0010) [2023-12-27 01:58:28,517][105692] Updated weights for policy 0, policy_version 1452980 (0.0006) [2023-12-27 01:58:28,576][105692] Updated weights for policy 0, policy_version 1452990 (0.0008) [2023-12-27 01:58:28,634][105692] Updated weights for policy 0, policy_version 1453000 (0.0008) [2023-12-27 01:58:29,190][105620] Updated weights for policy 1, policy_version 1455364 (0.0010) [2023-12-27 01:58:29,262][105620] Updated weights for policy 1, policy_version 1455374 (0.0007) [2023-12-27 01:58:29,317][105620] Updated weights for policy 1, policy_version 1455384 (0.0008) [2023-12-27 01:58:29,331][105692] Updated weights for policy 0, policy_version 1453010 (0.0008) [2023-12-27 01:58:29,394][105692] Updated weights for policy 0, policy_version 1453020 (0.0008) [2023-12-27 01:58:29,451][105692] Updated weights for policy 0, policy_version 1453030 (0.0005) [2023-12-27 01:58:29,510][105692] Updated weights for policy 0, policy_version 1453040 (0.0005) [2023-12-27 01:58:29,951][105620] Updated weights for policy 1, policy_version 1455394 (0.0007) [2023-12-27 01:58:30,000][105620] Updated weights for policy 1, policy_version 1455404 (0.0005) [2023-12-27 01:58:30,058][105620] Updated weights for policy 1, policy_version 1455414 (0.0007) [2023-12-27 01:58:30,097][105692] Updated weights for policy 0, policy_version 1453050 (0.0010) [2023-12-27 01:58:30,116][105620] Updated weights for policy 1, policy_version 1455424 (0.0007) [2023-12-27 01:58:30,161][105692] Updated weights for policy 0, policy_version 1453060 (0.0009) [2023-12-27 01:58:30,222][105692] Updated weights for policy 0, policy_version 1453070 (0.0010) [2023-12-27 01:58:30,720][105620] Updated weights for policy 1, policy_version 1455434 (0.0010) [2023-12-27 01:58:30,794][105620] Updated weights for policy 1, policy_version 1455444 (0.0009) [2023-12-27 01:58:30,855][105620] Updated weights for policy 1, policy_version 1455454 (0.0007) [2023-12-27 01:58:30,875][105692] Updated weights for policy 0, policy_version 1453080 (0.0008) [2023-12-27 01:58:30,941][105692] Updated weights for policy 0, policy_version 1453090 (0.0008) [2023-12-27 01:58:31,004][105692] Updated weights for policy 0, policy_version 1453100 (0.0009) [2023-12-27 01:58:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 744693760. Throughput: 0: 9718.2, 1: 9729.9. Samples: 744655020. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:58:31,062][104569] Avg episode reward: [(0, '9078.259'), (1, '8537.321')] [2023-12-27 01:58:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001453104_372047872.pth... [2023-12-27 01:58:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001455456_372645888.pth... [2023-12-27 01:58:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001451952_371752960.pth [2023-12-27 01:58:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001454336_372359168.pth [2023-12-27 01:58:31,520][105620] Updated weights for policy 1, policy_version 1455464 (0.0005) [2023-12-27 01:58:31,576][105620] Updated weights for policy 1, policy_version 1455474 (0.0005) [2023-12-27 01:58:31,641][105620] Updated weights for policy 1, policy_version 1455484 (0.0007) [2023-12-27 01:58:31,801][105692] Updated weights for policy 0, policy_version 1453110 (0.0007) [2023-12-27 01:58:31,861][105692] Updated weights for policy 0, policy_version 1453120 (0.0005) [2023-12-27 01:58:31,916][105692] Updated weights for policy 0, policy_version 1453130 (0.0005) [2023-12-27 01:58:32,305][105620] Updated weights for policy 1, policy_version 1455494 (0.0008) [2023-12-27 01:58:32,369][105620] Updated weights for policy 1, policy_version 1455504 (0.0008) [2023-12-27 01:58:32,427][105620] Updated weights for policy 1, policy_version 1455514 (0.0006) [2023-12-27 01:58:32,574][105692] Updated weights for policy 0, policy_version 1453140 (0.0007) [2023-12-27 01:58:32,635][105692] Updated weights for policy 0, policy_version 1453150 (0.0009) [2023-12-27 01:58:32,690][105692] Updated weights for policy 0, policy_version 1453160 (0.0009) [2023-12-27 01:58:33,103][105620] Updated weights for policy 1, policy_version 1455524 (0.0005) [2023-12-27 01:58:33,148][105620] Updated weights for policy 1, policy_version 1455534 (0.0005) [2023-12-27 01:58:33,196][105620] Updated weights for policy 1, policy_version 1455544 (0.0005) [2023-12-27 01:58:33,518][105692] Updated weights for policy 0, policy_version 1453170 (0.0007) [2023-12-27 01:58:33,565][105692] Updated weights for policy 0, policy_version 1453180 (0.0008) [2023-12-27 01:58:33,612][105692] Updated weights for policy 0, policy_version 1453190 (0.0009) [2023-12-27 01:58:33,663][105692] Updated weights for policy 0, policy_version 1453200 (0.0009) [2023-12-27 01:58:33,809][105620] Updated weights for policy 1, policy_version 1455554 (0.0006) [2023-12-27 01:58:33,870][105620] Updated weights for policy 1, policy_version 1455564 (0.0009) [2023-12-27 01:58:33,936][105620] Updated weights for policy 1, policy_version 1455574 (0.0009) [2023-12-27 01:58:33,996][105620] Updated weights for policy 1, policy_version 1455584 (0.0009) [2023-12-27 01:58:34,470][105692] Updated weights for policy 0, policy_version 1453210 (0.0009) [2023-12-27 01:58:34,536][105692] Updated weights for policy 0, policy_version 1453220 (0.0009) [2023-12-27 01:58:34,591][105692] Updated weights for policy 0, policy_version 1453230 (0.0010) [2023-12-27 01:58:34,681][105620] Updated weights for policy 1, policy_version 1455594 (0.0008) [2023-12-27 01:58:34,746][105620] Updated weights for policy 1, policy_version 1455604 (0.0009) [2023-12-27 01:58:34,806][105620] Updated weights for policy 1, policy_version 1455614 (0.0008) [2023-12-27 01:58:35,381][105692] Updated weights for policy 0, policy_version 1453240 (0.0009) [2023-12-27 01:58:35,428][105692] Updated weights for policy 0, policy_version 1453250 (0.0008) [2023-12-27 01:58:35,483][105692] Updated weights for policy 0, policy_version 1453260 (0.0007) [2023-12-27 01:58:35,492][105620] Updated weights for policy 1, policy_version 1455624 (0.0008) [2023-12-27 01:58:35,554][105620] Updated weights for policy 1, policy_version 1455634 (0.0009) [2023-12-27 01:58:35,615][105620] Updated weights for policy 1, policy_version 1455644 (0.0009) [2023-12-27 01:58:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 744783872. Throughput: 0: 9789.7, 1: 9754.7. Samples: 744774684. Policy #0 lag: (min: 25.0, avg: 38.6, max: 57.0) [2023-12-27 01:58:36,063][104569] Avg episode reward: [(0, '8894.358'), (1, '9084.925')] [2023-12-27 01:58:36,228][105692] Updated weights for policy 0, policy_version 1453270 (0.0009) [2023-12-27 01:58:36,289][105692] Updated weights for policy 0, policy_version 1453280 (0.0008) [2023-12-27 01:58:36,357][105692] Updated weights for policy 0, policy_version 1453290 (0.0008) [2023-12-27 01:58:36,417][105620] Updated weights for policy 1, policy_version 1455654 (0.0009) [2023-12-27 01:58:36,476][105620] Updated weights for policy 1, policy_version 1455664 (0.0010) [2023-12-27 01:58:36,541][105620] Updated weights for policy 1, policy_version 1455674 (0.0010) [2023-12-27 01:58:37,131][105692] Updated weights for policy 0, policy_version 1453300 (0.0008) [2023-12-27 01:58:37,185][105692] Updated weights for policy 0, policy_version 1453310 (0.0008) [2023-12-27 01:58:37,235][105692] Updated weights for policy 0, policy_version 1453320 (0.0008) [2023-12-27 01:58:37,277][105620] Updated weights for policy 1, policy_version 1455684 (0.0010) [2023-12-27 01:58:37,339][105620] Updated weights for policy 1, policy_version 1455694 (0.0010) [2023-12-27 01:58:37,398][105620] Updated weights for policy 1, policy_version 1455704 (0.0011) [2023-12-27 01:58:38,045][105692] Updated weights for policy 0, policy_version 1453330 (0.0009) [2023-12-27 01:58:38,102][105692] Updated weights for policy 0, policy_version 1453340 (0.0009) [2023-12-27 01:58:38,154][105692] Updated weights for policy 0, policy_version 1453350 (0.0008) [2023-12-27 01:58:38,176][105620] Updated weights for policy 1, policy_version 1455714 (0.0011) [2023-12-27 01:58:38,211][105692] Updated weights for policy 0, policy_version 1453360 (0.0009) [2023-12-27 01:58:38,234][105620] Updated weights for policy 1, policy_version 1455724 (0.0010) [2023-12-27 01:58:38,291][105620] Updated weights for policy 1, policy_version 1455734 (0.0010) [2023-12-27 01:58:38,355][105620] Updated weights for policy 1, policy_version 1455744 (0.0011) [2023-12-27 01:58:38,995][105692] Updated weights for policy 0, policy_version 1453370 (0.0008) [2023-12-27 01:58:39,050][105692] Updated weights for policy 0, policy_version 1453380 (0.0008) [2023-12-27 01:58:39,086][105620] Updated weights for policy 1, policy_version 1455754 (0.0010) [2023-12-27 01:58:39,102][105692] Updated weights for policy 0, policy_version 1453390 (0.0007) [2023-12-27 01:58:39,147][105620] Updated weights for policy 1, policy_version 1455764 (0.0010) [2023-12-27 01:58:39,206][105620] Updated weights for policy 1, policy_version 1455774 (0.0010) [2023-12-27 01:58:39,925][105692] Updated weights for policy 0, policy_version 1453400 (0.0009) [2023-12-27 01:58:39,992][105692] Updated weights for policy 0, policy_version 1453410 (0.0007) [2023-12-27 01:58:40,015][105620] Updated weights for policy 1, policy_version 1455784 (0.0008) [2023-12-27 01:58:40,057][105692] Updated weights for policy 0, policy_version 1453420 (0.0009) [2023-12-27 01:58:40,073][105620] Updated weights for policy 1, policy_version 1455794 (0.0006) [2023-12-27 01:58:40,128][105620] Updated weights for policy 1, policy_version 1455804 (0.0007) [2023-12-27 01:58:40,831][105692] Updated weights for policy 0, policy_version 1453430 (0.0009) [2023-12-27 01:58:40,895][105692] Updated weights for policy 0, policy_version 1453440 (0.0007) [2023-12-27 01:58:40,899][105620] Updated weights for policy 1, policy_version 1455814 (0.0008) [2023-12-27 01:58:40,956][105620] Updated weights for policy 1, policy_version 1455824 (0.0009) [2023-12-27 01:58:40,959][105692] Updated weights for policy 0, policy_version 1453450 (0.0005) [2023-12-27 01:58:41,018][105620] Updated weights for policy 1, policy_version 1455834 (0.0009) [2023-12-27 01:58:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 744882176. Throughput: 0: 9691.5, 1: 9667.7. Samples: 744884476. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 01:58:41,063][104569] Avg episode reward: [(0, '8710.725'), (1, '9174.935')] [2023-12-27 01:58:41,757][105692] Updated weights for policy 0, policy_version 1453460 (0.0006) [2023-12-27 01:58:41,763][105620] Updated weights for policy 1, policy_version 1455844 (0.0008) [2023-12-27 01:58:41,816][105692] Updated weights for policy 0, policy_version 1453470 (0.0007) [2023-12-27 01:58:41,827][105620] Updated weights for policy 1, policy_version 1455854 (0.0006) [2023-12-27 01:58:41,871][105692] Updated weights for policy 0, policy_version 1453480 (0.0005) [2023-12-27 01:58:41,887][105620] Updated weights for policy 1, policy_version 1455864 (0.0009) [2023-12-27 01:58:42,542][105620] Updated weights for policy 1, policy_version 1455874 (0.0008) [2023-12-27 01:58:42,602][105620] Updated weights for policy 1, policy_version 1455884 (0.0007) [2023-12-27 01:58:42,641][105692] Updated weights for policy 0, policy_version 1453490 (0.0006) [2023-12-27 01:58:42,675][105620] Updated weights for policy 1, policy_version 1455894 (0.0005) [2023-12-27 01:58:42,708][105692] Updated weights for policy 0, policy_version 1453500 (0.0006) [2023-12-27 01:58:42,741][105620] Updated weights for policy 1, policy_version 1455904 (0.0006) [2023-12-27 01:58:42,771][105692] Updated weights for policy 0, policy_version 1453510 (0.0006) [2023-12-27 01:58:42,839][105692] Updated weights for policy 0, policy_version 1453520 (0.0008) [2023-12-27 01:58:43,372][105620] Updated weights for policy 1, policy_version 1455914 (0.0007) [2023-12-27 01:58:43,393][105692] Updated weights for policy 0, policy_version 1453530 (0.0008) [2023-12-27 01:58:43,434][105620] Updated weights for policy 1, policy_version 1455924 (0.0005) [2023-12-27 01:58:43,450][105692] Updated weights for policy 0, policy_version 1453540 (0.0009) [2023-12-27 01:58:43,486][105620] Updated weights for policy 1, policy_version 1455934 (0.0005) [2023-12-27 01:58:43,507][105692] Updated weights for policy 0, policy_version 1453550 (0.0008) [2023-12-27 01:58:44,092][105620] Updated weights for policy 1, policy_version 1455944 (0.0006) [2023-12-27 01:58:44,157][105620] Updated weights for policy 1, policy_version 1455954 (0.0008) [2023-12-27 01:58:44,215][105620] Updated weights for policy 1, policy_version 1455964 (0.0009) [2023-12-27 01:58:44,347][105692] Updated weights for policy 0, policy_version 1453560 (0.0009) [2023-12-27 01:58:44,401][105692] Updated weights for policy 0, policy_version 1453570 (0.0009) [2023-12-27 01:58:44,471][105692] Updated weights for policy 0, policy_version 1453580 (0.0009) [2023-12-27 01:58:44,787][105620] Updated weights for policy 1, policy_version 1455974 (0.0008) [2023-12-27 01:58:44,846][105620] Updated weights for policy 1, policy_version 1455984 (0.0010) [2023-12-27 01:58:44,902][105620] Updated weights for policy 1, policy_version 1455994 (0.0009) [2023-12-27 01:58:45,250][105692] Updated weights for policy 0, policy_version 1453590 (0.0010) [2023-12-27 01:58:45,305][105692] Updated weights for policy 0, policy_version 1453600 (0.0008) [2023-12-27 01:58:45,363][105692] Updated weights for policy 0, policy_version 1453610 (0.0008) [2023-12-27 01:58:45,701][105620] Updated weights for policy 1, policy_version 1456004 (0.0008) [2023-12-27 01:58:45,753][105620] Updated weights for policy 1, policy_version 1456014 (0.0008) [2023-12-27 01:58:45,808][105620] Updated weights for policy 1, policy_version 1456024 (0.0009) [2023-12-27 01:58:46,018][105692] Updated weights for policy 0, policy_version 1453620 (0.0009) [2023-12-27 01:58:46,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 744972288. Throughput: 0: 9616.5, 1: 9686.2. Samples: 744943600. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 01:58:46,062][104569] Avg episode reward: [(0, '8439.232'), (1, '8986.869')] [2023-12-27 01:58:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001456032_372793344.pth... [2023-12-27 01:58:46,066][105692] Updated weights for policy 0, policy_version 1453630 (0.0010) [2023-12-27 01:58:46,069][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001454880_372498432.pth [2023-12-27 01:58:46,122][105692] Updated weights for policy 0, policy_version 1453640 (0.0011) [2023-12-27 01:58:46,161][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001453648_372187136.pth... [2023-12-27 01:58:46,164][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001452528_371900416.pth [2023-12-27 01:58:46,638][105620] Updated weights for policy 1, policy_version 1456034 (0.0009) [2023-12-27 01:58:46,673][105692] Updated weights for policy 0, policy_version 1453650 (0.0006) [2023-12-27 01:58:46,706][105620] Updated weights for policy 1, policy_version 1456044 (0.0007) [2023-12-27 01:58:46,734][105692] Updated weights for policy 0, policy_version 1453660 (0.0005) [2023-12-27 01:58:46,770][105620] Updated weights for policy 1, policy_version 1456054 (0.0009) [2023-12-27 01:58:46,787][105692] Updated weights for policy 0, policy_version 1453670 (0.0005) [2023-12-27 01:58:46,830][105620] Updated weights for policy 1, policy_version 1456064 (0.0009) [2023-12-27 01:58:46,840][105692] Updated weights for policy 0, policy_version 1453680 (0.0007) [2023-12-27 01:58:47,374][105692] Updated weights for policy 0, policy_version 1453690 (0.0005) [2023-12-27 01:58:47,435][105692] Updated weights for policy 0, policy_version 1453700 (0.0005) [2023-12-27 01:58:47,482][105692] Updated weights for policy 0, policy_version 1453710 (0.0005) [2023-12-27 01:58:47,585][105620] Updated weights for policy 1, policy_version 1456074 (0.0006) [2023-12-27 01:58:47,645][105620] Updated weights for policy 1, policy_version 1456084 (0.0006) [2023-12-27 01:58:47,707][105620] Updated weights for policy 1, policy_version 1456094 (0.0009) [2023-12-27 01:58:48,049][105692] Updated weights for policy 0, policy_version 1453720 (0.0007) [2023-12-27 01:58:48,112][105692] Updated weights for policy 0, policy_version 1453730 (0.0008) [2023-12-27 01:58:48,173][105692] Updated weights for policy 0, policy_version 1453740 (0.0008) [2023-12-27 01:58:48,415][105620] Updated weights for policy 1, policy_version 1456104 (0.0010) [2023-12-27 01:58:48,471][105620] Updated weights for policy 1, policy_version 1456114 (0.0011) [2023-12-27 01:58:48,541][105620] Updated weights for policy 1, policy_version 1456124 (0.0011) [2023-12-27 01:58:48,880][105692] Updated weights for policy 0, policy_version 1453750 (0.0006) [2023-12-27 01:58:48,937][105692] Updated weights for policy 0, policy_version 1453760 (0.0005) [2023-12-27 01:58:49,000][105692] Updated weights for policy 0, policy_version 1453770 (0.0005) [2023-12-27 01:58:49,303][105620] Updated weights for policy 1, policy_version 1456134 (0.0009) [2023-12-27 01:58:49,371][105620] Updated weights for policy 1, policy_version 1456144 (0.0008) [2023-12-27 01:58:49,435][105620] Updated weights for policy 1, policy_version 1456154 (0.0008) [2023-12-27 01:58:49,647][105692] Updated weights for policy 0, policy_version 1453780 (0.0007) [2023-12-27 01:58:49,707][105692] Updated weights for policy 0, policy_version 1453790 (0.0009) [2023-12-27 01:58:49,763][105692] Updated weights for policy 0, policy_version 1453800 (0.0011) [2023-12-27 01:58:50,187][105620] Updated weights for policy 1, policy_version 1456164 (0.0008) [2023-12-27 01:58:50,234][105620] Updated weights for policy 1, policy_version 1456174 (0.0008) [2023-12-27 01:58:50,289][105620] Updated weights for policy 1, policy_version 1456184 (0.0007) [2023-12-27 01:58:50,517][105692] Updated weights for policy 0, policy_version 1453810 (0.0010) [2023-12-27 01:58:50,583][105692] Updated weights for policy 0, policy_version 1453820 (0.0010) [2023-12-27 01:58:50,636][105692] Updated weights for policy 0, policy_version 1453830 (0.0010) [2023-12-27 01:58:50,692][105692] Updated weights for policy 0, policy_version 1453840 (0.0009) [2023-12-27 01:58:51,000][105620] Updated weights for policy 1, policy_version 1456194 (0.0007) [2023-12-27 01:58:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 745070592. Throughput: 0: 9734.0, 1: 9692.1. Samples: 745064040. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 01:58:51,063][105620] Updated weights for policy 1, policy_version 1456204 (0.0009) [2023-12-27 01:58:51,063][104569] Avg episode reward: [(0, '8528.748'), (1, '8898.627')] [2023-12-27 01:58:51,118][105620] Updated weights for policy 1, policy_version 1456214 (0.0008) [2023-12-27 01:58:51,174][105620] Updated weights for policy 1, policy_version 1456224 (0.0008) [2023-12-27 01:58:51,505][105692] Updated weights for policy 0, policy_version 1453850 (0.0010) [2023-12-27 01:58:51,564][105692] Updated weights for policy 0, policy_version 1453860 (0.0009) [2023-12-27 01:58:51,633][105692] Updated weights for policy 0, policy_version 1453870 (0.0009) [2023-12-27 01:58:51,895][105620] Updated weights for policy 1, policy_version 1456234 (0.0008) [2023-12-27 01:58:51,955][105620] Updated weights for policy 1, policy_version 1456244 (0.0007) [2023-12-27 01:58:52,022][105620] Updated weights for policy 1, policy_version 1456254 (0.0009) [2023-12-27 01:58:52,441][105692] Updated weights for policy 0, policy_version 1453880 (0.0009) [2023-12-27 01:58:52,493][105692] Updated weights for policy 0, policy_version 1453890 (0.0009) [2023-12-27 01:58:52,550][105692] Updated weights for policy 0, policy_version 1453900 (0.0009) [2023-12-27 01:58:52,776][105620] Updated weights for policy 1, policy_version 1456264 (0.0009) [2023-12-27 01:58:52,834][105620] Updated weights for policy 1, policy_version 1456274 (0.0009) [2023-12-27 01:58:52,892][105620] Updated weights for policy 1, policy_version 1456284 (0.0009) [2023-12-27 01:58:53,318][105692] Updated weights for policy 0, policy_version 1453910 (0.0009) [2023-12-27 01:58:53,376][105692] Updated weights for policy 0, policy_version 1453920 (0.0009) [2023-12-27 01:58:53,428][105692] Updated weights for policy 0, policy_version 1453930 (0.0009) [2023-12-27 01:58:53,621][105620] Updated weights for policy 1, policy_version 1456294 (0.0008) [2023-12-27 01:58:53,672][105620] Updated weights for policy 1, policy_version 1456304 (0.0008) [2023-12-27 01:58:53,720][105620] Updated weights for policy 1, policy_version 1456314 (0.0008) [2023-12-27 01:58:54,138][105692] Updated weights for policy 0, policy_version 1453940 (0.0009) [2023-12-27 01:58:54,184][105692] Updated weights for policy 0, policy_version 1453950 (0.0008) [2023-12-27 01:58:54,236][105692] Updated weights for policy 0, policy_version 1453960 (0.0005) [2023-12-27 01:58:54,518][105620] Updated weights for policy 1, policy_version 1456324 (0.0006) [2023-12-27 01:58:54,564][105620] Updated weights for policy 1, policy_version 1456334 (0.0005) [2023-12-27 01:58:54,612][105620] Updated weights for policy 1, policy_version 1456344 (0.0011) [2023-12-27 01:58:54,873][105692] Updated weights for policy 0, policy_version 1453970 (0.0006) [2023-12-27 01:58:54,918][105692] Updated weights for policy 0, policy_version 1453980 (0.0010) [2023-12-27 01:58:54,971][105692] Updated weights for policy 0, policy_version 1453990 (0.0010) [2023-12-27 01:58:55,026][105692] Updated weights for policy 0, policy_version 1454000 (0.0011) [2023-12-27 01:58:55,333][105620] Updated weights for policy 1, policy_version 1456354 (0.0010) [2023-12-27 01:58:55,392][105620] Updated weights for policy 1, policy_version 1456364 (0.0006) [2023-12-27 01:58:55,455][105620] Updated weights for policy 1, policy_version 1456374 (0.0008) [2023-12-27 01:58:55,513][105620] Updated weights for policy 1, policy_version 1456384 (0.0010) [2023-12-27 01:58:55,809][105692] Updated weights for policy 0, policy_version 1454010 (0.0011) [2023-12-27 01:58:55,864][105692] Updated weights for policy 0, policy_version 1454020 (0.0010) [2023-12-27 01:58:55,918][105692] Updated weights for policy 0, policy_version 1454030 (0.0009) [2023-12-27 01:58:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 745168896. Throughput: 0: 9650.6, 1: 9640.8. Samples: 745177136. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 01:58:56,062][104569] Avg episode reward: [(0, '8526.572'), (1, '9264.673')] [2023-12-27 01:58:56,153][105620] Updated weights for policy 1, policy_version 1456394 (0.0011) [2023-12-27 01:58:56,212][105620] Updated weights for policy 1, policy_version 1456404 (0.0010) [2023-12-27 01:58:56,271][105620] Updated weights for policy 1, policy_version 1456414 (0.0010) [2023-12-27 01:58:56,508][105692] Updated weights for policy 0, policy_version 1454040 (0.0011) [2023-12-27 01:58:56,571][105692] Updated weights for policy 0, policy_version 1454050 (0.0008) [2023-12-27 01:58:56,638][105692] Updated weights for policy 0, policy_version 1454060 (0.0006) [2023-12-27 01:58:56,958][105620] Updated weights for policy 1, policy_version 1456424 (0.0006) [2023-12-27 01:58:57,011][105620] Updated weights for policy 1, policy_version 1456434 (0.0006) [2023-12-27 01:58:57,061][105620] Updated weights for policy 1, policy_version 1456444 (0.0008) [2023-12-27 01:58:57,199][105692] Updated weights for policy 0, policy_version 1454070 (0.0008) [2023-12-27 01:58:57,247][105692] Updated weights for policy 0, policy_version 1454080 (0.0010) [2023-12-27 01:58:57,295][105692] Updated weights for policy 0, policy_version 1454090 (0.0010) [2023-12-27 01:58:57,686][105620] Updated weights for policy 1, policy_version 1456454 (0.0007) [2023-12-27 01:58:57,749][105620] Updated weights for policy 1, policy_version 1456464 (0.0008) [2023-12-27 01:58:57,801][105620] Updated weights for policy 1, policy_version 1456474 (0.0008) [2023-12-27 01:58:57,988][105692] Updated weights for policy 0, policy_version 1454100 (0.0010) [2023-12-27 01:58:58,039][105692] Updated weights for policy 0, policy_version 1454110 (0.0010) [2023-12-27 01:58:58,086][105692] Updated weights for policy 0, policy_version 1454120 (0.0010) [2023-12-27 01:58:58,464][105620] Updated weights for policy 1, policy_version 1456484 (0.0007) [2023-12-27 01:58:58,527][105620] Updated weights for policy 1, policy_version 1456494 (0.0009) [2023-12-27 01:58:58,611][105620] Updated weights for policy 1, policy_version 1456504 (0.0008) [2023-12-27 01:58:58,830][105692] Updated weights for policy 0, policy_version 1454130 (0.0010) [2023-12-27 01:58:58,898][105692] Updated weights for policy 0, policy_version 1454140 (0.0008) [2023-12-27 01:58:58,969][105692] Updated weights for policy 0, policy_version 1454150 (0.0009) [2023-12-27 01:58:59,041][105692] Updated weights for policy 0, policy_version 1454160 (0.0008) [2023-12-27 01:58:59,446][105620] Updated weights for policy 1, policy_version 1456514 (0.0008) [2023-12-27 01:58:59,510][105620] Updated weights for policy 1, policy_version 1456524 (0.0008) [2023-12-27 01:58:59,567][105620] Updated weights for policy 1, policy_version 1456534 (0.0008) [2023-12-27 01:58:59,631][105620] Updated weights for policy 1, policy_version 1456544 (0.0009) [2023-12-27 01:58:59,823][105692] Updated weights for policy 0, policy_version 1454170 (0.0010) [2023-12-27 01:58:59,888][105692] Updated weights for policy 0, policy_version 1454180 (0.0011) [2023-12-27 01:58:59,937][105692] Updated weights for policy 0, policy_version 1454190 (0.0010) [2023-12-27 01:59:00,358][105620] Updated weights for policy 1, policy_version 1456554 (0.0010) [2023-12-27 01:59:00,422][105620] Updated weights for policy 1, policy_version 1456564 (0.0007) [2023-12-27 01:59:00,480][105620] Updated weights for policy 1, policy_version 1456574 (0.0010) [2023-12-27 01:59:00,664][105692] Updated weights for policy 0, policy_version 1454200 (0.0006) [2023-12-27 01:59:00,712][105692] Updated weights for policy 0, policy_version 1454210 (0.0008) [2023-12-27 01:59:00,766][105692] Updated weights for policy 0, policy_version 1454220 (0.0009) [2023-12-27 01:59:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 745267200. Throughput: 0: 9727.1, 1: 9707.5. Samples: 745239644. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 01:59:01,062][104569] Avg episode reward: [(0, '8708.682'), (1, '9081.074')] [2023-12-27 01:59:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001454224_372334592.pth... [2023-12-27 01:59:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001453104_372047872.pth [2023-12-27 01:59:01,123][105620] Updated weights for policy 1, policy_version 1456584 (0.0008) [2023-12-27 01:59:01,187][105620] Updated weights for policy 1, policy_version 1456594 (0.0010) [2023-12-27 01:59:01,243][105620] Updated weights for policy 1, policy_version 1456604 (0.0010) [2023-12-27 01:59:01,266][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001456608_372940800.pth... [2023-12-27 01:59:01,271][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001455456_372645888.pth [2023-12-27 01:59:01,553][105692] Updated weights for policy 0, policy_version 1454230 (0.0007) [2023-12-27 01:59:01,603][105692] Updated weights for policy 0, policy_version 1454240 (0.0005) [2023-12-27 01:59:01,668][105692] Updated weights for policy 0, policy_version 1454250 (0.0008) [2023-12-27 01:59:01,948][105620] Updated weights for policy 1, policy_version 1456614 (0.0010) [2023-12-27 01:59:02,013][105620] Updated weights for policy 1, policy_version 1456624 (0.0011) [2023-12-27 01:59:02,075][105620] Updated weights for policy 1, policy_version 1456634 (0.0010) [2023-12-27 01:59:02,307][105692] Updated weights for policy 0, policy_version 1454260 (0.0006) [2023-12-27 01:59:02,371][105692] Updated weights for policy 0, policy_version 1454270 (0.0007) [2023-12-27 01:59:02,431][105692] Updated weights for policy 0, policy_version 1454280 (0.0005) [2023-12-27 01:59:02,820][105620] Updated weights for policy 1, policy_version 1456644 (0.0010) [2023-12-27 01:59:02,876][105620] Updated weights for policy 1, policy_version 1456654 (0.0010) [2023-12-27 01:59:02,929][105620] Updated weights for policy 1, policy_version 1456664 (0.0010) [2023-12-27 01:59:03,069][105692] Updated weights for policy 0, policy_version 1454290 (0.0005) [2023-12-27 01:59:03,126][105692] Updated weights for policy 0, policy_version 1454300 (0.0005) [2023-12-27 01:59:03,187][105692] Updated weights for policy 0, policy_version 1454310 (0.0005) [2023-12-27 01:59:03,238][105692] Updated weights for policy 0, policy_version 1454320 (0.0006) [2023-12-27 01:59:03,594][105620] Updated weights for policy 1, policy_version 1456674 (0.0010) [2023-12-27 01:59:03,657][105620] Updated weights for policy 1, policy_version 1456684 (0.0008) [2023-12-27 01:59:03,706][105620] Updated weights for policy 1, policy_version 1456694 (0.0007) [2023-12-27 01:59:03,754][105620] Updated weights for policy 1, policy_version 1456704 (0.0008) [2023-12-27 01:59:03,977][105692] Updated weights for policy 0, policy_version 1454330 (0.0010) [2023-12-27 01:59:04,028][105692] Updated weights for policy 0, policy_version 1454340 (0.0007) [2023-12-27 01:59:04,086][105692] Updated weights for policy 0, policy_version 1454350 (0.0005) [2023-12-27 01:59:04,464][105620] Updated weights for policy 1, policy_version 1456714 (0.0010) [2023-12-27 01:59:04,513][105620] Updated weights for policy 1, policy_version 1456724 (0.0010) [2023-12-27 01:59:04,563][105620] Updated weights for policy 1, policy_version 1456734 (0.0009) [2023-12-27 01:59:04,660][105692] Updated weights for policy 0, policy_version 1454360 (0.0005) [2023-12-27 01:59:04,709][105692] Updated weights for policy 0, policy_version 1454370 (0.0005) [2023-12-27 01:59:04,762][105692] Updated weights for policy 0, policy_version 1454380 (0.0005) [2023-12-27 01:59:05,219][105620] Updated weights for policy 1, policy_version 1456744 (0.0008) [2023-12-27 01:59:05,289][105620] Updated weights for policy 1, policy_version 1456754 (0.0011) [2023-12-27 01:59:05,346][105620] Updated weights for policy 1, policy_version 1456764 (0.0011) [2023-12-27 01:59:05,357][105692] Updated weights for policy 0, policy_version 1454390 (0.0006) [2023-12-27 01:59:05,409][105692] Updated weights for policy 0, policy_version 1454400 (0.0005) [2023-12-27 01:59:05,463][105692] Updated weights for policy 0, policy_version 1454410 (0.0005) [2023-12-27 01:59:06,004][105620] Updated weights for policy 1, policy_version 1456774 (0.0007) [2023-12-27 01:59:06,056][105620] Updated weights for policy 1, policy_version 1456784 (0.0005) [2023-12-27 01:59:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 745365504. Throughput: 0: 9705.4, 1: 9743.8. Samples: 745356876. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 01:59:06,062][104569] Avg episode reward: [(0, '8708.419'), (1, '8801.903')] [2023-12-27 01:59:06,112][105620] Updated weights for policy 1, policy_version 1456794 (0.0006) [2023-12-27 01:59:06,191][105692] Updated weights for policy 0, policy_version 1454420 (0.0007) [2023-12-27 01:59:06,259][105692] Updated weights for policy 0, policy_version 1454430 (0.0008) [2023-12-27 01:59:06,325][105692] Updated weights for policy 0, policy_version 1454440 (0.0009) [2023-12-27 01:59:06,801][105620] Updated weights for policy 1, policy_version 1456804 (0.0009) [2023-12-27 01:59:06,856][105620] Updated weights for policy 1, policy_version 1456814 (0.0005) [2023-12-27 01:59:06,896][105692] Updated weights for policy 0, policy_version 1454450 (0.0007) [2023-12-27 01:59:06,925][105620] Updated weights for policy 1, policy_version 1456824 (0.0007) [2023-12-27 01:59:06,953][105692] Updated weights for policy 0, policy_version 1454460 (0.0006) [2023-12-27 01:59:07,014][105692] Updated weights for policy 0, policy_version 1454470 (0.0010) [2023-12-27 01:59:07,076][105692] Updated weights for policy 0, policy_version 1454480 (0.0010) [2023-12-27 01:59:07,600][105620] Updated weights for policy 1, policy_version 1456834 (0.0007) [2023-12-27 01:59:07,645][105620] Updated weights for policy 1, policy_version 1456844 (0.0008) [2023-12-27 01:59:07,696][105620] Updated weights for policy 1, policy_version 1456854 (0.0008) [2023-12-27 01:59:07,754][105692] Updated weights for policy 0, policy_version 1454490 (0.0010) [2023-12-27 01:59:07,760][105620] Updated weights for policy 1, policy_version 1456864 (0.0006) [2023-12-27 01:59:07,818][105692] Updated weights for policy 0, policy_version 1454500 (0.0008) [2023-12-27 01:59:07,884][105692] Updated weights for policy 0, policy_version 1454510 (0.0008) [2023-12-27 01:59:08,356][105620] Updated weights for policy 1, policy_version 1456874 (0.0006) [2023-12-27 01:59:08,423][105620] Updated weights for policy 1, policy_version 1456884 (0.0006) [2023-12-27 01:59:08,487][105620] Updated weights for policy 1, policy_version 1456894 (0.0007) [2023-12-27 01:59:08,507][105692] Updated weights for policy 0, policy_version 1454520 (0.0007) [2023-12-27 01:59:08,560][105692] Updated weights for policy 0, policy_version 1454530 (0.0010) [2023-12-27 01:59:08,619][105692] Updated weights for policy 0, policy_version 1454540 (0.0010) [2023-12-27 01:59:09,170][105620] Updated weights for policy 1, policy_version 1456904 (0.0011) [2023-12-27 01:59:09,233][105620] Updated weights for policy 1, policy_version 1456914 (0.0011) [2023-12-27 01:59:09,292][105620] Updated weights for policy 1, policy_version 1456924 (0.0011) [2023-12-27 01:59:09,334][105692] Updated weights for policy 0, policy_version 1454550 (0.0008) [2023-12-27 01:59:09,403][105692] Updated weights for policy 0, policy_version 1454560 (0.0007) [2023-12-27 01:59:09,470][105692] Updated weights for policy 0, policy_version 1454570 (0.0008) [2023-12-27 01:59:10,047][105620] Updated weights for policy 1, policy_version 1456934 (0.0009) [2023-12-27 01:59:10,114][105620] Updated weights for policy 1, policy_version 1456944 (0.0008) [2023-12-27 01:59:10,178][105620] Updated weights for policy 1, policy_version 1456954 (0.0008) [2023-12-27 01:59:10,217][105692] Updated weights for policy 0, policy_version 1454580 (0.0009) [2023-12-27 01:59:10,272][105692] Updated weights for policy 0, policy_version 1454590 (0.0010) [2023-12-27 01:59:10,330][105692] Updated weights for policy 0, policy_version 1454600 (0.0010) [2023-12-27 01:59:10,848][105620] Updated weights for policy 1, policy_version 1456964 (0.0006) [2023-12-27 01:59:10,897][105620] Updated weights for policy 1, policy_version 1456974 (0.0007) [2023-12-27 01:59:10,945][105620] Updated weights for policy 1, policy_version 1456984 (0.0008) [2023-12-27 01:59:11,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 745472000. Throughput: 0: 9809.9, 1: 9844.6. Samples: 745478956. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 01:59:11,062][104569] Avg episode reward: [(0, '8624.681'), (1, '8893.850')] [2023-12-27 01:59:11,148][105692] Updated weights for policy 0, policy_version 1454610 (0.0009) [2023-12-27 01:59:11,212][105692] Updated weights for policy 0, policy_version 1454620 (0.0009) [2023-12-27 01:59:11,279][105692] Updated weights for policy 0, policy_version 1454630 (0.0009) [2023-12-27 01:59:11,332][105692] Updated weights for policy 0, policy_version 1454640 (0.0010) [2023-12-27 01:59:11,713][105620] Updated weights for policy 1, policy_version 1456994 (0.0008) [2023-12-27 01:59:11,781][105620] Updated weights for policy 1, policy_version 1457004 (0.0009) [2023-12-27 01:59:11,842][105620] Updated weights for policy 1, policy_version 1457014 (0.0009) [2023-12-27 01:59:11,908][105620] Updated weights for policy 1, policy_version 1457024 (0.0007) [2023-12-27 01:59:12,175][105692] Updated weights for policy 0, policy_version 1454651 (0.0010) [2023-12-27 01:59:12,228][105692] Updated weights for policy 0, policy_version 1454661 (0.0009) [2023-12-27 01:59:12,293][105692] Updated weights for policy 0, policy_version 1454671 (0.0009) [2023-12-27 01:59:12,695][105620] Updated weights for policy 1, policy_version 1457034 (0.0009) [2023-12-27 01:59:12,759][105620] Updated weights for policy 1, policy_version 1457044 (0.0007) [2023-12-27 01:59:12,823][105620] Updated weights for policy 1, policy_version 1457054 (0.0007) [2023-12-27 01:59:12,967][105692] Updated weights for policy 0, policy_version 1454681 (0.0006) [2023-12-27 01:59:13,014][105692] Updated weights for policy 0, policy_version 1454691 (0.0006) [2023-12-27 01:59:13,061][105692] Updated weights for policy 0, policy_version 1454702 (0.0007) [2023-12-27 01:59:13,555][105620] Updated weights for policy 1, policy_version 1457064 (0.0007) [2023-12-27 01:59:13,605][105620] Updated weights for policy 1, policy_version 1457074 (0.0005) [2023-12-27 01:59:13,651][105620] Updated weights for policy 1, policy_version 1457084 (0.0007) [2023-12-27 01:59:13,738][105692] Updated weights for policy 0, policy_version 1454712 (0.0005) [2023-12-27 01:59:13,798][105692] Updated weights for policy 0, policy_version 1454722 (0.0005) [2023-12-27 01:59:13,873][105692] Updated weights for policy 0, policy_version 1454732 (0.0005) [2023-12-27 01:59:14,295][105620] Updated weights for policy 1, policy_version 1457094 (0.0009) [2023-12-27 01:59:14,351][105620] Updated weights for policy 1, policy_version 1457104 (0.0008) [2023-12-27 01:59:14,410][105620] Updated weights for policy 1, policy_version 1457114 (0.0008) [2023-12-27 01:59:14,510][105692] Updated weights for policy 0, policy_version 1454742 (0.0006) [2023-12-27 01:59:14,560][105692] Updated weights for policy 0, policy_version 1454752 (0.0006) [2023-12-27 01:59:14,609][105692] Updated weights for policy 0, policy_version 1454762 (0.0005) [2023-12-27 01:59:15,202][105620] Updated weights for policy 1, policy_version 1457124 (0.0008) [2023-12-27 01:59:15,264][105620] Updated weights for policy 1, policy_version 1457134 (0.0009) [2023-12-27 01:59:15,305][105692] Updated weights for policy 0, policy_version 1454772 (0.0007) [2023-12-27 01:59:15,324][105620] Updated weights for policy 1, policy_version 1457144 (0.0009) [2023-12-27 01:59:15,364][105692] Updated weights for policy 0, policy_version 1454782 (0.0010) [2023-12-27 01:59:15,422][105692] Updated weights for policy 0, policy_version 1454792 (0.0009) [2023-12-27 01:59:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.6, 300 sec: 19633.0). Total num frames: 745562112. Throughput: 0: 9760.7, 1: 9787.4. Samples: 745534688. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 01:59:16,063][104569] Avg episode reward: [(0, '8620.472'), (1, '8867.900')] [2023-12-27 01:59:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001454800_372482048.pth... [2023-12-27 01:59:16,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001453648_372187136.pth [2023-12-27 01:59:16,092][105620] Updated weights for policy 1, policy_version 1457154 (0.0009) [2023-12-27 01:59:16,154][105620] Updated weights for policy 1, policy_version 1457164 (0.0008) [2023-12-27 01:59:16,156][105692] Updated weights for policy 0, policy_version 1454802 (0.0011) [2023-12-27 01:59:16,208][105692] Updated weights for policy 0, policy_version 1454812 (0.0010) [2023-12-27 01:59:16,210][105620] Updated weights for policy 1, policy_version 1457174 (0.0006) [2023-12-27 01:59:16,252][105692] Updated weights for policy 0, policy_version 1454822 (0.0010) [2023-12-27 01:59:16,270][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001457184_373088256.pth... [2023-12-27 01:59:16,271][105620] Updated weights for policy 1, policy_version 1457184 (0.0007) [2023-12-27 01:59:16,273][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001456032_372793344.pth [2023-12-27 01:59:16,310][105692] Updated weights for policy 0, policy_version 1454832 (0.0010) [2023-12-27 01:59:16,946][105692] Updated weights for policy 0, policy_version 1454842 (0.0005) [2023-12-27 01:59:17,002][105692] Updated weights for policy 0, policy_version 1454852 (0.0005) [2023-12-27 01:59:17,051][105620] Updated weights for policy 1, policy_version 1457194 (0.0010) [2023-12-27 01:59:17,058][105692] Updated weights for policy 0, policy_version 1454862 (0.0005) [2023-12-27 01:59:17,117][105620] Updated weights for policy 1, policy_version 1457204 (0.0010) [2023-12-27 01:59:17,182][105620] Updated weights for policy 1, policy_version 1457214 (0.0011) [2023-12-27 01:59:17,591][105692] Updated weights for policy 0, policy_version 1454872 (0.0005) [2023-12-27 01:59:17,637][105692] Updated weights for policy 0, policy_version 1454882 (0.0005) [2023-12-27 01:59:17,683][105692] Updated weights for policy 0, policy_version 1454892 (0.0005) [2023-12-27 01:59:17,751][105620] Updated weights for policy 1, policy_version 1457224 (0.0006) [2023-12-27 01:59:17,798][105620] Updated weights for policy 1, policy_version 1457234 (0.0005) [2023-12-27 01:59:17,851][105620] Updated weights for policy 1, policy_version 1457244 (0.0009) [2023-12-27 01:59:18,225][105692] Updated weights for policy 0, policy_version 1454902 (0.0007) [2023-12-27 01:59:18,276][105692] Updated weights for policy 0, policy_version 1454912 (0.0009) [2023-12-27 01:59:18,331][105692] Updated weights for policy 0, policy_version 1454922 (0.0009) [2023-12-27 01:59:18,490][105620] Updated weights for policy 1, policy_version 1457254 (0.0009) [2023-12-27 01:59:18,548][105620] Updated weights for policy 1, policy_version 1457264 (0.0009) [2023-12-27 01:59:18,609][105620] Updated weights for policy 1, policy_version 1457274 (0.0009) [2023-12-27 01:59:19,099][105692] Updated weights for policy 0, policy_version 1454932 (0.0010) [2023-12-27 01:59:19,147][105692] Updated weights for policy 0, policy_version 1454942 (0.0009) [2023-12-27 01:59:19,199][105692] Updated weights for policy 0, policy_version 1454952 (0.0009) [2023-12-27 01:59:19,365][105620] Updated weights for policy 1, policy_version 1457284 (0.0010) [2023-12-27 01:59:19,432][105620] Updated weights for policy 1, policy_version 1457294 (0.0010) [2023-12-27 01:59:19,499][105620] Updated weights for policy 1, policy_version 1457304 (0.0008) [2023-12-27 01:59:20,018][105692] Updated weights for policy 0, policy_version 1454962 (0.0009) [2023-12-27 01:59:20,081][105692] Updated weights for policy 0, policy_version 1454972 (0.0009) [2023-12-27 01:59:20,137][105692] Updated weights for policy 0, policy_version 1454982 (0.0009) [2023-12-27 01:59:20,194][105692] Updated weights for policy 0, policy_version 1454992 (0.0009) [2023-12-27 01:59:20,266][105620] Updated weights for policy 1, policy_version 1457314 (0.0008) [2023-12-27 01:59:20,328][105620] Updated weights for policy 1, policy_version 1457324 (0.0009) [2023-12-27 01:59:20,387][105620] Updated weights for policy 1, policy_version 1457334 (0.0010) [2023-12-27 01:59:20,445][105620] Updated weights for policy 1, policy_version 1457344 (0.0008) [2023-12-27 01:59:20,999][105692] Updated weights for policy 0, policy_version 1455002 (0.0006) [2023-12-27 01:59:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 745660416. Throughput: 0: 9882.4, 1: 9706.1. Samples: 745656164. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 01:59:21,062][104569] Avg episode reward: [(0, '8893.835'), (1, '6238.040')] [2023-12-27 01:59:21,074][105692] Updated weights for policy 0, policy_version 1455012 (0.0008) [2023-12-27 01:59:21,142][105692] Updated weights for policy 0, policy_version 1455022 (0.0007) [2023-12-27 01:59:21,222][105620] Updated weights for policy 1, policy_version 1457354 (0.0010) [2023-12-27 01:59:21,289][105620] Updated weights for policy 1, policy_version 1457364 (0.0009) [2023-12-27 01:59:21,357][105620] Updated weights for policy 1, policy_version 1457374 (0.0010) [2023-12-27 01:59:21,830][105692] Updated weights for policy 0, policy_version 1455032 (0.0008) [2023-12-27 01:59:21,884][105692] Updated weights for policy 0, policy_version 1455042 (0.0009) [2023-12-27 01:59:21,946][105692] Updated weights for policy 0, policy_version 1455052 (0.0008) [2023-12-27 01:59:22,148][105620] Updated weights for policy 1, policy_version 1457384 (0.0009) [2023-12-27 01:59:22,199][105620] Updated weights for policy 1, policy_version 1457394 (0.0008) [2023-12-27 01:59:22,250][105620] Updated weights for policy 1, policy_version 1457404 (0.0009) [2023-12-27 01:59:22,644][105692] Updated weights for policy 0, policy_version 1455062 (0.0009) [2023-12-27 01:59:22,710][105692] Updated weights for policy 0, policy_version 1455072 (0.0007) [2023-12-27 01:59:22,783][105692] Updated weights for policy 0, policy_version 1455082 (0.0007) [2023-12-27 01:59:23,070][105620] Updated weights for policy 1, policy_version 1457414 (0.0009) [2023-12-27 01:59:23,134][105620] Updated weights for policy 1, policy_version 1457424 (0.0007) [2023-12-27 01:59:23,180][105620] Updated weights for policy 1, policy_version 1457434 (0.0005) [2023-12-27 01:59:23,543][105692] Updated weights for policy 0, policy_version 1455092 (0.0008) [2023-12-27 01:59:23,593][105692] Updated weights for policy 0, policy_version 1455102 (0.0005) [2023-12-27 01:59:23,647][105692] Updated weights for policy 0, policy_version 1455112 (0.0005) [2023-12-27 01:59:23,844][105620] Updated weights for policy 1, policy_version 1457444 (0.0007) [2023-12-27 01:59:23,901][105620] Updated weights for policy 1, policy_version 1457454 (0.0009) [2023-12-27 01:59:23,963][105620] Updated weights for policy 1, policy_version 1457464 (0.0009) [2023-12-27 01:59:24,352][105692] Updated weights for policy 0, policy_version 1455122 (0.0008) [2023-12-27 01:59:24,414][105692] Updated weights for policy 0, policy_version 1455132 (0.0008) [2023-12-27 01:59:24,469][105692] Updated weights for policy 0, policy_version 1455142 (0.0010) [2023-12-27 01:59:24,523][105692] Updated weights for policy 0, policy_version 1455152 (0.0010) [2023-12-27 01:59:24,636][105620] Updated weights for policy 1, policy_version 1457474 (0.0008) [2023-12-27 01:59:24,694][105620] Updated weights for policy 1, policy_version 1457484 (0.0005) [2023-12-27 01:59:24,748][105620] Updated weights for policy 1, policy_version 1457494 (0.0005) [2023-12-27 01:59:24,815][105620] Updated weights for policy 1, policy_version 1457504 (0.0005) [2023-12-27 01:59:25,227][105692] Updated weights for policy 0, policy_version 1455162 (0.0006) [2023-12-27 01:59:25,283][105692] Updated weights for policy 0, policy_version 1455172 (0.0006) [2023-12-27 01:59:25,338][105692] Updated weights for policy 0, policy_version 1455182 (0.0006) [2023-12-27 01:59:25,403][105620] Updated weights for policy 1, policy_version 1457514 (0.0010) [2023-12-27 01:59:25,457][105620] Updated weights for policy 1, policy_version 1457524 (0.0010) [2023-12-27 01:59:25,505][105620] Updated weights for policy 1, policy_version 1457534 (0.0010) [2023-12-27 01:59:25,977][105692] Updated weights for policy 0, policy_version 1455192 (0.0008) [2023-12-27 01:59:26,026][105692] Updated weights for policy 0, policy_version 1455202 (0.0008) [2023-12-27 01:59:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 745758720. Throughput: 0: 9970.3, 1: 9742.8. Samples: 745771564. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 01:59:26,062][104569] Avg episode reward: [(0, '8630.259'), (1, '6639.971')] [2023-12-27 01:59:26,087][105692] Updated weights for policy 0, policy_version 1455212 (0.0008) [2023-12-27 01:59:26,254][105620] Updated weights for policy 1, policy_version 1457544 (0.0010) [2023-12-27 01:59:26,312][105620] Updated weights for policy 1, policy_version 1457554 (0.0010) [2023-12-27 01:59:26,371][105620] Updated weights for policy 1, policy_version 1457564 (0.0011) [2023-12-27 01:59:26,764][105692] Updated weights for policy 0, policy_version 1455222 (0.0007) [2023-12-27 01:59:26,829][105692] Updated weights for policy 0, policy_version 1455232 (0.0006) [2023-12-27 01:59:26,896][105692] Updated weights for policy 0, policy_version 1455242 (0.0005) [2023-12-27 01:59:27,112][105620] Updated weights for policy 1, policy_version 1457574 (0.0011) [2023-12-27 01:59:27,166][105620] Updated weights for policy 1, policy_version 1457584 (0.0010) [2023-12-27 01:59:27,217][105620] Updated weights for policy 1, policy_version 1457594 (0.0010) [2023-12-27 01:59:27,402][105692] Updated weights for policy 0, policy_version 1455252 (0.0005) [2023-12-27 01:59:27,469][105692] Updated weights for policy 0, policy_version 1455262 (0.0005) [2023-12-27 01:59:27,539][105692] Updated weights for policy 0, policy_version 1455272 (0.0005) [2023-12-27 01:59:27,951][105620] Updated weights for policy 1, policy_version 1457604 (0.0010) [2023-12-27 01:59:28,006][105620] Updated weights for policy 1, policy_version 1457614 (0.0009) [2023-12-27 01:59:28,061][105620] Updated weights for policy 1, policy_version 1457624 (0.0005) [2023-12-27 01:59:28,125][105692] Updated weights for policy 0, policy_version 1455282 (0.0006) [2023-12-27 01:59:28,192][105692] Updated weights for policy 0, policy_version 1455292 (0.0010) [2023-12-27 01:59:28,242][105692] Updated weights for policy 0, policy_version 1455302 (0.0009) [2023-12-27 01:59:28,301][105692] Updated weights for policy 0, policy_version 1455312 (0.0005) [2023-12-27 01:59:28,671][105620] Updated weights for policy 1, policy_version 1457634 (0.0006) [2023-12-27 01:59:28,736][105620] Updated weights for policy 1, policy_version 1457644 (0.0007) [2023-12-27 01:59:28,788][105620] Updated weights for policy 1, policy_version 1457654 (0.0005) [2023-12-27 01:59:28,835][105620] Updated weights for policy 1, policy_version 1457664 (0.0007) [2023-12-27 01:59:28,891][105692] Updated weights for policy 0, policy_version 1455322 (0.0010) [2023-12-27 01:59:28,949][105692] Updated weights for policy 0, policy_version 1455332 (0.0010) [2023-12-27 01:59:29,009][105692] Updated weights for policy 0, policy_version 1455342 (0.0008) [2023-12-27 01:59:29,541][105620] Updated weights for policy 1, policy_version 1457674 (0.0008) [2023-12-27 01:59:29,593][105620] Updated weights for policy 1, policy_version 1457684 (0.0008) [2023-12-27 01:59:29,654][105620] Updated weights for policy 1, policy_version 1457694 (0.0007) [2023-12-27 01:59:29,683][105692] Updated weights for policy 0, policy_version 1455352 (0.0009) [2023-12-27 01:59:29,737][105692] Updated weights for policy 0, policy_version 1455362 (0.0007) [2023-12-27 01:59:29,794][105692] Updated weights for policy 0, policy_version 1455372 (0.0007) [2023-12-27 01:59:30,338][105620] Updated weights for policy 1, policy_version 1457704 (0.0005) [2023-12-27 01:59:30,394][105620] Updated weights for policy 1, policy_version 1457714 (0.0005) [2023-12-27 01:59:30,457][105620] Updated weights for policy 1, policy_version 1457724 (0.0006) [2023-12-27 01:59:30,566][105692] Updated weights for policy 0, policy_version 1455382 (0.0009) [2023-12-27 01:59:30,629][105692] Updated weights for policy 0, policy_version 1455392 (0.0010) [2023-12-27 01:59:30,690][105692] Updated weights for policy 0, policy_version 1455402 (0.0010) [2023-12-27 01:59:30,996][105620] Updated weights for policy 1, policy_version 1457734 (0.0009) [2023-12-27 01:59:31,051][105620] Updated weights for policy 1, policy_version 1457744 (0.0010) [2023-12-27 01:59:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 745865216. Throughput: 0: 10079.7, 1: 9734.4. Samples: 745835232. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 01:59:31,062][104569] Avg episode reward: [(0, '8261.655'), (1, '8435.190')] [2023-12-27 01:59:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001455408_372637696.pth... [2023-12-27 01:59:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001454224_372334592.pth [2023-12-27 01:59:31,099][105620] Updated weights for policy 1, policy_version 1457754 (0.0010) [2023-12-27 01:59:31,135][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001457760_373235712.pth... [2023-12-27 01:59:31,141][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001456608_372940800.pth [2023-12-27 01:59:31,396][105692] Updated weights for policy 0, policy_version 1455412 (0.0009) [2023-12-27 01:59:31,455][105692] Updated weights for policy 0, policy_version 1455422 (0.0008) [2023-12-27 01:59:31,513][105692] Updated weights for policy 0, policy_version 1455432 (0.0011) [2023-12-27 01:59:31,892][105620] Updated weights for policy 1, policy_version 1457764 (0.0010) [2023-12-27 01:59:31,953][105620] Updated weights for policy 1, policy_version 1457774 (0.0009) [2023-12-27 01:59:32,019][105620] Updated weights for policy 1, policy_version 1457784 (0.0009) [2023-12-27 01:59:32,163][105692] Updated weights for policy 0, policy_version 1455442 (0.0010) [2023-12-27 01:59:32,224][105692] Updated weights for policy 0, policy_version 1455452 (0.0009) [2023-12-27 01:59:32,280][105692] Updated weights for policy 0, policy_version 1455462 (0.0008) [2023-12-27 01:59:32,344][105692] Updated weights for policy 0, policy_version 1455472 (0.0007) [2023-12-27 01:59:32,742][105620] Updated weights for policy 1, policy_version 1457794 (0.0009) [2023-12-27 01:59:32,799][105620] Updated weights for policy 1, policy_version 1457804 (0.0008) [2023-12-27 01:59:32,859][105620] Updated weights for policy 1, policy_version 1457814 (0.0008) [2023-12-27 01:59:32,919][105620] Updated weights for policy 1, policy_version 1457824 (0.0008) [2023-12-27 01:59:33,080][105692] Updated weights for policy 0, policy_version 1455482 (0.0009) [2023-12-27 01:59:33,131][105692] Updated weights for policy 0, policy_version 1455492 (0.0008) [2023-12-27 01:59:33,182][105692] Updated weights for policy 0, policy_version 1455502 (0.0005) [2023-12-27 01:59:33,636][105620] Updated weights for policy 1, policy_version 1457834 (0.0005) [2023-12-27 01:59:33,690][105620] Updated weights for policy 1, policy_version 1457844 (0.0008) [2023-12-27 01:59:33,736][105620] Updated weights for policy 1, policy_version 1457854 (0.0008) [2023-12-27 01:59:33,841][105692] Updated weights for policy 0, policy_version 1455512 (0.0008) [2023-12-27 01:59:33,890][105692] Updated weights for policy 0, policy_version 1455522 (0.0009) [2023-12-27 01:59:33,937][105692] Updated weights for policy 0, policy_version 1455532 (0.0008) [2023-12-27 01:59:34,431][105620] Updated weights for policy 1, policy_version 1457864 (0.0009) [2023-12-27 01:59:34,497][105620] Updated weights for policy 1, policy_version 1457874 (0.0008) [2023-12-27 01:59:34,564][105620] Updated weights for policy 1, policy_version 1457884 (0.0006) [2023-12-27 01:59:34,770][105692] Updated weights for policy 0, policy_version 1455542 (0.0010) [2023-12-27 01:59:34,829][105692] Updated weights for policy 0, policy_version 1455552 (0.0011) [2023-12-27 01:59:34,886][105692] Updated weights for policy 0, policy_version 1455562 (0.0011) [2023-12-27 01:59:35,273][105620] Updated weights for policy 1, policy_version 1457894 (0.0007) [2023-12-27 01:59:35,338][105620] Updated weights for policy 1, policy_version 1457904 (0.0008) [2023-12-27 01:59:35,393][105620] Updated weights for policy 1, policy_version 1457914 (0.0008) [2023-12-27 01:59:35,626][105692] Updated weights for policy 0, policy_version 1455572 (0.0011) [2023-12-27 01:59:35,674][105692] Updated weights for policy 0, policy_version 1455582 (0.0010) [2023-12-27 01:59:35,721][105692] Updated weights for policy 0, policy_version 1455592 (0.0010) [2023-12-27 01:59:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 745963520. Throughput: 0: 9981.1, 1: 9803.2. Samples: 745954332. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 01:59:36,063][104569] Avg episode reward: [(0, '8346.294'), (1, '8911.120')] [2023-12-27 01:59:36,129][105620] Updated weights for policy 1, policy_version 1457924 (0.0008) [2023-12-27 01:59:36,189][105620] Updated weights for policy 1, policy_version 1457934 (0.0005) [2023-12-27 01:59:36,250][105620] Updated weights for policy 1, policy_version 1457944 (0.0008) [2023-12-27 01:59:36,463][105692] Updated weights for policy 0, policy_version 1455602 (0.0010) [2023-12-27 01:59:36,534][105692] Updated weights for policy 0, policy_version 1455612 (0.0007) [2023-12-27 01:59:36,609][105692] Updated weights for policy 0, policy_version 1455622 (0.0010) [2023-12-27 01:59:36,682][105692] Updated weights for policy 0, policy_version 1455632 (0.0011) [2023-12-27 01:59:36,834][105620] Updated weights for policy 1, policy_version 1457954 (0.0007) [2023-12-27 01:59:36,898][105620] Updated weights for policy 1, policy_version 1457964 (0.0008) [2023-12-27 01:59:36,950][105620] Updated weights for policy 1, policy_version 1457974 (0.0007) [2023-12-27 01:59:37,014][105620] Updated weights for policy 1, policy_version 1457984 (0.0009) [2023-12-27 01:59:37,380][105692] Updated weights for policy 0, policy_version 1455642 (0.0011) [2023-12-27 01:59:37,425][105692] Updated weights for policy 0, policy_version 1455652 (0.0010) [2023-12-27 01:59:37,474][105692] Updated weights for policy 0, policy_version 1455662 (0.0010) [2023-12-27 01:59:37,632][105620] Updated weights for policy 1, policy_version 1457994 (0.0008) [2023-12-27 01:59:37,700][105620] Updated weights for policy 1, policy_version 1458004 (0.0008) [2023-12-27 01:59:37,766][105620] Updated weights for policy 1, policy_version 1458014 (0.0008) [2023-12-27 01:59:38,250][105692] Updated weights for policy 0, policy_version 1455672 (0.0011) [2023-12-27 01:59:38,300][105692] Updated weights for policy 0, policy_version 1455682 (0.0011) [2023-12-27 01:59:38,362][105692] Updated weights for policy 0, policy_version 1455692 (0.0011) [2023-12-27 01:59:38,418][105620] Updated weights for policy 1, policy_version 1458024 (0.0008) [2023-12-27 01:59:38,485][105620] Updated weights for policy 1, policy_version 1458034 (0.0005) [2023-12-27 01:59:38,547][105620] Updated weights for policy 1, policy_version 1458044 (0.0005) [2023-12-27 01:59:39,122][105692] Updated weights for policy 0, policy_version 1455702 (0.0011) [2023-12-27 01:59:39,180][105692] Updated weights for policy 0, policy_version 1455712 (0.0011) [2023-12-27 01:59:39,215][105620] Updated weights for policy 1, policy_version 1458054 (0.0008) [2023-12-27 01:59:39,237][105692] Updated weights for policy 0, policy_version 1455722 (0.0011) [2023-12-27 01:59:39,277][105620] Updated weights for policy 1, policy_version 1458064 (0.0007) [2023-12-27 01:59:39,335][105620] Updated weights for policy 1, policy_version 1458074 (0.0009) [2023-12-27 01:59:39,907][105692] Updated weights for policy 0, policy_version 1455732 (0.0010) [2023-12-27 01:59:39,964][105692] Updated weights for policy 0, policy_version 1455742 (0.0009) [2023-12-27 01:59:40,019][105692] Updated weights for policy 0, policy_version 1455752 (0.0008) [2023-12-27 01:59:40,179][105620] Updated weights for policy 1, policy_version 1458084 (0.0007) [2023-12-27 01:59:40,240][105620] Updated weights for policy 1, policy_version 1458094 (0.0007) [2023-12-27 01:59:40,300][105620] Updated weights for policy 1, policy_version 1458104 (0.0007) [2023-12-27 01:59:40,727][105692] Updated weights for policy 0, policy_version 1455762 (0.0005) [2023-12-27 01:59:40,787][105692] Updated weights for policy 0, policy_version 1455772 (0.0008) [2023-12-27 01:59:40,846][105692] Updated weights for policy 0, policy_version 1455782 (0.0011) [2023-12-27 01:59:40,895][105620] Updated weights for policy 1, policy_version 1458114 (0.0008) [2023-12-27 01:59:40,911][105692] Updated weights for policy 0, policy_version 1455792 (0.0010) [2023-12-27 01:59:40,946][105620] Updated weights for policy 1, policy_version 1458124 (0.0006) [2023-12-27 01:59:40,998][105620] Updated weights for policy 1, policy_version 1458134 (0.0007) [2023-12-27 01:59:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 746061824. Throughput: 0: 10008.4, 1: 9881.7. Samples: 746072196. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 01:59:41,063][104569] Avg episode reward: [(0, '8440.980'), (1, '8805.675')] [2023-12-27 01:59:41,078][105620] Updated weights for policy 1, policy_version 1458144 (0.0009) [2023-12-27 01:59:41,655][105692] Updated weights for policy 0, policy_version 1455802 (0.0008) [2023-12-27 01:59:41,714][105692] Updated weights for policy 0, policy_version 1455812 (0.0009) [2023-12-27 01:59:41,782][105692] Updated weights for policy 0, policy_version 1455822 (0.0008) [2023-12-27 01:59:41,801][105620] Updated weights for policy 1, policy_version 1458154 (0.0007) [2023-12-27 01:59:41,857][105620] Updated weights for policy 1, policy_version 1458164 (0.0008) [2023-12-27 01:59:41,917][105620] Updated weights for policy 1, policy_version 1458174 (0.0006) [2023-12-27 01:59:42,529][105692] Updated weights for policy 0, policy_version 1455832 (0.0009) [2023-12-27 01:59:42,592][105692] Updated weights for policy 0, policy_version 1455842 (0.0009) [2023-12-27 01:59:42,653][105692] Updated weights for policy 0, policy_version 1455852 (0.0008) [2023-12-27 01:59:42,663][105620] Updated weights for policy 1, policy_version 1458184 (0.0007) [2023-12-27 01:59:42,737][105620] Updated weights for policy 1, policy_version 1458194 (0.0009) [2023-12-27 01:59:42,804][105620] Updated weights for policy 1, policy_version 1458204 (0.0008) [2023-12-27 01:59:43,338][105620] Updated weights for policy 1, policy_version 1458214 (0.0007) [2023-12-27 01:59:43,383][105692] Updated weights for policy 0, policy_version 1455862 (0.0008) [2023-12-27 01:59:43,384][105620] Updated weights for policy 1, policy_version 1458224 (0.0005) [2023-12-27 01:59:43,430][105620] Updated weights for policy 1, policy_version 1458234 (0.0005) [2023-12-27 01:59:43,442][105692] Updated weights for policy 0, policy_version 1455872 (0.0010) [2023-12-27 01:59:43,493][105692] Updated weights for policy 0, policy_version 1455882 (0.0010) [2023-12-27 01:59:43,966][105620] Updated weights for policy 1, policy_version 1458244 (0.0006) [2023-12-27 01:59:44,023][105620] Updated weights for policy 1, policy_version 1458254 (0.0006) [2023-12-27 01:59:44,086][105620] Updated weights for policy 1, policy_version 1458264 (0.0005) [2023-12-27 01:59:44,086][105692] Updated weights for policy 0, policy_version 1455892 (0.0008) [2023-12-27 01:59:44,149][105692] Updated weights for policy 0, policy_version 1455902 (0.0006) [2023-12-27 01:59:44,221][105692] Updated weights for policy 0, policy_version 1455912 (0.0006) [2023-12-27 01:59:44,731][105692] Updated weights for policy 0, policy_version 1455922 (0.0006) [2023-12-27 01:59:44,791][105692] Updated weights for policy 0, policy_version 1455932 (0.0007) [2023-12-27 01:59:44,808][105620] Updated weights for policy 1, policy_version 1458274 (0.0007) [2023-12-27 01:59:44,847][105692] Updated weights for policy 0, policy_version 1455942 (0.0008) [2023-12-27 01:59:44,859][105620] Updated weights for policy 1, policy_version 1458284 (0.0006) [2023-12-27 01:59:44,907][105692] Updated weights for policy 0, policy_version 1455952 (0.0009) [2023-12-27 01:59:44,922][105620] Updated weights for policy 1, policy_version 1458294 (0.0006) [2023-12-27 01:59:44,982][105620] Updated weights for policy 1, policy_version 1458304 (0.0006) [2023-12-27 01:59:45,539][105620] Updated weights for policy 1, policy_version 1458314 (0.0011) [2023-12-27 01:59:45,601][105620] Updated weights for policy 1, policy_version 1458324 (0.0011) [2023-12-27 01:59:45,666][105620] Updated weights for policy 1, policy_version 1458334 (0.0008) [2023-12-27 01:59:45,740][105692] Updated weights for policy 0, policy_version 1455962 (0.0005) [2023-12-27 01:59:45,786][105692] Updated weights for policy 0, policy_version 1455972 (0.0005) [2023-12-27 01:59:45,838][105692] Updated weights for policy 0, policy_version 1455982 (0.0005) [2023-12-27 01:59:46,062][104569] Fps is (10 sec: 20479.3, 60 sec: 19933.8, 300 sec: 19688.5). Total num frames: 746168320. Throughput: 0: 9928.6, 1: 9904.8. Samples: 746132152. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 01:59:46,063][104569] Avg episode reward: [(0, '8710.716'), (1, '8781.596')] [2023-12-27 01:59:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001455984_372785152.pth... [2023-12-27 01:59:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001458336_373383168.pth... [2023-12-27 01:59:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001454800_372482048.pth [2023-12-27 01:59:46,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001457184_373088256.pth [2023-12-27 01:59:46,204][105620] Updated weights for policy 1, policy_version 1458344 (0.0005) [2023-12-27 01:59:46,256][105620] Updated weights for policy 1, policy_version 1458354 (0.0009) [2023-12-27 01:59:46,320][105620] Updated weights for policy 1, policy_version 1458364 (0.0007) [2023-12-27 01:59:46,372][105692] Updated weights for policy 0, policy_version 1455992 (0.0008) [2023-12-27 01:59:46,431][105692] Updated weights for policy 0, policy_version 1456002 (0.0010) [2023-12-27 01:59:46,484][105692] Updated weights for policy 0, policy_version 1456012 (0.0008) [2023-12-27 01:59:46,899][105620] Updated weights for policy 1, policy_version 1458374 (0.0006) [2023-12-27 01:59:46,962][105620] Updated weights for policy 1, policy_version 1458384 (0.0007) [2023-12-27 01:59:47,014][105620] Updated weights for policy 1, policy_version 1458394 (0.0011) [2023-12-27 01:59:47,062][105692] Updated weights for policy 0, policy_version 1456022 (0.0007) [2023-12-27 01:59:47,108][105692] Updated weights for policy 0, policy_version 1456032 (0.0005) [2023-12-27 01:59:47,152][105692] Updated weights for policy 0, policy_version 1456042 (0.0005) [2023-12-27 01:59:47,685][105620] Updated weights for policy 1, policy_version 1458404 (0.0008) [2023-12-27 01:59:47,749][105620] Updated weights for policy 1, policy_version 1458414 (0.0006) [2023-12-27 01:59:47,806][105620] Updated weights for policy 1, policy_version 1458424 (0.0005) [2023-12-27 01:59:47,890][105692] Updated weights for policy 0, policy_version 1456052 (0.0007) [2023-12-27 01:59:47,945][105692] Updated weights for policy 0, policy_version 1456062 (0.0008) [2023-12-27 01:59:48,004][105692] Updated weights for policy 0, policy_version 1456072 (0.0006) [2023-12-27 01:59:48,455][105620] Updated weights for policy 1, policy_version 1458434 (0.0006) [2023-12-27 01:59:48,507][105620] Updated weights for policy 1, policy_version 1458444 (0.0011) [2023-12-27 01:59:48,570][105620] Updated weights for policy 1, policy_version 1458454 (0.0011) [2023-12-27 01:59:48,636][105620] Updated weights for policy 1, policy_version 1458464 (0.0007) [2023-12-27 01:59:48,673][105692] Updated weights for policy 0, policy_version 1456082 (0.0007) [2023-12-27 01:59:48,740][105692] Updated weights for policy 0, policy_version 1456092 (0.0006) [2023-12-27 01:59:48,808][105692] Updated weights for policy 0, policy_version 1456102 (0.0006) [2023-12-27 01:59:48,878][105692] Updated weights for policy 0, policy_version 1456112 (0.0005) [2023-12-27 01:59:49,207][105620] Updated weights for policy 1, policy_version 1458474 (0.0005) [2023-12-27 01:59:49,276][105620] Updated weights for policy 1, policy_version 1458484 (0.0009) [2023-12-27 01:59:49,347][105620] Updated weights for policy 1, policy_version 1458494 (0.0010) [2023-12-27 01:59:49,535][105692] Updated weights for policy 0, policy_version 1456122 (0.0008) [2023-12-27 01:59:49,593][105692] Updated weights for policy 0, policy_version 1456132 (0.0009) [2023-12-27 01:59:49,652][105692] Updated weights for policy 0, policy_version 1456142 (0.0008) [2023-12-27 01:59:50,060][105620] Updated weights for policy 1, policy_version 1458504 (0.0006) [2023-12-27 01:59:50,130][105620] Updated weights for policy 1, policy_version 1458514 (0.0007) [2023-12-27 01:59:50,185][105620] Updated weights for policy 1, policy_version 1458524 (0.0009) [2023-12-27 01:59:50,362][105692] Updated weights for policy 0, policy_version 1456152 (0.0008) [2023-12-27 01:59:50,416][105692] Updated weights for policy 0, policy_version 1456162 (0.0009) [2023-12-27 01:59:50,472][105692] Updated weights for policy 0, policy_version 1456172 (0.0008) [2023-12-27 01:59:50,869][105620] Updated weights for policy 1, policy_version 1458534 (0.0010) [2023-12-27 01:59:50,925][105620] Updated weights for policy 1, policy_version 1458544 (0.0005) [2023-12-27 01:59:50,987][105620] Updated weights for policy 1, policy_version 1458554 (0.0008) [2023-12-27 01:59:51,062][104569] Fps is (10 sec: 21299.5, 60 sec: 20070.4, 300 sec: 19688.6). Total num frames: 746274816. Throughput: 0: 10058.9, 1: 10045.1. Samples: 746261552. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 01:59:51,062][104569] Avg episode reward: [(0, '9073.896'), (1, '8684.656')] [2023-12-27 01:59:51,259][105692] Updated weights for policy 0, policy_version 1456182 (0.0007) [2023-12-27 01:59:51,319][105692] Updated weights for policy 0, policy_version 1456192 (0.0008) [2023-12-27 01:59:51,395][105692] Updated weights for policy 0, policy_version 1456202 (0.0009) [2023-12-27 01:59:51,763][105620] Updated weights for policy 1, policy_version 1458564 (0.0010) [2023-12-27 01:59:51,826][105620] Updated weights for policy 1, policy_version 1458574 (0.0009) [2023-12-27 01:59:51,893][105620] Updated weights for policy 1, policy_version 1458584 (0.0007) [2023-12-27 01:59:52,132][105692] Updated weights for policy 0, policy_version 1456213 (0.0010) [2023-12-27 01:59:52,196][105692] Updated weights for policy 0, policy_version 1456223 (0.0010) [2023-12-27 01:59:52,250][105692] Updated weights for policy 0, policy_version 1456233 (0.0010) [2023-12-27 01:59:52,590][105620] Updated weights for policy 1, policy_version 1458594 (0.0008) [2023-12-27 01:59:52,650][105620] Updated weights for policy 1, policy_version 1458604 (0.0006) [2023-12-27 01:59:52,701][105620] Updated weights for policy 1, policy_version 1458614 (0.0005) [2023-12-27 01:59:52,756][105620] Updated weights for policy 1, policy_version 1458624 (0.0008) [2023-12-27 01:59:53,020][105692] Updated weights for policy 0, policy_version 1456243 (0.0009) [2023-12-27 01:59:53,083][105692] Updated weights for policy 0, policy_version 1456253 (0.0009) [2023-12-27 01:59:53,140][105692] Updated weights for policy 0, policy_version 1456263 (0.0008) [2023-12-27 01:59:53,496][105620] Updated weights for policy 1, policy_version 1458634 (0.0007) [2023-12-27 01:59:53,565][105620] Updated weights for policy 1, policy_version 1458644 (0.0009) [2023-12-27 01:59:53,618][105620] Updated weights for policy 1, policy_version 1458654 (0.0011) [2023-12-27 01:59:53,832][105692] Updated weights for policy 0, policy_version 1456273 (0.0007) [2023-12-27 01:59:53,885][105692] Updated weights for policy 0, policy_version 1456283 (0.0005) [2023-12-27 01:59:53,939][105692] Updated weights for policy 0, policy_version 1456293 (0.0008) [2023-12-27 01:59:54,001][105692] Updated weights for policy 0, policy_version 1456303 (0.0010) [2023-12-27 01:59:54,327][105620] Updated weights for policy 1, policy_version 1458664 (0.0008) [2023-12-27 01:59:54,374][105620] Updated weights for policy 1, policy_version 1458674 (0.0008) [2023-12-27 01:59:54,427][105620] Updated weights for policy 1, policy_version 1458684 (0.0008) [2023-12-27 01:59:54,717][105692] Updated weights for policy 0, policy_version 1456313 (0.0010) [2023-12-27 01:59:54,770][105692] Updated weights for policy 0, policy_version 1456323 (0.0010) [2023-12-27 01:59:54,822][105692] Updated weights for policy 0, policy_version 1456333 (0.0010) [2023-12-27 01:59:55,151][105620] Updated weights for policy 1, policy_version 1458694 (0.0008) [2023-12-27 01:59:55,202][105620] Updated weights for policy 1, policy_version 1458704 (0.0008) [2023-12-27 01:59:55,253][105620] Updated weights for policy 1, policy_version 1458714 (0.0008) [2023-12-27 01:59:55,579][105692] Updated weights for policy 0, policy_version 1456343 (0.0010) [2023-12-27 01:59:55,641][105692] Updated weights for policy 0, policy_version 1456353 (0.0010) [2023-12-27 01:59:55,702][105692] Updated weights for policy 0, policy_version 1456363 (0.0010) [2023-12-27 01:59:55,913][105620] Updated weights for policy 1, policy_version 1458724 (0.0006) [2023-12-27 01:59:55,971][105620] Updated weights for policy 1, policy_version 1458734 (0.0005) [2023-12-27 01:59:56,035][105620] Updated weights for policy 1, policy_version 1458744 (0.0006) [2023-12-27 01:59:56,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19933.8, 300 sec: 19660.8). Total num frames: 746364928. Throughput: 0: 9964.6, 1: 9992.2. Samples: 746377012. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 01:59:56,062][104569] Avg episode reward: [(0, '8803.358'), (1, '8611.646')] [2023-12-27 01:59:56,379][105692] Updated weights for policy 0, policy_version 1456373 (0.0009) [2023-12-27 01:59:56,444][105692] Updated weights for policy 0, policy_version 1456383 (0.0011) [2023-12-27 01:59:56,496][105692] Updated weights for policy 0, policy_version 1456393 (0.0010) [2023-12-27 01:59:56,616][105620] Updated weights for policy 1, policy_version 1458754 (0.0006) [2023-12-27 01:59:56,665][105620] Updated weights for policy 1, policy_version 1458764 (0.0008) [2023-12-27 01:59:56,726][105620] Updated weights for policy 1, policy_version 1458774 (0.0010) [2023-12-27 01:59:56,783][105620] Updated weights for policy 1, policy_version 1458784 (0.0008) [2023-12-27 01:59:57,177][105692] Updated weights for policy 0, policy_version 1456403 (0.0010) [2023-12-27 01:59:57,228][105692] Updated weights for policy 0, policy_version 1456413 (0.0010) [2023-12-27 01:59:57,275][105692] Updated weights for policy 0, policy_version 1456423 (0.0010) [2023-12-27 01:59:57,522][105620] Updated weights for policy 1, policy_version 1458794 (0.0008) [2023-12-27 01:59:57,576][105620] Updated weights for policy 1, policy_version 1458804 (0.0008) [2023-12-27 01:59:57,631][105620] Updated weights for policy 1, policy_version 1458814 (0.0008) [2023-12-27 01:59:57,940][105692] Updated weights for policy 0, policy_version 1456433 (0.0010) [2023-12-27 01:59:57,998][105692] Updated weights for policy 0, policy_version 1456443 (0.0005) [2023-12-27 01:59:58,061][105692] Updated weights for policy 0, policy_version 1456453 (0.0007) [2023-12-27 01:59:58,123][105692] Updated weights for policy 0, policy_version 1456463 (0.0007) [2023-12-27 01:59:58,360][105620] Updated weights for policy 1, policy_version 1458824 (0.0008) [2023-12-27 01:59:58,426][105620] Updated weights for policy 1, policy_version 1458834 (0.0007) [2023-12-27 01:59:58,495][105620] Updated weights for policy 1, policy_version 1458844 (0.0008) [2023-12-27 01:59:58,926][105692] Updated weights for policy 0, policy_version 1456473 (0.0009) [2023-12-27 01:59:58,988][105692] Updated weights for policy 0, policy_version 1456483 (0.0007) [2023-12-27 01:59:59,048][105692] Updated weights for policy 0, policy_version 1456493 (0.0006) [2023-12-27 01:59:59,263][105620] Updated weights for policy 1, policy_version 1458854 (0.0008) [2023-12-27 01:59:59,324][105620] Updated weights for policy 1, policy_version 1458864 (0.0010) [2023-12-27 01:59:59,391][105620] Updated weights for policy 1, policy_version 1458874 (0.0011) [2023-12-27 01:59:59,804][105692] Updated weights for policy 0, policy_version 1456503 (0.0008) [2023-12-27 01:59:59,866][105692] Updated weights for policy 0, policy_version 1456513 (0.0009) [2023-12-27 01:59:59,923][105692] Updated weights for policy 0, policy_version 1456523 (0.0008) [2023-12-27 02:00:00,203][105620] Updated weights for policy 1, policy_version 1458884 (0.0011) [2023-12-27 02:00:00,262][105620] Updated weights for policy 1, policy_version 1458894 (0.0011) [2023-12-27 02:00:00,320][105620] Updated weights for policy 1, policy_version 1458904 (0.0010) [2023-12-27 02:00:00,695][105692] Updated weights for policy 0, policy_version 1456533 (0.0007) [2023-12-27 02:00:00,746][105692] Updated weights for policy 0, policy_version 1456543 (0.0005) [2023-12-27 02:00:00,793][105692] Updated weights for policy 0, policy_version 1456553 (0.0005) [2023-12-27 02:00:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19933.9, 300 sec: 19660.8). Total num frames: 746463232. Throughput: 0: 10008.6, 1: 10030.6. Samples: 746436448. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 02:00:01,063][104569] Avg episode reward: [(0, '8621.674'), (1, '8799.954')] [2023-12-27 02:00:01,068][105620] Updated weights for policy 1, policy_version 1458914 (0.0010) [2023-12-27 02:00:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001456560_372932608.pth... [2023-12-27 02:00:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001455408_372637696.pth [2023-12-27 02:00:01,135][105620] Updated weights for policy 1, policy_version 1458924 (0.0007) [2023-12-27 02:00:01,196][105620] Updated weights for policy 1, policy_version 1458934 (0.0009) [2023-12-27 02:00:01,256][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001458944_373538816.pth... [2023-12-27 02:00:01,258][105620] Updated weights for policy 1, policy_version 1458944 (0.0009) [2023-12-27 02:00:01,260][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001457760_373235712.pth [2023-12-27 02:00:01,449][105692] Updated weights for policy 0, policy_version 1456563 (0.0005) [2023-12-27 02:00:01,502][105692] Updated weights for policy 0, policy_version 1456573 (0.0006) [2023-12-27 02:00:01,555][105692] Updated weights for policy 0, policy_version 1456583 (0.0005) [2023-12-27 02:00:02,015][105620] Updated weights for policy 1, policy_version 1458954 (0.0008) [2023-12-27 02:00:02,087][105620] Updated weights for policy 1, policy_version 1458964 (0.0005) [2023-12-27 02:00:02,156][105620] Updated weights for policy 1, policy_version 1458974 (0.0010) [2023-12-27 02:00:02,231][105692] Updated weights for policy 0, policy_version 1456593 (0.0007) [2023-12-27 02:00:02,298][105692] Updated weights for policy 0, policy_version 1456603 (0.0009) [2023-12-27 02:00:02,359][105692] Updated weights for policy 0, policy_version 1456613 (0.0008) [2023-12-27 02:00:02,417][105692] Updated weights for policy 0, policy_version 1456623 (0.0008) [2023-12-27 02:00:02,870][105620] Updated weights for policy 1, policy_version 1458984 (0.0010) [2023-12-27 02:00:02,925][105620] Updated weights for policy 1, policy_version 1458994 (0.0010) [2023-12-27 02:00:02,984][105620] Updated weights for policy 1, policy_version 1459004 (0.0010) [2023-12-27 02:00:03,109][105692] Updated weights for policy 0, policy_version 1456633 (0.0005) [2023-12-27 02:00:03,155][105692] Updated weights for policy 0, policy_version 1456643 (0.0005) [2023-12-27 02:00:03,200][105692] Updated weights for policy 0, policy_version 1456653 (0.0005) [2023-12-27 02:00:03,573][105620] Updated weights for policy 1, policy_version 1459014 (0.0008) [2023-12-27 02:00:03,625][105620] Updated weights for policy 1, policy_version 1459024 (0.0005) [2023-12-27 02:00:03,680][105620] Updated weights for policy 1, policy_version 1459034 (0.0006) [2023-12-27 02:00:03,917][105692] Updated weights for policy 0, policy_version 1456663 (0.0006) [2023-12-27 02:00:03,976][105692] Updated weights for policy 0, policy_version 1456673 (0.0006) [2023-12-27 02:00:04,045][105692] Updated weights for policy 0, policy_version 1456683 (0.0006) [2023-12-27 02:00:04,395][105620] Updated weights for policy 1, policy_version 1459044 (0.0008) [2023-12-27 02:00:04,447][105620] Updated weights for policy 1, policy_version 1459054 (0.0005) [2023-12-27 02:00:04,492][105620] Updated weights for policy 1, policy_version 1459064 (0.0005) [2023-12-27 02:00:04,735][105692] Updated weights for policy 0, policy_version 1456693 (0.0007) [2023-12-27 02:00:04,783][105692] Updated weights for policy 0, policy_version 1456703 (0.0005) [2023-12-27 02:00:04,838][105692] Updated weights for policy 0, policy_version 1456713 (0.0005) [2023-12-27 02:00:05,039][105620] Updated weights for policy 1, policy_version 1459074 (0.0006) [2023-12-27 02:00:05,101][105620] Updated weights for policy 1, policy_version 1459084 (0.0011) [2023-12-27 02:00:05,160][105620] Updated weights for policy 1, policy_version 1459094 (0.0010) [2023-12-27 02:00:05,221][105620] Updated weights for policy 1, policy_version 1459104 (0.0010) [2023-12-27 02:00:05,467][105692] Updated weights for policy 0, policy_version 1456723 (0.0007) [2023-12-27 02:00:05,514][105692] Updated weights for policy 0, policy_version 1456733 (0.0010) [2023-12-27 02:00:05,565][105692] Updated weights for policy 0, policy_version 1456743 (0.0010) [2023-12-27 02:00:05,877][105620] Updated weights for policy 1, policy_version 1459114 (0.0005) [2023-12-27 02:00:05,934][105620] Updated weights for policy 1, policy_version 1459124 (0.0009) [2023-12-27 02:00:05,983][105620] Updated weights for policy 1, policy_version 1459134 (0.0010) [2023-12-27 02:00:06,062][104569] Fps is (10 sec: 20480.1, 60 sec: 20070.4, 300 sec: 19688.6). Total num frames: 746569728. Throughput: 0: 9890.5, 1: 10062.9. Samples: 746554068. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 02:00:06,062][104569] Avg episode reward: [(0, '8438.282'), (1, '9006.558')] [2023-12-27 02:00:06,319][105692] Updated weights for policy 0, policy_version 1456753 (0.0010) [2023-12-27 02:00:06,383][105692] Updated weights for policy 0, policy_version 1456763 (0.0010) [2023-12-27 02:00:06,442][105692] Updated weights for policy 0, policy_version 1456773 (0.0010) [2023-12-27 02:00:06,511][105692] Updated weights for policy 0, policy_version 1456783 (0.0005) [2023-12-27 02:00:06,699][105620] Updated weights for policy 1, policy_version 1459144 (0.0011) [2023-12-27 02:00:06,768][105620] Updated weights for policy 1, policy_version 1459154 (0.0011) [2023-12-27 02:00:06,836][105620] Updated weights for policy 1, policy_version 1459164 (0.0010) [2023-12-27 02:00:07,097][105692] Updated weights for policy 0, policy_version 1456793 (0.0008) [2023-12-27 02:00:07,156][105692] Updated weights for policy 0, policy_version 1456803 (0.0005) [2023-12-27 02:00:07,207][105692] Updated weights for policy 0, policy_version 1456813 (0.0005) [2023-12-27 02:00:07,581][105620] Updated weights for policy 1, policy_version 1459174 (0.0010) [2023-12-27 02:00:07,648][105620] Updated weights for policy 1, policy_version 1459184 (0.0011) [2023-12-27 02:00:07,697][105620] Updated weights for policy 1, policy_version 1459194 (0.0010) [2023-12-27 02:00:07,733][105692] Updated weights for policy 0, policy_version 1456823 (0.0005) [2023-12-27 02:00:07,788][105692] Updated weights for policy 0, policy_version 1456833 (0.0005) [2023-12-27 02:00:07,834][105692] Updated weights for policy 0, policy_version 1456843 (0.0005) [2023-12-27 02:00:08,444][105692] Updated weights for policy 0, policy_version 1456853 (0.0008) [2023-12-27 02:00:08,470][105620] Updated weights for policy 1, policy_version 1459204 (0.0011) [2023-12-27 02:00:08,504][105692] Updated weights for policy 0, policy_version 1456863 (0.0010) [2023-12-27 02:00:08,533][105620] Updated weights for policy 1, policy_version 1459214 (0.0011) [2023-12-27 02:00:08,565][105692] Updated weights for policy 0, policy_version 1456873 (0.0010) [2023-12-27 02:00:08,593][105620] Updated weights for policy 1, policy_version 1459224 (0.0011) [2023-12-27 02:00:09,263][105692] Updated weights for policy 0, policy_version 1456883 (0.0010) [2023-12-27 02:00:09,314][105692] Updated weights for policy 0, policy_version 1456893 (0.0007) [2023-12-27 02:00:09,315][105620] Updated weights for policy 1, policy_version 1459234 (0.0010) [2023-12-27 02:00:09,379][105692] Updated weights for policy 0, policy_version 1456903 (0.0010) [2023-12-27 02:00:09,382][105620] Updated weights for policy 1, policy_version 1459244 (0.0008) [2023-12-27 02:00:09,443][105620] Updated weights for policy 1, policy_version 1459254 (0.0008) [2023-12-27 02:00:09,496][105620] Updated weights for policy 1, policy_version 1459264 (0.0008) [2023-12-27 02:00:10,131][105692] Updated weights for policy 0, policy_version 1456913 (0.0013) [2023-12-27 02:00:10,201][105692] Updated weights for policy 0, policy_version 1456923 (0.0007) [2023-12-27 02:00:10,232][105620] Updated weights for policy 1, policy_version 1459274 (0.0009) [2023-12-27 02:00:10,267][105692] Updated weights for policy 0, policy_version 1456933 (0.0011) [2023-12-27 02:00:10,288][105620] Updated weights for policy 1, policy_version 1459284 (0.0008) [2023-12-27 02:00:10,331][105692] Updated weights for policy 0, policy_version 1456943 (0.0011) [2023-12-27 02:00:10,344][105620] Updated weights for policy 1, policy_version 1459294 (0.0006) [2023-12-27 02:00:10,993][105692] Updated weights for policy 0, policy_version 1456953 (0.0011) [2023-12-27 02:00:11,032][105620] Updated weights for policy 1, policy_version 1459304 (0.0010) [2023-12-27 02:00:11,060][105692] Updated weights for policy 0, policy_version 1456963 (0.0010) [2023-12-27 02:00:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 746659840. Throughput: 0: 9999.5, 1: 10076.4. Samples: 746674980. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 02:00:11,063][104569] Avg episode reward: [(0, '8252.436'), (1, '8986.638')] [2023-12-27 02:00:11,100][105620] Updated weights for policy 1, policy_version 1459314 (0.0011) [2023-12-27 02:00:11,120][105692] Updated weights for policy 0, policy_version 1456973 (0.0010) [2023-12-27 02:00:11,165][105620] Updated weights for policy 1, policy_version 1459324 (0.0011) [2023-12-27 02:00:11,900][105692] Updated weights for policy 0, policy_version 1456983 (0.0009) [2023-12-27 02:00:11,948][105620] Updated weights for policy 1, policy_version 1459334 (0.0011) [2023-12-27 02:00:11,958][105692] Updated weights for policy 0, policy_version 1456993 (0.0007) [2023-12-27 02:00:12,008][105620] Updated weights for policy 1, policy_version 1459344 (0.0011) [2023-12-27 02:00:12,016][105692] Updated weights for policy 0, policy_version 1457003 (0.0006) [2023-12-27 02:00:12,068][105620] Updated weights for policy 1, policy_version 1459354 (0.0011) [2023-12-27 02:00:12,779][105692] Updated weights for policy 0, policy_version 1457013 (0.0007) [2023-12-27 02:00:12,794][105620] Updated weights for policy 1, policy_version 1459364 (0.0010) [2023-12-27 02:00:12,830][105692] Updated weights for policy 0, policy_version 1457023 (0.0006) [2023-12-27 02:00:12,845][105620] Updated weights for policy 1, policy_version 1459374 (0.0007) [2023-12-27 02:00:12,877][105692] Updated weights for policy 0, policy_version 1457033 (0.0006) [2023-12-27 02:00:12,896][105620] Updated weights for policy 1, policy_version 1459384 (0.0007) [2023-12-27 02:00:13,544][105692] Updated weights for policy 0, policy_version 1457043 (0.0008) [2023-12-27 02:00:13,607][105692] Updated weights for policy 0, policy_version 1457053 (0.0010) [2023-12-27 02:00:13,617][105620] Updated weights for policy 1, policy_version 1459394 (0.0008) [2023-12-27 02:00:13,675][105620] Updated weights for policy 1, policy_version 1459404 (0.0008) [2023-12-27 02:00:13,677][105692] Updated weights for policy 0, policy_version 1457063 (0.0009) [2023-12-27 02:00:13,731][105620] Updated weights for policy 1, policy_version 1459414 (0.0011) [2023-12-27 02:00:13,791][105620] Updated weights for policy 1, policy_version 1459424 (0.0010) [2023-12-27 02:00:14,227][105692] Updated weights for policy 0, policy_version 1457073 (0.0006) [2023-12-27 02:00:14,284][105692] Updated weights for policy 0, policy_version 1457083 (0.0008) [2023-12-27 02:00:14,342][105692] Updated weights for policy 0, policy_version 1457093 (0.0009) [2023-12-27 02:00:14,405][105692] Updated weights for policy 0, policy_version 1457103 (0.0008) [2023-12-27 02:00:14,453][105620] Updated weights for policy 1, policy_version 1459434 (0.0008) [2023-12-27 02:00:14,511][105620] Updated weights for policy 1, policy_version 1459444 (0.0009) [2023-12-27 02:00:14,558][105620] Updated weights for policy 1, policy_version 1459454 (0.0008) [2023-12-27 02:00:15,137][105692] Updated weights for policy 0, policy_version 1457113 (0.0008) [2023-12-27 02:00:15,198][105692] Updated weights for policy 0, policy_version 1457123 (0.0008) [2023-12-27 02:00:15,256][105692] Updated weights for policy 0, policy_version 1457133 (0.0009) [2023-12-27 02:00:15,293][105620] Updated weights for policy 1, policy_version 1459464 (0.0006) [2023-12-27 02:00:15,357][105620] Updated weights for policy 1, policy_version 1459474 (0.0005) [2023-12-27 02:00:15,420][105620] Updated weights for policy 1, policy_version 1459484 (0.0008) [2023-12-27 02:00:15,941][105692] Updated weights for policy 0, policy_version 1457143 (0.0006) [2023-12-27 02:00:15,981][105620] Updated weights for policy 1, policy_version 1459494 (0.0008) [2023-12-27 02:00:16,006][105692] Updated weights for policy 0, policy_version 1457153 (0.0005) [2023-12-27 02:00:16,040][105620] Updated weights for policy 1, policy_version 1459504 (0.0010) [2023-12-27 02:00:16,059][105692] Updated weights for policy 0, policy_version 1457163 (0.0005) [2023-12-27 02:00:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19934.0, 300 sec: 19660.8). Total num frames: 746758144. Throughput: 0: 9910.1, 1: 10033.4. Samples: 746732692. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 02:00:16,062][104569] Avg episode reward: [(0, '8255.896'), (1, '9079.811')] [2023-12-27 02:00:16,087][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001457168_373088256.pth... [2023-12-27 02:00:16,090][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001455984_372785152.pth [2023-12-27 02:00:16,103][105620] Updated weights for policy 1, policy_version 1459514 (0.0011) [2023-12-27 02:00:16,135][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001459520_373686272.pth... [2023-12-27 02:00:16,138][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001458336_373383168.pth [2023-12-27 02:00:16,554][105692] Updated weights for policy 0, policy_version 1457173 (0.0005) [2023-12-27 02:00:16,620][105692] Updated weights for policy 0, policy_version 1457183 (0.0005) [2023-12-27 02:00:16,677][105692] Updated weights for policy 0, policy_version 1457193 (0.0007) [2023-12-27 02:00:16,814][105620] Updated weights for policy 1, policy_version 1459524 (0.0008) [2023-12-27 02:00:16,867][105620] Updated weights for policy 1, policy_version 1459534 (0.0007) [2023-12-27 02:00:16,916][105620] Updated weights for policy 1, policy_version 1459544 (0.0010) [2023-12-27 02:00:17,209][105692] Updated weights for policy 0, policy_version 1457203 (0.0006) [2023-12-27 02:00:17,258][105692] Updated weights for policy 0, policy_version 1457213 (0.0008) [2023-12-27 02:00:17,309][105692] Updated weights for policy 0, policy_version 1457223 (0.0008) [2023-12-27 02:00:17,611][105620] Updated weights for policy 1, policy_version 1459554 (0.0006) [2023-12-27 02:00:17,663][105620] Updated weights for policy 1, policy_version 1459564 (0.0005) [2023-12-27 02:00:17,716][105620] Updated weights for policy 1, policy_version 1459574 (0.0005) [2023-12-27 02:00:17,772][105620] Updated weights for policy 1, policy_version 1459584 (0.0005) [2023-12-27 02:00:18,104][105692] Updated weights for policy 0, policy_version 1457233 (0.0008) [2023-12-27 02:00:18,151][105692] Updated weights for policy 0, policy_version 1457243 (0.0008) [2023-12-27 02:00:18,203][105692] Updated weights for policy 0, policy_version 1457253 (0.0008) [2023-12-27 02:00:18,252][105692] Updated weights for policy 0, policy_version 1457263 (0.0005) [2023-12-27 02:00:18,422][105620] Updated weights for policy 1, policy_version 1459594 (0.0006) [2023-12-27 02:00:18,482][105620] Updated weights for policy 1, policy_version 1459604 (0.0006) [2023-12-27 02:00:18,547][105620] Updated weights for policy 1, policy_version 1459614 (0.0008) [2023-12-27 02:00:19,032][105692] Updated weights for policy 0, policy_version 1457273 (0.0005) [2023-12-27 02:00:19,089][105692] Updated weights for policy 0, policy_version 1457283 (0.0005) [2023-12-27 02:00:19,139][105692] Updated weights for policy 0, policy_version 1457293 (0.0006) [2023-12-27 02:00:19,168][105620] Updated weights for policy 1, policy_version 1459624 (0.0010) [2023-12-27 02:00:19,228][105620] Updated weights for policy 1, policy_version 1459634 (0.0010) [2023-12-27 02:00:19,296][105620] Updated weights for policy 1, policy_version 1459644 (0.0010) [2023-12-27 02:00:19,737][105692] Updated weights for policy 0, policy_version 1457303 (0.0006) [2023-12-27 02:00:19,787][105692] Updated weights for policy 0, policy_version 1457313 (0.0006) [2023-12-27 02:00:19,849][105692] Updated weights for policy 0, policy_version 1457323 (0.0007) [2023-12-27 02:00:20,124][105620] Updated weights for policy 1, policy_version 1459654 (0.0008) [2023-12-27 02:00:20,190][105620] Updated weights for policy 1, policy_version 1459664 (0.0010) [2023-12-27 02:00:20,245][105620] Updated weights for policy 1, policy_version 1459674 (0.0009) [2023-12-27 02:00:20,504][105692] Updated weights for policy 0, policy_version 1457333 (0.0008) [2023-12-27 02:00:20,565][105692] Updated weights for policy 0, policy_version 1457343 (0.0009) [2023-12-27 02:00:20,623][105692] Updated weights for policy 0, policy_version 1457353 (0.0009) [2023-12-27 02:00:20,998][105620] Updated weights for policy 1, policy_version 1459684 (0.0009) [2023-12-27 02:00:21,063][105620] Updated weights for policy 1, policy_version 1459694 (0.0010) [2023-12-27 02:00:21,062][104569] Fps is (10 sec: 20479.4, 60 sec: 20070.3, 300 sec: 19660.8). Total num frames: 746864640. Throughput: 0: 10011.1, 1: 10055.8. Samples: 746857348. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 02:00:21,063][104569] Avg episode reward: [(0, '8531.659'), (1, '9172.021')] [2023-12-27 02:00:21,132][105620] Updated weights for policy 1, policy_version 1459704 (0.0011) [2023-12-27 02:00:21,265][105692] Updated weights for policy 0, policy_version 1457363 (0.0007) [2023-12-27 02:00:21,325][105692] Updated weights for policy 0, policy_version 1457373 (0.0010) [2023-12-27 02:00:21,395][105692] Updated weights for policy 0, policy_version 1457383 (0.0007) [2023-12-27 02:00:21,891][105620] Updated weights for policy 1, policy_version 1459714 (0.0011) [2023-12-27 02:00:21,953][105620] Updated weights for policy 1, policy_version 1459724 (0.0009) [2023-12-27 02:00:22,011][105620] Updated weights for policy 1, policy_version 1459734 (0.0009) [2023-12-27 02:00:22,069][105620] Updated weights for policy 1, policy_version 1459744 (0.0009) [2023-12-27 02:00:22,206][105692] Updated weights for policy 0, policy_version 1457393 (0.0009) [2023-12-27 02:00:22,263][105692] Updated weights for policy 0, policy_version 1457403 (0.0009) [2023-12-27 02:00:22,327][105692] Updated weights for policy 0, policy_version 1457413 (0.0009) [2023-12-27 02:00:22,388][105692] Updated weights for policy 0, policy_version 1457423 (0.0009) [2023-12-27 02:00:22,831][105620] Updated weights for policy 1, policy_version 1459754 (0.0009) [2023-12-27 02:00:22,888][105620] Updated weights for policy 1, policy_version 1459764 (0.0007) [2023-12-27 02:00:22,937][105620] Updated weights for policy 1, policy_version 1459774 (0.0008) [2023-12-27 02:00:23,180][105692] Updated weights for policy 0, policy_version 1457433 (0.0008) [2023-12-27 02:00:23,244][105692] Updated weights for policy 0, policy_version 1457443 (0.0009) [2023-12-27 02:00:23,307][105692] Updated weights for policy 0, policy_version 1457453 (0.0009) [2023-12-27 02:00:23,734][105620] Updated weights for policy 1, policy_version 1459784 (0.0009) [2023-12-27 02:00:23,791][105620] Updated weights for policy 1, policy_version 1459794 (0.0009) [2023-12-27 02:00:23,849][105620] Updated weights for policy 1, policy_version 1459804 (0.0009) [2023-12-27 02:00:24,069][105692] Updated weights for policy 0, policy_version 1457463 (0.0009) [2023-12-27 02:00:24,129][105692] Updated weights for policy 0, policy_version 1457473 (0.0009) [2023-12-27 02:00:24,188][105692] Updated weights for policy 0, policy_version 1457483 (0.0009) [2023-12-27 02:00:24,578][105620] Updated weights for policy 1, policy_version 1459814 (0.0007) [2023-12-27 02:00:24,640][105620] Updated weights for policy 1, policy_version 1459824 (0.0005) [2023-12-27 02:00:24,694][105620] Updated weights for policy 1, policy_version 1459834 (0.0009) [2023-12-27 02:00:25,036][105692] Updated weights for policy 0, policy_version 1457493 (0.0008) [2023-12-27 02:00:25,091][105692] Updated weights for policy 0, policy_version 1457503 (0.0008) [2023-12-27 02:00:25,157][105692] Updated weights for policy 0, policy_version 1457513 (0.0010) [2023-12-27 02:00:25,231][105620] Updated weights for policy 1, policy_version 1459844 (0.0008) [2023-12-27 02:00:25,299][105620] Updated weights for policy 1, policy_version 1459854 (0.0005) [2023-12-27 02:00:25,357][105620] Updated weights for policy 1, policy_version 1459864 (0.0005) [2023-12-27 02:00:25,974][105620] Updated weights for policy 1, policy_version 1459874 (0.0006) [2023-12-27 02:00:25,978][105692] Updated weights for policy 0, policy_version 1457523 (0.0010) [2023-12-27 02:00:26,019][105620] Updated weights for policy 1, policy_version 1459884 (0.0009) [2023-12-27 02:00:26,029][105692] Updated weights for policy 0, policy_version 1457533 (0.0009) [2023-12-27 02:00:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19933.9, 300 sec: 19660.8). Total num frames: 746954752. Throughput: 0: 9969.6, 1: 9999.0. Samples: 746970780. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 02:00:26,063][104569] Avg episode reward: [(0, '8890.513'), (1, '9079.628')] [2023-12-27 02:00:26,067][105620] Updated weights for policy 1, policy_version 1459894 (0.0006) [2023-12-27 02:00:26,085][105692] Updated weights for policy 0, policy_version 1457543 (0.0008) [2023-12-27 02:00:26,116][105620] Updated weights for policy 1, policy_version 1459904 (0.0006) [2023-12-27 02:00:26,760][105620] Updated weights for policy 1, policy_version 1459914 (0.0005) [2023-12-27 02:00:26,815][105620] Updated weights for policy 1, policy_version 1459924 (0.0005) [2023-12-27 02:00:26,885][105620] Updated weights for policy 1, policy_version 1459934 (0.0005) [2023-12-27 02:00:26,893][105692] Updated weights for policy 0, policy_version 1457553 (0.0007) [2023-12-27 02:00:26,957][105692] Updated weights for policy 0, policy_version 1457563 (0.0009) [2023-12-27 02:00:27,015][105692] Updated weights for policy 0, policy_version 1457573 (0.0009) [2023-12-27 02:00:27,061][105692] Updated weights for policy 0, policy_version 1457583 (0.0009) [2023-12-27 02:00:27,510][105620] Updated weights for policy 1, policy_version 1459944 (0.0008) [2023-12-27 02:00:27,556][105620] Updated weights for policy 1, policy_version 1459954 (0.0008) [2023-12-27 02:00:27,609][105620] Updated weights for policy 1, policy_version 1459964 (0.0009) [2023-12-27 02:00:27,802][105692] Updated weights for policy 0, policy_version 1457593 (0.0009) [2023-12-27 02:00:27,848][105692] Updated weights for policy 0, policy_version 1457603 (0.0008) [2023-12-27 02:00:27,906][105692] Updated weights for policy 0, policy_version 1457613 (0.0009) [2023-12-27 02:00:28,323][105620] Updated weights for policy 1, policy_version 1459974 (0.0009) [2023-12-27 02:00:28,383][105620] Updated weights for policy 1, policy_version 1459984 (0.0008) [2023-12-27 02:00:28,440][105620] Updated weights for policy 1, policy_version 1459994 (0.0010) [2023-12-27 02:00:28,640][105692] Updated weights for policy 0, policy_version 1457623 (0.0007) [2023-12-27 02:00:28,702][105692] Updated weights for policy 0, policy_version 1457633 (0.0008) [2023-12-27 02:00:28,770][105692] Updated weights for policy 0, policy_version 1457643 (0.0007) [2023-12-27 02:00:29,108][105620] Updated weights for policy 1, policy_version 1460004 (0.0008) [2023-12-27 02:00:29,172][105620] Updated weights for policy 1, policy_version 1460014 (0.0006) [2023-12-27 02:00:29,234][105620] Updated weights for policy 1, policy_version 1460024 (0.0007) [2023-12-27 02:00:29,439][105692] Updated weights for policy 0, policy_version 1457653 (0.0007) [2023-12-27 02:00:29,498][105692] Updated weights for policy 0, policy_version 1457663 (0.0009) [2023-12-27 02:00:29,556][105692] Updated weights for policy 0, policy_version 1457673 (0.0009) [2023-12-27 02:00:29,883][105620] Updated weights for policy 1, policy_version 1460034 (0.0007) [2023-12-27 02:00:29,949][105620] Updated weights for policy 1, policy_version 1460044 (0.0008) [2023-12-27 02:00:30,006][105620] Updated weights for policy 1, policy_version 1460054 (0.0009) [2023-12-27 02:00:30,064][105620] Updated weights for policy 1, policy_version 1460064 (0.0010) [2023-12-27 02:00:30,264][105692] Updated weights for policy 0, policy_version 1457683 (0.0008) [2023-12-27 02:00:30,331][105692] Updated weights for policy 0, policy_version 1457693 (0.0005) [2023-12-27 02:00:30,396][105692] Updated weights for policy 0, policy_version 1457703 (0.0007) [2023-12-27 02:00:30,843][105620] Updated weights for policy 1, policy_version 1460074 (0.0008) [2023-12-27 02:00:30,903][105620] Updated weights for policy 1, policy_version 1460084 (0.0009) [2023-12-27 02:00:30,961][105620] Updated weights for policy 1, policy_version 1460094 (0.0009) [2023-12-27 02:00:30,975][105692] Updated weights for policy 0, policy_version 1457713 (0.0006) [2023-12-27 02:00:31,027][105692] Updated weights for policy 0, policy_version 1457723 (0.0008) [2023-12-27 02:00:31,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19933.8, 300 sec: 19688.6). Total num frames: 747061248. Throughput: 0: 9968.0, 1: 9975.9. Samples: 747029620. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 02:00:31,063][104569] Avg episode reward: [(0, '8431.315'), (1, '8985.697')] [2023-12-27 02:00:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001460096_373833728.pth... [2023-12-27 02:00:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001458944_373538816.pth [2023-12-27 02:00:31,093][105692] Updated weights for policy 0, policy_version 1457733 (0.0007) [2023-12-27 02:00:31,153][105692] Updated weights for policy 0, policy_version 1457743 (0.0007) [2023-12-27 02:00:31,156][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001457744_373235712.pth... [2023-12-27 02:00:31,161][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001456560_372932608.pth [2023-12-27 02:00:31,742][105620] Updated weights for policy 1, policy_version 1460104 (0.0009) [2023-12-27 02:00:31,801][105620] Updated weights for policy 1, policy_version 1460114 (0.0005) [2023-12-27 02:00:31,847][105620] Updated weights for policy 1, policy_version 1460124 (0.0005) [2023-12-27 02:00:31,935][105692] Updated weights for policy 0, policy_version 1457753 (0.0009) [2023-12-27 02:00:31,996][105692] Updated weights for policy 0, policy_version 1457763 (0.0010) [2023-12-27 02:00:32,059][105692] Updated weights for policy 0, policy_version 1457773 (0.0009) [2023-12-27 02:00:32,544][105620] Updated weights for policy 1, policy_version 1460134 (0.0008) [2023-12-27 02:00:32,590][105620] Updated weights for policy 1, policy_version 1460144 (0.0008) [2023-12-27 02:00:32,644][105620] Updated weights for policy 1, policy_version 1460154 (0.0009) [2023-12-27 02:00:32,841][105692] Updated weights for policy 0, policy_version 1457783 (0.0009) [2023-12-27 02:00:32,891][105692] Updated weights for policy 0, policy_version 1457793 (0.0009) [2023-12-27 02:00:32,938][105692] Updated weights for policy 0, policy_version 1457803 (0.0009) [2023-12-27 02:00:33,455][105620] Updated weights for policy 1, policy_version 1460164 (0.0009) [2023-12-27 02:00:33,511][105620] Updated weights for policy 1, policy_version 1460174 (0.0007) [2023-12-27 02:00:33,566][105620] Updated weights for policy 1, policy_version 1460184 (0.0005) [2023-12-27 02:00:33,629][105692] Updated weights for policy 0, policy_version 1457814 (0.0009) [2023-12-27 02:00:33,681][105692] Updated weights for policy 0, policy_version 1457825 (0.0010) [2023-12-27 02:00:33,735][105692] Updated weights for policy 0, policy_version 1457836 (0.0010) [2023-12-27 02:00:34,104][105620] Updated weights for policy 1, policy_version 1460194 (0.0005) [2023-12-27 02:00:34,177][105620] Updated weights for policy 1, policy_version 1460204 (0.0007) [2023-12-27 02:00:34,231][105620] Updated weights for policy 1, policy_version 1460214 (0.0007) [2023-12-27 02:00:34,293][105620] Updated weights for policy 1, policy_version 1460224 (0.0009) [2023-12-27 02:00:34,617][105692] Updated weights for policy 0, policy_version 1457846 (0.0010) [2023-12-27 02:00:34,673][105692] Updated weights for policy 0, policy_version 1457856 (0.0009) [2023-12-27 02:00:34,731][105692] Updated weights for policy 0, policy_version 1457866 (0.0009) [2023-12-27 02:00:34,936][105620] Updated weights for policy 1, policy_version 1460235 (0.0009) [2023-12-27 02:00:34,984][105620] Updated weights for policy 1, policy_version 1460245 (0.0008) [2023-12-27 02:00:35,036][105620] Updated weights for policy 1, policy_version 1460255 (0.0008) [2023-12-27 02:00:35,458][105692] Updated weights for policy 0, policy_version 1457876 (0.0009) [2023-12-27 02:00:35,514][105692] Updated weights for policy 0, policy_version 1457886 (0.0008) [2023-12-27 02:00:35,583][105692] Updated weights for policy 0, policy_version 1457896 (0.0007) [2023-12-27 02:00:35,750][105620] Updated weights for policy 1, policy_version 1460265 (0.0008) [2023-12-27 02:00:35,802][105620] Updated weights for policy 1, policy_version 1460275 (0.0009) [2023-12-27 02:00:35,855][105620] Updated weights for policy 1, policy_version 1460286 (0.0009) [2023-12-27 02:00:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19933.9, 300 sec: 19688.6). Total num frames: 747159552. Throughput: 0: 9807.9, 1: 9866.9. Samples: 747146920. Policy #0 lag: (min: 20.0, avg: 27.4, max: 52.0) [2023-12-27 02:00:36,062][104569] Avg episode reward: [(0, '8707.340'), (1, '8983.925')] [2023-12-27 02:00:36,208][105692] Updated weights for policy 0, policy_version 1457906 (0.0010) [2023-12-27 02:00:36,271][105692] Updated weights for policy 0, policy_version 1457916 (0.0011) [2023-12-27 02:00:36,334][105692] Updated weights for policy 0, policy_version 1457926 (0.0011) [2023-12-27 02:00:36,398][105692] Updated weights for policy 0, policy_version 1457936 (0.0011) [2023-12-27 02:00:36,739][105620] Updated weights for policy 1, policy_version 1460296 (0.0010) [2023-12-27 02:00:36,792][105620] Updated weights for policy 1, policy_version 1460307 (0.0010) [2023-12-27 02:00:36,849][105620] Updated weights for policy 1, policy_version 1460317 (0.0010) [2023-12-27 02:00:37,014][105692] Updated weights for policy 0, policy_version 1457946 (0.0007) [2023-12-27 02:00:37,073][105692] Updated weights for policy 0, policy_version 1457956 (0.0011) [2023-12-27 02:00:37,131][105692] Updated weights for policy 0, policy_version 1457966 (0.0010) [2023-12-27 02:00:37,581][105620] Updated weights for policy 1, policy_version 1460327 (0.0008) [2023-12-27 02:00:37,637][105620] Updated weights for policy 1, policy_version 1460337 (0.0008) [2023-12-27 02:00:37,700][105620] Updated weights for policy 1, policy_version 1460347 (0.0008) [2023-12-27 02:00:37,838][105692] Updated weights for policy 0, policy_version 1457976 (0.0011) [2023-12-27 02:00:37,890][105692] Updated weights for policy 0, policy_version 1457986 (0.0011) [2023-12-27 02:00:37,940][105692] Updated weights for policy 0, policy_version 1457996 (0.0011) [2023-12-27 02:00:38,454][105620] Updated weights for policy 1, policy_version 1460357 (0.0010) [2023-12-27 02:00:38,510][105620] Updated weights for policy 1, policy_version 1460367 (0.0011) [2023-12-27 02:00:38,566][105620] Updated weights for policy 1, policy_version 1460377 (0.0005) [2023-12-27 02:00:38,724][105692] Updated weights for policy 0, policy_version 1458006 (0.0011) [2023-12-27 02:00:38,787][105692] Updated weights for policy 0, policy_version 1458016 (0.0011) [2023-12-27 02:00:38,853][105692] Updated weights for policy 0, policy_version 1458026 (0.0011) [2023-12-27 02:00:39,218][105620] Updated weights for policy 1, policy_version 1460387 (0.0008) [2023-12-27 02:00:39,283][105620] Updated weights for policy 1, policy_version 1460397 (0.0008) [2023-12-27 02:00:39,348][105620] Updated weights for policy 1, policy_version 1460407 (0.0007) [2023-12-27 02:00:39,581][105692] Updated weights for policy 0, policy_version 1458036 (0.0011) [2023-12-27 02:00:39,647][105692] Updated weights for policy 0, policy_version 1458046 (0.0011) [2023-12-27 02:00:39,703][105692] Updated weights for policy 0, policy_version 1458056 (0.0010) [2023-12-27 02:00:40,114][105620] Updated weights for policy 1, policy_version 1460417 (0.0009) [2023-12-27 02:00:40,166][105620] Updated weights for policy 1, policy_version 1460427 (0.0008) [2023-12-27 02:00:40,217][105620] Updated weights for policy 1, policy_version 1460437 (0.0008) [2023-12-27 02:00:40,271][105620] Updated weights for policy 1, policy_version 1460447 (0.0008) [2023-12-27 02:00:40,486][105692] Updated weights for policy 0, policy_version 1458066 (0.0010) [2023-12-27 02:00:40,555][105692] Updated weights for policy 0, policy_version 1458076 (0.0009) [2023-12-27 02:00:40,604][105692] Updated weights for policy 0, policy_version 1458086 (0.0009) [2023-12-27 02:00:40,653][105692] Updated weights for policy 0, policy_version 1458096 (0.0009) [2023-12-27 02:00:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 747249664. Throughput: 0: 9840.8, 1: 9810.9. Samples: 747261340. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:00:41,063][104569] Avg episode reward: [(0, '8160.600'), (1, '9169.496')] [2023-12-27 02:00:41,085][105620] Updated weights for policy 1, policy_version 1460457 (0.0009) [2023-12-27 02:00:41,154][105620] Updated weights for policy 1, policy_version 1460467 (0.0009) [2023-12-27 02:00:41,216][105620] Updated weights for policy 1, policy_version 1460477 (0.0009) [2023-12-27 02:00:41,362][105692] Updated weights for policy 0, policy_version 1458106 (0.0009) [2023-12-27 02:00:41,431][105692] Updated weights for policy 0, policy_version 1458116 (0.0007) [2023-12-27 02:00:41,494][105692] Updated weights for policy 0, policy_version 1458126 (0.0006) [2023-12-27 02:00:42,002][105620] Updated weights for policy 1, policy_version 1460487 (0.0009) [2023-12-27 02:00:42,070][105620] Updated weights for policy 1, policy_version 1460497 (0.0009) [2023-12-27 02:00:42,131][105620] Updated weights for policy 1, policy_version 1460507 (0.0009) [2023-12-27 02:00:42,189][105692] Updated weights for policy 0, policy_version 1458136 (0.0008) [2023-12-27 02:00:42,245][105692] Updated weights for policy 0, policy_version 1458146 (0.0009) [2023-12-27 02:00:42,307][105692] Updated weights for policy 0, policy_version 1458156 (0.0009) [2023-12-27 02:00:42,741][105620] Updated weights for policy 1, policy_version 1460517 (0.0006) [2023-12-27 02:00:42,800][105620] Updated weights for policy 1, policy_version 1460527 (0.0007) [2023-12-27 02:00:42,862][105620] Updated weights for policy 1, policy_version 1460537 (0.0008) [2023-12-27 02:00:43,106][105692] Updated weights for policy 0, policy_version 1458166 (0.0009) [2023-12-27 02:00:43,158][105692] Updated weights for policy 0, policy_version 1458176 (0.0009) [2023-12-27 02:00:43,206][105692] Updated weights for policy 0, policy_version 1458186 (0.0009) [2023-12-27 02:00:43,606][105620] Updated weights for policy 1, policy_version 1460547 (0.0007) [2023-12-27 02:00:43,659][105620] Updated weights for policy 1, policy_version 1460557 (0.0005) [2023-12-27 02:00:43,710][105620] Updated weights for policy 1, policy_version 1460567 (0.0005) [2023-12-27 02:00:43,859][105692] Updated weights for policy 0, policy_version 1458196 (0.0005) [2023-12-27 02:00:43,921][105692] Updated weights for policy 0, policy_version 1458206 (0.0006) [2023-12-27 02:00:43,973][105692] Updated weights for policy 0, policy_version 1458217 (0.0009) [2023-12-27 02:00:44,315][105620] Updated weights for policy 1, policy_version 1460577 (0.0005) [2023-12-27 02:00:44,363][105620] Updated weights for policy 1, policy_version 1460587 (0.0009) [2023-12-27 02:00:44,417][105620] Updated weights for policy 1, policy_version 1460597 (0.0010) [2023-12-27 02:00:44,471][105620] Updated weights for policy 1, policy_version 1460608 (0.0010) [2023-12-27 02:00:44,644][105692] Updated weights for policy 0, policy_version 1458227 (0.0011) [2023-12-27 02:00:44,695][105692] Updated weights for policy 0, policy_version 1458237 (0.0009) [2023-12-27 02:00:44,759][105692] Updated weights for policy 0, policy_version 1458247 (0.0007) [2023-12-27 02:00:45,314][105620] Updated weights for policy 1, policy_version 1460618 (0.0009) [2023-12-27 02:00:45,386][105620] Updated weights for policy 1, policy_version 1460628 (0.0009) [2023-12-27 02:00:45,446][105620] Updated weights for policy 1, policy_version 1460638 (0.0009) [2023-12-27 02:00:45,503][105692] Updated weights for policy 0, policy_version 1458257 (0.0009) [2023-12-27 02:00:45,570][105692] Updated weights for policy 0, policy_version 1458267 (0.0006) [2023-12-27 02:00:45,637][105692] Updated weights for policy 0, policy_version 1458277 (0.0006) [2023-12-27 02:00:45,709][105692] Updated weights for policy 0, policy_version 1458287 (0.0005) [2023-12-27 02:00:46,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 747347968. Throughput: 0: 9807.6, 1: 9802.2. Samples: 747318892. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:00:46,063][104569] Avg episode reward: [(0, '8070.284'), (1, '8990.043')] [2023-12-27 02:00:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001458288_373374976.pth... [2023-12-27 02:00:46,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001460640_373972992.pth... [2023-12-27 02:00:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001457168_373088256.pth [2023-12-27 02:00:46,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001459520_373686272.pth [2023-12-27 02:00:46,218][105692] Updated weights for policy 0, policy_version 1458297 (0.0005) [2023-12-27 02:00:46,258][105620] Updated weights for policy 1, policy_version 1460648 (0.0006) [2023-12-27 02:00:46,279][105692] Updated weights for policy 0, policy_version 1458307 (0.0008) [2023-12-27 02:00:46,319][105620] Updated weights for policy 1, policy_version 1460658 (0.0009) [2023-12-27 02:00:46,337][105692] Updated weights for policy 0, policy_version 1458317 (0.0010) [2023-12-27 02:00:46,372][105620] Updated weights for policy 1, policy_version 1460668 (0.0006) [2023-12-27 02:00:47,019][105620] Updated weights for policy 1, policy_version 1460678 (0.0008) [2023-12-27 02:00:47,055][105692] Updated weights for policy 0, policy_version 1458327 (0.0010) [2023-12-27 02:00:47,073][105620] Updated weights for policy 1, policy_version 1460688 (0.0005) [2023-12-27 02:00:47,106][105692] Updated weights for policy 0, policy_version 1458337 (0.0010) [2023-12-27 02:00:47,125][105620] Updated weights for policy 1, policy_version 1460698 (0.0007) [2023-12-27 02:00:47,161][105692] Updated weights for policy 0, policy_version 1458347 (0.0010) [2023-12-27 02:00:47,849][105692] Updated weights for policy 0, policy_version 1458357 (0.0008) [2023-12-27 02:00:47,911][105692] Updated weights for policy 0, policy_version 1458367 (0.0005) [2023-12-27 02:00:47,940][105620] Updated weights for policy 1, policy_version 1460708 (0.0007) [2023-12-27 02:00:47,975][105692] Updated weights for policy 0, policy_version 1458377 (0.0006) [2023-12-27 02:00:47,996][105620] Updated weights for policy 1, policy_version 1460718 (0.0008) [2023-12-27 02:00:48,043][105620] Updated weights for policy 1, policy_version 1460728 (0.0008) [2023-12-27 02:00:48,550][105692] Updated weights for policy 0, policy_version 1458387 (0.0007) [2023-12-27 02:00:48,599][105692] Updated weights for policy 0, policy_version 1458397 (0.0010) [2023-12-27 02:00:48,612][105585] KL-divergence is very high: 178.9495 [2023-12-27 02:00:48,648][105692] Updated weights for policy 0, policy_version 1458407 (0.0010) [2023-12-27 02:00:48,653][105585] KL-divergence is very high: 223.4808 [2023-12-27 02:00:48,875][105620] Updated weights for policy 1, policy_version 1460738 (0.0007) [2023-12-27 02:00:48,945][105620] Updated weights for policy 1, policy_version 1460748 (0.0008) [2023-12-27 02:00:49,008][105620] Updated weights for policy 1, policy_version 1460758 (0.0008) [2023-12-27 02:00:49,076][105620] Updated weights for policy 1, policy_version 1460768 (0.0007) [2023-12-27 02:00:49,388][105692] Updated weights for policy 0, policy_version 1458417 (0.0010) [2023-12-27 02:00:49,449][105692] Updated weights for policy 0, policy_version 1458427 (0.0006) [2023-12-27 02:00:49,510][105692] Updated weights for policy 0, policy_version 1458437 (0.0007) [2023-12-27 02:00:49,557][105692] Updated weights for policy 0, policy_version 1458447 (0.0006) [2023-12-27 02:00:49,878][105620] Updated weights for policy 1, policy_version 1460778 (0.0009) [2023-12-27 02:00:49,940][105620] Updated weights for policy 1, policy_version 1460788 (0.0009) [2023-12-27 02:00:50,010][105620] Updated weights for policy 1, policy_version 1460798 (0.0006) [2023-12-27 02:00:50,162][105692] Updated weights for policy 0, policy_version 1458457 (0.0006) [2023-12-27 02:00:50,231][105692] Updated weights for policy 0, policy_version 1458467 (0.0007) [2023-12-27 02:00:50,283][105692] Updated weights for policy 0, policy_version 1458477 (0.0011) [2023-12-27 02:00:50,645][105620] Updated weights for policy 1, policy_version 1460808 (0.0009) [2023-12-27 02:00:50,695][105620] Updated weights for policy 1, policy_version 1460818 (0.0008) [2023-12-27 02:00:50,752][105620] Updated weights for policy 1, policy_version 1460828 (0.0008) [2023-12-27 02:00:50,925][105692] Updated weights for policy 0, policy_version 1458487 (0.0007) [2023-12-27 02:00:50,980][105692] Updated weights for policy 0, policy_version 1458497 (0.0006) [2023-12-27 02:00:51,047][105692] Updated weights for policy 0, policy_version 1458507 (0.0009) [2023-12-27 02:00:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 747446272. Throughput: 0: 9902.7, 1: 9691.9. Samples: 747435824. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:00:51,062][104569] Avg episode reward: [(0, '8435.515'), (1, '8718.136')] [2023-12-27 02:00:51,558][105620] Updated weights for policy 1, policy_version 1460838 (0.0008) [2023-12-27 02:00:51,625][105620] Updated weights for policy 1, policy_version 1460848 (0.0007) [2023-12-27 02:00:51,699][105620] Updated weights for policy 1, policy_version 1460858 (0.0009) [2023-12-27 02:00:51,729][105692] Updated weights for policy 0, policy_version 1458517 (0.0007) [2023-12-27 02:00:51,793][105692] Updated weights for policy 0, policy_version 1458527 (0.0006) [2023-12-27 02:00:51,854][105692] Updated weights for policy 0, policy_version 1458537 (0.0006) [2023-12-27 02:00:52,504][105692] Updated weights for policy 0, policy_version 1458547 (0.0008) [2023-12-27 02:00:52,511][105620] Updated weights for policy 1, policy_version 1460868 (0.0009) [2023-12-27 02:00:52,560][105692] Updated weights for policy 0, policy_version 1458557 (0.0007) [2023-12-27 02:00:52,565][105620] Updated weights for policy 1, policy_version 1460878 (0.0007) [2023-12-27 02:00:52,620][105692] Updated weights for policy 0, policy_version 1458567 (0.0007) [2023-12-27 02:00:52,626][105620] Updated weights for policy 1, policy_version 1460888 (0.0008) [2023-12-27 02:00:53,325][105692] Updated weights for policy 0, policy_version 1458577 (0.0006) [2023-12-27 02:00:53,391][105692] Updated weights for policy 0, policy_version 1458587 (0.0009) [2023-12-27 02:00:53,403][105620] Updated weights for policy 1, policy_version 1460898 (0.0008) [2023-12-27 02:00:53,451][105692] Updated weights for policy 0, policy_version 1458597 (0.0011) [2023-12-27 02:00:53,461][105620] Updated weights for policy 1, policy_version 1460908 (0.0006) [2023-12-27 02:00:53,503][105692] Updated weights for policy 0, policy_version 1458607 (0.0010) [2023-12-27 02:00:53,517][105620] Updated weights for policy 1, policy_version 1460918 (0.0005) [2023-12-27 02:00:53,581][105620] Updated weights for policy 1, policy_version 1460928 (0.0008) [2023-12-27 02:00:54,253][105692] Updated weights for policy 0, policy_version 1458617 (0.0009) [2023-12-27 02:00:54,312][105692] Updated weights for policy 0, policy_version 1458627 (0.0008) [2023-12-27 02:00:54,364][105620] Updated weights for policy 1, policy_version 1460938 (0.0009) [2023-12-27 02:00:54,380][105692] Updated weights for policy 0, policy_version 1458637 (0.0008) [2023-12-27 02:00:54,430][105620] Updated weights for policy 1, policy_version 1460948 (0.0009) [2023-12-27 02:00:54,497][105620] Updated weights for policy 1, policy_version 1460958 (0.0008) [2023-12-27 02:00:55,153][105692] Updated weights for policy 0, policy_version 1458647 (0.0008) [2023-12-27 02:00:55,220][105692] Updated weights for policy 0, policy_version 1458657 (0.0008) [2023-12-27 02:00:55,276][105620] Updated weights for policy 1, policy_version 1460968 (0.0010) [2023-12-27 02:00:55,279][105692] Updated weights for policy 0, policy_version 1458667 (0.0008) [2023-12-27 02:00:55,340][105620] Updated weights for policy 1, policy_version 1460978 (0.0011) [2023-12-27 02:00:55,394][105620] Updated weights for policy 1, policy_version 1460988 (0.0011) [2023-12-27 02:00:55,951][105692] Updated weights for policy 0, policy_version 1458677 (0.0008) [2023-12-27 02:00:55,999][105692] Updated weights for policy 0, policy_version 1458687 (0.0008) [2023-12-27 02:00:56,048][105692] Updated weights for policy 0, policy_version 1458697 (0.0008) [2023-12-27 02:00:56,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 747536384. Throughput: 0: 9835.0, 1: 9615.6. Samples: 747550256. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:00:56,062][104569] Avg episode reward: [(0, '7885.577'), (1, '8538.379')] [2023-12-27 02:00:56,105][105620] Updated weights for policy 1, policy_version 1460998 (0.0011) [2023-12-27 02:00:56,163][105620] Updated weights for policy 1, policy_version 1461008 (0.0010) [2023-12-27 02:00:56,215][105620] Updated weights for policy 1, policy_version 1461018 (0.0010) [2023-12-27 02:00:56,812][105692] Updated weights for policy 0, policy_version 1458707 (0.0008) [2023-12-27 02:00:56,874][105692] Updated weights for policy 0, policy_version 1458717 (0.0008) [2023-12-27 02:00:56,937][105692] Updated weights for policy 0, policy_version 1458727 (0.0008) [2023-12-27 02:00:56,979][105620] Updated weights for policy 1, policy_version 1461028 (0.0011) [2023-12-27 02:00:57,035][105620] Updated weights for policy 1, policy_version 1461038 (0.0009) [2023-12-27 02:00:57,087][105620] Updated weights for policy 1, policy_version 1461048 (0.0008) [2023-12-27 02:00:57,557][105692] Updated weights for policy 0, policy_version 1458737 (0.0008) [2023-12-27 02:00:57,623][105692] Updated weights for policy 0, policy_version 1458747 (0.0007) [2023-12-27 02:00:57,681][105692] Updated weights for policy 0, policy_version 1458757 (0.0008) [2023-12-27 02:00:57,740][105692] Updated weights for policy 0, policy_version 1458767 (0.0008) [2023-12-27 02:00:57,824][105620] Updated weights for policy 1, policy_version 1461058 (0.0010) [2023-12-27 02:00:57,875][105620] Updated weights for policy 1, policy_version 1461068 (0.0010) [2023-12-27 02:00:57,925][105620] Updated weights for policy 1, policy_version 1461078 (0.0010) [2023-12-27 02:00:57,972][105620] Updated weights for policy 1, policy_version 1461088 (0.0010) [2023-12-27 02:00:58,450][105692] Updated weights for policy 0, policy_version 1458777 (0.0008) [2023-12-27 02:00:58,507][105692] Updated weights for policy 0, policy_version 1458787 (0.0008) [2023-12-27 02:00:58,557][105692] Updated weights for policy 0, policy_version 1458797 (0.0008) [2023-12-27 02:00:58,829][105620] Updated weights for policy 1, policy_version 1461098 (0.0009) [2023-12-27 02:00:58,902][105620] Updated weights for policy 1, policy_version 1461108 (0.0010) [2023-12-27 02:00:58,967][105620] Updated weights for policy 1, policy_version 1461118 (0.0010) [2023-12-27 02:00:59,373][105692] Updated weights for policy 0, policy_version 1458808 (0.0009) [2023-12-27 02:00:59,429][105692] Updated weights for policy 0, policy_version 1458818 (0.0011) [2023-12-27 02:00:59,485][105692] Updated weights for policy 0, policy_version 1458828 (0.0010) [2023-12-27 02:00:59,704][105620] Updated weights for policy 1, policy_version 1461128 (0.0011) [2023-12-27 02:00:59,763][105620] Updated weights for policy 1, policy_version 1461138 (0.0010) [2023-12-27 02:00:59,820][105620] Updated weights for policy 1, policy_version 1461148 (0.0009) [2023-12-27 02:01:00,194][105692] Updated weights for policy 0, policy_version 1458838 (0.0010) [2023-12-27 02:01:00,258][105692] Updated weights for policy 0, policy_version 1458848 (0.0006) [2023-12-27 02:01:00,327][105692] Updated weights for policy 0, policy_version 1458858 (0.0008) [2023-12-27 02:01:00,498][105620] Updated weights for policy 1, policy_version 1461158 (0.0009) [2023-12-27 02:01:00,560][105620] Updated weights for policy 1, policy_version 1461168 (0.0011) [2023-12-27 02:01:00,624][105620] Updated weights for policy 1, policy_version 1461178 (0.0008) [2023-12-27 02:01:00,908][105692] Updated weights for policy 0, policy_version 1458868 (0.0007) [2023-12-27 02:01:00,965][105692] Updated weights for policy 0, policy_version 1458878 (0.0005) [2023-12-27 02:01:01,023][105692] Updated weights for policy 0, policy_version 1458888 (0.0007) [2023-12-27 02:01:01,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 747634688. Throughput: 0: 9843.9, 1: 9593.9. Samples: 747607396. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:01:01,063][104569] Avg episode reward: [(0, '7799.969'), (1, '8539.921')] [2023-12-27 02:01:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001461184_374112256.pth... [2023-12-27 02:01:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001460096_373833728.pth [2023-12-27 02:01:01,075][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001458896_373530624.pth... [2023-12-27 02:01:01,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001457744_373235712.pth [2023-12-27 02:01:01,380][105620] Updated weights for policy 1, policy_version 1461188 (0.0010) [2023-12-27 02:01:01,443][105620] Updated weights for policy 1, policy_version 1461198 (0.0008) [2023-12-27 02:01:01,491][105620] Updated weights for policy 1, policy_version 1461208 (0.0008) [2023-12-27 02:01:01,674][105692] Updated weights for policy 0, policy_version 1458898 (0.0009) [2023-12-27 02:01:01,734][105692] Updated weights for policy 0, policy_version 1458908 (0.0010) [2023-12-27 02:01:01,795][105692] Updated weights for policy 0, policy_version 1458918 (0.0011) [2023-12-27 02:01:01,849][105692] Updated weights for policy 0, policy_version 1458928 (0.0010) [2023-12-27 02:01:02,295][105620] Updated weights for policy 1, policy_version 1461218 (0.0008) [2023-12-27 02:01:02,370][105620] Updated weights for policy 1, policy_version 1461228 (0.0007) [2023-12-27 02:01:02,435][105620] Updated weights for policy 1, policy_version 1461238 (0.0006) [2023-12-27 02:01:02,500][105620] Updated weights for policy 1, policy_version 1461248 (0.0006) [2023-12-27 02:01:02,614][105692] Updated weights for policy 0, policy_version 1458938 (0.0011) [2023-12-27 02:01:02,674][105692] Updated weights for policy 0, policy_version 1458948 (0.0011) [2023-12-27 02:01:02,733][105692] Updated weights for policy 0, policy_version 1458958 (0.0011) [2023-12-27 02:01:03,216][105620] Updated weights for policy 1, policy_version 1461258 (0.0011) [2023-12-27 02:01:03,268][105620] Updated weights for policy 1, policy_version 1461268 (0.0011) [2023-12-27 02:01:03,325][105620] Updated weights for policy 1, policy_version 1461278 (0.0011) [2023-12-27 02:01:03,499][105692] Updated weights for policy 0, policy_version 1458968 (0.0009) [2023-12-27 02:01:03,549][105692] Updated weights for policy 0, policy_version 1458978 (0.0008) [2023-12-27 02:01:03,605][105692] Updated weights for policy 0, policy_version 1458988 (0.0008) [2023-12-27 02:01:04,089][105620] Updated weights for policy 1, policy_version 1461288 (0.0010) [2023-12-27 02:01:04,155][105620] Updated weights for policy 1, policy_version 1461298 (0.0008) [2023-12-27 02:01:04,224][105620] Updated weights for policy 1, policy_version 1461308 (0.0010) [2023-12-27 02:01:04,287][105692] Updated weights for policy 0, policy_version 1458998 (0.0010) [2023-12-27 02:01:04,351][105692] Updated weights for policy 0, policy_version 1459008 (0.0010) [2023-12-27 02:01:04,417][105692] Updated weights for policy 0, policy_version 1459018 (0.0007) [2023-12-27 02:01:04,969][105620] Updated weights for policy 1, policy_version 1461318 (0.0007) [2023-12-27 02:01:05,028][105620] Updated weights for policy 1, policy_version 1461328 (0.0007) [2023-12-27 02:01:05,088][105620] Updated weights for policy 1, policy_version 1461338 (0.0011) [2023-12-27 02:01:05,145][105692] Updated weights for policy 0, policy_version 1459028 (0.0011) [2023-12-27 02:01:05,200][105692] Updated weights for policy 0, policy_version 1459038 (0.0011) [2023-12-27 02:01:05,259][105692] Updated weights for policy 0, policy_version 1459048 (0.0010) [2023-12-27 02:01:05,820][105620] Updated weights for policy 1, policy_version 1461348 (0.0011) [2023-12-27 02:01:05,883][105620] Updated weights for policy 1, policy_version 1461358 (0.0009) [2023-12-27 02:01:05,932][105620] Updated weights for policy 1, policy_version 1461368 (0.0010) [2023-12-27 02:01:06,020][105692] Updated weights for policy 0, policy_version 1459058 (0.0011) [2023-12-27 02:01:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 747732992. Throughput: 0: 9734.7, 1: 9488.3. Samples: 747722376. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:01:06,062][104569] Avg episode reward: [(0, '8074.332'), (1, '8717.995')] [2023-12-27 02:01:06,076][105692] Updated weights for policy 0, policy_version 1459068 (0.0010) [2023-12-27 02:01:06,140][105692] Updated weights for policy 0, policy_version 1459078 (0.0009) [2023-12-27 02:01:06,201][105692] Updated weights for policy 0, policy_version 1459088 (0.0008) [2023-12-27 02:01:06,711][105620] Updated weights for policy 1, policy_version 1461378 (0.0011) [2023-12-27 02:01:06,775][105620] Updated weights for policy 1, policy_version 1461388 (0.0011) [2023-12-27 02:01:06,840][105620] Updated weights for policy 1, policy_version 1461398 (0.0011) [2023-12-27 02:01:06,895][105692] Updated weights for policy 0, policy_version 1459098 (0.0011) [2023-12-27 02:01:06,909][105620] Updated weights for policy 1, policy_version 1461408 (0.0011) [2023-12-27 02:01:06,954][105692] Updated weights for policy 0, policy_version 1459108 (0.0010) [2023-12-27 02:01:07,011][105692] Updated weights for policy 0, policy_version 1459118 (0.0005) [2023-12-27 02:01:07,669][105620] Updated weights for policy 1, policy_version 1461418 (0.0011) [2023-12-27 02:01:07,724][105692] Updated weights for policy 0, policy_version 1459128 (0.0009) [2023-12-27 02:01:07,727][105620] Updated weights for policy 1, policy_version 1461428 (0.0011) [2023-12-27 02:01:07,778][105692] Updated weights for policy 0, policy_version 1459138 (0.0010) [2023-12-27 02:01:07,784][105620] Updated weights for policy 1, policy_version 1461438 (0.0011) [2023-12-27 02:01:07,827][105692] Updated weights for policy 0, policy_version 1459148 (0.0010) [2023-12-27 02:01:08,444][105692] Updated weights for policy 0, policy_version 1459158 (0.0009) [2023-12-27 02:01:08,512][105692] Updated weights for policy 0, policy_version 1459168 (0.0008) [2023-12-27 02:01:08,527][105620] Updated weights for policy 1, policy_version 1461448 (0.0008) [2023-12-27 02:01:08,577][105692] Updated weights for policy 0, policy_version 1459178 (0.0006) [2023-12-27 02:01:08,591][105620] Updated weights for policy 1, policy_version 1461458 (0.0008) [2023-12-27 02:01:08,652][105620] Updated weights for policy 1, policy_version 1461468 (0.0009) [2023-12-27 02:01:09,218][105692] Updated weights for policy 0, policy_version 1459188 (0.0008) [2023-12-27 02:01:09,287][105692] Updated weights for policy 0, policy_version 1459198 (0.0008) [2023-12-27 02:01:09,351][105692] Updated weights for policy 0, policy_version 1459208 (0.0006) [2023-12-27 02:01:09,466][105620] Updated weights for policy 1, policy_version 1461478 (0.0009) [2023-12-27 02:01:09,525][105620] Updated weights for policy 1, policy_version 1461488 (0.0009) [2023-12-27 02:01:09,576][105620] Updated weights for policy 1, policy_version 1461498 (0.0009) [2023-12-27 02:01:10,068][105692] Updated weights for policy 0, policy_version 1459218 (0.0008) [2023-12-27 02:01:10,132][105692] Updated weights for policy 0, policy_version 1459228 (0.0007) [2023-12-27 02:01:10,186][105692] Updated weights for policy 0, policy_version 1459238 (0.0006) [2023-12-27 02:01:10,237][105692] Updated weights for policy 0, policy_version 1459248 (0.0006) [2023-12-27 02:01:10,415][105620] Updated weights for policy 1, policy_version 1461508 (0.0010) [2023-12-27 02:01:10,476][105620] Updated weights for policy 1, policy_version 1461518 (0.0009) [2023-12-27 02:01:10,543][105620] Updated weights for policy 1, policy_version 1461528 (0.0009) [2023-12-27 02:01:10,892][105692] Updated weights for policy 0, policy_version 1459258 (0.0006) [2023-12-27 02:01:10,963][105692] Updated weights for policy 0, policy_version 1459268 (0.0005) [2023-12-27 02:01:11,033][105692] Updated weights for policy 0, policy_version 1459278 (0.0006) [2023-12-27 02:01:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 747831296. Throughput: 0: 9837.8, 1: 9388.9. Samples: 747835980. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:01:11,063][104569] Avg episode reward: [(0, '7799.322'), (1, '8989.072')] [2023-12-27 02:01:11,227][105620] Updated weights for policy 1, policy_version 1461538 (0.0008) [2023-12-27 02:01:11,285][105620] Updated weights for policy 1, policy_version 1461548 (0.0007) [2023-12-27 02:01:11,346][105620] Updated weights for policy 1, policy_version 1461558 (0.0006) [2023-12-27 02:01:11,416][105620] Updated weights for policy 1, policy_version 1461568 (0.0008) [2023-12-27 02:01:11,731][105692] Updated weights for policy 0, policy_version 1459288 (0.0010) [2023-12-27 02:01:11,799][105692] Updated weights for policy 0, policy_version 1459298 (0.0010) [2023-12-27 02:01:11,869][105692] Updated weights for policy 0, policy_version 1459308 (0.0011) [2023-12-27 02:01:12,105][105620] Updated weights for policy 1, policy_version 1461578 (0.0007) [2023-12-27 02:01:12,164][105620] Updated weights for policy 1, policy_version 1461588 (0.0008) [2023-12-27 02:01:12,219][105620] Updated weights for policy 1, policy_version 1461598 (0.0011) [2023-12-27 02:01:12,613][105692] Updated weights for policy 0, policy_version 1459318 (0.0007) [2023-12-27 02:01:12,681][105692] Updated weights for policy 0, policy_version 1459328 (0.0006) [2023-12-27 02:01:12,739][105692] Updated weights for policy 0, policy_version 1459338 (0.0007) [2023-12-27 02:01:12,930][105620] Updated weights for policy 1, policy_version 1461608 (0.0011) [2023-12-27 02:01:12,991][105620] Updated weights for policy 1, policy_version 1461618 (0.0010) [2023-12-27 02:01:13,053][105620] Updated weights for policy 1, policy_version 1461628 (0.0010) [2023-12-27 02:01:13,400][105692] Updated weights for policy 0, policy_version 1459348 (0.0008) [2023-12-27 02:01:13,459][105692] Updated weights for policy 0, policy_version 1459358 (0.0005) [2023-12-27 02:01:13,525][105692] Updated weights for policy 0, policy_version 1459368 (0.0010) [2023-12-27 02:01:13,691][105620] Updated weights for policy 1, policy_version 1461638 (0.0009) [2023-12-27 02:01:13,744][105620] Updated weights for policy 1, policy_version 1461648 (0.0008) [2023-12-27 02:01:13,793][105620] Updated weights for policy 1, policy_version 1461658 (0.0008) [2023-12-27 02:01:14,235][105692] Updated weights for policy 0, policy_version 1459378 (0.0010) [2023-12-27 02:01:14,293][105692] Updated weights for policy 0, policy_version 1459388 (0.0010) [2023-12-27 02:01:14,347][105692] Updated weights for policy 0, policy_version 1459398 (0.0009) [2023-12-27 02:01:14,401][105692] Updated weights for policy 0, policy_version 1459408 (0.0010) [2023-12-27 02:01:14,481][105620] Updated weights for policy 1, policy_version 1461668 (0.0008) [2023-12-27 02:01:14,525][105620] Updated weights for policy 1, policy_version 1461678 (0.0008) [2023-12-27 02:01:14,578][105620] Updated weights for policy 1, policy_version 1461688 (0.0009) [2023-12-27 02:01:15,183][105692] Updated weights for policy 0, policy_version 1459418 (0.0011) [2023-12-27 02:01:15,228][105692] Updated weights for policy 0, policy_version 1459428 (0.0011) [2023-12-27 02:01:15,274][105692] Updated weights for policy 0, policy_version 1459438 (0.0011) [2023-12-27 02:01:15,397][105620] Updated weights for policy 1, policy_version 1461698 (0.0008) [2023-12-27 02:01:15,449][105620] Updated weights for policy 1, policy_version 1461708 (0.0008) [2023-12-27 02:01:15,498][105620] Updated weights for policy 1, policy_version 1461718 (0.0008) [2023-12-27 02:01:15,544][105620] Updated weights for policy 1, policy_version 1461728 (0.0008) [2023-12-27 02:01:16,044][105692] Updated weights for policy 0, policy_version 1459448 (0.0011) [2023-12-27 02:01:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 747921408. Throughput: 0: 9862.8, 1: 9380.9. Samples: 747895584. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:01:16,062][104569] Avg episode reward: [(0, '7710.650'), (1, '9168.498')] [2023-12-27 02:01:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001461728_374251520.pth... [2023-12-27 02:01:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001460640_373972992.pth [2023-12-27 02:01:16,100][105692] Updated weights for policy 0, policy_version 1459458 (0.0011) [2023-12-27 02:01:16,155][105692] Updated weights for policy 0, policy_version 1459468 (0.0010) [2023-12-27 02:01:16,176][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001459472_373678080.pth... [2023-12-27 02:01:16,180][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001458288_373374976.pth [2023-12-27 02:01:16,260][105620] Updated weights for policy 1, policy_version 1461738 (0.0008) [2023-12-27 02:01:16,317][105620] Updated weights for policy 1, policy_version 1461748 (0.0008) [2023-12-27 02:01:16,369][105620] Updated weights for policy 1, policy_version 1461758 (0.0008) [2023-12-27 02:01:16,857][105692] Updated weights for policy 0, policy_version 1459478 (0.0007) [2023-12-27 02:01:16,903][105692] Updated weights for policy 0, policy_version 1459488 (0.0005) [2023-12-27 02:01:16,949][105692] Updated weights for policy 0, policy_version 1459498 (0.0005) [2023-12-27 02:01:17,220][105620] Updated weights for policy 1, policy_version 1461768 (0.0009) [2023-12-27 02:01:17,281][105620] Updated weights for policy 1, policy_version 1461778 (0.0010) [2023-12-27 02:01:17,343][105620] Updated weights for policy 1, policy_version 1461789 (0.0009) [2023-12-27 02:01:17,528][105692] Updated weights for policy 0, policy_version 1459508 (0.0007) [2023-12-27 02:01:17,589][105692] Updated weights for policy 0, policy_version 1459518 (0.0009) [2023-12-27 02:01:17,651][105692] Updated weights for policy 0, policy_version 1459528 (0.0009) [2023-12-27 02:01:18,045][105620] Updated weights for policy 1, policy_version 1461799 (0.0008) [2023-12-27 02:01:18,103][105620] Updated weights for policy 1, policy_version 1461809 (0.0008) [2023-12-27 02:01:18,166][105620] Updated weights for policy 1, policy_version 1461819 (0.0009) [2023-12-27 02:01:18,361][105692] Updated weights for policy 0, policy_version 1459538 (0.0009) [2023-12-27 02:01:18,423][105692] Updated weights for policy 0, policy_version 1459548 (0.0008) [2023-12-27 02:01:18,485][105692] Updated weights for policy 0, policy_version 1459558 (0.0009) [2023-12-27 02:01:18,544][105692] Updated weights for policy 0, policy_version 1459568 (0.0009) [2023-12-27 02:01:18,984][105620] Updated weights for policy 1, policy_version 1461829 (0.0008) [2023-12-27 02:01:19,042][105620] Updated weights for policy 1, policy_version 1461839 (0.0008) [2023-12-27 02:01:19,090][105620] Updated weights for policy 1, policy_version 1461849 (0.0007) [2023-12-27 02:01:19,255][105692] Updated weights for policy 0, policy_version 1459578 (0.0010) [2023-12-27 02:01:19,311][105692] Updated weights for policy 0, policy_version 1459588 (0.0010) [2023-12-27 02:01:19,387][105692] Updated weights for policy 0, policy_version 1459598 (0.0009) [2023-12-27 02:01:19,860][105620] Updated weights for policy 1, policy_version 1461859 (0.0008) [2023-12-27 02:01:19,928][105620] Updated weights for policy 1, policy_version 1461869 (0.0006) [2023-12-27 02:01:19,992][105620] Updated weights for policy 1, policy_version 1461879 (0.0009) [2023-12-27 02:01:20,020][105692] Updated weights for policy 0, policy_version 1459608 (0.0010) [2023-12-27 02:01:20,080][105692] Updated weights for policy 0, policy_version 1459618 (0.0011) [2023-12-27 02:01:20,139][105692] Updated weights for policy 0, policy_version 1459628 (0.0011) [2023-12-27 02:01:20,740][105620] Updated weights for policy 1, policy_version 1461889 (0.0008) [2023-12-27 02:01:20,803][105620] Updated weights for policy 1, policy_version 1461899 (0.0011) [2023-12-27 02:01:20,862][105620] Updated weights for policy 1, policy_version 1461909 (0.0011) [2023-12-27 02:01:20,884][105692] Updated weights for policy 0, policy_version 1459638 (0.0007) [2023-12-27 02:01:20,926][105620] Updated weights for policy 1, policy_version 1461919 (0.0011) [2023-12-27 02:01:20,948][105692] Updated weights for policy 0, policy_version 1459648 (0.0006) [2023-12-27 02:01:21,016][105692] Updated weights for policy 0, policy_version 1459658 (0.0007) [2023-12-27 02:01:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 748027904. Throughput: 0: 9904.8, 1: 9295.7. Samples: 748010944. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:01:21,062][104569] Avg episode reward: [(0, '8436.789'), (1, '9169.445')] [2023-12-27 02:01:21,745][105620] Updated weights for policy 1, policy_version 1461929 (0.0008) [2023-12-27 02:01:21,785][105692] Updated weights for policy 0, policy_version 1459668 (0.0009) [2023-12-27 02:01:21,796][105620] Updated weights for policy 1, policy_version 1461939 (0.0006) [2023-12-27 02:01:21,845][105692] Updated weights for policy 0, policy_version 1459678 (0.0006) [2023-12-27 02:01:21,846][105620] Updated weights for policy 1, policy_version 1461949 (0.0008) [2023-12-27 02:01:21,905][105692] Updated weights for policy 0, policy_version 1459688 (0.0005) [2023-12-27 02:01:22,642][105692] Updated weights for policy 0, policy_version 1459698 (0.0009) [2023-12-27 02:01:22,672][105620] Updated weights for policy 1, policy_version 1461959 (0.0007) [2023-12-27 02:01:22,706][105692] Updated weights for policy 0, policy_version 1459708 (0.0008) [2023-12-27 02:01:22,733][105620] Updated weights for policy 1, policy_version 1461969 (0.0007) [2023-12-27 02:01:22,765][105692] Updated weights for policy 0, policy_version 1459718 (0.0008) [2023-12-27 02:01:22,795][105620] Updated weights for policy 1, policy_version 1461979 (0.0007) [2023-12-27 02:01:22,831][105692] Updated weights for policy 0, policy_version 1459728 (0.0008) [2023-12-27 02:01:23,490][105692] Updated weights for policy 0, policy_version 1459738 (0.0005) [2023-12-27 02:01:23,533][105620] Updated weights for policy 1, policy_version 1461989 (0.0009) [2023-12-27 02:01:23,552][105692] Updated weights for policy 0, policy_version 1459748 (0.0008) [2023-12-27 02:01:23,597][105620] Updated weights for policy 1, policy_version 1461999 (0.0006) [2023-12-27 02:01:23,613][105692] Updated weights for policy 0, policy_version 1459758 (0.0011) [2023-12-27 02:01:23,667][105620] Updated weights for policy 1, policy_version 1462009 (0.0006) [2023-12-27 02:01:24,226][105620] Updated weights for policy 1, policy_version 1462019 (0.0006) [2023-12-27 02:01:24,280][105620] Updated weights for policy 1, policy_version 1462029 (0.0005) [2023-12-27 02:01:24,321][105692] Updated weights for policy 0, policy_version 1459768 (0.0010) [2023-12-27 02:01:24,340][105620] Updated weights for policy 1, policy_version 1462039 (0.0006) [2023-12-27 02:01:24,376][105692] Updated weights for policy 0, policy_version 1459778 (0.0010) [2023-12-27 02:01:24,431][105692] Updated weights for policy 0, policy_version 1459788 (0.0010) [2023-12-27 02:01:24,951][105620] Updated weights for policy 1, policy_version 1462049 (0.0010) [2023-12-27 02:01:25,020][105620] Updated weights for policy 1, policy_version 1462059 (0.0006) [2023-12-27 02:01:25,076][105620] Updated weights for policy 1, policy_version 1462069 (0.0006) [2023-12-27 02:01:25,137][105620] Updated weights for policy 1, policy_version 1462079 (0.0006) [2023-12-27 02:01:25,181][105692] Updated weights for policy 0, policy_version 1459798 (0.0011) [2023-12-27 02:01:25,236][105692] Updated weights for policy 0, policy_version 1459808 (0.0010) [2023-12-27 02:01:25,302][105692] Updated weights for policy 0, policy_version 1459818 (0.0010) [2023-12-27 02:01:25,715][105620] Updated weights for policy 1, policy_version 1462089 (0.0009) [2023-12-27 02:01:25,775][105620] Updated weights for policy 1, policy_version 1462099 (0.0009) [2023-12-27 02:01:25,850][105620] Updated weights for policy 1, policy_version 1462109 (0.0008) [2023-12-27 02:01:26,007][105692] Updated weights for policy 0, policy_version 1459828 (0.0010) [2023-12-27 02:01:26,060][105692] Updated weights for policy 0, policy_version 1459838 (0.0008) [2023-12-27 02:01:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19605.2). Total num frames: 748118016. Throughput: 0: 9884.0, 1: 9367.2. Samples: 748127644. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:01:26,063][104569] Avg episode reward: [(0, '8896.671'), (1, '9173.193')] [2023-12-27 02:01:26,121][105692] Updated weights for policy 0, policy_version 1459848 (0.0006) [2023-12-27 02:01:26,570][105620] Updated weights for policy 1, policy_version 1462119 (0.0009) [2023-12-27 02:01:26,636][105620] Updated weights for policy 1, policy_version 1462129 (0.0010) [2023-12-27 02:01:26,700][105620] Updated weights for policy 1, policy_version 1462139 (0.0010) [2023-12-27 02:01:26,758][105692] Updated weights for policy 0, policy_version 1459858 (0.0007) [2023-12-27 02:01:26,812][105692] Updated weights for policy 0, policy_version 1459868 (0.0005) [2023-12-27 02:01:26,869][105692] Updated weights for policy 0, policy_version 1459878 (0.0005) [2023-12-27 02:01:26,928][105692] Updated weights for policy 0, policy_version 1459888 (0.0006) [2023-12-27 02:01:27,404][105620] Updated weights for policy 1, policy_version 1462149 (0.0008) [2023-12-27 02:01:27,464][105620] Updated weights for policy 1, policy_version 1462159 (0.0008) [2023-12-27 02:01:27,531][105620] Updated weights for policy 1, policy_version 1462169 (0.0010) [2023-12-27 02:01:27,637][105692] Updated weights for policy 0, policy_version 1459898 (0.0008) [2023-12-27 02:01:27,682][105692] Updated weights for policy 0, policy_version 1459908 (0.0008) [2023-12-27 02:01:27,734][105692] Updated weights for policy 0, policy_version 1459918 (0.0008) [2023-12-27 02:01:28,167][105620] Updated weights for policy 1, policy_version 1462179 (0.0010) [2023-12-27 02:01:28,215][105620] Updated weights for policy 1, policy_version 1462189 (0.0010) [2023-12-27 02:01:28,267][105620] Updated weights for policy 1, policy_version 1462199 (0.0010) [2023-12-27 02:01:28,375][105692] Updated weights for policy 0, policy_version 1459928 (0.0008) [2023-12-27 02:01:28,427][105692] Updated weights for policy 0, policy_version 1459938 (0.0009) [2023-12-27 02:01:28,493][105692] Updated weights for policy 0, policy_version 1459948 (0.0010) [2023-12-27 02:01:28,902][105620] Updated weights for policy 1, policy_version 1462209 (0.0009) [2023-12-27 02:01:28,968][105620] Updated weights for policy 1, policy_version 1462219 (0.0007) [2023-12-27 02:01:29,027][105620] Updated weights for policy 1, policy_version 1462229 (0.0008) [2023-12-27 02:01:29,078][105620] Updated weights for policy 1, policy_version 1462239 (0.0008) [2023-12-27 02:01:29,215][105692] Updated weights for policy 0, policy_version 1459958 (0.0011) [2023-12-27 02:01:29,282][105692] Updated weights for policy 0, policy_version 1459969 (0.0010) [2023-12-27 02:01:29,335][105692] Updated weights for policy 0, policy_version 1459979 (0.0009) [2023-12-27 02:01:29,801][105620] Updated weights for policy 1, policy_version 1462249 (0.0006) [2023-12-27 02:01:29,864][105620] Updated weights for policy 1, policy_version 1462259 (0.0009) [2023-12-27 02:01:29,920][105620] Updated weights for policy 1, policy_version 1462269 (0.0007) [2023-12-27 02:01:30,197][105692] Updated weights for policy 0, policy_version 1459989 (0.0009) [2023-12-27 02:01:30,258][105692] Updated weights for policy 0, policy_version 1459999 (0.0009) [2023-12-27 02:01:30,316][105692] Updated weights for policy 0, policy_version 1460009 (0.0009) [2023-12-27 02:01:30,562][105620] Updated weights for policy 1, policy_version 1462279 (0.0008) [2023-12-27 02:01:30,610][105620] Updated weights for policy 1, policy_version 1462289 (0.0007) [2023-12-27 02:01:30,654][105620] Updated weights for policy 1, policy_version 1462299 (0.0008) [2023-12-27 02:01:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 748216320. Throughput: 0: 9941.4, 1: 9408.9. Samples: 748189652. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:01:31,063][104569] Avg episode reward: [(0, '8807.182'), (1, '9081.451')] [2023-12-27 02:01:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001462304_374398976.pth... [2023-12-27 02:01:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001461184_374112256.pth [2023-12-27 02:01:31,089][105692] Updated weights for policy 0, policy_version 1460019 (0.0009) [2023-12-27 02:01:31,154][105692] Updated weights for policy 0, policy_version 1460029 (0.0010) [2023-12-27 02:01:31,202][105692] Updated weights for policy 0, policy_version 1460039 (0.0010) [2023-12-27 02:01:31,252][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001460048_373825536.pth... [2023-12-27 02:01:31,255][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001458896_373530624.pth [2023-12-27 02:01:31,461][105620] Updated weights for policy 1, policy_version 1462309 (0.0007) [2023-12-27 02:01:31,516][105620] Updated weights for policy 1, policy_version 1462319 (0.0008) [2023-12-27 02:01:31,566][105620] Updated weights for policy 1, policy_version 1462329 (0.0008) [2023-12-27 02:01:31,967][105692] Updated weights for policy 0, policy_version 1460049 (0.0010) [2023-12-27 02:01:32,023][105692] Updated weights for policy 0, policy_version 1460059 (0.0010) [2023-12-27 02:01:32,077][105692] Updated weights for policy 0, policy_version 1460069 (0.0008) [2023-12-27 02:01:32,139][105692] Updated weights for policy 0, policy_version 1460079 (0.0005) [2023-12-27 02:01:32,382][105620] Updated weights for policy 1, policy_version 1462339 (0.0008) [2023-12-27 02:01:32,435][105620] Updated weights for policy 1, policy_version 1462349 (0.0007) [2023-12-27 02:01:32,491][105620] Updated weights for policy 1, policy_version 1462359 (0.0005) [2023-12-27 02:01:32,929][105692] Updated weights for policy 0, policy_version 1460089 (0.0009) [2023-12-27 02:01:32,984][105692] Updated weights for policy 0, policy_version 1460099 (0.0009) [2023-12-27 02:01:33,044][105692] Updated weights for policy 0, policy_version 1460109 (0.0009) [2023-12-27 02:01:33,063][105620] Updated weights for policy 1, policy_version 1462369 (0.0005) [2023-12-27 02:01:33,116][105620] Updated weights for policy 1, policy_version 1462379 (0.0008) [2023-12-27 02:01:33,169][105620] Updated weights for policy 1, policy_version 1462390 (0.0009) [2023-12-27 02:01:33,226][105620] Updated weights for policy 1, policy_version 1462400 (0.0010) [2023-12-27 02:01:33,656][105692] Updated weights for policy 0, policy_version 1460119 (0.0009) [2023-12-27 02:01:33,716][105692] Updated weights for policy 0, policy_version 1460129 (0.0009) [2023-12-27 02:01:33,776][105692] Updated weights for policy 0, policy_version 1460139 (0.0009) [2023-12-27 02:01:34,043][105620] Updated weights for policy 1, policy_version 1462410 (0.0009) [2023-12-27 02:01:34,090][105620] Updated weights for policy 1, policy_version 1462420 (0.0009) [2023-12-27 02:01:34,154][105620] Updated weights for policy 1, policy_version 1462430 (0.0008) [2023-12-27 02:01:34,497][105692] Updated weights for policy 0, policy_version 1460149 (0.0007) [2023-12-27 02:01:34,565][105692] Updated weights for policy 0, policy_version 1460159 (0.0005) [2023-12-27 02:01:34,631][105692] Updated weights for policy 0, policy_version 1460169 (0.0008) [2023-12-27 02:01:34,849][105620] Updated weights for policy 1, policy_version 1462440 (0.0009) [2023-12-27 02:01:34,931][105620] Updated weights for policy 1, policy_version 1462450 (0.0009) [2023-12-27 02:01:34,985][105620] Updated weights for policy 1, policy_version 1462460 (0.0009) [2023-12-27 02:01:35,288][105692] Updated weights for policy 0, policy_version 1460179 (0.0008) [2023-12-27 02:01:35,340][105692] Updated weights for policy 0, policy_version 1460189 (0.0005) [2023-12-27 02:01:35,393][105692] Updated weights for policy 0, policy_version 1460199 (0.0005) [2023-12-27 02:01:35,789][105620] Updated weights for policy 1, policy_version 1462470 (0.0009) [2023-12-27 02:01:35,841][105620] Updated weights for policy 1, policy_version 1462480 (0.0009) [2023-12-27 02:01:35,895][105620] Updated weights for policy 1, policy_version 1462490 (0.0009) [2023-12-27 02:01:36,026][105692] Updated weights for policy 0, policy_version 1460209 (0.0006) [2023-12-27 02:01:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.1, 300 sec: 19605.3). Total num frames: 748314624. Throughput: 0: 9815.0, 1: 9483.1. Samples: 748304244. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:01:36,063][104569] Avg episode reward: [(0, '8624.740'), (1, '8987.726')] [2023-12-27 02:01:36,076][105692] Updated weights for policy 0, policy_version 1460219 (0.0008) [2023-12-27 02:01:36,136][105692] Updated weights for policy 0, policy_version 1460229 (0.0009) [2023-12-27 02:01:36,187][105692] Updated weights for policy 0, policy_version 1460239 (0.0008) [2023-12-27 02:01:36,691][105620] Updated weights for policy 1, policy_version 1462500 (0.0009) [2023-12-27 02:01:36,755][105620] Updated weights for policy 1, policy_version 1462510 (0.0008) [2023-12-27 02:01:36,801][105620] Updated weights for policy 1, policy_version 1462520 (0.0008) [2023-12-27 02:01:36,976][105692] Updated weights for policy 0, policy_version 1460249 (0.0010) [2023-12-27 02:01:37,027][105692] Updated weights for policy 0, policy_version 1460259 (0.0010) [2023-12-27 02:01:37,086][105692] Updated weights for policy 0, policy_version 1460269 (0.0010) [2023-12-27 02:01:37,578][105620] Updated weights for policy 1, policy_version 1462530 (0.0008) [2023-12-27 02:01:37,640][105620] Updated weights for policy 1, policy_version 1462540 (0.0009) [2023-12-27 02:01:37,705][105620] Updated weights for policy 1, policy_version 1462550 (0.0009) [2023-12-27 02:01:37,757][105620] Updated weights for policy 1, policy_version 1462560 (0.0009) [2023-12-27 02:01:37,817][105692] Updated weights for policy 0, policy_version 1460279 (0.0009) [2023-12-27 02:01:37,868][105692] Updated weights for policy 0, policy_version 1460289 (0.0009) [2023-12-27 02:01:37,933][105692] Updated weights for policy 0, policy_version 1460299 (0.0008) [2023-12-27 02:01:38,548][105620] Updated weights for policy 1, policy_version 1462570 (0.0009) [2023-12-27 02:01:38,606][105620] Updated weights for policy 1, policy_version 1462580 (0.0009) [2023-12-27 02:01:38,665][105692] Updated weights for policy 0, policy_version 1460309 (0.0009) [2023-12-27 02:01:38,669][105620] Updated weights for policy 1, policy_version 1462590 (0.0008) [2023-12-27 02:01:38,728][105692] Updated weights for policy 0, policy_version 1460319 (0.0007) [2023-12-27 02:01:38,787][105692] Updated weights for policy 0, policy_version 1460329 (0.0006) [2023-12-27 02:01:39,457][105620] Updated weights for policy 1, policy_version 1462600 (0.0008) [2023-12-27 02:01:39,496][105692] Updated weights for policy 0, policy_version 1460339 (0.0007) [2023-12-27 02:01:39,515][105620] Updated weights for policy 1, policy_version 1462610 (0.0007) [2023-12-27 02:01:39,561][105692] Updated weights for policy 0, policy_version 1460349 (0.0008) [2023-12-27 02:01:39,581][105620] Updated weights for policy 1, policy_version 1462620 (0.0006) [2023-12-27 02:01:39,621][105692] Updated weights for policy 0, policy_version 1460359 (0.0007) [2023-12-27 02:01:40,270][105620] Updated weights for policy 1, policy_version 1462630 (0.0007) [2023-12-27 02:01:40,325][105620] Updated weights for policy 1, policy_version 1462640 (0.0007) [2023-12-27 02:01:40,364][105692] Updated weights for policy 0, policy_version 1460369 (0.0009) [2023-12-27 02:01:40,387][105620] Updated weights for policy 1, policy_version 1462650 (0.0005) [2023-12-27 02:01:40,414][105692] Updated weights for policy 0, policy_version 1460379 (0.0009) [2023-12-27 02:01:40,468][105692] Updated weights for policy 0, policy_version 1460389 (0.0010) [2023-12-27 02:01:40,525][105692] Updated weights for policy 0, policy_version 1460400 (0.0012) [2023-12-27 02:01:40,957][105620] Updated weights for policy 1, policy_version 1462660 (0.0005) [2023-12-27 02:01:41,022][105620] Updated weights for policy 1, policy_version 1462670 (0.0007) [2023-12-27 02:01:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 748404736. Throughput: 0: 9765.3, 1: 9508.8. Samples: 748417592. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:01:41,062][104569] Avg episode reward: [(0, '8712.265'), (1, '8621.601')] [2023-12-27 02:01:41,089][105620] Updated weights for policy 1, policy_version 1462680 (0.0008) [2023-12-27 02:01:41,304][105692] Updated weights for policy 0, policy_version 1460410 (0.0011) [2023-12-27 02:01:41,367][105692] Updated weights for policy 0, policy_version 1460420 (0.0011) [2023-12-27 02:01:41,427][105692] Updated weights for policy 0, policy_version 1460430 (0.0011) [2023-12-27 02:01:41,788][105620] Updated weights for policy 1, policy_version 1462690 (0.0008) [2023-12-27 02:01:41,857][105620] Updated weights for policy 1, policy_version 1462700 (0.0007) [2023-12-27 02:01:41,922][105620] Updated weights for policy 1, policy_version 1462710 (0.0008) [2023-12-27 02:01:41,989][105620] Updated weights for policy 1, policy_version 1462720 (0.0008) [2023-12-27 02:01:42,189][105692] Updated weights for policy 0, policy_version 1460440 (0.0007) [2023-12-27 02:01:42,242][105692] Updated weights for policy 0, policy_version 1460450 (0.0008) [2023-12-27 02:01:42,305][105692] Updated weights for policy 0, policy_version 1460460 (0.0009) [2023-12-27 02:01:42,739][105620] Updated weights for policy 1, policy_version 1462730 (0.0009) [2023-12-27 02:01:42,785][105620] Updated weights for policy 1, policy_version 1462740 (0.0008) [2023-12-27 02:01:42,839][105620] Updated weights for policy 1, policy_version 1462750 (0.0009) [2023-12-27 02:01:43,076][105692] Updated weights for policy 0, policy_version 1460470 (0.0009) [2023-12-27 02:01:43,132][105692] Updated weights for policy 0, policy_version 1460480 (0.0010) [2023-12-27 02:01:43,184][105692] Updated weights for policy 0, policy_version 1460490 (0.0009) [2023-12-27 02:01:43,489][105620] Updated weights for policy 1, policy_version 1462760 (0.0006) [2023-12-27 02:01:43,560][105620] Updated weights for policy 1, policy_version 1462770 (0.0005) [2023-12-27 02:01:43,620][105620] Updated weights for policy 1, policy_version 1462780 (0.0005) [2023-12-27 02:01:43,959][105692] Updated weights for policy 0, policy_version 1460500 (0.0010) [2023-12-27 02:01:44,032][105692] Updated weights for policy 0, policy_version 1460510 (0.0011) [2023-12-27 02:01:44,097][105620] Updated weights for policy 1, policy_version 1462790 (0.0006) [2023-12-27 02:01:44,107][105692] Updated weights for policy 0, policy_version 1460520 (0.0011) [2023-12-27 02:01:44,164][105620] Updated weights for policy 1, policy_version 1462800 (0.0006) [2023-12-27 02:01:44,220][105620] Updated weights for policy 1, policy_version 1462810 (0.0006) [2023-12-27 02:01:44,806][105692] Updated weights for policy 0, policy_version 1460530 (0.0011) [2023-12-27 02:01:44,879][105692] Updated weights for policy 0, policy_version 1460540 (0.0011) [2023-12-27 02:01:44,919][105620] Updated weights for policy 1, policy_version 1462820 (0.0006) [2023-12-27 02:01:44,946][105692] Updated weights for policy 0, policy_version 1460550 (0.0011) [2023-12-27 02:01:44,976][105620] Updated weights for policy 1, policy_version 1462830 (0.0006) [2023-12-27 02:01:45,003][105692] Updated weights for policy 0, policy_version 1460560 (0.0011) [2023-12-27 02:01:45,037][105620] Updated weights for policy 1, policy_version 1462840 (0.0006) [2023-12-27 02:01:45,742][105692] Updated weights for policy 0, policy_version 1460570 (0.0010) [2023-12-27 02:01:45,747][105620] Updated weights for policy 1, policy_version 1462850 (0.0006) [2023-12-27 02:01:45,790][105692] Updated weights for policy 0, policy_version 1460580 (0.0010) [2023-12-27 02:01:45,801][105620] Updated weights for policy 1, policy_version 1462860 (0.0007) [2023-12-27 02:01:45,845][105692] Updated weights for policy 0, policy_version 1460590 (0.0010) [2023-12-27 02:01:45,847][105620] Updated weights for policy 1, policy_version 1462870 (0.0008) [2023-12-27 02:01:45,892][105620] Updated weights for policy 1, policy_version 1462880 (0.0008) [2023-12-27 02:01:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 748511232. Throughput: 0: 9717.1, 1: 9592.8. Samples: 748476340. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:01:46,063][104569] Avg episode reward: [(0, '8891.699'), (1, '8717.971')] [2023-12-27 02:01:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001460592_373964800.pth... [2023-12-27 02:01:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001462880_374546432.pth... [2023-12-27 02:01:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001459472_373678080.pth [2023-12-27 02:01:46,078][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_001460592_373964800.pth [2023-12-27 02:01:46,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001461728_374251520.pth [2023-12-27 02:01:46,081][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_001462880_374546432.pth [2023-12-27 02:01:46,608][105620] Updated weights for policy 1, policy_version 1462890 (0.0007) [2023-12-27 02:01:46,626][105692] Updated weights for policy 0, policy_version 1460600 (0.0008) [2023-12-27 02:01:46,667][105620] Updated weights for policy 1, policy_version 1462900 (0.0008) [2023-12-27 02:01:46,682][105692] Updated weights for policy 0, policy_version 1460610 (0.0006) [2023-12-27 02:01:46,716][105620] Updated weights for policy 1, policy_version 1462910 (0.0006) [2023-12-27 02:01:46,738][105692] Updated weights for policy 0, policy_version 1460620 (0.0007) [2023-12-27 02:01:47,384][105620] Updated weights for policy 1, policy_version 1462920 (0.0009) [2023-12-27 02:01:47,430][105620] Updated weights for policy 1, policy_version 1462930 (0.0009) [2023-12-27 02:01:47,449][105692] Updated weights for policy 0, policy_version 1460630 (0.0007) [2023-12-27 02:01:47,478][105620] Updated weights for policy 1, policy_version 1462940 (0.0006) [2023-12-27 02:01:47,506][105692] Updated weights for policy 0, policy_version 1460640 (0.0007) [2023-12-27 02:01:47,557][105692] Updated weights for policy 0, policy_version 1460650 (0.0008) [2023-12-27 02:01:48,247][105620] Updated weights for policy 1, policy_version 1462950 (0.0007) [2023-12-27 02:01:48,282][105692] Updated weights for policy 0, policy_version 1460660 (0.0008) [2023-12-27 02:01:48,296][105620] Updated weights for policy 1, policy_version 1462960 (0.0007) [2023-12-27 02:01:48,357][105620] Updated weights for policy 1, policy_version 1462970 (0.0007) [2023-12-27 02:01:48,372][105692] Updated weights for policy 0, policy_version 1460670 (0.0006) [2023-12-27 02:01:48,424][105692] Updated weights for policy 0, policy_version 1460680 (0.0008) [2023-12-27 02:01:48,982][105620] Updated weights for policy 1, policy_version 1462980 (0.0009) [2023-12-27 02:01:49,044][105620] Updated weights for policy 1, policy_version 1462990 (0.0010) [2023-12-27 02:01:49,106][105620] Updated weights for policy 1, policy_version 1463000 (0.0007) [2023-12-27 02:01:49,207][105692] Updated weights for policy 0, policy_version 1460690 (0.0009) [2023-12-27 02:01:49,279][105692] Updated weights for policy 0, policy_version 1460700 (0.0008) [2023-12-27 02:01:49,311][105585] KL-divergence is very high: 103.9713 [2023-12-27 02:01:49,335][105692] Updated weights for policy 0, policy_version 1460710 (0.0009) [2023-12-27 02:01:49,362][105585] KL-divergence is very high: 124.2429 [2023-12-27 02:01:49,401][105692] Updated weights for policy 0, policy_version 1460720 (0.0008) [2023-12-27 02:01:49,778][105620] Updated weights for policy 1, policy_version 1463010 (0.0008) [2023-12-27 02:01:49,832][105620] Updated weights for policy 1, policy_version 1463020 (0.0011) [2023-12-27 02:01:49,892][105620] Updated weights for policy 1, policy_version 1463030 (0.0011) [2023-12-27 02:01:49,957][105620] Updated weights for policy 1, policy_version 1463040 (0.0012) [2023-12-27 02:01:50,069][105692] Updated weights for policy 0, policy_version 1460730 (0.0009) [2023-12-27 02:01:50,137][105692] Updated weights for policy 0, policy_version 1460740 (0.0007) [2023-12-27 02:01:50,195][105692] Updated weights for policy 0, policy_version 1460750 (0.0005) [2023-12-27 02:01:50,694][105620] Updated weights for policy 1, policy_version 1463050 (0.0011) [2023-12-27 02:01:50,760][105620] Updated weights for policy 1, policy_version 1463060 (0.0011) [2023-12-27 02:01:50,827][105620] Updated weights for policy 1, policy_version 1463070 (0.0009) [2023-12-27 02:01:50,839][105692] Updated weights for policy 0, policy_version 1460760 (0.0007) [2023-12-27 02:01:50,891][105692] Updated weights for policy 0, policy_version 1460770 (0.0010) [2023-12-27 02:01:50,958][105692] Updated weights for policy 0, policy_version 1460780 (0.0011) [2023-12-27 02:01:51,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 748609536. Throughput: 0: 9663.4, 1: 9688.0. Samples: 748593192. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:01:51,063][104569] Avg episode reward: [(0, '8246.044'), (1, '8814.424')] [2023-12-27 02:01:51,549][105620] Updated weights for policy 1, policy_version 1463080 (0.0009) [2023-12-27 02:01:51,601][105620] Updated weights for policy 1, policy_version 1463090 (0.0010) [2023-12-27 02:01:51,670][105620] Updated weights for policy 1, policy_version 1463100 (0.0010) [2023-12-27 02:01:51,687][105692] Updated weights for policy 0, policy_version 1460790 (0.0011) [2023-12-27 02:01:51,751][105692] Updated weights for policy 0, policy_version 1460800 (0.0009) [2023-12-27 02:01:51,807][105692] Updated weights for policy 0, policy_version 1460810 (0.0011) [2023-12-27 02:01:52,451][105620] Updated weights for policy 1, policy_version 1463110 (0.0009) [2023-12-27 02:01:52,505][105620] Updated weights for policy 1, policy_version 1463120 (0.0009) [2023-12-27 02:01:52,555][105620] Updated weights for policy 1, policy_version 1463130 (0.0008) [2023-12-27 02:01:52,580][105692] Updated weights for policy 0, policy_version 1460820 (0.0010) [2023-12-27 02:01:52,635][105692] Updated weights for policy 0, policy_version 1460830 (0.0009) [2023-12-27 02:01:52,694][105692] Updated weights for policy 0, policy_version 1460840 (0.0009) [2023-12-27 02:01:53,295][105620] Updated weights for policy 1, policy_version 1463140 (0.0007) [2023-12-27 02:01:53,350][105620] Updated weights for policy 1, policy_version 1463150 (0.0009) [2023-12-27 02:01:53,410][105620] Updated weights for policy 1, policy_version 1463160 (0.0009) [2023-12-27 02:01:53,451][105692] Updated weights for policy 0, policy_version 1460850 (0.0009) [2023-12-27 02:01:53,507][105692] Updated weights for policy 0, policy_version 1460860 (0.0007) [2023-12-27 02:01:53,560][105692] Updated weights for policy 0, policy_version 1460870 (0.0008) [2023-12-27 02:01:53,609][105692] Updated weights for policy 0, policy_version 1460880 (0.0009) [2023-12-27 02:01:54,140][105620] Updated weights for policy 1, policy_version 1463170 (0.0008) [2023-12-27 02:01:54,205][105620] Updated weights for policy 1, policy_version 1463180 (0.0008) [2023-12-27 02:01:54,258][105620] Updated weights for policy 1, policy_version 1463190 (0.0006) [2023-12-27 02:01:54,320][105620] Updated weights for policy 1, policy_version 1463200 (0.0008) [2023-12-27 02:01:54,420][105692] Updated weights for policy 0, policy_version 1460890 (0.0009) [2023-12-27 02:01:54,477][105692] Updated weights for policy 0, policy_version 1460900 (0.0009) [2023-12-27 02:01:54,537][105692] Updated weights for policy 0, policy_version 1460910 (0.0007) [2023-12-27 02:01:54,966][105620] Updated weights for policy 1, policy_version 1463210 (0.0009) [2023-12-27 02:01:55,016][105620] Updated weights for policy 1, policy_version 1463220 (0.0009) [2023-12-27 02:01:55,066][105620] Updated weights for policy 1, policy_version 1463230 (0.0006) [2023-12-27 02:01:55,273][105692] Updated weights for policy 0, policy_version 1460920 (0.0009) [2023-12-27 02:01:55,328][105692] Updated weights for policy 0, policy_version 1460930 (0.0008) [2023-12-27 02:01:55,393][105692] Updated weights for policy 0, policy_version 1460940 (0.0005) [2023-12-27 02:01:55,868][105620] Updated weights for policy 1, policy_version 1463240 (0.0007) [2023-12-27 02:01:55,922][105620] Updated weights for policy 1, policy_version 1463250 (0.0008) [2023-12-27 02:01:55,975][105620] Updated weights for policy 1, policy_version 1463260 (0.0008) [2023-12-27 02:01:56,034][105692] Updated weights for policy 0, policy_version 1460950 (0.0008) [2023-12-27 02:01:56,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 748699648. Throughput: 0: 9616.7, 1: 9762.8. Samples: 748708060. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:01:56,063][104569] Avg episode reward: [(0, '8246.688'), (1, '8717.379')] [2023-12-27 02:01:56,087][105692] Updated weights for policy 0, policy_version 1460960 (0.0010) [2023-12-27 02:01:56,138][105692] Updated weights for policy 0, policy_version 1460970 (0.0010) [2023-12-27 02:01:56,686][105620] Updated weights for policy 1, policy_version 1463270 (0.0006) [2023-12-27 02:01:56,734][105620] Updated weights for policy 1, policy_version 1463280 (0.0005) [2023-12-27 02:01:56,788][105620] Updated weights for policy 1, policy_version 1463290 (0.0008) [2023-12-27 02:01:56,870][105692] Updated weights for policy 0, policy_version 1460980 (0.0010) [2023-12-27 02:01:56,920][105692] Updated weights for policy 0, policy_version 1460990 (0.0010) [2023-12-27 02:01:56,975][105692] Updated weights for policy 0, policy_version 1461000 (0.0010) [2023-12-27 02:01:57,508][105620] Updated weights for policy 1, policy_version 1463300 (0.0008) [2023-12-27 02:01:57,568][105620] Updated weights for policy 1, policy_version 1463310 (0.0008) [2023-12-27 02:01:57,626][105620] Updated weights for policy 1, policy_version 1463320 (0.0010) [2023-12-27 02:01:57,684][105692] Updated weights for policy 0, policy_version 1461010 (0.0010) [2023-12-27 02:01:57,752][105692] Updated weights for policy 0, policy_version 1461020 (0.0008) [2023-12-27 02:01:57,816][105692] Updated weights for policy 0, policy_version 1461030 (0.0007) [2023-12-27 02:01:57,877][105692] Updated weights for policy 0, policy_version 1461040 (0.0009) [2023-12-27 02:01:58,433][105620] Updated weights for policy 1, policy_version 1463330 (0.0009) [2023-12-27 02:01:58,499][105620] Updated weights for policy 1, policy_version 1463340 (0.0008) [2023-12-27 02:01:58,561][105620] Updated weights for policy 1, policy_version 1463350 (0.0007) [2023-12-27 02:01:58,593][105692] Updated weights for policy 0, policy_version 1461050 (0.0008) [2023-12-27 02:01:58,626][105620] Updated weights for policy 1, policy_version 1463360 (0.0007) [2023-12-27 02:01:58,654][105692] Updated weights for policy 0, policy_version 1461060 (0.0008) [2023-12-27 02:01:58,719][105692] Updated weights for policy 0, policy_version 1461070 (0.0008) [2023-12-27 02:01:59,470][105620] Updated weights for policy 1, policy_version 1463370 (0.0011) [2023-12-27 02:01:59,524][105620] Updated weights for policy 1, policy_version 1463380 (0.0006) [2023-12-27 02:01:59,579][105692] Updated weights for policy 0, policy_version 1461080 (0.0006) [2023-12-27 02:01:59,582][105620] Updated weights for policy 1, policy_version 1463390 (0.0006) [2023-12-27 02:01:59,635][105692] Updated weights for policy 0, policy_version 1461090 (0.0005) [2023-12-27 02:01:59,688][105692] Updated weights for policy 0, policy_version 1461101 (0.0009) [2023-12-27 02:02:00,225][105620] Updated weights for policy 1, policy_version 1463400 (0.0005) [2023-12-27 02:02:00,291][105620] Updated weights for policy 1, policy_version 1463410 (0.0006) [2023-12-27 02:02:00,356][105620] Updated weights for policy 1, policy_version 1463420 (0.0011) [2023-12-27 02:02:00,399][105692] Updated weights for policy 0, policy_version 1461111 (0.0007) [2023-12-27 02:02:00,457][105692] Updated weights for policy 0, policy_version 1461121 (0.0006) [2023-12-27 02:02:00,505][105692] Updated weights for policy 0, policy_version 1461131 (0.0008) [2023-12-27 02:02:00,956][105620] Updated weights for policy 1, policy_version 1463430 (0.0009) [2023-12-27 02:02:01,011][105620] Updated weights for policy 1, policy_version 1463440 (0.0010) [2023-12-27 02:02:01,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 748789760. Throughput: 0: 9620.2, 1: 9714.2. Samples: 748765632. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:02:01,063][104569] Avg episode reward: [(0, '8893.988'), (1, '8804.980')] [2023-12-27 02:02:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001461136_374104064.pth... [2023-12-27 02:02:01,070][105620] Updated weights for policy 1, policy_version 1463450 (0.0011) [2023-12-27 02:02:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001460048_373825536.pth [2023-12-27 02:02:01,094][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001463456_374693888.pth... [2023-12-27 02:02:01,096][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001462304_374398976.pth [2023-12-27 02:02:01,278][105692] Updated weights for policy 0, policy_version 1461141 (0.0009) [2023-12-27 02:02:01,331][105692] Updated weights for policy 0, policy_version 1461151 (0.0008) [2023-12-27 02:02:01,397][105692] Updated weights for policy 0, policy_version 1461161 (0.0009) [2023-12-27 02:02:01,800][105620] Updated weights for policy 1, policy_version 1463460 (0.0011) [2023-12-27 02:02:01,856][105620] Updated weights for policy 1, policy_version 1463470 (0.0011) [2023-12-27 02:02:01,907][105620] Updated weights for policy 1, policy_version 1463480 (0.0010) [2023-12-27 02:02:02,087][105692] Updated weights for policy 0, policy_version 1461171 (0.0007) [2023-12-27 02:02:02,148][105692] Updated weights for policy 0, policy_version 1461181 (0.0009) [2023-12-27 02:02:02,200][105692] Updated weights for policy 0, policy_version 1461191 (0.0010) [2023-12-27 02:02:02,525][105620] Updated weights for policy 1, policy_version 1463490 (0.0008) [2023-12-27 02:02:02,579][105620] Updated weights for policy 1, policy_version 1463500 (0.0005) [2023-12-27 02:02:02,633][105620] Updated weights for policy 1, policy_version 1463510 (0.0007) [2023-12-27 02:02:02,686][105620] Updated weights for policy 1, policy_version 1463520 (0.0006) [2023-12-27 02:02:03,072][105692] Updated weights for policy 0, policy_version 1461201 (0.0009) [2023-12-27 02:02:03,133][105692] Updated weights for policy 0, policy_version 1461211 (0.0008) [2023-12-27 02:02:03,181][105692] Updated weights for policy 0, policy_version 1461221 (0.0008) [2023-12-27 02:02:03,228][105692] Updated weights for policy 0, policy_version 1461231 (0.0008) [2023-12-27 02:02:03,312][105620] Updated weights for policy 1, policy_version 1463530 (0.0010) [2023-12-27 02:02:03,359][105620] Updated weights for policy 1, policy_version 1463540 (0.0010) [2023-12-27 02:02:03,414][105620] Updated weights for policy 1, policy_version 1463550 (0.0010) [2023-12-27 02:02:03,971][105692] Updated weights for policy 0, policy_version 1461241 (0.0008) [2023-12-27 02:02:04,024][105692] Updated weights for policy 0, policy_version 1461251 (0.0008) [2023-12-27 02:02:04,074][105692] Updated weights for policy 0, policy_version 1461261 (0.0008) [2023-12-27 02:02:04,143][105620] Updated weights for policy 1, policy_version 1463560 (0.0010) [2023-12-27 02:02:04,199][105620] Updated weights for policy 1, policy_version 1463570 (0.0010) [2023-12-27 02:02:04,267][105620] Updated weights for policy 1, policy_version 1463580 (0.0010) [2023-12-27 02:02:04,682][105692] Updated weights for policy 0, policy_version 1461271 (0.0006) [2023-12-27 02:02:04,728][105692] Updated weights for policy 0, policy_version 1461281 (0.0005) [2023-12-27 02:02:04,783][105692] Updated weights for policy 0, policy_version 1461291 (0.0006) [2023-12-27 02:02:04,972][105620] Updated weights for policy 1, policy_version 1463590 (0.0007) [2023-12-27 02:02:05,030][105620] Updated weights for policy 1, policy_version 1463600 (0.0006) [2023-12-27 02:02:05,091][105620] Updated weights for policy 1, policy_version 1463610 (0.0006) [2023-12-27 02:02:05,596][105692] Updated weights for policy 0, policy_version 1461301 (0.0008) [2023-12-27 02:02:05,620][105620] Updated weights for policy 1, policy_version 1463620 (0.0007) [2023-12-27 02:02:05,648][105692] Updated weights for policy 0, policy_version 1461311 (0.0005) [2023-12-27 02:02:05,683][105620] Updated weights for policy 1, policy_version 1463630 (0.0008) [2023-12-27 02:02:05,706][105692] Updated weights for policy 0, policy_version 1461321 (0.0005) [2023-12-27 02:02:05,742][105620] Updated weights for policy 1, policy_version 1463640 (0.0010) [2023-12-27 02:02:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 748896256. Throughput: 0: 9560.6, 1: 9824.3. Samples: 748883264. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:02:06,063][104569] Avg episode reward: [(0, '8984.350'), (1, '8895.010')] [2023-12-27 02:02:06,245][105692] Updated weights for policy 0, policy_version 1461331 (0.0006) [2023-12-27 02:02:06,312][105692] Updated weights for policy 0, policy_version 1461341 (0.0007) [2023-12-27 02:02:06,370][105620] Updated weights for policy 1, policy_version 1463650 (0.0009) [2023-12-27 02:02:06,375][105692] Updated weights for policy 0, policy_version 1461351 (0.0009) [2023-12-27 02:02:06,432][105620] Updated weights for policy 1, policy_version 1463660 (0.0006) [2023-12-27 02:02:06,487][105620] Updated weights for policy 1, policy_version 1463670 (0.0009) [2023-12-27 02:02:06,558][105620] Updated weights for policy 1, policy_version 1463680 (0.0011) [2023-12-27 02:02:07,027][105692] Updated weights for policy 0, policy_version 1461361 (0.0007) [2023-12-27 02:02:07,084][105692] Updated weights for policy 0, policy_version 1461371 (0.0005) [2023-12-27 02:02:07,151][105692] Updated weights for policy 0, policy_version 1461381 (0.0006) [2023-12-27 02:02:07,206][105692] Updated weights for policy 0, policy_version 1461391 (0.0006) [2023-12-27 02:02:07,276][105620] Updated weights for policy 1, policy_version 1463690 (0.0011) [2023-12-27 02:02:07,342][105620] Updated weights for policy 1, policy_version 1463700 (0.0011) [2023-12-27 02:02:07,403][105620] Updated weights for policy 1, policy_version 1463710 (0.0011) [2023-12-27 02:02:07,809][105692] Updated weights for policy 0, policy_version 1461401 (0.0005) [2023-12-27 02:02:07,860][105692] Updated weights for policy 0, policy_version 1461411 (0.0005) [2023-12-27 02:02:07,908][105692] Updated weights for policy 0, policy_version 1461421 (0.0005) [2023-12-27 02:02:08,114][105620] Updated weights for policy 1, policy_version 1463720 (0.0011) [2023-12-27 02:02:08,170][105620] Updated weights for policy 1, policy_version 1463730 (0.0011) [2023-12-27 02:02:08,225][105620] Updated weights for policy 1, policy_version 1463740 (0.0010) [2023-12-27 02:02:08,511][105692] Updated weights for policy 0, policy_version 1461431 (0.0009) [2023-12-27 02:02:08,556][105692] Updated weights for policy 0, policy_version 1461441 (0.0010) [2023-12-27 02:02:08,612][105692] Updated weights for policy 0, policy_version 1461451 (0.0010) [2023-12-27 02:02:08,996][105620] Updated weights for policy 1, policy_version 1463750 (0.0011) [2023-12-27 02:02:09,059][105620] Updated weights for policy 1, policy_version 1463760 (0.0011) [2023-12-27 02:02:09,128][105620] Updated weights for policy 1, policy_version 1463770 (0.0011) [2023-12-27 02:02:09,380][105692] Updated weights for policy 0, policy_version 1461461 (0.0010) [2023-12-27 02:02:09,448][105692] Updated weights for policy 0, policy_version 1461471 (0.0007) [2023-12-27 02:02:09,519][105692] Updated weights for policy 0, policy_version 1461481 (0.0005) [2023-12-27 02:02:09,894][105620] Updated weights for policy 1, policy_version 1463780 (0.0010) [2023-12-27 02:02:09,962][105620] Updated weights for policy 1, policy_version 1463790 (0.0009) [2023-12-27 02:02:10,031][105620] Updated weights for policy 1, policy_version 1463800 (0.0008) [2023-12-27 02:02:10,171][105692] Updated weights for policy 0, policy_version 1461491 (0.0005) [2023-12-27 02:02:10,226][105692] Updated weights for policy 0, policy_version 1461501 (0.0008) [2023-12-27 02:02:10,291][105692] Updated weights for policy 0, policy_version 1461511 (0.0012) [2023-12-27 02:02:10,673][105620] Updated weights for policy 1, policy_version 1463810 (0.0009) [2023-12-27 02:02:10,732][105620] Updated weights for policy 1, policy_version 1463820 (0.0006) [2023-12-27 02:02:10,800][105620] Updated weights for policy 1, policy_version 1463830 (0.0008) [2023-12-27 02:02:10,853][105620] Updated weights for policy 1, policy_version 1463840 (0.0008) [2023-12-27 02:02:10,989][105692] Updated weights for policy 0, policy_version 1461521 (0.0011) [2023-12-27 02:02:11,052][105692] Updated weights for policy 0, policy_version 1461531 (0.0011) [2023-12-27 02:02:11,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 748994560. Throughput: 0: 9673.8, 1: 9836.7. Samples: 749005616. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:02:11,063][104569] Avg episode reward: [(0, '8617.628'), (1, '8807.751')] [2023-12-27 02:02:11,112][105692] Updated weights for policy 0, policy_version 1461541 (0.0011) [2023-12-27 02:02:11,174][105692] Updated weights for policy 0, policy_version 1461551 (0.0011) [2023-12-27 02:02:11,605][105620] Updated weights for policy 1, policy_version 1463850 (0.0008) [2023-12-27 02:02:11,670][105620] Updated weights for policy 1, policy_version 1463860 (0.0008) [2023-12-27 02:02:11,737][105620] Updated weights for policy 1, policy_version 1463870 (0.0008) [2023-12-27 02:02:11,966][105692] Updated weights for policy 0, policy_version 1461561 (0.0009) [2023-12-27 02:02:12,025][105692] Updated weights for policy 0, policy_version 1461571 (0.0009) [2023-12-27 02:02:12,083][105692] Updated weights for policy 0, policy_version 1461581 (0.0009) [2023-12-27 02:02:12,491][105620] Updated weights for policy 1, policy_version 1463880 (0.0010) [2023-12-27 02:02:12,560][105620] Updated weights for policy 1, policy_version 1463890 (0.0006) [2023-12-27 02:02:12,628][105620] Updated weights for policy 1, policy_version 1463900 (0.0008) [2023-12-27 02:02:12,798][105692] Updated weights for policy 0, policy_version 1461591 (0.0009) [2023-12-27 02:02:12,848][105692] Updated weights for policy 0, policy_version 1461601 (0.0009) [2023-12-27 02:02:12,894][105692] Updated weights for policy 0, policy_version 1461611 (0.0009) [2023-12-27 02:02:13,237][105620] Updated weights for policy 1, policy_version 1463910 (0.0005) [2023-12-27 02:02:13,298][105620] Updated weights for policy 1, policy_version 1463920 (0.0005) [2023-12-27 02:02:13,354][105620] Updated weights for policy 1, policy_version 1463930 (0.0005) [2023-12-27 02:02:13,723][105692] Updated weights for policy 0, policy_version 1461621 (0.0007) [2023-12-27 02:02:13,779][105692] Updated weights for policy 0, policy_version 1461631 (0.0005) [2023-12-27 02:02:13,832][105692] Updated weights for policy 0, policy_version 1461641 (0.0009) [2023-12-27 02:02:13,906][105620] Updated weights for policy 1, policy_version 1463940 (0.0006) [2023-12-27 02:02:13,974][105620] Updated weights for policy 1, policy_version 1463950 (0.0010) [2023-12-27 02:02:14,036][105620] Updated weights for policy 1, policy_version 1463960 (0.0010) [2023-12-27 02:02:14,433][105692] Updated weights for policy 0, policy_version 1461651 (0.0009) [2023-12-27 02:02:14,494][105692] Updated weights for policy 0, policy_version 1461661 (0.0006) [2023-12-27 02:02:14,555][105692] Updated weights for policy 0, policy_version 1461671 (0.0005) [2023-12-27 02:02:14,684][105620] Updated weights for policy 1, policy_version 1463970 (0.0010) [2023-12-27 02:02:14,742][105620] Updated weights for policy 1, policy_version 1463980 (0.0010) [2023-12-27 02:02:14,809][105620] Updated weights for policy 1, policy_version 1463990 (0.0011) [2023-12-27 02:02:14,871][105620] Updated weights for policy 1, policy_version 1464000 (0.0011) [2023-12-27 02:02:15,220][105692] Updated weights for policy 0, policy_version 1461681 (0.0006) [2023-12-27 02:02:15,277][105692] Updated weights for policy 0, policy_version 1461691 (0.0009) [2023-12-27 02:02:15,336][105692] Updated weights for policy 0, policy_version 1461701 (0.0010) [2023-12-27 02:02:15,395][105692] Updated weights for policy 0, policy_version 1461711 (0.0006) [2023-12-27 02:02:15,558][105620] Updated weights for policy 1, policy_version 1464010 (0.0009) [2023-12-27 02:02:15,615][105620] Updated weights for policy 1, policy_version 1464020 (0.0008) [2023-12-27 02:02:15,677][105620] Updated weights for policy 1, policy_version 1464030 (0.0009) [2023-12-27 02:02:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 749092864. Throughput: 0: 9584.7, 1: 9834.7. Samples: 749063528. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:02:16,063][104569] Avg episode reward: [(0, '8160.145'), (1, '8719.020')] [2023-12-27 02:02:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001464032_374841344.pth... [2023-12-27 02:02:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001462880_374546432.pth [2023-12-27 02:02:16,120][105692] Updated weights for policy 0, policy_version 1461721 (0.0009) [2023-12-27 02:02:16,186][105692] Updated weights for policy 0, policy_version 1461731 (0.0008) [2023-12-27 02:02:16,253][105692] Updated weights for policy 0, policy_version 1461741 (0.0008) [2023-12-27 02:02:16,269][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001461744_374259712.pth... [2023-12-27 02:02:16,274][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001460592_373964800.pth [2023-12-27 02:02:16,420][105620] Updated weights for policy 1, policy_version 1464040 (0.0008) [2023-12-27 02:02:16,481][105620] Updated weights for policy 1, policy_version 1464050 (0.0009) [2023-12-27 02:02:16,538][105620] Updated weights for policy 1, policy_version 1464060 (0.0009) [2023-12-27 02:02:16,968][105692] Updated weights for policy 0, policy_version 1461751 (0.0009) [2023-12-27 02:02:17,026][105692] Updated weights for policy 0, policy_version 1461761 (0.0009) [2023-12-27 02:02:17,087][105692] Updated weights for policy 0, policy_version 1461771 (0.0009) [2023-12-27 02:02:17,260][105620] Updated weights for policy 1, policy_version 1464070 (0.0007) [2023-12-27 02:02:17,320][105620] Updated weights for policy 1, policy_version 1464080 (0.0007) [2023-12-27 02:02:17,379][105620] Updated weights for policy 1, policy_version 1464090 (0.0008) [2023-12-27 02:02:17,801][105692] Updated weights for policy 0, policy_version 1461781 (0.0008) [2023-12-27 02:02:17,856][105692] Updated weights for policy 0, policy_version 1461791 (0.0010) [2023-12-27 02:02:17,905][105692] Updated weights for policy 0, policy_version 1461801 (0.0009) [2023-12-27 02:02:17,970][105620] Updated weights for policy 1, policy_version 1464100 (0.0005) [2023-12-27 02:02:18,041][105620] Updated weights for policy 1, policy_version 1464110 (0.0006) [2023-12-27 02:02:18,099][105620] Updated weights for policy 1, policy_version 1464120 (0.0006) [2023-12-27 02:02:18,623][105692] Updated weights for policy 0, policy_version 1461811 (0.0008) [2023-12-27 02:02:18,687][105692] Updated weights for policy 0, policy_version 1461821 (0.0011) [2023-12-27 02:02:18,739][105692] Updated weights for policy 0, policy_version 1461831 (0.0009) [2023-12-27 02:02:18,759][105620] Updated weights for policy 1, policy_version 1464130 (0.0006) [2023-12-27 02:02:18,820][105620] Updated weights for policy 1, policy_version 1464140 (0.0007) [2023-12-27 02:02:18,880][105620] Updated weights for policy 1, policy_version 1464150 (0.0008) [2023-12-27 02:02:18,935][105620] Updated weights for policy 1, policy_version 1464160 (0.0009) [2023-12-27 02:02:19,483][105692] Updated weights for policy 0, policy_version 1461841 (0.0008) [2023-12-27 02:02:19,548][105692] Updated weights for policy 0, policy_version 1461851 (0.0007) [2023-12-27 02:02:19,608][105692] Updated weights for policy 0, policy_version 1461861 (0.0006) [2023-12-27 02:02:19,669][105692] Updated weights for policy 0, policy_version 1461871 (0.0009) [2023-12-27 02:02:19,719][105620] Updated weights for policy 1, policy_version 1464170 (0.0009) [2023-12-27 02:02:19,786][105620] Updated weights for policy 1, policy_version 1464180 (0.0009) [2023-12-27 02:02:19,852][105620] Updated weights for policy 1, policy_version 1464190 (0.0008) [2023-12-27 02:02:20,349][105692] Updated weights for policy 0, policy_version 1461881 (0.0009) [2023-12-27 02:02:20,413][105692] Updated weights for policy 0, policy_version 1461891 (0.0009) [2023-12-27 02:02:20,469][105692] Updated weights for policy 0, policy_version 1461901 (0.0009) [2023-12-27 02:02:20,636][105620] Updated weights for policy 1, policy_version 1464200 (0.0008) [2023-12-27 02:02:20,697][105620] Updated weights for policy 1, policy_version 1464210 (0.0008) [2023-12-27 02:02:20,762][105620] Updated weights for policy 1, policy_version 1464220 (0.0009) [2023-12-27 02:02:21,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 749191168. Throughput: 0: 9648.7, 1: 9853.6. Samples: 749181844. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:02:21,062][104569] Avg episode reward: [(0, '8337.892'), (1, '8898.048')] [2023-12-27 02:02:21,314][105692] Updated weights for policy 0, policy_version 1461911 (0.0010) [2023-12-27 02:02:21,380][105692] Updated weights for policy 0, policy_version 1461921 (0.0009) [2023-12-27 02:02:21,443][105692] Updated weights for policy 0, policy_version 1461931 (0.0010) [2023-12-27 02:02:21,571][105620] Updated weights for policy 1, policy_version 1464230 (0.0008) [2023-12-27 02:02:21,643][105620] Updated weights for policy 1, policy_version 1464240 (0.0007) [2023-12-27 02:02:21,702][105620] Updated weights for policy 1, policy_version 1464250 (0.0008) [2023-12-27 02:02:22,205][105692] Updated weights for policy 0, policy_version 1461941 (0.0008) [2023-12-27 02:02:22,271][105692] Updated weights for policy 0, policy_version 1461951 (0.0007) [2023-12-27 02:02:22,334][105692] Updated weights for policy 0, policy_version 1461961 (0.0008) [2023-12-27 02:02:22,454][105620] Updated weights for policy 1, policy_version 1464260 (0.0009) [2023-12-27 02:02:22,513][105620] Updated weights for policy 1, policy_version 1464270 (0.0008) [2023-12-27 02:02:22,579][105620] Updated weights for policy 1, policy_version 1464280 (0.0009) [2023-12-27 02:02:23,067][105692] Updated weights for policy 0, policy_version 1461971 (0.0012) [2023-12-27 02:02:23,122][105692] Updated weights for policy 0, policy_version 1461981 (0.0009) [2023-12-27 02:02:23,184][105692] Updated weights for policy 0, policy_version 1461991 (0.0009) [2023-12-27 02:02:23,301][105620] Updated weights for policy 1, policy_version 1464290 (0.0008) [2023-12-27 02:02:23,353][105620] Updated weights for policy 1, policy_version 1464300 (0.0005) [2023-12-27 02:02:23,405][105620] Updated weights for policy 1, policy_version 1464310 (0.0005) [2023-12-27 02:02:23,462][105620] Updated weights for policy 1, policy_version 1464320 (0.0005) [2023-12-27 02:02:23,873][105692] Updated weights for policy 0, policy_version 1462001 (0.0009) [2023-12-27 02:02:23,921][105692] Updated weights for policy 0, policy_version 1462011 (0.0005) [2023-12-27 02:02:23,990][105692] Updated weights for policy 0, policy_version 1462021 (0.0006) [2023-12-27 02:02:24,053][105692] Updated weights for policy 0, policy_version 1462031 (0.0006) [2023-12-27 02:02:24,265][105620] Updated weights for policy 1, policy_version 1464330 (0.0008) [2023-12-27 02:02:24,329][105620] Updated weights for policy 1, policy_version 1464340 (0.0009) [2023-12-27 02:02:24,389][105620] Updated weights for policy 1, policy_version 1464350 (0.0009) [2023-12-27 02:02:24,669][105692] Updated weights for policy 0, policy_version 1462041 (0.0009) [2023-12-27 02:02:24,724][105692] Updated weights for policy 0, policy_version 1462051 (0.0009) [2023-12-27 02:02:24,775][105692] Updated weights for policy 0, policy_version 1462061 (0.0007) [2023-12-27 02:02:25,227][105620] Updated weights for policy 1, policy_version 1464360 (0.0008) [2023-12-27 02:02:25,274][105620] Updated weights for policy 1, policy_version 1464370 (0.0008) [2023-12-27 02:02:25,320][105620] Updated weights for policy 1, policy_version 1464380 (0.0009) [2023-12-27 02:02:25,380][105692] Updated weights for policy 0, policy_version 1462071 (0.0007) [2023-12-27 02:02:25,435][105692] Updated weights for policy 0, policy_version 1462081 (0.0005) [2023-12-27 02:02:25,487][105692] Updated weights for policy 0, policy_version 1462091 (0.0009) [2023-12-27 02:02:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 749281280. Throughput: 0: 9678.3, 1: 9809.8. Samples: 749294556. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:02:26,063][104569] Avg episode reward: [(0, '8617.309'), (1, '8985.433')] [2023-12-27 02:02:26,096][105620] Updated weights for policy 1, policy_version 1464390 (0.0009) [2023-12-27 02:02:26,149][105620] Updated weights for policy 1, policy_version 1464400 (0.0009) [2023-12-27 02:02:26,211][105692] Updated weights for policy 0, policy_version 1462101 (0.0008) [2023-12-27 02:02:26,214][105620] Updated weights for policy 1, policy_version 1464410 (0.0008) [2023-12-27 02:02:26,259][105692] Updated weights for policy 0, policy_version 1462111 (0.0008) [2023-12-27 02:02:26,305][105692] Updated weights for policy 0, policy_version 1462121 (0.0007) [2023-12-27 02:02:27,004][105620] Updated weights for policy 1, policy_version 1464420 (0.0007) [2023-12-27 02:02:27,039][105692] Updated weights for policy 0, policy_version 1462131 (0.0006) [2023-12-27 02:02:27,057][105620] Updated weights for policy 1, policy_version 1464430 (0.0006) [2023-12-27 02:02:27,100][105620] Updated weights for policy 1, policy_version 1464440 (0.0008) [2023-12-27 02:02:27,102][105692] Updated weights for policy 0, policy_version 1462141 (0.0008) [2023-12-27 02:02:27,155][105692] Updated weights for policy 0, policy_version 1462151 (0.0008) [2023-12-27 02:02:27,841][105692] Updated weights for policy 0, policy_version 1462161 (0.0009) [2023-12-27 02:02:27,877][105620] Updated weights for policy 1, policy_version 1464450 (0.0006) [2023-12-27 02:02:27,892][105692] Updated weights for policy 0, policy_version 1462171 (0.0008) [2023-12-27 02:02:27,929][105620] Updated weights for policy 1, policy_version 1464460 (0.0007) [2023-12-27 02:02:27,955][105692] Updated weights for policy 0, policy_version 1462181 (0.0008) [2023-12-27 02:02:27,987][105620] Updated weights for policy 1, policy_version 1464470 (0.0007) [2023-12-27 02:02:28,013][105692] Updated weights for policy 0, policy_version 1462191 (0.0005) [2023-12-27 02:02:28,038][105620] Updated weights for policy 1, policy_version 1464480 (0.0008) [2023-12-27 02:02:28,728][105692] Updated weights for policy 0, policy_version 1462201 (0.0006) [2023-12-27 02:02:28,786][105692] Updated weights for policy 0, policy_version 1462211 (0.0005) [2023-12-27 02:02:28,793][105620] Updated weights for policy 1, policy_version 1464490 (0.0008) [2023-12-27 02:02:28,850][105692] Updated weights for policy 0, policy_version 1462221 (0.0007) [2023-12-27 02:02:28,863][105620] Updated weights for policy 1, policy_version 1464500 (0.0006) [2023-12-27 02:02:28,925][105620] Updated weights for policy 1, policy_version 1464510 (0.0007) [2023-12-27 02:02:29,544][105692] Updated weights for policy 0, policy_version 1462231 (0.0007) [2023-12-27 02:02:29,605][105692] Updated weights for policy 0, policy_version 1462241 (0.0006) [2023-12-27 02:02:29,639][105620] Updated weights for policy 1, policy_version 1464520 (0.0011) [2023-12-27 02:02:29,668][105692] Updated weights for policy 0, policy_version 1462251 (0.0006) [2023-12-27 02:02:29,696][105620] Updated weights for policy 1, policy_version 1464530 (0.0011) [2023-12-27 02:02:29,753][105620] Updated weights for policy 1, policy_version 1464540 (0.0011) [2023-12-27 02:02:30,278][105692] Updated weights for policy 0, policy_version 1462261 (0.0005) [2023-12-27 02:02:30,343][105692] Updated weights for policy 0, policy_version 1462271 (0.0008) [2023-12-27 02:02:30,412][105692] Updated weights for policy 0, policy_version 1462281 (0.0009) [2023-12-27 02:02:30,413][105620] Updated weights for policy 1, policy_version 1464550 (0.0009) [2023-12-27 02:02:30,462][105620] Updated weights for policy 1, policy_version 1464560 (0.0007) [2023-12-27 02:02:30,514][105620] Updated weights for policy 1, policy_version 1464570 (0.0006) [2023-12-27 02:02:31,017][105692] Updated weights for policy 0, policy_version 1462291 (0.0007) [2023-12-27 02:02:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 749379584. Throughput: 0: 9716.2, 1: 9739.9. Samples: 749351860. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:02:31,062][104569] Avg episode reward: [(0, '8710.733'), (1, '8710.512')] [2023-12-27 02:02:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001464576_374980608.pth... [2023-12-27 02:02:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001463456_374693888.pth [2023-12-27 02:02:31,080][105692] Updated weights for policy 0, policy_version 1462301 (0.0010) [2023-12-27 02:02:31,141][105692] Updated weights for policy 0, policy_version 1462311 (0.0010) [2023-12-27 02:02:31,187][105620] Updated weights for policy 1, policy_version 1464580 (0.0008) [2023-12-27 02:02:31,201][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001462320_374407168.pth... [2023-12-27 02:02:31,205][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001461136_374104064.pth [2023-12-27 02:02:31,248][105620] Updated weights for policy 1, policy_version 1464590 (0.0008) [2023-12-27 02:02:31,307][105620] Updated weights for policy 1, policy_version 1464600 (0.0006) [2023-12-27 02:02:31,867][105692] Updated weights for policy 0, policy_version 1462321 (0.0009) [2023-12-27 02:02:31,932][105692] Updated weights for policy 0, policy_version 1462331 (0.0007) [2023-12-27 02:02:31,984][105692] Updated weights for policy 0, policy_version 1462341 (0.0005) [2023-12-27 02:02:32,043][105692] Updated weights for policy 0, policy_version 1462351 (0.0008) [2023-12-27 02:02:32,049][105620] Updated weights for policy 1, policy_version 1464610 (0.0008) [2023-12-27 02:02:32,101][105620] Updated weights for policy 1, policy_version 1464620 (0.0009) [2023-12-27 02:02:32,152][105620] Updated weights for policy 1, policy_version 1464630 (0.0010) [2023-12-27 02:02:32,214][105620] Updated weights for policy 1, policy_version 1464640 (0.0011) [2023-12-27 02:02:32,793][105692] Updated weights for policy 0, policy_version 1462361 (0.0010) [2023-12-27 02:02:32,850][105692] Updated weights for policy 0, policy_version 1462371 (0.0009) [2023-12-27 02:02:32,854][105620] Updated weights for policy 1, policy_version 1464650 (0.0006) [2023-12-27 02:02:32,906][105620] Updated weights for policy 1, policy_version 1464660 (0.0005) [2023-12-27 02:02:32,909][105692] Updated weights for policy 0, policy_version 1462381 (0.0009) [2023-12-27 02:02:32,961][105620] Updated weights for policy 1, policy_version 1464670 (0.0005) [2023-12-27 02:02:33,507][105620] Updated weights for policy 1, policy_version 1464680 (0.0009) [2023-12-27 02:02:33,568][105620] Updated weights for policy 1, policy_version 1464690 (0.0010) [2023-12-27 02:02:33,625][105620] Updated weights for policy 1, policy_version 1464700 (0.0010) [2023-12-27 02:02:33,753][105692] Updated weights for policy 0, policy_version 1462391 (0.0008) [2023-12-27 02:02:33,797][105692] Updated weights for policy 0, policy_version 1462401 (0.0008) [2023-12-27 02:02:33,848][105692] Updated weights for policy 0, policy_version 1462411 (0.0007) [2023-12-27 02:02:34,263][105620] Updated weights for policy 1, policy_version 1464710 (0.0010) [2023-12-27 02:02:34,333][105620] Updated weights for policy 1, policy_version 1464720 (0.0011) [2023-12-27 02:02:34,395][105620] Updated weights for policy 1, policy_version 1464730 (0.0011) [2023-12-27 02:02:34,632][105692] Updated weights for policy 0, policy_version 1462421 (0.0007) [2023-12-27 02:02:34,693][105692] Updated weights for policy 0, policy_version 1462431 (0.0008) [2023-12-27 02:02:34,750][105692] Updated weights for policy 0, policy_version 1462441 (0.0008) [2023-12-27 02:02:35,116][105620] Updated weights for policy 1, policy_version 1464740 (0.0010) [2023-12-27 02:02:35,185][105620] Updated weights for policy 1, policy_version 1464750 (0.0011) [2023-12-27 02:02:35,243][105620] Updated weights for policy 1, policy_version 1464760 (0.0010) [2023-12-27 02:02:35,362][105692] Updated weights for policy 0, policy_version 1462451 (0.0008) [2023-12-27 02:02:35,406][105692] Updated weights for policy 0, policy_version 1462461 (0.0008) [2023-12-27 02:02:35,450][105692] Updated weights for policy 0, policy_version 1462471 (0.0007) [2023-12-27 02:02:35,976][105620] Updated weights for policy 1, policy_version 1464770 (0.0010) [2023-12-27 02:02:36,037][105620] Updated weights for policy 1, policy_version 1464780 (0.0010) [2023-12-27 02:02:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 749477888. Throughput: 0: 9762.5, 1: 9776.5. Samples: 749472444. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-12-27 02:02:36,062][104569] Avg episode reward: [(0, '8261.004'), (1, '8894.663')] [2023-12-27 02:02:36,092][105620] Updated weights for policy 1, policy_version 1464790 (0.0010) [2023-12-27 02:02:36,154][105620] Updated weights for policy 1, policy_version 1464800 (0.0010) [2023-12-27 02:02:36,232][105692] Updated weights for policy 0, policy_version 1462481 (0.0008) [2023-12-27 02:02:36,288][105692] Updated weights for policy 0, policy_version 1462491 (0.0008) [2023-12-27 02:02:36,349][105692] Updated weights for policy 0, policy_version 1462501 (0.0008) [2023-12-27 02:02:36,405][105692] Updated weights for policy 0, policy_version 1462511 (0.0009) [2023-12-27 02:02:36,912][105620] Updated weights for policy 1, policy_version 1464810 (0.0010) [2023-12-27 02:02:36,967][105620] Updated weights for policy 1, policy_version 1464820 (0.0010) [2023-12-27 02:02:37,015][105620] Updated weights for policy 1, policy_version 1464830 (0.0010) [2023-12-27 02:02:37,180][105692] Updated weights for policy 0, policy_version 1462521 (0.0008) [2023-12-27 02:02:37,225][105692] Updated weights for policy 0, policy_version 1462531 (0.0007) [2023-12-27 02:02:37,276][105692] Updated weights for policy 0, policy_version 1462541 (0.0006) [2023-12-27 02:02:37,727][105620] Updated weights for policy 1, policy_version 1464840 (0.0007) [2023-12-27 02:02:37,783][105620] Updated weights for policy 1, policy_version 1464850 (0.0006) [2023-12-27 02:02:37,838][105620] Updated weights for policy 1, policy_version 1464860 (0.0006) [2023-12-27 02:02:38,036][105692] Updated weights for policy 0, policy_version 1462551 (0.0007) [2023-12-27 02:02:38,104][105692] Updated weights for policy 0, policy_version 1462561 (0.0008) [2023-12-27 02:02:38,157][105692] Updated weights for policy 0, policy_version 1462571 (0.0010) [2023-12-27 02:02:38,503][105620] Updated weights for policy 1, policy_version 1464870 (0.0006) [2023-12-27 02:02:38,560][105620] Updated weights for policy 1, policy_version 1464880 (0.0009) [2023-12-27 02:02:38,622][105620] Updated weights for policy 1, policy_version 1464890 (0.0009) [2023-12-27 02:02:38,846][105692] Updated weights for policy 0, policy_version 1462581 (0.0009) [2023-12-27 02:02:38,901][105692] Updated weights for policy 0, policy_version 1462591 (0.0009) [2023-12-27 02:02:38,955][105692] Updated weights for policy 0, policy_version 1462601 (0.0009) [2023-12-27 02:02:39,358][105620] Updated weights for policy 1, policy_version 1464900 (0.0009) [2023-12-27 02:02:39,429][105620] Updated weights for policy 1, policy_version 1464910 (0.0008) [2023-12-27 02:02:39,495][105620] Updated weights for policy 1, policy_version 1464920 (0.0008) [2023-12-27 02:02:39,717][105692] Updated weights for policy 0, policy_version 1462611 (0.0008) [2023-12-27 02:02:39,775][105692] Updated weights for policy 0, policy_version 1462621 (0.0009) [2023-12-27 02:02:39,840][105692] Updated weights for policy 0, policy_version 1462631 (0.0009) [2023-12-27 02:02:40,250][105620] Updated weights for policy 1, policy_version 1464930 (0.0009) [2023-12-27 02:02:40,308][105620] Updated weights for policy 1, policy_version 1464940 (0.0009) [2023-12-27 02:02:40,365][105620] Updated weights for policy 1, policy_version 1464950 (0.0009) [2023-12-27 02:02:40,411][105620] Updated weights for policy 1, policy_version 1464960 (0.0008) [2023-12-27 02:02:40,560][105692] Updated weights for policy 0, policy_version 1462641 (0.0006) [2023-12-27 02:02:40,627][105692] Updated weights for policy 0, policy_version 1462651 (0.0009) [2023-12-27 02:02:40,693][105692] Updated weights for policy 0, policy_version 1462661 (0.0009) [2023-12-27 02:02:40,758][105692] Updated weights for policy 0, policy_version 1462671 (0.0009) [2023-12-27 02:02:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 749576192. Throughput: 0: 9762.5, 1: 9767.6. Samples: 749586912. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:02:41,062][104569] Avg episode reward: [(0, '8075.314'), (1, '8989.795')] [2023-12-27 02:02:41,202][105620] Updated weights for policy 1, policy_version 1464970 (0.0007) [2023-12-27 02:02:41,264][105620] Updated weights for policy 1, policy_version 1464980 (0.0010) [2023-12-27 02:02:41,331][105620] Updated weights for policy 1, policy_version 1464990 (0.0008) [2023-12-27 02:02:41,514][105692] Updated weights for policy 0, policy_version 1462681 (0.0009) [2023-12-27 02:02:41,563][105692] Updated weights for policy 0, policy_version 1462691 (0.0008) [2023-12-27 02:02:41,619][105692] Updated weights for policy 0, policy_version 1462701 (0.0009) [2023-12-27 02:02:42,131][105620] Updated weights for policy 1, policy_version 1465000 (0.0009) [2023-12-27 02:02:42,185][105620] Updated weights for policy 1, policy_version 1465010 (0.0009) [2023-12-27 02:02:42,247][105620] Updated weights for policy 1, policy_version 1465020 (0.0009) [2023-12-27 02:02:42,323][105692] Updated weights for policy 0, policy_version 1462711 (0.0009) [2023-12-27 02:02:42,393][105692] Updated weights for policy 0, policy_version 1462721 (0.0009) [2023-12-27 02:02:42,447][105692] Updated weights for policy 0, policy_version 1462731 (0.0009) [2023-12-27 02:02:42,887][105620] Updated weights for policy 1, policy_version 1465030 (0.0008) [2023-12-27 02:02:42,933][105620] Updated weights for policy 1, policy_version 1465040 (0.0005) [2023-12-27 02:02:42,993][105620] Updated weights for policy 1, policy_version 1465050 (0.0005) [2023-12-27 02:02:43,142][105692] Updated weights for policy 0, policy_version 1462741 (0.0007) [2023-12-27 02:02:43,200][105692] Updated weights for policy 0, policy_version 1462751 (0.0005) [2023-12-27 02:02:43,267][105692] Updated weights for policy 0, policy_version 1462761 (0.0007) [2023-12-27 02:02:43,755][105620] Updated weights for policy 1, policy_version 1465060 (0.0007) [2023-12-27 02:02:43,783][105692] Updated weights for policy 0, policy_version 1462771 (0.0007) [2023-12-27 02:02:43,809][105620] Updated weights for policy 1, policy_version 1465070 (0.0010) [2023-12-27 02:02:43,842][105692] Updated weights for policy 0, policy_version 1462781 (0.0006) [2023-12-27 02:02:43,856][105620] Updated weights for policy 1, policy_version 1465080 (0.0008) [2023-12-27 02:02:43,911][105692] Updated weights for policy 0, policy_version 1462791 (0.0010) [2023-12-27 02:02:44,427][105620] Updated weights for policy 1, policy_version 1465090 (0.0006) [2023-12-27 02:02:44,482][105620] Updated weights for policy 1, policy_version 1465100 (0.0009) [2023-12-27 02:02:44,533][105620] Updated weights for policy 1, policy_version 1465110 (0.0006) [2023-12-27 02:02:44,544][105692] Updated weights for policy 0, policy_version 1462801 (0.0010) [2023-12-27 02:02:44,593][105620] Updated weights for policy 1, policy_version 1465120 (0.0006) [2023-12-27 02:02:44,609][105692] Updated weights for policy 0, policy_version 1462811 (0.0008) [2023-12-27 02:02:44,670][105692] Updated weights for policy 0, policy_version 1462821 (0.0008) [2023-12-27 02:02:44,730][105692] Updated weights for policy 0, policy_version 1462831 (0.0008) [2023-12-27 02:02:45,293][105620] Updated weights for policy 1, policy_version 1465130 (0.0005) [2023-12-27 02:02:45,357][105620] Updated weights for policy 1, policy_version 1465140 (0.0007) [2023-12-27 02:02:45,420][105620] Updated weights for policy 1, policy_version 1465150 (0.0007) [2023-12-27 02:02:45,486][105692] Updated weights for policy 0, policy_version 1462841 (0.0009) [2023-12-27 02:02:45,552][105692] Updated weights for policy 0, policy_version 1462851 (0.0009) [2023-12-27 02:02:45,614][105692] Updated weights for policy 0, policy_version 1462861 (0.0010) [2023-12-27 02:02:45,968][105620] Updated weights for policy 1, policy_version 1465160 (0.0005) [2023-12-27 02:02:46,021][105620] Updated weights for policy 1, policy_version 1465170 (0.0005) [2023-12-27 02:02:46,062][104569] Fps is (10 sec: 19659.5, 60 sec: 19387.6, 300 sec: 19577.5). Total num frames: 749674496. Throughput: 0: 9773.5, 1: 9768.2. Samples: 749645020. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:02:46,064][104569] Avg episode reward: [(0, '8346.594'), (1, '9174.442')] [2023-12-27 02:02:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001462864_374546432.pth... [2023-12-27 02:02:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001461744_374259712.pth [2023-12-27 02:02:46,079][105620] Updated weights for policy 1, policy_version 1465180 (0.0005) [2023-12-27 02:02:46,098][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001465184_375136256.pth... [2023-12-27 02:02:46,101][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001464032_374841344.pth [2023-12-27 02:02:46,510][105692] Updated weights for policy 0, policy_version 1462871 (0.0008) [2023-12-27 02:02:46,562][105692] Updated weights for policy 0, policy_version 1462881 (0.0009) [2023-12-27 02:02:46,592][105620] Updated weights for policy 1, policy_version 1465190 (0.0005) [2023-12-27 02:02:46,614][105692] Updated weights for policy 0, policy_version 1462891 (0.0008) [2023-12-27 02:02:46,645][105620] Updated weights for policy 1, policy_version 1465200 (0.0005) [2023-12-27 02:02:46,704][105620] Updated weights for policy 1, policy_version 1465210 (0.0008) [2023-12-27 02:02:47,300][105620] Updated weights for policy 1, policy_version 1465220 (0.0008) [2023-12-27 02:02:47,360][105620] Updated weights for policy 1, policy_version 1465230 (0.0009) [2023-12-27 02:02:47,419][105620] Updated weights for policy 1, policy_version 1465240 (0.0007) [2023-12-27 02:02:47,467][105692] Updated weights for policy 0, policy_version 1462901 (0.0008) [2023-12-27 02:02:47,514][105692] Updated weights for policy 0, policy_version 1462911 (0.0009) [2023-12-27 02:02:47,568][105692] Updated weights for policy 0, policy_version 1462921 (0.0009) [2023-12-27 02:02:48,083][105620] Updated weights for policy 1, policy_version 1465250 (0.0007) [2023-12-27 02:02:48,130][105620] Updated weights for policy 1, policy_version 1465260 (0.0008) [2023-12-27 02:02:48,179][105620] Updated weights for policy 1, policy_version 1465270 (0.0008) [2023-12-27 02:02:48,226][105620] Updated weights for policy 1, policy_version 1465280 (0.0009) [2023-12-27 02:02:48,365][105692] Updated weights for policy 0, policy_version 1462931 (0.0009) [2023-12-27 02:02:48,419][105692] Updated weights for policy 0, policy_version 1462941 (0.0009) [2023-12-27 02:02:48,475][105692] Updated weights for policy 0, policy_version 1462951 (0.0009) [2023-12-27 02:02:48,985][105620] Updated weights for policy 1, policy_version 1465290 (0.0010) [2023-12-27 02:02:49,037][105620] Updated weights for policy 1, policy_version 1465300 (0.0010) [2023-12-27 02:02:49,096][105620] Updated weights for policy 1, policy_version 1465310 (0.0010) [2023-12-27 02:02:49,269][105692] Updated weights for policy 0, policy_version 1462961 (0.0009) [2023-12-27 02:02:49,325][105692] Updated weights for policy 0, policy_version 1462971 (0.0008) [2023-12-27 02:02:49,389][105692] Updated weights for policy 0, policy_version 1462981 (0.0009) [2023-12-27 02:02:49,446][105692] Updated weights for policy 0, policy_version 1462991 (0.0008) [2023-12-27 02:02:49,890][105620] Updated weights for policy 1, policy_version 1465320 (0.0010) [2023-12-27 02:02:49,954][105620] Updated weights for policy 1, policy_version 1465330 (0.0011) [2023-12-27 02:02:50,014][105620] Updated weights for policy 1, policy_version 1465340 (0.0011) [2023-12-27 02:02:50,229][105692] Updated weights for policy 0, policy_version 1463001 (0.0009) [2023-12-27 02:02:50,288][105692] Updated weights for policy 0, policy_version 1463011 (0.0009) [2023-12-27 02:02:50,347][105692] Updated weights for policy 0, policy_version 1463021 (0.0009) [2023-12-27 02:02:50,711][105620] Updated weights for policy 1, policy_version 1465350 (0.0011) [2023-12-27 02:02:50,768][105620] Updated weights for policy 1, policy_version 1465360 (0.0011) [2023-12-27 02:02:50,826][105620] Updated weights for policy 1, policy_version 1465370 (0.0011) [2023-12-27 02:02:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 749772800. Throughput: 0: 9724.1, 1: 9838.9. Samples: 749763600. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:02:51,062][104569] Avg episode reward: [(0, '8253.640'), (1, '9262.710')] [2023-12-27 02:02:51,162][105692] Updated weights for policy 0, policy_version 1463031 (0.0009) [2023-12-27 02:02:51,215][105692] Updated weights for policy 0, policy_version 1463041 (0.0008) [2023-12-27 02:02:51,275][105692] Updated weights for policy 0, policy_version 1463051 (0.0009) [2023-12-27 02:02:51,562][105620] Updated weights for policy 1, policy_version 1465380 (0.0011) [2023-12-27 02:02:51,611][105620] Updated weights for policy 1, policy_version 1465390 (0.0010) [2023-12-27 02:02:51,679][105620] Updated weights for policy 1, policy_version 1465400 (0.0009) [2023-12-27 02:02:51,996][105692] Updated weights for policy 0, policy_version 1463061 (0.0009) [2023-12-27 02:02:52,053][105692] Updated weights for policy 0, policy_version 1463071 (0.0009) [2023-12-27 02:02:52,113][105692] Updated weights for policy 0, policy_version 1463081 (0.0009) [2023-12-27 02:02:52,417][105620] Updated weights for policy 1, policy_version 1465410 (0.0011) [2023-12-27 02:02:52,482][105620] Updated weights for policy 1, policy_version 1465420 (0.0007) [2023-12-27 02:02:52,540][105620] Updated weights for policy 1, policy_version 1465430 (0.0008) [2023-12-27 02:02:52,605][105620] Updated weights for policy 1, policy_version 1465440 (0.0009) [2023-12-27 02:02:52,893][105692] Updated weights for policy 0, policy_version 1463091 (0.0009) [2023-12-27 02:02:52,951][105692] Updated weights for policy 0, policy_version 1463101 (0.0009) [2023-12-27 02:02:52,999][105692] Updated weights for policy 0, policy_version 1463111 (0.0009) [2023-12-27 02:02:53,350][105620] Updated weights for policy 1, policy_version 1465450 (0.0008) [2023-12-27 02:02:53,421][105620] Updated weights for policy 1, policy_version 1465460 (0.0009) [2023-12-27 02:02:53,480][105620] Updated weights for policy 1, policy_version 1465470 (0.0009) [2023-12-27 02:02:53,781][105692] Updated weights for policy 0, policy_version 1463121 (0.0009) [2023-12-27 02:02:53,836][105692] Updated weights for policy 0, policy_version 1463131 (0.0009) [2023-12-27 02:02:53,894][105692] Updated weights for policy 0, policy_version 1463141 (0.0009) [2023-12-27 02:02:53,941][105692] Updated weights for policy 0, policy_version 1463151 (0.0009) [2023-12-27 02:02:54,221][105620] Updated weights for policy 1, policy_version 1465480 (0.0009) [2023-12-27 02:02:54,273][105620] Updated weights for policy 1, policy_version 1465490 (0.0009) [2023-12-27 02:02:54,324][105620] Updated weights for policy 1, policy_version 1465500 (0.0009) [2023-12-27 02:02:54,731][105692] Updated weights for policy 0, policy_version 1463161 (0.0009) [2023-12-27 02:02:54,787][105692] Updated weights for policy 0, policy_version 1463171 (0.0008) [2023-12-27 02:02:54,848][105692] Updated weights for policy 0, policy_version 1463181 (0.0009) [2023-12-27 02:02:55,072][105620] Updated weights for policy 1, policy_version 1465510 (0.0010) [2023-12-27 02:02:55,124][105620] Updated weights for policy 1, policy_version 1465520 (0.0010) [2023-12-27 02:02:55,175][105620] Updated weights for policy 1, policy_version 1465530 (0.0010) [2023-12-27 02:02:55,499][105692] Updated weights for policy 0, policy_version 1463191 (0.0008) [2023-12-27 02:02:55,551][105692] Updated weights for policy 0, policy_version 1463201 (0.0008) [2023-12-27 02:02:55,612][105692] Updated weights for policy 0, policy_version 1463212 (0.0010) [2023-12-27 02:02:55,883][105620] Updated weights for policy 1, policy_version 1465540 (0.0010) [2023-12-27 02:02:55,935][105620] Updated weights for policy 1, policy_version 1465551 (0.0010) [2023-12-27 02:02:55,996][105620] Updated weights for policy 1, policy_version 1465561 (0.0010) [2023-12-27 02:02:56,062][104569] Fps is (10 sec: 19662.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 749871104. Throughput: 0: 9582.8, 1: 9776.7. Samples: 749876788. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:02:56,062][104569] Avg episode reward: [(0, '8341.678'), (1, '9262.645')] [2023-12-27 02:02:56,351][105692] Updated weights for policy 0, policy_version 1463222 (0.0008) [2023-12-27 02:02:56,404][105692] Updated weights for policy 0, policy_version 1463232 (0.0005) [2023-12-27 02:02:56,458][105692] Updated weights for policy 0, policy_version 1463242 (0.0005) [2023-12-27 02:02:56,651][105620] Updated weights for policy 1, policy_version 1465571 (0.0009) [2023-12-27 02:02:56,703][105620] Updated weights for policy 1, policy_version 1465581 (0.0007) [2023-12-27 02:02:56,763][105620] Updated weights for policy 1, policy_version 1465591 (0.0007) [2023-12-27 02:02:57,133][105692] Updated weights for policy 0, policy_version 1463252 (0.0007) [2023-12-27 02:02:57,188][105692] Updated weights for policy 0, policy_version 1463262 (0.0010) [2023-12-27 02:02:57,245][105692] Updated weights for policy 0, policy_version 1463272 (0.0010) [2023-12-27 02:02:57,352][105620] Updated weights for policy 1, policy_version 1465601 (0.0006) [2023-12-27 02:02:57,414][105620] Updated weights for policy 1, policy_version 1465611 (0.0005) [2023-12-27 02:02:57,482][105620] Updated weights for policy 1, policy_version 1465621 (0.0008) [2023-12-27 02:02:57,550][105620] Updated weights for policy 1, policy_version 1465631 (0.0009) [2023-12-27 02:02:57,822][105692] Updated weights for policy 0, policy_version 1463282 (0.0006) [2023-12-27 02:02:57,891][105692] Updated weights for policy 0, policy_version 1463292 (0.0007) [2023-12-27 02:02:57,944][105692] Updated weights for policy 0, policy_version 1463302 (0.0005) [2023-12-27 02:02:57,990][105692] Updated weights for policy 0, policy_version 1463312 (0.0005) [2023-12-27 02:02:58,113][105620] Updated weights for policy 1, policy_version 1465641 (0.0005) [2023-12-27 02:02:58,184][105620] Updated weights for policy 1, policy_version 1465651 (0.0007) [2023-12-27 02:02:58,243][105620] Updated weights for policy 1, policy_version 1465661 (0.0008) [2023-12-27 02:02:58,698][105692] Updated weights for policy 0, policy_version 1463322 (0.0009) [2023-12-27 02:02:58,762][105692] Updated weights for policy 0, policy_version 1463332 (0.0011) [2023-12-27 02:02:58,826][105692] Updated weights for policy 0, policy_version 1463342 (0.0009) [2023-12-27 02:02:58,927][105620] Updated weights for policy 1, policy_version 1465671 (0.0008) [2023-12-27 02:02:58,986][105620] Updated weights for policy 1, policy_version 1465681 (0.0009) [2023-12-27 02:02:59,050][105620] Updated weights for policy 1, policy_version 1465691 (0.0008) [2023-12-27 02:02:59,593][105692] Updated weights for policy 0, policy_version 1463352 (0.0009) [2023-12-27 02:02:59,648][105692] Updated weights for policy 0, policy_version 1463362 (0.0009) [2023-12-27 02:02:59,700][105692] Updated weights for policy 0, policy_version 1463372 (0.0009) [2023-12-27 02:02:59,779][105620] Updated weights for policy 1, policy_version 1465701 (0.0008) [2023-12-27 02:02:59,829][105620] Updated weights for policy 1, policy_version 1465711 (0.0009) [2023-12-27 02:02:59,896][105620] Updated weights for policy 1, policy_version 1465721 (0.0008) [2023-12-27 02:03:00,507][105692] Updated weights for policy 0, policy_version 1463382 (0.0009) [2023-12-27 02:03:00,557][105692] Updated weights for policy 0, policy_version 1463392 (0.0009) [2023-12-27 02:03:00,617][105692] Updated weights for policy 0, policy_version 1463402 (0.0013) [2023-12-27 02:03:00,617][105620] Updated weights for policy 1, policy_version 1465731 (0.0008) [2023-12-27 02:03:00,666][105620] Updated weights for policy 1, policy_version 1465741 (0.0005) [2023-12-27 02:03:00,734][105620] Updated weights for policy 1, policy_version 1465751 (0.0006) [2023-12-27 02:03:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 749969408. Throughput: 0: 9670.3, 1: 9813.7. Samples: 749940308. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:03:01,063][104569] Avg episode reward: [(0, '8346.385'), (1, '9079.688')] [2023-12-27 02:03:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001463408_374685696.pth... [2023-12-27 02:03:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001465760_375283712.pth... [2023-12-27 02:03:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001464576_374980608.pth [2023-12-27 02:03:01,085][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001462320_374407168.pth [2023-12-27 02:03:01,425][105620] Updated weights for policy 1, policy_version 1465761 (0.0008) [2023-12-27 02:03:01,431][105692] Updated weights for policy 0, policy_version 1463412 (0.0010) [2023-12-27 02:03:01,484][105620] Updated weights for policy 1, policy_version 1465771 (0.0006) [2023-12-27 02:03:01,490][105692] Updated weights for policy 0, policy_version 1463422 (0.0011) [2023-12-27 02:03:01,545][105620] Updated weights for policy 1, policy_version 1465781 (0.0006) [2023-12-27 02:03:01,551][105692] Updated weights for policy 0, policy_version 1463432 (0.0011) [2023-12-27 02:03:01,607][105620] Updated weights for policy 1, policy_version 1465791 (0.0007) [2023-12-27 02:03:02,188][105692] Updated weights for policy 0, policy_version 1463442 (0.0010) [2023-12-27 02:03:02,234][105620] Updated weights for policy 1, policy_version 1465801 (0.0009) [2023-12-27 02:03:02,242][105692] Updated weights for policy 0, policy_version 1463452 (0.0005) [2023-12-27 02:03:02,293][105620] Updated weights for policy 1, policy_version 1465811 (0.0008) [2023-12-27 02:03:02,306][105692] Updated weights for policy 0, policy_version 1463462 (0.0008) [2023-12-27 02:03:02,350][105620] Updated weights for policy 1, policy_version 1465821 (0.0007) [2023-12-27 02:03:02,364][105692] Updated weights for policy 0, policy_version 1463472 (0.0008) [2023-12-27 02:03:02,994][105692] Updated weights for policy 0, policy_version 1463482 (0.0010) [2023-12-27 02:03:03,042][105692] Updated weights for policy 0, policy_version 1463492 (0.0010) [2023-12-27 02:03:03,093][105692] Updated weights for policy 0, policy_version 1463502 (0.0005) [2023-12-27 02:03:03,169][105620] Updated weights for policy 1, policy_version 1465831 (0.0010) [2023-12-27 02:03:03,214][105620] Updated weights for policy 1, policy_version 1465841 (0.0010) [2023-12-27 02:03:03,260][105620] Updated weights for policy 1, policy_version 1465851 (0.0009) [2023-12-27 02:03:03,742][105692] Updated weights for policy 0, policy_version 1463512 (0.0006) [2023-12-27 02:03:03,803][105692] Updated weights for policy 0, policy_version 1463522 (0.0006) [2023-12-27 02:03:03,854][105692] Updated weights for policy 0, policy_version 1463532 (0.0008) [2023-12-27 02:03:03,915][105620] Updated weights for policy 1, policy_version 1465861 (0.0007) [2023-12-27 02:03:03,974][105620] Updated weights for policy 1, policy_version 1465871 (0.0008) [2023-12-27 02:03:04,033][105620] Updated weights for policy 1, policy_version 1465881 (0.0008) [2023-12-27 02:03:04,479][105692] Updated weights for policy 0, policy_version 1463542 (0.0009) [2023-12-27 02:03:04,530][105692] Updated weights for policy 0, policy_version 1463553 (0.0009) [2023-12-27 02:03:04,580][105692] Updated weights for policy 0, policy_version 1463564 (0.0008) [2023-12-27 02:03:04,863][105620] Updated weights for policy 1, policy_version 1465891 (0.0009) [2023-12-27 02:03:04,917][105620] Updated weights for policy 1, policy_version 1465901 (0.0010) [2023-12-27 02:03:04,975][105620] Updated weights for policy 1, policy_version 1465911 (0.0010) [2023-12-27 02:03:05,167][105692] Updated weights for policy 0, policy_version 1463574 (0.0005) [2023-12-27 02:03:05,229][105692] Updated weights for policy 0, policy_version 1463584 (0.0005) [2023-12-27 02:03:05,301][105692] Updated weights for policy 0, policy_version 1463594 (0.0005) [2023-12-27 02:03:05,834][105620] Updated weights for policy 1, policy_version 1465921 (0.0009) [2023-12-27 02:03:05,887][105620] Updated weights for policy 1, policy_version 1465931 (0.0008) [2023-12-27 02:03:05,906][105692] Updated weights for policy 0, policy_version 1463604 (0.0006) [2023-12-27 02:03:05,936][105620] Updated weights for policy 1, policy_version 1465941 (0.0006) [2023-12-27 02:03:05,958][105692] Updated weights for policy 0, policy_version 1463614 (0.0007) [2023-12-27 02:03:05,988][105620] Updated weights for policy 1, policy_version 1465951 (0.0006) [2023-12-27 02:03:06,011][105692] Updated weights for policy 0, policy_version 1463624 (0.0007) [2023-12-27 02:03:06,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 750075904. Throughput: 0: 9648.4, 1: 9793.0. Samples: 750056712. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:03:06,063][104569] Avg episode reward: [(0, '8717.027'), (1, '8804.093')] [2023-12-27 02:03:06,756][105620] Updated weights for policy 1, policy_version 1465961 (0.0008) [2023-12-27 02:03:06,810][105620] Updated weights for policy 1, policy_version 1465971 (0.0011) [2023-12-27 02:03:06,811][105692] Updated weights for policy 0, policy_version 1463634 (0.0007) [2023-12-27 02:03:06,866][105620] Updated weights for policy 1, policy_version 1465981 (0.0011) [2023-12-27 02:03:06,872][105692] Updated weights for policy 0, policy_version 1463644 (0.0007) [2023-12-27 02:03:06,941][105692] Updated weights for policy 0, policy_version 1463654 (0.0008) [2023-12-27 02:03:07,012][105692] Updated weights for policy 0, policy_version 1463664 (0.0010) [2023-12-27 02:03:07,553][105620] Updated weights for policy 1, policy_version 1465991 (0.0011) [2023-12-27 02:03:07,616][105620] Updated weights for policy 1, policy_version 1466001 (0.0010) [2023-12-27 02:03:07,679][105620] Updated weights for policy 1, policy_version 1466011 (0.0009) [2023-12-27 02:03:07,768][105692] Updated weights for policy 0, policy_version 1463674 (0.0008) [2023-12-27 02:03:07,818][105692] Updated weights for policy 0, policy_version 1463684 (0.0008) [2023-12-27 02:03:07,875][105692] Updated weights for policy 0, policy_version 1463694 (0.0007) [2023-12-27 02:03:08,390][105620] Updated weights for policy 1, policy_version 1466021 (0.0009) [2023-12-27 02:03:08,460][105620] Updated weights for policy 1, policy_version 1466031 (0.0010) [2023-12-27 02:03:08,527][105620] Updated weights for policy 1, policy_version 1466041 (0.0010) [2023-12-27 02:03:08,572][105692] Updated weights for policy 0, policy_version 1463704 (0.0006) [2023-12-27 02:03:08,623][105692] Updated weights for policy 0, policy_version 1463714 (0.0009) [2023-12-27 02:03:08,682][105692] Updated weights for policy 0, policy_version 1463724 (0.0011) [2023-12-27 02:03:09,286][105692] Updated weights for policy 0, policy_version 1463734 (0.0008) [2023-12-27 02:03:09,288][105620] Updated weights for policy 1, policy_version 1466051 (0.0009) [2023-12-27 02:03:09,353][105620] Updated weights for policy 1, policy_version 1466061 (0.0007) [2023-12-27 02:03:09,353][105692] Updated weights for policy 0, policy_version 1463744 (0.0007) [2023-12-27 02:03:09,423][105692] Updated weights for policy 0, policy_version 1463754 (0.0010) [2023-12-27 02:03:09,426][105620] Updated weights for policy 1, policy_version 1466071 (0.0009) [2023-12-27 02:03:10,097][105620] Updated weights for policy 1, policy_version 1466081 (0.0008) [2023-12-27 02:03:10,164][105620] Updated weights for policy 1, policy_version 1466091 (0.0008) [2023-12-27 02:03:10,170][105692] Updated weights for policy 0, policy_version 1463764 (0.0008) [2023-12-27 02:03:10,225][105620] Updated weights for policy 1, policy_version 1466101 (0.0008) [2023-12-27 02:03:10,231][105692] Updated weights for policy 0, policy_version 1463774 (0.0006) [2023-12-27 02:03:10,283][105620] Updated weights for policy 1, policy_version 1466111 (0.0007) [2023-12-27 02:03:10,292][105692] Updated weights for policy 0, policy_version 1463784 (0.0007) [2023-12-27 02:03:10,957][105620] Updated weights for policy 1, policy_version 1466121 (0.0008) [2023-12-27 02:03:11,019][105620] Updated weights for policy 1, policy_version 1466131 (0.0009) [2023-12-27 02:03:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19521.9). Total num frames: 750157824. Throughput: 0: 9670.0, 1: 9858.9. Samples: 750173360. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:03:11,063][104569] Avg episode reward: [(0, '8896.506'), (1, '8529.058')] [2023-12-27 02:03:11,069][105692] Updated weights for policy 0, policy_version 1463794 (0.0009) [2023-12-27 02:03:11,088][105620] Updated weights for policy 1, policy_version 1466141 (0.0009) [2023-12-27 02:03:11,132][105692] Updated weights for policy 0, policy_version 1463804 (0.0007) [2023-12-27 02:03:11,196][105692] Updated weights for policy 0, policy_version 1463814 (0.0007) [2023-12-27 02:03:11,255][105692] Updated weights for policy 0, policy_version 1463824 (0.0006) [2023-12-27 02:03:11,922][105692] Updated weights for policy 0, policy_version 1463834 (0.0008) [2023-12-27 02:03:11,956][105620] Updated weights for policy 1, policy_version 1466151 (0.0009) [2023-12-27 02:03:11,981][105692] Updated weights for policy 0, policy_version 1463844 (0.0006) [2023-12-27 02:03:12,021][105620] Updated weights for policy 1, policy_version 1466161 (0.0010) [2023-12-27 02:03:12,048][105692] Updated weights for policy 0, policy_version 1463854 (0.0006) [2023-12-27 02:03:12,091][105620] Updated weights for policy 1, policy_version 1466171 (0.0010) [2023-12-27 02:03:12,712][105692] Updated weights for policy 0, policy_version 1463864 (0.0007) [2023-12-27 02:03:12,768][105692] Updated weights for policy 0, policy_version 1463874 (0.0008) [2023-12-27 02:03:12,828][105692] Updated weights for policy 0, policy_version 1463884 (0.0008) [2023-12-27 02:03:12,841][105620] Updated weights for policy 1, policy_version 1466181 (0.0011) [2023-12-27 02:03:12,900][105620] Updated weights for policy 1, policy_version 1466191 (0.0011) [2023-12-27 02:03:12,953][105620] Updated weights for policy 1, policy_version 1466201 (0.0010) [2023-12-27 02:03:13,484][105692] Updated weights for policy 0, policy_version 1463894 (0.0005) [2023-12-27 02:03:13,534][105692] Updated weights for policy 0, policy_version 1463904 (0.0005) [2023-12-27 02:03:13,585][105692] Updated weights for policy 0, policy_version 1463914 (0.0005) [2023-12-27 02:03:13,698][105620] Updated weights for policy 1, policy_version 1466211 (0.0011) [2023-12-27 02:03:13,753][105620] Updated weights for policy 1, policy_version 1466221 (0.0011) [2023-12-27 02:03:13,808][105620] Updated weights for policy 1, policy_version 1466231 (0.0010) [2023-12-27 02:03:14,294][105692] Updated weights for policy 0, policy_version 1463924 (0.0007) [2023-12-27 02:03:14,345][105692] Updated weights for policy 0, policy_version 1463934 (0.0007) [2023-12-27 02:03:14,391][105692] Updated weights for policy 0, policy_version 1463945 (0.0008) [2023-12-27 02:03:14,561][105620] Updated weights for policy 1, policy_version 1466241 (0.0010) [2023-12-27 02:03:14,633][105620] Updated weights for policy 1, policy_version 1466251 (0.0010) [2023-12-27 02:03:14,694][105620] Updated weights for policy 1, policy_version 1466261 (0.0010) [2023-12-27 02:03:14,752][105620] Updated weights for policy 1, policy_version 1466271 (0.0010) [2023-12-27 02:03:15,186][105692] Updated weights for policy 0, policy_version 1463955 (0.0008) [2023-12-27 02:03:15,236][105692] Updated weights for policy 0, policy_version 1463965 (0.0008) [2023-12-27 02:03:15,281][105692] Updated weights for policy 0, policy_version 1463975 (0.0008) [2023-12-27 02:03:15,490][105620] Updated weights for policy 1, policy_version 1466281 (0.0011) [2023-12-27 02:03:15,542][105620] Updated weights for policy 1, policy_version 1466291 (0.0011) [2023-12-27 02:03:15,592][105620] Updated weights for policy 1, policy_version 1466301 (0.0010) [2023-12-27 02:03:15,912][105692] Updated weights for policy 0, policy_version 1463985 (0.0008) [2023-12-27 02:03:15,961][105692] Updated weights for policy 0, policy_version 1463995 (0.0005) [2023-12-27 02:03:16,007][105692] Updated weights for policy 0, policy_version 1464005 (0.0005) [2023-12-27 02:03:16,061][105692] Updated weights for policy 0, policy_version 1464015 (0.0005) [2023-12-27 02:03:16,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 750256128. Throughput: 0: 9691.8, 1: 9831.5. Samples: 750230412. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:03:16,063][104569] Avg episode reward: [(0, '8985.012'), (1, '8530.133')] [2023-12-27 02:03:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001464016_374841344.pth... [2023-12-27 02:03:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001466304_375422976.pth... [2023-12-27 02:03:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001462864_374546432.pth [2023-12-27 02:03:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001465184_375136256.pth [2023-12-27 02:03:16,364][105620] Updated weights for policy 1, policy_version 1466311 (0.0007) [2023-12-27 02:03:16,429][105620] Updated weights for policy 1, policy_version 1466321 (0.0008) [2023-12-27 02:03:16,491][105620] Updated weights for policy 1, policy_version 1466331 (0.0010) [2023-12-27 02:03:16,601][105692] Updated weights for policy 0, policy_version 1464025 (0.0005) [2023-12-27 02:03:16,654][105692] Updated weights for policy 0, policy_version 1464035 (0.0005) [2023-12-27 02:03:16,715][105692] Updated weights for policy 0, policy_version 1464045 (0.0005) [2023-12-27 02:03:17,113][105620] Updated weights for policy 1, policy_version 1466341 (0.0010) [2023-12-27 02:03:17,161][105620] Updated weights for policy 1, policy_version 1466351 (0.0010) [2023-12-27 02:03:17,213][105620] Updated weights for policy 1, policy_version 1466361 (0.0010) [2023-12-27 02:03:17,219][105692] Updated weights for policy 0, policy_version 1464055 (0.0005) [2023-12-27 02:03:17,272][105692] Updated weights for policy 0, policy_version 1464065 (0.0009) [2023-12-27 02:03:17,330][105692] Updated weights for policy 0, policy_version 1464075 (0.0010) [2023-12-27 02:03:17,977][105620] Updated weights for policy 1, policy_version 1466371 (0.0010) [2023-12-27 02:03:18,016][105692] Updated weights for policy 0, policy_version 1464085 (0.0010) [2023-12-27 02:03:18,032][105620] Updated weights for policy 1, policy_version 1466381 (0.0010) [2023-12-27 02:03:18,072][105692] Updated weights for policy 0, policy_version 1464095 (0.0010) [2023-12-27 02:03:18,076][105620] Updated weights for policy 1, policy_version 1466391 (0.0007) [2023-12-27 02:03:18,124][105692] Updated weights for policy 0, policy_version 1464105 (0.0010) [2023-12-27 02:03:18,708][105620] Updated weights for policy 1, policy_version 1466401 (0.0005) [2023-12-27 02:03:18,766][105620] Updated weights for policy 1, policy_version 1466411 (0.0007) [2023-12-27 02:03:18,834][105620] Updated weights for policy 1, policy_version 1466421 (0.0011) [2023-12-27 02:03:18,898][105620] Updated weights for policy 1, policy_version 1466431 (0.0011) [2023-12-27 02:03:18,937][105692] Updated weights for policy 0, policy_version 1464115 (0.0009) [2023-12-27 02:03:19,004][105692] Updated weights for policy 0, policy_version 1464125 (0.0005) [2023-12-27 02:03:19,066][105692] Updated weights for policy 0, policy_version 1464135 (0.0007) [2023-12-27 02:03:19,598][105620] Updated weights for policy 1, policy_version 1466441 (0.0010) [2023-12-27 02:03:19,660][105620] Updated weights for policy 1, policy_version 1466451 (0.0010) [2023-12-27 02:03:19,716][105620] Updated weights for policy 1, policy_version 1466461 (0.0010) [2023-12-27 02:03:19,721][105692] Updated weights for policy 0, policy_version 1464145 (0.0006) [2023-12-27 02:03:19,778][105692] Updated weights for policy 0, policy_version 1464155 (0.0007) [2023-12-27 02:03:19,844][105692] Updated weights for policy 0, policy_version 1464165 (0.0007) [2023-12-27 02:03:19,907][105692] Updated weights for policy 0, policy_version 1464175 (0.0007) [2023-12-27 02:03:20,366][105620] Updated weights for policy 1, policy_version 1466471 (0.0009) [2023-12-27 02:03:20,427][105620] Updated weights for policy 1, policy_version 1466481 (0.0008) [2023-12-27 02:03:20,489][105620] Updated weights for policy 1, policy_version 1466491 (0.0008) [2023-12-27 02:03:20,772][105692] Updated weights for policy 0, policy_version 1464185 (0.0006) [2023-12-27 02:03:20,840][105692] Updated weights for policy 0, policy_version 1464195 (0.0009) [2023-12-27 02:03:20,898][105692] Updated weights for policy 0, policy_version 1464205 (0.0010) [2023-12-27 02:03:21,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 750362624. Throughput: 0: 9788.8, 1: 9749.4. Samples: 750351664. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:03:21,062][104569] Avg episode reward: [(0, '9168.315'), (1, '8898.023')] [2023-12-27 02:03:21,149][105620] Updated weights for policy 1, policy_version 1466501 (0.0009) [2023-12-27 02:03:21,205][105620] Updated weights for policy 1, policy_version 1466511 (0.0008) [2023-12-27 02:03:21,268][105620] Updated weights for policy 1, policy_version 1466521 (0.0008) [2023-12-27 02:03:21,698][105692] Updated weights for policy 0, policy_version 1464215 (0.0009) [2023-12-27 02:03:21,763][105692] Updated weights for policy 0, policy_version 1464225 (0.0010) [2023-12-27 02:03:21,827][105692] Updated weights for policy 0, policy_version 1464235 (0.0010) [2023-12-27 02:03:21,964][105620] Updated weights for policy 1, policy_version 1466531 (0.0008) [2023-12-27 02:03:22,016][105620] Updated weights for policy 1, policy_version 1466541 (0.0009) [2023-12-27 02:03:22,084][105620] Updated weights for policy 1, policy_version 1466551 (0.0006) [2023-12-27 02:03:22,630][105692] Updated weights for policy 0, policy_version 1464246 (0.0007) [2023-12-27 02:03:22,693][105692] Updated weights for policy 0, policy_version 1464256 (0.0007) [2023-12-27 02:03:22,749][105692] Updated weights for policy 0, policy_version 1464266 (0.0006) [2023-12-27 02:03:22,843][105620] Updated weights for policy 1, policy_version 1466561 (0.0006) [2023-12-27 02:03:22,906][105620] Updated weights for policy 1, policy_version 1466571 (0.0010) [2023-12-27 02:03:22,971][105620] Updated weights for policy 1, policy_version 1466582 (0.0010) [2023-12-27 02:03:23,037][105620] Updated weights for policy 1, policy_version 1466592 (0.0009) [2023-12-27 02:03:23,393][105692] Updated weights for policy 0, policy_version 1464276 (0.0007) [2023-12-27 02:03:23,441][105692] Updated weights for policy 0, policy_version 1464286 (0.0009) [2023-12-27 02:03:23,494][105692] Updated weights for policy 0, policy_version 1464296 (0.0009) [2023-12-27 02:03:23,823][105620] Updated weights for policy 1, policy_version 1466602 (0.0010) [2023-12-27 02:03:23,889][105620] Updated weights for policy 1, policy_version 1466612 (0.0010) [2023-12-27 02:03:23,947][105620] Updated weights for policy 1, policy_version 1466622 (0.0009) [2023-12-27 02:03:24,156][105692] Updated weights for policy 0, policy_version 1464306 (0.0009) [2023-12-27 02:03:24,208][105692] Updated weights for policy 0, policy_version 1464316 (0.0010) [2023-12-27 02:03:24,261][105692] Updated weights for policy 0, policy_version 1464326 (0.0010) [2023-12-27 02:03:24,309][105692] Updated weights for policy 0, policy_version 1464336 (0.0008) [2023-12-27 02:03:24,663][105620] Updated weights for policy 1, policy_version 1466632 (0.0009) [2023-12-27 02:03:24,726][105620] Updated weights for policy 1, policy_version 1466642 (0.0009) [2023-12-27 02:03:24,777][105620] Updated weights for policy 1, policy_version 1466652 (0.0009) [2023-12-27 02:03:25,104][105692] Updated weights for policy 0, policy_version 1464346 (0.0009) [2023-12-27 02:03:25,170][105692] Updated weights for policy 0, policy_version 1464356 (0.0010) [2023-12-27 02:03:25,232][105692] Updated weights for policy 0, policy_version 1464366 (0.0009) [2023-12-27 02:03:25,478][105620] Updated weights for policy 1, policy_version 1466662 (0.0008) [2023-12-27 02:03:25,526][105620] Updated weights for policy 1, policy_version 1466672 (0.0005) [2023-12-27 02:03:25,577][105620] Updated weights for policy 1, policy_version 1466682 (0.0005) [2023-12-27 02:03:26,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 750452736. Throughput: 0: 9721.5, 1: 9798.0. Samples: 750465292. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:03:26,062][104569] Avg episode reward: [(0, '8801.742'), (1, '8910.591')] [2023-12-27 02:03:26,072][105692] Updated weights for policy 0, policy_version 1464376 (0.0009) [2023-12-27 02:03:26,123][105692] Updated weights for policy 0, policy_version 1464386 (0.0009) [2023-12-27 02:03:26,174][105692] Updated weights for policy 0, policy_version 1464396 (0.0008) [2023-12-27 02:03:26,179][105620] Updated weights for policy 1, policy_version 1466692 (0.0007) [2023-12-27 02:03:26,236][105620] Updated weights for policy 1, policy_version 1466702 (0.0009) [2023-12-27 02:03:26,296][105620] Updated weights for policy 1, policy_version 1466712 (0.0007) [2023-12-27 02:03:26,913][105620] Updated weights for policy 1, policy_version 1466722 (0.0008) [2023-12-27 02:03:26,955][105692] Updated weights for policy 0, policy_version 1464406 (0.0007) [2023-12-27 02:03:26,970][105620] Updated weights for policy 1, policy_version 1466732 (0.0006) [2023-12-27 02:03:27,012][105692] Updated weights for policy 0, policy_version 1464416 (0.0010) [2023-12-27 02:03:27,031][105620] Updated weights for policy 1, policy_version 1466742 (0.0007) [2023-12-27 02:03:27,057][105692] Updated weights for policy 0, policy_version 1464426 (0.0006) [2023-12-27 02:03:27,075][105620] Updated weights for policy 1, policy_version 1466752 (0.0010) [2023-12-27 02:03:27,748][105620] Updated weights for policy 1, policy_version 1466762 (0.0010) [2023-12-27 02:03:27,792][105620] Updated weights for policy 1, policy_version 1466772 (0.0010) [2023-12-27 02:03:27,843][105692] Updated weights for policy 0, policy_version 1464436 (0.0006) [2023-12-27 02:03:27,860][105620] Updated weights for policy 1, policy_version 1466782 (0.0010) [2023-12-27 02:03:27,893][105692] Updated weights for policy 0, policy_version 1464446 (0.0006) [2023-12-27 02:03:27,946][105692] Updated weights for policy 0, policy_version 1464456 (0.0005) [2023-12-27 02:03:28,542][105620] Updated weights for policy 1, policy_version 1466792 (0.0009) [2023-12-27 02:03:28,596][105620] Updated weights for policy 1, policy_version 1466802 (0.0006) [2023-12-27 02:03:28,618][105692] Updated weights for policy 0, policy_version 1464466 (0.0005) [2023-12-27 02:03:28,654][105620] Updated weights for policy 1, policy_version 1466812 (0.0007) [2023-12-27 02:03:28,670][105692] Updated weights for policy 0, policy_version 1464476 (0.0005) [2023-12-27 02:03:28,725][105692] Updated weights for policy 0, policy_version 1464486 (0.0007) [2023-12-27 02:03:28,780][105692] Updated weights for policy 0, policy_version 1464496 (0.0009) [2023-12-27 02:03:29,388][105620] Updated weights for policy 1, policy_version 1466822 (0.0009) [2023-12-27 02:03:29,440][105620] Updated weights for policy 1, policy_version 1466832 (0.0008) [2023-12-27 02:03:29,476][105692] Updated weights for policy 0, policy_version 1464506 (0.0008) [2023-12-27 02:03:29,494][105620] Updated weights for policy 1, policy_version 1466843 (0.0006) [2023-12-27 02:03:29,537][105692] Updated weights for policy 0, policy_version 1464516 (0.0007) [2023-12-27 02:03:29,602][105692] Updated weights for policy 0, policy_version 1464526 (0.0007) [2023-12-27 02:03:30,278][105620] Updated weights for policy 1, policy_version 1466853 (0.0007) [2023-12-27 02:03:30,343][105620] Updated weights for policy 1, policy_version 1466863 (0.0008) [2023-12-27 02:03:30,363][105692] Updated weights for policy 0, policy_version 1464536 (0.0008) [2023-12-27 02:03:30,401][105620] Updated weights for policy 1, policy_version 1466873 (0.0008) [2023-12-27 02:03:30,419][105692] Updated weights for policy 0, policy_version 1464546 (0.0010) [2023-12-27 02:03:30,475][105692] Updated weights for policy 0, policy_version 1464556 (0.0008) [2023-12-27 02:03:31,055][105620] Updated weights for policy 1, policy_version 1466883 (0.0007) [2023-12-27 02:03:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 750551040. Throughput: 0: 9677.6, 1: 9865.6. Samples: 750524456. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:03:31,062][104569] Avg episode reward: [(0, '8892.581'), (1, '8630.399')] [2023-12-27 02:03:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001464560_374980608.pth... [2023-12-27 02:03:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001463408_374685696.pth [2023-12-27 02:03:31,116][105620] Updated weights for policy 1, policy_version 1466893 (0.0010) [2023-12-27 02:03:31,171][105692] Updated weights for policy 0, policy_version 1464566 (0.0010) [2023-12-27 02:03:31,180][105620] Updated weights for policy 1, policy_version 1466903 (0.0010) [2023-12-27 02:03:31,227][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001466912_375578624.pth... [2023-12-27 02:03:31,231][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001465760_375283712.pth [2023-12-27 02:03:31,233][105692] Updated weights for policy 0, policy_version 1464576 (0.0011) [2023-12-27 02:03:31,289][105692] Updated weights for policy 0, policy_version 1464586 (0.0011) [2023-12-27 02:03:31,845][105620] Updated weights for policy 1, policy_version 1466913 (0.0007) [2023-12-27 02:03:31,908][105620] Updated weights for policy 1, policy_version 1466923 (0.0006) [2023-12-27 02:03:31,969][105620] Updated weights for policy 1, policy_version 1466933 (0.0007) [2023-12-27 02:03:32,028][105620] Updated weights for policy 1, policy_version 1466943 (0.0010) [2023-12-27 02:03:32,071][105692] Updated weights for policy 0, policy_version 1464596 (0.0011) [2023-12-27 02:03:32,129][105692] Updated weights for policy 0, policy_version 1464606 (0.0006) [2023-12-27 02:03:32,191][105692] Updated weights for policy 0, policy_version 1464616 (0.0011) [2023-12-27 02:03:32,755][105620] Updated weights for policy 1, policy_version 1466953 (0.0007) [2023-12-27 02:03:32,800][105692] Updated weights for policy 0, policy_version 1464626 (0.0007) [2023-12-27 02:03:32,816][105620] Updated weights for policy 1, policy_version 1466963 (0.0005) [2023-12-27 02:03:32,856][105692] Updated weights for policy 0, policy_version 1464636 (0.0011) [2023-12-27 02:03:32,882][105620] Updated weights for policy 1, policy_version 1466973 (0.0006) [2023-12-27 02:03:32,916][105692] Updated weights for policy 0, policy_version 1464646 (0.0010) [2023-12-27 02:03:32,968][105692] Updated weights for policy 0, policy_version 1464656 (0.0005) [2023-12-27 02:03:33,395][105620] Updated weights for policy 1, policy_version 1466983 (0.0005) [2023-12-27 02:03:33,453][105620] Updated weights for policy 1, policy_version 1466993 (0.0005) [2023-12-27 02:03:33,510][105620] Updated weights for policy 1, policy_version 1467003 (0.0005) [2023-12-27 02:03:33,578][105692] Updated weights for policy 0, policy_version 1464666 (0.0006) [2023-12-27 02:03:33,625][105692] Updated weights for policy 0, policy_version 1464676 (0.0006) [2023-12-27 02:03:33,673][105692] Updated weights for policy 0, policy_version 1464686 (0.0006) [2023-12-27 02:03:34,162][105620] Updated weights for policy 1, policy_version 1467013 (0.0008) [2023-12-27 02:03:34,224][105620] Updated weights for policy 1, policy_version 1467023 (0.0011) [2023-12-27 02:03:34,238][105692] Updated weights for policy 0, policy_version 1464696 (0.0007) [2023-12-27 02:03:34,284][105620] Updated weights for policy 1, policy_version 1467033 (0.0011) [2023-12-27 02:03:34,299][105692] Updated weights for policy 0, policy_version 1464706 (0.0007) [2023-12-27 02:03:34,360][105692] Updated weights for policy 0, policy_version 1464716 (0.0007) [2023-12-27 02:03:34,997][105620] Updated weights for policy 1, policy_version 1467043 (0.0009) [2023-12-27 02:03:35,052][105620] Updated weights for policy 1, policy_version 1467053 (0.0006) [2023-12-27 02:03:35,070][105692] Updated weights for policy 0, policy_version 1464726 (0.0008) [2023-12-27 02:03:35,105][105620] Updated weights for policy 1, policy_version 1467063 (0.0006) [2023-12-27 02:03:35,120][105692] Updated weights for policy 0, policy_version 1464736 (0.0009) [2023-12-27 02:03:35,186][105692] Updated weights for policy 0, policy_version 1464746 (0.0009) [2023-12-27 02:03:35,690][105620] Updated weights for policy 1, policy_version 1467073 (0.0006) [2023-12-27 02:03:35,757][105620] Updated weights for policy 1, policy_version 1467083 (0.0005) [2023-12-27 02:03:35,809][105620] Updated weights for policy 1, policy_version 1467093 (0.0005) [2023-12-27 02:03:35,858][105620] Updated weights for policy 1, policy_version 1467103 (0.0005) [2023-12-27 02:03:36,042][105692] Updated weights for policy 0, policy_version 1464756 (0.0010) [2023-12-27 02:03:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 750657536. Throughput: 0: 9841.8, 1: 9790.2. Samples: 750647040. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:03:36,062][104569] Avg episode reward: [(0, '8252.062'), (1, '8697.471')] [2023-12-27 02:03:36,101][105692] Updated weights for policy 0, policy_version 1464768 (0.0010) [2023-12-27 02:03:36,168][105692] Updated weights for policy 0, policy_version 1464778 (0.0008) [2023-12-27 02:03:36,434][105620] Updated weights for policy 1, policy_version 1467113 (0.0008) [2023-12-27 02:03:36,495][105620] Updated weights for policy 1, policy_version 1467123 (0.0006) [2023-12-27 02:03:36,554][105620] Updated weights for policy 1, policy_version 1467133 (0.0007) [2023-12-27 02:03:37,042][105692] Updated weights for policy 0, policy_version 1464788 (0.0009) [2023-12-27 02:03:37,102][105692] Updated weights for policy 0, policy_version 1464798 (0.0007) [2023-12-27 02:03:37,167][105692] Updated weights for policy 0, policy_version 1464808 (0.0009) [2023-12-27 02:03:37,173][105620] Updated weights for policy 1, policy_version 1467143 (0.0011) [2023-12-27 02:03:37,232][105620] Updated weights for policy 1, policy_version 1467153 (0.0011) [2023-12-27 02:03:37,292][105620] Updated weights for policy 1, policy_version 1467163 (0.0011) [2023-12-27 02:03:37,835][105692] Updated weights for policy 0, policy_version 1464818 (0.0010) [2023-12-27 02:03:37,902][105692] Updated weights for policy 0, policy_version 1464828 (0.0009) [2023-12-27 02:03:37,953][105692] Updated weights for policy 0, policy_version 1464838 (0.0006) [2023-12-27 02:03:38,006][105692] Updated weights for policy 0, policy_version 1464848 (0.0008) [2023-12-27 02:03:38,013][105620] Updated weights for policy 1, policy_version 1467173 (0.0008) [2023-12-27 02:03:38,075][105620] Updated weights for policy 1, policy_version 1467183 (0.0005) [2023-12-27 02:03:38,130][105620] Updated weights for policy 1, policy_version 1467193 (0.0008) [2023-12-27 02:03:38,746][105692] Updated weights for policy 0, policy_version 1464858 (0.0009) [2023-12-27 02:03:38,806][105692] Updated weights for policy 0, policy_version 1464868 (0.0006) [2023-12-27 02:03:38,817][105620] Updated weights for policy 1, policy_version 1467203 (0.0010) [2023-12-27 02:03:38,865][105692] Updated weights for policy 0, policy_version 1464878 (0.0006) [2023-12-27 02:03:38,876][105620] Updated weights for policy 1, policy_version 1467213 (0.0010) [2023-12-27 02:03:38,935][105620] Updated weights for policy 1, policy_version 1467223 (0.0010) [2023-12-27 02:03:39,581][105692] Updated weights for policy 0, policy_version 1464888 (0.0006) [2023-12-27 02:03:39,646][105692] Updated weights for policy 0, policy_version 1464898 (0.0006) [2023-12-27 02:03:39,685][105620] Updated weights for policy 1, policy_version 1467233 (0.0011) [2023-12-27 02:03:39,706][105692] Updated weights for policy 0, policy_version 1464908 (0.0011) [2023-12-27 02:03:39,754][105620] Updated weights for policy 1, policy_version 1467243 (0.0010) [2023-12-27 02:03:39,819][105620] Updated weights for policy 1, policy_version 1467253 (0.0009) [2023-12-27 02:03:39,892][105620] Updated weights for policy 1, policy_version 1467263 (0.0009) [2023-12-27 02:03:40,358][105692] Updated weights for policy 0, policy_version 1464918 (0.0008) [2023-12-27 02:03:40,407][105692] Updated weights for policy 0, policy_version 1464928 (0.0009) [2023-12-27 02:03:40,458][105692] Updated weights for policy 0, policy_version 1464938 (0.0009) [2023-12-27 02:03:40,598][105620] Updated weights for policy 1, policy_version 1467273 (0.0009) [2023-12-27 02:03:40,647][105620] Updated weights for policy 1, policy_version 1467283 (0.0009) [2023-12-27 02:03:40,703][105620] Updated weights for policy 1, policy_version 1467293 (0.0008) [2023-12-27 02:03:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 750755840. Throughput: 0: 9842.8, 1: 9875.6. Samples: 750764120. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:03:41,062][104569] Avg episode reward: [(0, '8065.817'), (1, '8580.013')] [2023-12-27 02:03:41,087][105692] Updated weights for policy 0, policy_version 1464948 (0.0008) [2023-12-27 02:03:41,147][105692] Updated weights for policy 0, policy_version 1464958 (0.0010) [2023-12-27 02:03:41,207][105692] Updated weights for policy 0, policy_version 1464968 (0.0011) [2023-12-27 02:03:41,647][105620] Updated weights for policy 1, policy_version 1467303 (0.0008) [2023-12-27 02:03:41,712][105620] Updated weights for policy 1, policy_version 1467313 (0.0008) [2023-12-27 02:03:41,783][105620] Updated weights for policy 1, policy_version 1467323 (0.0008) [2023-12-27 02:03:41,964][105692] Updated weights for policy 0, policy_version 1464978 (0.0010) [2023-12-27 02:03:42,022][105692] Updated weights for policy 0, policy_version 1464988 (0.0009) [2023-12-27 02:03:42,085][105692] Updated weights for policy 0, policy_version 1464998 (0.0009) [2023-12-27 02:03:42,142][105692] Updated weights for policy 0, policy_version 1465008 (0.0009) [2023-12-27 02:03:42,521][105620] Updated weights for policy 1, policy_version 1467333 (0.0007) [2023-12-27 02:03:42,584][105620] Updated weights for policy 1, policy_version 1467343 (0.0005) [2023-12-27 02:03:42,643][105620] Updated weights for policy 1, policy_version 1467353 (0.0006) [2023-12-27 02:03:42,898][105692] Updated weights for policy 0, policy_version 1465018 (0.0005) [2023-12-27 02:03:42,958][105692] Updated weights for policy 0, policy_version 1465028 (0.0008) [2023-12-27 02:03:43,013][105692] Updated weights for policy 0, policy_version 1465038 (0.0006) [2023-12-27 02:03:43,195][105620] Updated weights for policy 1, policy_version 1467363 (0.0006) [2023-12-27 02:03:43,249][105620] Updated weights for policy 1, policy_version 1467373 (0.0005) [2023-12-27 02:03:43,316][105620] Updated weights for policy 1, policy_version 1467383 (0.0005) [2023-12-27 02:03:43,630][105692] Updated weights for policy 0, policy_version 1465048 (0.0006) [2023-12-27 02:03:43,681][105692] Updated weights for policy 0, policy_version 1465058 (0.0005) [2023-12-27 02:03:43,723][105692] Updated weights for policy 0, policy_version 1465068 (0.0008) [2023-12-27 02:03:43,944][105620] Updated weights for policy 1, policy_version 1467393 (0.0006) [2023-12-27 02:03:44,009][105620] Updated weights for policy 1, policy_version 1467403 (0.0008) [2023-12-27 02:03:44,072][105620] Updated weights for policy 1, policy_version 1467413 (0.0010) [2023-12-27 02:03:44,142][105620] Updated weights for policy 1, policy_version 1467423 (0.0009) [2023-12-27 02:03:44,327][105692] Updated weights for policy 0, policy_version 1465078 (0.0007) [2023-12-27 02:03:44,377][105692] Updated weights for policy 0, policy_version 1465088 (0.0009) [2023-12-27 02:03:44,424][105692] Updated weights for policy 0, policy_version 1465098 (0.0010) [2023-12-27 02:03:44,973][105620] Updated weights for policy 1, policy_version 1467433 (0.0009) [2023-12-27 02:03:45,031][105620] Updated weights for policy 1, policy_version 1467443 (0.0010) [2023-12-27 02:03:45,060][105692] Updated weights for policy 0, policy_version 1465108 (0.0006) [2023-12-27 02:03:45,092][105620] Updated weights for policy 1, policy_version 1467453 (0.0008) [2023-12-27 02:03:45,122][105692] Updated weights for policy 0, policy_version 1465118 (0.0007) [2023-12-27 02:03:45,187][105692] Updated weights for policy 0, policy_version 1465128 (0.0011) [2023-12-27 02:03:45,738][105620] Updated weights for policy 1, policy_version 1467463 (0.0006) [2023-12-27 02:03:45,786][105620] Updated weights for policy 1, policy_version 1467473 (0.0005) [2023-12-27 02:03:45,825][105692] Updated weights for policy 0, policy_version 1465138 (0.0011) [2023-12-27 02:03:45,833][105620] Updated weights for policy 1, policy_version 1467483 (0.0005) [2023-12-27 02:03:45,869][105692] Updated weights for policy 0, policy_version 1465148 (0.0010) [2023-12-27 02:03:45,914][105692] Updated weights for policy 0, policy_version 1465158 (0.0010) [2023-12-27 02:03:45,965][105692] Updated weights for policy 0, policy_version 1465168 (0.0010) [2023-12-27 02:03:46,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.5, 300 sec: 19633.0). Total num frames: 750862336. Throughput: 0: 9818.4, 1: 9833.4. Samples: 750824636. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:03:46,062][104569] Avg episode reward: [(0, '8156.866'), (1, '8563.999')] [2023-12-27 02:03:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001465168_375136256.pth... [2023-12-27 02:03:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001467488_375726080.pth... [2023-12-27 02:03:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001464016_374841344.pth [2023-12-27 02:03:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001466304_375422976.pth [2023-12-27 02:03:46,376][105620] Updated weights for policy 1, policy_version 1467493 (0.0006) [2023-12-27 02:03:46,419][105620] Updated weights for policy 1, policy_version 1467503 (0.0007) [2023-12-27 02:03:46,467][105620] Updated weights for policy 1, policy_version 1467513 (0.0008) [2023-12-27 02:03:46,711][105692] Updated weights for policy 0, policy_version 1465178 (0.0010) [2023-12-27 02:03:46,772][105692] Updated weights for policy 0, policy_version 1465188 (0.0010) [2023-12-27 02:03:46,840][105692] Updated weights for policy 0, policy_version 1465198 (0.0010) [2023-12-27 02:03:47,116][105620] Updated weights for policy 1, policy_version 1467523 (0.0008) [2023-12-27 02:03:47,161][105620] Updated weights for policy 1, policy_version 1467533 (0.0008) [2023-12-27 02:03:47,213][105620] Updated weights for policy 1, policy_version 1467543 (0.0008) [2023-12-27 02:03:47,553][105692] Updated weights for policy 0, policy_version 1465208 (0.0010) [2023-12-27 02:03:47,604][105692] Updated weights for policy 0, policy_version 1465218 (0.0010) [2023-12-27 02:03:47,666][105692] Updated weights for policy 0, policy_version 1465228 (0.0010) [2023-12-27 02:03:47,962][105620] Updated weights for policy 1, policy_version 1467553 (0.0007) [2023-12-27 02:03:48,014][105620] Updated weights for policy 1, policy_version 1467563 (0.0005) [2023-12-27 02:03:48,063][105620] Updated weights for policy 1, policy_version 1467573 (0.0006) [2023-12-27 02:03:48,109][105620] Updated weights for policy 1, policy_version 1467583 (0.0005) [2023-12-27 02:03:48,355][105692] Updated weights for policy 0, policy_version 1465238 (0.0011) [2023-12-27 02:03:48,414][105692] Updated weights for policy 0, policy_version 1465248 (0.0008) [2023-12-27 02:03:48,474][105692] Updated weights for policy 0, policy_version 1465258 (0.0009) [2023-12-27 02:03:48,782][105620] Updated weights for policy 1, policy_version 1467593 (0.0009) [2023-12-27 02:03:48,831][105620] Updated weights for policy 1, policy_version 1467603 (0.0007) [2023-12-27 02:03:48,888][105620] Updated weights for policy 1, policy_version 1467613 (0.0006) [2023-12-27 02:03:49,056][105692] Updated weights for policy 0, policy_version 1465268 (0.0007) [2023-12-27 02:03:49,105][105692] Updated weights for policy 0, policy_version 1465278 (0.0006) [2023-12-27 02:03:49,152][105692] Updated weights for policy 0, policy_version 1465288 (0.0009) [2023-12-27 02:03:49,667][105620] Updated weights for policy 1, policy_version 1467623 (0.0008) [2023-12-27 02:03:49,728][105620] Updated weights for policy 1, policy_version 1467633 (0.0009) [2023-12-27 02:03:49,792][105620] Updated weights for policy 1, policy_version 1467643 (0.0009) [2023-12-27 02:03:49,928][105692] Updated weights for policy 0, policy_version 1465298 (0.0009) [2023-12-27 02:03:49,998][105692] Updated weights for policy 0, policy_version 1465308 (0.0009) [2023-12-27 02:03:50,068][105692] Updated weights for policy 0, policy_version 1465318 (0.0010) [2023-12-27 02:03:50,132][105692] Updated weights for policy 0, policy_version 1465328 (0.0010) [2023-12-27 02:03:50,539][105620] Updated weights for policy 1, policy_version 1467653 (0.0010) [2023-12-27 02:03:50,606][105620] Updated weights for policy 1, policy_version 1467663 (0.0010) [2023-12-27 02:03:50,661][105620] Updated weights for policy 1, policy_version 1467673 (0.0006) [2023-12-27 02:03:50,812][105692] Updated weights for policy 0, policy_version 1465338 (0.0008) [2023-12-27 02:03:50,862][105692] Updated weights for policy 0, policy_version 1465348 (0.0008) [2023-12-27 02:03:50,914][105692] Updated weights for policy 0, policy_version 1465358 (0.0008) [2023-12-27 02:03:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 750960640. Throughput: 0: 9905.5, 1: 9876.8. Samples: 750946908. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:03:51,062][104569] Avg episode reward: [(0, '8433.090'), (1, '8641.521')] [2023-12-27 02:03:51,293][105620] Updated weights for policy 1, policy_version 1467683 (0.0007) [2023-12-27 02:03:51,358][105620] Updated weights for policy 1, policy_version 1467693 (0.0011) [2023-12-27 02:03:51,427][105620] Updated weights for policy 1, policy_version 1467703 (0.0011) [2023-12-27 02:03:51,676][105692] Updated weights for policy 0, policy_version 1465368 (0.0007) [2023-12-27 02:03:51,746][105692] Updated weights for policy 0, policy_version 1465378 (0.0009) [2023-12-27 02:03:51,806][105692] Updated weights for policy 0, policy_version 1465388 (0.0008) [2023-12-27 02:03:52,188][105620] Updated weights for policy 1, policy_version 1467713 (0.0011) [2023-12-27 02:03:52,243][105620] Updated weights for policy 1, policy_version 1467723 (0.0010) [2023-12-27 02:03:52,305][105620] Updated weights for policy 1, policy_version 1467733 (0.0010) [2023-12-27 02:03:52,367][105620] Updated weights for policy 1, policy_version 1467743 (0.0010) [2023-12-27 02:03:52,587][105692] Updated weights for policy 0, policy_version 1465398 (0.0007) [2023-12-27 02:03:52,653][105692] Updated weights for policy 0, policy_version 1465408 (0.0005) [2023-12-27 02:03:52,702][105692] Updated weights for policy 0, policy_version 1465418 (0.0006) [2023-12-27 02:03:53,108][105620] Updated weights for policy 1, policy_version 1467753 (0.0010) [2023-12-27 02:03:53,182][105620] Updated weights for policy 1, policy_version 1467763 (0.0008) [2023-12-27 02:03:53,247][105620] Updated weights for policy 1, policy_version 1467773 (0.0009) [2023-12-27 02:03:53,342][105692] Updated weights for policy 0, policy_version 1465428 (0.0007) [2023-12-27 02:03:53,402][105692] Updated weights for policy 0, policy_version 1465438 (0.0005) [2023-12-27 02:03:53,448][105692] Updated weights for policy 0, policy_version 1465448 (0.0005) [2023-12-27 02:03:53,887][105620] Updated weights for policy 1, policy_version 1467783 (0.0007) [2023-12-27 02:03:53,938][105620] Updated weights for policy 1, policy_version 1467793 (0.0005) [2023-12-27 02:03:53,964][105692] Updated weights for policy 0, policy_version 1465458 (0.0006) [2023-12-27 02:03:53,986][105620] Updated weights for policy 1, policy_version 1467803 (0.0006) [2023-12-27 02:03:54,012][105692] Updated weights for policy 0, policy_version 1465468 (0.0008) [2023-12-27 02:03:54,068][105692] Updated weights for policy 0, policy_version 1465479 (0.0011) [2023-12-27 02:03:54,652][105620] Updated weights for policy 1, policy_version 1467813 (0.0008) [2023-12-27 02:03:54,699][105620] Updated weights for policy 1, policy_version 1467823 (0.0010) [2023-12-27 02:03:54,747][105620] Updated weights for policy 1, policy_version 1467833 (0.0010) [2023-12-27 02:03:54,798][105692] Updated weights for policy 0, policy_version 1465489 (0.0009) [2023-12-27 02:03:54,857][105692] Updated weights for policy 0, policy_version 1465499 (0.0006) [2023-12-27 02:03:54,920][105692] Updated weights for policy 0, policy_version 1465509 (0.0008) [2023-12-27 02:03:54,985][105692] Updated weights for policy 0, policy_version 1465519 (0.0008) [2023-12-27 02:03:55,468][105620] Updated weights for policy 1, policy_version 1467843 (0.0009) [2023-12-27 02:03:55,532][105620] Updated weights for policy 1, policy_version 1467853 (0.0010) [2023-12-27 02:03:55,587][105620] Updated weights for policy 1, policy_version 1467863 (0.0010) [2023-12-27 02:03:55,692][105692] Updated weights for policy 0, policy_version 1465529 (0.0009) [2023-12-27 02:03:55,753][105692] Updated weights for policy 0, policy_version 1465539 (0.0008) [2023-12-27 02:03:55,817][105692] Updated weights for policy 0, policy_version 1465549 (0.0009) [2023-12-27 02:03:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 751058944. Throughput: 0: 9913.1, 1: 9925.4. Samples: 751066092. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:03:56,062][104569] Avg episode reward: [(0, '8801.445'), (1, '8268.628')] [2023-12-27 02:03:56,292][105620] Updated weights for policy 1, policy_version 1467873 (0.0010) [2023-12-27 02:03:56,343][105620] Updated weights for policy 1, policy_version 1467883 (0.0006) [2023-12-27 02:03:56,389][105620] Updated weights for policy 1, policy_version 1467893 (0.0006) [2023-12-27 02:03:56,419][105692] Updated weights for policy 0, policy_version 1465559 (0.0006) [2023-12-27 02:03:56,443][105620] Updated weights for policy 1, policy_version 1467903 (0.0010) [2023-12-27 02:03:56,472][105692] Updated weights for policy 0, policy_version 1465569 (0.0005) [2023-12-27 02:03:56,528][105692] Updated weights for policy 0, policy_version 1465579 (0.0007) [2023-12-27 02:03:57,070][105620] Updated weights for policy 1, policy_version 1467913 (0.0006) [2023-12-27 02:03:57,115][105620] Updated weights for policy 1, policy_version 1467923 (0.0005) [2023-12-27 02:03:57,160][105620] Updated weights for policy 1, policy_version 1467933 (0.0005) [2023-12-27 02:03:57,317][105692] Updated weights for policy 0, policy_version 1465589 (0.0008) [2023-12-27 02:03:57,368][105692] Updated weights for policy 0, policy_version 1465599 (0.0008) [2023-12-27 02:03:57,422][105692] Updated weights for policy 0, policy_version 1465609 (0.0008) [2023-12-27 02:03:57,735][105620] Updated weights for policy 1, policy_version 1467943 (0.0005) [2023-12-27 02:03:57,793][105620] Updated weights for policy 1, policy_version 1467953 (0.0005) [2023-12-27 02:03:57,844][105620] Updated weights for policy 1, policy_version 1467963 (0.0005) [2023-12-27 02:03:58,187][105692] Updated weights for policy 0, policy_version 1465619 (0.0008) [2023-12-27 02:03:58,248][105692] Updated weights for policy 0, policy_version 1465629 (0.0008) [2023-12-27 02:03:58,311][105692] Updated weights for policy 0, policy_version 1465639 (0.0008) [2023-12-27 02:03:58,426][105620] Updated weights for policy 1, policy_version 1467973 (0.0007) [2023-12-27 02:03:58,492][105620] Updated weights for policy 1, policy_version 1467983 (0.0008) [2023-12-27 02:03:58,558][105620] Updated weights for policy 1, policy_version 1467993 (0.0008) [2023-12-27 02:03:59,204][105692] Updated weights for policy 0, policy_version 1465650 (0.0008) [2023-12-27 02:03:59,272][105692] Updated weights for policy 0, policy_version 1465660 (0.0010) [2023-12-27 02:03:59,299][105620] Updated weights for policy 1, policy_version 1468003 (0.0010) [2023-12-27 02:03:59,326][105692] Updated weights for policy 0, policy_version 1465670 (0.0011) [2023-12-27 02:03:59,361][105620] Updated weights for policy 1, policy_version 1468013 (0.0008) [2023-12-27 02:03:59,395][105692] Updated weights for policy 0, policy_version 1465680 (0.0010) [2023-12-27 02:03:59,427][105620] Updated weights for policy 1, policy_version 1468023 (0.0010) [2023-12-27 02:04:00,100][105692] Updated weights for policy 0, policy_version 1465690 (0.0011) [2023-12-27 02:04:00,156][105692] Updated weights for policy 0, policy_version 1465700 (0.0009) [2023-12-27 02:04:00,176][105620] Updated weights for policy 1, policy_version 1468033 (0.0011) [2023-12-27 02:04:00,218][105692] Updated weights for policy 0, policy_version 1465710 (0.0005) [2023-12-27 02:04:00,235][105620] Updated weights for policy 1, policy_version 1468043 (0.0011) [2023-12-27 02:04:00,298][105620] Updated weights for policy 1, policy_version 1468053 (0.0011) [2023-12-27 02:04:00,357][105620] Updated weights for policy 1, policy_version 1468063 (0.0011) [2023-12-27 02:04:00,889][105692] Updated weights for policy 0, policy_version 1465720 (0.0005) [2023-12-27 02:04:00,952][105692] Updated weights for policy 0, policy_version 1465730 (0.0005) [2023-12-27 02:04:01,019][105692] Updated weights for policy 0, policy_version 1465740 (0.0006) [2023-12-27 02:04:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 751157248. Throughput: 0: 9868.2, 1: 10058.1. Samples: 751127092. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:04:01,063][104569] Avg episode reward: [(0, '8891.937'), (1, '8460.850')] [2023-12-27 02:04:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001465744_375283712.pth... [2023-12-27 02:04:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001464560_374980608.pth [2023-12-27 02:04:01,096][105620] Updated weights for policy 1, policy_version 1468073 (0.0010) [2023-12-27 02:04:01,160][105620] Updated weights for policy 1, policy_version 1468083 (0.0010) [2023-12-27 02:04:01,219][105620] Updated weights for policy 1, policy_version 1468093 (0.0010) [2023-12-27 02:04:01,231][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001468096_375881728.pth... [2023-12-27 02:04:01,234][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001466912_375578624.pth [2023-12-27 02:04:01,779][105692] Updated weights for policy 0, policy_version 1465750 (0.0009) [2023-12-27 02:04:01,847][105692] Updated weights for policy 0, policy_version 1465760 (0.0008) [2023-12-27 02:04:01,858][105620] Updated weights for policy 1, policy_version 1468103 (0.0007) [2023-12-27 02:04:01,909][105692] Updated weights for policy 0, policy_version 1465770 (0.0007) [2023-12-27 02:04:01,923][105620] Updated weights for policy 1, policy_version 1468113 (0.0007) [2023-12-27 02:04:01,993][105620] Updated weights for policy 1, policy_version 1468123 (0.0010) [2023-12-27 02:04:02,584][105692] Updated weights for policy 0, policy_version 1465780 (0.0007) [2023-12-27 02:04:02,631][105692] Updated weights for policy 0, policy_version 1465790 (0.0009) [2023-12-27 02:04:02,682][105692] Updated weights for policy 0, policy_version 1465800 (0.0009) [2023-12-27 02:04:02,706][105620] Updated weights for policy 1, policy_version 1468133 (0.0008) [2023-12-27 02:04:02,763][105620] Updated weights for policy 1, policy_version 1468143 (0.0009) [2023-12-27 02:04:02,813][105620] Updated weights for policy 1, policy_version 1468153 (0.0008) [2023-12-27 02:04:03,459][105692] Updated weights for policy 0, policy_version 1465810 (0.0007) [2023-12-27 02:04:03,513][105692] Updated weights for policy 0, policy_version 1465820 (0.0009) [2023-12-27 02:04:03,571][105620] Updated weights for policy 1, policy_version 1468163 (0.0009) [2023-12-27 02:04:03,573][105692] Updated weights for policy 0, policy_version 1465830 (0.0008) [2023-12-27 02:04:03,632][105692] Updated weights for policy 0, policy_version 1465840 (0.0007) [2023-12-27 02:04:03,634][105620] Updated weights for policy 1, policy_version 1468173 (0.0006) [2023-12-27 02:04:03,680][105620] Updated weights for policy 1, policy_version 1468183 (0.0007) [2023-12-27 02:04:04,297][105692] Updated weights for policy 0, policy_version 1465850 (0.0007) [2023-12-27 02:04:04,353][105692] Updated weights for policy 0, policy_version 1465860 (0.0009) [2023-12-27 02:04:04,408][105692] Updated weights for policy 0, policy_version 1465870 (0.0009) [2023-12-27 02:04:04,488][105620] Updated weights for policy 1, policy_version 1468193 (0.0008) [2023-12-27 02:04:04,559][105620] Updated weights for policy 1, policy_version 1468203 (0.0010) [2023-12-27 02:04:04,622][105620] Updated weights for policy 1, policy_version 1468213 (0.0009) [2023-12-27 02:04:04,678][105620] Updated weights for policy 1, policy_version 1468223 (0.0007) [2023-12-27 02:04:05,037][105692] Updated weights for policy 0, policy_version 1465880 (0.0006) [2023-12-27 02:04:05,100][105692] Updated weights for policy 0, policy_version 1465890 (0.0006) [2023-12-27 02:04:05,160][105692] Updated weights for policy 0, policy_version 1465900 (0.0006) [2023-12-27 02:04:05,470][105620] Updated weights for policy 1, policy_version 1468233 (0.0009) [2023-12-27 02:04:05,533][105620] Updated weights for policy 1, policy_version 1468243 (0.0009) [2023-12-27 02:04:05,591][105620] Updated weights for policy 1, policy_version 1468253 (0.0006) [2023-12-27 02:04:05,815][105692] Updated weights for policy 0, policy_version 1465910 (0.0006) [2023-12-27 02:04:05,883][105692] Updated weights for policy 0, policy_version 1465920 (0.0008) [2023-12-27 02:04:05,951][105692] Updated weights for policy 0, policy_version 1465930 (0.0010) [2023-12-27 02:04:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 751255552. Throughput: 0: 9756.6, 1: 10002.7. Samples: 751240832. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:04:06,062][104569] Avg episode reward: [(0, '8802.885'), (1, '8530.657')] [2023-12-27 02:04:06,146][105620] Updated weights for policy 1, policy_version 1468263 (0.0009) [2023-12-27 02:04:06,206][105620] Updated weights for policy 1, policy_version 1468273 (0.0011) [2023-12-27 02:04:06,267][105620] Updated weights for policy 1, policy_version 1468283 (0.0011) [2023-12-27 02:04:06,646][105692] Updated weights for policy 0, policy_version 1465940 (0.0009) [2023-12-27 02:04:06,715][105692] Updated weights for policy 0, policy_version 1465950 (0.0006) [2023-12-27 02:04:06,774][105692] Updated weights for policy 0, policy_version 1465960 (0.0010) [2023-12-27 02:04:06,969][105620] Updated weights for policy 1, policy_version 1468293 (0.0009) [2023-12-27 02:04:07,024][105620] Updated weights for policy 1, policy_version 1468303 (0.0010) [2023-12-27 02:04:07,091][105620] Updated weights for policy 1, policy_version 1468313 (0.0011) [2023-12-27 02:04:07,447][105692] Updated weights for policy 0, policy_version 1465970 (0.0009) [2023-12-27 02:04:07,497][105692] Updated weights for policy 0, policy_version 1465980 (0.0005) [2023-12-27 02:04:07,554][105692] Updated weights for policy 0, policy_version 1465990 (0.0006) [2023-12-27 02:04:07,616][105692] Updated weights for policy 0, policy_version 1466000 (0.0010) [2023-12-27 02:04:07,842][105620] Updated weights for policy 1, policy_version 1468323 (0.0011) [2023-12-27 02:04:07,900][105620] Updated weights for policy 1, policy_version 1468333 (0.0010) [2023-12-27 02:04:07,966][105620] Updated weights for policy 1, policy_version 1468343 (0.0010) [2023-12-27 02:04:08,266][105692] Updated weights for policy 0, policy_version 1466010 (0.0010) [2023-12-27 02:04:08,318][105692] Updated weights for policy 0, policy_version 1466020 (0.0010) [2023-12-27 02:04:08,374][105692] Updated weights for policy 0, policy_version 1466030 (0.0011) [2023-12-27 02:04:08,643][105620] Updated weights for policy 1, policy_version 1468353 (0.0010) [2023-12-27 02:04:08,703][105620] Updated weights for policy 1, policy_version 1468363 (0.0005) [2023-12-27 02:04:08,765][105620] Updated weights for policy 1, policy_version 1468373 (0.0005) [2023-12-27 02:04:08,826][105620] Updated weights for policy 1, policy_version 1468383 (0.0008) [2023-12-27 02:04:09,059][105692] Updated weights for policy 0, policy_version 1466040 (0.0010) [2023-12-27 02:04:09,108][105692] Updated weights for policy 0, policy_version 1466050 (0.0008) [2023-12-27 02:04:09,156][105692] Updated weights for policy 0, policy_version 1466060 (0.0008) [2023-12-27 02:04:09,541][105620] Updated weights for policy 1, policy_version 1468393 (0.0011) [2023-12-27 02:04:09,594][105620] Updated weights for policy 1, policy_version 1468403 (0.0010) [2023-12-27 02:04:09,652][105620] Updated weights for policy 1, policy_version 1468413 (0.0011) [2023-12-27 02:04:09,973][105692] Updated weights for policy 0, policy_version 1466070 (0.0008) [2023-12-27 02:04:10,044][105692] Updated weights for policy 0, policy_version 1466080 (0.0008) [2023-12-27 02:04:10,107][105692] Updated weights for policy 0, policy_version 1466090 (0.0008) [2023-12-27 02:04:10,445][105620] Updated weights for policy 1, policy_version 1468423 (0.0009) [2023-12-27 02:04:10,516][105620] Updated weights for policy 1, policy_version 1468433 (0.0006) [2023-12-27 02:04:10,576][105620] Updated weights for policy 1, policy_version 1468443 (0.0008) [2023-12-27 02:04:10,853][105692] Updated weights for policy 0, policy_version 1466100 (0.0008) [2023-12-27 02:04:10,907][105692] Updated weights for policy 0, policy_version 1466110 (0.0010) [2023-12-27 02:04:10,960][105692] Updated weights for policy 0, policy_version 1466120 (0.0010) [2023-12-27 02:04:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19933.9, 300 sec: 19633.1). Total num frames: 751353856. Throughput: 0: 9872.8, 1: 10010.2. Samples: 751360028. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:04:11,062][104569] Avg episode reward: [(0, '8987.269'), (1, '8538.698')] [2023-12-27 02:04:11,247][105620] Updated weights for policy 1, policy_version 1468453 (0.0009) [2023-12-27 02:04:11,303][105620] Updated weights for policy 1, policy_version 1468463 (0.0009) [2023-12-27 02:04:11,366][105620] Updated weights for policy 1, policy_version 1468473 (0.0009) [2023-12-27 02:04:11,805][105692] Updated weights for policy 0, policy_version 1466130 (0.0009) [2023-12-27 02:04:11,862][105692] Updated weights for policy 0, policy_version 1466140 (0.0009) [2023-12-27 02:04:11,929][105692] Updated weights for policy 0, policy_version 1466150 (0.0007) [2023-12-27 02:04:11,992][105692] Updated weights for policy 0, policy_version 1466160 (0.0009) [2023-12-27 02:04:12,183][105620] Updated weights for policy 1, policy_version 1468483 (0.0007) [2023-12-27 02:04:12,249][105620] Updated weights for policy 1, policy_version 1468493 (0.0007) [2023-12-27 02:04:12,315][105620] Updated weights for policy 1, policy_version 1468503 (0.0006) [2023-12-27 02:04:12,772][105692] Updated weights for policy 0, policy_version 1466170 (0.0008) [2023-12-27 02:04:12,802][105585] KL-divergence is very high: 145.1535 [2023-12-27 02:04:12,829][105692] Updated weights for policy 0, policy_version 1466180 (0.0005) [2023-12-27 02:04:12,850][105585] KL-divergence is very high: 256.7437 [2023-12-27 02:04:12,894][105692] Updated weights for policy 0, policy_version 1466190 (0.0005) [2023-12-27 02:04:12,902][105585] KL-divergence is very high: 282.7993 [2023-12-27 02:04:13,058][105620] Updated weights for policy 1, policy_version 1468513 (0.0009) [2023-12-27 02:04:13,110][105620] Updated weights for policy 1, policy_version 1468523 (0.0009) [2023-12-27 02:04:13,166][105620] Updated weights for policy 1, policy_version 1468533 (0.0005) [2023-12-27 02:04:13,224][105620] Updated weights for policy 1, policy_version 1468543 (0.0009) [2023-12-27 02:04:13,433][105692] Updated weights for policy 0, policy_version 1466200 (0.0005) [2023-12-27 02:04:13,491][105692] Updated weights for policy 0, policy_version 1466210 (0.0006) [2023-12-27 02:04:13,536][105692] Updated weights for policy 0, policy_version 1466220 (0.0006) [2023-12-27 02:04:14,051][105620] Updated weights for policy 1, policy_version 1468553 (0.0008) [2023-12-27 02:04:14,098][105620] Updated weights for policy 1, policy_version 1468563 (0.0009) [2023-12-27 02:04:14,139][105692] Updated weights for policy 0, policy_version 1466230 (0.0007) [2023-12-27 02:04:14,156][105620] Updated weights for policy 1, policy_version 1468573 (0.0006) [2023-12-27 02:04:14,204][105692] Updated weights for policy 0, policy_version 1466240 (0.0008) [2023-12-27 02:04:14,267][105692] Updated weights for policy 0, policy_version 1466250 (0.0009) [2023-12-27 02:04:14,893][105620] Updated weights for policy 1, policy_version 1468583 (0.0006) [2023-12-27 02:04:14,962][105620] Updated weights for policy 1, policy_version 1468593 (0.0009) [2023-12-27 02:04:14,981][105692] Updated weights for policy 0, policy_version 1466260 (0.0008) [2023-12-27 02:04:15,021][105620] Updated weights for policy 1, policy_version 1468603 (0.0008) [2023-12-27 02:04:15,047][105692] Updated weights for policy 0, policy_version 1466270 (0.0009) [2023-12-27 02:04:15,113][105692] Updated weights for policy 0, policy_version 1466280 (0.0011) [2023-12-27 02:04:15,620][105620] Updated weights for policy 1, policy_version 1468613 (0.0008) [2023-12-27 02:04:15,674][105620] Updated weights for policy 1, policy_version 1468623 (0.0007) [2023-12-27 02:04:15,739][105620] Updated weights for policy 1, policy_version 1468633 (0.0010) [2023-12-27 02:04:15,793][105692] Updated weights for policy 0, policy_version 1466290 (0.0009) [2023-12-27 02:04:15,845][105692] Updated weights for policy 0, policy_version 1466300 (0.0005) [2023-12-27 02:04:15,896][105692] Updated weights for policy 0, policy_version 1466310 (0.0009) [2023-12-27 02:04:15,948][105692] Updated weights for policy 0, policy_version 1466320 (0.0010) [2023-12-27 02:04:16,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 751452160. Throughput: 0: 9904.3, 1: 9912.4. Samples: 751416212. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:04:16,063][104569] Avg episode reward: [(0, '9078.041'), (1, '8554.763')] [2023-12-27 02:04:16,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001468640_376020992.pth... [2023-12-27 02:04:16,074][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001466320_375431168.pth... [2023-12-27 02:04:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001465168_375136256.pth [2023-12-27 02:04:16,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001467488_375726080.pth [2023-12-27 02:04:16,334][105620] Updated weights for policy 1, policy_version 1468643 (0.0009) [2023-12-27 02:04:16,406][105620] Updated weights for policy 1, policy_version 1468653 (0.0006) [2023-12-27 02:04:16,481][105620] Updated weights for policy 1, policy_version 1468663 (0.0006) [2023-12-27 02:04:16,695][105692] Updated weights for policy 0, policy_version 1466330 (0.0010) [2023-12-27 02:04:16,754][105692] Updated weights for policy 0, policy_version 1466340 (0.0007) [2023-12-27 02:04:16,807][105692] Updated weights for policy 0, policy_version 1466350 (0.0006) [2023-12-27 02:04:17,046][105620] Updated weights for policy 1, policy_version 1468673 (0.0007) [2023-12-27 02:04:17,114][105620] Updated weights for policy 1, policy_version 1468683 (0.0010) [2023-12-27 02:04:17,162][105620] Updated weights for policy 1, policy_version 1468693 (0.0010) [2023-12-27 02:04:17,230][105620] Updated weights for policy 1, policy_version 1468703 (0.0010) [2023-12-27 02:04:17,437][105692] Updated weights for policy 0, policy_version 1466360 (0.0006) [2023-12-27 02:04:17,501][105692] Updated weights for policy 0, policy_version 1466370 (0.0007) [2023-12-27 02:04:17,552][105692] Updated weights for policy 0, policy_version 1466380 (0.0007) [2023-12-27 02:04:17,953][105620] Updated weights for policy 1, policy_version 1468713 (0.0010) [2023-12-27 02:04:18,012][105620] Updated weights for policy 1, policy_version 1468723 (0.0005) [2023-12-27 02:04:18,070][105620] Updated weights for policy 1, policy_version 1468733 (0.0008) [2023-12-27 02:04:18,145][105692] Updated weights for policy 0, policy_version 1466390 (0.0008) [2023-12-27 02:04:18,195][105692] Updated weights for policy 0, policy_version 1466400 (0.0007) [2023-12-27 02:04:18,244][105692] Updated weights for policy 0, policy_version 1466410 (0.0007) [2023-12-27 02:04:18,779][105620] Updated weights for policy 1, policy_version 1468743 (0.0010) [2023-12-27 02:04:18,839][105620] Updated weights for policy 1, policy_version 1468753 (0.0011) [2023-12-27 02:04:18,895][105620] Updated weights for policy 1, policy_version 1468763 (0.0011) [2023-12-27 02:04:18,956][105692] Updated weights for policy 0, policy_version 1466420 (0.0007) [2023-12-27 02:04:19,001][105692] Updated weights for policy 0, policy_version 1466430 (0.0008) [2023-12-27 02:04:19,065][105692] Updated weights for policy 0, policy_version 1466440 (0.0007) [2023-12-27 02:04:19,631][105620] Updated weights for policy 1, policy_version 1468773 (0.0008) [2023-12-27 02:04:19,692][105620] Updated weights for policy 1, policy_version 1468783 (0.0006) [2023-12-27 02:04:19,752][105620] Updated weights for policy 1, policy_version 1468793 (0.0006) [2023-12-27 02:04:19,887][105692] Updated weights for policy 0, policy_version 1466450 (0.0008) [2023-12-27 02:04:19,953][105692] Updated weights for policy 0, policy_version 1466460 (0.0008) [2023-12-27 02:04:20,011][105692] Updated weights for policy 0, policy_version 1466470 (0.0006) [2023-12-27 02:04:20,070][105692] Updated weights for policy 0, policy_version 1466480 (0.0006) [2023-12-27 02:04:20,424][105620] Updated weights for policy 1, policy_version 1468803 (0.0009) [2023-12-27 02:04:20,492][105620] Updated weights for policy 1, policy_version 1468813 (0.0007) [2023-12-27 02:04:20,551][105620] Updated weights for policy 1, policy_version 1468823 (0.0009) [2023-12-27 02:04:20,822][105692] Updated weights for policy 0, policy_version 1466490 (0.0009) [2023-12-27 02:04:20,890][105692] Updated weights for policy 0, policy_version 1466500 (0.0010) [2023-12-27 02:04:20,957][105692] Updated weights for policy 0, policy_version 1466510 (0.0009) [2023-12-27 02:04:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 751550464. Throughput: 0: 9881.3, 1: 9927.8. Samples: 751538452. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:04:21,063][104569] Avg episode reward: [(0, '8898.157'), (1, '8940.192')] [2023-12-27 02:04:21,298][105620] Updated weights for policy 1, policy_version 1468833 (0.0009) [2023-12-27 02:04:21,370][105620] Updated weights for policy 1, policy_version 1468843 (0.0009) [2023-12-27 02:04:21,433][105620] Updated weights for policy 1, policy_version 1468853 (0.0009) [2023-12-27 02:04:21,499][105620] Updated weights for policy 1, policy_version 1468863 (0.0009) [2023-12-27 02:04:21,714][105692] Updated weights for policy 0, policy_version 1466520 (0.0009) [2023-12-27 02:04:21,775][105692] Updated weights for policy 0, policy_version 1466530 (0.0008) [2023-12-27 02:04:21,820][105692] Updated weights for policy 0, policy_version 1466540 (0.0008) [2023-12-27 02:04:22,243][105620] Updated weights for policy 1, policy_version 1468873 (0.0010) [2023-12-27 02:04:22,307][105620] Updated weights for policy 1, policy_version 1468883 (0.0007) [2023-12-27 02:04:22,375][105620] Updated weights for policy 1, policy_version 1468893 (0.0008) [2023-12-27 02:04:22,587][105692] Updated weights for policy 0, policy_version 1466550 (0.0009) [2023-12-27 02:04:22,656][105692] Updated weights for policy 0, policy_version 1466560 (0.0010) [2023-12-27 02:04:22,708][105692] Updated weights for policy 0, policy_version 1466570 (0.0010) [2023-12-27 02:04:23,042][105620] Updated weights for policy 1, policy_version 1468903 (0.0007) [2023-12-27 02:04:23,109][105620] Updated weights for policy 1, policy_version 1468913 (0.0006) [2023-12-27 02:04:23,157][105620] Updated weights for policy 1, policy_version 1468923 (0.0005) [2023-12-27 02:04:23,541][105692] Updated weights for policy 0, policy_version 1466580 (0.0009) [2023-12-27 02:04:23,599][105692] Updated weights for policy 0, policy_version 1466590 (0.0008) [2023-12-27 02:04:23,650][105692] Updated weights for policy 0, policy_version 1466600 (0.0007) [2023-12-27 02:04:23,861][105620] Updated weights for policy 1, policy_version 1468933 (0.0010) [2023-12-27 02:04:23,909][105620] Updated weights for policy 1, policy_version 1468943 (0.0010) [2023-12-27 02:04:23,960][105620] Updated weights for policy 1, policy_version 1468953 (0.0010) [2023-12-27 02:04:24,358][105692] Updated weights for policy 0, policy_version 1466610 (0.0008) [2023-12-27 02:04:24,424][105692] Updated weights for policy 0, policy_version 1466620 (0.0008) [2023-12-27 02:04:24,480][105692] Updated weights for policy 0, policy_version 1466630 (0.0009) [2023-12-27 02:04:24,662][105620] Updated weights for policy 1, policy_version 1468963 (0.0010) [2023-12-27 02:04:24,720][105620] Updated weights for policy 1, policy_version 1468973 (0.0007) [2023-12-27 02:04:24,775][105620] Updated weights for policy 1, policy_version 1468983 (0.0008) [2023-12-27 02:04:25,326][105692] Updated weights for policy 0, policy_version 1466641 (0.0010) [2023-12-27 02:04:25,373][105620] Updated weights for policy 1, policy_version 1468993 (0.0010) [2023-12-27 02:04:25,386][105692] Updated weights for policy 0, policy_version 1466651 (0.0007) [2023-12-27 02:04:25,432][105620] Updated weights for policy 1, policy_version 1469003 (0.0011) [2023-12-27 02:04:25,439][105692] Updated weights for policy 0, policy_version 1466661 (0.0007) [2023-12-27 02:04:25,491][105620] Updated weights for policy 1, policy_version 1469013 (0.0010) [2023-12-27 02:04:25,497][105692] Updated weights for policy 0, policy_version 1466671 (0.0008) [2023-12-27 02:04:25,550][105620] Updated weights for policy 1, policy_version 1469023 (0.0010) [2023-12-27 02:04:26,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 751640576. Throughput: 0: 9824.2, 1: 9879.8. Samples: 751650804. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:04:26,063][104569] Avg episode reward: [(0, '8346.905'), (1, '8864.052')] [2023-12-27 02:04:26,234][105692] Updated weights for policy 0, policy_version 1466681 (0.0005) [2023-12-27 02:04:26,285][105692] Updated weights for policy 0, policy_version 1466691 (0.0005) [2023-12-27 02:04:26,291][105620] Updated weights for policy 1, policy_version 1469033 (0.0010) [2023-12-27 02:04:26,341][105692] Updated weights for policy 0, policy_version 1466701 (0.0006) [2023-12-27 02:04:26,352][105620] Updated weights for policy 1, policy_version 1469043 (0.0006) [2023-12-27 02:04:26,404][105620] Updated weights for policy 1, policy_version 1469053 (0.0008) [2023-12-27 02:04:27,032][105620] Updated weights for policy 1, policy_version 1469063 (0.0010) [2023-12-27 02:04:27,038][105692] Updated weights for policy 0, policy_version 1466711 (0.0006) [2023-12-27 02:04:27,088][105692] Updated weights for policy 0, policy_version 1466721 (0.0005) [2023-12-27 02:04:27,090][105620] Updated weights for policy 1, policy_version 1469073 (0.0010) [2023-12-27 02:04:27,141][105692] Updated weights for policy 0, policy_version 1466731 (0.0005) [2023-12-27 02:04:27,152][105620] Updated weights for policy 1, policy_version 1469083 (0.0010) [2023-12-27 02:04:27,747][105620] Updated weights for policy 1, policy_version 1469093 (0.0007) [2023-12-27 02:04:27,798][105620] Updated weights for policy 1, policy_version 1469103 (0.0005) [2023-12-27 02:04:27,853][105620] Updated weights for policy 1, policy_version 1469113 (0.0005) [2023-12-27 02:04:27,912][105692] Updated weights for policy 0, policy_version 1466741 (0.0009) [2023-12-27 02:04:27,973][105692] Updated weights for policy 0, policy_version 1466751 (0.0009) [2023-12-27 02:04:28,034][105692] Updated weights for policy 0, policy_version 1466761 (0.0009) [2023-12-27 02:04:28,399][105620] Updated weights for policy 1, policy_version 1469123 (0.0007) [2023-12-27 02:04:28,450][105620] Updated weights for policy 1, policy_version 1469133 (0.0010) [2023-12-27 02:04:28,502][105620] Updated weights for policy 1, policy_version 1469143 (0.0010) [2023-12-27 02:04:28,882][105692] Updated weights for policy 0, policy_version 1466771 (0.0010) [2023-12-27 02:04:28,952][105692] Updated weights for policy 0, policy_version 1466781 (0.0010) [2023-12-27 02:04:29,009][105692] Updated weights for policy 0, policy_version 1466791 (0.0010) [2023-12-27 02:04:29,122][105620] Updated weights for policy 1, policy_version 1469153 (0.0010) [2023-12-27 02:04:29,181][105620] Updated weights for policy 1, policy_version 1469163 (0.0008) [2023-12-27 02:04:29,246][105620] Updated weights for policy 1, policy_version 1469173 (0.0009) [2023-12-27 02:04:29,300][105620] Updated weights for policy 1, policy_version 1469183 (0.0008) [2023-12-27 02:04:29,701][105692] Updated weights for policy 0, policy_version 1466801 (0.0009) [2023-12-27 02:04:29,763][105692] Updated weights for policy 0, policy_version 1466811 (0.0007) [2023-12-27 02:04:29,818][105692] Updated weights for policy 0, policy_version 1466821 (0.0008) [2023-12-27 02:04:29,881][105692] Updated weights for policy 0, policy_version 1466831 (0.0009) [2023-12-27 02:04:30,003][105620] Updated weights for policy 1, policy_version 1469193 (0.0010) [2023-12-27 02:04:30,051][105620] Updated weights for policy 1, policy_version 1469203 (0.0010) [2023-12-27 02:04:30,109][105620] Updated weights for policy 1, policy_version 1469213 (0.0010) [2023-12-27 02:04:30,646][105692] Updated weights for policy 0, policy_version 1466841 (0.0008) [2023-12-27 02:04:30,700][105692] Updated weights for policy 0, policy_version 1466851 (0.0008) [2023-12-27 02:04:30,747][105692] Updated weights for policy 0, policy_version 1466861 (0.0008) [2023-12-27 02:04:30,847][105620] Updated weights for policy 1, policy_version 1469223 (0.0010) [2023-12-27 02:04:30,894][105620] Updated weights for policy 1, policy_version 1469233 (0.0010) [2023-12-27 02:04:30,951][105620] Updated weights for policy 1, policy_version 1469243 (0.0010) [2023-12-27 02:04:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 751747072. Throughput: 0: 9780.4, 1: 9944.6. Samples: 751712264. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:04:31,062][104569] Avg episode reward: [(0, '8345.713'), (1, '8989.643')] [2023-12-27 02:04:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001466864_375570432.pth... [2023-12-27 02:04:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001469248_376176640.pth... [2023-12-27 02:04:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001468096_375881728.pth [2023-12-27 02:04:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001465744_375283712.pth [2023-12-27 02:04:31,516][105692] Updated weights for policy 0, policy_version 1466871 (0.0009) [2023-12-27 02:04:31,580][105692] Updated weights for policy 0, policy_version 1466881 (0.0008) [2023-12-27 02:04:31,651][105692] Updated weights for policy 0, policy_version 1466891 (0.0009) [2023-12-27 02:04:31,700][105620] Updated weights for policy 1, policy_version 1469253 (0.0009) [2023-12-27 02:04:31,758][105620] Updated weights for policy 1, policy_version 1469263 (0.0005) [2023-12-27 02:04:31,809][105620] Updated weights for policy 1, policy_version 1469273 (0.0005) [2023-12-27 02:04:32,419][105692] Updated weights for policy 0, policy_version 1466901 (0.0008) [2023-12-27 02:04:32,475][105692] Updated weights for policy 0, policy_version 1466911 (0.0009) [2023-12-27 02:04:32,476][105620] Updated weights for policy 1, policy_version 1469283 (0.0005) [2023-12-27 02:04:32,530][105620] Updated weights for policy 1, policy_version 1469293 (0.0005) [2023-12-27 02:04:32,544][105692] Updated weights for policy 0, policy_version 1466921 (0.0008) [2023-12-27 02:04:32,586][105620] Updated weights for policy 1, policy_version 1469303 (0.0006) [2023-12-27 02:04:33,112][105692] Updated weights for policy 0, policy_version 1466931 (0.0008) [2023-12-27 02:04:33,176][105692] Updated weights for policy 0, policy_version 1466941 (0.0008) [2023-12-27 02:04:33,181][105620] Updated weights for policy 1, policy_version 1469313 (0.0006) [2023-12-27 02:04:33,234][105620] Updated weights for policy 1, policy_version 1469323 (0.0006) [2023-12-27 02:04:33,237][105692] Updated weights for policy 0, policy_version 1466951 (0.0008) [2023-12-27 02:04:33,288][105620] Updated weights for policy 1, policy_version 1469333 (0.0008) [2023-12-27 02:04:33,344][105620] Updated weights for policy 1, policy_version 1469343 (0.0010) [2023-12-27 02:04:33,779][105692] Updated weights for policy 0, policy_version 1466961 (0.0005) [2023-12-27 02:04:33,834][105692] Updated weights for policy 0, policy_version 1466971 (0.0005) [2023-12-27 02:04:33,881][105692] Updated weights for policy 0, policy_version 1466981 (0.0005) [2023-12-27 02:04:33,922][105692] Updated weights for policy 0, policy_version 1466991 (0.0005) [2023-12-27 02:04:34,176][105620] Updated weights for policy 1, policy_version 1469353 (0.0006) [2023-12-27 02:04:34,239][105620] Updated weights for policy 1, policy_version 1469363 (0.0006) [2023-12-27 02:04:34,302][105620] Updated weights for policy 1, policy_version 1469373 (0.0009) [2023-12-27 02:04:34,518][105692] Updated weights for policy 0, policy_version 1467001 (0.0006) [2023-12-27 02:04:34,578][105692] Updated weights for policy 0, policy_version 1467011 (0.0006) [2023-12-27 02:04:34,644][105692] Updated weights for policy 0, policy_version 1467021 (0.0007) [2023-12-27 02:04:35,120][105620] Updated weights for policy 1, policy_version 1469383 (0.0008) [2023-12-27 02:04:35,179][105620] Updated weights for policy 1, policy_version 1469393 (0.0007) [2023-12-27 02:04:35,219][105692] Updated weights for policy 0, policy_version 1467031 (0.0007) [2023-12-27 02:04:35,230][105620] Updated weights for policy 1, policy_version 1469403 (0.0008) [2023-12-27 02:04:35,278][105692] Updated weights for policy 0, policy_version 1467041 (0.0005) [2023-12-27 02:04:35,338][105692] Updated weights for policy 0, policy_version 1467051 (0.0005) [2023-12-27 02:04:35,930][105692] Updated weights for policy 0, policy_version 1467061 (0.0008) [2023-12-27 02:04:35,979][105692] Updated weights for policy 0, policy_version 1467071 (0.0010) [2023-12-27 02:04:36,031][105692] Updated weights for policy 0, policy_version 1467081 (0.0010) [2023-12-27 02:04:36,042][105620] Updated weights for policy 1, policy_version 1469413 (0.0008) [2023-12-27 02:04:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 751837184. Throughput: 0: 9750.4, 1: 9920.2. Samples: 751832084. Policy #0 lag: (min: 27.0, avg: 35.1, max: 59.0) [2023-12-27 02:04:36,063][104569] Avg episode reward: [(0, '8255.854'), (1, '8901.503')] [2023-12-27 02:04:36,102][105620] Updated weights for policy 1, policy_version 1469423 (0.0008) [2023-12-27 02:04:36,177][105620] Updated weights for policy 1, policy_version 1469433 (0.0007) [2023-12-27 02:04:36,814][105692] Updated weights for policy 0, policy_version 1467091 (0.0011) [2023-12-27 02:04:36,872][105692] Updated weights for policy 0, policy_version 1467101 (0.0010) [2023-12-27 02:04:36,921][105692] Updated weights for policy 0, policy_version 1467111 (0.0010) [2023-12-27 02:04:36,923][105620] Updated weights for policy 1, policy_version 1469443 (0.0008) [2023-12-27 02:04:36,979][105620] Updated weights for policy 1, policy_version 1469453 (0.0007) [2023-12-27 02:04:37,037][105620] Updated weights for policy 1, policy_version 1469463 (0.0008) [2023-12-27 02:04:37,673][105692] Updated weights for policy 0, policy_version 1467121 (0.0010) [2023-12-27 02:04:37,721][105620] Updated weights for policy 1, policy_version 1469473 (0.0009) [2023-12-27 02:04:37,733][105692] Updated weights for policy 0, policy_version 1467131 (0.0008) [2023-12-27 02:04:37,774][105620] Updated weights for policy 1, policy_version 1469483 (0.0009) [2023-12-27 02:04:37,796][105692] Updated weights for policy 0, policy_version 1467141 (0.0010) [2023-12-27 02:04:37,821][105620] Updated weights for policy 1, policy_version 1469493 (0.0007) [2023-12-27 02:04:37,855][105692] Updated weights for policy 0, policy_version 1467151 (0.0011) [2023-12-27 02:04:37,882][105620] Updated weights for policy 1, policy_version 1469503 (0.0006) [2023-12-27 02:04:38,553][105692] Updated weights for policy 0, policy_version 1467161 (0.0010) [2023-12-27 02:04:38,616][105692] Updated weights for policy 0, policy_version 1467171 (0.0008) [2023-12-27 02:04:38,616][105620] Updated weights for policy 1, policy_version 1469513 (0.0006) [2023-12-27 02:04:38,676][105692] Updated weights for policy 0, policy_version 1467181 (0.0006) [2023-12-27 02:04:38,680][105620] Updated weights for policy 1, policy_version 1469523 (0.0007) [2023-12-27 02:04:38,757][105620] Updated weights for policy 1, policy_version 1469533 (0.0010) [2023-12-27 02:04:39,237][105692] Updated weights for policy 0, policy_version 1467191 (0.0007) [2023-12-27 02:04:39,299][105692] Updated weights for policy 0, policy_version 1467201 (0.0009) [2023-12-27 02:04:39,361][105692] Updated weights for policy 0, policy_version 1467211 (0.0008) [2023-12-27 02:04:39,545][105620] Updated weights for policy 1, policy_version 1469543 (0.0009) [2023-12-27 02:04:39,598][105620] Updated weights for policy 1, policy_version 1469553 (0.0006) [2023-12-27 02:04:39,654][105620] Updated weights for policy 1, policy_version 1469563 (0.0008) [2023-12-27 02:04:40,126][105692] Updated weights for policy 0, policy_version 1467221 (0.0009) [2023-12-27 02:04:40,189][105692] Updated weights for policy 0, policy_version 1467231 (0.0011) [2023-12-27 02:04:40,245][105692] Updated weights for policy 0, policy_version 1467241 (0.0011) [2023-12-27 02:04:40,403][105620] Updated weights for policy 1, policy_version 1469573 (0.0009) [2023-12-27 02:04:40,465][105620] Updated weights for policy 1, policy_version 1469583 (0.0008) [2023-12-27 02:04:40,532][105620] Updated weights for policy 1, policy_version 1469593 (0.0007) [2023-12-27 02:04:41,018][105692] Updated weights for policy 0, policy_version 1467251 (0.0009) [2023-12-27 02:04:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 751935488. Throughput: 0: 9759.5, 1: 9852.1. Samples: 751948612. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:04:41,062][104569] Avg episode reward: [(0, '8441.166'), (1, '8899.888')] [2023-12-27 02:04:41,087][105692] Updated weights for policy 0, policy_version 1467261 (0.0009) [2023-12-27 02:04:41,143][105692] Updated weights for policy 0, policy_version 1467271 (0.0010) [2023-12-27 02:04:41,154][105620] Updated weights for policy 1, policy_version 1469603 (0.0008) [2023-12-27 02:04:41,214][105620] Updated weights for policy 1, policy_version 1469613 (0.0007) [2023-12-27 02:04:41,282][105620] Updated weights for policy 1, policy_version 1469623 (0.0008) [2023-12-27 02:04:41,934][105692] Updated weights for policy 0, policy_version 1467281 (0.0009) [2023-12-27 02:04:41,992][105692] Updated weights for policy 0, policy_version 1467291 (0.0005) [2023-12-27 02:04:42,017][105620] Updated weights for policy 1, policy_version 1469633 (0.0007) [2023-12-27 02:04:42,043][105692] Updated weights for policy 0, policy_version 1467301 (0.0005) [2023-12-27 02:04:42,080][105620] Updated weights for policy 1, policy_version 1469643 (0.0007) [2023-12-27 02:04:42,103][105692] Updated weights for policy 0, policy_version 1467311 (0.0011) [2023-12-27 02:04:42,139][105620] Updated weights for policy 1, policy_version 1469653 (0.0007) [2023-12-27 02:04:42,194][105620] Updated weights for policy 1, policy_version 1469663 (0.0008) [2023-12-27 02:04:42,775][105692] Updated weights for policy 0, policy_version 1467321 (0.0011) [2023-12-27 02:04:42,827][105692] Updated weights for policy 0, policy_version 1467331 (0.0011) [2023-12-27 02:04:42,875][105692] Updated weights for policy 0, policy_version 1467341 (0.0010) [2023-12-27 02:04:42,950][105620] Updated weights for policy 1, policy_version 1469673 (0.0008) [2023-12-27 02:04:43,001][105620] Updated weights for policy 1, policy_version 1469683 (0.0006) [2023-12-27 02:04:43,055][105620] Updated weights for policy 1, policy_version 1469693 (0.0005) [2023-12-27 02:04:43,583][105692] Updated weights for policy 0, policy_version 1467351 (0.0010) [2023-12-27 02:04:43,647][105692] Updated weights for policy 0, policy_version 1467361 (0.0010) [2023-12-27 02:04:43,676][105620] Updated weights for policy 1, policy_version 1469703 (0.0006) [2023-12-27 02:04:43,702][105692] Updated weights for policy 0, policy_version 1467371 (0.0010) [2023-12-27 02:04:43,737][105620] Updated weights for policy 1, policy_version 1469713 (0.0006) [2023-12-27 02:04:43,797][105620] Updated weights for policy 1, policy_version 1469723 (0.0008) [2023-12-27 02:04:44,440][105692] Updated weights for policy 0, policy_version 1467381 (0.0010) [2023-12-27 02:04:44,502][105692] Updated weights for policy 0, policy_version 1467391 (0.0010) [2023-12-27 02:04:44,534][105620] Updated weights for policy 1, policy_version 1469733 (0.0008) [2023-12-27 02:04:44,567][105692] Updated weights for policy 0, policy_version 1467401 (0.0010) [2023-12-27 02:04:44,598][105620] Updated weights for policy 1, policy_version 1469743 (0.0009) [2023-12-27 02:04:44,652][105620] Updated weights for policy 1, policy_version 1469753 (0.0009) [2023-12-27 02:04:45,306][105692] Updated weights for policy 0, policy_version 1467411 (0.0010) [2023-12-27 02:04:45,355][105692] Updated weights for policy 0, policy_version 1467421 (0.0010) [2023-12-27 02:04:45,404][105620] Updated weights for policy 1, policy_version 1469763 (0.0009) [2023-12-27 02:04:45,414][105692] Updated weights for policy 0, policy_version 1467431 (0.0008) [2023-12-27 02:04:45,467][105620] Updated weights for policy 1, policy_version 1469773 (0.0011) [2023-12-27 02:04:45,527][105620] Updated weights for policy 1, policy_version 1469783 (0.0011) [2023-12-27 02:04:46,062][104569] Fps is (10 sec: 19659.7, 60 sec: 19524.1, 300 sec: 19521.9). Total num frames: 752033792. Throughput: 0: 9776.6, 1: 9780.9. Samples: 752007192. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:04:46,064][104569] Avg episode reward: [(0, '8624.166'), (1, '8626.088')] [2023-12-27 02:04:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001469792_376315904.pth... [2023-12-27 02:04:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001468640_376020992.pth [2023-12-27 02:04:46,075][105692] Updated weights for policy 0, policy_version 1467441 (0.0006) [2023-12-27 02:04:46,126][105692] Updated weights for policy 0, policy_version 1467451 (0.0010) [2023-12-27 02:04:46,190][105692] Updated weights for policy 0, policy_version 1467461 (0.0010) [2023-12-27 02:04:46,245][105692] Updated weights for policy 0, policy_version 1467471 (0.0010) [2023-12-27 02:04:46,245][105620] Updated weights for policy 1, policy_version 1469793 (0.0011) [2023-12-27 02:04:46,248][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001467472_375726080.pth... [2023-12-27 02:04:46,251][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001466320_375431168.pth [2023-12-27 02:04:46,300][105620] Updated weights for policy 1, policy_version 1469803 (0.0008) [2023-12-27 02:04:46,358][105620] Updated weights for policy 1, policy_version 1469813 (0.0008) [2023-12-27 02:04:46,409][105620] Updated weights for policy 1, policy_version 1469823 (0.0007) [2023-12-27 02:04:46,980][105692] Updated weights for policy 0, policy_version 1467481 (0.0008) [2023-12-27 02:04:47,028][105692] Updated weights for policy 0, policy_version 1467491 (0.0005) [2023-12-27 02:04:47,085][105692] Updated weights for policy 0, policy_version 1467501 (0.0005) [2023-12-27 02:04:47,162][105620] Updated weights for policy 1, policy_version 1469833 (0.0011) [2023-12-27 02:04:47,220][105620] Updated weights for policy 1, policy_version 1469843 (0.0010) [2023-12-27 02:04:47,279][105620] Updated weights for policy 1, policy_version 1469853 (0.0010) [2023-12-27 02:04:47,629][105692] Updated weights for policy 0, policy_version 1467511 (0.0005) [2023-12-27 02:04:47,677][105692] Updated weights for policy 0, policy_version 1467521 (0.0005) [2023-12-27 02:04:47,730][105692] Updated weights for policy 0, policy_version 1467531 (0.0005) [2023-12-27 02:04:47,980][105620] Updated weights for policy 1, policy_version 1469863 (0.0009) [2023-12-27 02:04:48,050][105620] Updated weights for policy 1, policy_version 1469873 (0.0010) [2023-12-27 02:04:48,117][105620] Updated weights for policy 1, policy_version 1469883 (0.0008) [2023-12-27 02:04:48,289][105692] Updated weights for policy 0, policy_version 1467541 (0.0006) [2023-12-27 02:04:48,356][105692] Updated weights for policy 0, policy_version 1467551 (0.0008) [2023-12-27 02:04:48,410][105692] Updated weights for policy 0, policy_version 1467561 (0.0007) [2023-12-27 02:04:48,848][105620] Updated weights for policy 1, policy_version 1469893 (0.0007) [2023-12-27 02:04:48,906][105620] Updated weights for policy 1, policy_version 1469903 (0.0008) [2023-12-27 02:04:48,972][105620] Updated weights for policy 1, policy_version 1469913 (0.0008) [2023-12-27 02:04:49,095][105692] Updated weights for policy 0, policy_version 1467571 (0.0005) [2023-12-27 02:04:49,151][105692] Updated weights for policy 0, policy_version 1467581 (0.0006) [2023-12-27 02:04:49,211][105692] Updated weights for policy 0, policy_version 1467591 (0.0006) [2023-12-27 02:04:49,784][105620] Updated weights for policy 1, policy_version 1469923 (0.0008) [2023-12-27 02:04:49,843][105620] Updated weights for policy 1, policy_version 1469933 (0.0008) [2023-12-27 02:04:49,883][105692] Updated weights for policy 0, policy_version 1467601 (0.0009) [2023-12-27 02:04:49,907][105620] Updated weights for policy 1, policy_version 1469943 (0.0008) [2023-12-27 02:04:49,948][105692] Updated weights for policy 0, policy_version 1467611 (0.0007) [2023-12-27 02:04:50,011][105692] Updated weights for policy 0, policy_version 1467621 (0.0008) [2023-12-27 02:04:50,063][105692] Updated weights for policy 0, policy_version 1467631 (0.0009) [2023-12-27 02:04:50,680][105620] Updated weights for policy 1, policy_version 1469953 (0.0009) [2023-12-27 02:04:50,741][105620] Updated weights for policy 1, policy_version 1469963 (0.0008) [2023-12-27 02:04:50,756][105692] Updated weights for policy 0, policy_version 1467641 (0.0006) [2023-12-27 02:04:50,792][105620] Updated weights for policy 1, policy_version 1469973 (0.0007) [2023-12-27 02:04:50,818][105692] Updated weights for policy 0, policy_version 1467651 (0.0008) [2023-12-27 02:04:50,842][105620] Updated weights for policy 1, policy_version 1469983 (0.0006) [2023-12-27 02:04:50,886][105692] Updated weights for policy 0, policy_version 1467661 (0.0007) [2023-12-27 02:04:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 752140288. Throughput: 0: 9876.3, 1: 9775.7. Samples: 752125168. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:04:51,062][104569] Avg episode reward: [(0, '8530.040'), (1, '8625.252')] [2023-12-27 02:04:51,586][105692] Updated weights for policy 0, policy_version 1467671 (0.0007) [2023-12-27 02:04:51,648][105692] Updated weights for policy 0, policy_version 1467681 (0.0008) [2023-12-27 02:04:51,690][105620] Updated weights for policy 1, policy_version 1469993 (0.0008) [2023-12-27 02:04:51,706][105692] Updated weights for policy 0, policy_version 1467691 (0.0008) [2023-12-27 02:04:51,768][105620] Updated weights for policy 1, policy_version 1470003 (0.0008) [2023-12-27 02:04:51,824][105620] Updated weights for policy 1, policy_version 1470013 (0.0009) [2023-12-27 02:04:52,503][105692] Updated weights for policy 0, policy_version 1467701 (0.0008) [2023-12-27 02:04:52,561][105692] Updated weights for policy 0, policy_version 1467711 (0.0009) [2023-12-27 02:04:52,580][105620] Updated weights for policy 1, policy_version 1470023 (0.0008) [2023-12-27 02:04:52,619][105692] Updated weights for policy 0, policy_version 1467721 (0.0006) [2023-12-27 02:04:52,641][105620] Updated weights for policy 1, policy_version 1470033 (0.0008) [2023-12-27 02:04:52,700][105620] Updated weights for policy 1, policy_version 1470043 (0.0008) [2023-12-27 02:04:53,351][105620] Updated weights for policy 1, policy_version 1470053 (0.0009) [2023-12-27 02:04:53,405][105692] Updated weights for policy 0, policy_version 1467731 (0.0006) [2023-12-27 02:04:53,411][105620] Updated weights for policy 1, policy_version 1470063 (0.0008) [2023-12-27 02:04:53,455][105692] Updated weights for policy 0, policy_version 1467741 (0.0010) [2023-12-27 02:04:53,470][105620] Updated weights for policy 1, policy_version 1470073 (0.0007) [2023-12-27 02:04:53,504][105692] Updated weights for policy 0, policy_version 1467751 (0.0010) [2023-12-27 02:04:54,191][105692] Updated weights for policy 0, policy_version 1467761 (0.0010) [2023-12-27 02:04:54,209][105620] Updated weights for policy 1, policy_version 1470083 (0.0006) [2023-12-27 02:04:54,243][105692] Updated weights for policy 0, policy_version 1467771 (0.0010) [2023-12-27 02:04:54,267][105620] Updated weights for policy 1, policy_version 1470093 (0.0007) [2023-12-27 02:04:54,288][105692] Updated weights for policy 0, policy_version 1467781 (0.0010) [2023-12-27 02:04:54,322][105620] Updated weights for policy 1, policy_version 1470103 (0.0006) [2023-12-27 02:04:54,337][105692] Updated weights for policy 0, policy_version 1467791 (0.0010) [2023-12-27 02:04:55,016][105692] Updated weights for policy 0, policy_version 1467801 (0.0011) [2023-12-27 02:04:55,076][105692] Updated weights for policy 0, policy_version 1467811 (0.0011) [2023-12-27 02:04:55,080][105620] Updated weights for policy 1, policy_version 1470113 (0.0007) [2023-12-27 02:04:55,128][105692] Updated weights for policy 0, policy_version 1467821 (0.0010) [2023-12-27 02:04:55,144][105620] Updated weights for policy 1, policy_version 1470123 (0.0005) [2023-12-27 02:04:55,209][105620] Updated weights for policy 1, policy_version 1470133 (0.0008) [2023-12-27 02:04:55,270][105620] Updated weights for policy 1, policy_version 1470143 (0.0010) [2023-12-27 02:04:55,747][105692] Updated weights for policy 0, policy_version 1467831 (0.0007) [2023-12-27 02:04:55,809][105692] Updated weights for policy 0, policy_version 1467841 (0.0010) [2023-12-27 02:04:55,862][105692] Updated weights for policy 0, policy_version 1467851 (0.0010) [2023-12-27 02:04:55,915][105620] Updated weights for policy 1, policy_version 1470153 (0.0008) [2023-12-27 02:04:55,967][105620] Updated weights for policy 1, policy_version 1470163 (0.0010) [2023-12-27 02:04:56,026][105620] Updated weights for policy 1, policy_version 1470173 (0.0010) [2023-12-27 02:04:56,062][104569] Fps is (10 sec: 20481.2, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 752238592. Throughput: 0: 9853.6, 1: 9722.0. Samples: 752240932. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:04:56,063][104569] Avg episode reward: [(0, '8259.296'), (1, '9081.100')] [2023-12-27 02:04:56,489][105692] Updated weights for policy 0, policy_version 1467861 (0.0008) [2023-12-27 02:04:56,544][105692] Updated weights for policy 0, policy_version 1467871 (0.0008) [2023-12-27 02:04:56,600][105692] Updated weights for policy 0, policy_version 1467881 (0.0010) [2023-12-27 02:04:56,634][105620] Updated weights for policy 1, policy_version 1470183 (0.0007) [2023-12-27 02:04:56,695][105620] Updated weights for policy 1, policy_version 1470193 (0.0005) [2023-12-27 02:04:56,758][105620] Updated weights for policy 1, policy_version 1470203 (0.0005) [2023-12-27 02:04:57,184][105692] Updated weights for policy 0, policy_version 1467891 (0.0008) [2023-12-27 02:04:57,243][105692] Updated weights for policy 0, policy_version 1467901 (0.0006) [2023-12-27 02:04:57,291][105692] Updated weights for policy 0, policy_version 1467911 (0.0010) [2023-12-27 02:04:57,368][105620] Updated weights for policy 1, policy_version 1470213 (0.0008) [2023-12-27 02:04:57,423][105620] Updated weights for policy 1, policy_version 1470223 (0.0010) [2023-12-27 02:04:57,483][105620] Updated weights for policy 1, policy_version 1470233 (0.0010) [2023-12-27 02:04:57,996][105692] Updated weights for policy 0, policy_version 1467921 (0.0008) [2023-12-27 02:04:58,066][105692] Updated weights for policy 0, policy_version 1467931 (0.0010) [2023-12-27 02:04:58,123][105692] Updated weights for policy 0, policy_version 1467941 (0.0008) [2023-12-27 02:04:58,187][105692] Updated weights for policy 0, policy_version 1467951 (0.0011) [2023-12-27 02:04:58,223][105620] Updated weights for policy 1, policy_version 1470243 (0.0010) [2023-12-27 02:04:58,281][105620] Updated weights for policy 1, policy_version 1470253 (0.0010) [2023-12-27 02:04:58,347][105620] Updated weights for policy 1, policy_version 1470263 (0.0011) [2023-12-27 02:04:58,951][105692] Updated weights for policy 0, policy_version 1467961 (0.0010) [2023-12-27 02:04:59,014][105692] Updated weights for policy 0, policy_version 1467971 (0.0010) [2023-12-27 02:04:59,063][105692] Updated weights for policy 0, policy_version 1467981 (0.0011) [2023-12-27 02:04:59,134][105620] Updated weights for policy 1, policy_version 1470273 (0.0009) [2023-12-27 02:04:59,199][105620] Updated weights for policy 1, policy_version 1470283 (0.0008) [2023-12-27 02:04:59,272][105620] Updated weights for policy 1, policy_version 1470293 (0.0008) [2023-12-27 02:04:59,340][105620] Updated weights for policy 1, policy_version 1470303 (0.0008) [2023-12-27 02:04:59,886][105692] Updated weights for policy 0, policy_version 1467991 (0.0008) [2023-12-27 02:04:59,954][105692] Updated weights for policy 0, policy_version 1468001 (0.0009) [2023-12-27 02:05:00,022][105692] Updated weights for policy 0, policy_version 1468011 (0.0007) [2023-12-27 02:05:00,070][105620] Updated weights for policy 1, policy_version 1470313 (0.0009) [2023-12-27 02:05:00,123][105620] Updated weights for policy 1, policy_version 1470323 (0.0008) [2023-12-27 02:05:00,171][105620] Updated weights for policy 1, policy_version 1470333 (0.0008) [2023-12-27 02:05:00,610][105692] Updated weights for policy 0, policy_version 1468021 (0.0005) [2023-12-27 02:05:00,660][105692] Updated weights for policy 0, policy_version 1468031 (0.0008) [2023-12-27 02:05:00,714][105692] Updated weights for policy 0, policy_version 1468041 (0.0009) [2023-12-27 02:05:01,000][105620] Updated weights for policy 1, policy_version 1470343 (0.0009) [2023-12-27 02:05:01,061][105620] Updated weights for policy 1, policy_version 1470353 (0.0009) [2023-12-27 02:05:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 752328704. Throughput: 0: 9899.1, 1: 9790.5. Samples: 752302240. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:05:01,062][104569] Avg episode reward: [(0, '8350.727'), (1, '8901.682')] [2023-12-27 02:05:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001468048_375873536.pth... [2023-12-27 02:05:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001466864_375570432.pth [2023-12-27 02:05:01,127][105620] Updated weights for policy 1, policy_version 1470363 (0.0009) [2023-12-27 02:05:01,159][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001470368_376463360.pth... [2023-12-27 02:05:01,163][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001469248_376176640.pth [2023-12-27 02:05:01,463][105692] Updated weights for policy 0, policy_version 1468051 (0.0009) [2023-12-27 02:05:01,529][105692] Updated weights for policy 0, policy_version 1468061 (0.0009) [2023-12-27 02:05:01,595][105692] Updated weights for policy 0, policy_version 1468071 (0.0009) [2023-12-27 02:05:01,825][105620] Updated weights for policy 1, policy_version 1470373 (0.0008) [2023-12-27 02:05:01,872][105620] Updated weights for policy 1, policy_version 1470383 (0.0009) [2023-12-27 02:05:01,922][105620] Updated weights for policy 1, policy_version 1470393 (0.0007) [2023-12-27 02:05:02,389][105692] Updated weights for policy 0, policy_version 1468081 (0.0009) [2023-12-27 02:05:02,451][105692] Updated weights for policy 0, policy_version 1468091 (0.0008) [2023-12-27 02:05:02,506][105692] Updated weights for policy 0, policy_version 1468101 (0.0008) [2023-12-27 02:05:02,561][105692] Updated weights for policy 0, policy_version 1468111 (0.0008) [2023-12-27 02:05:02,621][105620] Updated weights for policy 1, policy_version 1470403 (0.0010) [2023-12-27 02:05:02,676][105620] Updated weights for policy 1, policy_version 1470413 (0.0010) [2023-12-27 02:05:02,728][105620] Updated weights for policy 1, policy_version 1470423 (0.0010) [2023-12-27 02:05:03,249][105692] Updated weights for policy 0, policy_version 1468121 (0.0008) [2023-12-27 02:05:03,312][105692] Updated weights for policy 0, policy_version 1468131 (0.0007) [2023-12-27 02:05:03,326][105620] Updated weights for policy 1, policy_version 1470433 (0.0010) [2023-12-27 02:05:03,372][105692] Updated weights for policy 0, policy_version 1468141 (0.0006) [2023-12-27 02:05:03,380][105620] Updated weights for policy 1, policy_version 1470443 (0.0005) [2023-12-27 02:05:03,438][105620] Updated weights for policy 1, policy_version 1470453 (0.0005) [2023-12-27 02:05:03,506][105620] Updated weights for policy 1, policy_version 1470463 (0.0005) [2023-12-27 02:05:03,908][105692] Updated weights for policy 0, policy_version 1468151 (0.0006) [2023-12-27 02:05:03,964][105692] Updated weights for policy 0, policy_version 1468161 (0.0008) [2023-12-27 02:05:04,023][105692] Updated weights for policy 0, policy_version 1468171 (0.0007) [2023-12-27 02:05:04,034][105620] Updated weights for policy 1, policy_version 1470473 (0.0007) [2023-12-27 02:05:04,095][105620] Updated weights for policy 1, policy_version 1470483 (0.0008) [2023-12-27 02:05:04,155][105620] Updated weights for policy 1, policy_version 1470493 (0.0009) [2023-12-27 02:05:04,766][105692] Updated weights for policy 0, policy_version 1468181 (0.0008) [2023-12-27 02:05:04,813][105692] Updated weights for policy 0, policy_version 1468191 (0.0008) [2023-12-27 02:05:04,874][105692] Updated weights for policy 0, policy_version 1468201 (0.0009) [2023-12-27 02:05:04,925][105620] Updated weights for policy 1, policy_version 1470503 (0.0007) [2023-12-27 02:05:04,974][105620] Updated weights for policy 1, policy_version 1470513 (0.0007) [2023-12-27 02:05:05,028][105620] Updated weights for policy 1, policy_version 1470523 (0.0009) [2023-12-27 02:05:05,498][105692] Updated weights for policy 0, policy_version 1468211 (0.0008) [2023-12-27 02:05:05,527][105585] KL-divergence is very high: 102.0654 [2023-12-27 02:05:05,538][105585] KL-divergence is very high: 124.6456 [2023-12-27 02:05:05,549][105692] Updated weights for policy 0, policy_version 1468221 (0.0005) [2023-12-27 02:05:05,570][105585] KL-divergence is very high: 193.4205 [2023-12-27 02:05:05,582][105585] KL-divergence is very high: 186.6100 [2023-12-27 02:05:05,605][105692] Updated weights for policy 0, policy_version 1468231 (0.0005) [2023-12-27 02:05:05,615][105585] KL-divergence is very high: 218.1669 [2023-12-27 02:05:05,624][105585] KL-divergence is very high: 196.9076 [2023-12-27 02:05:05,705][105620] Updated weights for policy 1, policy_version 1470533 (0.0010) [2023-12-27 02:05:05,769][105620] Updated weights for policy 1, policy_version 1470544 (0.0009) [2023-12-27 02:05:05,821][105620] Updated weights for policy 1, policy_version 1470554 (0.0010) [2023-12-27 02:05:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 752435200. Throughput: 0: 9850.3, 1: 9749.6. Samples: 752420448. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:05:06,062][104569] Avg episode reward: [(0, '7897.775'), (1, '8901.542')] [2023-12-27 02:05:06,137][105692] Updated weights for policy 0, policy_version 1468241 (0.0006) [2023-12-27 02:05:06,195][105692] Updated weights for policy 0, policy_version 1468251 (0.0008) [2023-12-27 02:05:06,247][105692] Updated weights for policy 0, policy_version 1468261 (0.0008) [2023-12-27 02:05:06,308][105692] Updated weights for policy 0, policy_version 1468271 (0.0009) [2023-12-27 02:05:06,583][105620] Updated weights for policy 1, policy_version 1470564 (0.0010) [2023-12-27 02:05:06,639][105620] Updated weights for policy 1, policy_version 1470574 (0.0011) [2023-12-27 02:05:06,703][105620] Updated weights for policy 1, policy_version 1470584 (0.0010) [2023-12-27 02:05:07,100][105692] Updated weights for policy 0, policy_version 1468281 (0.0010) [2023-12-27 02:05:07,171][105692] Updated weights for policy 0, policy_version 1468291 (0.0011) [2023-12-27 02:05:07,234][105692] Updated weights for policy 0, policy_version 1468301 (0.0011) [2023-12-27 02:05:07,333][105620] Updated weights for policy 1, policy_version 1470594 (0.0008) [2023-12-27 02:05:07,389][105620] Updated weights for policy 1, policy_version 1470604 (0.0005) [2023-12-27 02:05:07,440][105620] Updated weights for policy 1, policy_version 1470614 (0.0006) [2023-12-27 02:05:07,487][105620] Updated weights for policy 1, policy_version 1470624 (0.0005) [2023-12-27 02:05:07,884][105692] Updated weights for policy 0, policy_version 1468311 (0.0007) [2023-12-27 02:05:07,944][105692] Updated weights for policy 0, policy_version 1468321 (0.0006) [2023-12-27 02:05:08,017][105692] Updated weights for policy 0, policy_version 1468331 (0.0006) [2023-12-27 02:05:08,114][105620] Updated weights for policy 1, policy_version 1470634 (0.0005) [2023-12-27 02:05:08,183][105620] Updated weights for policy 1, policy_version 1470644 (0.0005) [2023-12-27 02:05:08,251][105620] Updated weights for policy 1, policy_version 1470654 (0.0005) [2023-12-27 02:05:08,746][105692] Updated weights for policy 0, policy_version 1468341 (0.0007) [2023-12-27 02:05:08,799][105692] Updated weights for policy 0, policy_version 1468351 (0.0008) [2023-12-27 02:05:08,857][105620] Updated weights for policy 1, policy_version 1470664 (0.0007) [2023-12-27 02:05:08,859][105692] Updated weights for policy 0, policy_version 1468361 (0.0008) [2023-12-27 02:05:08,909][105620] Updated weights for policy 1, policy_version 1470674 (0.0005) [2023-12-27 02:05:08,958][105620] Updated weights for policy 1, policy_version 1470684 (0.0005) [2023-12-27 02:05:09,578][105620] Updated weights for policy 1, policy_version 1470694 (0.0006) [2023-12-27 02:05:09,591][105692] Updated weights for policy 0, policy_version 1468371 (0.0008) [2023-12-27 02:05:09,645][105620] Updated weights for policy 1, policy_version 1470704 (0.0011) [2023-12-27 02:05:09,651][105692] Updated weights for policy 0, policy_version 1468381 (0.0005) [2023-12-27 02:05:09,705][105620] Updated weights for policy 1, policy_version 1470714 (0.0011) [2023-12-27 02:05:09,712][105692] Updated weights for policy 0, policy_version 1468391 (0.0009) [2023-12-27 02:05:10,418][105692] Updated weights for policy 0, policy_version 1468401 (0.0010) [2023-12-27 02:05:10,476][105692] Updated weights for policy 0, policy_version 1468411 (0.0008) [2023-12-27 02:05:10,486][105620] Updated weights for policy 1, policy_version 1470724 (0.0009) [2023-12-27 02:05:10,541][105692] Updated weights for policy 0, policy_version 1468421 (0.0008) [2023-12-27 02:05:10,547][105620] Updated weights for policy 1, policy_version 1470734 (0.0007) [2023-12-27 02:05:10,602][105692] Updated weights for policy 0, policy_version 1468431 (0.0008) [2023-12-27 02:05:10,611][105620] Updated weights for policy 1, policy_version 1470744 (0.0008) [2023-12-27 02:05:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 752533504. Throughput: 0: 10002.7, 1: 9802.2. Samples: 752542028. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:05:11,063][104569] Avg episode reward: [(0, '7985.762'), (1, '9084.337')] [2023-12-27 02:05:11,340][105692] Updated weights for policy 0, policy_version 1468441 (0.0010) [2023-12-27 02:05:11,348][105620] Updated weights for policy 1, policy_version 1470754 (0.0009) [2023-12-27 02:05:11,410][105692] Updated weights for policy 0, policy_version 1468451 (0.0009) [2023-12-27 02:05:11,425][105620] Updated weights for policy 1, policy_version 1470764 (0.0007) [2023-12-27 02:05:11,477][105692] Updated weights for policy 0, policy_version 1468461 (0.0007) [2023-12-27 02:05:11,491][105620] Updated weights for policy 1, policy_version 1470774 (0.0008) [2023-12-27 02:05:11,566][105620] Updated weights for policy 1, policy_version 1470784 (0.0009) [2023-12-27 02:05:12,201][105692] Updated weights for policy 0, policy_version 1468471 (0.0010) [2023-12-27 02:05:12,261][105692] Updated weights for policy 0, policy_version 1468481 (0.0010) [2023-12-27 02:05:12,287][105620] Updated weights for policy 1, policy_version 1470794 (0.0007) [2023-12-27 02:05:12,328][105692] Updated weights for policy 0, policy_version 1468491 (0.0010) [2023-12-27 02:05:12,352][105620] Updated weights for policy 1, policy_version 1470804 (0.0009) [2023-12-27 02:05:12,415][105620] Updated weights for policy 1, policy_version 1470814 (0.0007) [2023-12-27 02:05:13,007][105692] Updated weights for policy 0, policy_version 1468501 (0.0011) [2023-12-27 02:05:13,059][105692] Updated weights for policy 0, policy_version 1468511 (0.0010) [2023-12-27 02:05:13,089][105620] Updated weights for policy 1, policy_version 1470824 (0.0006) [2023-12-27 02:05:13,111][105692] Updated weights for policy 0, policy_version 1468521 (0.0010) [2023-12-27 02:05:13,141][105620] Updated weights for policy 1, policy_version 1470834 (0.0005) [2023-12-27 02:05:13,196][105620] Updated weights for policy 1, policy_version 1470844 (0.0005) [2023-12-27 02:05:13,713][105692] Updated weights for policy 0, policy_version 1468531 (0.0007) [2023-12-27 02:05:13,744][105620] Updated weights for policy 1, policy_version 1470854 (0.0005) [2023-12-27 02:05:13,764][105692] Updated weights for policy 0, policy_version 1468541 (0.0010) [2023-12-27 02:05:13,811][105620] Updated weights for policy 1, policy_version 1470864 (0.0005) [2023-12-27 02:05:13,819][105692] Updated weights for policy 0, policy_version 1468551 (0.0010) [2023-12-27 02:05:13,875][105620] Updated weights for policy 1, policy_version 1470874 (0.0005) [2023-12-27 02:05:14,489][105620] Updated weights for policy 1, policy_version 1470884 (0.0009) [2023-12-27 02:05:14,545][105620] Updated weights for policy 1, policy_version 1470894 (0.0010) [2023-12-27 02:05:14,572][105692] Updated weights for policy 0, policy_version 1468561 (0.0010) [2023-12-27 02:05:14,604][105620] Updated weights for policy 1, policy_version 1470904 (0.0010) [2023-12-27 02:05:14,630][105692] Updated weights for policy 0, policy_version 1468571 (0.0005) [2023-12-27 02:05:14,687][105692] Updated weights for policy 0, policy_version 1468581 (0.0005) [2023-12-27 02:05:14,756][105692] Updated weights for policy 0, policy_version 1468591 (0.0009) [2023-12-27 02:05:15,381][105692] Updated weights for policy 0, policy_version 1468601 (0.0008) [2023-12-27 02:05:15,383][105620] Updated weights for policy 1, policy_version 1470914 (0.0010) [2023-12-27 02:05:15,434][105692] Updated weights for policy 0, policy_version 1468611 (0.0007) [2023-12-27 02:05:15,446][105620] Updated weights for policy 1, policy_version 1470924 (0.0011) [2023-12-27 02:05:15,494][105692] Updated weights for policy 0, policy_version 1468621 (0.0009) [2023-12-27 02:05:15,506][105620] Updated weights for policy 1, policy_version 1470934 (0.0011) [2023-12-27 02:05:15,570][105620] Updated weights for policy 1, policy_version 1470944 (0.0011) [2023-12-27 02:05:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 752631808. Throughput: 0: 10053.7, 1: 9752.8. Samples: 752603556. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:05:16,062][104569] Avg episode reward: [(0, '8253.937'), (1, '9085.085')] [2023-12-27 02:05:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001468624_376020992.pth... [2023-12-27 02:05:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001470944_376610816.pth... [2023-12-27 02:05:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001467472_375726080.pth [2023-12-27 02:05:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001469792_376315904.pth [2023-12-27 02:05:16,209][105692] Updated weights for policy 0, policy_version 1468631 (0.0007) [2023-12-27 02:05:16,257][105692] Updated weights for policy 0, policy_version 1468641 (0.0005) [2023-12-27 02:05:16,290][105620] Updated weights for policy 1, policy_version 1470954 (0.0010) [2023-12-27 02:05:16,309][105692] Updated weights for policy 0, policy_version 1468651 (0.0005) [2023-12-27 02:05:16,342][105620] Updated weights for policy 1, policy_version 1470964 (0.0010) [2023-12-27 02:05:16,397][105620] Updated weights for policy 1, policy_version 1470974 (0.0010) [2023-12-27 02:05:17,038][105692] Updated weights for policy 0, policy_version 1468661 (0.0006) [2023-12-27 02:05:17,092][105692] Updated weights for policy 0, policy_version 1468671 (0.0005) [2023-12-27 02:05:17,138][105692] Updated weights for policy 0, policy_version 1468681 (0.0005) [2023-12-27 02:05:17,155][105620] Updated weights for policy 1, policy_version 1470984 (0.0010) [2023-12-27 02:05:17,204][105620] Updated weights for policy 1, policy_version 1470994 (0.0010) [2023-12-27 02:05:17,252][105620] Updated weights for policy 1, policy_version 1471004 (0.0010) [2023-12-27 02:05:17,816][105692] Updated weights for policy 0, policy_version 1468691 (0.0007) [2023-12-27 02:05:17,883][105692] Updated weights for policy 0, policy_version 1468701 (0.0005) [2023-12-27 02:05:17,957][105692] Updated weights for policy 0, policy_version 1468711 (0.0005) [2023-12-27 02:05:18,030][105620] Updated weights for policy 1, policy_version 1471014 (0.0010) [2023-12-27 02:05:18,102][105620] Updated weights for policy 1, policy_version 1471024 (0.0011) [2023-12-27 02:05:18,162][105620] Updated weights for policy 1, policy_version 1471034 (0.0010) [2023-12-27 02:05:18,524][105692] Updated weights for policy 0, policy_version 1468721 (0.0009) [2023-12-27 02:05:18,591][105692] Updated weights for policy 0, policy_version 1468731 (0.0007) [2023-12-27 02:05:18,658][105692] Updated weights for policy 0, policy_version 1468741 (0.0005) [2023-12-27 02:05:18,716][105692] Updated weights for policy 0, policy_version 1468751 (0.0006) [2023-12-27 02:05:18,876][105620] Updated weights for policy 1, policy_version 1471044 (0.0010) [2023-12-27 02:05:18,927][105620] Updated weights for policy 1, policy_version 1471055 (0.0010) [2023-12-27 02:05:18,980][105620] Updated weights for policy 1, policy_version 1471065 (0.0010) [2023-12-27 02:05:19,284][105692] Updated weights for policy 0, policy_version 1468761 (0.0010) [2023-12-27 02:05:19,350][105692] Updated weights for policy 0, policy_version 1468771 (0.0009) [2023-12-27 02:05:19,411][105692] Updated weights for policy 0, policy_version 1468781 (0.0008) [2023-12-27 02:05:19,758][105620] Updated weights for policy 1, policy_version 1471075 (0.0010) [2023-12-27 02:05:19,810][105620] Updated weights for policy 1, policy_version 1471085 (0.0009) [2023-12-27 02:05:19,874][105620] Updated weights for policy 1, policy_version 1471095 (0.0008) [2023-12-27 02:05:20,104][105692] Updated weights for policy 0, policy_version 1468791 (0.0006) [2023-12-27 02:05:20,166][105692] Updated weights for policy 0, policy_version 1468801 (0.0007) [2023-12-27 02:05:20,226][105692] Updated weights for policy 0, policy_version 1468811 (0.0009) [2023-12-27 02:05:20,754][105620] Updated weights for policy 1, policy_version 1471105 (0.0009) [2023-12-27 02:05:20,821][105620] Updated weights for policy 1, policy_version 1471115 (0.0008) [2023-12-27 02:05:20,843][105692] Updated weights for policy 0, policy_version 1468821 (0.0010) [2023-12-27 02:05:20,882][105620] Updated weights for policy 1, policy_version 1471125 (0.0007) [2023-12-27 02:05:20,892][105692] Updated weights for policy 0, policy_version 1468831 (0.0010) [2023-12-27 02:05:20,938][105692] Updated weights for policy 0, policy_version 1468841 (0.0005) [2023-12-27 02:05:20,941][105620] Updated weights for policy 1, policy_version 1471135 (0.0007) [2023-12-27 02:05:21,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 752738304. Throughput: 0: 10061.6, 1: 9695.3. Samples: 752721144. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:05:21,063][104569] Avg episode reward: [(0, '7997.378'), (1, '9082.106')] [2023-12-27 02:05:21,705][105620] Updated weights for policy 1, policy_version 1471145 (0.0009) [2023-12-27 02:05:21,726][105692] Updated weights for policy 0, policy_version 1468851 (0.0011) [2023-12-27 02:05:21,773][105620] Updated weights for policy 1, policy_version 1471155 (0.0007) [2023-12-27 02:05:21,785][105692] Updated weights for policy 0, policy_version 1468861 (0.0009) [2023-12-27 02:05:21,842][105620] Updated weights for policy 1, policy_version 1471165 (0.0007) [2023-12-27 02:05:21,851][105692] Updated weights for policy 0, policy_version 1468871 (0.0007) [2023-12-27 02:05:22,578][105620] Updated weights for policy 1, policy_version 1471175 (0.0008) [2023-12-27 02:05:22,588][105692] Updated weights for policy 0, policy_version 1468881 (0.0007) [2023-12-27 02:05:22,644][105620] Updated weights for policy 1, policy_version 1471185 (0.0008) [2023-12-27 02:05:22,649][105692] Updated weights for policy 0, policy_version 1468891 (0.0006) [2023-12-27 02:05:22,710][105620] Updated weights for policy 1, policy_version 1471195 (0.0008) [2023-12-27 02:05:22,712][105692] Updated weights for policy 0, policy_version 1468901 (0.0007) [2023-12-27 02:05:22,777][105692] Updated weights for policy 0, policy_version 1468911 (0.0009) [2023-12-27 02:05:23,423][105620] Updated weights for policy 1, policy_version 1471205 (0.0008) [2023-12-27 02:05:23,468][105620] Updated weights for policy 1, policy_version 1471215 (0.0009) [2023-12-27 02:05:23,504][105692] Updated weights for policy 0, policy_version 1468921 (0.0010) [2023-12-27 02:05:23,516][105620] Updated weights for policy 1, policy_version 1471225 (0.0010) [2023-12-27 02:05:23,549][105692] Updated weights for policy 0, policy_version 1468931 (0.0006) [2023-12-27 02:05:23,597][105692] Updated weights for policy 0, policy_version 1468941 (0.0006) [2023-12-27 02:05:24,304][105620] Updated weights for policy 1, policy_version 1471235 (0.0009) [2023-12-27 02:05:24,342][105692] Updated weights for policy 0, policy_version 1468951 (0.0008) [2023-12-27 02:05:24,359][105620] Updated weights for policy 1, policy_version 1471245 (0.0007) [2023-12-27 02:05:24,392][105692] Updated weights for policy 0, policy_version 1468961 (0.0006) [2023-12-27 02:05:24,417][105620] Updated weights for policy 1, policy_version 1471255 (0.0007) [2023-12-27 02:05:24,443][105692] Updated weights for policy 0, policy_version 1468971 (0.0005) [2023-12-27 02:05:25,002][105692] Updated weights for policy 0, policy_version 1468981 (0.0005) [2023-12-27 02:05:25,064][105692] Updated weights for policy 0, policy_version 1468991 (0.0005) [2023-12-27 02:05:25,113][105692] Updated weights for policy 0, policy_version 1469001 (0.0009) [2023-12-27 02:05:25,170][105620] Updated weights for policy 1, policy_version 1471265 (0.0009) [2023-12-27 02:05:25,230][105620] Updated weights for policy 1, policy_version 1471275 (0.0009) [2023-12-27 02:05:25,286][105620] Updated weights for policy 1, policy_version 1471285 (0.0010) [2023-12-27 02:05:25,342][105620] Updated weights for policy 1, policy_version 1471295 (0.0011) [2023-12-27 02:05:25,795][105692] Updated weights for policy 0, policy_version 1469012 (0.0008) [2023-12-27 02:05:25,851][105692] Updated weights for policy 0, policy_version 1469022 (0.0006) [2023-12-27 02:05:25,915][105692] Updated weights for policy 0, policy_version 1469032 (0.0006) [2023-12-27 02:05:25,989][105620] Updated weights for policy 1, policy_version 1471305 (0.0006) [2023-12-27 02:05:26,044][105620] Updated weights for policy 1, policy_version 1471315 (0.0005) [2023-12-27 02:05:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 752828416. Throughput: 0: 10052.6, 1: 9693.0. Samples: 752837164. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:05:26,062][104569] Avg episode reward: [(0, '8543.084'), (1, '9174.427')] [2023-12-27 02:05:26,099][105620] Updated weights for policy 1, policy_version 1471325 (0.0005) [2023-12-27 02:05:26,481][105692] Updated weights for policy 0, policy_version 1469042 (0.0005) [2023-12-27 02:05:26,530][105692] Updated weights for policy 0, policy_version 1469052 (0.0005) [2023-12-27 02:05:26,582][105692] Updated weights for policy 0, policy_version 1469062 (0.0005) [2023-12-27 02:05:26,628][105692] Updated weights for policy 0, policy_version 1469072 (0.0005) [2023-12-27 02:05:26,652][105620] Updated weights for policy 1, policy_version 1471335 (0.0005) [2023-12-27 02:05:26,704][105620] Updated weights for policy 1, policy_version 1471345 (0.0005) [2023-12-27 02:05:26,755][105620] Updated weights for policy 1, policy_version 1471355 (0.0005) [2023-12-27 02:05:27,314][105692] Updated weights for policy 0, policy_version 1469082 (0.0008) [2023-12-27 02:05:27,364][105692] Updated weights for policy 0, policy_version 1469092 (0.0008) [2023-12-27 02:05:27,417][105692] Updated weights for policy 0, policy_version 1469102 (0.0008) [2023-12-27 02:05:27,435][105620] Updated weights for policy 1, policy_version 1471365 (0.0008) [2023-12-27 02:05:27,496][105620] Updated weights for policy 1, policy_version 1471375 (0.0009) [2023-12-27 02:05:27,555][105620] Updated weights for policy 1, policy_version 1471385 (0.0009) [2023-12-27 02:05:28,001][105692] Updated weights for policy 0, policy_version 1469112 (0.0005) [2023-12-27 02:05:28,056][105692] Updated weights for policy 0, policy_version 1469122 (0.0005) [2023-12-27 02:05:28,108][105692] Updated weights for policy 0, policy_version 1469132 (0.0006) [2023-12-27 02:05:28,231][105620] Updated weights for policy 1, policy_version 1471395 (0.0010) [2023-12-27 02:05:28,284][105620] Updated weights for policy 1, policy_version 1471406 (0.0010) [2023-12-27 02:05:28,343][105620] Updated weights for policy 1, policy_version 1471416 (0.0009) [2023-12-27 02:05:28,703][105692] Updated weights for policy 0, policy_version 1469142 (0.0007) [2023-12-27 02:05:28,757][105692] Updated weights for policy 0, policy_version 1469152 (0.0009) [2023-12-27 02:05:28,814][105692] Updated weights for policy 0, policy_version 1469162 (0.0010) [2023-12-27 02:05:29,067][105620] Updated weights for policy 1, policy_version 1471426 (0.0007) [2023-12-27 02:05:29,122][105620] Updated weights for policy 1, policy_version 1471436 (0.0008) [2023-12-27 02:05:29,172][105620] Updated weights for policy 1, policy_version 1471446 (0.0008) [2023-12-27 02:05:29,220][105620] Updated weights for policy 1, policy_version 1471456 (0.0008) [2023-12-27 02:05:29,565][105692] Updated weights for policy 0, policy_version 1469172 (0.0010) [2023-12-27 02:05:29,620][105692] Updated weights for policy 0, policy_version 1469182 (0.0010) [2023-12-27 02:05:29,666][105692] Updated weights for policy 0, policy_version 1469192 (0.0007) [2023-12-27 02:05:29,856][105620] Updated weights for policy 1, policy_version 1471466 (0.0006) [2023-12-27 02:05:29,911][105620] Updated weights for policy 1, policy_version 1471476 (0.0006) [2023-12-27 02:05:29,975][105620] Updated weights for policy 1, policy_version 1471486 (0.0008) [2023-12-27 02:05:30,288][105692] Updated weights for policy 0, policy_version 1469202 (0.0005) [2023-12-27 02:05:30,358][105692] Updated weights for policy 0, policy_version 1469212 (0.0006) [2023-12-27 02:05:30,417][105692] Updated weights for policy 0, policy_version 1469222 (0.0006) [2023-12-27 02:05:30,477][105692] Updated weights for policy 0, policy_version 1469232 (0.0005) [2023-12-27 02:05:30,660][105620] Updated weights for policy 1, policy_version 1471496 (0.0008) [2023-12-27 02:05:30,714][105620] Updated weights for policy 1, policy_version 1471506 (0.0007) [2023-12-27 02:05:30,781][105620] Updated weights for policy 1, policy_version 1471516 (0.0005) [2023-12-27 02:05:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 752934912. Throughput: 0: 10159.7, 1: 9743.6. Samples: 752902828. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:05:31,062][104569] Avg episode reward: [(0, '8445.027'), (1, '9081.992')] [2023-12-27 02:05:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001469232_376176640.pth... [2023-12-27 02:05:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001471520_376758272.pth... [2023-12-27 02:05:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001470368_376463360.pth [2023-12-27 02:05:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001468048_375873536.pth [2023-12-27 02:05:31,220][105692] Updated weights for policy 0, policy_version 1469242 (0.0009) [2023-12-27 02:05:31,282][105692] Updated weights for policy 0, policy_version 1469252 (0.0006) [2023-12-27 02:05:31,347][105692] Updated weights for policy 0, policy_version 1469262 (0.0008) [2023-12-27 02:05:31,417][105620] Updated weights for policy 1, policy_version 1471526 (0.0007) [2023-12-27 02:05:31,481][105620] Updated weights for policy 1, policy_version 1471536 (0.0009) [2023-12-27 02:05:31,538][105620] Updated weights for policy 1, policy_version 1471546 (0.0008) [2023-12-27 02:05:32,013][105692] Updated weights for policy 0, policy_version 1469272 (0.0006) [2023-12-27 02:05:32,063][105692] Updated weights for policy 0, policy_version 1469282 (0.0005) [2023-12-27 02:05:32,129][105692] Updated weights for policy 0, policy_version 1469292 (0.0009) [2023-12-27 02:05:32,218][105620] Updated weights for policy 1, policy_version 1471556 (0.0009) [2023-12-27 02:05:32,274][105620] Updated weights for policy 1, policy_version 1471566 (0.0009) [2023-12-27 02:05:32,334][105620] Updated weights for policy 1, policy_version 1471576 (0.0008) [2023-12-27 02:05:32,897][105692] Updated weights for policy 0, policy_version 1469302 (0.0009) [2023-12-27 02:05:32,950][105692] Updated weights for policy 0, policy_version 1469312 (0.0009) [2023-12-27 02:05:32,997][105620] Updated weights for policy 1, policy_version 1471586 (0.0008) [2023-12-27 02:05:32,997][105692] Updated weights for policy 0, policy_version 1469322 (0.0007) [2023-12-27 02:05:33,053][105620] Updated weights for policy 1, policy_version 1471596 (0.0008) [2023-12-27 02:05:33,110][105620] Updated weights for policy 1, policy_version 1471606 (0.0009) [2023-12-27 02:05:33,166][105620] Updated weights for policy 1, policy_version 1471616 (0.0008) [2023-12-27 02:05:33,602][105692] Updated weights for policy 0, policy_version 1469332 (0.0006) [2023-12-27 02:05:33,667][105692] Updated weights for policy 0, policy_version 1469342 (0.0005) [2023-12-27 02:05:33,713][105692] Updated weights for policy 0, policy_version 1469352 (0.0005) [2023-12-27 02:05:34,034][105620] Updated weights for policy 1, policy_version 1471626 (0.0009) [2023-12-27 02:05:34,090][105620] Updated weights for policy 1, policy_version 1471636 (0.0009) [2023-12-27 02:05:34,139][105620] Updated weights for policy 1, policy_version 1471646 (0.0008) [2023-12-27 02:05:34,321][105692] Updated weights for policy 0, policy_version 1469362 (0.0007) [2023-12-27 02:05:34,387][105692] Updated weights for policy 0, policy_version 1469372 (0.0009) [2023-12-27 02:05:34,439][105692] Updated weights for policy 0, policy_version 1469382 (0.0009) [2023-12-27 02:05:34,502][105692] Updated weights for policy 0, policy_version 1469392 (0.0009) [2023-12-27 02:05:34,864][105620] Updated weights for policy 1, policy_version 1471656 (0.0006) [2023-12-27 02:05:34,925][105620] Updated weights for policy 1, policy_version 1471666 (0.0006) [2023-12-27 02:05:34,978][105620] Updated weights for policy 1, policy_version 1471676 (0.0010) [2023-12-27 02:05:35,226][105692] Updated weights for policy 0, policy_version 1469402 (0.0007) [2023-12-27 02:05:35,279][105692] Updated weights for policy 0, policy_version 1469412 (0.0007) [2023-12-27 02:05:35,329][105692] Updated weights for policy 0, policy_version 1469422 (0.0009) [2023-12-27 02:05:35,732][105620] Updated weights for policy 1, policy_version 1471686 (0.0007) [2023-12-27 02:05:35,790][105620] Updated weights for policy 1, policy_version 1471696 (0.0005) [2023-12-27 02:05:35,844][105620] Updated weights for policy 1, policy_version 1471706 (0.0005) [2023-12-27 02:05:36,054][105692] Updated weights for policy 0, policy_version 1469432 (0.0007) [2023-12-27 02:05:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 753033216. Throughput: 0: 10128.1, 1: 9830.8. Samples: 753023320. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:05:36,062][104569] Avg episode reward: [(0, '8352.586'), (1, '9082.213')] [2023-12-27 02:05:36,116][105692] Updated weights for policy 0, policy_version 1469442 (0.0008) [2023-12-27 02:05:36,180][105692] Updated weights for policy 0, policy_version 1469452 (0.0007) [2023-12-27 02:05:36,575][105620] Updated weights for policy 1, policy_version 1471716 (0.0009) [2023-12-27 02:05:36,642][105620] Updated weights for policy 1, policy_version 1471726 (0.0010) [2023-12-27 02:05:36,708][105620] Updated weights for policy 1, policy_version 1471736 (0.0010) [2023-12-27 02:05:36,800][105692] Updated weights for policy 0, policy_version 1469462 (0.0007) [2023-12-27 02:05:36,860][105692] Updated weights for policy 0, policy_version 1469472 (0.0010) [2023-12-27 02:05:36,920][105692] Updated weights for policy 0, policy_version 1469482 (0.0009) [2023-12-27 02:05:37,507][105620] Updated weights for policy 1, policy_version 1471746 (0.0009) [2023-12-27 02:05:37,559][105620] Updated weights for policy 1, policy_version 1471756 (0.0009) [2023-12-27 02:05:37,596][105692] Updated weights for policy 0, policy_version 1469492 (0.0008) [2023-12-27 02:05:37,619][105620] Updated weights for policy 1, policy_version 1471766 (0.0009) [2023-12-27 02:05:37,657][105692] Updated weights for policy 0, policy_version 1469502 (0.0008) [2023-12-27 02:05:37,676][105620] Updated weights for policy 1, policy_version 1471776 (0.0008) [2023-12-27 02:05:37,717][105692] Updated weights for policy 0, policy_version 1469512 (0.0007) [2023-12-27 02:05:38,421][105692] Updated weights for policy 0, policy_version 1469522 (0.0010) [2023-12-27 02:05:38,463][105620] Updated weights for policy 1, policy_version 1471786 (0.0006) [2023-12-27 02:05:38,485][105692] Updated weights for policy 0, policy_version 1469532 (0.0010) [2023-12-27 02:05:38,524][105620] Updated weights for policy 1, policy_version 1471796 (0.0008) [2023-12-27 02:05:38,543][105692] Updated weights for policy 0, policy_version 1469542 (0.0007) [2023-12-27 02:05:38,581][105620] Updated weights for policy 1, policy_version 1471806 (0.0008) [2023-12-27 02:05:38,601][105692] Updated weights for policy 0, policy_version 1469552 (0.0006) [2023-12-27 02:05:39,271][105692] Updated weights for policy 0, policy_version 1469562 (0.0008) [2023-12-27 02:05:39,337][105692] Updated weights for policy 0, policy_version 1469572 (0.0007) [2023-12-27 02:05:39,403][105692] Updated weights for policy 0, policy_version 1469582 (0.0009) [2023-12-27 02:05:39,409][105620] Updated weights for policy 1, policy_version 1471816 (0.0008) [2023-12-27 02:05:39,465][105620] Updated weights for policy 1, policy_version 1471826 (0.0009) [2023-12-27 02:05:39,520][105620] Updated weights for policy 1, policy_version 1471836 (0.0009) [2023-12-27 02:05:40,182][105692] Updated weights for policy 0, policy_version 1469592 (0.0009) [2023-12-27 02:05:40,245][105692] Updated weights for policy 0, policy_version 1469602 (0.0009) [2023-12-27 02:05:40,295][105620] Updated weights for policy 1, policy_version 1471846 (0.0008) [2023-12-27 02:05:40,295][105692] Updated weights for policy 0, policy_version 1469612 (0.0008) [2023-12-27 02:05:40,352][105620] Updated weights for policy 1, policy_version 1471856 (0.0007) [2023-12-27 02:05:40,404][105620] Updated weights for policy 1, policy_version 1471866 (0.0009) [2023-12-27 02:05:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 753123328. Throughput: 0: 10138.9, 1: 9759.4. Samples: 753136356. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:05:41,062][104569] Avg episode reward: [(0, '8533.882'), (1, '9176.375')] [2023-12-27 02:05:41,099][105692] Updated weights for policy 0, policy_version 1469622 (0.0009) [2023-12-27 02:05:41,170][105692] Updated weights for policy 0, policy_version 1469632 (0.0008) [2023-12-27 02:05:41,212][105620] Updated weights for policy 1, policy_version 1471876 (0.0008) [2023-12-27 02:05:41,231][105692] Updated weights for policy 0, policy_version 1469642 (0.0007) [2023-12-27 02:05:41,275][105620] Updated weights for policy 1, policy_version 1471886 (0.0006) [2023-12-27 02:05:41,345][105620] Updated weights for policy 1, policy_version 1471896 (0.0006) [2023-12-27 02:05:42,029][105692] Updated weights for policy 0, policy_version 1469652 (0.0009) [2023-12-27 02:05:42,085][105692] Updated weights for policy 0, policy_version 1469662 (0.0009) [2023-12-27 02:05:42,095][105620] Updated weights for policy 1, policy_version 1471906 (0.0009) [2023-12-27 02:05:42,147][105692] Updated weights for policy 0, policy_version 1469672 (0.0007) [2023-12-27 02:05:42,149][105620] Updated weights for policy 1, policy_version 1471916 (0.0006) [2023-12-27 02:05:42,212][105620] Updated weights for policy 1, policy_version 1471926 (0.0007) [2023-12-27 02:05:42,275][105620] Updated weights for policy 1, policy_version 1471936 (0.0009) [2023-12-27 02:05:42,889][105692] Updated weights for policy 0, policy_version 1469682 (0.0007) [2023-12-27 02:05:42,937][105692] Updated weights for policy 0, policy_version 1469692 (0.0009) [2023-12-27 02:05:42,986][105692] Updated weights for policy 0, policy_version 1469702 (0.0009) [2023-12-27 02:05:43,036][105692] Updated weights for policy 0, policy_version 1469712 (0.0009) [2023-12-27 02:05:43,062][105620] Updated weights for policy 1, policy_version 1471946 (0.0008) [2023-12-27 02:05:43,117][105620] Updated weights for policy 1, policy_version 1471956 (0.0009) [2023-12-27 02:05:43,180][105620] Updated weights for policy 1, policy_version 1471966 (0.0011) [2023-12-27 02:05:43,807][105692] Updated weights for policy 0, policy_version 1469722 (0.0009) [2023-12-27 02:05:43,863][105692] Updated weights for policy 0, policy_version 1469732 (0.0009) [2023-12-27 02:05:43,902][105620] Updated weights for policy 1, policy_version 1471976 (0.0006) [2023-12-27 02:05:43,911][105692] Updated weights for policy 0, policy_version 1469742 (0.0008) [2023-12-27 02:05:43,960][105620] Updated weights for policy 1, policy_version 1471986 (0.0009) [2023-12-27 02:05:44,018][105620] Updated weights for policy 1, policy_version 1471996 (0.0010) [2023-12-27 02:05:44,664][105620] Updated weights for policy 1, policy_version 1472006 (0.0008) [2023-12-27 02:05:44,666][105692] Updated weights for policy 0, policy_version 1469752 (0.0007) [2023-12-27 02:05:44,713][105620] Updated weights for policy 1, policy_version 1472016 (0.0007) [2023-12-27 02:05:44,715][105692] Updated weights for policy 0, policy_version 1469762 (0.0006) [2023-12-27 02:05:44,768][105692] Updated weights for policy 0, policy_version 1469772 (0.0007) [2023-12-27 02:05:44,770][105620] Updated weights for policy 1, policy_version 1472026 (0.0007) [2023-12-27 02:05:45,529][105620] Updated weights for policy 1, policy_version 1472036 (0.0008) [2023-12-27 02:05:45,561][105692] Updated weights for policy 0, policy_version 1469782 (0.0008) [2023-12-27 02:05:45,579][105620] Updated weights for policy 1, policy_version 1472046 (0.0007) [2023-12-27 02:05:45,624][105692] Updated weights for policy 0, policy_version 1469792 (0.0008) [2023-12-27 02:05:45,635][105620] Updated weights for policy 1, policy_version 1472056 (0.0006) [2023-12-27 02:05:45,687][105692] Updated weights for policy 0, policy_version 1469802 (0.0008) [2023-12-27 02:05:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19797.5, 300 sec: 19577.5). Total num frames: 753221632. Throughput: 0: 10049.3, 1: 9711.7. Samples: 753191480. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:05:46,062][104569] Avg episode reward: [(0, '8626.775'), (1, '9083.646')] [2023-12-27 02:05:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001472064_376897536.pth... [2023-12-27 02:05:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001469808_376324096.pth... [2023-12-27 02:05:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001470944_376610816.pth [2023-12-27 02:05:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001468624_376020992.pth [2023-12-27 02:05:46,397][105620] Updated weights for policy 1, policy_version 1472066 (0.0006) [2023-12-27 02:05:46,425][105692] Updated weights for policy 0, policy_version 1469812 (0.0009) [2023-12-27 02:05:46,448][105620] Updated weights for policy 1, policy_version 1472076 (0.0005) [2023-12-27 02:05:46,490][105692] Updated weights for policy 0, policy_version 1469822 (0.0008) [2023-12-27 02:05:46,498][105620] Updated weights for policy 1, policy_version 1472086 (0.0005) [2023-12-27 02:05:46,548][105692] Updated weights for policy 0, policy_version 1469832 (0.0009) [2023-12-27 02:05:46,551][105620] Updated weights for policy 1, policy_version 1472096 (0.0006) [2023-12-27 02:05:47,112][105620] Updated weights for policy 1, policy_version 1472106 (0.0005) [2023-12-27 02:05:47,157][105620] Updated weights for policy 1, policy_version 1472116 (0.0006) [2023-12-27 02:05:47,204][105620] Updated weights for policy 1, policy_version 1472126 (0.0008) [2023-12-27 02:05:47,375][105692] Updated weights for policy 0, policy_version 1469842 (0.0010) [2023-12-27 02:05:47,429][105692] Updated weights for policy 0, policy_version 1469852 (0.0009) [2023-12-27 02:05:47,491][105692] Updated weights for policy 0, policy_version 1469862 (0.0007) [2023-12-27 02:05:47,552][105692] Updated weights for policy 0, policy_version 1469872 (0.0009) [2023-12-27 02:05:47,945][105620] Updated weights for policy 1, policy_version 1472136 (0.0009) [2023-12-27 02:05:48,010][105620] Updated weights for policy 1, policy_version 1472146 (0.0009) [2023-12-27 02:05:48,068][105620] Updated weights for policy 1, policy_version 1472156 (0.0009) [2023-12-27 02:05:48,281][105692] Updated weights for policy 0, policy_version 1469882 (0.0009) [2023-12-27 02:05:48,344][105692] Updated weights for policy 0, policy_version 1469892 (0.0009) [2023-12-27 02:05:48,412][105692] Updated weights for policy 0, policy_version 1469902 (0.0008) [2023-12-27 02:05:48,752][105620] Updated weights for policy 1, policy_version 1472166 (0.0009) [2023-12-27 02:05:48,816][105620] Updated weights for policy 1, policy_version 1472176 (0.0010) [2023-12-27 02:05:48,872][105620] Updated weights for policy 1, policy_version 1472186 (0.0009) [2023-12-27 02:05:49,166][105692] Updated weights for policy 0, policy_version 1469912 (0.0009) [2023-12-27 02:05:49,227][105692] Updated weights for policy 0, policy_version 1469922 (0.0008) [2023-12-27 02:05:49,291][105692] Updated weights for policy 0, policy_version 1469932 (0.0008) [2023-12-27 02:05:49,562][105620] Updated weights for policy 1, policy_version 1472196 (0.0008) [2023-12-27 02:05:49,625][105620] Updated weights for policy 1, policy_version 1472206 (0.0005) [2023-12-27 02:05:49,690][105620] Updated weights for policy 1, policy_version 1472216 (0.0005) [2023-12-27 02:05:50,127][105692] Updated weights for policy 0, policy_version 1469942 (0.0010) [2023-12-27 02:05:50,186][105692] Updated weights for policy 0, policy_version 1469952 (0.0010) [2023-12-27 02:05:50,245][105692] Updated weights for policy 0, policy_version 1469962 (0.0011) [2023-12-27 02:05:50,295][105620] Updated weights for policy 1, policy_version 1472226 (0.0006) [2023-12-27 02:05:50,358][105620] Updated weights for policy 1, policy_version 1472236 (0.0008) [2023-12-27 02:05:50,414][105620] Updated weights for policy 1, policy_version 1472246 (0.0008) [2023-12-27 02:05:50,478][105620] Updated weights for policy 1, policy_version 1472256 (0.0008) [2023-12-27 02:05:51,003][105692] Updated weights for policy 0, policy_version 1469972 (0.0009) [2023-12-27 02:05:51,062][104569] Fps is (10 sec: 18840.7, 60 sec: 19524.1, 300 sec: 19577.5). Total num frames: 753311744. Throughput: 0: 9963.7, 1: 9736.6. Samples: 753306972. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:05:51,063][104569] Avg episode reward: [(0, '8626.447'), (1, '9171.856')] [2023-12-27 02:05:51,066][105692] Updated weights for policy 0, policy_version 1469982 (0.0007) [2023-12-27 02:05:51,136][105692] Updated weights for policy 0, policy_version 1469992 (0.0008) [2023-12-27 02:05:51,212][105620] Updated weights for policy 1, policy_version 1472266 (0.0008) [2023-12-27 02:05:51,262][105620] Updated weights for policy 1, policy_version 1472276 (0.0009) [2023-12-27 02:05:51,317][105620] Updated weights for policy 1, policy_version 1472286 (0.0009) [2023-12-27 02:05:51,808][105692] Updated weights for policy 0, policy_version 1470002 (0.0006) [2023-12-27 02:05:51,876][105692] Updated weights for policy 0, policy_version 1470012 (0.0006) [2023-12-27 02:05:51,942][105692] Updated weights for policy 0, policy_version 1470022 (0.0007) [2023-12-27 02:05:52,002][105692] Updated weights for policy 0, policy_version 1470032 (0.0009) [2023-12-27 02:05:52,146][105620] Updated weights for policy 1, policy_version 1472296 (0.0007) [2023-12-27 02:05:52,206][105620] Updated weights for policy 1, policy_version 1472306 (0.0007) [2023-12-27 02:05:52,273][105620] Updated weights for policy 1, policy_version 1472316 (0.0009) [2023-12-27 02:05:52,722][105692] Updated weights for policy 0, policy_version 1470042 (0.0008) [2023-12-27 02:05:52,781][105692] Updated weights for policy 0, policy_version 1470052 (0.0008) [2023-12-27 02:05:52,841][105692] Updated weights for policy 0, policy_version 1470062 (0.0009) [2023-12-27 02:05:52,981][105620] Updated weights for policy 1, policy_version 1472326 (0.0010) [2023-12-27 02:05:53,032][105620] Updated weights for policy 1, policy_version 1472336 (0.0010) [2023-12-27 02:05:53,087][105620] Updated weights for policy 1, policy_version 1472346 (0.0010) [2023-12-27 02:05:53,576][105692] Updated weights for policy 0, policy_version 1470072 (0.0009) [2023-12-27 02:05:53,635][105692] Updated weights for policy 0, policy_version 1470082 (0.0009) [2023-12-27 02:05:53,685][105620] Updated weights for policy 1, policy_version 1472356 (0.0008) [2023-12-27 02:05:53,687][105692] Updated weights for policy 0, policy_version 1470092 (0.0010) [2023-12-27 02:05:53,738][105620] Updated weights for policy 1, policy_version 1472366 (0.0005) [2023-12-27 02:05:53,790][105620] Updated weights for policy 1, policy_version 1472376 (0.0006) [2023-12-27 02:05:54,418][105692] Updated weights for policy 0, policy_version 1470102 (0.0007) [2023-12-27 02:05:54,484][105692] Updated weights for policy 0, policy_version 1470112 (0.0007) [2023-12-27 02:05:54,501][105620] Updated weights for policy 1, policy_version 1472386 (0.0010) [2023-12-27 02:05:54,543][105692] Updated weights for policy 0, policy_version 1470122 (0.0010) [2023-12-27 02:05:54,555][105620] Updated weights for policy 1, policy_version 1472396 (0.0010) [2023-12-27 02:05:54,612][105620] Updated weights for policy 1, policy_version 1472406 (0.0010) [2023-12-27 02:05:54,661][105620] Updated weights for policy 1, policy_version 1472416 (0.0007) [2023-12-27 02:05:55,234][105620] Updated weights for policy 1, policy_version 1472426 (0.0005) [2023-12-27 02:05:55,305][105620] Updated weights for policy 1, policy_version 1472436 (0.0005) [2023-12-27 02:05:55,360][105692] Updated weights for policy 0, policy_version 1470132 (0.0008) [2023-12-27 02:05:55,372][105620] Updated weights for policy 1, policy_version 1472446 (0.0007) [2023-12-27 02:05:55,408][105692] Updated weights for policy 0, policy_version 1470142 (0.0007) [2023-12-27 02:05:55,460][105692] Updated weights for policy 0, policy_version 1470152 (0.0005) [2023-12-27 02:05:56,036][105620] Updated weights for policy 1, policy_version 1472456 (0.0010) [2023-12-27 02:05:56,051][105692] Updated weights for policy 0, policy_version 1470162 (0.0006) [2023-12-27 02:05:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 753410048. Throughput: 0: 9861.2, 1: 9715.3. Samples: 753422968. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:05:56,062][104569] Avg episode reward: [(0, '8441.881'), (1, '9082.481')] [2023-12-27 02:05:56,093][105620] Updated weights for policy 1, policy_version 1472466 (0.0010) [2023-12-27 02:05:56,099][105692] Updated weights for policy 0, policy_version 1470172 (0.0008) [2023-12-27 02:05:56,148][105620] Updated weights for policy 1, policy_version 1472476 (0.0010) [2023-12-27 02:05:56,150][105692] Updated weights for policy 0, policy_version 1470182 (0.0008) [2023-12-27 02:05:56,202][105692] Updated weights for policy 0, policy_version 1470192 (0.0008) [2023-12-27 02:05:56,892][105692] Updated weights for policy 0, policy_version 1470202 (0.0006) [2023-12-27 02:05:56,931][105620] Updated weights for policy 1, policy_version 1472486 (0.0010) [2023-12-27 02:05:56,950][105692] Updated weights for policy 0, policy_version 1470212 (0.0005) [2023-12-27 02:05:56,990][105620] Updated weights for policy 1, policy_version 1472496 (0.0011) [2023-12-27 02:05:57,004][105692] Updated weights for policy 0, policy_version 1470222 (0.0006) [2023-12-27 02:05:57,038][105620] Updated weights for policy 1, policy_version 1472506 (0.0010) [2023-12-27 02:05:57,581][105692] Updated weights for policy 0, policy_version 1470232 (0.0007) [2023-12-27 02:05:57,628][105692] Updated weights for policy 0, policy_version 1470242 (0.0010) [2023-12-27 02:05:57,676][105692] Updated weights for policy 0, policy_version 1470252 (0.0010) [2023-12-27 02:05:57,771][105620] Updated weights for policy 1, policy_version 1472516 (0.0010) [2023-12-27 02:05:57,827][105620] Updated weights for policy 1, policy_version 1472526 (0.0011) [2023-12-27 02:05:57,875][105620] Updated weights for policy 1, policy_version 1472536 (0.0010) [2023-12-27 02:05:58,361][105692] Updated weights for policy 0, policy_version 1470262 (0.0011) [2023-12-27 02:05:58,427][105692] Updated weights for policy 0, policy_version 1470272 (0.0008) [2023-12-27 02:05:58,503][105692] Updated weights for policy 0, policy_version 1470282 (0.0008) [2023-12-27 02:05:58,652][105620] Updated weights for policy 1, policy_version 1472546 (0.0010) [2023-12-27 02:05:58,715][105620] Updated weights for policy 1, policy_version 1472556 (0.0008) [2023-12-27 02:05:58,796][105620] Updated weights for policy 1, policy_version 1472567 (0.0008) [2023-12-27 02:05:59,345][105692] Updated weights for policy 0, policy_version 1470292 (0.0008) [2023-12-27 02:05:59,407][105692] Updated weights for policy 0, policy_version 1470302 (0.0009) [2023-12-27 02:05:59,474][105692] Updated weights for policy 0, policy_version 1470312 (0.0010) [2023-12-27 02:05:59,542][105620] Updated weights for policy 1, policy_version 1472577 (0.0012) [2023-12-27 02:05:59,592][105620] Updated weights for policy 1, policy_version 1472587 (0.0009) [2023-12-27 02:05:59,646][105620] Updated weights for policy 1, policy_version 1472597 (0.0009) [2023-12-27 02:05:59,696][105620] Updated weights for policy 1, policy_version 1472607 (0.0009) [2023-12-27 02:06:00,292][105692] Updated weights for policy 0, policy_version 1470322 (0.0010) [2023-12-27 02:06:00,338][105620] Updated weights for policy 1, policy_version 1472617 (0.0005) [2023-12-27 02:06:00,347][105692] Updated weights for policy 0, policy_version 1470332 (0.0009) [2023-12-27 02:06:00,399][105620] Updated weights for policy 1, policy_version 1472627 (0.0005) [2023-12-27 02:06:00,416][105692] Updated weights for policy 0, policy_version 1470342 (0.0008) [2023-12-27 02:06:00,458][105620] Updated weights for policy 1, policy_version 1472637 (0.0007) [2023-12-27 02:06:00,472][105692] Updated weights for policy 0, policy_version 1470352 (0.0006) [2023-12-27 02:06:01,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 753508352. Throughput: 0: 9904.7, 1: 9627.8. Samples: 753482520. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:06:01,063][104569] Avg episode reward: [(0, '8623.878'), (1, '9082.722')] [2023-12-27 02:06:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001470352_376463360.pth... [2023-12-27 02:06:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001469232_376176640.pth [2023-12-27 02:06:01,102][105620] Updated weights for policy 1, policy_version 1472647 (0.0008) [2023-12-27 02:06:01,170][105620] Updated weights for policy 1, policy_version 1472657 (0.0009) [2023-12-27 02:06:01,235][105620] Updated weights for policy 1, policy_version 1472667 (0.0008) [2023-12-27 02:06:01,259][105692] Updated weights for policy 0, policy_version 1470362 (0.0008) [2023-12-27 02:06:01,264][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001472672_377053184.pth... [2023-12-27 02:06:01,268][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001471520_376758272.pth [2023-12-27 02:06:01,324][105692] Updated weights for policy 0, policy_version 1470372 (0.0009) [2023-12-27 02:06:01,393][105692] Updated weights for policy 0, policy_version 1470382 (0.0009) [2023-12-27 02:06:01,982][105620] Updated weights for policy 1, policy_version 1472677 (0.0007) [2023-12-27 02:06:02,046][105620] Updated weights for policy 1, policy_version 1472687 (0.0007) [2023-12-27 02:06:02,095][105620] Updated weights for policy 1, policy_version 1472697 (0.0010) [2023-12-27 02:06:02,201][105692] Updated weights for policy 0, policy_version 1470392 (0.0009) [2023-12-27 02:06:02,262][105692] Updated weights for policy 0, policy_version 1470402 (0.0010) [2023-12-27 02:06:02,320][105692] Updated weights for policy 0, policy_version 1470412 (0.0009) [2023-12-27 02:06:02,707][105620] Updated weights for policy 1, policy_version 1472707 (0.0006) [2023-12-27 02:06:02,771][105620] Updated weights for policy 1, policy_version 1472717 (0.0009) [2023-12-27 02:06:02,830][105620] Updated weights for policy 1, policy_version 1472727 (0.0009) [2023-12-27 02:06:03,128][105692] Updated weights for policy 0, policy_version 1470422 (0.0009) [2023-12-27 02:06:03,182][105692] Updated weights for policy 0, policy_version 1470432 (0.0010) [2023-12-27 02:06:03,242][105692] Updated weights for policy 0, policy_version 1470442 (0.0009) [2023-12-27 02:06:03,483][105620] Updated weights for policy 1, policy_version 1472737 (0.0009) [2023-12-27 02:06:03,537][105620] Updated weights for policy 1, policy_version 1472747 (0.0006) [2023-12-27 02:06:03,588][105620] Updated weights for policy 1, policy_version 1472757 (0.0006) [2023-12-27 02:06:03,636][105620] Updated weights for policy 1, policy_version 1472767 (0.0005) [2023-12-27 02:06:04,112][105692] Updated weights for policy 0, policy_version 1470452 (0.0010) [2023-12-27 02:06:04,168][105692] Updated weights for policy 0, policy_version 1470462 (0.0009) [2023-12-27 02:06:04,228][105692] Updated weights for policy 0, policy_version 1470472 (0.0009) [2023-12-27 02:06:04,297][105620] Updated weights for policy 1, policy_version 1472777 (0.0008) [2023-12-27 02:06:04,370][105620] Updated weights for policy 1, policy_version 1472787 (0.0008) [2023-12-27 02:06:04,437][105620] Updated weights for policy 1, policy_version 1472797 (0.0009) [2023-12-27 02:06:05,026][105692] Updated weights for policy 0, policy_version 1470482 (0.0009) [2023-12-27 02:06:05,082][105620] Updated weights for policy 1, policy_version 1472807 (0.0007) [2023-12-27 02:06:05,084][105692] Updated weights for policy 0, policy_version 1470492 (0.0008) [2023-12-27 02:06:05,135][105620] Updated weights for policy 1, policy_version 1472817 (0.0008) [2023-12-27 02:06:05,139][105692] Updated weights for policy 0, policy_version 1470502 (0.0007) [2023-12-27 02:06:05,188][105692] Updated weights for policy 0, policy_version 1470512 (0.0006) [2023-12-27 02:06:05,200][105620] Updated weights for policy 1, policy_version 1472827 (0.0005) [2023-12-27 02:06:05,872][105620] Updated weights for policy 1, policy_version 1472837 (0.0005) [2023-12-27 02:06:05,938][105620] Updated weights for policy 1, policy_version 1472847 (0.0005) [2023-12-27 02:06:05,941][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000009 [2023-12-27 02:06:05,976][105692] Updated weights for policy 0, policy_version 1470522 (0.0009) [2023-12-27 02:06:06,029][105692] Updated weights for policy 0, policy_version 1470533 (0.0010) [2023-12-27 02:06:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 753606656. Throughput: 0: 9690.1, 1: 9771.1. Samples: 753596900. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:06:06,063][104569] Avg episode reward: [(0, '8625.621'), (1, '9083.629')] [2023-12-27 02:06:06,074][105692] Updated weights for policy 0, policy_version 1470543 (0.0008) [2023-12-27 02:06:06,697][105620] Updated weights for policy 1, policy_version 1472857 (0.0010) [2023-12-27 02:06:06,722][105692] Updated weights for policy 0, policy_version 1470553 (0.0006) [2023-12-27 02:06:06,746][105620] Updated weights for policy 1, policy_version 1472867 (0.0010) [2023-12-27 02:06:06,781][105692] Updated weights for policy 0, policy_version 1470563 (0.0006) [2023-12-27 02:06:06,805][105620] Updated weights for policy 1, policy_version 1472877 (0.0010) [2023-12-27 02:06:06,833][105692] Updated weights for policy 0, policy_version 1470573 (0.0005) [2023-12-27 02:06:07,376][105692] Updated weights for policy 0, policy_version 1470583 (0.0005) [2023-12-27 02:06:07,429][105692] Updated weights for policy 0, policy_version 1470593 (0.0006) [2023-12-27 02:06:07,495][105692] Updated weights for policy 0, policy_version 1470603 (0.0006) [2023-12-27 02:06:07,570][105620] Updated weights for policy 1, policy_version 1472887 (0.0010) [2023-12-27 02:06:07,632][105620] Updated weights for policy 1, policy_version 1472897 (0.0010) [2023-12-27 02:06:07,692][105620] Updated weights for policy 1, policy_version 1472907 (0.0010) [2023-12-27 02:06:08,181][105692] Updated weights for policy 0, policy_version 1470613 (0.0008) [2023-12-27 02:06:08,240][105692] Updated weights for policy 0, policy_version 1470623 (0.0008) [2023-12-27 02:06:08,299][105692] Updated weights for policy 0, policy_version 1470633 (0.0008) [2023-12-27 02:06:08,409][105620] Updated weights for policy 1, policy_version 1472917 (0.0010) [2023-12-27 02:06:08,468][105620] Updated weights for policy 1, policy_version 1472927 (0.0010) [2023-12-27 02:06:08,533][105620] Updated weights for policy 1, policy_version 1472937 (0.0010) [2023-12-27 02:06:09,048][105692] Updated weights for policy 0, policy_version 1470643 (0.0007) [2023-12-27 02:06:09,114][105692] Updated weights for policy 0, policy_version 1470653 (0.0005) [2023-12-27 02:06:09,163][105692] Updated weights for policy 0, policy_version 1470663 (0.0005) [2023-12-27 02:06:09,316][105620] Updated weights for policy 1, policy_version 1472947 (0.0010) [2023-12-27 02:06:09,378][105620] Updated weights for policy 1, policy_version 1472957 (0.0010) [2023-12-27 02:06:09,447][105620] Updated weights for policy 1, policy_version 1472967 (0.0009) [2023-12-27 02:06:09,853][105692] Updated weights for policy 0, policy_version 1470673 (0.0005) [2023-12-27 02:06:09,911][105692] Updated weights for policy 0, policy_version 1470683 (0.0009) [2023-12-27 02:06:09,972][105692] Updated weights for policy 0, policy_version 1470693 (0.0008) [2023-12-27 02:06:10,024][105692] Updated weights for policy 0, policy_version 1470703 (0.0008) [2023-12-27 02:06:10,190][105620] Updated weights for policy 1, policy_version 1472977 (0.0008) [2023-12-27 02:06:10,250][105620] Updated weights for policy 1, policy_version 1472987 (0.0009) [2023-12-27 02:06:10,313][105620] Updated weights for policy 1, policy_version 1472997 (0.0008) [2023-12-27 02:06:10,380][105620] Updated weights for policy 1, policy_version 1473007 (0.0008) [2023-12-27 02:06:10,763][105692] Updated weights for policy 0, policy_version 1470713 (0.0010) [2023-12-27 02:06:10,828][105692] Updated weights for policy 0, policy_version 1470723 (0.0009) [2023-12-27 02:06:10,888][105692] Updated weights for policy 0, policy_version 1470733 (0.0011) [2023-12-27 02:06:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 753704960. Throughput: 0: 9678.6, 1: 9800.9. Samples: 753713740. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:06:11,063][104569] Avg episode reward: [(0, '8262.742'), (1, '9175.831')] [2023-12-27 02:06:11,231][105620] Updated weights for policy 1, policy_version 1473017 (0.0007) [2023-12-27 02:06:11,291][105620] Updated weights for policy 1, policy_version 1473027 (0.0009) [2023-12-27 02:06:11,346][105620] Updated weights for policy 1, policy_version 1473037 (0.0009) [2023-12-27 02:06:11,647][105692] Updated weights for policy 0, policy_version 1470743 (0.0009) [2023-12-27 02:06:11,703][105692] Updated weights for policy 0, policy_version 1470753 (0.0009) [2023-12-27 02:06:11,771][105692] Updated weights for policy 0, policy_version 1470763 (0.0007) [2023-12-27 02:06:12,131][105620] Updated weights for policy 1, policy_version 1473047 (0.0009) [2023-12-27 02:06:12,183][105620] Updated weights for policy 1, policy_version 1473057 (0.0007) [2023-12-27 02:06:12,235][105620] Updated weights for policy 1, policy_version 1473067 (0.0006) [2023-12-27 02:06:12,506][105692] Updated weights for policy 0, policy_version 1470773 (0.0007) [2023-12-27 02:06:12,571][105692] Updated weights for policy 0, policy_version 1470783 (0.0008) [2023-12-27 02:06:12,624][105692] Updated weights for policy 0, policy_version 1470793 (0.0008) [2023-12-27 02:06:12,993][105620] Updated weights for policy 1, policy_version 1473077 (0.0009) [2023-12-27 02:06:13,043][105620] Updated weights for policy 1, policy_version 1473087 (0.0008) [2023-12-27 02:06:13,093][105620] Updated weights for policy 1, policy_version 1473097 (0.0007) [2023-12-27 02:06:13,440][105692] Updated weights for policy 0, policy_version 1470803 (0.0009) [2023-12-27 02:06:13,493][105692] Updated weights for policy 0, policy_version 1470813 (0.0010) [2023-12-27 02:06:13,547][105692] Updated weights for policy 0, policy_version 1470823 (0.0010) [2023-12-27 02:06:13,653][105620] Updated weights for policy 1, policy_version 1473107 (0.0005) [2023-12-27 02:06:13,701][105620] Updated weights for policy 1, policy_version 1473117 (0.0005) [2023-12-27 02:06:13,750][105620] Updated weights for policy 1, policy_version 1473127 (0.0005) [2023-12-27 02:06:14,270][105620] Updated weights for policy 1, policy_version 1473137 (0.0006) [2023-12-27 02:06:14,325][105620] Updated weights for policy 1, policy_version 1473147 (0.0005) [2023-12-27 02:06:14,376][105620] Updated weights for policy 1, policy_version 1473157 (0.0005) [2023-12-27 02:06:14,427][105620] Updated weights for policy 1, policy_version 1473167 (0.0005) [2023-12-27 02:06:14,494][105692] Updated weights for policy 0, policy_version 1470834 (0.0010) [2023-12-27 02:06:14,549][105692] Updated weights for policy 0, policy_version 1470845 (0.0010) [2023-12-27 02:06:14,603][105692] Updated weights for policy 0, policy_version 1470856 (0.0010) [2023-12-27 02:06:14,987][105620] Updated weights for policy 1, policy_version 1473177 (0.0010) [2023-12-27 02:06:15,054][105620] Updated weights for policy 1, policy_version 1473187 (0.0011) [2023-12-27 02:06:15,118][105620] Updated weights for policy 1, policy_version 1473197 (0.0011) [2023-12-27 02:06:15,449][105692] Updated weights for policy 0, policy_version 1470867 (0.0010) [2023-12-27 02:06:15,498][105692] Updated weights for policy 0, policy_version 1470877 (0.0008) [2023-12-27 02:06:15,552][105692] Updated weights for policy 0, policy_version 1470887 (0.0008) [2023-12-27 02:06:15,862][105620] Updated weights for policy 1, policy_version 1473207 (0.0011) [2023-12-27 02:06:15,920][105620] Updated weights for policy 1, policy_version 1473217 (0.0010) [2023-12-27 02:06:15,980][105620] Updated weights for policy 1, policy_version 1473227 (0.0010) [2023-12-27 02:06:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 753803264. Throughput: 0: 9523.4, 1: 9776.3. Samples: 753771312. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:06:16,062][104569] Avg episode reward: [(0, '8536.008'), (1, '9082.858')] [2023-12-27 02:06:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001473232_377200640.pth... [2023-12-27 02:06:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001470896_376602624.pth... [2023-12-27 02:06:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001472064_376897536.pth [2023-12-27 02:06:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001469808_376324096.pth [2023-12-27 02:06:16,281][105692] Updated weights for policy 0, policy_version 1470897 (0.0008) [2023-12-27 02:06:16,336][105692] Updated weights for policy 0, policy_version 1470907 (0.0011) [2023-12-27 02:06:16,386][105692] Updated weights for policy 0, policy_version 1470917 (0.0011) [2023-12-27 02:06:16,447][105692] Updated weights for policy 0, policy_version 1470927 (0.0008) [2023-12-27 02:06:16,561][105620] Updated weights for policy 1, policy_version 1473237 (0.0008) [2023-12-27 02:06:16,617][105620] Updated weights for policy 1, policy_version 1473247 (0.0006) [2023-12-27 02:06:16,681][105620] Updated weights for policy 1, policy_version 1473257 (0.0005) [2023-12-27 02:06:17,153][105692] Updated weights for policy 0, policy_version 1470937 (0.0007) [2023-12-27 02:06:17,183][105620] Updated weights for policy 1, policy_version 1473267 (0.0006) [2023-12-27 02:06:17,202][105692] Updated weights for policy 0, policy_version 1470948 (0.0008) [2023-12-27 02:06:17,234][105620] Updated weights for policy 1, policy_version 1473277 (0.0009) [2023-12-27 02:06:17,248][105692] Updated weights for policy 0, policy_version 1470958 (0.0008) [2023-12-27 02:06:17,293][105620] Updated weights for policy 1, policy_version 1473287 (0.0009) [2023-12-27 02:06:17,963][105692] Updated weights for policy 0, policy_version 1470968 (0.0008) [2023-12-27 02:06:18,021][105692] Updated weights for policy 0, policy_version 1470978 (0.0009) [2023-12-27 02:06:18,067][105692] Updated weights for policy 0, policy_version 1470988 (0.0009) [2023-12-27 02:06:18,075][105620] Updated weights for policy 1, policy_version 1473297 (0.0009) [2023-12-27 02:06:18,136][105620] Updated weights for policy 1, policy_version 1473307 (0.0007) [2023-12-27 02:06:18,184][105620] Updated weights for policy 1, policy_version 1473317 (0.0007) [2023-12-27 02:06:18,242][105620] Updated weights for policy 1, policy_version 1473327 (0.0008) [2023-12-27 02:06:18,814][105692] Updated weights for policy 0, policy_version 1470998 (0.0009) [2023-12-27 02:06:18,865][105692] Updated weights for policy 0, policy_version 1471008 (0.0009) [2023-12-27 02:06:18,915][105692] Updated weights for policy 0, policy_version 1471018 (0.0009) [2023-12-27 02:06:18,962][105620] Updated weights for policy 1, policy_version 1473337 (0.0009) [2023-12-27 02:06:19,020][105620] Updated weights for policy 1, policy_version 1473347 (0.0009) [2023-12-27 02:06:19,080][105620] Updated weights for policy 1, policy_version 1473357 (0.0008) [2023-12-27 02:06:19,736][105692] Updated weights for policy 0, policy_version 1471028 (0.0009) [2023-12-27 02:06:19,795][105620] Updated weights for policy 1, policy_version 1473367 (0.0008) [2023-12-27 02:06:19,799][105692] Updated weights for policy 0, policy_version 1471038 (0.0007) [2023-12-27 02:06:19,857][105620] Updated weights for policy 1, policy_version 1473377 (0.0009) [2023-12-27 02:06:19,863][105692] Updated weights for policy 0, policy_version 1471048 (0.0009) [2023-12-27 02:06:19,922][105620] Updated weights for policy 1, policy_version 1473387 (0.0007) [2023-12-27 02:06:20,618][105692] Updated weights for policy 0, policy_version 1471058 (0.0006) [2023-12-27 02:06:20,665][105620] Updated weights for policy 1, policy_version 1473397 (0.0008) [2023-12-27 02:06:20,686][105692] Updated weights for policy 0, policy_version 1471068 (0.0006) [2023-12-27 02:06:20,716][105620] Updated weights for policy 1, policy_version 1473407 (0.0007) [2023-12-27 02:06:20,754][105692] Updated weights for policy 0, policy_version 1471078 (0.0009) [2023-12-27 02:06:20,769][105620] Updated weights for policy 1, policy_version 1473417 (0.0006) [2023-12-27 02:06:20,819][105692] Updated weights for policy 0, policy_version 1471088 (0.0009) [2023-12-27 02:06:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 753901568. Throughput: 0: 9401.0, 1: 9849.8. Samples: 753889604. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:06:21,063][104569] Avg episode reward: [(0, '8808.537'), (1, '8898.441')] [2023-12-27 02:06:21,554][105620] Updated weights for policy 1, policy_version 1473427 (0.0008) [2023-12-27 02:06:21,564][105692] Updated weights for policy 0, policy_version 1471098 (0.0005) [2023-12-27 02:06:21,607][105620] Updated weights for policy 1, policy_version 1473437 (0.0008) [2023-12-27 02:06:21,636][105692] Updated weights for policy 0, policy_version 1471108 (0.0007) [2023-12-27 02:06:21,672][105620] Updated weights for policy 1, policy_version 1473447 (0.0007) [2023-12-27 02:06:21,703][105692] Updated weights for policy 0, policy_version 1471118 (0.0009) [2023-12-27 02:06:22,434][105692] Updated weights for policy 0, policy_version 1471128 (0.0009) [2023-12-27 02:06:22,493][105692] Updated weights for policy 0, policy_version 1471138 (0.0008) [2023-12-27 02:06:22,498][105620] Updated weights for policy 1, policy_version 1473457 (0.0009) [2023-12-27 02:06:22,548][105692] Updated weights for policy 0, policy_version 1471148 (0.0008) [2023-12-27 02:06:22,557][105620] Updated weights for policy 1, policy_version 1473467 (0.0010) [2023-12-27 02:06:22,621][105620] Updated weights for policy 1, policy_version 1473477 (0.0009) [2023-12-27 02:06:22,683][105620] Updated weights for policy 1, policy_version 1473487 (0.0008) [2023-12-27 02:06:23,222][105692] Updated weights for policy 0, policy_version 1471158 (0.0009) [2023-12-27 02:06:23,278][105692] Updated weights for policy 0, policy_version 1471168 (0.0009) [2023-12-27 02:06:23,341][105692] Updated weights for policy 0, policy_version 1471178 (0.0008) [2023-12-27 02:06:23,436][105620] Updated weights for policy 1, policy_version 1473497 (0.0009) [2023-12-27 02:06:23,494][105620] Updated weights for policy 1, policy_version 1473508 (0.0010) [2023-12-27 02:06:23,548][105620] Updated weights for policy 1, policy_version 1473519 (0.0010) [2023-12-27 02:06:23,938][105692] Updated weights for policy 0, policy_version 1471188 (0.0008) [2023-12-27 02:06:23,999][105692] Updated weights for policy 0, policy_version 1471198 (0.0008) [2023-12-27 02:06:24,055][105692] Updated weights for policy 0, policy_version 1471208 (0.0005) [2023-12-27 02:06:24,320][105620] Updated weights for policy 1, policy_version 1473529 (0.0010) [2023-12-27 02:06:24,384][105620] Updated weights for policy 1, policy_version 1473539 (0.0006) [2023-12-27 02:06:24,448][105620] Updated weights for policy 1, policy_version 1473549 (0.0009) [2023-12-27 02:06:24,608][105692] Updated weights for policy 0, policy_version 1471218 (0.0006) [2023-12-27 02:06:24,656][105692] Updated weights for policy 0, policy_version 1471228 (0.0005) [2023-12-27 02:06:24,709][105692] Updated weights for policy 0, policy_version 1471238 (0.0005) [2023-12-27 02:06:24,754][105692] Updated weights for policy 0, policy_version 1471248 (0.0005) [2023-12-27 02:06:25,003][105620] Updated weights for policy 1, policy_version 1473559 (0.0007) [2023-12-27 02:06:25,069][105620] Updated weights for policy 1, policy_version 1473569 (0.0008) [2023-12-27 02:06:25,125][105620] Updated weights for policy 1, policy_version 1473579 (0.0005) [2023-12-27 02:06:25,328][105692] Updated weights for policy 0, policy_version 1471258 (0.0005) [2023-12-27 02:06:25,388][105692] Updated weights for policy 0, policy_version 1471268 (0.0005) [2023-12-27 02:06:25,432][105692] Updated weights for policy 0, policy_version 1471278 (0.0005) [2023-12-27 02:06:25,644][105620] Updated weights for policy 1, policy_version 1473589 (0.0008) [2023-12-27 02:06:25,705][105620] Updated weights for policy 1, policy_version 1473599 (0.0010) [2023-12-27 02:06:25,765][105620] Updated weights for policy 1, policy_version 1473609 (0.0009) [2023-12-27 02:06:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 753999872. Throughput: 0: 9455.5, 1: 9959.6. Samples: 754010032. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:06:26,062][104569] Avg episode reward: [(0, '8715.471'), (1, '9172.322')] [2023-12-27 02:06:26,074][105692] Updated weights for policy 0, policy_version 1471288 (0.0008) [2023-12-27 02:06:26,133][105692] Updated weights for policy 0, policy_version 1471298 (0.0009) [2023-12-27 02:06:26,189][105692] Updated weights for policy 0, policy_version 1471308 (0.0008) [2023-12-27 02:06:26,546][105620] Updated weights for policy 1, policy_version 1473619 (0.0010) [2023-12-27 02:06:26,605][105620] Updated weights for policy 1, policy_version 1473629 (0.0006) [2023-12-27 02:06:26,666][105620] Updated weights for policy 1, policy_version 1473639 (0.0006) [2023-12-27 02:06:26,843][105692] Updated weights for policy 0, policy_version 1471318 (0.0008) [2023-12-27 02:06:26,902][105692] Updated weights for policy 0, policy_version 1471328 (0.0008) [2023-12-27 02:06:26,973][105692] Updated weights for policy 0, policy_version 1471338 (0.0008) [2023-12-27 02:06:27,264][105620] Updated weights for policy 1, policy_version 1473649 (0.0006) [2023-12-27 02:06:27,315][105620] Updated weights for policy 1, policy_version 1473659 (0.0009) [2023-12-27 02:06:27,366][105620] Updated weights for policy 1, policy_version 1473669 (0.0005) [2023-12-27 02:06:27,426][105620] Updated weights for policy 1, policy_version 1473679 (0.0005) [2023-12-27 02:06:27,781][105692] Updated weights for policy 0, policy_version 1471348 (0.0009) [2023-12-27 02:06:27,832][105692] Updated weights for policy 0, policy_version 1471358 (0.0010) [2023-12-27 02:06:27,883][105692] Updated weights for policy 0, policy_version 1471368 (0.0010) [2023-12-27 02:06:28,015][105620] Updated weights for policy 1, policy_version 1473689 (0.0010) [2023-12-27 02:06:28,066][105620] Updated weights for policy 1, policy_version 1473699 (0.0010) [2023-12-27 02:06:28,126][105620] Updated weights for policy 1, policy_version 1473709 (0.0007) [2023-12-27 02:06:28,571][105692] Updated weights for policy 0, policy_version 1471378 (0.0010) [2023-12-27 02:06:28,628][105692] Updated weights for policy 0, policy_version 1471388 (0.0009) [2023-12-27 02:06:28,680][105692] Updated weights for policy 0, policy_version 1471398 (0.0007) [2023-12-27 02:06:28,715][105620] Updated weights for policy 1, policy_version 1473719 (0.0009) [2023-12-27 02:06:28,726][105692] Updated weights for policy 0, policy_version 1471408 (0.0005) [2023-12-27 02:06:28,777][105620] Updated weights for policy 1, policy_version 1473729 (0.0011) [2023-12-27 02:06:28,832][105620] Updated weights for policy 1, policy_version 1473739 (0.0010) [2023-12-27 02:06:29,440][105692] Updated weights for policy 0, policy_version 1471418 (0.0005) [2023-12-27 02:06:29,498][105692] Updated weights for policy 0, policy_version 1471428 (0.0006) [2023-12-27 02:06:29,530][105620] Updated weights for policy 1, policy_version 1473749 (0.0008) [2023-12-27 02:06:29,557][105692] Updated weights for policy 0, policy_version 1471438 (0.0008) [2023-12-27 02:06:29,592][105620] Updated weights for policy 1, policy_version 1473759 (0.0006) [2023-12-27 02:06:29,651][105620] Updated weights for policy 1, policy_version 1473769 (0.0006) [2023-12-27 02:06:30,278][105692] Updated weights for policy 0, policy_version 1471448 (0.0006) [2023-12-27 02:06:30,295][105620] Updated weights for policy 1, policy_version 1473779 (0.0006) [2023-12-27 02:06:30,331][105692] Updated weights for policy 0, policy_version 1471458 (0.0008) [2023-12-27 02:06:30,355][105620] Updated weights for policy 1, policy_version 1473789 (0.0007) [2023-12-27 02:06:30,385][105692] Updated weights for policy 0, policy_version 1471468 (0.0009) [2023-12-27 02:06:30,415][105620] Updated weights for policy 1, policy_version 1473799 (0.0008) [2023-12-27 02:06:31,016][105620] Updated weights for policy 1, policy_version 1473809 (0.0008) [2023-12-27 02:06:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 754098176. Throughput: 0: 9500.3, 1: 10056.1. Samples: 754071516. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:06:31,062][104569] Avg episode reward: [(0, '8442.760'), (1, '9088.406')] [2023-12-27 02:06:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001471472_376750080.pth... [2023-12-27 02:06:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001470352_376463360.pth [2023-12-27 02:06:31,083][105620] Updated weights for policy 1, policy_version 1473819 (0.0009) [2023-12-27 02:06:31,143][105620] Updated weights for policy 1, policy_version 1473829 (0.0009) [2023-12-27 02:06:31,173][105692] Updated weights for policy 0, policy_version 1471478 (0.0007) [2023-12-27 02:06:31,209][105620] Updated weights for policy 1, policy_version 1473839 (0.0009) [2023-12-27 02:06:31,214][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001473840_377356288.pth... [2023-12-27 02:06:31,218][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001472672_377053184.pth [2023-12-27 02:06:31,232][105692] Updated weights for policy 0, policy_version 1471488 (0.0006) [2023-12-27 02:06:31,296][105692] Updated weights for policy 0, policy_version 1471498 (0.0009) [2023-12-27 02:06:31,962][105620] Updated weights for policy 1, policy_version 1473849 (0.0009) [2023-12-27 02:06:32,011][105620] Updated weights for policy 1, policy_version 1473859 (0.0008) [2023-12-27 02:06:32,031][105692] Updated weights for policy 0, policy_version 1471508 (0.0009) [2023-12-27 02:06:32,068][105620] Updated weights for policy 1, policy_version 1473869 (0.0006) [2023-12-27 02:06:32,090][105692] Updated weights for policy 0, policy_version 1471518 (0.0011) [2023-12-27 02:06:32,140][105692] Updated weights for policy 0, policy_version 1471528 (0.0007) [2023-12-27 02:06:32,789][105692] Updated weights for policy 0, policy_version 1471538 (0.0007) [2023-12-27 02:06:32,838][105692] Updated weights for policy 0, policy_version 1471548 (0.0010) [2023-12-27 02:06:32,895][105692] Updated weights for policy 0, policy_version 1471558 (0.0011) [2023-12-27 02:06:32,901][105620] Updated weights for policy 1, policy_version 1473879 (0.0006) [2023-12-27 02:06:32,950][105692] Updated weights for policy 0, policy_version 1471568 (0.0011) [2023-12-27 02:06:32,961][105620] Updated weights for policy 1, policy_version 1473889 (0.0006) [2023-12-27 02:06:33,022][105620] Updated weights for policy 1, policy_version 1473899 (0.0008) [2023-12-27 02:06:33,637][105692] Updated weights for policy 0, policy_version 1471578 (0.0005) [2023-12-27 02:06:33,683][105620] Updated weights for policy 1, policy_version 1473909 (0.0008) [2023-12-27 02:06:33,701][105692] Updated weights for policy 0, policy_version 1471588 (0.0007) [2023-12-27 02:06:33,739][105620] Updated weights for policy 1, policy_version 1473919 (0.0007) [2023-12-27 02:06:33,761][105692] Updated weights for policy 0, policy_version 1471598 (0.0006) [2023-12-27 02:06:33,790][105620] Updated weights for policy 1, policy_version 1473929 (0.0006) [2023-12-27 02:06:34,487][105692] Updated weights for policy 0, policy_version 1471608 (0.0009) [2023-12-27 02:06:34,487][105620] Updated weights for policy 1, policy_version 1473939 (0.0006) [2023-12-27 02:06:34,543][105692] Updated weights for policy 0, policy_version 1471618 (0.0007) [2023-12-27 02:06:34,544][105620] Updated weights for policy 1, policy_version 1473949 (0.0009) [2023-12-27 02:06:34,601][105692] Updated weights for policy 0, policy_version 1471628 (0.0007) [2023-12-27 02:06:34,603][105620] Updated weights for policy 1, policy_version 1473959 (0.0007) [2023-12-27 02:06:35,316][105620] Updated weights for policy 1, policy_version 1473969 (0.0008) [2023-12-27 02:06:35,351][105692] Updated weights for policy 0, policy_version 1471638 (0.0007) [2023-12-27 02:06:35,369][105620] Updated weights for policy 1, policy_version 1473979 (0.0008) [2023-12-27 02:06:35,411][105692] Updated weights for policy 0, policy_version 1471648 (0.0007) [2023-12-27 02:06:35,426][105620] Updated weights for policy 1, policy_version 1473989 (0.0006) [2023-12-27 02:06:35,476][105692] Updated weights for policy 0, policy_version 1471658 (0.0008) [2023-12-27 02:06:35,482][105620] Updated weights for policy 1, policy_version 1473999 (0.0006) [2023-12-27 02:06:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 754196480. Throughput: 0: 9581.3, 1: 10013.2. Samples: 754188716. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:06:36,063][104569] Avg episode reward: [(0, '8539.076'), (1, '9091.646')] [2023-12-27 02:06:36,072][105620] Updated weights for policy 1, policy_version 1474009 (0.0005) [2023-12-27 02:06:36,133][105620] Updated weights for policy 1, policy_version 1474019 (0.0009) [2023-12-27 02:06:36,200][105620] Updated weights for policy 1, policy_version 1474029 (0.0008) [2023-12-27 02:06:36,274][105692] Updated weights for policy 0, policy_version 1471668 (0.0008) [2023-12-27 02:06:36,335][105692] Updated weights for policy 0, policy_version 1471678 (0.0009) [2023-12-27 02:06:36,390][105692] Updated weights for policy 0, policy_version 1471688 (0.0009) [2023-12-27 02:06:36,823][105620] Updated weights for policy 1, policy_version 1474039 (0.0010) [2023-12-27 02:06:36,886][105620] Updated weights for policy 1, policy_version 1474049 (0.0011) [2023-12-27 02:06:36,949][105620] Updated weights for policy 1, policy_version 1474059 (0.0011) [2023-12-27 02:06:37,225][105692] Updated weights for policy 0, policy_version 1471698 (0.0009) [2023-12-27 02:06:37,281][105692] Updated weights for policy 0, policy_version 1471708 (0.0009) [2023-12-27 02:06:37,338][105692] Updated weights for policy 0, policy_version 1471718 (0.0009) [2023-12-27 02:06:37,619][105620] Updated weights for policy 1, policy_version 1474069 (0.0008) [2023-12-27 02:06:37,667][105620] Updated weights for policy 1, policy_version 1474079 (0.0008) [2023-12-27 02:06:37,718][105620] Updated weights for policy 1, policy_version 1474089 (0.0009) [2023-12-27 02:06:38,171][105692] Updated weights for policy 0, policy_version 1471729 (0.0010) [2023-12-27 02:06:38,220][105692] Updated weights for policy 0, policy_version 1471739 (0.0008) [2023-12-27 02:06:38,272][105692] Updated weights for policy 0, policy_version 1471749 (0.0009) [2023-12-27 02:06:38,327][105692] Updated weights for policy 0, policy_version 1471759 (0.0009) [2023-12-27 02:06:38,399][105620] Updated weights for policy 1, policy_version 1474099 (0.0009) [2023-12-27 02:06:38,465][105620] Updated weights for policy 1, policy_version 1474109 (0.0006) [2023-12-27 02:06:38,533][105620] Updated weights for policy 1, policy_version 1474119 (0.0006) [2023-12-27 02:06:39,081][105620] Updated weights for policy 1, policy_version 1474129 (0.0008) [2023-12-27 02:06:39,151][105620] Updated weights for policy 1, policy_version 1474139 (0.0006) [2023-12-27 02:06:39,217][105620] Updated weights for policy 1, policy_version 1474149 (0.0008) [2023-12-27 02:06:39,222][105692] Updated weights for policy 0, policy_version 1471769 (0.0007) [2023-12-27 02:06:39,282][105620] Updated weights for policy 1, policy_version 1474159 (0.0009) [2023-12-27 02:06:39,288][105692] Updated weights for policy 0, policy_version 1471779 (0.0008) [2023-12-27 02:06:39,361][105692] Updated weights for policy 0, policy_version 1471789 (0.0008) [2023-12-27 02:06:40,012][105620] Updated weights for policy 1, policy_version 1474169 (0.0008) [2023-12-27 02:06:40,072][105620] Updated weights for policy 1, policy_version 1474179 (0.0008) [2023-12-27 02:06:40,085][105692] Updated weights for policy 0, policy_version 1471799 (0.0008) [2023-12-27 02:06:40,131][105620] Updated weights for policy 1, policy_version 1474189 (0.0008) [2023-12-27 02:06:40,141][105692] Updated weights for policy 0, policy_version 1471809 (0.0006) [2023-12-27 02:06:40,205][105692] Updated weights for policy 0, policy_version 1471819 (0.0010) [2023-12-27 02:06:40,928][105620] Updated weights for policy 1, policy_version 1474199 (0.0009) [2023-12-27 02:06:40,931][105692] Updated weights for policy 0, policy_version 1471829 (0.0008) [2023-12-27 02:06:40,976][105620] Updated weights for policy 1, policy_version 1474209 (0.0007) [2023-12-27 02:06:40,979][105692] Updated weights for policy 0, policy_version 1471839 (0.0007) [2023-12-27 02:06:41,026][105620] Updated weights for policy 1, policy_version 1474219 (0.0008) [2023-12-27 02:06:41,034][105692] Updated weights for policy 0, policy_version 1471849 (0.0007) [2023-12-27 02:06:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 754294784. Throughput: 0: 9516.3, 1: 10047.7. Samples: 754303352. Policy #0 lag: (min: 7.0, avg: 9.6, max: 39.0) [2023-12-27 02:06:41,062][104569] Avg episode reward: [(0, '8717.410'), (1, '9085.100')] [2023-12-27 02:06:41,832][105620] Updated weights for policy 1, policy_version 1474229 (0.0009) [2023-12-27 02:06:41,862][105692] Updated weights for policy 0, policy_version 1471859 (0.0008) [2023-12-27 02:06:41,900][105620] Updated weights for policy 1, policy_version 1474239 (0.0011) [2023-12-27 02:06:41,921][105692] Updated weights for policy 0, policy_version 1471869 (0.0007) [2023-12-27 02:06:41,966][105620] Updated weights for policy 1, policy_version 1474249 (0.0010) [2023-12-27 02:06:41,980][105692] Updated weights for policy 0, policy_version 1471879 (0.0006) [2023-12-27 02:06:42,674][105620] Updated weights for policy 1, policy_version 1474259 (0.0010) [2023-12-27 02:06:42,680][105692] Updated weights for policy 0, policy_version 1471889 (0.0006) [2023-12-27 02:06:42,736][105620] Updated weights for policy 1, policy_version 1474269 (0.0006) [2023-12-27 02:06:42,738][105692] Updated weights for policy 0, policy_version 1471899 (0.0006) [2023-12-27 02:06:42,794][105692] Updated weights for policy 0, policy_version 1471909 (0.0006) [2023-12-27 02:06:42,795][105620] Updated weights for policy 1, policy_version 1474279 (0.0009) [2023-12-27 02:06:42,850][105692] Updated weights for policy 0, policy_version 1471919 (0.0006) [2023-12-27 02:06:43,494][105692] Updated weights for policy 0, policy_version 1471929 (0.0011) [2023-12-27 02:06:43,539][105620] Updated weights for policy 1, policy_version 1474289 (0.0008) [2023-12-27 02:06:43,547][105692] Updated weights for policy 0, policy_version 1471939 (0.0011) [2023-12-27 02:06:43,595][105620] Updated weights for policy 1, policy_version 1474299 (0.0007) [2023-12-27 02:06:43,610][105692] Updated weights for policy 0, policy_version 1471949 (0.0010) [2023-12-27 02:06:43,641][105620] Updated weights for policy 1, policy_version 1474309 (0.0006) [2023-12-27 02:06:43,697][105620] Updated weights for policy 1, policy_version 1474319 (0.0006) [2023-12-27 02:06:44,283][105620] Updated weights for policy 1, policy_version 1474329 (0.0007) [2023-12-27 02:06:44,332][105692] Updated weights for policy 0, policy_version 1471959 (0.0007) [2023-12-27 02:06:44,334][105620] Updated weights for policy 1, policy_version 1474339 (0.0007) [2023-12-27 02:06:44,391][105620] Updated weights for policy 1, policy_version 1474349 (0.0007) [2023-12-27 02:06:44,393][105692] Updated weights for policy 0, policy_version 1471969 (0.0006) [2023-12-27 02:06:44,456][105692] Updated weights for policy 0, policy_version 1471979 (0.0008) [2023-12-27 02:06:45,005][105620] Updated weights for policy 1, policy_version 1474359 (0.0009) [2023-12-27 02:06:45,075][105620] Updated weights for policy 1, policy_version 1474369 (0.0008) [2023-12-27 02:06:45,135][105620] Updated weights for policy 1, policy_version 1474379 (0.0006) [2023-12-27 02:06:45,150][105692] Updated weights for policy 0, policy_version 1471989 (0.0011) [2023-12-27 02:06:45,211][105692] Updated weights for policy 0, policy_version 1471999 (0.0011) [2023-12-27 02:06:45,275][105692] Updated weights for policy 0, policy_version 1472009 (0.0011) [2023-12-27 02:06:45,864][105620] Updated weights for policy 1, policy_version 1474389 (0.0010) [2023-12-27 02:06:45,915][105620] Updated weights for policy 1, policy_version 1474399 (0.0010) [2023-12-27 02:06:45,965][105620] Updated weights for policy 1, policy_version 1474409 (0.0009) [2023-12-27 02:06:45,975][105692] Updated weights for policy 0, policy_version 1472019 (0.0009) [2023-12-27 02:06:46,030][105692] Updated weights for policy 0, policy_version 1472029 (0.0008) [2023-12-27 02:06:46,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19524.2, 300 sec: 19605.2). Total num frames: 754393088. Throughput: 0: 9449.0, 1: 10082.1. Samples: 754361424. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:06:46,063][104569] Avg episode reward: [(0, '8802.895'), (1, '9174.488')] [2023-12-27 02:06:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001474416_377503744.pth... [2023-12-27 02:06:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001473232_377200640.pth [2023-12-27 02:06:46,077][105692] Updated weights for policy 0, policy_version 1472039 (0.0010) [2023-12-27 02:06:46,122][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001472048_376897536.pth... [2023-12-27 02:06:46,126][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001470896_376602624.pth [2023-12-27 02:06:46,719][105620] Updated weights for policy 1, policy_version 1474419 (0.0005) [2023-12-27 02:06:46,767][105620] Updated weights for policy 1, policy_version 1474429 (0.0005) [2023-12-27 02:06:46,790][105692] Updated weights for policy 0, policy_version 1472049 (0.0010) [2023-12-27 02:06:46,815][105620] Updated weights for policy 1, policy_version 1474439 (0.0007) [2023-12-27 02:06:46,848][105692] Updated weights for policy 0, policy_version 1472059 (0.0010) [2023-12-27 02:06:46,907][105692] Updated weights for policy 0, policy_version 1472069 (0.0010) [2023-12-27 02:06:46,962][105692] Updated weights for policy 0, policy_version 1472079 (0.0010) [2023-12-27 02:06:47,413][105620] Updated weights for policy 1, policy_version 1474449 (0.0010) [2023-12-27 02:06:47,465][105620] Updated weights for policy 1, policy_version 1474459 (0.0005) [2023-12-27 02:06:47,514][105620] Updated weights for policy 1, policy_version 1474469 (0.0007) [2023-12-27 02:06:47,566][105692] Updated weights for policy 0, policy_version 1472089 (0.0009) [2023-12-27 02:06:47,572][105620] Updated weights for policy 1, policy_version 1474479 (0.0007) [2023-12-27 02:06:47,617][105692] Updated weights for policy 0, policy_version 1472099 (0.0010) [2023-12-27 02:06:47,675][105692] Updated weights for policy 0, policy_version 1472109 (0.0010) [2023-12-27 02:06:48,218][105620] Updated weights for policy 1, policy_version 1474489 (0.0010) [2023-12-27 02:06:48,276][105620] Updated weights for policy 1, policy_version 1474499 (0.0010) [2023-12-27 02:06:48,336][105620] Updated weights for policy 1, policy_version 1474509 (0.0010) [2023-12-27 02:06:48,407][105692] Updated weights for policy 0, policy_version 1472119 (0.0011) [2023-12-27 02:06:48,460][105692] Updated weights for policy 0, policy_version 1472129 (0.0010) [2023-12-27 02:06:48,519][105692] Updated weights for policy 0, policy_version 1472139 (0.0010) [2023-12-27 02:06:48,960][105620] Updated weights for policy 1, policy_version 1474519 (0.0009) [2023-12-27 02:06:49,023][105620] Updated weights for policy 1, policy_version 1474529 (0.0008) [2023-12-27 02:06:49,085][105620] Updated weights for policy 1, policy_version 1474539 (0.0008) [2023-12-27 02:06:49,273][105692] Updated weights for policy 0, policy_version 1472149 (0.0011) [2023-12-27 02:06:49,332][105692] Updated weights for policy 0, policy_version 1472159 (0.0011) [2023-12-27 02:06:49,401][105692] Updated weights for policy 0, policy_version 1472169 (0.0011) [2023-12-27 02:06:49,855][105620] Updated weights for policy 1, policy_version 1474549 (0.0009) [2023-12-27 02:06:49,924][105620] Updated weights for policy 1, policy_version 1474559 (0.0008) [2023-12-27 02:06:49,989][105620] Updated weights for policy 1, policy_version 1474569 (0.0008) [2023-12-27 02:06:50,114][105692] Updated weights for policy 0, policy_version 1472179 (0.0009) [2023-12-27 02:06:50,162][105692] Updated weights for policy 0, policy_version 1472189 (0.0005) [2023-12-27 02:06:50,224][105692] Updated weights for policy 0, policy_version 1472199 (0.0007) [2023-12-27 02:06:50,705][105620] Updated weights for policy 1, policy_version 1474579 (0.0008) [2023-12-27 02:06:50,762][105620] Updated weights for policy 1, policy_version 1474589 (0.0006) [2023-12-27 02:06:50,818][105620] Updated weights for policy 1, policy_version 1474599 (0.0009) [2023-12-27 02:06:50,948][105692] Updated weights for policy 0, policy_version 1472209 (0.0008) [2023-12-27 02:06:51,003][105692] Updated weights for policy 0, policy_version 1472219 (0.0010) [2023-12-27 02:06:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19661.0, 300 sec: 19633.0). Total num frames: 754491392. Throughput: 0: 9606.1, 1: 10069.9. Samples: 754482316. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:06:51,062][104569] Avg episode reward: [(0, '8800.919'), (1, '9176.729')] [2023-12-27 02:06:51,072][105692] Updated weights for policy 0, policy_version 1472229 (0.0009) [2023-12-27 02:06:51,134][105692] Updated weights for policy 0, policy_version 1472239 (0.0008) [2023-12-27 02:06:51,575][105620] Updated weights for policy 1, policy_version 1474609 (0.0009) [2023-12-27 02:06:51,637][105620] Updated weights for policy 1, policy_version 1474619 (0.0009) [2023-12-27 02:06:51,699][105620] Updated weights for policy 1, policy_version 1474629 (0.0009) [2023-12-27 02:06:51,767][105620] Updated weights for policy 1, policy_version 1474639 (0.0008) [2023-12-27 02:06:51,905][105692] Updated weights for policy 0, policy_version 1472249 (0.0010) [2023-12-27 02:06:51,950][105692] Updated weights for policy 0, policy_version 1472259 (0.0010) [2023-12-27 02:06:51,996][105692] Updated weights for policy 0, policy_version 1472269 (0.0010) [2023-12-27 02:06:52,540][105620] Updated weights for policy 1, policy_version 1474649 (0.0008) [2023-12-27 02:06:52,602][105620] Updated weights for policy 1, policy_version 1474659 (0.0007) [2023-12-27 02:06:52,651][105620] Updated weights for policy 1, policy_version 1474669 (0.0008) [2023-12-27 02:06:52,683][105692] Updated weights for policy 0, policy_version 1472279 (0.0007) [2023-12-27 02:06:52,742][105692] Updated weights for policy 0, policy_version 1472289 (0.0007) [2023-12-27 02:06:52,798][105692] Updated weights for policy 0, policy_version 1472299 (0.0007) [2023-12-27 02:06:53,365][105692] Updated weights for policy 0, policy_version 1472309 (0.0006) [2023-12-27 02:06:53,427][105692] Updated weights for policy 0, policy_version 1472319 (0.0005) [2023-12-27 02:06:53,456][105620] Updated weights for policy 1, policy_version 1474679 (0.0008) [2023-12-27 02:06:53,479][105692] Updated weights for policy 0, policy_version 1472329 (0.0006) [2023-12-27 02:06:53,514][105620] Updated weights for policy 1, policy_version 1474689 (0.0006) [2023-12-27 02:06:53,567][105620] Updated weights for policy 1, policy_version 1474699 (0.0007) [2023-12-27 02:06:54,126][105692] Updated weights for policy 0, policy_version 1472339 (0.0008) [2023-12-27 02:06:54,185][105692] Updated weights for policy 0, policy_version 1472349 (0.0009) [2023-12-27 02:06:54,243][105692] Updated weights for policy 0, policy_version 1472359 (0.0009) [2023-12-27 02:06:54,316][105620] Updated weights for policy 1, policy_version 1474709 (0.0009) [2023-12-27 02:06:54,382][105620] Updated weights for policy 1, policy_version 1474719 (0.0010) [2023-12-27 02:06:54,436][105620] Updated weights for policy 1, policy_version 1474729 (0.0010) [2023-12-27 02:06:54,855][105692] Updated weights for policy 0, policy_version 1472369 (0.0009) [2023-12-27 02:06:54,911][105692] Updated weights for policy 0, policy_version 1472379 (0.0005) [2023-12-27 02:06:54,970][105692] Updated weights for policy 0, policy_version 1472389 (0.0006) [2023-12-27 02:06:55,025][105692] Updated weights for policy 0, policy_version 1472399 (0.0005) [2023-12-27 02:06:55,328][105620] Updated weights for policy 1, policy_version 1474739 (0.0010) [2023-12-27 02:06:55,391][105620] Updated weights for policy 1, policy_version 1474749 (0.0010) [2023-12-27 02:06:55,449][105620] Updated weights for policy 1, policy_version 1474759 (0.0010) [2023-12-27 02:06:55,553][105692] Updated weights for policy 0, policy_version 1472409 (0.0009) [2023-12-27 02:06:55,611][105692] Updated weights for policy 0, policy_version 1472419 (0.0009) [2023-12-27 02:06:55,672][105692] Updated weights for policy 0, policy_version 1472429 (0.0008) [2023-12-27 02:06:56,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 754589696. Throughput: 0: 9691.1, 1: 9979.3. Samples: 754598908. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:06:56,062][104569] Avg episode reward: [(0, '8714.032'), (1, '9174.332')] [2023-12-27 02:06:56,283][105620] Updated weights for policy 1, policy_version 1474770 (0.0009) [2023-12-27 02:06:56,307][105692] Updated weights for policy 0, policy_version 1472439 (0.0008) [2023-12-27 02:06:56,329][105620] Updated weights for policy 1, policy_version 1474780 (0.0007) [2023-12-27 02:06:56,358][105692] Updated weights for policy 0, policy_version 1472449 (0.0008) [2023-12-27 02:06:56,377][105620] Updated weights for policy 1, policy_version 1474790 (0.0006) [2023-12-27 02:06:56,412][105692] Updated weights for policy 0, policy_version 1472459 (0.0008) [2023-12-27 02:06:56,428][105620] Updated weights for policy 1, policy_version 1474800 (0.0007) [2023-12-27 02:06:57,149][105692] Updated weights for policy 0, policy_version 1472469 (0.0007) [2023-12-27 02:06:57,181][105620] Updated weights for policy 1, policy_version 1474810 (0.0005) [2023-12-27 02:06:57,212][105692] Updated weights for policy 0, policy_version 1472479 (0.0006) [2023-12-27 02:06:57,230][105620] Updated weights for policy 1, policy_version 1474820 (0.0005) [2023-12-27 02:06:57,268][105692] Updated weights for policy 0, policy_version 1472489 (0.0009) [2023-12-27 02:06:57,282][105620] Updated weights for policy 1, policy_version 1474830 (0.0006) [2023-12-27 02:06:57,846][105692] Updated weights for policy 0, policy_version 1472499 (0.0006) [2023-12-27 02:06:57,868][105620] Updated weights for policy 1, policy_version 1474840 (0.0007) [2023-12-27 02:06:57,897][105692] Updated weights for policy 0, policy_version 1472509 (0.0010) [2023-12-27 02:06:57,919][105620] Updated weights for policy 1, policy_version 1474850 (0.0007) [2023-12-27 02:06:57,945][105692] Updated weights for policy 0, policy_version 1472519 (0.0008) [2023-12-27 02:06:57,976][105620] Updated weights for policy 1, policy_version 1474860 (0.0006) [2023-12-27 02:06:58,640][105620] Updated weights for policy 1, policy_version 1474870 (0.0008) [2023-12-27 02:06:58,714][105620] Updated weights for policy 1, policy_version 1474880 (0.0008) [2023-12-27 02:06:58,783][105620] Updated weights for policy 1, policy_version 1474890 (0.0008) [2023-12-27 02:06:58,842][105692] Updated weights for policy 0, policy_version 1472529 (0.0007) [2023-12-27 02:06:58,900][105692] Updated weights for policy 0, policy_version 1472539 (0.0008) [2023-12-27 02:06:58,962][105692] Updated weights for policy 0, policy_version 1472549 (0.0008) [2023-12-27 02:06:59,024][105692] Updated weights for policy 0, policy_version 1472559 (0.0008) [2023-12-27 02:06:59,578][105620] Updated weights for policy 1, policy_version 1474900 (0.0007) [2023-12-27 02:06:59,638][105620] Updated weights for policy 1, policy_version 1474910 (0.0007) [2023-12-27 02:06:59,693][105620] Updated weights for policy 1, policy_version 1474920 (0.0010) [2023-12-27 02:06:59,804][105692] Updated weights for policy 0, policy_version 1472569 (0.0008) [2023-12-27 02:06:59,866][105692] Updated weights for policy 0, policy_version 1472579 (0.0008) [2023-12-27 02:06:59,919][105692] Updated weights for policy 0, policy_version 1472589 (0.0009) [2023-12-27 02:07:00,296][105620] Updated weights for policy 1, policy_version 1474930 (0.0010) [2023-12-27 02:07:00,348][105620] Updated weights for policy 1, policy_version 1474940 (0.0010) [2023-12-27 02:07:00,404][105620] Updated weights for policy 1, policy_version 1474950 (0.0007) [2023-12-27 02:07:00,459][105620] Updated weights for policy 1, policy_version 1474960 (0.0005) [2023-12-27 02:07:00,773][105692] Updated weights for policy 0, policy_version 1472600 (0.0009) [2023-12-27 02:07:00,830][105692] Updated weights for policy 0, policy_version 1472610 (0.0008) [2023-12-27 02:07:00,883][105692] Updated weights for policy 0, policy_version 1472620 (0.0008) [2023-12-27 02:07:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.9, 300 sec: 19633.0). Total num frames: 754688000. Throughput: 0: 9759.3, 1: 9967.5. Samples: 754659016. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:07:01,062][104569] Avg episode reward: [(0, '8256.790'), (1, '9081.873')] [2023-12-27 02:07:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001474960_377643008.pth... [2023-12-27 02:07:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001472624_377044992.pth... [2023-12-27 02:07:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001473840_377356288.pth [2023-12-27 02:07:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001471472_376750080.pth [2023-12-27 02:07:01,146][105620] Updated weights for policy 1, policy_version 1474970 (0.0006) [2023-12-27 02:07:01,207][105620] Updated weights for policy 1, policy_version 1474980 (0.0007) [2023-12-27 02:07:01,272][105620] Updated weights for policy 1, policy_version 1474990 (0.0008) [2023-12-27 02:07:01,710][105692] Updated weights for policy 0, policy_version 1472630 (0.0007) [2023-12-27 02:07:01,774][105692] Updated weights for policy 0, policy_version 1472640 (0.0008) [2023-12-27 02:07:01,826][105692] Updated weights for policy 0, policy_version 1472650 (0.0008) [2023-12-27 02:07:01,912][105620] Updated weights for policy 1, policy_version 1475000 (0.0006) [2023-12-27 02:07:01,972][105620] Updated weights for policy 1, policy_version 1475010 (0.0005) [2023-12-27 02:07:02,038][105620] Updated weights for policy 1, policy_version 1475020 (0.0006) [2023-12-27 02:07:02,532][105692] Updated weights for policy 0, policy_version 1472660 (0.0008) [2023-12-27 02:07:02,583][105692] Updated weights for policy 0, policy_version 1472671 (0.0009) [2023-12-27 02:07:02,633][105692] Updated weights for policy 0, policy_version 1472681 (0.0009) [2023-12-27 02:07:02,754][105620] Updated weights for policy 1, policy_version 1475030 (0.0006) [2023-12-27 02:07:02,809][105620] Updated weights for policy 1, policy_version 1475040 (0.0010) [2023-12-27 02:07:02,868][105620] Updated weights for policy 1, policy_version 1475050 (0.0008) [2023-12-27 02:07:03,318][105692] Updated weights for policy 0, policy_version 1472691 (0.0008) [2023-12-27 02:07:03,363][105692] Updated weights for policy 0, policy_version 1472701 (0.0005) [2023-12-27 02:07:03,411][105692] Updated weights for policy 0, policy_version 1472711 (0.0005) [2023-12-27 02:07:03,556][105620] Updated weights for policy 1, policy_version 1475060 (0.0009) [2023-12-27 02:07:03,619][105620] Updated weights for policy 1, policy_version 1475070 (0.0009) [2023-12-27 02:07:03,677][105620] Updated weights for policy 1, policy_version 1475080 (0.0007) [2023-12-27 02:07:04,004][105692] Updated weights for policy 0, policy_version 1472721 (0.0005) [2023-12-27 02:07:04,067][105692] Updated weights for policy 0, policy_version 1472731 (0.0009) [2023-12-27 02:07:04,123][105692] Updated weights for policy 0, policy_version 1472741 (0.0009) [2023-12-27 02:07:04,189][105692] Updated weights for policy 0, policy_version 1472751 (0.0007) [2023-12-27 02:07:04,364][105620] Updated weights for policy 1, policy_version 1475090 (0.0005) [2023-12-27 02:07:04,431][105620] Updated weights for policy 1, policy_version 1475100 (0.0009) [2023-12-27 02:07:04,488][105620] Updated weights for policy 1, policy_version 1475110 (0.0008) [2023-12-27 02:07:04,551][105620] Updated weights for policy 1, policy_version 1475120 (0.0008) [2023-12-27 02:07:04,887][105692] Updated weights for policy 0, policy_version 1472761 (0.0008) [2023-12-27 02:07:04,942][105692] Updated weights for policy 0, policy_version 1472771 (0.0009) [2023-12-27 02:07:04,989][105692] Updated weights for policy 0, policy_version 1472781 (0.0009) [2023-12-27 02:07:05,317][105620] Updated weights for policy 1, policy_version 1475130 (0.0009) [2023-12-27 02:07:05,373][105620] Updated weights for policy 1, policy_version 1475140 (0.0008) [2023-12-27 02:07:05,431][105620] Updated weights for policy 1, policy_version 1475150 (0.0009) [2023-12-27 02:07:05,756][105692] Updated weights for policy 0, policy_version 1472791 (0.0009) [2023-12-27 02:07:05,816][105692] Updated weights for policy 0, policy_version 1472801 (0.0009) [2023-12-27 02:07:05,882][105692] Updated weights for policy 0, policy_version 1472811 (0.0009) [2023-12-27 02:07:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 754786304. Throughput: 0: 9788.7, 1: 9892.7. Samples: 754775268. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:07:06,062][104569] Avg episode reward: [(0, '8349.809'), (1, '8990.985')] [2023-12-27 02:07:06,173][105620] Updated weights for policy 1, policy_version 1475160 (0.0008) [2023-12-27 02:07:06,232][105620] Updated weights for policy 1, policy_version 1475170 (0.0009) [2023-12-27 02:07:06,298][105620] Updated weights for policy 1, policy_version 1475180 (0.0009) [2023-12-27 02:07:06,608][105692] Updated weights for policy 0, policy_version 1472821 (0.0008) [2023-12-27 02:07:06,675][105692] Updated weights for policy 0, policy_version 1472831 (0.0006) [2023-12-27 02:07:06,744][105692] Updated weights for policy 0, policy_version 1472841 (0.0008) [2023-12-27 02:07:06,984][105620] Updated weights for policy 1, policy_version 1475190 (0.0009) [2023-12-27 02:07:07,044][105620] Updated weights for policy 1, policy_version 1475200 (0.0007) [2023-12-27 02:07:07,106][105620] Updated weights for policy 1, policy_version 1475210 (0.0009) [2023-12-27 02:07:07,467][105692] Updated weights for policy 0, policy_version 1472851 (0.0009) [2023-12-27 02:07:07,526][105692] Updated weights for policy 0, policy_version 1472861 (0.0009) [2023-12-27 02:07:07,576][105692] Updated weights for policy 0, policy_version 1472871 (0.0010) [2023-12-27 02:07:07,808][105620] Updated weights for policy 1, policy_version 1475220 (0.0008) [2023-12-27 02:07:07,866][105620] Updated weights for policy 1, policy_version 1475230 (0.0008) [2023-12-27 02:07:07,918][105620] Updated weights for policy 1, policy_version 1475240 (0.0008) [2023-12-27 02:07:08,268][105692] Updated weights for policy 0, policy_version 1472881 (0.0010) [2023-12-27 02:07:08,322][105692] Updated weights for policy 0, policy_version 1472891 (0.0011) [2023-12-27 02:07:08,386][105692] Updated weights for policy 0, policy_version 1472901 (0.0011) [2023-12-27 02:07:08,449][105692] Updated weights for policy 0, policy_version 1472911 (0.0011) [2023-12-27 02:07:08,710][105620] Updated weights for policy 1, policy_version 1475250 (0.0009) [2023-12-27 02:07:08,764][105620] Updated weights for policy 1, policy_version 1475260 (0.0008) [2023-12-27 02:07:08,824][105620] Updated weights for policy 1, policy_version 1475270 (0.0009) [2023-12-27 02:07:08,882][105620] Updated weights for policy 1, policy_version 1475280 (0.0009) [2023-12-27 02:07:09,106][105692] Updated weights for policy 0, policy_version 1472921 (0.0010) [2023-12-27 02:07:09,167][105692] Updated weights for policy 0, policy_version 1472931 (0.0008) [2023-12-27 02:07:09,222][105692] Updated weights for policy 0, policy_version 1472941 (0.0006) [2023-12-27 02:07:09,645][105620] Updated weights for policy 1, policy_version 1475290 (0.0008) [2023-12-27 02:07:09,715][105620] Updated weights for policy 1, policy_version 1475300 (0.0009) [2023-12-27 02:07:09,773][105620] Updated weights for policy 1, policy_version 1475310 (0.0008) [2023-12-27 02:07:09,926][105692] Updated weights for policy 0, policy_version 1472951 (0.0009) [2023-12-27 02:07:09,993][105692] Updated weights for policy 0, policy_version 1472961 (0.0011) [2023-12-27 02:07:10,062][105692] Updated weights for policy 0, policy_version 1472971 (0.0010) [2023-12-27 02:07:10,459][105620] Updated weights for policy 1, policy_version 1475320 (0.0008) [2023-12-27 02:07:10,532][105620] Updated weights for policy 1, policy_version 1475330 (0.0009) [2023-12-27 02:07:10,601][105620] Updated weights for policy 1, policy_version 1475340 (0.0008) [2023-12-27 02:07:10,819][105692] Updated weights for policy 0, policy_version 1472981 (0.0010) [2023-12-27 02:07:10,884][105692] Updated weights for policy 0, policy_version 1472991 (0.0011) [2023-12-27 02:07:10,940][105692] Updated weights for policy 0, policy_version 1473001 (0.0006) [2023-12-27 02:07:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 754884608. Throughput: 0: 9702.9, 1: 9849.8. Samples: 754889904. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:07:11,063][104569] Avg episode reward: [(0, '8896.218'), (1, '8719.015')] [2023-12-27 02:07:11,420][105620] Updated weights for policy 1, policy_version 1475350 (0.0008) [2023-12-27 02:07:11,479][105620] Updated weights for policy 1, policy_version 1475360 (0.0008) [2023-12-27 02:07:11,538][105620] Updated weights for policy 1, policy_version 1475370 (0.0008) [2023-12-27 02:07:11,668][105692] Updated weights for policy 0, policy_version 1473011 (0.0009) [2023-12-27 02:07:11,739][105692] Updated weights for policy 0, policy_version 1473021 (0.0009) [2023-12-27 02:07:11,794][105692] Updated weights for policy 0, policy_version 1473031 (0.0009) [2023-12-27 02:07:12,291][105620] Updated weights for policy 1, policy_version 1475380 (0.0008) [2023-12-27 02:07:12,362][105620] Updated weights for policy 1, policy_version 1475390 (0.0009) [2023-12-27 02:07:12,419][105620] Updated weights for policy 1, policy_version 1475400 (0.0010) [2023-12-27 02:07:12,513][105692] Updated weights for policy 0, policy_version 1473041 (0.0009) [2023-12-27 02:07:12,573][105692] Updated weights for policy 0, policy_version 1473051 (0.0006) [2023-12-27 02:07:12,635][105692] Updated weights for policy 0, policy_version 1473061 (0.0006) [2023-12-27 02:07:12,695][105692] Updated weights for policy 0, policy_version 1473071 (0.0008) [2023-12-27 02:07:13,213][105620] Updated weights for policy 1, policy_version 1475410 (0.0010) [2023-12-27 02:07:13,278][105620] Updated weights for policy 1, policy_version 1475420 (0.0009) [2023-12-27 02:07:13,343][105620] Updated weights for policy 1, policy_version 1475430 (0.0009) [2023-12-27 02:07:13,389][105692] Updated weights for policy 0, policy_version 1473081 (0.0007) [2023-12-27 02:07:13,406][105620] Updated weights for policy 1, policy_version 1475440 (0.0008) [2023-12-27 02:07:13,444][105692] Updated weights for policy 0, policy_version 1473091 (0.0008) [2023-12-27 02:07:13,506][105692] Updated weights for policy 0, policy_version 1473101 (0.0009) [2023-12-27 02:07:14,121][105620] Updated weights for policy 1, policy_version 1475450 (0.0010) [2023-12-27 02:07:14,169][105620] Updated weights for policy 1, policy_version 1475460 (0.0010) [2023-12-27 02:07:14,214][105620] Updated weights for policy 1, policy_version 1475470 (0.0010) [2023-12-27 02:07:14,227][105692] Updated weights for policy 0, policy_version 1473111 (0.0007) [2023-12-27 02:07:14,300][105692] Updated weights for policy 0, policy_version 1473121 (0.0010) [2023-12-27 02:07:14,350][105692] Updated weights for policy 0, policy_version 1473131 (0.0009) [2023-12-27 02:07:14,887][105620] Updated weights for policy 1, policy_version 1475480 (0.0008) [2023-12-27 02:07:14,954][105620] Updated weights for policy 1, policy_version 1475490 (0.0010) [2023-12-27 02:07:15,012][105620] Updated weights for policy 1, policy_version 1475500 (0.0010) [2023-12-27 02:07:15,108][105692] Updated weights for policy 0, policy_version 1473141 (0.0008) [2023-12-27 02:07:15,175][105692] Updated weights for policy 0, policy_version 1473151 (0.0006) [2023-12-27 02:07:15,241][105692] Updated weights for policy 0, policy_version 1473161 (0.0005) [2023-12-27 02:07:15,681][105620] Updated weights for policy 1, policy_version 1475510 (0.0011) [2023-12-27 02:07:15,743][105620] Updated weights for policy 1, policy_version 1475520 (0.0010) [2023-12-27 02:07:15,762][105692] Updated weights for policy 0, policy_version 1473171 (0.0006) [2023-12-27 02:07:15,802][105620] Updated weights for policy 1, policy_version 1475530 (0.0009) [2023-12-27 02:07:15,819][105692] Updated weights for policy 0, policy_version 1473181 (0.0008) [2023-12-27 02:07:15,871][105692] Updated weights for policy 0, policy_version 1473191 (0.0009) [2023-12-27 02:07:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 754982912. Throughput: 0: 9689.8, 1: 9728.4. Samples: 754945336. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:07:16,062][104569] Avg episode reward: [(0, '7985.706'), (1, '8992.266')] [2023-12-27 02:07:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001473200_377192448.pth... [2023-12-27 02:07:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001475536_377790464.pth... [2023-12-27 02:07:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001472048_376897536.pth [2023-12-27 02:07:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001474416_377503744.pth [2023-12-27 02:07:16,349][105620] Updated weights for policy 1, policy_version 1475540 (0.0006) [2023-12-27 02:07:16,400][105620] Updated weights for policy 1, policy_version 1475550 (0.0005) [2023-12-27 02:07:16,455][105620] Updated weights for policy 1, policy_version 1475560 (0.0005) [2023-12-27 02:07:16,556][105692] Updated weights for policy 0, policy_version 1473201 (0.0009) [2023-12-27 02:07:16,604][105692] Updated weights for policy 0, policy_version 1473211 (0.0005) [2023-12-27 02:07:16,651][105692] Updated weights for policy 0, policy_version 1473221 (0.0005) [2023-12-27 02:07:16,701][105692] Updated weights for policy 0, policy_version 1473231 (0.0005) [2023-12-27 02:07:17,097][105620] Updated weights for policy 1, policy_version 1475570 (0.0005) [2023-12-27 02:07:17,148][105620] Updated weights for policy 1, policy_version 1475580 (0.0005) [2023-12-27 02:07:17,197][105620] Updated weights for policy 1, policy_version 1475590 (0.0009) [2023-12-27 02:07:17,249][105620] Updated weights for policy 1, policy_version 1475600 (0.0010) [2023-12-27 02:07:17,420][105692] Updated weights for policy 0, policy_version 1473241 (0.0005) [2023-12-27 02:07:17,473][105692] Updated weights for policy 0, policy_version 1473251 (0.0006) [2023-12-27 02:07:17,528][105692] Updated weights for policy 0, policy_version 1473261 (0.0006) [2023-12-27 02:07:17,979][105620] Updated weights for policy 1, policy_version 1475610 (0.0010) [2023-12-27 02:07:18,031][105620] Updated weights for policy 1, policy_version 1475620 (0.0009) [2023-12-27 02:07:18,083][105620] Updated weights for policy 1, policy_version 1475630 (0.0011) [2023-12-27 02:07:18,194][105692] Updated weights for policy 0, policy_version 1473271 (0.0007) [2023-12-27 02:07:18,251][105692] Updated weights for policy 0, policy_version 1473281 (0.0007) [2023-12-27 02:07:18,306][105692] Updated weights for policy 0, policy_version 1473291 (0.0008) [2023-12-27 02:07:18,894][105692] Updated weights for policy 0, policy_version 1473301 (0.0007) [2023-12-27 02:07:18,895][105620] Updated weights for policy 1, policy_version 1475640 (0.0010) [2023-12-27 02:07:18,956][105620] Updated weights for policy 1, policy_version 1475650 (0.0005) [2023-12-27 02:07:18,961][105692] Updated weights for policy 0, policy_version 1473311 (0.0005) [2023-12-27 02:07:19,017][105620] Updated weights for policy 1, policy_version 1475660 (0.0009) [2023-12-27 02:07:19,024][105692] Updated weights for policy 0, policy_version 1473321 (0.0006) [2023-12-27 02:07:19,606][105692] Updated weights for policy 0, policy_version 1473331 (0.0007) [2023-12-27 02:07:19,661][105692] Updated weights for policy 0, policy_version 1473341 (0.0009) [2023-12-27 02:07:19,722][105692] Updated weights for policy 0, policy_version 1473351 (0.0009) [2023-12-27 02:07:19,747][105620] Updated weights for policy 1, policy_version 1475670 (0.0010) [2023-12-27 02:07:19,804][105620] Updated weights for policy 1, policy_version 1475680 (0.0007) [2023-12-27 02:07:19,871][105620] Updated weights for policy 1, policy_version 1475690 (0.0009) [2023-12-27 02:07:20,399][105692] Updated weights for policy 0, policy_version 1473361 (0.0009) [2023-12-27 02:07:20,462][105692] Updated weights for policy 0, policy_version 1473371 (0.0009) [2023-12-27 02:07:20,525][105692] Updated weights for policy 0, policy_version 1473381 (0.0009) [2023-12-27 02:07:20,587][105692] Updated weights for policy 0, policy_version 1473391 (0.0009) [2023-12-27 02:07:20,670][105620] Updated weights for policy 1, policy_version 1475700 (0.0010) [2023-12-27 02:07:20,720][105620] Updated weights for policy 1, policy_version 1475710 (0.0009) [2023-12-27 02:07:20,784][105620] Updated weights for policy 1, policy_version 1475720 (0.0009) [2023-12-27 02:07:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 755081216. Throughput: 0: 9805.3, 1: 9774.1. Samples: 755069792. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:07:21,062][104569] Avg episode reward: [(0, '7894.976'), (1, '8899.173')] [2023-12-27 02:07:21,299][105692] Updated weights for policy 0, policy_version 1473401 (0.0009) [2023-12-27 02:07:21,361][105692] Updated weights for policy 0, policy_version 1473411 (0.0007) [2023-12-27 02:07:21,420][105692] Updated weights for policy 0, policy_version 1473421 (0.0007) [2023-12-27 02:07:21,640][105620] Updated weights for policy 1, policy_version 1475730 (0.0009) [2023-12-27 02:07:21,703][105620] Updated weights for policy 1, policy_version 1475740 (0.0009) [2023-12-27 02:07:21,773][105620] Updated weights for policy 1, policy_version 1475750 (0.0009) [2023-12-27 02:07:21,830][105620] Updated weights for policy 1, policy_version 1475760 (0.0009) [2023-12-27 02:07:22,140][105692] Updated weights for policy 0, policy_version 1473431 (0.0008) [2023-12-27 02:07:22,192][105692] Updated weights for policy 0, policy_version 1473441 (0.0009) [2023-12-27 02:07:22,261][105692] Updated weights for policy 0, policy_version 1473451 (0.0009) [2023-12-27 02:07:22,628][105620] Updated weights for policy 1, policy_version 1475770 (0.0008) [2023-12-27 02:07:22,687][105620] Updated weights for policy 1, policy_version 1475780 (0.0009) [2023-12-27 02:07:22,744][105620] Updated weights for policy 1, policy_version 1475790 (0.0009) [2023-12-27 02:07:23,019][105692] Updated weights for policy 0, policy_version 1473461 (0.0009) [2023-12-27 02:07:23,075][105692] Updated weights for policy 0, policy_version 1473471 (0.0009) [2023-12-27 02:07:23,138][105692] Updated weights for policy 0, policy_version 1473481 (0.0009) [2023-12-27 02:07:23,508][105620] Updated weights for policy 1, policy_version 1475800 (0.0009) [2023-12-27 02:07:23,559][105620] Updated weights for policy 1, policy_version 1475810 (0.0009) [2023-12-27 02:07:23,606][105620] Updated weights for policy 1, policy_version 1475820 (0.0008) [2023-12-27 02:07:23,895][105692] Updated weights for policy 0, policy_version 1473491 (0.0009) [2023-12-27 02:07:23,958][105692] Updated weights for policy 0, policy_version 1473501 (0.0009) [2023-12-27 02:07:24,012][105692] Updated weights for policy 0, policy_version 1473511 (0.0009) [2023-12-27 02:07:24,358][105620] Updated weights for policy 1, policy_version 1475830 (0.0009) [2023-12-27 02:07:24,421][105620] Updated weights for policy 1, policy_version 1475840 (0.0007) [2023-12-27 02:07:24,476][105620] Updated weights for policy 1, policy_version 1475850 (0.0009) [2023-12-27 02:07:24,749][105692] Updated weights for policy 0, policy_version 1473521 (0.0006) [2023-12-27 02:07:24,800][105692] Updated weights for policy 0, policy_version 1473531 (0.0009) [2023-12-27 02:07:24,849][105692] Updated weights for policy 0, policy_version 1473541 (0.0008) [2023-12-27 02:07:24,901][105692] Updated weights for policy 0, policy_version 1473551 (0.0009) [2023-12-27 02:07:25,169][105620] Updated weights for policy 1, policy_version 1475860 (0.0008) [2023-12-27 02:07:25,216][105620] Updated weights for policy 1, policy_version 1475870 (0.0009) [2023-12-27 02:07:25,260][105620] Updated weights for policy 1, policy_version 1475880 (0.0010) [2023-12-27 02:07:25,649][105692] Updated weights for policy 0, policy_version 1473561 (0.0007) [2023-12-27 02:07:25,710][105692] Updated weights for policy 0, policy_version 1473571 (0.0007) [2023-12-27 02:07:25,772][105692] Updated weights for policy 0, policy_version 1473581 (0.0008) [2023-12-27 02:07:25,934][105620] Updated weights for policy 1, policy_version 1475890 (0.0010) [2023-12-27 02:07:25,991][105620] Updated weights for policy 1, policy_version 1475900 (0.0010) [2023-12-27 02:07:26,046][105620] Updated weights for policy 1, policy_version 1475910 (0.0008) [2023-12-27 02:07:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 755171328. Throughput: 0: 9904.8, 1: 9635.8. Samples: 755182680. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:07:26,063][104569] Avg episode reward: [(0, '8533.209'), (1, '8898.207')] [2023-12-27 02:07:26,110][105620] Updated weights for policy 1, policy_version 1475920 (0.0010) [2023-12-27 02:07:26,415][105692] Updated weights for policy 0, policy_version 1473591 (0.0006) [2023-12-27 02:07:26,477][105692] Updated weights for policy 0, policy_version 1473601 (0.0005) [2023-12-27 02:07:26,528][105692] Updated weights for policy 0, policy_version 1473611 (0.0005) [2023-12-27 02:07:26,758][105620] Updated weights for policy 1, policy_version 1475930 (0.0007) [2023-12-27 02:07:26,830][105620] Updated weights for policy 1, policy_version 1475940 (0.0007) [2023-12-27 02:07:26,892][105620] Updated weights for policy 1, policy_version 1475950 (0.0010) [2023-12-27 02:07:27,242][105692] Updated weights for policy 0, policy_version 1473621 (0.0007) [2023-12-27 02:07:27,294][105692] Updated weights for policy 0, policy_version 1473631 (0.0009) [2023-12-27 02:07:27,355][105692] Updated weights for policy 0, policy_version 1473641 (0.0009) [2023-12-27 02:07:27,462][105620] Updated weights for policy 1, policy_version 1475960 (0.0007) [2023-12-27 02:07:27,508][105620] Updated weights for policy 1, policy_version 1475970 (0.0005) [2023-12-27 02:07:27,551][105620] Updated weights for policy 1, policy_version 1475980 (0.0005) [2023-12-27 02:07:28,091][105620] Updated weights for policy 1, policy_version 1475990 (0.0008) [2023-12-27 02:07:28,104][105692] Updated weights for policy 0, policy_version 1473651 (0.0009) [2023-12-27 02:07:28,146][105620] Updated weights for policy 1, policy_version 1476000 (0.0010) [2023-12-27 02:07:28,152][105692] Updated weights for policy 0, policy_version 1473661 (0.0010) [2023-12-27 02:07:28,203][105692] Updated weights for policy 0, policy_version 1473671 (0.0010) [2023-12-27 02:07:28,204][105620] Updated weights for policy 1, policy_version 1476010 (0.0010) [2023-12-27 02:07:28,787][105620] Updated weights for policy 1, policy_version 1476020 (0.0008) [2023-12-27 02:07:28,844][105620] Updated weights for policy 1, policy_version 1476030 (0.0005) [2023-12-27 02:07:28,893][105620] Updated weights for policy 1, policy_version 1476040 (0.0005) [2023-12-27 02:07:28,970][105692] Updated weights for policy 0, policy_version 1473681 (0.0010) [2023-12-27 02:07:29,039][105692] Updated weights for policy 0, policy_version 1473691 (0.0011) [2023-12-27 02:07:29,109][105692] Updated weights for policy 0, policy_version 1473701 (0.0010) [2023-12-27 02:07:29,171][105692] Updated weights for policy 0, policy_version 1473711 (0.0010) [2023-12-27 02:07:29,527][105620] Updated weights for policy 1, policy_version 1476050 (0.0007) [2023-12-27 02:07:29,578][105620] Updated weights for policy 1, policy_version 1476060 (0.0010) [2023-12-27 02:07:29,647][105620] Updated weights for policy 1, policy_version 1476070 (0.0008) [2023-12-27 02:07:29,711][105620] Updated weights for policy 1, policy_version 1476080 (0.0010) [2023-12-27 02:07:29,881][105692] Updated weights for policy 0, policy_version 1473721 (0.0009) [2023-12-27 02:07:29,945][105692] Updated weights for policy 0, policy_version 1473731 (0.0008) [2023-12-27 02:07:29,993][105692] Updated weights for policy 0, policy_version 1473741 (0.0009) [2023-12-27 02:07:30,431][105620] Updated weights for policy 1, policy_version 1476090 (0.0009) [2023-12-27 02:07:30,495][105620] Updated weights for policy 1, policy_version 1476100 (0.0009) [2023-12-27 02:07:30,544][105620] Updated weights for policy 1, policy_version 1476110 (0.0006) [2023-12-27 02:07:30,702][105692] Updated weights for policy 0, policy_version 1473751 (0.0006) [2023-12-27 02:07:30,755][105692] Updated weights for policy 0, policy_version 1473761 (0.0005) [2023-12-27 02:07:30,808][105692] Updated weights for policy 0, policy_version 1473771 (0.0005) [2023-12-27 02:07:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 755277824. Throughput: 0: 9902.1, 1: 9766.7. Samples: 755246516. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:07:31,063][104569] Avg episode reward: [(0, '8525.414'), (1, '9263.686')] [2023-12-27 02:07:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001473776_377339904.pth... [2023-12-27 02:07:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001476112_377937920.pth... [2023-12-27 02:07:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001472624_377044992.pth [2023-12-27 02:07:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001474960_377643008.pth [2023-12-27 02:07:31,209][105620] Updated weights for policy 1, policy_version 1476120 (0.0006) [2023-12-27 02:07:31,270][105620] Updated weights for policy 1, policy_version 1476130 (0.0007) [2023-12-27 02:07:31,330][105620] Updated weights for policy 1, policy_version 1476140 (0.0005) [2023-12-27 02:07:31,492][105692] Updated weights for policy 0, policy_version 1473781 (0.0006) [2023-12-27 02:07:31,561][105692] Updated weights for policy 0, policy_version 1473791 (0.0005) [2023-12-27 02:07:31,630][105692] Updated weights for policy 0, policy_version 1473801 (0.0007) [2023-12-27 02:07:32,042][105620] Updated weights for policy 1, policy_version 1476150 (0.0008) [2023-12-27 02:07:32,104][105620] Updated weights for policy 1, policy_version 1476160 (0.0009) [2023-12-27 02:07:32,158][105620] Updated weights for policy 1, policy_version 1476170 (0.0009) [2023-12-27 02:07:32,355][105692] Updated weights for policy 0, policy_version 1473811 (0.0010) [2023-12-27 02:07:32,422][105692] Updated weights for policy 0, policy_version 1473821 (0.0009) [2023-12-27 02:07:32,484][105692] Updated weights for policy 0, policy_version 1473831 (0.0009) [2023-12-27 02:07:32,896][105620] Updated weights for policy 1, policy_version 1476180 (0.0008) [2023-12-27 02:07:32,965][105620] Updated weights for policy 1, policy_version 1476190 (0.0005) [2023-12-27 02:07:33,032][105620] Updated weights for policy 1, policy_version 1476200 (0.0006) [2023-12-27 02:07:33,239][105692] Updated weights for policy 0, policy_version 1473841 (0.0009) [2023-12-27 02:07:33,288][105692] Updated weights for policy 0, policy_version 1473851 (0.0010) [2023-12-27 02:07:33,352][105692] Updated weights for policy 0, policy_version 1473861 (0.0010) [2023-12-27 02:07:33,419][105692] Updated weights for policy 0, policy_version 1473871 (0.0009) [2023-12-27 02:07:33,668][105620] Updated weights for policy 1, policy_version 1476210 (0.0008) [2023-12-27 02:07:33,727][105620] Updated weights for policy 1, policy_version 1476220 (0.0009) [2023-12-27 02:07:33,781][105620] Updated weights for policy 1, policy_version 1476230 (0.0009) [2023-12-27 02:07:33,834][105620] Updated weights for policy 1, policy_version 1476240 (0.0008) [2023-12-27 02:07:34,056][105692] Updated weights for policy 0, policy_version 1473881 (0.0010) [2023-12-27 02:07:34,114][105692] Updated weights for policy 0, policy_version 1473891 (0.0010) [2023-12-27 02:07:34,182][105692] Updated weights for policy 0, policy_version 1473901 (0.0011) [2023-12-27 02:07:34,625][105620] Updated weights for policy 1, policy_version 1476250 (0.0008) [2023-12-27 02:07:34,673][105620] Updated weights for policy 1, policy_version 1476260 (0.0008) [2023-12-27 02:07:34,732][105620] Updated weights for policy 1, policy_version 1476270 (0.0008) [2023-12-27 02:07:34,913][105692] Updated weights for policy 0, policy_version 1473911 (0.0011) [2023-12-27 02:07:34,972][105692] Updated weights for policy 0, policy_version 1473921 (0.0010) [2023-12-27 02:07:35,027][105692] Updated weights for policy 0, policy_version 1473931 (0.0011) [2023-12-27 02:07:35,455][105620] Updated weights for policy 1, policy_version 1476280 (0.0006) [2023-12-27 02:07:35,514][105620] Updated weights for policy 1, policy_version 1476290 (0.0008) [2023-12-27 02:07:35,564][105620] Updated weights for policy 1, policy_version 1476300 (0.0009) [2023-12-27 02:07:35,733][105692] Updated weights for policy 0, policy_version 1473941 (0.0011) [2023-12-27 02:07:35,795][105692] Updated weights for policy 0, policy_version 1473951 (0.0010) [2023-12-27 02:07:35,856][105692] Updated weights for policy 0, policy_version 1473961 (0.0010) [2023-12-27 02:07:36,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 755376128. Throughput: 0: 9887.7, 1: 9687.5. Samples: 755363200. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:07:36,062][104569] Avg episode reward: [(0, '8621.320'), (1, '9263.444')] [2023-12-27 02:07:36,277][105620] Updated weights for policy 1, policy_version 1476310 (0.0008) [2023-12-27 02:07:36,344][105620] Updated weights for policy 1, policy_version 1476320 (0.0008) [2023-12-27 02:07:36,409][105620] Updated weights for policy 1, policy_version 1476330 (0.0008) [2023-12-27 02:07:36,605][105692] Updated weights for policy 0, policy_version 1473971 (0.0010) [2023-12-27 02:07:36,668][105692] Updated weights for policy 0, policy_version 1473981 (0.0011) [2023-12-27 02:07:36,730][105692] Updated weights for policy 0, policy_version 1473991 (0.0011) [2023-12-27 02:07:37,183][105620] Updated weights for policy 1, policy_version 1476340 (0.0009) [2023-12-27 02:07:37,246][105620] Updated weights for policy 1, policy_version 1476350 (0.0009) [2023-12-27 02:07:37,298][105620] Updated weights for policy 1, policy_version 1476360 (0.0009) [2023-12-27 02:07:37,374][105692] Updated weights for policy 0, policy_version 1474001 (0.0010) [2023-12-27 02:07:37,444][105692] Updated weights for policy 0, policy_version 1474011 (0.0008) [2023-12-27 02:07:37,501][105692] Updated weights for policy 0, policy_version 1474021 (0.0007) [2023-12-27 02:07:37,558][105692] Updated weights for policy 0, policy_version 1474031 (0.0006) [2023-12-27 02:07:38,103][105620] Updated weights for policy 1, policy_version 1476371 (0.0009) [2023-12-27 02:07:38,158][105620] Updated weights for policy 1, policy_version 1476381 (0.0008) [2023-12-27 02:07:38,224][105620] Updated weights for policy 1, policy_version 1476391 (0.0005) [2023-12-27 02:07:38,269][105692] Updated weights for policy 0, policy_version 1474041 (0.0011) [2023-12-27 02:07:38,341][105692] Updated weights for policy 0, policy_version 1474051 (0.0011) [2023-12-27 02:07:38,406][105692] Updated weights for policy 0, policy_version 1474061 (0.0011) [2023-12-27 02:07:38,936][105620] Updated weights for policy 1, policy_version 1476401 (0.0005) [2023-12-27 02:07:38,992][105620] Updated weights for policy 1, policy_version 1476411 (0.0006) [2023-12-27 02:07:39,055][105620] Updated weights for policy 1, policy_version 1476421 (0.0009) [2023-12-27 02:07:39,109][105620] Updated weights for policy 1, policy_version 1476431 (0.0010) [2023-12-27 02:07:39,143][105692] Updated weights for policy 0, policy_version 1474071 (0.0010) [2023-12-27 02:07:39,202][105692] Updated weights for policy 0, policy_version 1474081 (0.0010) [2023-12-27 02:07:39,266][105692] Updated weights for policy 0, policy_version 1474091 (0.0011) [2023-12-27 02:07:39,806][105620] Updated weights for policy 1, policy_version 1476441 (0.0009) [2023-12-27 02:07:39,876][105620] Updated weights for policy 1, policy_version 1476451 (0.0008) [2023-12-27 02:07:39,936][105620] Updated weights for policy 1, policy_version 1476461 (0.0007) [2023-12-27 02:07:40,036][105692] Updated weights for policy 0, policy_version 1474101 (0.0011) [2023-12-27 02:07:40,102][105692] Updated weights for policy 0, policy_version 1474111 (0.0011) [2023-12-27 02:07:40,162][105692] Updated weights for policy 0, policy_version 1474121 (0.0011) [2023-12-27 02:07:40,610][105620] Updated weights for policy 1, policy_version 1476471 (0.0008) [2023-12-27 02:07:40,669][105620] Updated weights for policy 1, policy_version 1476481 (0.0009) [2023-12-27 02:07:40,728][105620] Updated weights for policy 1, policy_version 1476491 (0.0009) [2023-12-27 02:07:40,904][105692] Updated weights for policy 0, policy_version 1474131 (0.0010) [2023-12-27 02:07:40,961][105692] Updated weights for policy 0, policy_version 1474141 (0.0009) [2023-12-27 02:07:41,023][105692] Updated weights for policy 0, policy_version 1474151 (0.0009) [2023-12-27 02:07:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19633.1). Total num frames: 755466240. Throughput: 0: 9745.7, 1: 9776.6. Samples: 755477412. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:07:41,063][104569] Avg episode reward: [(0, '8534.813'), (1, '9173.395')] [2023-12-27 02:07:41,513][105620] Updated weights for policy 1, policy_version 1476501 (0.0009) [2023-12-27 02:07:41,573][105620] Updated weights for policy 1, policy_version 1476511 (0.0010) [2023-12-27 02:07:41,637][105620] Updated weights for policy 1, policy_version 1476521 (0.0011) [2023-12-27 02:07:41,859][105692] Updated weights for policy 0, policy_version 1474161 (0.0008) [2023-12-27 02:07:41,919][105692] Updated weights for policy 0, policy_version 1474171 (0.0010) [2023-12-27 02:07:41,975][105692] Updated weights for policy 0, policy_version 1474181 (0.0009) [2023-12-27 02:07:42,026][105692] Updated weights for policy 0, policy_version 1474191 (0.0009) [2023-12-27 02:07:42,348][105620] Updated weights for policy 1, policy_version 1476531 (0.0009) [2023-12-27 02:07:42,408][105620] Updated weights for policy 1, policy_version 1476541 (0.0007) [2023-12-27 02:07:42,467][105620] Updated weights for policy 1, policy_version 1476551 (0.0009) [2023-12-27 02:07:42,875][105692] Updated weights for policy 0, policy_version 1474201 (0.0010) [2023-12-27 02:07:42,929][105692] Updated weights for policy 0, policy_version 1474211 (0.0008) [2023-12-27 02:07:42,983][105692] Updated weights for policy 0, policy_version 1474221 (0.0010) [2023-12-27 02:07:43,141][105620] Updated weights for policy 1, policy_version 1476561 (0.0009) [2023-12-27 02:07:43,203][105620] Updated weights for policy 1, policy_version 1476571 (0.0007) [2023-12-27 02:07:43,263][105620] Updated weights for policy 1, policy_version 1476581 (0.0005) [2023-12-27 02:07:43,310][105620] Updated weights for policy 1, policy_version 1476591 (0.0006) [2023-12-27 02:07:43,852][105692] Updated weights for policy 0, policy_version 1474231 (0.0009) [2023-12-27 02:07:43,883][105620] Updated weights for policy 1, policy_version 1476601 (0.0005) [2023-12-27 02:07:43,905][105692] Updated weights for policy 0, policy_version 1474241 (0.0009) [2023-12-27 02:07:43,931][105620] Updated weights for policy 1, policy_version 1476611 (0.0005) [2023-12-27 02:07:43,952][105692] Updated weights for policy 0, policy_version 1474251 (0.0008) [2023-12-27 02:07:43,976][105620] Updated weights for policy 1, policy_version 1476621 (0.0005) [2023-12-27 02:07:44,549][105620] Updated weights for policy 1, policy_version 1476631 (0.0005) [2023-12-27 02:07:44,607][105620] Updated weights for policy 1, policy_version 1476641 (0.0008) [2023-12-27 02:07:44,623][105692] Updated weights for policy 0, policy_version 1474261 (0.0009) [2023-12-27 02:07:44,660][105620] Updated weights for policy 1, policy_version 1476651 (0.0010) [2023-12-27 02:07:44,682][105692] Updated weights for policy 0, policy_version 1474271 (0.0008) [2023-12-27 02:07:44,741][105692] Updated weights for policy 0, policy_version 1474281 (0.0008) [2023-12-27 02:07:45,334][105620] Updated weights for policy 1, policy_version 1476661 (0.0010) [2023-12-27 02:07:45,398][105620] Updated weights for policy 1, policy_version 1476671 (0.0009) [2023-12-27 02:07:45,461][105620] Updated weights for policy 1, policy_version 1476681 (0.0009) [2023-12-27 02:07:45,485][105692] Updated weights for policy 0, policy_version 1474291 (0.0007) [2023-12-27 02:07:45,538][105692] Updated weights for policy 0, policy_version 1474301 (0.0008) [2023-12-27 02:07:45,588][105692] Updated weights for policy 0, policy_version 1474311 (0.0010) [2023-12-27 02:07:46,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 755564544. Throughput: 0: 9652.5, 1: 9791.0. Samples: 755533984. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:07:46,063][104569] Avg episode reward: [(0, '8622.358'), (1, '8991.322')] [2023-12-27 02:07:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001476688_378085376.pth... [2023-12-27 02:07:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001474320_377479168.pth... [2023-12-27 02:07:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001473200_377192448.pth [2023-12-27 02:07:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001475536_377790464.pth [2023-12-27 02:07:46,171][105620] Updated weights for policy 1, policy_version 1476691 (0.0008) [2023-12-27 02:07:46,221][105620] Updated weights for policy 1, policy_version 1476701 (0.0006) [2023-12-27 02:07:46,276][105620] Updated weights for policy 1, policy_version 1476711 (0.0005) [2023-12-27 02:07:46,404][105692] Updated weights for policy 0, policy_version 1474321 (0.0010) [2023-12-27 02:07:46,462][105692] Updated weights for policy 0, policy_version 1474331 (0.0009) [2023-12-27 02:07:46,509][105692] Updated weights for policy 0, policy_version 1474341 (0.0009) [2023-12-27 02:07:46,558][105692] Updated weights for policy 0, policy_version 1474351 (0.0009) [2023-12-27 02:07:46,931][105620] Updated weights for policy 1, policy_version 1476721 (0.0006) [2023-12-27 02:07:46,994][105620] Updated weights for policy 1, policy_version 1476731 (0.0005) [2023-12-27 02:07:47,055][105620] Updated weights for policy 1, policy_version 1476741 (0.0005) [2023-12-27 02:07:47,106][105620] Updated weights for policy 1, policy_version 1476751 (0.0009) [2023-12-27 02:07:47,408][105692] Updated weights for policy 0, policy_version 1474361 (0.0008) [2023-12-27 02:07:47,471][105692] Updated weights for policy 0, policy_version 1474371 (0.0007) [2023-12-27 02:07:47,528][105692] Updated weights for policy 0, policy_version 1474381 (0.0005) [2023-12-27 02:07:47,805][105620] Updated weights for policy 1, policy_version 1476761 (0.0010) [2023-12-27 02:07:47,852][105620] Updated weights for policy 1, policy_version 1476771 (0.0008) [2023-12-27 02:07:47,906][105620] Updated weights for policy 1, policy_version 1476781 (0.0009) [2023-12-27 02:07:48,126][105692] Updated weights for policy 0, policy_version 1474391 (0.0008) [2023-12-27 02:07:48,182][105692] Updated weights for policy 0, policy_version 1474401 (0.0009) [2023-12-27 02:07:48,243][105692] Updated weights for policy 0, policy_version 1474411 (0.0008) [2023-12-27 02:07:48,688][105620] Updated weights for policy 1, policy_version 1476791 (0.0009) [2023-12-27 02:07:48,749][105620] Updated weights for policy 1, policy_version 1476801 (0.0010) [2023-12-27 02:07:48,805][105620] Updated weights for policy 1, policy_version 1476811 (0.0011) [2023-12-27 02:07:49,000][105692] Updated weights for policy 0, policy_version 1474421 (0.0008) [2023-12-27 02:07:49,049][105692] Updated weights for policy 0, policy_version 1474431 (0.0005) [2023-12-27 02:07:49,094][105692] Updated weights for policy 0, policy_version 1474441 (0.0007) [2023-12-27 02:07:49,565][105620] Updated weights for policy 1, policy_version 1476821 (0.0011) [2023-12-27 02:07:49,627][105620] Updated weights for policy 1, policy_version 1476831 (0.0010) [2023-12-27 02:07:49,682][105620] Updated weights for policy 1, policy_version 1476841 (0.0010) [2023-12-27 02:07:49,787][105692] Updated weights for policy 0, policy_version 1474451 (0.0009) [2023-12-27 02:07:49,861][105692] Updated weights for policy 0, policy_version 1474461 (0.0011) [2023-12-27 02:07:49,929][105692] Updated weights for policy 0, policy_version 1474471 (0.0011) [2023-12-27 02:07:50,452][105620] Updated weights for policy 1, policy_version 1476851 (0.0010) [2023-12-27 02:07:50,511][105620] Updated weights for policy 1, policy_version 1476861 (0.0010) [2023-12-27 02:07:50,576][105620] Updated weights for policy 1, policy_version 1476871 (0.0011) [2023-12-27 02:07:50,604][105692] Updated weights for policy 0, policy_version 1474481 (0.0010) [2023-12-27 02:07:50,661][105692] Updated weights for policy 0, policy_version 1474491 (0.0006) [2023-12-27 02:07:50,712][105692] Updated weights for policy 0, policy_version 1474501 (0.0006) [2023-12-27 02:07:50,767][105692] Updated weights for policy 0, policy_version 1474511 (0.0006) [2023-12-27 02:07:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 755662848. Throughput: 0: 9674.7, 1: 9799.1. Samples: 755651592. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:07:51,062][104569] Avg episode reward: [(0, '8533.256'), (1, '8804.895')] [2023-12-27 02:07:51,355][105620] Updated weights for policy 1, policy_version 1476881 (0.0011) [2023-12-27 02:07:51,417][105620] Updated weights for policy 1, policy_version 1476891 (0.0011) [2023-12-27 02:07:51,478][105620] Updated weights for policy 1, policy_version 1476901 (0.0011) [2023-12-27 02:07:51,509][105692] Updated weights for policy 0, policy_version 1474521 (0.0010) [2023-12-27 02:07:51,536][105620] Updated weights for policy 1, policy_version 1476911 (0.0010) [2023-12-27 02:07:51,567][105692] Updated weights for policy 0, policy_version 1474531 (0.0008) [2023-12-27 02:07:51,632][105692] Updated weights for policy 0, policy_version 1474541 (0.0008) [2023-12-27 02:07:52,287][105620] Updated weights for policy 1, policy_version 1476921 (0.0006) [2023-12-27 02:07:52,344][105620] Updated weights for policy 1, policy_version 1476931 (0.0007) [2023-12-27 02:07:52,367][105692] Updated weights for policy 0, policy_version 1474551 (0.0010) [2023-12-27 02:07:52,403][105620] Updated weights for policy 1, policy_version 1476941 (0.0009) [2023-12-27 02:07:52,428][105692] Updated weights for policy 0, policy_version 1474561 (0.0007) [2023-12-27 02:07:52,479][105692] Updated weights for policy 0, policy_version 1474571 (0.0008) [2023-12-27 02:07:53,062][105620] Updated weights for policy 1, policy_version 1476951 (0.0006) [2023-12-27 02:07:53,108][105620] Updated weights for policy 1, policy_version 1476961 (0.0005) [2023-12-27 02:07:53,154][105620] Updated weights for policy 1, policy_version 1476971 (0.0005) [2023-12-27 02:07:53,305][105692] Updated weights for policy 0, policy_version 1474581 (0.0011) [2023-12-27 02:07:53,360][105692] Updated weights for policy 0, policy_version 1474591 (0.0009) [2023-12-27 02:07:53,408][105692] Updated weights for policy 0, policy_version 1474601 (0.0008) [2023-12-27 02:07:53,782][105620] Updated weights for policy 1, policy_version 1476981 (0.0009) [2023-12-27 02:07:53,830][105620] Updated weights for policy 1, policy_version 1476991 (0.0010) [2023-12-27 02:07:53,882][105620] Updated weights for policy 1, policy_version 1477001 (0.0010) [2023-12-27 02:07:54,195][105692] Updated weights for policy 0, policy_version 1474611 (0.0008) [2023-12-27 02:07:54,250][105692] Updated weights for policy 0, policy_version 1474621 (0.0009) [2023-12-27 02:07:54,303][105692] Updated weights for policy 0, policy_version 1474631 (0.0010) [2023-12-27 02:07:54,561][105620] Updated weights for policy 1, policy_version 1477011 (0.0009) [2023-12-27 02:07:54,624][105620] Updated weights for policy 1, policy_version 1477021 (0.0009) [2023-12-27 02:07:54,686][105620] Updated weights for policy 1, policy_version 1477031 (0.0009) [2023-12-27 02:07:55,115][105692] Updated weights for policy 0, policy_version 1474641 (0.0009) [2023-12-27 02:07:55,177][105692] Updated weights for policy 0, policy_version 1474651 (0.0010) [2023-12-27 02:07:55,227][105692] Updated weights for policy 0, policy_version 1474662 (0.0009) [2023-12-27 02:07:55,275][105692] Updated weights for policy 0, policy_version 1474672 (0.0009) [2023-12-27 02:07:55,292][105620] Updated weights for policy 1, policy_version 1477041 (0.0009) [2023-12-27 02:07:55,344][105620] Updated weights for policy 1, policy_version 1477051 (0.0008) [2023-12-27 02:07:55,402][105620] Updated weights for policy 1, policy_version 1477061 (0.0005) [2023-12-27 02:07:55,451][105620] Updated weights for policy 1, policy_version 1477071 (0.0005) [2023-12-27 02:07:56,005][105692] Updated weights for policy 0, policy_version 1474682 (0.0005) [2023-12-27 02:07:56,034][105620] Updated weights for policy 1, policy_version 1477081 (0.0005) [2023-12-27 02:07:56,055][105692] Updated weights for policy 0, policy_version 1474692 (0.0006) [2023-12-27 02:07:56,062][104569] Fps is (10 sec: 18842.4, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 755752960. Throughput: 0: 9609.0, 1: 9906.3. Samples: 755768088. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:07:56,062][104569] Avg episode reward: [(0, '8350.608'), (1, '9079.195')] [2023-12-27 02:07:56,093][105620] Updated weights for policy 1, policy_version 1477091 (0.0006) [2023-12-27 02:07:56,107][105692] Updated weights for policy 0, policy_version 1474702 (0.0008) [2023-12-27 02:07:56,154][105620] Updated weights for policy 1, policy_version 1477101 (0.0006) [2023-12-27 02:07:56,777][105620] Updated weights for policy 1, policy_version 1477111 (0.0006) [2023-12-27 02:07:56,825][105620] Updated weights for policy 1, policy_version 1477121 (0.0005) [2023-12-27 02:07:56,876][105620] Updated weights for policy 1, policy_version 1477131 (0.0005) [2023-12-27 02:07:56,877][105692] Updated weights for policy 0, policy_version 1474712 (0.0007) [2023-12-27 02:07:56,923][105692] Updated weights for policy 0, policy_version 1474722 (0.0005) [2023-12-27 02:07:56,979][105692] Updated weights for policy 0, policy_version 1474732 (0.0006) [2023-12-27 02:07:57,569][105620] Updated weights for policy 1, policy_version 1477141 (0.0007) [2023-12-27 02:07:57,621][105620] Updated weights for policy 1, policy_version 1477151 (0.0008) [2023-12-27 02:07:57,669][105620] Updated weights for policy 1, policy_version 1477161 (0.0008) [2023-12-27 02:07:57,702][105692] Updated weights for policy 0, policy_version 1474742 (0.0009) [2023-12-27 02:07:57,750][105692] Updated weights for policy 0, policy_version 1474752 (0.0010) [2023-12-27 02:07:57,797][105692] Updated weights for policy 0, policy_version 1474762 (0.0010) [2023-12-27 02:07:58,473][105620] Updated weights for policy 1, policy_version 1477171 (0.0007) [2023-12-27 02:07:58,532][105620] Updated weights for policy 1, policy_version 1477181 (0.0008) [2023-12-27 02:07:58,579][105692] Updated weights for policy 0, policy_version 1474772 (0.0009) [2023-12-27 02:07:58,596][105620] Updated weights for policy 1, policy_version 1477191 (0.0009) [2023-12-27 02:07:58,636][105692] Updated weights for policy 0, policy_version 1474782 (0.0007) [2023-12-27 02:07:58,696][105692] Updated weights for policy 0, policy_version 1474792 (0.0007) [2023-12-27 02:07:59,407][105620] Updated weights for policy 1, policy_version 1477201 (0.0007) [2023-12-27 02:07:59,479][105620] Updated weights for policy 1, policy_version 1477211 (0.0008) [2023-12-27 02:07:59,545][105620] Updated weights for policy 1, policy_version 1477221 (0.0008) [2023-12-27 02:07:59,570][105692] Updated weights for policy 0, policy_version 1474802 (0.0008) [2023-12-27 02:07:59,611][105620] Updated weights for policy 1, policy_version 1477231 (0.0008) [2023-12-27 02:07:59,633][105692] Updated weights for policy 0, policy_version 1474812 (0.0007) [2023-12-27 02:07:59,696][105692] Updated weights for policy 0, policy_version 1474822 (0.0010) [2023-12-27 02:07:59,762][105692] Updated weights for policy 0, policy_version 1474832 (0.0010) [2023-12-27 02:08:00,210][105620] Updated weights for policy 1, policy_version 1477241 (0.0010) [2023-12-27 02:08:00,283][105620] Updated weights for policy 1, policy_version 1477251 (0.0011) [2023-12-27 02:08:00,342][105620] Updated weights for policy 1, policy_version 1477261 (0.0011) [2023-12-27 02:08:00,538][105692] Updated weights for policy 0, policy_version 1474842 (0.0005) [2023-12-27 02:08:00,594][105692] Updated weights for policy 0, policy_version 1474852 (0.0007) [2023-12-27 02:08:00,644][105692] Updated weights for policy 0, policy_version 1474862 (0.0008) [2023-12-27 02:08:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 755851264. Throughput: 0: 9603.9, 1: 9954.0. Samples: 755825444. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:08:01,063][104569] Avg episode reward: [(0, '7705.256'), (1, '9263.541')] [2023-12-27 02:08:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001474864_377618432.pth... [2023-12-27 02:08:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001473776_377339904.pth [2023-12-27 02:08:01,078][105620] Updated weights for policy 1, policy_version 1477271 (0.0011) [2023-12-27 02:08:01,142][105620] Updated weights for policy 1, policy_version 1477281 (0.0011) [2023-12-27 02:08:01,200][105692] Updated weights for policy 0, policy_version 1474872 (0.0008) [2023-12-27 02:08:01,205][105620] Updated weights for policy 1, policy_version 1477291 (0.0008) [2023-12-27 02:08:01,232][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001477296_378241024.pth... [2023-12-27 02:08:01,235][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001476112_377937920.pth [2023-12-27 02:08:01,263][105692] Updated weights for policy 0, policy_version 1474882 (0.0006) [2023-12-27 02:08:01,321][105692] Updated weights for policy 0, policy_version 1474892 (0.0006) [2023-12-27 02:08:01,947][105620] Updated weights for policy 1, policy_version 1477301 (0.0009) [2023-12-27 02:08:01,999][105620] Updated weights for policy 1, policy_version 1477311 (0.0010) [2023-12-27 02:08:02,051][105620] Updated weights for policy 1, policy_version 1477321 (0.0010) [2023-12-27 02:08:02,085][105692] Updated weights for policy 0, policy_version 1474902 (0.0007) [2023-12-27 02:08:02,145][105692] Updated weights for policy 0, policy_version 1474912 (0.0008) [2023-12-27 02:08:02,202][105692] Updated weights for policy 0, policy_version 1474922 (0.0008) [2023-12-27 02:08:02,819][105620] Updated weights for policy 1, policy_version 1477331 (0.0010) [2023-12-27 02:08:02,884][105620] Updated weights for policy 1, policy_version 1477341 (0.0011) [2023-12-27 02:08:02,941][105620] Updated weights for policy 1, policy_version 1477351 (0.0010) [2023-12-27 02:08:02,960][105692] Updated weights for policy 0, policy_version 1474932 (0.0007) [2023-12-27 02:08:03,006][105692] Updated weights for policy 0, policy_version 1474942 (0.0006) [2023-12-27 02:08:03,056][105692] Updated weights for policy 0, policy_version 1474952 (0.0009) [2023-12-27 02:08:03,529][105620] Updated weights for policy 1, policy_version 1477361 (0.0008) [2023-12-27 02:08:03,589][105620] Updated weights for policy 1, policy_version 1477371 (0.0008) [2023-12-27 02:08:03,644][105620] Updated weights for policy 1, policy_version 1477381 (0.0009) [2023-12-27 02:08:03,691][105620] Updated weights for policy 1, policy_version 1477391 (0.0009) [2023-12-27 02:08:03,856][105692] Updated weights for policy 0, policy_version 1474962 (0.0009) [2023-12-27 02:08:03,915][105692] Updated weights for policy 0, policy_version 1474972 (0.0008) [2023-12-27 02:08:03,975][105692] Updated weights for policy 0, policy_version 1474982 (0.0005) [2023-12-27 02:08:04,037][105692] Updated weights for policy 0, policy_version 1474992 (0.0006) [2023-12-27 02:08:04,403][105620] Updated weights for policy 1, policy_version 1477401 (0.0007) [2023-12-27 02:08:04,467][105620] Updated weights for policy 1, policy_version 1477411 (0.0009) [2023-12-27 02:08:04,528][105620] Updated weights for policy 1, policy_version 1477421 (0.0009) [2023-12-27 02:08:04,797][105692] Updated weights for policy 0, policy_version 1475002 (0.0009) [2023-12-27 02:08:04,844][105692] Updated weights for policy 0, policy_version 1475012 (0.0009) [2023-12-27 02:08:04,891][105692] Updated weights for policy 0, policy_version 1475022 (0.0008) [2023-12-27 02:08:05,273][105620] Updated weights for policy 1, policy_version 1477431 (0.0009) [2023-12-27 02:08:05,323][105620] Updated weights for policy 1, policy_version 1477441 (0.0008) [2023-12-27 02:08:05,373][105620] Updated weights for policy 1, policy_version 1477451 (0.0009) [2023-12-27 02:08:05,686][105692] Updated weights for policy 0, policy_version 1475032 (0.0009) [2023-12-27 02:08:05,738][105692] Updated weights for policy 0, policy_version 1475042 (0.0009) [2023-12-27 02:08:05,794][105692] Updated weights for policy 0, policy_version 1475052 (0.0009) [2023-12-27 02:08:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 755949568. Throughput: 0: 9432.4, 1: 9904.3. Samples: 755939940. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:08:06,063][104569] Avg episode reward: [(0, '8254.872'), (1, '9171.419')] [2023-12-27 02:08:06,171][105620] Updated weights for policy 1, policy_version 1477461 (0.0009) [2023-12-27 02:08:06,234][105620] Updated weights for policy 1, policy_version 1477471 (0.0009) [2023-12-27 02:08:06,295][105620] Updated weights for policy 1, policy_version 1477481 (0.0010) [2023-12-27 02:08:06,510][105692] Updated weights for policy 0, policy_version 1475062 (0.0009) [2023-12-27 02:08:06,563][105692] Updated weights for policy 0, policy_version 1475072 (0.0010) [2023-12-27 02:08:06,619][105692] Updated weights for policy 0, policy_version 1475082 (0.0011) [2023-12-27 02:08:07,093][105620] Updated weights for policy 1, policy_version 1477491 (0.0009) [2023-12-27 02:08:07,157][105620] Updated weights for policy 1, policy_version 1477501 (0.0010) [2023-12-27 02:08:07,214][105620] Updated weights for policy 1, policy_version 1477511 (0.0009) [2023-12-27 02:08:07,320][105692] Updated weights for policy 0, policy_version 1475092 (0.0009) [2023-12-27 02:08:07,375][105692] Updated weights for policy 0, policy_version 1475102 (0.0007) [2023-12-27 02:08:07,425][105692] Updated weights for policy 0, policy_version 1475112 (0.0010) [2023-12-27 02:08:07,999][105692] Updated weights for policy 0, policy_version 1475122 (0.0010) [2023-12-27 02:08:08,053][105692] Updated weights for policy 0, policy_version 1475132 (0.0010) [2023-12-27 02:08:08,070][105620] Updated weights for policy 1, policy_version 1477521 (0.0009) [2023-12-27 02:08:08,115][105692] Updated weights for policy 0, policy_version 1475142 (0.0010) [2023-12-27 02:08:08,126][105620] Updated weights for policy 1, policy_version 1477531 (0.0006) [2023-12-27 02:08:08,170][105692] Updated weights for policy 0, policy_version 1475152 (0.0010) [2023-12-27 02:08:08,188][105620] Updated weights for policy 1, policy_version 1477541 (0.0007) [2023-12-27 02:08:08,244][105620] Updated weights for policy 1, policy_version 1477551 (0.0005) [2023-12-27 02:08:08,824][105692] Updated weights for policy 0, policy_version 1475162 (0.0011) [2023-12-27 02:08:08,877][105692] Updated weights for policy 0, policy_version 1475172 (0.0006) [2023-12-27 02:08:08,888][105620] Updated weights for policy 1, policy_version 1477561 (0.0009) [2023-12-27 02:08:08,927][105692] Updated weights for policy 0, policy_version 1475182 (0.0005) [2023-12-27 02:08:08,945][105620] Updated weights for policy 1, policy_version 1477571 (0.0010) [2023-12-27 02:08:09,006][105620] Updated weights for policy 1, policy_version 1477581 (0.0010) [2023-12-27 02:08:09,605][105692] Updated weights for policy 0, policy_version 1475192 (0.0008) [2023-12-27 02:08:09,669][105692] Updated weights for policy 0, policy_version 1475202 (0.0007) [2023-12-27 02:08:09,741][105692] Updated weights for policy 0, policy_version 1475212 (0.0005) [2023-12-27 02:08:09,845][105620] Updated weights for policy 1, policy_version 1477591 (0.0008) [2023-12-27 02:08:09,903][105620] Updated weights for policy 1, policy_version 1477601 (0.0008) [2023-12-27 02:08:09,963][105620] Updated weights for policy 1, policy_version 1477611 (0.0008) [2023-12-27 02:08:10,455][105692] Updated weights for policy 0, policy_version 1475222 (0.0007) [2023-12-27 02:08:10,519][105692] Updated weights for policy 0, policy_version 1475232 (0.0005) [2023-12-27 02:08:10,578][105692] Updated weights for policy 0, policy_version 1475242 (0.0006) [2023-12-27 02:08:10,688][105620] Updated weights for policy 1, policy_version 1477621 (0.0009) [2023-12-27 02:08:10,750][105620] Updated weights for policy 1, policy_version 1477631 (0.0010) [2023-12-27 02:08:10,803][105620] Updated weights for policy 1, policy_version 1477641 (0.0009) [2023-12-27 02:08:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 756047872. Throughput: 0: 9509.1, 1: 9871.1. Samples: 756054788. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:08:11,062][104569] Avg episode reward: [(0, '8987.617'), (1, '9171.442')] [2023-12-27 02:08:11,182][105692] Updated weights for policy 0, policy_version 1475252 (0.0008) [2023-12-27 02:08:11,234][105692] Updated weights for policy 0, policy_version 1475262 (0.0009) [2023-12-27 02:08:11,289][105692] Updated weights for policy 0, policy_version 1475272 (0.0009) [2023-12-27 02:08:11,675][105620] Updated weights for policy 1, policy_version 1477651 (0.0010) [2023-12-27 02:08:11,745][105620] Updated weights for policy 1, policy_version 1477661 (0.0009) [2023-12-27 02:08:11,801][105620] Updated weights for policy 1, policy_version 1477671 (0.0008) [2023-12-27 02:08:12,063][105692] Updated weights for policy 0, policy_version 1475282 (0.0010) [2023-12-27 02:08:12,117][105692] Updated weights for policy 0, policy_version 1475292 (0.0011) [2023-12-27 02:08:12,192][105692] Updated weights for policy 0, policy_version 1475302 (0.0011) [2023-12-27 02:08:12,252][105692] Updated weights for policy 0, policy_version 1475312 (0.0011) [2023-12-27 02:08:12,568][105620] Updated weights for policy 1, policy_version 1477681 (0.0008) [2023-12-27 02:08:12,627][105620] Updated weights for policy 1, policy_version 1477691 (0.0007) [2023-12-27 02:08:12,683][105620] Updated weights for policy 1, policy_version 1477701 (0.0008) [2023-12-27 02:08:12,742][105620] Updated weights for policy 1, policy_version 1477711 (0.0008) [2023-12-27 02:08:12,963][105692] Updated weights for policy 0, policy_version 1475322 (0.0009) [2023-12-27 02:08:13,022][105692] Updated weights for policy 0, policy_version 1475332 (0.0011) [2023-12-27 02:08:13,067][105692] Updated weights for policy 0, policy_version 1475342 (0.0010) [2023-12-27 02:08:13,472][105620] Updated weights for policy 1, policy_version 1477721 (0.0008) [2023-12-27 02:08:13,523][105620] Updated weights for policy 1, policy_version 1477731 (0.0008) [2023-12-27 02:08:13,574][105620] Updated weights for policy 1, policy_version 1477741 (0.0007) [2023-12-27 02:08:13,798][105692] Updated weights for policy 0, policy_version 1475352 (0.0011) [2023-12-27 02:08:13,856][105692] Updated weights for policy 0, policy_version 1475362 (0.0010) [2023-12-27 02:08:13,911][105692] Updated weights for policy 0, policy_version 1475372 (0.0008) [2023-12-27 02:08:14,428][105620] Updated weights for policy 1, policy_version 1477751 (0.0007) [2023-12-27 02:08:14,450][105692] Updated weights for policy 0, policy_version 1475382 (0.0009) [2023-12-27 02:08:14,490][105620] Updated weights for policy 1, policy_version 1477761 (0.0008) [2023-12-27 02:08:14,498][105692] Updated weights for policy 0, policy_version 1475392 (0.0005) [2023-12-27 02:08:14,545][105692] Updated weights for policy 0, policy_version 1475402 (0.0005) [2023-12-27 02:08:14,553][105620] Updated weights for policy 1, policy_version 1477771 (0.0008) [2023-12-27 02:08:15,292][105692] Updated weights for policy 0, policy_version 1475412 (0.0009) [2023-12-27 02:08:15,323][105620] Updated weights for policy 1, policy_version 1477781 (0.0006) [2023-12-27 02:08:15,348][105692] Updated weights for policy 0, policy_version 1475422 (0.0011) [2023-12-27 02:08:15,371][105620] Updated weights for policy 1, policy_version 1477791 (0.0008) [2023-12-27 02:08:15,401][105692] Updated weights for policy 0, policy_version 1475432 (0.0011) [2023-12-27 02:08:15,423][105620] Updated weights for policy 1, policy_version 1477801 (0.0006) [2023-12-27 02:08:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 756137984. Throughput: 0: 9508.8, 1: 9712.4. Samples: 756111468. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:08:16,062][104569] Avg episode reward: [(0, '9078.595'), (1, '9355.934')] [2023-12-27 02:08:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001475440_377765888.pth... [2023-12-27 02:08:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001477808_378372096.pth... [2023-12-27 02:08:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001474320_377479168.pth [2023-12-27 02:08:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001476688_378085376.pth [2023-12-27 02:08:16,117][105692] Updated weights for policy 0, policy_version 1475442 (0.0010) [2023-12-27 02:08:16,150][105620] Updated weights for policy 1, policy_version 1477811 (0.0007) [2023-12-27 02:08:16,181][105692] Updated weights for policy 0, policy_version 1475452 (0.0006) [2023-12-27 02:08:16,211][105620] Updated weights for policy 1, policy_version 1477821 (0.0006) [2023-12-27 02:08:16,240][105692] Updated weights for policy 0, policy_version 1475462 (0.0010) [2023-12-27 02:08:16,256][105620] Updated weights for policy 1, policy_version 1477831 (0.0005) [2023-12-27 02:08:16,300][105692] Updated weights for policy 0, policy_version 1475472 (0.0011) [2023-12-27 02:08:16,971][105692] Updated weights for policy 0, policy_version 1475482 (0.0009) [2023-12-27 02:08:16,989][105620] Updated weights for policy 1, policy_version 1477841 (0.0005) [2023-12-27 02:08:17,030][105692] Updated weights for policy 0, policy_version 1475492 (0.0009) [2023-12-27 02:08:17,049][105620] Updated weights for policy 1, policy_version 1477851 (0.0008) [2023-12-27 02:08:17,089][105692] Updated weights for policy 0, policy_version 1475502 (0.0007) [2023-12-27 02:08:17,112][105620] Updated weights for policy 1, policy_version 1477861 (0.0006) [2023-12-27 02:08:17,181][105620] Updated weights for policy 1, policy_version 1477871 (0.0008) [2023-12-27 02:08:17,686][105692] Updated weights for policy 0, policy_version 1475512 (0.0006) [2023-12-27 02:08:17,739][105692] Updated weights for policy 0, policy_version 1475522 (0.0006) [2023-12-27 02:08:17,789][105692] Updated weights for policy 0, policy_version 1475532 (0.0005) [2023-12-27 02:08:18,002][105620] Updated weights for policy 1, policy_version 1477881 (0.0009) [2023-12-27 02:08:18,061][105620] Updated weights for policy 1, policy_version 1477891 (0.0009) [2023-12-27 02:08:18,118][105620] Updated weights for policy 1, policy_version 1477901 (0.0009) [2023-12-27 02:08:18,313][105692] Updated weights for policy 0, policy_version 1475542 (0.0005) [2023-12-27 02:08:18,378][105692] Updated weights for policy 0, policy_version 1475552 (0.0008) [2023-12-27 02:08:18,448][105692] Updated weights for policy 0, policy_version 1475562 (0.0008) [2023-12-27 02:08:18,971][105620] Updated weights for policy 1, policy_version 1477911 (0.0009) [2023-12-27 02:08:19,029][105620] Updated weights for policy 1, policy_version 1477921 (0.0008) [2023-12-27 02:08:19,095][105620] Updated weights for policy 1, policy_version 1477931 (0.0008) [2023-12-27 02:08:19,096][105692] Updated weights for policy 0, policy_version 1475572 (0.0007) [2023-12-27 02:08:19,159][105692] Updated weights for policy 0, policy_version 1475582 (0.0006) [2023-12-27 02:08:19,225][105692] Updated weights for policy 0, policy_version 1475592 (0.0006) [2023-12-27 02:08:19,883][105620] Updated weights for policy 1, policy_version 1477941 (0.0009) [2023-12-27 02:08:19,932][105692] Updated weights for policy 0, policy_version 1475602 (0.0008) [2023-12-27 02:08:19,954][105620] Updated weights for policy 1, policy_version 1477951 (0.0008) [2023-12-27 02:08:19,998][105692] Updated weights for policy 0, policy_version 1475612 (0.0010) [2023-12-27 02:08:20,016][105620] Updated weights for policy 1, policy_version 1477961 (0.0006) [2023-12-27 02:08:20,058][105692] Updated weights for policy 0, policy_version 1475622 (0.0010) [2023-12-27 02:08:20,119][105692] Updated weights for policy 0, policy_version 1475632 (0.0011) [2023-12-27 02:08:20,797][105620] Updated weights for policy 1, policy_version 1477971 (0.0010) [2023-12-27 02:08:20,838][105692] Updated weights for policy 0, policy_version 1475642 (0.0006) [2023-12-27 02:08:20,860][105620] Updated weights for policy 1, policy_version 1477981 (0.0009) [2023-12-27 02:08:20,909][105692] Updated weights for policy 0, policy_version 1475652 (0.0005) [2023-12-27 02:08:20,918][105620] Updated weights for policy 1, policy_version 1477991 (0.0009) [2023-12-27 02:08:20,963][105692] Updated weights for policy 0, policy_version 1475662 (0.0007) [2023-12-27 02:08:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 756244480. Throughput: 0: 9636.6, 1: 9596.1. Samples: 756228672. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:08:21,063][104569] Avg episode reward: [(0, '8800.180'), (1, '9264.593')] [2023-12-27 02:08:21,584][105692] Updated weights for policy 0, policy_version 1475672 (0.0008) [2023-12-27 02:08:21,663][105692] Updated weights for policy 0, policy_version 1475682 (0.0009) [2023-12-27 02:08:21,730][105692] Updated weights for policy 0, policy_version 1475692 (0.0009) [2023-12-27 02:08:21,810][105620] Updated weights for policy 1, policy_version 1478001 (0.0007) [2023-12-27 02:08:21,881][105620] Updated weights for policy 1, policy_version 1478011 (0.0008) [2023-12-27 02:08:21,947][105620] Updated weights for policy 1, policy_version 1478021 (0.0009) [2023-12-27 02:08:22,005][105620] Updated weights for policy 1, policy_version 1478031 (0.0009) [2023-12-27 02:08:22,299][105692] Updated weights for policy 0, policy_version 1475702 (0.0009) [2023-12-27 02:08:22,363][105692] Updated weights for policy 0, policy_version 1475712 (0.0011) [2023-12-27 02:08:22,423][105692] Updated weights for policy 0, policy_version 1475722 (0.0011) [2023-12-27 02:08:22,830][105620] Updated weights for policy 1, policy_version 1478041 (0.0009) [2023-12-27 02:08:22,883][105620] Updated weights for policy 1, policy_version 1478051 (0.0008) [2023-12-27 02:08:22,946][105620] Updated weights for policy 1, policy_version 1478061 (0.0006) [2023-12-27 02:08:23,183][105692] Updated weights for policy 0, policy_version 1475732 (0.0011) [2023-12-27 02:08:23,241][105692] Updated weights for policy 0, policy_version 1475742 (0.0010) [2023-12-27 02:08:23,302][105692] Updated weights for policy 0, policy_version 1475752 (0.0010) [2023-12-27 02:08:23,658][105620] Updated weights for policy 1, policy_version 1478071 (0.0007) [2023-12-27 02:08:23,722][105620] Updated weights for policy 1, policy_version 1478081 (0.0008) [2023-12-27 02:08:23,778][105620] Updated weights for policy 1, policy_version 1478091 (0.0008) [2023-12-27 02:08:23,995][105692] Updated weights for policy 0, policy_version 1475762 (0.0009) [2023-12-27 02:08:24,051][105692] Updated weights for policy 0, policy_version 1475772 (0.0005) [2023-12-27 02:08:24,118][105692] Updated weights for policy 0, policy_version 1475782 (0.0005) [2023-12-27 02:08:24,176][105692] Updated weights for policy 0, policy_version 1475792 (0.0005) [2023-12-27 02:08:24,556][105620] Updated weights for policy 1, policy_version 1478101 (0.0010) [2023-12-27 02:08:24,609][105620] Updated weights for policy 1, policy_version 1478111 (0.0010) [2023-12-27 02:08:24,657][105620] Updated weights for policy 1, policy_version 1478121 (0.0010) [2023-12-27 02:08:24,708][105692] Updated weights for policy 0, policy_version 1475802 (0.0007) [2023-12-27 02:08:24,773][105692] Updated weights for policy 0, policy_version 1475812 (0.0007) [2023-12-27 02:08:24,827][105692] Updated weights for policy 0, policy_version 1475822 (0.0007) [2023-12-27 02:08:25,420][105620] Updated weights for policy 1, policy_version 1478131 (0.0010) [2023-12-27 02:08:25,478][105620] Updated weights for policy 1, policy_version 1478141 (0.0010) [2023-12-27 02:08:25,496][105692] Updated weights for policy 0, policy_version 1475832 (0.0007) [2023-12-27 02:08:25,536][105620] Updated weights for policy 1, policy_version 1478151 (0.0010) [2023-12-27 02:08:25,552][105692] Updated weights for policy 0, policy_version 1475842 (0.0006) [2023-12-27 02:08:25,615][105692] Updated weights for policy 0, policy_version 1475852 (0.0007) [2023-12-27 02:08:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 756334592. Throughput: 0: 9746.5, 1: 9509.2. Samples: 756343920. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:08:26,063][104569] Avg episode reward: [(0, '8618.379'), (1, '9175.800')] [2023-12-27 02:08:26,224][105692] Updated weights for policy 0, policy_version 1475862 (0.0007) [2023-12-27 02:08:26,272][105692] Updated weights for policy 0, policy_version 1475872 (0.0008) [2023-12-27 02:08:26,272][105620] Updated weights for policy 1, policy_version 1478161 (0.0010) [2023-12-27 02:08:26,321][105692] Updated weights for policy 0, policy_version 1475882 (0.0006) [2023-12-27 02:08:26,323][105620] Updated weights for policy 1, policy_version 1478171 (0.0010) [2023-12-27 02:08:26,371][105620] Updated weights for policy 1, policy_version 1478181 (0.0010) [2023-12-27 02:08:26,416][105620] Updated weights for policy 1, policy_version 1478191 (0.0010) [2023-12-27 02:08:27,061][105692] Updated weights for policy 0, policy_version 1475892 (0.0005) [2023-12-27 02:08:27,117][105692] Updated weights for policy 0, policy_version 1475902 (0.0008) [2023-12-27 02:08:27,169][105692] Updated weights for policy 0, policy_version 1475912 (0.0006) [2023-12-27 02:08:27,182][105620] Updated weights for policy 1, policy_version 1478201 (0.0011) [2023-12-27 02:08:27,237][105620] Updated weights for policy 1, policy_version 1478211 (0.0010) [2023-12-27 02:08:27,297][105620] Updated weights for policy 1, policy_version 1478221 (0.0010) [2023-12-27 02:08:27,853][105692] Updated weights for policy 0, policy_version 1475922 (0.0007) [2023-12-27 02:08:27,908][105692] Updated weights for policy 0, policy_version 1475932 (0.0010) [2023-12-27 02:08:27,957][105692] Updated weights for policy 0, policy_version 1475942 (0.0010) [2023-12-27 02:08:28,005][105692] Updated weights for policy 0, policy_version 1475952 (0.0010) [2023-12-27 02:08:28,077][105620] Updated weights for policy 1, policy_version 1478231 (0.0011) [2023-12-27 02:08:28,130][105620] Updated weights for policy 1, policy_version 1478241 (0.0008) [2023-12-27 02:08:28,183][105620] Updated weights for policy 1, policy_version 1478251 (0.0010) [2023-12-27 02:08:28,637][105692] Updated weights for policy 0, policy_version 1475962 (0.0009) [2023-12-27 02:08:28,699][105692] Updated weights for policy 0, policy_version 1475972 (0.0010) [2023-12-27 02:08:28,764][105692] Updated weights for policy 0, policy_version 1475982 (0.0010) [2023-12-27 02:08:28,975][105620] Updated weights for policy 1, policy_version 1478261 (0.0009) [2023-12-27 02:08:29,022][105620] Updated weights for policy 1, policy_version 1478271 (0.0008) [2023-12-27 02:08:29,077][105620] Updated weights for policy 1, policy_version 1478281 (0.0008) [2023-12-27 02:08:29,420][105692] Updated weights for policy 0, policy_version 1475992 (0.0006) [2023-12-27 02:08:29,469][105692] Updated weights for policy 0, policy_version 1476002 (0.0005) [2023-12-27 02:08:29,517][105692] Updated weights for policy 0, policy_version 1476012 (0.0005) [2023-12-27 02:08:29,826][105620] Updated weights for policy 1, policy_version 1478291 (0.0008) [2023-12-27 02:08:29,889][105620] Updated weights for policy 1, policy_version 1478301 (0.0007) [2023-12-27 02:08:29,960][105620] Updated weights for policy 1, policy_version 1478311 (0.0007) [2023-12-27 02:08:30,158][105692] Updated weights for policy 0, policy_version 1476022 (0.0005) [2023-12-27 02:08:30,224][105692] Updated weights for policy 0, policy_version 1476032 (0.0010) [2023-12-27 02:08:30,292][105692] Updated weights for policy 0, policy_version 1476042 (0.0010) [2023-12-27 02:08:30,632][105620] Updated weights for policy 1, policy_version 1478321 (0.0006) [2023-12-27 02:08:30,676][105620] Updated weights for policy 1, policy_version 1478331 (0.0008) [2023-12-27 02:08:30,732][105620] Updated weights for policy 1, policy_version 1478341 (0.0009) [2023-12-27 02:08:30,790][105620] Updated weights for policy 1, policy_version 1478352 (0.0009) [2023-12-27 02:08:30,922][105692] Updated weights for policy 0, policy_version 1476052 (0.0008) [2023-12-27 02:08:30,970][105692] Updated weights for policy 0, policy_version 1476062 (0.0005) [2023-12-27 02:08:31,017][105692] Updated weights for policy 0, policy_version 1476072 (0.0006) [2023-12-27 02:08:31,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 756432896. Throughput: 0: 9870.2, 1: 9446.0. Samples: 756403208. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:08:31,062][104569] Avg episode reward: [(0, '8708.732'), (1, '9086.604')] [2023-12-27 02:08:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001478352_378511360.pth... [2023-12-27 02:08:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001476080_377929728.pth... [2023-12-27 02:08:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001477296_378241024.pth [2023-12-27 02:08:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001474864_377618432.pth [2023-12-27 02:08:31,503][105620] Updated weights for policy 1, policy_version 1478362 (0.0006) [2023-12-27 02:08:31,570][105620] Updated weights for policy 1, policy_version 1478372 (0.0008) [2023-12-27 02:08:31,635][105620] Updated weights for policy 1, policy_version 1478382 (0.0007) [2023-12-27 02:08:31,770][105692] Updated weights for policy 0, policy_version 1476082 (0.0011) [2023-12-27 02:08:31,822][105692] Updated weights for policy 0, policy_version 1476092 (0.0010) [2023-12-27 02:08:31,871][105692] Updated weights for policy 0, policy_version 1476102 (0.0010) [2023-12-27 02:08:31,924][105692] Updated weights for policy 0, policy_version 1476112 (0.0007) [2023-12-27 02:08:32,376][105620] Updated weights for policy 1, policy_version 1478392 (0.0008) [2023-12-27 02:08:32,432][105620] Updated weights for policy 1, policy_version 1478402 (0.0007) [2023-12-27 02:08:32,496][105620] Updated weights for policy 1, policy_version 1478412 (0.0009) [2023-12-27 02:08:32,619][105692] Updated weights for policy 0, policy_version 1476122 (0.0009) [2023-12-27 02:08:32,672][105692] Updated weights for policy 0, policy_version 1476132 (0.0009) [2023-12-27 02:08:32,730][105692] Updated weights for policy 0, policy_version 1476142 (0.0010) [2023-12-27 02:08:33,212][105620] Updated weights for policy 1, policy_version 1478422 (0.0008) [2023-12-27 02:08:33,261][105620] Updated weights for policy 1, policy_version 1478432 (0.0008) [2023-12-27 02:08:33,311][105620] Updated weights for policy 1, policy_version 1478442 (0.0009) [2023-12-27 02:08:33,498][105692] Updated weights for policy 0, policy_version 1476152 (0.0009) [2023-12-27 02:08:33,554][105692] Updated weights for policy 0, policy_version 1476162 (0.0008) [2023-12-27 02:08:33,600][105692] Updated weights for policy 0, policy_version 1476172 (0.0008) [2023-12-27 02:08:34,019][105620] Updated weights for policy 1, policy_version 1478452 (0.0009) [2023-12-27 02:08:34,070][105620] Updated weights for policy 1, policy_version 1478462 (0.0009) [2023-12-27 02:08:34,117][105620] Updated weights for policy 1, policy_version 1478472 (0.0008) [2023-12-27 02:08:34,392][105692] Updated weights for policy 0, policy_version 1476182 (0.0009) [2023-12-27 02:08:34,444][105692] Updated weights for policy 0, policy_version 1476192 (0.0008) [2023-12-27 02:08:34,509][105692] Updated weights for policy 0, policy_version 1476202 (0.0009) [2023-12-27 02:08:34,893][105620] Updated weights for policy 1, policy_version 1478482 (0.0009) [2023-12-27 02:08:34,951][105620] Updated weights for policy 1, policy_version 1478492 (0.0009) [2023-12-27 02:08:34,999][105620] Updated weights for policy 1, policy_version 1478502 (0.0008) [2023-12-27 02:08:35,053][105620] Updated weights for policy 1, policy_version 1478512 (0.0006) [2023-12-27 02:08:35,317][105692] Updated weights for policy 0, policy_version 1476212 (0.0010) [2023-12-27 02:08:35,369][105692] Updated weights for policy 0, policy_version 1476222 (0.0010) [2023-12-27 02:08:35,417][105692] Updated weights for policy 0, policy_version 1476232 (0.0006) [2023-12-27 02:08:35,769][105620] Updated weights for policy 1, policy_version 1478522 (0.0010) [2023-12-27 02:08:35,831][105620] Updated weights for policy 1, policy_version 1478532 (0.0011) [2023-12-27 02:08:35,895][105620] Updated weights for policy 1, policy_version 1478542 (0.0011) [2023-12-27 02:08:36,005][105692] Updated weights for policy 0, policy_version 1476242 (0.0005) [2023-12-27 02:08:36,054][105692] Updated weights for policy 0, policy_version 1476252 (0.0005) [2023-12-27 02:08:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 756531200. Throughput: 0: 9895.5, 1: 9403.4. Samples: 756520040. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:08:36,063][104569] Avg episode reward: [(0, '8343.586'), (1, '8993.031')] [2023-12-27 02:08:36,114][105692] Updated weights for policy 0, policy_version 1476262 (0.0007) [2023-12-27 02:08:36,179][105692] Updated weights for policy 0, policy_version 1476272 (0.0011) [2023-12-27 02:08:36,661][105620] Updated weights for policy 1, policy_version 1478552 (0.0011) [2023-12-27 02:08:36,725][105620] Updated weights for policy 1, policy_version 1478562 (0.0011) [2023-12-27 02:08:36,784][105620] Updated weights for policy 1, policy_version 1478572 (0.0011) [2023-12-27 02:08:36,924][105692] Updated weights for policy 0, policy_version 1476282 (0.0008) [2023-12-27 02:08:36,984][105692] Updated weights for policy 0, policy_version 1476292 (0.0008) [2023-12-27 02:08:37,045][105692] Updated weights for policy 0, policy_version 1476302 (0.0008) [2023-12-27 02:08:37,534][105620] Updated weights for policy 1, policy_version 1478582 (0.0011) [2023-12-27 02:08:37,593][105620] Updated weights for policy 1, policy_version 1478592 (0.0010) [2023-12-27 02:08:37,643][105692] Updated weights for policy 0, policy_version 1476312 (0.0006) [2023-12-27 02:08:37,656][105620] Updated weights for policy 1, policy_version 1478602 (0.0010) [2023-12-27 02:08:37,700][105692] Updated weights for policy 0, policy_version 1476322 (0.0005) [2023-12-27 02:08:37,762][105692] Updated weights for policy 0, policy_version 1476332 (0.0005) [2023-12-27 02:08:38,396][105620] Updated weights for policy 1, policy_version 1478612 (0.0010) [2023-12-27 02:08:38,431][105692] Updated weights for policy 0, policy_version 1476342 (0.0006) [2023-12-27 02:08:38,456][105620] Updated weights for policy 1, policy_version 1478622 (0.0011) [2023-12-27 02:08:38,480][105692] Updated weights for policy 0, policy_version 1476352 (0.0009) [2023-12-27 02:08:38,515][105620] Updated weights for policy 1, policy_version 1478632 (0.0011) [2023-12-27 02:08:38,539][105692] Updated weights for policy 0, policy_version 1476362 (0.0007) [2023-12-27 02:08:39,270][105620] Updated weights for policy 1, policy_version 1478642 (0.0011) [2023-12-27 02:08:39,313][105692] Updated weights for policy 0, policy_version 1476372 (0.0007) [2023-12-27 02:08:39,323][105620] Updated weights for policy 1, policy_version 1478652 (0.0011) [2023-12-27 02:08:39,382][105692] Updated weights for policy 0, policy_version 1476382 (0.0007) [2023-12-27 02:08:39,396][105620] Updated weights for policy 1, policy_version 1478662 (0.0009) [2023-12-27 02:08:39,449][105692] Updated weights for policy 0, policy_version 1476392 (0.0007) [2023-12-27 02:08:39,459][105620] Updated weights for policy 1, policy_version 1478672 (0.0009) [2023-12-27 02:08:40,139][105692] Updated weights for policy 0, policy_version 1476402 (0.0008) [2023-12-27 02:08:40,181][105620] Updated weights for policy 1, policy_version 1478682 (0.0007) [2023-12-27 02:08:40,193][105692] Updated weights for policy 0, policy_version 1476412 (0.0007) [2023-12-27 02:08:40,241][105620] Updated weights for policy 1, policy_version 1478692 (0.0011) [2023-12-27 02:08:40,258][105692] Updated weights for policy 0, policy_version 1476422 (0.0009) [2023-12-27 02:08:40,304][105620] Updated weights for policy 1, policy_version 1478702 (0.0011) [2023-12-27 02:08:40,318][105692] Updated weights for policy 0, policy_version 1476432 (0.0011) [2023-12-27 02:08:41,014][105620] Updated weights for policy 1, policy_version 1478712 (0.0007) [2023-12-27 02:08:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 756621312. Throughput: 0: 10002.7, 1: 9293.8. Samples: 756636432. Policy #0 lag: (min: 17.0, avg: 31.6, max: 32.0) [2023-12-27 02:08:41,063][104569] Avg episode reward: [(0, '8163.729'), (1, '8901.163')] [2023-12-27 02:08:41,077][105692] Updated weights for policy 0, policy_version 1476442 (0.0010) [2023-12-27 02:08:41,085][105620] Updated weights for policy 1, policy_version 1478722 (0.0013) [2023-12-27 02:08:41,135][105692] Updated weights for policy 0, policy_version 1476452 (0.0012) [2023-12-27 02:08:41,140][105620] Updated weights for policy 1, policy_version 1478732 (0.0009) [2023-12-27 02:08:41,201][105692] Updated weights for policy 0, policy_version 1476462 (0.0011) [2023-12-27 02:08:41,931][105620] Updated weights for policy 1, policy_version 1478742 (0.0008) [2023-12-27 02:08:41,965][105692] Updated weights for policy 0, policy_version 1476472 (0.0010) [2023-12-27 02:08:41,992][105620] Updated weights for policy 1, policy_version 1478752 (0.0005) [2023-12-27 02:08:42,026][105692] Updated weights for policy 0, policy_version 1476482 (0.0011) [2023-12-27 02:08:42,055][105620] Updated weights for policy 1, policy_version 1478762 (0.0006) [2023-12-27 02:08:42,090][105692] Updated weights for policy 0, policy_version 1476492 (0.0011) [2023-12-27 02:08:42,672][105620] Updated weights for policy 1, policy_version 1478772 (0.0007) [2023-12-27 02:08:42,733][105620] Updated weights for policy 1, policy_version 1478782 (0.0009) [2023-12-27 02:08:42,800][105620] Updated weights for policy 1, policy_version 1478792 (0.0009) [2023-12-27 02:08:42,837][105692] Updated weights for policy 0, policy_version 1476502 (0.0008) [2023-12-27 02:08:42,896][105692] Updated weights for policy 0, policy_version 1476512 (0.0005) [2023-12-27 02:08:42,953][105692] Updated weights for policy 0, policy_version 1476522 (0.0005) [2023-12-27 02:08:43,397][105620] Updated weights for policy 1, policy_version 1478802 (0.0009) [2023-12-27 02:08:43,445][105620] Updated weights for policy 1, policy_version 1478812 (0.0008) [2023-12-27 02:08:43,498][105620] Updated weights for policy 1, policy_version 1478822 (0.0007) [2023-12-27 02:08:43,566][105620] Updated weights for policy 1, policy_version 1478832 (0.0005) [2023-12-27 02:08:43,586][105692] Updated weights for policy 0, policy_version 1476532 (0.0007) [2023-12-27 02:08:43,638][105692] Updated weights for policy 0, policy_version 1476542 (0.0010) [2023-12-27 02:08:43,699][105692] Updated weights for policy 0, policy_version 1476552 (0.0010) [2023-12-27 02:08:44,218][105620] Updated weights for policy 1, policy_version 1478842 (0.0006) [2023-12-27 02:08:44,270][105620] Updated weights for policy 1, policy_version 1478852 (0.0005) [2023-12-27 02:08:44,319][105620] Updated weights for policy 1, policy_version 1478862 (0.0005) [2023-12-27 02:08:44,325][105692] Updated weights for policy 0, policy_version 1476562 (0.0010) [2023-12-27 02:08:44,394][105692] Updated weights for policy 0, policy_version 1476572 (0.0011) [2023-12-27 02:08:44,462][105692] Updated weights for policy 0, policy_version 1476582 (0.0010) [2023-12-27 02:08:44,517][105692] Updated weights for policy 0, policy_version 1476592 (0.0010) [2023-12-27 02:08:44,880][105620] Updated weights for policy 1, policy_version 1478872 (0.0010) [2023-12-27 02:08:44,944][105620] Updated weights for policy 1, policy_version 1478882 (0.0011) [2023-12-27 02:08:45,005][105620] Updated weights for policy 1, policy_version 1478892 (0.0011) [2023-12-27 02:08:45,245][105692] Updated weights for policy 0, policy_version 1476602 (0.0010) [2023-12-27 02:08:45,304][105692] Updated weights for policy 0, policy_version 1476612 (0.0010) [2023-12-27 02:08:45,362][105692] Updated weights for policy 0, policy_version 1476622 (0.0011) [2023-12-27 02:08:45,671][105620] Updated weights for policy 1, policy_version 1478902 (0.0007) [2023-12-27 02:08:45,734][105620] Updated weights for policy 1, policy_version 1478912 (0.0007) [2023-12-27 02:08:45,793][105620] Updated weights for policy 1, policy_version 1478922 (0.0005) [2023-12-27 02:08:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 756727808. Throughput: 0: 10012.6, 1: 9337.1. Samples: 756696180. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:08:46,062][104569] Avg episode reward: [(0, '8712.907'), (1, '8905.251')] [2023-12-27 02:08:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001478928_378658816.pth... [2023-12-27 02:08:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001477808_378372096.pth [2023-12-27 02:08:46,119][105692] Updated weights for policy 0, policy_version 1476632 (0.0010) [2023-12-27 02:08:46,169][105692] Updated weights for policy 0, policy_version 1476642 (0.0005) [2023-12-27 02:08:46,222][105692] Updated weights for policy 0, policy_version 1476652 (0.0007) [2023-12-27 02:08:46,240][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001476656_378077184.pth... [2023-12-27 02:08:46,244][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001475440_377765888.pth [2023-12-27 02:08:46,358][105620] Updated weights for policy 1, policy_version 1478932 (0.0007) [2023-12-27 02:08:46,414][105620] Updated weights for policy 1, policy_version 1478942 (0.0010) [2023-12-27 02:08:46,474][105620] Updated weights for policy 1, policy_version 1478952 (0.0008) [2023-12-27 02:08:46,921][105692] Updated weights for policy 0, policy_version 1476662 (0.0007) [2023-12-27 02:08:46,978][105692] Updated weights for policy 0, policy_version 1476672 (0.0006) [2023-12-27 02:08:47,035][105692] Updated weights for policy 0, policy_version 1476682 (0.0010) [2023-12-27 02:08:47,186][105620] Updated weights for policy 1, policy_version 1478962 (0.0007) [2023-12-27 02:08:47,241][105620] Updated weights for policy 1, policy_version 1478972 (0.0006) [2023-12-27 02:08:47,296][105620] Updated weights for policy 1, policy_version 1478982 (0.0006) [2023-12-27 02:08:47,351][105620] Updated weights for policy 1, policy_version 1478992 (0.0006) [2023-12-27 02:08:47,643][105692] Updated weights for policy 0, policy_version 1476692 (0.0009) [2023-12-27 02:08:47,705][105692] Updated weights for policy 0, policy_version 1476702 (0.0010) [2023-12-27 02:08:47,771][105692] Updated weights for policy 0, policy_version 1476712 (0.0010) [2023-12-27 02:08:47,959][105620] Updated weights for policy 1, policy_version 1479002 (0.0008) [2023-12-27 02:08:48,011][105620] Updated weights for policy 1, policy_version 1479012 (0.0008) [2023-12-27 02:08:48,063][105620] Updated weights for policy 1, policy_version 1479022 (0.0008) [2023-12-27 02:08:48,501][105692] Updated weights for policy 0, policy_version 1476722 (0.0010) [2023-12-27 02:08:48,559][105692] Updated weights for policy 0, policy_version 1476732 (0.0009) [2023-12-27 02:08:48,615][105692] Updated weights for policy 0, policy_version 1476742 (0.0009) [2023-12-27 02:08:48,673][105692] Updated weights for policy 0, policy_version 1476752 (0.0008) [2023-12-27 02:08:48,859][105620] Updated weights for policy 1, policy_version 1479032 (0.0009) [2023-12-27 02:08:48,913][105620] Updated weights for policy 1, policy_version 1479042 (0.0009) [2023-12-27 02:08:48,977][105620] Updated weights for policy 1, policy_version 1479052 (0.0009) [2023-12-27 02:08:49,447][105692] Updated weights for policy 0, policy_version 1476762 (0.0008) [2023-12-27 02:08:49,518][105692] Updated weights for policy 0, policy_version 1476772 (0.0007) [2023-12-27 02:08:49,584][105692] Updated weights for policy 0, policy_version 1476782 (0.0005) [2023-12-27 02:08:49,759][105620] Updated weights for policy 1, policy_version 1479062 (0.0007) [2023-12-27 02:08:49,817][105620] Updated weights for policy 1, policy_version 1479072 (0.0006) [2023-12-27 02:08:49,885][105620] Updated weights for policy 1, policy_version 1479082 (0.0008) [2023-12-27 02:08:50,223][105692] Updated weights for policy 0, policy_version 1476792 (0.0009) [2023-12-27 02:08:50,284][105692] Updated weights for policy 0, policy_version 1476802 (0.0006) [2023-12-27 02:08:50,347][105692] Updated weights for policy 0, policy_version 1476812 (0.0006) [2023-12-27 02:08:50,666][105620] Updated weights for policy 1, policy_version 1479092 (0.0007) [2023-12-27 02:08:50,721][105620] Updated weights for policy 1, policy_version 1479102 (0.0008) [2023-12-27 02:08:50,777][105620] Updated weights for policy 1, policy_version 1479112 (0.0008) [2023-12-27 02:08:50,949][105692] Updated weights for policy 0, policy_version 1476822 (0.0009) [2023-12-27 02:08:51,008][105692] Updated weights for policy 0, policy_version 1476832 (0.0011) [2023-12-27 02:08:51,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 756826112. Throughput: 0: 10078.8, 1: 9391.9. Samples: 756816124. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:08:51,062][104569] Avg episode reward: [(0, '8713.863'), (1, '8997.666')] [2023-12-27 02:08:51,072][105692] Updated weights for policy 0, policy_version 1476842 (0.0011) [2023-12-27 02:08:51,430][105620] Updated weights for policy 1, policy_version 1479122 (0.0005) [2023-12-27 02:08:51,488][105620] Updated weights for policy 1, policy_version 1479132 (0.0005) [2023-12-27 02:08:51,540][105620] Updated weights for policy 1, policy_version 1479142 (0.0006) [2023-12-27 02:08:51,602][105620] Updated weights for policy 1, policy_version 1479152 (0.0009) [2023-12-27 02:08:51,882][105692] Updated weights for policy 0, policy_version 1476852 (0.0009) [2023-12-27 02:08:51,934][105692] Updated weights for policy 0, policy_version 1476862 (0.0005) [2023-12-27 02:08:52,003][105692] Updated weights for policy 0, policy_version 1476872 (0.0006) [2023-12-27 02:08:52,391][105620] Updated weights for policy 1, policy_version 1479162 (0.0008) [2023-12-27 02:08:52,452][105620] Updated weights for policy 1, policy_version 1479172 (0.0008) [2023-12-27 02:08:52,519][105620] Updated weights for policy 1, policy_version 1479182 (0.0007) [2023-12-27 02:08:52,601][105692] Updated weights for policy 0, policy_version 1476882 (0.0007) [2023-12-27 02:08:52,655][105692] Updated weights for policy 0, policy_version 1476892 (0.0009) [2023-12-27 02:08:52,722][105692] Updated weights for policy 0, policy_version 1476902 (0.0010) [2023-12-27 02:08:52,788][105692] Updated weights for policy 0, policy_version 1476912 (0.0010) [2023-12-27 02:08:53,156][105620] Updated weights for policy 1, policy_version 1479192 (0.0009) [2023-12-27 02:08:53,207][105620] Updated weights for policy 1, policy_version 1479202 (0.0008) [2023-12-27 02:08:53,258][105620] Updated weights for policy 1, policy_version 1479213 (0.0010) [2023-12-27 02:08:53,493][105692] Updated weights for policy 0, policy_version 1476922 (0.0009) [2023-12-27 02:08:53,549][105692] Updated weights for policy 0, policy_version 1476932 (0.0009) [2023-12-27 02:08:53,600][105692] Updated weights for policy 0, policy_version 1476943 (0.0010) [2023-12-27 02:08:53,859][105620] Updated weights for policy 1, policy_version 1479224 (0.0007) [2023-12-27 02:08:53,905][105620] Updated weights for policy 1, policy_version 1479234 (0.0005) [2023-12-27 02:08:53,956][105620] Updated weights for policy 1, policy_version 1479244 (0.0005) [2023-12-27 02:08:54,427][105692] Updated weights for policy 0, policy_version 1476953 (0.0006) [2023-12-27 02:08:54,485][105692] Updated weights for policy 0, policy_version 1476963 (0.0005) [2023-12-27 02:08:54,496][105620] Updated weights for policy 1, policy_version 1479254 (0.0006) [2023-12-27 02:08:54,533][105692] Updated weights for policy 0, policy_version 1476973 (0.0005) [2023-12-27 02:08:54,549][105620] Updated weights for policy 1, policy_version 1479264 (0.0008) [2023-12-27 02:08:54,604][105620] Updated weights for policy 1, policy_version 1479274 (0.0009) [2023-12-27 02:08:55,169][105692] Updated weights for policy 0, policy_version 1476983 (0.0008) [2023-12-27 02:08:55,220][105692] Updated weights for policy 0, policy_version 1476993 (0.0009) [2023-12-27 02:08:55,269][105620] Updated weights for policy 1, policy_version 1479284 (0.0010) [2023-12-27 02:08:55,275][105692] Updated weights for policy 0, policy_version 1477003 (0.0007) [2023-12-27 02:08:55,319][105620] Updated weights for policy 1, policy_version 1479294 (0.0006) [2023-12-27 02:08:55,365][105620] Updated weights for policy 1, policy_version 1479304 (0.0009) [2023-12-27 02:08:56,058][105692] Updated weights for policy 0, policy_version 1477013 (0.0008) [2023-12-27 02:08:56,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 756924416. Throughput: 0: 10055.5, 1: 9559.8. Samples: 756937480. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:08:56,063][104569] Avg episode reward: [(0, '8347.774'), (1, '8995.635')] [2023-12-27 02:08:56,089][105620] Updated weights for policy 1, policy_version 1479314 (0.0007) [2023-12-27 02:08:56,116][105692] Updated weights for policy 0, policy_version 1477023 (0.0009) [2023-12-27 02:08:56,138][105620] Updated weights for policy 1, policy_version 1479324 (0.0008) [2023-12-27 02:08:56,176][105692] Updated weights for policy 0, policy_version 1477033 (0.0009) [2023-12-27 02:08:56,187][105620] Updated weights for policy 1, policy_version 1479334 (0.0006) [2023-12-27 02:08:56,238][105620] Updated weights for policy 1, policy_version 1479344 (0.0007) [2023-12-27 02:08:56,915][105620] Updated weights for policy 1, policy_version 1479354 (0.0009) [2023-12-27 02:08:56,967][105620] Updated weights for policy 1, policy_version 1479364 (0.0008) [2023-12-27 02:08:56,974][105692] Updated weights for policy 0, policy_version 1477043 (0.0007) [2023-12-27 02:08:57,024][105620] Updated weights for policy 1, policy_version 1479374 (0.0008) [2023-12-27 02:08:57,030][105692] Updated weights for policy 0, policy_version 1477053 (0.0007) [2023-12-27 02:08:57,098][105692] Updated weights for policy 0, policy_version 1477063 (0.0008) [2023-12-27 02:08:57,664][105620] Updated weights for policy 1, policy_version 1479384 (0.0006) [2023-12-27 02:08:57,712][105620] Updated weights for policy 1, policy_version 1479394 (0.0006) [2023-12-27 02:08:57,757][105620] Updated weights for policy 1, policy_version 1479404 (0.0010) [2023-12-27 02:08:57,859][105692] Updated weights for policy 0, policy_version 1477073 (0.0008) [2023-12-27 02:08:57,907][105692] Updated weights for policy 0, policy_version 1477083 (0.0008) [2023-12-27 02:08:57,967][105692] Updated weights for policy 0, policy_version 1477093 (0.0008) [2023-12-27 02:08:58,031][105692] Updated weights for policy 0, policy_version 1477103 (0.0009) [2023-12-27 02:08:58,447][105620] Updated weights for policy 1, policy_version 1479414 (0.0010) [2023-12-27 02:08:58,507][105620] Updated weights for policy 1, policy_version 1479424 (0.0011) [2023-12-27 02:08:58,571][105620] Updated weights for policy 1, policy_version 1479434 (0.0010) [2023-12-27 02:08:58,850][105692] Updated weights for policy 0, policy_version 1477113 (0.0009) [2023-12-27 02:08:58,905][105692] Updated weights for policy 0, policy_version 1477123 (0.0008) [2023-12-27 02:08:58,959][105692] Updated weights for policy 0, policy_version 1477133 (0.0008) [2023-12-27 02:08:59,244][105620] Updated weights for policy 1, policy_version 1479444 (0.0011) [2023-12-27 02:08:59,303][105620] Updated weights for policy 1, policy_version 1479454 (0.0010) [2023-12-27 02:08:59,362][105620] Updated weights for policy 1, policy_version 1479464 (0.0011) [2023-12-27 02:08:59,772][105692] Updated weights for policy 0, policy_version 1477143 (0.0006) [2023-12-27 02:08:59,840][105692] Updated weights for policy 0, policy_version 1477153 (0.0007) [2023-12-27 02:08:59,906][105692] Updated weights for policy 0, policy_version 1477163 (0.0006) [2023-12-27 02:09:00,039][105620] Updated weights for policy 1, policy_version 1479474 (0.0010) [2023-12-27 02:09:00,104][105620] Updated weights for policy 1, policy_version 1479484 (0.0011) [2023-12-27 02:09:00,173][105620] Updated weights for policy 1, policy_version 1479494 (0.0011) [2023-12-27 02:09:00,238][105620] Updated weights for policy 1, policy_version 1479504 (0.0011) [2023-12-27 02:09:00,510][105692] Updated weights for policy 0, policy_version 1477173 (0.0008) [2023-12-27 02:09:00,570][105692] Updated weights for policy 0, policy_version 1477183 (0.0007) [2023-12-27 02:09:00,622][105692] Updated weights for policy 0, policy_version 1477193 (0.0008) [2023-12-27 02:09:00,925][105620] Updated weights for policy 1, policy_version 1479514 (0.0005) [2023-12-27 02:09:00,977][105620] Updated weights for policy 1, policy_version 1479524 (0.0005) [2023-12-27 02:09:01,040][105620] Updated weights for policy 1, policy_version 1479535 (0.0007) [2023-12-27 02:09:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 757030912. Throughput: 0: 9994.6, 1: 9642.0. Samples: 756995116. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:09:01,064][104569] Avg episode reward: [(0, '8526.348'), (1, '8902.776')] [2023-12-27 02:09:01,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001477200_378216448.pth... [2023-12-27 02:09:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001479536_378814464.pth... [2023-12-27 02:09:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001478352_378511360.pth [2023-12-27 02:09:01,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001476080_377929728.pth [2023-12-27 02:09:01,428][105692] Updated weights for policy 0, policy_version 1477203 (0.0007) [2023-12-27 02:09:01,479][105692] Updated weights for policy 0, policy_version 1477213 (0.0006) [2023-12-27 02:09:01,527][105692] Updated weights for policy 0, policy_version 1477223 (0.0008) [2023-12-27 02:09:01,758][105620] Updated weights for policy 1, policy_version 1479545 (0.0011) [2023-12-27 02:09:01,816][105620] Updated weights for policy 1, policy_version 1479555 (0.0010) [2023-12-27 02:09:01,881][105620] Updated weights for policy 1, policy_version 1479565 (0.0006) [2023-12-27 02:09:02,191][105692] Updated weights for policy 0, policy_version 1477233 (0.0008) [2023-12-27 02:09:02,250][105692] Updated weights for policy 0, policy_version 1477243 (0.0006) [2023-12-27 02:09:02,314][105692] Updated weights for policy 0, policy_version 1477253 (0.0009) [2023-12-27 02:09:02,372][105692] Updated weights for policy 0, policy_version 1477263 (0.0009) [2023-12-27 02:09:02,625][105620] Updated weights for policy 1, policy_version 1479575 (0.0009) [2023-12-27 02:09:02,683][105620] Updated weights for policy 1, policy_version 1479585 (0.0010) [2023-12-27 02:09:02,739][105620] Updated weights for policy 1, policy_version 1479595 (0.0009) [2023-12-27 02:09:03,096][105692] Updated weights for policy 0, policy_version 1477273 (0.0009) [2023-12-27 02:09:03,167][105692] Updated weights for policy 0, policy_version 1477283 (0.0009) [2023-12-27 02:09:03,233][105692] Updated weights for policy 0, policy_version 1477293 (0.0009) [2023-12-27 02:09:03,401][105620] Updated weights for policy 1, policy_version 1479605 (0.0009) [2023-12-27 02:09:03,456][105620] Updated weights for policy 1, policy_version 1479615 (0.0006) [2023-12-27 02:09:03,514][105620] Updated weights for policy 1, policy_version 1479625 (0.0009) [2023-12-27 02:09:03,906][105692] Updated weights for policy 0, policy_version 1477303 (0.0008) [2023-12-27 02:09:03,969][105692] Updated weights for policy 0, policy_version 1477313 (0.0006) [2023-12-27 02:09:04,038][105692] Updated weights for policy 0, policy_version 1477323 (0.0005) [2023-12-27 02:09:04,199][105620] Updated weights for policy 1, policy_version 1479635 (0.0010) [2023-12-27 02:09:04,251][105620] Updated weights for policy 1, policy_version 1479645 (0.0011) [2023-12-27 02:09:04,305][105620] Updated weights for policy 1, policy_version 1479655 (0.0010) [2023-12-27 02:09:04,714][105692] Updated weights for policy 0, policy_version 1477333 (0.0008) [2023-12-27 02:09:04,766][105692] Updated weights for policy 0, policy_version 1477343 (0.0008) [2023-12-27 02:09:04,827][105692] Updated weights for policy 0, policy_version 1477353 (0.0005) [2023-12-27 02:09:05,088][105620] Updated weights for policy 1, policy_version 1479665 (0.0011) [2023-12-27 02:09:05,144][105620] Updated weights for policy 1, policy_version 1479675 (0.0011) [2023-12-27 02:09:05,195][105620] Updated weights for policy 1, policy_version 1479685 (0.0009) [2023-12-27 02:09:05,240][105620] Updated weights for policy 1, policy_version 1479695 (0.0010) [2023-12-27 02:09:05,461][105692] Updated weights for policy 0, policy_version 1477363 (0.0007) [2023-12-27 02:09:05,510][105692] Updated weights for policy 0, policy_version 1477373 (0.0008) [2023-12-27 02:09:05,574][105692] Updated weights for policy 0, policy_version 1477383 (0.0009) [2023-12-27 02:09:05,944][105620] Updated weights for policy 1, policy_version 1479705 (0.0006) [2023-12-27 02:09:06,001][105620] Updated weights for policy 1, policy_version 1479715 (0.0005) [2023-12-27 02:09:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 757121024. Throughput: 0: 9866.7, 1: 9786.7. Samples: 757113080. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:09:06,063][104569] Avg episode reward: [(0, '8984.210'), (1, '8992.357')] [2023-12-27 02:09:06,069][105620] Updated weights for policy 1, policy_version 1479725 (0.0011) [2023-12-27 02:09:06,356][105692] Updated weights for policy 0, policy_version 1477393 (0.0009) [2023-12-27 02:09:06,419][105692] Updated weights for policy 0, policy_version 1477403 (0.0009) [2023-12-27 02:09:06,487][105692] Updated weights for policy 0, policy_version 1477413 (0.0007) [2023-12-27 02:09:06,536][105692] Updated weights for policy 0, policy_version 1477423 (0.0009) [2023-12-27 02:09:06,787][105620] Updated weights for policy 1, policy_version 1479735 (0.0008) [2023-12-27 02:09:06,847][105620] Updated weights for policy 1, policy_version 1479745 (0.0011) [2023-12-27 02:09:06,910][105620] Updated weights for policy 1, policy_version 1479755 (0.0009) [2023-12-27 02:09:07,242][105692] Updated weights for policy 0, policy_version 1477433 (0.0007) [2023-12-27 02:09:07,301][105692] Updated weights for policy 0, policy_version 1477443 (0.0010) [2023-12-27 02:09:07,367][105692] Updated weights for policy 0, policy_version 1477453 (0.0006) [2023-12-27 02:09:07,560][105620] Updated weights for policy 1, policy_version 1479765 (0.0008) [2023-12-27 02:09:07,627][105620] Updated weights for policy 1, policy_version 1479775 (0.0011) [2023-12-27 02:09:07,687][105620] Updated weights for policy 1, policy_version 1479785 (0.0008) [2023-12-27 02:09:07,970][105692] Updated weights for policy 0, policy_version 1477463 (0.0005) [2023-12-27 02:09:08,023][105692] Updated weights for policy 0, policy_version 1477473 (0.0006) [2023-12-27 02:09:08,079][105692] Updated weights for policy 0, policy_version 1477483 (0.0010) [2023-12-27 02:09:08,298][105620] Updated weights for policy 1, policy_version 1479795 (0.0007) [2023-12-27 02:09:08,360][105620] Updated weights for policy 1, policy_version 1479805 (0.0008) [2023-12-27 02:09:08,420][105620] Updated weights for policy 1, policy_version 1479815 (0.0008) [2023-12-27 02:09:08,821][105692] Updated weights for policy 0, policy_version 1477493 (0.0010) [2023-12-27 02:09:08,871][105692] Updated weights for policy 0, policy_version 1477503 (0.0008) [2023-12-27 02:09:08,919][105692] Updated weights for policy 0, policy_version 1477513 (0.0005) [2023-12-27 02:09:09,085][105620] Updated weights for policy 1, policy_version 1479825 (0.0008) [2023-12-27 02:09:09,139][105620] Updated weights for policy 1, policy_version 1479835 (0.0008) [2023-12-27 02:09:09,205][105620] Updated weights for policy 1, policy_version 1479845 (0.0009) [2023-12-27 02:09:09,265][105620] Updated weights for policy 1, policy_version 1479855 (0.0008) [2023-12-27 02:09:09,540][105692] Updated weights for policy 0, policy_version 1477523 (0.0007) [2023-12-27 02:09:09,606][105692] Updated weights for policy 0, policy_version 1477533 (0.0010) [2023-12-27 02:09:09,670][105692] Updated weights for policy 0, policy_version 1477543 (0.0010) [2023-12-27 02:09:09,963][105620] Updated weights for policy 1, policy_version 1479865 (0.0009) [2023-12-27 02:09:10,028][105620] Updated weights for policy 1, policy_version 1479875 (0.0008) [2023-12-27 02:09:10,091][105620] Updated weights for policy 1, policy_version 1479885 (0.0009) [2023-12-27 02:09:10,468][105692] Updated weights for policy 0, policy_version 1477553 (0.0009) [2023-12-27 02:09:10,531][105692] Updated weights for policy 0, policy_version 1477563 (0.0009) [2023-12-27 02:09:10,591][105692] Updated weights for policy 0, policy_version 1477573 (0.0009) [2023-12-27 02:09:10,655][105692] Updated weights for policy 0, policy_version 1477583 (0.0009) [2023-12-27 02:09:10,833][105620] Updated weights for policy 1, policy_version 1479895 (0.0008) [2023-12-27 02:09:10,893][105620] Updated weights for policy 1, policy_version 1479905 (0.0008) [2023-12-27 02:09:10,941][105620] Updated weights for policy 1, policy_version 1479915 (0.0009) [2023-12-27 02:09:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 757227520. Throughput: 0: 9804.5, 1: 9931.3. Samples: 757232028. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:09:11,062][104569] Avg episode reward: [(0, '9080.380'), (1, '9082.160')] [2023-12-27 02:09:11,397][105692] Updated weights for policy 0, policy_version 1477593 (0.0008) [2023-12-27 02:09:11,457][105692] Updated weights for policy 0, policy_version 1477603 (0.0008) [2023-12-27 02:09:11,510][105692] Updated weights for policy 0, policy_version 1477613 (0.0007) [2023-12-27 02:09:11,733][105620] Updated weights for policy 1, policy_version 1479925 (0.0009) [2023-12-27 02:09:11,785][105620] Updated weights for policy 1, policy_version 1479935 (0.0009) [2023-12-27 02:09:11,832][105620] Updated weights for policy 1, policy_version 1479945 (0.0008) [2023-12-27 02:09:12,264][105692] Updated weights for policy 0, policy_version 1477623 (0.0007) [2023-12-27 02:09:12,326][105692] Updated weights for policy 0, policy_version 1477633 (0.0007) [2023-12-27 02:09:12,400][105692] Updated weights for policy 0, policy_version 1477643 (0.0009) [2023-12-27 02:09:12,688][105620] Updated weights for policy 1, policy_version 1479955 (0.0009) [2023-12-27 02:09:12,743][105620] Updated weights for policy 1, policy_version 1479965 (0.0010) [2023-12-27 02:09:12,811][105620] Updated weights for policy 1, policy_version 1479975 (0.0006) [2023-12-27 02:09:13,017][105692] Updated weights for policy 0, policy_version 1477653 (0.0008) [2023-12-27 02:09:13,080][105692] Updated weights for policy 0, policy_version 1477663 (0.0008) [2023-12-27 02:09:13,142][105692] Updated weights for policy 0, policy_version 1477673 (0.0009) [2023-12-27 02:09:13,528][105620] Updated weights for policy 1, policy_version 1479985 (0.0009) [2023-12-27 02:09:13,583][105620] Updated weights for policy 1, policy_version 1479995 (0.0005) [2023-12-27 02:09:13,632][105620] Updated weights for policy 1, policy_version 1480005 (0.0005) [2023-12-27 02:09:13,685][105620] Updated weights for policy 1, policy_version 1480015 (0.0006) [2023-12-27 02:09:13,717][105692] Updated weights for policy 0, policy_version 1477683 (0.0008) [2023-12-27 02:09:13,777][105692] Updated weights for policy 0, policy_version 1477693 (0.0005) [2023-12-27 02:09:13,825][105692] Updated weights for policy 0, policy_version 1477703 (0.0005) [2023-12-27 02:09:14,344][105620] Updated weights for policy 1, policy_version 1480025 (0.0005) [2023-12-27 02:09:14,367][105692] Updated weights for policy 0, policy_version 1477713 (0.0005) [2023-12-27 02:09:14,403][105620] Updated weights for policy 1, policy_version 1480035 (0.0010) [2023-12-27 02:09:14,430][105692] Updated weights for policy 0, policy_version 1477723 (0.0006) [2023-12-27 02:09:14,467][105620] Updated weights for policy 1, policy_version 1480045 (0.0005) [2023-12-27 02:09:14,498][105692] Updated weights for policy 0, policy_version 1477733 (0.0005) [2023-12-27 02:09:14,564][105692] Updated weights for policy 0, policy_version 1477743 (0.0005) [2023-12-27 02:09:15,002][105620] Updated weights for policy 1, policy_version 1480055 (0.0007) [2023-12-27 02:09:15,070][105620] Updated weights for policy 1, policy_version 1480065 (0.0008) [2023-12-27 02:09:15,128][105692] Updated weights for policy 0, policy_version 1477753 (0.0007) [2023-12-27 02:09:15,135][105620] Updated weights for policy 1, policy_version 1480075 (0.0007) [2023-12-27 02:09:15,187][105692] Updated weights for policy 0, policy_version 1477763 (0.0009) [2023-12-27 02:09:15,249][105692] Updated weights for policy 0, policy_version 1477773 (0.0008) [2023-12-27 02:09:15,720][105620] Updated weights for policy 1, policy_version 1480085 (0.0005) [2023-12-27 02:09:15,773][105620] Updated weights for policy 1, policy_version 1480095 (0.0005) [2023-12-27 02:09:15,839][105620] Updated weights for policy 1, policy_version 1480105 (0.0005) [2023-12-27 02:09:16,029][105692] Updated weights for policy 0, policy_version 1477783 (0.0006) [2023-12-27 02:09:16,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.2, 300 sec: 19577.5). Total num frames: 757325824. Throughput: 0: 9775.9, 1: 9927.8. Samples: 757289884. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:09:16,063][104569] Avg episode reward: [(0, '8437.307'), (1, '9082.996')] [2023-12-27 02:09:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001480112_378961920.pth... [2023-12-27 02:09:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001478928_378658816.pth [2023-12-27 02:09:16,091][105692] Updated weights for policy 0, policy_version 1477793 (0.0007) [2023-12-27 02:09:16,156][105692] Updated weights for policy 0, policy_version 1477803 (0.0010) [2023-12-27 02:09:16,177][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001477808_378372096.pth... [2023-12-27 02:09:16,180][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001476656_378077184.pth [2023-12-27 02:09:16,366][105620] Updated weights for policy 1, policy_version 1480115 (0.0005) [2023-12-27 02:09:16,420][105620] Updated weights for policy 1, policy_version 1480125 (0.0005) [2023-12-27 02:09:16,489][105620] Updated weights for policy 1, policy_version 1480135 (0.0005) [2023-12-27 02:09:16,702][105692] Updated weights for policy 0, policy_version 1477813 (0.0008) [2023-12-27 02:09:16,753][105692] Updated weights for policy 0, policy_version 1477823 (0.0008) [2023-12-27 02:09:16,804][105692] Updated weights for policy 0, policy_version 1477833 (0.0010) [2023-12-27 02:09:17,016][105620] Updated weights for policy 1, policy_version 1480145 (0.0006) [2023-12-27 02:09:17,067][105620] Updated weights for policy 1, policy_version 1480155 (0.0005) [2023-12-27 02:09:17,129][105620] Updated weights for policy 1, policy_version 1480165 (0.0005) [2023-12-27 02:09:17,183][105620] Updated weights for policy 1, policy_version 1480175 (0.0005) [2023-12-27 02:09:17,462][105692] Updated weights for policy 0, policy_version 1477843 (0.0008) [2023-12-27 02:09:17,521][105692] Updated weights for policy 0, policy_version 1477853 (0.0005) [2023-12-27 02:09:17,573][105692] Updated weights for policy 0, policy_version 1477863 (0.0005) [2023-12-27 02:09:17,805][105620] Updated weights for policy 1, policy_version 1480185 (0.0005) [2023-12-27 02:09:17,858][105620] Updated weights for policy 1, policy_version 1480195 (0.0005) [2023-12-27 02:09:17,922][105620] Updated weights for policy 1, policy_version 1480205 (0.0005) [2023-12-27 02:09:18,289][105692] Updated weights for policy 0, policy_version 1477873 (0.0006) [2023-12-27 02:09:18,348][105692] Updated weights for policy 0, policy_version 1477883 (0.0009) [2023-12-27 02:09:18,398][105692] Updated weights for policy 0, policy_version 1477893 (0.0008) [2023-12-27 02:09:18,452][105692] Updated weights for policy 0, policy_version 1477903 (0.0007) [2023-12-27 02:09:18,465][105620] Updated weights for policy 1, policy_version 1480215 (0.0009) [2023-12-27 02:09:18,527][105620] Updated weights for policy 1, policy_version 1480225 (0.0010) [2023-12-27 02:09:18,580][105620] Updated weights for policy 1, policy_version 1480235 (0.0010) [2023-12-27 02:09:19,299][105692] Updated weights for policy 0, policy_version 1477913 (0.0008) [2023-12-27 02:09:19,349][105620] Updated weights for policy 1, policy_version 1480245 (0.0009) [2023-12-27 02:09:19,360][105692] Updated weights for policy 0, policy_version 1477923 (0.0007) [2023-12-27 02:09:19,412][105620] Updated weights for policy 1, policy_version 1480255 (0.0010) [2023-12-27 02:09:19,422][105692] Updated weights for policy 0, policy_version 1477933 (0.0007) [2023-12-27 02:09:19,492][105620] Updated weights for policy 1, policy_version 1480265 (0.0010) [2023-12-27 02:09:20,130][105692] Updated weights for policy 0, policy_version 1477943 (0.0007) [2023-12-27 02:09:20,194][105692] Updated weights for policy 0, policy_version 1477953 (0.0007) [2023-12-27 02:09:20,261][105692] Updated weights for policy 0, policy_version 1477963 (0.0009) [2023-12-27 02:09:20,318][105620] Updated weights for policy 1, policy_version 1480275 (0.0009) [2023-12-27 02:09:20,374][105620] Updated weights for policy 1, policy_version 1480285 (0.0008) [2023-12-27 02:09:20,437][105620] Updated weights for policy 1, policy_version 1480295 (0.0008) [2023-12-27 02:09:20,944][105692] Updated weights for policy 0, policy_version 1477973 (0.0010) [2023-12-27 02:09:21,006][105692] Updated weights for policy 0, policy_version 1477983 (0.0010) [2023-12-27 02:09:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 757424128. Throughput: 0: 9853.6, 1: 10110.7. Samples: 757418432. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:09:21,062][104569] Avg episode reward: [(0, '8164.385'), (1, '9264.720')] [2023-12-27 02:09:21,075][105692] Updated weights for policy 0, policy_version 1477993 (0.0011) [2023-12-27 02:09:21,239][105620] Updated weights for policy 1, policy_version 1480305 (0.0008) [2023-12-27 02:09:21,302][105620] Updated weights for policy 1, policy_version 1480315 (0.0005) [2023-12-27 02:09:21,371][105620] Updated weights for policy 1, policy_version 1480325 (0.0008) [2023-12-27 02:09:21,426][105620] Updated weights for policy 1, policy_version 1480335 (0.0009) [2023-12-27 02:09:21,844][105692] Updated weights for policy 0, policy_version 1478003 (0.0008) [2023-12-27 02:09:21,903][105692] Updated weights for policy 0, policy_version 1478013 (0.0006) [2023-12-27 02:09:21,972][105692] Updated weights for policy 0, policy_version 1478023 (0.0007) [2023-12-27 02:09:22,210][105620] Updated weights for policy 1, policy_version 1480345 (0.0008) [2023-12-27 02:09:22,279][105620] Updated weights for policy 1, policy_version 1480355 (0.0008) [2023-12-27 02:09:22,350][105620] Updated weights for policy 1, policy_version 1480365 (0.0008) [2023-12-27 02:09:22,582][105692] Updated weights for policy 0, policy_version 1478033 (0.0006) [2023-12-27 02:09:22,644][105692] Updated weights for policy 0, policy_version 1478043 (0.0008) [2023-12-27 02:09:22,704][105692] Updated weights for policy 0, policy_version 1478053 (0.0010) [2023-12-27 02:09:22,763][105692] Updated weights for policy 0, policy_version 1478063 (0.0010) [2023-12-27 02:09:23,066][105620] Updated weights for policy 1, policy_version 1480375 (0.0009) [2023-12-27 02:09:23,119][105620] Updated weights for policy 1, policy_version 1480385 (0.0008) [2023-12-27 02:09:23,178][105620] Updated weights for policy 1, policy_version 1480395 (0.0010) [2023-12-27 02:09:23,429][105692] Updated weights for policy 0, policy_version 1478073 (0.0008) [2023-12-27 02:09:23,482][105692] Updated weights for policy 0, policy_version 1478083 (0.0008) [2023-12-27 02:09:23,534][105692] Updated weights for policy 0, policy_version 1478093 (0.0009) [2023-12-27 02:09:23,882][105620] Updated weights for policy 1, policy_version 1480405 (0.0007) [2023-12-27 02:09:23,944][105620] Updated weights for policy 1, policy_version 1480415 (0.0006) [2023-12-27 02:09:24,011][105620] Updated weights for policy 1, policy_version 1480425 (0.0005) [2023-12-27 02:09:24,251][105692] Updated weights for policy 0, policy_version 1478103 (0.0009) [2023-12-27 02:09:24,296][105692] Updated weights for policy 0, policy_version 1478113 (0.0010) [2023-12-27 02:09:24,359][105692] Updated weights for policy 0, policy_version 1478123 (0.0010) [2023-12-27 02:09:24,577][105620] Updated weights for policy 1, policy_version 1480435 (0.0007) [2023-12-27 02:09:24,637][105620] Updated weights for policy 1, policy_version 1480445 (0.0011) [2023-12-27 02:09:24,702][105620] Updated weights for policy 1, policy_version 1480455 (0.0010) [2023-12-27 02:09:25,105][105692] Updated weights for policy 0, policy_version 1478133 (0.0011) [2023-12-27 02:09:25,173][105692] Updated weights for policy 0, policy_version 1478143 (0.0010) [2023-12-27 02:09:25,235][105692] Updated weights for policy 0, policy_version 1478153 (0.0009) [2023-12-27 02:09:25,397][105620] Updated weights for policy 1, policy_version 1480465 (0.0010) [2023-12-27 02:09:25,454][105620] Updated weights for policy 1, policy_version 1480475 (0.0009) [2023-12-27 02:09:25,508][105620] Updated weights for policy 1, policy_version 1480485 (0.0010) [2023-12-27 02:09:25,558][105620] Updated weights for policy 1, policy_version 1480495 (0.0011) [2023-12-27 02:09:25,762][105692] Updated weights for policy 0, policy_version 1478163 (0.0005) [2023-12-27 02:09:25,815][105692] Updated weights for policy 0, policy_version 1478173 (0.0005) [2023-12-27 02:09:25,873][105692] Updated weights for policy 0, policy_version 1478183 (0.0005) [2023-12-27 02:09:26,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19933.8, 300 sec: 19605.2). Total num frames: 757530624. Throughput: 0: 9860.8, 1: 10127.5. Samples: 757535912. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:09:26,063][104569] Avg episode reward: [(0, '8621.587'), (1, '9172.329')] [2023-12-27 02:09:26,249][105620] Updated weights for policy 1, policy_version 1480505 (0.0006) [2023-12-27 02:09:26,298][105620] Updated weights for policy 1, policy_version 1480515 (0.0006) [2023-12-27 02:09:26,344][105620] Updated weights for policy 1, policy_version 1480525 (0.0005) [2023-12-27 02:09:26,466][105692] Updated weights for policy 0, policy_version 1478193 (0.0009) [2023-12-27 02:09:26,521][105692] Updated weights for policy 0, policy_version 1478203 (0.0010) [2023-12-27 02:09:26,573][105692] Updated weights for policy 0, policy_version 1478213 (0.0010) [2023-12-27 02:09:26,631][105692] Updated weights for policy 0, policy_version 1478223 (0.0010) [2023-12-27 02:09:26,960][105620] Updated weights for policy 1, policy_version 1480535 (0.0007) [2023-12-27 02:09:27,021][105620] Updated weights for policy 1, policy_version 1480545 (0.0008) [2023-12-27 02:09:27,081][105620] Updated weights for policy 1, policy_version 1480555 (0.0008) [2023-12-27 02:09:27,363][105692] Updated weights for policy 0, policy_version 1478233 (0.0010) [2023-12-27 02:09:27,428][105692] Updated weights for policy 0, policy_version 1478243 (0.0010) [2023-12-27 02:09:27,486][105692] Updated weights for policy 0, policy_version 1478253 (0.0010) [2023-12-27 02:09:27,722][105620] Updated weights for policy 1, policy_version 1480565 (0.0008) [2023-12-27 02:09:27,769][105620] Updated weights for policy 1, policy_version 1480575 (0.0008) [2023-12-27 02:09:27,840][105620] Updated weights for policy 1, policy_version 1480585 (0.0009) [2023-12-27 02:09:28,221][105692] Updated weights for policy 0, policy_version 1478263 (0.0008) [2023-12-27 02:09:28,273][105692] Updated weights for policy 0, policy_version 1478273 (0.0010) [2023-12-27 02:09:28,328][105692] Updated weights for policy 0, policy_version 1478283 (0.0010) [2023-12-27 02:09:28,656][105620] Updated weights for policy 1, policy_version 1480595 (0.0008) [2023-12-27 02:09:28,716][105620] Updated weights for policy 1, policy_version 1480605 (0.0008) [2023-12-27 02:09:28,779][105620] Updated weights for policy 1, policy_version 1480615 (0.0008) [2023-12-27 02:09:29,045][105692] Updated weights for policy 0, policy_version 1478293 (0.0008) [2023-12-27 02:09:29,103][105692] Updated weights for policy 0, policy_version 1478303 (0.0005) [2023-12-27 02:09:29,157][105692] Updated weights for policy 0, policy_version 1478313 (0.0007) [2023-12-27 02:09:29,585][105620] Updated weights for policy 1, policy_version 1480625 (0.0008) [2023-12-27 02:09:29,646][105620] Updated weights for policy 1, policy_version 1480635 (0.0009) [2023-12-27 02:09:29,700][105620] Updated weights for policy 1, policy_version 1480645 (0.0009) [2023-12-27 02:09:29,761][105620] Updated weights for policy 1, policy_version 1480655 (0.0009) [2023-12-27 02:09:29,859][105692] Updated weights for policy 0, policy_version 1478323 (0.0008) [2023-12-27 02:09:29,915][105692] Updated weights for policy 0, policy_version 1478333 (0.0008) [2023-12-27 02:09:29,976][105692] Updated weights for policy 0, policy_version 1478343 (0.0008) [2023-12-27 02:09:30,525][105620] Updated weights for policy 1, policy_version 1480665 (0.0010) [2023-12-27 02:09:30,579][105620] Updated weights for policy 1, policy_version 1480675 (0.0008) [2023-12-27 02:09:30,633][105620] Updated weights for policy 1, policy_version 1480685 (0.0009) [2023-12-27 02:09:30,715][105692] Updated weights for policy 0, policy_version 1478353 (0.0009) [2023-12-27 02:09:30,770][105692] Updated weights for policy 0, policy_version 1478363 (0.0009) [2023-12-27 02:09:30,821][105692] Updated weights for policy 0, policy_version 1478373 (0.0009) [2023-12-27 02:09:30,872][105692] Updated weights for policy 0, policy_version 1478383 (0.0009) [2023-12-27 02:09:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 757628928. Throughput: 0: 9897.2, 1: 10121.2. Samples: 757597012. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:09:31,062][104569] Avg episode reward: [(0, '8435.808'), (1, '9263.458')] [2023-12-27 02:09:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001478384_378519552.pth... [2023-12-27 02:09:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001480688_379109376.pth... [2023-12-27 02:09:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001477200_378216448.pth [2023-12-27 02:09:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001479536_378814464.pth [2023-12-27 02:09:31,373][105620] Updated weights for policy 1, policy_version 1480695 (0.0008) [2023-12-27 02:09:31,423][105620] Updated weights for policy 1, policy_version 1480705 (0.0008) [2023-12-27 02:09:31,471][105620] Updated weights for policy 1, policy_version 1480715 (0.0009) [2023-12-27 02:09:31,708][105692] Updated weights for policy 0, policy_version 1478393 (0.0009) [2023-12-27 02:09:31,763][105692] Updated weights for policy 0, policy_version 1478403 (0.0009) [2023-12-27 02:09:31,813][105692] Updated weights for policy 0, policy_version 1478413 (0.0008) [2023-12-27 02:09:32,172][105620] Updated weights for policy 1, policy_version 1480725 (0.0007) [2023-12-27 02:09:32,238][105620] Updated weights for policy 1, policy_version 1480735 (0.0010) [2023-12-27 02:09:32,307][105620] Updated weights for policy 1, policy_version 1480745 (0.0009) [2023-12-27 02:09:32,595][105692] Updated weights for policy 0, policy_version 1478423 (0.0009) [2023-12-27 02:09:32,644][105692] Updated weights for policy 0, policy_version 1478433 (0.0009) [2023-12-27 02:09:32,700][105692] Updated weights for policy 0, policy_version 1478443 (0.0009) [2023-12-27 02:09:33,077][105620] Updated weights for policy 1, policy_version 1480755 (0.0009) [2023-12-27 02:09:33,146][105620] Updated weights for policy 1, policy_version 1480765 (0.0009) [2023-12-27 02:09:33,203][105620] Updated weights for policy 1, policy_version 1480776 (0.0010) [2023-12-27 02:09:33,376][105692] Updated weights for policy 0, policy_version 1478453 (0.0007) [2023-12-27 02:09:33,440][105692] Updated weights for policy 0, policy_version 1478463 (0.0005) [2023-12-27 02:09:33,511][105692] Updated weights for policy 0, policy_version 1478473 (0.0005) [2023-12-27 02:09:34,064][105620] Updated weights for policy 1, policy_version 1480786 (0.0009) [2023-12-27 02:09:34,089][105692] Updated weights for policy 0, policy_version 1478483 (0.0007) [2023-12-27 02:09:34,123][105620] Updated weights for policy 1, policy_version 1480796 (0.0008) [2023-12-27 02:09:34,141][105692] Updated weights for policy 0, policy_version 1478493 (0.0007) [2023-12-27 02:09:34,189][105620] Updated weights for policy 1, policy_version 1480806 (0.0007) [2023-12-27 02:09:34,204][105692] Updated weights for policy 0, policy_version 1478503 (0.0008) [2023-12-27 02:09:34,246][105620] Updated weights for policy 1, policy_version 1480816 (0.0006) [2023-12-27 02:09:34,833][105692] Updated weights for policy 0, policy_version 1478513 (0.0008) [2023-12-27 02:09:34,895][105692] Updated weights for policy 0, policy_version 1478523 (0.0009) [2023-12-27 02:09:34,957][105692] Updated weights for policy 0, policy_version 1478533 (0.0009) [2023-12-27 02:09:35,018][105692] Updated weights for policy 0, policy_version 1478543 (0.0006) [2023-12-27 02:09:35,087][105620] Updated weights for policy 1, policy_version 1480826 (0.0009) [2023-12-27 02:09:35,151][105620] Updated weights for policy 1, policy_version 1480836 (0.0009) [2023-12-27 02:09:35,211][105620] Updated weights for policy 1, policy_version 1480846 (0.0009) [2023-12-27 02:09:35,611][105692] Updated weights for policy 0, policy_version 1478553 (0.0005) [2023-12-27 02:09:35,671][105692] Updated weights for policy 0, policy_version 1478563 (0.0006) [2023-12-27 02:09:35,731][105692] Updated weights for policy 0, policy_version 1478573 (0.0009) [2023-12-27 02:09:36,046][105620] Updated weights for policy 1, policy_version 1480856 (0.0010) [2023-12-27 02:09:36,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 757719040. Throughput: 0: 9907.4, 1: 9966.2. Samples: 757710436. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:09:36,062][104569] Avg episode reward: [(0, '8342.412'), (1, '8988.858')] [2023-12-27 02:09:36,103][105620] Updated weights for policy 1, policy_version 1480866 (0.0010) [2023-12-27 02:09:36,173][105620] Updated weights for policy 1, policy_version 1480876 (0.0008) [2023-12-27 02:09:36,311][105692] Updated weights for policy 0, policy_version 1478583 (0.0010) [2023-12-27 02:09:36,373][105692] Updated weights for policy 0, policy_version 1478593 (0.0010) [2023-12-27 02:09:36,432][105692] Updated weights for policy 0, policy_version 1478603 (0.0009) [2023-12-27 02:09:36,906][105620] Updated weights for policy 1, policy_version 1480886 (0.0008) [2023-12-27 02:09:36,964][105620] Updated weights for policy 1, policy_version 1480896 (0.0007) [2023-12-27 02:09:37,026][105620] Updated weights for policy 1, policy_version 1480906 (0.0009) [2023-12-27 02:09:37,217][105692] Updated weights for policy 0, policy_version 1478613 (0.0009) [2023-12-27 02:09:37,281][105692] Updated weights for policy 0, policy_version 1478623 (0.0009) [2023-12-27 02:09:37,335][105692] Updated weights for policy 0, policy_version 1478633 (0.0008) [2023-12-27 02:09:37,741][105620] Updated weights for policy 1, policy_version 1480916 (0.0009) [2023-12-27 02:09:37,807][105620] Updated weights for policy 1, policy_version 1480926 (0.0009) [2023-12-27 02:09:37,869][105620] Updated weights for policy 1, policy_version 1480936 (0.0009) [2023-12-27 02:09:38,056][105692] Updated weights for policy 0, policy_version 1478643 (0.0006) [2023-12-27 02:09:38,118][105692] Updated weights for policy 0, policy_version 1478653 (0.0009) [2023-12-27 02:09:38,176][105692] Updated weights for policy 0, policy_version 1478663 (0.0008) [2023-12-27 02:09:38,603][105620] Updated weights for policy 1, policy_version 1480946 (0.0008) [2023-12-27 02:09:38,652][105620] Updated weights for policy 1, policy_version 1480956 (0.0006) [2023-12-27 02:09:38,699][105620] Updated weights for policy 1, policy_version 1480966 (0.0009) [2023-12-27 02:09:38,749][105620] Updated weights for policy 1, policy_version 1480976 (0.0007) [2023-12-27 02:09:38,926][105692] Updated weights for policy 0, policy_version 1478673 (0.0009) [2023-12-27 02:09:38,985][105692] Updated weights for policy 0, policy_version 1478683 (0.0009) [2023-12-27 02:09:39,043][105692] Updated weights for policy 0, policy_version 1478693 (0.0009) [2023-12-27 02:09:39,098][105692] Updated weights for policy 0, policy_version 1478703 (0.0009) [2023-12-27 02:09:39,505][105620] Updated weights for policy 1, policy_version 1480986 (0.0009) [2023-12-27 02:09:39,570][105620] Updated weights for policy 1, policy_version 1480996 (0.0008) [2023-12-27 02:09:39,633][105620] Updated weights for policy 1, policy_version 1481006 (0.0007) [2023-12-27 02:09:39,908][105692] Updated weights for policy 0, policy_version 1478713 (0.0007) [2023-12-27 02:09:39,968][105692] Updated weights for policy 0, policy_version 1478723 (0.0010) [2023-12-27 02:09:40,021][105692] Updated weights for policy 0, policy_version 1478733 (0.0009) [2023-12-27 02:09:40,365][105620] Updated weights for policy 1, policy_version 1481016 (0.0010) [2023-12-27 02:09:40,424][105620] Updated weights for policy 1, policy_version 1481026 (0.0009) [2023-12-27 02:09:40,480][105620] Updated weights for policy 1, policy_version 1481036 (0.0009) [2023-12-27 02:09:40,724][105692] Updated weights for policy 0, policy_version 1478743 (0.0006) [2023-12-27 02:09:40,783][105692] Updated weights for policy 0, policy_version 1478753 (0.0005) [2023-12-27 02:09:40,842][105692] Updated weights for policy 0, policy_version 1478763 (0.0008) [2023-12-27 02:09:41,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 757817344. Throughput: 0: 9890.7, 1: 9817.6. Samples: 757824352. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:09:41,063][104569] Avg episode reward: [(0, '8164.229'), (1, '8896.773')] [2023-12-27 02:09:41,304][105620] Updated weights for policy 1, policy_version 1481046 (0.0008) [2023-12-27 02:09:41,373][105620] Updated weights for policy 1, policy_version 1481056 (0.0009) [2023-12-27 02:09:41,435][105620] Updated weights for policy 1, policy_version 1481066 (0.0009) [2023-12-27 02:09:41,551][105692] Updated weights for policy 0, policy_version 1478773 (0.0010) [2023-12-27 02:09:41,605][105692] Updated weights for policy 0, policy_version 1478783 (0.0010) [2023-12-27 02:09:41,670][105692] Updated weights for policy 0, policy_version 1478793 (0.0008) [2023-12-27 02:09:42,130][105620] Updated weights for policy 1, policy_version 1481076 (0.0008) [2023-12-27 02:09:42,183][105620] Updated weights for policy 1, policy_version 1481086 (0.0006) [2023-12-27 02:09:42,238][105620] Updated weights for policy 1, policy_version 1481096 (0.0005) [2023-12-27 02:09:42,521][105692] Updated weights for policy 0, policy_version 1478803 (0.0009) [2023-12-27 02:09:42,581][105692] Updated weights for policy 0, policy_version 1478813 (0.0009) [2023-12-27 02:09:42,638][105692] Updated weights for policy 0, policy_version 1478823 (0.0009) [2023-12-27 02:09:42,884][105620] Updated weights for policy 1, policy_version 1481106 (0.0008) [2023-12-27 02:09:42,939][105620] Updated weights for policy 1, policy_version 1481116 (0.0007) [2023-12-27 02:09:43,002][105620] Updated weights for policy 1, policy_version 1481126 (0.0008) [2023-12-27 02:09:43,058][105620] Updated weights for policy 1, policy_version 1481136 (0.0008) [2023-12-27 02:09:43,394][105692] Updated weights for policy 0, policy_version 1478833 (0.0009) [2023-12-27 02:09:43,448][105692] Updated weights for policy 0, policy_version 1478843 (0.0006) [2023-12-27 02:09:43,508][105692] Updated weights for policy 0, policy_version 1478853 (0.0005) [2023-12-27 02:09:43,575][105692] Updated weights for policy 0, policy_version 1478863 (0.0009) [2023-12-27 02:09:43,870][105620] Updated weights for policy 1, policy_version 1481146 (0.0009) [2023-12-27 02:09:43,922][105620] Updated weights for policy 1, policy_version 1481156 (0.0008) [2023-12-27 02:09:43,973][105620] Updated weights for policy 1, policy_version 1481166 (0.0009) [2023-12-27 02:09:44,231][105692] Updated weights for policy 0, policy_version 1478873 (0.0009) [2023-12-27 02:09:44,290][105692] Updated weights for policy 0, policy_version 1478883 (0.0010) [2023-12-27 02:09:44,343][105692] Updated weights for policy 0, policy_version 1478893 (0.0009) [2023-12-27 02:09:44,701][105620] Updated weights for policy 1, policy_version 1481176 (0.0009) [2023-12-27 02:09:44,751][105620] Updated weights for policy 1, policy_version 1481186 (0.0008) [2023-12-27 02:09:44,811][105620] Updated weights for policy 1, policy_version 1481196 (0.0008) [2023-12-27 02:09:45,152][105692] Updated weights for policy 0, policy_version 1478903 (0.0009) [2023-12-27 02:09:45,216][105692] Updated weights for policy 0, policy_version 1478913 (0.0009) [2023-12-27 02:09:45,275][105692] Updated weights for policy 0, policy_version 1478923 (0.0010) [2023-12-27 02:09:45,542][105620] Updated weights for policy 1, policy_version 1481206 (0.0008) [2023-12-27 02:09:45,609][105620] Updated weights for policy 1, policy_version 1481216 (0.0010) [2023-12-27 02:09:45,674][105620] Updated weights for policy 1, policy_version 1481226 (0.0005) [2023-12-27 02:09:45,891][105692] Updated weights for policy 0, policy_version 1478933 (0.0007) [2023-12-27 02:09:45,939][105692] Updated weights for policy 0, policy_version 1478943 (0.0005) [2023-12-27 02:09:46,004][105692] Updated weights for policy 0, policy_version 1478953 (0.0005) [2023-12-27 02:09:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 757915648. Throughput: 0: 9921.3, 1: 9760.1. Samples: 757880780. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:09:46,063][104569] Avg episode reward: [(0, '8256.899'), (1, '9173.517')] [2023-12-27 02:09:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001481232_379248640.pth... [2023-12-27 02:09:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001478960_378667008.pth... [2023-12-27 02:09:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001480112_378961920.pth [2023-12-27 02:09:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001477808_378372096.pth [2023-12-27 02:09:46,221][105620] Updated weights for policy 1, policy_version 1481236 (0.0007) [2023-12-27 02:09:46,274][105620] Updated weights for policy 1, policy_version 1481246 (0.0009) [2023-12-27 02:09:46,328][105620] Updated weights for policy 1, policy_version 1481257 (0.0010) [2023-12-27 02:09:46,510][105692] Updated weights for policy 0, policy_version 1478963 (0.0005) [2023-12-27 02:09:46,564][105692] Updated weights for policy 0, policy_version 1478973 (0.0006) [2023-12-27 02:09:46,621][105692] Updated weights for policy 0, policy_version 1478983 (0.0006) [2023-12-27 02:09:46,991][105620] Updated weights for policy 1, policy_version 1481267 (0.0006) [2023-12-27 02:09:47,039][105620] Updated weights for policy 1, policy_version 1481277 (0.0005) [2023-12-27 02:09:47,083][105620] Updated weights for policy 1, policy_version 1481287 (0.0005) [2023-12-27 02:09:47,298][105692] Updated weights for policy 0, policy_version 1478993 (0.0005) [2023-12-27 02:09:47,347][105692] Updated weights for policy 0, policy_version 1479003 (0.0005) [2023-12-27 02:09:47,394][105692] Updated weights for policy 0, policy_version 1479013 (0.0005) [2023-12-27 02:09:47,443][105692] Updated weights for policy 0, policy_version 1479023 (0.0006) [2023-12-27 02:09:47,762][105620] Updated weights for policy 1, policy_version 1481297 (0.0007) [2023-12-27 02:09:47,814][105620] Updated weights for policy 1, policy_version 1481307 (0.0008) [2023-12-27 02:09:47,859][105620] Updated weights for policy 1, policy_version 1481317 (0.0008) [2023-12-27 02:09:47,909][105620] Updated weights for policy 1, policy_version 1481327 (0.0010) [2023-12-27 02:09:48,124][105692] Updated weights for policy 0, policy_version 1479033 (0.0008) [2023-12-27 02:09:48,170][105692] Updated weights for policy 0, policy_version 1479043 (0.0008) [2023-12-27 02:09:48,219][105692] Updated weights for policy 0, policy_version 1479053 (0.0008) [2023-12-27 02:09:48,679][105620] Updated weights for policy 1, policy_version 1481337 (0.0010) [2023-12-27 02:09:48,731][105620] Updated weights for policy 1, policy_version 1481347 (0.0010) [2023-12-27 02:09:48,789][105620] Updated weights for policy 1, policy_version 1481357 (0.0010) [2023-12-27 02:09:48,990][105692] Updated weights for policy 0, policy_version 1479063 (0.0009) [2023-12-27 02:09:49,047][105692] Updated weights for policy 0, policy_version 1479073 (0.0008) [2023-12-27 02:09:49,113][105692] Updated weights for policy 0, policy_version 1479083 (0.0009) [2023-12-27 02:09:49,542][105620] Updated weights for policy 1, policy_version 1481367 (0.0010) [2023-12-27 02:09:49,591][105620] Updated weights for policy 1, policy_version 1481377 (0.0010) [2023-12-27 02:09:49,643][105620] Updated weights for policy 1, policy_version 1481387 (0.0010) [2023-12-27 02:09:49,843][105692] Updated weights for policy 0, policy_version 1479093 (0.0009) [2023-12-27 02:09:49,908][105692] Updated weights for policy 0, policy_version 1479103 (0.0007) [2023-12-27 02:09:49,980][105692] Updated weights for policy 0, policy_version 1479113 (0.0009) [2023-12-27 02:09:50,317][105620] Updated weights for policy 1, policy_version 1481397 (0.0006) [2023-12-27 02:09:50,379][105620] Updated weights for policy 1, policy_version 1481407 (0.0010) [2023-12-27 02:09:50,438][105620] Updated weights for policy 1, policy_version 1481417 (0.0010) [2023-12-27 02:09:50,682][105692] Updated weights for policy 0, policy_version 1479123 (0.0009) [2023-12-27 02:09:50,738][105692] Updated weights for policy 0, policy_version 1479133 (0.0009) [2023-12-27 02:09:50,789][105692] Updated weights for policy 0, policy_version 1479143 (0.0009) [2023-12-27 02:09:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 758013952. Throughput: 0: 9971.9, 1: 9782.1. Samples: 758002004. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:09:51,062][104569] Avg episode reward: [(0, '8621.571'), (1, '9174.098')] [2023-12-27 02:09:51,192][105620] Updated weights for policy 1, policy_version 1481427 (0.0010) [2023-12-27 02:09:51,256][105620] Updated weights for policy 1, policy_version 1481437 (0.0011) [2023-12-27 02:09:51,330][105620] Updated weights for policy 1, policy_version 1481447 (0.0011) [2023-12-27 02:09:51,589][105692] Updated weights for policy 0, policy_version 1479153 (0.0007) [2023-12-27 02:09:51,657][105692] Updated weights for policy 0, policy_version 1479163 (0.0009) [2023-12-27 02:09:51,725][105692] Updated weights for policy 0, policy_version 1479173 (0.0008) [2023-12-27 02:09:51,793][105692] Updated weights for policy 0, policy_version 1479183 (0.0006) [2023-12-27 02:09:52,099][105620] Updated weights for policy 1, policy_version 1481457 (0.0009) [2023-12-27 02:09:52,165][105620] Updated weights for policy 1, policy_version 1481467 (0.0009) [2023-12-27 02:09:52,226][105620] Updated weights for policy 1, policy_version 1481477 (0.0009) [2023-12-27 02:09:52,285][105620] Updated weights for policy 1, policy_version 1481487 (0.0009) [2023-12-27 02:09:52,533][105692] Updated weights for policy 0, policy_version 1479193 (0.0006) [2023-12-27 02:09:52,592][105692] Updated weights for policy 0, policy_version 1479203 (0.0008) [2023-12-27 02:09:52,652][105692] Updated weights for policy 0, policy_version 1479213 (0.0009) [2023-12-27 02:09:53,009][105620] Updated weights for policy 1, policy_version 1481497 (0.0008) [2023-12-27 02:09:53,073][105620] Updated weights for policy 1, policy_version 1481507 (0.0006) [2023-12-27 02:09:53,133][105620] Updated weights for policy 1, policy_version 1481517 (0.0010) [2023-12-27 02:09:53,435][105692] Updated weights for policy 0, policy_version 1479223 (0.0008) [2023-12-27 02:09:53,503][105692] Updated weights for policy 0, policy_version 1479233 (0.0010) [2023-12-27 02:09:53,570][105692] Updated weights for policy 0, policy_version 1479243 (0.0010) [2023-12-27 02:09:53,755][105620] Updated weights for policy 1, policy_version 1481527 (0.0009) [2023-12-27 02:09:53,809][105620] Updated weights for policy 1, policy_version 1481537 (0.0010) [2023-12-27 02:09:53,868][105620] Updated weights for policy 1, policy_version 1481547 (0.0006) [2023-12-27 02:09:54,374][105692] Updated weights for policy 0, policy_version 1479253 (0.0009) [2023-12-27 02:09:54,422][105692] Updated weights for policy 0, policy_version 1479263 (0.0009) [2023-12-27 02:09:54,475][105692] Updated weights for policy 0, policy_version 1479273 (0.0009) [2023-12-27 02:09:54,538][105620] Updated weights for policy 1, policy_version 1481557 (0.0005) [2023-12-27 02:09:54,587][105620] Updated weights for policy 1, policy_version 1481567 (0.0008) [2023-12-27 02:09:54,633][105620] Updated weights for policy 1, policy_version 1481577 (0.0008) [2023-12-27 02:09:55,279][105620] Updated weights for policy 1, policy_version 1481587 (0.0009) [2023-12-27 02:09:55,322][105692] Updated weights for policy 0, policy_version 1479283 (0.0009) [2023-12-27 02:09:55,340][105620] Updated weights for policy 1, policy_version 1481597 (0.0009) [2023-12-27 02:09:55,372][105692] Updated weights for policy 0, policy_version 1479293 (0.0009) [2023-12-27 02:09:55,401][105620] Updated weights for policy 1, policy_version 1481607 (0.0009) [2023-12-27 02:09:55,422][105692] Updated weights for policy 0, policy_version 1479303 (0.0008) [2023-12-27 02:09:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 758104064. Throughput: 0: 9861.3, 1: 9766.2. Samples: 758115268. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:09:56,063][104569] Avg episode reward: [(0, '8167.411'), (1, '9173.828')] [2023-12-27 02:09:56,140][105620] Updated weights for policy 1, policy_version 1481617 (0.0008) [2023-12-27 02:09:56,196][105692] Updated weights for policy 0, policy_version 1479313 (0.0009) [2023-12-27 02:09:56,202][105620] Updated weights for policy 1, policy_version 1481627 (0.0009) [2023-12-27 02:09:56,256][105692] Updated weights for policy 0, policy_version 1479323 (0.0005) [2023-12-27 02:09:56,261][105620] Updated weights for policy 1, policy_version 1481637 (0.0008) [2023-12-27 02:09:56,308][105692] Updated weights for policy 0, policy_version 1479333 (0.0006) [2023-12-27 02:09:56,317][105620] Updated weights for policy 1, policy_version 1481647 (0.0008) [2023-12-27 02:09:56,368][105692] Updated weights for policy 0, policy_version 1479343 (0.0008) [2023-12-27 02:09:56,987][105620] Updated weights for policy 1, policy_version 1481657 (0.0006) [2023-12-27 02:09:57,043][105620] Updated weights for policy 1, policy_version 1481667 (0.0006) [2023-12-27 02:09:57,097][105620] Updated weights for policy 1, policy_version 1481677 (0.0009) [2023-12-27 02:09:57,168][105692] Updated weights for policy 0, policy_version 1479353 (0.0009) [2023-12-27 02:09:57,216][105692] Updated weights for policy 0, policy_version 1479363 (0.0009) [2023-12-27 02:09:57,264][105692] Updated weights for policy 0, policy_version 1479374 (0.0009) [2023-12-27 02:09:57,760][105620] Updated weights for policy 1, policy_version 1481687 (0.0008) [2023-12-27 02:09:57,809][105620] Updated weights for policy 1, policy_version 1481697 (0.0008) [2023-12-27 02:09:57,854][105620] Updated weights for policy 1, policy_version 1481707 (0.0008) [2023-12-27 02:09:58,083][105692] Updated weights for policy 0, policy_version 1479384 (0.0009) [2023-12-27 02:09:58,149][105692] Updated weights for policy 0, policy_version 1479394 (0.0009) [2023-12-27 02:09:58,217][105692] Updated weights for policy 0, policy_version 1479404 (0.0009) [2023-12-27 02:09:58,588][105620] Updated weights for policy 1, policy_version 1481717 (0.0007) [2023-12-27 02:09:58,659][105620] Updated weights for policy 1, policy_version 1481727 (0.0007) [2023-12-27 02:09:58,724][105620] Updated weights for policy 1, policy_version 1481737 (0.0008) [2023-12-27 02:09:59,002][105692] Updated weights for policy 0, policy_version 1479414 (0.0008) [2023-12-27 02:09:59,070][105692] Updated weights for policy 0, policy_version 1479424 (0.0009) [2023-12-27 02:09:59,132][105692] Updated weights for policy 0, policy_version 1479434 (0.0010) [2023-12-27 02:09:59,412][105620] Updated weights for policy 1, policy_version 1481747 (0.0009) [2023-12-27 02:09:59,469][105620] Updated weights for policy 1, policy_version 1481757 (0.0008) [2023-12-27 02:09:59,526][105620] Updated weights for policy 1, policy_version 1481767 (0.0009) [2023-12-27 02:09:59,945][105692] Updated weights for policy 0, policy_version 1479444 (0.0009) [2023-12-27 02:10:00,002][105692] Updated weights for policy 0, policy_version 1479454 (0.0009) [2023-12-27 02:10:00,061][105692] Updated weights for policy 0, policy_version 1479464 (0.0009) [2023-12-27 02:10:00,203][105620] Updated weights for policy 1, policy_version 1481777 (0.0009) [2023-12-27 02:10:00,254][105620] Updated weights for policy 1, policy_version 1481787 (0.0009) [2023-12-27 02:10:00,307][105620] Updated weights for policy 1, policy_version 1481797 (0.0009) [2023-12-27 02:10:00,353][105620] Updated weights for policy 1, policy_version 1481807 (0.0008) [2023-12-27 02:10:00,865][105692] Updated weights for policy 0, policy_version 1479474 (0.0009) [2023-12-27 02:10:00,913][105692] Updated weights for policy 0, policy_version 1479484 (0.0009) [2023-12-27 02:10:00,959][105692] Updated weights for policy 0, policy_version 1479494 (0.0009) [2023-12-27 02:10:01,006][105692] Updated weights for policy 0, policy_version 1479504 (0.0009) [2023-12-27 02:10:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 758202368. Throughput: 0: 9790.0, 1: 9808.6. Samples: 758171816. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:10:01,062][104569] Avg episode reward: [(0, '8441.208'), (1, '9265.206')] [2023-12-27 02:10:01,064][105620] Updated weights for policy 1, policy_version 1481817 (0.0008) [2023-12-27 02:10:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001479504_378806272.pth... [2023-12-27 02:10:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001478384_378519552.pth [2023-12-27 02:10:01,124][105620] Updated weights for policy 1, policy_version 1481827 (0.0008) [2023-12-27 02:10:01,185][105620] Updated weights for policy 1, policy_version 1481837 (0.0008) [2023-12-27 02:10:01,200][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001481840_379404288.pth... [2023-12-27 02:10:01,206][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001480688_379109376.pth [2023-12-27 02:10:01,849][105620] Updated weights for policy 1, policy_version 1481847 (0.0007) [2023-12-27 02:10:01,888][105692] Updated weights for policy 0, policy_version 1479514 (0.0008) [2023-12-27 02:10:01,907][105620] Updated weights for policy 1, policy_version 1481857 (0.0009) [2023-12-27 02:10:01,947][105692] Updated weights for policy 0, policy_version 1479524 (0.0007) [2023-12-27 02:10:01,950][105620] Updated weights for policy 1, policy_version 1481867 (0.0007) [2023-12-27 02:10:02,014][105692] Updated weights for policy 0, policy_version 1479534 (0.0006) [2023-12-27 02:10:02,674][105620] Updated weights for policy 1, policy_version 1481877 (0.0008) [2023-12-27 02:10:02,724][105620] Updated weights for policy 1, policy_version 1481887 (0.0009) [2023-12-27 02:10:02,781][105620] Updated weights for policy 1, policy_version 1481897 (0.0009) [2023-12-27 02:10:02,783][105692] Updated weights for policy 0, policy_version 1479544 (0.0008) [2023-12-27 02:10:02,838][105692] Updated weights for policy 0, policy_version 1479554 (0.0007) [2023-12-27 02:10:02,897][105692] Updated weights for policy 0, policy_version 1479564 (0.0009) [2023-12-27 02:10:03,534][105620] Updated weights for policy 1, policy_version 1481907 (0.0008) [2023-12-27 02:10:03,580][105620] Updated weights for policy 1, policy_version 1481917 (0.0009) [2023-12-27 02:10:03,626][105620] Updated weights for policy 1, policy_version 1481927 (0.0008) [2023-12-27 02:10:03,661][105692] Updated weights for policy 0, policy_version 1479574 (0.0008) [2023-12-27 02:10:03,725][105692] Updated weights for policy 0, policy_version 1479584 (0.0008) [2023-12-27 02:10:03,790][105692] Updated weights for policy 0, policy_version 1479594 (0.0010) [2023-12-27 02:10:04,324][105620] Updated weights for policy 1, policy_version 1481937 (0.0007) [2023-12-27 02:10:04,369][105620] Updated weights for policy 1, policy_version 1481947 (0.0010) [2023-12-27 02:10:04,426][105620] Updated weights for policy 1, policy_version 1481957 (0.0010) [2023-12-27 02:10:04,491][105620] Updated weights for policy 1, policy_version 1481967 (0.0011) [2023-12-27 02:10:04,615][105692] Updated weights for policy 0, policy_version 1479604 (0.0009) [2023-12-27 02:10:04,680][105692] Updated weights for policy 0, policy_version 1479614 (0.0009) [2023-12-27 02:10:04,740][105692] Updated weights for policy 0, policy_version 1479624 (0.0008) [2023-12-27 02:10:05,235][105620] Updated weights for policy 1, policy_version 1481977 (0.0010) [2023-12-27 02:10:05,289][105620] Updated weights for policy 1, policy_version 1481987 (0.0010) [2023-12-27 02:10:05,344][105620] Updated weights for policy 1, policy_version 1481997 (0.0010) [2023-12-27 02:10:05,484][105692] Updated weights for policy 0, policy_version 1479634 (0.0008) [2023-12-27 02:10:05,547][105692] Updated weights for policy 0, policy_version 1479644 (0.0008) [2023-12-27 02:10:05,594][105692] Updated weights for policy 0, policy_version 1479654 (0.0008) [2023-12-27 02:10:05,642][105692] Updated weights for policy 0, policy_version 1479664 (0.0008) [2023-12-27 02:10:06,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 758292480. Throughput: 0: 9579.7, 1: 9658.6. Samples: 758284156. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:10:06,062][104569] Avg episode reward: [(0, '8715.685'), (1, '9176.392')] [2023-12-27 02:10:06,122][105620] Updated weights for policy 1, policy_version 1482007 (0.0011) [2023-12-27 02:10:06,183][105620] Updated weights for policy 1, policy_version 1482017 (0.0012) [2023-12-27 02:10:06,235][105620] Updated weights for policy 1, policy_version 1482027 (0.0010) [2023-12-27 02:10:06,324][105692] Updated weights for policy 0, policy_version 1479674 (0.0005) [2023-12-27 02:10:06,377][105692] Updated weights for policy 0, policy_version 1479684 (0.0008) [2023-12-27 02:10:06,429][105692] Updated weights for policy 0, policy_version 1479694 (0.0008) [2023-12-27 02:10:06,984][105620] Updated weights for policy 1, policy_version 1482037 (0.0011) [2023-12-27 02:10:07,037][105620] Updated weights for policy 1, policy_version 1482047 (0.0010) [2023-12-27 02:10:07,096][105620] Updated weights for policy 1, policy_version 1482057 (0.0010) [2023-12-27 02:10:07,167][105692] Updated weights for policy 0, policy_version 1479704 (0.0008) [2023-12-27 02:10:07,226][105692] Updated weights for policy 0, policy_version 1479714 (0.0008) [2023-12-27 02:10:07,287][105692] Updated weights for policy 0, policy_version 1479724 (0.0008) [2023-12-27 02:10:07,859][105620] Updated weights for policy 1, policy_version 1482067 (0.0010) [2023-12-27 02:10:07,912][105620] Updated weights for policy 1, policy_version 1482077 (0.0005) [2023-12-27 02:10:07,976][105620] Updated weights for policy 1, policy_version 1482087 (0.0005) [2023-12-27 02:10:08,005][105692] Updated weights for policy 0, policy_version 1479734 (0.0009) [2023-12-27 02:10:08,054][105692] Updated weights for policy 0, policy_version 1479744 (0.0006) [2023-12-27 02:10:08,109][105692] Updated weights for policy 0, policy_version 1479754 (0.0005) [2023-12-27 02:10:08,555][105620] Updated weights for policy 1, policy_version 1482097 (0.0005) [2023-12-27 02:10:08,618][105620] Updated weights for policy 1, policy_version 1482107 (0.0010) [2023-12-27 02:10:08,676][105692] Updated weights for policy 0, policy_version 1479764 (0.0006) [2023-12-27 02:10:08,682][105620] Updated weights for policy 1, policy_version 1482117 (0.0009) [2023-12-27 02:10:08,728][105692] Updated weights for policy 0, policy_version 1479774 (0.0006) [2023-12-27 02:10:08,738][105620] Updated weights for policy 1, policy_version 1482127 (0.0009) [2023-12-27 02:10:08,782][105692] Updated weights for policy 0, policy_version 1479784 (0.0007) [2023-12-27 02:10:09,451][105692] Updated weights for policy 0, policy_version 1479794 (0.0006) [2023-12-27 02:10:09,508][105692] Updated weights for policy 0, policy_version 1479804 (0.0008) [2023-12-27 02:10:09,543][105620] Updated weights for policy 1, policy_version 1482137 (0.0007) [2023-12-27 02:10:09,570][105692] Updated weights for policy 0, policy_version 1479814 (0.0008) [2023-12-27 02:10:09,601][105620] Updated weights for policy 1, policy_version 1482147 (0.0006) [2023-12-27 02:10:09,635][105692] Updated weights for policy 0, policy_version 1479824 (0.0007) [2023-12-27 02:10:09,660][105620] Updated weights for policy 1, policy_version 1482157 (0.0007) [2023-12-27 02:10:10,409][105692] Updated weights for policy 0, policy_version 1479834 (0.0009) [2023-12-27 02:10:10,416][105620] Updated weights for policy 1, policy_version 1482167 (0.0007) [2023-12-27 02:10:10,467][105692] Updated weights for policy 0, policy_version 1479844 (0.0006) [2023-12-27 02:10:10,481][105620] Updated weights for policy 1, policy_version 1482177 (0.0008) [2023-12-27 02:10:10,522][105692] Updated weights for policy 0, policy_version 1479854 (0.0005) [2023-12-27 02:10:10,546][105620] Updated weights for policy 1, policy_version 1482187 (0.0009) [2023-12-27 02:10:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 758390784. Throughput: 0: 9561.6, 1: 9637.7. Samples: 758399876. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:10:11,062][104569] Avg episode reward: [(0, '8348.894'), (1, '8994.881')] [2023-12-27 02:10:11,165][105692] Updated weights for policy 0, policy_version 1479864 (0.0008) [2023-12-27 02:10:11,231][105692] Updated weights for policy 0, policy_version 1479874 (0.0007) [2023-12-27 02:10:11,292][105692] Updated weights for policy 0, policy_version 1479884 (0.0010) [2023-12-27 02:10:11,381][105620] Updated weights for policy 1, policy_version 1482197 (0.0008) [2023-12-27 02:10:11,440][105620] Updated weights for policy 1, policy_version 1482207 (0.0009) [2023-12-27 02:10:11,499][105620] Updated weights for policy 1, policy_version 1482217 (0.0005) [2023-12-27 02:10:12,100][105692] Updated weights for policy 0, policy_version 1479894 (0.0008) [2023-12-27 02:10:12,155][105692] Updated weights for policy 0, policy_version 1479904 (0.0006) [2023-12-27 02:10:12,217][105692] Updated weights for policy 0, policy_version 1479914 (0.0005) [2023-12-27 02:10:12,254][105620] Updated weights for policy 1, policy_version 1482227 (0.0009) [2023-12-27 02:10:12,321][105620] Updated weights for policy 1, policy_version 1482237 (0.0011) [2023-12-27 02:10:12,388][105620] Updated weights for policy 1, policy_version 1482247 (0.0012) [2023-12-27 02:10:12,892][105692] Updated weights for policy 0, policy_version 1479924 (0.0008) [2023-12-27 02:10:12,948][105692] Updated weights for policy 0, policy_version 1479934 (0.0008) [2023-12-27 02:10:13,004][105692] Updated weights for policy 0, policy_version 1479944 (0.0007) [2023-12-27 02:10:13,047][105620] Updated weights for policy 1, policy_version 1482257 (0.0009) [2023-12-27 02:10:13,096][105620] Updated weights for policy 1, policy_version 1482267 (0.0005) [2023-12-27 02:10:13,156][105620] Updated weights for policy 1, policy_version 1482277 (0.0010) [2023-12-27 02:10:13,214][105620] Updated weights for policy 1, policy_version 1482287 (0.0010) [2023-12-27 02:10:13,625][105692] Updated weights for policy 0, policy_version 1479954 (0.0009) [2023-12-27 02:10:13,677][105692] Updated weights for policy 0, policy_version 1479964 (0.0010) [2023-12-27 02:10:13,740][105692] Updated weights for policy 0, policy_version 1479974 (0.0010) [2023-12-27 02:10:13,796][105620] Updated weights for policy 1, policy_version 1482297 (0.0006) [2023-12-27 02:10:13,799][105692] Updated weights for policy 0, policy_version 1479984 (0.0010) [2023-12-27 02:10:13,857][105620] Updated weights for policy 1, policy_version 1482307 (0.0007) [2023-12-27 02:10:13,927][105620] Updated weights for policy 1, policy_version 1482318 (0.0010) [2023-12-27 02:10:14,446][105692] Updated weights for policy 0, policy_version 1479994 (0.0010) [2023-12-27 02:10:14,503][105692] Updated weights for policy 0, policy_version 1480004 (0.0010) [2023-12-27 02:10:14,566][105692] Updated weights for policy 0, policy_version 1480014 (0.0007) [2023-12-27 02:10:14,704][105620] Updated weights for policy 1, policy_version 1482328 (0.0008) [2023-12-27 02:10:14,752][105620] Updated weights for policy 1, policy_version 1482338 (0.0008) [2023-12-27 02:10:14,813][105620] Updated weights for policy 1, policy_version 1482348 (0.0008) [2023-12-27 02:10:15,328][105692] Updated weights for policy 0, policy_version 1480024 (0.0011) [2023-12-27 02:10:15,386][105692] Updated weights for policy 0, policy_version 1480034 (0.0011) [2023-12-27 02:10:15,449][105692] Updated weights for policy 0, policy_version 1480044 (0.0011) [2023-12-27 02:10:15,608][105620] Updated weights for policy 1, policy_version 1482358 (0.0008) [2023-12-27 02:10:15,667][105620] Updated weights for policy 1, policy_version 1482368 (0.0006) [2023-12-27 02:10:15,723][105620] Updated weights for policy 1, policy_version 1482378 (0.0009) [2023-12-27 02:10:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 758489088. Throughput: 0: 9554.4, 1: 9634.1. Samples: 758460496. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:10:16,062][104569] Avg episode reward: [(0, '8530.088'), (1, '8993.559')] [2023-12-27 02:10:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001482384_379543552.pth... [2023-12-27 02:10:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001480048_378945536.pth... [2023-12-27 02:10:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001481232_379248640.pth [2023-12-27 02:10:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001478960_378667008.pth [2023-12-27 02:10:16,158][105692] Updated weights for policy 0, policy_version 1480054 (0.0011) [2023-12-27 02:10:16,213][105692] Updated weights for policy 0, policy_version 1480064 (0.0010) [2023-12-27 02:10:16,274][105692] Updated weights for policy 0, policy_version 1480074 (0.0010) [2023-12-27 02:10:16,307][105620] Updated weights for policy 1, policy_version 1482388 (0.0008) [2023-12-27 02:10:16,357][105620] Updated weights for policy 1, policy_version 1482398 (0.0005) [2023-12-27 02:10:16,406][105620] Updated weights for policy 1, policy_version 1482408 (0.0006) [2023-12-27 02:10:16,868][105692] Updated weights for policy 0, policy_version 1480084 (0.0008) [2023-12-27 02:10:16,926][105692] Updated weights for policy 0, policy_version 1480094 (0.0010) [2023-12-27 02:10:16,974][105692] Updated weights for policy 0, policy_version 1480104 (0.0010) [2023-12-27 02:10:17,101][105620] Updated weights for policy 1, policy_version 1482418 (0.0006) [2023-12-27 02:10:17,158][105620] Updated weights for policy 1, policy_version 1482428 (0.0010) [2023-12-27 02:10:17,207][105620] Updated weights for policy 1, policy_version 1482438 (0.0010) [2023-12-27 02:10:17,254][105620] Updated weights for policy 1, policy_version 1482448 (0.0010) [2023-12-27 02:10:17,543][105692] Updated weights for policy 0, policy_version 1480114 (0.0011) [2023-12-27 02:10:17,603][105692] Updated weights for policy 0, policy_version 1480124 (0.0006) [2023-12-27 02:10:17,654][105692] Updated weights for policy 0, policy_version 1480134 (0.0005) [2023-12-27 02:10:17,718][105692] Updated weights for policy 0, policy_version 1480144 (0.0009) [2023-12-27 02:10:18,002][105620] Updated weights for policy 1, policy_version 1482458 (0.0006) [2023-12-27 02:10:18,070][105620] Updated weights for policy 1, policy_version 1482468 (0.0006) [2023-12-27 02:10:18,127][105620] Updated weights for policy 1, policy_version 1482478 (0.0005) [2023-12-27 02:10:18,283][105692] Updated weights for policy 0, policy_version 1480154 (0.0007) [2023-12-27 02:10:18,331][105692] Updated weights for policy 0, policy_version 1480164 (0.0010) [2023-12-27 02:10:18,385][105692] Updated weights for policy 0, policy_version 1480174 (0.0008) [2023-12-27 02:10:18,650][105620] Updated weights for policy 1, policy_version 1482488 (0.0005) [2023-12-27 02:10:18,705][105620] Updated weights for policy 1, policy_version 1482498 (0.0006) [2023-12-27 02:10:18,759][105620] Updated weights for policy 1, policy_version 1482508 (0.0011) [2023-12-27 02:10:19,194][105692] Updated weights for policy 0, policy_version 1480184 (0.0008) [2023-12-27 02:10:19,257][105692] Updated weights for policy 0, policy_version 1480194 (0.0011) [2023-12-27 02:10:19,325][105692] Updated weights for policy 0, policy_version 1480204 (0.0007) [2023-12-27 02:10:19,379][105620] Updated weights for policy 1, policy_version 1482518 (0.0010) [2023-12-27 02:10:19,438][105620] Updated weights for policy 1, policy_version 1482528 (0.0007) [2023-12-27 02:10:19,502][105620] Updated weights for policy 1, policy_version 1482538 (0.0008) [2023-12-27 02:10:20,057][105692] Updated weights for policy 0, policy_version 1480214 (0.0008) [2023-12-27 02:10:20,107][105620] Updated weights for policy 1, policy_version 1482548 (0.0006) [2023-12-27 02:10:20,113][105692] Updated weights for policy 0, policy_version 1480224 (0.0008) [2023-12-27 02:10:20,157][105620] Updated weights for policy 1, policy_version 1482558 (0.0007) [2023-12-27 02:10:20,163][105692] Updated weights for policy 0, policy_version 1480234 (0.0008) [2023-12-27 02:10:20,208][105620] Updated weights for policy 1, policy_version 1482568 (0.0007) [2023-12-27 02:10:20,823][105692] Updated weights for policy 0, policy_version 1480244 (0.0008) [2023-12-27 02:10:20,892][105692] Updated weights for policy 0, policy_version 1480254 (0.0008) [2023-12-27 02:10:20,955][105692] Updated weights for policy 0, policy_version 1480264 (0.0009) [2023-12-27 02:10:21,033][105620] Updated weights for policy 1, policy_version 1482578 (0.0009) [2023-12-27 02:10:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 758595584. Throughput: 0: 9621.8, 1: 9798.5. Samples: 758584348. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:10:21,062][104569] Avg episode reward: [(0, '8441.650'), (1, '9174.647')] [2023-12-27 02:10:21,098][105620] Updated weights for policy 1, policy_version 1482588 (0.0009) [2023-12-27 02:10:21,168][105620] Updated weights for policy 1, policy_version 1482598 (0.0008) [2023-12-27 02:10:21,232][105620] Updated weights for policy 1, policy_version 1482608 (0.0009) [2023-12-27 02:10:21,717][105692] Updated weights for policy 0, policy_version 1480274 (0.0008) [2023-12-27 02:10:21,786][105692] Updated weights for policy 0, policy_version 1480284 (0.0008) [2023-12-27 02:10:21,842][105692] Updated weights for policy 0, policy_version 1480294 (0.0008) [2023-12-27 02:10:21,902][105692] Updated weights for policy 0, policy_version 1480304 (0.0009) [2023-12-27 02:10:22,054][105620] Updated weights for policy 1, policy_version 1482618 (0.0011) [2023-12-27 02:10:22,118][105620] Updated weights for policy 1, policy_version 1482628 (0.0011) [2023-12-27 02:10:22,181][105620] Updated weights for policy 1, policy_version 1482638 (0.0011) [2023-12-27 02:10:22,701][105692] Updated weights for policy 0, policy_version 1480314 (0.0007) [2023-12-27 02:10:22,765][105692] Updated weights for policy 0, policy_version 1480324 (0.0008) [2023-12-27 02:10:22,834][105692] Updated weights for policy 0, policy_version 1480334 (0.0008) [2023-12-27 02:10:22,950][105620] Updated weights for policy 1, policy_version 1482648 (0.0010) [2023-12-27 02:10:23,002][105620] Updated weights for policy 1, policy_version 1482658 (0.0009) [2023-12-27 02:10:23,055][105620] Updated weights for policy 1, policy_version 1482668 (0.0007) [2023-12-27 02:10:23,464][105692] Updated weights for policy 0, policy_version 1480344 (0.0010) [2023-12-27 02:10:23,522][105692] Updated weights for policy 0, policy_version 1480355 (0.0010) [2023-12-27 02:10:23,579][105692] Updated weights for policy 0, policy_version 1480365 (0.0009) [2023-12-27 02:10:23,705][105620] Updated weights for policy 1, policy_version 1482678 (0.0006) [2023-12-27 02:10:23,762][105620] Updated weights for policy 1, policy_version 1482688 (0.0008) [2023-12-27 02:10:23,818][105620] Updated weights for policy 1, policy_version 1482698 (0.0007) [2023-12-27 02:10:24,314][105692] Updated weights for policy 0, policy_version 1480375 (0.0009) [2023-12-27 02:10:24,372][105692] Updated weights for policy 0, policy_version 1480385 (0.0009) [2023-12-27 02:10:24,431][105692] Updated weights for policy 0, policy_version 1480395 (0.0009) [2023-12-27 02:10:24,496][105620] Updated weights for policy 1, policy_version 1482708 (0.0008) [2023-12-27 02:10:24,561][105620] Updated weights for policy 1, policy_version 1482718 (0.0009) [2023-12-27 02:10:24,630][105620] Updated weights for policy 1, policy_version 1482728 (0.0009) [2023-12-27 02:10:25,081][105692] Updated weights for policy 0, policy_version 1480405 (0.0007) [2023-12-27 02:10:25,142][105692] Updated weights for policy 0, policy_version 1480415 (0.0005) [2023-12-27 02:10:25,193][105692] Updated weights for policy 0, policy_version 1480425 (0.0009) [2023-12-27 02:10:25,430][105620] Updated weights for policy 1, policy_version 1482738 (0.0009) [2023-12-27 02:10:25,479][105620] Updated weights for policy 1, policy_version 1482748 (0.0008) [2023-12-27 02:10:25,540][105620] Updated weights for policy 1, policy_version 1482758 (0.0009) [2023-12-27 02:10:25,590][105620] Updated weights for policy 1, policy_version 1482768 (0.0008) [2023-12-27 02:10:25,836][105692] Updated weights for policy 0, policy_version 1480435 (0.0009) [2023-12-27 02:10:25,882][105692] Updated weights for policy 0, policy_version 1480445 (0.0008) [2023-12-27 02:10:25,941][105692] Updated weights for policy 0, policy_version 1480455 (0.0005) [2023-12-27 02:10:26,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 758693888. Throughput: 0: 9613.0, 1: 9807.3. Samples: 758698264. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:10:26,062][104569] Avg episode reward: [(0, '8713.902'), (1, '9263.622')] [2023-12-27 02:10:26,403][105620] Updated weights for policy 1, policy_version 1482778 (0.0005) [2023-12-27 02:10:26,448][105620] Updated weights for policy 1, policy_version 1482788 (0.0005) [2023-12-27 02:10:26,500][105620] Updated weights for policy 1, policy_version 1482798 (0.0005) [2023-12-27 02:10:26,524][105692] Updated weights for policy 0, policy_version 1480465 (0.0005) [2023-12-27 02:10:26,594][105692] Updated weights for policy 0, policy_version 1480475 (0.0006) [2023-12-27 02:10:26,659][105692] Updated weights for policy 0, policy_version 1480485 (0.0006) [2023-12-27 02:10:26,721][105692] Updated weights for policy 0, policy_version 1480495 (0.0005) [2023-12-27 02:10:27,055][105620] Updated weights for policy 1, policy_version 1482808 (0.0005) [2023-12-27 02:10:27,123][105620] Updated weights for policy 1, policy_version 1482818 (0.0005) [2023-12-27 02:10:27,189][105620] Updated weights for policy 1, policy_version 1482828 (0.0005) [2023-12-27 02:10:27,232][105692] Updated weights for policy 0, policy_version 1480505 (0.0005) [2023-12-27 02:10:27,297][105692] Updated weights for policy 0, policy_version 1480515 (0.0006) [2023-12-27 02:10:27,353][105692] Updated weights for policy 0, policy_version 1480525 (0.0008) [2023-12-27 02:10:27,748][105620] Updated weights for policy 1, policy_version 1482838 (0.0007) [2023-12-27 02:10:27,801][105620] Updated weights for policy 1, policy_version 1482848 (0.0005) [2023-12-27 02:10:27,849][105620] Updated weights for policy 1, policy_version 1482858 (0.0006) [2023-12-27 02:10:28,167][105692] Updated weights for policy 0, policy_version 1480535 (0.0009) [2023-12-27 02:10:28,218][105692] Updated weights for policy 0, policy_version 1480545 (0.0009) [2023-12-27 02:10:28,272][105692] Updated weights for policy 0, policy_version 1480555 (0.0009) [2023-12-27 02:10:28,456][105620] Updated weights for policy 1, policy_version 1482868 (0.0007) [2023-12-27 02:10:28,507][105620] Updated weights for policy 1, policy_version 1482878 (0.0010) [2023-12-27 02:10:28,562][105620] Updated weights for policy 1, policy_version 1482888 (0.0010) [2023-12-27 02:10:29,016][105692] Updated weights for policy 0, policy_version 1480565 (0.0010) [2023-12-27 02:10:29,072][105692] Updated weights for policy 0, policy_version 1480575 (0.0009) [2023-12-27 02:10:29,135][105692] Updated weights for policy 0, policy_version 1480585 (0.0010) [2023-12-27 02:10:29,234][105620] Updated weights for policy 1, policy_version 1482898 (0.0010) [2023-12-27 02:10:29,294][105620] Updated weights for policy 1, policy_version 1482908 (0.0006) [2023-12-27 02:10:29,366][105620] Updated weights for policy 1, policy_version 1482918 (0.0010) [2023-12-27 02:10:29,837][105692] Updated weights for policy 0, policy_version 1480596 (0.0010) [2023-12-27 02:10:29,899][105692] Updated weights for policy 0, policy_version 1480606 (0.0009) [2023-12-27 02:10:29,958][105692] Updated weights for policy 0, policy_version 1480616 (0.0009) [2023-12-27 02:10:30,122][105620] Updated weights for policy 1, policy_version 1482929 (0.0010) [2023-12-27 02:10:30,176][105620] Updated weights for policy 1, policy_version 1482939 (0.0009) [2023-12-27 02:10:30,230][105620] Updated weights for policy 1, policy_version 1482949 (0.0009) [2023-12-27 02:10:30,275][105620] Updated weights for policy 1, policy_version 1482959 (0.0008) [2023-12-27 02:10:30,705][105692] Updated weights for policy 0, policy_version 1480626 (0.0009) [2023-12-27 02:10:30,755][105692] Updated weights for policy 0, policy_version 1480636 (0.0009) [2023-12-27 02:10:30,801][105692] Updated weights for policy 0, policy_version 1480646 (0.0009) [2023-12-27 02:10:30,855][105692] Updated weights for policy 0, policy_version 1480656 (0.0009) [2023-12-27 02:10:31,005][105620] Updated weights for policy 1, policy_version 1482969 (0.0006) [2023-12-27 02:10:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 758792192. Throughput: 0: 9690.5, 1: 9909.4. Samples: 758762776. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:10:31,062][104569] Avg episode reward: [(0, '8993.975'), (1, '9262.343')] [2023-12-27 02:10:31,064][105620] Updated weights for policy 1, policy_version 1482979 (0.0008) [2023-12-27 02:10:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001480656_379101184.pth... [2023-12-27 02:10:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001479504_378806272.pth [2023-12-27 02:10:31,132][105620] Updated weights for policy 1, policy_version 1482989 (0.0009) [2023-12-27 02:10:31,148][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001482992_379699200.pth... [2023-12-27 02:10:31,154][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001481840_379404288.pth [2023-12-27 02:10:31,633][105692] Updated weights for policy 0, policy_version 1480666 (0.0009) [2023-12-27 02:10:31,691][105692] Updated weights for policy 0, policy_version 1480676 (0.0010) [2023-12-27 02:10:31,757][105692] Updated weights for policy 0, policy_version 1480686 (0.0009) [2023-12-27 02:10:31,786][105620] Updated weights for policy 1, policy_version 1482999 (0.0007) [2023-12-27 02:10:31,846][105620] Updated weights for policy 1, policy_version 1483009 (0.0008) [2023-12-27 02:10:31,913][105620] Updated weights for policy 1, policy_version 1483019 (0.0008) [2023-12-27 02:10:32,521][105620] Updated weights for policy 1, policy_version 1483029 (0.0010) [2023-12-27 02:10:32,575][105620] Updated weights for policy 1, policy_version 1483039 (0.0008) [2023-12-27 02:10:32,589][105692] Updated weights for policy 0, policy_version 1480696 (0.0007) [2023-12-27 02:10:32,625][105620] Updated weights for policy 1, policy_version 1483049 (0.0008) [2023-12-27 02:10:32,652][105692] Updated weights for policy 0, policy_version 1480706 (0.0008) [2023-12-27 02:10:32,702][105692] Updated weights for policy 0, policy_version 1480716 (0.0007) [2023-12-27 02:10:33,301][105620] Updated weights for policy 1, policy_version 1483059 (0.0007) [2023-12-27 02:10:33,361][105692] Updated weights for policy 0, policy_version 1480726 (0.0007) [2023-12-27 02:10:33,373][105620] Updated weights for policy 1, policy_version 1483069 (0.0006) [2023-12-27 02:10:33,417][105692] Updated weights for policy 0, policy_version 1480736 (0.0005) [2023-12-27 02:10:33,432][105620] Updated weights for policy 1, policy_version 1483079 (0.0008) [2023-12-27 02:10:33,463][105692] Updated weights for policy 0, policy_version 1480746 (0.0005) [2023-12-27 02:10:33,976][105692] Updated weights for policy 0, policy_version 1480756 (0.0007) [2023-12-27 02:10:34,005][105620] Updated weights for policy 1, policy_version 1483089 (0.0007) [2023-12-27 02:10:34,029][105692] Updated weights for policy 0, policy_version 1480766 (0.0009) [2023-12-27 02:10:34,051][105620] Updated weights for policy 1, policy_version 1483099 (0.0007) [2023-12-27 02:10:34,080][105585] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000010 [2023-12-27 02:10:34,081][105692] Updated weights for policy 0, policy_version 1480776 (0.0008) [2023-12-27 02:10:34,097][105620] Updated weights for policy 1, policy_version 1483109 (0.0007) [2023-12-27 02:10:34,150][105620] Updated weights for policy 1, policy_version 1483119 (0.0008) [2023-12-27 02:10:34,841][105620] Updated weights for policy 1, policy_version 1483129 (0.0008) [2023-12-27 02:10:34,845][105692] Updated weights for policy 0, policy_version 1480786 (0.0006) [2023-12-27 02:10:34,900][105692] Updated weights for policy 0, policy_version 1480796 (0.0006) [2023-12-27 02:10:34,902][105620] Updated weights for policy 1, policy_version 1483139 (0.0008) [2023-12-27 02:10:34,961][105692] Updated weights for policy 0, policy_version 1480806 (0.0007) [2023-12-27 02:10:34,962][105620] Updated weights for policy 1, policy_version 1483149 (0.0008) [2023-12-27 02:10:35,526][105692] Updated weights for policy 0, policy_version 1480816 (0.0006) [2023-12-27 02:10:35,588][105692] Updated weights for policy 0, policy_version 1480826 (0.0008) [2023-12-27 02:10:35,650][105692] Updated weights for policy 0, policy_version 1480836 (0.0009) [2023-12-27 02:10:35,799][105620] Updated weights for policy 1, policy_version 1483159 (0.0009) [2023-12-27 02:10:35,853][105620] Updated weights for policy 1, policy_version 1483169 (0.0008) [2023-12-27 02:10:35,914][105620] Updated weights for policy 1, policy_version 1483179 (0.0010) [2023-12-27 02:10:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 758898688. Throughput: 0: 9661.2, 1: 9934.6. Samples: 758883812. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:10:36,062][104569] Avg episode reward: [(0, '8632.573'), (1, '9352.314')] [2023-12-27 02:10:36,375][105692] Updated weights for policy 0, policy_version 1480846 (0.0009) [2023-12-27 02:10:36,431][105692] Updated weights for policy 0, policy_version 1480856 (0.0009) [2023-12-27 02:10:36,494][105692] Updated weights for policy 0, policy_version 1480866 (0.0009) [2023-12-27 02:10:36,628][105620] Updated weights for policy 1, policy_version 1483189 (0.0009) [2023-12-27 02:10:36,693][105620] Updated weights for policy 1, policy_version 1483199 (0.0008) [2023-12-27 02:10:36,751][105620] Updated weights for policy 1, policy_version 1483209 (0.0009) [2023-12-27 02:10:37,305][105692] Updated weights for policy 0, policy_version 1480876 (0.0009) [2023-12-27 02:10:37,367][105692] Updated weights for policy 0, policy_version 1480886 (0.0008) [2023-12-27 02:10:37,388][105620] Updated weights for policy 1, policy_version 1483219 (0.0009) [2023-12-27 02:10:37,434][105692] Updated weights for policy 0, policy_version 1480896 (0.0008) [2023-12-27 02:10:37,440][105620] Updated weights for policy 1, policy_version 1483229 (0.0010) [2023-12-27 02:10:37,498][105620] Updated weights for policy 1, policy_version 1483239 (0.0010) [2023-12-27 02:10:38,094][105692] Updated weights for policy 0, policy_version 1480906 (0.0008) [2023-12-27 02:10:38,158][105692] Updated weights for policy 0, policy_version 1480916 (0.0009) [2023-12-27 02:10:38,222][105692] Updated weights for policy 0, policy_version 1480926 (0.0008) [2023-12-27 02:10:38,242][105620] Updated weights for policy 1, policy_version 1483249 (0.0010) [2023-12-27 02:10:38,279][105692] Updated weights for policy 0, policy_version 1480936 (0.0008) [2023-12-27 02:10:38,300][105620] Updated weights for policy 1, policy_version 1483259 (0.0010) [2023-12-27 02:10:38,359][105620] Updated weights for policy 1, policy_version 1483269 (0.0009) [2023-12-27 02:10:38,411][105620] Updated weights for policy 1, policy_version 1483279 (0.0009) [2023-12-27 02:10:38,902][105692] Updated weights for policy 0, policy_version 1480946 (0.0007) [2023-12-27 02:10:38,962][105692] Updated weights for policy 0, policy_version 1480956 (0.0006) [2023-12-27 02:10:39,022][105692] Updated weights for policy 0, policy_version 1480966 (0.0007) [2023-12-27 02:10:39,125][105620] Updated weights for policy 1, policy_version 1483289 (0.0010) [2023-12-27 02:10:39,188][105620] Updated weights for policy 1, policy_version 1483299 (0.0011) [2023-12-27 02:10:39,256][105620] Updated weights for policy 1, policy_version 1483309 (0.0011) [2023-12-27 02:10:39,716][105692] Updated weights for policy 0, policy_version 1480976 (0.0009) [2023-12-27 02:10:39,774][105692] Updated weights for policy 0, policy_version 1480986 (0.0010) [2023-12-27 02:10:39,838][105692] Updated weights for policy 0, policy_version 1480996 (0.0008) [2023-12-27 02:10:39,980][105620] Updated weights for policy 1, policy_version 1483319 (0.0011) [2023-12-27 02:10:40,037][105620] Updated weights for policy 1, policy_version 1483329 (0.0011) [2023-12-27 02:10:40,093][105620] Updated weights for policy 1, policy_version 1483339 (0.0010) [2023-12-27 02:10:40,644][105692] Updated weights for policy 0, policy_version 1481006 (0.0008) [2023-12-27 02:10:40,714][105692] Updated weights for policy 0, policy_version 1481016 (0.0008) [2023-12-27 02:10:40,751][105620] Updated weights for policy 1, policy_version 1483349 (0.0011) [2023-12-27 02:10:40,774][105692] Updated weights for policy 0, policy_version 1481026 (0.0008) [2023-12-27 02:10:40,807][105620] Updated weights for policy 1, policy_version 1483359 (0.0010) [2023-12-27 02:10:40,853][105620] Updated weights for policy 1, policy_version 1483369 (0.0010) [2023-12-27 02:10:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 758996992. Throughput: 0: 9780.7, 1: 9907.3. Samples: 759001224. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 02:10:41,062][104569] Avg episode reward: [(0, '8627.961'), (1, '9168.862')] [2023-12-27 02:10:41,569][105692] Updated weights for policy 0, policy_version 1481036 (0.0008) [2023-12-27 02:10:41,623][105692] Updated weights for policy 0, policy_version 1481046 (0.0009) [2023-12-27 02:10:41,682][105620] Updated weights for policy 1, policy_version 1483379 (0.0010) [2023-12-27 02:10:41,684][105692] Updated weights for policy 0, policy_version 1481056 (0.0008) [2023-12-27 02:10:41,749][105620] Updated weights for policy 1, policy_version 1483389 (0.0011) [2023-12-27 02:10:41,809][105620] Updated weights for policy 1, policy_version 1483399 (0.0009) [2023-12-27 02:10:42,479][105620] Updated weights for policy 1, policy_version 1483409 (0.0010) [2023-12-27 02:10:42,543][105620] Updated weights for policy 1, policy_version 1483419 (0.0009) [2023-12-27 02:10:42,562][105692] Updated weights for policy 0, policy_version 1481066 (0.0007) [2023-12-27 02:10:42,602][105620] Updated weights for policy 1, policy_version 1483429 (0.0010) [2023-12-27 02:10:42,621][105692] Updated weights for policy 0, policy_version 1481076 (0.0009) [2023-12-27 02:10:42,664][105620] Updated weights for policy 1, policy_version 1483439 (0.0006) [2023-12-27 02:10:42,682][105692] Updated weights for policy 0, policy_version 1481086 (0.0008) [2023-12-27 02:10:42,742][105692] Updated weights for policy 0, policy_version 1481096 (0.0008) [2023-12-27 02:10:43,389][105620] Updated weights for policy 1, policy_version 1483449 (0.0009) [2023-12-27 02:10:43,440][105620] Updated weights for policy 1, policy_version 1483459 (0.0007) [2023-12-27 02:10:43,475][105692] Updated weights for policy 0, policy_version 1481106 (0.0007) [2023-12-27 02:10:43,501][105620] Updated weights for policy 1, policy_version 1483469 (0.0007) [2023-12-27 02:10:43,539][105692] Updated weights for policy 0, policy_version 1481116 (0.0008) [2023-12-27 02:10:43,603][105692] Updated weights for policy 0, policy_version 1481126 (0.0006) [2023-12-27 02:10:44,219][105692] Updated weights for policy 0, policy_version 1481136 (0.0008) [2023-12-27 02:10:44,273][105692] Updated weights for policy 0, policy_version 1481146 (0.0008) [2023-12-27 02:10:44,298][105620] Updated weights for policy 1, policy_version 1483479 (0.0010) [2023-12-27 02:10:44,331][105692] Updated weights for policy 0, policy_version 1481156 (0.0006) [2023-12-27 02:10:44,356][105620] Updated weights for policy 1, policy_version 1483489 (0.0010) [2023-12-27 02:10:44,408][105620] Updated weights for policy 1, policy_version 1483499 (0.0010) [2023-12-27 02:10:45,077][105692] Updated weights for policy 0, policy_version 1481166 (0.0008) [2023-12-27 02:10:45,127][105620] Updated weights for policy 1, policy_version 1483509 (0.0008) [2023-12-27 02:10:45,138][105692] Updated weights for policy 0, policy_version 1481176 (0.0009) [2023-12-27 02:10:45,191][105620] Updated weights for policy 1, policy_version 1483519 (0.0007) [2023-12-27 02:10:45,198][105692] Updated weights for policy 0, policy_version 1481186 (0.0006) [2023-12-27 02:10:45,254][105620] Updated weights for policy 1, policy_version 1483529 (0.0009) [2023-12-27 02:10:45,917][105692] Updated weights for policy 0, policy_version 1481196 (0.0010) [2023-12-27 02:10:45,978][105692] Updated weights for policy 0, policy_version 1481206 (0.0008) [2023-12-27 02:10:46,005][105620] Updated weights for policy 1, policy_version 1483539 (0.0008) [2023-12-27 02:10:46,040][105692] Updated weights for policy 0, policy_version 1481216 (0.0009) [2023-12-27 02:10:46,061][105620] Updated weights for policy 1, policy_version 1483549 (0.0006) [2023-12-27 02:10:46,062][104569] Fps is (10 sec: 18022.1, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 759078912. Throughput: 0: 9775.7, 1: 9877.2. Samples: 759056200. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:10:46,063][104569] Avg episode reward: [(0, '8900.017'), (1, '9170.199')] [2023-12-27 02:10:46,084][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001481224_379248640.pth... [2023-12-27 02:10:46,088][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001480048_378945536.pth [2023-12-27 02:10:46,118][105620] Updated weights for policy 1, policy_version 1483559 (0.0006) [2023-12-27 02:10:46,175][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001483568_379846656.pth... [2023-12-27 02:10:46,179][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001482384_379543552.pth [2023-12-27 02:10:46,659][105620] Updated weights for policy 1, policy_version 1483569 (0.0005) [2023-12-27 02:10:46,713][105620] Updated weights for policy 1, policy_version 1483579 (0.0005) [2023-12-27 02:10:46,765][105620] Updated weights for policy 1, policy_version 1483589 (0.0005) [2023-12-27 02:10:46,814][105620] Updated weights for policy 1, policy_version 1483599 (0.0005) [2023-12-27 02:10:46,872][105692] Updated weights for policy 0, policy_version 1481226 (0.0008) [2023-12-27 02:10:46,919][105692] Updated weights for policy 0, policy_version 1481236 (0.0009) [2023-12-27 02:10:46,972][105692] Updated weights for policy 0, policy_version 1481246 (0.0009) [2023-12-27 02:10:47,024][105692] Updated weights for policy 0, policy_version 1481256 (0.0010) [2023-12-27 02:10:47,358][105620] Updated weights for policy 1, policy_version 1483609 (0.0009) [2023-12-27 02:10:47,419][105620] Updated weights for policy 1, policy_version 1483619 (0.0009) [2023-12-27 02:10:47,485][105620] Updated weights for policy 1, policy_version 1483629 (0.0009) [2023-12-27 02:10:47,730][105692] Updated weights for policy 0, policy_version 1481266 (0.0010) [2023-12-27 02:10:47,775][105692] Updated weights for policy 0, policy_version 1481276 (0.0010) [2023-12-27 02:10:47,819][105692] Updated weights for policy 0, policy_version 1481286 (0.0007) [2023-12-27 02:10:48,214][105620] Updated weights for policy 1, policy_version 1483639 (0.0011) [2023-12-27 02:10:48,267][105620] Updated weights for policy 1, policy_version 1483649 (0.0007) [2023-12-27 02:10:48,336][105620] Updated weights for policy 1, policy_version 1483659 (0.0006) [2023-12-27 02:10:48,638][105692] Updated weights for policy 0, policy_version 1481296 (0.0009) [2023-12-27 02:10:48,692][105692] Updated weights for policy 0, policy_version 1481306 (0.0009) [2023-12-27 02:10:48,745][105692] Updated weights for policy 0, policy_version 1481316 (0.0010) [2023-12-27 02:10:49,005][105620] Updated weights for policy 1, policy_version 1483669 (0.0009) [2023-12-27 02:10:49,056][105620] Updated weights for policy 1, policy_version 1483679 (0.0009) [2023-12-27 02:10:49,109][105620] Updated weights for policy 1, policy_version 1483689 (0.0011) [2023-12-27 02:10:49,552][105692] Updated weights for policy 0, policy_version 1481326 (0.0009) [2023-12-27 02:10:49,608][105692] Updated weights for policy 0, policy_version 1481336 (0.0009) [2023-12-27 02:10:49,662][105692] Updated weights for policy 0, policy_version 1481346 (0.0010) [2023-12-27 02:10:49,840][105620] Updated weights for policy 1, policy_version 1483699 (0.0010) [2023-12-27 02:10:49,900][105620] Updated weights for policy 1, policy_version 1483709 (0.0011) [2023-12-27 02:10:49,965][105620] Updated weights for policy 1, policy_version 1483719 (0.0009) [2023-12-27 02:10:50,518][105692] Updated weights for policy 0, policy_version 1481356 (0.0009) [2023-12-27 02:10:50,579][105692] Updated weights for policy 0, policy_version 1481366 (0.0009) [2023-12-27 02:10:50,609][105620] Updated weights for policy 1, policy_version 1483729 (0.0007) [2023-12-27 02:10:50,638][105692] Updated weights for policy 0, policy_version 1481376 (0.0008) [2023-12-27 02:10:50,674][105620] Updated weights for policy 1, policy_version 1483739 (0.0005) [2023-12-27 02:10:50,729][105620] Updated weights for policy 1, policy_version 1483749 (0.0005) [2023-12-27 02:10:50,786][105620] Updated weights for policy 1, policy_version 1483759 (0.0008) [2023-12-27 02:10:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 759185408. Throughput: 0: 9859.9, 1: 9915.4. Samples: 759174044. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:10:51,063][104569] Avg episode reward: [(0, '8536.787'), (1, '9355.237')] [2023-12-27 02:10:51,304][105692] Updated weights for policy 0, policy_version 1481386 (0.0006) [2023-12-27 02:10:51,366][105692] Updated weights for policy 0, policy_version 1481396 (0.0008) [2023-12-27 02:10:51,429][105692] Updated weights for policy 0, policy_version 1481406 (0.0008) [2023-12-27 02:10:51,480][105692] Updated weights for policy 0, policy_version 1481416 (0.0008) [2023-12-27 02:10:51,487][105620] Updated weights for policy 1, policy_version 1483769 (0.0006) [2023-12-27 02:10:51,552][105620] Updated weights for policy 1, policy_version 1483779 (0.0008) [2023-12-27 02:10:51,622][105620] Updated weights for policy 1, policy_version 1483789 (0.0010) [2023-12-27 02:10:52,162][105692] Updated weights for policy 0, policy_version 1481426 (0.0009) [2023-12-27 02:10:52,213][105692] Updated weights for policy 0, policy_version 1481436 (0.0008) [2023-12-27 02:10:52,268][105692] Updated weights for policy 0, policy_version 1481446 (0.0009) [2023-12-27 02:10:52,410][105620] Updated weights for policy 1, policy_version 1483799 (0.0008) [2023-12-27 02:10:52,470][105620] Updated weights for policy 1, policy_version 1483809 (0.0009) [2023-12-27 02:10:52,531][105620] Updated weights for policy 1, policy_version 1483819 (0.0009) [2023-12-27 02:10:52,987][105692] Updated weights for policy 0, policy_version 1481456 (0.0006) [2023-12-27 02:10:53,044][105692] Updated weights for policy 0, policy_version 1481466 (0.0007) [2023-12-27 02:10:53,096][105692] Updated weights for policy 0, policy_version 1481476 (0.0009) [2023-12-27 02:10:53,301][105620] Updated weights for policy 1, policy_version 1483829 (0.0007) [2023-12-27 02:10:53,363][105620] Updated weights for policy 1, policy_version 1483839 (0.0007) [2023-12-27 02:10:53,424][105620] Updated weights for policy 1, policy_version 1483849 (0.0009) [2023-12-27 02:10:53,811][105692] Updated weights for policy 0, policy_version 1481486 (0.0009) [2023-12-27 02:10:53,858][105692] Updated weights for policy 0, policy_version 1481496 (0.0009) [2023-12-27 02:10:53,905][105692] Updated weights for policy 0, policy_version 1481506 (0.0009) [2023-12-27 02:10:54,137][105620] Updated weights for policy 1, policy_version 1483859 (0.0008) [2023-12-27 02:10:54,187][105620] Updated weights for policy 1, policy_version 1483869 (0.0008) [2023-12-27 02:10:54,242][105620] Updated weights for policy 1, policy_version 1483879 (0.0006) [2023-12-27 02:10:54,708][105692] Updated weights for policy 0, policy_version 1481516 (0.0009) [2023-12-27 02:10:54,760][105692] Updated weights for policy 0, policy_version 1481526 (0.0009) [2023-12-27 02:10:54,808][105692] Updated weights for policy 0, policy_version 1481536 (0.0009) [2023-12-27 02:10:54,955][105620] Updated weights for policy 1, policy_version 1483889 (0.0005) [2023-12-27 02:10:55,021][105620] Updated weights for policy 1, policy_version 1483899 (0.0009) [2023-12-27 02:10:55,080][105620] Updated weights for policy 1, policy_version 1483909 (0.0009) [2023-12-27 02:10:55,140][105620] Updated weights for policy 1, policy_version 1483919 (0.0009) [2023-12-27 02:10:55,482][105692] Updated weights for policy 0, policy_version 1481546 (0.0009) [2023-12-27 02:10:55,538][105692] Updated weights for policy 0, policy_version 1481556 (0.0005) [2023-12-27 02:10:55,601][105692] Updated weights for policy 0, policy_version 1481566 (0.0008) [2023-12-27 02:10:55,659][105692] Updated weights for policy 0, policy_version 1481576 (0.0010) [2023-12-27 02:10:55,861][105620] Updated weights for policy 1, policy_version 1483929 (0.0009) [2023-12-27 02:10:55,918][105620] Updated weights for policy 1, policy_version 1483939 (0.0009) [2023-12-27 02:10:55,976][105620] Updated weights for policy 1, policy_version 1483949 (0.0009) [2023-12-27 02:10:56,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 759283712. Throughput: 0: 9821.5, 1: 9946.1. Samples: 759289420. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:10:56,062][104569] Avg episode reward: [(0, '8261.164'), (1, '9355.092')] [2023-12-27 02:10:56,346][105692] Updated weights for policy 0, policy_version 1481586 (0.0009) [2023-12-27 02:10:56,416][105692] Updated weights for policy 0, policy_version 1481596 (0.0009) [2023-12-27 02:10:56,478][105692] Updated weights for policy 0, policy_version 1481606 (0.0009) [2023-12-27 02:10:56,722][105620] Updated weights for policy 1, policy_version 1483959 (0.0009) [2023-12-27 02:10:56,787][105620] Updated weights for policy 1, policy_version 1483969 (0.0009) [2023-12-27 02:10:56,852][105620] Updated weights for policy 1, policy_version 1483979 (0.0010) [2023-12-27 02:10:57,128][105692] Updated weights for policy 0, policy_version 1481616 (0.0009) [2023-12-27 02:10:57,182][105692] Updated weights for policy 0, policy_version 1481626 (0.0009) [2023-12-27 02:10:57,239][105692] Updated weights for policy 0, policy_version 1481636 (0.0009) [2023-12-27 02:10:57,614][105620] Updated weights for policy 1, policy_version 1483989 (0.0009) [2023-12-27 02:10:57,660][105620] Updated weights for policy 1, policy_version 1483999 (0.0009) [2023-12-27 02:10:57,706][105620] Updated weights for policy 1, policy_version 1484009 (0.0008) [2023-12-27 02:10:57,973][105692] Updated weights for policy 0, policy_version 1481646 (0.0009) [2023-12-27 02:10:58,037][105692] Updated weights for policy 0, policy_version 1481656 (0.0005) [2023-12-27 02:10:58,090][105692] Updated weights for policy 0, policy_version 1481666 (0.0005) [2023-12-27 02:10:58,520][105620] Updated weights for policy 1, policy_version 1484019 (0.0009) [2023-12-27 02:10:58,579][105620] Updated weights for policy 1, policy_version 1484029 (0.0009) [2023-12-27 02:10:58,644][105620] Updated weights for policy 1, policy_version 1484039 (0.0010) [2023-12-27 02:10:58,791][105692] Updated weights for policy 0, policy_version 1481676 (0.0008) [2023-12-27 02:10:58,854][105692] Updated weights for policy 0, policy_version 1481686 (0.0009) [2023-12-27 02:10:58,919][105692] Updated weights for policy 0, policy_version 1481696 (0.0008) [2023-12-27 02:10:59,451][105620] Updated weights for policy 1, policy_version 1484049 (0.0009) [2023-12-27 02:10:59,508][105620] Updated weights for policy 1, policy_version 1484059 (0.0010) [2023-12-27 02:10:59,566][105620] Updated weights for policy 1, policy_version 1484069 (0.0010) [2023-12-27 02:10:59,607][105692] Updated weights for policy 0, policy_version 1481706 (0.0009) [2023-12-27 02:10:59,622][105620] Updated weights for policy 1, policy_version 1484079 (0.0006) [2023-12-27 02:10:59,668][105692] Updated weights for policy 0, policy_version 1481716 (0.0010) [2023-12-27 02:10:59,730][105692] Updated weights for policy 0, policy_version 1481726 (0.0010) [2023-12-27 02:10:59,795][105692] Updated weights for policy 0, policy_version 1481736 (0.0009) [2023-12-27 02:11:00,343][105620] Updated weights for policy 1, policy_version 1484089 (0.0009) [2023-12-27 02:11:00,392][105620] Updated weights for policy 1, policy_version 1484099 (0.0008) [2023-12-27 02:11:00,456][105620] Updated weights for policy 1, policy_version 1484109 (0.0005) [2023-12-27 02:11:00,491][105692] Updated weights for policy 0, policy_version 1481746 (0.0009) [2023-12-27 02:11:00,557][105692] Updated weights for policy 0, policy_version 1481756 (0.0011) [2023-12-27 02:11:00,622][105692] Updated weights for policy 0, policy_version 1481766 (0.0010) [2023-12-27 02:11:01,002][105620] Updated weights for policy 1, policy_version 1484119 (0.0005) [2023-12-27 02:11:01,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 759373824. Throughput: 0: 9805.7, 1: 9869.4. Samples: 759345876. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:11:01,062][104569] Avg episode reward: [(0, '8346.618'), (1, '9264.857')] [2023-12-27 02:11:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001481768_379387904.pth... [2023-12-27 02:11:01,068][105620] Updated weights for policy 1, policy_version 1484129 (0.0006) [2023-12-27 02:11:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001480656_379101184.pth [2023-12-27 02:11:01,138][105620] Updated weights for policy 1, policy_version 1484139 (0.0007) [2023-12-27 02:11:01,161][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001484144_379994112.pth... [2023-12-27 02:11:01,164][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001482992_379699200.pth [2023-12-27 02:11:01,406][105692] Updated weights for policy 0, policy_version 1481776 (0.0009) [2023-12-27 02:11:01,466][105692] Updated weights for policy 0, policy_version 1481786 (0.0007) [2023-12-27 02:11:01,528][105692] Updated weights for policy 0, policy_version 1481796 (0.0009) [2023-12-27 02:11:01,831][105620] Updated weights for policy 1, policy_version 1484149 (0.0009) [2023-12-27 02:11:01,886][105620] Updated weights for policy 1, policy_version 1484159 (0.0010) [2023-12-27 02:11:01,931][105620] Updated weights for policy 1, policy_version 1484169 (0.0010) [2023-12-27 02:11:02,294][105692] Updated weights for policy 0, policy_version 1481806 (0.0009) [2023-12-27 02:11:02,346][105692] Updated weights for policy 0, policy_version 1481816 (0.0008) [2023-12-27 02:11:02,402][105692] Updated weights for policy 0, policy_version 1481826 (0.0009) [2023-12-27 02:11:02,606][105620] Updated weights for policy 1, policy_version 1484179 (0.0010) [2023-12-27 02:11:02,656][105620] Updated weights for policy 1, policy_version 1484189 (0.0008) [2023-12-27 02:11:02,713][105620] Updated weights for policy 1, policy_version 1484199 (0.0009) [2023-12-27 02:11:03,156][105692] Updated weights for policy 0, policy_version 1481836 (0.0010) [2023-12-27 02:11:03,219][105692] Updated weights for policy 0, policy_version 1481846 (0.0008) [2023-12-27 02:11:03,279][105692] Updated weights for policy 0, policy_version 1481856 (0.0007) [2023-12-27 02:11:03,377][105620] Updated weights for policy 1, policy_version 1484209 (0.0008) [2023-12-27 02:11:03,430][105620] Updated weights for policy 1, policy_version 1484219 (0.0006) [2023-12-27 02:11:03,473][105620] Updated weights for policy 1, policy_version 1484229 (0.0005) [2023-12-27 02:11:03,520][105620] Updated weights for policy 1, policy_version 1484239 (0.0005) [2023-12-27 02:11:04,026][105692] Updated weights for policy 0, policy_version 1481866 (0.0009) [2023-12-27 02:11:04,083][105692] Updated weights for policy 0, policy_version 1481876 (0.0010) [2023-12-27 02:11:04,103][105620] Updated weights for policy 1, policy_version 1484249 (0.0007) [2023-12-27 02:11:04,143][105692] Updated weights for policy 0, policy_version 1481886 (0.0008) [2023-12-27 02:11:04,170][105620] Updated weights for policy 1, policy_version 1484259 (0.0009) [2023-12-27 02:11:04,201][105692] Updated weights for policy 0, policy_version 1481896 (0.0006) [2023-12-27 02:11:04,233][105620] Updated weights for policy 1, policy_version 1484269 (0.0008) [2023-12-27 02:11:04,924][105620] Updated weights for policy 1, policy_version 1484279 (0.0006) [2023-12-27 02:11:04,930][105692] Updated weights for policy 0, policy_version 1481906 (0.0005) [2023-12-27 02:11:04,977][105692] Updated weights for policy 0, policy_version 1481916 (0.0005) [2023-12-27 02:11:04,984][105620] Updated weights for policy 1, policy_version 1484289 (0.0005) [2023-12-27 02:11:05,032][105692] Updated weights for policy 0, policy_version 1481926 (0.0006) [2023-12-27 02:11:05,037][105620] Updated weights for policy 1, policy_version 1484299 (0.0007) [2023-12-27 02:11:05,681][105620] Updated weights for policy 1, policy_version 1484309 (0.0007) [2023-12-27 02:11:05,699][105692] Updated weights for policy 0, policy_version 1481936 (0.0006) [2023-12-27 02:11:05,736][105620] Updated weights for policy 1, policy_version 1484319 (0.0008) [2023-12-27 02:11:05,760][105692] Updated weights for policy 0, policy_version 1481946 (0.0005) [2023-12-27 02:11:05,804][105620] Updated weights for policy 1, policy_version 1484329 (0.0008) [2023-12-27 02:11:05,824][105692] Updated weights for policy 0, policy_version 1481956 (0.0005) [2023-12-27 02:11:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 759480320. Throughput: 0: 9659.8, 1: 9891.7. Samples: 759464168. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:11:06,062][104569] Avg episode reward: [(0, '8619.876'), (1, '9264.732')] [2023-12-27 02:11:06,527][105620] Updated weights for policy 1, policy_version 1484339 (0.0009) [2023-12-27 02:11:06,553][105692] Updated weights for policy 0, policy_version 1481966 (0.0006) [2023-12-27 02:11:06,576][105620] Updated weights for policy 1, policy_version 1484349 (0.0010) [2023-12-27 02:11:06,615][105692] Updated weights for policy 0, policy_version 1481976 (0.0006) [2023-12-27 02:11:06,631][105620] Updated weights for policy 1, policy_version 1484359 (0.0010) [2023-12-27 02:11:06,676][105692] Updated weights for policy 0, policy_version 1481986 (0.0010) [2023-12-27 02:11:07,352][105620] Updated weights for policy 1, policy_version 1484369 (0.0010) [2023-12-27 02:11:07,409][105620] Updated weights for policy 1, policy_version 1484379 (0.0005) [2023-12-27 02:11:07,469][105620] Updated weights for policy 1, policy_version 1484389 (0.0009) [2023-12-27 02:11:07,495][105692] Updated weights for policy 0, policy_version 1481996 (0.0008) [2023-12-27 02:11:07,522][105620] Updated weights for policy 1, policy_version 1484399 (0.0009) [2023-12-27 02:11:07,543][105692] Updated weights for policy 0, policy_version 1482006 (0.0009) [2023-12-27 02:11:07,587][105692] Updated weights for policy 0, policy_version 1482016 (0.0006) [2023-12-27 02:11:08,237][105620] Updated weights for policy 1, policy_version 1484409 (0.0010) [2023-12-27 02:11:08,295][105620] Updated weights for policy 1, policy_version 1484419 (0.0010) [2023-12-27 02:11:08,353][105620] Updated weights for policy 1, policy_version 1484429 (0.0010) [2023-12-27 02:11:08,420][105692] Updated weights for policy 0, policy_version 1482026 (0.0008) [2023-12-27 02:11:08,484][105692] Updated weights for policy 0, policy_version 1482036 (0.0008) [2023-12-27 02:11:08,539][105692] Updated weights for policy 0, policy_version 1482046 (0.0008) [2023-12-27 02:11:08,585][105692] Updated weights for policy 0, policy_version 1482056 (0.0008) [2023-12-27 02:11:09,054][105620] Updated weights for policy 1, policy_version 1484439 (0.0011) [2023-12-27 02:11:09,117][105620] Updated weights for policy 1, policy_version 1484449 (0.0010) [2023-12-27 02:11:09,183][105620] Updated weights for policy 1, policy_version 1484459 (0.0010) [2023-12-27 02:11:09,418][105692] Updated weights for policy 0, policy_version 1482066 (0.0009) [2023-12-27 02:11:09,487][105692] Updated weights for policy 0, policy_version 1482076 (0.0006) [2023-12-27 02:11:09,552][105692] Updated weights for policy 0, policy_version 1482086 (0.0009) [2023-12-27 02:11:09,939][105620] Updated weights for policy 1, policy_version 1484469 (0.0009) [2023-12-27 02:11:09,997][105620] Updated weights for policy 1, policy_version 1484479 (0.0009) [2023-12-27 02:11:10,055][105620] Updated weights for policy 1, policy_version 1484489 (0.0009) [2023-12-27 02:11:10,300][105692] Updated weights for policy 0, policy_version 1482096 (0.0009) [2023-12-27 02:11:10,360][105692] Updated weights for policy 0, policy_version 1482106 (0.0009) [2023-12-27 02:11:10,414][105692] Updated weights for policy 0, policy_version 1482116 (0.0008) [2023-12-27 02:11:10,748][105620] Updated weights for policy 1, policy_version 1484499 (0.0008) [2023-12-27 02:11:10,814][105620] Updated weights for policy 1, policy_version 1484509 (0.0007) [2023-12-27 02:11:10,879][105620] Updated weights for policy 1, policy_version 1484519 (0.0006) [2023-12-27 02:11:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 759570432. Throughput: 0: 9589.8, 1: 9964.8. Samples: 759578220. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:11:11,062][104569] Avg episode reward: [(0, '8712.650'), (1, '9173.854')] [2023-12-27 02:11:11,224][105692] Updated weights for policy 0, policy_version 1482126 (0.0009) [2023-12-27 02:11:11,282][105692] Updated weights for policy 0, policy_version 1482136 (0.0009) [2023-12-27 02:11:11,340][105692] Updated weights for policy 0, policy_version 1482146 (0.0009) [2023-12-27 02:11:11,499][105620] Updated weights for policy 1, policy_version 1484529 (0.0006) [2023-12-27 02:11:11,559][105620] Updated weights for policy 1, policy_version 1484539 (0.0011) [2023-12-27 02:11:11,619][105620] Updated weights for policy 1, policy_version 1484549 (0.0010) [2023-12-27 02:11:11,682][105620] Updated weights for policy 1, policy_version 1484559 (0.0009) [2023-12-27 02:11:12,214][105692] Updated weights for policy 0, policy_version 1482156 (0.0008) [2023-12-27 02:11:12,273][105692] Updated weights for policy 0, policy_version 1482166 (0.0008) [2023-12-27 02:11:12,282][105620] Updated weights for policy 1, policy_version 1484569 (0.0008) [2023-12-27 02:11:12,330][105692] Updated weights for policy 0, policy_version 1482176 (0.0008) [2023-12-27 02:11:12,348][105620] Updated weights for policy 1, policy_version 1484579 (0.0008) [2023-12-27 02:11:12,410][105620] Updated weights for policy 1, policy_version 1484589 (0.0011) [2023-12-27 02:11:13,111][105620] Updated weights for policy 1, policy_version 1484599 (0.0010) [2023-12-27 02:11:13,137][105692] Updated weights for policy 0, policy_version 1482186 (0.0006) [2023-12-27 02:11:13,161][105620] Updated weights for policy 1, policy_version 1484609 (0.0010) [2023-12-27 02:11:13,197][105692] Updated weights for policy 0, policy_version 1482196 (0.0006) [2023-12-27 02:11:13,210][105620] Updated weights for policy 1, policy_version 1484619 (0.0010) [2023-12-27 02:11:13,258][105692] Updated weights for policy 0, policy_version 1482206 (0.0006) [2023-12-27 02:11:13,316][105692] Updated weights for policy 0, policy_version 1482216 (0.0009) [2023-12-27 02:11:13,909][105620] Updated weights for policy 1, policy_version 1484629 (0.0009) [2023-12-27 02:11:13,957][105620] Updated weights for policy 1, policy_version 1484639 (0.0008) [2023-12-27 02:11:14,014][105620] Updated weights for policy 1, policy_version 1484649 (0.0010) [2023-12-27 02:11:14,037][105692] Updated weights for policy 0, policy_version 1482226 (0.0011) [2023-12-27 02:11:14,096][105692] Updated weights for policy 0, policy_version 1482236 (0.0010) [2023-12-27 02:11:14,156][105692] Updated weights for policy 0, policy_version 1482246 (0.0008) [2023-12-27 02:11:14,681][105620] Updated weights for policy 1, policy_version 1484659 (0.0007) [2023-12-27 02:11:14,744][105620] Updated weights for policy 1, policy_version 1484669 (0.0006) [2023-12-27 02:11:14,809][105620] Updated weights for policy 1, policy_version 1484679 (0.0010) [2023-12-27 02:11:14,842][105692] Updated weights for policy 0, policy_version 1482256 (0.0011) [2023-12-27 02:11:14,864][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000006 [2023-12-27 02:11:14,902][105692] Updated weights for policy 0, policy_version 1482266 (0.0011) [2023-12-27 02:11:14,969][105692] Updated weights for policy 0, policy_version 1482276 (0.0011) [2023-12-27 02:11:15,573][105620] Updated weights for policy 1, policy_version 1484689 (0.0011) [2023-12-27 02:11:15,624][105620] Updated weights for policy 1, policy_version 1484699 (0.0010) [2023-12-27 02:11:15,680][105620] Updated weights for policy 1, policy_version 1484709 (0.0011) [2023-12-27 02:11:15,710][105692] Updated weights for policy 0, policy_version 1482286 (0.0008) [2023-12-27 02:11:15,728][105620] Updated weights for policy 1, policy_version 1484719 (0.0010) [2023-12-27 02:11:15,768][105692] Updated weights for policy 0, policy_version 1482296 (0.0007) [2023-12-27 02:11:15,832][105692] Updated weights for policy 0, policy_version 1482306 (0.0008) [2023-12-27 02:11:16,062][104569] Fps is (10 sec: 18840.7, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 759668736. Throughput: 0: 9462.7, 1: 9912.1. Samples: 759634648. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:11:16,063][104569] Avg episode reward: [(0, '8810.119'), (1, '9081.471')] [2023-12-27 02:11:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001482312_379527168.pth... [2023-12-27 02:11:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001484720_380141568.pth... [2023-12-27 02:11:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001481224_379248640.pth [2023-12-27 02:11:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001483568_379846656.pth [2023-12-27 02:11:16,485][105620] Updated weights for policy 1, policy_version 1484729 (0.0010) [2023-12-27 02:11:16,543][105620] Updated weights for policy 1, policy_version 1484739 (0.0010) [2023-12-27 02:11:16,549][105692] Updated weights for policy 0, policy_version 1482316 (0.0007) [2023-12-27 02:11:16,597][105692] Updated weights for policy 0, policy_version 1482326 (0.0006) [2023-12-27 02:11:16,602][105620] Updated weights for policy 1, policy_version 1484749 (0.0011) [2023-12-27 02:11:16,643][105692] Updated weights for policy 0, policy_version 1482336 (0.0008) [2023-12-27 02:11:17,272][105692] Updated weights for policy 0, policy_version 1482346 (0.0009) [2023-12-27 02:11:17,318][105620] Updated weights for policy 1, policy_version 1484759 (0.0010) [2023-12-27 02:11:17,323][105692] Updated weights for policy 0, policy_version 1482356 (0.0005) [2023-12-27 02:11:17,374][105692] Updated weights for policy 0, policy_version 1482366 (0.0005) [2023-12-27 02:11:17,377][105620] Updated weights for policy 1, policy_version 1484769 (0.0010) [2023-12-27 02:11:17,428][105692] Updated weights for policy 0, policy_version 1482376 (0.0005) [2023-12-27 02:11:17,435][105620] Updated weights for policy 1, policy_version 1484779 (0.0010) [2023-12-27 02:11:18,064][105692] Updated weights for policy 0, policy_version 1482386 (0.0010) [2023-12-27 02:11:18,122][105692] Updated weights for policy 0, policy_version 1482396 (0.0010) [2023-12-27 02:11:18,170][105620] Updated weights for policy 1, policy_version 1484789 (0.0010) [2023-12-27 02:11:18,179][105692] Updated weights for policy 0, policy_version 1482406 (0.0011) [2023-12-27 02:11:18,225][105620] Updated weights for policy 1, policy_version 1484799 (0.0010) [2023-12-27 02:11:18,283][105620] Updated weights for policy 1, policy_version 1484809 (0.0010) [2023-12-27 02:11:18,814][105692] Updated weights for policy 0, policy_version 1482416 (0.0010) [2023-12-27 02:11:18,870][105692] Updated weights for policy 0, policy_version 1482426 (0.0010) [2023-12-27 02:11:18,915][105692] Updated weights for policy 0, policy_version 1482436 (0.0010) [2023-12-27 02:11:19,028][105620] Updated weights for policy 1, policy_version 1484819 (0.0010) [2023-12-27 02:11:19,083][105620] Updated weights for policy 1, policy_version 1484829 (0.0010) [2023-12-27 02:11:19,137][105620] Updated weights for policy 1, policy_version 1484839 (0.0010) [2023-12-27 02:11:19,649][105692] Updated weights for policy 0, policy_version 1482446 (0.0007) [2023-12-27 02:11:19,715][105692] Updated weights for policy 0, policy_version 1482456 (0.0006) [2023-12-27 02:11:19,778][105692] Updated weights for policy 0, policy_version 1482466 (0.0008) [2023-12-27 02:11:19,861][105620] Updated weights for policy 1, policy_version 1484849 (0.0010) [2023-12-27 02:11:19,927][105620] Updated weights for policy 1, policy_version 1484859 (0.0007) [2023-12-27 02:11:19,993][105620] Updated weights for policy 1, policy_version 1484869 (0.0008) [2023-12-27 02:11:20,061][105620] Updated weights for policy 1, policy_version 1484879 (0.0009) [2023-12-27 02:11:20,066][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000009 [2023-12-27 02:11:20,626][105692] Updated weights for policy 0, policy_version 1482476 (0.0009) [2023-12-27 02:11:20,685][105692] Updated weights for policy 0, policy_version 1482486 (0.0007) [2023-12-27 02:11:20,726][105620] Updated weights for policy 1, policy_version 1484889 (0.0009) [2023-12-27 02:11:20,741][105692] Updated weights for policy 0, policy_version 1482496 (0.0005) [2023-12-27 02:11:20,780][105620] Updated weights for policy 1, policy_version 1484899 (0.0008) [2023-12-27 02:11:20,843][105620] Updated weights for policy 1, policy_version 1484909 (0.0009) [2023-12-27 02:11:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 759767040. Throughput: 0: 9521.5, 1: 9817.9. Samples: 759754088. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:11:21,063][104569] Avg episode reward: [(0, '8534.638'), (1, '9262.123')] [2023-12-27 02:11:21,440][105692] Updated weights for policy 0, policy_version 1482506 (0.0006) [2023-12-27 02:11:21,504][105692] Updated weights for policy 0, policy_version 1482516 (0.0009) [2023-12-27 02:11:21,570][105692] Updated weights for policy 0, policy_version 1482526 (0.0009) [2023-12-27 02:11:21,630][105692] Updated weights for policy 0, policy_version 1482536 (0.0009) [2023-12-27 02:11:21,654][105620] Updated weights for policy 1, policy_version 1484919 (0.0010) [2023-12-27 02:11:21,713][105620] Updated weights for policy 1, policy_version 1484929 (0.0009) [2023-12-27 02:11:21,773][105620] Updated weights for policy 1, policy_version 1484939 (0.0008) [2023-12-27 02:11:22,441][105692] Updated weights for policy 0, policy_version 1482546 (0.0008) [2023-12-27 02:11:22,491][105692] Updated weights for policy 0, policy_version 1482556 (0.0009) [2023-12-27 02:11:22,539][105692] Updated weights for policy 0, policy_version 1482566 (0.0009) [2023-12-27 02:11:22,567][105620] Updated weights for policy 1, policy_version 1484949 (0.0009) [2023-12-27 02:11:22,617][105620] Updated weights for policy 1, policy_version 1484959 (0.0007) [2023-12-27 02:11:22,671][105620] Updated weights for policy 1, policy_version 1484969 (0.0006) [2023-12-27 02:11:23,276][105692] Updated weights for policy 0, policy_version 1482576 (0.0007) [2023-12-27 02:11:23,288][105620] Updated weights for policy 1, policy_version 1484979 (0.0007) [2023-12-27 02:11:23,329][105692] Updated weights for policy 0, policy_version 1482586 (0.0009) [2023-12-27 02:11:23,336][105620] Updated weights for policy 1, policy_version 1484989 (0.0009) [2023-12-27 02:11:23,380][105692] Updated weights for policy 0, policy_version 1482596 (0.0008) [2023-12-27 02:11:23,384][105620] Updated weights for policy 1, policy_version 1484999 (0.0009) [2023-12-27 02:11:24,016][105692] Updated weights for policy 0, policy_version 1482606 (0.0005) [2023-12-27 02:11:24,021][105620] Updated weights for policy 1, policy_version 1485009 (0.0009) [2023-12-27 02:11:24,065][105692] Updated weights for policy 0, policy_version 1482616 (0.0005) [2023-12-27 02:11:24,078][105620] Updated weights for policy 1, policy_version 1485019 (0.0005) [2023-12-27 02:11:24,122][105692] Updated weights for policy 0, policy_version 1482626 (0.0005) [2023-12-27 02:11:24,136][105620] Updated weights for policy 1, policy_version 1485029 (0.0005) [2023-12-27 02:11:24,204][105620] Updated weights for policy 1, policy_version 1485039 (0.0005) [2023-12-27 02:11:24,740][105692] Updated weights for policy 0, policy_version 1482636 (0.0007) [2023-12-27 02:11:24,793][105692] Updated weights for policy 0, policy_version 1482646 (0.0010) [2023-12-27 02:11:24,829][105620] Updated weights for policy 1, policy_version 1485049 (0.0005) [2023-12-27 02:11:24,840][105692] Updated weights for policy 0, policy_version 1482656 (0.0008) [2023-12-27 02:11:24,885][105620] Updated weights for policy 1, policy_version 1485059 (0.0005) [2023-12-27 02:11:24,952][105620] Updated weights for policy 1, policy_version 1485069 (0.0005) [2023-12-27 02:11:25,492][105620] Updated weights for policy 1, policy_version 1485079 (0.0007) [2023-12-27 02:11:25,553][105620] Updated weights for policy 1, policy_version 1485089 (0.0007) [2023-12-27 02:11:25,599][105620] Updated weights for policy 1, policy_version 1485099 (0.0005) [2023-12-27 02:11:25,700][105692] Updated weights for policy 0, policy_version 1482666 (0.0009) [2023-12-27 02:11:25,759][105692] Updated weights for policy 0, policy_version 1482676 (0.0009) [2023-12-27 02:11:25,818][105692] Updated weights for policy 0, policy_version 1482687 (0.0010) [2023-12-27 02:11:26,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 759865344. Throughput: 0: 9475.6, 1: 9927.6. Samples: 759874372. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:11:26,063][104569] Avg episode reward: [(0, '8712.204'), (1, '9079.999')] [2023-12-27 02:11:26,144][105620] Updated weights for policy 1, policy_version 1485109 (0.0005) [2023-12-27 02:11:26,195][105620] Updated weights for policy 1, policy_version 1485119 (0.0008) [2023-12-27 02:11:26,260][105620] Updated weights for policy 1, policy_version 1485129 (0.0010) [2023-12-27 02:11:26,662][105692] Updated weights for policy 0, policy_version 1482697 (0.0010) [2023-12-27 02:11:26,713][105692] Updated weights for policy 0, policy_version 1482707 (0.0010) [2023-12-27 02:11:26,775][105692] Updated weights for policy 0, policy_version 1482717 (0.0010) [2023-12-27 02:11:26,833][105692] Updated weights for policy 0, policy_version 1482727 (0.0010) [2023-12-27 02:11:26,935][105620] Updated weights for policy 1, policy_version 1485139 (0.0008) [2023-12-27 02:11:26,979][105620] Updated weights for policy 1, policy_version 1485149 (0.0010) [2023-12-27 02:11:27,026][105620] Updated weights for policy 1, policy_version 1485159 (0.0006) [2023-12-27 02:11:27,577][105620] Updated weights for policy 1, policy_version 1485169 (0.0005) [2023-12-27 02:11:27,582][105692] Updated weights for policy 0, policy_version 1482737 (0.0010) [2023-12-27 02:11:27,623][105620] Updated weights for policy 1, policy_version 1485179 (0.0005) [2023-12-27 02:11:27,636][105692] Updated weights for policy 0, policy_version 1482747 (0.0009) [2023-12-27 02:11:27,672][105620] Updated weights for policy 1, policy_version 1485189 (0.0008) [2023-12-27 02:11:27,696][105692] Updated weights for policy 0, policy_version 1482757 (0.0005) [2023-12-27 02:11:27,717][105620] Updated weights for policy 1, policy_version 1485199 (0.0009) [2023-12-27 02:11:28,250][105692] Updated weights for policy 0, policy_version 1482767 (0.0007) [2023-12-27 02:11:28,306][105692] Updated weights for policy 0, policy_version 1482777 (0.0010) [2023-12-27 02:11:28,368][105692] Updated weights for policy 0, policy_version 1482787 (0.0008) [2023-12-27 02:11:28,404][105620] Updated weights for policy 1, policy_version 1485209 (0.0007) [2023-12-27 02:11:28,469][105620] Updated weights for policy 1, policy_version 1485219 (0.0010) [2023-12-27 02:11:28,535][105620] Updated weights for policy 1, policy_version 1485229 (0.0008) [2023-12-27 02:11:28,934][105692] Updated weights for policy 0, policy_version 1482797 (0.0006) [2023-12-27 02:11:28,988][105692] Updated weights for policy 0, policy_version 1482807 (0.0010) [2023-12-27 02:11:29,037][105692] Updated weights for policy 0, policy_version 1482817 (0.0010) [2023-12-27 02:11:29,201][105620] Updated weights for policy 1, policy_version 1485239 (0.0010) [2023-12-27 02:11:29,258][105620] Updated weights for policy 1, policy_version 1485249 (0.0011) [2023-12-27 02:11:29,318][105620] Updated weights for policy 1, policy_version 1485259 (0.0011) [2023-12-27 02:11:29,736][105692] Updated weights for policy 0, policy_version 1482827 (0.0010) [2023-12-27 02:11:29,787][105692] Updated weights for policy 0, policy_version 1482837 (0.0010) [2023-12-27 02:11:29,843][105692] Updated weights for policy 0, policy_version 1482847 (0.0010) [2023-12-27 02:11:30,078][105620] Updated weights for policy 1, policy_version 1485269 (0.0011) [2023-12-27 02:11:30,144][105620] Updated weights for policy 1, policy_version 1485279 (0.0006) [2023-12-27 02:11:30,205][105620] Updated weights for policy 1, policy_version 1485289 (0.0008) [2023-12-27 02:11:30,536][105692] Updated weights for policy 0, policy_version 1482857 (0.0008) [2023-12-27 02:11:30,584][105692] Updated weights for policy 0, policy_version 1482867 (0.0008) [2023-12-27 02:11:30,636][105692] Updated weights for policy 0, policy_version 1482877 (0.0008) [2023-12-27 02:11:30,681][105692] Updated weights for policy 0, policy_version 1482887 (0.0008) [2023-12-27 02:11:30,851][105620] Updated weights for policy 1, policy_version 1485299 (0.0009) [2023-12-27 02:11:30,906][105620] Updated weights for policy 1, policy_version 1485309 (0.0006) [2023-12-27 02:11:30,959][105620] Updated weights for policy 1, policy_version 1485319 (0.0005) [2023-12-27 02:11:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 759971840. Throughput: 0: 9536.3, 1: 10025.2. Samples: 759936464. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:11:31,062][104569] Avg episode reward: [(0, '8622.102'), (1, '9077.847')] [2023-12-27 02:11:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001482888_379674624.pth... [2023-12-27 02:11:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001485328_380297216.pth... [2023-12-27 02:11:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001481768_379387904.pth [2023-12-27 02:11:31,090][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001484144_379994112.pth [2023-12-27 02:11:31,457][105692] Updated weights for policy 0, policy_version 1482897 (0.0008) [2023-12-27 02:11:31,518][105692] Updated weights for policy 0, policy_version 1482907 (0.0008) [2023-12-27 02:11:31,564][105692] Updated weights for policy 0, policy_version 1482917 (0.0008) [2023-12-27 02:11:31,677][105620] Updated weights for policy 1, policy_version 1485329 (0.0006) [2023-12-27 02:11:31,736][105620] Updated weights for policy 1, policy_version 1485339 (0.0010) [2023-12-27 02:11:31,786][105620] Updated weights for policy 1, policy_version 1485349 (0.0009) [2023-12-27 02:11:31,835][105620] Updated weights for policy 1, policy_version 1485359 (0.0008) [2023-12-27 02:11:32,353][105692] Updated weights for policy 0, policy_version 1482927 (0.0010) [2023-12-27 02:11:32,416][105692] Updated weights for policy 0, policy_version 1482937 (0.0011) [2023-12-27 02:11:32,471][105692] Updated weights for policy 0, policy_version 1482947 (0.0010) [2023-12-27 02:11:32,555][105620] Updated weights for policy 1, policy_version 1485369 (0.0008) [2023-12-27 02:11:32,607][105620] Updated weights for policy 1, policy_version 1485379 (0.0008) [2023-12-27 02:11:32,660][105620] Updated weights for policy 1, policy_version 1485389 (0.0007) [2023-12-27 02:11:33,207][105692] Updated weights for policy 0, policy_version 1482957 (0.0010) [2023-12-27 02:11:33,274][105692] Updated weights for policy 0, policy_version 1482967 (0.0010) [2023-12-27 02:11:33,328][105692] Updated weights for policy 0, policy_version 1482977 (0.0010) [2023-12-27 02:11:33,412][105620] Updated weights for policy 1, policy_version 1485399 (0.0005) [2023-12-27 02:11:33,467][105620] Updated weights for policy 1, policy_version 1485409 (0.0007) [2023-12-27 02:11:33,531][105620] Updated weights for policy 1, policy_version 1485419 (0.0010) [2023-12-27 02:11:34,034][105692] Updated weights for policy 0, policy_version 1482987 (0.0008) [2023-12-27 02:11:34,103][105692] Updated weights for policy 0, policy_version 1482997 (0.0005) [2023-12-27 02:11:34,166][105692] Updated weights for policy 0, policy_version 1483007 (0.0007) [2023-12-27 02:11:34,178][105620] Updated weights for policy 1, policy_version 1485429 (0.0010) [2023-12-27 02:11:34,237][105620] Updated weights for policy 1, policy_version 1485439 (0.0010) [2023-12-27 02:11:34,299][105620] Updated weights for policy 1, policy_version 1485449 (0.0010) [2023-12-27 02:11:34,887][105692] Updated weights for policy 0, policy_version 1483017 (0.0008) [2023-12-27 02:11:34,951][105692] Updated weights for policy 0, policy_version 1483027 (0.0008) [2023-12-27 02:11:35,007][105692] Updated weights for policy 0, policy_version 1483037 (0.0009) [2023-12-27 02:11:35,047][105620] Updated weights for policy 1, policy_version 1485459 (0.0010) [2023-12-27 02:11:35,073][105692] Updated weights for policy 0, policy_version 1483047 (0.0006) [2023-12-27 02:11:35,098][105620] Updated weights for policy 1, policy_version 1485469 (0.0010) [2023-12-27 02:11:35,150][105620] Updated weights for policy 1, policy_version 1485479 (0.0010) [2023-12-27 02:11:35,778][105620] Updated weights for policy 1, policy_version 1485489 (0.0010) [2023-12-27 02:11:35,838][105620] Updated weights for policy 1, policy_version 1485499 (0.0008) [2023-12-27 02:11:35,877][105692] Updated weights for policy 0, policy_version 1483057 (0.0008) [2023-12-27 02:11:35,903][105620] Updated weights for policy 1, policy_version 1485509 (0.0005) [2023-12-27 02:11:35,944][105692] Updated weights for policy 0, policy_version 1483067 (0.0011) [2023-12-27 02:11:35,969][105620] Updated weights for policy 1, policy_version 1485519 (0.0006) [2023-12-27 02:11:36,002][105692] Updated weights for policy 0, policy_version 1483077 (0.0011) [2023-12-27 02:11:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 760070144. Throughput: 0: 9589.4, 1: 9975.9. Samples: 760054484. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:11:36,063][104569] Avg episode reward: [(0, '8530.777'), (1, '9170.038')] [2023-12-27 02:11:36,708][105620] Updated weights for policy 1, policy_version 1485529 (0.0008) [2023-12-27 02:11:36,749][105692] Updated weights for policy 0, policy_version 1483087 (0.0009) [2023-12-27 02:11:36,764][105620] Updated weights for policy 1, policy_version 1485539 (0.0005) [2023-12-27 02:11:36,806][105692] Updated weights for policy 0, policy_version 1483097 (0.0008) [2023-12-27 02:11:36,822][105620] Updated weights for policy 1, policy_version 1485549 (0.0006) [2023-12-27 02:11:36,865][105692] Updated weights for policy 0, policy_version 1483107 (0.0010) [2023-12-27 02:11:37,461][105620] Updated weights for policy 1, policy_version 1485559 (0.0006) [2023-12-27 02:11:37,513][105620] Updated weights for policy 1, policy_version 1485569 (0.0010) [2023-12-27 02:11:37,571][105620] Updated weights for policy 1, policy_version 1485579 (0.0010) [2023-12-27 02:11:37,653][105692] Updated weights for policy 0, policy_version 1483117 (0.0008) [2023-12-27 02:11:37,708][105692] Updated weights for policy 0, policy_version 1483127 (0.0009) [2023-12-27 02:11:37,764][105692] Updated weights for policy 0, policy_version 1483137 (0.0008) [2023-12-27 02:11:38,253][105620] Updated weights for policy 1, policy_version 1485589 (0.0010) [2023-12-27 02:11:38,304][105620] Updated weights for policy 1, policy_version 1485599 (0.0010) [2023-12-27 02:11:38,373][105620] Updated weights for policy 1, policy_version 1485609 (0.0008) [2023-12-27 02:11:38,524][105692] Updated weights for policy 0, policy_version 1483147 (0.0009) [2023-12-27 02:11:38,579][105692] Updated weights for policy 0, policy_version 1483157 (0.0010) [2023-12-27 02:11:38,642][105692] Updated weights for policy 0, policy_version 1483167 (0.0010) [2023-12-27 02:11:38,954][105620] Updated weights for policy 1, policy_version 1485619 (0.0007) [2023-12-27 02:11:39,022][105620] Updated weights for policy 1, policy_version 1485629 (0.0010) [2023-12-27 02:11:39,070][105620] Updated weights for policy 1, policy_version 1485639 (0.0009) [2023-12-27 02:11:39,518][105692] Updated weights for policy 0, policy_version 1483177 (0.0010) [2023-12-27 02:11:39,576][105692] Updated weights for policy 0, policy_version 1483187 (0.0006) [2023-12-27 02:11:39,634][105692] Updated weights for policy 0, policy_version 1483197 (0.0005) [2023-12-27 02:11:39,692][105692] Updated weights for policy 0, policy_version 1483207 (0.0008) [2023-12-27 02:11:39,700][105620] Updated weights for policy 1, policy_version 1485649 (0.0006) [2023-12-27 02:11:39,760][105620] Updated weights for policy 1, policy_version 1485659 (0.0010) [2023-12-27 02:11:39,816][105620] Updated weights for policy 1, policy_version 1485669 (0.0010) [2023-12-27 02:11:39,874][105620] Updated weights for policy 1, policy_version 1485679 (0.0011) [2023-12-27 02:11:40,414][105692] Updated weights for policy 0, policy_version 1483217 (0.0009) [2023-12-27 02:11:40,476][105692] Updated weights for policy 0, policy_version 1483227 (0.0008) [2023-12-27 02:11:40,545][105692] Updated weights for policy 0, policy_version 1483237 (0.0008) [2023-12-27 02:11:40,667][105620] Updated weights for policy 1, policy_version 1485689 (0.0011) [2023-12-27 02:11:40,719][105620] Updated weights for policy 1, policy_version 1485699 (0.0010) [2023-12-27 02:11:40,768][105620] Updated weights for policy 1, policy_version 1485709 (0.0010) [2023-12-27 02:11:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 760160256. Throughput: 0: 9496.4, 1: 10051.2. Samples: 760169064. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:11:41,062][104569] Avg episode reward: [(0, '8526.223'), (1, '9078.754')] [2023-12-27 02:11:41,366][105692] Updated weights for policy 0, policy_version 1483247 (0.0009) [2023-12-27 02:11:41,432][105692] Updated weights for policy 0, policy_version 1483257 (0.0009) [2023-12-27 02:11:41,480][105620] Updated weights for policy 1, policy_version 1485719 (0.0008) [2023-12-27 02:11:41,491][105692] Updated weights for policy 0, policy_version 1483267 (0.0009) [2023-12-27 02:11:41,535][105620] Updated weights for policy 1, policy_version 1485729 (0.0007) [2023-12-27 02:11:41,597][105620] Updated weights for policy 1, policy_version 1485739 (0.0009) [2023-12-27 02:11:42,142][105692] Updated weights for policy 0, policy_version 1483277 (0.0008) [2023-12-27 02:11:42,195][105692] Updated weights for policy 0, policy_version 1483287 (0.0009) [2023-12-27 02:11:42,252][105692] Updated weights for policy 0, policy_version 1483297 (0.0006) [2023-12-27 02:11:42,449][105620] Updated weights for policy 1, policy_version 1485749 (0.0009) [2023-12-27 02:11:42,504][105620] Updated weights for policy 1, policy_version 1485759 (0.0008) [2023-12-27 02:11:42,559][105620] Updated weights for policy 1, policy_version 1485769 (0.0009) [2023-12-27 02:11:43,079][105692] Updated weights for policy 0, policy_version 1483307 (0.0009) [2023-12-27 02:11:43,143][105692] Updated weights for policy 0, policy_version 1483317 (0.0010) [2023-12-27 02:11:43,187][105620] Updated weights for policy 1, policy_version 1485779 (0.0009) [2023-12-27 02:11:43,198][105692] Updated weights for policy 0, policy_version 1483327 (0.0008) [2023-12-27 02:11:43,242][105620] Updated weights for policy 1, policy_version 1485789 (0.0007) [2023-12-27 02:11:43,295][105620] Updated weights for policy 1, policy_version 1485799 (0.0006) [2023-12-27 02:11:43,960][105620] Updated weights for policy 1, policy_version 1485809 (0.0006) [2023-12-27 02:11:43,961][105692] Updated weights for policy 0, policy_version 1483337 (0.0009) [2023-12-27 02:11:44,023][105620] Updated weights for policy 1, policy_version 1485819 (0.0008) [2023-12-27 02:11:44,026][105692] Updated weights for policy 0, policy_version 1483347 (0.0008) [2023-12-27 02:11:44,081][105620] Updated weights for policy 1, policy_version 1485829 (0.0006) [2023-12-27 02:11:44,087][105692] Updated weights for policy 0, policy_version 1483357 (0.0010) [2023-12-27 02:11:44,138][105620] Updated weights for policy 1, policy_version 1485839 (0.0006) [2023-12-27 02:11:44,147][105692] Updated weights for policy 0, policy_version 1483367 (0.0011) [2023-12-27 02:11:44,782][105692] Updated weights for policy 0, policy_version 1483377 (0.0009) [2023-12-27 02:11:44,842][105692] Updated weights for policy 0, policy_version 1483387 (0.0011) [2023-12-27 02:11:44,846][105620] Updated weights for policy 1, policy_version 1485849 (0.0007) [2023-12-27 02:11:44,898][105692] Updated weights for policy 0, policy_version 1483397 (0.0011) [2023-12-27 02:11:44,900][105620] Updated weights for policy 1, policy_version 1485859 (0.0007) [2023-12-27 02:11:44,954][105620] Updated weights for policy 1, policy_version 1485869 (0.0007) [2023-12-27 02:11:45,634][105692] Updated weights for policy 0, policy_version 1483407 (0.0011) [2023-12-27 02:11:45,700][105692] Updated weights for policy 0, policy_version 1483417 (0.0011) [2023-12-27 02:11:45,703][105620] Updated weights for policy 1, policy_version 1485879 (0.0006) [2023-12-27 02:11:45,760][105692] Updated weights for policy 0, policy_version 1483427 (0.0007) [2023-12-27 02:11:45,771][105620] Updated weights for policy 1, policy_version 1485889 (0.0009) [2023-12-27 02:11:45,835][105620] Updated weights for policy 1, policy_version 1485899 (0.0010) [2023-12-27 02:11:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 760258560. Throughput: 0: 9445.4, 1: 10110.3. Samples: 760225884. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:11:46,063][104569] Avg episode reward: [(0, '8801.070'), (1, '9077.780')] [2023-12-27 02:11:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001485904_380444672.pth... [2023-12-27 02:11:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001483432_379813888.pth... [2023-12-27 02:11:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001484720_380141568.pth [2023-12-27 02:11:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001482312_379527168.pth [2023-12-27 02:11:46,367][105692] Updated weights for policy 0, policy_version 1483437 (0.0008) [2023-12-27 02:11:46,428][105692] Updated weights for policy 0, policy_version 1483448 (0.0010) [2023-12-27 02:11:46,468][105620] Updated weights for policy 1, policy_version 1485909 (0.0010) [2023-12-27 02:11:46,495][105692] Updated weights for policy 0, policy_version 1483458 (0.0007) [2023-12-27 02:11:46,526][105620] Updated weights for policy 1, policy_version 1485919 (0.0008) [2023-12-27 02:11:46,585][105620] Updated weights for policy 1, policy_version 1485929 (0.0008) [2023-12-27 02:11:47,184][105692] Updated weights for policy 0, policy_version 1483468 (0.0006) [2023-12-27 02:11:47,203][105620] Updated weights for policy 1, policy_version 1485939 (0.0009) [2023-12-27 02:11:47,231][105692] Updated weights for policy 0, policy_version 1483478 (0.0006) [2023-12-27 02:11:47,259][105620] Updated weights for policy 1, policy_version 1485949 (0.0008) [2023-12-27 02:11:47,286][105692] Updated weights for policy 0, policy_version 1483488 (0.0007) [2023-12-27 02:11:47,318][105620] Updated weights for policy 1, policy_version 1485959 (0.0008) [2023-12-27 02:11:47,923][105692] Updated weights for policy 0, policy_version 1483498 (0.0007) [2023-12-27 02:11:47,971][105692] Updated weights for policy 0, policy_version 1483508 (0.0006) [2023-12-27 02:11:48,027][105692] Updated weights for policy 0, policy_version 1483518 (0.0008) [2023-12-27 02:11:48,068][105620] Updated weights for policy 1, policy_version 1485969 (0.0008) [2023-12-27 02:11:48,085][105692] Updated weights for policy 0, policy_version 1483528 (0.0007) [2023-12-27 02:11:48,130][105620] Updated weights for policy 1, policy_version 1485979 (0.0010) [2023-12-27 02:11:48,195][105620] Updated weights for policy 1, policy_version 1485989 (0.0010) [2023-12-27 02:11:48,257][105620] Updated weights for policy 1, policy_version 1485999 (0.0010) [2023-12-27 02:11:48,726][105692] Updated weights for policy 0, policy_version 1483538 (0.0009) [2023-12-27 02:11:48,778][105692] Updated weights for policy 0, policy_version 1483548 (0.0010) [2023-12-27 02:11:48,836][105692] Updated weights for policy 0, policy_version 1483558 (0.0007) [2023-12-27 02:11:48,994][105620] Updated weights for policy 1, policy_version 1486009 (0.0011) [2023-12-27 02:11:49,052][105620] Updated weights for policy 1, policy_version 1486019 (0.0010) [2023-12-27 02:11:49,127][105620] Updated weights for policy 1, policy_version 1486029 (0.0010) [2023-12-27 02:11:49,468][105692] Updated weights for policy 0, policy_version 1483568 (0.0007) [2023-12-27 02:11:49,527][105692] Updated weights for policy 0, policy_version 1483578 (0.0008) [2023-12-27 02:11:49,580][105692] Updated weights for policy 0, policy_version 1483588 (0.0008) [2023-12-27 02:11:49,912][105620] Updated weights for policy 1, policy_version 1486039 (0.0010) [2023-12-27 02:11:49,980][105620] Updated weights for policy 1, policy_version 1486049 (0.0011) [2023-12-27 02:11:50,050][105620] Updated weights for policy 1, policy_version 1486059 (0.0011) [2023-12-27 02:11:50,271][105692] Updated weights for policy 0, policy_version 1483598 (0.0007) [2023-12-27 02:11:50,330][105692] Updated weights for policy 0, policy_version 1483608 (0.0008) [2023-12-27 02:11:50,391][105692] Updated weights for policy 0, policy_version 1483618 (0.0008) [2023-12-27 02:11:50,812][105620] Updated weights for policy 1, policy_version 1486069 (0.0011) [2023-12-27 02:11:50,862][105620] Updated weights for policy 1, policy_version 1486079 (0.0011) [2023-12-27 02:11:50,919][105620] Updated weights for policy 1, policy_version 1486089 (0.0011) [2023-12-27 02:11:51,023][105692] Updated weights for policy 0, policy_version 1483628 (0.0009) [2023-12-27 02:11:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 760356864. Throughput: 0: 9605.0, 1: 10013.8. Samples: 760347016. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:11:51,062][104569] Avg episode reward: [(0, '8713.291'), (1, '9261.078')] [2023-12-27 02:11:51,084][105692] Updated weights for policy 0, policy_version 1483638 (0.0008) [2023-12-27 02:11:51,156][105692] Updated weights for policy 0, policy_version 1483648 (0.0008) [2023-12-27 02:11:51,673][105620] Updated weights for policy 1, policy_version 1486099 (0.0010) [2023-12-27 02:11:51,742][105620] Updated weights for policy 1, policy_version 1486109 (0.0008) [2023-12-27 02:11:51,797][105620] Updated weights for policy 1, policy_version 1486119 (0.0008) [2023-12-27 02:11:51,830][105692] Updated weights for policy 0, policy_version 1483658 (0.0009) [2023-12-27 02:11:51,895][105692] Updated weights for policy 0, policy_version 1483668 (0.0008) [2023-12-27 02:11:51,963][105692] Updated weights for policy 0, policy_version 1483678 (0.0011) [2023-12-27 02:11:52,030][105692] Updated weights for policy 0, policy_version 1483688 (0.0011) [2023-12-27 02:11:52,524][105620] Updated weights for policy 1, policy_version 1486129 (0.0008) [2023-12-27 02:11:52,587][105620] Updated weights for policy 1, policy_version 1486139 (0.0009) [2023-12-27 02:11:52,650][105620] Updated weights for policy 1, policy_version 1486149 (0.0009) [2023-12-27 02:11:52,713][105620] Updated weights for policy 1, policy_version 1486159 (0.0008) [2023-12-27 02:11:52,768][105692] Updated weights for policy 0, policy_version 1483698 (0.0005) [2023-12-27 02:11:52,826][105692] Updated weights for policy 0, policy_version 1483708 (0.0006) [2023-12-27 02:11:52,884][105692] Updated weights for policy 0, policy_version 1483718 (0.0009) [2023-12-27 02:11:53,450][105692] Updated weights for policy 0, policy_version 1483728 (0.0006) [2023-12-27 02:11:53,502][105692] Updated weights for policy 0, policy_version 1483738 (0.0005) [2023-12-27 02:11:53,558][105692] Updated weights for policy 0, policy_version 1483748 (0.0006) [2023-12-27 02:11:53,566][105620] Updated weights for policy 1, policy_version 1486169 (0.0009) [2023-12-27 02:11:53,619][105620] Updated weights for policy 1, policy_version 1486179 (0.0006) [2023-12-27 02:11:53,688][105620] Updated weights for policy 1, policy_version 1486189 (0.0005) [2023-12-27 02:11:54,196][105692] Updated weights for policy 0, policy_version 1483758 (0.0005) [2023-12-27 02:11:54,245][105692] Updated weights for policy 0, policy_version 1483768 (0.0005) [2023-12-27 02:11:54,299][105692] Updated weights for policy 0, policy_version 1483778 (0.0005) [2023-12-27 02:11:54,433][105620] Updated weights for policy 1, policy_version 1486199 (0.0008) [2023-12-27 02:11:54,477][105620] Updated weights for policy 1, policy_version 1486209 (0.0008) [2023-12-27 02:11:54,526][105620] Updated weights for policy 1, policy_version 1486219 (0.0008) [2023-12-27 02:11:54,943][105692] Updated weights for policy 0, policy_version 1483788 (0.0009) [2023-12-27 02:11:54,993][105692] Updated weights for policy 0, policy_version 1483798 (0.0010) [2023-12-27 02:11:55,043][105692] Updated weights for policy 0, policy_version 1483808 (0.0010) [2023-12-27 02:11:55,299][105620] Updated weights for policy 1, policy_version 1486229 (0.0007) [2023-12-27 02:11:55,342][105620] Updated weights for policy 1, policy_version 1486239 (0.0005) [2023-12-27 02:11:55,390][105620] Updated weights for policy 1, policy_version 1486249 (0.0006) [2023-12-27 02:11:55,676][105692] Updated weights for policy 0, policy_version 1483818 (0.0009) [2023-12-27 02:11:55,737][105692] Updated weights for policy 0, policy_version 1483828 (0.0006) [2023-12-27 02:11:55,798][105692] Updated weights for policy 0, policy_version 1483838 (0.0006) [2023-12-27 02:11:55,858][105692] Updated weights for policy 0, policy_version 1483848 (0.0006) [2023-12-27 02:11:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 760455168. Throughput: 0: 9768.4, 1: 9935.6. Samples: 760464900. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:11:56,062][104569] Avg episode reward: [(0, '8805.291'), (1, '9168.505')] [2023-12-27 02:11:56,107][105620] Updated weights for policy 1, policy_version 1486259 (0.0007) [2023-12-27 02:11:56,161][105620] Updated weights for policy 1, policy_version 1486269 (0.0005) [2023-12-27 02:11:56,205][105620] Updated weights for policy 1, policy_version 1486279 (0.0006) [2023-12-27 02:11:56,584][105692] Updated weights for policy 0, policy_version 1483858 (0.0010) [2023-12-27 02:11:56,632][105692] Updated weights for policy 0, policy_version 1483868 (0.0010) [2023-12-27 02:11:56,682][105692] Updated weights for policy 0, policy_version 1483878 (0.0010) [2023-12-27 02:11:56,812][105620] Updated weights for policy 1, policy_version 1486289 (0.0008) [2023-12-27 02:11:56,866][105620] Updated weights for policy 1, policy_version 1486299 (0.0008) [2023-12-27 02:11:56,926][105620] Updated weights for policy 1, policy_version 1486309 (0.0006) [2023-12-27 02:11:56,985][105620] Updated weights for policy 1, policy_version 1486319 (0.0006) [2023-12-27 02:11:57,357][105692] Updated weights for policy 0, policy_version 1483888 (0.0010) [2023-12-27 02:11:57,417][105692] Updated weights for policy 0, policy_version 1483898 (0.0009) [2023-12-27 02:11:57,473][105692] Updated weights for policy 0, policy_version 1483908 (0.0005) [2023-12-27 02:11:57,771][105620] Updated weights for policy 1, policy_version 1486329 (0.0008) [2023-12-27 02:11:57,826][105620] Updated weights for policy 1, policy_version 1486339 (0.0008) [2023-12-27 02:11:57,882][105620] Updated weights for policy 1, policy_version 1486349 (0.0009) [2023-12-27 02:11:58,033][105692] Updated weights for policy 0, policy_version 1483918 (0.0007) [2023-12-27 02:11:58,089][105692] Updated weights for policy 0, policy_version 1483928 (0.0005) [2023-12-27 02:11:58,140][105692] Updated weights for policy 0, policy_version 1483938 (0.0005) [2023-12-27 02:11:58,696][105620] Updated weights for policy 1, policy_version 1486359 (0.0009) [2023-12-27 02:11:58,762][105620] Updated weights for policy 1, policy_version 1486369 (0.0008) [2023-12-27 02:11:58,828][105620] Updated weights for policy 1, policy_version 1486379 (0.0008) [2023-12-27 02:11:58,940][105692] Updated weights for policy 0, policy_version 1483948 (0.0007) [2023-12-27 02:11:59,004][105692] Updated weights for policy 0, policy_version 1483958 (0.0009) [2023-12-27 02:11:59,062][105692] Updated weights for policy 0, policy_version 1483968 (0.0009) [2023-12-27 02:11:59,620][105620] Updated weights for policy 1, policy_version 1486389 (0.0006) [2023-12-27 02:11:59,670][105620] Updated weights for policy 1, policy_version 1486399 (0.0005) [2023-12-27 02:11:59,727][105620] Updated weights for policy 1, policy_version 1486409 (0.0008) [2023-12-27 02:11:59,862][105692] Updated weights for policy 0, policy_version 1483978 (0.0009) [2023-12-27 02:11:59,916][105692] Updated weights for policy 0, policy_version 1483988 (0.0008) [2023-12-27 02:11:59,974][105692] Updated weights for policy 0, policy_version 1483998 (0.0008) [2023-12-27 02:12:00,031][105692] Updated weights for policy 0, policy_version 1484008 (0.0006) [2023-12-27 02:12:00,482][105620] Updated weights for policy 1, policy_version 1486419 (0.0005) [2023-12-27 02:12:00,547][105620] Updated weights for policy 1, policy_version 1486429 (0.0005) [2023-12-27 02:12:00,613][105620] Updated weights for policy 1, policy_version 1486439 (0.0006) [2023-12-27 02:12:00,669][105692] Updated weights for policy 0, policy_version 1484018 (0.0008) [2023-12-27 02:12:00,722][105692] Updated weights for policy 0, policy_version 1484028 (0.0008) [2023-12-27 02:12:00,768][105692] Updated weights for policy 0, policy_version 1484038 (0.0009) [2023-12-27 02:12:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 760553472. Throughput: 0: 9881.6, 1: 9875.4. Samples: 760523704. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:12:01,063][104569] Avg episode reward: [(0, '8894.454'), (1, '8987.691')] [2023-12-27 02:12:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001484040_379969536.pth... [2023-12-27 02:12:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001486448_380583936.pth... [2023-12-27 02:12:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001485328_380297216.pth [2023-12-27 02:12:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001482888_379674624.pth [2023-12-27 02:12:01,194][105620] Updated weights for policy 1, policy_version 1486449 (0.0006) [2023-12-27 02:12:01,248][105620] Updated weights for policy 1, policy_version 1486459 (0.0008) [2023-12-27 02:12:01,321][105620] Updated weights for policy 1, policy_version 1486469 (0.0005) [2023-12-27 02:12:01,379][105620] Updated weights for policy 1, policy_version 1486479 (0.0007) [2023-12-27 02:12:01,562][105692] Updated weights for policy 0, policy_version 1484048 (0.0009) [2023-12-27 02:12:01,624][105692] Updated weights for policy 0, policy_version 1484058 (0.0006) [2023-12-27 02:12:01,681][105692] Updated weights for policy 0, policy_version 1484068 (0.0010) [2023-12-27 02:12:02,067][105620] Updated weights for policy 1, policy_version 1486489 (0.0008) [2023-12-27 02:12:02,123][105620] Updated weights for policy 1, policy_version 1486499 (0.0009) [2023-12-27 02:12:02,181][105620] Updated weights for policy 1, policy_version 1486509 (0.0010) [2023-12-27 02:12:02,448][105692] Updated weights for policy 0, policy_version 1484079 (0.0009) [2023-12-27 02:12:02,506][105692] Updated weights for policy 0, policy_version 1484089 (0.0008) [2023-12-27 02:12:02,562][105692] Updated weights for policy 0, policy_version 1484099 (0.0007) [2023-12-27 02:12:02,971][105620] Updated weights for policy 1, policy_version 1486519 (0.0010) [2023-12-27 02:12:03,019][105620] Updated weights for policy 1, policy_version 1486530 (0.0008) [2023-12-27 02:12:03,070][105620] Updated weights for policy 1, policy_version 1486540 (0.0009) [2023-12-27 02:12:03,256][105692] Updated weights for policy 0, policy_version 1484109 (0.0010) [2023-12-27 02:12:03,310][105692] Updated weights for policy 0, policy_version 1484119 (0.0010) [2023-12-27 02:12:03,357][105692] Updated weights for policy 0, policy_version 1484129 (0.0010) [2023-12-27 02:12:03,745][105620] Updated weights for policy 1, policy_version 1486550 (0.0010) [2023-12-27 02:12:03,792][105620] Updated weights for policy 1, policy_version 1486560 (0.0010) [2023-12-27 02:12:03,841][105620] Updated weights for policy 1, policy_version 1486570 (0.0010) [2023-12-27 02:12:04,074][105692] Updated weights for policy 0, policy_version 1484139 (0.0009) [2023-12-27 02:12:04,131][105692] Updated weights for policy 0, policy_version 1484149 (0.0011) [2023-12-27 02:12:04,180][105692] Updated weights for policy 0, policy_version 1484159 (0.0010) [2023-12-27 02:12:04,572][105620] Updated weights for policy 1, policy_version 1486580 (0.0010) [2023-12-27 02:12:04,631][105620] Updated weights for policy 1, policy_version 1486590 (0.0009) [2023-12-27 02:12:04,695][105620] Updated weights for policy 1, policy_version 1486600 (0.0010) [2023-12-27 02:12:04,872][105692] Updated weights for policy 0, policy_version 1484169 (0.0010) [2023-12-27 02:12:04,937][105692] Updated weights for policy 0, policy_version 1484179 (0.0006) [2023-12-27 02:12:04,997][105692] Updated weights for policy 0, policy_version 1484189 (0.0005) [2023-12-27 02:12:05,055][105692] Updated weights for policy 0, policy_version 1484199 (0.0010) [2023-12-27 02:12:05,497][105620] Updated weights for policy 1, policy_version 1486610 (0.0009) [2023-12-27 02:12:05,552][105620] Updated weights for policy 1, policy_version 1486620 (0.0008) [2023-12-27 02:12:05,615][105620] Updated weights for policy 1, policy_version 1486630 (0.0009) [2023-12-27 02:12:05,667][105620] Updated weights for policy 1, policy_version 1486640 (0.0008) [2023-12-27 02:12:05,701][105692] Updated weights for policy 0, policy_version 1484209 (0.0010) [2023-12-27 02:12:05,759][105692] Updated weights for policy 0, policy_version 1484219 (0.0010) [2023-12-27 02:12:05,810][105692] Updated weights for policy 0, policy_version 1484229 (0.0010) [2023-12-27 02:12:06,062][104569] Fps is (10 sec: 19660.0, 60 sec: 19524.1, 300 sec: 19549.7). Total num frames: 760651776. Throughput: 0: 9768.1, 1: 9903.9. Samples: 760639336. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:12:06,064][104569] Avg episode reward: [(0, '8802.807'), (1, '9080.081')] [2023-12-27 02:12:06,474][105692] Updated weights for policy 0, policy_version 1484239 (0.0011) [2023-12-27 02:12:06,488][105620] Updated weights for policy 1, policy_version 1486650 (0.0007) [2023-12-27 02:12:06,523][105692] Updated weights for policy 0, policy_version 1484249 (0.0009) [2023-12-27 02:12:06,540][105620] Updated weights for policy 1, policy_version 1486660 (0.0007) [2023-12-27 02:12:06,572][105692] Updated weights for policy 0, policy_version 1484259 (0.0005) [2023-12-27 02:12:06,602][105620] Updated weights for policy 1, policy_version 1486670 (0.0008) [2023-12-27 02:12:07,253][105692] Updated weights for policy 0, policy_version 1484269 (0.0008) [2023-12-27 02:12:07,305][105692] Updated weights for policy 0, policy_version 1484279 (0.0010) [2023-12-27 02:12:07,354][105692] Updated weights for policy 0, policy_version 1484289 (0.0009) [2023-12-27 02:12:07,420][105620] Updated weights for policy 1, policy_version 1486680 (0.0009) [2023-12-27 02:12:07,476][105620] Updated weights for policy 1, policy_version 1486690 (0.0010) [2023-12-27 02:12:07,529][105620] Updated weights for policy 1, policy_version 1486700 (0.0009) [2023-12-27 02:12:07,932][105692] Updated weights for policy 0, policy_version 1484299 (0.0006) [2023-12-27 02:12:07,980][105692] Updated weights for policy 0, policy_version 1484309 (0.0005) [2023-12-27 02:12:08,029][105692] Updated weights for policy 0, policy_version 1484319 (0.0006) [2023-12-27 02:12:08,414][105620] Updated weights for policy 1, policy_version 1486710 (0.0009) [2023-12-27 02:12:08,476][105620] Updated weights for policy 1, policy_version 1486720 (0.0009) [2023-12-27 02:12:08,529][105620] Updated weights for policy 1, policy_version 1486730 (0.0009) [2023-12-27 02:12:08,760][105692] Updated weights for policy 0, policy_version 1484329 (0.0010) [2023-12-27 02:12:08,829][105692] Updated weights for policy 0, policy_version 1484339 (0.0006) [2023-12-27 02:12:08,893][105692] Updated weights for policy 0, policy_version 1484349 (0.0006) [2023-12-27 02:12:08,962][105692] Updated weights for policy 0, policy_version 1484359 (0.0006) [2023-12-27 02:12:09,186][105620] Updated weights for policy 1, policy_version 1486740 (0.0008) [2023-12-27 02:12:09,252][105620] Updated weights for policy 1, policy_version 1486750 (0.0007) [2023-12-27 02:12:09,319][105620] Updated weights for policy 1, policy_version 1486760 (0.0006) [2023-12-27 02:12:09,638][105692] Updated weights for policy 0, policy_version 1484369 (0.0009) [2023-12-27 02:12:09,710][105692] Updated weights for policy 0, policy_version 1484379 (0.0010) [2023-12-27 02:12:09,774][105692] Updated weights for policy 0, policy_version 1484389 (0.0009) [2023-12-27 02:12:09,929][105620] Updated weights for policy 1, policy_version 1486770 (0.0008) [2023-12-27 02:12:09,995][105620] Updated weights for policy 1, policy_version 1486780 (0.0009) [2023-12-27 02:12:10,063][105620] Updated weights for policy 1, policy_version 1486790 (0.0007) [2023-12-27 02:12:10,127][105620] Updated weights for policy 1, policy_version 1486800 (0.0007) [2023-12-27 02:12:10,542][105692] Updated weights for policy 0, policy_version 1484399 (0.0007) [2023-12-27 02:12:10,595][105692] Updated weights for policy 0, policy_version 1484409 (0.0009) [2023-12-27 02:12:10,640][105692] Updated weights for policy 0, policy_version 1484419 (0.0010) [2023-12-27 02:12:10,835][105620] Updated weights for policy 1, policy_version 1486810 (0.0010) [2023-12-27 02:12:10,902][105620] Updated weights for policy 1, policy_version 1486820 (0.0010) [2023-12-27 02:12:10,963][105620] Updated weights for policy 1, policy_version 1486830 (0.0009) [2023-12-27 02:12:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 760750080. Throughput: 0: 9859.1, 1: 9734.0. Samples: 760756064. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:12:11,063][104569] Avg episode reward: [(0, '8530.324'), (1, '9171.258')] [2023-12-27 02:12:11,377][105692] Updated weights for policy 0, policy_version 1484429 (0.0010) [2023-12-27 02:12:11,434][105692] Updated weights for policy 0, policy_version 1484439 (0.0009) [2023-12-27 02:12:11,486][105692] Updated weights for policy 0, policy_version 1484449 (0.0009) [2023-12-27 02:12:11,708][105620] Updated weights for policy 1, policy_version 1486840 (0.0008) [2023-12-27 02:12:11,783][105620] Updated weights for policy 1, policy_version 1486850 (0.0009) [2023-12-27 02:12:11,832][105620] Updated weights for policy 1, policy_version 1486860 (0.0008) [2023-12-27 02:12:12,241][105692] Updated weights for policy 0, policy_version 1484459 (0.0008) [2023-12-27 02:12:12,299][105692] Updated weights for policy 0, policy_version 1484469 (0.0008) [2023-12-27 02:12:12,355][105692] Updated weights for policy 0, policy_version 1484479 (0.0008) [2023-12-27 02:12:12,635][105620] Updated weights for policy 1, policy_version 1486870 (0.0009) [2023-12-27 02:12:12,703][105620] Updated weights for policy 1, policy_version 1486880 (0.0010) [2023-12-27 02:12:12,762][105620] Updated weights for policy 1, policy_version 1486890 (0.0010) [2023-12-27 02:12:13,003][105692] Updated weights for policy 0, policy_version 1484489 (0.0008) [2023-12-27 02:12:13,053][105692] Updated weights for policy 0, policy_version 1484499 (0.0009) [2023-12-27 02:12:13,100][105692] Updated weights for policy 0, policy_version 1484509 (0.0008) [2023-12-27 02:12:13,150][105692] Updated weights for policy 0, policy_version 1484519 (0.0009) [2023-12-27 02:12:13,541][105620] Updated weights for policy 1, policy_version 1486900 (0.0007) [2023-12-27 02:12:13,609][105620] Updated weights for policy 1, policy_version 1486910 (0.0005) [2023-12-27 02:12:13,671][105620] Updated weights for policy 1, policy_version 1486920 (0.0007) [2023-12-27 02:12:13,936][105692] Updated weights for policy 0, policy_version 1484529 (0.0005) [2023-12-27 02:12:13,988][105692] Updated weights for policy 0, policy_version 1484539 (0.0005) [2023-12-27 02:12:14,035][105692] Updated weights for policy 0, policy_version 1484549 (0.0005) [2023-12-27 02:12:14,247][105620] Updated weights for policy 1, policy_version 1486930 (0.0009) [2023-12-27 02:12:14,312][105620] Updated weights for policy 1, policy_version 1486940 (0.0010) [2023-12-27 02:12:14,363][105620] Updated weights for policy 1, policy_version 1486950 (0.0010) [2023-12-27 02:12:14,416][105620] Updated weights for policy 1, policy_version 1486960 (0.0010) [2023-12-27 02:12:14,639][105692] Updated weights for policy 0, policy_version 1484559 (0.0007) [2023-12-27 02:12:14,706][105692] Updated weights for policy 0, policy_version 1484569 (0.0008) [2023-12-27 02:12:14,775][105692] Updated weights for policy 0, policy_version 1484579 (0.0009) [2023-12-27 02:12:15,115][105620] Updated weights for policy 1, policy_version 1486970 (0.0010) [2023-12-27 02:12:15,174][105620] Updated weights for policy 1, policy_version 1486980 (0.0006) [2023-12-27 02:12:15,234][105620] Updated weights for policy 1, policy_version 1486990 (0.0005) [2023-12-27 02:12:15,543][105692] Updated weights for policy 0, policy_version 1484589 (0.0008) [2023-12-27 02:12:15,599][105692] Updated weights for policy 0, policy_version 1484599 (0.0008) [2023-12-27 02:12:15,655][105692] Updated weights for policy 0, policy_version 1484609 (0.0008) [2023-12-27 02:12:15,954][105620] Updated weights for policy 1, policy_version 1487000 (0.0009) [2023-12-27 02:12:16,008][105620] Updated weights for policy 1, policy_version 1487011 (0.0010) [2023-12-27 02:12:16,016][105586] KL-divergence is very high: 102.5729 [2023-12-27 02:12:16,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19524.4, 300 sec: 19521.9). Total num frames: 760840192. Throughput: 0: 9851.6, 1: 9619.7. Samples: 760812676. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:12:16,062][104569] Avg episode reward: [(0, '7980.250'), (1, '9080.177')] [2023-12-27 02:12:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001484616_380116992.pth... [2023-12-27 02:12:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001483432_379813888.pth [2023-12-27 02:12:16,074][105620] Updated weights for policy 1, policy_version 1487021 (0.0009) [2023-12-27 02:12:16,089][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001487024_380731392.pth... [2023-12-27 02:12:16,094][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001485904_380444672.pth [2023-12-27 02:12:16,367][105692] Updated weights for policy 0, policy_version 1484619 (0.0008) [2023-12-27 02:12:16,413][105692] Updated weights for policy 0, policy_version 1484629 (0.0005) [2023-12-27 02:12:16,461][105692] Updated weights for policy 0, policy_version 1484639 (0.0005) [2023-12-27 02:12:16,722][105620] Updated weights for policy 1, policy_version 1487031 (0.0007) [2023-12-27 02:12:16,770][105620] Updated weights for policy 1, policy_version 1487041 (0.0005) [2023-12-27 02:12:16,830][105620] Updated weights for policy 1, policy_version 1487051 (0.0005) [2023-12-27 02:12:17,088][105692] Updated weights for policy 0, policy_version 1484649 (0.0005) [2023-12-27 02:12:17,133][105692] Updated weights for policy 0, policy_version 1484659 (0.0005) [2023-12-27 02:12:17,184][105692] Updated weights for policy 0, policy_version 1484669 (0.0010) [2023-12-27 02:12:17,239][105692] Updated weights for policy 0, policy_version 1484679 (0.0010) [2023-12-27 02:12:17,399][105620] Updated weights for policy 1, policy_version 1487061 (0.0007) [2023-12-27 02:12:17,465][105620] Updated weights for policy 1, policy_version 1487071 (0.0008) [2023-12-27 02:12:17,528][105620] Updated weights for policy 1, policy_version 1487081 (0.0010) [2023-12-27 02:12:17,866][105692] Updated weights for policy 0, policy_version 1484689 (0.0009) [2023-12-27 02:12:17,927][105692] Updated weights for policy 0, policy_version 1484699 (0.0009) [2023-12-27 02:12:17,978][105692] Updated weights for policy 0, policy_version 1484709 (0.0009) [2023-12-27 02:12:18,311][105620] Updated weights for policy 1, policy_version 1487091 (0.0010) [2023-12-27 02:12:18,377][105620] Updated weights for policy 1, policy_version 1487101 (0.0009) [2023-12-27 02:12:18,428][105620] Updated weights for policy 1, policy_version 1487111 (0.0009) [2023-12-27 02:12:18,727][105692] Updated weights for policy 0, policy_version 1484719 (0.0007) [2023-12-27 02:12:18,783][105692] Updated weights for policy 0, policy_version 1484729 (0.0006) [2023-12-27 02:12:18,835][105692] Updated weights for policy 0, policy_version 1484739 (0.0009) [2023-12-27 02:12:19,221][105620] Updated weights for policy 1, policy_version 1487121 (0.0010) [2023-12-27 02:12:19,281][105620] Updated weights for policy 1, policy_version 1487131 (0.0009) [2023-12-27 02:12:19,353][105620] Updated weights for policy 1, policy_version 1487141 (0.0009) [2023-12-27 02:12:19,421][105620] Updated weights for policy 1, policy_version 1487151 (0.0010) [2023-12-27 02:12:19,520][105692] Updated weights for policy 0, policy_version 1484749 (0.0009) [2023-12-27 02:12:19,584][105692] Updated weights for policy 0, policy_version 1484759 (0.0009) [2023-12-27 02:12:19,644][105692] Updated weights for policy 0, policy_version 1484769 (0.0007) [2023-12-27 02:12:20,121][105620] Updated weights for policy 1, policy_version 1487161 (0.0009) [2023-12-27 02:12:20,184][105620] Updated weights for policy 1, policy_version 1487171 (0.0009) [2023-12-27 02:12:20,239][105620] Updated weights for policy 1, policy_version 1487181 (0.0009) [2023-12-27 02:12:20,412][105692] Updated weights for policy 0, policy_version 1484779 (0.0009) [2023-12-27 02:12:20,470][105692] Updated weights for policy 0, policy_version 1484789 (0.0009) [2023-12-27 02:12:20,517][105692] Updated weights for policy 0, policy_version 1484799 (0.0008) [2023-12-27 02:12:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 760938496. Throughput: 0: 9903.0, 1: 9644.0. Samples: 760934096. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:12:21,062][104569] Avg episode reward: [(0, '8257.227'), (1, '9172.657')] [2023-12-27 02:12:21,081][105620] Updated weights for policy 1, policy_version 1487191 (0.0009) [2023-12-27 02:12:21,148][105620] Updated weights for policy 1, policy_version 1487201 (0.0009) [2023-12-27 02:12:21,195][105692] Updated weights for policy 0, policy_version 1484809 (0.0008) [2023-12-27 02:12:21,215][105620] Updated weights for policy 1, policy_version 1487211 (0.0009) [2023-12-27 02:12:21,260][105692] Updated weights for policy 0, policy_version 1484819 (0.0007) [2023-12-27 02:12:21,312][105692] Updated weights for policy 0, policy_version 1484829 (0.0006) [2023-12-27 02:12:21,385][105692] Updated weights for policy 0, policy_version 1484839 (0.0011) [2023-12-27 02:12:22,019][105620] Updated weights for policy 1, policy_version 1487221 (0.0009) [2023-12-27 02:12:22,071][105620] Updated weights for policy 1, policy_version 1487231 (0.0009) [2023-12-27 02:12:22,109][105692] Updated weights for policy 0, policy_version 1484849 (0.0010) [2023-12-27 02:12:22,123][105620] Updated weights for policy 1, policy_version 1487241 (0.0006) [2023-12-27 02:12:22,169][105692] Updated weights for policy 0, policy_version 1484859 (0.0011) [2023-12-27 02:12:22,229][105692] Updated weights for policy 0, policy_version 1484869 (0.0010) [2023-12-27 02:12:22,928][105620] Updated weights for policy 1, policy_version 1487251 (0.0007) [2023-12-27 02:12:22,954][105692] Updated weights for policy 0, policy_version 1484879 (0.0011) [2023-12-27 02:12:22,988][105620] Updated weights for policy 1, policy_version 1487261 (0.0005) [2023-12-27 02:12:23,009][105692] Updated weights for policy 0, policy_version 1484889 (0.0010) [2023-12-27 02:12:23,047][105620] Updated weights for policy 1, policy_version 1487271 (0.0006) [2023-12-27 02:12:23,061][105692] Updated weights for policy 0, policy_version 1484899 (0.0010) [2023-12-27 02:12:23,824][105620] Updated weights for policy 1, policy_version 1487281 (0.0006) [2023-12-27 02:12:23,830][105692] Updated weights for policy 0, policy_version 1484909 (0.0010) [2023-12-27 02:12:23,886][105620] Updated weights for policy 1, policy_version 1487291 (0.0006) [2023-12-27 02:12:23,888][105692] Updated weights for policy 0, policy_version 1484919 (0.0010) [2023-12-27 02:12:23,933][105620] Updated weights for policy 1, policy_version 1487301 (0.0005) [2023-12-27 02:12:23,939][105692] Updated weights for policy 0, policy_version 1484929 (0.0010) [2023-12-27 02:12:23,993][105620] Updated weights for policy 1, policy_version 1487311 (0.0007) [2023-12-27 02:12:24,678][105692] Updated weights for policy 0, policy_version 1484939 (0.0010) [2023-12-27 02:12:24,733][105692] Updated weights for policy 0, policy_version 1484949 (0.0011) [2023-12-27 02:12:24,736][105620] Updated weights for policy 1, policy_version 1487321 (0.0008) [2023-12-27 02:12:24,783][105692] Updated weights for policy 0, policy_version 1484959 (0.0010) [2023-12-27 02:12:24,789][105620] Updated weights for policy 1, policy_version 1487331 (0.0005) [2023-12-27 02:12:24,838][105620] Updated weights for policy 1, policy_version 1487341 (0.0006) [2023-12-27 02:12:25,397][105692] Updated weights for policy 0, policy_version 1484969 (0.0010) [2023-12-27 02:12:25,464][105692] Updated weights for policy 0, policy_version 1484979 (0.0005) [2023-12-27 02:12:25,502][105620] Updated weights for policy 1, policy_version 1487351 (0.0006) [2023-12-27 02:12:25,529][105692] Updated weights for policy 0, policy_version 1484989 (0.0005) [2023-12-27 02:12:25,561][105620] Updated weights for policy 1, policy_version 1487361 (0.0005) [2023-12-27 02:12:25,588][105692] Updated weights for policy 0, policy_version 1484999 (0.0005) [2023-12-27 02:12:25,627][105620] Updated weights for policy 1, policy_version 1487371 (0.0006) [2023-12-27 02:12:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 761036800. Throughput: 0: 10017.2, 1: 9501.8. Samples: 761047420. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:12:26,063][104569] Avg episode reward: [(0, '8352.837'), (1, '9083.312')] [2023-12-27 02:12:26,069][105692] Updated weights for policy 0, policy_version 1485009 (0.0005) [2023-12-27 02:12:26,125][105692] Updated weights for policy 0, policy_version 1485019 (0.0005) [2023-12-27 02:12:26,196][105692] Updated weights for policy 0, policy_version 1485029 (0.0005) [2023-12-27 02:12:26,405][105620] Updated weights for policy 1, policy_version 1487381 (0.0007) [2023-12-27 02:12:26,463][105620] Updated weights for policy 1, policy_version 1487391 (0.0006) [2023-12-27 02:12:26,514][105620] Updated weights for policy 1, policy_version 1487401 (0.0006) [2023-12-27 02:12:26,715][105692] Updated weights for policy 0, policy_version 1485039 (0.0005) [2023-12-27 02:12:26,772][105692] Updated weights for policy 0, policy_version 1485049 (0.0007) [2023-12-27 02:12:26,830][105692] Updated weights for policy 0, policy_version 1485059 (0.0010) [2023-12-27 02:12:27,117][105620] Updated weights for policy 1, policy_version 1487411 (0.0005) [2023-12-27 02:12:27,170][105620] Updated weights for policy 1, policy_version 1487421 (0.0005) [2023-12-27 02:12:27,233][105620] Updated weights for policy 1, policy_version 1487431 (0.0006) [2023-12-27 02:12:27,433][105692] Updated weights for policy 0, policy_version 1485069 (0.0010) [2023-12-27 02:12:27,487][105692] Updated weights for policy 0, policy_version 1485079 (0.0010) [2023-12-27 02:12:27,534][105692] Updated weights for policy 0, policy_version 1485089 (0.0010) [2023-12-27 02:12:27,816][105620] Updated weights for policy 1, policy_version 1487441 (0.0006) [2023-12-27 02:12:27,868][105620] Updated weights for policy 1, policy_version 1487451 (0.0009) [2023-12-27 02:12:27,922][105620] Updated weights for policy 1, policy_version 1487461 (0.0010) [2023-12-27 02:12:27,979][105620] Updated weights for policy 1, policy_version 1487471 (0.0010) [2023-12-27 02:12:28,204][105692] Updated weights for policy 0, policy_version 1485099 (0.0010) [2023-12-27 02:12:28,256][105692] Updated weights for policy 0, policy_version 1485109 (0.0010) [2023-12-27 02:12:28,300][105692] Updated weights for policy 0, policy_version 1485119 (0.0010) [2023-12-27 02:12:28,695][105620] Updated weights for policy 1, policy_version 1487481 (0.0008) [2023-12-27 02:12:28,762][105620] Updated weights for policy 1, policy_version 1487491 (0.0009) [2023-12-27 02:12:28,833][105620] Updated weights for policy 1, policy_version 1487501 (0.0009) [2023-12-27 02:12:28,930][105692] Updated weights for policy 0, policy_version 1485129 (0.0009) [2023-12-27 02:12:28,996][105692] Updated weights for policy 0, policy_version 1485139 (0.0008) [2023-12-27 02:12:29,056][105692] Updated weights for policy 0, policy_version 1485149 (0.0006) [2023-12-27 02:12:29,112][105692] Updated weights for policy 0, policy_version 1485159 (0.0005) [2023-12-27 02:12:29,642][105620] Updated weights for policy 1, policy_version 1487511 (0.0008) [2023-12-27 02:12:29,694][105620] Updated weights for policy 1, policy_version 1487521 (0.0008) [2023-12-27 02:12:29,739][105620] Updated weights for policy 1, policy_version 1487531 (0.0008) [2023-12-27 02:12:29,797][105692] Updated weights for policy 0, policy_version 1485169 (0.0010) [2023-12-27 02:12:29,862][105692] Updated weights for policy 0, policy_version 1485179 (0.0011) [2023-12-27 02:12:29,922][105692] Updated weights for policy 0, policy_version 1485189 (0.0011) [2023-12-27 02:12:30,420][105620] Updated weights for policy 1, policy_version 1487541 (0.0009) [2023-12-27 02:12:30,468][105620] Updated weights for policy 1, policy_version 1487551 (0.0008) [2023-12-27 02:12:30,520][105620] Updated weights for policy 1, policy_version 1487561 (0.0008) [2023-12-27 02:12:30,643][105692] Updated weights for policy 0, policy_version 1485199 (0.0010) [2023-12-27 02:12:30,701][105692] Updated weights for policy 0, policy_version 1485209 (0.0010) [2023-12-27 02:12:30,760][105692] Updated weights for policy 0, policy_version 1485219 (0.0009) [2023-12-27 02:12:31,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 761143296. Throughput: 0: 10184.7, 1: 9526.5. Samples: 761112888. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:12:31,063][104569] Avg episode reward: [(0, '8354.240'), (1, '9171.735')] [2023-12-27 02:12:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001485224_380272640.pth... [2023-12-27 02:12:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001487568_380870656.pth... [2023-12-27 02:12:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001484040_379969536.pth [2023-12-27 02:12:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001486448_380583936.pth [2023-12-27 02:12:31,169][105620] Updated weights for policy 1, policy_version 1487571 (0.0008) [2023-12-27 02:12:31,218][105620] Updated weights for policy 1, policy_version 1487581 (0.0010) [2023-12-27 02:12:31,273][105620] Updated weights for policy 1, policy_version 1487591 (0.0010) [2023-12-27 02:12:31,459][105692] Updated weights for policy 0, policy_version 1485229 (0.0010) [2023-12-27 02:12:31,520][105692] Updated weights for policy 0, policy_version 1485239 (0.0010) [2023-12-27 02:12:31,579][105692] Updated weights for policy 0, policy_version 1485249 (0.0010) [2023-12-27 02:12:31,941][105620] Updated weights for policy 1, policy_version 1487601 (0.0010) [2023-12-27 02:12:31,997][105620] Updated weights for policy 1, policy_version 1487611 (0.0005) [2023-12-27 02:12:32,050][105620] Updated weights for policy 1, policy_version 1487621 (0.0007) [2023-12-27 02:12:32,098][105620] Updated weights for policy 1, policy_version 1487631 (0.0010) [2023-12-27 02:12:32,260][105692] Updated weights for policy 0, policy_version 1485259 (0.0008) [2023-12-27 02:12:32,311][105692] Updated weights for policy 0, policy_version 1485269 (0.0006) [2023-12-27 02:12:32,372][105692] Updated weights for policy 0, policy_version 1485279 (0.0007) [2023-12-27 02:12:32,708][105620] Updated weights for policy 1, policy_version 1487641 (0.0010) [2023-12-27 02:12:32,761][105620] Updated weights for policy 1, policy_version 1487651 (0.0010) [2023-12-27 02:12:32,827][105620] Updated weights for policy 1, policy_version 1487661 (0.0010) [2023-12-27 02:12:32,963][105692] Updated weights for policy 0, policy_version 1485289 (0.0009) [2023-12-27 02:12:33,011][105692] Updated weights for policy 0, policy_version 1485299 (0.0008) [2023-12-27 02:12:33,055][105692] Updated weights for policy 0, policy_version 1485309 (0.0008) [2023-12-27 02:12:33,099][105692] Updated weights for policy 0, policy_version 1485319 (0.0007) [2023-12-27 02:12:33,499][105620] Updated weights for policy 1, policy_version 1487671 (0.0010) [2023-12-27 02:12:33,543][105620] Updated weights for policy 1, policy_version 1487681 (0.0010) [2023-12-27 02:12:33,588][105620] Updated weights for policy 1, policy_version 1487691 (0.0009) [2023-12-27 02:12:33,961][105692] Updated weights for policy 0, policy_version 1485329 (0.0010) [2023-12-27 02:12:34,018][105692] Updated weights for policy 0, policy_version 1485339 (0.0008) [2023-12-27 02:12:34,082][105692] Updated weights for policy 0, policy_version 1485349 (0.0008) [2023-12-27 02:12:34,189][105620] Updated weights for policy 1, policy_version 1487701 (0.0006) [2023-12-27 02:12:34,249][105620] Updated weights for policy 1, policy_version 1487711 (0.0006) [2023-12-27 02:12:34,305][105620] Updated weights for policy 1, policy_version 1487721 (0.0006) [2023-12-27 02:12:34,906][105692] Updated weights for policy 0, policy_version 1485359 (0.0009) [2023-12-27 02:12:34,962][105692] Updated weights for policy 0, policy_version 1485369 (0.0009) [2023-12-27 02:12:34,981][105620] Updated weights for policy 1, policy_version 1487731 (0.0009) [2023-12-27 02:12:35,019][105692] Updated weights for policy 0, policy_version 1485379 (0.0006) [2023-12-27 02:12:35,036][105620] Updated weights for policy 1, policy_version 1487741 (0.0010) [2023-12-27 02:12:35,073][105586] KL-divergence is very high: 105.7259 [2023-12-27 02:12:35,089][105586] KL-divergence is very high: 104.7915 [2023-12-27 02:12:35,101][105620] Updated weights for policy 1, policy_version 1487751 (0.0010) [2023-12-27 02:12:35,122][105586] KL-divergence is very high: 103.3525 [2023-12-27 02:12:35,745][105692] Updated weights for policy 0, policy_version 1485389 (0.0006) [2023-12-27 02:12:35,814][105692] Updated weights for policy 0, policy_version 1485399 (0.0006) [2023-12-27 02:12:35,840][105620] Updated weights for policy 1, policy_version 1487761 (0.0010) [2023-12-27 02:12:35,877][105692] Updated weights for policy 0, policy_version 1485409 (0.0007) [2023-12-27 02:12:35,902][105620] Updated weights for policy 1, policy_version 1487771 (0.0006) [2023-12-27 02:12:35,966][105620] Updated weights for policy 1, policy_version 1487781 (0.0006) [2023-12-27 02:12:36,018][105620] Updated weights for policy 1, policy_version 1487791 (0.0005) [2023-12-27 02:12:36,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 761249792. Throughput: 0: 10087.3, 1: 9625.0. Samples: 761234068. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:12:36,063][104569] Avg episode reward: [(0, '8714.124'), (1, '8179.035')] [2023-12-27 02:12:36,480][105692] Updated weights for policy 0, policy_version 1485419 (0.0005) [2023-12-27 02:12:36,544][105692] Updated weights for policy 0, policy_version 1485429 (0.0008) [2023-12-27 02:12:36,601][105692] Updated weights for policy 0, policy_version 1485439 (0.0006) [2023-12-27 02:12:36,626][105620] Updated weights for policy 1, policy_version 1487801 (0.0010) [2023-12-27 02:12:36,690][105620] Updated weights for policy 1, policy_version 1487811 (0.0010) [2023-12-27 02:12:36,757][105620] Updated weights for policy 1, policy_version 1487821 (0.0005) [2023-12-27 02:12:37,303][105620] Updated weights for policy 1, policy_version 1487831 (0.0009) [2023-12-27 02:12:37,351][105620] Updated weights for policy 1, policy_version 1487841 (0.0010) [2023-12-27 02:12:37,399][105620] Updated weights for policy 1, policy_version 1487851 (0.0010) [2023-12-27 02:12:37,409][105692] Updated weights for policy 0, policy_version 1485449 (0.0008) [2023-12-27 02:12:37,463][105692] Updated weights for policy 0, policy_version 1485459 (0.0005) [2023-12-27 02:12:37,535][105692] Updated weights for policy 0, policy_version 1485469 (0.0006) [2023-12-27 02:12:37,588][105692] Updated weights for policy 0, policy_version 1485479 (0.0005) [2023-12-27 02:12:38,051][105620] Updated weights for policy 1, policy_version 1487861 (0.0010) [2023-12-27 02:12:38,107][105620] Updated weights for policy 1, policy_version 1487871 (0.0009) [2023-12-27 02:12:38,142][105692] Updated weights for policy 0, policy_version 1485489 (0.0007) [2023-12-27 02:12:38,171][105620] Updated weights for policy 1, policy_version 1487881 (0.0005) [2023-12-27 02:12:38,203][105692] Updated weights for policy 0, policy_version 1485499 (0.0006) [2023-12-27 02:12:38,269][105692] Updated weights for policy 0, policy_version 1485509 (0.0008) [2023-12-27 02:12:38,829][105692] Updated weights for policy 0, policy_version 1485519 (0.0010) [2023-12-27 02:12:38,850][105620] Updated weights for policy 1, policy_version 1487891 (0.0007) [2023-12-27 02:12:38,880][105692] Updated weights for policy 0, policy_version 1485529 (0.0010) [2023-12-27 02:12:38,913][105620] Updated weights for policy 1, policy_version 1487901 (0.0006) [2023-12-27 02:12:38,938][105692] Updated weights for policy 0, policy_version 1485539 (0.0011) [2023-12-27 02:12:38,969][105620] Updated weights for policy 1, policy_version 1487911 (0.0010) [2023-12-27 02:12:39,655][105692] Updated weights for policy 0, policy_version 1485549 (0.0008) [2023-12-27 02:12:39,716][105692] Updated weights for policy 0, policy_version 1485559 (0.0006) [2023-12-27 02:12:39,782][105692] Updated weights for policy 0, policy_version 1485569 (0.0010) [2023-12-27 02:12:39,805][105620] Updated weights for policy 1, policy_version 1487921 (0.0010) [2023-12-27 02:12:39,877][105620] Updated weights for policy 1, policy_version 1487931 (0.0008) [2023-12-27 02:12:39,944][105620] Updated weights for policy 1, policy_version 1487941 (0.0010) [2023-12-27 02:12:40,010][105620] Updated weights for policy 1, policy_version 1487951 (0.0009) [2023-12-27 02:12:40,522][105692] Updated weights for policy 0, policy_version 1485579 (0.0008) [2023-12-27 02:12:40,571][105692] Updated weights for policy 0, policy_version 1485589 (0.0008) [2023-12-27 02:12:40,618][105692] Updated weights for policy 0, policy_version 1485599 (0.0009) [2023-12-27 02:12:40,746][105620] Updated weights for policy 1, policy_version 1487961 (0.0009) [2023-12-27 02:12:40,797][105620] Updated weights for policy 1, policy_version 1487971 (0.0009) [2023-12-27 02:12:40,849][105620] Updated weights for policy 1, policy_version 1487981 (0.0009) [2023-12-27 02:12:41,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 761348096. Throughput: 0: 10039.1, 1: 9717.8. Samples: 761353964. Policy #0 lag: (min: 1.0, avg: 7.4, max: 33.0) [2023-12-27 02:12:41,063][104569] Avg episode reward: [(0, '8442.038'), (1, '7994.929')] [2023-12-27 02:12:41,436][105692] Updated weights for policy 0, policy_version 1485609 (0.0009) [2023-12-27 02:12:41,500][105692] Updated weights for policy 0, policy_version 1485619 (0.0009) [2023-12-27 02:12:41,563][105692] Updated weights for policy 0, policy_version 1485629 (0.0009) [2023-12-27 02:12:41,623][105692] Updated weights for policy 0, policy_version 1485639 (0.0009) [2023-12-27 02:12:41,652][105620] Updated weights for policy 1, policy_version 1487991 (0.0008) [2023-12-27 02:12:41,723][105620] Updated weights for policy 1, policy_version 1488001 (0.0008) [2023-12-27 02:12:41,787][105620] Updated weights for policy 1, policy_version 1488011 (0.0008) [2023-12-27 02:12:42,450][105620] Updated weights for policy 1, policy_version 1488021 (0.0006) [2023-12-27 02:12:42,455][105692] Updated weights for policy 0, policy_version 1485649 (0.0010) [2023-12-27 02:12:42,512][105620] Updated weights for policy 1, policy_version 1488031 (0.0006) [2023-12-27 02:12:42,514][105692] Updated weights for policy 0, policy_version 1485659 (0.0007) [2023-12-27 02:12:42,570][105620] Updated weights for policy 1, policy_version 1488041 (0.0007) [2023-12-27 02:12:42,572][105692] Updated weights for policy 0, policy_version 1485669 (0.0006) [2023-12-27 02:12:43,287][105620] Updated weights for policy 1, policy_version 1488051 (0.0007) [2023-12-27 02:12:43,337][105620] Updated weights for policy 1, policy_version 1488061 (0.0006) [2023-12-27 02:12:43,359][105692] Updated weights for policy 0, policy_version 1485679 (0.0008) [2023-12-27 02:12:43,385][105620] Updated weights for policy 1, policy_version 1488071 (0.0006) [2023-12-27 02:12:43,418][105692] Updated weights for policy 0, policy_version 1485689 (0.0009) [2023-12-27 02:12:43,478][105692] Updated weights for policy 0, policy_version 1485699 (0.0009) [2023-12-27 02:12:43,931][105620] Updated weights for policy 1, policy_version 1488081 (0.0007) [2023-12-27 02:12:43,985][105620] Updated weights for policy 1, policy_version 1488091 (0.0009) [2023-12-27 02:12:44,050][105620] Updated weights for policy 1, policy_version 1488101 (0.0009) [2023-12-27 02:12:44,107][105620] Updated weights for policy 1, policy_version 1488111 (0.0008) [2023-12-27 02:12:44,321][105692] Updated weights for policy 0, policy_version 1485709 (0.0009) [2023-12-27 02:12:44,382][105692] Updated weights for policy 0, policy_version 1485719 (0.0009) [2023-12-27 02:12:44,451][105692] Updated weights for policy 0, policy_version 1485729 (0.0009) [2023-12-27 02:12:44,846][105620] Updated weights for policy 1, policy_version 1488121 (0.0006) [2023-12-27 02:12:44,909][105620] Updated weights for policy 1, policy_version 1488131 (0.0009) [2023-12-27 02:12:44,965][105620] Updated weights for policy 1, policy_version 1488141 (0.0009) [2023-12-27 02:12:45,138][105692] Updated weights for policy 0, policy_version 1485739 (0.0008) [2023-12-27 02:12:45,209][105692] Updated weights for policy 0, policy_version 1485749 (0.0006) [2023-12-27 02:12:45,281][105692] Updated weights for policy 0, policy_version 1485759 (0.0005) [2023-12-27 02:12:45,621][105620] Updated weights for policy 1, policy_version 1488151 (0.0008) [2023-12-27 02:12:45,667][105620] Updated weights for policy 1, policy_version 1488161 (0.0008) [2023-12-27 02:12:45,714][105620] Updated weights for policy 1, policy_version 1488171 (0.0009) [2023-12-27 02:12:45,947][105692] Updated weights for policy 0, policy_version 1485769 (0.0006) [2023-12-27 02:12:46,004][105692] Updated weights for policy 0, policy_version 1485779 (0.0010) [2023-12-27 02:12:46,062][105692] Updated weights for policy 0, policy_version 1485789 (0.0010) [2023-12-27 02:12:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 761438208. Throughput: 0: 9926.5, 1: 9784.8. Samples: 761410712. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:12:46,063][104569] Avg episode reward: [(0, '8445.802'), (1, '8777.581')] [2023-12-27 02:12:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001488176_381026304.pth... [2023-12-27 02:12:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001487024_380731392.pth [2023-12-27 02:12:46,120][105692] Updated weights for policy 0, policy_version 1485799 (0.0009) [2023-12-27 02:12:46,124][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001485800_380420096.pth... [2023-12-27 02:12:46,127][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001484616_380116992.pth [2023-12-27 02:12:46,385][105620] Updated weights for policy 1, policy_version 1488181 (0.0008) [2023-12-27 02:12:46,432][105620] Updated weights for policy 1, policy_version 1488191 (0.0009) [2023-12-27 02:12:46,485][105620] Updated weights for policy 1, policy_version 1488201 (0.0008) [2023-12-27 02:12:46,928][105692] Updated weights for policy 0, policy_version 1485809 (0.0006) [2023-12-27 02:12:46,979][105692] Updated weights for policy 0, policy_version 1485819 (0.0005) [2023-12-27 02:12:47,034][105692] Updated weights for policy 0, policy_version 1485829 (0.0005) [2023-12-27 02:12:47,096][105620] Updated weights for policy 1, policy_version 1488211 (0.0008) [2023-12-27 02:12:47,145][105620] Updated weights for policy 1, policy_version 1488221 (0.0006) [2023-12-27 02:12:47,196][105620] Updated weights for policy 1, policy_version 1488231 (0.0005) [2023-12-27 02:12:47,588][105692] Updated weights for policy 0, policy_version 1485839 (0.0006) [2023-12-27 02:12:47,643][105692] Updated weights for policy 0, policy_version 1485849 (0.0010) [2023-12-27 02:12:47,702][105692] Updated weights for policy 0, policy_version 1485859 (0.0010) [2023-12-27 02:12:47,718][105620] Updated weights for policy 1, policy_version 1488241 (0.0005) [2023-12-27 02:12:47,771][105620] Updated weights for policy 1, policy_version 1488251 (0.0009) [2023-12-27 02:12:47,827][105620] Updated weights for policy 1, policy_version 1488261 (0.0009) [2023-12-27 02:12:47,890][105620] Updated weights for policy 1, policy_version 1488271 (0.0008) [2023-12-27 02:12:48,434][105692] Updated weights for policy 0, policy_version 1485869 (0.0011) [2023-12-27 02:12:48,493][105692] Updated weights for policy 0, policy_version 1485879 (0.0010) [2023-12-27 02:12:48,548][105692] Updated weights for policy 0, policy_version 1485889 (0.0010) [2023-12-27 02:12:48,570][105620] Updated weights for policy 1, policy_version 1488281 (0.0005) [2023-12-27 02:12:48,632][105620] Updated weights for policy 1, policy_version 1488291 (0.0007) [2023-12-27 02:12:48,693][105620] Updated weights for policy 1, policy_version 1488301 (0.0008) [2023-12-27 02:12:49,243][105692] Updated weights for policy 0, policy_version 1485899 (0.0010) [2023-12-27 02:12:49,305][105692] Updated weights for policy 0, policy_version 1485909 (0.0010) [2023-12-27 02:12:49,376][105692] Updated weights for policy 0, policy_version 1485919 (0.0011) [2023-12-27 02:12:49,461][105620] Updated weights for policy 1, policy_version 1488311 (0.0010) [2023-12-27 02:12:49,521][105620] Updated weights for policy 1, policy_version 1488321 (0.0010) [2023-12-27 02:12:49,576][105620] Updated weights for policy 1, policy_version 1488331 (0.0010) [2023-12-27 02:12:50,094][105692] Updated weights for policy 0, policy_version 1485929 (0.0010) [2023-12-27 02:12:50,152][105692] Updated weights for policy 0, policy_version 1485939 (0.0007) [2023-12-27 02:12:50,215][105692] Updated weights for policy 0, policy_version 1485949 (0.0011) [2023-12-27 02:12:50,279][105692] Updated weights for policy 0, policy_version 1485959 (0.0011) [2023-12-27 02:12:50,349][105620] Updated weights for policy 1, policy_version 1488341 (0.0009) [2023-12-27 02:12:50,412][105620] Updated weights for policy 1, policy_version 1488351 (0.0011) [2023-12-27 02:12:50,479][105620] Updated weights for policy 1, policy_version 1488361 (0.0011) [2023-12-27 02:12:50,969][105692] Updated weights for policy 0, policy_version 1485969 (0.0010) [2023-12-27 02:12:51,029][105692] Updated weights for policy 0, policy_version 1485979 (0.0010) [2023-12-27 02:12:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 761536512. Throughput: 0: 9965.3, 1: 9847.9. Samples: 761530924. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:12:51,062][104569] Avg episode reward: [(0, '8811.090'), (1, '9167.327')] [2023-12-27 02:12:51,089][105692] Updated weights for policy 0, policy_version 1485989 (0.0010) [2023-12-27 02:12:51,154][105620] Updated weights for policy 1, policy_version 1488371 (0.0010) [2023-12-27 02:12:51,203][105620] Updated weights for policy 1, policy_version 1488381 (0.0011) [2023-12-27 02:12:51,263][105620] Updated weights for policy 1, policy_version 1488391 (0.0009) [2023-12-27 02:12:51,882][105692] Updated weights for policy 0, policy_version 1485999 (0.0009) [2023-12-27 02:12:51,947][105692] Updated weights for policy 0, policy_version 1486009 (0.0009) [2023-12-27 02:12:51,998][105692] Updated weights for policy 0, policy_version 1486019 (0.0009) [2023-12-27 02:12:52,014][105620] Updated weights for policy 1, policy_version 1488401 (0.0009) [2023-12-27 02:12:52,063][105620] Updated weights for policy 1, policy_version 1488411 (0.0008) [2023-12-27 02:12:52,110][105620] Updated weights for policy 1, policy_version 1488421 (0.0009) [2023-12-27 02:12:52,161][105620] Updated weights for policy 1, policy_version 1488431 (0.0009) [2023-12-27 02:12:52,794][105692] Updated weights for policy 0, policy_version 1486029 (0.0008) [2023-12-27 02:12:52,853][105692] Updated weights for policy 0, policy_version 1486039 (0.0009) [2023-12-27 02:12:52,897][105620] Updated weights for policy 1, policy_version 1488441 (0.0006) [2023-12-27 02:12:52,917][105692] Updated weights for policy 0, policy_version 1486049 (0.0009) [2023-12-27 02:12:52,944][105620] Updated weights for policy 1, policy_version 1488451 (0.0006) [2023-12-27 02:12:52,993][105620] Updated weights for policy 1, policy_version 1488461 (0.0005) [2023-12-27 02:12:53,716][105692] Updated weights for policy 0, policy_version 1486059 (0.0009) [2023-12-27 02:12:53,718][105620] Updated weights for policy 1, policy_version 1488471 (0.0007) [2023-12-27 02:12:53,773][105692] Updated weights for policy 0, policy_version 1486069 (0.0006) [2023-12-27 02:12:53,781][105620] Updated weights for policy 1, policy_version 1488481 (0.0009) [2023-12-27 02:12:53,821][105692] Updated weights for policy 0, policy_version 1486079 (0.0006) [2023-12-27 02:12:53,832][105620] Updated weights for policy 1, policy_version 1488491 (0.0009) [2023-12-27 02:12:54,496][105692] Updated weights for policy 0, policy_version 1486089 (0.0008) [2023-12-27 02:12:54,549][105692] Updated weights for policy 0, policy_version 1486099 (0.0009) [2023-12-27 02:12:54,580][105620] Updated weights for policy 1, policy_version 1488501 (0.0006) [2023-12-27 02:12:54,610][105692] Updated weights for policy 0, policy_version 1486109 (0.0010) [2023-12-27 02:12:54,646][105620] Updated weights for policy 1, policy_version 1488511 (0.0005) [2023-12-27 02:12:54,670][105692] Updated weights for policy 0, policy_version 1486119 (0.0008) [2023-12-27 02:12:54,698][105620] Updated weights for policy 1, policy_version 1488521 (0.0006) [2023-12-27 02:12:55,384][105620] Updated weights for policy 1, policy_version 1488531 (0.0008) [2023-12-27 02:12:55,426][105692] Updated weights for policy 0, policy_version 1486129 (0.0007) [2023-12-27 02:12:55,436][105620] Updated weights for policy 1, policy_version 1488541 (0.0007) [2023-12-27 02:12:55,475][105692] Updated weights for policy 0, policy_version 1486139 (0.0006) [2023-12-27 02:12:55,492][105620] Updated weights for policy 1, policy_version 1488551 (0.0006) [2023-12-27 02:12:55,528][105692] Updated weights for policy 0, policy_version 1486149 (0.0010) [2023-12-27 02:12:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 761634816. Throughput: 0: 9842.9, 1: 9894.6. Samples: 761644252. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:12:56,062][104569] Avg episode reward: [(0, '8714.169'), (1, '9262.810')] [2023-12-27 02:12:56,152][105692] Updated weights for policy 0, policy_version 1486159 (0.0010) [2023-12-27 02:12:56,208][105692] Updated weights for policy 0, policy_version 1486169 (0.0008) [2023-12-27 02:12:56,270][105692] Updated weights for policy 0, policy_version 1486179 (0.0006) [2023-12-27 02:12:56,310][105620] Updated weights for policy 1, policy_version 1488561 (0.0007) [2023-12-27 02:12:56,368][105620] Updated weights for policy 1, policy_version 1488572 (0.0010) [2023-12-27 02:12:56,425][105620] Updated weights for policy 1, policy_version 1488582 (0.0009) [2023-12-27 02:12:56,478][105620] Updated weights for policy 1, policy_version 1488592 (0.0009) [2023-12-27 02:12:56,813][105692] Updated weights for policy 0, policy_version 1486189 (0.0006) [2023-12-27 02:12:56,869][105692] Updated weights for policy 0, policy_version 1486199 (0.0005) [2023-12-27 02:12:56,930][105692] Updated weights for policy 0, policy_version 1486209 (0.0005) [2023-12-27 02:12:57,359][105620] Updated weights for policy 1, policy_version 1488602 (0.0008) [2023-12-27 02:12:57,415][105620] Updated weights for policy 1, policy_version 1488613 (0.0010) [2023-12-27 02:12:57,476][105620] Updated weights for policy 1, policy_version 1488623 (0.0010) [2023-12-27 02:12:57,525][105692] Updated weights for policy 0, policy_version 1486219 (0.0005) [2023-12-27 02:12:57,588][105692] Updated weights for policy 0, policy_version 1486229 (0.0007) [2023-12-27 02:12:57,655][105692] Updated weights for policy 0, policy_version 1486239 (0.0009) [2023-12-27 02:12:58,268][105692] Updated weights for policy 0, policy_version 1486249 (0.0009) [2023-12-27 02:12:58,315][105620] Updated weights for policy 1, policy_version 1488633 (0.0006) [2023-12-27 02:12:58,332][105692] Updated weights for policy 0, policy_version 1486259 (0.0008) [2023-12-27 02:12:58,386][105620] Updated weights for policy 1, policy_version 1488643 (0.0008) [2023-12-27 02:12:58,395][105692] Updated weights for policy 0, policy_version 1486269 (0.0009) [2023-12-27 02:12:58,456][105620] Updated weights for policy 1, policy_version 1488653 (0.0009) [2023-12-27 02:12:58,458][105692] Updated weights for policy 0, policy_version 1486279 (0.0007) [2023-12-27 02:12:59,181][105692] Updated weights for policy 0, policy_version 1486289 (0.0010) [2023-12-27 02:12:59,232][105620] Updated weights for policy 1, policy_version 1488663 (0.0007) [2023-12-27 02:12:59,235][105692] Updated weights for policy 0, policy_version 1486299 (0.0010) [2023-12-27 02:12:59,290][105620] Updated weights for policy 1, policy_version 1488673 (0.0006) [2023-12-27 02:12:59,296][105692] Updated weights for policy 0, policy_version 1486309 (0.0011) [2023-12-27 02:12:59,349][105620] Updated weights for policy 1, policy_version 1488683 (0.0006) [2023-12-27 02:13:00,031][105692] Updated weights for policy 0, policy_version 1486319 (0.0011) [2023-12-27 02:13:00,098][105692] Updated weights for policy 0, policy_version 1486329 (0.0009) [2023-12-27 02:13:00,147][105692] Updated weights for policy 0, policy_version 1486339 (0.0007) [2023-12-27 02:13:00,181][105620] Updated weights for policy 1, policy_version 1488693 (0.0007) [2023-12-27 02:13:00,236][105620] Updated weights for policy 1, policy_version 1488703 (0.0008) [2023-12-27 02:13:00,294][105620] Updated weights for policy 1, policy_version 1488713 (0.0007) [2023-12-27 02:13:00,806][105692] Updated weights for policy 0, policy_version 1486349 (0.0009) [2023-12-27 02:13:00,865][105692] Updated weights for policy 0, policy_version 1486359 (0.0010) [2023-12-27 02:13:00,927][105692] Updated weights for policy 0, policy_version 1486369 (0.0010) [2023-12-27 02:13:01,035][105620] Updated weights for policy 1, policy_version 1488723 (0.0006) [2023-12-27 02:13:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 761733120. Throughput: 0: 9945.0, 1: 9850.5. Samples: 761703472. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:13:01,062][104569] Avg episode reward: [(0, '8528.498'), (1, '9079.629')] [2023-12-27 02:13:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001486376_380567552.pth... [2023-12-27 02:13:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001485224_380272640.pth [2023-12-27 02:13:01,095][105620] Updated weights for policy 1, policy_version 1488733 (0.0009) [2023-12-27 02:13:01,159][105620] Updated weights for policy 1, policy_version 1488743 (0.0008) [2023-12-27 02:13:01,209][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001488752_381173760.pth... [2023-12-27 02:13:01,214][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001487568_380870656.pth [2023-12-27 02:13:01,663][105692] Updated weights for policy 0, policy_version 1486379 (0.0008) [2023-12-27 02:13:01,727][105692] Updated weights for policy 0, policy_version 1486389 (0.0009) [2023-12-27 02:13:01,787][105692] Updated weights for policy 0, policy_version 1486399 (0.0010) [2023-12-27 02:13:01,902][105620] Updated weights for policy 1, policy_version 1488753 (0.0008) [2023-12-27 02:13:01,970][105620] Updated weights for policy 1, policy_version 1488763 (0.0008) [2023-12-27 02:13:02,023][105620] Updated weights for policy 1, policy_version 1488773 (0.0008) [2023-12-27 02:13:02,077][105620] Updated weights for policy 1, policy_version 1488783 (0.0009) [2023-12-27 02:13:02,554][105692] Updated weights for policy 0, policy_version 1486409 (0.0010) [2023-12-27 02:13:02,610][105692] Updated weights for policy 0, policy_version 1486419 (0.0008) [2023-12-27 02:13:02,671][105692] Updated weights for policy 0, policy_version 1486429 (0.0010) [2023-12-27 02:13:02,715][105692] Updated weights for policy 0, policy_version 1486439 (0.0010) [2023-12-27 02:13:02,792][105620] Updated weights for policy 1, policy_version 1488793 (0.0010) [2023-12-27 02:13:02,840][105620] Updated weights for policy 1, policy_version 1488803 (0.0010) [2023-12-27 02:13:02,887][105620] Updated weights for policy 1, policy_version 1488813 (0.0010) [2023-12-27 02:13:03,326][105692] Updated weights for policy 0, policy_version 1486449 (0.0006) [2023-12-27 02:13:03,377][105692] Updated weights for policy 0, policy_version 1486459 (0.0005) [2023-12-27 02:13:03,428][105692] Updated weights for policy 0, policy_version 1486469 (0.0005) [2023-12-27 02:13:03,613][105620] Updated weights for policy 1, policy_version 1488823 (0.0010) [2023-12-27 02:13:03,657][105620] Updated weights for policy 1, policy_version 1488833 (0.0010) [2023-12-27 02:13:03,701][105620] Updated weights for policy 1, policy_version 1488843 (0.0010) [2023-12-27 02:13:04,023][105692] Updated weights for policy 0, policy_version 1486479 (0.0007) [2023-12-27 02:13:04,082][105692] Updated weights for policy 0, policy_version 1486489 (0.0007) [2023-12-27 02:13:04,139][105692] Updated weights for policy 0, policy_version 1486499 (0.0007) [2023-12-27 02:13:04,492][105620] Updated weights for policy 1, policy_version 1488853 (0.0010) [2023-12-27 02:13:04,548][105620] Updated weights for policy 1, policy_version 1488863 (0.0008) [2023-12-27 02:13:04,607][105620] Updated weights for policy 1, policy_version 1488873 (0.0006) [2023-12-27 02:13:04,869][105692] Updated weights for policy 0, policy_version 1486509 (0.0011) [2023-12-27 02:13:04,922][105692] Updated weights for policy 0, policy_version 1486519 (0.0008) [2023-12-27 02:13:04,984][105692] Updated weights for policy 0, policy_version 1486529 (0.0011) [2023-12-27 02:13:05,227][105620] Updated weights for policy 1, policy_version 1488883 (0.0008) [2023-12-27 02:13:05,292][105620] Updated weights for policy 1, policy_version 1488893 (0.0010) [2023-12-27 02:13:05,346][105620] Updated weights for policy 1, policy_version 1488903 (0.0010) [2023-12-27 02:13:05,669][105692] Updated weights for policy 0, policy_version 1486539 (0.0008) [2023-12-27 02:13:05,717][105692] Updated weights for policy 0, policy_version 1486549 (0.0008) [2023-12-27 02:13:05,771][105692] Updated weights for policy 0, policy_version 1486559 (0.0009) [2023-12-27 02:13:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 761831424. Throughput: 0: 9926.8, 1: 9790.9. Samples: 761821396. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:13:06,063][104569] Avg episode reward: [(0, '8622.466'), (1, '9076.490')] [2023-12-27 02:13:06,071][105620] Updated weights for policy 1, policy_version 1488913 (0.0010) [2023-12-27 02:13:06,139][105620] Updated weights for policy 1, policy_version 1488923 (0.0007) [2023-12-27 02:13:06,201][105620] Updated weights for policy 1, policy_version 1488933 (0.0009) [2023-12-27 02:13:06,258][105620] Updated weights for policy 1, policy_version 1488943 (0.0009) [2023-12-27 02:13:06,496][105692] Updated weights for policy 0, policy_version 1486569 (0.0009) [2023-12-27 02:13:06,552][105692] Updated weights for policy 0, policy_version 1486579 (0.0009) [2023-12-27 02:13:06,607][105692] Updated weights for policy 0, policy_version 1486589 (0.0009) [2023-12-27 02:13:06,670][105692] Updated weights for policy 0, policy_version 1486599 (0.0009) [2023-12-27 02:13:06,995][105620] Updated weights for policy 1, policy_version 1488953 (0.0006) [2023-12-27 02:13:07,050][105620] Updated weights for policy 1, policy_version 1488963 (0.0005) [2023-12-27 02:13:07,113][105620] Updated weights for policy 1, policy_version 1488973 (0.0006) [2023-12-27 02:13:07,472][105692] Updated weights for policy 0, policy_version 1486609 (0.0009) [2023-12-27 02:13:07,526][105692] Updated weights for policy 0, policy_version 1486619 (0.0009) [2023-12-27 02:13:07,583][105692] Updated weights for policy 0, policy_version 1486629 (0.0009) [2023-12-27 02:13:07,760][105620] Updated weights for policy 1, policy_version 1488983 (0.0008) [2023-12-27 02:13:07,821][105620] Updated weights for policy 1, policy_version 1488993 (0.0009) [2023-12-27 02:13:07,885][105620] Updated weights for policy 1, policy_version 1489003 (0.0008) [2023-12-27 02:13:08,246][105692] Updated weights for policy 0, policy_version 1486639 (0.0007) [2023-12-27 02:13:08,309][105692] Updated weights for policy 0, policy_version 1486649 (0.0006) [2023-12-27 02:13:08,380][105692] Updated weights for policy 0, policy_version 1486659 (0.0008) [2023-12-27 02:13:08,537][105620] Updated weights for policy 1, policy_version 1489013 (0.0007) [2023-12-27 02:13:08,584][105620] Updated weights for policy 1, policy_version 1489023 (0.0009) [2023-12-27 02:13:08,637][105620] Updated weights for policy 1, policy_version 1489033 (0.0010) [2023-12-27 02:13:09,164][105692] Updated weights for policy 0, policy_version 1486669 (0.0009) [2023-12-27 02:13:09,223][105692] Updated weights for policy 0, policy_version 1486679 (0.0010) [2023-12-27 02:13:09,243][105620] Updated weights for policy 1, policy_version 1489043 (0.0007) [2023-12-27 02:13:09,291][105692] Updated weights for policy 0, policy_version 1486689 (0.0009) [2023-12-27 02:13:09,305][105620] Updated weights for policy 1, policy_version 1489053 (0.0007) [2023-12-27 02:13:09,367][105620] Updated weights for policy 1, policy_version 1489063 (0.0007) [2023-12-27 02:13:10,100][105692] Updated weights for policy 0, policy_version 1486699 (0.0009) [2023-12-27 02:13:10,167][105692] Updated weights for policy 0, policy_version 1486709 (0.0011) [2023-12-27 02:13:10,184][105620] Updated weights for policy 1, policy_version 1489073 (0.0008) [2023-12-27 02:13:10,227][105692] Updated weights for policy 0, policy_version 1486719 (0.0011) [2023-12-27 02:13:10,248][105620] Updated weights for policy 1, policy_version 1489083 (0.0006) [2023-12-27 02:13:10,304][105620] Updated weights for policy 1, policy_version 1489093 (0.0006) [2023-12-27 02:13:10,365][105620] Updated weights for policy 1, policy_version 1489103 (0.0006) [2023-12-27 02:13:10,860][105692] Updated weights for policy 0, policy_version 1486729 (0.0010) [2023-12-27 02:13:10,921][105692] Updated weights for policy 0, policy_version 1486739 (0.0006) [2023-12-27 02:13:10,976][105620] Updated weights for policy 1, policy_version 1489113 (0.0008) [2023-12-27 02:13:10,984][105692] Updated weights for policy 0, policy_version 1486749 (0.0011) [2023-12-27 02:13:11,029][105620] Updated weights for policy 1, policy_version 1489123 (0.0007) [2023-12-27 02:13:11,051][105692] Updated weights for policy 0, policy_version 1486759 (0.0010) [2023-12-27 02:13:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 761929728. Throughput: 0: 9880.6, 1: 9917.4. Samples: 761938324. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:13:11,062][104569] Avg episode reward: [(0, '8801.673'), (1, '9259.167')] [2023-12-27 02:13:11,093][105620] Updated weights for policy 1, policy_version 1489133 (0.0009) [2023-12-27 02:13:11,775][105692] Updated weights for policy 0, policy_version 1486769 (0.0010) [2023-12-27 02:13:11,831][105692] Updated weights for policy 0, policy_version 1486779 (0.0011) [2023-12-27 02:13:11,847][105620] Updated weights for policy 1, policy_version 1489143 (0.0006) [2023-12-27 02:13:11,898][105692] Updated weights for policy 0, policy_version 1486789 (0.0011) [2023-12-27 02:13:11,909][105620] Updated weights for policy 1, policy_version 1489153 (0.0006) [2023-12-27 02:13:11,972][105620] Updated weights for policy 1, policy_version 1489163 (0.0008) [2023-12-27 02:13:12,661][105692] Updated weights for policy 0, policy_version 1486799 (0.0011) [2023-12-27 02:13:12,710][105692] Updated weights for policy 0, policy_version 1486809 (0.0011) [2023-12-27 02:13:12,711][105620] Updated weights for policy 1, policy_version 1489173 (0.0007) [2023-12-27 02:13:12,762][105692] Updated weights for policy 0, policy_version 1486819 (0.0010) [2023-12-27 02:13:12,776][105620] Updated weights for policy 1, policy_version 1489183 (0.0007) [2023-12-27 02:13:12,826][105620] Updated weights for policy 1, policy_version 1489193 (0.0007) [2023-12-27 02:13:13,380][105692] Updated weights for policy 0, policy_version 1486829 (0.0008) [2023-12-27 02:13:13,394][105620] Updated weights for policy 1, policy_version 1489203 (0.0006) [2023-12-27 02:13:13,439][105692] Updated weights for policy 0, policy_version 1486839 (0.0006) [2023-12-27 02:13:13,459][105620] Updated weights for policy 1, policy_version 1489213 (0.0009) [2023-12-27 02:13:13,494][105692] Updated weights for policy 0, policy_version 1486849 (0.0005) [2023-12-27 02:13:13,526][105620] Updated weights for policy 1, policy_version 1489223 (0.0008) [2023-12-27 02:13:14,038][105692] Updated weights for policy 0, policy_version 1486859 (0.0006) [2023-12-27 02:13:14,097][105692] Updated weights for policy 0, policy_version 1486869 (0.0009) [2023-12-27 02:13:14,159][105692] Updated weights for policy 0, policy_version 1486879 (0.0006) [2023-12-27 02:13:14,414][105620] Updated weights for policy 1, policy_version 1489233 (0.0009) [2023-12-27 02:13:14,480][105620] Updated weights for policy 1, policy_version 1489243 (0.0008) [2023-12-27 02:13:14,543][105620] Updated weights for policy 1, policy_version 1489253 (0.0008) [2023-12-27 02:13:14,604][105620] Updated weights for policy 1, policy_version 1489263 (0.0009) [2023-12-27 02:13:14,710][105692] Updated weights for policy 0, policy_version 1486889 (0.0005) [2023-12-27 02:13:14,769][105692] Updated weights for policy 0, policy_version 1486899 (0.0007) [2023-12-27 02:13:14,831][105692] Updated weights for policy 0, policy_version 1486909 (0.0007) [2023-12-27 02:13:14,890][105692] Updated weights for policy 0, policy_version 1486919 (0.0010) [2023-12-27 02:13:15,395][105620] Updated weights for policy 1, policy_version 1489273 (0.0008) [2023-12-27 02:13:15,451][105620] Updated weights for policy 1, policy_version 1489283 (0.0008) [2023-12-27 02:13:15,509][105620] Updated weights for policy 1, policy_version 1489293 (0.0008) [2023-12-27 02:13:15,574][105692] Updated weights for policy 0, policy_version 1486929 (0.0011) [2023-12-27 02:13:15,636][105692] Updated weights for policy 0, policy_version 1486939 (0.0010) [2023-12-27 02:13:15,702][105692] Updated weights for policy 0, policy_version 1486949 (0.0011) [2023-12-27 02:13:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 762028032. Throughput: 0: 9785.7, 1: 9873.5. Samples: 761997552. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:13:16,062][104569] Avg episode reward: [(0, '8714.307'), (1, '9350.642')] [2023-12-27 02:13:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001486952_380715008.pth... [2023-12-27 02:13:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001489296_381313024.pth... [2023-12-27 02:13:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001485800_380420096.pth [2023-12-27 02:13:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001488176_381026304.pth [2023-12-27 02:13:16,252][105620] Updated weights for policy 1, policy_version 1489303 (0.0006) [2023-12-27 02:13:16,309][105620] Updated weights for policy 1, policy_version 1489313 (0.0005) [2023-12-27 02:13:16,310][105692] Updated weights for policy 0, policy_version 1486959 (0.0007) [2023-12-27 02:13:16,371][105620] Updated weights for policy 1, policy_version 1489323 (0.0008) [2023-12-27 02:13:16,376][105692] Updated weights for policy 0, policy_version 1486969 (0.0006) [2023-12-27 02:13:16,432][105692] Updated weights for policy 0, policy_version 1486979 (0.0008) [2023-12-27 02:13:17,111][105620] Updated weights for policy 1, policy_version 1489333 (0.0009) [2023-12-27 02:13:17,117][105692] Updated weights for policy 0, policy_version 1486989 (0.0008) [2023-12-27 02:13:17,165][105692] Updated weights for policy 0, policy_version 1486999 (0.0006) [2023-12-27 02:13:17,170][105620] Updated weights for policy 1, policy_version 1489343 (0.0008) [2023-12-27 02:13:17,222][105692] Updated weights for policy 0, policy_version 1487009 (0.0009) [2023-12-27 02:13:17,237][105620] Updated weights for policy 1, policy_version 1489353 (0.0007) [2023-12-27 02:13:17,904][105620] Updated weights for policy 1, policy_version 1489363 (0.0009) [2023-12-27 02:13:17,955][105620] Updated weights for policy 1, policy_version 1489373 (0.0009) [2023-12-27 02:13:17,999][105692] Updated weights for policy 0, policy_version 1487019 (0.0007) [2023-12-27 02:13:18,018][105620] Updated weights for policy 1, policy_version 1489383 (0.0007) [2023-12-27 02:13:18,056][105692] Updated weights for policy 0, policy_version 1487029 (0.0008) [2023-12-27 02:13:18,118][105692] Updated weights for policy 0, policy_version 1487039 (0.0009) [2023-12-27 02:13:18,668][105620] Updated weights for policy 1, policy_version 1489393 (0.0006) [2023-12-27 02:13:18,731][105620] Updated weights for policy 1, policy_version 1489403 (0.0006) [2023-12-27 02:13:18,804][105620] Updated weights for policy 1, policy_version 1489413 (0.0007) [2023-12-27 02:13:18,877][105620] Updated weights for policy 1, policy_version 1489423 (0.0006) [2023-12-27 02:13:18,900][105692] Updated weights for policy 0, policy_version 1487049 (0.0009) [2023-12-27 02:13:18,956][105692] Updated weights for policy 0, policy_version 1487059 (0.0010) [2023-12-27 02:13:19,010][105692] Updated weights for policy 0, policy_version 1487069 (0.0010) [2023-12-27 02:13:19,063][105692] Updated weights for policy 0, policy_version 1487079 (0.0010) [2023-12-27 02:13:19,432][105620] Updated weights for policy 1, policy_version 1489433 (0.0009) [2023-12-27 02:13:19,495][105620] Updated weights for policy 1, policy_version 1489443 (0.0008) [2023-12-27 02:13:19,551][105620] Updated weights for policy 1, policy_version 1489453 (0.0008) [2023-12-27 02:13:19,863][105692] Updated weights for policy 0, policy_version 1487089 (0.0009) [2023-12-27 02:13:19,929][105692] Updated weights for policy 0, policy_version 1487099 (0.0008) [2023-12-27 02:13:19,988][105692] Updated weights for policy 0, policy_version 1487109 (0.0006) [2023-12-27 02:13:20,270][105620] Updated weights for policy 1, policy_version 1489463 (0.0008) [2023-12-27 02:13:20,329][105620] Updated weights for policy 1, policy_version 1489473 (0.0008) [2023-12-27 02:13:20,388][105620] Updated weights for policy 1, policy_version 1489483 (0.0010) [2023-12-27 02:13:20,719][105692] Updated weights for policy 0, policy_version 1487119 (0.0008) [2023-12-27 02:13:20,775][105692] Updated weights for policy 0, policy_version 1487129 (0.0009) [2023-12-27 02:13:20,834][105692] Updated weights for policy 0, policy_version 1487139 (0.0009) [2023-12-27 02:13:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 762126336. Throughput: 0: 9845.0, 1: 9778.9. Samples: 762117144. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:13:21,062][104569] Avg episode reward: [(0, '8439.840'), (1, '9261.947')] [2023-12-27 02:13:21,168][105620] Updated weights for policy 1, policy_version 1489493 (0.0010) [2023-12-27 02:13:21,223][105620] Updated weights for policy 1, policy_version 1489503 (0.0009) [2023-12-27 02:13:21,287][105620] Updated weights for policy 1, policy_version 1489513 (0.0009) [2023-12-27 02:13:21,561][105692] Updated weights for policy 0, policy_version 1487149 (0.0008) [2023-12-27 02:13:21,630][105692] Updated weights for policy 0, policy_version 1487159 (0.0007) [2023-12-27 02:13:21,687][105692] Updated weights for policy 0, policy_version 1487169 (0.0007) [2023-12-27 02:13:22,045][105620] Updated weights for policy 1, policy_version 1489523 (0.0009) [2023-12-27 02:13:22,095][105620] Updated weights for policy 1, policy_version 1489533 (0.0010) [2023-12-27 02:13:22,145][105620] Updated weights for policy 1, policy_version 1489543 (0.0011) [2023-12-27 02:13:22,462][105692] Updated weights for policy 0, policy_version 1487179 (0.0009) [2023-12-27 02:13:22,523][105692] Updated weights for policy 0, policy_version 1487189 (0.0010) [2023-12-27 02:13:22,590][105692] Updated weights for policy 0, policy_version 1487199 (0.0010) [2023-12-27 02:13:22,928][105620] Updated weights for policy 1, policy_version 1489553 (0.0011) [2023-12-27 02:13:22,994][105620] Updated weights for policy 1, policy_version 1489563 (0.0011) [2023-12-27 02:13:23,053][105620] Updated weights for policy 1, policy_version 1489573 (0.0011) [2023-12-27 02:13:23,100][105620] Updated weights for policy 1, policy_version 1489583 (0.0011) [2023-12-27 02:13:23,224][105692] Updated weights for policy 0, policy_version 1487209 (0.0010) [2023-12-27 02:13:23,289][105692] Updated weights for policy 0, policy_version 1487219 (0.0007) [2023-12-27 02:13:23,352][105692] Updated weights for policy 0, policy_version 1487229 (0.0008) [2023-12-27 02:13:23,398][105692] Updated weights for policy 0, policy_version 1487239 (0.0005) [2023-12-27 02:13:23,920][105692] Updated weights for policy 0, policy_version 1487249 (0.0006) [2023-12-27 02:13:23,950][105620] Updated weights for policy 1, policy_version 1489593 (0.0006) [2023-12-27 02:13:23,984][105692] Updated weights for policy 0, policy_version 1487259 (0.0005) [2023-12-27 02:13:24,003][105620] Updated weights for policy 1, policy_version 1489603 (0.0009) [2023-12-27 02:13:24,040][105692] Updated weights for policy 0, policy_version 1487269 (0.0005) [2023-12-27 02:13:24,056][105620] Updated weights for policy 1, policy_version 1489613 (0.0009) [2023-12-27 02:13:24,559][105692] Updated weights for policy 0, policy_version 1487279 (0.0005) [2023-12-27 02:13:24,623][105692] Updated weights for policy 0, policy_version 1487289 (0.0006) [2023-12-27 02:13:24,684][105692] Updated weights for policy 0, policy_version 1487299 (0.0010) [2023-12-27 02:13:24,724][105620] Updated weights for policy 1, policy_version 1489623 (0.0009) [2023-12-27 02:13:24,792][105620] Updated weights for policy 1, policy_version 1489633 (0.0010) [2023-12-27 02:13:24,856][105620] Updated weights for policy 1, policy_version 1489643 (0.0010) [2023-12-27 02:13:25,240][105692] Updated weights for policy 0, policy_version 1487309 (0.0006) [2023-12-27 02:13:25,304][105692] Updated weights for policy 0, policy_version 1487319 (0.0008) [2023-12-27 02:13:25,355][105692] Updated weights for policy 0, policy_version 1487329 (0.0008) [2023-12-27 02:13:25,563][105620] Updated weights for policy 1, policy_version 1489653 (0.0010) [2023-12-27 02:13:25,617][105620] Updated weights for policy 1, policy_version 1489663 (0.0010) [2023-12-27 02:13:25,672][105620] Updated weights for policy 1, policy_version 1489673 (0.0010) [2023-12-27 02:13:26,010][105692] Updated weights for policy 0, policy_version 1487339 (0.0010) [2023-12-27 02:13:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 762224640. Throughput: 0: 9899.2, 1: 9697.6. Samples: 762235820. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:13:26,062][104569] Avg episode reward: [(0, '8531.676'), (1, '9262.587')] [2023-12-27 02:13:26,064][105692] Updated weights for policy 0, policy_version 1487349 (0.0010) [2023-12-27 02:13:26,122][105692] Updated weights for policy 0, policy_version 1487359 (0.0010) [2023-12-27 02:13:26,358][105620] Updated weights for policy 1, policy_version 1489683 (0.0008) [2023-12-27 02:13:26,407][105620] Updated weights for policy 1, policy_version 1489693 (0.0010) [2023-12-27 02:13:26,455][105620] Updated weights for policy 1, policy_version 1489703 (0.0007) [2023-12-27 02:13:26,845][105692] Updated weights for policy 0, policy_version 1487369 (0.0010) [2023-12-27 02:13:26,904][105692] Updated weights for policy 0, policy_version 1487379 (0.0008) [2023-12-27 02:13:26,954][105692] Updated weights for policy 0, policy_version 1487390 (0.0009) [2023-12-27 02:13:27,020][105620] Updated weights for policy 1, policy_version 1489713 (0.0006) [2023-12-27 02:13:27,073][105620] Updated weights for policy 1, policy_version 1489723 (0.0007) [2023-12-27 02:13:27,131][105620] Updated weights for policy 1, policy_version 1489733 (0.0010) [2023-12-27 02:13:27,190][105620] Updated weights for policy 1, policy_version 1489743 (0.0011) [2023-12-27 02:13:27,597][105692] Updated weights for policy 0, policy_version 1487401 (0.0010) [2023-12-27 02:13:27,657][105692] Updated weights for policy 0, policy_version 1487411 (0.0005) [2023-12-27 02:13:27,711][105692] Updated weights for policy 0, policy_version 1487421 (0.0006) [2023-12-27 02:13:27,755][105692] Updated weights for policy 0, policy_version 1487431 (0.0008) [2023-12-27 02:13:27,885][105620] Updated weights for policy 1, policy_version 1489753 (0.0006) [2023-12-27 02:13:27,929][105620] Updated weights for policy 1, policy_version 1489763 (0.0005) [2023-12-27 02:13:27,972][105620] Updated weights for policy 1, policy_version 1489773 (0.0005) [2023-12-27 02:13:28,395][105692] Updated weights for policy 0, policy_version 1487441 (0.0007) [2023-12-27 02:13:28,462][105692] Updated weights for policy 0, policy_version 1487451 (0.0005) [2023-12-27 02:13:28,521][105692] Updated weights for policy 0, policy_version 1487461 (0.0005) [2023-12-27 02:13:28,589][105620] Updated weights for policy 1, policy_version 1489783 (0.0009) [2023-12-27 02:13:28,654][105620] Updated weights for policy 1, policy_version 1489793 (0.0009) [2023-12-27 02:13:28,712][105620] Updated weights for policy 1, policy_version 1489803 (0.0005) [2023-12-27 02:13:29,237][105620] Updated weights for policy 1, policy_version 1489813 (0.0007) [2023-12-27 02:13:29,245][105692] Updated weights for policy 0, policy_version 1487471 (0.0007) [2023-12-27 02:13:29,299][105620] Updated weights for policy 1, policy_version 1489823 (0.0007) [2023-12-27 02:13:29,303][105692] Updated weights for policy 0, policy_version 1487481 (0.0008) [2023-12-27 02:13:29,363][105620] Updated weights for policy 1, policy_version 1489833 (0.0008) [2023-12-27 02:13:29,372][105692] Updated weights for policy 0, policy_version 1487491 (0.0009) [2023-12-27 02:13:30,038][105620] Updated weights for policy 1, policy_version 1489843 (0.0006) [2023-12-27 02:13:30,099][105620] Updated weights for policy 1, policy_version 1489853 (0.0007) [2023-12-27 02:13:30,102][105692] Updated weights for policy 0, policy_version 1487501 (0.0006) [2023-12-27 02:13:30,155][105620] Updated weights for policy 1, policy_version 1489863 (0.0009) [2023-12-27 02:13:30,161][105692] Updated weights for policy 0, policy_version 1487511 (0.0005) [2023-12-27 02:13:30,208][105692] Updated weights for policy 0, policy_version 1487521 (0.0005) [2023-12-27 02:13:30,765][105620] Updated weights for policy 1, policy_version 1489873 (0.0010) [2023-12-27 02:13:30,818][105620] Updated weights for policy 1, policy_version 1489883 (0.0005) [2023-12-27 02:13:30,870][105620] Updated weights for policy 1, policy_version 1489893 (0.0005) [2023-12-27 02:13:30,916][105620] Updated weights for policy 1, policy_version 1489903 (0.0005) [2023-12-27 02:13:30,919][105692] Updated weights for policy 0, policy_version 1487531 (0.0007) [2023-12-27 02:13:30,972][105692] Updated weights for policy 0, policy_version 1487541 (0.0010) [2023-12-27 02:13:31,027][105692] Updated weights for policy 0, policy_version 1487551 (0.0009) [2023-12-27 02:13:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 762331136. Throughput: 0: 10033.1, 1: 9742.7. Samples: 762300620. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:13:31,062][104569] Avg episode reward: [(0, '8897.649'), (1, '9352.461')] [2023-12-27 02:13:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001489904_381468672.pth... [2023-12-27 02:13:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001488752_381173760.pth [2023-12-27 02:13:31,084][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001487560_380870656.pth... [2023-12-27 02:13:31,091][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001486376_380567552.pth [2023-12-27 02:13:31,530][105620] Updated weights for policy 1, policy_version 1489913 (0.0005) [2023-12-27 02:13:31,587][105620] Updated weights for policy 1, policy_version 1489923 (0.0007) [2023-12-27 02:13:31,656][105620] Updated weights for policy 1, policy_version 1489933 (0.0009) [2023-12-27 02:13:31,856][105692] Updated weights for policy 0, policy_version 1487561 (0.0009) [2023-12-27 02:13:31,920][105692] Updated weights for policy 0, policy_version 1487571 (0.0009) [2023-12-27 02:13:31,985][105692] Updated weights for policy 0, policy_version 1487581 (0.0005) [2023-12-27 02:13:32,046][105692] Updated weights for policy 0, policy_version 1487591 (0.0005) [2023-12-27 02:13:32,438][105620] Updated weights for policy 1, policy_version 1489943 (0.0010) [2023-12-27 02:13:32,505][105620] Updated weights for policy 1, policy_version 1489953 (0.0009) [2023-12-27 02:13:32,570][105620] Updated weights for policy 1, policy_version 1489963 (0.0009) [2023-12-27 02:13:32,626][105692] Updated weights for policy 0, policy_version 1487601 (0.0006) [2023-12-27 02:13:32,687][105692] Updated weights for policy 0, policy_version 1487611 (0.0005) [2023-12-27 02:13:32,746][105692] Updated weights for policy 0, policy_version 1487621 (0.0005) [2023-12-27 02:13:33,248][105620] Updated weights for policy 1, policy_version 1489973 (0.0007) [2023-12-27 02:13:33,298][105620] Updated weights for policy 1, policy_version 1489983 (0.0005) [2023-12-27 02:13:33,343][105620] Updated weights for policy 1, policy_version 1489993 (0.0005) [2023-12-27 02:13:33,377][105692] Updated weights for policy 0, policy_version 1487631 (0.0008) [2023-12-27 02:13:33,426][105692] Updated weights for policy 0, policy_version 1487641 (0.0009) [2023-12-27 02:13:33,474][105692] Updated weights for policy 0, policy_version 1487651 (0.0009) [2023-12-27 02:13:33,892][105620] Updated weights for policy 1, policy_version 1490003 (0.0006) [2023-12-27 02:13:33,951][105620] Updated weights for policy 1, policy_version 1490013 (0.0010) [2023-12-27 02:13:34,008][105620] Updated weights for policy 1, policy_version 1490023 (0.0010) [2023-12-27 02:13:34,357][105692] Updated weights for policy 0, policy_version 1487662 (0.0010) [2023-12-27 02:13:34,424][105692] Updated weights for policy 0, policy_version 1487672 (0.0010) [2023-12-27 02:13:34,484][105692] Updated weights for policy 0, policy_version 1487682 (0.0009) [2023-12-27 02:13:34,590][105620] Updated weights for policy 1, policy_version 1490033 (0.0010) [2023-12-27 02:13:34,660][105620] Updated weights for policy 1, policy_version 1490043 (0.0005) [2023-12-27 02:13:34,724][105620] Updated weights for policy 1, policy_version 1490053 (0.0005) [2023-12-27 02:13:34,792][105620] Updated weights for policy 1, policy_version 1490063 (0.0007) [2023-12-27 02:13:35,252][105692] Updated weights for policy 0, policy_version 1487692 (0.0010) [2023-12-27 02:13:35,316][105692] Updated weights for policy 0, policy_version 1487702 (0.0009) [2023-12-27 02:13:35,353][105620] Updated weights for policy 1, policy_version 1490073 (0.0010) [2023-12-27 02:13:35,367][105692] Updated weights for policy 0, policy_version 1487712 (0.0006) [2023-12-27 02:13:35,405][105620] Updated weights for policy 1, policy_version 1490083 (0.0009) [2023-12-27 02:13:35,460][105620] Updated weights for policy 1, policy_version 1490093 (0.0009) [2023-12-27 02:13:35,966][105692] Updated weights for policy 0, policy_version 1487722 (0.0007) [2023-12-27 02:13:36,022][105692] Updated weights for policy 0, policy_version 1487732 (0.0007) [2023-12-27 02:13:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 762429440. Throughput: 0: 9990.4, 1: 9814.6. Samples: 762422148. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:13:36,063][104569] Avg episode reward: [(0, '8902.852'), (1, '9078.589')] [2023-12-27 02:13:36,083][105692] Updated weights for policy 0, policy_version 1487742 (0.0009) [2023-12-27 02:13:36,140][105692] Updated weights for policy 0, policy_version 1487752 (0.0009) [2023-12-27 02:13:36,238][105620] Updated weights for policy 1, policy_version 1490103 (0.0006) [2023-12-27 02:13:36,297][105620] Updated weights for policy 1, policy_version 1490113 (0.0005) [2023-12-27 02:13:36,367][105620] Updated weights for policy 1, policy_version 1490123 (0.0005) [2023-12-27 02:13:36,957][105692] Updated weights for policy 0, policy_version 1487762 (0.0008) [2023-12-27 02:13:37,001][105620] Updated weights for policy 1, policy_version 1490133 (0.0008) [2023-12-27 02:13:37,011][105692] Updated weights for policy 0, policy_version 1487772 (0.0007) [2023-12-27 02:13:37,049][105620] Updated weights for policy 1, policy_version 1490143 (0.0010) [2023-12-27 02:13:37,071][105692] Updated weights for policy 0, policy_version 1487782 (0.0005) [2023-12-27 02:13:37,108][105620] Updated weights for policy 1, policy_version 1490153 (0.0010) [2023-12-27 02:13:37,762][105692] Updated weights for policy 0, policy_version 1487792 (0.0010) [2023-12-27 02:13:37,832][105692] Updated weights for policy 0, policy_version 1487802 (0.0011) [2023-12-27 02:13:37,881][105620] Updated weights for policy 1, policy_version 1490163 (0.0010) [2023-12-27 02:13:37,895][105692] Updated weights for policy 0, policy_version 1487812 (0.0009) [2023-12-27 02:13:37,938][105620] Updated weights for policy 1, policy_version 1490173 (0.0007) [2023-12-27 02:13:37,997][105620] Updated weights for policy 1, policy_version 1490183 (0.0008) [2023-12-27 02:13:38,563][105692] Updated weights for policy 0, policy_version 1487822 (0.0009) [2023-12-27 02:13:38,626][105692] Updated weights for policy 0, policy_version 1487832 (0.0006) [2023-12-27 02:13:38,683][105692] Updated weights for policy 0, policy_version 1487842 (0.0009) [2023-12-27 02:13:38,759][105620] Updated weights for policy 1, policy_version 1490193 (0.0008) [2023-12-27 02:13:38,809][105620] Updated weights for policy 1, policy_version 1490203 (0.0009) [2023-12-27 02:13:38,867][105620] Updated weights for policy 1, policy_version 1490213 (0.0009) [2023-12-27 02:13:38,930][105620] Updated weights for policy 1, policy_version 1490223 (0.0009) [2023-12-27 02:13:39,385][105692] Updated weights for policy 0, policy_version 1487852 (0.0009) [2023-12-27 02:13:39,445][105692] Updated weights for policy 0, policy_version 1487862 (0.0009) [2023-12-27 02:13:39,505][105692] Updated weights for policy 0, policy_version 1487872 (0.0010) [2023-12-27 02:13:39,629][105620] Updated weights for policy 1, policy_version 1490233 (0.0008) [2023-12-27 02:13:39,692][105620] Updated weights for policy 1, policy_version 1490243 (0.0006) [2023-12-27 02:13:39,755][105620] Updated weights for policy 1, policy_version 1490253 (0.0008) [2023-12-27 02:13:40,335][105692] Updated weights for policy 0, policy_version 1487882 (0.0008) [2023-12-27 02:13:40,400][105692] Updated weights for policy 0, policy_version 1487892 (0.0008) [2023-12-27 02:13:40,465][105692] Updated weights for policy 0, policy_version 1487902 (0.0008) [2023-12-27 02:13:40,487][105620] Updated weights for policy 1, policy_version 1490263 (0.0009) [2023-12-27 02:13:40,526][105692] Updated weights for policy 0, policy_version 1487912 (0.0007) [2023-12-27 02:13:40,549][105620] Updated weights for policy 1, policy_version 1490273 (0.0009) [2023-12-27 02:13:40,610][105620] Updated weights for policy 1, policy_version 1490283 (0.0007) [2023-12-27 02:13:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 762527744. Throughput: 0: 10026.8, 1: 9824.6. Samples: 762537568. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:13:41,063][104569] Avg episode reward: [(0, '8815.686'), (1, '9079.105')] [2023-12-27 02:13:41,290][105692] Updated weights for policy 0, policy_version 1487922 (0.0011) [2023-12-27 02:13:41,323][105620] Updated weights for policy 1, policy_version 1490293 (0.0008) [2023-12-27 02:13:41,355][105692] Updated weights for policy 0, policy_version 1487932 (0.0009) [2023-12-27 02:13:41,388][105620] Updated weights for policy 1, policy_version 1490303 (0.0008) [2023-12-27 02:13:41,423][105692] Updated weights for policy 0, policy_version 1487942 (0.0008) [2023-12-27 02:13:41,449][105620] Updated weights for policy 1, policy_version 1490313 (0.0009) [2023-12-27 02:13:42,200][105692] Updated weights for policy 0, policy_version 1487952 (0.0008) [2023-12-27 02:13:42,213][105620] Updated weights for policy 1, policy_version 1490323 (0.0010) [2023-12-27 02:13:42,253][105692] Updated weights for policy 0, policy_version 1487962 (0.0008) [2023-12-27 02:13:42,273][105620] Updated weights for policy 1, policy_version 1490333 (0.0008) [2023-12-27 02:13:42,314][105692] Updated weights for policy 0, policy_version 1487972 (0.0008) [2023-12-27 02:13:42,342][105620] Updated weights for policy 1, policy_version 1490343 (0.0008) [2023-12-27 02:13:42,992][105692] Updated weights for policy 0, policy_version 1487982 (0.0007) [2023-12-27 02:13:43,054][105692] Updated weights for policy 0, policy_version 1487992 (0.0009) [2023-12-27 02:13:43,104][105692] Updated weights for policy 0, policy_version 1488002 (0.0011) [2023-12-27 02:13:43,144][105620] Updated weights for policy 1, policy_version 1490353 (0.0008) [2023-12-27 02:13:43,200][105620] Updated weights for policy 1, policy_version 1490363 (0.0008) [2023-12-27 02:13:43,257][105620] Updated weights for policy 1, policy_version 1490373 (0.0008) [2023-12-27 02:13:43,312][105620] Updated weights for policy 1, policy_version 1490383 (0.0009) [2023-12-27 02:13:43,690][105692] Updated weights for policy 0, policy_version 1488012 (0.0010) [2023-12-27 02:13:43,744][105692] Updated weights for policy 0, policy_version 1488022 (0.0010) [2023-12-27 02:13:43,806][105692] Updated weights for policy 0, policy_version 1488032 (0.0010) [2023-12-27 02:13:44,019][105620] Updated weights for policy 1, policy_version 1490393 (0.0006) [2023-12-27 02:13:44,083][105620] Updated weights for policy 1, policy_version 1490403 (0.0005) [2023-12-27 02:13:44,141][105620] Updated weights for policy 1, policy_version 1490413 (0.0008) [2023-12-27 02:13:44,479][105692] Updated weights for policy 0, policy_version 1488042 (0.0010) [2023-12-27 02:13:44,527][105692] Updated weights for policy 0, policy_version 1488052 (0.0008) [2023-12-27 02:13:44,575][105692] Updated weights for policy 0, policy_version 1488062 (0.0007) [2023-12-27 02:13:44,638][105692] Updated weights for policy 0, policy_version 1488072 (0.0009) [2023-12-27 02:13:44,833][105620] Updated weights for policy 1, policy_version 1490423 (0.0010) [2023-12-27 02:13:44,886][105620] Updated weights for policy 1, policy_version 1490433 (0.0011) [2023-12-27 02:13:44,935][105620] Updated weights for policy 1, policy_version 1490443 (0.0011) [2023-12-27 02:13:45,495][105692] Updated weights for policy 0, policy_version 1488082 (0.0010) [2023-12-27 02:13:45,552][105692] Updated weights for policy 0, policy_version 1488092 (0.0011) [2023-12-27 02:13:45,568][105620] Updated weights for policy 1, policy_version 1490453 (0.0008) [2023-12-27 02:13:45,608][105692] Updated weights for policy 0, policy_version 1488102 (0.0011) [2023-12-27 02:13:45,621][105620] Updated weights for policy 1, policy_version 1490463 (0.0008) [2023-12-27 02:13:45,669][105620] Updated weights for policy 1, policy_version 1490473 (0.0010) [2023-12-27 02:13:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 762626048. Throughput: 0: 9932.2, 1: 9879.2. Samples: 762594988. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:13:46,063][104569] Avg episode reward: [(0, '8716.107'), (1, '9261.270')] [2023-12-27 02:13:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001488104_381009920.pth... [2023-12-27 02:13:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001490480_381616128.pth... [2023-12-27 02:13:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001489296_381313024.pth [2023-12-27 02:13:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001486952_380715008.pth [2023-12-27 02:13:46,315][105620] Updated weights for policy 1, policy_version 1490483 (0.0010) [2023-12-27 02:13:46,329][105692] Updated weights for policy 0, policy_version 1488112 (0.0010) [2023-12-27 02:13:46,374][105620] Updated weights for policy 1, policy_version 1490493 (0.0010) [2023-12-27 02:13:46,387][105692] Updated weights for policy 0, policy_version 1488122 (0.0010) [2023-12-27 02:13:46,432][105620] Updated weights for policy 1, policy_version 1490503 (0.0010) [2023-12-27 02:13:46,446][105692] Updated weights for policy 0, policy_version 1488132 (0.0011) [2023-12-27 02:13:47,148][105692] Updated weights for policy 0, policy_version 1488142 (0.0011) [2023-12-27 02:13:47,171][105620] Updated weights for policy 1, policy_version 1490513 (0.0010) [2023-12-27 02:13:47,210][105692] Updated weights for policy 0, policy_version 1488152 (0.0010) [2023-12-27 02:13:47,223][105620] Updated weights for policy 1, policy_version 1490523 (0.0010) [2023-12-27 02:13:47,268][105692] Updated weights for policy 0, policy_version 1488162 (0.0010) [2023-12-27 02:13:47,274][105620] Updated weights for policy 1, policy_version 1490533 (0.0010) [2023-12-27 02:13:47,335][105620] Updated weights for policy 1, policy_version 1490543 (0.0010) [2023-12-27 02:13:47,875][105692] Updated weights for policy 0, policy_version 1488172 (0.0009) [2023-12-27 02:13:47,933][105692] Updated weights for policy 0, policy_version 1488182 (0.0008) [2023-12-27 02:13:47,992][105692] Updated weights for policy 0, policy_version 1488192 (0.0008) [2023-12-27 02:13:48,081][105620] Updated weights for policy 1, policy_version 1490553 (0.0010) [2023-12-27 02:13:48,146][105620] Updated weights for policy 1, policy_version 1490563 (0.0011) [2023-12-27 02:13:48,210][105620] Updated weights for policy 1, policy_version 1490573 (0.0011) [2023-12-27 02:13:48,734][105692] Updated weights for policy 0, policy_version 1488202 (0.0007) [2023-12-27 02:13:48,796][105692] Updated weights for policy 0, policy_version 1488212 (0.0006) [2023-12-27 02:13:48,856][105692] Updated weights for policy 0, policy_version 1488222 (0.0006) [2023-12-27 02:13:48,914][105692] Updated weights for policy 0, policy_version 1488232 (0.0007) [2023-12-27 02:13:48,956][105620] Updated weights for policy 1, policy_version 1490583 (0.0009) [2023-12-27 02:13:49,011][105620] Updated weights for policy 1, policy_version 1490593 (0.0005) [2023-12-27 02:13:49,067][105620] Updated weights for policy 1, policy_version 1490603 (0.0009) [2023-12-27 02:13:49,569][105692] Updated weights for policy 0, policy_version 1488242 (0.0008) [2023-12-27 02:13:49,625][105692] Updated weights for policy 0, policy_version 1488252 (0.0011) [2023-12-27 02:13:49,675][105692] Updated weights for policy 0, policy_version 1488262 (0.0007) [2023-12-27 02:13:49,691][105620] Updated weights for policy 1, policy_version 1490613 (0.0010) [2023-12-27 02:13:49,753][105620] Updated weights for policy 1, policy_version 1490623 (0.0011) [2023-12-27 02:13:49,818][105620] Updated weights for policy 1, policy_version 1490633 (0.0010) [2023-12-27 02:13:50,397][105692] Updated weights for policy 0, policy_version 1488272 (0.0006) [2023-12-27 02:13:50,461][105692] Updated weights for policy 0, policy_version 1488282 (0.0006) [2023-12-27 02:13:50,530][105692] Updated weights for policy 0, policy_version 1488292 (0.0005) [2023-12-27 02:13:50,552][105620] Updated weights for policy 1, policy_version 1490643 (0.0010) [2023-12-27 02:13:50,619][105620] Updated weights for policy 1, policy_version 1490653 (0.0008) [2023-12-27 02:13:50,683][105620] Updated weights for policy 1, policy_version 1490663 (0.0007) [2023-12-27 02:13:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 762724352. Throughput: 0: 9904.4, 1: 9941.5. Samples: 762714460. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:13:51,062][104569] Avg episode reward: [(0, '8446.128'), (1, '9261.159')] [2023-12-27 02:13:51,181][105692] Updated weights for policy 0, policy_version 1488302 (0.0009) [2023-12-27 02:13:51,231][105692] Updated weights for policy 0, policy_version 1488312 (0.0011) [2023-12-27 02:13:51,298][105692] Updated weights for policy 0, policy_version 1488322 (0.0011) [2023-12-27 02:13:51,389][105620] Updated weights for policy 1, policy_version 1490673 (0.0007) [2023-12-27 02:13:51,441][105620] Updated weights for policy 1, policy_version 1490683 (0.0008) [2023-12-27 02:13:51,489][105620] Updated weights for policy 1, policy_version 1490693 (0.0008) [2023-12-27 02:13:51,534][105620] Updated weights for policy 1, policy_version 1490703 (0.0007) [2023-12-27 02:13:52,037][105692] Updated weights for policy 0, policy_version 1488332 (0.0012) [2023-12-27 02:13:52,104][105692] Updated weights for policy 0, policy_version 1488342 (0.0010) [2023-12-27 02:13:52,159][105692] Updated weights for policy 0, policy_version 1488352 (0.0010) [2023-12-27 02:13:52,254][105620] Updated weights for policy 1, policy_version 1490713 (0.0008) [2023-12-27 02:13:52,316][105620] Updated weights for policy 1, policy_version 1490723 (0.0010) [2023-12-27 02:13:52,382][105620] Updated weights for policy 1, policy_version 1490733 (0.0009) [2023-12-27 02:13:52,789][105692] Updated weights for policy 0, policy_version 1488362 (0.0010) [2023-12-27 02:13:52,850][105692] Updated weights for policy 0, policy_version 1488372 (0.0009) [2023-12-27 02:13:52,902][105692] Updated weights for policy 0, policy_version 1488382 (0.0011) [2023-12-27 02:13:52,954][105692] Updated weights for policy 0, policy_version 1488392 (0.0010) [2023-12-27 02:13:53,242][105620] Updated weights for policy 1, policy_version 1490744 (0.0009) [2023-12-27 02:13:53,287][105620] Updated weights for policy 1, policy_version 1490754 (0.0008) [2023-12-27 02:13:53,339][105620] Updated weights for policy 1, policy_version 1490764 (0.0009) [2023-12-27 02:13:53,574][105692] Updated weights for policy 0, policy_version 1488402 (0.0006) [2023-12-27 02:13:53,628][105692] Updated weights for policy 0, policy_version 1488412 (0.0006) [2023-12-27 02:13:53,677][105692] Updated weights for policy 0, policy_version 1488422 (0.0010) [2023-12-27 02:13:54,171][105620] Updated weights for policy 1, policy_version 1490774 (0.0010) [2023-12-27 02:13:54,219][105620] Updated weights for policy 1, policy_version 1490784 (0.0008) [2023-12-27 02:13:54,271][105620] Updated weights for policy 1, policy_version 1490794 (0.0008) [2023-12-27 02:13:54,403][105692] Updated weights for policy 0, policy_version 1488432 (0.0010) [2023-12-27 02:13:54,462][105692] Updated weights for policy 0, policy_version 1488442 (0.0010) [2023-12-27 02:13:54,517][105692] Updated weights for policy 0, policy_version 1488452 (0.0011) [2023-12-27 02:13:55,060][105620] Updated weights for policy 1, policy_version 1490804 (0.0008) [2023-12-27 02:13:55,126][105620] Updated weights for policy 1, policy_version 1490814 (0.0008) [2023-12-27 02:13:55,171][105692] Updated weights for policy 0, policy_version 1488462 (0.0011) [2023-12-27 02:13:55,182][105620] Updated weights for policy 1, policy_version 1490824 (0.0005) [2023-12-27 02:13:55,233][105692] Updated weights for policy 0, policy_version 1488472 (0.0010) [2023-12-27 02:13:55,277][105692] Updated weights for policy 0, policy_version 1488482 (0.0010) [2023-12-27 02:13:55,810][105620] Updated weights for policy 1, policy_version 1490834 (0.0005) [2023-12-27 02:13:55,868][105620] Updated weights for policy 1, policy_version 1490844 (0.0006) [2023-12-27 02:13:55,919][105620] Updated weights for policy 1, policy_version 1490854 (0.0005) [2023-12-27 02:13:55,983][105620] Updated weights for policy 1, policy_version 1490864 (0.0005) [2023-12-27 02:13:56,028][105692] Updated weights for policy 0, policy_version 1488492 (0.0010) [2023-12-27 02:13:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 762822656. Throughput: 0: 9995.6, 1: 9856.4. Samples: 762831664. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:13:56,062][104569] Avg episode reward: [(0, '8539.753'), (1, '9260.032')] [2023-12-27 02:13:56,077][105692] Updated weights for policy 0, policy_version 1488502 (0.0010) [2023-12-27 02:13:56,126][105692] Updated weights for policy 0, policy_version 1488512 (0.0010) [2023-12-27 02:13:56,515][105620] Updated weights for policy 1, policy_version 1490874 (0.0006) [2023-12-27 02:13:56,575][105620] Updated weights for policy 1, policy_version 1490884 (0.0005) [2023-12-27 02:13:56,628][105620] Updated weights for policy 1, policy_version 1490894 (0.0006) [2023-12-27 02:13:56,893][105692] Updated weights for policy 0, policy_version 1488522 (0.0011) [2023-12-27 02:13:56,947][105692] Updated weights for policy 0, policy_version 1488532 (0.0010) [2023-12-27 02:13:56,999][105692] Updated weights for policy 0, policy_version 1488542 (0.0010) [2023-12-27 02:13:57,050][105692] Updated weights for policy 0, policy_version 1488552 (0.0010) [2023-12-27 02:13:57,131][105620] Updated weights for policy 1, policy_version 1490904 (0.0005) [2023-12-27 02:13:57,182][105620] Updated weights for policy 1, policy_version 1490914 (0.0005) [2023-12-27 02:13:57,236][105620] Updated weights for policy 1, policy_version 1490924 (0.0005) [2023-12-27 02:13:57,692][105692] Updated weights for policy 0, policy_version 1488562 (0.0005) [2023-12-27 02:13:57,729][105585] KL-divergence is very high: 143.0009 [2023-12-27 02:13:57,752][105692] Updated weights for policy 0, policy_version 1488572 (0.0008) [2023-12-27 02:13:57,771][105585] KL-divergence is very high: 140.4120 [2023-12-27 02:13:57,800][105692] Updated weights for policy 0, policy_version 1488582 (0.0010) [2023-12-27 02:13:57,936][105620] Updated weights for policy 1, policy_version 1490934 (0.0008) [2023-12-27 02:13:57,983][105620] Updated weights for policy 1, policy_version 1490944 (0.0010) [2023-12-27 02:13:58,038][105620] Updated weights for policy 1, policy_version 1490954 (0.0010) [2023-12-27 02:13:58,529][105692] Updated weights for policy 0, policy_version 1488592 (0.0011) [2023-12-27 02:13:58,591][105692] Updated weights for policy 0, policy_version 1488602 (0.0009) [2023-12-27 02:13:58,656][105692] Updated weights for policy 0, policy_version 1488612 (0.0007) [2023-12-27 02:13:58,869][105620] Updated weights for policy 1, policy_version 1490964 (0.0013) [2023-12-27 02:13:58,935][105620] Updated weights for policy 1, policy_version 1490975 (0.0011) [2023-12-27 02:13:59,004][105620] Updated weights for policy 1, policy_version 1490985 (0.0008) [2023-12-27 02:13:59,426][105692] Updated weights for policy 0, policy_version 1488622 (0.0007) [2023-12-27 02:13:59,486][105692] Updated weights for policy 0, policy_version 1488632 (0.0006) [2023-12-27 02:13:59,548][105692] Updated weights for policy 0, policy_version 1488642 (0.0005) [2023-12-27 02:13:59,746][105620] Updated weights for policy 1, policy_version 1490995 (0.0007) [2023-12-27 02:13:59,809][105620] Updated weights for policy 1, policy_version 1491005 (0.0007) [2023-12-27 02:13:59,873][105620] Updated weights for policy 1, policy_version 1491015 (0.0010) [2023-12-27 02:14:00,189][105692] Updated weights for policy 0, policy_version 1488652 (0.0007) [2023-12-27 02:14:00,241][105692] Updated weights for policy 0, policy_version 1488662 (0.0008) [2023-12-27 02:14:00,292][105692] Updated weights for policy 0, policy_version 1488672 (0.0008) [2023-12-27 02:14:00,612][105620] Updated weights for policy 1, policy_version 1491025 (0.0008) [2023-12-27 02:14:00,669][105620] Updated weights for policy 1, policy_version 1491035 (0.0009) [2023-12-27 02:14:00,727][105620] Updated weights for policy 1, policy_version 1491045 (0.0009) [2023-12-27 02:14:00,789][105620] Updated weights for policy 1, policy_version 1491055 (0.0009) [2023-12-27 02:14:01,045][105692] Updated weights for policy 0, policy_version 1488682 (0.0008) [2023-12-27 02:14:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 762920960. Throughput: 0: 9979.9, 1: 9936.1. Samples: 762893772. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:14:01,062][104569] Avg episode reward: [(0, '8349.357'), (1, '9167.310')] [2023-12-27 02:14:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001491056_381763584.pth... [2023-12-27 02:14:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001489904_381468672.pth [2023-12-27 02:14:01,100][105692] Updated weights for policy 0, policy_version 1488692 (0.0009) [2023-12-27 02:14:01,161][105692] Updated weights for policy 0, policy_version 1488702 (0.0009) [2023-12-27 02:14:01,210][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001488712_381165568.pth... [2023-12-27 02:14:01,210][105692] Updated weights for policy 0, policy_version 1488712 (0.0008) [2023-12-27 02:14:01,213][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001487560_380870656.pth [2023-12-27 02:14:01,533][105620] Updated weights for policy 1, policy_version 1491065 (0.0010) [2023-12-27 02:14:01,589][105620] Updated weights for policy 1, policy_version 1491075 (0.0008) [2023-12-27 02:14:01,647][105620] Updated weights for policy 1, policy_version 1491085 (0.0009) [2023-12-27 02:14:01,997][105692] Updated weights for policy 0, policy_version 1488722 (0.0006) [2023-12-27 02:14:02,049][105692] Updated weights for policy 0, policy_version 1488732 (0.0005) [2023-12-27 02:14:02,107][105692] Updated weights for policy 0, policy_version 1488742 (0.0006) [2023-12-27 02:14:02,398][105620] Updated weights for policy 1, policy_version 1491095 (0.0010) [2023-12-27 02:14:02,463][105620] Updated weights for policy 1, policy_version 1491105 (0.0010) [2023-12-27 02:14:02,524][105620] Updated weights for policy 1, policy_version 1491115 (0.0010) [2023-12-27 02:14:02,742][105692] Updated weights for policy 0, policy_version 1488752 (0.0008) [2023-12-27 02:14:02,800][105692] Updated weights for policy 0, policy_version 1488762 (0.0005) [2023-12-27 02:14:02,863][105692] Updated weights for policy 0, policy_version 1488772 (0.0005) [2023-12-27 02:14:03,267][105620] Updated weights for policy 1, policy_version 1491125 (0.0011) [2023-12-27 02:14:03,318][105620] Updated weights for policy 1, policy_version 1491135 (0.0010) [2023-12-27 02:14:03,370][105620] Updated weights for policy 1, policy_version 1491145 (0.0009) [2023-12-27 02:14:03,513][105692] Updated weights for policy 0, policy_version 1488782 (0.0009) [2023-12-27 02:14:03,561][105692] Updated weights for policy 0, policy_version 1488792 (0.0010) [2023-12-27 02:14:03,609][105692] Updated weights for policy 0, policy_version 1488802 (0.0010) [2023-12-27 02:14:03,994][105620] Updated weights for policy 1, policy_version 1491155 (0.0007) [2023-12-27 02:14:04,056][105620] Updated weights for policy 1, policy_version 1491165 (0.0011) [2023-12-27 02:14:04,124][105620] Updated weights for policy 1, policy_version 1491175 (0.0012) [2023-12-27 02:14:04,313][105692] Updated weights for policy 0, policy_version 1488812 (0.0009) [2023-12-27 02:14:04,374][105692] Updated weights for policy 0, policy_version 1488822 (0.0006) [2023-12-27 02:14:04,439][105692] Updated weights for policy 0, policy_version 1488832 (0.0006) [2023-12-27 02:14:04,880][105620] Updated weights for policy 1, policy_version 1491185 (0.0010) [2023-12-27 02:14:04,933][105620] Updated weights for policy 1, policy_version 1491195 (0.0009) [2023-12-27 02:14:04,991][105620] Updated weights for policy 1, policy_version 1491205 (0.0010) [2023-12-27 02:14:05,039][105620] Updated weights for policy 1, policy_version 1491215 (0.0010) [2023-12-27 02:14:05,143][105692] Updated weights for policy 0, policy_version 1488842 (0.0007) [2023-12-27 02:14:05,196][105692] Updated weights for policy 0, policy_version 1488852 (0.0007) [2023-12-27 02:14:05,251][105692] Updated weights for policy 0, policy_version 1488862 (0.0006) [2023-12-27 02:14:05,299][105692] Updated weights for policy 0, policy_version 1488872 (0.0008) [2023-12-27 02:14:05,717][105620] Updated weights for policy 1, policy_version 1491225 (0.0009) [2023-12-27 02:14:05,764][105620] Updated weights for policy 1, policy_version 1491235 (0.0009) [2023-12-27 02:14:05,810][105620] Updated weights for policy 1, policy_version 1491245 (0.0008) [2023-12-27 02:14:06,046][105692] Updated weights for policy 0, policy_version 1488882 (0.0005) [2023-12-27 02:14:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 763019264. Throughput: 0: 9940.1, 1: 9904.4. Samples: 763010148. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:14:06,063][104569] Avg episode reward: [(0, '8530.887'), (1, '9259.190')] [2023-12-27 02:14:06,103][105692] Updated weights for policy 0, policy_version 1488892 (0.0006) [2023-12-27 02:14:06,170][105692] Updated weights for policy 0, policy_version 1488902 (0.0006) [2023-12-27 02:14:06,627][105620] Updated weights for policy 1, policy_version 1491255 (0.0010) [2023-12-27 02:14:06,693][105620] Updated weights for policy 1, policy_version 1491265 (0.0010) [2023-12-27 02:14:06,756][105620] Updated weights for policy 1, policy_version 1491275 (0.0007) [2023-12-27 02:14:06,764][105692] Updated weights for policy 0, policy_version 1488912 (0.0008) [2023-12-27 02:14:06,825][105692] Updated weights for policy 0, policy_version 1488922 (0.0008) [2023-12-27 02:14:06,887][105692] Updated weights for policy 0, policy_version 1488932 (0.0009) [2023-12-27 02:14:07,369][105620] Updated weights for policy 1, policy_version 1491285 (0.0009) [2023-12-27 02:14:07,432][105620] Updated weights for policy 1, policy_version 1491295 (0.0010) [2023-12-27 02:14:07,481][105620] Updated weights for policy 1, policy_version 1491305 (0.0010) [2023-12-27 02:14:07,691][105692] Updated weights for policy 0, policy_version 1488942 (0.0010) [2023-12-27 02:14:07,753][105692] Updated weights for policy 0, policy_version 1488952 (0.0011) [2023-12-27 02:14:07,805][105692] Updated weights for policy 0, policy_version 1488962 (0.0008) [2023-12-27 02:14:08,220][105620] Updated weights for policy 1, policy_version 1491315 (0.0010) [2023-12-27 02:14:08,274][105620] Updated weights for policy 1, policy_version 1491325 (0.0010) [2023-12-27 02:14:08,329][105620] Updated weights for policy 1, policy_version 1491335 (0.0010) [2023-12-27 02:14:08,382][105692] Updated weights for policy 0, policy_version 1488972 (0.0007) [2023-12-27 02:14:08,442][105692] Updated weights for policy 0, policy_version 1488982 (0.0011) [2023-12-27 02:14:08,501][105692] Updated weights for policy 0, policy_version 1488992 (0.0011) [2023-12-27 02:14:09,082][105620] Updated weights for policy 1, policy_version 1491345 (0.0009) [2023-12-27 02:14:09,130][105620] Updated weights for policy 1, policy_version 1491355 (0.0010) [2023-12-27 02:14:09,187][105620] Updated weights for policy 1, policy_version 1491365 (0.0011) [2023-12-27 02:14:09,245][105620] Updated weights for policy 1, policy_version 1491375 (0.0010) [2023-12-27 02:14:09,250][105692] Updated weights for policy 0, policy_version 1489002 (0.0010) [2023-12-27 02:14:09,312][105692] Updated weights for policy 0, policy_version 1489012 (0.0010) [2023-12-27 02:14:09,380][105692] Updated weights for policy 0, policy_version 1489022 (0.0009) [2023-12-27 02:14:09,443][105692] Updated weights for policy 0, policy_version 1489032 (0.0008) [2023-12-27 02:14:10,025][105620] Updated weights for policy 1, policy_version 1491385 (0.0009) [2023-12-27 02:14:10,085][105620] Updated weights for policy 1, policy_version 1491395 (0.0007) [2023-12-27 02:14:10,138][105620] Updated weights for policy 1, policy_version 1491405 (0.0008) [2023-12-27 02:14:10,245][105692] Updated weights for policy 0, policy_version 1489042 (0.0011) [2023-12-27 02:14:10,301][105692] Updated weights for policy 0, policy_version 1489052 (0.0010) [2023-12-27 02:14:10,363][105692] Updated weights for policy 0, policy_version 1489062 (0.0010) [2023-12-27 02:14:10,810][105620] Updated weights for policy 1, policy_version 1491415 (0.0009) [2023-12-27 02:14:10,866][105620] Updated weights for policy 1, policy_version 1491425 (0.0008) [2023-12-27 02:14:10,920][105620] Updated weights for policy 1, policy_version 1491435 (0.0009) [2023-12-27 02:14:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 763117568. Throughput: 0: 9825.1, 1: 9947.6. Samples: 763125596. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:14:11,062][104569] Avg episode reward: [(0, '9172.298'), (1, '9170.673')] [2023-12-27 02:14:11,134][105692] Updated weights for policy 0, policy_version 1489072 (0.0012) [2023-12-27 02:14:11,202][105692] Updated weights for policy 0, policy_version 1489082 (0.0010) [2023-12-27 02:14:11,266][105692] Updated weights for policy 0, policy_version 1489092 (0.0008) [2023-12-27 02:14:11,723][105620] Updated weights for policy 1, policy_version 1491445 (0.0009) [2023-12-27 02:14:11,785][105620] Updated weights for policy 1, policy_version 1491455 (0.0009) [2023-12-27 02:14:11,840][105620] Updated weights for policy 1, policy_version 1491465 (0.0010) [2023-12-27 02:14:12,006][105692] Updated weights for policy 0, policy_version 1489102 (0.0007) [2023-12-27 02:14:12,059][105692] Updated weights for policy 0, policy_version 1489112 (0.0006) [2023-12-27 02:14:12,123][105692] Updated weights for policy 0, policy_version 1489122 (0.0009) [2023-12-27 02:14:12,560][105620] Updated weights for policy 1, policy_version 1491475 (0.0008) [2023-12-27 02:14:12,616][105620] Updated weights for policy 1, policy_version 1491485 (0.0009) [2023-12-27 02:14:12,667][105620] Updated weights for policy 1, policy_version 1491495 (0.0010) [2023-12-27 02:14:12,803][105692] Updated weights for policy 0, policy_version 1489132 (0.0008) [2023-12-27 02:14:12,853][105692] Updated weights for policy 0, policy_version 1489142 (0.0006) [2023-12-27 02:14:12,901][105692] Updated weights for policy 0, policy_version 1489152 (0.0009) [2023-12-27 02:14:13,418][105620] Updated weights for policy 1, policy_version 1491505 (0.0008) [2023-12-27 02:14:13,479][105620] Updated weights for policy 1, policy_version 1491515 (0.0006) [2023-12-27 02:14:13,543][105620] Updated weights for policy 1, policy_version 1491525 (0.0009) [2023-12-27 02:14:13,603][105620] Updated weights for policy 1, policy_version 1491535 (0.0009) [2023-12-27 02:14:13,636][105692] Updated weights for policy 0, policy_version 1489162 (0.0007) [2023-12-27 02:14:13,691][105692] Updated weights for policy 0, policy_version 1489172 (0.0009) [2023-12-27 02:14:13,747][105692] Updated weights for policy 0, policy_version 1489182 (0.0009) [2023-12-27 02:14:13,802][105692] Updated weights for policy 0, policy_version 1489192 (0.0009) [2023-12-27 02:14:14,353][105620] Updated weights for policy 1, policy_version 1491545 (0.0010) [2023-12-27 02:14:14,419][105620] Updated weights for policy 1, policy_version 1491555 (0.0009) [2023-12-27 02:14:14,484][105620] Updated weights for policy 1, policy_version 1491565 (0.0008) [2023-12-27 02:14:14,488][105692] Updated weights for policy 0, policy_version 1489202 (0.0006) [2023-12-27 02:14:14,542][105692] Updated weights for policy 0, policy_version 1489212 (0.0008) [2023-12-27 02:14:14,597][105692] Updated weights for policy 0, policy_version 1489222 (0.0009) [2023-12-27 02:14:15,171][105620] Updated weights for policy 1, policy_version 1491575 (0.0007) [2023-12-27 02:14:15,244][105620] Updated weights for policy 1, policy_version 1491585 (0.0009) [2023-12-27 02:14:15,309][105620] Updated weights for policy 1, policy_version 1491595 (0.0007) [2023-12-27 02:14:15,314][105692] Updated weights for policy 0, policy_version 1489232 (0.0008) [2023-12-27 02:14:15,367][105692] Updated weights for policy 0, policy_version 1489242 (0.0007) [2023-12-27 02:14:15,426][105692] Updated weights for policy 0, policy_version 1489252 (0.0006) [2023-12-27 02:14:16,020][105620] Updated weights for policy 1, policy_version 1491605 (0.0008) [2023-12-27 02:14:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.7, 300 sec: 19605.2). Total num frames: 763207680. Throughput: 0: 9758.2, 1: 9839.2. Samples: 763182504. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:14:16,063][104569] Avg episode reward: [(0, '9172.589'), (1, '8985.836')] [2023-12-27 02:14:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001489256_381304832.pth... [2023-12-27 02:14:16,071][105620] Updated weights for policy 1, policy_version 1491615 (0.0009) [2023-12-27 02:14:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001488104_381009920.pth [2023-12-27 02:14:16,132][105620] Updated weights for policy 1, policy_version 1491625 (0.0009) [2023-12-27 02:14:16,162][105692] Updated weights for policy 0, policy_version 1489262 (0.0008) [2023-12-27 02:14:16,168][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001491632_381911040.pth... [2023-12-27 02:14:16,173][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001490480_381616128.pth [2023-12-27 02:14:16,222][105692] Updated weights for policy 0, policy_version 1489272 (0.0009) [2023-12-27 02:14:16,284][105692] Updated weights for policy 0, policy_version 1489282 (0.0010) [2023-12-27 02:14:16,768][105620] Updated weights for policy 1, policy_version 1491635 (0.0006) [2023-12-27 02:14:16,822][105620] Updated weights for policy 1, policy_version 1491645 (0.0007) [2023-12-27 02:14:16,877][105620] Updated weights for policy 1, policy_version 1491656 (0.0010) [2023-12-27 02:14:16,932][105692] Updated weights for policy 0, policy_version 1489292 (0.0008) [2023-12-27 02:14:16,981][105692] Updated weights for policy 0, policy_version 1489302 (0.0007) [2023-12-27 02:14:17,029][105692] Updated weights for policy 0, policy_version 1489312 (0.0007) [2023-12-27 02:14:17,586][105620] Updated weights for policy 1, policy_version 1491666 (0.0010) [2023-12-27 02:14:17,637][105620] Updated weights for policy 1, policy_version 1491676 (0.0010) [2023-12-27 02:14:17,685][105620] Updated weights for policy 1, policy_version 1491686 (0.0010) [2023-12-27 02:14:17,734][105620] Updated weights for policy 1, policy_version 1491696 (0.0010) [2023-12-27 02:14:17,799][105692] Updated weights for policy 0, policy_version 1489322 (0.0007) [2023-12-27 02:14:17,858][105692] Updated weights for policy 0, policy_version 1489332 (0.0008) [2023-12-27 02:14:17,907][105692] Updated weights for policy 0, policy_version 1489342 (0.0008) [2023-12-27 02:14:17,952][105692] Updated weights for policy 0, policy_version 1489352 (0.0008) [2023-12-27 02:14:18,427][105620] Updated weights for policy 1, policy_version 1491706 (0.0007) [2023-12-27 02:14:18,482][105620] Updated weights for policy 1, policy_version 1491716 (0.0009) [2023-12-27 02:14:18,532][105620] Updated weights for policy 1, policy_version 1491726 (0.0009) [2023-12-27 02:14:18,752][105692] Updated weights for policy 0, policy_version 1489362 (0.0009) [2023-12-27 02:14:18,815][105692] Updated weights for policy 0, policy_version 1489372 (0.0009) [2023-12-27 02:14:18,877][105692] Updated weights for policy 0, policy_version 1489382 (0.0009) [2023-12-27 02:14:19,215][105620] Updated weights for policy 1, policy_version 1491736 (0.0009) [2023-12-27 02:14:19,286][105620] Updated weights for policy 1, policy_version 1491746 (0.0009) [2023-12-27 02:14:19,355][105620] Updated weights for policy 1, policy_version 1491756 (0.0009) [2023-12-27 02:14:19,689][105692] Updated weights for policy 0, policy_version 1489392 (0.0010) [2023-12-27 02:14:19,748][105692] Updated weights for policy 0, policy_version 1489402 (0.0009) [2023-12-27 02:14:19,805][105692] Updated weights for policy 0, policy_version 1489412 (0.0009) [2023-12-27 02:14:20,119][105620] Updated weights for policy 1, policy_version 1491766 (0.0008) [2023-12-27 02:14:20,184][105620] Updated weights for policy 1, policy_version 1491776 (0.0008) [2023-12-27 02:14:20,244][105620] Updated weights for policy 1, policy_version 1491786 (0.0008) [2023-12-27 02:14:20,697][105692] Updated weights for policy 0, policy_version 1489422 (0.0009) [2023-12-27 02:14:20,753][105692] Updated weights for policy 0, policy_version 1489432 (0.0010) [2023-12-27 02:14:20,812][105692] Updated weights for policy 0, policy_version 1489442 (0.0009) [2023-12-27 02:14:20,962][105620] Updated weights for policy 1, policy_version 1491796 (0.0008) [2023-12-27 02:14:21,018][105620] Updated weights for policy 1, policy_version 1491806 (0.0006) [2023-12-27 02:14:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 763305984. Throughput: 0: 9773.9, 1: 9712.0. Samples: 763299012. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:14:21,062][104569] Avg episode reward: [(0, '8719.775'), (1, '8798.755')] [2023-12-27 02:14:21,089][105620] Updated weights for policy 1, policy_version 1491816 (0.0009) [2023-12-27 02:14:21,655][105692] Updated weights for policy 0, policy_version 1489452 (0.0008) [2023-12-27 02:14:21,712][105692] Updated weights for policy 0, policy_version 1489462 (0.0006) [2023-12-27 02:14:21,737][105620] Updated weights for policy 1, policy_version 1491826 (0.0008) [2023-12-27 02:14:21,776][105692] Updated weights for policy 0, policy_version 1489472 (0.0007) [2023-12-27 02:14:21,795][105620] Updated weights for policy 1, policy_version 1491836 (0.0009) [2023-12-27 02:14:21,847][105620] Updated weights for policy 1, policy_version 1491846 (0.0008) [2023-12-27 02:14:21,908][105620] Updated weights for policy 1, policy_version 1491856 (0.0009) [2023-12-27 02:14:22,484][105692] Updated weights for policy 0, policy_version 1489482 (0.0007) [2023-12-27 02:14:22,532][105692] Updated weights for policy 0, policy_version 1489492 (0.0009) [2023-12-27 02:14:22,582][105692] Updated weights for policy 0, policy_version 1489502 (0.0009) [2023-12-27 02:14:22,643][105692] Updated weights for policy 0, policy_version 1489512 (0.0007) [2023-12-27 02:14:22,709][105620] Updated weights for policy 1, policy_version 1491866 (0.0009) [2023-12-27 02:14:22,770][105620] Updated weights for policy 1, policy_version 1491876 (0.0009) [2023-12-27 02:14:22,837][105620] Updated weights for policy 1, policy_version 1491886 (0.0010) [2023-12-27 02:14:23,382][105692] Updated weights for policy 0, policy_version 1489522 (0.0009) [2023-12-27 02:14:23,434][105692] Updated weights for policy 0, policy_version 1489532 (0.0009) [2023-12-27 02:14:23,485][105692] Updated weights for policy 0, policy_version 1489542 (0.0008) [2023-12-27 02:14:23,542][105620] Updated weights for policy 1, policy_version 1491896 (0.0008) [2023-12-27 02:14:23,596][105620] Updated weights for policy 1, policy_version 1491906 (0.0009) [2023-12-27 02:14:23,649][105620] Updated weights for policy 1, policy_version 1491916 (0.0010) [2023-12-27 02:14:24,247][105692] Updated weights for policy 0, policy_version 1489552 (0.0006) [2023-12-27 02:14:24,305][105692] Updated weights for policy 0, policy_version 1489562 (0.0007) [2023-12-27 02:14:24,364][105692] Updated weights for policy 0, policy_version 1489572 (0.0008) [2023-12-27 02:14:24,417][105620] Updated weights for policy 1, policy_version 1491926 (0.0008) [2023-12-27 02:14:24,474][105620] Updated weights for policy 1, policy_version 1491936 (0.0007) [2023-12-27 02:14:24,519][105620] Updated weights for policy 1, policy_version 1491946 (0.0008) [2023-12-27 02:14:25,093][105620] Updated weights for policy 1, policy_version 1491956 (0.0006) [2023-12-27 02:14:25,150][105620] Updated weights for policy 1, policy_version 1491966 (0.0005) [2023-12-27 02:14:25,158][105692] Updated weights for policy 0, policy_version 1489582 (0.0009) [2023-12-27 02:14:25,200][105620] Updated weights for policy 1, policy_version 1491976 (0.0008) [2023-12-27 02:14:25,213][105692] Updated weights for policy 0, policy_version 1489592 (0.0010) [2023-12-27 02:14:25,260][105692] Updated weights for policy 0, policy_version 1489602 (0.0006) [2023-12-27 02:14:25,778][105620] Updated weights for policy 1, policy_version 1491986 (0.0009) [2023-12-27 02:14:25,835][105620] Updated weights for policy 1, policy_version 1491996 (0.0005) [2023-12-27 02:14:25,884][105620] Updated weights for policy 1, policy_version 1492006 (0.0008) [2023-12-27 02:14:25,928][105620] Updated weights for policy 1, policy_version 1492016 (0.0010) [2023-12-27 02:14:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 763404288. Throughput: 0: 9681.8, 1: 9764.1. Samples: 763412632. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:14:26,063][104569] Avg episode reward: [(0, '8263.466'), (1, '8904.051')] [2023-12-27 02:14:26,119][105692] Updated weights for policy 0, policy_version 1489612 (0.0008) [2023-12-27 02:14:26,167][105692] Updated weights for policy 0, policy_version 1489622 (0.0008) [2023-12-27 02:14:26,215][105692] Updated weights for policy 0, policy_version 1489632 (0.0008) [2023-12-27 02:14:26,661][105620] Updated weights for policy 1, policy_version 1492026 (0.0010) [2023-12-27 02:14:26,719][105620] Updated weights for policy 1, policy_version 1492036 (0.0010) [2023-12-27 02:14:26,776][105620] Updated weights for policy 1, policy_version 1492046 (0.0010) [2023-12-27 02:14:27,022][105692] Updated weights for policy 0, policy_version 1489642 (0.0008) [2023-12-27 02:14:27,080][105692] Updated weights for policy 0, policy_version 1489652 (0.0010) [2023-12-27 02:14:27,135][105692] Updated weights for policy 0, policy_version 1489664 (0.0010) [2023-12-27 02:14:27,363][105620] Updated weights for policy 1, policy_version 1492056 (0.0010) [2023-12-27 02:14:27,411][105620] Updated weights for policy 1, policy_version 1492066 (0.0010) [2023-12-27 02:14:27,460][105620] Updated weights for policy 1, policy_version 1492076 (0.0010) [2023-12-27 02:14:27,979][105692] Updated weights for policy 0, policy_version 1489675 (0.0010) [2023-12-27 02:14:28,036][105692] Updated weights for policy 0, policy_version 1489685 (0.0010) [2023-12-27 02:14:28,083][105620] Updated weights for policy 1, policy_version 1492086 (0.0007) [2023-12-27 02:14:28,089][105692] Updated weights for policy 0, policy_version 1489695 (0.0009) [2023-12-27 02:14:28,136][105620] Updated weights for policy 1, policy_version 1492096 (0.0005) [2023-12-27 02:14:28,192][105620] Updated weights for policy 1, policy_version 1492106 (0.0005) [2023-12-27 02:14:28,725][105620] Updated weights for policy 1, policy_version 1492116 (0.0005) [2023-12-27 02:14:28,781][105620] Updated weights for policy 1, policy_version 1492126 (0.0005) [2023-12-27 02:14:28,835][105620] Updated weights for policy 1, policy_version 1492136 (0.0005) [2023-12-27 02:14:28,998][105692] Updated weights for policy 0, policy_version 1489705 (0.0009) [2023-12-27 02:14:29,057][105692] Updated weights for policy 0, policy_version 1489715 (0.0009) [2023-12-27 02:14:29,113][105692] Updated weights for policy 0, policy_version 1489725 (0.0008) [2023-12-27 02:14:29,168][105692] Updated weights for policy 0, policy_version 1489735 (0.0009) [2023-12-27 02:14:29,456][105620] Updated weights for policy 1, policy_version 1492146 (0.0007) [2023-12-27 02:14:29,514][105620] Updated weights for policy 1, policy_version 1492156 (0.0006) [2023-12-27 02:14:29,565][105620] Updated weights for policy 1, policy_version 1492166 (0.0005) [2023-12-27 02:14:29,618][105620] Updated weights for policy 1, policy_version 1492176 (0.0008) [2023-12-27 02:14:29,996][105692] Updated weights for policy 0, policy_version 1489745 (0.0009) [2023-12-27 02:14:30,051][105692] Updated weights for policy 0, policy_version 1489755 (0.0008) [2023-12-27 02:14:30,107][105692] Updated weights for policy 0, policy_version 1489765 (0.0010) [2023-12-27 02:14:30,316][105620] Updated weights for policy 1, policy_version 1492186 (0.0010) [2023-12-27 02:14:30,381][105620] Updated weights for policy 1, policy_version 1492196 (0.0010) [2023-12-27 02:14:30,447][105620] Updated weights for policy 1, policy_version 1492206 (0.0010) [2023-12-27 02:14:30,888][105692] Updated weights for policy 0, policy_version 1489775 (0.0007) [2023-12-27 02:14:30,942][105692] Updated weights for policy 0, policy_version 1489785 (0.0008) [2023-12-27 02:14:31,019][105692] Updated weights for policy 0, policy_version 1489795 (0.0008) [2023-12-27 02:14:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 763502592. Throughput: 0: 9596.3, 1: 9886.3. Samples: 763471704. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:14:31,062][104569] Avg episode reward: [(0, '8443.571'), (1, '9005.489')] [2023-12-27 02:14:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001489800_381444096.pth... [2023-12-27 02:14:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001492208_382058496.pth... [2023-12-27 02:14:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001488712_381165568.pth [2023-12-27 02:14:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001491056_381763584.pth [2023-12-27 02:14:31,183][105620] Updated weights for policy 1, policy_version 1492216 (0.0011) [2023-12-27 02:14:31,239][105620] Updated weights for policy 1, policy_version 1492226 (0.0010) [2023-12-27 02:14:31,298][105620] Updated weights for policy 1, policy_version 1492236 (0.0009) [2023-12-27 02:14:31,806][105692] Updated weights for policy 0, policy_version 1489805 (0.0008) [2023-12-27 02:14:31,858][105692] Updated weights for policy 0, policy_version 1489815 (0.0006) [2023-12-27 02:14:31,906][105692] Updated weights for policy 0, policy_version 1489825 (0.0005) [2023-12-27 02:14:31,997][105620] Updated weights for policy 1, policy_version 1492246 (0.0008) [2023-12-27 02:14:32,052][105620] Updated weights for policy 1, policy_version 1492256 (0.0010) [2023-12-27 02:14:32,105][105620] Updated weights for policy 1, policy_version 1492266 (0.0010) [2023-12-27 02:14:32,480][105692] Updated weights for policy 0, policy_version 1489835 (0.0007) [2023-12-27 02:14:32,535][105692] Updated weights for policy 0, policy_version 1489845 (0.0009) [2023-12-27 02:14:32,593][105692] Updated weights for policy 0, policy_version 1489855 (0.0010) [2023-12-27 02:14:32,785][105620] Updated weights for policy 1, policy_version 1492276 (0.0009) [2023-12-27 02:14:32,846][105620] Updated weights for policy 1, policy_version 1492286 (0.0008) [2023-12-27 02:14:32,913][105620] Updated weights for policy 1, policy_version 1492296 (0.0007) [2023-12-27 02:14:33,249][105692] Updated weights for policy 0, policy_version 1489865 (0.0010) [2023-12-27 02:14:33,302][105692] Updated weights for policy 0, policy_version 1489875 (0.0010) [2023-12-27 02:14:33,357][105692] Updated weights for policy 0, policy_version 1489885 (0.0010) [2023-12-27 02:14:33,418][105692] Updated weights for policy 0, policy_version 1489895 (0.0010) [2023-12-27 02:14:33,626][105620] Updated weights for policy 1, policy_version 1492306 (0.0006) [2023-12-27 02:14:33,678][105620] Updated weights for policy 1, policy_version 1492316 (0.0005) [2023-12-27 02:14:33,736][105620] Updated weights for policy 1, policy_version 1492326 (0.0007) [2023-12-27 02:14:33,797][105620] Updated weights for policy 1, policy_version 1492336 (0.0005) [2023-12-27 02:14:34,159][105692] Updated weights for policy 0, policy_version 1489905 (0.0009) [2023-12-27 02:14:34,214][105692] Updated weights for policy 0, policy_version 1489915 (0.0011) [2023-12-27 02:14:34,262][105692] Updated weights for policy 0, policy_version 1489925 (0.0011) [2023-12-27 02:14:34,347][105620] Updated weights for policy 1, policy_version 1492346 (0.0009) [2023-12-27 02:14:34,398][105620] Updated weights for policy 1, policy_version 1492356 (0.0005) [2023-12-27 02:14:34,460][105620] Updated weights for policy 1, policy_version 1492366 (0.0006) [2023-12-27 02:14:35,048][105692] Updated weights for policy 0, policy_version 1489935 (0.0011) [2023-12-27 02:14:35,108][105692] Updated weights for policy 0, policy_version 1489945 (0.0010) [2023-12-27 02:14:35,113][105620] Updated weights for policy 1, policy_version 1492376 (0.0010) [2023-12-27 02:14:35,167][105692] Updated weights for policy 0, policy_version 1489955 (0.0010) [2023-12-27 02:14:35,168][105620] Updated weights for policy 1, policy_version 1492386 (0.0010) [2023-12-27 02:14:35,224][105620] Updated weights for policy 1, policy_version 1492396 (0.0006) [2023-12-27 02:14:35,914][105692] Updated weights for policy 0, policy_version 1489965 (0.0008) [2023-12-27 02:14:35,957][105620] Updated weights for policy 1, policy_version 1492406 (0.0008) [2023-12-27 02:14:35,971][105692] Updated weights for policy 0, policy_version 1489975 (0.0008) [2023-12-27 02:14:36,011][105620] Updated weights for policy 1, policy_version 1492416 (0.0007) [2023-12-27 02:14:36,026][105692] Updated weights for policy 0, policy_version 1489985 (0.0010) [2023-12-27 02:14:36,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 763592704. Throughput: 0: 9538.8, 1: 9915.7. Samples: 763589916. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:14:36,062][104569] Avg episode reward: [(0, '8623.265'), (1, '9178.181')] [2023-12-27 02:14:36,067][105620] Updated weights for policy 1, policy_version 1492426 (0.0005) [2023-12-27 02:14:36,758][105692] Updated weights for policy 0, policy_version 1489995 (0.0010) [2023-12-27 02:14:36,814][105692] Updated weights for policy 0, policy_version 1490005 (0.0010) [2023-12-27 02:14:36,816][105620] Updated weights for policy 1, policy_version 1492436 (0.0007) [2023-12-27 02:14:36,864][105620] Updated weights for policy 1, policy_version 1492446 (0.0006) [2023-12-27 02:14:36,873][105692] Updated weights for policy 0, policy_version 1490015 (0.0011) [2023-12-27 02:14:36,913][105620] Updated weights for policy 1, policy_version 1492456 (0.0009) [2023-12-27 02:14:37,625][105620] Updated weights for policy 1, policy_version 1492466 (0.0008) [2023-12-27 02:14:37,631][105692] Updated weights for policy 0, policy_version 1490025 (0.0010) [2023-12-27 02:14:37,680][105620] Updated weights for policy 1, policy_version 1492476 (0.0006) [2023-12-27 02:14:37,691][105692] Updated weights for policy 0, policy_version 1490035 (0.0011) [2023-12-27 02:14:37,740][105620] Updated weights for policy 1, policy_version 1492486 (0.0007) [2023-12-27 02:14:37,748][105692] Updated weights for policy 0, policy_version 1490045 (0.0011) [2023-12-27 02:14:37,796][105692] Updated weights for policy 0, policy_version 1490055 (0.0010) [2023-12-27 02:14:37,804][105620] Updated weights for policy 1, policy_version 1492496 (0.0008) [2023-12-27 02:14:38,411][105620] Updated weights for policy 1, policy_version 1492506 (0.0006) [2023-12-27 02:14:38,480][105620] Updated weights for policy 1, policy_version 1492516 (0.0005) [2023-12-27 02:14:38,551][105620] Updated weights for policy 1, policy_version 1492526 (0.0006) [2023-12-27 02:14:38,594][105692] Updated weights for policy 0, policy_version 1490065 (0.0007) [2023-12-27 02:14:38,651][105692] Updated weights for policy 0, policy_version 1490075 (0.0008) [2023-12-27 02:14:38,700][105692] Updated weights for policy 0, policy_version 1490085 (0.0008) [2023-12-27 02:14:39,178][105620] Updated weights for policy 1, policy_version 1492536 (0.0006) [2023-12-27 02:14:39,238][105620] Updated weights for policy 1, policy_version 1492546 (0.0007) [2023-12-27 02:14:39,301][105620] Updated weights for policy 1, policy_version 1492556 (0.0008) [2023-12-27 02:14:39,501][105692] Updated weights for policy 0, policy_version 1490095 (0.0008) [2023-12-27 02:14:39,554][105692] Updated weights for policy 0, policy_version 1490105 (0.0008) [2023-12-27 02:14:39,598][105692] Updated weights for policy 0, policy_version 1490115 (0.0008) [2023-12-27 02:14:39,920][105620] Updated weights for policy 1, policy_version 1492566 (0.0009) [2023-12-27 02:14:39,984][105620] Updated weights for policy 1, policy_version 1492576 (0.0008) [2023-12-27 02:14:40,054][105620] Updated weights for policy 1, policy_version 1492586 (0.0007) [2023-12-27 02:14:40,380][105692] Updated weights for policy 0, policy_version 1490125 (0.0011) [2023-12-27 02:14:40,446][105692] Updated weights for policy 0, policy_version 1490135 (0.0011) [2023-12-27 02:14:40,512][105692] Updated weights for policy 0, policy_version 1490145 (0.0010) [2023-12-27 02:14:40,693][105620] Updated weights for policy 1, policy_version 1492596 (0.0010) [2023-12-27 02:14:40,744][105620] Updated weights for policy 1, policy_version 1492606 (0.0010) [2023-12-27 02:14:40,808][105620] Updated weights for policy 1, policy_version 1492616 (0.0009) [2023-12-27 02:14:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 763699200. Throughput: 0: 9412.4, 1: 10019.5. Samples: 763706100. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 02:14:41,062][104569] Avg episode reward: [(0, '8533.005'), (1, '9352.810')] [2023-12-27 02:14:41,221][105692] Updated weights for policy 0, policy_version 1490155 (0.0009) [2023-12-27 02:14:41,293][105692] Updated weights for policy 0, policy_version 1490165 (0.0008) [2023-12-27 02:14:41,351][105692] Updated weights for policy 0, policy_version 1490175 (0.0008) [2023-12-27 02:14:41,610][105620] Updated weights for policy 1, policy_version 1492626 (0.0009) [2023-12-27 02:14:41,683][105620] Updated weights for policy 1, policy_version 1492636 (0.0008) [2023-12-27 02:14:41,755][105620] Updated weights for policy 1, policy_version 1492646 (0.0009) [2023-12-27 02:14:41,824][105620] Updated weights for policy 1, policy_version 1492656 (0.0009) [2023-12-27 02:14:42,158][105692] Updated weights for policy 0, policy_version 1490185 (0.0009) [2023-12-27 02:14:42,219][105692] Updated weights for policy 0, policy_version 1490195 (0.0009) [2023-12-27 02:14:42,279][105692] Updated weights for policy 0, policy_version 1490205 (0.0008) [2023-12-27 02:14:42,334][105692] Updated weights for policy 0, policy_version 1490215 (0.0009) [2023-12-27 02:14:42,605][105620] Updated weights for policy 1, policy_version 1492666 (0.0008) [2023-12-27 02:14:42,669][105620] Updated weights for policy 1, policy_version 1492676 (0.0005) [2023-12-27 02:14:42,732][105620] Updated weights for policy 1, policy_version 1492686 (0.0005) [2023-12-27 02:14:43,133][105692] Updated weights for policy 0, policy_version 1490225 (0.0006) [2023-12-27 02:14:43,190][105692] Updated weights for policy 0, policy_version 1490235 (0.0006) [2023-12-27 02:14:43,248][105692] Updated weights for policy 0, policy_version 1490245 (0.0009) [2023-12-27 02:14:43,379][105620] Updated weights for policy 1, policy_version 1492696 (0.0008) [2023-12-27 02:14:43,442][105620] Updated weights for policy 1, policy_version 1492706 (0.0009) [2023-12-27 02:14:43,496][105620] Updated weights for policy 1, policy_version 1492716 (0.0009) [2023-12-27 02:14:43,973][105692] Updated weights for policy 0, policy_version 1490255 (0.0009) [2023-12-27 02:14:44,019][105692] Updated weights for policy 0, policy_version 1490265 (0.0008) [2023-12-27 02:14:44,084][105692] Updated weights for policy 0, policy_version 1490275 (0.0009) [2023-12-27 02:14:44,238][105620] Updated weights for policy 1, policy_version 1492726 (0.0008) [2023-12-27 02:14:44,288][105620] Updated weights for policy 1, policy_version 1492736 (0.0008) [2023-12-27 02:14:44,353][105620] Updated weights for policy 1, policy_version 1492746 (0.0008) [2023-12-27 02:14:44,910][105692] Updated weights for policy 0, policy_version 1490285 (0.0009) [2023-12-27 02:14:44,970][105692] Updated weights for policy 0, policy_version 1490295 (0.0009) [2023-12-27 02:14:45,006][105620] Updated weights for policy 1, policy_version 1492756 (0.0009) [2023-12-27 02:14:45,033][105692] Updated weights for policy 0, policy_version 1490305 (0.0007) [2023-12-27 02:14:45,068][105620] Updated weights for policy 1, policy_version 1492766 (0.0007) [2023-12-27 02:14:45,131][105620] Updated weights for policy 1, policy_version 1492776 (0.0008) [2023-12-27 02:14:45,784][105692] Updated weights for policy 0, policy_version 1490315 (0.0007) [2023-12-27 02:14:45,846][105692] Updated weights for policy 0, policy_version 1490325 (0.0009) [2023-12-27 02:14:45,897][105620] Updated weights for policy 1, policy_version 1492786 (0.0009) [2023-12-27 02:14:45,904][105692] Updated weights for policy 0, policy_version 1490335 (0.0009) [2023-12-27 02:14:45,947][105620] Updated weights for policy 1, policy_version 1492796 (0.0006) [2023-12-27 02:14:45,995][105620] Updated weights for policy 1, policy_version 1492806 (0.0009) [2023-12-27 02:14:46,050][105620] Updated weights for policy 1, policy_version 1492816 (0.0008) [2023-12-27 02:14:46,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 763797504. Throughput: 0: 9356.0, 1: 9927.3. Samples: 763761520. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:14:46,063][104569] Avg episode reward: [(0, '8442.756'), (1, '9263.606')] [2023-12-27 02:14:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001490344_381583360.pth... [2023-12-27 02:14:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001492816_382214144.pth... [2023-12-27 02:14:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001489256_381304832.pth [2023-12-27 02:14:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001491632_381911040.pth [2023-12-27 02:14:46,673][105692] Updated weights for policy 0, policy_version 1490345 (0.0007) [2023-12-27 02:14:46,738][105692] Updated weights for policy 0, policy_version 1490355 (0.0008) [2023-12-27 02:14:46,804][105692] Updated weights for policy 0, policy_version 1490365 (0.0007) [2023-12-27 02:14:46,811][105620] Updated weights for policy 1, policy_version 1492826 (0.0010) [2023-12-27 02:14:46,860][105620] Updated weights for policy 1, policy_version 1492836 (0.0007) [2023-12-27 02:14:46,864][105692] Updated weights for policy 0, policy_version 1490375 (0.0011) [2023-12-27 02:14:46,909][105620] Updated weights for policy 1, policy_version 1492846 (0.0005) [2023-12-27 02:14:47,519][105692] Updated weights for policy 0, policy_version 1490385 (0.0011) [2023-12-27 02:14:47,557][105620] Updated weights for policy 1, policy_version 1492856 (0.0006) [2023-12-27 02:14:47,578][105692] Updated weights for policy 0, policy_version 1490395 (0.0011) [2023-12-27 02:14:47,608][105620] Updated weights for policy 1, policy_version 1492866 (0.0005) [2023-12-27 02:14:47,640][105692] Updated weights for policy 0, policy_version 1490405 (0.0011) [2023-12-27 02:14:47,663][105620] Updated weights for policy 1, policy_version 1492876 (0.0006) [2023-12-27 02:14:48,251][105620] Updated weights for policy 1, policy_version 1492886 (0.0008) [2023-12-27 02:14:48,310][105620] Updated weights for policy 1, policy_version 1492896 (0.0008) [2023-12-27 02:14:48,372][105620] Updated weights for policy 1, policy_version 1492906 (0.0008) [2023-12-27 02:14:48,378][105692] Updated weights for policy 0, policy_version 1490415 (0.0008) [2023-12-27 02:14:48,443][105692] Updated weights for policy 0, policy_version 1490425 (0.0008) [2023-12-27 02:14:48,505][105692] Updated weights for policy 0, policy_version 1490435 (0.0009) [2023-12-27 02:14:49,142][105620] Updated weights for policy 1, policy_version 1492916 (0.0007) [2023-12-27 02:14:49,168][105692] Updated weights for policy 0, policy_version 1490445 (0.0007) [2023-12-27 02:14:49,190][105620] Updated weights for policy 1, policy_version 1492926 (0.0009) [2023-12-27 02:14:49,242][105692] Updated weights for policy 0, policy_version 1490455 (0.0007) [2023-12-27 02:14:49,252][105620] Updated weights for policy 1, policy_version 1492936 (0.0009) [2023-12-27 02:14:49,300][105692] Updated weights for policy 0, policy_version 1490465 (0.0006) [2023-12-27 02:14:49,990][105692] Updated weights for policy 0, policy_version 1490475 (0.0007) [2023-12-27 02:14:50,053][105692] Updated weights for policy 0, policy_version 1490485 (0.0009) [2023-12-27 02:14:50,062][105620] Updated weights for policy 1, policy_version 1492946 (0.0009) [2023-12-27 02:14:50,114][105692] Updated weights for policy 0, policy_version 1490495 (0.0006) [2023-12-27 02:14:50,124][105620] Updated weights for policy 1, policy_version 1492956 (0.0009) [2023-12-27 02:14:50,185][105620] Updated weights for policy 1, policy_version 1492966 (0.0009) [2023-12-27 02:14:50,252][105620] Updated weights for policy 1, policy_version 1492976 (0.0008) [2023-12-27 02:14:50,834][105692] Updated weights for policy 0, policy_version 1490505 (0.0007) [2023-12-27 02:14:50,893][105692] Updated weights for policy 0, policy_version 1490515 (0.0009) [2023-12-27 02:14:50,952][105692] Updated weights for policy 0, policy_version 1490525 (0.0009) [2023-12-27 02:14:51,015][105692] Updated weights for policy 0, policy_version 1490535 (0.0006) [2023-12-27 02:14:51,021][105620] Updated weights for policy 1, policy_version 1492986 (0.0008) [2023-12-27 02:14:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 763887616. Throughput: 0: 9304.6, 1: 9963.6. Samples: 763877216. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:14:51,063][104569] Avg episode reward: [(0, '8081.768'), (1, '9189.841')] [2023-12-27 02:14:51,080][105620] Updated weights for policy 1, policy_version 1492996 (0.0007) [2023-12-27 02:14:51,144][105620] Updated weights for policy 1, policy_version 1493006 (0.0008) [2023-12-27 02:14:51,804][105692] Updated weights for policy 0, policy_version 1490545 (0.0008) [2023-12-27 02:14:51,863][105692] Updated weights for policy 0, policy_version 1490555 (0.0007) [2023-12-27 02:14:51,886][105620] Updated weights for policy 1, policy_version 1493016 (0.0010) [2023-12-27 02:14:51,921][105692] Updated weights for policy 0, policy_version 1490565 (0.0006) [2023-12-27 02:14:51,942][105620] Updated weights for policy 1, policy_version 1493026 (0.0011) [2023-12-27 02:14:52,001][105620] Updated weights for policy 1, policy_version 1493036 (0.0011) [2023-12-27 02:14:52,682][105692] Updated weights for policy 0, policy_version 1490575 (0.0009) [2023-12-27 02:14:52,734][105692] Updated weights for policy 0, policy_version 1490585 (0.0008) [2023-12-27 02:14:52,756][105620] Updated weights for policy 1, policy_version 1493046 (0.0010) [2023-12-27 02:14:52,786][105692] Updated weights for policy 0, policy_version 1490595 (0.0006) [2023-12-27 02:14:52,811][105620] Updated weights for policy 1, policy_version 1493056 (0.0010) [2023-12-27 02:14:52,864][105620] Updated weights for policy 1, policy_version 1493066 (0.0010) [2023-12-27 02:14:53,499][105692] Updated weights for policy 0, policy_version 1490605 (0.0008) [2023-12-27 02:14:53,568][105692] Updated weights for policy 0, policy_version 1490615 (0.0011) [2023-12-27 02:14:53,628][105620] Updated weights for policy 1, policy_version 1493076 (0.0010) [2023-12-27 02:14:53,633][105692] Updated weights for policy 0, policy_version 1490625 (0.0011) [2023-12-27 02:14:53,676][105620] Updated weights for policy 1, policy_version 1493086 (0.0010) [2023-12-27 02:14:53,723][105620] Updated weights for policy 1, policy_version 1493096 (0.0010) [2023-12-27 02:14:54,287][105692] Updated weights for policy 0, policy_version 1490635 (0.0010) [2023-12-27 02:14:54,353][105692] Updated weights for policy 0, policy_version 1490645 (0.0011) [2023-12-27 02:14:54,417][105692] Updated weights for policy 0, policy_version 1490655 (0.0011) [2023-12-27 02:14:54,475][105620] Updated weights for policy 1, policy_version 1493106 (0.0010) [2023-12-27 02:14:54,527][105620] Updated weights for policy 1, policy_version 1493116 (0.0010) [2023-12-27 02:14:54,586][105620] Updated weights for policy 1, policy_version 1493126 (0.0010) [2023-12-27 02:14:54,634][105620] Updated weights for policy 1, policy_version 1493136 (0.0010) [2023-12-27 02:14:55,052][105692] Updated weights for policy 0, policy_version 1490665 (0.0011) [2023-12-27 02:14:55,102][105692] Updated weights for policy 0, policy_version 1490675 (0.0009) [2023-12-27 02:14:55,158][105692] Updated weights for policy 0, policy_version 1490685 (0.0009) [2023-12-27 02:14:55,208][105692] Updated weights for policy 0, policy_version 1490695 (0.0009) [2023-12-27 02:14:55,380][105620] Updated weights for policy 1, policy_version 1493146 (0.0009) [2023-12-27 02:14:55,427][105620] Updated weights for policy 1, policy_version 1493156 (0.0008) [2023-12-27 02:14:55,477][105620] Updated weights for policy 1, policy_version 1493166 (0.0009) [2023-12-27 02:14:55,937][105692] Updated weights for policy 0, policy_version 1490705 (0.0006) [2023-12-27 02:14:55,998][105692] Updated weights for policy 0, policy_version 1490715 (0.0006) [2023-12-27 02:14:56,061][105692] Updated weights for policy 0, policy_version 1490725 (0.0006) [2023-12-27 02:14:56,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 763977728. Throughput: 0: 9309.1, 1: 9933.6. Samples: 763991516. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:14:56,063][104569] Avg episode reward: [(0, '8262.267'), (1, '9207.930')] [2023-12-27 02:14:56,144][105620] Updated weights for policy 1, policy_version 1493176 (0.0011) [2023-12-27 02:14:56,200][105620] Updated weights for policy 1, policy_version 1493186 (0.0010) [2023-12-27 02:14:56,248][105620] Updated weights for policy 1, policy_version 1493196 (0.0010) [2023-12-27 02:14:56,589][105692] Updated weights for policy 0, policy_version 1490735 (0.0006) [2023-12-27 02:14:56,647][105692] Updated weights for policy 0, policy_version 1490745 (0.0005) [2023-12-27 02:14:56,703][105692] Updated weights for policy 0, policy_version 1490755 (0.0005) [2023-12-27 02:14:56,912][105620] Updated weights for policy 1, policy_version 1493206 (0.0008) [2023-12-27 02:14:56,968][105620] Updated weights for policy 1, policy_version 1493216 (0.0005) [2023-12-27 02:14:57,033][105620] Updated weights for policy 1, policy_version 1493226 (0.0010) [2023-12-27 02:14:57,234][105692] Updated weights for policy 0, policy_version 1490765 (0.0005) [2023-12-27 02:14:57,300][105692] Updated weights for policy 0, policy_version 1490775 (0.0005) [2023-12-27 02:14:57,357][105692] Updated weights for policy 0, policy_version 1490785 (0.0006) [2023-12-27 02:14:57,701][105620] Updated weights for policy 1, policy_version 1493236 (0.0007) [2023-12-27 02:14:57,755][105620] Updated weights for policy 1, policy_version 1493246 (0.0008) [2023-12-27 02:14:57,810][105620] Updated weights for policy 1, policy_version 1493256 (0.0005) [2023-12-27 02:14:58,065][105692] Updated weights for policy 0, policy_version 1490795 (0.0007) [2023-12-27 02:14:58,121][105692] Updated weights for policy 0, policy_version 1490805 (0.0009) [2023-12-27 02:14:58,180][105692] Updated weights for policy 0, policy_version 1490815 (0.0008) [2023-12-27 02:14:58,399][105620] Updated weights for policy 1, policy_version 1493266 (0.0006) [2023-12-27 02:14:58,461][105620] Updated weights for policy 1, policy_version 1493276 (0.0009) [2023-12-27 02:14:58,528][105620] Updated weights for policy 1, policy_version 1493286 (0.0009) [2023-12-27 02:14:59,024][105692] Updated weights for policy 0, policy_version 1490825 (0.0008) [2023-12-27 02:14:59,073][105692] Updated weights for policy 0, policy_version 1490835 (0.0010) [2023-12-27 02:14:59,121][105692] Updated weights for policy 0, policy_version 1490845 (0.0009) [2023-12-27 02:14:59,172][105692] Updated weights for policy 0, policy_version 1490855 (0.0009) [2023-12-27 02:14:59,294][105620] Updated weights for policy 1, policy_version 1493297 (0.0009) [2023-12-27 02:14:59,359][105620] Updated weights for policy 1, policy_version 1493307 (0.0009) [2023-12-27 02:14:59,416][105620] Updated weights for policy 1, policy_version 1493317 (0.0009) [2023-12-27 02:14:59,467][105620] Updated weights for policy 1, policy_version 1493327 (0.0007) [2023-12-27 02:14:59,943][105692] Updated weights for policy 0, policy_version 1490865 (0.0009) [2023-12-27 02:15:00,005][105692] Updated weights for policy 0, policy_version 1490875 (0.0007) [2023-12-27 02:15:00,061][105692] Updated weights for policy 0, policy_version 1490885 (0.0008) [2023-12-27 02:15:00,190][105620] Updated weights for policy 1, policy_version 1493337 (0.0008) [2023-12-27 02:15:00,241][105620] Updated weights for policy 1, policy_version 1493347 (0.0010) [2023-12-27 02:15:00,295][105620] Updated weights for policy 1, policy_version 1493357 (0.0010) [2023-12-27 02:15:00,772][105692] Updated weights for policy 0, policy_version 1490895 (0.0007) [2023-12-27 02:15:00,818][105692] Updated weights for policy 0, policy_version 1490905 (0.0005) [2023-12-27 02:15:00,871][105692] Updated weights for policy 0, policy_version 1490915 (0.0005) [2023-12-27 02:15:01,021][105620] Updated weights for policy 1, policy_version 1493367 (0.0011) [2023-12-27 02:15:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 764084224. Throughput: 0: 9393.8, 1: 9995.7. Samples: 764055028. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:15:01,062][104569] Avg episode reward: [(0, '7982.412'), (1, '9281.295')] [2023-12-27 02:15:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001490920_381730816.pth... [2023-12-27 02:15:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001489800_381444096.pth [2023-12-27 02:15:01,084][105620] Updated weights for policy 1, policy_version 1493377 (0.0011) [2023-12-27 02:15:01,150][105620] Updated weights for policy 1, policy_version 1493387 (0.0009) [2023-12-27 02:15:01,176][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001493392_382361600.pth... [2023-12-27 02:15:01,181][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001492208_382058496.pth [2023-12-27 02:15:01,567][105692] Updated weights for policy 0, policy_version 1490925 (0.0006) [2023-12-27 02:15:01,626][105692] Updated weights for policy 0, policy_version 1490935 (0.0006) [2023-12-27 02:15:01,694][105692] Updated weights for policy 0, policy_version 1490945 (0.0008) [2023-12-27 02:15:01,793][105620] Updated weights for policy 1, policy_version 1493397 (0.0007) [2023-12-27 02:15:01,851][105620] Updated weights for policy 1, policy_version 1493407 (0.0010) [2023-12-27 02:15:01,911][105620] Updated weights for policy 1, policy_version 1493417 (0.0011) [2023-12-27 02:15:02,425][105692] Updated weights for policy 0, policy_version 1490955 (0.0008) [2023-12-27 02:15:02,486][105692] Updated weights for policy 0, policy_version 1490965 (0.0009) [2023-12-27 02:15:02,545][105692] Updated weights for policy 0, policy_version 1490975 (0.0009) [2023-12-27 02:15:02,565][105620] Updated weights for policy 1, policy_version 1493427 (0.0006) [2023-12-27 02:15:02,622][105620] Updated weights for policy 1, policy_version 1493437 (0.0007) [2023-12-27 02:15:02,683][105620] Updated weights for policy 1, policy_version 1493447 (0.0009) [2023-12-27 02:15:03,270][105620] Updated weights for policy 1, policy_version 1493457 (0.0009) [2023-12-27 02:15:03,289][105692] Updated weights for policy 0, policy_version 1490985 (0.0009) [2023-12-27 02:15:03,325][105620] Updated weights for policy 1, policy_version 1493467 (0.0006) [2023-12-27 02:15:03,350][105692] Updated weights for policy 0, policy_version 1490995 (0.0008) [2023-12-27 02:15:03,378][105620] Updated weights for policy 1, policy_version 1493477 (0.0005) [2023-12-27 02:15:03,406][105692] Updated weights for policy 0, policy_version 1491005 (0.0009) [2023-12-27 02:15:03,434][105620] Updated weights for policy 1, policy_version 1493487 (0.0005) [2023-12-27 02:15:03,468][105692] Updated weights for policy 0, policy_version 1491015 (0.0009) [2023-12-27 02:15:04,076][105620] Updated weights for policy 1, policy_version 1493497 (0.0008) [2023-12-27 02:15:04,145][105620] Updated weights for policy 1, policy_version 1493507 (0.0009) [2023-12-27 02:15:04,202][105620] Updated weights for policy 1, policy_version 1493517 (0.0009) [2023-12-27 02:15:04,283][105692] Updated weights for policy 0, policy_version 1491025 (0.0009) [2023-12-27 02:15:04,347][105692] Updated weights for policy 0, policy_version 1491035 (0.0008) [2023-12-27 02:15:04,410][105692] Updated weights for policy 0, policy_version 1491045 (0.0005) [2023-12-27 02:15:05,000][105620] Updated weights for policy 1, policy_version 1493527 (0.0009) [2023-12-27 02:15:05,054][105620] Updated weights for policy 1, policy_version 1493537 (0.0007) [2023-12-27 02:15:05,067][105692] Updated weights for policy 0, policy_version 1491055 (0.0008) [2023-12-27 02:15:05,104][105620] Updated weights for policy 1, policy_version 1493547 (0.0009) [2023-12-27 02:15:05,126][105692] Updated weights for policy 0, policy_version 1491065 (0.0008) [2023-12-27 02:15:05,182][105692] Updated weights for policy 0, policy_version 1491075 (0.0008) [2023-12-27 02:15:05,833][105620] Updated weights for policy 1, policy_version 1493557 (0.0009) [2023-12-27 02:15:05,857][105692] Updated weights for policy 0, policy_version 1491085 (0.0008) [2023-12-27 02:15:05,879][105620] Updated weights for policy 1, policy_version 1493567 (0.0008) [2023-12-27 02:15:05,902][105692] Updated weights for policy 0, policy_version 1491095 (0.0006) [2023-12-27 02:15:05,939][105620] Updated weights for policy 1, policy_version 1493577 (0.0008) [2023-12-27 02:15:05,961][105692] Updated weights for policy 0, policy_version 1491105 (0.0007) [2023-12-27 02:15:06,062][104569] Fps is (10 sec: 21298.9, 60 sec: 19524.2, 300 sec: 19660.8). Total num frames: 764190720. Throughput: 0: 9374.5, 1: 10038.3. Samples: 764172596. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:15:06,063][104569] Avg episode reward: [(0, '7983.152'), (1, '9261.739')] [2023-12-27 02:15:06,714][105692] Updated weights for policy 0, policy_version 1491115 (0.0008) [2023-12-27 02:15:06,726][105620] Updated weights for policy 1, policy_version 1493587 (0.0007) [2023-12-27 02:15:06,775][105692] Updated weights for policy 0, policy_version 1491125 (0.0008) [2023-12-27 02:15:06,785][105620] Updated weights for policy 1, policy_version 1493597 (0.0007) [2023-12-27 02:15:06,834][105692] Updated weights for policy 0, policy_version 1491135 (0.0007) [2023-12-27 02:15:06,845][105620] Updated weights for policy 1, policy_version 1493607 (0.0007) [2023-12-27 02:15:07,505][105692] Updated weights for policy 0, policy_version 1491145 (0.0007) [2023-12-27 02:15:07,557][105692] Updated weights for policy 0, policy_version 1491155 (0.0005) [2023-12-27 02:15:07,610][105692] Updated weights for policy 0, policy_version 1491165 (0.0005) [2023-12-27 02:15:07,629][105620] Updated weights for policy 1, policy_version 1493617 (0.0006) [2023-12-27 02:15:07,670][105692] Updated weights for policy 0, policy_version 1491175 (0.0009) [2023-12-27 02:15:07,689][105620] Updated weights for policy 1, policy_version 1493627 (0.0006) [2023-12-27 02:15:07,742][105620] Updated weights for policy 1, policy_version 1493637 (0.0005) [2023-12-27 02:15:07,794][105620] Updated weights for policy 1, policy_version 1493647 (0.0005) [2023-12-27 02:15:08,300][105692] Updated weights for policy 0, policy_version 1491185 (0.0006) [2023-12-27 02:15:08,359][105692] Updated weights for policy 0, policy_version 1491195 (0.0007) [2023-12-27 02:15:08,407][105692] Updated weights for policy 0, policy_version 1491205 (0.0008) [2023-12-27 02:15:08,519][105620] Updated weights for policy 1, policy_version 1493657 (0.0008) [2023-12-27 02:15:08,581][105620] Updated weights for policy 1, policy_version 1493667 (0.0009) [2023-12-27 02:15:08,641][105620] Updated weights for policy 1, policy_version 1493677 (0.0006) [2023-12-27 02:15:09,208][105692] Updated weights for policy 0, policy_version 1491215 (0.0006) [2023-12-27 02:15:09,264][105620] Updated weights for policy 1, policy_version 1493687 (0.0008) [2023-12-27 02:15:09,276][105692] Updated weights for policy 0, policy_version 1491225 (0.0007) [2023-12-27 02:15:09,318][105620] Updated weights for policy 1, policy_version 1493697 (0.0010) [2023-12-27 02:15:09,340][105692] Updated weights for policy 0, policy_version 1491235 (0.0007) [2023-12-27 02:15:09,387][105620] Updated weights for policy 1, policy_version 1493707 (0.0011) [2023-12-27 02:15:10,009][105692] Updated weights for policy 0, policy_version 1491245 (0.0008) [2023-12-27 02:15:10,081][105692] Updated weights for policy 0, policy_version 1491255 (0.0007) [2023-12-27 02:15:10,136][105692] Updated weights for policy 0, policy_version 1491265 (0.0008) [2023-12-27 02:15:10,153][105620] Updated weights for policy 1, policy_version 1493717 (0.0010) [2023-12-27 02:15:10,213][105620] Updated weights for policy 1, policy_version 1493727 (0.0010) [2023-12-27 02:15:10,274][105620] Updated weights for policy 1, policy_version 1493737 (0.0011) [2023-12-27 02:15:10,768][105692] Updated weights for policy 0, policy_version 1491275 (0.0007) [2023-12-27 02:15:10,816][105692] Updated weights for policy 0, policy_version 1491285 (0.0007) [2023-12-27 02:15:10,865][105692] Updated weights for policy 0, policy_version 1491295 (0.0008) [2023-12-27 02:15:11,002][105620] Updated weights for policy 1, policy_version 1493747 (0.0010) [2023-12-27 02:15:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 764280832. Throughput: 0: 9516.0, 1: 9970.8. Samples: 764289540. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:15:11,063][104569] Avg episode reward: [(0, '8074.666'), (1, '9087.095')] [2023-12-27 02:15:11,068][105620] Updated weights for policy 1, policy_version 1493757 (0.0011) [2023-12-27 02:15:11,140][105620] Updated weights for policy 1, policy_version 1493767 (0.0011) [2023-12-27 02:15:11,539][105692] Updated weights for policy 0, policy_version 1491305 (0.0008) [2023-12-27 02:15:11,601][105692] Updated weights for policy 0, policy_version 1491315 (0.0008) [2023-12-27 02:15:11,669][105692] Updated weights for policy 0, policy_version 1491325 (0.0008) [2023-12-27 02:15:11,735][105692] Updated weights for policy 0, policy_version 1491335 (0.0008) [2023-12-27 02:15:11,906][105620] Updated weights for policy 1, policy_version 1493777 (0.0010) [2023-12-27 02:15:11,959][105620] Updated weights for policy 1, policy_version 1493787 (0.0006) [2023-12-27 02:15:12,017][105620] Updated weights for policy 1, policy_version 1493797 (0.0005) [2023-12-27 02:15:12,079][105620] Updated weights for policy 1, policy_version 1493807 (0.0006) [2023-12-27 02:15:12,508][105692] Updated weights for policy 0, policy_version 1491345 (0.0006) [2023-12-27 02:15:12,569][105692] Updated weights for policy 0, policy_version 1491355 (0.0008) [2023-12-27 02:15:12,626][105692] Updated weights for policy 0, policy_version 1491365 (0.0009) [2023-12-27 02:15:12,744][105620] Updated weights for policy 1, policy_version 1493817 (0.0009) [2023-12-27 02:15:12,811][105620] Updated weights for policy 1, policy_version 1493827 (0.0010) [2023-12-27 02:15:12,872][105620] Updated weights for policy 1, policy_version 1493837 (0.0007) [2023-12-27 02:15:13,392][105692] Updated weights for policy 0, policy_version 1491375 (0.0009) [2023-12-27 02:15:13,442][105692] Updated weights for policy 0, policy_version 1491385 (0.0009) [2023-12-27 02:15:13,489][105692] Updated weights for policy 0, policy_version 1491395 (0.0007) [2023-12-27 02:15:13,560][105620] Updated weights for policy 1, policy_version 1493847 (0.0007) [2023-12-27 02:15:13,614][105620] Updated weights for policy 1, policy_version 1493857 (0.0009) [2023-12-27 02:15:13,660][105620] Updated weights for policy 1, policy_version 1493867 (0.0008) [2023-12-27 02:15:14,167][105692] Updated weights for policy 0, policy_version 1491405 (0.0007) [2023-12-27 02:15:14,236][105692] Updated weights for policy 0, policy_version 1491415 (0.0006) [2023-12-27 02:15:14,297][105692] Updated weights for policy 0, policy_version 1491425 (0.0005) [2023-12-27 02:15:14,408][105620] Updated weights for policy 1, policy_version 1493877 (0.0009) [2023-12-27 02:15:14,462][105620] Updated weights for policy 1, policy_version 1493887 (0.0008) [2023-12-27 02:15:14,516][105620] Updated weights for policy 1, policy_version 1493897 (0.0008) [2023-12-27 02:15:14,906][105692] Updated weights for policy 0, policy_version 1491435 (0.0005) [2023-12-27 02:15:14,969][105692] Updated weights for policy 0, policy_version 1491445 (0.0006) [2023-12-27 02:15:15,031][105692] Updated weights for policy 0, policy_version 1491455 (0.0008) [2023-12-27 02:15:15,315][105620] Updated weights for policy 1, policy_version 1493907 (0.0009) [2023-12-27 02:15:15,368][105620] Updated weights for policy 1, policy_version 1493917 (0.0011) [2023-12-27 02:15:15,421][105620] Updated weights for policy 1, policy_version 1493927 (0.0010) [2023-12-27 02:15:15,788][105692] Updated weights for policy 0, policy_version 1491465 (0.0009) [2023-12-27 02:15:15,853][105692] Updated weights for policy 0, policy_version 1491475 (0.0007) [2023-12-27 02:15:15,915][105692] Updated weights for policy 0, policy_version 1491485 (0.0006) [2023-12-27 02:15:15,983][105692] Updated weights for policy 0, policy_version 1491495 (0.0008) [2023-12-27 02:15:16,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 764379136. Throughput: 0: 9574.5, 1: 9878.8. Samples: 764347104. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:15:16,063][104569] Avg episode reward: [(0, '8073.286'), (1, '9177.070')] [2023-12-27 02:15:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001491496_381878272.pth... [2023-12-27 02:15:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001493936_382500864.pth... [2023-12-27 02:15:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001490344_381583360.pth [2023-12-27 02:15:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001492816_382214144.pth [2023-12-27 02:15:16,140][105620] Updated weights for policy 1, policy_version 1493937 (0.0010) [2023-12-27 02:15:16,209][105620] Updated weights for policy 1, policy_version 1493947 (0.0006) [2023-12-27 02:15:16,271][105620] Updated weights for policy 1, policy_version 1493957 (0.0005) [2023-12-27 02:15:16,331][105620] Updated weights for policy 1, policy_version 1493967 (0.0005) [2023-12-27 02:15:16,648][105692] Updated weights for policy 0, policy_version 1491505 (0.0010) [2023-12-27 02:15:16,699][105692] Updated weights for policy 0, policy_version 1491515 (0.0010) [2023-12-27 02:15:16,751][105692] Updated weights for policy 0, policy_version 1491525 (0.0010) [2023-12-27 02:15:16,920][105620] Updated weights for policy 1, policy_version 1493977 (0.0006) [2023-12-27 02:15:16,972][105620] Updated weights for policy 1, policy_version 1493987 (0.0007) [2023-12-27 02:15:17,025][105620] Updated weights for policy 1, policy_version 1493997 (0.0005) [2023-12-27 02:15:17,501][105692] Updated weights for policy 0, policy_version 1491535 (0.0011) [2023-12-27 02:15:17,559][105692] Updated weights for policy 0, policy_version 1491545 (0.0010) [2023-12-27 02:15:17,566][105620] Updated weights for policy 1, policy_version 1494007 (0.0006) [2023-12-27 02:15:17,617][105692] Updated weights for policy 0, policy_version 1491555 (0.0010) [2023-12-27 02:15:17,623][105620] Updated weights for policy 1, policy_version 1494017 (0.0006) [2023-12-27 02:15:17,680][105620] Updated weights for policy 1, policy_version 1494027 (0.0007) [2023-12-27 02:15:18,382][105692] Updated weights for policy 0, policy_version 1491565 (0.0011) [2023-12-27 02:15:18,400][105620] Updated weights for policy 1, policy_version 1494037 (0.0007) [2023-12-27 02:15:18,437][105692] Updated weights for policy 0, policy_version 1491575 (0.0011) [2023-12-27 02:15:18,460][105620] Updated weights for policy 1, policy_version 1494047 (0.0006) [2023-12-27 02:15:18,482][105692] Updated weights for policy 0, policy_version 1491585 (0.0010) [2023-12-27 02:15:18,516][105620] Updated weights for policy 1, policy_version 1494057 (0.0005) [2023-12-27 02:15:19,068][105620] Updated weights for policy 1, policy_version 1494067 (0.0006) [2023-12-27 02:15:19,114][105620] Updated weights for policy 1, policy_version 1494077 (0.0005) [2023-12-27 02:15:19,163][105620] Updated weights for policy 1, policy_version 1494087 (0.0006) [2023-12-27 02:15:19,221][105692] Updated weights for policy 0, policy_version 1491595 (0.0011) [2023-12-27 02:15:19,284][105692] Updated weights for policy 0, policy_version 1491605 (0.0010) [2023-12-27 02:15:19,344][105692] Updated weights for policy 0, policy_version 1491615 (0.0011) [2023-12-27 02:15:19,809][105620] Updated weights for policy 1, policy_version 1494097 (0.0007) [2023-12-27 02:15:19,873][105620] Updated weights for policy 1, policy_version 1494107 (0.0008) [2023-12-27 02:15:19,931][105620] Updated weights for policy 1, policy_version 1494117 (0.0009) [2023-12-27 02:15:19,997][105620] Updated weights for policy 1, policy_version 1494127 (0.0008) [2023-12-27 02:15:20,077][105692] Updated weights for policy 0, policy_version 1491625 (0.0009) [2023-12-27 02:15:20,136][105692] Updated weights for policy 0, policy_version 1491635 (0.0010) [2023-12-27 02:15:20,196][105692] Updated weights for policy 0, policy_version 1491645 (0.0011) [2023-12-27 02:15:20,256][105692] Updated weights for policy 0, policy_version 1491655 (0.0011) [2023-12-27 02:15:20,772][105620] Updated weights for policy 1, policy_version 1494137 (0.0008) [2023-12-27 02:15:20,829][105620] Updated weights for policy 1, policy_version 1494147 (0.0008) [2023-12-27 02:15:20,878][105620] Updated weights for policy 1, policy_version 1494157 (0.0008) [2023-12-27 02:15:20,992][105692] Updated weights for policy 0, policy_version 1491665 (0.0010) [2023-12-27 02:15:21,054][105692] Updated weights for policy 0, policy_version 1491675 (0.0011) [2023-12-27 02:15:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 764477440. Throughput: 0: 9623.4, 1: 9893.2. Samples: 764468168. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:15:21,063][104569] Avg episode reward: [(0, '8260.779'), (1, '9259.438')] [2023-12-27 02:15:21,107][105692] Updated weights for policy 0, policy_version 1491685 (0.0011) [2023-12-27 02:15:21,733][105620] Updated weights for policy 1, policy_version 1494167 (0.0007) [2023-12-27 02:15:21,793][105620] Updated weights for policy 1, policy_version 1494177 (0.0008) [2023-12-27 02:15:21,840][105692] Updated weights for policy 0, policy_version 1491695 (0.0009) [2023-12-27 02:15:21,843][105620] Updated weights for policy 1, policy_version 1494187 (0.0006) [2023-12-27 02:15:21,909][105692] Updated weights for policy 0, policy_version 1491705 (0.0008) [2023-12-27 02:15:21,975][105692] Updated weights for policy 0, policy_version 1491715 (0.0008) [2023-12-27 02:15:22,554][105620] Updated weights for policy 1, policy_version 1494197 (0.0009) [2023-12-27 02:15:22,616][105620] Updated weights for policy 1, policy_version 1494207 (0.0009) [2023-12-27 02:15:22,663][105620] Updated weights for policy 1, policy_version 1494217 (0.0008) [2023-12-27 02:15:22,739][105692] Updated weights for policy 0, policy_version 1491725 (0.0009) [2023-12-27 02:15:22,794][105692] Updated weights for policy 0, policy_version 1491735 (0.0009) [2023-12-27 02:15:22,840][105692] Updated weights for policy 0, policy_version 1491745 (0.0008) [2023-12-27 02:15:23,356][105620] Updated weights for policy 1, policy_version 1494227 (0.0006) [2023-12-27 02:15:23,412][105620] Updated weights for policy 1, policy_version 1494237 (0.0009) [2023-12-27 02:15:23,467][105620] Updated weights for policy 1, policy_version 1494247 (0.0009) [2023-12-27 02:15:23,646][105692] Updated weights for policy 0, policy_version 1491755 (0.0009) [2023-12-27 02:15:23,697][105692] Updated weights for policy 0, policy_version 1491765 (0.0009) [2023-12-27 02:15:23,743][105692] Updated weights for policy 0, policy_version 1491775 (0.0009) [2023-12-27 02:15:24,233][105620] Updated weights for policy 1, policy_version 1494257 (0.0009) [2023-12-27 02:15:24,286][105620] Updated weights for policy 1, policy_version 1494267 (0.0008) [2023-12-27 02:15:24,349][105620] Updated weights for policy 1, policy_version 1494277 (0.0005) [2023-12-27 02:15:24,396][105620] Updated weights for policy 1, policy_version 1494287 (0.0007) [2023-12-27 02:15:24,534][105692] Updated weights for policy 0, policy_version 1491785 (0.0007) [2023-12-27 02:15:24,585][105692] Updated weights for policy 0, policy_version 1491795 (0.0009) [2023-12-27 02:15:24,636][105692] Updated weights for policy 0, policy_version 1491805 (0.0009) [2023-12-27 02:15:24,683][105692] Updated weights for policy 0, policy_version 1491815 (0.0009) [2023-12-27 02:15:25,164][105620] Updated weights for policy 1, policy_version 1494298 (0.0010) [2023-12-27 02:15:25,217][105620] Updated weights for policy 1, policy_version 1494309 (0.0010) [2023-12-27 02:15:25,298][105620] Updated weights for policy 1, policy_version 1494319 (0.0009) [2023-12-27 02:15:25,302][105692] Updated weights for policy 0, policy_version 1491825 (0.0006) [2023-12-27 02:15:25,360][105692] Updated weights for policy 0, policy_version 1491835 (0.0009) [2023-12-27 02:15:25,408][105692] Updated weights for policy 0, policy_version 1491845 (0.0009) [2023-12-27 02:15:26,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 764567552. Throughput: 0: 9667.8, 1: 9753.5. Samples: 764580064. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:15:26,063][104569] Avg episode reward: [(0, '7810.766'), (1, '9258.434')] [2023-12-27 02:15:26,067][105692] Updated weights for policy 0, policy_version 1491855 (0.0009) [2023-12-27 02:15:26,106][105620] Updated weights for policy 1, policy_version 1494329 (0.0007) [2023-12-27 02:15:26,124][105692] Updated weights for policy 0, policy_version 1491865 (0.0007) [2023-12-27 02:15:26,155][105620] Updated weights for policy 1, policy_version 1494339 (0.0007) [2023-12-27 02:15:26,178][105692] Updated weights for policy 0, policy_version 1491875 (0.0007) [2023-12-27 02:15:26,200][105620] Updated weights for policy 1, policy_version 1494349 (0.0007) [2023-12-27 02:15:26,862][105692] Updated weights for policy 0, policy_version 1491885 (0.0008) [2023-12-27 02:15:26,912][105692] Updated weights for policy 0, policy_version 1491895 (0.0008) [2023-12-27 02:15:26,928][105620] Updated weights for policy 1, policy_version 1494359 (0.0007) [2023-12-27 02:15:26,962][105692] Updated weights for policy 0, policy_version 1491905 (0.0008) [2023-12-27 02:15:26,976][105620] Updated weights for policy 1, policy_version 1494369 (0.0007) [2023-12-27 02:15:27,023][105620] Updated weights for policy 1, policy_version 1494379 (0.0008) [2023-12-27 02:15:27,617][105620] Updated weights for policy 1, policy_version 1494389 (0.0009) [2023-12-27 02:15:27,668][105620] Updated weights for policy 1, policy_version 1494399 (0.0009) [2023-12-27 02:15:27,723][105620] Updated weights for policy 1, policy_version 1494409 (0.0009) [2023-12-27 02:15:27,778][105692] Updated weights for policy 0, policy_version 1491915 (0.0007) [2023-12-27 02:15:27,825][105692] Updated weights for policy 0, policy_version 1491925 (0.0009) [2023-12-27 02:15:27,874][105692] Updated weights for policy 0, policy_version 1491935 (0.0008) [2023-12-27 02:15:28,423][105620] Updated weights for policy 1, policy_version 1494419 (0.0009) [2023-12-27 02:15:28,473][105620] Updated weights for policy 1, policy_version 1494429 (0.0009) [2023-12-27 02:15:28,528][105620] Updated weights for policy 1, policy_version 1494439 (0.0009) [2023-12-27 02:15:28,604][105692] Updated weights for policy 0, policy_version 1491945 (0.0006) [2023-12-27 02:15:28,668][105692] Updated weights for policy 0, policy_version 1491955 (0.0009) [2023-12-27 02:15:28,728][105692] Updated weights for policy 0, policy_version 1491965 (0.0009) [2023-12-27 02:15:28,791][105692] Updated weights for policy 0, policy_version 1491975 (0.0009) [2023-12-27 02:15:29,262][105620] Updated weights for policy 1, policy_version 1494449 (0.0009) [2023-12-27 02:15:29,328][105620] Updated weights for policy 1, policy_version 1494459 (0.0008) [2023-12-27 02:15:29,390][105620] Updated weights for policy 1, policy_version 1494469 (0.0007) [2023-12-27 02:15:29,443][105692] Updated weights for policy 0, policy_version 1491985 (0.0011) [2023-12-27 02:15:29,452][105620] Updated weights for policy 1, policy_version 1494479 (0.0007) [2023-12-27 02:15:29,512][105692] Updated weights for policy 0, policy_version 1491995 (0.0011) [2023-12-27 02:15:29,566][105692] Updated weights for policy 0, policy_version 1492005 (0.0011) [2023-12-27 02:15:30,181][105692] Updated weights for policy 0, policy_version 1492015 (0.0011) [2023-12-27 02:15:30,183][105620] Updated weights for policy 1, policy_version 1494489 (0.0007) [2023-12-27 02:15:30,229][105692] Updated weights for policy 0, policy_version 1492025 (0.0010) [2023-12-27 02:15:30,243][105620] Updated weights for policy 1, policy_version 1494499 (0.0006) [2023-12-27 02:15:30,281][105692] Updated weights for policy 0, policy_version 1492035 (0.0010) [2023-12-27 02:15:30,295][105620] Updated weights for policy 1, policy_version 1494509 (0.0005) [2023-12-27 02:15:30,898][105620] Updated weights for policy 1, policy_version 1494519 (0.0006) [2023-12-27 02:15:30,939][105692] Updated weights for policy 0, policy_version 1492045 (0.0011) [2023-12-27 02:15:30,964][105620] Updated weights for policy 1, policy_version 1494529 (0.0009) [2023-12-27 02:15:31,002][105692] Updated weights for policy 0, policy_version 1492055 (0.0011) [2023-12-27 02:15:31,022][105620] Updated weights for policy 1, policy_version 1494539 (0.0009) [2023-12-27 02:15:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 764674048. Throughput: 0: 9726.2, 1: 9805.6. Samples: 764640448. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:15:31,062][104569] Avg episode reward: [(0, '7902.971'), (1, '9259.487')] [2023-12-27 02:15:31,067][105692] Updated weights for policy 0, policy_version 1492065 (0.0011) [2023-12-27 02:15:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001494544_382656512.pth... [2023-12-27 02:15:31,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001493392_382361600.pth [2023-12-27 02:15:31,112][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001492072_382025728.pth... [2023-12-27 02:15:31,116][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001490920_381730816.pth [2023-12-27 02:15:31,694][105620] Updated weights for policy 1, policy_version 1494549 (0.0010) [2023-12-27 02:15:31,763][105620] Updated weights for policy 1, policy_version 1494559 (0.0011) [2023-12-27 02:15:31,807][105692] Updated weights for policy 0, policy_version 1492075 (0.0008) [2023-12-27 02:15:31,828][105620] Updated weights for policy 1, policy_version 1494569 (0.0011) [2023-12-27 02:15:31,864][105692] Updated weights for policy 0, policy_version 1492085 (0.0010) [2023-12-27 02:15:31,926][105692] Updated weights for policy 0, policy_version 1492095 (0.0011) [2023-12-27 02:15:32,512][105620] Updated weights for policy 1, policy_version 1494579 (0.0010) [2023-12-27 02:15:32,578][105620] Updated weights for policy 1, policy_version 1494589 (0.0011) [2023-12-27 02:15:32,579][105692] Updated weights for policy 0, policy_version 1492105 (0.0010) [2023-12-27 02:15:32,635][105692] Updated weights for policy 0, policy_version 1492115 (0.0007) [2023-12-27 02:15:32,640][105620] Updated weights for policy 1, policy_version 1494599 (0.0010) [2023-12-27 02:15:32,693][105692] Updated weights for policy 0, policy_version 1492125 (0.0007) [2023-12-27 02:15:32,752][105692] Updated weights for policy 0, policy_version 1492135 (0.0005) [2023-12-27 02:15:33,340][105620] Updated weights for policy 1, policy_version 1494609 (0.0010) [2023-12-27 02:15:33,387][105692] Updated weights for policy 0, policy_version 1492145 (0.0007) [2023-12-27 02:15:33,400][105620] Updated weights for policy 1, policy_version 1494619 (0.0010) [2023-12-27 02:15:33,446][105692] Updated weights for policy 0, policy_version 1492155 (0.0008) [2023-12-27 02:15:33,465][105620] Updated weights for policy 1, policy_version 1494629 (0.0010) [2023-12-27 02:15:33,500][105692] Updated weights for policy 0, policy_version 1492165 (0.0007) [2023-12-27 02:15:33,531][105620] Updated weights for policy 1, policy_version 1494639 (0.0007) [2023-12-27 02:15:34,047][105692] Updated weights for policy 0, policy_version 1492175 (0.0008) [2023-12-27 02:15:34,096][105620] Updated weights for policy 1, policy_version 1494649 (0.0006) [2023-12-27 02:15:34,103][105692] Updated weights for policy 0, policy_version 1492185 (0.0008) [2023-12-27 02:15:34,164][105620] Updated weights for policy 1, policy_version 1494659 (0.0008) [2023-12-27 02:15:34,164][105692] Updated weights for policy 0, policy_version 1492195 (0.0008) [2023-12-27 02:15:34,213][105620] Updated weights for policy 1, policy_version 1494669 (0.0006) [2023-12-27 02:15:34,845][105620] Updated weights for policy 1, policy_version 1494679 (0.0008) [2023-12-27 02:15:34,893][105620] Updated weights for policy 1, policy_version 1494689 (0.0009) [2023-12-27 02:15:34,942][105620] Updated weights for policy 1, policy_version 1494700 (0.0008) [2023-12-27 02:15:34,972][105692] Updated weights for policy 0, policy_version 1492205 (0.0007) [2023-12-27 02:15:35,026][105692] Updated weights for policy 0, policy_version 1492215 (0.0009) [2023-12-27 02:15:35,084][105692] Updated weights for policy 0, policy_version 1492225 (0.0009) [2023-12-27 02:15:35,696][105620] Updated weights for policy 1, policy_version 1494710 (0.0006) [2023-12-27 02:15:35,728][105692] Updated weights for policy 0, policy_version 1492235 (0.0008) [2023-12-27 02:15:35,755][105620] Updated weights for policy 1, policy_version 1494720 (0.0008) [2023-12-27 02:15:35,781][105692] Updated weights for policy 0, policy_version 1492245 (0.0008) [2023-12-27 02:15:35,809][105620] Updated weights for policy 1, policy_version 1494730 (0.0005) [2023-12-27 02:15:35,844][105692] Updated weights for policy 0, policy_version 1492255 (0.0008) [2023-12-27 02:15:36,062][104569] Fps is (10 sec: 21299.7, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 764780544. Throughput: 0: 9843.7, 1: 9878.9. Samples: 764764736. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:15:36,063][104569] Avg episode reward: [(0, '7990.712'), (1, '9258.325')] [2023-12-27 02:15:36,453][105620] Updated weights for policy 1, policy_version 1494740 (0.0006) [2023-12-27 02:15:36,515][105620] Updated weights for policy 1, policy_version 1494750 (0.0009) [2023-12-27 02:15:36,579][105620] Updated weights for policy 1, policy_version 1494760 (0.0009) [2023-12-27 02:15:36,646][105692] Updated weights for policy 0, policy_version 1492265 (0.0009) [2023-12-27 02:15:36,700][105692] Updated weights for policy 0, policy_version 1492275 (0.0008) [2023-12-27 02:15:36,757][105692] Updated weights for policy 0, policy_version 1492285 (0.0008) [2023-12-27 02:15:36,812][105692] Updated weights for policy 0, policy_version 1492295 (0.0008) [2023-12-27 02:15:37,326][105620] Updated weights for policy 1, policy_version 1494770 (0.0009) [2023-12-27 02:15:37,382][105620] Updated weights for policy 1, policy_version 1494780 (0.0009) [2023-12-27 02:15:37,441][105620] Updated weights for policy 1, policy_version 1494790 (0.0009) [2023-12-27 02:15:37,500][105620] Updated weights for policy 1, policy_version 1494800 (0.0009) [2023-12-27 02:15:37,579][105692] Updated weights for policy 0, policy_version 1492305 (0.0010) [2023-12-27 02:15:37,634][105692] Updated weights for policy 0, policy_version 1492315 (0.0010) [2023-12-27 02:15:37,692][105692] Updated weights for policy 0, policy_version 1492325 (0.0010) [2023-12-27 02:15:38,153][105620] Updated weights for policy 1, policy_version 1494810 (0.0009) [2023-12-27 02:15:38,219][105620] Updated weights for policy 1, policy_version 1494820 (0.0009) [2023-12-27 02:15:38,272][105620] Updated weights for policy 1, policy_version 1494830 (0.0009) [2023-12-27 02:15:38,525][105692] Updated weights for policy 0, policy_version 1492335 (0.0009) [2023-12-27 02:15:38,595][105692] Updated weights for policy 0, policy_version 1492345 (0.0009) [2023-12-27 02:15:38,657][105692] Updated weights for policy 0, policy_version 1492355 (0.0009) [2023-12-27 02:15:39,021][105620] Updated weights for policy 1, policy_version 1494840 (0.0009) [2023-12-27 02:15:39,072][105620] Updated weights for policy 1, policy_version 1494850 (0.0008) [2023-12-27 02:15:39,133][105620] Updated weights for policy 1, policy_version 1494860 (0.0009) [2023-12-27 02:15:39,405][105692] Updated weights for policy 0, policy_version 1492365 (0.0008) [2023-12-27 02:15:39,479][105692] Updated weights for policy 0, policy_version 1492375 (0.0010) [2023-12-27 02:15:39,543][105692] Updated weights for policy 0, policy_version 1492385 (0.0010) [2023-12-27 02:15:39,959][105620] Updated weights for policy 1, policy_version 1494870 (0.0008) [2023-12-27 02:15:40,012][105620] Updated weights for policy 1, policy_version 1494880 (0.0007) [2023-12-27 02:15:40,072][105620] Updated weights for policy 1, policy_version 1494890 (0.0006) [2023-12-27 02:15:40,219][105692] Updated weights for policy 0, policy_version 1492395 (0.0007) [2023-12-27 02:15:40,269][105692] Updated weights for policy 0, policy_version 1492405 (0.0009) [2023-12-27 02:15:40,317][105692] Updated weights for policy 0, policy_version 1492415 (0.0009) [2023-12-27 02:15:40,684][105620] Updated weights for policy 1, policy_version 1494900 (0.0009) [2023-12-27 02:15:40,739][105620] Updated weights for policy 1, policy_version 1494910 (0.0009) [2023-12-27 02:15:40,798][105620] Updated weights for policy 1, policy_version 1494920 (0.0010) [2023-12-27 02:15:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 764870656. Throughput: 0: 9789.4, 1: 9941.4. Samples: 764879404. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:15:41,062][104569] Avg episode reward: [(0, '7721.435'), (1, '9257.608')] [2023-12-27 02:15:41,069][105692] Updated weights for policy 0, policy_version 1492425 (0.0007) [2023-12-27 02:15:41,134][105692] Updated weights for policy 0, policy_version 1492435 (0.0009) [2023-12-27 02:15:41,197][105692] Updated weights for policy 0, policy_version 1492445 (0.0010) [2023-12-27 02:15:41,249][105692] Updated weights for policy 0, policy_version 1492456 (0.0009) [2023-12-27 02:15:41,488][105620] Updated weights for policy 1, policy_version 1494930 (0.0006) [2023-12-27 02:15:41,546][105620] Updated weights for policy 1, policy_version 1494940 (0.0006) [2023-12-27 02:15:41,617][105620] Updated weights for policy 1, policy_version 1494950 (0.0006) [2023-12-27 02:15:41,680][105620] Updated weights for policy 1, policy_version 1494960 (0.0008) [2023-12-27 02:15:42,157][105692] Updated weights for policy 0, policy_version 1492466 (0.0010) [2023-12-27 02:15:42,215][105692] Updated weights for policy 0, policy_version 1492476 (0.0009) [2023-12-27 02:15:42,276][105692] Updated weights for policy 0, policy_version 1492486 (0.0008) [2023-12-27 02:15:42,295][105620] Updated weights for policy 1, policy_version 1494970 (0.0007) [2023-12-27 02:15:42,360][105620] Updated weights for policy 1, policy_version 1494980 (0.0009) [2023-12-27 02:15:42,416][105620] Updated weights for policy 1, policy_version 1494990 (0.0008) [2023-12-27 02:15:43,087][105692] Updated weights for policy 0, policy_version 1492496 (0.0009) [2023-12-27 02:15:43,115][105620] Updated weights for policy 1, policy_version 1495000 (0.0006) [2023-12-27 02:15:43,149][105692] Updated weights for policy 0, policy_version 1492506 (0.0008) [2023-12-27 02:15:43,174][105620] Updated weights for policy 1, policy_version 1495010 (0.0006) [2023-12-27 02:15:43,205][105692] Updated weights for policy 0, policy_version 1492516 (0.0009) [2023-12-27 02:15:43,226][105620] Updated weights for policy 1, policy_version 1495020 (0.0006) [2023-12-27 02:15:43,833][105620] Updated weights for policy 1, policy_version 1495030 (0.0008) [2023-12-27 02:15:43,880][105620] Updated weights for policy 1, policy_version 1495040 (0.0009) [2023-12-27 02:15:43,939][105620] Updated weights for policy 1, policy_version 1495051 (0.0009) [2023-12-27 02:15:43,969][105692] Updated weights for policy 0, policy_version 1492526 (0.0006) [2023-12-27 02:15:44,018][105692] Updated weights for policy 0, policy_version 1492536 (0.0009) [2023-12-27 02:15:44,065][105692] Updated weights for policy 0, policy_version 1492546 (0.0009) [2023-12-27 02:15:44,634][105620] Updated weights for policy 1, policy_version 1495061 (0.0006) [2023-12-27 02:15:44,698][105620] Updated weights for policy 1, policy_version 1495071 (0.0007) [2023-12-27 02:15:44,752][105620] Updated weights for policy 1, policy_version 1495081 (0.0009) [2023-12-27 02:15:44,819][105692] Updated weights for policy 0, policy_version 1492556 (0.0008) [2023-12-27 02:15:44,883][105692] Updated weights for policy 0, policy_version 1492566 (0.0008) [2023-12-27 02:15:44,946][105692] Updated weights for policy 0, policy_version 1492576 (0.0008) [2023-12-27 02:15:45,496][105620] Updated weights for policy 1, policy_version 1495091 (0.0009) [2023-12-27 02:15:45,555][105620] Updated weights for policy 1, policy_version 1495101 (0.0009) [2023-12-27 02:15:45,614][105620] Updated weights for policy 1, policy_version 1495111 (0.0009) [2023-12-27 02:15:45,690][105692] Updated weights for policy 0, policy_version 1492586 (0.0009) [2023-12-27 02:15:45,757][105692] Updated weights for policy 0, policy_version 1492596 (0.0009) [2023-12-27 02:15:45,820][105692] Updated weights for policy 0, policy_version 1492606 (0.0009) [2023-12-27 02:15:45,879][105692] Updated weights for policy 0, policy_version 1492616 (0.0009) [2023-12-27 02:15:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 764968960. Throughput: 0: 9659.0, 1: 9951.0. Samples: 764937480. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:15:46,063][104569] Avg episode reward: [(0, '7905.429'), (1, '9254.223')] [2023-12-27 02:15:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001492616_382164992.pth... [2023-12-27 02:15:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001495120_382803968.pth... [2023-12-27 02:15:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001491496_381878272.pth [2023-12-27 02:15:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001493936_382500864.pth [2023-12-27 02:15:46,379][105620] Updated weights for policy 1, policy_version 1495121 (0.0009) [2023-12-27 02:15:46,427][105620] Updated weights for policy 1, policy_version 1495131 (0.0007) [2023-12-27 02:15:46,491][105620] Updated weights for policy 1, policy_version 1495141 (0.0008) [2023-12-27 02:15:46,558][105620] Updated weights for policy 1, policy_version 1495151 (0.0009) [2023-12-27 02:15:46,637][105692] Updated weights for policy 0, policy_version 1492626 (0.0011) [2023-12-27 02:15:46,690][105692] Updated weights for policy 0, policy_version 1492636 (0.0010) [2023-12-27 02:15:46,746][105692] Updated weights for policy 0, policy_version 1492646 (0.0011) [2023-12-27 02:15:47,314][105620] Updated weights for policy 1, policy_version 1495161 (0.0006) [2023-12-27 02:15:47,381][105620] Updated weights for policy 1, policy_version 1495171 (0.0006) [2023-12-27 02:15:47,446][105620] Updated weights for policy 1, policy_version 1495181 (0.0007) [2023-12-27 02:15:47,474][105692] Updated weights for policy 0, policy_version 1492656 (0.0011) [2023-12-27 02:15:47,519][105692] Updated weights for policy 0, policy_version 1492666 (0.0010) [2023-12-27 02:15:47,567][105692] Updated weights for policy 0, policy_version 1492676 (0.0010) [2023-12-27 02:15:48,018][105620] Updated weights for policy 1, policy_version 1495191 (0.0009) [2023-12-27 02:15:48,071][105620] Updated weights for policy 1, policy_version 1495201 (0.0009) [2023-12-27 02:15:48,138][105620] Updated weights for policy 1, policy_version 1495211 (0.0008) [2023-12-27 02:15:48,300][105692] Updated weights for policy 0, policy_version 1492686 (0.0010) [2023-12-27 02:15:48,370][105692] Updated weights for policy 0, policy_version 1492696 (0.0008) [2023-12-27 02:15:48,432][105692] Updated weights for policy 0, policy_version 1492706 (0.0009) [2023-12-27 02:15:48,860][105620] Updated weights for policy 1, policy_version 1495221 (0.0011) [2023-12-27 02:15:48,927][105620] Updated weights for policy 1, policy_version 1495231 (0.0011) [2023-12-27 02:15:48,987][105620] Updated weights for policy 1, policy_version 1495241 (0.0011) [2023-12-27 02:15:49,078][105692] Updated weights for policy 0, policy_version 1492716 (0.0010) [2023-12-27 02:15:49,131][105692] Updated weights for policy 0, policy_version 1492726 (0.0007) [2023-12-27 02:15:49,189][105692] Updated weights for policy 0, policy_version 1492736 (0.0007) [2023-12-27 02:15:49,750][105620] Updated weights for policy 1, policy_version 1495251 (0.0010) [2023-12-27 02:15:49,812][105620] Updated weights for policy 1, policy_version 1495261 (0.0009) [2023-12-27 02:15:49,881][105620] Updated weights for policy 1, policy_version 1495271 (0.0008) [2023-12-27 02:15:49,958][105692] Updated weights for policy 0, policy_version 1492746 (0.0008) [2023-12-27 02:15:50,021][105692] Updated weights for policy 0, policy_version 1492756 (0.0009) [2023-12-27 02:15:50,075][105692] Updated weights for policy 0, policy_version 1492766 (0.0009) [2023-12-27 02:15:50,134][105692] Updated weights for policy 0, policy_version 1492776 (0.0009) [2023-12-27 02:15:50,640][105620] Updated weights for policy 1, policy_version 1495281 (0.0009) [2023-12-27 02:15:50,702][105620] Updated weights for policy 1, policy_version 1495291 (0.0009) [2023-12-27 02:15:50,765][105620] Updated weights for policy 1, policy_version 1495301 (0.0009) [2023-12-27 02:15:50,841][105620] Updated weights for policy 1, policy_version 1495311 (0.0009) [2023-12-27 02:15:50,913][105692] Updated weights for policy 0, policy_version 1492786 (0.0010) [2023-12-27 02:15:50,967][105692] Updated weights for policy 0, policy_version 1492796 (0.0010) [2023-12-27 02:15:51,023][105692] Updated weights for policy 0, policy_version 1492806 (0.0010) [2023-12-27 02:15:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 765067264. Throughput: 0: 9671.7, 1: 9877.7. Samples: 765052312. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:15:51,062][104569] Avg episode reward: [(0, '8264.667'), (1, '9254.851')] [2023-12-27 02:15:51,513][105620] Updated weights for policy 1, policy_version 1495321 (0.0006) [2023-12-27 02:15:51,583][105620] Updated weights for policy 1, policy_version 1495331 (0.0006) [2023-12-27 02:15:51,657][105620] Updated weights for policy 1, policy_version 1495341 (0.0008) [2023-12-27 02:15:51,931][105692] Updated weights for policy 0, policy_version 1492816 (0.0010) [2023-12-27 02:15:51,992][105692] Updated weights for policy 0, policy_version 1492826 (0.0010) [2023-12-27 02:15:52,046][105692] Updated weights for policy 0, policy_version 1492836 (0.0008) [2023-12-27 02:15:52,235][105620] Updated weights for policy 1, policy_version 1495351 (0.0008) [2023-12-27 02:15:52,289][105620] Updated weights for policy 1, policy_version 1495361 (0.0009) [2023-12-27 02:15:52,338][105620] Updated weights for policy 1, policy_version 1495371 (0.0009) [2023-12-27 02:15:52,836][105692] Updated weights for policy 0, policy_version 1492846 (0.0009) [2023-12-27 02:15:52,894][105692] Updated weights for policy 0, policy_version 1492856 (0.0009) [2023-12-27 02:15:52,944][105692] Updated weights for policy 0, policy_version 1492866 (0.0008) [2023-12-27 02:15:53,135][105620] Updated weights for policy 1, policy_version 1495381 (0.0008) [2023-12-27 02:15:53,185][105620] Updated weights for policy 1, policy_version 1495391 (0.0009) [2023-12-27 02:15:53,235][105620] Updated weights for policy 1, policy_version 1495401 (0.0009) [2023-12-27 02:15:53,690][105692] Updated weights for policy 0, policy_version 1492876 (0.0009) [2023-12-27 02:15:53,741][105692] Updated weights for policy 0, policy_version 1492886 (0.0009) [2023-12-27 02:15:53,788][105692] Updated weights for policy 0, policy_version 1492896 (0.0009) [2023-12-27 02:15:53,986][105620] Updated weights for policy 1, policy_version 1495411 (0.0008) [2023-12-27 02:15:54,035][105620] Updated weights for policy 1, policy_version 1495421 (0.0008) [2023-12-27 02:15:54,105][105620] Updated weights for policy 1, policy_version 1495431 (0.0009) [2023-12-27 02:15:54,433][105692] Updated weights for policy 0, policy_version 1492907 (0.0008) [2023-12-27 02:15:54,489][105692] Updated weights for policy 0, policy_version 1492917 (0.0006) [2023-12-27 02:15:54,550][105692] Updated weights for policy 0, policy_version 1492927 (0.0006) [2023-12-27 02:15:54,943][105620] Updated weights for policy 1, policy_version 1495441 (0.0010) [2023-12-27 02:15:55,005][105620] Updated weights for policy 1, policy_version 1495451 (0.0009) [2023-12-27 02:15:55,062][105620] Updated weights for policy 1, policy_version 1495461 (0.0010) [2023-12-27 02:15:55,117][105620] Updated weights for policy 1, policy_version 1495471 (0.0009) [2023-12-27 02:15:55,150][105692] Updated weights for policy 0, policy_version 1492937 (0.0006) [2023-12-27 02:15:55,212][105692] Updated weights for policy 0, policy_version 1492947 (0.0009) [2023-12-27 02:15:55,271][105692] Updated weights for policy 0, policy_version 1492957 (0.0009) [2023-12-27 02:15:55,329][105692] Updated weights for policy 0, policy_version 1492967 (0.0006) [2023-12-27 02:15:55,890][105620] Updated weights for policy 1, policy_version 1495481 (0.0009) [2023-12-27 02:15:55,948][105620] Updated weights for policy 1, policy_version 1495491 (0.0009) [2023-12-27 02:15:55,966][105692] Updated weights for policy 0, policy_version 1492977 (0.0007) [2023-12-27 02:15:56,001][105620] Updated weights for policy 1, policy_version 1495501 (0.0006) [2023-12-27 02:15:56,019][105692] Updated weights for policy 0, policy_version 1492987 (0.0008) [2023-12-27 02:15:56,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 765157376. Throughput: 0: 9611.1, 1: 9870.5. Samples: 765166212. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:15:56,062][104569] Avg episode reward: [(0, '8083.693'), (1, '9347.600')] [2023-12-27 02:15:56,086][105692] Updated weights for policy 0, policy_version 1492997 (0.0009) [2023-12-27 02:15:56,706][105620] Updated weights for policy 1, policy_version 1495511 (0.0009) [2023-12-27 02:15:56,754][105620] Updated weights for policy 1, policy_version 1495521 (0.0010) [2023-12-27 02:15:56,801][105620] Updated weights for policy 1, policy_version 1495531 (0.0010) [2023-12-27 02:15:56,807][105692] Updated weights for policy 0, policy_version 1493007 (0.0007) [2023-12-27 02:15:56,857][105692] Updated weights for policy 0, policy_version 1493017 (0.0008) [2023-12-27 02:15:56,912][105692] Updated weights for policy 0, policy_version 1493027 (0.0006) [2023-12-27 02:15:57,382][105620] Updated weights for policy 1, policy_version 1495541 (0.0009) [2023-12-27 02:15:57,446][105620] Updated weights for policy 1, policy_version 1495551 (0.0010) [2023-12-27 02:15:57,506][105620] Updated weights for policy 1, policy_version 1495561 (0.0010) [2023-12-27 02:15:57,520][105692] Updated weights for policy 0, policy_version 1493037 (0.0006) [2023-12-27 02:15:57,567][105692] Updated weights for policy 0, policy_version 1493047 (0.0008) [2023-12-27 02:15:57,627][105692] Updated weights for policy 0, policy_version 1493057 (0.0008) [2023-12-27 02:15:58,226][105620] Updated weights for policy 1, policy_version 1495571 (0.0009) [2023-12-27 02:15:58,291][105620] Updated weights for policy 1, policy_version 1495581 (0.0010) [2023-12-27 02:15:58,382][105620] Updated weights for policy 1, policy_version 1495591 (0.0011) [2023-12-27 02:15:58,408][105692] Updated weights for policy 0, policy_version 1493067 (0.0009) [2023-12-27 02:15:58,476][105692] Updated weights for policy 0, policy_version 1493077 (0.0010) [2023-12-27 02:15:58,537][105692] Updated weights for policy 0, policy_version 1493087 (0.0010) [2023-12-27 02:15:59,207][105620] Updated weights for policy 1, policy_version 1495601 (0.0007) [2023-12-27 02:15:59,280][105620] Updated weights for policy 1, policy_version 1495611 (0.0009) [2023-12-27 02:15:59,347][105620] Updated weights for policy 1, policy_version 1495621 (0.0009) [2023-12-27 02:15:59,356][105692] Updated weights for policy 0, policy_version 1493097 (0.0008) [2023-12-27 02:15:59,415][105620] Updated weights for policy 1, policy_version 1495631 (0.0009) [2023-12-27 02:15:59,422][105692] Updated weights for policy 0, policy_version 1493107 (0.0008) [2023-12-27 02:15:59,482][105692] Updated weights for policy 0, policy_version 1493117 (0.0006) [2023-12-27 02:15:59,537][105692] Updated weights for policy 0, policy_version 1493127 (0.0005) [2023-12-27 02:16:00,186][105620] Updated weights for policy 1, policy_version 1495641 (0.0009) [2023-12-27 02:16:00,204][105692] Updated weights for policy 0, policy_version 1493137 (0.0008) [2023-12-27 02:16:00,232][105620] Updated weights for policy 1, policy_version 1495651 (0.0006) [2023-12-27 02:16:00,261][105692] Updated weights for policy 0, policy_version 1493147 (0.0009) [2023-12-27 02:16:00,280][105620] Updated weights for policy 1, policy_version 1495661 (0.0007) [2023-12-27 02:16:00,323][105692] Updated weights for policy 0, policy_version 1493157 (0.0007) [2023-12-27 02:16:01,024][105692] Updated weights for policy 0, policy_version 1493167 (0.0008) [2023-12-27 02:16:01,055][105620] Updated weights for policy 1, policy_version 1495671 (0.0007) [2023-12-27 02:16:01,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 765247488. Throughput: 0: 9642.2, 1: 9871.2. Samples: 765225208. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:16:01,062][104569] Avg episode reward: [(0, '8442.211'), (1, '9256.068')] [2023-12-27 02:16:01,082][105692] Updated weights for policy 0, policy_version 1493177 (0.0007) [2023-12-27 02:16:01,117][105620] Updated weights for policy 1, policy_version 1495681 (0.0007) [2023-12-27 02:16:01,144][105692] Updated weights for policy 0, policy_version 1493188 (0.0007) [2023-12-27 02:16:01,171][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001493192_382312448.pth... [2023-12-27 02:16:01,176][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001492072_382025728.pth [2023-12-27 02:16:01,184][105620] Updated weights for policy 1, policy_version 1495691 (0.0007) [2023-12-27 02:16:01,212][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001495696_382951424.pth... [2023-12-27 02:16:01,215][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001494544_382656512.pth [2023-12-27 02:16:01,894][105620] Updated weights for policy 1, policy_version 1495701 (0.0008) [2023-12-27 02:16:01,959][105620] Updated weights for policy 1, policy_version 1495711 (0.0007) [2023-12-27 02:16:01,982][105692] Updated weights for policy 0, policy_version 1493198 (0.0007) [2023-12-27 02:16:02,029][105620] Updated weights for policy 1, policy_version 1495721 (0.0008) [2023-12-27 02:16:02,044][105692] Updated weights for policy 0, policy_version 1493208 (0.0007) [2023-12-27 02:16:02,108][105692] Updated weights for policy 0, policy_version 1493218 (0.0008) [2023-12-27 02:16:02,710][105620] Updated weights for policy 1, policy_version 1495731 (0.0008) [2023-12-27 02:16:02,774][105620] Updated weights for policy 1, policy_version 1495741 (0.0010) [2023-12-27 02:16:02,836][105620] Updated weights for policy 1, policy_version 1495751 (0.0008) [2023-12-27 02:16:02,896][105692] Updated weights for policy 0, policy_version 1493228 (0.0008) [2023-12-27 02:16:02,962][105692] Updated weights for policy 0, policy_version 1493238 (0.0009) [2023-12-27 02:16:03,029][105692] Updated weights for policy 0, policy_version 1493248 (0.0010) [2023-12-27 02:16:03,443][105620] Updated weights for policy 1, policy_version 1495761 (0.0009) [2023-12-27 02:16:03,498][105620] Updated weights for policy 1, policy_version 1495771 (0.0007) [2023-12-27 02:16:03,555][105620] Updated weights for policy 1, policy_version 1495781 (0.0009) [2023-12-27 02:16:03,608][105620] Updated weights for policy 1, policy_version 1495791 (0.0007) [2023-12-27 02:16:03,864][105692] Updated weights for policy 0, policy_version 1493258 (0.0009) [2023-12-27 02:16:03,927][105692] Updated weights for policy 0, policy_version 1493268 (0.0010) [2023-12-27 02:16:03,987][105692] Updated weights for policy 0, policy_version 1493278 (0.0010) [2023-12-27 02:16:04,051][105692] Updated weights for policy 0, policy_version 1493288 (0.0009) [2023-12-27 02:16:04,305][105620] Updated weights for policy 1, policy_version 1495801 (0.0008) [2023-12-27 02:16:04,386][105620] Updated weights for policy 1, policy_version 1495811 (0.0009) [2023-12-27 02:16:04,449][105620] Updated weights for policy 1, policy_version 1495821 (0.0007) [2023-12-27 02:16:04,860][105692] Updated weights for policy 0, policy_version 1493298 (0.0010) [2023-12-27 02:16:04,928][105692] Updated weights for policy 0, policy_version 1493308 (0.0009) [2023-12-27 02:16:04,988][105692] Updated weights for policy 0, policy_version 1493318 (0.0009) [2023-12-27 02:16:05,149][105620] Updated weights for policy 1, policy_version 1495831 (0.0007) [2023-12-27 02:16:05,216][105620] Updated weights for policy 1, policy_version 1495841 (0.0010) [2023-12-27 02:16:05,275][105620] Updated weights for policy 1, policy_version 1495851 (0.0011) [2023-12-27 02:16:05,694][105692] Updated weights for policy 0, policy_version 1493328 (0.0008) [2023-12-27 02:16:05,756][105692] Updated weights for policy 0, policy_version 1493338 (0.0007) [2023-12-27 02:16:05,811][105692] Updated weights for policy 0, policy_version 1493348 (0.0008) [2023-12-27 02:16:05,969][105620] Updated weights for policy 1, policy_version 1495861 (0.0008) [2023-12-27 02:16:06,033][105620] Updated weights for policy 1, policy_version 1495871 (0.0005) [2023-12-27 02:16:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.3, 300 sec: 19577.5). Total num frames: 765345792. Throughput: 0: 9529.8, 1: 9763.0. Samples: 765336340. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:16:06,062][104569] Avg episode reward: [(0, '8353.570'), (1, '9256.666')] [2023-12-27 02:16:06,089][105620] Updated weights for policy 1, policy_version 1495881 (0.0008) [2023-12-27 02:16:06,565][105692] Updated weights for policy 0, policy_version 1493358 (0.0008) [2023-12-27 02:16:06,629][105692] Updated weights for policy 0, policy_version 1493368 (0.0007) [2023-12-27 02:16:06,686][105692] Updated weights for policy 0, policy_version 1493378 (0.0008) [2023-12-27 02:16:06,785][105620] Updated weights for policy 1, policy_version 1495891 (0.0009) [2023-12-27 02:16:06,831][105620] Updated weights for policy 1, policy_version 1495901 (0.0005) [2023-12-27 02:16:06,886][105620] Updated weights for policy 1, policy_version 1495911 (0.0005) [2023-12-27 02:16:07,416][105620] Updated weights for policy 1, policy_version 1495921 (0.0005) [2023-12-27 02:16:07,471][105692] Updated weights for policy 0, policy_version 1493388 (0.0009) [2023-12-27 02:16:07,486][105620] Updated weights for policy 1, policy_version 1495931 (0.0005) [2023-12-27 02:16:07,521][105692] Updated weights for policy 0, policy_version 1493398 (0.0009) [2023-12-27 02:16:07,552][105620] Updated weights for policy 1, policy_version 1495941 (0.0005) [2023-12-27 02:16:07,576][105692] Updated weights for policy 0, policy_version 1493409 (0.0007) [2023-12-27 02:16:07,618][105620] Updated weights for policy 1, policy_version 1495951 (0.0007) [2023-12-27 02:16:08,214][105620] Updated weights for policy 1, policy_version 1495961 (0.0011) [2023-12-27 02:16:08,262][105620] Updated weights for policy 1, policy_version 1495971 (0.0010) [2023-12-27 02:16:08,314][105620] Updated weights for policy 1, policy_version 1495981 (0.0010) [2023-12-27 02:16:08,364][105692] Updated weights for policy 0, policy_version 1493419 (0.0006) [2023-12-27 02:16:08,428][105692] Updated weights for policy 0, policy_version 1493429 (0.0008) [2023-12-27 02:16:08,485][105692] Updated weights for policy 0, policy_version 1493439 (0.0006) [2023-12-27 02:16:09,081][105692] Updated weights for policy 0, policy_version 1493449 (0.0005) [2023-12-27 02:16:09,106][105620] Updated weights for policy 1, policy_version 1495991 (0.0010) [2023-12-27 02:16:09,142][105692] Updated weights for policy 0, policy_version 1493459 (0.0005) [2023-12-27 02:16:09,169][105620] Updated weights for policy 1, policy_version 1496001 (0.0010) [2023-12-27 02:16:09,195][105692] Updated weights for policy 0, policy_version 1493469 (0.0005) [2023-12-27 02:16:09,236][105620] Updated weights for policy 1, policy_version 1496011 (0.0009) [2023-12-27 02:16:09,259][105692] Updated weights for policy 0, policy_version 1493479 (0.0006) [2023-12-27 02:16:09,904][105692] Updated weights for policy 0, policy_version 1493489 (0.0009) [2023-12-27 02:16:09,970][105692] Updated weights for policy 0, policy_version 1493499 (0.0010) [2023-12-27 02:16:10,013][105620] Updated weights for policy 1, policy_version 1496021 (0.0010) [2023-12-27 02:16:10,038][105692] Updated weights for policy 0, policy_version 1493509 (0.0006) [2023-12-27 02:16:10,077][105620] Updated weights for policy 1, policy_version 1496031 (0.0011) [2023-12-27 02:16:10,144][105620] Updated weights for policy 1, policy_version 1496041 (0.0011) [2023-12-27 02:16:10,672][105692] Updated weights for policy 0, policy_version 1493519 (0.0005) [2023-12-27 02:16:10,730][105692] Updated weights for policy 0, policy_version 1493529 (0.0005) [2023-12-27 02:16:10,780][105692] Updated weights for policy 0, policy_version 1493539 (0.0010) [2023-12-27 02:16:10,893][105620] Updated weights for policy 1, policy_version 1496051 (0.0010) [2023-12-27 02:16:10,951][105620] Updated weights for policy 1, policy_version 1496061 (0.0009) [2023-12-27 02:16:11,007][105620] Updated weights for policy 1, policy_version 1496071 (0.0011) [2023-12-27 02:16:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 765444096. Throughput: 0: 9569.7, 1: 9874.1. Samples: 765455028. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:16:11,062][104569] Avg episode reward: [(0, '8079.589'), (1, '9350.010')] [2023-12-27 02:16:11,473][105692] Updated weights for policy 0, policy_version 1493549 (0.0010) [2023-12-27 02:16:11,526][105692] Updated weights for policy 0, policy_version 1493559 (0.0011) [2023-12-27 02:16:11,575][105692] Updated weights for policy 0, policy_version 1493569 (0.0010) [2023-12-27 02:16:11,852][105620] Updated weights for policy 1, policy_version 1496081 (0.0010) [2023-12-27 02:16:11,915][105620] Updated weights for policy 1, policy_version 1496091 (0.0009) [2023-12-27 02:16:11,969][105620] Updated weights for policy 1, policy_version 1496101 (0.0011) [2023-12-27 02:16:12,029][105620] Updated weights for policy 1, policy_version 1496111 (0.0011) [2023-12-27 02:16:12,383][105692] Updated weights for policy 0, policy_version 1493579 (0.0011) [2023-12-27 02:16:12,447][105692] Updated weights for policy 0, policy_version 1493589 (0.0010) [2023-12-27 02:16:12,511][105692] Updated weights for policy 0, policy_version 1493599 (0.0006) [2023-12-27 02:16:12,849][105620] Updated weights for policy 1, policy_version 1496121 (0.0009) [2023-12-27 02:16:12,902][105620] Updated weights for policy 1, policy_version 1496131 (0.0008) [2023-12-27 02:16:12,958][105620] Updated weights for policy 1, policy_version 1496141 (0.0008) [2023-12-27 02:16:13,199][105692] Updated weights for policy 0, policy_version 1493609 (0.0009) [2023-12-27 02:16:13,256][105692] Updated weights for policy 0, policy_version 1493619 (0.0005) [2023-12-27 02:16:13,318][105692] Updated weights for policy 0, policy_version 1493629 (0.0008) [2023-12-27 02:16:13,366][105692] Updated weights for policy 0, policy_version 1493639 (0.0010) [2023-12-27 02:16:13,756][105620] Updated weights for policy 1, policy_version 1496151 (0.0007) [2023-12-27 02:16:13,807][105620] Updated weights for policy 1, policy_version 1496161 (0.0007) [2023-12-27 02:16:13,863][105620] Updated weights for policy 1, policy_version 1496171 (0.0008) [2023-12-27 02:16:14,080][105692] Updated weights for policy 0, policy_version 1493649 (0.0010) [2023-12-27 02:16:14,138][105692] Updated weights for policy 0, policy_version 1493659 (0.0010) [2023-12-27 02:16:14,186][105692] Updated weights for policy 0, policy_version 1493669 (0.0010) [2023-12-27 02:16:14,659][105620] Updated weights for policy 1, policy_version 1496181 (0.0009) [2023-12-27 02:16:14,710][105620] Updated weights for policy 1, policy_version 1496191 (0.0008) [2023-12-27 02:16:14,769][105620] Updated weights for policy 1, policy_version 1496201 (0.0008) [2023-12-27 02:16:14,893][105692] Updated weights for policy 0, policy_version 1493679 (0.0011) [2023-12-27 02:16:14,959][105692] Updated weights for policy 0, policy_version 1493689 (0.0011) [2023-12-27 02:16:15,020][105692] Updated weights for policy 0, policy_version 1493699 (0.0009) [2023-12-27 02:16:15,566][105620] Updated weights for policy 1, policy_version 1496211 (0.0008) [2023-12-27 02:16:15,618][105620] Updated weights for policy 1, policy_version 1496221 (0.0008) [2023-12-27 02:16:15,673][105620] Updated weights for policy 1, policy_version 1496231 (0.0008) [2023-12-27 02:16:15,746][105692] Updated weights for policy 0, policy_version 1493709 (0.0011) [2023-12-27 02:16:15,798][105692] Updated weights for policy 0, policy_version 1493719 (0.0010) [2023-12-27 02:16:15,849][105692] Updated weights for policy 0, policy_version 1493729 (0.0010) [2023-12-27 02:16:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 765542400. Throughput: 0: 9550.0, 1: 9778.4. Samples: 765510224. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:16:16,063][104569] Avg episode reward: [(0, '8349.164'), (1, '9258.662')] [2023-12-27 02:16:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001493736_382451712.pth... [2023-12-27 02:16:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001496240_383090688.pth... [2023-12-27 02:16:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001495120_382803968.pth [2023-12-27 02:16:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001492616_382164992.pth [2023-12-27 02:16:16,287][105620] Updated weights for policy 1, policy_version 1496241 (0.0005) [2023-12-27 02:16:16,354][105620] Updated weights for policy 1, policy_version 1496251 (0.0007) [2023-12-27 02:16:16,417][105620] Updated weights for policy 1, policy_version 1496261 (0.0008) [2023-12-27 02:16:16,474][105620] Updated weights for policy 1, policy_version 1496271 (0.0008) [2023-12-27 02:16:16,583][105692] Updated weights for policy 0, policy_version 1493739 (0.0008) [2023-12-27 02:16:16,640][105692] Updated weights for policy 0, policy_version 1493749 (0.0011) [2023-12-27 02:16:16,692][105692] Updated weights for policy 0, policy_version 1493759 (0.0010) [2023-12-27 02:16:17,113][105620] Updated weights for policy 1, policy_version 1496281 (0.0010) [2023-12-27 02:16:17,167][105620] Updated weights for policy 1, policy_version 1496291 (0.0010) [2023-12-27 02:16:17,225][105620] Updated weights for policy 1, policy_version 1496301 (0.0010) [2023-12-27 02:16:17,398][105692] Updated weights for policy 0, policy_version 1493769 (0.0011) [2023-12-27 02:16:17,449][105692] Updated weights for policy 0, policy_version 1493779 (0.0010) [2023-12-27 02:16:17,494][105692] Updated weights for policy 0, policy_version 1493789 (0.0010) [2023-12-27 02:16:17,548][105692] Updated weights for policy 0, policy_version 1493799 (0.0010) [2023-12-27 02:16:17,970][105620] Updated weights for policy 1, policy_version 1496311 (0.0009) [2023-12-27 02:16:18,025][105620] Updated weights for policy 1, policy_version 1496321 (0.0010) [2023-12-27 02:16:18,079][105620] Updated weights for policy 1, policy_version 1496331 (0.0010) [2023-12-27 02:16:18,249][105692] Updated weights for policy 0, policy_version 1493809 (0.0010) [2023-12-27 02:16:18,303][105692] Updated weights for policy 0, policy_version 1493819 (0.0010) [2023-12-27 02:16:18,371][105692] Updated weights for policy 0, policy_version 1493829 (0.0011) [2023-12-27 02:16:18,831][105620] Updated weights for policy 1, policy_version 1496341 (0.0010) [2023-12-27 02:16:18,895][105620] Updated weights for policy 1, policy_version 1496351 (0.0010) [2023-12-27 02:16:18,955][105620] Updated weights for policy 1, policy_version 1496361 (0.0010) [2023-12-27 02:16:19,102][105692] Updated weights for policy 0, policy_version 1493839 (0.0010) [2023-12-27 02:16:19,167][105692] Updated weights for policy 0, policy_version 1493849 (0.0010) [2023-12-27 02:16:19,229][105692] Updated weights for policy 0, policy_version 1493859 (0.0011) [2023-12-27 02:16:19,714][105620] Updated weights for policy 1, policy_version 1496371 (0.0010) [2023-12-27 02:16:19,783][105620] Updated weights for policy 1, policy_version 1496381 (0.0008) [2023-12-27 02:16:19,856][105620] Updated weights for policy 1, policy_version 1496391 (0.0011) [2023-12-27 02:16:20,007][105692] Updated weights for policy 0, policy_version 1493869 (0.0008) [2023-12-27 02:16:20,071][105692] Updated weights for policy 0, policy_version 1493879 (0.0007) [2023-12-27 02:16:20,129][105692] Updated weights for policy 0, policy_version 1493889 (0.0008) [2023-12-27 02:16:20,574][105620] Updated weights for policy 1, policy_version 1496401 (0.0010) [2023-12-27 02:16:20,632][105620] Updated weights for policy 1, policy_version 1496411 (0.0010) [2023-12-27 02:16:20,694][105620] Updated weights for policy 1, policy_version 1496421 (0.0006) [2023-12-27 02:16:20,761][105620] Updated weights for policy 1, policy_version 1496431 (0.0009) [2023-12-27 02:16:20,889][105692] Updated weights for policy 0, policy_version 1493899 (0.0008) [2023-12-27 02:16:20,952][105692] Updated weights for policy 0, policy_version 1493909 (0.0009) [2023-12-27 02:16:21,001][105692] Updated weights for policy 0, policy_version 1493919 (0.0009) [2023-12-27 02:16:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 765640704. Throughput: 0: 9455.8, 1: 9678.5. Samples: 765625780. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:16:21,062][104569] Avg episode reward: [(0, '8715.518'), (1, '9167.041')] [2023-12-27 02:16:21,451][105620] Updated weights for policy 1, policy_version 1496441 (0.0009) [2023-12-27 02:16:21,509][105620] Updated weights for policy 1, policy_version 1496451 (0.0008) [2023-12-27 02:16:21,559][105620] Updated weights for policy 1, policy_version 1496461 (0.0009) [2023-12-27 02:16:21,826][105692] Updated weights for policy 0, policy_version 1493929 (0.0009) [2023-12-27 02:16:21,886][105692] Updated weights for policy 0, policy_version 1493939 (0.0007) [2023-12-27 02:16:21,946][105692] Updated weights for policy 0, policy_version 1493949 (0.0008) [2023-12-27 02:16:22,001][105692] Updated weights for policy 0, policy_version 1493959 (0.0008) [2023-12-27 02:16:22,400][105620] Updated weights for policy 1, policy_version 1496471 (0.0008) [2023-12-27 02:16:22,471][105620] Updated weights for policy 1, policy_version 1496481 (0.0008) [2023-12-27 02:16:22,526][105620] Updated weights for policy 1, policy_version 1496491 (0.0008) [2023-12-27 02:16:22,733][105692] Updated weights for policy 0, policy_version 1493969 (0.0010) [2023-12-27 02:16:22,792][105692] Updated weights for policy 0, policy_version 1493979 (0.0010) [2023-12-27 02:16:22,861][105692] Updated weights for policy 0, policy_version 1493989 (0.0010) [2023-12-27 02:16:23,296][105620] Updated weights for policy 1, policy_version 1496501 (0.0008) [2023-12-27 02:16:23,342][105620] Updated weights for policy 1, policy_version 1496511 (0.0008) [2023-12-27 02:16:23,397][105620] Updated weights for policy 1, policy_version 1496521 (0.0008) [2023-12-27 02:16:23,592][105692] Updated weights for policy 0, policy_version 1493999 (0.0010) [2023-12-27 02:16:23,647][105692] Updated weights for policy 0, policy_version 1494009 (0.0008) [2023-12-27 02:16:23,695][105692] Updated weights for policy 0, policy_version 1494019 (0.0010) [2023-12-27 02:16:24,138][105620] Updated weights for policy 1, policy_version 1496531 (0.0008) [2023-12-27 02:16:24,196][105620] Updated weights for policy 1, policy_version 1496541 (0.0009) [2023-12-27 02:16:24,256][105620] Updated weights for policy 1, policy_version 1496551 (0.0008) [2023-12-27 02:16:24,449][105692] Updated weights for policy 0, policy_version 1494029 (0.0010) [2023-12-27 02:16:24,503][105692] Updated weights for policy 0, policy_version 1494039 (0.0008) [2023-12-27 02:16:24,550][105692] Updated weights for policy 0, policy_version 1494049 (0.0009) [2023-12-27 02:16:25,025][105620] Updated weights for policy 1, policy_version 1496561 (0.0009) [2023-12-27 02:16:25,082][105620] Updated weights for policy 1, policy_version 1496571 (0.0009) [2023-12-27 02:16:25,147][105620] Updated weights for policy 1, policy_version 1496581 (0.0008) [2023-12-27 02:16:25,198][105620] Updated weights for policy 1, policy_version 1496591 (0.0009) [2023-12-27 02:16:25,239][105692] Updated weights for policy 0, policy_version 1494059 (0.0009) [2023-12-27 02:16:25,289][105692] Updated weights for policy 0, policy_version 1494069 (0.0009) [2023-12-27 02:16:25,335][105692] Updated weights for policy 0, policy_version 1494079 (0.0008) [2023-12-27 02:16:25,837][105620] Updated weights for policy 1, policy_version 1496601 (0.0006) [2023-12-27 02:16:25,898][105620] Updated weights for policy 1, policy_version 1496611 (0.0005) [2023-12-27 02:16:25,958][105620] Updated weights for policy 1, policy_version 1496621 (0.0010) [2023-12-27 02:16:26,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.8, 300 sec: 19521.9). Total num frames: 765730816. Throughput: 0: 9457.2, 1: 9630.7. Samples: 765738364. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:16:26,063][104569] Avg episode reward: [(0, '8899.124'), (1, '9075.472')] [2023-12-27 02:16:26,218][105692] Updated weights for policy 0, policy_version 1494089 (0.0009) [2023-12-27 02:16:26,274][105692] Updated weights for policy 0, policy_version 1494099 (0.0009) [2023-12-27 02:16:26,334][105692] Updated weights for policy 0, policy_version 1494109 (0.0009) [2023-12-27 02:16:26,387][105692] Updated weights for policy 0, policy_version 1494119 (0.0009) [2023-12-27 02:16:26,525][105620] Updated weights for policy 1, policy_version 1496631 (0.0007) [2023-12-27 02:16:26,586][105620] Updated weights for policy 1, policy_version 1496641 (0.0010) [2023-12-27 02:16:26,638][105620] Updated weights for policy 1, policy_version 1496651 (0.0011) [2023-12-27 02:16:27,080][105692] Updated weights for policy 0, policy_version 1494129 (0.0009) [2023-12-27 02:16:27,128][105692] Updated weights for policy 0, policy_version 1494139 (0.0010) [2023-12-27 02:16:27,176][105692] Updated weights for policy 0, policy_version 1494149 (0.0007) [2023-12-27 02:16:27,244][105620] Updated weights for policy 1, policy_version 1496661 (0.0008) [2023-12-27 02:16:27,303][105620] Updated weights for policy 1, policy_version 1496671 (0.0005) [2023-12-27 02:16:27,364][105620] Updated weights for policy 1, policy_version 1496681 (0.0010) [2023-12-27 02:16:27,843][105692] Updated weights for policy 0, policy_version 1494159 (0.0005) [2023-12-27 02:16:27,894][105692] Updated weights for policy 0, policy_version 1494169 (0.0005) [2023-12-27 02:16:27,947][105620] Updated weights for policy 1, policy_version 1496691 (0.0009) [2023-12-27 02:16:27,951][105692] Updated weights for policy 0, policy_version 1494179 (0.0008) [2023-12-27 02:16:27,992][105620] Updated weights for policy 1, policy_version 1496701 (0.0007) [2023-12-27 02:16:28,053][105620] Updated weights for policy 1, policy_version 1496711 (0.0006) [2023-12-27 02:16:28,597][105692] Updated weights for policy 0, policy_version 1494189 (0.0010) [2023-12-27 02:16:28,653][105692] Updated weights for policy 0, policy_version 1494199 (0.0010) [2023-12-27 02:16:28,683][105620] Updated weights for policy 1, policy_version 1496721 (0.0008) [2023-12-27 02:16:28,708][105692] Updated weights for policy 0, policy_version 1494209 (0.0010) [2023-12-27 02:16:28,731][105620] Updated weights for policy 1, policy_version 1496731 (0.0010) [2023-12-27 02:16:28,789][105620] Updated weights for policy 1, policy_version 1496741 (0.0010) [2023-12-27 02:16:28,840][105620] Updated weights for policy 1, policy_version 1496751 (0.0010) [2023-12-27 02:16:29,399][105692] Updated weights for policy 0, policy_version 1494219 (0.0009) [2023-12-27 02:16:29,451][105692] Updated weights for policy 0, policy_version 1494229 (0.0006) [2023-12-27 02:16:29,496][105692] Updated weights for policy 0, policy_version 1494239 (0.0007) [2023-12-27 02:16:29,548][105620] Updated weights for policy 1, policy_version 1496761 (0.0010) [2023-12-27 02:16:29,613][105620] Updated weights for policy 1, policy_version 1496771 (0.0011) [2023-12-27 02:16:29,667][105620] Updated weights for policy 1, policy_version 1496781 (0.0010) [2023-12-27 02:16:30,159][105692] Updated weights for policy 0, policy_version 1494249 (0.0006) [2023-12-27 02:16:30,207][105692] Updated weights for policy 0, policy_version 1494259 (0.0007) [2023-12-27 02:16:30,255][105692] Updated weights for policy 0, policy_version 1494269 (0.0008) [2023-12-27 02:16:30,311][105692] Updated weights for policy 0, policy_version 1494279 (0.0008) [2023-12-27 02:16:30,405][105620] Updated weights for policy 1, policy_version 1496791 (0.0010) [2023-12-27 02:16:30,453][105620] Updated weights for policy 1, policy_version 1496801 (0.0010) [2023-12-27 02:16:30,519][105620] Updated weights for policy 1, policy_version 1496811 (0.0010) [2023-12-27 02:16:30,957][105692] Updated weights for policy 0, policy_version 1494289 (0.0008) [2023-12-27 02:16:31,002][105692] Updated weights for policy 0, policy_version 1494299 (0.0005) [2023-12-27 02:16:31,060][105692] Updated weights for policy 0, policy_version 1494309 (0.0006) [2023-12-27 02:16:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 765829120. Throughput: 0: 9533.3, 1: 9674.2. Samples: 765801816. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:16:31,062][104569] Avg episode reward: [(0, '8623.924'), (1, '9168.022')] [2023-12-27 02:16:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001496816_383238144.pth... [2023-12-27 02:16:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001495696_382951424.pth [2023-12-27 02:16:31,076][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001494312_382599168.pth... [2023-12-27 02:16:31,098][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001493192_382312448.pth [2023-12-27 02:16:31,274][105620] Updated weights for policy 1, policy_version 1496821 (0.0011) [2023-12-27 02:16:31,337][105620] Updated weights for policy 1, policy_version 1496831 (0.0011) [2023-12-27 02:16:31,400][105620] Updated weights for policy 1, policy_version 1496841 (0.0011) [2023-12-27 02:16:31,808][105692] Updated weights for policy 0, policy_version 1494319 (0.0009) [2023-12-27 02:16:31,856][105692] Updated weights for policy 0, policy_version 1494329 (0.0009) [2023-12-27 02:16:31,909][105692] Updated weights for policy 0, policy_version 1494339 (0.0008) [2023-12-27 02:16:32,099][105620] Updated weights for policy 1, policy_version 1496851 (0.0009) [2023-12-27 02:16:32,158][105620] Updated weights for policy 1, policy_version 1496861 (0.0008) [2023-12-27 02:16:32,211][105620] Updated weights for policy 1, policy_version 1496871 (0.0009) [2023-12-27 02:16:32,715][105692] Updated weights for policy 0, policy_version 1494349 (0.0009) [2023-12-27 02:16:32,774][105692] Updated weights for policy 0, policy_version 1494359 (0.0008) [2023-12-27 02:16:32,830][105692] Updated weights for policy 0, policy_version 1494369 (0.0006) [2023-12-27 02:16:32,940][105620] Updated weights for policy 1, policy_version 1496881 (0.0008) [2023-12-27 02:16:32,992][105620] Updated weights for policy 1, policy_version 1496891 (0.0009) [2023-12-27 02:16:33,043][105620] Updated weights for policy 1, policy_version 1496901 (0.0010) [2023-12-27 02:16:33,091][105620] Updated weights for policy 1, policy_version 1496912 (0.0008) [2023-12-27 02:16:33,471][105692] Updated weights for policy 0, policy_version 1494379 (0.0006) [2023-12-27 02:16:33,523][105692] Updated weights for policy 0, policy_version 1494389 (0.0006) [2023-12-27 02:16:33,571][105692] Updated weights for policy 0, policy_version 1494399 (0.0009) [2023-12-27 02:16:33,822][105620] Updated weights for policy 1, policy_version 1496922 (0.0005) [2023-12-27 02:16:33,866][105620] Updated weights for policy 1, policy_version 1496932 (0.0005) [2023-12-27 02:16:33,911][105620] Updated weights for policy 1, policy_version 1496942 (0.0005) [2023-12-27 02:16:34,110][105692] Updated weights for policy 0, policy_version 1494409 (0.0009) [2023-12-27 02:16:34,175][105692] Updated weights for policy 0, policy_version 1494419 (0.0007) [2023-12-27 02:16:34,222][105692] Updated weights for policy 0, policy_version 1494429 (0.0008) [2023-12-27 02:16:34,274][105692] Updated weights for policy 0, policy_version 1494439 (0.0008) [2023-12-27 02:16:34,587][105620] Updated weights for policy 1, policy_version 1496952 (0.0010) [2023-12-27 02:16:34,647][105620] Updated weights for policy 1, policy_version 1496962 (0.0011) [2023-12-27 02:16:34,716][105620] Updated weights for policy 1, policy_version 1496972 (0.0010) [2023-12-27 02:16:35,013][105692] Updated weights for policy 0, policy_version 1494449 (0.0007) [2023-12-27 02:16:35,078][105692] Updated weights for policy 0, policy_version 1494459 (0.0006) [2023-12-27 02:16:35,148][105692] Updated weights for policy 0, policy_version 1494469 (0.0006) [2023-12-27 02:16:35,408][105620] Updated weights for policy 1, policy_version 1496982 (0.0009) [2023-12-27 02:16:35,457][105620] Updated weights for policy 1, policy_version 1496992 (0.0010) [2023-12-27 02:16:35,506][105620] Updated weights for policy 1, policy_version 1497002 (0.0009) [2023-12-27 02:16:35,787][105692] Updated weights for policy 0, policy_version 1494479 (0.0006) [2023-12-27 02:16:35,846][105692] Updated weights for policy 0, policy_version 1494489 (0.0005) [2023-12-27 02:16:35,913][105692] Updated weights for policy 0, policy_version 1494499 (0.0009) [2023-12-27 02:16:36,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 765935616. Throughput: 0: 9633.2, 1: 9691.5. Samples: 765921928. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:16:36,063][104569] Avg episode reward: [(0, '8167.460'), (1, '9167.611')] [2023-12-27 02:16:36,193][105620] Updated weights for policy 1, policy_version 1497012 (0.0007) [2023-12-27 02:16:36,242][105620] Updated weights for policy 1, policy_version 1497022 (0.0008) [2023-12-27 02:16:36,301][105620] Updated weights for policy 1, policy_version 1497032 (0.0009) [2023-12-27 02:16:36,579][105692] Updated weights for policy 0, policy_version 1494509 (0.0008) [2023-12-27 02:16:36,642][105692] Updated weights for policy 0, policy_version 1494519 (0.0006) [2023-12-27 02:16:36,700][105692] Updated weights for policy 0, policy_version 1494529 (0.0007) [2023-12-27 02:16:37,063][105620] Updated weights for policy 1, policy_version 1497042 (0.0008) [2023-12-27 02:16:37,126][105620] Updated weights for policy 1, policy_version 1497052 (0.0006) [2023-12-27 02:16:37,192][105620] Updated weights for policy 1, policy_version 1497062 (0.0005) [2023-12-27 02:16:37,256][105620] Updated weights for policy 1, policy_version 1497072 (0.0006) [2023-12-27 02:16:37,490][105692] Updated weights for policy 0, policy_version 1494539 (0.0009) [2023-12-27 02:16:37,540][105692] Updated weights for policy 0, policy_version 1494549 (0.0007) [2023-12-27 02:16:37,601][105692] Updated weights for policy 0, policy_version 1494559 (0.0007) [2023-12-27 02:16:37,849][105620] Updated weights for policy 1, policy_version 1497082 (0.0010) [2023-12-27 02:16:37,908][105620] Updated weights for policy 1, policy_version 1497092 (0.0010) [2023-12-27 02:16:37,956][105620] Updated weights for policy 1, policy_version 1497102 (0.0008) [2023-12-27 02:16:38,297][105692] Updated weights for policy 0, policy_version 1494569 (0.0006) [2023-12-27 02:16:38,365][105692] Updated weights for policy 0, policy_version 1494579 (0.0009) [2023-12-27 02:16:38,428][105692] Updated weights for policy 0, policy_version 1494589 (0.0008) [2023-12-27 02:16:38,488][105692] Updated weights for policy 0, policy_version 1494599 (0.0008) [2023-12-27 02:16:38,616][105620] Updated weights for policy 1, policy_version 1497112 (0.0009) [2023-12-27 02:16:38,668][105620] Updated weights for policy 1, policy_version 1497122 (0.0010) [2023-12-27 02:16:38,728][105620] Updated weights for policy 1, policy_version 1497132 (0.0011) [2023-12-27 02:16:39,199][105692] Updated weights for policy 0, policy_version 1494609 (0.0006) [2023-12-27 02:16:39,265][105692] Updated weights for policy 0, policy_version 1494619 (0.0009) [2023-12-27 02:16:39,333][105692] Updated weights for policy 0, policy_version 1494629 (0.0010) [2023-12-27 02:16:39,486][105620] Updated weights for policy 1, policy_version 1497142 (0.0010) [2023-12-27 02:16:39,551][105620] Updated weights for policy 1, policy_version 1497152 (0.0009) [2023-12-27 02:16:39,603][105620] Updated weights for policy 1, policy_version 1497162 (0.0009) [2023-12-27 02:16:40,084][105692] Updated weights for policy 0, policy_version 1494639 (0.0007) [2023-12-27 02:16:40,153][105692] Updated weights for policy 0, policy_version 1494649 (0.0006) [2023-12-27 02:16:40,222][105692] Updated weights for policy 0, policy_version 1494659 (0.0008) [2023-12-27 02:16:40,415][105620] Updated weights for policy 1, policy_version 1497172 (0.0009) [2023-12-27 02:16:40,482][105620] Updated weights for policy 1, policy_version 1497182 (0.0011) [2023-12-27 02:16:40,552][105620] Updated weights for policy 1, policy_version 1497192 (0.0010) [2023-12-27 02:16:40,782][105692] Updated weights for policy 0, policy_version 1494669 (0.0007) [2023-12-27 02:16:40,839][105692] Updated weights for policy 0, policy_version 1494679 (0.0005) [2023-12-27 02:16:40,901][105692] Updated weights for policy 0, policy_version 1494689 (0.0005) [2023-12-27 02:16:41,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 766033920. Throughput: 0: 9672.2, 1: 9752.6. Samples: 766040328. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 02:16:41,063][104569] Avg episode reward: [(0, '7814.293'), (1, '9169.992')] [2023-12-27 02:16:41,259][105620] Updated weights for policy 1, policy_version 1497202 (0.0010) [2023-12-27 02:16:41,325][105620] Updated weights for policy 1, policy_version 1497212 (0.0008) [2023-12-27 02:16:41,403][105620] Updated weights for policy 1, policy_version 1497222 (0.0009) [2023-12-27 02:16:41,469][105620] Updated weights for policy 1, policy_version 1497232 (0.0011) [2023-12-27 02:16:41,612][105692] Updated weights for policy 0, policy_version 1494699 (0.0006) [2023-12-27 02:16:41,677][105692] Updated weights for policy 0, policy_version 1494709 (0.0009) [2023-12-27 02:16:41,746][105692] Updated weights for policy 0, policy_version 1494719 (0.0009) [2023-12-27 02:16:42,122][105620] Updated weights for policy 1, policy_version 1497242 (0.0011) [2023-12-27 02:16:42,193][105620] Updated weights for policy 1, policy_version 1497252 (0.0011) [2023-12-27 02:16:42,259][105620] Updated weights for policy 1, policy_version 1497262 (0.0011) [2023-12-27 02:16:42,516][105692] Updated weights for policy 0, policy_version 1494729 (0.0008) [2023-12-27 02:16:42,570][105692] Updated weights for policy 0, policy_version 1494739 (0.0006) [2023-12-27 02:16:42,625][105692] Updated weights for policy 0, policy_version 1494749 (0.0006) [2023-12-27 02:16:42,685][105692] Updated weights for policy 0, policy_version 1494759 (0.0007) [2023-12-27 02:16:42,863][105620] Updated weights for policy 1, policy_version 1497272 (0.0009) [2023-12-27 02:16:42,920][105620] Updated weights for policy 1, policy_version 1497282 (0.0010) [2023-12-27 02:16:42,973][105620] Updated weights for policy 1, policy_version 1497292 (0.0011) [2023-12-27 02:16:43,371][105692] Updated weights for policy 0, policy_version 1494769 (0.0008) [2023-12-27 02:16:43,423][105692] Updated weights for policy 0, policy_version 1494779 (0.0008) [2023-12-27 02:16:43,479][105692] Updated weights for policy 0, policy_version 1494789 (0.0008) [2023-12-27 02:16:43,718][105620] Updated weights for policy 1, policy_version 1497302 (0.0007) [2023-12-27 02:16:43,775][105620] Updated weights for policy 1, policy_version 1497312 (0.0008) [2023-12-27 02:16:43,827][105620] Updated weights for policy 1, policy_version 1497322 (0.0010) [2023-12-27 02:16:44,075][105692] Updated weights for policy 0, policy_version 1494799 (0.0006) [2023-12-27 02:16:44,137][105692] Updated weights for policy 0, policy_version 1494809 (0.0005) [2023-12-27 02:16:44,200][105692] Updated weights for policy 0, policy_version 1494819 (0.0007) [2023-12-27 02:16:44,502][105620] Updated weights for policy 1, policy_version 1497332 (0.0011) [2023-12-27 02:16:44,561][105620] Updated weights for policy 1, policy_version 1497342 (0.0010) [2023-12-27 02:16:44,621][105620] Updated weights for policy 1, policy_version 1497352 (0.0008) [2023-12-27 02:16:44,831][105692] Updated weights for policy 0, policy_version 1494829 (0.0008) [2023-12-27 02:16:44,891][105692] Updated weights for policy 0, policy_version 1494839 (0.0006) [2023-12-27 02:16:44,955][105692] Updated weights for policy 0, policy_version 1494849 (0.0007) [2023-12-27 02:16:45,364][105620] Updated weights for policy 1, policy_version 1497362 (0.0006) [2023-12-27 02:16:45,427][105620] Updated weights for policy 1, policy_version 1497372 (0.0010) [2023-12-27 02:16:45,487][105620] Updated weights for policy 1, policy_version 1497382 (0.0009) [2023-12-27 02:16:45,536][105620] Updated weights for policy 1, policy_version 1497392 (0.0008) [2023-12-27 02:16:45,547][105692] Updated weights for policy 0, policy_version 1494859 (0.0009) [2023-12-27 02:16:45,595][105692] Updated weights for policy 0, policy_version 1494869 (0.0009) [2023-12-27 02:16:45,643][105692] Updated weights for policy 0, policy_version 1494879 (0.0009) [2023-12-27 02:16:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 766132224. Throughput: 0: 9658.1, 1: 9763.3. Samples: 766099176. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:16:46,063][104569] Avg episode reward: [(0, '7904.017'), (1, '9082.485')] [2023-12-27 02:16:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001497392_383385600.pth... [2023-12-27 02:16:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001494888_382746624.pth... [2023-12-27 02:16:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001496240_383090688.pth [2023-12-27 02:16:46,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001493736_382451712.pth [2023-12-27 02:16:46,339][105620] Updated weights for policy 1, policy_version 1497402 (0.0011) [2023-12-27 02:16:46,378][105692] Updated weights for policy 0, policy_version 1494889 (0.0009) [2023-12-27 02:16:46,397][105620] Updated weights for policy 1, policy_version 1497412 (0.0010) [2023-12-27 02:16:46,428][105692] Updated weights for policy 0, policy_version 1494899 (0.0005) [2023-12-27 02:16:46,458][105620] Updated weights for policy 1, policy_version 1497422 (0.0010) [2023-12-27 02:16:46,487][105692] Updated weights for policy 0, policy_version 1494909 (0.0005) [2023-12-27 02:16:46,541][105692] Updated weights for policy 0, policy_version 1494919 (0.0006) [2023-12-27 02:16:47,213][105620] Updated weights for policy 1, policy_version 1497432 (0.0011) [2023-12-27 02:16:47,239][105692] Updated weights for policy 0, policy_version 1494929 (0.0006) [2023-12-27 02:16:47,279][105620] Updated weights for policy 1, policy_version 1497442 (0.0010) [2023-12-27 02:16:47,302][105692] Updated weights for policy 0, policy_version 1494939 (0.0007) [2023-12-27 02:16:47,342][105620] Updated weights for policy 1, policy_version 1497452 (0.0010) [2023-12-27 02:16:47,361][105692] Updated weights for policy 0, policy_version 1494949 (0.0007) [2023-12-27 02:16:47,938][105620] Updated weights for policy 1, policy_version 1497462 (0.0006) [2023-12-27 02:16:47,988][105620] Updated weights for policy 1, policy_version 1497472 (0.0009) [2023-12-27 02:16:48,047][105620] Updated weights for policy 1, policy_version 1497482 (0.0009) [2023-12-27 02:16:48,122][105692] Updated weights for policy 0, policy_version 1494959 (0.0008) [2023-12-27 02:16:48,174][105692] Updated weights for policy 0, policy_version 1494969 (0.0008) [2023-12-27 02:16:48,230][105692] Updated weights for policy 0, policy_version 1494979 (0.0008) [2023-12-27 02:16:48,805][105620] Updated weights for policy 1, policy_version 1497492 (0.0010) [2023-12-27 02:16:48,860][105620] Updated weights for policy 1, policy_version 1497502 (0.0009) [2023-12-27 02:16:48,916][105620] Updated weights for policy 1, policy_version 1497512 (0.0010) [2023-12-27 02:16:48,992][105692] Updated weights for policy 0, policy_version 1494989 (0.0009) [2023-12-27 02:16:49,058][105692] Updated weights for policy 0, policy_version 1494999 (0.0009) [2023-12-27 02:16:49,120][105692] Updated weights for policy 0, policy_version 1495009 (0.0009) [2023-12-27 02:16:49,584][105620] Updated weights for policy 1, policy_version 1497522 (0.0006) [2023-12-27 02:16:49,646][105620] Updated weights for policy 1, policy_version 1497532 (0.0008) [2023-12-27 02:16:49,701][105620] Updated weights for policy 1, policy_version 1497542 (0.0005) [2023-12-27 02:16:49,759][105620] Updated weights for policy 1, policy_version 1497552 (0.0005) [2023-12-27 02:16:49,960][105692] Updated weights for policy 0, policy_version 1495019 (0.0008) [2023-12-27 02:16:50,027][105692] Updated weights for policy 0, policy_version 1495029 (0.0008) [2023-12-27 02:16:50,090][105692] Updated weights for policy 0, policy_version 1495039 (0.0008) [2023-12-27 02:16:50,447][105620] Updated weights for policy 1, policy_version 1497562 (0.0009) [2023-12-27 02:16:50,496][105620] Updated weights for policy 1, policy_version 1497572 (0.0005) [2023-12-27 02:16:50,548][105620] Updated weights for policy 1, policy_version 1497582 (0.0006) [2023-12-27 02:16:50,850][105692] Updated weights for policy 0, policy_version 1495049 (0.0008) [2023-12-27 02:16:50,902][105692] Updated weights for policy 0, policy_version 1495059 (0.0009) [2023-12-27 02:16:50,960][105692] Updated weights for policy 0, policy_version 1495069 (0.0010) [2023-12-27 02:16:51,024][105692] Updated weights for policy 0, policy_version 1495079 (0.0010) [2023-12-27 02:16:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 766230528. Throughput: 0: 9805.8, 1: 9787.9. Samples: 766218060. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:16:51,063][104569] Avg episode reward: [(0, '8628.463'), (1, '9080.010')] [2023-12-27 02:16:51,228][105620] Updated weights for policy 1, policy_version 1497592 (0.0007) [2023-12-27 02:16:51,295][105620] Updated weights for policy 1, policy_version 1497602 (0.0008) [2023-12-27 02:16:51,363][105620] Updated weights for policy 1, policy_version 1497612 (0.0008) [2023-12-27 02:16:51,831][105692] Updated weights for policy 0, policy_version 1495089 (0.0006) [2023-12-27 02:16:51,890][105692] Updated weights for policy 0, policy_version 1495099 (0.0010) [2023-12-27 02:16:51,949][105692] Updated weights for policy 0, policy_version 1495109 (0.0011) [2023-12-27 02:16:52,097][105620] Updated weights for policy 1, policy_version 1497622 (0.0009) [2023-12-27 02:16:52,159][105620] Updated weights for policy 1, policy_version 1497632 (0.0010) [2023-12-27 02:16:52,220][105620] Updated weights for policy 1, policy_version 1497642 (0.0009) [2023-12-27 02:16:52,682][105692] Updated weights for policy 0, policy_version 1495119 (0.0007) [2023-12-27 02:16:52,733][105692] Updated weights for policy 0, policy_version 1495129 (0.0005) [2023-12-27 02:16:52,784][105692] Updated weights for policy 0, policy_version 1495139 (0.0006) [2023-12-27 02:16:52,947][105620] Updated weights for policy 1, policy_version 1497652 (0.0008) [2023-12-27 02:16:53,017][105620] Updated weights for policy 1, policy_version 1497662 (0.0007) [2023-12-27 02:16:53,082][105620] Updated weights for policy 1, policy_version 1497672 (0.0010) [2023-12-27 02:16:53,492][105692] Updated weights for policy 0, policy_version 1495149 (0.0008) [2023-12-27 02:16:53,540][105692] Updated weights for policy 0, policy_version 1495159 (0.0008) [2023-12-27 02:16:53,590][105692] Updated weights for policy 0, policy_version 1495169 (0.0009) [2023-12-27 02:16:53,789][105620] Updated weights for policy 1, policy_version 1497682 (0.0010) [2023-12-27 02:16:53,847][105620] Updated weights for policy 1, policy_version 1497692 (0.0010) [2023-12-27 02:16:53,895][105620] Updated weights for policy 1, policy_version 1497702 (0.0010) [2023-12-27 02:16:53,949][105620] Updated weights for policy 1, policy_version 1497712 (0.0010) [2023-12-27 02:16:54,302][105692] Updated weights for policy 0, policy_version 1495179 (0.0010) [2023-12-27 02:16:54,348][105692] Updated weights for policy 0, policy_version 1495189 (0.0010) [2023-12-27 02:16:54,400][105692] Updated weights for policy 0, policy_version 1495199 (0.0009) [2023-12-27 02:16:54,702][105620] Updated weights for policy 1, policy_version 1497722 (0.0009) [2023-12-27 02:16:54,764][105620] Updated weights for policy 1, policy_version 1497732 (0.0010) [2023-12-27 02:16:54,822][105620] Updated weights for policy 1, policy_version 1497742 (0.0010) [2023-12-27 02:16:55,081][105692] Updated weights for policy 0, policy_version 1495209 (0.0008) [2023-12-27 02:16:55,128][105692] Updated weights for policy 0, policy_version 1495219 (0.0007) [2023-12-27 02:16:55,172][105692] Updated weights for policy 0, policy_version 1495229 (0.0010) [2023-12-27 02:16:55,224][105692] Updated weights for policy 0, policy_version 1495239 (0.0010) [2023-12-27 02:16:55,527][105620] Updated weights for policy 1, policy_version 1497752 (0.0010) [2023-12-27 02:16:55,581][105620] Updated weights for policy 1, policy_version 1497762 (0.0010) [2023-12-27 02:16:55,639][105620] Updated weights for policy 1, policy_version 1497772 (0.0010) [2023-12-27 02:16:55,861][105692] Updated weights for policy 0, policy_version 1495249 (0.0006) [2023-12-27 02:16:55,907][105585] KL-divergence is very high: 147.9799 [2023-12-27 02:16:55,907][105692] Updated weights for policy 0, policy_version 1495259 (0.0005) [2023-12-27 02:16:55,948][105585] KL-divergence is very high: 202.5831 [2023-12-27 02:16:55,959][105692] Updated weights for policy 0, policy_version 1495269 (0.0008) [2023-12-27 02:16:56,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 766328832. Throughput: 0: 9767.7, 1: 9733.9. Samples: 766332596. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:16:56,062][104569] Avg episode reward: [(0, '8539.619'), (1, '8990.521')] [2023-12-27 02:16:56,387][105620] Updated weights for policy 1, policy_version 1497782 (0.0010) [2023-12-27 02:16:56,439][105620] Updated weights for policy 1, policy_version 1497792 (0.0010) [2023-12-27 02:16:56,500][105620] Updated weights for policy 1, policy_version 1497802 (0.0010) [2023-12-27 02:16:56,591][105692] Updated weights for policy 0, policy_version 1495279 (0.0008) [2023-12-27 02:16:56,653][105692] Updated weights for policy 0, policy_version 1495289 (0.0008) [2023-12-27 02:16:56,709][105692] Updated weights for policy 0, policy_version 1495299 (0.0007) [2023-12-27 02:16:57,184][105620] Updated weights for policy 1, policy_version 1497812 (0.0009) [2023-12-27 02:16:57,231][105620] Updated weights for policy 1, policy_version 1497822 (0.0009) [2023-12-27 02:16:57,280][105620] Updated weights for policy 1, policy_version 1497832 (0.0009) [2023-12-27 02:16:57,478][105692] Updated weights for policy 0, policy_version 1495309 (0.0008) [2023-12-27 02:16:57,528][105692] Updated weights for policy 0, policy_version 1495319 (0.0009) [2023-12-27 02:16:57,575][105692] Updated weights for policy 0, policy_version 1495329 (0.0008) [2023-12-27 02:16:57,921][105620] Updated weights for policy 1, policy_version 1497842 (0.0009) [2023-12-27 02:16:57,971][105620] Updated weights for policy 1, policy_version 1497852 (0.0010) [2023-12-27 02:16:58,039][105620] Updated weights for policy 1, policy_version 1497862 (0.0010) [2023-12-27 02:16:58,100][105620] Updated weights for policy 1, policy_version 1497872 (0.0010) [2023-12-27 02:16:58,300][105692] Updated weights for policy 0, policy_version 1495339 (0.0008) [2023-12-27 02:16:58,361][105692] Updated weights for policy 0, policy_version 1495349 (0.0008) [2023-12-27 02:16:58,424][105692] Updated weights for policy 0, policy_version 1495359 (0.0007) [2023-12-27 02:16:58,881][105620] Updated weights for policy 1, policy_version 1497882 (0.0010) [2023-12-27 02:16:58,957][105620] Updated weights for policy 1, policy_version 1497892 (0.0011) [2023-12-27 02:16:59,008][105620] Updated weights for policy 1, policy_version 1497902 (0.0008) [2023-12-27 02:16:59,259][105692] Updated weights for policy 0, policy_version 1495369 (0.0010) [2023-12-27 02:16:59,326][105692] Updated weights for policy 0, policy_version 1495379 (0.0010) [2023-12-27 02:16:59,392][105692] Updated weights for policy 0, policy_version 1495389 (0.0008) [2023-12-27 02:16:59,453][105692] Updated weights for policy 0, policy_version 1495399 (0.0008) [2023-12-27 02:16:59,813][105620] Updated weights for policy 1, policy_version 1497912 (0.0007) [2023-12-27 02:16:59,881][105620] Updated weights for policy 1, policy_version 1497922 (0.0006) [2023-12-27 02:16:59,946][105620] Updated weights for policy 1, policy_version 1497932 (0.0008) [2023-12-27 02:17:00,142][105692] Updated weights for policy 0, policy_version 1495409 (0.0008) [2023-12-27 02:17:00,208][105692] Updated weights for policy 0, policy_version 1495419 (0.0007) [2023-12-27 02:17:00,272][105692] Updated weights for policy 0, policy_version 1495429 (0.0008) [2023-12-27 02:17:00,664][105620] Updated weights for policy 1, policy_version 1497942 (0.0007) [2023-12-27 02:17:00,728][105620] Updated weights for policy 1, policy_version 1497952 (0.0008) [2023-12-27 02:17:00,776][105620] Updated weights for policy 1, policy_version 1497962 (0.0005) [2023-12-27 02:17:00,924][105692] Updated weights for policy 0, policy_version 1495439 (0.0008) [2023-12-27 02:17:00,970][105692] Updated weights for policy 0, policy_version 1495449 (0.0008) [2023-12-27 02:17:01,028][105692] Updated weights for policy 0, policy_version 1495459 (0.0009) [2023-12-27 02:17:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 766427136. Throughput: 0: 9797.9, 1: 9808.4. Samples: 766392504. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:17:01,063][104569] Avg episode reward: [(0, '8180.361'), (1, '9080.814')] [2023-12-27 02:17:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001497968_383533056.pth... [2023-12-27 02:17:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001495464_382894080.pth... [2023-12-27 02:17:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001496816_383238144.pth [2023-12-27 02:17:01,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001494312_382599168.pth [2023-12-27 02:17:01,488][105620] Updated weights for policy 1, policy_version 1497972 (0.0006) [2023-12-27 02:17:01,550][105620] Updated weights for policy 1, policy_version 1497982 (0.0008) [2023-12-27 02:17:01,612][105620] Updated weights for policy 1, policy_version 1497992 (0.0009) [2023-12-27 02:17:01,847][105692] Updated weights for policy 0, policy_version 1495469 (0.0010) [2023-12-27 02:17:01,908][105692] Updated weights for policy 0, policy_version 1495479 (0.0010) [2023-12-27 02:17:01,953][105692] Updated weights for policy 0, policy_version 1495489 (0.0010) [2023-12-27 02:17:02,253][105620] Updated weights for policy 1, policy_version 1498002 (0.0008) [2023-12-27 02:17:02,300][105620] Updated weights for policy 1, policy_version 1498012 (0.0006) [2023-12-27 02:17:02,347][105620] Updated weights for policy 1, policy_version 1498022 (0.0008) [2023-12-27 02:17:02,406][105620] Updated weights for policy 1, policy_version 1498032 (0.0008) [2023-12-27 02:17:02,671][105692] Updated weights for policy 0, policy_version 1495499 (0.0010) [2023-12-27 02:17:02,730][105692] Updated weights for policy 0, policy_version 1495509 (0.0009) [2023-12-27 02:17:02,788][105692] Updated weights for policy 0, policy_version 1495519 (0.0009) [2023-12-27 02:17:03,077][105620] Updated weights for policy 1, policy_version 1498042 (0.0006) [2023-12-27 02:17:03,141][105620] Updated weights for policy 1, policy_version 1498052 (0.0008) [2023-12-27 02:17:03,205][105620] Updated weights for policy 1, policy_version 1498062 (0.0008) [2023-12-27 02:17:03,447][105692] Updated weights for policy 0, policy_version 1495529 (0.0006) [2023-12-27 02:17:03,509][105692] Updated weights for policy 0, policy_version 1495539 (0.0005) [2023-12-27 02:17:03,576][105692] Updated weights for policy 0, policy_version 1495549 (0.0007) [2023-12-27 02:17:03,628][105692] Updated weights for policy 0, policy_version 1495559 (0.0009) [2023-12-27 02:17:03,904][105620] Updated weights for policy 1, policy_version 1498072 (0.0007) [2023-12-27 02:17:03,971][105620] Updated weights for policy 1, policy_version 1498082 (0.0006) [2023-12-27 02:17:04,037][105620] Updated weights for policy 1, policy_version 1498092 (0.0010) [2023-12-27 02:17:04,257][105692] Updated weights for policy 0, policy_version 1495569 (0.0010) [2023-12-27 02:17:04,313][105692] Updated weights for policy 0, policy_version 1495579 (0.0011) [2023-12-27 02:17:04,373][105692] Updated weights for policy 0, policy_version 1495589 (0.0011) [2023-12-27 02:17:04,684][105620] Updated weights for policy 1, policy_version 1498102 (0.0011) [2023-12-27 02:17:04,757][105620] Updated weights for policy 1, policy_version 1498112 (0.0011) [2023-12-27 02:17:04,820][105620] Updated weights for policy 1, policy_version 1498122 (0.0008) [2023-12-27 02:17:05,057][105692] Updated weights for policy 0, policy_version 1495599 (0.0010) [2023-12-27 02:17:05,106][105692] Updated weights for policy 0, policy_version 1495609 (0.0010) [2023-12-27 02:17:05,154][105692] Updated weights for policy 0, policy_version 1495619 (0.0010) [2023-12-27 02:17:05,472][105620] Updated weights for policy 1, policy_version 1498132 (0.0009) [2023-12-27 02:17:05,530][105620] Updated weights for policy 1, policy_version 1498142 (0.0010) [2023-12-27 02:17:05,595][105620] Updated weights for policy 1, policy_version 1498152 (0.0010) [2023-12-27 02:17:05,893][105692] Updated weights for policy 0, policy_version 1495629 (0.0010) [2023-12-27 02:17:05,948][105692] Updated weights for policy 0, policy_version 1495641 (0.0010) [2023-12-27 02:17:06,002][105692] Updated weights for policy 0, policy_version 1495653 (0.0011) [2023-12-27 02:17:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 766525440. Throughput: 0: 9804.8, 1: 9853.9. Samples: 766510420. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:17:06,062][104569] Avg episode reward: [(0, '8266.090'), (1, '9259.396')] [2023-12-27 02:17:06,206][105620] Updated weights for policy 1, policy_version 1498162 (0.0010) [2023-12-27 02:17:06,262][105620] Updated weights for policy 1, policy_version 1498172 (0.0011) [2023-12-27 02:17:06,318][105620] Updated weights for policy 1, policy_version 1498182 (0.0011) [2023-12-27 02:17:06,370][105620] Updated weights for policy 1, policy_version 1498192 (0.0011) [2023-12-27 02:17:06,901][105692] Updated weights for policy 0, policy_version 1495663 (0.0009) [2023-12-27 02:17:06,964][105692] Updated weights for policy 0, policy_version 1495673 (0.0009) [2023-12-27 02:17:07,024][105692] Updated weights for policy 0, policy_version 1495683 (0.0008) [2023-12-27 02:17:07,043][105620] Updated weights for policy 1, policy_version 1498202 (0.0011) [2023-12-27 02:17:07,092][105620] Updated weights for policy 1, policy_version 1498212 (0.0010) [2023-12-27 02:17:07,144][105620] Updated weights for policy 1, policy_version 1498222 (0.0007) [2023-12-27 02:17:07,667][105692] Updated weights for policy 0, policy_version 1495693 (0.0009) [2023-12-27 02:17:07,720][105692] Updated weights for policy 0, policy_version 1495703 (0.0009) [2023-12-27 02:17:07,781][105692] Updated weights for policy 0, policy_version 1495713 (0.0008) [2023-12-27 02:17:07,799][105620] Updated weights for policy 1, policy_version 1498232 (0.0010) [2023-12-27 02:17:07,861][105620] Updated weights for policy 1, policy_version 1498242 (0.0010) [2023-12-27 02:17:07,922][105620] Updated weights for policy 1, policy_version 1498252 (0.0010) [2023-12-27 02:17:08,567][105692] Updated weights for policy 0, policy_version 1495723 (0.0007) [2023-12-27 02:17:08,616][105692] Updated weights for policy 0, policy_version 1495733 (0.0008) [2023-12-27 02:17:08,627][105620] Updated weights for policy 1, policy_version 1498262 (0.0008) [2023-12-27 02:17:08,670][105692] Updated weights for policy 0, policy_version 1495743 (0.0007) [2023-12-27 02:17:08,680][105620] Updated weights for policy 1, policy_version 1498272 (0.0007) [2023-12-27 02:17:08,738][105620] Updated weights for policy 1, policy_version 1498282 (0.0006) [2023-12-27 02:17:09,480][105620] Updated weights for policy 1, policy_version 1498292 (0.0009) [2023-12-27 02:17:09,510][105692] Updated weights for policy 0, policy_version 1495753 (0.0007) [2023-12-27 02:17:09,535][105620] Updated weights for policy 1, policy_version 1498302 (0.0009) [2023-12-27 02:17:09,571][105692] Updated weights for policy 0, policy_version 1495763 (0.0008) [2023-12-27 02:17:09,587][105620] Updated weights for policy 1, policy_version 1498312 (0.0009) [2023-12-27 02:17:09,632][105692] Updated weights for policy 0, policy_version 1495773 (0.0007) [2023-12-27 02:17:09,689][105692] Updated weights for policy 0, policy_version 1495783 (0.0009) [2023-12-27 02:17:10,292][105620] Updated weights for policy 1, policy_version 1498322 (0.0009) [2023-12-27 02:17:10,352][105620] Updated weights for policy 1, policy_version 1498332 (0.0008) [2023-12-27 02:17:10,407][105620] Updated weights for policy 1, policy_version 1498342 (0.0010) [2023-12-27 02:17:10,464][105620] Updated weights for policy 1, policy_version 1498352 (0.0008) [2023-12-27 02:17:10,479][105692] Updated weights for policy 0, policy_version 1495793 (0.0010) [2023-12-27 02:17:10,545][105692] Updated weights for policy 0, policy_version 1495803 (0.0011) [2023-12-27 02:17:10,618][105692] Updated weights for policy 0, policy_version 1495813 (0.0010) [2023-12-27 02:17:11,045][105620] Updated weights for policy 1, policy_version 1498362 (0.0006) [2023-12-27 02:17:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 766615552. Throughput: 0: 9808.1, 1: 9967.9. Samples: 766628280. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:17:11,063][104569] Avg episode reward: [(0, '8625.028'), (1, '9351.676')] [2023-12-27 02:17:11,108][105620] Updated weights for policy 1, policy_version 1498372 (0.0010) [2023-12-27 02:17:11,171][105620] Updated weights for policy 1, policy_version 1498382 (0.0009) [2023-12-27 02:17:11,360][105692] Updated weights for policy 0, policy_version 1495823 (0.0010) [2023-12-27 02:17:11,420][105692] Updated weights for policy 0, policy_version 1495833 (0.0009) [2023-12-27 02:17:11,473][105692] Updated weights for policy 0, policy_version 1495843 (0.0006) [2023-12-27 02:17:11,899][105620] Updated weights for policy 1, policy_version 1498392 (0.0007) [2023-12-27 02:17:11,958][105620] Updated weights for policy 1, policy_version 1498402 (0.0005) [2023-12-27 02:17:12,016][105620] Updated weights for policy 1, policy_version 1498412 (0.0006) [2023-12-27 02:17:12,192][105692] Updated weights for policy 0, policy_version 1495853 (0.0008) [2023-12-27 02:17:12,254][105692] Updated weights for policy 0, policy_version 1495863 (0.0009) [2023-12-27 02:17:12,321][105692] Updated weights for policy 0, policy_version 1495873 (0.0008) [2023-12-27 02:17:12,701][105620] Updated weights for policy 1, policy_version 1498422 (0.0007) [2023-12-27 02:17:12,756][105620] Updated weights for policy 1, policy_version 1498432 (0.0005) [2023-12-27 02:17:12,815][105620] Updated weights for policy 1, policy_version 1498442 (0.0005) [2023-12-27 02:17:13,072][105692] Updated weights for policy 0, policy_version 1495883 (0.0009) [2023-12-27 02:17:13,130][105692] Updated weights for policy 0, policy_version 1495893 (0.0009) [2023-12-27 02:17:13,191][105692] Updated weights for policy 0, policy_version 1495903 (0.0006) [2023-12-27 02:17:13,549][105620] Updated weights for policy 1, policy_version 1498452 (0.0007) [2023-12-27 02:17:13,618][105620] Updated weights for policy 1, policy_version 1498462 (0.0009) [2023-12-27 02:17:13,685][105620] Updated weights for policy 1, policy_version 1498472 (0.0009) [2023-12-27 02:17:13,790][105692] Updated weights for policy 0, policy_version 1495913 (0.0005) [2023-12-27 02:17:13,842][105692] Updated weights for policy 0, policy_version 1495923 (0.0008) [2023-12-27 02:17:13,889][105692] Updated weights for policy 0, policy_version 1495933 (0.0009) [2023-12-27 02:17:13,939][105692] Updated weights for policy 0, policy_version 1495943 (0.0008) [2023-12-27 02:17:14,456][105620] Updated weights for policy 1, policy_version 1498482 (0.0010) [2023-12-27 02:17:14,513][105620] Updated weights for policy 1, policy_version 1498492 (0.0009) [2023-12-27 02:17:14,574][105620] Updated weights for policy 1, policy_version 1498502 (0.0010) [2023-12-27 02:17:14,622][105620] Updated weights for policy 1, policy_version 1498512 (0.0009) [2023-12-27 02:17:14,702][105692] Updated weights for policy 0, policy_version 1495953 (0.0009) [2023-12-27 02:17:14,769][105692] Updated weights for policy 0, policy_version 1495963 (0.0009) [2023-12-27 02:17:14,830][105692] Updated weights for policy 0, policy_version 1495973 (0.0009) [2023-12-27 02:17:15,409][105620] Updated weights for policy 1, policy_version 1498522 (0.0009) [2023-12-27 02:17:15,465][105620] Updated weights for policy 1, policy_version 1498532 (0.0009) [2023-12-27 02:17:15,521][105620] Updated weights for policy 1, policy_version 1498542 (0.0010) [2023-12-27 02:17:15,561][105692] Updated weights for policy 0, policy_version 1495983 (0.0007) [2023-12-27 02:17:15,607][105692] Updated weights for policy 0, policy_version 1495993 (0.0005) [2023-12-27 02:17:15,665][105692] Updated weights for policy 0, policy_version 1496003 (0.0006) [2023-12-27 02:17:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 766713856. Throughput: 0: 9774.9, 1: 9854.7. Samples: 766685148. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:17:16,062][104569] Avg episode reward: [(0, '8445.324'), (1, '9171.605')] [2023-12-27 02:17:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001496008_383033344.pth... [2023-12-27 02:17:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001498544_383680512.pth... [2023-12-27 02:17:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001494888_382746624.pth [2023-12-27 02:17:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001497392_383385600.pth [2023-12-27 02:17:16,260][105692] Updated weights for policy 0, policy_version 1496013 (0.0009) [2023-12-27 02:17:16,316][105692] Updated weights for policy 0, policy_version 1496023 (0.0009) [2023-12-27 02:17:16,370][105620] Updated weights for policy 1, policy_version 1498552 (0.0007) [2023-12-27 02:17:16,371][105692] Updated weights for policy 0, policy_version 1496033 (0.0010) [2023-12-27 02:17:16,429][105620] Updated weights for policy 1, policy_version 1498562 (0.0009) [2023-12-27 02:17:16,483][105620] Updated weights for policy 1, policy_version 1498572 (0.0009) [2023-12-27 02:17:17,073][105692] Updated weights for policy 0, policy_version 1496043 (0.0009) [2023-12-27 02:17:17,140][105692] Updated weights for policy 0, policy_version 1496053 (0.0011) [2023-12-27 02:17:17,178][105620] Updated weights for policy 1, policy_version 1498582 (0.0008) [2023-12-27 02:17:17,192][105692] Updated weights for policy 0, policy_version 1496063 (0.0010) [2023-12-27 02:17:17,238][105620] Updated weights for policy 1, policy_version 1498592 (0.0006) [2023-12-27 02:17:17,304][105620] Updated weights for policy 1, policy_version 1498602 (0.0008) [2023-12-27 02:17:17,845][105692] Updated weights for policy 0, policy_version 1496073 (0.0010) [2023-12-27 02:17:17,891][105692] Updated weights for policy 0, policy_version 1496083 (0.0005) [2023-12-27 02:17:17,942][105692] Updated weights for policy 0, policy_version 1496093 (0.0006) [2023-12-27 02:17:17,995][105692] Updated weights for policy 0, policy_version 1496103 (0.0008) [2023-12-27 02:17:18,016][105620] Updated weights for policy 1, policy_version 1498612 (0.0007) [2023-12-27 02:17:18,082][105620] Updated weights for policy 1, policy_version 1498622 (0.0010) [2023-12-27 02:17:18,143][105620] Updated weights for policy 1, policy_version 1498632 (0.0009) [2023-12-27 02:17:18,738][105692] Updated weights for policy 0, policy_version 1496113 (0.0009) [2023-12-27 02:17:18,786][105692] Updated weights for policy 0, policy_version 1496123 (0.0008) [2023-12-27 02:17:18,838][105692] Updated weights for policy 0, policy_version 1496133 (0.0009) [2023-12-27 02:17:18,851][105620] Updated weights for policy 1, policy_version 1498642 (0.0007) [2023-12-27 02:17:18,902][105620] Updated weights for policy 1, policy_version 1498652 (0.0010) [2023-12-27 02:17:18,954][105620] Updated weights for policy 1, policy_version 1498662 (0.0009) [2023-12-27 02:17:19,004][105620] Updated weights for policy 1, policy_version 1498672 (0.0008) [2023-12-27 02:17:19,576][105692] Updated weights for policy 0, policy_version 1496143 (0.0009) [2023-12-27 02:17:19,637][105692] Updated weights for policy 0, policy_version 1496153 (0.0010) [2023-12-27 02:17:19,692][105692] Updated weights for policy 0, policy_version 1496163 (0.0010) [2023-12-27 02:17:19,812][105620] Updated weights for policy 1, policy_version 1498682 (0.0007) [2023-12-27 02:17:19,879][105620] Updated weights for policy 1, policy_version 1498692 (0.0009) [2023-12-27 02:17:19,940][105620] Updated weights for policy 1, policy_version 1498702 (0.0010) [2023-12-27 02:17:20,435][105692] Updated weights for policy 0, policy_version 1496173 (0.0008) [2023-12-27 02:17:20,500][105692] Updated weights for policy 0, policy_version 1496183 (0.0006) [2023-12-27 02:17:20,562][105692] Updated weights for policy 0, policy_version 1496193 (0.0006) [2023-12-27 02:17:20,683][105620] Updated weights for policy 1, policy_version 1498712 (0.0010) [2023-12-27 02:17:20,749][105620] Updated weights for policy 1, policy_version 1498722 (0.0010) [2023-12-27 02:17:20,818][105620] Updated weights for policy 1, policy_version 1498732 (0.0009) [2023-12-27 02:17:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 766812160. Throughput: 0: 9750.5, 1: 9789.8. Samples: 766801240. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:17:21,062][104569] Avg episode reward: [(0, '8085.895'), (1, '9078.954')] [2023-12-27 02:17:21,138][105692] Updated weights for policy 0, policy_version 1496203 (0.0008) [2023-12-27 02:17:21,186][105692] Updated weights for policy 0, policy_version 1496213 (0.0009) [2023-12-27 02:17:21,233][105692] Updated weights for policy 0, policy_version 1496223 (0.0008) [2023-12-27 02:17:21,635][105620] Updated weights for policy 1, policy_version 1498742 (0.0007) [2023-12-27 02:17:21,700][105620] Updated weights for policy 1, policy_version 1498752 (0.0008) [2023-12-27 02:17:21,769][105620] Updated weights for policy 1, policy_version 1498762 (0.0007) [2023-12-27 02:17:22,091][105692] Updated weights for policy 0, policy_version 1496233 (0.0009) [2023-12-27 02:17:22,155][105692] Updated weights for policy 0, policy_version 1496243 (0.0010) [2023-12-27 02:17:22,215][105692] Updated weights for policy 0, policy_version 1496253 (0.0009) [2023-12-27 02:17:22,278][105692] Updated weights for policy 0, policy_version 1496263 (0.0009) [2023-12-27 02:17:22,470][105620] Updated weights for policy 1, policy_version 1498772 (0.0008) [2023-12-27 02:17:22,531][105620] Updated weights for policy 1, policy_version 1498782 (0.0008) [2023-12-27 02:17:22,593][105620] Updated weights for policy 1, policy_version 1498792 (0.0009) [2023-12-27 02:17:23,034][105692] Updated weights for policy 0, policy_version 1496273 (0.0007) [2023-12-27 02:17:23,094][105692] Updated weights for policy 0, policy_version 1496283 (0.0007) [2023-12-27 02:17:23,163][105692] Updated weights for policy 0, policy_version 1496293 (0.0006) [2023-12-27 02:17:23,377][105620] Updated weights for policy 1, policy_version 1498802 (0.0008) [2023-12-27 02:17:23,429][105620] Updated weights for policy 1, policy_version 1498812 (0.0008) [2023-12-27 02:17:23,491][105620] Updated weights for policy 1, policy_version 1498822 (0.0006) [2023-12-27 02:17:23,555][105620] Updated weights for policy 1, policy_version 1498832 (0.0008) [2023-12-27 02:17:23,732][105692] Updated weights for policy 0, policy_version 1496303 (0.0005) [2023-12-27 02:17:23,794][105692] Updated weights for policy 0, policy_version 1496313 (0.0005) [2023-12-27 02:17:23,851][105692] Updated weights for policy 0, policy_version 1496323 (0.0005) [2023-12-27 02:17:24,318][105620] Updated weights for policy 1, policy_version 1498842 (0.0009) [2023-12-27 02:17:24,372][105620] Updated weights for policy 1, policy_version 1498852 (0.0009) [2023-12-27 02:17:24,417][105620] Updated weights for policy 1, policy_version 1498862 (0.0010) [2023-12-27 02:17:24,443][105692] Updated weights for policy 0, policy_version 1496333 (0.0007) [2023-12-27 02:17:24,494][105692] Updated weights for policy 0, policy_version 1496343 (0.0010) [2023-12-27 02:17:24,546][105692] Updated weights for policy 0, policy_version 1496353 (0.0010) [2023-12-27 02:17:25,165][105620] Updated weights for policy 1, policy_version 1498872 (0.0007) [2023-12-27 02:17:25,214][105620] Updated weights for policy 1, policy_version 1498882 (0.0005) [2023-12-27 02:17:25,231][105692] Updated weights for policy 0, policy_version 1496363 (0.0010) [2023-12-27 02:17:25,267][105620] Updated weights for policy 1, policy_version 1498892 (0.0005) [2023-12-27 02:17:25,287][105692] Updated weights for policy 0, policy_version 1496373 (0.0008) [2023-12-27 02:17:25,340][105692] Updated weights for policy 0, policy_version 1496383 (0.0009) [2023-12-27 02:17:25,964][105620] Updated weights for policy 1, policy_version 1498902 (0.0005) [2023-12-27 02:17:26,017][105620] Updated weights for policy 1, policy_version 1498912 (0.0006) [2023-12-27 02:17:26,019][105692] Updated weights for policy 0, policy_version 1496393 (0.0009) [2023-12-27 02:17:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 766902272. Throughput: 0: 9769.9, 1: 9710.8. Samples: 766916956. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:17:26,063][104569] Avg episode reward: [(0, '8361.084'), (1, '9074.361')] [2023-12-27 02:17:26,070][105692] Updated weights for policy 0, policy_version 1496403 (0.0010) [2023-12-27 02:17:26,073][105620] Updated weights for policy 1, policy_version 1498922 (0.0006) [2023-12-27 02:17:26,123][105692] Updated weights for policy 0, policy_version 1496413 (0.0010) [2023-12-27 02:17:26,181][105692] Updated weights for policy 0, policy_version 1496423 (0.0010) [2023-12-27 02:17:26,734][105620] Updated weights for policy 1, policy_version 1498932 (0.0007) [2023-12-27 02:17:26,794][105620] Updated weights for policy 1, policy_version 1498942 (0.0009) [2023-12-27 02:17:26,856][105620] Updated weights for policy 1, policy_version 1498952 (0.0007) [2023-12-27 02:17:26,910][105692] Updated weights for policy 0, policy_version 1496433 (0.0007) [2023-12-27 02:17:26,969][105692] Updated weights for policy 0, policy_version 1496443 (0.0008) [2023-12-27 02:17:27,033][105692] Updated weights for policy 0, policy_version 1496453 (0.0005) [2023-12-27 02:17:27,539][105620] Updated weights for policy 1, policy_version 1498962 (0.0007) [2023-12-27 02:17:27,605][105620] Updated weights for policy 1, policy_version 1498972 (0.0006) [2023-12-27 02:17:27,642][105692] Updated weights for policy 0, policy_version 1496463 (0.0006) [2023-12-27 02:17:27,657][105620] Updated weights for policy 1, policy_version 1498982 (0.0005) [2023-12-27 02:17:27,697][105692] Updated weights for policy 0, policy_version 1496473 (0.0006) [2023-12-27 02:17:27,709][105620] Updated weights for policy 1, policy_version 1498992 (0.0007) [2023-12-27 02:17:27,756][105692] Updated weights for policy 0, policy_version 1496483 (0.0006) [2023-12-27 02:17:28,335][105620] Updated weights for policy 1, policy_version 1499002 (0.0008) [2023-12-27 02:17:28,394][105620] Updated weights for policy 1, policy_version 1499012 (0.0010) [2023-12-27 02:17:28,445][105692] Updated weights for policy 0, policy_version 1496493 (0.0007) [2023-12-27 02:17:28,446][105620] Updated weights for policy 1, policy_version 1499022 (0.0011) [2023-12-27 02:17:28,503][105692] Updated weights for policy 0, policy_version 1496503 (0.0008) [2023-12-27 02:17:28,555][105692] Updated weights for policy 0, policy_version 1496513 (0.0008) [2023-12-27 02:17:29,171][105620] Updated weights for policy 1, policy_version 1499032 (0.0006) [2023-12-27 02:17:29,221][105620] Updated weights for policy 1, policy_version 1499042 (0.0006) [2023-12-27 02:17:29,221][105692] Updated weights for policy 0, policy_version 1496523 (0.0009) [2023-12-27 02:17:29,279][105692] Updated weights for policy 0, policy_version 1496533 (0.0006) [2023-12-27 02:17:29,292][105620] Updated weights for policy 1, policy_version 1499052 (0.0008) [2023-12-27 02:17:29,344][105692] Updated weights for policy 0, policy_version 1496543 (0.0008) [2023-12-27 02:17:29,879][105620] Updated weights for policy 1, policy_version 1499062 (0.0008) [2023-12-27 02:17:29,938][105620] Updated weights for policy 1, policy_version 1499072 (0.0008) [2023-12-27 02:17:29,997][105620] Updated weights for policy 1, policy_version 1499082 (0.0008) [2023-12-27 02:17:30,129][105692] Updated weights for policy 0, policy_version 1496553 (0.0008) [2023-12-27 02:17:30,183][105692] Updated weights for policy 0, policy_version 1496563 (0.0006) [2023-12-27 02:17:30,240][105692] Updated weights for policy 0, policy_version 1496573 (0.0007) [2023-12-27 02:17:30,303][105692] Updated weights for policy 0, policy_version 1496583 (0.0007) [2023-12-27 02:17:30,759][105620] Updated weights for policy 1, policy_version 1499092 (0.0009) [2023-12-27 02:17:30,810][105620] Updated weights for policy 1, policy_version 1499102 (0.0010) [2023-12-27 02:17:30,858][105620] Updated weights for policy 1, policy_version 1499112 (0.0010) [2023-12-27 02:17:31,032][105692] Updated weights for policy 0, policy_version 1496593 (0.0006) [2023-12-27 02:17:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 767008768. Throughput: 0: 9826.7, 1: 9728.8. Samples: 766979168. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:17:31,063][104569] Avg episode reward: [(0, '8536.275'), (1, '9257.700')] [2023-12-27 02:17:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001499120_383827968.pth... [2023-12-27 02:17:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001497968_383533056.pth [2023-12-27 02:17:31,091][105692] Updated weights for policy 0, policy_version 1496603 (0.0009) [2023-12-27 02:17:31,149][105692] Updated weights for policy 0, policy_version 1496613 (0.0008) [2023-12-27 02:17:31,163][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001496616_383188992.pth... [2023-12-27 02:17:31,166][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001495464_382894080.pth [2023-12-27 02:17:31,670][105620] Updated weights for policy 1, policy_version 1499122 (0.0010) [2023-12-27 02:17:31,724][105620] Updated weights for policy 1, policy_version 1499132 (0.0008) [2023-12-27 02:17:31,776][105692] Updated weights for policy 0, policy_version 1496623 (0.0009) [2023-12-27 02:17:31,786][105620] Updated weights for policy 1, policy_version 1499142 (0.0009) [2023-12-27 02:17:31,840][105692] Updated weights for policy 0, policy_version 1496633 (0.0007) [2023-12-27 02:17:31,846][105620] Updated weights for policy 1, policy_version 1499152 (0.0006) [2023-12-27 02:17:31,902][105692] Updated weights for policy 0, policy_version 1496643 (0.0009) [2023-12-27 02:17:32,553][105620] Updated weights for policy 1, policy_version 1499162 (0.0009) [2023-12-27 02:17:32,607][105620] Updated weights for policy 1, policy_version 1499172 (0.0009) [2023-12-27 02:17:32,663][105620] Updated weights for policy 1, policy_version 1499182 (0.0008) [2023-12-27 02:17:32,676][105692] Updated weights for policy 0, policy_version 1496653 (0.0009) [2023-12-27 02:17:32,733][105692] Updated weights for policy 0, policy_version 1496663 (0.0009) [2023-12-27 02:17:32,789][105692] Updated weights for policy 0, policy_version 1496673 (0.0009) [2023-12-27 02:17:33,422][105620] Updated weights for policy 1, policy_version 1499192 (0.0008) [2023-12-27 02:17:33,485][105620] Updated weights for policy 1, policy_version 1499202 (0.0008) [2023-12-27 02:17:33,530][105620] Updated weights for policy 1, policy_version 1499212 (0.0008) [2023-12-27 02:17:33,540][105692] Updated weights for policy 0, policy_version 1496683 (0.0008) [2023-12-27 02:17:33,589][105692] Updated weights for policy 0, policy_version 1496693 (0.0008) [2023-12-27 02:17:33,635][105692] Updated weights for policy 0, policy_version 1496703 (0.0008) [2023-12-27 02:17:34,297][105692] Updated weights for policy 0, policy_version 1496713 (0.0008) [2023-12-27 02:17:34,303][105620] Updated weights for policy 1, policy_version 1499222 (0.0008) [2023-12-27 02:17:34,350][105692] Updated weights for policy 0, policy_version 1496723 (0.0007) [2023-12-27 02:17:34,366][105620] Updated weights for policy 1, policy_version 1499232 (0.0010) [2023-12-27 02:17:34,402][105692] Updated weights for policy 0, policy_version 1496733 (0.0008) [2023-12-27 02:17:34,426][105620] Updated weights for policy 1, policy_version 1499242 (0.0009) [2023-12-27 02:17:34,453][105692] Updated weights for policy 0, policy_version 1496743 (0.0008) [2023-12-27 02:17:35,118][105620] Updated weights for policy 1, policy_version 1499252 (0.0008) [2023-12-27 02:17:35,178][105620] Updated weights for policy 1, policy_version 1499262 (0.0011) [2023-12-27 02:17:35,193][105692] Updated weights for policy 0, policy_version 1496753 (0.0008) [2023-12-27 02:17:35,244][105620] Updated weights for policy 1, policy_version 1499272 (0.0011) [2023-12-27 02:17:35,255][105692] Updated weights for policy 0, policy_version 1496763 (0.0005) [2023-12-27 02:17:35,314][105692] Updated weights for policy 0, policy_version 1496773 (0.0005) [2023-12-27 02:17:35,937][105620] Updated weights for policy 1, policy_version 1499282 (0.0010) [2023-12-27 02:17:35,991][105620] Updated weights for policy 1, policy_version 1499292 (0.0005) [2023-12-27 02:17:35,991][105692] Updated weights for policy 0, policy_version 1496783 (0.0009) [2023-12-27 02:17:36,052][105620] Updated weights for policy 1, policy_version 1499302 (0.0008) [2023-12-27 02:17:36,053][105692] Updated weights for policy 0, policy_version 1496793 (0.0010) [2023-12-27 02:17:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 767098880. Throughput: 0: 9786.6, 1: 9708.4. Samples: 767095332. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:17:36,062][104569] Avg episode reward: [(0, '8360.981'), (1, '9167.662')] [2023-12-27 02:17:36,116][105620] Updated weights for policy 1, policy_version 1499312 (0.0008) [2023-12-27 02:17:36,119][105692] Updated weights for policy 0, policy_version 1496803 (0.0011) [2023-12-27 02:17:36,795][105692] Updated weights for policy 0, policy_version 1496813 (0.0009) [2023-12-27 02:17:36,832][105620] Updated weights for policy 1, policy_version 1499322 (0.0008) [2023-12-27 02:17:36,844][105692] Updated weights for policy 0, policy_version 1496823 (0.0008) [2023-12-27 02:17:36,889][105620] Updated weights for policy 1, policy_version 1499332 (0.0007) [2023-12-27 02:17:36,895][105692] Updated weights for policy 0, policy_version 1496833 (0.0008) [2023-12-27 02:17:36,944][105620] Updated weights for policy 1, policy_version 1499342 (0.0011) [2023-12-27 02:17:37,606][105692] Updated weights for policy 0, policy_version 1496843 (0.0006) [2023-12-27 02:17:37,619][105620] Updated weights for policy 1, policy_version 1499352 (0.0007) [2023-12-27 02:17:37,667][105692] Updated weights for policy 0, policy_version 1496853 (0.0009) [2023-12-27 02:17:37,679][105620] Updated weights for policy 1, policy_version 1499362 (0.0007) [2023-12-27 02:17:37,727][105692] Updated weights for policy 0, policy_version 1496863 (0.0008) [2023-12-27 02:17:37,730][105620] Updated weights for policy 1, policy_version 1499372 (0.0006) [2023-12-27 02:17:38,430][105620] Updated weights for policy 1, policy_version 1499382 (0.0006) [2023-12-27 02:17:38,485][105620] Updated weights for policy 1, policy_version 1499392 (0.0005) [2023-12-27 02:17:38,524][105692] Updated weights for policy 0, policy_version 1496873 (0.0008) [2023-12-27 02:17:38,548][105620] Updated weights for policy 1, policy_version 1499402 (0.0005) [2023-12-27 02:17:38,585][105692] Updated weights for policy 0, policy_version 1496883 (0.0007) [2023-12-27 02:17:38,645][105692] Updated weights for policy 0, policy_version 1496893 (0.0006) [2023-12-27 02:17:38,707][105692] Updated weights for policy 0, policy_version 1496903 (0.0006) [2023-12-27 02:17:39,077][105620] Updated weights for policy 1, policy_version 1499412 (0.0008) [2023-12-27 02:17:39,122][105620] Updated weights for policy 1, policy_version 1499422 (0.0010) [2023-12-27 02:17:39,175][105620] Updated weights for policy 1, policy_version 1499432 (0.0011) [2023-12-27 02:17:39,285][105692] Updated weights for policy 0, policy_version 1496913 (0.0007) [2023-12-27 02:17:39,343][105692] Updated weights for policy 0, policy_version 1496923 (0.0006) [2023-12-27 02:17:39,416][105692] Updated weights for policy 0, policy_version 1496933 (0.0010) [2023-12-27 02:17:39,923][105620] Updated weights for policy 1, policy_version 1499442 (0.0010) [2023-12-27 02:17:39,981][105620] Updated weights for policy 1, policy_version 1499452 (0.0009) [2023-12-27 02:17:40,039][105620] Updated weights for policy 1, policy_version 1499462 (0.0010) [2023-12-27 02:17:40,108][105620] Updated weights for policy 1, policy_version 1499472 (0.0010) [2023-12-27 02:17:40,122][105692] Updated weights for policy 0, policy_version 1496943 (0.0008) [2023-12-27 02:17:40,181][105692] Updated weights for policy 0, policy_version 1496953 (0.0009) [2023-12-27 02:17:40,238][105692] Updated weights for policy 0, policy_version 1496963 (0.0011) [2023-12-27 02:17:40,861][105620] Updated weights for policy 1, policy_version 1499482 (0.0009) [2023-12-27 02:17:40,927][105620] Updated weights for policy 1, policy_version 1499492 (0.0009) [2023-12-27 02:17:40,934][105692] Updated weights for policy 0, policy_version 1496973 (0.0008) [2023-12-27 02:17:40,985][105620] Updated weights for policy 1, policy_version 1499502 (0.0007) [2023-12-27 02:17:40,994][105692] Updated weights for policy 0, policy_version 1496983 (0.0011) [2023-12-27 02:17:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 767205376. Throughput: 0: 9845.6, 1: 9775.5. Samples: 767215548. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:17:41,063][104569] Avg episode reward: [(0, '8002.662'), (1, '8895.938')] [2023-12-27 02:17:41,063][105692] Updated weights for policy 0, policy_version 1496993 (0.0009) [2023-12-27 02:17:41,660][105620] Updated weights for policy 1, policy_version 1499512 (0.0007) [2023-12-27 02:17:41,728][105620] Updated weights for policy 1, policy_version 1499522 (0.0007) [2023-12-27 02:17:41,790][105620] Updated weights for policy 1, policy_version 1499532 (0.0008) [2023-12-27 02:17:41,816][105692] Updated weights for policy 0, policy_version 1497003 (0.0008) [2023-12-27 02:17:41,870][105692] Updated weights for policy 0, policy_version 1497013 (0.0010) [2023-12-27 02:17:41,922][105692] Updated weights for policy 0, policy_version 1497023 (0.0008) [2023-12-27 02:17:42,544][105620] Updated weights for policy 1, policy_version 1499542 (0.0008) [2023-12-27 02:17:42,606][105620] Updated weights for policy 1, policy_version 1499552 (0.0010) [2023-12-27 02:17:42,635][105692] Updated weights for policy 0, policy_version 1497033 (0.0010) [2023-12-27 02:17:42,663][105620] Updated weights for policy 1, policy_version 1499562 (0.0009) [2023-12-27 02:17:42,693][105692] Updated weights for policy 0, policy_version 1497043 (0.0011) [2023-12-27 02:17:42,745][105692] Updated weights for policy 0, policy_version 1497053 (0.0011) [2023-12-27 02:17:42,808][105692] Updated weights for policy 0, policy_version 1497063 (0.0011) [2023-12-27 02:17:43,433][105692] Updated weights for policy 0, policy_version 1497073 (0.0010) [2023-12-27 02:17:43,458][105620] Updated weights for policy 1, policy_version 1499572 (0.0005) [2023-12-27 02:17:43,486][105692] Updated weights for policy 0, policy_version 1497083 (0.0009) [2023-12-27 02:17:43,510][105620] Updated weights for policy 1, policy_version 1499582 (0.0005) [2023-12-27 02:17:43,549][105692] Updated weights for policy 0, policy_version 1497093 (0.0008) [2023-12-27 02:17:43,558][105620] Updated weights for policy 1, policy_version 1499592 (0.0005) [2023-12-27 02:17:44,105][105620] Updated weights for policy 1, policy_version 1499602 (0.0007) [2023-12-27 02:17:44,164][105620] Updated weights for policy 1, policy_version 1499612 (0.0007) [2023-12-27 02:17:44,228][105620] Updated weights for policy 1, policy_version 1499622 (0.0009) [2023-12-27 02:17:44,283][105620] Updated weights for policy 1, policy_version 1499632 (0.0009) [2023-12-27 02:17:44,313][105692] Updated weights for policy 0, policy_version 1497103 (0.0008) [2023-12-27 02:17:44,371][105692] Updated weights for policy 0, policy_version 1497113 (0.0010) [2023-12-27 02:17:44,429][105692] Updated weights for policy 0, policy_version 1497123 (0.0009) [2023-12-27 02:17:44,987][105620] Updated weights for policy 1, policy_version 1499642 (0.0009) [2023-12-27 02:17:45,037][105620] Updated weights for policy 1, policy_version 1499652 (0.0011) [2023-12-27 02:17:45,097][105620] Updated weights for policy 1, policy_version 1499662 (0.0011) [2023-12-27 02:17:45,231][105692] Updated weights for policy 0, policy_version 1497133 (0.0009) [2023-12-27 02:17:45,294][105692] Updated weights for policy 0, policy_version 1497143 (0.0009) [2023-12-27 02:17:45,358][105692] Updated weights for policy 0, policy_version 1497153 (0.0009) [2023-12-27 02:17:45,808][105620] Updated weights for policy 1, policy_version 1499672 (0.0007) [2023-12-27 02:17:45,863][105620] Updated weights for policy 1, policy_version 1499682 (0.0006) [2023-12-27 02:17:45,925][105620] Updated weights for policy 1, policy_version 1499692 (0.0009) [2023-12-27 02:17:46,062][104569] Fps is (10 sec: 20479.2, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 767303680. Throughput: 0: 9822.6, 1: 9768.6. Samples: 767274112. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:17:46,063][104569] Avg episode reward: [(0, '8354.423'), (1, '8900.778')] [2023-12-27 02:17:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001497160_383328256.pth... [2023-12-27 02:17:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001499696_383975424.pth... [2023-12-27 02:17:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001496008_383033344.pth [2023-12-27 02:17:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001498544_383680512.pth [2023-12-27 02:17:46,150][105692] Updated weights for policy 0, policy_version 1497163 (0.0010) [2023-12-27 02:17:46,200][105692] Updated weights for policy 0, policy_version 1497173 (0.0009) [2023-12-27 02:17:46,262][105692] Updated weights for policy 0, policy_version 1497183 (0.0009) [2023-12-27 02:17:46,641][105620] Updated weights for policy 1, policy_version 1499702 (0.0010) [2023-12-27 02:17:46,691][105620] Updated weights for policy 1, policy_version 1499712 (0.0009) [2023-12-27 02:17:46,748][105620] Updated weights for policy 1, policy_version 1499722 (0.0005) [2023-12-27 02:17:47,060][105692] Updated weights for policy 0, policy_version 1497193 (0.0008) [2023-12-27 02:17:47,123][105692] Updated weights for policy 0, policy_version 1497203 (0.0009) [2023-12-27 02:17:47,180][105692] Updated weights for policy 0, policy_version 1497213 (0.0008) [2023-12-27 02:17:47,239][105692] Updated weights for policy 0, policy_version 1497223 (0.0008) [2023-12-27 02:17:47,454][105620] Updated weights for policy 1, policy_version 1499732 (0.0009) [2023-12-27 02:17:47,506][105620] Updated weights for policy 1, policy_version 1499742 (0.0009) [2023-12-27 02:17:47,559][105620] Updated weights for policy 1, policy_version 1499752 (0.0008) [2023-12-27 02:17:47,922][105692] Updated weights for policy 0, policy_version 1497233 (0.0009) [2023-12-27 02:17:47,976][105692] Updated weights for policy 0, policy_version 1497243 (0.0008) [2023-12-27 02:17:48,030][105692] Updated weights for policy 0, policy_version 1497253 (0.0009) [2023-12-27 02:17:48,358][105620] Updated weights for policy 1, policy_version 1499762 (0.0008) [2023-12-27 02:17:48,424][105620] Updated weights for policy 1, policy_version 1499772 (0.0008) [2023-12-27 02:17:48,488][105620] Updated weights for policy 1, policy_version 1499782 (0.0008) [2023-12-27 02:17:48,551][105620] Updated weights for policy 1, policy_version 1499792 (0.0007) [2023-12-27 02:17:48,842][105692] Updated weights for policy 0, policy_version 1497263 (0.0008) [2023-12-27 02:17:48,901][105692] Updated weights for policy 0, policy_version 1497273 (0.0009) [2023-12-27 02:17:48,967][105692] Updated weights for policy 0, policy_version 1497283 (0.0010) [2023-12-27 02:17:49,180][105620] Updated weights for policy 1, policy_version 1499802 (0.0010) [2023-12-27 02:17:49,243][105620] Updated weights for policy 1, policy_version 1499812 (0.0009) [2023-12-27 02:17:49,297][105620] Updated weights for policy 1, policy_version 1499822 (0.0006) [2023-12-27 02:17:49,745][105692] Updated weights for policy 0, policy_version 1497293 (0.0010) [2023-12-27 02:17:49,790][105692] Updated weights for policy 0, policy_version 1497303 (0.0010) [2023-12-27 02:17:49,847][105692] Updated weights for policy 0, policy_version 1497313 (0.0010) [2023-12-27 02:17:50,053][105620] Updated weights for policy 1, policy_version 1499832 (0.0010) [2023-12-27 02:17:50,115][105620] Updated weights for policy 1, policy_version 1499842 (0.0010) [2023-12-27 02:17:50,174][105620] Updated weights for policy 1, policy_version 1499852 (0.0010) [2023-12-27 02:17:50,584][105692] Updated weights for policy 0, policy_version 1497323 (0.0010) [2023-12-27 02:17:50,642][105692] Updated weights for policy 0, policy_version 1497333 (0.0011) [2023-12-27 02:17:50,692][105692] Updated weights for policy 0, policy_version 1497343 (0.0010) [2023-12-27 02:17:50,860][105620] Updated weights for policy 1, policy_version 1499862 (0.0011) [2023-12-27 02:17:50,909][105620] Updated weights for policy 1, policy_version 1499872 (0.0010) [2023-12-27 02:17:50,958][105620] Updated weights for policy 1, policy_version 1499882 (0.0010) [2023-12-27 02:17:51,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 767401984. Throughput: 0: 9737.2, 1: 9757.6. Samples: 767387688. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:17:51,062][104569] Avg episode reward: [(0, '8624.099'), (1, '8896.362')] [2023-12-27 02:17:51,463][105692] Updated weights for policy 0, policy_version 1497353 (0.0011) [2023-12-27 02:17:51,535][105692] Updated weights for policy 0, policy_version 1497363 (0.0008) [2023-12-27 02:17:51,596][105692] Updated weights for policy 0, policy_version 1497373 (0.0005) [2023-12-27 02:17:51,658][105692] Updated weights for policy 0, policy_version 1497383 (0.0008) [2023-12-27 02:17:51,695][105620] Updated weights for policy 1, policy_version 1499892 (0.0009) [2023-12-27 02:17:51,763][105620] Updated weights for policy 1, policy_version 1499902 (0.0008) [2023-12-27 02:17:51,829][105620] Updated weights for policy 1, policy_version 1499912 (0.0008) [2023-12-27 02:17:52,340][105692] Updated weights for policy 0, policy_version 1497393 (0.0008) [2023-12-27 02:17:52,407][105692] Updated weights for policy 0, policy_version 1497403 (0.0009) [2023-12-27 02:17:52,440][105620] Updated weights for policy 1, policy_version 1499922 (0.0006) [2023-12-27 02:17:52,469][105692] Updated weights for policy 0, policy_version 1497413 (0.0010) [2023-12-27 02:17:52,499][105620] Updated weights for policy 1, policy_version 1499932 (0.0008) [2023-12-27 02:17:52,556][105620] Updated weights for policy 1, policy_version 1499942 (0.0008) [2023-12-27 02:17:52,620][105620] Updated weights for policy 1, policy_version 1499952 (0.0009) [2023-12-27 02:17:53,264][105620] Updated weights for policy 1, policy_version 1499962 (0.0008) [2023-12-27 02:17:53,285][105692] Updated weights for policy 0, policy_version 1497423 (0.0009) [2023-12-27 02:17:53,326][105620] Updated weights for policy 1, policy_version 1499972 (0.0009) [2023-12-27 02:17:53,336][105692] Updated weights for policy 0, policy_version 1497433 (0.0008) [2023-12-27 02:17:53,371][105620] Updated weights for policy 1, policy_version 1499982 (0.0006) [2023-12-27 02:17:53,385][105692] Updated weights for policy 0, policy_version 1497443 (0.0007) [2023-12-27 02:17:54,033][105620] Updated weights for policy 1, policy_version 1499992 (0.0005) [2023-12-27 02:17:54,083][105620] Updated weights for policy 1, policy_version 1500002 (0.0006) [2023-12-27 02:17:54,142][105620] Updated weights for policy 1, policy_version 1500012 (0.0005) [2023-12-27 02:17:54,202][105692] Updated weights for policy 0, policy_version 1497453 (0.0009) [2023-12-27 02:17:54,273][105692] Updated weights for policy 0, policy_version 1497463 (0.0010) [2023-12-27 02:17:54,338][105692] Updated weights for policy 0, policy_version 1497473 (0.0009) [2023-12-27 02:17:54,708][105620] Updated weights for policy 1, policy_version 1500022 (0.0009) [2023-12-27 02:17:54,757][105620] Updated weights for policy 1, policy_version 1500032 (0.0010) [2023-12-27 02:17:54,812][105620] Updated weights for policy 1, policy_version 1500042 (0.0010) [2023-12-27 02:17:55,056][105692] Updated weights for policy 0, policy_version 1497483 (0.0010) [2023-12-27 02:17:55,111][105692] Updated weights for policy 0, policy_version 1497493 (0.0010) [2023-12-27 02:17:55,166][105692] Updated weights for policy 0, policy_version 1497503 (0.0006) [2023-12-27 02:17:55,591][105620] Updated weights for policy 1, policy_version 1500052 (0.0010) [2023-12-27 02:17:55,649][105620] Updated weights for policy 1, policy_version 1500062 (0.0010) [2023-12-27 02:17:55,707][105620] Updated weights for policy 1, policy_version 1500072 (0.0010) [2023-12-27 02:17:55,793][105692] Updated weights for policy 0, policy_version 1497513 (0.0005) [2023-12-27 02:17:55,854][105692] Updated weights for policy 0, policy_version 1497523 (0.0005) [2023-12-27 02:17:55,922][105692] Updated weights for policy 0, policy_version 1497533 (0.0005) [2023-12-27 02:17:55,986][105692] Updated weights for policy 0, policy_version 1497543 (0.0009) [2023-12-27 02:17:56,062][104569] Fps is (10 sec: 19661.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 767500288. Throughput: 0: 9742.0, 1: 9740.5. Samples: 767504992. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:17:56,062][104569] Avg episode reward: [(0, '8638.687'), (1, '8897.664')] [2023-12-27 02:17:56,441][105620] Updated weights for policy 1, policy_version 1500082 (0.0009) [2023-12-27 02:17:56,500][105620] Updated weights for policy 1, policy_version 1500092 (0.0005) [2023-12-27 02:17:56,559][105692] Updated weights for policy 0, policy_version 1497553 (0.0010) [2023-12-27 02:17:56,561][105620] Updated weights for policy 1, policy_version 1500102 (0.0005) [2023-12-27 02:17:56,608][105692] Updated weights for policy 0, policy_version 1497563 (0.0006) [2023-12-27 02:17:56,622][105620] Updated weights for policy 1, policy_version 1500112 (0.0008) [2023-12-27 02:17:56,654][105692] Updated weights for policy 0, policy_version 1497573 (0.0005) [2023-12-27 02:17:57,218][105692] Updated weights for policy 0, policy_version 1497583 (0.0006) [2023-12-27 02:17:57,274][105692] Updated weights for policy 0, policy_version 1497593 (0.0006) [2023-12-27 02:17:57,319][105620] Updated weights for policy 1, policy_version 1500122 (0.0005) [2023-12-27 02:17:57,326][105692] Updated weights for policy 0, policy_version 1497603 (0.0009) [2023-12-27 02:17:57,369][105620] Updated weights for policy 1, policy_version 1500132 (0.0005) [2023-12-27 02:17:57,419][105620] Updated weights for policy 1, policy_version 1500142 (0.0005) [2023-12-27 02:17:57,958][105692] Updated weights for policy 0, policy_version 1497613 (0.0010) [2023-12-27 02:17:58,009][105692] Updated weights for policy 0, policy_version 1497623 (0.0010) [2023-12-27 02:17:58,070][105692] Updated weights for policy 0, policy_version 1497633 (0.0010) [2023-12-27 02:17:58,095][105620] Updated weights for policy 1, policy_version 1500152 (0.0007) [2023-12-27 02:17:58,150][105620] Updated weights for policy 1, policy_version 1500162 (0.0008) [2023-12-27 02:17:58,217][105620] Updated weights for policy 1, policy_version 1500172 (0.0009) [2023-12-27 02:17:58,848][105692] Updated weights for policy 0, policy_version 1497643 (0.0010) [2023-12-27 02:17:58,914][105692] Updated weights for policy 0, policy_version 1497653 (0.0008) [2023-12-27 02:17:58,986][105692] Updated weights for policy 0, policy_version 1497663 (0.0008) [2023-12-27 02:17:59,056][105620] Updated weights for policy 1, policy_version 1500182 (0.0007) [2023-12-27 02:17:59,117][105620] Updated weights for policy 1, policy_version 1500192 (0.0008) [2023-12-27 02:17:59,179][105620] Updated weights for policy 1, policy_version 1500202 (0.0008) [2023-12-27 02:17:59,681][105692] Updated weights for policy 0, policy_version 1497673 (0.0007) [2023-12-27 02:17:59,731][105692] Updated weights for policy 0, policy_version 1497683 (0.0009) [2023-12-27 02:17:59,782][105692] Updated weights for policy 0, policy_version 1497693 (0.0008) [2023-12-27 02:17:59,840][105692] Updated weights for policy 0, policy_version 1497703 (0.0009) [2023-12-27 02:17:59,944][105620] Updated weights for policy 1, policy_version 1500212 (0.0008) [2023-12-27 02:18:00,001][105620] Updated weights for policy 1, policy_version 1500222 (0.0009) [2023-12-27 02:18:00,056][105620] Updated weights for policy 1, policy_version 1500232 (0.0010) [2023-12-27 02:18:00,624][105692] Updated weights for policy 0, policy_version 1497713 (0.0006) [2023-12-27 02:18:00,664][105620] Updated weights for policy 1, policy_version 1500242 (0.0008) [2023-12-27 02:18:00,686][105692] Updated weights for policy 0, policy_version 1497723 (0.0007) [2023-12-27 02:18:00,739][105692] Updated weights for policy 0, policy_version 1497733 (0.0008) [2023-12-27 02:18:00,743][105620] Updated weights for policy 1, policy_version 1500252 (0.0005) [2023-12-27 02:18:00,805][105620] Updated weights for policy 1, policy_version 1500262 (0.0005) [2023-12-27 02:18:00,870][105620] Updated weights for policy 1, policy_version 1500272 (0.0005) [2023-12-27 02:18:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 767598592. Throughput: 0: 9842.3, 1: 9748.2. Samples: 767566720. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:18:01,062][104569] Avg episode reward: [(0, '8547.604'), (1, '9082.307')] [2023-12-27 02:18:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001497736_383475712.pth... [2023-12-27 02:18:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001500272_384122880.pth... [2023-12-27 02:18:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001499120_383827968.pth [2023-12-27 02:18:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001496616_383188992.pth [2023-12-27 02:18:01,425][105692] Updated weights for policy 0, policy_version 1497743 (0.0008) [2023-12-27 02:18:01,458][105620] Updated weights for policy 1, policy_version 1500282 (0.0009) [2023-12-27 02:18:01,481][105692] Updated weights for policy 0, policy_version 1497753 (0.0006) [2023-12-27 02:18:01,522][105620] Updated weights for policy 1, policy_version 1500292 (0.0009) [2023-12-27 02:18:01,537][105692] Updated weights for policy 0, policy_version 1497763 (0.0008) [2023-12-27 02:18:01,584][105620] Updated weights for policy 1, policy_version 1500302 (0.0009) [2023-12-27 02:18:02,202][105692] Updated weights for policy 0, policy_version 1497773 (0.0008) [2023-12-27 02:18:02,261][105692] Updated weights for policy 0, policy_version 1497783 (0.0009) [2023-12-27 02:18:02,328][105692] Updated weights for policy 0, policy_version 1497793 (0.0009) [2023-12-27 02:18:02,367][105620] Updated weights for policy 1, policy_version 1500312 (0.0007) [2023-12-27 02:18:02,434][105620] Updated weights for policy 1, policy_version 1500322 (0.0009) [2023-12-27 02:18:02,500][105620] Updated weights for policy 1, policy_version 1500332 (0.0009) [2023-12-27 02:18:02,967][105692] Updated weights for policy 0, policy_version 1497803 (0.0007) [2023-12-27 02:18:03,018][105692] Updated weights for policy 0, policy_version 1497813 (0.0005) [2023-12-27 02:18:03,064][105692] Updated weights for policy 0, policy_version 1497823 (0.0008) [2023-12-27 02:18:03,301][105620] Updated weights for policy 1, policy_version 1500342 (0.0008) [2023-12-27 02:18:03,358][105620] Updated weights for policy 1, policy_version 1500352 (0.0008) [2023-12-27 02:18:03,403][105620] Updated weights for policy 1, policy_version 1500362 (0.0008) [2023-12-27 02:18:03,788][105692] Updated weights for policy 0, policy_version 1497833 (0.0009) [2023-12-27 02:18:03,839][105692] Updated weights for policy 0, policy_version 1497843 (0.0008) [2023-12-27 02:18:03,901][105692] Updated weights for policy 0, policy_version 1497853 (0.0008) [2023-12-27 02:18:03,950][105692] Updated weights for policy 0, policy_version 1497863 (0.0009) [2023-12-27 02:18:04,069][105620] Updated weights for policy 1, policy_version 1500373 (0.0010) [2023-12-27 02:18:04,121][105620] Updated weights for policy 1, policy_version 1500383 (0.0010) [2023-12-27 02:18:04,170][105620] Updated weights for policy 1, policy_version 1500393 (0.0010) [2023-12-27 02:18:04,763][105692] Updated weights for policy 0, policy_version 1497873 (0.0009) [2023-12-27 02:18:04,825][105692] Updated weights for policy 0, policy_version 1497883 (0.0010) [2023-12-27 02:18:04,894][105692] Updated weights for policy 0, policy_version 1497893 (0.0009) [2023-12-27 02:18:04,920][105620] Updated weights for policy 1, policy_version 1500403 (0.0009) [2023-12-27 02:18:04,979][105620] Updated weights for policy 1, policy_version 1500413 (0.0007) [2023-12-27 02:18:05,037][105620] Updated weights for policy 1, policy_version 1500423 (0.0010) [2023-12-27 02:18:05,472][105692] Updated weights for policy 0, policy_version 1497903 (0.0008) [2023-12-27 02:18:05,516][105692] Updated weights for policy 0, policy_version 1497913 (0.0007) [2023-12-27 02:18:05,572][105692] Updated weights for policy 0, policy_version 1497923 (0.0008) [2023-12-27 02:18:05,760][105620] Updated weights for policy 1, policy_version 1500433 (0.0010) [2023-12-27 02:18:05,818][105620] Updated weights for policy 1, policy_version 1500443 (0.0007) [2023-12-27 02:18:05,871][105620] Updated weights for policy 1, policy_version 1500453 (0.0005) [2023-12-27 02:18:05,925][105620] Updated weights for policy 1, policy_version 1500463 (0.0007) [2023-12-27 02:18:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 767696896. Throughput: 0: 9796.5, 1: 9805.5. Samples: 767683332. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:18:06,063][104569] Avg episode reward: [(0, '7811.698'), (1, '9169.912')] [2023-12-27 02:18:06,359][105692] Updated weights for policy 0, policy_version 1497933 (0.0007) [2023-12-27 02:18:06,419][105692] Updated weights for policy 0, policy_version 1497943 (0.0008) [2023-12-27 02:18:06,469][105692] Updated weights for policy 0, policy_version 1497953 (0.0008) [2023-12-27 02:18:06,602][105620] Updated weights for policy 1, policy_version 1500473 (0.0010) [2023-12-27 02:18:06,663][105620] Updated weights for policy 1, policy_version 1500483 (0.0011) [2023-12-27 02:18:06,724][105620] Updated weights for policy 1, policy_version 1500493 (0.0011) [2023-12-27 02:18:07,257][105692] Updated weights for policy 0, policy_version 1497963 (0.0010) [2023-12-27 02:18:07,310][105692] Updated weights for policy 0, policy_version 1497973 (0.0011) [2023-12-27 02:18:07,318][105620] Updated weights for policy 1, policy_version 1500503 (0.0006) [2023-12-27 02:18:07,369][105692] Updated weights for policy 0, policy_version 1497983 (0.0011) [2023-12-27 02:18:07,373][105620] Updated weights for policy 1, policy_version 1500513 (0.0010) [2023-12-27 02:18:07,437][105620] Updated weights for policy 1, policy_version 1500523 (0.0011) [2023-12-27 02:18:08,054][105692] Updated weights for policy 0, policy_version 1497993 (0.0009) [2023-12-27 02:18:08,075][105620] Updated weights for policy 1, policy_version 1500533 (0.0008) [2023-12-27 02:18:08,112][105692] Updated weights for policy 0, policy_version 1498003 (0.0010) [2023-12-27 02:18:08,139][105620] Updated weights for policy 1, policy_version 1500543 (0.0008) [2023-12-27 02:18:08,168][105692] Updated weights for policy 0, policy_version 1498013 (0.0011) [2023-12-27 02:18:08,194][105620] Updated weights for policy 1, policy_version 1500553 (0.0010) [2023-12-27 02:18:08,219][105692] Updated weights for policy 0, policy_version 1498023 (0.0010) [2023-12-27 02:18:08,908][105620] Updated weights for policy 1, policy_version 1500563 (0.0010) [2023-12-27 02:18:08,965][105620] Updated weights for policy 1, policy_version 1500573 (0.0009) [2023-12-27 02:18:08,982][105692] Updated weights for policy 0, policy_version 1498033 (0.0008) [2023-12-27 02:18:09,021][105620] Updated weights for policy 1, policy_version 1500583 (0.0009) [2023-12-27 02:18:09,038][105692] Updated weights for policy 0, policy_version 1498043 (0.0008) [2023-12-27 02:18:09,095][105692] Updated weights for policy 0, policy_version 1498053 (0.0006) [2023-12-27 02:18:09,748][105620] Updated weights for policy 1, policy_version 1500593 (0.0010) [2023-12-27 02:18:09,806][105620] Updated weights for policy 1, policy_version 1500603 (0.0006) [2023-12-27 02:18:09,870][105620] Updated weights for policy 1, policy_version 1500613 (0.0007) [2023-12-27 02:18:09,873][105692] Updated weights for policy 0, policy_version 1498063 (0.0008) [2023-12-27 02:18:09,929][105692] Updated weights for policy 0, policy_version 1498073 (0.0009) [2023-12-27 02:18:09,936][105620] Updated weights for policy 1, policy_version 1500623 (0.0006) [2023-12-27 02:18:09,995][105692] Updated weights for policy 0, policy_version 1498083 (0.0009) [2023-12-27 02:18:10,631][105620] Updated weights for policy 1, policy_version 1500633 (0.0009) [2023-12-27 02:18:10,691][105620] Updated weights for policy 1, policy_version 1500643 (0.0008) [2023-12-27 02:18:10,750][105620] Updated weights for policy 1, policy_version 1500653 (0.0008) [2023-12-27 02:18:10,762][105692] Updated weights for policy 0, policy_version 1498093 (0.0009) [2023-12-27 02:18:10,820][105692] Updated weights for policy 0, policy_version 1498104 (0.0010) [2023-12-27 02:18:10,875][105692] Updated weights for policy 0, policy_version 1498115 (0.0010) [2023-12-27 02:18:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 767795200. Throughput: 0: 9738.7, 1: 9913.9. Samples: 767801320. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:18:11,062][104569] Avg episode reward: [(0, '8085.661'), (1, '9351.205')] [2023-12-27 02:18:11,404][105620] Updated weights for policy 1, policy_version 1500663 (0.0008) [2023-12-27 02:18:11,467][105620] Updated weights for policy 1, policy_version 1500673 (0.0005) [2023-12-27 02:18:11,528][105620] Updated weights for policy 1, policy_version 1500683 (0.0005) [2023-12-27 02:18:11,688][105692] Updated weights for policy 0, policy_version 1498125 (0.0009) [2023-12-27 02:18:11,750][105692] Updated weights for policy 0, policy_version 1498135 (0.0009) [2023-12-27 02:18:11,814][105692] Updated weights for policy 0, policy_version 1498145 (0.0010) [2023-12-27 02:18:12,227][105620] Updated weights for policy 1, policy_version 1500693 (0.0007) [2023-12-27 02:18:12,295][105620] Updated weights for policy 1, policy_version 1500703 (0.0009) [2023-12-27 02:18:12,358][105620] Updated weights for policy 1, policy_version 1500713 (0.0008) [2023-12-27 02:18:12,608][105692] Updated weights for policy 0, policy_version 1498155 (0.0009) [2023-12-27 02:18:12,666][105692] Updated weights for policy 0, policy_version 1498165 (0.0010) [2023-12-27 02:18:12,727][105692] Updated weights for policy 0, policy_version 1498175 (0.0009) [2023-12-27 02:18:13,073][105620] Updated weights for policy 1, policy_version 1500723 (0.0009) [2023-12-27 02:18:13,122][105620] Updated weights for policy 1, policy_version 1500733 (0.0009) [2023-12-27 02:18:13,174][105620] Updated weights for policy 1, policy_version 1500743 (0.0009) [2023-12-27 02:18:13,447][105692] Updated weights for policy 0, policy_version 1498185 (0.0009) [2023-12-27 02:18:13,515][105692] Updated weights for policy 0, policy_version 1498195 (0.0008) [2023-12-27 02:18:13,566][105692] Updated weights for policy 0, policy_version 1498205 (0.0008) [2023-12-27 02:18:13,620][105692] Updated weights for policy 0, policy_version 1498215 (0.0005) [2023-12-27 02:18:13,977][105620] Updated weights for policy 1, policy_version 1500753 (0.0010) [2023-12-27 02:18:14,029][105620] Updated weights for policy 1, policy_version 1500763 (0.0011) [2023-12-27 02:18:14,087][105620] Updated weights for policy 1, policy_version 1500773 (0.0010) [2023-12-27 02:18:14,149][105620] Updated weights for policy 1, policy_version 1500783 (0.0010) [2023-12-27 02:18:14,350][105692] Updated weights for policy 0, policy_version 1498225 (0.0006) [2023-12-27 02:18:14,411][105692] Updated weights for policy 0, policy_version 1498235 (0.0008) [2023-12-27 02:18:14,470][105692] Updated weights for policy 0, policy_version 1498245 (0.0006) [2023-12-27 02:18:14,860][105620] Updated weights for policy 1, policy_version 1500793 (0.0006) [2023-12-27 02:18:14,929][105620] Updated weights for policy 1, policy_version 1500803 (0.0006) [2023-12-27 02:18:14,999][105620] Updated weights for policy 1, policy_version 1500813 (0.0009) [2023-12-27 02:18:15,048][105692] Updated weights for policy 0, policy_version 1498255 (0.0006) [2023-12-27 02:18:15,118][105692] Updated weights for policy 0, policy_version 1498265 (0.0007) [2023-12-27 02:18:15,186][105692] Updated weights for policy 0, policy_version 1498275 (0.0010) [2023-12-27 02:18:15,604][105620] Updated weights for policy 1, policy_version 1500823 (0.0010) [2023-12-27 02:18:15,665][105620] Updated weights for policy 1, policy_version 1500833 (0.0010) [2023-12-27 02:18:15,727][105620] Updated weights for policy 1, policy_version 1500843 (0.0010) [2023-12-27 02:18:15,901][105692] Updated weights for policy 0, policy_version 1498285 (0.0009) [2023-12-27 02:18:15,953][105692] Updated weights for policy 0, policy_version 1498295 (0.0008) [2023-12-27 02:18:16,009][105692] Updated weights for policy 0, policy_version 1498305 (0.0008) [2023-12-27 02:18:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 767893504. Throughput: 0: 9636.2, 1: 9871.3. Samples: 767857008. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:18:16,063][104569] Avg episode reward: [(0, '8455.675'), (1, '9172.803')] [2023-12-27 02:18:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001498312_383623168.pth... [2023-12-27 02:18:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001500848_384270336.pth... [2023-12-27 02:18:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001499696_383975424.pth [2023-12-27 02:18:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001497160_383328256.pth [2023-12-27 02:18:16,442][105620] Updated weights for policy 1, policy_version 1500853 (0.0011) [2023-12-27 02:18:16,505][105620] Updated weights for policy 1, policy_version 1500863 (0.0011) [2023-12-27 02:18:16,570][105620] Updated weights for policy 1, policy_version 1500873 (0.0010) [2023-12-27 02:18:16,788][105692] Updated weights for policy 0, policy_version 1498315 (0.0008) [2023-12-27 02:18:16,841][105692] Updated weights for policy 0, policy_version 1498325 (0.0008) [2023-12-27 02:18:16,890][105692] Updated weights for policy 0, policy_version 1498335 (0.0008) [2023-12-27 02:18:17,311][105620] Updated weights for policy 1, policy_version 1500883 (0.0011) [2023-12-27 02:18:17,377][105620] Updated weights for policy 1, policy_version 1500893 (0.0011) [2023-12-27 02:18:17,439][105620] Updated weights for policy 1, policy_version 1500903 (0.0010) [2023-12-27 02:18:17,655][105692] Updated weights for policy 0, policy_version 1498345 (0.0008) [2023-12-27 02:18:17,715][105692] Updated weights for policy 0, policy_version 1498355 (0.0006) [2023-12-27 02:18:17,772][105692] Updated weights for policy 0, policy_version 1498365 (0.0008) [2023-12-27 02:18:17,826][105692] Updated weights for policy 0, policy_version 1498375 (0.0007) [2023-12-27 02:18:18,147][105620] Updated weights for policy 1, policy_version 1500913 (0.0010) [2023-12-27 02:18:18,202][105620] Updated weights for policy 1, policy_version 1500923 (0.0005) [2023-12-27 02:18:18,257][105620] Updated weights for policy 1, policy_version 1500933 (0.0011) [2023-12-27 02:18:18,306][105620] Updated weights for policy 1, policy_version 1500943 (0.0011) [2023-12-27 02:18:18,566][105692] Updated weights for policy 0, policy_version 1498385 (0.0010) [2023-12-27 02:18:18,625][105692] Updated weights for policy 0, policy_version 1498395 (0.0011) [2023-12-27 02:18:18,684][105692] Updated weights for policy 0, policy_version 1498405 (0.0011) [2023-12-27 02:18:18,882][105620] Updated weights for policy 1, policy_version 1500953 (0.0006) [2023-12-27 02:18:18,949][105620] Updated weights for policy 1, policy_version 1500963 (0.0008) [2023-12-27 02:18:19,016][105620] Updated weights for policy 1, policy_version 1500973 (0.0006) [2023-12-27 02:18:19,360][105692] Updated weights for policy 0, policy_version 1498415 (0.0009) [2023-12-27 02:18:19,420][105692] Updated weights for policy 0, policy_version 1498425 (0.0007) [2023-12-27 02:18:19,480][105692] Updated weights for policy 0, policy_version 1498435 (0.0008) [2023-12-27 02:18:19,705][105620] Updated weights for policy 1, policy_version 1500983 (0.0006) [2023-12-27 02:18:19,770][105620] Updated weights for policy 1, policy_version 1500993 (0.0006) [2023-12-27 02:18:19,841][105620] Updated weights for policy 1, policy_version 1501003 (0.0010) [2023-12-27 02:18:20,265][105692] Updated weights for policy 0, policy_version 1498445 (0.0009) [2023-12-27 02:18:20,331][105692] Updated weights for policy 0, policy_version 1498455 (0.0009) [2023-12-27 02:18:20,397][105692] Updated weights for policy 0, policy_version 1498465 (0.0009) [2023-12-27 02:18:20,550][105620] Updated weights for policy 1, policy_version 1501013 (0.0007) [2023-12-27 02:18:20,617][105620] Updated weights for policy 1, policy_version 1501023 (0.0009) [2023-12-27 02:18:20,668][105620] Updated weights for policy 1, policy_version 1501033 (0.0008) [2023-12-27 02:18:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 767983616. Throughput: 0: 9631.7, 1: 9924.4. Samples: 767975360. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:18:21,063][104569] Avg episode reward: [(0, '8086.231'), (1, '8991.726')] [2023-12-27 02:18:21,141][105692] Updated weights for policy 0, policy_version 1498475 (0.0009) [2023-12-27 02:18:21,208][105692] Updated weights for policy 0, policy_version 1498485 (0.0010) [2023-12-27 02:18:21,271][105692] Updated weights for policy 0, policy_version 1498495 (0.0008) [2023-12-27 02:18:21,446][105620] Updated weights for policy 1, policy_version 1501043 (0.0009) [2023-12-27 02:18:21,506][105620] Updated weights for policy 1, policy_version 1501053 (0.0008) [2023-12-27 02:18:21,566][105620] Updated weights for policy 1, policy_version 1501063 (0.0008) [2023-12-27 02:18:22,097][105692] Updated weights for policy 0, policy_version 1498505 (0.0008) [2023-12-27 02:18:22,148][105692] Updated weights for policy 0, policy_version 1498515 (0.0005) [2023-12-27 02:18:22,213][105692] Updated weights for policy 0, policy_version 1498525 (0.0007) [2023-12-27 02:18:22,285][105620] Updated weights for policy 1, policy_version 1501073 (0.0009) [2023-12-27 02:18:22,287][105692] Updated weights for policy 0, policy_version 1498535 (0.0008) [2023-12-27 02:18:22,350][105620] Updated weights for policy 1, policy_version 1501083 (0.0009) [2023-12-27 02:18:22,416][105620] Updated weights for policy 1, policy_version 1501093 (0.0007) [2023-12-27 02:18:22,472][105620] Updated weights for policy 1, policy_version 1501103 (0.0009) [2023-12-27 02:18:23,000][105692] Updated weights for policy 0, policy_version 1498545 (0.0009) [2023-12-27 02:18:23,055][105692] Updated weights for policy 0, policy_version 1498555 (0.0009) [2023-12-27 02:18:23,112][105692] Updated weights for policy 0, policy_version 1498565 (0.0009) [2023-12-27 02:18:23,183][105620] Updated weights for policy 1, policy_version 1501113 (0.0011) [2023-12-27 02:18:23,239][105620] Updated weights for policy 1, policy_version 1501123 (0.0007) [2023-12-27 02:18:23,295][105620] Updated weights for policy 1, policy_version 1501133 (0.0005) [2023-12-27 02:18:23,825][105692] Updated weights for policy 0, policy_version 1498575 (0.0010) [2023-12-27 02:18:23,836][105620] Updated weights for policy 1, policy_version 1501143 (0.0005) [2023-12-27 02:18:23,879][105692] Updated weights for policy 0, policy_version 1498585 (0.0010) [2023-12-27 02:18:23,894][105620] Updated weights for policy 1, policy_version 1501153 (0.0005) [2023-12-27 02:18:23,931][105692] Updated weights for policy 0, policy_version 1498595 (0.0010) [2023-12-27 02:18:23,944][105620] Updated weights for policy 1, policy_version 1501163 (0.0005) [2023-12-27 02:18:24,508][105620] Updated weights for policy 1, policy_version 1501173 (0.0008) [2023-12-27 02:18:24,569][105620] Updated weights for policy 1, policy_version 1501183 (0.0010) [2023-12-27 02:18:24,635][105620] Updated weights for policy 1, policy_version 1501193 (0.0009) [2023-12-27 02:18:24,657][105692] Updated weights for policy 0, policy_version 1498605 (0.0008) [2023-12-27 02:18:24,706][105692] Updated weights for policy 0, policy_version 1498615 (0.0009) [2023-12-27 02:18:24,764][105692] Updated weights for policy 0, policy_version 1498625 (0.0010) [2023-12-27 02:18:25,302][105620] Updated weights for policy 1, policy_version 1501203 (0.0009) [2023-12-27 02:18:25,358][105620] Updated weights for policy 1, policy_version 1501213 (0.0006) [2023-12-27 02:18:25,422][105620] Updated weights for policy 1, policy_version 1501223 (0.0010) [2023-12-27 02:18:25,457][105692] Updated weights for policy 0, policy_version 1498635 (0.0008) [2023-12-27 02:18:25,504][105692] Updated weights for policy 0, policy_version 1498645 (0.0010) [2023-12-27 02:18:25,553][105692] Updated weights for policy 0, policy_version 1498655 (0.0010) [2023-12-27 02:18:26,056][105620] Updated weights for policy 1, policy_version 1501233 (0.0006) [2023-12-27 02:18:26,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 768081920. Throughput: 0: 9556.9, 1: 9951.4. Samples: 768093420. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:18:26,062][104569] Avg episode reward: [(0, '7991.051'), (1, '8817.860')] [2023-12-27 02:18:26,109][105620] Updated weights for policy 1, policy_version 1501243 (0.0008) [2023-12-27 02:18:26,159][105620] Updated weights for policy 1, policy_version 1501253 (0.0007) [2023-12-27 02:18:26,216][105620] Updated weights for policy 1, policy_version 1501263 (0.0006) [2023-12-27 02:18:26,309][105692] Updated weights for policy 0, policy_version 1498665 (0.0010) [2023-12-27 02:18:26,377][105692] Updated weights for policy 0, policy_version 1498675 (0.0010) [2023-12-27 02:18:26,441][105692] Updated weights for policy 0, policy_version 1498685 (0.0010) [2023-12-27 02:18:26,489][105692] Updated weights for policy 0, policy_version 1498695 (0.0010) [2023-12-27 02:18:26,802][105620] Updated weights for policy 1, policy_version 1501273 (0.0006) [2023-12-27 02:18:26,851][105620] Updated weights for policy 1, policy_version 1501283 (0.0008) [2023-12-27 02:18:26,896][105620] Updated weights for policy 1, policy_version 1501293 (0.0008) [2023-12-27 02:18:27,191][105692] Updated weights for policy 0, policy_version 1498705 (0.0010) [2023-12-27 02:18:27,255][105692] Updated weights for policy 0, policy_version 1498715 (0.0010) [2023-12-27 02:18:27,315][105692] Updated weights for policy 0, policy_version 1498725 (0.0010) [2023-12-27 02:18:27,523][105620] Updated weights for policy 1, policy_version 1501303 (0.0006) [2023-12-27 02:18:27,582][105620] Updated weights for policy 1, policy_version 1501313 (0.0005) [2023-12-27 02:18:27,639][105620] Updated weights for policy 1, policy_version 1501323 (0.0005) [2023-12-27 02:18:28,044][105692] Updated weights for policy 0, policy_version 1498735 (0.0010) [2023-12-27 02:18:28,088][105692] Updated weights for policy 0, policy_version 1498745 (0.0010) [2023-12-27 02:18:28,114][105585] KL-divergence is very high: 138.5806 [2023-12-27 02:18:28,128][105620] Updated weights for policy 1, policy_version 1501333 (0.0005) [2023-12-27 02:18:28,142][105692] Updated weights for policy 0, policy_version 1498755 (0.0010) [2023-12-27 02:18:28,160][105585] KL-divergence is very high: 169.4123 [2023-12-27 02:18:28,179][105620] Updated weights for policy 1, policy_version 1501343 (0.0005) [2023-12-27 02:18:28,227][105620] Updated weights for policy 1, policy_version 1501353 (0.0005) [2023-12-27 02:18:28,835][105620] Updated weights for policy 1, policy_version 1501363 (0.0007) [2023-12-27 02:18:28,894][105692] Updated weights for policy 0, policy_version 1498765 (0.0010) [2023-12-27 02:18:28,898][105620] Updated weights for policy 1, policy_version 1501373 (0.0009) [2023-12-27 02:18:28,946][105692] Updated weights for policy 0, policy_version 1498775 (0.0010) [2023-12-27 02:18:28,961][105620] Updated weights for policy 1, policy_version 1501383 (0.0011) [2023-12-27 02:18:29,001][105692] Updated weights for policy 0, policy_version 1498785 (0.0010) [2023-12-27 02:18:29,687][105620] Updated weights for policy 1, policy_version 1501393 (0.0011) [2023-12-27 02:18:29,736][105692] Updated weights for policy 0, policy_version 1498795 (0.0011) [2023-12-27 02:18:29,749][105620] Updated weights for policy 1, policy_version 1501403 (0.0010) [2023-12-27 02:18:29,795][105692] Updated weights for policy 0, policy_version 1498805 (0.0010) [2023-12-27 02:18:29,797][105620] Updated weights for policy 1, policy_version 1501413 (0.0010) [2023-12-27 02:18:29,857][105620] Updated weights for policy 1, policy_version 1501423 (0.0011) [2023-12-27 02:18:29,857][105692] Updated weights for policy 0, policy_version 1498815 (0.0009) [2023-12-27 02:18:30,534][105692] Updated weights for policy 0, policy_version 1498825 (0.0008) [2023-12-27 02:18:30,579][105620] Updated weights for policy 1, policy_version 1501433 (0.0006) [2023-12-27 02:18:30,582][105692] Updated weights for policy 0, policy_version 1498835 (0.0008) [2023-12-27 02:18:30,634][105620] Updated weights for policy 1, policy_version 1501443 (0.0005) [2023-12-27 02:18:30,638][105692] Updated weights for policy 0, policy_version 1498845 (0.0005) [2023-12-27 02:18:30,690][105620] Updated weights for policy 1, policy_version 1501453 (0.0005) [2023-12-27 02:18:30,694][105692] Updated weights for policy 0, policy_version 1498855 (0.0009) [2023-12-27 02:18:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 768188416. Throughput: 0: 9545.3, 1: 10076.3. Samples: 768157080. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:18:31,062][104569] Avg episode reward: [(0, '8082.190'), (1, '8999.167')] [2023-12-27 02:18:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001498856_383762432.pth... [2023-12-27 02:18:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001501456_384425984.pth... [2023-12-27 02:18:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001500272_384122880.pth [2023-12-27 02:18:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001497736_383475712.pth [2023-12-27 02:18:31,267][105620] Updated weights for policy 1, policy_version 1501463 (0.0007) [2023-12-27 02:18:31,323][105620] Updated weights for policy 1, policy_version 1501473 (0.0011) [2023-12-27 02:18:31,331][105692] Updated weights for policy 0, policy_version 1498865 (0.0011) [2023-12-27 02:18:31,386][105620] Updated weights for policy 1, policy_version 1501483 (0.0009) [2023-12-27 02:18:31,392][105692] Updated weights for policy 0, policy_version 1498875 (0.0008) [2023-12-27 02:18:31,448][105692] Updated weights for policy 0, policy_version 1498885 (0.0008) [2023-12-27 02:18:32,045][105620] Updated weights for policy 1, policy_version 1501493 (0.0008) [2023-12-27 02:18:32,110][105620] Updated weights for policy 1, policy_version 1501503 (0.0006) [2023-12-27 02:18:32,175][105620] Updated weights for policy 1, policy_version 1501513 (0.0006) [2023-12-27 02:18:32,225][105692] Updated weights for policy 0, policy_version 1498895 (0.0006) [2023-12-27 02:18:32,284][105692] Updated weights for policy 0, policy_version 1498905 (0.0007) [2023-12-27 02:18:32,349][105692] Updated weights for policy 0, policy_version 1498915 (0.0009) [2023-12-27 02:18:32,812][105620] Updated weights for policy 1, policy_version 1501523 (0.0009) [2023-12-27 02:18:32,858][105620] Updated weights for policy 1, policy_version 1501533 (0.0007) [2023-12-27 02:18:32,901][105620] Updated weights for policy 1, policy_version 1501543 (0.0005) [2023-12-27 02:18:33,089][105692] Updated weights for policy 0, policy_version 1498925 (0.0009) [2023-12-27 02:18:33,145][105692] Updated weights for policy 0, policy_version 1498935 (0.0009) [2023-12-27 02:18:33,202][105692] Updated weights for policy 0, policy_version 1498945 (0.0009) [2023-12-27 02:18:33,495][105620] Updated weights for policy 1, policy_version 1501553 (0.0005) [2023-12-27 02:18:33,560][105620] Updated weights for policy 1, policy_version 1501563 (0.0005) [2023-12-27 02:18:33,622][105620] Updated weights for policy 1, policy_version 1501573 (0.0006) [2023-12-27 02:18:33,675][105620] Updated weights for policy 1, policy_version 1501583 (0.0005) [2023-12-27 02:18:33,889][105692] Updated weights for policy 0, policy_version 1498955 (0.0008) [2023-12-27 02:18:33,938][105692] Updated weights for policy 0, policy_version 1498965 (0.0011) [2023-12-27 02:18:33,983][105692] Updated weights for policy 0, policy_version 1498975 (0.0011) [2023-12-27 02:18:34,336][105620] Updated weights for policy 1, policy_version 1501593 (0.0007) [2023-12-27 02:18:34,399][105620] Updated weights for policy 1, policy_version 1501603 (0.0010) [2023-12-27 02:18:34,462][105620] Updated weights for policy 1, policy_version 1501613 (0.0006) [2023-12-27 02:18:34,786][105692] Updated weights for policy 0, policy_version 1498985 (0.0010) [2023-12-27 02:18:34,841][105692] Updated weights for policy 0, policy_version 1498995 (0.0010) [2023-12-27 02:18:34,899][105692] Updated weights for policy 0, policy_version 1499005 (0.0011) [2023-12-27 02:18:34,965][105692] Updated weights for policy 0, policy_version 1499015 (0.0010) [2023-12-27 02:18:35,119][105620] Updated weights for policy 1, policy_version 1501623 (0.0009) [2023-12-27 02:18:35,183][105620] Updated weights for policy 1, policy_version 1501633 (0.0010) [2023-12-27 02:18:35,247][105620] Updated weights for policy 1, policy_version 1501643 (0.0010) [2023-12-27 02:18:35,692][105692] Updated weights for policy 0, policy_version 1499025 (0.0010) [2023-12-27 02:18:35,736][105692] Updated weights for policy 0, policy_version 1499035 (0.0010) [2023-12-27 02:18:35,780][105692] Updated weights for policy 0, policy_version 1499045 (0.0010) [2023-12-27 02:18:35,920][105620] Updated weights for policy 1, policy_version 1501653 (0.0010) [2023-12-27 02:18:35,982][105620] Updated weights for policy 1, policy_version 1501663 (0.0010) [2023-12-27 02:18:36,033][105620] Updated weights for policy 1, policy_version 1501673 (0.0010) [2023-12-27 02:18:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 768286720. Throughput: 0: 9628.6, 1: 10163.7. Samples: 768278344. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:18:36,062][104569] Avg episode reward: [(0, '7900.650'), (1, '9172.371')] [2023-12-27 02:18:36,566][105692] Updated weights for policy 0, policy_version 1499055 (0.0011) [2023-12-27 02:18:36,621][105692] Updated weights for policy 0, policy_version 1499065 (0.0011) [2023-12-27 02:18:36,680][105692] Updated weights for policy 0, policy_version 1499075 (0.0010) [2023-12-27 02:18:36,756][105620] Updated weights for policy 1, policy_version 1501683 (0.0010) [2023-12-27 02:18:36,803][105620] Updated weights for policy 1, policy_version 1501693 (0.0006) [2023-12-27 02:18:36,850][105620] Updated weights for policy 1, policy_version 1501703 (0.0005) [2023-12-27 02:18:37,376][105692] Updated weights for policy 0, policy_version 1499085 (0.0010) [2023-12-27 02:18:37,429][105692] Updated weights for policy 0, policy_version 1499095 (0.0010) [2023-12-27 02:18:37,432][105620] Updated weights for policy 1, policy_version 1501713 (0.0006) [2023-12-27 02:18:37,478][105692] Updated weights for policy 0, policy_version 1499105 (0.0011) [2023-12-27 02:18:37,482][105620] Updated weights for policy 1, policy_version 1501723 (0.0011) [2023-12-27 02:18:37,534][105620] Updated weights for policy 1, policy_version 1501733 (0.0011) [2023-12-27 02:18:37,584][105620] Updated weights for policy 1, policy_version 1501743 (0.0010) [2023-12-27 02:18:38,255][105692] Updated weights for policy 0, policy_version 1499115 (0.0009) [2023-12-27 02:18:38,307][105692] Updated weights for policy 0, policy_version 1499125 (0.0005) [2023-12-27 02:18:38,368][105692] Updated weights for policy 0, policy_version 1499135 (0.0009) [2023-12-27 02:18:38,370][105620] Updated weights for policy 1, policy_version 1501753 (0.0011) [2023-12-27 02:18:38,434][105620] Updated weights for policy 1, policy_version 1501763 (0.0008) [2023-12-27 02:18:38,499][105620] Updated weights for policy 1, policy_version 1501773 (0.0006) [2023-12-27 02:18:39,034][105692] Updated weights for policy 0, policy_version 1499145 (0.0010) [2023-12-27 02:18:39,083][105692] Updated weights for policy 0, policy_version 1499155 (0.0006) [2023-12-27 02:18:39,090][105620] Updated weights for policy 1, policy_version 1501783 (0.0005) [2023-12-27 02:18:39,133][105692] Updated weights for policy 0, policy_version 1499165 (0.0008) [2023-12-27 02:18:39,142][105620] Updated weights for policy 1, policy_version 1501793 (0.0006) [2023-12-27 02:18:39,192][105692] Updated weights for policy 0, policy_version 1499175 (0.0006) [2023-12-27 02:18:39,203][105620] Updated weights for policy 1, policy_version 1501803 (0.0009) [2023-12-27 02:18:39,892][105692] Updated weights for policy 0, policy_version 1499185 (0.0009) [2023-12-27 02:18:39,957][105692] Updated weights for policy 0, policy_version 1499195 (0.0008) [2023-12-27 02:18:40,017][105620] Updated weights for policy 1, policy_version 1501813 (0.0008) [2023-12-27 02:18:40,023][105692] Updated weights for policy 0, policy_version 1499205 (0.0008) [2023-12-27 02:18:40,079][105620] Updated weights for policy 1, policy_version 1501823 (0.0008) [2023-12-27 02:18:40,146][105620] Updated weights for policy 1, policy_version 1501833 (0.0009) [2023-12-27 02:18:40,711][105692] Updated weights for policy 0, policy_version 1499215 (0.0007) [2023-12-27 02:18:40,768][105692] Updated weights for policy 0, policy_version 1499225 (0.0010) [2023-12-27 02:18:40,820][105692] Updated weights for policy 0, policy_version 1499235 (0.0009) [2023-12-27 02:18:40,892][105620] Updated weights for policy 1, policy_version 1501843 (0.0009) [2023-12-27 02:18:40,955][105620] Updated weights for policy 1, policy_version 1501853 (0.0008) [2023-12-27 02:18:41,015][105620] Updated weights for policy 1, policy_version 1501863 (0.0009) [2023-12-27 02:18:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 768385024. Throughput: 0: 9682.1, 1: 10115.3. Samples: 768395876. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 02:18:41,062][104569] Avg episode reward: [(0, '8359.407'), (1, '9001.881')] [2023-12-27 02:18:41,573][105692] Updated weights for policy 0, policy_version 1499245 (0.0009) [2023-12-27 02:18:41,631][105692] Updated weights for policy 0, policy_version 1499255 (0.0009) [2023-12-27 02:18:41,698][105692] Updated weights for policy 0, policy_version 1499265 (0.0009) [2023-12-27 02:18:41,835][105620] Updated weights for policy 1, policy_version 1501873 (0.0007) [2023-12-27 02:18:41,887][105620] Updated weights for policy 1, policy_version 1501883 (0.0009) [2023-12-27 02:18:41,941][105620] Updated weights for policy 1, policy_version 1501893 (0.0008) [2023-12-27 02:18:42,007][105620] Updated weights for policy 1, policy_version 1501903 (0.0009) [2023-12-27 02:18:42,425][105692] Updated weights for policy 0, policy_version 1499275 (0.0009) [2023-12-27 02:18:42,484][105692] Updated weights for policy 0, policy_version 1499285 (0.0009) [2023-12-27 02:18:42,531][105692] Updated weights for policy 0, policy_version 1499295 (0.0009) [2023-12-27 02:18:42,760][105620] Updated weights for policy 1, policy_version 1501913 (0.0009) [2023-12-27 02:18:42,816][105620] Updated weights for policy 1, policy_version 1501923 (0.0007) [2023-12-27 02:18:42,878][105620] Updated weights for policy 1, policy_version 1501933 (0.0005) [2023-12-27 02:18:43,290][105692] Updated weights for policy 0, policy_version 1499305 (0.0010) [2023-12-27 02:18:43,344][105692] Updated weights for policy 0, policy_version 1499315 (0.0009) [2023-12-27 02:18:43,391][105692] Updated weights for policy 0, policy_version 1499325 (0.0009) [2023-12-27 02:18:43,438][105692] Updated weights for policy 0, policy_version 1499335 (0.0009) [2023-12-27 02:18:43,582][105620] Updated weights for policy 1, policy_version 1501943 (0.0008) [2023-12-27 02:18:43,629][105620] Updated weights for policy 1, policy_version 1501953 (0.0008) [2023-12-27 02:18:43,675][105620] Updated weights for policy 1, policy_version 1501963 (0.0009) [2023-12-27 02:18:44,218][105692] Updated weights for policy 0, policy_version 1499345 (0.0008) [2023-12-27 02:18:44,284][105692] Updated weights for policy 0, policy_version 1499355 (0.0009) [2023-12-27 02:18:44,343][105692] Updated weights for policy 0, policy_version 1499365 (0.0009) [2023-12-27 02:18:44,437][105620] Updated weights for policy 1, policy_version 1501973 (0.0008) [2023-12-27 02:18:44,488][105620] Updated weights for policy 1, policy_version 1501983 (0.0008) [2023-12-27 02:18:44,541][105620] Updated weights for policy 1, policy_version 1501993 (0.0009) [2023-12-27 02:18:44,970][105692] Updated weights for policy 0, policy_version 1499375 (0.0007) [2023-12-27 02:18:45,033][105692] Updated weights for policy 0, policy_version 1499385 (0.0007) [2023-12-27 02:18:45,093][105692] Updated weights for policy 0, policy_version 1499395 (0.0008) [2023-12-27 02:18:45,345][105620] Updated weights for policy 1, policy_version 1502004 (0.0009) [2023-12-27 02:18:45,393][105620] Updated weights for policy 1, policy_version 1502014 (0.0009) [2023-12-27 02:18:45,441][105620] Updated weights for policy 1, policy_version 1502024 (0.0009) [2023-12-27 02:18:45,822][105692] Updated weights for policy 0, policy_version 1499405 (0.0009) [2023-12-27 02:18:45,879][105692] Updated weights for policy 0, policy_version 1499415 (0.0008) [2023-12-27 02:18:45,937][105692] Updated weights for policy 0, policy_version 1499425 (0.0010) [2023-12-27 02:18:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.9, 300 sec: 19521.9). Total num frames: 768483328. Throughput: 0: 9559.3, 1: 10099.2. Samples: 768451352. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:18:46,062][104569] Avg episode reward: [(0, '8267.133'), (1, '9001.730')] [2023-12-27 02:18:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001502032_384573440.pth... [2023-12-27 02:18:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001499432_383909888.pth... [2023-12-27 02:18:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001500848_384270336.pth [2023-12-27 02:18:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001498312_383623168.pth [2023-12-27 02:18:46,249][105620] Updated weights for policy 1, policy_version 1502034 (0.0009) [2023-12-27 02:18:46,307][105620] Updated weights for policy 1, policy_version 1502045 (0.0010) [2023-12-27 02:18:46,378][105620] Updated weights for policy 1, policy_version 1502055 (0.0010) [2023-12-27 02:18:46,508][105692] Updated weights for policy 0, policy_version 1499435 (0.0009) [2023-12-27 02:18:46,575][105692] Updated weights for policy 0, policy_version 1499445 (0.0007) [2023-12-27 02:18:46,640][105692] Updated weights for policy 0, policy_version 1499455 (0.0011) [2023-12-27 02:18:47,194][105620] Updated weights for policy 1, policy_version 1502065 (0.0010) [2023-12-27 02:18:47,248][105620] Updated weights for policy 1, policy_version 1502075 (0.0010) [2023-12-27 02:18:47,286][105692] Updated weights for policy 0, policy_version 1499465 (0.0010) [2023-12-27 02:18:47,299][105620] Updated weights for policy 1, policy_version 1502086 (0.0009) [2023-12-27 02:18:47,344][105692] Updated weights for policy 0, policy_version 1499475 (0.0009) [2023-12-27 02:18:47,354][105620] Updated weights for policy 1, policy_version 1502096 (0.0006) [2023-12-27 02:18:47,399][105692] Updated weights for policy 0, policy_version 1499485 (0.0010) [2023-12-27 02:18:47,454][105692] Updated weights for policy 0, policy_version 1499495 (0.0010) [2023-12-27 02:18:48,092][105692] Updated weights for policy 0, policy_version 1499505 (0.0009) [2023-12-27 02:18:48,153][105692] Updated weights for policy 0, policy_version 1499515 (0.0010) [2023-12-27 02:18:48,214][105692] Updated weights for policy 0, policy_version 1499525 (0.0010) [2023-12-27 02:18:48,228][105620] Updated weights for policy 1, policy_version 1502106 (0.0006) [2023-12-27 02:18:48,287][105620] Updated weights for policy 1, policy_version 1502116 (0.0008) [2023-12-27 02:18:48,353][105620] Updated weights for policy 1, policy_version 1502126 (0.0008) [2023-12-27 02:18:48,891][105692] Updated weights for policy 0, policy_version 1499535 (0.0011) [2023-12-27 02:18:48,945][105692] Updated weights for policy 0, policy_version 1499545 (0.0010) [2023-12-27 02:18:49,010][105692] Updated weights for policy 0, policy_version 1499555 (0.0010) [2023-12-27 02:18:49,106][105620] Updated weights for policy 1, policy_version 1502136 (0.0009) [2023-12-27 02:18:49,155][105620] Updated weights for policy 1, policy_version 1502146 (0.0008) [2023-12-27 02:18:49,204][105620] Updated weights for policy 1, policy_version 1502156 (0.0008) [2023-12-27 02:18:49,775][105692] Updated weights for policy 0, policy_version 1499565 (0.0010) [2023-12-27 02:18:49,835][105692] Updated weights for policy 0, policy_version 1499575 (0.0009) [2023-12-27 02:18:49,902][105692] Updated weights for policy 0, policy_version 1499585 (0.0009) [2023-12-27 02:18:49,960][105620] Updated weights for policy 1, policy_version 1502166 (0.0008) [2023-12-27 02:18:50,023][105620] Updated weights for policy 1, policy_version 1502176 (0.0009) [2023-12-27 02:18:50,085][105620] Updated weights for policy 1, policy_version 1502186 (0.0009) [2023-12-27 02:18:50,656][105692] Updated weights for policy 0, policy_version 1499595 (0.0008) [2023-12-27 02:18:50,716][105692] Updated weights for policy 0, policy_version 1499605 (0.0009) [2023-12-27 02:18:50,775][105692] Updated weights for policy 0, policy_version 1499615 (0.0009) [2023-12-27 02:18:50,867][105620] Updated weights for policy 1, policy_version 1502196 (0.0008) [2023-12-27 02:18:50,932][105620] Updated weights for policy 1, policy_version 1502206 (0.0008) [2023-12-27 02:18:50,994][105620] Updated weights for policy 1, policy_version 1502216 (0.0008) [2023-12-27 02:18:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 768581632. Throughput: 0: 9629.5, 1: 10002.5. Samples: 768566772. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:18:51,063][104569] Avg episode reward: [(0, '8175.357'), (1, '9091.720')] [2023-12-27 02:18:51,562][105692] Updated weights for policy 0, policy_version 1499625 (0.0008) [2023-12-27 02:18:51,619][105692] Updated weights for policy 0, policy_version 1499635 (0.0009) [2023-12-27 02:18:51,670][105692] Updated weights for policy 0, policy_version 1499645 (0.0008) [2023-12-27 02:18:51,752][105620] Updated weights for policy 1, policy_version 1502226 (0.0009) [2023-12-27 02:18:51,753][105692] Updated weights for policy 0, policy_version 1499655 (0.0010) [2023-12-27 02:18:51,808][105620] Updated weights for policy 1, policy_version 1502236 (0.0009) [2023-12-27 02:18:51,865][105620] Updated weights for policy 1, policy_version 1502246 (0.0010) [2023-12-27 02:18:51,918][105620] Updated weights for policy 1, policy_version 1502256 (0.0009) [2023-12-27 02:18:52,498][105692] Updated weights for policy 0, policy_version 1499665 (0.0008) [2023-12-27 02:18:52,562][105692] Updated weights for policy 0, policy_version 1499675 (0.0009) [2023-12-27 02:18:52,624][105692] Updated weights for policy 0, policy_version 1499685 (0.0010) [2023-12-27 02:18:52,712][105620] Updated weights for policy 1, policy_version 1502266 (0.0008) [2023-12-27 02:18:52,778][105620] Updated weights for policy 1, policy_version 1502276 (0.0008) [2023-12-27 02:18:52,830][105620] Updated weights for policy 1, policy_version 1502286 (0.0009) [2023-12-27 02:18:53,349][105692] Updated weights for policy 0, policy_version 1499695 (0.0009) [2023-12-27 02:18:53,400][105692] Updated weights for policy 0, policy_version 1499705 (0.0009) [2023-12-27 02:18:53,457][105692] Updated weights for policy 0, policy_version 1499715 (0.0008) [2023-12-27 02:18:53,609][105620] Updated weights for policy 1, policy_version 1502296 (0.0008) [2023-12-27 02:18:53,671][105620] Updated weights for policy 1, policy_version 1502306 (0.0009) [2023-12-27 02:18:53,727][105620] Updated weights for policy 1, policy_version 1502316 (0.0009) [2023-12-27 02:18:54,215][105692] Updated weights for policy 0, policy_version 1499725 (0.0007) [2023-12-27 02:18:54,273][105692] Updated weights for policy 0, policy_version 1499735 (0.0008) [2023-12-27 02:18:54,331][105692] Updated weights for policy 0, policy_version 1499745 (0.0009) [2023-12-27 02:18:54,483][105620] Updated weights for policy 1, policy_version 1502326 (0.0008) [2023-12-27 02:18:54,537][105620] Updated weights for policy 1, policy_version 1502336 (0.0009) [2023-12-27 02:18:54,593][105620] Updated weights for policy 1, policy_version 1502346 (0.0009) [2023-12-27 02:18:55,106][105692] Updated weights for policy 0, policy_version 1499755 (0.0009) [2023-12-27 02:18:55,160][105692] Updated weights for policy 0, policy_version 1499765 (0.0009) [2023-12-27 02:18:55,218][105692] Updated weights for policy 0, policy_version 1499775 (0.0009) [2023-12-27 02:18:55,280][105620] Updated weights for policy 1, policy_version 1502356 (0.0009) [2023-12-27 02:18:55,345][105620] Updated weights for policy 1, policy_version 1502366 (0.0008) [2023-12-27 02:18:55,410][105620] Updated weights for policy 1, policy_version 1502376 (0.0007) [2023-12-27 02:18:55,965][105692] Updated weights for policy 0, policy_version 1499785 (0.0009) [2023-12-27 02:18:56,011][105692] Updated weights for policy 0, policy_version 1499795 (0.0007) [2023-12-27 02:18:56,062][104569] Fps is (10 sec: 18022.6, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 768663552. Throughput: 0: 9587.7, 1: 9877.1. Samples: 768677236. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:18:56,062][104569] Avg episode reward: [(0, '8719.325'), (1, '9169.012')] [2023-12-27 02:18:56,069][105692] Updated weights for policy 0, policy_version 1499805 (0.0009) [2023-12-27 02:18:56,125][105692] Updated weights for policy 0, policy_version 1499815 (0.0008) [2023-12-27 02:18:56,132][105620] Updated weights for policy 1, policy_version 1502386 (0.0008) [2023-12-27 02:18:56,179][105620] Updated weights for policy 1, policy_version 1502396 (0.0008) [2023-12-27 02:18:56,226][105620] Updated weights for policy 1, policy_version 1502406 (0.0009) [2023-12-27 02:18:56,278][105620] Updated weights for policy 1, policy_version 1502416 (0.0010) [2023-12-27 02:18:56,772][105692] Updated weights for policy 0, policy_version 1499825 (0.0009) [2023-12-27 02:18:56,824][105692] Updated weights for policy 0, policy_version 1499835 (0.0009) [2023-12-27 02:18:56,880][105692] Updated weights for policy 0, policy_version 1499845 (0.0009) [2023-12-27 02:18:56,990][105620] Updated weights for policy 1, policy_version 1502426 (0.0009) [2023-12-27 02:18:57,041][105620] Updated weights for policy 1, policy_version 1502436 (0.0009) [2023-12-27 02:18:57,088][105620] Updated weights for policy 1, policy_version 1502446 (0.0009) [2023-12-27 02:18:57,674][105692] Updated weights for policy 0, policy_version 1499855 (0.0009) [2023-12-27 02:18:57,721][105692] Updated weights for policy 0, policy_version 1499865 (0.0009) [2023-12-27 02:18:57,767][105692] Updated weights for policy 0, policy_version 1499875 (0.0008) [2023-12-27 02:18:57,822][105620] Updated weights for policy 1, policy_version 1502456 (0.0007) [2023-12-27 02:18:57,874][105620] Updated weights for policy 1, policy_version 1502466 (0.0008) [2023-12-27 02:18:57,921][105620] Updated weights for policy 1, policy_version 1502476 (0.0008) [2023-12-27 02:18:58,576][105692] Updated weights for policy 0, policy_version 1499885 (0.0008) [2023-12-27 02:18:58,640][105692] Updated weights for policy 0, policy_version 1499895 (0.0008) [2023-12-27 02:18:58,710][105692] Updated weights for policy 0, policy_version 1499905 (0.0008) [2023-12-27 02:18:58,744][105620] Updated weights for policy 1, policy_version 1502486 (0.0008) [2023-12-27 02:18:58,809][105620] Updated weights for policy 1, policy_version 1502496 (0.0009) [2023-12-27 02:18:58,883][105620] Updated weights for policy 1, policy_version 1502507 (0.0008) [2023-12-27 02:18:59,452][105692] Updated weights for policy 0, policy_version 1499915 (0.0009) [2023-12-27 02:18:59,520][105692] Updated weights for policy 0, policy_version 1499925 (0.0010) [2023-12-27 02:18:59,581][105692] Updated weights for policy 0, policy_version 1499935 (0.0009) [2023-12-27 02:18:59,708][105620] Updated weights for policy 1, policy_version 1502517 (0.0006) [2023-12-27 02:18:59,766][105620] Updated weights for policy 1, policy_version 1502527 (0.0008) [2023-12-27 02:18:59,827][105620] Updated weights for policy 1, policy_version 1502537 (0.0009) [2023-12-27 02:19:00,400][105692] Updated weights for policy 0, policy_version 1499945 (0.0009) [2023-12-27 02:19:00,460][105692] Updated weights for policy 0, policy_version 1499955 (0.0009) [2023-12-27 02:19:00,471][105620] Updated weights for policy 1, policy_version 1502547 (0.0008) [2023-12-27 02:19:00,518][105692] Updated weights for policy 0, policy_version 1499965 (0.0007) [2023-12-27 02:19:00,526][105620] Updated weights for policy 1, policy_version 1502557 (0.0008) [2023-12-27 02:19:00,568][105692] Updated weights for policy 0, policy_version 1499975 (0.0005) [2023-12-27 02:19:00,586][105620] Updated weights for policy 1, policy_version 1502567 (0.0009) [2023-12-27 02:19:01,062][104569] Fps is (10 sec: 18022.6, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 768761856. Throughput: 0: 9622.3, 1: 9857.6. Samples: 768733596. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:19:01,062][104569] Avg episode reward: [(0, '8716.329'), (1, '9078.112')] [2023-12-27 02:19:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001499976_384049152.pth... [2023-12-27 02:19:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001502576_384712704.pth... [2023-12-27 02:19:01,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001498856_383762432.pth [2023-12-27 02:19:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001501456_384425984.pth [2023-12-27 02:19:01,289][105692] Updated weights for policy 0, policy_version 1499985 (0.0008) [2023-12-27 02:19:01,344][105692] Updated weights for policy 0, policy_version 1499995 (0.0009) [2023-12-27 02:19:01,345][105620] Updated weights for policy 1, policy_version 1502577 (0.0010) [2023-12-27 02:19:01,409][105692] Updated weights for policy 0, policy_version 1500005 (0.0007) [2023-12-27 02:19:01,411][105620] Updated weights for policy 1, policy_version 1502587 (0.0008) [2023-12-27 02:19:01,460][105620] Updated weights for policy 1, policy_version 1502597 (0.0008) [2023-12-27 02:19:01,514][105620] Updated weights for policy 1, policy_version 1502608 (0.0010) [2023-12-27 02:19:02,156][105692] Updated weights for policy 0, policy_version 1500015 (0.0008) [2023-12-27 02:19:02,213][105692] Updated weights for policy 0, policy_version 1500025 (0.0008) [2023-12-27 02:19:02,264][105620] Updated weights for policy 1, policy_version 1502618 (0.0006) [2023-12-27 02:19:02,270][105692] Updated weights for policy 0, policy_version 1500035 (0.0007) [2023-12-27 02:19:02,324][105620] Updated weights for policy 1, policy_version 1502628 (0.0008) [2023-12-27 02:19:02,393][105620] Updated weights for policy 1, policy_version 1502638 (0.0009) [2023-12-27 02:19:02,904][105692] Updated weights for policy 0, policy_version 1500045 (0.0009) [2023-12-27 02:19:02,967][105692] Updated weights for policy 0, policy_version 1500055 (0.0009) [2023-12-27 02:19:03,015][105692] Updated weights for policy 0, policy_version 1500065 (0.0009) [2023-12-27 02:19:03,218][105620] Updated weights for policy 1, policy_version 1502648 (0.0009) [2023-12-27 02:19:03,275][105620] Updated weights for policy 1, policy_version 1502658 (0.0009) [2023-12-27 02:19:03,322][105620] Updated weights for policy 1, policy_version 1502668 (0.0009) [2023-12-27 02:19:03,780][105692] Updated weights for policy 0, policy_version 1500075 (0.0009) [2023-12-27 02:19:03,839][105692] Updated weights for policy 0, policy_version 1500085 (0.0009) [2023-12-27 02:19:03,903][105692] Updated weights for policy 0, policy_version 1500095 (0.0010) [2023-12-27 02:19:04,015][105620] Updated weights for policy 1, policy_version 1502678 (0.0007) [2023-12-27 02:19:04,082][105620] Updated weights for policy 1, policy_version 1502688 (0.0005) [2023-12-27 02:19:04,147][105620] Updated weights for policy 1, policy_version 1502698 (0.0005) [2023-12-27 02:19:04,699][105620] Updated weights for policy 1, policy_version 1502708 (0.0007) [2023-12-27 02:19:04,750][105692] Updated weights for policy 0, policy_version 1500105 (0.0010) [2023-12-27 02:19:04,752][105620] Updated weights for policy 1, policy_version 1502718 (0.0006) [2023-12-27 02:19:04,799][105692] Updated weights for policy 0, policy_version 1500115 (0.0008) [2023-12-27 02:19:04,801][105620] Updated weights for policy 1, policy_version 1502728 (0.0005) [2023-12-27 02:19:04,846][105692] Updated weights for policy 0, policy_version 1500125 (0.0006) [2023-12-27 02:19:04,898][105692] Updated weights for policy 0, policy_version 1500135 (0.0009) [2023-12-27 02:19:05,474][105620] Updated weights for policy 1, policy_version 1502738 (0.0007) [2023-12-27 02:19:05,535][105620] Updated weights for policy 1, policy_version 1502748 (0.0005) [2023-12-27 02:19:05,603][105620] Updated weights for policy 1, policy_version 1502758 (0.0007) [2023-12-27 02:19:05,650][105620] Updated weights for policy 1, policy_version 1502768 (0.0008) [2023-12-27 02:19:05,713][105692] Updated weights for policy 0, policy_version 1500146 (0.0010) [2023-12-27 02:19:05,767][105692] Updated weights for policy 0, policy_version 1500156 (0.0010) [2023-12-27 02:19:05,819][105692] Updated weights for policy 0, policy_version 1500166 (0.0010) [2023-12-27 02:19:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 768860160. Throughput: 0: 9561.8, 1: 9830.2. Samples: 768847996. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:19:06,062][104569] Avg episode reward: [(0, '8540.141'), (1, '9078.257')] [2023-12-27 02:19:06,241][105620] Updated weights for policy 1, policy_version 1502778 (0.0009) [2023-12-27 02:19:06,289][105620] Updated weights for policy 1, policy_version 1502788 (0.0009) [2023-12-27 02:19:06,344][105620] Updated weights for policy 1, policy_version 1502798 (0.0009) [2023-12-27 02:19:06,633][105692] Updated weights for policy 0, policy_version 1500176 (0.0009) [2023-12-27 02:19:06,688][105692] Updated weights for policy 0, policy_version 1500186 (0.0009) [2023-12-27 02:19:06,744][105692] Updated weights for policy 0, policy_version 1500196 (0.0009) [2023-12-27 02:19:07,119][105620] Updated weights for policy 1, policy_version 1502808 (0.0008) [2023-12-27 02:19:07,170][105620] Updated weights for policy 1, policy_version 1502818 (0.0009) [2023-12-27 02:19:07,221][105620] Updated weights for policy 1, policy_version 1502828 (0.0006) [2023-12-27 02:19:07,573][105692] Updated weights for policy 0, policy_version 1500206 (0.0009) [2023-12-27 02:19:07,624][105692] Updated weights for policy 0, policy_version 1500216 (0.0008) [2023-12-27 02:19:07,677][105692] Updated weights for policy 0, policy_version 1500226 (0.0009) [2023-12-27 02:19:07,836][105620] Updated weights for policy 1, policy_version 1502838 (0.0008) [2023-12-27 02:19:07,899][105620] Updated weights for policy 1, policy_version 1502848 (0.0006) [2023-12-27 02:19:07,967][105620] Updated weights for policy 1, policy_version 1502858 (0.0005) [2023-12-27 02:19:08,555][105692] Updated weights for policy 0, policy_version 1500236 (0.0008) [2023-12-27 02:19:08,582][105620] Updated weights for policy 1, policy_version 1502868 (0.0008) [2023-12-27 02:19:08,601][105692] Updated weights for policy 0, policy_version 1500246 (0.0006) [2023-12-27 02:19:08,639][105620] Updated weights for policy 1, policy_version 1502878 (0.0011) [2023-12-27 02:19:08,650][105692] Updated weights for policy 0, policy_version 1500256 (0.0009) [2023-12-27 02:19:08,692][105620] Updated weights for policy 1, policy_version 1502888 (0.0011) [2023-12-27 02:19:09,455][105692] Updated weights for policy 0, policy_version 1500266 (0.0006) [2023-12-27 02:19:09,456][105620] Updated weights for policy 1, policy_version 1502898 (0.0010) [2023-12-27 02:19:09,515][105620] Updated weights for policy 1, policy_version 1502908 (0.0008) [2023-12-27 02:19:09,517][105692] Updated weights for policy 0, policy_version 1500276 (0.0007) [2023-12-27 02:19:09,564][105620] Updated weights for policy 1, policy_version 1502918 (0.0008) [2023-12-27 02:19:09,574][105692] Updated weights for policy 0, policy_version 1500286 (0.0008) [2023-12-27 02:19:09,609][105620] Updated weights for policy 1, policy_version 1502928 (0.0008) [2023-12-27 02:19:09,624][105692] Updated weights for policy 0, policy_version 1500296 (0.0008) [2023-12-27 02:19:10,321][105692] Updated weights for policy 0, policy_version 1500306 (0.0009) [2023-12-27 02:19:10,351][105620] Updated weights for policy 1, policy_version 1502938 (0.0010) [2023-12-27 02:19:10,382][105692] Updated weights for policy 0, policy_version 1500316 (0.0007) [2023-12-27 02:19:10,407][105620] Updated weights for policy 1, policy_version 1502948 (0.0010) [2023-12-27 02:19:10,440][105692] Updated weights for policy 0, policy_version 1500326 (0.0007) [2023-12-27 02:19:10,466][105620] Updated weights for policy 1, policy_version 1502958 (0.0011) [2023-12-27 02:19:11,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 768950272. Throughput: 0: 9497.0, 1: 9800.7. Samples: 768961816. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:19:11,063][104569] Avg episode reward: [(0, '8637.878'), (1, '9169.405')] [2023-12-27 02:19:11,129][105692] Updated weights for policy 0, policy_version 1500336 (0.0007) [2023-12-27 02:19:11,193][105692] Updated weights for policy 0, policy_version 1500346 (0.0009) [2023-12-27 02:19:11,249][105620] Updated weights for policy 1, policy_version 1502968 (0.0011) [2023-12-27 02:19:11,261][105692] Updated weights for policy 0, policy_version 1500356 (0.0007) [2023-12-27 02:19:11,306][105620] Updated weights for policy 1, policy_version 1502978 (0.0010) [2023-12-27 02:19:11,373][105620] Updated weights for policy 1, policy_version 1502988 (0.0010) [2023-12-27 02:19:11,981][105692] Updated weights for policy 0, policy_version 1500366 (0.0007) [2023-12-27 02:19:12,042][105692] Updated weights for policy 0, policy_version 1500376 (0.0008) [2023-12-27 02:19:12,102][105692] Updated weights for policy 0, policy_version 1500386 (0.0008) [2023-12-27 02:19:12,185][105620] Updated weights for policy 1, policy_version 1502998 (0.0009) [2023-12-27 02:19:12,255][105620] Updated weights for policy 1, policy_version 1503008 (0.0007) [2023-12-27 02:19:12,321][105620] Updated weights for policy 1, policy_version 1503018 (0.0010) [2023-12-27 02:19:12,822][105692] Updated weights for policy 0, policy_version 1500396 (0.0008) [2023-12-27 02:19:12,886][105692] Updated weights for policy 0, policy_version 1500406 (0.0008) [2023-12-27 02:19:12,944][105692] Updated weights for policy 0, policy_version 1500416 (0.0006) [2023-12-27 02:19:13,047][105620] Updated weights for policy 1, policy_version 1503028 (0.0009) [2023-12-27 02:19:13,100][105620] Updated weights for policy 1, policy_version 1503038 (0.0010) [2023-12-27 02:19:13,148][105620] Updated weights for policy 1, policy_version 1503048 (0.0005) [2023-12-27 02:19:13,622][105692] Updated weights for policy 0, policy_version 1500426 (0.0009) [2023-12-27 02:19:13,670][105692] Updated weights for policy 0, policy_version 1500436 (0.0010) [2023-12-27 02:19:13,723][105692] Updated weights for policy 0, policy_version 1500446 (0.0011) [2023-12-27 02:19:13,777][105692] Updated weights for policy 0, policy_version 1500456 (0.0011) [2023-12-27 02:19:13,801][105620] Updated weights for policy 1, policy_version 1503058 (0.0006) [2023-12-27 02:19:13,849][105620] Updated weights for policy 1, policy_version 1503068 (0.0010) [2023-12-27 02:19:13,914][105620] Updated weights for policy 1, policy_version 1503078 (0.0010) [2023-12-27 02:19:13,961][105620] Updated weights for policy 1, policy_version 1503088 (0.0010) [2023-12-27 02:19:14,425][105692] Updated weights for policy 0, policy_version 1500466 (0.0005) [2023-12-27 02:19:14,479][105692] Updated weights for policy 0, policy_version 1500476 (0.0005) [2023-12-27 02:19:14,525][105692] Updated weights for policy 0, policy_version 1500486 (0.0005) [2023-12-27 02:19:14,651][105620] Updated weights for policy 1, policy_version 1503098 (0.0006) [2023-12-27 02:19:14,708][105620] Updated weights for policy 1, policy_version 1503108 (0.0005) [2023-12-27 02:19:14,764][105620] Updated weights for policy 1, policy_version 1503118 (0.0006) [2023-12-27 02:19:15,214][105692] Updated weights for policy 0, policy_version 1500496 (0.0010) [2023-12-27 02:19:15,274][105692] Updated weights for policy 0, policy_version 1500506 (0.0011) [2023-12-27 02:19:15,333][105692] Updated weights for policy 0, policy_version 1500516 (0.0011) [2023-12-27 02:19:15,481][105620] Updated weights for policy 1, policy_version 1503128 (0.0010) [2023-12-27 02:19:15,534][105620] Updated weights for policy 1, policy_version 1503138 (0.0011) [2023-12-27 02:19:15,585][105620] Updated weights for policy 1, policy_version 1503148 (0.0008) [2023-12-27 02:19:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.3, 300 sec: 19466.4). Total num frames: 769048576. Throughput: 0: 9522.3, 1: 9657.6. Samples: 769020176. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:19:16,063][104569] Avg episode reward: [(0, '8729.468'), (1, '9261.966')] [2023-12-27 02:19:16,068][105692] Updated weights for policy 0, policy_version 1500526 (0.0009) [2023-12-27 02:19:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001503152_384860160.pth... [2023-12-27 02:19:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001502032_384573440.pth [2023-12-27 02:19:16,128][105692] Updated weights for policy 0, policy_version 1500536 (0.0005) [2023-12-27 02:19:16,182][105692] Updated weights for policy 0, policy_version 1500546 (0.0005) [2023-12-27 02:19:16,218][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001500552_384196608.pth... [2023-12-27 02:19:16,222][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001499432_383909888.pth [2023-12-27 02:19:16,258][105620] Updated weights for policy 1, policy_version 1503158 (0.0005) [2023-12-27 02:19:16,320][105620] Updated weights for policy 1, policy_version 1503168 (0.0005) [2023-12-27 02:19:16,382][105620] Updated weights for policy 1, policy_version 1503178 (0.0009) [2023-12-27 02:19:16,877][105692] Updated weights for policy 0, policy_version 1500556 (0.0007) [2023-12-27 02:19:16,946][105692] Updated weights for policy 0, policy_version 1500566 (0.0009) [2023-12-27 02:19:16,965][105620] Updated weights for policy 1, policy_version 1503188 (0.0008) [2023-12-27 02:19:17,009][105692] Updated weights for policy 0, policy_version 1500576 (0.0008) [2023-12-27 02:19:17,019][105620] Updated weights for policy 1, policy_version 1503198 (0.0007) [2023-12-27 02:19:17,066][105620] Updated weights for policy 1, policy_version 1503208 (0.0008) [2023-12-27 02:19:17,682][105692] Updated weights for policy 0, policy_version 1500586 (0.0008) [2023-12-27 02:19:17,739][105692] Updated weights for policy 0, policy_version 1500596 (0.0008) [2023-12-27 02:19:17,792][105692] Updated weights for policy 0, policy_version 1500606 (0.0008) [2023-12-27 02:19:17,822][105620] Updated weights for policy 1, policy_version 1503218 (0.0010) [2023-12-27 02:19:17,840][105692] Updated weights for policy 0, policy_version 1500616 (0.0007) [2023-12-27 02:19:17,870][105620] Updated weights for policy 1, policy_version 1503228 (0.0010) [2023-12-27 02:19:17,930][105620] Updated weights for policy 1, policy_version 1503238 (0.0011) [2023-12-27 02:19:17,980][105620] Updated weights for policy 1, policy_version 1503248 (0.0011) [2023-12-27 02:19:18,617][105692] Updated weights for policy 0, policy_version 1500626 (0.0010) [2023-12-27 02:19:18,663][105692] Updated weights for policy 0, policy_version 1500636 (0.0011) [2023-12-27 02:19:18,711][105692] Updated weights for policy 0, policy_version 1500646 (0.0011) [2023-12-27 02:19:18,749][105620] Updated weights for policy 1, policy_version 1503258 (0.0011) [2023-12-27 02:19:18,812][105620] Updated weights for policy 1, policy_version 1503268 (0.0011) [2023-12-27 02:19:18,882][105620] Updated weights for policy 1, policy_version 1503278 (0.0011) [2023-12-27 02:19:19,474][105692] Updated weights for policy 0, policy_version 1500656 (0.0011) [2023-12-27 02:19:19,534][105692] Updated weights for policy 0, policy_version 1500666 (0.0011) [2023-12-27 02:19:19,593][105692] Updated weights for policy 0, policy_version 1500676 (0.0010) [2023-12-27 02:19:19,606][105620] Updated weights for policy 1, policy_version 1503288 (0.0010) [2023-12-27 02:19:19,663][105620] Updated weights for policy 1, policy_version 1503298 (0.0010) [2023-12-27 02:19:19,724][105620] Updated weights for policy 1, policy_version 1503308 (0.0010) [2023-12-27 02:19:20,362][105692] Updated weights for policy 0, policy_version 1500686 (0.0011) [2023-12-27 02:19:20,425][105692] Updated weights for policy 0, policy_version 1500696 (0.0011) [2023-12-27 02:19:20,464][105620] Updated weights for policy 1, policy_version 1503318 (0.0011) [2023-12-27 02:19:20,474][105692] Updated weights for policy 0, policy_version 1500706 (0.0011) [2023-12-27 02:19:20,520][105620] Updated weights for policy 1, policy_version 1503328 (0.0010) [2023-12-27 02:19:20,577][105620] Updated weights for policy 1, policy_version 1503338 (0.0010) [2023-12-27 02:19:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 769146880. Throughput: 0: 9537.1, 1: 9577.3. Samples: 769138492. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:19:21,062][104569] Avg episode reward: [(0, '8367.855'), (1, '9172.327')] [2023-12-27 02:19:21,282][105692] Updated weights for policy 0, policy_version 1500716 (0.0010) [2023-12-27 02:19:21,323][105620] Updated weights for policy 1, policy_version 1503348 (0.0011) [2023-12-27 02:19:21,343][105692] Updated weights for policy 0, policy_version 1500726 (0.0007) [2023-12-27 02:19:21,384][105620] Updated weights for policy 1, policy_version 1503358 (0.0013) [2023-12-27 02:19:21,410][105692] Updated weights for policy 0, policy_version 1500736 (0.0007) [2023-12-27 02:19:21,449][105620] Updated weights for policy 1, policy_version 1503368 (0.0010) [2023-12-27 02:19:22,158][105692] Updated weights for policy 0, policy_version 1500746 (0.0006) [2023-12-27 02:19:22,191][105620] Updated weights for policy 1, policy_version 1503378 (0.0008) [2023-12-27 02:19:22,217][105692] Updated weights for policy 0, policy_version 1500756 (0.0007) [2023-12-27 02:19:22,254][105620] Updated weights for policy 1, policy_version 1503388 (0.0011) [2023-12-27 02:19:22,279][105692] Updated weights for policy 0, policy_version 1500766 (0.0007) [2023-12-27 02:19:22,319][105620] Updated weights for policy 1, policy_version 1503398 (0.0011) [2023-12-27 02:19:22,349][105692] Updated weights for policy 0, policy_version 1500776 (0.0006) [2023-12-27 02:19:23,027][105620] Updated weights for policy 1, policy_version 1503409 (0.0009) [2023-12-27 02:19:23,094][105620] Updated weights for policy 1, policy_version 1503419 (0.0007) [2023-12-27 02:19:23,145][105620] Updated weights for policy 1, policy_version 1503429 (0.0009) [2023-12-27 02:19:23,160][105692] Updated weights for policy 0, policy_version 1500786 (0.0007) [2023-12-27 02:19:23,202][105620] Updated weights for policy 1, policy_version 1503439 (0.0008) [2023-12-27 02:19:23,209][105692] Updated weights for policy 0, policy_version 1500796 (0.0006) [2023-12-27 02:19:23,265][105692] Updated weights for policy 0, policy_version 1500806 (0.0008) [2023-12-27 02:19:23,902][105620] Updated weights for policy 1, policy_version 1503449 (0.0006) [2023-12-27 02:19:23,964][105620] Updated weights for policy 1, policy_version 1503459 (0.0008) [2023-12-27 02:19:23,997][105692] Updated weights for policy 0, policy_version 1500816 (0.0006) [2023-12-27 02:19:24,028][105620] Updated weights for policy 1, policy_version 1503469 (0.0008) [2023-12-27 02:19:24,054][105692] Updated weights for policy 0, policy_version 1500826 (0.0006) [2023-12-27 02:19:24,108][105692] Updated weights for policy 0, policy_version 1500836 (0.0009) [2023-12-27 02:19:24,718][105620] Updated weights for policy 1, policy_version 1503479 (0.0006) [2023-12-27 02:19:24,772][105620] Updated weights for policy 1, policy_version 1503489 (0.0005) [2023-12-27 02:19:24,820][105692] Updated weights for policy 0, policy_version 1500846 (0.0008) [2023-12-27 02:19:24,840][105620] Updated weights for policy 1, policy_version 1503499 (0.0005) [2023-12-27 02:19:24,873][105692] Updated weights for policy 0, policy_version 1500856 (0.0007) [2023-12-27 02:19:24,925][105692] Updated weights for policy 0, policy_version 1500866 (0.0010) [2023-12-27 02:19:25,434][105620] Updated weights for policy 1, policy_version 1503509 (0.0006) [2023-12-27 02:19:25,487][105620] Updated weights for policy 1, policy_version 1503519 (0.0005) [2023-12-27 02:19:25,551][105620] Updated weights for policy 1, policy_version 1503529 (0.0008) [2023-12-27 02:19:25,640][105692] Updated weights for policy 0, policy_version 1500876 (0.0010) [2023-12-27 02:19:25,690][105692] Updated weights for policy 0, policy_version 1500886 (0.0010) [2023-12-27 02:19:25,755][105692] Updated weights for policy 0, policy_version 1500896 (0.0010) [2023-12-27 02:19:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 769245184. Throughput: 0: 9477.7, 1: 9607.4. Samples: 769254704. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:19:26,062][104569] Avg episode reward: [(0, '8092.963'), (1, '9080.428')] [2023-12-27 02:19:26,154][105620] Updated weights for policy 1, policy_version 1503539 (0.0007) [2023-12-27 02:19:26,203][105620] Updated weights for policy 1, policy_version 1503549 (0.0009) [2023-12-27 02:19:26,257][105620] Updated weights for policy 1, policy_version 1503559 (0.0008) [2023-12-27 02:19:26,439][105692] Updated weights for policy 0, policy_version 1500906 (0.0009) [2023-12-27 02:19:26,497][105692] Updated weights for policy 0, policy_version 1500916 (0.0006) [2023-12-27 02:19:26,564][105692] Updated weights for policy 0, policy_version 1500926 (0.0007) [2023-12-27 02:19:26,626][105692] Updated weights for policy 0, policy_version 1500936 (0.0010) [2023-12-27 02:19:27,075][105620] Updated weights for policy 1, policy_version 1503569 (0.0008) [2023-12-27 02:19:27,132][105620] Updated weights for policy 1, policy_version 1503579 (0.0005) [2023-12-27 02:19:27,192][105620] Updated weights for policy 1, policy_version 1503589 (0.0005) [2023-12-27 02:19:27,203][105692] Updated weights for policy 0, policy_version 1500946 (0.0008) [2023-12-27 02:19:27,241][105620] Updated weights for policy 1, policy_version 1503599 (0.0005) [2023-12-27 02:19:27,249][105692] Updated weights for policy 0, policy_version 1500956 (0.0010) [2023-12-27 02:19:27,297][105692] Updated weights for policy 0, policy_version 1500966 (0.0005) [2023-12-27 02:19:27,796][105620] Updated weights for policy 1, policy_version 1503609 (0.0008) [2023-12-27 02:19:27,854][105620] Updated weights for policy 1, policy_version 1503619 (0.0010) [2023-12-27 02:19:27,860][105692] Updated weights for policy 0, policy_version 1500976 (0.0005) [2023-12-27 02:19:27,908][105620] Updated weights for policy 1, policy_version 1503629 (0.0010) [2023-12-27 02:19:27,912][105692] Updated weights for policy 0, policy_version 1500986 (0.0005) [2023-12-27 02:19:27,962][105692] Updated weights for policy 0, policy_version 1500996 (0.0005) [2023-12-27 02:19:28,513][105620] Updated weights for policy 1, policy_version 1503639 (0.0007) [2023-12-27 02:19:28,567][105620] Updated weights for policy 1, policy_version 1503649 (0.0005) [2023-12-27 02:19:28,626][105620] Updated weights for policy 1, policy_version 1503659 (0.0007) [2023-12-27 02:19:28,627][105692] Updated weights for policy 0, policy_version 1501006 (0.0008) [2023-12-27 02:19:28,681][105692] Updated weights for policy 0, policy_version 1501016 (0.0010) [2023-12-27 02:19:28,739][105692] Updated weights for policy 0, policy_version 1501026 (0.0010) [2023-12-27 02:19:29,308][105620] Updated weights for policy 1, policy_version 1503669 (0.0011) [2023-12-27 02:19:29,373][105620] Updated weights for policy 1, policy_version 1503679 (0.0011) [2023-12-27 02:19:29,435][105620] Updated weights for policy 1, policy_version 1503689 (0.0010) [2023-12-27 02:19:29,488][105692] Updated weights for policy 0, policy_version 1501036 (0.0010) [2023-12-27 02:19:29,539][105692] Updated weights for policy 0, policy_version 1501046 (0.0010) [2023-12-27 02:19:29,590][105692] Updated weights for policy 0, policy_version 1501056 (0.0010) [2023-12-27 02:19:30,162][105620] Updated weights for policy 1, policy_version 1503699 (0.0009) [2023-12-27 02:19:30,221][105620] Updated weights for policy 1, policy_version 1503709 (0.0006) [2023-12-27 02:19:30,280][105620] Updated weights for policy 1, policy_version 1503719 (0.0007) [2023-12-27 02:19:30,306][105692] Updated weights for policy 0, policy_version 1501066 (0.0009) [2023-12-27 02:19:30,365][105692] Updated weights for policy 0, policy_version 1501076 (0.0007) [2023-12-27 02:19:30,423][105692] Updated weights for policy 0, policy_version 1501086 (0.0005) [2023-12-27 02:19:30,472][105692] Updated weights for policy 0, policy_version 1501096 (0.0009) [2023-12-27 02:19:30,907][105620] Updated weights for policy 1, policy_version 1503729 (0.0010) [2023-12-27 02:19:30,966][105620] Updated weights for policy 1, policy_version 1503739 (0.0009) [2023-12-27 02:19:31,031][105620] Updated weights for policy 1, policy_version 1503749 (0.0007) [2023-12-27 02:19:31,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19251.1, 300 sec: 19494.2). Total num frames: 769343488. Throughput: 0: 9599.1, 1: 9683.2. Samples: 769319056. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:19:31,063][104569] Avg episode reward: [(0, '8084.290'), (1, '9080.512')] [2023-12-27 02:19:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001501096_384335872.pth... [2023-12-27 02:19:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001499976_384049152.pth [2023-12-27 02:19:31,090][105620] Updated weights for policy 1, policy_version 1503759 (0.0010) [2023-12-27 02:19:31,092][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001503760_385015808.pth... [2023-12-27 02:19:31,095][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001502576_384712704.pth [2023-12-27 02:19:31,180][105692] Updated weights for policy 0, policy_version 1501106 (0.0011) [2023-12-27 02:19:31,239][105692] Updated weights for policy 0, policy_version 1501116 (0.0010) [2023-12-27 02:19:31,308][105692] Updated weights for policy 0, policy_version 1501126 (0.0010) [2023-12-27 02:19:31,792][105620] Updated weights for policy 1, policy_version 1503769 (0.0008) [2023-12-27 02:19:31,847][105620] Updated weights for policy 1, policy_version 1503779 (0.0008) [2023-12-27 02:19:31,900][105620] Updated weights for policy 1, policy_version 1503789 (0.0009) [2023-12-27 02:19:32,040][105692] Updated weights for policy 0, policy_version 1501136 (0.0010) [2023-12-27 02:19:32,102][105692] Updated weights for policy 0, policy_version 1501147 (0.0009) [2023-12-27 02:19:32,160][105692] Updated weights for policy 0, policy_version 1501157 (0.0006) [2023-12-27 02:19:32,537][105620] Updated weights for policy 1, policy_version 1503799 (0.0006) [2023-12-27 02:19:32,589][105620] Updated weights for policy 1, policy_version 1503809 (0.0005) [2023-12-27 02:19:32,636][105620] Updated weights for policy 1, policy_version 1503819 (0.0005) [2023-12-27 02:19:32,828][105692] Updated weights for policy 0, policy_version 1501167 (0.0007) [2023-12-27 02:19:32,885][105692] Updated weights for policy 0, policy_version 1501177 (0.0006) [2023-12-27 02:19:32,939][105692] Updated weights for policy 0, policy_version 1501187 (0.0008) [2023-12-27 02:19:33,252][105620] Updated weights for policy 1, policy_version 1503829 (0.0007) [2023-12-27 02:19:33,312][105620] Updated weights for policy 1, policy_version 1503839 (0.0008) [2023-12-27 02:19:33,375][105620] Updated weights for policy 1, policy_version 1503849 (0.0008) [2023-12-27 02:19:33,656][105692] Updated weights for policy 0, policy_version 1501197 (0.0010) [2023-12-27 02:19:33,700][105692] Updated weights for policy 0, policy_version 1501207 (0.0010) [2023-12-27 02:19:33,749][105692] Updated weights for policy 0, policy_version 1501217 (0.0010) [2023-12-27 02:19:33,960][105620] Updated weights for policy 1, policy_version 1503859 (0.0005) [2023-12-27 02:19:34,024][105620] Updated weights for policy 1, policy_version 1503869 (0.0008) [2023-12-27 02:19:34,072][105620] Updated weights for policy 1, policy_version 1503879 (0.0005) [2023-12-27 02:19:34,530][105692] Updated weights for policy 0, policy_version 1501227 (0.0010) [2023-12-27 02:19:34,593][105692] Updated weights for policy 0, policy_version 1501237 (0.0010) [2023-12-27 02:19:34,653][105620] Updated weights for policy 1, policy_version 1503889 (0.0008) [2023-12-27 02:19:34,659][105692] Updated weights for policy 0, policy_version 1501247 (0.0010) [2023-12-27 02:19:34,709][105620] Updated weights for policy 1, policy_version 1503899 (0.0006) [2023-12-27 02:19:34,765][105620] Updated weights for policy 1, policy_version 1503909 (0.0008) [2023-12-27 02:19:34,818][105620] Updated weights for policy 1, policy_version 1503919 (0.0008) [2023-12-27 02:19:35,410][105692] Updated weights for policy 0, policy_version 1501257 (0.0011) [2023-12-27 02:19:35,445][105620] Updated weights for policy 1, policy_version 1503929 (0.0007) [2023-12-27 02:19:35,463][105692] Updated weights for policy 0, policy_version 1501267 (0.0011) [2023-12-27 02:19:35,502][105620] Updated weights for policy 1, policy_version 1503939 (0.0006) [2023-12-27 02:19:35,528][105692] Updated weights for policy 0, policy_version 1501277 (0.0010) [2023-12-27 02:19:35,559][105620] Updated weights for policy 1, policy_version 1503949 (0.0006) [2023-12-27 02:19:35,583][105692] Updated weights for policy 0, policy_version 1501287 (0.0010) [2023-12-27 02:19:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 769449984. Throughput: 0: 9519.2, 1: 9887.6. Samples: 769440076. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:19:36,062][104569] Avg episode reward: [(0, '8176.132'), (1, '9169.887')] [2023-12-27 02:19:36,180][105620] Updated weights for policy 1, policy_version 1503959 (0.0005) [2023-12-27 02:19:36,240][105620] Updated weights for policy 1, policy_version 1503969 (0.0005) [2023-12-27 02:19:36,277][105692] Updated weights for policy 0, policy_version 1501297 (0.0011) [2023-12-27 02:19:36,303][105620] Updated weights for policy 1, policy_version 1503979 (0.0009) [2023-12-27 02:19:36,337][105692] Updated weights for policy 0, policy_version 1501307 (0.0010) [2023-12-27 02:19:36,403][105692] Updated weights for policy 0, policy_version 1501317 (0.0007) [2023-12-27 02:19:36,887][105620] Updated weights for policy 1, policy_version 1503989 (0.0006) [2023-12-27 02:19:36,948][105620] Updated weights for policy 1, policy_version 1503999 (0.0006) [2023-12-27 02:19:37,004][105620] Updated weights for policy 1, policy_version 1504009 (0.0005) [2023-12-27 02:19:37,138][105692] Updated weights for policy 0, policy_version 1501327 (0.0011) [2023-12-27 02:19:37,202][105692] Updated weights for policy 0, policy_version 1501337 (0.0011) [2023-12-27 02:19:37,267][105692] Updated weights for policy 0, policy_version 1501347 (0.0010) [2023-12-27 02:19:37,707][105620] Updated weights for policy 1, policy_version 1504019 (0.0007) [2023-12-27 02:19:37,776][105620] Updated weights for policy 1, policy_version 1504029 (0.0008) [2023-12-27 02:19:37,828][105620] Updated weights for policy 1, policy_version 1504039 (0.0008) [2023-12-27 02:19:38,010][105692] Updated weights for policy 0, policy_version 1501357 (0.0011) [2023-12-27 02:19:38,072][105692] Updated weights for policy 0, policy_version 1501367 (0.0010) [2023-12-27 02:19:38,127][105692] Updated weights for policy 0, policy_version 1501377 (0.0010) [2023-12-27 02:19:38,604][105620] Updated weights for policy 1, policy_version 1504049 (0.0008) [2023-12-27 02:19:38,652][105620] Updated weights for policy 1, policy_version 1504059 (0.0008) [2023-12-27 02:19:38,705][105620] Updated weights for policy 1, policy_version 1504069 (0.0008) [2023-12-27 02:19:38,761][105620] Updated weights for policy 1, policy_version 1504079 (0.0008) [2023-12-27 02:19:38,896][105692] Updated weights for policy 0, policy_version 1501387 (0.0011) [2023-12-27 02:19:38,951][105692] Updated weights for policy 0, policy_version 1501397 (0.0010) [2023-12-27 02:19:38,998][105692] Updated weights for policy 0, policy_version 1501407 (0.0010) [2023-12-27 02:19:39,555][105620] Updated weights for policy 1, policy_version 1504089 (0.0008) [2023-12-27 02:19:39,619][105620] Updated weights for policy 1, policy_version 1504099 (0.0008) [2023-12-27 02:19:39,690][105620] Updated weights for policy 1, policy_version 1504109 (0.0008) [2023-12-27 02:19:39,781][105692] Updated weights for policy 0, policy_version 1501417 (0.0010) [2023-12-27 02:19:39,847][105692] Updated weights for policy 0, policy_version 1501427 (0.0011) [2023-12-27 02:19:39,918][105692] Updated weights for policy 0, policy_version 1501437 (0.0008) [2023-12-27 02:19:39,983][105692] Updated weights for policy 0, policy_version 1501447 (0.0006) [2023-12-27 02:19:40,531][105620] Updated weights for policy 1, policy_version 1504119 (0.0008) [2023-12-27 02:19:40,539][105692] Updated weights for policy 0, policy_version 1501457 (0.0006) [2023-12-27 02:19:40,578][105620] Updated weights for policy 1, policy_version 1504129 (0.0008) [2023-12-27 02:19:40,598][105692] Updated weights for policy 0, policy_version 1501467 (0.0007) [2023-12-27 02:19:40,622][105620] Updated weights for policy 1, policy_version 1504139 (0.0008) [2023-12-27 02:19:40,668][105692] Updated weights for policy 0, policy_version 1501477 (0.0007) [2023-12-27 02:19:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 769548288. Throughput: 0: 9573.3, 1: 9967.8. Samples: 769556588. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:19:41,063][104569] Avg episode reward: [(0, '7905.507'), (1, '9259.941')] [2023-12-27 02:19:41,392][105692] Updated weights for policy 0, policy_version 1501487 (0.0008) [2023-12-27 02:19:41,413][105620] Updated weights for policy 1, policy_version 1504149 (0.0007) [2023-12-27 02:19:41,455][105692] Updated weights for policy 0, policy_version 1501497 (0.0007) [2023-12-27 02:19:41,470][105620] Updated weights for policy 1, policy_version 1504159 (0.0008) [2023-12-27 02:19:41,514][105692] Updated weights for policy 0, policy_version 1501507 (0.0008) [2023-12-27 02:19:41,526][105620] Updated weights for policy 1, policy_version 1504169 (0.0008) [2023-12-27 02:19:42,249][105692] Updated weights for policy 0, policy_version 1501517 (0.0008) [2023-12-27 02:19:42,307][105620] Updated weights for policy 1, policy_version 1504179 (0.0007) [2023-12-27 02:19:42,309][105692] Updated weights for policy 0, policy_version 1501527 (0.0008) [2023-12-27 02:19:42,373][105620] Updated weights for policy 1, policy_version 1504189 (0.0009) [2023-12-27 02:19:42,379][105692] Updated weights for policy 0, policy_version 1501537 (0.0007) [2023-12-27 02:19:42,437][105620] Updated weights for policy 1, policy_version 1504199 (0.0007) [2023-12-27 02:19:43,121][105692] Updated weights for policy 0, policy_version 1501547 (0.0006) [2023-12-27 02:19:43,178][105692] Updated weights for policy 0, policy_version 1501557 (0.0005) [2023-12-27 02:19:43,205][105620] Updated weights for policy 1, policy_version 1504209 (0.0009) [2023-12-27 02:19:43,231][105692] Updated weights for policy 0, policy_version 1501567 (0.0005) [2023-12-27 02:19:43,256][105620] Updated weights for policy 1, policy_version 1504219 (0.0007) [2023-12-27 02:19:43,312][105620] Updated weights for policy 1, policy_version 1504229 (0.0009) [2023-12-27 02:19:43,358][105620] Updated weights for policy 1, policy_version 1504239 (0.0008) [2023-12-27 02:19:43,937][105692] Updated weights for policy 0, policy_version 1501577 (0.0006) [2023-12-27 02:19:43,995][105692] Updated weights for policy 0, policy_version 1501587 (0.0009) [2023-12-27 02:19:44,047][105692] Updated weights for policy 0, policy_version 1501597 (0.0007) [2023-12-27 02:19:44,061][105620] Updated weights for policy 1, policy_version 1504249 (0.0007) [2023-12-27 02:19:44,106][105692] Updated weights for policy 0, policy_version 1501607 (0.0007) [2023-12-27 02:19:44,111][105620] Updated weights for policy 1, policy_version 1504259 (0.0008) [2023-12-27 02:19:44,168][105620] Updated weights for policy 1, policy_version 1504269 (0.0009) [2023-12-27 02:19:44,812][105692] Updated weights for policy 0, policy_version 1501617 (0.0009) [2023-12-27 02:19:44,868][105692] Updated weights for policy 0, policy_version 1501627 (0.0007) [2023-12-27 02:19:44,936][105692] Updated weights for policy 0, policy_version 1501637 (0.0008) [2023-12-27 02:19:44,974][105620] Updated weights for policy 1, policy_version 1504279 (0.0011) [2023-12-27 02:19:45,025][105620] Updated weights for policy 1, policy_version 1504289 (0.0011) [2023-12-27 02:19:45,082][105620] Updated weights for policy 1, policy_version 1504299 (0.0011) [2023-12-27 02:19:45,578][105692] Updated weights for policy 0, policy_version 1501647 (0.0008) [2023-12-27 02:19:45,628][105692] Updated weights for policy 0, policy_version 1501657 (0.0008) [2023-12-27 02:19:45,676][105692] Updated weights for policy 0, policy_version 1501667 (0.0008) [2023-12-27 02:19:45,812][105620] Updated weights for policy 1, policy_version 1504309 (0.0009) [2023-12-27 02:19:45,857][105620] Updated weights for policy 1, policy_version 1504319 (0.0007) [2023-12-27 02:19:45,903][105620] Updated weights for policy 1, policy_version 1504329 (0.0005) [2023-12-27 02:19:46,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19387.6, 300 sec: 19521.9). Total num frames: 769646592. Throughput: 0: 9585.3, 1: 9964.4. Samples: 769613340. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:19:46,063][104569] Avg episode reward: [(0, '8091.624'), (1, '9173.697')] [2023-12-27 02:19:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001501672_384483328.pth... [2023-12-27 02:19:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001504336_385163264.pth... [2023-12-27 02:19:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001500552_384196608.pth [2023-12-27 02:19:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001503152_384860160.pth [2023-12-27 02:19:46,290][105692] Updated weights for policy 0, policy_version 1501677 (0.0008) [2023-12-27 02:19:46,344][105692] Updated weights for policy 0, policy_version 1501687 (0.0009) [2023-12-27 02:19:46,396][105692] Updated weights for policy 0, policy_version 1501697 (0.0009) [2023-12-27 02:19:46,544][105620] Updated weights for policy 1, policy_version 1504339 (0.0006) [2023-12-27 02:19:46,612][105620] Updated weights for policy 1, policy_version 1504349 (0.0005) [2023-12-27 02:19:46,670][105620] Updated weights for policy 1, policy_version 1504359 (0.0007) [2023-12-27 02:19:47,178][105692] Updated weights for policy 0, policy_version 1501707 (0.0009) [2023-12-27 02:19:47,230][105692] Updated weights for policy 0, policy_version 1501717 (0.0009) [2023-12-27 02:19:47,280][105620] Updated weights for policy 1, policy_version 1504369 (0.0006) [2023-12-27 02:19:47,282][105692] Updated weights for policy 0, policy_version 1501727 (0.0009) [2023-12-27 02:19:47,335][105620] Updated weights for policy 1, policy_version 1504379 (0.0008) [2023-12-27 02:19:47,389][105620] Updated weights for policy 1, policy_version 1504389 (0.0009) [2023-12-27 02:19:47,440][105620] Updated weights for policy 1, policy_version 1504400 (0.0010) [2023-12-27 02:19:47,872][105692] Updated weights for policy 0, policy_version 1501737 (0.0006) [2023-12-27 02:19:47,921][105692] Updated weights for policy 0, policy_version 1501747 (0.0008) [2023-12-27 02:19:47,973][105692] Updated weights for policy 0, policy_version 1501757 (0.0008) [2023-12-27 02:19:48,030][105692] Updated weights for policy 0, policy_version 1501767 (0.0009) [2023-12-27 02:19:48,156][105620] Updated weights for policy 1, policy_version 1504410 (0.0006) [2023-12-27 02:19:48,221][105620] Updated weights for policy 1, policy_version 1504420 (0.0015) [2023-12-27 02:19:48,289][105620] Updated weights for policy 1, policy_version 1504430 (0.0011) [2023-12-27 02:19:48,857][105692] Updated weights for policy 0, policy_version 1501777 (0.0010) [2023-12-27 02:19:48,912][105692] Updated weights for policy 0, policy_version 1501787 (0.0010) [2023-12-27 02:19:48,924][105620] Updated weights for policy 1, policy_version 1504440 (0.0008) [2023-12-27 02:19:48,969][105692] Updated weights for policy 0, policy_version 1501797 (0.0007) [2023-12-27 02:19:48,982][105620] Updated weights for policy 1, policy_version 1504450 (0.0010) [2023-12-27 02:19:49,034][105620] Updated weights for policy 1, policy_version 1504460 (0.0010) [2023-12-27 02:19:49,591][105692] Updated weights for policy 0, policy_version 1501807 (0.0008) [2023-12-27 02:19:49,636][105692] Updated weights for policy 0, policy_version 1501817 (0.0008) [2023-12-27 02:19:49,681][105692] Updated weights for policy 0, policy_version 1501827 (0.0007) [2023-12-27 02:19:49,812][105620] Updated weights for policy 1, policy_version 1504470 (0.0010) [2023-12-27 02:19:49,880][105620] Updated weights for policy 1, policy_version 1504480 (0.0009) [2023-12-27 02:19:49,943][105620] Updated weights for policy 1, policy_version 1504490 (0.0011) [2023-12-27 02:19:50,396][105692] Updated weights for policy 0, policy_version 1501837 (0.0007) [2023-12-27 02:19:50,452][105692] Updated weights for policy 0, policy_version 1501847 (0.0009) [2023-12-27 02:19:50,499][105692] Updated weights for policy 0, policy_version 1501857 (0.0008) [2023-12-27 02:19:50,707][105620] Updated weights for policy 1, policy_version 1504500 (0.0010) [2023-12-27 02:19:50,768][105620] Updated weights for policy 1, policy_version 1504510 (0.0009) [2023-12-27 02:19:50,834][105620] Updated weights for policy 1, policy_version 1504520 (0.0009) [2023-12-27 02:19:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 769744896. Throughput: 0: 9694.1, 1: 9980.9. Samples: 769733372. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:19:51,062][104569] Avg episode reward: [(0, '7900.085'), (1, '8810.788')] [2023-12-27 02:19:51,221][105692] Updated weights for policy 0, policy_version 1501867 (0.0008) [2023-12-27 02:19:51,285][105692] Updated weights for policy 0, policy_version 1501877 (0.0008) [2023-12-27 02:19:51,344][105692] Updated weights for policy 0, policy_version 1501887 (0.0009) [2023-12-27 02:19:51,636][105620] Updated weights for policy 1, policy_version 1504530 (0.0009) [2023-12-27 02:19:51,696][105620] Updated weights for policy 1, policy_version 1504540 (0.0009) [2023-12-27 02:19:51,762][105620] Updated weights for policy 1, policy_version 1504550 (0.0010) [2023-12-27 02:19:51,823][105620] Updated weights for policy 1, policy_version 1504560 (0.0007) [2023-12-27 02:19:52,084][105692] Updated weights for policy 0, policy_version 1501897 (0.0009) [2023-12-27 02:19:52,143][105692] Updated weights for policy 0, policy_version 1501907 (0.0009) [2023-12-27 02:19:52,209][105692] Updated weights for policy 0, policy_version 1501917 (0.0010) [2023-12-27 02:19:52,274][105692] Updated weights for policy 0, policy_version 1501927 (0.0009) [2023-12-27 02:19:52,599][105620] Updated weights for policy 1, policy_version 1504570 (0.0010) [2023-12-27 02:19:52,655][105620] Updated weights for policy 1, policy_version 1504580 (0.0009) [2023-12-27 02:19:52,716][105620] Updated weights for policy 1, policy_version 1504590 (0.0009) [2023-12-27 02:19:52,955][105692] Updated weights for policy 0, policy_version 1501937 (0.0009) [2023-12-27 02:19:53,018][105692] Updated weights for policy 0, policy_version 1501947 (0.0009) [2023-12-27 02:19:53,074][105692] Updated weights for policy 0, policy_version 1501957 (0.0008) [2023-12-27 02:19:53,498][105620] Updated weights for policy 1, policy_version 1504600 (0.0009) [2023-12-27 02:19:53,560][105620] Updated weights for policy 1, policy_version 1504610 (0.0008) [2023-12-27 02:19:53,617][105620] Updated weights for policy 1, policy_version 1504620 (0.0007) [2023-12-27 02:19:53,823][105692] Updated weights for policy 0, policy_version 1501967 (0.0010) [2023-12-27 02:19:53,888][105692] Updated weights for policy 0, policy_version 1501977 (0.0010) [2023-12-27 02:19:53,951][105692] Updated weights for policy 0, policy_version 1501987 (0.0009) [2023-12-27 02:19:54,253][105620] Updated weights for policy 1, policy_version 1504630 (0.0006) [2023-12-27 02:19:54,302][105620] Updated weights for policy 1, policy_version 1504640 (0.0005) [2023-12-27 02:19:54,360][105620] Updated weights for policy 1, policy_version 1504650 (0.0006) [2023-12-27 02:19:54,730][105692] Updated weights for policy 0, policy_version 1501997 (0.0009) [2023-12-27 02:19:54,778][105692] Updated weights for policy 0, policy_version 1502007 (0.0008) [2023-12-27 02:19:54,825][105692] Updated weights for policy 0, policy_version 1502017 (0.0008) [2023-12-27 02:19:54,909][105620] Updated weights for policy 1, policy_version 1504660 (0.0006) [2023-12-27 02:19:54,956][105620] Updated weights for policy 1, policy_version 1504670 (0.0006) [2023-12-27 02:19:55,016][105620] Updated weights for policy 1, policy_version 1504680 (0.0010) [2023-12-27 02:19:55,667][105620] Updated weights for policy 1, policy_version 1504690 (0.0010) [2023-12-27 02:19:55,704][105692] Updated weights for policy 0, policy_version 1502027 (0.0008) [2023-12-27 02:19:55,729][105620] Updated weights for policy 1, policy_version 1504700 (0.0010) [2023-12-27 02:19:55,751][105692] Updated weights for policy 0, policy_version 1502037 (0.0005) [2023-12-27 02:19:55,773][105620] Updated weights for policy 1, policy_version 1504710 (0.0010) [2023-12-27 02:19:55,810][105692] Updated weights for policy 0, policy_version 1502047 (0.0006) [2023-12-27 02:19:55,820][105620] Updated weights for policy 1, policy_version 1504720 (0.0010) [2023-12-27 02:19:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 769843200. Throughput: 0: 9756.4, 1: 9936.5. Samples: 769848000. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:19:56,063][104569] Avg episode reward: [(0, '8267.256'), (1, '8811.789')] [2023-12-27 02:19:56,498][105620] Updated weights for policy 1, policy_version 1504730 (0.0010) [2023-12-27 02:19:56,532][105692] Updated weights for policy 0, policy_version 1502057 (0.0007) [2023-12-27 02:19:56,560][105620] Updated weights for policy 1, policy_version 1504740 (0.0010) [2023-12-27 02:19:56,583][105692] Updated weights for policy 0, policy_version 1502067 (0.0006) [2023-12-27 02:19:56,622][105620] Updated weights for policy 1, policy_version 1504750 (0.0010) [2023-12-27 02:19:56,636][105692] Updated weights for policy 0, policy_version 1502077 (0.0008) [2023-12-27 02:19:56,696][105692] Updated weights for policy 0, policy_version 1502087 (0.0006) [2023-12-27 02:19:57,349][105620] Updated weights for policy 1, policy_version 1504760 (0.0010) [2023-12-27 02:19:57,356][105692] Updated weights for policy 0, policy_version 1502097 (0.0007) [2023-12-27 02:19:57,402][105692] Updated weights for policy 0, policy_version 1502107 (0.0009) [2023-12-27 02:19:57,411][105620] Updated weights for policy 1, policy_version 1504770 (0.0010) [2023-12-27 02:19:57,452][105692] Updated weights for policy 0, policy_version 1502117 (0.0005) [2023-12-27 02:19:57,462][105620] Updated weights for policy 1, policy_version 1504780 (0.0010) [2023-12-27 02:19:58,163][105692] Updated weights for policy 0, policy_version 1502127 (0.0007) [2023-12-27 02:19:58,169][105620] Updated weights for policy 1, policy_version 1504790 (0.0009) [2023-12-27 02:19:58,225][105692] Updated weights for policy 0, policy_version 1502137 (0.0006) [2023-12-27 02:19:58,230][105620] Updated weights for policy 1, policy_version 1504800 (0.0010) [2023-12-27 02:19:58,285][105692] Updated weights for policy 0, policy_version 1502147 (0.0007) [2023-12-27 02:19:58,290][105620] Updated weights for policy 1, policy_version 1504810 (0.0011) [2023-12-27 02:19:59,054][105692] Updated weights for policy 0, policy_version 1502157 (0.0007) [2023-12-27 02:19:59,102][105692] Updated weights for policy 0, policy_version 1502167 (0.0007) [2023-12-27 02:19:59,137][105620] Updated weights for policy 1, policy_version 1504820 (0.0009) [2023-12-27 02:19:59,156][105692] Updated weights for policy 0, policy_version 1502177 (0.0007) [2023-12-27 02:19:59,195][105620] Updated weights for policy 1, policy_version 1504830 (0.0008) [2023-12-27 02:19:59,264][105620] Updated weights for policy 1, policy_version 1504840 (0.0009) [2023-12-27 02:19:59,932][105620] Updated weights for policy 1, policy_version 1504850 (0.0009) [2023-12-27 02:19:59,995][105620] Updated weights for policy 1, policy_version 1504860 (0.0009) [2023-12-27 02:20:00,005][105692] Updated weights for policy 0, policy_version 1502187 (0.0007) [2023-12-27 02:20:00,057][105620] Updated weights for policy 1, policy_version 1504870 (0.0009) [2023-12-27 02:20:00,067][105586] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000004 [2023-12-27 02:20:00,070][105692] Updated weights for policy 0, policy_version 1502197 (0.0007) [2023-12-27 02:20:00,123][105692] Updated weights for policy 0, policy_version 1502207 (0.0009) [2023-12-27 02:20:00,726][105620] Updated weights for policy 1, policy_version 1504880 (0.0006) [2023-12-27 02:20:00,774][105620] Updated weights for policy 1, policy_version 1504890 (0.0005) [2023-12-27 02:20:00,844][105620] Updated weights for policy 1, policy_version 1504900 (0.0005) [2023-12-27 02:20:00,945][105692] Updated weights for policy 0, policy_version 1502217 (0.0008) [2023-12-27 02:20:01,006][105692] Updated weights for policy 0, policy_version 1502227 (0.0007) [2023-12-27 02:20:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 769933312. Throughput: 0: 9743.4, 1: 9940.1. Samples: 769905932. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:20:01,063][104569] Avg episode reward: [(0, '8086.991'), (1, '9262.697')] [2023-12-27 02:20:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001504904_385310720.pth... [2023-12-27 02:20:01,069][105692] Updated weights for policy 0, policy_version 1502237 (0.0008) [2023-12-27 02:20:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001503760_385015808.pth [2023-12-27 02:20:01,128][105692] Updated weights for policy 0, policy_version 1502247 (0.0009) [2023-12-27 02:20:01,134][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001502248_384630784.pth... [2023-12-27 02:20:01,138][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001501096_384335872.pth [2023-12-27 02:20:01,491][105620] Updated weights for policy 1, policy_version 1504910 (0.0008) [2023-12-27 02:20:01,553][105620] Updated weights for policy 1, policy_version 1504920 (0.0010) [2023-12-27 02:20:01,612][105620] Updated weights for policy 1, policy_version 1504930 (0.0010) [2023-12-27 02:20:01,876][105692] Updated weights for policy 0, policy_version 1502257 (0.0008) [2023-12-27 02:20:01,930][105692] Updated weights for policy 0, policy_version 1502267 (0.0008) [2023-12-27 02:20:01,988][105692] Updated weights for policy 0, policy_version 1502277 (0.0007) [2023-12-27 02:20:02,295][105620] Updated weights for policy 1, policy_version 1504940 (0.0010) [2023-12-27 02:20:02,354][105620] Updated weights for policy 1, policy_version 1504950 (0.0007) [2023-12-27 02:20:02,413][105620] Updated weights for policy 1, policy_version 1504960 (0.0009) [2023-12-27 02:20:02,640][105692] Updated weights for policy 0, policy_version 1502287 (0.0007) [2023-12-27 02:20:02,690][105692] Updated weights for policy 0, policy_version 1502297 (0.0008) [2023-12-27 02:20:02,744][105692] Updated weights for policy 0, policy_version 1502307 (0.0010) [2023-12-27 02:20:03,179][105620] Updated weights for policy 1, policy_version 1504970 (0.0009) [2023-12-27 02:20:03,244][105620] Updated weights for policy 1, policy_version 1504980 (0.0007) [2023-12-27 02:20:03,306][105620] Updated weights for policy 1, policy_version 1504990 (0.0008) [2023-12-27 02:20:03,367][105620] Updated weights for policy 1, policy_version 1505000 (0.0009) [2023-12-27 02:20:03,474][105692] Updated weights for policy 0, policy_version 1502317 (0.0010) [2023-12-27 02:20:03,527][105692] Updated weights for policy 0, policy_version 1502327 (0.0010) [2023-12-27 02:20:03,586][105692] Updated weights for policy 0, policy_version 1502337 (0.0010) [2023-12-27 02:20:04,147][105620] Updated weights for policy 1, policy_version 1505010 (0.0008) [2023-12-27 02:20:04,216][105620] Updated weights for policy 1, policy_version 1505020 (0.0009) [2023-12-27 02:20:04,276][105620] Updated weights for policy 1, policy_version 1505030 (0.0010) [2023-12-27 02:20:04,285][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000006 [2023-12-27 02:20:04,319][105692] Updated weights for policy 0, policy_version 1502347 (0.0010) [2023-12-27 02:20:04,366][105692] Updated weights for policy 0, policy_version 1502357 (0.0010) [2023-12-27 02:20:04,422][105692] Updated weights for policy 0, policy_version 1502367 (0.0010) [2023-12-27 02:20:04,986][105620] Updated weights for policy 1, policy_version 1505040 (0.0008) [2023-12-27 02:20:05,041][105620] Updated weights for policy 1, policy_version 1505050 (0.0008) [2023-12-27 02:20:05,100][105620] Updated weights for policy 1, policy_version 1505060 (0.0007) [2023-12-27 02:20:05,186][105692] Updated weights for policy 0, policy_version 1502377 (0.0011) [2023-12-27 02:20:05,251][105692] Updated weights for policy 0, policy_version 1502387 (0.0010) [2023-12-27 02:20:05,308][105692] Updated weights for policy 0, policy_version 1502397 (0.0010) [2023-12-27 02:20:05,363][105692] Updated weights for policy 0, policy_version 1502407 (0.0010) [2023-12-27 02:20:05,769][105620] Updated weights for policy 1, policy_version 1505070 (0.0008) [2023-12-27 02:20:05,823][105620] Updated weights for policy 1, policy_version 1505080 (0.0010) [2023-12-27 02:20:05,871][105620] Updated weights for policy 1, policy_version 1505090 (0.0010) [2023-12-27 02:20:06,029][105692] Updated weights for policy 0, policy_version 1502417 (0.0006) [2023-12-27 02:20:06,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 770031616. Throughput: 0: 9674.5, 1: 9920.7. Samples: 770020276. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:20:06,062][104569] Avg episode reward: [(0, '8355.637'), (1, '9352.106')] [2023-12-27 02:20:06,076][105692] Updated weights for policy 0, policy_version 1502427 (0.0005) [2023-12-27 02:20:06,138][105692] Updated weights for policy 0, policy_version 1502437 (0.0009) [2023-12-27 02:20:06,594][105620] Updated weights for policy 1, policy_version 1505100 (0.0010) [2023-12-27 02:20:06,650][105620] Updated weights for policy 1, policy_version 1505110 (0.0010) [2023-12-27 02:20:06,702][105620] Updated weights for policy 1, policy_version 1505120 (0.0010) [2023-12-27 02:20:06,836][105692] Updated weights for policy 0, policy_version 1502447 (0.0009) [2023-12-27 02:20:06,891][105692] Updated weights for policy 0, policy_version 1502457 (0.0008) [2023-12-27 02:20:06,943][105692] Updated weights for policy 0, policy_version 1502467 (0.0008) [2023-12-27 02:20:07,464][105620] Updated weights for policy 1, policy_version 1505130 (0.0010) [2023-12-27 02:20:07,508][105620] Updated weights for policy 1, policy_version 1505140 (0.0010) [2023-12-27 02:20:07,570][105620] Updated weights for policy 1, policy_version 1505150 (0.0010) [2023-12-27 02:20:07,633][105620] Updated weights for policy 1, policy_version 1505160 (0.0005) [2023-12-27 02:20:07,719][105692] Updated weights for policy 0, policy_version 1502477 (0.0007) [2023-12-27 02:20:07,771][105692] Updated weights for policy 0, policy_version 1502487 (0.0005) [2023-12-27 02:20:07,827][105692] Updated weights for policy 0, policy_version 1502497 (0.0005) [2023-12-27 02:20:08,334][105620] Updated weights for policy 1, policy_version 1505170 (0.0008) [2023-12-27 02:20:08,398][105620] Updated weights for policy 1, policy_version 1505180 (0.0007) [2023-12-27 02:20:08,463][105620] Updated weights for policy 1, policy_version 1505190 (0.0005) [2023-12-27 02:20:08,484][105692] Updated weights for policy 0, policy_version 1502507 (0.0007) [2023-12-27 02:20:08,548][105692] Updated weights for policy 0, policy_version 1502517 (0.0010) [2023-12-27 02:20:08,609][105692] Updated weights for policy 0, policy_version 1502527 (0.0010) [2023-12-27 02:20:09,138][105620] Updated weights for policy 1, policy_version 1505200 (0.0008) [2023-12-27 02:20:09,198][105620] Updated weights for policy 1, policy_version 1505210 (0.0008) [2023-12-27 02:20:09,264][105620] Updated weights for policy 1, policy_version 1505220 (0.0008) [2023-12-27 02:20:09,386][105692] Updated weights for policy 0, policy_version 1502537 (0.0009) [2023-12-27 02:20:09,449][105692] Updated weights for policy 0, policy_version 1502547 (0.0008) [2023-12-27 02:20:09,506][105692] Updated weights for policy 0, policy_version 1502557 (0.0006) [2023-12-27 02:20:09,558][105692] Updated weights for policy 0, policy_version 1502567 (0.0006) [2023-12-27 02:20:10,054][105620] Updated weights for policy 1, policy_version 1505230 (0.0009) [2023-12-27 02:20:10,117][105620] Updated weights for policy 1, policy_version 1505240 (0.0008) [2023-12-27 02:20:10,178][105620] Updated weights for policy 1, policy_version 1505250 (0.0008) [2023-12-27 02:20:10,282][105692] Updated weights for policy 0, policy_version 1502577 (0.0009) [2023-12-27 02:20:10,343][105692] Updated weights for policy 0, policy_version 1502587 (0.0008) [2023-12-27 02:20:10,402][105692] Updated weights for policy 0, policy_version 1502597 (0.0009) [2023-12-27 02:20:10,931][105620] Updated weights for policy 1, policy_version 1505260 (0.0009) [2023-12-27 02:20:10,993][105620] Updated weights for policy 1, policy_version 1505270 (0.0008) [2023-12-27 02:20:11,059][105620] Updated weights for policy 1, policy_version 1505280 (0.0007) [2023-12-27 02:20:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 770121728. Throughput: 0: 9732.2, 1: 9867.4. Samples: 770136688. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:20:11,063][104569] Avg episode reward: [(0, '8813.968'), (1, '9260.222')] [2023-12-27 02:20:11,073][105692] Updated weights for policy 0, policy_version 1502607 (0.0009) [2023-12-27 02:20:11,137][105692] Updated weights for policy 0, policy_version 1502617 (0.0009) [2023-12-27 02:20:11,198][105692] Updated weights for policy 0, policy_version 1502627 (0.0009) [2023-12-27 02:20:11,726][105620] Updated weights for policy 1, policy_version 1505290 (0.0006) [2023-12-27 02:20:11,785][105620] Updated weights for policy 1, policy_version 1505300 (0.0008) [2023-12-27 02:20:11,841][105620] Updated weights for policy 1, policy_version 1505310 (0.0008) [2023-12-27 02:20:11,903][105620] Updated weights for policy 1, policy_version 1505320 (0.0009) [2023-12-27 02:20:12,031][105692] Updated weights for policy 0, policy_version 1502637 (0.0010) [2023-12-27 02:20:12,088][105692] Updated weights for policy 0, policy_version 1502647 (0.0011) [2023-12-27 02:20:12,140][105692] Updated weights for policy 0, policy_version 1502657 (0.0011) [2023-12-27 02:20:12,560][105620] Updated weights for policy 1, policy_version 1505330 (0.0010) [2023-12-27 02:20:12,617][105620] Updated weights for policy 1, policy_version 1505340 (0.0011) [2023-12-27 02:20:12,666][105620] Updated weights for policy 1, policy_version 1505350 (0.0011) [2023-12-27 02:20:12,844][105692] Updated weights for policy 0, policy_version 1502667 (0.0009) [2023-12-27 02:20:12,895][105692] Updated weights for policy 0, policy_version 1502677 (0.0007) [2023-12-27 02:20:12,947][105692] Updated weights for policy 0, policy_version 1502687 (0.0011) [2023-12-27 02:20:13,369][105620] Updated weights for policy 1, policy_version 1505360 (0.0007) [2023-12-27 02:20:13,426][105620] Updated weights for policy 1, policy_version 1505370 (0.0007) [2023-12-27 02:20:13,494][105620] Updated weights for policy 1, policy_version 1505380 (0.0005) [2023-12-27 02:20:13,551][105692] Updated weights for policy 0, policy_version 1502697 (0.0010) [2023-12-27 02:20:13,617][105692] Updated weights for policy 0, policy_version 1502707 (0.0010) [2023-12-27 02:20:13,671][105692] Updated weights for policy 0, policy_version 1502717 (0.0007) [2023-12-27 02:20:13,723][105692] Updated weights for policy 0, policy_version 1502727 (0.0011) [2023-12-27 02:20:14,004][105620] Updated weights for policy 1, policy_version 1505390 (0.0005) [2023-12-27 02:20:14,063][105620] Updated weights for policy 1, policy_version 1505400 (0.0008) [2023-12-27 02:20:14,124][105620] Updated weights for policy 1, policy_version 1505410 (0.0009) [2023-12-27 02:20:14,430][105692] Updated weights for policy 0, policy_version 1502737 (0.0006) [2023-12-27 02:20:14,486][105692] Updated weights for policy 0, policy_version 1502747 (0.0010) [2023-12-27 02:20:14,541][105692] Updated weights for policy 0, policy_version 1502757 (0.0010) [2023-12-27 02:20:14,755][105620] Updated weights for policy 1, policy_version 1505420 (0.0008) [2023-12-27 02:20:14,820][105620] Updated weights for policy 1, policy_version 1505430 (0.0008) [2023-12-27 02:20:14,879][105620] Updated weights for policy 1, policy_version 1505440 (0.0008) [2023-12-27 02:20:15,255][105692] Updated weights for policy 0, policy_version 1502767 (0.0010) [2023-12-27 02:20:15,318][105692] Updated weights for policy 0, policy_version 1502777 (0.0011) [2023-12-27 02:20:15,383][105692] Updated weights for policy 0, policy_version 1502787 (0.0011) [2023-12-27 02:20:15,652][105620] Updated weights for policy 1, policy_version 1505450 (0.0008) [2023-12-27 02:20:15,705][105620] Updated weights for policy 1, policy_version 1505460 (0.0008) [2023-12-27 02:20:15,761][105620] Updated weights for policy 1, policy_version 1505470 (0.0008) [2023-12-27 02:20:15,816][105620] Updated weights for policy 1, policy_version 1505480 (0.0008) [2023-12-27 02:20:16,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19660.7, 300 sec: 19494.2). Total num frames: 770228224. Throughput: 0: 9665.1, 1: 9893.6. Samples: 770199200. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:20:16,063][104569] Avg episode reward: [(0, '8724.319'), (1, '9260.292')] [2023-12-27 02:20:16,074][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001505480_385458176.pth... [2023-12-27 02:20:16,074][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001502792_384770048.pth... [2023-12-27 02:20:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001504336_385163264.pth [2023-12-27 02:20:16,080][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001501672_384483328.pth [2023-12-27 02:20:16,119][105692] Updated weights for policy 0, policy_version 1502797 (0.0011) [2023-12-27 02:20:16,163][105692] Updated weights for policy 0, policy_version 1502807 (0.0010) [2023-12-27 02:20:16,217][105692] Updated weights for policy 0, policy_version 1502817 (0.0010) [2023-12-27 02:20:16,575][105620] Updated weights for policy 1, policy_version 1505490 (0.0008) [2023-12-27 02:20:16,622][105620] Updated weights for policy 1, policy_version 1505500 (0.0008) [2023-12-27 02:20:16,673][105620] Updated weights for policy 1, policy_version 1505510 (0.0009) [2023-12-27 02:20:16,961][105692] Updated weights for policy 0, policy_version 1502827 (0.0010) [2023-12-27 02:20:17,009][105692] Updated weights for policy 0, policy_version 1502837 (0.0009) [2023-12-27 02:20:17,056][105692] Updated weights for policy 0, policy_version 1502847 (0.0008) [2023-12-27 02:20:17,450][105620] Updated weights for policy 1, policy_version 1505520 (0.0009) [2023-12-27 02:20:17,508][105620] Updated weights for policy 1, policy_version 1505530 (0.0009) [2023-12-27 02:20:17,559][105620] Updated weights for policy 1, policy_version 1505540 (0.0008) [2023-12-27 02:20:17,836][105692] Updated weights for policy 0, policy_version 1502857 (0.0009) [2023-12-27 02:20:17,898][105692] Updated weights for policy 0, policy_version 1502867 (0.0009) [2023-12-27 02:20:17,954][105692] Updated weights for policy 0, policy_version 1502877 (0.0008) [2023-12-27 02:20:18,014][105692] Updated weights for policy 0, policy_version 1502887 (0.0007) [2023-12-27 02:20:18,322][105620] Updated weights for policy 1, policy_version 1505550 (0.0009) [2023-12-27 02:20:18,383][105620] Updated weights for policy 1, policy_version 1505560 (0.0009) [2023-12-27 02:20:18,445][105620] Updated weights for policy 1, policy_version 1505570 (0.0008) [2023-12-27 02:20:18,738][105692] Updated weights for policy 0, policy_version 1502897 (0.0006) [2023-12-27 02:20:18,796][105692] Updated weights for policy 0, policy_version 1502907 (0.0007) [2023-12-27 02:20:18,860][105692] Updated weights for policy 0, policy_version 1502917 (0.0006) [2023-12-27 02:20:19,283][105620] Updated weights for policy 1, policy_version 1505580 (0.0007) [2023-12-27 02:20:19,349][105620] Updated weights for policy 1, policy_version 1505590 (0.0008) [2023-12-27 02:20:19,417][105620] Updated weights for policy 1, policy_version 1505600 (0.0006) [2023-12-27 02:20:19,487][105692] Updated weights for policy 0, policy_version 1502927 (0.0008) [2023-12-27 02:20:19,554][105692] Updated weights for policy 0, policy_version 1502937 (0.0009) [2023-12-27 02:20:19,614][105692] Updated weights for policy 0, policy_version 1502947 (0.0010) [2023-12-27 02:20:20,098][105620] Updated weights for policy 1, policy_version 1505610 (0.0006) [2023-12-27 02:20:20,170][105620] Updated weights for policy 1, policy_version 1505620 (0.0007) [2023-12-27 02:20:20,226][105620] Updated weights for policy 1, policy_version 1505630 (0.0009) [2023-12-27 02:20:20,294][105620] Updated weights for policy 1, policy_version 1505640 (0.0006) [2023-12-27 02:20:20,380][105692] Updated weights for policy 0, policy_version 1502957 (0.0009) [2023-12-27 02:20:20,446][105692] Updated weights for policy 0, policy_version 1502967 (0.0009) [2023-12-27 02:20:20,498][105692] Updated weights for policy 0, policy_version 1502977 (0.0010) [2023-12-27 02:20:20,957][105620] Updated weights for policy 1, policy_version 1505650 (0.0008) [2023-12-27 02:20:21,016][105620] Updated weights for policy 1, policy_version 1505660 (0.0008) [2023-12-27 02:20:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 770318336. Throughput: 0: 9667.1, 1: 9732.4. Samples: 770313052. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:20:21,062][104569] Avg episode reward: [(0, '8361.939'), (1, '9260.168')] [2023-12-27 02:20:21,079][105620] Updated weights for policy 1, policy_version 1505670 (0.0008) [2023-12-27 02:20:21,291][105692] Updated weights for policy 0, policy_version 1502987 (0.0009) [2023-12-27 02:20:21,359][105692] Updated weights for policy 0, policy_version 1502997 (0.0009) [2023-12-27 02:20:21,429][105692] Updated weights for policy 0, policy_version 1503007 (0.0009) [2023-12-27 02:20:21,843][105620] Updated weights for policy 1, policy_version 1505680 (0.0009) [2023-12-27 02:20:21,906][105620] Updated weights for policy 1, policy_version 1505690 (0.0009) [2023-12-27 02:20:21,970][105620] Updated weights for policy 1, policy_version 1505700 (0.0009) [2023-12-27 02:20:22,247][105692] Updated weights for policy 0, policy_version 1503017 (0.0010) [2023-12-27 02:20:22,318][105692] Updated weights for policy 0, policy_version 1503027 (0.0010) [2023-12-27 02:20:22,387][105692] Updated weights for policy 0, policy_version 1503037 (0.0010) [2023-12-27 02:20:22,438][105692] Updated weights for policy 0, policy_version 1503047 (0.0011) [2023-12-27 02:20:22,762][105620] Updated weights for policy 1, policy_version 1505710 (0.0009) [2023-12-27 02:20:22,826][105620] Updated weights for policy 1, policy_version 1505720 (0.0008) [2023-12-27 02:20:22,890][105620] Updated weights for policy 1, policy_version 1505730 (0.0009) [2023-12-27 02:20:23,193][105692] Updated weights for policy 0, policy_version 1503057 (0.0009) [2023-12-27 02:20:23,252][105692] Updated weights for policy 0, policy_version 1503067 (0.0009) [2023-12-27 02:20:23,306][105692] Updated weights for policy 0, policy_version 1503077 (0.0008) [2023-12-27 02:20:23,642][105620] Updated weights for policy 1, policy_version 1505740 (0.0009) [2023-12-27 02:20:23,692][105620] Updated weights for policy 1, policy_version 1505750 (0.0006) [2023-12-27 02:20:23,743][105620] Updated weights for policy 1, policy_version 1505760 (0.0009) [2023-12-27 02:20:24,092][105692] Updated weights for policy 0, policy_version 1503087 (0.0008) [2023-12-27 02:20:24,153][105692] Updated weights for policy 0, policy_version 1503097 (0.0009) [2023-12-27 02:20:24,212][105692] Updated weights for policy 0, policy_version 1503107 (0.0009) [2023-12-27 02:20:24,403][105620] Updated weights for policy 1, policy_version 1505770 (0.0007) [2023-12-27 02:20:24,468][105620] Updated weights for policy 1, policy_version 1505780 (0.0008) [2023-12-27 02:20:24,536][105620] Updated weights for policy 1, policy_version 1505790 (0.0008) [2023-12-27 02:20:24,603][105620] Updated weights for policy 1, policy_version 1505800 (0.0007) [2023-12-27 02:20:24,870][105692] Updated weights for policy 0, policy_version 1503117 (0.0007) [2023-12-27 02:20:24,924][105692] Updated weights for policy 0, policy_version 1503127 (0.0006) [2023-12-27 02:20:24,976][105692] Updated weights for policy 0, policy_version 1503137 (0.0008) [2023-12-27 02:20:25,326][105620] Updated weights for policy 1, policy_version 1505810 (0.0006) [2023-12-27 02:20:25,377][105620] Updated weights for policy 1, policy_version 1505820 (0.0005) [2023-12-27 02:20:25,440][105620] Updated weights for policy 1, policy_version 1505830 (0.0006) [2023-12-27 02:20:25,643][105692] Updated weights for policy 0, policy_version 1503147 (0.0007) [2023-12-27 02:20:25,697][105692] Updated weights for policy 0, policy_version 1503157 (0.0005) [2023-12-27 02:20:25,758][105692] Updated weights for policy 0, policy_version 1503167 (0.0005) [2023-12-27 02:20:25,972][105620] Updated weights for policy 1, policy_version 1505840 (0.0005) [2023-12-27 02:20:26,022][105620] Updated weights for policy 1, policy_version 1505850 (0.0008) [2023-12-27 02:20:26,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 770416640. Throughput: 0: 9610.7, 1: 9740.9. Samples: 770427408. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:20:26,063][104569] Avg episode reward: [(0, '8269.836'), (1, '9168.968')] [2023-12-27 02:20:26,073][105620] Updated weights for policy 1, policy_version 1505860 (0.0009) [2023-12-27 02:20:26,328][105692] Updated weights for policy 0, policy_version 1503177 (0.0005) [2023-12-27 02:20:26,389][105692] Updated weights for policy 0, policy_version 1503187 (0.0008) [2023-12-27 02:20:26,454][105692] Updated weights for policy 0, policy_version 1503197 (0.0007) [2023-12-27 02:20:26,512][105692] Updated weights for policy 0, policy_version 1503207 (0.0006) [2023-12-27 02:20:26,800][105620] Updated weights for policy 1, policy_version 1505871 (0.0009) [2023-12-27 02:20:26,850][105620] Updated weights for policy 1, policy_version 1505881 (0.0006) [2023-12-27 02:20:26,898][105620] Updated weights for policy 1, policy_version 1505891 (0.0005) [2023-12-27 02:20:27,190][105692] Updated weights for policy 0, policy_version 1503217 (0.0009) [2023-12-27 02:20:27,238][105692] Updated weights for policy 0, policy_version 1503227 (0.0008) [2023-12-27 02:20:27,287][105692] Updated weights for policy 0, policy_version 1503237 (0.0009) [2023-12-27 02:20:27,582][105620] Updated weights for policy 1, policy_version 1505901 (0.0007) [2023-12-27 02:20:27,647][105620] Updated weights for policy 1, policy_version 1505911 (0.0009) [2023-12-27 02:20:27,708][105620] Updated weights for policy 1, policy_version 1505921 (0.0009) [2023-12-27 02:20:28,089][105692] Updated weights for policy 0, policy_version 1503247 (0.0009) [2023-12-27 02:20:28,140][105692] Updated weights for policy 0, policy_version 1503257 (0.0009) [2023-12-27 02:20:28,186][105692] Updated weights for policy 0, policy_version 1503267 (0.0009) [2023-12-27 02:20:28,375][105620] Updated weights for policy 1, policy_version 1505931 (0.0009) [2023-12-27 02:20:28,434][105620] Updated weights for policy 1, policy_version 1505941 (0.0009) [2023-12-27 02:20:28,496][105620] Updated weights for policy 1, policy_version 1505951 (0.0009) [2023-12-27 02:20:28,999][105692] Updated weights for policy 0, policy_version 1503277 (0.0009) [2023-12-27 02:20:29,053][105692] Updated weights for policy 0, policy_version 1503287 (0.0009) [2023-12-27 02:20:29,104][105692] Updated weights for policy 0, policy_version 1503297 (0.0009) [2023-12-27 02:20:29,201][105620] Updated weights for policy 1, policy_version 1505961 (0.0009) [2023-12-27 02:20:29,267][105620] Updated weights for policy 1, policy_version 1505971 (0.0009) [2023-12-27 02:20:29,325][105620] Updated weights for policy 1, policy_version 1505981 (0.0008) [2023-12-27 02:20:29,394][105620] Updated weights for policy 1, policy_version 1505991 (0.0008) [2023-12-27 02:20:29,882][105692] Updated weights for policy 0, policy_version 1503307 (0.0009) [2023-12-27 02:20:29,948][105692] Updated weights for policy 0, policy_version 1503317 (0.0009) [2023-12-27 02:20:30,007][105692] Updated weights for policy 0, policy_version 1503327 (0.0009) [2023-12-27 02:20:30,142][105620] Updated weights for policy 1, policy_version 1506001 (0.0006) [2023-12-27 02:20:30,211][105620] Updated weights for policy 1, policy_version 1506011 (0.0007) [2023-12-27 02:20:30,269][105620] Updated weights for policy 1, policy_version 1506021 (0.0009) [2023-12-27 02:20:30,801][105692] Updated weights for policy 0, policy_version 1503337 (0.0010) [2023-12-27 02:20:30,857][105692] Updated weights for policy 0, policy_version 1503347 (0.0008) [2023-12-27 02:20:30,904][105692] Updated weights for policy 0, policy_version 1503357 (0.0008) [2023-12-27 02:20:30,955][105692] Updated weights for policy 0, policy_version 1503367 (0.0007) [2023-12-27 02:20:30,957][105620] Updated weights for policy 1, policy_version 1506031 (0.0009) [2023-12-27 02:20:31,003][105620] Updated weights for policy 1, policy_version 1506041 (0.0008) [2023-12-27 02:20:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 770514944. Throughput: 0: 9631.8, 1: 9786.5. Samples: 770487160. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:20:31,063][104569] Avg episode reward: [(0, '8356.686'), (1, '9261.454')] [2023-12-27 02:20:31,067][105620] Updated weights for policy 1, policy_version 1506051 (0.0009) [2023-12-27 02:20:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001503368_384917504.pth... [2023-12-27 02:20:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001502248_384630784.pth [2023-12-27 02:20:31,098][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001506056_385605632.pth... [2023-12-27 02:20:31,103][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001504904_385310720.pth [2023-12-27 02:20:31,702][105692] Updated weights for policy 0, policy_version 1503377 (0.0008) [2023-12-27 02:20:31,769][105692] Updated weights for policy 0, policy_version 1503387 (0.0008) [2023-12-27 02:20:31,827][105692] Updated weights for policy 0, policy_version 1503397 (0.0009) [2023-12-27 02:20:31,880][105620] Updated weights for policy 1, policy_version 1506061 (0.0010) [2023-12-27 02:20:31,932][105620] Updated weights for policy 1, policy_version 1506071 (0.0010) [2023-12-27 02:20:31,980][105620] Updated weights for policy 1, policy_version 1506081 (0.0010) [2023-12-27 02:20:32,620][105692] Updated weights for policy 0, policy_version 1503407 (0.0008) [2023-12-27 02:20:32,665][105620] Updated weights for policy 1, policy_version 1506091 (0.0010) [2023-12-27 02:20:32,682][105692] Updated weights for policy 0, policy_version 1503417 (0.0008) [2023-12-27 02:20:32,716][105620] Updated weights for policy 1, policy_version 1506101 (0.0010) [2023-12-27 02:20:32,738][105692] Updated weights for policy 0, policy_version 1503427 (0.0005) [2023-12-27 02:20:32,771][105620] Updated weights for policy 1, policy_version 1506111 (0.0010) [2023-12-27 02:20:33,413][105620] Updated weights for policy 1, policy_version 1506121 (0.0010) [2023-12-27 02:20:33,460][105620] Updated weights for policy 1, policy_version 1506131 (0.0010) [2023-12-27 02:20:33,478][105692] Updated weights for policy 0, policy_version 1503437 (0.0005) [2023-12-27 02:20:33,511][105620] Updated weights for policy 1, policy_version 1506141 (0.0010) [2023-12-27 02:20:33,533][105692] Updated weights for policy 0, policy_version 1503447 (0.0005) [2023-12-27 02:20:33,565][105620] Updated weights for policy 1, policy_version 1506151 (0.0010) [2023-12-27 02:20:33,595][105692] Updated weights for policy 0, policy_version 1503457 (0.0006) [2023-12-27 02:20:34,314][105692] Updated weights for policy 0, policy_version 1503467 (0.0009) [2023-12-27 02:20:34,333][105620] Updated weights for policy 1, policy_version 1506161 (0.0008) [2023-12-27 02:20:34,364][105692] Updated weights for policy 0, policy_version 1503477 (0.0006) [2023-12-27 02:20:34,387][105620] Updated weights for policy 1, policy_version 1506171 (0.0007) [2023-12-27 02:20:34,402][105585] KL-divergence is very high: 101.6120 [2023-12-27 02:20:34,413][105692] Updated weights for policy 0, policy_version 1503487 (0.0006) [2023-12-27 02:20:34,453][105620] Updated weights for policy 1, policy_version 1506181 (0.0008) [2023-12-27 02:20:35,143][105620] Updated weights for policy 1, policy_version 1506191 (0.0007) [2023-12-27 02:20:35,193][105692] Updated weights for policy 0, policy_version 1503497 (0.0007) [2023-12-27 02:20:35,220][105620] Updated weights for policy 1, policy_version 1506201 (0.0008) [2023-12-27 02:20:35,247][105692] Updated weights for policy 0, policy_version 1503507 (0.0006) [2023-12-27 02:20:35,276][105620] Updated weights for policy 1, policy_version 1506211 (0.0008) [2023-12-27 02:20:35,296][105692] Updated weights for policy 0, policy_version 1503517 (0.0007) [2023-12-27 02:20:35,345][105692] Updated weights for policy 0, policy_version 1503527 (0.0009) [2023-12-27 02:20:36,017][105692] Updated weights for policy 0, policy_version 1503537 (0.0009) [2023-12-27 02:20:36,057][105620] Updated weights for policy 1, policy_version 1506221 (0.0008) [2023-12-27 02:20:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 770605056. Throughput: 0: 9517.8, 1: 9766.0. Samples: 770601144. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:20:36,063][104569] Avg episode reward: [(0, '7813.114'), (1, '9353.079')] [2023-12-27 02:20:36,064][105692] Updated weights for policy 0, policy_version 1503547 (0.0008) [2023-12-27 02:20:36,114][105692] Updated weights for policy 0, policy_version 1503557 (0.0006) [2023-12-27 02:20:36,118][105620] Updated weights for policy 1, policy_version 1506231 (0.0009) [2023-12-27 02:20:36,167][105620] Updated weights for policy 1, policy_version 1506241 (0.0007) [2023-12-27 02:20:36,857][105620] Updated weights for policy 1, policy_version 1506251 (0.0011) [2023-12-27 02:20:36,909][105620] Updated weights for policy 1, policy_version 1506261 (0.0010) [2023-12-27 02:20:36,965][105620] Updated weights for policy 1, policy_version 1506271 (0.0010) [2023-12-27 02:20:36,990][105692] Updated weights for policy 0, policy_version 1503567 (0.0010) [2023-12-27 02:20:37,049][105692] Updated weights for policy 0, policy_version 1503577 (0.0011) [2023-12-27 02:20:37,111][105692] Updated weights for policy 0, policy_version 1503587 (0.0010) [2023-12-27 02:20:37,720][105620] Updated weights for policy 1, policy_version 1506281 (0.0011) [2023-12-27 02:20:37,779][105620] Updated weights for policy 1, policy_version 1506291 (0.0010) [2023-12-27 02:20:37,807][105692] Updated weights for policy 0, policy_version 1503597 (0.0008) [2023-12-27 02:20:37,839][105620] Updated weights for policy 1, policy_version 1506301 (0.0010) [2023-12-27 02:20:37,884][105692] Updated weights for policy 0, policy_version 1503607 (0.0010) [2023-12-27 02:20:37,897][105620] Updated weights for policy 1, policy_version 1506311 (0.0010) [2023-12-27 02:20:37,945][105692] Updated weights for policy 0, policy_version 1503617 (0.0010) [2023-12-27 02:20:38,636][105620] Updated weights for policy 1, policy_version 1506321 (0.0008) [2023-12-27 02:20:38,653][105692] Updated weights for policy 0, policy_version 1503627 (0.0010) [2023-12-27 02:20:38,695][105620] Updated weights for policy 1, policy_version 1506331 (0.0006) [2023-12-27 02:20:38,708][105692] Updated weights for policy 0, policy_version 1503637 (0.0010) [2023-12-27 02:20:38,749][105620] Updated weights for policy 1, policy_version 1506341 (0.0005) [2023-12-27 02:20:38,762][105692] Updated weights for policy 0, policy_version 1503647 (0.0010) [2023-12-27 02:20:39,542][105692] Updated weights for policy 0, policy_version 1503657 (0.0010) [2023-12-27 02:20:39,568][105620] Updated weights for policy 1, policy_version 1506351 (0.0006) [2023-12-27 02:20:39,605][105692] Updated weights for policy 0, policy_version 1503667 (0.0010) [2023-12-27 02:20:39,627][105620] Updated weights for policy 1, policy_version 1506361 (0.0007) [2023-12-27 02:20:39,650][105692] Updated weights for policy 0, policy_version 1503677 (0.0010) [2023-12-27 02:20:39,687][105620] Updated weights for policy 1, policy_version 1506371 (0.0005) [2023-12-27 02:20:39,705][105692] Updated weights for policy 0, policy_version 1503687 (0.0010) [2023-12-27 02:20:40,444][105692] Updated weights for policy 0, policy_version 1503697 (0.0006) [2023-12-27 02:20:40,509][105692] Updated weights for policy 0, policy_version 1503707 (0.0009) [2023-12-27 02:20:40,519][105620] Updated weights for policy 1, policy_version 1506381 (0.0006) [2023-12-27 02:20:40,571][105692] Updated weights for policy 0, policy_version 1503717 (0.0007) [2023-12-27 02:20:40,584][105620] Updated weights for policy 1, policy_version 1506391 (0.0008) [2023-12-27 02:20:40,643][105620] Updated weights for policy 1, policy_version 1506401 (0.0011) [2023-12-27 02:20:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 770703360. Throughput: 0: 9537.4, 1: 9692.0. Samples: 770713324. Policy #0 lag: (min: 13.0, avg: 14.5, max: 45.0) [2023-12-27 02:20:41,063][104569] Avg episode reward: [(0, '7634.922'), (1, '9261.038')] [2023-12-27 02:20:41,169][105692] Updated weights for policy 0, policy_version 1503727 (0.0007) [2023-12-27 02:20:41,236][105692] Updated weights for policy 0, policy_version 1503737 (0.0008) [2023-12-27 02:20:41,300][105692] Updated weights for policy 0, policy_version 1503747 (0.0006) [2023-12-27 02:20:41,422][105620] Updated weights for policy 1, policy_version 1506411 (0.0010) [2023-12-27 02:20:41,483][105620] Updated weights for policy 1, policy_version 1506421 (0.0009) [2023-12-27 02:20:41,546][105620] Updated weights for policy 1, policy_version 1506431 (0.0009) [2023-12-27 02:20:41,975][105692] Updated weights for policy 0, policy_version 1503757 (0.0007) [2023-12-27 02:20:42,036][105692] Updated weights for policy 0, policy_version 1503767 (0.0008) [2023-12-27 02:20:42,088][105692] Updated weights for policy 0, policy_version 1503777 (0.0007) [2023-12-27 02:20:42,363][105620] Updated weights for policy 1, policy_version 1506441 (0.0008) [2023-12-27 02:20:42,423][105620] Updated weights for policy 1, policy_version 1506451 (0.0009) [2023-12-27 02:20:42,482][105620] Updated weights for policy 1, policy_version 1506461 (0.0011) [2023-12-27 02:20:42,545][105620] Updated weights for policy 1, policy_version 1506471 (0.0010) [2023-12-27 02:20:42,780][105692] Updated weights for policy 0, policy_version 1503787 (0.0009) [2023-12-27 02:20:42,837][105692] Updated weights for policy 0, policy_version 1503797 (0.0010) [2023-12-27 02:20:42,904][105692] Updated weights for policy 0, policy_version 1503807 (0.0005) [2023-12-27 02:20:43,229][105620] Updated weights for policy 1, policy_version 1506481 (0.0006) [2023-12-27 02:20:43,293][105620] Updated weights for policy 1, policy_version 1506491 (0.0005) [2023-12-27 02:20:43,345][105620] Updated weights for policy 1, policy_version 1506501 (0.0005) [2023-12-27 02:20:43,511][105692] Updated weights for policy 0, policy_version 1503817 (0.0005) [2023-12-27 02:20:43,555][105692] Updated weights for policy 0, policy_version 1503827 (0.0005) [2023-12-27 02:20:43,609][105692] Updated weights for policy 0, policy_version 1503837 (0.0005) [2023-12-27 02:20:43,659][105692] Updated weights for policy 0, policy_version 1503847 (0.0005) [2023-12-27 02:20:43,946][105620] Updated weights for policy 1, policy_version 1506511 (0.0008) [2023-12-27 02:20:44,002][105620] Updated weights for policy 1, policy_version 1506521 (0.0009) [2023-12-27 02:20:44,050][105620] Updated weights for policy 1, policy_version 1506531 (0.0009) [2023-12-27 02:20:44,256][105692] Updated weights for policy 0, policy_version 1503857 (0.0008) [2023-12-27 02:20:44,308][105692] Updated weights for policy 0, policy_version 1503867 (0.0008) [2023-12-27 02:20:44,357][105692] Updated weights for policy 0, policy_version 1503877 (0.0008) [2023-12-27 02:20:44,911][105620] Updated weights for policy 1, policy_version 1506542 (0.0010) [2023-12-27 02:20:44,939][105692] Updated weights for policy 0, policy_version 1503887 (0.0009) [2023-12-27 02:20:44,971][105620] Updated weights for policy 1, policy_version 1506552 (0.0008) [2023-12-27 02:20:44,999][105692] Updated weights for policy 0, policy_version 1503897 (0.0007) [2023-12-27 02:20:45,023][105620] Updated weights for policy 1, policy_version 1506562 (0.0007) [2023-12-27 02:20:45,060][105692] Updated weights for policy 0, policy_version 1503907 (0.0009) [2023-12-27 02:20:45,760][105692] Updated weights for policy 0, policy_version 1503917 (0.0009) [2023-12-27 02:20:45,810][105692] Updated weights for policy 0, policy_version 1503927 (0.0008) [2023-12-27 02:20:45,820][105620] Updated weights for policy 1, policy_version 1506572 (0.0008) [2023-12-27 02:20:45,863][105692] Updated weights for policy 0, policy_version 1503937 (0.0006) [2023-12-27 02:20:45,877][105620] Updated weights for policy 1, policy_version 1506582 (0.0008) [2023-12-27 02:20:45,937][105620] Updated weights for policy 1, policy_version 1506592 (0.0008) [2023-12-27 02:20:46,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 770809856. Throughput: 0: 9583.2, 1: 9695.9. Samples: 770773496. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:20:46,063][104569] Avg episode reward: [(0, '8542.230'), (1, '9260.760')] [2023-12-27 02:20:46,073][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001503944_385064960.pth... [2023-12-27 02:20:46,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001506600_385744896.pth... [2023-12-27 02:20:46,081][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001505480_385458176.pth [2023-12-27 02:20:46,083][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001502792_384770048.pth [2023-12-27 02:20:46,529][105692] Updated weights for policy 0, policy_version 1503947 (0.0006) [2023-12-27 02:20:46,596][105692] Updated weights for policy 0, policy_version 1503958 (0.0007) [2023-12-27 02:20:46,647][105692] Updated weights for policy 0, policy_version 1503968 (0.0007) [2023-12-27 02:20:46,748][105620] Updated weights for policy 1, policy_version 1506602 (0.0009) [2023-12-27 02:20:46,803][105620] Updated weights for policy 1, policy_version 1506612 (0.0009) [2023-12-27 02:20:46,850][105620] Updated weights for policy 1, policy_version 1506622 (0.0009) [2023-12-27 02:20:46,898][105620] Updated weights for policy 1, policy_version 1506632 (0.0009) [2023-12-27 02:20:47,250][105692] Updated weights for policy 0, policy_version 1503978 (0.0007) [2023-12-27 02:20:47,319][105692] Updated weights for policy 0, policy_version 1503988 (0.0005) [2023-12-27 02:20:47,387][105692] Updated weights for policy 0, policy_version 1503998 (0.0005) [2023-12-27 02:20:47,433][105692] Updated weights for policy 0, policy_version 1504008 (0.0005) [2023-12-27 02:20:47,806][105620] Updated weights for policy 1, policy_version 1506642 (0.0009) [2023-12-27 02:20:47,856][105620] Updated weights for policy 1, policy_version 1506652 (0.0009) [2023-12-27 02:20:47,916][105620] Updated weights for policy 1, policy_version 1506662 (0.0009) [2023-12-27 02:20:47,997][105692] Updated weights for policy 0, policy_version 1504018 (0.0009) [2023-12-27 02:20:48,058][105692] Updated weights for policy 0, policy_version 1504028 (0.0009) [2023-12-27 02:20:48,118][105692] Updated weights for policy 0, policy_version 1504038 (0.0006) [2023-12-27 02:20:48,736][105692] Updated weights for policy 0, policy_version 1504048 (0.0007) [2023-12-27 02:20:48,783][105620] Updated weights for policy 1, policy_version 1506672 (0.0009) [2023-12-27 02:20:48,803][105692] Updated weights for policy 0, policy_version 1504058 (0.0007) [2023-12-27 02:20:48,838][105620] Updated weights for policy 1, policy_version 1506682 (0.0008) [2023-12-27 02:20:48,857][105692] Updated weights for policy 0, policy_version 1504068 (0.0008) [2023-12-27 02:20:48,889][105620] Updated weights for policy 1, policy_version 1506692 (0.0007) [2023-12-27 02:20:49,631][105692] Updated weights for policy 0, policy_version 1504078 (0.0008) [2023-12-27 02:20:49,691][105620] Updated weights for policy 1, policy_version 1506702 (0.0010) [2023-12-27 02:20:49,693][105692] Updated weights for policy 0, policy_version 1504088 (0.0007) [2023-12-27 02:20:49,749][105620] Updated weights for policy 1, policy_version 1506712 (0.0011) [2023-12-27 02:20:49,755][105692] Updated weights for policy 0, policy_version 1504098 (0.0006) [2023-12-27 02:20:49,810][105620] Updated weights for policy 1, policy_version 1506722 (0.0011) [2023-12-27 02:20:50,540][105692] Updated weights for policy 0, policy_version 1504108 (0.0007) [2023-12-27 02:20:50,593][105620] Updated weights for policy 1, policy_version 1506732 (0.0011) [2023-12-27 02:20:50,601][105692] Updated weights for policy 0, policy_version 1504118 (0.0009) [2023-12-27 02:20:50,645][105620] Updated weights for policy 1, policy_version 1506742 (0.0010) [2023-12-27 02:20:50,651][105692] Updated weights for policy 0, policy_version 1504128 (0.0008) [2023-12-27 02:20:50,705][105620] Updated weights for policy 1, policy_version 1506752 (0.0011) [2023-12-27 02:20:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.1, 300 sec: 19466.4). Total num frames: 770899968. Throughput: 0: 9778.7, 1: 9529.9. Samples: 770889168. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:20:51,063][104569] Avg episode reward: [(0, '8906.613'), (1, '9262.894')] [2023-12-27 02:20:51,427][105692] Updated weights for policy 0, policy_version 1504138 (0.0005) [2023-12-27 02:20:51,482][105692] Updated weights for policy 0, policy_version 1504148 (0.0006) [2023-12-27 02:20:51,490][105620] Updated weights for policy 1, policy_version 1506762 (0.0010) [2023-12-27 02:20:51,540][105692] Updated weights for policy 0, policy_version 1504158 (0.0005) [2023-12-27 02:20:51,554][105620] Updated weights for policy 1, policy_version 1506772 (0.0008) [2023-12-27 02:20:51,595][105692] Updated weights for policy 0, policy_version 1504168 (0.0005) [2023-12-27 02:20:51,619][105620] Updated weights for policy 1, policy_version 1506782 (0.0007) [2023-12-27 02:20:51,676][105620] Updated weights for policy 1, policy_version 1506792 (0.0007) [2023-12-27 02:20:52,361][105692] Updated weights for policy 0, policy_version 1504178 (0.0008) [2023-12-27 02:20:52,401][105620] Updated weights for policy 1, policy_version 1506802 (0.0007) [2023-12-27 02:20:52,418][105692] Updated weights for policy 0, policy_version 1504188 (0.0008) [2023-12-27 02:20:52,458][105620] Updated weights for policy 1, policy_version 1506812 (0.0007) [2023-12-27 02:20:52,473][105692] Updated weights for policy 0, policy_version 1504198 (0.0009) [2023-12-27 02:20:52,516][105620] Updated weights for policy 1, policy_version 1506822 (0.0005) [2023-12-27 02:20:53,155][105620] Updated weights for policy 1, policy_version 1506832 (0.0006) [2023-12-27 02:20:53,215][105620] Updated weights for policy 1, policy_version 1506842 (0.0006) [2023-12-27 02:20:53,271][105620] Updated weights for policy 1, policy_version 1506852 (0.0008) [2023-12-27 02:20:53,320][105692] Updated weights for policy 0, policy_version 1504208 (0.0010) [2023-12-27 02:20:53,378][105692] Updated weights for policy 0, policy_version 1504218 (0.0008) [2023-12-27 02:20:53,445][105692] Updated weights for policy 0, policy_version 1504228 (0.0010) [2023-12-27 02:20:53,840][105620] Updated weights for policy 1, policy_version 1506862 (0.0008) [2023-12-27 02:20:53,901][105620] Updated weights for policy 1, policy_version 1506872 (0.0009) [2023-12-27 02:20:53,962][105620] Updated weights for policy 1, policy_version 1506882 (0.0009) [2023-12-27 02:20:54,162][105692] Updated weights for policy 0, policy_version 1504238 (0.0010) [2023-12-27 02:20:54,218][105692] Updated weights for policy 0, policy_version 1504248 (0.0006) [2023-12-27 02:20:54,264][105692] Updated weights for policy 0, policy_version 1504258 (0.0006) [2023-12-27 02:20:54,736][105620] Updated weights for policy 1, policy_version 1506892 (0.0009) [2023-12-27 02:20:54,783][105620] Updated weights for policy 1, policy_version 1506902 (0.0009) [2023-12-27 02:20:54,835][105620] Updated weights for policy 1, policy_version 1506912 (0.0009) [2023-12-27 02:20:55,001][105692] Updated weights for policy 0, policy_version 1504268 (0.0008) [2023-12-27 02:20:55,063][105692] Updated weights for policy 0, policy_version 1504278 (0.0008) [2023-12-27 02:20:55,124][105692] Updated weights for policy 0, policy_version 1504288 (0.0009) [2023-12-27 02:20:55,538][105620] Updated weights for policy 1, policy_version 1506922 (0.0008) [2023-12-27 02:20:55,594][105620] Updated weights for policy 1, policy_version 1506932 (0.0005) [2023-12-27 02:20:55,654][105620] Updated weights for policy 1, policy_version 1506942 (0.0007) [2023-12-27 02:20:55,708][105620] Updated weights for policy 1, policy_version 1506952 (0.0008) [2023-12-27 02:20:55,826][105692] Updated weights for policy 0, policy_version 1504298 (0.0007) [2023-12-27 02:20:55,885][105692] Updated weights for policy 0, policy_version 1504308 (0.0005) [2023-12-27 02:20:55,940][105692] Updated weights for policy 0, policy_version 1504318 (0.0005) [2023-12-27 02:20:55,999][105692] Updated weights for policy 0, policy_version 1504328 (0.0005) [2023-12-27 02:20:56,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19251.3, 300 sec: 19494.2). Total num frames: 770998272. Throughput: 0: 9701.5, 1: 9552.1. Samples: 771003100. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:20:56,062][104569] Avg episode reward: [(0, '8811.805'), (1, '9262.388')] [2023-12-27 02:20:56,409][105620] Updated weights for policy 1, policy_version 1506962 (0.0009) [2023-12-27 02:20:56,463][105620] Updated weights for policy 1, policy_version 1506972 (0.0009) [2023-12-27 02:20:56,514][105620] Updated weights for policy 1, policy_version 1506982 (0.0009) [2023-12-27 02:20:56,641][105692] Updated weights for policy 0, policy_version 1504338 (0.0009) [2023-12-27 02:20:56,703][105692] Updated weights for policy 0, policy_version 1504348 (0.0009) [2023-12-27 02:20:56,765][105692] Updated weights for policy 0, policy_version 1504358 (0.0009) [2023-12-27 02:20:57,238][105620] Updated weights for policy 1, policy_version 1506992 (0.0010) [2023-12-27 02:20:57,290][105620] Updated weights for policy 1, policy_version 1507002 (0.0011) [2023-12-27 02:20:57,345][105620] Updated weights for policy 1, policy_version 1507012 (0.0009) [2023-12-27 02:20:57,419][105692] Updated weights for policy 0, policy_version 1504368 (0.0010) [2023-12-27 02:20:57,488][105692] Updated weights for policy 0, policy_version 1504378 (0.0011) [2023-12-27 02:20:57,552][105692] Updated weights for policy 0, policy_version 1504388 (0.0010) [2023-12-27 02:20:57,937][105620] Updated weights for policy 1, policy_version 1507022 (0.0005) [2023-12-27 02:20:58,006][105620] Updated weights for policy 1, policy_version 1507032 (0.0006) [2023-12-27 02:20:58,061][105620] Updated weights for policy 1, policy_version 1507042 (0.0005) [2023-12-27 02:20:58,208][105692] Updated weights for policy 0, policy_version 1504398 (0.0009) [2023-12-27 02:20:58,278][105692] Updated weights for policy 0, policy_version 1504408 (0.0007) [2023-12-27 02:20:58,349][105692] Updated weights for policy 0, policy_version 1504418 (0.0008) [2023-12-27 02:20:58,653][105620] Updated weights for policy 1, policy_version 1507052 (0.0008) [2023-12-27 02:20:58,715][105620] Updated weights for policy 1, policy_version 1507062 (0.0010) [2023-12-27 02:20:58,780][105620] Updated weights for policy 1, policy_version 1507072 (0.0008) [2023-12-27 02:20:59,082][105692] Updated weights for policy 0, policy_version 1504428 (0.0007) [2023-12-27 02:20:59,152][105692] Updated weights for policy 0, policy_version 1504438 (0.0006) [2023-12-27 02:20:59,208][105692] Updated weights for policy 0, policy_version 1504448 (0.0005) [2023-12-27 02:20:59,470][105620] Updated weights for policy 1, policy_version 1507082 (0.0009) [2023-12-27 02:20:59,518][105620] Updated weights for policy 1, policy_version 1507092 (0.0010) [2023-12-27 02:20:59,567][105620] Updated weights for policy 1, policy_version 1507102 (0.0010) [2023-12-27 02:20:59,620][105620] Updated weights for policy 1, policy_version 1507112 (0.0010) [2023-12-27 02:20:59,891][105692] Updated weights for policy 0, policy_version 1504458 (0.0008) [2023-12-27 02:20:59,950][105692] Updated weights for policy 0, policy_version 1504468 (0.0009) [2023-12-27 02:21:00,008][105692] Updated weights for policy 0, policy_version 1504478 (0.0009) [2023-12-27 02:21:00,067][105692] Updated weights for policy 0, policy_version 1504488 (0.0008) [2023-12-27 02:21:00,400][105620] Updated weights for policy 1, policy_version 1507122 (0.0008) [2023-12-27 02:21:00,453][105620] Updated weights for policy 1, policy_version 1507132 (0.0009) [2023-12-27 02:21:00,501][105620] Updated weights for policy 1, policy_version 1507142 (0.0009) [2023-12-27 02:21:00,799][105692] Updated weights for policy 0, policy_version 1504498 (0.0009) [2023-12-27 02:21:00,850][105692] Updated weights for policy 0, policy_version 1504508 (0.0008) [2023-12-27 02:21:00,908][105692] Updated weights for policy 0, policy_version 1504518 (0.0009) [2023-12-27 02:21:01,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 771096576. Throughput: 0: 9718.3, 1: 9533.0. Samples: 771065504. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:21:01,062][104569] Avg episode reward: [(0, '8539.546'), (1, '9260.115')] [2023-12-27 02:21:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001504520_385212416.pth... [2023-12-27 02:21:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001507144_385884160.pth... [2023-12-27 02:21:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001506056_385605632.pth [2023-12-27 02:21:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001503368_384917504.pth [2023-12-27 02:21:01,230][105620] Updated weights for policy 1, policy_version 1507152 (0.0009) [2023-12-27 02:21:01,295][105620] Updated weights for policy 1, policy_version 1507162 (0.0007) [2023-12-27 02:21:01,356][105620] Updated weights for policy 1, policy_version 1507172 (0.0008) [2023-12-27 02:21:01,731][105692] Updated weights for policy 0, policy_version 1504528 (0.0007) [2023-12-27 02:21:01,780][105692] Updated weights for policy 0, policy_version 1504538 (0.0008) [2023-12-27 02:21:01,836][105692] Updated weights for policy 0, policy_version 1504548 (0.0009) [2023-12-27 02:21:02,127][105620] Updated weights for policy 1, policy_version 1507182 (0.0007) [2023-12-27 02:21:02,191][105620] Updated weights for policy 1, policy_version 1507192 (0.0008) [2023-12-27 02:21:02,257][105620] Updated weights for policy 1, policy_version 1507202 (0.0007) [2023-12-27 02:21:02,616][105692] Updated weights for policy 0, policy_version 1504558 (0.0009) [2023-12-27 02:21:02,671][105692] Updated weights for policy 0, policy_version 1504568 (0.0008) [2023-12-27 02:21:02,719][105692] Updated weights for policy 0, policy_version 1504578 (0.0008) [2023-12-27 02:21:02,923][105620] Updated weights for policy 1, policy_version 1507212 (0.0008) [2023-12-27 02:21:02,979][105620] Updated weights for policy 1, policy_version 1507222 (0.0006) [2023-12-27 02:21:03,032][105620] Updated weights for policy 1, policy_version 1507232 (0.0006) [2023-12-27 02:21:03,449][105692] Updated weights for policy 0, policy_version 1504588 (0.0008) [2023-12-27 02:21:03,496][105692] Updated weights for policy 0, policy_version 1504598 (0.0009) [2023-12-27 02:21:03,542][105692] Updated weights for policy 0, policy_version 1504608 (0.0009) [2023-12-27 02:21:03,700][105620] Updated weights for policy 1, policy_version 1507242 (0.0009) [2023-12-27 02:21:03,751][105620] Updated weights for policy 1, policy_version 1507252 (0.0008) [2023-12-27 02:21:03,802][105620] Updated weights for policy 1, policy_version 1507262 (0.0008) [2023-12-27 02:21:03,857][105620] Updated weights for policy 1, policy_version 1507272 (0.0008) [2023-12-27 02:21:04,225][105692] Updated weights for policy 0, policy_version 1504618 (0.0008) [2023-12-27 02:21:04,289][105692] Updated weights for policy 0, policy_version 1504628 (0.0006) [2023-12-27 02:21:04,356][105692] Updated weights for policy 0, policy_version 1504638 (0.0006) [2023-12-27 02:21:04,424][105692] Updated weights for policy 0, policy_version 1504648 (0.0007) [2023-12-27 02:21:04,753][105620] Updated weights for policy 1, policy_version 1507282 (0.0008) [2023-12-27 02:21:04,808][105620] Updated weights for policy 1, policy_version 1507292 (0.0008) [2023-12-27 02:21:04,874][105620] Updated weights for policy 1, policy_version 1507302 (0.0009) [2023-12-27 02:21:05,013][105692] Updated weights for policy 0, policy_version 1504658 (0.0006) [2023-12-27 02:21:05,057][105692] Updated weights for policy 0, policy_version 1504668 (0.0010) [2023-12-27 02:21:05,102][105692] Updated weights for policy 0, policy_version 1504678 (0.0010) [2023-12-27 02:21:05,539][105620] Updated weights for policy 1, policy_version 1507312 (0.0007) [2023-12-27 02:21:05,599][105620] Updated weights for policy 1, policy_version 1507322 (0.0010) [2023-12-27 02:21:05,650][105620] Updated weights for policy 1, policy_version 1507332 (0.0010) [2023-12-27 02:21:05,778][105692] Updated weights for policy 0, policy_version 1504688 (0.0006) [2023-12-27 02:21:05,844][105692] Updated weights for policy 0, policy_version 1504698 (0.0005) [2023-12-27 02:21:05,900][105692] Updated weights for policy 0, policy_version 1504708 (0.0005) [2023-12-27 02:21:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 771194880. Throughput: 0: 9720.1, 1: 9547.4. Samples: 771180088. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:21:06,062][104569] Avg episode reward: [(0, '8813.777'), (1, '9260.294')] [2023-12-27 02:21:06,405][105620] Updated weights for policy 1, policy_version 1507342 (0.0009) [2023-12-27 02:21:06,473][105620] Updated weights for policy 1, policy_version 1507352 (0.0006) [2023-12-27 02:21:06,508][105692] Updated weights for policy 0, policy_version 1504718 (0.0008) [2023-12-27 02:21:06,531][105620] Updated weights for policy 1, policy_version 1507362 (0.0008) [2023-12-27 02:21:06,571][105692] Updated weights for policy 0, policy_version 1504728 (0.0011) [2023-12-27 02:21:06,638][105692] Updated weights for policy 0, policy_version 1504738 (0.0011) [2023-12-27 02:21:07,263][105620] Updated weights for policy 1, policy_version 1507372 (0.0010) [2023-12-27 02:21:07,308][105620] Updated weights for policy 1, policy_version 1507382 (0.0010) [2023-12-27 02:21:07,337][105692] Updated weights for policy 0, policy_version 1504748 (0.0010) [2023-12-27 02:21:07,359][105620] Updated weights for policy 1, policy_version 1507392 (0.0010) [2023-12-27 02:21:07,386][105692] Updated weights for policy 0, policy_version 1504758 (0.0005) [2023-12-27 02:21:07,437][105692] Updated weights for policy 0, policy_version 1504768 (0.0008) [2023-12-27 02:21:08,099][105692] Updated weights for policy 0, policy_version 1504778 (0.0009) [2023-12-27 02:21:08,100][105620] Updated weights for policy 1, policy_version 1507402 (0.0010) [2023-12-27 02:21:08,149][105692] Updated weights for policy 0, policy_version 1504788 (0.0010) [2023-12-27 02:21:08,161][105620] Updated weights for policy 1, policy_version 1507412 (0.0010) [2023-12-27 02:21:08,205][105692] Updated weights for policy 0, policy_version 1504798 (0.0006) [2023-12-27 02:21:08,220][105620] Updated weights for policy 1, policy_version 1507422 (0.0010) [2023-12-27 02:21:08,269][105692] Updated weights for policy 0, policy_version 1504808 (0.0006) [2023-12-27 02:21:08,285][105620] Updated weights for policy 1, policy_version 1507432 (0.0010) [2023-12-27 02:21:08,853][105692] Updated weights for policy 0, policy_version 1504818 (0.0005) [2023-12-27 02:21:08,918][105692] Updated weights for policy 0, policy_version 1504828 (0.0009) [2023-12-27 02:21:08,980][105692] Updated weights for policy 0, policy_version 1504838 (0.0011) [2023-12-27 02:21:08,992][105620] Updated weights for policy 1, policy_version 1507442 (0.0010) [2023-12-27 02:21:09,057][105620] Updated weights for policy 1, policy_version 1507452 (0.0010) [2023-12-27 02:21:09,122][105620] Updated weights for policy 1, policy_version 1507462 (0.0010) [2023-12-27 02:21:09,690][105692] Updated weights for policy 0, policy_version 1504848 (0.0011) [2023-12-27 02:21:09,750][105692] Updated weights for policy 0, policy_version 1504858 (0.0011) [2023-12-27 02:21:09,810][105692] Updated weights for policy 0, policy_version 1504868 (0.0011) [2023-12-27 02:21:09,869][105620] Updated weights for policy 1, policy_version 1507472 (0.0011) [2023-12-27 02:21:09,921][105620] Updated weights for policy 1, policy_version 1507482 (0.0010) [2023-12-27 02:21:09,992][105620] Updated weights for policy 1, policy_version 1507492 (0.0010) [2023-12-27 02:21:10,466][105692] Updated weights for policy 0, policy_version 1504878 (0.0007) [2023-12-27 02:21:10,523][105692] Updated weights for policy 0, policy_version 1504888 (0.0008) [2023-12-27 02:21:10,579][105692] Updated weights for policy 0, policy_version 1504898 (0.0010) [2023-12-27 02:21:10,702][105620] Updated weights for policy 1, policy_version 1507502 (0.0010) [2023-12-27 02:21:10,758][105620] Updated weights for policy 1, policy_version 1507512 (0.0010) [2023-12-27 02:21:10,805][105620] Updated weights for policy 1, policy_version 1507522 (0.0010) [2023-12-27 02:21:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 771293184. Throughput: 0: 9900.4, 1: 9520.2. Samples: 771301332. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:21:11,062][104569] Avg episode reward: [(0, '8993.159'), (1, '9082.095')] [2023-12-27 02:21:11,153][105692] Updated weights for policy 0, policy_version 1504908 (0.0010) [2023-12-27 02:21:11,220][105692] Updated weights for policy 0, policy_version 1504918 (0.0011) [2023-12-27 02:21:11,285][105692] Updated weights for policy 0, policy_version 1504928 (0.0009) [2023-12-27 02:21:11,605][105620] Updated weights for policy 1, policy_version 1507532 (0.0009) [2023-12-27 02:21:11,668][105620] Updated weights for policy 1, policy_version 1507542 (0.0009) [2023-12-27 02:21:11,738][105620] Updated weights for policy 1, policy_version 1507552 (0.0009) [2023-12-27 02:21:12,022][105692] Updated weights for policy 0, policy_version 1504938 (0.0011) [2023-12-27 02:21:12,078][105692] Updated weights for policy 0, policy_version 1504948 (0.0011) [2023-12-27 02:21:12,130][105692] Updated weights for policy 0, policy_version 1504958 (0.0010) [2023-12-27 02:21:12,186][105692] Updated weights for policy 0, policy_version 1504968 (0.0011) [2023-12-27 02:21:12,360][105620] Updated weights for policy 1, policy_version 1507562 (0.0009) [2023-12-27 02:21:12,433][105620] Updated weights for policy 1, policy_version 1507572 (0.0006) [2023-12-27 02:21:12,503][105620] Updated weights for policy 1, policy_version 1507582 (0.0006) [2023-12-27 02:21:12,570][105620] Updated weights for policy 1, policy_version 1507592 (0.0006) [2023-12-27 02:21:12,971][105692] Updated weights for policy 0, policy_version 1504978 (0.0008) [2023-12-27 02:21:13,037][105692] Updated weights for policy 0, policy_version 1504988 (0.0005) [2023-12-27 02:21:13,101][105692] Updated weights for policy 0, policy_version 1504998 (0.0005) [2023-12-27 02:21:13,189][105620] Updated weights for policy 1, policy_version 1507602 (0.0008) [2023-12-27 02:21:13,245][105620] Updated weights for policy 1, policy_version 1507612 (0.0007) [2023-12-27 02:21:13,294][105620] Updated weights for policy 1, policy_version 1507622 (0.0008) [2023-12-27 02:21:13,660][105692] Updated weights for policy 0, policy_version 1505008 (0.0006) [2023-12-27 02:21:13,712][105692] Updated weights for policy 0, policy_version 1505018 (0.0005) [2023-12-27 02:21:13,755][105692] Updated weights for policy 0, policy_version 1505028 (0.0006) [2023-12-27 02:21:14,061][105620] Updated weights for policy 1, policy_version 1507632 (0.0010) [2023-12-27 02:21:14,127][105620] Updated weights for policy 1, policy_version 1507642 (0.0010) [2023-12-27 02:21:14,193][105620] Updated weights for policy 1, policy_version 1507652 (0.0011) [2023-12-27 02:21:14,354][105692] Updated weights for policy 0, policy_version 1505038 (0.0006) [2023-12-27 02:21:14,404][105692] Updated weights for policy 0, policy_version 1505048 (0.0005) [2023-12-27 02:21:14,458][105692] Updated weights for policy 0, policy_version 1505058 (0.0010) [2023-12-27 02:21:14,842][105620] Updated weights for policy 1, policy_version 1507662 (0.0009) [2023-12-27 02:21:14,898][105620] Updated weights for policy 1, policy_version 1507672 (0.0008) [2023-12-27 02:21:14,949][105620] Updated weights for policy 1, policy_version 1507682 (0.0008) [2023-12-27 02:21:15,170][105692] Updated weights for policy 0, policy_version 1505068 (0.0010) [2023-12-27 02:21:15,230][105692] Updated weights for policy 0, policy_version 1505078 (0.0008) [2023-12-27 02:21:15,297][105692] Updated weights for policy 0, policy_version 1505088 (0.0008) [2023-12-27 02:21:15,750][105620] Updated weights for policy 1, policy_version 1507692 (0.0010) [2023-12-27 02:21:15,821][105620] Updated weights for policy 1, policy_version 1507702 (0.0010) [2023-12-27 02:21:15,887][105620] Updated weights for policy 1, policy_version 1507712 (0.0010) [2023-12-27 02:21:15,998][105692] Updated weights for policy 0, policy_version 1505098 (0.0008) [2023-12-27 02:21:16,055][105692] Updated weights for policy 0, policy_version 1505108 (0.0007) [2023-12-27 02:21:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 771391488. Throughput: 0: 9909.7, 1: 9531.3. Samples: 771362008. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:21:16,063][104569] Avg episode reward: [(0, '8721.408'), (1, '8990.005')] [2023-12-27 02:21:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001507720_386031616.pth... [2023-12-27 02:21:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001506600_385744896.pth [2023-12-27 02:21:16,114][105692] Updated weights for policy 0, policy_version 1505118 (0.0008) [2023-12-27 02:21:16,169][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001505128_385368064.pth... [2023-12-27 02:21:16,171][105692] Updated weights for policy 0, policy_version 1505128 (0.0008) [2023-12-27 02:21:16,173][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001503944_385064960.pth [2023-12-27 02:21:16,606][105620] Updated weights for policy 1, policy_version 1507722 (0.0011) [2023-12-27 02:21:16,662][105620] Updated weights for policy 1, policy_version 1507732 (0.0010) [2023-12-27 02:21:16,716][105620] Updated weights for policy 1, policy_version 1507742 (0.0010) [2023-12-27 02:21:16,781][105620] Updated weights for policy 1, policy_version 1507752 (0.0011) [2023-12-27 02:21:16,888][105692] Updated weights for policy 0, policy_version 1505138 (0.0008) [2023-12-27 02:21:16,946][105692] Updated weights for policy 0, policy_version 1505148 (0.0007) [2023-12-27 02:21:17,004][105692] Updated weights for policy 0, policy_version 1505158 (0.0007) [2023-12-27 02:21:17,540][105620] Updated weights for policy 1, policy_version 1507762 (0.0011) [2023-12-27 02:21:17,600][105620] Updated weights for policy 1, policy_version 1507772 (0.0011) [2023-12-27 02:21:17,660][105620] Updated weights for policy 1, policy_version 1507782 (0.0011) [2023-12-27 02:21:17,720][105692] Updated weights for policy 0, policy_version 1505168 (0.0006) [2023-12-27 02:21:17,782][105692] Updated weights for policy 0, policy_version 1505178 (0.0006) [2023-12-27 02:21:17,839][105692] Updated weights for policy 0, policy_version 1505188 (0.0006) [2023-12-27 02:21:18,409][105620] Updated weights for policy 1, policy_version 1507792 (0.0011) [2023-12-27 02:21:18,468][105620] Updated weights for policy 1, policy_version 1507802 (0.0011) [2023-12-27 02:21:18,491][105692] Updated weights for policy 0, policy_version 1505198 (0.0006) [2023-12-27 02:21:18,521][105620] Updated weights for policy 1, policy_version 1507812 (0.0011) [2023-12-27 02:21:18,551][105692] Updated weights for policy 0, policy_version 1505208 (0.0009) [2023-12-27 02:21:18,606][105692] Updated weights for policy 0, policy_version 1505218 (0.0008) [2023-12-27 02:21:19,279][105620] Updated weights for policy 1, policy_version 1507822 (0.0011) [2023-12-27 02:21:19,304][105692] Updated weights for policy 0, policy_version 1505228 (0.0011) [2023-12-27 02:21:19,332][105620] Updated weights for policy 1, policy_version 1507832 (0.0011) [2023-12-27 02:21:19,374][105692] Updated weights for policy 0, policy_version 1505238 (0.0010) [2023-12-27 02:21:19,402][105620] Updated weights for policy 1, policy_version 1507842 (0.0011) [2023-12-27 02:21:19,440][105692] Updated weights for policy 0, policy_version 1505248 (0.0010) [2023-12-27 02:21:20,125][105692] Updated weights for policy 0, policy_version 1505258 (0.0011) [2023-12-27 02:21:20,184][105620] Updated weights for policy 1, policy_version 1507852 (0.0011) [2023-12-27 02:21:20,187][105692] Updated weights for policy 0, policy_version 1505268 (0.0011) [2023-12-27 02:21:20,249][105692] Updated weights for policy 0, policy_version 1505278 (0.0011) [2023-12-27 02:21:20,252][105620] Updated weights for policy 1, policy_version 1507862 (0.0011) [2023-12-27 02:21:20,317][105692] Updated weights for policy 0, policy_version 1505288 (0.0011) [2023-12-27 02:21:20,317][105620] Updated weights for policy 1, policy_version 1507872 (0.0011) [2023-12-27 02:21:21,046][105692] Updated weights for policy 0, policy_version 1505298 (0.0011) [2023-12-27 02:21:21,062][104569] Fps is (10 sec: 18840.6, 60 sec: 19387.6, 300 sec: 19494.2). Total num frames: 771481600. Throughput: 0: 10018.7, 1: 9462.2. Samples: 771477796. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:21:21,063][104569] Avg episode reward: [(0, '8264.553'), (1, '8991.428')] [2023-12-27 02:21:21,091][105620] Updated weights for policy 1, policy_version 1507882 (0.0011) [2023-12-27 02:21:21,107][105692] Updated weights for policy 0, policy_version 1505308 (0.0011) [2023-12-27 02:21:21,156][105620] Updated weights for policy 1, policy_version 1507892 (0.0010) [2023-12-27 02:21:21,164][105692] Updated weights for policy 0, policy_version 1505318 (0.0011) [2023-12-27 02:21:21,212][105620] Updated weights for policy 1, policy_version 1507902 (0.0010) [2023-12-27 02:21:21,276][105620] Updated weights for policy 1, policy_version 1507912 (0.0011) [2023-12-27 02:21:21,999][105692] Updated weights for policy 0, policy_version 1505328 (0.0008) [2023-12-27 02:21:22,036][105620] Updated weights for policy 1, policy_version 1507922 (0.0010) [2023-12-27 02:21:22,059][105692] Updated weights for policy 0, policy_version 1505338 (0.0006) [2023-12-27 02:21:22,088][105620] Updated weights for policy 1, policy_version 1507932 (0.0010) [2023-12-27 02:21:22,118][105692] Updated weights for policy 0, policy_version 1505348 (0.0005) [2023-12-27 02:21:22,144][105620] Updated weights for policy 1, policy_version 1507942 (0.0010) [2023-12-27 02:21:22,849][105692] Updated weights for policy 0, policy_version 1505358 (0.0007) [2023-12-27 02:21:22,889][105620] Updated weights for policy 1, policy_version 1507952 (0.0008) [2023-12-27 02:21:22,913][105692] Updated weights for policy 0, policy_version 1505368 (0.0008) [2023-12-27 02:21:22,956][105620] Updated weights for policy 1, policy_version 1507962 (0.0006) [2023-12-27 02:21:22,970][105692] Updated weights for policy 0, policy_version 1505378 (0.0008) [2023-12-27 02:21:23,016][105620] Updated weights for policy 1, policy_version 1507972 (0.0008) [2023-12-27 02:21:23,698][105620] Updated weights for policy 1, policy_version 1507982 (0.0010) [2023-12-27 02:21:23,747][105620] Updated weights for policy 1, policy_version 1507992 (0.0009) [2023-12-27 02:21:23,756][105692] Updated weights for policy 0, policy_version 1505388 (0.0007) [2023-12-27 02:21:23,801][105620] Updated weights for policy 1, policy_version 1508002 (0.0007) [2023-12-27 02:21:23,819][105692] Updated weights for policy 0, policy_version 1505398 (0.0007) [2023-12-27 02:21:23,874][105692] Updated weights for policy 0, policy_version 1505408 (0.0008) [2023-12-27 02:21:24,452][105620] Updated weights for policy 1, policy_version 1508012 (0.0006) [2023-12-27 02:21:24,516][105620] Updated weights for policy 1, policy_version 1508022 (0.0007) [2023-12-27 02:21:24,542][105692] Updated weights for policy 0, policy_version 1505418 (0.0009) [2023-12-27 02:21:24,569][105620] Updated weights for policy 1, policy_version 1508032 (0.0009) [2023-12-27 02:21:24,597][105692] Updated weights for policy 0, policy_version 1505428 (0.0010) [2023-12-27 02:21:24,652][105692] Updated weights for policy 0, policy_version 1505438 (0.0007) [2023-12-27 02:21:24,713][105692] Updated weights for policy 0, policy_version 1505448 (0.0006) [2023-12-27 02:21:25,197][105620] Updated weights for policy 1, policy_version 1508042 (0.0009) [2023-12-27 02:21:25,243][105620] Updated weights for policy 1, policy_version 1508052 (0.0005) [2023-12-27 02:21:25,297][105620] Updated weights for policy 1, policy_version 1508062 (0.0005) [2023-12-27 02:21:25,337][105692] Updated weights for policy 0, policy_version 1505458 (0.0009) [2023-12-27 02:21:25,349][105620] Updated weights for policy 1, policy_version 1508072 (0.0005) [2023-12-27 02:21:25,397][105692] Updated weights for policy 0, policy_version 1505468 (0.0009) [2023-12-27 02:21:25,452][105692] Updated weights for policy 0, policy_version 1505478 (0.0009) [2023-12-27 02:21:26,042][105620] Updated weights for policy 1, policy_version 1508082 (0.0010) [2023-12-27 02:21:26,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 771579904. Throughput: 0: 10026.2, 1: 9545.5. Samples: 771594048. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:21:26,062][104569] Avg episode reward: [(0, '7814.942'), (1, '9258.965')] [2023-12-27 02:21:26,096][105620] Updated weights for policy 1, policy_version 1508092 (0.0006) [2023-12-27 02:21:26,159][105620] Updated weights for policy 1, policy_version 1508102 (0.0008) [2023-12-27 02:21:26,186][105692] Updated weights for policy 0, policy_version 1505488 (0.0009) [2023-12-27 02:21:26,233][105692] Updated weights for policy 0, policy_version 1505498 (0.0009) [2023-12-27 02:21:26,292][105692] Updated weights for policy 0, policy_version 1505508 (0.0009) [2023-12-27 02:21:26,916][105692] Updated weights for policy 0, policy_version 1505518 (0.0007) [2023-12-27 02:21:26,950][105620] Updated weights for policy 1, policy_version 1508112 (0.0006) [2023-12-27 02:21:26,969][105692] Updated weights for policy 0, policy_version 1505528 (0.0005) [2023-12-27 02:21:26,997][105620] Updated weights for policy 1, policy_version 1508122 (0.0005) [2023-12-27 02:21:27,030][105692] Updated weights for policy 0, policy_version 1505538 (0.0005) [2023-12-27 02:21:27,044][105620] Updated weights for policy 1, policy_version 1508132 (0.0005) [2023-12-27 02:21:27,583][105692] Updated weights for policy 0, policy_version 1505548 (0.0007) [2023-12-27 02:21:27,600][105620] Updated weights for policy 1, policy_version 1508142 (0.0005) [2023-12-27 02:21:27,646][105692] Updated weights for policy 0, policy_version 1505558 (0.0005) [2023-12-27 02:21:27,646][105620] Updated weights for policy 1, policy_version 1508152 (0.0005) [2023-12-27 02:21:27,696][105620] Updated weights for policy 1, policy_version 1508162 (0.0005) [2023-12-27 02:21:27,704][105692] Updated weights for policy 0, policy_version 1505568 (0.0005) [2023-12-27 02:21:28,262][105620] Updated weights for policy 1, policy_version 1508172 (0.0008) [2023-12-27 02:21:28,321][105620] Updated weights for policy 1, policy_version 1508182 (0.0009) [2023-12-27 02:21:28,382][105620] Updated weights for policy 1, policy_version 1508192 (0.0010) [2023-12-27 02:21:28,435][105692] Updated weights for policy 0, policy_version 1505578 (0.0007) [2023-12-27 02:21:28,500][105692] Updated weights for policy 0, policy_version 1505588 (0.0005) [2023-12-27 02:21:28,559][105692] Updated weights for policy 0, policy_version 1505598 (0.0005) [2023-12-27 02:21:28,615][105692] Updated weights for policy 0, policy_version 1505608 (0.0005) [2023-12-27 02:21:29,193][105620] Updated weights for policy 1, policy_version 1508202 (0.0009) [2023-12-27 02:21:29,263][105620] Updated weights for policy 1, policy_version 1508212 (0.0008) [2023-12-27 02:21:29,286][105692] Updated weights for policy 0, policy_version 1505618 (0.0008) [2023-12-27 02:21:29,312][105620] Updated weights for policy 1, policy_version 1508222 (0.0006) [2023-12-27 02:21:29,349][105692] Updated weights for policy 0, policy_version 1505628 (0.0009) [2023-12-27 02:21:29,375][105620] Updated weights for policy 1, policy_version 1508232 (0.0010) [2023-12-27 02:21:29,415][105692] Updated weights for policy 0, policy_version 1505638 (0.0009) [2023-12-27 02:21:30,050][105692] Updated weights for policy 0, policy_version 1505648 (0.0009) [2023-12-27 02:21:30,105][105692] Updated weights for policy 0, policy_version 1505658 (0.0009) [2023-12-27 02:21:30,171][105692] Updated weights for policy 0, policy_version 1505668 (0.0009) [2023-12-27 02:21:30,185][105620] Updated weights for policy 1, policy_version 1508242 (0.0006) [2023-12-27 02:21:30,238][105620] Updated weights for policy 1, policy_version 1508252 (0.0008) [2023-12-27 02:21:30,300][105620] Updated weights for policy 1, policy_version 1508262 (0.0009) [2023-12-27 02:21:30,862][105692] Updated weights for policy 0, policy_version 1505678 (0.0009) [2023-12-27 02:21:30,920][105692] Updated weights for policy 0, policy_version 1505688 (0.0009) [2023-12-27 02:21:30,974][105692] Updated weights for policy 0, policy_version 1505698 (0.0010) [2023-12-27 02:21:31,062][104569] Fps is (10 sec: 20481.0, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 771686400. Throughput: 0: 10034.5, 1: 9592.0. Samples: 771656680. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:21:31,062][104569] Avg episode reward: [(0, '8630.443'), (1, '9167.364')] [2023-12-27 02:21:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001505704_385515520.pth... [2023-12-27 02:21:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001504520_385212416.pth [2023-12-27 02:21:31,096][105620] Updated weights for policy 1, policy_version 1508272 (0.0008) [2023-12-27 02:21:31,165][105620] Updated weights for policy 1, policy_version 1508282 (0.0009) [2023-12-27 02:21:31,221][105620] Updated weights for policy 1, policy_version 1508292 (0.0008) [2023-12-27 02:21:31,244][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001508296_386179072.pth... [2023-12-27 02:21:31,252][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001507144_385884160.pth [2023-12-27 02:21:31,728][105692] Updated weights for policy 0, policy_version 1505708 (0.0010) [2023-12-27 02:21:31,776][105692] Updated weights for policy 0, policy_version 1505718 (0.0010) [2023-12-27 02:21:31,839][105692] Updated weights for policy 0, policy_version 1505728 (0.0010) [2023-12-27 02:21:31,970][105620] Updated weights for policy 1, policy_version 1508302 (0.0009) [2023-12-27 02:21:32,018][105620] Updated weights for policy 1, policy_version 1508312 (0.0008) [2023-12-27 02:21:32,077][105620] Updated weights for policy 1, policy_version 1508322 (0.0008) [2023-12-27 02:21:32,590][105692] Updated weights for policy 0, policy_version 1505738 (0.0010) [2023-12-27 02:21:32,642][105692] Updated weights for policy 0, policy_version 1505748 (0.0011) [2023-12-27 02:21:32,686][105692] Updated weights for policy 0, policy_version 1505758 (0.0010) [2023-12-27 02:21:32,734][105692] Updated weights for policy 0, policy_version 1505768 (0.0010) [2023-12-27 02:21:32,832][105620] Updated weights for policy 1, policy_version 1508332 (0.0007) [2023-12-27 02:21:32,895][105620] Updated weights for policy 1, policy_version 1508342 (0.0005) [2023-12-27 02:21:32,952][105620] Updated weights for policy 1, policy_version 1508352 (0.0005) [2023-12-27 02:21:33,448][105692] Updated weights for policy 0, policy_version 1505778 (0.0010) [2023-12-27 02:21:33,459][105620] Updated weights for policy 1, policy_version 1508362 (0.0006) [2023-12-27 02:21:33,503][105692] Updated weights for policy 0, policy_version 1505788 (0.0010) [2023-12-27 02:21:33,505][105620] Updated weights for policy 1, policy_version 1508372 (0.0008) [2023-12-27 02:21:33,552][105620] Updated weights for policy 1, policy_version 1508382 (0.0006) [2023-12-27 02:21:33,561][105692] Updated weights for policy 0, policy_version 1505798 (0.0009) [2023-12-27 02:21:33,602][105620] Updated weights for policy 1, policy_version 1508392 (0.0009) [2023-12-27 02:21:34,270][105692] Updated weights for policy 0, policy_version 1505808 (0.0010) [2023-12-27 02:21:34,333][105692] Updated weights for policy 0, policy_version 1505818 (0.0010) [2023-12-27 02:21:34,350][105620] Updated weights for policy 1, policy_version 1508402 (0.0005) [2023-12-27 02:21:34,392][105692] Updated weights for policy 0, policy_version 1505828 (0.0010) [2023-12-27 02:21:34,406][105620] Updated weights for policy 1, policy_version 1508412 (0.0006) [2023-12-27 02:21:34,463][105620] Updated weights for policy 1, policy_version 1508422 (0.0008) [2023-12-27 02:21:35,129][105620] Updated weights for policy 1, policy_version 1508432 (0.0006) [2023-12-27 02:21:35,142][105692] Updated weights for policy 0, policy_version 1505838 (0.0010) [2023-12-27 02:21:35,196][105620] Updated weights for policy 1, policy_version 1508442 (0.0005) [2023-12-27 02:21:35,197][105692] Updated weights for policy 0, policy_version 1505848 (0.0010) [2023-12-27 02:21:35,251][105620] Updated weights for policy 1, policy_version 1508452 (0.0005) [2023-12-27 02:21:35,258][105692] Updated weights for policy 0, policy_version 1505858 (0.0010) [2023-12-27 02:21:35,824][105620] Updated weights for policy 1, policy_version 1508462 (0.0007) [2023-12-27 02:21:35,891][105620] Updated weights for policy 1, policy_version 1508472 (0.0007) [2023-12-27 02:21:35,950][105620] Updated weights for policy 1, policy_version 1508482 (0.0010) [2023-12-27 02:21:35,987][105692] Updated weights for policy 0, policy_version 1505868 (0.0010) [2023-12-27 02:21:36,039][105692] Updated weights for policy 0, policy_version 1505878 (0.0010) [2023-12-27 02:21:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 771784704. Throughput: 0: 9907.3, 1: 9745.7. Samples: 771773552. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:21:36,063][104569] Avg episode reward: [(0, '8630.257'), (1, '9167.204')] [2023-12-27 02:21:36,087][105692] Updated weights for policy 0, policy_version 1505888 (0.0010) [2023-12-27 02:21:36,711][105620] Updated weights for policy 1, policy_version 1508492 (0.0011) [2023-12-27 02:21:36,771][105692] Updated weights for policy 0, policy_version 1505898 (0.0006) [2023-12-27 02:21:36,774][105620] Updated weights for policy 1, policy_version 1508502 (0.0011) [2023-12-27 02:21:36,826][105692] Updated weights for policy 0, policy_version 1505908 (0.0006) [2023-12-27 02:21:36,831][105620] Updated weights for policy 1, policy_version 1508512 (0.0010) [2023-12-27 02:21:36,888][105692] Updated weights for policy 0, policy_version 1505918 (0.0006) [2023-12-27 02:21:36,943][105692] Updated weights for policy 0, policy_version 1505928 (0.0008) [2023-12-27 02:21:37,442][105620] Updated weights for policy 1, policy_version 1508522 (0.0010) [2023-12-27 02:21:37,498][105620] Updated weights for policy 1, policy_version 1508532 (0.0005) [2023-12-27 02:21:37,548][105620] Updated weights for policy 1, policy_version 1508542 (0.0009) [2023-12-27 02:21:37,600][105620] Updated weights for policy 1, policy_version 1508552 (0.0010) [2023-12-27 02:21:37,645][105692] Updated weights for policy 0, policy_version 1505938 (0.0008) [2023-12-27 02:21:37,697][105692] Updated weights for policy 0, policy_version 1505948 (0.0008) [2023-12-27 02:21:37,757][105692] Updated weights for policy 0, policy_version 1505958 (0.0008) [2023-12-27 02:21:38,349][105620] Updated weights for policy 1, policy_version 1508562 (0.0007) [2023-12-27 02:21:38,415][105620] Updated weights for policy 1, policy_version 1508572 (0.0009) [2023-12-27 02:21:38,476][105620] Updated weights for policy 1, policy_version 1508582 (0.0009) [2023-12-27 02:21:38,525][105692] Updated weights for policy 0, policy_version 1505968 (0.0009) [2023-12-27 02:21:38,587][105692] Updated weights for policy 0, policy_version 1505978 (0.0008) [2023-12-27 02:21:38,647][105692] Updated weights for policy 0, policy_version 1505988 (0.0010) [2023-12-27 02:21:39,222][105620] Updated weights for policy 1, policy_version 1508592 (0.0008) [2023-12-27 02:21:39,285][105620] Updated weights for policy 1, policy_version 1508602 (0.0008) [2023-12-27 02:21:39,332][105620] Updated weights for policy 1, policy_version 1508612 (0.0009) [2023-12-27 02:21:39,406][105692] Updated weights for policy 0, policy_version 1505998 (0.0010) [2023-12-27 02:21:39,473][105692] Updated weights for policy 0, policy_version 1506008 (0.0009) [2023-12-27 02:21:39,535][105692] Updated weights for policy 0, policy_version 1506018 (0.0006) [2023-12-27 02:21:40,214][105692] Updated weights for policy 0, policy_version 1506028 (0.0006) [2023-12-27 02:21:40,216][105620] Updated weights for policy 1, policy_version 1508622 (0.0009) [2023-12-27 02:21:40,265][105692] Updated weights for policy 0, policy_version 1506038 (0.0007) [2023-12-27 02:21:40,280][105620] Updated weights for policy 1, policy_version 1508632 (0.0008) [2023-12-27 02:21:40,320][105692] Updated weights for policy 0, policy_version 1506048 (0.0008) [2023-12-27 02:21:40,330][105620] Updated weights for policy 1, policy_version 1508642 (0.0007) [2023-12-27 02:21:40,948][105692] Updated weights for policy 0, policy_version 1506058 (0.0006) [2023-12-27 02:21:41,007][105692] Updated weights for policy 0, policy_version 1506068 (0.0009) [2023-12-27 02:21:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 771874816. Throughput: 0: 9980.0, 1: 9712.6. Samples: 771889268. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:21:41,062][104569] Avg episode reward: [(0, '8536.890'), (1, '9088.141')] [2023-12-27 02:21:41,076][105692] Updated weights for policy 0, policy_version 1506078 (0.0008) [2023-12-27 02:21:41,128][105692] Updated weights for policy 0, policy_version 1506088 (0.0008) [2023-12-27 02:21:41,158][105620] Updated weights for policy 1, policy_version 1508652 (0.0008) [2023-12-27 02:21:41,221][105620] Updated weights for policy 1, policy_version 1508662 (0.0007) [2023-12-27 02:21:41,288][105620] Updated weights for policy 1, policy_version 1508672 (0.0008) [2023-12-27 02:21:41,877][105692] Updated weights for policy 0, policy_version 1506098 (0.0009) [2023-12-27 02:21:41,933][105692] Updated weights for policy 0, policy_version 1506108 (0.0009) [2023-12-27 02:21:41,984][105692] Updated weights for policy 0, policy_version 1506118 (0.0008) [2023-12-27 02:21:42,078][105620] Updated weights for policy 1, policy_version 1508682 (0.0009) [2023-12-27 02:21:42,136][105620] Updated weights for policy 1, policy_version 1508692 (0.0009) [2023-12-27 02:21:42,199][105620] Updated weights for policy 1, policy_version 1508702 (0.0009) [2023-12-27 02:21:42,257][105620] Updated weights for policy 1, policy_version 1508712 (0.0009) [2023-12-27 02:21:42,798][105692] Updated weights for policy 0, policy_version 1506128 (0.0009) [2023-12-27 02:21:42,845][105692] Updated weights for policy 0, policy_version 1506138 (0.0009) [2023-12-27 02:21:42,903][105692] Updated weights for policy 0, policy_version 1506148 (0.0008) [2023-12-27 02:21:42,941][105620] Updated weights for policy 1, policy_version 1508722 (0.0007) [2023-12-27 02:21:42,997][105620] Updated weights for policy 1, policy_version 1508732 (0.0008) [2023-12-27 02:21:43,052][105620] Updated weights for policy 1, policy_version 1508742 (0.0008) [2023-12-27 02:21:43,559][105692] Updated weights for policy 0, policy_version 1506158 (0.0008) [2023-12-27 02:21:43,607][105692] Updated weights for policy 0, policy_version 1506168 (0.0010) [2023-12-27 02:21:43,651][105692] Updated weights for policy 0, policy_version 1506178 (0.0009) [2023-12-27 02:21:43,873][105620] Updated weights for policy 1, policy_version 1508752 (0.0009) [2023-12-27 02:21:43,936][105620] Updated weights for policy 1, policy_version 1508762 (0.0008) [2023-12-27 02:21:43,983][105620] Updated weights for policy 1, policy_version 1508772 (0.0009) [2023-12-27 02:21:44,320][105692] Updated weights for policy 0, policy_version 1506188 (0.0007) [2023-12-27 02:21:44,381][105692] Updated weights for policy 0, policy_version 1506198 (0.0010) [2023-12-27 02:21:44,429][105692] Updated weights for policy 0, policy_version 1506208 (0.0010) [2023-12-27 02:21:44,763][105620] Updated weights for policy 1, policy_version 1508782 (0.0008) [2023-12-27 02:21:44,825][105620] Updated weights for policy 1, policy_version 1508792 (0.0007) [2023-12-27 02:21:44,889][105620] Updated weights for policy 1, policy_version 1508802 (0.0007) [2023-12-27 02:21:45,120][105692] Updated weights for policy 0, policy_version 1506218 (0.0010) [2023-12-27 02:21:45,168][105692] Updated weights for policy 0, policy_version 1506228 (0.0009) [2023-12-27 02:21:45,228][105692] Updated weights for policy 0, policy_version 1506238 (0.0009) [2023-12-27 02:21:45,276][105692] Updated weights for policy 0, policy_version 1506248 (0.0009) [2023-12-27 02:21:45,584][105620] Updated weights for policy 1, policy_version 1508812 (0.0009) [2023-12-27 02:21:45,639][105620] Updated weights for policy 1, policy_version 1508822 (0.0009) [2023-12-27 02:21:45,689][105620] Updated weights for policy 1, policy_version 1508833 (0.0009) [2023-12-27 02:21:45,960][105692] Updated weights for policy 0, policy_version 1506258 (0.0005) [2023-12-27 02:21:46,014][105692] Updated weights for policy 0, policy_version 1506268 (0.0005) [2023-12-27 02:21:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 771973120. Throughput: 0: 9949.5, 1: 9616.0. Samples: 771945952. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:21:46,063][104569] Avg episode reward: [(0, '8721.840'), (1, '8996.471')] [2023-12-27 02:21:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001508840_386318336.pth... [2023-12-27 02:21:46,072][105692] Updated weights for policy 0, policy_version 1506278 (0.0005) [2023-12-27 02:21:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001507720_386031616.pth [2023-12-27 02:21:46,074][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_001508840_386318336.pth [2023-12-27 02:21:46,081][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001506280_385662976.pth... [2023-12-27 02:21:46,085][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001505128_385368064.pth [2023-12-27 02:21:46,086][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_001506280_385662976.pth [2023-12-27 02:21:46,603][105620] Updated weights for policy 1, policy_version 1508844 (0.0010) [2023-12-27 02:21:46,645][105692] Updated weights for policy 0, policy_version 1506288 (0.0006) [2023-12-27 02:21:46,652][105620] Updated weights for policy 1, policy_version 1508854 (0.0008) [2023-12-27 02:21:46,703][105692] Updated weights for policy 0, policy_version 1506298 (0.0006) [2023-12-27 02:21:46,715][105620] Updated weights for policy 1, policy_version 1508864 (0.0008) [2023-12-27 02:21:46,761][105692] Updated weights for policy 0, policy_version 1506308 (0.0006) [2023-12-27 02:21:47,297][105692] Updated weights for policy 0, policy_version 1506318 (0.0006) [2023-12-27 02:21:47,341][105692] Updated weights for policy 0, policy_version 1506328 (0.0005) [2023-12-27 02:21:47,401][105692] Updated weights for policy 0, policy_version 1506338 (0.0005) [2023-12-27 02:21:47,573][105620] Updated weights for policy 1, policy_version 1508874 (0.0009) [2023-12-27 02:21:47,637][105620] Updated weights for policy 1, policy_version 1508884 (0.0008) [2023-12-27 02:21:47,700][105620] Updated weights for policy 1, policy_version 1508894 (0.0009) [2023-12-27 02:21:47,763][105620] Updated weights for policy 1, policy_version 1508904 (0.0009) [2023-12-27 02:21:48,109][105692] Updated weights for policy 0, policy_version 1506348 (0.0009) [2023-12-27 02:21:48,168][105692] Updated weights for policy 0, policy_version 1506358 (0.0007) [2023-12-27 02:21:48,223][105692] Updated weights for policy 0, policy_version 1506368 (0.0005) [2023-12-27 02:21:48,540][105620] Updated weights for policy 1, policy_version 1508914 (0.0009) [2023-12-27 02:21:48,587][105620] Updated weights for policy 1, policy_version 1508924 (0.0008) [2023-12-27 02:21:48,638][105620] Updated weights for policy 1, policy_version 1508934 (0.0009) [2023-12-27 02:21:48,873][105692] Updated weights for policy 0, policy_version 1506378 (0.0005) [2023-12-27 02:21:48,933][105692] Updated weights for policy 0, policy_version 1506388 (0.0009) [2023-12-27 02:21:48,996][105692] Updated weights for policy 0, policy_version 1506398 (0.0009) [2023-12-27 02:21:49,060][105692] Updated weights for policy 0, policy_version 1506408 (0.0010) [2023-12-27 02:21:49,454][105620] Updated weights for policy 1, policy_version 1508944 (0.0009) [2023-12-27 02:21:49,513][105620] Updated weights for policy 1, policy_version 1508954 (0.0008) [2023-12-27 02:21:49,563][105620] Updated weights for policy 1, policy_version 1508964 (0.0009) [2023-12-27 02:21:49,813][105692] Updated weights for policy 0, policy_version 1506418 (0.0008) [2023-12-27 02:21:49,879][105692] Updated weights for policy 0, policy_version 1506428 (0.0009) [2023-12-27 02:21:49,949][105692] Updated weights for policy 0, policy_version 1506438 (0.0007) [2023-12-27 02:21:50,290][105620] Updated weights for policy 1, policy_version 1508974 (0.0009) [2023-12-27 02:21:50,357][105620] Updated weights for policy 1, policy_version 1508984 (0.0006) [2023-12-27 02:21:50,427][105620] Updated weights for policy 1, policy_version 1508994 (0.0006) [2023-12-27 02:21:50,783][105692] Updated weights for policy 0, policy_version 1506448 (0.0010) [2023-12-27 02:21:50,844][105692] Updated weights for policy 0, policy_version 1506458 (0.0009) [2023-12-27 02:21:50,901][105692] Updated weights for policy 0, policy_version 1506468 (0.0009) [2023-12-27 02:21:51,058][105620] Updated weights for policy 1, policy_version 1509004 (0.0007) [2023-12-27 02:21:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 772071424. Throughput: 0: 10068.3, 1: 9533.1. Samples: 772062152. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:21:51,062][104569] Avg episode reward: [(0, '8448.125'), (1, '9079.447')] [2023-12-27 02:21:51,123][105620] Updated weights for policy 1, policy_version 1509014 (0.0008) [2023-12-27 02:21:51,183][105620] Updated weights for policy 1, policy_version 1509024 (0.0007) [2023-12-27 02:21:51,715][105692] Updated weights for policy 0, policy_version 1506478 (0.0009) [2023-12-27 02:21:51,779][105692] Updated weights for policy 0, policy_version 1506488 (0.0009) [2023-12-27 02:21:51,842][105692] Updated weights for policy 0, policy_version 1506498 (0.0010) [2023-12-27 02:21:51,897][105620] Updated weights for policy 1, policy_version 1509034 (0.0009) [2023-12-27 02:21:51,961][105620] Updated weights for policy 1, policy_version 1509044 (0.0009) [2023-12-27 02:21:52,019][105620] Updated weights for policy 1, policy_version 1509054 (0.0009) [2023-12-27 02:21:52,074][105620] Updated weights for policy 1, policy_version 1509064 (0.0009) [2023-12-27 02:21:52,592][105692] Updated weights for policy 0, policy_version 1506508 (0.0008) [2023-12-27 02:21:52,651][105692] Updated weights for policy 0, policy_version 1506518 (0.0005) [2023-12-27 02:21:52,706][105692] Updated weights for policy 0, policy_version 1506528 (0.0005) [2023-12-27 02:21:52,881][105620] Updated weights for policy 1, policy_version 1509074 (0.0010) [2023-12-27 02:21:52,939][105620] Updated weights for policy 1, policy_version 1509084 (0.0009) [2023-12-27 02:21:53,000][105620] Updated weights for policy 1, policy_version 1509094 (0.0006) [2023-12-27 02:21:53,339][105692] Updated weights for policy 0, policy_version 1506538 (0.0006) [2023-12-27 02:21:53,400][105692] Updated weights for policy 0, policy_version 1506548 (0.0009) [2023-12-27 02:21:53,454][105692] Updated weights for policy 0, policy_version 1506558 (0.0008) [2023-12-27 02:21:53,511][105692] Updated weights for policy 0, policy_version 1506568 (0.0009) [2023-12-27 02:21:53,741][105620] Updated weights for policy 1, policy_version 1509104 (0.0008) [2023-12-27 02:21:53,803][105620] Updated weights for policy 1, policy_version 1509114 (0.0009) [2023-12-27 02:21:53,854][105620] Updated weights for policy 1, policy_version 1509124 (0.0009) [2023-12-27 02:21:54,215][105692] Updated weights for policy 0, policy_version 1506578 (0.0007) [2023-12-27 02:21:54,279][105692] Updated weights for policy 0, policy_version 1506588 (0.0006) [2023-12-27 02:21:54,341][105692] Updated weights for policy 0, policy_version 1506598 (0.0005) [2023-12-27 02:21:54,686][105620] Updated weights for policy 1, policy_version 1509134 (0.0010) [2023-12-27 02:21:54,747][105620] Updated weights for policy 1, policy_version 1509144 (0.0009) [2023-12-27 02:21:54,801][105620] Updated weights for policy 1, policy_version 1509154 (0.0009) [2023-12-27 02:21:54,909][105692] Updated weights for policy 0, policy_version 1506608 (0.0005) [2023-12-27 02:21:54,970][105692] Updated weights for policy 0, policy_version 1506618 (0.0006) [2023-12-27 02:21:55,021][105692] Updated weights for policy 0, policy_version 1506628 (0.0006) [2023-12-27 02:21:55,634][105620] Updated weights for policy 1, policy_version 1509164 (0.0009) [2023-12-27 02:21:55,674][105692] Updated weights for policy 0, policy_version 1506638 (0.0006) [2023-12-27 02:21:55,684][105620] Updated weights for policy 1, policy_version 1509174 (0.0008) [2023-12-27 02:21:55,730][105692] Updated weights for policy 0, policy_version 1506648 (0.0007) [2023-12-27 02:21:55,749][105620] Updated weights for policy 1, policy_version 1509184 (0.0006) [2023-12-27 02:21:55,791][105692] Updated weights for policy 0, policy_version 1506658 (0.0007) [2023-12-27 02:21:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 772169728. Throughput: 0: 9957.6, 1: 9476.7. Samples: 772175876. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:21:56,063][104569] Avg episode reward: [(0, '8088.175'), (1, '9169.089')] [2023-12-27 02:21:56,364][105692] Updated weights for policy 0, policy_version 1506668 (0.0005) [2023-12-27 02:21:56,422][105692] Updated weights for policy 0, policy_version 1506678 (0.0006) [2023-12-27 02:21:56,477][105692] Updated weights for policy 0, policy_version 1506688 (0.0009) [2023-12-27 02:21:56,583][105620] Updated weights for policy 1, policy_version 1509194 (0.0008) [2023-12-27 02:21:56,630][105620] Updated weights for policy 1, policy_version 1509204 (0.0009) [2023-12-27 02:21:56,681][105620] Updated weights for policy 1, policy_version 1509214 (0.0009) [2023-12-27 02:21:56,732][105620] Updated weights for policy 1, policy_version 1509224 (0.0006) [2023-12-27 02:21:57,140][105692] Updated weights for policy 0, policy_version 1506698 (0.0009) [2023-12-27 02:21:57,199][105692] Updated weights for policy 0, policy_version 1506708 (0.0008) [2023-12-27 02:21:57,266][105692] Updated weights for policy 0, policy_version 1506718 (0.0008) [2023-12-27 02:21:57,328][105692] Updated weights for policy 0, policy_version 1506728 (0.0008) [2023-12-27 02:21:57,528][105620] Updated weights for policy 1, policy_version 1509234 (0.0009) [2023-12-27 02:21:57,575][105620] Updated weights for policy 1, policy_version 1509244 (0.0008) [2023-12-27 02:21:57,625][105620] Updated weights for policy 1, policy_version 1509254 (0.0009) [2023-12-27 02:21:57,959][105692] Updated weights for policy 0, policy_version 1506738 (0.0008) [2023-12-27 02:21:58,002][105585] KL-divergence is very high: 138.1876 [2023-12-27 02:21:58,013][105692] Updated weights for policy 0, policy_version 1506748 (0.0005) [2023-12-27 02:21:58,049][105585] KL-divergence is very high: 148.1640 [2023-12-27 02:21:58,071][105692] Updated weights for policy 0, policy_version 1506758 (0.0005) [2023-12-27 02:21:58,476][105620] Updated weights for policy 1, policy_version 1509264 (0.0009) [2023-12-27 02:21:58,540][105620] Updated weights for policy 1, policy_version 1509274 (0.0008) [2023-12-27 02:21:58,593][105620] Updated weights for policy 1, policy_version 1509284 (0.0008) [2023-12-27 02:21:58,761][105692] Updated weights for policy 0, policy_version 1506768 (0.0007) [2023-12-27 02:21:58,825][105692] Updated weights for policy 0, policy_version 1506778 (0.0009) [2023-12-27 02:21:58,891][105692] Updated weights for policy 0, policy_version 1506788 (0.0009) [2023-12-27 02:21:59,406][105620] Updated weights for policy 1, policy_version 1509294 (0.0007) [2023-12-27 02:21:59,470][105620] Updated weights for policy 1, policy_version 1509304 (0.0006) [2023-12-27 02:21:59,536][105620] Updated weights for policy 1, policy_version 1509314 (0.0006) [2023-12-27 02:21:59,721][105692] Updated weights for policy 0, policy_version 1506798 (0.0007) [2023-12-27 02:21:59,782][105692] Updated weights for policy 0, policy_version 1506808 (0.0009) [2023-12-27 02:21:59,857][105692] Updated weights for policy 0, policy_version 1506819 (0.0009) [2023-12-27 02:22:00,187][105620] Updated weights for policy 1, policy_version 1509324 (0.0008) [2023-12-27 02:22:00,238][105620] Updated weights for policy 1, policy_version 1509334 (0.0010) [2023-12-27 02:22:00,290][105620] Updated weights for policy 1, policy_version 1509344 (0.0010) [2023-12-27 02:22:00,522][105692] Updated weights for policy 0, policy_version 1506829 (0.0011) [2023-12-27 02:22:00,577][105692] Updated weights for policy 0, policy_version 1506839 (0.0010) [2023-12-27 02:22:00,628][105692] Updated weights for policy 0, policy_version 1506849 (0.0010) [2023-12-27 02:22:00,859][105620] Updated weights for policy 1, policy_version 1509354 (0.0009) [2023-12-27 02:22:00,907][105620] Updated weights for policy 1, policy_version 1509364 (0.0005) [2023-12-27 02:22:00,960][105620] Updated weights for policy 1, policy_version 1509374 (0.0005) [2023-12-27 02:22:01,015][105620] Updated weights for policy 1, policy_version 1509384 (0.0006) [2023-12-27 02:22:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 772268032. Throughput: 0: 9993.1, 1: 9375.4. Samples: 772233588. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:22:01,062][104569] Avg episode reward: [(0, '7992.561'), (1, '9350.163')] [2023-12-27 02:22:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001506856_385810432.pth... [2023-12-27 02:22:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001509384_386457600.pth... [2023-12-27 02:22:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001505704_385515520.pth [2023-12-27 02:22:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001508296_386179072.pth [2023-12-27 02:22:01,223][105692] Updated weights for policy 0, policy_version 1506859 (0.0008) [2023-12-27 02:22:01,283][105692] Updated weights for policy 0, policy_version 1506869 (0.0011) [2023-12-27 02:22:01,340][105692] Updated weights for policy 0, policy_version 1506879 (0.0010) [2023-12-27 02:22:01,672][105620] Updated weights for policy 1, policy_version 1509394 (0.0007) [2023-12-27 02:22:01,742][105620] Updated weights for policy 1, policy_version 1509404 (0.0008) [2023-12-27 02:22:01,798][105620] Updated weights for policy 1, policy_version 1509414 (0.0006) [2023-12-27 02:22:02,107][105692] Updated weights for policy 0, policy_version 1506889 (0.0008) [2023-12-27 02:22:02,172][105692] Updated weights for policy 0, policy_version 1506899 (0.0010) [2023-12-27 02:22:02,230][105692] Updated weights for policy 0, policy_version 1506909 (0.0009) [2023-12-27 02:22:02,297][105692] Updated weights for policy 0, policy_version 1506919 (0.0010) [2023-12-27 02:22:02,428][105620] Updated weights for policy 1, policy_version 1509424 (0.0005) [2023-12-27 02:22:02,489][105620] Updated weights for policy 1, policy_version 1509434 (0.0007) [2023-12-27 02:22:02,537][105620] Updated weights for policy 1, policy_version 1509444 (0.0008) [2023-12-27 02:22:03,028][105692] Updated weights for policy 0, policy_version 1506929 (0.0010) [2023-12-27 02:22:03,082][105620] Updated weights for policy 1, policy_version 1509454 (0.0008) [2023-12-27 02:22:03,090][105692] Updated weights for policy 0, policy_version 1506939 (0.0011) [2023-12-27 02:22:03,143][105620] Updated weights for policy 1, policy_version 1509464 (0.0007) [2023-12-27 02:22:03,148][105692] Updated weights for policy 0, policy_version 1506949 (0.0010) [2023-12-27 02:22:03,200][105620] Updated weights for policy 1, policy_version 1509474 (0.0007) [2023-12-27 02:22:03,886][105692] Updated weights for policy 0, policy_version 1506959 (0.0011) [2023-12-27 02:22:03,910][105620] Updated weights for policy 1, policy_version 1509484 (0.0008) [2023-12-27 02:22:03,949][105692] Updated weights for policy 0, policy_version 1506969 (0.0009) [2023-12-27 02:22:03,966][105620] Updated weights for policy 1, policy_version 1509494 (0.0007) [2023-12-27 02:22:04,005][105692] Updated weights for policy 0, policy_version 1506979 (0.0005) [2023-12-27 02:22:04,027][105620] Updated weights for policy 1, policy_version 1509504 (0.0009) [2023-12-27 02:22:04,576][105692] Updated weights for policy 0, policy_version 1506989 (0.0006) [2023-12-27 02:22:04,642][105692] Updated weights for policy 0, policy_version 1506999 (0.0009) [2023-12-27 02:22:04,700][105692] Updated weights for policy 0, policy_version 1507009 (0.0010) [2023-12-27 02:22:04,879][105620] Updated weights for policy 1, policy_version 1509514 (0.0009) [2023-12-27 02:22:04,940][105620] Updated weights for policy 1, policy_version 1509524 (0.0008) [2023-12-27 02:22:04,990][105620] Updated weights for policy 1, policy_version 1509534 (0.0008) [2023-12-27 02:22:05,049][105620] Updated weights for policy 1, policy_version 1509544 (0.0007) [2023-12-27 02:22:05,336][105692] Updated weights for policy 0, policy_version 1507019 (0.0007) [2023-12-27 02:22:05,396][105692] Updated weights for policy 0, policy_version 1507029 (0.0007) [2023-12-27 02:22:05,449][105692] Updated weights for policy 0, policy_version 1507039 (0.0005) [2023-12-27 02:22:05,770][105620] Updated weights for policy 1, policy_version 1509554 (0.0005) [2023-12-27 02:22:05,816][105620] Updated weights for policy 1, policy_version 1509564 (0.0005) [2023-12-27 02:22:05,869][105620] Updated weights for policy 1, policy_version 1509574 (0.0005) [2023-12-27 02:22:05,973][105692] Updated weights for policy 0, policy_version 1507049 (0.0006) [2023-12-27 02:22:06,019][105692] Updated weights for policy 0, policy_version 1507059 (0.0008) [2023-12-27 02:22:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 772366336. Throughput: 0: 9964.9, 1: 9529.7. Samples: 772355044. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:22:06,062][104569] Avg episode reward: [(0, '7903.865'), (1, '9350.414')] [2023-12-27 02:22:06,070][105692] Updated weights for policy 0, policy_version 1507069 (0.0006) [2023-12-27 02:22:06,128][105692] Updated weights for policy 0, policy_version 1507079 (0.0009) [2023-12-27 02:22:06,512][105620] Updated weights for policy 1, policy_version 1509584 (0.0008) [2023-12-27 02:22:06,578][105620] Updated weights for policy 1, policy_version 1509594 (0.0010) [2023-12-27 02:22:06,641][105620] Updated weights for policy 1, policy_version 1509604 (0.0009) [2023-12-27 02:22:06,877][105692] Updated weights for policy 0, policy_version 1507089 (0.0007) [2023-12-27 02:22:06,929][105692] Updated weights for policy 0, policy_version 1507099 (0.0005) [2023-12-27 02:22:06,990][105692] Updated weights for policy 0, policy_version 1507109 (0.0006) [2023-12-27 02:22:07,488][105620] Updated weights for policy 1, policy_version 1509614 (0.0008) [2023-12-27 02:22:07,544][105620] Updated weights for policy 1, policy_version 1509624 (0.0008) [2023-12-27 02:22:07,601][105620] Updated weights for policy 1, policy_version 1509634 (0.0009) [2023-12-27 02:22:07,604][105692] Updated weights for policy 0, policy_version 1507119 (0.0008) [2023-12-27 02:22:07,664][105692] Updated weights for policy 0, policy_version 1507129 (0.0008) [2023-12-27 02:22:07,712][105692] Updated weights for policy 0, policy_version 1507139 (0.0008) [2023-12-27 02:22:08,318][105620] Updated weights for policy 1, policy_version 1509644 (0.0007) [2023-12-27 02:22:08,389][105620] Updated weights for policy 1, policy_version 1509654 (0.0009) [2023-12-27 02:22:08,445][105620] Updated weights for policy 1, policy_version 1509664 (0.0008) [2023-12-27 02:22:08,468][105692] Updated weights for policy 0, policy_version 1507149 (0.0007) [2023-12-27 02:22:08,530][105692] Updated weights for policy 0, policy_version 1507159 (0.0005) [2023-12-27 02:22:08,581][105692] Updated weights for policy 0, policy_version 1507169 (0.0005) [2023-12-27 02:22:09,146][105692] Updated weights for policy 0, policy_version 1507179 (0.0005) [2023-12-27 02:22:09,183][105620] Updated weights for policy 1, policy_version 1509674 (0.0007) [2023-12-27 02:22:09,199][105692] Updated weights for policy 0, policy_version 1507189 (0.0005) [2023-12-27 02:22:09,249][105620] Updated weights for policy 1, policy_version 1509684 (0.0006) [2023-12-27 02:22:09,259][105692] Updated weights for policy 0, policy_version 1507199 (0.0009) [2023-12-27 02:22:09,319][105620] Updated weights for policy 1, policy_version 1509694 (0.0007) [2023-12-27 02:22:09,390][105620] Updated weights for policy 1, policy_version 1509704 (0.0008) [2023-12-27 02:22:09,969][105692] Updated weights for policy 0, policy_version 1507209 (0.0008) [2023-12-27 02:22:10,033][105692] Updated weights for policy 0, policy_version 1507219 (0.0006) [2023-12-27 02:22:10,099][105692] Updated weights for policy 0, policy_version 1507229 (0.0007) [2023-12-27 02:22:10,100][105620] Updated weights for policy 1, policy_version 1509714 (0.0011) [2023-12-27 02:22:10,160][105620] Updated weights for policy 1, policy_version 1509724 (0.0010) [2023-12-27 02:22:10,163][105692] Updated weights for policy 0, policy_version 1507239 (0.0011) [2023-12-27 02:22:10,224][105620] Updated weights for policy 1, policy_version 1509734 (0.0010) [2023-12-27 02:22:10,828][105692] Updated weights for policy 0, policy_version 1507249 (0.0007) [2023-12-27 02:22:10,845][105620] Updated weights for policy 1, policy_version 1509744 (0.0011) [2023-12-27 02:22:10,886][105692] Updated weights for policy 0, policy_version 1507259 (0.0006) [2023-12-27 02:22:10,907][105620] Updated weights for policy 1, policy_version 1509754 (0.0007) [2023-12-27 02:22:10,951][105692] Updated weights for policy 0, policy_version 1507269 (0.0007) [2023-12-27 02:22:10,967][105620] Updated weights for policy 1, policy_version 1509764 (0.0007) [2023-12-27 02:22:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 772472832. Throughput: 0: 10084.7, 1: 9504.0. Samples: 772475540. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:22:11,062][104569] Avg episode reward: [(0, '7736.889'), (1, '9351.089')] [2023-12-27 02:22:11,667][105692] Updated weights for policy 0, policy_version 1507279 (0.0007) [2023-12-27 02:22:11,730][105692] Updated weights for policy 0, policy_version 1507289 (0.0009) [2023-12-27 02:22:11,750][105620] Updated weights for policy 1, policy_version 1509774 (0.0010) [2023-12-27 02:22:11,790][105692] Updated weights for policy 0, policy_version 1507299 (0.0009) [2023-12-27 02:22:11,807][105620] Updated weights for policy 1, policy_version 1509784 (0.0010) [2023-12-27 02:22:11,864][105620] Updated weights for policy 1, policy_version 1509794 (0.0011) [2023-12-27 02:22:12,554][105692] Updated weights for policy 0, policy_version 1507309 (0.0007) [2023-12-27 02:22:12,615][105692] Updated weights for policy 0, policy_version 1507319 (0.0009) [2023-12-27 02:22:12,636][105620] Updated weights for policy 1, policy_version 1509804 (0.0010) [2023-12-27 02:22:12,668][105692] Updated weights for policy 0, policy_version 1507329 (0.0006) [2023-12-27 02:22:12,698][105620] Updated weights for policy 1, policy_version 1509814 (0.0008) [2023-12-27 02:22:12,751][105620] Updated weights for policy 1, policy_version 1509824 (0.0008) [2023-12-27 02:22:13,341][105692] Updated weights for policy 0, policy_version 1507339 (0.0005) [2023-12-27 02:22:13,396][105692] Updated weights for policy 0, policy_version 1507349 (0.0005) [2023-12-27 02:22:13,463][105692] Updated weights for policy 0, policy_version 1507359 (0.0007) [2023-12-27 02:22:13,562][105620] Updated weights for policy 1, policy_version 1509834 (0.0009) [2023-12-27 02:22:13,614][105620] Updated weights for policy 1, policy_version 1509844 (0.0011) [2023-12-27 02:22:13,671][105620] Updated weights for policy 1, policy_version 1509854 (0.0008) [2023-12-27 02:22:13,720][105620] Updated weights for policy 1, policy_version 1509864 (0.0005) [2023-12-27 02:22:14,225][105692] Updated weights for policy 0, policy_version 1507369 (0.0008) [2023-12-27 02:22:14,279][105692] Updated weights for policy 0, policy_version 1507379 (0.0009) [2023-12-27 02:22:14,308][105620] Updated weights for policy 1, policy_version 1509874 (0.0006) [2023-12-27 02:22:14,337][105692] Updated weights for policy 0, policy_version 1507389 (0.0008) [2023-12-27 02:22:14,373][105620] Updated weights for policy 1, policy_version 1509884 (0.0008) [2023-12-27 02:22:14,401][105692] Updated weights for policy 0, policy_version 1507399 (0.0006) [2023-12-27 02:22:14,430][105620] Updated weights for policy 1, policy_version 1509894 (0.0007) [2023-12-27 02:22:14,987][105620] Updated weights for policy 1, policy_version 1509904 (0.0008) [2023-12-27 02:22:15,040][105620] Updated weights for policy 1, policy_version 1509914 (0.0011) [2023-12-27 02:22:15,097][105620] Updated weights for policy 1, policy_version 1509924 (0.0011) [2023-12-27 02:22:15,233][105692] Updated weights for policy 0, policy_version 1507410 (0.0009) [2023-12-27 02:22:15,280][105692] Updated weights for policy 0, policy_version 1507420 (0.0008) [2023-12-27 02:22:15,326][105692] Updated weights for policy 0, policy_version 1507430 (0.0008) [2023-12-27 02:22:15,919][105620] Updated weights for policy 1, policy_version 1509934 (0.0010) [2023-12-27 02:22:15,975][105620] Updated weights for policy 1, policy_version 1509944 (0.0008) [2023-12-27 02:22:16,025][105620] Updated weights for policy 1, policy_version 1509954 (0.0009) [2023-12-27 02:22:16,048][105692] Updated weights for policy 0, policy_version 1507440 (0.0006) [2023-12-27 02:22:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.4, 300 sec: 19494.2). Total num frames: 772562944. Throughput: 0: 10037.4, 1: 9437.5. Samples: 772533052. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:22:16,062][104569] Avg episode reward: [(0, '8005.857'), (1, '9259.324')] [2023-12-27 02:22:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001509960_386605056.pth... [2023-12-27 02:22:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001508840_386318336.pth [2023-12-27 02:22:16,116][105692] Updated weights for policy 0, policy_version 1507450 (0.0007) [2023-12-27 02:22:16,174][105692] Updated weights for policy 0, policy_version 1507460 (0.0009) [2023-12-27 02:22:16,197][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001507464_385966080.pth... [2023-12-27 02:22:16,200][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001506280_385662976.pth [2023-12-27 02:22:16,695][105620] Updated weights for policy 1, policy_version 1509964 (0.0008) [2023-12-27 02:22:16,754][105620] Updated weights for policy 1, policy_version 1509974 (0.0006) [2023-12-27 02:22:16,814][105620] Updated weights for policy 1, policy_version 1509984 (0.0005) [2023-12-27 02:22:16,894][105692] Updated weights for policy 0, policy_version 1507470 (0.0009) [2023-12-27 02:22:16,944][105692] Updated weights for policy 0, policy_version 1507480 (0.0010) [2023-12-27 02:22:16,998][105692] Updated weights for policy 0, policy_version 1507490 (0.0008) [2023-12-27 02:22:17,357][105620] Updated weights for policy 1, policy_version 1509994 (0.0006) [2023-12-27 02:22:17,405][105620] Updated weights for policy 1, policy_version 1510004 (0.0010) [2023-12-27 02:22:17,458][105620] Updated weights for policy 1, policy_version 1510014 (0.0011) [2023-12-27 02:22:17,523][105620] Updated weights for policy 1, policy_version 1510024 (0.0010) [2023-12-27 02:22:17,849][105692] Updated weights for policy 0, policy_version 1507500 (0.0009) [2023-12-27 02:22:17,900][105692] Updated weights for policy 0, policy_version 1507510 (0.0009) [2023-12-27 02:22:17,960][105692] Updated weights for policy 0, policy_version 1507520 (0.0008) [2023-12-27 02:22:18,156][105620] Updated weights for policy 1, policy_version 1510034 (0.0005) [2023-12-27 02:22:18,214][105620] Updated weights for policy 1, policy_version 1510044 (0.0005) [2023-12-27 02:22:18,271][105620] Updated weights for policy 1, policy_version 1510054 (0.0005) [2023-12-27 02:22:18,754][105692] Updated weights for policy 0, policy_version 1507530 (0.0009) [2023-12-27 02:22:18,805][105692] Updated weights for policy 0, policy_version 1507540 (0.0009) [2023-12-27 02:22:18,853][105692] Updated weights for policy 0, policy_version 1507550 (0.0008) [2023-12-27 02:22:18,898][105692] Updated weights for policy 0, policy_version 1507560 (0.0009) [2023-12-27 02:22:18,925][105620] Updated weights for policy 1, policy_version 1510064 (0.0008) [2023-12-27 02:22:18,990][105620] Updated weights for policy 1, policy_version 1510074 (0.0007) [2023-12-27 02:22:19,040][105620] Updated weights for policy 1, policy_version 1510084 (0.0008) [2023-12-27 02:22:19,712][105620] Updated weights for policy 1, policy_version 1510094 (0.0008) [2023-12-27 02:22:19,739][105692] Updated weights for policy 0, policy_version 1507570 (0.0006) [2023-12-27 02:22:19,775][105620] Updated weights for policy 1, policy_version 1510104 (0.0009) [2023-12-27 02:22:19,803][105692] Updated weights for policy 0, policy_version 1507580 (0.0008) [2023-12-27 02:22:19,843][105620] Updated weights for policy 1, policy_version 1510114 (0.0007) [2023-12-27 02:22:19,871][105692] Updated weights for policy 0, policy_version 1507590 (0.0007) [2023-12-27 02:22:20,470][105620] Updated weights for policy 1, policy_version 1510124 (0.0008) [2023-12-27 02:22:20,529][105620] Updated weights for policy 1, policy_version 1510134 (0.0009) [2023-12-27 02:22:20,595][105620] Updated weights for policy 1, policy_version 1510144 (0.0009) [2023-12-27 02:22:20,610][105692] Updated weights for policy 0, policy_version 1507600 (0.0007) [2023-12-27 02:22:20,673][105692] Updated weights for policy 0, policy_version 1507610 (0.0009) [2023-12-27 02:22:20,729][105692] Updated weights for policy 0, policy_version 1507620 (0.0008) [2023-12-27 02:22:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.9, 300 sec: 19521.9). Total num frames: 772661248. Throughput: 0: 9921.3, 1: 9553.4. Samples: 772649916. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:22:21,063][104569] Avg episode reward: [(0, '7916.865'), (1, '9259.249')] [2023-12-27 02:22:21,379][105620] Updated weights for policy 1, policy_version 1510154 (0.0007) [2023-12-27 02:22:21,442][105620] Updated weights for policy 1, policy_version 1510164 (0.0008) [2023-12-27 02:22:21,494][105620] Updated weights for policy 1, policy_version 1510174 (0.0008) [2023-12-27 02:22:21,506][105692] Updated weights for policy 0, policy_version 1507630 (0.0008) [2023-12-27 02:22:21,546][105620] Updated weights for policy 1, policy_version 1510184 (0.0008) [2023-12-27 02:22:21,561][105692] Updated weights for policy 0, policy_version 1507640 (0.0007) [2023-12-27 02:22:21,626][105692] Updated weights for policy 0, policy_version 1507650 (0.0009) [2023-12-27 02:22:22,340][105620] Updated weights for policy 1, policy_version 1510194 (0.0009) [2023-12-27 02:22:22,365][105692] Updated weights for policy 0, policy_version 1507660 (0.0009) [2023-12-27 02:22:22,404][105620] Updated weights for policy 1, policy_version 1510204 (0.0007) [2023-12-27 02:22:22,428][105692] Updated weights for policy 0, policy_version 1507670 (0.0008) [2023-12-27 02:22:22,465][105620] Updated weights for policy 1, policy_version 1510214 (0.0009) [2023-12-27 02:22:22,486][105692] Updated weights for policy 0, policy_version 1507680 (0.0007) [2023-12-27 02:22:23,213][105692] Updated weights for policy 0, policy_version 1507690 (0.0009) [2023-12-27 02:22:23,252][105620] Updated weights for policy 1, policy_version 1510224 (0.0005) [2023-12-27 02:22:23,264][105692] Updated weights for policy 0, policy_version 1507700 (0.0009) [2023-12-27 02:22:23,311][105620] Updated weights for policy 1, policy_version 1510234 (0.0005) [2023-12-27 02:22:23,332][105692] Updated weights for policy 0, policy_version 1507710 (0.0007) [2023-12-27 02:22:23,370][105620] Updated weights for policy 1, policy_version 1510244 (0.0008) [2023-12-27 02:22:23,385][105692] Updated weights for policy 0, policy_version 1507720 (0.0006) [2023-12-27 02:22:23,913][105620] Updated weights for policy 1, policy_version 1510254 (0.0007) [2023-12-27 02:22:23,970][105620] Updated weights for policy 1, policy_version 1510264 (0.0006) [2023-12-27 02:22:24,027][105620] Updated weights for policy 1, policy_version 1510274 (0.0009) [2023-12-27 02:22:24,117][105692] Updated weights for policy 0, policy_version 1507730 (0.0006) [2023-12-27 02:22:24,183][105692] Updated weights for policy 0, policy_version 1507740 (0.0010) [2023-12-27 02:22:24,242][105692] Updated weights for policy 0, policy_version 1507750 (0.0006) [2023-12-27 02:22:24,616][105620] Updated weights for policy 1, policy_version 1510284 (0.0008) [2023-12-27 02:22:24,672][105620] Updated weights for policy 1, policy_version 1510294 (0.0007) [2023-12-27 02:22:24,730][105620] Updated weights for policy 1, policy_version 1510304 (0.0010) [2023-12-27 02:22:24,927][105692] Updated weights for policy 0, policy_version 1507760 (0.0010) [2023-12-27 02:22:24,985][105692] Updated weights for policy 0, policy_version 1507770 (0.0010) [2023-12-27 02:22:25,047][105692] Updated weights for policy 0, policy_version 1507780 (0.0010) [2023-12-27 02:22:25,269][105620] Updated weights for policy 1, policy_version 1510314 (0.0005) [2023-12-27 02:22:25,332][105620] Updated weights for policy 1, policy_version 1510324 (0.0006) [2023-12-27 02:22:25,384][105620] Updated weights for policy 1, policy_version 1510334 (0.0010) [2023-12-27 02:22:25,445][105620] Updated weights for policy 1, policy_version 1510344 (0.0010) [2023-12-27 02:22:25,764][105692] Updated weights for policy 0, policy_version 1507790 (0.0010) [2023-12-27 02:22:25,808][105692] Updated weights for policy 0, policy_version 1507800 (0.0010) [2023-12-27 02:22:25,859][105692] Updated weights for policy 0, policy_version 1507810 (0.0010) [2023-12-27 02:22:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 772759552. Throughput: 0: 9890.0, 1: 9661.8. Samples: 772769100. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:22:26,063][104569] Avg episode reward: [(0, '8181.654'), (1, '9083.204')] [2023-12-27 02:22:26,079][105620] Updated weights for policy 1, policy_version 1510354 (0.0010) [2023-12-27 02:22:26,125][105620] Updated weights for policy 1, policy_version 1510364 (0.0010) [2023-12-27 02:22:26,173][105620] Updated weights for policy 1, policy_version 1510374 (0.0010) [2023-12-27 02:22:26,628][105692] Updated weights for policy 0, policy_version 1507820 (0.0008) [2023-12-27 02:22:26,682][105692] Updated weights for policy 0, policy_version 1507830 (0.0005) [2023-12-27 02:22:26,738][105692] Updated weights for policy 0, policy_version 1507840 (0.0005) [2023-12-27 02:22:26,898][105620] Updated weights for policy 1, policy_version 1510384 (0.0006) [2023-12-27 02:22:26,967][105620] Updated weights for policy 1, policy_version 1510394 (0.0005) [2023-12-27 02:22:27,038][105620] Updated weights for policy 1, policy_version 1510404 (0.0005) [2023-12-27 02:22:27,313][105692] Updated weights for policy 0, policy_version 1507850 (0.0005) [2023-12-27 02:22:27,366][105692] Updated weights for policy 0, policy_version 1507860 (0.0005) [2023-12-27 02:22:27,422][105692] Updated weights for policy 0, policy_version 1507870 (0.0005) [2023-12-27 02:22:27,472][105692] Updated weights for policy 0, policy_version 1507880 (0.0005) [2023-12-27 02:22:27,588][105620] Updated weights for policy 1, policy_version 1510414 (0.0005) [2023-12-27 02:22:27,644][105620] Updated weights for policy 1, policy_version 1510424 (0.0005) [2023-12-27 02:22:27,711][105620] Updated weights for policy 1, policy_version 1510434 (0.0006) [2023-12-27 02:22:28,167][105692] Updated weights for policy 0, policy_version 1507890 (0.0008) [2023-12-27 02:22:28,215][105692] Updated weights for policy 0, policy_version 1507900 (0.0008) [2023-12-27 02:22:28,258][105692] Updated weights for policy 0, policy_version 1507910 (0.0007) [2023-12-27 02:22:28,339][105620] Updated weights for policy 1, policy_version 1510444 (0.0007) [2023-12-27 02:22:28,398][105620] Updated weights for policy 1, policy_version 1510454 (0.0010) [2023-12-27 02:22:28,473][105620] Updated weights for policy 1, policy_version 1510464 (0.0011) [2023-12-27 02:22:28,917][105692] Updated weights for policy 0, policy_version 1507920 (0.0009) [2023-12-27 02:22:28,979][105692] Updated weights for policy 0, policy_version 1507930 (0.0009) [2023-12-27 02:22:29,038][105692] Updated weights for policy 0, policy_version 1507940 (0.0009) [2023-12-27 02:22:29,218][105620] Updated weights for policy 1, policy_version 1510474 (0.0010) [2023-12-27 02:22:29,285][105620] Updated weights for policy 1, policy_version 1510484 (0.0009) [2023-12-27 02:22:29,347][105620] Updated weights for policy 1, policy_version 1510494 (0.0009) [2023-12-27 02:22:29,413][105620] Updated weights for policy 1, policy_version 1510504 (0.0009) [2023-12-27 02:22:29,815][105692] Updated weights for policy 0, policy_version 1507950 (0.0009) [2023-12-27 02:22:29,870][105692] Updated weights for policy 0, policy_version 1507960 (0.0008) [2023-12-27 02:22:29,933][105692] Updated weights for policy 0, policy_version 1507970 (0.0007) [2023-12-27 02:22:30,138][105620] Updated weights for policy 1, policy_version 1510514 (0.0009) [2023-12-27 02:22:30,195][105620] Updated weights for policy 1, policy_version 1510524 (0.0009) [2023-12-27 02:22:30,244][105620] Updated weights for policy 1, policy_version 1510534 (0.0008) [2023-12-27 02:22:30,672][105692] Updated weights for policy 0, policy_version 1507980 (0.0008) [2023-12-27 02:22:30,731][105692] Updated weights for policy 0, policy_version 1507990 (0.0009) [2023-12-27 02:22:30,788][105692] Updated weights for policy 0, policy_version 1508000 (0.0008) [2023-12-27 02:22:30,970][105620] Updated weights for policy 1, policy_version 1510545 (0.0010) [2023-12-27 02:22:31,021][105620] Updated weights for policy 1, policy_version 1510555 (0.0008) [2023-12-27 02:22:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 772857856. Throughput: 0: 9922.1, 1: 9745.6. Samples: 772830992. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:22:31,063][104569] Avg episode reward: [(0, '8626.171'), (1, '9083.603')] [2023-12-27 02:22:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001508008_386105344.pth... [2023-12-27 02:22:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001506856_385810432.pth [2023-12-27 02:22:31,085][105620] Updated weights for policy 1, policy_version 1510565 (0.0009) [2023-12-27 02:22:31,101][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001510568_386760704.pth... [2023-12-27 02:22:31,106][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001509384_386457600.pth [2023-12-27 02:22:31,445][105692] Updated weights for policy 0, policy_version 1508010 (0.0009) [2023-12-27 02:22:31,508][105692] Updated weights for policy 0, policy_version 1508020 (0.0009) [2023-12-27 02:22:31,568][105692] Updated weights for policy 0, policy_version 1508030 (0.0009) [2023-12-27 02:22:31,635][105692] Updated weights for policy 0, policy_version 1508040 (0.0007) [2023-12-27 02:22:31,912][105620] Updated weights for policy 1, policy_version 1510575 (0.0006) [2023-12-27 02:22:31,965][105620] Updated weights for policy 1, policy_version 1510585 (0.0005) [2023-12-27 02:22:32,030][105620] Updated weights for policy 1, policy_version 1510595 (0.0009) [2023-12-27 02:22:32,366][105692] Updated weights for policy 0, policy_version 1508050 (0.0008) [2023-12-27 02:22:32,424][105692] Updated weights for policy 0, policy_version 1508060 (0.0008) [2023-12-27 02:22:32,476][105692] Updated weights for policy 0, policy_version 1508070 (0.0005) [2023-12-27 02:22:32,712][105620] Updated weights for policy 1, policy_version 1510605 (0.0007) [2023-12-27 02:22:32,771][105620] Updated weights for policy 1, policy_version 1510615 (0.0006) [2023-12-27 02:22:32,829][105620] Updated weights for policy 1, policy_version 1510625 (0.0011) [2023-12-27 02:22:33,137][105692] Updated weights for policy 0, policy_version 1508080 (0.0008) [2023-12-27 02:22:33,195][105692] Updated weights for policy 0, policy_version 1508090 (0.0008) [2023-12-27 02:22:33,250][105692] Updated weights for policy 0, policy_version 1508100 (0.0008) [2023-12-27 02:22:33,520][105620] Updated weights for policy 1, policy_version 1510635 (0.0011) [2023-12-27 02:22:33,571][105620] Updated weights for policy 1, policy_version 1510645 (0.0010) [2023-12-27 02:22:33,629][105620] Updated weights for policy 1, policy_version 1510655 (0.0010) [2023-12-27 02:22:33,997][105692] Updated weights for policy 0, policy_version 1508110 (0.0006) [2023-12-27 02:22:34,050][105692] Updated weights for policy 0, policy_version 1508120 (0.0005) [2023-12-27 02:22:34,103][105692] Updated weights for policy 0, policy_version 1508130 (0.0008) [2023-12-27 02:22:34,305][105620] Updated weights for policy 1, policy_version 1510665 (0.0010) [2023-12-27 02:22:34,365][105620] Updated weights for policy 1, policy_version 1510675 (0.0010) [2023-12-27 02:22:34,424][105620] Updated weights for policy 1, policy_version 1510685 (0.0011) [2023-12-27 02:22:34,483][105620] Updated weights for policy 1, policy_version 1510695 (0.0010) [2023-12-27 02:22:34,852][105692] Updated weights for policy 0, policy_version 1508140 (0.0009) [2023-12-27 02:22:34,912][105692] Updated weights for policy 0, policy_version 1508150 (0.0008) [2023-12-27 02:22:34,965][105692] Updated weights for policy 0, policy_version 1508160 (0.0010) [2023-12-27 02:22:35,196][105620] Updated weights for policy 1, policy_version 1510705 (0.0006) [2023-12-27 02:22:35,260][105620] Updated weights for policy 1, policy_version 1510715 (0.0006) [2023-12-27 02:22:35,321][105620] Updated weights for policy 1, policy_version 1510725 (0.0008) [2023-12-27 02:22:35,784][105692] Updated weights for policy 0, policy_version 1508170 (0.0009) [2023-12-27 02:22:35,834][105692] Updated weights for policy 0, policy_version 1508180 (0.0009) [2023-12-27 02:22:35,881][105692] Updated weights for policy 0, policy_version 1508190 (0.0008) [2023-12-27 02:22:35,927][105692] Updated weights for policy 0, policy_version 1508200 (0.0008) [2023-12-27 02:22:35,966][105620] Updated weights for policy 1, policy_version 1510735 (0.0009) [2023-12-27 02:22:36,019][105620] Updated weights for policy 1, policy_version 1510745 (0.0009) [2023-12-27 02:22:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 772956160. Throughput: 0: 9805.2, 1: 9847.6. Samples: 772946532. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:22:36,062][104569] Avg episode reward: [(0, '8629.639'), (1, '9262.293')] [2023-12-27 02:22:36,075][105620] Updated weights for policy 1, policy_version 1510755 (0.0005) [2023-12-27 02:22:36,706][105692] Updated weights for policy 0, policy_version 1508210 (0.0010) [2023-12-27 02:22:36,767][105692] Updated weights for policy 0, policy_version 1508220 (0.0009) [2023-12-27 02:22:36,804][105620] Updated weights for policy 1, policy_version 1510765 (0.0008) [2023-12-27 02:22:36,826][105692] Updated weights for policy 0, policy_version 1508230 (0.0008) [2023-12-27 02:22:36,863][105620] Updated weights for policy 1, policy_version 1510775 (0.0006) [2023-12-27 02:22:36,927][105620] Updated weights for policy 1, policy_version 1510785 (0.0006) [2023-12-27 02:22:37,474][105692] Updated weights for policy 0, policy_version 1508240 (0.0006) [2023-12-27 02:22:37,522][105692] Updated weights for policy 0, policy_version 1508250 (0.0005) [2023-12-27 02:22:37,569][105692] Updated weights for policy 0, policy_version 1508260 (0.0005) [2023-12-27 02:22:37,574][105620] Updated weights for policy 1, policy_version 1510795 (0.0007) [2023-12-27 02:22:37,636][105620] Updated weights for policy 1, policy_version 1510805 (0.0009) [2023-12-27 02:22:37,696][105620] Updated weights for policy 1, policy_version 1510815 (0.0009) [2023-12-27 02:22:38,207][105692] Updated weights for policy 0, policy_version 1508270 (0.0008) [2023-12-27 02:22:38,265][105692] Updated weights for policy 0, policy_version 1508280 (0.0010) [2023-12-27 02:22:38,317][105692] Updated weights for policy 0, policy_version 1508290 (0.0010) [2023-12-27 02:22:38,446][105620] Updated weights for policy 1, policy_version 1510825 (0.0007) [2023-12-27 02:22:38,501][105620] Updated weights for policy 1, policy_version 1510835 (0.0009) [2023-12-27 02:22:38,561][105620] Updated weights for policy 1, policy_version 1510845 (0.0009) [2023-12-27 02:22:38,609][105620] Updated weights for policy 1, policy_version 1510855 (0.0008) [2023-12-27 02:22:39,040][105692] Updated weights for policy 0, policy_version 1508300 (0.0010) [2023-12-27 02:22:39,098][105692] Updated weights for policy 0, policy_version 1508310 (0.0010) [2023-12-27 02:22:39,164][105692] Updated weights for policy 0, policy_version 1508320 (0.0010) [2023-12-27 02:22:39,389][105620] Updated weights for policy 1, policy_version 1510865 (0.0010) [2023-12-27 02:22:39,455][105620] Updated weights for policy 1, policy_version 1510875 (0.0007) [2023-12-27 02:22:39,509][105620] Updated weights for policy 1, policy_version 1510885 (0.0006) [2023-12-27 02:22:39,950][105692] Updated weights for policy 0, policy_version 1508330 (0.0010) [2023-12-27 02:22:40,004][105692] Updated weights for policy 0, policy_version 1508340 (0.0009) [2023-12-27 02:22:40,052][105692] Updated weights for policy 0, policy_version 1508350 (0.0007) [2023-12-27 02:22:40,104][105692] Updated weights for policy 0, policy_version 1508360 (0.0008) [2023-12-27 02:22:40,261][105620] Updated weights for policy 1, policy_version 1510895 (0.0011) [2023-12-27 02:22:40,323][105620] Updated weights for policy 1, policy_version 1510905 (0.0010) [2023-12-27 02:22:40,387][105620] Updated weights for policy 1, policy_version 1510915 (0.0010) [2023-12-27 02:22:40,886][105692] Updated weights for policy 0, policy_version 1508370 (0.0011) [2023-12-27 02:22:40,941][105692] Updated weights for policy 0, policy_version 1508380 (0.0010) [2023-12-27 02:22:40,999][105692] Updated weights for policy 0, policy_version 1508390 (0.0010) [2023-12-27 02:22:41,048][105620] Updated weights for policy 1, policy_version 1510925 (0.0010) [2023-12-27 02:22:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 773054464. Throughput: 0: 9784.3, 1: 9930.1. Samples: 773063024. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 02:22:41,063][104569] Avg episode reward: [(0, '8357.690'), (1, '9174.077')] [2023-12-27 02:22:41,103][105620] Updated weights for policy 1, policy_version 1510935 (0.0009) [2023-12-27 02:22:41,168][105620] Updated weights for policy 1, policy_version 1510945 (0.0010) [2023-12-27 02:22:41,796][105692] Updated weights for policy 0, policy_version 1508400 (0.0007) [2023-12-27 02:22:41,841][105620] Updated weights for policy 1, policy_version 1510955 (0.0010) [2023-12-27 02:22:41,856][105692] Updated weights for policy 0, policy_version 1508410 (0.0005) [2023-12-27 02:22:41,899][105620] Updated weights for policy 1, policy_version 1510965 (0.0008) [2023-12-27 02:22:41,914][105692] Updated weights for policy 0, policy_version 1508420 (0.0006) [2023-12-27 02:22:41,958][105620] Updated weights for policy 1, policy_version 1510975 (0.0008) [2023-12-27 02:22:42,589][105692] Updated weights for policy 0, policy_version 1508430 (0.0006) [2023-12-27 02:22:42,649][105692] Updated weights for policy 0, policy_version 1508440 (0.0006) [2023-12-27 02:22:42,704][105692] Updated weights for policy 0, policy_version 1508450 (0.0006) [2023-12-27 02:22:42,753][105620] Updated weights for policy 1, policy_version 1510985 (0.0008) [2023-12-27 02:22:42,808][105620] Updated weights for policy 1, policy_version 1510995 (0.0010) [2023-12-27 02:22:42,874][105620] Updated weights for policy 1, policy_version 1511005 (0.0009) [2023-12-27 02:22:42,932][105620] Updated weights for policy 1, policy_version 1511015 (0.0010) [2023-12-27 02:22:43,415][105692] Updated weights for policy 0, policy_version 1508460 (0.0008) [2023-12-27 02:22:43,469][105692] Updated weights for policy 0, policy_version 1508470 (0.0008) [2023-12-27 02:22:43,524][105692] Updated weights for policy 0, policy_version 1508480 (0.0008) [2023-12-27 02:22:43,618][105620] Updated weights for policy 1, policy_version 1511025 (0.0008) [2023-12-27 02:22:43,687][105620] Updated weights for policy 1, policy_version 1511035 (0.0005) [2023-12-27 02:22:43,738][105620] Updated weights for policy 1, policy_version 1511045 (0.0005) [2023-12-27 02:22:44,252][105692] Updated weights for policy 0, policy_version 1508490 (0.0009) [2023-12-27 02:22:44,312][105692] Updated weights for policy 0, policy_version 1508500 (0.0009) [2023-12-27 02:22:44,363][105692] Updated weights for policy 0, policy_version 1508510 (0.0009) [2023-12-27 02:22:44,426][105692] Updated weights for policy 0, policy_version 1508520 (0.0009) [2023-12-27 02:22:44,432][105620] Updated weights for policy 1, policy_version 1511055 (0.0006) [2023-12-27 02:22:44,483][105620] Updated weights for policy 1, policy_version 1511065 (0.0006) [2023-12-27 02:22:44,538][105620] Updated weights for policy 1, policy_version 1511075 (0.0005) [2023-12-27 02:22:45,186][105620] Updated weights for policy 1, policy_version 1511085 (0.0008) [2023-12-27 02:22:45,245][105620] Updated weights for policy 1, policy_version 1511095 (0.0008) [2023-12-27 02:22:45,247][105692] Updated weights for policy 0, policy_version 1508530 (0.0007) [2023-12-27 02:22:45,303][105620] Updated weights for policy 1, policy_version 1511105 (0.0007) [2023-12-27 02:22:45,305][105692] Updated weights for policy 0, policy_version 1508540 (0.0006) [2023-12-27 02:22:45,362][105692] Updated weights for policy 0, policy_version 1508550 (0.0006) [2023-12-27 02:22:45,979][105692] Updated weights for policy 0, policy_version 1508560 (0.0008) [2023-12-27 02:22:46,044][105692] Updated weights for policy 0, policy_version 1508570 (0.0009) [2023-12-27 02:22:46,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 773144576. Throughput: 0: 9706.7, 1: 10007.0. Samples: 773120712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:22:46,063][104569] Avg episode reward: [(0, '8634.387'), (1, '9263.772')] [2023-12-27 02:22:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001511112_386899968.pth... [2023-12-27 02:22:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001509960_386605056.pth [2023-12-27 02:22:46,102][105692] Updated weights for policy 0, policy_version 1508580 (0.0009) [2023-12-27 02:22:46,114][105620] Updated weights for policy 1, policy_version 1511115 (0.0007) [2023-12-27 02:22:46,119][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001508584_386252800.pth... [2023-12-27 02:22:46,124][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001507464_385966080.pth [2023-12-27 02:22:46,169][105620] Updated weights for policy 1, policy_version 1511125 (0.0007) [2023-12-27 02:22:46,231][105620] Updated weights for policy 1, policy_version 1511135 (0.0009) [2023-12-27 02:22:46,817][105692] Updated weights for policy 0, policy_version 1508590 (0.0009) [2023-12-27 02:22:46,871][105692] Updated weights for policy 0, policy_version 1508600 (0.0009) [2023-12-27 02:22:46,922][105692] Updated weights for policy 0, policy_version 1508610 (0.0009) [2023-12-27 02:22:46,962][105620] Updated weights for policy 1, policy_version 1511145 (0.0008) [2023-12-27 02:22:47,010][105620] Updated weights for policy 1, policy_version 1511155 (0.0005) [2023-12-27 02:22:47,063][105620] Updated weights for policy 1, policy_version 1511165 (0.0005) [2023-12-27 02:22:47,120][105620] Updated weights for policy 1, policy_version 1511175 (0.0006) [2023-12-27 02:22:47,727][105692] Updated weights for policy 0, policy_version 1508620 (0.0009) [2023-12-27 02:22:47,751][105620] Updated weights for policy 1, policy_version 1511185 (0.0006) [2023-12-27 02:22:47,793][105692] Updated weights for policy 0, policy_version 1508630 (0.0008) [2023-12-27 02:22:47,796][105620] Updated weights for policy 1, policy_version 1511195 (0.0006) [2023-12-27 02:22:47,848][105692] Updated weights for policy 0, policy_version 1508640 (0.0007) [2023-12-27 02:22:47,856][105620] Updated weights for policy 1, policy_version 1511205 (0.0008) [2023-12-27 02:22:48,530][105692] Updated weights for policy 0, policy_version 1508650 (0.0007) [2023-12-27 02:22:48,598][105692] Updated weights for policy 0, policy_version 1508660 (0.0006) [2023-12-27 02:22:48,643][105620] Updated weights for policy 1, policy_version 1511216 (0.0006) [2023-12-27 02:22:48,668][105692] Updated weights for policy 0, policy_version 1508670 (0.0009) [2023-12-27 02:22:48,699][105620] Updated weights for policy 1, policy_version 1511226 (0.0006) [2023-12-27 02:22:48,737][105692] Updated weights for policy 0, policy_version 1508680 (0.0009) [2023-12-27 02:22:48,760][105620] Updated weights for policy 1, policy_version 1511236 (0.0005) [2023-12-27 02:22:49,366][105620] Updated weights for policy 1, policy_version 1511246 (0.0007) [2023-12-27 02:22:49,425][105620] Updated weights for policy 1, policy_version 1511256 (0.0008) [2023-12-27 02:22:49,482][105620] Updated weights for policy 1, policy_version 1511266 (0.0008) [2023-12-27 02:22:49,489][105692] Updated weights for policy 0, policy_version 1508690 (0.0006) [2023-12-27 02:22:49,535][105692] Updated weights for policy 0, policy_version 1508700 (0.0008) [2023-12-27 02:22:49,586][105692] Updated weights for policy 0, policy_version 1508710 (0.0008) [2023-12-27 02:22:50,251][105620] Updated weights for policy 1, policy_version 1511276 (0.0008) [2023-12-27 02:22:50,309][105620] Updated weights for policy 1, policy_version 1511286 (0.0009) [2023-12-27 02:22:50,361][105692] Updated weights for policy 0, policy_version 1508720 (0.0009) [2023-12-27 02:22:50,368][105620] Updated weights for policy 1, policy_version 1511296 (0.0006) [2023-12-27 02:22:50,418][105692] Updated weights for policy 0, policy_version 1508730 (0.0010) [2023-12-27 02:22:50,483][105692] Updated weights for policy 0, policy_version 1508740 (0.0009) [2023-12-27 02:22:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 773242880. Throughput: 0: 9663.0, 1: 9934.7. Samples: 773236944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:22:51,063][104569] Avg episode reward: [(0, '8538.688'), (1, '9351.709')] [2023-12-27 02:22:51,144][105692] Updated weights for policy 0, policy_version 1508750 (0.0009) [2023-12-27 02:22:51,180][105620] Updated weights for policy 1, policy_version 1511306 (0.0006) [2023-12-27 02:22:51,201][105692] Updated weights for policy 0, policy_version 1508760 (0.0008) [2023-12-27 02:22:51,243][105620] Updated weights for policy 1, policy_version 1511316 (0.0008) [2023-12-27 02:22:51,255][105692] Updated weights for policy 0, policy_version 1508770 (0.0008) [2023-12-27 02:22:51,308][105620] Updated weights for policy 1, policy_version 1511326 (0.0008) [2023-12-27 02:22:51,382][105620] Updated weights for policy 1, policy_version 1511336 (0.0009) [2023-12-27 02:22:52,035][105692] Updated weights for policy 0, policy_version 1508780 (0.0008) [2023-12-27 02:22:52,092][105692] Updated weights for policy 0, policy_version 1508790 (0.0005) [2023-12-27 02:22:52,148][105692] Updated weights for policy 0, policy_version 1508800 (0.0008) [2023-12-27 02:22:52,163][105620] Updated weights for policy 1, policy_version 1511346 (0.0007) [2023-12-27 02:22:52,219][105620] Updated weights for policy 1, policy_version 1511356 (0.0008) [2023-12-27 02:22:52,277][105620] Updated weights for policy 1, policy_version 1511366 (0.0008) [2023-12-27 02:22:52,827][105692] Updated weights for policy 0, policy_version 1508810 (0.0007) [2023-12-27 02:22:52,878][105692] Updated weights for policy 0, policy_version 1508820 (0.0009) [2023-12-27 02:22:52,933][105692] Updated weights for policy 0, policy_version 1508830 (0.0009) [2023-12-27 02:22:52,985][105692] Updated weights for policy 0, policy_version 1508840 (0.0009) [2023-12-27 02:22:53,085][105620] Updated weights for policy 1, policy_version 1511376 (0.0009) [2023-12-27 02:22:53,131][105620] Updated weights for policy 1, policy_version 1511386 (0.0009) [2023-12-27 02:22:53,181][105620] Updated weights for policy 1, policy_version 1511396 (0.0008) [2023-12-27 02:22:53,751][105692] Updated weights for policy 0, policy_version 1508850 (0.0009) [2023-12-27 02:22:53,798][105692] Updated weights for policy 0, policy_version 1508860 (0.0008) [2023-12-27 02:22:53,845][105692] Updated weights for policy 0, policy_version 1508870 (0.0009) [2023-12-27 02:22:53,945][105620] Updated weights for policy 1, policy_version 1511406 (0.0010) [2023-12-27 02:22:54,001][105620] Updated weights for policy 1, policy_version 1511416 (0.0009) [2023-12-27 02:22:54,049][105620] Updated weights for policy 1, policy_version 1511426 (0.0009) [2023-12-27 02:22:54,637][105692] Updated weights for policy 0, policy_version 1508880 (0.0009) [2023-12-27 02:22:54,697][105692] Updated weights for policy 0, policy_version 1508890 (0.0006) [2023-12-27 02:22:54,752][105692] Updated weights for policy 0, policy_version 1508900 (0.0008) [2023-12-27 02:22:54,823][105620] Updated weights for policy 1, policy_version 1511436 (0.0009) [2023-12-27 02:22:54,874][105620] Updated weights for policy 1, policy_version 1511446 (0.0009) [2023-12-27 02:22:54,921][105620] Updated weights for policy 1, policy_version 1511456 (0.0009) [2023-12-27 02:22:55,438][105692] Updated weights for policy 0, policy_version 1508910 (0.0010) [2023-12-27 02:22:55,502][105692] Updated weights for policy 0, policy_version 1508920 (0.0010) [2023-12-27 02:22:55,564][105692] Updated weights for policy 0, policy_version 1508930 (0.0010) [2023-12-27 02:22:55,691][105620] Updated weights for policy 1, policy_version 1511466 (0.0008) [2023-12-27 02:22:55,742][105620] Updated weights for policy 1, policy_version 1511476 (0.0008) [2023-12-27 02:22:55,790][105620] Updated weights for policy 1, policy_version 1511486 (0.0008) [2023-12-27 02:22:55,838][105620] Updated weights for policy 1, policy_version 1511496 (0.0007) [2023-12-27 02:22:56,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 773341184. Throughput: 0: 9544.0, 1: 9849.2. Samples: 773348232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:22:56,062][104569] Avg episode reward: [(0, '8082.004'), (1, '9351.298')] [2023-12-27 02:22:56,278][105692] Updated weights for policy 0, policy_version 1508940 (0.0010) [2023-12-27 02:22:56,332][105692] Updated weights for policy 0, policy_version 1508950 (0.0010) [2023-12-27 02:22:56,387][105692] Updated weights for policy 0, policy_version 1508960 (0.0007) [2023-12-27 02:22:56,615][105620] Updated weights for policy 1, policy_version 1511506 (0.0008) [2023-12-27 02:22:56,660][105620] Updated weights for policy 1, policy_version 1511516 (0.0008) [2023-12-27 02:22:56,707][105620] Updated weights for policy 1, policy_version 1511526 (0.0008) [2023-12-27 02:22:57,024][105692] Updated weights for policy 0, policy_version 1508970 (0.0009) [2023-12-27 02:22:57,076][105692] Updated weights for policy 0, policy_version 1508980 (0.0005) [2023-12-27 02:22:57,129][105692] Updated weights for policy 0, policy_version 1508990 (0.0008) [2023-12-27 02:22:57,187][105692] Updated weights for policy 0, policy_version 1509000 (0.0008) [2023-12-27 02:22:57,615][105620] Updated weights for policy 1, policy_version 1511536 (0.0009) [2023-12-27 02:22:57,665][105620] Updated weights for policy 1, policy_version 1511546 (0.0010) [2023-12-27 02:22:57,705][105692] Updated weights for policy 0, policy_version 1509010 (0.0006) [2023-12-27 02:22:57,724][105620] Updated weights for policy 1, policy_version 1511556 (0.0008) [2023-12-27 02:22:57,759][105692] Updated weights for policy 0, policy_version 1509020 (0.0006) [2023-12-27 02:22:57,815][105692] Updated weights for policy 0, policy_version 1509030 (0.0005) [2023-12-27 02:22:58,429][105692] Updated weights for policy 0, policy_version 1509040 (0.0009) [2023-12-27 02:22:58,495][105692] Updated weights for policy 0, policy_version 1509050 (0.0011) [2023-12-27 02:22:58,498][105620] Updated weights for policy 1, policy_version 1511566 (0.0008) [2023-12-27 02:22:58,555][105692] Updated weights for policy 0, policy_version 1509060 (0.0010) [2023-12-27 02:22:58,558][105620] Updated weights for policy 1, policy_version 1511576 (0.0011) [2023-12-27 02:22:58,636][105620] Updated weights for policy 1, policy_version 1511586 (0.0010) [2023-12-27 02:22:59,385][105620] Updated weights for policy 1, policy_version 1511596 (0.0011) [2023-12-27 02:22:59,430][105692] Updated weights for policy 0, policy_version 1509070 (0.0010) [2023-12-27 02:22:59,438][105620] Updated weights for policy 1, policy_version 1511606 (0.0011) [2023-12-27 02:22:59,488][105692] Updated weights for policy 0, policy_version 1509080 (0.0011) [2023-12-27 02:22:59,490][105620] Updated weights for policy 1, policy_version 1511616 (0.0011) [2023-12-27 02:22:59,548][105692] Updated weights for policy 0, policy_version 1509090 (0.0011) [2023-12-27 02:23:00,241][105692] Updated weights for policy 0, policy_version 1509100 (0.0009) [2023-12-27 02:23:00,257][105620] Updated weights for policy 1, policy_version 1511626 (0.0010) [2023-12-27 02:23:00,297][105692] Updated weights for policy 0, policy_version 1509110 (0.0007) [2023-12-27 02:23:00,325][105620] Updated weights for policy 1, policy_version 1511636 (0.0011) [2023-12-27 02:23:00,351][105692] Updated weights for policy 0, policy_version 1509120 (0.0006) [2023-12-27 02:23:00,392][105620] Updated weights for policy 1, policy_version 1511646 (0.0009) [2023-12-27 02:23:00,461][105620] Updated weights for policy 1, policy_version 1511656 (0.0011) [2023-12-27 02:23:00,926][105692] Updated weights for policy 0, policy_version 1509130 (0.0006) [2023-12-27 02:23:00,979][105692] Updated weights for policy 0, policy_version 1509140 (0.0006) [2023-12-27 02:23:01,038][105692] Updated weights for policy 0, policy_version 1509150 (0.0006) [2023-12-27 02:23:01,040][105620] Updated weights for policy 1, policy_version 1511666 (0.0008) [2023-12-27 02:23:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 773431296. Throughput: 0: 9612.5, 1: 9818.5. Samples: 773407448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:23:01,062][104569] Avg episode reward: [(0, '8352.166'), (1, '9260.900')] [2023-12-27 02:23:01,099][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001509160_386400256.pth... [2023-12-27 02:23:01,099][105692] Updated weights for policy 0, policy_version 1509160 (0.0006) [2023-12-27 02:23:01,102][105620] Updated weights for policy 1, policy_version 1511676 (0.0010) [2023-12-27 02:23:01,103][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001508008_386105344.pth [2023-12-27 02:23:01,172][105620] Updated weights for policy 1, policy_version 1511686 (0.0010) [2023-12-27 02:23:01,182][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001511688_387047424.pth... [2023-12-27 02:23:01,186][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001510568_386760704.pth [2023-12-27 02:23:01,764][105692] Updated weights for policy 0, policy_version 1509170 (0.0009) [2023-12-27 02:23:01,826][105692] Updated weights for policy 0, policy_version 1509180 (0.0007) [2023-12-27 02:23:01,886][105692] Updated weights for policy 0, policy_version 1509190 (0.0006) [2023-12-27 02:23:01,914][105620] Updated weights for policy 1, policy_version 1511696 (0.0007) [2023-12-27 02:23:01,976][105620] Updated weights for policy 1, policy_version 1511706 (0.0005) [2023-12-27 02:23:02,032][105620] Updated weights for policy 1, policy_version 1511716 (0.0006) [2023-12-27 02:23:02,618][105692] Updated weights for policy 0, policy_version 1509200 (0.0007) [2023-12-27 02:23:02,665][105692] Updated weights for policy 0, policy_version 1509210 (0.0008) [2023-12-27 02:23:02,674][105620] Updated weights for policy 1, policy_version 1511726 (0.0007) [2023-12-27 02:23:02,714][105692] Updated weights for policy 0, policy_version 1509220 (0.0006) [2023-12-27 02:23:02,730][105620] Updated weights for policy 1, policy_version 1511736 (0.0009) [2023-12-27 02:23:02,789][105620] Updated weights for policy 1, policy_version 1511746 (0.0009) [2023-12-27 02:23:03,428][105620] Updated weights for policy 1, policy_version 1511756 (0.0008) [2023-12-27 02:23:03,479][105620] Updated weights for policy 1, policy_version 1511766 (0.0009) [2023-12-27 02:23:03,520][105692] Updated weights for policy 0, policy_version 1509230 (0.0007) [2023-12-27 02:23:03,535][105620] Updated weights for policy 1, policy_version 1511776 (0.0008) [2023-12-27 02:23:03,581][105692] Updated weights for policy 0, policy_version 1509240 (0.0007) [2023-12-27 02:23:03,648][105692] Updated weights for policy 0, policy_version 1509250 (0.0005) [2023-12-27 02:23:04,313][105692] Updated weights for policy 0, policy_version 1509260 (0.0009) [2023-12-27 02:23:04,344][105620] Updated weights for policy 1, policy_version 1511786 (0.0007) [2023-12-27 02:23:04,379][105692] Updated weights for policy 0, policy_version 1509270 (0.0011) [2023-12-27 02:23:04,405][105620] Updated weights for policy 1, policy_version 1511796 (0.0006) [2023-12-27 02:23:04,440][105692] Updated weights for policy 0, policy_version 1509280 (0.0011) [2023-12-27 02:23:04,463][105620] Updated weights for policy 1, policy_version 1511806 (0.0006) [2023-12-27 02:23:04,524][105620] Updated weights for policy 1, policy_version 1511816 (0.0007) [2023-12-27 02:23:05,159][105692] Updated weights for policy 0, policy_version 1509290 (0.0010) [2023-12-27 02:23:05,212][105692] Updated weights for policy 0, policy_version 1509300 (0.0008) [2023-12-27 02:23:05,239][105620] Updated weights for policy 1, policy_version 1511826 (0.0008) [2023-12-27 02:23:05,270][105692] Updated weights for policy 0, policy_version 1509310 (0.0007) [2023-12-27 02:23:05,289][105620] Updated weights for policy 1, policy_version 1511836 (0.0006) [2023-12-27 02:23:05,327][105692] Updated weights for policy 0, policy_version 1509320 (0.0008) [2023-12-27 02:23:05,342][105620] Updated weights for policy 1, policy_version 1511846 (0.0006) [2023-12-27 02:23:06,026][105620] Updated weights for policy 1, policy_version 1511856 (0.0009) [2023-12-27 02:23:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 773529600. Throughput: 0: 9724.6, 1: 9705.0. Samples: 773524244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:23:06,062][104569] Avg episode reward: [(0, '8712.798'), (1, '9260.457')] [2023-12-27 02:23:06,081][105620] Updated weights for policy 1, policy_version 1511866 (0.0009) [2023-12-27 02:23:06,147][105692] Updated weights for policy 0, policy_version 1509330 (0.0008) [2023-12-27 02:23:06,149][105620] Updated weights for policy 1, policy_version 1511876 (0.0007) [2023-12-27 02:23:06,205][105692] Updated weights for policy 0, policy_version 1509340 (0.0009) [2023-12-27 02:23:06,260][105692] Updated weights for policy 0, policy_version 1509351 (0.0009) [2023-12-27 02:23:06,774][105620] Updated weights for policy 1, policy_version 1511886 (0.0008) [2023-12-27 02:23:06,830][105620] Updated weights for policy 1, policy_version 1511896 (0.0010) [2023-12-27 02:23:06,881][105620] Updated weights for policy 1, policy_version 1511906 (0.0009) [2023-12-27 02:23:07,024][105692] Updated weights for policy 0, policy_version 1509361 (0.0007) [2023-12-27 02:23:07,088][105692] Updated weights for policy 0, policy_version 1509371 (0.0008) [2023-12-27 02:23:07,152][105692] Updated weights for policy 0, policy_version 1509381 (0.0008) [2023-12-27 02:23:07,565][105620] Updated weights for policy 1, policy_version 1511916 (0.0010) [2023-12-27 02:23:07,617][105620] Updated weights for policy 1, policy_version 1511926 (0.0009) [2023-12-27 02:23:07,674][105620] Updated weights for policy 1, policy_version 1511936 (0.0008) [2023-12-27 02:23:07,826][105692] Updated weights for policy 0, policy_version 1509391 (0.0010) [2023-12-27 02:23:07,874][105692] Updated weights for policy 0, policy_version 1509401 (0.0010) [2023-12-27 02:23:07,922][105692] Updated weights for policy 0, policy_version 1509411 (0.0010) [2023-12-27 02:23:08,389][105620] Updated weights for policy 1, policy_version 1511946 (0.0009) [2023-12-27 02:23:08,442][105620] Updated weights for policy 1, policy_version 1511956 (0.0011) [2023-12-27 02:23:08,501][105620] Updated weights for policy 1, policy_version 1511966 (0.0011) [2023-12-27 02:23:08,557][105620] Updated weights for policy 1, policy_version 1511976 (0.0010) [2023-12-27 02:23:08,635][105692] Updated weights for policy 0, policy_version 1509421 (0.0009) [2023-12-27 02:23:08,689][105692] Updated weights for policy 0, policy_version 1509431 (0.0008) [2023-12-27 02:23:08,747][105692] Updated weights for policy 0, policy_version 1509441 (0.0008) [2023-12-27 02:23:09,349][105620] Updated weights for policy 1, policy_version 1511986 (0.0007) [2023-12-27 02:23:09,422][105620] Updated weights for policy 1, policy_version 1511996 (0.0008) [2023-12-27 02:23:09,478][105692] Updated weights for policy 0, policy_version 1509451 (0.0009) [2023-12-27 02:23:09,480][105620] Updated weights for policy 1, policy_version 1512006 (0.0006) [2023-12-27 02:23:09,541][105692] Updated weights for policy 0, policy_version 1509461 (0.0011) [2023-12-27 02:23:09,606][105692] Updated weights for policy 0, policy_version 1509471 (0.0011) [2023-12-27 02:23:10,211][105620] Updated weights for policy 1, policy_version 1512016 (0.0007) [2023-12-27 02:23:10,274][105620] Updated weights for policy 1, policy_version 1512026 (0.0009) [2023-12-27 02:23:10,336][105620] Updated weights for policy 1, policy_version 1512036 (0.0009) [2023-12-27 02:23:10,385][105692] Updated weights for policy 0, policy_version 1509481 (0.0011) [2023-12-27 02:23:10,450][105692] Updated weights for policy 0, policy_version 1509491 (0.0011) [2023-12-27 02:23:10,518][105692] Updated weights for policy 0, policy_version 1509501 (0.0011) [2023-12-27 02:23:10,586][105692] Updated weights for policy 0, policy_version 1509511 (0.0010) [2023-12-27 02:23:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19438.7). Total num frames: 773627904. Throughput: 0: 9719.5, 1: 9642.2. Samples: 773640372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:23:11,062][104569] Avg episode reward: [(0, '8532.168'), (1, '9260.336')] [2023-12-27 02:23:11,094][105620] Updated weights for policy 1, policy_version 1512046 (0.0008) [2023-12-27 02:23:11,164][105620] Updated weights for policy 1, policy_version 1512056 (0.0009) [2023-12-27 02:23:11,233][105620] Updated weights for policy 1, policy_version 1512066 (0.0007) [2023-12-27 02:23:11,347][105692] Updated weights for policy 0, policy_version 1509521 (0.0011) [2023-12-27 02:23:11,417][105692] Updated weights for policy 0, policy_version 1509531 (0.0010) [2023-12-27 02:23:11,486][105692] Updated weights for policy 0, policy_version 1509541 (0.0011) [2023-12-27 02:23:12,005][105620] Updated weights for policy 1, policy_version 1512076 (0.0008) [2023-12-27 02:23:12,074][105620] Updated weights for policy 1, policy_version 1512086 (0.0008) [2023-12-27 02:23:12,148][105620] Updated weights for policy 1, policy_version 1512096 (0.0009) [2023-12-27 02:23:12,430][105692] Updated weights for policy 0, policy_version 1509551 (0.0009) [2023-12-27 02:23:12,498][105692] Updated weights for policy 0, policy_version 1509561 (0.0007) [2023-12-27 02:23:12,569][105692] Updated weights for policy 0, policy_version 1509571 (0.0008) [2023-12-27 02:23:13,044][105620] Updated weights for policy 1, policy_version 1512106 (0.0010) [2023-12-27 02:23:13,116][105620] Updated weights for policy 1, policy_version 1512116 (0.0011) [2023-12-27 02:23:13,185][105620] Updated weights for policy 1, policy_version 1512126 (0.0010) [2023-12-27 02:23:13,275][105620] Updated weights for policy 1, policy_version 1512136 (0.0011) [2023-12-27 02:23:13,356][105692] Updated weights for policy 0, policy_version 1509581 (0.0009) [2023-12-27 02:23:13,414][105692] Updated weights for policy 0, policy_version 1509591 (0.0008) [2023-12-27 02:23:13,476][105692] Updated weights for policy 0, policy_version 1509601 (0.0009) [2023-12-27 02:23:14,084][105620] Updated weights for policy 1, policy_version 1512146 (0.0009) [2023-12-27 02:23:14,156][105620] Updated weights for policy 1, policy_version 1512156 (0.0009) [2023-12-27 02:23:14,219][105620] Updated weights for policy 1, policy_version 1512166 (0.0008) [2023-12-27 02:23:14,325][105692] Updated weights for policy 0, policy_version 1509611 (0.0009) [2023-12-27 02:23:14,387][105692] Updated weights for policy 0, policy_version 1509621 (0.0009) [2023-12-27 02:23:14,443][105692] Updated weights for policy 0, policy_version 1509631 (0.0009) [2023-12-27 02:23:14,915][105620] Updated weights for policy 1, policy_version 1512176 (0.0009) [2023-12-27 02:23:14,973][105620] Updated weights for policy 1, policy_version 1512186 (0.0009) [2023-12-27 02:23:15,041][105620] Updated weights for policy 1, policy_version 1512196 (0.0009) [2023-12-27 02:23:15,190][105692] Updated weights for policy 0, policy_version 1509641 (0.0009) [2023-12-27 02:23:15,257][105692] Updated weights for policy 0, policy_version 1509651 (0.0010) [2023-12-27 02:23:15,317][105692] Updated weights for policy 0, policy_version 1509661 (0.0006) [2023-12-27 02:23:15,384][105692] Updated weights for policy 0, policy_version 1509671 (0.0008) [2023-12-27 02:23:15,878][105620] Updated weights for policy 1, policy_version 1512206 (0.0010) [2023-12-27 02:23:15,945][105620] Updated weights for policy 1, policy_version 1512216 (0.0010) [2023-12-27 02:23:16,003][105620] Updated weights for policy 1, policy_version 1512226 (0.0008) [2023-12-27 02:23:16,026][105692] Updated weights for policy 0, policy_version 1509681 (0.0007) [2023-12-27 02:23:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 773718016. Throughput: 0: 9584.5, 1: 9504.6. Samples: 773690004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:23:16,062][104569] Avg episode reward: [(0, '8353.235'), (1, '9260.547')] [2023-12-27 02:23:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001512232_387186688.pth... [2023-12-27 02:23:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001511112_386899968.pth [2023-12-27 02:23:16,080][105692] Updated weights for policy 0, policy_version 1509691 (0.0008) [2023-12-27 02:23:16,134][105692] Updated weights for policy 0, policy_version 1509701 (0.0010) [2023-12-27 02:23:16,152][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001509704_386539520.pth... [2023-12-27 02:23:16,156][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001508584_386252800.pth [2023-12-27 02:23:16,665][105620] Updated weights for policy 1, policy_version 1512236 (0.0008) [2023-12-27 02:23:16,720][105620] Updated weights for policy 1, policy_version 1512246 (0.0009) [2023-12-27 02:23:16,768][105620] Updated weights for policy 1, policy_version 1512256 (0.0009) [2023-12-27 02:23:16,943][105692] Updated weights for policy 0, policy_version 1509711 (0.0007) [2023-12-27 02:23:17,008][105692] Updated weights for policy 0, policy_version 1509721 (0.0006) [2023-12-27 02:23:17,059][105692] Updated weights for policy 0, policy_version 1509731 (0.0008) [2023-12-27 02:23:17,557][105620] Updated weights for policy 1, policy_version 1512266 (0.0008) [2023-12-27 02:23:17,626][105620] Updated weights for policy 1, policy_version 1512276 (0.0006) [2023-12-27 02:23:17,696][105620] Updated weights for policy 1, policy_version 1512286 (0.0005) [2023-12-27 02:23:17,767][105620] Updated weights for policy 1, policy_version 1512296 (0.0006) [2023-12-27 02:23:17,828][105692] Updated weights for policy 0, policy_version 1509741 (0.0010) [2023-12-27 02:23:17,892][105692] Updated weights for policy 0, policy_version 1509751 (0.0008) [2023-12-27 02:23:17,956][105692] Updated weights for policy 0, policy_version 1509761 (0.0008) [2023-12-27 02:23:18,487][105620] Updated weights for policy 1, policy_version 1512306 (0.0008) [2023-12-27 02:23:18,543][105620] Updated weights for policy 1, policy_version 1512316 (0.0008) [2023-12-27 02:23:18,599][105620] Updated weights for policy 1, policy_version 1512326 (0.0007) [2023-12-27 02:23:18,779][105692] Updated weights for policy 0, policy_version 1509771 (0.0009) [2023-12-27 02:23:18,845][105692] Updated weights for policy 0, policy_version 1509781 (0.0009) [2023-12-27 02:23:18,911][105692] Updated weights for policy 0, policy_version 1509791 (0.0010) [2023-12-27 02:23:19,422][105620] Updated weights for policy 1, policy_version 1512336 (0.0008) [2023-12-27 02:23:19,482][105620] Updated weights for policy 1, policy_version 1512346 (0.0009) [2023-12-27 02:23:19,555][105620] Updated weights for policy 1, policy_version 1512356 (0.0009) [2023-12-27 02:23:19,749][105692] Updated weights for policy 0, policy_version 1509801 (0.0009) [2023-12-27 02:23:19,813][105692] Updated weights for policy 0, policy_version 1509811 (0.0011) [2023-12-27 02:23:19,881][105692] Updated weights for policy 0, policy_version 1509821 (0.0009) [2023-12-27 02:23:19,951][105692] Updated weights for policy 0, policy_version 1509831 (0.0011) [2023-12-27 02:23:20,389][105620] Updated weights for policy 1, policy_version 1512366 (0.0009) [2023-12-27 02:23:20,455][105620] Updated weights for policy 1, policy_version 1512376 (0.0008) [2023-12-27 02:23:20,521][105620] Updated weights for policy 1, policy_version 1512386 (0.0009) [2023-12-27 02:23:20,843][105692] Updated weights for policy 0, policy_version 1509841 (0.0010) [2023-12-27 02:23:20,916][105692] Updated weights for policy 0, policy_version 1509851 (0.0009) [2023-12-27 02:23:20,991][105692] Updated weights for policy 0, policy_version 1509861 (0.0009) [2023-12-27 02:23:21,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19114.7, 300 sec: 19410.9). Total num frames: 773808128. Throughput: 0: 9494.8, 1: 9443.5. Samples: 773798756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:23:21,062][104569] Avg episode reward: [(0, '8087.245'), (1, '9168.760')] [2023-12-27 02:23:21,403][105620] Updated weights for policy 1, policy_version 1512396 (0.0008) [2023-12-27 02:23:21,461][105620] Updated weights for policy 1, policy_version 1512406 (0.0008) [2023-12-27 02:23:21,526][105620] Updated weights for policy 1, policy_version 1512416 (0.0009) [2023-12-27 02:23:21,865][105692] Updated weights for policy 0, policy_version 1509871 (0.0010) [2023-12-27 02:23:21,925][105692] Updated weights for policy 0, policy_version 1509881 (0.0010) [2023-12-27 02:23:21,993][105692] Updated weights for policy 0, policy_version 1509891 (0.0008) [2023-12-27 02:23:22,293][105620] Updated weights for policy 1, policy_version 1512426 (0.0010) [2023-12-27 02:23:22,378][105620] Updated weights for policy 1, policy_version 1512436 (0.0010) [2023-12-27 02:23:22,447][105620] Updated weights for policy 1, policy_version 1512446 (0.0008) [2023-12-27 02:23:22,500][105620] Updated weights for policy 1, policy_version 1512456 (0.0009) [2023-12-27 02:23:22,801][105692] Updated weights for policy 0, policy_version 1509901 (0.0010) [2023-12-27 02:23:22,867][105692] Updated weights for policy 0, policy_version 1509911 (0.0009) [2023-12-27 02:23:22,934][105692] Updated weights for policy 0, policy_version 1509921 (0.0009) [2023-12-27 02:23:23,356][105620] Updated weights for policy 1, policy_version 1512466 (0.0010) [2023-12-27 02:23:23,417][105620] Updated weights for policy 1, policy_version 1512476 (0.0010) [2023-12-27 02:23:23,480][105620] Updated weights for policy 1, policy_version 1512486 (0.0009) [2023-12-27 02:23:23,618][105692] Updated weights for policy 0, policy_version 1509931 (0.0008) [2023-12-27 02:23:23,676][105692] Updated weights for policy 0, policy_version 1509941 (0.0010) [2023-12-27 02:23:23,736][105692] Updated weights for policy 0, policy_version 1509951 (0.0007) [2023-12-27 02:23:24,282][105620] Updated weights for policy 1, policy_version 1512496 (0.0008) [2023-12-27 02:23:24,354][105620] Updated weights for policy 1, policy_version 1512506 (0.0008) [2023-12-27 02:23:24,426][105620] Updated weights for policy 1, policy_version 1512516 (0.0009) [2023-12-27 02:23:24,569][105692] Updated weights for policy 0, policy_version 1509961 (0.0007) [2023-12-27 02:23:24,639][105692] Updated weights for policy 0, policy_version 1509971 (0.0009) [2023-12-27 02:23:24,701][105692] Updated weights for policy 0, policy_version 1509981 (0.0010) [2023-12-27 02:23:24,765][105692] Updated weights for policy 0, policy_version 1509991 (0.0009) [2023-12-27 02:23:25,212][105620] Updated weights for policy 1, policy_version 1512526 (0.0008) [2023-12-27 02:23:25,279][105620] Updated weights for policy 1, policy_version 1512536 (0.0008) [2023-12-27 02:23:25,346][105620] Updated weights for policy 1, policy_version 1512546 (0.0006) [2023-12-27 02:23:25,593][105692] Updated weights for policy 0, policy_version 1510001 (0.0009) [2023-12-27 02:23:25,664][105692] Updated weights for policy 0, policy_version 1510011 (0.0009) [2023-12-27 02:23:25,723][105692] Updated weights for policy 0, policy_version 1510021 (0.0009) [2023-12-27 02:23:26,062][104569] Fps is (10 sec: 17203.0, 60 sec: 18841.6, 300 sec: 19327.6). Total num frames: 773890048. Throughput: 0: 9355.6, 1: 9264.2. Samples: 773900916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:23:26,063][104569] Avg episode reward: [(0, '8263.036'), (1, '9079.530')] [2023-12-27 02:23:26,107][105620] Updated weights for policy 1, policy_version 1512556 (0.0009) [2023-12-27 02:23:26,171][105620] Updated weights for policy 1, policy_version 1512566 (0.0009) [2023-12-27 02:23:26,221][105620] Updated weights for policy 1, policy_version 1512576 (0.0009) [2023-12-27 02:23:26,506][105692] Updated weights for policy 0, policy_version 1510031 (0.0010) [2023-12-27 02:23:26,571][105692] Updated weights for policy 0, policy_version 1510041 (0.0009) [2023-12-27 02:23:26,628][105692] Updated weights for policy 0, policy_version 1510051 (0.0005) [2023-12-27 02:23:27,092][105620] Updated weights for policy 1, policy_version 1512586 (0.0009) [2023-12-27 02:23:27,155][105620] Updated weights for policy 1, policy_version 1512596 (0.0009) [2023-12-27 02:23:27,204][105620] Updated weights for policy 1, policy_version 1512606 (0.0009) [2023-12-27 02:23:27,259][105620] Updated weights for policy 1, policy_version 1512616 (0.0009) [2023-12-27 02:23:27,284][105692] Updated weights for policy 0, policy_version 1510061 (0.0007) [2023-12-27 02:23:27,351][105692] Updated weights for policy 0, policy_version 1510071 (0.0008) [2023-12-27 02:23:27,410][105692] Updated weights for policy 0, policy_version 1510081 (0.0010) [2023-12-27 02:23:27,990][105620] Updated weights for policy 1, policy_version 1512626 (0.0009) [2023-12-27 02:23:28,058][105620] Updated weights for policy 1, policy_version 1512636 (0.0008) [2023-12-27 02:23:28,126][105620] Updated weights for policy 1, policy_version 1512646 (0.0010) [2023-12-27 02:23:28,191][105692] Updated weights for policy 0, policy_version 1510091 (0.0008) [2023-12-27 02:23:28,264][105692] Updated weights for policy 0, policy_version 1510101 (0.0009) [2023-12-27 02:23:28,326][105692] Updated weights for policy 0, policy_version 1510111 (0.0009) [2023-12-27 02:23:29,009][105620] Updated weights for policy 1, policy_version 1512656 (0.0011) [2023-12-27 02:23:29,075][105620] Updated weights for policy 1, policy_version 1512666 (0.0011) [2023-12-27 02:23:29,140][105620] Updated weights for policy 1, policy_version 1512676 (0.0009) [2023-12-27 02:23:29,213][105692] Updated weights for policy 0, policy_version 1510121 (0.0010) [2023-12-27 02:23:29,285][105692] Updated weights for policy 0, policy_version 1510131 (0.0009) [2023-12-27 02:23:29,356][105692] Updated weights for policy 0, policy_version 1510141 (0.0008) [2023-12-27 02:23:29,420][105692] Updated weights for policy 0, policy_version 1510151 (0.0008) [2023-12-27 02:23:29,884][105620] Updated weights for policy 1, policy_version 1512686 (0.0008) [2023-12-27 02:23:29,952][105620] Updated weights for policy 1, policy_version 1512696 (0.0010) [2023-12-27 02:23:30,018][105620] Updated weights for policy 1, policy_version 1512706 (0.0008) [2023-12-27 02:23:30,273][105692] Updated weights for policy 0, policy_version 1510161 (0.0009) [2023-12-27 02:23:30,343][105692] Updated weights for policy 0, policy_version 1510171 (0.0009) [2023-12-27 02:23:30,413][105692] Updated weights for policy 0, policy_version 1510181 (0.0009) [2023-12-27 02:23:30,773][105620] Updated weights for policy 1, policy_version 1512716 (0.0011) [2023-12-27 02:23:30,842][105620] Updated weights for policy 1, policy_version 1512726 (0.0011) [2023-12-27 02:23:30,910][105620] Updated weights for policy 1, policy_version 1512736 (0.0011) [2023-12-27 02:23:31,062][104569] Fps is (10 sec: 17203.2, 60 sec: 18705.0, 300 sec: 19299.8). Total num frames: 773980160. Throughput: 0: 9325.6, 1: 9184.0. Samples: 773953640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:23:31,063][104569] Avg episode reward: [(0, '8530.431'), (1, '9077.154')] [2023-12-27 02:23:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001510184_386662400.pth... [2023-12-27 02:23:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001512744_387317760.pth... [2023-12-27 02:23:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001509160_386400256.pth [2023-12-27 02:23:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001511688_387047424.pth [2023-12-27 02:23:31,227][105692] Updated weights for policy 0, policy_version 1510191 (0.0008) [2023-12-27 02:23:31,298][105692] Updated weights for policy 0, policy_version 1510201 (0.0009) [2023-12-27 02:23:31,368][105692] Updated weights for policy 0, policy_version 1510211 (0.0008) [2023-12-27 02:23:31,831][105620] Updated weights for policy 1, policy_version 1512746 (0.0011) [2023-12-27 02:23:31,889][105620] Updated weights for policy 1, policy_version 1512756 (0.0010) [2023-12-27 02:23:31,950][105620] Updated weights for policy 1, policy_version 1512766 (0.0011) [2023-12-27 02:23:32,015][105620] Updated weights for policy 1, policy_version 1512776 (0.0011) [2023-12-27 02:23:32,187][105692] Updated weights for policy 0, policy_version 1510221 (0.0007) [2023-12-27 02:23:32,259][105692] Updated weights for policy 0, policy_version 1510231 (0.0006) [2023-12-27 02:23:32,320][105692] Updated weights for policy 0, policy_version 1510241 (0.0008) [2023-12-27 02:23:32,761][105620] Updated weights for policy 1, policy_version 1512786 (0.0008) [2023-12-27 02:23:32,822][105620] Updated weights for policy 1, policy_version 1512796 (0.0008) [2023-12-27 02:23:32,891][105620] Updated weights for policy 1, policy_version 1512806 (0.0007) [2023-12-27 02:23:33,145][105692] Updated weights for policy 0, policy_version 1510251 (0.0009) [2023-12-27 02:23:33,200][105692] Updated weights for policy 0, policy_version 1510262 (0.0010) [2023-12-27 02:23:33,259][105692] Updated weights for policy 0, policy_version 1510273 (0.0009) [2023-12-27 02:23:33,492][105620] Updated weights for policy 1, policy_version 1512816 (0.0006) [2023-12-27 02:23:33,557][105620] Updated weights for policy 1, policy_version 1512826 (0.0006) [2023-12-27 02:23:33,618][105620] Updated weights for policy 1, policy_version 1512836 (0.0009) [2023-12-27 02:23:34,046][105692] Updated weights for policy 0, policy_version 1510283 (0.0010) [2023-12-27 02:23:34,102][105692] Updated weights for policy 0, policy_version 1510293 (0.0009) [2023-12-27 02:23:34,161][105692] Updated weights for policy 0, policy_version 1510303 (0.0008) [2023-12-27 02:23:34,370][105620] Updated weights for policy 1, policy_version 1512846 (0.0009) [2023-12-27 02:23:34,440][105620] Updated weights for policy 1, policy_version 1512856 (0.0008) [2023-12-27 02:23:34,507][105620] Updated weights for policy 1, policy_version 1512866 (0.0008) [2023-12-27 02:23:35,014][105692] Updated weights for policy 0, policy_version 1510313 (0.0009) [2023-12-27 02:23:35,088][105692] Updated weights for policy 0, policy_version 1510323 (0.0009) [2023-12-27 02:23:35,160][105692] Updated weights for policy 0, policy_version 1510333 (0.0009) [2023-12-27 02:23:35,234][105692] Updated weights for policy 0, policy_version 1510343 (0.0010) [2023-12-27 02:23:35,373][105620] Updated weights for policy 1, policy_version 1512876 (0.0008) [2023-12-27 02:23:35,429][105620] Updated weights for policy 1, policy_version 1512886 (0.0009) [2023-12-27 02:23:35,496][105620] Updated weights for policy 1, policy_version 1512896 (0.0009) [2023-12-27 02:23:35,996][105692] Updated weights for policy 0, policy_version 1510353 (0.0006) [2023-12-27 02:23:36,062][104569] Fps is (10 sec: 17203.2, 60 sec: 18432.0, 300 sec: 19244.3). Total num frames: 774062080. Throughput: 0: 9176.1, 1: 9071.4. Samples: 774058080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:23:36,062][105692] Updated weights for policy 0, policy_version 1510363 (0.0008) [2023-12-27 02:23:36,062][104569] Avg episode reward: [(0, '8715.617'), (1, '9258.745')] [2023-12-27 02:23:36,129][105692] Updated weights for policy 0, policy_version 1510373 (0.0009) [2023-12-27 02:23:36,290][105620] Updated weights for policy 1, policy_version 1512906 (0.0010) [2023-12-27 02:23:36,353][105620] Updated weights for policy 1, policy_version 1512916 (0.0007) [2023-12-27 02:23:36,405][105620] Updated weights for policy 1, policy_version 1512926 (0.0009) [2023-12-27 02:23:36,477][105620] Updated weights for policy 1, policy_version 1512936 (0.0008) [2023-12-27 02:23:36,791][105692] Updated weights for policy 0, policy_version 1510383 (0.0010) [2023-12-27 02:23:36,845][105692] Updated weights for policy 0, policy_version 1510393 (0.0010) [2023-12-27 02:23:36,897][105692] Updated weights for policy 0, policy_version 1510403 (0.0009) [2023-12-27 02:23:37,317][105620] Updated weights for policy 1, policy_version 1512946 (0.0010) [2023-12-27 02:23:37,382][105620] Updated weights for policy 1, policy_version 1512956 (0.0008) [2023-12-27 02:23:37,448][105620] Updated weights for policy 1, policy_version 1512966 (0.0009) [2023-12-27 02:23:37,657][105692] Updated weights for policy 0, policy_version 1510413 (0.0009) [2023-12-27 02:23:37,722][105692] Updated weights for policy 0, policy_version 1510423 (0.0009) [2023-12-27 02:23:37,787][105692] Updated weights for policy 0, policy_version 1510433 (0.0006) [2023-12-27 02:23:38,138][105620] Updated weights for policy 1, policy_version 1512976 (0.0008) [2023-12-27 02:23:38,207][105620] Updated weights for policy 1, policy_version 1512986 (0.0007) [2023-12-27 02:23:38,273][105620] Updated weights for policy 1, policy_version 1512996 (0.0010) [2023-12-27 02:23:38,508][105692] Updated weights for policy 0, policy_version 1510443 (0.0008) [2023-12-27 02:23:38,573][105692] Updated weights for policy 0, policy_version 1510453 (0.0009) [2023-12-27 02:23:38,641][105692] Updated weights for policy 0, policy_version 1510463 (0.0009) [2023-12-27 02:23:39,083][105620] Updated weights for policy 1, policy_version 1513006 (0.0010) [2023-12-27 02:23:39,147][105620] Updated weights for policy 1, policy_version 1513016 (0.0008) [2023-12-27 02:23:39,209][105620] Updated weights for policy 1, policy_version 1513026 (0.0010) [2023-12-27 02:23:39,481][105692] Updated weights for policy 0, policy_version 1510473 (0.0011) [2023-12-27 02:23:39,556][105692] Updated weights for policy 0, policy_version 1510483 (0.0007) [2023-12-27 02:23:39,624][105692] Updated weights for policy 0, policy_version 1510493 (0.0008) [2023-12-27 02:23:39,683][105692] Updated weights for policy 0, policy_version 1510503 (0.0009) [2023-12-27 02:23:40,126][105620] Updated weights for policy 1, policy_version 1513036 (0.0008) [2023-12-27 02:23:40,199][105620] Updated weights for policy 1, policy_version 1513046 (0.0006) [2023-12-27 02:23:40,267][105620] Updated weights for policy 1, policy_version 1513056 (0.0007) [2023-12-27 02:23:40,537][105692] Updated weights for policy 0, policy_version 1510513 (0.0008) [2023-12-27 02:23:40,595][105692] Updated weights for policy 0, policy_version 1510523 (0.0007) [2023-12-27 02:23:40,646][105692] Updated weights for policy 0, policy_version 1510533 (0.0008) [2023-12-27 02:23:41,027][105620] Updated weights for policy 1, policy_version 1513066 (0.0009) [2023-12-27 02:23:41,062][104569] Fps is (10 sec: 17203.2, 60 sec: 18295.4, 300 sec: 19216.5). Total num frames: 774152192. Throughput: 0: 9098.9, 1: 9033.0. Samples: 774164168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:23:41,063][104569] Avg episode reward: [(0, '8079.587'), (1, '9262.892')] [2023-12-27 02:23:41,091][105620] Updated weights for policy 1, policy_version 1513076 (0.0009) [2023-12-27 02:23:41,158][105620] Updated weights for policy 1, policy_version 1513086 (0.0009) [2023-12-27 02:23:41,221][105620] Updated weights for policy 1, policy_version 1513096 (0.0007) [2023-12-27 02:23:41,418][105692] Updated weights for policy 0, policy_version 1510543 (0.0009) [2023-12-27 02:23:41,488][105692] Updated weights for policy 0, policy_version 1510553 (0.0008) [2023-12-27 02:23:41,555][105692] Updated weights for policy 0, policy_version 1510563 (0.0009) [2023-12-27 02:23:42,068][105620] Updated weights for policy 1, policy_version 1513106 (0.0009) [2023-12-27 02:23:42,131][105620] Updated weights for policy 1, policy_version 1513116 (0.0009) [2023-12-27 02:23:42,184][105620] Updated weights for policy 1, policy_version 1513126 (0.0009) [2023-12-27 02:23:42,369][105692] Updated weights for policy 0, policy_version 1510573 (0.0010) [2023-12-27 02:23:42,436][105692] Updated weights for policy 0, policy_version 1510583 (0.0008) [2023-12-27 02:23:42,501][105692] Updated weights for policy 0, policy_version 1510593 (0.0009) [2023-12-27 02:23:43,041][105620] Updated weights for policy 1, policy_version 1513136 (0.0006) [2023-12-27 02:23:43,110][105620] Updated weights for policy 1, policy_version 1513146 (0.0007) [2023-12-27 02:23:43,180][105620] Updated weights for policy 1, policy_version 1513156 (0.0008) [2023-12-27 02:23:43,262][105692] Updated weights for policy 0, policy_version 1510603 (0.0008) [2023-12-27 02:23:43,321][105692] Updated weights for policy 0, policy_version 1510613 (0.0010) [2023-12-27 02:23:43,390][105692] Updated weights for policy 0, policy_version 1510623 (0.0008) [2023-12-27 02:23:43,956][105620] Updated weights for policy 1, policy_version 1513166 (0.0007) [2023-12-27 02:23:44,017][105620] Updated weights for policy 1, policy_version 1513176 (0.0010) [2023-12-27 02:23:44,087][105620] Updated weights for policy 1, policy_version 1513186 (0.0009) [2023-12-27 02:23:44,095][105692] Updated weights for policy 0, policy_version 1510633 (0.0008) [2023-12-27 02:23:44,151][105692] Updated weights for policy 0, policy_version 1510643 (0.0008) [2023-12-27 02:23:44,208][105692] Updated weights for policy 0, policy_version 1510653 (0.0009) [2023-12-27 02:23:44,275][105692] Updated weights for policy 0, policy_version 1510663 (0.0010) [2023-12-27 02:23:44,784][105620] Updated weights for policy 1, policy_version 1513196 (0.0009) [2023-12-27 02:23:44,849][105620] Updated weights for policy 1, policy_version 1513206 (0.0009) [2023-12-27 02:23:44,909][105620] Updated weights for policy 1, policy_version 1513216 (0.0007) [2023-12-27 02:23:45,138][105692] Updated weights for policy 0, policy_version 1510673 (0.0009) [2023-12-27 02:23:45,216][105692] Updated weights for policy 0, policy_version 1510683 (0.0009) [2023-12-27 02:23:45,283][105692] Updated weights for policy 0, policy_version 1510693 (0.0009) [2023-12-27 02:23:45,728][105620] Updated weights for policy 1, policy_version 1513226 (0.0009) [2023-12-27 02:23:45,783][105620] Updated weights for policy 1, policy_version 1513236 (0.0009) [2023-12-27 02:23:45,832][105620] Updated weights for policy 1, policy_version 1513246 (0.0010) [2023-12-27 02:23:45,888][105620] Updated weights for policy 1, policy_version 1513256 (0.0009) [2023-12-27 02:23:46,062][104569] Fps is (10 sec: 18022.5, 60 sec: 18295.6, 300 sec: 19188.7). Total num frames: 774242304. Throughput: 0: 8960.7, 1: 9015.3. Samples: 774216368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:23:46,062][104569] Avg episode reward: [(0, '7988.994'), (1, '9262.621')] [2023-12-27 02:23:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001513256_387448832.pth... [2023-12-27 02:23:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001512232_387186688.pth [2023-12-27 02:23:46,074][105692] Updated weights for policy 0, policy_version 1510703 (0.0009) [2023-12-27 02:23:46,128][105692] Updated weights for policy 0, policy_version 1510713 (0.0009) [2023-12-27 02:23:46,185][105692] Updated weights for policy 0, policy_version 1510723 (0.0009) [2023-12-27 02:23:46,212][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001510728_386801664.pth... [2023-12-27 02:23:46,216][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001509704_386539520.pth [2023-12-27 02:23:46,739][105620] Updated weights for policy 1, policy_version 1513266 (0.0008) [2023-12-27 02:23:46,796][105620] Updated weights for policy 1, policy_version 1513276 (0.0008) [2023-12-27 02:23:46,861][105620] Updated weights for policy 1, policy_version 1513286 (0.0008) [2023-12-27 02:23:46,937][105692] Updated weights for policy 0, policy_version 1510733 (0.0009) [2023-12-27 02:23:47,006][105692] Updated weights for policy 0, policy_version 1510743 (0.0008) [2023-12-27 02:23:47,063][105692] Updated weights for policy 0, policy_version 1510753 (0.0009) [2023-12-27 02:23:47,693][105620] Updated weights for policy 1, policy_version 1513296 (0.0009) [2023-12-27 02:23:47,763][105620] Updated weights for policy 1, policy_version 1513306 (0.0008) [2023-12-27 02:23:47,782][105692] Updated weights for policy 0, policy_version 1510763 (0.0007) [2023-12-27 02:23:47,831][105620] Updated weights for policy 1, policy_version 1513316 (0.0008) [2023-12-27 02:23:47,852][105692] Updated weights for policy 0, policy_version 1510773 (0.0011) [2023-12-27 02:23:47,919][105692] Updated weights for policy 0, policy_version 1510783 (0.0010) [2023-12-27 02:23:48,681][105620] Updated weights for policy 1, policy_version 1513326 (0.0008) [2023-12-27 02:23:48,754][105620] Updated weights for policy 1, policy_version 1513336 (0.0010) [2023-12-27 02:23:48,801][105692] Updated weights for policy 0, policy_version 1510793 (0.0011) [2023-12-27 02:23:48,821][105620] Updated weights for policy 1, policy_version 1513346 (0.0010) [2023-12-27 02:23:48,877][105692] Updated weights for policy 0, policy_version 1510803 (0.0012) [2023-12-27 02:23:48,937][105692] Updated weights for policy 0, policy_version 1510813 (0.0008) [2023-12-27 02:23:48,997][105692] Updated weights for policy 0, policy_version 1510823 (0.0008) [2023-12-27 02:23:49,616][105620] Updated weights for policy 1, policy_version 1513356 (0.0010) [2023-12-27 02:23:49,686][105620] Updated weights for policy 1, policy_version 1513366 (0.0007) [2023-12-27 02:23:49,761][105620] Updated weights for policy 1, policy_version 1513376 (0.0008) [2023-12-27 02:23:49,783][105692] Updated weights for policy 0, policy_version 1510833 (0.0008) [2023-12-27 02:23:49,855][105692] Updated weights for policy 0, policy_version 1510843 (0.0008) [2023-12-27 02:23:49,922][105692] Updated weights for policy 0, policy_version 1510853 (0.0009) [2023-12-27 02:23:50,792][105620] Updated weights for policy 1, policy_version 1513386 (0.0010) [2023-12-27 02:23:50,870][105620] Updated weights for policy 1, policy_version 1513396 (0.0009) [2023-12-27 02:23:50,939][105692] Updated weights for policy 0, policy_version 1510864 (0.0009) [2023-12-27 02:23:50,954][105620] Updated weights for policy 1, policy_version 1513406 (0.0012) [2023-12-27 02:23:51,015][105692] Updated weights for policy 0, policy_version 1510874 (0.0007) [2023-12-27 02:23:51,054][105620] Updated weights for policy 1, policy_version 1513416 (0.0013) [2023-12-27 02:23:51,062][104569] Fps is (10 sec: 17203.2, 60 sec: 18022.4, 300 sec: 19188.7). Total num frames: 774324224. Throughput: 0: 8839.7, 1: 8855.5. Samples: 774320532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:23:51,063][104569] Avg episode reward: [(0, '8174.543'), (1, '9261.999')] [2023-12-27 02:23:51,104][105692] Updated weights for policy 0, policy_version 1510884 (0.0008) [2023-12-27 02:23:51,925][105620] Updated weights for policy 1, policy_version 1513426 (0.0011) [2023-12-27 02:23:51,990][105620] Updated weights for policy 1, policy_version 1513436 (0.0011) [2023-12-27 02:23:51,992][105692] Updated weights for policy 0, policy_version 1510894 (0.0013) [2023-12-27 02:23:52,050][105620] Updated weights for policy 1, policy_version 1513446 (0.0011) [2023-12-27 02:23:52,052][105692] Updated weights for policy 0, policy_version 1510904 (0.0007) [2023-12-27 02:23:52,111][105692] Updated weights for policy 0, policy_version 1510914 (0.0008) [2023-12-27 02:23:52,765][105620] Updated weights for policy 1, policy_version 1513456 (0.0006) [2023-12-27 02:23:52,828][105620] Updated weights for policy 1, policy_version 1513466 (0.0008) [2023-12-27 02:23:52,881][105692] Updated weights for policy 0, policy_version 1510924 (0.0008) [2023-12-27 02:23:52,890][105620] Updated weights for policy 1, policy_version 1513476 (0.0008) [2023-12-27 02:23:52,946][105692] Updated weights for policy 0, policy_version 1510934 (0.0006) [2023-12-27 02:23:53,013][105692] Updated weights for policy 0, policy_version 1510944 (0.0008) [2023-12-27 02:23:53,465][105620] Updated weights for policy 1, policy_version 1513486 (0.0009) [2023-12-27 02:23:53,524][105620] Updated weights for policy 1, policy_version 1513496 (0.0011) [2023-12-27 02:23:53,583][105620] Updated weights for policy 1, policy_version 1513506 (0.0010) [2023-12-27 02:23:53,761][105692] Updated weights for policy 0, policy_version 1510954 (0.0009) [2023-12-27 02:23:53,807][105692] Updated weights for policy 0, policy_version 1510964 (0.0005) [2023-12-27 02:23:53,861][105692] Updated weights for policy 0, policy_version 1510974 (0.0005) [2023-12-27 02:23:53,928][105692] Updated weights for policy 0, policy_version 1510984 (0.0005) [2023-12-27 02:23:54,154][105620] Updated weights for policy 1, policy_version 1513516 (0.0006) [2023-12-27 02:23:54,206][105620] Updated weights for policy 1, policy_version 1513526 (0.0005) [2023-12-27 02:23:54,257][105620] Updated weights for policy 1, policy_version 1513536 (0.0007) [2023-12-27 02:23:54,674][105692] Updated weights for policy 0, policy_version 1510994 (0.0008) [2023-12-27 02:23:54,735][105692] Updated weights for policy 0, policy_version 1511004 (0.0008) [2023-12-27 02:23:54,797][105692] Updated weights for policy 0, policy_version 1511014 (0.0009) [2023-12-27 02:23:54,912][105620] Updated weights for policy 1, policy_version 1513546 (0.0010) [2023-12-27 02:23:54,973][105620] Updated weights for policy 1, policy_version 1513556 (0.0009) [2023-12-27 02:23:55,042][105620] Updated weights for policy 1, policy_version 1513566 (0.0009) [2023-12-27 02:23:55,111][105620] Updated weights for policy 1, policy_version 1513576 (0.0010) [2023-12-27 02:23:55,528][105692] Updated weights for policy 0, policy_version 1511024 (0.0008) [2023-12-27 02:23:55,584][105692] Updated weights for policy 0, policy_version 1511034 (0.0008) [2023-12-27 02:23:55,638][105692] Updated weights for policy 0, policy_version 1511044 (0.0008) [2023-12-27 02:23:55,852][105620] Updated weights for policy 1, policy_version 1513586 (0.0010) [2023-12-27 02:23:55,900][105620] Updated weights for policy 1, policy_version 1513596 (0.0010) [2023-12-27 02:23:55,948][105620] Updated weights for policy 1, policy_version 1513606 (0.0010) [2023-12-27 02:23:56,062][104569] Fps is (10 sec: 18022.5, 60 sec: 18022.4, 300 sec: 19188.7). Total num frames: 774422528. Throughput: 0: 8718.7, 1: 8780.0. Samples: 774427812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:23:56,062][104569] Avg episode reward: [(0, '8175.075'), (1, '9257.492')] [2023-12-27 02:23:56,410][105692] Updated weights for policy 0, policy_version 1511054 (0.0007) [2023-12-27 02:23:56,463][105692] Updated weights for policy 0, policy_version 1511064 (0.0007) [2023-12-27 02:23:56,522][105692] Updated weights for policy 0, policy_version 1511075 (0.0010) [2023-12-27 02:23:56,630][105620] Updated weights for policy 1, policy_version 1513616 (0.0006) [2023-12-27 02:23:56,683][105620] Updated weights for policy 1, policy_version 1513626 (0.0005) [2023-12-27 02:23:56,736][105620] Updated weights for policy 1, policy_version 1513636 (0.0006) [2023-12-27 02:23:57,255][105692] Updated weights for policy 0, policy_version 1511086 (0.0008) [2023-12-27 02:23:57,293][105620] Updated weights for policy 1, policy_version 1513646 (0.0009) [2023-12-27 02:23:57,319][105692] Updated weights for policy 0, policy_version 1511096 (0.0008) [2023-12-27 02:23:57,353][105620] Updated weights for policy 1, policy_version 1513656 (0.0006) [2023-12-27 02:23:57,374][105692] Updated weights for policy 0, policy_version 1511107 (0.0010) [2023-12-27 02:23:57,411][105620] Updated weights for policy 1, policy_version 1513666 (0.0005) [2023-12-27 02:23:57,998][105620] Updated weights for policy 1, policy_version 1513676 (0.0009) [2023-12-27 02:23:58,035][105692] Updated weights for policy 0, policy_version 1511117 (0.0008) [2023-12-27 02:23:58,042][105620] Updated weights for policy 1, policy_version 1513686 (0.0008) [2023-12-27 02:23:58,086][105620] Updated weights for policy 1, policy_version 1513696 (0.0007) [2023-12-27 02:23:58,087][105692] Updated weights for policy 0, policy_version 1511127 (0.0009) [2023-12-27 02:23:58,133][105692] Updated weights for policy 0, policy_version 1511137 (0.0007) [2023-12-27 02:23:58,929][105620] Updated weights for policy 1, policy_version 1513706 (0.0008) [2023-12-27 02:23:58,989][105620] Updated weights for policy 1, policy_version 1513716 (0.0009) [2023-12-27 02:23:59,020][105692] Updated weights for policy 0, policy_version 1511147 (0.0009) [2023-12-27 02:23:59,045][105620] Updated weights for policy 1, policy_version 1513726 (0.0009) [2023-12-27 02:23:59,082][105692] Updated weights for policy 0, policy_version 1511157 (0.0010) [2023-12-27 02:23:59,105][105620] Updated weights for policy 1, policy_version 1513736 (0.0010) [2023-12-27 02:23:59,144][105692] Updated weights for policy 0, policy_version 1511167 (0.0010) [2023-12-27 02:23:59,891][105692] Updated weights for policy 0, policy_version 1511177 (0.0010) [2023-12-27 02:23:59,935][105620] Updated weights for policy 1, policy_version 1513746 (0.0009) [2023-12-27 02:23:59,953][105692] Updated weights for policy 0, policy_version 1511187 (0.0007) [2023-12-27 02:24:00,001][105620] Updated weights for policy 1, policy_version 1513756 (0.0011) [2023-12-27 02:24:00,020][105692] Updated weights for policy 0, policy_version 1511197 (0.0006) [2023-12-27 02:24:00,068][105620] Updated weights for policy 1, policy_version 1513766 (0.0011) [2023-12-27 02:24:00,075][105692] Updated weights for policy 0, policy_version 1511207 (0.0008) [2023-12-27 02:24:00,792][105620] Updated weights for policy 1, policy_version 1513776 (0.0010) [2023-12-27 02:24:00,799][105692] Updated weights for policy 0, policy_version 1511217 (0.0010) [2023-12-27 02:24:00,840][105620] Updated weights for policy 1, policy_version 1513786 (0.0010) [2023-12-27 02:24:00,850][105692] Updated weights for policy 0, policy_version 1511227 (0.0010) [2023-12-27 02:24:00,886][105620] Updated weights for policy 1, policy_version 1513796 (0.0010) [2023-12-27 02:24:00,901][105692] Updated weights for policy 0, policy_version 1511237 (0.0010) [2023-12-27 02:24:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 18158.9, 300 sec: 19188.7). Total num frames: 774520832. Throughput: 0: 8807.3, 1: 8934.3. Samples: 774488376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:24:01,063][104569] Avg episode reward: [(0, '8356.972'), (1, '8893.249')] [2023-12-27 02:24:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001511240_386932736.pth... [2023-12-27 02:24:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001513800_387588096.pth... [2023-12-27 02:24:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001512744_387317760.pth [2023-12-27 02:24:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001510184_386662400.pth [2023-12-27 02:24:01,657][105692] Updated weights for policy 0, policy_version 1511247 (0.0007) [2023-12-27 02:24:01,698][105620] Updated weights for policy 1, policy_version 1513806 (0.0010) [2023-12-27 02:24:01,719][105692] Updated weights for policy 0, policy_version 1511257 (0.0008) [2023-12-27 02:24:01,761][105620] Updated weights for policy 1, policy_version 1513816 (0.0009) [2023-12-27 02:24:01,772][105692] Updated weights for policy 0, policy_version 1511267 (0.0006) [2023-12-27 02:24:01,819][105620] Updated weights for policy 1, policy_version 1513826 (0.0009) [2023-12-27 02:24:02,425][105692] Updated weights for policy 0, policy_version 1511277 (0.0007) [2023-12-27 02:24:02,485][105692] Updated weights for policy 0, policy_version 1511287 (0.0008) [2023-12-27 02:24:02,550][105692] Updated weights for policy 0, policy_version 1511297 (0.0010) [2023-12-27 02:24:02,604][105620] Updated weights for policy 1, policy_version 1513836 (0.0009) [2023-12-27 02:24:02,673][105620] Updated weights for policy 1, policy_version 1513846 (0.0009) [2023-12-27 02:24:02,734][105620] Updated weights for policy 1, policy_version 1513856 (0.0008) [2023-12-27 02:24:03,261][105692] Updated weights for policy 0, policy_version 1511307 (0.0009) [2023-12-27 02:24:03,327][105692] Updated weights for policy 0, policy_version 1511317 (0.0005) [2023-12-27 02:24:03,378][105692] Updated weights for policy 0, policy_version 1511327 (0.0005) [2023-12-27 02:24:03,426][105620] Updated weights for policy 1, policy_version 1513866 (0.0008) [2023-12-27 02:24:03,486][105620] Updated weights for policy 1, policy_version 1513876 (0.0007) [2023-12-27 02:24:03,552][105620] Updated weights for policy 1, policy_version 1513886 (0.0007) [2023-12-27 02:24:03,611][105620] Updated weights for policy 1, policy_version 1513896 (0.0005) [2023-12-27 02:24:03,986][105692] Updated weights for policy 0, policy_version 1511337 (0.0006) [2023-12-27 02:24:04,049][105692] Updated weights for policy 0, policy_version 1511347 (0.0009) [2023-12-27 02:24:04,111][105692] Updated weights for policy 0, policy_version 1511357 (0.0009) [2023-12-27 02:24:04,181][105692] Updated weights for policy 0, policy_version 1511367 (0.0008) [2023-12-27 02:24:04,231][105620] Updated weights for policy 1, policy_version 1513906 (0.0008) [2023-12-27 02:24:04,297][105620] Updated weights for policy 1, policy_version 1513916 (0.0008) [2023-12-27 02:24:04,359][105620] Updated weights for policy 1, policy_version 1513926 (0.0006) [2023-12-27 02:24:04,922][105692] Updated weights for policy 0, policy_version 1511377 (0.0008) [2023-12-27 02:24:04,967][105692] Updated weights for policy 0, policy_version 1511387 (0.0008) [2023-12-27 02:24:05,023][105692] Updated weights for policy 0, policy_version 1511397 (0.0008) [2023-12-27 02:24:05,089][105620] Updated weights for policy 1, policy_version 1513936 (0.0011) [2023-12-27 02:24:05,151][105620] Updated weights for policy 1, policy_version 1513946 (0.0010) [2023-12-27 02:24:05,200][105620] Updated weights for policy 1, policy_version 1513956 (0.0010) [2023-12-27 02:24:05,823][105692] Updated weights for policy 0, policy_version 1511407 (0.0008) [2023-12-27 02:24:05,881][105692] Updated weights for policy 0, policy_version 1511417 (0.0008) [2023-12-27 02:24:05,922][105620] Updated weights for policy 1, policy_version 1513966 (0.0008) [2023-12-27 02:24:05,947][105692] Updated weights for policy 0, policy_version 1511427 (0.0007) [2023-12-27 02:24:05,985][105620] Updated weights for policy 1, policy_version 1513976 (0.0008) [2023-12-27 02:24:06,037][105620] Updated weights for policy 1, policy_version 1513986 (0.0009) [2023-12-27 02:24:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 18022.4, 300 sec: 19188.7). Total num frames: 774610944. Throughput: 0: 8894.0, 1: 8975.2. Samples: 774602868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:24:06,063][104569] Avg episode reward: [(0, '8353.434'), (1, '8803.890')] [2023-12-27 02:24:06,750][105692] Updated weights for policy 0, policy_version 1511437 (0.0009) [2023-12-27 02:24:06,806][105620] Updated weights for policy 1, policy_version 1513996 (0.0007) [2023-12-27 02:24:06,808][105692] Updated weights for policy 0, policy_version 1511447 (0.0008) [2023-12-27 02:24:06,858][105692] Updated weights for policy 0, policy_version 1511457 (0.0006) [2023-12-27 02:24:06,867][105620] Updated weights for policy 1, policy_version 1514006 (0.0008) [2023-12-27 02:24:06,923][105620] Updated weights for policy 1, policy_version 1514016 (0.0008) [2023-12-27 02:24:07,645][105692] Updated weights for policy 0, policy_version 1511467 (0.0007) [2023-12-27 02:24:07,684][105620] Updated weights for policy 1, policy_version 1514026 (0.0009) [2023-12-27 02:24:07,711][105692] Updated weights for policy 0, policy_version 1511477 (0.0008) [2023-12-27 02:24:07,735][105620] Updated weights for policy 1, policy_version 1514036 (0.0005) [2023-12-27 02:24:07,766][105692] Updated weights for policy 0, policy_version 1511487 (0.0009) [2023-12-27 02:24:07,783][105620] Updated weights for policy 1, policy_version 1514046 (0.0006) [2023-12-27 02:24:07,833][105620] Updated weights for policy 1, policy_version 1514056 (0.0007) [2023-12-27 02:24:08,535][105692] Updated weights for policy 0, policy_version 1511497 (0.0008) [2023-12-27 02:24:08,582][105620] Updated weights for policy 1, policy_version 1514066 (0.0007) [2023-12-27 02:24:08,594][105692] Updated weights for policy 0, policy_version 1511507 (0.0007) [2023-12-27 02:24:08,645][105620] Updated weights for policy 1, policy_version 1514076 (0.0008) [2023-12-27 02:24:08,660][105692] Updated weights for policy 0, policy_version 1511517 (0.0007) [2023-12-27 02:24:08,708][105620] Updated weights for policy 1, policy_version 1514086 (0.0009) [2023-12-27 02:24:08,724][105692] Updated weights for policy 0, policy_version 1511527 (0.0011) [2023-12-27 02:24:09,404][105620] Updated weights for policy 1, policy_version 1514096 (0.0008) [2023-12-27 02:24:09,468][105620] Updated weights for policy 1, policy_version 1514106 (0.0008) [2023-12-27 02:24:09,535][105620] Updated weights for policy 1, policy_version 1514116 (0.0008) [2023-12-27 02:24:09,563][105692] Updated weights for policy 0, policy_version 1511537 (0.0007) [2023-12-27 02:24:09,621][105692] Updated weights for policy 0, policy_version 1511547 (0.0009) [2023-12-27 02:24:09,676][105692] Updated weights for policy 0, policy_version 1511557 (0.0009) [2023-12-27 02:24:10,189][105620] Updated weights for policy 1, policy_version 1514126 (0.0008) [2023-12-27 02:24:10,252][105620] Updated weights for policy 1, policy_version 1514136 (0.0008) [2023-12-27 02:24:10,323][105620] Updated weights for policy 1, policy_version 1514146 (0.0005) [2023-12-27 02:24:10,507][105692] Updated weights for policy 0, policy_version 1511567 (0.0007) [2023-12-27 02:24:10,569][105692] Updated weights for policy 0, policy_version 1511577 (0.0011) [2023-12-27 02:24:10,626][105692] Updated weights for policy 0, policy_version 1511587 (0.0011) [2023-12-27 02:24:11,001][105620] Updated weights for policy 1, policy_version 1514156 (0.0007) [2023-12-27 02:24:11,057][105620] Updated weights for policy 1, policy_version 1514166 (0.0008) [2023-12-27 02:24:11,062][104569] Fps is (10 sec: 18022.6, 60 sec: 17885.9, 300 sec: 19161.0). Total num frames: 774701056. Throughput: 0: 8941.6, 1: 9121.8. Samples: 774713768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:24:11,063][104569] Avg episode reward: [(0, '8170.246'), (1, '8985.134')] [2023-12-27 02:24:11,119][105620] Updated weights for policy 1, policy_version 1514176 (0.0008) [2023-12-27 02:24:11,355][105692] Updated weights for policy 0, policy_version 1511597 (0.0010) [2023-12-27 02:24:11,421][105692] Updated weights for policy 0, policy_version 1511607 (0.0007) [2023-12-27 02:24:11,469][105692] Updated weights for policy 0, policy_version 1511617 (0.0009) [2023-12-27 02:24:11,886][105620] Updated weights for policy 1, policy_version 1514186 (0.0008) [2023-12-27 02:24:11,942][105620] Updated weights for policy 1, policy_version 1514196 (0.0009) [2023-12-27 02:24:11,997][105620] Updated weights for policy 1, policy_version 1514206 (0.0009) [2023-12-27 02:24:12,058][105620] Updated weights for policy 1, policy_version 1514216 (0.0009) [2023-12-27 02:24:12,253][105692] Updated weights for policy 0, policy_version 1511627 (0.0011) [2023-12-27 02:24:12,320][105692] Updated weights for policy 0, policy_version 1511637 (0.0007) [2023-12-27 02:24:12,383][105692] Updated weights for policy 0, policy_version 1511647 (0.0010) [2023-12-27 02:24:12,761][105620] Updated weights for policy 1, policy_version 1514226 (0.0007) [2023-12-27 02:24:12,809][105620] Updated weights for policy 1, policy_version 1514236 (0.0010) [2023-12-27 02:24:12,867][105620] Updated weights for policy 1, policy_version 1514246 (0.0010) [2023-12-27 02:24:13,190][105692] Updated weights for policy 0, policy_version 1511657 (0.0010) [2023-12-27 02:24:13,244][105692] Updated weights for policy 0, policy_version 1511667 (0.0009) [2023-12-27 02:24:13,295][105692] Updated weights for policy 0, policy_version 1511677 (0.0009) [2023-12-27 02:24:13,346][105692] Updated weights for policy 0, policy_version 1511687 (0.0007) [2023-12-27 02:24:13,599][105620] Updated weights for policy 1, policy_version 1514256 (0.0009) [2023-12-27 02:24:13,660][105620] Updated weights for policy 1, policy_version 1514266 (0.0009) [2023-12-27 02:24:13,716][105620] Updated weights for policy 1, policy_version 1514276 (0.0008) [2023-12-27 02:24:14,091][105692] Updated weights for policy 0, policy_version 1511697 (0.0010) [2023-12-27 02:24:14,153][105692] Updated weights for policy 0, policy_version 1511707 (0.0011) [2023-12-27 02:24:14,211][105692] Updated weights for policy 0, policy_version 1511717 (0.0010) [2023-12-27 02:24:14,504][105620] Updated weights for policy 1, policy_version 1514286 (0.0008) [2023-12-27 02:24:14,570][105620] Updated weights for policy 1, policy_version 1514296 (0.0007) [2023-12-27 02:24:14,631][105620] Updated weights for policy 1, policy_version 1514306 (0.0005) [2023-12-27 02:24:14,965][105692] Updated weights for policy 0, policy_version 1511727 (0.0010) [2023-12-27 02:24:15,028][105692] Updated weights for policy 0, policy_version 1511737 (0.0010) [2023-12-27 02:24:15,092][105692] Updated weights for policy 0, policy_version 1511747 (0.0009) [2023-12-27 02:24:15,312][105620] Updated weights for policy 1, policy_version 1514316 (0.0006) [2023-12-27 02:24:15,371][105620] Updated weights for policy 1, policy_version 1514326 (0.0010) [2023-12-27 02:24:15,418][105620] Updated weights for policy 1, policy_version 1514336 (0.0009) [2023-12-27 02:24:15,829][105692] Updated weights for policy 0, policy_version 1511757 (0.0010) [2023-12-27 02:24:15,877][105692] Updated weights for policy 0, policy_version 1511767 (0.0010) [2023-12-27 02:24:15,924][105692] Updated weights for policy 0, policy_version 1511777 (0.0010) [2023-12-27 02:24:16,062][104569] Fps is (10 sec: 18841.0, 60 sec: 18022.3, 300 sec: 19160.9). Total num frames: 774799360. Throughput: 0: 8929.6, 1: 9176.0. Samples: 774768396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:24:16,063][104569] Avg episode reward: [(0, '8173.404'), (1, '9166.909')] [2023-12-27 02:24:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001514344_387727360.pth... [2023-12-27 02:24:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001511784_387072000.pth... [2023-12-27 02:24:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001513256_387448832.pth [2023-12-27 02:24:16,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001510728_386801664.pth [2023-12-27 02:24:16,107][105620] Updated weights for policy 1, policy_version 1514346 (0.0008) [2023-12-27 02:24:16,155][105620] Updated weights for policy 1, policy_version 1514356 (0.0008) [2023-12-27 02:24:16,207][105620] Updated weights for policy 1, policy_version 1514366 (0.0007) [2023-12-27 02:24:16,261][105620] Updated weights for policy 1, policy_version 1514376 (0.0009) [2023-12-27 02:24:16,618][105692] Updated weights for policy 0, policy_version 1511787 (0.0009) [2023-12-27 02:24:16,664][105692] Updated weights for policy 0, policy_version 1511797 (0.0005) [2023-12-27 02:24:16,711][105692] Updated weights for policy 0, policy_version 1511807 (0.0005) [2023-12-27 02:24:16,986][105620] Updated weights for policy 1, policy_version 1514386 (0.0009) [2023-12-27 02:24:17,044][105620] Updated weights for policy 1, policy_version 1514396 (0.0008) [2023-12-27 02:24:17,107][105620] Updated weights for policy 1, policy_version 1514406 (0.0007) [2023-12-27 02:24:17,335][105692] Updated weights for policy 0, policy_version 1511817 (0.0006) [2023-12-27 02:24:17,389][105692] Updated weights for policy 0, policy_version 1511827 (0.0006) [2023-12-27 02:24:17,440][105692] Updated weights for policy 0, policy_version 1511837 (0.0005) [2023-12-27 02:24:17,491][105692] Updated weights for policy 0, policy_version 1511847 (0.0010) [2023-12-27 02:24:17,792][105620] Updated weights for policy 1, policy_version 1514416 (0.0009) [2023-12-27 02:24:17,846][105620] Updated weights for policy 1, policy_version 1514426 (0.0008) [2023-12-27 02:24:17,901][105620] Updated weights for policy 1, policy_version 1514436 (0.0008) [2023-12-27 02:24:18,163][105692] Updated weights for policy 0, policy_version 1511857 (0.0006) [2023-12-27 02:24:18,184][105585] KL-divergence is very high: 189.5209 [2023-12-27 02:24:18,227][105692] Updated weights for policy 0, policy_version 1511867 (0.0005) [2023-12-27 02:24:18,232][105585] KL-divergence is very high: 359.0958 [2023-12-27 02:24:18,278][105585] KL-divergence is very high: 433.9598 [2023-12-27 02:24:18,283][105692] Updated weights for policy 0, policy_version 1511877 (0.0005) [2023-12-27 02:24:18,526][105620] Updated weights for policy 1, policy_version 1514446 (0.0009) [2023-12-27 02:24:18,575][105620] Updated weights for policy 1, policy_version 1514456 (0.0008) [2023-12-27 02:24:18,633][105620] Updated weights for policy 1, policy_version 1514466 (0.0005) [2023-12-27 02:24:18,907][105692] Updated weights for policy 0, policy_version 1511887 (0.0010) [2023-12-27 02:24:18,966][105692] Updated weights for policy 0, policy_version 1511897 (0.0011) [2023-12-27 02:24:19,026][105692] Updated weights for policy 0, policy_version 1511907 (0.0011) [2023-12-27 02:24:19,298][105620] Updated weights for policy 1, policy_version 1514476 (0.0008) [2023-12-27 02:24:19,367][105620] Updated weights for policy 1, policy_version 1514486 (0.0009) [2023-12-27 02:24:19,426][105620] Updated weights for policy 1, policy_version 1514496 (0.0006) [2023-12-27 02:24:19,825][105692] Updated weights for policy 0, policy_version 1511917 (0.0009) [2023-12-27 02:24:19,895][105692] Updated weights for policy 0, policy_version 1511927 (0.0008) [2023-12-27 02:24:19,961][105692] Updated weights for policy 0, policy_version 1511937 (0.0007) [2023-12-27 02:24:20,237][105620] Updated weights for policy 1, policy_version 1514506 (0.0006) [2023-12-27 02:24:20,289][105620] Updated weights for policy 1, policy_version 1514516 (0.0009) [2023-12-27 02:24:20,351][105620] Updated weights for policy 1, policy_version 1514526 (0.0010) [2023-12-27 02:24:20,409][105620] Updated weights for policy 1, policy_version 1514536 (0.0009) [2023-12-27 02:24:20,574][105692] Updated weights for policy 0, policy_version 1511947 (0.0009) [2023-12-27 02:24:20,641][105692] Updated weights for policy 0, policy_version 1511957 (0.0007) [2023-12-27 02:24:20,702][105692] Updated weights for policy 0, policy_version 1511967 (0.0009) [2023-12-27 02:24:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 18158.9, 300 sec: 19160.9). Total num frames: 774897664. Throughput: 0: 9147.6, 1: 9319.2. Samples: 774889084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:24:21,063][104569] Avg episode reward: [(0, '8357.514'), (1, '9166.345')] [2023-12-27 02:24:21,200][105620] Updated weights for policy 1, policy_version 1514546 (0.0007) [2023-12-27 02:24:21,259][105620] Updated weights for policy 1, policy_version 1514556 (0.0006) [2023-12-27 02:24:21,333][105620] Updated weights for policy 1, policy_version 1514566 (0.0009) [2023-12-27 02:24:21,456][105692] Updated weights for policy 0, policy_version 1511977 (0.0010) [2023-12-27 02:24:21,506][105692] Updated weights for policy 0, policy_version 1511987 (0.0010) [2023-12-27 02:24:21,558][105692] Updated weights for policy 0, policy_version 1511997 (0.0010) [2023-12-27 02:24:21,611][105692] Updated weights for policy 0, policy_version 1512007 (0.0010) [2023-12-27 02:24:22,068][105620] Updated weights for policy 1, policy_version 1514576 (0.0009) [2023-12-27 02:24:22,126][105620] Updated weights for policy 1, policy_version 1514586 (0.0008) [2023-12-27 02:24:22,179][105620] Updated weights for policy 1, policy_version 1514596 (0.0008) [2023-12-27 02:24:22,431][105692] Updated weights for policy 0, policy_version 1512017 (0.0010) [2023-12-27 02:24:22,498][105692] Updated weights for policy 0, policy_version 1512027 (0.0009) [2023-12-27 02:24:22,559][105692] Updated weights for policy 0, policy_version 1512037 (0.0010) [2023-12-27 02:24:22,991][105620] Updated weights for policy 1, policy_version 1514606 (0.0009) [2023-12-27 02:24:23,041][105620] Updated weights for policy 1, policy_version 1514616 (0.0009) [2023-12-27 02:24:23,100][105620] Updated weights for policy 1, policy_version 1514626 (0.0006) [2023-12-27 02:24:23,239][105692] Updated weights for policy 0, policy_version 1512047 (0.0009) [2023-12-27 02:24:23,298][105692] Updated weights for policy 0, policy_version 1512057 (0.0010) [2023-12-27 02:24:23,365][105692] Updated weights for policy 0, policy_version 1512067 (0.0010) [2023-12-27 02:24:23,783][105620] Updated weights for policy 1, policy_version 1514636 (0.0010) [2023-12-27 02:24:23,842][105620] Updated weights for policy 1, policy_version 1514646 (0.0010) [2023-12-27 02:24:23,898][105620] Updated weights for policy 1, policy_version 1514656 (0.0011) [2023-12-27 02:24:24,001][105692] Updated weights for policy 0, policy_version 1512077 (0.0010) [2023-12-27 02:24:24,058][105692] Updated weights for policy 0, policy_version 1512087 (0.0011) [2023-12-27 02:24:24,119][105692] Updated weights for policy 0, policy_version 1512097 (0.0010) [2023-12-27 02:24:24,627][105620] Updated weights for policy 1, policy_version 1514666 (0.0010) [2023-12-27 02:24:24,673][105620] Updated weights for policy 1, policy_version 1514676 (0.0008) [2023-12-27 02:24:24,722][105620] Updated weights for policy 1, policy_version 1514686 (0.0010) [2023-12-27 02:24:24,766][105620] Updated weights for policy 1, policy_version 1514696 (0.0010) [2023-12-27 02:24:24,823][105692] Updated weights for policy 0, policy_version 1512107 (0.0009) [2023-12-27 02:24:24,883][105692] Updated weights for policy 0, policy_version 1512117 (0.0008) [2023-12-27 02:24:24,952][105692] Updated weights for policy 0, policy_version 1512127 (0.0011) [2023-12-27 02:24:25,344][105620] Updated weights for policy 1, policy_version 1514706 (0.0007) [2023-12-27 02:24:25,395][105620] Updated weights for policy 1, policy_version 1514716 (0.0008) [2023-12-27 02:24:25,450][105620] Updated weights for policy 1, policy_version 1514726 (0.0008) [2023-12-27 02:24:25,675][105692] Updated weights for policy 0, policy_version 1512137 (0.0010) [2023-12-27 02:24:25,733][105692] Updated weights for policy 0, policy_version 1512147 (0.0010) [2023-12-27 02:24:25,778][105692] Updated weights for policy 0, policy_version 1512157 (0.0010) [2023-12-27 02:24:25,826][105692] Updated weights for policy 0, policy_version 1512167 (0.0010) [2023-12-27 02:24:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 18432.0, 300 sec: 19161.0). Total num frames: 774995968. Throughput: 0: 9246.0, 1: 9437.1. Samples: 775004904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:24:26,062][104569] Avg episode reward: [(0, '8445.538'), (1, '9079.843')] [2023-12-27 02:24:26,187][105620] Updated weights for policy 1, policy_version 1514736 (0.0010) [2023-12-27 02:24:26,249][105620] Updated weights for policy 1, policy_version 1514746 (0.0010) [2023-12-27 02:24:26,309][105620] Updated weights for policy 1, policy_version 1514756 (0.0007) [2023-12-27 02:24:26,587][105692] Updated weights for policy 0, policy_version 1512177 (0.0010) [2023-12-27 02:24:26,638][105692] Updated weights for policy 0, policy_version 1512187 (0.0010) [2023-12-27 02:24:26,682][105692] Updated weights for policy 0, policy_version 1512197 (0.0010) [2023-12-27 02:24:26,983][105620] Updated weights for policy 1, policy_version 1514766 (0.0008) [2023-12-27 02:24:27,035][105620] Updated weights for policy 1, policy_version 1514776 (0.0007) [2023-12-27 02:24:27,092][105620] Updated weights for policy 1, policy_version 1514786 (0.0008) [2023-12-27 02:24:27,492][105692] Updated weights for policy 0, policy_version 1512207 (0.0008) [2023-12-27 02:24:27,545][105692] Updated weights for policy 0, policy_version 1512217 (0.0006) [2023-12-27 02:24:27,596][105692] Updated weights for policy 0, policy_version 1512227 (0.0005) [2023-12-27 02:24:27,709][105620] Updated weights for policy 1, policy_version 1514796 (0.0009) [2023-12-27 02:24:27,759][105620] Updated weights for policy 1, policy_version 1514806 (0.0008) [2023-12-27 02:24:27,809][105620] Updated weights for policy 1, policy_version 1514816 (0.0005) [2023-12-27 02:24:28,315][105692] Updated weights for policy 0, policy_version 1512237 (0.0005) [2023-12-27 02:24:28,377][105692] Updated weights for policy 0, policy_version 1512247 (0.0007) [2023-12-27 02:24:28,416][105620] Updated weights for policy 1, policy_version 1514826 (0.0006) [2023-12-27 02:24:28,436][105692] Updated weights for policy 0, policy_version 1512257 (0.0008) [2023-12-27 02:24:28,475][105620] Updated weights for policy 1, policy_version 1514836 (0.0009) [2023-12-27 02:24:28,526][105620] Updated weights for policy 1, policy_version 1514846 (0.0010) [2023-12-27 02:24:28,581][105620] Updated weights for policy 1, policy_version 1514856 (0.0010) [2023-12-27 02:24:29,088][105692] Updated weights for policy 0, policy_version 1512267 (0.0007) [2023-12-27 02:24:29,149][105692] Updated weights for policy 0, policy_version 1512277 (0.0007) [2023-12-27 02:24:29,200][105692] Updated weights for policy 0, policy_version 1512287 (0.0007) [2023-12-27 02:24:29,213][105620] Updated weights for policy 1, policy_version 1514866 (0.0010) [2023-12-27 02:24:29,277][105620] Updated weights for policy 1, policy_version 1514876 (0.0010) [2023-12-27 02:24:29,343][105620] Updated weights for policy 1, policy_version 1514886 (0.0010) [2023-12-27 02:24:29,938][105692] Updated weights for policy 0, policy_version 1512297 (0.0010) [2023-12-27 02:24:30,000][105692] Updated weights for policy 0, policy_version 1512307 (0.0010) [2023-12-27 02:24:30,010][105620] Updated weights for policy 1, policy_version 1514896 (0.0010) [2023-12-27 02:24:30,058][105620] Updated weights for policy 1, policy_version 1514906 (0.0010) [2023-12-27 02:24:30,059][105692] Updated weights for policy 0, policy_version 1512317 (0.0010) [2023-12-27 02:24:30,110][105620] Updated weights for policy 1, policy_version 1514916 (0.0010) [2023-12-27 02:24:30,118][105692] Updated weights for policy 0, policy_version 1512327 (0.0010) [2023-12-27 02:24:30,764][105620] Updated weights for policy 1, policy_version 1514926 (0.0010) [2023-12-27 02:24:30,808][105620] Updated weights for policy 1, policy_version 1514936 (0.0010) [2023-12-27 02:24:30,834][105692] Updated weights for policy 0, policy_version 1512337 (0.0010) [2023-12-27 02:24:30,862][105620] Updated weights for policy 1, policy_version 1514946 (0.0010) [2023-12-27 02:24:30,882][105692] Updated weights for policy 0, policy_version 1512347 (0.0010) [2023-12-27 02:24:30,936][105692] Updated weights for policy 0, policy_version 1512357 (0.0010) [2023-12-27 02:24:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 18705.1, 300 sec: 19160.9). Total num frames: 775102464. Throughput: 0: 9282.9, 1: 9585.8. Samples: 775065464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:24:31,063][104569] Avg episode reward: [(0, '8356.483'), (1, '9082.541')] [2023-12-27 02:24:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001512360_387219456.pth... [2023-12-27 02:24:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001514952_387883008.pth... [2023-12-27 02:24:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001513800_387588096.pth [2023-12-27 02:24:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001511240_386932736.pth [2023-12-27 02:24:31,641][105620] Updated weights for policy 1, policy_version 1514956 (0.0010) [2023-12-27 02:24:31,704][105620] Updated weights for policy 1, policy_version 1514966 (0.0009) [2023-12-27 02:24:31,722][105692] Updated weights for policy 0, policy_version 1512367 (0.0010) [2023-12-27 02:24:31,772][105620] Updated weights for policy 1, policy_version 1514977 (0.0009) [2023-12-27 02:24:31,785][105692] Updated weights for policy 0, policy_version 1512377 (0.0011) [2023-12-27 02:24:31,846][105692] Updated weights for policy 0, policy_version 1512387 (0.0010) [2023-12-27 02:24:32,445][105620] Updated weights for policy 1, policy_version 1514987 (0.0010) [2023-12-27 02:24:32,510][105620] Updated weights for policy 1, policy_version 1514997 (0.0010) [2023-12-27 02:24:32,557][105692] Updated weights for policy 0, policy_version 1512397 (0.0006) [2023-12-27 02:24:32,574][105620] Updated weights for policy 1, policy_version 1515007 (0.0007) [2023-12-27 02:24:32,619][105692] Updated weights for policy 0, policy_version 1512407 (0.0006) [2023-12-27 02:24:32,677][105692] Updated weights for policy 0, policy_version 1512417 (0.0005) [2023-12-27 02:24:33,219][105620] Updated weights for policy 1, policy_version 1515017 (0.0008) [2023-12-27 02:24:33,273][105692] Updated weights for policy 0, policy_version 1512427 (0.0007) [2023-12-27 02:24:33,276][105620] Updated weights for policy 1, policy_version 1515027 (0.0010) [2023-12-27 02:24:33,317][105692] Updated weights for policy 0, policy_version 1512437 (0.0010) [2023-12-27 02:24:33,320][105620] Updated weights for policy 1, policy_version 1515037 (0.0010) [2023-12-27 02:24:33,363][105692] Updated weights for policy 0, policy_version 1512447 (0.0011) [2023-12-27 02:24:33,381][105620] Updated weights for policy 1, policy_version 1515047 (0.0010) [2023-12-27 02:24:33,915][105692] Updated weights for policy 0, policy_version 1512457 (0.0006) [2023-12-27 02:24:33,960][105692] Updated weights for policy 0, policy_version 1512467 (0.0005) [2023-12-27 02:24:34,010][105692] Updated weights for policy 0, policy_version 1512477 (0.0005) [2023-12-27 02:24:34,065][105692] Updated weights for policy 0, policy_version 1512487 (0.0005) [2023-12-27 02:24:34,139][105620] Updated weights for policy 1, policy_version 1515057 (0.0010) [2023-12-27 02:24:34,202][105620] Updated weights for policy 1, policy_version 1515067 (0.0008) [2023-12-27 02:24:34,266][105620] Updated weights for policy 1, policy_version 1515077 (0.0008) [2023-12-27 02:24:34,763][105692] Updated weights for policy 0, policy_version 1512497 (0.0010) [2023-12-27 02:24:34,823][105692] Updated weights for policy 0, policy_version 1512507 (0.0011) [2023-12-27 02:24:34,863][105620] Updated weights for policy 1, policy_version 1515087 (0.0006) [2023-12-27 02:24:34,887][105692] Updated weights for policy 0, policy_version 1512517 (0.0011) [2023-12-27 02:24:34,922][105620] Updated weights for policy 1, policy_version 1515097 (0.0006) [2023-12-27 02:24:34,994][105620] Updated weights for policy 1, policy_version 1515107 (0.0010) [2023-12-27 02:24:35,497][105692] Updated weights for policy 0, policy_version 1512527 (0.0010) [2023-12-27 02:24:35,554][105692] Updated weights for policy 0, policy_version 1512537 (0.0011) [2023-12-27 02:24:35,557][105620] Updated weights for policy 1, policy_version 1515117 (0.0007) [2023-12-27 02:24:35,607][105692] Updated weights for policy 0, policy_version 1512547 (0.0011) [2023-12-27 02:24:35,613][105620] Updated weights for policy 1, policy_version 1515127 (0.0005) [2023-12-27 02:24:35,685][105620] Updated weights for policy 1, policy_version 1515137 (0.0009) [2023-12-27 02:24:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 18978.2, 300 sec: 19161.0). Total num frames: 775200768. Throughput: 0: 9463.7, 1: 9806.3. Samples: 775187680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:24:36,062][104569] Avg episode reward: [(0, '8352.606'), (1, '9349.935')] [2023-12-27 02:24:36,347][105692] Updated weights for policy 0, policy_version 1512557 (0.0008) [2023-12-27 02:24:36,389][105620] Updated weights for policy 1, policy_version 1515147 (0.0011) [2023-12-27 02:24:36,412][105692] Updated weights for policy 0, policy_version 1512567 (0.0008) [2023-12-27 02:24:36,456][105620] Updated weights for policy 1, policy_version 1515157 (0.0011) [2023-12-27 02:24:36,472][105692] Updated weights for policy 0, policy_version 1512577 (0.0007) [2023-12-27 02:24:36,523][105620] Updated weights for policy 1, policy_version 1515167 (0.0010) [2023-12-27 02:24:37,246][105692] Updated weights for policy 0, policy_version 1512587 (0.0009) [2023-12-27 02:24:37,253][105620] Updated weights for policy 1, policy_version 1515177 (0.0010) [2023-12-27 02:24:37,292][105692] Updated weights for policy 0, policy_version 1512597 (0.0005) [2023-12-27 02:24:37,298][105620] Updated weights for policy 1, policy_version 1515187 (0.0010) [2023-12-27 02:24:37,336][105692] Updated weights for policy 0, policy_version 1512607 (0.0005) [2023-12-27 02:24:37,349][105620] Updated weights for policy 1, policy_version 1515197 (0.0010) [2023-12-27 02:24:37,412][105620] Updated weights for policy 1, policy_version 1515207 (0.0010) [2023-12-27 02:24:38,135][105692] Updated weights for policy 0, policy_version 1512617 (0.0009) [2023-12-27 02:24:38,172][105620] Updated weights for policy 1, policy_version 1515217 (0.0010) [2023-12-27 02:24:38,191][105692] Updated weights for policy 0, policy_version 1512627 (0.0008) [2023-12-27 02:24:38,220][105620] Updated weights for policy 1, policy_version 1515227 (0.0010) [2023-12-27 02:24:38,262][105620] Updated weights for policy 1, policy_version 1515237 (0.0010) [2023-12-27 02:24:38,263][105692] Updated weights for policy 0, policy_version 1512637 (0.0005) [2023-12-27 02:24:38,323][105692] Updated weights for policy 0, policy_version 1512647 (0.0009) [2023-12-27 02:24:39,005][105692] Updated weights for policy 0, policy_version 1512657 (0.0007) [2023-12-27 02:24:39,013][105620] Updated weights for policy 1, policy_version 1515247 (0.0008) [2023-12-27 02:24:39,076][105620] Updated weights for policy 1, policy_version 1515257 (0.0005) [2023-12-27 02:24:39,077][105692] Updated weights for policy 0, policy_version 1512667 (0.0005) [2023-12-27 02:24:39,143][105620] Updated weights for policy 1, policy_version 1515267 (0.0005) [2023-12-27 02:24:39,145][105692] Updated weights for policy 0, policy_version 1512677 (0.0006) [2023-12-27 02:24:39,807][105620] Updated weights for policy 1, policy_version 1515277 (0.0007) [2023-12-27 02:24:39,865][105692] Updated weights for policy 0, policy_version 1512687 (0.0008) [2023-12-27 02:24:39,870][105620] Updated weights for policy 1, policy_version 1515287 (0.0006) [2023-12-27 02:24:39,922][105692] Updated weights for policy 0, policy_version 1512697 (0.0009) [2023-12-27 02:24:39,932][105620] Updated weights for policy 1, policy_version 1515297 (0.0008) [2023-12-27 02:24:39,988][105692] Updated weights for policy 0, policy_version 1512707 (0.0008) [2023-12-27 02:24:40,618][105620] Updated weights for policy 1, policy_version 1515307 (0.0008) [2023-12-27 02:24:40,672][105620] Updated weights for policy 1, policy_version 1515317 (0.0010) [2023-12-27 02:24:40,721][105692] Updated weights for policy 0, policy_version 1512717 (0.0009) [2023-12-27 02:24:40,732][105620] Updated weights for policy 1, policy_version 1515327 (0.0008) [2023-12-27 02:24:40,783][105692] Updated weights for policy 0, policy_version 1512727 (0.0006) [2023-12-27 02:24:40,837][105692] Updated weights for policy 0, policy_version 1512737 (0.0006) [2023-12-27 02:24:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19114.7, 300 sec: 19161.0). Total num frames: 775299072. Throughput: 0: 9599.2, 1: 9859.8. Samples: 775303468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:24:41,063][104569] Avg episode reward: [(0, '8626.362'), (1, '9350.756')] [2023-12-27 02:24:41,507][105692] Updated weights for policy 0, policy_version 1512747 (0.0006) [2023-12-27 02:24:41,546][105620] Updated weights for policy 1, policy_version 1515337 (0.0008) [2023-12-27 02:24:41,572][105692] Updated weights for policy 0, policy_version 1512757 (0.0007) [2023-12-27 02:24:41,604][105620] Updated weights for policy 1, policy_version 1515347 (0.0008) [2023-12-27 02:24:41,637][105692] Updated weights for policy 0, policy_version 1512767 (0.0008) [2023-12-27 02:24:41,674][105620] Updated weights for policy 1, policy_version 1515357 (0.0007) [2023-12-27 02:24:41,736][105620] Updated weights for policy 1, policy_version 1515367 (0.0007) [2023-12-27 02:24:42,424][105692] Updated weights for policy 0, policy_version 1512777 (0.0008) [2023-12-27 02:24:42,430][105620] Updated weights for policy 1, policy_version 1515377 (0.0010) [2023-12-27 02:24:42,482][105620] Updated weights for policy 1, policy_version 1515387 (0.0011) [2023-12-27 02:24:42,488][105692] Updated weights for policy 0, policy_version 1512787 (0.0008) [2023-12-27 02:24:42,545][105692] Updated weights for policy 0, policy_version 1512797 (0.0007) [2023-12-27 02:24:42,549][105620] Updated weights for policy 1, policy_version 1515397 (0.0010) [2023-12-27 02:24:42,609][105692] Updated weights for policy 0, policy_version 1512807 (0.0008) [2023-12-27 02:24:43,171][105692] Updated weights for policy 0, policy_version 1512817 (0.0010) [2023-12-27 02:24:43,215][105692] Updated weights for policy 0, policy_version 1512827 (0.0010) [2023-12-27 02:24:43,263][105692] Updated weights for policy 0, policy_version 1512837 (0.0010) [2023-12-27 02:24:43,304][105620] Updated weights for policy 1, policy_version 1515407 (0.0009) [2023-12-27 02:24:43,358][105620] Updated weights for policy 1, policy_version 1515417 (0.0009) [2023-12-27 02:24:43,411][105620] Updated weights for policy 1, policy_version 1515427 (0.0009) [2023-12-27 02:24:43,945][105692] Updated weights for policy 0, policy_version 1512847 (0.0008) [2023-12-27 02:24:44,006][105692] Updated weights for policy 0, policy_version 1512857 (0.0008) [2023-12-27 02:24:44,075][105692] Updated weights for policy 0, policy_version 1512867 (0.0008) [2023-12-27 02:24:44,199][105620] Updated weights for policy 1, policy_version 1515437 (0.0010) [2023-12-27 02:24:44,261][105620] Updated weights for policy 1, policy_version 1515447 (0.0010) [2023-12-27 02:24:44,324][105620] Updated weights for policy 1, policy_version 1515458 (0.0010) [2023-12-27 02:24:44,619][105692] Updated weights for policy 0, policy_version 1512877 (0.0007) [2023-12-27 02:24:44,677][105692] Updated weights for policy 0, policy_version 1512887 (0.0009) [2023-12-27 02:24:44,735][105692] Updated weights for policy 0, policy_version 1512897 (0.0008) [2023-12-27 02:24:45,145][105620] Updated weights for policy 1, policy_version 1515468 (0.0009) [2023-12-27 02:24:45,193][105620] Updated weights for policy 1, policy_version 1515478 (0.0009) [2023-12-27 02:24:45,241][105620] Updated weights for policy 1, policy_version 1515488 (0.0009) [2023-12-27 02:24:45,437][105692] Updated weights for policy 0, policy_version 1512907 (0.0009) [2023-12-27 02:24:45,508][105692] Updated weights for policy 0, policy_version 1512917 (0.0009) [2023-12-27 02:24:45,575][105692] Updated weights for policy 0, policy_version 1512927 (0.0009) [2023-12-27 02:24:45,993][105620] Updated weights for policy 1, policy_version 1515498 (0.0009) [2023-12-27 02:24:46,046][105620] Updated weights for policy 1, policy_version 1515508 (0.0005) [2023-12-27 02:24:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.7, 300 sec: 19133.2). Total num frames: 775389184. Throughput: 0: 9638.2, 1: 9765.6. Samples: 775361544. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:24:46,062][104569] Avg episode reward: [(0, '7986.796'), (1, '9258.718')] [2023-12-27 02:24:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001512936_387366912.pth... [2023-12-27 02:24:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001511784_387072000.pth [2023-12-27 02:24:46,111][105620] Updated weights for policy 1, policy_version 1515518 (0.0005) [2023-12-27 02:24:46,156][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001515528_388030464.pth... [2023-12-27 02:24:46,158][105620] Updated weights for policy 1, policy_version 1515528 (0.0005) [2023-12-27 02:24:46,159][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001514344_387727360.pth [2023-12-27 02:24:46,324][105692] Updated weights for policy 0, policy_version 1512937 (0.0008) [2023-12-27 02:24:46,375][105692] Updated weights for policy 0, policy_version 1512947 (0.0005) [2023-12-27 02:24:46,420][105692] Updated weights for policy 0, policy_version 1512957 (0.0005) [2023-12-27 02:24:46,467][105692] Updated weights for policy 0, policy_version 1512967 (0.0005) [2023-12-27 02:24:46,909][105620] Updated weights for policy 1, policy_version 1515538 (0.0009) [2023-12-27 02:24:46,967][105620] Updated weights for policy 1, policy_version 1515549 (0.0010) [2023-12-27 02:24:47,027][105620] Updated weights for policy 1, policy_version 1515559 (0.0008) [2023-12-27 02:24:47,071][105692] Updated weights for policy 0, policy_version 1512977 (0.0008) [2023-12-27 02:24:47,142][105692] Updated weights for policy 0, policy_version 1512987 (0.0006) [2023-12-27 02:24:47,201][105692] Updated weights for policy 0, policy_version 1512997 (0.0010) [2023-12-27 02:24:47,751][105620] Updated weights for policy 1, policy_version 1515569 (0.0007) [2023-12-27 02:24:47,804][105620] Updated weights for policy 1, policy_version 1515579 (0.0008) [2023-12-27 02:24:47,863][105620] Updated weights for policy 1, policy_version 1515589 (0.0009) [2023-12-27 02:24:47,889][105692] Updated weights for policy 0, policy_version 1513007 (0.0010) [2023-12-27 02:24:47,950][105692] Updated weights for policy 0, policy_version 1513017 (0.0010) [2023-12-27 02:24:47,998][105692] Updated weights for policy 0, policy_version 1513027 (0.0010) [2023-12-27 02:24:48,590][105620] Updated weights for policy 1, policy_version 1515599 (0.0008) [2023-12-27 02:24:48,650][105620] Updated weights for policy 1, policy_version 1515609 (0.0008) [2023-12-27 02:24:48,661][105692] Updated weights for policy 0, policy_version 1513037 (0.0006) [2023-12-27 02:24:48,713][105620] Updated weights for policy 1, policy_version 1515619 (0.0008) [2023-12-27 02:24:48,725][105692] Updated weights for policy 0, policy_version 1513047 (0.0006) [2023-12-27 02:24:48,784][105692] Updated weights for policy 0, policy_version 1513057 (0.0009) [2023-12-27 02:24:49,439][105692] Updated weights for policy 0, policy_version 1513067 (0.0010) [2023-12-27 02:24:49,494][105692] Updated weights for policy 0, policy_version 1513077 (0.0008) [2023-12-27 02:24:49,524][105620] Updated weights for policy 1, policy_version 1515629 (0.0009) [2023-12-27 02:24:49,549][105692] Updated weights for policy 0, policy_version 1513087 (0.0008) [2023-12-27 02:24:49,583][105620] Updated weights for policy 1, policy_version 1515639 (0.0006) [2023-12-27 02:24:49,645][105620] Updated weights for policy 1, policy_version 1515649 (0.0010) [2023-12-27 02:24:50,199][105692] Updated weights for policy 0, policy_version 1513097 (0.0009) [2023-12-27 02:24:50,249][105692] Updated weights for policy 0, policy_version 1513107 (0.0006) [2023-12-27 02:24:50,303][105692] Updated weights for policy 0, policy_version 1513117 (0.0007) [2023-12-27 02:24:50,329][105620] Updated weights for policy 1, policy_version 1515659 (0.0008) [2023-12-27 02:24:50,360][105692] Updated weights for policy 0, policy_version 1513127 (0.0009) [2023-12-27 02:24:50,394][105620] Updated weights for policy 1, policy_version 1515669 (0.0008) [2023-12-27 02:24:50,454][105620] Updated weights for policy 1, policy_version 1515679 (0.0007) [2023-12-27 02:24:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19133.2). Total num frames: 775487488. Throughput: 0: 9729.4, 1: 9739.4. Samples: 775478964. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:24:51,062][104569] Avg episode reward: [(0, '8349.592'), (1, '9079.574')] [2023-12-27 02:24:51,091][105692] Updated weights for policy 0, policy_version 1513137 (0.0010) [2023-12-27 02:24:51,151][105692] Updated weights for policy 0, policy_version 1513147 (0.0010) [2023-12-27 02:24:51,205][105692] Updated weights for policy 0, policy_version 1513157 (0.0008) [2023-12-27 02:24:51,211][105620] Updated weights for policy 1, policy_version 1515689 (0.0006) [2023-12-27 02:24:51,271][105620] Updated weights for policy 1, policy_version 1515699 (0.0011) [2023-12-27 02:24:51,326][105620] Updated weights for policy 1, policy_version 1515709 (0.0010) [2023-12-27 02:24:51,394][105620] Updated weights for policy 1, policy_version 1515719 (0.0011) [2023-12-27 02:24:51,962][105692] Updated weights for policy 0, policy_version 1513167 (0.0010) [2023-12-27 02:24:52,021][105692] Updated weights for policy 0, policy_version 1513177 (0.0010) [2023-12-27 02:24:52,078][105692] Updated weights for policy 0, policy_version 1513187 (0.0010) [2023-12-27 02:24:52,149][105620] Updated weights for policy 1, policy_version 1515729 (0.0011) [2023-12-27 02:24:52,207][105620] Updated weights for policy 1, policy_version 1515739 (0.0010) [2023-12-27 02:24:52,262][105620] Updated weights for policy 1, policy_version 1515749 (0.0010) [2023-12-27 02:24:52,841][105692] Updated weights for policy 0, policy_version 1513197 (0.0008) [2023-12-27 02:24:52,901][105692] Updated weights for policy 0, policy_version 1513207 (0.0008) [2023-12-27 02:24:52,958][105692] Updated weights for policy 0, policy_version 1513217 (0.0007) [2023-12-27 02:24:53,014][105620] Updated weights for policy 1, policy_version 1515759 (0.0009) [2023-12-27 02:24:53,081][105620] Updated weights for policy 1, policy_version 1515769 (0.0006) [2023-12-27 02:24:53,145][105620] Updated weights for policy 1, policy_version 1515779 (0.0005) [2023-12-27 02:24:53,740][105692] Updated weights for policy 0, policy_version 1513227 (0.0009) [2023-12-27 02:24:53,793][105692] Updated weights for policy 0, policy_version 1513237 (0.0009) [2023-12-27 02:24:53,796][105620] Updated weights for policy 1, policy_version 1515789 (0.0006) [2023-12-27 02:24:53,841][105620] Updated weights for policy 1, policy_version 1515799 (0.0005) [2023-12-27 02:24:53,844][105692] Updated weights for policy 0, policy_version 1513247 (0.0007) [2023-12-27 02:24:53,893][105620] Updated weights for policy 1, policy_version 1515809 (0.0005) [2023-12-27 02:24:54,513][105620] Updated weights for policy 1, policy_version 1515819 (0.0007) [2023-12-27 02:24:54,575][105620] Updated weights for policy 1, policy_version 1515829 (0.0009) [2023-12-27 02:24:54,634][105620] Updated weights for policy 1, policy_version 1515839 (0.0009) [2023-12-27 02:24:54,662][105692] Updated weights for policy 0, policy_version 1513257 (0.0008) [2023-12-27 02:24:54,706][105692] Updated weights for policy 0, policy_version 1513267 (0.0007) [2023-12-27 02:24:54,753][105692] Updated weights for policy 0, policy_version 1513277 (0.0009) [2023-12-27 02:24:54,800][105692] Updated weights for policy 0, policy_version 1513287 (0.0009) [2023-12-27 02:24:55,397][105620] Updated weights for policy 1, policy_version 1515849 (0.0008) [2023-12-27 02:24:55,447][105620] Updated weights for policy 1, policy_version 1515859 (0.0009) [2023-12-27 02:24:55,505][105620] Updated weights for policy 1, policy_version 1515869 (0.0009) [2023-12-27 02:24:55,562][105620] Updated weights for policy 1, policy_version 1515879 (0.0007) [2023-12-27 02:24:55,573][105692] Updated weights for policy 0, policy_version 1513297 (0.0008) [2023-12-27 02:24:55,624][105692] Updated weights for policy 0, policy_version 1513307 (0.0009) [2023-12-27 02:24:55,677][105692] Updated weights for policy 0, policy_version 1513317 (0.0009) [2023-12-27 02:24:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19161.0). Total num frames: 775585792. Throughput: 0: 9806.3, 1: 9754.9. Samples: 775594024. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:24:56,062][104569] Avg episode reward: [(0, '8812.956'), (1, '8713.358')] [2023-12-27 02:24:56,258][105620] Updated weights for policy 1, policy_version 1515890 (0.0009) [2023-12-27 02:24:56,311][105692] Updated weights for policy 0, policy_version 1513327 (0.0007) [2023-12-27 02:24:56,311][105620] Updated weights for policy 1, policy_version 1515901 (0.0008) [2023-12-27 02:24:56,367][105692] Updated weights for policy 0, policy_version 1513337 (0.0006) [2023-12-27 02:24:56,375][105620] Updated weights for policy 1, policy_version 1515911 (0.0005) [2023-12-27 02:24:56,421][105692] Updated weights for policy 0, policy_version 1513347 (0.0007) [2023-12-27 02:24:56,991][105692] Updated weights for policy 0, policy_version 1513357 (0.0005) [2023-12-27 02:24:57,052][105692] Updated weights for policy 0, policy_version 1513367 (0.0005) [2023-12-27 02:24:57,058][105620] Updated weights for policy 1, policy_version 1515921 (0.0009) [2023-12-27 02:24:57,107][105692] Updated weights for policy 0, policy_version 1513377 (0.0005) [2023-12-27 02:24:57,114][105620] Updated weights for policy 1, policy_version 1515931 (0.0009) [2023-12-27 02:24:57,168][105620] Updated weights for policy 1, policy_version 1515941 (0.0009) [2023-12-27 02:24:57,628][105692] Updated weights for policy 0, policy_version 1513387 (0.0005) [2023-12-27 02:24:57,689][105692] Updated weights for policy 0, policy_version 1513397 (0.0005) [2023-12-27 02:24:57,738][105692] Updated weights for policy 0, policy_version 1513407 (0.0006) [2023-12-27 02:24:57,807][105620] Updated weights for policy 1, policy_version 1515952 (0.0006) [2023-12-27 02:24:57,859][105620] Updated weights for policy 1, policy_version 1515962 (0.0005) [2023-12-27 02:24:57,915][105620] Updated weights for policy 1, policy_version 1515972 (0.0006) [2023-12-27 02:24:58,336][105692] Updated weights for policy 0, policy_version 1513417 (0.0006) [2023-12-27 02:24:58,399][105692] Updated weights for policy 0, policy_version 1513427 (0.0013) [2023-12-27 02:24:58,461][105692] Updated weights for policy 0, policy_version 1513437 (0.0011) [2023-12-27 02:24:58,529][105692] Updated weights for policy 0, policy_version 1513447 (0.0011) [2023-12-27 02:24:58,634][105620] Updated weights for policy 1, policy_version 1515982 (0.0007) [2023-12-27 02:24:58,699][105620] Updated weights for policy 1, policy_version 1515992 (0.0006) [2023-12-27 02:24:58,763][105620] Updated weights for policy 1, policy_version 1516002 (0.0008) [2023-12-27 02:24:59,263][105692] Updated weights for policy 0, policy_version 1513457 (0.0008) [2023-12-27 02:24:59,331][105692] Updated weights for policy 0, policy_version 1513467 (0.0005) [2023-12-27 02:24:59,400][105692] Updated weights for policy 0, policy_version 1513477 (0.0010) [2023-12-27 02:24:59,522][105620] Updated weights for policy 1, policy_version 1516012 (0.0008) [2023-12-27 02:24:59,578][105620] Updated weights for policy 1, policy_version 1516022 (0.0008) [2023-12-27 02:24:59,630][105620] Updated weights for policy 1, policy_version 1516032 (0.0008) [2023-12-27 02:25:00,122][105692] Updated weights for policy 0, policy_version 1513487 (0.0011) [2023-12-27 02:25:00,181][105692] Updated weights for policy 0, policy_version 1513497 (0.0010) [2023-12-27 02:25:00,232][105692] Updated weights for policy 0, policy_version 1513507 (0.0010) [2023-12-27 02:25:00,344][105620] Updated weights for policy 1, policy_version 1516042 (0.0007) [2023-12-27 02:25:00,399][105620] Updated weights for policy 1, policy_version 1516052 (0.0006) [2023-12-27 02:25:00,446][105620] Updated weights for policy 1, policy_version 1516062 (0.0008) [2023-12-27 02:25:00,497][105620] Updated weights for policy 1, policy_version 1516072 (0.0007) [2023-12-27 02:25:00,938][105692] Updated weights for policy 0, policy_version 1513517 (0.0008) [2023-12-27 02:25:00,990][105692] Updated weights for policy 0, policy_version 1513527 (0.0007) [2023-12-27 02:25:01,045][105692] Updated weights for policy 0, policy_version 1513537 (0.0009) [2023-12-27 02:25:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19160.9). Total num frames: 775684096. Throughput: 0: 9972.2, 1: 9824.8. Samples: 775659256. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:25:01,062][104569] Avg episode reward: [(0, '8720.037'), (1, '8805.341')] [2023-12-27 02:25:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001516072_388169728.pth... [2023-12-27 02:25:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001514952_387883008.pth [2023-12-27 02:25:01,089][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001513544_387522560.pth... [2023-12-27 02:25:01,093][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001512360_387219456.pth [2023-12-27 02:25:01,250][105620] Updated weights for policy 1, policy_version 1516082 (0.0009) [2023-12-27 02:25:01,303][105620] Updated weights for policy 1, policy_version 1516092 (0.0009) [2023-12-27 02:25:01,361][105620] Updated weights for policy 1, policy_version 1516102 (0.0009) [2023-12-27 02:25:01,760][105692] Updated weights for policy 0, policy_version 1513547 (0.0007) [2023-12-27 02:25:01,821][105692] Updated weights for policy 0, policy_version 1513557 (0.0009) [2023-12-27 02:25:01,879][105692] Updated weights for policy 0, policy_version 1513567 (0.0009) [2023-12-27 02:25:02,108][105620] Updated weights for policy 1, policy_version 1516112 (0.0008) [2023-12-27 02:25:02,166][105620] Updated weights for policy 1, policy_version 1516122 (0.0005) [2023-12-27 02:25:02,220][105620] Updated weights for policy 1, policy_version 1516132 (0.0005) [2023-12-27 02:25:02,703][105692] Updated weights for policy 0, policy_version 1513577 (0.0009) [2023-12-27 02:25:02,766][105692] Updated weights for policy 0, policy_version 1513587 (0.0009) [2023-12-27 02:25:02,818][105692] Updated weights for policy 0, policy_version 1513597 (0.0008) [2023-12-27 02:25:02,835][105620] Updated weights for policy 1, policy_version 1516142 (0.0006) [2023-12-27 02:25:02,873][105692] Updated weights for policy 0, policy_version 1513607 (0.0011) [2023-12-27 02:25:02,886][105620] Updated weights for policy 1, policy_version 1516152 (0.0005) [2023-12-27 02:25:02,942][105620] Updated weights for policy 1, policy_version 1516162 (0.0009) [2023-12-27 02:25:03,474][105692] Updated weights for policy 0, policy_version 1513617 (0.0010) [2023-12-27 02:25:03,525][105692] Updated weights for policy 0, policy_version 1513627 (0.0010) [2023-12-27 02:25:03,579][105692] Updated weights for policy 0, policy_version 1513637 (0.0010) [2023-12-27 02:25:03,692][105620] Updated weights for policy 1, policy_version 1516172 (0.0008) [2023-12-27 02:25:03,750][105620] Updated weights for policy 1, policy_version 1516182 (0.0007) [2023-12-27 02:25:03,804][105620] Updated weights for policy 1, policy_version 1516192 (0.0008) [2023-12-27 02:25:04,297][105692] Updated weights for policy 0, policy_version 1513647 (0.0007) [2023-12-27 02:25:04,350][105692] Updated weights for policy 0, policy_version 1513657 (0.0006) [2023-12-27 02:25:04,412][105692] Updated weights for policy 0, policy_version 1513667 (0.0009) [2023-12-27 02:25:04,582][105620] Updated weights for policy 1, policy_version 1516202 (0.0007) [2023-12-27 02:25:04,640][105620] Updated weights for policy 1, policy_version 1516212 (0.0010) [2023-12-27 02:25:04,695][105620] Updated weights for policy 1, policy_version 1516222 (0.0009) [2023-12-27 02:25:04,757][105620] Updated weights for policy 1, policy_version 1516232 (0.0009) [2023-12-27 02:25:05,036][105692] Updated weights for policy 0, policy_version 1513677 (0.0008) [2023-12-27 02:25:05,091][105692] Updated weights for policy 0, policy_version 1513687 (0.0009) [2023-12-27 02:25:05,146][105692] Updated weights for policy 0, policy_version 1513697 (0.0009) [2023-12-27 02:25:05,550][105620] Updated weights for policy 1, policy_version 1516242 (0.0009) [2023-12-27 02:25:05,617][105620] Updated weights for policy 1, policy_version 1516252 (0.0008) [2023-12-27 02:25:05,680][105620] Updated weights for policy 1, policy_version 1516262 (0.0008) [2023-12-27 02:25:05,904][105692] Updated weights for policy 0, policy_version 1513707 (0.0009) [2023-12-27 02:25:05,963][105692] Updated weights for policy 0, policy_version 1513717 (0.0010) [2023-12-27 02:25:06,024][105692] Updated weights for policy 0, policy_version 1513727 (0.0010) [2023-12-27 02:25:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19188.7). Total num frames: 775782400. Throughput: 0: 9940.8, 1: 9745.3. Samples: 775774956. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:25:06,063][104569] Avg episode reward: [(0, '8256.504'), (1, '9082.013')] [2023-12-27 02:25:06,435][105620] Updated weights for policy 1, policy_version 1516272 (0.0007) [2023-12-27 02:25:06,499][105620] Updated weights for policy 1, policy_version 1516282 (0.0010) [2023-12-27 02:25:06,559][105620] Updated weights for policy 1, policy_version 1516292 (0.0008) [2023-12-27 02:25:06,845][105692] Updated weights for policy 0, policy_version 1513737 (0.0009) [2023-12-27 02:25:06,894][105692] Updated weights for policy 0, policy_version 1513747 (0.0008) [2023-12-27 02:25:06,950][105692] Updated weights for policy 0, policy_version 1513757 (0.0008) [2023-12-27 02:25:07,015][105692] Updated weights for policy 0, policy_version 1513767 (0.0009) [2023-12-27 02:25:07,254][105620] Updated weights for policy 1, policy_version 1516302 (0.0006) [2023-12-27 02:25:07,308][105620] Updated weights for policy 1, policy_version 1516312 (0.0005) [2023-12-27 02:25:07,365][105620] Updated weights for policy 1, policy_version 1516322 (0.0006) [2023-12-27 02:25:07,821][105692] Updated weights for policy 0, policy_version 1513777 (0.0009) [2023-12-27 02:25:07,868][105692] Updated weights for policy 0, policy_version 1513787 (0.0009) [2023-12-27 02:25:07,921][105692] Updated weights for policy 0, policy_version 1513798 (0.0010) [2023-12-27 02:25:07,979][105620] Updated weights for policy 1, policy_version 1516332 (0.0008) [2023-12-27 02:25:08,029][105620] Updated weights for policy 1, policy_version 1516342 (0.0005) [2023-12-27 02:25:08,089][105620] Updated weights for policy 1, policy_version 1516352 (0.0006) [2023-12-27 02:25:08,752][105692] Updated weights for policy 0, policy_version 1513808 (0.0009) [2023-12-27 02:25:08,789][105620] Updated weights for policy 1, policy_version 1516362 (0.0009) [2023-12-27 02:25:08,804][105692] Updated weights for policy 0, policy_version 1513818 (0.0007) [2023-12-27 02:25:08,847][105620] Updated weights for policy 1, policy_version 1516372 (0.0008) [2023-12-27 02:25:08,858][105692] Updated weights for policy 0, policy_version 1513828 (0.0007) [2023-12-27 02:25:08,905][105620] Updated weights for policy 1, policy_version 1516382 (0.0010) [2023-12-27 02:25:08,957][105620] Updated weights for policy 1, policy_version 1516392 (0.0009) [2023-12-27 02:25:09,609][105692] Updated weights for policy 0, policy_version 1513838 (0.0006) [2023-12-27 02:25:09,676][105692] Updated weights for policy 0, policy_version 1513848 (0.0006) [2023-12-27 02:25:09,719][105620] Updated weights for policy 1, policy_version 1516402 (0.0009) [2023-12-27 02:25:09,740][105692] Updated weights for policy 0, policy_version 1513858 (0.0006) [2023-12-27 02:25:09,779][105620] Updated weights for policy 1, policy_version 1516412 (0.0008) [2023-12-27 02:25:09,850][105620] Updated weights for policy 1, policy_version 1516422 (0.0008) [2023-12-27 02:25:10,466][105692] Updated weights for policy 0, policy_version 1513868 (0.0007) [2023-12-27 02:25:10,532][105692] Updated weights for policy 0, policy_version 1513878 (0.0009) [2023-12-27 02:25:10,596][105692] Updated weights for policy 0, policy_version 1513888 (0.0009) [2023-12-27 02:25:10,622][105620] Updated weights for policy 1, policy_version 1516432 (0.0008) [2023-12-27 02:25:10,684][105620] Updated weights for policy 1, policy_version 1516442 (0.0007) [2023-12-27 02:25:10,737][105620] Updated weights for policy 1, policy_version 1516452 (0.0007) [2023-12-27 02:25:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19161.0). Total num frames: 775880704. Throughput: 0: 9868.8, 1: 9754.8. Samples: 775887968. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:25:11,062][104569] Avg episode reward: [(0, '8712.645'), (1, '8987.754')] [2023-12-27 02:25:11,324][105692] Updated weights for policy 0, policy_version 1513898 (0.0008) [2023-12-27 02:25:11,401][105692] Updated weights for policy 0, policy_version 1513908 (0.0008) [2023-12-27 02:25:11,449][105620] Updated weights for policy 1, policy_version 1516462 (0.0007) [2023-12-27 02:25:11,467][105692] Updated weights for policy 0, policy_version 1513918 (0.0009) [2023-12-27 02:25:11,514][105620] Updated weights for policy 1, policy_version 1516472 (0.0007) [2023-12-27 02:25:11,527][105692] Updated weights for policy 0, policy_version 1513928 (0.0010) [2023-12-27 02:25:11,574][105620] Updated weights for policy 1, policy_version 1516482 (0.0007) [2023-12-27 02:25:12,205][105692] Updated weights for policy 0, policy_version 1513938 (0.0008) [2023-12-27 02:25:12,272][105692] Updated weights for policy 0, policy_version 1513948 (0.0009) [2023-12-27 02:25:12,336][105692] Updated weights for policy 0, policy_version 1513958 (0.0009) [2023-12-27 02:25:12,343][105620] Updated weights for policy 1, policy_version 1516492 (0.0007) [2023-12-27 02:25:12,406][105620] Updated weights for policy 1, policy_version 1516502 (0.0008) [2023-12-27 02:25:12,469][105620] Updated weights for policy 1, policy_version 1516512 (0.0009) [2023-12-27 02:25:13,121][105692] Updated weights for policy 0, policy_version 1513968 (0.0009) [2023-12-27 02:25:13,172][105620] Updated weights for policy 1, policy_version 1516522 (0.0008) [2023-12-27 02:25:13,185][105692] Updated weights for policy 0, policy_version 1513978 (0.0009) [2023-12-27 02:25:13,228][105620] Updated weights for policy 1, policy_version 1516532 (0.0005) [2023-12-27 02:25:13,242][105692] Updated weights for policy 0, policy_version 1513988 (0.0009) [2023-12-27 02:25:13,288][105620] Updated weights for policy 1, policy_version 1516542 (0.0007) [2023-12-27 02:25:13,349][105620] Updated weights for policy 1, policy_version 1516552 (0.0005) [2023-12-27 02:25:13,982][105620] Updated weights for policy 1, policy_version 1516562 (0.0009) [2023-12-27 02:25:14,042][105620] Updated weights for policy 1, policy_version 1516573 (0.0010) [2023-12-27 02:25:14,042][105692] Updated weights for policy 0, policy_version 1513998 (0.0006) [2023-12-27 02:25:14,097][105620] Updated weights for policy 1, policy_version 1516583 (0.0009) [2023-12-27 02:25:14,100][105692] Updated weights for policy 0, policy_version 1514008 (0.0006) [2023-12-27 02:25:14,158][105692] Updated weights for policy 0, policy_version 1514018 (0.0006) [2023-12-27 02:25:14,745][105620] Updated weights for policy 1, policy_version 1516593 (0.0008) [2023-12-27 02:25:14,807][105620] Updated weights for policy 1, policy_version 1516603 (0.0008) [2023-12-27 02:25:14,862][105692] Updated weights for policy 0, policy_version 1514028 (0.0006) [2023-12-27 02:25:14,868][105620] Updated weights for policy 1, policy_version 1516613 (0.0010) [2023-12-27 02:25:14,929][105692] Updated weights for policy 0, policy_version 1514038 (0.0009) [2023-12-27 02:25:14,988][105692] Updated weights for policy 0, policy_version 1514048 (0.0009) [2023-12-27 02:25:15,634][105620] Updated weights for policy 1, policy_version 1516623 (0.0007) [2023-12-27 02:25:15,707][105620] Updated weights for policy 1, policy_version 1516633 (0.0005) [2023-12-27 02:25:15,767][105692] Updated weights for policy 0, policy_version 1514058 (0.0010) [2023-12-27 02:25:15,773][105620] Updated weights for policy 1, policy_version 1516643 (0.0006) [2023-12-27 02:25:15,818][105692] Updated weights for policy 0, policy_version 1514068 (0.0009) [2023-12-27 02:25:15,875][105692] Updated weights for policy 0, policy_version 1514079 (0.0009) [2023-12-27 02:25:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19188.7). Total num frames: 775979008. Throughput: 0: 9845.7, 1: 9697.2. Samples: 775944896. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:25:16,063][104569] Avg episode reward: [(0, '9171.902'), (1, '9082.372')] [2023-12-27 02:25:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001514088_387661824.pth... [2023-12-27 02:25:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001516648_388317184.pth... [2023-12-27 02:25:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001515528_388030464.pth [2023-12-27 02:25:16,080][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001512936_387366912.pth [2023-12-27 02:25:16,410][105620] Updated weights for policy 1, policy_version 1516653 (0.0007) [2023-12-27 02:25:16,456][105620] Updated weights for policy 1, policy_version 1516663 (0.0008) [2023-12-27 02:25:16,510][105620] Updated weights for policy 1, policy_version 1516673 (0.0009) [2023-12-27 02:25:16,654][105692] Updated weights for policy 0, policy_version 1514089 (0.0009) [2023-12-27 02:25:16,708][105692] Updated weights for policy 0, policy_version 1514099 (0.0009) [2023-12-27 02:25:16,756][105692] Updated weights for policy 0, policy_version 1514109 (0.0009) [2023-12-27 02:25:16,811][105692] Updated weights for policy 0, policy_version 1514119 (0.0009) [2023-12-27 02:25:17,261][105620] Updated weights for policy 1, policy_version 1516683 (0.0009) [2023-12-27 02:25:17,318][105620] Updated weights for policy 1, policy_version 1516693 (0.0008) [2023-12-27 02:25:17,378][105620] Updated weights for policy 1, policy_version 1516703 (0.0008) [2023-12-27 02:25:17,564][105692] Updated weights for policy 0, policy_version 1514129 (0.0009) [2023-12-27 02:25:17,618][105692] Updated weights for policy 0, policy_version 1514139 (0.0009) [2023-12-27 02:25:17,680][105692] Updated weights for policy 0, policy_version 1514149 (0.0009) [2023-12-27 02:25:18,121][105620] Updated weights for policy 1, policy_version 1516713 (0.0010) [2023-12-27 02:25:18,181][105620] Updated weights for policy 1, policy_version 1516723 (0.0008) [2023-12-27 02:25:18,246][105620] Updated weights for policy 1, policy_version 1516733 (0.0009) [2023-12-27 02:25:18,315][105620] Updated weights for policy 1, policy_version 1516743 (0.0009) [2023-12-27 02:25:18,412][105692] Updated weights for policy 0, policy_version 1514159 (0.0009) [2023-12-27 02:25:18,479][105692] Updated weights for policy 0, policy_version 1514169 (0.0009) [2023-12-27 02:25:18,538][105692] Updated weights for policy 0, policy_version 1514179 (0.0008) [2023-12-27 02:25:19,104][105620] Updated weights for policy 1, policy_version 1516753 (0.0008) [2023-12-27 02:25:19,154][105620] Updated weights for policy 1, policy_version 1516763 (0.0009) [2023-12-27 02:25:19,191][105692] Updated weights for policy 0, policy_version 1514189 (0.0006) [2023-12-27 02:25:19,219][105620] Updated weights for policy 1, policy_version 1516773 (0.0008) [2023-12-27 02:25:19,251][105692] Updated weights for policy 0, policy_version 1514199 (0.0008) [2023-12-27 02:25:19,314][105692] Updated weights for policy 0, policy_version 1514209 (0.0009) [2023-12-27 02:25:19,997][105620] Updated weights for policy 1, policy_version 1516783 (0.0009) [2023-12-27 02:25:20,062][105620] Updated weights for policy 1, policy_version 1516793 (0.0009) [2023-12-27 02:25:20,077][105692] Updated weights for policy 0, policy_version 1514219 (0.0008) [2023-12-27 02:25:20,121][105620] Updated weights for policy 1, policy_version 1516803 (0.0007) [2023-12-27 02:25:20,136][105692] Updated weights for policy 0, policy_version 1514229 (0.0006) [2023-12-27 02:25:20,197][105692] Updated weights for policy 0, policy_version 1514239 (0.0008) [2023-12-27 02:25:20,875][105620] Updated weights for policy 1, policy_version 1516813 (0.0009) [2023-12-27 02:25:20,930][105620] Updated weights for policy 1, policy_version 1516823 (0.0009) [2023-12-27 02:25:20,975][105692] Updated weights for policy 0, policy_version 1514249 (0.0010) [2023-12-27 02:25:20,986][105620] Updated weights for policy 1, policy_version 1516833 (0.0008) [2023-12-27 02:25:21,034][105692] Updated weights for policy 0, policy_version 1514259 (0.0008) [2023-12-27 02:25:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19161.0). Total num frames: 776069120. Throughput: 0: 9753.7, 1: 9605.2. Samples: 776058828. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:25:21,063][104569] Avg episode reward: [(0, '8621.457'), (1, '9172.642')] [2023-12-27 02:25:21,096][105692] Updated weights for policy 0, policy_version 1514269 (0.0008) [2023-12-27 02:25:21,162][105692] Updated weights for policy 0, policy_version 1514279 (0.0009) [2023-12-27 02:25:21,772][105620] Updated weights for policy 1, policy_version 1516843 (0.0008) [2023-12-27 02:25:21,829][105620] Updated weights for policy 1, policy_version 1516853 (0.0010) [2023-12-27 02:25:21,892][105620] Updated weights for policy 1, policy_version 1516863 (0.0009) [2023-12-27 02:25:21,974][105692] Updated weights for policy 0, policy_version 1514289 (0.0008) [2023-12-27 02:25:22,040][105692] Updated weights for policy 0, policy_version 1514299 (0.0009) [2023-12-27 02:25:22,108][105692] Updated weights for policy 0, policy_version 1514309 (0.0010) [2023-12-27 02:25:22,647][105620] Updated weights for policy 1, policy_version 1516873 (0.0008) [2023-12-27 02:25:22,713][105620] Updated weights for policy 1, policy_version 1516883 (0.0009) [2023-12-27 02:25:22,765][105620] Updated weights for policy 1, policy_version 1516893 (0.0009) [2023-12-27 02:25:22,820][105620] Updated weights for policy 1, policy_version 1516903 (0.0009) [2023-12-27 02:25:22,885][105692] Updated weights for policy 0, policy_version 1514319 (0.0009) [2023-12-27 02:25:22,939][105692] Updated weights for policy 0, policy_version 1514329 (0.0009) [2023-12-27 02:25:22,990][105692] Updated weights for policy 0, policy_version 1514339 (0.0009) [2023-12-27 02:25:23,511][105620] Updated weights for policy 1, policy_version 1516913 (0.0010) [2023-12-27 02:25:23,557][105620] Updated weights for policy 1, policy_version 1516923 (0.0006) [2023-12-27 02:25:23,610][105620] Updated weights for policy 1, policy_version 1516933 (0.0005) [2023-12-27 02:25:23,842][105692] Updated weights for policy 0, policy_version 1514349 (0.0008) [2023-12-27 02:25:23,894][105692] Updated weights for policy 0, policy_version 1514359 (0.0009) [2023-12-27 02:25:23,952][105692] Updated weights for policy 0, policy_version 1514369 (0.0008) [2023-12-27 02:25:24,285][105620] Updated weights for policy 1, policy_version 1516943 (0.0007) [2023-12-27 02:25:24,336][105620] Updated weights for policy 1, policy_version 1516953 (0.0009) [2023-12-27 02:25:24,392][105620] Updated weights for policy 1, policy_version 1516963 (0.0009) [2023-12-27 02:25:24,732][105692] Updated weights for policy 0, policy_version 1514379 (0.0009) [2023-12-27 02:25:24,797][105692] Updated weights for policy 0, policy_version 1514389 (0.0009) [2023-12-27 02:25:24,858][105692] Updated weights for policy 0, policy_version 1514399 (0.0009) [2023-12-27 02:25:25,140][105620] Updated weights for policy 1, policy_version 1516973 (0.0009) [2023-12-27 02:25:25,194][105620] Updated weights for policy 1, policy_version 1516983 (0.0008) [2023-12-27 02:25:25,257][105620] Updated weights for policy 1, policy_version 1516993 (0.0009) [2023-12-27 02:25:25,628][105692] Updated weights for policy 0, policy_version 1514409 (0.0009) [2023-12-27 02:25:25,682][105692] Updated weights for policy 0, policy_version 1514419 (0.0009) [2023-12-27 02:25:25,736][105692] Updated weights for policy 0, policy_version 1514429 (0.0009) [2023-12-27 02:25:25,795][105692] Updated weights for policy 0, policy_version 1514439 (0.0009) [2023-12-27 02:25:25,978][105620] Updated weights for policy 1, policy_version 1517003 (0.0008) [2023-12-27 02:25:26,043][105620] Updated weights for policy 1, policy_version 1517013 (0.0008) [2023-12-27 02:25:26,062][104569] Fps is (10 sec: 18022.7, 60 sec: 19387.7, 300 sec: 19133.2). Total num frames: 776159232. Throughput: 0: 9669.9, 1: 9576.3. Samples: 776169544. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:25:26,063][104569] Avg episode reward: [(0, '8348.098'), (1, '9172.379')] [2023-12-27 02:25:26,101][105620] Updated weights for policy 1, policy_version 1517023 (0.0008) [2023-12-27 02:25:26,572][105692] Updated weights for policy 0, policy_version 1514449 (0.0009) [2023-12-27 02:25:26,635][105692] Updated weights for policy 0, policy_version 1514459 (0.0009) [2023-12-27 02:25:26,696][105692] Updated weights for policy 0, policy_version 1514469 (0.0009) [2023-12-27 02:25:26,771][105620] Updated weights for policy 1, policy_version 1517033 (0.0008) [2023-12-27 02:25:26,820][105620] Updated weights for policy 1, policy_version 1517043 (0.0008) [2023-12-27 02:25:26,867][105620] Updated weights for policy 1, policy_version 1517053 (0.0009) [2023-12-27 02:25:26,922][105620] Updated weights for policy 1, policy_version 1517063 (0.0009) [2023-12-27 02:25:27,470][105692] Updated weights for policy 0, policy_version 1514479 (0.0009) [2023-12-27 02:25:27,524][105692] Updated weights for policy 0, policy_version 1514489 (0.0008) [2023-12-27 02:25:27,571][105692] Updated weights for policy 0, policy_version 1514499 (0.0009) [2023-12-27 02:25:27,662][105620] Updated weights for policy 1, policy_version 1517073 (0.0009) [2023-12-27 02:25:27,719][105620] Updated weights for policy 1, policy_version 1517083 (0.0009) [2023-12-27 02:25:27,780][105620] Updated weights for policy 1, policy_version 1517093 (0.0005) [2023-12-27 02:25:28,398][105620] Updated weights for policy 1, policy_version 1517103 (0.0008) [2023-12-27 02:25:28,405][105692] Updated weights for policy 0, policy_version 1514509 (0.0007) [2023-12-27 02:25:28,451][105620] Updated weights for policy 1, policy_version 1517113 (0.0006) [2023-12-27 02:25:28,465][105692] Updated weights for policy 0, policy_version 1514519 (0.0009) [2023-12-27 02:25:28,506][105620] Updated weights for policy 1, policy_version 1517123 (0.0006) [2023-12-27 02:25:28,524][105692] Updated weights for policy 0, policy_version 1514529 (0.0009) [2023-12-27 02:25:29,167][105620] Updated weights for policy 1, policy_version 1517133 (0.0007) [2023-12-27 02:25:29,215][105620] Updated weights for policy 1, policy_version 1517143 (0.0009) [2023-12-27 02:25:29,279][105620] Updated weights for policy 1, policy_version 1517153 (0.0009) [2023-12-27 02:25:29,301][105692] Updated weights for policy 0, policy_version 1514539 (0.0009) [2023-12-27 02:25:29,363][105692] Updated weights for policy 0, policy_version 1514549 (0.0007) [2023-12-27 02:25:29,428][105692] Updated weights for policy 0, policy_version 1514559 (0.0007) [2023-12-27 02:25:30,044][105620] Updated weights for policy 1, policy_version 1517163 (0.0008) [2023-12-27 02:25:30,116][105620] Updated weights for policy 1, policy_version 1517173 (0.0009) [2023-12-27 02:25:30,165][105620] Updated weights for policy 1, policy_version 1517183 (0.0008) [2023-12-27 02:25:30,183][105692] Updated weights for policy 0, policy_version 1514569 (0.0009) [2023-12-27 02:25:30,239][105692] Updated weights for policy 0, policy_version 1514579 (0.0009) [2023-12-27 02:25:30,297][105692] Updated weights for policy 0, policy_version 1514589 (0.0009) [2023-12-27 02:25:30,355][105692] Updated weights for policy 0, policy_version 1514599 (0.0009) [2023-12-27 02:25:30,918][105620] Updated weights for policy 1, policy_version 1517193 (0.0008) [2023-12-27 02:25:30,976][105620] Updated weights for policy 1, policy_version 1517203 (0.0007) [2023-12-27 02:25:31,032][105620] Updated weights for policy 1, policy_version 1517213 (0.0008) [2023-12-27 02:25:31,046][105692] Updated weights for policy 0, policy_version 1514609 (0.0009) [2023-12-27 02:25:31,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19114.7, 300 sec: 19133.2). Total num frames: 776249344. Throughput: 0: 9580.1, 1: 9650.8. Samples: 776226936. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:25:31,062][104569] Avg episode reward: [(0, '8807.999'), (1, '9173.965')] [2023-12-27 02:25:31,101][105620] Updated weights for policy 1, policy_version 1517223 (0.0008) [2023-12-27 02:25:31,105][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001517224_388464640.pth... [2023-12-27 02:25:31,108][105692] Updated weights for policy 0, policy_version 1514619 (0.0006) [2023-12-27 02:25:31,109][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001516072_388169728.pth [2023-12-27 02:25:31,171][105692] Updated weights for policy 0, policy_version 1514629 (0.0009) [2023-12-27 02:25:31,182][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001514632_387801088.pth... [2023-12-27 02:25:31,186][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001513544_387522560.pth [2023-12-27 02:25:31,872][105620] Updated weights for policy 1, policy_version 1517233 (0.0005) [2023-12-27 02:25:31,937][105692] Updated weights for policy 0, policy_version 1514639 (0.0009) [2023-12-27 02:25:31,940][105620] Updated weights for policy 1, policy_version 1517243 (0.0005) [2023-12-27 02:25:31,994][105620] Updated weights for policy 1, policy_version 1517253 (0.0006) [2023-12-27 02:25:31,995][105692] Updated weights for policy 0, policy_version 1514649 (0.0009) [2023-12-27 02:25:32,046][105692] Updated weights for policy 0, policy_version 1514659 (0.0008) [2023-12-27 02:25:32,650][105620] Updated weights for policy 1, policy_version 1517263 (0.0007) [2023-12-27 02:25:32,707][105620] Updated weights for policy 1, policy_version 1517273 (0.0008) [2023-12-27 02:25:32,752][105620] Updated weights for policy 1, policy_version 1517283 (0.0010) [2023-12-27 02:25:32,869][105692] Updated weights for policy 0, policy_version 1514669 (0.0007) [2023-12-27 02:25:32,935][105692] Updated weights for policy 0, policy_version 1514679 (0.0006) [2023-12-27 02:25:33,002][105692] Updated weights for policy 0, policy_version 1514689 (0.0005) [2023-12-27 02:25:33,466][105620] Updated weights for policy 1, policy_version 1517293 (0.0010) [2023-12-27 02:25:33,518][105620] Updated weights for policy 1, policy_version 1517303 (0.0010) [2023-12-27 02:25:33,569][105620] Updated weights for policy 1, policy_version 1517313 (0.0010) [2023-12-27 02:25:33,582][105692] Updated weights for policy 0, policy_version 1514699 (0.0006) [2023-12-27 02:25:33,638][105692] Updated weights for policy 0, policy_version 1514709 (0.0006) [2023-12-27 02:25:33,698][105692] Updated weights for policy 0, policy_version 1514719 (0.0009) [2023-12-27 02:25:34,295][105620] Updated weights for policy 1, policy_version 1517323 (0.0010) [2023-12-27 02:25:34,351][105620] Updated weights for policy 1, policy_version 1517333 (0.0011) [2023-12-27 02:25:34,406][105620] Updated weights for policy 1, policy_version 1517343 (0.0011) [2023-12-27 02:25:34,455][105692] Updated weights for policy 0, policy_version 1514729 (0.0008) [2023-12-27 02:25:34,516][105692] Updated weights for policy 0, policy_version 1514739 (0.0008) [2023-12-27 02:25:34,573][105692] Updated weights for policy 0, policy_version 1514749 (0.0008) [2023-12-27 02:25:34,618][105692] Updated weights for policy 0, policy_version 1514759 (0.0008) [2023-12-27 02:25:35,082][105620] Updated weights for policy 1, policy_version 1517353 (0.0010) [2023-12-27 02:25:35,140][105620] Updated weights for policy 1, policy_version 1517363 (0.0005) [2023-12-27 02:25:35,206][105620] Updated weights for policy 1, policy_version 1517373 (0.0006) [2023-12-27 02:25:35,257][105620] Updated weights for policy 1, policy_version 1517383 (0.0007) [2023-12-27 02:25:35,469][105692] Updated weights for policy 0, policy_version 1514769 (0.0009) [2023-12-27 02:25:35,525][105692] Updated weights for policy 0, policy_version 1514779 (0.0009) [2023-12-27 02:25:35,578][105692] Updated weights for policy 0, policy_version 1514789 (0.0009) [2023-12-27 02:25:35,859][105620] Updated weights for policy 1, policy_version 1517393 (0.0009) [2023-12-27 02:25:35,905][105620] Updated weights for policy 1, policy_version 1517403 (0.0006) [2023-12-27 02:25:35,958][105620] Updated weights for policy 1, policy_version 1517413 (0.0005) [2023-12-27 02:25:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19161.0). Total num frames: 776355840. Throughput: 0: 9456.3, 1: 9704.4. Samples: 776341196. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:25:36,062][104569] Avg episode reward: [(0, '8806.581'), (1, '9169.834')] [2023-12-27 02:25:36,419][105692] Updated weights for policy 0, policy_version 1514799 (0.0009) [2023-12-27 02:25:36,488][105692] Updated weights for policy 0, policy_version 1514809 (0.0010) [2023-12-27 02:25:36,563][105692] Updated weights for policy 0, policy_version 1514819 (0.0009) [2023-12-27 02:25:36,611][105620] Updated weights for policy 1, policy_version 1517423 (0.0005) [2023-12-27 02:25:36,660][105620] Updated weights for policy 1, policy_version 1517433 (0.0005) [2023-12-27 02:25:36,706][105620] Updated weights for policy 1, policy_version 1517443 (0.0007) [2023-12-27 02:25:37,355][105692] Updated weights for policy 0, policy_version 1514829 (0.0009) [2023-12-27 02:25:37,407][105692] Updated weights for policy 0, policy_version 1514839 (0.0009) [2023-12-27 02:25:37,456][105620] Updated weights for policy 1, policy_version 1517453 (0.0008) [2023-12-27 02:25:37,466][105692] Updated weights for policy 0, policy_version 1514849 (0.0007) [2023-12-27 02:25:37,507][105620] Updated weights for policy 1, policy_version 1517463 (0.0006) [2023-12-27 02:25:37,559][105620] Updated weights for policy 1, policy_version 1517473 (0.0009) [2023-12-27 02:25:38,268][105620] Updated weights for policy 1, policy_version 1517483 (0.0009) [2023-12-27 02:25:38,282][105692] Updated weights for policy 0, policy_version 1514859 (0.0008) [2023-12-27 02:25:38,322][105620] Updated weights for policy 1, policy_version 1517493 (0.0006) [2023-12-27 02:25:38,345][105692] Updated weights for policy 0, policy_version 1514869 (0.0008) [2023-12-27 02:25:38,388][105620] Updated weights for policy 1, policy_version 1517503 (0.0008) [2023-12-27 02:25:38,419][105692] Updated weights for policy 0, policy_version 1514879 (0.0006) [2023-12-27 02:25:39,048][105692] Updated weights for policy 0, policy_version 1514889 (0.0007) [2023-12-27 02:25:39,072][105620] Updated weights for policy 1, policy_version 1517513 (0.0008) [2023-12-27 02:25:39,098][105692] Updated weights for policy 0, policy_version 1514899 (0.0006) [2023-12-27 02:25:39,132][105620] Updated weights for policy 1, policy_version 1517523 (0.0010) [2023-12-27 02:25:39,152][105692] Updated weights for policy 0, policy_version 1514909 (0.0005) [2023-12-27 02:25:39,185][105620] Updated weights for policy 1, policy_version 1517533 (0.0007) [2023-12-27 02:25:39,205][105692] Updated weights for policy 0, policy_version 1514919 (0.0006) [2023-12-27 02:25:39,255][105620] Updated weights for policy 1, policy_version 1517543 (0.0007) [2023-12-27 02:25:39,885][105692] Updated weights for policy 0, policy_version 1514929 (0.0009) [2023-12-27 02:25:39,950][105692] Updated weights for policy 0, policy_version 1514939 (0.0008) [2023-12-27 02:25:39,999][105620] Updated weights for policy 1, policy_version 1517553 (0.0006) [2023-12-27 02:25:40,017][105692] Updated weights for policy 0, policy_version 1514949 (0.0009) [2023-12-27 02:25:40,059][105620] Updated weights for policy 1, policy_version 1517563 (0.0007) [2023-12-27 02:25:40,112][105620] Updated weights for policy 1, policy_version 1517573 (0.0011) [2023-12-27 02:25:40,721][105692] Updated weights for policy 0, policy_version 1514959 (0.0009) [2023-12-27 02:25:40,770][105692] Updated weights for policy 0, policy_version 1514969 (0.0011) [2023-12-27 02:25:40,819][105692] Updated weights for policy 0, policy_version 1514979 (0.0010) [2023-12-27 02:25:40,895][105620] Updated weights for policy 1, policy_version 1517583 (0.0010) [2023-12-27 02:25:40,941][105620] Updated weights for policy 1, policy_version 1517593 (0.0006) [2023-12-27 02:25:41,002][105620] Updated weights for policy 1, policy_version 1517603 (0.0006) [2023-12-27 02:25:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19251.2, 300 sec: 19133.2). Total num frames: 776454144. Throughput: 0: 9412.6, 1: 9737.4. Samples: 776455776. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:25:41,062][104569] Avg episode reward: [(0, '8627.872'), (1, '9077.485')] [2023-12-27 02:25:41,604][105692] Updated weights for policy 0, policy_version 1514989 (0.0009) [2023-12-27 02:25:41,668][105692] Updated weights for policy 0, policy_version 1514999 (0.0008) [2023-12-27 02:25:41,740][105692] Updated weights for policy 0, policy_version 1515009 (0.0009) [2023-12-27 02:25:41,778][105620] Updated weights for policy 1, policy_version 1517613 (0.0008) [2023-12-27 02:25:41,838][105620] Updated weights for policy 1, policy_version 1517623 (0.0011) [2023-12-27 02:25:41,902][105620] Updated weights for policy 1, policy_version 1517633 (0.0011) [2023-12-27 02:25:42,499][105692] Updated weights for policy 0, policy_version 1515019 (0.0008) [2023-12-27 02:25:42,561][105692] Updated weights for policy 0, policy_version 1515029 (0.0009) [2023-12-27 02:25:42,618][105692] Updated weights for policy 0, policy_version 1515039 (0.0008) [2023-12-27 02:25:42,645][105620] Updated weights for policy 1, policy_version 1517643 (0.0009) [2023-12-27 02:25:42,709][105620] Updated weights for policy 1, policy_version 1517653 (0.0006) [2023-12-27 02:25:42,778][105620] Updated weights for policy 1, policy_version 1517663 (0.0006) [2023-12-27 02:25:43,287][105692] Updated weights for policy 0, policy_version 1515049 (0.0008) [2023-12-27 02:25:43,338][105692] Updated weights for policy 0, policy_version 1515059 (0.0009) [2023-12-27 02:25:43,386][105692] Updated weights for policy 0, policy_version 1515069 (0.0009) [2023-12-27 02:25:43,434][105692] Updated weights for policy 0, policy_version 1515079 (0.0009) [2023-12-27 02:25:43,453][105620] Updated weights for policy 1, policy_version 1517673 (0.0009) [2023-12-27 02:25:43,510][105620] Updated weights for policy 1, policy_version 1517683 (0.0009) [2023-12-27 02:25:43,556][105620] Updated weights for policy 1, policy_version 1517693 (0.0009) [2023-12-27 02:25:43,623][105620] Updated weights for policy 1, policy_version 1517703 (0.0007) [2023-12-27 02:25:44,188][105620] Updated weights for policy 1, policy_version 1517713 (0.0006) [2023-12-27 02:25:44,247][105620] Updated weights for policy 1, policy_version 1517723 (0.0005) [2023-12-27 02:25:44,278][105692] Updated weights for policy 0, policy_version 1515089 (0.0009) [2023-12-27 02:25:44,304][105620] Updated weights for policy 1, policy_version 1517733 (0.0007) [2023-12-27 02:25:44,334][105692] Updated weights for policy 0, policy_version 1515099 (0.0009) [2023-12-27 02:25:44,387][105692] Updated weights for policy 0, policy_version 1515109 (0.0009) [2023-12-27 02:25:45,021][105692] Updated weights for policy 0, policy_version 1515119 (0.0008) [2023-12-27 02:25:45,076][105692] Updated weights for policy 0, policy_version 1515129 (0.0010) [2023-12-27 02:25:45,103][105620] Updated weights for policy 1, policy_version 1517743 (0.0008) [2023-12-27 02:25:45,129][105692] Updated weights for policy 0, policy_version 1515139 (0.0007) [2023-12-27 02:25:45,161][105620] Updated weights for policy 1, policy_version 1517753 (0.0006) [2023-12-27 02:25:45,214][105620] Updated weights for policy 1, policy_version 1517763 (0.0009) [2023-12-27 02:25:45,860][105692] Updated weights for policy 0, policy_version 1515149 (0.0008) [2023-12-27 02:25:45,906][105692] Updated weights for policy 0, policy_version 1515159 (0.0009) [2023-12-27 02:25:45,934][105620] Updated weights for policy 1, policy_version 1517773 (0.0007) [2023-12-27 02:25:45,958][105692] Updated weights for policy 0, policy_version 1515169 (0.0009) [2023-12-27 02:25:45,988][105620] Updated weights for policy 1, policy_version 1517783 (0.0008) [2023-12-27 02:25:46,049][105620] Updated weights for policy 1, policy_version 1517793 (0.0009) [2023-12-27 02:25:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19133.2). Total num frames: 776544256. Throughput: 0: 9285.4, 1: 9693.9. Samples: 776513320. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:25:46,062][104569] Avg episode reward: [(0, '8721.979'), (1, '9259.002')] [2023-12-27 02:25:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001515176_387940352.pth... [2023-12-27 02:25:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001514088_387661824.pth [2023-12-27 02:25:46,087][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001517800_388612096.pth... [2023-12-27 02:25:46,091][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001516648_388317184.pth [2023-12-27 02:25:46,714][105692] Updated weights for policy 0, policy_version 1515179 (0.0009) [2023-12-27 02:25:46,760][105692] Updated weights for policy 0, policy_version 1515189 (0.0008) [2023-12-27 02:25:46,798][105620] Updated weights for policy 1, policy_version 1517803 (0.0008) [2023-12-27 02:25:46,812][105692] Updated weights for policy 0, policy_version 1515199 (0.0008) [2023-12-27 02:25:46,853][105620] Updated weights for policy 1, policy_version 1517813 (0.0006) [2023-12-27 02:25:46,893][105586] KL-divergence is very high: 119.1688 [2023-12-27 02:25:46,900][105620] Updated weights for policy 1, policy_version 1517823 (0.0009) [2023-12-27 02:25:46,931][105586] KL-divergence is very high: 132.6332 [2023-12-27 02:25:47,533][105692] Updated weights for policy 0, policy_version 1515209 (0.0007) [2023-12-27 02:25:47,593][105692] Updated weights for policy 0, policy_version 1515219 (0.0006) [2023-12-27 02:25:47,658][105692] Updated weights for policy 0, policy_version 1515229 (0.0007) [2023-12-27 02:25:47,714][105620] Updated weights for policy 1, policy_version 1517834 (0.0008) [2023-12-27 02:25:47,716][105692] Updated weights for policy 0, policy_version 1515239 (0.0010) [2023-12-27 02:25:47,775][105620] Updated weights for policy 1, policy_version 1517844 (0.0008) [2023-12-27 02:25:47,825][105620] Updated weights for policy 1, policy_version 1517854 (0.0008) [2023-12-27 02:25:47,872][105620] Updated weights for policy 1, policy_version 1517864 (0.0008) [2023-12-27 02:25:48,352][105692] Updated weights for policy 0, policy_version 1515249 (0.0007) [2023-12-27 02:25:48,401][105692] Updated weights for policy 0, policy_version 1515259 (0.0010) [2023-12-27 02:25:48,455][105692] Updated weights for policy 0, policy_version 1515269 (0.0008) [2023-12-27 02:25:48,637][105620] Updated weights for policy 1, policy_version 1517874 (0.0010) [2023-12-27 02:25:48,699][105620] Updated weights for policy 1, policy_version 1517884 (0.0009) [2023-12-27 02:25:48,756][105620] Updated weights for policy 1, policy_version 1517894 (0.0008) [2023-12-27 02:25:49,046][105692] Updated weights for policy 0, policy_version 1515279 (0.0007) [2023-12-27 02:25:49,103][105692] Updated weights for policy 0, policy_version 1515289 (0.0010) [2023-12-27 02:25:49,155][105692] Updated weights for policy 0, policy_version 1515299 (0.0010) [2023-12-27 02:25:49,563][105620] Updated weights for policy 1, policy_version 1517904 (0.0008) [2023-12-27 02:25:49,633][105620] Updated weights for policy 1, policy_version 1517914 (0.0008) [2023-12-27 02:25:49,695][105620] Updated weights for policy 1, policy_version 1517924 (0.0008) [2023-12-27 02:25:49,874][105692] Updated weights for policy 0, policy_version 1515309 (0.0011) [2023-12-27 02:25:49,942][105692] Updated weights for policy 0, policy_version 1515319 (0.0011) [2023-12-27 02:25:50,012][105692] Updated weights for policy 0, policy_version 1515329 (0.0010) [2023-12-27 02:25:50,465][105620] Updated weights for policy 1, policy_version 1517934 (0.0008) [2023-12-27 02:25:50,521][105620] Updated weights for policy 1, policy_version 1517944 (0.0009) [2023-12-27 02:25:50,589][105620] Updated weights for policy 1, policy_version 1517954 (0.0008) [2023-12-27 02:25:50,759][105692] Updated weights for policy 0, policy_version 1515339 (0.0010) [2023-12-27 02:25:50,822][105692] Updated weights for policy 0, policy_version 1515349 (0.0011) [2023-12-27 02:25:50,870][105692] Updated weights for policy 0, policy_version 1515359 (0.0010) [2023-12-27 02:25:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19133.2). Total num frames: 776642560. Throughput: 0: 9319.6, 1: 9655.4. Samples: 776628828. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:25:51,062][104569] Avg episode reward: [(0, '8902.390'), (1, '8985.239')] [2023-12-27 02:25:51,366][105620] Updated weights for policy 1, policy_version 1517964 (0.0008) [2023-12-27 02:25:51,437][105620] Updated weights for policy 1, policy_version 1517974 (0.0009) [2023-12-27 02:25:51,500][105620] Updated weights for policy 1, policy_version 1517984 (0.0010) [2023-12-27 02:25:51,587][105692] Updated weights for policy 0, policy_version 1515369 (0.0010) [2023-12-27 02:25:51,649][105692] Updated weights for policy 0, policy_version 1515379 (0.0008) [2023-12-27 02:25:51,715][105692] Updated weights for policy 0, policy_version 1515389 (0.0008) [2023-12-27 02:25:51,779][105692] Updated weights for policy 0, policy_version 1515399 (0.0007) [2023-12-27 02:25:52,316][105620] Updated weights for policy 1, policy_version 1517994 (0.0010) [2023-12-27 02:25:52,378][105620] Updated weights for policy 1, policy_version 1518004 (0.0009) [2023-12-27 02:25:52,435][105692] Updated weights for policy 0, policy_version 1515409 (0.0008) [2023-12-27 02:25:52,437][105620] Updated weights for policy 1, policy_version 1518014 (0.0009) [2023-12-27 02:25:52,499][105620] Updated weights for policy 1, policy_version 1518024 (0.0008) [2023-12-27 02:25:52,501][105692] Updated weights for policy 0, policy_version 1515419 (0.0006) [2023-12-27 02:25:52,563][105692] Updated weights for policy 0, policy_version 1515429 (0.0009) [2023-12-27 02:25:53,252][105620] Updated weights for policy 1, policy_version 1518034 (0.0006) [2023-12-27 02:25:53,314][105620] Updated weights for policy 1, policy_version 1518044 (0.0009) [2023-12-27 02:25:53,330][105692] Updated weights for policy 0, policy_version 1515439 (0.0007) [2023-12-27 02:25:53,366][105620] Updated weights for policy 1, policy_version 1518054 (0.0007) [2023-12-27 02:25:53,377][105692] Updated weights for policy 0, policy_version 1515449 (0.0006) [2023-12-27 02:25:53,424][105692] Updated weights for policy 0, policy_version 1515459 (0.0009) [2023-12-27 02:25:54,053][105692] Updated weights for policy 0, policy_version 1515469 (0.0009) [2023-12-27 02:25:54,105][105692] Updated weights for policy 0, policy_version 1515479 (0.0009) [2023-12-27 02:25:54,145][105620] Updated weights for policy 1, policy_version 1518064 (0.0008) [2023-12-27 02:25:54,156][105692] Updated weights for policy 0, policy_version 1515489 (0.0008) [2023-12-27 02:25:54,230][105620] Updated weights for policy 1, policy_version 1518074 (0.0009) [2023-12-27 02:25:54,284][105620] Updated weights for policy 1, policy_version 1518084 (0.0008) [2023-12-27 02:25:54,799][105692] Updated weights for policy 0, policy_version 1515499 (0.0007) [2023-12-27 02:25:54,861][105692] Updated weights for policy 0, policy_version 1515509 (0.0010) [2023-12-27 02:25:54,919][105692] Updated weights for policy 0, policy_version 1515519 (0.0009) [2023-12-27 02:25:55,084][105620] Updated weights for policy 1, policy_version 1518094 (0.0008) [2023-12-27 02:25:55,151][105620] Updated weights for policy 1, policy_version 1518104 (0.0008) [2023-12-27 02:25:55,220][105620] Updated weights for policy 1, policy_version 1518114 (0.0008) [2023-12-27 02:25:55,636][105692] Updated weights for policy 0, policy_version 1515529 (0.0007) [2023-12-27 02:25:55,697][105692] Updated weights for policy 0, policy_version 1515539 (0.0010) [2023-12-27 02:25:55,761][105692] Updated weights for policy 0, policy_version 1515549 (0.0010) [2023-12-27 02:25:55,766][105585] KL-divergence is very high: 147.9151 [2023-12-27 02:25:55,816][105585] KL-divergence is very high: 142.5533 [2023-12-27 02:25:55,822][105692] Updated weights for policy 0, policy_version 1515559 (0.0010) [2023-12-27 02:25:55,928][105620] Updated weights for policy 1, policy_version 1518124 (0.0007) [2023-12-27 02:25:55,997][105620] Updated weights for policy 1, policy_version 1518134 (0.0010) [2023-12-27 02:25:56,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19114.6, 300 sec: 19105.4). Total num frames: 776732672. Throughput: 0: 9414.5, 1: 9579.2. Samples: 776742688. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:25:56,063][104569] Avg episode reward: [(0, '8625.411'), (1, '8985.436')] [2023-12-27 02:25:56,075][105620] Updated weights for policy 1, policy_version 1518144 (0.0010) [2023-12-27 02:25:56,512][105692] Updated weights for policy 0, policy_version 1515569 (0.0006) [2023-12-27 02:25:56,567][105692] Updated weights for policy 0, policy_version 1515579 (0.0005) [2023-12-27 02:25:56,617][105692] Updated weights for policy 0, policy_version 1515589 (0.0008) [2023-12-27 02:25:56,839][105620] Updated weights for policy 1, policy_version 1518154 (0.0009) [2023-12-27 02:25:56,896][105620] Updated weights for policy 1, policy_version 1518164 (0.0009) [2023-12-27 02:25:56,956][105620] Updated weights for policy 1, policy_version 1518174 (0.0009) [2023-12-27 02:25:57,228][105692] Updated weights for policy 0, policy_version 1515599 (0.0009) [2023-12-27 02:25:57,278][105692] Updated weights for policy 0, policy_version 1515609 (0.0009) [2023-12-27 02:25:57,328][105692] Updated weights for policy 0, policy_version 1515619 (0.0008) [2023-12-27 02:25:57,729][105620] Updated weights for policy 1, policy_version 1518185 (0.0010) [2023-12-27 02:25:57,777][105620] Updated weights for policy 1, policy_version 1518195 (0.0009) [2023-12-27 02:25:57,836][105620] Updated weights for policy 1, policy_version 1518205 (0.0009) [2023-12-27 02:25:57,898][105620] Updated weights for policy 1, policy_version 1518215 (0.0010) [2023-12-27 02:25:58,040][105692] Updated weights for policy 0, policy_version 1515629 (0.0007) [2023-12-27 02:25:58,095][105692] Updated weights for policy 0, policy_version 1515639 (0.0005) [2023-12-27 02:25:58,154][105692] Updated weights for policy 0, policy_version 1515649 (0.0006) [2023-12-27 02:25:58,756][105620] Updated weights for policy 1, policy_version 1518225 (0.0007) [2023-12-27 02:25:58,825][105620] Updated weights for policy 1, policy_version 1518235 (0.0006) [2023-12-27 02:25:58,887][105620] Updated weights for policy 1, policy_version 1518245 (0.0008) [2023-12-27 02:25:58,913][105692] Updated weights for policy 0, policy_version 1515659 (0.0008) [2023-12-27 02:25:58,971][105692] Updated weights for policy 0, policy_version 1515669 (0.0008) [2023-12-27 02:25:59,037][105692] Updated weights for policy 0, policy_version 1515679 (0.0009) [2023-12-27 02:25:59,584][105620] Updated weights for policy 1, policy_version 1518255 (0.0006) [2023-12-27 02:25:59,638][105620] Updated weights for policy 1, policy_version 1518265 (0.0006) [2023-12-27 02:25:59,692][105620] Updated weights for policy 1, policy_version 1518275 (0.0009) [2023-12-27 02:25:59,835][105692] Updated weights for policy 0, policy_version 1515689 (0.0009) [2023-12-27 02:25:59,901][105692] Updated weights for policy 0, policy_version 1515699 (0.0010) [2023-12-27 02:25:59,960][105692] Updated weights for policy 0, policy_version 1515709 (0.0008) [2023-12-27 02:26:00,014][105692] Updated weights for policy 0, policy_version 1515719 (0.0010) [2023-12-27 02:26:00,393][105620] Updated weights for policy 1, policy_version 1518285 (0.0008) [2023-12-27 02:26:00,444][105620] Updated weights for policy 1, policy_version 1518295 (0.0005) [2023-12-27 02:26:00,494][105620] Updated weights for policy 1, policy_version 1518305 (0.0005) [2023-12-27 02:26:00,786][105692] Updated weights for policy 0, policy_version 1515729 (0.0010) [2023-12-27 02:26:00,854][105692] Updated weights for policy 0, policy_version 1515739 (0.0009) [2023-12-27 02:26:00,921][105692] Updated weights for policy 0, policy_version 1515749 (0.0007) [2023-12-27 02:26:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19105.4). Total num frames: 776830976. Throughput: 0: 9485.8, 1: 9497.7. Samples: 776799148. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:26:01,062][104569] Avg episode reward: [(0, '8170.724'), (1, '9351.131')] [2023-12-27 02:26:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001518312_388743168.pth... [2023-12-27 02:26:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001515752_388087808.pth... [2023-12-27 02:26:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001517224_388464640.pth [2023-12-27 02:26:01,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001514632_387801088.pth [2023-12-27 02:26:01,128][105620] Updated weights for policy 1, policy_version 1518315 (0.0006) [2023-12-27 02:26:01,192][105620] Updated weights for policy 1, policy_version 1518325 (0.0008) [2023-12-27 02:26:01,257][105620] Updated weights for policy 1, policy_version 1518335 (0.0008) [2023-12-27 02:26:01,593][105692] Updated weights for policy 0, policy_version 1515759 (0.0007) [2023-12-27 02:26:01,657][105692] Updated weights for policy 0, policy_version 1515769 (0.0009) [2023-12-27 02:26:01,721][105692] Updated weights for policy 0, policy_version 1515779 (0.0009) [2023-12-27 02:26:01,978][105620] Updated weights for policy 1, policy_version 1518345 (0.0008) [2023-12-27 02:26:02,026][105620] Updated weights for policy 1, policy_version 1518355 (0.0010) [2023-12-27 02:26:02,091][105620] Updated weights for policy 1, policy_version 1518365 (0.0010) [2023-12-27 02:26:02,139][105620] Updated weights for policy 1, policy_version 1518375 (0.0010) [2023-12-27 02:26:02,329][105692] Updated weights for policy 0, policy_version 1515789 (0.0007) [2023-12-27 02:26:02,389][105692] Updated weights for policy 0, policy_version 1515799 (0.0008) [2023-12-27 02:26:02,441][105692] Updated weights for policy 0, policy_version 1515809 (0.0008) [2023-12-27 02:26:02,885][105620] Updated weights for policy 1, policy_version 1518385 (0.0011) [2023-12-27 02:26:02,942][105620] Updated weights for policy 1, policy_version 1518395 (0.0011) [2023-12-27 02:26:02,994][105620] Updated weights for policy 1, policy_version 1518405 (0.0010) [2023-12-27 02:26:03,141][105692] Updated weights for policy 0, policy_version 1515819 (0.0007) [2023-12-27 02:26:03,195][105692] Updated weights for policy 0, policy_version 1515829 (0.0005) [2023-12-27 02:26:03,244][105692] Updated weights for policy 0, policy_version 1515839 (0.0005) [2023-12-27 02:26:03,712][105620] Updated weights for policy 1, policy_version 1518415 (0.0009) [2023-12-27 02:26:03,772][105620] Updated weights for policy 1, policy_version 1518425 (0.0010) [2023-12-27 02:26:03,825][105620] Updated weights for policy 1, policy_version 1518435 (0.0009) [2023-12-27 02:26:03,963][105692] Updated weights for policy 0, policy_version 1515850 (0.0008) [2023-12-27 02:26:04,018][105692] Updated weights for policy 0, policy_version 1515860 (0.0009) [2023-12-27 02:26:04,074][105692] Updated weights for policy 0, policy_version 1515870 (0.0007) [2023-12-27 02:26:04,137][105692] Updated weights for policy 0, policy_version 1515880 (0.0009) [2023-12-27 02:26:04,627][105620] Updated weights for policy 1, policy_version 1518445 (0.0010) [2023-12-27 02:26:04,682][105620] Updated weights for policy 1, policy_version 1518455 (0.0010) [2023-12-27 02:26:04,734][105620] Updated weights for policy 1, policy_version 1518465 (0.0010) [2023-12-27 02:26:04,901][105692] Updated weights for policy 0, policy_version 1515890 (0.0010) [2023-12-27 02:26:04,954][105692] Updated weights for policy 0, policy_version 1515900 (0.0011) [2023-12-27 02:26:05,008][105692] Updated weights for policy 0, policy_version 1515911 (0.0011) [2023-12-27 02:26:05,314][105620] Updated weights for policy 1, policy_version 1518475 (0.0009) [2023-12-27 02:26:05,366][105620] Updated weights for policy 1, policy_version 1518485 (0.0005) [2023-12-27 02:26:05,416][105620] Updated weights for policy 1, policy_version 1518495 (0.0005) [2023-12-27 02:26:05,888][105692] Updated weights for policy 0, policy_version 1515921 (0.0007) [2023-12-27 02:26:05,944][105692] Updated weights for policy 0, policy_version 1515931 (0.0008) [2023-12-27 02:26:06,000][105692] Updated weights for policy 0, policy_version 1515941 (0.0008) [2023-12-27 02:26:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19114.6, 300 sec: 19105.4). Total num frames: 776929280. Throughput: 0: 9499.4, 1: 9544.9. Samples: 776915828. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:26:06,064][104569] Avg episode reward: [(0, '8082.815'), (1, '9167.706')] [2023-12-27 02:26:06,095][105620] Updated weights for policy 1, policy_version 1518505 (0.0006) [2023-12-27 02:26:06,157][105620] Updated weights for policy 1, policy_version 1518515 (0.0010) [2023-12-27 02:26:06,216][105620] Updated weights for policy 1, policy_version 1518525 (0.0010) [2023-12-27 02:26:06,272][105620] Updated weights for policy 1, policy_version 1518535 (0.0010) [2023-12-27 02:26:06,671][105692] Updated weights for policy 0, policy_version 1515951 (0.0008) [2023-12-27 02:26:06,725][105692] Updated weights for policy 0, policy_version 1515961 (0.0006) [2023-12-27 02:26:06,788][105692] Updated weights for policy 0, policy_version 1515971 (0.0005) [2023-12-27 02:26:07,020][105620] Updated weights for policy 1, policy_version 1518545 (0.0010) [2023-12-27 02:26:07,075][105620] Updated weights for policy 1, policy_version 1518555 (0.0010) [2023-12-27 02:26:07,128][105620] Updated weights for policy 1, policy_version 1518565 (0.0008) [2023-12-27 02:26:07,531][105692] Updated weights for policy 0, policy_version 1515981 (0.0008) [2023-12-27 02:26:07,589][105692] Updated weights for policy 0, policy_version 1515991 (0.0008) [2023-12-27 02:26:07,645][105692] Updated weights for policy 0, policy_version 1516001 (0.0008) [2023-12-27 02:26:07,818][105620] Updated weights for policy 1, policy_version 1518575 (0.0009) [2023-12-27 02:26:07,882][105620] Updated weights for policy 1, policy_version 1518585 (0.0010) [2023-12-27 02:26:07,934][105620] Updated weights for policy 1, policy_version 1518595 (0.0009) [2023-12-27 02:26:08,444][105692] Updated weights for policy 0, policy_version 1516011 (0.0008) [2023-12-27 02:26:08,511][105692] Updated weights for policy 0, policy_version 1516021 (0.0009) [2023-12-27 02:26:08,575][105692] Updated weights for policy 0, policy_version 1516031 (0.0009) [2023-12-27 02:26:08,680][105620] Updated weights for policy 1, policy_version 1518605 (0.0010) [2023-12-27 02:26:08,747][105620] Updated weights for policy 1, policy_version 1518615 (0.0010) [2023-12-27 02:26:08,811][105620] Updated weights for policy 1, policy_version 1518625 (0.0008) [2023-12-27 02:26:09,290][105692] Updated weights for policy 0, policy_version 1516041 (0.0008) [2023-12-27 02:26:09,361][105692] Updated weights for policy 0, policy_version 1516051 (0.0010) [2023-12-27 02:26:09,428][105692] Updated weights for policy 0, policy_version 1516061 (0.0010) [2023-12-27 02:26:09,489][105692] Updated weights for policy 0, policy_version 1516071 (0.0009) [2023-12-27 02:26:09,593][105620] Updated weights for policy 1, policy_version 1518635 (0.0009) [2023-12-27 02:26:09,655][105620] Updated weights for policy 1, policy_version 1518645 (0.0009) [2023-12-27 02:26:09,717][105620] Updated weights for policy 1, policy_version 1518655 (0.0009) [2023-12-27 02:26:10,311][105692] Updated weights for policy 0, policy_version 1516081 (0.0010) [2023-12-27 02:26:10,365][105692] Updated weights for policy 0, policy_version 1516092 (0.0008) [2023-12-27 02:26:10,419][105692] Updated weights for policy 0, policy_version 1516102 (0.0006) [2023-12-27 02:26:10,426][105620] Updated weights for policy 1, policy_version 1518665 (0.0009) [2023-12-27 02:26:10,489][105620] Updated weights for policy 1, policy_version 1518675 (0.0009) [2023-12-27 02:26:10,552][105620] Updated weights for policy 1, policy_version 1518685 (0.0008) [2023-12-27 02:26:10,614][105620] Updated weights for policy 1, policy_version 1518695 (0.0008) [2023-12-27 02:26:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 18978.1, 300 sec: 19077.7). Total num frames: 777019392. Throughput: 0: 9526.6, 1: 9575.9. Samples: 777029156. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:26:11,063][104569] Avg episode reward: [(0, '8172.259'), (1, '9167.139')] [2023-12-27 02:26:11,178][105692] Updated weights for policy 0, policy_version 1516112 (0.0008) [2023-12-27 02:26:11,248][105692] Updated weights for policy 0, policy_version 1516122 (0.0006) [2023-12-27 02:26:11,292][105585] KL-divergence is very high: 100.4801 [2023-12-27 02:26:11,320][105692] Updated weights for policy 0, policy_version 1516132 (0.0008) [2023-12-27 02:26:11,363][105620] Updated weights for policy 1, policy_version 1518705 (0.0008) [2023-12-27 02:26:11,425][105620] Updated weights for policy 1, policy_version 1518715 (0.0009) [2023-12-27 02:26:11,484][105620] Updated weights for policy 1, policy_version 1518725 (0.0009) [2023-12-27 02:26:12,074][105692] Updated weights for policy 0, policy_version 1516142 (0.0008) [2023-12-27 02:26:12,129][105692] Updated weights for policy 0, policy_version 1516152 (0.0006) [2023-12-27 02:26:12,181][105692] Updated weights for policy 0, policy_version 1516162 (0.0006) [2023-12-27 02:26:12,292][105620] Updated weights for policy 1, policy_version 1518735 (0.0008) [2023-12-27 02:26:12,354][105620] Updated weights for policy 1, policy_version 1518745 (0.0008) [2023-12-27 02:26:12,421][105620] Updated weights for policy 1, policy_version 1518755 (0.0009) [2023-12-27 02:26:12,848][105692] Updated weights for policy 0, policy_version 1516172 (0.0007) [2023-12-27 02:26:12,907][105692] Updated weights for policy 0, policy_version 1516182 (0.0008) [2023-12-27 02:26:12,974][105692] Updated weights for policy 0, policy_version 1516192 (0.0005) [2023-12-27 02:26:13,245][105620] Updated weights for policy 1, policy_version 1518765 (0.0009) [2023-12-27 02:26:13,295][105620] Updated weights for policy 1, policy_version 1518775 (0.0009) [2023-12-27 02:26:13,346][105620] Updated weights for policy 1, policy_version 1518785 (0.0009) [2023-12-27 02:26:13,549][105692] Updated weights for policy 0, policy_version 1516202 (0.0006) [2023-12-27 02:26:13,606][105692] Updated weights for policy 0, policy_version 1516212 (0.0009) [2023-12-27 02:26:13,666][105692] Updated weights for policy 0, policy_version 1516222 (0.0009) [2023-12-27 02:26:13,721][105692] Updated weights for policy 0, policy_version 1516232 (0.0010) [2023-12-27 02:26:14,053][105620] Updated weights for policy 1, policy_version 1518795 (0.0007) [2023-12-27 02:26:14,123][105620] Updated weights for policy 1, policy_version 1518805 (0.0005) [2023-12-27 02:26:14,178][105620] Updated weights for policy 1, policy_version 1518815 (0.0005) [2023-12-27 02:26:14,471][105692] Updated weights for policy 0, policy_version 1516242 (0.0006) [2023-12-27 02:26:14,524][105692] Updated weights for policy 0, policy_version 1516254 (0.0010) [2023-12-27 02:26:14,574][105692] Updated weights for policy 0, policy_version 1516264 (0.0008) [2023-12-27 02:26:14,720][105620] Updated weights for policy 1, policy_version 1518825 (0.0005) [2023-12-27 02:26:14,782][105620] Updated weights for policy 1, policy_version 1518835 (0.0008) [2023-12-27 02:26:14,851][105620] Updated weights for policy 1, policy_version 1518845 (0.0008) [2023-12-27 02:26:14,920][105620] Updated weights for policy 1, policy_version 1518855 (0.0009) [2023-12-27 02:26:15,295][105692] Updated weights for policy 0, policy_version 1516274 (0.0006) [2023-12-27 02:26:15,363][105692] Updated weights for policy 0, policy_version 1516284 (0.0006) [2023-12-27 02:26:15,430][105692] Updated weights for policy 0, policy_version 1516294 (0.0007) [2023-12-27 02:26:15,636][105620] Updated weights for policy 1, policy_version 1518865 (0.0009) [2023-12-27 02:26:15,687][105620] Updated weights for policy 1, policy_version 1518875 (0.0009) [2023-12-27 02:26:15,745][105620] Updated weights for policy 1, policy_version 1518885 (0.0009) [2023-12-27 02:26:16,062][104569] Fps is (10 sec: 18841.9, 60 sec: 18978.2, 300 sec: 19105.4). Total num frames: 777117696. Throughput: 0: 9611.3, 1: 9493.9. Samples: 777086672. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:26:16,063][104569] Avg episode reward: [(0, '8356.731'), (1, '9348.912')] [2023-12-27 02:26:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001518888_388890624.pth... [2023-12-27 02:26:16,069][105692] Updated weights for policy 0, policy_version 1516304 (0.0009) [2023-12-27 02:26:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001517800_388612096.pth [2023-12-27 02:26:16,139][105692] Updated weights for policy 0, policy_version 1516314 (0.0010) [2023-12-27 02:26:16,209][105692] Updated weights for policy 0, policy_version 1516324 (0.0010) [2023-12-27 02:26:16,234][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001516328_388235264.pth... [2023-12-27 02:26:16,238][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001515176_387940352.pth [2023-12-27 02:26:16,476][105620] Updated weights for policy 1, policy_version 1518895 (0.0010) [2023-12-27 02:26:16,520][105620] Updated weights for policy 1, policy_version 1518905 (0.0010) [2023-12-27 02:26:16,571][105620] Updated weights for policy 1, policy_version 1518915 (0.0010) [2023-12-27 02:26:17,066][105692] Updated weights for policy 0, policy_version 1516334 (0.0010) [2023-12-27 02:26:17,122][105692] Updated weights for policy 0, policy_version 1516344 (0.0010) [2023-12-27 02:26:17,150][105620] Updated weights for policy 1, policy_version 1518925 (0.0008) [2023-12-27 02:26:17,178][105692] Updated weights for policy 0, policy_version 1516354 (0.0008) [2023-12-27 02:26:17,207][105620] Updated weights for policy 1, policy_version 1518935 (0.0006) [2023-12-27 02:26:17,276][105620] Updated weights for policy 1, policy_version 1518945 (0.0005) [2023-12-27 02:26:17,798][105692] Updated weights for policy 0, policy_version 1516364 (0.0007) [2023-12-27 02:26:17,847][105692] Updated weights for policy 0, policy_version 1516374 (0.0008) [2023-12-27 02:26:17,900][105692] Updated weights for policy 0, policy_version 1516384 (0.0008) [2023-12-27 02:26:17,933][105620] Updated weights for policy 1, policy_version 1518955 (0.0010) [2023-12-27 02:26:17,981][105620] Updated weights for policy 1, policy_version 1518965 (0.0010) [2023-12-27 02:26:18,026][105620] Updated weights for policy 1, policy_version 1518975 (0.0010) [2023-12-27 02:26:18,651][105692] Updated weights for policy 0, policy_version 1516394 (0.0008) [2023-12-27 02:26:18,710][105692] Updated weights for policy 0, policy_version 1516404 (0.0010) [2023-12-27 02:26:18,770][105692] Updated weights for policy 0, policy_version 1516414 (0.0011) [2023-12-27 02:26:18,794][105620] Updated weights for policy 1, policy_version 1518985 (0.0010) [2023-12-27 02:26:18,827][105692] Updated weights for policy 0, policy_version 1516424 (0.0009) [2023-12-27 02:26:18,855][105620] Updated weights for policy 1, policy_version 1518995 (0.0009) [2023-12-27 02:26:18,909][105620] Updated weights for policy 1, policy_version 1519005 (0.0010) [2023-12-27 02:26:18,970][105620] Updated weights for policy 1, policy_version 1519015 (0.0009) [2023-12-27 02:26:19,469][105692] Updated weights for policy 0, policy_version 1516434 (0.0006) [2023-12-27 02:26:19,536][105692] Updated weights for policy 0, policy_version 1516444 (0.0007) [2023-12-27 02:26:19,596][105692] Updated weights for policy 0, policy_version 1516454 (0.0005) [2023-12-27 02:26:19,838][105620] Updated weights for policy 1, policy_version 1519025 (0.0009) [2023-12-27 02:26:19,904][105620] Updated weights for policy 1, policy_version 1519035 (0.0008) [2023-12-27 02:26:19,967][105620] Updated weights for policy 1, policy_version 1519045 (0.0009) [2023-12-27 02:26:20,264][105692] Updated weights for policy 0, policy_version 1516464 (0.0008) [2023-12-27 02:26:20,327][105692] Updated weights for policy 0, policy_version 1516474 (0.0009) [2023-12-27 02:26:20,390][105692] Updated weights for policy 0, policy_version 1516484 (0.0009) [2023-12-27 02:26:20,694][105620] Updated weights for policy 1, policy_version 1519055 (0.0009) [2023-12-27 02:26:20,759][105620] Updated weights for policy 1, policy_version 1519065 (0.0008) [2023-12-27 02:26:20,815][105620] Updated weights for policy 1, policy_version 1519075 (0.0009) [2023-12-27 02:26:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19114.7, 300 sec: 19105.4). Total num frames: 777216000. Throughput: 0: 9661.2, 1: 9532.8. Samples: 777204924. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:26:21,062][104569] Avg episode reward: [(0, '8721.347'), (1, '9256.693')] [2023-12-27 02:26:21,200][105692] Updated weights for policy 0, policy_version 1516494 (0.0010) [2023-12-27 02:26:21,263][105692] Updated weights for policy 0, policy_version 1516504 (0.0009) [2023-12-27 02:26:21,327][105692] Updated weights for policy 0, policy_version 1516514 (0.0009) [2023-12-27 02:26:21,562][105620] Updated weights for policy 1, policy_version 1519085 (0.0008) [2023-12-27 02:26:21,622][105620] Updated weights for policy 1, policy_version 1519095 (0.0008) [2023-12-27 02:26:21,690][105620] Updated weights for policy 1, policy_version 1519105 (0.0008) [2023-12-27 02:26:22,104][105692] Updated weights for policy 0, policy_version 1516524 (0.0009) [2023-12-27 02:26:22,152][105692] Updated weights for policy 0, policy_version 1516534 (0.0009) [2023-12-27 02:26:22,204][105692] Updated weights for policy 0, policy_version 1516544 (0.0009) [2023-12-27 02:26:22,422][105620] Updated weights for policy 1, policy_version 1519115 (0.0008) [2023-12-27 02:26:22,473][105620] Updated weights for policy 1, policy_version 1519125 (0.0006) [2023-12-27 02:26:22,529][105620] Updated weights for policy 1, policy_version 1519135 (0.0006) [2023-12-27 02:26:23,108][105692] Updated weights for policy 0, policy_version 1516554 (0.0009) [2023-12-27 02:26:23,114][105620] Updated weights for policy 1, policy_version 1519145 (0.0008) [2023-12-27 02:26:23,161][105692] Updated weights for policy 0, policy_version 1516564 (0.0009) [2023-12-27 02:26:23,170][105620] Updated weights for policy 1, policy_version 1519155 (0.0005) [2023-12-27 02:26:23,214][105692] Updated weights for policy 0, policy_version 1516574 (0.0009) [2023-12-27 02:26:23,217][105620] Updated weights for policy 1, policy_version 1519165 (0.0006) [2023-12-27 02:26:23,261][105620] Updated weights for policy 1, policy_version 1519175 (0.0005) [2023-12-27 02:26:23,280][105692] Updated weights for policy 0, policy_version 1516584 (0.0008) [2023-12-27 02:26:23,901][105620] Updated weights for policy 1, policy_version 1519185 (0.0009) [2023-12-27 02:26:23,953][105620] Updated weights for policy 1, policy_version 1519195 (0.0009) [2023-12-27 02:26:24,016][105620] Updated weights for policy 1, policy_version 1519205 (0.0009) [2023-12-27 02:26:24,085][105692] Updated weights for policy 0, policy_version 1516594 (0.0009) [2023-12-27 02:26:24,133][105692] Updated weights for policy 0, policy_version 1516604 (0.0009) [2023-12-27 02:26:24,191][105692] Updated weights for policy 0, policy_version 1516614 (0.0008) [2023-12-27 02:26:24,814][105620] Updated weights for policy 1, policy_version 1519215 (0.0009) [2023-12-27 02:26:24,864][105620] Updated weights for policy 1, policy_version 1519225 (0.0008) [2023-12-27 02:26:24,891][105692] Updated weights for policy 0, policy_version 1516624 (0.0008) [2023-12-27 02:26:24,914][105620] Updated weights for policy 1, policy_version 1519235 (0.0006) [2023-12-27 02:26:24,940][105692] Updated weights for policy 0, policy_version 1516634 (0.0008) [2023-12-27 02:26:24,998][105692] Updated weights for policy 0, policy_version 1516644 (0.0009) [2023-12-27 02:26:25,541][105620] Updated weights for policy 1, policy_version 1519245 (0.0009) [2023-12-27 02:26:25,590][105620] Updated weights for policy 1, policy_version 1519255 (0.0010) [2023-12-27 02:26:25,637][105620] Updated weights for policy 1, policy_version 1519265 (0.0010) [2023-12-27 02:26:25,824][105692] Updated weights for policy 0, policy_version 1516654 (0.0009) [2023-12-27 02:26:25,871][105692] Updated weights for policy 0, policy_version 1516664 (0.0008) [2023-12-27 02:26:25,919][105692] Updated weights for policy 0, policy_version 1516674 (0.0008) [2023-12-27 02:26:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19077.6). Total num frames: 777314304. Throughput: 0: 9632.4, 1: 9540.5. Samples: 777318556. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:26:26,062][104569] Avg episode reward: [(0, '8357.535'), (1, '9164.478')] [2023-12-27 02:26:26,400][105620] Updated weights for policy 1, policy_version 1519275 (0.0010) [2023-12-27 02:26:26,445][105620] Updated weights for policy 1, policy_version 1519285 (0.0010) [2023-12-27 02:26:26,489][105620] Updated weights for policy 1, policy_version 1519295 (0.0010) [2023-12-27 02:26:26,685][105692] Updated weights for policy 0, policy_version 1516684 (0.0009) [2023-12-27 02:26:26,737][105692] Updated weights for policy 0, policy_version 1516694 (0.0008) [2023-12-27 02:26:26,790][105692] Updated weights for policy 0, policy_version 1516704 (0.0010) [2023-12-27 02:26:27,127][105620] Updated weights for policy 1, policy_version 1519305 (0.0007) [2023-12-27 02:26:27,184][105620] Updated weights for policy 1, policy_version 1519315 (0.0009) [2023-12-27 02:26:27,235][105620] Updated weights for policy 1, policy_version 1519325 (0.0010) [2023-12-27 02:26:27,292][105620] Updated weights for policy 1, policy_version 1519335 (0.0011) [2023-12-27 02:26:27,449][105692] Updated weights for policy 0, policy_version 1516714 (0.0006) [2023-12-27 02:26:27,501][105692] Updated weights for policy 0, policy_version 1516724 (0.0005) [2023-12-27 02:26:27,566][105692] Updated weights for policy 0, policy_version 1516734 (0.0005) [2023-12-27 02:26:27,622][105692] Updated weights for policy 0, policy_version 1516744 (0.0005) [2023-12-27 02:26:27,915][105620] Updated weights for policy 1, policy_version 1519345 (0.0008) [2023-12-27 02:26:27,967][105620] Updated weights for policy 1, policy_version 1519355 (0.0010) [2023-12-27 02:26:28,029][105620] Updated weights for policy 1, policy_version 1519366 (0.0009) [2023-12-27 02:26:28,212][105692] Updated weights for policy 0, policy_version 1516754 (0.0005) [2023-12-27 02:26:28,283][105692] Updated weights for policy 0, policy_version 1516764 (0.0009) [2023-12-27 02:26:28,350][105692] Updated weights for policy 0, policy_version 1516774 (0.0011) [2023-12-27 02:26:28,694][105620] Updated weights for policy 1, policy_version 1519376 (0.0008) [2023-12-27 02:26:28,754][105620] Updated weights for policy 1, policy_version 1519386 (0.0008) [2023-12-27 02:26:28,809][105620] Updated weights for policy 1, policy_version 1519396 (0.0008) [2023-12-27 02:26:29,033][105692] Updated weights for policy 0, policy_version 1516784 (0.0009) [2023-12-27 02:26:29,095][105692] Updated weights for policy 0, policy_version 1516794 (0.0006) [2023-12-27 02:26:29,154][105692] Updated weights for policy 0, policy_version 1516804 (0.0005) [2023-12-27 02:26:29,585][105620] Updated weights for policy 1, policy_version 1519406 (0.0008) [2023-12-27 02:26:29,641][105620] Updated weights for policy 1, policy_version 1519416 (0.0009) [2023-12-27 02:26:29,704][105620] Updated weights for policy 1, policy_version 1519426 (0.0009) [2023-12-27 02:26:29,761][105692] Updated weights for policy 0, policy_version 1516814 (0.0006) [2023-12-27 02:26:29,826][105692] Updated weights for policy 0, policy_version 1516824 (0.0007) [2023-12-27 02:26:29,888][105692] Updated weights for policy 0, policy_version 1516834 (0.0006) [2023-12-27 02:26:30,525][105620] Updated weights for policy 1, policy_version 1519436 (0.0009) [2023-12-27 02:26:30,542][105692] Updated weights for policy 0, policy_version 1516844 (0.0008) [2023-12-27 02:26:30,579][105620] Updated weights for policy 1, policy_version 1519446 (0.0007) [2023-12-27 02:26:30,593][105692] Updated weights for policy 0, policy_version 1516854 (0.0010) [2023-12-27 02:26:30,631][105620] Updated weights for policy 1, policy_version 1519456 (0.0005) [2023-12-27 02:26:30,644][105692] Updated weights for policy 0, policy_version 1516864 (0.0010) [2023-12-27 02:26:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19077.6). Total num frames: 777412608. Throughput: 0: 9692.0, 1: 9582.8. Samples: 777380688. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:26:31,063][104569] Avg episode reward: [(0, '8539.155'), (1, '9164.827')] [2023-12-27 02:26:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001516872_388374528.pth... [2023-12-27 02:26:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001519464_389038080.pth... [2023-12-27 02:26:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001515752_388087808.pth [2023-12-27 02:26:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001518312_388743168.pth [2023-12-27 02:26:31,389][105620] Updated weights for policy 1, policy_version 1519466 (0.0006) [2023-12-27 02:26:31,405][105692] Updated weights for policy 0, policy_version 1516874 (0.0010) [2023-12-27 02:26:31,448][105620] Updated weights for policy 1, policy_version 1519476 (0.0007) [2023-12-27 02:26:31,461][105692] Updated weights for policy 0, policy_version 1516884 (0.0010) [2023-12-27 02:26:31,500][105620] Updated weights for policy 1, policy_version 1519486 (0.0006) [2023-12-27 02:26:31,519][105692] Updated weights for policy 0, policy_version 1516894 (0.0010) [2023-12-27 02:26:31,552][105620] Updated weights for policy 1, policy_version 1519496 (0.0007) [2023-12-27 02:26:31,574][105692] Updated weights for policy 0, policy_version 1516904 (0.0009) [2023-12-27 02:26:32,219][105620] Updated weights for policy 1, policy_version 1519506 (0.0005) [2023-12-27 02:26:32,233][105692] Updated weights for policy 0, policy_version 1516914 (0.0009) [2023-12-27 02:26:32,278][105620] Updated weights for policy 1, policy_version 1519516 (0.0006) [2023-12-27 02:26:32,289][105692] Updated weights for policy 0, policy_version 1516924 (0.0009) [2023-12-27 02:26:32,338][105620] Updated weights for policy 1, policy_version 1519526 (0.0006) [2023-12-27 02:26:32,353][105692] Updated weights for policy 0, policy_version 1516934 (0.0009) [2023-12-27 02:26:32,961][105620] Updated weights for policy 1, policy_version 1519536 (0.0007) [2023-12-27 02:26:33,015][105620] Updated weights for policy 1, policy_version 1519546 (0.0005) [2023-12-27 02:26:33,079][105620] Updated weights for policy 1, policy_version 1519556 (0.0006) [2023-12-27 02:26:33,163][105692] Updated weights for policy 0, policy_version 1516944 (0.0006) [2023-12-27 02:26:33,208][105692] Updated weights for policy 0, policy_version 1516954 (0.0005) [2023-12-27 02:26:33,250][105692] Updated weights for policy 0, policy_version 1516964 (0.0005) [2023-12-27 02:26:33,791][105692] Updated weights for policy 0, policy_version 1516974 (0.0007) [2023-12-27 02:26:33,821][105620] Updated weights for policy 1, policy_version 1519566 (0.0009) [2023-12-27 02:26:33,840][105692] Updated weights for policy 0, policy_version 1516984 (0.0008) [2023-12-27 02:26:33,878][105620] Updated weights for policy 1, policy_version 1519576 (0.0009) [2023-12-27 02:26:33,885][105692] Updated weights for policy 0, policy_version 1516994 (0.0006) [2023-12-27 02:26:33,930][105620] Updated weights for policy 1, policy_version 1519586 (0.0008) [2023-12-27 02:26:34,657][105620] Updated weights for policy 1, policy_version 1519596 (0.0009) [2023-12-27 02:26:34,680][105692] Updated weights for policy 0, policy_version 1517004 (0.0007) [2023-12-27 02:26:34,719][105620] Updated weights for policy 1, policy_version 1519606 (0.0007) [2023-12-27 02:26:34,744][105692] Updated weights for policy 0, policy_version 1517014 (0.0008) [2023-12-27 02:26:34,780][105620] Updated weights for policy 1, policy_version 1519616 (0.0006) [2023-12-27 02:26:34,802][105692] Updated weights for policy 0, policy_version 1517024 (0.0008) [2023-12-27 02:26:35,474][105620] Updated weights for policy 1, policy_version 1519626 (0.0009) [2023-12-27 02:26:35,508][105692] Updated weights for policy 0, policy_version 1517034 (0.0009) [2023-12-27 02:26:35,539][105620] Updated weights for policy 1, policy_version 1519636 (0.0007) [2023-12-27 02:26:35,566][105692] Updated weights for policy 0, policy_version 1517044 (0.0007) [2023-12-27 02:26:35,596][105620] Updated weights for policy 1, policy_version 1519646 (0.0009) [2023-12-27 02:26:35,617][105692] Updated weights for policy 0, policy_version 1517054 (0.0005) [2023-12-27 02:26:35,651][105620] Updated weights for policy 1, policy_version 1519656 (0.0009) [2023-12-27 02:26:35,665][105692] Updated weights for policy 0, policy_version 1517064 (0.0005) [2023-12-27 02:26:36,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19251.1, 300 sec: 19105.4). Total num frames: 777510912. Throughput: 0: 9697.3, 1: 9643.4. Samples: 777499164. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:26:36,063][104569] Avg episode reward: [(0, '8536.955'), (1, '8984.841')] [2023-12-27 02:26:36,292][105620] Updated weights for policy 1, policy_version 1519666 (0.0007) [2023-12-27 02:26:36,321][105692] Updated weights for policy 0, policy_version 1517074 (0.0011) [2023-12-27 02:26:36,360][105620] Updated weights for policy 1, policy_version 1519676 (0.0006) [2023-12-27 02:26:36,381][105692] Updated weights for policy 0, policy_version 1517084 (0.0011) [2023-12-27 02:26:36,419][105620] Updated weights for policy 1, policy_version 1519686 (0.0008) [2023-12-27 02:26:36,435][105692] Updated weights for policy 0, policy_version 1517094 (0.0009) [2023-12-27 02:26:37,078][105620] Updated weights for policy 1, policy_version 1519696 (0.0006) [2023-12-27 02:26:37,142][105620] Updated weights for policy 1, policy_version 1519706 (0.0008) [2023-12-27 02:26:37,193][105620] Updated weights for policy 1, policy_version 1519716 (0.0008) [2023-12-27 02:26:37,248][105692] Updated weights for policy 0, policy_version 1517104 (0.0008) [2023-12-27 02:26:37,310][105692] Updated weights for policy 0, policy_version 1517114 (0.0007) [2023-12-27 02:26:37,376][105692] Updated weights for policy 0, policy_version 1517124 (0.0008) [2023-12-27 02:26:37,915][105620] Updated weights for policy 1, policy_version 1519726 (0.0010) [2023-12-27 02:26:37,959][105620] Updated weights for policy 1, policy_version 1519736 (0.0010) [2023-12-27 02:26:38,004][105620] Updated weights for policy 1, policy_version 1519746 (0.0010) [2023-12-27 02:26:38,109][105692] Updated weights for policy 0, policy_version 1517134 (0.0007) [2023-12-27 02:26:38,159][105692] Updated weights for policy 0, policy_version 1517144 (0.0008) [2023-12-27 02:26:38,212][105692] Updated weights for policy 0, policy_version 1517154 (0.0008) [2023-12-27 02:26:38,790][105620] Updated weights for policy 1, policy_version 1519756 (0.0010) [2023-12-27 02:26:38,848][105620] Updated weights for policy 1, policy_version 1519766 (0.0010) [2023-12-27 02:26:38,896][105620] Updated weights for policy 1, policy_version 1519776 (0.0010) [2023-12-27 02:26:39,015][105692] Updated weights for policy 0, policy_version 1517164 (0.0008) [2023-12-27 02:26:39,063][105692] Updated weights for policy 0, policy_version 1517174 (0.0008) [2023-12-27 02:26:39,116][105692] Updated weights for policy 0, policy_version 1517184 (0.0008) [2023-12-27 02:26:39,680][105620] Updated weights for policy 1, policy_version 1519786 (0.0010) [2023-12-27 02:26:39,743][105620] Updated weights for policy 1, policy_version 1519796 (0.0011) [2023-12-27 02:26:39,800][105620] Updated weights for policy 1, policy_version 1519806 (0.0011) [2023-12-27 02:26:39,865][105620] Updated weights for policy 1, policy_version 1519816 (0.0009) [2023-12-27 02:26:39,946][105692] Updated weights for policy 0, policy_version 1517194 (0.0008) [2023-12-27 02:26:40,008][105692] Updated weights for policy 0, policy_version 1517204 (0.0009) [2023-12-27 02:26:40,069][105692] Updated weights for policy 0, policy_version 1517214 (0.0008) [2023-12-27 02:26:40,132][105692] Updated weights for policy 0, policy_version 1517224 (0.0009) [2023-12-27 02:26:40,650][105620] Updated weights for policy 1, policy_version 1519826 (0.0011) [2023-12-27 02:26:40,698][105620] Updated weights for policy 1, policy_version 1519836 (0.0010) [2023-12-27 02:26:40,754][105620] Updated weights for policy 1, policy_version 1519846 (0.0010) [2023-12-27 02:26:40,907][105692] Updated weights for policy 0, policy_version 1517234 (0.0008) [2023-12-27 02:26:40,969][105692] Updated weights for policy 0, policy_version 1517244 (0.0008) [2023-12-27 02:26:41,027][105692] Updated weights for policy 0, policy_version 1517254 (0.0007) [2023-12-27 02:26:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19105.4). Total num frames: 777609216. Throughput: 0: 9612.1, 1: 9717.6. Samples: 777612524. Policy #0 lag: (min: 1.0, avg: 24.7, max: 33.0) [2023-12-27 02:26:41,062][104569] Avg episode reward: [(0, '8172.923'), (1, '8988.082')] [2023-12-27 02:26:41,562][105620] Updated weights for policy 1, policy_version 1519856 (0.0011) [2023-12-27 02:26:41,627][105620] Updated weights for policy 1, policy_version 1519866 (0.0011) [2023-12-27 02:26:41,691][105620] Updated weights for policy 1, policy_version 1519876 (0.0010) [2023-12-27 02:26:41,880][105692] Updated weights for policy 0, policy_version 1517264 (0.0008) [2023-12-27 02:26:41,933][105692] Updated weights for policy 0, policy_version 1517274 (0.0008) [2023-12-27 02:26:41,980][105692] Updated weights for policy 0, policy_version 1517284 (0.0008) [2023-12-27 02:26:42,490][105620] Updated weights for policy 1, policy_version 1519886 (0.0010) [2023-12-27 02:26:42,549][105620] Updated weights for policy 1, policy_version 1519896 (0.0010) [2023-12-27 02:26:42,616][105620] Updated weights for policy 1, policy_version 1519906 (0.0010) [2023-12-27 02:26:42,820][105692] Updated weights for policy 0, policy_version 1517294 (0.0008) [2023-12-27 02:26:42,873][105692] Updated weights for policy 0, policy_version 1517304 (0.0008) [2023-12-27 02:26:42,921][105692] Updated weights for policy 0, policy_version 1517314 (0.0008) [2023-12-27 02:26:43,357][105620] Updated weights for policy 1, policy_version 1519916 (0.0010) [2023-12-27 02:26:43,407][105620] Updated weights for policy 1, policy_version 1519926 (0.0010) [2023-12-27 02:26:43,465][105620] Updated weights for policy 1, policy_version 1519936 (0.0009) [2023-12-27 02:26:43,711][105692] Updated weights for policy 0, policy_version 1517324 (0.0008) [2023-12-27 02:26:43,766][105692] Updated weights for policy 0, policy_version 1517334 (0.0008) [2023-12-27 02:26:43,822][105692] Updated weights for policy 0, policy_version 1517344 (0.0008) [2023-12-27 02:26:44,204][105620] Updated weights for policy 1, policy_version 1519946 (0.0010) [2023-12-27 02:26:44,252][105620] Updated weights for policy 1, policy_version 1519956 (0.0010) [2023-12-27 02:26:44,313][105620] Updated weights for policy 1, policy_version 1519966 (0.0010) [2023-12-27 02:26:44,379][105620] Updated weights for policy 1, policy_version 1519976 (0.0010) [2023-12-27 02:26:44,593][105692] Updated weights for policy 0, policy_version 1517354 (0.0008) [2023-12-27 02:26:44,646][105692] Updated weights for policy 0, policy_version 1517364 (0.0008) [2023-12-27 02:26:44,697][105692] Updated weights for policy 0, policy_version 1517374 (0.0008) [2023-12-27 02:26:44,746][105692] Updated weights for policy 0, policy_version 1517384 (0.0008) [2023-12-27 02:26:45,170][105620] Updated weights for policy 1, policy_version 1519986 (0.0011) [2023-12-27 02:26:45,234][105620] Updated weights for policy 1, policy_version 1519996 (0.0011) [2023-12-27 02:26:45,302][105620] Updated weights for policy 1, policy_version 1520006 (0.0011) [2023-12-27 02:26:45,557][105692] Updated weights for policy 0, policy_version 1517394 (0.0009) [2023-12-27 02:26:45,604][105692] Updated weights for policy 0, policy_version 1517404 (0.0009) [2023-12-27 02:26:45,655][105692] Updated weights for policy 0, policy_version 1517414 (0.0009) [2023-12-27 02:26:45,950][105620] Updated weights for policy 1, policy_version 1520016 (0.0007) [2023-12-27 02:26:46,006][105620] Updated weights for policy 1, policy_version 1520026 (0.0008) [2023-12-27 02:26:46,057][105620] Updated weights for policy 1, policy_version 1520036 (0.0009) [2023-12-27 02:26:46,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19114.6, 300 sec: 19049.9). Total num frames: 777691136. Throughput: 0: 9525.1, 1: 9748.1. Samples: 777666448. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:26:46,063][104569] Avg episode reward: [(0, '8541.735'), (1, '9261.967')] [2023-12-27 02:26:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001517416_388513792.pth... [2023-12-27 02:26:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001516328_388235264.pth [2023-12-27 02:26:46,075][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001520040_389185536.pth... [2023-12-27 02:26:46,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001518888_388890624.pth [2023-12-27 02:26:46,401][105692] Updated weights for policy 0, policy_version 1517424 (0.0009) [2023-12-27 02:26:46,449][105692] Updated weights for policy 0, policy_version 1517435 (0.0009) [2023-12-27 02:26:46,502][105692] Updated weights for policy 0, policy_version 1517445 (0.0009) [2023-12-27 02:26:46,833][105620] Updated weights for policy 1, policy_version 1520046 (0.0010) [2023-12-27 02:26:46,896][105620] Updated weights for policy 1, policy_version 1520056 (0.0009) [2023-12-27 02:26:46,952][105620] Updated weights for policy 1, policy_version 1520066 (0.0008) [2023-12-27 02:26:47,150][105692] Updated weights for policy 0, policy_version 1517455 (0.0007) [2023-12-27 02:26:47,202][105692] Updated weights for policy 0, policy_version 1517465 (0.0008) [2023-12-27 02:26:47,251][105692] Updated weights for policy 0, policy_version 1517475 (0.0005) [2023-12-27 02:26:47,698][105620] Updated weights for policy 1, policy_version 1520076 (0.0009) [2023-12-27 02:26:47,751][105620] Updated weights for policy 1, policy_version 1520086 (0.0008) [2023-12-27 02:26:47,804][105620] Updated weights for policy 1, policy_version 1520097 (0.0009) [2023-12-27 02:26:47,857][105692] Updated weights for policy 0, policy_version 1517485 (0.0005) [2023-12-27 02:26:47,912][105692] Updated weights for policy 0, policy_version 1517495 (0.0005) [2023-12-27 02:26:47,967][105692] Updated weights for policy 0, policy_version 1517505 (0.0009) [2023-12-27 02:26:48,605][105692] Updated weights for policy 0, policy_version 1517515 (0.0009) [2023-12-27 02:26:48,645][105620] Updated weights for policy 1, policy_version 1520108 (0.0010) [2023-12-27 02:26:48,658][105692] Updated weights for policy 0, policy_version 1517525 (0.0006) [2023-12-27 02:26:48,697][105620] Updated weights for policy 1, policy_version 1520118 (0.0007) [2023-12-27 02:26:48,719][105692] Updated weights for policy 0, policy_version 1517535 (0.0010) [2023-12-27 02:26:48,749][105620] Updated weights for policy 1, policy_version 1520128 (0.0006) [2023-12-27 02:26:49,386][105692] Updated weights for policy 0, policy_version 1517545 (0.0010) [2023-12-27 02:26:49,390][105620] Updated weights for policy 1, policy_version 1520138 (0.0007) [2023-12-27 02:26:49,447][105692] Updated weights for policy 0, policy_version 1517555 (0.0008) [2023-12-27 02:26:49,460][105620] Updated weights for policy 1, policy_version 1520148 (0.0008) [2023-12-27 02:26:49,510][105692] Updated weights for policy 0, policy_version 1517565 (0.0008) [2023-12-27 02:26:49,521][105620] Updated weights for policy 1, policy_version 1520158 (0.0007) [2023-12-27 02:26:49,566][105692] Updated weights for policy 0, policy_version 1517575 (0.0007) [2023-12-27 02:26:49,583][105620] Updated weights for policy 1, policy_version 1520168 (0.0007) [2023-12-27 02:26:50,192][105620] Updated weights for policy 1, policy_version 1520178 (0.0005) [2023-12-27 02:26:50,248][105620] Updated weights for policy 1, policy_version 1520188 (0.0008) [2023-12-27 02:26:50,264][105692] Updated weights for policy 0, policy_version 1517585 (0.0010) [2023-12-27 02:26:50,311][105620] Updated weights for policy 1, policy_version 1520198 (0.0009) [2023-12-27 02:26:50,330][105692] Updated weights for policy 0, policy_version 1517595 (0.0011) [2023-12-27 02:26:50,393][105692] Updated weights for policy 0, policy_version 1517605 (0.0010) [2023-12-27 02:26:50,972][105620] Updated weights for policy 1, policy_version 1520208 (0.0005) [2023-12-27 02:26:51,024][105620] Updated weights for policy 1, policy_version 1520218 (0.0008) [2023-12-27 02:26:51,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19114.7, 300 sec: 19049.9). Total num frames: 777789440. Throughput: 0: 9573.1, 1: 9727.7. Samples: 777784360. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:26:51,062][104569] Avg episode reward: [(0, '8721.715'), (1, '9264.062')] [2023-12-27 02:26:51,089][105620] Updated weights for policy 1, policy_version 1520228 (0.0009) [2023-12-27 02:26:51,188][105692] Updated weights for policy 0, policy_version 1517615 (0.0010) [2023-12-27 02:26:51,246][105692] Updated weights for policy 0, policy_version 1517625 (0.0009) [2023-12-27 02:26:51,311][105692] Updated weights for policy 0, policy_version 1517635 (0.0009) [2023-12-27 02:26:51,823][105620] Updated weights for policy 1, policy_version 1520238 (0.0009) [2023-12-27 02:26:51,879][105620] Updated weights for policy 1, policy_version 1520248 (0.0010) [2023-12-27 02:26:51,936][105620] Updated weights for policy 1, policy_version 1520258 (0.0010) [2023-12-27 02:26:52,018][105692] Updated weights for policy 0, policy_version 1517645 (0.0007) [2023-12-27 02:26:52,073][105692] Updated weights for policy 0, policy_version 1517655 (0.0005) [2023-12-27 02:26:52,128][105692] Updated weights for policy 0, policy_version 1517665 (0.0005) [2023-12-27 02:26:52,731][105692] Updated weights for policy 0, policy_version 1517675 (0.0008) [2023-12-27 02:26:52,783][105692] Updated weights for policy 0, policy_version 1517685 (0.0010) [2023-12-27 02:26:52,805][105620] Updated weights for policy 1, policy_version 1520268 (0.0009) [2023-12-27 02:26:52,839][105692] Updated weights for policy 0, policy_version 1517695 (0.0011) [2023-12-27 02:26:52,864][105620] Updated weights for policy 1, policy_version 1520278 (0.0008) [2023-12-27 02:26:52,916][105620] Updated weights for policy 1, policy_version 1520288 (0.0007) [2023-12-27 02:26:53,586][105692] Updated weights for policy 0, policy_version 1517705 (0.0011) [2023-12-27 02:26:53,622][105620] Updated weights for policy 1, policy_version 1520298 (0.0008) [2023-12-27 02:26:53,652][105692] Updated weights for policy 0, policy_version 1517715 (0.0011) [2023-12-27 02:26:53,677][105620] Updated weights for policy 1, policy_version 1520308 (0.0005) [2023-12-27 02:26:53,704][105692] Updated weights for policy 0, policy_version 1517725 (0.0011) [2023-12-27 02:26:53,737][105620] Updated weights for policy 1, policy_version 1520318 (0.0005) [2023-12-27 02:26:53,755][105692] Updated weights for policy 0, policy_version 1517735 (0.0010) [2023-12-27 02:26:53,795][105620] Updated weights for policy 1, policy_version 1520328 (0.0005) [2023-12-27 02:26:54,351][105620] Updated weights for policy 1, policy_version 1520338 (0.0010) [2023-12-27 02:26:54,403][105620] Updated weights for policy 1, policy_version 1520348 (0.0008) [2023-12-27 02:26:54,411][105692] Updated weights for policy 0, policy_version 1517745 (0.0006) [2023-12-27 02:26:54,457][105620] Updated weights for policy 1, policy_version 1520358 (0.0005) [2023-12-27 02:26:54,462][105692] Updated weights for policy 0, policy_version 1517755 (0.0006) [2023-12-27 02:26:54,515][105692] Updated weights for policy 0, policy_version 1517765 (0.0009) [2023-12-27 02:26:55,117][105620] Updated weights for policy 1, policy_version 1520368 (0.0010) [2023-12-27 02:26:55,165][105692] Updated weights for policy 0, policy_version 1517776 (0.0011) [2023-12-27 02:26:55,176][105620] Updated weights for policy 1, policy_version 1520378 (0.0011) [2023-12-27 02:26:55,221][105692] Updated weights for policy 0, policy_version 1517786 (0.0007) [2023-12-27 02:26:55,229][105620] Updated weights for policy 1, policy_version 1520388 (0.0011) [2023-12-27 02:26:55,277][105692] Updated weights for policy 0, policy_version 1517796 (0.0005) [2023-12-27 02:26:55,872][105620] Updated weights for policy 1, policy_version 1520398 (0.0007) [2023-12-27 02:26:55,933][105620] Updated weights for policy 1, policy_version 1520408 (0.0005) [2023-12-27 02:26:55,960][105692] Updated weights for policy 0, policy_version 1517806 (0.0008) [2023-12-27 02:26:55,987][105620] Updated weights for policy 1, policy_version 1520418 (0.0005) [2023-12-27 02:26:56,011][105692] Updated weights for policy 0, policy_version 1517816 (0.0010) [2023-12-27 02:26:56,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19387.8, 300 sec: 19077.6). Total num frames: 777895936. Throughput: 0: 9700.7, 1: 9772.9. Samples: 777905468. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:26:56,062][104569] Avg episode reward: [(0, '8628.578'), (1, '9263.793')] [2023-12-27 02:26:56,067][105692] Updated weights for policy 0, policy_version 1517826 (0.0010) [2023-12-27 02:26:56,500][105620] Updated weights for policy 1, policy_version 1520428 (0.0005) [2023-12-27 02:26:56,568][105620] Updated weights for policy 1, policy_version 1520438 (0.0005) [2023-12-27 02:26:56,634][105620] Updated weights for policy 1, policy_version 1520448 (0.0005) [2023-12-27 02:26:56,823][105692] Updated weights for policy 0, policy_version 1517836 (0.0010) [2023-12-27 02:26:56,870][105692] Updated weights for policy 0, policy_version 1517846 (0.0010) [2023-12-27 02:26:56,923][105692] Updated weights for policy 0, policy_version 1517856 (0.0010) [2023-12-27 02:26:57,249][105620] Updated weights for policy 1, policy_version 1520458 (0.0006) [2023-12-27 02:26:57,311][105620] Updated weights for policy 1, policy_version 1520468 (0.0009) [2023-12-27 02:26:57,373][105620] Updated weights for policy 1, policy_version 1520478 (0.0010) [2023-12-27 02:26:57,427][105620] Updated weights for policy 1, policy_version 1520488 (0.0010) [2023-12-27 02:26:57,664][105692] Updated weights for policy 0, policy_version 1517866 (0.0010) [2023-12-27 02:26:57,729][105692] Updated weights for policy 0, policy_version 1517876 (0.0011) [2023-12-27 02:26:57,788][105692] Updated weights for policy 0, policy_version 1517886 (0.0011) [2023-12-27 02:26:57,842][105692] Updated weights for policy 0, policy_version 1517896 (0.0010) [2023-12-27 02:26:58,158][105620] Updated weights for policy 1, policy_version 1520498 (0.0010) [2023-12-27 02:26:58,223][105620] Updated weights for policy 1, policy_version 1520508 (0.0008) [2023-12-27 02:26:58,276][105620] Updated weights for policy 1, policy_version 1520518 (0.0011) [2023-12-27 02:26:58,583][105692] Updated weights for policy 0, policy_version 1517906 (0.0007) [2023-12-27 02:26:58,654][105692] Updated weights for policy 0, policy_version 1517916 (0.0007) [2023-12-27 02:26:58,719][105692] Updated weights for policy 0, policy_version 1517926 (0.0010) [2023-12-27 02:26:59,061][105620] Updated weights for policy 1, policy_version 1520528 (0.0007) [2023-12-27 02:26:59,107][105620] Updated weights for policy 1, policy_version 1520538 (0.0006) [2023-12-27 02:26:59,167][105620] Updated weights for policy 1, policy_version 1520548 (0.0006) [2023-12-27 02:26:59,526][105692] Updated weights for policy 0, policy_version 1517936 (0.0008) [2023-12-27 02:26:59,585][105692] Updated weights for policy 0, policy_version 1517947 (0.0010) [2023-12-27 02:26:59,637][105692] Updated weights for policy 0, policy_version 1517957 (0.0007) [2023-12-27 02:26:59,849][105620] Updated weights for policy 1, policy_version 1520558 (0.0008) [2023-12-27 02:26:59,914][105620] Updated weights for policy 1, policy_version 1520568 (0.0008) [2023-12-27 02:26:59,987][105620] Updated weights for policy 1, policy_version 1520578 (0.0008) [2023-12-27 02:27:00,227][105692] Updated weights for policy 0, policy_version 1517967 (0.0008) [2023-12-27 02:27:00,278][105692] Updated weights for policy 0, policy_version 1517977 (0.0007) [2023-12-27 02:27:00,347][105692] Updated weights for policy 0, policy_version 1517987 (0.0010) [2023-12-27 02:27:00,770][105620] Updated weights for policy 1, policy_version 1520588 (0.0009) [2023-12-27 02:27:00,819][105620] Updated weights for policy 1, policy_version 1520598 (0.0008) [2023-12-27 02:27:00,870][105620] Updated weights for policy 1, policy_version 1520608 (0.0009) [2023-12-27 02:27:00,976][105692] Updated weights for policy 0, policy_version 1517997 (0.0009) [2023-12-27 02:27:01,026][105692] Updated weights for policy 0, policy_version 1518007 (0.0009) [2023-12-27 02:27:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19387.7, 300 sec: 19077.6). Total num frames: 777994240. Throughput: 0: 9656.2, 1: 9850.9. Samples: 777964488. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:27:01,063][104569] Avg episode reward: [(0, '8716.799'), (1, '9258.925')] [2023-12-27 02:27:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001520616_389332992.pth... [2023-12-27 02:27:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001519464_389038080.pth [2023-12-27 02:27:01,088][105692] Updated weights for policy 0, policy_version 1518017 (0.0008) [2023-12-27 02:27:01,134][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001518024_388669440.pth... [2023-12-27 02:27:01,139][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001516872_388374528.pth [2023-12-27 02:27:01,664][105620] Updated weights for policy 1, policy_version 1520618 (0.0010) [2023-12-27 02:27:01,726][105620] Updated weights for policy 1, policy_version 1520628 (0.0010) [2023-12-27 02:27:01,785][105620] Updated weights for policy 1, policy_version 1520638 (0.0006) [2023-12-27 02:27:01,834][105692] Updated weights for policy 0, policy_version 1518027 (0.0007) [2023-12-27 02:27:01,840][105620] Updated weights for policy 1, policy_version 1520648 (0.0005) [2023-12-27 02:27:01,883][105692] Updated weights for policy 0, policy_version 1518037 (0.0005) [2023-12-27 02:27:01,938][105692] Updated weights for policy 0, policy_version 1518047 (0.0005) [2023-12-27 02:27:02,541][105620] Updated weights for policy 1, policy_version 1520658 (0.0008) [2023-12-27 02:27:02,608][105620] Updated weights for policy 1, policy_version 1520668 (0.0011) [2023-12-27 02:27:02,625][105692] Updated weights for policy 0, policy_version 1518057 (0.0006) [2023-12-27 02:27:02,667][105620] Updated weights for policy 1, policy_version 1520678 (0.0011) [2023-12-27 02:27:02,685][105692] Updated weights for policy 0, policy_version 1518067 (0.0005) [2023-12-27 02:27:02,750][105692] Updated weights for policy 0, policy_version 1518077 (0.0007) [2023-12-27 02:27:02,808][105692] Updated weights for policy 0, policy_version 1518087 (0.0008) [2023-12-27 02:27:03,332][105620] Updated weights for policy 1, policy_version 1520688 (0.0007) [2023-12-27 02:27:03,388][105620] Updated weights for policy 1, policy_version 1520698 (0.0005) [2023-12-27 02:27:03,410][105692] Updated weights for policy 0, policy_version 1518097 (0.0010) [2023-12-27 02:27:03,438][105620] Updated weights for policy 1, policy_version 1520708 (0.0005) [2023-12-27 02:27:03,461][105692] Updated weights for policy 0, policy_version 1518107 (0.0010) [2023-12-27 02:27:03,509][105692] Updated weights for policy 0, policy_version 1518117 (0.0010) [2023-12-27 02:27:03,963][105620] Updated weights for policy 1, policy_version 1520718 (0.0008) [2023-12-27 02:27:04,028][105620] Updated weights for policy 1, policy_version 1520728 (0.0010) [2023-12-27 02:27:04,087][105620] Updated weights for policy 1, policy_version 1520738 (0.0010) [2023-12-27 02:27:04,156][105692] Updated weights for policy 0, policy_version 1518127 (0.0009) [2023-12-27 02:27:04,216][105692] Updated weights for policy 0, policy_version 1518137 (0.0008) [2023-12-27 02:27:04,269][105692] Updated weights for policy 0, policy_version 1518147 (0.0006) [2023-12-27 02:27:04,816][105620] Updated weights for policy 1, policy_version 1520748 (0.0007) [2023-12-27 02:27:04,868][105692] Updated weights for policy 0, policy_version 1518157 (0.0005) [2023-12-27 02:27:04,878][105620] Updated weights for policy 1, policy_version 1520758 (0.0010) [2023-12-27 02:27:04,932][105692] Updated weights for policy 0, policy_version 1518167 (0.0006) [2023-12-27 02:27:04,947][105620] Updated weights for policy 1, policy_version 1520768 (0.0010) [2023-12-27 02:27:04,993][105692] Updated weights for policy 0, policy_version 1518177 (0.0007) [2023-12-27 02:27:05,613][105620] Updated weights for policy 1, policy_version 1520778 (0.0009) [2023-12-27 02:27:05,679][105620] Updated weights for policy 1, policy_version 1520788 (0.0005) [2023-12-27 02:27:05,730][105620] Updated weights for policy 1, policy_version 1520798 (0.0005) [2023-12-27 02:27:05,775][105620] Updated weights for policy 1, policy_version 1520808 (0.0005) [2023-12-27 02:27:05,777][105692] Updated weights for policy 0, policy_version 1518187 (0.0008) [2023-12-27 02:27:05,830][105692] Updated weights for policy 0, policy_version 1518197 (0.0010) [2023-12-27 02:27:05,889][105692] Updated weights for policy 0, policy_version 1518207 (0.0008) [2023-12-27 02:27:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.4, 300 sec: 19077.6). Total num frames: 778100736. Throughput: 0: 9718.0, 1: 9848.8. Samples: 778085428. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:27:06,062][104569] Avg episode reward: [(0, '8902.501'), (1, '9170.729')] [2023-12-27 02:27:06,442][105620] Updated weights for policy 1, policy_version 1520818 (0.0007) [2023-12-27 02:27:06,501][105620] Updated weights for policy 1, policy_version 1520828 (0.0005) [2023-12-27 02:27:06,563][105620] Updated weights for policy 1, policy_version 1520838 (0.0005) [2023-12-27 02:27:06,716][105692] Updated weights for policy 0, policy_version 1518217 (0.0006) [2023-12-27 02:27:06,772][105692] Updated weights for policy 0, policy_version 1518227 (0.0010) [2023-12-27 02:27:06,823][105692] Updated weights for policy 0, policy_version 1518237 (0.0009) [2023-12-27 02:27:06,872][105692] Updated weights for policy 0, policy_version 1518247 (0.0008) [2023-12-27 02:27:07,164][105620] Updated weights for policy 1, policy_version 1520848 (0.0010) [2023-12-27 02:27:07,226][105620] Updated weights for policy 1, policy_version 1520858 (0.0011) [2023-12-27 02:27:07,291][105620] Updated weights for policy 1, policy_version 1520868 (0.0011) [2023-12-27 02:27:07,672][105692] Updated weights for policy 0, policy_version 1518257 (0.0010) [2023-12-27 02:27:07,731][105692] Updated weights for policy 0, policy_version 1518267 (0.0009) [2023-12-27 02:27:07,796][105692] Updated weights for policy 0, policy_version 1518277 (0.0010) [2023-12-27 02:27:07,995][105620] Updated weights for policy 1, policy_version 1520878 (0.0010) [2023-12-27 02:27:08,042][105620] Updated weights for policy 1, policy_version 1520888 (0.0010) [2023-12-27 02:27:08,086][105620] Updated weights for policy 1, policy_version 1520898 (0.0010) [2023-12-27 02:27:08,515][105692] Updated weights for policy 0, policy_version 1518287 (0.0011) [2023-12-27 02:27:08,580][105692] Updated weights for policy 0, policy_version 1518297 (0.0010) [2023-12-27 02:27:08,643][105692] Updated weights for policy 0, policy_version 1518307 (0.0010) [2023-12-27 02:27:08,751][105620] Updated weights for policy 1, policy_version 1520908 (0.0008) [2023-12-27 02:27:08,808][105620] Updated weights for policy 1, policy_version 1520918 (0.0005) [2023-12-27 02:27:08,855][105620] Updated weights for policy 1, policy_version 1520928 (0.0005) [2023-12-27 02:27:09,401][105692] Updated weights for policy 0, policy_version 1518317 (0.0010) [2023-12-27 02:27:09,462][105692] Updated weights for policy 0, policy_version 1518327 (0.0009) [2023-12-27 02:27:09,468][105620] Updated weights for policy 1, policy_version 1520938 (0.0007) [2023-12-27 02:27:09,521][105692] Updated weights for policy 0, policy_version 1518337 (0.0010) [2023-12-27 02:27:09,522][105620] Updated weights for policy 1, policy_version 1520948 (0.0006) [2023-12-27 02:27:09,576][105620] Updated weights for policy 1, policy_version 1520958 (0.0009) [2023-12-27 02:27:09,625][105620] Updated weights for policy 1, policy_version 1520968 (0.0008) [2023-12-27 02:27:10,247][105692] Updated weights for policy 0, policy_version 1518347 (0.0009) [2023-12-27 02:27:10,305][105692] Updated weights for policy 0, policy_version 1518357 (0.0009) [2023-12-27 02:27:10,368][105692] Updated weights for policy 0, policy_version 1518367 (0.0009) [2023-12-27 02:27:10,429][105620] Updated weights for policy 1, policy_version 1520978 (0.0007) [2023-12-27 02:27:10,492][105620] Updated weights for policy 1, policy_version 1520988 (0.0007) [2023-12-27 02:27:10,563][105620] Updated weights for policy 1, policy_version 1520998 (0.0008) [2023-12-27 02:27:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19077.6). Total num frames: 778190848. Throughput: 0: 9766.9, 1: 9878.4. Samples: 778202596. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:27:11,063][104569] Avg episode reward: [(0, '8808.701'), (1, '9262.809')] [2023-12-27 02:27:11,080][105692] Updated weights for policy 0, policy_version 1518377 (0.0007) [2023-12-27 02:27:11,146][105692] Updated weights for policy 0, policy_version 1518387 (0.0010) [2023-12-27 02:27:11,206][105692] Updated weights for policy 0, policy_version 1518397 (0.0009) [2023-12-27 02:27:11,276][105692] Updated weights for policy 0, policy_version 1518407 (0.0009) [2023-12-27 02:27:11,331][105620] Updated weights for policy 1, policy_version 1521008 (0.0009) [2023-12-27 02:27:11,396][105620] Updated weights for policy 1, policy_version 1521018 (0.0007) [2023-12-27 02:27:11,458][105620] Updated weights for policy 1, policy_version 1521028 (0.0008) [2023-12-27 02:27:12,052][105692] Updated weights for policy 0, policy_version 1518417 (0.0008) [2023-12-27 02:27:12,121][105692] Updated weights for policy 0, policy_version 1518427 (0.0009) [2023-12-27 02:27:12,182][105692] Updated weights for policy 0, policy_version 1518437 (0.0008) [2023-12-27 02:27:12,238][105620] Updated weights for policy 1, policy_version 1521038 (0.0009) [2023-12-27 02:27:12,301][105620] Updated weights for policy 1, policy_version 1521048 (0.0008) [2023-12-27 02:27:12,374][105620] Updated weights for policy 1, policy_version 1521058 (0.0009) [2023-12-27 02:27:12,997][105692] Updated weights for policy 0, policy_version 1518447 (0.0007) [2023-12-27 02:27:13,069][105692] Updated weights for policy 0, policy_version 1518457 (0.0006) [2023-12-27 02:27:13,124][105692] Updated weights for policy 0, policy_version 1518467 (0.0009) [2023-12-27 02:27:13,132][105620] Updated weights for policy 1, policy_version 1521068 (0.0008) [2023-12-27 02:27:13,191][105620] Updated weights for policy 1, policy_version 1521078 (0.0007) [2023-12-27 02:27:13,249][105620] Updated weights for policy 1, policy_version 1521088 (0.0007) [2023-12-27 02:27:13,851][105620] Updated weights for policy 1, policy_version 1521098 (0.0009) [2023-12-27 02:27:13,891][105692] Updated weights for policy 0, policy_version 1518477 (0.0008) [2023-12-27 02:27:13,898][105620] Updated weights for policy 1, policy_version 1521108 (0.0007) [2023-12-27 02:27:13,940][105692] Updated weights for policy 0, policy_version 1518487 (0.0006) [2023-12-27 02:27:13,958][105620] Updated weights for policy 1, policy_version 1521118 (0.0007) [2023-12-27 02:27:13,994][105692] Updated weights for policy 0, policy_version 1518497 (0.0009) [2023-12-27 02:27:14,020][105620] Updated weights for policy 1, policy_version 1521128 (0.0008) [2023-12-27 02:27:14,596][105692] Updated weights for policy 0, policy_version 1518507 (0.0006) [2023-12-27 02:27:14,647][105692] Updated weights for policy 0, policy_version 1518517 (0.0005) [2023-12-27 02:27:14,701][105692] Updated weights for policy 0, policy_version 1518527 (0.0005) [2023-12-27 02:27:14,856][105620] Updated weights for policy 1, policy_version 1521138 (0.0008) [2023-12-27 02:27:14,910][105620] Updated weights for policy 1, policy_version 1521148 (0.0009) [2023-12-27 02:27:14,968][105620] Updated weights for policy 1, policy_version 1521158 (0.0008) [2023-12-27 02:27:15,384][105692] Updated weights for policy 0, policy_version 1518537 (0.0006) [2023-12-27 02:27:15,448][105692] Updated weights for policy 0, policy_version 1518547 (0.0009) [2023-12-27 02:27:15,507][105692] Updated weights for policy 0, policy_version 1518557 (0.0009) [2023-12-27 02:27:15,553][105692] Updated weights for policy 0, policy_version 1518567 (0.0008) [2023-12-27 02:27:15,762][105620] Updated weights for policy 1, policy_version 1521168 (0.0009) [2023-12-27 02:27:15,811][105620] Updated weights for policy 1, policy_version 1521178 (0.0008) [2023-12-27 02:27:15,861][105620] Updated weights for policy 1, policy_version 1521188 (0.0009) [2023-12-27 02:27:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19077.6). Total num frames: 778289152. Throughput: 0: 9689.4, 1: 9821.9. Samples: 778258692. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:27:16,062][104569] Avg episode reward: [(0, '8713.156'), (1, '9258.923')] [2023-12-27 02:27:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001518568_388808704.pth... [2023-12-27 02:27:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001521192_389480448.pth... [2023-12-27 02:27:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001517416_388513792.pth [2023-12-27 02:27:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001520040_389185536.pth [2023-12-27 02:27:16,279][105692] Updated weights for policy 0, policy_version 1518577 (0.0009) [2023-12-27 02:27:16,329][105692] Updated weights for policy 0, policy_version 1518587 (0.0009) [2023-12-27 02:27:16,383][105692] Updated weights for policy 0, policy_version 1518597 (0.0009) [2023-12-27 02:27:16,629][105620] Updated weights for policy 1, policy_version 1521198 (0.0009) [2023-12-27 02:27:16,678][105620] Updated weights for policy 1, policy_version 1521208 (0.0008) [2023-12-27 02:27:16,740][105620] Updated weights for policy 1, policy_version 1521218 (0.0008) [2023-12-27 02:27:17,120][105692] Updated weights for policy 0, policy_version 1518607 (0.0008) [2023-12-27 02:27:17,167][105692] Updated weights for policy 0, policy_version 1518617 (0.0009) [2023-12-27 02:27:17,218][105692] Updated weights for policy 0, policy_version 1518627 (0.0009) [2023-12-27 02:27:17,496][105620] Updated weights for policy 1, policy_version 1521229 (0.0009) [2023-12-27 02:27:17,550][105620] Updated weights for policy 1, policy_version 1521239 (0.0009) [2023-12-27 02:27:17,614][105620] Updated weights for policy 1, policy_version 1521249 (0.0008) [2023-12-27 02:27:18,003][105692] Updated weights for policy 0, policy_version 1518637 (0.0009) [2023-12-27 02:27:18,062][105692] Updated weights for policy 0, policy_version 1518647 (0.0008) [2023-12-27 02:27:18,112][105692] Updated weights for policy 0, policy_version 1518657 (0.0006) [2023-12-27 02:27:18,350][105620] Updated weights for policy 1, policy_version 1521259 (0.0009) [2023-12-27 02:27:18,412][105620] Updated weights for policy 1, policy_version 1521269 (0.0009) [2023-12-27 02:27:18,474][105620] Updated weights for policy 1, policy_version 1521279 (0.0009) [2023-12-27 02:27:18,809][105692] Updated weights for policy 0, policy_version 1518667 (0.0007) [2023-12-27 02:27:18,875][105692] Updated weights for policy 0, policy_version 1518677 (0.0007) [2023-12-27 02:27:18,932][105692] Updated weights for policy 0, policy_version 1518687 (0.0006) [2023-12-27 02:27:19,160][105620] Updated weights for policy 1, policy_version 1521289 (0.0008) [2023-12-27 02:27:19,217][105620] Updated weights for policy 1, policy_version 1521299 (0.0006) [2023-12-27 02:27:19,283][105620] Updated weights for policy 1, policy_version 1521309 (0.0009) [2023-12-27 02:27:19,338][105620] Updated weights for policy 1, policy_version 1521319 (0.0009) [2023-12-27 02:27:19,580][105692] Updated weights for policy 0, policy_version 1518697 (0.0006) [2023-12-27 02:27:19,637][105692] Updated weights for policy 0, policy_version 1518707 (0.0008) [2023-12-27 02:27:19,693][105692] Updated weights for policy 0, policy_version 1518717 (0.0008) [2023-12-27 02:27:19,752][105692] Updated weights for policy 0, policy_version 1518727 (0.0008) [2023-12-27 02:27:20,065][105620] Updated weights for policy 1, policy_version 1521329 (0.0009) [2023-12-27 02:27:20,126][105620] Updated weights for policy 1, policy_version 1521339 (0.0006) [2023-12-27 02:27:20,187][105620] Updated weights for policy 1, policy_version 1521349 (0.0010) [2023-12-27 02:27:20,492][105692] Updated weights for policy 0, policy_version 1518737 (0.0007) [2023-12-27 02:27:20,549][105692] Updated weights for policy 0, policy_version 1518747 (0.0007) [2023-12-27 02:27:20,616][105692] Updated weights for policy 0, policy_version 1518757 (0.0007) [2023-12-27 02:27:20,880][105620] Updated weights for policy 1, policy_version 1521359 (0.0011) [2023-12-27 02:27:20,939][105620] Updated weights for policy 1, policy_version 1521369 (0.0008) [2023-12-27 02:27:21,005][105620] Updated weights for policy 1, policy_version 1521379 (0.0008) [2023-12-27 02:27:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19077.6). Total num frames: 778387456. Throughput: 0: 9679.1, 1: 9765.9. Samples: 778374188. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:27:21,062][104569] Avg episode reward: [(0, '8808.360'), (1, '8987.173')] [2023-12-27 02:27:21,432][105692] Updated weights for policy 0, policy_version 1518767 (0.0006) [2023-12-27 02:27:21,500][105692] Updated weights for policy 0, policy_version 1518777 (0.0008) [2023-12-27 02:27:21,554][105692] Updated weights for policy 0, policy_version 1518787 (0.0009) [2023-12-27 02:27:21,707][105620] Updated weights for policy 1, policy_version 1521389 (0.0009) [2023-12-27 02:27:21,773][105620] Updated weights for policy 1, policy_version 1521399 (0.0010) [2023-12-27 02:27:21,839][105620] Updated weights for policy 1, policy_version 1521409 (0.0009) [2023-12-27 02:27:22,180][105692] Updated weights for policy 0, policy_version 1518797 (0.0007) [2023-12-27 02:27:22,244][105692] Updated weights for policy 0, policy_version 1518807 (0.0008) [2023-12-27 02:27:22,311][105692] Updated weights for policy 0, policy_version 1518817 (0.0008) [2023-12-27 02:27:22,635][105620] Updated weights for policy 1, policy_version 1521419 (0.0007) [2023-12-27 02:27:22,693][105620] Updated weights for policy 1, policy_version 1521429 (0.0009) [2023-12-27 02:27:22,752][105620] Updated weights for policy 1, policy_version 1521439 (0.0009) [2023-12-27 02:27:23,007][105692] Updated weights for policy 0, policy_version 1518827 (0.0009) [2023-12-27 02:27:23,077][105692] Updated weights for policy 0, policy_version 1518837 (0.0008) [2023-12-27 02:27:23,143][105692] Updated weights for policy 0, policy_version 1518847 (0.0008) [2023-12-27 02:27:23,441][105620] Updated weights for policy 1, policy_version 1521449 (0.0008) [2023-12-27 02:27:23,506][105620] Updated weights for policy 1, policy_version 1521459 (0.0009) [2023-12-27 02:27:23,564][105620] Updated weights for policy 1, policy_version 1521469 (0.0008) [2023-12-27 02:27:23,621][105620] Updated weights for policy 1, policy_version 1521479 (0.0005) [2023-12-27 02:27:23,802][105692] Updated weights for policy 0, policy_version 1518857 (0.0008) [2023-12-27 02:27:23,855][105692] Updated weights for policy 0, policy_version 1518867 (0.0009) [2023-12-27 02:27:23,915][105692] Updated weights for policy 0, policy_version 1518877 (0.0010) [2023-12-27 02:27:23,983][105692] Updated weights for policy 0, policy_version 1518887 (0.0006) [2023-12-27 02:27:24,196][105620] Updated weights for policy 1, policy_version 1521489 (0.0005) [2023-12-27 02:27:24,256][105620] Updated weights for policy 1, policy_version 1521499 (0.0006) [2023-12-27 02:27:24,306][105620] Updated weights for policy 1, policy_version 1521509 (0.0005) [2023-12-27 02:27:24,593][105692] Updated weights for policy 0, policy_version 1518897 (0.0010) [2023-12-27 02:27:24,655][105692] Updated weights for policy 0, policy_version 1518907 (0.0010) [2023-12-27 02:27:24,726][105692] Updated weights for policy 0, policy_version 1518917 (0.0010) [2023-12-27 02:27:24,852][105620] Updated weights for policy 1, policy_version 1521519 (0.0008) [2023-12-27 02:27:24,900][105620] Updated weights for policy 1, policy_version 1521529 (0.0008) [2023-12-27 02:27:24,947][105620] Updated weights for policy 1, policy_version 1521540 (0.0007) [2023-12-27 02:27:25,372][105692] Updated weights for policy 0, policy_version 1518927 (0.0007) [2023-12-27 02:27:25,422][105692] Updated weights for policy 0, policy_version 1518937 (0.0005) [2023-12-27 02:27:25,475][105692] Updated weights for policy 0, policy_version 1518947 (0.0008) [2023-12-27 02:27:25,729][105620] Updated weights for policy 1, policy_version 1521550 (0.0008) [2023-12-27 02:27:25,774][105620] Updated weights for policy 1, policy_version 1521560 (0.0008) [2023-12-27 02:27:25,824][105620] Updated weights for policy 1, policy_version 1521570 (0.0006) [2023-12-27 02:27:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19077.6). Total num frames: 778485760. Throughput: 0: 9773.0, 1: 9824.4. Samples: 778494404. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:27:26,062][104569] Avg episode reward: [(0, '8263.447'), (1, '8990.235')] [2023-12-27 02:27:26,096][105692] Updated weights for policy 0, policy_version 1518957 (0.0009) [2023-12-27 02:27:26,153][105692] Updated weights for policy 0, policy_version 1518967 (0.0009) [2023-12-27 02:27:26,213][105692] Updated weights for policy 0, policy_version 1518977 (0.0008) [2023-12-27 02:27:26,474][105620] Updated weights for policy 1, policy_version 1521580 (0.0007) [2023-12-27 02:27:26,522][105620] Updated weights for policy 1, policy_version 1521590 (0.0009) [2023-12-27 02:27:26,581][105620] Updated weights for policy 1, policy_version 1521600 (0.0009) [2023-12-27 02:27:26,953][105692] Updated weights for policy 0, policy_version 1518987 (0.0007) [2023-12-27 02:27:27,003][105692] Updated weights for policy 0, policy_version 1518997 (0.0009) [2023-12-27 02:27:27,049][105692] Updated weights for policy 0, policy_version 1519007 (0.0009) [2023-12-27 02:27:27,252][105620] Updated weights for policy 1, policy_version 1521610 (0.0009) [2023-12-27 02:27:27,311][105620] Updated weights for policy 1, policy_version 1521620 (0.0009) [2023-12-27 02:27:27,366][105620] Updated weights for policy 1, policy_version 1521630 (0.0009) [2023-12-27 02:27:27,422][105620] Updated weights for policy 1, policy_version 1521640 (0.0009) [2023-12-27 02:27:27,842][105692] Updated weights for policy 0, policy_version 1519017 (0.0009) [2023-12-27 02:27:27,897][105692] Updated weights for policy 0, policy_version 1519027 (0.0007) [2023-12-27 02:27:27,963][105692] Updated weights for policy 0, policy_version 1519037 (0.0005) [2023-12-27 02:27:28,019][105692] Updated weights for policy 0, policy_version 1519047 (0.0007) [2023-12-27 02:27:28,132][105620] Updated weights for policy 1, policy_version 1521650 (0.0006) [2023-12-27 02:27:28,193][105620] Updated weights for policy 1, policy_version 1521660 (0.0005) [2023-12-27 02:27:28,250][105620] Updated weights for policy 1, policy_version 1521670 (0.0007) [2023-12-27 02:27:28,776][105692] Updated weights for policy 0, policy_version 1519057 (0.0008) [2023-12-27 02:27:28,831][105692] Updated weights for policy 0, policy_version 1519067 (0.0008) [2023-12-27 02:27:28,890][105692] Updated weights for policy 0, policy_version 1519077 (0.0008) [2023-12-27 02:27:28,978][105620] Updated weights for policy 1, policy_version 1521680 (0.0007) [2023-12-27 02:27:29,028][105620] Updated weights for policy 1, policy_version 1521690 (0.0005) [2023-12-27 02:27:29,081][105620] Updated weights for policy 1, policy_version 1521700 (0.0005) [2023-12-27 02:27:29,675][105620] Updated weights for policy 1, policy_version 1521710 (0.0005) [2023-12-27 02:27:29,738][105620] Updated weights for policy 1, policy_version 1521720 (0.0007) [2023-12-27 02:27:29,748][105692] Updated weights for policy 0, policy_version 1519087 (0.0008) [2023-12-27 02:27:29,795][105620] Updated weights for policy 1, policy_version 1521730 (0.0007) [2023-12-27 02:27:29,805][105692] Updated weights for policy 0, policy_version 1519097 (0.0006) [2023-12-27 02:27:29,869][105692] Updated weights for policy 0, policy_version 1519107 (0.0007) [2023-12-27 02:27:30,545][105620] Updated weights for policy 1, policy_version 1521740 (0.0007) [2023-12-27 02:27:30,565][105692] Updated weights for policy 0, policy_version 1519117 (0.0010) [2023-12-27 02:27:30,608][105620] Updated weights for policy 1, policy_version 1521750 (0.0006) [2023-12-27 02:27:30,623][105692] Updated weights for policy 0, policy_version 1519127 (0.0007) [2023-12-27 02:27:30,663][105620] Updated weights for policy 1, policy_version 1521760 (0.0007) [2023-12-27 02:27:30,674][105692] Updated weights for policy 0, policy_version 1519137 (0.0007) [2023-12-27 02:27:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19077.6). Total num frames: 778584064. Throughput: 0: 9822.0, 1: 9892.5. Samples: 778553596. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:27:31,062][104569] Avg episode reward: [(0, '8169.744'), (1, '8993.230')] [2023-12-27 02:27:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001519144_388956160.pth... [2023-12-27 02:27:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001521768_389627904.pth... [2023-12-27 02:27:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001518024_388669440.pth [2023-12-27 02:27:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001520616_389332992.pth [2023-12-27 02:27:31,352][105620] Updated weights for policy 1, policy_version 1521770 (0.0007) [2023-12-27 02:27:31,412][105620] Updated weights for policy 1, policy_version 1521780 (0.0009) [2023-12-27 02:27:31,439][105692] Updated weights for policy 0, policy_version 1519147 (0.0009) [2023-12-27 02:27:31,469][105620] Updated weights for policy 1, policy_version 1521790 (0.0008) [2023-12-27 02:27:31,488][105692] Updated weights for policy 0, policy_version 1519157 (0.0006) [2023-12-27 02:27:31,529][105620] Updated weights for policy 1, policy_version 1521800 (0.0007) [2023-12-27 02:27:31,547][105692] Updated weights for policy 0, policy_version 1519167 (0.0009) [2023-12-27 02:27:32,247][105692] Updated weights for policy 0, policy_version 1519177 (0.0009) [2023-12-27 02:27:32,301][105620] Updated weights for policy 1, policy_version 1521810 (0.0006) [2023-12-27 02:27:32,312][105692] Updated weights for policy 0, policy_version 1519187 (0.0007) [2023-12-27 02:27:32,356][105620] Updated weights for policy 1, policy_version 1521820 (0.0009) [2023-12-27 02:27:32,374][105692] Updated weights for policy 0, policy_version 1519197 (0.0010) [2023-12-27 02:27:32,414][105620] Updated weights for policy 1, policy_version 1521830 (0.0010) [2023-12-27 02:27:32,430][105692] Updated weights for policy 0, policy_version 1519207 (0.0010) [2023-12-27 02:27:32,982][105620] Updated weights for policy 1, policy_version 1521840 (0.0010) [2023-12-27 02:27:33,040][105620] Updated weights for policy 1, policy_version 1521850 (0.0008) [2023-12-27 02:27:33,114][105620] Updated weights for policy 1, policy_version 1521860 (0.0008) [2023-12-27 02:27:33,168][105692] Updated weights for policy 0, policy_version 1519217 (0.0010) [2023-12-27 02:27:33,215][105692] Updated weights for policy 0, policy_version 1519227 (0.0010) [2023-12-27 02:27:33,273][105692] Updated weights for policy 0, policy_version 1519237 (0.0010) [2023-12-27 02:27:33,743][105620] Updated weights for policy 1, policy_version 1521870 (0.0009) [2023-12-27 02:27:33,800][105620] Updated weights for policy 1, policy_version 1521880 (0.0009) [2023-12-27 02:27:33,846][105620] Updated weights for policy 1, policy_version 1521890 (0.0005) [2023-12-27 02:27:33,943][105692] Updated weights for policy 0, policy_version 1519247 (0.0009) [2023-12-27 02:27:33,994][105692] Updated weights for policy 0, policy_version 1519257 (0.0005) [2023-12-27 02:27:34,045][105692] Updated weights for policy 0, policy_version 1519267 (0.0005) [2023-12-27 02:27:34,490][105620] Updated weights for policy 1, policy_version 1521900 (0.0010) [2023-12-27 02:27:34,557][105620] Updated weights for policy 1, policy_version 1521910 (0.0010) [2023-12-27 02:27:34,616][105620] Updated weights for policy 1, policy_version 1521920 (0.0010) [2023-12-27 02:27:34,765][105692] Updated weights for policy 0, policy_version 1519277 (0.0008) [2023-12-27 02:27:34,826][105692] Updated weights for policy 0, policy_version 1519287 (0.0010) [2023-12-27 02:27:34,885][105692] Updated weights for policy 0, policy_version 1519297 (0.0010) [2023-12-27 02:27:35,350][105620] Updated weights for policy 1, policy_version 1521930 (0.0010) [2023-12-27 02:27:35,401][105620] Updated weights for policy 1, policy_version 1521940 (0.0010) [2023-12-27 02:27:35,465][105620] Updated weights for policy 1, policy_version 1521950 (0.0008) [2023-12-27 02:27:35,531][105620] Updated weights for policy 1, policy_version 1521960 (0.0007) [2023-12-27 02:27:35,568][105692] Updated weights for policy 0, policy_version 1519307 (0.0010) [2023-12-27 02:27:35,619][105692] Updated weights for policy 0, policy_version 1519317 (0.0010) [2023-12-27 02:27:35,670][105692] Updated weights for policy 0, policy_version 1519327 (0.0010) [2023-12-27 02:27:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19077.6). Total num frames: 778682368. Throughput: 0: 9756.3, 1: 9971.4. Samples: 778672112. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:27:36,063][104569] Avg episode reward: [(0, '8259.926'), (1, '9082.354')] [2023-12-27 02:27:36,207][105620] Updated weights for policy 1, policy_version 1521970 (0.0009) [2023-12-27 02:27:36,259][105620] Updated weights for policy 1, policy_version 1521980 (0.0010) [2023-12-27 02:27:36,315][105620] Updated weights for policy 1, policy_version 1521990 (0.0010) [2023-12-27 02:27:36,403][105692] Updated weights for policy 0, policy_version 1519337 (0.0010) [2023-12-27 02:27:36,463][105692] Updated weights for policy 0, policy_version 1519347 (0.0011) [2023-12-27 02:27:36,532][105692] Updated weights for policy 0, policy_version 1519357 (0.0010) [2023-12-27 02:27:36,607][105692] Updated weights for policy 0, policy_version 1519367 (0.0008) [2023-12-27 02:27:36,966][105620] Updated weights for policy 1, policy_version 1522000 (0.0008) [2023-12-27 02:27:37,032][105620] Updated weights for policy 1, policy_version 1522010 (0.0007) [2023-12-27 02:27:37,095][105620] Updated weights for policy 1, policy_version 1522020 (0.0007) [2023-12-27 02:27:37,305][105692] Updated weights for policy 0, policy_version 1519377 (0.0010) [2023-12-27 02:27:37,368][105692] Updated weights for policy 0, policy_version 1519387 (0.0011) [2023-12-27 02:27:37,425][105692] Updated weights for policy 0, policy_version 1519397 (0.0011) [2023-12-27 02:27:37,741][105620] Updated weights for policy 1, policy_version 1522030 (0.0007) [2023-12-27 02:27:37,804][105620] Updated weights for policy 1, policy_version 1522040 (0.0005) [2023-12-27 02:27:37,868][105620] Updated weights for policy 1, policy_version 1522050 (0.0006) [2023-12-27 02:27:38,072][105692] Updated weights for policy 0, policy_version 1519407 (0.0009) [2023-12-27 02:27:38,128][105692] Updated weights for policy 0, policy_version 1519417 (0.0010) [2023-12-27 02:27:38,181][105692] Updated weights for policy 0, policy_version 1519427 (0.0010) [2023-12-27 02:27:38,429][105620] Updated weights for policy 1, policy_version 1522060 (0.0005) [2023-12-27 02:27:38,494][105620] Updated weights for policy 1, policy_version 1522070 (0.0005) [2023-12-27 02:27:38,555][105620] Updated weights for policy 1, policy_version 1522080 (0.0006) [2023-12-27 02:27:38,906][105692] Updated weights for policy 0, policy_version 1519437 (0.0010) [2023-12-27 02:27:38,965][105692] Updated weights for policy 0, policy_version 1519447 (0.0010) [2023-12-27 02:27:39,017][105692] Updated weights for policy 0, policy_version 1519457 (0.0009) [2023-12-27 02:27:39,158][105620] Updated weights for policy 1, policy_version 1522090 (0.0006) [2023-12-27 02:27:39,209][105620] Updated weights for policy 1, policy_version 1522100 (0.0006) [2023-12-27 02:27:39,271][105620] Updated weights for policy 1, policy_version 1522110 (0.0009) [2023-12-27 02:27:39,321][105620] Updated weights for policy 1, policy_version 1522120 (0.0008) [2023-12-27 02:27:39,828][105692] Updated weights for policy 0, policy_version 1519467 (0.0007) [2023-12-27 02:27:39,893][105692] Updated weights for policy 0, policy_version 1519477 (0.0009) [2023-12-27 02:27:39,955][105692] Updated weights for policy 0, policy_version 1519487 (0.0009) [2023-12-27 02:27:40,015][105620] Updated weights for policy 1, policy_version 1522130 (0.0010) [2023-12-27 02:27:40,078][105620] Updated weights for policy 1, policy_version 1522140 (0.0011) [2023-12-27 02:27:40,131][105620] Updated weights for policy 1, policy_version 1522150 (0.0011) [2023-12-27 02:27:40,745][105692] Updated weights for policy 0, policy_version 1519497 (0.0010) [2023-12-27 02:27:40,810][105692] Updated weights for policy 0, policy_version 1519507 (0.0007) [2023-12-27 02:27:40,815][105620] Updated weights for policy 1, policy_version 1522160 (0.0007) [2023-12-27 02:27:40,870][105692] Updated weights for policy 0, policy_version 1519517 (0.0010) [2023-12-27 02:27:40,877][105620] Updated weights for policy 1, policy_version 1522170 (0.0006) [2023-12-27 02:27:40,927][105692] Updated weights for policy 0, policy_version 1519527 (0.0011) [2023-12-27 02:27:40,932][105620] Updated weights for policy 1, policy_version 1522180 (0.0006) [2023-12-27 02:27:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19133.2). Total num frames: 778788864. Throughput: 0: 9687.6, 1: 10021.0. Samples: 778792356. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:27:41,062][104569] Avg episode reward: [(0, '8352.856'), (1, '9077.786')] [2023-12-27 02:27:41,669][105620] Updated weights for policy 1, policy_version 1522190 (0.0008) [2023-12-27 02:27:41,689][105692] Updated weights for policy 0, policy_version 1519537 (0.0008) [2023-12-27 02:27:41,737][105620] Updated weights for policy 1, policy_version 1522200 (0.0010) [2023-12-27 02:27:41,755][105692] Updated weights for policy 0, policy_version 1519547 (0.0008) [2023-12-27 02:27:41,798][105620] Updated weights for policy 1, policy_version 1522210 (0.0006) [2023-12-27 02:27:41,823][105692] Updated weights for policy 0, policy_version 1519557 (0.0009) [2023-12-27 02:27:42,510][105620] Updated weights for policy 1, policy_version 1522220 (0.0008) [2023-12-27 02:27:42,565][105620] Updated weights for policy 1, policy_version 1522230 (0.0009) [2023-12-27 02:27:42,609][105692] Updated weights for policy 0, policy_version 1519567 (0.0007) [2023-12-27 02:27:42,621][105620] Updated weights for policy 1, policy_version 1522240 (0.0008) [2023-12-27 02:27:42,663][105692] Updated weights for policy 0, policy_version 1519577 (0.0007) [2023-12-27 02:27:42,717][105692] Updated weights for policy 0, policy_version 1519587 (0.0009) [2023-12-27 02:27:43,382][105620] Updated weights for policy 1, policy_version 1522250 (0.0008) [2023-12-27 02:27:43,437][105620] Updated weights for policy 1, policy_version 1522260 (0.0009) [2023-12-27 02:27:43,474][105692] Updated weights for policy 0, policy_version 1519597 (0.0007) [2023-12-27 02:27:43,488][105620] Updated weights for policy 1, policy_version 1522270 (0.0007) [2023-12-27 02:27:43,531][105692] Updated weights for policy 0, policy_version 1519607 (0.0008) [2023-12-27 02:27:43,537][105620] Updated weights for policy 1, policy_version 1522280 (0.0006) [2023-12-27 02:27:43,591][105692] Updated weights for policy 0, policy_version 1519617 (0.0009) [2023-12-27 02:27:44,159][105620] Updated weights for policy 1, policy_version 1522290 (0.0009) [2023-12-27 02:27:44,214][105620] Updated weights for policy 1, policy_version 1522300 (0.0008) [2023-12-27 02:27:44,242][105692] Updated weights for policy 0, policy_version 1519627 (0.0011) [2023-12-27 02:27:44,272][105620] Updated weights for policy 1, policy_version 1522310 (0.0007) [2023-12-27 02:27:44,290][105692] Updated weights for policy 0, policy_version 1519637 (0.0010) [2023-12-27 02:27:44,335][105692] Updated weights for policy 0, policy_version 1519647 (0.0010) [2023-12-27 02:27:44,908][105620] Updated weights for policy 1, policy_version 1522320 (0.0006) [2023-12-27 02:27:44,973][105620] Updated weights for policy 1, policy_version 1522330 (0.0008) [2023-12-27 02:27:45,038][105620] Updated weights for policy 1, policy_version 1522340 (0.0009) [2023-12-27 02:27:45,133][105692] Updated weights for policy 0, policy_version 1519657 (0.0010) [2023-12-27 02:27:45,198][105692] Updated weights for policy 0, policy_version 1519667 (0.0011) [2023-12-27 02:27:45,269][105692] Updated weights for policy 0, policy_version 1519677 (0.0011) [2023-12-27 02:27:45,331][105692] Updated weights for policy 0, policy_version 1519687 (0.0011) [2023-12-27 02:27:45,680][105620] Updated weights for policy 1, policy_version 1522350 (0.0011) [2023-12-27 02:27:45,736][105620] Updated weights for policy 1, policy_version 1522360 (0.0010) [2023-12-27 02:27:45,791][105620] Updated weights for policy 1, policy_version 1522370 (0.0006) [2023-12-27 02:27:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19105.4). Total num frames: 778878976. Throughput: 0: 9660.3, 1: 9970.3. Samples: 778847864. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:27:46,062][104569] Avg episode reward: [(0, '8810.823'), (1, '8985.981')] [2023-12-27 02:27:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001522376_389783552.pth... [2023-12-27 02:27:46,069][105692] Updated weights for policy 0, policy_version 1519697 (0.0010) [2023-12-27 02:27:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001521192_389480448.pth [2023-12-27 02:27:46,129][105692] Updated weights for policy 0, policy_version 1519707 (0.0009) [2023-12-27 02:27:46,182][105692] Updated weights for policy 0, policy_version 1519717 (0.0008) [2023-12-27 02:27:46,197][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001519720_389103616.pth... [2023-12-27 02:27:46,201][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001518568_388808704.pth [2023-12-27 02:27:46,481][105620] Updated weights for policy 1, policy_version 1522380 (0.0010) [2023-12-27 02:27:46,536][105620] Updated weights for policy 1, policy_version 1522390 (0.0010) [2023-12-27 02:27:46,605][105620] Updated weights for policy 1, policy_version 1522400 (0.0010) [2023-12-27 02:27:46,772][105692] Updated weights for policy 0, policy_version 1519727 (0.0006) [2023-12-27 02:27:46,835][105692] Updated weights for policy 0, policy_version 1519737 (0.0005) [2023-12-27 02:27:46,884][105692] Updated weights for policy 0, policy_version 1519747 (0.0005) [2023-12-27 02:27:47,396][105620] Updated weights for policy 1, policy_version 1522410 (0.0009) [2023-12-27 02:27:47,417][105692] Updated weights for policy 0, policy_version 1519757 (0.0006) [2023-12-27 02:27:47,453][105620] Updated weights for policy 1, policy_version 1522420 (0.0007) [2023-12-27 02:27:47,479][105692] Updated weights for policy 0, policy_version 1519767 (0.0005) [2023-12-27 02:27:47,509][105620] Updated weights for policy 1, policy_version 1522430 (0.0009) [2023-12-27 02:27:47,533][105692] Updated weights for policy 0, policy_version 1519777 (0.0005) [2023-12-27 02:27:47,563][105620] Updated weights for policy 1, policy_version 1522440 (0.0008) [2023-12-27 02:27:48,261][105692] Updated weights for policy 0, policy_version 1519787 (0.0007) [2023-12-27 02:27:48,303][105620] Updated weights for policy 1, policy_version 1522450 (0.0007) [2023-12-27 02:27:48,325][105692] Updated weights for policy 0, policy_version 1519797 (0.0008) [2023-12-27 02:27:48,359][105620] Updated weights for policy 1, policy_version 1522460 (0.0008) [2023-12-27 02:27:48,391][105692] Updated weights for policy 0, policy_version 1519807 (0.0008) [2023-12-27 02:27:48,413][105620] Updated weights for policy 1, policy_version 1522470 (0.0007) [2023-12-27 02:27:49,145][105692] Updated weights for policy 0, policy_version 1519817 (0.0007) [2023-12-27 02:27:49,192][105620] Updated weights for policy 1, policy_version 1522480 (0.0008) [2023-12-27 02:27:49,210][105692] Updated weights for policy 0, policy_version 1519827 (0.0006) [2023-12-27 02:27:49,254][105620] Updated weights for policy 1, policy_version 1522490 (0.0007) [2023-12-27 02:27:49,271][105692] Updated weights for policy 0, policy_version 1519837 (0.0008) [2023-12-27 02:27:49,320][105620] Updated weights for policy 1, policy_version 1522500 (0.0008) [2023-12-27 02:27:49,321][105692] Updated weights for policy 0, policy_version 1519847 (0.0008) [2023-12-27 02:27:49,978][105692] Updated weights for policy 0, policy_version 1519857 (0.0008) [2023-12-27 02:27:50,034][105692] Updated weights for policy 0, policy_version 1519867 (0.0009) [2023-12-27 02:27:50,084][105692] Updated weights for policy 0, policy_version 1519877 (0.0008) [2023-12-27 02:27:50,098][105620] Updated weights for policy 1, policy_version 1522510 (0.0007) [2023-12-27 02:27:50,160][105620] Updated weights for policy 1, policy_version 1522520 (0.0009) [2023-12-27 02:27:50,224][105620] Updated weights for policy 1, policy_version 1522530 (0.0008) [2023-12-27 02:27:50,879][105692] Updated weights for policy 0, policy_version 1519887 (0.0008) [2023-12-27 02:27:50,936][105692] Updated weights for policy 0, policy_version 1519897 (0.0009) [2023-12-27 02:27:50,954][105620] Updated weights for policy 1, policy_version 1522540 (0.0008) [2023-12-27 02:27:50,997][105692] Updated weights for policy 0, policy_version 1519907 (0.0007) [2023-12-27 02:27:51,007][105620] Updated weights for policy 1, policy_version 1522550 (0.0006) [2023-12-27 02:27:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.3, 300 sec: 19105.4). Total num frames: 778977280. Throughput: 0: 9643.7, 1: 9944.5. Samples: 778966900. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:27:51,062][104569] Avg episode reward: [(0, '8718.082'), (1, '9078.482')] [2023-12-27 02:27:51,070][105620] Updated weights for policy 1, policy_version 1522560 (0.0007) [2023-12-27 02:27:51,758][105620] Updated weights for policy 1, policy_version 1522570 (0.0007) [2023-12-27 02:27:51,814][105620] Updated weights for policy 1, policy_version 1522580 (0.0009) [2023-12-27 02:27:51,855][105692] Updated weights for policy 0, policy_version 1519917 (0.0008) [2023-12-27 02:27:51,869][105620] Updated weights for policy 1, policy_version 1522590 (0.0009) [2023-12-27 02:27:51,914][105692] Updated weights for policy 0, policy_version 1519927 (0.0007) [2023-12-27 02:27:51,916][105620] Updated weights for policy 1, policy_version 1522600 (0.0008) [2023-12-27 02:27:51,968][105692] Updated weights for policy 0, policy_version 1519937 (0.0009) [2023-12-27 02:27:52,625][105620] Updated weights for policy 1, policy_version 1522610 (0.0009) [2023-12-27 02:27:52,683][105620] Updated weights for policy 1, policy_version 1522620 (0.0009) [2023-12-27 02:27:52,711][105692] Updated weights for policy 0, policy_version 1519947 (0.0008) [2023-12-27 02:27:52,743][105620] Updated weights for policy 1, policy_version 1522630 (0.0008) [2023-12-27 02:27:52,770][105692] Updated weights for policy 0, policy_version 1519957 (0.0006) [2023-12-27 02:27:52,824][105692] Updated weights for policy 0, policy_version 1519967 (0.0010) [2023-12-27 02:27:53,415][105620] Updated weights for policy 1, policy_version 1522640 (0.0008) [2023-12-27 02:27:53,477][105620] Updated weights for policy 1, policy_version 1522650 (0.0009) [2023-12-27 02:27:53,511][105692] Updated weights for policy 0, policy_version 1519977 (0.0010) [2023-12-27 02:27:53,531][105620] Updated weights for policy 1, policy_version 1522660 (0.0010) [2023-12-27 02:27:53,568][105692] Updated weights for policy 0, policy_version 1519987 (0.0005) [2023-12-27 02:27:53,623][105692] Updated weights for policy 0, policy_version 1519997 (0.0005) [2023-12-27 02:27:53,677][105692] Updated weights for policy 0, policy_version 1520007 (0.0005) [2023-12-27 02:27:54,234][105692] Updated weights for policy 0, policy_version 1520017 (0.0005) [2023-12-27 02:27:54,290][105692] Updated weights for policy 0, policy_version 1520027 (0.0009) [2023-12-27 02:27:54,335][105692] Updated weights for policy 0, policy_version 1520037 (0.0010) [2023-12-27 02:27:54,401][105620] Updated weights for policy 1, policy_version 1522670 (0.0007) [2023-12-27 02:27:54,471][105620] Updated weights for policy 1, policy_version 1522680 (0.0006) [2023-12-27 02:27:54,541][105620] Updated weights for policy 1, policy_version 1522690 (0.0007) [2023-12-27 02:27:54,975][105692] Updated weights for policy 0, policy_version 1520047 (0.0007) [2023-12-27 02:27:55,035][105692] Updated weights for policy 0, policy_version 1520057 (0.0008) [2023-12-27 02:27:55,092][105692] Updated weights for policy 0, policy_version 1520067 (0.0008) [2023-12-27 02:27:55,136][105620] Updated weights for policy 1, policy_version 1522700 (0.0008) [2023-12-27 02:27:55,197][105620] Updated weights for policy 1, policy_version 1522710 (0.0008) [2023-12-27 02:27:55,257][105620] Updated weights for policy 1, policy_version 1522720 (0.0009) [2023-12-27 02:27:55,716][105692] Updated weights for policy 0, policy_version 1520077 (0.0008) [2023-12-27 02:27:55,774][105692] Updated weights for policy 0, policy_version 1520087 (0.0005) [2023-12-27 02:27:55,817][105692] Updated weights for policy 0, policy_version 1520097 (0.0005) [2023-12-27 02:27:55,925][105620] Updated weights for policy 1, policy_version 1522730 (0.0008) [2023-12-27 02:27:55,984][105620] Updated weights for policy 1, policy_version 1522740 (0.0005) [2023-12-27 02:27:56,048][105620] Updated weights for policy 1, policy_version 1522750 (0.0005) [2023-12-27 02:27:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19133.2). Total num frames: 779075584. Throughput: 0: 9736.0, 1: 9899.5. Samples: 779086192. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:27:56,062][104569] Avg episode reward: [(0, '8625.468'), (1, '9170.492')] [2023-12-27 02:27:56,108][105620] Updated weights for policy 1, policy_version 1522760 (0.0005) [2023-12-27 02:27:56,519][105692] Updated weights for policy 0, policy_version 1520107 (0.0007) [2023-12-27 02:27:56,566][105692] Updated weights for policy 0, policy_version 1520117 (0.0008) [2023-12-27 02:27:56,619][105692] Updated weights for policy 0, policy_version 1520128 (0.0011) [2023-12-27 02:27:56,690][105620] Updated weights for policy 1, policy_version 1522770 (0.0005) [2023-12-27 02:27:56,745][105620] Updated weights for policy 1, policy_version 1522780 (0.0006) [2023-12-27 02:27:56,790][105620] Updated weights for policy 1, policy_version 1522790 (0.0008) [2023-12-27 02:27:57,398][105692] Updated weights for policy 0, policy_version 1520138 (0.0009) [2023-12-27 02:27:57,460][105692] Updated weights for policy 0, policy_version 1520148 (0.0009) [2023-12-27 02:27:57,504][105692] Updated weights for policy 0, policy_version 1520158 (0.0010) [2023-12-27 02:27:57,527][105620] Updated weights for policy 1, policy_version 1522800 (0.0007) [2023-12-27 02:27:57,552][105692] Updated weights for policy 0, policy_version 1520168 (0.0010) [2023-12-27 02:27:57,576][105620] Updated weights for policy 1, policy_version 1522810 (0.0006) [2023-12-27 02:27:57,630][105620] Updated weights for policy 1, policy_version 1522820 (0.0007) [2023-12-27 02:27:58,216][105692] Updated weights for policy 0, policy_version 1520178 (0.0007) [2023-12-27 02:27:58,275][105692] Updated weights for policy 0, policy_version 1520188 (0.0008) [2023-12-27 02:27:58,338][105692] Updated weights for policy 0, policy_version 1520198 (0.0008) [2023-12-27 02:27:58,451][105620] Updated weights for policy 1, policy_version 1522830 (0.0008) [2023-12-27 02:27:58,517][105620] Updated weights for policy 1, policy_version 1522840 (0.0008) [2023-12-27 02:27:58,593][105620] Updated weights for policy 1, policy_version 1522850 (0.0008) [2023-12-27 02:27:59,112][105692] Updated weights for policy 0, policy_version 1520208 (0.0008) [2023-12-27 02:27:59,181][105692] Updated weights for policy 0, policy_version 1520218 (0.0008) [2023-12-27 02:27:59,249][105692] Updated weights for policy 0, policy_version 1520228 (0.0009) [2023-12-27 02:27:59,461][105620] Updated weights for policy 1, policy_version 1522860 (0.0007) [2023-12-27 02:27:59,531][105620] Updated weights for policy 1, policy_version 1522870 (0.0005) [2023-12-27 02:27:59,598][105620] Updated weights for policy 1, policy_version 1522880 (0.0006) [2023-12-27 02:28:00,025][105692] Updated weights for policy 0, policy_version 1520238 (0.0008) [2023-12-27 02:28:00,074][105692] Updated weights for policy 0, policy_version 1520248 (0.0009) [2023-12-27 02:28:00,126][105692] Updated weights for policy 0, policy_version 1520258 (0.0009) [2023-12-27 02:28:00,270][105620] Updated weights for policy 1, policy_version 1522890 (0.0009) [2023-12-27 02:28:00,326][105620] Updated weights for policy 1, policy_version 1522900 (0.0011) [2023-12-27 02:28:00,385][105620] Updated weights for policy 1, policy_version 1522910 (0.0011) [2023-12-27 02:28:00,440][105620] Updated weights for policy 1, policy_version 1522920 (0.0010) [2023-12-27 02:28:00,884][105692] Updated weights for policy 0, policy_version 1520268 (0.0009) [2023-12-27 02:28:00,939][105692] Updated weights for policy 0, policy_version 1520278 (0.0010) [2023-12-27 02:28:00,990][105692] Updated weights for policy 0, policy_version 1520288 (0.0007) [2023-12-27 02:28:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19133.2). Total num frames: 779173888. Throughput: 0: 9784.6, 1: 9895.8. Samples: 779144312. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:28:01,062][104569] Avg episode reward: [(0, '8262.273'), (1, '9260.182')] [2023-12-27 02:28:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001520296_389251072.pth... [2023-12-27 02:28:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001519144_388956160.pth [2023-12-27 02:28:01,143][105620] Updated weights for policy 1, policy_version 1522930 (0.0008) [2023-12-27 02:28:01,206][105620] Updated weights for policy 1, policy_version 1522940 (0.0008) [2023-12-27 02:28:01,270][105620] Updated weights for policy 1, policy_version 1522950 (0.0009) [2023-12-27 02:28:01,281][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001522952_389931008.pth... [2023-12-27 02:28:01,285][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001521768_389627904.pth [2023-12-27 02:28:01,672][105692] Updated weights for policy 0, policy_version 1520298 (0.0007) [2023-12-27 02:28:01,739][105692] Updated weights for policy 0, policy_version 1520308 (0.0010) [2023-12-27 02:28:01,796][105692] Updated weights for policy 0, policy_version 1520318 (0.0008) [2023-12-27 02:28:01,855][105692] Updated weights for policy 0, policy_version 1520328 (0.0008) [2023-12-27 02:28:02,046][105620] Updated weights for policy 1, policy_version 1522960 (0.0011) [2023-12-27 02:28:02,094][105620] Updated weights for policy 1, policy_version 1522970 (0.0011) [2023-12-27 02:28:02,146][105620] Updated weights for policy 1, policy_version 1522980 (0.0011) [2023-12-27 02:28:02,614][105692] Updated weights for policy 0, policy_version 1520338 (0.0009) [2023-12-27 02:28:02,669][105692] Updated weights for policy 0, policy_version 1520348 (0.0009) [2023-12-27 02:28:02,727][105692] Updated weights for policy 0, policy_version 1520358 (0.0010) [2023-12-27 02:28:02,833][105620] Updated weights for policy 1, policy_version 1522990 (0.0009) [2023-12-27 02:28:02,886][105620] Updated weights for policy 1, policy_version 1523000 (0.0009) [2023-12-27 02:28:02,946][105620] Updated weights for policy 1, policy_version 1523010 (0.0007) [2023-12-27 02:28:03,450][105692] Updated weights for policy 0, policy_version 1520368 (0.0007) [2023-12-27 02:28:03,488][105620] Updated weights for policy 1, policy_version 1523020 (0.0005) [2023-12-27 02:28:03,504][105692] Updated weights for policy 0, policy_version 1520378 (0.0009) [2023-12-27 02:28:03,541][105620] Updated weights for policy 1, policy_version 1523030 (0.0006) [2023-12-27 02:28:03,556][105692] Updated weights for policy 0, policy_version 1520388 (0.0006) [2023-12-27 02:28:03,596][105620] Updated weights for policy 1, policy_version 1523040 (0.0007) [2023-12-27 02:28:04,167][105692] Updated weights for policy 0, policy_version 1520398 (0.0007) [2023-12-27 02:28:04,228][105692] Updated weights for policy 0, policy_version 1520408 (0.0009) [2023-12-27 02:28:04,270][105620] Updated weights for policy 1, policy_version 1523050 (0.0012) [2023-12-27 02:28:04,295][105692] Updated weights for policy 0, policy_version 1520418 (0.0006) [2023-12-27 02:28:04,332][105620] Updated weights for policy 1, policy_version 1523060 (0.0008) [2023-12-27 02:28:04,392][105620] Updated weights for policy 1, policy_version 1523070 (0.0010) [2023-12-27 02:28:04,450][105620] Updated weights for policy 1, policy_version 1523080 (0.0010) [2023-12-27 02:28:04,985][105692] Updated weights for policy 0, policy_version 1520428 (0.0007) [2023-12-27 02:28:05,037][105692] Updated weights for policy 0, policy_version 1520438 (0.0009) [2023-12-27 02:28:05,087][105692] Updated weights for policy 0, policy_version 1520448 (0.0008) [2023-12-27 02:28:05,223][105620] Updated weights for policy 1, policy_version 1523090 (0.0005) [2023-12-27 02:28:05,268][105620] Updated weights for policy 1, policy_version 1523100 (0.0005) [2023-12-27 02:28:05,320][105620] Updated weights for policy 1, policy_version 1523110 (0.0006) [2023-12-27 02:28:05,807][105692] Updated weights for policy 0, policy_version 1520458 (0.0008) [2023-12-27 02:28:05,863][105692] Updated weights for policy 0, policy_version 1520468 (0.0005) [2023-12-27 02:28:05,916][105692] Updated weights for policy 0, policy_version 1520478 (0.0006) [2023-12-27 02:28:05,964][105692] Updated weights for policy 0, policy_version 1520488 (0.0010) [2023-12-27 02:28:06,051][105620] Updated weights for policy 1, policy_version 1523120 (0.0006) [2023-12-27 02:28:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19133.2). Total num frames: 779272192. Throughput: 0: 9730.9, 1: 9986.2. Samples: 779261452. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:28:06,062][104569] Avg episode reward: [(0, '8444.292'), (1, '8987.108')] [2023-12-27 02:28:06,106][105620] Updated weights for policy 1, policy_version 1523130 (0.0006) [2023-12-27 02:28:06,166][105620] Updated weights for policy 1, policy_version 1523140 (0.0007) [2023-12-27 02:28:06,693][105692] Updated weights for policy 0, policy_version 1520498 (0.0011) [2023-12-27 02:28:06,737][105620] Updated weights for policy 1, policy_version 1523150 (0.0006) [2023-12-27 02:28:06,758][105692] Updated weights for policy 0, policy_version 1520508 (0.0011) [2023-12-27 02:28:06,797][105620] Updated weights for policy 1, policy_version 1523160 (0.0006) [2023-12-27 02:28:06,818][105692] Updated weights for policy 0, policy_version 1520518 (0.0011) [2023-12-27 02:28:06,858][105620] Updated weights for policy 1, policy_version 1523170 (0.0007) [2023-12-27 02:28:07,481][105692] Updated weights for policy 0, policy_version 1520528 (0.0010) [2023-12-27 02:28:07,543][105692] Updated weights for policy 0, policy_version 1520538 (0.0012) [2023-12-27 02:28:07,600][105692] Updated weights for policy 0, policy_version 1520548 (0.0010) [2023-12-27 02:28:07,640][105620] Updated weights for policy 1, policy_version 1523180 (0.0007) [2023-12-27 02:28:07,692][105620] Updated weights for policy 1, policy_version 1523190 (0.0005) [2023-12-27 02:28:07,746][105620] Updated weights for policy 1, policy_version 1523200 (0.0007) [2023-12-27 02:28:08,317][105692] Updated weights for policy 0, policy_version 1520558 (0.0007) [2023-12-27 02:28:08,377][105692] Updated weights for policy 0, policy_version 1520568 (0.0008) [2023-12-27 02:28:08,432][105692] Updated weights for policy 0, policy_version 1520578 (0.0008) [2023-12-27 02:28:08,509][105620] Updated weights for policy 1, policy_version 1523210 (0.0008) [2023-12-27 02:28:08,568][105620] Updated weights for policy 1, policy_version 1523220 (0.0008) [2023-12-27 02:28:08,622][105620] Updated weights for policy 1, policy_version 1523230 (0.0008) [2023-12-27 02:28:08,680][105620] Updated weights for policy 1, policy_version 1523240 (0.0009) [2023-12-27 02:28:09,161][105692] Updated weights for policy 0, policy_version 1520588 (0.0009) [2023-12-27 02:28:09,218][105692] Updated weights for policy 0, policy_version 1520598 (0.0011) [2023-12-27 02:28:09,286][105692] Updated weights for policy 0, policy_version 1520608 (0.0011) [2023-12-27 02:28:09,398][105620] Updated weights for policy 1, policy_version 1523250 (0.0008) [2023-12-27 02:28:09,462][105620] Updated weights for policy 1, policy_version 1523260 (0.0008) [2023-12-27 02:28:09,515][105620] Updated weights for policy 1, policy_version 1523270 (0.0008) [2023-12-27 02:28:10,063][105692] Updated weights for policy 0, policy_version 1520618 (0.0008) [2023-12-27 02:28:10,126][105692] Updated weights for policy 0, policy_version 1520628 (0.0010) [2023-12-27 02:28:10,193][105692] Updated weights for policy 0, policy_version 1520638 (0.0010) [2023-12-27 02:28:10,256][105692] Updated weights for policy 0, policy_version 1520648 (0.0011) [2023-12-27 02:28:10,295][105620] Updated weights for policy 1, policy_version 1523280 (0.0008) [2023-12-27 02:28:10,347][105620] Updated weights for policy 1, policy_version 1523290 (0.0008) [2023-12-27 02:28:10,417][105620] Updated weights for policy 1, policy_version 1523300 (0.0008) [2023-12-27 02:28:10,986][105692] Updated weights for policy 0, policy_version 1520658 (0.0010) [2023-12-27 02:28:11,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19524.2, 300 sec: 19133.2). Total num frames: 779362304. Throughput: 0: 9685.1, 1: 9924.9. Samples: 779376860. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:28:11,063][104569] Avg episode reward: [(0, '8901.300'), (1, '8896.157')] [2023-12-27 02:28:11,069][105692] Updated weights for policy 0, policy_version 1520668 (0.0011) [2023-12-27 02:28:11,132][105692] Updated weights for policy 0, policy_version 1520678 (0.0011) [2023-12-27 02:28:11,174][105620] Updated weights for policy 1, policy_version 1523310 (0.0007) [2023-12-27 02:28:11,236][105620] Updated weights for policy 1, policy_version 1523320 (0.0009) [2023-12-27 02:28:11,308][105620] Updated weights for policy 1, policy_version 1523330 (0.0008) [2023-12-27 02:28:11,948][105692] Updated weights for policy 0, policy_version 1520688 (0.0011) [2023-12-27 02:28:12,009][105692] Updated weights for policy 0, policy_version 1520698 (0.0011) [2023-12-27 02:28:12,061][105620] Updated weights for policy 1, policy_version 1523340 (0.0010) [2023-12-27 02:28:12,063][105692] Updated weights for policy 0, policy_version 1520708 (0.0010) [2023-12-27 02:28:12,129][105620] Updated weights for policy 1, policy_version 1523350 (0.0011) [2023-12-27 02:28:12,182][105620] Updated weights for policy 1, policy_version 1523360 (0.0011) [2023-12-27 02:28:12,704][105692] Updated weights for policy 0, policy_version 1520718 (0.0007) [2023-12-27 02:28:12,758][105692] Updated weights for policy 0, policy_version 1520728 (0.0006) [2023-12-27 02:28:12,811][105692] Updated weights for policy 0, policy_version 1520738 (0.0005) [2023-12-27 02:28:12,861][105620] Updated weights for policy 1, policy_version 1523370 (0.0010) [2023-12-27 02:28:12,936][105620] Updated weights for policy 1, policy_version 1523380 (0.0008) [2023-12-27 02:28:13,001][105620] Updated weights for policy 1, policy_version 1523390 (0.0008) [2023-12-27 02:28:13,066][105620] Updated weights for policy 1, policy_version 1523400 (0.0008) [2023-12-27 02:28:13,353][105692] Updated weights for policy 0, policy_version 1520748 (0.0008) [2023-12-27 02:28:13,404][105692] Updated weights for policy 0, policy_version 1520758 (0.0009) [2023-12-27 02:28:13,468][105692] Updated weights for policy 0, policy_version 1520768 (0.0007) [2023-12-27 02:28:13,735][105620] Updated weights for policy 1, policy_version 1523410 (0.0010) [2023-12-27 02:28:13,797][105620] Updated weights for policy 1, policy_version 1523420 (0.0010) [2023-12-27 02:28:13,855][105620] Updated weights for policy 1, policy_version 1523430 (0.0010) [2023-12-27 02:28:14,029][105692] Updated weights for policy 0, policy_version 1520778 (0.0008) [2023-12-27 02:28:14,093][105692] Updated weights for policy 0, policy_version 1520788 (0.0005) [2023-12-27 02:28:14,150][105692] Updated weights for policy 0, policy_version 1520798 (0.0009) [2023-12-27 02:28:14,210][105692] Updated weights for policy 0, policy_version 1520808 (0.0010) [2023-12-27 02:28:14,520][105620] Updated weights for policy 1, policy_version 1523440 (0.0010) [2023-12-27 02:28:14,578][105620] Updated weights for policy 1, policy_version 1523450 (0.0010) [2023-12-27 02:28:14,636][105620] Updated weights for policy 1, policy_version 1523460 (0.0010) [2023-12-27 02:28:14,844][105692] Updated weights for policy 0, policy_version 1520818 (0.0008) [2023-12-27 02:28:14,904][105692] Updated weights for policy 0, policy_version 1520828 (0.0011) [2023-12-27 02:28:14,962][105692] Updated weights for policy 0, policy_version 1520838 (0.0010) [2023-12-27 02:28:15,400][105620] Updated weights for policy 1, policy_version 1523470 (0.0010) [2023-12-27 02:28:15,462][105620] Updated weights for policy 1, policy_version 1523480 (0.0011) [2023-12-27 02:28:15,522][105620] Updated weights for policy 1, policy_version 1523490 (0.0011) [2023-12-27 02:28:15,720][105692] Updated weights for policy 0, policy_version 1520848 (0.0010) [2023-12-27 02:28:15,778][105692] Updated weights for policy 0, policy_version 1520858 (0.0008) [2023-12-27 02:28:15,837][105692] Updated weights for policy 0, policy_version 1520868 (0.0008) [2023-12-27 02:28:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19188.7). Total num frames: 779468800. Throughput: 0: 9733.4, 1: 9884.5. Samples: 779436404. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:28:16,062][104569] Avg episode reward: [(0, '8632.362'), (1, '9260.865')] [2023-12-27 02:28:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001523496_390070272.pth... [2023-12-27 02:28:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001520872_389398528.pth... [2023-12-27 02:28:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001522376_389783552.pth [2023-12-27 02:28:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001519720_389103616.pth [2023-12-27 02:28:16,225][105620] Updated weights for policy 1, policy_version 1523500 (0.0008) [2023-12-27 02:28:16,299][105620] Updated weights for policy 1, policy_version 1523510 (0.0007) [2023-12-27 02:28:16,360][105620] Updated weights for policy 1, policy_version 1523520 (0.0007) [2023-12-27 02:28:16,513][105692] Updated weights for policy 0, policy_version 1520878 (0.0010) [2023-12-27 02:28:16,576][105692] Updated weights for policy 0, policy_version 1520888 (0.0010) [2023-12-27 02:28:16,630][105692] Updated weights for policy 0, policy_version 1520898 (0.0010) [2023-12-27 02:28:16,927][105620] Updated weights for policy 1, policy_version 1523530 (0.0005) [2023-12-27 02:28:16,974][105620] Updated weights for policy 1, policy_version 1523540 (0.0005) [2023-12-27 02:28:17,025][105620] Updated weights for policy 1, policy_version 1523550 (0.0007) [2023-12-27 02:28:17,083][105620] Updated weights for policy 1, policy_version 1523560 (0.0008) [2023-12-27 02:28:17,389][105692] Updated weights for policy 0, policy_version 1520908 (0.0010) [2023-12-27 02:28:17,450][105692] Updated weights for policy 0, policy_version 1520918 (0.0010) [2023-12-27 02:28:17,511][105692] Updated weights for policy 0, policy_version 1520928 (0.0010) [2023-12-27 02:28:17,665][105620] Updated weights for policy 1, policy_version 1523570 (0.0008) [2023-12-27 02:28:17,711][105620] Updated weights for policy 1, policy_version 1523580 (0.0008) [2023-12-27 02:28:17,758][105620] Updated weights for policy 1, policy_version 1523590 (0.0009) [2023-12-27 02:28:18,154][105692] Updated weights for policy 0, policy_version 1520938 (0.0008) [2023-12-27 02:28:18,208][105692] Updated weights for policy 0, policy_version 1520948 (0.0010) [2023-12-27 02:28:18,260][105692] Updated weights for policy 0, policy_version 1520958 (0.0009) [2023-12-27 02:28:18,330][105692] Updated weights for policy 0, policy_version 1520968 (0.0010) [2023-12-27 02:28:18,399][105620] Updated weights for policy 1, policy_version 1523600 (0.0009) [2023-12-27 02:28:18,458][105620] Updated weights for policy 1, policy_version 1523610 (0.0010) [2023-12-27 02:28:18,510][105620] Updated weights for policy 1, policy_version 1523620 (0.0010) [2023-12-27 02:28:19,145][105620] Updated weights for policy 1, policy_version 1523630 (0.0009) [2023-12-27 02:28:19,177][105692] Updated weights for policy 0, policy_version 1520978 (0.0010) [2023-12-27 02:28:19,200][105620] Updated weights for policy 1, policy_version 1523640 (0.0010) [2023-12-27 02:28:19,231][105692] Updated weights for policy 0, policy_version 1520988 (0.0006) [2023-12-27 02:28:19,265][105620] Updated weights for policy 1, policy_version 1523650 (0.0011) [2023-12-27 02:28:19,296][105692] Updated weights for policy 0, policy_version 1520998 (0.0009) [2023-12-27 02:28:19,953][105620] Updated weights for policy 1, policy_version 1523660 (0.0010) [2023-12-27 02:28:20,013][105620] Updated weights for policy 1, policy_version 1523670 (0.0009) [2023-12-27 02:28:20,071][105620] Updated weights for policy 1, policy_version 1523680 (0.0007) [2023-12-27 02:28:20,105][105692] Updated weights for policy 0, policy_version 1521008 (0.0007) [2023-12-27 02:28:20,167][105692] Updated weights for policy 0, policy_version 1521018 (0.0009) [2023-12-27 02:28:20,225][105692] Updated weights for policy 0, policy_version 1521028 (0.0009) [2023-12-27 02:28:20,782][105620] Updated weights for policy 1, policy_version 1523690 (0.0007) [2023-12-27 02:28:20,849][105620] Updated weights for policy 1, policy_version 1523700 (0.0007) [2023-12-27 02:28:20,921][105620] Updated weights for policy 1, policy_version 1523710 (0.0006) [2023-12-27 02:28:20,995][105620] Updated weights for policy 1, policy_version 1523720 (0.0007) [2023-12-27 02:28:21,038][105692] Updated weights for policy 0, policy_version 1521038 (0.0009) [2023-12-27 02:28:21,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19660.8, 300 sec: 19244.3). Total num frames: 779567104. Throughput: 0: 9779.0, 1: 9928.5. Samples: 779558952. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:28:21,063][104569] Avg episode reward: [(0, '8631.826'), (1, '9167.938')] [2023-12-27 02:28:21,090][105692] Updated weights for policy 0, policy_version 1521048 (0.0009) [2023-12-27 02:28:21,156][105692] Updated weights for policy 0, policy_version 1521058 (0.0008) [2023-12-27 02:28:21,684][105620] Updated weights for policy 1, policy_version 1523730 (0.0009) [2023-12-27 02:28:21,758][105620] Updated weights for policy 1, policy_version 1523740 (0.0008) [2023-12-27 02:28:21,828][105620] Updated weights for policy 1, policy_version 1523750 (0.0008) [2023-12-27 02:28:21,858][105692] Updated weights for policy 0, policy_version 1521068 (0.0010) [2023-12-27 02:28:21,915][105692] Updated weights for policy 0, policy_version 1521078 (0.0008) [2023-12-27 02:28:21,968][105692] Updated weights for policy 0, policy_version 1521088 (0.0007) [2023-12-27 02:28:22,529][105620] Updated weights for policy 1, policy_version 1523760 (0.0006) [2023-12-27 02:28:22,591][105620] Updated weights for policy 1, policy_version 1523770 (0.0008) [2023-12-27 02:28:22,648][105620] Updated weights for policy 1, policy_version 1523780 (0.0008) [2023-12-27 02:28:22,789][105692] Updated weights for policy 0, policy_version 1521098 (0.0009) [2023-12-27 02:28:22,852][105692] Updated weights for policy 0, policy_version 1521108 (0.0009) [2023-12-27 02:28:22,917][105692] Updated weights for policy 0, policy_version 1521118 (0.0010) [2023-12-27 02:28:22,977][105692] Updated weights for policy 0, policy_version 1521128 (0.0009) [2023-12-27 02:28:23,367][105620] Updated weights for policy 1, policy_version 1523790 (0.0010) [2023-12-27 02:28:23,412][105586] KL-divergence is very high: 102.4688 [2023-12-27 02:28:23,414][105620] Updated weights for policy 1, policy_version 1523800 (0.0009) [2023-12-27 02:28:23,451][105586] KL-divergence is very high: 130.1137 [2023-12-27 02:28:23,459][105620] Updated weights for policy 1, policy_version 1523810 (0.0008) [2023-12-27 02:28:23,732][105692] Updated weights for policy 0, policy_version 1521138 (0.0009) [2023-12-27 02:28:23,790][105692] Updated weights for policy 0, policy_version 1521148 (0.0009) [2023-12-27 02:28:23,848][105692] Updated weights for policy 0, policy_version 1521158 (0.0009) [2023-12-27 02:28:24,208][105620] Updated weights for policy 1, policy_version 1523820 (0.0010) [2023-12-27 02:28:24,272][105620] Updated weights for policy 1, policy_version 1523830 (0.0008) [2023-12-27 02:28:24,340][105620] Updated weights for policy 1, policy_version 1523840 (0.0007) [2023-12-27 02:28:24,529][105692] Updated weights for policy 0, policy_version 1521168 (0.0009) [2023-12-27 02:28:24,597][105692] Updated weights for policy 0, policy_version 1521178 (0.0009) [2023-12-27 02:28:24,645][105692] Updated weights for policy 0, policy_version 1521188 (0.0009) [2023-12-27 02:28:25,073][105620] Updated weights for policy 1, policy_version 1523850 (0.0006) [2023-12-27 02:28:25,139][105620] Updated weights for policy 1, policy_version 1523860 (0.0010) [2023-12-27 02:28:25,198][105620] Updated weights for policy 1, policy_version 1523870 (0.0009) [2023-12-27 02:28:25,246][105692] Updated weights for policy 0, policy_version 1521198 (0.0008) [2023-12-27 02:28:25,256][105620] Updated weights for policy 1, policy_version 1523880 (0.0007) [2023-12-27 02:28:25,295][105692] Updated weights for policy 0, policy_version 1521208 (0.0009) [2023-12-27 02:28:25,340][105692] Updated weights for policy 0, policy_version 1521218 (0.0010) [2023-12-27 02:28:26,026][105620] Updated weights for policy 1, policy_version 1523890 (0.0009) [2023-12-27 02:28:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19244.3). Total num frames: 779657216. Throughput: 0: 9767.5, 1: 9790.4. Samples: 779672460. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:28:26,062][104569] Avg episode reward: [(0, '8631.397'), (1, '8712.273')] [2023-12-27 02:28:26,077][105620] Updated weights for policy 1, policy_version 1523900 (0.0009) [2023-12-27 02:28:26,088][105692] Updated weights for policy 0, policy_version 1521228 (0.0009) [2023-12-27 02:28:26,137][105692] Updated weights for policy 0, policy_version 1521238 (0.0006) [2023-12-27 02:28:26,138][105620] Updated weights for policy 1, policy_version 1523910 (0.0008) [2023-12-27 02:28:26,182][105692] Updated weights for policy 0, policy_version 1521248 (0.0008) [2023-12-27 02:28:26,800][105692] Updated weights for policy 0, policy_version 1521258 (0.0007) [2023-12-27 02:28:26,849][105620] Updated weights for policy 1, policy_version 1523920 (0.0008) [2023-12-27 02:28:26,860][105692] Updated weights for policy 0, policy_version 1521268 (0.0005) [2023-12-27 02:28:26,904][105620] Updated weights for policy 1, policy_version 1523930 (0.0008) [2023-12-27 02:28:26,917][105692] Updated weights for policy 0, policy_version 1521278 (0.0010) [2023-12-27 02:28:26,951][105620] Updated weights for policy 1, policy_version 1523940 (0.0006) [2023-12-27 02:28:26,960][105692] Updated weights for policy 0, policy_version 1521288 (0.0010) [2023-12-27 02:28:27,594][105692] Updated weights for policy 0, policy_version 1521298 (0.0009) [2023-12-27 02:28:27,640][105692] Updated weights for policy 0, policy_version 1521308 (0.0008) [2023-12-27 02:28:27,689][105692] Updated weights for policy 0, policy_version 1521318 (0.0008) [2023-12-27 02:28:27,727][105620] Updated weights for policy 1, policy_version 1523950 (0.0008) [2023-12-27 02:28:27,776][105620] Updated weights for policy 1, policy_version 1523960 (0.0008) [2023-12-27 02:28:27,834][105620] Updated weights for policy 1, policy_version 1523970 (0.0009) [2023-12-27 02:28:28,498][105692] Updated weights for policy 0, policy_version 1521328 (0.0009) [2023-12-27 02:28:28,542][105620] Updated weights for policy 1, policy_version 1523980 (0.0008) [2023-12-27 02:28:28,558][105692] Updated weights for policy 0, policy_version 1521338 (0.0008) [2023-12-27 02:28:28,601][105620] Updated weights for policy 1, policy_version 1523990 (0.0007) [2023-12-27 02:28:28,618][105692] Updated weights for policy 0, policy_version 1521348 (0.0009) [2023-12-27 02:28:28,662][105620] Updated weights for policy 1, policy_version 1524000 (0.0008) [2023-12-27 02:28:29,344][105692] Updated weights for policy 0, policy_version 1521358 (0.0009) [2023-12-27 02:28:29,416][105692] Updated weights for policy 0, policy_version 1521368 (0.0009) [2023-12-27 02:28:29,443][105620] Updated weights for policy 1, policy_version 1524010 (0.0008) [2023-12-27 02:28:29,473][105692] Updated weights for policy 0, policy_version 1521378 (0.0007) [2023-12-27 02:28:29,502][105620] Updated weights for policy 1, policy_version 1524020 (0.0008) [2023-12-27 02:28:29,553][105620] Updated weights for policy 1, policy_version 1524030 (0.0009) [2023-12-27 02:28:29,611][105620] Updated weights for policy 1, policy_version 1524040 (0.0005) [2023-12-27 02:28:30,254][105620] Updated weights for policy 1, policy_version 1524050 (0.0005) [2023-12-27 02:28:30,291][105692] Updated weights for policy 0, policy_version 1521388 (0.0010) [2023-12-27 02:28:30,311][105620] Updated weights for policy 1, policy_version 1524060 (0.0006) [2023-12-27 02:28:30,353][105692] Updated weights for policy 0, policy_version 1521398 (0.0007) [2023-12-27 02:28:30,368][105620] Updated weights for policy 1, policy_version 1524070 (0.0009) [2023-12-27 02:28:30,404][105692] Updated weights for policy 0, policy_version 1521408 (0.0008) [2023-12-27 02:28:31,003][105620] Updated weights for policy 1, policy_version 1524080 (0.0010) [2023-12-27 02:28:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19299.8). Total num frames: 779755520. Throughput: 0: 9840.9, 1: 9794.2. Samples: 779731440. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:28:31,062][104569] Avg episode reward: [(0, '8446.966'), (1, '8716.879')] [2023-12-27 02:28:31,064][105620] Updated weights for policy 1, policy_version 1524090 (0.0010) [2023-12-27 02:28:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001521416_389537792.pth... [2023-12-27 02:28:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001520296_389251072.pth [2023-12-27 02:28:31,121][105620] Updated weights for policy 1, policy_version 1524100 (0.0006) [2023-12-27 02:28:31,147][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001524104_390225920.pth... [2023-12-27 02:28:31,153][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001522952_389931008.pth [2023-12-27 02:28:31,210][105692] Updated weights for policy 0, policy_version 1521418 (0.0005) [2023-12-27 02:28:31,275][105692] Updated weights for policy 0, policy_version 1521428 (0.0008) [2023-12-27 02:28:31,337][105692] Updated weights for policy 0, policy_version 1521438 (0.0009) [2023-12-27 02:28:31,401][105692] Updated weights for policy 0, policy_version 1521448 (0.0008) [2023-12-27 02:28:31,870][105620] Updated weights for policy 1, policy_version 1524110 (0.0007) [2023-12-27 02:28:31,919][105620] Updated weights for policy 1, policy_version 1524120 (0.0008) [2023-12-27 02:28:31,978][105620] Updated weights for policy 1, policy_version 1524130 (0.0008) [2023-12-27 02:28:32,139][105692] Updated weights for policy 0, policy_version 1521458 (0.0008) [2023-12-27 02:28:32,191][105692] Updated weights for policy 0, policy_version 1521468 (0.0009) [2023-12-27 02:28:32,244][105692] Updated weights for policy 0, policy_version 1521479 (0.0010) [2023-12-27 02:28:32,675][105620] Updated weights for policy 1, policy_version 1524140 (0.0008) [2023-12-27 02:28:32,727][105620] Updated weights for policy 1, policy_version 1524150 (0.0008) [2023-12-27 02:28:32,776][105620] Updated weights for policy 1, policy_version 1524160 (0.0007) [2023-12-27 02:28:33,043][105692] Updated weights for policy 0, policy_version 1521489 (0.0009) [2023-12-27 02:28:33,093][105692] Updated weights for policy 0, policy_version 1521499 (0.0009) [2023-12-27 02:28:33,143][105692] Updated weights for policy 0, policy_version 1521510 (0.0009) [2023-12-27 02:28:33,315][105620] Updated weights for policy 1, policy_version 1524170 (0.0005) [2023-12-27 02:28:33,370][105620] Updated weights for policy 1, policy_version 1524180 (0.0006) [2023-12-27 02:28:33,427][105620] Updated weights for policy 1, policy_version 1524190 (0.0006) [2023-12-27 02:28:33,481][105620] Updated weights for policy 1, policy_version 1524200 (0.0005) [2023-12-27 02:28:34,010][105620] Updated weights for policy 1, policy_version 1524210 (0.0008) [2023-12-27 02:28:34,047][105692] Updated weights for policy 0, policy_version 1521520 (0.0007) [2023-12-27 02:28:34,071][105620] Updated weights for policy 1, policy_version 1524220 (0.0011) [2023-12-27 02:28:34,099][105692] Updated weights for policy 0, policy_version 1521530 (0.0010) [2023-12-27 02:28:34,126][105620] Updated weights for policy 1, policy_version 1524230 (0.0011) [2023-12-27 02:28:34,162][105692] Updated weights for policy 0, policy_version 1521540 (0.0007) [2023-12-27 02:28:34,876][105692] Updated weights for policy 0, policy_version 1521550 (0.0007) [2023-12-27 02:28:34,878][105620] Updated weights for policy 1, policy_version 1524240 (0.0011) [2023-12-27 02:28:34,927][105692] Updated weights for policy 0, policy_version 1521560 (0.0006) [2023-12-27 02:28:34,937][105620] Updated weights for policy 1, policy_version 1524250 (0.0010) [2023-12-27 02:28:34,983][105692] Updated weights for policy 0, policy_version 1521570 (0.0005) [2023-12-27 02:28:34,990][105620] Updated weights for policy 1, policy_version 1524260 (0.0011) [2023-12-27 02:28:35,736][105692] Updated weights for policy 0, policy_version 1521580 (0.0005) [2023-12-27 02:28:35,738][105620] Updated weights for policy 1, policy_version 1524270 (0.0011) [2023-12-27 02:28:35,786][105620] Updated weights for policy 1, policy_version 1524280 (0.0010) [2023-12-27 02:28:35,798][105692] Updated weights for policy 0, policy_version 1521590 (0.0009) [2023-12-27 02:28:35,848][105620] Updated weights for policy 1, policy_version 1524290 (0.0011) [2023-12-27 02:28:35,851][105692] Updated weights for policy 0, policy_version 1521600 (0.0006) [2023-12-27 02:28:36,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.8, 300 sec: 19355.3). Total num frames: 779862016. Throughput: 0: 9679.3, 1: 9897.1. Samples: 779847840. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:28:36,063][104569] Avg episode reward: [(0, '8627.306'), (1, '9081.475')] [2023-12-27 02:28:36,576][105620] Updated weights for policy 1, policy_version 1524300 (0.0009) [2023-12-27 02:28:36,611][105692] Updated weights for policy 0, policy_version 1521610 (0.0007) [2023-12-27 02:28:36,640][105620] Updated weights for policy 1, policy_version 1524310 (0.0009) [2023-12-27 02:28:36,671][105692] Updated weights for policy 0, policy_version 1521620 (0.0011) [2023-12-27 02:28:36,704][105620] Updated weights for policy 1, policy_version 1524320 (0.0011) [2023-12-27 02:28:36,725][105692] Updated weights for policy 0, policy_version 1521630 (0.0011) [2023-12-27 02:28:36,781][105692] Updated weights for policy 0, policy_version 1521640 (0.0008) [2023-12-27 02:28:37,412][105692] Updated weights for policy 0, policy_version 1521650 (0.0011) [2023-12-27 02:28:37,430][105620] Updated weights for policy 1, policy_version 1524330 (0.0011) [2023-12-27 02:28:37,463][105692] Updated weights for policy 0, policy_version 1521660 (0.0010) [2023-12-27 02:28:37,490][105620] Updated weights for policy 1, policy_version 1524340 (0.0011) [2023-12-27 02:28:37,522][105692] Updated weights for policy 0, policy_version 1521670 (0.0011) [2023-12-27 02:28:37,545][105620] Updated weights for policy 1, policy_version 1524350 (0.0010) [2023-12-27 02:28:37,591][105620] Updated weights for policy 1, policy_version 1524360 (0.0011) [2023-12-27 02:28:38,219][105692] Updated weights for policy 0, policy_version 1521680 (0.0006) [2023-12-27 02:28:38,276][105692] Updated weights for policy 0, policy_version 1521690 (0.0005) [2023-12-27 02:28:38,336][105692] Updated weights for policy 0, policy_version 1521700 (0.0006) [2023-12-27 02:28:38,376][105620] Updated weights for policy 1, policy_version 1524370 (0.0007) [2023-12-27 02:28:38,431][105620] Updated weights for policy 1, policy_version 1524380 (0.0005) [2023-12-27 02:28:38,482][105620] Updated weights for policy 1, policy_version 1524390 (0.0005) [2023-12-27 02:28:39,043][105692] Updated weights for policy 0, policy_version 1521710 (0.0006) [2023-12-27 02:28:39,092][105692] Updated weights for policy 0, policy_version 1521720 (0.0005) [2023-12-27 02:28:39,138][105692] Updated weights for policy 0, policy_version 1521730 (0.0005) [2023-12-27 02:28:39,208][105620] Updated weights for policy 1, policy_version 1524400 (0.0009) [2023-12-27 02:28:39,279][105620] Updated weights for policy 1, policy_version 1524410 (0.0009) [2023-12-27 02:28:39,343][105620] Updated weights for policy 1, policy_version 1524420 (0.0007) [2023-12-27 02:28:39,891][105692] Updated weights for policy 0, policy_version 1521740 (0.0007) [2023-12-27 02:28:39,950][105692] Updated weights for policy 0, policy_version 1521750 (0.0007) [2023-12-27 02:28:40,014][105692] Updated weights for policy 0, policy_version 1521760 (0.0008) [2023-12-27 02:28:40,028][105620] Updated weights for policy 1, policy_version 1524430 (0.0007) [2023-12-27 02:28:40,087][105620] Updated weights for policy 1, policy_version 1524440 (0.0008) [2023-12-27 02:28:40,146][105620] Updated weights for policy 1, policy_version 1524450 (0.0009) [2023-12-27 02:28:40,732][105692] Updated weights for policy 0, policy_version 1521770 (0.0007) [2023-12-27 02:28:40,800][105692] Updated weights for policy 0, policy_version 1521780 (0.0009) [2023-12-27 02:28:40,858][105692] Updated weights for policy 0, policy_version 1521790 (0.0009) [2023-12-27 02:28:40,908][105620] Updated weights for policy 1, policy_version 1524460 (0.0008) [2023-12-27 02:28:40,926][105692] Updated weights for policy 0, policy_version 1521800 (0.0008) [2023-12-27 02:28:40,966][105620] Updated weights for policy 1, policy_version 1524470 (0.0007) [2023-12-27 02:28:41,026][105620] Updated weights for policy 1, policy_version 1524480 (0.0011) [2023-12-27 02:28:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 779952128. Throughput: 0: 9660.0, 1: 9833.6. Samples: 779963404. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:28:41,063][104569] Avg episode reward: [(0, '8537.037'), (1, '8992.339')] [2023-12-27 02:28:41,721][105692] Updated weights for policy 0, policy_version 1521810 (0.0009) [2023-12-27 02:28:41,786][105620] Updated weights for policy 1, policy_version 1524490 (0.0009) [2023-12-27 02:28:41,790][105692] Updated weights for policy 0, policy_version 1521820 (0.0009) [2023-12-27 02:28:41,846][105620] Updated weights for policy 1, policy_version 1524500 (0.0006) [2023-12-27 02:28:41,848][105692] Updated weights for policy 0, policy_version 1521830 (0.0007) [2023-12-27 02:28:41,899][105620] Updated weights for policy 1, policy_version 1524510 (0.0006) [2023-12-27 02:28:41,960][105620] Updated weights for policy 1, policy_version 1524520 (0.0005) [2023-12-27 02:28:42,645][105620] Updated weights for policy 1, policy_version 1524530 (0.0008) [2023-12-27 02:28:42,672][105692] Updated weights for policy 0, policy_version 1521840 (0.0007) [2023-12-27 02:28:42,697][105620] Updated weights for policy 1, policy_version 1524540 (0.0008) [2023-12-27 02:28:42,723][105692] Updated weights for policy 0, policy_version 1521850 (0.0008) [2023-12-27 02:28:42,756][105620] Updated weights for policy 1, policy_version 1524550 (0.0008) [2023-12-27 02:28:42,778][105692] Updated weights for policy 0, policy_version 1521860 (0.0010) [2023-12-27 02:28:43,492][105620] Updated weights for policy 1, policy_version 1524560 (0.0009) [2023-12-27 02:28:43,538][105620] Updated weights for policy 1, policy_version 1524570 (0.0008) [2023-12-27 02:28:43,544][105692] Updated weights for policy 0, policy_version 1521870 (0.0008) [2023-12-27 02:28:43,598][105620] Updated weights for policy 1, policy_version 1524580 (0.0008) [2023-12-27 02:28:43,599][105692] Updated weights for policy 0, policy_version 1521880 (0.0006) [2023-12-27 02:28:43,652][105692] Updated weights for policy 0, policy_version 1521890 (0.0009) [2023-12-27 02:28:44,351][105620] Updated weights for policy 1, policy_version 1524590 (0.0010) [2023-12-27 02:28:44,402][105692] Updated weights for policy 0, policy_version 1521900 (0.0009) [2023-12-27 02:28:44,411][105620] Updated weights for policy 1, policy_version 1524600 (0.0008) [2023-12-27 02:28:44,452][105692] Updated weights for policy 0, policy_version 1521910 (0.0009) [2023-12-27 02:28:44,474][105620] Updated weights for policy 1, policy_version 1524610 (0.0006) [2023-12-27 02:28:44,503][105692] Updated weights for policy 0, policy_version 1521921 (0.0009) [2023-12-27 02:28:45,192][105620] Updated weights for policy 1, policy_version 1524620 (0.0006) [2023-12-27 02:28:45,255][105620] Updated weights for policy 1, policy_version 1524630 (0.0006) [2023-12-27 02:28:45,279][105692] Updated weights for policy 0, policy_version 1521932 (0.0010) [2023-12-27 02:28:45,316][105620] Updated weights for policy 1, policy_version 1524640 (0.0005) [2023-12-27 02:28:45,346][105692] Updated weights for policy 0, policy_version 1521942 (0.0010) [2023-12-27 02:28:45,413][105692] Updated weights for policy 0, policy_version 1521952 (0.0009) [2023-12-27 02:28:46,023][105620] Updated weights for policy 1, policy_version 1524650 (0.0006) [2023-12-27 02:28:46,062][104569] Fps is (10 sec: 18021.9, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 780042240. Throughput: 0: 9587.2, 1: 9841.5. Samples: 780018612. Policy #0 lag: (min: 27.0, avg: 27.4, max: 45.0) [2023-12-27 02:28:46,063][104569] Avg episode reward: [(0, '8082.629'), (1, '8992.555')] [2023-12-27 02:28:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001521960_389677056.pth... [2023-12-27 02:28:46,071][105620] Updated weights for policy 1, policy_version 1524660 (0.0005) [2023-12-27 02:28:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001520872_389398528.pth [2023-12-27 02:28:46,101][105692] Updated weights for policy 0, policy_version 1521962 (0.0009) [2023-12-27 02:28:46,128][105620] Updated weights for policy 1, policy_version 1524670 (0.0006) [2023-12-27 02:28:46,170][105692] Updated weights for policy 0, policy_version 1521972 (0.0009) [2023-12-27 02:28:46,184][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001524680_390373376.pth... [2023-12-27 02:28:46,184][105620] Updated weights for policy 1, policy_version 1524680 (0.0005) [2023-12-27 02:28:46,187][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001523496_390070272.pth [2023-12-27 02:28:46,239][105692] Updated weights for policy 0, policy_version 1521982 (0.0009) [2023-12-27 02:28:46,308][105692] Updated weights for policy 0, policy_version 1521992 (0.0010) [2023-12-27 02:28:46,834][105620] Updated weights for policy 1, policy_version 1524690 (0.0011) [2023-12-27 02:28:46,893][105620] Updated weights for policy 1, policy_version 1524700 (0.0011) [2023-12-27 02:28:46,953][105620] Updated weights for policy 1, policy_version 1524710 (0.0007) [2023-12-27 02:28:46,975][105692] Updated weights for policy 0, policy_version 1522002 (0.0009) [2023-12-27 02:28:47,030][105692] Updated weights for policy 0, policy_version 1522012 (0.0009) [2023-12-27 02:28:47,087][105692] Updated weights for policy 0, policy_version 1522022 (0.0006) [2023-12-27 02:28:47,562][105620] Updated weights for policy 1, policy_version 1524720 (0.0007) [2023-12-27 02:28:47,620][105620] Updated weights for policy 1, policy_version 1524730 (0.0005) [2023-12-27 02:28:47,674][105620] Updated weights for policy 1, policy_version 1524740 (0.0010) [2023-12-27 02:28:47,819][105692] Updated weights for policy 0, policy_version 1522032 (0.0008) [2023-12-27 02:28:47,873][105692] Updated weights for policy 0, policy_version 1522042 (0.0009) [2023-12-27 02:28:47,927][105692] Updated weights for policy 0, policy_version 1522052 (0.0009) [2023-12-27 02:28:48,279][105620] Updated weights for policy 1, policy_version 1524750 (0.0010) [2023-12-27 02:28:48,333][105620] Updated weights for policy 1, policy_version 1524760 (0.0010) [2023-12-27 02:28:48,397][105620] Updated weights for policy 1, policy_version 1524770 (0.0010) [2023-12-27 02:28:48,652][105692] Updated weights for policy 0, policy_version 1522062 (0.0009) [2023-12-27 02:28:48,709][105692] Updated weights for policy 0, policy_version 1522072 (0.0008) [2023-12-27 02:28:48,767][105692] Updated weights for policy 0, policy_version 1522082 (0.0006) [2023-12-27 02:28:49,150][105620] Updated weights for policy 1, policy_version 1524780 (0.0011) [2023-12-27 02:28:49,198][105620] Updated weights for policy 1, policy_version 1524790 (0.0010) [2023-12-27 02:28:49,259][105620] Updated weights for policy 1, policy_version 1524800 (0.0011) [2023-12-27 02:28:49,513][105692] Updated weights for policy 0, policy_version 1522092 (0.0010) [2023-12-27 02:28:49,568][105692] Updated weights for policy 0, policy_version 1522102 (0.0009) [2023-12-27 02:28:49,636][105692] Updated weights for policy 0, policy_version 1522112 (0.0006) [2023-12-27 02:28:49,897][105620] Updated weights for policy 1, policy_version 1524810 (0.0009) [2023-12-27 02:28:49,962][105620] Updated weights for policy 1, policy_version 1524821 (0.0009) [2023-12-27 02:28:49,983][105586] KL-divergence is very high: 112.1263 [2023-12-27 02:28:50,018][105620] Updated weights for policy 1, policy_version 1524831 (0.0007) [2023-12-27 02:28:50,032][105586] KL-divergence is very high: 118.2874 [2023-12-27 02:28:50,310][105692] Updated weights for policy 0, policy_version 1522122 (0.0007) [2023-12-27 02:28:50,364][105692] Updated weights for policy 0, policy_version 1522132 (0.0009) [2023-12-27 02:28:50,430][105692] Updated weights for policy 0, policy_version 1522142 (0.0010) [2023-12-27 02:28:50,481][105692] Updated weights for policy 0, policy_version 1522152 (0.0008) [2023-12-27 02:28:50,786][105620] Updated weights for policy 1, policy_version 1524841 (0.0009) [2023-12-27 02:28:50,851][105620] Updated weights for policy 1, policy_version 1524851 (0.0008) [2023-12-27 02:28:50,917][105620] Updated weights for policy 1, policy_version 1524861 (0.0007) [2023-12-27 02:28:50,980][105620] Updated weights for policy 1, policy_version 1524871 (0.0007) [2023-12-27 02:28:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 780148736. Throughput: 0: 9583.1, 1: 9881.1. Samples: 780137344. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:28:51,063][104569] Avg episode reward: [(0, '8159.165'), (1, '9171.079')] [2023-12-27 02:28:51,183][105692] Updated weights for policy 0, policy_version 1522162 (0.0008) [2023-12-27 02:28:51,240][105692] Updated weights for policy 0, policy_version 1522172 (0.0006) [2023-12-27 02:28:51,300][105692] Updated weights for policy 0, policy_version 1522182 (0.0006) [2023-12-27 02:28:51,748][105620] Updated weights for policy 1, policy_version 1524881 (0.0008) [2023-12-27 02:28:51,811][105620] Updated weights for policy 1, policy_version 1524891 (0.0008) [2023-12-27 02:28:51,871][105620] Updated weights for policy 1, policy_version 1524901 (0.0008) [2023-12-27 02:28:52,019][105692] Updated weights for policy 0, policy_version 1522192 (0.0008) [2023-12-27 02:28:52,084][105692] Updated weights for policy 0, policy_version 1522202 (0.0007) [2023-12-27 02:28:52,142][105692] Updated weights for policy 0, policy_version 1522212 (0.0005) [2023-12-27 02:28:52,658][105620] Updated weights for policy 1, policy_version 1524911 (0.0008) [2023-12-27 02:28:52,714][105620] Updated weights for policy 1, policy_version 1524921 (0.0008) [2023-12-27 02:28:52,773][105620] Updated weights for policy 1, policy_version 1524931 (0.0008) [2023-12-27 02:28:52,852][105692] Updated weights for policy 0, policy_version 1522222 (0.0008) [2023-12-27 02:28:52,911][105692] Updated weights for policy 0, policy_version 1522232 (0.0011) [2023-12-27 02:28:52,970][105692] Updated weights for policy 0, policy_version 1522242 (0.0011) [2023-12-27 02:28:53,532][105692] Updated weights for policy 0, policy_version 1522252 (0.0006) [2023-12-27 02:28:53,547][105620] Updated weights for policy 1, policy_version 1524941 (0.0008) [2023-12-27 02:28:53,598][105692] Updated weights for policy 0, policy_version 1522262 (0.0005) [2023-12-27 02:28:53,607][105620] Updated weights for policy 1, policy_version 1524951 (0.0005) [2023-12-27 02:28:53,660][105692] Updated weights for policy 0, policy_version 1522272 (0.0010) [2023-12-27 02:28:53,671][105620] Updated weights for policy 1, policy_version 1524961 (0.0006) [2023-12-27 02:28:54,331][105692] Updated weights for policy 0, policy_version 1522282 (0.0011) [2023-12-27 02:28:54,385][105692] Updated weights for policy 0, policy_version 1522292 (0.0010) [2023-12-27 02:28:54,407][105620] Updated weights for policy 1, policy_version 1524971 (0.0009) [2023-12-27 02:28:54,447][105692] Updated weights for policy 0, policy_version 1522302 (0.0010) [2023-12-27 02:28:54,466][105620] Updated weights for policy 1, policy_version 1524981 (0.0010) [2023-12-27 02:28:54,506][105692] Updated weights for policy 0, policy_version 1522312 (0.0011) [2023-12-27 02:28:54,523][105620] Updated weights for policy 1, policy_version 1524991 (0.0010) [2023-12-27 02:28:55,226][105692] Updated weights for policy 0, policy_version 1522322 (0.0010) [2023-12-27 02:28:55,240][105620] Updated weights for policy 1, policy_version 1525001 (0.0008) [2023-12-27 02:28:55,280][105692] Updated weights for policy 0, policy_version 1522332 (0.0010) [2023-12-27 02:28:55,297][105620] Updated weights for policy 1, policy_version 1525011 (0.0010) [2023-12-27 02:28:55,323][105692] Updated weights for policy 0, policy_version 1522342 (0.0010) [2023-12-27 02:28:55,355][105620] Updated weights for policy 1, policy_version 1525021 (0.0010) [2023-12-27 02:28:55,415][105620] Updated weights for policy 1, policy_version 1525031 (0.0010) [2023-12-27 02:28:56,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 780238848. Throughput: 0: 9648.0, 1: 9825.8. Samples: 780253176. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:28:56,062][104569] Avg episode reward: [(0, '8443.215'), (1, '9171.118')] [2023-12-27 02:28:56,094][105692] Updated weights for policy 0, policy_version 1522352 (0.0008) [2023-12-27 02:28:56,149][105620] Updated weights for policy 1, policy_version 1525041 (0.0010) [2023-12-27 02:28:56,152][105692] Updated weights for policy 0, policy_version 1522362 (0.0007) [2023-12-27 02:28:56,207][105620] Updated weights for policy 1, policy_version 1525051 (0.0010) [2023-12-27 02:28:56,208][105692] Updated weights for policy 0, policy_version 1522372 (0.0008) [2023-12-27 02:28:56,266][105620] Updated weights for policy 1, policy_version 1525061 (0.0010) [2023-12-27 02:28:56,987][105692] Updated weights for policy 0, policy_version 1522382 (0.0007) [2023-12-27 02:28:57,003][105620] Updated weights for policy 1, policy_version 1525071 (0.0010) [2023-12-27 02:28:57,033][105692] Updated weights for policy 0, policy_version 1522392 (0.0005) [2023-12-27 02:28:57,051][105620] Updated weights for policy 1, policy_version 1525081 (0.0010) [2023-12-27 02:28:57,086][105692] Updated weights for policy 0, policy_version 1522402 (0.0005) [2023-12-27 02:28:57,095][105620] Updated weights for policy 1, policy_version 1525091 (0.0010) [2023-12-27 02:28:57,711][105692] Updated weights for policy 0, policy_version 1522412 (0.0006) [2023-12-27 02:28:57,722][105620] Updated weights for policy 1, policy_version 1525101 (0.0010) [2023-12-27 02:28:57,767][105692] Updated weights for policy 0, policy_version 1522422 (0.0005) [2023-12-27 02:28:57,784][105620] Updated weights for policy 1, policy_version 1525111 (0.0008) [2023-12-27 02:28:57,822][105692] Updated weights for policy 0, policy_version 1522432 (0.0009) [2023-12-27 02:28:57,834][105620] Updated weights for policy 1, policy_version 1525121 (0.0006) [2023-12-27 02:28:58,442][105692] Updated weights for policy 0, policy_version 1522442 (0.0009) [2023-12-27 02:28:58,456][105620] Updated weights for policy 1, policy_version 1525131 (0.0005) [2023-12-27 02:28:58,503][105692] Updated weights for policy 0, policy_version 1522452 (0.0009) [2023-12-27 02:28:58,516][105620] Updated weights for policy 1, policy_version 1525141 (0.0008) [2023-12-27 02:28:58,565][105692] Updated weights for policy 0, policy_version 1522462 (0.0007) [2023-12-27 02:28:58,580][105620] Updated weights for policy 1, policy_version 1525152 (0.0008) [2023-12-27 02:28:58,625][105692] Updated weights for policy 0, policy_version 1522472 (0.0009) [2023-12-27 02:28:59,402][105692] Updated weights for policy 0, policy_version 1522482 (0.0006) [2023-12-27 02:28:59,423][105620] Updated weights for policy 1, policy_version 1525162 (0.0008) [2023-12-27 02:28:59,462][105692] Updated weights for policy 0, policy_version 1522492 (0.0006) [2023-12-27 02:28:59,480][105620] Updated weights for policy 1, policy_version 1525172 (0.0008) [2023-12-27 02:28:59,524][105692] Updated weights for policy 0, policy_version 1522502 (0.0007) [2023-12-27 02:28:59,542][105620] Updated weights for policy 1, policy_version 1525182 (0.0007) [2023-12-27 02:28:59,604][105620] Updated weights for policy 1, policy_version 1525192 (0.0009) [2023-12-27 02:29:00,248][105692] Updated weights for policy 0, policy_version 1522512 (0.0009) [2023-12-27 02:29:00,298][105692] Updated weights for policy 0, policy_version 1522522 (0.0008) [2023-12-27 02:29:00,309][105620] Updated weights for policy 1, policy_version 1525202 (0.0008) [2023-12-27 02:29:00,355][105692] Updated weights for policy 0, policy_version 1522532 (0.0011) [2023-12-27 02:29:00,367][105620] Updated weights for policy 1, policy_version 1525212 (0.0008) [2023-12-27 02:29:00,424][105620] Updated weights for policy 1, policy_version 1525222 (0.0008) [2023-12-27 02:29:00,955][105692] Updated weights for policy 0, policy_version 1522542 (0.0007) [2023-12-27 02:29:01,009][105692] Updated weights for policy 0, policy_version 1522552 (0.0006) [2023-12-27 02:29:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 780337152. Throughput: 0: 9634.9, 1: 9851.0. Samples: 780313272. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:29:01,063][104569] Avg episode reward: [(0, '8628.062'), (1, '9353.613')] [2023-12-27 02:29:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001525224_390512640.pth... [2023-12-27 02:29:01,068][105692] Updated weights for policy 0, policy_version 1522562 (0.0010) [2023-12-27 02:29:01,086][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001524104_390225920.pth [2023-12-27 02:29:01,100][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001522568_389832704.pth... [2023-12-27 02:29:01,104][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001521416_389537792.pth [2023-12-27 02:29:01,271][105620] Updated weights for policy 1, policy_version 1525232 (0.0007) [2023-12-27 02:29:01,326][105620] Updated weights for policy 1, policy_version 1525242 (0.0007) [2023-12-27 02:29:01,396][105620] Updated weights for policy 1, policy_version 1525252 (0.0008) [2023-12-27 02:29:01,818][105692] Updated weights for policy 0, policy_version 1522572 (0.0011) [2023-12-27 02:29:01,883][105692] Updated weights for policy 0, policy_version 1522582 (0.0010) [2023-12-27 02:29:01,941][105692] Updated weights for policy 0, policy_version 1522592 (0.0010) [2023-12-27 02:29:02,146][105620] Updated weights for policy 1, policy_version 1525262 (0.0007) [2023-12-27 02:29:02,209][105620] Updated weights for policy 1, policy_version 1525272 (0.0006) [2023-12-27 02:29:02,271][105620] Updated weights for policy 1, policy_version 1525282 (0.0007) [2023-12-27 02:29:02,675][105692] Updated weights for policy 0, policy_version 1522602 (0.0010) [2023-12-27 02:29:02,730][105692] Updated weights for policy 0, policy_version 1522612 (0.0010) [2023-12-27 02:29:02,785][105692] Updated weights for policy 0, policy_version 1522622 (0.0010) [2023-12-27 02:29:02,827][105620] Updated weights for policy 1, policy_version 1525292 (0.0006) [2023-12-27 02:29:02,842][105692] Updated weights for policy 0, policy_version 1522632 (0.0009) [2023-12-27 02:29:02,877][105620] Updated weights for policy 1, policy_version 1525302 (0.0005) [2023-12-27 02:29:02,931][105620] Updated weights for policy 1, policy_version 1525312 (0.0005) [2023-12-27 02:29:03,462][105620] Updated weights for policy 1, policy_version 1525322 (0.0005) [2023-12-27 02:29:03,497][105692] Updated weights for policy 0, policy_version 1522642 (0.0005) [2023-12-27 02:29:03,518][105620] Updated weights for policy 1, policy_version 1525332 (0.0005) [2023-12-27 02:29:03,546][105692] Updated weights for policy 0, policy_version 1522652 (0.0005) [2023-12-27 02:29:03,569][105620] Updated weights for policy 1, policy_version 1525342 (0.0005) [2023-12-27 02:29:03,592][105692] Updated weights for policy 0, policy_version 1522662 (0.0006) [2023-12-27 02:29:03,615][105620] Updated weights for policy 1, policy_version 1525352 (0.0005) [2023-12-27 02:29:04,199][105692] Updated weights for policy 0, policy_version 1522672 (0.0006) [2023-12-27 02:29:04,223][105620] Updated weights for policy 1, policy_version 1525362 (0.0006) [2023-12-27 02:29:04,267][105692] Updated weights for policy 0, policy_version 1522682 (0.0006) [2023-12-27 02:29:04,277][105620] Updated weights for policy 1, policy_version 1525372 (0.0007) [2023-12-27 02:29:04,330][105620] Updated weights for policy 1, policy_version 1525382 (0.0008) [2023-12-27 02:29:04,332][105692] Updated weights for policy 0, policy_version 1522692 (0.0006) [2023-12-27 02:29:04,913][105620] Updated weights for policy 1, policy_version 1525392 (0.0005) [2023-12-27 02:29:04,959][105620] Updated weights for policy 1, policy_version 1525402 (0.0005) [2023-12-27 02:29:04,979][105692] Updated weights for policy 0, policy_version 1522702 (0.0011) [2023-12-27 02:29:05,008][105620] Updated weights for policy 1, policy_version 1525412 (0.0006) [2023-12-27 02:29:05,031][105692] Updated weights for policy 0, policy_version 1522712 (0.0010) [2023-12-27 02:29:05,089][105692] Updated weights for policy 0, policy_version 1522722 (0.0010) [2023-12-27 02:29:05,569][105620] Updated weights for policy 1, policy_version 1525422 (0.0005) [2023-12-27 02:29:05,620][105620] Updated weights for policy 1, policy_version 1525432 (0.0005) [2023-12-27 02:29:05,674][105620] Updated weights for policy 1, policy_version 1525442 (0.0006) [2023-12-27 02:29:05,841][105692] Updated weights for policy 0, policy_version 1522732 (0.0010) [2023-12-27 02:29:05,896][105692] Updated weights for policy 0, policy_version 1522742 (0.0010) [2023-12-27 02:29:05,941][105692] Updated weights for policy 0, policy_version 1522752 (0.0010) [2023-12-27 02:29:06,062][104569] Fps is (10 sec: 21298.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 780451840. Throughput: 0: 9657.4, 1: 9827.5. Samples: 780435776. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:29:06,063][104569] Avg episode reward: [(0, '8723.135'), (1, '9173.202')] [2023-12-27 02:29:06,349][105620] Updated weights for policy 1, policy_version 1525452 (0.0005) [2023-12-27 02:29:06,409][105620] Updated weights for policy 1, policy_version 1525462 (0.0005) [2023-12-27 02:29:06,474][105620] Updated weights for policy 1, policy_version 1525472 (0.0007) [2023-12-27 02:29:06,649][105692] Updated weights for policy 0, policy_version 1522762 (0.0010) [2023-12-27 02:29:06,712][105692] Updated weights for policy 0, policy_version 1522772 (0.0011) [2023-12-27 02:29:06,778][105692] Updated weights for policy 0, policy_version 1522782 (0.0011) [2023-12-27 02:29:06,842][105692] Updated weights for policy 0, policy_version 1522792 (0.0011) [2023-12-27 02:29:07,144][105620] Updated weights for policy 1, policy_version 1525482 (0.0008) [2023-12-27 02:29:07,208][105620] Updated weights for policy 1, policy_version 1525492 (0.0005) [2023-12-27 02:29:07,266][105620] Updated weights for policy 1, policy_version 1525502 (0.0007) [2023-12-27 02:29:07,327][105620] Updated weights for policy 1, policy_version 1525512 (0.0008) [2023-12-27 02:29:07,566][105692] Updated weights for policy 0, policy_version 1522802 (0.0005) [2023-12-27 02:29:07,632][105692] Updated weights for policy 0, policy_version 1522812 (0.0011) [2023-12-27 02:29:07,697][105692] Updated weights for policy 0, policy_version 1522822 (0.0010) [2023-12-27 02:29:07,941][105620] Updated weights for policy 1, policy_version 1525522 (0.0007) [2023-12-27 02:29:08,009][105620] Updated weights for policy 1, policy_version 1525532 (0.0007) [2023-12-27 02:29:08,066][105620] Updated weights for policy 1, policy_version 1525542 (0.0008) [2023-12-27 02:29:08,409][105692] Updated weights for policy 0, policy_version 1522832 (0.0010) [2023-12-27 02:29:08,471][105692] Updated weights for policy 0, policy_version 1522842 (0.0010) [2023-12-27 02:29:08,533][105692] Updated weights for policy 0, policy_version 1522852 (0.0010) [2023-12-27 02:29:08,796][105620] Updated weights for policy 1, policy_version 1525552 (0.0006) [2023-12-27 02:29:08,853][105620] Updated weights for policy 1, policy_version 1525562 (0.0006) [2023-12-27 02:29:08,915][105620] Updated weights for policy 1, policy_version 1525572 (0.0005) [2023-12-27 02:29:09,170][105692] Updated weights for policy 0, policy_version 1522862 (0.0008) [2023-12-27 02:29:09,230][105692] Updated weights for policy 0, policy_version 1522872 (0.0007) [2023-12-27 02:29:09,292][105692] Updated weights for policy 0, policy_version 1522882 (0.0010) [2023-12-27 02:29:09,611][105620] Updated weights for policy 1, policy_version 1525582 (0.0008) [2023-12-27 02:29:09,677][105620] Updated weights for policy 1, policy_version 1525592 (0.0011) [2023-12-27 02:29:09,740][105620] Updated weights for policy 1, policy_version 1525602 (0.0011) [2023-12-27 02:29:10,049][105692] Updated weights for policy 0, policy_version 1522892 (0.0010) [2023-12-27 02:29:10,115][105692] Updated weights for policy 0, policy_version 1522902 (0.0010) [2023-12-27 02:29:10,178][105692] Updated weights for policy 0, policy_version 1522912 (0.0011) [2023-12-27 02:29:10,444][105620] Updated weights for policy 1, policy_version 1525612 (0.0010) [2023-12-27 02:29:10,506][105620] Updated weights for policy 1, policy_version 1525622 (0.0010) [2023-12-27 02:29:10,565][105620] Updated weights for policy 1, policy_version 1525632 (0.0010) [2023-12-27 02:29:10,924][105692] Updated weights for policy 0, policy_version 1522922 (0.0010) [2023-12-27 02:29:10,981][105692] Updated weights for policy 0, policy_version 1522932 (0.0009) [2023-12-27 02:29:11,048][105692] Updated weights for policy 0, policy_version 1522942 (0.0008) [2023-12-27 02:29:11,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.9, 300 sec: 19466.4). Total num frames: 780541952. Throughput: 0: 9672.7, 1: 9936.9. Samples: 780554892. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:29:11,062][104569] Avg episode reward: [(0, '8634.229'), (1, '9078.943')] [2023-12-27 02:29:11,109][105692] Updated weights for policy 0, policy_version 1522952 (0.0006) [2023-12-27 02:29:11,302][105620] Updated weights for policy 1, policy_version 1525642 (0.0010) [2023-12-27 02:29:11,356][105620] Updated weights for policy 1, policy_version 1525652 (0.0007) [2023-12-27 02:29:11,419][105620] Updated weights for policy 1, policy_version 1525662 (0.0009) [2023-12-27 02:29:11,470][105620] Updated weights for policy 1, policy_version 1525672 (0.0008) [2023-12-27 02:29:11,873][105692] Updated weights for policy 0, policy_version 1522962 (0.0008) [2023-12-27 02:29:11,940][105692] Updated weights for policy 0, policy_version 1522972 (0.0008) [2023-12-27 02:29:11,999][105692] Updated weights for policy 0, policy_version 1522982 (0.0008) [2023-12-27 02:29:12,341][105620] Updated weights for policy 1, policy_version 1525682 (0.0008) [2023-12-27 02:29:12,406][105620] Updated weights for policy 1, policy_version 1525692 (0.0007) [2023-12-27 02:29:12,465][105620] Updated weights for policy 1, policy_version 1525702 (0.0009) [2023-12-27 02:29:12,786][105692] Updated weights for policy 0, policy_version 1522992 (0.0009) [2023-12-27 02:29:12,853][105692] Updated weights for policy 0, policy_version 1523002 (0.0010) [2023-12-27 02:29:12,907][105692] Updated weights for policy 0, policy_version 1523012 (0.0009) [2023-12-27 02:29:13,198][105620] Updated weights for policy 1, policy_version 1525712 (0.0010) [2023-12-27 02:29:13,260][105620] Updated weights for policy 1, policy_version 1525722 (0.0009) [2023-12-27 02:29:13,318][105620] Updated weights for policy 1, policy_version 1525732 (0.0009) [2023-12-27 02:29:13,689][105692] Updated weights for policy 0, policy_version 1523022 (0.0007) [2023-12-27 02:29:13,737][105692] Updated weights for policy 0, policy_version 1523032 (0.0006) [2023-12-27 02:29:13,793][105692] Updated weights for policy 0, policy_version 1523042 (0.0006) [2023-12-27 02:29:13,902][105620] Updated weights for policy 1, policy_version 1525742 (0.0007) [2023-12-27 02:29:13,961][105620] Updated weights for policy 1, policy_version 1525752 (0.0010) [2023-12-27 02:29:14,026][105620] Updated weights for policy 1, policy_version 1525762 (0.0008) [2023-12-27 02:29:14,408][105692] Updated weights for policy 0, policy_version 1523052 (0.0007) [2023-12-27 02:29:14,477][105692] Updated weights for policy 0, policy_version 1523062 (0.0010) [2023-12-27 02:29:14,532][105692] Updated weights for policy 0, policy_version 1523072 (0.0010) [2023-12-27 02:29:14,661][105620] Updated weights for policy 1, policy_version 1525772 (0.0005) [2023-12-27 02:29:14,720][105620] Updated weights for policy 1, policy_version 1525782 (0.0009) [2023-12-27 02:29:14,788][105620] Updated weights for policy 1, policy_version 1525792 (0.0009) [2023-12-27 02:29:15,288][105692] Updated weights for policy 0, policy_version 1523082 (0.0010) [2023-12-27 02:29:15,352][105692] Updated weights for policy 0, policy_version 1523092 (0.0008) [2023-12-27 02:29:15,414][105692] Updated weights for policy 0, policy_version 1523102 (0.0008) [2023-12-27 02:29:15,481][105692] Updated weights for policy 0, policy_version 1523112 (0.0005) [2023-12-27 02:29:15,556][105620] Updated weights for policy 1, policy_version 1525802 (0.0009) [2023-12-27 02:29:15,615][105620] Updated weights for policy 1, policy_version 1525812 (0.0009) [2023-12-27 02:29:15,675][105620] Updated weights for policy 1, policy_version 1525822 (0.0009) [2023-12-27 02:29:15,738][105620] Updated weights for policy 1, policy_version 1525832 (0.0010) [2023-12-27 02:29:16,061][105692] Updated weights for policy 0, policy_version 1523122 (0.0006) [2023-12-27 02:29:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 780640256. Throughput: 0: 9591.2, 1: 9939.7. Samples: 780610336. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:29:16,063][104569] Avg episode reward: [(0, '8268.810'), (1, '9083.992')] [2023-12-27 02:29:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001525832_390668288.pth... [2023-12-27 02:29:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001524680_390373376.pth [2023-12-27 02:29:16,113][105692] Updated weights for policy 0, policy_version 1523132 (0.0006) [2023-12-27 02:29:16,169][105692] Updated weights for policy 0, policy_version 1523142 (0.0005) [2023-12-27 02:29:16,180][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001523144_389980160.pth... [2023-12-27 02:29:16,184][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001521960_389677056.pth [2023-12-27 02:29:16,540][105620] Updated weights for policy 1, policy_version 1525842 (0.0006) [2023-12-27 02:29:16,596][105620] Updated weights for policy 1, policy_version 1525852 (0.0006) [2023-12-27 02:29:16,645][105620] Updated weights for policy 1, policy_version 1525862 (0.0008) [2023-12-27 02:29:16,854][105692] Updated weights for policy 0, policy_version 1523152 (0.0008) [2023-12-27 02:29:16,912][105692] Updated weights for policy 0, policy_version 1523162 (0.0010) [2023-12-27 02:29:16,972][105692] Updated weights for policy 0, policy_version 1523173 (0.0010) [2023-12-27 02:29:17,300][105620] Updated weights for policy 1, policy_version 1525872 (0.0009) [2023-12-27 02:29:17,354][105620] Updated weights for policy 1, policy_version 1525882 (0.0009) [2023-12-27 02:29:17,401][105620] Updated weights for policy 1, policy_version 1525892 (0.0009) [2023-12-27 02:29:17,770][105692] Updated weights for policy 0, policy_version 1523184 (0.0009) [2023-12-27 02:29:17,822][105692] Updated weights for policy 0, policy_version 1523194 (0.0009) [2023-12-27 02:29:17,869][105692] Updated weights for policy 0, policy_version 1523204 (0.0008) [2023-12-27 02:29:18,121][105620] Updated weights for policy 1, policy_version 1525902 (0.0008) [2023-12-27 02:29:18,182][105620] Updated weights for policy 1, policy_version 1525912 (0.0009) [2023-12-27 02:29:18,242][105620] Updated weights for policy 1, policy_version 1525922 (0.0009) [2023-12-27 02:29:18,672][105692] Updated weights for policy 0, policy_version 1523214 (0.0009) [2023-12-27 02:29:18,720][105692] Updated weights for policy 0, policy_version 1523224 (0.0009) [2023-12-27 02:29:18,782][105692] Updated weights for policy 0, policy_version 1523234 (0.0009) [2023-12-27 02:29:18,959][105620] Updated weights for policy 1, policy_version 1525932 (0.0008) [2023-12-27 02:29:19,016][105620] Updated weights for policy 1, policy_version 1525942 (0.0007) [2023-12-27 02:29:19,087][105620] Updated weights for policy 1, policy_version 1525952 (0.0006) [2023-12-27 02:29:19,644][105692] Updated weights for policy 0, policy_version 1523244 (0.0009) [2023-12-27 02:29:19,704][105692] Updated weights for policy 0, policy_version 1523254 (0.0009) [2023-12-27 02:29:19,763][105692] Updated weights for policy 0, policy_version 1523264 (0.0009) [2023-12-27 02:29:19,772][105620] Updated weights for policy 1, policy_version 1525962 (0.0007) [2023-12-27 02:29:19,835][105620] Updated weights for policy 1, policy_version 1525972 (0.0007) [2023-12-27 02:29:19,904][105620] Updated weights for policy 1, policy_version 1525982 (0.0010) [2023-12-27 02:29:19,971][105620] Updated weights for policy 1, policy_version 1525992 (0.0010) [2023-12-27 02:29:20,506][105692] Updated weights for policy 0, policy_version 1523274 (0.0008) [2023-12-27 02:29:20,576][105692] Updated weights for policy 0, policy_version 1523284 (0.0008) [2023-12-27 02:29:20,632][105692] Updated weights for policy 0, policy_version 1523294 (0.0009) [2023-12-27 02:29:20,680][105692] Updated weights for policy 0, policy_version 1523304 (0.0008) [2023-12-27 02:29:20,693][105620] Updated weights for policy 1, policy_version 1526002 (0.0008) [2023-12-27 02:29:20,761][105620] Updated weights for policy 1, policy_version 1526012 (0.0009) [2023-12-27 02:29:20,827][105620] Updated weights for policy 1, policy_version 1526022 (0.0009) [2023-12-27 02:29:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 780738560. Throughput: 0: 9694.3, 1: 9830.4. Samples: 780726452. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:29:21,062][104569] Avg episode reward: [(0, '7995.865'), (1, '8909.464')] [2023-12-27 02:29:21,463][105692] Updated weights for policy 0, policy_version 1523314 (0.0009) [2023-12-27 02:29:21,515][105692] Updated weights for policy 0, policy_version 1523324 (0.0009) [2023-12-27 02:29:21,573][105692] Updated weights for policy 0, policy_version 1523334 (0.0010) [2023-12-27 02:29:21,580][105620] Updated weights for policy 1, policy_version 1526032 (0.0006) [2023-12-27 02:29:21,638][105620] Updated weights for policy 1, policy_version 1526042 (0.0009) [2023-12-27 02:29:21,695][105620] Updated weights for policy 1, policy_version 1526052 (0.0009) [2023-12-27 02:29:22,279][105692] Updated weights for policy 0, policy_version 1523344 (0.0009) [2023-12-27 02:29:22,348][105692] Updated weights for policy 0, policy_version 1523354 (0.0008) [2023-12-27 02:29:22,411][105692] Updated weights for policy 0, policy_version 1523364 (0.0007) [2023-12-27 02:29:22,535][105620] Updated weights for policy 1, policy_version 1526062 (0.0008) [2023-12-27 02:29:22,593][105620] Updated weights for policy 1, policy_version 1526072 (0.0009) [2023-12-27 02:29:22,648][105620] Updated weights for policy 1, policy_version 1526082 (0.0009) [2023-12-27 02:29:23,178][105692] Updated weights for policy 0, policy_version 1523374 (0.0009) [2023-12-27 02:29:23,241][105692] Updated weights for policy 0, policy_version 1523384 (0.0009) [2023-12-27 02:29:23,300][105692] Updated weights for policy 0, policy_version 1523394 (0.0009) [2023-12-27 02:29:23,402][105620] Updated weights for policy 1, policy_version 1526092 (0.0009) [2023-12-27 02:29:23,473][105620] Updated weights for policy 1, policy_version 1526102 (0.0010) [2023-12-27 02:29:23,542][105620] Updated weights for policy 1, policy_version 1526112 (0.0009) [2023-12-27 02:29:23,923][105692] Updated weights for policy 0, policy_version 1523404 (0.0008) [2023-12-27 02:29:23,978][105692] Updated weights for policy 0, policy_version 1523414 (0.0006) [2023-12-27 02:29:24,025][105692] Updated weights for policy 0, policy_version 1523424 (0.0009) [2023-12-27 02:29:24,354][105620] Updated weights for policy 1, policy_version 1526122 (0.0009) [2023-12-27 02:29:24,412][105620] Updated weights for policy 1, policy_version 1526133 (0.0010) [2023-12-27 02:29:24,466][105620] Updated weights for policy 1, policy_version 1526143 (0.0008) [2023-12-27 02:29:24,683][105692] Updated weights for policy 0, policy_version 1523434 (0.0007) [2023-12-27 02:29:24,733][105692] Updated weights for policy 0, policy_version 1523444 (0.0005) [2023-12-27 02:29:24,782][105692] Updated weights for policy 0, policy_version 1523454 (0.0005) [2023-12-27 02:29:24,827][105692] Updated weights for policy 0, policy_version 1523464 (0.0005) [2023-12-27 02:29:25,300][105620] Updated weights for policy 1, policy_version 1526153 (0.0009) [2023-12-27 02:29:25,361][105620] Updated weights for policy 1, policy_version 1526163 (0.0008) [2023-12-27 02:29:25,396][105692] Updated weights for policy 0, policy_version 1523474 (0.0011) [2023-12-27 02:29:25,413][105620] Updated weights for policy 1, policy_version 1526173 (0.0009) [2023-12-27 02:29:25,455][105692] Updated weights for policy 0, policy_version 1523484 (0.0010) [2023-12-27 02:29:25,465][105620] Updated weights for policy 1, policy_version 1526183 (0.0010) [2023-12-27 02:29:25,509][105692] Updated weights for policy 0, policy_version 1523494 (0.0006) [2023-12-27 02:29:26,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 780828672. Throughput: 0: 9707.4, 1: 9772.4. Samples: 780839992. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:29:26,062][104569] Avg episode reward: [(0, '8539.415'), (1, '8994.216')] [2023-12-27 02:29:26,117][105692] Updated weights for policy 0, policy_version 1523504 (0.0005) [2023-12-27 02:29:26,173][105692] Updated weights for policy 0, policy_version 1523514 (0.0006) [2023-12-27 02:29:26,212][105620] Updated weights for policy 1, policy_version 1526193 (0.0010) [2023-12-27 02:29:26,219][105692] Updated weights for policy 0, policy_version 1523524 (0.0005) [2023-12-27 02:29:26,265][105620] Updated weights for policy 1, policy_version 1526203 (0.0010) [2023-12-27 02:29:26,323][105620] Updated weights for policy 1, policy_version 1526213 (0.0010) [2023-12-27 02:29:26,772][105692] Updated weights for policy 0, policy_version 1523534 (0.0005) [2023-12-27 02:29:26,819][105692] Updated weights for policy 0, policy_version 1523544 (0.0007) [2023-12-27 02:29:26,866][105692] Updated weights for policy 0, policy_version 1523554 (0.0010) [2023-12-27 02:29:27,026][105620] Updated weights for policy 1, policy_version 1526223 (0.0010) [2023-12-27 02:29:27,081][105620] Updated weights for policy 1, policy_version 1526233 (0.0010) [2023-12-27 02:29:27,135][105620] Updated weights for policy 1, policy_version 1526243 (0.0010) [2023-12-27 02:29:27,593][105692] Updated weights for policy 0, policy_version 1523564 (0.0010) [2023-12-27 02:29:27,654][105692] Updated weights for policy 0, policy_version 1523574 (0.0008) [2023-12-27 02:29:27,717][105692] Updated weights for policy 0, policy_version 1523584 (0.0009) [2023-12-27 02:29:27,806][105620] Updated weights for policy 1, policy_version 1526253 (0.0010) [2023-12-27 02:29:27,874][105620] Updated weights for policy 1, policy_version 1526263 (0.0010) [2023-12-27 02:29:27,942][105620] Updated weights for policy 1, policy_version 1526273 (0.0010) [2023-12-27 02:29:28,415][105692] Updated weights for policy 0, policy_version 1523594 (0.0009) [2023-12-27 02:29:28,479][105692] Updated weights for policy 0, policy_version 1523604 (0.0011) [2023-12-27 02:29:28,537][105692] Updated weights for policy 0, policy_version 1523614 (0.0010) [2023-12-27 02:29:28,586][105620] Updated weights for policy 1, policy_version 1526283 (0.0011) [2023-12-27 02:29:28,596][105692] Updated weights for policy 0, policy_version 1523624 (0.0010) [2023-12-27 02:29:28,643][105620] Updated weights for policy 1, policy_version 1526293 (0.0010) [2023-12-27 02:29:28,698][105620] Updated weights for policy 1, policy_version 1526303 (0.0010) [2023-12-27 02:29:28,727][105586] KL-divergence is very high: 100.2374 [2023-12-27 02:29:29,230][105692] Updated weights for policy 0, policy_version 1523634 (0.0007) [2023-12-27 02:29:29,287][105692] Updated weights for policy 0, policy_version 1523644 (0.0006) [2023-12-27 02:29:29,294][105620] Updated weights for policy 1, policy_version 1526313 (0.0008) [2023-12-27 02:29:29,347][105692] Updated weights for policy 0, policy_version 1523654 (0.0009) [2023-12-27 02:29:29,354][105620] Updated weights for policy 1, policy_version 1526323 (0.0010) [2023-12-27 02:29:29,426][105620] Updated weights for policy 1, policy_version 1526333 (0.0006) [2023-12-27 02:29:29,496][105620] Updated weights for policy 1, policy_version 1526343 (0.0005) [2023-12-27 02:29:30,045][105692] Updated weights for policy 0, policy_version 1523664 (0.0011) [2023-12-27 02:29:30,097][105692] Updated weights for policy 0, policy_version 1523674 (0.0010) [2023-12-27 02:29:30,114][105620] Updated weights for policy 1, policy_version 1526353 (0.0007) [2023-12-27 02:29:30,155][105692] Updated weights for policy 0, policy_version 1523684 (0.0010) [2023-12-27 02:29:30,173][105620] Updated weights for policy 1, policy_version 1526363 (0.0005) [2023-12-27 02:29:30,237][105620] Updated weights for policy 1, policy_version 1526373 (0.0008) [2023-12-27 02:29:30,893][105620] Updated weights for policy 1, policy_version 1526383 (0.0008) [2023-12-27 02:29:30,913][105692] Updated weights for policy 0, policy_version 1523694 (0.0011) [2023-12-27 02:29:30,950][105620] Updated weights for policy 1, policy_version 1526393 (0.0009) [2023-12-27 02:29:30,964][105692] Updated weights for policy 0, policy_version 1523704 (0.0010) [2023-12-27 02:29:31,013][105620] Updated weights for policy 1, policy_version 1526403 (0.0006) [2023-12-27 02:29:31,026][105692] Updated weights for policy 0, policy_version 1523714 (0.0010) [2023-12-27 02:29:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 780935168. Throughput: 0: 9832.7, 1: 9816.1. Samples: 780902800. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:29:31,062][104569] Avg episode reward: [(0, '8532.240'), (1, '8904.859')] [2023-12-27 02:29:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001526408_390815744.pth... [2023-12-27 02:29:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001525224_390512640.pth [2023-12-27 02:29:31,078][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001523720_390127616.pth... [2023-12-27 02:29:31,082][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001522568_389832704.pth [2023-12-27 02:29:31,691][105692] Updated weights for policy 0, policy_version 1523724 (0.0010) [2023-12-27 02:29:31,756][105692] Updated weights for policy 0, policy_version 1523734 (0.0009) [2023-12-27 02:29:31,815][105692] Updated weights for policy 0, policy_version 1523744 (0.0008) [2023-12-27 02:29:31,817][105620] Updated weights for policy 1, policy_version 1526413 (0.0008) [2023-12-27 02:29:31,879][105620] Updated weights for policy 1, policy_version 1526423 (0.0008) [2023-12-27 02:29:31,934][105620] Updated weights for policy 1, policy_version 1526433 (0.0008) [2023-12-27 02:29:32,619][105692] Updated weights for policy 0, policy_version 1523754 (0.0006) [2023-12-27 02:29:32,657][105620] Updated weights for policy 1, policy_version 1526443 (0.0008) [2023-12-27 02:29:32,668][105692] Updated weights for policy 0, policy_version 1523764 (0.0008) [2023-12-27 02:29:32,721][105692] Updated weights for policy 0, policy_version 1523774 (0.0008) [2023-12-27 02:29:32,723][105620] Updated weights for policy 1, policy_version 1526453 (0.0009) [2023-12-27 02:29:32,782][105620] Updated weights for policy 1, policy_version 1526463 (0.0006) [2023-12-27 02:29:32,783][105692] Updated weights for policy 0, policy_version 1523784 (0.0007) [2023-12-27 02:29:33,303][105620] Updated weights for policy 1, policy_version 1526473 (0.0005) [2023-12-27 02:29:33,354][105620] Updated weights for policy 1, policy_version 1526483 (0.0005) [2023-12-27 02:29:33,404][105620] Updated weights for policy 1, policy_version 1526493 (0.0009) [2023-12-27 02:29:33,456][105620] Updated weights for policy 1, policy_version 1526503 (0.0007) [2023-12-27 02:29:33,483][105692] Updated weights for policy 0, policy_version 1523794 (0.0008) [2023-12-27 02:29:33,532][105692] Updated weights for policy 0, policy_version 1523804 (0.0008) [2023-12-27 02:29:33,596][105692] Updated weights for policy 0, policy_version 1523814 (0.0008) [2023-12-27 02:29:34,155][105692] Updated weights for policy 0, policy_version 1523824 (0.0006) [2023-12-27 02:29:34,218][105692] Updated weights for policy 0, policy_version 1523834 (0.0008) [2023-12-27 02:29:34,284][105692] Updated weights for policy 0, policy_version 1523844 (0.0007) [2023-12-27 02:29:34,285][105620] Updated weights for policy 1, policy_version 1526513 (0.0009) [2023-12-27 02:29:34,341][105620] Updated weights for policy 1, policy_version 1526523 (0.0009) [2023-12-27 02:29:34,405][105620] Updated weights for policy 1, policy_version 1526533 (0.0007) [2023-12-27 02:29:34,924][105692] Updated weights for policy 0, policy_version 1523854 (0.0008) [2023-12-27 02:29:34,968][105692] Updated weights for policy 0, policy_version 1523864 (0.0010) [2023-12-27 02:29:35,023][105692] Updated weights for policy 0, policy_version 1523874 (0.0010) [2023-12-27 02:29:35,131][105620] Updated weights for policy 1, policy_version 1526543 (0.0008) [2023-12-27 02:29:35,182][105620] Updated weights for policy 1, policy_version 1526553 (0.0007) [2023-12-27 02:29:35,226][105620] Updated weights for policy 1, policy_version 1526563 (0.0005) [2023-12-27 02:29:35,748][105692] Updated weights for policy 0, policy_version 1523884 (0.0008) [2023-12-27 02:29:35,798][105692] Updated weights for policy 0, policy_version 1523894 (0.0009) [2023-12-27 02:29:35,830][105620] Updated weights for policy 1, policy_version 1526573 (0.0005) [2023-12-27 02:29:35,850][105692] Updated weights for policy 0, policy_version 1523904 (0.0010) [2023-12-27 02:29:35,893][105620] Updated weights for policy 1, policy_version 1526583 (0.0009) [2023-12-27 02:29:35,949][105620] Updated weights for policy 1, policy_version 1526593 (0.0006) [2023-12-27 02:29:36,062][104569] Fps is (10 sec: 21298.7, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 781041664. Throughput: 0: 9907.3, 1: 9785.2. Samples: 781023508. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:29:36,063][104569] Avg episode reward: [(0, '8082.003'), (1, '8997.885')] [2023-12-27 02:29:36,552][105692] Updated weights for policy 0, policy_version 1523914 (0.0010) [2023-12-27 02:29:36,607][105620] Updated weights for policy 1, policy_version 1526603 (0.0005) [2023-12-27 02:29:36,607][105692] Updated weights for policy 0, policy_version 1523924 (0.0010) [2023-12-27 02:29:36,663][105692] Updated weights for policy 0, policy_version 1523934 (0.0011) [2023-12-27 02:29:36,664][105620] Updated weights for policy 1, policy_version 1526613 (0.0009) [2023-12-27 02:29:36,716][105620] Updated weights for policy 1, policy_version 1526623 (0.0010) [2023-12-27 02:29:36,718][105692] Updated weights for policy 0, policy_version 1523944 (0.0010) [2023-12-27 02:29:37,461][105620] Updated weights for policy 1, policy_version 1526633 (0.0011) [2023-12-27 02:29:37,478][105692] Updated weights for policy 0, policy_version 1523954 (0.0011) [2023-12-27 02:29:37,512][105620] Updated weights for policy 1, policy_version 1526643 (0.0010) [2023-12-27 02:29:37,533][105692] Updated weights for policy 0, policy_version 1523964 (0.0010) [2023-12-27 02:29:37,567][105620] Updated weights for policy 1, policy_version 1526653 (0.0010) [2023-12-27 02:29:37,589][105692] Updated weights for policy 0, policy_version 1523974 (0.0010) [2023-12-27 02:29:37,632][105620] Updated weights for policy 1, policy_version 1526663 (0.0010) [2023-12-27 02:29:38,227][105692] Updated weights for policy 0, policy_version 1523984 (0.0009) [2023-12-27 02:29:38,294][105692] Updated weights for policy 0, policy_version 1523994 (0.0011) [2023-12-27 02:29:38,339][105620] Updated weights for policy 1, policy_version 1526673 (0.0007) [2023-12-27 02:29:38,358][105692] Updated weights for policy 0, policy_version 1524004 (0.0011) [2023-12-27 02:29:38,395][105620] Updated weights for policy 1, policy_version 1526683 (0.0007) [2023-12-27 02:29:38,453][105620] Updated weights for policy 1, policy_version 1526693 (0.0011) [2023-12-27 02:29:39,053][105692] Updated weights for policy 0, policy_version 1524014 (0.0011) [2023-12-27 02:29:39,056][105620] Updated weights for policy 1, policy_version 1526703 (0.0007) [2023-12-27 02:29:39,103][105620] Updated weights for policy 1, policy_version 1526713 (0.0005) [2023-12-27 02:29:39,116][105692] Updated weights for policy 0, policy_version 1524024 (0.0011) [2023-12-27 02:29:39,164][105620] Updated weights for policy 1, policy_version 1526723 (0.0006) [2023-12-27 02:29:39,175][105692] Updated weights for policy 0, policy_version 1524034 (0.0011) [2023-12-27 02:29:39,870][105620] Updated weights for policy 1, policy_version 1526733 (0.0011) [2023-12-27 02:29:39,919][105692] Updated weights for policy 0, policy_version 1524044 (0.0009) [2023-12-27 02:29:39,937][105620] Updated weights for policy 1, policy_version 1526743 (0.0011) [2023-12-27 02:29:39,986][105692] Updated weights for policy 0, policy_version 1524054 (0.0007) [2023-12-27 02:29:40,002][105620] Updated weights for policy 1, policy_version 1526753 (0.0011) [2023-12-27 02:29:40,051][105692] Updated weights for policy 0, policy_version 1524064 (0.0006) [2023-12-27 02:29:40,694][105692] Updated weights for policy 0, policy_version 1524074 (0.0007) [2023-12-27 02:29:40,755][105692] Updated weights for policy 0, policy_version 1524084 (0.0008) [2023-12-27 02:29:40,763][105620] Updated weights for policy 1, policy_version 1526763 (0.0010) [2023-12-27 02:29:40,819][105620] Updated weights for policy 1, policy_version 1526773 (0.0011) [2023-12-27 02:29:40,825][105692] Updated weights for policy 0, policy_version 1524094 (0.0005) [2023-12-27 02:29:40,869][105620] Updated weights for policy 1, policy_version 1526783 (0.0010) [2023-12-27 02:29:40,887][105692] Updated weights for policy 0, policy_version 1524104 (0.0006) [2023-12-27 02:29:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19494.2). Total num frames: 781139968. Throughput: 0: 9872.5, 1: 9901.5. Samples: 781143008. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:29:41,062][104569] Avg episode reward: [(0, '8354.872'), (1, '9263.962')] [2023-12-27 02:29:41,599][105692] Updated weights for policy 0, policy_version 1524114 (0.0009) [2023-12-27 02:29:41,671][105692] Updated weights for policy 0, policy_version 1524124 (0.0009) [2023-12-27 02:29:41,704][105620] Updated weights for policy 1, policy_version 1526793 (0.0010) [2023-12-27 02:29:41,743][105692] Updated weights for policy 0, policy_version 1524134 (0.0009) [2023-12-27 02:29:41,765][105620] Updated weights for policy 1, policy_version 1526803 (0.0007) [2023-12-27 02:29:41,816][105620] Updated weights for policy 1, policy_version 1526813 (0.0006) [2023-12-27 02:29:41,870][105620] Updated weights for policy 1, policy_version 1526823 (0.0006) [2023-12-27 02:29:42,564][105692] Updated weights for policy 0, policy_version 1524144 (0.0008) [2023-12-27 02:29:42,566][105620] Updated weights for policy 1, policy_version 1526833 (0.0007) [2023-12-27 02:29:42,627][105692] Updated weights for policy 0, policy_version 1524154 (0.0007) [2023-12-27 02:29:42,632][105620] Updated weights for policy 1, policy_version 1526843 (0.0005) [2023-12-27 02:29:42,684][105692] Updated weights for policy 0, policy_version 1524164 (0.0009) [2023-12-27 02:29:42,686][105620] Updated weights for policy 1, policy_version 1526853 (0.0005) [2023-12-27 02:29:43,359][105692] Updated weights for policy 0, policy_version 1524174 (0.0010) [2023-12-27 02:29:43,377][105620] Updated weights for policy 1, policy_version 1526863 (0.0005) [2023-12-27 02:29:43,415][105692] Updated weights for policy 0, policy_version 1524184 (0.0010) [2023-12-27 02:29:43,438][105620] Updated weights for policy 1, policy_version 1526873 (0.0006) [2023-12-27 02:29:43,464][105692] Updated weights for policy 0, policy_version 1524194 (0.0010) [2023-12-27 02:29:43,500][105620] Updated weights for policy 1, policy_version 1526883 (0.0006) [2023-12-27 02:29:44,092][105692] Updated weights for policy 0, policy_version 1524204 (0.0008) [2023-12-27 02:29:44,141][105620] Updated weights for policy 1, policy_version 1526893 (0.0009) [2023-12-27 02:29:44,146][105692] Updated weights for policy 0, policy_version 1524214 (0.0008) [2023-12-27 02:29:44,204][105620] Updated weights for policy 1, policy_version 1526903 (0.0007) [2023-12-27 02:29:44,209][105692] Updated weights for policy 0, policy_version 1524224 (0.0007) [2023-12-27 02:29:44,262][105620] Updated weights for policy 1, policy_version 1526913 (0.0005) [2023-12-27 02:29:44,920][105692] Updated weights for policy 0, policy_version 1524234 (0.0011) [2023-12-27 02:29:44,927][105620] Updated weights for policy 1, policy_version 1526923 (0.0007) [2023-12-27 02:29:44,980][105692] Updated weights for policy 0, policy_version 1524244 (0.0011) [2023-12-27 02:29:44,988][105620] Updated weights for policy 1, policy_version 1526933 (0.0006) [2023-12-27 02:29:45,041][105692] Updated weights for policy 0, policy_version 1524254 (0.0011) [2023-12-27 02:29:45,048][105620] Updated weights for policy 1, policy_version 1526943 (0.0006) [2023-12-27 02:29:45,101][105692] Updated weights for policy 0, policy_version 1524264 (0.0011) [2023-12-27 02:29:45,761][105620] Updated weights for policy 1, policy_version 1526953 (0.0006) [2023-12-27 02:29:45,789][105692] Updated weights for policy 0, policy_version 1524274 (0.0006) [2023-12-27 02:29:45,816][105620] Updated weights for policy 1, policy_version 1526963 (0.0007) [2023-12-27 02:29:45,843][105692] Updated weights for policy 0, policy_version 1524284 (0.0006) [2023-12-27 02:29:45,867][105620] Updated weights for policy 1, policy_version 1526973 (0.0007) [2023-12-27 02:29:45,890][105692] Updated weights for policy 0, policy_version 1524294 (0.0006) [2023-12-27 02:29:45,910][105620] Updated weights for policy 1, policy_version 1526983 (0.0006) [2023-12-27 02:29:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19933.9, 300 sec: 19494.2). Total num frames: 781238272. Throughput: 0: 9841.0, 1: 9896.9. Samples: 781201484. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:29:46,063][104569] Avg episode reward: [(0, '8632.715'), (1, '9082.528')] [2023-12-27 02:29:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001526984_390963200.pth... [2023-12-27 02:29:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001524296_390275072.pth... [2023-12-27 02:29:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001525832_390668288.pth [2023-12-27 02:29:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001523144_389980160.pth [2023-12-27 02:29:46,514][105620] Updated weights for policy 1, policy_version 1526993 (0.0010) [2023-12-27 02:29:46,572][105620] Updated weights for policy 1, policy_version 1527003 (0.0010) [2023-12-27 02:29:46,634][105620] Updated weights for policy 1, policy_version 1527013 (0.0010) [2023-12-27 02:29:46,684][105692] Updated weights for policy 0, policy_version 1524304 (0.0008) [2023-12-27 02:29:46,733][105692] Updated weights for policy 0, policy_version 1524314 (0.0006) [2023-12-27 02:29:46,784][105692] Updated weights for policy 0, policy_version 1524324 (0.0008) [2023-12-27 02:29:47,319][105620] Updated weights for policy 1, policy_version 1527023 (0.0010) [2023-12-27 02:29:47,376][105620] Updated weights for policy 1, policy_version 1527033 (0.0010) [2023-12-27 02:29:47,434][105620] Updated weights for policy 1, policy_version 1527043 (0.0010) [2023-12-27 02:29:47,506][105692] Updated weights for policy 0, policy_version 1524334 (0.0008) [2023-12-27 02:29:47,563][105692] Updated weights for policy 0, policy_version 1524344 (0.0008) [2023-12-27 02:29:47,619][105692] Updated weights for policy 0, policy_version 1524354 (0.0005) [2023-12-27 02:29:48,182][105620] Updated weights for policy 1, policy_version 1527053 (0.0011) [2023-12-27 02:29:48,241][105692] Updated weights for policy 0, policy_version 1524364 (0.0005) [2023-12-27 02:29:48,244][105620] Updated weights for policy 1, policy_version 1527063 (0.0005) [2023-12-27 02:29:48,302][105620] Updated weights for policy 1, policy_version 1527073 (0.0007) [2023-12-27 02:29:48,303][105692] Updated weights for policy 0, policy_version 1524374 (0.0006) [2023-12-27 02:29:48,368][105692] Updated weights for policy 0, policy_version 1524384 (0.0006) [2023-12-27 02:29:49,013][105692] Updated weights for policy 0, policy_version 1524394 (0.0006) [2023-12-27 02:29:49,024][105620] Updated weights for policy 1, policy_version 1527083 (0.0011) [2023-12-27 02:29:49,058][105692] Updated weights for policy 0, policy_version 1524404 (0.0006) [2023-12-27 02:29:49,082][105620] Updated weights for policy 1, policy_version 1527093 (0.0010) [2023-12-27 02:29:49,113][105692] Updated weights for policy 0, policy_version 1524414 (0.0010) [2023-12-27 02:29:49,136][105620] Updated weights for policy 1, policy_version 1527103 (0.0010) [2023-12-27 02:29:49,174][105692] Updated weights for policy 0, policy_version 1524424 (0.0008) [2023-12-27 02:29:49,872][105620] Updated weights for policy 1, policy_version 1527113 (0.0010) [2023-12-27 02:29:49,883][105692] Updated weights for policy 0, policy_version 1524434 (0.0007) [2023-12-27 02:29:49,932][105620] Updated weights for policy 1, policy_version 1527123 (0.0008) [2023-12-27 02:29:49,935][105692] Updated weights for policy 0, policy_version 1524444 (0.0006) [2023-12-27 02:29:49,994][105620] Updated weights for policy 1, policy_version 1527133 (0.0008) [2023-12-27 02:29:50,008][105692] Updated weights for policy 0, policy_version 1524454 (0.0009) [2023-12-27 02:29:50,061][105620] Updated weights for policy 1, policy_version 1527143 (0.0008) [2023-12-27 02:29:50,688][105692] Updated weights for policy 0, policy_version 1524464 (0.0006) [2023-12-27 02:29:50,749][105692] Updated weights for policy 0, policy_version 1524474 (0.0008) [2023-12-27 02:29:50,802][105692] Updated weights for policy 0, policy_version 1524484 (0.0007) [2023-12-27 02:29:50,834][105620] Updated weights for policy 1, policy_version 1527153 (0.0010) [2023-12-27 02:29:50,903][105620] Updated weights for policy 1, policy_version 1527163 (0.0011) [2023-12-27 02:29:50,969][105620] Updated weights for policy 1, policy_version 1527173 (0.0011) [2023-12-27 02:29:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 781336576. Throughput: 0: 9846.0, 1: 9829.3. Samples: 781321164. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:29:51,063][104569] Avg episode reward: [(0, '8627.604'), (1, '8812.298')] [2023-12-27 02:29:51,562][105692] Updated weights for policy 0, policy_version 1524494 (0.0006) [2023-12-27 02:29:51,629][105692] Updated weights for policy 0, policy_version 1524504 (0.0007) [2023-12-27 02:29:51,689][105692] Updated weights for policy 0, policy_version 1524514 (0.0010) [2023-12-27 02:29:51,716][105620] Updated weights for policy 1, policy_version 1527183 (0.0008) [2023-12-27 02:29:51,783][105620] Updated weights for policy 1, policy_version 1527193 (0.0010) [2023-12-27 02:29:51,853][105620] Updated weights for policy 1, policy_version 1527203 (0.0011) [2023-12-27 02:29:52,414][105692] Updated weights for policy 0, policy_version 1524524 (0.0010) [2023-12-27 02:29:52,463][105692] Updated weights for policy 0, policy_version 1524534 (0.0010) [2023-12-27 02:29:52,519][105692] Updated weights for policy 0, policy_version 1524544 (0.0010) [2023-12-27 02:29:52,550][105620] Updated weights for policy 1, policy_version 1527213 (0.0009) [2023-12-27 02:29:52,604][105620] Updated weights for policy 1, policy_version 1527223 (0.0010) [2023-12-27 02:29:52,649][105620] Updated weights for policy 1, policy_version 1527233 (0.0010) [2023-12-27 02:29:53,243][105620] Updated weights for policy 1, policy_version 1527243 (0.0009) [2023-12-27 02:29:53,285][105692] Updated weights for policy 0, policy_version 1524554 (0.0010) [2023-12-27 02:29:53,314][105620] Updated weights for policy 1, policy_version 1527253 (0.0006) [2023-12-27 02:29:53,348][105692] Updated weights for policy 0, policy_version 1524564 (0.0010) [2023-12-27 02:29:53,371][105620] Updated weights for policy 1, policy_version 1527263 (0.0007) [2023-12-27 02:29:53,410][105692] Updated weights for policy 0, policy_version 1524574 (0.0011) [2023-12-27 02:29:53,463][105692] Updated weights for policy 0, policy_version 1524584 (0.0007) [2023-12-27 02:29:54,100][105692] Updated weights for policy 0, policy_version 1524594 (0.0009) [2023-12-27 02:29:54,132][105620] Updated weights for policy 1, policy_version 1527273 (0.0007) [2023-12-27 02:29:54,152][105692] Updated weights for policy 0, policy_version 1524604 (0.0009) [2023-12-27 02:29:54,197][105620] Updated weights for policy 1, policy_version 1527283 (0.0008) [2023-12-27 02:29:54,206][105692] Updated weights for policy 0, policy_version 1524614 (0.0009) [2023-12-27 02:29:54,253][105620] Updated weights for policy 1, policy_version 1527293 (0.0007) [2023-12-27 02:29:54,308][105620] Updated weights for policy 1, policy_version 1527303 (0.0009) [2023-12-27 02:29:54,910][105620] Updated weights for policy 1, policy_version 1527313 (0.0006) [2023-12-27 02:29:54,971][105620] Updated weights for policy 1, policy_version 1527323 (0.0006) [2023-12-27 02:29:55,023][105620] Updated weights for policy 1, policy_version 1527333 (0.0006) [2023-12-27 02:29:55,081][105692] Updated weights for policy 0, policy_version 1524624 (0.0009) [2023-12-27 02:29:55,153][105692] Updated weights for policy 0, policy_version 1524634 (0.0010) [2023-12-27 02:29:55,215][105692] Updated weights for policy 0, policy_version 1524644 (0.0007) [2023-12-27 02:29:55,619][105620] Updated weights for policy 1, policy_version 1527343 (0.0006) [2023-12-27 02:29:55,672][105620] Updated weights for policy 1, policy_version 1527353 (0.0009) [2023-12-27 02:29:55,726][105620] Updated weights for policy 1, policy_version 1527363 (0.0010) [2023-12-27 02:29:55,857][105692] Updated weights for policy 0, policy_version 1524654 (0.0008) [2023-12-27 02:29:55,912][105692] Updated weights for policy 0, policy_version 1524664 (0.0008) [2023-12-27 02:29:55,969][105692] Updated weights for policy 0, policy_version 1524674 (0.0005) [2023-12-27 02:29:56,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19933.9, 300 sec: 19494.2). Total num frames: 781434880. Throughput: 0: 9839.0, 1: 9796.3. Samples: 781438480. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:29:56,062][104569] Avg episode reward: [(0, '8446.553'), (1, '8716.828')] [2023-12-27 02:29:56,308][105620] Updated weights for policy 1, policy_version 1527373 (0.0008) [2023-12-27 02:29:56,369][105620] Updated weights for policy 1, policy_version 1527383 (0.0005) [2023-12-27 02:29:56,431][105620] Updated weights for policy 1, policy_version 1527393 (0.0005) [2023-12-27 02:29:56,638][105692] Updated weights for policy 0, policy_version 1524684 (0.0007) [2023-12-27 02:29:56,693][105692] Updated weights for policy 0, policy_version 1524695 (0.0010) [2023-12-27 02:29:56,746][105692] Updated weights for policy 0, policy_version 1524705 (0.0009) [2023-12-27 02:29:56,989][105620] Updated weights for policy 1, policy_version 1527403 (0.0007) [2023-12-27 02:29:57,043][105620] Updated weights for policy 1, policy_version 1527413 (0.0010) [2023-12-27 02:29:57,096][105620] Updated weights for policy 1, policy_version 1527423 (0.0010) [2023-12-27 02:29:57,572][105692] Updated weights for policy 0, policy_version 1524715 (0.0009) [2023-12-27 02:29:57,624][105692] Updated weights for policy 0, policy_version 1524725 (0.0009) [2023-12-27 02:29:57,678][105692] Updated weights for policy 0, policy_version 1524735 (0.0009) [2023-12-27 02:29:57,733][105620] Updated weights for policy 1, policy_version 1527433 (0.0010) [2023-12-27 02:29:57,791][105620] Updated weights for policy 1, policy_version 1527443 (0.0005) [2023-12-27 02:29:57,843][105620] Updated weights for policy 1, policy_version 1527453 (0.0005) [2023-12-27 02:29:57,897][105620] Updated weights for policy 1, policy_version 1527463 (0.0005) [2023-12-27 02:29:58,434][105692] Updated weights for policy 0, policy_version 1524745 (0.0008) [2023-12-27 02:29:58,511][105692] Updated weights for policy 0, policy_version 1524755 (0.0008) [2023-12-27 02:29:58,554][105620] Updated weights for policy 1, policy_version 1527473 (0.0007) [2023-12-27 02:29:58,576][105692] Updated weights for policy 0, policy_version 1524765 (0.0007) [2023-12-27 02:29:58,630][105620] Updated weights for policy 1, policy_version 1527483 (0.0007) [2023-12-27 02:29:58,635][105692] Updated weights for policy 0, policy_version 1524775 (0.0010) [2023-12-27 02:29:58,683][105620] Updated weights for policy 1, policy_version 1527493 (0.0008) [2023-12-27 02:29:59,441][105692] Updated weights for policy 0, policy_version 1524785 (0.0007) [2023-12-27 02:29:59,493][105692] Updated weights for policy 0, policy_version 1524795 (0.0005) [2023-12-27 02:29:59,561][105692] Updated weights for policy 0, policy_version 1524805 (0.0005) [2023-12-27 02:29:59,578][105620] Updated weights for policy 1, policy_version 1527503 (0.0007) [2023-12-27 02:29:59,647][105620] Updated weights for policy 1, policy_version 1527513 (0.0009) [2023-12-27 02:29:59,712][105620] Updated weights for policy 1, policy_version 1527523 (0.0009) [2023-12-27 02:30:00,159][105692] Updated weights for policy 0, policy_version 1524815 (0.0006) [2023-12-27 02:30:00,221][105692] Updated weights for policy 0, policy_version 1524825 (0.0006) [2023-12-27 02:30:00,279][105692] Updated weights for policy 0, policy_version 1524835 (0.0005) [2023-12-27 02:30:00,498][105620] Updated weights for policy 1, policy_version 1527533 (0.0009) [2023-12-27 02:30:00,555][105620] Updated weights for policy 1, policy_version 1527543 (0.0010) [2023-12-27 02:30:00,613][105620] Updated weights for policy 1, policy_version 1527553 (0.0010) [2023-12-27 02:30:00,803][105692] Updated weights for policy 0, policy_version 1524845 (0.0005) [2023-12-27 02:30:00,850][105692] Updated weights for policy 0, policy_version 1524855 (0.0005) [2023-12-27 02:30:00,901][105692] Updated weights for policy 0, policy_version 1524865 (0.0007) [2023-12-27 02:30:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19494.2). Total num frames: 781533184. Throughput: 0: 9887.5, 1: 9884.3. Samples: 781500064. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:30:01,063][104569] Avg episode reward: [(0, '8266.928'), (1, '8988.360')] [2023-12-27 02:30:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001524872_390422528.pth... [2023-12-27 02:30:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001527560_391110656.pth... [2023-12-27 02:30:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001523720_390127616.pth [2023-12-27 02:30:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001526408_390815744.pth [2023-12-27 02:30:01,500][105620] Updated weights for policy 1, policy_version 1527563 (0.0010) [2023-12-27 02:30:01,559][105620] Updated weights for policy 1, policy_version 1527573 (0.0009) [2023-12-27 02:30:01,623][105620] Updated weights for policy 1, policy_version 1527583 (0.0008) [2023-12-27 02:30:01,638][105692] Updated weights for policy 0, policy_version 1524875 (0.0008) [2023-12-27 02:30:01,700][105692] Updated weights for policy 0, policy_version 1524885 (0.0007) [2023-12-27 02:30:01,736][105585] KL-divergence is very high: 112.3687 [2023-12-27 02:30:01,765][105692] Updated weights for policy 0, policy_version 1524895 (0.0006) [2023-12-27 02:30:02,339][105620] Updated weights for policy 1, policy_version 1527593 (0.0007) [2023-12-27 02:30:02,408][105620] Updated weights for policy 1, policy_version 1527603 (0.0008) [2023-12-27 02:30:02,464][105620] Updated weights for policy 1, policy_version 1527613 (0.0008) [2023-12-27 02:30:02,499][105692] Updated weights for policy 0, policy_version 1524905 (0.0008) [2023-12-27 02:30:02,523][105620] Updated weights for policy 1, policy_version 1527623 (0.0007) [2023-12-27 02:30:02,560][105692] Updated weights for policy 0, policy_version 1524915 (0.0010) [2023-12-27 02:30:02,619][105692] Updated weights for policy 0, policy_version 1524925 (0.0010) [2023-12-27 02:30:02,676][105692] Updated weights for policy 0, policy_version 1524935 (0.0010) [2023-12-27 02:30:03,285][105620] Updated weights for policy 1, policy_version 1527633 (0.0010) [2023-12-27 02:30:03,347][105620] Updated weights for policy 1, policy_version 1527643 (0.0010) [2023-12-27 02:30:03,349][105692] Updated weights for policy 0, policy_version 1524945 (0.0006) [2023-12-27 02:30:03,395][105692] Updated weights for policy 0, policy_version 1524955 (0.0006) [2023-12-27 02:30:03,396][105620] Updated weights for policy 1, policy_version 1527653 (0.0010) [2023-12-27 02:30:03,440][105692] Updated weights for policy 0, policy_version 1524965 (0.0006) [2023-12-27 02:30:04,066][105620] Updated weights for policy 1, policy_version 1527663 (0.0007) [2023-12-27 02:30:04,132][105620] Updated weights for policy 1, policy_version 1527673 (0.0006) [2023-12-27 02:30:04,201][105620] Updated weights for policy 1, policy_version 1527683 (0.0011) [2023-12-27 02:30:04,232][105692] Updated weights for policy 0, policy_version 1524975 (0.0006) [2023-12-27 02:30:04,295][105692] Updated weights for policy 0, policy_version 1524985 (0.0008) [2023-12-27 02:30:04,353][105692] Updated weights for policy 0, policy_version 1524995 (0.0009) [2023-12-27 02:30:04,909][105620] Updated weights for policy 1, policy_version 1527693 (0.0011) [2023-12-27 02:30:04,967][105620] Updated weights for policy 1, policy_version 1527703 (0.0010) [2023-12-27 02:30:05,026][105620] Updated weights for policy 1, policy_version 1527713 (0.0011) [2023-12-27 02:30:05,072][105692] Updated weights for policy 0, policy_version 1525005 (0.0007) [2023-12-27 02:30:05,136][105692] Updated weights for policy 0, policy_version 1525015 (0.0007) [2023-12-27 02:30:05,200][105692] Updated weights for policy 0, policy_version 1525025 (0.0007) [2023-12-27 02:30:05,752][105620] Updated weights for policy 1, policy_version 1527723 (0.0011) [2023-12-27 02:30:05,796][105620] Updated weights for policy 1, policy_version 1527733 (0.0010) [2023-12-27 02:30:05,844][105620] Updated weights for policy 1, policy_version 1527743 (0.0010) [2023-12-27 02:30:05,916][105692] Updated weights for policy 0, policy_version 1525035 (0.0008) [2023-12-27 02:30:05,970][105692] Updated weights for policy 0, policy_version 1525045 (0.0008) [2023-12-27 02:30:06,029][105692] Updated weights for policy 0, policy_version 1525055 (0.0005) [2023-12-27 02:30:06,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 781623296. Throughput: 0: 9911.0, 1: 9830.6. Samples: 781614824. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:30:06,063][104569] Avg episode reward: [(0, '8450.778'), (1, '8989.061')] [2023-12-27 02:30:06,538][105620] Updated weights for policy 1, policy_version 1527753 (0.0010) [2023-12-27 02:30:06,600][105620] Updated weights for policy 1, policy_version 1527763 (0.0011) [2023-12-27 02:30:06,657][105620] Updated weights for policy 1, policy_version 1527773 (0.0011) [2023-12-27 02:30:06,679][105692] Updated weights for policy 0, policy_version 1525065 (0.0005) [2023-12-27 02:30:06,714][105620] Updated weights for policy 1, policy_version 1527783 (0.0011) [2023-12-27 02:30:06,742][105692] Updated weights for policy 0, policy_version 1525075 (0.0008) [2023-12-27 02:30:06,805][105692] Updated weights for policy 0, policy_version 1525085 (0.0008) [2023-12-27 02:30:06,869][105692] Updated weights for policy 0, policy_version 1525095 (0.0008) [2023-12-27 02:30:07,340][105620] Updated weights for policy 1, policy_version 1527793 (0.0006) [2023-12-27 02:30:07,397][105620] Updated weights for policy 1, policy_version 1527803 (0.0010) [2023-12-27 02:30:07,450][105620] Updated weights for policy 1, policy_version 1527813 (0.0011) [2023-12-27 02:30:07,712][105692] Updated weights for policy 0, policy_version 1525105 (0.0008) [2023-12-27 02:30:07,771][105692] Updated weights for policy 0, policy_version 1525115 (0.0008) [2023-12-27 02:30:07,829][105692] Updated weights for policy 0, policy_version 1525125 (0.0007) [2023-12-27 02:30:08,098][105620] Updated weights for policy 1, policy_version 1527823 (0.0011) [2023-12-27 02:30:08,167][105620] Updated weights for policy 1, policy_version 1527833 (0.0011) [2023-12-27 02:30:08,222][105620] Updated weights for policy 1, policy_version 1527843 (0.0011) [2023-12-27 02:30:08,550][105692] Updated weights for policy 0, policy_version 1525135 (0.0008) [2023-12-27 02:30:08,606][105692] Updated weights for policy 0, policy_version 1525145 (0.0008) [2023-12-27 02:30:08,658][105692] Updated weights for policy 0, policy_version 1525155 (0.0008) [2023-12-27 02:30:08,953][105620] Updated weights for policy 1, policy_version 1527853 (0.0008) [2023-12-27 02:30:09,016][105620] Updated weights for policy 1, policy_version 1527863 (0.0007) [2023-12-27 02:30:09,078][105620] Updated weights for policy 1, policy_version 1527873 (0.0006) [2023-12-27 02:30:09,485][105692] Updated weights for policy 0, policy_version 1525165 (0.0008) [2023-12-27 02:30:09,550][105692] Updated weights for policy 0, policy_version 1525175 (0.0009) [2023-12-27 02:30:09,612][105692] Updated weights for policy 0, policy_version 1525185 (0.0009) [2023-12-27 02:30:09,748][105620] Updated weights for policy 1, policy_version 1527883 (0.0007) [2023-12-27 02:30:09,812][105620] Updated weights for policy 1, policy_version 1527893 (0.0011) [2023-12-27 02:30:09,878][105620] Updated weights for policy 1, policy_version 1527903 (0.0009) [2023-12-27 02:30:10,330][105692] Updated weights for policy 0, policy_version 1525195 (0.0008) [2023-12-27 02:30:10,389][105692] Updated weights for policy 0, policy_version 1525205 (0.0006) [2023-12-27 02:30:10,448][105692] Updated weights for policy 0, policy_version 1525215 (0.0007) [2023-12-27 02:30:10,644][105620] Updated weights for policy 1, policy_version 1527913 (0.0011) [2023-12-27 02:30:10,695][105620] Updated weights for policy 1, policy_version 1527923 (0.0010) [2023-12-27 02:30:10,754][105620] Updated weights for policy 1, policy_version 1527933 (0.0011) [2023-12-27 02:30:10,811][105620] Updated weights for policy 1, policy_version 1527943 (0.0006) [2023-12-27 02:30:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 781721600. Throughput: 0: 9849.3, 1: 9950.6. Samples: 781730988. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:30:11,062][104569] Avg episode reward: [(0, '8449.057'), (1, '9170.768')] [2023-12-27 02:30:11,073][105692] Updated weights for policy 0, policy_version 1525225 (0.0007) [2023-12-27 02:30:11,145][105692] Updated weights for policy 0, policy_version 1525235 (0.0007) [2023-12-27 02:30:11,211][105692] Updated weights for policy 0, policy_version 1525245 (0.0009) [2023-12-27 02:30:11,277][105692] Updated weights for policy 0, policy_version 1525255 (0.0009) [2023-12-27 02:30:11,573][105620] Updated weights for policy 1, policy_version 1527953 (0.0009) [2023-12-27 02:30:11,629][105620] Updated weights for policy 1, policy_version 1527963 (0.0009) [2023-12-27 02:30:11,694][105620] Updated weights for policy 1, policy_version 1527973 (0.0009) [2023-12-27 02:30:12,033][105692] Updated weights for policy 0, policy_version 1525265 (0.0009) [2023-12-27 02:30:12,095][105692] Updated weights for policy 0, policy_version 1525275 (0.0009) [2023-12-27 02:30:12,158][105692] Updated weights for policy 0, policy_version 1525285 (0.0009) [2023-12-27 02:30:12,469][105620] Updated weights for policy 1, policy_version 1527983 (0.0009) [2023-12-27 02:30:12,532][105620] Updated weights for policy 1, policy_version 1527993 (0.0008) [2023-12-27 02:30:12,594][105620] Updated weights for policy 1, policy_version 1528003 (0.0009) [2023-12-27 02:30:12,910][105692] Updated weights for policy 0, policy_version 1525295 (0.0009) [2023-12-27 02:30:12,962][105692] Updated weights for policy 0, policy_version 1525305 (0.0009) [2023-12-27 02:30:13,030][105692] Updated weights for policy 0, policy_version 1525315 (0.0008) [2023-12-27 02:30:13,307][105620] Updated weights for policy 1, policy_version 1528013 (0.0007) [2023-12-27 02:30:13,361][105620] Updated weights for policy 1, policy_version 1528023 (0.0005) [2023-12-27 02:30:13,412][105620] Updated weights for policy 1, policy_version 1528033 (0.0005) [2023-12-27 02:30:13,852][105692] Updated weights for policy 0, policy_version 1525325 (0.0009) [2023-12-27 02:30:13,915][105692] Updated weights for policy 0, policy_version 1525335 (0.0009) [2023-12-27 02:30:13,971][105692] Updated weights for policy 0, policy_version 1525345 (0.0008) [2023-12-27 02:30:13,977][105620] Updated weights for policy 1, policy_version 1528043 (0.0006) [2023-12-27 02:30:14,032][105620] Updated weights for policy 1, policy_version 1528053 (0.0008) [2023-12-27 02:30:14,092][105620] Updated weights for policy 1, policy_version 1528063 (0.0008) [2023-12-27 02:30:14,770][105692] Updated weights for policy 0, policy_version 1525355 (0.0006) [2023-12-27 02:30:14,805][105620] Updated weights for policy 1, policy_version 1528073 (0.0009) [2023-12-27 02:30:14,833][105692] Updated weights for policy 0, policy_version 1525365 (0.0008) [2023-12-27 02:30:14,863][105620] Updated weights for policy 1, policy_version 1528083 (0.0012) [2023-12-27 02:30:14,885][105692] Updated weights for policy 0, policy_version 1525375 (0.0008) [2023-12-27 02:30:14,920][105620] Updated weights for policy 1, policy_version 1528093 (0.0011) [2023-12-27 02:30:14,976][105620] Updated weights for policy 1, policy_version 1528103 (0.0010) [2023-12-27 02:30:15,670][105620] Updated weights for policy 1, policy_version 1528113 (0.0009) [2023-12-27 02:30:15,682][105692] Updated weights for policy 0, policy_version 1525385 (0.0008) [2023-12-27 02:30:15,723][105620] Updated weights for policy 1, policy_version 1528123 (0.0006) [2023-12-27 02:30:15,734][105692] Updated weights for policy 0, policy_version 1525395 (0.0008) [2023-12-27 02:30:15,782][105620] Updated weights for policy 1, policy_version 1528133 (0.0009) [2023-12-27 02:30:15,793][105692] Updated weights for policy 0, policy_version 1525405 (0.0007) [2023-12-27 02:30:15,850][105692] Updated weights for policy 0, policy_version 1525415 (0.0006) [2023-12-27 02:30:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19660.7, 300 sec: 19494.2). Total num frames: 781819904. Throughput: 0: 9745.4, 1: 9930.3. Samples: 781788216. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:30:16,064][104569] Avg episode reward: [(0, '8539.617'), (1, '9084.662')] [2023-12-27 02:30:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001525416_390561792.pth... [2023-12-27 02:30:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001528136_391258112.pth... [2023-12-27 02:30:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001524296_390275072.pth [2023-12-27 02:30:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001526984_390963200.pth [2023-12-27 02:30:16,424][105692] Updated weights for policy 0, policy_version 1525425 (0.0005) [2023-12-27 02:30:16,428][105620] Updated weights for policy 1, policy_version 1528143 (0.0010) [2023-12-27 02:30:16,481][105620] Updated weights for policy 1, policy_version 1528153 (0.0007) [2023-12-27 02:30:16,487][105692] Updated weights for policy 0, policy_version 1525435 (0.0006) [2023-12-27 02:30:16,539][105620] Updated weights for policy 1, policy_version 1528163 (0.0006) [2023-12-27 02:30:16,547][105692] Updated weights for policy 0, policy_version 1525445 (0.0008) [2023-12-27 02:30:17,114][105620] Updated weights for policy 1, policy_version 1528173 (0.0006) [2023-12-27 02:30:17,183][105620] Updated weights for policy 1, policy_version 1528183 (0.0007) [2023-12-27 02:30:17,244][105692] Updated weights for policy 0, policy_version 1525455 (0.0009) [2023-12-27 02:30:17,247][105620] Updated weights for policy 1, policy_version 1528193 (0.0008) [2023-12-27 02:30:17,300][105692] Updated weights for policy 0, policy_version 1525465 (0.0011) [2023-12-27 02:30:17,357][105692] Updated weights for policy 0, policy_version 1525475 (0.0010) [2023-12-27 02:30:17,911][105620] Updated weights for policy 1, policy_version 1528203 (0.0010) [2023-12-27 02:30:17,947][105692] Updated weights for policy 0, policy_version 1525485 (0.0008) [2023-12-27 02:30:17,969][105620] Updated weights for policy 1, policy_version 1528213 (0.0011) [2023-12-27 02:30:17,999][105692] Updated weights for policy 0, policy_version 1525495 (0.0010) [2023-12-27 02:30:18,029][105620] Updated weights for policy 1, policy_version 1528223 (0.0008) [2023-12-27 02:30:18,055][105692] Updated weights for policy 0, policy_version 1525505 (0.0010) [2023-12-27 02:30:18,742][105620] Updated weights for policy 1, policy_version 1528233 (0.0006) [2023-12-27 02:30:18,801][105620] Updated weights for policy 1, policy_version 1528243 (0.0011) [2023-12-27 02:30:18,804][105692] Updated weights for policy 0, policy_version 1525515 (0.0010) [2023-12-27 02:30:18,861][105620] Updated weights for policy 1, policy_version 1528253 (0.0011) [2023-12-27 02:30:18,864][105692] Updated weights for policy 0, policy_version 1525525 (0.0010) [2023-12-27 02:30:18,921][105620] Updated weights for policy 1, policy_version 1528263 (0.0011) [2023-12-27 02:30:18,923][105692] Updated weights for policy 0, policy_version 1525535 (0.0009) [2023-12-27 02:30:19,597][105620] Updated weights for policy 1, policy_version 1528273 (0.0006) [2023-12-27 02:30:19,659][105620] Updated weights for policy 1, policy_version 1528283 (0.0008) [2023-12-27 02:30:19,677][105692] Updated weights for policy 0, policy_version 1525545 (0.0006) [2023-12-27 02:30:19,721][105620] Updated weights for policy 1, policy_version 1528293 (0.0005) [2023-12-27 02:30:19,739][105692] Updated weights for policy 0, policy_version 1525555 (0.0008) [2023-12-27 02:30:19,801][105692] Updated weights for policy 0, policy_version 1525565 (0.0007) [2023-12-27 02:30:19,867][105692] Updated weights for policy 0, policy_version 1525575 (0.0008) [2023-12-27 02:30:20,429][105620] Updated weights for policy 1, policy_version 1528303 (0.0008) [2023-12-27 02:30:20,480][105620] Updated weights for policy 1, policy_version 1528313 (0.0009) [2023-12-27 02:30:20,540][105620] Updated weights for policy 1, policy_version 1528323 (0.0009) [2023-12-27 02:30:20,619][105692] Updated weights for policy 0, policy_version 1525585 (0.0009) [2023-12-27 02:30:20,672][105692] Updated weights for policy 0, policy_version 1525596 (0.0009) [2023-12-27 02:30:20,721][105692] Updated weights for policy 0, policy_version 1525606 (0.0009) [2023-12-27 02:30:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 781918208. Throughput: 0: 9700.8, 1: 9960.3. Samples: 781908256. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:30:21,062][104569] Avg episode reward: [(0, '8810.612'), (1, '8993.437')] [2023-12-27 02:30:21,313][105620] Updated weights for policy 1, policy_version 1528333 (0.0008) [2023-12-27 02:30:21,385][105620] Updated weights for policy 1, policy_version 1528343 (0.0008) [2023-12-27 02:30:21,455][105620] Updated weights for policy 1, policy_version 1528353 (0.0007) [2023-12-27 02:30:21,564][105692] Updated weights for policy 0, policy_version 1525616 (0.0009) [2023-12-27 02:30:21,623][105692] Updated weights for policy 0, policy_version 1525626 (0.0009) [2023-12-27 02:30:21,689][105692] Updated weights for policy 0, policy_version 1525636 (0.0009) [2023-12-27 02:30:22,185][105620] Updated weights for policy 1, policy_version 1528363 (0.0009) [2023-12-27 02:30:22,248][105620] Updated weights for policy 1, policy_version 1528373 (0.0009) [2023-12-27 02:30:22,319][105620] Updated weights for policy 1, policy_version 1528383 (0.0010) [2023-12-27 02:30:22,476][105692] Updated weights for policy 0, policy_version 1525646 (0.0009) [2023-12-27 02:30:22,529][105692] Updated weights for policy 0, policy_version 1525656 (0.0009) [2023-12-27 02:30:22,578][105692] Updated weights for policy 0, policy_version 1525666 (0.0009) [2023-12-27 02:30:23,112][105620] Updated weights for policy 1, policy_version 1528393 (0.0010) [2023-12-27 02:30:23,178][105620] Updated weights for policy 1, policy_version 1528403 (0.0009) [2023-12-27 02:30:23,230][105620] Updated weights for policy 1, policy_version 1528413 (0.0009) [2023-12-27 02:30:23,278][105620] Updated weights for policy 1, policy_version 1528423 (0.0009) [2023-12-27 02:30:23,369][105692] Updated weights for policy 0, policy_version 1525676 (0.0009) [2023-12-27 02:30:23,432][105692] Updated weights for policy 0, policy_version 1525686 (0.0009) [2023-12-27 02:30:23,492][105692] Updated weights for policy 0, policy_version 1525696 (0.0009) [2023-12-27 02:30:24,036][105620] Updated weights for policy 1, policy_version 1528433 (0.0009) [2023-12-27 02:30:24,092][105620] Updated weights for policy 1, policy_version 1528443 (0.0009) [2023-12-27 02:30:24,148][105620] Updated weights for policy 1, policy_version 1528453 (0.0009) [2023-12-27 02:30:24,227][105692] Updated weights for policy 0, policy_version 1525706 (0.0009) [2023-12-27 02:30:24,278][105692] Updated weights for policy 0, policy_version 1525716 (0.0009) [2023-12-27 02:30:24,326][105692] Updated weights for policy 0, policy_version 1525726 (0.0009) [2023-12-27 02:30:24,384][105692] Updated weights for policy 0, policy_version 1525736 (0.0005) [2023-12-27 02:30:24,865][105620] Updated weights for policy 1, policy_version 1528463 (0.0006) [2023-12-27 02:30:24,911][105620] Updated weights for policy 1, policy_version 1528473 (0.0005) [2023-12-27 02:30:24,961][105620] Updated weights for policy 1, policy_version 1528483 (0.0005) [2023-12-27 02:30:25,126][105692] Updated weights for policy 0, policy_version 1525746 (0.0008) [2023-12-27 02:30:25,183][105692] Updated weights for policy 0, policy_version 1525756 (0.0009) [2023-12-27 02:30:25,241][105692] Updated weights for policy 0, policy_version 1525766 (0.0009) [2023-12-27 02:30:25,646][105620] Updated weights for policy 1, policy_version 1528493 (0.0007) [2023-12-27 02:30:25,696][105620] Updated weights for policy 1, policy_version 1528503 (0.0008) [2023-12-27 02:30:25,743][105620] Updated weights for policy 1, policy_version 1528513 (0.0008) [2023-12-27 02:30:25,998][105692] Updated weights for policy 0, policy_version 1525776 (0.0009) [2023-12-27 02:30:26,050][105692] Updated weights for policy 0, policy_version 1525786 (0.0007) [2023-12-27 02:30:26,062][104569] Fps is (10 sec: 18842.3, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 782008320. Throughput: 0: 9592.2, 1: 9861.7. Samples: 782018432. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:30:26,063][104569] Avg episode reward: [(0, '8723.959'), (1, '9081.907')] [2023-12-27 02:30:26,103][105692] Updated weights for policy 0, policy_version 1525796 (0.0007) [2023-12-27 02:30:26,517][105620] Updated weights for policy 1, policy_version 1528523 (0.0009) [2023-12-27 02:30:26,566][105620] Updated weights for policy 1, policy_version 1528533 (0.0008) [2023-12-27 02:30:26,615][105620] Updated weights for policy 1, policy_version 1528543 (0.0008) [2023-12-27 02:30:26,841][105692] Updated weights for policy 0, policy_version 1525806 (0.0009) [2023-12-27 02:30:26,887][105692] Updated weights for policy 0, policy_version 1525816 (0.0009) [2023-12-27 02:30:26,934][105692] Updated weights for policy 0, policy_version 1525826 (0.0008) [2023-12-27 02:30:27,363][105620] Updated weights for policy 1, policy_version 1528553 (0.0009) [2023-12-27 02:30:27,424][105620] Updated weights for policy 1, policy_version 1528563 (0.0005) [2023-12-27 02:30:27,476][105620] Updated weights for policy 1, policy_version 1528573 (0.0005) [2023-12-27 02:30:27,523][105620] Updated weights for policy 1, policy_version 1528583 (0.0005) [2023-12-27 02:30:27,663][105692] Updated weights for policy 0, policy_version 1525836 (0.0007) [2023-12-27 02:30:27,713][105692] Updated weights for policy 0, policy_version 1525846 (0.0005) [2023-12-27 02:30:27,761][105692] Updated weights for policy 0, policy_version 1525856 (0.0005) [2023-12-27 02:30:28,178][105620] Updated weights for policy 1, policy_version 1528593 (0.0008) [2023-12-27 02:30:28,232][105620] Updated weights for policy 1, policy_version 1528603 (0.0009) [2023-12-27 02:30:28,289][105620] Updated weights for policy 1, policy_version 1528613 (0.0009) [2023-12-27 02:30:28,419][105692] Updated weights for policy 0, policy_version 1525866 (0.0006) [2023-12-27 02:30:28,467][105692] Updated weights for policy 0, policy_version 1525876 (0.0008) [2023-12-27 02:30:28,514][105692] Updated weights for policy 0, policy_version 1525886 (0.0009) [2023-12-27 02:30:28,563][105692] Updated weights for policy 0, policy_version 1525896 (0.0009) [2023-12-27 02:30:29,043][105620] Updated weights for policy 1, policy_version 1528623 (0.0009) [2023-12-27 02:30:29,108][105620] Updated weights for policy 1, policy_version 1528633 (0.0009) [2023-12-27 02:30:29,168][105620] Updated weights for policy 1, policy_version 1528643 (0.0009) [2023-12-27 02:30:29,380][105692] Updated weights for policy 0, policy_version 1525906 (0.0009) [2023-12-27 02:30:29,447][105692] Updated weights for policy 0, policy_version 1525916 (0.0009) [2023-12-27 02:30:29,505][105692] Updated weights for policy 0, policy_version 1525926 (0.0009) [2023-12-27 02:30:29,894][105620] Updated weights for policy 1, policy_version 1528653 (0.0009) [2023-12-27 02:30:29,961][105620] Updated weights for policy 1, policy_version 1528663 (0.0009) [2023-12-27 02:30:30,019][105620] Updated weights for policy 1, policy_version 1528673 (0.0008) [2023-12-27 02:30:30,242][105692] Updated weights for policy 0, policy_version 1525936 (0.0009) [2023-12-27 02:30:30,291][105692] Updated weights for policy 0, policy_version 1525946 (0.0008) [2023-12-27 02:30:30,344][105692] Updated weights for policy 0, policy_version 1525956 (0.0007) [2023-12-27 02:30:30,716][105620] Updated weights for policy 1, policy_version 1528683 (0.0009) [2023-12-27 02:30:30,773][105620] Updated weights for policy 1, policy_version 1528693 (0.0008) [2023-12-27 02:30:30,829][105620] Updated weights for policy 1, policy_version 1528703 (0.0006) [2023-12-27 02:30:30,978][105692] Updated weights for policy 0, policy_version 1525966 (0.0007) [2023-12-27 02:30:31,039][105692] Updated weights for policy 0, policy_version 1525976 (0.0008) [2023-12-27 02:30:31,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 782106624. Throughput: 0: 9608.5, 1: 9857.6. Samples: 782077456. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:30:31,063][104569] Avg episode reward: [(0, '8631.981'), (1, '9078.798')] [2023-12-27 02:30:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001528712_391405568.pth... [2023-12-27 02:30:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001527560_391110656.pth [2023-12-27 02:30:31,094][105692] Updated weights for policy 0, policy_version 1525986 (0.0011) [2023-12-27 02:30:31,119][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001525992_390709248.pth... [2023-12-27 02:30:31,123][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001524872_390422528.pth [2023-12-27 02:30:31,508][105620] Updated weights for policy 1, policy_version 1528713 (0.0006) [2023-12-27 02:30:31,576][105620] Updated weights for policy 1, policy_version 1528723 (0.0010) [2023-12-27 02:30:31,640][105620] Updated weights for policy 1, policy_version 1528733 (0.0009) [2023-12-27 02:30:31,702][105620] Updated weights for policy 1, policy_version 1528743 (0.0009) [2023-12-27 02:30:31,823][105692] Updated weights for policy 0, policy_version 1525996 (0.0010) [2023-12-27 02:30:31,880][105692] Updated weights for policy 0, policy_version 1526006 (0.0008) [2023-12-27 02:30:31,934][105692] Updated weights for policy 0, policy_version 1526016 (0.0005) [2023-12-27 02:30:32,463][105620] Updated weights for policy 1, policy_version 1528753 (0.0009) [2023-12-27 02:30:32,531][105620] Updated weights for policy 1, policy_version 1528763 (0.0009) [2023-12-27 02:30:32,597][105620] Updated weights for policy 1, policy_version 1528773 (0.0009) [2023-12-27 02:30:32,649][105692] Updated weights for policy 0, policy_version 1526026 (0.0007) [2023-12-27 02:30:32,712][105692] Updated weights for policy 0, policy_version 1526036 (0.0009) [2023-12-27 02:30:32,776][105692] Updated weights for policy 0, policy_version 1526046 (0.0009) [2023-12-27 02:30:32,832][105692] Updated weights for policy 0, policy_version 1526056 (0.0010) [2023-12-27 02:30:33,339][105620] Updated weights for policy 1, policy_version 1528783 (0.0007) [2023-12-27 02:30:33,402][105620] Updated weights for policy 1, policy_version 1528793 (0.0007) [2023-12-27 02:30:33,464][105620] Updated weights for policy 1, policy_version 1528803 (0.0007) [2023-12-27 02:30:33,546][105692] Updated weights for policy 0, policy_version 1526066 (0.0009) [2023-12-27 02:30:33,595][105692] Updated weights for policy 0, policy_version 1526076 (0.0008) [2023-12-27 02:30:33,655][105692] Updated weights for policy 0, policy_version 1526086 (0.0006) [2023-12-27 02:30:34,089][105620] Updated weights for policy 1, policy_version 1528813 (0.0007) [2023-12-27 02:30:34,149][105620] Updated weights for policy 1, policy_version 1528823 (0.0008) [2023-12-27 02:30:34,216][105620] Updated weights for policy 1, policy_version 1528833 (0.0009) [2023-12-27 02:30:34,391][105692] Updated weights for policy 0, policy_version 1526096 (0.0010) [2023-12-27 02:30:34,448][105692] Updated weights for policy 0, policy_version 1526106 (0.0010) [2023-12-27 02:30:34,510][105692] Updated weights for policy 0, policy_version 1526116 (0.0009) [2023-12-27 02:30:34,844][105620] Updated weights for policy 1, policy_version 1528843 (0.0006) [2023-12-27 02:30:34,907][105620] Updated weights for policy 1, policy_version 1528853 (0.0009) [2023-12-27 02:30:34,965][105620] Updated weights for policy 1, policy_version 1528863 (0.0006) [2023-12-27 02:30:35,351][105692] Updated weights for policy 0, policy_version 1526126 (0.0007) [2023-12-27 02:30:35,403][105692] Updated weights for policy 0, policy_version 1526136 (0.0006) [2023-12-27 02:30:35,456][105692] Updated weights for policy 0, policy_version 1526146 (0.0005) [2023-12-27 02:30:35,687][105620] Updated weights for policy 1, policy_version 1528873 (0.0006) [2023-12-27 02:30:35,740][105620] Updated weights for policy 1, policy_version 1528883 (0.0005) [2023-12-27 02:30:35,785][105620] Updated weights for policy 1, policy_version 1528893 (0.0007) [2023-12-27 02:30:35,835][105620] Updated weights for policy 1, policy_version 1528903 (0.0008) [2023-12-27 02:30:36,032][105692] Updated weights for policy 0, policy_version 1526156 (0.0007) [2023-12-27 02:30:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 782204928. Throughput: 0: 9556.1, 1: 9852.7. Samples: 782194560. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:30:36,062][104569] Avg episode reward: [(0, '8722.727'), (1, '9081.920')] [2023-12-27 02:30:36,086][105692] Updated weights for policy 0, policy_version 1526167 (0.0010) [2023-12-27 02:30:36,143][105692] Updated weights for policy 0, policy_version 1526177 (0.0009) [2023-12-27 02:30:36,591][105620] Updated weights for policy 1, policy_version 1528913 (0.0008) [2023-12-27 02:30:36,638][105620] Updated weights for policy 1, policy_version 1528923 (0.0008) [2023-12-27 02:30:36,696][105620] Updated weights for policy 1, policy_version 1528933 (0.0009) [2023-12-27 02:30:36,938][105692] Updated weights for policy 0, policy_version 1526187 (0.0009) [2023-12-27 02:30:36,998][105692] Updated weights for policy 0, policy_version 1526197 (0.0009) [2023-12-27 02:30:37,054][105692] Updated weights for policy 0, policy_version 1526207 (0.0009) [2023-12-27 02:30:37,430][105620] Updated weights for policy 1, policy_version 1528943 (0.0008) [2023-12-27 02:30:37,491][105620] Updated weights for policy 1, policy_version 1528953 (0.0009) [2023-12-27 02:30:37,549][105620] Updated weights for policy 1, policy_version 1528963 (0.0009) [2023-12-27 02:30:37,832][105692] Updated weights for policy 0, policy_version 1526217 (0.0009) [2023-12-27 02:30:37,896][105692] Updated weights for policy 0, policy_version 1526227 (0.0010) [2023-12-27 02:30:37,943][105692] Updated weights for policy 0, policy_version 1526237 (0.0009) [2023-12-27 02:30:37,993][105692] Updated weights for policy 0, policy_version 1526247 (0.0009) [2023-12-27 02:30:38,262][105620] Updated weights for policy 1, policy_version 1528973 (0.0009) [2023-12-27 02:30:38,317][105620] Updated weights for policy 1, policy_version 1528983 (0.0009) [2023-12-27 02:30:38,376][105620] Updated weights for policy 1, policy_version 1528993 (0.0007) [2023-12-27 02:30:38,759][105692] Updated weights for policy 0, policy_version 1526257 (0.0009) [2023-12-27 02:30:38,810][105692] Updated weights for policy 0, policy_version 1526267 (0.0009) [2023-12-27 02:30:38,873][105692] Updated weights for policy 0, policy_version 1526277 (0.0009) [2023-12-27 02:30:39,134][105620] Updated weights for policy 1, policy_version 1529003 (0.0008) [2023-12-27 02:30:39,181][105620] Updated weights for policy 1, policy_version 1529013 (0.0009) [2023-12-27 02:30:39,251][105620] Updated weights for policy 1, policy_version 1529023 (0.0008) [2023-12-27 02:30:39,706][105692] Updated weights for policy 0, policy_version 1526287 (0.0009) [2023-12-27 02:30:39,770][105692] Updated weights for policy 0, policy_version 1526297 (0.0009) [2023-12-27 02:30:39,828][105692] Updated weights for policy 0, policy_version 1526307 (0.0009) [2023-12-27 02:30:39,913][105620] Updated weights for policy 1, policy_version 1529033 (0.0009) [2023-12-27 02:30:39,979][105620] Updated weights for policy 1, policy_version 1529043 (0.0009) [2023-12-27 02:30:40,042][105620] Updated weights for policy 1, policy_version 1529053 (0.0009) [2023-12-27 02:30:40,100][105620] Updated weights for policy 1, policy_version 1529063 (0.0010) [2023-12-27 02:30:40,452][105692] Updated weights for policy 0, policy_version 1526317 (0.0008) [2023-12-27 02:30:40,508][105692] Updated weights for policy 0, policy_version 1526327 (0.0009) [2023-12-27 02:30:40,562][105692] Updated weights for policy 0, policy_version 1526337 (0.0009) [2023-12-27 02:30:40,907][105620] Updated weights for policy 1, policy_version 1529073 (0.0009) [2023-12-27 02:30:40,960][105620] Updated weights for policy 1, policy_version 1529083 (0.0009) [2023-12-27 02:30:41,020][105620] Updated weights for policy 1, policy_version 1529094 (0.0009) [2023-12-27 02:30:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 782303232. Throughput: 0: 9546.9, 1: 9775.7. Samples: 782308000. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:30:41,062][104569] Avg episode reward: [(0, '8361.458'), (1, '9083.673')] [2023-12-27 02:30:41,310][105692] Updated weights for policy 0, policy_version 1526347 (0.0009) [2023-12-27 02:30:41,373][105692] Updated weights for policy 0, policy_version 1526357 (0.0010) [2023-12-27 02:30:41,440][105692] Updated weights for policy 0, policy_version 1526367 (0.0009) [2023-12-27 02:30:41,835][105620] Updated weights for policy 1, policy_version 1529104 (0.0006) [2023-12-27 02:30:41,903][105620] Updated weights for policy 1, policy_version 1529114 (0.0008) [2023-12-27 02:30:41,969][105620] Updated weights for policy 1, policy_version 1529124 (0.0010) [2023-12-27 02:30:42,150][105692] Updated weights for policy 0, policy_version 1526377 (0.0008) [2023-12-27 02:30:42,205][105692] Updated weights for policy 0, policy_version 1526387 (0.0008) [2023-12-27 02:30:42,272][105692] Updated weights for policy 0, policy_version 1526397 (0.0009) [2023-12-27 02:30:42,333][105692] Updated weights for policy 0, policy_version 1526407 (0.0009) [2023-12-27 02:30:42,660][105620] Updated weights for policy 1, policy_version 1529134 (0.0007) [2023-12-27 02:30:42,712][105620] Updated weights for policy 1, policy_version 1529144 (0.0005) [2023-12-27 02:30:42,763][105620] Updated weights for policy 1, policy_version 1529154 (0.0005) [2023-12-27 02:30:43,108][105692] Updated weights for policy 0, policy_version 1526417 (0.0009) [2023-12-27 02:30:43,173][105692] Updated weights for policy 0, policy_version 1526427 (0.0009) [2023-12-27 02:30:43,243][105692] Updated weights for policy 0, policy_version 1526437 (0.0010) [2023-12-27 02:30:43,330][105620] Updated weights for policy 1, policy_version 1529164 (0.0006) [2023-12-27 02:30:43,384][105620] Updated weights for policy 1, policy_version 1529174 (0.0010) [2023-12-27 02:30:43,438][105620] Updated weights for policy 1, policy_version 1529184 (0.0010) [2023-12-27 02:30:44,004][105620] Updated weights for policy 1, policy_version 1529194 (0.0007) [2023-12-27 02:30:44,070][105620] Updated weights for policy 1, policy_version 1529204 (0.0008) [2023-12-27 02:30:44,088][105692] Updated weights for policy 0, policy_version 1526447 (0.0008) [2023-12-27 02:30:44,123][105620] Updated weights for policy 1, policy_version 1529214 (0.0007) [2023-12-27 02:30:44,144][105692] Updated weights for policy 0, policy_version 1526457 (0.0007) [2023-12-27 02:30:44,179][105620] Updated weights for policy 1, policy_version 1529224 (0.0007) [2023-12-27 02:30:44,194][105692] Updated weights for policy 0, policy_version 1526467 (0.0007) [2023-12-27 02:30:44,924][105692] Updated weights for policy 0, policy_version 1526477 (0.0008) [2023-12-27 02:30:44,951][105620] Updated weights for policy 1, policy_version 1529234 (0.0007) [2023-12-27 02:30:44,985][105692] Updated weights for policy 0, policy_version 1526487 (0.0007) [2023-12-27 02:30:45,005][105620] Updated weights for policy 1, policy_version 1529244 (0.0006) [2023-12-27 02:30:45,053][105692] Updated weights for policy 0, policy_version 1526497 (0.0007) [2023-12-27 02:30:45,063][105620] Updated weights for policy 1, policy_version 1529254 (0.0009) [2023-12-27 02:30:45,627][105692] Updated weights for policy 0, policy_version 1526507 (0.0006) [2023-12-27 02:30:45,691][105692] Updated weights for policy 0, policy_version 1526517 (0.0007) [2023-12-27 02:30:45,736][105692] Updated weights for policy 0, policy_version 1526527 (0.0005) [2023-12-27 02:30:45,899][105620] Updated weights for policy 1, policy_version 1529264 (0.0006) [2023-12-27 02:30:45,965][105620] Updated weights for policy 1, policy_version 1529274 (0.0005) [2023-12-27 02:30:46,016][105620] Updated weights for policy 1, policy_version 1529284 (0.0005) [2023-12-27 02:30:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 782401536. Throughput: 0: 9519.2, 1: 9751.4. Samples: 782367240. Policy #0 lag: (min: 28.0, avg: 29.8, max: 49.0) [2023-12-27 02:30:46,063][104569] Avg episode reward: [(0, '8356.372'), (1, '8989.190')] [2023-12-27 02:30:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001526536_390848512.pth... [2023-12-27 02:30:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001529288_391553024.pth... [2023-12-27 02:30:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001525416_390561792.pth [2023-12-27 02:30:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001528136_391258112.pth [2023-12-27 02:30:46,521][105692] Updated weights for policy 0, policy_version 1526537 (0.0009) [2023-12-27 02:30:46,588][105692] Updated weights for policy 0, policy_version 1526548 (0.0009) [2023-12-27 02:30:46,638][105692] Updated weights for policy 0, policy_version 1526558 (0.0009) [2023-12-27 02:30:46,665][105620] Updated weights for policy 1, policy_version 1529294 (0.0006) [2023-12-27 02:30:46,687][105692] Updated weights for policy 0, policy_version 1526568 (0.0006) [2023-12-27 02:30:46,713][105620] Updated weights for policy 1, policy_version 1529304 (0.0008) [2023-12-27 02:30:46,759][105620] Updated weights for policy 1, policy_version 1529314 (0.0008) [2023-12-27 02:30:47,323][105692] Updated weights for policy 0, policy_version 1526578 (0.0010) [2023-12-27 02:30:47,381][105692] Updated weights for policy 0, policy_version 1526588 (0.0010) [2023-12-27 02:30:47,432][105692] Updated weights for policy 0, policy_version 1526598 (0.0010) [2023-12-27 02:30:47,599][105620] Updated weights for policy 1, policy_version 1529324 (0.0008) [2023-12-27 02:30:47,661][105620] Updated weights for policy 1, policy_version 1529334 (0.0006) [2023-12-27 02:30:47,721][105620] Updated weights for policy 1, policy_version 1529344 (0.0007) [2023-12-27 02:30:48,190][105692] Updated weights for policy 0, policy_version 1526608 (0.0011) [2023-12-27 02:30:48,246][105692] Updated weights for policy 0, policy_version 1526618 (0.0010) [2023-12-27 02:30:48,305][105692] Updated weights for policy 0, policy_version 1526628 (0.0010) [2023-12-27 02:30:48,418][105620] Updated weights for policy 1, policy_version 1529354 (0.0006) [2023-12-27 02:30:48,488][105620] Updated weights for policy 1, policy_version 1529364 (0.0005) [2023-12-27 02:30:48,547][105620] Updated weights for policy 1, policy_version 1529374 (0.0006) [2023-12-27 02:30:48,608][105620] Updated weights for policy 1, policy_version 1529384 (0.0006) [2023-12-27 02:30:49,059][105692] Updated weights for policy 0, policy_version 1526638 (0.0010) [2023-12-27 02:30:49,103][105692] Updated weights for policy 0, policy_version 1526648 (0.0010) [2023-12-27 02:30:49,154][105692] Updated weights for policy 0, policy_version 1526658 (0.0010) [2023-12-27 02:30:49,173][105620] Updated weights for policy 1, policy_version 1529394 (0.0006) [2023-12-27 02:30:49,232][105620] Updated weights for policy 1, policy_version 1529404 (0.0007) [2023-12-27 02:30:49,283][105620] Updated weights for policy 1, policy_version 1529414 (0.0008) [2023-12-27 02:30:49,914][105692] Updated weights for policy 0, policy_version 1526668 (0.0010) [2023-12-27 02:30:49,978][105692] Updated weights for policy 0, policy_version 1526678 (0.0010) [2023-12-27 02:30:50,025][105620] Updated weights for policy 1, policy_version 1529424 (0.0010) [2023-12-27 02:30:50,034][105692] Updated weights for policy 0, policy_version 1526688 (0.0009) [2023-12-27 02:30:50,088][105620] Updated weights for policy 1, policy_version 1529434 (0.0010) [2023-12-27 02:30:50,157][105620] Updated weights for policy 1, policy_version 1529444 (0.0010) [2023-12-27 02:30:50,740][105692] Updated weights for policy 0, policy_version 1526698 (0.0006) [2023-12-27 02:30:50,810][105692] Updated weights for policy 0, policy_version 1526708 (0.0005) [2023-12-27 02:30:50,875][105692] Updated weights for policy 0, policy_version 1526718 (0.0009) [2023-12-27 02:30:50,921][105620] Updated weights for policy 1, policy_version 1529454 (0.0007) [2023-12-27 02:30:50,936][105692] Updated weights for policy 0, policy_version 1526728 (0.0008) [2023-12-27 02:30:50,969][105620] Updated weights for policy 1, policy_version 1529464 (0.0009) [2023-12-27 02:30:51,017][105620] Updated weights for policy 1, policy_version 1529474 (0.0009) [2023-12-27 02:30:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 782499840. Throughput: 0: 9503.3, 1: 9798.7. Samples: 782483412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:30:51,063][104569] Avg episode reward: [(0, '8900.119'), (1, '8989.640')] [2023-12-27 02:30:51,611][105692] Updated weights for policy 0, policy_version 1526738 (0.0009) [2023-12-27 02:30:51,677][105692] Updated weights for policy 0, policy_version 1526748 (0.0007) [2023-12-27 02:30:51,743][105692] Updated weights for policy 0, policy_version 1526758 (0.0009) [2023-12-27 02:30:51,780][105620] Updated weights for policy 1, policy_version 1529484 (0.0009) [2023-12-27 02:30:51,832][105620] Updated weights for policy 1, policy_version 1529494 (0.0011) [2023-12-27 02:30:51,885][105620] Updated weights for policy 1, policy_version 1529504 (0.0011) [2023-12-27 02:30:52,475][105692] Updated weights for policy 0, policy_version 1526768 (0.0008) [2023-12-27 02:30:52,531][105692] Updated weights for policy 0, policy_version 1526778 (0.0008) [2023-12-27 02:30:52,591][105692] Updated weights for policy 0, policy_version 1526788 (0.0009) [2023-12-27 02:30:52,651][105620] Updated weights for policy 1, policy_version 1529514 (0.0010) [2023-12-27 02:30:52,718][105620] Updated weights for policy 1, policy_version 1529524 (0.0008) [2023-12-27 02:30:52,782][105620] Updated weights for policy 1, policy_version 1529534 (0.0010) [2023-12-27 02:30:52,837][105620] Updated weights for policy 1, policy_version 1529544 (0.0009) [2023-12-27 02:30:53,382][105692] Updated weights for policy 0, policy_version 1526798 (0.0009) [2023-12-27 02:30:53,431][105692] Updated weights for policy 0, policy_version 1526808 (0.0008) [2023-12-27 02:30:53,494][105692] Updated weights for policy 0, policy_version 1526818 (0.0007) [2023-12-27 02:30:53,518][105620] Updated weights for policy 1, policy_version 1529554 (0.0008) [2023-12-27 02:30:53,584][105620] Updated weights for policy 1, policy_version 1529564 (0.0008) [2023-12-27 02:30:53,642][105620] Updated weights for policy 1, policy_version 1529574 (0.0009) [2023-12-27 02:30:54,275][105692] Updated weights for policy 0, policy_version 1526828 (0.0008) [2023-12-27 02:30:54,333][105692] Updated weights for policy 0, policy_version 1526838 (0.0008) [2023-12-27 02:30:54,396][105692] Updated weights for policy 0, policy_version 1526848 (0.0006) [2023-12-27 02:30:54,405][105620] Updated weights for policy 1, policy_version 1529584 (0.0009) [2023-12-27 02:30:54,461][105620] Updated weights for policy 1, policy_version 1529594 (0.0006) [2023-12-27 02:30:54,527][105620] Updated weights for policy 1, policy_version 1529604 (0.0006) [2023-12-27 02:30:55,111][105620] Updated weights for policy 1, policy_version 1529614 (0.0006) [2023-12-27 02:30:55,174][105620] Updated weights for policy 1, policy_version 1529624 (0.0006) [2023-12-27 02:30:55,239][105620] Updated weights for policy 1, policy_version 1529634 (0.0006) [2023-12-27 02:30:55,288][105692] Updated weights for policy 0, policy_version 1526858 (0.0007) [2023-12-27 02:30:55,340][105692] Updated weights for policy 0, policy_version 1526868 (0.0009) [2023-12-27 02:30:55,392][105692] Updated weights for policy 0, policy_version 1526878 (0.0009) [2023-12-27 02:30:55,452][105692] Updated weights for policy 0, policy_version 1526888 (0.0010) [2023-12-27 02:30:55,870][105620] Updated weights for policy 1, policy_version 1529644 (0.0007) [2023-12-27 02:30:55,922][105620] Updated weights for policy 1, policy_version 1529654 (0.0005) [2023-12-27 02:30:55,965][105620] Updated weights for policy 1, policy_version 1529664 (0.0005) [2023-12-27 02:30:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 782589952. Throughput: 0: 9452.9, 1: 9783.8. Samples: 782596640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:30:56,063][104569] Avg episode reward: [(0, '8993.281'), (1, '9171.598')] [2023-12-27 02:30:56,280][105692] Updated weights for policy 0, policy_version 1526898 (0.0010) [2023-12-27 02:30:56,339][105692] Updated weights for policy 0, policy_version 1526908 (0.0009) [2023-12-27 02:30:56,397][105692] Updated weights for policy 0, policy_version 1526918 (0.0009) [2023-12-27 02:30:56,591][105620] Updated weights for policy 1, policy_version 1529674 (0.0005) [2023-12-27 02:30:56,652][105620] Updated weights for policy 1, policy_version 1529684 (0.0005) [2023-12-27 02:30:56,721][105620] Updated weights for policy 1, policy_version 1529694 (0.0005) [2023-12-27 02:30:56,789][105620] Updated weights for policy 1, policy_version 1529704 (0.0005) [2023-12-27 02:30:57,087][105692] Updated weights for policy 0, policy_version 1526928 (0.0009) [2023-12-27 02:30:57,135][105692] Updated weights for policy 0, policy_version 1526938 (0.0008) [2023-12-27 02:30:57,190][105692] Updated weights for policy 0, policy_version 1526948 (0.0008) [2023-12-27 02:30:57,445][105620] Updated weights for policy 1, policy_version 1529714 (0.0011) [2023-12-27 02:30:57,506][105620] Updated weights for policy 1, policy_version 1529724 (0.0009) [2023-12-27 02:30:57,565][105620] Updated weights for policy 1, policy_version 1529734 (0.0009) [2023-12-27 02:30:57,955][105692] Updated weights for policy 0, policy_version 1526958 (0.0007) [2023-12-27 02:30:58,018][105692] Updated weights for policy 0, policy_version 1526968 (0.0008) [2023-12-27 02:30:58,085][105692] Updated weights for policy 0, policy_version 1526978 (0.0008) [2023-12-27 02:30:58,313][105620] Updated weights for policy 1, policy_version 1529744 (0.0010) [2023-12-27 02:30:58,390][105620] Updated weights for policy 1, policy_version 1529754 (0.0009) [2023-12-27 02:30:58,457][105620] Updated weights for policy 1, policy_version 1529764 (0.0010) [2023-12-27 02:30:58,941][105692] Updated weights for policy 0, policy_version 1526988 (0.0007) [2023-12-27 02:30:58,997][105692] Updated weights for policy 0, policy_version 1526998 (0.0008) [2023-12-27 02:30:59,049][105692] Updated weights for policy 0, policy_version 1527008 (0.0008) [2023-12-27 02:30:59,257][105620] Updated weights for policy 1, policy_version 1529774 (0.0009) [2023-12-27 02:30:59,325][105620] Updated weights for policy 1, policy_version 1529784 (0.0008) [2023-12-27 02:30:59,407][105620] Updated weights for policy 1, policy_version 1529794 (0.0008) [2023-12-27 02:31:00,016][105692] Updated weights for policy 0, policy_version 1527018 (0.0008) [2023-12-27 02:31:00,035][105620] Updated weights for policy 1, policy_version 1529804 (0.0007) [2023-12-27 02:31:00,075][105692] Updated weights for policy 0, policy_version 1527028 (0.0007) [2023-12-27 02:31:00,100][105620] Updated weights for policy 1, policy_version 1529814 (0.0007) [2023-12-27 02:31:00,132][105692] Updated weights for policy 0, policy_version 1527038 (0.0007) [2023-12-27 02:31:00,169][105620] Updated weights for policy 1, policy_version 1529824 (0.0008) [2023-12-27 02:31:00,195][105692] Updated weights for policy 0, policy_version 1527048 (0.0010) [2023-12-27 02:31:00,924][105692] Updated weights for policy 0, policy_version 1527058 (0.0011) [2023-12-27 02:31:00,926][105620] Updated weights for policy 1, policy_version 1529834 (0.0008) [2023-12-27 02:31:00,978][105692] Updated weights for policy 0, policy_version 1527068 (0.0010) [2023-12-27 02:31:00,983][105620] Updated weights for policy 1, policy_version 1529844 (0.0008) [2023-12-27 02:31:01,041][105620] Updated weights for policy 1, policy_version 1529854 (0.0006) [2023-12-27 02:31:01,042][105692] Updated weights for policy 0, policy_version 1527078 (0.0010) [2023-12-27 02:31:01,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19114.7, 300 sec: 19494.2). Total num frames: 782680064. Throughput: 0: 9463.6, 1: 9767.9. Samples: 782653628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:31:01,062][104569] Avg episode reward: [(0, '8718.701'), (1, '9353.706')] [2023-12-27 02:31:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001527080_390987776.pth... [2023-12-27 02:31:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001525992_390709248.pth [2023-12-27 02:31:01,095][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001529864_391700480.pth... [2023-12-27 02:31:01,098][105620] Updated weights for policy 1, policy_version 1529864 (0.0009) [2023-12-27 02:31:01,102][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001528712_391405568.pth [2023-12-27 02:31:01,848][105692] Updated weights for policy 0, policy_version 1527088 (0.0011) [2023-12-27 02:31:01,883][105620] Updated weights for policy 1, policy_version 1529874 (0.0007) [2023-12-27 02:31:01,909][105692] Updated weights for policy 0, policy_version 1527098 (0.0008) [2023-12-27 02:31:01,942][105620] Updated weights for policy 1, policy_version 1529884 (0.0009) [2023-12-27 02:31:01,966][105692] Updated weights for policy 0, policy_version 1527108 (0.0006) [2023-12-27 02:31:01,997][105620] Updated weights for policy 1, policy_version 1529894 (0.0008) [2023-12-27 02:31:02,699][105692] Updated weights for policy 0, policy_version 1527118 (0.0009) [2023-12-27 02:31:02,764][105692] Updated weights for policy 0, policy_version 1527128 (0.0010) [2023-12-27 02:31:02,766][105620] Updated weights for policy 1, policy_version 1529904 (0.0007) [2023-12-27 02:31:02,829][105620] Updated weights for policy 1, policy_version 1529914 (0.0009) [2023-12-27 02:31:02,832][105692] Updated weights for policy 0, policy_version 1527138 (0.0010) [2023-12-27 02:31:02,884][105620] Updated weights for policy 1, policy_version 1529924 (0.0008) [2023-12-27 02:31:03,594][105692] Updated weights for policy 0, policy_version 1527148 (0.0011) [2023-12-27 02:31:03,620][105620] Updated weights for policy 1, policy_version 1529934 (0.0006) [2023-12-27 02:31:03,650][105692] Updated weights for policy 0, policy_version 1527158 (0.0011) [2023-12-27 02:31:03,668][105620] Updated weights for policy 1, policy_version 1529945 (0.0006) [2023-12-27 02:31:03,705][105692] Updated weights for policy 0, policy_version 1527168 (0.0010) [2023-12-27 02:31:03,719][105620] Updated weights for policy 1, policy_version 1529955 (0.0010) [2023-12-27 02:31:04,479][105692] Updated weights for policy 0, policy_version 1527178 (0.0011) [2023-12-27 02:31:04,531][105620] Updated weights for policy 1, policy_version 1529965 (0.0006) [2023-12-27 02:31:04,543][105692] Updated weights for policy 0, policy_version 1527188 (0.0009) [2023-12-27 02:31:04,590][105620] Updated weights for policy 1, policy_version 1529975 (0.0006) [2023-12-27 02:31:04,603][105692] Updated weights for policy 0, policy_version 1527198 (0.0010) [2023-12-27 02:31:04,656][105620] Updated weights for policy 1, policy_version 1529985 (0.0006) [2023-12-27 02:31:04,669][105692] Updated weights for policy 0, policy_version 1527208 (0.0011) [2023-12-27 02:31:05,408][105620] Updated weights for policy 1, policy_version 1529995 (0.0008) [2023-12-27 02:31:05,414][105692] Updated weights for policy 0, policy_version 1527218 (0.0011) [2023-12-27 02:31:05,461][105620] Updated weights for policy 1, policy_version 1530005 (0.0006) [2023-12-27 02:31:05,474][105692] Updated weights for policy 0, policy_version 1527228 (0.0011) [2023-12-27 02:31:05,512][105620] Updated weights for policy 1, policy_version 1530015 (0.0008) [2023-12-27 02:31:05,534][105692] Updated weights for policy 0, policy_version 1527238 (0.0011) [2023-12-27 02:31:06,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19114.7, 300 sec: 19494.2). Total num frames: 782770176. Throughput: 0: 9335.2, 1: 9635.8. Samples: 782761952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:31:06,062][104569] Avg episode reward: [(0, '8361.706'), (1, '8993.837')] [2023-12-27 02:31:06,135][105620] Updated weights for policy 1, policy_version 1530025 (0.0007) [2023-12-27 02:31:06,195][105620] Updated weights for policy 1, policy_version 1530035 (0.0011) [2023-12-27 02:31:06,256][105620] Updated weights for policy 1, policy_version 1530045 (0.0011) [2023-12-27 02:31:06,286][105692] Updated weights for policy 0, policy_version 1527248 (0.0011) [2023-12-27 02:31:06,317][105620] Updated weights for policy 1, policy_version 1530055 (0.0011) [2023-12-27 02:31:06,346][105692] Updated weights for policy 0, policy_version 1527258 (0.0011) [2023-12-27 02:31:06,414][105692] Updated weights for policy 0, policy_version 1527268 (0.0011) [2023-12-27 02:31:07,030][105620] Updated weights for policy 1, policy_version 1530065 (0.0006) [2023-12-27 02:31:07,085][105620] Updated weights for policy 1, policy_version 1530075 (0.0011) [2023-12-27 02:31:07,138][105620] Updated weights for policy 1, policy_version 1530085 (0.0011) [2023-12-27 02:31:07,175][105692] Updated weights for policy 0, policy_version 1527278 (0.0011) [2023-12-27 02:31:07,240][105692] Updated weights for policy 0, policy_version 1527288 (0.0011) [2023-12-27 02:31:07,292][105692] Updated weights for policy 0, policy_version 1527298 (0.0011) [2023-12-27 02:31:07,745][105620] Updated weights for policy 1, policy_version 1530095 (0.0007) [2023-12-27 02:31:07,805][105620] Updated weights for policy 1, policy_version 1530105 (0.0006) [2023-12-27 02:31:07,868][105620] Updated weights for policy 1, policy_version 1530115 (0.0007) [2023-12-27 02:31:08,045][105692] Updated weights for policy 0, policy_version 1527308 (0.0010) [2023-12-27 02:31:08,096][105692] Updated weights for policy 0, policy_version 1527318 (0.0010) [2023-12-27 02:31:08,153][105692] Updated weights for policy 0, policy_version 1527328 (0.0010) [2023-12-27 02:31:08,576][105620] Updated weights for policy 1, policy_version 1530125 (0.0008) [2023-12-27 02:31:08,641][105620] Updated weights for policy 1, policy_version 1530135 (0.0011) [2023-12-27 02:31:08,693][105620] Updated weights for policy 1, policy_version 1530145 (0.0010) [2023-12-27 02:31:08,838][105692] Updated weights for policy 0, policy_version 1527338 (0.0009) [2023-12-27 02:31:08,903][105692] Updated weights for policy 0, policy_version 1527348 (0.0007) [2023-12-27 02:31:08,959][105692] Updated weights for policy 0, policy_version 1527358 (0.0007) [2023-12-27 02:31:09,011][105692] Updated weights for policy 0, policy_version 1527368 (0.0011) [2023-12-27 02:31:09,483][105620] Updated weights for policy 1, policy_version 1530155 (0.0009) [2023-12-27 02:31:09,543][105620] Updated weights for policy 1, policy_version 1530165 (0.0008) [2023-12-27 02:31:09,606][105620] Updated weights for policy 1, policy_version 1530175 (0.0008) [2023-12-27 02:31:09,655][105692] Updated weights for policy 0, policy_version 1527378 (0.0006) [2023-12-27 02:31:09,713][105692] Updated weights for policy 0, policy_version 1527388 (0.0007) [2023-12-27 02:31:09,772][105692] Updated weights for policy 0, policy_version 1527398 (0.0006) [2023-12-27 02:31:10,361][105620] Updated weights for policy 1, policy_version 1530185 (0.0008) [2023-12-27 02:31:10,418][105620] Updated weights for policy 1, policy_version 1530195 (0.0006) [2023-12-27 02:31:10,477][105620] Updated weights for policy 1, policy_version 1530205 (0.0006) [2023-12-27 02:31:10,546][105692] Updated weights for policy 0, policy_version 1527408 (0.0007) [2023-12-27 02:31:10,547][105620] Updated weights for policy 1, policy_version 1530215 (0.0009) [2023-12-27 02:31:10,608][105692] Updated weights for policy 0, policy_version 1527418 (0.0008) [2023-12-27 02:31:10,667][105692] Updated weights for policy 0, policy_version 1527428 (0.0009) [2023-12-27 02:31:11,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19114.6, 300 sec: 19494.2). Total num frames: 782868480. Throughput: 0: 9391.6, 1: 9734.7. Samples: 782879116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:31:11,063][104569] Avg episode reward: [(0, '8186.403'), (1, '8903.722')] [2023-12-27 02:31:11,182][105620] Updated weights for policy 1, policy_version 1530225 (0.0010) [2023-12-27 02:31:11,234][105620] Updated weights for policy 1, policy_version 1530235 (0.0010) [2023-12-27 02:31:11,295][105620] Updated weights for policy 1, policy_version 1530245 (0.0011) [2023-12-27 02:31:11,313][105692] Updated weights for policy 0, policy_version 1527438 (0.0007) [2023-12-27 02:31:11,385][105692] Updated weights for policy 0, policy_version 1527448 (0.0008) [2023-12-27 02:31:11,441][105692] Updated weights for policy 0, policy_version 1527458 (0.0008) [2023-12-27 02:31:12,067][105620] Updated weights for policy 1, policy_version 1530255 (0.0010) [2023-12-27 02:31:12,117][105620] Updated weights for policy 1, policy_version 1530265 (0.0011) [2023-12-27 02:31:12,179][105620] Updated weights for policy 1, policy_version 1530275 (0.0011) [2023-12-27 02:31:12,218][105692] Updated weights for policy 0, policy_version 1527468 (0.0008) [2023-12-27 02:31:12,279][105692] Updated weights for policy 0, policy_version 1527478 (0.0009) [2023-12-27 02:31:12,348][105692] Updated weights for policy 0, policy_version 1527488 (0.0009) [2023-12-27 02:31:12,879][105620] Updated weights for policy 1, policy_version 1530285 (0.0009) [2023-12-27 02:31:12,922][105620] Updated weights for policy 1, policy_version 1530295 (0.0005) [2023-12-27 02:31:12,975][105620] Updated weights for policy 1, policy_version 1530305 (0.0005) [2023-12-27 02:31:13,175][105692] Updated weights for policy 0, policy_version 1527498 (0.0008) [2023-12-27 02:31:13,231][105692] Updated weights for policy 0, policy_version 1527508 (0.0008) [2023-12-27 02:31:13,276][105692] Updated weights for policy 0, policy_version 1527518 (0.0008) [2023-12-27 02:31:13,332][105692] Updated weights for policy 0, policy_version 1527528 (0.0008) [2023-12-27 02:31:13,665][105620] Updated weights for policy 1, policy_version 1530315 (0.0010) [2023-12-27 02:31:13,720][105620] Updated weights for policy 1, policy_version 1530325 (0.0009) [2023-12-27 02:31:13,776][105620] Updated weights for policy 1, policy_version 1530335 (0.0005) [2023-12-27 02:31:14,003][105692] Updated weights for policy 0, policy_version 1527538 (0.0005) [2023-12-27 02:31:14,067][105692] Updated weights for policy 0, policy_version 1527548 (0.0008) [2023-12-27 02:31:14,136][105692] Updated weights for policy 0, policy_version 1527558 (0.0006) [2023-12-27 02:31:14,402][105620] Updated weights for policy 1, policy_version 1530345 (0.0006) [2023-12-27 02:31:14,452][105620] Updated weights for policy 1, policy_version 1530355 (0.0010) [2023-12-27 02:31:14,507][105620] Updated weights for policy 1, policy_version 1530365 (0.0010) [2023-12-27 02:31:14,575][105620] Updated weights for policy 1, policy_version 1530375 (0.0010) [2023-12-27 02:31:14,720][105692] Updated weights for policy 0, policy_version 1527568 (0.0006) [2023-12-27 02:31:14,779][105692] Updated weights for policy 0, policy_version 1527578 (0.0009) [2023-12-27 02:31:14,841][105692] Updated weights for policy 0, policy_version 1527588 (0.0008) [2023-12-27 02:31:15,240][105620] Updated weights for policy 1, policy_version 1530385 (0.0010) [2023-12-27 02:31:15,286][105620] Updated weights for policy 1, policy_version 1530395 (0.0010) [2023-12-27 02:31:15,341][105620] Updated weights for policy 1, policy_version 1530405 (0.0008) [2023-12-27 02:31:15,613][105692] Updated weights for policy 0, policy_version 1527598 (0.0009) [2023-12-27 02:31:15,678][105692] Updated weights for policy 0, policy_version 1527608 (0.0009) [2023-12-27 02:31:15,740][105692] Updated weights for policy 0, policy_version 1527618 (0.0008) [2023-12-27 02:31:16,047][105620] Updated weights for policy 1, policy_version 1530415 (0.0008) [2023-12-27 02:31:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19114.8, 300 sec: 19494.2). Total num frames: 782966784. Throughput: 0: 9344.4, 1: 9741.2. Samples: 782936304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:31:16,063][104569] Avg episode reward: [(0, '8539.401'), (1, '8990.247')] [2023-12-27 02:31:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001527624_391127040.pth... [2023-12-27 02:31:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001526536_390848512.pth [2023-12-27 02:31:16,105][105620] Updated weights for policy 1, policy_version 1530425 (0.0009) [2023-12-27 02:31:16,169][105620] Updated weights for policy 1, policy_version 1530435 (0.0009) [2023-12-27 02:31:16,197][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001530440_391847936.pth... [2023-12-27 02:31:16,201][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001529288_391553024.pth [2023-12-27 02:31:16,420][105692] Updated weights for policy 0, policy_version 1527628 (0.0007) [2023-12-27 02:31:16,481][105692] Updated weights for policy 0, policy_version 1527638 (0.0008) [2023-12-27 02:31:16,545][105692] Updated weights for policy 0, policy_version 1527648 (0.0008) [2023-12-27 02:31:16,912][105620] Updated weights for policy 1, policy_version 1530445 (0.0010) [2023-12-27 02:31:16,973][105620] Updated weights for policy 1, policy_version 1530455 (0.0010) [2023-12-27 02:31:17,034][105620] Updated weights for policy 1, policy_version 1530465 (0.0010) [2023-12-27 02:31:17,184][105692] Updated weights for policy 0, policy_version 1527658 (0.0007) [2023-12-27 02:31:17,248][105692] Updated weights for policy 0, policy_version 1527668 (0.0005) [2023-12-27 02:31:17,297][105692] Updated weights for policy 0, policy_version 1527678 (0.0005) [2023-12-27 02:31:17,353][105692] Updated weights for policy 0, policy_version 1527688 (0.0008) [2023-12-27 02:31:17,774][105620] Updated weights for policy 1, policy_version 1530475 (0.0009) [2023-12-27 02:31:17,826][105620] Updated weights for policy 1, policy_version 1530485 (0.0006) [2023-12-27 02:31:17,888][105620] Updated weights for policy 1, policy_version 1530495 (0.0005) [2023-12-27 02:31:17,954][105692] Updated weights for policy 0, policy_version 1527698 (0.0006) [2023-12-27 02:31:18,018][105692] Updated weights for policy 0, policy_version 1527708 (0.0005) [2023-12-27 02:31:18,082][105692] Updated weights for policy 0, policy_version 1527718 (0.0008) [2023-12-27 02:31:18,551][105620] Updated weights for policy 1, policy_version 1530505 (0.0005) [2023-12-27 02:31:18,612][105620] Updated weights for policy 1, policy_version 1530515 (0.0009) [2023-12-27 02:31:18,671][105620] Updated weights for policy 1, policy_version 1530525 (0.0011) [2023-12-27 02:31:18,737][105620] Updated weights for policy 1, policy_version 1530535 (0.0010) [2023-12-27 02:31:18,785][105692] Updated weights for policy 0, policy_version 1527728 (0.0011) [2023-12-27 02:31:18,851][105692] Updated weights for policy 0, policy_version 1527738 (0.0011) [2023-12-27 02:31:18,908][105692] Updated weights for policy 0, policy_version 1527748 (0.0010) [2023-12-27 02:31:19,493][105620] Updated weights for policy 1, policy_version 1530545 (0.0011) [2023-12-27 02:31:19,549][105620] Updated weights for policy 1, policy_version 1530555 (0.0011) [2023-12-27 02:31:19,612][105620] Updated weights for policy 1, policy_version 1530565 (0.0011) [2023-12-27 02:31:19,672][105692] Updated weights for policy 0, policy_version 1527758 (0.0008) [2023-12-27 02:31:19,721][105692] Updated weights for policy 0, policy_version 1527768 (0.0008) [2023-12-27 02:31:19,781][105692] Updated weights for policy 0, policy_version 1527778 (0.0008) [2023-12-27 02:31:20,386][105620] Updated weights for policy 1, policy_version 1530575 (0.0011) [2023-12-27 02:31:20,439][105620] Updated weights for policy 1, policy_version 1530585 (0.0009) [2023-12-27 02:31:20,496][105692] Updated weights for policy 0, policy_version 1527788 (0.0006) [2023-12-27 02:31:20,507][105620] Updated weights for policy 1, policy_version 1530595 (0.0008) [2023-12-27 02:31:20,556][105692] Updated weights for policy 0, policy_version 1527798 (0.0005) [2023-12-27 02:31:20,623][105692] Updated weights for policy 0, policy_version 1527808 (0.0008) [2023-12-27 02:31:21,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19114.7, 300 sec: 19494.2). Total num frames: 783065088. Throughput: 0: 9417.0, 1: 9726.6. Samples: 783056020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:31:21,062][104569] Avg episode reward: [(0, '8537.249'), (1, '8805.871')] [2023-12-27 02:31:21,209][105620] Updated weights for policy 1, policy_version 1530605 (0.0009) [2023-12-27 02:31:21,276][105620] Updated weights for policy 1, policy_version 1530615 (0.0007) [2023-12-27 02:31:21,340][105620] Updated weights for policy 1, policy_version 1530625 (0.0008) [2023-12-27 02:31:21,447][105692] Updated weights for policy 0, policy_version 1527818 (0.0008) [2023-12-27 02:31:21,510][105692] Updated weights for policy 0, policy_version 1527828 (0.0010) [2023-12-27 02:31:21,575][105692] Updated weights for policy 0, policy_version 1527838 (0.0010) [2023-12-27 02:31:21,630][105692] Updated weights for policy 0, policy_version 1527848 (0.0009) [2023-12-27 02:31:22,013][105620] Updated weights for policy 1, policy_version 1530635 (0.0009) [2023-12-27 02:31:22,060][105620] Updated weights for policy 1, policy_version 1530645 (0.0009) [2023-12-27 02:31:22,114][105620] Updated weights for policy 1, policy_version 1530655 (0.0008) [2023-12-27 02:31:22,465][105692] Updated weights for policy 0, policy_version 1527858 (0.0009) [2023-12-27 02:31:22,529][105692] Updated weights for policy 0, policy_version 1527868 (0.0009) [2023-12-27 02:31:22,585][105692] Updated weights for policy 0, policy_version 1527878 (0.0010) [2023-12-27 02:31:22,821][105620] Updated weights for policy 1, policy_version 1530665 (0.0010) [2023-12-27 02:31:22,882][105620] Updated weights for policy 1, policy_version 1530675 (0.0008) [2023-12-27 02:31:22,945][105620] Updated weights for policy 1, policy_version 1530685 (0.0009) [2023-12-27 02:31:23,003][105620] Updated weights for policy 1, policy_version 1530695 (0.0009) [2023-12-27 02:31:23,425][105692] Updated weights for policy 0, policy_version 1527889 (0.0011) [2023-12-27 02:31:23,481][105692] Updated weights for policy 0, policy_version 1527900 (0.0009) [2023-12-27 02:31:23,545][105692] Updated weights for policy 0, policy_version 1527910 (0.0005) [2023-12-27 02:31:23,632][105620] Updated weights for policy 1, policy_version 1530705 (0.0010) [2023-12-27 02:31:23,683][105620] Updated weights for policy 1, policy_version 1530715 (0.0010) [2023-12-27 02:31:23,736][105620] Updated weights for policy 1, policy_version 1530725 (0.0009) [2023-12-27 02:31:24,187][105692] Updated weights for policy 0, policy_version 1527920 (0.0009) [2023-12-27 02:31:24,242][105692] Updated weights for policy 0, policy_version 1527930 (0.0010) [2023-12-27 02:31:24,309][105692] Updated weights for policy 0, policy_version 1527940 (0.0010) [2023-12-27 02:31:24,462][105620] Updated weights for policy 1, policy_version 1530735 (0.0011) [2023-12-27 02:31:24,529][105620] Updated weights for policy 1, policy_version 1530745 (0.0010) [2023-12-27 02:31:24,588][105620] Updated weights for policy 1, policy_version 1530755 (0.0011) [2023-12-27 02:31:24,906][105692] Updated weights for policy 0, policy_version 1527950 (0.0009) [2023-12-27 02:31:24,953][105692] Updated weights for policy 0, policy_version 1527960 (0.0010) [2023-12-27 02:31:24,998][105692] Updated weights for policy 0, policy_version 1527970 (0.0010) [2023-12-27 02:31:25,274][105620] Updated weights for policy 1, policy_version 1530765 (0.0011) [2023-12-27 02:31:25,337][105620] Updated weights for policy 1, policy_version 1530775 (0.0011) [2023-12-27 02:31:25,400][105620] Updated weights for policy 1, policy_version 1530785 (0.0006) [2023-12-27 02:31:25,687][105692] Updated weights for policy 0, policy_version 1527980 (0.0009) [2023-12-27 02:31:25,751][105692] Updated weights for policy 0, policy_version 1527990 (0.0006) [2023-12-27 02:31:25,809][105692] Updated weights for policy 0, policy_version 1528000 (0.0005) [2023-12-27 02:31:26,048][105620] Updated weights for policy 1, policy_version 1530795 (0.0006) [2023-12-27 02:31:26,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 783163392. Throughput: 0: 9421.0, 1: 9788.9. Samples: 783172448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:31:26,063][104569] Avg episode reward: [(0, '8177.625'), (1, '8899.374')] [2023-12-27 02:31:26,107][105620] Updated weights for policy 1, policy_version 1530805 (0.0005) [2023-12-27 02:31:26,161][105620] Updated weights for policy 1, policy_version 1530815 (0.0005) [2023-12-27 02:31:26,387][105692] Updated weights for policy 0, policy_version 1528010 (0.0005) [2023-12-27 02:31:26,441][105692] Updated weights for policy 0, policy_version 1528020 (0.0005) [2023-12-27 02:31:26,497][105692] Updated weights for policy 0, policy_version 1528030 (0.0005) [2023-12-27 02:31:26,554][105692] Updated weights for policy 0, policy_version 1528040 (0.0006) [2023-12-27 02:31:26,813][105620] Updated weights for policy 1, policy_version 1530825 (0.0007) [2023-12-27 02:31:26,859][105620] Updated weights for policy 1, policy_version 1530835 (0.0005) [2023-12-27 02:31:26,906][105620] Updated weights for policy 1, policy_version 1530845 (0.0005) [2023-12-27 02:31:26,954][105620] Updated weights for policy 1, policy_version 1530855 (0.0005) [2023-12-27 02:31:27,182][105692] Updated weights for policy 0, policy_version 1528050 (0.0009) [2023-12-27 02:31:27,239][105692] Updated weights for policy 0, policy_version 1528060 (0.0010) [2023-12-27 02:31:27,291][105692] Updated weights for policy 0, policy_version 1528070 (0.0009) [2023-12-27 02:31:27,500][105620] Updated weights for policy 1, policy_version 1530865 (0.0005) [2023-12-27 02:31:27,554][105620] Updated weights for policy 1, policy_version 1530875 (0.0005) [2023-12-27 02:31:27,614][105620] Updated weights for policy 1, policy_version 1530885 (0.0008) [2023-12-27 02:31:27,861][105692] Updated weights for policy 0, policy_version 1528080 (0.0005) [2023-12-27 02:31:27,919][105692] Updated weights for policy 0, policy_version 1528090 (0.0005) [2023-12-27 02:31:27,967][105692] Updated weights for policy 0, policy_version 1528100 (0.0005) [2023-12-27 02:31:28,246][105620] Updated weights for policy 1, policy_version 1530895 (0.0009) [2023-12-27 02:31:28,299][105620] Updated weights for policy 1, policy_version 1530905 (0.0009) [2023-12-27 02:31:28,365][105620] Updated weights for policy 1, policy_version 1530915 (0.0008) [2023-12-27 02:31:28,525][105692] Updated weights for policy 0, policy_version 1528110 (0.0005) [2023-12-27 02:31:28,594][105692] Updated weights for policy 0, policy_version 1528120 (0.0009) [2023-12-27 02:31:28,652][105692] Updated weights for policy 0, policy_version 1528130 (0.0009) [2023-12-27 02:31:28,996][105620] Updated weights for policy 1, policy_version 1530925 (0.0005) [2023-12-27 02:31:29,052][105620] Updated weights for policy 1, policy_version 1530935 (0.0005) [2023-12-27 02:31:29,107][105620] Updated weights for policy 1, policy_version 1530945 (0.0005) [2023-12-27 02:31:29,451][105692] Updated weights for policy 0, policy_version 1528140 (0.0008) [2023-12-27 02:31:29,517][105692] Updated weights for policy 0, policy_version 1528150 (0.0005) [2023-12-27 02:31:29,592][105692] Updated weights for policy 0, policy_version 1528160 (0.0008) [2023-12-27 02:31:29,658][105620] Updated weights for policy 1, policy_version 1530955 (0.0006) [2023-12-27 02:31:29,718][105620] Updated weights for policy 1, policy_version 1530965 (0.0007) [2023-12-27 02:31:29,775][105620] Updated weights for policy 1, policy_version 1530975 (0.0008) [2023-12-27 02:31:30,247][105692] Updated weights for policy 0, policy_version 1528170 (0.0008) [2023-12-27 02:31:30,302][105692] Updated weights for policy 0, policy_version 1528180 (0.0005) [2023-12-27 02:31:30,360][105692] Updated weights for policy 0, policy_version 1528190 (0.0008) [2023-12-27 02:31:30,407][105620] Updated weights for policy 1, policy_version 1530985 (0.0007) [2023-12-27 02:31:30,421][105692] Updated weights for policy 0, policy_version 1528200 (0.0007) [2023-12-27 02:31:30,465][105620] Updated weights for policy 1, policy_version 1530995 (0.0010) [2023-12-27 02:31:30,516][105620] Updated weights for policy 1, policy_version 1531005 (0.0010) [2023-12-27 02:31:30,575][105620] Updated weights for policy 1, policy_version 1531015 (0.0010) [2023-12-27 02:31:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 783269888. Throughput: 0: 9584.2, 1: 9830.8. Samples: 783240916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:31:31,062][104569] Avg episode reward: [(0, '8630.313'), (1, '8996.394')] [2023-12-27 02:31:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001528200_391274496.pth... [2023-12-27 02:31:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001531016_391995392.pth... [2023-12-27 02:31:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001529864_391700480.pth [2023-12-27 02:31:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001527080_390987776.pth [2023-12-27 02:31:31,161][105692] Updated weights for policy 0, policy_version 1528210 (0.0009) [2023-12-27 02:31:31,225][105692] Updated weights for policy 0, policy_version 1528220 (0.0006) [2023-12-27 02:31:31,286][105692] Updated weights for policy 0, policy_version 1528230 (0.0009) [2023-12-27 02:31:31,294][105620] Updated weights for policy 1, policy_version 1531025 (0.0007) [2023-12-27 02:31:31,354][105620] Updated weights for policy 1, policy_version 1531035 (0.0006) [2023-12-27 02:31:31,418][105620] Updated weights for policy 1, policy_version 1531045 (0.0009) [2023-12-27 02:31:32,041][105620] Updated weights for policy 1, policy_version 1531055 (0.0007) [2023-12-27 02:31:32,088][105692] Updated weights for policy 0, policy_version 1528240 (0.0008) [2023-12-27 02:31:32,094][105620] Updated weights for policy 1, policy_version 1531065 (0.0007) [2023-12-27 02:31:32,134][105692] Updated weights for policy 0, policy_version 1528250 (0.0007) [2023-12-27 02:31:32,152][105620] Updated weights for policy 1, policy_version 1531075 (0.0008) [2023-12-27 02:31:32,177][105692] Updated weights for policy 0, policy_version 1528260 (0.0008) [2023-12-27 02:31:32,807][105620] Updated weights for policy 1, policy_version 1531085 (0.0007) [2023-12-27 02:31:32,867][105620] Updated weights for policy 1, policy_version 1531095 (0.0006) [2023-12-27 02:31:32,931][105620] Updated weights for policy 1, policy_version 1531105 (0.0009) [2023-12-27 02:31:33,009][105692] Updated weights for policy 0, policy_version 1528270 (0.0009) [2023-12-27 02:31:33,056][105692] Updated weights for policy 0, policy_version 1528280 (0.0008) [2023-12-27 02:31:33,105][105692] Updated weights for policy 0, policy_version 1528290 (0.0009) [2023-12-27 02:31:33,609][105620] Updated weights for policy 1, policy_version 1531115 (0.0009) [2023-12-27 02:31:33,671][105620] Updated weights for policy 1, policy_version 1531125 (0.0010) [2023-12-27 02:31:33,731][105620] Updated weights for policy 1, policy_version 1531135 (0.0010) [2023-12-27 02:31:33,878][105692] Updated weights for policy 0, policy_version 1528300 (0.0009) [2023-12-27 02:31:33,929][105692] Updated weights for policy 0, policy_version 1528310 (0.0009) [2023-12-27 02:31:33,980][105692] Updated weights for policy 0, policy_version 1528320 (0.0008) [2023-12-27 02:31:34,452][105620] Updated weights for policy 1, policy_version 1531145 (0.0008) [2023-12-27 02:31:34,504][105620] Updated weights for policy 1, policy_version 1531155 (0.0010) [2023-12-27 02:31:34,579][105620] Updated weights for policy 1, policy_version 1531165 (0.0007) [2023-12-27 02:31:34,630][105620] Updated weights for policy 1, policy_version 1531175 (0.0008) [2023-12-27 02:31:34,750][105692] Updated weights for policy 0, policy_version 1528330 (0.0009) [2023-12-27 02:31:34,813][105692] Updated weights for policy 0, policy_version 1528340 (0.0009) [2023-12-27 02:31:34,880][105692] Updated weights for policy 0, policy_version 1528350 (0.0008) [2023-12-27 02:31:34,946][105692] Updated weights for policy 0, policy_version 1528360 (0.0011) [2023-12-27 02:31:35,374][105620] Updated weights for policy 1, policy_version 1531185 (0.0010) [2023-12-27 02:31:35,422][105620] Updated weights for policy 1, policy_version 1531195 (0.0010) [2023-12-27 02:31:35,480][105620] Updated weights for policy 1, policy_version 1531205 (0.0010) [2023-12-27 02:31:35,659][105692] Updated weights for policy 0, policy_version 1528370 (0.0008) [2023-12-27 02:31:35,714][105692] Updated weights for policy 0, policy_version 1528380 (0.0011) [2023-12-27 02:31:35,758][105692] Updated weights for policy 0, policy_version 1528390 (0.0010) [2023-12-27 02:31:36,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 783368192. Throughput: 0: 9502.7, 1: 9931.3. Samples: 783357940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:31:36,063][104569] Avg episode reward: [(0, '9174.066'), (1, '8727.131')] [2023-12-27 02:31:36,118][105620] Updated weights for policy 1, policy_version 1531215 (0.0008) [2023-12-27 02:31:36,178][105620] Updated weights for policy 1, policy_version 1531225 (0.0009) [2023-12-27 02:31:36,229][105620] Updated weights for policy 1, policy_version 1531235 (0.0010) [2023-12-27 02:31:36,545][105692] Updated weights for policy 0, policy_version 1528400 (0.0011) [2023-12-27 02:31:36,594][105692] Updated weights for policy 0, policy_version 1528410 (0.0011) [2023-12-27 02:31:36,647][105692] Updated weights for policy 0, policy_version 1528420 (0.0011) [2023-12-27 02:31:36,794][105620] Updated weights for policy 1, policy_version 1531245 (0.0006) [2023-12-27 02:31:36,854][105620] Updated weights for policy 1, policy_version 1531255 (0.0006) [2023-12-27 02:31:36,915][105620] Updated weights for policy 1, policy_version 1531265 (0.0006) [2023-12-27 02:31:37,354][105692] Updated weights for policy 0, policy_version 1528430 (0.0008) [2023-12-27 02:31:37,412][105692] Updated weights for policy 0, policy_version 1528440 (0.0010) [2023-12-27 02:31:37,477][105692] Updated weights for policy 0, policy_version 1528450 (0.0010) [2023-12-27 02:31:37,571][105620] Updated weights for policy 1, policy_version 1531275 (0.0007) [2023-12-27 02:31:37,629][105620] Updated weights for policy 1, policy_version 1531285 (0.0006) [2023-12-27 02:31:37,685][105620] Updated weights for policy 1, policy_version 1531295 (0.0008) [2023-12-27 02:31:38,167][105692] Updated weights for policy 0, policy_version 1528460 (0.0010) [2023-12-27 02:31:38,215][105692] Updated weights for policy 0, policy_version 1528470 (0.0010) [2023-12-27 02:31:38,262][105692] Updated weights for policy 0, policy_version 1528480 (0.0010) [2023-12-27 02:31:38,370][105620] Updated weights for policy 1, policy_version 1531305 (0.0008) [2023-12-27 02:31:38,435][105620] Updated weights for policy 1, policy_version 1531315 (0.0008) [2023-12-27 02:31:38,499][105620] Updated weights for policy 1, policy_version 1531325 (0.0008) [2023-12-27 02:31:38,565][105620] Updated weights for policy 1, policy_version 1531335 (0.0008) [2023-12-27 02:31:39,049][105692] Updated weights for policy 0, policy_version 1528490 (0.0010) [2023-12-27 02:31:39,104][105692] Updated weights for policy 0, policy_version 1528500 (0.0010) [2023-12-27 02:31:39,169][105692] Updated weights for policy 0, policy_version 1528510 (0.0010) [2023-12-27 02:31:39,215][105620] Updated weights for policy 1, policy_version 1531345 (0.0007) [2023-12-27 02:31:39,229][105692] Updated weights for policy 0, policy_version 1528520 (0.0011) [2023-12-27 02:31:39,286][105620] Updated weights for policy 1, policy_version 1531355 (0.0008) [2023-12-27 02:31:39,351][105620] Updated weights for policy 1, policy_version 1531365 (0.0008) [2023-12-27 02:31:40,031][105692] Updated weights for policy 0, policy_version 1528530 (0.0009) [2023-12-27 02:31:40,081][105692] Updated weights for policy 0, policy_version 1528540 (0.0009) [2023-12-27 02:31:40,104][105620] Updated weights for policy 1, policy_version 1531375 (0.0006) [2023-12-27 02:31:40,140][105692] Updated weights for policy 0, policy_version 1528550 (0.0009) [2023-12-27 02:31:40,165][105620] Updated weights for policy 1, policy_version 1531385 (0.0007) [2023-12-27 02:31:40,223][105620] Updated weights for policy 1, policy_version 1531395 (0.0008) [2023-12-27 02:31:40,857][105692] Updated weights for policy 0, policy_version 1528560 (0.0007) [2023-12-27 02:31:40,913][105692] Updated weights for policy 0, policy_version 1528570 (0.0009) [2023-12-27 02:31:40,959][105692] Updated weights for policy 0, policy_version 1528580 (0.0009) [2023-12-27 02:31:40,974][105620] Updated weights for policy 1, policy_version 1531405 (0.0008) [2023-12-27 02:31:41,034][105620] Updated weights for policy 1, policy_version 1531415 (0.0007) [2023-12-27 02:31:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 783466496. Throughput: 0: 9551.0, 1: 9975.5. Samples: 783475336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:31:41,063][104569] Avg episode reward: [(0, '9173.938'), (1, '8542.682')] [2023-12-27 02:31:41,100][105620] Updated weights for policy 1, policy_version 1531425 (0.0008) [2023-12-27 02:31:41,782][105692] Updated weights for policy 0, policy_version 1528590 (0.0009) [2023-12-27 02:31:41,838][105692] Updated weights for policy 0, policy_version 1528600 (0.0009) [2023-12-27 02:31:41,896][105692] Updated weights for policy 0, policy_version 1528610 (0.0009) [2023-12-27 02:31:41,897][105620] Updated weights for policy 1, policy_version 1531435 (0.0007) [2023-12-27 02:31:41,960][105620] Updated weights for policy 1, policy_version 1531445 (0.0007) [2023-12-27 02:31:42,015][105620] Updated weights for policy 1, policy_version 1531455 (0.0009) [2023-12-27 02:31:42,675][105692] Updated weights for policy 0, policy_version 1528620 (0.0008) [2023-12-27 02:31:42,740][105692] Updated weights for policy 0, policy_version 1528630 (0.0008) [2023-12-27 02:31:42,778][105620] Updated weights for policy 1, policy_version 1531465 (0.0009) [2023-12-27 02:31:42,802][105692] Updated weights for policy 0, policy_version 1528640 (0.0009) [2023-12-27 02:31:42,838][105620] Updated weights for policy 1, policy_version 1531475 (0.0008) [2023-12-27 02:31:42,889][105620] Updated weights for policy 1, policy_version 1531485 (0.0007) [2023-12-27 02:31:42,942][105620] Updated weights for policy 1, policy_version 1531495 (0.0005) [2023-12-27 02:31:43,526][105620] Updated weights for policy 1, policy_version 1531505 (0.0006) [2023-12-27 02:31:43,592][105620] Updated weights for policy 1, policy_version 1531515 (0.0005) [2023-12-27 02:31:43,653][105620] Updated weights for policy 1, policy_version 1531525 (0.0005) [2023-12-27 02:31:43,654][105692] Updated weights for policy 0, policy_version 1528650 (0.0009) [2023-12-27 02:31:43,706][105692] Updated weights for policy 0, policy_version 1528661 (0.0009) [2023-12-27 02:31:43,761][105692] Updated weights for policy 0, policy_version 1528673 (0.0010) [2023-12-27 02:31:44,274][105620] Updated weights for policy 1, policy_version 1531535 (0.0007) [2023-12-27 02:31:44,331][105620] Updated weights for policy 1, policy_version 1531545 (0.0009) [2023-12-27 02:31:44,394][105620] Updated weights for policy 1, policy_version 1531555 (0.0009) [2023-12-27 02:31:44,505][105692] Updated weights for policy 0, policy_version 1528684 (0.0009) [2023-12-27 02:31:44,554][105692] Updated weights for policy 0, policy_version 1528694 (0.0009) [2023-12-27 02:31:44,609][105692] Updated weights for policy 0, policy_version 1528704 (0.0009) [2023-12-27 02:31:45,113][105620] Updated weights for policy 1, policy_version 1531565 (0.0009) [2023-12-27 02:31:45,163][105620] Updated weights for policy 1, policy_version 1531575 (0.0008) [2023-12-27 02:31:45,211][105620] Updated weights for policy 1, policy_version 1531585 (0.0008) [2023-12-27 02:31:45,435][105692] Updated weights for policy 0, policy_version 1528714 (0.0009) [2023-12-27 02:31:45,487][105692] Updated weights for policy 0, policy_version 1528724 (0.0010) [2023-12-27 02:31:45,550][105692] Updated weights for policy 0, policy_version 1528734 (0.0011) [2023-12-27 02:31:45,606][105692] Updated weights for policy 0, policy_version 1528744 (0.0011) [2023-12-27 02:31:45,948][105620] Updated weights for policy 1, policy_version 1531595 (0.0008) [2023-12-27 02:31:46,007][105620] Updated weights for policy 1, policy_version 1531606 (0.0011) [2023-12-27 02:31:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 783556608. Throughput: 0: 9523.3, 1: 9994.3. Samples: 783531920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:31:46,062][104569] Avg episode reward: [(0, '8898.075'), (1, '8809.254')] [2023-12-27 02:31:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001528744_391413760.pth... [2023-12-27 02:31:46,069][105620] Updated weights for policy 1, policy_version 1531617 (0.0010) [2023-12-27 02:31:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001527624_391127040.pth [2023-12-27 02:31:46,103][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001531624_392151040.pth... [2023-12-27 02:31:46,108][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001530440_391847936.pth [2023-12-27 02:31:46,303][105692] Updated weights for policy 0, policy_version 1528754 (0.0009) [2023-12-27 02:31:46,362][105692] Updated weights for policy 0, policy_version 1528764 (0.0009) [2023-12-27 02:31:46,416][105692] Updated weights for policy 0, policy_version 1528774 (0.0008) [2023-12-27 02:31:46,766][105620] Updated weights for policy 1, policy_version 1531627 (0.0009) [2023-12-27 02:31:46,821][105620] Updated weights for policy 1, policy_version 1531637 (0.0009) [2023-12-27 02:31:46,868][105620] Updated weights for policy 1, policy_version 1531647 (0.0009) [2023-12-27 02:31:47,256][105692] Updated weights for policy 0, policy_version 1528784 (0.0009) [2023-12-27 02:31:47,316][105692] Updated weights for policy 0, policy_version 1528794 (0.0009) [2023-12-27 02:31:47,379][105692] Updated weights for policy 0, policy_version 1528804 (0.0009) [2023-12-27 02:31:47,469][105620] Updated weights for policy 1, policy_version 1531657 (0.0008) [2023-12-27 02:31:47,519][105620] Updated weights for policy 1, policy_version 1531667 (0.0005) [2023-12-27 02:31:47,574][105620] Updated weights for policy 1, policy_version 1531677 (0.0005) [2023-12-27 02:31:47,630][105620] Updated weights for policy 1, policy_version 1531687 (0.0005) [2023-12-27 02:31:48,107][105692] Updated weights for policy 0, policy_version 1528814 (0.0007) [2023-12-27 02:31:48,154][105692] Updated weights for policy 0, policy_version 1528824 (0.0009) [2023-12-27 02:31:48,209][105692] Updated weights for policy 0, policy_version 1528834 (0.0009) [2023-12-27 02:31:48,341][105620] Updated weights for policy 1, policy_version 1531697 (0.0008) [2023-12-27 02:31:48,406][105620] Updated weights for policy 1, policy_version 1531707 (0.0009) [2023-12-27 02:31:48,470][105620] Updated weights for policy 1, policy_version 1531717 (0.0009) [2023-12-27 02:31:48,812][105692] Updated weights for policy 0, policy_version 1528844 (0.0007) [2023-12-27 02:31:48,882][105692] Updated weights for policy 0, policy_version 1528854 (0.0005) [2023-12-27 02:31:48,943][105692] Updated weights for policy 0, policy_version 1528864 (0.0006) [2023-12-27 02:31:49,351][105620] Updated weights for policy 1, policy_version 1531727 (0.0010) [2023-12-27 02:31:49,409][105620] Updated weights for policy 1, policy_version 1531737 (0.0009) [2023-12-27 02:31:49,461][105620] Updated weights for policy 1, policy_version 1531747 (0.0008) [2023-12-27 02:31:49,590][105692] Updated weights for policy 0, policy_version 1528874 (0.0005) [2023-12-27 02:31:49,655][105692] Updated weights for policy 0, policy_version 1528884 (0.0005) [2023-12-27 02:31:49,712][105692] Updated weights for policy 0, policy_version 1528894 (0.0005) [2023-12-27 02:31:49,772][105692] Updated weights for policy 0, policy_version 1528904 (0.0005) [2023-12-27 02:31:50,264][105620] Updated weights for policy 1, policy_version 1531757 (0.0009) [2023-12-27 02:31:50,330][105620] Updated weights for policy 1, policy_version 1531767 (0.0009) [2023-12-27 02:31:50,388][105620] Updated weights for policy 1, policy_version 1531777 (0.0009) [2023-12-27 02:31:50,440][105692] Updated weights for policy 0, policy_version 1528914 (0.0007) [2023-12-27 02:31:50,503][105692] Updated weights for policy 0, policy_version 1528924 (0.0009) [2023-12-27 02:31:50,550][105692] Updated weights for policy 0, policy_version 1528934 (0.0009) [2023-12-27 02:31:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 783654912. Throughput: 0: 9635.7, 1: 10040.7. Samples: 783647392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:31:51,062][104569] Avg episode reward: [(0, '8532.117'), (1, '8811.481')] [2023-12-27 02:31:51,083][105620] Updated weights for policy 1, policy_version 1531787 (0.0007) [2023-12-27 02:31:51,145][105620] Updated weights for policy 1, policy_version 1531797 (0.0008) [2023-12-27 02:31:51,208][105620] Updated weights for policy 1, policy_version 1531807 (0.0008) [2023-12-27 02:31:51,241][105692] Updated weights for policy 0, policy_version 1528944 (0.0008) [2023-12-27 02:31:51,303][105692] Updated weights for policy 0, policy_version 1528954 (0.0010) [2023-12-27 02:31:51,369][105692] Updated weights for policy 0, policy_version 1528964 (0.0011) [2023-12-27 02:31:51,948][105620] Updated weights for policy 1, policy_version 1531817 (0.0008) [2023-12-27 02:31:51,996][105620] Updated weights for policy 1, policy_version 1531827 (0.0009) [2023-12-27 02:31:52,057][105620] Updated weights for policy 1, policy_version 1531837 (0.0008) [2023-12-27 02:31:52,071][105692] Updated weights for policy 0, policy_version 1528974 (0.0008) [2023-12-27 02:31:52,119][105620] Updated weights for policy 1, policy_version 1531847 (0.0008) [2023-12-27 02:31:52,137][105692] Updated weights for policy 0, policy_version 1528984 (0.0007) [2023-12-27 02:31:52,197][105692] Updated weights for policy 0, policy_version 1528994 (0.0009) [2023-12-27 02:31:52,883][105692] Updated weights for policy 0, policy_version 1529004 (0.0009) [2023-12-27 02:31:52,939][105692] Updated weights for policy 0, policy_version 1529014 (0.0010) [2023-12-27 02:31:52,954][105620] Updated weights for policy 1, policy_version 1531857 (0.0007) [2023-12-27 02:31:52,985][105692] Updated weights for policy 0, policy_version 1529024 (0.0010) [2023-12-27 02:31:53,008][105620] Updated weights for policy 1, policy_version 1531867 (0.0006) [2023-12-27 02:31:53,071][105620] Updated weights for policy 1, policy_version 1531877 (0.0008) [2023-12-27 02:31:53,676][105692] Updated weights for policy 0, policy_version 1529034 (0.0009) [2023-12-27 02:31:53,707][105620] Updated weights for policy 1, policy_version 1531887 (0.0008) [2023-12-27 02:31:53,727][105692] Updated weights for policy 0, policy_version 1529044 (0.0005) [2023-12-27 02:31:53,767][105620] Updated weights for policy 1, policy_version 1531897 (0.0009) [2023-12-27 02:31:53,788][105692] Updated weights for policy 0, policy_version 1529054 (0.0009) [2023-12-27 02:31:53,818][105620] Updated weights for policy 1, policy_version 1531907 (0.0006) [2023-12-27 02:31:53,836][105692] Updated weights for policy 0, policy_version 1529064 (0.0010) [2023-12-27 02:31:54,486][105692] Updated weights for policy 0, policy_version 1529074 (0.0010) [2023-12-27 02:31:54,528][105620] Updated weights for policy 1, policy_version 1531917 (0.0007) [2023-12-27 02:31:54,547][105692] Updated weights for policy 0, policy_version 1529084 (0.0010) [2023-12-27 02:31:54,586][105620] Updated weights for policy 1, policy_version 1531927 (0.0008) [2023-12-27 02:31:54,599][105692] Updated weights for policy 0, policy_version 1529094 (0.0010) [2023-12-27 02:31:54,648][105620] Updated weights for policy 1, policy_version 1531937 (0.0008) [2023-12-27 02:31:55,335][105692] Updated weights for policy 0, policy_version 1529104 (0.0010) [2023-12-27 02:31:55,347][105620] Updated weights for policy 1, policy_version 1531947 (0.0008) [2023-12-27 02:31:55,389][105692] Updated weights for policy 0, policy_version 1529114 (0.0010) [2023-12-27 02:31:55,408][105620] Updated weights for policy 1, policy_version 1531957 (0.0010) [2023-12-27 02:31:55,448][105692] Updated weights for policy 0, policy_version 1529124 (0.0010) [2023-12-27 02:31:55,456][105620] Updated weights for policy 1, policy_version 1531967 (0.0010) [2023-12-27 02:31:56,038][105620] Updated weights for policy 1, policy_version 1531977 (0.0010) [2023-12-27 02:31:56,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 783753216. Throughput: 0: 9707.1, 1: 10001.3. Samples: 783765996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:31:56,063][104569] Avg episode reward: [(0, '8443.380'), (1, '8815.934')] [2023-12-27 02:31:56,093][105620] Updated weights for policy 1, policy_version 1531987 (0.0010) [2023-12-27 02:31:56,142][105620] Updated weights for policy 1, policy_version 1531997 (0.0011) [2023-12-27 02:31:56,175][105692] Updated weights for policy 0, policy_version 1529134 (0.0009) [2023-12-27 02:31:56,203][105620] Updated weights for policy 1, policy_version 1532007 (0.0011) [2023-12-27 02:31:56,234][105692] Updated weights for policy 0, policy_version 1529144 (0.0006) [2023-12-27 02:31:56,292][105692] Updated weights for policy 0, policy_version 1529154 (0.0006) [2023-12-27 02:31:56,926][105620] Updated weights for policy 1, policy_version 1532017 (0.0011) [2023-12-27 02:31:56,984][105620] Updated weights for policy 1, policy_version 1532027 (0.0010) [2023-12-27 02:31:56,990][105692] Updated weights for policy 0, policy_version 1529164 (0.0009) [2023-12-27 02:31:57,042][105620] Updated weights for policy 1, policy_version 1532037 (0.0011) [2023-12-27 02:31:57,050][105692] Updated weights for policy 0, policy_version 1529174 (0.0006) [2023-12-27 02:31:57,110][105692] Updated weights for policy 0, policy_version 1529184 (0.0005) [2023-12-27 02:31:57,666][105620] Updated weights for policy 1, policy_version 1532047 (0.0007) [2023-12-27 02:31:57,718][105620] Updated weights for policy 1, policy_version 1532057 (0.0006) [2023-12-27 02:31:57,723][105692] Updated weights for policy 0, policy_version 1529194 (0.0005) [2023-12-27 02:31:57,774][105620] Updated weights for policy 1, policy_version 1532067 (0.0006) [2023-12-27 02:31:57,786][105692] Updated weights for policy 0, policy_version 1529204 (0.0007) [2023-12-27 02:31:57,846][105692] Updated weights for policy 0, policy_version 1529214 (0.0005) [2023-12-27 02:31:57,914][105692] Updated weights for policy 0, policy_version 1529224 (0.0005) [2023-12-27 02:31:58,494][105620] Updated weights for policy 1, policy_version 1532077 (0.0009) [2023-12-27 02:31:58,504][105692] Updated weights for policy 0, policy_version 1529234 (0.0008) [2023-12-27 02:31:58,554][105620] Updated weights for policy 1, policy_version 1532087 (0.0011) [2023-12-27 02:31:58,564][105692] Updated weights for policy 0, policy_version 1529244 (0.0007) [2023-12-27 02:31:58,619][105620] Updated weights for policy 1, policy_version 1532098 (0.0014) [2023-12-27 02:31:58,629][105692] Updated weights for policy 0, policy_version 1529254 (0.0007) [2023-12-27 02:31:59,426][105692] Updated weights for policy 0, policy_version 1529264 (0.0008) [2023-12-27 02:31:59,469][105620] Updated weights for policy 1, policy_version 1532108 (0.0011) [2023-12-27 02:31:59,484][105692] Updated weights for policy 0, policy_version 1529274 (0.0008) [2023-12-27 02:31:59,522][105620] Updated weights for policy 1, policy_version 1532118 (0.0011) [2023-12-27 02:31:59,539][105692] Updated weights for policy 0, policy_version 1529284 (0.0008) [2023-12-27 02:31:59,574][105620] Updated weights for policy 1, policy_version 1532128 (0.0010) [2023-12-27 02:32:00,258][105692] Updated weights for policy 0, policy_version 1529294 (0.0006) [2023-12-27 02:32:00,323][105692] Updated weights for policy 0, policy_version 1529304 (0.0007) [2023-12-27 02:32:00,338][105620] Updated weights for policy 1, policy_version 1532138 (0.0010) [2023-12-27 02:32:00,378][105692] Updated weights for policy 0, policy_version 1529314 (0.0009) [2023-12-27 02:32:00,397][105620] Updated weights for policy 1, policy_version 1532148 (0.0007) [2023-12-27 02:32:00,452][105620] Updated weights for policy 1, policy_version 1532158 (0.0007) [2023-12-27 02:32:00,513][105620] Updated weights for policy 1, policy_version 1532168 (0.0009) [2023-12-27 02:32:00,945][105692] Updated weights for policy 0, policy_version 1529324 (0.0008) [2023-12-27 02:32:00,990][105692] Updated weights for policy 0, policy_version 1529334 (0.0005) [2023-12-27 02:32:01,047][105692] Updated weights for policy 0, policy_version 1529344 (0.0007) [2023-12-27 02:32:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 783851520. Throughput: 0: 9790.5, 1: 10000.3. Samples: 783826888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:32:01,062][104569] Avg episode reward: [(0, '8535.330'), (1, '8992.892')] [2023-12-27 02:32:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001532168_392290304.pth... [2023-12-27 02:32:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001531016_391995392.pth [2023-12-27 02:32:01,090][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001529352_391569408.pth... [2023-12-27 02:32:01,093][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001528200_391274496.pth [2023-12-27 02:32:01,291][105620] Updated weights for policy 1, policy_version 1532178 (0.0006) [2023-12-27 02:32:01,358][105620] Updated weights for policy 1, policy_version 1532188 (0.0007) [2023-12-27 02:32:01,415][105620] Updated weights for policy 1, policy_version 1532198 (0.0008) [2023-12-27 02:32:01,724][105692] Updated weights for policy 0, policy_version 1529354 (0.0009) [2023-12-27 02:32:01,780][105692] Updated weights for policy 0, policy_version 1529364 (0.0010) [2023-12-27 02:32:01,841][105692] Updated weights for policy 0, policy_version 1529374 (0.0005) [2023-12-27 02:32:01,903][105692] Updated weights for policy 0, policy_version 1529384 (0.0007) [2023-12-27 02:32:02,190][105620] Updated weights for policy 1, policy_version 1532208 (0.0010) [2023-12-27 02:32:02,248][105620] Updated weights for policy 1, policy_version 1532218 (0.0010) [2023-12-27 02:32:02,308][105620] Updated weights for policy 1, policy_version 1532228 (0.0011) [2023-12-27 02:32:02,591][105692] Updated weights for policy 0, policy_version 1529394 (0.0010) [2023-12-27 02:32:02,655][105692] Updated weights for policy 0, policy_version 1529404 (0.0010) [2023-12-27 02:32:02,727][105692] Updated weights for policy 0, policy_version 1529414 (0.0010) [2023-12-27 02:32:02,967][105620] Updated weights for policy 1, policy_version 1532238 (0.0010) [2023-12-27 02:32:03,027][105620] Updated weights for policy 1, policy_version 1532248 (0.0005) [2023-12-27 02:32:03,085][105620] Updated weights for policy 1, policy_version 1532258 (0.0005) [2023-12-27 02:32:03,441][105692] Updated weights for policy 0, policy_version 1529424 (0.0010) [2023-12-27 02:32:03,489][105692] Updated weights for policy 0, policy_version 1529434 (0.0010) [2023-12-27 02:32:03,533][105692] Updated weights for policy 0, policy_version 1529444 (0.0010) [2023-12-27 02:32:03,616][105620] Updated weights for policy 1, policy_version 1532268 (0.0005) [2023-12-27 02:32:03,667][105620] Updated weights for policy 1, policy_version 1532278 (0.0005) [2023-12-27 02:32:03,732][105620] Updated weights for policy 1, policy_version 1532288 (0.0005) [2023-12-27 02:32:04,313][105620] Updated weights for policy 1, policy_version 1532298 (0.0006) [2023-12-27 02:32:04,318][105692] Updated weights for policy 0, policy_version 1529454 (0.0011) [2023-12-27 02:32:04,376][105620] Updated weights for policy 1, policy_version 1532308 (0.0008) [2023-12-27 02:32:04,379][105692] Updated weights for policy 0, policy_version 1529464 (0.0009) [2023-12-27 02:32:04,439][105620] Updated weights for policy 1, policy_version 1532318 (0.0009) [2023-12-27 02:32:04,440][105692] Updated weights for policy 0, policy_version 1529474 (0.0011) [2023-12-27 02:32:04,502][105620] Updated weights for policy 1, policy_version 1532328 (0.0008) [2023-12-27 02:32:05,110][105692] Updated weights for policy 0, policy_version 1529484 (0.0011) [2023-12-27 02:32:05,135][105620] Updated weights for policy 1, policy_version 1532338 (0.0005) [2023-12-27 02:32:05,162][105692] Updated weights for policy 0, policy_version 1529494 (0.0010) [2023-12-27 02:32:05,187][105620] Updated weights for policy 1, policy_version 1532348 (0.0005) [2023-12-27 02:32:05,214][105692] Updated weights for policy 0, policy_version 1529504 (0.0011) [2023-12-27 02:32:05,241][105620] Updated weights for policy 1, policy_version 1532358 (0.0005) [2023-12-27 02:32:05,774][105620] Updated weights for policy 1, policy_version 1532368 (0.0005) [2023-12-27 02:32:05,832][105620] Updated weights for policy 1, policy_version 1532378 (0.0005) [2023-12-27 02:32:05,899][105620] Updated weights for policy 1, policy_version 1532388 (0.0005) [2023-12-27 02:32:05,926][105692] Updated weights for policy 0, policy_version 1529514 (0.0011) [2023-12-27 02:32:05,980][105692] Updated weights for policy 0, policy_version 1529524 (0.0010) [2023-12-27 02:32:06,028][105692] Updated weights for policy 0, policy_version 1529534 (0.0010) [2023-12-27 02:32:06,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 783958016. Throughput: 0: 9736.3, 1: 10048.6. Samples: 783946344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:32:06,063][104569] Avg episode reward: [(0, '8718.706'), (1, '8808.487')] [2023-12-27 02:32:06,073][105692] Updated weights for policy 0, policy_version 1529544 (0.0010) [2023-12-27 02:32:06,600][105620] Updated weights for policy 1, policy_version 1532398 (0.0007) [2023-12-27 02:32:06,666][105620] Updated weights for policy 1, policy_version 1532408 (0.0005) [2023-12-27 02:32:06,732][105620] Updated weights for policy 1, policy_version 1532418 (0.0008) [2023-12-27 02:32:06,846][105692] Updated weights for policy 0, policy_version 1529554 (0.0010) [2023-12-27 02:32:06,908][105692] Updated weights for policy 0, policy_version 1529564 (0.0010) [2023-12-27 02:32:06,971][105692] Updated weights for policy 0, policy_version 1529574 (0.0010) [2023-12-27 02:32:07,450][105620] Updated weights for policy 1, policy_version 1532428 (0.0009) [2023-12-27 02:32:07,513][105620] Updated weights for policy 1, policy_version 1532438 (0.0011) [2023-12-27 02:32:07,566][105620] Updated weights for policy 1, policy_version 1532448 (0.0010) [2023-12-27 02:32:07,601][105692] Updated weights for policy 0, policy_version 1529584 (0.0011) [2023-12-27 02:32:07,655][105692] Updated weights for policy 0, policy_version 1529594 (0.0011) [2023-12-27 02:32:07,708][105692] Updated weights for policy 0, policy_version 1529604 (0.0006) [2023-12-27 02:32:08,248][105620] Updated weights for policy 1, policy_version 1532458 (0.0010) [2023-12-27 02:32:08,303][105620] Updated weights for policy 1, policy_version 1532468 (0.0005) [2023-12-27 02:32:08,374][105620] Updated weights for policy 1, policy_version 1532478 (0.0008) [2023-12-27 02:32:08,436][105620] Updated weights for policy 1, policy_version 1532488 (0.0007) [2023-12-27 02:32:08,441][105692] Updated weights for policy 0, policy_version 1529614 (0.0009) [2023-12-27 02:32:08,511][105692] Updated weights for policy 0, policy_version 1529624 (0.0011) [2023-12-27 02:32:08,574][105692] Updated weights for policy 0, policy_version 1529634 (0.0011) [2023-12-27 02:32:09,095][105620] Updated weights for policy 1, policy_version 1532498 (0.0010) [2023-12-27 02:32:09,147][105620] Updated weights for policy 1, policy_version 1532508 (0.0010) [2023-12-27 02:32:09,153][105692] Updated weights for policy 0, policy_version 1529644 (0.0009) [2023-12-27 02:32:09,199][105620] Updated weights for policy 1, policy_version 1532518 (0.0010) [2023-12-27 02:32:09,206][105692] Updated weights for policy 0, policy_version 1529654 (0.0008) [2023-12-27 02:32:09,271][105692] Updated weights for policy 0, policy_version 1529664 (0.0010) [2023-12-27 02:32:09,955][105620] Updated weights for policy 1, policy_version 1532528 (0.0008) [2023-12-27 02:32:10,021][105620] Updated weights for policy 1, policy_version 1532538 (0.0008) [2023-12-27 02:32:10,036][105692] Updated weights for policy 0, policy_version 1529674 (0.0010) [2023-12-27 02:32:10,084][105620] Updated weights for policy 1, policy_version 1532548 (0.0008) [2023-12-27 02:32:10,097][105692] Updated weights for policy 0, policy_version 1529684 (0.0011) [2023-12-27 02:32:10,153][105692] Updated weights for policy 0, policy_version 1529694 (0.0011) [2023-12-27 02:32:10,213][105692] Updated weights for policy 0, policy_version 1529704 (0.0011) [2023-12-27 02:32:10,845][105620] Updated weights for policy 1, policy_version 1532558 (0.0009) [2023-12-27 02:32:10,907][105620] Updated weights for policy 1, policy_version 1532568 (0.0010) [2023-12-27 02:32:10,963][105620] Updated weights for policy 1, policy_version 1532578 (0.0010) [2023-12-27 02:32:10,972][105692] Updated weights for policy 0, policy_version 1529714 (0.0010) [2023-12-27 02:32:11,021][105692] Updated weights for policy 0, policy_version 1529724 (0.0010) [2023-12-27 02:32:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 784056320. Throughput: 0: 9800.5, 1: 10064.1. Samples: 784066352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:32:11,062][104569] Avg episode reward: [(0, '8537.705'), (1, '8806.235')] [2023-12-27 02:32:11,090][105692] Updated weights for policy 0, policy_version 1529734 (0.0010) [2023-12-27 02:32:11,646][105620] Updated weights for policy 1, policy_version 1532588 (0.0008) [2023-12-27 02:32:11,716][105620] Updated weights for policy 1, policy_version 1532598 (0.0009) [2023-12-27 02:32:11,778][105620] Updated weights for policy 1, policy_version 1532608 (0.0008) [2023-12-27 02:32:11,867][105692] Updated weights for policy 0, policy_version 1529744 (0.0011) [2023-12-27 02:32:11,931][105692] Updated weights for policy 0, policy_version 1529754 (0.0011) [2023-12-27 02:32:11,995][105692] Updated weights for policy 0, policy_version 1529764 (0.0011) [2023-12-27 02:32:12,568][105620] Updated weights for policy 1, policy_version 1532618 (0.0009) [2023-12-27 02:32:12,614][105620] Updated weights for policy 1, policy_version 1532628 (0.0008) [2023-12-27 02:32:12,673][105620] Updated weights for policy 1, policy_version 1532638 (0.0007) [2023-12-27 02:32:12,732][105620] Updated weights for policy 1, policy_version 1532648 (0.0008) [2023-12-27 02:32:12,743][105692] Updated weights for policy 0, policy_version 1529774 (0.0011) [2023-12-27 02:32:12,798][105692] Updated weights for policy 0, policy_version 1529784 (0.0011) [2023-12-27 02:32:12,862][105692] Updated weights for policy 0, policy_version 1529794 (0.0011) [2023-12-27 02:32:13,461][105692] Updated weights for policy 0, policy_version 1529804 (0.0010) [2023-12-27 02:32:13,520][105692] Updated weights for policy 0, policy_version 1529814 (0.0010) [2023-12-27 02:32:13,573][105620] Updated weights for policy 1, policy_version 1532658 (0.0009) [2023-12-27 02:32:13,578][105692] Updated weights for policy 0, policy_version 1529824 (0.0010) [2023-12-27 02:32:13,632][105620] Updated weights for policy 1, policy_version 1532668 (0.0006) [2023-12-27 02:32:13,686][105620] Updated weights for policy 1, policy_version 1532678 (0.0008) [2023-12-27 02:32:14,326][105692] Updated weights for policy 0, policy_version 1529834 (0.0010) [2023-12-27 02:32:14,387][105692] Updated weights for policy 0, policy_version 1529844 (0.0010) [2023-12-27 02:32:14,445][105692] Updated weights for policy 0, policy_version 1529854 (0.0010) [2023-12-27 02:32:14,447][105620] Updated weights for policy 1, policy_version 1532688 (0.0006) [2023-12-27 02:32:14,503][105620] Updated weights for policy 1, policy_version 1532698 (0.0005) [2023-12-27 02:32:14,504][105692] Updated weights for policy 0, policy_version 1529864 (0.0011) [2023-12-27 02:32:14,565][105620] Updated weights for policy 1, policy_version 1532708 (0.0008) [2023-12-27 02:32:15,238][105692] Updated weights for policy 0, policy_version 1529874 (0.0011) [2023-12-27 02:32:15,298][105692] Updated weights for policy 0, policy_version 1529884 (0.0011) [2023-12-27 02:32:15,326][105620] Updated weights for policy 1, policy_version 1532718 (0.0009) [2023-12-27 02:32:15,357][105692] Updated weights for policy 0, policy_version 1529894 (0.0011) [2023-12-27 02:32:15,380][105620] Updated weights for policy 1, policy_version 1532728 (0.0006) [2023-12-27 02:32:15,440][105620] Updated weights for policy 1, policy_version 1532738 (0.0008) [2023-12-27 02:32:16,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 784146432. Throughput: 0: 9669.6, 1: 9915.4. Samples: 784122240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:32:16,062][104569] Avg episode reward: [(0, '8900.677'), (1, '8902.340')] [2023-12-27 02:32:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001532744_392437760.pth... [2023-12-27 02:32:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001531624_392151040.pth [2023-12-27 02:32:16,115][105692] Updated weights for policy 0, policy_version 1529904 (0.0009) [2023-12-27 02:32:16,172][105692] Updated weights for policy 0, policy_version 1529914 (0.0008) [2023-12-27 02:32:16,201][105620] Updated weights for policy 1, policy_version 1532748 (0.0008) [2023-12-27 02:32:16,227][105692] Updated weights for policy 0, policy_version 1529924 (0.0006) [2023-12-27 02:32:16,244][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001529928_391716864.pth... [2023-12-27 02:32:16,265][105620] Updated weights for policy 1, policy_version 1532758 (0.0007) [2023-12-27 02:32:16,268][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001528744_391413760.pth [2023-12-27 02:32:16,324][105620] Updated weights for policy 1, policy_version 1532768 (0.0009) [2023-12-27 02:32:16,954][105692] Updated weights for policy 0, policy_version 1529934 (0.0006) [2023-12-27 02:32:17,004][105620] Updated weights for policy 1, policy_version 1532778 (0.0008) [2023-12-27 02:32:17,021][105692] Updated weights for policy 0, policy_version 1529944 (0.0009) [2023-12-27 02:32:17,073][105620] Updated weights for policy 1, policy_version 1532788 (0.0005) [2023-12-27 02:32:17,083][105692] Updated weights for policy 0, policy_version 1529954 (0.0007) [2023-12-27 02:32:17,130][105620] Updated weights for policy 1, policy_version 1532798 (0.0005) [2023-12-27 02:32:17,178][105620] Updated weights for policy 1, policy_version 1532808 (0.0008) [2023-12-27 02:32:17,779][105692] Updated weights for policy 0, policy_version 1529964 (0.0009) [2023-12-27 02:32:17,829][105692] Updated weights for policy 0, policy_version 1529974 (0.0009) [2023-12-27 02:32:17,875][105692] Updated weights for policy 0, policy_version 1529984 (0.0007) [2023-12-27 02:32:17,882][105620] Updated weights for policy 1, policy_version 1532818 (0.0008) [2023-12-27 02:32:17,944][105620] Updated weights for policy 1, policy_version 1532828 (0.0006) [2023-12-27 02:32:18,006][105620] Updated weights for policy 1, policy_version 1532838 (0.0006) [2023-12-27 02:32:18,670][105620] Updated weights for policy 1, policy_version 1532848 (0.0010) [2023-12-27 02:32:18,681][105692] Updated weights for policy 0, policy_version 1529994 (0.0007) [2023-12-27 02:32:18,732][105620] Updated weights for policy 1, policy_version 1532858 (0.0010) [2023-12-27 02:32:18,739][105692] Updated weights for policy 0, policy_version 1530004 (0.0006) [2023-12-27 02:32:18,792][105620] Updated weights for policy 1, policy_version 1532868 (0.0010) [2023-12-27 02:32:18,793][105692] Updated weights for policy 0, policy_version 1530014 (0.0008) [2023-12-27 02:32:18,848][105692] Updated weights for policy 0, policy_version 1530024 (0.0007) [2023-12-27 02:32:19,505][105620] Updated weights for policy 1, policy_version 1532878 (0.0009) [2023-12-27 02:32:19,557][105692] Updated weights for policy 0, policy_version 1530034 (0.0007) [2023-12-27 02:32:19,572][105620] Updated weights for policy 1, policy_version 1532888 (0.0008) [2023-12-27 02:32:19,613][105692] Updated weights for policy 0, policy_version 1530044 (0.0008) [2023-12-27 02:32:19,640][105620] Updated weights for policy 1, policy_version 1532898 (0.0005) [2023-12-27 02:32:19,665][105692] Updated weights for policy 0, policy_version 1530054 (0.0008) [2023-12-27 02:32:20,390][105692] Updated weights for policy 0, policy_version 1530064 (0.0008) [2023-12-27 02:32:20,411][105620] Updated weights for policy 1, policy_version 1532908 (0.0008) [2023-12-27 02:32:20,451][105692] Updated weights for policy 0, policy_version 1530074 (0.0007) [2023-12-27 02:32:20,470][105620] Updated weights for policy 1, policy_version 1532918 (0.0007) [2023-12-27 02:32:20,513][105692] Updated weights for policy 0, policy_version 1530084 (0.0009) [2023-12-27 02:32:20,531][105620] Updated weights for policy 1, policy_version 1532928 (0.0006) [2023-12-27 02:32:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 784244736. Throughput: 0: 9715.2, 1: 9813.3. Samples: 784236720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:32:21,062][104569] Avg episode reward: [(0, '9084.520'), (1, '8998.577')] [2023-12-27 02:32:21,221][105620] Updated weights for policy 1, policy_version 1532938 (0.0009) [2023-12-27 02:32:21,281][105620] Updated weights for policy 1, policy_version 1532948 (0.0009) [2023-12-27 02:32:21,343][105620] Updated weights for policy 1, policy_version 1532958 (0.0008) [2023-12-27 02:32:21,347][105692] Updated weights for policy 0, policy_version 1530094 (0.0008) [2023-12-27 02:32:21,408][105620] Updated weights for policy 1, policy_version 1532968 (0.0009) [2023-12-27 02:32:21,413][105692] Updated weights for policy 0, policy_version 1530104 (0.0008) [2023-12-27 02:32:21,484][105692] Updated weights for policy 0, policy_version 1530114 (0.0006) [2023-12-27 02:32:22,132][105692] Updated weights for policy 0, policy_version 1530124 (0.0007) [2023-12-27 02:32:22,181][105692] Updated weights for policy 0, policy_version 1530134 (0.0005) [2023-12-27 02:32:22,251][105692] Updated weights for policy 0, policy_version 1530144 (0.0007) [2023-12-27 02:32:22,309][105620] Updated weights for policy 1, policy_version 1532978 (0.0007) [2023-12-27 02:32:22,371][105620] Updated weights for policy 1, policy_version 1532988 (0.0008) [2023-12-27 02:32:22,428][105620] Updated weights for policy 1, policy_version 1532998 (0.0008) [2023-12-27 02:32:22,949][105692] Updated weights for policy 0, policy_version 1530154 (0.0011) [2023-12-27 02:32:23,002][105692] Updated weights for policy 0, policy_version 1530164 (0.0009) [2023-12-27 02:32:23,052][105692] Updated weights for policy 0, policy_version 1530174 (0.0010) [2023-12-27 02:32:23,097][105692] Updated weights for policy 0, policy_version 1530184 (0.0010) [2023-12-27 02:32:23,201][105620] Updated weights for policy 1, policy_version 1533008 (0.0009) [2023-12-27 02:32:23,257][105620] Updated weights for policy 1, policy_version 1533019 (0.0010) [2023-12-27 02:32:23,309][105620] Updated weights for policy 1, policy_version 1533029 (0.0009) [2023-12-27 02:32:23,772][105692] Updated weights for policy 0, policy_version 1530194 (0.0008) [2023-12-27 02:32:23,823][105692] Updated weights for policy 0, policy_version 1530204 (0.0009) [2023-12-27 02:32:23,870][105692] Updated weights for policy 0, policy_version 1530214 (0.0008) [2023-12-27 02:32:24,133][105620] Updated weights for policy 1, policy_version 1533039 (0.0008) [2023-12-27 02:32:24,187][105620] Updated weights for policy 1, policy_version 1533049 (0.0009) [2023-12-27 02:32:24,246][105620] Updated weights for policy 1, policy_version 1533060 (0.0011) [2023-12-27 02:32:24,494][105692] Updated weights for policy 0, policy_version 1530224 (0.0007) [2023-12-27 02:32:24,551][105692] Updated weights for policy 0, policy_version 1530234 (0.0010) [2023-12-27 02:32:24,611][105692] Updated weights for policy 0, policy_version 1530244 (0.0011) [2023-12-27 02:32:24,982][105620] Updated weights for policy 1, policy_version 1533070 (0.0007) [2023-12-27 02:32:25,052][105620] Updated weights for policy 1, policy_version 1533080 (0.0009) [2023-12-27 02:32:25,113][105620] Updated weights for policy 1, policy_version 1533090 (0.0005) [2023-12-27 02:32:25,346][105692] Updated weights for policy 0, policy_version 1530254 (0.0010) [2023-12-27 02:32:25,398][105692] Updated weights for policy 0, policy_version 1530264 (0.0010) [2023-12-27 02:32:25,456][105692] Updated weights for policy 0, policy_version 1530274 (0.0010) [2023-12-27 02:32:25,716][105620] Updated weights for policy 1, policy_version 1533100 (0.0005) [2023-12-27 02:32:25,761][105620] Updated weights for policy 1, policy_version 1533110 (0.0005) [2023-12-27 02:32:25,809][105620] Updated weights for policy 1, policy_version 1533120 (0.0006) [2023-12-27 02:32:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.9, 300 sec: 19522.0). Total num frames: 784343040. Throughput: 0: 9764.0, 1: 9709.5. Samples: 784351644. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:32:26,062][104569] Avg episode reward: [(0, '8993.385'), (1, '9175.331')] [2023-12-27 02:32:26,109][105692] Updated weights for policy 0, policy_version 1530284 (0.0008) [2023-12-27 02:32:26,167][105692] Updated weights for policy 0, policy_version 1530294 (0.0005) [2023-12-27 02:32:26,228][105692] Updated weights for policy 0, policy_version 1530304 (0.0005) [2023-12-27 02:32:26,616][105620] Updated weights for policy 1, policy_version 1533131 (0.0010) [2023-12-27 02:32:26,681][105620] Updated weights for policy 1, policy_version 1533141 (0.0009) [2023-12-27 02:32:26,735][105620] Updated weights for policy 1, policy_version 1533151 (0.0009) [2023-12-27 02:32:26,894][105692] Updated weights for policy 0, policy_version 1530314 (0.0007) [2023-12-27 02:32:26,945][105692] Updated weights for policy 0, policy_version 1530325 (0.0010) [2023-12-27 02:32:26,999][105692] Updated weights for policy 0, policy_version 1530336 (0.0010) [2023-12-27 02:32:27,440][105620] Updated weights for policy 1, policy_version 1533161 (0.0009) [2023-12-27 02:32:27,488][105620] Updated weights for policy 1, policy_version 1533171 (0.0008) [2023-12-27 02:32:27,535][105620] Updated weights for policy 1, policy_version 1533181 (0.0009) [2023-12-27 02:32:27,580][105620] Updated weights for policy 1, policy_version 1533191 (0.0008) [2023-12-27 02:32:27,802][105692] Updated weights for policy 0, policy_version 1530347 (0.0010) [2023-12-27 02:32:27,856][105692] Updated weights for policy 0, policy_version 1530357 (0.0009) [2023-12-27 02:32:27,903][105692] Updated weights for policy 0, policy_version 1530367 (0.0009) [2023-12-27 02:32:28,262][105620] Updated weights for policy 1, policy_version 1533201 (0.0009) [2023-12-27 02:32:28,308][105620] Updated weights for policy 1, policy_version 1533211 (0.0009) [2023-12-27 02:32:28,365][105620] Updated weights for policy 1, policy_version 1533221 (0.0009) [2023-12-27 02:32:28,666][105692] Updated weights for policy 0, policy_version 1530377 (0.0008) [2023-12-27 02:32:28,735][105692] Updated weights for policy 0, policy_version 1530387 (0.0006) [2023-12-27 02:32:28,803][105692] Updated weights for policy 0, policy_version 1530397 (0.0006) [2023-12-27 02:32:28,869][105692] Updated weights for policy 0, policy_version 1530407 (0.0006) [2023-12-27 02:32:29,053][105620] Updated weights for policy 1, policy_version 1533231 (0.0009) [2023-12-27 02:32:29,112][105620] Updated weights for policy 1, policy_version 1533241 (0.0010) [2023-12-27 02:32:29,163][105620] Updated weights for policy 1, policy_version 1533251 (0.0010) [2023-12-27 02:32:29,431][105692] Updated weights for policy 0, policy_version 1530417 (0.0006) [2023-12-27 02:32:29,494][105692] Updated weights for policy 0, policy_version 1530427 (0.0006) [2023-12-27 02:32:29,552][105692] Updated weights for policy 0, policy_version 1530437 (0.0005) [2023-12-27 02:32:29,869][105620] Updated weights for policy 1, policy_version 1533261 (0.0007) [2023-12-27 02:32:29,934][105620] Updated weights for policy 1, policy_version 1533271 (0.0006) [2023-12-27 02:32:29,986][105620] Updated weights for policy 1, policy_version 1533281 (0.0009) [2023-12-27 02:32:30,197][105692] Updated weights for policy 0, policy_version 1530447 (0.0008) [2023-12-27 02:32:30,248][105692] Updated weights for policy 0, policy_version 1530457 (0.0009) [2023-12-27 02:32:30,305][105692] Updated weights for policy 0, policy_version 1530467 (0.0008) [2023-12-27 02:32:30,667][105620] Updated weights for policy 1, policy_version 1533291 (0.0008) [2023-12-27 02:32:30,731][105620] Updated weights for policy 1, policy_version 1533301 (0.0008) [2023-12-27 02:32:30,796][105620] Updated weights for policy 1, policy_version 1533311 (0.0009) [2023-12-27 02:32:30,984][105692] Updated weights for policy 0, policy_version 1530477 (0.0009) [2023-12-27 02:32:31,048][105692] Updated weights for policy 0, policy_version 1530487 (0.0009) [2023-12-27 02:32:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 784441344. Throughput: 0: 9825.8, 1: 9702.3. Samples: 784410684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:32:31,063][104569] Avg episode reward: [(0, '9081.185'), (1, '9169.761')] [2023-12-27 02:32:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001533320_392585216.pth... [2023-12-27 02:32:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001532168_392290304.pth [2023-12-27 02:32:31,111][105692] Updated weights for policy 0, policy_version 1530497 (0.0009) [2023-12-27 02:32:31,154][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001530504_391864320.pth... [2023-12-27 02:32:31,159][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001529352_391569408.pth [2023-12-27 02:32:31,489][105620] Updated weights for policy 1, policy_version 1533321 (0.0009) [2023-12-27 02:32:31,550][105620] Updated weights for policy 1, policy_version 1533331 (0.0009) [2023-12-27 02:32:31,617][105620] Updated weights for policy 1, policy_version 1533341 (0.0010) [2023-12-27 02:32:31,682][105620] Updated weights for policy 1, policy_version 1533351 (0.0009) [2023-12-27 02:32:31,895][105692] Updated weights for policy 0, policy_version 1530507 (0.0008) [2023-12-27 02:32:31,963][105692] Updated weights for policy 0, policy_version 1530517 (0.0009) [2023-12-27 02:32:32,025][105692] Updated weights for policy 0, policy_version 1530527 (0.0008) [2023-12-27 02:32:32,390][105620] Updated weights for policy 1, policy_version 1533361 (0.0008) [2023-12-27 02:32:32,450][105620] Updated weights for policy 1, policy_version 1533371 (0.0008) [2023-12-27 02:32:32,509][105620] Updated weights for policy 1, policy_version 1533381 (0.0008) [2023-12-27 02:32:32,731][105692] Updated weights for policy 0, policy_version 1530537 (0.0009) [2023-12-27 02:32:32,786][105692] Updated weights for policy 0, policy_version 1530547 (0.0005) [2023-12-27 02:32:32,860][105692] Updated weights for policy 0, policy_version 1530557 (0.0005) [2023-12-27 02:32:32,923][105692] Updated weights for policy 0, policy_version 1530567 (0.0006) [2023-12-27 02:32:33,173][105620] Updated weights for policy 1, policy_version 1533391 (0.0007) [2023-12-27 02:32:33,227][105620] Updated weights for policy 1, policy_version 1533401 (0.0005) [2023-12-27 02:32:33,283][105620] Updated weights for policy 1, policy_version 1533411 (0.0005) [2023-12-27 02:32:33,622][105692] Updated weights for policy 0, policy_version 1530577 (0.0005) [2023-12-27 02:32:33,673][105692] Updated weights for policy 0, policy_version 1530587 (0.0005) [2023-12-27 02:32:33,723][105692] Updated weights for policy 0, policy_version 1530597 (0.0005) [2023-12-27 02:32:33,887][105620] Updated weights for policy 1, policy_version 1533421 (0.0008) [2023-12-27 02:32:33,938][105620] Updated weights for policy 1, policy_version 1533431 (0.0010) [2023-12-27 02:32:34,000][105620] Updated weights for policy 1, policy_version 1533441 (0.0010) [2023-12-27 02:32:34,376][105692] Updated weights for policy 0, policy_version 1530607 (0.0008) [2023-12-27 02:32:34,430][105692] Updated weights for policy 0, policy_version 1530617 (0.0010) [2023-12-27 02:32:34,493][105692] Updated weights for policy 0, policy_version 1530627 (0.0010) [2023-12-27 02:32:34,620][105620] Updated weights for policy 1, policy_version 1533451 (0.0009) [2023-12-27 02:32:34,669][105620] Updated weights for policy 1, policy_version 1533461 (0.0009) [2023-12-27 02:32:34,721][105620] Updated weights for policy 1, policy_version 1533471 (0.0009) [2023-12-27 02:32:35,270][105692] Updated weights for policy 0, policy_version 1530637 (0.0010) [2023-12-27 02:32:35,318][105692] Updated weights for policy 0, policy_version 1530647 (0.0009) [2023-12-27 02:32:35,368][105692] Updated weights for policy 0, policy_version 1530657 (0.0009) [2023-12-27 02:32:35,506][105620] Updated weights for policy 1, policy_version 1533481 (0.0009) [2023-12-27 02:32:35,560][105620] Updated weights for policy 1, policy_version 1533491 (0.0005) [2023-12-27 02:32:35,605][105620] Updated weights for policy 1, policy_version 1533501 (0.0005) [2023-12-27 02:32:35,651][105620] Updated weights for policy 1, policy_version 1533511 (0.0005) [2023-12-27 02:32:36,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 784539648. Throughput: 0: 9868.8, 1: 9784.1. Samples: 784531772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:32:36,063][104569] Avg episode reward: [(0, '9081.436'), (1, '9081.435')] [2023-12-27 02:32:36,168][105692] Updated weights for policy 0, policy_version 1530667 (0.0008) [2023-12-27 02:32:36,218][105692] Updated weights for policy 0, policy_version 1530677 (0.0008) [2023-12-27 02:32:36,271][105692] Updated weights for policy 0, policy_version 1530687 (0.0008) [2023-12-27 02:32:36,383][105620] Updated weights for policy 1, policy_version 1533521 (0.0010) [2023-12-27 02:32:36,444][105620] Updated weights for policy 1, policy_version 1533531 (0.0011) [2023-12-27 02:32:36,504][105620] Updated weights for policy 1, policy_version 1533541 (0.0011) [2023-12-27 02:32:37,056][105692] Updated weights for policy 0, policy_version 1530697 (0.0008) [2023-12-27 02:32:37,108][105692] Updated weights for policy 0, policy_version 1530707 (0.0011) [2023-12-27 02:32:37,164][105692] Updated weights for policy 0, policy_version 1530717 (0.0010) [2023-12-27 02:32:37,214][105692] Updated weights for policy 0, policy_version 1530727 (0.0011) [2023-12-27 02:32:37,227][105620] Updated weights for policy 1, policy_version 1533551 (0.0007) [2023-12-27 02:32:37,294][105620] Updated weights for policy 1, policy_version 1533561 (0.0008) [2023-12-27 02:32:37,355][105620] Updated weights for policy 1, policy_version 1533571 (0.0006) [2023-12-27 02:32:37,987][105692] Updated weights for policy 0, policy_version 1530737 (0.0010) [2023-12-27 02:32:38,031][105620] Updated weights for policy 1, policy_version 1533581 (0.0005) [2023-12-27 02:32:38,051][105692] Updated weights for policy 0, policy_version 1530747 (0.0011) [2023-12-27 02:32:38,102][105620] Updated weights for policy 1, policy_version 1533591 (0.0005) [2023-12-27 02:32:38,115][105692] Updated weights for policy 0, policy_version 1530757 (0.0011) [2023-12-27 02:32:38,163][105620] Updated weights for policy 1, policy_version 1533601 (0.0007) [2023-12-27 02:32:38,808][105692] Updated weights for policy 0, policy_version 1530767 (0.0010) [2023-12-27 02:32:38,862][105692] Updated weights for policy 0, policy_version 1530777 (0.0010) [2023-12-27 02:32:38,874][105620] Updated weights for policy 1, policy_version 1533611 (0.0008) [2023-12-27 02:32:38,918][105692] Updated weights for policy 0, policy_version 1530787 (0.0010) [2023-12-27 02:32:38,926][105620] Updated weights for policy 1, policy_version 1533621 (0.0009) [2023-12-27 02:32:38,978][105620] Updated weights for policy 1, policy_version 1533631 (0.0008) [2023-12-27 02:32:39,729][105692] Updated weights for policy 0, policy_version 1530797 (0.0008) [2023-12-27 02:32:39,787][105692] Updated weights for policy 0, policy_version 1530807 (0.0008) [2023-12-27 02:32:39,797][105620] Updated weights for policy 1, policy_version 1533641 (0.0008) [2023-12-27 02:32:39,848][105692] Updated weights for policy 0, policy_version 1530817 (0.0009) [2023-12-27 02:32:39,865][105620] Updated weights for policy 1, policy_version 1533651 (0.0007) [2023-12-27 02:32:39,924][105620] Updated weights for policy 1, policy_version 1533661 (0.0008) [2023-12-27 02:32:39,989][105620] Updated weights for policy 1, policy_version 1533671 (0.0008) [2023-12-27 02:32:40,587][105692] Updated weights for policy 0, policy_version 1530827 (0.0008) [2023-12-27 02:32:40,656][105692] Updated weights for policy 0, policy_version 1530837 (0.0006) [2023-12-27 02:32:40,693][105620] Updated weights for policy 1, policy_version 1533681 (0.0007) [2023-12-27 02:32:40,725][105692] Updated weights for policy 0, policy_version 1530847 (0.0005) [2023-12-27 02:32:40,744][105620] Updated weights for policy 1, policy_version 1533691 (0.0007) [2023-12-27 02:32:40,797][105620] Updated weights for policy 1, policy_version 1533701 (0.0006) [2023-12-27 02:32:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 784637952. Throughput: 0: 9775.6, 1: 9770.3. Samples: 784645556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:32:41,063][104569] Avg episode reward: [(0, '9083.921'), (1, '8998.698')] [2023-12-27 02:32:41,359][105692] Updated weights for policy 0, policy_version 1530857 (0.0009) [2023-12-27 02:32:41,419][105692] Updated weights for policy 0, policy_version 1530867 (0.0008) [2023-12-27 02:32:41,464][105692] Updated weights for policy 0, policy_version 1530877 (0.0008) [2023-12-27 02:32:41,517][105692] Updated weights for policy 0, policy_version 1530887 (0.0008) [2023-12-27 02:32:41,533][105620] Updated weights for policy 1, policy_version 1533711 (0.0010) [2023-12-27 02:32:41,597][105620] Updated weights for policy 1, policy_version 1533721 (0.0011) [2023-12-27 02:32:41,663][105620] Updated weights for policy 1, policy_version 1533731 (0.0009) [2023-12-27 02:32:42,268][105692] Updated weights for policy 0, policy_version 1530897 (0.0007) [2023-12-27 02:32:42,331][105692] Updated weights for policy 0, policy_version 1530907 (0.0008) [2023-12-27 02:32:42,382][105620] Updated weights for policy 1, policy_version 1533741 (0.0009) [2023-12-27 02:32:42,397][105692] Updated weights for policy 0, policy_version 1530917 (0.0008) [2023-12-27 02:32:42,448][105620] Updated weights for policy 1, policy_version 1533751 (0.0009) [2023-12-27 02:32:42,516][105620] Updated weights for policy 1, policy_version 1533761 (0.0010) [2023-12-27 02:32:43,034][105692] Updated weights for policy 0, policy_version 1530927 (0.0009) [2023-12-27 02:32:43,095][105692] Updated weights for policy 0, policy_version 1530937 (0.0009) [2023-12-27 02:32:43,157][105692] Updated weights for policy 0, policy_version 1530947 (0.0009) [2023-12-27 02:32:43,295][105620] Updated weights for policy 1, policy_version 1533771 (0.0010) [2023-12-27 02:32:43,359][105620] Updated weights for policy 1, policy_version 1533781 (0.0009) [2023-12-27 02:32:43,428][105620] Updated weights for policy 1, policy_version 1533791 (0.0009) [2023-12-27 02:32:43,907][105692] Updated weights for policy 0, policy_version 1530957 (0.0009) [2023-12-27 02:32:43,964][105692] Updated weights for policy 0, policy_version 1530967 (0.0009) [2023-12-27 02:32:44,018][105692] Updated weights for policy 0, policy_version 1530977 (0.0009) [2023-12-27 02:32:44,147][105620] Updated weights for policy 1, policy_version 1533801 (0.0009) [2023-12-27 02:32:44,215][105620] Updated weights for policy 1, policy_version 1533811 (0.0008) [2023-12-27 02:32:44,276][105620] Updated weights for policy 1, policy_version 1533821 (0.0009) [2023-12-27 02:32:44,336][105620] Updated weights for policy 1, policy_version 1533831 (0.0009) [2023-12-27 02:32:44,744][105692] Updated weights for policy 0, policy_version 1530987 (0.0009) [2023-12-27 02:32:44,805][105692] Updated weights for policy 0, policy_version 1530997 (0.0008) [2023-12-27 02:32:44,853][105692] Updated weights for policy 0, policy_version 1531007 (0.0009) [2023-12-27 02:32:45,099][105620] Updated weights for policy 1, policy_version 1533841 (0.0009) [2023-12-27 02:32:45,164][105620] Updated weights for policy 1, policy_version 1533851 (0.0009) [2023-12-27 02:32:45,229][105620] Updated weights for policy 1, policy_version 1533861 (0.0008) [2023-12-27 02:32:45,660][105692] Updated weights for policy 0, policy_version 1531017 (0.0009) [2023-12-27 02:32:45,726][105692] Updated weights for policy 0, policy_version 1531027 (0.0009) [2023-12-27 02:32:45,787][105692] Updated weights for policy 0, policy_version 1531037 (0.0009) [2023-12-27 02:32:45,841][105692] Updated weights for policy 0, policy_version 1531047 (0.0008) [2023-12-27 02:32:45,900][105620] Updated weights for policy 1, policy_version 1533871 (0.0009) [2023-12-27 02:32:45,950][105620] Updated weights for policy 1, policy_version 1533881 (0.0009) [2023-12-27 02:32:46,009][105620] Updated weights for policy 1, policy_version 1533891 (0.0010) [2023-12-27 02:32:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 784736256. Throughput: 0: 9741.3, 1: 9729.9. Samples: 784703096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:32:46,063][104569] Avg episode reward: [(0, '9085.348'), (1, '8996.124')] [2023-12-27 02:32:46,073][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001531048_392003584.pth... [2023-12-27 02:32:46,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001533896_392732672.pth... [2023-12-27 02:32:46,081][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001529928_391716864.pth [2023-12-27 02:32:46,092][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001532744_392437760.pth [2023-12-27 02:32:46,523][105692] Updated weights for policy 0, policy_version 1531057 (0.0009) [2023-12-27 02:32:46,585][105692] Updated weights for policy 0, policy_version 1531067 (0.0010) [2023-12-27 02:32:46,640][105692] Updated weights for policy 0, policy_version 1531077 (0.0010) [2023-12-27 02:32:46,796][105620] Updated weights for policy 1, policy_version 1533901 (0.0007) [2023-12-27 02:32:46,842][105620] Updated weights for policy 1, policy_version 1533911 (0.0005) [2023-12-27 02:32:46,852][105586] KL-divergence is very high: 117.9718 [2023-12-27 02:32:46,898][105586] KL-divergence is very high: 143.1711 [2023-12-27 02:32:46,899][105620] Updated weights for policy 1, policy_version 1533921 (0.0006) [2023-12-27 02:32:47,297][105692] Updated weights for policy 0, policy_version 1531087 (0.0007) [2023-12-27 02:32:47,348][105692] Updated weights for policy 0, policy_version 1531097 (0.0005) [2023-12-27 02:32:47,408][105692] Updated weights for policy 0, policy_version 1531107 (0.0005) [2023-12-27 02:32:47,566][105620] Updated weights for policy 1, policy_version 1533931 (0.0007) [2023-12-27 02:32:47,613][105620] Updated weights for policy 1, policy_version 1533941 (0.0005) [2023-12-27 02:32:47,661][105620] Updated weights for policy 1, policy_version 1533951 (0.0006) [2023-12-27 02:32:48,054][105692] Updated weights for policy 0, policy_version 1531117 (0.0010) [2023-12-27 02:32:48,121][105692] Updated weights for policy 0, policy_version 1531128 (0.0009) [2023-12-27 02:32:48,175][105692] Updated weights for policy 0, policy_version 1531138 (0.0006) [2023-12-27 02:32:48,309][105620] Updated weights for policy 1, policy_version 1533961 (0.0006) [2023-12-27 02:32:48,368][105620] Updated weights for policy 1, policy_version 1533971 (0.0008) [2023-12-27 02:32:48,413][105620] Updated weights for policy 1, policy_version 1533981 (0.0010) [2023-12-27 02:32:48,473][105620] Updated weights for policy 1, policy_version 1533991 (0.0011) [2023-12-27 02:32:48,846][105692] Updated weights for policy 0, policy_version 1531148 (0.0008) [2023-12-27 02:32:48,908][105692] Updated weights for policy 0, policy_version 1531158 (0.0007) [2023-12-27 02:32:48,963][105692] Updated weights for policy 0, policy_version 1531168 (0.0006) [2023-12-27 02:32:49,140][105620] Updated weights for policy 1, policy_version 1534001 (0.0010) [2023-12-27 02:32:49,193][105620] Updated weights for policy 1, policy_version 1534011 (0.0010) [2023-12-27 02:32:49,254][105620] Updated weights for policy 1, policy_version 1534021 (0.0011) [2023-12-27 02:32:49,617][105692] Updated weights for policy 0, policy_version 1531178 (0.0007) [2023-12-27 02:32:49,669][105692] Updated weights for policy 0, policy_version 1531188 (0.0011) [2023-12-27 02:32:49,738][105692] Updated weights for policy 0, policy_version 1531198 (0.0011) [2023-12-27 02:32:49,805][105692] Updated weights for policy 0, policy_version 1531208 (0.0011) [2023-12-27 02:32:49,892][105620] Updated weights for policy 1, policy_version 1534031 (0.0008) [2023-12-27 02:32:49,964][105620] Updated weights for policy 1, policy_version 1534041 (0.0007) [2023-12-27 02:32:50,027][105620] Updated weights for policy 1, policy_version 1534051 (0.0007) [2023-12-27 02:32:50,580][105692] Updated weights for policy 0, policy_version 1531218 (0.0009) [2023-12-27 02:32:50,588][105620] Updated weights for policy 1, policy_version 1534061 (0.0006) [2023-12-27 02:32:50,641][105692] Updated weights for policy 0, policy_version 1531228 (0.0010) [2023-12-27 02:32:50,656][105620] Updated weights for policy 1, policy_version 1534071 (0.0007) [2023-12-27 02:32:50,710][105692] Updated weights for policy 0, policy_version 1531238 (0.0007) [2023-12-27 02:32:50,720][105620] Updated weights for policy 1, policy_version 1534081 (0.0007) [2023-12-27 02:32:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 784834560. Throughput: 0: 9768.8, 1: 9724.0. Samples: 784823512. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:32:51,062][104569] Avg episode reward: [(0, '9083.231'), (1, '9173.420')] [2023-12-27 02:32:51,316][105692] Updated weights for policy 0, policy_version 1531248 (0.0007) [2023-12-27 02:32:51,386][105692] Updated weights for policy 0, policy_version 1531258 (0.0007) [2023-12-27 02:32:51,444][105692] Updated weights for policy 0, policy_version 1531268 (0.0006) [2023-12-27 02:32:51,473][105620] Updated weights for policy 1, policy_version 1534091 (0.0007) [2023-12-27 02:32:51,525][105620] Updated weights for policy 1, policy_version 1534101 (0.0009) [2023-12-27 02:32:51,582][105620] Updated weights for policy 1, policy_version 1534111 (0.0009) [2023-12-27 02:32:52,136][105692] Updated weights for policy 0, policy_version 1531278 (0.0008) [2023-12-27 02:32:52,199][105692] Updated weights for policy 0, policy_version 1531288 (0.0009) [2023-12-27 02:32:52,255][105692] Updated weights for policy 0, policy_version 1531298 (0.0009) [2023-12-27 02:32:52,372][105620] Updated weights for policy 1, policy_version 1534121 (0.0009) [2023-12-27 02:32:52,427][105620] Updated weights for policy 1, policy_version 1534131 (0.0010) [2023-12-27 02:32:52,487][105620] Updated weights for policy 1, policy_version 1534141 (0.0007) [2023-12-27 02:32:52,551][105620] Updated weights for policy 1, policy_version 1534151 (0.0006) [2023-12-27 02:32:53,094][105692] Updated weights for policy 0, policy_version 1531308 (0.0009) [2023-12-27 02:32:53,154][105692] Updated weights for policy 0, policy_version 1531318 (0.0008) [2023-12-27 02:32:53,191][105620] Updated weights for policy 1, policy_version 1534161 (0.0008) [2023-12-27 02:32:53,207][105692] Updated weights for policy 0, policy_version 1531328 (0.0007) [2023-12-27 02:32:53,243][105620] Updated weights for policy 1, policy_version 1534171 (0.0006) [2023-12-27 02:32:53,295][105620] Updated weights for policy 1, policy_version 1534181 (0.0009) [2023-12-27 02:32:53,815][105692] Updated weights for policy 0, policy_version 1531338 (0.0007) [2023-12-27 02:32:53,875][105692] Updated weights for policy 0, policy_version 1531348 (0.0005) [2023-12-27 02:32:53,931][105692] Updated weights for policy 0, policy_version 1531358 (0.0007) [2023-12-27 02:32:53,979][105692] Updated weights for policy 0, policy_version 1531368 (0.0010) [2023-12-27 02:32:54,002][105620] Updated weights for policy 1, policy_version 1534192 (0.0006) [2023-12-27 02:32:54,050][105620] Updated weights for policy 1, policy_version 1534202 (0.0008) [2023-12-27 02:32:54,105][105620] Updated weights for policy 1, policy_version 1534212 (0.0008) [2023-12-27 02:32:54,605][105692] Updated weights for policy 0, policy_version 1531378 (0.0009) [2023-12-27 02:32:54,656][105692] Updated weights for policy 0, policy_version 1531388 (0.0009) [2023-12-27 02:32:54,700][105692] Updated weights for policy 0, policy_version 1531398 (0.0008) [2023-12-27 02:32:54,913][105620] Updated weights for policy 1, policy_version 1534222 (0.0009) [2023-12-27 02:32:54,969][105620] Updated weights for policy 1, policy_version 1534232 (0.0009) [2023-12-27 02:32:55,029][105620] Updated weights for policy 1, policy_version 1534242 (0.0008) [2023-12-27 02:32:55,474][105692] Updated weights for policy 0, policy_version 1531408 (0.0009) [2023-12-27 02:32:55,518][105692] Updated weights for policy 0, policy_version 1531418 (0.0010) [2023-12-27 02:32:55,575][105692] Updated weights for policy 0, policy_version 1531428 (0.0010) [2023-12-27 02:32:55,801][105620] Updated weights for policy 1, policy_version 1534252 (0.0008) [2023-12-27 02:32:55,845][105620] Updated weights for policy 1, policy_version 1534262 (0.0008) [2023-12-27 02:32:55,889][105620] Updated weights for policy 1, policy_version 1534272 (0.0008) [2023-12-27 02:32:56,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19660.9, 300 sec: 19522.0). Total num frames: 784932864. Throughput: 0: 9745.4, 1: 9663.5. Samples: 784939752. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:32:56,062][104569] Avg episode reward: [(0, '8717.030'), (1, '9265.712')] [2023-12-27 02:32:56,323][105692] Updated weights for policy 0, policy_version 1531438 (0.0010) [2023-12-27 02:32:56,372][105692] Updated weights for policy 0, policy_version 1531448 (0.0010) [2023-12-27 02:32:56,443][105692] Updated weights for policy 0, policy_version 1531458 (0.0010) [2023-12-27 02:32:56,686][105620] Updated weights for policy 1, policy_version 1534282 (0.0008) [2023-12-27 02:32:56,743][105620] Updated weights for policy 1, policy_version 1534292 (0.0010) [2023-12-27 02:32:56,800][105620] Updated weights for policy 1, policy_version 1534302 (0.0009) [2023-12-27 02:32:56,850][105620] Updated weights for policy 1, policy_version 1534312 (0.0010) [2023-12-27 02:32:57,102][105692] Updated weights for policy 0, policy_version 1531468 (0.0010) [2023-12-27 02:32:57,146][105692] Updated weights for policy 0, policy_version 1531478 (0.0010) [2023-12-27 02:32:57,197][105692] Updated weights for policy 0, policy_version 1531488 (0.0010) [2023-12-27 02:32:57,601][105620] Updated weights for policy 1, policy_version 1534322 (0.0010) [2023-12-27 02:32:57,644][105620] Updated weights for policy 1, policy_version 1534332 (0.0010) [2023-12-27 02:32:57,690][105620] Updated weights for policy 1, policy_version 1534342 (0.0010) [2023-12-27 02:32:57,897][105692] Updated weights for policy 0, policy_version 1531498 (0.0010) [2023-12-27 02:32:57,948][105692] Updated weights for policy 0, policy_version 1531508 (0.0010) [2023-12-27 02:32:58,002][105692] Updated weights for policy 0, policy_version 1531518 (0.0010) [2023-12-27 02:32:58,069][105692] Updated weights for policy 0, policy_version 1531528 (0.0008) [2023-12-27 02:32:58,331][105620] Updated weights for policy 1, policy_version 1534352 (0.0008) [2023-12-27 02:32:58,400][105620] Updated weights for policy 1, policy_version 1534362 (0.0007) [2023-12-27 02:32:58,477][105620] Updated weights for policy 1, policy_version 1534372 (0.0008) [2023-12-27 02:32:58,772][105692] Updated weights for policy 0, policy_version 1531538 (0.0009) [2023-12-27 02:32:58,841][105692] Updated weights for policy 0, policy_version 1531548 (0.0008) [2023-12-27 02:32:58,911][105692] Updated weights for policy 0, policy_version 1531558 (0.0009) [2023-12-27 02:32:59,175][105620] Updated weights for policy 1, policy_version 1534382 (0.0007) [2023-12-27 02:32:59,230][105620] Updated weights for policy 1, policy_version 1534392 (0.0006) [2023-12-27 02:32:59,285][105620] Updated weights for policy 1, policy_version 1534402 (0.0008) [2023-12-27 02:32:59,562][105692] Updated weights for policy 0, policy_version 1531568 (0.0008) [2023-12-27 02:32:59,620][105692] Updated weights for policy 0, policy_version 1531578 (0.0007) [2023-12-27 02:32:59,676][105692] Updated weights for policy 0, policy_version 1531588 (0.0007) [2023-12-27 02:32:59,943][105620] Updated weights for policy 1, policy_version 1534412 (0.0007) [2023-12-27 02:33:00,010][105620] Updated weights for policy 1, policy_version 1534422 (0.0010) [2023-12-27 02:33:00,074][105620] Updated weights for policy 1, policy_version 1534432 (0.0012) [2023-12-27 02:33:00,339][105692] Updated weights for policy 0, policy_version 1531598 (0.0009) [2023-12-27 02:33:00,398][105692] Updated weights for policy 0, policy_version 1531608 (0.0009) [2023-12-27 02:33:00,468][105692] Updated weights for policy 0, policy_version 1531618 (0.0008) [2023-12-27 02:33:00,772][105620] Updated weights for policy 1, policy_version 1534442 (0.0010) [2023-12-27 02:33:00,827][105620] Updated weights for policy 1, policy_version 1534452 (0.0007) [2023-12-27 02:33:00,881][105620] Updated weights for policy 1, policy_version 1534462 (0.0005) [2023-12-27 02:33:00,927][105620] Updated weights for policy 1, policy_version 1534472 (0.0005) [2023-12-27 02:33:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 785031168. Throughput: 0: 9768.2, 1: 9720.7. Samples: 784999244. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:33:01,062][104569] Avg episode reward: [(0, '8262.814'), (1, '9173.211')] [2023-12-27 02:33:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001531624_392151040.pth... [2023-12-27 02:33:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001534472_392880128.pth... [2023-12-27 02:33:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001530504_391864320.pth [2023-12-27 02:33:01,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001533320_392585216.pth [2023-12-27 02:33:01,115][105692] Updated weights for policy 0, policy_version 1531628 (0.0009) [2023-12-27 02:33:01,180][105692] Updated weights for policy 0, policy_version 1531638 (0.0011) [2023-12-27 02:33:01,239][105692] Updated weights for policy 0, policy_version 1531648 (0.0010) [2023-12-27 02:33:01,636][105620] Updated weights for policy 1, policy_version 1534482 (0.0008) [2023-12-27 02:33:01,694][105620] Updated weights for policy 1, policy_version 1534492 (0.0007) [2023-12-27 02:33:01,757][105620] Updated weights for policy 1, policy_version 1534502 (0.0006) [2023-12-27 02:33:02,000][105692] Updated weights for policy 0, policy_version 1531658 (0.0011) [2023-12-27 02:33:02,055][105692] Updated weights for policy 0, policy_version 1531668 (0.0010) [2023-12-27 02:33:02,110][105692] Updated weights for policy 0, policy_version 1531678 (0.0010) [2023-12-27 02:33:02,154][105692] Updated weights for policy 0, policy_version 1531688 (0.0010) [2023-12-27 02:33:02,440][105620] Updated weights for policy 1, policy_version 1534512 (0.0005) [2023-12-27 02:33:02,500][105620] Updated weights for policy 1, policy_version 1534522 (0.0006) [2023-12-27 02:33:02,565][105620] Updated weights for policy 1, policy_version 1534532 (0.0007) [2023-12-27 02:33:02,920][105692] Updated weights for policy 0, policy_version 1531698 (0.0011) [2023-12-27 02:33:02,981][105692] Updated weights for policy 0, policy_version 1531708 (0.0010) [2023-12-27 02:33:03,038][105692] Updated weights for policy 0, policy_version 1531718 (0.0010) [2023-12-27 02:33:03,145][105620] Updated weights for policy 1, policy_version 1534542 (0.0008) [2023-12-27 02:33:03,205][105620] Updated weights for policy 1, policy_version 1534552 (0.0011) [2023-12-27 02:33:03,251][105620] Updated weights for policy 1, policy_version 1534562 (0.0009) [2023-12-27 02:33:03,744][105692] Updated weights for policy 0, policy_version 1531728 (0.0007) [2023-12-27 02:33:03,799][105692] Updated weights for policy 0, policy_version 1531738 (0.0005) [2023-12-27 02:33:03,860][105692] Updated weights for policy 0, policy_version 1531748 (0.0006) [2023-12-27 02:33:03,879][105620] Updated weights for policy 1, policy_version 1534572 (0.0007) [2023-12-27 02:33:03,941][105620] Updated weights for policy 1, policy_version 1534582 (0.0008) [2023-12-27 02:33:03,999][105620] Updated weights for policy 1, policy_version 1534592 (0.0006) [2023-12-27 02:33:04,477][105692] Updated weights for policy 0, policy_version 1531758 (0.0009) [2023-12-27 02:33:04,545][105692] Updated weights for policy 0, policy_version 1531768 (0.0011) [2023-12-27 02:33:04,580][105620] Updated weights for policy 1, policy_version 1534602 (0.0007) [2023-12-27 02:33:04,606][105692] Updated weights for policy 0, policy_version 1531778 (0.0007) [2023-12-27 02:33:04,641][105620] Updated weights for policy 1, policy_version 1534612 (0.0007) [2023-12-27 02:33:04,708][105620] Updated weights for policy 1, policy_version 1534622 (0.0008) [2023-12-27 02:33:04,772][105620] Updated weights for policy 1, policy_version 1534632 (0.0010) [2023-12-27 02:33:05,252][105692] Updated weights for policy 0, policy_version 1531788 (0.0009) [2023-12-27 02:33:05,316][105692] Updated weights for policy 0, policy_version 1531798 (0.0009) [2023-12-27 02:33:05,345][105620] Updated weights for policy 1, policy_version 1534642 (0.0005) [2023-12-27 02:33:05,379][105692] Updated weights for policy 0, policy_version 1531808 (0.0008) [2023-12-27 02:33:05,401][105620] Updated weights for policy 1, policy_version 1534652 (0.0005) [2023-12-27 02:33:05,469][105620] Updated weights for policy 1, policy_version 1534662 (0.0008) [2023-12-27 02:33:06,031][105620] Updated weights for policy 1, policy_version 1534672 (0.0010) [2023-12-27 02:33:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.4, 300 sec: 19549.7). Total num frames: 785129472. Throughput: 0: 9832.7, 1: 9835.9. Samples: 785121808. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:33:06,062][104569] Avg episode reward: [(0, '8354.737'), (1, '8991.640')] [2023-12-27 02:33:06,096][105620] Updated weights for policy 1, policy_version 1534682 (0.0010) [2023-12-27 02:33:06,157][105620] Updated weights for policy 1, policy_version 1534692 (0.0009) [2023-12-27 02:33:06,163][105692] Updated weights for policy 0, policy_version 1531818 (0.0007) [2023-12-27 02:33:06,226][105692] Updated weights for policy 0, policy_version 1531828 (0.0007) [2023-12-27 02:33:06,298][105692] Updated weights for policy 0, policy_version 1531838 (0.0008) [2023-12-27 02:33:06,365][105692] Updated weights for policy 0, policy_version 1531848 (0.0008) [2023-12-27 02:33:06,938][105620] Updated weights for policy 1, policy_version 1534702 (0.0011) [2023-12-27 02:33:06,990][105620] Updated weights for policy 1, policy_version 1534712 (0.0010) [2023-12-27 02:33:07,044][105620] Updated weights for policy 1, policy_version 1534722 (0.0010) [2023-12-27 02:33:07,116][105692] Updated weights for policy 0, policy_version 1531858 (0.0008) [2023-12-27 02:33:07,163][105692] Updated weights for policy 0, policy_version 1531868 (0.0007) [2023-12-27 02:33:07,214][105692] Updated weights for policy 0, policy_version 1531878 (0.0005) [2023-12-27 02:33:07,688][105620] Updated weights for policy 1, policy_version 1534732 (0.0007) [2023-12-27 02:33:07,747][105620] Updated weights for policy 1, policy_version 1534742 (0.0005) [2023-12-27 02:33:07,801][105620] Updated weights for policy 1, policy_version 1534752 (0.0005) [2023-12-27 02:33:08,093][105692] Updated weights for policy 0, policy_version 1531888 (0.0008) [2023-12-27 02:33:08,141][105692] Updated weights for policy 0, policy_version 1531898 (0.0007) [2023-12-27 02:33:08,197][105692] Updated weights for policy 0, policy_version 1531908 (0.0008) [2023-12-27 02:33:08,385][105620] Updated weights for policy 1, policy_version 1534762 (0.0006) [2023-12-27 02:33:08,445][105620] Updated weights for policy 1, policy_version 1534772 (0.0010) [2023-12-27 02:33:08,517][105620] Updated weights for policy 1, policy_version 1534782 (0.0007) [2023-12-27 02:33:08,566][105620] Updated weights for policy 1, policy_version 1534792 (0.0010) [2023-12-27 02:33:08,866][105692] Updated weights for policy 0, policy_version 1531918 (0.0008) [2023-12-27 02:33:08,925][105692] Updated weights for policy 0, policy_version 1531928 (0.0008) [2023-12-27 02:33:08,988][105692] Updated weights for policy 0, policy_version 1531938 (0.0008) [2023-12-27 02:33:09,233][105620] Updated weights for policy 1, policy_version 1534802 (0.0010) [2023-12-27 02:33:09,282][105620] Updated weights for policy 1, policy_version 1534812 (0.0007) [2023-12-27 02:33:09,346][105620] Updated weights for policy 1, policy_version 1534822 (0.0009) [2023-12-27 02:33:09,839][105692] Updated weights for policy 0, policy_version 1531948 (0.0008) [2023-12-27 02:33:09,901][105692] Updated weights for policy 0, policy_version 1531958 (0.0009) [2023-12-27 02:33:09,972][105692] Updated weights for policy 0, policy_version 1531968 (0.0008) [2023-12-27 02:33:10,071][105620] Updated weights for policy 1, policy_version 1534832 (0.0008) [2023-12-27 02:33:10,135][105620] Updated weights for policy 1, policy_version 1534842 (0.0009) [2023-12-27 02:33:10,195][105620] Updated weights for policy 1, policy_version 1534852 (0.0009) [2023-12-27 02:33:10,683][105692] Updated weights for policy 0, policy_version 1531978 (0.0008) [2023-12-27 02:33:10,738][105692] Updated weights for policy 0, policy_version 1531988 (0.0009) [2023-12-27 02:33:10,789][105692] Updated weights for policy 0, policy_version 1531998 (0.0008) [2023-12-27 02:33:10,803][105620] Updated weights for policy 1, policy_version 1534862 (0.0009) [2023-12-27 02:33:10,842][105692] Updated weights for policy 0, policy_version 1532008 (0.0006) [2023-12-27 02:33:10,854][105620] Updated weights for policy 1, policy_version 1534872 (0.0007) [2023-12-27 02:33:10,909][105620] Updated weights for policy 1, policy_version 1534882 (0.0010) [2023-12-27 02:33:11,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 785235968. Throughput: 0: 9756.4, 1: 9996.8. Samples: 785240540. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:33:11,062][104569] Avg episode reward: [(0, '8900.807'), (1, '8992.576')] [2023-12-27 02:33:11,661][105692] Updated weights for policy 0, policy_version 1532018 (0.0010) [2023-12-27 02:33:11,692][105620] Updated weights for policy 1, policy_version 1534892 (0.0010) [2023-12-27 02:33:11,721][105692] Updated weights for policy 0, policy_version 1532028 (0.0007) [2023-12-27 02:33:11,758][105620] Updated weights for policy 1, policy_version 1534902 (0.0008) [2023-12-27 02:33:11,780][105692] Updated weights for policy 0, policy_version 1532038 (0.0007) [2023-12-27 02:33:11,824][105620] Updated weights for policy 1, policy_version 1534912 (0.0009) [2023-12-27 02:33:12,549][105620] Updated weights for policy 1, policy_version 1534922 (0.0006) [2023-12-27 02:33:12,563][105692] Updated weights for policy 0, policy_version 1532048 (0.0006) [2023-12-27 02:33:12,600][105620] Updated weights for policy 1, policy_version 1534932 (0.0008) [2023-12-27 02:33:12,615][105692] Updated weights for policy 0, policy_version 1532058 (0.0008) [2023-12-27 02:33:12,655][105620] Updated weights for policy 1, policy_version 1534942 (0.0006) [2023-12-27 02:33:12,668][105692] Updated weights for policy 0, policy_version 1532068 (0.0006) [2023-12-27 02:33:12,712][105620] Updated weights for policy 1, policy_version 1534952 (0.0008) [2023-12-27 02:33:13,284][105692] Updated weights for policy 0, policy_version 1532078 (0.0008) [2023-12-27 02:33:13,336][105692] Updated weights for policy 0, policy_version 1532088 (0.0009) [2023-12-27 02:33:13,388][105692] Updated weights for policy 0, policy_version 1532098 (0.0009) [2023-12-27 02:33:13,552][105620] Updated weights for policy 1, policy_version 1534963 (0.0010) [2023-12-27 02:33:13,607][105620] Updated weights for policy 1, policy_version 1534974 (0.0008) [2023-12-27 02:33:13,674][105620] Updated weights for policy 1, policy_version 1534984 (0.0005) [2023-12-27 02:33:14,087][105692] Updated weights for policy 0, policy_version 1532108 (0.0008) [2023-12-27 02:33:14,141][105692] Updated weights for policy 0, policy_version 1532118 (0.0010) [2023-12-27 02:33:14,193][105692] Updated weights for policy 0, policy_version 1532128 (0.0009) [2023-12-27 02:33:14,238][105620] Updated weights for policy 1, policy_version 1534994 (0.0005) [2023-12-27 02:33:14,288][105620] Updated weights for policy 1, policy_version 1535004 (0.0008) [2023-12-27 02:33:14,335][105620] Updated weights for policy 1, policy_version 1535014 (0.0009) [2023-12-27 02:33:14,861][105692] Updated weights for policy 0, policy_version 1532138 (0.0008) [2023-12-27 02:33:14,926][105692] Updated weights for policy 0, policy_version 1532148 (0.0006) [2023-12-27 02:33:14,985][105692] Updated weights for policy 0, policy_version 1532158 (0.0006) [2023-12-27 02:33:15,041][105692] Updated weights for policy 0, policy_version 1532168 (0.0006) [2023-12-27 02:33:15,173][105620] Updated weights for policy 1, policy_version 1535024 (0.0009) [2023-12-27 02:33:15,222][105620] Updated weights for policy 1, policy_version 1535034 (0.0009) [2023-12-27 02:33:15,270][105620] Updated weights for policy 1, policy_version 1535044 (0.0008) [2023-12-27 02:33:15,715][105692] Updated weights for policy 0, policy_version 1532178 (0.0005) [2023-12-27 02:33:15,781][105692] Updated weights for policy 0, policy_version 1532188 (0.0007) [2023-12-27 02:33:15,845][105692] Updated weights for policy 0, policy_version 1532198 (0.0007) [2023-12-27 02:33:16,017][105620] Updated weights for policy 1, policy_version 1535054 (0.0009) [2023-12-27 02:33:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 785326080. Throughput: 0: 9736.8, 1: 9964.8. Samples: 785297256. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:33:16,062][105620] Updated weights for policy 1, policy_version 1535064 (0.0008) [2023-12-27 02:33:16,063][104569] Avg episode reward: [(0, '8803.150'), (1, '8900.735')] [2023-12-27 02:33:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001532200_392298496.pth... [2023-12-27 02:33:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001531048_392003584.pth [2023-12-27 02:33:16,111][105620] Updated weights for policy 1, policy_version 1535074 (0.0010) [2023-12-27 02:33:16,145][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001535080_393035776.pth... [2023-12-27 02:33:16,148][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001533896_392732672.pth [2023-12-27 02:33:16,393][105692] Updated weights for policy 0, policy_version 1532208 (0.0007) [2023-12-27 02:33:16,456][105692] Updated weights for policy 0, policy_version 1532218 (0.0008) [2023-12-27 02:33:16,504][105692] Updated weights for policy 0, policy_version 1532228 (0.0007) [2023-12-27 02:33:16,785][105620] Updated weights for policy 1, policy_version 1535084 (0.0008) [2023-12-27 02:33:16,842][105620] Updated weights for policy 1, policy_version 1535094 (0.0009) [2023-12-27 02:33:16,901][105620] Updated weights for policy 1, policy_version 1535104 (0.0010) [2023-12-27 02:33:17,188][105692] Updated weights for policy 0, policy_version 1532238 (0.0010) [2023-12-27 02:33:17,238][105692] Updated weights for policy 0, policy_version 1532248 (0.0010) [2023-12-27 02:33:17,286][105692] Updated weights for policy 0, policy_version 1532258 (0.0010) [2023-12-27 02:33:17,593][105620] Updated weights for policy 1, policy_version 1535114 (0.0010) [2023-12-27 02:33:17,641][105620] Updated weights for policy 1, policy_version 1535124 (0.0008) [2023-12-27 02:33:17,697][105620] Updated weights for policy 1, policy_version 1535135 (0.0009) [2023-12-27 02:33:17,935][105692] Updated weights for policy 0, policy_version 1532268 (0.0008) [2023-12-27 02:33:17,993][105692] Updated weights for policy 0, policy_version 1532278 (0.0008) [2023-12-27 02:33:18,043][105692] Updated weights for policy 0, policy_version 1532288 (0.0011) [2023-12-27 02:33:18,516][105620] Updated weights for policy 1, policy_version 1535145 (0.0009) [2023-12-27 02:33:18,581][105620] Updated weights for policy 1, policy_version 1535155 (0.0010) [2023-12-27 02:33:18,646][105620] Updated weights for policy 1, policy_version 1535165 (0.0010) [2023-12-27 02:33:18,710][105620] Updated weights for policy 1, policy_version 1535175 (0.0010) [2023-12-27 02:33:18,748][105692] Updated weights for policy 0, policy_version 1532298 (0.0010) [2023-12-27 02:33:18,812][105692] Updated weights for policy 0, policy_version 1532308 (0.0010) [2023-12-27 02:33:18,871][105692] Updated weights for policy 0, policy_version 1532318 (0.0010) [2023-12-27 02:33:18,930][105692] Updated weights for policy 0, policy_version 1532328 (0.0010) [2023-12-27 02:33:19,395][105620] Updated weights for policy 1, policy_version 1535185 (0.0010) [2023-12-27 02:33:19,454][105620] Updated weights for policy 1, policy_version 1535195 (0.0010) [2023-12-27 02:33:19,514][105620] Updated weights for policy 1, policy_version 1535205 (0.0010) [2023-12-27 02:33:19,674][105692] Updated weights for policy 0, policy_version 1532338 (0.0006) [2023-12-27 02:33:19,744][105692] Updated weights for policy 0, policy_version 1532348 (0.0005) [2023-12-27 02:33:19,806][105692] Updated weights for policy 0, policy_version 1532358 (0.0006) [2023-12-27 02:33:20,314][105620] Updated weights for policy 1, policy_version 1535215 (0.0010) [2023-12-27 02:33:20,382][105620] Updated weights for policy 1, policy_version 1535225 (0.0007) [2023-12-27 02:33:20,412][105692] Updated weights for policy 0, policy_version 1532368 (0.0006) [2023-12-27 02:33:20,445][105620] Updated weights for policy 1, policy_version 1535235 (0.0008) [2023-12-27 02:33:20,460][105692] Updated weights for policy 0, policy_version 1532378 (0.0007) [2023-12-27 02:33:20,511][105692] Updated weights for policy 0, policy_version 1532388 (0.0010) [2023-12-27 02:33:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 785424384. Throughput: 0: 9780.0, 1: 9893.1. Samples: 785417060. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:33:21,063][104569] Avg episode reward: [(0, '8715.863'), (1, '8724.502')] [2023-12-27 02:33:21,152][105620] Updated weights for policy 1, policy_version 1535245 (0.0008) [2023-12-27 02:33:21,219][105620] Updated weights for policy 1, policy_version 1535255 (0.0006) [2023-12-27 02:33:21,282][105620] Updated weights for policy 1, policy_version 1535265 (0.0006) [2023-12-27 02:33:21,329][105692] Updated weights for policy 0, policy_version 1532398 (0.0008) [2023-12-27 02:33:21,400][105692] Updated weights for policy 0, policy_version 1532408 (0.0008) [2023-12-27 02:33:21,467][105692] Updated weights for policy 0, policy_version 1532418 (0.0007) [2023-12-27 02:33:22,010][105620] Updated weights for policy 1, policy_version 1535275 (0.0010) [2023-12-27 02:33:22,064][105620] Updated weights for policy 1, policy_version 1535285 (0.0011) [2023-12-27 02:33:22,126][105620] Updated weights for policy 1, policy_version 1535295 (0.0010) [2023-12-27 02:33:22,235][105692] Updated weights for policy 0, policy_version 1532428 (0.0008) [2023-12-27 02:33:22,293][105692] Updated weights for policy 0, policy_version 1532438 (0.0006) [2023-12-27 02:33:22,354][105692] Updated weights for policy 0, policy_version 1532448 (0.0008) [2023-12-27 02:33:22,868][105620] Updated weights for policy 1, policy_version 1535305 (0.0006) [2023-12-27 02:33:22,931][105620] Updated weights for policy 1, policy_version 1535315 (0.0009) [2023-12-27 02:33:22,981][105620] Updated weights for policy 1, policy_version 1535325 (0.0009) [2023-12-27 02:33:23,034][105620] Updated weights for policy 1, policy_version 1535335 (0.0009) [2023-12-27 02:33:23,083][105692] Updated weights for policy 0, policy_version 1532458 (0.0007) [2023-12-27 02:33:23,143][105692] Updated weights for policy 0, policy_version 1532468 (0.0009) [2023-12-27 02:33:23,205][105692] Updated weights for policy 0, policy_version 1532478 (0.0009) [2023-12-27 02:33:23,265][105692] Updated weights for policy 0, policy_version 1532488 (0.0009) [2023-12-27 02:33:23,801][105620] Updated weights for policy 1, policy_version 1535345 (0.0009) [2023-12-27 02:33:23,863][105620] Updated weights for policy 1, policy_version 1535355 (0.0005) [2023-12-27 02:33:23,872][105692] Updated weights for policy 0, policy_version 1532498 (0.0009) [2023-12-27 02:33:23,922][105620] Updated weights for policy 1, policy_version 1535365 (0.0005) [2023-12-27 02:33:23,935][105692] Updated weights for policy 0, policy_version 1532508 (0.0009) [2023-12-27 02:33:23,989][105692] Updated weights for policy 0, policy_version 1532518 (0.0010) [2023-12-27 02:33:24,516][105620] Updated weights for policy 1, policy_version 1535375 (0.0007) [2023-12-27 02:33:24,575][105620] Updated weights for policy 1, policy_version 1535385 (0.0008) [2023-12-27 02:33:24,642][105620] Updated weights for policy 1, policy_version 1535395 (0.0009) [2023-12-27 02:33:24,832][105692] Updated weights for policy 0, policy_version 1532528 (0.0010) [2023-12-27 02:33:24,890][105692] Updated weights for policy 0, policy_version 1532538 (0.0006) [2023-12-27 02:33:24,938][105692] Updated weights for policy 0, policy_version 1532548 (0.0009) [2023-12-27 02:33:25,384][105620] Updated weights for policy 1, policy_version 1535405 (0.0009) [2023-12-27 02:33:25,448][105620] Updated weights for policy 1, policy_version 1535415 (0.0008) [2023-12-27 02:33:25,509][105620] Updated weights for policy 1, policy_version 1535425 (0.0009) [2023-12-27 02:33:25,688][105692] Updated weights for policy 0, policy_version 1532558 (0.0009) [2023-12-27 02:33:25,744][105692] Updated weights for policy 0, policy_version 1532568 (0.0009) [2023-12-27 02:33:25,800][105692] Updated weights for policy 0, policy_version 1532578 (0.0009) [2023-12-27 02:33:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 785522688. Throughput: 0: 9800.7, 1: 9893.7. Samples: 785531812. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:33:26,063][104569] Avg episode reward: [(0, '8993.715'), (1, '8365.525')] [2023-12-27 02:33:26,179][105620] Updated weights for policy 1, policy_version 1535435 (0.0009) [2023-12-27 02:33:26,243][105620] Updated weights for policy 1, policy_version 1535445 (0.0008) [2023-12-27 02:33:26,303][105620] Updated weights for policy 1, policy_version 1535455 (0.0005) [2023-12-27 02:33:26,562][105692] Updated weights for policy 0, policy_version 1532588 (0.0009) [2023-12-27 02:33:26,615][105692] Updated weights for policy 0, policy_version 1532598 (0.0006) [2023-12-27 02:33:26,663][105692] Updated weights for policy 0, policy_version 1532608 (0.0005) [2023-12-27 02:33:26,982][105620] Updated weights for policy 1, policy_version 1535465 (0.0006) [2023-12-27 02:33:27,043][105620] Updated weights for policy 1, policy_version 1535475 (0.0008) [2023-12-27 02:33:27,102][105620] Updated weights for policy 1, policy_version 1535485 (0.0006) [2023-12-27 02:33:27,161][105620] Updated weights for policy 1, policy_version 1535495 (0.0010) [2023-12-27 02:33:27,341][105692] Updated weights for policy 0, policy_version 1532618 (0.0009) [2023-12-27 02:33:27,392][105692] Updated weights for policy 0, policy_version 1532628 (0.0009) [2023-12-27 02:33:27,447][105692] Updated weights for policy 0, policy_version 1532638 (0.0012) [2023-12-27 02:33:27,828][105620] Updated weights for policy 1, policy_version 1535505 (0.0009) [2023-12-27 02:33:27,876][105620] Updated weights for policy 1, policy_version 1535515 (0.0005) [2023-12-27 02:33:27,933][105620] Updated weights for policy 1, policy_version 1535525 (0.0005) [2023-12-27 02:33:28,268][105692] Updated weights for policy 0, policy_version 1532649 (0.0010) [2023-12-27 02:33:28,317][105692] Updated weights for policy 0, policy_version 1532659 (0.0008) [2023-12-27 02:33:28,378][105692] Updated weights for policy 0, policy_version 1532669 (0.0009) [2023-12-27 02:33:28,430][105692] Updated weights for policy 0, policy_version 1532679 (0.0009) [2023-12-27 02:33:28,568][105620] Updated weights for policy 1, policy_version 1535535 (0.0008) [2023-12-27 02:33:28,627][105620] Updated weights for policy 1, policy_version 1535545 (0.0008) [2023-12-27 02:33:28,685][105620] Updated weights for policy 1, policy_version 1535555 (0.0009) [2023-12-27 02:33:29,228][105692] Updated weights for policy 0, policy_version 1532689 (0.0007) [2023-12-27 02:33:29,288][105692] Updated weights for policy 0, policy_version 1532699 (0.0008) [2023-12-27 02:33:29,352][105692] Updated weights for policy 0, policy_version 1532709 (0.0008) [2023-12-27 02:33:29,402][105620] Updated weights for policy 1, policy_version 1535565 (0.0007) [2023-12-27 02:33:29,471][105620] Updated weights for policy 1, policy_version 1535575 (0.0005) [2023-12-27 02:33:29,532][105620] Updated weights for policy 1, policy_version 1535585 (0.0005) [2023-12-27 02:33:30,104][105620] Updated weights for policy 1, policy_version 1535595 (0.0006) [2023-12-27 02:33:30,150][105692] Updated weights for policy 0, policy_version 1532719 (0.0008) [2023-12-27 02:33:30,155][105620] Updated weights for policy 1, policy_version 1535605 (0.0008) [2023-12-27 02:33:30,202][105692] Updated weights for policy 0, policy_version 1532729 (0.0006) [2023-12-27 02:33:30,204][105620] Updated weights for policy 1, policy_version 1535615 (0.0010) [2023-12-27 02:33:30,262][105692] Updated weights for policy 0, policy_version 1532739 (0.0005) [2023-12-27 02:33:30,858][105692] Updated weights for policy 0, policy_version 1532749 (0.0007) [2023-12-27 02:33:30,901][105692] Updated weights for policy 0, policy_version 1532759 (0.0005) [2023-12-27 02:33:30,946][105620] Updated weights for policy 1, policy_version 1535625 (0.0011) [2023-12-27 02:33:30,954][105692] Updated weights for policy 0, policy_version 1532769 (0.0008) [2023-12-27 02:33:30,993][105620] Updated weights for policy 1, policy_version 1535635 (0.0010) [2023-12-27 02:33:31,048][105620] Updated weights for policy 1, policy_version 1535645 (0.0011) [2023-12-27 02:33:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 785620992. Throughput: 0: 9758.1, 1: 9962.6. Samples: 785590524. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:33:31,062][104569] Avg episode reward: [(0, '8993.360'), (1, '8811.286')] [2023-12-27 02:33:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001532776_392445952.pth... [2023-12-27 02:33:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001531624_392151040.pth [2023-12-27 02:33:31,118][105620] Updated weights for policy 1, policy_version 1535655 (0.0011) [2023-12-27 02:33:31,124][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001535656_393183232.pth... [2023-12-27 02:33:31,129][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001534472_392880128.pth [2023-12-27 02:33:31,675][105692] Updated weights for policy 0, policy_version 1532779 (0.0007) [2023-12-27 02:33:31,738][105692] Updated weights for policy 0, policy_version 1532789 (0.0008) [2023-12-27 02:33:31,795][105620] Updated weights for policy 1, policy_version 1535665 (0.0011) [2023-12-27 02:33:31,797][105692] Updated weights for policy 0, policy_version 1532799 (0.0008) [2023-12-27 02:33:31,854][105620] Updated weights for policy 1, policy_version 1535675 (0.0011) [2023-12-27 02:33:31,914][105620] Updated weights for policy 1, policy_version 1535685 (0.0010) [2023-12-27 02:33:32,552][105692] Updated weights for policy 0, policy_version 1532809 (0.0008) [2023-12-27 02:33:32,618][105692] Updated weights for policy 0, policy_version 1532819 (0.0005) [2023-12-27 02:33:32,666][105620] Updated weights for policy 1, policy_version 1535695 (0.0010) [2023-12-27 02:33:32,673][105692] Updated weights for policy 0, policy_version 1532829 (0.0006) [2023-12-27 02:33:32,719][105620] Updated weights for policy 1, policy_version 1535705 (0.0011) [2023-12-27 02:33:32,732][105692] Updated weights for policy 0, policy_version 1532839 (0.0007) [2023-12-27 02:33:32,778][105620] Updated weights for policy 1, policy_version 1535715 (0.0011) [2023-12-27 02:33:33,440][105620] Updated weights for policy 1, policy_version 1535725 (0.0011) [2023-12-27 02:33:33,473][105692] Updated weights for policy 0, policy_version 1532849 (0.0007) [2023-12-27 02:33:33,498][105620] Updated weights for policy 1, policy_version 1535735 (0.0010) [2023-12-27 02:33:33,527][105692] Updated weights for policy 0, policy_version 1532859 (0.0005) [2023-12-27 02:33:33,563][105620] Updated weights for policy 1, policy_version 1535745 (0.0010) [2023-12-27 02:33:33,577][105692] Updated weights for policy 0, policy_version 1532869 (0.0005) [2023-12-27 02:33:34,252][105692] Updated weights for policy 0, policy_version 1532879 (0.0008) [2023-12-27 02:33:34,306][105620] Updated weights for policy 1, policy_version 1535755 (0.0010) [2023-12-27 02:33:34,325][105692] Updated weights for policy 0, policy_version 1532889 (0.0009) [2023-12-27 02:33:34,369][105620] Updated weights for policy 1, policy_version 1535765 (0.0006) [2023-12-27 02:33:34,392][105692] Updated weights for policy 0, policy_version 1532899 (0.0008) [2023-12-27 02:33:34,435][105620] Updated weights for policy 1, policy_version 1535775 (0.0006) [2023-12-27 02:33:35,007][105692] Updated weights for policy 0, policy_version 1532909 (0.0008) [2023-12-27 02:33:35,066][105692] Updated weights for policy 0, policy_version 1532919 (0.0010) [2023-12-27 02:33:35,101][105620] Updated weights for policy 1, policy_version 1535785 (0.0008) [2023-12-27 02:33:35,118][105692] Updated weights for policy 0, policy_version 1532929 (0.0008) [2023-12-27 02:33:35,163][105620] Updated weights for policy 1, policy_version 1535795 (0.0009) [2023-12-27 02:33:35,210][105620] Updated weights for policy 1, policy_version 1535805 (0.0005) [2023-12-27 02:33:35,259][105620] Updated weights for policy 1, policy_version 1535815 (0.0008) [2023-12-27 02:33:35,752][105692] Updated weights for policy 0, policy_version 1532939 (0.0007) [2023-12-27 02:33:35,808][105692] Updated weights for policy 0, policy_version 1532949 (0.0009) [2023-12-27 02:33:35,834][105620] Updated weights for policy 1, policy_version 1535825 (0.0006) [2023-12-27 02:33:35,860][105692] Updated weights for policy 0, policy_version 1532959 (0.0008) [2023-12-27 02:33:35,896][105620] Updated weights for policy 1, policy_version 1535835 (0.0006) [2023-12-27 02:33:35,943][105620] Updated weights for policy 1, policy_version 1535845 (0.0010) [2023-12-27 02:33:36,062][104569] Fps is (10 sec: 20480.7, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 785727488. Throughput: 0: 9720.0, 1: 9948.5. Samples: 785708596. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:33:36,062][104569] Avg episode reward: [(0, '8726.710'), (1, '9083.206')] [2023-12-27 02:33:36,598][105620] Updated weights for policy 1, policy_version 1535855 (0.0007) [2023-12-27 02:33:36,655][105620] Updated weights for policy 1, policy_version 1535865 (0.0008) [2023-12-27 02:33:36,713][105692] Updated weights for policy 0, policy_version 1532969 (0.0007) [2023-12-27 02:33:36,714][105620] Updated weights for policy 1, policy_version 1535875 (0.0007) [2023-12-27 02:33:36,778][105692] Updated weights for policy 0, policy_version 1532979 (0.0007) [2023-12-27 02:33:36,838][105692] Updated weights for policy 0, policy_version 1532989 (0.0008) [2023-12-27 02:33:36,897][105692] Updated weights for policy 0, policy_version 1532999 (0.0008) [2023-12-27 02:33:37,322][105620] Updated weights for policy 1, policy_version 1535885 (0.0011) [2023-12-27 02:33:37,387][105620] Updated weights for policy 1, policy_version 1535895 (0.0008) [2023-12-27 02:33:37,445][105620] Updated weights for policy 1, policy_version 1535905 (0.0007) [2023-12-27 02:33:37,751][105692] Updated weights for policy 0, policy_version 1533009 (0.0008) [2023-12-27 02:33:37,821][105692] Updated weights for policy 0, policy_version 1533019 (0.0008) [2023-12-27 02:33:37,884][105692] Updated weights for policy 0, policy_version 1533029 (0.0008) [2023-12-27 02:33:38,145][105620] Updated weights for policy 1, policy_version 1535915 (0.0011) [2023-12-27 02:33:38,210][105620] Updated weights for policy 1, policy_version 1535925 (0.0007) [2023-12-27 02:33:38,265][105620] Updated weights for policy 1, policy_version 1535935 (0.0006) [2023-12-27 02:33:38,666][105692] Updated weights for policy 0, policy_version 1533039 (0.0008) [2023-12-27 02:33:38,737][105692] Updated weights for policy 0, policy_version 1533049 (0.0008) [2023-12-27 02:33:38,800][105692] Updated weights for policy 0, policy_version 1533059 (0.0008) [2023-12-27 02:33:38,940][105620] Updated weights for policy 1, policy_version 1535945 (0.0007) [2023-12-27 02:33:38,995][105620] Updated weights for policy 1, policy_version 1535955 (0.0005) [2023-12-27 02:33:39,049][105620] Updated weights for policy 1, policy_version 1535965 (0.0010) [2023-12-27 02:33:39,106][105620] Updated weights for policy 1, policy_version 1535975 (0.0007) [2023-12-27 02:33:39,610][105692] Updated weights for policy 0, policy_version 1533069 (0.0009) [2023-12-27 02:33:39,680][105692] Updated weights for policy 0, policy_version 1533079 (0.0009) [2023-12-27 02:33:39,747][105692] Updated weights for policy 0, policy_version 1533089 (0.0009) [2023-12-27 02:33:39,823][105620] Updated weights for policy 1, policy_version 1535985 (0.0008) [2023-12-27 02:33:39,887][105620] Updated weights for policy 1, policy_version 1535995 (0.0007) [2023-12-27 02:33:39,959][105620] Updated weights for policy 1, policy_version 1536005 (0.0009) [2023-12-27 02:33:40,571][105692] Updated weights for policy 0, policy_version 1533099 (0.0008) [2023-12-27 02:33:40,616][105620] Updated weights for policy 1, policy_version 1536015 (0.0007) [2023-12-27 02:33:40,635][105692] Updated weights for policy 0, policy_version 1533109 (0.0008) [2023-12-27 02:33:40,678][105620] Updated weights for policy 1, policy_version 1536025 (0.0007) [2023-12-27 02:33:40,696][105692] Updated weights for policy 0, policy_version 1533119 (0.0007) [2023-12-27 02:33:40,735][105620] Updated weights for policy 1, policy_version 1536035 (0.0007) [2023-12-27 02:33:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 785817600. Throughput: 0: 9621.2, 1: 10069.7. Samples: 785825844. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:33:41,062][104569] Avg episode reward: [(0, '8541.043'), (1, '9082.940')] [2023-12-27 02:33:41,397][105620] Updated weights for policy 1, policy_version 1536045 (0.0009) [2023-12-27 02:33:41,453][105620] Updated weights for policy 1, policy_version 1536055 (0.0009) [2023-12-27 02:33:41,506][105692] Updated weights for policy 0, policy_version 1533129 (0.0006) [2023-12-27 02:33:41,510][105620] Updated weights for policy 1, policy_version 1536065 (0.0009) [2023-12-27 02:33:41,570][105692] Updated weights for policy 0, policy_version 1533139 (0.0006) [2023-12-27 02:33:41,635][105692] Updated weights for policy 0, policy_version 1533149 (0.0007) [2023-12-27 02:33:41,698][105692] Updated weights for policy 0, policy_version 1533159 (0.0010) [2023-12-27 02:33:42,292][105620] Updated weights for policy 1, policy_version 1536075 (0.0009) [2023-12-27 02:33:42,355][105620] Updated weights for policy 1, policy_version 1536085 (0.0008) [2023-12-27 02:33:42,395][105692] Updated weights for policy 0, policy_version 1533169 (0.0008) [2023-12-27 02:33:42,415][105620] Updated weights for policy 1, policy_version 1536095 (0.0006) [2023-12-27 02:33:42,461][105692] Updated weights for policy 0, policy_version 1533179 (0.0008) [2023-12-27 02:33:42,519][105692] Updated weights for policy 0, policy_version 1533189 (0.0006) [2023-12-27 02:33:43,142][105620] Updated weights for policy 1, policy_version 1536105 (0.0006) [2023-12-27 02:33:43,144][105692] Updated weights for policy 0, policy_version 1533199 (0.0008) [2023-12-27 02:33:43,194][105620] Updated weights for policy 1, policy_version 1536115 (0.0007) [2023-12-27 02:33:43,215][105692] Updated weights for policy 0, policy_version 1533209 (0.0008) [2023-12-27 02:33:43,239][105620] Updated weights for policy 1, policy_version 1536125 (0.0009) [2023-12-27 02:33:43,277][105692] Updated weights for policy 0, policy_version 1533219 (0.0008) [2023-12-27 02:33:43,287][105620] Updated weights for policy 1, policy_version 1536135 (0.0005) [2023-12-27 02:33:43,977][105620] Updated weights for policy 1, policy_version 1536145 (0.0005) [2023-12-27 02:33:44,012][105692] Updated weights for policy 0, policy_version 1533229 (0.0009) [2023-12-27 02:33:44,036][105620] Updated weights for policy 1, policy_version 1536155 (0.0005) [2023-12-27 02:33:44,061][105692] Updated weights for policy 0, policy_version 1533239 (0.0009) [2023-12-27 02:33:44,089][105620] Updated weights for policy 1, policy_version 1536165 (0.0005) [2023-12-27 02:33:44,110][105692] Updated weights for policy 0, policy_version 1533249 (0.0008) [2023-12-27 02:33:44,800][105692] Updated weights for policy 0, policy_version 1533259 (0.0009) [2023-12-27 02:33:44,820][105620] Updated weights for policy 1, policy_version 1536175 (0.0007) [2023-12-27 02:33:44,862][105692] Updated weights for policy 0, policy_version 1533269 (0.0006) [2023-12-27 02:33:44,883][105620] Updated weights for policy 1, policy_version 1536185 (0.0009) [2023-12-27 02:33:44,928][105692] Updated weights for policy 0, policy_version 1533279 (0.0006) [2023-12-27 02:33:44,943][105620] Updated weights for policy 1, policy_version 1536195 (0.0009) [2023-12-27 02:33:45,465][105692] Updated weights for policy 0, policy_version 1533289 (0.0006) [2023-12-27 02:33:45,534][105692] Updated weights for policy 0, policy_version 1533299 (0.0005) [2023-12-27 02:33:45,590][105692] Updated weights for policy 0, policy_version 1533309 (0.0006) [2023-12-27 02:33:45,641][105692] Updated weights for policy 0, policy_version 1533319 (0.0008) [2023-12-27 02:33:45,767][105620] Updated weights for policy 1, policy_version 1536205 (0.0008) [2023-12-27 02:33:45,834][105620] Updated weights for policy 1, policy_version 1536215 (0.0005) [2023-12-27 02:33:45,890][105620] Updated weights for policy 1, policy_version 1536225 (0.0006) [2023-12-27 02:33:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 785915904. Throughput: 0: 9591.3, 1: 10061.8. Samples: 785883632. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:33:46,062][104569] Avg episode reward: [(0, '8632.277'), (1, '9172.303')] [2023-12-27 02:33:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001533320_392585216.pth... [2023-12-27 02:33:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001536232_393330688.pth... [2023-12-27 02:33:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001532200_392298496.pth [2023-12-27 02:33:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001535080_393035776.pth [2023-12-27 02:33:46,300][105692] Updated weights for policy 0, policy_version 1533329 (0.0005) [2023-12-27 02:33:46,362][105692] Updated weights for policy 0, policy_version 1533339 (0.0005) [2023-12-27 02:33:46,422][105692] Updated weights for policy 0, policy_version 1533349 (0.0007) [2023-12-27 02:33:46,569][105620] Updated weights for policy 1, policy_version 1536235 (0.0006) [2023-12-27 02:33:46,623][105620] Updated weights for policy 1, policy_version 1536245 (0.0005) [2023-12-27 02:33:46,668][105620] Updated weights for policy 1, policy_version 1536255 (0.0005) [2023-12-27 02:33:47,151][105692] Updated weights for policy 0, policy_version 1533359 (0.0009) [2023-12-27 02:33:47,208][105692] Updated weights for policy 0, policy_version 1533369 (0.0009) [2023-12-27 02:33:47,264][105692] Updated weights for policy 0, policy_version 1533379 (0.0009) [2023-12-27 02:33:47,325][105620] Updated weights for policy 1, policy_version 1536265 (0.0005) [2023-12-27 02:33:47,377][105620] Updated weights for policy 1, policy_version 1536275 (0.0005) [2023-12-27 02:33:47,431][105620] Updated weights for policy 1, policy_version 1536285 (0.0007) [2023-12-27 02:33:47,485][105620] Updated weights for policy 1, policy_version 1536295 (0.0009) [2023-12-27 02:33:47,922][105692] Updated weights for policy 0, policy_version 1533389 (0.0010) [2023-12-27 02:33:47,976][105692] Updated weights for policy 0, policy_version 1533399 (0.0008) [2023-12-27 02:33:48,040][105692] Updated weights for policy 0, policy_version 1533409 (0.0007) [2023-12-27 02:33:48,099][105620] Updated weights for policy 1, policy_version 1536305 (0.0007) [2023-12-27 02:33:48,149][105620] Updated weights for policy 1, policy_version 1536315 (0.0008) [2023-12-27 02:33:48,200][105620] Updated weights for policy 1, policy_version 1536325 (0.0009) [2023-12-27 02:33:48,824][105692] Updated weights for policy 0, policy_version 1533419 (0.0007) [2023-12-27 02:33:48,856][105620] Updated weights for policy 1, policy_version 1536335 (0.0007) [2023-12-27 02:33:48,892][105692] Updated weights for policy 0, policy_version 1533429 (0.0007) [2023-12-27 02:33:48,923][105620] Updated weights for policy 1, policy_version 1536345 (0.0006) [2023-12-27 02:33:48,943][105692] Updated weights for policy 0, policy_version 1533439 (0.0009) [2023-12-27 02:33:48,979][105620] Updated weights for policy 1, policy_version 1536355 (0.0007) [2023-12-27 02:33:49,585][105620] Updated weights for policy 1, policy_version 1536365 (0.0007) [2023-12-27 02:33:49,647][105620] Updated weights for policy 1, policy_version 1536375 (0.0009) [2023-12-27 02:33:49,705][105620] Updated weights for policy 1, policy_version 1536385 (0.0009) [2023-12-27 02:33:49,774][105692] Updated weights for policy 0, policy_version 1533449 (0.0007) [2023-12-27 02:33:49,843][105692] Updated weights for policy 0, policy_version 1533459 (0.0009) [2023-12-27 02:33:49,911][105692] Updated weights for policy 0, policy_version 1533469 (0.0009) [2023-12-27 02:33:49,974][105692] Updated weights for policy 0, policy_version 1533479 (0.0009) [2023-12-27 02:33:50,472][105620] Updated weights for policy 1, policy_version 1536395 (0.0009) [2023-12-27 02:33:50,527][105620] Updated weights for policy 1, policy_version 1536405 (0.0009) [2023-12-27 02:33:50,583][105620] Updated weights for policy 1, policy_version 1536415 (0.0009) [2023-12-27 02:33:50,716][105692] Updated weights for policy 0, policy_version 1533489 (0.0009) [2023-12-27 02:33:50,772][105692] Updated weights for policy 0, policy_version 1533499 (0.0009) [2023-12-27 02:33:50,824][105692] Updated weights for policy 0, policy_version 1533509 (0.0010) [2023-12-27 02:33:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 786014208. Throughput: 0: 9576.8, 1: 10006.8. Samples: 786003072. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:33:51,063][104569] Avg episode reward: [(0, '8719.846'), (1, '9174.247')] [2023-12-27 02:33:51,301][105620] Updated weights for policy 1, policy_version 1536425 (0.0009) [2023-12-27 02:33:51,359][105620] Updated weights for policy 1, policy_version 1536435 (0.0008) [2023-12-27 02:33:51,427][105620] Updated weights for policy 1, policy_version 1536445 (0.0008) [2023-12-27 02:33:51,482][105620] Updated weights for policy 1, policy_version 1536455 (0.0008) [2023-12-27 02:33:51,635][105692] Updated weights for policy 0, policy_version 1533519 (0.0009) [2023-12-27 02:33:51,700][105692] Updated weights for policy 0, policy_version 1533529 (0.0007) [2023-12-27 02:33:51,771][105692] Updated weights for policy 0, policy_version 1533539 (0.0010) [2023-12-27 02:33:52,156][105620] Updated weights for policy 1, policy_version 1536465 (0.0006) [2023-12-27 02:33:52,230][105620] Updated weights for policy 1, policy_version 1536475 (0.0010) [2023-12-27 02:33:52,295][105620] Updated weights for policy 1, policy_version 1536485 (0.0008) [2023-12-27 02:33:52,390][105692] Updated weights for policy 0, policy_version 1533549 (0.0008) [2023-12-27 02:33:52,443][105692] Updated weights for policy 0, policy_version 1533559 (0.0008) [2023-12-27 02:33:52,514][105692] Updated weights for policy 0, policy_version 1533569 (0.0010) [2023-12-27 02:33:52,856][105620] Updated weights for policy 1, policy_version 1536495 (0.0009) [2023-12-27 02:33:52,919][105620] Updated weights for policy 1, policy_version 1536505 (0.0008) [2023-12-27 02:33:52,990][105620] Updated weights for policy 1, policy_version 1536515 (0.0005) [2023-12-27 02:33:53,215][105692] Updated weights for policy 0, policy_version 1533579 (0.0009) [2023-12-27 02:33:53,274][105692] Updated weights for policy 0, policy_version 1533589 (0.0006) [2023-12-27 02:33:53,335][105692] Updated weights for policy 0, policy_version 1533599 (0.0006) [2023-12-27 02:33:53,627][105620] Updated weights for policy 1, policy_version 1536525 (0.0008) [2023-12-27 02:33:53,687][105620] Updated weights for policy 1, policy_version 1536535 (0.0010) [2023-12-27 02:33:53,745][105620] Updated weights for policy 1, policy_version 1536545 (0.0010) [2023-12-27 02:33:54,064][105692] Updated weights for policy 0, policy_version 1533609 (0.0009) [2023-12-27 02:33:54,125][105692] Updated weights for policy 0, policy_version 1533619 (0.0010) [2023-12-27 02:33:54,178][105692] Updated weights for policy 0, policy_version 1533629 (0.0007) [2023-12-27 02:33:54,234][105692] Updated weights for policy 0, policy_version 1533639 (0.0005) [2023-12-27 02:33:54,374][105620] Updated weights for policy 1, policy_version 1536555 (0.0007) [2023-12-27 02:33:54,423][105620] Updated weights for policy 1, policy_version 1536565 (0.0005) [2023-12-27 02:33:54,483][105620] Updated weights for policy 1, policy_version 1536575 (0.0007) [2023-12-27 02:33:54,849][105692] Updated weights for policy 0, policy_version 1533649 (0.0005) [2023-12-27 02:33:54,910][105692] Updated weights for policy 0, policy_version 1533659 (0.0005) [2023-12-27 02:33:54,976][105692] Updated weights for policy 0, policy_version 1533669 (0.0006) [2023-12-27 02:33:55,195][105620] Updated weights for policy 1, policy_version 1536585 (0.0010) [2023-12-27 02:33:55,254][105620] Updated weights for policy 1, policy_version 1536595 (0.0010) [2023-12-27 02:33:55,315][105620] Updated weights for policy 1, policy_version 1536605 (0.0010) [2023-12-27 02:33:55,373][105620] Updated weights for policy 1, policy_version 1536615 (0.0011) [2023-12-27 02:33:55,551][105692] Updated weights for policy 0, policy_version 1533679 (0.0008) [2023-12-27 02:33:55,603][105692] Updated weights for policy 0, policy_version 1533689 (0.0009) [2023-12-27 02:33:55,669][105692] Updated weights for policy 0, policy_version 1533699 (0.0007) [2023-12-27 02:33:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 786112512. Throughput: 0: 9674.9, 1: 9951.6. Samples: 786123736. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:33:56,063][104569] Avg episode reward: [(0, '8448.404'), (1, '8900.800')] [2023-12-27 02:33:56,148][105620] Updated weights for policy 1, policy_version 1536625 (0.0010) [2023-12-27 02:33:56,209][105620] Updated weights for policy 1, policy_version 1536635 (0.0010) [2023-12-27 02:33:56,277][105620] Updated weights for policy 1, policy_version 1536645 (0.0010) [2023-12-27 02:33:56,313][105692] Updated weights for policy 0, policy_version 1533709 (0.0005) [2023-12-27 02:33:56,363][105692] Updated weights for policy 0, policy_version 1533719 (0.0005) [2023-12-27 02:33:56,412][105692] Updated weights for policy 0, policy_version 1533729 (0.0007) [2023-12-27 02:33:56,910][105620] Updated weights for policy 1, policy_version 1536655 (0.0007) [2023-12-27 02:33:56,958][105620] Updated weights for policy 1, policy_version 1536665 (0.0010) [2023-12-27 02:33:57,020][105620] Updated weights for policy 1, policy_version 1536675 (0.0010) [2023-12-27 02:33:57,219][105692] Updated weights for policy 0, policy_version 1533739 (0.0008) [2023-12-27 02:33:57,272][105692] Updated weights for policy 0, policy_version 1533749 (0.0010) [2023-12-27 02:33:57,320][105692] Updated weights for policy 0, policy_version 1533759 (0.0007) [2023-12-27 02:33:57,697][105620] Updated weights for policy 1, policy_version 1536685 (0.0009) [2023-12-27 02:33:57,765][105620] Updated weights for policy 1, policy_version 1536695 (0.0005) [2023-12-27 02:33:57,816][105620] Updated weights for policy 1, policy_version 1536705 (0.0009) [2023-12-27 02:33:57,903][105692] Updated weights for policy 0, policy_version 1533769 (0.0008) [2023-12-27 02:33:57,958][105692] Updated weights for policy 0, policy_version 1533779 (0.0005) [2023-12-27 02:33:58,011][105692] Updated weights for policy 0, policy_version 1533789 (0.0006) [2023-12-27 02:33:58,054][105692] Updated weights for policy 0, policy_version 1533799 (0.0008) [2023-12-27 02:33:58,564][105620] Updated weights for policy 1, policy_version 1536715 (0.0009) [2023-12-27 02:33:58,627][105620] Updated weights for policy 1, policy_version 1536725 (0.0008) [2023-12-27 02:33:58,690][105620] Updated weights for policy 1, policy_version 1536735 (0.0008) [2023-12-27 02:33:58,854][105692] Updated weights for policy 0, policy_version 1533809 (0.0007) [2023-12-27 02:33:58,919][105692] Updated weights for policy 0, policy_version 1533819 (0.0009) [2023-12-27 02:33:58,985][105692] Updated weights for policy 0, policy_version 1533829 (0.0009) [2023-12-27 02:33:59,578][105620] Updated weights for policy 1, policy_version 1536745 (0.0009) [2023-12-27 02:33:59,641][105620] Updated weights for policy 1, policy_version 1536755 (0.0006) [2023-12-27 02:33:59,642][105692] Updated weights for policy 0, policy_version 1533839 (0.0008) [2023-12-27 02:33:59,699][105692] Updated weights for policy 0, policy_version 1533849 (0.0009) [2023-12-27 02:33:59,699][105620] Updated weights for policy 1, policy_version 1536765 (0.0005) [2023-12-27 02:33:59,753][105620] Updated weights for policy 1, policy_version 1536775 (0.0005) [2023-12-27 02:33:59,755][105692] Updated weights for policy 0, policy_version 1533859 (0.0008) [2023-12-27 02:34:00,396][105620] Updated weights for policy 1, policy_version 1536785 (0.0007) [2023-12-27 02:34:00,441][105620] Updated weights for policy 1, policy_version 1536795 (0.0010) [2023-12-27 02:34:00,487][105620] Updated weights for policy 1, policy_version 1536805 (0.0009) [2023-12-27 02:34:00,548][105692] Updated weights for policy 0, policy_version 1533869 (0.0009) [2023-12-27 02:34:00,598][105692] Updated weights for policy 0, policy_version 1533879 (0.0008) [2023-12-27 02:34:00,652][105692] Updated weights for policy 0, policy_version 1533889 (0.0008) [2023-12-27 02:34:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 786210816. Throughput: 0: 9711.6, 1: 9973.3. Samples: 786183072. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:34:01,065][104569] Avg episode reward: [(0, '8446.704'), (1, '8995.233')] [2023-12-27 02:34:01,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001533896_392732672.pth... [2023-12-27 02:34:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001536808_393478144.pth... [2023-12-27 02:34:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001535656_393183232.pth [2023-12-27 02:34:01,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001532776_392445952.pth [2023-12-27 02:34:01,137][105620] Updated weights for policy 1, policy_version 1536815 (0.0009) [2023-12-27 02:34:01,192][105620] Updated weights for policy 1, policy_version 1536825 (0.0010) [2023-12-27 02:34:01,245][105620] Updated weights for policy 1, policy_version 1536835 (0.0010) [2023-12-27 02:34:01,488][105692] Updated weights for policy 0, policy_version 1533899 (0.0008) [2023-12-27 02:34:01,546][105692] Updated weights for policy 0, policy_version 1533909 (0.0010) [2023-12-27 02:34:01,601][105692] Updated weights for policy 0, policy_version 1533920 (0.0010) [2023-12-27 02:34:01,919][105620] Updated weights for policy 1, policy_version 1536845 (0.0010) [2023-12-27 02:34:01,978][105620] Updated weights for policy 1, policy_version 1536855 (0.0011) [2023-12-27 02:34:02,040][105620] Updated weights for policy 1, policy_version 1536865 (0.0011) [2023-12-27 02:34:02,407][105692] Updated weights for policy 0, policy_version 1533930 (0.0008) [2023-12-27 02:34:02,465][105692] Updated weights for policy 0, policy_version 1533940 (0.0010) [2023-12-27 02:34:02,528][105692] Updated weights for policy 0, policy_version 1533950 (0.0010) [2023-12-27 02:34:02,581][105692] Updated weights for policy 0, policy_version 1533960 (0.0009) [2023-12-27 02:34:02,658][105620] Updated weights for policy 1, policy_version 1536875 (0.0009) [2023-12-27 02:34:02,707][105620] Updated weights for policy 1, policy_version 1536885 (0.0005) [2023-12-27 02:34:02,758][105620] Updated weights for policy 1, policy_version 1536895 (0.0005) [2023-12-27 02:34:03,300][105620] Updated weights for policy 1, policy_version 1536905 (0.0006) [2023-12-27 02:34:03,355][105620] Updated weights for policy 1, policy_version 1536915 (0.0011) [2023-12-27 02:34:03,413][105620] Updated weights for policy 1, policy_version 1536925 (0.0011) [2023-12-27 02:34:03,460][105692] Updated weights for policy 0, policy_version 1533970 (0.0007) [2023-12-27 02:34:03,482][105620] Updated weights for policy 1, policy_version 1536935 (0.0010) [2023-12-27 02:34:03,510][105692] Updated weights for policy 0, policy_version 1533980 (0.0007) [2023-12-27 02:34:03,559][105692] Updated weights for policy 0, policy_version 1533990 (0.0008) [2023-12-27 02:34:04,253][105620] Updated weights for policy 1, policy_version 1536945 (0.0010) [2023-12-27 02:34:04,313][105692] Updated weights for policy 0, policy_version 1534000 (0.0006) [2023-12-27 02:34:04,320][105620] Updated weights for policy 1, policy_version 1536955 (0.0009) [2023-12-27 02:34:04,377][105692] Updated weights for policy 0, policy_version 1534010 (0.0007) [2023-12-27 02:34:04,379][105620] Updated weights for policy 1, policy_version 1536965 (0.0008) [2023-12-27 02:34:04,441][105692] Updated weights for policy 0, policy_version 1534020 (0.0007) [2023-12-27 02:34:05,015][105692] Updated weights for policy 0, policy_version 1534030 (0.0007) [2023-12-27 02:34:05,067][105692] Updated weights for policy 0, policy_version 1534040 (0.0005) [2023-12-27 02:34:05,125][105692] Updated weights for policy 0, policy_version 1534050 (0.0007) [2023-12-27 02:34:05,140][105620] Updated weights for policy 1, policy_version 1536975 (0.0009) [2023-12-27 02:34:05,194][105620] Updated weights for policy 1, policy_version 1536985 (0.0008) [2023-12-27 02:34:05,247][105620] Updated weights for policy 1, policy_version 1536995 (0.0010) [2023-12-27 02:34:05,776][105692] Updated weights for policy 0, policy_version 1534060 (0.0008) [2023-12-27 02:34:05,820][105692] Updated weights for policy 0, policy_version 1534070 (0.0008) [2023-12-27 02:34:05,850][105620] Updated weights for policy 1, policy_version 1537005 (0.0010) [2023-12-27 02:34:05,869][105692] Updated weights for policy 0, policy_version 1534080 (0.0008) [2023-12-27 02:34:05,910][105620] Updated weights for policy 1, policy_version 1537015 (0.0009) [2023-12-27 02:34:05,962][105620] Updated weights for policy 1, policy_version 1537025 (0.0010) [2023-12-27 02:34:06,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 786317312. Throughput: 0: 9544.3, 1: 10022.5. Samples: 786297568. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:34:06,063][104569] Avg episode reward: [(0, '8718.067'), (1, '8996.463')] [2023-12-27 02:34:06,588][105692] Updated weights for policy 0, policy_version 1534090 (0.0008) [2023-12-27 02:34:06,640][105692] Updated weights for policy 0, policy_version 1534100 (0.0008) [2023-12-27 02:34:06,673][105620] Updated weights for policy 1, policy_version 1537035 (0.0007) [2023-12-27 02:34:06,692][105692] Updated weights for policy 0, policy_version 1534110 (0.0007) [2023-12-27 02:34:06,733][105620] Updated weights for policy 1, policy_version 1537045 (0.0011) [2023-12-27 02:34:06,749][105692] Updated weights for policy 0, policy_version 1534120 (0.0007) [2023-12-27 02:34:06,790][105620] Updated weights for policy 1, policy_version 1537055 (0.0011) [2023-12-27 02:34:07,502][105692] Updated weights for policy 0, policy_version 1534130 (0.0008) [2023-12-27 02:34:07,546][105620] Updated weights for policy 1, policy_version 1537065 (0.0011) [2023-12-27 02:34:07,555][105692] Updated weights for policy 0, policy_version 1534140 (0.0009) [2023-12-27 02:34:07,603][105620] Updated weights for policy 1, policy_version 1537075 (0.0010) [2023-12-27 02:34:07,606][105692] Updated weights for policy 0, policy_version 1534150 (0.0005) [2023-12-27 02:34:07,655][105620] Updated weights for policy 1, policy_version 1537085 (0.0010) [2023-12-27 02:34:07,702][105620] Updated weights for policy 1, policy_version 1537095 (0.0010) [2023-12-27 02:34:08,253][105692] Updated weights for policy 0, policy_version 1534160 (0.0008) [2023-12-27 02:34:08,311][105692] Updated weights for policy 0, policy_version 1534170 (0.0008) [2023-12-27 02:34:08,363][105692] Updated weights for policy 0, policy_version 1534180 (0.0009) [2023-12-27 02:34:08,496][105620] Updated weights for policy 1, policy_version 1537105 (0.0010) [2023-12-27 02:34:08,544][105620] Updated weights for policy 1, policy_version 1537115 (0.0010) [2023-12-27 02:34:08,600][105620] Updated weights for policy 1, policy_version 1537125 (0.0009) [2023-12-27 02:34:09,197][105692] Updated weights for policy 0, policy_version 1534190 (0.0008) [2023-12-27 02:34:09,233][105620] Updated weights for policy 1, policy_version 1537135 (0.0009) [2023-12-27 02:34:09,261][105692] Updated weights for policy 0, policy_version 1534200 (0.0007) [2023-12-27 02:34:09,284][105620] Updated weights for policy 1, policy_version 1537145 (0.0007) [2023-12-27 02:34:09,326][105692] Updated weights for policy 0, policy_version 1534210 (0.0008) [2023-12-27 02:34:09,334][105620] Updated weights for policy 1, policy_version 1537155 (0.0005) [2023-12-27 02:34:10,117][105620] Updated weights for policy 1, policy_version 1537165 (0.0008) [2023-12-27 02:34:10,152][105692] Updated weights for policy 0, policy_version 1534220 (0.0008) [2023-12-27 02:34:10,178][105620] Updated weights for policy 1, policy_version 1537175 (0.0009) [2023-12-27 02:34:10,207][105692] Updated weights for policy 0, policy_version 1534230 (0.0009) [2023-12-27 02:34:10,210][105586] KL-divergence is very high: 151.2065 [2023-12-27 02:34:10,241][105620] Updated weights for policy 1, policy_version 1537185 (0.0008) [2023-12-27 02:34:10,261][105586] KL-divergence is very high: 291.9630 [2023-12-27 02:34:10,264][105692] Updated weights for policy 0, policy_version 1534240 (0.0006) [2023-12-27 02:34:11,023][105620] Updated weights for policy 1, policy_version 1537195 (0.0009) [2023-12-27 02:34:11,030][105692] Updated weights for policy 0, policy_version 1534250 (0.0007) [2023-12-27 02:34:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 786399232. Throughput: 0: 9583.1, 1: 10048.0. Samples: 786415204. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:34:11,062][104569] Avg episode reward: [(0, '8627.345'), (1, '8900.349')] [2023-12-27 02:34:11,086][105620] Updated weights for policy 1, policy_version 1537205 (0.0009) [2023-12-27 02:34:11,092][105692] Updated weights for policy 0, policy_version 1534260 (0.0007) [2023-12-27 02:34:11,157][105692] Updated weights for policy 0, policy_version 1534270 (0.0008) [2023-12-27 02:34:11,159][105620] Updated weights for policy 1, policy_version 1537215 (0.0007) [2023-12-27 02:34:11,211][105692] Updated weights for policy 0, policy_version 1534280 (0.0006) [2023-12-27 02:34:11,824][105692] Updated weights for policy 0, policy_version 1534290 (0.0008) [2023-12-27 02:34:11,864][105620] Updated weights for policy 1, policy_version 1537225 (0.0009) [2023-12-27 02:34:11,882][105692] Updated weights for policy 0, policy_version 1534300 (0.0009) [2023-12-27 02:34:11,927][105620] Updated weights for policy 1, policy_version 1537235 (0.0008) [2023-12-27 02:34:11,928][105692] Updated weights for policy 0, policy_version 1534310 (0.0008) [2023-12-27 02:34:11,986][105620] Updated weights for policy 1, policy_version 1537245 (0.0008) [2023-12-27 02:34:12,045][105620] Updated weights for policy 1, policy_version 1537255 (0.0008) [2023-12-27 02:34:12,602][105692] Updated weights for policy 0, policy_version 1534320 (0.0009) [2023-12-27 02:34:12,666][105692] Updated weights for policy 0, policy_version 1534330 (0.0009) [2023-12-27 02:34:12,725][105692] Updated weights for policy 0, policy_version 1534340 (0.0009) [2023-12-27 02:34:12,788][105620] Updated weights for policy 1, policy_version 1537265 (0.0009) [2023-12-27 02:34:12,851][105620] Updated weights for policy 1, policy_version 1537275 (0.0009) [2023-12-27 02:34:12,911][105620] Updated weights for policy 1, policy_version 1537285 (0.0007) [2023-12-27 02:34:13,493][105692] Updated weights for policy 0, policy_version 1534350 (0.0008) [2023-12-27 02:34:13,551][105692] Updated weights for policy 0, policy_version 1534360 (0.0009) [2023-12-27 02:34:13,587][105620] Updated weights for policy 1, policy_version 1537295 (0.0005) [2023-12-27 02:34:13,611][105692] Updated weights for policy 0, policy_version 1534370 (0.0009) [2023-12-27 02:34:13,648][105620] Updated weights for policy 1, policy_version 1537305 (0.0005) [2023-12-27 02:34:13,704][105620] Updated weights for policy 1, policy_version 1537315 (0.0005) [2023-12-27 02:34:14,261][105620] Updated weights for policy 1, policy_version 1537325 (0.0007) [2023-12-27 02:34:14,321][105620] Updated weights for policy 1, policy_version 1537335 (0.0008) [2023-12-27 02:34:14,389][105620] Updated weights for policy 1, policy_version 1537345 (0.0010) [2023-12-27 02:34:14,436][105692] Updated weights for policy 0, policy_version 1534380 (0.0008) [2023-12-27 02:34:14,501][105692] Updated weights for policy 0, policy_version 1534390 (0.0007) [2023-12-27 02:34:14,564][105692] Updated weights for policy 0, policy_version 1534400 (0.0006) [2023-12-27 02:34:15,037][105620] Updated weights for policy 1, policy_version 1537355 (0.0010) [2023-12-27 02:34:15,099][105620] Updated weights for policy 1, policy_version 1537365 (0.0006) [2023-12-27 02:34:15,165][105620] Updated weights for policy 1, policy_version 1537375 (0.0006) [2023-12-27 02:34:15,236][105692] Updated weights for policy 0, policy_version 1534410 (0.0006) [2023-12-27 02:34:15,306][105692] Updated weights for policy 0, policy_version 1534420 (0.0008) [2023-12-27 02:34:15,367][105692] Updated weights for policy 0, policy_version 1534430 (0.0009) [2023-12-27 02:34:15,430][105692] Updated weights for policy 0, policy_version 1534440 (0.0008) [2023-12-27 02:34:15,776][105620] Updated weights for policy 1, policy_version 1537385 (0.0005) [2023-12-27 02:34:15,840][105620] Updated weights for policy 1, policy_version 1537395 (0.0005) [2023-12-27 02:34:15,887][105620] Updated weights for policy 1, policy_version 1537405 (0.0005) [2023-12-27 02:34:15,932][105620] Updated weights for policy 1, policy_version 1537415 (0.0010) [2023-12-27 02:34:16,028][105692] Updated weights for policy 0, policy_version 1534450 (0.0008) [2023-12-27 02:34:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 786505728. Throughput: 0: 9616.7, 1: 10006.2. Samples: 786473556. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:34:16,063][104569] Avg episode reward: [(0, '8447.513'), (1, '8809.146')] [2023-12-27 02:34:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001537416_393633792.pth... [2023-12-27 02:34:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001536232_393330688.pth [2023-12-27 02:34:16,088][105692] Updated weights for policy 0, policy_version 1534460 (0.0008) [2023-12-27 02:34:16,144][105692] Updated weights for policy 0, policy_version 1534470 (0.0009) [2023-12-27 02:34:16,156][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001534472_392880128.pth... [2023-12-27 02:34:16,160][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001533320_392585216.pth [2023-12-27 02:34:16,654][105620] Updated weights for policy 1, policy_version 1537425 (0.0011) [2023-12-27 02:34:16,712][105620] Updated weights for policy 1, policy_version 1537435 (0.0010) [2023-12-27 02:34:16,758][105620] Updated weights for policy 1, policy_version 1537445 (0.0010) [2023-12-27 02:34:16,886][105692] Updated weights for policy 0, policy_version 1534480 (0.0010) [2023-12-27 02:34:16,944][105692] Updated weights for policy 0, policy_version 1534490 (0.0010) [2023-12-27 02:34:16,992][105692] Updated weights for policy 0, policy_version 1534500 (0.0010) [2023-12-27 02:34:17,505][105620] Updated weights for policy 1, policy_version 1537455 (0.0009) [2023-12-27 02:34:17,567][105620] Updated weights for policy 1, policy_version 1537465 (0.0010) [2023-12-27 02:34:17,635][105620] Updated weights for policy 1, policy_version 1537475 (0.0010) [2023-12-27 02:34:17,732][105692] Updated weights for policy 0, policy_version 1534510 (0.0010) [2023-12-27 02:34:17,791][105692] Updated weights for policy 0, policy_version 1534520 (0.0010) [2023-12-27 02:34:17,839][105692] Updated weights for policy 0, policy_version 1534530 (0.0010) [2023-12-27 02:34:18,333][105620] Updated weights for policy 1, policy_version 1537485 (0.0010) [2023-12-27 02:34:18,392][105620] Updated weights for policy 1, policy_version 1537495 (0.0010) [2023-12-27 02:34:18,447][105620] Updated weights for policy 1, policy_version 1537505 (0.0009) [2023-12-27 02:34:18,541][105692] Updated weights for policy 0, policy_version 1534540 (0.0008) [2023-12-27 02:34:18,606][105692] Updated weights for policy 0, policy_version 1534550 (0.0005) [2023-12-27 02:34:18,672][105692] Updated weights for policy 0, policy_version 1534560 (0.0006) [2023-12-27 02:34:19,175][105620] Updated weights for policy 1, policy_version 1537515 (0.0010) [2023-12-27 02:34:19,238][105620] Updated weights for policy 1, policy_version 1537525 (0.0010) [2023-12-27 02:34:19,297][105692] Updated weights for policy 0, policy_version 1534570 (0.0008) [2023-12-27 02:34:19,312][105620] Updated weights for policy 1, policy_version 1537535 (0.0008) [2023-12-27 02:34:19,363][105692] Updated weights for policy 0, policy_version 1534580 (0.0007) [2023-12-27 02:34:19,431][105692] Updated weights for policy 0, policy_version 1534590 (0.0009) [2023-12-27 02:34:19,501][105692] Updated weights for policy 0, policy_version 1534600 (0.0009) [2023-12-27 02:34:20,001][105620] Updated weights for policy 1, policy_version 1537545 (0.0009) [2023-12-27 02:34:20,058][105620] Updated weights for policy 1, policy_version 1537555 (0.0009) [2023-12-27 02:34:20,120][105620] Updated weights for policy 1, policy_version 1537565 (0.0009) [2023-12-27 02:34:20,181][105620] Updated weights for policy 1, policy_version 1537575 (0.0008) [2023-12-27 02:34:20,369][105692] Updated weights for policy 0, policy_version 1534610 (0.0010) [2023-12-27 02:34:20,431][105692] Updated weights for policy 0, policy_version 1534620 (0.0009) [2023-12-27 02:34:20,481][105692] Updated weights for policy 0, policy_version 1534630 (0.0010) [2023-12-27 02:34:20,916][105620] Updated weights for policy 1, policy_version 1537585 (0.0009) [2023-12-27 02:34:20,965][105620] Updated weights for policy 1, policy_version 1537595 (0.0007) [2023-12-27 02:34:21,017][105620] Updated weights for policy 1, policy_version 1537605 (0.0005) [2023-12-27 02:34:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 786604032. Throughput: 0: 9629.4, 1: 10029.1. Samples: 786593228. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:34:21,062][104569] Avg episode reward: [(0, '8539.016'), (1, '8716.278')] [2023-12-27 02:34:21,242][105692] Updated weights for policy 0, policy_version 1534641 (0.0009) [2023-12-27 02:34:21,309][105692] Updated weights for policy 0, policy_version 1534651 (0.0009) [2023-12-27 02:34:21,373][105692] Updated weights for policy 0, policy_version 1534661 (0.0009) [2023-12-27 02:34:21,789][105620] Updated weights for policy 1, policy_version 1537615 (0.0008) [2023-12-27 02:34:21,856][105620] Updated weights for policy 1, policy_version 1537625 (0.0007) [2023-12-27 02:34:21,925][105620] Updated weights for policy 1, policy_version 1537635 (0.0006) [2023-12-27 02:34:22,169][105692] Updated weights for policy 0, policy_version 1534671 (0.0010) [2023-12-27 02:34:22,226][105692] Updated weights for policy 0, policy_version 1534681 (0.0011) [2023-12-27 02:34:22,290][105692] Updated weights for policy 0, policy_version 1534691 (0.0011) [2023-12-27 02:34:22,666][105620] Updated weights for policy 1, policy_version 1537645 (0.0008) [2023-12-27 02:34:22,725][105620] Updated weights for policy 1, policy_version 1537655 (0.0009) [2023-12-27 02:34:22,774][105620] Updated weights for policy 1, policy_version 1537665 (0.0009) [2023-12-27 02:34:23,050][105692] Updated weights for policy 0, policy_version 1534701 (0.0009) [2023-12-27 02:34:23,110][105692] Updated weights for policy 0, policy_version 1534711 (0.0005) [2023-12-27 02:34:23,167][105692] Updated weights for policy 0, policy_version 1534721 (0.0005) [2023-12-27 02:34:23,476][105620] Updated weights for policy 1, policy_version 1537675 (0.0008) [2023-12-27 02:34:23,528][105620] Updated weights for policy 1, policy_version 1537685 (0.0009) [2023-12-27 02:34:23,575][105620] Updated weights for policy 1, policy_version 1537695 (0.0008) [2023-12-27 02:34:23,906][105692] Updated weights for policy 0, policy_version 1534731 (0.0009) [2023-12-27 02:34:23,958][105692] Updated weights for policy 0, policy_version 1534741 (0.0008) [2023-12-27 02:34:24,013][105692] Updated weights for policy 0, policy_version 1534751 (0.0005) [2023-12-27 02:34:24,395][105620] Updated weights for policy 1, policy_version 1537705 (0.0008) [2023-12-27 02:34:24,456][105620] Updated weights for policy 1, policy_version 1537715 (0.0005) [2023-12-27 02:34:24,518][105620] Updated weights for policy 1, policy_version 1537725 (0.0005) [2023-12-27 02:34:24,582][105620] Updated weights for policy 1, policy_version 1537735 (0.0005) [2023-12-27 02:34:24,664][105692] Updated weights for policy 0, policy_version 1534761 (0.0006) [2023-12-27 02:34:24,715][105692] Updated weights for policy 0, policy_version 1534771 (0.0006) [2023-12-27 02:34:24,779][105692] Updated weights for policy 0, policy_version 1534781 (0.0005) [2023-12-27 02:34:24,832][105692] Updated weights for policy 0, policy_version 1534791 (0.0005) [2023-12-27 02:34:25,138][105620] Updated weights for policy 1, policy_version 1537745 (0.0006) [2023-12-27 02:34:25,193][105620] Updated weights for policy 1, policy_version 1537755 (0.0009) [2023-12-27 02:34:25,249][105620] Updated weights for policy 1, policy_version 1537765 (0.0008) [2023-12-27 02:34:25,403][105692] Updated weights for policy 0, policy_version 1534801 (0.0005) [2023-12-27 02:34:25,454][105692] Updated weights for policy 0, policy_version 1534811 (0.0005) [2023-12-27 02:34:25,499][105692] Updated weights for policy 0, policy_version 1534821 (0.0005) [2023-12-27 02:34:26,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 786694144. Throughput: 0: 9705.5, 1: 9901.1. Samples: 786708140. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:34:26,062][104569] Avg episode reward: [(0, '8362.096'), (1, '8629.122')] [2023-12-27 02:34:26,070][105620] Updated weights for policy 1, policy_version 1537775 (0.0009) [2023-12-27 02:34:26,131][105620] Updated weights for policy 1, policy_version 1537785 (0.0009) [2023-12-27 02:34:26,161][105692] Updated weights for policy 0, policy_version 1534831 (0.0006) [2023-12-27 02:34:26,195][105620] Updated weights for policy 1, policy_version 1537795 (0.0008) [2023-12-27 02:34:26,222][105692] Updated weights for policy 0, policy_version 1534841 (0.0006) [2023-12-27 02:34:26,284][105692] Updated weights for policy 0, policy_version 1534851 (0.0008) [2023-12-27 02:34:26,943][105692] Updated weights for policy 0, policy_version 1534861 (0.0008) [2023-12-27 02:34:26,953][105620] Updated weights for policy 1, policy_version 1537805 (0.0007) [2023-12-27 02:34:26,972][105586] KL-divergence is very high: 187.3203 [2023-12-27 02:34:26,996][105692] Updated weights for policy 0, policy_version 1534871 (0.0006) [2023-12-27 02:34:26,999][105620] Updated weights for policy 1, policy_version 1537815 (0.0008) [2023-12-27 02:34:27,008][105586] KL-divergence is very high: 303.7753 [2023-12-27 02:34:27,045][105620] Updated weights for policy 1, policy_version 1537825 (0.0007) [2023-12-27 02:34:27,046][105586] KL-divergence is very high: 326.6179 [2023-12-27 02:34:27,054][105692] Updated weights for policy 0, policy_version 1534881 (0.0009) [2023-12-27 02:34:27,741][105692] Updated weights for policy 0, policy_version 1534891 (0.0008) [2023-12-27 02:34:27,796][105692] Updated weights for policy 0, policy_version 1534901 (0.0010) [2023-12-27 02:34:27,850][105692] Updated weights for policy 0, policy_version 1534911 (0.0010) [2023-12-27 02:34:27,851][105620] Updated weights for policy 1, policy_version 1537835 (0.0008) [2023-12-27 02:34:27,897][105620] Updated weights for policy 1, policy_version 1537845 (0.0007) [2023-12-27 02:34:27,953][105620] Updated weights for policy 1, policy_version 1537855 (0.0009) [2023-12-27 02:34:28,611][105692] Updated weights for policy 0, policy_version 1534921 (0.0010) [2023-12-27 02:34:28,673][105692] Updated weights for policy 0, policy_version 1534931 (0.0010) [2023-12-27 02:34:28,712][105620] Updated weights for policy 1, policy_version 1537865 (0.0008) [2023-12-27 02:34:28,731][105692] Updated weights for policy 0, policy_version 1534941 (0.0010) [2023-12-27 02:34:28,768][105620] Updated weights for policy 1, policy_version 1537875 (0.0005) [2023-12-27 02:34:28,789][105692] Updated weights for policy 0, policy_version 1534951 (0.0010) [2023-12-27 02:34:28,833][105620] Updated weights for policy 1, policy_version 1537885 (0.0007) [2023-12-27 02:34:28,899][105620] Updated weights for policy 1, policy_version 1537895 (0.0008) [2023-12-27 02:34:29,521][105692] Updated weights for policy 0, policy_version 1534961 (0.0010) [2023-12-27 02:34:29,583][105692] Updated weights for policy 0, policy_version 1534971 (0.0010) [2023-12-27 02:34:29,644][105692] Updated weights for policy 0, policy_version 1534981 (0.0010) [2023-12-27 02:34:29,655][105620] Updated weights for policy 1, policy_version 1537905 (0.0007) [2023-12-27 02:34:29,715][105620] Updated weights for policy 1, policy_version 1537915 (0.0005) [2023-12-27 02:34:29,770][105620] Updated weights for policy 1, policy_version 1537925 (0.0007) [2023-12-27 02:34:30,260][105692] Updated weights for policy 0, policy_version 1534991 (0.0009) [2023-12-27 02:34:30,316][105692] Updated weights for policy 0, policy_version 1535001 (0.0010) [2023-12-27 02:34:30,383][105692] Updated weights for policy 0, policy_version 1535011 (0.0011) [2023-12-27 02:34:30,438][105620] Updated weights for policy 1, policy_version 1537935 (0.0009) [2023-12-27 02:34:30,494][105620] Updated weights for policy 1, policy_version 1537945 (0.0006) [2023-12-27 02:34:30,562][105620] Updated weights for policy 1, policy_version 1537955 (0.0009) [2023-12-27 02:34:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 786792448. Throughput: 0: 9736.2, 1: 9858.3. Samples: 786765388. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:34:31,063][104569] Avg episode reward: [(0, '8724.245'), (1, '8816.983')] [2023-12-27 02:34:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001535016_393019392.pth... [2023-12-27 02:34:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001537960_393773056.pth... [2023-12-27 02:34:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001536808_393478144.pth [2023-12-27 02:34:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001533896_392732672.pth [2023-12-27 02:34:31,108][105692] Updated weights for policy 0, policy_version 1535021 (0.0010) [2023-12-27 02:34:31,168][105692] Updated weights for policy 0, policy_version 1535031 (0.0010) [2023-12-27 02:34:31,202][105620] Updated weights for policy 1, policy_version 1537965 (0.0006) [2023-12-27 02:34:31,235][105692] Updated weights for policy 0, policy_version 1535041 (0.0011) [2023-12-27 02:34:31,265][105620] Updated weights for policy 1, policy_version 1537975 (0.0006) [2023-12-27 02:34:31,316][105620] Updated weights for policy 1, policy_version 1537985 (0.0007) [2023-12-27 02:34:31,873][105692] Updated weights for policy 0, policy_version 1535051 (0.0011) [2023-12-27 02:34:31,942][105692] Updated weights for policy 0, policy_version 1535061 (0.0010) [2023-12-27 02:34:32,001][105692] Updated weights for policy 0, policy_version 1535071 (0.0010) [2023-12-27 02:34:32,031][105620] Updated weights for policy 1, policy_version 1537995 (0.0007) [2023-12-27 02:34:32,095][105620] Updated weights for policy 1, policy_version 1538005 (0.0007) [2023-12-27 02:34:32,158][105620] Updated weights for policy 1, policy_version 1538015 (0.0009) [2023-12-27 02:34:32,632][105692] Updated weights for policy 0, policy_version 1535081 (0.0010) [2023-12-27 02:34:32,694][105692] Updated weights for policy 0, policy_version 1535091 (0.0010) [2023-12-27 02:34:32,752][105692] Updated weights for policy 0, policy_version 1535101 (0.0010) [2023-12-27 02:34:32,798][105620] Updated weights for policy 1, policy_version 1538025 (0.0009) [2023-12-27 02:34:32,811][105692] Updated weights for policy 0, policy_version 1535111 (0.0010) [2023-12-27 02:34:32,851][105620] Updated weights for policy 1, policy_version 1538035 (0.0008) [2023-12-27 02:34:32,902][105620] Updated weights for policy 1, policy_version 1538045 (0.0007) [2023-12-27 02:34:32,960][105620] Updated weights for policy 1, policy_version 1538055 (0.0007) [2023-12-27 02:34:33,476][105692] Updated weights for policy 0, policy_version 1535121 (0.0010) [2023-12-27 02:34:33,526][105692] Updated weights for policy 0, policy_version 1535131 (0.0009) [2023-12-27 02:34:33,549][105620] Updated weights for policy 1, policy_version 1538065 (0.0010) [2023-12-27 02:34:33,590][105692] Updated weights for policy 0, policy_version 1535141 (0.0006) [2023-12-27 02:34:33,604][105620] Updated weights for policy 1, policy_version 1538075 (0.0005) [2023-12-27 02:34:33,650][105620] Updated weights for policy 1, policy_version 1538085 (0.0005) [2023-12-27 02:34:34,226][105692] Updated weights for policy 0, policy_version 1535151 (0.0006) [2023-12-27 02:34:34,272][105692] Updated weights for policy 0, policy_version 1535161 (0.0008) [2023-12-27 02:34:34,286][105620] Updated weights for policy 1, policy_version 1538095 (0.0007) [2023-12-27 02:34:34,322][105692] Updated weights for policy 0, policy_version 1535171 (0.0007) [2023-12-27 02:34:34,335][105620] Updated weights for policy 1, policy_version 1538105 (0.0007) [2023-12-27 02:34:34,390][105620] Updated weights for policy 1, policy_version 1538115 (0.0008) [2023-12-27 02:34:35,116][105692] Updated weights for policy 0, policy_version 1535181 (0.0006) [2023-12-27 02:34:35,164][105620] Updated weights for policy 1, policy_version 1538125 (0.0010) [2023-12-27 02:34:35,181][105692] Updated weights for policy 0, policy_version 1535191 (0.0006) [2023-12-27 02:34:35,223][105620] Updated weights for policy 1, policy_version 1538135 (0.0008) [2023-12-27 02:34:35,238][105692] Updated weights for policy 0, policy_version 1535201 (0.0007) [2023-12-27 02:34:35,275][105620] Updated weights for policy 1, policy_version 1538145 (0.0005) [2023-12-27 02:34:35,902][105620] Updated weights for policy 1, policy_version 1538155 (0.0007) [2023-12-27 02:34:35,929][105692] Updated weights for policy 0, policy_version 1535211 (0.0009) [2023-12-27 02:34:35,957][105620] Updated weights for policy 1, policy_version 1538165 (0.0010) [2023-12-27 02:34:35,977][105692] Updated weights for policy 0, policy_version 1535221 (0.0008) [2023-12-27 02:34:36,010][105620] Updated weights for policy 1, policy_version 1538175 (0.0010) [2023-12-27 02:34:36,027][105692] Updated weights for policy 0, policy_version 1535231 (0.0008) [2023-12-27 02:34:36,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 786898944. Throughput: 0: 9780.6, 1: 9882.0. Samples: 786887892. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:34:36,062][104569] Avg episode reward: [(0, '8900.992'), (1, '8819.126')] [2023-12-27 02:34:36,722][105692] Updated weights for policy 0, policy_version 1535241 (0.0006) [2023-12-27 02:34:36,770][105692] Updated weights for policy 0, policy_version 1535251 (0.0008) [2023-12-27 02:34:36,777][105620] Updated weights for policy 1, policy_version 1538185 (0.0010) [2023-12-27 02:34:36,830][105692] Updated weights for policy 0, policy_version 1535261 (0.0008) [2023-12-27 02:34:36,836][105620] Updated weights for policy 1, policy_version 1538195 (0.0006) [2023-12-27 02:34:36,890][105620] Updated weights for policy 1, policy_version 1538205 (0.0005) [2023-12-27 02:34:36,892][105692] Updated weights for policy 0, policy_version 1535271 (0.0008) [2023-12-27 02:34:36,950][105620] Updated weights for policy 1, policy_version 1538215 (0.0007) [2023-12-27 02:34:37,536][105620] Updated weights for policy 1, policy_version 1538225 (0.0008) [2023-12-27 02:34:37,597][105620] Updated weights for policy 1, policy_version 1538235 (0.0008) [2023-12-27 02:34:37,656][105620] Updated weights for policy 1, policy_version 1538245 (0.0008) [2023-12-27 02:34:37,689][105692] Updated weights for policy 0, policy_version 1535281 (0.0010) [2023-12-27 02:34:37,745][105692] Updated weights for policy 0, policy_version 1535291 (0.0010) [2023-12-27 02:34:37,802][105692] Updated weights for policy 0, policy_version 1535301 (0.0011) [2023-12-27 02:34:38,368][105620] Updated weights for policy 1, policy_version 1538255 (0.0009) [2023-12-27 02:34:38,433][105620] Updated weights for policy 1, policy_version 1538265 (0.0006) [2023-12-27 02:34:38,487][105620] Updated weights for policy 1, policy_version 1538275 (0.0005) [2023-12-27 02:34:38,510][105692] Updated weights for policy 0, policy_version 1535311 (0.0007) [2023-12-27 02:34:38,564][105692] Updated weights for policy 0, policy_version 1535321 (0.0005) [2023-12-27 02:34:38,623][105692] Updated weights for policy 0, policy_version 1535331 (0.0008) [2023-12-27 02:34:39,216][105620] Updated weights for policy 1, policy_version 1538285 (0.0007) [2023-12-27 02:34:39,277][105620] Updated weights for policy 1, policy_version 1538295 (0.0007) [2023-12-27 02:34:39,344][105620] Updated weights for policy 1, policy_version 1538305 (0.0007) [2023-12-27 02:34:39,350][105692] Updated weights for policy 0, policy_version 1535341 (0.0010) [2023-12-27 02:34:39,418][105692] Updated weights for policy 0, policy_version 1535351 (0.0010) [2023-12-27 02:34:39,481][105692] Updated weights for policy 0, policy_version 1535361 (0.0011) [2023-12-27 02:34:40,073][105620] Updated weights for policy 1, policy_version 1538315 (0.0008) [2023-12-27 02:34:40,136][105620] Updated weights for policy 1, policy_version 1538325 (0.0008) [2023-12-27 02:34:40,171][105692] Updated weights for policy 0, policy_version 1535371 (0.0009) [2023-12-27 02:34:40,195][105620] Updated weights for policy 1, policy_version 1538335 (0.0008) [2023-12-27 02:34:40,226][105692] Updated weights for policy 0, policy_version 1535381 (0.0006) [2023-12-27 02:34:40,285][105692] Updated weights for policy 0, policy_version 1535391 (0.0007) [2023-12-27 02:34:40,956][105620] Updated weights for policy 1, policy_version 1538345 (0.0008) [2023-12-27 02:34:41,025][105620] Updated weights for policy 1, policy_version 1538355 (0.0006) [2023-12-27 02:34:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 786989056. Throughput: 0: 9731.7, 1: 9854.5. Samples: 787005116. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:34:41,063][104569] Avg episode reward: [(0, '8715.687'), (1, '8721.376')] [2023-12-27 02:34:41,084][105692] Updated weights for policy 0, policy_version 1535401 (0.0008) [2023-12-27 02:34:41,090][105620] Updated weights for policy 1, policy_version 1538365 (0.0008) [2023-12-27 02:34:41,148][105692] Updated weights for policy 0, policy_version 1535411 (0.0009) [2023-12-27 02:34:41,149][105620] Updated weights for policy 1, policy_version 1538375 (0.0007) [2023-12-27 02:34:41,207][105692] Updated weights for policy 0, policy_version 1535421 (0.0009) [2023-12-27 02:34:41,268][105692] Updated weights for policy 0, policy_version 1535432 (0.0012) [2023-12-27 02:34:41,803][105620] Updated weights for policy 1, policy_version 1538385 (0.0009) [2023-12-27 02:34:41,867][105620] Updated weights for policy 1, policy_version 1538395 (0.0008) [2023-12-27 02:34:41,928][105620] Updated weights for policy 1, policy_version 1538405 (0.0008) [2023-12-27 02:34:42,114][105692] Updated weights for policy 0, policy_version 1535442 (0.0009) [2023-12-27 02:34:42,170][105692] Updated weights for policy 0, policy_version 1535452 (0.0009) [2023-12-27 02:34:42,221][105692] Updated weights for policy 0, policy_version 1535462 (0.0010) [2023-12-27 02:34:42,673][105620] Updated weights for policy 1, policy_version 1538415 (0.0008) [2023-12-27 02:34:42,729][105620] Updated weights for policy 1, policy_version 1538425 (0.0007) [2023-12-27 02:34:42,791][105620] Updated weights for policy 1, policy_version 1538435 (0.0007) [2023-12-27 02:34:43,064][105692] Updated weights for policy 0, policy_version 1535472 (0.0010) [2023-12-27 02:34:43,113][105692] Updated weights for policy 0, policy_version 1535482 (0.0009) [2023-12-27 02:34:43,165][105692] Updated weights for policy 0, policy_version 1535493 (0.0010) [2023-12-27 02:34:43,398][105620] Updated weights for policy 1, policy_version 1538445 (0.0007) [2023-12-27 02:34:43,460][105620] Updated weights for policy 1, policy_version 1538455 (0.0006) [2023-12-27 02:34:43,519][105620] Updated weights for policy 1, policy_version 1538465 (0.0005) [2023-12-27 02:34:43,934][105692] Updated weights for policy 0, policy_version 1535503 (0.0010) [2023-12-27 02:34:43,993][105692] Updated weights for policy 0, policy_version 1535513 (0.0008) [2023-12-27 02:34:44,046][105692] Updated weights for policy 0, policy_version 1535523 (0.0008) [2023-12-27 02:34:44,126][105620] Updated weights for policy 1, policy_version 1538475 (0.0005) [2023-12-27 02:34:44,176][105620] Updated weights for policy 1, policy_version 1538485 (0.0005) [2023-12-27 02:34:44,232][105620] Updated weights for policy 1, policy_version 1538495 (0.0007) [2023-12-27 02:34:44,687][105692] Updated weights for policy 0, policy_version 1535533 (0.0008) [2023-12-27 02:34:44,756][105692] Updated weights for policy 0, policy_version 1535543 (0.0011) [2023-12-27 02:34:44,766][105620] Updated weights for policy 1, policy_version 1538505 (0.0009) [2023-12-27 02:34:44,820][105692] Updated weights for policy 0, policy_version 1535553 (0.0010) [2023-12-27 02:34:44,832][105620] Updated weights for policy 1, policy_version 1538515 (0.0011) [2023-12-27 02:34:44,892][105620] Updated weights for policy 1, policy_version 1538525 (0.0011) [2023-12-27 02:34:44,956][105620] Updated weights for policy 1, policy_version 1538535 (0.0010) [2023-12-27 02:34:45,516][105692] Updated weights for policy 0, policy_version 1535563 (0.0009) [2023-12-27 02:34:45,564][105692] Updated weights for policy 0, policy_version 1535573 (0.0006) [2023-12-27 02:34:45,610][105692] Updated weights for policy 0, policy_version 1535583 (0.0005) [2023-12-27 02:34:45,699][105620] Updated weights for policy 1, policy_version 1538545 (0.0011) [2023-12-27 02:34:45,754][105620] Updated weights for policy 1, policy_version 1538555 (0.0010) [2023-12-27 02:34:45,802][105620] Updated weights for policy 1, policy_version 1538565 (0.0011) [2023-12-27 02:34:46,062][104569] Fps is (10 sec: 19659.7, 60 sec: 19660.6, 300 sec: 19521.9). Total num frames: 787095552. Throughput: 0: 9638.3, 1: 9887.1. Samples: 787061724. Policy #0 lag: (min: 26.0, avg: 31.9, max: 32.0) [2023-12-27 02:34:46,064][104569] Avg episode reward: [(0, '8443.213'), (1, '8444.854')] [2023-12-27 02:34:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001538568_393928704.pth... [2023-12-27 02:34:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001535592_393166848.pth... [2023-12-27 02:34:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001537416_393633792.pth [2023-12-27 02:34:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001534472_392880128.pth [2023-12-27 02:34:46,188][105692] Updated weights for policy 0, policy_version 1535593 (0.0006) [2023-12-27 02:34:46,237][105692] Updated weights for policy 0, policy_version 1535603 (0.0008) [2023-12-27 02:34:46,300][105692] Updated weights for policy 0, policy_version 1535613 (0.0010) [2023-12-27 02:34:46,358][105692] Updated weights for policy 0, policy_version 1535623 (0.0010) [2023-12-27 02:34:46,481][105620] Updated weights for policy 1, policy_version 1538575 (0.0011) [2023-12-27 02:34:46,541][105620] Updated weights for policy 1, policy_version 1538585 (0.0010) [2023-12-27 02:34:46,600][105620] Updated weights for policy 1, policy_version 1538595 (0.0005) [2023-12-27 02:34:47,102][105692] Updated weights for policy 0, policy_version 1535633 (0.0009) [2023-12-27 02:34:47,171][105692] Updated weights for policy 0, policy_version 1535643 (0.0010) [2023-12-27 02:34:47,226][105692] Updated weights for policy 0, policy_version 1535653 (0.0010) [2023-12-27 02:34:47,247][105620] Updated weights for policy 1, policy_version 1538605 (0.0005) [2023-12-27 02:34:47,298][105620] Updated weights for policy 1, policy_version 1538615 (0.0005) [2023-12-27 02:34:47,352][105620] Updated weights for policy 1, policy_version 1538625 (0.0005) [2023-12-27 02:34:47,967][105692] Updated weights for policy 0, policy_version 1535663 (0.0007) [2023-12-27 02:34:48,019][105692] Updated weights for policy 0, policy_version 1535673 (0.0008) [2023-12-27 02:34:48,047][105620] Updated weights for policy 1, policy_version 1538635 (0.0008) [2023-12-27 02:34:48,073][105692] Updated weights for policy 0, policy_version 1535683 (0.0007) [2023-12-27 02:34:48,106][105620] Updated weights for policy 1, policy_version 1538645 (0.0011) [2023-12-27 02:34:48,165][105620] Updated weights for policy 1, policy_version 1538655 (0.0011) [2023-12-27 02:34:48,761][105692] Updated weights for policy 0, policy_version 1535693 (0.0007) [2023-12-27 02:34:48,820][105692] Updated weights for policy 0, policy_version 1535703 (0.0009) [2023-12-27 02:34:48,887][105692] Updated weights for policy 0, policy_version 1535713 (0.0009) [2023-12-27 02:34:48,914][105620] Updated weights for policy 1, policy_version 1538665 (0.0010) [2023-12-27 02:34:48,963][105620] Updated weights for policy 1, policy_version 1538675 (0.0009) [2023-12-27 02:34:49,013][105620] Updated weights for policy 1, policy_version 1538685 (0.0008) [2023-12-27 02:34:49,062][105620] Updated weights for policy 1, policy_version 1538695 (0.0005) [2023-12-27 02:34:49,703][105620] Updated weights for policy 1, policy_version 1538705 (0.0009) [2023-12-27 02:34:49,724][105692] Updated weights for policy 0, policy_version 1535723 (0.0007) [2023-12-27 02:34:49,757][105620] Updated weights for policy 1, policy_version 1538715 (0.0005) [2023-12-27 02:34:49,785][105692] Updated weights for policy 0, policy_version 1535733 (0.0009) [2023-12-27 02:34:49,818][105620] Updated weights for policy 1, policy_version 1538725 (0.0007) [2023-12-27 02:34:49,846][105692] Updated weights for policy 0, policy_version 1535743 (0.0007) [2023-12-27 02:34:50,570][105692] Updated weights for policy 0, policy_version 1535753 (0.0007) [2023-12-27 02:34:50,590][105620] Updated weights for policy 1, policy_version 1538735 (0.0008) [2023-12-27 02:34:50,634][105692] Updated weights for policy 0, policy_version 1535763 (0.0008) [2023-12-27 02:34:50,655][105620] Updated weights for policy 1, policy_version 1538745 (0.0005) [2023-12-27 02:34:50,693][105692] Updated weights for policy 0, policy_version 1535773 (0.0006) [2023-12-27 02:34:50,722][105620] Updated weights for policy 1, policy_version 1538755 (0.0006) [2023-12-27 02:34:50,757][105692] Updated weights for policy 0, policy_version 1535783 (0.0006) [2023-12-27 02:34:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 787193856. Throughput: 0: 9744.8, 1: 9933.2. Samples: 787183076. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:34:51,063][104569] Avg episode reward: [(0, '8542.893'), (1, '8536.579')] [2023-12-27 02:34:51,370][105620] Updated weights for policy 1, policy_version 1538765 (0.0008) [2023-12-27 02:34:51,435][105620] Updated weights for policy 1, policy_version 1538775 (0.0008) [2023-12-27 02:34:51,502][105620] Updated weights for policy 1, policy_version 1538786 (0.0007) [2023-12-27 02:34:51,514][105692] Updated weights for policy 0, policy_version 1535793 (0.0008) [2023-12-27 02:34:51,583][105692] Updated weights for policy 0, policy_version 1535803 (0.0006) [2023-12-27 02:34:51,648][105692] Updated weights for policy 0, policy_version 1535813 (0.0008) [2023-12-27 02:34:52,216][105620] Updated weights for policy 1, policy_version 1538796 (0.0006) [2023-12-27 02:34:52,283][105620] Updated weights for policy 1, policy_version 1538806 (0.0006) [2023-12-27 02:34:52,332][105692] Updated weights for policy 0, policy_version 1535823 (0.0008) [2023-12-27 02:34:52,345][105620] Updated weights for policy 1, policy_version 1538816 (0.0006) [2023-12-27 02:34:52,403][105692] Updated weights for policy 0, policy_version 1535833 (0.0007) [2023-12-27 02:34:52,461][105692] Updated weights for policy 0, policy_version 1535843 (0.0009) [2023-12-27 02:34:53,008][105620] Updated weights for policy 1, policy_version 1538826 (0.0007) [2023-12-27 02:34:53,077][105620] Updated weights for policy 1, policy_version 1538836 (0.0005) [2023-12-27 02:34:53,137][105620] Updated weights for policy 1, policy_version 1538846 (0.0009) [2023-12-27 02:34:53,193][105620] Updated weights for policy 1, policy_version 1538856 (0.0009) [2023-12-27 02:34:53,201][105692] Updated weights for policy 0, policy_version 1535853 (0.0008) [2023-12-27 02:34:53,255][105692] Updated weights for policy 0, policy_version 1535863 (0.0009) [2023-12-27 02:34:53,303][105692] Updated weights for policy 0, policy_version 1535873 (0.0009) [2023-12-27 02:34:53,930][105620] Updated weights for policy 1, policy_version 1538866 (0.0009) [2023-12-27 02:34:53,984][105620] Updated weights for policy 1, policy_version 1538876 (0.0009) [2023-12-27 02:34:54,030][105620] Updated weights for policy 1, policy_version 1538886 (0.0009) [2023-12-27 02:34:54,037][105692] Updated weights for policy 0, policy_version 1535883 (0.0008) [2023-12-27 02:34:54,083][105692] Updated weights for policy 0, policy_version 1535893 (0.0008) [2023-12-27 02:34:54,138][105692] Updated weights for policy 0, policy_version 1535903 (0.0009) [2023-12-27 02:34:54,768][105620] Updated weights for policy 1, policy_version 1538896 (0.0009) [2023-12-27 02:34:54,815][105692] Updated weights for policy 0, policy_version 1535913 (0.0007) [2023-12-27 02:34:54,823][105620] Updated weights for policy 1, policy_version 1538906 (0.0009) [2023-12-27 02:34:54,866][105692] Updated weights for policy 0, policy_version 1535923 (0.0006) [2023-12-27 02:34:54,870][105620] Updated weights for policy 1, policy_version 1538916 (0.0009) [2023-12-27 02:34:54,925][105692] Updated weights for policy 0, policy_version 1535933 (0.0005) [2023-12-27 02:34:54,995][105692] Updated weights for policy 0, policy_version 1535943 (0.0010) [2023-12-27 02:34:55,574][105620] Updated weights for policy 1, policy_version 1538926 (0.0010) [2023-12-27 02:34:55,628][105620] Updated weights for policy 1, policy_version 1538936 (0.0010) [2023-12-27 02:34:55,688][105620] Updated weights for policy 1, policy_version 1538946 (0.0010) [2023-12-27 02:34:55,690][105692] Updated weights for policy 0, policy_version 1535953 (0.0011) [2023-12-27 02:34:55,742][105692] Updated weights for policy 0, policy_version 1535963 (0.0011) [2023-12-27 02:34:55,796][105692] Updated weights for policy 0, policy_version 1535973 (0.0010) [2023-12-27 02:34:56,062][104569] Fps is (10 sec: 19661.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 787292160. Throughput: 0: 9726.6, 1: 9921.3. Samples: 787299360. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:34:56,063][104569] Avg episode reward: [(0, '8813.050'), (1, '8636.115')] [2023-12-27 02:34:56,425][105620] Updated weights for policy 1, policy_version 1538956 (0.0008) [2023-12-27 02:34:56,490][105620] Updated weights for policy 1, policy_version 1538966 (0.0010) [2023-12-27 02:34:56,545][105620] Updated weights for policy 1, policy_version 1538976 (0.0007) [2023-12-27 02:34:56,580][105692] Updated weights for policy 0, policy_version 1535983 (0.0010) [2023-12-27 02:34:56,632][105692] Updated weights for policy 0, policy_version 1535993 (0.0010) [2023-12-27 02:34:56,689][105692] Updated weights for policy 0, policy_version 1536003 (0.0009) [2023-12-27 02:34:57,165][105620] Updated weights for policy 1, policy_version 1538986 (0.0006) [2023-12-27 02:34:57,224][105620] Updated weights for policy 1, policy_version 1538996 (0.0010) [2023-12-27 02:34:57,279][105620] Updated weights for policy 1, policy_version 1539006 (0.0010) [2023-12-27 02:34:57,338][105620] Updated weights for policy 1, policy_version 1539016 (0.0011) [2023-12-27 02:34:57,404][105692] Updated weights for policy 0, policy_version 1536013 (0.0008) [2023-12-27 02:34:57,471][105692] Updated weights for policy 0, policy_version 1536023 (0.0007) [2023-12-27 02:34:57,519][105692] Updated weights for policy 0, policy_version 1536033 (0.0008) [2023-12-27 02:34:58,069][105620] Updated weights for policy 1, policy_version 1539026 (0.0009) [2023-12-27 02:34:58,123][105620] Updated weights for policy 1, policy_version 1539036 (0.0010) [2023-12-27 02:34:58,152][105692] Updated weights for policy 0, policy_version 1536043 (0.0009) [2023-12-27 02:34:58,185][105620] Updated weights for policy 1, policy_version 1539046 (0.0009) [2023-12-27 02:34:58,216][105692] Updated weights for policy 0, policy_version 1536053 (0.0008) [2023-12-27 02:34:58,278][105692] Updated weights for policy 0, policy_version 1536063 (0.0008) [2023-12-27 02:34:58,944][105620] Updated weights for policy 1, policy_version 1539056 (0.0008) [2023-12-27 02:34:59,007][105620] Updated weights for policy 1, policy_version 1539066 (0.0006) [2023-12-27 02:34:59,041][105692] Updated weights for policy 0, policy_version 1536073 (0.0008) [2023-12-27 02:34:59,072][105620] Updated weights for policy 1, policy_version 1539076 (0.0008) [2023-12-27 02:34:59,110][105692] Updated weights for policy 0, policy_version 1536083 (0.0007) [2023-12-27 02:34:59,175][105692] Updated weights for policy 0, policy_version 1536093 (0.0009) [2023-12-27 02:34:59,242][105692] Updated weights for policy 0, policy_version 1536103 (0.0009) [2023-12-27 02:34:59,810][105620] Updated weights for policy 1, policy_version 1539086 (0.0008) [2023-12-27 02:34:59,873][105620] Updated weights for policy 1, policy_version 1539096 (0.0010) [2023-12-27 02:34:59,933][105620] Updated weights for policy 1, policy_version 1539106 (0.0010) [2023-12-27 02:34:59,978][105692] Updated weights for policy 0, policy_version 1536113 (0.0008) [2023-12-27 02:35:00,038][105692] Updated weights for policy 0, policy_version 1536123 (0.0009) [2023-12-27 02:35:00,092][105692] Updated weights for policy 0, policy_version 1536134 (0.0010) [2023-12-27 02:35:00,687][105620] Updated weights for policy 1, policy_version 1539116 (0.0008) [2023-12-27 02:35:00,733][105692] Updated weights for policy 0, policy_version 1536144 (0.0009) [2023-12-27 02:35:00,743][105620] Updated weights for policy 1, policy_version 1539126 (0.0008) [2023-12-27 02:35:00,789][105692] Updated weights for policy 0, policy_version 1536154 (0.0006) [2023-12-27 02:35:00,804][105620] Updated weights for policy 1, policy_version 1539136 (0.0009) [2023-12-27 02:35:00,846][105692] Updated weights for policy 0, policy_version 1536164 (0.0007) [2023-12-27 02:35:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 787390464. Throughput: 0: 9719.5, 1: 9918.0. Samples: 787357240. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:35:01,062][104569] Avg episode reward: [(0, '8808.138'), (1, '8817.557')] [2023-12-27 02:35:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001536168_393314304.pth... [2023-12-27 02:35:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001539144_394076160.pth... [2023-12-27 02:35:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001537960_393773056.pth [2023-12-27 02:35:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001535016_393019392.pth [2023-12-27 02:35:01,431][105620] Updated weights for policy 1, policy_version 1539146 (0.0008) [2023-12-27 02:35:01,489][105620] Updated weights for policy 1, policy_version 1539156 (0.0007) [2023-12-27 02:35:01,550][105620] Updated weights for policy 1, policy_version 1539166 (0.0009) [2023-12-27 02:35:01,612][105620] Updated weights for policy 1, policy_version 1539176 (0.0007) [2023-12-27 02:35:01,652][105692] Updated weights for policy 0, policy_version 1536174 (0.0008) [2023-12-27 02:35:01,719][105692] Updated weights for policy 0, policy_version 1536184 (0.0009) [2023-12-27 02:35:01,788][105692] Updated weights for policy 0, policy_version 1536194 (0.0008) [2023-12-27 02:35:02,225][105620] Updated weights for policy 1, policy_version 1539186 (0.0007) [2023-12-27 02:35:02,286][105620] Updated weights for policy 1, policy_version 1539196 (0.0007) [2023-12-27 02:35:02,343][105620] Updated weights for policy 1, policy_version 1539206 (0.0007) [2023-12-27 02:35:02,542][105692] Updated weights for policy 0, policy_version 1536204 (0.0009) [2023-12-27 02:35:02,607][105692] Updated weights for policy 0, policy_version 1536214 (0.0010) [2023-12-27 02:35:02,666][105692] Updated weights for policy 0, policy_version 1536224 (0.0010) [2023-12-27 02:35:03,016][105620] Updated weights for policy 1, policy_version 1539216 (0.0008) [2023-12-27 02:35:03,073][105620] Updated weights for policy 1, policy_version 1539226 (0.0008) [2023-12-27 02:35:03,126][105620] Updated weights for policy 1, policy_version 1539236 (0.0008) [2023-12-27 02:35:03,395][105692] Updated weights for policy 0, policy_version 1536234 (0.0010) [2023-12-27 02:35:03,439][105692] Updated weights for policy 0, policy_version 1536244 (0.0010) [2023-12-27 02:35:03,492][105692] Updated weights for policy 0, policy_version 1536254 (0.0010) [2023-12-27 02:35:03,543][105692] Updated weights for policy 0, policy_version 1536264 (0.0009) [2023-12-27 02:35:03,793][105620] Updated weights for policy 1, policy_version 1539246 (0.0008) [2023-12-27 02:35:03,853][105620] Updated weights for policy 1, policy_version 1539256 (0.0008) [2023-12-27 02:35:03,910][105620] Updated weights for policy 1, policy_version 1539266 (0.0008) [2023-12-27 02:35:04,294][105692] Updated weights for policy 0, policy_version 1536274 (0.0011) [2023-12-27 02:35:04,350][105692] Updated weights for policy 0, policy_version 1536284 (0.0010) [2023-12-27 02:35:04,409][105692] Updated weights for policy 0, policy_version 1536294 (0.0010) [2023-12-27 02:35:04,680][105620] Updated weights for policy 1, policy_version 1539276 (0.0009) [2023-12-27 02:35:04,738][105620] Updated weights for policy 1, policy_version 1539286 (0.0011) [2023-12-27 02:35:04,794][105620] Updated weights for policy 1, policy_version 1539296 (0.0007) [2023-12-27 02:35:05,123][105692] Updated weights for policy 0, policy_version 1536304 (0.0010) [2023-12-27 02:35:05,180][105692] Updated weights for policy 0, policy_version 1536314 (0.0008) [2023-12-27 02:35:05,235][105692] Updated weights for policy 0, policy_version 1536324 (0.0006) [2023-12-27 02:35:05,481][105620] Updated weights for policy 1, policy_version 1539306 (0.0007) [2023-12-27 02:35:05,526][105620] Updated weights for policy 1, policy_version 1539316 (0.0010) [2023-12-27 02:35:05,581][105620] Updated weights for policy 1, policy_version 1539326 (0.0010) [2023-12-27 02:35:05,636][105620] Updated weights for policy 1, policy_version 1539336 (0.0010) [2023-12-27 02:35:05,832][105692] Updated weights for policy 0, policy_version 1536334 (0.0008) [2023-12-27 02:35:05,892][105692] Updated weights for policy 0, policy_version 1536344 (0.0010) [2023-12-27 02:35:05,944][105692] Updated weights for policy 0, policy_version 1536354 (0.0010) [2023-12-27 02:35:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 787488768. Throughput: 0: 9661.5, 1: 9896.4. Samples: 787473336. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:35:06,062][104569] Avg episode reward: [(0, '8812.233'), (1, '9170.610')] [2023-12-27 02:35:06,316][105620] Updated weights for policy 1, policy_version 1539346 (0.0010) [2023-12-27 02:35:06,382][105620] Updated weights for policy 1, policy_version 1539356 (0.0010) [2023-12-27 02:35:06,448][105620] Updated weights for policy 1, policy_version 1539366 (0.0010) [2023-12-27 02:35:06,701][105692] Updated weights for policy 0, policy_version 1536364 (0.0008) [2023-12-27 02:35:06,769][105692] Updated weights for policy 0, policy_version 1536374 (0.0008) [2023-12-27 02:35:06,834][105692] Updated weights for policy 0, policy_version 1536384 (0.0010) [2023-12-27 02:35:07,187][105620] Updated weights for policy 1, policy_version 1539376 (0.0010) [2023-12-27 02:35:07,246][105620] Updated weights for policy 1, policy_version 1539386 (0.0010) [2023-12-27 02:35:07,308][105620] Updated weights for policy 1, policy_version 1539396 (0.0011) [2023-12-27 02:35:07,512][105692] Updated weights for policy 0, policy_version 1536394 (0.0010) [2023-12-27 02:35:07,575][105692] Updated weights for policy 0, policy_version 1536404 (0.0010) [2023-12-27 02:35:07,638][105692] Updated weights for policy 0, policy_version 1536414 (0.0011) [2023-12-27 02:35:07,697][105692] Updated weights for policy 0, policy_version 1536424 (0.0010) [2023-12-27 02:35:08,044][105620] Updated weights for policy 1, policy_version 1539406 (0.0008) [2023-12-27 02:35:08,102][105620] Updated weights for policy 1, policy_version 1539416 (0.0008) [2023-12-27 02:35:08,161][105620] Updated weights for policy 1, policy_version 1539426 (0.0008) [2023-12-27 02:35:08,433][105692] Updated weights for policy 0, policy_version 1536434 (0.0010) [2023-12-27 02:35:08,486][105692] Updated weights for policy 0, policy_version 1536444 (0.0010) [2023-12-27 02:35:08,538][105692] Updated weights for policy 0, policy_version 1536454 (0.0010) [2023-12-27 02:35:08,949][105620] Updated weights for policy 1, policy_version 1539436 (0.0008) [2023-12-27 02:35:09,009][105620] Updated weights for policy 1, policy_version 1539446 (0.0006) [2023-12-27 02:35:09,075][105620] Updated weights for policy 1, policy_version 1539456 (0.0005) [2023-12-27 02:35:09,170][105692] Updated weights for policy 0, policy_version 1536464 (0.0006) [2023-12-27 02:35:09,222][105692] Updated weights for policy 0, policy_version 1536474 (0.0006) [2023-12-27 02:35:09,280][105692] Updated weights for policy 0, policy_version 1536484 (0.0009) [2023-12-27 02:35:09,646][105620] Updated weights for policy 1, policy_version 1539466 (0.0006) [2023-12-27 02:35:09,708][105620] Updated weights for policy 1, policy_version 1539476 (0.0010) [2023-12-27 02:35:09,772][105620] Updated weights for policy 1, policy_version 1539486 (0.0011) [2023-12-27 02:35:09,844][105620] Updated weights for policy 1, policy_version 1539496 (0.0010) [2023-12-27 02:35:10,075][105692] Updated weights for policy 0, policy_version 1536494 (0.0009) [2023-12-27 02:35:10,125][105692] Updated weights for policy 0, policy_version 1536504 (0.0008) [2023-12-27 02:35:10,174][105692] Updated weights for policy 0, policy_version 1536514 (0.0008) [2023-12-27 02:35:10,609][105620] Updated weights for policy 1, policy_version 1539506 (0.0011) [2023-12-27 02:35:10,673][105620] Updated weights for policy 1, policy_version 1539516 (0.0011) [2023-12-27 02:35:10,733][105620] Updated weights for policy 1, policy_version 1539526 (0.0011) [2023-12-27 02:35:10,902][105692] Updated weights for policy 0, policy_version 1536524 (0.0007) [2023-12-27 02:35:10,949][105692] Updated weights for policy 0, policy_version 1536534 (0.0009) [2023-12-27 02:35:11,011][105692] Updated weights for policy 0, policy_version 1536544 (0.0008) [2023-12-27 02:35:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 787578880. Throughput: 0: 9698.9, 1: 9908.4. Samples: 787590464. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:35:11,063][104569] Avg episode reward: [(0, '8813.943'), (1, '9078.080')] [2023-12-27 02:35:11,516][105620] Updated weights for policy 1, policy_version 1539536 (0.0008) [2023-12-27 02:35:11,565][105620] Updated weights for policy 1, policy_version 1539546 (0.0008) [2023-12-27 02:35:11,619][105620] Updated weights for policy 1, policy_version 1539556 (0.0008) [2023-12-27 02:35:11,718][105692] Updated weights for policy 0, policy_version 1536554 (0.0008) [2023-12-27 02:35:11,786][105692] Updated weights for policy 0, policy_version 1536564 (0.0009) [2023-12-27 02:35:11,842][105692] Updated weights for policy 0, policy_version 1536574 (0.0010) [2023-12-27 02:35:11,895][105692] Updated weights for policy 0, policy_version 1536584 (0.0009) [2023-12-27 02:35:12,439][105620] Updated weights for policy 1, policy_version 1539566 (0.0009) [2023-12-27 02:35:12,506][105620] Updated weights for policy 1, policy_version 1539576 (0.0008) [2023-12-27 02:35:12,563][105620] Updated weights for policy 1, policy_version 1539586 (0.0008) [2023-12-27 02:35:12,652][105692] Updated weights for policy 0, policy_version 1536594 (0.0011) [2023-12-27 02:35:12,713][105692] Updated weights for policy 0, policy_version 1536604 (0.0011) [2023-12-27 02:35:12,758][105692] Updated weights for policy 0, policy_version 1536614 (0.0011) [2023-12-27 02:35:13,338][105620] Updated weights for policy 1, policy_version 1539596 (0.0008) [2023-12-27 02:35:13,397][105620] Updated weights for policy 1, policy_version 1539606 (0.0008) [2023-12-27 02:35:13,445][105620] Updated weights for policy 1, policy_version 1539616 (0.0007) [2023-12-27 02:35:13,519][105692] Updated weights for policy 0, policy_version 1536624 (0.0010) [2023-12-27 02:35:13,581][105692] Updated weights for policy 0, policy_version 1536634 (0.0010) [2023-12-27 02:35:13,638][105692] Updated weights for policy 0, policy_version 1536644 (0.0010) [2023-12-27 02:35:14,225][105620] Updated weights for policy 1, policy_version 1539626 (0.0007) [2023-12-27 02:35:14,270][105620] Updated weights for policy 1, policy_version 1539636 (0.0008) [2023-12-27 02:35:14,320][105620] Updated weights for policy 1, policy_version 1539646 (0.0009) [2023-12-27 02:35:14,352][105692] Updated weights for policy 0, policy_version 1536654 (0.0008) [2023-12-27 02:35:14,366][105620] Updated weights for policy 1, policy_version 1539656 (0.0007) [2023-12-27 02:35:14,408][105692] Updated weights for policy 0, policy_version 1536664 (0.0008) [2023-12-27 02:35:14,469][105692] Updated weights for policy 0, policy_version 1536674 (0.0007) [2023-12-27 02:35:15,187][105620] Updated weights for policy 1, policy_version 1539666 (0.0009) [2023-12-27 02:35:15,193][105692] Updated weights for policy 0, policy_version 1536684 (0.0008) [2023-12-27 02:35:15,233][105620] Updated weights for policy 1, policy_version 1539676 (0.0006) [2023-12-27 02:35:15,251][105692] Updated weights for policy 0, policy_version 1536694 (0.0007) [2023-12-27 02:35:15,287][105620] Updated weights for policy 1, policy_version 1539686 (0.0006) [2023-12-27 02:35:15,307][105692] Updated weights for policy 0, policy_version 1536704 (0.0007) [2023-12-27 02:35:15,973][105692] Updated weights for policy 0, policy_version 1536714 (0.0008) [2023-12-27 02:35:16,027][105692] Updated weights for policy 0, policy_version 1536724 (0.0005) [2023-12-27 02:35:16,059][105620] Updated weights for policy 1, policy_version 1539696 (0.0005) [2023-12-27 02:35:16,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 787668992. Throughput: 0: 9672.5, 1: 9900.9. Samples: 787646192. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:35:16,063][104569] Avg episode reward: [(0, '8450.770'), (1, '9169.887')] [2023-12-27 02:35:16,082][105692] Updated weights for policy 0, policy_version 1536734 (0.0005) [2023-12-27 02:35:16,117][105620] Updated weights for policy 1, policy_version 1539706 (0.0010) [2023-12-27 02:35:16,136][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001536744_393461760.pth... [2023-12-27 02:35:16,138][105692] Updated weights for policy 0, policy_version 1536744 (0.0006) [2023-12-27 02:35:16,140][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001535592_393166848.pth [2023-12-27 02:35:16,164][105620] Updated weights for policy 1, policy_version 1539716 (0.0007) [2023-12-27 02:35:16,186][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001539720_394223616.pth... [2023-12-27 02:35:16,190][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001538568_393928704.pth [2023-12-27 02:35:16,716][105692] Updated weights for policy 0, policy_version 1536754 (0.0010) [2023-12-27 02:35:16,734][105620] Updated weights for policy 1, policy_version 1539726 (0.0005) [2023-12-27 02:35:16,773][105692] Updated weights for policy 0, policy_version 1536764 (0.0009) [2023-12-27 02:35:16,796][105620] Updated weights for policy 1, policy_version 1539736 (0.0005) [2023-12-27 02:35:16,828][105692] Updated weights for policy 0, policy_version 1536774 (0.0005) [2023-12-27 02:35:16,858][105620] Updated weights for policy 1, policy_version 1539746 (0.0008) [2023-12-27 02:35:17,510][105620] Updated weights for policy 1, policy_version 1539757 (0.0009) [2023-12-27 02:35:17,520][105692] Updated weights for policy 0, policy_version 1536784 (0.0009) [2023-12-27 02:35:17,556][105620] Updated weights for policy 1, policy_version 1539767 (0.0006) [2023-12-27 02:35:17,569][105692] Updated weights for policy 0, policy_version 1536794 (0.0008) [2023-12-27 02:35:17,614][105620] Updated weights for policy 1, policy_version 1539777 (0.0007) [2023-12-27 02:35:17,620][105692] Updated weights for policy 0, policy_version 1536804 (0.0007) [2023-12-27 02:35:18,298][105620] Updated weights for policy 1, policy_version 1539787 (0.0008) [2023-12-27 02:35:18,357][105620] Updated weights for policy 1, policy_version 1539797 (0.0007) [2023-12-27 02:35:18,368][105692] Updated weights for policy 0, policy_version 1536814 (0.0006) [2023-12-27 02:35:18,417][105620] Updated weights for policy 1, policy_version 1539807 (0.0006) [2023-12-27 02:35:18,427][105692] Updated weights for policy 0, policy_version 1536824 (0.0009) [2023-12-27 02:35:18,494][105692] Updated weights for policy 0, policy_version 1536834 (0.0008) [2023-12-27 02:35:19,113][105620] Updated weights for policy 1, policy_version 1539817 (0.0006) [2023-12-27 02:35:19,171][105620] Updated weights for policy 1, policy_version 1539827 (0.0005) [2023-12-27 02:35:19,261][105620] Updated weights for policy 1, policy_version 1539837 (0.0008) [2023-12-27 02:35:19,284][105692] Updated weights for policy 0, policy_version 1536844 (0.0009) [2023-12-27 02:35:19,316][105620] Updated weights for policy 1, policy_version 1539847 (0.0006) [2023-12-27 02:35:19,350][105692] Updated weights for policy 0, policy_version 1536854 (0.0008) [2023-12-27 02:35:19,414][105692] Updated weights for policy 0, policy_version 1536864 (0.0010) [2023-12-27 02:35:19,950][105620] Updated weights for policy 1, policy_version 1539857 (0.0009) [2023-12-27 02:35:20,006][105620] Updated weights for policy 1, policy_version 1539867 (0.0008) [2023-12-27 02:35:20,074][105620] Updated weights for policy 1, policy_version 1539877 (0.0008) [2023-12-27 02:35:20,229][105692] Updated weights for policy 0, policy_version 1536874 (0.0009) [2023-12-27 02:35:20,292][105692] Updated weights for policy 0, policy_version 1536884 (0.0008) [2023-12-27 02:35:20,354][105692] Updated weights for policy 0, policy_version 1536894 (0.0008) [2023-12-27 02:35:20,414][105692] Updated weights for policy 0, policy_version 1536904 (0.0008) [2023-12-27 02:35:20,812][105620] Updated weights for policy 1, policy_version 1539887 (0.0009) [2023-12-27 02:35:20,865][105620] Updated weights for policy 1, policy_version 1539897 (0.0009) [2023-12-27 02:35:20,920][105620] Updated weights for policy 1, policy_version 1539907 (0.0009) [2023-12-27 02:35:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 787775488. Throughput: 0: 9618.6, 1: 9882.2. Samples: 787765428. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:35:21,062][104569] Avg episode reward: [(0, '8450.453'), (1, '8721.479')] [2023-12-27 02:35:21,209][105692] Updated weights for policy 0, policy_version 1536914 (0.0009) [2023-12-27 02:35:21,269][105692] Updated weights for policy 0, policy_version 1536924 (0.0010) [2023-12-27 02:35:21,335][105692] Updated weights for policy 0, policy_version 1536934 (0.0009) [2023-12-27 02:35:21,741][105620] Updated weights for policy 1, policy_version 1539917 (0.0008) [2023-12-27 02:35:21,805][105620] Updated weights for policy 1, policy_version 1539927 (0.0006) [2023-12-27 02:35:21,869][105620] Updated weights for policy 1, policy_version 1539937 (0.0005) [2023-12-27 02:35:22,125][105692] Updated weights for policy 0, policy_version 1536944 (0.0007) [2023-12-27 02:35:22,181][105692] Updated weights for policy 0, policy_version 1536954 (0.0008) [2023-12-27 02:35:22,235][105692] Updated weights for policy 0, policy_version 1536964 (0.0008) [2023-12-27 02:35:22,620][105620] Updated weights for policy 1, policy_version 1539947 (0.0008) [2023-12-27 02:35:22,665][105620] Updated weights for policy 1, policy_version 1539957 (0.0010) [2023-12-27 02:35:22,718][105620] Updated weights for policy 1, policy_version 1539967 (0.0010) [2023-12-27 02:35:23,040][105692] Updated weights for policy 0, policy_version 1536974 (0.0010) [2023-12-27 02:35:23,086][105692] Updated weights for policy 0, policy_version 1536984 (0.0010) [2023-12-27 02:35:23,154][105692] Updated weights for policy 0, policy_version 1536994 (0.0011) [2023-12-27 02:35:23,414][105620] Updated weights for policy 1, policy_version 1539977 (0.0010) [2023-12-27 02:35:23,465][105620] Updated weights for policy 1, policy_version 1539987 (0.0010) [2023-12-27 02:35:23,520][105620] Updated weights for policy 1, policy_version 1539997 (0.0010) [2023-12-27 02:35:23,582][105620] Updated weights for policy 1, policy_version 1540007 (0.0010) [2023-12-27 02:35:23,887][105692] Updated weights for policy 0, policy_version 1537004 (0.0009) [2023-12-27 02:35:23,953][105692] Updated weights for policy 0, policy_version 1537014 (0.0006) [2023-12-27 02:35:24,018][105692] Updated weights for policy 0, policy_version 1537024 (0.0006) [2023-12-27 02:35:24,343][105620] Updated weights for policy 1, policy_version 1540017 (0.0010) [2023-12-27 02:35:24,395][105620] Updated weights for policy 1, policy_version 1540027 (0.0010) [2023-12-27 02:35:24,446][105620] Updated weights for policy 1, policy_version 1540037 (0.0010) [2023-12-27 02:35:24,604][105692] Updated weights for policy 0, policy_version 1537034 (0.0009) [2023-12-27 02:35:24,665][105692] Updated weights for policy 0, policy_version 1537044 (0.0005) [2023-12-27 02:35:24,725][105692] Updated weights for policy 0, policy_version 1537054 (0.0005) [2023-12-27 02:35:24,790][105692] Updated weights for policy 0, policy_version 1537064 (0.0008) [2023-12-27 02:35:25,210][105620] Updated weights for policy 1, policy_version 1540047 (0.0010) [2023-12-27 02:35:25,257][105620] Updated weights for policy 1, policy_version 1540057 (0.0010) [2023-12-27 02:35:25,305][105620] Updated weights for policy 1, policy_version 1540067 (0.0010) [2023-12-27 02:35:25,343][105692] Updated weights for policy 0, policy_version 1537074 (0.0009) [2023-12-27 02:35:25,394][105692] Updated weights for policy 0, policy_version 1537084 (0.0010) [2023-12-27 02:35:25,441][105692] Updated weights for policy 0, policy_version 1537094 (0.0010) [2023-12-27 02:35:26,057][105620] Updated weights for policy 1, policy_version 1540077 (0.0010) [2023-12-27 02:35:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 787865600. Throughput: 0: 9614.2, 1: 9817.5. Samples: 787879540. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:35:26,062][104569] Avg episode reward: [(0, '8362.327'), (1, '8813.442')] [2023-12-27 02:35:26,119][105620] Updated weights for policy 1, policy_version 1540087 (0.0010) [2023-12-27 02:35:26,176][105692] Updated weights for policy 0, policy_version 1537104 (0.0009) [2023-12-27 02:35:26,182][105620] Updated weights for policy 1, policy_version 1540097 (0.0011) [2023-12-27 02:35:26,241][105692] Updated weights for policy 0, policy_version 1537114 (0.0009) [2023-12-27 02:35:26,300][105692] Updated weights for policy 0, policy_version 1537124 (0.0006) [2023-12-27 02:35:26,882][105620] Updated weights for policy 1, policy_version 1540107 (0.0009) [2023-12-27 02:35:26,916][105692] Updated weights for policy 0, policy_version 1537134 (0.0011) [2023-12-27 02:35:26,929][105620] Updated weights for policy 1, policy_version 1540117 (0.0010) [2023-12-27 02:35:26,964][105692] Updated weights for policy 0, policy_version 1537144 (0.0010) [2023-12-27 02:35:26,977][105620] Updated weights for policy 1, policy_version 1540127 (0.0010) [2023-12-27 02:35:27,015][105692] Updated weights for policy 0, policy_version 1537154 (0.0010) [2023-12-27 02:35:27,639][105620] Updated weights for policy 1, policy_version 1540137 (0.0010) [2023-12-27 02:35:27,642][105692] Updated weights for policy 0, policy_version 1537164 (0.0010) [2023-12-27 02:35:27,693][105620] Updated weights for policy 1, policy_version 1540147 (0.0007) [2023-12-27 02:35:27,702][105692] Updated weights for policy 0, policy_version 1537174 (0.0008) [2023-12-27 02:35:27,743][105620] Updated weights for policy 1, policy_version 1540157 (0.0006) [2023-12-27 02:35:27,762][105692] Updated weights for policy 0, policy_version 1537184 (0.0011) [2023-12-27 02:35:27,804][105620] Updated weights for policy 1, policy_version 1540167 (0.0005) [2023-12-27 02:35:28,428][105620] Updated weights for policy 1, policy_version 1540177 (0.0007) [2023-12-27 02:35:28,486][105620] Updated weights for policy 1, policy_version 1540187 (0.0010) [2023-12-27 02:35:28,493][105692] Updated weights for policy 0, policy_version 1537194 (0.0011) [2023-12-27 02:35:28,549][105620] Updated weights for policy 1, policy_version 1540197 (0.0010) [2023-12-27 02:35:28,549][105692] Updated weights for policy 0, policy_version 1537204 (0.0011) [2023-12-27 02:35:28,609][105692] Updated weights for policy 0, policy_version 1537214 (0.0010) [2023-12-27 02:35:28,668][105692] Updated weights for policy 0, policy_version 1537224 (0.0010) [2023-12-27 02:35:29,249][105620] Updated weights for policy 1, policy_version 1540207 (0.0009) [2023-12-27 02:35:29,297][105692] Updated weights for policy 0, policy_version 1537234 (0.0006) [2023-12-27 02:35:29,303][105620] Updated weights for policy 1, policy_version 1540217 (0.0011) [2023-12-27 02:35:29,364][105692] Updated weights for policy 0, policy_version 1537244 (0.0007) [2023-12-27 02:35:29,366][105620] Updated weights for policy 1, policy_version 1540227 (0.0011) [2023-12-27 02:35:29,427][105692] Updated weights for policy 0, policy_version 1537254 (0.0006) [2023-12-27 02:35:29,965][105692] Updated weights for policy 0, policy_version 1537264 (0.0008) [2023-12-27 02:35:30,023][105692] Updated weights for policy 0, policy_version 1537274 (0.0010) [2023-12-27 02:35:30,068][105692] Updated weights for policy 0, policy_version 1537284 (0.0010) [2023-12-27 02:35:30,092][105620] Updated weights for policy 1, policy_version 1540237 (0.0008) [2023-12-27 02:35:30,146][105620] Updated weights for policy 1, policy_version 1540247 (0.0005) [2023-12-27 02:35:30,211][105620] Updated weights for policy 1, policy_version 1540257 (0.0010) [2023-12-27 02:35:30,662][105692] Updated weights for policy 0, policy_version 1537294 (0.0010) [2023-12-27 02:35:30,719][105692] Updated weights for policy 0, policy_version 1537304 (0.0010) [2023-12-27 02:35:30,774][105692] Updated weights for policy 0, policy_version 1537314 (0.0010) [2023-12-27 02:35:30,795][105620] Updated weights for policy 1, policy_version 1540267 (0.0007) [2023-12-27 02:35:30,854][105620] Updated weights for policy 1, policy_version 1540277 (0.0007) [2023-12-27 02:35:30,905][105620] Updated weights for policy 1, policy_version 1540287 (0.0010) [2023-12-27 02:35:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 787980288. Throughput: 0: 9724.6, 1: 9823.7. Samples: 787941388. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:35:31,063][104569] Avg episode reward: [(0, '8179.624'), (1, '9266.806')] [2023-12-27 02:35:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001537320_393609216.pth... [2023-12-27 02:35:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001540296_394371072.pth... [2023-12-27 02:35:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001539144_394076160.pth [2023-12-27 02:35:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001536168_393314304.pth [2023-12-27 02:35:31,513][105692] Updated weights for policy 0, policy_version 1537324 (0.0010) [2023-12-27 02:35:31,572][105692] Updated weights for policy 0, policy_version 1537334 (0.0010) [2023-12-27 02:35:31,577][105620] Updated weights for policy 1, policy_version 1540297 (0.0010) [2023-12-27 02:35:31,624][105692] Updated weights for policy 0, policy_version 1537344 (0.0010) [2023-12-27 02:35:31,638][105620] Updated weights for policy 1, policy_version 1540307 (0.0007) [2023-12-27 02:35:31,701][105620] Updated weights for policy 1, policy_version 1540317 (0.0007) [2023-12-27 02:35:31,763][105620] Updated weights for policy 1, policy_version 1540327 (0.0007) [2023-12-27 02:35:32,309][105692] Updated weights for policy 0, policy_version 1537354 (0.0010) [2023-12-27 02:35:32,361][105692] Updated weights for policy 0, policy_version 1537364 (0.0009) [2023-12-27 02:35:32,427][105692] Updated weights for policy 0, policy_version 1537374 (0.0010) [2023-12-27 02:35:32,492][105692] Updated weights for policy 0, policy_version 1537384 (0.0009) [2023-12-27 02:35:32,513][105620] Updated weights for policy 1, policy_version 1540337 (0.0007) [2023-12-27 02:35:32,565][105620] Updated weights for policy 1, policy_version 1540347 (0.0009) [2023-12-27 02:35:32,613][105620] Updated weights for policy 1, policy_version 1540357 (0.0009) [2023-12-27 02:35:33,253][105620] Updated weights for policy 1, policy_version 1540367 (0.0006) [2023-12-27 02:35:33,254][105692] Updated weights for policy 0, policy_version 1537394 (0.0005) [2023-12-27 02:35:33,315][105692] Updated weights for policy 0, policy_version 1537404 (0.0005) [2023-12-27 02:35:33,317][105620] Updated weights for policy 1, policy_version 1540377 (0.0005) [2023-12-27 02:35:33,362][105692] Updated weights for policy 0, policy_version 1537414 (0.0007) [2023-12-27 02:35:33,378][105620] Updated weights for policy 1, policy_version 1540387 (0.0007) [2023-12-27 02:35:33,900][105620] Updated weights for policy 1, policy_version 1540397 (0.0007) [2023-12-27 02:35:33,942][105692] Updated weights for policy 0, policy_version 1537424 (0.0006) [2023-12-27 02:35:33,949][105620] Updated weights for policy 1, policy_version 1540407 (0.0008) [2023-12-27 02:35:33,987][105692] Updated weights for policy 0, policy_version 1537434 (0.0007) [2023-12-27 02:35:33,997][105620] Updated weights for policy 1, policy_version 1540417 (0.0007) [2023-12-27 02:35:34,030][105692] Updated weights for policy 0, policy_version 1537444 (0.0007) [2023-12-27 02:35:34,630][105692] Updated weights for policy 0, policy_version 1537454 (0.0005) [2023-12-27 02:35:34,688][105692] Updated weights for policy 0, policy_version 1537464 (0.0006) [2023-12-27 02:35:34,754][105692] Updated weights for policy 0, policy_version 1537474 (0.0008) [2023-12-27 02:35:34,867][105620] Updated weights for policy 1, policy_version 1540427 (0.0007) [2023-12-27 02:35:34,931][105620] Updated weights for policy 1, policy_version 1540437 (0.0010) [2023-12-27 02:35:34,999][105620] Updated weights for policy 1, policy_version 1540447 (0.0010) [2023-12-27 02:35:35,376][105692] Updated weights for policy 0, policy_version 1537484 (0.0010) [2023-12-27 02:35:35,430][105692] Updated weights for policy 0, policy_version 1537494 (0.0010) [2023-12-27 02:35:35,489][105692] Updated weights for policy 0, policy_version 1537504 (0.0011) [2023-12-27 02:35:35,672][105620] Updated weights for policy 1, policy_version 1540457 (0.0010) [2023-12-27 02:35:35,723][105620] Updated weights for policy 1, policy_version 1540467 (0.0006) [2023-12-27 02:35:35,780][105620] Updated weights for policy 1, policy_version 1540477 (0.0008) [2023-12-27 02:35:35,838][105620] Updated weights for policy 1, policy_version 1540487 (0.0008) [2023-12-27 02:35:36,062][104569] Fps is (10 sec: 21298.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 788078592. Throughput: 0: 9832.8, 1: 9786.0. Samples: 788065924. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:35:36,063][104569] Avg episode reward: [(0, '7991.388'), (1, '8809.763')] [2023-12-27 02:35:36,139][105692] Updated weights for policy 0, policy_version 1537514 (0.0009) [2023-12-27 02:35:36,195][105692] Updated weights for policy 0, policy_version 1537524 (0.0006) [2023-12-27 02:35:36,247][105692] Updated weights for policy 0, policy_version 1537534 (0.0010) [2023-12-27 02:35:36,302][105692] Updated weights for policy 0, policy_version 1537544 (0.0009) [2023-12-27 02:35:36,511][105620] Updated weights for policy 1, policy_version 1540497 (0.0009) [2023-12-27 02:35:36,568][105620] Updated weights for policy 1, policy_version 1540507 (0.0010) [2023-12-27 02:35:36,629][105620] Updated weights for policy 1, policy_version 1540517 (0.0011) [2023-12-27 02:35:36,994][105692] Updated weights for policy 0, policy_version 1537554 (0.0010) [2023-12-27 02:35:37,048][105692] Updated weights for policy 0, policy_version 1537564 (0.0008) [2023-12-27 02:35:37,110][105692] Updated weights for policy 0, policy_version 1537574 (0.0010) [2023-12-27 02:35:37,384][105620] Updated weights for policy 1, policy_version 1540527 (0.0008) [2023-12-27 02:35:37,455][105620] Updated weights for policy 1, policy_version 1540537 (0.0006) [2023-12-27 02:35:37,511][105620] Updated weights for policy 1, policy_version 1540547 (0.0006) [2023-12-27 02:35:37,708][105692] Updated weights for policy 0, policy_version 1537584 (0.0010) [2023-12-27 02:35:37,763][105692] Updated weights for policy 0, policy_version 1537594 (0.0010) [2023-12-27 02:35:37,808][105692] Updated weights for policy 0, policy_version 1537604 (0.0010) [2023-12-27 02:35:38,139][105620] Updated weights for policy 1, policy_version 1540557 (0.0007) [2023-12-27 02:35:38,192][105620] Updated weights for policy 1, policy_version 1540567 (0.0008) [2023-12-27 02:35:38,241][105620] Updated weights for policy 1, policy_version 1540577 (0.0008) [2023-12-27 02:35:38,586][105692] Updated weights for policy 0, policy_version 1537614 (0.0008) [2023-12-27 02:35:38,656][105692] Updated weights for policy 0, policy_version 1537624 (0.0005) [2023-12-27 02:35:38,718][105692] Updated weights for policy 0, policy_version 1537634 (0.0006) [2023-12-27 02:35:39,124][105620] Updated weights for policy 1, policy_version 1540587 (0.0007) [2023-12-27 02:35:39,185][105620] Updated weights for policy 1, policy_version 1540597 (0.0009) [2023-12-27 02:35:39,245][105620] Updated weights for policy 1, policy_version 1540607 (0.0008) [2023-12-27 02:35:39,298][105692] Updated weights for policy 0, policy_version 1537644 (0.0007) [2023-12-27 02:35:39,371][105692] Updated weights for policy 0, policy_version 1537654 (0.0007) [2023-12-27 02:35:39,440][105692] Updated weights for policy 0, policy_version 1537664 (0.0008) [2023-12-27 02:35:40,033][105620] Updated weights for policy 1, policy_version 1540617 (0.0008) [2023-12-27 02:35:40,100][105620] Updated weights for policy 1, policy_version 1540627 (0.0008) [2023-12-27 02:35:40,158][105620] Updated weights for policy 1, policy_version 1540637 (0.0009) [2023-12-27 02:35:40,211][105692] Updated weights for policy 0, policy_version 1537674 (0.0008) [2023-12-27 02:35:40,223][105620] Updated weights for policy 1, policy_version 1540647 (0.0009) [2023-12-27 02:35:40,274][105692] Updated weights for policy 0, policy_version 1537684 (0.0008) [2023-12-27 02:35:40,334][105692] Updated weights for policy 0, policy_version 1537694 (0.0009) [2023-12-27 02:35:40,398][105692] Updated weights for policy 0, policy_version 1537704 (0.0010) [2023-12-27 02:35:40,949][105620] Updated weights for policy 1, policy_version 1540657 (0.0005) [2023-12-27 02:35:41,018][105620] Updated weights for policy 1, policy_version 1540667 (0.0006) [2023-12-27 02:35:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 788168704. Throughput: 0: 9916.2, 1: 9750.7. Samples: 788184368. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:35:41,063][104569] Avg episode reward: [(0, '8449.070'), (1, '8450.891')] [2023-12-27 02:35:41,083][105620] Updated weights for policy 1, policy_version 1540677 (0.0008) [2023-12-27 02:35:41,155][105692] Updated weights for policy 0, policy_version 1537714 (0.0008) [2023-12-27 02:35:41,218][105692] Updated weights for policy 0, policy_version 1537724 (0.0009) [2023-12-27 02:35:41,283][105692] Updated weights for policy 0, policy_version 1537734 (0.0007) [2023-12-27 02:35:41,798][105620] Updated weights for policy 1, policy_version 1540687 (0.0009) [2023-12-27 02:35:41,857][105620] Updated weights for policy 1, policy_version 1540697 (0.0009) [2023-12-27 02:35:41,910][105620] Updated weights for policy 1, policy_version 1540707 (0.0010) [2023-12-27 02:35:42,032][105692] Updated weights for policy 0, policy_version 1537744 (0.0007) [2023-12-27 02:35:42,097][105692] Updated weights for policy 0, policy_version 1537754 (0.0008) [2023-12-27 02:35:42,165][105692] Updated weights for policy 0, policy_version 1537764 (0.0008) [2023-12-27 02:35:42,647][105620] Updated weights for policy 1, policy_version 1540717 (0.0009) [2023-12-27 02:35:42,714][105620] Updated weights for policy 1, policy_version 1540727 (0.0009) [2023-12-27 02:35:42,780][105620] Updated weights for policy 1, policy_version 1540737 (0.0010) [2023-12-27 02:35:42,830][105692] Updated weights for policy 0, policy_version 1537774 (0.0007) [2023-12-27 02:35:42,890][105692] Updated weights for policy 0, policy_version 1537784 (0.0005) [2023-12-27 02:35:42,946][105692] Updated weights for policy 0, policy_version 1537794 (0.0005) [2023-12-27 02:35:43,557][105692] Updated weights for policy 0, policy_version 1537804 (0.0007) [2023-12-27 02:35:43,600][105620] Updated weights for policy 1, policy_version 1540747 (0.0008) [2023-12-27 02:35:43,621][105692] Updated weights for policy 0, policy_version 1537814 (0.0006) [2023-12-27 02:35:43,654][105620] Updated weights for policy 1, policy_version 1540757 (0.0007) [2023-12-27 02:35:43,678][105692] Updated weights for policy 0, policy_version 1537824 (0.0008) [2023-12-27 02:35:43,709][105620] Updated weights for policy 1, policy_version 1540767 (0.0008) [2023-12-27 02:35:44,366][105692] Updated weights for policy 0, policy_version 1537834 (0.0006) [2023-12-27 02:35:44,419][105692] Updated weights for policy 0, policy_version 1537844 (0.0005) [2023-12-27 02:35:44,470][105692] Updated weights for policy 0, policy_version 1537854 (0.0007) [2023-12-27 02:35:44,501][105620] Updated weights for policy 1, policy_version 1540777 (0.0008) [2023-12-27 02:35:44,523][105692] Updated weights for policy 0, policy_version 1537864 (0.0005) [2023-12-27 02:35:44,560][105620] Updated weights for policy 1, policy_version 1540787 (0.0009) [2023-12-27 02:35:44,613][105620] Updated weights for policy 1, policy_version 1540797 (0.0009) [2023-12-27 02:35:44,674][105620] Updated weights for policy 1, policy_version 1540807 (0.0010) [2023-12-27 02:35:45,147][105692] Updated weights for policy 0, policy_version 1537874 (0.0009) [2023-12-27 02:35:45,210][105692] Updated weights for policy 0, policy_version 1537884 (0.0008) [2023-12-27 02:35:45,272][105692] Updated weights for policy 0, policy_version 1537894 (0.0005) [2023-12-27 02:35:45,432][105620] Updated weights for policy 1, policy_version 1540817 (0.0009) [2023-12-27 02:35:45,499][105620] Updated weights for policy 1, policy_version 1540827 (0.0009) [2023-12-27 02:35:45,562][105620] Updated weights for policy 1, policy_version 1540837 (0.0008) [2023-12-27 02:35:45,959][105692] Updated weights for policy 0, policy_version 1537904 (0.0009) [2023-12-27 02:35:46,021][105692] Updated weights for policy 0, policy_version 1537915 (0.0010) [2023-12-27 02:35:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.4, 300 sec: 19549.7). Total num frames: 788267008. Throughput: 0: 9938.3, 1: 9699.5. Samples: 788240944. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:35:46,063][104569] Avg episode reward: [(0, '8725.780'), (1, '8903.370')] [2023-12-27 02:35:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001540840_394510336.pth... [2023-12-27 02:35:46,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001539720_394223616.pth [2023-12-27 02:35:46,080][105692] Updated weights for policy 0, policy_version 1537925 (0.0010) [2023-12-27 02:35:46,098][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001537928_393764864.pth... [2023-12-27 02:35:46,103][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001536744_393461760.pth [2023-12-27 02:35:46,214][105620] Updated weights for policy 1, policy_version 1540847 (0.0007) [2023-12-27 02:35:46,264][105620] Updated weights for policy 1, policy_version 1540857 (0.0008) [2023-12-27 02:35:46,318][105620] Updated weights for policy 1, policy_version 1540867 (0.0009) [2023-12-27 02:35:46,808][105692] Updated weights for policy 0, policy_version 1537935 (0.0009) [2023-12-27 02:35:46,869][105692] Updated weights for policy 0, policy_version 1537945 (0.0008) [2023-12-27 02:35:46,930][105692] Updated weights for policy 0, policy_version 1537955 (0.0008) [2023-12-27 02:35:47,026][105620] Updated weights for policy 1, policy_version 1540877 (0.0008) [2023-12-27 02:35:47,082][105620] Updated weights for policy 1, policy_version 1540887 (0.0005) [2023-12-27 02:35:47,134][105620] Updated weights for policy 1, policy_version 1540897 (0.0006) [2023-12-27 02:35:47,733][105620] Updated weights for policy 1, policy_version 1540907 (0.0008) [2023-12-27 02:35:47,739][105692] Updated weights for policy 0, policy_version 1537965 (0.0008) [2023-12-27 02:35:47,784][105620] Updated weights for policy 1, policy_version 1540917 (0.0006) [2023-12-27 02:35:47,793][105692] Updated weights for policy 0, policy_version 1537975 (0.0009) [2023-12-27 02:35:47,844][105620] Updated weights for policy 1, policy_version 1540927 (0.0008) [2023-12-27 02:35:47,847][105692] Updated weights for policy 0, policy_version 1537985 (0.0006) [2023-12-27 02:35:48,511][105620] Updated weights for policy 1, policy_version 1540937 (0.0010) [2023-12-27 02:35:48,571][105620] Updated weights for policy 1, policy_version 1540947 (0.0005) [2023-12-27 02:35:48,628][105620] Updated weights for policy 1, policy_version 1540957 (0.0006) [2023-12-27 02:35:48,666][105692] Updated weights for policy 0, policy_version 1537995 (0.0010) [2023-12-27 02:35:48,689][105620] Updated weights for policy 1, policy_version 1540967 (0.0007) [2023-12-27 02:35:48,733][105692] Updated weights for policy 0, policy_version 1538005 (0.0009) [2023-12-27 02:35:48,786][105692] Updated weights for policy 0, policy_version 1538015 (0.0008) [2023-12-27 02:35:49,399][105620] Updated weights for policy 1, policy_version 1540977 (0.0008) [2023-12-27 02:35:49,462][105620] Updated weights for policy 1, policy_version 1540987 (0.0009) [2023-12-27 02:35:49,524][105620] Updated weights for policy 1, policy_version 1540997 (0.0009) [2023-12-27 02:35:49,527][105692] Updated weights for policy 0, policy_version 1538025 (0.0008) [2023-12-27 02:35:49,580][105692] Updated weights for policy 0, policy_version 1538035 (0.0007) [2023-12-27 02:35:49,643][105692] Updated weights for policy 0, policy_version 1538045 (0.0009) [2023-12-27 02:35:49,702][105692] Updated weights for policy 0, policy_version 1538055 (0.0009) [2023-12-27 02:35:50,267][105620] Updated weights for policy 1, policy_version 1541007 (0.0008) [2023-12-27 02:35:50,312][105620] Updated weights for policy 1, policy_version 1541017 (0.0010) [2023-12-27 02:35:50,365][105620] Updated weights for policy 1, policy_version 1541027 (0.0008) [2023-12-27 02:35:50,500][105692] Updated weights for policy 0, policy_version 1538065 (0.0009) [2023-12-27 02:35:50,566][105692] Updated weights for policy 0, policy_version 1538075 (0.0008) [2023-12-27 02:35:50,629][105692] Updated weights for policy 0, policy_version 1538085 (0.0008) [2023-12-27 02:35:51,061][105620] Updated weights for policy 1, policy_version 1541037 (0.0009) [2023-12-27 02:35:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 788365312. Throughput: 0: 9959.2, 1: 9696.4. Samples: 788357840. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:35:51,062][104569] Avg episode reward: [(0, '8628.822'), (1, '8538.892')] [2023-12-27 02:35:51,127][105620] Updated weights for policy 1, policy_version 1541047 (0.0009) [2023-12-27 02:35:51,186][105620] Updated weights for policy 1, policy_version 1541057 (0.0008) [2023-12-27 02:35:51,444][105692] Updated weights for policy 0, policy_version 1538095 (0.0008) [2023-12-27 02:35:51,511][105692] Updated weights for policy 0, policy_version 1538105 (0.0008) [2023-12-27 02:35:51,577][105692] Updated weights for policy 0, policy_version 1538115 (0.0007) [2023-12-27 02:35:51,903][105620] Updated weights for policy 1, policy_version 1541067 (0.0006) [2023-12-27 02:35:51,955][105620] Updated weights for policy 1, policy_version 1541077 (0.0007) [2023-12-27 02:35:52,018][105620] Updated weights for policy 1, policy_version 1541087 (0.0008) [2023-12-27 02:35:52,309][105692] Updated weights for policy 0, policy_version 1538125 (0.0007) [2023-12-27 02:35:52,371][105692] Updated weights for policy 0, policy_version 1538135 (0.0009) [2023-12-27 02:35:52,433][105692] Updated weights for policy 0, policy_version 1538145 (0.0009) [2023-12-27 02:35:52,701][105620] Updated weights for policy 1, policy_version 1541097 (0.0007) [2023-12-27 02:35:52,757][105620] Updated weights for policy 1, policy_version 1541107 (0.0006) [2023-12-27 02:35:52,801][105586] KL-divergence is very high: 117.7363 [2023-12-27 02:35:52,818][105620] Updated weights for policy 1, policy_version 1541117 (0.0006) [2023-12-27 02:35:52,855][105586] KL-divergence is very high: 123.1581 [2023-12-27 02:35:52,886][105620] Updated weights for policy 1, policy_version 1541127 (0.0008) [2023-12-27 02:35:53,263][105692] Updated weights for policy 0, policy_version 1538155 (0.0010) [2023-12-27 02:35:53,315][105692] Updated weights for policy 0, policy_version 1538165 (0.0010) [2023-12-27 02:35:53,373][105692] Updated weights for policy 0, policy_version 1538175 (0.0010) [2023-12-27 02:35:53,491][105620] Updated weights for policy 1, policy_version 1541137 (0.0006) [2023-12-27 02:35:53,551][105620] Updated weights for policy 1, policy_version 1541147 (0.0005) [2023-12-27 02:35:53,601][105620] Updated weights for policy 1, policy_version 1541157 (0.0006) [2023-12-27 02:35:54,173][105620] Updated weights for policy 1, policy_version 1541167 (0.0006) [2023-12-27 02:35:54,222][105620] Updated weights for policy 1, policy_version 1541177 (0.0005) [2023-12-27 02:35:54,246][105692] Updated weights for policy 0, policy_version 1538185 (0.0009) [2023-12-27 02:35:54,273][105620] Updated weights for policy 1, policy_version 1541187 (0.0005) [2023-12-27 02:35:54,302][105692] Updated weights for policy 0, policy_version 1538195 (0.0009) [2023-12-27 02:35:54,358][105692] Updated weights for policy 0, policy_version 1538205 (0.0009) [2023-12-27 02:35:54,422][105692] Updated weights for policy 0, policy_version 1538215 (0.0010) [2023-12-27 02:35:54,929][105620] Updated weights for policy 1, policy_version 1541197 (0.0007) [2023-12-27 02:35:54,991][105620] Updated weights for policy 1, policy_version 1541207 (0.0009) [2023-12-27 02:35:55,043][105620] Updated weights for policy 1, policy_version 1541217 (0.0009) [2023-12-27 02:35:55,213][105692] Updated weights for policy 0, policy_version 1538225 (0.0009) [2023-12-27 02:35:55,265][105692] Updated weights for policy 0, policy_version 1538235 (0.0009) [2023-12-27 02:35:55,321][105692] Updated weights for policy 0, policy_version 1538245 (0.0009) [2023-12-27 02:35:55,784][105620] Updated weights for policy 1, policy_version 1541227 (0.0008) [2023-12-27 02:35:55,845][105620] Updated weights for policy 1, policy_version 1541237 (0.0005) [2023-12-27 02:35:55,902][105620] Updated weights for policy 1, policy_version 1541247 (0.0005) [2023-12-27 02:35:56,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 788463616. Throughput: 0: 9805.0, 1: 9790.7. Samples: 788472268. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:35:56,062][104569] Avg episode reward: [(0, '7904.233'), (1, '8180.621')] [2023-12-27 02:35:56,137][105692] Updated weights for policy 0, policy_version 1538255 (0.0009) [2023-12-27 02:35:56,191][105692] Updated weights for policy 0, policy_version 1538265 (0.0009) [2023-12-27 02:35:56,243][105692] Updated weights for policy 0, policy_version 1538275 (0.0009) [2023-12-27 02:35:56,558][105620] Updated weights for policy 1, policy_version 1541257 (0.0006) [2023-12-27 02:35:56,604][105620] Updated weights for policy 1, policy_version 1541267 (0.0010) [2023-12-27 02:35:56,655][105620] Updated weights for policy 1, policy_version 1541277 (0.0009) [2023-12-27 02:35:56,712][105620] Updated weights for policy 1, policy_version 1541287 (0.0008) [2023-12-27 02:35:57,081][105692] Updated weights for policy 0, policy_version 1538285 (0.0009) [2023-12-27 02:35:57,135][105692] Updated weights for policy 0, policy_version 1538295 (0.0009) [2023-12-27 02:35:57,190][105692] Updated weights for policy 0, policy_version 1538305 (0.0009) [2023-12-27 02:35:57,322][105620] Updated weights for policy 1, policy_version 1541297 (0.0007) [2023-12-27 02:35:57,388][105620] Updated weights for policy 1, policy_version 1541307 (0.0006) [2023-12-27 02:35:57,449][105620] Updated weights for policy 1, policy_version 1541317 (0.0007) [2023-12-27 02:35:57,865][105692] Updated weights for policy 0, policy_version 1538315 (0.0009) [2023-12-27 02:35:57,913][105692] Updated weights for policy 0, policy_version 1538325 (0.0009) [2023-12-27 02:35:57,966][105692] Updated weights for policy 0, policy_version 1538335 (0.0008) [2023-12-27 02:35:58,108][105620] Updated weights for policy 1, policy_version 1541327 (0.0005) [2023-12-27 02:35:58,168][105620] Updated weights for policy 1, policy_version 1541337 (0.0007) [2023-12-27 02:35:58,231][105620] Updated weights for policy 1, policy_version 1541347 (0.0009) [2023-12-27 02:35:58,773][105692] Updated weights for policy 0, policy_version 1538345 (0.0009) [2023-12-27 02:35:58,840][105692] Updated weights for policy 0, policy_version 1538355 (0.0009) [2023-12-27 02:35:58,901][105692] Updated weights for policy 0, policy_version 1538365 (0.0009) [2023-12-27 02:35:58,961][105692] Updated weights for policy 0, policy_version 1538375 (0.0010) [2023-12-27 02:35:58,999][105620] Updated weights for policy 1, policy_version 1541357 (0.0007) [2023-12-27 02:35:59,046][105620] Updated weights for policy 1, policy_version 1541367 (0.0006) [2023-12-27 02:35:59,101][105620] Updated weights for policy 1, policy_version 1541377 (0.0009) [2023-12-27 02:35:59,734][105692] Updated weights for policy 0, policy_version 1538385 (0.0007) [2023-12-27 02:35:59,769][105620] Updated weights for policy 1, policy_version 1541387 (0.0009) [2023-12-27 02:35:59,791][105692] Updated weights for policy 0, policy_version 1538395 (0.0005) [2023-12-27 02:35:59,819][105620] Updated weights for policy 1, policy_version 1541397 (0.0008) [2023-12-27 02:35:59,854][105692] Updated weights for policy 0, policy_version 1538405 (0.0008) [2023-12-27 02:35:59,889][105620] Updated weights for policy 1, policy_version 1541407 (0.0008) [2023-12-27 02:36:00,578][105620] Updated weights for policy 1, policy_version 1541417 (0.0010) [2023-12-27 02:36:00,587][105692] Updated weights for policy 0, policy_version 1538415 (0.0008) [2023-12-27 02:36:00,626][105620] Updated weights for policy 1, policy_version 1541427 (0.0007) [2023-12-27 02:36:00,640][105692] Updated weights for policy 0, policy_version 1538425 (0.0007) [2023-12-27 02:36:00,677][105620] Updated weights for policy 1, policy_version 1541437 (0.0006) [2023-12-27 02:36:00,696][105692] Updated weights for policy 0, policy_version 1538435 (0.0006) [2023-12-27 02:36:00,716][105585] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000008 [2023-12-27 02:36:00,726][105620] Updated weights for policy 1, policy_version 1541447 (0.0006) [2023-12-27 02:36:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 788561920. Throughput: 0: 9774.4, 1: 9867.7. Samples: 788530084. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:36:01,062][104569] Avg episode reward: [(0, '7900.092'), (1, '8538.860')] [2023-12-27 02:36:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001538440_393895936.pth... [2023-12-27 02:36:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001541448_394665984.pth... [2023-12-27 02:36:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001537320_393609216.pth [2023-12-27 02:36:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001540296_394371072.pth [2023-12-27 02:36:01,367][105692] Updated weights for policy 0, policy_version 1538445 (0.0007) [2023-12-27 02:36:01,438][105692] Updated weights for policy 0, policy_version 1538455 (0.0006) [2023-12-27 02:36:01,499][105620] Updated weights for policy 1, policy_version 1541457 (0.0008) [2023-12-27 02:36:01,506][105692] Updated weights for policy 0, policy_version 1538465 (0.0007) [2023-12-27 02:36:01,559][105620] Updated weights for policy 1, policy_version 1541467 (0.0007) [2023-12-27 02:36:01,618][105620] Updated weights for policy 1, policy_version 1541477 (0.0006) [2023-12-27 02:36:02,112][105692] Updated weights for policy 0, policy_version 1538475 (0.0009) [2023-12-27 02:36:02,169][105692] Updated weights for policy 0, policy_version 1538485 (0.0010) [2023-12-27 02:36:02,223][105620] Updated weights for policy 1, policy_version 1541487 (0.0007) [2023-12-27 02:36:02,223][105692] Updated weights for policy 0, policy_version 1538495 (0.0009) [2023-12-27 02:36:02,283][105620] Updated weights for policy 1, policy_version 1541497 (0.0006) [2023-12-27 02:36:02,343][105620] Updated weights for policy 1, policy_version 1541507 (0.0007) [2023-12-27 02:36:02,995][105692] Updated weights for policy 0, policy_version 1538505 (0.0009) [2023-12-27 02:36:03,044][105692] Updated weights for policy 0, policy_version 1538515 (0.0010) [2023-12-27 02:36:03,066][105620] Updated weights for policy 1, policy_version 1541517 (0.0008) [2023-12-27 02:36:03,099][105692] Updated weights for policy 0, policy_version 1538525 (0.0007) [2023-12-27 02:36:03,121][105620] Updated weights for policy 1, policy_version 1541527 (0.0008) [2023-12-27 02:36:03,147][105692] Updated weights for policy 0, policy_version 1538535 (0.0005) [2023-12-27 02:36:03,175][105620] Updated weights for policy 1, policy_version 1541537 (0.0008) [2023-12-27 02:36:03,805][105692] Updated weights for policy 0, policy_version 1538545 (0.0010) [2023-12-27 02:36:03,871][105692] Updated weights for policy 0, policy_version 1538555 (0.0011) [2023-12-27 02:36:03,925][105692] Updated weights for policy 0, policy_version 1538565 (0.0011) [2023-12-27 02:36:03,997][105620] Updated weights for policy 1, policy_version 1541547 (0.0009) [2023-12-27 02:36:04,053][105620] Updated weights for policy 1, policy_version 1541557 (0.0009) [2023-12-27 02:36:04,110][105620] Updated weights for policy 1, policy_version 1541567 (0.0009) [2023-12-27 02:36:04,598][105692] Updated weights for policy 0, policy_version 1538575 (0.0011) [2023-12-27 02:36:04,662][105692] Updated weights for policy 0, policy_version 1538585 (0.0010) [2023-12-27 02:36:04,721][105692] Updated weights for policy 0, policy_version 1538595 (0.0011) [2023-12-27 02:36:04,820][105620] Updated weights for policy 1, policy_version 1541577 (0.0009) [2023-12-27 02:36:04,869][105620] Updated weights for policy 1, policy_version 1541587 (0.0010) [2023-12-27 02:36:04,915][105620] Updated weights for policy 1, policy_version 1541597 (0.0010) [2023-12-27 02:36:04,964][105620] Updated weights for policy 1, policy_version 1541607 (0.0010) [2023-12-27 02:36:05,303][105692] Updated weights for policy 0, policy_version 1538605 (0.0008) [2023-12-27 02:36:05,367][105692] Updated weights for policy 0, policy_version 1538615 (0.0008) [2023-12-27 02:36:05,415][105692] Updated weights for policy 0, policy_version 1538625 (0.0010) [2023-12-27 02:36:05,627][105620] Updated weights for policy 1, policy_version 1541617 (0.0009) [2023-12-27 02:36:05,694][105620] Updated weights for policy 1, policy_version 1541627 (0.0007) [2023-12-27 02:36:05,755][105620] Updated weights for policy 1, policy_version 1541637 (0.0005) [2023-12-27 02:36:06,007][105692] Updated weights for policy 0, policy_version 1538635 (0.0010) [2023-12-27 02:36:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 788660224. Throughput: 0: 9783.0, 1: 9835.6. Samples: 788648268. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:36:06,063][104569] Avg episode reward: [(0, '8623.650'), (1, '8990.325')] [2023-12-27 02:36:06,069][105692] Updated weights for policy 0, policy_version 1538645 (0.0009) [2023-12-27 02:36:06,133][105692] Updated weights for policy 0, policy_version 1538655 (0.0008) [2023-12-27 02:36:06,446][105620] Updated weights for policy 1, policy_version 1541647 (0.0008) [2023-12-27 02:36:06,504][105620] Updated weights for policy 1, policy_version 1541657 (0.0009) [2023-12-27 02:36:06,558][105620] Updated weights for policy 1, policy_version 1541667 (0.0009) [2023-12-27 02:36:06,809][105692] Updated weights for policy 0, policy_version 1538665 (0.0008) [2023-12-27 02:36:06,867][105692] Updated weights for policy 0, policy_version 1538675 (0.0009) [2023-12-27 02:36:06,930][105692] Updated weights for policy 0, policy_version 1538685 (0.0010) [2023-12-27 02:36:06,991][105692] Updated weights for policy 0, policy_version 1538695 (0.0009) [2023-12-27 02:36:07,351][105620] Updated weights for policy 1, policy_version 1541677 (0.0010) [2023-12-27 02:36:07,413][105620] Updated weights for policy 1, policy_version 1541687 (0.0008) [2023-12-27 02:36:07,463][105620] Updated weights for policy 1, policy_version 1541697 (0.0008) [2023-12-27 02:36:07,741][105692] Updated weights for policy 0, policy_version 1538705 (0.0006) [2023-12-27 02:36:07,792][105692] Updated weights for policy 0, policy_version 1538715 (0.0008) [2023-12-27 02:36:07,849][105692] Updated weights for policy 0, policy_version 1538725 (0.0010) [2023-12-27 02:36:08,233][105620] Updated weights for policy 1, policy_version 1541707 (0.0008) [2023-12-27 02:36:08,286][105620] Updated weights for policy 1, policy_version 1541717 (0.0010) [2023-12-27 02:36:08,341][105620] Updated weights for policy 1, policy_version 1541727 (0.0008) [2023-12-27 02:36:08,520][105692] Updated weights for policy 0, policy_version 1538735 (0.0008) [2023-12-27 02:36:08,567][105692] Updated weights for policy 0, policy_version 1538745 (0.0009) [2023-12-27 02:36:08,618][105692] Updated weights for policy 0, policy_version 1538755 (0.0009) [2023-12-27 02:36:09,068][105620] Updated weights for policy 1, policy_version 1541737 (0.0007) [2023-12-27 02:36:09,116][105620] Updated weights for policy 1, policy_version 1541747 (0.0008) [2023-12-27 02:36:09,171][105620] Updated weights for policy 1, policy_version 1541757 (0.0006) [2023-12-27 02:36:09,226][105620] Updated weights for policy 1, policy_version 1541767 (0.0006) [2023-12-27 02:36:09,478][105692] Updated weights for policy 0, policy_version 1538765 (0.0008) [2023-12-27 02:36:09,543][105692] Updated weights for policy 0, policy_version 1538775 (0.0007) [2023-12-27 02:36:09,616][105692] Updated weights for policy 0, policy_version 1538785 (0.0006) [2023-12-27 02:36:10,012][105620] Updated weights for policy 1, policy_version 1541777 (0.0009) [2023-12-27 02:36:10,080][105620] Updated weights for policy 1, policy_version 1541787 (0.0009) [2023-12-27 02:36:10,130][105620] Updated weights for policy 1, policy_version 1541797 (0.0009) [2023-12-27 02:36:10,217][105692] Updated weights for policy 0, policy_version 1538795 (0.0006) [2023-12-27 02:36:10,272][105692] Updated weights for policy 0, policy_version 1538805 (0.0005) [2023-12-27 02:36:10,336][105692] Updated weights for policy 0, policy_version 1538815 (0.0008) [2023-12-27 02:36:10,944][105620] Updated weights for policy 1, policy_version 1541807 (0.0010) [2023-12-27 02:36:11,000][105620] Updated weights for policy 1, policy_version 1541817 (0.0010) [2023-12-27 02:36:11,054][105692] Updated weights for policy 0, policy_version 1538825 (0.0010) [2023-12-27 02:36:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 788750336. Throughput: 0: 9861.5, 1: 9845.3. Samples: 788766348. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:36:11,062][104569] Avg episode reward: [(0, '8989.047'), (1, '8901.181')] [2023-12-27 02:36:11,068][105620] Updated weights for policy 1, policy_version 1541827 (0.0012) [2023-12-27 02:36:11,116][105692] Updated weights for policy 0, policy_version 1538835 (0.0008) [2023-12-27 02:36:11,181][105692] Updated weights for policy 0, policy_version 1538845 (0.0011) [2023-12-27 02:36:11,239][105692] Updated weights for policy 0, policy_version 1538855 (0.0011) [2023-12-27 02:36:11,963][105620] Updated weights for policy 1, policy_version 1541837 (0.0010) [2023-12-27 02:36:12,022][105692] Updated weights for policy 0, policy_version 1538865 (0.0008) [2023-12-27 02:36:12,026][105620] Updated weights for policy 1, policy_version 1541847 (0.0010) [2023-12-27 02:36:12,086][105620] Updated weights for policy 1, policy_version 1541857 (0.0011) [2023-12-27 02:36:12,088][105692] Updated weights for policy 0, policy_version 1538875 (0.0007) [2023-12-27 02:36:12,153][105692] Updated weights for policy 0, policy_version 1538885 (0.0006) [2023-12-27 02:36:12,812][105620] Updated weights for policy 1, policy_version 1541867 (0.0011) [2023-12-27 02:36:12,854][105692] Updated weights for policy 0, policy_version 1538895 (0.0006) [2023-12-27 02:36:12,874][105620] Updated weights for policy 1, policy_version 1541877 (0.0011) [2023-12-27 02:36:12,909][105692] Updated weights for policy 0, policy_version 1538905 (0.0009) [2023-12-27 02:36:12,930][105620] Updated weights for policy 1, policy_version 1541887 (0.0010) [2023-12-27 02:36:12,965][105692] Updated weights for policy 0, policy_version 1538915 (0.0010) [2023-12-27 02:36:13,543][105620] Updated weights for policy 1, policy_version 1541897 (0.0010) [2023-12-27 02:36:13,591][105620] Updated weights for policy 1, policy_version 1541907 (0.0005) [2023-12-27 02:36:13,646][105620] Updated weights for policy 1, policy_version 1541917 (0.0005) [2023-12-27 02:36:13,690][105692] Updated weights for policy 0, policy_version 1538925 (0.0008) [2023-12-27 02:36:13,700][105620] Updated weights for policy 1, policy_version 1541927 (0.0007) [2023-12-27 02:36:13,746][105692] Updated weights for policy 0, policy_version 1538935 (0.0005) [2023-12-27 02:36:13,792][105692] Updated weights for policy 0, policy_version 1538945 (0.0005) [2023-12-27 02:36:14,325][105692] Updated weights for policy 0, policy_version 1538955 (0.0005) [2023-12-27 02:36:14,365][105620] Updated weights for policy 1, policy_version 1541937 (0.0006) [2023-12-27 02:36:14,381][105692] Updated weights for policy 0, policy_version 1538965 (0.0006) [2023-12-27 02:36:14,422][105620] Updated weights for policy 1, policy_version 1541947 (0.0005) [2023-12-27 02:36:14,444][105692] Updated weights for policy 0, policy_version 1538975 (0.0008) [2023-12-27 02:36:14,470][105620] Updated weights for policy 1, policy_version 1541957 (0.0007) [2023-12-27 02:36:15,090][105620] Updated weights for policy 1, policy_version 1541967 (0.0006) [2023-12-27 02:36:15,151][105620] Updated weights for policy 1, policy_version 1541977 (0.0006) [2023-12-27 02:36:15,153][105692] Updated weights for policy 0, policy_version 1538985 (0.0010) [2023-12-27 02:36:15,218][105620] Updated weights for policy 1, policy_version 1541987 (0.0006) [2023-12-27 02:36:15,220][105692] Updated weights for policy 0, policy_version 1538995 (0.0011) [2023-12-27 02:36:15,283][105692] Updated weights for policy 0, policy_version 1539005 (0.0011) [2023-12-27 02:36:15,346][105692] Updated weights for policy 0, policy_version 1539015 (0.0011) [2023-12-27 02:36:15,802][105620] Updated weights for policy 1, policy_version 1541997 (0.0007) [2023-12-27 02:36:15,865][105620] Updated weights for policy 1, policy_version 1542007 (0.0006) [2023-12-27 02:36:15,921][105620] Updated weights for policy 1, policy_version 1542017 (0.0007) [2023-12-27 02:36:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 788856832. Throughput: 0: 9797.8, 1: 9797.6. Samples: 788823180. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:36:16,062][104569] Avg episode reward: [(0, '8811.486'), (1, '8993.201')] [2023-12-27 02:36:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001542024_394813440.pth... [2023-12-27 02:36:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001540840_394510336.pth [2023-12-27 02:36:16,088][105692] Updated weights for policy 0, policy_version 1539025 (0.0010) [2023-12-27 02:36:16,142][105692] Updated weights for policy 0, policy_version 1539035 (0.0010) [2023-12-27 02:36:16,197][105692] Updated weights for policy 0, policy_version 1539045 (0.0010) [2023-12-27 02:36:16,210][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001539048_394051584.pth... [2023-12-27 02:36:16,214][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001537928_393764864.pth [2023-12-27 02:36:16,653][105620] Updated weights for policy 1, policy_version 1542027 (0.0009) [2023-12-27 02:36:16,709][105620] Updated weights for policy 1, policy_version 1542037 (0.0011) [2023-12-27 02:36:16,769][105620] Updated weights for policy 1, policy_version 1542047 (0.0011) [2023-12-27 02:36:16,912][105692] Updated weights for policy 0, policy_version 1539055 (0.0007) [2023-12-27 02:36:16,967][105692] Updated weights for policy 0, policy_version 1539065 (0.0005) [2023-12-27 02:36:17,016][105692] Updated weights for policy 0, policy_version 1539075 (0.0008) [2023-12-27 02:36:17,483][105620] Updated weights for policy 1, policy_version 1542057 (0.0010) [2023-12-27 02:36:17,541][105620] Updated weights for policy 1, policy_version 1542067 (0.0005) [2023-12-27 02:36:17,597][105620] Updated weights for policy 1, policy_version 1542077 (0.0005) [2023-12-27 02:36:17,648][105620] Updated weights for policy 1, policy_version 1542087 (0.0005) [2023-12-27 02:36:17,741][105692] Updated weights for policy 0, policy_version 1539085 (0.0011) [2023-12-27 02:36:17,802][105692] Updated weights for policy 0, policy_version 1539095 (0.0010) [2023-12-27 02:36:17,847][105692] Updated weights for policy 0, policy_version 1539105 (0.0010) [2023-12-27 02:36:18,211][105620] Updated weights for policy 1, policy_version 1542097 (0.0008) [2023-12-27 02:36:18,284][105620] Updated weights for policy 1, policy_version 1542107 (0.0007) [2023-12-27 02:36:18,355][105620] Updated weights for policy 1, policy_version 1542117 (0.0007) [2023-12-27 02:36:18,481][105692] Updated weights for policy 0, policy_version 1539115 (0.0010) [2023-12-27 02:36:18,549][105692] Updated weights for policy 0, policy_version 1539125 (0.0010) [2023-12-27 02:36:18,613][105692] Updated weights for policy 0, policy_version 1539135 (0.0008) [2023-12-27 02:36:18,987][105620] Updated weights for policy 1, policy_version 1542127 (0.0010) [2023-12-27 02:36:19,045][105620] Updated weights for policy 1, policy_version 1542137 (0.0010) [2023-12-27 02:36:19,110][105620] Updated weights for policy 1, policy_version 1542147 (0.0010) [2023-12-27 02:36:19,300][105692] Updated weights for policy 0, policy_version 1539145 (0.0005) [2023-12-27 02:36:19,345][105692] Updated weights for policy 0, policy_version 1539155 (0.0008) [2023-12-27 02:36:19,408][105692] Updated weights for policy 0, policy_version 1539165 (0.0008) [2023-12-27 02:36:19,467][105692] Updated weights for policy 0, policy_version 1539175 (0.0009) [2023-12-27 02:36:19,882][105620] Updated weights for policy 1, policy_version 1542157 (0.0010) [2023-12-27 02:36:19,945][105620] Updated weights for policy 1, policy_version 1542167 (0.0009) [2023-12-27 02:36:20,003][105620] Updated weights for policy 1, policy_version 1542177 (0.0009) [2023-12-27 02:36:20,313][105692] Updated weights for policy 0, policy_version 1539185 (0.0008) [2023-12-27 02:36:20,373][105692] Updated weights for policy 0, policy_version 1539195 (0.0008) [2023-12-27 02:36:20,432][105692] Updated weights for policy 0, policy_version 1539205 (0.0009) [2023-12-27 02:36:20,799][105620] Updated weights for policy 1, policy_version 1542187 (0.0009) [2023-12-27 02:36:20,860][105620] Updated weights for policy 1, policy_version 1542197 (0.0008) [2023-12-27 02:36:20,917][105620] Updated weights for policy 1, policy_version 1542207 (0.0008) [2023-12-27 02:36:21,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 788955136. Throughput: 0: 9713.6, 1: 9838.8. Samples: 788945780. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:36:21,063][104569] Avg episode reward: [(0, '8184.274'), (1, '8994.980')] [2023-12-27 02:36:21,188][105692] Updated weights for policy 0, policy_version 1539215 (0.0010) [2023-12-27 02:36:21,245][105692] Updated weights for policy 0, policy_version 1539225 (0.0011) [2023-12-27 02:36:21,302][105692] Updated weights for policy 0, policy_version 1539235 (0.0011) [2023-12-27 02:36:21,696][105620] Updated weights for policy 1, policy_version 1542217 (0.0005) [2023-12-27 02:36:21,770][105620] Updated weights for policy 1, policy_version 1542227 (0.0009) [2023-12-27 02:36:21,840][105620] Updated weights for policy 1, policy_version 1542237 (0.0010) [2023-12-27 02:36:21,904][105620] Updated weights for policy 1, policy_version 1542247 (0.0009) [2023-12-27 02:36:21,962][105692] Updated weights for policy 0, policy_version 1539245 (0.0010) [2023-12-27 02:36:22,025][105692] Updated weights for policy 0, policy_version 1539255 (0.0009) [2023-12-27 02:36:22,087][105692] Updated weights for policy 0, policy_version 1539265 (0.0009) [2023-12-27 02:36:22,655][105620] Updated weights for policy 1, policy_version 1542257 (0.0010) [2023-12-27 02:36:22,710][105620] Updated weights for policy 1, policy_version 1542267 (0.0009) [2023-12-27 02:36:22,766][105620] Updated weights for policy 1, policy_version 1542277 (0.0009) [2023-12-27 02:36:22,841][105692] Updated weights for policy 0, policy_version 1539275 (0.0009) [2023-12-27 02:36:22,903][105692] Updated weights for policy 0, policy_version 1539285 (0.0009) [2023-12-27 02:36:22,968][105692] Updated weights for policy 0, policy_version 1539295 (0.0009) [2023-12-27 02:36:23,557][105620] Updated weights for policy 1, policy_version 1542287 (0.0009) [2023-12-27 02:36:23,615][105620] Updated weights for policy 1, policy_version 1542297 (0.0009) [2023-12-27 02:36:23,658][105692] Updated weights for policy 0, policy_version 1539305 (0.0009) [2023-12-27 02:36:23,677][105620] Updated weights for policy 1, policy_version 1542307 (0.0008) [2023-12-27 02:36:23,704][105692] Updated weights for policy 0, policy_version 1539315 (0.0005) [2023-12-27 02:36:23,746][105692] Updated weights for policy 0, policy_version 1539325 (0.0006) [2023-12-27 02:36:23,793][105692] Updated weights for policy 0, policy_version 1539335 (0.0009) [2023-12-27 02:36:24,440][105692] Updated weights for policy 0, policy_version 1539345 (0.0005) [2023-12-27 02:36:24,493][105692] Updated weights for policy 0, policy_version 1539355 (0.0005) [2023-12-27 02:36:24,534][105620] Updated weights for policy 1, policy_version 1542317 (0.0008) [2023-12-27 02:36:24,545][105692] Updated weights for policy 0, policy_version 1539365 (0.0006) [2023-12-27 02:36:24,590][105620] Updated weights for policy 1, policy_version 1542327 (0.0007) [2023-12-27 02:36:24,641][105620] Updated weights for policy 1, policy_version 1542337 (0.0009) [2023-12-27 02:36:25,245][105692] Updated weights for policy 0, policy_version 1539375 (0.0009) [2023-12-27 02:36:25,297][105692] Updated weights for policy 0, policy_version 1539385 (0.0009) [2023-12-27 02:36:25,343][105692] Updated weights for policy 0, policy_version 1539395 (0.0008) [2023-12-27 02:36:25,402][105620] Updated weights for policy 1, policy_version 1542347 (0.0009) [2023-12-27 02:36:25,450][105620] Updated weights for policy 1, policy_version 1542357 (0.0009) [2023-12-27 02:36:25,505][105620] Updated weights for policy 1, policy_version 1542367 (0.0009) [2023-12-27 02:36:25,982][105692] Updated weights for policy 0, policy_version 1539405 (0.0010) [2023-12-27 02:36:26,034][105692] Updated weights for policy 0, policy_version 1539415 (0.0010) [2023-12-27 02:36:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 789045248. Throughput: 0: 9655.0, 1: 9752.5. Samples: 789057704. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:36:26,062][104569] Avg episode reward: [(0, '8455.481'), (1, '8815.091')] [2023-12-27 02:36:26,086][105692] Updated weights for policy 0, policy_version 1539425 (0.0011) [2023-12-27 02:36:26,199][105620] Updated weights for policy 1, policy_version 1542377 (0.0009) [2023-12-27 02:36:26,250][105620] Updated weights for policy 1, policy_version 1542387 (0.0005) [2023-12-27 02:36:26,305][105620] Updated weights for policy 1, policy_version 1542397 (0.0005) [2023-12-27 02:36:26,358][105620] Updated weights for policy 1, policy_version 1542407 (0.0005) [2023-12-27 02:36:26,762][105692] Updated weights for policy 0, policy_version 1539435 (0.0010) [2023-12-27 02:36:26,821][105692] Updated weights for policy 0, policy_version 1539445 (0.0010) [2023-12-27 02:36:26,875][105692] Updated weights for policy 0, policy_version 1539455 (0.0010) [2023-12-27 02:36:26,955][105620] Updated weights for policy 1, policy_version 1542417 (0.0005) [2023-12-27 02:36:27,006][105620] Updated weights for policy 1, policy_version 1542427 (0.0006) [2023-12-27 02:36:27,058][105620] Updated weights for policy 1, policy_version 1542437 (0.0005) [2023-12-27 02:36:27,626][105692] Updated weights for policy 0, policy_version 1539465 (0.0007) [2023-12-27 02:36:27,676][105620] Updated weights for policy 1, policy_version 1542447 (0.0005) [2023-12-27 02:36:27,683][105692] Updated weights for policy 0, policy_version 1539475 (0.0005) [2023-12-27 02:36:27,744][105692] Updated weights for policy 0, policy_version 1539485 (0.0005) [2023-12-27 02:36:27,746][105620] Updated weights for policy 1, policy_version 1542457 (0.0005) [2023-12-27 02:36:27,798][105620] Updated weights for policy 1, policy_version 1542467 (0.0008) [2023-12-27 02:36:27,806][105692] Updated weights for policy 0, policy_version 1539495 (0.0007) [2023-12-27 02:36:28,366][105620] Updated weights for policy 1, policy_version 1542477 (0.0009) [2023-12-27 02:36:28,428][105620] Updated weights for policy 1, policy_version 1542487 (0.0009) [2023-12-27 02:36:28,487][105620] Updated weights for policy 1, policy_version 1542497 (0.0010) [2023-12-27 02:36:28,527][105692] Updated weights for policy 0, policy_version 1539505 (0.0006) [2023-12-27 02:36:28,586][105692] Updated weights for policy 0, policy_version 1539515 (0.0010) [2023-12-27 02:36:28,640][105692] Updated weights for policy 0, policy_version 1539525 (0.0010) [2023-12-27 02:36:29,105][105620] Updated weights for policy 1, policy_version 1542507 (0.0010) [2023-12-27 02:36:29,160][105620] Updated weights for policy 1, policy_version 1542517 (0.0009) [2023-12-27 02:36:29,228][105620] Updated weights for policy 1, policy_version 1542527 (0.0006) [2023-12-27 02:36:29,347][105692] Updated weights for policy 0, policy_version 1539535 (0.0011) [2023-12-27 02:36:29,410][105692] Updated weights for policy 0, policy_version 1539545 (0.0011) [2023-12-27 02:36:29,482][105692] Updated weights for policy 0, policy_version 1539555 (0.0011) [2023-12-27 02:36:29,825][105620] Updated weights for policy 1, policy_version 1542537 (0.0006) [2023-12-27 02:36:29,888][105620] Updated weights for policy 1, policy_version 1542547 (0.0009) [2023-12-27 02:36:29,950][105620] Updated weights for policy 1, policy_version 1542557 (0.0009) [2023-12-27 02:36:30,009][105620] Updated weights for policy 1, policy_version 1542567 (0.0008) [2023-12-27 02:36:30,198][105692] Updated weights for policy 0, policy_version 1539565 (0.0011) [2023-12-27 02:36:30,253][105692] Updated weights for policy 0, policy_version 1539575 (0.0010) [2023-12-27 02:36:30,315][105692] Updated weights for policy 0, policy_version 1539585 (0.0010) [2023-12-27 02:36:30,683][105620] Updated weights for policy 1, policy_version 1542577 (0.0006) [2023-12-27 02:36:30,735][105620] Updated weights for policy 1, policy_version 1542587 (0.0008) [2023-12-27 02:36:30,793][105620] Updated weights for policy 1, policy_version 1542597 (0.0010) [2023-12-27 02:36:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 789151744. Throughput: 0: 9663.6, 1: 9932.2. Samples: 789122752. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:36:31,063][104569] Avg episode reward: [(0, '8452.622'), (1, '8810.385')] [2023-12-27 02:36:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001542600_394960896.pth... [2023-12-27 02:36:31,070][105692] Updated weights for policy 0, policy_version 1539595 (0.0011) [2023-12-27 02:36:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001541448_394665984.pth [2023-12-27 02:36:31,132][105692] Updated weights for policy 0, policy_version 1539605 (0.0011) [2023-12-27 02:36:31,195][105692] Updated weights for policy 0, policy_version 1539615 (0.0011) [2023-12-27 02:36:31,242][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001539624_394199040.pth... [2023-12-27 02:36:31,246][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001538440_393895936.pth [2023-12-27 02:36:31,506][105620] Updated weights for policy 1, policy_version 1542607 (0.0010) [2023-12-27 02:36:31,567][105620] Updated weights for policy 1, policy_version 1542617 (0.0010) [2023-12-27 02:36:31,623][105620] Updated weights for policy 1, policy_version 1542627 (0.0009) [2023-12-27 02:36:31,971][105692] Updated weights for policy 0, policy_version 1539625 (0.0010) [2023-12-27 02:36:32,033][105692] Updated weights for policy 0, policy_version 1539635 (0.0009) [2023-12-27 02:36:32,095][105692] Updated weights for policy 0, policy_version 1539645 (0.0009) [2023-12-27 02:36:32,157][105692] Updated weights for policy 0, policy_version 1539655 (0.0009) [2023-12-27 02:36:32,345][105620] Updated weights for policy 1, policy_version 1542637 (0.0008) [2023-12-27 02:36:32,401][105620] Updated weights for policy 1, policy_version 1542647 (0.0010) [2023-12-27 02:36:32,448][105620] Updated weights for policy 1, policy_version 1542657 (0.0008) [2023-12-27 02:36:32,959][105692] Updated weights for policy 0, policy_version 1539665 (0.0011) [2023-12-27 02:36:33,024][105692] Updated weights for policy 0, policy_version 1539675 (0.0011) [2023-12-27 02:36:33,083][105692] Updated weights for policy 0, policy_version 1539685 (0.0011) [2023-12-27 02:36:33,095][105620] Updated weights for policy 1, policy_version 1542667 (0.0007) [2023-12-27 02:36:33,160][105620] Updated weights for policy 1, policy_version 1542677 (0.0007) [2023-12-27 02:36:33,204][105620] Updated weights for policy 1, policy_version 1542687 (0.0008) [2023-12-27 02:36:33,738][105692] Updated weights for policy 0, policy_version 1539695 (0.0007) [2023-12-27 02:36:33,798][105692] Updated weights for policy 0, policy_version 1539705 (0.0006) [2023-12-27 02:36:33,841][105692] Updated weights for policy 0, policy_version 1539715 (0.0005) [2023-12-27 02:36:33,982][105620] Updated weights for policy 1, policy_version 1542697 (0.0009) [2023-12-27 02:36:34,033][105620] Updated weights for policy 1, policy_version 1542707 (0.0007) [2023-12-27 02:36:34,080][105620] Updated weights for policy 1, policy_version 1542717 (0.0007) [2023-12-27 02:36:34,129][105620] Updated weights for policy 1, policy_version 1542727 (0.0008) [2023-12-27 02:36:34,501][105692] Updated weights for policy 0, policy_version 1539725 (0.0008) [2023-12-27 02:36:34,557][105692] Updated weights for policy 0, policy_version 1539735 (0.0010) [2023-12-27 02:36:34,626][105692] Updated weights for policy 0, policy_version 1539745 (0.0011) [2023-12-27 02:36:34,901][105620] Updated weights for policy 1, policy_version 1542737 (0.0006) [2023-12-27 02:36:34,946][105620] Updated weights for policy 1, policy_version 1542747 (0.0005) [2023-12-27 02:36:34,992][105620] Updated weights for policy 1, policy_version 1542757 (0.0005) [2023-12-27 02:36:35,375][105692] Updated weights for policy 0, policy_version 1539755 (0.0011) [2023-12-27 02:36:35,436][105692] Updated weights for policy 0, policy_version 1539765 (0.0010) [2023-12-27 02:36:35,490][105692] Updated weights for policy 0, policy_version 1539775 (0.0010) [2023-12-27 02:36:35,678][105620] Updated weights for policy 1, policy_version 1542767 (0.0008) [2023-12-27 02:36:35,728][105620] Updated weights for policy 1, policy_version 1542777 (0.0007) [2023-12-27 02:36:35,772][105620] Updated weights for policy 1, policy_version 1542787 (0.0008) [2023-12-27 02:36:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 789250048. Throughput: 0: 9656.3, 1: 9933.7. Samples: 789239388. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:36:36,062][104569] Avg episode reward: [(0, '8363.164'), (1, '8536.492')] [2023-12-27 02:36:36,234][105692] Updated weights for policy 0, policy_version 1539785 (0.0011) [2023-12-27 02:36:36,293][105692] Updated weights for policy 0, policy_version 1539795 (0.0011) [2023-12-27 02:36:36,352][105692] Updated weights for policy 0, policy_version 1539805 (0.0010) [2023-12-27 02:36:36,406][105692] Updated weights for policy 0, policy_version 1539815 (0.0011) [2023-12-27 02:36:36,553][105620] Updated weights for policy 1, policy_version 1542797 (0.0008) [2023-12-27 02:36:36,615][105620] Updated weights for policy 1, policy_version 1542807 (0.0008) [2023-12-27 02:36:36,682][105620] Updated weights for policy 1, policy_version 1542817 (0.0008) [2023-12-27 02:36:37,179][105692] Updated weights for policy 0, policy_version 1539825 (0.0011) [2023-12-27 02:36:37,238][105692] Updated weights for policy 0, policy_version 1539835 (0.0011) [2023-12-27 02:36:37,303][105692] Updated weights for policy 0, policy_version 1539845 (0.0011) [2023-12-27 02:36:37,427][105620] Updated weights for policy 1, policy_version 1542827 (0.0008) [2023-12-27 02:36:37,479][105620] Updated weights for policy 1, policy_version 1542837 (0.0008) [2023-12-27 02:36:37,527][105620] Updated weights for policy 1, policy_version 1542847 (0.0008) [2023-12-27 02:36:38,050][105692] Updated weights for policy 0, policy_version 1539855 (0.0010) [2023-12-27 02:36:38,108][105692] Updated weights for policy 0, policy_version 1539865 (0.0011) [2023-12-27 02:36:38,164][105692] Updated weights for policy 0, policy_version 1539875 (0.0009) [2023-12-27 02:36:38,267][105620] Updated weights for policy 1, policy_version 1542857 (0.0008) [2023-12-27 02:36:38,341][105620] Updated weights for policy 1, policy_version 1542867 (0.0006) [2023-12-27 02:36:38,401][105620] Updated weights for policy 1, policy_version 1542877 (0.0009) [2023-12-27 02:36:38,467][105620] Updated weights for policy 1, policy_version 1542887 (0.0009) [2023-12-27 02:36:38,919][105692] Updated weights for policy 0, policy_version 1539885 (0.0009) [2023-12-27 02:36:38,982][105692] Updated weights for policy 0, policy_version 1539895 (0.0011) [2023-12-27 02:36:39,038][105692] Updated weights for policy 0, policy_version 1539905 (0.0010) [2023-12-27 02:36:39,164][105620] Updated weights for policy 1, policy_version 1542897 (0.0008) [2023-12-27 02:36:39,217][105620] Updated weights for policy 1, policy_version 1542907 (0.0008) [2023-12-27 02:36:39,277][105620] Updated weights for policy 1, policy_version 1542917 (0.0008) [2023-12-27 02:36:39,772][105692] Updated weights for policy 0, policy_version 1539915 (0.0011) [2023-12-27 02:36:39,840][105692] Updated weights for policy 0, policy_version 1539925 (0.0009) [2023-12-27 02:36:39,908][105692] Updated weights for policy 0, policy_version 1539935 (0.0006) [2023-12-27 02:36:40,088][105620] Updated weights for policy 1, policy_version 1542927 (0.0009) [2023-12-27 02:36:40,157][105620] Updated weights for policy 1, policy_version 1542937 (0.0009) [2023-12-27 02:36:40,214][105620] Updated weights for policy 1, policy_version 1542947 (0.0009) [2023-12-27 02:36:40,606][105692] Updated weights for policy 0, policy_version 1539945 (0.0009) [2023-12-27 02:36:40,658][105692] Updated weights for policy 0, policy_version 1539955 (0.0009) [2023-12-27 02:36:40,706][105692] Updated weights for policy 0, policy_version 1539965 (0.0005) [2023-12-27 02:36:40,763][105692] Updated weights for policy 0, policy_version 1539975 (0.0007) [2023-12-27 02:36:41,024][105620] Updated weights for policy 1, policy_version 1542957 (0.0007) [2023-12-27 02:36:41,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 789340160. Throughput: 0: 9740.2, 1: 9796.2. Samples: 789351404. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:36:41,062][104569] Avg episode reward: [(0, '8999.293'), (1, '8629.864')] [2023-12-27 02:36:41,087][105620] Updated weights for policy 1, policy_version 1542967 (0.0008) [2023-12-27 02:36:41,145][105620] Updated weights for policy 1, policy_version 1542977 (0.0009) [2023-12-27 02:36:41,559][105692] Updated weights for policy 0, policy_version 1539985 (0.0006) [2023-12-27 02:36:41,627][105692] Updated weights for policy 0, policy_version 1539995 (0.0007) [2023-12-27 02:36:41,690][105692] Updated weights for policy 0, policy_version 1540005 (0.0010) [2023-12-27 02:36:41,966][105620] Updated weights for policy 1, policy_version 1542987 (0.0010) [2023-12-27 02:36:42,028][105620] Updated weights for policy 1, policy_version 1542997 (0.0009) [2023-12-27 02:36:42,090][105620] Updated weights for policy 1, policy_version 1543007 (0.0009) [2023-12-27 02:36:42,452][105692] Updated weights for policy 0, policy_version 1540015 (0.0008) [2023-12-27 02:36:42,516][105692] Updated weights for policy 0, policy_version 1540025 (0.0009) [2023-12-27 02:36:42,577][105692] Updated weights for policy 0, policy_version 1540035 (0.0011) [2023-12-27 02:36:42,853][105620] Updated weights for policy 1, policy_version 1543017 (0.0009) [2023-12-27 02:36:42,919][105620] Updated weights for policy 1, policy_version 1543027 (0.0009) [2023-12-27 02:36:42,982][105620] Updated weights for policy 1, policy_version 1543037 (0.0008) [2023-12-27 02:36:43,043][105620] Updated weights for policy 1, policy_version 1543047 (0.0008) [2023-12-27 02:36:43,311][105692] Updated weights for policy 0, policy_version 1540045 (0.0011) [2023-12-27 02:36:43,366][105692] Updated weights for policy 0, policy_version 1540055 (0.0010) [2023-12-27 02:36:43,416][105692] Updated weights for policy 0, policy_version 1540065 (0.0006) [2023-12-27 02:36:43,730][105620] Updated weights for policy 1, policy_version 1543057 (0.0006) [2023-12-27 02:36:43,778][105620] Updated weights for policy 1, policy_version 1543067 (0.0005) [2023-12-27 02:36:43,835][105620] Updated weights for policy 1, policy_version 1543077 (0.0005) [2023-12-27 02:36:43,981][105692] Updated weights for policy 0, policy_version 1540075 (0.0005) [2023-12-27 02:36:44,037][105692] Updated weights for policy 0, policy_version 1540085 (0.0005) [2023-12-27 02:36:44,094][105692] Updated weights for policy 0, policy_version 1540095 (0.0008) [2023-12-27 02:36:44,398][105620] Updated weights for policy 1, policy_version 1543087 (0.0007) [2023-12-27 02:36:44,450][105620] Updated weights for policy 1, policy_version 1543097 (0.0010) [2023-12-27 02:36:44,507][105620] Updated weights for policy 1, policy_version 1543107 (0.0009) [2023-12-27 02:36:44,808][105692] Updated weights for policy 0, policy_version 1540105 (0.0009) [2023-12-27 02:36:44,878][105692] Updated weights for policy 0, policy_version 1540115 (0.0006) [2023-12-27 02:36:44,944][105692] Updated weights for policy 0, policy_version 1540125 (0.0007) [2023-12-27 02:36:44,999][105692] Updated weights for policy 0, policy_version 1540135 (0.0009) [2023-12-27 02:36:45,300][105620] Updated weights for policy 1, policy_version 1543117 (0.0007) [2023-12-27 02:36:45,372][105620] Updated weights for policy 1, policy_version 1543127 (0.0005) [2023-12-27 02:36:45,443][105620] Updated weights for policy 1, policy_version 1543137 (0.0008) [2023-12-27 02:36:45,646][105692] Updated weights for policy 0, policy_version 1540145 (0.0009) [2023-12-27 02:36:45,707][105692] Updated weights for policy 0, policy_version 1540155 (0.0010) [2023-12-27 02:36:45,766][105692] Updated weights for policy 0, policy_version 1540165 (0.0010) [2023-12-27 02:36:45,964][105620] Updated weights for policy 1, policy_version 1543147 (0.0008) [2023-12-27 02:36:46,027][105620] Updated weights for policy 1, policy_version 1543157 (0.0006) [2023-12-27 02:36:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 789438464. Throughput: 0: 9759.1, 1: 9744.9. Samples: 789407764. Policy #0 lag: (min: 6.0, avg: 13.8, max: 38.0) [2023-12-27 02:36:46,062][104569] Avg episode reward: [(0, '8633.575'), (1, '8992.459')] [2023-12-27 02:36:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001540168_394338304.pth... [2023-12-27 02:36:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001539048_394051584.pth [2023-12-27 02:36:46,079][105620] Updated weights for policy 1, policy_version 1543167 (0.0010) [2023-12-27 02:36:46,128][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001543176_395108352.pth... [2023-12-27 02:36:46,131][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001542024_394813440.pth [2023-12-27 02:36:46,615][105620] Updated weights for policy 1, policy_version 1543177 (0.0010) [2023-12-27 02:36:46,635][105692] Updated weights for policy 0, policy_version 1540175 (0.0009) [2023-12-27 02:36:46,660][105620] Updated weights for policy 1, policy_version 1543187 (0.0005) [2023-12-27 02:36:46,691][105692] Updated weights for policy 0, policy_version 1540185 (0.0009) [2023-12-27 02:36:46,705][105620] Updated weights for policy 1, policy_version 1543197 (0.0005) [2023-12-27 02:36:46,750][105692] Updated weights for policy 0, policy_version 1540195 (0.0009) [2023-12-27 02:36:46,758][105620] Updated weights for policy 1, policy_version 1543207 (0.0005) [2023-12-27 02:36:47,307][105620] Updated weights for policy 1, policy_version 1543217 (0.0010) [2023-12-27 02:36:47,378][105620] Updated weights for policy 1, policy_version 1543227 (0.0006) [2023-12-27 02:36:47,432][105620] Updated weights for policy 1, policy_version 1543237 (0.0005) [2023-12-27 02:36:47,524][105692] Updated weights for policy 0, policy_version 1540205 (0.0010) [2023-12-27 02:36:47,576][105692] Updated weights for policy 0, policy_version 1540215 (0.0010) [2023-12-27 02:36:47,634][105692] Updated weights for policy 0, policy_version 1540225 (0.0005) [2023-12-27 02:36:48,108][105620] Updated weights for policy 1, policy_version 1543247 (0.0009) [2023-12-27 02:36:48,166][105620] Updated weights for policy 1, policy_version 1543257 (0.0010) [2023-12-27 02:36:48,221][105620] Updated weights for policy 1, policy_version 1543268 (0.0007) [2023-12-27 02:36:48,249][105692] Updated weights for policy 0, policy_version 1540235 (0.0006) [2023-12-27 02:36:48,305][105692] Updated weights for policy 0, policy_version 1540245 (0.0005) [2023-12-27 02:36:48,371][105692] Updated weights for policy 0, policy_version 1540255 (0.0008) [2023-12-27 02:36:48,991][105620] Updated weights for policy 1, policy_version 1543278 (0.0007) [2023-12-27 02:36:49,054][105620] Updated weights for policy 1, policy_version 1543288 (0.0009) [2023-12-27 02:36:49,065][105692] Updated weights for policy 0, policy_version 1540265 (0.0009) [2023-12-27 02:36:49,106][105620] Updated weights for policy 1, policy_version 1543298 (0.0008) [2023-12-27 02:36:49,122][105692] Updated weights for policy 0, policy_version 1540275 (0.0007) [2023-12-27 02:36:49,192][105692] Updated weights for policy 0, policy_version 1540285 (0.0006) [2023-12-27 02:36:49,251][105692] Updated weights for policy 0, policy_version 1540295 (0.0009) [2023-12-27 02:36:49,856][105620] Updated weights for policy 1, policy_version 1543308 (0.0007) [2023-12-27 02:36:49,919][105620] Updated weights for policy 1, policy_version 1543318 (0.0008) [2023-12-27 02:36:49,985][105620] Updated weights for policy 1, policy_version 1543328 (0.0008) [2023-12-27 02:36:49,992][105692] Updated weights for policy 0, policy_version 1540305 (0.0006) [2023-12-27 02:36:50,053][105692] Updated weights for policy 0, policy_version 1540315 (0.0006) [2023-12-27 02:36:50,117][105692] Updated weights for policy 0, policy_version 1540325 (0.0007) [2023-12-27 02:36:50,654][105620] Updated weights for policy 1, policy_version 1543338 (0.0008) [2023-12-27 02:36:50,706][105620] Updated weights for policy 1, policy_version 1543348 (0.0008) [2023-12-27 02:36:50,754][105620] Updated weights for policy 1, policy_version 1543358 (0.0009) [2023-12-27 02:36:50,814][105620] Updated weights for policy 1, policy_version 1543368 (0.0009) [2023-12-27 02:36:50,889][105692] Updated weights for policy 0, policy_version 1540335 (0.0008) [2023-12-27 02:36:50,947][105692] Updated weights for policy 0, policy_version 1540345 (0.0009) [2023-12-27 02:36:50,994][105692] Updated weights for policy 0, policy_version 1540355 (0.0009) [2023-12-27 02:36:51,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 789544960. Throughput: 0: 9751.9, 1: 9838.7. Samples: 789529844. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:36:51,062][104569] Avg episode reward: [(0, '8446.887'), (1, '9087.793')] [2023-12-27 02:36:51,581][105620] Updated weights for policy 1, policy_version 1543378 (0.0010) [2023-12-27 02:36:51,648][105620] Updated weights for policy 1, policy_version 1543388 (0.0009) [2023-12-27 02:36:51,712][105620] Updated weights for policy 1, policy_version 1543398 (0.0008) [2023-12-27 02:36:51,791][105692] Updated weights for policy 0, policy_version 1540365 (0.0009) [2023-12-27 02:36:51,854][105692] Updated weights for policy 0, policy_version 1540375 (0.0009) [2023-12-27 02:36:51,912][105692] Updated weights for policy 0, policy_version 1540385 (0.0009) [2023-12-27 02:36:52,457][105620] Updated weights for policy 1, policy_version 1543408 (0.0010) [2023-12-27 02:36:52,519][105620] Updated weights for policy 1, policy_version 1543418 (0.0008) [2023-12-27 02:36:52,585][105620] Updated weights for policy 1, policy_version 1543428 (0.0006) [2023-12-27 02:36:52,701][105692] Updated weights for policy 0, policy_version 1540395 (0.0009) [2023-12-27 02:36:52,757][105692] Updated weights for policy 0, policy_version 1540405 (0.0011) [2023-12-27 02:36:52,816][105692] Updated weights for policy 0, policy_version 1540415 (0.0011) [2023-12-27 02:36:53,199][105620] Updated weights for policy 1, policy_version 1543438 (0.0005) [2023-12-27 02:36:53,253][105620] Updated weights for policy 1, policy_version 1543448 (0.0005) [2023-12-27 02:36:53,310][105620] Updated weights for policy 1, policy_version 1543458 (0.0005) [2023-12-27 02:36:53,607][105692] Updated weights for policy 0, policy_version 1540425 (0.0011) [2023-12-27 02:36:53,662][105692] Updated weights for policy 0, policy_version 1540435 (0.0010) [2023-12-27 02:36:53,719][105692] Updated weights for policy 0, policy_version 1540445 (0.0005) [2023-12-27 02:36:53,772][105692] Updated weights for policy 0, policy_version 1540455 (0.0007) [2023-12-27 02:36:53,873][105620] Updated weights for policy 1, policy_version 1543468 (0.0007) [2023-12-27 02:36:53,944][105620] Updated weights for policy 1, policy_version 1543478 (0.0010) [2023-12-27 02:36:53,992][105620] Updated weights for policy 1, policy_version 1543488 (0.0010) [2023-12-27 02:36:54,367][105692] Updated weights for policy 0, policy_version 1540465 (0.0008) [2023-12-27 02:36:54,426][105692] Updated weights for policy 0, policy_version 1540475 (0.0008) [2023-12-27 02:36:54,471][105692] Updated weights for policy 0, policy_version 1540485 (0.0008) [2023-12-27 02:36:54,730][105620] Updated weights for policy 1, policy_version 1543498 (0.0010) [2023-12-27 02:36:54,781][105620] Updated weights for policy 1, policy_version 1543508 (0.0006) [2023-12-27 02:36:54,837][105620] Updated weights for policy 1, policy_version 1543518 (0.0005) [2023-12-27 02:36:54,893][105620] Updated weights for policy 1, policy_version 1543528 (0.0005) [2023-12-27 02:36:55,215][105692] Updated weights for policy 0, policy_version 1540495 (0.0006) [2023-12-27 02:36:55,261][105692] Updated weights for policy 0, policy_version 1540505 (0.0005) [2023-12-27 02:36:55,315][105692] Updated weights for policy 0, policy_version 1540515 (0.0005) [2023-12-27 02:36:55,449][105620] Updated weights for policy 1, policy_version 1543538 (0.0010) [2023-12-27 02:36:55,504][105620] Updated weights for policy 1, policy_version 1543548 (0.0007) [2023-12-27 02:36:55,550][105620] Updated weights for policy 1, policy_version 1543558 (0.0005) [2023-12-27 02:36:55,935][105692] Updated weights for policy 0, policy_version 1540525 (0.0008) [2023-12-27 02:36:55,997][105692] Updated weights for policy 0, policy_version 1540535 (0.0010) [2023-12-27 02:36:56,046][105692] Updated weights for policy 0, policy_version 1540545 (0.0009) [2023-12-27 02:36:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 789635072. Throughput: 0: 9671.0, 1: 9964.0. Samples: 789649924. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:36:56,062][104569] Avg episode reward: [(0, '8815.180'), (1, '8906.571')] [2023-12-27 02:36:56,083][105620] Updated weights for policy 1, policy_version 1543568 (0.0009) [2023-12-27 02:36:56,149][105620] Updated weights for policy 1, policy_version 1543579 (0.0011) [2023-12-27 02:36:56,201][105620] Updated weights for policy 1, policy_version 1543589 (0.0010) [2023-12-27 02:36:56,706][105692] Updated weights for policy 0, policy_version 1540555 (0.0006) [2023-12-27 02:36:56,753][105692] Updated weights for policy 0, policy_version 1540565 (0.0009) [2023-12-27 02:36:56,800][105692] Updated weights for policy 0, policy_version 1540575 (0.0008) [2023-12-27 02:36:56,919][105620] Updated weights for policy 1, policy_version 1543599 (0.0009) [2023-12-27 02:36:56,970][105620] Updated weights for policy 1, policy_version 1543609 (0.0009) [2023-12-27 02:36:57,024][105620] Updated weights for policy 1, policy_version 1543620 (0.0010) [2023-12-27 02:36:57,487][105692] Updated weights for policy 0, policy_version 1540585 (0.0008) [2023-12-27 02:36:57,545][105692] Updated weights for policy 0, policy_version 1540595 (0.0009) [2023-12-27 02:36:57,593][105692] Updated weights for policy 0, policy_version 1540605 (0.0005) [2023-12-27 02:36:57,646][105692] Updated weights for policy 0, policy_version 1540615 (0.0005) [2023-12-27 02:36:57,804][105620] Updated weights for policy 1, policy_version 1543630 (0.0010) [2023-12-27 02:36:57,860][105620] Updated weights for policy 1, policy_version 1543640 (0.0009) [2023-12-27 02:36:57,914][105620] Updated weights for policy 1, policy_version 1543650 (0.0009) [2023-12-27 02:36:58,222][105692] Updated weights for policy 0, policy_version 1540625 (0.0008) [2023-12-27 02:36:58,278][105692] Updated weights for policy 0, policy_version 1540635 (0.0009) [2023-12-27 02:36:58,337][105692] Updated weights for policy 0, policy_version 1540645 (0.0009) [2023-12-27 02:36:58,807][105620] Updated weights for policy 1, policy_version 1543660 (0.0009) [2023-12-27 02:36:58,871][105620] Updated weights for policy 1, policy_version 1543670 (0.0009) [2023-12-27 02:36:58,934][105620] Updated weights for policy 1, policy_version 1543680 (0.0009) [2023-12-27 02:36:59,199][105692] Updated weights for policy 0, policy_version 1540655 (0.0008) [2023-12-27 02:36:59,268][105692] Updated weights for policy 0, policy_version 1540665 (0.0009) [2023-12-27 02:36:59,331][105692] Updated weights for policy 0, policy_version 1540675 (0.0010) [2023-12-27 02:36:59,673][105620] Updated weights for policy 1, policy_version 1543690 (0.0007) [2023-12-27 02:36:59,730][105620] Updated weights for policy 1, policy_version 1543700 (0.0006) [2023-12-27 02:36:59,778][105620] Updated weights for policy 1, policy_version 1543710 (0.0008) [2023-12-27 02:36:59,834][105620] Updated weights for policy 1, policy_version 1543720 (0.0007) [2023-12-27 02:37:00,064][105692] Updated weights for policy 0, policy_version 1540685 (0.0009) [2023-12-27 02:37:00,122][105692] Updated weights for policy 0, policy_version 1540695 (0.0010) [2023-12-27 02:37:00,178][105692] Updated weights for policy 0, policy_version 1540705 (0.0010) [2023-12-27 02:37:00,561][105620] Updated weights for policy 1, policy_version 1543730 (0.0008) [2023-12-27 02:37:00,616][105620] Updated weights for policy 1, policy_version 1543741 (0.0008) [2023-12-27 02:37:00,665][105620] Updated weights for policy 1, policy_version 1543751 (0.0008) [2023-12-27 02:37:00,906][105692] Updated weights for policy 0, policy_version 1540715 (0.0010) [2023-12-27 02:37:00,950][105692] Updated weights for policy 0, policy_version 1540725 (0.0010) [2023-12-27 02:37:01,004][105692] Updated weights for policy 0, policy_version 1540735 (0.0010) [2023-12-27 02:37:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 789741568. Throughput: 0: 9737.4, 1: 9941.5. Samples: 789708732. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:37:01,063][104569] Avg episode reward: [(0, '8548.579'), (1, '8902.396')] [2023-12-27 02:37:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001540744_394485760.pth... [2023-12-27 02:37:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001543752_395255808.pth... [2023-12-27 02:37:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001539624_394199040.pth [2023-12-27 02:37:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001542600_394960896.pth [2023-12-27 02:37:01,329][105620] Updated weights for policy 1, policy_version 1543761 (0.0008) [2023-12-27 02:37:01,390][105620] Updated weights for policy 1, policy_version 1543771 (0.0006) [2023-12-27 02:37:01,453][105620] Updated weights for policy 1, policy_version 1543781 (0.0007) [2023-12-27 02:37:01,769][105692] Updated weights for policy 0, policy_version 1540745 (0.0010) [2023-12-27 02:37:01,835][105692] Updated weights for policy 0, policy_version 1540755 (0.0008) [2023-12-27 02:37:01,895][105692] Updated weights for policy 0, policy_version 1540765 (0.0008) [2023-12-27 02:37:01,957][105692] Updated weights for policy 0, policy_version 1540775 (0.0005) [2023-12-27 02:37:02,228][105620] Updated weights for policy 1, policy_version 1543791 (0.0009) [2023-12-27 02:37:02,286][105620] Updated weights for policy 1, policy_version 1543801 (0.0011) [2023-12-27 02:37:02,353][105620] Updated weights for policy 1, policy_version 1543811 (0.0010) [2023-12-27 02:37:02,606][105692] Updated weights for policy 0, policy_version 1540785 (0.0010) [2023-12-27 02:37:02,660][105692] Updated weights for policy 0, policy_version 1540795 (0.0009) [2023-12-27 02:37:02,713][105692] Updated weights for policy 0, policy_version 1540805 (0.0010) [2023-12-27 02:37:02,957][105620] Updated weights for policy 1, policy_version 1543821 (0.0009) [2023-12-27 02:37:03,019][105620] Updated weights for policy 1, policy_version 1543831 (0.0008) [2023-12-27 02:37:03,075][105620] Updated weights for policy 1, policy_version 1543841 (0.0005) [2023-12-27 02:37:03,428][105692] Updated weights for policy 0, policy_version 1540815 (0.0007) [2023-12-27 02:37:03,486][105692] Updated weights for policy 0, policy_version 1540825 (0.0010) [2023-12-27 02:37:03,543][105692] Updated weights for policy 0, policy_version 1540835 (0.0008) [2023-12-27 02:37:03,774][105620] Updated weights for policy 1, policy_version 1543851 (0.0009) [2023-12-27 02:37:03,831][105620] Updated weights for policy 1, policy_version 1543861 (0.0009) [2023-12-27 02:37:03,889][105620] Updated weights for policy 1, policy_version 1543871 (0.0008) [2023-12-27 02:37:04,226][105692] Updated weights for policy 0, policy_version 1540845 (0.0008) [2023-12-27 02:37:04,282][105692] Updated weights for policy 0, policy_version 1540855 (0.0011) [2023-12-27 02:37:04,334][105692] Updated weights for policy 0, policy_version 1540865 (0.0011) [2023-12-27 02:37:04,636][105620] Updated weights for policy 1, policy_version 1543881 (0.0007) [2023-12-27 02:37:04,684][105620] Updated weights for policy 1, policy_version 1543891 (0.0010) [2023-12-27 02:37:04,743][105620] Updated weights for policy 1, policy_version 1543901 (0.0010) [2023-12-27 02:37:04,805][105620] Updated weights for policy 1, policy_version 1543911 (0.0008) [2023-12-27 02:37:05,033][105692] Updated weights for policy 0, policy_version 1540875 (0.0009) [2023-12-27 02:37:05,082][105692] Updated weights for policy 0, policy_version 1540885 (0.0007) [2023-12-27 02:37:05,133][105692] Updated weights for policy 0, policy_version 1540895 (0.0009) [2023-12-27 02:37:05,557][105620] Updated weights for policy 1, policy_version 1543921 (0.0010) [2023-12-27 02:37:05,606][105620] Updated weights for policy 1, policy_version 1543931 (0.0010) [2023-12-27 02:37:05,668][105620] Updated weights for policy 1, policy_version 1543941 (0.0010) [2023-12-27 02:37:05,890][105692] Updated weights for policy 0, policy_version 1540905 (0.0008) [2023-12-27 02:37:05,942][105692] Updated weights for policy 0, policy_version 1540915 (0.0008) [2023-12-27 02:37:05,986][105692] Updated weights for policy 0, policy_version 1540925 (0.0007) [2023-12-27 02:37:06,037][105692] Updated weights for policy 0, policy_version 1540935 (0.0008) [2023-12-27 02:37:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 789839872. Throughput: 0: 9663.6, 1: 9871.7. Samples: 789824864. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:37:06,062][104569] Avg episode reward: [(0, '8629.898'), (1, '8900.725')] [2023-12-27 02:37:06,427][105620] Updated weights for policy 1, policy_version 1543951 (0.0011) [2023-12-27 02:37:06,480][105620] Updated weights for policy 1, policy_version 1543961 (0.0010) [2023-12-27 02:37:06,543][105620] Updated weights for policy 1, policy_version 1543971 (0.0011) [2023-12-27 02:37:06,568][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000003 [2023-12-27 02:37:06,832][105692] Updated weights for policy 0, policy_version 1540945 (0.0007) [2023-12-27 02:37:06,888][105692] Updated weights for policy 0, policy_version 1540955 (0.0010) [2023-12-27 02:37:06,946][105692] Updated weights for policy 0, policy_version 1540965 (0.0010) [2023-12-27 02:37:07,304][105620] Updated weights for policy 1, policy_version 1543981 (0.0010) [2023-12-27 02:37:07,359][105620] Updated weights for policy 1, policy_version 1543991 (0.0010) [2023-12-27 02:37:07,414][105620] Updated weights for policy 1, policy_version 1544001 (0.0010) [2023-12-27 02:37:07,573][105692] Updated weights for policy 0, policy_version 1540975 (0.0010) [2023-12-27 02:37:07,630][105692] Updated weights for policy 0, policy_version 1540985 (0.0010) [2023-12-27 02:37:07,680][105692] Updated weights for policy 0, policy_version 1540995 (0.0007) [2023-12-27 02:37:08,157][105620] Updated weights for policy 1, policy_version 1544011 (0.0009) [2023-12-27 02:37:08,219][105620] Updated weights for policy 1, policy_version 1544021 (0.0005) [2023-12-27 02:37:08,277][105620] Updated weights for policy 1, policy_version 1544031 (0.0005) [2023-12-27 02:37:08,327][105692] Updated weights for policy 0, policy_version 1541005 (0.0006) [2023-12-27 02:37:08,393][105692] Updated weights for policy 0, policy_version 1541015 (0.0009) [2023-12-27 02:37:08,454][105692] Updated weights for policy 0, policy_version 1541025 (0.0008) [2023-12-27 02:37:08,801][105620] Updated weights for policy 1, policy_version 1544041 (0.0007) [2023-12-27 02:37:08,865][105620] Updated weights for policy 1, policy_version 1544051 (0.0008) [2023-12-27 02:37:08,924][105620] Updated weights for policy 1, policy_version 1544061 (0.0007) [2023-12-27 02:37:08,982][105620] Updated weights for policy 1, policy_version 1544071 (0.0009) [2023-12-27 02:37:09,257][105692] Updated weights for policy 0, policy_version 1541035 (0.0010) [2023-12-27 02:37:09,319][105692] Updated weights for policy 0, policy_version 1541045 (0.0010) [2023-12-27 02:37:09,389][105692] Updated weights for policy 0, policy_version 1541055 (0.0011) [2023-12-27 02:37:09,691][105620] Updated weights for policy 1, policy_version 1544081 (0.0011) [2023-12-27 02:37:09,754][105620] Updated weights for policy 1, policy_version 1544091 (0.0011) [2023-12-27 02:37:09,821][105620] Updated weights for policy 1, policy_version 1544101 (0.0010) [2023-12-27 02:37:10,197][105692] Updated weights for policy 0, policy_version 1541065 (0.0010) [2023-12-27 02:37:10,255][105692] Updated weights for policy 0, policy_version 1541075 (0.0009) [2023-12-27 02:37:10,317][105692] Updated weights for policy 0, policy_version 1541085 (0.0008) [2023-12-27 02:37:10,382][105692] Updated weights for policy 0, policy_version 1541095 (0.0008) [2023-12-27 02:37:10,476][105620] Updated weights for policy 1, policy_version 1544111 (0.0010) [2023-12-27 02:37:10,543][105620] Updated weights for policy 1, policy_version 1544121 (0.0011) [2023-12-27 02:37:10,607][105620] Updated weights for policy 1, policy_version 1544131 (0.0011) [2023-12-27 02:37:11,054][105692] Updated weights for policy 0, policy_version 1541105 (0.0010) [2023-12-27 02:37:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 789929984. Throughput: 0: 9626.9, 1: 10017.3. Samples: 789941696. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:37:11,062][104569] Avg episode reward: [(0, '8808.797'), (1, '8808.985')] [2023-12-27 02:37:11,113][105692] Updated weights for policy 0, policy_version 1541115 (0.0011) [2023-12-27 02:37:11,176][105692] Updated weights for policy 0, policy_version 1541125 (0.0010) [2023-12-27 02:37:11,265][105620] Updated weights for policy 1, policy_version 1544141 (0.0010) [2023-12-27 02:37:11,325][105620] Updated weights for policy 1, policy_version 1544151 (0.0009) [2023-12-27 02:37:11,391][105620] Updated weights for policy 1, policy_version 1544161 (0.0009) [2023-12-27 02:37:11,886][105692] Updated weights for policy 0, policy_version 1541135 (0.0011) [2023-12-27 02:37:11,948][105692] Updated weights for policy 0, policy_version 1541145 (0.0010) [2023-12-27 02:37:12,011][105692] Updated weights for policy 0, policy_version 1541155 (0.0009) [2023-12-27 02:37:12,155][105620] Updated weights for policy 1, policy_version 1544171 (0.0009) [2023-12-27 02:37:12,214][105620] Updated weights for policy 1, policy_version 1544181 (0.0008) [2023-12-27 02:37:12,278][105620] Updated weights for policy 1, policy_version 1544191 (0.0009) [2023-12-27 02:37:12,755][105692] Updated weights for policy 0, policy_version 1541165 (0.0011) [2023-12-27 02:37:12,819][105692] Updated weights for policy 0, policy_version 1541175 (0.0011) [2023-12-27 02:37:12,882][105692] Updated weights for policy 0, policy_version 1541185 (0.0011) [2023-12-27 02:37:12,999][105620] Updated weights for policy 1, policy_version 1544201 (0.0008) [2023-12-27 02:37:13,056][105620] Updated weights for policy 1, policy_version 1544211 (0.0007) [2023-12-27 02:37:13,115][105620] Updated weights for policy 1, policy_version 1544221 (0.0009) [2023-12-27 02:37:13,175][105620] Updated weights for policy 1, policy_version 1544231 (0.0011) [2023-12-27 02:37:13,624][105692] Updated weights for policy 0, policy_version 1541195 (0.0011) [2023-12-27 02:37:13,687][105692] Updated weights for policy 0, policy_version 1541205 (0.0011) [2023-12-27 02:37:13,745][105692] Updated weights for policy 0, policy_version 1541215 (0.0011) [2023-12-27 02:37:13,903][105620] Updated weights for policy 1, policy_version 1544241 (0.0008) [2023-12-27 02:37:13,950][105620] Updated weights for policy 1, policy_version 1544251 (0.0005) [2023-12-27 02:37:14,004][105620] Updated weights for policy 1, policy_version 1544261 (0.0007) [2023-12-27 02:37:14,438][105692] Updated weights for policy 0, policy_version 1541225 (0.0011) [2023-12-27 02:37:14,487][105692] Updated weights for policy 0, policy_version 1541235 (0.0011) [2023-12-27 02:37:14,548][105692] Updated weights for policy 0, policy_version 1541245 (0.0010) [2023-12-27 02:37:14,600][105692] Updated weights for policy 0, policy_version 1541255 (0.0010) [2023-12-27 02:37:14,794][105620] Updated weights for policy 1, policy_version 1544271 (0.0008) [2023-12-27 02:37:14,853][105620] Updated weights for policy 1, policy_version 1544281 (0.0007) [2023-12-27 02:37:14,909][105620] Updated weights for policy 1, policy_version 1544291 (0.0006) [2023-12-27 02:37:15,405][105692] Updated weights for policy 0, policy_version 1541265 (0.0011) [2023-12-27 02:37:15,468][105692] Updated weights for policy 0, policy_version 1541275 (0.0011) [2023-12-27 02:37:15,527][105620] Updated weights for policy 1, policy_version 1544301 (0.0006) [2023-12-27 02:37:15,532][105692] Updated weights for policy 0, policy_version 1541285 (0.0010) [2023-12-27 02:37:15,578][105620] Updated weights for policy 1, policy_version 1544311 (0.0006) [2023-12-27 02:37:15,627][105620] Updated weights for policy 1, policy_version 1544321 (0.0005) [2023-12-27 02:37:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 790028288. Throughput: 0: 9609.3, 1: 9874.4. Samples: 789999512. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:37:16,062][104569] Avg episode reward: [(0, '8719.846'), (1, '8993.827')] [2023-12-27 02:37:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001541288_394625024.pth... [2023-12-27 02:37:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001544328_395403264.pth... [2023-12-27 02:37:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001540168_394338304.pth [2023-12-27 02:37:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001543176_395108352.pth [2023-12-27 02:37:16,210][105692] Updated weights for policy 0, policy_version 1541295 (0.0009) [2023-12-27 02:37:16,261][105692] Updated weights for policy 0, policy_version 1541305 (0.0008) [2023-12-27 02:37:16,315][105692] Updated weights for policy 0, policy_version 1541315 (0.0009) [2023-12-27 02:37:16,342][105620] Updated weights for policy 1, policy_version 1544331 (0.0005) [2023-12-27 02:37:16,393][105620] Updated weights for policy 1, policy_version 1544341 (0.0007) [2023-12-27 02:37:16,436][105586] KL-divergence is very high: 146.6242 [2023-12-27 02:37:16,447][105620] Updated weights for policy 1, policy_version 1544351 (0.0010) [2023-12-27 02:37:16,476][105586] KL-divergence is very high: 207.7990 [2023-12-27 02:37:17,073][105620] Updated weights for policy 1, policy_version 1544361 (0.0010) [2023-12-27 02:37:17,119][105620] Updated weights for policy 1, policy_version 1544371 (0.0005) [2023-12-27 02:37:17,151][105692] Updated weights for policy 0, policy_version 1541325 (0.0008) [2023-12-27 02:37:17,173][105620] Updated weights for policy 1, policy_version 1544381 (0.0005) [2023-12-27 02:37:17,200][105692] Updated weights for policy 0, policy_version 1541335 (0.0009) [2023-12-27 02:37:17,227][105620] Updated weights for policy 1, policy_version 1544391 (0.0005) [2023-12-27 02:37:17,251][105692] Updated weights for policy 0, policy_version 1541345 (0.0008) [2023-12-27 02:37:17,810][105620] Updated weights for policy 1, policy_version 1544401 (0.0010) [2023-12-27 02:37:17,878][105620] Updated weights for policy 1, policy_version 1544411 (0.0007) [2023-12-27 02:37:17,939][105620] Updated weights for policy 1, policy_version 1544421 (0.0005) [2023-12-27 02:37:17,974][105692] Updated weights for policy 0, policy_version 1541355 (0.0007) [2023-12-27 02:37:18,028][105692] Updated weights for policy 0, policy_version 1541365 (0.0009) [2023-12-27 02:37:18,094][105692] Updated weights for policy 0, policy_version 1541375 (0.0008) [2023-12-27 02:37:18,591][105620] Updated weights for policy 1, policy_version 1544431 (0.0008) [2023-12-27 02:37:18,652][105620] Updated weights for policy 1, policy_version 1544441 (0.0006) [2023-12-27 02:37:18,703][105620] Updated weights for policy 1, policy_version 1544451 (0.0005) [2023-12-27 02:37:18,894][105692] Updated weights for policy 0, policy_version 1541385 (0.0007) [2023-12-27 02:37:18,954][105692] Updated weights for policy 0, policy_version 1541395 (0.0010) [2023-12-27 02:37:19,018][105692] Updated weights for policy 0, policy_version 1541405 (0.0009) [2023-12-27 02:37:19,075][105692] Updated weights for policy 0, policy_version 1541415 (0.0009) [2023-12-27 02:37:19,329][105620] Updated weights for policy 1, policy_version 1544461 (0.0007) [2023-12-27 02:37:19,396][105620] Updated weights for policy 1, policy_version 1544471 (0.0008) [2023-12-27 02:37:19,463][105620] Updated weights for policy 1, policy_version 1544481 (0.0006) [2023-12-27 02:37:19,898][105692] Updated weights for policy 0, policy_version 1541425 (0.0009) [2023-12-27 02:37:19,955][105692] Updated weights for policy 0, policy_version 1541435 (0.0009) [2023-12-27 02:37:20,010][105692] Updated weights for policy 0, policy_version 1541445 (0.0008) [2023-12-27 02:37:20,158][105620] Updated weights for policy 1, policy_version 1544491 (0.0008) [2023-12-27 02:37:20,224][105620] Updated weights for policy 1, policy_version 1544501 (0.0009) [2023-12-27 02:37:20,290][105620] Updated weights for policy 1, policy_version 1544511 (0.0009) [2023-12-27 02:37:20,806][105692] Updated weights for policy 0, policy_version 1541455 (0.0009) [2023-12-27 02:37:20,863][105692] Updated weights for policy 0, policy_version 1541465 (0.0008) [2023-12-27 02:37:20,913][105692] Updated weights for policy 0, policy_version 1541475 (0.0008) [2023-12-27 02:37:21,056][105620] Updated weights for policy 1, policy_version 1544521 (0.0009) [2023-12-27 02:37:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 790126592. Throughput: 0: 9567.5, 1: 9940.4. Samples: 790117248. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:37:21,063][104569] Avg episode reward: [(0, '8634.031'), (1, '8905.009')] [2023-12-27 02:37:21,120][105620] Updated weights for policy 1, policy_version 1544531 (0.0011) [2023-12-27 02:37:21,186][105620] Updated weights for policy 1, policy_version 1544541 (0.0006) [2023-12-27 02:37:21,244][105620] Updated weights for policy 1, policy_version 1544551 (0.0007) [2023-12-27 02:37:21,755][105692] Updated weights for policy 0, policy_version 1541485 (0.0008) [2023-12-27 02:37:21,823][105692] Updated weights for policy 0, policy_version 1541495 (0.0005) [2023-12-27 02:37:21,885][105692] Updated weights for policy 0, policy_version 1541505 (0.0005) [2023-12-27 02:37:21,921][105620] Updated weights for policy 1, policy_version 1544561 (0.0009) [2023-12-27 02:37:21,975][105620] Updated weights for policy 1, policy_version 1544571 (0.0010) [2023-12-27 02:37:22,036][105620] Updated weights for policy 1, policy_version 1544581 (0.0009) [2023-12-27 02:37:22,543][105692] Updated weights for policy 0, policy_version 1541515 (0.0005) [2023-12-27 02:37:22,605][105692] Updated weights for policy 0, policy_version 1541525 (0.0006) [2023-12-27 02:37:22,664][105692] Updated weights for policy 0, policy_version 1541535 (0.0006) [2023-12-27 02:37:22,829][105620] Updated weights for policy 1, policy_version 1544591 (0.0010) [2023-12-27 02:37:22,880][105620] Updated weights for policy 1, policy_version 1544601 (0.0009) [2023-12-27 02:37:22,927][105620] Updated weights for policy 1, policy_version 1544611 (0.0008) [2023-12-27 02:37:23,363][105692] Updated weights for policy 0, policy_version 1541545 (0.0007) [2023-12-27 02:37:23,419][105692] Updated weights for policy 0, policy_version 1541555 (0.0009) [2023-12-27 02:37:23,474][105692] Updated weights for policy 0, policy_version 1541565 (0.0009) [2023-12-27 02:37:23,521][105692] Updated weights for policy 0, policy_version 1541575 (0.0009) [2023-12-27 02:37:23,683][105620] Updated weights for policy 1, policy_version 1544621 (0.0009) [2023-12-27 02:37:23,744][105620] Updated weights for policy 1, policy_version 1544631 (0.0009) [2023-12-27 02:37:23,801][105620] Updated weights for policy 1, policy_version 1544641 (0.0008) [2023-12-27 02:37:24,294][105692] Updated weights for policy 0, policy_version 1541585 (0.0008) [2023-12-27 02:37:24,357][105692] Updated weights for policy 0, policy_version 1541595 (0.0008) [2023-12-27 02:37:24,406][105692] Updated weights for policy 0, policy_version 1541605 (0.0008) [2023-12-27 02:37:24,538][105620] Updated weights for policy 1, policy_version 1544651 (0.0009) [2023-12-27 02:37:24,582][105620] Updated weights for policy 1, policy_version 1544661 (0.0010) [2023-12-27 02:37:24,641][105620] Updated weights for policy 1, policy_version 1544671 (0.0011) [2023-12-27 02:37:25,246][105620] Updated weights for policy 1, policy_version 1544681 (0.0010) [2023-12-27 02:37:25,253][105692] Updated weights for policy 0, policy_version 1541615 (0.0009) [2023-12-27 02:37:25,307][105620] Updated weights for policy 1, policy_version 1544691 (0.0006) [2023-12-27 02:37:25,310][105692] Updated weights for policy 0, policy_version 1541625 (0.0008) [2023-12-27 02:37:25,363][105620] Updated weights for policy 1, policy_version 1544701 (0.0005) [2023-12-27 02:37:25,365][105692] Updated weights for policy 0, policy_version 1541635 (0.0008) [2023-12-27 02:37:25,420][105620] Updated weights for policy 1, policy_version 1544711 (0.0005) [2023-12-27 02:37:26,038][105620] Updated weights for policy 1, policy_version 1544721 (0.0007) [2023-12-27 02:37:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 790216704. Throughput: 0: 9526.1, 1: 9998.9. Samples: 790230024. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:37:26,062][104569] Avg episode reward: [(0, '8722.537'), (1, '8993.983')] [2023-12-27 02:37:26,089][105620] Updated weights for policy 1, policy_version 1544731 (0.0005) [2023-12-27 02:37:26,149][105620] Updated weights for policy 1, policy_version 1544741 (0.0008) [2023-12-27 02:37:26,170][105692] Updated weights for policy 0, policy_version 1541645 (0.0010) [2023-12-27 02:37:26,226][105692] Updated weights for policy 0, policy_version 1541655 (0.0009) [2023-12-27 02:37:26,281][105692] Updated weights for policy 0, policy_version 1541666 (0.0010) [2023-12-27 02:37:26,705][105620] Updated weights for policy 1, policy_version 1544751 (0.0006) [2023-12-27 02:37:26,759][105620] Updated weights for policy 1, policy_version 1544761 (0.0005) [2023-12-27 02:37:26,812][105620] Updated weights for policy 1, policy_version 1544771 (0.0005) [2023-12-27 02:37:27,195][105692] Updated weights for policy 0, policy_version 1541677 (0.0009) [2023-12-27 02:37:27,261][105692] Updated weights for policy 0, policy_version 1541687 (0.0009) [2023-12-27 02:37:27,323][105692] Updated weights for policy 0, policy_version 1541697 (0.0009) [2023-12-27 02:37:27,398][105620] Updated weights for policy 1, policy_version 1544781 (0.0006) [2023-12-27 02:37:27,444][105620] Updated weights for policy 1, policy_version 1544791 (0.0009) [2023-12-27 02:37:27,496][105620] Updated weights for policy 1, policy_version 1544801 (0.0005) [2023-12-27 02:37:28,085][105692] Updated weights for policy 0, policy_version 1541707 (0.0009) [2023-12-27 02:37:28,136][105692] Updated weights for policy 0, policy_version 1541717 (0.0009) [2023-12-27 02:37:28,189][105692] Updated weights for policy 0, policy_version 1541727 (0.0009) [2023-12-27 02:37:28,222][105620] Updated weights for policy 1, policy_version 1544811 (0.0006) [2023-12-27 02:37:28,271][105620] Updated weights for policy 1, policy_version 1544821 (0.0009) [2023-12-27 02:37:28,321][105620] Updated weights for policy 1, policy_version 1544831 (0.0009) [2023-12-27 02:37:28,969][105692] Updated weights for policy 0, policy_version 1541737 (0.0008) [2023-12-27 02:37:29,028][105692] Updated weights for policy 0, policy_version 1541747 (0.0010) [2023-12-27 02:37:29,073][105620] Updated weights for policy 1, policy_version 1544841 (0.0009) [2023-12-27 02:37:29,083][105692] Updated weights for policy 0, policy_version 1541757 (0.0009) [2023-12-27 02:37:29,127][105620] Updated weights for policy 1, policy_version 1544851 (0.0007) [2023-12-27 02:37:29,142][105692] Updated weights for policy 0, policy_version 1541767 (0.0006) [2023-12-27 02:37:29,179][105620] Updated weights for policy 1, policy_version 1544862 (0.0008) [2023-12-27 02:37:29,238][105620] Updated weights for policy 1, policy_version 1544872 (0.0008) [2023-12-27 02:37:29,907][105692] Updated weights for policy 0, policy_version 1541777 (0.0008) [2023-12-27 02:37:29,970][105692] Updated weights for policy 0, policy_version 1541787 (0.0011) [2023-12-27 02:37:30,032][105692] Updated weights for policy 0, policy_version 1541797 (0.0009) [2023-12-27 02:37:30,053][105620] Updated weights for policy 1, policy_version 1544882 (0.0009) [2023-12-27 02:37:30,112][105620] Updated weights for policy 1, policy_version 1544892 (0.0009) [2023-12-27 02:37:30,143][105586] KL-divergence is very high: 178.6970 [2023-12-27 02:37:30,170][105620] Updated weights for policy 1, policy_version 1544902 (0.0009) [2023-12-27 02:37:30,779][105620] Updated weights for policy 1, policy_version 1544912 (0.0009) [2023-12-27 02:37:30,850][105620] Updated weights for policy 1, policy_version 1544922 (0.0009) [2023-12-27 02:37:30,851][105692] Updated weights for policy 0, policy_version 1541807 (0.0007) [2023-12-27 02:37:30,910][105620] Updated weights for policy 1, policy_version 1544932 (0.0008) [2023-12-27 02:37:30,930][105692] Updated weights for policy 0, policy_version 1541817 (0.0008) [2023-12-27 02:37:30,986][105692] Updated weights for policy 0, policy_version 1541827 (0.0009) [2023-12-27 02:37:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 790323200. Throughput: 0: 9474.8, 1: 10085.8. Samples: 790287988. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:37:31,062][104569] Avg episode reward: [(0, '8627.156'), (1, '9264.520')] [2023-12-27 02:37:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001541832_394764288.pth... [2023-12-27 02:37:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001544936_395558912.pth... [2023-12-27 02:37:31,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001540744_394485760.pth [2023-12-27 02:37:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001543752_395255808.pth [2023-12-27 02:37:31,534][105620] Updated weights for policy 1, policy_version 1544942 (0.0008) [2023-12-27 02:37:31,587][105620] Updated weights for policy 1, policy_version 1544952 (0.0009) [2023-12-27 02:37:31,642][105620] Updated weights for policy 1, policy_version 1544962 (0.0009) [2023-12-27 02:37:31,774][105692] Updated weights for policy 0, policy_version 1541837 (0.0009) [2023-12-27 02:37:31,825][105692] Updated weights for policy 0, policy_version 1541847 (0.0009) [2023-12-27 02:37:31,879][105692] Updated weights for policy 0, policy_version 1541857 (0.0009) [2023-12-27 02:37:32,397][105620] Updated weights for policy 1, policy_version 1544972 (0.0008) [2023-12-27 02:37:32,457][105620] Updated weights for policy 1, policy_version 1544982 (0.0009) [2023-12-27 02:37:32,524][105620] Updated weights for policy 1, policy_version 1544992 (0.0010) [2023-12-27 02:37:32,612][105692] Updated weights for policy 0, policy_version 1541867 (0.0009) [2023-12-27 02:37:32,669][105692] Updated weights for policy 0, policy_version 1541877 (0.0010) [2023-12-27 02:37:32,732][105692] Updated weights for policy 0, policy_version 1541887 (0.0009) [2023-12-27 02:37:33,277][105620] Updated weights for policy 1, policy_version 1545002 (0.0009) [2023-12-27 02:37:33,329][105620] Updated weights for policy 1, policy_version 1545012 (0.0009) [2023-12-27 02:37:33,358][105692] Updated weights for policy 0, policy_version 1541897 (0.0006) [2023-12-27 02:37:33,377][105620] Updated weights for policy 1, policy_version 1545022 (0.0008) [2023-12-27 02:37:33,410][105692] Updated weights for policy 0, policy_version 1541907 (0.0006) [2023-12-27 02:37:33,436][105620] Updated weights for policy 1, policy_version 1545032 (0.0008) [2023-12-27 02:37:33,467][105692] Updated weights for policy 0, policy_version 1541917 (0.0005) [2023-12-27 02:37:33,522][105692] Updated weights for policy 0, policy_version 1541927 (0.0006) [2023-12-27 02:37:34,106][105692] Updated weights for policy 0, policy_version 1541937 (0.0008) [2023-12-27 02:37:34,169][105692] Updated weights for policy 0, policy_version 1541947 (0.0009) [2023-12-27 02:37:34,232][105692] Updated weights for policy 0, policy_version 1541957 (0.0009) [2023-12-27 02:37:34,286][105620] Updated weights for policy 1, policy_version 1545042 (0.0009) [2023-12-27 02:37:34,337][105620] Updated weights for policy 1, policy_version 1545052 (0.0008) [2023-12-27 02:37:34,398][105620] Updated weights for policy 1, policy_version 1545062 (0.0008) [2023-12-27 02:37:34,969][105692] Updated weights for policy 0, policy_version 1541967 (0.0008) [2023-12-27 02:37:35,024][105692] Updated weights for policy 0, policy_version 1541977 (0.0010) [2023-12-27 02:37:35,078][105692] Updated weights for policy 0, policy_version 1541987 (0.0006) [2023-12-27 02:37:35,159][105620] Updated weights for policy 1, policy_version 1545072 (0.0009) [2023-12-27 02:37:35,220][105620] Updated weights for policy 1, policy_version 1545082 (0.0009) [2023-12-27 02:37:35,290][105620] Updated weights for policy 1, policy_version 1545092 (0.0010) [2023-12-27 02:37:35,685][105692] Updated weights for policy 0, policy_version 1541997 (0.0007) [2023-12-27 02:37:35,736][105692] Updated weights for policy 0, policy_version 1542007 (0.0009) [2023-12-27 02:37:35,795][105692] Updated weights for policy 0, policy_version 1542017 (0.0009) [2023-12-27 02:37:36,029][105620] Updated weights for policy 1, policy_version 1545102 (0.0010) [2023-12-27 02:37:36,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 790413312. Throughput: 0: 9450.6, 1: 9951.3. Samples: 790402936. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:37:36,063][104569] Avg episode reward: [(0, '8356.649'), (1, '8912.691')] [2023-12-27 02:37:36,088][105620] Updated weights for policy 1, policy_version 1545112 (0.0009) [2023-12-27 02:37:36,153][105620] Updated weights for policy 1, policy_version 1545122 (0.0008) [2023-12-27 02:37:36,616][105692] Updated weights for policy 0, policy_version 1542027 (0.0010) [2023-12-27 02:37:36,687][105692] Updated weights for policy 0, policy_version 1542037 (0.0009) [2023-12-27 02:37:36,750][105692] Updated weights for policy 0, policy_version 1542047 (0.0009) [2023-12-27 02:37:36,841][105620] Updated weights for policy 1, policy_version 1545132 (0.0007) [2023-12-27 02:37:36,890][105620] Updated weights for policy 1, policy_version 1545142 (0.0008) [2023-12-27 02:37:36,952][105620] Updated weights for policy 1, policy_version 1545152 (0.0009) [2023-12-27 02:37:37,549][105692] Updated weights for policy 0, policy_version 1542057 (0.0009) [2023-12-27 02:37:37,604][105692] Updated weights for policy 0, policy_version 1542068 (0.0010) [2023-12-27 02:37:37,624][105620] Updated weights for policy 1, policy_version 1545162 (0.0008) [2023-12-27 02:37:37,653][105692] Updated weights for policy 0, policy_version 1542078 (0.0010) [2023-12-27 02:37:37,672][105620] Updated weights for policy 1, policy_version 1545172 (0.0006) [2023-12-27 02:37:37,712][105692] Updated weights for policy 0, policy_version 1542088 (0.0008) [2023-12-27 02:37:37,725][105620] Updated weights for policy 1, policy_version 1545182 (0.0006) [2023-12-27 02:37:37,771][105620] Updated weights for policy 1, policy_version 1545192 (0.0005) [2023-12-27 02:37:38,501][105620] Updated weights for policy 1, policy_version 1545202 (0.0009) [2023-12-27 02:37:38,522][105692] Updated weights for policy 0, policy_version 1542098 (0.0008) [2023-12-27 02:37:38,562][105620] Updated weights for policy 1, policy_version 1545212 (0.0009) [2023-12-27 02:37:38,586][105692] Updated weights for policy 0, policy_version 1542108 (0.0008) [2023-12-27 02:37:38,623][105620] Updated weights for policy 1, policy_version 1545222 (0.0010) [2023-12-27 02:37:38,646][105692] Updated weights for policy 0, policy_version 1542118 (0.0006) [2023-12-27 02:37:39,234][105692] Updated weights for policy 0, policy_version 1542128 (0.0008) [2023-12-27 02:37:39,297][105692] Updated weights for policy 0, policy_version 1542138 (0.0008) [2023-12-27 02:37:39,344][105620] Updated weights for policy 1, policy_version 1545232 (0.0011) [2023-12-27 02:37:39,364][105692] Updated weights for policy 0, policy_version 1542148 (0.0009) [2023-12-27 02:37:39,414][105620] Updated weights for policy 1, policy_version 1545242 (0.0009) [2023-12-27 02:37:39,476][105620] Updated weights for policy 1, policy_version 1545252 (0.0011) [2023-12-27 02:37:40,082][105692] Updated weights for policy 0, policy_version 1542158 (0.0008) [2023-12-27 02:37:40,139][105692] Updated weights for policy 0, policy_version 1542169 (0.0010) [2023-12-27 02:37:40,197][105692] Updated weights for policy 0, policy_version 1542180 (0.0010) [2023-12-27 02:37:40,215][105620] Updated weights for policy 1, policy_version 1545262 (0.0007) [2023-12-27 02:37:40,279][105620] Updated weights for policy 1, policy_version 1545272 (0.0008) [2023-12-27 02:37:40,337][105620] Updated weights for policy 1, policy_version 1545282 (0.0008) [2023-12-27 02:37:41,024][105692] Updated weights for policy 0, policy_version 1542190 (0.0009) [2023-12-27 02:37:41,043][105620] Updated weights for policy 1, policy_version 1545292 (0.0007) [2023-12-27 02:37:41,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 790503424. Throughput: 0: 9434.6, 1: 9852.2. Samples: 790517832. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:37:41,063][104569] Avg episode reward: [(0, '8176.433'), (1, '8559.604')] [2023-12-27 02:37:41,085][105692] Updated weights for policy 0, policy_version 1542200 (0.0008) [2023-12-27 02:37:41,113][105620] Updated weights for policy 1, policy_version 1545302 (0.0008) [2023-12-27 02:37:41,145][105692] Updated weights for policy 0, policy_version 1542210 (0.0008) [2023-12-27 02:37:41,181][105620] Updated weights for policy 1, policy_version 1545312 (0.0008) [2023-12-27 02:37:41,883][105692] Updated weights for policy 0, policy_version 1542220 (0.0007) [2023-12-27 02:37:41,938][105692] Updated weights for policy 0, policy_version 1542230 (0.0009) [2023-12-27 02:37:41,961][105620] Updated weights for policy 1, policy_version 1545322 (0.0010) [2023-12-27 02:37:42,000][105692] Updated weights for policy 0, policy_version 1542240 (0.0006) [2023-12-27 02:37:42,019][105620] Updated weights for policy 1, policy_version 1545332 (0.0009) [2023-12-27 02:37:42,068][105620] Updated weights for policy 1, policy_version 1545342 (0.0008) [2023-12-27 02:37:42,122][105620] Updated weights for policy 1, policy_version 1545352 (0.0008) [2023-12-27 02:37:42,835][105692] Updated weights for policy 0, policy_version 1542250 (0.0007) [2023-12-27 02:37:42,865][105620] Updated weights for policy 1, policy_version 1545362 (0.0008) [2023-12-27 02:37:42,880][105692] Updated weights for policy 0, policy_version 1542260 (0.0005) [2023-12-27 02:37:42,927][105620] Updated weights for policy 1, policy_version 1545372 (0.0010) [2023-12-27 02:37:42,942][105692] Updated weights for policy 0, policy_version 1542270 (0.0008) [2023-12-27 02:37:42,991][105620] Updated weights for policy 1, policy_version 1545382 (0.0009) [2023-12-27 02:37:42,993][105692] Updated weights for policy 0, policy_version 1542280 (0.0005) [2023-12-27 02:37:43,656][105692] Updated weights for policy 0, policy_version 1542290 (0.0009) [2023-12-27 02:37:43,705][105692] Updated weights for policy 0, policy_version 1542300 (0.0009) [2023-12-27 02:37:43,763][105620] Updated weights for policy 1, policy_version 1545392 (0.0008) [2023-12-27 02:37:43,765][105692] Updated weights for policy 0, policy_version 1542310 (0.0006) [2023-12-27 02:37:43,829][105620] Updated weights for policy 1, policy_version 1545402 (0.0009) [2023-12-27 02:37:43,889][105620] Updated weights for policy 1, policy_version 1545412 (0.0008) [2023-12-27 02:37:44,544][105692] Updated weights for policy 0, policy_version 1542320 (0.0008) [2023-12-27 02:37:44,596][105692] Updated weights for policy 0, policy_version 1542330 (0.0008) [2023-12-27 02:37:44,650][105620] Updated weights for policy 1, policy_version 1545423 (0.0010) [2023-12-27 02:37:44,659][105692] Updated weights for policy 0, policy_version 1542340 (0.0007) [2023-12-27 02:37:44,715][105620] Updated weights for policy 1, policy_version 1545433 (0.0006) [2023-12-27 02:37:44,781][105620] Updated weights for policy 1, policy_version 1545443 (0.0007) [2023-12-27 02:37:45,446][105692] Updated weights for policy 0, policy_version 1542350 (0.0008) [2023-12-27 02:37:45,497][105692] Updated weights for policy 0, policy_version 1542360 (0.0007) [2023-12-27 02:37:45,499][105620] Updated weights for policy 1, policy_version 1545453 (0.0008) [2023-12-27 02:37:45,549][105692] Updated weights for policy 0, policy_version 1542370 (0.0007) [2023-12-27 02:37:45,561][105620] Updated weights for policy 1, policy_version 1545463 (0.0011) [2023-12-27 02:37:45,613][105620] Updated weights for policy 1, policy_version 1545473 (0.0010) [2023-12-27 02:37:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 790601728. Throughput: 0: 9374.4, 1: 9832.9. Samples: 790573060. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:37:46,063][104569] Avg episode reward: [(0, '8718.132'), (1, '8554.277')] [2023-12-27 02:37:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001545480_395698176.pth... [2023-12-27 02:37:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001542376_394903552.pth... [2023-12-27 02:37:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001544328_395403264.pth [2023-12-27 02:37:46,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001541288_394625024.pth [2023-12-27 02:37:46,187][105620] Updated weights for policy 1, policy_version 1545483 (0.0010) [2023-12-27 02:37:46,252][105620] Updated weights for policy 1, policy_version 1545493 (0.0011) [2023-12-27 02:37:46,310][105620] Updated weights for policy 1, policy_version 1545503 (0.0010) [2023-12-27 02:37:46,422][105692] Updated weights for policy 0, policy_version 1542380 (0.0009) [2023-12-27 02:37:46,483][105692] Updated weights for policy 0, policy_version 1542390 (0.0009) [2023-12-27 02:37:46,541][105692] Updated weights for policy 0, policy_version 1542400 (0.0009) [2023-12-27 02:37:46,949][105620] Updated weights for policy 1, policy_version 1545513 (0.0009) [2023-12-27 02:37:47,003][105620] Updated weights for policy 1, policy_version 1545523 (0.0005) [2023-12-27 02:37:47,062][105620] Updated weights for policy 1, policy_version 1545533 (0.0008) [2023-12-27 02:37:47,123][105620] Updated weights for policy 1, policy_version 1545543 (0.0009) [2023-12-27 02:37:47,356][105692] Updated weights for policy 0, policy_version 1542410 (0.0009) [2023-12-27 02:37:47,410][105692] Updated weights for policy 0, policy_version 1542421 (0.0010) [2023-12-27 02:37:47,460][105692] Updated weights for policy 0, policy_version 1542431 (0.0009) [2023-12-27 02:37:47,719][105620] Updated weights for policy 1, policy_version 1545553 (0.0006) [2023-12-27 02:37:47,781][105620] Updated weights for policy 1, policy_version 1545563 (0.0008) [2023-12-27 02:37:47,836][105620] Updated weights for policy 1, policy_version 1545573 (0.0007) [2023-12-27 02:37:48,314][105692] Updated weights for policy 0, policy_version 1542443 (0.0010) [2023-12-27 02:37:48,374][105692] Updated weights for policy 0, policy_version 1542453 (0.0008) [2023-12-27 02:37:48,443][105692] Updated weights for policy 0, policy_version 1542463 (0.0009) [2023-12-27 02:37:48,495][105620] Updated weights for policy 1, policy_version 1545583 (0.0007) [2023-12-27 02:37:48,562][105620] Updated weights for policy 1, policy_version 1545593 (0.0008) [2023-12-27 02:37:48,626][105620] Updated weights for policy 1, policy_version 1545603 (0.0008) [2023-12-27 02:37:49,261][105620] Updated weights for policy 1, policy_version 1545613 (0.0008) [2023-12-27 02:37:49,291][105692] Updated weights for policy 0, policy_version 1542473 (0.0007) [2023-12-27 02:37:49,328][105620] Updated weights for policy 1, policy_version 1545623 (0.0005) [2023-12-27 02:37:49,355][105692] Updated weights for policy 0, policy_version 1542483 (0.0008) [2023-12-27 02:37:49,395][105620] Updated weights for policy 1, policy_version 1545633 (0.0008) [2023-12-27 02:37:49,421][105692] Updated weights for policy 0, policy_version 1542493 (0.0008) [2023-12-27 02:37:49,488][105692] Updated weights for policy 0, policy_version 1542503 (0.0006) [2023-12-27 02:37:50,054][105620] Updated weights for policy 1, policy_version 1545643 (0.0007) [2023-12-27 02:37:50,109][105620] Updated weights for policy 1, policy_version 1545653 (0.0007) [2023-12-27 02:37:50,124][105692] Updated weights for policy 0, policy_version 1542513 (0.0006) [2023-12-27 02:37:50,168][105620] Updated weights for policy 1, policy_version 1545663 (0.0008) [2023-12-27 02:37:50,177][105692] Updated weights for policy 0, policy_version 1542523 (0.0005) [2023-12-27 02:37:50,230][105692] Updated weights for policy 0, policy_version 1542533 (0.0006) [2023-12-27 02:37:50,892][105692] Updated weights for policy 0, policy_version 1542543 (0.0008) [2023-12-27 02:37:50,909][105620] Updated weights for policy 1, policy_version 1545673 (0.0007) [2023-12-27 02:37:50,952][105692] Updated weights for policy 0, policy_version 1542553 (0.0007) [2023-12-27 02:37:50,970][105620] Updated weights for policy 1, policy_version 1545683 (0.0008) [2023-12-27 02:37:51,012][105692] Updated weights for policy 0, policy_version 1542563 (0.0006) [2023-12-27 02:37:51,034][105620] Updated weights for policy 1, policy_version 1545693 (0.0008) [2023-12-27 02:37:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 790700032. Throughput: 0: 9267.6, 1: 9921.3. Samples: 790688368. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:37:51,062][104569] Avg episode reward: [(0, '8811.166'), (1, '9087.938')] [2023-12-27 02:37:51,103][105620] Updated weights for policy 1, policy_version 1545703 (0.0008) [2023-12-27 02:37:51,824][105620] Updated weights for policy 1, policy_version 1545713 (0.0007) [2023-12-27 02:37:51,833][105692] Updated weights for policy 0, policy_version 1542573 (0.0008) [2023-12-27 02:37:51,869][105620] Updated weights for policy 1, policy_version 1545723 (0.0006) [2023-12-27 02:37:51,887][105692] Updated weights for policy 0, policy_version 1542583 (0.0007) [2023-12-27 02:37:51,920][105620] Updated weights for policy 1, policy_version 1545733 (0.0006) [2023-12-27 02:37:51,935][105692] Updated weights for policy 0, policy_version 1542593 (0.0007) [2023-12-27 02:37:52,617][105620] Updated weights for policy 1, policy_version 1545743 (0.0009) [2023-12-27 02:37:52,679][105620] Updated weights for policy 1, policy_version 1545753 (0.0010) [2023-12-27 02:37:52,688][105692] Updated weights for policy 0, policy_version 1542603 (0.0009) [2023-12-27 02:37:52,728][105620] Updated weights for policy 1, policy_version 1545763 (0.0008) [2023-12-27 02:37:52,743][105692] Updated weights for policy 0, policy_version 1542613 (0.0007) [2023-12-27 02:37:52,801][105692] Updated weights for policy 0, policy_version 1542623 (0.0006) [2023-12-27 02:37:53,370][105692] Updated weights for policy 0, policy_version 1542633 (0.0005) [2023-12-27 02:37:53,431][105692] Updated weights for policy 0, policy_version 1542643 (0.0006) [2023-12-27 02:37:53,472][105620] Updated weights for policy 1, policy_version 1545773 (0.0010) [2023-12-27 02:37:53,494][105692] Updated weights for policy 0, policy_version 1542653 (0.0005) [2023-12-27 02:37:53,551][105692] Updated weights for policy 0, policy_version 1542663 (0.0005) [2023-12-27 02:37:53,559][105620] Updated weights for policy 1, policy_version 1545783 (0.0011) [2023-12-27 02:37:53,613][105620] Updated weights for policy 1, policy_version 1545793 (0.0010) [2023-12-27 02:37:54,218][105692] Updated weights for policy 0, policy_version 1542673 (0.0008) [2023-12-27 02:37:54,270][105692] Updated weights for policy 0, policy_version 1542683 (0.0008) [2023-12-27 02:37:54,328][105692] Updated weights for policy 0, policy_version 1542693 (0.0008) [2023-12-27 02:37:54,336][105620] Updated weights for policy 1, policy_version 1545803 (0.0010) [2023-12-27 02:37:54,398][105620] Updated weights for policy 1, policy_version 1545813 (0.0011) [2023-12-27 02:37:54,453][105620] Updated weights for policy 1, policy_version 1545823 (0.0010) [2023-12-27 02:37:55,093][105620] Updated weights for policy 1, policy_version 1545833 (0.0010) [2023-12-27 02:37:55,151][105620] Updated weights for policy 1, policy_version 1545843 (0.0008) [2023-12-27 02:37:55,157][105692] Updated weights for policy 0, policy_version 1542703 (0.0008) [2023-12-27 02:37:55,205][105620] Updated weights for policy 1, policy_version 1545853 (0.0010) [2023-12-27 02:37:55,210][105692] Updated weights for policy 0, policy_version 1542713 (0.0009) [2023-12-27 02:37:55,256][105620] Updated weights for policy 1, policy_version 1545863 (0.0010) [2023-12-27 02:37:55,263][105692] Updated weights for policy 0, policy_version 1542723 (0.0007) [2023-12-27 02:37:55,956][105620] Updated weights for policy 1, policy_version 1545873 (0.0010) [2023-12-27 02:37:56,007][105620] Updated weights for policy 1, policy_version 1545883 (0.0010) [2023-12-27 02:37:56,052][105620] Updated weights for policy 1, policy_version 1545893 (0.0010) [2023-12-27 02:37:56,057][105692] Updated weights for policy 0, policy_version 1542733 (0.0008) [2023-12-27 02:37:56,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 790790144. Throughput: 0: 9300.3, 1: 9894.7. Samples: 790805472. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:37:56,062][104569] Avg episode reward: [(0, '8814.417'), (1, '9084.380')] [2023-12-27 02:37:56,116][105692] Updated weights for policy 0, policy_version 1542743 (0.0007) [2023-12-27 02:37:56,175][105692] Updated weights for policy 0, policy_version 1542753 (0.0008) [2023-12-27 02:37:56,762][105620] Updated weights for policy 1, policy_version 1545903 (0.0007) [2023-12-27 02:37:56,817][105620] Updated weights for policy 1, policy_version 1545913 (0.0005) [2023-12-27 02:37:56,870][105620] Updated weights for policy 1, policy_version 1545923 (0.0005) [2023-12-27 02:37:57,001][105692] Updated weights for policy 0, policy_version 1542763 (0.0009) [2023-12-27 02:37:57,059][105692] Updated weights for policy 0, policy_version 1542773 (0.0010) [2023-12-27 02:37:57,113][105692] Updated weights for policy 0, policy_version 1542783 (0.0010) [2023-12-27 02:37:57,389][105620] Updated weights for policy 1, policy_version 1545933 (0.0008) [2023-12-27 02:37:57,438][105620] Updated weights for policy 1, policy_version 1545943 (0.0011) [2023-12-27 02:37:57,494][105620] Updated weights for policy 1, policy_version 1545953 (0.0011) [2023-12-27 02:37:57,900][105692] Updated weights for policy 0, policy_version 1542793 (0.0009) [2023-12-27 02:37:57,964][105692] Updated weights for policy 0, policy_version 1542803 (0.0008) [2023-12-27 02:37:58,027][105692] Updated weights for policy 0, policy_version 1542813 (0.0008) [2023-12-27 02:37:58,092][105692] Updated weights for policy 0, policy_version 1542823 (0.0008) [2023-12-27 02:37:58,250][105620] Updated weights for policy 1, policy_version 1545963 (0.0010) [2023-12-27 02:37:58,308][105620] Updated weights for policy 1, policy_version 1545973 (0.0008) [2023-12-27 02:37:58,374][105620] Updated weights for policy 1, policy_version 1545983 (0.0007) [2023-12-27 02:37:58,797][105692] Updated weights for policy 0, policy_version 1542833 (0.0009) [2023-12-27 02:37:58,862][105692] Updated weights for policy 0, policy_version 1542843 (0.0008) [2023-12-27 02:37:58,933][105692] Updated weights for policy 0, policy_version 1542853 (0.0007) [2023-12-27 02:37:59,218][105620] Updated weights for policy 1, policy_version 1545993 (0.0009) [2023-12-27 02:37:59,280][105620] Updated weights for policy 1, policy_version 1546003 (0.0009) [2023-12-27 02:37:59,341][105620] Updated weights for policy 1, policy_version 1546013 (0.0009) [2023-12-27 02:37:59,402][105620] Updated weights for policy 1, policy_version 1546023 (0.0008) [2023-12-27 02:37:59,754][105692] Updated weights for policy 0, policy_version 1542863 (0.0010) [2023-12-27 02:37:59,810][105692] Updated weights for policy 0, policy_version 1542873 (0.0011) [2023-12-27 02:37:59,878][105692] Updated weights for policy 0, policy_version 1542883 (0.0009) [2023-12-27 02:38:00,211][105620] Updated weights for policy 1, policy_version 1546033 (0.0009) [2023-12-27 02:38:00,256][105620] Updated weights for policy 1, policy_version 1546043 (0.0008) [2023-12-27 02:38:00,314][105620] Updated weights for policy 1, policy_version 1546053 (0.0008) [2023-12-27 02:38:00,570][105692] Updated weights for policy 0, policy_version 1542893 (0.0010) [2023-12-27 02:38:00,618][105692] Updated weights for policy 0, policy_version 1542903 (0.0010) [2023-12-27 02:38:00,671][105692] Updated weights for policy 0, policy_version 1542913 (0.0008) [2023-12-27 02:38:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19521.9). Total num frames: 790888448. Throughput: 0: 9251.2, 1: 9951.1. Samples: 790863616. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:38:01,062][104569] Avg episode reward: [(0, '8904.668'), (1, '8902.693')] [2023-12-27 02:38:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001542920_395042816.pth... [2023-12-27 02:38:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001546056_395845632.pth... [2023-12-27 02:38:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001541832_394764288.pth [2023-12-27 02:38:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001544936_395558912.pth [2023-12-27 02:38:01,165][105620] Updated weights for policy 1, policy_version 1546063 (0.0007) [2023-12-27 02:38:01,229][105620] Updated weights for policy 1, policy_version 1546073 (0.0008) [2023-12-27 02:38:01,276][105692] Updated weights for policy 0, policy_version 1542923 (0.0006) [2023-12-27 02:38:01,292][105620] Updated weights for policy 1, policy_version 1546083 (0.0008) [2023-12-27 02:38:01,333][105692] Updated weights for policy 0, policy_version 1542933 (0.0007) [2023-12-27 02:38:01,399][105692] Updated weights for policy 0, policy_version 1542943 (0.0007) [2023-12-27 02:38:01,959][105620] Updated weights for policy 1, policy_version 1546093 (0.0010) [2023-12-27 02:38:02,017][105620] Updated weights for policy 1, policy_version 1546103 (0.0009) [2023-12-27 02:38:02,031][105692] Updated weights for policy 0, policy_version 1542953 (0.0006) [2023-12-27 02:38:02,072][105620] Updated weights for policy 1, policy_version 1546113 (0.0007) [2023-12-27 02:38:02,091][105692] Updated weights for policy 0, policy_version 1542963 (0.0006) [2023-12-27 02:38:02,157][105692] Updated weights for policy 0, policy_version 1542973 (0.0011) [2023-12-27 02:38:02,219][105692] Updated weights for policy 0, policy_version 1542983 (0.0011) [2023-12-27 02:38:02,717][105620] Updated weights for policy 1, policy_version 1546123 (0.0005) [2023-12-27 02:38:02,775][105620] Updated weights for policy 1, policy_version 1546133 (0.0008) [2023-12-27 02:38:02,830][105620] Updated weights for policy 1, policy_version 1546143 (0.0010) [2023-12-27 02:38:02,867][105692] Updated weights for policy 0, policy_version 1542993 (0.0006) [2023-12-27 02:38:02,915][105692] Updated weights for policy 0, policy_version 1543003 (0.0005) [2023-12-27 02:38:02,972][105692] Updated weights for policy 0, policy_version 1543013 (0.0005) [2023-12-27 02:38:03,491][105692] Updated weights for policy 0, policy_version 1543023 (0.0005) [2023-12-27 02:38:03,544][105692] Updated weights for policy 0, policy_version 1543033 (0.0008) [2023-12-27 02:38:03,547][105620] Updated weights for policy 1, policy_version 1546153 (0.0009) [2023-12-27 02:38:03,592][105692] Updated weights for policy 0, policy_version 1543043 (0.0006) [2023-12-27 02:38:03,607][105620] Updated weights for policy 1, policy_version 1546163 (0.0010) [2023-12-27 02:38:03,658][105620] Updated weights for policy 1, policy_version 1546173 (0.0010) [2023-12-27 02:38:03,707][105620] Updated weights for policy 1, policy_version 1546183 (0.0006) [2023-12-27 02:38:04,308][105620] Updated weights for policy 1, policy_version 1546193 (0.0007) [2023-12-27 02:38:04,367][105620] Updated weights for policy 1, policy_version 1546203 (0.0010) [2023-12-27 02:38:04,413][105692] Updated weights for policy 0, policy_version 1543053 (0.0007) [2023-12-27 02:38:04,427][105620] Updated weights for policy 1, policy_version 1546213 (0.0011) [2023-12-27 02:38:04,470][105692] Updated weights for policy 0, policy_version 1543063 (0.0007) [2023-12-27 02:38:04,537][105692] Updated weights for policy 0, policy_version 1543073 (0.0008) [2023-12-27 02:38:05,174][105620] Updated weights for policy 1, policy_version 1546223 (0.0007) [2023-12-27 02:38:05,232][105620] Updated weights for policy 1, policy_version 1546233 (0.0005) [2023-12-27 02:38:05,279][105692] Updated weights for policy 0, policy_version 1543083 (0.0007) [2023-12-27 02:38:05,287][105620] Updated weights for policy 1, policy_version 1546243 (0.0005) [2023-12-27 02:38:05,337][105692] Updated weights for policy 0, policy_version 1543094 (0.0007) [2023-12-27 02:38:05,405][105692] Updated weights for policy 0, policy_version 1543104 (0.0006) [2023-12-27 02:38:05,872][105620] Updated weights for policy 1, policy_version 1546253 (0.0007) [2023-12-27 02:38:05,916][105620] Updated weights for policy 1, policy_version 1546263 (0.0005) [2023-12-27 02:38:05,976][105620] Updated weights for policy 1, policy_version 1546273 (0.0006) [2023-12-27 02:38:06,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 790994944. Throughput: 0: 9374.8, 1: 9839.8. Samples: 790981908. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:38:06,063][104569] Avg episode reward: [(0, '8721.125'), (1, '8900.074')] [2023-12-27 02:38:06,096][105692] Updated weights for policy 0, policy_version 1543114 (0.0006) [2023-12-27 02:38:06,160][105692] Updated weights for policy 0, policy_version 1543124 (0.0007) [2023-12-27 02:38:06,222][105692] Updated weights for policy 0, policy_version 1543134 (0.0009) [2023-12-27 02:38:06,277][105692] Updated weights for policy 0, policy_version 1543144 (0.0010) [2023-12-27 02:38:06,518][105620] Updated weights for policy 1, policy_version 1546283 (0.0005) [2023-12-27 02:38:06,577][105620] Updated weights for policy 1, policy_version 1546293 (0.0007) [2023-12-27 02:38:06,636][105620] Updated weights for policy 1, policy_version 1546303 (0.0010) [2023-12-27 02:38:07,156][105692] Updated weights for policy 0, policy_version 1543154 (0.0008) [2023-12-27 02:38:07,209][105692] Updated weights for policy 0, policy_version 1543164 (0.0008) [2023-12-27 02:38:07,214][105620] Updated weights for policy 1, policy_version 1546313 (0.0006) [2023-12-27 02:38:07,264][105692] Updated weights for policy 0, policy_version 1543174 (0.0005) [2023-12-27 02:38:07,266][105620] Updated weights for policy 1, policy_version 1546323 (0.0010) [2023-12-27 02:38:07,317][105620] Updated weights for policy 1, policy_version 1546333 (0.0010) [2023-12-27 02:38:07,372][105620] Updated weights for policy 1, policy_version 1546343 (0.0010) [2023-12-27 02:38:08,033][105692] Updated weights for policy 0, policy_version 1543184 (0.0007) [2023-12-27 02:38:08,102][105692] Updated weights for policy 0, policy_version 1543194 (0.0007) [2023-12-27 02:38:08,119][105620] Updated weights for policy 1, policy_version 1546353 (0.0007) [2023-12-27 02:38:08,167][105692] Updated weights for policy 0, policy_version 1543204 (0.0005) [2023-12-27 02:38:08,173][105620] Updated weights for policy 1, policy_version 1546363 (0.0007) [2023-12-27 02:38:08,231][105620] Updated weights for policy 1, policy_version 1546373 (0.0006) [2023-12-27 02:38:08,838][105692] Updated weights for policy 0, policy_version 1543214 (0.0005) [2023-12-27 02:38:08,895][105692] Updated weights for policy 0, policy_version 1543224 (0.0005) [2023-12-27 02:38:08,924][105620] Updated weights for policy 1, policy_version 1546383 (0.0005) [2023-12-27 02:38:08,951][105692] Updated weights for policy 0, policy_version 1543234 (0.0006) [2023-12-27 02:38:08,981][105620] Updated weights for policy 1, policy_version 1546393 (0.0010) [2023-12-27 02:38:09,027][105620] Updated weights for policy 1, policy_version 1546403 (0.0010) [2023-12-27 02:38:09,660][105692] Updated weights for policy 0, policy_version 1543244 (0.0007) [2023-12-27 02:38:09,716][105692] Updated weights for policy 0, policy_version 1543254 (0.0008) [2023-12-27 02:38:09,778][105692] Updated weights for policy 0, policy_version 1543264 (0.0008) [2023-12-27 02:38:09,780][105620] Updated weights for policy 1, policy_version 1546413 (0.0010) [2023-12-27 02:38:09,846][105620] Updated weights for policy 1, policy_version 1546423 (0.0011) [2023-12-27 02:38:09,910][105620] Updated weights for policy 1, policy_version 1546433 (0.0008) [2023-12-27 02:38:10,541][105620] Updated weights for policy 1, policy_version 1546443 (0.0007) [2023-12-27 02:38:10,573][105692] Updated weights for policy 0, policy_version 1543274 (0.0007) [2023-12-27 02:38:10,605][105620] Updated weights for policy 1, policy_version 1546453 (0.0007) [2023-12-27 02:38:10,632][105692] Updated weights for policy 0, policy_version 1543284 (0.0007) [2023-12-27 02:38:10,659][105620] Updated weights for policy 1, policy_version 1546463 (0.0006) [2023-12-27 02:38:10,691][105692] Updated weights for policy 0, policy_version 1543294 (0.0008) [2023-12-27 02:38:10,742][105692] Updated weights for policy 0, policy_version 1543304 (0.0009) [2023-12-27 02:38:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 791093248. Throughput: 0: 9409.5, 1: 9942.3. Samples: 791100856. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:38:11,063][104569] Avg episode reward: [(0, '8263.512'), (1, '8989.334')] [2023-12-27 02:38:11,333][105620] Updated weights for policy 1, policy_version 1546473 (0.0006) [2023-12-27 02:38:11,401][105620] Updated weights for policy 1, policy_version 1546483 (0.0010) [2023-12-27 02:38:11,465][105620] Updated weights for policy 1, policy_version 1546493 (0.0010) [2023-12-27 02:38:11,533][105620] Updated weights for policy 1, policy_version 1546503 (0.0010) [2023-12-27 02:38:11,589][105692] Updated weights for policy 0, policy_version 1543314 (0.0008) [2023-12-27 02:38:11,653][105692] Updated weights for policy 0, policy_version 1543324 (0.0007) [2023-12-27 02:38:11,724][105692] Updated weights for policy 0, policy_version 1543334 (0.0008) [2023-12-27 02:38:12,330][105620] Updated weights for policy 1, policy_version 1546513 (0.0010) [2023-12-27 02:38:12,400][105620] Updated weights for policy 1, policy_version 1546523 (0.0008) [2023-12-27 02:38:12,456][105620] Updated weights for policy 1, policy_version 1546533 (0.0008) [2023-12-27 02:38:12,560][105692] Updated weights for policy 0, policy_version 1543344 (0.0009) [2023-12-27 02:38:12,613][105692] Updated weights for policy 0, policy_version 1543354 (0.0009) [2023-12-27 02:38:12,668][105692] Updated weights for policy 0, policy_version 1543364 (0.0009) [2023-12-27 02:38:13,208][105620] Updated weights for policy 1, policy_version 1546543 (0.0009) [2023-12-27 02:38:13,254][105620] Updated weights for policy 1, policy_version 1546553 (0.0007) [2023-12-27 02:38:13,304][105620] Updated weights for policy 1, policy_version 1546563 (0.0005) [2023-12-27 02:38:13,449][105692] Updated weights for policy 0, policy_version 1543374 (0.0008) [2023-12-27 02:38:13,510][105692] Updated weights for policy 0, policy_version 1543384 (0.0009) [2023-12-27 02:38:13,569][105692] Updated weights for policy 0, policy_version 1543394 (0.0008) [2023-12-27 02:38:13,971][105620] Updated weights for policy 1, policy_version 1546573 (0.0008) [2023-12-27 02:38:14,023][105620] Updated weights for policy 1, policy_version 1546583 (0.0010) [2023-12-27 02:38:14,072][105620] Updated weights for policy 1, policy_version 1546593 (0.0010) [2023-12-27 02:38:14,293][105692] Updated weights for policy 0, policy_version 1543404 (0.0009) [2023-12-27 02:38:14,354][105692] Updated weights for policy 0, policy_version 1543414 (0.0009) [2023-12-27 02:38:14,411][105692] Updated weights for policy 0, policy_version 1543424 (0.0008) [2023-12-27 02:38:14,774][105620] Updated weights for policy 1, policy_version 1546603 (0.0010) [2023-12-27 02:38:14,834][105620] Updated weights for policy 1, policy_version 1546613 (0.0011) [2023-12-27 02:38:14,894][105620] Updated weights for policy 1, policy_version 1546623 (0.0011) [2023-12-27 02:38:15,172][105692] Updated weights for policy 0, policy_version 1543434 (0.0008) [2023-12-27 02:38:15,226][105692] Updated weights for policy 0, policy_version 1543444 (0.0010) [2023-12-27 02:38:15,280][105692] Updated weights for policy 0, policy_version 1543454 (0.0010) [2023-12-27 02:38:15,328][105692] Updated weights for policy 0, policy_version 1543464 (0.0008) [2023-12-27 02:38:15,513][105620] Updated weights for policy 1, policy_version 1546633 (0.0011) [2023-12-27 02:38:15,574][105620] Updated weights for policy 1, policy_version 1546643 (0.0011) [2023-12-27 02:38:15,628][105620] Updated weights for policy 1, policy_version 1546653 (0.0011) [2023-12-27 02:38:15,691][105620] Updated weights for policy 1, policy_version 1546663 (0.0011) [2023-12-27 02:38:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.1, 300 sec: 19521.9). Total num frames: 791183360. Throughput: 0: 9411.3, 1: 9873.0. Samples: 791155788. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:38:16,063][104569] Avg episode reward: [(0, '7990.222'), (1, '8989.506')] [2023-12-27 02:38:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001543464_395182080.pth... [2023-12-27 02:38:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001546664_396001280.pth... [2023-12-27 02:38:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001542376_394903552.pth [2023-12-27 02:38:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001545480_395698176.pth [2023-12-27 02:38:16,204][105692] Updated weights for policy 0, policy_version 1543474 (0.0009) [2023-12-27 02:38:16,269][105692] Updated weights for policy 0, policy_version 1543484 (0.0010) [2023-12-27 02:38:16,310][105620] Updated weights for policy 1, policy_version 1546673 (0.0006) [2023-12-27 02:38:16,330][105692] Updated weights for policy 0, policy_version 1543494 (0.0009) [2023-12-27 02:38:16,367][105620] Updated weights for policy 1, policy_version 1546683 (0.0005) [2023-12-27 02:38:16,423][105620] Updated weights for policy 1, policy_version 1546693 (0.0005) [2023-12-27 02:38:17,092][105620] Updated weights for policy 1, policy_version 1546703 (0.0009) [2023-12-27 02:38:17,133][105692] Updated weights for policy 0, policy_version 1543504 (0.0007) [2023-12-27 02:38:17,147][105620] Updated weights for policy 1, policy_version 1546713 (0.0010) [2023-12-27 02:38:17,185][105692] Updated weights for policy 0, policy_version 1543514 (0.0006) [2023-12-27 02:38:17,205][105620] Updated weights for policy 1, policy_version 1546723 (0.0006) [2023-12-27 02:38:17,248][105692] Updated weights for policy 0, policy_version 1543524 (0.0009) [2023-12-27 02:38:17,882][105692] Updated weights for policy 0, policy_version 1543534 (0.0010) [2023-12-27 02:38:17,919][105620] Updated weights for policy 1, policy_version 1546733 (0.0008) [2023-12-27 02:38:17,938][105692] Updated weights for policy 0, policy_version 1543544 (0.0011) [2023-12-27 02:38:17,975][105620] Updated weights for policy 1, policy_version 1546743 (0.0010) [2023-12-27 02:38:17,994][105692] Updated weights for policy 0, policy_version 1543554 (0.0011) [2023-12-27 02:38:18,034][105620] Updated weights for policy 1, policy_version 1546753 (0.0010) [2023-12-27 02:38:18,663][105620] Updated weights for policy 1, policy_version 1546763 (0.0009) [2023-12-27 02:38:18,685][105692] Updated weights for policy 0, policy_version 1543564 (0.0011) [2023-12-27 02:38:18,716][105620] Updated weights for policy 1, policy_version 1546773 (0.0005) [2023-12-27 02:38:18,738][105692] Updated weights for policy 0, policy_version 1543574 (0.0010) [2023-12-27 02:38:18,769][105620] Updated weights for policy 1, policy_version 1546783 (0.0008) [2023-12-27 02:38:18,796][105692] Updated weights for policy 0, policy_version 1543584 (0.0008) [2023-12-27 02:38:19,539][105620] Updated weights for policy 1, policy_version 1546793 (0.0008) [2023-12-27 02:38:19,559][105692] Updated weights for policy 0, policy_version 1543594 (0.0010) [2023-12-27 02:38:19,604][105620] Updated weights for policy 1, policy_version 1546803 (0.0008) [2023-12-27 02:38:19,618][105692] Updated weights for policy 0, policy_version 1543604 (0.0007) [2023-12-27 02:38:19,665][105620] Updated weights for policy 1, policy_version 1546813 (0.0009) [2023-12-27 02:38:19,675][105692] Updated weights for policy 0, policy_version 1543614 (0.0006) [2023-12-27 02:38:19,714][105620] Updated weights for policy 1, policy_version 1546823 (0.0006) [2023-12-27 02:38:19,721][105585] KL-divergence is very high: 133.9805 [2023-12-27 02:38:19,739][105692] Updated weights for policy 0, policy_version 1543624 (0.0006) [2023-12-27 02:38:20,487][105692] Updated weights for policy 0, policy_version 1543634 (0.0009) [2023-12-27 02:38:20,499][105620] Updated weights for policy 1, policy_version 1546833 (0.0010) [2023-12-27 02:38:20,547][105620] Updated weights for policy 1, policy_version 1546843 (0.0010) [2023-12-27 02:38:20,550][105692] Updated weights for policy 0, policy_version 1543644 (0.0007) [2023-12-27 02:38:20,612][105620] Updated weights for policy 1, policy_version 1546853 (0.0008) [2023-12-27 02:38:20,615][105692] Updated weights for policy 0, policy_version 1543654 (0.0007) [2023-12-27 02:38:21,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 791281664. Throughput: 0: 9374.9, 1: 9944.5. Samples: 791272304. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:38:21,062][104569] Avg episode reward: [(0, '8084.659'), (1, '9081.560')] [2023-12-27 02:38:21,387][105692] Updated weights for policy 0, policy_version 1543664 (0.0008) [2023-12-27 02:38:21,423][105620] Updated weights for policy 1, policy_version 1546863 (0.0006) [2023-12-27 02:38:21,446][105692] Updated weights for policy 0, policy_version 1543674 (0.0008) [2023-12-27 02:38:21,470][105620] Updated weights for policy 1, policy_version 1546873 (0.0006) [2023-12-27 02:38:21,509][105692] Updated weights for policy 0, policy_version 1543684 (0.0008) [2023-12-27 02:38:21,523][105620] Updated weights for policy 1, policy_version 1546883 (0.0006) [2023-12-27 02:38:22,289][105692] Updated weights for policy 0, policy_version 1543694 (0.0009) [2023-12-27 02:38:22,294][105620] Updated weights for policy 1, policy_version 1546893 (0.0009) [2023-12-27 02:38:22,357][105620] Updated weights for policy 1, policy_version 1546903 (0.0007) [2023-12-27 02:38:22,358][105692] Updated weights for policy 0, policy_version 1543704 (0.0009) [2023-12-27 02:38:22,422][105692] Updated weights for policy 0, policy_version 1543714 (0.0011) [2023-12-27 02:38:22,424][105620] Updated weights for policy 1, policy_version 1546913 (0.0009) [2023-12-27 02:38:23,141][105620] Updated weights for policy 1, policy_version 1546923 (0.0007) [2023-12-27 02:38:23,187][105692] Updated weights for policy 0, policy_version 1543724 (0.0009) [2023-12-27 02:38:23,199][105620] Updated weights for policy 1, policy_version 1546933 (0.0009) [2023-12-27 02:38:23,254][105692] Updated weights for policy 0, policy_version 1543734 (0.0005) [2023-12-27 02:38:23,259][105620] Updated weights for policy 1, policy_version 1546943 (0.0008) [2023-12-27 02:38:23,312][105692] Updated weights for policy 0, policy_version 1543744 (0.0005) [2023-12-27 02:38:23,817][105692] Updated weights for policy 0, policy_version 1543754 (0.0005) [2023-12-27 02:38:23,868][105620] Updated weights for policy 1, policy_version 1546953 (0.0008) [2023-12-27 02:38:23,868][105692] Updated weights for policy 0, policy_version 1543764 (0.0005) [2023-12-27 02:38:23,921][105620] Updated weights for policy 1, policy_version 1546963 (0.0005) [2023-12-27 02:38:23,923][105692] Updated weights for policy 0, policy_version 1543774 (0.0006) [2023-12-27 02:38:23,973][105692] Updated weights for policy 0, policy_version 1543784 (0.0006) [2023-12-27 02:38:23,976][105620] Updated weights for policy 1, policy_version 1546973 (0.0006) [2023-12-27 02:38:24,034][105620] Updated weights for policy 1, policy_version 1546983 (0.0010) [2023-12-27 02:38:24,526][105692] Updated weights for policy 0, policy_version 1543794 (0.0005) [2023-12-27 02:38:24,592][105692] Updated weights for policy 0, policy_version 1543804 (0.0006) [2023-12-27 02:38:24,648][105692] Updated weights for policy 0, policy_version 1543814 (0.0008) [2023-12-27 02:38:24,776][105620] Updated weights for policy 1, policy_version 1546993 (0.0010) [2023-12-27 02:38:24,835][105620] Updated weights for policy 1, policy_version 1547003 (0.0010) [2023-12-27 02:38:24,901][105620] Updated weights for policy 1, policy_version 1547013 (0.0009) [2023-12-27 02:38:25,260][105692] Updated weights for policy 0, policy_version 1543824 (0.0009) [2023-12-27 02:38:25,317][105692] Updated weights for policy 0, policy_version 1543834 (0.0010) [2023-12-27 02:38:25,371][105692] Updated weights for policy 0, policy_version 1543844 (0.0010) [2023-12-27 02:38:25,650][105620] Updated weights for policy 1, policy_version 1547023 (0.0006) [2023-12-27 02:38:25,713][105620] Updated weights for policy 1, policy_version 1547033 (0.0005) [2023-12-27 02:38:25,780][105620] Updated weights for policy 1, policy_version 1547043 (0.0005) [2023-12-27 02:38:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 791379968. Throughput: 0: 9465.7, 1: 9943.2. Samples: 791391232. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:38:26,062][104569] Avg episode reward: [(0, '8449.000'), (1, '9172.739')] [2023-12-27 02:38:26,080][105692] Updated weights for policy 0, policy_version 1543854 (0.0007) [2023-12-27 02:38:26,133][105692] Updated weights for policy 0, policy_version 1543864 (0.0011) [2023-12-27 02:38:26,184][105692] Updated weights for policy 0, policy_version 1543874 (0.0009) [2023-12-27 02:38:26,348][105620] Updated weights for policy 1, policy_version 1547053 (0.0007) [2023-12-27 02:38:26,416][105620] Updated weights for policy 1, policy_version 1547063 (0.0008) [2023-12-27 02:38:26,484][105620] Updated weights for policy 1, policy_version 1547073 (0.0009) [2023-12-27 02:38:26,764][105692] Updated weights for policy 0, policy_version 1543884 (0.0006) [2023-12-27 02:38:26,818][105692] Updated weights for policy 0, policy_version 1543894 (0.0008) [2023-12-27 02:38:26,869][105692] Updated weights for policy 0, policy_version 1543904 (0.0008) [2023-12-27 02:38:27,245][105620] Updated weights for policy 1, policy_version 1547083 (0.0009) [2023-12-27 02:38:27,292][105620] Updated weights for policy 1, policy_version 1547093 (0.0010) [2023-12-27 02:38:27,350][105620] Updated weights for policy 1, policy_version 1547103 (0.0010) [2023-12-27 02:38:27,620][105692] Updated weights for policy 0, policy_version 1543914 (0.0008) [2023-12-27 02:38:27,683][105692] Updated weights for policy 0, policy_version 1543924 (0.0006) [2023-12-27 02:38:27,743][105692] Updated weights for policy 0, policy_version 1543934 (0.0006) [2023-12-27 02:38:27,803][105692] Updated weights for policy 0, policy_version 1543944 (0.0009) [2023-12-27 02:38:28,079][105620] Updated weights for policy 1, policy_version 1547113 (0.0010) [2023-12-27 02:38:28,143][105620] Updated weights for policy 1, policy_version 1547123 (0.0010) [2023-12-27 02:38:28,209][105620] Updated weights for policy 1, policy_version 1547133 (0.0011) [2023-12-27 02:38:28,271][105620] Updated weights for policy 1, policy_version 1547143 (0.0010) [2023-12-27 02:38:28,485][105692] Updated weights for policy 0, policy_version 1543954 (0.0009) [2023-12-27 02:38:28,542][105692] Updated weights for policy 0, policy_version 1543964 (0.0010) [2023-12-27 02:38:28,596][105692] Updated weights for policy 0, policy_version 1543975 (0.0009) [2023-12-27 02:38:28,928][105620] Updated weights for policy 1, policy_version 1547153 (0.0009) [2023-12-27 02:38:28,982][105620] Updated weights for policy 1, policy_version 1547163 (0.0009) [2023-12-27 02:38:29,029][105620] Updated weights for policy 1, policy_version 1547173 (0.0008) [2023-12-27 02:38:29,418][105692] Updated weights for policy 0, policy_version 1543985 (0.0010) [2023-12-27 02:38:29,479][105692] Updated weights for policy 0, policy_version 1543995 (0.0009) [2023-12-27 02:38:29,536][105692] Updated weights for policy 0, policy_version 1544005 (0.0009) [2023-12-27 02:38:29,809][105620] Updated weights for policy 1, policy_version 1547183 (0.0009) [2023-12-27 02:38:29,872][105620] Updated weights for policy 1, policy_version 1547193 (0.0009) [2023-12-27 02:38:29,929][105620] Updated weights for policy 1, policy_version 1547203 (0.0008) [2023-12-27 02:38:30,273][105692] Updated weights for policy 0, policy_version 1544015 (0.0008) [2023-12-27 02:38:30,337][105692] Updated weights for policy 0, policy_version 1544025 (0.0009) [2023-12-27 02:38:30,393][105692] Updated weights for policy 0, policy_version 1544035 (0.0008) [2023-12-27 02:38:30,592][105620] Updated weights for policy 1, policy_version 1547213 (0.0006) [2023-12-27 02:38:30,638][105620] Updated weights for policy 1, policy_version 1547223 (0.0005) [2023-12-27 02:38:30,690][105620] Updated weights for policy 1, policy_version 1547233 (0.0005) [2023-12-27 02:38:31,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19251.1, 300 sec: 19494.2). Total num frames: 791478272. Throughput: 0: 9512.5, 1: 9996.4. Samples: 791450964. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:38:31,063][104569] Avg episode reward: [(0, '8715.765'), (1, '8905.885')] [2023-12-27 02:38:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001544040_395329536.pth... [2023-12-27 02:38:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001547240_396148736.pth... [2023-12-27 02:38:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001542920_395042816.pth [2023-12-27 02:38:31,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001546056_395845632.pth [2023-12-27 02:38:31,241][105692] Updated weights for policy 0, policy_version 1544045 (0.0009) [2023-12-27 02:38:31,277][105620] Updated weights for policy 1, policy_version 1547243 (0.0006) [2023-12-27 02:38:31,306][105692] Updated weights for policy 0, policy_version 1544055 (0.0008) [2023-12-27 02:38:31,339][105620] Updated weights for policy 1, policy_version 1547253 (0.0008) [2023-12-27 02:38:31,371][105692] Updated weights for policy 0, policy_version 1544065 (0.0007) [2023-12-27 02:38:31,405][105620] Updated weights for policy 1, policy_version 1547263 (0.0008) [2023-12-27 02:38:32,088][105692] Updated weights for policy 0, policy_version 1544075 (0.0008) [2023-12-27 02:38:32,142][105692] Updated weights for policy 0, policy_version 1544085 (0.0009) [2023-12-27 02:38:32,146][105620] Updated weights for policy 1, policy_version 1547273 (0.0008) [2023-12-27 02:38:32,199][105692] Updated weights for policy 0, policy_version 1544095 (0.0008) [2023-12-27 02:38:32,208][105620] Updated weights for policy 1, policy_version 1547283 (0.0006) [2023-12-27 02:38:32,267][105620] Updated weights for policy 1, policy_version 1547293 (0.0008) [2023-12-27 02:38:32,323][105620] Updated weights for policy 1, policy_version 1547303 (0.0009) [2023-12-27 02:38:32,914][105692] Updated weights for policy 0, policy_version 1544105 (0.0007) [2023-12-27 02:38:32,969][105692] Updated weights for policy 0, policy_version 1544115 (0.0009) [2023-12-27 02:38:33,024][105692] Updated weights for policy 0, policy_version 1544125 (0.0008) [2023-12-27 02:38:33,067][105692] Updated weights for policy 0, policy_version 1544135 (0.0008) [2023-12-27 02:38:33,069][105620] Updated weights for policy 1, policy_version 1547313 (0.0008) [2023-12-27 02:38:33,118][105620] Updated weights for policy 1, policy_version 1547323 (0.0008) [2023-12-27 02:38:33,174][105620] Updated weights for policy 1, policy_version 1547333 (0.0009) [2023-12-27 02:38:33,743][105692] Updated weights for policy 0, policy_version 1544145 (0.0005) [2023-12-27 02:38:33,798][105692] Updated weights for policy 0, policy_version 1544155 (0.0005) [2023-12-27 02:38:33,843][105692] Updated weights for policy 0, policy_version 1544165 (0.0005) [2023-12-27 02:38:33,892][105620] Updated weights for policy 1, policy_version 1547343 (0.0008) [2023-12-27 02:38:33,937][105620] Updated weights for policy 1, policy_version 1547353 (0.0007) [2023-12-27 02:38:33,982][105620] Updated weights for policy 1, policy_version 1547363 (0.0005) [2023-12-27 02:38:34,586][105692] Updated weights for policy 0, policy_version 1544175 (0.0007) [2023-12-27 02:38:34,638][105692] Updated weights for policy 0, policy_version 1544185 (0.0010) [2023-12-27 02:38:34,680][105620] Updated weights for policy 1, policy_version 1547373 (0.0007) [2023-12-27 02:38:34,689][105692] Updated weights for policy 0, policy_version 1544195 (0.0010) [2023-12-27 02:38:34,739][105620] Updated weights for policy 1, policy_version 1547383 (0.0008) [2023-12-27 02:38:34,799][105620] Updated weights for policy 1, policy_version 1547393 (0.0008) [2023-12-27 02:38:35,433][105620] Updated weights for policy 1, policy_version 1547403 (0.0005) [2023-12-27 02:38:35,486][105620] Updated weights for policy 1, policy_version 1547413 (0.0005) [2023-12-27 02:38:35,492][105692] Updated weights for policy 0, policy_version 1544205 (0.0007) [2023-12-27 02:38:35,542][105620] Updated weights for policy 1, policy_version 1547423 (0.0005) [2023-12-27 02:38:35,542][105692] Updated weights for policy 0, policy_version 1544215 (0.0009) [2023-12-27 02:38:35,592][105692] Updated weights for policy 0, policy_version 1544225 (0.0010) [2023-12-27 02:38:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 791576576. Throughput: 0: 9610.9, 1: 9916.0. Samples: 791567076. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:38:36,062][104569] Avg episode reward: [(0, '8528.179'), (1, '8540.392')] [2023-12-27 02:38:36,072][105620] Updated weights for policy 1, policy_version 1547433 (0.0006) [2023-12-27 02:38:36,149][105620] Updated weights for policy 1, policy_version 1547443 (0.0006) [2023-12-27 02:38:36,208][105620] Updated weights for policy 1, policy_version 1547453 (0.0008) [2023-12-27 02:38:36,258][105620] Updated weights for policy 1, policy_version 1547463 (0.0008) [2023-12-27 02:38:36,266][105692] Updated weights for policy 0, policy_version 1544235 (0.0009) [2023-12-27 02:38:36,328][105692] Updated weights for policy 0, policy_version 1544245 (0.0009) [2023-12-27 02:38:36,393][105692] Updated weights for policy 0, policy_version 1544255 (0.0009) [2023-12-27 02:38:36,986][105620] Updated weights for policy 1, policy_version 1547473 (0.0008) [2023-12-27 02:38:37,037][105620] Updated weights for policy 1, policy_version 1547483 (0.0008) [2023-12-27 02:38:37,090][105620] Updated weights for policy 1, policy_version 1547493 (0.0008) [2023-12-27 02:38:37,181][105692] Updated weights for policy 0, policy_version 1544265 (0.0008) [2023-12-27 02:38:37,243][105692] Updated weights for policy 0, policy_version 1544275 (0.0011) [2023-12-27 02:38:37,305][105692] Updated weights for policy 0, policy_version 1544285 (0.0009) [2023-12-27 02:38:37,363][105692] Updated weights for policy 0, policy_version 1544295 (0.0010) [2023-12-27 02:38:37,859][105620] Updated weights for policy 1, policy_version 1547503 (0.0006) [2023-12-27 02:38:37,915][105620] Updated weights for policy 1, policy_version 1547513 (0.0005) [2023-12-27 02:38:37,973][105620] Updated weights for policy 1, policy_version 1547523 (0.0006) [2023-12-27 02:38:38,099][105692] Updated weights for policy 0, policy_version 1544305 (0.0011) [2023-12-27 02:38:38,160][105692] Updated weights for policy 0, policy_version 1544315 (0.0010) [2023-12-27 02:38:38,215][105692] Updated weights for policy 0, policy_version 1544325 (0.0010) [2023-12-27 02:38:38,657][105620] Updated weights for policy 1, policy_version 1547533 (0.0010) [2023-12-27 02:38:38,722][105620] Updated weights for policy 1, policy_version 1547543 (0.0010) [2023-12-27 02:38:38,781][105620] Updated weights for policy 1, policy_version 1547553 (0.0011) [2023-12-27 02:38:38,958][105692] Updated weights for policy 0, policy_version 1544335 (0.0011) [2023-12-27 02:38:39,020][105692] Updated weights for policy 0, policy_version 1544345 (0.0011) [2023-12-27 02:38:39,084][105692] Updated weights for policy 0, policy_version 1544355 (0.0010) [2023-12-27 02:38:39,524][105620] Updated weights for policy 1, policy_version 1547563 (0.0011) [2023-12-27 02:38:39,594][105620] Updated weights for policy 1, policy_version 1547573 (0.0010) [2023-12-27 02:38:39,655][105620] Updated weights for policy 1, policy_version 1547583 (0.0008) [2023-12-27 02:38:39,699][105586] KL-divergence is very high: 106.3302 [2023-12-27 02:38:39,743][105692] Updated weights for policy 0, policy_version 1544365 (0.0010) [2023-12-27 02:38:39,804][105692] Updated weights for policy 0, policy_version 1544375 (0.0009) [2023-12-27 02:38:39,874][105692] Updated weights for policy 0, policy_version 1544385 (0.0008) [2023-12-27 02:38:40,340][105620] Updated weights for policy 1, policy_version 1547593 (0.0009) [2023-12-27 02:38:40,401][105620] Updated weights for policy 1, policy_version 1547603 (0.0006) [2023-12-27 02:38:40,464][105620] Updated weights for policy 1, policy_version 1547613 (0.0007) [2023-12-27 02:38:40,527][105620] Updated weights for policy 1, policy_version 1547623 (0.0008) [2023-12-27 02:38:40,754][105692] Updated weights for policy 0, policy_version 1544395 (0.0009) [2023-12-27 02:38:40,814][105692] Updated weights for policy 0, policy_version 1544406 (0.0011) [2023-12-27 02:38:40,867][105692] Updated weights for policy 0, policy_version 1544418 (0.0010) [2023-12-27 02:38:41,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 791674880. Throughput: 0: 9546.3, 1: 9973.4. Samples: 791683860. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:38:41,062][104569] Avg episode reward: [(0, '8164.071'), (1, '8900.038')] [2023-12-27 02:38:41,142][105620] Updated weights for policy 1, policy_version 1547633 (0.0010) [2023-12-27 02:38:41,202][105620] Updated weights for policy 1, policy_version 1547643 (0.0010) [2023-12-27 02:38:41,260][105620] Updated weights for policy 1, policy_version 1547653 (0.0006) [2023-12-27 02:38:41,810][105692] Updated weights for policy 0, policy_version 1544428 (0.0009) [2023-12-27 02:38:41,874][105692] Updated weights for policy 0, policy_version 1544438 (0.0008) [2023-12-27 02:38:41,931][105620] Updated weights for policy 1, policy_version 1547663 (0.0007) [2023-12-27 02:38:41,934][105692] Updated weights for policy 0, policy_version 1544448 (0.0006) [2023-12-27 02:38:41,992][105620] Updated weights for policy 1, policy_version 1547673 (0.0008) [2023-12-27 02:38:42,058][105620] Updated weights for policy 1, policy_version 1547683 (0.0006) [2023-12-27 02:38:42,682][105692] Updated weights for policy 0, policy_version 1544458 (0.0008) [2023-12-27 02:38:42,747][105692] Updated weights for policy 0, policy_version 1544469 (0.0009) [2023-12-27 02:38:42,803][105620] Updated weights for policy 1, policy_version 1547693 (0.0007) [2023-12-27 02:38:42,809][105692] Updated weights for policy 0, policy_version 1544479 (0.0008) [2023-12-27 02:38:42,855][105620] Updated weights for policy 1, policy_version 1547703 (0.0007) [2023-12-27 02:38:42,913][105620] Updated weights for policy 1, policy_version 1547713 (0.0008) [2023-12-27 02:38:43,520][105692] Updated weights for policy 0, policy_version 1544489 (0.0008) [2023-12-27 02:38:43,576][105692] Updated weights for policy 0, policy_version 1544499 (0.0009) [2023-12-27 02:38:43,622][105692] Updated weights for policy 0, policy_version 1544509 (0.0008) [2023-12-27 02:38:43,670][105692] Updated weights for policy 0, policy_version 1544519 (0.0009) [2023-12-27 02:38:43,704][105620] Updated weights for policy 1, policy_version 1547723 (0.0009) [2023-12-27 02:38:43,751][105620] Updated weights for policy 1, policy_version 1547733 (0.0008) [2023-12-27 02:38:43,814][105620] Updated weights for policy 1, policy_version 1547743 (0.0010) [2023-12-27 02:38:44,400][105692] Updated weights for policy 0, policy_version 1544529 (0.0007) [2023-12-27 02:38:44,463][105692] Updated weights for policy 0, policy_version 1544539 (0.0010) [2023-12-27 02:38:44,525][105692] Updated weights for policy 0, policy_version 1544549 (0.0009) [2023-12-27 02:38:44,604][105620] Updated weights for policy 1, policy_version 1547753 (0.0010) [2023-12-27 02:38:44,662][105620] Updated weights for policy 1, policy_version 1547763 (0.0009) [2023-12-27 02:38:44,721][105620] Updated weights for policy 1, policy_version 1547773 (0.0006) [2023-12-27 02:38:44,796][105620] Updated weights for policy 1, policy_version 1547783 (0.0007) [2023-12-27 02:38:45,205][105692] Updated weights for policy 0, policy_version 1544559 (0.0007) [2023-12-27 02:38:45,260][105692] Updated weights for policy 0, policy_version 1544569 (0.0009) [2023-12-27 02:38:45,313][105692] Updated weights for policy 0, policy_version 1544579 (0.0009) [2023-12-27 02:38:45,582][105620] Updated weights for policy 1, policy_version 1547793 (0.0008) [2023-12-27 02:38:45,653][105620] Updated weights for policy 1, policy_version 1547803 (0.0007) [2023-12-27 02:38:45,704][105620] Updated weights for policy 1, policy_version 1547813 (0.0008) [2023-12-27 02:38:46,046][105692] Updated weights for policy 0, policy_version 1544589 (0.0009) [2023-12-27 02:38:46,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 791764992. Throughput: 0: 9532.9, 1: 9911.2. Samples: 791738608. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-12-27 02:38:46,063][104569] Avg episode reward: [(0, '8167.123'), (1, '9174.764')] [2023-12-27 02:38:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001547816_396296192.pth... [2023-12-27 02:38:46,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001546664_396001280.pth [2023-12-27 02:38:46,097][105692] Updated weights for policy 0, policy_version 1544599 (0.0009) [2023-12-27 02:38:46,149][105692] Updated weights for policy 0, policy_version 1544609 (0.0005) [2023-12-27 02:38:46,194][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001544616_395476992.pth... [2023-12-27 02:38:46,199][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001543464_395182080.pth [2023-12-27 02:38:46,481][105620] Updated weights for policy 1, policy_version 1547823 (0.0009) [2023-12-27 02:38:46,535][105620] Updated weights for policy 1, policy_version 1547833 (0.0009) [2023-12-27 02:38:46,592][105620] Updated weights for policy 1, policy_version 1547843 (0.0008) [2023-12-27 02:38:46,825][105692] Updated weights for policy 0, policy_version 1544619 (0.0007) [2023-12-27 02:38:46,892][105692] Updated weights for policy 0, policy_version 1544629 (0.0008) [2023-12-27 02:38:46,953][105692] Updated weights for policy 0, policy_version 1544639 (0.0007) [2023-12-27 02:38:47,270][105620] Updated weights for policy 1, policy_version 1547853 (0.0007) [2023-12-27 02:38:47,336][105620] Updated weights for policy 1, policy_version 1547863 (0.0007) [2023-12-27 02:38:47,387][105620] Updated weights for policy 1, policy_version 1547873 (0.0010) [2023-12-27 02:38:47,538][105692] Updated weights for policy 0, policy_version 1544649 (0.0006) [2023-12-27 02:38:47,593][105692] Updated weights for policy 0, policy_version 1544659 (0.0010) [2023-12-27 02:38:47,648][105692] Updated weights for policy 0, policy_version 1544669 (0.0010) [2023-12-27 02:38:47,696][105692] Updated weights for policy 0, policy_version 1544679 (0.0010) [2023-12-27 02:38:47,960][105620] Updated weights for policy 1, policy_version 1547883 (0.0009) [2023-12-27 02:38:48,011][105620] Updated weights for policy 1, policy_version 1547893 (0.0006) [2023-12-27 02:38:48,079][105620] Updated weights for policy 1, policy_version 1547903 (0.0008) [2023-12-27 02:38:48,403][105692] Updated weights for policy 0, policy_version 1544689 (0.0007) [2023-12-27 02:38:48,477][105692] Updated weights for policy 0, policy_version 1544699 (0.0010) [2023-12-27 02:38:48,551][105692] Updated weights for policy 0, policy_version 1544709 (0.0006) [2023-12-27 02:38:48,712][105620] Updated weights for policy 1, policy_version 1547913 (0.0010) [2023-12-27 02:38:48,782][105620] Updated weights for policy 1, policy_version 1547923 (0.0009) [2023-12-27 02:38:48,854][105620] Updated weights for policy 1, policy_version 1547933 (0.0009) [2023-12-27 02:38:48,915][105620] Updated weights for policy 1, policy_version 1547943 (0.0009) [2023-12-27 02:38:49,123][105692] Updated weights for policy 0, policy_version 1544719 (0.0009) [2023-12-27 02:38:49,178][105692] Updated weights for policy 0, policy_version 1544729 (0.0010) [2023-12-27 02:38:49,245][105692] Updated weights for policy 0, policy_version 1544739 (0.0011) [2023-12-27 02:38:49,628][105620] Updated weights for policy 1, policy_version 1547953 (0.0006) [2023-12-27 02:38:49,696][105620] Updated weights for policy 1, policy_version 1547963 (0.0005) [2023-12-27 02:38:49,761][105620] Updated weights for policy 1, policy_version 1547973 (0.0008) [2023-12-27 02:38:49,983][105692] Updated weights for policy 0, policy_version 1544749 (0.0009) [2023-12-27 02:38:50,042][105692] Updated weights for policy 0, policy_version 1544759 (0.0008) [2023-12-27 02:38:50,096][105692] Updated weights for policy 0, policy_version 1544769 (0.0008) [2023-12-27 02:38:50,353][105620] Updated weights for policy 1, policy_version 1547983 (0.0008) [2023-12-27 02:38:50,419][105620] Updated weights for policy 1, policy_version 1547993 (0.0006) [2023-12-27 02:38:50,478][105620] Updated weights for policy 1, policy_version 1548003 (0.0005) [2023-12-27 02:38:50,898][105692] Updated weights for policy 0, policy_version 1544779 (0.0008) [2023-12-27 02:38:50,960][105692] Updated weights for policy 0, policy_version 1544789 (0.0009) [2023-12-27 02:38:51,025][105692] Updated weights for policy 0, policy_version 1544799 (0.0009) [2023-12-27 02:38:51,054][105620] Updated weights for policy 1, policy_version 1548013 (0.0007) [2023-12-27 02:38:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 791863296. Throughput: 0: 9547.8, 1: 9944.8. Samples: 791859076. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:38:51,063][104569] Avg episode reward: [(0, '8443.150'), (1, '9081.760')] [2023-12-27 02:38:51,113][105620] Updated weights for policy 1, policy_version 1548023 (0.0007) [2023-12-27 02:38:51,178][105620] Updated weights for policy 1, policy_version 1548033 (0.0010) [2023-12-27 02:38:51,804][105692] Updated weights for policy 0, policy_version 1544809 (0.0008) [2023-12-27 02:38:51,869][105692] Updated weights for policy 0, policy_version 1544819 (0.0010) [2023-12-27 02:38:51,923][105620] Updated weights for policy 1, policy_version 1548043 (0.0008) [2023-12-27 02:38:51,928][105692] Updated weights for policy 0, policy_version 1544829 (0.0010) [2023-12-27 02:38:51,983][105692] Updated weights for policy 0, policy_version 1544839 (0.0008) [2023-12-27 02:38:51,985][105620] Updated weights for policy 1, policy_version 1548053 (0.0006) [2023-12-27 02:38:52,041][105620] Updated weights for policy 1, policy_version 1548063 (0.0009) [2023-12-27 02:38:52,745][105692] Updated weights for policy 0, policy_version 1544849 (0.0009) [2023-12-27 02:38:52,765][105620] Updated weights for policy 1, policy_version 1548073 (0.0005) [2023-12-27 02:38:52,803][105692] Updated weights for policy 0, policy_version 1544859 (0.0008) [2023-12-27 02:38:52,827][105620] Updated weights for policy 1, policy_version 1548083 (0.0007) [2023-12-27 02:38:52,863][105692] Updated weights for policy 0, policy_version 1544869 (0.0007) [2023-12-27 02:38:52,882][105620] Updated weights for policy 1, policy_version 1548093 (0.0007) [2023-12-27 02:38:52,934][105620] Updated weights for policy 1, policy_version 1548103 (0.0009) [2023-12-27 02:38:53,540][105692] Updated weights for policy 0, policy_version 1544879 (0.0007) [2023-12-27 02:38:53,602][105692] Updated weights for policy 0, policy_version 1544889 (0.0006) [2023-12-27 02:38:53,654][105692] Updated weights for policy 0, policy_version 1544899 (0.0005) [2023-12-27 02:38:53,720][105620] Updated weights for policy 1, policy_version 1548113 (0.0010) [2023-12-27 02:38:53,778][105620] Updated weights for policy 1, policy_version 1548123 (0.0010) [2023-12-27 02:38:53,830][105620] Updated weights for policy 1, policy_version 1548133 (0.0009) [2023-12-27 02:38:54,325][105692] Updated weights for policy 0, policy_version 1544909 (0.0005) [2023-12-27 02:38:54,379][105692] Updated weights for policy 0, policy_version 1544919 (0.0005) [2023-12-27 02:38:54,428][105692] Updated weights for policy 0, policy_version 1544929 (0.0005) [2023-12-27 02:38:54,511][105620] Updated weights for policy 1, policy_version 1548143 (0.0009) [2023-12-27 02:38:54,580][105620] Updated weights for policy 1, policy_version 1548153 (0.0009) [2023-12-27 02:38:54,643][105620] Updated weights for policy 1, policy_version 1548163 (0.0009) [2023-12-27 02:38:55,058][105692] Updated weights for policy 0, policy_version 1544939 (0.0006) [2023-12-27 02:38:55,122][105692] Updated weights for policy 0, policy_version 1544949 (0.0009) [2023-12-27 02:38:55,182][105692] Updated weights for policy 0, policy_version 1544959 (0.0008) [2023-12-27 02:38:55,339][105620] Updated weights for policy 1, policy_version 1548173 (0.0007) [2023-12-27 02:38:55,402][105620] Updated weights for policy 1, policy_version 1548183 (0.0007) [2023-12-27 02:38:55,461][105620] Updated weights for policy 1, policy_version 1548193 (0.0011) [2023-12-27 02:38:55,774][105692] Updated weights for policy 0, policy_version 1544969 (0.0010) [2023-12-27 02:38:55,821][105692] Updated weights for policy 0, policy_version 1544979 (0.0008) [2023-12-27 02:38:55,877][105692] Updated weights for policy 0, policy_version 1544989 (0.0006) [2023-12-27 02:38:55,933][105692] Updated weights for policy 0, policy_version 1544999 (0.0005) [2023-12-27 02:38:56,039][105620] Updated weights for policy 1, policy_version 1548203 (0.0010) [2023-12-27 02:38:56,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 791969792. Throughput: 0: 9607.7, 1: 9886.5. Samples: 791978096. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:38:56,063][104569] Avg episode reward: [(0, '8261.798'), (1, '8992.288')] [2023-12-27 02:38:56,098][105620] Updated weights for policy 1, policy_version 1548213 (0.0011) [2023-12-27 02:38:56,146][105620] Updated weights for policy 1, policy_version 1548223 (0.0010) [2023-12-27 02:38:56,451][105692] Updated weights for policy 0, policy_version 1545009 (0.0010) [2023-12-27 02:38:56,502][105692] Updated weights for policy 0, policy_version 1545019 (0.0010) [2023-12-27 02:38:56,555][105692] Updated weights for policy 0, policy_version 1545029 (0.0009) [2023-12-27 02:38:56,887][105620] Updated weights for policy 1, policy_version 1548233 (0.0010) [2023-12-27 02:38:56,945][105620] Updated weights for policy 1, policy_version 1548243 (0.0007) [2023-12-27 02:38:57,004][105620] Updated weights for policy 1, policy_version 1548253 (0.0010) [2023-12-27 02:38:57,062][105620] Updated weights for policy 1, policy_version 1548263 (0.0011) [2023-12-27 02:38:57,203][105692] Updated weights for policy 0, policy_version 1545039 (0.0008) [2023-12-27 02:38:57,251][105692] Updated weights for policy 0, policy_version 1545049 (0.0008) [2023-12-27 02:38:57,307][105692] Updated weights for policy 0, policy_version 1545059 (0.0008) [2023-12-27 02:38:57,788][105620] Updated weights for policy 1, policy_version 1548273 (0.0010) [2023-12-27 02:38:57,834][105620] Updated weights for policy 1, policy_version 1548283 (0.0010) [2023-12-27 02:38:57,888][105620] Updated weights for policy 1, policy_version 1548293 (0.0010) [2023-12-27 02:38:58,041][105692] Updated weights for policy 0, policy_version 1545069 (0.0008) [2023-12-27 02:38:58,093][105692] Updated weights for policy 0, policy_version 1545079 (0.0008) [2023-12-27 02:38:58,147][105692] Updated weights for policy 0, policy_version 1545089 (0.0008) [2023-12-27 02:38:58,760][105620] Updated weights for policy 1, policy_version 1548303 (0.0008) [2023-12-27 02:38:58,814][105620] Updated weights for policy 1, policy_version 1548313 (0.0007) [2023-12-27 02:38:58,863][105620] Updated weights for policy 1, policy_version 1548323 (0.0008) [2023-12-27 02:38:58,977][105692] Updated weights for policy 0, policy_version 1545099 (0.0007) [2023-12-27 02:38:59,030][105692] Updated weights for policy 0, policy_version 1545109 (0.0006) [2023-12-27 02:38:59,076][105692] Updated weights for policy 0, policy_version 1545119 (0.0007) [2023-12-27 02:38:59,549][105620] Updated weights for policy 1, policy_version 1548333 (0.0007) [2023-12-27 02:38:59,606][105620] Updated weights for policy 1, policy_version 1548343 (0.0005) [2023-12-27 02:38:59,668][105620] Updated weights for policy 1, policy_version 1548353 (0.0005) [2023-12-27 02:38:59,844][105692] Updated weights for policy 0, policy_version 1545129 (0.0006) [2023-12-27 02:38:59,902][105692] Updated weights for policy 0, policy_version 1545139 (0.0009) [2023-12-27 02:38:59,966][105692] Updated weights for policy 0, policy_version 1545149 (0.0009) [2023-12-27 02:39:00,027][105692] Updated weights for policy 0, policy_version 1545159 (0.0008) [2023-12-27 02:39:00,260][105620] Updated weights for policy 1, policy_version 1548363 (0.0006) [2023-12-27 02:39:00,311][105620] Updated weights for policy 1, policy_version 1548373 (0.0010) [2023-12-27 02:39:00,368][105620] Updated weights for policy 1, policy_version 1548383 (0.0010) [2023-12-27 02:39:00,760][105692] Updated weights for policy 0, policy_version 1545169 (0.0006) [2023-12-27 02:39:00,817][105692] Updated weights for policy 0, policy_version 1545179 (0.0005) [2023-12-27 02:39:00,877][105692] Updated weights for policy 0, policy_version 1545189 (0.0005) [2023-12-27 02:39:01,046][105620] Updated weights for policy 1, policy_version 1548393 (0.0008) [2023-12-27 02:39:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 792068096. Throughput: 0: 9723.8, 1: 9869.9. Samples: 792037500. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:39:01,063][104569] Avg episode reward: [(0, '8164.672'), (1, '8993.738')] [2023-12-27 02:39:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001545192_395624448.pth... [2023-12-27 02:39:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001544040_395329536.pth [2023-12-27 02:39:01,106][105620] Updated weights for policy 1, policy_version 1548403 (0.0006) [2023-12-27 02:39:01,168][105620] Updated weights for policy 1, policy_version 1548413 (0.0007) [2023-12-27 02:39:01,224][105620] Updated weights for policy 1, policy_version 1548423 (0.0008) [2023-12-27 02:39:01,227][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001548424_396451840.pth... [2023-12-27 02:39:01,231][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001547240_396148736.pth [2023-12-27 02:39:01,485][105692] Updated weights for policy 0, policy_version 1545199 (0.0008) [2023-12-27 02:39:01,540][105692] Updated weights for policy 0, policy_version 1545209 (0.0008) [2023-12-27 02:39:01,612][105692] Updated weights for policy 0, policy_version 1545219 (0.0010) [2023-12-27 02:39:01,968][105620] Updated weights for policy 1, policy_version 1548433 (0.0008) [2023-12-27 02:39:02,028][105620] Updated weights for policy 1, policy_version 1548443 (0.0008) [2023-12-27 02:39:02,083][105620] Updated weights for policy 1, policy_version 1548453 (0.0009) [2023-12-27 02:39:02,353][105692] Updated weights for policy 0, policy_version 1545229 (0.0008) [2023-12-27 02:39:02,419][105692] Updated weights for policy 0, policy_version 1545239 (0.0006) [2023-12-27 02:39:02,488][105692] Updated weights for policy 0, policy_version 1545249 (0.0007) [2023-12-27 02:39:02,754][105620] Updated weights for policy 1, policy_version 1548463 (0.0006) [2023-12-27 02:39:02,817][105620] Updated weights for policy 1, policy_version 1548473 (0.0005) [2023-12-27 02:39:02,873][105620] Updated weights for policy 1, policy_version 1548483 (0.0006) [2023-12-27 02:39:03,165][105692] Updated weights for policy 0, policy_version 1545259 (0.0010) [2023-12-27 02:39:03,221][105692] Updated weights for policy 0, policy_version 1545269 (0.0010) [2023-12-27 02:39:03,281][105692] Updated weights for policy 0, policy_version 1545279 (0.0009) [2023-12-27 02:39:03,458][105620] Updated weights for policy 1, policy_version 1548493 (0.0008) [2023-12-27 02:39:03,514][105620] Updated weights for policy 1, policy_version 1548503 (0.0005) [2023-12-27 02:39:03,576][105620] Updated weights for policy 1, policy_version 1548513 (0.0005) [2023-12-27 02:39:03,959][105692] Updated weights for policy 0, policy_version 1545289 (0.0006) [2023-12-27 02:39:04,008][105692] Updated weights for policy 0, policy_version 1545299 (0.0010) [2023-12-27 02:39:04,053][105692] Updated weights for policy 0, policy_version 1545309 (0.0010) [2023-12-27 02:39:04,102][105692] Updated weights for policy 0, policy_version 1545319 (0.0010) [2023-12-27 02:39:04,238][105620] Updated weights for policy 1, policy_version 1548523 (0.0006) [2023-12-27 02:39:04,293][105620] Updated weights for policy 1, policy_version 1548533 (0.0008) [2023-12-27 02:39:04,344][105620] Updated weights for policy 1, policy_version 1548543 (0.0007) [2023-12-27 02:39:04,842][105692] Updated weights for policy 0, policy_version 1545329 (0.0010) [2023-12-27 02:39:04,904][105692] Updated weights for policy 0, policy_version 1545339 (0.0008) [2023-12-27 02:39:04,972][105692] Updated weights for policy 0, policy_version 1545349 (0.0010) [2023-12-27 02:39:05,022][105620] Updated weights for policy 1, policy_version 1548553 (0.0008) [2023-12-27 02:39:05,083][105620] Updated weights for policy 1, policy_version 1548563 (0.0006) [2023-12-27 02:39:05,145][105620] Updated weights for policy 1, policy_version 1548573 (0.0008) [2023-12-27 02:39:05,208][105620] Updated weights for policy 1, policy_version 1548583 (0.0005) [2023-12-27 02:39:05,709][105692] Updated weights for policy 0, policy_version 1545359 (0.0010) [2023-12-27 02:39:05,777][105692] Updated weights for policy 0, policy_version 1545369 (0.0010) [2023-12-27 02:39:05,785][105620] Updated weights for policy 1, policy_version 1548593 (0.0006) [2023-12-27 02:39:05,835][105692] Updated weights for policy 0, policy_version 1545379 (0.0010) [2023-12-27 02:39:05,849][105620] Updated weights for policy 1, policy_version 1548603 (0.0005) [2023-12-27 02:39:05,908][105620] Updated weights for policy 1, policy_version 1548613 (0.0005) [2023-12-27 02:39:06,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 792174592. Throughput: 0: 9772.7, 1: 9918.7. Samples: 792158420. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:39:06,062][104569] Avg episode reward: [(0, '7722.166'), (1, '8719.697')] [2023-12-27 02:39:06,509][105620] Updated weights for policy 1, policy_version 1548623 (0.0006) [2023-12-27 02:39:06,555][105620] Updated weights for policy 1, policy_version 1548633 (0.0008) [2023-12-27 02:39:06,565][105692] Updated weights for policy 0, policy_version 1545389 (0.0009) [2023-12-27 02:39:06,608][105620] Updated weights for policy 1, policy_version 1548643 (0.0006) [2023-12-27 02:39:06,622][105692] Updated weights for policy 0, policy_version 1545399 (0.0011) [2023-12-27 02:39:06,681][105692] Updated weights for policy 0, policy_version 1545409 (0.0011) [2023-12-27 02:39:07,321][105692] Updated weights for policy 0, policy_version 1545419 (0.0009) [2023-12-27 02:39:07,357][105620] Updated weights for policy 1, policy_version 1548653 (0.0006) [2023-12-27 02:39:07,373][105692] Updated weights for policy 0, policy_version 1545429 (0.0005) [2023-12-27 02:39:07,419][105620] Updated weights for policy 1, policy_version 1548663 (0.0008) [2023-12-27 02:39:07,425][105692] Updated weights for policy 0, policy_version 1545439 (0.0005) [2023-12-27 02:39:07,484][105620] Updated weights for policy 1, policy_version 1548673 (0.0009) [2023-12-27 02:39:07,973][105692] Updated weights for policy 0, policy_version 1545449 (0.0006) [2023-12-27 02:39:08,039][105692] Updated weights for policy 0, policy_version 1545459 (0.0008) [2023-12-27 02:39:08,103][105692] Updated weights for policy 0, policy_version 1545469 (0.0009) [2023-12-27 02:39:08,168][105692] Updated weights for policy 0, policy_version 1545479 (0.0009) [2023-12-27 02:39:08,256][105620] Updated weights for policy 1, policy_version 1548683 (0.0009) [2023-12-27 02:39:08,315][105620] Updated weights for policy 1, policy_version 1548693 (0.0010) [2023-12-27 02:39:08,384][105620] Updated weights for policy 1, policy_version 1548703 (0.0009) [2023-12-27 02:39:08,895][105692] Updated weights for policy 0, policy_version 1545489 (0.0009) [2023-12-27 02:39:08,953][105692] Updated weights for policy 0, policy_version 1545499 (0.0009) [2023-12-27 02:39:09,012][105692] Updated weights for policy 0, policy_version 1545509 (0.0009) [2023-12-27 02:39:09,132][105620] Updated weights for policy 1, policy_version 1548713 (0.0009) [2023-12-27 02:39:09,201][105620] Updated weights for policy 1, policy_version 1548723 (0.0009) [2023-12-27 02:39:09,272][105620] Updated weights for policy 1, policy_version 1548733 (0.0009) [2023-12-27 02:39:09,337][105620] Updated weights for policy 1, policy_version 1548743 (0.0009) [2023-12-27 02:39:09,771][105692] Updated weights for policy 0, policy_version 1545519 (0.0009) [2023-12-27 02:39:09,838][105692] Updated weights for policy 0, policy_version 1545529 (0.0008) [2023-12-27 02:39:09,895][105692] Updated weights for policy 0, policy_version 1545539 (0.0009) [2023-12-27 02:39:10,125][105620] Updated weights for policy 1, policy_version 1548753 (0.0006) [2023-12-27 02:39:10,187][105620] Updated weights for policy 1, policy_version 1548763 (0.0006) [2023-12-27 02:39:10,258][105620] Updated weights for policy 1, policy_version 1548773 (0.0006) [2023-12-27 02:39:10,645][105692] Updated weights for policy 0, policy_version 1545549 (0.0009) [2023-12-27 02:39:10,693][105692] Updated weights for policy 0, policy_version 1545559 (0.0009) [2023-12-27 02:39:10,745][105692] Updated weights for policy 0, policy_version 1545569 (0.0009) [2023-12-27 02:39:10,942][105620] Updated weights for policy 1, policy_version 1548783 (0.0009) [2023-12-27 02:39:10,995][105620] Updated weights for policy 1, policy_version 1548793 (0.0008) [2023-12-27 02:39:11,059][105620] Updated weights for policy 1, policy_version 1548803 (0.0007) [2023-12-27 02:39:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 792264704. Throughput: 0: 9745.2, 1: 9931.0. Samples: 792276664. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:39:11,063][104569] Avg episode reward: [(0, '8366.426'), (1, '8628.656')] [2023-12-27 02:39:11,576][105692] Updated weights for policy 0, policy_version 1545579 (0.0008) [2023-12-27 02:39:11,647][105692] Updated weights for policy 0, policy_version 1545589 (0.0009) [2023-12-27 02:39:11,710][105692] Updated weights for policy 0, policy_version 1545599 (0.0008) [2023-12-27 02:39:11,739][105620] Updated weights for policy 1, policy_version 1548813 (0.0007) [2023-12-27 02:39:11,805][105620] Updated weights for policy 1, policy_version 1548823 (0.0008) [2023-12-27 02:39:11,874][105620] Updated weights for policy 1, policy_version 1548833 (0.0009) [2023-12-27 02:39:12,404][105692] Updated weights for policy 0, policy_version 1545609 (0.0008) [2023-12-27 02:39:12,453][105692] Updated weights for policy 0, policy_version 1545619 (0.0009) [2023-12-27 02:39:12,502][105692] Updated weights for policy 0, policy_version 1545629 (0.0009) [2023-12-27 02:39:12,559][105692] Updated weights for policy 0, policy_version 1545640 (0.0010) [2023-12-27 02:39:12,573][105620] Updated weights for policy 1, policy_version 1548843 (0.0007) [2023-12-27 02:39:12,635][105620] Updated weights for policy 1, policy_version 1548853 (0.0007) [2023-12-27 02:39:12,704][105620] Updated weights for policy 1, policy_version 1548863 (0.0009) [2023-12-27 02:39:13,305][105692] Updated weights for policy 0, policy_version 1545650 (0.0007) [2023-12-27 02:39:13,370][105692] Updated weights for policy 0, policy_version 1545660 (0.0007) [2023-12-27 02:39:13,436][105692] Updated weights for policy 0, policy_version 1545670 (0.0007) [2023-12-27 02:39:13,447][105620] Updated weights for policy 1, policy_version 1548873 (0.0009) [2023-12-27 02:39:13,510][105620] Updated weights for policy 1, policy_version 1548883 (0.0008) [2023-12-27 02:39:13,566][105620] Updated weights for policy 1, policy_version 1548893 (0.0009) [2023-12-27 02:39:13,630][105620] Updated weights for policy 1, policy_version 1548903 (0.0009) [2023-12-27 02:39:14,116][105692] Updated weights for policy 0, policy_version 1545680 (0.0010) [2023-12-27 02:39:14,180][105692] Updated weights for policy 0, policy_version 1545690 (0.0010) [2023-12-27 02:39:14,235][105692] Updated weights for policy 0, policy_version 1545700 (0.0010) [2023-12-27 02:39:14,327][105620] Updated weights for policy 1, policy_version 1548913 (0.0010) [2023-12-27 02:39:14,382][105620] Updated weights for policy 1, policy_version 1548923 (0.0010) [2023-12-27 02:39:14,446][105620] Updated weights for policy 1, policy_version 1548933 (0.0010) [2023-12-27 02:39:14,964][105692] Updated weights for policy 0, policy_version 1545710 (0.0010) [2023-12-27 02:39:15,018][105692] Updated weights for policy 0, policy_version 1545720 (0.0010) [2023-12-27 02:39:15,071][105692] Updated weights for policy 0, policy_version 1545730 (0.0011) [2023-12-27 02:39:15,183][105620] Updated weights for policy 1, policy_version 1548943 (0.0009) [2023-12-27 02:39:15,248][105620] Updated weights for policy 1, policy_version 1548953 (0.0008) [2023-12-27 02:39:15,316][105620] Updated weights for policy 1, policy_version 1548963 (0.0010) [2023-12-27 02:39:15,705][105692] Updated weights for policy 0, policy_version 1545740 (0.0009) [2023-12-27 02:39:15,770][105692] Updated weights for policy 0, policy_version 1545750 (0.0008) [2023-12-27 02:39:15,836][105692] Updated weights for policy 0, policy_version 1545760 (0.0010) [2023-12-27 02:39:16,022][105620] Updated weights for policy 1, policy_version 1548973 (0.0007) [2023-12-27 02:39:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 792363008. Throughput: 0: 9682.9, 1: 9919.2. Samples: 792333060. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:39:16,062][104569] Avg episode reward: [(0, '8537.479'), (1, '8810.615')] [2023-12-27 02:39:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001545768_395771904.pth... [2023-12-27 02:39:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001544616_395476992.pth [2023-12-27 02:39:16,077][105620] Updated weights for policy 1, policy_version 1548983 (0.0007) [2023-12-27 02:39:16,136][105620] Updated weights for policy 1, policy_version 1548993 (0.0005) [2023-12-27 02:39:16,174][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001549000_396599296.pth... [2023-12-27 02:39:16,178][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001547816_396296192.pth [2023-12-27 02:39:16,537][105692] Updated weights for policy 0, policy_version 1545770 (0.0010) [2023-12-27 02:39:16,586][105692] Updated weights for policy 0, policy_version 1545780 (0.0010) [2023-12-27 02:39:16,633][105692] Updated weights for policy 0, policy_version 1545790 (0.0010) [2023-12-27 02:39:16,681][105692] Updated weights for policy 0, policy_version 1545800 (0.0010) [2023-12-27 02:39:16,781][105620] Updated weights for policy 1, policy_version 1549003 (0.0006) [2023-12-27 02:39:16,840][105620] Updated weights for policy 1, policy_version 1549013 (0.0008) [2023-12-27 02:39:16,898][105620] Updated weights for policy 1, policy_version 1549023 (0.0008) [2023-12-27 02:39:17,445][105692] Updated weights for policy 0, policy_version 1545810 (0.0010) [2023-12-27 02:39:17,493][105620] Updated weights for policy 1, policy_version 1549033 (0.0007) [2023-12-27 02:39:17,500][105692] Updated weights for policy 0, policy_version 1545820 (0.0010) [2023-12-27 02:39:17,539][105620] Updated weights for policy 1, policy_version 1549043 (0.0005) [2023-12-27 02:39:17,548][105692] Updated weights for policy 0, policy_version 1545830 (0.0010) [2023-12-27 02:39:17,587][105620] Updated weights for policy 1, policy_version 1549053 (0.0005) [2023-12-27 02:39:17,633][105620] Updated weights for policy 1, policy_version 1549063 (0.0005) [2023-12-27 02:39:18,182][105620] Updated weights for policy 1, policy_version 1549073 (0.0005) [2023-12-27 02:39:18,228][105620] Updated weights for policy 1, policy_version 1549083 (0.0005) [2023-12-27 02:39:18,283][105620] Updated weights for policy 1, policy_version 1549093 (0.0006) [2023-12-27 02:39:18,319][105692] Updated weights for policy 0, policy_version 1545840 (0.0011) [2023-12-27 02:39:18,385][105692] Updated weights for policy 0, policy_version 1545850 (0.0011) [2023-12-27 02:39:18,430][105692] Updated weights for policy 0, policy_version 1545860 (0.0011) [2023-12-27 02:39:19,002][105620] Updated weights for policy 1, policy_version 1549103 (0.0006) [2023-12-27 02:39:19,063][105620] Updated weights for policy 1, policy_version 1549113 (0.0006) [2023-12-27 02:39:19,124][105620] Updated weights for policy 1, policy_version 1549123 (0.0005) [2023-12-27 02:39:19,166][105692] Updated weights for policy 0, policy_version 1545870 (0.0007) [2023-12-27 02:39:19,228][105692] Updated weights for policy 0, policy_version 1545880 (0.0006) [2023-12-27 02:39:19,297][105692] Updated weights for policy 0, policy_version 1545890 (0.0008) [2023-12-27 02:39:19,750][105620] Updated weights for policy 1, policy_version 1549133 (0.0006) [2023-12-27 02:39:19,811][105620] Updated weights for policy 1, policy_version 1549143 (0.0011) [2023-12-27 02:39:19,879][105620] Updated weights for policy 1, policy_version 1549153 (0.0011) [2023-12-27 02:39:20,059][105692] Updated weights for policy 0, policy_version 1545900 (0.0009) [2023-12-27 02:39:20,124][105692] Updated weights for policy 0, policy_version 1545910 (0.0011) [2023-12-27 02:39:20,188][105692] Updated weights for policy 0, policy_version 1545920 (0.0011) [2023-12-27 02:39:20,613][105620] Updated weights for policy 1, policy_version 1549163 (0.0010) [2023-12-27 02:39:20,674][105620] Updated weights for policy 1, policy_version 1549173 (0.0008) [2023-12-27 02:39:20,737][105620] Updated weights for policy 1, policy_version 1549183 (0.0008) [2023-12-27 02:39:20,915][105692] Updated weights for policy 0, policy_version 1545930 (0.0011) [2023-12-27 02:39:20,979][105692] Updated weights for policy 0, policy_version 1545940 (0.0007) [2023-12-27 02:39:21,047][105692] Updated weights for policy 0, policy_version 1545950 (0.0010) [2023-12-27 02:39:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 792461312. Throughput: 0: 9726.8, 1: 9994.5. Samples: 792454540. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:39:21,063][104569] Avg episode reward: [(0, '8351.046'), (1, '8993.319')] [2023-12-27 02:39:21,108][105692] Updated weights for policy 0, policy_version 1545960 (0.0011) [2023-12-27 02:39:21,613][105620] Updated weights for policy 1, policy_version 1549193 (0.0009) [2023-12-27 02:39:21,679][105620] Updated weights for policy 1, policy_version 1549203 (0.0009) [2023-12-27 02:39:21,753][105620] Updated weights for policy 1, policy_version 1549213 (0.0008) [2023-12-27 02:39:21,766][105692] Updated weights for policy 0, policy_version 1545970 (0.0008) [2023-12-27 02:39:21,819][105620] Updated weights for policy 1, policy_version 1549223 (0.0006) [2023-12-27 02:39:21,828][105692] Updated weights for policy 0, policy_version 1545980 (0.0009) [2023-12-27 02:39:21,892][105692] Updated weights for policy 0, policy_version 1545990 (0.0009) [2023-12-27 02:39:22,579][105620] Updated weights for policy 1, policy_version 1549233 (0.0007) [2023-12-27 02:39:22,641][105620] Updated weights for policy 1, policy_version 1549243 (0.0007) [2023-12-27 02:39:22,672][105692] Updated weights for policy 0, policy_version 1546000 (0.0007) [2023-12-27 02:39:22,708][105620] Updated weights for policy 1, policy_version 1549253 (0.0006) [2023-12-27 02:39:22,735][105692] Updated weights for policy 0, policy_version 1546010 (0.0006) [2023-12-27 02:39:22,803][105692] Updated weights for policy 0, policy_version 1546020 (0.0007) [2023-12-27 02:39:23,382][105692] Updated weights for policy 0, policy_version 1546030 (0.0008) [2023-12-27 02:39:23,443][105692] Updated weights for policy 0, policy_version 1546040 (0.0011) [2023-12-27 02:39:23,443][105620] Updated weights for policy 1, policy_version 1549263 (0.0009) [2023-12-27 02:39:23,502][105620] Updated weights for policy 1, policy_version 1549273 (0.0010) [2023-12-27 02:39:23,502][105692] Updated weights for policy 0, policy_version 1546050 (0.0011) [2023-12-27 02:39:23,558][105620] Updated weights for policy 1, policy_version 1549283 (0.0008) [2023-12-27 02:39:24,057][105692] Updated weights for policy 0, policy_version 1546060 (0.0008) [2023-12-27 02:39:24,110][105692] Updated weights for policy 0, policy_version 1546070 (0.0005) [2023-12-27 02:39:24,166][105692] Updated weights for policy 0, policy_version 1546080 (0.0005) [2023-12-27 02:39:24,433][105620] Updated weights for policy 1, policy_version 1549293 (0.0009) [2023-12-27 02:39:24,495][105620] Updated weights for policy 1, policy_version 1549303 (0.0007) [2023-12-27 02:39:24,558][105620] Updated weights for policy 1, policy_version 1549313 (0.0008) [2023-12-27 02:39:24,809][105692] Updated weights for policy 0, policy_version 1546090 (0.0006) [2023-12-27 02:39:24,860][105692] Updated weights for policy 0, policy_version 1546100 (0.0010) [2023-12-27 02:39:24,908][105692] Updated weights for policy 0, policy_version 1546110 (0.0010) [2023-12-27 02:39:24,965][105692] Updated weights for policy 0, policy_version 1546120 (0.0010) [2023-12-27 02:39:25,247][105620] Updated weights for policy 1, policy_version 1549323 (0.0008) [2023-12-27 02:39:25,317][105620] Updated weights for policy 1, policy_version 1549333 (0.0005) [2023-12-27 02:39:25,368][105620] Updated weights for policy 1, policy_version 1549343 (0.0005) [2023-12-27 02:39:25,737][105692] Updated weights for policy 0, policy_version 1546130 (0.0008) [2023-12-27 02:39:25,797][105692] Updated weights for policy 0, policy_version 1546140 (0.0009) [2023-12-27 02:39:25,850][105692] Updated weights for policy 0, policy_version 1546150 (0.0009) [2023-12-27 02:39:25,882][105620] Updated weights for policy 1, policy_version 1549353 (0.0005) [2023-12-27 02:39:25,943][105620] Updated weights for policy 1, policy_version 1549363 (0.0007) [2023-12-27 02:39:26,007][105620] Updated weights for policy 1, policy_version 1549373 (0.0009) [2023-12-27 02:39:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 792559616. Throughput: 0: 9832.9, 1: 9886.2. Samples: 792571224. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:39:26,063][104569] Avg episode reward: [(0, '8716.206'), (1, '8989.417')] [2023-12-27 02:39:26,069][105620] Updated weights for policy 1, policy_version 1549383 (0.0010) [2023-12-27 02:39:26,587][105692] Updated weights for policy 0, policy_version 1546160 (0.0007) [2023-12-27 02:39:26,649][105692] Updated weights for policy 0, policy_version 1546170 (0.0005) [2023-12-27 02:39:26,715][105692] Updated weights for policy 0, policy_version 1546180 (0.0006) [2023-12-27 02:39:26,816][105620] Updated weights for policy 1, policy_version 1549393 (0.0011) [2023-12-27 02:39:26,862][105620] Updated weights for policy 1, policy_version 1549403 (0.0006) [2023-12-27 02:39:26,909][105620] Updated weights for policy 1, policy_version 1549413 (0.0005) [2023-12-27 02:39:27,287][105692] Updated weights for policy 0, policy_version 1546190 (0.0008) [2023-12-27 02:39:27,355][105692] Updated weights for policy 0, policy_version 1546200 (0.0009) [2023-12-27 02:39:27,418][105692] Updated weights for policy 0, policy_version 1546210 (0.0006) [2023-12-27 02:39:27,460][105620] Updated weights for policy 1, policy_version 1549423 (0.0005) [2023-12-27 02:39:27,510][105620] Updated weights for policy 1, policy_version 1549433 (0.0006) [2023-12-27 02:39:27,564][105620] Updated weights for policy 1, policy_version 1549443 (0.0010) [2023-12-27 02:39:28,000][105692] Updated weights for policy 0, policy_version 1546220 (0.0008) [2023-12-27 02:39:28,047][105692] Updated weights for policy 0, policy_version 1546230 (0.0007) [2023-12-27 02:39:28,090][105692] Updated weights for policy 0, policy_version 1546240 (0.0008) [2023-12-27 02:39:28,235][105620] Updated weights for policy 1, policy_version 1549453 (0.0010) [2023-12-27 02:39:28,291][105620] Updated weights for policy 1, policy_version 1549463 (0.0011) [2023-12-27 02:39:28,356][105620] Updated weights for policy 1, policy_version 1549473 (0.0010) [2023-12-27 02:39:28,908][105692] Updated weights for policy 0, policy_version 1546250 (0.0008) [2023-12-27 02:39:28,968][105692] Updated weights for policy 0, policy_version 1546260 (0.0009) [2023-12-27 02:39:29,016][105692] Updated weights for policy 0, policy_version 1546270 (0.0007) [2023-12-27 02:39:29,052][105620] Updated weights for policy 1, policy_version 1549483 (0.0007) [2023-12-27 02:39:29,074][105692] Updated weights for policy 0, policy_version 1546280 (0.0005) [2023-12-27 02:39:29,103][105620] Updated weights for policy 1, policy_version 1549494 (0.0010) [2023-12-27 02:39:29,156][105620] Updated weights for policy 1, policy_version 1549505 (0.0009) [2023-12-27 02:39:29,733][105692] Updated weights for policy 0, policy_version 1546290 (0.0006) [2023-12-27 02:39:29,782][105692] Updated weights for policy 0, policy_version 1546300 (0.0008) [2023-12-27 02:39:29,837][105692] Updated weights for policy 0, policy_version 1546310 (0.0006) [2023-12-27 02:39:30,058][105620] Updated weights for policy 1, policy_version 1549515 (0.0009) [2023-12-27 02:39:30,106][105620] Updated weights for policy 1, policy_version 1549525 (0.0008) [2023-12-27 02:39:30,165][105620] Updated weights for policy 1, policy_version 1549535 (0.0008) [2023-12-27 02:39:30,551][105692] Updated weights for policy 0, policy_version 1546320 (0.0009) [2023-12-27 02:39:30,603][105692] Updated weights for policy 0, policy_version 1546330 (0.0010) [2023-12-27 02:39:30,661][105692] Updated weights for policy 0, policy_version 1546340 (0.0010) [2023-12-27 02:39:30,937][105620] Updated weights for policy 1, policy_version 1549545 (0.0008) [2023-12-27 02:39:31,000][105620] Updated weights for policy 1, policy_version 1549555 (0.0009) [2023-12-27 02:39:31,062][105620] Updated weights for policy 1, policy_version 1549565 (0.0007) [2023-12-27 02:39:31,062][104569] Fps is (10 sec: 19660.0, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 792657920. Throughput: 0: 9919.3, 1: 9951.4. Samples: 792632796. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:39:31,063][104569] Avg episode reward: [(0, '8628.436'), (1, '8897.212')] [2023-12-27 02:39:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001546344_395919360.pth... [2023-12-27 02:39:31,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001545192_395624448.pth [2023-12-27 02:39:31,123][105620] Updated weights for policy 1, policy_version 1549575 (0.0009) [2023-12-27 02:39:31,130][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001549576_396746752.pth... [2023-12-27 02:39:31,133][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001548424_396451840.pth [2023-12-27 02:39:31,384][105692] Updated weights for policy 0, policy_version 1546350 (0.0009) [2023-12-27 02:39:31,439][105692] Updated weights for policy 0, policy_version 1546360 (0.0008) [2023-12-27 02:39:31,487][105692] Updated weights for policy 0, policy_version 1546370 (0.0008) [2023-12-27 02:39:31,879][105620] Updated weights for policy 1, policy_version 1549585 (0.0009) [2023-12-27 02:39:31,935][105620] Updated weights for policy 1, policy_version 1549595 (0.0009) [2023-12-27 02:39:31,986][105620] Updated weights for policy 1, policy_version 1549605 (0.0009) [2023-12-27 02:39:32,287][105692] Updated weights for policy 0, policy_version 1546380 (0.0009) [2023-12-27 02:39:32,347][105692] Updated weights for policy 0, policy_version 1546390 (0.0009) [2023-12-27 02:39:32,413][105692] Updated weights for policy 0, policy_version 1546400 (0.0009) [2023-12-27 02:39:32,625][105620] Updated weights for policy 1, policy_version 1549615 (0.0006) [2023-12-27 02:39:32,676][105620] Updated weights for policy 1, policy_version 1549625 (0.0005) [2023-12-27 02:39:32,730][105620] Updated weights for policy 1, policy_version 1549635 (0.0005) [2023-12-27 02:39:33,288][105620] Updated weights for policy 1, policy_version 1549645 (0.0005) [2023-12-27 02:39:33,295][105692] Updated weights for policy 0, policy_version 1546410 (0.0009) [2023-12-27 02:39:33,346][105620] Updated weights for policy 1, policy_version 1549655 (0.0006) [2023-12-27 02:39:33,355][105692] Updated weights for policy 0, policy_version 1546420 (0.0008) [2023-12-27 02:39:33,397][105620] Updated weights for policy 1, policy_version 1549665 (0.0006) [2023-12-27 02:39:33,411][105692] Updated weights for policy 0, policy_version 1546430 (0.0008) [2023-12-27 02:39:33,461][105692] Updated weights for policy 0, policy_version 1546440 (0.0007) [2023-12-27 02:39:34,159][105620] Updated weights for policy 1, policy_version 1549675 (0.0006) [2023-12-27 02:39:34,166][105692] Updated weights for policy 0, policy_version 1546450 (0.0009) [2023-12-27 02:39:34,218][105620] Updated weights for policy 1, policy_version 1549685 (0.0008) [2023-12-27 02:39:34,221][105692] Updated weights for policy 0, policy_version 1546460 (0.0007) [2023-12-27 02:39:34,279][105620] Updated weights for policy 1, policy_version 1549695 (0.0007) [2023-12-27 02:39:34,284][105692] Updated weights for policy 0, policy_version 1546470 (0.0009) [2023-12-27 02:39:35,014][105692] Updated weights for policy 0, policy_version 1546480 (0.0006) [2023-12-27 02:39:35,082][105692] Updated weights for policy 0, policy_version 1546490 (0.0008) [2023-12-27 02:39:35,091][105620] Updated weights for policy 1, policy_version 1549705 (0.0008) [2023-12-27 02:39:35,144][105692] Updated weights for policy 0, policy_version 1546500 (0.0007) [2023-12-27 02:39:35,145][105620] Updated weights for policy 1, policy_version 1549715 (0.0005) [2023-12-27 02:39:35,201][105620] Updated weights for policy 1, policy_version 1549725 (0.0005) [2023-12-27 02:39:35,262][105620] Updated weights for policy 1, policy_version 1549735 (0.0005) [2023-12-27 02:39:35,841][105620] Updated weights for policy 1, policy_version 1549745 (0.0005) [2023-12-27 02:39:35,899][105620] Updated weights for policy 1, policy_version 1549755 (0.0006) [2023-12-27 02:39:35,906][105692] Updated weights for policy 0, policy_version 1546510 (0.0008) [2023-12-27 02:39:35,954][105620] Updated weights for policy 1, policy_version 1549765 (0.0010) [2023-12-27 02:39:35,956][105692] Updated weights for policy 0, policy_version 1546520 (0.0006) [2023-12-27 02:39:36,002][105692] Updated weights for policy 0, policy_version 1546530 (0.0007) [2023-12-27 02:39:36,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 792764416. Throughput: 0: 9810.9, 1: 9916.6. Samples: 792746808. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:39:36,062][104569] Avg episode reward: [(0, '8450.218'), (1, '8899.805')] [2023-12-27 02:39:36,564][105620] Updated weights for policy 1, policy_version 1549775 (0.0007) [2023-12-27 02:39:36,631][105620] Updated weights for policy 1, policy_version 1549785 (0.0006) [2023-12-27 02:39:36,701][105620] Updated weights for policy 1, policy_version 1549795 (0.0006) [2023-12-27 02:39:36,863][105692] Updated weights for policy 0, policy_version 1546540 (0.0008) [2023-12-27 02:39:36,923][105692] Updated weights for policy 0, policy_version 1546550 (0.0009) [2023-12-27 02:39:36,983][105692] Updated weights for policy 0, policy_version 1546560 (0.0008) [2023-12-27 02:39:37,319][105620] Updated weights for policy 1, policy_version 1549805 (0.0008) [2023-12-27 02:39:37,382][105620] Updated weights for policy 1, policy_version 1549815 (0.0008) [2023-12-27 02:39:37,445][105620] Updated weights for policy 1, policy_version 1549825 (0.0009) [2023-12-27 02:39:37,776][105692] Updated weights for policy 0, policy_version 1546570 (0.0008) [2023-12-27 02:39:37,827][105692] Updated weights for policy 0, policy_version 1546580 (0.0009) [2023-12-27 02:39:37,885][105692] Updated weights for policy 0, policy_version 1546590 (0.0008) [2023-12-27 02:39:37,942][105692] Updated weights for policy 0, policy_version 1546600 (0.0008) [2023-12-27 02:39:38,153][105620] Updated weights for policy 1, policy_version 1549835 (0.0008) [2023-12-27 02:39:38,201][105620] Updated weights for policy 1, policy_version 1549845 (0.0005) [2023-12-27 02:39:38,245][105620] Updated weights for policy 1, policy_version 1549855 (0.0005) [2023-12-27 02:39:38,824][105620] Updated weights for policy 1, policy_version 1549865 (0.0005) [2023-12-27 02:39:38,825][105692] Updated weights for policy 0, policy_version 1546610 (0.0010) [2023-12-27 02:39:38,876][105620] Updated weights for policy 1, policy_version 1549875 (0.0006) [2023-12-27 02:39:38,887][105692] Updated weights for policy 0, policy_version 1546620 (0.0009) [2023-12-27 02:39:38,930][105620] Updated weights for policy 1, policy_version 1549885 (0.0006) [2023-12-27 02:39:38,957][105692] Updated weights for policy 0, policy_version 1546630 (0.0008) [2023-12-27 02:39:38,993][105620] Updated weights for policy 1, policy_version 1549895 (0.0010) [2023-12-27 02:39:39,700][105620] Updated weights for policy 1, policy_version 1549905 (0.0007) [2023-12-27 02:39:39,717][105692] Updated weights for policy 0, policy_version 1546640 (0.0010) [2023-12-27 02:39:39,756][105620] Updated weights for policy 1, policy_version 1549915 (0.0005) [2023-12-27 02:39:39,770][105692] Updated weights for policy 0, policy_version 1546650 (0.0011) [2023-12-27 02:39:39,813][105620] Updated weights for policy 1, policy_version 1549925 (0.0005) [2023-12-27 02:39:39,827][105692] Updated weights for policy 0, policy_version 1546660 (0.0010) [2023-12-27 02:39:40,512][105620] Updated weights for policy 1, policy_version 1549935 (0.0006) [2023-12-27 02:39:40,570][105620] Updated weights for policy 1, policy_version 1549945 (0.0008) [2023-12-27 02:39:40,633][105620] Updated weights for policy 1, policy_version 1549955 (0.0009) [2023-12-27 02:39:40,662][105692] Updated weights for policy 0, policy_version 1546670 (0.0007) [2023-12-27 02:39:40,725][105692] Updated weights for policy 0, policy_version 1546680 (0.0008) [2023-12-27 02:39:40,783][105692] Updated weights for policy 0, policy_version 1546690 (0.0009) [2023-12-27 02:39:41,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 792854528. Throughput: 0: 9680.0, 1: 9968.2. Samples: 792862264. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:39:41,063][104569] Avg episode reward: [(0, '8270.722'), (1, '9083.865')] [2023-12-27 02:39:41,371][105620] Updated weights for policy 1, policy_version 1549965 (0.0008) [2023-12-27 02:39:41,442][105620] Updated weights for policy 1, policy_version 1549975 (0.0008) [2023-12-27 02:39:41,496][105620] Updated weights for policy 1, policy_version 1549985 (0.0008) [2023-12-27 02:39:41,560][105692] Updated weights for policy 0, policy_version 1546700 (0.0009) [2023-12-27 02:39:41,620][105692] Updated weights for policy 0, policy_version 1546710 (0.0009) [2023-12-27 02:39:41,685][105692] Updated weights for policy 0, policy_version 1546720 (0.0008) [2023-12-27 02:39:42,226][105620] Updated weights for policy 1, policy_version 1549995 (0.0009) [2023-12-27 02:39:42,285][105620] Updated weights for policy 1, policy_version 1550005 (0.0008) [2023-12-27 02:39:42,342][105620] Updated weights for policy 1, policy_version 1550015 (0.0008) [2023-12-27 02:39:42,399][105692] Updated weights for policy 0, policy_version 1546730 (0.0009) [2023-12-27 02:39:42,447][105692] Updated weights for policy 0, policy_version 1546740 (0.0010) [2023-12-27 02:39:42,495][105692] Updated weights for policy 0, policy_version 1546750 (0.0010) [2023-12-27 02:39:42,561][105692] Updated weights for policy 0, policy_version 1546760 (0.0011) [2023-12-27 02:39:43,157][105620] Updated weights for policy 1, policy_version 1550025 (0.0008) [2023-12-27 02:39:43,214][105620] Updated weights for policy 1, policy_version 1550035 (0.0010) [2023-12-27 02:39:43,236][105692] Updated weights for policy 0, policy_version 1546770 (0.0005) [2023-12-27 02:39:43,265][105620] Updated weights for policy 1, policy_version 1550045 (0.0009) [2023-12-27 02:39:43,290][105692] Updated weights for policy 0, policy_version 1546780 (0.0005) [2023-12-27 02:39:43,323][105620] Updated weights for policy 1, policy_version 1550055 (0.0009) [2023-12-27 02:39:43,353][105692] Updated weights for policy 0, policy_version 1546790 (0.0007) [2023-12-27 02:39:43,897][105692] Updated weights for policy 0, policy_version 1546800 (0.0005) [2023-12-27 02:39:43,944][105692] Updated weights for policy 0, policy_version 1546810 (0.0005) [2023-12-27 02:39:43,993][105692] Updated weights for policy 0, policy_version 1546820 (0.0008) [2023-12-27 02:39:44,246][105620] Updated weights for policy 1, policy_version 1550065 (0.0008) [2023-12-27 02:39:44,305][105620] Updated weights for policy 1, policy_version 1550075 (0.0008) [2023-12-27 02:39:44,361][105620] Updated weights for policy 1, policy_version 1550085 (0.0008) [2023-12-27 02:39:44,648][105692] Updated weights for policy 0, policy_version 1546830 (0.0011) [2023-12-27 02:39:44,704][105692] Updated weights for policy 0, policy_version 1546840 (0.0010) [2023-12-27 02:39:44,763][105692] Updated weights for policy 0, policy_version 1546850 (0.0011) [2023-12-27 02:39:45,094][105620] Updated weights for policy 1, policy_version 1550095 (0.0008) [2023-12-27 02:39:45,154][105620] Updated weights for policy 1, policy_version 1550105 (0.0008) [2023-12-27 02:39:45,212][105620] Updated weights for policy 1, policy_version 1550115 (0.0008) [2023-12-27 02:39:45,521][105692] Updated weights for policy 0, policy_version 1546860 (0.0011) [2023-12-27 02:39:45,587][105692] Updated weights for policy 0, policy_version 1546870 (0.0011) [2023-12-27 02:39:45,646][105692] Updated weights for policy 0, policy_version 1546880 (0.0011) [2023-12-27 02:39:45,999][105620] Updated weights for policy 1, policy_version 1550125 (0.0007) [2023-12-27 02:39:46,055][105620] Updated weights for policy 1, policy_version 1550135 (0.0008) [2023-12-27 02:39:46,062][104569] Fps is (10 sec: 18022.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 792944640. Throughput: 0: 9639.5, 1: 9929.2. Samples: 792918096. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:39:46,062][104569] Avg episode reward: [(0, '8541.640'), (1, '8991.737')] [2023-12-27 02:39:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001546888_396058624.pth... [2023-12-27 02:39:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001545768_395771904.pth [2023-12-27 02:39:46,120][105620] Updated weights for policy 1, policy_version 1550145 (0.0008) [2023-12-27 02:39:46,156][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001550152_396894208.pth... [2023-12-27 02:39:46,160][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001549000_396599296.pth [2023-12-27 02:39:46,388][105692] Updated weights for policy 0, policy_version 1546890 (0.0011) [2023-12-27 02:39:46,446][105692] Updated weights for policy 0, policy_version 1546900 (0.0010) [2023-12-27 02:39:46,504][105692] Updated weights for policy 0, policy_version 1546910 (0.0010) [2023-12-27 02:39:46,559][105692] Updated weights for policy 0, policy_version 1546920 (0.0010) [2023-12-27 02:39:46,853][105620] Updated weights for policy 1, policy_version 1550155 (0.0008) [2023-12-27 02:39:46,910][105620] Updated weights for policy 1, policy_version 1550165 (0.0008) [2023-12-27 02:39:46,959][105620] Updated weights for policy 1, policy_version 1550175 (0.0007) [2023-12-27 02:39:47,253][105692] Updated weights for policy 0, policy_version 1546930 (0.0005) [2023-12-27 02:39:47,309][105692] Updated weights for policy 0, policy_version 1546940 (0.0009) [2023-12-27 02:39:47,367][105692] Updated weights for policy 0, policy_version 1546950 (0.0010) [2023-12-27 02:39:47,828][105620] Updated weights for policy 1, policy_version 1550185 (0.0008) [2023-12-27 02:39:47,891][105620] Updated weights for policy 1, policy_version 1550195 (0.0008) [2023-12-27 02:39:47,947][105620] Updated weights for policy 1, policy_version 1550205 (0.0008) [2023-12-27 02:39:47,979][105692] Updated weights for policy 0, policy_version 1546960 (0.0009) [2023-12-27 02:39:47,993][105620] Updated weights for policy 1, policy_version 1550215 (0.0007) [2023-12-27 02:39:48,040][105692] Updated weights for policy 0, policy_version 1546970 (0.0010) [2023-12-27 02:39:48,105][105692] Updated weights for policy 0, policy_version 1546980 (0.0010) [2023-12-27 02:39:48,755][105620] Updated weights for policy 1, policy_version 1550225 (0.0006) [2023-12-27 02:39:48,824][105620] Updated weights for policy 1, policy_version 1550235 (0.0006) [2023-12-27 02:39:48,847][105692] Updated weights for policy 0, policy_version 1546990 (0.0010) [2023-12-27 02:39:48,877][105620] Updated weights for policy 1, policy_version 1550245 (0.0006) [2023-12-27 02:39:48,910][105692] Updated weights for policy 0, policy_version 1547000 (0.0011) [2023-12-27 02:39:48,982][105692] Updated weights for policy 0, policy_version 1547010 (0.0011) [2023-12-27 02:39:49,455][105620] Updated weights for policy 1, policy_version 1550255 (0.0007) [2023-12-27 02:39:49,511][105620] Updated weights for policy 1, policy_version 1550265 (0.0008) [2023-12-27 02:39:49,574][105620] Updated weights for policy 1, policy_version 1550275 (0.0008) [2023-12-27 02:39:49,718][105692] Updated weights for policy 0, policy_version 1547020 (0.0010) [2023-12-27 02:39:49,771][105692] Updated weights for policy 0, policy_version 1547030 (0.0010) [2023-12-27 02:39:49,830][105692] Updated weights for policy 0, policy_version 1547040 (0.0009) [2023-12-27 02:39:50,322][105620] Updated weights for policy 1, policy_version 1550285 (0.0008) [2023-12-27 02:39:50,380][105620] Updated weights for policy 1, policy_version 1550295 (0.0009) [2023-12-27 02:39:50,435][105620] Updated weights for policy 1, policy_version 1550305 (0.0009) [2023-12-27 02:39:50,577][105692] Updated weights for policy 0, policy_version 1547050 (0.0011) [2023-12-27 02:39:50,637][105692] Updated weights for policy 0, policy_version 1547060 (0.0008) [2023-12-27 02:39:50,698][105692] Updated weights for policy 0, policy_version 1547070 (0.0010) [2023-12-27 02:39:50,762][105692] Updated weights for policy 0, policy_version 1547080 (0.0008) [2023-12-27 02:39:51,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 793042944. Throughput: 0: 9672.3, 1: 9777.2. Samples: 793033648. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:39:51,062][104569] Avg episode reward: [(0, '8265.987'), (1, '8988.943')] [2023-12-27 02:39:51,256][105620] Updated weights for policy 1, policy_version 1550315 (0.0010) [2023-12-27 02:39:51,323][105620] Updated weights for policy 1, policy_version 1550325 (0.0009) [2023-12-27 02:39:51,384][105620] Updated weights for policy 1, policy_version 1550335 (0.0009) [2023-12-27 02:39:51,504][105692] Updated weights for policy 0, policy_version 1547090 (0.0009) [2023-12-27 02:39:51,564][105692] Updated weights for policy 0, policy_version 1547100 (0.0009) [2023-12-27 02:39:51,623][105692] Updated weights for policy 0, policy_version 1547110 (0.0009) [2023-12-27 02:39:52,136][105620] Updated weights for policy 1, policy_version 1550345 (0.0009) [2023-12-27 02:39:52,195][105620] Updated weights for policy 1, policy_version 1550355 (0.0008) [2023-12-27 02:39:52,259][105620] Updated weights for policy 1, policy_version 1550365 (0.0009) [2023-12-27 02:39:52,323][105620] Updated weights for policy 1, policy_version 1550375 (0.0009) [2023-12-27 02:39:52,413][105692] Updated weights for policy 0, policy_version 1547120 (0.0009) [2023-12-27 02:39:52,468][105692] Updated weights for policy 0, policy_version 1547130 (0.0009) [2023-12-27 02:39:52,532][105692] Updated weights for policy 0, policy_version 1547140 (0.0010) [2023-12-27 02:39:52,992][105620] Updated weights for policy 1, policy_version 1550385 (0.0009) [2023-12-27 02:39:53,047][105620] Updated weights for policy 1, policy_version 1550395 (0.0009) [2023-12-27 02:39:53,105][105620] Updated weights for policy 1, policy_version 1550405 (0.0009) [2023-12-27 02:39:53,352][105692] Updated weights for policy 0, policy_version 1547150 (0.0009) [2023-12-27 02:39:53,407][105692] Updated weights for policy 0, policy_version 1547160 (0.0009) [2023-12-27 02:39:53,465][105692] Updated weights for policy 0, policy_version 1547170 (0.0009) [2023-12-27 02:39:53,949][105620] Updated weights for policy 1, policy_version 1550415 (0.0009) [2023-12-27 02:39:54,007][105620] Updated weights for policy 1, policy_version 1550425 (0.0009) [2023-12-27 02:39:54,056][105692] Updated weights for policy 0, policy_version 1547180 (0.0007) [2023-12-27 02:39:54,073][105620] Updated weights for policy 1, policy_version 1550435 (0.0009) [2023-12-27 02:39:54,120][105692] Updated weights for policy 0, policy_version 1547190 (0.0005) [2023-12-27 02:39:54,182][105692] Updated weights for policy 0, policy_version 1547200 (0.0005) [2023-12-27 02:39:54,849][105692] Updated weights for policy 0, policy_version 1547210 (0.0008) [2023-12-27 02:39:54,862][105620] Updated weights for policy 1, policy_version 1550445 (0.0010) [2023-12-27 02:39:54,901][105692] Updated weights for policy 0, policy_version 1547220 (0.0008) [2023-12-27 02:39:54,916][105620] Updated weights for policy 1, policy_version 1550455 (0.0009) [2023-12-27 02:39:54,947][105692] Updated weights for policy 0, policy_version 1547230 (0.0007) [2023-12-27 02:39:54,967][105620] Updated weights for policy 1, policy_version 1550465 (0.0007) [2023-12-27 02:39:54,998][105692] Updated weights for policy 0, policy_version 1547240 (0.0005) [2023-12-27 02:39:55,587][105692] Updated weights for policy 0, policy_version 1547250 (0.0008) [2023-12-27 02:39:55,641][105692] Updated weights for policy 0, policy_version 1547260 (0.0008) [2023-12-27 02:39:55,692][105692] Updated weights for policy 0, policy_version 1547270 (0.0009) [2023-12-27 02:39:55,841][105620] Updated weights for policy 1, policy_version 1550475 (0.0009) [2023-12-27 02:39:55,903][105620] Updated weights for policy 1, policy_version 1550485 (0.0009) [2023-12-27 02:39:55,955][105620] Updated weights for policy 1, policy_version 1550495 (0.0009) [2023-12-27 02:39:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 793141248. Throughput: 0: 9661.2, 1: 9652.8. Samples: 793145800. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:39:56,063][104569] Avg episode reward: [(0, '7994.310'), (1, '9172.895')] [2023-12-27 02:39:56,344][105692] Updated weights for policy 0, policy_version 1547280 (0.0008) [2023-12-27 02:39:56,418][105692] Updated weights for policy 0, policy_version 1547290 (0.0009) [2023-12-27 02:39:56,484][105692] Updated weights for policy 0, policy_version 1547300 (0.0008) [2023-12-27 02:39:56,777][105620] Updated weights for policy 1, policy_version 1550505 (0.0009) [2023-12-27 02:39:56,827][105620] Updated weights for policy 1, policy_version 1550515 (0.0009) [2023-12-27 02:39:56,872][105620] Updated weights for policy 1, policy_version 1550525 (0.0008) [2023-12-27 02:39:56,922][105620] Updated weights for policy 1, policy_version 1550535 (0.0009) [2023-12-27 02:39:57,165][105692] Updated weights for policy 0, policy_version 1547310 (0.0007) [2023-12-27 02:39:57,225][105692] Updated weights for policy 0, policy_version 1547320 (0.0009) [2023-12-27 02:39:57,271][105692] Updated weights for policy 0, policy_version 1547330 (0.0005) [2023-12-27 02:39:57,710][105620] Updated weights for policy 1, policy_version 1550545 (0.0008) [2023-12-27 02:39:57,759][105620] Updated weights for policy 1, policy_version 1550555 (0.0009) [2023-12-27 02:39:57,809][105620] Updated weights for policy 1, policy_version 1550565 (0.0008) [2023-12-27 02:39:57,996][105692] Updated weights for policy 0, policy_version 1547340 (0.0008) [2023-12-27 02:39:58,043][105692] Updated weights for policy 0, policy_version 1547350 (0.0008) [2023-12-27 02:39:58,094][105692] Updated weights for policy 0, policy_version 1547360 (0.0009) [2023-12-27 02:39:58,632][105620] Updated weights for policy 1, policy_version 1550575 (0.0007) [2023-12-27 02:39:58,693][105620] Updated weights for policy 1, policy_version 1550585 (0.0006) [2023-12-27 02:39:58,765][105620] Updated weights for policy 1, policy_version 1550595 (0.0007) [2023-12-27 02:39:58,988][105692] Updated weights for policy 0, policy_version 1547370 (0.0009) [2023-12-27 02:39:59,046][105692] Updated weights for policy 0, policy_version 1547380 (0.0009) [2023-12-27 02:39:59,101][105692] Updated weights for policy 0, policy_version 1547390 (0.0009) [2023-12-27 02:39:59,160][105692] Updated weights for policy 0, policy_version 1547400 (0.0008) [2023-12-27 02:39:59,551][105620] Updated weights for policy 1, policy_version 1550605 (0.0008) [2023-12-27 02:39:59,623][105620] Updated weights for policy 1, policy_version 1550615 (0.0010) [2023-12-27 02:39:59,694][105620] Updated weights for policy 1, policy_version 1550625 (0.0010) [2023-12-27 02:39:59,803][105692] Updated weights for policy 0, policy_version 1547410 (0.0008) [2023-12-27 02:39:59,868][105692] Updated weights for policy 0, policy_version 1547420 (0.0008) [2023-12-27 02:39:59,934][105692] Updated weights for policy 0, policy_version 1547430 (0.0008) [2023-12-27 02:40:00,407][105620] Updated weights for policy 1, policy_version 1550635 (0.0010) [2023-12-27 02:40:00,465][105620] Updated weights for policy 1, policy_version 1550645 (0.0010) [2023-12-27 02:40:00,522][105620] Updated weights for policy 1, policy_version 1550655 (0.0010) [2023-12-27 02:40:00,669][105692] Updated weights for policy 0, policy_version 1547440 (0.0009) [2023-12-27 02:40:00,717][105692] Updated weights for policy 0, policy_version 1547450 (0.0009) [2023-12-27 02:40:00,764][105692] Updated weights for policy 0, policy_version 1547460 (0.0009) [2023-12-27 02:40:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 793231360. Throughput: 0: 9693.0, 1: 9608.0. Samples: 793201604. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:40:01,062][104569] Avg episode reward: [(0, '8624.750'), (1, '8993.376')] [2023-12-27 02:40:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001547464_396206080.pth... [2023-12-27 02:40:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001550664_397025280.pth... [2023-12-27 02:40:01,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001549576_396746752.pth [2023-12-27 02:40:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001546344_395919360.pth [2023-12-27 02:40:01,345][105620] Updated weights for policy 1, policy_version 1550665 (0.0009) [2023-12-27 02:40:01,409][105620] Updated weights for policy 1, policy_version 1550675 (0.0009) [2023-12-27 02:40:01,459][105620] Updated weights for policy 1, policy_version 1550685 (0.0008) [2023-12-27 02:40:01,473][105692] Updated weights for policy 0, policy_version 1547470 (0.0007) [2023-12-27 02:40:01,508][105620] Updated weights for policy 1, policy_version 1550695 (0.0006) [2023-12-27 02:40:01,531][105692] Updated weights for policy 0, policy_version 1547480 (0.0008) [2023-12-27 02:40:01,587][105692] Updated weights for policy 0, policy_version 1547491 (0.0010) [2023-12-27 02:40:02,219][105692] Updated weights for policy 0, policy_version 1547501 (0.0007) [2023-12-27 02:40:02,274][105692] Updated weights for policy 0, policy_version 1547511 (0.0005) [2023-12-27 02:40:02,328][105692] Updated weights for policy 0, policy_version 1547521 (0.0007) [2023-12-27 02:40:02,354][105620] Updated weights for policy 1, policy_version 1550705 (0.0008) [2023-12-27 02:40:02,413][105620] Updated weights for policy 1, policy_version 1550715 (0.0008) [2023-12-27 02:40:02,468][105620] Updated weights for policy 1, policy_version 1550725 (0.0009) [2023-12-27 02:40:02,916][105692] Updated weights for policy 0, policy_version 1547531 (0.0007) [2023-12-27 02:40:02,976][105692] Updated weights for policy 0, policy_version 1547541 (0.0005) [2023-12-27 02:40:03,044][105692] Updated weights for policy 0, policy_version 1547551 (0.0008) [2023-12-27 02:40:03,088][105620] Updated weights for policy 1, policy_version 1550735 (0.0008) [2023-12-27 02:40:03,155][105620] Updated weights for policy 1, policy_version 1550745 (0.0008) [2023-12-27 02:40:03,221][105620] Updated weights for policy 1, policy_version 1550755 (0.0008) [2023-12-27 02:40:03,733][105692] Updated weights for policy 0, policy_version 1547561 (0.0008) [2023-12-27 02:40:03,790][105692] Updated weights for policy 0, policy_version 1547571 (0.0009) [2023-12-27 02:40:03,849][105620] Updated weights for policy 1, policy_version 1550765 (0.0008) [2023-12-27 02:40:03,854][105692] Updated weights for policy 0, policy_version 1547581 (0.0008) [2023-12-27 02:40:03,909][105620] Updated weights for policy 1, policy_version 1550775 (0.0008) [2023-12-27 02:40:03,911][105692] Updated weights for policy 0, policy_version 1547591 (0.0007) [2023-12-27 02:40:03,969][105620] Updated weights for policy 1, policy_version 1550785 (0.0008) [2023-12-27 02:40:04,605][105692] Updated weights for policy 0, policy_version 1547601 (0.0005) [2023-12-27 02:40:04,650][105692] Updated weights for policy 0, policy_version 1547611 (0.0006) [2023-12-27 02:40:04,701][105692] Updated weights for policy 0, policy_version 1547621 (0.0010) [2023-12-27 02:40:04,762][105620] Updated weights for policy 1, policy_version 1550795 (0.0009) [2023-12-27 02:40:04,805][105620] Updated weights for policy 1, policy_version 1550805 (0.0006) [2023-12-27 02:40:04,851][105620] Updated weights for policy 1, policy_version 1550815 (0.0005) [2023-12-27 02:40:05,392][105692] Updated weights for policy 0, policy_version 1547631 (0.0010) [2023-12-27 02:40:05,437][105620] Updated weights for policy 1, policy_version 1550825 (0.0006) [2023-12-27 02:40:05,443][105692] Updated weights for policy 0, policy_version 1547641 (0.0010) [2023-12-27 02:40:05,493][105620] Updated weights for policy 1, policy_version 1550835 (0.0010) [2023-12-27 02:40:05,498][105692] Updated weights for policy 0, policy_version 1547651 (0.0010) [2023-12-27 02:40:05,545][105620] Updated weights for policy 1, policy_version 1550845 (0.0010) [2023-12-27 02:40:05,600][105620] Updated weights for policy 1, policy_version 1550855 (0.0010) [2023-12-27 02:40:06,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 793329664. Throughput: 0: 9746.5, 1: 9461.4. Samples: 793318896. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:40:06,062][104569] Avg episode reward: [(0, '8626.664'), (1, '9174.521')] [2023-12-27 02:40:06,132][105692] Updated weights for policy 0, policy_version 1547661 (0.0010) [2023-12-27 02:40:06,191][105692] Updated weights for policy 0, policy_version 1547671 (0.0010) [2023-12-27 02:40:06,250][105692] Updated weights for policy 0, policy_version 1547681 (0.0011) [2023-12-27 02:40:06,254][105620] Updated weights for policy 1, policy_version 1550865 (0.0011) [2023-12-27 02:40:06,317][105620] Updated weights for policy 1, policy_version 1550875 (0.0011) [2023-12-27 02:40:06,376][105620] Updated weights for policy 1, policy_version 1550885 (0.0011) [2023-12-27 02:40:06,973][105692] Updated weights for policy 0, policy_version 1547691 (0.0009) [2023-12-27 02:40:07,022][105692] Updated weights for policy 0, policy_version 1547701 (0.0005) [2023-12-27 02:40:07,080][105692] Updated weights for policy 0, policy_version 1547711 (0.0010) [2023-12-27 02:40:07,120][105620] Updated weights for policy 1, policy_version 1550895 (0.0010) [2023-12-27 02:40:07,167][105620] Updated weights for policy 1, policy_version 1550905 (0.0007) [2023-12-27 02:40:07,234][105620] Updated weights for policy 1, policy_version 1550915 (0.0005) [2023-12-27 02:40:07,649][105692] Updated weights for policy 0, policy_version 1547721 (0.0010) [2023-12-27 02:40:07,717][105692] Updated weights for policy 0, policy_version 1547731 (0.0008) [2023-12-27 02:40:07,773][105692] Updated weights for policy 0, policy_version 1547741 (0.0006) [2023-12-27 02:40:07,820][105692] Updated weights for policy 0, policy_version 1547751 (0.0005) [2023-12-27 02:40:07,961][105620] Updated weights for policy 1, policy_version 1550925 (0.0009) [2023-12-27 02:40:08,020][105620] Updated weights for policy 1, policy_version 1550935 (0.0010) [2023-12-27 02:40:08,066][105620] Updated weights for policy 1, policy_version 1550945 (0.0010) [2023-12-27 02:40:08,090][105586] KL-divergence is very high: 186.7185 [2023-12-27 02:40:08,458][105692] Updated weights for policy 0, policy_version 1547761 (0.0010) [2023-12-27 02:40:08,519][105692] Updated weights for policy 0, policy_version 1547771 (0.0011) [2023-12-27 02:40:08,576][105692] Updated weights for policy 0, policy_version 1547781 (0.0011) [2023-12-27 02:40:08,860][105620] Updated weights for policy 1, policy_version 1550955 (0.0010) [2023-12-27 02:40:08,908][105620] Updated weights for policy 1, policy_version 1550965 (0.0010) [2023-12-27 02:40:08,957][105620] Updated weights for policy 1, policy_version 1550975 (0.0010) [2023-12-27 02:40:09,309][105692] Updated weights for policy 0, policy_version 1547791 (0.0009) [2023-12-27 02:40:09,378][105692] Updated weights for policy 0, policy_version 1547801 (0.0010) [2023-12-27 02:40:09,442][105692] Updated weights for policy 0, policy_version 1547811 (0.0010) [2023-12-27 02:40:09,636][105620] Updated weights for policy 1, policy_version 1550985 (0.0010) [2023-12-27 02:40:09,703][105620] Updated weights for policy 1, policy_version 1550995 (0.0009) [2023-12-27 02:40:09,762][105620] Updated weights for policy 1, policy_version 1551005 (0.0008) [2023-12-27 02:40:09,829][105620] Updated weights for policy 1, policy_version 1551015 (0.0009) [2023-12-27 02:40:10,166][105692] Updated weights for policy 0, policy_version 1547821 (0.0010) [2023-12-27 02:40:10,220][105692] Updated weights for policy 0, policy_version 1547831 (0.0010) [2023-12-27 02:40:10,278][105692] Updated weights for policy 0, policy_version 1547841 (0.0010) [2023-12-27 02:40:10,558][105620] Updated weights for policy 1, policy_version 1551025 (0.0009) [2023-12-27 02:40:10,620][105620] Updated weights for policy 1, policy_version 1551035 (0.0010) [2023-12-27 02:40:10,631][105586] KL-divergence is very high: 147.2543 [2023-12-27 02:40:10,678][105586] KL-divergence is very high: 173.2751 [2023-12-27 02:40:10,679][105620] Updated weights for policy 1, policy_version 1551045 (0.0011) [2023-12-27 02:40:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 793427968. Throughput: 0: 9754.3, 1: 9538.9. Samples: 793439416. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:40:11,063][104569] Avg episode reward: [(0, '8811.483'), (1, '8994.223')] [2023-12-27 02:40:11,100][105692] Updated weights for policy 0, policy_version 1547851 (0.0009) [2023-12-27 02:40:11,159][105692] Updated weights for policy 0, policy_version 1547861 (0.0009) [2023-12-27 02:40:11,223][105692] Updated weights for policy 0, policy_version 1547871 (0.0009) [2023-12-27 02:40:11,395][105620] Updated weights for policy 1, policy_version 1551055 (0.0008) [2023-12-27 02:40:11,462][105620] Updated weights for policy 1, policy_version 1551065 (0.0009) [2023-12-27 02:40:11,523][105620] Updated weights for policy 1, policy_version 1551075 (0.0008) [2023-12-27 02:40:12,057][105692] Updated weights for policy 0, policy_version 1547881 (0.0009) [2023-12-27 02:40:12,120][105692] Updated weights for policy 0, policy_version 1547891 (0.0009) [2023-12-27 02:40:12,186][105692] Updated weights for policy 0, policy_version 1547901 (0.0009) [2023-12-27 02:40:12,218][105620] Updated weights for policy 1, policy_version 1551085 (0.0006) [2023-12-27 02:40:12,244][105692] Updated weights for policy 0, policy_version 1547911 (0.0006) [2023-12-27 02:40:12,283][105620] Updated weights for policy 1, policy_version 1551095 (0.0009) [2023-12-27 02:40:12,342][105620] Updated weights for policy 1, policy_version 1551105 (0.0009) [2023-12-27 02:40:12,983][105692] Updated weights for policy 0, policy_version 1547921 (0.0008) [2023-12-27 02:40:13,045][105692] Updated weights for policy 0, policy_version 1547931 (0.0009) [2023-12-27 02:40:13,103][105692] Updated weights for policy 0, policy_version 1547941 (0.0009) [2023-12-27 02:40:13,112][105620] Updated weights for policy 1, policy_version 1551115 (0.0008) [2023-12-27 02:40:13,177][105620] Updated weights for policy 1, policy_version 1551125 (0.0008) [2023-12-27 02:40:13,225][105620] Updated weights for policy 1, policy_version 1551135 (0.0007) [2023-12-27 02:40:13,809][105692] Updated weights for policy 0, policy_version 1547951 (0.0009) [2023-12-27 02:40:13,862][105692] Updated weights for policy 0, policy_version 1547961 (0.0009) [2023-12-27 02:40:13,916][105692] Updated weights for policy 0, policy_version 1547971 (0.0009) [2023-12-27 02:40:13,976][105620] Updated weights for policy 1, policy_version 1551145 (0.0008) [2023-12-27 02:40:14,023][105620] Updated weights for policy 1, policy_version 1551155 (0.0008) [2023-12-27 02:40:14,084][105620] Updated weights for policy 1, policy_version 1551165 (0.0007) [2023-12-27 02:40:14,151][105620] Updated weights for policy 1, policy_version 1551175 (0.0009) [2023-12-27 02:40:14,700][105692] Updated weights for policy 0, policy_version 1547981 (0.0009) [2023-12-27 02:40:14,750][105692] Updated weights for policy 0, policy_version 1547991 (0.0009) [2023-12-27 02:40:14,808][105692] Updated weights for policy 0, policy_version 1548001 (0.0009) [2023-12-27 02:40:14,892][105620] Updated weights for policy 1, policy_version 1551185 (0.0008) [2023-12-27 02:40:14,959][105620] Updated weights for policy 1, policy_version 1551195 (0.0009) [2023-12-27 02:40:15,023][105620] Updated weights for policy 1, policy_version 1551205 (0.0009) [2023-12-27 02:40:15,479][105692] Updated weights for policy 0, policy_version 1548011 (0.0009) [2023-12-27 02:40:15,540][105692] Updated weights for policy 0, policy_version 1548021 (0.0006) [2023-12-27 02:40:15,604][105692] Updated weights for policy 0, policy_version 1548031 (0.0009) [2023-12-27 02:40:15,822][105620] Updated weights for policy 1, policy_version 1551215 (0.0009) [2023-12-27 02:40:15,867][105620] Updated weights for policy 1, policy_version 1551225 (0.0008) [2023-12-27 02:40:15,913][105620] Updated weights for policy 1, policy_version 1551235 (0.0008) [2023-12-27 02:40:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 793526272. Throughput: 0: 9665.4, 1: 9485.0. Samples: 793494556. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:40:16,062][104569] Avg episode reward: [(0, '8988.627'), (1, '8717.845')] [2023-12-27 02:40:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001548040_396353536.pth... [2023-12-27 02:40:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001551240_397172736.pth... [2023-12-27 02:40:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001550152_396894208.pth [2023-12-27 02:40:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001546888_396058624.pth [2023-12-27 02:40:16,249][105692] Updated weights for policy 0, policy_version 1548041 (0.0008) [2023-12-27 02:40:16,316][105692] Updated weights for policy 0, policy_version 1548051 (0.0006) [2023-12-27 02:40:16,383][105692] Updated weights for policy 0, policy_version 1548061 (0.0006) [2023-12-27 02:40:16,444][105692] Updated weights for policy 0, policy_version 1548071 (0.0010) [2023-12-27 02:40:16,747][105620] Updated weights for policy 1, policy_version 1551245 (0.0009) [2023-12-27 02:40:16,804][105620] Updated weights for policy 1, policy_version 1551255 (0.0010) [2023-12-27 02:40:16,858][105620] Updated weights for policy 1, policy_version 1551265 (0.0009) [2023-12-27 02:40:17,026][105692] Updated weights for policy 0, policy_version 1548081 (0.0007) [2023-12-27 02:40:17,084][105692] Updated weights for policy 0, policy_version 1548091 (0.0011) [2023-12-27 02:40:17,142][105692] Updated weights for policy 0, policy_version 1548101 (0.0010) [2023-12-27 02:40:17,649][105620] Updated weights for policy 1, policy_version 1551276 (0.0009) [2023-12-27 02:40:17,701][105620] Updated weights for policy 1, policy_version 1551286 (0.0008) [2023-12-27 02:40:17,759][105620] Updated weights for policy 1, policy_version 1551297 (0.0009) [2023-12-27 02:40:17,780][105692] Updated weights for policy 0, policy_version 1548111 (0.0007) [2023-12-27 02:40:17,833][105692] Updated weights for policy 0, policy_version 1548121 (0.0008) [2023-12-27 02:40:17,887][105692] Updated weights for policy 0, policy_version 1548131 (0.0005) [2023-12-27 02:40:18,434][105692] Updated weights for policy 0, policy_version 1548141 (0.0006) [2023-12-27 02:40:18,483][105692] Updated weights for policy 0, policy_version 1548151 (0.0005) [2023-12-27 02:40:18,537][105692] Updated weights for policy 0, policy_version 1548161 (0.0005) [2023-12-27 02:40:18,573][105620] Updated weights for policy 1, policy_version 1551307 (0.0007) [2023-12-27 02:40:18,627][105620] Updated weights for policy 1, policy_version 1551318 (0.0010) [2023-12-27 02:40:18,686][105620] Updated weights for policy 1, policy_version 1551328 (0.0010) [2023-12-27 02:40:19,118][105692] Updated weights for policy 0, policy_version 1548171 (0.0006) [2023-12-27 02:40:19,170][105692] Updated weights for policy 0, policy_version 1548181 (0.0005) [2023-12-27 02:40:19,230][105692] Updated weights for policy 0, policy_version 1548191 (0.0006) [2023-12-27 02:40:19,564][105620] Updated weights for policy 1, policy_version 1551338 (0.0010) [2023-12-27 02:40:19,618][105620] Updated weights for policy 1, policy_version 1551348 (0.0008) [2023-12-27 02:40:19,675][105620] Updated weights for policy 1, policy_version 1551358 (0.0009) [2023-12-27 02:40:19,731][105620] Updated weights for policy 1, policy_version 1551368 (0.0010) [2023-12-27 02:40:19,870][105692] Updated weights for policy 0, policy_version 1548201 (0.0011) [2023-12-27 02:40:19,937][105692] Updated weights for policy 0, policy_version 1548211 (0.0009) [2023-12-27 02:40:20,002][105692] Updated weights for policy 0, policy_version 1548221 (0.0009) [2023-12-27 02:40:20,070][105692] Updated weights for policy 0, policy_version 1548231 (0.0011) [2023-12-27 02:40:20,488][105620] Updated weights for policy 1, policy_version 1551378 (0.0006) [2023-12-27 02:40:20,524][105586] KL-divergence is very high: 142.8751 [2023-12-27 02:40:20,546][105620] Updated weights for policy 1, policy_version 1551388 (0.0006) [2023-12-27 02:40:20,573][105586] KL-divergence is very high: 152.0445 [2023-12-27 02:40:20,610][105620] Updated weights for policy 1, policy_version 1551398 (0.0009) [2023-12-27 02:40:20,830][105692] Updated weights for policy 0, policy_version 1548241 (0.0009) [2023-12-27 02:40:20,888][105692] Updated weights for policy 0, policy_version 1548251 (0.0005) [2023-12-27 02:40:20,940][105692] Updated weights for policy 0, policy_version 1548261 (0.0008) [2023-12-27 02:40:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 793624576. Throughput: 0: 9843.5, 1: 9365.1. Samples: 793611196. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:40:21,063][104569] Avg episode reward: [(0, '8626.870'), (1, '8802.315')] [2023-12-27 02:40:21,366][105620] Updated weights for policy 1, policy_version 1551408 (0.0010) [2023-12-27 02:40:21,434][105620] Updated weights for policy 1, policy_version 1551418 (0.0011) [2023-12-27 02:40:21,493][105620] Updated weights for policy 1, policy_version 1551428 (0.0010) [2023-12-27 02:40:21,684][105692] Updated weights for policy 0, policy_version 1548271 (0.0009) [2023-12-27 02:40:21,756][105692] Updated weights for policy 0, policy_version 1548281 (0.0008) [2023-12-27 02:40:21,820][105692] Updated weights for policy 0, policy_version 1548291 (0.0010) [2023-12-27 02:40:22,145][105620] Updated weights for policy 1, policy_version 1551438 (0.0008) [2023-12-27 02:40:22,208][105620] Updated weights for policy 1, policy_version 1551448 (0.0005) [2023-12-27 02:40:22,278][105620] Updated weights for policy 1, policy_version 1551458 (0.0007) [2023-12-27 02:40:22,588][105692] Updated weights for policy 0, policy_version 1548301 (0.0011) [2023-12-27 02:40:22,655][105692] Updated weights for policy 0, policy_version 1548311 (0.0010) [2023-12-27 02:40:22,714][105692] Updated weights for policy 0, policy_version 1548321 (0.0011) [2023-12-27 02:40:22,986][105620] Updated weights for policy 1, policy_version 1551468 (0.0009) [2023-12-27 02:40:23,035][105620] Updated weights for policy 1, policy_version 1551478 (0.0010) [2023-12-27 02:40:23,098][105620] Updated weights for policy 1, policy_version 1551488 (0.0011) [2023-12-27 02:40:23,414][105692] Updated weights for policy 0, policy_version 1548331 (0.0011) [2023-12-27 02:40:23,482][105692] Updated weights for policy 0, policy_version 1548341 (0.0007) [2023-12-27 02:40:23,539][105692] Updated weights for policy 0, policy_version 1548351 (0.0005) [2023-12-27 02:40:23,848][105620] Updated weights for policy 1, policy_version 1551498 (0.0011) [2023-12-27 02:40:23,907][105620] Updated weights for policy 1, policy_version 1551508 (0.0011) [2023-12-27 02:40:23,970][105620] Updated weights for policy 1, policy_version 1551518 (0.0011) [2023-12-27 02:40:24,034][105620] Updated weights for policy 1, policy_version 1551528 (0.0011) [2023-12-27 02:40:24,180][105692] Updated weights for policy 0, policy_version 1548361 (0.0006) [2023-12-27 02:40:24,241][105692] Updated weights for policy 0, policy_version 1548371 (0.0011) [2023-12-27 02:40:24,308][105692] Updated weights for policy 0, policy_version 1548381 (0.0011) [2023-12-27 02:40:24,371][105692] Updated weights for policy 0, policy_version 1548391 (0.0011) [2023-12-27 02:40:24,778][105620] Updated weights for policy 1, policy_version 1551538 (0.0010) [2023-12-27 02:40:24,836][105620] Updated weights for policy 1, policy_version 1551548 (0.0010) [2023-12-27 02:40:24,894][105620] Updated weights for policy 1, policy_version 1551558 (0.0010) [2023-12-27 02:40:25,097][105692] Updated weights for policy 0, policy_version 1548401 (0.0011) [2023-12-27 02:40:25,155][105692] Updated weights for policy 0, policy_version 1548411 (0.0010) [2023-12-27 02:40:25,214][105692] Updated weights for policy 0, policy_version 1548421 (0.0010) [2023-12-27 02:40:25,536][105620] Updated weights for policy 1, policy_version 1551568 (0.0006) [2023-12-27 02:40:25,594][105620] Updated weights for policy 1, policy_version 1551578 (0.0006) [2023-12-27 02:40:25,649][105620] Updated weights for policy 1, policy_version 1551588 (0.0010) [2023-12-27 02:40:25,844][105692] Updated weights for policy 0, policy_version 1548431 (0.0007) [2023-12-27 02:40:25,894][105692] Updated weights for policy 0, policy_version 1548441 (0.0006) [2023-12-27 02:40:25,955][105692] Updated weights for policy 0, policy_version 1548451 (0.0005) [2023-12-27 02:40:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 793722880. Throughput: 0: 9963.4, 1: 9273.6. Samples: 793727928. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:40:26,062][104569] Avg episode reward: [(0, '8444.660'), (1, '8987.555')] [2023-12-27 02:40:26,313][105620] Updated weights for policy 1, policy_version 1551598 (0.0010) [2023-12-27 02:40:26,370][105620] Updated weights for policy 1, policy_version 1551608 (0.0010) [2023-12-27 02:40:26,432][105620] Updated weights for policy 1, policy_version 1551618 (0.0010) [2023-12-27 02:40:26,514][105692] Updated weights for policy 0, policy_version 1548461 (0.0008) [2023-12-27 02:40:26,563][105692] Updated weights for policy 0, policy_version 1548471 (0.0008) [2023-12-27 02:40:26,608][105692] Updated weights for policy 0, policy_version 1548481 (0.0008) [2023-12-27 02:40:27,125][105620] Updated weights for policy 1, policy_version 1551628 (0.0010) [2023-12-27 02:40:27,176][105620] Updated weights for policy 1, policy_version 1551638 (0.0007) [2023-12-27 02:40:27,236][105620] Updated weights for policy 1, policy_version 1551648 (0.0006) [2023-12-27 02:40:27,460][105692] Updated weights for policy 0, policy_version 1548491 (0.0008) [2023-12-27 02:40:27,529][105692] Updated weights for policy 0, policy_version 1548501 (0.0009) [2023-12-27 02:40:27,593][105692] Updated weights for policy 0, policy_version 1548511 (0.0010) [2023-12-27 02:40:27,776][105620] Updated weights for policy 1, policy_version 1551658 (0.0007) [2023-12-27 02:40:27,841][105620] Updated weights for policy 1, policy_version 1551668 (0.0010) [2023-12-27 02:40:27,905][105620] Updated weights for policy 1, policy_version 1551678 (0.0010) [2023-12-27 02:40:27,962][105620] Updated weights for policy 1, policy_version 1551688 (0.0010) [2023-12-27 02:40:28,296][105692] Updated weights for policy 0, policy_version 1548521 (0.0009) [2023-12-27 02:40:28,359][105692] Updated weights for policy 0, policy_version 1548531 (0.0008) [2023-12-27 02:40:28,410][105692] Updated weights for policy 0, policy_version 1548541 (0.0007) [2023-12-27 02:40:28,462][105692] Updated weights for policy 0, policy_version 1548551 (0.0008) [2023-12-27 02:40:28,685][105620] Updated weights for policy 1, policy_version 1551698 (0.0011) [2023-12-27 02:40:28,751][105620] Updated weights for policy 1, policy_version 1551708 (0.0010) [2023-12-27 02:40:28,805][105620] Updated weights for policy 1, policy_version 1551718 (0.0010) [2023-12-27 02:40:29,158][105692] Updated weights for policy 0, policy_version 1548561 (0.0007) [2023-12-27 02:40:29,201][105692] Updated weights for policy 0, policy_version 1548571 (0.0007) [2023-12-27 02:40:29,265][105692] Updated weights for policy 0, policy_version 1548581 (0.0008) [2023-12-27 02:40:29,503][105620] Updated weights for policy 1, policy_version 1551728 (0.0008) [2023-12-27 02:40:29,555][105620] Updated weights for policy 1, policy_version 1551738 (0.0005) [2023-12-27 02:40:29,609][105620] Updated weights for policy 1, policy_version 1551748 (0.0005) [2023-12-27 02:40:29,967][105692] Updated weights for policy 0, policy_version 1548591 (0.0007) [2023-12-27 02:40:30,030][105692] Updated weights for policy 0, policy_version 1548601 (0.0008) [2023-12-27 02:40:30,096][105692] Updated weights for policy 0, policy_version 1548611 (0.0008) [2023-12-27 02:40:30,316][105620] Updated weights for policy 1, policy_version 1551758 (0.0009) [2023-12-27 02:40:30,368][105620] Updated weights for policy 1, policy_version 1551768 (0.0010) [2023-12-27 02:40:30,425][105620] Updated weights for policy 1, policy_version 1551778 (0.0009) [2023-12-27 02:40:30,774][105692] Updated weights for policy 0, policy_version 1548621 (0.0008) [2023-12-27 02:40:30,826][105692] Updated weights for policy 0, policy_version 1548632 (0.0010) [2023-12-27 02:40:30,880][105692] Updated weights for policy 0, policy_version 1548643 (0.0010) [2023-12-27 02:40:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.9, 300 sec: 19466.4). Total num frames: 793821184. Throughput: 0: 9973.8, 1: 9384.9. Samples: 793789236. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:40:31,062][104569] Avg episode reward: [(0, '8533.965'), (1, '9174.211')] [2023-12-27 02:40:31,063][105620] Updated weights for policy 1, policy_version 1551788 (0.0006) [2023-12-27 02:40:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001548648_396509184.pth... [2023-12-27 02:40:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001547464_396206080.pth [2023-12-27 02:40:31,126][105620] Updated weights for policy 1, policy_version 1551798 (0.0010) [2023-12-27 02:40:31,195][105620] Updated weights for policy 1, policy_version 1551808 (0.0008) [2023-12-27 02:40:31,244][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001551816_397320192.pth... [2023-12-27 02:40:31,248][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001550664_397025280.pth [2023-12-27 02:40:31,662][105692] Updated weights for policy 0, policy_version 1548653 (0.0008) [2023-12-27 02:40:31,711][105692] Updated weights for policy 0, policy_version 1548663 (0.0006) [2023-12-27 02:40:31,771][105692] Updated weights for policy 0, policy_version 1548673 (0.0008) [2023-12-27 02:40:31,908][105620] Updated weights for policy 1, policy_version 1551818 (0.0011) [2023-12-27 02:40:31,968][105620] Updated weights for policy 1, policy_version 1551828 (0.0006) [2023-12-27 02:40:32,026][105620] Updated weights for policy 1, policy_version 1551838 (0.0008) [2023-12-27 02:40:32,080][105620] Updated weights for policy 1, policy_version 1551848 (0.0010) [2023-12-27 02:40:32,521][105692] Updated weights for policy 0, policy_version 1548683 (0.0008) [2023-12-27 02:40:32,573][105692] Updated weights for policy 0, policy_version 1548693 (0.0008) [2023-12-27 02:40:32,629][105692] Updated weights for policy 0, policy_version 1548703 (0.0008) [2023-12-27 02:40:32,783][105620] Updated weights for policy 1, policy_version 1551858 (0.0011) [2023-12-27 02:40:32,848][105620] Updated weights for policy 1, policy_version 1551868 (0.0010) [2023-12-27 02:40:32,916][105620] Updated weights for policy 1, policy_version 1551878 (0.0010) [2023-12-27 02:40:33,197][105692] Updated weights for policy 0, policy_version 1548713 (0.0007) [2023-12-27 02:40:33,254][105692] Updated weights for policy 0, policy_version 1548723 (0.0007) [2023-12-27 02:40:33,299][105692] Updated weights for policy 0, policy_version 1548733 (0.0006) [2023-12-27 02:40:33,346][105692] Updated weights for policy 0, policy_version 1548743 (0.0006) [2023-12-27 02:40:33,626][105620] Updated weights for policy 1, policy_version 1551888 (0.0011) [2023-12-27 02:40:33,680][105620] Updated weights for policy 1, policy_version 1551898 (0.0010) [2023-12-27 02:40:33,735][105620] Updated weights for policy 1, policy_version 1551908 (0.0010) [2023-12-27 02:40:33,943][105692] Updated weights for policy 0, policy_version 1548753 (0.0005) [2023-12-27 02:40:33,997][105692] Updated weights for policy 0, policy_version 1548763 (0.0005) [2023-12-27 02:40:34,045][105692] Updated weights for policy 0, policy_version 1548773 (0.0005) [2023-12-27 02:40:34,409][105620] Updated weights for policy 1, policy_version 1551918 (0.0011) [2023-12-27 02:40:34,458][105620] Updated weights for policy 1, policy_version 1551928 (0.0011) [2023-12-27 02:40:34,515][105620] Updated weights for policy 1, policy_version 1551938 (0.0011) [2023-12-27 02:40:34,683][105692] Updated weights for policy 0, policy_version 1548783 (0.0009) [2023-12-27 02:40:34,743][105692] Updated weights for policy 0, policy_version 1548793 (0.0010) [2023-12-27 02:40:34,795][105692] Updated weights for policy 0, policy_version 1548803 (0.0010) [2023-12-27 02:40:35,159][105620] Updated weights for policy 1, policy_version 1551948 (0.0009) [2023-12-27 02:40:35,217][105620] Updated weights for policy 1, policy_version 1551958 (0.0010) [2023-12-27 02:40:35,262][105620] Updated weights for policy 1, policy_version 1551968 (0.0010) [2023-12-27 02:40:35,555][105692] Updated weights for policy 0, policy_version 1548813 (0.0010) [2023-12-27 02:40:35,606][105692] Updated weights for policy 0, policy_version 1548823 (0.0010) [2023-12-27 02:40:35,657][105692] Updated weights for policy 0, policy_version 1548833 (0.0010) [2023-12-27 02:40:35,910][105620] Updated weights for policy 1, policy_version 1551978 (0.0007) [2023-12-27 02:40:35,959][105620] Updated weights for policy 1, policy_version 1551988 (0.0006) [2023-12-27 02:40:36,006][105620] Updated weights for policy 1, policy_version 1551998 (0.0005) [2023-12-27 02:40:36,052][105620] Updated weights for policy 1, policy_version 1552008 (0.0007) [2023-12-27 02:40:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 793927680. Throughput: 0: 10032.6, 1: 9476.7. Samples: 793911568. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:40:36,062][104569] Avg episode reward: [(0, '8534.560'), (1, '9085.111')] [2023-12-27 02:40:36,342][105692] Updated weights for policy 0, policy_version 1548843 (0.0007) [2023-12-27 02:40:36,404][105692] Updated weights for policy 0, policy_version 1548853 (0.0010) [2023-12-27 02:40:36,453][105692] Updated weights for policy 0, policy_version 1548863 (0.0010) [2023-12-27 02:40:36,794][105620] Updated weights for policy 1, policy_version 1552018 (0.0010) [2023-12-27 02:40:36,845][105620] Updated weights for policy 1, policy_version 1552028 (0.0010) [2023-12-27 02:40:36,903][105620] Updated weights for policy 1, policy_version 1552038 (0.0010) [2023-12-27 02:40:37,163][105692] Updated weights for policy 0, policy_version 1548873 (0.0010) [2023-12-27 02:40:37,211][105692] Updated weights for policy 0, policy_version 1548883 (0.0010) [2023-12-27 02:40:37,263][105692] Updated weights for policy 0, policy_version 1548893 (0.0010) [2023-12-27 02:40:37,307][105692] Updated weights for policy 0, policy_version 1548903 (0.0010) [2023-12-27 02:40:37,613][105620] Updated weights for policy 1, policy_version 1552048 (0.0007) [2023-12-27 02:40:37,663][105620] Updated weights for policy 1, policy_version 1552058 (0.0010) [2023-12-27 02:40:37,711][105620] Updated weights for policy 1, policy_version 1552068 (0.0010) [2023-12-27 02:40:38,035][105692] Updated weights for policy 0, policy_version 1548913 (0.0010) [2023-12-27 02:40:38,089][105692] Updated weights for policy 0, policy_version 1548923 (0.0010) [2023-12-27 02:40:38,157][105692] Updated weights for policy 0, policy_version 1548933 (0.0010) [2023-12-27 02:40:38,442][105620] Updated weights for policy 1, policy_version 1552078 (0.0008) [2023-12-27 02:40:38,498][105620] Updated weights for policy 1, policy_version 1552088 (0.0005) [2023-12-27 02:40:38,556][105620] Updated weights for policy 1, policy_version 1552098 (0.0009) [2023-12-27 02:40:38,948][105692] Updated weights for policy 0, policy_version 1548943 (0.0009) [2023-12-27 02:40:39,009][105692] Updated weights for policy 0, policy_version 1548953 (0.0008) [2023-12-27 02:40:39,065][105692] Updated weights for policy 0, policy_version 1548963 (0.0008) [2023-12-27 02:40:39,238][105620] Updated weights for policy 1, policy_version 1552108 (0.0011) [2023-12-27 02:40:39,299][105620] Updated weights for policy 1, policy_version 1552118 (0.0011) [2023-12-27 02:40:39,354][105620] Updated weights for policy 1, policy_version 1552128 (0.0009) [2023-12-27 02:40:39,853][105692] Updated weights for policy 0, policy_version 1548973 (0.0009) [2023-12-27 02:40:39,921][105692] Updated weights for policy 0, policy_version 1548983 (0.0010) [2023-12-27 02:40:39,992][105692] Updated weights for policy 0, policy_version 1548993 (0.0008) [2023-12-27 02:40:40,112][105620] Updated weights for policy 1, policy_version 1552138 (0.0009) [2023-12-27 02:40:40,177][105620] Updated weights for policy 1, policy_version 1552148 (0.0008) [2023-12-27 02:40:40,237][105620] Updated weights for policy 1, policy_version 1552158 (0.0009) [2023-12-27 02:40:40,294][105620] Updated weights for policy 1, policy_version 1552168 (0.0009) [2023-12-27 02:40:40,733][105692] Updated weights for policy 0, policy_version 1549003 (0.0008) [2023-12-27 02:40:40,792][105692] Updated weights for policy 0, policy_version 1549013 (0.0008) [2023-12-27 02:40:40,848][105692] Updated weights for policy 0, policy_version 1549023 (0.0009) [2023-12-27 02:40:41,037][105620] Updated weights for policy 1, policy_version 1552178 (0.0007) [2023-12-27 02:40:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 794017792. Throughput: 0: 9974.5, 1: 9613.5. Samples: 794027260. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:40:41,063][104569] Avg episode reward: [(0, '8445.601'), (1, '8815.388')] [2023-12-27 02:40:41,106][105620] Updated weights for policy 1, policy_version 1552188 (0.0010) [2023-12-27 02:40:41,172][105620] Updated weights for policy 1, policy_version 1552198 (0.0007) [2023-12-27 02:40:41,619][105692] Updated weights for policy 0, policy_version 1549033 (0.0009) [2023-12-27 02:40:41,686][105692] Updated weights for policy 0, policy_version 1549043 (0.0009) [2023-12-27 02:40:41,756][105692] Updated weights for policy 0, policy_version 1549053 (0.0008) [2023-12-27 02:40:41,813][105692] Updated weights for policy 0, policy_version 1549063 (0.0008) [2023-12-27 02:40:41,978][105620] Updated weights for policy 1, policy_version 1552208 (0.0011) [2023-12-27 02:40:42,035][105620] Updated weights for policy 1, policy_version 1552218 (0.0011) [2023-12-27 02:40:42,102][105620] Updated weights for policy 1, policy_version 1552228 (0.0011) [2023-12-27 02:40:42,558][105692] Updated weights for policy 0, policy_version 1549073 (0.0006) [2023-12-27 02:40:42,611][105692] Updated weights for policy 0, policy_version 1549083 (0.0008) [2023-12-27 02:40:42,662][105692] Updated weights for policy 0, policy_version 1549093 (0.0005) [2023-12-27 02:40:42,822][105620] Updated weights for policy 1, policy_version 1552238 (0.0008) [2023-12-27 02:40:42,880][105620] Updated weights for policy 1, policy_version 1552248 (0.0006) [2023-12-27 02:40:42,933][105620] Updated weights for policy 1, policy_version 1552258 (0.0005) [2023-12-27 02:40:43,344][105692] Updated weights for policy 0, policy_version 1549103 (0.0006) [2023-12-27 02:40:43,406][105692] Updated weights for policy 0, policy_version 1549113 (0.0008) [2023-12-27 02:40:43,464][105692] Updated weights for policy 0, policy_version 1549123 (0.0007) [2023-12-27 02:40:43,487][105620] Updated weights for policy 1, policy_version 1552268 (0.0007) [2023-12-27 02:40:43,542][105620] Updated weights for policy 1, policy_version 1552278 (0.0011) [2023-12-27 02:40:43,586][105620] Updated weights for policy 1, policy_version 1552288 (0.0010) [2023-12-27 02:40:44,014][105692] Updated weights for policy 0, policy_version 1549133 (0.0005) [2023-12-27 02:40:44,077][105692] Updated weights for policy 0, policy_version 1549143 (0.0008) [2023-12-27 02:40:44,146][105692] Updated weights for policy 0, policy_version 1549153 (0.0008) [2023-12-27 02:40:44,342][105620] Updated weights for policy 1, policy_version 1552298 (0.0008) [2023-12-27 02:40:44,394][105620] Updated weights for policy 1, policy_version 1552308 (0.0011) [2023-12-27 02:40:44,460][105620] Updated weights for policy 1, policy_version 1552318 (0.0011) [2023-12-27 02:40:44,525][105620] Updated weights for policy 1, policy_version 1552328 (0.0010) [2023-12-27 02:40:44,684][105692] Updated weights for policy 0, policy_version 1549163 (0.0008) [2023-12-27 02:40:44,728][105692] Updated weights for policy 0, policy_version 1549173 (0.0005) [2023-12-27 02:40:44,782][105692] Updated weights for policy 0, policy_version 1549183 (0.0007) [2023-12-27 02:40:45,281][105620] Updated weights for policy 1, policy_version 1552338 (0.0007) [2023-12-27 02:40:45,342][105620] Updated weights for policy 1, policy_version 1552348 (0.0008) [2023-12-27 02:40:45,400][105620] Updated weights for policy 1, policy_version 1552358 (0.0009) [2023-12-27 02:40:45,513][105692] Updated weights for policy 0, policy_version 1549193 (0.0008) [2023-12-27 02:40:45,578][105692] Updated weights for policy 0, policy_version 1549203 (0.0010) [2023-12-27 02:40:45,633][105692] Updated weights for policy 0, policy_version 1549213 (0.0010) [2023-12-27 02:40:45,691][105692] Updated weights for policy 0, policy_version 1549223 (0.0010) [2023-12-27 02:40:46,021][105620] Updated weights for policy 1, policy_version 1552368 (0.0006) [2023-12-27 02:40:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 794116096. Throughput: 0: 9963.4, 1: 9679.6. Samples: 794085540. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-12-27 02:40:46,062][104569] Avg episode reward: [(0, '8262.229'), (1, '8817.897')] [2023-12-27 02:40:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001549224_396656640.pth... [2023-12-27 02:40:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001548040_396353536.pth [2023-12-27 02:40:46,074][105620] Updated weights for policy 1, policy_version 1552378 (0.0008) [2023-12-27 02:40:46,130][105620] Updated weights for policy 1, policy_version 1552388 (0.0010) [2023-12-27 02:40:46,147][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001552392_397467648.pth... [2023-12-27 02:40:46,150][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001551240_397172736.pth [2023-12-27 02:40:46,426][105692] Updated weights for policy 0, policy_version 1549233 (0.0010) [2023-12-27 02:40:46,484][105692] Updated weights for policy 0, policy_version 1549243 (0.0010) [2023-12-27 02:40:46,542][105692] Updated weights for policy 0, policy_version 1549253 (0.0010) [2023-12-27 02:40:46,762][105620] Updated weights for policy 1, policy_version 1552398 (0.0011) [2023-12-27 02:40:46,820][105620] Updated weights for policy 1, policy_version 1552408 (0.0010) [2023-12-27 02:40:46,875][105620] Updated weights for policy 1, policy_version 1552418 (0.0010) [2023-12-27 02:40:47,215][105692] Updated weights for policy 0, policy_version 1549263 (0.0010) [2023-12-27 02:40:47,270][105692] Updated weights for policy 0, policy_version 1549273 (0.0009) [2023-12-27 02:40:47,321][105692] Updated weights for policy 0, policy_version 1549283 (0.0005) [2023-12-27 02:40:47,549][105620] Updated weights for policy 1, policy_version 1552428 (0.0010) [2023-12-27 02:40:47,601][105620] Updated weights for policy 1, policy_version 1552438 (0.0010) [2023-12-27 02:40:47,660][105620] Updated weights for policy 1, policy_version 1552448 (0.0010) [2023-12-27 02:40:47,891][105692] Updated weights for policy 0, policy_version 1549293 (0.0005) [2023-12-27 02:40:47,952][105692] Updated weights for policy 0, policy_version 1549303 (0.0006) [2023-12-27 02:40:48,011][105692] Updated weights for policy 0, policy_version 1549313 (0.0009) [2023-12-27 02:40:48,405][105620] Updated weights for policy 1, policy_version 1552458 (0.0010) [2023-12-27 02:40:48,461][105620] Updated weights for policy 1, policy_version 1552468 (0.0010) [2023-12-27 02:40:48,506][105620] Updated weights for policy 1, policy_version 1552478 (0.0010) [2023-12-27 02:40:48,565][105620] Updated weights for policy 1, policy_version 1552488 (0.0010) [2023-12-27 02:40:48,695][105692] Updated weights for policy 0, policy_version 1549323 (0.0011) [2023-12-27 02:40:48,754][105692] Updated weights for policy 0, policy_version 1549333 (0.0011) [2023-12-27 02:40:48,812][105692] Updated weights for policy 0, policy_version 1549343 (0.0010) [2023-12-27 02:40:49,232][105620] Updated weights for policy 1, policy_version 1552498 (0.0007) [2023-12-27 02:40:49,291][105620] Updated weights for policy 1, policy_version 1552508 (0.0009) [2023-12-27 02:40:49,357][105620] Updated weights for policy 1, policy_version 1552518 (0.0010) [2023-12-27 02:40:49,562][105692] Updated weights for policy 0, policy_version 1549353 (0.0010) [2023-12-27 02:40:49,619][105692] Updated weights for policy 0, policy_version 1549363 (0.0008) [2023-12-27 02:40:49,667][105692] Updated weights for policy 0, policy_version 1549373 (0.0008) [2023-12-27 02:40:49,716][105692] Updated weights for policy 0, policy_version 1549383 (0.0008) [2023-12-27 02:40:50,084][105620] Updated weights for policy 1, policy_version 1552528 (0.0008) [2023-12-27 02:40:50,140][105620] Updated weights for policy 1, policy_version 1552538 (0.0008) [2023-12-27 02:40:50,186][105620] Updated weights for policy 1, policy_version 1552548 (0.0010) [2023-12-27 02:40:50,487][105692] Updated weights for policy 0, policy_version 1549393 (0.0008) [2023-12-27 02:40:50,547][105692] Updated weights for policy 0, policy_version 1549403 (0.0008) [2023-12-27 02:40:50,611][105692] Updated weights for policy 0, policy_version 1549413 (0.0008) [2023-12-27 02:40:51,008][105620] Updated weights for policy 1, policy_version 1552558 (0.0008) [2023-12-27 02:40:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 794214400. Throughput: 0: 10007.3, 1: 9765.9. Samples: 794208688. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:40:51,062][104569] Avg episode reward: [(0, '8256.615'), (1, '9083.628')] [2023-12-27 02:40:51,072][105620] Updated weights for policy 1, policy_version 1552568 (0.0008) [2023-12-27 02:40:51,137][105620] Updated weights for policy 1, policy_version 1552578 (0.0009) [2023-12-27 02:40:51,396][105692] Updated weights for policy 0, policy_version 1549423 (0.0009) [2023-12-27 02:40:51,459][105692] Updated weights for policy 0, policy_version 1549433 (0.0008) [2023-12-27 02:40:51,519][105692] Updated weights for policy 0, policy_version 1549443 (0.0007) [2023-12-27 02:40:51,826][105620] Updated weights for policy 1, policy_version 1552588 (0.0006) [2023-12-27 02:40:51,885][105620] Updated weights for policy 1, policy_version 1552598 (0.0009) [2023-12-27 02:40:51,943][105620] Updated weights for policy 1, policy_version 1552608 (0.0009) [2023-12-27 02:40:52,266][105692] Updated weights for policy 0, policy_version 1549453 (0.0006) [2023-12-27 02:40:52,333][105692] Updated weights for policy 0, policy_version 1549463 (0.0009) [2023-12-27 02:40:52,394][105692] Updated weights for policy 0, policy_version 1549473 (0.0007) [2023-12-27 02:40:52,603][105620] Updated weights for policy 1, policy_version 1552618 (0.0008) [2023-12-27 02:40:52,655][105620] Updated weights for policy 1, policy_version 1552628 (0.0007) [2023-12-27 02:40:52,714][105620] Updated weights for policy 1, policy_version 1552638 (0.0009) [2023-12-27 02:40:52,772][105620] Updated weights for policy 1, policy_version 1552648 (0.0009) [2023-12-27 02:40:53,139][105692] Updated weights for policy 0, policy_version 1549483 (0.0009) [2023-12-27 02:40:53,192][105692] Updated weights for policy 0, policy_version 1549493 (0.0010) [2023-12-27 02:40:53,247][105692] Updated weights for policy 0, policy_version 1549503 (0.0009) [2023-12-27 02:40:53,419][105620] Updated weights for policy 1, policy_version 1552658 (0.0005) [2023-12-27 02:40:53,465][105620] Updated weights for policy 1, policy_version 1552668 (0.0005) [2023-12-27 02:40:53,510][105620] Updated weights for policy 1, policy_version 1552678 (0.0009) [2023-12-27 02:40:53,938][105692] Updated weights for policy 0, policy_version 1549513 (0.0008) [2023-12-27 02:40:53,994][105692] Updated weights for policy 0, policy_version 1549523 (0.0005) [2023-12-27 02:40:54,058][105692] Updated weights for policy 0, policy_version 1549533 (0.0009) [2023-12-27 02:40:54,074][105620] Updated weights for policy 1, policy_version 1552688 (0.0009) [2023-12-27 02:40:54,120][105692] Updated weights for policy 0, policy_version 1549543 (0.0011) [2023-12-27 02:40:54,131][105620] Updated weights for policy 1, policy_version 1552698 (0.0010) [2023-12-27 02:40:54,184][105620] Updated weights for policy 1, policy_version 1552708 (0.0007) [2023-12-27 02:40:54,804][105692] Updated weights for policy 0, policy_version 1549553 (0.0006) [2023-12-27 02:40:54,858][105692] Updated weights for policy 0, policy_version 1549563 (0.0005) [2023-12-27 02:40:54,876][105620] Updated weights for policy 1, policy_version 1552718 (0.0009) [2023-12-27 02:40:54,909][105692] Updated weights for policy 0, policy_version 1549573 (0.0005) [2023-12-27 02:40:54,932][105620] Updated weights for policy 1, policy_version 1552728 (0.0010) [2023-12-27 02:40:54,998][105620] Updated weights for policy 1, policy_version 1552738 (0.0007) [2023-12-27 02:40:55,542][105692] Updated weights for policy 0, policy_version 1549583 (0.0006) [2023-12-27 02:40:55,590][105620] Updated weights for policy 1, policy_version 1552748 (0.0010) [2023-12-27 02:40:55,595][105692] Updated weights for policy 0, policy_version 1549593 (0.0006) [2023-12-27 02:40:55,641][105692] Updated weights for policy 0, policy_version 1549603 (0.0005) [2023-12-27 02:40:55,642][105620] Updated weights for policy 1, policy_version 1552758 (0.0010) [2023-12-27 02:40:55,690][105620] Updated weights for policy 1, policy_version 1552768 (0.0008) [2023-12-27 02:40:56,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 794320896. Throughput: 0: 9948.6, 1: 9825.6. Samples: 794329256. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:40:56,062][104569] Avg episode reward: [(0, '8531.563'), (1, '9356.052')] [2023-12-27 02:40:56,225][105692] Updated weights for policy 0, policy_version 1549613 (0.0007) [2023-12-27 02:40:56,284][105692] Updated weights for policy 0, policy_version 1549623 (0.0009) [2023-12-27 02:40:56,350][105692] Updated weights for policy 0, policy_version 1549633 (0.0008) [2023-12-27 02:40:56,385][105620] Updated weights for policy 1, policy_version 1552778 (0.0006) [2023-12-27 02:40:56,453][105620] Updated weights for policy 1, policy_version 1552788 (0.0006) [2023-12-27 02:40:56,513][105620] Updated weights for policy 1, policy_version 1552798 (0.0005) [2023-12-27 02:40:56,574][105620] Updated weights for policy 1, policy_version 1552808 (0.0008) [2023-12-27 02:40:57,137][105692] Updated weights for policy 0, policy_version 1549643 (0.0006) [2023-12-27 02:40:57,157][105620] Updated weights for policy 1, policy_version 1552818 (0.0005) [2023-12-27 02:40:57,193][105692] Updated weights for policy 0, policy_version 1549653 (0.0008) [2023-12-27 02:40:57,210][105620] Updated weights for policy 1, policy_version 1552828 (0.0005) [2023-12-27 02:40:57,245][105692] Updated weights for policy 0, policy_version 1549663 (0.0007) [2023-12-27 02:40:57,257][105620] Updated weights for policy 1, policy_version 1552838 (0.0006) [2023-12-27 02:40:57,799][105620] Updated weights for policy 1, policy_version 1552848 (0.0006) [2023-12-27 02:40:57,859][105620] Updated weights for policy 1, policy_version 1552858 (0.0005) [2023-12-27 02:40:57,912][105620] Updated weights for policy 1, policy_version 1552868 (0.0006) [2023-12-27 02:40:58,090][105692] Updated weights for policy 0, policy_version 1549673 (0.0006) [2023-12-27 02:40:58,140][105692] Updated weights for policy 0, policy_version 1549683 (0.0008) [2023-12-27 02:40:58,200][105692] Updated weights for policy 0, policy_version 1549693 (0.0008) [2023-12-27 02:40:58,269][105692] Updated weights for policy 0, policy_version 1549703 (0.0008) [2023-12-27 02:40:58,578][105620] Updated weights for policy 1, policy_version 1552878 (0.0011) [2023-12-27 02:40:58,650][105620] Updated weights for policy 1, policy_version 1552888 (0.0009) [2023-12-27 02:40:58,713][105620] Updated weights for policy 1, policy_version 1552898 (0.0009) [2023-12-27 02:40:59,052][105692] Updated weights for policy 0, policy_version 1549713 (0.0009) [2023-12-27 02:40:59,106][105692] Updated weights for policy 0, policy_version 1549723 (0.0009) [2023-12-27 02:40:59,163][105692] Updated weights for policy 0, policy_version 1549733 (0.0008) [2023-12-27 02:40:59,488][105620] Updated weights for policy 1, policy_version 1552908 (0.0007) [2023-12-27 02:40:59,550][105620] Updated weights for policy 1, policy_version 1552918 (0.0005) [2023-12-27 02:40:59,620][105620] Updated weights for policy 1, policy_version 1552928 (0.0005) [2023-12-27 02:40:59,858][105692] Updated weights for policy 0, policy_version 1549743 (0.0007) [2023-12-27 02:40:59,920][105692] Updated weights for policy 0, policy_version 1549753 (0.0007) [2023-12-27 02:40:59,985][105692] Updated weights for policy 0, policy_version 1549763 (0.0006) [2023-12-27 02:41:00,218][105620] Updated weights for policy 1, policy_version 1552938 (0.0006) [2023-12-27 02:41:00,266][105620] Updated weights for policy 1, policy_version 1552948 (0.0010) [2023-12-27 02:41:00,325][105620] Updated weights for policy 1, policy_version 1552958 (0.0010) [2023-12-27 02:41:00,383][105620] Updated weights for policy 1, policy_version 1552968 (0.0010) [2023-12-27 02:41:00,624][105692] Updated weights for policy 0, policy_version 1549773 (0.0008) [2023-12-27 02:41:00,691][105692] Updated weights for policy 0, policy_version 1549783 (0.0010) [2023-12-27 02:41:00,757][105692] Updated weights for policy 0, policy_version 1549793 (0.0010) [2023-12-27 02:41:01,001][105620] Updated weights for policy 1, policy_version 1552978 (0.0010) [2023-12-27 02:41:01,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 794419200. Throughput: 0: 9979.5, 1: 9904.1. Samples: 794389316. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:41:01,062][104569] Avg episode reward: [(0, '9082.383'), (1, '9355.850')] [2023-12-27 02:41:01,063][105620] Updated weights for policy 1, policy_version 1552988 (0.0009) [2023-12-27 02:41:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001549800_396804096.pth... [2023-12-27 02:41:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001548648_396509184.pth [2023-12-27 02:41:01,132][105620] Updated weights for policy 1, policy_version 1552998 (0.0010) [2023-12-27 02:41:01,143][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001553000_397623296.pth... [2023-12-27 02:41:01,148][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001551816_397320192.pth [2023-12-27 02:41:01,558][105692] Updated weights for policy 0, policy_version 1549803 (0.0009) [2023-12-27 02:41:01,615][105692] Updated weights for policy 0, policy_version 1549813 (0.0008) [2023-12-27 02:41:01,676][105692] Updated weights for policy 0, policy_version 1549823 (0.0009) [2023-12-27 02:41:01,871][105620] Updated weights for policy 1, policy_version 1553008 (0.0010) [2023-12-27 02:41:01,923][105620] Updated weights for policy 1, policy_version 1553018 (0.0010) [2023-12-27 02:41:01,979][105620] Updated weights for policy 1, policy_version 1553028 (0.0011) [2023-12-27 02:41:02,381][105692] Updated weights for policy 0, policy_version 1549833 (0.0008) [2023-12-27 02:41:02,446][105692] Updated weights for policy 0, policy_version 1549843 (0.0006) [2023-12-27 02:41:02,507][105692] Updated weights for policy 0, policy_version 1549853 (0.0006) [2023-12-27 02:41:02,572][105692] Updated weights for policy 0, policy_version 1549863 (0.0006) [2023-12-27 02:41:02,716][105620] Updated weights for policy 1, policy_version 1553038 (0.0008) [2023-12-27 02:41:02,779][105620] Updated weights for policy 1, policy_version 1553048 (0.0005) [2023-12-27 02:41:02,849][105620] Updated weights for policy 1, policy_version 1553058 (0.0010) [2023-12-27 02:41:03,169][105692] Updated weights for policy 0, policy_version 1549873 (0.0006) [2023-12-27 02:41:03,227][105692] Updated weights for policy 0, policy_version 1549883 (0.0008) [2023-12-27 02:41:03,282][105692] Updated weights for policy 0, policy_version 1549893 (0.0008) [2023-12-27 02:41:03,537][105620] Updated weights for policy 1, policy_version 1553068 (0.0011) [2023-12-27 02:41:03,588][105620] Updated weights for policy 1, policy_version 1553078 (0.0009) [2023-12-27 02:41:03,636][105620] Updated weights for policy 1, policy_version 1553088 (0.0010) [2023-12-27 02:41:03,986][105692] Updated weights for policy 0, policy_version 1549903 (0.0007) [2023-12-27 02:41:04,038][105692] Updated weights for policy 0, policy_version 1549913 (0.0008) [2023-12-27 02:41:04,091][105692] Updated weights for policy 0, policy_version 1549923 (0.0009) [2023-12-27 02:41:04,418][105620] Updated weights for policy 1, policy_version 1553098 (0.0010) [2023-12-27 02:41:04,485][105620] Updated weights for policy 1, policy_version 1553108 (0.0011) [2023-12-27 02:41:04,541][105620] Updated weights for policy 1, policy_version 1553118 (0.0011) [2023-12-27 02:41:04,589][105620] Updated weights for policy 1, policy_version 1553128 (0.0010) [2023-12-27 02:41:04,851][105692] Updated weights for policy 0, policy_version 1549933 (0.0007) [2023-12-27 02:41:04,919][105692] Updated weights for policy 0, policy_version 1549943 (0.0005) [2023-12-27 02:41:04,969][105692] Updated weights for policy 0, policy_version 1549953 (0.0005) [2023-12-27 02:41:05,328][105620] Updated weights for policy 1, policy_version 1553138 (0.0010) [2023-12-27 02:41:05,383][105620] Updated weights for policy 1, policy_version 1553148 (0.0010) [2023-12-27 02:41:05,428][105620] Updated weights for policy 1, policy_version 1553158 (0.0009) [2023-12-27 02:41:05,621][105692] Updated weights for policy 0, policy_version 1549963 (0.0007) [2023-12-27 02:41:05,680][105692] Updated weights for policy 0, policy_version 1549973 (0.0010) [2023-12-27 02:41:05,731][105692] Updated weights for policy 0, policy_version 1549983 (0.0010) [2023-12-27 02:41:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 794517504. Throughput: 0: 9851.2, 1: 10057.4. Samples: 794507084. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:41:06,062][104569] Avg episode reward: [(0, '8721.400'), (1, '9175.147')] [2023-12-27 02:41:06,187][105620] Updated weights for policy 1, policy_version 1553168 (0.0007) [2023-12-27 02:41:06,250][105620] Updated weights for policy 1, policy_version 1553178 (0.0006) [2023-12-27 02:41:06,340][105620] Updated weights for policy 1, policy_version 1553188 (0.0011) [2023-12-27 02:41:06,471][105692] Updated weights for policy 0, policy_version 1549993 (0.0010) [2023-12-27 02:41:06,534][105692] Updated weights for policy 0, policy_version 1550003 (0.0008) [2023-12-27 02:41:06,588][105692] Updated weights for policy 0, policy_version 1550013 (0.0008) [2023-12-27 02:41:06,641][105692] Updated weights for policy 0, policy_version 1550023 (0.0008) [2023-12-27 02:41:07,045][105620] Updated weights for policy 1, policy_version 1553198 (0.0011) [2023-12-27 02:41:07,093][105620] Updated weights for policy 1, policy_version 1553208 (0.0010) [2023-12-27 02:41:07,145][105620] Updated weights for policy 1, policy_version 1553218 (0.0010) [2023-12-27 02:41:07,391][105692] Updated weights for policy 0, policy_version 1550033 (0.0009) [2023-12-27 02:41:07,448][105692] Updated weights for policy 0, policy_version 1550043 (0.0009) [2023-12-27 02:41:07,506][105692] Updated weights for policy 0, policy_version 1550053 (0.0009) [2023-12-27 02:41:07,874][105620] Updated weights for policy 1, policy_version 1553228 (0.0010) [2023-12-27 02:41:07,927][105620] Updated weights for policy 1, policy_version 1553238 (0.0008) [2023-12-27 02:41:07,988][105620] Updated weights for policy 1, policy_version 1553248 (0.0007) [2023-12-27 02:41:08,266][105692] Updated weights for policy 0, policy_version 1550063 (0.0007) [2023-12-27 02:41:08,326][105692] Updated weights for policy 0, policy_version 1550073 (0.0006) [2023-12-27 02:41:08,395][105692] Updated weights for policy 0, policy_version 1550083 (0.0009) [2023-12-27 02:41:08,716][105620] Updated weights for policy 1, policy_version 1553258 (0.0008) [2023-12-27 02:41:08,780][105620] Updated weights for policy 1, policy_version 1553268 (0.0005) [2023-12-27 02:41:08,851][105620] Updated weights for policy 1, policy_version 1553278 (0.0007) [2023-12-27 02:41:08,915][105620] Updated weights for policy 1, policy_version 1553288 (0.0011) [2023-12-27 02:41:09,018][105692] Updated weights for policy 0, policy_version 1550093 (0.0008) [2023-12-27 02:41:09,079][105692] Updated weights for policy 0, policy_version 1550103 (0.0005) [2023-12-27 02:41:09,146][105692] Updated weights for policy 0, policy_version 1550113 (0.0005) [2023-12-27 02:41:09,578][105620] Updated weights for policy 1, policy_version 1553298 (0.0011) [2023-12-27 02:41:09,641][105620] Updated weights for policy 1, policy_version 1553308 (0.0011) [2023-12-27 02:41:09,701][105620] Updated weights for policy 1, policy_version 1553318 (0.0011) [2023-12-27 02:41:09,834][105692] Updated weights for policy 0, policy_version 1550123 (0.0006) [2023-12-27 02:41:09,901][105692] Updated weights for policy 0, policy_version 1550133 (0.0008) [2023-12-27 02:41:09,966][105692] Updated weights for policy 0, policy_version 1550143 (0.0009) [2023-12-27 02:41:10,381][105620] Updated weights for policy 1, policy_version 1553328 (0.0007) [2023-12-27 02:41:10,446][105620] Updated weights for policy 1, policy_version 1553338 (0.0008) [2023-12-27 02:41:10,507][105620] Updated weights for policy 1, policy_version 1553348 (0.0006) [2023-12-27 02:41:10,785][105692] Updated weights for policy 0, policy_version 1550153 (0.0008) [2023-12-27 02:41:10,845][105692] Updated weights for policy 0, policy_version 1550163 (0.0008) [2023-12-27 02:41:10,901][105692] Updated weights for policy 0, policy_version 1550173 (0.0007) [2023-12-27 02:41:10,952][105692] Updated weights for policy 0, policy_version 1550183 (0.0005) [2023-12-27 02:41:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19521.9). Total num frames: 794615808. Throughput: 0: 9847.1, 1: 10062.3. Samples: 794623852. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:41:11,062][104569] Avg episode reward: [(0, '8629.112'), (1, '8991.528')] [2023-12-27 02:41:11,186][105620] Updated weights for policy 1, policy_version 1553358 (0.0009) [2023-12-27 02:41:11,247][105620] Updated weights for policy 1, policy_version 1553368 (0.0010) [2023-12-27 02:41:11,310][105620] Updated weights for policy 1, policy_version 1553378 (0.0008) [2023-12-27 02:41:11,619][105692] Updated weights for policy 0, policy_version 1550193 (0.0007) [2023-12-27 02:41:11,685][105692] Updated weights for policy 0, policy_version 1550203 (0.0009) [2023-12-27 02:41:11,757][105692] Updated weights for policy 0, policy_version 1550213 (0.0009) [2023-12-27 02:41:12,075][105620] Updated weights for policy 1, policy_version 1553388 (0.0011) [2023-12-27 02:41:12,139][105620] Updated weights for policy 1, policy_version 1553398 (0.0009) [2023-12-27 02:41:12,196][105620] Updated weights for policy 1, policy_version 1553408 (0.0009) [2023-12-27 02:41:12,503][105692] Updated weights for policy 0, policy_version 1550223 (0.0008) [2023-12-27 02:41:12,557][105692] Updated weights for policy 0, policy_version 1550233 (0.0007) [2023-12-27 02:41:12,615][105692] Updated weights for policy 0, policy_version 1550243 (0.0009) [2023-12-27 02:41:12,904][105620] Updated weights for policy 1, policy_version 1553418 (0.0008) [2023-12-27 02:41:12,970][105620] Updated weights for policy 1, policy_version 1553428 (0.0005) [2023-12-27 02:41:13,028][105620] Updated weights for policy 1, policy_version 1553438 (0.0007) [2023-12-27 02:41:13,076][105620] Updated weights for policy 1, policy_version 1553448 (0.0008) [2023-12-27 02:41:13,408][105692] Updated weights for policy 0, policy_version 1550253 (0.0009) [2023-12-27 02:41:13,459][105692] Updated weights for policy 0, policy_version 1550263 (0.0009) [2023-12-27 02:41:13,510][105692] Updated weights for policy 0, policy_version 1550273 (0.0009) [2023-12-27 02:41:13,776][105620] Updated weights for policy 1, policy_version 1553458 (0.0009) [2023-12-27 02:41:13,837][105620] Updated weights for policy 1, policy_version 1553468 (0.0008) [2023-12-27 02:41:13,898][105620] Updated weights for policy 1, policy_version 1553478 (0.0009) [2023-12-27 02:41:14,273][105692] Updated weights for policy 0, policy_version 1550283 (0.0009) [2023-12-27 02:41:14,337][105692] Updated weights for policy 0, policy_version 1550293 (0.0007) [2023-12-27 02:41:14,405][105692] Updated weights for policy 0, policy_version 1550303 (0.0008) [2023-12-27 02:41:14,609][105620] Updated weights for policy 1, policy_version 1553488 (0.0006) [2023-12-27 02:41:14,659][105620] Updated weights for policy 1, policy_version 1553498 (0.0006) [2023-12-27 02:41:14,728][105620] Updated weights for policy 1, policy_version 1553508 (0.0006) [2023-12-27 02:41:15,148][105692] Updated weights for policy 0, policy_version 1550313 (0.0010) [2023-12-27 02:41:15,214][105692] Updated weights for policy 0, policy_version 1550323 (0.0009) [2023-12-27 02:41:15,277][105692] Updated weights for policy 0, policy_version 1550333 (0.0010) [2023-12-27 02:41:15,334][105692] Updated weights for policy 0, policy_version 1550343 (0.0007) [2023-12-27 02:41:15,379][105620] Updated weights for policy 1, policy_version 1553518 (0.0008) [2023-12-27 02:41:15,433][105620] Updated weights for policy 1, policy_version 1553528 (0.0009) [2023-12-27 02:41:15,497][105620] Updated weights for policy 1, policy_version 1553538 (0.0009) [2023-12-27 02:41:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 794705920. Throughput: 0: 9806.4, 1: 9991.0. Samples: 794680120. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:41:16,062][104569] Avg episode reward: [(0, '8539.022'), (1, '8989.662')] [2023-12-27 02:41:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001553544_397762560.pth... [2023-12-27 02:41:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001552392_397467648.pth [2023-12-27 02:41:16,083][105692] Updated weights for policy 0, policy_version 1550353 (0.0009) [2023-12-27 02:41:16,133][105692] Updated weights for policy 0, policy_version 1550363 (0.0009) [2023-12-27 02:41:16,185][105692] Updated weights for policy 0, policy_version 1550373 (0.0009) [2023-12-27 02:41:16,198][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001550376_396951552.pth... [2023-12-27 02:41:16,203][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001549224_396656640.pth [2023-12-27 02:41:16,216][105620] Updated weights for policy 1, policy_version 1553548 (0.0007) [2023-12-27 02:41:16,275][105620] Updated weights for policy 1, policy_version 1553558 (0.0007) [2023-12-27 02:41:16,337][105620] Updated weights for policy 1, policy_version 1553568 (0.0009) [2023-12-27 02:41:16,937][105620] Updated weights for policy 1, policy_version 1553578 (0.0008) [2023-12-27 02:41:17,000][105620] Updated weights for policy 1, policy_version 1553588 (0.0006) [2023-12-27 02:41:17,042][105692] Updated weights for policy 0, policy_version 1550383 (0.0009) [2023-12-27 02:41:17,051][105620] Updated weights for policy 1, policy_version 1553598 (0.0009) [2023-12-27 02:41:17,099][105692] Updated weights for policy 0, policy_version 1550393 (0.0009) [2023-12-27 02:41:17,105][105620] Updated weights for policy 1, policy_version 1553608 (0.0006) [2023-12-27 02:41:17,155][105692] Updated weights for policy 0, policy_version 1550403 (0.0010) [2023-12-27 02:41:17,745][105692] Updated weights for policy 0, policy_version 1550413 (0.0008) [2023-12-27 02:41:17,772][105620] Updated weights for policy 1, policy_version 1553618 (0.0009) [2023-12-27 02:41:17,806][105692] Updated weights for policy 0, policy_version 1550423 (0.0005) [2023-12-27 02:41:17,829][105620] Updated weights for policy 1, policy_version 1553628 (0.0010) [2023-12-27 02:41:17,862][105692] Updated weights for policy 0, policy_version 1550433 (0.0007) [2023-12-27 02:41:17,884][105620] Updated weights for policy 1, policy_version 1553638 (0.0007) [2023-12-27 02:41:18,402][105692] Updated weights for policy 0, policy_version 1550443 (0.0010) [2023-12-27 02:41:18,471][105692] Updated weights for policy 0, policy_version 1550453 (0.0009) [2023-12-27 02:41:18,545][105692] Updated weights for policy 0, policy_version 1550463 (0.0007) [2023-12-27 02:41:18,711][105620] Updated weights for policy 1, policy_version 1553648 (0.0007) [2023-12-27 02:41:18,779][105620] Updated weights for policy 1, policy_version 1553658 (0.0010) [2023-12-27 02:41:18,828][105620] Updated weights for policy 1, policy_version 1553668 (0.0008) [2023-12-27 02:41:19,199][105692] Updated weights for policy 0, policy_version 1550473 (0.0008) [2023-12-27 02:41:19,266][105692] Updated weights for policy 0, policy_version 1550483 (0.0009) [2023-12-27 02:41:19,323][105692] Updated weights for policy 0, policy_version 1550493 (0.0010) [2023-12-27 02:41:19,384][105692] Updated weights for policy 0, policy_version 1550503 (0.0010) [2023-12-27 02:41:19,664][105620] Updated weights for policy 1, policy_version 1553678 (0.0007) [2023-12-27 02:41:19,723][105620] Updated weights for policy 1, policy_version 1553688 (0.0006) [2023-12-27 02:41:19,780][105620] Updated weights for policy 1, policy_version 1553698 (0.0008) [2023-12-27 02:41:20,116][105692] Updated weights for policy 0, policy_version 1550513 (0.0006) [2023-12-27 02:41:20,165][105692] Updated weights for policy 0, policy_version 1550523 (0.0005) [2023-12-27 02:41:20,219][105692] Updated weights for policy 0, policy_version 1550533 (0.0006) [2023-12-27 02:41:20,601][105620] Updated weights for policy 1, policy_version 1553708 (0.0008) [2023-12-27 02:41:20,668][105620] Updated weights for policy 1, policy_version 1553718 (0.0006) [2023-12-27 02:41:20,733][105620] Updated weights for policy 1, policy_version 1553728 (0.0008) [2023-12-27 02:41:20,917][105692] Updated weights for policy 0, policy_version 1550543 (0.0009) [2023-12-27 02:41:20,980][105692] Updated weights for policy 0, policy_version 1550553 (0.0011) [2023-12-27 02:41:21,042][105692] Updated weights for policy 0, policy_version 1550563 (0.0010) [2023-12-27 02:41:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 794804224. Throughput: 0: 9730.2, 1: 9959.0. Samples: 794797584. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:41:21,062][104569] Avg episode reward: [(0, '8361.068'), (1, '9081.584')] [2023-12-27 02:41:21,464][105620] Updated weights for policy 1, policy_version 1553738 (0.0008) [2023-12-27 02:41:21,524][105620] Updated weights for policy 1, policy_version 1553748 (0.0008) [2023-12-27 02:41:21,584][105620] Updated weights for policy 1, policy_version 1553758 (0.0009) [2023-12-27 02:41:21,655][105620] Updated weights for policy 1, policy_version 1553768 (0.0008) [2023-12-27 02:41:21,817][105692] Updated weights for policy 0, policy_version 1550573 (0.0008) [2023-12-27 02:41:21,876][105692] Updated weights for policy 0, policy_version 1550583 (0.0009) [2023-12-27 02:41:21,928][105692] Updated weights for policy 0, policy_version 1550593 (0.0011) [2023-12-27 02:41:22,386][105620] Updated weights for policy 1, policy_version 1553778 (0.0008) [2023-12-27 02:41:22,449][105620] Updated weights for policy 1, policy_version 1553788 (0.0008) [2023-12-27 02:41:22,512][105620] Updated weights for policy 1, policy_version 1553798 (0.0008) [2023-12-27 02:41:22,598][105692] Updated weights for policy 0, policy_version 1550603 (0.0006) [2023-12-27 02:41:22,668][105692] Updated weights for policy 0, policy_version 1550613 (0.0006) [2023-12-27 02:41:22,740][105692] Updated weights for policy 0, policy_version 1550623 (0.0007) [2023-12-27 02:41:23,267][105620] Updated weights for policy 1, policy_version 1553808 (0.0009) [2023-12-27 02:41:23,276][105692] Updated weights for policy 0, policy_version 1550633 (0.0007) [2023-12-27 02:41:23,330][105692] Updated weights for policy 0, policy_version 1550643 (0.0006) [2023-12-27 02:41:23,330][105620] Updated weights for policy 1, policy_version 1553818 (0.0010) [2023-12-27 02:41:23,373][105692] Updated weights for policy 0, policy_version 1550653 (0.0008) [2023-12-27 02:41:23,379][105620] Updated weights for policy 1, policy_version 1553828 (0.0010) [2023-12-27 02:41:23,419][105692] Updated weights for policy 0, policy_version 1550663 (0.0007) [2023-12-27 02:41:24,122][105620] Updated weights for policy 1, policy_version 1553838 (0.0010) [2023-12-27 02:41:24,125][105692] Updated weights for policy 0, policy_version 1550673 (0.0008) [2023-12-27 02:41:24,176][105692] Updated weights for policy 0, policy_version 1550683 (0.0006) [2023-12-27 02:41:24,177][105620] Updated weights for policy 1, policy_version 1553848 (0.0010) [2023-12-27 02:41:24,226][105620] Updated weights for policy 1, policy_version 1553858 (0.0010) [2023-12-27 02:41:24,232][105692] Updated weights for policy 0, policy_version 1550693 (0.0006) [2023-12-27 02:41:24,820][105620] Updated weights for policy 1, policy_version 1553868 (0.0008) [2023-12-27 02:41:24,875][105620] Updated weights for policy 1, policy_version 1553878 (0.0009) [2023-12-27 02:41:24,927][105620] Updated weights for policy 1, policy_version 1553888 (0.0009) [2023-12-27 02:41:25,060][105692] Updated weights for policy 0, policy_version 1550704 (0.0009) [2023-12-27 02:41:25,114][105692] Updated weights for policy 0, policy_version 1550714 (0.0010) [2023-12-27 02:41:25,170][105692] Updated weights for policy 0, policy_version 1550724 (0.0009) [2023-12-27 02:41:25,505][105620] Updated weights for policy 1, policy_version 1553898 (0.0007) [2023-12-27 02:41:25,571][105620] Updated weights for policy 1, policy_version 1553908 (0.0007) [2023-12-27 02:41:25,626][105620] Updated weights for policy 1, policy_version 1553918 (0.0006) [2023-12-27 02:41:25,673][105620] Updated weights for policy 1, policy_version 1553928 (0.0006) [2023-12-27 02:41:26,022][105692] Updated weights for policy 0, policy_version 1550734 (0.0011) [2023-12-27 02:41:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 794902528. Throughput: 0: 9774.2, 1: 9964.5. Samples: 794915504. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:41:26,062][104569] Avg episode reward: [(0, '8174.858'), (1, '8986.699')] [2023-12-27 02:41:26,074][105692] Updated weights for policy 0, policy_version 1550744 (0.0008) [2023-12-27 02:41:26,139][105692] Updated weights for policy 0, policy_version 1550754 (0.0009) [2023-12-27 02:41:26,392][105620] Updated weights for policy 1, policy_version 1553938 (0.0009) [2023-12-27 02:41:26,444][105620] Updated weights for policy 1, policy_version 1553948 (0.0010) [2023-12-27 02:41:26,502][105620] Updated weights for policy 1, policy_version 1553958 (0.0009) [2023-12-27 02:41:26,810][105692] Updated weights for policy 0, policy_version 1550764 (0.0008) [2023-12-27 02:41:26,863][105692] Updated weights for policy 0, policy_version 1550774 (0.0009) [2023-12-27 02:41:26,922][105692] Updated weights for policy 0, policy_version 1550784 (0.0009) [2023-12-27 02:41:27,300][105620] Updated weights for policy 1, policy_version 1553968 (0.0009) [2023-12-27 02:41:27,363][105620] Updated weights for policy 1, policy_version 1553978 (0.0010) [2023-12-27 02:41:27,410][105620] Updated weights for policy 1, policy_version 1553988 (0.0008) [2023-12-27 02:41:27,646][105692] Updated weights for policy 0, policy_version 1550794 (0.0009) [2023-12-27 02:41:27,697][105692] Updated weights for policy 0, policy_version 1550804 (0.0009) [2023-12-27 02:41:27,753][105692] Updated weights for policy 0, policy_version 1550814 (0.0009) [2023-12-27 02:41:27,802][105692] Updated weights for policy 0, policy_version 1550824 (0.0008) [2023-12-27 02:41:28,152][105620] Updated weights for policy 1, policy_version 1553998 (0.0009) [2023-12-27 02:41:28,209][105620] Updated weights for policy 1, policy_version 1554009 (0.0010) [2023-12-27 02:41:28,262][105620] Updated weights for policy 1, policy_version 1554019 (0.0009) [2023-12-27 02:41:28,423][105692] Updated weights for policy 0, policy_version 1550834 (0.0009) [2023-12-27 02:41:28,480][105692] Updated weights for policy 0, policy_version 1550844 (0.0005) [2023-12-27 02:41:28,533][105692] Updated weights for policy 0, policy_version 1550854 (0.0006) [2023-12-27 02:41:29,084][105620] Updated weights for policy 1, policy_version 1554029 (0.0009) [2023-12-27 02:41:29,133][105620] Updated weights for policy 1, policy_version 1554039 (0.0009) [2023-12-27 02:41:29,186][105620] Updated weights for policy 1, policy_version 1554049 (0.0009) [2023-12-27 02:41:29,259][105692] Updated weights for policy 0, policy_version 1550864 (0.0008) [2023-12-27 02:41:29,322][105692] Updated weights for policy 0, policy_version 1550874 (0.0008) [2023-12-27 02:41:29,383][105692] Updated weights for policy 0, policy_version 1550884 (0.0008) [2023-12-27 02:41:29,955][105620] Updated weights for policy 1, policy_version 1554059 (0.0007) [2023-12-27 02:41:29,992][105692] Updated weights for policy 0, policy_version 1550894 (0.0007) [2023-12-27 02:41:30,016][105620] Updated weights for policy 1, policy_version 1554069 (0.0005) [2023-12-27 02:41:30,049][105692] Updated weights for policy 0, policy_version 1550904 (0.0010) [2023-12-27 02:41:30,072][105620] Updated weights for policy 1, policy_version 1554079 (0.0006) [2023-12-27 02:41:30,103][105692] Updated weights for policy 0, policy_version 1550914 (0.0007) [2023-12-27 02:41:30,642][105620] Updated weights for policy 1, policy_version 1554089 (0.0006) [2023-12-27 02:41:30,702][105620] Updated weights for policy 1, policy_version 1554099 (0.0009) [2023-12-27 02:41:30,764][105620] Updated weights for policy 1, policy_version 1554109 (0.0009) [2023-12-27 02:41:30,818][105620] Updated weights for policy 1, policy_version 1554119 (0.0009) [2023-12-27 02:41:30,894][105692] Updated weights for policy 0, policy_version 1550924 (0.0009) [2023-12-27 02:41:30,940][105692] Updated weights for policy 0, policy_version 1550934 (0.0009) [2023-12-27 02:41:30,989][105692] Updated weights for policy 0, policy_version 1550944 (0.0009) [2023-12-27 02:41:31,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.3, 300 sec: 19521.9). Total num frames: 795009024. Throughput: 0: 9815.4, 1: 9913.9. Samples: 794973360. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:41:31,063][104569] Avg episode reward: [(0, '8720.822'), (1, '9170.609')] [2023-12-27 02:41:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001550952_397099008.pth... [2023-12-27 02:41:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001554120_397910016.pth... [2023-12-27 02:41:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001549800_396804096.pth [2023-12-27 02:41:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001553000_397623296.pth [2023-12-27 02:41:31,551][105620] Updated weights for policy 1, policy_version 1554129 (0.0006) [2023-12-27 02:41:31,609][105620] Updated weights for policy 1, policy_version 1554139 (0.0006) [2023-12-27 02:41:31,671][105620] Updated weights for policy 1, policy_version 1554149 (0.0009) [2023-12-27 02:41:31,786][105692] Updated weights for policy 0, policy_version 1550954 (0.0008) [2023-12-27 02:41:31,841][105692] Updated weights for policy 0, policy_version 1550965 (0.0010) [2023-12-27 02:41:31,895][105692] Updated weights for policy 0, policy_version 1550976 (0.0010) [2023-12-27 02:41:32,352][105620] Updated weights for policy 1, policy_version 1554159 (0.0007) [2023-12-27 02:41:32,407][105620] Updated weights for policy 1, policy_version 1554169 (0.0008) [2023-12-27 02:41:32,465][105620] Updated weights for policy 1, policy_version 1554179 (0.0009) [2023-12-27 02:41:32,655][105692] Updated weights for policy 0, policy_version 1550987 (0.0007) [2023-12-27 02:41:32,707][105692] Updated weights for policy 0, policy_version 1550997 (0.0007) [2023-12-27 02:41:32,758][105692] Updated weights for policy 0, policy_version 1551007 (0.0009) [2023-12-27 02:41:33,123][105620] Updated weights for policy 1, policy_version 1554189 (0.0007) [2023-12-27 02:41:33,169][105620] Updated weights for policy 1, policy_version 1554199 (0.0005) [2023-12-27 02:41:33,216][105620] Updated weights for policy 1, policy_version 1554209 (0.0005) [2023-12-27 02:41:33,388][105692] Updated weights for policy 0, policy_version 1551017 (0.0008) [2023-12-27 02:41:33,442][105692] Updated weights for policy 0, policy_version 1551027 (0.0007) [2023-12-27 02:41:33,500][105692] Updated weights for policy 0, policy_version 1551037 (0.0008) [2023-12-27 02:41:33,561][105692] Updated weights for policy 0, policy_version 1551047 (0.0008) [2023-12-27 02:41:33,879][105620] Updated weights for policy 1, policy_version 1554219 (0.0007) [2023-12-27 02:41:33,927][105620] Updated weights for policy 1, policy_version 1554229 (0.0010) [2023-12-27 02:41:33,998][105620] Updated weights for policy 1, policy_version 1554239 (0.0010) [2023-12-27 02:41:34,153][105692] Updated weights for policy 0, policy_version 1551057 (0.0006) [2023-12-27 02:41:34,207][105692] Updated weights for policy 0, policy_version 1551067 (0.0008) [2023-12-27 02:41:34,264][105692] Updated weights for policy 0, policy_version 1551077 (0.0008) [2023-12-27 02:41:34,762][105620] Updated weights for policy 1, policy_version 1554249 (0.0010) [2023-12-27 02:41:34,821][105620] Updated weights for policy 1, policy_version 1554259 (0.0010) [2023-12-27 02:41:34,875][105620] Updated weights for policy 1, policy_version 1554269 (0.0010) [2023-12-27 02:41:34,927][105620] Updated weights for policy 1, policy_version 1554279 (0.0010) [2023-12-27 02:41:35,028][105692] Updated weights for policy 0, policy_version 1551087 (0.0009) [2023-12-27 02:41:35,095][105692] Updated weights for policy 0, policy_version 1551097 (0.0010) [2023-12-27 02:41:35,160][105692] Updated weights for policy 0, policy_version 1551107 (0.0009) [2023-12-27 02:41:35,633][105620] Updated weights for policy 1, policy_version 1554289 (0.0009) [2023-12-27 02:41:35,684][105620] Updated weights for policy 1, policy_version 1554299 (0.0010) [2023-12-27 02:41:35,740][105620] Updated weights for policy 1, policy_version 1554309 (0.0009) [2023-12-27 02:41:35,964][105692] Updated weights for policy 0, policy_version 1551117 (0.0009) [2023-12-27 02:41:36,021][105692] Updated weights for policy 0, policy_version 1551127 (0.0010) [2023-12-27 02:41:36,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 795099136. Throughput: 0: 9745.3, 1: 9896.9. Samples: 795092588. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:41:36,063][104569] Avg episode reward: [(0, '8623.828'), (1, '9264.480')] [2023-12-27 02:41:36,083][105692] Updated weights for policy 0, policy_version 1551137 (0.0009) [2023-12-27 02:41:36,310][105620] Updated weights for policy 1, policy_version 1554319 (0.0007) [2023-12-27 02:41:36,377][105620] Updated weights for policy 1, policy_version 1554329 (0.0008) [2023-12-27 02:41:36,448][105620] Updated weights for policy 1, policy_version 1554339 (0.0009) [2023-12-27 02:41:36,982][105692] Updated weights for policy 0, policy_version 1551147 (0.0010) [2023-12-27 02:41:37,028][105620] Updated weights for policy 1, policy_version 1554349 (0.0009) [2023-12-27 02:41:37,032][105692] Updated weights for policy 0, policy_version 1551157 (0.0010) [2023-12-27 02:41:37,083][105692] Updated weights for policy 0, policy_version 1551167 (0.0008) [2023-12-27 02:41:37,084][105620] Updated weights for policy 1, policy_version 1554359 (0.0008) [2023-12-27 02:41:37,148][105620] Updated weights for policy 1, policy_version 1554369 (0.0005) [2023-12-27 02:41:37,767][105620] Updated weights for policy 1, policy_version 1554379 (0.0009) [2023-12-27 02:41:37,820][105620] Updated weights for policy 1, policy_version 1554389 (0.0006) [2023-12-27 02:41:37,880][105620] Updated weights for policy 1, policy_version 1554399 (0.0006) [2023-12-27 02:41:37,944][105692] Updated weights for policy 0, policy_version 1551177 (0.0010) [2023-12-27 02:41:38,001][105692] Updated weights for policy 0, policy_version 1551187 (0.0009) [2023-12-27 02:41:38,058][105692] Updated weights for policy 0, policy_version 1551197 (0.0008) [2023-12-27 02:41:38,108][105692] Updated weights for policy 0, policy_version 1551207 (0.0008) [2023-12-27 02:41:38,552][105620] Updated weights for policy 1, policy_version 1554409 (0.0006) [2023-12-27 02:41:38,611][105620] Updated weights for policy 1, policy_version 1554419 (0.0009) [2023-12-27 02:41:38,659][105620] Updated weights for policy 1, policy_version 1554429 (0.0009) [2023-12-27 02:41:38,714][105620] Updated weights for policy 1, policy_version 1554439 (0.0009) [2023-12-27 02:41:38,889][105692] Updated weights for policy 0, policy_version 1551217 (0.0008) [2023-12-27 02:41:38,940][105692] Updated weights for policy 0, policy_version 1551227 (0.0009) [2023-12-27 02:41:38,996][105692] Updated weights for policy 0, policy_version 1551237 (0.0009) [2023-12-27 02:41:39,517][105620] Updated weights for policy 1, policy_version 1554449 (0.0009) [2023-12-27 02:41:39,580][105620] Updated weights for policy 1, policy_version 1554459 (0.0008) [2023-12-27 02:41:39,647][105620] Updated weights for policy 1, policy_version 1554469 (0.0009) [2023-12-27 02:41:39,763][105692] Updated weights for policy 0, policy_version 1551247 (0.0010) [2023-12-27 02:41:39,828][105692] Updated weights for policy 0, policy_version 1551257 (0.0011) [2023-12-27 02:41:39,887][105692] Updated weights for policy 0, policy_version 1551267 (0.0009) [2023-12-27 02:41:40,426][105620] Updated weights for policy 1, policy_version 1554479 (0.0008) [2023-12-27 02:41:40,485][105620] Updated weights for policy 1, policy_version 1554489 (0.0009) [2023-12-27 02:41:40,543][105692] Updated weights for policy 0, policy_version 1551277 (0.0007) [2023-12-27 02:41:40,544][105620] Updated weights for policy 1, policy_version 1554499 (0.0009) [2023-12-27 02:41:40,612][105692] Updated weights for policy 0, policy_version 1551287 (0.0006) [2023-12-27 02:41:40,667][105692] Updated weights for policy 0, policy_version 1551297 (0.0011) [2023-12-27 02:41:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 795197440. Throughput: 0: 9638.2, 1: 9878.8. Samples: 795207520. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:41:41,062][104569] Avg episode reward: [(0, '8442.961'), (1, '8990.319')] [2023-12-27 02:41:41,213][105620] Updated weights for policy 1, policy_version 1554509 (0.0007) [2023-12-27 02:41:41,282][105620] Updated weights for policy 1, policy_version 1554519 (0.0007) [2023-12-27 02:41:41,330][105692] Updated weights for policy 0, policy_version 1551307 (0.0009) [2023-12-27 02:41:41,341][105620] Updated weights for policy 1, policy_version 1554529 (0.0009) [2023-12-27 02:41:41,399][105692] Updated weights for policy 0, policy_version 1551317 (0.0008) [2023-12-27 02:41:41,455][105692] Updated weights for policy 0, policy_version 1551327 (0.0009) [2023-12-27 02:41:42,087][105620] Updated weights for policy 1, policy_version 1554539 (0.0009) [2023-12-27 02:41:42,135][105620] Updated weights for policy 1, policy_version 1554549 (0.0009) [2023-12-27 02:41:42,188][105620] Updated weights for policy 1, policy_version 1554559 (0.0009) [2023-12-27 02:41:42,200][105692] Updated weights for policy 0, policy_version 1551337 (0.0006) [2023-12-27 02:41:42,268][105692] Updated weights for policy 0, policy_version 1551347 (0.0007) [2023-12-27 02:41:42,326][105692] Updated weights for policy 0, policy_version 1551357 (0.0009) [2023-12-27 02:41:42,390][105692] Updated weights for policy 0, policy_version 1551367 (0.0009) [2023-12-27 02:41:42,966][105620] Updated weights for policy 1, policy_version 1554569 (0.0007) [2023-12-27 02:41:43,033][105620] Updated weights for policy 1, policy_version 1554579 (0.0007) [2023-12-27 02:41:43,073][105692] Updated weights for policy 0, policy_version 1551377 (0.0007) [2023-12-27 02:41:43,096][105620] Updated weights for policy 1, policy_version 1554589 (0.0006) [2023-12-27 02:41:43,124][105692] Updated weights for policy 0, policy_version 1551387 (0.0005) [2023-12-27 02:41:43,155][105620] Updated weights for policy 1, policy_version 1554599 (0.0009) [2023-12-27 02:41:43,175][105692] Updated weights for policy 0, policy_version 1551397 (0.0005) [2023-12-27 02:41:43,884][105692] Updated weights for policy 0, policy_version 1551407 (0.0005) [2023-12-27 02:41:43,927][105620] Updated weights for policy 1, policy_version 1554609 (0.0010) [2023-12-27 02:41:43,945][105692] Updated weights for policy 0, policy_version 1551417 (0.0005) [2023-12-27 02:41:43,986][105620] Updated weights for policy 1, policy_version 1554619 (0.0011) [2023-12-27 02:41:44,004][105692] Updated weights for policy 0, policy_version 1551427 (0.0005) [2023-12-27 02:41:44,045][105620] Updated weights for policy 1, policy_version 1554629 (0.0011) [2023-12-27 02:41:44,768][105692] Updated weights for policy 0, policy_version 1551437 (0.0008) [2023-12-27 02:41:44,786][105620] Updated weights for policy 1, policy_version 1554639 (0.0011) [2023-12-27 02:41:44,829][105692] Updated weights for policy 0, policy_version 1551447 (0.0011) [2023-12-27 02:41:44,855][105620] Updated weights for policy 1, policy_version 1554649 (0.0011) [2023-12-27 02:41:44,892][105692] Updated weights for policy 0, policy_version 1551457 (0.0011) [2023-12-27 02:41:44,915][105620] Updated weights for policy 1, policy_version 1554659 (0.0011) [2023-12-27 02:41:45,589][105692] Updated weights for policy 0, policy_version 1551467 (0.0009) [2023-12-27 02:41:45,607][105620] Updated weights for policy 1, policy_version 1554669 (0.0008) [2023-12-27 02:41:45,641][105692] Updated weights for policy 0, policy_version 1551477 (0.0006) [2023-12-27 02:41:45,664][105620] Updated weights for policy 1, policy_version 1554679 (0.0006) [2023-12-27 02:41:45,695][105692] Updated weights for policy 0, policy_version 1551487 (0.0006) [2023-12-27 02:41:45,720][105620] Updated weights for policy 1, policy_version 1554689 (0.0005) [2023-12-27 02:41:46,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 795295744. Throughput: 0: 9673.2, 1: 9783.2. Samples: 795264856. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:41:46,062][104569] Avg episode reward: [(0, '8352.769'), (1, '8630.157')] [2023-12-27 02:41:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001551496_397238272.pth... [2023-12-27 02:41:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001554696_398057472.pth... [2023-12-27 02:41:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001553544_397762560.pth [2023-12-27 02:41:46,074][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_001554696_398057472.pth [2023-12-27 02:41:46,082][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001550376_396951552.pth [2023-12-27 02:41:46,082][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_001551496_397238272.pth [2023-12-27 02:41:46,390][105620] Updated weights for policy 1, policy_version 1554699 (0.0006) [2023-12-27 02:41:46,421][105692] Updated weights for policy 0, policy_version 1551497 (0.0009) [2023-12-27 02:41:46,440][105620] Updated weights for policy 1, policy_version 1554709 (0.0007) [2023-12-27 02:41:46,466][105692] Updated weights for policy 0, policy_version 1551507 (0.0011) [2023-12-27 02:41:46,492][105620] Updated weights for policy 1, policy_version 1554719 (0.0005) [2023-12-27 02:41:46,515][105692] Updated weights for policy 0, policy_version 1551517 (0.0010) [2023-12-27 02:41:46,572][105692] Updated weights for policy 0, policy_version 1551527 (0.0005) [2023-12-27 02:41:47,208][105692] Updated weights for policy 0, policy_version 1551537 (0.0010) [2023-12-27 02:41:47,256][105692] Updated weights for policy 0, policy_version 1551547 (0.0010) [2023-12-27 02:41:47,304][105692] Updated weights for policy 0, policy_version 1551557 (0.0010) [2023-12-27 02:41:47,328][105620] Updated weights for policy 1, policy_version 1554729 (0.0008) [2023-12-27 02:41:47,382][105620] Updated weights for policy 1, policy_version 1554739 (0.0008) [2023-12-27 02:41:47,429][105620] Updated weights for policy 1, policy_version 1554749 (0.0008) [2023-12-27 02:41:47,479][105620] Updated weights for policy 1, policy_version 1554759 (0.0007) [2023-12-27 02:41:47,996][105692] Updated weights for policy 0, policy_version 1551567 (0.0007) [2023-12-27 02:41:48,052][105692] Updated weights for policy 0, policy_version 1551577 (0.0005) [2023-12-27 02:41:48,108][105692] Updated weights for policy 0, policy_version 1551587 (0.0005) [2023-12-27 02:41:48,285][105620] Updated weights for policy 1, policy_version 1554769 (0.0011) [2023-12-27 02:41:48,360][105620] Updated weights for policy 1, policy_version 1554779 (0.0011) [2023-12-27 02:41:48,419][105620] Updated weights for policy 1, policy_version 1554789 (0.0010) [2023-12-27 02:41:48,728][105692] Updated weights for policy 0, policy_version 1551597 (0.0008) [2023-12-27 02:41:48,790][105692] Updated weights for policy 0, policy_version 1551607 (0.0011) [2023-12-27 02:41:48,855][105692] Updated weights for policy 0, policy_version 1551617 (0.0011) [2023-12-27 02:41:49,108][105620] Updated weights for policy 1, policy_version 1554799 (0.0009) [2023-12-27 02:41:49,168][105620] Updated weights for policy 1, policy_version 1554809 (0.0008) [2023-12-27 02:41:49,237][105620] Updated weights for policy 1, policy_version 1554819 (0.0008) [2023-12-27 02:41:49,613][105692] Updated weights for policy 0, policy_version 1551627 (0.0010) [2023-12-27 02:41:49,668][105692] Updated weights for policy 0, policy_version 1551637 (0.0010) [2023-12-27 02:41:49,723][105692] Updated weights for policy 0, policy_version 1551647 (0.0010) [2023-12-27 02:41:49,989][105620] Updated weights for policy 1, policy_version 1554829 (0.0008) [2023-12-27 02:41:50,042][105620] Updated weights for policy 1, policy_version 1554839 (0.0008) [2023-12-27 02:41:50,095][105620] Updated weights for policy 1, policy_version 1554849 (0.0008) [2023-12-27 02:41:50,477][105692] Updated weights for policy 0, policy_version 1551657 (0.0011) [2023-12-27 02:41:50,532][105692] Updated weights for policy 0, policy_version 1551667 (0.0010) [2023-12-27 02:41:50,595][105692] Updated weights for policy 0, policy_version 1551677 (0.0011) [2023-12-27 02:41:50,661][105692] Updated weights for policy 0, policy_version 1551687 (0.0011) [2023-12-27 02:41:50,815][105620] Updated weights for policy 1, policy_version 1554859 (0.0008) [2023-12-27 02:41:50,881][105620] Updated weights for policy 1, policy_version 1554869 (0.0006) [2023-12-27 02:41:50,931][105620] Updated weights for policy 1, policy_version 1554879 (0.0008) [2023-12-27 02:41:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 795394048. Throughput: 0: 9704.0, 1: 9736.0. Samples: 795381884. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:41:51,062][104569] Avg episode reward: [(0, '8168.678'), (1, '8353.626')] [2023-12-27 02:41:51,412][105692] Updated weights for policy 0, policy_version 1551697 (0.0011) [2023-12-27 02:41:51,465][105692] Updated weights for policy 0, policy_version 1551707 (0.0010) [2023-12-27 02:41:51,513][105692] Updated weights for policy 0, policy_version 1551717 (0.0010) [2023-12-27 02:41:51,632][105620] Updated weights for policy 1, policy_version 1554889 (0.0008) [2023-12-27 02:41:51,692][105620] Updated weights for policy 1, policy_version 1554899 (0.0008) [2023-12-27 02:41:51,758][105620] Updated weights for policy 1, policy_version 1554909 (0.0008) [2023-12-27 02:41:51,817][105620] Updated weights for policy 1, policy_version 1554919 (0.0010) [2023-12-27 02:41:52,202][105692] Updated weights for policy 0, policy_version 1551727 (0.0007) [2023-12-27 02:41:52,263][105692] Updated weights for policy 0, policy_version 1551737 (0.0009) [2023-12-27 02:41:52,326][105692] Updated weights for policy 0, policy_version 1551747 (0.0006) [2023-12-27 02:41:52,660][105620] Updated weights for policy 1, policy_version 1554929 (0.0008) [2023-12-27 02:41:52,727][105620] Updated weights for policy 1, policy_version 1554939 (0.0008) [2023-12-27 02:41:52,794][105620] Updated weights for policy 1, policy_version 1554949 (0.0007) [2023-12-27 02:41:53,014][105692] Updated weights for policy 0, policy_version 1551757 (0.0008) [2023-12-27 02:41:53,067][105692] Updated weights for policy 0, policy_version 1551767 (0.0005) [2023-12-27 02:41:53,128][105692] Updated weights for policy 0, policy_version 1551777 (0.0005) [2023-12-27 02:41:53,469][105620] Updated weights for policy 1, policy_version 1554959 (0.0006) [2023-12-27 02:41:53,516][105620] Updated weights for policy 1, policy_version 1554969 (0.0007) [2023-12-27 02:41:53,560][105620] Updated weights for policy 1, policy_version 1554979 (0.0007) [2023-12-27 02:41:53,697][105692] Updated weights for policy 0, policy_version 1551787 (0.0007) [2023-12-27 02:41:53,759][105692] Updated weights for policy 0, policy_version 1551797 (0.0010) [2023-12-27 02:41:53,810][105692] Updated weights for policy 0, policy_version 1551807 (0.0010) [2023-12-27 02:41:54,381][105620] Updated weights for policy 1, policy_version 1554989 (0.0009) [2023-12-27 02:41:54,406][105692] Updated weights for policy 0, policy_version 1551817 (0.0010) [2023-12-27 02:41:54,442][105620] Updated weights for policy 1, policy_version 1554999 (0.0009) [2023-12-27 02:41:54,458][105692] Updated weights for policy 0, policy_version 1551827 (0.0005) [2023-12-27 02:41:54,497][105620] Updated weights for policy 1, policy_version 1555009 (0.0005) [2023-12-27 02:41:54,504][105692] Updated weights for policy 0, policy_version 1551837 (0.0005) [2023-12-27 02:41:54,555][105692] Updated weights for policy 0, policy_version 1551847 (0.0005) [2023-12-27 02:41:55,065][105620] Updated weights for policy 1, policy_version 1555019 (0.0006) [2023-12-27 02:41:55,115][105620] Updated weights for policy 1, policy_version 1555029 (0.0006) [2023-12-27 02:41:55,169][105620] Updated weights for policy 1, policy_version 1555039 (0.0005) [2023-12-27 02:41:55,209][105692] Updated weights for policy 0, policy_version 1551857 (0.0009) [2023-12-27 02:41:55,267][105692] Updated weights for policy 0, policy_version 1551867 (0.0010) [2023-12-27 02:41:55,321][105692] Updated weights for policy 0, policy_version 1551877 (0.0010) [2023-12-27 02:41:55,709][105620] Updated weights for policy 1, policy_version 1555049 (0.0005) [2023-12-27 02:41:55,775][105620] Updated weights for policy 1, policy_version 1555059 (0.0005) [2023-12-27 02:41:55,833][105620] Updated weights for policy 1, policy_version 1555069 (0.0008) [2023-12-27 02:41:55,894][105620] Updated weights for policy 1, policy_version 1555079 (0.0009) [2023-12-27 02:41:56,033][105692] Updated weights for policy 0, policy_version 1551887 (0.0007) [2023-12-27 02:41:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 795492352. Throughput: 0: 9756.8, 1: 9757.7. Samples: 795502004. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:41:56,063][104569] Avg episode reward: [(0, '8715.232'), (1, '8269.334')] [2023-12-27 02:41:56,087][105692] Updated weights for policy 0, policy_version 1551897 (0.0005) [2023-12-27 02:41:56,151][105692] Updated weights for policy 0, policy_version 1551907 (0.0008) [2023-12-27 02:41:56,600][105620] Updated weights for policy 1, policy_version 1555089 (0.0009) [2023-12-27 02:41:56,654][105620] Updated weights for policy 1, policy_version 1555099 (0.0008) [2023-12-27 02:41:56,701][105620] Updated weights for policy 1, policy_version 1555109 (0.0009) [2023-12-27 02:41:56,853][105692] Updated weights for policy 0, policy_version 1551917 (0.0008) [2023-12-27 02:41:56,900][105692] Updated weights for policy 0, policy_version 1551927 (0.0009) [2023-12-27 02:41:56,947][105692] Updated weights for policy 0, policy_version 1551937 (0.0009) [2023-12-27 02:41:57,420][105620] Updated weights for policy 1, policy_version 1555119 (0.0006) [2023-12-27 02:41:57,482][105620] Updated weights for policy 1, policy_version 1555129 (0.0005) [2023-12-27 02:41:57,544][105620] Updated weights for policy 1, policy_version 1555139 (0.0007) [2023-12-27 02:41:57,747][105692] Updated weights for policy 0, policy_version 1551947 (0.0009) [2023-12-27 02:41:57,805][105692] Updated weights for policy 0, policy_version 1551957 (0.0009) [2023-12-27 02:41:57,865][105692] Updated weights for policy 0, policy_version 1551967 (0.0010) [2023-12-27 02:41:58,267][105620] Updated weights for policy 1, policy_version 1555149 (0.0008) [2023-12-27 02:41:58,340][105620] Updated weights for policy 1, policy_version 1555159 (0.0008) [2023-12-27 02:41:58,411][105620] Updated weights for policy 1, policy_version 1555169 (0.0007) [2023-12-27 02:41:58,616][105692] Updated weights for policy 0, policy_version 1551977 (0.0008) [2023-12-27 02:41:58,676][105692] Updated weights for policy 0, policy_version 1551987 (0.0009) [2023-12-27 02:41:58,739][105692] Updated weights for policy 0, policy_version 1551997 (0.0009) [2023-12-27 02:41:58,809][105692] Updated weights for policy 0, policy_version 1552007 (0.0009) [2023-12-27 02:41:59,190][105620] Updated weights for policy 1, policy_version 1555179 (0.0009) [2023-12-27 02:41:59,260][105620] Updated weights for policy 1, policy_version 1555189 (0.0012) [2023-12-27 02:41:59,327][105620] Updated weights for policy 1, policy_version 1555199 (0.0009) [2023-12-27 02:41:59,651][105692] Updated weights for policy 0, policy_version 1552017 (0.0006) [2023-12-27 02:41:59,703][105692] Updated weights for policy 0, policy_version 1552027 (0.0005) [2023-12-27 02:41:59,754][105692] Updated weights for policy 0, policy_version 1552037 (0.0006) [2023-12-27 02:42:00,161][105620] Updated weights for policy 1, policy_version 1555209 (0.0010) [2023-12-27 02:42:00,213][105620] Updated weights for policy 1, policy_version 1555219 (0.0008) [2023-12-27 02:42:00,264][105620] Updated weights for policy 1, policy_version 1555229 (0.0008) [2023-12-27 02:42:00,316][105620] Updated weights for policy 1, policy_version 1555239 (0.0008) [2023-12-27 02:42:00,380][105692] Updated weights for policy 0, policy_version 1552047 (0.0007) [2023-12-27 02:42:00,435][105692] Updated weights for policy 0, policy_version 1552057 (0.0007) [2023-12-27 02:42:00,490][105692] Updated weights for policy 0, policy_version 1552067 (0.0008) [2023-12-27 02:42:00,986][105620] Updated weights for policy 1, policy_version 1555250 (0.0007) [2023-12-27 02:42:01,048][105620] Updated weights for policy 1, policy_version 1555260 (0.0008) [2023-12-27 02:42:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 795582464. Throughput: 0: 9785.8, 1: 9761.0. Samples: 795559724. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:42:01,062][104569] Avg episode reward: [(0, '8900.838'), (1, '8725.073')] [2023-12-27 02:42:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001552072_397385728.pth... [2023-12-27 02:42:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001550952_397099008.pth [2023-12-27 02:42:01,103][105620] Updated weights for policy 1, policy_version 1555270 (0.0006) [2023-12-27 02:42:01,115][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001555272_398204928.pth... [2023-12-27 02:42:01,119][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001554120_397910016.pth [2023-12-27 02:42:01,299][105692] Updated weights for policy 0, policy_version 1552077 (0.0008) [2023-12-27 02:42:01,360][105692] Updated weights for policy 0, policy_version 1552087 (0.0009) [2023-12-27 02:42:01,416][105692] Updated weights for policy 0, policy_version 1552097 (0.0009) [2023-12-27 02:42:01,784][105620] Updated weights for policy 1, policy_version 1555280 (0.0008) [2023-12-27 02:42:01,830][105620] Updated weights for policy 1, policy_version 1555290 (0.0005) [2023-12-27 02:42:01,873][105620] Updated weights for policy 1, policy_version 1555300 (0.0005) [2023-12-27 02:42:02,271][105692] Updated weights for policy 0, policy_version 1552107 (0.0009) [2023-12-27 02:42:02,338][105692] Updated weights for policy 0, policy_version 1552117 (0.0008) [2023-12-27 02:42:02,402][105692] Updated weights for policy 0, policy_version 1552127 (0.0008) [2023-12-27 02:42:02,482][105620] Updated weights for policy 1, policy_version 1555310 (0.0007) [2023-12-27 02:42:02,536][105620] Updated weights for policy 1, policy_version 1555320 (0.0009) [2023-12-27 02:42:02,594][105620] Updated weights for policy 1, policy_version 1555331 (0.0010) [2023-12-27 02:42:03,011][105692] Updated weights for policy 0, policy_version 1552137 (0.0009) [2023-12-27 02:42:03,061][105692] Updated weights for policy 0, policy_version 1552147 (0.0009) [2023-12-27 02:42:03,108][105692] Updated weights for policy 0, policy_version 1552157 (0.0009) [2023-12-27 02:42:03,155][105692] Updated weights for policy 0, policy_version 1552167 (0.0009) [2023-12-27 02:42:03,350][105620] Updated weights for policy 1, policy_version 1555341 (0.0008) [2023-12-27 02:42:03,404][105620] Updated weights for policy 1, policy_version 1555351 (0.0005) [2023-12-27 02:42:03,471][105620] Updated weights for policy 1, policy_version 1555361 (0.0005) [2023-12-27 02:42:03,906][105692] Updated weights for policy 0, policy_version 1552177 (0.0008) [2023-12-27 02:42:03,974][105692] Updated weights for policy 0, policy_version 1552187 (0.0008) [2023-12-27 02:42:04,041][105620] Updated weights for policy 1, policy_version 1555371 (0.0005) [2023-12-27 02:42:04,042][105692] Updated weights for policy 0, policy_version 1552197 (0.0008) [2023-12-27 02:42:04,098][105620] Updated weights for policy 1, policy_version 1555381 (0.0007) [2023-12-27 02:42:04,160][105620] Updated weights for policy 1, policy_version 1555391 (0.0009) [2023-12-27 02:42:04,761][105692] Updated weights for policy 0, policy_version 1552207 (0.0009) [2023-12-27 02:42:04,826][105692] Updated weights for policy 0, policy_version 1552217 (0.0009) [2023-12-27 02:42:04,884][105692] Updated weights for policy 0, policy_version 1552227 (0.0009) [2023-12-27 02:42:04,903][105620] Updated weights for policy 1, policy_version 1555401 (0.0009) [2023-12-27 02:42:04,961][105620] Updated weights for policy 1, policy_version 1555411 (0.0009) [2023-12-27 02:42:05,012][105620] Updated weights for policy 1, policy_version 1555421 (0.0009) [2023-12-27 02:42:05,059][105620] Updated weights for policy 1, policy_version 1555431 (0.0009) [2023-12-27 02:42:05,560][105692] Updated weights for policy 0, policy_version 1552237 (0.0008) [2023-12-27 02:42:05,604][105692] Updated weights for policy 0, policy_version 1552247 (0.0008) [2023-12-27 02:42:05,647][105692] Updated weights for policy 0, policy_version 1552257 (0.0008) [2023-12-27 02:42:05,863][105620] Updated weights for policy 1, policy_version 1555441 (0.0007) [2023-12-27 02:42:05,932][105620] Updated weights for policy 1, policy_version 1555451 (0.0007) [2023-12-27 02:42:05,977][105620] Updated weights for policy 1, policy_version 1555461 (0.0010) [2023-12-27 02:42:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 795688960. Throughput: 0: 9723.4, 1: 9785.6. Samples: 795675488. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:42:06,063][104569] Avg episode reward: [(0, '8807.937'), (1, '8900.089')] [2023-12-27 02:42:06,349][105692] Updated weights for policy 0, policy_version 1552267 (0.0010) [2023-12-27 02:42:06,412][105692] Updated weights for policy 0, policy_version 1552277 (0.0011) [2023-12-27 02:42:06,465][105692] Updated weights for policy 0, policy_version 1552287 (0.0011) [2023-12-27 02:42:06,683][105620] Updated weights for policy 1, policy_version 1555471 (0.0009) [2023-12-27 02:42:06,742][105620] Updated weights for policy 1, policy_version 1555481 (0.0008) [2023-12-27 02:42:06,802][105620] Updated weights for policy 1, policy_version 1555491 (0.0008) [2023-12-27 02:42:07,194][105692] Updated weights for policy 0, policy_version 1552297 (0.0010) [2023-12-27 02:42:07,266][105692] Updated weights for policy 0, policy_version 1552307 (0.0010) [2023-12-27 02:42:07,333][105692] Updated weights for policy 0, policy_version 1552317 (0.0007) [2023-12-27 02:42:07,394][105692] Updated weights for policy 0, policy_version 1552327 (0.0009) [2023-12-27 02:42:07,485][105620] Updated weights for policy 1, policy_version 1555501 (0.0007) [2023-12-27 02:42:07,534][105620] Updated weights for policy 1, policy_version 1555511 (0.0005) [2023-12-27 02:42:07,579][105620] Updated weights for policy 1, policy_version 1555521 (0.0005) [2023-12-27 02:42:08,001][105692] Updated weights for policy 0, policy_version 1552337 (0.0010) [2023-12-27 02:42:08,050][105692] Updated weights for policy 0, policy_version 1552347 (0.0010) [2023-12-27 02:42:08,101][105692] Updated weights for policy 0, policy_version 1552357 (0.0010) [2023-12-27 02:42:08,122][105620] Updated weights for policy 1, policy_version 1555531 (0.0005) [2023-12-27 02:42:08,177][105620] Updated weights for policy 1, policy_version 1555541 (0.0006) [2023-12-27 02:42:08,230][105620] Updated weights for policy 1, policy_version 1555551 (0.0006) [2023-12-27 02:42:08,834][105620] Updated weights for policy 1, policy_version 1555561 (0.0005) [2023-12-27 02:42:08,853][105692] Updated weights for policy 0, policy_version 1552367 (0.0010) [2023-12-27 02:42:08,887][105620] Updated weights for policy 1, policy_version 1555571 (0.0005) [2023-12-27 02:42:08,908][105692] Updated weights for policy 0, policy_version 1552377 (0.0010) [2023-12-27 02:42:08,938][105620] Updated weights for policy 1, policy_version 1555581 (0.0006) [2023-12-27 02:42:08,963][105692] Updated weights for policy 0, policy_version 1552387 (0.0010) [2023-12-27 02:42:08,987][105620] Updated weights for policy 1, policy_version 1555591 (0.0009) [2023-12-27 02:42:09,710][105620] Updated weights for policy 1, policy_version 1555601 (0.0010) [2023-12-27 02:42:09,732][105692] Updated weights for policy 0, policy_version 1552397 (0.0010) [2023-12-27 02:42:09,771][105620] Updated weights for policy 1, policy_version 1555611 (0.0008) [2023-12-27 02:42:09,799][105692] Updated weights for policy 0, policy_version 1552407 (0.0011) [2023-12-27 02:42:09,838][105620] Updated weights for policy 1, policy_version 1555621 (0.0008) [2023-12-27 02:42:09,870][105692] Updated weights for policy 0, policy_version 1552417 (0.0008) [2023-12-27 02:42:10,518][105692] Updated weights for policy 0, policy_version 1552427 (0.0008) [2023-12-27 02:42:10,570][105692] Updated weights for policy 0, policy_version 1552437 (0.0006) [2023-12-27 02:42:10,625][105692] Updated weights for policy 0, policy_version 1552447 (0.0005) [2023-12-27 02:42:10,631][105620] Updated weights for policy 1, policy_version 1555631 (0.0009) [2023-12-27 02:42:10,684][105620] Updated weights for policy 1, policy_version 1555641 (0.0008) [2023-12-27 02:42:10,734][105620] Updated weights for policy 1, policy_version 1555651 (0.0009) [2023-12-27 02:42:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 795787264. Throughput: 0: 9752.4, 1: 9782.5. Samples: 795794572. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:42:11,062][104569] Avg episode reward: [(0, '8811.491'), (1, '9033.513')] [2023-12-27 02:42:11,310][105692] Updated weights for policy 0, policy_version 1552457 (0.0006) [2023-12-27 02:42:11,374][105692] Updated weights for policy 0, policy_version 1552467 (0.0009) [2023-12-27 02:42:11,438][105692] Updated weights for policy 0, policy_version 1552477 (0.0007) [2023-12-27 02:42:11,510][105692] Updated weights for policy 0, policy_version 1552487 (0.0007) [2023-12-27 02:42:11,516][105620] Updated weights for policy 1, policy_version 1555661 (0.0009) [2023-12-27 02:42:11,581][105620] Updated weights for policy 1, policy_version 1555671 (0.0010) [2023-12-27 02:42:11,585][105586] KL-divergence is very high: 109.0155 [2023-12-27 02:42:11,639][105586] KL-divergence is very high: 152.8228 [2023-12-27 02:42:11,646][105620] Updated weights for policy 1, policy_version 1555681 (0.0009) [2023-12-27 02:42:12,261][105692] Updated weights for policy 0, policy_version 1552497 (0.0009) [2023-12-27 02:42:12,326][105692] Updated weights for policy 0, policy_version 1552507 (0.0008) [2023-12-27 02:42:12,365][105620] Updated weights for policy 1, policy_version 1555691 (0.0008) [2023-12-27 02:42:12,393][105692] Updated weights for policy 0, policy_version 1552517 (0.0009) [2023-12-27 02:42:12,428][105620] Updated weights for policy 1, policy_version 1555701 (0.0008) [2023-12-27 02:42:12,479][105620] Updated weights for policy 1, policy_version 1555711 (0.0008) [2023-12-27 02:42:13,175][105692] Updated weights for policy 0, policy_version 1552527 (0.0007) [2023-12-27 02:42:13,232][105620] Updated weights for policy 1, policy_version 1555721 (0.0008) [2023-12-27 02:42:13,236][105692] Updated weights for policy 0, policy_version 1552537 (0.0008) [2023-12-27 02:42:13,291][105620] Updated weights for policy 1, policy_version 1555731 (0.0010) [2023-12-27 02:42:13,302][105692] Updated weights for policy 0, policy_version 1552547 (0.0008) [2023-12-27 02:42:13,344][105620] Updated weights for policy 1, policy_version 1555741 (0.0006) [2023-12-27 02:42:13,403][105620] Updated weights for policy 1, policy_version 1555751 (0.0010) [2023-12-27 02:42:13,968][105692] Updated weights for policy 0, policy_version 1552557 (0.0008) [2023-12-27 02:42:14,020][105692] Updated weights for policy 0, policy_version 1552567 (0.0005) [2023-12-27 02:42:14,079][105692] Updated weights for policy 0, policy_version 1552577 (0.0008) [2023-12-27 02:42:14,137][105620] Updated weights for policy 1, policy_version 1555761 (0.0008) [2023-12-27 02:42:14,195][105620] Updated weights for policy 1, policy_version 1555771 (0.0010) [2023-12-27 02:42:14,256][105620] Updated weights for policy 1, policy_version 1555781 (0.0007) [2023-12-27 02:42:14,686][105692] Updated weights for policy 0, policy_version 1552587 (0.0009) [2023-12-27 02:42:14,747][105692] Updated weights for policy 0, policy_version 1552597 (0.0008) [2023-12-27 02:42:14,812][105692] Updated weights for policy 0, policy_version 1552607 (0.0008) [2023-12-27 02:42:14,929][105620] Updated weights for policy 1, policy_version 1555791 (0.0009) [2023-12-27 02:42:14,983][105620] Updated weights for policy 1, policy_version 1555801 (0.0011) [2023-12-27 02:42:15,048][105620] Updated weights for policy 1, policy_version 1555811 (0.0011) [2023-12-27 02:42:15,486][105692] Updated weights for policy 0, policy_version 1552617 (0.0006) [2023-12-27 02:42:15,543][105692] Updated weights for policy 0, policy_version 1552627 (0.0010) [2023-12-27 02:42:15,605][105692] Updated weights for policy 0, policy_version 1552637 (0.0010) [2023-12-27 02:42:15,662][105692] Updated weights for policy 0, policy_version 1552647 (0.0010) [2023-12-27 02:42:15,805][105620] Updated weights for policy 1, policy_version 1555821 (0.0009) [2023-12-27 02:42:15,852][105620] Updated weights for policy 1, policy_version 1555831 (0.0010) [2023-12-27 02:42:15,903][105620] Updated weights for policy 1, policy_version 1555841 (0.0010) [2023-12-27 02:42:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 795885568. Throughput: 0: 9683.2, 1: 9804.7. Samples: 795850312. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:42:16,062][104569] Avg episode reward: [(0, '8448.564'), (1, '8859.693')] [2023-12-27 02:42:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001552648_397533184.pth... [2023-12-27 02:42:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001555848_398352384.pth... [2023-12-27 02:42:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001551496_397238272.pth [2023-12-27 02:42:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001554696_398057472.pth [2023-12-27 02:42:16,365][105692] Updated weights for policy 0, policy_version 1552657 (0.0006) [2023-12-27 02:42:16,420][105692] Updated weights for policy 0, policy_version 1552667 (0.0005) [2023-12-27 02:42:16,471][105692] Updated weights for policy 0, policy_version 1552677 (0.0006) [2023-12-27 02:42:16,678][105620] Updated weights for policy 1, policy_version 1555851 (0.0010) [2023-12-27 02:42:16,732][105620] Updated weights for policy 1, policy_version 1555861 (0.0010) [2023-12-27 02:42:16,791][105620] Updated weights for policy 1, policy_version 1555871 (0.0009) [2023-12-27 02:42:17,044][105692] Updated weights for policy 0, policy_version 1552687 (0.0006) [2023-12-27 02:42:17,098][105692] Updated weights for policy 0, policy_version 1552697 (0.0006) [2023-12-27 02:42:17,160][105692] Updated weights for policy 0, policy_version 1552707 (0.0008) [2023-12-27 02:42:17,386][105620] Updated weights for policy 1, policy_version 1555881 (0.0009) [2023-12-27 02:42:17,437][105620] Updated weights for policy 1, policy_version 1555891 (0.0005) [2023-12-27 02:42:17,493][105620] Updated weights for policy 1, policy_version 1555901 (0.0008) [2023-12-27 02:42:17,539][105620] Updated weights for policy 1, policy_version 1555911 (0.0009) [2023-12-27 02:42:17,872][105692] Updated weights for policy 0, policy_version 1552717 (0.0010) [2023-12-27 02:42:17,924][105692] Updated weights for policy 0, policy_version 1552727 (0.0009) [2023-12-27 02:42:17,976][105692] Updated weights for policy 0, policy_version 1552737 (0.0005) [2023-12-27 02:42:18,278][105620] Updated weights for policy 1, policy_version 1555921 (0.0006) [2023-12-27 02:42:18,333][105620] Updated weights for policy 1, policy_version 1555931 (0.0006) [2023-12-27 02:42:18,394][105620] Updated weights for policy 1, policy_version 1555941 (0.0010) [2023-12-27 02:42:18,627][105692] Updated weights for policy 0, policy_version 1552747 (0.0007) [2023-12-27 02:42:18,679][105692] Updated weights for policy 0, policy_version 1552758 (0.0010) [2023-12-27 02:42:18,732][105692] Updated weights for policy 0, policy_version 1552769 (0.0010) [2023-12-27 02:42:18,957][105620] Updated weights for policy 1, policy_version 1555951 (0.0007) [2023-12-27 02:42:19,010][105620] Updated weights for policy 1, policy_version 1555961 (0.0005) [2023-12-27 02:42:19,060][105620] Updated weights for policy 1, policy_version 1555971 (0.0005) [2023-12-27 02:42:19,572][105692] Updated weights for policy 0, policy_version 1552780 (0.0008) [2023-12-27 02:42:19,620][105692] Updated weights for policy 0, policy_version 1552790 (0.0005) [2023-12-27 02:42:19,675][105692] Updated weights for policy 0, policy_version 1552800 (0.0006) [2023-12-27 02:42:19,748][105620] Updated weights for policy 1, policy_version 1555981 (0.0008) [2023-12-27 02:42:19,816][105620] Updated weights for policy 1, policy_version 1555991 (0.0011) [2023-12-27 02:42:19,875][105620] Updated weights for policy 1, policy_version 1556001 (0.0011) [2023-12-27 02:42:20,400][105692] Updated weights for policy 0, policy_version 1552810 (0.0007) [2023-12-27 02:42:20,462][105692] Updated weights for policy 0, policy_version 1552820 (0.0005) [2023-12-27 02:42:20,529][105692] Updated weights for policy 0, policy_version 1552830 (0.0006) [2023-12-27 02:42:20,600][105692] Updated weights for policy 0, policy_version 1552840 (0.0007) [2023-12-27 02:42:20,656][105620] Updated weights for policy 1, policy_version 1556011 (0.0009) [2023-12-27 02:42:20,720][105620] Updated weights for policy 1, policy_version 1556021 (0.0011) [2023-12-27 02:42:20,789][105620] Updated weights for policy 1, policy_version 1556031 (0.0010) [2023-12-27 02:42:21,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 795983872. Throughput: 0: 9731.7, 1: 9827.1. Samples: 795972736. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:42:21,063][104569] Avg episode reward: [(0, '8350.318'), (1, '8818.666')] [2023-12-27 02:42:21,301][105692] Updated weights for policy 0, policy_version 1552850 (0.0008) [2023-12-27 02:42:21,370][105692] Updated weights for policy 0, policy_version 1552860 (0.0010) [2023-12-27 02:42:21,437][105692] Updated weights for policy 0, policy_version 1552870 (0.0009) [2023-12-27 02:42:21,545][105620] Updated weights for policy 1, policy_version 1556041 (0.0009) [2023-12-27 02:42:21,608][105620] Updated weights for policy 1, policy_version 1556051 (0.0006) [2023-12-27 02:42:21,669][105620] Updated weights for policy 1, policy_version 1556061 (0.0011) [2023-12-27 02:42:21,730][105620] Updated weights for policy 1, policy_version 1556071 (0.0009) [2023-12-27 02:42:22,259][105692] Updated weights for policy 0, policy_version 1552880 (0.0009) [2023-12-27 02:42:22,319][105692] Updated weights for policy 0, policy_version 1552890 (0.0009) [2023-12-27 02:42:22,387][105692] Updated weights for policy 0, policy_version 1552900 (0.0009) [2023-12-27 02:42:22,436][105620] Updated weights for policy 1, policy_version 1556081 (0.0008) [2023-12-27 02:42:22,496][105620] Updated weights for policy 1, policy_version 1556091 (0.0009) [2023-12-27 02:42:22,562][105620] Updated weights for policy 1, policy_version 1556101 (0.0008) [2023-12-27 02:42:23,172][105692] Updated weights for policy 0, policy_version 1552910 (0.0009) [2023-12-27 02:42:23,224][105692] Updated weights for policy 0, policy_version 1552920 (0.0009) [2023-12-27 02:42:23,277][105692] Updated weights for policy 0, policy_version 1552930 (0.0008) [2023-12-27 02:42:23,282][105620] Updated weights for policy 1, policy_version 1556111 (0.0007) [2023-12-27 02:42:23,351][105620] Updated weights for policy 1, policy_version 1556121 (0.0005) [2023-12-27 02:42:23,419][105620] Updated weights for policy 1, policy_version 1556131 (0.0006) [2023-12-27 02:42:23,965][105620] Updated weights for policy 1, policy_version 1556141 (0.0007) [2023-12-27 02:42:24,009][105620] Updated weights for policy 1, policy_version 1556151 (0.0005) [2023-12-27 02:42:24,059][105620] Updated weights for policy 1, policy_version 1556161 (0.0006) [2023-12-27 02:42:24,121][105692] Updated weights for policy 0, policy_version 1552940 (0.0009) [2023-12-27 02:42:24,171][105692] Updated weights for policy 0, policy_version 1552950 (0.0009) [2023-12-27 02:42:24,225][105692] Updated weights for policy 0, policy_version 1552960 (0.0009) [2023-12-27 02:42:24,715][105620] Updated weights for policy 1, policy_version 1556171 (0.0008) [2023-12-27 02:42:24,778][105620] Updated weights for policy 1, policy_version 1556181 (0.0005) [2023-12-27 02:42:24,829][105620] Updated weights for policy 1, policy_version 1556191 (0.0007) [2023-12-27 02:42:25,085][105692] Updated weights for policy 0, policy_version 1552970 (0.0010) [2023-12-27 02:42:25,139][105692] Updated weights for policy 0, policy_version 1552980 (0.0010) [2023-12-27 02:42:25,195][105692] Updated weights for policy 0, policy_version 1552990 (0.0009) [2023-12-27 02:42:25,256][105692] Updated weights for policy 0, policy_version 1553000 (0.0010) [2023-12-27 02:42:25,397][105620] Updated weights for policy 1, policy_version 1556201 (0.0007) [2023-12-27 02:42:25,453][105620] Updated weights for policy 1, policy_version 1556211 (0.0006) [2023-12-27 02:42:25,499][105620] Updated weights for policy 1, policy_version 1556221 (0.0008) [2023-12-27 02:42:25,552][105620] Updated weights for policy 1, policy_version 1556231 (0.0007) [2023-12-27 02:42:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 796073984. Throughput: 0: 9722.2, 1: 9849.2. Samples: 796088228. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:42:26,062][104569] Avg episode reward: [(0, '8439.569'), (1, '9264.802')] [2023-12-27 02:42:26,127][105620] Updated weights for policy 1, policy_version 1556241 (0.0006) [2023-12-27 02:42:26,135][105692] Updated weights for policy 0, policy_version 1553010 (0.0007) [2023-12-27 02:42:26,186][105692] Updated weights for policy 0, policy_version 1553020 (0.0007) [2023-12-27 02:42:26,190][105620] Updated weights for policy 1, policy_version 1556251 (0.0005) [2023-12-27 02:42:26,231][105692] Updated weights for policy 0, policy_version 1553030 (0.0007) [2023-12-27 02:42:26,252][105620] Updated weights for policy 1, policy_version 1556261 (0.0005) [2023-12-27 02:42:26,883][105692] Updated weights for policy 0, policy_version 1553040 (0.0009) [2023-12-27 02:42:26,940][105692] Updated weights for policy 0, policy_version 1553050 (0.0009) [2023-12-27 02:42:26,961][105620] Updated weights for policy 1, policy_version 1556271 (0.0005) [2023-12-27 02:42:26,996][105692] Updated weights for policy 0, policy_version 1553060 (0.0009) [2023-12-27 02:42:27,010][105620] Updated weights for policy 1, policy_version 1556281 (0.0005) [2023-12-27 02:42:27,059][105620] Updated weights for policy 1, policy_version 1556291 (0.0005) [2023-12-27 02:42:27,563][105620] Updated weights for policy 1, policy_version 1556301 (0.0005) [2023-12-27 02:42:27,620][105620] Updated weights for policy 1, policy_version 1556311 (0.0005) [2023-12-27 02:42:27,667][105620] Updated weights for policy 1, policy_version 1556321 (0.0005) [2023-12-27 02:42:27,859][105692] Updated weights for policy 0, policy_version 1553070 (0.0010) [2023-12-27 02:42:27,920][105692] Updated weights for policy 0, policy_version 1553080 (0.0006) [2023-12-27 02:42:27,972][105692] Updated weights for policy 0, policy_version 1553090 (0.0005) [2023-12-27 02:42:28,215][105620] Updated weights for policy 1, policy_version 1556331 (0.0005) [2023-12-27 02:42:28,271][105620] Updated weights for policy 1, policy_version 1556341 (0.0005) [2023-12-27 02:42:28,334][105620] Updated weights for policy 1, policy_version 1556351 (0.0006) [2023-12-27 02:42:28,344][105586] KL-divergence is very high: 108.2201 [2023-12-27 02:42:28,581][105692] Updated weights for policy 0, policy_version 1553100 (0.0006) [2023-12-27 02:42:28,643][105692] Updated weights for policy 0, policy_version 1553110 (0.0006) [2023-12-27 02:42:28,699][105692] Updated weights for policy 0, policy_version 1553120 (0.0006) [2023-12-27 02:42:28,991][105620] Updated weights for policy 1, policy_version 1556361 (0.0006) [2023-12-27 02:42:29,049][105620] Updated weights for policy 1, policy_version 1556371 (0.0005) [2023-12-27 02:42:29,107][105620] Updated weights for policy 1, policy_version 1556381 (0.0005) [2023-12-27 02:42:29,155][105620] Updated weights for policy 1, policy_version 1556391 (0.0005) [2023-12-27 02:42:29,256][105692] Updated weights for policy 0, policy_version 1553130 (0.0006) [2023-12-27 02:42:29,323][105692] Updated weights for policy 0, policy_version 1553140 (0.0007) [2023-12-27 02:42:29,391][105692] Updated weights for policy 0, policy_version 1553150 (0.0011) [2023-12-27 02:42:29,458][105692] Updated weights for policy 0, policy_version 1553160 (0.0011) [2023-12-27 02:42:29,832][105620] Updated weights for policy 1, policy_version 1556401 (0.0008) [2023-12-27 02:42:29,899][105620] Updated weights for policy 1, policy_version 1556411 (0.0008) [2023-12-27 02:42:29,966][105620] Updated weights for policy 1, policy_version 1556421 (0.0008) [2023-12-27 02:42:30,164][105692] Updated weights for policy 0, policy_version 1553170 (0.0010) [2023-12-27 02:42:30,212][105692] Updated weights for policy 0, policy_version 1553180 (0.0010) [2023-12-27 02:42:30,263][105692] Updated weights for policy 0, policy_version 1553190 (0.0010) [2023-12-27 02:42:30,628][105620] Updated weights for policy 1, policy_version 1556431 (0.0006) [2023-12-27 02:42:30,682][105620] Updated weights for policy 1, policy_version 1556441 (0.0008) [2023-12-27 02:42:30,739][105620] Updated weights for policy 1, policy_version 1556451 (0.0006) [2023-12-27 02:42:30,943][105692] Updated weights for policy 0, policy_version 1553200 (0.0011) [2023-12-27 02:42:31,001][105692] Updated weights for policy 0, policy_version 1553210 (0.0008) [2023-12-27 02:42:31,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 796180480. Throughput: 0: 9708.6, 1: 9985.6. Samples: 796151096. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:42:31,062][104569] Avg episode reward: [(0, '8624.849'), (1, '9263.878')] [2023-12-27 02:42:31,063][105692] Updated weights for policy 0, policy_version 1553220 (0.0008) [2023-12-27 02:42:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001556456_398508032.pth... [2023-12-27 02:42:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001555272_398204928.pth [2023-12-27 02:42:31,088][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001553224_397680640.pth... [2023-12-27 02:42:31,092][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001552072_397385728.pth [2023-12-27 02:42:31,479][105620] Updated weights for policy 1, policy_version 1556461 (0.0007) [2023-12-27 02:42:31,526][105620] Updated weights for policy 1, policy_version 1556471 (0.0009) [2023-12-27 02:42:31,591][105620] Updated weights for policy 1, policy_version 1556481 (0.0009) [2023-12-27 02:42:31,750][105692] Updated weights for policy 0, policy_version 1553230 (0.0006) [2023-12-27 02:42:31,794][105692] Updated weights for policy 0, policy_version 1553240 (0.0005) [2023-12-27 02:42:31,861][105692] Updated weights for policy 0, policy_version 1553250 (0.0006) [2023-12-27 02:42:32,375][105620] Updated weights for policy 1, policy_version 1556491 (0.0009) [2023-12-27 02:42:32,433][105620] Updated weights for policy 1, policy_version 1556501 (0.0010) [2023-12-27 02:42:32,487][105620] Updated weights for policy 1, policy_version 1556511 (0.0009) [2023-12-27 02:42:32,560][105692] Updated weights for policy 0, policy_version 1553260 (0.0008) [2023-12-27 02:42:32,614][105692] Updated weights for policy 0, policy_version 1553270 (0.0005) [2023-12-27 02:42:32,663][105692] Updated weights for policy 0, policy_version 1553280 (0.0008) [2023-12-27 02:42:33,263][105620] Updated weights for policy 1, policy_version 1556521 (0.0008) [2023-12-27 02:42:33,323][105620] Updated weights for policy 1, policy_version 1556531 (0.0008) [2023-12-27 02:42:33,375][105620] Updated weights for policy 1, policy_version 1556541 (0.0008) [2023-12-27 02:42:33,395][105692] Updated weights for policy 0, policy_version 1553290 (0.0009) [2023-12-27 02:42:33,429][105620] Updated weights for policy 1, policy_version 1556551 (0.0008) [2023-12-27 02:42:33,448][105692] Updated weights for policy 0, policy_version 1553300 (0.0007) [2023-12-27 02:42:33,495][105692] Updated weights for policy 0, policy_version 1553310 (0.0010) [2023-12-27 02:42:33,545][105692] Updated weights for policy 0, policy_version 1553320 (0.0008) [2023-12-27 02:42:34,118][105692] Updated weights for policy 0, policy_version 1553330 (0.0005) [2023-12-27 02:42:34,179][105692] Updated weights for policy 0, policy_version 1553340 (0.0008) [2023-12-27 02:42:34,239][105692] Updated weights for policy 0, policy_version 1553350 (0.0006) [2023-12-27 02:42:34,291][105620] Updated weights for policy 1, policy_version 1556561 (0.0008) [2023-12-27 02:42:34,352][105620] Updated weights for policy 1, policy_version 1556571 (0.0009) [2023-12-27 02:42:34,411][105620] Updated weights for policy 1, policy_version 1556581 (0.0009) [2023-12-27 02:42:34,817][105692] Updated weights for policy 0, policy_version 1553360 (0.0008) [2023-12-27 02:42:34,884][105692] Updated weights for policy 0, policy_version 1553370 (0.0010) [2023-12-27 02:42:34,949][105692] Updated weights for policy 0, policy_version 1553380 (0.0010) [2023-12-27 02:42:35,238][105620] Updated weights for policy 1, policy_version 1556591 (0.0008) [2023-12-27 02:42:35,296][105620] Updated weights for policy 1, policy_version 1556601 (0.0008) [2023-12-27 02:42:35,348][105620] Updated weights for policy 1, policy_version 1556611 (0.0008) [2023-12-27 02:42:35,665][105692] Updated weights for policy 0, policy_version 1553390 (0.0010) [2023-12-27 02:42:35,716][105692] Updated weights for policy 0, policy_version 1553400 (0.0010) [2023-12-27 02:42:35,771][105692] Updated weights for policy 0, policy_version 1553410 (0.0010) [2023-12-27 02:42:35,969][105620] Updated weights for policy 1, policy_version 1556621 (0.0008) [2023-12-27 02:42:36,027][105620] Updated weights for policy 1, policy_version 1556631 (0.0006) [2023-12-27 02:42:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 796278784. Throughput: 0: 9772.3, 1: 9958.0. Samples: 796269748. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:42:36,063][104569] Avg episode reward: [(0, '8716.685'), (1, '9080.086')] [2023-12-27 02:42:36,083][105620] Updated weights for policy 1, policy_version 1556641 (0.0011) [2023-12-27 02:42:36,496][105692] Updated weights for policy 0, policy_version 1553420 (0.0008) [2023-12-27 02:42:36,562][105692] Updated weights for policy 0, policy_version 1553430 (0.0009) [2023-12-27 02:42:36,626][105692] Updated weights for policy 0, policy_version 1553440 (0.0008) [2023-12-27 02:42:36,799][105620] Updated weights for policy 1, policy_version 1556651 (0.0011) [2023-12-27 02:42:36,855][105620] Updated weights for policy 1, policy_version 1556661 (0.0011) [2023-12-27 02:42:36,907][105620] Updated weights for policy 1, policy_version 1556671 (0.0010) [2023-12-27 02:42:37,245][105692] Updated weights for policy 0, policy_version 1553450 (0.0009) [2023-12-27 02:42:37,297][105692] Updated weights for policy 0, policy_version 1553460 (0.0010) [2023-12-27 02:42:37,349][105692] Updated weights for policy 0, policy_version 1553470 (0.0010) [2023-12-27 02:42:37,401][105692] Updated weights for policy 0, policy_version 1553480 (0.0010) [2023-12-27 02:42:37,643][105620] Updated weights for policy 1, policy_version 1556681 (0.0011) [2023-12-27 02:42:37,703][105620] Updated weights for policy 1, policy_version 1556691 (0.0011) [2023-12-27 02:42:37,773][105620] Updated weights for policy 1, policy_version 1556701 (0.0011) [2023-12-27 02:42:37,832][105620] Updated weights for policy 1, policy_version 1556711 (0.0011) [2023-12-27 02:42:38,041][105692] Updated weights for policy 0, policy_version 1553490 (0.0006) [2023-12-27 02:42:38,096][105692] Updated weights for policy 0, policy_version 1553500 (0.0006) [2023-12-27 02:42:38,153][105692] Updated weights for policy 0, policy_version 1553510 (0.0005) [2023-12-27 02:42:38,499][105620] Updated weights for policy 1, policy_version 1556721 (0.0010) [2023-12-27 02:42:38,554][105620] Updated weights for policy 1, policy_version 1556731 (0.0009) [2023-12-27 02:42:38,608][105620] Updated weights for policy 1, policy_version 1556741 (0.0010) [2023-12-27 02:42:38,837][105692] Updated weights for policy 0, policy_version 1553520 (0.0008) [2023-12-27 02:42:38,886][105692] Updated weights for policy 0, policy_version 1553530 (0.0009) [2023-12-27 02:42:38,937][105692] Updated weights for policy 0, policy_version 1553540 (0.0009) [2023-12-27 02:42:39,300][105620] Updated weights for policy 1, policy_version 1556751 (0.0006) [2023-12-27 02:42:39,364][105620] Updated weights for policy 1, policy_version 1556761 (0.0008) [2023-12-27 02:42:39,430][105620] Updated weights for policy 1, policy_version 1556771 (0.0007) [2023-12-27 02:42:39,867][105692] Updated weights for policy 0, policy_version 1553550 (0.0008) [2023-12-27 02:42:39,930][105692] Updated weights for policy 0, policy_version 1553560 (0.0009) [2023-12-27 02:42:39,992][105692] Updated weights for policy 0, policy_version 1553570 (0.0009) [2023-12-27 02:42:40,070][105620] Updated weights for policy 1, policy_version 1556781 (0.0008) [2023-12-27 02:42:40,128][105620] Updated weights for policy 1, policy_version 1556791 (0.0008) [2023-12-27 02:42:40,178][105620] Updated weights for policy 1, policy_version 1556801 (0.0009) [2023-12-27 02:42:40,749][105692] Updated weights for policy 0, policy_version 1553580 (0.0009) [2023-12-27 02:42:40,808][105692] Updated weights for policy 0, policy_version 1553590 (0.0009) [2023-12-27 02:42:40,868][105692] Updated weights for policy 0, policy_version 1553600 (0.0009) [2023-12-27 02:42:40,945][105620] Updated weights for policy 1, policy_version 1556811 (0.0009) [2023-12-27 02:42:40,997][105620] Updated weights for policy 1, policy_version 1556821 (0.0009) [2023-12-27 02:42:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 796377088. Throughput: 0: 9723.8, 1: 9958.4. Samples: 796387704. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:42:41,063][104569] Avg episode reward: [(0, '8719.934'), (1, '9172.216')] [2023-12-27 02:42:41,063][105620] Updated weights for policy 1, policy_version 1556831 (0.0007) [2023-12-27 02:42:41,691][105692] Updated weights for policy 0, policy_version 1553610 (0.0009) [2023-12-27 02:42:41,761][105692] Updated weights for policy 0, policy_version 1553620 (0.0008) [2023-12-27 02:42:41,824][105692] Updated weights for policy 0, policy_version 1553630 (0.0008) [2023-12-27 02:42:41,880][105620] Updated weights for policy 1, policy_version 1556841 (0.0008) [2023-12-27 02:42:41,882][105692] Updated weights for policy 0, policy_version 1553640 (0.0008) [2023-12-27 02:42:41,947][105620] Updated weights for policy 1, policy_version 1556851 (0.0009) [2023-12-27 02:42:42,002][105620] Updated weights for policy 1, policy_version 1556861 (0.0009) [2023-12-27 02:42:42,061][105620] Updated weights for policy 1, policy_version 1556871 (0.0009) [2023-12-27 02:42:42,648][105692] Updated weights for policy 0, policy_version 1553650 (0.0009) [2023-12-27 02:42:42,699][105692] Updated weights for policy 0, policy_version 1553660 (0.0008) [2023-12-27 02:42:42,744][105692] Updated weights for policy 0, policy_version 1553670 (0.0005) [2023-12-27 02:42:42,751][105620] Updated weights for policy 1, policy_version 1556881 (0.0009) [2023-12-27 02:42:42,808][105620] Updated weights for policy 1, policy_version 1556891 (0.0009) [2023-12-27 02:42:42,864][105620] Updated weights for policy 1, policy_version 1556901 (0.0009) [2023-12-27 02:42:43,359][105692] Updated weights for policy 0, policy_version 1553680 (0.0005) [2023-12-27 02:42:43,414][105692] Updated weights for policy 0, policy_version 1553690 (0.0006) [2023-12-27 02:42:43,466][105692] Updated weights for policy 0, policy_version 1553700 (0.0008) [2023-12-27 02:42:43,705][105620] Updated weights for policy 1, policy_version 1556911 (0.0010) [2023-12-27 02:42:43,762][105620] Updated weights for policy 1, policy_version 1556922 (0.0010) [2023-12-27 02:42:43,818][105620] Updated weights for policy 1, policy_version 1556932 (0.0009) [2023-12-27 02:42:44,038][105692] Updated weights for policy 0, policy_version 1553710 (0.0007) [2023-12-27 02:42:44,101][105692] Updated weights for policy 0, policy_version 1553720 (0.0010) [2023-12-27 02:42:44,153][105692] Updated weights for policy 0, policy_version 1553730 (0.0010) [2023-12-27 02:42:44,686][105620] Updated weights for policy 1, policy_version 1556942 (0.0010) [2023-12-27 02:42:44,738][105620] Updated weights for policy 1, policy_version 1556952 (0.0009) [2023-12-27 02:42:44,764][105692] Updated weights for policy 0, policy_version 1553740 (0.0010) [2023-12-27 02:42:44,797][105620] Updated weights for policy 1, policy_version 1556962 (0.0007) [2023-12-27 02:42:44,827][105692] Updated weights for policy 0, policy_version 1553750 (0.0010) [2023-12-27 02:42:44,899][105692] Updated weights for policy 0, policy_version 1553760 (0.0011) [2023-12-27 02:42:45,571][105620] Updated weights for policy 1, policy_version 1556972 (0.0007) [2023-12-27 02:42:45,607][105692] Updated weights for policy 0, policy_version 1553770 (0.0006) [2023-12-27 02:42:45,636][105620] Updated weights for policy 1, policy_version 1556982 (0.0007) [2023-12-27 02:42:45,666][105692] Updated weights for policy 0, policy_version 1553780 (0.0006) [2023-12-27 02:42:45,698][105620] Updated weights for policy 1, policy_version 1556992 (0.0005) [2023-12-27 02:42:45,724][105692] Updated weights for policy 0, policy_version 1553790 (0.0006) [2023-12-27 02:42:45,775][105692] Updated weights for policy 0, policy_version 1553800 (0.0007) [2023-12-27 02:42:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 796475392. Throughput: 0: 9703.8, 1: 9919.9. Samples: 796442792. Policy #0 lag: (min: 33.0, avg: 46.6, max: 48.0) [2023-12-27 02:42:46,062][104569] Avg episode reward: [(0, '8810.109'), (1, '9355.638')] [2023-12-27 02:42:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001553800_397828096.pth... [2023-12-27 02:42:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001557000_398647296.pth... [2023-12-27 02:42:46,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001552648_397533184.pth [2023-12-27 02:42:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001555848_398352384.pth [2023-12-27 02:42:46,343][105620] Updated weights for policy 1, policy_version 1557002 (0.0006) [2023-12-27 02:42:46,398][105620] Updated weights for policy 1, policy_version 1557012 (0.0010) [2023-12-27 02:42:46,456][105620] Updated weights for policy 1, policy_version 1557022 (0.0010) [2023-12-27 02:42:46,464][105692] Updated weights for policy 0, policy_version 1553810 (0.0010) [2023-12-27 02:42:46,511][105620] Updated weights for policy 1, policy_version 1557032 (0.0010) [2023-12-27 02:42:46,511][105692] Updated weights for policy 0, policy_version 1553820 (0.0010) [2023-12-27 02:42:46,556][105692] Updated weights for policy 0, policy_version 1553830 (0.0010) [2023-12-27 02:42:47,143][105620] Updated weights for policy 1, policy_version 1557042 (0.0006) [2023-12-27 02:42:47,199][105620] Updated weights for policy 1, policy_version 1557052 (0.0006) [2023-12-27 02:42:47,246][105620] Updated weights for policy 1, policy_version 1557062 (0.0005) [2023-12-27 02:42:47,363][105692] Updated weights for policy 0, policy_version 1553840 (0.0010) [2023-12-27 02:42:47,410][105692] Updated weights for policy 0, policy_version 1553850 (0.0012) [2023-12-27 02:42:47,454][105692] Updated weights for policy 0, policy_version 1553860 (0.0010) [2023-12-27 02:42:47,925][105620] Updated weights for policy 1, policy_version 1557072 (0.0008) [2023-12-27 02:42:47,982][105620] Updated weights for policy 1, policy_version 1557082 (0.0010) [2023-12-27 02:42:48,042][105620] Updated weights for policy 1, policy_version 1557093 (0.0008) [2023-12-27 02:42:48,147][105692] Updated weights for policy 0, policy_version 1553870 (0.0009) [2023-12-27 02:42:48,208][105692] Updated weights for policy 0, policy_version 1553880 (0.0009) [2023-12-27 02:42:48,259][105692] Updated weights for policy 0, policy_version 1553890 (0.0009) [2023-12-27 02:42:48,684][105620] Updated weights for policy 1, policy_version 1557103 (0.0005) [2023-12-27 02:42:48,737][105620] Updated weights for policy 1, policy_version 1557113 (0.0006) [2023-12-27 02:42:48,790][105620] Updated weights for policy 1, policy_version 1557123 (0.0005) [2023-12-27 02:42:49,065][105692] Updated weights for policy 0, policy_version 1553900 (0.0009) [2023-12-27 02:42:49,117][105692] Updated weights for policy 0, policy_version 1553910 (0.0010) [2023-12-27 02:42:49,166][105692] Updated weights for policy 0, policy_version 1553920 (0.0011) [2023-12-27 02:42:49,474][105620] Updated weights for policy 1, policy_version 1557133 (0.0008) [2023-12-27 02:42:49,525][105620] Updated weights for policy 1, policy_version 1557143 (0.0010) [2023-12-27 02:42:49,576][105620] Updated weights for policy 1, policy_version 1557153 (0.0010) [2023-12-27 02:42:49,936][105692] Updated weights for policy 0, policy_version 1553930 (0.0012) [2023-12-27 02:42:50,004][105692] Updated weights for policy 0, policy_version 1553940 (0.0011) [2023-12-27 02:42:50,064][105692] Updated weights for policy 0, policy_version 1553950 (0.0011) [2023-12-27 02:42:50,123][105692] Updated weights for policy 0, policy_version 1553960 (0.0010) [2023-12-27 02:42:50,368][105620] Updated weights for policy 1, policy_version 1557163 (0.0010) [2023-12-27 02:42:50,425][105620] Updated weights for policy 1, policy_version 1557173 (0.0009) [2023-12-27 02:42:50,477][105620] Updated weights for policy 1, policy_version 1557183 (0.0008) [2023-12-27 02:42:50,839][105692] Updated weights for policy 0, policy_version 1553970 (0.0011) [2023-12-27 02:42:50,898][105692] Updated weights for policy 0, policy_version 1553980 (0.0007) [2023-12-27 02:42:50,960][105692] Updated weights for policy 0, policy_version 1553990 (0.0005) [2023-12-27 02:42:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 796573696. Throughput: 0: 9780.5, 1: 9919.2. Samples: 796561976. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:42:51,062][104569] Avg episode reward: [(0, '8620.941'), (1, '9263.393')] [2023-12-27 02:42:51,185][105620] Updated weights for policy 1, policy_version 1557193 (0.0008) [2023-12-27 02:42:51,237][105620] Updated weights for policy 1, policy_version 1557203 (0.0008) [2023-12-27 02:42:51,299][105620] Updated weights for policy 1, policy_version 1557213 (0.0008) [2023-12-27 02:42:51,352][105620] Updated weights for policy 1, policy_version 1557223 (0.0008) [2023-12-27 02:42:51,566][105692] Updated weights for policy 0, policy_version 1554000 (0.0010) [2023-12-27 02:42:51,621][105692] Updated weights for policy 0, policy_version 1554010 (0.0008) [2023-12-27 02:42:51,688][105692] Updated weights for policy 0, policy_version 1554020 (0.0006) [2023-12-27 02:42:52,120][105620] Updated weights for policy 1, policy_version 1557233 (0.0010) [2023-12-27 02:42:52,176][105620] Updated weights for policy 1, policy_version 1557243 (0.0009) [2023-12-27 02:42:52,235][105620] Updated weights for policy 1, policy_version 1557253 (0.0009) [2023-12-27 02:42:52,364][105692] Updated weights for policy 0, policy_version 1554030 (0.0009) [2023-12-27 02:42:52,425][105692] Updated weights for policy 0, policy_version 1554040 (0.0008) [2023-12-27 02:42:52,487][105692] Updated weights for policy 0, policy_version 1554050 (0.0009) [2023-12-27 02:42:53,056][105620] Updated weights for policy 1, policy_version 1557263 (0.0010) [2023-12-27 02:42:53,114][105620] Updated weights for policy 1, policy_version 1557273 (0.0010) [2023-12-27 02:42:53,150][105692] Updated weights for policy 0, policy_version 1554060 (0.0007) [2023-12-27 02:42:53,164][105620] Updated weights for policy 1, policy_version 1557283 (0.0009) [2023-12-27 02:42:53,207][105692] Updated weights for policy 0, policy_version 1554070 (0.0005) [2023-12-27 02:42:53,262][105692] Updated weights for policy 0, policy_version 1554080 (0.0005) [2023-12-27 02:42:53,773][105692] Updated weights for policy 0, policy_version 1554090 (0.0005) [2023-12-27 02:42:53,824][105692] Updated weights for policy 0, policy_version 1554100 (0.0005) [2023-12-27 02:42:53,874][105692] Updated weights for policy 0, policy_version 1554110 (0.0005) [2023-12-27 02:42:53,937][105692] Updated weights for policy 0, policy_version 1554120 (0.0005) [2023-12-27 02:42:54,070][105620] Updated weights for policy 1, policy_version 1557293 (0.0009) [2023-12-27 02:42:54,115][105620] Updated weights for policy 1, policy_version 1557303 (0.0010) [2023-12-27 02:42:54,163][105620] Updated weights for policy 1, policy_version 1557313 (0.0009) [2023-12-27 02:42:54,505][105692] Updated weights for policy 0, policy_version 1554130 (0.0007) [2023-12-27 02:42:54,564][105692] Updated weights for policy 0, policy_version 1554140 (0.0008) [2023-12-27 02:42:54,620][105692] Updated weights for policy 0, policy_version 1554150 (0.0008) [2023-12-27 02:42:54,913][105620] Updated weights for policy 1, policy_version 1557323 (0.0007) [2023-12-27 02:42:54,978][105586] KL-divergence is very high: 177.0471 [2023-12-27 02:42:54,985][105620] Updated weights for policy 1, policy_version 1557333 (0.0008) [2023-12-27 02:42:55,015][105586] KL-divergence is very high: 142.9422 [2023-12-27 02:42:55,038][105586] KL-divergence is very high: 331.3683 [2023-12-27 02:42:55,058][105620] Updated weights for policy 1, policy_version 1557343 (0.0009) [2023-12-27 02:42:55,074][105586] KL-divergence is very high: 168.1203 [2023-12-27 02:42:55,095][105586] KL-divergence is very high: 364.1226 [2023-12-27 02:42:55,302][105692] Updated weights for policy 0, policy_version 1554160 (0.0006) [2023-12-27 02:42:55,359][105692] Updated weights for policy 0, policy_version 1554170 (0.0005) [2023-12-27 02:42:55,412][105692] Updated weights for policy 0, policy_version 1554180 (0.0005) [2023-12-27 02:42:55,758][105620] Updated weights for policy 1, policy_version 1557353 (0.0011) [2023-12-27 02:42:55,830][105620] Updated weights for policy 1, policy_version 1557363 (0.0010) [2023-12-27 02:42:55,888][105620] Updated weights for policy 1, policy_version 1557373 (0.0010) [2023-12-27 02:42:55,945][105620] Updated weights for policy 1, policy_version 1557383 (0.0010) [2023-12-27 02:42:55,968][105692] Updated weights for policy 0, policy_version 1554190 (0.0006) [2023-12-27 02:42:56,024][105692] Updated weights for policy 0, policy_version 1554200 (0.0006) [2023-12-27 02:42:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 796672000. Throughput: 0: 9888.0, 1: 9808.2. Samples: 796680900. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:42:56,062][104569] Avg episode reward: [(0, '8713.007'), (1, '8989.707')] [2023-12-27 02:42:56,077][105692] Updated weights for policy 0, policy_version 1554210 (0.0006) [2023-12-27 02:42:56,669][105620] Updated weights for policy 1, policy_version 1557393 (0.0011) [2023-12-27 02:42:56,722][105620] Updated weights for policy 1, policy_version 1557403 (0.0010) [2023-12-27 02:42:56,778][105620] Updated weights for policy 1, policy_version 1557413 (0.0010) [2023-12-27 02:42:56,787][105692] Updated weights for policy 0, policy_version 1554220 (0.0008) [2023-12-27 02:42:56,841][105692] Updated weights for policy 0, policy_version 1554230 (0.0008) [2023-12-27 02:42:56,895][105692] Updated weights for policy 0, policy_version 1554240 (0.0008) [2023-12-27 02:42:57,427][105620] Updated weights for policy 1, policy_version 1557423 (0.0010) [2023-12-27 02:42:57,471][105620] Updated weights for policy 1, policy_version 1557433 (0.0010) [2023-12-27 02:42:57,515][105620] Updated weights for policy 1, policy_version 1557443 (0.0010) [2023-12-27 02:42:57,709][105692] Updated weights for policy 0, policy_version 1554250 (0.0008) [2023-12-27 02:42:57,766][105692] Updated weights for policy 0, policy_version 1554260 (0.0008) [2023-12-27 02:42:57,820][105692] Updated weights for policy 0, policy_version 1554270 (0.0008) [2023-12-27 02:42:57,873][105692] Updated weights for policy 0, policy_version 1554280 (0.0007) [2023-12-27 02:42:58,246][105620] Updated weights for policy 1, policy_version 1557453 (0.0009) [2023-12-27 02:42:58,306][105620] Updated weights for policy 1, policy_version 1557463 (0.0008) [2023-12-27 02:42:58,373][105620] Updated weights for policy 1, policy_version 1557473 (0.0008) [2023-12-27 02:42:58,655][105692] Updated weights for policy 0, policy_version 1554290 (0.0011) [2023-12-27 02:42:58,716][105692] Updated weights for policy 0, policy_version 1554300 (0.0010) [2023-12-27 02:42:58,789][105692] Updated weights for policy 0, policy_version 1554310 (0.0010) [2023-12-27 02:42:59,246][105620] Updated weights for policy 1, policy_version 1557483 (0.0009) [2023-12-27 02:42:59,315][105620] Updated weights for policy 1, policy_version 1557493 (0.0010) [2023-12-27 02:42:59,379][105620] Updated weights for policy 1, policy_version 1557503 (0.0008) [2023-12-27 02:42:59,546][105692] Updated weights for policy 0, policy_version 1554320 (0.0006) [2023-12-27 02:42:59,607][105692] Updated weights for policy 0, policy_version 1554330 (0.0006) [2023-12-27 02:42:59,672][105692] Updated weights for policy 0, policy_version 1554340 (0.0005) [2023-12-27 02:43:00,009][105620] Updated weights for policy 1, policy_version 1557513 (0.0006) [2023-12-27 02:43:00,074][105620] Updated weights for policy 1, policy_version 1557523 (0.0009) [2023-12-27 02:43:00,135][105620] Updated weights for policy 1, policy_version 1557533 (0.0008) [2023-12-27 02:43:00,200][105620] Updated weights for policy 1, policy_version 1557543 (0.0009) [2023-12-27 02:43:00,296][105692] Updated weights for policy 0, policy_version 1554350 (0.0007) [2023-12-27 02:43:00,363][105692] Updated weights for policy 0, policy_version 1554360 (0.0009) [2023-12-27 02:43:00,421][105692] Updated weights for policy 0, policy_version 1554370 (0.0008) [2023-12-27 02:43:00,950][105620] Updated weights for policy 1, policy_version 1557553 (0.0007) [2023-12-27 02:43:01,001][105620] Updated weights for policy 1, policy_version 1557563 (0.0008) [2023-12-27 02:43:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 796762112. Throughput: 0: 9918.9, 1: 9819.8. Samples: 796738556. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:43:01,062][105620] Updated weights for policy 1, policy_version 1557573 (0.0011) [2023-12-27 02:43:01,062][104569] Avg episode reward: [(0, '8355.561'), (1, '8991.758')] [2023-12-27 02:43:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001554376_397975552.pth... [2023-12-27 02:43:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001553224_397680640.pth [2023-12-27 02:43:01,078][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001557576_398794752.pth... [2023-12-27 02:43:01,081][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001556456_398508032.pth [2023-12-27 02:43:01,214][105692] Updated weights for policy 0, policy_version 1554380 (0.0010) [2023-12-27 02:43:01,278][105692] Updated weights for policy 0, policy_version 1554390 (0.0011) [2023-12-27 02:43:01,328][105692] Updated weights for policy 0, policy_version 1554400 (0.0011) [2023-12-27 02:43:01,775][105620] Updated weights for policy 1, policy_version 1557583 (0.0010) [2023-12-27 02:43:01,827][105620] Updated weights for policy 1, policy_version 1557593 (0.0009) [2023-12-27 02:43:01,881][105620] Updated weights for policy 1, policy_version 1557603 (0.0009) [2023-12-27 02:43:02,065][105692] Updated weights for policy 0, policy_version 1554410 (0.0010) [2023-12-27 02:43:02,117][105692] Updated weights for policy 0, policy_version 1554420 (0.0011) [2023-12-27 02:43:02,169][105692] Updated weights for policy 0, policy_version 1554430 (0.0011) [2023-12-27 02:43:02,221][105692] Updated weights for policy 0, policy_version 1554440 (0.0011) [2023-12-27 02:43:02,566][105620] Updated weights for policy 1, policy_version 1557613 (0.0009) [2023-12-27 02:43:02,618][105620] Updated weights for policy 1, policy_version 1557623 (0.0010) [2023-12-27 02:43:02,672][105620] Updated weights for policy 1, policy_version 1557633 (0.0006) [2023-12-27 02:43:02,974][105692] Updated weights for policy 0, policy_version 1554450 (0.0008) [2023-12-27 02:43:03,029][105692] Updated weights for policy 0, policy_version 1554460 (0.0008) [2023-12-27 02:43:03,077][105692] Updated weights for policy 0, policy_version 1554470 (0.0008) [2023-12-27 02:43:03,393][105620] Updated weights for policy 1, policy_version 1557643 (0.0008) [2023-12-27 02:43:03,450][105620] Updated weights for policy 1, policy_version 1557653 (0.0005) [2023-12-27 02:43:03,500][105620] Updated weights for policy 1, policy_version 1557663 (0.0009) [2023-12-27 02:43:03,821][105692] Updated weights for policy 0, policy_version 1554480 (0.0010) [2023-12-27 02:43:03,882][105692] Updated weights for policy 0, policy_version 1554490 (0.0011) [2023-12-27 02:43:03,941][105692] Updated weights for policy 0, policy_version 1554500 (0.0011) [2023-12-27 02:43:04,204][105620] Updated weights for policy 1, policy_version 1557673 (0.0010) [2023-12-27 02:43:04,271][105620] Updated weights for policy 1, policy_version 1557683 (0.0011) [2023-12-27 02:43:04,334][105620] Updated weights for policy 1, policy_version 1557693 (0.0011) [2023-12-27 02:43:04,396][105620] Updated weights for policy 1, policy_version 1557703 (0.0008) [2023-12-27 02:43:04,716][105692] Updated weights for policy 0, policy_version 1554510 (0.0011) [2023-12-27 02:43:04,764][105692] Updated weights for policy 0, policy_version 1554520 (0.0010) [2023-12-27 02:43:04,816][105692] Updated weights for policy 0, policy_version 1554530 (0.0010) [2023-12-27 02:43:04,943][105620] Updated weights for policy 1, policy_version 1557713 (0.0005) [2023-12-27 02:43:04,991][105620] Updated weights for policy 1, policy_version 1557723 (0.0005) [2023-12-27 02:43:05,050][105620] Updated weights for policy 1, policy_version 1557733 (0.0006) [2023-12-27 02:43:05,563][105692] Updated weights for policy 0, policy_version 1554540 (0.0010) [2023-12-27 02:43:05,583][105620] Updated weights for policy 1, policy_version 1557743 (0.0009) [2023-12-27 02:43:05,624][105692] Updated weights for policy 0, policy_version 1554550 (0.0010) [2023-12-27 02:43:05,641][105620] Updated weights for policy 1, policy_version 1557753 (0.0010) [2023-12-27 02:43:05,682][105692] Updated weights for policy 0, policy_version 1554560 (0.0010) [2023-12-27 02:43:05,695][105620] Updated weights for policy 1, policy_version 1557763 (0.0010) [2023-12-27 02:43:06,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 796868608. Throughput: 0: 9801.1, 1: 9814.0. Samples: 796855412. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:43:06,063][104569] Avg episode reward: [(0, '7807.941'), (1, '9083.465')] [2023-12-27 02:43:06,262][105692] Updated weights for policy 0, policy_version 1554570 (0.0006) [2023-12-27 02:43:06,329][105692] Updated weights for policy 0, policy_version 1554580 (0.0011) [2023-12-27 02:43:06,395][105692] Updated weights for policy 0, policy_version 1554590 (0.0011) [2023-12-27 02:43:06,448][105620] Updated weights for policy 1, policy_version 1557773 (0.0010) [2023-12-27 02:43:06,459][105692] Updated weights for policy 0, policy_version 1554600 (0.0011) [2023-12-27 02:43:06,507][105620] Updated weights for policy 1, policy_version 1557783 (0.0007) [2023-12-27 02:43:06,566][105620] Updated weights for policy 1, policy_version 1557793 (0.0008) [2023-12-27 02:43:07,143][105692] Updated weights for policy 0, policy_version 1554610 (0.0011) [2023-12-27 02:43:07,210][105692] Updated weights for policy 0, policy_version 1554620 (0.0011) [2023-12-27 02:43:07,276][105692] Updated weights for policy 0, policy_version 1554630 (0.0011) [2023-12-27 02:43:07,292][105620] Updated weights for policy 1, policy_version 1557803 (0.0009) [2023-12-27 02:43:07,350][105620] Updated weights for policy 1, policy_version 1557813 (0.0010) [2023-12-27 02:43:07,408][105620] Updated weights for policy 1, policy_version 1557823 (0.0010) [2023-12-27 02:43:08,012][105692] Updated weights for policy 0, policy_version 1554640 (0.0009) [2023-12-27 02:43:08,063][105692] Updated weights for policy 0, policy_version 1554650 (0.0009) [2023-12-27 02:43:08,093][105620] Updated weights for policy 1, policy_version 1557833 (0.0010) [2023-12-27 02:43:08,119][105692] Updated weights for policy 0, policy_version 1554660 (0.0009) [2023-12-27 02:43:08,157][105620] Updated weights for policy 1, policy_version 1557843 (0.0006) [2023-12-27 02:43:08,207][105620] Updated weights for policy 1, policy_version 1557853 (0.0005) [2023-12-27 02:43:08,254][105620] Updated weights for policy 1, policy_version 1557863 (0.0006) [2023-12-27 02:43:08,905][105692] Updated weights for policy 0, policy_version 1554670 (0.0011) [2023-12-27 02:43:08,947][105620] Updated weights for policy 1, policy_version 1557873 (0.0008) [2023-12-27 02:43:08,964][105692] Updated weights for policy 0, policy_version 1554680 (0.0011) [2023-12-27 02:43:09,003][105620] Updated weights for policy 1, policy_version 1557883 (0.0006) [2023-12-27 02:43:09,021][105692] Updated weights for policy 0, policy_version 1554690 (0.0010) [2023-12-27 02:43:09,054][105620] Updated weights for policy 1, policy_version 1557893 (0.0006) [2023-12-27 02:43:09,800][105692] Updated weights for policy 0, policy_version 1554700 (0.0011) [2023-12-27 02:43:09,832][105620] Updated weights for policy 1, policy_version 1557903 (0.0007) [2023-12-27 02:43:09,864][105692] Updated weights for policy 0, policy_version 1554710 (0.0009) [2023-12-27 02:43:09,897][105620] Updated weights for policy 1, policy_version 1557913 (0.0010) [2023-12-27 02:43:09,933][105692] Updated weights for policy 0, policy_version 1554720 (0.0008) [2023-12-27 02:43:09,971][105620] Updated weights for policy 1, policy_version 1557923 (0.0009) [2023-12-27 02:43:10,651][105692] Updated weights for policy 0, policy_version 1554730 (0.0007) [2023-12-27 02:43:10,713][105692] Updated weights for policy 0, policy_version 1554740 (0.0011) [2023-12-27 02:43:10,732][105620] Updated weights for policy 1, policy_version 1557933 (0.0007) [2023-12-27 02:43:10,769][105692] Updated weights for policy 0, policy_version 1554750 (0.0011) [2023-12-27 02:43:10,788][105620] Updated weights for policy 1, policy_version 1557943 (0.0005) [2023-12-27 02:43:10,831][105692] Updated weights for policy 0, policy_version 1554760 (0.0010) [2023-12-27 02:43:10,847][105620] Updated weights for policy 1, policy_version 1557953 (0.0006) [2023-12-27 02:43:11,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.7, 300 sec: 19605.3). Total num frames: 796966912. Throughput: 0: 9915.2, 1: 9728.0. Samples: 796972180. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:43:11,063][104569] Avg episode reward: [(0, '8442.253'), (1, '8989.881')] [2023-12-27 02:43:11,513][105692] Updated weights for policy 0, policy_version 1554770 (0.0009) [2023-12-27 02:43:11,575][105692] Updated weights for policy 0, policy_version 1554780 (0.0009) [2023-12-27 02:43:11,590][105620] Updated weights for policy 1, policy_version 1557963 (0.0007) [2023-12-27 02:43:11,641][105692] Updated weights for policy 0, policy_version 1554790 (0.0006) [2023-12-27 02:43:11,657][105620] Updated weights for policy 1, policy_version 1557973 (0.0009) [2023-12-27 02:43:11,708][105620] Updated weights for policy 1, policy_version 1557983 (0.0009) [2023-12-27 02:43:12,364][105692] Updated weights for policy 0, policy_version 1554800 (0.0008) [2023-12-27 02:43:12,433][105692] Updated weights for policy 0, policy_version 1554810 (0.0007) [2023-12-27 02:43:12,490][105620] Updated weights for policy 1, policy_version 1557993 (0.0006) [2023-12-27 02:43:12,495][105692] Updated weights for policy 0, policy_version 1554820 (0.0006) [2023-12-27 02:43:12,551][105620] Updated weights for policy 1, policy_version 1558003 (0.0009) [2023-12-27 02:43:12,611][105620] Updated weights for policy 1, policy_version 1558013 (0.0009) [2023-12-27 02:43:12,672][105620] Updated weights for policy 1, policy_version 1558023 (0.0008) [2023-12-27 02:43:13,154][105692] Updated weights for policy 0, policy_version 1554830 (0.0008) [2023-12-27 02:43:13,206][105692] Updated weights for policy 0, policy_version 1554840 (0.0009) [2023-12-27 02:43:13,264][105692] Updated weights for policy 0, policy_version 1554850 (0.0009) [2023-12-27 02:43:13,440][105620] Updated weights for policy 1, policy_version 1558033 (0.0008) [2023-12-27 02:43:13,497][105620] Updated weights for policy 1, policy_version 1558043 (0.0009) [2023-12-27 02:43:13,553][105620] Updated weights for policy 1, policy_version 1558053 (0.0008) [2023-12-27 02:43:13,986][105692] Updated weights for policy 0, policy_version 1554860 (0.0009) [2023-12-27 02:43:14,046][105692] Updated weights for policy 0, policy_version 1554870 (0.0006) [2023-12-27 02:43:14,111][105692] Updated weights for policy 0, policy_version 1554880 (0.0007) [2023-12-27 02:43:14,384][105620] Updated weights for policy 1, policy_version 1558063 (0.0008) [2023-12-27 02:43:14,436][105620] Updated weights for policy 1, policy_version 1558073 (0.0009) [2023-12-27 02:43:14,489][105620] Updated weights for policy 1, policy_version 1558083 (0.0008) [2023-12-27 02:43:14,712][105692] Updated weights for policy 0, policy_version 1554890 (0.0005) [2023-12-27 02:43:14,782][105692] Updated weights for policy 0, policy_version 1554900 (0.0006) [2023-12-27 02:43:14,846][105692] Updated weights for policy 0, policy_version 1554910 (0.0006) [2023-12-27 02:43:14,910][105692] Updated weights for policy 0, policy_version 1554920 (0.0008) [2023-12-27 02:43:15,363][105620] Updated weights for policy 1, policy_version 1558093 (0.0007) [2023-12-27 02:43:15,420][105620] Updated weights for policy 1, policy_version 1558103 (0.0005) [2023-12-27 02:43:15,472][105620] Updated weights for policy 1, policy_version 1558113 (0.0006) [2023-12-27 02:43:15,665][105692] Updated weights for policy 0, policy_version 1554930 (0.0009) [2023-12-27 02:43:15,729][105692] Updated weights for policy 0, policy_version 1554940 (0.0008) [2023-12-27 02:43:15,782][105692] Updated weights for policy 0, policy_version 1554950 (0.0009) [2023-12-27 02:43:16,041][105620] Updated weights for policy 1, policy_version 1558123 (0.0005) [2023-12-27 02:43:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 797057024. Throughput: 0: 9919.1, 1: 9575.8. Samples: 797028372. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:43:16,063][104569] Avg episode reward: [(0, '8630.032'), (1, '8991.576')] [2023-12-27 02:43:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001554952_398123008.pth... [2023-12-27 02:43:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001553800_397828096.pth [2023-12-27 02:43:16,101][105620] Updated weights for policy 1, policy_version 1558133 (0.0010) [2023-12-27 02:43:16,161][105620] Updated weights for policy 1, policy_version 1558143 (0.0008) [2023-12-27 02:43:16,219][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001558152_398942208.pth... [2023-12-27 02:43:16,223][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001557000_398647296.pth [2023-12-27 02:43:16,684][105692] Updated weights for policy 0, policy_version 1554960 (0.0006) [2023-12-27 02:43:16,746][105692] Updated weights for policy 0, policy_version 1554970 (0.0007) [2023-12-27 02:43:16,748][105620] Updated weights for policy 1, policy_version 1558153 (0.0006) [2023-12-27 02:43:16,806][105620] Updated weights for policy 1, policy_version 1558163 (0.0010) [2023-12-27 02:43:16,808][105692] Updated weights for policy 0, policy_version 1554980 (0.0005) [2023-12-27 02:43:16,854][105620] Updated weights for policy 1, policy_version 1558173 (0.0010) [2023-12-27 02:43:16,901][105620] Updated weights for policy 1, policy_version 1558183 (0.0010) [2023-12-27 02:43:17,394][105692] Updated weights for policy 0, policy_version 1554990 (0.0007) [2023-12-27 02:43:17,446][105692] Updated weights for policy 0, policy_version 1555000 (0.0008) [2023-12-27 02:43:17,494][105692] Updated weights for policy 0, policy_version 1555010 (0.0009) [2023-12-27 02:43:17,665][105620] Updated weights for policy 1, policy_version 1558193 (0.0010) [2023-12-27 02:43:17,733][105620] Updated weights for policy 1, policy_version 1558203 (0.0010) [2023-12-27 02:43:17,801][105620] Updated weights for policy 1, policy_version 1558213 (0.0010) [2023-12-27 02:43:18,140][105692] Updated weights for policy 0, policy_version 1555020 (0.0008) [2023-12-27 02:43:18,189][105692] Updated weights for policy 0, policy_version 1555030 (0.0007) [2023-12-27 02:43:18,249][105692] Updated weights for policy 0, policy_version 1555040 (0.0008) [2023-12-27 02:43:18,533][105620] Updated weights for policy 1, policy_version 1558223 (0.0011) [2023-12-27 02:43:18,596][105620] Updated weights for policy 1, policy_version 1558233 (0.0011) [2023-12-27 02:43:18,645][105620] Updated weights for policy 1, policy_version 1558243 (0.0010) [2023-12-27 02:43:18,878][105692] Updated weights for policy 0, policy_version 1555050 (0.0009) [2023-12-27 02:43:18,943][105692] Updated weights for policy 0, policy_version 1555060 (0.0007) [2023-12-27 02:43:18,998][105692] Updated weights for policy 0, policy_version 1555070 (0.0008) [2023-12-27 02:43:19,058][105692] Updated weights for policy 0, policy_version 1555080 (0.0008) [2023-12-27 02:43:19,380][105620] Updated weights for policy 1, policy_version 1558253 (0.0011) [2023-12-27 02:43:19,441][105620] Updated weights for policy 1, policy_version 1558263 (0.0010) [2023-12-27 02:43:19,502][105620] Updated weights for policy 1, policy_version 1558273 (0.0010) [2023-12-27 02:43:19,790][105692] Updated weights for policy 0, policy_version 1555090 (0.0008) [2023-12-27 02:43:19,852][105692] Updated weights for policy 0, policy_version 1555100 (0.0009) [2023-12-27 02:43:19,910][105692] Updated weights for policy 0, policy_version 1555110 (0.0009) [2023-12-27 02:43:20,319][105620] Updated weights for policy 1, policy_version 1558283 (0.0010) [2023-12-27 02:43:20,379][105620] Updated weights for policy 1, policy_version 1558293 (0.0007) [2023-12-27 02:43:20,432][105620] Updated weights for policy 1, policy_version 1558303 (0.0010) [2023-12-27 02:43:20,760][105692] Updated weights for policy 0, policy_version 1555120 (0.0008) [2023-12-27 02:43:20,824][105692] Updated weights for policy 0, policy_version 1555130 (0.0008) [2023-12-27 02:43:20,888][105692] Updated weights for policy 0, policy_version 1555140 (0.0008) [2023-12-27 02:43:21,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 797155328. Throughput: 0: 9857.6, 1: 9624.1. Samples: 797146424. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:43:21,062][104569] Avg episode reward: [(0, '8169.379'), (1, '8903.088')] [2023-12-27 02:43:21,204][105620] Updated weights for policy 1, policy_version 1558313 (0.0009) [2023-12-27 02:43:21,273][105620] Updated weights for policy 1, policy_version 1558323 (0.0008) [2023-12-27 02:43:21,337][105620] Updated weights for policy 1, policy_version 1558333 (0.0008) [2023-12-27 02:43:21,407][105620] Updated weights for policy 1, policy_version 1558343 (0.0009) [2023-12-27 02:43:21,623][105692] Updated weights for policy 0, policy_version 1555150 (0.0010) [2023-12-27 02:43:21,686][105692] Updated weights for policy 0, policy_version 1555160 (0.0011) [2023-12-27 02:43:21,755][105692] Updated weights for policy 0, policy_version 1555170 (0.0012) [2023-12-27 02:43:22,159][105620] Updated weights for policy 1, policy_version 1558353 (0.0008) [2023-12-27 02:43:22,233][105620] Updated weights for policy 1, policy_version 1558363 (0.0009) [2023-12-27 02:43:22,298][105620] Updated weights for policy 1, policy_version 1558373 (0.0007) [2023-12-27 02:43:22,441][105692] Updated weights for policy 0, policy_version 1555180 (0.0010) [2023-12-27 02:43:22,491][105692] Updated weights for policy 0, policy_version 1555190 (0.0010) [2023-12-27 02:43:22,552][105692] Updated weights for policy 0, policy_version 1555200 (0.0008) [2023-12-27 02:43:23,020][105620] Updated weights for policy 1, policy_version 1558383 (0.0008) [2023-12-27 02:43:23,067][105620] Updated weights for policy 1, policy_version 1558393 (0.0009) [2023-12-27 02:43:23,114][105620] Updated weights for policy 1, policy_version 1558403 (0.0009) [2023-12-27 02:43:23,230][105692] Updated weights for policy 0, policy_version 1555210 (0.0009) [2023-12-27 02:43:23,287][105692] Updated weights for policy 0, policy_version 1555220 (0.0009) [2023-12-27 02:43:23,343][105692] Updated weights for policy 0, policy_version 1555230 (0.0010) [2023-12-27 02:43:23,405][105692] Updated weights for policy 0, policy_version 1555240 (0.0008) [2023-12-27 02:43:23,815][105620] Updated weights for policy 1, policy_version 1558413 (0.0009) [2023-12-27 02:43:23,869][105620] Updated weights for policy 1, policy_version 1558423 (0.0008) [2023-12-27 02:43:23,916][105620] Updated weights for policy 1, policy_version 1558433 (0.0008) [2023-12-27 02:43:24,107][105692] Updated weights for policy 0, policy_version 1555250 (0.0008) [2023-12-27 02:43:24,155][105692] Updated weights for policy 0, policy_version 1555260 (0.0008) [2023-12-27 02:43:24,203][105692] Updated weights for policy 0, policy_version 1555270 (0.0009) [2023-12-27 02:43:24,680][105620] Updated weights for policy 1, policy_version 1558443 (0.0008) [2023-12-27 02:43:24,742][105620] Updated weights for policy 1, policy_version 1558453 (0.0006) [2023-12-27 02:43:24,811][105620] Updated weights for policy 1, policy_version 1558463 (0.0005) [2023-12-27 02:43:24,937][105692] Updated weights for policy 0, policy_version 1555280 (0.0006) [2023-12-27 02:43:24,994][105692] Updated weights for policy 0, policy_version 1555290 (0.0008) [2023-12-27 02:43:25,049][105692] Updated weights for policy 0, policy_version 1555300 (0.0010) [2023-12-27 02:43:25,336][105620] Updated weights for policy 1, policy_version 1558473 (0.0007) [2023-12-27 02:43:25,408][105620] Updated weights for policy 1, policy_version 1558483 (0.0007) [2023-12-27 02:43:25,470][105620] Updated weights for policy 1, policy_version 1558493 (0.0006) [2023-12-27 02:43:25,536][105620] Updated weights for policy 1, policy_version 1558503 (0.0009) [2023-12-27 02:43:25,681][105692] Updated weights for policy 0, policy_version 1555310 (0.0009) [2023-12-27 02:43:25,741][105692] Updated weights for policy 0, policy_version 1555320 (0.0008) [2023-12-27 02:43:25,807][105692] Updated weights for policy 0, policy_version 1555330 (0.0008) [2023-12-27 02:43:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 797253632. Throughput: 0: 9861.4, 1: 9591.8. Samples: 797263100. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:43:26,063][104569] Avg episode reward: [(0, '8441.829'), (1, '9083.633')] [2023-12-27 02:43:26,155][105620] Updated weights for policy 1, policy_version 1558513 (0.0009) [2023-12-27 02:43:26,208][105620] Updated weights for policy 1, policy_version 1558523 (0.0008) [2023-12-27 02:43:26,262][105620] Updated weights for policy 1, policy_version 1558533 (0.0005) [2023-12-27 02:43:26,657][105692] Updated weights for policy 0, policy_version 1555340 (0.0009) [2023-12-27 02:43:26,711][105692] Updated weights for policy 0, policy_version 1555350 (0.0010) [2023-12-27 02:43:26,769][105692] Updated weights for policy 0, policy_version 1555361 (0.0010) [2023-12-27 02:43:26,808][105620] Updated weights for policy 1, policy_version 1558543 (0.0005) [2023-12-27 02:43:26,863][105620] Updated weights for policy 1, policy_version 1558553 (0.0005) [2023-12-27 02:43:26,913][105620] Updated weights for policy 1, policy_version 1558563 (0.0005) [2023-12-27 02:43:27,421][105620] Updated weights for policy 1, policy_version 1558573 (0.0008) [2023-12-27 02:43:27,485][105620] Updated weights for policy 1, policy_version 1558583 (0.0010) [2023-12-27 02:43:27,550][105620] Updated weights for policy 1, policy_version 1558593 (0.0010) [2023-12-27 02:43:27,559][105692] Updated weights for policy 0, policy_version 1555371 (0.0009) [2023-12-27 02:43:27,610][105692] Updated weights for policy 0, policy_version 1555381 (0.0010) [2023-12-27 02:43:27,657][105692] Updated weights for policy 0, policy_version 1555391 (0.0009) [2023-12-27 02:43:28,284][105692] Updated weights for policy 0, policy_version 1555401 (0.0006) [2023-12-27 02:43:28,306][105620] Updated weights for policy 1, policy_version 1558603 (0.0010) [2023-12-27 02:43:28,340][105692] Updated weights for policy 0, policy_version 1555411 (0.0009) [2023-12-27 02:43:28,367][105620] Updated weights for policy 1, policy_version 1558613 (0.0008) [2023-12-27 02:43:28,398][105692] Updated weights for policy 0, policy_version 1555421 (0.0009) [2023-12-27 02:43:28,429][105620] Updated weights for policy 1, policy_version 1558623 (0.0008) [2023-12-27 02:43:28,459][105692] Updated weights for policy 0, policy_version 1555431 (0.0007) [2023-12-27 02:43:29,148][105692] Updated weights for policy 0, policy_version 1555441 (0.0008) [2023-12-27 02:43:29,197][105620] Updated weights for policy 1, policy_version 1558633 (0.0007) [2023-12-27 02:43:29,207][105692] Updated weights for policy 0, policy_version 1555451 (0.0008) [2023-12-27 02:43:29,264][105620] Updated weights for policy 1, policy_version 1558643 (0.0010) [2023-12-27 02:43:29,266][105692] Updated weights for policy 0, policy_version 1555461 (0.0007) [2023-12-27 02:43:29,323][105620] Updated weights for policy 1, policy_version 1558653 (0.0010) [2023-12-27 02:43:29,390][105620] Updated weights for policy 1, policy_version 1558663 (0.0008) [2023-12-27 02:43:29,980][105620] Updated weights for policy 1, policy_version 1558673 (0.0009) [2023-12-27 02:43:29,988][105692] Updated weights for policy 0, policy_version 1555471 (0.0008) [2023-12-27 02:43:30,032][105620] Updated weights for policy 1, policy_version 1558683 (0.0007) [2023-12-27 02:43:30,038][105692] Updated weights for policy 0, policy_version 1555481 (0.0007) [2023-12-27 02:43:30,088][105692] Updated weights for policy 0, policy_version 1555491 (0.0007) [2023-12-27 02:43:30,090][105620] Updated weights for policy 1, policy_version 1558693 (0.0008) [2023-12-27 02:43:30,787][105620] Updated weights for policy 1, policy_version 1558703 (0.0006) [2023-12-27 02:43:30,841][105620] Updated weights for policy 1, policy_version 1558713 (0.0006) [2023-12-27 02:43:30,861][105692] Updated weights for policy 0, policy_version 1555501 (0.0009) [2023-12-27 02:43:30,901][105620] Updated weights for policy 1, policy_version 1558723 (0.0008) [2023-12-27 02:43:30,908][105692] Updated weights for policy 0, policy_version 1555511 (0.0006) [2023-12-27 02:43:30,959][105692] Updated weights for policy 0, policy_version 1555521 (0.0008) [2023-12-27 02:43:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 797360128. Throughput: 0: 9839.0, 1: 9725.6. Samples: 797323200. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:43:31,062][104569] Avg episode reward: [(0, '8809.058'), (1, '9083.540')] [2023-12-27 02:43:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001555528_398270464.pth... [2023-12-27 02:43:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001558728_399089664.pth... [2023-12-27 02:43:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001554376_397975552.pth [2023-12-27 02:43:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001557576_398794752.pth [2023-12-27 02:43:31,591][105620] Updated weights for policy 1, policy_version 1558733 (0.0010) [2023-12-27 02:43:31,655][105620] Updated weights for policy 1, policy_version 1558743 (0.0008) [2023-12-27 02:43:31,683][105692] Updated weights for policy 0, policy_version 1555531 (0.0008) [2023-12-27 02:43:31,710][105620] Updated weights for policy 1, policy_version 1558753 (0.0009) [2023-12-27 02:43:31,755][105692] Updated weights for policy 0, policy_version 1555541 (0.0007) [2023-12-27 02:43:31,819][105692] Updated weights for policy 0, policy_version 1555551 (0.0008) [2023-12-27 02:43:32,281][105620] Updated weights for policy 1, policy_version 1558763 (0.0008) [2023-12-27 02:43:32,346][105620] Updated weights for policy 1, policy_version 1558773 (0.0006) [2023-12-27 02:43:32,411][105620] Updated weights for policy 1, policy_version 1558783 (0.0007) [2023-12-27 02:43:32,441][105692] Updated weights for policy 0, policy_version 1555561 (0.0008) [2023-12-27 02:43:32,507][105692] Updated weights for policy 0, policy_version 1555571 (0.0008) [2023-12-27 02:43:32,576][105692] Updated weights for policy 0, policy_version 1555581 (0.0008) [2023-12-27 02:43:32,637][105692] Updated weights for policy 0, policy_version 1555591 (0.0008) [2023-12-27 02:43:33,015][105620] Updated weights for policy 1, policy_version 1558793 (0.0009) [2023-12-27 02:43:33,071][105620] Updated weights for policy 1, policy_version 1558803 (0.0005) [2023-12-27 02:43:33,130][105620] Updated weights for policy 1, policy_version 1558813 (0.0005) [2023-12-27 02:43:33,188][105620] Updated weights for policy 1, policy_version 1558823 (0.0005) [2023-12-27 02:43:33,414][105692] Updated weights for policy 0, policy_version 1555601 (0.0010) [2023-12-27 02:43:33,467][105692] Updated weights for policy 0, policy_version 1555612 (0.0010) [2023-12-27 02:43:33,512][105692] Updated weights for policy 0, policy_version 1555622 (0.0008) [2023-12-27 02:43:33,757][105620] Updated weights for policy 1, policy_version 1558833 (0.0008) [2023-12-27 02:43:33,820][105620] Updated weights for policy 1, policy_version 1558843 (0.0009) [2023-12-27 02:43:33,883][105620] Updated weights for policy 1, policy_version 1558853 (0.0006) [2023-12-27 02:43:34,264][105692] Updated weights for policy 0, policy_version 1555632 (0.0010) [2023-12-27 02:43:34,326][105692] Updated weights for policy 0, policy_version 1555642 (0.0010) [2023-12-27 02:43:34,387][105692] Updated weights for policy 0, policy_version 1555652 (0.0010) [2023-12-27 02:43:34,502][105620] Updated weights for policy 1, policy_version 1558863 (0.0009) [2023-12-27 02:43:34,557][105620] Updated weights for policy 1, policy_version 1558873 (0.0010) [2023-12-27 02:43:34,613][105620] Updated weights for policy 1, policy_version 1558883 (0.0010) [2023-12-27 02:43:35,234][105692] Updated weights for policy 0, policy_version 1555662 (0.0008) [2023-12-27 02:43:35,287][105692] Updated weights for policy 0, policy_version 1555672 (0.0008) [2023-12-27 02:43:35,314][105620] Updated weights for policy 1, policy_version 1558893 (0.0009) [2023-12-27 02:43:35,334][105692] Updated weights for policy 0, policy_version 1555682 (0.0007) [2023-12-27 02:43:35,368][105620] Updated weights for policy 1, policy_version 1558903 (0.0005) [2023-12-27 02:43:35,423][105620] Updated weights for policy 1, policy_version 1558913 (0.0005) [2023-12-27 02:43:36,003][105620] Updated weights for policy 1, policy_version 1558923 (0.0005) [2023-12-27 02:43:36,056][105620] Updated weights for policy 1, policy_version 1558933 (0.0010) [2023-12-27 02:43:36,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 797450240. Throughput: 0: 9782.5, 1: 9834.1. Samples: 797444724. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:43:36,062][104569] Avg episode reward: [(0, '8452.382'), (1, '8995.385')] [2023-12-27 02:43:36,112][105620] Updated weights for policy 1, policy_version 1558943 (0.0010) [2023-12-27 02:43:36,176][105692] Updated weights for policy 0, policy_version 1555693 (0.0008) [2023-12-27 02:43:36,240][105692] Updated weights for policy 0, policy_version 1555703 (0.0009) [2023-12-27 02:43:36,300][105692] Updated weights for policy 0, policy_version 1555713 (0.0008) [2023-12-27 02:43:36,851][105620] Updated weights for policy 1, policy_version 1558953 (0.0009) [2023-12-27 02:43:36,915][105620] Updated weights for policy 1, policy_version 1558963 (0.0009) [2023-12-27 02:43:36,970][105620] Updated weights for policy 1, policy_version 1558973 (0.0010) [2023-12-27 02:43:37,022][105620] Updated weights for policy 1, policy_version 1558983 (0.0010) [2023-12-27 02:43:37,037][105692] Updated weights for policy 0, policy_version 1555723 (0.0009) [2023-12-27 02:43:37,094][105692] Updated weights for policy 0, policy_version 1555733 (0.0008) [2023-12-27 02:43:37,146][105692] Updated weights for policy 0, policy_version 1555743 (0.0008) [2023-12-27 02:43:37,712][105620] Updated weights for policy 1, policy_version 1558993 (0.0011) [2023-12-27 02:43:37,768][105620] Updated weights for policy 1, policy_version 1559003 (0.0011) [2023-12-27 02:43:37,820][105620] Updated weights for policy 1, policy_version 1559013 (0.0010) [2023-12-27 02:43:37,961][105692] Updated weights for policy 0, policy_version 1555753 (0.0009) [2023-12-27 02:43:38,018][105692] Updated weights for policy 0, policy_version 1555763 (0.0010) [2023-12-27 02:43:38,063][105692] Updated weights for policy 0, policy_version 1555773 (0.0008) [2023-12-27 02:43:38,126][105692] Updated weights for policy 0, policy_version 1555783 (0.0009) [2023-12-27 02:43:38,450][105620] Updated weights for policy 1, policy_version 1559023 (0.0007) [2023-12-27 02:43:38,512][105620] Updated weights for policy 1, policy_version 1559033 (0.0005) [2023-12-27 02:43:38,575][105620] Updated weights for policy 1, policy_version 1559043 (0.0005) [2023-12-27 02:43:38,946][105692] Updated weights for policy 0, policy_version 1555793 (0.0010) [2023-12-27 02:43:39,010][105692] Updated weights for policy 0, policy_version 1555803 (0.0010) [2023-12-27 02:43:39,071][105692] Updated weights for policy 0, policy_version 1555813 (0.0009) [2023-12-27 02:43:39,103][105620] Updated weights for policy 1, policy_version 1559053 (0.0006) [2023-12-27 02:43:39,165][105620] Updated weights for policy 1, policy_version 1559063 (0.0007) [2023-12-27 02:43:39,225][105620] Updated weights for policy 1, policy_version 1559073 (0.0011) [2023-12-27 02:43:39,831][105692] Updated weights for policy 0, policy_version 1555823 (0.0009) [2023-12-27 02:43:39,888][105692] Updated weights for policy 0, policy_version 1555833 (0.0009) [2023-12-27 02:43:39,944][105692] Updated weights for policy 0, policy_version 1555843 (0.0008) [2023-12-27 02:43:39,976][105620] Updated weights for policy 1, policy_version 1559083 (0.0011) [2023-12-27 02:43:40,040][105620] Updated weights for policy 1, policy_version 1559093 (0.0011) [2023-12-27 02:43:40,101][105620] Updated weights for policy 1, policy_version 1559103 (0.0007) [2023-12-27 02:43:40,705][105692] Updated weights for policy 0, policy_version 1555853 (0.0008) [2023-12-27 02:43:40,758][105620] Updated weights for policy 1, policy_version 1559113 (0.0009) [2023-12-27 02:43:40,764][105692] Updated weights for policy 0, policy_version 1555863 (0.0008) [2023-12-27 02:43:40,814][105692] Updated weights for policy 0, policy_version 1555873 (0.0007) [2023-12-27 02:43:40,845][105620] Updated weights for policy 1, policy_version 1559123 (0.0010) [2023-12-27 02:43:40,897][105620] Updated weights for policy 1, policy_version 1559133 (0.0010) [2023-12-27 02:43:40,950][105620] Updated weights for policy 1, policy_version 1559143 (0.0010) [2023-12-27 02:43:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 797556736. Throughput: 0: 9546.3, 1: 10020.1. Samples: 797561388. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:43:41,062][104569] Avg episode reward: [(0, '8085.145'), (1, '9081.218')] [2023-12-27 02:43:41,561][105692] Updated weights for policy 0, policy_version 1555883 (0.0006) [2023-12-27 02:43:41,625][105692] Updated weights for policy 0, policy_version 1555893 (0.0008) [2023-12-27 02:43:41,693][105692] Updated weights for policy 0, policy_version 1555903 (0.0009) [2023-12-27 02:43:41,719][105620] Updated weights for policy 1, policy_version 1559153 (0.0007) [2023-12-27 02:43:41,782][105620] Updated weights for policy 1, policy_version 1559163 (0.0009) [2023-12-27 02:43:41,847][105620] Updated weights for policy 1, policy_version 1559173 (0.0009) [2023-12-27 02:43:42,383][105692] Updated weights for policy 0, policy_version 1555913 (0.0009) [2023-12-27 02:43:42,450][105692] Updated weights for policy 0, policy_version 1555923 (0.0010) [2023-12-27 02:43:42,513][105692] Updated weights for policy 0, policy_version 1555933 (0.0009) [2023-12-27 02:43:42,564][105692] Updated weights for policy 0, policy_version 1555943 (0.0009) [2023-12-27 02:43:42,601][105620] Updated weights for policy 1, policy_version 1559183 (0.0009) [2023-12-27 02:43:42,656][105620] Updated weights for policy 1, policy_version 1559193 (0.0010) [2023-12-27 02:43:42,713][105620] Updated weights for policy 1, policy_version 1559204 (0.0010) [2023-12-27 02:43:43,230][105692] Updated weights for policy 0, policy_version 1555953 (0.0007) [2023-12-27 02:43:43,291][105692] Updated weights for policy 0, policy_version 1555963 (0.0008) [2023-12-27 02:43:43,354][105692] Updated weights for policy 0, policy_version 1555973 (0.0009) [2023-12-27 02:43:43,487][105620] Updated weights for policy 1, policy_version 1559214 (0.0009) [2023-12-27 02:43:43,548][105620] Updated weights for policy 1, policy_version 1559224 (0.0005) [2023-12-27 02:43:43,611][105620] Updated weights for policy 1, policy_version 1559234 (0.0005) [2023-12-27 02:43:44,101][105692] Updated weights for policy 0, policy_version 1555984 (0.0010) [2023-12-27 02:43:44,162][105692] Updated weights for policy 0, policy_version 1555994 (0.0009) [2023-12-27 02:43:44,224][105692] Updated weights for policy 0, policy_version 1556004 (0.0010) [2023-12-27 02:43:44,261][105620] Updated weights for policy 1, policy_version 1559244 (0.0005) [2023-12-27 02:43:44,319][105620] Updated weights for policy 1, policy_version 1559254 (0.0005) [2023-12-27 02:43:44,375][105620] Updated weights for policy 1, policy_version 1559264 (0.0005) [2023-12-27 02:43:44,923][105692] Updated weights for policy 0, policy_version 1556014 (0.0010) [2023-12-27 02:43:44,925][105620] Updated weights for policy 1, policy_version 1559274 (0.0005) [2023-12-27 02:43:44,983][105620] Updated weights for policy 1, policy_version 1559284 (0.0008) [2023-12-27 02:43:44,986][105692] Updated weights for policy 0, policy_version 1556024 (0.0010) [2023-12-27 02:43:45,041][105620] Updated weights for policy 1, policy_version 1559294 (0.0006) [2023-12-27 02:43:45,046][105692] Updated weights for policy 0, policy_version 1556034 (0.0011) [2023-12-27 02:43:45,096][105620] Updated weights for policy 1, policy_version 1559304 (0.0008) [2023-12-27 02:43:45,714][105692] Updated weights for policy 0, policy_version 1556044 (0.0011) [2023-12-27 02:43:45,766][105692] Updated weights for policy 0, policy_version 1556054 (0.0010) [2023-12-27 02:43:45,815][105692] Updated weights for policy 0, policy_version 1556064 (0.0010) [2023-12-27 02:43:45,883][105620] Updated weights for policy 1, policy_version 1559314 (0.0007) [2023-12-27 02:43:45,944][105620] Updated weights for policy 1, policy_version 1559324 (0.0007) [2023-12-27 02:43:46,002][105620] Updated weights for policy 1, policy_version 1559334 (0.0007) [2023-12-27 02:43:46,062][104569] Fps is (10 sec: 20479.2, 60 sec: 19660.7, 300 sec: 19633.0). Total num frames: 797655040. Throughput: 0: 9546.2, 1: 9993.6. Samples: 797617856. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:43:46,064][104569] Avg episode reward: [(0, '8540.395'), (1, '8809.694')] [2023-12-27 02:43:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001559336_399245312.pth... [2023-12-27 02:43:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001556072_398409728.pth... [2023-12-27 02:43:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001554952_398123008.pth [2023-12-27 02:43:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001558152_398942208.pth [2023-12-27 02:43:46,518][105692] Updated weights for policy 0, policy_version 1556074 (0.0010) [2023-12-27 02:43:46,571][105692] Updated weights for policy 0, policy_version 1556084 (0.0010) [2023-12-27 02:43:46,611][105620] Updated weights for policy 1, policy_version 1559344 (0.0005) [2023-12-27 02:43:46,631][105692] Updated weights for policy 0, policy_version 1556094 (0.0008) [2023-12-27 02:43:46,660][105620] Updated weights for policy 1, policy_version 1559354 (0.0005) [2023-12-27 02:43:46,681][105692] Updated weights for policy 0, policy_version 1556104 (0.0009) [2023-12-27 02:43:46,708][105620] Updated weights for policy 1, policy_version 1559364 (0.0008) [2023-12-27 02:43:47,451][105620] Updated weights for policy 1, policy_version 1559374 (0.0010) [2023-12-27 02:43:47,475][105692] Updated weights for policy 0, policy_version 1556114 (0.0008) [2023-12-27 02:43:47,507][105620] Updated weights for policy 1, policy_version 1559384 (0.0010) [2023-12-27 02:43:47,530][105692] Updated weights for policy 0, policy_version 1556124 (0.0008) [2023-12-27 02:43:47,557][105620] Updated weights for policy 1, policy_version 1559394 (0.0007) [2023-12-27 02:43:47,587][105692] Updated weights for policy 0, policy_version 1556134 (0.0007) [2023-12-27 02:43:48,312][105620] Updated weights for policy 1, policy_version 1559404 (0.0007) [2023-12-27 02:43:48,346][105692] Updated weights for policy 0, policy_version 1556144 (0.0008) [2023-12-27 02:43:48,379][105620] Updated weights for policy 1, policy_version 1559414 (0.0006) [2023-12-27 02:43:48,402][105692] Updated weights for policy 0, policy_version 1556154 (0.0009) [2023-12-27 02:43:48,438][105620] Updated weights for policy 1, policy_version 1559424 (0.0006) [2023-12-27 02:43:48,453][105692] Updated weights for policy 0, policy_version 1556164 (0.0009) [2023-12-27 02:43:49,066][105620] Updated weights for policy 1, policy_version 1559434 (0.0006) [2023-12-27 02:43:49,128][105620] Updated weights for policy 1, policy_version 1559444 (0.0006) [2023-12-27 02:43:49,181][105620] Updated weights for policy 1, policy_version 1559454 (0.0008) [2023-12-27 02:43:49,237][105620] Updated weights for policy 1, policy_version 1559464 (0.0009) [2023-12-27 02:43:49,251][105692] Updated weights for policy 0, policy_version 1556174 (0.0007) [2023-12-27 02:43:49,310][105692] Updated weights for policy 0, policy_version 1556184 (0.0010) [2023-12-27 02:43:49,376][105692] Updated weights for policy 0, policy_version 1556194 (0.0009) [2023-12-27 02:43:49,940][105620] Updated weights for policy 1, policy_version 1559474 (0.0008) [2023-12-27 02:43:50,001][105620] Updated weights for policy 1, policy_version 1559484 (0.0009) [2023-12-27 02:43:50,067][105620] Updated weights for policy 1, policy_version 1559494 (0.0007) [2023-12-27 02:43:50,138][105692] Updated weights for policy 0, policy_version 1556204 (0.0007) [2023-12-27 02:43:50,192][105692] Updated weights for policy 0, policy_version 1556214 (0.0008) [2023-12-27 02:43:50,256][105692] Updated weights for policy 0, policy_version 1556224 (0.0010) [2023-12-27 02:43:50,704][105620] Updated weights for policy 1, policy_version 1559504 (0.0009) [2023-12-27 02:43:50,756][105620] Updated weights for policy 1, policy_version 1559514 (0.0009) [2023-12-27 02:43:50,809][105620] Updated weights for policy 1, policy_version 1559524 (0.0009) [2023-12-27 02:43:51,008][105692] Updated weights for policy 0, policy_version 1556234 (0.0009) [2023-12-27 02:43:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 797745152. Throughput: 0: 9555.6, 1: 10007.4. Samples: 797735744. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:43:51,063][104569] Avg episode reward: [(0, '8360.905'), (1, '8719.104')] [2023-12-27 02:43:51,078][105692] Updated weights for policy 0, policy_version 1556244 (0.0008) [2023-12-27 02:43:51,158][105692] Updated weights for policy 0, policy_version 1556254 (0.0007) [2023-12-27 02:43:51,224][105692] Updated weights for policy 0, policy_version 1556264 (0.0008) [2023-12-27 02:43:51,553][105620] Updated weights for policy 1, policy_version 1559534 (0.0007) [2023-12-27 02:43:51,621][105620] Updated weights for policy 1, policy_version 1559544 (0.0006) [2023-12-27 02:43:51,687][105620] Updated weights for policy 1, policy_version 1559554 (0.0009) [2023-12-27 02:43:51,950][105692] Updated weights for policy 0, policy_version 1556274 (0.0008) [2023-12-27 02:43:52,009][105692] Updated weights for policy 0, policy_version 1556284 (0.0008) [2023-12-27 02:43:52,066][105692] Updated weights for policy 0, policy_version 1556294 (0.0008) [2023-12-27 02:43:52,431][105620] Updated weights for policy 1, policy_version 1559564 (0.0010) [2023-12-27 02:43:52,477][105620] Updated weights for policy 1, policy_version 1559574 (0.0011) [2023-12-27 02:43:52,529][105620] Updated weights for policy 1, policy_version 1559584 (0.0011) [2023-12-27 02:43:52,754][105692] Updated weights for policy 0, policy_version 1556304 (0.0006) [2023-12-27 02:43:52,810][105692] Updated weights for policy 0, policy_version 1556314 (0.0005) [2023-12-27 02:43:52,870][105692] Updated weights for policy 0, policy_version 1556324 (0.0007) [2023-12-27 02:43:53,300][105620] Updated weights for policy 1, policy_version 1559594 (0.0011) [2023-12-27 02:43:53,354][105620] Updated weights for policy 1, policy_version 1559604 (0.0010) [2023-12-27 02:43:53,408][105620] Updated weights for policy 1, policy_version 1559614 (0.0010) [2023-12-27 02:43:53,476][105620] Updated weights for policy 1, policy_version 1559624 (0.0007) [2023-12-27 02:43:53,567][105692] Updated weights for policy 0, policy_version 1556334 (0.0011) [2023-12-27 02:43:53,632][105692] Updated weights for policy 0, policy_version 1556344 (0.0010) [2023-12-27 02:43:53,697][105692] Updated weights for policy 0, policy_version 1556354 (0.0011) [2023-12-27 02:43:54,070][105620] Updated weights for policy 1, policy_version 1559634 (0.0011) [2023-12-27 02:43:54,131][105620] Updated weights for policy 1, policy_version 1559644 (0.0011) [2023-12-27 02:43:54,196][105620] Updated weights for policy 1, policy_version 1559654 (0.0011) [2023-12-27 02:43:54,422][105692] Updated weights for policy 0, policy_version 1556364 (0.0011) [2023-12-27 02:43:54,480][105692] Updated weights for policy 0, policy_version 1556374 (0.0006) [2023-12-27 02:43:54,533][105692] Updated weights for policy 0, policy_version 1556384 (0.0005) [2023-12-27 02:43:54,938][105620] Updated weights for policy 1, policy_version 1559664 (0.0007) [2023-12-27 02:43:55,003][105620] Updated weights for policy 1, policy_version 1559674 (0.0010) [2023-12-27 02:43:55,055][105620] Updated weights for policy 1, policy_version 1559684 (0.0010) [2023-12-27 02:43:55,231][105692] Updated weights for policy 0, policy_version 1556394 (0.0009) [2023-12-27 02:43:55,293][105692] Updated weights for policy 0, policy_version 1556404 (0.0009) [2023-12-27 02:43:55,352][105692] Updated weights for policy 0, policy_version 1556414 (0.0010) [2023-12-27 02:43:55,420][105692] Updated weights for policy 0, policy_version 1556424 (0.0008) [2023-12-27 02:43:55,683][105620] Updated weights for policy 1, policy_version 1559694 (0.0007) [2023-12-27 02:43:55,739][105620] Updated weights for policy 1, policy_version 1559704 (0.0006) [2023-12-27 02:43:55,794][105620] Updated weights for policy 1, policy_version 1559714 (0.0010) [2023-12-27 02:43:56,062][104569] Fps is (10 sec: 18842.3, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 797843456. Throughput: 0: 9547.9, 1: 10032.7. Samples: 797853304. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:43:56,062][104569] Avg episode reward: [(0, '8719.946'), (1, '8809.106')] [2023-12-27 02:43:56,144][105692] Updated weights for policy 0, policy_version 1556434 (0.0010) [2023-12-27 02:43:56,196][105692] Updated weights for policy 0, policy_version 1556444 (0.0010) [2023-12-27 02:43:56,248][105692] Updated weights for policy 0, policy_version 1556454 (0.0010) [2023-12-27 02:43:56,495][105620] Updated weights for policy 1, policy_version 1559724 (0.0010) [2023-12-27 02:43:56,548][105620] Updated weights for policy 1, policy_version 1559734 (0.0008) [2023-12-27 02:43:56,601][105620] Updated weights for policy 1, policy_version 1559744 (0.0005) [2023-12-27 02:43:56,989][105692] Updated weights for policy 0, policy_version 1556464 (0.0010) [2023-12-27 02:43:57,043][105692] Updated weights for policy 0, policy_version 1556474 (0.0010) [2023-12-27 02:43:57,089][105692] Updated weights for policy 0, policy_version 1556484 (0.0007) [2023-12-27 02:43:57,328][105620] Updated weights for policy 1, policy_version 1559754 (0.0009) [2023-12-27 02:43:57,379][105620] Updated weights for policy 1, policy_version 1559764 (0.0010) [2023-12-27 02:43:57,436][105620] Updated weights for policy 1, policy_version 1559774 (0.0010) [2023-12-27 02:43:57,499][105620] Updated weights for policy 1, policy_version 1559784 (0.0010) [2023-12-27 02:43:57,735][105692] Updated weights for policy 0, policy_version 1556494 (0.0007) [2023-12-27 02:43:57,790][105692] Updated weights for policy 0, policy_version 1556504 (0.0008) [2023-12-27 02:43:57,845][105692] Updated weights for policy 0, policy_version 1556514 (0.0008) [2023-12-27 02:43:58,228][105620] Updated weights for policy 1, policy_version 1559794 (0.0010) [2023-12-27 02:43:58,284][105620] Updated weights for policy 1, policy_version 1559804 (0.0008) [2023-12-27 02:43:58,345][105620] Updated weights for policy 1, policy_version 1559814 (0.0010) [2023-12-27 02:43:58,612][105692] Updated weights for policy 0, policy_version 1556524 (0.0009) [2023-12-27 02:43:58,680][105692] Updated weights for policy 0, policy_version 1556534 (0.0008) [2023-12-27 02:43:58,741][105692] Updated weights for policy 0, policy_version 1556544 (0.0007) [2023-12-27 02:43:59,160][105620] Updated weights for policy 1, policy_version 1559824 (0.0009) [2023-12-27 02:43:59,224][105620] Updated weights for policy 1, policy_version 1559834 (0.0008) [2023-12-27 02:43:59,288][105620] Updated weights for policy 1, policy_version 1559844 (0.0008) [2023-12-27 02:43:59,510][105692] Updated weights for policy 0, policy_version 1556554 (0.0007) [2023-12-27 02:43:59,557][105692] Updated weights for policy 0, policy_version 1556564 (0.0005) [2023-12-27 02:43:59,610][105692] Updated weights for policy 0, policy_version 1556574 (0.0005) [2023-12-27 02:43:59,667][105692] Updated weights for policy 0, policy_version 1556584 (0.0005) [2023-12-27 02:43:59,930][105620] Updated weights for policy 1, policy_version 1559854 (0.0010) [2023-12-27 02:43:59,995][105620] Updated weights for policy 1, policy_version 1559864 (0.0010) [2023-12-27 02:44:00,058][105620] Updated weights for policy 1, policy_version 1559874 (0.0010) [2023-12-27 02:44:00,355][105692] Updated weights for policy 0, policy_version 1556594 (0.0007) [2023-12-27 02:44:00,415][105692] Updated weights for policy 0, policy_version 1556604 (0.0008) [2023-12-27 02:44:00,464][105692] Updated weights for policy 0, policy_version 1556614 (0.0008) [2023-12-27 02:44:00,781][105620] Updated weights for policy 1, policy_version 1559884 (0.0009) [2023-12-27 02:44:00,843][105620] Updated weights for policy 1, policy_version 1559894 (0.0008) [2023-12-27 02:44:00,907][105620] Updated weights for policy 1, policy_version 1559904 (0.0010) [2023-12-27 02:44:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 797941760. Throughput: 0: 9551.9, 1: 10054.2. Samples: 797910640. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:44:01,062][104569] Avg episode reward: [(0, '8808.032'), (1, '8901.943')] [2023-12-27 02:44:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001556616_398548992.pth... [2023-12-27 02:44:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001559912_399392768.pth... [2023-12-27 02:44:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001555528_398270464.pth [2023-12-27 02:44:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001558728_399089664.pth [2023-12-27 02:44:01,196][105692] Updated weights for policy 0, policy_version 1556624 (0.0008) [2023-12-27 02:44:01,257][105692] Updated weights for policy 0, policy_version 1556634 (0.0008) [2023-12-27 02:44:01,310][105692] Updated weights for policy 0, policy_version 1556644 (0.0008) [2023-12-27 02:44:01,666][105620] Updated weights for policy 1, policy_version 1559914 (0.0010) [2023-12-27 02:44:01,732][105620] Updated weights for policy 1, policy_version 1559924 (0.0010) [2023-12-27 02:44:01,783][105620] Updated weights for policy 1, policy_version 1559934 (0.0009) [2023-12-27 02:44:01,843][105620] Updated weights for policy 1, policy_version 1559944 (0.0009) [2023-12-27 02:44:02,086][105692] Updated weights for policy 0, policy_version 1556654 (0.0007) [2023-12-27 02:44:02,144][105692] Updated weights for policy 0, policy_version 1556664 (0.0009) [2023-12-27 02:44:02,199][105692] Updated weights for policy 0, policy_version 1556674 (0.0009) [2023-12-27 02:44:02,605][105620] Updated weights for policy 1, policy_version 1559954 (0.0007) [2023-12-27 02:44:02,656][105620] Updated weights for policy 1, policy_version 1559964 (0.0005) [2023-12-27 02:44:02,710][105620] Updated weights for policy 1, policy_version 1559974 (0.0005) [2023-12-27 02:44:02,957][105692] Updated weights for policy 0, policy_version 1556684 (0.0009) [2023-12-27 02:44:03,016][105692] Updated weights for policy 0, policy_version 1556694 (0.0009) [2023-12-27 02:44:03,077][105692] Updated weights for policy 0, policy_version 1556704 (0.0009) [2023-12-27 02:44:03,298][105620] Updated weights for policy 1, policy_version 1559984 (0.0005) [2023-12-27 02:44:03,342][105620] Updated weights for policy 1, policy_version 1559994 (0.0005) [2023-12-27 02:44:03,389][105620] Updated weights for policy 1, policy_version 1560004 (0.0006) [2023-12-27 02:44:03,930][105692] Updated weights for policy 0, policy_version 1556714 (0.0009) [2023-12-27 02:44:03,956][105620] Updated weights for policy 1, policy_version 1560014 (0.0008) [2023-12-27 02:44:03,988][105692] Updated weights for policy 0, policy_version 1556724 (0.0008) [2023-12-27 02:44:04,012][105620] Updated weights for policy 1, policy_version 1560024 (0.0008) [2023-12-27 02:44:04,046][105692] Updated weights for policy 0, policy_version 1556734 (0.0008) [2023-12-27 02:44:04,060][105620] Updated weights for policy 1, policy_version 1560034 (0.0008) [2023-12-27 02:44:04,109][105692] Updated weights for policy 0, policy_version 1556744 (0.0007) [2023-12-27 02:44:04,787][105620] Updated weights for policy 1, policy_version 1560044 (0.0009) [2023-12-27 02:44:04,832][105692] Updated weights for policy 0, policy_version 1556754 (0.0011) [2023-12-27 02:44:04,850][105620] Updated weights for policy 1, policy_version 1560054 (0.0011) [2023-12-27 02:44:04,895][105692] Updated weights for policy 0, policy_version 1556764 (0.0011) [2023-12-27 02:44:04,914][105620] Updated weights for policy 1, policy_version 1560064 (0.0010) [2023-12-27 02:44:04,957][105692] Updated weights for policy 0, policy_version 1556774 (0.0011) [2023-12-27 02:44:05,521][105620] Updated weights for policy 1, policy_version 1560074 (0.0009) [2023-12-27 02:44:05,580][105620] Updated weights for policy 1, policy_version 1560084 (0.0005) [2023-12-27 02:44:05,626][105620] Updated weights for policy 1, policy_version 1560094 (0.0005) [2023-12-27 02:44:05,680][105620] Updated weights for policy 1, policy_version 1560104 (0.0007) [2023-12-27 02:44:05,688][105692] Updated weights for policy 0, policy_version 1556784 (0.0011) [2023-12-27 02:44:05,746][105692] Updated weights for policy 0, policy_version 1556794 (0.0010) [2023-12-27 02:44:05,795][105692] Updated weights for policy 0, policy_version 1556804 (0.0005) [2023-12-27 02:44:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 798040064. Throughput: 0: 9454.2, 1: 10105.4. Samples: 798026608. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:44:06,063][104569] Avg episode reward: [(0, '8627.582'), (1, '8999.318')] [2023-12-27 02:44:06,235][105620] Updated weights for policy 1, policy_version 1560114 (0.0008) [2023-12-27 02:44:06,290][105620] Updated weights for policy 1, policy_version 1560124 (0.0006) [2023-12-27 02:44:06,360][105620] Updated weights for policy 1, policy_version 1560134 (0.0006) [2023-12-27 02:44:06,593][105692] Updated weights for policy 0, policy_version 1556814 (0.0009) [2023-12-27 02:44:06,649][105692] Updated weights for policy 0, policy_version 1556824 (0.0009) [2023-12-27 02:44:06,710][105692] Updated weights for policy 0, policy_version 1556834 (0.0009) [2023-12-27 02:44:06,960][105620] Updated weights for policy 1, policy_version 1560144 (0.0005) [2023-12-27 02:44:07,019][105620] Updated weights for policy 1, policy_version 1560154 (0.0008) [2023-12-27 02:44:07,078][105620] Updated weights for policy 1, policy_version 1560164 (0.0008) [2023-12-27 02:44:07,536][105692] Updated weights for policy 0, policy_version 1556844 (0.0010) [2023-12-27 02:44:07,591][105692] Updated weights for policy 0, policy_version 1556854 (0.0010) [2023-12-27 02:44:07,648][105692] Updated weights for policy 0, policy_version 1556864 (0.0010) [2023-12-27 02:44:07,803][105620] Updated weights for policy 1, policy_version 1560174 (0.0009) [2023-12-27 02:44:07,860][105620] Updated weights for policy 1, policy_version 1560184 (0.0010) [2023-12-27 02:44:07,918][105620] Updated weights for policy 1, policy_version 1560194 (0.0010) [2023-12-27 02:44:08,385][105692] Updated weights for policy 0, policy_version 1556874 (0.0010) [2023-12-27 02:44:08,441][105692] Updated weights for policy 0, policy_version 1556884 (0.0011) [2023-12-27 02:44:08,513][105692] Updated weights for policy 0, policy_version 1556894 (0.0011) [2023-12-27 02:44:08,582][105692] Updated weights for policy 0, policy_version 1556904 (0.0011) [2023-12-27 02:44:08,589][105620] Updated weights for policy 1, policy_version 1560204 (0.0008) [2023-12-27 02:44:08,646][105620] Updated weights for policy 1, policy_version 1560214 (0.0005) [2023-12-27 02:44:08,709][105620] Updated weights for policy 1, policy_version 1560224 (0.0005) [2023-12-27 02:44:09,201][105692] Updated weights for policy 0, policy_version 1556914 (0.0011) [2023-12-27 02:44:09,266][105692] Updated weights for policy 0, policy_version 1556924 (0.0011) [2023-12-27 02:44:09,318][105692] Updated weights for policy 0, policy_version 1556934 (0.0010) [2023-12-27 02:44:09,414][105620] Updated weights for policy 1, policy_version 1560234 (0.0006) [2023-12-27 02:44:09,475][105620] Updated weights for policy 1, policy_version 1560244 (0.0008) [2023-12-27 02:44:09,530][105620] Updated weights for policy 1, policy_version 1560254 (0.0008) [2023-12-27 02:44:09,586][105620] Updated weights for policy 1, policy_version 1560264 (0.0008) [2023-12-27 02:44:10,104][105692] Updated weights for policy 0, policy_version 1556944 (0.0011) [2023-12-27 02:44:10,164][105692] Updated weights for policy 0, policy_version 1556954 (0.0011) [2023-12-27 02:44:10,213][105692] Updated weights for policy 0, policy_version 1556964 (0.0011) [2023-12-27 02:44:10,422][105620] Updated weights for policy 1, policy_version 1560274 (0.0010) [2023-12-27 02:44:10,485][105620] Updated weights for policy 1, policy_version 1560284 (0.0010) [2023-12-27 02:44:10,554][105620] Updated weights for policy 1, policy_version 1560294 (0.0010) [2023-12-27 02:44:10,845][105692] Updated weights for policy 0, policy_version 1556974 (0.0007) [2023-12-27 02:44:10,890][105692] Updated weights for policy 0, policy_version 1556984 (0.0009) [2023-12-27 02:44:10,938][105692] Updated weights for policy 0, policy_version 1556994 (0.0009) [2023-12-27 02:44:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 798138368. Throughput: 0: 9404.6, 1: 10176.7. Samples: 798144252. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:44:11,062][104569] Avg episode reward: [(0, '8808.110'), (1, '9001.136')] [2023-12-27 02:44:11,381][105620] Updated weights for policy 1, policy_version 1560304 (0.0008) [2023-12-27 02:44:11,435][105620] Updated weights for policy 1, policy_version 1560314 (0.0005) [2023-12-27 02:44:11,495][105620] Updated weights for policy 1, policy_version 1560324 (0.0006) [2023-12-27 02:44:11,739][105692] Updated weights for policy 0, policy_version 1557004 (0.0009) [2023-12-27 02:44:11,800][105692] Updated weights for policy 0, policy_version 1557014 (0.0007) [2023-12-27 02:44:11,855][105692] Updated weights for policy 0, policy_version 1557024 (0.0009) [2023-12-27 02:44:12,238][105620] Updated weights for policy 1, policy_version 1560334 (0.0008) [2023-12-27 02:44:12,296][105620] Updated weights for policy 1, policy_version 1560344 (0.0009) [2023-12-27 02:44:12,358][105620] Updated weights for policy 1, policy_version 1560354 (0.0008) [2023-12-27 02:44:12,592][105692] Updated weights for policy 0, policy_version 1557034 (0.0010) [2023-12-27 02:44:12,656][105692] Updated weights for policy 0, policy_version 1557044 (0.0009) [2023-12-27 02:44:12,717][105692] Updated weights for policy 0, policy_version 1557054 (0.0009) [2023-12-27 02:44:12,783][105692] Updated weights for policy 0, policy_version 1557064 (0.0009) [2023-12-27 02:44:13,069][105620] Updated weights for policy 1, policy_version 1560364 (0.0008) [2023-12-27 02:44:13,119][105620] Updated weights for policy 1, policy_version 1560374 (0.0008) [2023-12-27 02:44:13,166][105620] Updated weights for policy 1, policy_version 1560384 (0.0008) [2023-12-27 02:44:13,513][105692] Updated weights for policy 0, policy_version 1557074 (0.0009) [2023-12-27 02:44:13,576][105692] Updated weights for policy 0, policy_version 1557084 (0.0009) [2023-12-27 02:44:13,625][105692] Updated weights for policy 0, policy_version 1557094 (0.0008) [2023-12-27 02:44:13,957][105620] Updated weights for policy 1, policy_version 1560394 (0.0009) [2023-12-27 02:44:14,012][105620] Updated weights for policy 1, policy_version 1560404 (0.0009) [2023-12-27 02:44:14,073][105620] Updated weights for policy 1, policy_version 1560414 (0.0009) [2023-12-27 02:44:14,134][105620] Updated weights for policy 1, policy_version 1560424 (0.0008) [2023-12-27 02:44:14,284][105692] Updated weights for policy 0, policy_version 1557104 (0.0009) [2023-12-27 02:44:14,345][105692] Updated weights for policy 0, policy_version 1557114 (0.0009) [2023-12-27 02:44:14,398][105692] Updated weights for policy 0, policy_version 1557124 (0.0008) [2023-12-27 02:44:14,784][105620] Updated weights for policy 1, policy_version 1560434 (0.0009) [2023-12-27 02:44:14,847][105620] Updated weights for policy 1, policy_version 1560444 (0.0011) [2023-12-27 02:44:14,910][105620] Updated weights for policy 1, policy_version 1560454 (0.0011) [2023-12-27 02:44:15,231][105692] Updated weights for policy 0, policy_version 1557134 (0.0008) [2023-12-27 02:44:15,287][105692] Updated weights for policy 0, policy_version 1557144 (0.0009) [2023-12-27 02:44:15,339][105692] Updated weights for policy 0, policy_version 1557154 (0.0008) [2023-12-27 02:44:15,630][105620] Updated weights for policy 1, policy_version 1560464 (0.0007) [2023-12-27 02:44:15,687][105620] Updated weights for policy 1, policy_version 1560474 (0.0008) [2023-12-27 02:44:15,753][105620] Updated weights for policy 1, policy_version 1560484 (0.0009) [2023-12-27 02:44:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 798228480. Throughput: 0: 9434.5, 1: 10069.8. Samples: 798200896. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:44:16,063][104569] Avg episode reward: [(0, '8529.473'), (1, '9090.713')] [2023-12-27 02:44:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001557160_398688256.pth... [2023-12-27 02:44:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001560488_399540224.pth... [2023-12-27 02:44:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001556072_398409728.pth [2023-12-27 02:44:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001559336_399245312.pth [2023-12-27 02:44:16,115][105692] Updated weights for policy 0, policy_version 1557164 (0.0008) [2023-12-27 02:44:16,167][105692] Updated weights for policy 0, policy_version 1557174 (0.0005) [2023-12-27 02:44:16,219][105692] Updated weights for policy 0, policy_version 1557184 (0.0006) [2023-12-27 02:44:16,390][105620] Updated weights for policy 1, policy_version 1560494 (0.0010) [2023-12-27 02:44:16,450][105620] Updated weights for policy 1, policy_version 1560504 (0.0011) [2023-12-27 02:44:16,508][105620] Updated weights for policy 1, policy_version 1560514 (0.0010) [2023-12-27 02:44:16,908][105692] Updated weights for policy 0, policy_version 1557194 (0.0007) [2023-12-27 02:44:16,968][105692] Updated weights for policy 0, policy_version 1557204 (0.0010) [2023-12-27 02:44:17,024][105692] Updated weights for policy 0, policy_version 1557214 (0.0011) [2023-12-27 02:44:17,026][105585] KL-divergence is very high: 102.9769 [2023-12-27 02:44:17,071][105585] KL-divergence is very high: 110.8938 [2023-12-27 02:44:17,082][105692] Updated weights for policy 0, policy_version 1557224 (0.0006) [2023-12-27 02:44:17,189][105620] Updated weights for policy 1, policy_version 1560524 (0.0009) [2023-12-27 02:44:17,243][105620] Updated weights for policy 1, policy_version 1560534 (0.0006) [2023-12-27 02:44:17,300][105620] Updated weights for policy 1, policy_version 1560544 (0.0005) [2023-12-27 02:44:17,807][105692] Updated weights for policy 0, policy_version 1557234 (0.0010) [2023-12-27 02:44:17,866][105692] Updated weights for policy 0, policy_version 1557244 (0.0009) [2023-12-27 02:44:17,918][105692] Updated weights for policy 0, policy_version 1557254 (0.0009) [2023-12-27 02:44:17,931][105620] Updated weights for policy 1, policy_version 1560554 (0.0007) [2023-12-27 02:44:17,992][105620] Updated weights for policy 1, policy_version 1560564 (0.0005) [2023-12-27 02:44:18,054][105620] Updated weights for policy 1, policy_version 1560574 (0.0005) [2023-12-27 02:44:18,111][105620] Updated weights for policy 1, policy_version 1560584 (0.0005) [2023-12-27 02:44:18,690][105620] Updated weights for policy 1, policy_version 1560594 (0.0008) [2023-12-27 02:44:18,692][105692] Updated weights for policy 0, policy_version 1557264 (0.0008) [2023-12-27 02:44:18,747][105620] Updated weights for policy 1, policy_version 1560604 (0.0006) [2023-12-27 02:44:18,749][105692] Updated weights for policy 0, policy_version 1557274 (0.0007) [2023-12-27 02:44:18,803][105620] Updated weights for policy 1, policy_version 1560614 (0.0007) [2023-12-27 02:44:18,809][105692] Updated weights for policy 0, policy_version 1557284 (0.0007) [2023-12-27 02:44:19,474][105692] Updated weights for policy 0, policy_version 1557294 (0.0007) [2023-12-27 02:44:19,543][105692] Updated weights for policy 0, policy_version 1557304 (0.0006) [2023-12-27 02:44:19,607][105692] Updated weights for policy 0, policy_version 1557314 (0.0006) [2023-12-27 02:44:19,609][105620] Updated weights for policy 1, policy_version 1560624 (0.0008) [2023-12-27 02:44:19,677][105620] Updated weights for policy 1, policy_version 1560634 (0.0010) [2023-12-27 02:44:19,744][105620] Updated weights for policy 1, policy_version 1560644 (0.0010) [2023-12-27 02:44:20,297][105692] Updated weights for policy 0, policy_version 1557324 (0.0007) [2023-12-27 02:44:20,362][105692] Updated weights for policy 0, policy_version 1557334 (0.0009) [2023-12-27 02:44:20,424][105692] Updated weights for policy 0, policy_version 1557344 (0.0007) [2023-12-27 02:44:20,442][105620] Updated weights for policy 1, policy_version 1560654 (0.0009) [2023-12-27 02:44:20,500][105620] Updated weights for policy 1, policy_version 1560664 (0.0008) [2023-12-27 02:44:20,565][105620] Updated weights for policy 1, policy_version 1560674 (0.0008) [2023-12-27 02:44:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 798326784. Throughput: 0: 9445.8, 1: 9978.4. Samples: 798318812. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:44:21,062][104569] Avg episode reward: [(0, '8625.142'), (1, '9267.582')] [2023-12-27 02:44:21,137][105692] Updated weights for policy 0, policy_version 1557354 (0.0006) [2023-12-27 02:44:21,197][105692] Updated weights for policy 0, policy_version 1557364 (0.0009) [2023-12-27 02:44:21,261][105692] Updated weights for policy 0, policy_version 1557374 (0.0008) [2023-12-27 02:44:21,316][105692] Updated weights for policy 0, policy_version 1557384 (0.0009) [2023-12-27 02:44:21,343][105620] Updated weights for policy 1, policy_version 1560684 (0.0008) [2023-12-27 02:44:21,414][105620] Updated weights for policy 1, policy_version 1560694 (0.0009) [2023-12-27 02:44:21,465][105620] Updated weights for policy 1, policy_version 1560704 (0.0009) [2023-12-27 02:44:22,084][105692] Updated weights for policy 0, policy_version 1557394 (0.0009) [2023-12-27 02:44:22,143][105692] Updated weights for policy 0, policy_version 1557404 (0.0009) [2023-12-27 02:44:22,205][105692] Updated weights for policy 0, policy_version 1557414 (0.0008) [2023-12-27 02:44:22,244][105620] Updated weights for policy 1, policy_version 1560714 (0.0009) [2023-12-27 02:44:22,304][105620] Updated weights for policy 1, policy_version 1560724 (0.0010) [2023-12-27 02:44:22,371][105620] Updated weights for policy 1, policy_version 1560734 (0.0010) [2023-12-27 02:44:22,430][105620] Updated weights for policy 1, policy_version 1560744 (0.0009) [2023-12-27 02:44:22,890][105692] Updated weights for policy 0, policy_version 1557424 (0.0009) [2023-12-27 02:44:22,948][105692] Updated weights for policy 0, policy_version 1557434 (0.0010) [2023-12-27 02:44:23,034][105692] Updated weights for policy 0, policy_version 1557444 (0.0009) [2023-12-27 02:44:23,223][105620] Updated weights for policy 1, policy_version 1560754 (0.0009) [2023-12-27 02:44:23,285][105620] Updated weights for policy 1, policy_version 1560764 (0.0009) [2023-12-27 02:44:23,336][105620] Updated weights for policy 1, policy_version 1560774 (0.0009) [2023-12-27 02:44:23,757][105692] Updated weights for policy 0, policy_version 1557454 (0.0009) [2023-12-27 02:44:23,810][105692] Updated weights for policy 0, policy_version 1557464 (0.0009) [2023-12-27 02:44:23,857][105692] Updated weights for policy 0, policy_version 1557474 (0.0008) [2023-12-27 02:44:24,100][105620] Updated weights for policy 1, policy_version 1560784 (0.0009) [2023-12-27 02:44:24,153][105620] Updated weights for policy 1, policy_version 1560794 (0.0009) [2023-12-27 02:44:24,212][105620] Updated weights for policy 1, policy_version 1560804 (0.0009) [2023-12-27 02:44:24,620][105692] Updated weights for policy 0, policy_version 1557484 (0.0009) [2023-12-27 02:44:24,671][105692] Updated weights for policy 0, policy_version 1557494 (0.0009) [2023-12-27 02:44:24,733][105692] Updated weights for policy 0, policy_version 1557504 (0.0009) [2023-12-27 02:44:24,955][105620] Updated weights for policy 1, policy_version 1560814 (0.0008) [2023-12-27 02:44:25,011][105620] Updated weights for policy 1, policy_version 1560824 (0.0005) [2023-12-27 02:44:25,067][105620] Updated weights for policy 1, policy_version 1560834 (0.0005) [2023-12-27 02:44:25,441][105692] Updated weights for policy 0, policy_version 1557514 (0.0009) [2023-12-27 02:44:25,492][105692] Updated weights for policy 0, policy_version 1557524 (0.0007) [2023-12-27 02:44:25,537][105692] Updated weights for policy 0, policy_version 1557534 (0.0008) [2023-12-27 02:44:25,587][105692] Updated weights for policy 0, policy_version 1557544 (0.0006) [2023-12-27 02:44:25,679][105620] Updated weights for policy 1, policy_version 1560844 (0.0008) [2023-12-27 02:44:25,733][105620] Updated weights for policy 1, policy_version 1560854 (0.0006) [2023-12-27 02:44:25,788][105620] Updated weights for policy 1, policy_version 1560864 (0.0005) [2023-12-27 02:44:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.8). Total num frames: 798425088. Throughput: 0: 9519.0, 1: 9852.0. Samples: 798433084. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:44:26,063][104569] Avg episode reward: [(0, '8810.492'), (1, '9086.014')] [2023-12-27 02:44:26,285][105692] Updated weights for policy 0, policy_version 1557554 (0.0009) [2023-12-27 02:44:26,340][105692] Updated weights for policy 0, policy_version 1557564 (0.0009) [2023-12-27 02:44:26,397][105692] Updated weights for policy 0, policy_version 1557574 (0.0009) [2023-12-27 02:44:26,417][105620] Updated weights for policy 1, policy_version 1560874 (0.0005) [2023-12-27 02:44:26,479][105620] Updated weights for policy 1, policy_version 1560884 (0.0006) [2023-12-27 02:44:26,539][105620] Updated weights for policy 1, policy_version 1560894 (0.0008) [2023-12-27 02:44:26,594][105620] Updated weights for policy 1, policy_version 1560904 (0.0006) [2023-12-27 02:44:27,103][105692] Updated weights for policy 0, policy_version 1557584 (0.0010) [2023-12-27 02:44:27,155][105692] Updated weights for policy 0, policy_version 1557594 (0.0010) [2023-12-27 02:44:27,214][105692] Updated weights for policy 0, policy_version 1557604 (0.0011) [2023-12-27 02:44:27,221][105620] Updated weights for policy 1, policy_version 1560914 (0.0011) [2023-12-27 02:44:27,275][105620] Updated weights for policy 1, policy_version 1560924 (0.0010) [2023-12-27 02:44:27,334][105620] Updated weights for policy 1, policy_version 1560934 (0.0008) [2023-12-27 02:44:27,852][105692] Updated weights for policy 0, policy_version 1557614 (0.0007) [2023-12-27 02:44:27,901][105692] Updated weights for policy 0, policy_version 1557624 (0.0005) [2023-12-27 02:44:27,955][105692] Updated weights for policy 0, policy_version 1557634 (0.0005) [2023-12-27 02:44:28,023][105620] Updated weights for policy 1, policy_version 1560944 (0.0009) [2023-12-27 02:44:28,082][105620] Updated weights for policy 1, policy_version 1560954 (0.0009) [2023-12-27 02:44:28,137][105620] Updated weights for policy 1, policy_version 1560964 (0.0010) [2023-12-27 02:44:28,533][105692] Updated weights for policy 0, policy_version 1557644 (0.0005) [2023-12-27 02:44:28,592][105692] Updated weights for policy 0, policy_version 1557654 (0.0009) [2023-12-27 02:44:28,650][105692] Updated weights for policy 0, policy_version 1557664 (0.0010) [2023-12-27 02:44:28,808][105620] Updated weights for policy 1, policy_version 1560974 (0.0010) [2023-12-27 02:44:28,857][105620] Updated weights for policy 1, policy_version 1560984 (0.0010) [2023-12-27 02:44:28,904][105620] Updated weights for policy 1, policy_version 1560994 (0.0010) [2023-12-27 02:44:29,229][105692] Updated weights for policy 0, policy_version 1557674 (0.0010) [2023-12-27 02:44:29,287][105692] Updated weights for policy 0, policy_version 1557684 (0.0010) [2023-12-27 02:44:29,340][105692] Updated weights for policy 0, policy_version 1557694 (0.0007) [2023-12-27 02:44:29,402][105692] Updated weights for policy 0, policy_version 1557704 (0.0010) [2023-12-27 02:44:29,620][105620] Updated weights for policy 1, policy_version 1561004 (0.0009) [2023-12-27 02:44:29,686][105620] Updated weights for policy 1, policy_version 1561014 (0.0008) [2023-12-27 02:44:29,749][105620] Updated weights for policy 1, policy_version 1561024 (0.0008) [2023-12-27 02:44:30,148][105692] Updated weights for policy 0, policy_version 1557714 (0.0008) [2023-12-27 02:44:30,209][105692] Updated weights for policy 0, policy_version 1557724 (0.0010) [2023-12-27 02:44:30,259][105692] Updated weights for policy 0, policy_version 1557734 (0.0009) [2023-12-27 02:44:30,470][105620] Updated weights for policy 1, policy_version 1561034 (0.0008) [2023-12-27 02:44:30,527][105620] Updated weights for policy 1, policy_version 1561044 (0.0009) [2023-12-27 02:44:30,582][105620] Updated weights for policy 1, policy_version 1561054 (0.0009) [2023-12-27 02:44:30,642][105620] Updated weights for policy 1, policy_version 1561064 (0.0009) [2023-12-27 02:44:30,941][105692] Updated weights for policy 0, policy_version 1557744 (0.0010) [2023-12-27 02:44:31,002][105692] Updated weights for policy 0, policy_version 1557754 (0.0010) [2023-12-27 02:44:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 798523392. Throughput: 0: 9599.4, 1: 9926.3. Samples: 798496504. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:44:31,062][104569] Avg episode reward: [(0, '8529.152'), (1, '8994.554')] [2023-12-27 02:44:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001561064_399687680.pth... [2023-12-27 02:44:31,070][105692] Updated weights for policy 0, policy_version 1557764 (0.0009) [2023-12-27 02:44:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001559912_399392768.pth [2023-12-27 02:44:31,094][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001557768_398843904.pth... [2023-12-27 02:44:31,098][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001556616_398548992.pth [2023-12-27 02:44:31,392][105620] Updated weights for policy 1, policy_version 1561074 (0.0009) [2023-12-27 02:44:31,452][105620] Updated weights for policy 1, policy_version 1561084 (0.0008) [2023-12-27 02:44:31,509][105620] Updated weights for policy 1, policy_version 1561094 (0.0007) [2023-12-27 02:44:31,810][105692] Updated weights for policy 0, policy_version 1557774 (0.0008) [2023-12-27 02:44:31,868][105692] Updated weights for policy 0, policy_version 1557784 (0.0005) [2023-12-27 02:44:31,914][105692] Updated weights for policy 0, policy_version 1557794 (0.0005) [2023-12-27 02:44:32,285][105620] Updated weights for policy 1, policy_version 1561104 (0.0009) [2023-12-27 02:44:32,330][105620] Updated weights for policy 1, policy_version 1561114 (0.0007) [2023-12-27 02:44:32,397][105620] Updated weights for policy 1, policy_version 1561124 (0.0008) [2023-12-27 02:44:32,536][105692] Updated weights for policy 0, policy_version 1557804 (0.0007) [2023-12-27 02:44:32,600][105692] Updated weights for policy 0, policy_version 1557814 (0.0009) [2023-12-27 02:44:32,656][105692] Updated weights for policy 0, policy_version 1557824 (0.0010) [2023-12-27 02:44:33,125][105620] Updated weights for policy 1, policy_version 1561134 (0.0007) [2023-12-27 02:44:33,183][105620] Updated weights for policy 1, policy_version 1561144 (0.0006) [2023-12-27 02:44:33,239][105620] Updated weights for policy 1, policy_version 1561154 (0.0005) [2023-12-27 02:44:33,269][105692] Updated weights for policy 0, policy_version 1557834 (0.0008) [2023-12-27 02:44:33,327][105692] Updated weights for policy 0, policy_version 1557844 (0.0005) [2023-12-27 02:44:33,375][105692] Updated weights for policy 0, policy_version 1557854 (0.0005) [2023-12-27 02:44:33,421][105692] Updated weights for policy 0, policy_version 1557864 (0.0005) [2023-12-27 02:44:33,948][105620] Updated weights for policy 1, policy_version 1561164 (0.0007) [2023-12-27 02:44:33,986][105692] Updated weights for policy 0, policy_version 1557874 (0.0006) [2023-12-27 02:44:34,002][105620] Updated weights for policy 1, policy_version 1561174 (0.0007) [2023-12-27 02:44:34,034][105692] Updated weights for policy 0, policy_version 1557884 (0.0010) [2023-12-27 02:44:34,060][105620] Updated weights for policy 1, policy_version 1561184 (0.0006) [2023-12-27 02:44:34,082][105692] Updated weights for policy 0, policy_version 1557894 (0.0010) [2023-12-27 02:44:34,735][105620] Updated weights for policy 1, policy_version 1561194 (0.0006) [2023-12-27 02:44:34,797][105620] Updated weights for policy 1, policy_version 1561204 (0.0008) [2023-12-27 02:44:34,852][105620] Updated weights for policy 1, policy_version 1561214 (0.0007) [2023-12-27 02:44:34,858][105692] Updated weights for policy 0, policy_version 1557904 (0.0011) [2023-12-27 02:44:34,914][105620] Updated weights for policy 1, policy_version 1561224 (0.0008) [2023-12-27 02:44:34,920][105692] Updated weights for policy 0, policy_version 1557914 (0.0010) [2023-12-27 02:44:34,982][105692] Updated weights for policy 0, policy_version 1557924 (0.0011) [2023-12-27 02:44:35,600][105692] Updated weights for policy 0, policy_version 1557934 (0.0007) [2023-12-27 02:44:35,666][105692] Updated weights for policy 0, policy_version 1557944 (0.0005) [2023-12-27 02:44:35,711][105620] Updated weights for policy 1, policy_version 1561234 (0.0007) [2023-12-27 02:44:35,730][105692] Updated weights for policy 0, policy_version 1557954 (0.0005) [2023-12-27 02:44:35,763][105620] Updated weights for policy 1, policy_version 1561244 (0.0009) [2023-12-27 02:44:35,816][105620] Updated weights for policy 1, policy_version 1561254 (0.0009) [2023-12-27 02:44:36,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 798629888. Throughput: 0: 9720.8, 1: 9867.6. Samples: 798617220. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:44:36,062][104569] Avg episode reward: [(0, '8530.249'), (1, '8897.899')] [2023-12-27 02:44:36,266][105692] Updated weights for policy 0, policy_version 1557964 (0.0007) [2023-12-27 02:44:36,322][105692] Updated weights for policy 0, policy_version 1557974 (0.0011) [2023-12-27 02:44:36,381][105692] Updated weights for policy 0, policy_version 1557984 (0.0011) [2023-12-27 02:44:36,624][105620] Updated weights for policy 1, policy_version 1561264 (0.0008) [2023-12-27 02:44:36,687][105620] Updated weights for policy 1, policy_version 1561274 (0.0008) [2023-12-27 02:44:36,753][105620] Updated weights for policy 1, policy_version 1561284 (0.0006) [2023-12-27 02:44:37,124][105692] Updated weights for policy 0, policy_version 1557994 (0.0011) [2023-12-27 02:44:37,175][105692] Updated weights for policy 0, policy_version 1558004 (0.0011) [2023-12-27 02:44:37,223][105692] Updated weights for policy 0, policy_version 1558014 (0.0010) [2023-12-27 02:44:37,280][105692] Updated weights for policy 0, policy_version 1558024 (0.0006) [2023-12-27 02:44:37,411][105620] Updated weights for policy 1, policy_version 1561294 (0.0008) [2023-12-27 02:44:37,464][105620] Updated weights for policy 1, policy_version 1561304 (0.0008) [2023-12-27 02:44:37,516][105620] Updated weights for policy 1, policy_version 1561314 (0.0008) [2023-12-27 02:44:37,994][105692] Updated weights for policy 0, policy_version 1558034 (0.0010) [2023-12-27 02:44:38,052][105692] Updated weights for policy 0, policy_version 1558044 (0.0010) [2023-12-27 02:44:38,109][105692] Updated weights for policy 0, policy_version 1558054 (0.0010) [2023-12-27 02:44:38,279][105620] Updated weights for policy 1, policy_version 1561324 (0.0009) [2023-12-27 02:44:38,331][105620] Updated weights for policy 1, policy_version 1561334 (0.0008) [2023-12-27 02:44:38,397][105620] Updated weights for policy 1, policy_version 1561344 (0.0009) [2023-12-27 02:44:38,747][105692] Updated weights for policy 0, policy_version 1558064 (0.0006) [2023-12-27 02:44:38,794][105692] Updated weights for policy 0, policy_version 1558074 (0.0005) [2023-12-27 02:44:38,850][105692] Updated weights for policy 0, policy_version 1558084 (0.0005) [2023-12-27 02:44:39,272][105620] Updated weights for policy 1, policy_version 1561354 (0.0008) [2023-12-27 02:44:39,332][105620] Updated weights for policy 1, policy_version 1561364 (0.0009) [2023-12-27 02:44:39,401][105620] Updated weights for policy 1, policy_version 1561374 (0.0008) [2023-12-27 02:44:39,454][105692] Updated weights for policy 0, policy_version 1558094 (0.0008) [2023-12-27 02:44:39,460][105620] Updated weights for policy 1, policy_version 1561384 (0.0007) [2023-12-27 02:44:39,508][105692] Updated weights for policy 0, policy_version 1558104 (0.0005) [2023-12-27 02:44:39,563][105692] Updated weights for policy 0, policy_version 1558114 (0.0005) [2023-12-27 02:44:40,242][105620] Updated weights for policy 1, policy_version 1561394 (0.0009) [2023-12-27 02:44:40,293][105692] Updated weights for policy 0, policy_version 1558124 (0.0009) [2023-12-27 02:44:40,304][105620] Updated weights for policy 1, policy_version 1561404 (0.0008) [2023-12-27 02:44:40,354][105692] Updated weights for policy 0, policy_version 1558134 (0.0008) [2023-12-27 02:44:40,361][105620] Updated weights for policy 1, policy_version 1561414 (0.0006) [2023-12-27 02:44:40,412][105692] Updated weights for policy 0, policy_version 1558144 (0.0008) [2023-12-27 02:44:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 798720000. Throughput: 0: 9813.3, 1: 9733.3. Samples: 798732904. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:44:41,062][104569] Avg episode reward: [(0, '8623.439'), (1, '8989.478')] [2023-12-27 02:44:41,087][105620] Updated weights for policy 1, policy_version 1561424 (0.0007) [2023-12-27 02:44:41,099][105692] Updated weights for policy 0, policy_version 1558154 (0.0009) [2023-12-27 02:44:41,157][105620] Updated weights for policy 1, policy_version 1561434 (0.0008) [2023-12-27 02:44:41,166][105692] Updated weights for policy 0, policy_version 1558164 (0.0009) [2023-12-27 02:44:41,220][105620] Updated weights for policy 1, policy_version 1561444 (0.0005) [2023-12-27 02:44:41,220][105692] Updated weights for policy 0, policy_version 1558174 (0.0008) [2023-12-27 02:44:41,277][105692] Updated weights for policy 0, policy_version 1558184 (0.0009) [2023-12-27 02:44:41,944][105620] Updated weights for policy 1, policy_version 1561454 (0.0008) [2023-12-27 02:44:41,995][105620] Updated weights for policy 1, policy_version 1561464 (0.0008) [2023-12-27 02:44:42,045][105620] Updated weights for policy 1, policy_version 1561474 (0.0008) [2023-12-27 02:44:42,136][105692] Updated weights for policy 0, policy_version 1558194 (0.0009) [2023-12-27 02:44:42,188][105692] Updated weights for policy 0, policy_version 1558204 (0.0010) [2023-12-27 02:44:42,245][105692] Updated weights for policy 0, policy_version 1558215 (0.0009) [2023-12-27 02:44:42,785][105620] Updated weights for policy 1, policy_version 1561484 (0.0009) [2023-12-27 02:44:42,843][105620] Updated weights for policy 1, policy_version 1561494 (0.0009) [2023-12-27 02:44:42,902][105620] Updated weights for policy 1, policy_version 1561504 (0.0009) [2023-12-27 02:44:43,045][105692] Updated weights for policy 0, policy_version 1558225 (0.0010) [2023-12-27 02:44:43,109][105692] Updated weights for policy 0, policy_version 1558235 (0.0011) [2023-12-27 02:44:43,181][105692] Updated weights for policy 0, policy_version 1558245 (0.0009) [2023-12-27 02:44:43,595][105620] Updated weights for policy 1, policy_version 1561514 (0.0008) [2023-12-27 02:44:43,665][105620] Updated weights for policy 1, policy_version 1561524 (0.0005) [2023-12-27 02:44:43,728][105620] Updated weights for policy 1, policy_version 1561534 (0.0010) [2023-12-27 02:44:43,788][105620] Updated weights for policy 1, policy_version 1561544 (0.0009) [2023-12-27 02:44:44,007][105692] Updated weights for policy 0, policy_version 1558255 (0.0010) [2023-12-27 02:44:44,057][105692] Updated weights for policy 0, policy_version 1558265 (0.0010) [2023-12-27 02:44:44,110][105692] Updated weights for policy 0, policy_version 1558275 (0.0009) [2023-12-27 02:44:44,287][105620] Updated weights for policy 1, policy_version 1561554 (0.0009) [2023-12-27 02:44:44,323][105586] KL-divergence is very high: 142.5462 [2023-12-27 02:44:44,335][105620] Updated weights for policy 1, policy_version 1561564 (0.0009) [2023-12-27 02:44:44,362][105586] KL-divergence is very high: 141.0162 [2023-12-27 02:44:44,388][105620] Updated weights for policy 1, policy_version 1561574 (0.0009) [2023-12-27 02:44:44,843][105692] Updated weights for policy 0, policy_version 1558285 (0.0010) [2023-12-27 02:44:44,900][105692] Updated weights for policy 0, policy_version 1558295 (0.0008) [2023-12-27 02:44:44,959][105692] Updated weights for policy 0, policy_version 1558305 (0.0007) [2023-12-27 02:44:45,136][105620] Updated weights for policy 1, policy_version 1561584 (0.0010) [2023-12-27 02:44:45,203][105620] Updated weights for policy 1, policy_version 1561594 (0.0011) [2023-12-27 02:44:45,270][105620] Updated weights for policy 1, policy_version 1561604 (0.0011) [2023-12-27 02:44:45,575][105692] Updated weights for policy 0, policy_version 1558315 (0.0006) [2023-12-27 02:44:45,634][105692] Updated weights for policy 0, policy_version 1558325 (0.0005) [2023-12-27 02:44:45,678][105692] Updated weights for policy 0, policy_version 1558335 (0.0008) [2023-12-27 02:44:46,013][105620] Updated weights for policy 1, policy_version 1561614 (0.0010) [2023-12-27 02:44:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 798818304. Throughput: 0: 9764.5, 1: 9780.2. Samples: 798790152. Policy #0 lag: (min: 12.0, avg: 13.9, max: 44.0) [2023-12-27 02:44:46,062][104569] Avg episode reward: [(0, '8437.148'), (1, '9265.005')] [2023-12-27 02:44:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001558344_398991360.pth... [2023-12-27 02:44:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001557160_398688256.pth [2023-12-27 02:44:46,084][105620] Updated weights for policy 1, policy_version 1561624 (0.0011) [2023-12-27 02:44:46,143][105620] Updated weights for policy 1, policy_version 1561634 (0.0010) [2023-12-27 02:44:46,177][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001561640_399835136.pth... [2023-12-27 02:44:46,182][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001560488_399540224.pth [2023-12-27 02:44:46,388][105692] Updated weights for policy 0, policy_version 1558345 (0.0008) [2023-12-27 02:44:46,450][105692] Updated weights for policy 0, policy_version 1558355 (0.0008) [2023-12-27 02:44:46,502][105692] Updated weights for policy 0, policy_version 1558365 (0.0008) [2023-12-27 02:44:46,555][105692] Updated weights for policy 0, policy_version 1558375 (0.0008) [2023-12-27 02:44:46,845][105620] Updated weights for policy 1, policy_version 1561644 (0.0010) [2023-12-27 02:44:46,900][105620] Updated weights for policy 1, policy_version 1561654 (0.0010) [2023-12-27 02:44:46,949][105620] Updated weights for policy 1, policy_version 1561664 (0.0010) [2023-12-27 02:44:47,328][105692] Updated weights for policy 0, policy_version 1558385 (0.0008) [2023-12-27 02:44:47,393][105692] Updated weights for policy 0, policy_version 1558395 (0.0008) [2023-12-27 02:44:47,448][105692] Updated weights for policy 0, policy_version 1558405 (0.0008) [2023-12-27 02:44:47,716][105620] Updated weights for policy 1, policy_version 1561674 (0.0010) [2023-12-27 02:44:47,764][105620] Updated weights for policy 1, policy_version 1561684 (0.0010) [2023-12-27 02:44:47,816][105620] Updated weights for policy 1, policy_version 1561694 (0.0010) [2023-12-27 02:44:47,871][105620] Updated weights for policy 1, policy_version 1561704 (0.0011) [2023-12-27 02:44:48,214][105692] Updated weights for policy 0, policy_version 1558415 (0.0008) [2023-12-27 02:44:48,268][105692] Updated weights for policy 0, policy_version 1558425 (0.0008) [2023-12-27 02:44:48,327][105692] Updated weights for policy 0, policy_version 1558435 (0.0007) [2023-12-27 02:44:48,643][105620] Updated weights for policy 1, policy_version 1561714 (0.0011) [2023-12-27 02:44:48,696][105620] Updated weights for policy 1, policy_version 1561724 (0.0011) [2023-12-27 02:44:48,749][105620] Updated weights for policy 1, policy_version 1561734 (0.0011) [2023-12-27 02:44:48,756][105586] KL-divergence is very high: 103.0328 [2023-12-27 02:44:49,025][105692] Updated weights for policy 0, policy_version 1558445 (0.0009) [2023-12-27 02:44:49,073][105692] Updated weights for policy 0, policy_version 1558455 (0.0010) [2023-12-27 02:44:49,127][105692] Updated weights for policy 0, policy_version 1558465 (0.0010) [2023-12-27 02:44:49,496][105620] Updated weights for policy 1, policy_version 1561744 (0.0011) [2023-12-27 02:44:49,564][105620] Updated weights for policy 1, policy_version 1561754 (0.0011) [2023-12-27 02:44:49,620][105620] Updated weights for policy 1, policy_version 1561764 (0.0011) [2023-12-27 02:44:49,870][105692] Updated weights for policy 0, policy_version 1558475 (0.0010) [2023-12-27 02:44:49,942][105692] Updated weights for policy 0, policy_version 1558485 (0.0009) [2023-12-27 02:44:50,003][105692] Updated weights for policy 0, policy_version 1558495 (0.0006) [2023-12-27 02:44:50,385][105620] Updated weights for policy 1, policy_version 1561774 (0.0011) [2023-12-27 02:44:50,451][105620] Updated weights for policy 1, policy_version 1561784 (0.0011) [2023-12-27 02:44:50,517][105620] Updated weights for policy 1, policy_version 1561794 (0.0011) [2023-12-27 02:44:50,615][105692] Updated weights for policy 0, policy_version 1558505 (0.0007) [2023-12-27 02:44:50,672][105692] Updated weights for policy 0, policy_version 1558515 (0.0007) [2023-12-27 02:44:50,743][105692] Updated weights for policy 0, policy_version 1558525 (0.0005) [2023-12-27 02:44:50,809][105692] Updated weights for policy 0, policy_version 1558535 (0.0007) [2023-12-27 02:44:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 798916608. Throughput: 0: 9817.2, 1: 9728.3. Samples: 798906152. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:44:51,063][104569] Avg episode reward: [(0, '8526.062'), (1, '9082.918')] [2023-12-27 02:44:51,290][105620] Updated weights for policy 1, policy_version 1561804 (0.0010) [2023-12-27 02:44:51,339][105620] Updated weights for policy 1, policy_version 1561814 (0.0008) [2023-12-27 02:44:51,408][105620] Updated weights for policy 1, policy_version 1561824 (0.0007) [2023-12-27 02:44:51,536][105692] Updated weights for policy 0, policy_version 1558545 (0.0009) [2023-12-27 02:44:51,596][105692] Updated weights for policy 0, policy_version 1558555 (0.0007) [2023-12-27 02:44:51,666][105692] Updated weights for policy 0, policy_version 1558565 (0.0008) [2023-12-27 02:44:52,049][105620] Updated weights for policy 1, policy_version 1561834 (0.0009) [2023-12-27 02:44:52,103][105620] Updated weights for policy 1, policy_version 1561844 (0.0009) [2023-12-27 02:44:52,157][105620] Updated weights for policy 1, policy_version 1561854 (0.0008) [2023-12-27 02:44:52,204][105620] Updated weights for policy 1, policy_version 1561864 (0.0009) [2023-12-27 02:44:52,395][105692] Updated weights for policy 0, policy_version 1558575 (0.0008) [2023-12-27 02:44:52,453][105692] Updated weights for policy 0, policy_version 1558585 (0.0009) [2023-12-27 02:44:52,514][105692] Updated weights for policy 0, policy_version 1558595 (0.0009) [2023-12-27 02:44:52,960][105620] Updated weights for policy 1, policy_version 1561874 (0.0009) [2023-12-27 02:44:53,010][105620] Updated weights for policy 1, policy_version 1561884 (0.0009) [2023-12-27 02:44:53,071][105620] Updated weights for policy 1, policy_version 1561894 (0.0010) [2023-12-27 02:44:53,260][105692] Updated weights for policy 0, policy_version 1558605 (0.0009) [2023-12-27 02:44:53,317][105692] Updated weights for policy 0, policy_version 1558615 (0.0010) [2023-12-27 02:44:53,370][105692] Updated weights for policy 0, policy_version 1558625 (0.0009) [2023-12-27 02:44:53,707][105620] Updated weights for policy 1, policy_version 1561904 (0.0011) [2023-12-27 02:44:53,765][105620] Updated weights for policy 1, policy_version 1561914 (0.0011) [2023-12-27 02:44:53,830][105620] Updated weights for policy 1, policy_version 1561924 (0.0010) [2023-12-27 02:44:54,221][105692] Updated weights for policy 0, policy_version 1558636 (0.0009) [2023-12-27 02:44:54,280][105692] Updated weights for policy 0, policy_version 1558646 (0.0008) [2023-12-27 02:44:54,347][105692] Updated weights for policy 0, policy_version 1558656 (0.0006) [2023-12-27 02:44:54,551][105620] Updated weights for policy 1, policy_version 1561934 (0.0008) [2023-12-27 02:44:54,613][105620] Updated weights for policy 1, policy_version 1561944 (0.0005) [2023-12-27 02:44:54,678][105620] Updated weights for policy 1, policy_version 1561954 (0.0007) [2023-12-27 02:44:54,883][105692] Updated weights for policy 0, policy_version 1558666 (0.0006) [2023-12-27 02:44:54,939][105692] Updated weights for policy 0, policy_version 1558676 (0.0005) [2023-12-27 02:44:55,005][105692] Updated weights for policy 0, policy_version 1558686 (0.0006) [2023-12-27 02:44:55,074][105692] Updated weights for policy 0, policy_version 1558696 (0.0006) [2023-12-27 02:44:55,280][105620] Updated weights for policy 1, policy_version 1561964 (0.0008) [2023-12-27 02:44:55,346][105620] Updated weights for policy 1, policy_version 1561974 (0.0005) [2023-12-27 02:44:55,413][105620] Updated weights for policy 1, policy_version 1561984 (0.0009) [2023-12-27 02:44:55,587][105692] Updated weights for policy 0, policy_version 1558706 (0.0006) [2023-12-27 02:44:55,639][105692] Updated weights for policy 0, policy_version 1558716 (0.0005) [2023-12-27 02:44:55,700][105692] Updated weights for policy 0, policy_version 1558726 (0.0005) [2023-12-27 02:44:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 799014912. Throughput: 0: 9915.4, 1: 9692.6. Samples: 799026612. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:44:56,062][104569] Avg episode reward: [(0, '8257.434'), (1, '8990.383')] [2023-12-27 02:44:56,086][105620] Updated weights for policy 1, policy_version 1561994 (0.0010) [2023-12-27 02:44:56,141][105620] Updated weights for policy 1, policy_version 1562004 (0.0011) [2023-12-27 02:44:56,196][105620] Updated weights for policy 1, policy_version 1562014 (0.0011) [2023-12-27 02:44:56,209][105692] Updated weights for policy 0, policy_version 1558736 (0.0005) [2023-12-27 02:44:56,254][105620] Updated weights for policy 1, policy_version 1562024 (0.0011) [2023-12-27 02:44:56,256][105692] Updated weights for policy 0, policy_version 1558746 (0.0005) [2023-12-27 02:44:56,313][105692] Updated weights for policy 0, policy_version 1558756 (0.0005) [2023-12-27 02:44:56,827][105620] Updated weights for policy 1, policy_version 1562034 (0.0005) [2023-12-27 02:44:56,880][105620] Updated weights for policy 1, policy_version 1562044 (0.0008) [2023-12-27 02:44:56,895][105692] Updated weights for policy 0, policy_version 1558766 (0.0008) [2023-12-27 02:44:56,938][105620] Updated weights for policy 1, policy_version 1562054 (0.0010) [2023-12-27 02:44:56,952][105692] Updated weights for policy 0, policy_version 1558776 (0.0010) [2023-12-27 02:44:57,013][105692] Updated weights for policy 0, policy_version 1558786 (0.0010) [2023-12-27 02:44:57,615][105620] Updated weights for policy 1, policy_version 1562064 (0.0010) [2023-12-27 02:44:57,640][105692] Updated weights for policy 0, policy_version 1558796 (0.0008) [2023-12-27 02:44:57,682][105620] Updated weights for policy 1, policy_version 1562074 (0.0010) [2023-12-27 02:44:57,702][105692] Updated weights for policy 0, policy_version 1558806 (0.0005) [2023-12-27 02:44:57,734][105620] Updated weights for policy 1, policy_version 1562084 (0.0010) [2023-12-27 02:44:57,748][105692] Updated weights for policy 0, policy_version 1558816 (0.0006) [2023-12-27 02:44:58,410][105692] Updated weights for policy 0, policy_version 1558826 (0.0008) [2023-12-27 02:44:58,448][105620] Updated weights for policy 1, policy_version 1562094 (0.0010) [2023-12-27 02:44:58,482][105692] Updated weights for policy 0, policy_version 1558836 (0.0010) [2023-12-27 02:44:58,514][105620] Updated weights for policy 1, policy_version 1562104 (0.0011) [2023-12-27 02:44:58,545][105692] Updated weights for policy 0, policy_version 1558846 (0.0008) [2023-12-27 02:44:58,578][105620] Updated weights for policy 1, policy_version 1562114 (0.0012) [2023-12-27 02:44:58,610][105692] Updated weights for policy 0, policy_version 1558856 (0.0008) [2023-12-27 02:44:59,386][105620] Updated weights for policy 1, policy_version 1562124 (0.0010) [2023-12-27 02:44:59,390][105692] Updated weights for policy 0, policy_version 1558866 (0.0010) [2023-12-27 02:44:59,444][105620] Updated weights for policy 1, policy_version 1562134 (0.0008) [2023-12-27 02:44:59,448][105692] Updated weights for policy 0, policy_version 1558876 (0.0005) [2023-12-27 02:44:59,501][105620] Updated weights for policy 1, policy_version 1562144 (0.0008) [2023-12-27 02:44:59,505][105692] Updated weights for policy 0, policy_version 1558886 (0.0005) [2023-12-27 02:45:00,166][105692] Updated weights for policy 0, policy_version 1558896 (0.0007) [2023-12-27 02:45:00,216][105692] Updated weights for policy 0, policy_version 1558906 (0.0008) [2023-12-27 02:45:00,241][105620] Updated weights for policy 1, policy_version 1562154 (0.0008) [2023-12-27 02:45:00,266][105692] Updated weights for policy 0, policy_version 1558916 (0.0006) [2023-12-27 02:45:00,301][105620] Updated weights for policy 1, policy_version 1562164 (0.0011) [2023-12-27 02:45:00,366][105620] Updated weights for policy 1, policy_version 1562174 (0.0009) [2023-12-27 02:45:00,424][105620] Updated weights for policy 1, policy_version 1562184 (0.0010) [2023-12-27 02:45:00,961][105692] Updated weights for policy 0, policy_version 1558926 (0.0006) [2023-12-27 02:45:01,025][105692] Updated weights for policy 0, policy_version 1558936 (0.0008) [2023-12-27 02:45:01,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 799113216. Throughput: 0: 10013.5, 1: 9748.0. Samples: 799090160. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:45:01,062][104569] Avg episode reward: [(0, '8628.558'), (1, '8993.855')] [2023-12-27 02:45:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001562184_399974400.pth... [2023-12-27 02:45:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001561064_399687680.pth [2023-12-27 02:45:01,084][105692] Updated weights for policy 0, policy_version 1558946 (0.0008) [2023-12-27 02:45:01,116][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001558952_399147008.pth... [2023-12-27 02:45:01,119][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001557768_398843904.pth [2023-12-27 02:45:01,155][105620] Updated weights for policy 1, policy_version 1562194 (0.0006) [2023-12-27 02:45:01,228][105620] Updated weights for policy 1, policy_version 1562204 (0.0008) [2023-12-27 02:45:01,283][105620] Updated weights for policy 1, policy_version 1562214 (0.0008) [2023-12-27 02:45:01,799][105692] Updated weights for policy 0, policy_version 1558956 (0.0006) [2023-12-27 02:45:01,855][105692] Updated weights for policy 0, policy_version 1558966 (0.0007) [2023-12-27 02:45:01,912][105692] Updated weights for policy 0, policy_version 1558976 (0.0008) [2023-12-27 02:45:01,962][105620] Updated weights for policy 1, policy_version 1562224 (0.0008) [2023-12-27 02:45:02,021][105620] Updated weights for policy 1, policy_version 1562234 (0.0008) [2023-12-27 02:45:02,078][105620] Updated weights for policy 1, policy_version 1562244 (0.0010) [2023-12-27 02:45:02,574][105692] Updated weights for policy 0, policy_version 1558986 (0.0006) [2023-12-27 02:45:02,633][105692] Updated weights for policy 0, policy_version 1558996 (0.0005) [2023-12-27 02:45:02,690][105692] Updated weights for policy 0, policy_version 1559006 (0.0006) [2023-12-27 02:45:02,746][105692] Updated weights for policy 0, policy_version 1559016 (0.0006) [2023-12-27 02:45:02,930][105620] Updated weights for policy 1, policy_version 1562254 (0.0009) [2023-12-27 02:45:02,989][105620] Updated weights for policy 1, policy_version 1562264 (0.0008) [2023-12-27 02:45:03,045][105620] Updated weights for policy 1, policy_version 1562274 (0.0008) [2023-12-27 02:45:03,346][105692] Updated weights for policy 0, policy_version 1559026 (0.0010) [2023-12-27 02:45:03,400][105692] Updated weights for policy 0, policy_version 1559036 (0.0010) [2023-12-27 02:45:03,464][105692] Updated weights for policy 0, policy_version 1559046 (0.0010) [2023-12-27 02:45:03,824][105620] Updated weights for policy 1, policy_version 1562284 (0.0009) [2023-12-27 02:45:03,886][105620] Updated weights for policy 1, policy_version 1562294 (0.0010) [2023-12-27 02:45:03,945][105620] Updated weights for policy 1, policy_version 1562304 (0.0010) [2023-12-27 02:45:04,149][105692] Updated weights for policy 0, policy_version 1559056 (0.0008) [2023-12-27 02:45:04,199][105692] Updated weights for policy 0, policy_version 1559066 (0.0008) [2023-12-27 02:45:04,251][105692] Updated weights for policy 0, policy_version 1559076 (0.0009) [2023-12-27 02:45:04,694][105620] Updated weights for policy 1, policy_version 1562314 (0.0009) [2023-12-27 02:45:04,755][105620] Updated weights for policy 1, policy_version 1562324 (0.0010) [2023-12-27 02:45:04,818][105620] Updated weights for policy 1, policy_version 1562334 (0.0011) [2023-12-27 02:45:04,876][105620] Updated weights for policy 1, policy_version 1562344 (0.0010) [2023-12-27 02:45:05,018][105692] Updated weights for policy 0, policy_version 1559086 (0.0009) [2023-12-27 02:45:05,084][105692] Updated weights for policy 0, policy_version 1559096 (0.0009) [2023-12-27 02:45:05,136][105692] Updated weights for policy 0, policy_version 1559106 (0.0010) [2023-12-27 02:45:05,542][105620] Updated weights for policy 1, policy_version 1562354 (0.0008) [2023-12-27 02:45:05,590][105620] Updated weights for policy 1, policy_version 1562364 (0.0007) [2023-12-27 02:45:05,643][105620] Updated weights for policy 1, policy_version 1562374 (0.0008) [2023-12-27 02:45:05,826][105692] Updated weights for policy 0, policy_version 1559116 (0.0011) [2023-12-27 02:45:05,888][105692] Updated weights for policy 0, policy_version 1559126 (0.0011) [2023-12-27 02:45:05,937][105692] Updated weights for policy 0, policy_version 1559136 (0.0010) [2023-12-27 02:45:06,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 799219712. Throughput: 0: 10074.8, 1: 9628.2. Samples: 799205448. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:45:06,063][104569] Avg episode reward: [(0, '8810.330'), (1, '8722.698')] [2023-12-27 02:45:06,428][105620] Updated weights for policy 1, policy_version 1562384 (0.0006) [2023-12-27 02:45:06,484][105620] Updated weights for policy 1, policy_version 1562394 (0.0005) [2023-12-27 02:45:06,542][105620] Updated weights for policy 1, policy_version 1562404 (0.0008) [2023-12-27 02:45:06,605][105692] Updated weights for policy 0, policy_version 1559146 (0.0009) [2023-12-27 02:45:06,664][105692] Updated weights for policy 0, policy_version 1559156 (0.0011) [2023-12-27 02:45:06,730][105692] Updated weights for policy 0, policy_version 1559166 (0.0011) [2023-12-27 02:45:06,800][105692] Updated weights for policy 0, policy_version 1559176 (0.0011) [2023-12-27 02:45:07,164][105620] Updated weights for policy 1, policy_version 1562414 (0.0007) [2023-12-27 02:45:07,228][105620] Updated weights for policy 1, policy_version 1562424 (0.0005) [2023-12-27 02:45:07,300][105620] Updated weights for policy 1, policy_version 1562434 (0.0005) [2023-12-27 02:45:07,465][105692] Updated weights for policy 0, policy_version 1559186 (0.0011) [2023-12-27 02:45:07,525][105692] Updated weights for policy 0, policy_version 1559196 (0.0011) [2023-12-27 02:45:07,576][105692] Updated weights for policy 0, policy_version 1559206 (0.0010) [2023-12-27 02:45:07,832][105620] Updated weights for policy 1, policy_version 1562444 (0.0007) [2023-12-27 02:45:07,892][105620] Updated weights for policy 1, policy_version 1562454 (0.0010) [2023-12-27 02:45:07,956][105620] Updated weights for policy 1, policy_version 1562464 (0.0007) [2023-12-27 02:45:08,272][105692] Updated weights for policy 0, policy_version 1559216 (0.0009) [2023-12-27 02:45:08,323][105692] Updated weights for policy 0, policy_version 1559226 (0.0010) [2023-12-27 02:45:08,388][105692] Updated weights for policy 0, policy_version 1559236 (0.0010) [2023-12-27 02:45:08,542][105620] Updated weights for policy 1, policy_version 1562474 (0.0005) [2023-12-27 02:45:08,600][105620] Updated weights for policy 1, policy_version 1562484 (0.0005) [2023-12-27 02:45:08,651][105620] Updated weights for policy 1, policy_version 1562494 (0.0005) [2023-12-27 02:45:08,714][105620] Updated weights for policy 1, policy_version 1562504 (0.0007) [2023-12-27 02:45:09,139][105692] Updated weights for policy 0, policy_version 1559246 (0.0011) [2023-12-27 02:45:09,205][105692] Updated weights for policy 0, policy_version 1559256 (0.0011) [2023-12-27 02:45:09,272][105692] Updated weights for policy 0, policy_version 1559266 (0.0010) [2023-12-27 02:45:09,491][105620] Updated weights for policy 1, policy_version 1562514 (0.0008) [2023-12-27 02:45:09,552][105620] Updated weights for policy 1, policy_version 1562524 (0.0008) [2023-12-27 02:45:09,587][105586] KL-divergence is very high: 110.4971 [2023-12-27 02:45:09,611][105620] Updated weights for policy 1, policy_version 1562534 (0.0008) [2023-12-27 02:45:10,037][105692] Updated weights for policy 0, policy_version 1559276 (0.0010) [2023-12-27 02:45:10,108][105692] Updated weights for policy 0, policy_version 1559286 (0.0011) [2023-12-27 02:45:10,179][105692] Updated weights for policy 0, policy_version 1559296 (0.0011) [2023-12-27 02:45:10,246][105620] Updated weights for policy 1, policy_version 1562544 (0.0006) [2023-12-27 02:45:10,290][105586] KL-divergence is very high: 102.0717 [2023-12-27 02:45:10,299][105620] Updated weights for policy 1, policy_version 1562554 (0.0005) [2023-12-27 02:45:10,328][105586] KL-divergence is very high: 104.2704 [2023-12-27 02:45:10,349][105620] Updated weights for policy 1, policy_version 1562564 (0.0007) [2023-12-27 02:45:10,868][105692] Updated weights for policy 0, policy_version 1559306 (0.0010) [2023-12-27 02:45:10,915][105620] Updated weights for policy 1, policy_version 1562574 (0.0008) [2023-12-27 02:45:10,924][105692] Updated weights for policy 0, policy_version 1559316 (0.0006) [2023-12-27 02:45:10,964][105620] Updated weights for policy 1, policy_version 1562584 (0.0009) [2023-12-27 02:45:10,979][105692] Updated weights for policy 0, policy_version 1559326 (0.0005) [2023-12-27 02:45:11,017][105620] Updated weights for policy 1, policy_version 1562594 (0.0009) [2023-12-27 02:45:11,043][105692] Updated weights for policy 0, policy_version 1559336 (0.0007) [2023-12-27 02:45:11,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 799326208. Throughput: 0: 10095.0, 1: 9782.5. Samples: 799327572. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:45:11,062][104569] Avg episode reward: [(0, '8900.871'), (1, '8908.952')] [2023-12-27 02:45:11,780][105620] Updated weights for policy 1, policy_version 1562604 (0.0007) [2023-12-27 02:45:11,822][105692] Updated weights for policy 0, policy_version 1559346 (0.0008) [2023-12-27 02:45:11,846][105620] Updated weights for policy 1, policy_version 1562614 (0.0006) [2023-12-27 02:45:11,887][105692] Updated weights for policy 0, policy_version 1559356 (0.0008) [2023-12-27 02:45:11,908][105620] Updated weights for policy 1, policy_version 1562624 (0.0006) [2023-12-27 02:45:11,948][105692] Updated weights for policy 0, policy_version 1559366 (0.0007) [2023-12-27 02:45:12,614][105692] Updated weights for policy 0, policy_version 1559376 (0.0006) [2023-12-27 02:45:12,651][105620] Updated weights for policy 1, policy_version 1562634 (0.0007) [2023-12-27 02:45:12,677][105692] Updated weights for policy 0, policy_version 1559386 (0.0008) [2023-12-27 02:45:12,709][105620] Updated weights for policy 1, policy_version 1562644 (0.0006) [2023-12-27 02:45:12,739][105692] Updated weights for policy 0, policy_version 1559396 (0.0008) [2023-12-27 02:45:12,766][105620] Updated weights for policy 1, policy_version 1562654 (0.0006) [2023-12-27 02:45:12,824][105620] Updated weights for policy 1, policy_version 1562664 (0.0006) [2023-12-27 02:45:13,465][105692] Updated weights for policy 0, policy_version 1559406 (0.0008) [2023-12-27 02:45:13,475][105620] Updated weights for policy 1, policy_version 1562674 (0.0007) [2023-12-27 02:45:13,526][105692] Updated weights for policy 0, policy_version 1559416 (0.0007) [2023-12-27 02:45:13,528][105620] Updated weights for policy 1, policy_version 1562684 (0.0006) [2023-12-27 02:45:13,583][105620] Updated weights for policy 1, policy_version 1562694 (0.0007) [2023-12-27 02:45:13,585][105692] Updated weights for policy 0, policy_version 1559426 (0.0009) [2023-12-27 02:45:14,220][105692] Updated weights for policy 0, policy_version 1559436 (0.0007) [2023-12-27 02:45:14,272][105692] Updated weights for policy 0, policy_version 1559447 (0.0010) [2023-12-27 02:45:14,311][105620] Updated weights for policy 1, policy_version 1562704 (0.0006) [2023-12-27 02:45:14,319][105692] Updated weights for policy 0, policy_version 1559457 (0.0009) [2023-12-27 02:45:14,369][105620] Updated weights for policy 1, policy_version 1562714 (0.0007) [2023-12-27 02:45:14,418][105620] Updated weights for policy 1, policy_version 1562724 (0.0010) [2023-12-27 02:45:15,068][105620] Updated weights for policy 1, policy_version 1562734 (0.0010) [2023-12-27 02:45:15,126][105692] Updated weights for policy 0, policy_version 1559467 (0.0008) [2023-12-27 02:45:15,131][105620] Updated weights for policy 1, policy_version 1562744 (0.0010) [2023-12-27 02:45:15,187][105692] Updated weights for policy 0, policy_version 1559477 (0.0011) [2023-12-27 02:45:15,196][105620] Updated weights for policy 1, policy_version 1562754 (0.0011) [2023-12-27 02:45:15,248][105692] Updated weights for policy 0, policy_version 1559487 (0.0010) [2023-12-27 02:45:15,872][105620] Updated weights for policy 1, policy_version 1562764 (0.0011) [2023-12-27 02:45:15,934][105620] Updated weights for policy 1, policy_version 1562774 (0.0010) [2023-12-27 02:45:15,956][105692] Updated weights for policy 0, policy_version 1559497 (0.0007) [2023-12-27 02:45:15,999][105620] Updated weights for policy 1, policy_version 1562784 (0.0008) [2023-12-27 02:45:16,003][105692] Updated weights for policy 0, policy_version 1559507 (0.0005) [2023-12-27 02:45:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 799416320. Throughput: 0: 10016.0, 1: 9719.4. Samples: 799384600. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:45:16,063][104569] Avg episode reward: [(0, '8715.032'), (1, '9001.435')] [2023-12-27 02:45:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001562792_400130048.pth... [2023-12-27 02:45:16,071][105692] Updated weights for policy 0, policy_version 1559517 (0.0006) [2023-12-27 02:45:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001561640_399835136.pth [2023-12-27 02:45:16,136][105692] Updated weights for policy 0, policy_version 1559527 (0.0009) [2023-12-27 02:45:16,143][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001559528_399294464.pth... [2023-12-27 02:45:16,151][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001558344_398991360.pth [2023-12-27 02:45:16,687][105620] Updated weights for policy 1, policy_version 1562794 (0.0007) [2023-12-27 02:45:16,741][105620] Updated weights for policy 1, policy_version 1562804 (0.0005) [2023-12-27 02:45:16,789][105620] Updated weights for policy 1, policy_version 1562814 (0.0005) [2023-12-27 02:45:16,804][105692] Updated weights for policy 0, policy_version 1559537 (0.0006) [2023-12-27 02:45:16,840][105620] Updated weights for policy 1, policy_version 1562824 (0.0005) [2023-12-27 02:45:16,862][105692] Updated weights for policy 0, policy_version 1559547 (0.0008) [2023-12-27 02:45:16,915][105692] Updated weights for policy 0, policy_version 1559557 (0.0006) [2023-12-27 02:45:17,412][105620] Updated weights for policy 1, policy_version 1562834 (0.0005) [2023-12-27 02:45:17,468][105620] Updated weights for policy 1, policy_version 1562844 (0.0006) [2023-12-27 02:45:17,473][105692] Updated weights for policy 0, policy_version 1559567 (0.0008) [2023-12-27 02:45:17,526][105692] Updated weights for policy 0, policy_version 1559577 (0.0009) [2023-12-27 02:45:17,532][105620] Updated weights for policy 1, policy_version 1562854 (0.0005) [2023-12-27 02:45:17,578][105692] Updated weights for policy 0, policy_version 1559587 (0.0010) [2023-12-27 02:45:18,130][105620] Updated weights for policy 1, policy_version 1562864 (0.0009) [2023-12-27 02:45:18,182][105620] Updated weights for policy 1, policy_version 1562874 (0.0010) [2023-12-27 02:45:18,244][105620] Updated weights for policy 1, policy_version 1562884 (0.0010) [2023-12-27 02:45:18,384][105692] Updated weights for policy 0, policy_version 1559597 (0.0009) [2023-12-27 02:45:18,451][105692] Updated weights for policy 0, policy_version 1559607 (0.0008) [2023-12-27 02:45:18,508][105692] Updated weights for policy 0, policy_version 1559617 (0.0008) [2023-12-27 02:45:19,049][105620] Updated weights for policy 1, policy_version 1562894 (0.0011) [2023-12-27 02:45:19,108][105620] Updated weights for policy 1, policy_version 1562904 (0.0011) [2023-12-27 02:45:19,159][105620] Updated weights for policy 1, policy_version 1562914 (0.0011) [2023-12-27 02:45:19,206][105692] Updated weights for policy 0, policy_version 1559627 (0.0009) [2023-12-27 02:45:19,270][105692] Updated weights for policy 0, policy_version 1559637 (0.0008) [2023-12-27 02:45:19,337][105692] Updated weights for policy 0, policy_version 1559647 (0.0008) [2023-12-27 02:45:19,969][105620] Updated weights for policy 1, policy_version 1562924 (0.0008) [2023-12-27 02:45:20,030][105620] Updated weights for policy 1, policy_version 1562934 (0.0007) [2023-12-27 02:45:20,091][105620] Updated weights for policy 1, policy_version 1562944 (0.0008) [2023-12-27 02:45:20,114][105692] Updated weights for policy 0, policy_version 1559657 (0.0009) [2023-12-27 02:45:20,173][105692] Updated weights for policy 0, policy_version 1559667 (0.0008) [2023-12-27 02:45:20,231][105692] Updated weights for policy 0, policy_version 1559677 (0.0008) [2023-12-27 02:45:20,290][105692] Updated weights for policy 0, policy_version 1559687 (0.0008) [2023-12-27 02:45:20,826][105620] Updated weights for policy 1, policy_version 1562954 (0.0009) [2023-12-27 02:45:20,879][105620] Updated weights for policy 1, policy_version 1562964 (0.0011) [2023-12-27 02:45:20,942][105620] Updated weights for policy 1, policy_version 1562974 (0.0011) [2023-12-27 02:45:21,016][105620] Updated weights for policy 1, policy_version 1562984 (0.0011) [2023-12-27 02:45:21,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 799514624. Throughput: 0: 9942.9, 1: 9794.7. Samples: 799505416. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:45:21,063][104569] Avg episode reward: [(0, '8443.904'), (1, '9356.870')] [2023-12-27 02:45:21,065][105692] Updated weights for policy 0, policy_version 1559697 (0.0007) [2023-12-27 02:45:21,130][105692] Updated weights for policy 0, policy_version 1559707 (0.0006) [2023-12-27 02:45:21,193][105692] Updated weights for policy 0, policy_version 1559717 (0.0008) [2023-12-27 02:45:21,857][105620] Updated weights for policy 1, policy_version 1562994 (0.0006) [2023-12-27 02:45:21,885][105692] Updated weights for policy 0, policy_version 1559727 (0.0008) [2023-12-27 02:45:21,921][105620] Updated weights for policy 1, policy_version 1563004 (0.0006) [2023-12-27 02:45:21,947][105692] Updated weights for policy 0, policy_version 1559737 (0.0007) [2023-12-27 02:45:21,986][105620] Updated weights for policy 1, policy_version 1563014 (0.0009) [2023-12-27 02:45:22,003][105692] Updated weights for policy 0, policy_version 1559747 (0.0007) [2023-12-27 02:45:22,677][105620] Updated weights for policy 1, policy_version 1563024 (0.0009) [2023-12-27 02:45:22,735][105620] Updated weights for policy 1, policy_version 1563034 (0.0009) [2023-12-27 02:45:22,759][105692] Updated weights for policy 0, policy_version 1559757 (0.0009) [2023-12-27 02:45:22,789][105620] Updated weights for policy 1, policy_version 1563044 (0.0005) [2023-12-27 02:45:22,813][105692] Updated weights for policy 0, policy_version 1559767 (0.0009) [2023-12-27 02:45:22,868][105692] Updated weights for policy 0, policy_version 1559777 (0.0009) [2023-12-27 02:45:23,521][105620] Updated weights for policy 1, policy_version 1563054 (0.0006) [2023-12-27 02:45:23,578][105620] Updated weights for policy 1, policy_version 1563064 (0.0005) [2023-12-27 02:45:23,631][105620] Updated weights for policy 1, policy_version 1563074 (0.0006) [2023-12-27 02:45:23,638][105692] Updated weights for policy 0, policy_version 1559787 (0.0008) [2023-12-27 02:45:23,701][105692] Updated weights for policy 0, policy_version 1559797 (0.0008) [2023-12-27 02:45:23,758][105692] Updated weights for policy 0, policy_version 1559807 (0.0009) [2023-12-27 02:45:24,197][105620] Updated weights for policy 1, policy_version 1563084 (0.0006) [2023-12-27 02:45:24,247][105620] Updated weights for policy 1, policy_version 1563094 (0.0005) [2023-12-27 02:45:24,312][105620] Updated weights for policy 1, policy_version 1563104 (0.0006) [2023-12-27 02:45:24,507][105692] Updated weights for policy 0, policy_version 1559817 (0.0010) [2023-12-27 02:45:24,562][105692] Updated weights for policy 0, policy_version 1559827 (0.0008) [2023-12-27 02:45:24,623][105692] Updated weights for policy 0, policy_version 1559837 (0.0007) [2023-12-27 02:45:24,680][105692] Updated weights for policy 0, policy_version 1559847 (0.0008) [2023-12-27 02:45:24,977][105620] Updated weights for policy 1, policy_version 1563114 (0.0008) [2023-12-27 02:45:25,034][105620] Updated weights for policy 1, policy_version 1563124 (0.0005) [2023-12-27 02:45:25,103][105620] Updated weights for policy 1, policy_version 1563134 (0.0006) [2023-12-27 02:45:25,159][105620] Updated weights for policy 1, policy_version 1563144 (0.0005) [2023-12-27 02:45:25,260][105692] Updated weights for policy 0, policy_version 1559857 (0.0007) [2023-12-27 02:45:25,322][105692] Updated weights for policy 0, policy_version 1559867 (0.0010) [2023-12-27 02:45:25,381][105692] Updated weights for policy 0, policy_version 1559877 (0.0010) [2023-12-27 02:45:25,743][105620] Updated weights for policy 1, policy_version 1563154 (0.0005) [2023-12-27 02:45:25,804][105620] Updated weights for policy 1, policy_version 1563164 (0.0005) [2023-12-27 02:45:25,862][105620] Updated weights for policy 1, policy_version 1563174 (0.0005) [2023-12-27 02:45:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 799612928. Throughput: 0: 9848.7, 1: 9944.2. Samples: 799623588. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:45:26,063][104569] Avg episode reward: [(0, '8536.308'), (1, '9266.574')] [2023-12-27 02:45:26,103][105692] Updated weights for policy 0, policy_version 1559887 (0.0010) [2023-12-27 02:45:26,155][105692] Updated weights for policy 0, policy_version 1559897 (0.0010) [2023-12-27 02:45:26,203][105692] Updated weights for policy 0, policy_version 1559907 (0.0010) [2023-12-27 02:45:26,453][105620] Updated weights for policy 1, policy_version 1563184 (0.0007) [2023-12-27 02:45:26,498][105620] Updated weights for policy 1, policy_version 1563194 (0.0008) [2023-12-27 02:45:26,550][105620] Updated weights for policy 1, policy_version 1563204 (0.0008) [2023-12-27 02:45:26,968][105692] Updated weights for policy 0, policy_version 1559917 (0.0010) [2023-12-27 02:45:27,016][105692] Updated weights for policy 0, policy_version 1559927 (0.0010) [2023-12-27 02:45:27,064][105692] Updated weights for policy 0, policy_version 1559937 (0.0010) [2023-12-27 02:45:27,164][105620] Updated weights for policy 1, policy_version 1563214 (0.0006) [2023-12-27 02:45:27,227][105620] Updated weights for policy 1, policy_version 1563224 (0.0005) [2023-12-27 02:45:27,275][105620] Updated weights for policy 1, policy_version 1563234 (0.0005) [2023-12-27 02:45:27,797][105620] Updated weights for policy 1, policy_version 1563244 (0.0006) [2023-12-27 02:45:27,819][105692] Updated weights for policy 0, policy_version 1559947 (0.0010) [2023-12-27 02:45:27,843][105620] Updated weights for policy 1, policy_version 1563254 (0.0005) [2023-12-27 02:45:27,863][105692] Updated weights for policy 0, policy_version 1559957 (0.0010) [2023-12-27 02:45:27,889][105620] Updated weights for policy 1, policy_version 1563264 (0.0005) [2023-12-27 02:45:27,914][105692] Updated weights for policy 0, policy_version 1559967 (0.0010) [2023-12-27 02:45:28,548][105620] Updated weights for policy 1, policy_version 1563274 (0.0006) [2023-12-27 02:45:28,599][105620] Updated weights for policy 1, policy_version 1563284 (0.0005) [2023-12-27 02:45:28,655][105620] Updated weights for policy 1, policy_version 1563294 (0.0005) [2023-12-27 02:45:28,684][105692] Updated weights for policy 0, policy_version 1559977 (0.0010) [2023-12-27 02:45:28,716][105620] Updated weights for policy 1, policy_version 1563304 (0.0005) [2023-12-27 02:45:28,746][105692] Updated weights for policy 0, policy_version 1559987 (0.0011) [2023-12-27 02:45:28,795][105692] Updated weights for policy 0, policy_version 1559997 (0.0010) [2023-12-27 02:45:28,843][105692] Updated weights for policy 0, policy_version 1560007 (0.0010) [2023-12-27 02:45:29,240][105620] Updated weights for policy 1, policy_version 1563314 (0.0008) [2023-12-27 02:45:29,301][105620] Updated weights for policy 1, policy_version 1563324 (0.0011) [2023-12-27 02:45:29,364][105620] Updated weights for policy 1, policy_version 1563334 (0.0011) [2023-12-27 02:45:29,589][105692] Updated weights for policy 0, policy_version 1560017 (0.0010) [2023-12-27 02:45:29,640][105692] Updated weights for policy 0, policy_version 1560027 (0.0010) [2023-12-27 02:45:29,689][105692] Updated weights for policy 0, policy_version 1560037 (0.0010) [2023-12-27 02:45:29,990][105620] Updated weights for policy 1, policy_version 1563344 (0.0009) [2023-12-27 02:45:30,051][105620] Updated weights for policy 1, policy_version 1563354 (0.0008) [2023-12-27 02:45:30,115][105620] Updated weights for policy 1, policy_version 1563364 (0.0005) [2023-12-27 02:45:30,460][105692] Updated weights for policy 0, policy_version 1560047 (0.0007) [2023-12-27 02:45:30,523][105692] Updated weights for policy 0, policy_version 1560057 (0.0005) [2023-12-27 02:45:30,578][105692] Updated weights for policy 0, policy_version 1560067 (0.0010) [2023-12-27 02:45:30,783][105620] Updated weights for policy 1, policy_version 1563374 (0.0007) [2023-12-27 02:45:30,838][105620] Updated weights for policy 1, policy_version 1563384 (0.0010) [2023-12-27 02:45:30,889][105620] Updated weights for policy 1, policy_version 1563394 (0.0010) [2023-12-27 02:45:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 799719424. Throughput: 0: 9876.5, 1: 10051.8. Samples: 799686924. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:45:31,062][104569] Avg episode reward: [(0, '8265.188'), (1, '9174.045')] [2023-12-27 02:45:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001560072_399433728.pth... [2023-12-27 02:45:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001563400_400285696.pth... [2023-12-27 02:45:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001558952_399147008.pth [2023-12-27 02:45:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001562184_399974400.pth [2023-12-27 02:45:31,332][105692] Updated weights for policy 0, policy_version 1560077 (0.0010) [2023-12-27 02:45:31,387][105692] Updated weights for policy 0, policy_version 1560087 (0.0007) [2023-12-27 02:45:31,452][105692] Updated weights for policy 0, policy_version 1560097 (0.0007) [2023-12-27 02:45:31,555][105620] Updated weights for policy 1, policy_version 1563404 (0.0010) [2023-12-27 02:45:31,618][105620] Updated weights for policy 1, policy_version 1563414 (0.0011) [2023-12-27 02:45:31,677][105620] Updated weights for policy 1, policy_version 1563424 (0.0010) [2023-12-27 02:45:32,145][105692] Updated weights for policy 0, policy_version 1560107 (0.0007) [2023-12-27 02:45:32,197][105692] Updated weights for policy 0, policy_version 1560117 (0.0010) [2023-12-27 02:45:32,245][105692] Updated weights for policy 0, policy_version 1560127 (0.0010) [2023-12-27 02:45:32,441][105620] Updated weights for policy 1, policy_version 1563434 (0.0011) [2023-12-27 02:45:32,489][105620] Updated weights for policy 1, policy_version 1563444 (0.0010) [2023-12-27 02:45:32,544][105620] Updated weights for policy 1, policy_version 1563454 (0.0010) [2023-12-27 02:45:32,603][105620] Updated weights for policy 1, policy_version 1563464 (0.0010) [2023-12-27 02:45:33,050][105692] Updated weights for policy 0, policy_version 1560137 (0.0010) [2023-12-27 02:45:33,098][105692] Updated weights for policy 0, policy_version 1560147 (0.0007) [2023-12-27 02:45:33,155][105692] Updated weights for policy 0, policy_version 1560157 (0.0005) [2023-12-27 02:45:33,211][105692] Updated weights for policy 0, policy_version 1560167 (0.0009) [2023-12-27 02:45:33,228][105620] Updated weights for policy 1, policy_version 1563474 (0.0007) [2023-12-27 02:45:33,279][105620] Updated weights for policy 1, policy_version 1563484 (0.0005) [2023-12-27 02:45:33,333][105620] Updated weights for policy 1, policy_version 1563494 (0.0005) [2023-12-27 02:45:33,864][105692] Updated weights for policy 0, policy_version 1560178 (0.0009) [2023-12-27 02:45:33,913][105692] Updated weights for policy 0, policy_version 1560189 (0.0009) [2023-12-27 02:45:33,917][105620] Updated weights for policy 1, policy_version 1563504 (0.0005) [2023-12-27 02:45:33,963][105692] Updated weights for policy 0, policy_version 1560199 (0.0011) [2023-12-27 02:45:33,974][105620] Updated weights for policy 1, policy_version 1563514 (0.0006) [2023-12-27 02:45:34,027][105620] Updated weights for policy 1, policy_version 1563524 (0.0005) [2023-12-27 02:45:34,627][105620] Updated weights for policy 1, policy_version 1563534 (0.0006) [2023-12-27 02:45:34,687][105620] Updated weights for policy 1, policy_version 1563544 (0.0008) [2023-12-27 02:45:34,693][105692] Updated weights for policy 0, policy_version 1560209 (0.0006) [2023-12-27 02:45:34,748][105620] Updated weights for policy 1, policy_version 1563554 (0.0007) [2023-12-27 02:45:34,750][105692] Updated weights for policy 0, policy_version 1560219 (0.0006) [2023-12-27 02:45:34,809][105692] Updated weights for policy 0, policy_version 1560229 (0.0007) [2023-12-27 02:45:35,479][105620] Updated weights for policy 1, policy_version 1563564 (0.0007) [2023-12-27 02:45:35,543][105620] Updated weights for policy 1, policy_version 1563574 (0.0008) [2023-12-27 02:45:35,597][105692] Updated weights for policy 0, policy_version 1560239 (0.0009) [2023-12-27 02:45:35,603][105620] Updated weights for policy 1, policy_version 1563584 (0.0006) [2023-12-27 02:45:35,659][105692] Updated weights for policy 0, policy_version 1560249 (0.0008) [2023-12-27 02:45:35,705][105692] Updated weights for policy 0, policy_version 1560259 (0.0008) [2023-12-27 02:45:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 799817728. Throughput: 0: 9868.5, 1: 10167.2. Samples: 799807756. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:45:36,062][104569] Avg episode reward: [(0, '8172.846'), (1, '9264.376')] [2023-12-27 02:45:36,324][105620] Updated weights for policy 1, policy_version 1563594 (0.0009) [2023-12-27 02:45:36,383][105620] Updated weights for policy 1, policy_version 1563604 (0.0008) [2023-12-27 02:45:36,438][105620] Updated weights for policy 1, policy_version 1563614 (0.0008) [2023-12-27 02:45:36,440][105692] Updated weights for policy 0, policy_version 1560269 (0.0008) [2023-12-27 02:45:36,500][105692] Updated weights for policy 0, policy_version 1560279 (0.0006) [2023-12-27 02:45:36,505][105620] Updated weights for policy 1, policy_version 1563624 (0.0010) [2023-12-27 02:45:36,560][105692] Updated weights for policy 0, policy_version 1560289 (0.0009) [2023-12-27 02:45:37,287][105620] Updated weights for policy 1, policy_version 1563634 (0.0006) [2023-12-27 02:45:37,348][105620] Updated weights for policy 1, policy_version 1563644 (0.0009) [2023-12-27 02:45:37,367][105692] Updated weights for policy 0, policy_version 1560299 (0.0009) [2023-12-27 02:45:37,408][105620] Updated weights for policy 1, policy_version 1563654 (0.0006) [2023-12-27 02:45:37,415][105692] Updated weights for policy 0, policy_version 1560309 (0.0005) [2023-12-27 02:45:37,473][105692] Updated weights for policy 0, policy_version 1560319 (0.0008) [2023-12-27 02:45:37,999][105620] Updated weights for policy 1, policy_version 1563664 (0.0008) [2023-12-27 02:45:38,056][105620] Updated weights for policy 1, policy_version 1563674 (0.0008) [2023-12-27 02:45:38,105][105620] Updated weights for policy 1, policy_version 1563684 (0.0008) [2023-12-27 02:45:38,189][105692] Updated weights for policy 0, policy_version 1560329 (0.0011) [2023-12-27 02:45:38,253][105692] Updated weights for policy 0, policy_version 1560339 (0.0010) [2023-12-27 02:45:38,312][105692] Updated weights for policy 0, policy_version 1560349 (0.0010) [2023-12-27 02:45:38,377][105692] Updated weights for policy 0, policy_version 1560359 (0.0010) [2023-12-27 02:45:38,943][105620] Updated weights for policy 1, policy_version 1563694 (0.0009) [2023-12-27 02:45:38,962][105692] Updated weights for policy 0, policy_version 1560369 (0.0006) [2023-12-27 02:45:39,001][105620] Updated weights for policy 1, policy_version 1563704 (0.0009) [2023-12-27 02:45:39,017][105692] Updated weights for policy 0, policy_version 1560379 (0.0005) [2023-12-27 02:45:39,053][105620] Updated weights for policy 1, policy_version 1563714 (0.0009) [2023-12-27 02:45:39,067][105692] Updated weights for policy 0, policy_version 1560389 (0.0005) [2023-12-27 02:45:39,733][105692] Updated weights for policy 0, policy_version 1560399 (0.0009) [2023-12-27 02:45:39,793][105692] Updated weights for policy 0, policy_version 1560409 (0.0011) [2023-12-27 02:45:39,853][105692] Updated weights for policy 0, policy_version 1560419 (0.0011) [2023-12-27 02:45:39,938][105620] Updated weights for policy 1, policy_version 1563724 (0.0008) [2023-12-27 02:45:39,999][105620] Updated weights for policy 1, policy_version 1563734 (0.0006) [2023-12-27 02:45:40,058][105620] Updated weights for policy 1, policy_version 1563744 (0.0005) [2023-12-27 02:45:40,619][105692] Updated weights for policy 0, policy_version 1560429 (0.0011) [2023-12-27 02:45:40,678][105692] Updated weights for policy 0, policy_version 1560439 (0.0011) [2023-12-27 02:45:40,681][105620] Updated weights for policy 1, policy_version 1563754 (0.0006) [2023-12-27 02:45:40,734][105692] Updated weights for policy 0, policy_version 1560449 (0.0010) [2023-12-27 02:45:40,741][105620] Updated weights for policy 1, policy_version 1563764 (0.0005) [2023-12-27 02:45:40,795][105620] Updated weights for policy 1, policy_version 1563774 (0.0005) [2023-12-27 02:45:40,846][105620] Updated weights for policy 1, policy_version 1563784 (0.0007) [2023-12-27 02:45:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19933.8, 300 sec: 19660.8). Total num frames: 799916032. Throughput: 0: 9813.2, 1: 10106.2. Samples: 799922988. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:45:41,063][104569] Avg episode reward: [(0, '8533.376'), (1, '9356.774')] [2023-12-27 02:45:41,525][105692] Updated weights for policy 0, policy_version 1560459 (0.0008) [2023-12-27 02:45:41,551][105620] Updated weights for policy 1, policy_version 1563794 (0.0006) [2023-12-27 02:45:41,585][105692] Updated weights for policy 0, policy_version 1560469 (0.0008) [2023-12-27 02:45:41,614][105620] Updated weights for policy 1, policy_version 1563804 (0.0007) [2023-12-27 02:45:41,646][105692] Updated weights for policy 0, policy_version 1560479 (0.0008) [2023-12-27 02:45:41,682][105620] Updated weights for policy 1, policy_version 1563814 (0.0006) [2023-12-27 02:45:42,256][105692] Updated weights for policy 0, policy_version 1560489 (0.0007) [2023-12-27 02:45:42,314][105692] Updated weights for policy 0, policy_version 1560499 (0.0008) [2023-12-27 02:45:42,384][105692] Updated weights for policy 0, policy_version 1560509 (0.0008) [2023-12-27 02:45:42,443][105620] Updated weights for policy 1, policy_version 1563824 (0.0007) [2023-12-27 02:45:42,449][105692] Updated weights for policy 0, policy_version 1560519 (0.0006) [2023-12-27 02:45:42,502][105620] Updated weights for policy 1, policy_version 1563834 (0.0008) [2023-12-27 02:45:42,569][105620] Updated weights for policy 1, policy_version 1563844 (0.0009) [2023-12-27 02:45:43,122][105692] Updated weights for policy 0, policy_version 1560529 (0.0010) [2023-12-27 02:45:43,177][105692] Updated weights for policy 0, policy_version 1560539 (0.0011) [2023-12-27 02:45:43,226][105692] Updated weights for policy 0, policy_version 1560549 (0.0010) [2023-12-27 02:45:43,302][105620] Updated weights for policy 1, policy_version 1563854 (0.0009) [2023-12-27 02:45:43,353][105620] Updated weights for policy 1, policy_version 1563864 (0.0008) [2023-12-27 02:45:43,408][105620] Updated weights for policy 1, policy_version 1563874 (0.0008) [2023-12-27 02:45:43,975][105692] Updated weights for policy 0, policy_version 1560559 (0.0008) [2023-12-27 02:45:44,036][105692] Updated weights for policy 0, policy_version 1560569 (0.0005) [2023-12-27 02:45:44,103][105692] Updated weights for policy 0, policy_version 1560579 (0.0006) [2023-12-27 02:45:44,185][105620] Updated weights for policy 1, policy_version 1563884 (0.0008) [2023-12-27 02:45:44,232][105620] Updated weights for policy 1, policy_version 1563894 (0.0008) [2023-12-27 02:45:44,276][105620] Updated weights for policy 1, policy_version 1563904 (0.0008) [2023-12-27 02:45:44,792][105692] Updated weights for policy 0, policy_version 1560589 (0.0011) [2023-12-27 02:45:44,852][105692] Updated weights for policy 0, policy_version 1560599 (0.0011) [2023-12-27 02:45:44,908][105692] Updated weights for policy 0, policy_version 1560609 (0.0011) [2023-12-27 02:45:45,076][105620] Updated weights for policy 1, policy_version 1563914 (0.0008) [2023-12-27 02:45:45,136][105620] Updated weights for policy 1, policy_version 1563924 (0.0009) [2023-12-27 02:45:45,200][105620] Updated weights for policy 1, policy_version 1563934 (0.0008) [2023-12-27 02:45:45,267][105620] Updated weights for policy 1, policy_version 1563944 (0.0008) [2023-12-27 02:45:45,558][105692] Updated weights for policy 0, policy_version 1560619 (0.0009) [2023-12-27 02:45:45,618][105692] Updated weights for policy 0, policy_version 1560629 (0.0006) [2023-12-27 02:45:45,673][105692] Updated weights for policy 0, policy_version 1560639 (0.0006) [2023-12-27 02:45:46,059][105620] Updated weights for policy 1, policy_version 1563954 (0.0009) [2023-12-27 02:45:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 800006144. Throughput: 0: 9720.2, 1: 10057.2. Samples: 799980148. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:45:46,063][104569] Avg episode reward: [(0, '8806.760'), (1, '9266.997')] [2023-12-27 02:45:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001560648_399581184.pth... [2023-12-27 02:45:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001559528_399294464.pth [2023-12-27 02:45:46,121][105620] Updated weights for policy 1, policy_version 1563964 (0.0010) [2023-12-27 02:45:46,191][105620] Updated weights for policy 1, policy_version 1563974 (0.0010) [2023-12-27 02:45:46,203][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001563976_400433152.pth... [2023-12-27 02:45:46,208][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001562792_400130048.pth [2023-12-27 02:45:46,270][105692] Updated weights for policy 0, policy_version 1560649 (0.0009) [2023-12-27 02:45:46,329][105692] Updated weights for policy 0, policy_version 1560659 (0.0007) [2023-12-27 02:45:46,397][105692] Updated weights for policy 0, policy_version 1560669 (0.0011) [2023-12-27 02:45:46,465][105692] Updated weights for policy 0, policy_version 1560679 (0.0008) [2023-12-27 02:45:46,914][105620] Updated weights for policy 1, policy_version 1563984 (0.0006) [2023-12-27 02:45:46,975][105620] Updated weights for policy 1, policy_version 1563994 (0.0005) [2023-12-27 02:45:47,035][105620] Updated weights for policy 1, policy_version 1564004 (0.0008) [2023-12-27 02:45:47,169][105692] Updated weights for policy 0, policy_version 1560689 (0.0009) [2023-12-27 02:45:47,225][105692] Updated weights for policy 0, policy_version 1560699 (0.0009) [2023-12-27 02:45:47,275][105692] Updated weights for policy 0, policy_version 1560709 (0.0010) [2023-12-27 02:45:47,668][105620] Updated weights for policy 1, policy_version 1564014 (0.0009) [2023-12-27 02:45:47,724][105620] Updated weights for policy 1, policy_version 1564024 (0.0006) [2023-12-27 02:45:47,780][105620] Updated weights for policy 1, policy_version 1564034 (0.0005) [2023-12-27 02:45:48,101][105692] Updated weights for policy 0, policy_version 1560719 (0.0009) [2023-12-27 02:45:48,164][105692] Updated weights for policy 0, policy_version 1560730 (0.0012) [2023-12-27 02:45:48,218][105692] Updated weights for policy 0, policy_version 1560741 (0.0009) [2023-12-27 02:45:48,365][105620] Updated weights for policy 1, policy_version 1564044 (0.0006) [2023-12-27 02:45:48,431][105620] Updated weights for policy 1, policy_version 1564054 (0.0008) [2023-12-27 02:45:48,494][105620] Updated weights for policy 1, policy_version 1564064 (0.0008) [2023-12-27 02:45:49,056][105692] Updated weights for policy 0, policy_version 1560751 (0.0008) [2023-12-27 02:45:49,104][105692] Updated weights for policy 0, policy_version 1560761 (0.0009) [2023-12-27 02:45:49,154][105692] Updated weights for policy 0, policy_version 1560771 (0.0009) [2023-12-27 02:45:49,164][105620] Updated weights for policy 1, policy_version 1564074 (0.0007) [2023-12-27 02:45:49,218][105620] Updated weights for policy 1, policy_version 1564084 (0.0006) [2023-12-27 02:45:49,279][105620] Updated weights for policy 1, policy_version 1564094 (0.0007) [2023-12-27 02:45:49,339][105620] Updated weights for policy 1, policy_version 1564104 (0.0009) [2023-12-27 02:45:49,951][105692] Updated weights for policy 0, policy_version 1560781 (0.0009) [2023-12-27 02:45:50,001][105692] Updated weights for policy 0, policy_version 1560791 (0.0009) [2023-12-27 02:45:50,052][105692] Updated weights for policy 0, policy_version 1560801 (0.0008) [2023-12-27 02:45:50,096][105620] Updated weights for policy 1, policy_version 1564114 (0.0008) [2023-12-27 02:45:50,160][105620] Updated weights for policy 1, policy_version 1564124 (0.0007) [2023-12-27 02:45:50,218][105620] Updated weights for policy 1, policy_version 1564134 (0.0009) [2023-12-27 02:45:50,817][105692] Updated weights for policy 0, policy_version 1560811 (0.0009) [2023-12-27 02:45:50,880][105692] Updated weights for policy 0, policy_version 1560821 (0.0010) [2023-12-27 02:45:50,928][105692] Updated weights for policy 0, policy_version 1560831 (0.0007) [2023-12-27 02:45:50,946][105620] Updated weights for policy 1, policy_version 1564144 (0.0009) [2023-12-27 02:45:50,972][105586] KL-divergence is very high: 104.6514 [2023-12-27 02:45:51,012][105586] KL-divergence is very high: 120.1889 [2023-12-27 02:45:51,013][105620] Updated weights for policy 1, policy_version 1564154 (0.0008) [2023-12-27 02:45:51,026][105586] KL-divergence is very high: 183.5629 [2023-12-27 02:45:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 800104448. Throughput: 0: 9668.3, 1: 10138.4. Samples: 800096748. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:45:51,062][104569] Avg episode reward: [(0, '8715.590'), (1, '8812.956')] [2023-12-27 02:45:51,065][105586] KL-divergence is very high: 125.9984 [2023-12-27 02:45:51,078][105586] KL-divergence is very high: 183.0028 [2023-12-27 02:45:51,078][105620] Updated weights for policy 1, policy_version 1564164 (0.0009) [2023-12-27 02:45:51,710][105692] Updated weights for policy 0, policy_version 1560841 (0.0007) [2023-12-27 02:45:51,781][105692] Updated weights for policy 0, policy_version 1560851 (0.0008) [2023-12-27 02:45:51,849][105692] Updated weights for policy 0, policy_version 1560861 (0.0008) [2023-12-27 02:45:51,883][105620] Updated weights for policy 1, policy_version 1564174 (0.0008) [2023-12-27 02:45:51,913][105692] Updated weights for policy 0, policy_version 1560871 (0.0007) [2023-12-27 02:45:51,943][105620] Updated weights for policy 1, policy_version 1564184 (0.0008) [2023-12-27 02:45:52,002][105620] Updated weights for policy 1, policy_version 1564194 (0.0008) [2023-12-27 02:45:52,585][105692] Updated weights for policy 0, policy_version 1560881 (0.0010) [2023-12-27 02:45:52,653][105692] Updated weights for policy 0, policy_version 1560891 (0.0010) [2023-12-27 02:45:52,712][105692] Updated weights for policy 0, policy_version 1560901 (0.0008) [2023-12-27 02:45:52,713][105620] Updated weights for policy 1, policy_version 1564204 (0.0009) [2023-12-27 02:45:52,762][105620] Updated weights for policy 1, policy_version 1564214 (0.0008) [2023-12-27 02:45:52,816][105620] Updated weights for policy 1, policy_version 1564225 (0.0010) [2023-12-27 02:45:53,338][105692] Updated weights for policy 0, policy_version 1560911 (0.0006) [2023-12-27 02:45:53,396][105692] Updated weights for policy 0, policy_version 1560921 (0.0009) [2023-12-27 02:45:53,455][105692] Updated weights for policy 0, policy_version 1560931 (0.0009) [2023-12-27 02:45:53,658][105620] Updated weights for policy 1, policy_version 1564235 (0.0009) [2023-12-27 02:45:53,721][105620] Updated weights for policy 1, policy_version 1564245 (0.0009) [2023-12-27 02:45:53,776][105620] Updated weights for policy 1, policy_version 1564255 (0.0009) [2023-12-27 02:45:54,200][105692] Updated weights for policy 0, policy_version 1560941 (0.0008) [2023-12-27 02:45:54,253][105692] Updated weights for policy 0, policy_version 1560951 (0.0008) [2023-12-27 02:45:54,307][105692] Updated weights for policy 0, policy_version 1560961 (0.0010) [2023-12-27 02:45:54,456][105620] Updated weights for policy 1, policy_version 1564265 (0.0008) [2023-12-27 02:45:54,517][105620] Updated weights for policy 1, policy_version 1564275 (0.0005) [2023-12-27 02:45:54,581][105620] Updated weights for policy 1, policy_version 1564285 (0.0005) [2023-12-27 02:45:54,639][105620] Updated weights for policy 1, policy_version 1564295 (0.0008) [2023-12-27 02:45:55,078][105692] Updated weights for policy 0, policy_version 1560971 (0.0010) [2023-12-27 02:45:55,136][105692] Updated weights for policy 0, policy_version 1560981 (0.0010) [2023-12-27 02:45:55,188][105692] Updated weights for policy 0, policy_version 1560991 (0.0010) [2023-12-27 02:45:55,332][105620] Updated weights for policy 1, policy_version 1564305 (0.0008) [2023-12-27 02:45:55,390][105620] Updated weights for policy 1, policy_version 1564315 (0.0008) [2023-12-27 02:45:55,446][105620] Updated weights for policy 1, policy_version 1564325 (0.0008) [2023-12-27 02:45:55,937][105692] Updated weights for policy 0, policy_version 1561001 (0.0010) [2023-12-27 02:45:55,991][105692] Updated weights for policy 0, policy_version 1561011 (0.0010) [2023-12-27 02:45:56,042][105692] Updated weights for policy 0, policy_version 1561021 (0.0010) [2023-12-27 02:45:56,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 800194560. Throughput: 0: 9645.7, 1: 9951.6. Samples: 800209452. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:45:56,063][104569] Avg episode reward: [(0, '8806.871'), (1, '8721.746')] [2023-12-27 02:45:56,097][105692] Updated weights for policy 0, policy_version 1561031 (0.0010) [2023-12-27 02:45:56,208][105620] Updated weights for policy 1, policy_version 1564335 (0.0008) [2023-12-27 02:45:56,260][105620] Updated weights for policy 1, policy_version 1564345 (0.0008) [2023-12-27 02:45:56,313][105620] Updated weights for policy 1, policy_version 1564355 (0.0008) [2023-12-27 02:45:56,842][105692] Updated weights for policy 0, policy_version 1561041 (0.0010) [2023-12-27 02:45:56,892][105692] Updated weights for policy 0, policy_version 1561051 (0.0010) [2023-12-27 02:45:56,943][105692] Updated weights for policy 0, policy_version 1561061 (0.0010) [2023-12-27 02:45:57,101][105620] Updated weights for policy 1, policy_version 1564365 (0.0008) [2023-12-27 02:45:57,160][105620] Updated weights for policy 1, policy_version 1564375 (0.0008) [2023-12-27 02:45:57,212][105620] Updated weights for policy 1, policy_version 1564385 (0.0008) [2023-12-27 02:45:57,679][105692] Updated weights for policy 0, policy_version 1561071 (0.0007) [2023-12-27 02:45:57,741][105692] Updated weights for policy 0, policy_version 1561081 (0.0009) [2023-12-27 02:45:57,802][105692] Updated weights for policy 0, policy_version 1561091 (0.0010) [2023-12-27 02:45:57,991][105620] Updated weights for policy 1, policy_version 1564396 (0.0009) [2023-12-27 02:45:58,045][105620] Updated weights for policy 1, policy_version 1564406 (0.0010) [2023-12-27 02:45:58,100][105620] Updated weights for policy 1, policy_version 1564416 (0.0008) [2023-12-27 02:45:58,447][105692] Updated weights for policy 0, policy_version 1561101 (0.0010) [2023-12-27 02:45:58,510][105692] Updated weights for policy 0, policy_version 1561111 (0.0010) [2023-12-27 02:45:58,578][105692] Updated weights for policy 0, policy_version 1561121 (0.0010) [2023-12-27 02:45:58,919][105620] Updated weights for policy 1, policy_version 1564426 (0.0009) [2023-12-27 02:45:58,988][105620] Updated weights for policy 1, policy_version 1564436 (0.0008) [2023-12-27 02:45:59,049][105620] Updated weights for policy 1, policy_version 1564446 (0.0009) [2023-12-27 02:45:59,112][105620] Updated weights for policy 1, policy_version 1564456 (0.0008) [2023-12-27 02:45:59,517][105692] Updated weights for policy 0, policy_version 1561131 (0.0010) [2023-12-27 02:45:59,579][105692] Updated weights for policy 0, policy_version 1561141 (0.0008) [2023-12-27 02:45:59,638][105692] Updated weights for policy 0, policy_version 1561151 (0.0009) [2023-12-27 02:45:59,778][105620] Updated weights for policy 1, policy_version 1564466 (0.0009) [2023-12-27 02:45:59,843][105620] Updated weights for policy 1, policy_version 1564476 (0.0009) [2023-12-27 02:45:59,908][105620] Updated weights for policy 1, policy_version 1564486 (0.0009) [2023-12-27 02:46:00,404][105692] Updated weights for policy 0, policy_version 1561161 (0.0008) [2023-12-27 02:46:00,457][105692] Updated weights for policy 0, policy_version 1561171 (0.0010) [2023-12-27 02:46:00,509][105692] Updated weights for policy 0, policy_version 1561181 (0.0009) [2023-12-27 02:46:00,552][105620] Updated weights for policy 1, policy_version 1564496 (0.0007) [2023-12-27 02:46:00,560][105692] Updated weights for policy 0, policy_version 1561191 (0.0009) [2023-12-27 02:46:00,602][105620] Updated weights for policy 1, policy_version 1564506 (0.0006) [2023-12-27 02:46:00,656][105620] Updated weights for policy 1, policy_version 1564516 (0.0005) [2023-12-27 02:46:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 800292864. Throughput: 0: 9654.1, 1: 9922.0. Samples: 800265528. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:46:01,063][104569] Avg episode reward: [(0, '8624.503'), (1, '9085.903')] [2023-12-27 02:46:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001561192_399720448.pth... [2023-12-27 02:46:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001564520_400572416.pth... [2023-12-27 02:46:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001560072_399433728.pth [2023-12-27 02:46:01,092][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001563400_400285696.pth [2023-12-27 02:46:01,268][105692] Updated weights for policy 0, policy_version 1561201 (0.0010) [2023-12-27 02:46:01,320][105692] Updated weights for policy 0, policy_version 1561211 (0.0006) [2023-12-27 02:46:01,320][105620] Updated weights for policy 1, policy_version 1564526 (0.0008) [2023-12-27 02:46:01,388][105692] Updated weights for policy 0, policy_version 1561221 (0.0009) [2023-12-27 02:46:01,389][105620] Updated weights for policy 1, policy_version 1564536 (0.0011) [2023-12-27 02:46:01,449][105620] Updated weights for policy 1, policy_version 1564546 (0.0011) [2023-12-27 02:46:02,054][105692] Updated weights for policy 0, policy_version 1561231 (0.0008) [2023-12-27 02:46:02,110][105692] Updated weights for policy 0, policy_version 1561241 (0.0008) [2023-12-27 02:46:02,162][105692] Updated weights for policy 0, policy_version 1561251 (0.0008) [2023-12-27 02:46:02,209][105620] Updated weights for policy 1, policy_version 1564556 (0.0010) [2023-12-27 02:46:02,267][105620] Updated weights for policy 1, policy_version 1564566 (0.0010) [2023-12-27 02:46:02,326][105620] Updated weights for policy 1, policy_version 1564576 (0.0011) [2023-12-27 02:46:02,929][105692] Updated weights for policy 0, policy_version 1561261 (0.0007) [2023-12-27 02:46:02,982][105692] Updated weights for policy 0, policy_version 1561271 (0.0008) [2023-12-27 02:46:03,050][105692] Updated weights for policy 0, policy_version 1561281 (0.0008) [2023-12-27 02:46:03,106][105620] Updated weights for policy 1, policy_version 1564586 (0.0011) [2023-12-27 02:46:03,168][105620] Updated weights for policy 1, policy_version 1564596 (0.0010) [2023-12-27 02:46:03,223][105620] Updated weights for policy 1, policy_version 1564606 (0.0010) [2023-12-27 02:46:03,285][105620] Updated weights for policy 1, policy_version 1564616 (0.0010) [2023-12-27 02:46:03,885][105692] Updated weights for policy 0, policy_version 1561291 (0.0008) [2023-12-27 02:46:03,923][105620] Updated weights for policy 1, policy_version 1564626 (0.0011) [2023-12-27 02:46:03,946][105692] Updated weights for policy 0, policy_version 1561301 (0.0006) [2023-12-27 02:46:03,983][105620] Updated weights for policy 1, policy_version 1564636 (0.0011) [2023-12-27 02:46:04,012][105692] Updated weights for policy 0, policy_version 1561311 (0.0010) [2023-12-27 02:46:04,045][105620] Updated weights for policy 1, policy_version 1564646 (0.0011) [2023-12-27 02:46:04,808][105692] Updated weights for policy 0, policy_version 1561321 (0.0007) [2023-12-27 02:46:04,826][105620] Updated weights for policy 1, policy_version 1564656 (0.0011) [2023-12-27 02:46:04,868][105692] Updated weights for policy 0, policy_version 1561331 (0.0007) [2023-12-27 02:46:04,886][105620] Updated weights for policy 1, policy_version 1564666 (0.0011) [2023-12-27 02:46:04,929][105692] Updated weights for policy 0, policy_version 1561341 (0.0006) [2023-12-27 02:46:04,947][105620] Updated weights for policy 1, policy_version 1564676 (0.0010) [2023-12-27 02:46:04,990][105692] Updated weights for policy 0, policy_version 1561351 (0.0006) [2023-12-27 02:46:05,644][105620] Updated weights for policy 1, policy_version 1564686 (0.0007) [2023-12-27 02:46:05,698][105620] Updated weights for policy 1, policy_version 1564696 (0.0008) [2023-12-27 02:46:05,754][105620] Updated weights for policy 1, policy_version 1564706 (0.0011) [2023-12-27 02:46:05,788][105692] Updated weights for policy 0, policy_version 1561361 (0.0006) [2023-12-27 02:46:05,834][105692] Updated weights for policy 0, policy_version 1561371 (0.0008) [2023-12-27 02:46:05,883][105692] Updated weights for policy 0, policy_version 1561381 (0.0008) [2023-12-27 02:46:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 800391168. Throughput: 0: 9540.6, 1: 9855.8. Samples: 800378256. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:46:06,063][104569] Avg episode reward: [(0, '8073.911'), (1, '9091.403')] [2023-12-27 02:46:06,431][105620] Updated weights for policy 1, policy_version 1564716 (0.0011) [2023-12-27 02:46:06,487][105620] Updated weights for policy 1, policy_version 1564726 (0.0011) [2023-12-27 02:46:06,547][105620] Updated weights for policy 1, policy_version 1564736 (0.0011) [2023-12-27 02:46:06,758][105692] Updated weights for policy 0, policy_version 1561391 (0.0009) [2023-12-27 02:46:06,818][105692] Updated weights for policy 0, policy_version 1561401 (0.0008) [2023-12-27 02:46:06,878][105692] Updated weights for policy 0, policy_version 1561411 (0.0007) [2023-12-27 02:46:07,316][105620] Updated weights for policy 1, policy_version 1564746 (0.0011) [2023-12-27 02:46:07,384][105620] Updated weights for policy 1, policy_version 1564756 (0.0011) [2023-12-27 02:46:07,451][105620] Updated weights for policy 1, policy_version 1564766 (0.0011) [2023-12-27 02:46:07,516][105620] Updated weights for policy 1, policy_version 1564776 (0.0011) [2023-12-27 02:46:07,604][105692] Updated weights for policy 0, policy_version 1561421 (0.0006) [2023-12-27 02:46:07,663][105692] Updated weights for policy 0, policy_version 1561431 (0.0005) [2023-12-27 02:46:07,724][105692] Updated weights for policy 0, policy_version 1561441 (0.0006) [2023-12-27 02:46:08,244][105620] Updated weights for policy 1, policy_version 1564786 (0.0010) [2023-12-27 02:46:08,289][105620] Updated weights for policy 1, policy_version 1564796 (0.0010) [2023-12-27 02:46:08,342][105692] Updated weights for policy 0, policy_version 1561451 (0.0008) [2023-12-27 02:46:08,353][105620] Updated weights for policy 1, policy_version 1564806 (0.0011) [2023-12-27 02:46:08,412][105692] Updated weights for policy 0, policy_version 1561461 (0.0009) [2023-12-27 02:46:08,477][105692] Updated weights for policy 0, policy_version 1561471 (0.0008) [2023-12-27 02:46:09,106][105620] Updated weights for policy 1, policy_version 1564816 (0.0010) [2023-12-27 02:46:09,161][105620] Updated weights for policy 1, policy_version 1564826 (0.0010) [2023-12-27 02:46:09,195][105692] Updated weights for policy 0, policy_version 1561481 (0.0009) [2023-12-27 02:46:09,234][105620] Updated weights for policy 1, policy_version 1564836 (0.0010) [2023-12-27 02:46:09,260][105692] Updated weights for policy 0, policy_version 1561491 (0.0009) [2023-12-27 02:46:09,317][105692] Updated weights for policy 0, policy_version 1561501 (0.0007) [2023-12-27 02:46:09,390][105692] Updated weights for policy 0, policy_version 1561511 (0.0009) [2023-12-27 02:46:10,018][105620] Updated weights for policy 1, policy_version 1564846 (0.0008) [2023-12-27 02:46:10,078][105620] Updated weights for policy 1, policy_version 1564856 (0.0009) [2023-12-27 02:46:10,106][105586] KL-divergence is very high: 111.0334 [2023-12-27 02:46:10,136][105620] Updated weights for policy 1, policy_version 1564866 (0.0007) [2023-12-27 02:46:10,150][105692] Updated weights for policy 0, policy_version 1561521 (0.0008) [2023-12-27 02:46:10,158][105586] KL-divergence is very high: 121.2455 [2023-12-27 02:46:10,205][105692] Updated weights for policy 0, policy_version 1561531 (0.0009) [2023-12-27 02:46:10,259][105692] Updated weights for policy 0, policy_version 1561541 (0.0009) [2023-12-27 02:46:10,783][105620] Updated weights for policy 1, policy_version 1564876 (0.0006) [2023-12-27 02:46:10,853][105620] Updated weights for policy 1, policy_version 1564886 (0.0009) [2023-12-27 02:46:10,907][105620] Updated weights for policy 1, policy_version 1564896 (0.0010) [2023-12-27 02:46:11,051][105692] Updated weights for policy 0, policy_version 1561552 (0.0009) [2023-12-27 02:46:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 800481280. Throughput: 0: 9486.7, 1: 9783.0. Samples: 800490724. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:46:11,062][104569] Avg episode reward: [(0, '8163.851'), (1, '8819.223')] [2023-12-27 02:46:11,110][105692] Updated weights for policy 0, policy_version 1561562 (0.0007) [2023-12-27 02:46:11,175][105692] Updated weights for policy 0, policy_version 1561572 (0.0008) [2023-12-27 02:46:11,690][105620] Updated weights for policy 1, policy_version 1564906 (0.0010) [2023-12-27 02:46:11,758][105620] Updated weights for policy 1, policy_version 1564917 (0.0009) [2023-12-27 02:46:11,819][105620] Updated weights for policy 1, policy_version 1564927 (0.0007) [2023-12-27 02:46:11,886][105692] Updated weights for policy 0, policy_version 1561582 (0.0009) [2023-12-27 02:46:11,940][105692] Updated weights for policy 0, policy_version 1561592 (0.0010) [2023-12-27 02:46:12,002][105692] Updated weights for policy 0, policy_version 1561603 (0.0012) [2023-12-27 02:46:12,538][105620] Updated weights for policy 1, policy_version 1564937 (0.0010) [2023-12-27 02:46:12,596][105620] Updated weights for policy 1, policy_version 1564947 (0.0009) [2023-12-27 02:46:12,654][105620] Updated weights for policy 1, policy_version 1564957 (0.0009) [2023-12-27 02:46:12,715][105620] Updated weights for policy 1, policy_version 1564967 (0.0009) [2023-12-27 02:46:12,767][105692] Updated weights for policy 0, policy_version 1561613 (0.0009) [2023-12-27 02:46:12,824][105692] Updated weights for policy 0, policy_version 1561623 (0.0008) [2023-12-27 02:46:12,885][105692] Updated weights for policy 0, policy_version 1561633 (0.0007) [2023-12-27 02:46:13,440][105620] Updated weights for policy 1, policy_version 1564977 (0.0010) [2023-12-27 02:46:13,489][105620] Updated weights for policy 1, policy_version 1564987 (0.0008) [2023-12-27 02:46:13,543][105620] Updated weights for policy 1, policy_version 1564997 (0.0006) [2023-12-27 02:46:13,644][105692] Updated weights for policy 0, policy_version 1561643 (0.0006) [2023-12-27 02:46:13,698][105692] Updated weights for policy 0, policy_version 1561653 (0.0008) [2023-12-27 02:46:13,749][105692] Updated weights for policy 0, policy_version 1561663 (0.0008) [2023-12-27 02:46:14,298][105620] Updated weights for policy 1, policy_version 1565007 (0.0010) [2023-12-27 02:46:14,356][105620] Updated weights for policy 1, policy_version 1565017 (0.0010) [2023-12-27 02:46:14,417][105620] Updated weights for policy 1, policy_version 1565027 (0.0010) [2023-12-27 02:46:14,514][105692] Updated weights for policy 0, policy_version 1561673 (0.0008) [2023-12-27 02:46:14,567][105692] Updated weights for policy 0, policy_version 1561683 (0.0007) [2023-12-27 02:46:14,618][105692] Updated weights for policy 0, policy_version 1561693 (0.0005) [2023-12-27 02:46:14,678][105692] Updated weights for policy 0, policy_version 1561703 (0.0007) [2023-12-27 02:46:15,126][105620] Updated weights for policy 1, policy_version 1565037 (0.0009) [2023-12-27 02:46:15,192][105620] Updated weights for policy 1, policy_version 1565047 (0.0006) [2023-12-27 02:46:15,254][105620] Updated weights for policy 1, policy_version 1565057 (0.0007) [2023-12-27 02:46:15,432][105692] Updated weights for policy 0, policy_version 1561713 (0.0011) [2023-12-27 02:46:15,485][105692] Updated weights for policy 0, policy_version 1561723 (0.0011) [2023-12-27 02:46:15,544][105692] Updated weights for policy 0, policy_version 1561733 (0.0010) [2023-12-27 02:46:15,917][105620] Updated weights for policy 1, policy_version 1565067 (0.0008) [2023-12-27 02:46:15,965][105620] Updated weights for policy 1, policy_version 1565077 (0.0010) [2023-12-27 02:46:16,017][105620] Updated weights for policy 1, policy_version 1565087 (0.0010) [2023-12-27 02:46:16,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 800571392. Throughput: 0: 9484.4, 1: 9634.0. Samples: 800547256. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:46:16,063][104569] Avg episode reward: [(0, '8619.894'), (1, '8725.211')] [2023-12-27 02:46:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001561736_399859712.pth... [2023-12-27 02:46:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001560648_399581184.pth [2023-12-27 02:46:16,075][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001565096_400719872.pth... [2023-12-27 02:46:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001563976_400433152.pth [2023-12-27 02:46:16,183][105692] Updated weights for policy 0, policy_version 1561743 (0.0008) [2023-12-27 02:46:16,230][105692] Updated weights for policy 0, policy_version 1561753 (0.0005) [2023-12-27 02:46:16,276][105692] Updated weights for policy 0, policy_version 1561763 (0.0005) [2023-12-27 02:46:16,754][105620] Updated weights for policy 1, policy_version 1565097 (0.0010) [2023-12-27 02:46:16,809][105620] Updated weights for policy 1, policy_version 1565107 (0.0010) [2023-12-27 02:46:16,850][105692] Updated weights for policy 0, policy_version 1561773 (0.0008) [2023-12-27 02:46:16,868][105620] Updated weights for policy 1, policy_version 1565117 (0.0010) [2023-12-27 02:46:16,910][105692] Updated weights for policy 0, policy_version 1561783 (0.0011) [2023-12-27 02:46:16,920][105620] Updated weights for policy 1, policy_version 1565127 (0.0010) [2023-12-27 02:46:16,962][105692] Updated weights for policy 0, policy_version 1561793 (0.0011) [2023-12-27 02:46:17,677][105620] Updated weights for policy 1, policy_version 1565137 (0.0010) [2023-12-27 02:46:17,711][105586] KL-divergence is very high: 136.6513 [2023-12-27 02:46:17,725][105620] Updated weights for policy 1, policy_version 1565147 (0.0010) [2023-12-27 02:46:17,732][105692] Updated weights for policy 0, policy_version 1561803 (0.0009) [2023-12-27 02:46:17,753][105586] KL-divergence is very high: 179.6792 [2023-12-27 02:46:17,783][105620] Updated weights for policy 1, policy_version 1565157 (0.0010) [2023-12-27 02:46:17,786][105692] Updated weights for policy 0, policy_version 1561813 (0.0005) [2023-12-27 02:46:17,842][105692] Updated weights for policy 0, policy_version 1561823 (0.0005) [2023-12-27 02:46:18,425][105620] Updated weights for policy 1, policy_version 1565167 (0.0010) [2023-12-27 02:46:18,449][105692] Updated weights for policy 0, policy_version 1561833 (0.0007) [2023-12-27 02:46:18,482][105620] Updated weights for policy 1, policy_version 1565177 (0.0010) [2023-12-27 02:46:18,513][105692] Updated weights for policy 0, policy_version 1561843 (0.0011) [2023-12-27 02:46:18,547][105620] Updated weights for policy 1, policy_version 1565187 (0.0011) [2023-12-27 02:46:18,572][105692] Updated weights for policy 0, policy_version 1561853 (0.0010) [2023-12-27 02:46:18,637][105692] Updated weights for policy 0, policy_version 1561863 (0.0011) [2023-12-27 02:46:19,283][105620] Updated weights for policy 1, policy_version 1565197 (0.0011) [2023-12-27 02:46:19,344][105620] Updated weights for policy 1, policy_version 1565207 (0.0010) [2023-12-27 02:46:19,349][105692] Updated weights for policy 0, policy_version 1561873 (0.0009) [2023-12-27 02:46:19,402][105620] Updated weights for policy 1, policy_version 1565217 (0.0007) [2023-12-27 02:46:19,413][105692] Updated weights for policy 0, policy_version 1561883 (0.0009) [2023-12-27 02:46:19,479][105692] Updated weights for policy 0, policy_version 1561893 (0.0009) [2023-12-27 02:46:20,090][105620] Updated weights for policy 1, policy_version 1565227 (0.0007) [2023-12-27 02:46:20,154][105620] Updated weights for policy 1, policy_version 1565237 (0.0011) [2023-12-27 02:46:20,218][105620] Updated weights for policy 1, policy_version 1565247 (0.0009) [2023-12-27 02:46:20,285][105692] Updated weights for policy 0, policy_version 1561903 (0.0008) [2023-12-27 02:46:20,346][105692] Updated weights for policy 0, policy_version 1561913 (0.0008) [2023-12-27 02:46:20,402][105692] Updated weights for policy 0, policy_version 1561923 (0.0008) [2023-12-27 02:46:20,986][105620] Updated weights for policy 1, policy_version 1565257 (0.0011) [2023-12-27 02:46:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 800669696. Throughput: 0: 9531.7, 1: 9543.3. Samples: 800666132. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:46:21,063][104569] Avg episode reward: [(0, '8986.281'), (1, '8728.521')] [2023-12-27 02:46:21,071][105620] Updated weights for policy 1, policy_version 1565267 (0.0011) [2023-12-27 02:46:21,132][105620] Updated weights for policy 1, policy_version 1565277 (0.0011) [2023-12-27 02:46:21,190][105692] Updated weights for policy 0, policy_version 1561933 (0.0009) [2023-12-27 02:46:21,190][105620] Updated weights for policy 1, policy_version 1565287 (0.0011) [2023-12-27 02:46:21,242][105692] Updated weights for policy 0, policy_version 1561943 (0.0008) [2023-12-27 02:46:21,308][105692] Updated weights for policy 0, policy_version 1561953 (0.0009) [2023-12-27 02:46:21,973][105620] Updated weights for policy 1, policy_version 1565297 (0.0011) [2023-12-27 02:46:22,027][105620] Updated weights for policy 1, policy_version 1565307 (0.0011) [2023-12-27 02:46:22,084][105620] Updated weights for policy 1, policy_version 1565317 (0.0011) [2023-12-27 02:46:22,188][105692] Updated weights for policy 0, policy_version 1561963 (0.0008) [2023-12-27 02:46:22,241][105692] Updated weights for policy 0, policy_version 1561973 (0.0008) [2023-12-27 02:46:22,311][105692] Updated weights for policy 0, policy_version 1561983 (0.0009) [2023-12-27 02:46:22,863][105620] Updated weights for policy 1, policy_version 1565327 (0.0011) [2023-12-27 02:46:22,924][105620] Updated weights for policy 1, policy_version 1565337 (0.0011) [2023-12-27 02:46:22,926][105586] KL-divergence is very high: 131.7746 [2023-12-27 02:46:22,978][105586] KL-divergence is very high: 231.0011 [2023-12-27 02:46:22,992][105620] Updated weights for policy 1, policy_version 1565347 (0.0009) [2023-12-27 02:46:23,078][105692] Updated weights for policy 0, policy_version 1561993 (0.0009) [2023-12-27 02:46:23,137][105692] Updated weights for policy 0, policy_version 1562003 (0.0011) [2023-12-27 02:46:23,192][105692] Updated weights for policy 0, policy_version 1562013 (0.0010) [2023-12-27 02:46:23,251][105692] Updated weights for policy 0, policy_version 1562023 (0.0007) [2023-12-27 02:46:23,761][105620] Updated weights for policy 1, policy_version 1565357 (0.0009) [2023-12-27 02:46:23,820][105620] Updated weights for policy 1, policy_version 1565367 (0.0009) [2023-12-27 02:46:23,880][105620] Updated weights for policy 1, policy_version 1565377 (0.0009) [2023-12-27 02:46:23,903][105692] Updated weights for policy 0, policy_version 1562033 (0.0008) [2023-12-27 02:46:23,958][105692] Updated weights for policy 0, policy_version 1562043 (0.0008) [2023-12-27 02:46:24,017][105692] Updated weights for policy 0, policy_version 1562053 (0.0009) [2023-12-27 02:46:24,580][105620] Updated weights for policy 1, policy_version 1565387 (0.0009) [2023-12-27 02:46:24,638][105620] Updated weights for policy 1, policy_version 1565397 (0.0009) [2023-12-27 02:46:24,708][105620] Updated weights for policy 1, policy_version 1565407 (0.0009) [2023-12-27 02:46:24,806][105692] Updated weights for policy 0, policy_version 1562063 (0.0010) [2023-12-27 02:46:24,860][105692] Updated weights for policy 0, policy_version 1562073 (0.0009) [2023-12-27 02:46:24,908][105692] Updated weights for policy 0, policy_version 1562083 (0.0009) [2023-12-27 02:46:25,371][105620] Updated weights for policy 1, policy_version 1565417 (0.0006) [2023-12-27 02:46:25,428][105620] Updated weights for policy 1, policy_version 1565427 (0.0009) [2023-12-27 02:46:25,480][105620] Updated weights for policy 1, policy_version 1565437 (0.0009) [2023-12-27 02:46:25,528][105620] Updated weights for policy 1, policy_version 1565447 (0.0009) [2023-12-27 02:46:25,720][105692] Updated weights for policy 0, policy_version 1562093 (0.0009) [2023-12-27 02:46:25,770][105692] Updated weights for policy 0, policy_version 1562103 (0.0009) [2023-12-27 02:46:25,815][105692] Updated weights for policy 0, policy_version 1562113 (0.0008) [2023-12-27 02:46:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 800768000. Throughput: 0: 9456.5, 1: 9516.4. Samples: 800776764. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:46:26,062][104569] Avg episode reward: [(0, '8617.095'), (1, '8815.301')] [2023-12-27 02:46:26,254][105620] Updated weights for policy 1, policy_version 1565457 (0.0006) [2023-12-27 02:46:26,311][105620] Updated weights for policy 1, policy_version 1565468 (0.0010) [2023-12-27 02:46:26,367][105620] Updated weights for policy 1, policy_version 1565478 (0.0009) [2023-12-27 02:46:26,615][105692] Updated weights for policy 0, policy_version 1562123 (0.0009) [2023-12-27 02:46:26,659][105692] Updated weights for policy 0, policy_version 1562133 (0.0008) [2023-12-27 02:46:26,712][105692] Updated weights for policy 0, policy_version 1562143 (0.0009) [2023-12-27 02:46:26,946][105620] Updated weights for policy 1, policy_version 1565488 (0.0006) [2023-12-27 02:46:26,994][105620] Updated weights for policy 1, policy_version 1565498 (0.0009) [2023-12-27 02:46:27,045][105620] Updated weights for policy 1, policy_version 1565508 (0.0009) [2023-12-27 02:46:27,501][105692] Updated weights for policy 0, policy_version 1562153 (0.0009) [2023-12-27 02:46:27,555][105692] Updated weights for policy 0, policy_version 1562163 (0.0007) [2023-12-27 02:46:27,601][105692] Updated weights for policy 0, policy_version 1562173 (0.0009) [2023-12-27 02:46:27,651][105692] Updated weights for policy 0, policy_version 1562183 (0.0009) [2023-12-27 02:46:27,732][105620] Updated weights for policy 1, policy_version 1565518 (0.0008) [2023-12-27 02:46:27,789][105620] Updated weights for policy 1, policy_version 1565528 (0.0009) [2023-12-27 02:46:27,846][105620] Updated weights for policy 1, policy_version 1565538 (0.0009) [2023-12-27 02:46:28,425][105692] Updated weights for policy 0, policy_version 1562193 (0.0008) [2023-12-27 02:46:28,488][105692] Updated weights for policy 0, policy_version 1562203 (0.0007) [2023-12-27 02:46:28,513][105620] Updated weights for policy 1, policy_version 1565548 (0.0007) [2023-12-27 02:46:28,543][105692] Updated weights for policy 0, policy_version 1562213 (0.0005) [2023-12-27 02:46:28,566][105620] Updated weights for policy 1, policy_version 1565558 (0.0006) [2023-12-27 02:46:28,632][105620] Updated weights for policy 1, policy_version 1565568 (0.0007) [2023-12-27 02:46:29,214][105692] Updated weights for policy 0, policy_version 1562223 (0.0008) [2023-12-27 02:46:29,270][105692] Updated weights for policy 0, policy_version 1562233 (0.0008) [2023-12-27 02:46:29,334][105692] Updated weights for policy 0, policy_version 1562243 (0.0009) [2023-12-27 02:46:29,367][105620] Updated weights for policy 1, policy_version 1565578 (0.0007) [2023-12-27 02:46:29,415][105620] Updated weights for policy 1, policy_version 1565588 (0.0009) [2023-12-27 02:46:29,474][105620] Updated weights for policy 1, policy_version 1565598 (0.0008) [2023-12-27 02:46:29,534][105620] Updated weights for policy 1, policy_version 1565608 (0.0005) [2023-12-27 02:46:30,015][105692] Updated weights for policy 0, policy_version 1562253 (0.0009) [2023-12-27 02:46:30,083][105692] Updated weights for policy 0, policy_version 1562263 (0.0010) [2023-12-27 02:46:30,138][105692] Updated weights for policy 0, policy_version 1562273 (0.0007) [2023-12-27 02:46:30,157][105620] Updated weights for policy 1, policy_version 1565618 (0.0007) [2023-12-27 02:46:30,218][105620] Updated weights for policy 1, policy_version 1565628 (0.0007) [2023-12-27 02:46:30,273][105620] Updated weights for policy 1, policy_version 1565638 (0.0008) [2023-12-27 02:46:30,834][105692] Updated weights for policy 0, policy_version 1562283 (0.0010) [2023-12-27 02:46:30,888][105692] Updated weights for policy 0, policy_version 1562293 (0.0006) [2023-12-27 02:46:30,937][105692] Updated weights for policy 0, policy_version 1562303 (0.0005) [2023-12-27 02:46:31,039][105620] Updated weights for policy 1, policy_version 1565648 (0.0007) [2023-12-27 02:46:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19114.7, 300 sec: 19549.7). Total num frames: 800866304. Throughput: 0: 9420.9, 1: 9598.3. Samples: 800836008. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:46:31,062][104569] Avg episode reward: [(0, '8437.072'), (1, '8901.230')] [2023-12-27 02:46:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001562312_400007168.pth... [2023-12-27 02:46:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001561192_399720448.pth [2023-12-27 02:46:31,098][105620] Updated weights for policy 1, policy_version 1565658 (0.0008) [2023-12-27 02:46:31,157][105620] Updated weights for policy 1, policy_version 1565668 (0.0008) [2023-12-27 02:46:31,182][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001565672_400867328.pth... [2023-12-27 02:46:31,185][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001564520_400572416.pth [2023-12-27 02:46:31,635][105692] Updated weights for policy 0, policy_version 1562313 (0.0006) [2023-12-27 02:46:31,691][105692] Updated weights for policy 0, policy_version 1562323 (0.0009) [2023-12-27 02:46:31,755][105692] Updated weights for policy 0, policy_version 1562333 (0.0008) [2023-12-27 02:46:31,815][105692] Updated weights for policy 0, policy_version 1562343 (0.0007) [2023-12-27 02:46:31,878][105620] Updated weights for policy 1, policy_version 1565678 (0.0010) [2023-12-27 02:46:31,938][105620] Updated weights for policy 1, policy_version 1565688 (0.0011) [2023-12-27 02:46:31,993][105620] Updated weights for policy 1, policy_version 1565698 (0.0010) [2023-12-27 02:46:32,474][105692] Updated weights for policy 0, policy_version 1562353 (0.0009) [2023-12-27 02:46:32,523][105692] Updated weights for policy 0, policy_version 1562363 (0.0009) [2023-12-27 02:46:32,577][105692] Updated weights for policy 0, policy_version 1562374 (0.0010) [2023-12-27 02:46:32,653][105620] Updated weights for policy 1, policy_version 1565708 (0.0008) [2023-12-27 02:46:32,708][105620] Updated weights for policy 1, policy_version 1565718 (0.0005) [2023-12-27 02:46:32,769][105620] Updated weights for policy 1, policy_version 1565728 (0.0009) [2023-12-27 02:46:33,400][105692] Updated weights for policy 0, policy_version 1562384 (0.0008) [2023-12-27 02:46:33,447][105692] Updated weights for policy 0, policy_version 1562394 (0.0007) [2023-12-27 02:46:33,480][105620] Updated weights for policy 1, policy_version 1565738 (0.0010) [2023-12-27 02:46:33,490][105692] Updated weights for policy 0, policy_version 1562404 (0.0007) [2023-12-27 02:46:33,527][105620] Updated weights for policy 1, policy_version 1565748 (0.0010) [2023-12-27 02:46:33,571][105620] Updated weights for policy 1, policy_version 1565758 (0.0010) [2023-12-27 02:46:33,616][105620] Updated weights for policy 1, policy_version 1565768 (0.0007) [2023-12-27 02:46:34,328][105620] Updated weights for policy 1, policy_version 1565778 (0.0009) [2023-12-27 02:46:34,336][105692] Updated weights for policy 0, policy_version 1562414 (0.0006) [2023-12-27 02:46:34,393][105620] Updated weights for policy 1, policy_version 1565788 (0.0006) [2023-12-27 02:46:34,396][105692] Updated weights for policy 0, policy_version 1562424 (0.0008) [2023-12-27 02:46:34,447][105692] Updated weights for policy 0, policy_version 1562434 (0.0008) [2023-12-27 02:46:34,462][105620] Updated weights for policy 1, policy_version 1565798 (0.0007) [2023-12-27 02:46:35,121][105620] Updated weights for policy 1, policy_version 1565808 (0.0006) [2023-12-27 02:46:35,177][105620] Updated weights for policy 1, policy_version 1565818 (0.0005) [2023-12-27 02:46:35,233][105620] Updated weights for policy 1, policy_version 1565828 (0.0005) [2023-12-27 02:46:35,253][105692] Updated weights for policy 0, policy_version 1562444 (0.0008) [2023-12-27 02:46:35,310][105692] Updated weights for policy 0, policy_version 1562454 (0.0007) [2023-12-27 02:46:35,359][105692] Updated weights for policy 0, policy_version 1562464 (0.0005) [2023-12-27 02:46:35,889][105620] Updated weights for policy 1, policy_version 1565838 (0.0005) [2023-12-27 02:46:35,935][105620] Updated weights for policy 1, policy_version 1565848 (0.0005) [2023-12-27 02:46:35,972][105692] Updated weights for policy 0, policy_version 1562474 (0.0007) [2023-12-27 02:46:35,996][105620] Updated weights for policy 1, policy_version 1565858 (0.0005) [2023-12-27 02:46:36,020][105692] Updated weights for policy 0, policy_version 1562484 (0.0010) [2023-12-27 02:46:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19114.7, 300 sec: 19549.7). Total num frames: 800964608. Throughput: 0: 9415.7, 1: 9629.8. Samples: 800953792. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:46:36,062][104569] Avg episode reward: [(0, '8805.916'), (1, '9175.201')] [2023-12-27 02:46:36,065][105692] Updated weights for policy 0, policy_version 1562494 (0.0010) [2023-12-27 02:46:36,120][105692] Updated weights for policy 0, policy_version 1562504 (0.0010) [2023-12-27 02:46:36,700][105620] Updated weights for policy 1, policy_version 1565868 (0.0007) [2023-12-27 02:46:36,755][105620] Updated weights for policy 1, policy_version 1565878 (0.0009) [2023-12-27 02:46:36,828][105620] Updated weights for policy 1, policy_version 1565888 (0.0008) [2023-12-27 02:46:36,909][105692] Updated weights for policy 0, policy_version 1562514 (0.0007) [2023-12-27 02:46:36,981][105692] Updated weights for policy 0, policy_version 1562524 (0.0009) [2023-12-27 02:46:37,043][105692] Updated weights for policy 0, policy_version 1562534 (0.0010) [2023-12-27 02:46:37,383][105620] Updated weights for policy 1, policy_version 1565898 (0.0007) [2023-12-27 02:46:37,446][105620] Updated weights for policy 1, policy_version 1565908 (0.0007) [2023-12-27 02:46:37,510][105620] Updated weights for policy 1, policy_version 1565918 (0.0007) [2023-12-27 02:46:37,575][105620] Updated weights for policy 1, policy_version 1565928 (0.0008) [2023-12-27 02:46:37,904][105692] Updated weights for policy 0, policy_version 1562544 (0.0009) [2023-12-27 02:46:37,957][105692] Updated weights for policy 0, policy_version 1562554 (0.0009) [2023-12-27 02:46:38,011][105692] Updated weights for policy 0, policy_version 1562564 (0.0010) [2023-12-27 02:46:38,159][105620] Updated weights for policy 1, policy_version 1565938 (0.0008) [2023-12-27 02:46:38,209][105620] Updated weights for policy 1, policy_version 1565948 (0.0005) [2023-12-27 02:46:38,254][105620] Updated weights for policy 1, policy_version 1565958 (0.0005) [2023-12-27 02:46:38,867][105620] Updated weights for policy 1, policy_version 1565968 (0.0008) [2023-12-27 02:46:38,881][105692] Updated weights for policy 0, policy_version 1562574 (0.0008) [2023-12-27 02:46:38,917][105620] Updated weights for policy 1, policy_version 1565978 (0.0006) [2023-12-27 02:46:38,932][105692] Updated weights for policy 0, policy_version 1562584 (0.0006) [2023-12-27 02:46:38,971][105620] Updated weights for policy 1, policy_version 1565988 (0.0007) [2023-12-27 02:46:38,982][105692] Updated weights for policy 0, policy_version 1562594 (0.0006) [2023-12-27 02:46:39,766][105620] Updated weights for policy 1, policy_version 1565998 (0.0008) [2023-12-27 02:46:39,780][105692] Updated weights for policy 0, policy_version 1562604 (0.0007) [2023-12-27 02:46:39,820][105620] Updated weights for policy 1, policy_version 1566008 (0.0006) [2023-12-27 02:46:39,848][105692] Updated weights for policy 0, policy_version 1562614 (0.0008) [2023-12-27 02:46:39,888][105620] Updated weights for policy 1, policy_version 1566018 (0.0008) [2023-12-27 02:46:39,914][105692] Updated weights for policy 0, policy_version 1562624 (0.0008) [2023-12-27 02:46:40,650][105620] Updated weights for policy 1, policy_version 1566028 (0.0008) [2023-12-27 02:46:40,679][105692] Updated weights for policy 0, policy_version 1562634 (0.0008) [2023-12-27 02:46:40,705][105620] Updated weights for policy 1, policy_version 1566038 (0.0010) [2023-12-27 02:46:40,733][105692] Updated weights for policy 0, policy_version 1562644 (0.0006) [2023-12-27 02:46:40,764][105620] Updated weights for policy 1, policy_version 1566048 (0.0009) [2023-12-27 02:46:40,784][105692] Updated weights for policy 0, policy_version 1562654 (0.0007) [2023-12-27 02:46:40,843][105692] Updated weights for policy 0, policy_version 1562664 (0.0008) [2023-12-27 02:46:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19114.7, 300 sec: 19549.7). Total num frames: 801062912. Throughput: 0: 9350.1, 1: 9758.6. Samples: 801069340. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:46:41,062][104569] Avg episode reward: [(0, '8988.675'), (1, '9173.758')] [2023-12-27 02:46:41,551][105692] Updated weights for policy 0, policy_version 1562674 (0.0009) [2023-12-27 02:46:41,560][105620] Updated weights for policy 1, policy_version 1566058 (0.0008) [2023-12-27 02:46:41,620][105692] Updated weights for policy 0, policy_version 1562684 (0.0009) [2023-12-27 02:46:41,629][105620] Updated weights for policy 1, policy_version 1566068 (0.0007) [2023-12-27 02:46:41,687][105692] Updated weights for policy 0, policy_version 1562694 (0.0006) [2023-12-27 02:46:41,698][105620] Updated weights for policy 1, policy_version 1566078 (0.0007) [2023-12-27 02:46:41,766][105620] Updated weights for policy 1, policy_version 1566088 (0.0006) [2023-12-27 02:46:42,330][105620] Updated weights for policy 1, policy_version 1566098 (0.0010) [2023-12-27 02:46:42,396][105620] Updated weights for policy 1, policy_version 1566108 (0.0007) [2023-12-27 02:46:42,415][105692] Updated weights for policy 0, policy_version 1562704 (0.0009) [2023-12-27 02:46:42,453][105620] Updated weights for policy 1, policy_version 1566118 (0.0008) [2023-12-27 02:46:42,472][105692] Updated weights for policy 0, policy_version 1562714 (0.0007) [2023-12-27 02:46:42,529][105692] Updated weights for policy 0, policy_version 1562724 (0.0008) [2023-12-27 02:46:43,057][105620] Updated weights for policy 1, policy_version 1566128 (0.0006) [2023-12-27 02:46:43,111][105620] Updated weights for policy 1, policy_version 1566138 (0.0005) [2023-12-27 02:46:43,172][105620] Updated weights for policy 1, policy_version 1566148 (0.0005) [2023-12-27 02:46:43,356][105692] Updated weights for policy 0, policy_version 1562734 (0.0009) [2023-12-27 02:46:43,408][105692] Updated weights for policy 0, policy_version 1562744 (0.0009) [2023-12-27 02:46:43,460][105692] Updated weights for policy 0, policy_version 1562754 (0.0009) [2023-12-27 02:46:43,843][105620] Updated weights for policy 1, policy_version 1566158 (0.0008) [2023-12-27 02:46:43,901][105620] Updated weights for policy 1, policy_version 1566168 (0.0006) [2023-12-27 02:46:43,958][105620] Updated weights for policy 1, policy_version 1566178 (0.0005) [2023-12-27 02:46:44,236][105692] Updated weights for policy 0, policy_version 1562764 (0.0009) [2023-12-27 02:46:44,299][105692] Updated weights for policy 0, policy_version 1562774 (0.0009) [2023-12-27 02:46:44,360][105692] Updated weights for policy 0, policy_version 1562784 (0.0009) [2023-12-27 02:46:44,652][105620] Updated weights for policy 1, policy_version 1566188 (0.0006) [2023-12-27 02:46:44,707][105620] Updated weights for policy 1, policy_version 1566198 (0.0008) [2023-12-27 02:46:44,766][105620] Updated weights for policy 1, policy_version 1566208 (0.0009) [2023-12-27 02:46:45,112][105692] Updated weights for policy 0, policy_version 1562794 (0.0009) [2023-12-27 02:46:45,173][105692] Updated weights for policy 0, policy_version 1562804 (0.0009) [2023-12-27 02:46:45,237][105692] Updated weights for policy 0, policy_version 1562814 (0.0006) [2023-12-27 02:46:45,307][105692] Updated weights for policy 0, policy_version 1562824 (0.0006) [2023-12-27 02:46:45,401][105620] Updated weights for policy 1, policy_version 1566218 (0.0008) [2023-12-27 02:46:45,456][105620] Updated weights for policy 1, policy_version 1566228 (0.0007) [2023-12-27 02:46:45,506][105620] Updated weights for policy 1, policy_version 1566238 (0.0008) [2023-12-27 02:46:45,562][105620] Updated weights for policy 1, policy_version 1566248 (0.0006) [2023-12-27 02:46:46,035][105692] Updated weights for policy 0, policy_version 1562834 (0.0009) [2023-12-27 02:46:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19114.7, 300 sec: 19521.9). Total num frames: 801153024. Throughput: 0: 9308.2, 1: 9849.4. Samples: 801127620. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:46:46,062][104569] Avg episode reward: [(0, '8994.092'), (1, '8989.291')] [2023-12-27 02:46:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001566248_401014784.pth... [2023-12-27 02:46:46,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001565096_400719872.pth [2023-12-27 02:46:46,093][105692] Updated weights for policy 0, policy_version 1562845 (0.0010) [2023-12-27 02:46:46,152][105692] Updated weights for policy 0, policy_version 1562855 (0.0007) [2023-12-27 02:46:46,156][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001562856_400146432.pth... [2023-12-27 02:46:46,160][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001561736_399859712.pth [2023-12-27 02:46:46,177][105620] Updated weights for policy 1, policy_version 1566258 (0.0009) [2023-12-27 02:46:46,231][105620] Updated weights for policy 1, policy_version 1566268 (0.0009) [2023-12-27 02:46:46,277][105620] Updated weights for policy 1, policy_version 1566278 (0.0008) [2023-12-27 02:46:46,888][105692] Updated weights for policy 0, policy_version 1562865 (0.0009) [2023-12-27 02:46:46,948][105692] Updated weights for policy 0, policy_version 1562875 (0.0006) [2023-12-27 02:46:46,953][105620] Updated weights for policy 1, policy_version 1566288 (0.0008) [2023-12-27 02:46:47,008][105692] Updated weights for policy 0, policy_version 1562885 (0.0007) [2023-12-27 02:46:47,013][105620] Updated weights for policy 1, policy_version 1566298 (0.0006) [2023-12-27 02:46:47,081][105620] Updated weights for policy 1, policy_version 1566308 (0.0009) [2023-12-27 02:46:47,766][105620] Updated weights for policy 1, policy_version 1566318 (0.0007) [2023-12-27 02:46:47,772][105692] Updated weights for policy 0, policy_version 1562895 (0.0007) [2023-12-27 02:46:47,822][105620] Updated weights for policy 1, policy_version 1566328 (0.0008) [2023-12-27 02:46:47,828][105692] Updated weights for policy 0, policy_version 1562905 (0.0006) [2023-12-27 02:46:47,882][105620] Updated weights for policy 1, policy_version 1566338 (0.0007) [2023-12-27 02:46:47,888][105692] Updated weights for policy 0, policy_version 1562915 (0.0006) [2023-12-27 02:46:48,589][105620] Updated weights for policy 1, policy_version 1566348 (0.0008) [2023-12-27 02:46:48,631][105692] Updated weights for policy 0, policy_version 1562925 (0.0006) [2023-12-27 02:46:48,646][105620] Updated weights for policy 1, policy_version 1566358 (0.0008) [2023-12-27 02:46:48,689][105692] Updated weights for policy 0, policy_version 1562935 (0.0006) [2023-12-27 02:46:48,697][105620] Updated weights for policy 1, policy_version 1566368 (0.0008) [2023-12-27 02:46:48,751][105692] Updated weights for policy 0, policy_version 1562945 (0.0006) [2023-12-27 02:46:49,391][105620] Updated weights for policy 1, policy_version 1566378 (0.0007) [2023-12-27 02:46:49,455][105620] Updated weights for policy 1, policy_version 1566388 (0.0006) [2023-12-27 02:46:49,512][105620] Updated weights for policy 1, policy_version 1566398 (0.0008) [2023-12-27 02:46:49,531][105692] Updated weights for policy 0, policy_version 1562955 (0.0007) [2023-12-27 02:46:49,572][105620] Updated weights for policy 1, policy_version 1566408 (0.0008) [2023-12-27 02:46:49,589][105692] Updated weights for policy 0, policy_version 1562965 (0.0009) [2023-12-27 02:46:49,649][105692] Updated weights for policy 0, policy_version 1562975 (0.0010) [2023-12-27 02:46:50,308][105620] Updated weights for policy 1, policy_version 1566418 (0.0009) [2023-12-27 02:46:50,361][105620] Updated weights for policy 1, policy_version 1566428 (0.0009) [2023-12-27 02:46:50,413][105620] Updated weights for policy 1, policy_version 1566438 (0.0009) [2023-12-27 02:46:50,426][105692] Updated weights for policy 0, policy_version 1562985 (0.0009) [2023-12-27 02:46:50,484][105692] Updated weights for policy 0, policy_version 1562995 (0.0009) [2023-12-27 02:46:50,542][105692] Updated weights for policy 0, policy_version 1563005 (0.0009) [2023-12-27 02:46:50,604][105692] Updated weights for policy 0, policy_version 1563015 (0.0010) [2023-12-27 02:46:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.7, 300 sec: 19522.0). Total num frames: 801251328. Throughput: 0: 9345.4, 1: 9909.5. Samples: 801244728. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 02:46:51,062][104569] Avg episode reward: [(0, '8812.618'), (1, '8988.655')] [2023-12-27 02:46:51,197][105620] Updated weights for policy 1, policy_version 1566448 (0.0009) [2023-12-27 02:46:51,249][105620] Updated weights for policy 1, policy_version 1566458 (0.0009) [2023-12-27 02:46:51,316][105620] Updated weights for policy 1, policy_version 1566468 (0.0010) [2023-12-27 02:46:51,407][105692] Updated weights for policy 0, policy_version 1563025 (0.0008) [2023-12-27 02:46:51,460][105692] Updated weights for policy 0, policy_version 1563035 (0.0008) [2023-12-27 02:46:51,509][105692] Updated weights for policy 0, policy_version 1563045 (0.0008) [2023-12-27 02:46:52,121][105620] Updated weights for policy 1, policy_version 1566478 (0.0008) [2023-12-27 02:46:52,180][105620] Updated weights for policy 1, policy_version 1566488 (0.0008) [2023-12-27 02:46:52,234][105620] Updated weights for policy 1, policy_version 1566498 (0.0007) [2023-12-27 02:46:52,278][105692] Updated weights for policy 0, policy_version 1563055 (0.0010) [2023-12-27 02:46:52,334][105692] Updated weights for policy 0, policy_version 1563065 (0.0011) [2023-12-27 02:46:52,402][105692] Updated weights for policy 0, policy_version 1563075 (0.0011) [2023-12-27 02:46:52,921][105620] Updated weights for policy 1, policy_version 1566508 (0.0007) [2023-12-27 02:46:52,972][105620] Updated weights for policy 1, policy_version 1566518 (0.0005) [2023-12-27 02:46:53,023][105620] Updated weights for policy 1, policy_version 1566528 (0.0005) [2023-12-27 02:46:53,154][105692] Updated weights for policy 0, policy_version 1563085 (0.0011) [2023-12-27 02:46:53,212][105692] Updated weights for policy 0, policy_version 1563095 (0.0010) [2023-12-27 02:46:53,267][105692] Updated weights for policy 0, policy_version 1563105 (0.0009) [2023-12-27 02:46:53,567][105620] Updated weights for policy 1, policy_version 1566538 (0.0005) [2023-12-27 02:46:53,619][105620] Updated weights for policy 1, policy_version 1566548 (0.0005) [2023-12-27 02:46:53,680][105620] Updated weights for policy 1, policy_version 1566558 (0.0006) [2023-12-27 02:46:53,732][105620] Updated weights for policy 1, policy_version 1566568 (0.0005) [2023-12-27 02:46:53,998][105692] Updated weights for policy 0, policy_version 1563115 (0.0008) [2023-12-27 02:46:54,046][105692] Updated weights for policy 0, policy_version 1563125 (0.0005) [2023-12-27 02:46:54,094][105692] Updated weights for policy 0, policy_version 1563135 (0.0005) [2023-12-27 02:46:54,409][105620] Updated weights for policy 1, policy_version 1566578 (0.0008) [2023-12-27 02:46:54,468][105620] Updated weights for policy 1, policy_version 1566588 (0.0007) [2023-12-27 02:46:54,525][105620] Updated weights for policy 1, policy_version 1566598 (0.0006) [2023-12-27 02:46:54,815][105692] Updated weights for policy 0, policy_version 1563145 (0.0009) [2023-12-27 02:46:54,862][105692] Updated weights for policy 0, policy_version 1563155 (0.0010) [2023-12-27 02:46:54,910][105692] Updated weights for policy 0, policy_version 1563165 (0.0010) [2023-12-27 02:46:54,959][105692] Updated weights for policy 0, policy_version 1563175 (0.0010) [2023-12-27 02:46:55,142][105620] Updated weights for policy 1, policy_version 1566608 (0.0009) [2023-12-27 02:46:55,197][105620] Updated weights for policy 1, policy_version 1566618 (0.0010) [2023-12-27 02:46:55,253][105620] Updated weights for policy 1, policy_version 1566628 (0.0010) [2023-12-27 02:46:55,681][105692] Updated weights for policy 0, policy_version 1563185 (0.0010) [2023-12-27 02:46:55,736][105692] Updated weights for policy 0, policy_version 1563195 (0.0010) [2023-12-27 02:46:55,791][105692] Updated weights for policy 0, policy_version 1563205 (0.0010) [2023-12-27 02:46:55,966][105620] Updated weights for policy 1, policy_version 1566638 (0.0010) [2023-12-27 02:46:56,028][105620] Updated weights for policy 1, policy_version 1566648 (0.0010) [2023-12-27 02:46:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 801349632. Throughput: 0: 9359.6, 1: 9976.8. Samples: 801360860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:46:56,062][104569] Avg episode reward: [(0, '8809.543'), (1, '8992.306')] [2023-12-27 02:46:56,080][105620] Updated weights for policy 1, policy_version 1566658 (0.0010) [2023-12-27 02:46:56,505][105692] Updated weights for policy 0, policy_version 1563215 (0.0009) [2023-12-27 02:46:56,549][105692] Updated weights for policy 0, policy_version 1563225 (0.0007) [2023-12-27 02:46:56,594][105692] Updated weights for policy 0, policy_version 1563235 (0.0008) [2023-12-27 02:46:56,827][105620] Updated weights for policy 1, policy_version 1566668 (0.0010) [2023-12-27 02:46:56,884][105620] Updated weights for policy 1, policy_version 1566678 (0.0010) [2023-12-27 02:46:56,935][105620] Updated weights for policy 1, policy_version 1566688 (0.0010) [2023-12-27 02:46:57,326][105692] Updated weights for policy 0, policy_version 1563245 (0.0009) [2023-12-27 02:46:57,384][105692] Updated weights for policy 0, policy_version 1563255 (0.0009) [2023-12-27 02:46:57,445][105692] Updated weights for policy 0, policy_version 1563265 (0.0005) [2023-12-27 02:46:57,671][105620] Updated weights for policy 1, policy_version 1566698 (0.0009) [2023-12-27 02:46:57,726][105620] Updated weights for policy 1, policy_version 1566708 (0.0010) [2023-12-27 02:46:57,794][105620] Updated weights for policy 1, policy_version 1566718 (0.0010) [2023-12-27 02:46:57,845][105620] Updated weights for policy 1, policy_version 1566728 (0.0010) [2023-12-27 02:46:57,965][105692] Updated weights for policy 0, policy_version 1563275 (0.0005) [2023-12-27 02:46:58,017][105692] Updated weights for policy 0, policy_version 1563285 (0.0007) [2023-12-27 02:46:58,078][105692] Updated weights for policy 0, policy_version 1563295 (0.0007) [2023-12-27 02:46:58,607][105620] Updated weights for policy 1, policy_version 1566738 (0.0009) [2023-12-27 02:46:58,671][105620] Updated weights for policy 1, policy_version 1566748 (0.0009) [2023-12-27 02:46:58,741][105620] Updated weights for policy 1, policy_version 1566758 (0.0009) [2023-12-27 02:46:58,864][105692] Updated weights for policy 0, policy_version 1563305 (0.0008) [2023-12-27 02:46:58,928][105692] Updated weights for policy 0, policy_version 1563315 (0.0012) [2023-12-27 02:46:58,977][105692] Updated weights for policy 0, policy_version 1563325 (0.0009) [2023-12-27 02:46:59,042][105692] Updated weights for policy 0, policy_version 1563335 (0.0011) [2023-12-27 02:46:59,570][105620] Updated weights for policy 1, policy_version 1566768 (0.0008) [2023-12-27 02:46:59,633][105620] Updated weights for policy 1, policy_version 1566778 (0.0009) [2023-12-27 02:46:59,683][105620] Updated weights for policy 1, policy_version 1566788 (0.0009) [2023-12-27 02:46:59,822][105692] Updated weights for policy 0, policy_version 1563345 (0.0007) [2023-12-27 02:46:59,886][105692] Updated weights for policy 0, policy_version 1563355 (0.0009) [2023-12-27 02:46:59,954][105692] Updated weights for policy 0, policy_version 1563365 (0.0009) [2023-12-27 02:47:00,465][105620] Updated weights for policy 1, policy_version 1566798 (0.0007) [2023-12-27 02:47:00,533][105620] Updated weights for policy 1, policy_version 1566808 (0.0009) [2023-12-27 02:47:00,538][105692] Updated weights for policy 0, policy_version 1563375 (0.0006) [2023-12-27 02:47:00,589][105620] Updated weights for policy 1, policy_version 1566818 (0.0007) [2023-12-27 02:47:00,613][105692] Updated weights for policy 0, policy_version 1563385 (0.0007) [2023-12-27 02:47:00,669][105692] Updated weights for policy 0, policy_version 1563395 (0.0007) [2023-12-27 02:47:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.3, 300 sec: 19522.0). Total num frames: 801447936. Throughput: 0: 9425.2, 1: 9954.5. Samples: 801419344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:47:01,062][104569] Avg episode reward: [(0, '8716.606'), (1, '8724.363')] [2023-12-27 02:47:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001563400_400285696.pth... [2023-12-27 02:47:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001566824_401162240.pth... [2023-12-27 02:47:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001562312_400007168.pth [2023-12-27 02:47:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001565672_400867328.pth [2023-12-27 02:47:01,296][105620] Updated weights for policy 1, policy_version 1566828 (0.0009) [2023-12-27 02:47:01,334][105692] Updated weights for policy 0, policy_version 1563405 (0.0008) [2023-12-27 02:47:01,360][105620] Updated weights for policy 1, policy_version 1566838 (0.0007) [2023-12-27 02:47:01,399][105692] Updated weights for policy 0, policy_version 1563415 (0.0007) [2023-12-27 02:47:01,425][105620] Updated weights for policy 1, policy_version 1566848 (0.0009) [2023-12-27 02:47:01,460][105692] Updated weights for policy 0, policy_version 1563425 (0.0005) [2023-12-27 02:47:02,158][105692] Updated weights for policy 0, policy_version 1563435 (0.0006) [2023-12-27 02:47:02,187][105620] Updated weights for policy 1, policy_version 1566858 (0.0008) [2023-12-27 02:47:02,207][105692] Updated weights for policy 0, policy_version 1563445 (0.0008) [2023-12-27 02:47:02,242][105620] Updated weights for policy 1, policy_version 1566868 (0.0006) [2023-12-27 02:47:02,268][105692] Updated weights for policy 0, policy_version 1563455 (0.0007) [2023-12-27 02:47:02,302][105620] Updated weights for policy 1, policy_version 1566878 (0.0010) [2023-12-27 02:47:02,370][105620] Updated weights for policy 1, policy_version 1566888 (0.0011) [2023-12-27 02:47:02,988][105692] Updated weights for policy 0, policy_version 1563465 (0.0006) [2023-12-27 02:47:02,989][105620] Updated weights for policy 1, policy_version 1566898 (0.0010) [2023-12-27 02:47:03,034][105692] Updated weights for policy 0, policy_version 1563475 (0.0006) [2023-12-27 02:47:03,040][105620] Updated weights for policy 1, policy_version 1566908 (0.0010) [2023-12-27 02:47:03,076][105692] Updated weights for policy 0, policy_version 1563485 (0.0008) [2023-12-27 02:47:03,097][105620] Updated weights for policy 1, policy_version 1566918 (0.0010) [2023-12-27 02:47:03,121][105692] Updated weights for policy 0, policy_version 1563495 (0.0006) [2023-12-27 02:47:03,744][105620] Updated weights for policy 1, policy_version 1566928 (0.0010) [2023-12-27 02:47:03,805][105620] Updated weights for policy 1, policy_version 1566938 (0.0008) [2023-12-27 02:47:03,868][105620] Updated weights for policy 1, policy_version 1566948 (0.0011) [2023-12-27 02:47:03,940][105692] Updated weights for policy 0, policy_version 1563505 (0.0009) [2023-12-27 02:47:03,997][105692] Updated weights for policy 0, policy_version 1563515 (0.0010) [2023-12-27 02:47:04,061][105692] Updated weights for policy 0, policy_version 1563525 (0.0008) [2023-12-27 02:47:04,574][105620] Updated weights for policy 1, policy_version 1566958 (0.0011) [2023-12-27 02:47:04,634][105620] Updated weights for policy 1, policy_version 1566968 (0.0011) [2023-12-27 02:47:04,693][105620] Updated weights for policy 1, policy_version 1566978 (0.0011) [2023-12-27 02:47:04,822][105692] Updated weights for policy 0, policy_version 1563535 (0.0006) [2023-12-27 02:47:04,885][105692] Updated weights for policy 0, policy_version 1563545 (0.0005) [2023-12-27 02:47:04,942][105692] Updated weights for policy 0, policy_version 1563555 (0.0007) [2023-12-27 02:47:05,454][105620] Updated weights for policy 1, policy_version 1566988 (0.0011) [2023-12-27 02:47:05,514][105620] Updated weights for policy 1, policy_version 1566998 (0.0011) [2023-12-27 02:47:05,576][105620] Updated weights for policy 1, policy_version 1567008 (0.0010) [2023-12-27 02:47:05,602][105692] Updated weights for policy 0, policy_version 1563565 (0.0009) [2023-12-27 02:47:05,650][105692] Updated weights for policy 0, policy_version 1563575 (0.0007) [2023-12-27 02:47:05,694][105692] Updated weights for policy 0, policy_version 1563585 (0.0008) [2023-12-27 02:47:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 801546240. Throughput: 0: 9366.2, 1: 9933.2. Samples: 801534600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:47:06,062][104569] Avg episode reward: [(0, '8623.925'), (1, '8719.952')] [2023-12-27 02:47:06,319][105620] Updated weights for policy 1, policy_version 1567018 (0.0011) [2023-12-27 02:47:06,381][105620] Updated weights for policy 1, policy_version 1567028 (0.0011) [2023-12-27 02:47:06,441][105620] Updated weights for policy 1, policy_version 1567038 (0.0011) [2023-12-27 02:47:06,466][105692] Updated weights for policy 0, policy_version 1563595 (0.0007) [2023-12-27 02:47:06,505][105620] Updated weights for policy 1, policy_version 1567048 (0.0011) [2023-12-27 02:47:06,528][105692] Updated weights for policy 0, policy_version 1563605 (0.0006) [2023-12-27 02:47:06,588][105692] Updated weights for policy 0, policy_version 1563615 (0.0008) [2023-12-27 02:47:07,245][105620] Updated weights for policy 1, policy_version 1567058 (0.0007) [2023-12-27 02:47:07,302][105620] Updated weights for policy 1, policy_version 1567068 (0.0009) [2023-12-27 02:47:07,349][105692] Updated weights for policy 0, policy_version 1563625 (0.0008) [2023-12-27 02:47:07,350][105620] Updated weights for policy 1, policy_version 1567078 (0.0010) [2023-12-27 02:47:07,411][105692] Updated weights for policy 0, policy_version 1563635 (0.0008) [2023-12-27 02:47:07,468][105692] Updated weights for policy 0, policy_version 1563645 (0.0008) [2023-12-27 02:47:07,525][105692] Updated weights for policy 0, policy_version 1563655 (0.0008) [2023-12-27 02:47:08,015][105620] Updated weights for policy 1, policy_version 1567088 (0.0007) [2023-12-27 02:47:08,063][105620] Updated weights for policy 1, policy_version 1567098 (0.0010) [2023-12-27 02:47:08,116][105620] Updated weights for policy 1, policy_version 1567108 (0.0010) [2023-12-27 02:47:08,310][105692] Updated weights for policy 0, policy_version 1563665 (0.0009) [2023-12-27 02:47:08,377][105692] Updated weights for policy 0, policy_version 1563675 (0.0009) [2023-12-27 02:47:08,431][105692] Updated weights for policy 0, policy_version 1563685 (0.0010) [2023-12-27 02:47:08,755][105620] Updated weights for policy 1, policy_version 1567118 (0.0009) [2023-12-27 02:47:08,817][105620] Updated weights for policy 1, policy_version 1567128 (0.0010) [2023-12-27 02:47:08,868][105620] Updated weights for policy 1, policy_version 1567138 (0.0010) [2023-12-27 02:47:09,289][105692] Updated weights for policy 0, policy_version 1563695 (0.0010) [2023-12-27 02:47:09,341][105692] Updated weights for policy 0, policy_version 1563705 (0.0009) [2023-12-27 02:47:09,409][105692] Updated weights for policy 0, policy_version 1563715 (0.0010) [2023-12-27 02:47:09,545][105620] Updated weights for policy 1, policy_version 1567148 (0.0010) [2023-12-27 02:47:09,613][105620] Updated weights for policy 1, policy_version 1567158 (0.0006) [2023-12-27 02:47:09,666][105620] Updated weights for policy 1, policy_version 1567168 (0.0006) [2023-12-27 02:47:10,243][105620] Updated weights for policy 1, policy_version 1567178 (0.0007) [2023-12-27 02:47:10,306][105620] Updated weights for policy 1, policy_version 1567188 (0.0011) [2023-12-27 02:47:10,336][105692] Updated weights for policy 0, policy_version 1563725 (0.0007) [2023-12-27 02:47:10,362][105620] Updated weights for policy 1, policy_version 1567198 (0.0011) [2023-12-27 02:47:10,392][105692] Updated weights for policy 0, policy_version 1563735 (0.0005) [2023-12-27 02:47:10,422][105620] Updated weights for policy 1, policy_version 1567208 (0.0011) [2023-12-27 02:47:10,443][105692] Updated weights for policy 0, policy_version 1563745 (0.0008) [2023-12-27 02:47:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 801636352. Throughput: 0: 9346.7, 1: 10034.6. Samples: 801648920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:47:11,063][104569] Avg episode reward: [(0, '8353.699'), (1, '9082.001')] [2023-12-27 02:47:11,118][105620] Updated weights for policy 1, policy_version 1567218 (0.0010) [2023-12-27 02:47:11,180][105620] Updated weights for policy 1, policy_version 1567228 (0.0010) [2023-12-27 02:47:11,245][105620] Updated weights for policy 1, policy_version 1567238 (0.0011) [2023-12-27 02:47:11,267][105692] Updated weights for policy 0, policy_version 1563755 (0.0008) [2023-12-27 02:47:11,332][105692] Updated weights for policy 0, policy_version 1563765 (0.0006) [2023-12-27 02:47:11,398][105692] Updated weights for policy 0, policy_version 1563775 (0.0008) [2023-12-27 02:47:12,038][105620] Updated weights for policy 1, policy_version 1567248 (0.0011) [2023-12-27 02:47:12,102][105620] Updated weights for policy 1, policy_version 1567258 (0.0011) [2023-12-27 02:47:12,161][105620] Updated weights for policy 1, policy_version 1567268 (0.0011) [2023-12-27 02:47:12,183][105692] Updated weights for policy 0, policy_version 1563785 (0.0008) [2023-12-27 02:47:12,242][105692] Updated weights for policy 0, policy_version 1563795 (0.0008) [2023-12-27 02:47:12,299][105692] Updated weights for policy 0, policy_version 1563805 (0.0008) [2023-12-27 02:47:12,363][105692] Updated weights for policy 0, policy_version 1563815 (0.0009) [2023-12-27 02:47:12,814][105620] Updated weights for policy 1, policy_version 1567278 (0.0011) [2023-12-27 02:47:12,872][105620] Updated weights for policy 1, policy_version 1567288 (0.0006) [2023-12-27 02:47:12,934][105620] Updated weights for policy 1, policy_version 1567298 (0.0007) [2023-12-27 02:47:13,210][105692] Updated weights for policy 0, policy_version 1563825 (0.0008) [2023-12-27 02:47:13,268][105692] Updated weights for policy 0, policy_version 1563835 (0.0005) [2023-12-27 02:47:13,322][105692] Updated weights for policy 0, policy_version 1563845 (0.0005) [2023-12-27 02:47:13,646][105620] Updated weights for policy 1, policy_version 1567308 (0.0007) [2023-12-27 02:47:13,694][105620] Updated weights for policy 1, policy_version 1567318 (0.0009) [2023-12-27 02:47:13,755][105620] Updated weights for policy 1, policy_version 1567328 (0.0007) [2023-12-27 02:47:13,976][105692] Updated weights for policy 0, policy_version 1563855 (0.0009) [2023-12-27 02:47:14,034][105692] Updated weights for policy 0, policy_version 1563865 (0.0009) [2023-12-27 02:47:14,092][105692] Updated weights for policy 0, policy_version 1563875 (0.0009) [2023-12-27 02:47:14,372][105620] Updated weights for policy 1, policy_version 1567338 (0.0008) [2023-12-27 02:47:14,428][105620] Updated weights for policy 1, policy_version 1567348 (0.0008) [2023-12-27 02:47:14,474][105620] Updated weights for policy 1, policy_version 1567358 (0.0009) [2023-12-27 02:47:14,521][105620] Updated weights for policy 1, policy_version 1567368 (0.0009) [2023-12-27 02:47:14,908][105692] Updated weights for policy 0, policy_version 1563885 (0.0010) [2023-12-27 02:47:14,964][105692] Updated weights for policy 0, policy_version 1563895 (0.0009) [2023-12-27 02:47:15,022][105692] Updated weights for policy 0, policy_version 1563905 (0.0009) [2023-12-27 02:47:15,286][105620] Updated weights for policy 1, policy_version 1567378 (0.0007) [2023-12-27 02:47:15,351][105620] Updated weights for policy 1, policy_version 1567388 (0.0008) [2023-12-27 02:47:15,419][105620] Updated weights for policy 1, policy_version 1567398 (0.0008) [2023-12-27 02:47:15,764][105692] Updated weights for policy 0, policy_version 1563915 (0.0008) [2023-12-27 02:47:15,834][105692] Updated weights for policy 0, policy_version 1563925 (0.0005) [2023-12-27 02:47:15,886][105692] Updated weights for policy 0, policy_version 1563935 (0.0005) [2023-12-27 02:47:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 801734656. Throughput: 0: 9333.6, 1: 9987.8. Samples: 801705472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:47:16,062][104569] Avg episode reward: [(0, '8625.979'), (1, '8995.847')] [2023-12-27 02:47:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001563944_400424960.pth... [2023-12-27 02:47:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001562856_400146432.pth [2023-12-27 02:47:16,107][105620] Updated weights for policy 1, policy_version 1567408 (0.0008) [2023-12-27 02:47:16,164][105620] Updated weights for policy 1, policy_version 1567418 (0.0009) [2023-12-27 02:47:16,218][105620] Updated weights for policy 1, policy_version 1567428 (0.0010) [2023-12-27 02:47:16,238][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001567432_401317888.pth... [2023-12-27 02:47:16,243][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001566248_401014784.pth [2023-12-27 02:47:16,526][105692] Updated weights for policy 0, policy_version 1563945 (0.0006) [2023-12-27 02:47:16,594][105692] Updated weights for policy 0, policy_version 1563955 (0.0010) [2023-12-27 02:47:16,647][105692] Updated weights for policy 0, policy_version 1563965 (0.0010) [2023-12-27 02:47:16,699][105692] Updated weights for policy 0, policy_version 1563975 (0.0009) [2023-12-27 02:47:16,899][105620] Updated weights for policy 1, policy_version 1567438 (0.0006) [2023-12-27 02:47:16,964][105620] Updated weights for policy 1, policy_version 1567448 (0.0005) [2023-12-27 02:47:17,026][105620] Updated weights for policy 1, policy_version 1567458 (0.0008) [2023-12-27 02:47:17,504][105692] Updated weights for policy 0, policy_version 1563985 (0.0010) [2023-12-27 02:47:17,557][105692] Updated weights for policy 0, policy_version 1563995 (0.0011) [2023-12-27 02:47:17,622][105692] Updated weights for policy 0, policy_version 1564005 (0.0011) [2023-12-27 02:47:17,720][105620] Updated weights for policy 1, policy_version 1567468 (0.0010) [2023-12-27 02:47:17,785][105620] Updated weights for policy 1, policy_version 1567478 (0.0010) [2023-12-27 02:47:17,839][105620] Updated weights for policy 1, policy_version 1567488 (0.0010) [2023-12-27 02:47:18,279][105692] Updated weights for policy 0, policy_version 1564015 (0.0010) [2023-12-27 02:47:18,344][105692] Updated weights for policy 0, policy_version 1564026 (0.0010) [2023-12-27 02:47:18,410][105692] Updated weights for policy 0, policy_version 1564036 (0.0009) [2023-12-27 02:47:18,506][105620] Updated weights for policy 1, policy_version 1567498 (0.0010) [2023-12-27 02:47:18,574][105620] Updated weights for policy 1, policy_version 1567508 (0.0009) [2023-12-27 02:47:18,641][105620] Updated weights for policy 1, policy_version 1567518 (0.0009) [2023-12-27 02:47:18,707][105620] Updated weights for policy 1, policy_version 1567528 (0.0010) [2023-12-27 02:47:19,212][105692] Updated weights for policy 0, policy_version 1564046 (0.0010) [2023-12-27 02:47:19,280][105692] Updated weights for policy 0, policy_version 1564056 (0.0009) [2023-12-27 02:47:19,324][105620] Updated weights for policy 1, policy_version 1567538 (0.0008) [2023-12-27 02:47:19,348][105692] Updated weights for policy 0, policy_version 1564066 (0.0007) [2023-12-27 02:47:19,390][105620] Updated weights for policy 1, policy_version 1567548 (0.0010) [2023-12-27 02:47:19,445][105620] Updated weights for policy 1, policy_version 1567558 (0.0010) [2023-12-27 02:47:20,077][105692] Updated weights for policy 0, policy_version 1564076 (0.0007) [2023-12-27 02:47:20,141][105692] Updated weights for policy 0, policy_version 1564086 (0.0008) [2023-12-27 02:47:20,196][105692] Updated weights for policy 0, policy_version 1564096 (0.0007) [2023-12-27 02:47:20,218][105620] Updated weights for policy 1, policy_version 1567568 (0.0011) [2023-12-27 02:47:20,284][105620] Updated weights for policy 1, policy_version 1567578 (0.0010) [2023-12-27 02:47:20,330][105620] Updated weights for policy 1, policy_version 1567588 (0.0010) [2023-12-27 02:47:20,888][105692] Updated weights for policy 0, policy_version 1564106 (0.0006) [2023-12-27 02:47:20,957][105692] Updated weights for policy 0, policy_version 1564116 (0.0009) [2023-12-27 02:47:21,025][105692] Updated weights for policy 0, policy_version 1564126 (0.0008) [2023-12-27 02:47:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 801824768. Throughput: 0: 9287.8, 1: 9996.7. Samples: 801821596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:47:21,062][104569] Avg episode reward: [(0, '8715.972'), (1, '8448.849')] [2023-12-27 02:47:21,082][105620] Updated weights for policy 1, policy_version 1567598 (0.0009) [2023-12-27 02:47:21,094][105692] Updated weights for policy 0, policy_version 1564136 (0.0008) [2023-12-27 02:47:21,150][105620] Updated weights for policy 1, policy_version 1567608 (0.0010) [2023-12-27 02:47:21,201][105586] KL-divergence is very high: 100.8625 [2023-12-27 02:47:21,213][105620] Updated weights for policy 1, policy_version 1567618 (0.0011) [2023-12-27 02:47:21,856][105692] Updated weights for policy 0, policy_version 1564146 (0.0007) [2023-12-27 02:47:21,919][105692] Updated weights for policy 0, policy_version 1564156 (0.0008) [2023-12-27 02:47:21,975][105692] Updated weights for policy 0, policy_version 1564166 (0.0008) [2023-12-27 02:47:22,034][105620] Updated weights for policy 1, policy_version 1567628 (0.0010) [2023-12-27 02:47:22,094][105620] Updated weights for policy 1, policy_version 1567638 (0.0009) [2023-12-27 02:47:22,158][105620] Updated weights for policy 1, policy_version 1567648 (0.0010) [2023-12-27 02:47:22,759][105692] Updated weights for policy 0, policy_version 1564176 (0.0008) [2023-12-27 02:47:22,828][105692] Updated weights for policy 0, policy_version 1564186 (0.0008) [2023-12-27 02:47:22,892][105692] Updated weights for policy 0, policy_version 1564196 (0.0008) [2023-12-27 02:47:22,924][105620] Updated weights for policy 1, policy_version 1567658 (0.0009) [2023-12-27 02:47:22,974][105620] Updated weights for policy 1, policy_version 1567668 (0.0006) [2023-12-27 02:47:23,023][105620] Updated weights for policy 1, policy_version 1567678 (0.0010) [2023-12-27 02:47:23,082][105620] Updated weights for policy 1, policy_version 1567688 (0.0010) [2023-12-27 02:47:23,687][105692] Updated weights for policy 0, policy_version 1564206 (0.0008) [2023-12-27 02:47:23,759][105692] Updated weights for policy 0, policy_version 1564216 (0.0009) [2023-12-27 02:47:23,819][105692] Updated weights for policy 0, policy_version 1564226 (0.0007) [2023-12-27 02:47:23,840][105620] Updated weights for policy 1, policy_version 1567698 (0.0011) [2023-12-27 02:47:23,899][105620] Updated weights for policy 1, policy_version 1567708 (0.0010) [2023-12-27 02:47:23,948][105620] Updated weights for policy 1, policy_version 1567718 (0.0007) [2023-12-27 02:47:24,543][105692] Updated weights for policy 0, policy_version 1564236 (0.0006) [2023-12-27 02:47:24,603][105692] Updated weights for policy 0, policy_version 1564246 (0.0008) [2023-12-27 02:47:24,651][105692] Updated weights for policy 0, policy_version 1564256 (0.0010) [2023-12-27 02:47:24,662][105620] Updated weights for policy 1, policy_version 1567728 (0.0009) [2023-12-27 02:47:24,720][105620] Updated weights for policy 1, policy_version 1567738 (0.0010) [2023-12-27 02:47:24,785][105620] Updated weights for policy 1, policy_version 1567748 (0.0010) [2023-12-27 02:47:25,204][105692] Updated weights for policy 0, policy_version 1564266 (0.0009) [2023-12-27 02:47:25,268][105692] Updated weights for policy 0, policy_version 1564276 (0.0005) [2023-12-27 02:47:25,320][105692] Updated weights for policy 0, policy_version 1564286 (0.0005) [2023-12-27 02:47:25,369][105692] Updated weights for policy 0, policy_version 1564296 (0.0005) [2023-12-27 02:47:25,512][105620] Updated weights for policy 1, policy_version 1567758 (0.0009) [2023-12-27 02:47:25,560][105620] Updated weights for policy 1, policy_version 1567768 (0.0010) [2023-12-27 02:47:25,607][105620] Updated weights for policy 1, policy_version 1567778 (0.0010) [2023-12-27 02:47:26,001][105692] Updated weights for policy 0, policy_version 1564306 (0.0009) [2023-12-27 02:47:26,052][105692] Updated weights for policy 0, policy_version 1564316 (0.0009) [2023-12-27 02:47:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 801923072. Throughput: 0: 9385.3, 1: 9863.2. Samples: 801935524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:47:26,062][104569] Avg episode reward: [(0, '8532.491'), (1, '8446.791')] [2023-12-27 02:47:26,107][105692] Updated weights for policy 0, policy_version 1564326 (0.0010) [2023-12-27 02:47:26,386][105620] Updated weights for policy 1, policy_version 1567788 (0.0009) [2023-12-27 02:47:26,434][105620] Updated weights for policy 1, policy_version 1567798 (0.0008) [2023-12-27 02:47:26,483][105620] Updated weights for policy 1, policy_version 1567809 (0.0008) [2023-12-27 02:47:26,810][105692] Updated weights for policy 0, policy_version 1564336 (0.0009) [2023-12-27 02:47:26,864][105692] Updated weights for policy 0, policy_version 1564346 (0.0010) [2023-12-27 02:47:26,928][105692] Updated weights for policy 0, policy_version 1564356 (0.0010) [2023-12-27 02:47:27,262][105620] Updated weights for policy 1, policy_version 1567819 (0.0008) [2023-12-27 02:47:27,325][105620] Updated weights for policy 1, policy_version 1567829 (0.0010) [2023-12-27 02:47:27,376][105620] Updated weights for policy 1, policy_version 1567839 (0.0010) [2023-12-27 02:47:27,622][105692] Updated weights for policy 0, policy_version 1564366 (0.0008) [2023-12-27 02:47:27,686][105692] Updated weights for policy 0, policy_version 1564376 (0.0005) [2023-12-27 02:47:27,738][105692] Updated weights for policy 0, policy_version 1564386 (0.0005) [2023-12-27 02:47:28,007][105620] Updated weights for policy 1, policy_version 1567849 (0.0010) [2023-12-27 02:47:28,060][105620] Updated weights for policy 1, policy_version 1567860 (0.0008) [2023-12-27 02:47:28,113][105620] Updated weights for policy 1, policy_version 1567871 (0.0009) [2023-12-27 02:47:28,338][105692] Updated weights for policy 0, policy_version 1564396 (0.0008) [2023-12-27 02:47:28,399][105692] Updated weights for policy 0, policy_version 1564406 (0.0008) [2023-12-27 02:47:28,450][105692] Updated weights for policy 0, policy_version 1564416 (0.0009) [2023-12-27 02:47:28,844][105620] Updated weights for policy 1, policy_version 1567881 (0.0008) [2023-12-27 02:47:28,898][105620] Updated weights for policy 1, policy_version 1567891 (0.0010) [2023-12-27 02:47:28,957][105620] Updated weights for policy 1, policy_version 1567901 (0.0010) [2023-12-27 02:47:29,011][105620] Updated weights for policy 1, policy_version 1567911 (0.0009) [2023-12-27 02:47:29,136][105692] Updated weights for policy 0, policy_version 1564426 (0.0006) [2023-12-27 02:47:29,197][105692] Updated weights for policy 0, policy_version 1564436 (0.0009) [2023-12-27 02:47:29,259][105692] Updated weights for policy 0, policy_version 1564446 (0.0010) [2023-12-27 02:47:29,316][105692] Updated weights for policy 0, policy_version 1564456 (0.0009) [2023-12-27 02:47:29,708][105620] Updated weights for policy 1, policy_version 1567921 (0.0009) [2023-12-27 02:47:29,753][105620] Updated weights for policy 1, policy_version 1567931 (0.0005) [2023-12-27 02:47:29,802][105620] Updated weights for policy 1, policy_version 1567941 (0.0008) [2023-12-27 02:47:30,129][105692] Updated weights for policy 0, policy_version 1564466 (0.0009) [2023-12-27 02:47:30,190][105692] Updated weights for policy 0, policy_version 1564476 (0.0009) [2023-12-27 02:47:30,249][105692] Updated weights for policy 0, policy_version 1564486 (0.0010) [2023-12-27 02:47:30,484][105620] Updated weights for policy 1, policy_version 1567951 (0.0008) [2023-12-27 02:47:30,544][105620] Updated weights for policy 1, policy_version 1567961 (0.0009) [2023-12-27 02:47:30,595][105620] Updated weights for policy 1, policy_version 1567971 (0.0005) [2023-12-27 02:47:31,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19251.1, 300 sec: 19466.4). Total num frames: 802021376. Throughput: 0: 9453.7, 1: 9831.2. Samples: 801995448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:47:31,064][104569] Avg episode reward: [(0, '8807.796'), (1, '8810.480')] [2023-12-27 02:47:31,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001567976_401457152.pth... [2023-12-27 02:47:31,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001566824_401162240.pth [2023-12-27 02:47:31,087][105692] Updated weights for policy 0, policy_version 1564496 (0.0009) [2023-12-27 02:47:31,150][105692] Updated weights for policy 0, policy_version 1564506 (0.0009) [2023-12-27 02:47:31,180][105620] Updated weights for policy 1, policy_version 1567981 (0.0007) [2023-12-27 02:47:31,219][105692] Updated weights for policy 0, policy_version 1564516 (0.0007) [2023-12-27 02:47:31,238][105620] Updated weights for policy 1, policy_version 1567991 (0.0009) [2023-12-27 02:47:31,238][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001564520_400572416.pth... [2023-12-27 02:47:31,242][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001563400_400285696.pth [2023-12-27 02:47:31,302][105620] Updated weights for policy 1, policy_version 1568001 (0.0009) [2023-12-27 02:47:31,942][105692] Updated weights for policy 0, policy_version 1564526 (0.0008) [2023-12-27 02:47:32,006][105692] Updated weights for policy 0, policy_version 1564536 (0.0008) [2023-12-27 02:47:32,062][105620] Updated weights for policy 1, policy_version 1568011 (0.0009) [2023-12-27 02:47:32,068][105692] Updated weights for policy 0, policy_version 1564546 (0.0008) [2023-12-27 02:47:32,118][105620] Updated weights for policy 1, policy_version 1568021 (0.0006) [2023-12-27 02:47:32,180][105620] Updated weights for policy 1, policy_version 1568031 (0.0009) [2023-12-27 02:47:32,742][105692] Updated weights for policy 0, policy_version 1564556 (0.0007) [2023-12-27 02:47:32,801][105692] Updated weights for policy 0, policy_version 1564566 (0.0005) [2023-12-27 02:47:32,846][105620] Updated weights for policy 1, policy_version 1568041 (0.0009) [2023-12-27 02:47:32,868][105692] Updated weights for policy 0, policy_version 1564576 (0.0005) [2023-12-27 02:47:32,903][105620] Updated weights for policy 1, policy_version 1568051 (0.0009) [2023-12-27 02:47:32,960][105620] Updated weights for policy 1, policy_version 1568061 (0.0008) [2023-12-27 02:47:33,022][105620] Updated weights for policy 1, policy_version 1568071 (0.0008) [2023-12-27 02:47:33,378][105692] Updated weights for policy 0, policy_version 1564586 (0.0005) [2023-12-27 02:47:33,442][105692] Updated weights for policy 0, policy_version 1564596 (0.0005) [2023-12-27 02:47:33,497][105692] Updated weights for policy 0, policy_version 1564606 (0.0005) [2023-12-27 02:47:33,550][105692] Updated weights for policy 0, policy_version 1564616 (0.0005) [2023-12-27 02:47:33,593][105620] Updated weights for policy 1, policy_version 1568081 (0.0005) [2023-12-27 02:47:33,638][105620] Updated weights for policy 1, policy_version 1568091 (0.0005) [2023-12-27 02:47:33,683][105620] Updated weights for policy 1, policy_version 1568101 (0.0005) [2023-12-27 02:47:34,080][105692] Updated weights for policy 0, policy_version 1564626 (0.0006) [2023-12-27 02:47:34,133][105692] Updated weights for policy 0, policy_version 1564636 (0.0006) [2023-12-27 02:47:34,200][105692] Updated weights for policy 0, policy_version 1564646 (0.0008) [2023-12-27 02:47:34,234][105620] Updated weights for policy 1, policy_version 1568111 (0.0005) [2023-12-27 02:47:34,310][105620] Updated weights for policy 1, policy_version 1568121 (0.0006) [2023-12-27 02:47:34,378][105620] Updated weights for policy 1, policy_version 1568131 (0.0006) [2023-12-27 02:47:34,768][105692] Updated weights for policy 0, policy_version 1564656 (0.0006) [2023-12-27 02:47:34,831][105692] Updated weights for policy 0, policy_version 1564666 (0.0006) [2023-12-27 02:47:34,899][105692] Updated weights for policy 0, policy_version 1564676 (0.0006) [2023-12-27 02:47:34,970][105620] Updated weights for policy 1, policy_version 1568141 (0.0006) [2023-12-27 02:47:35,038][105620] Updated weights for policy 1, policy_version 1568151 (0.0006) [2023-12-27 02:47:35,103][105620] Updated weights for policy 1, policy_version 1568161 (0.0005) [2023-12-27 02:47:35,456][105692] Updated weights for policy 0, policy_version 1564686 (0.0009) [2023-12-27 02:47:35,511][105692] Updated weights for policy 0, policy_version 1564696 (0.0010) [2023-12-27 02:47:35,570][105692] Updated weights for policy 0, policy_version 1564706 (0.0010) [2023-12-27 02:47:35,705][105620] Updated weights for policy 1, policy_version 1568171 (0.0007) [2023-12-27 02:47:35,763][105620] Updated weights for policy 1, policy_version 1568181 (0.0010) [2023-12-27 02:47:35,827][105620] Updated weights for policy 1, policy_version 1568191 (0.0010) [2023-12-27 02:47:36,062][104569] Fps is (10 sec: 21299.0, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 802136064. Throughput: 0: 9564.8, 1: 9899.0. Samples: 802120600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:47:36,062][104569] Avg episode reward: [(0, '8808.523'), (1, '8808.728')] [2023-12-27 02:47:36,201][105692] Updated weights for policy 0, policy_version 1564716 (0.0010) [2023-12-27 02:47:36,262][105692] Updated weights for policy 0, policy_version 1564726 (0.0011) [2023-12-27 02:47:36,318][105692] Updated weights for policy 0, policy_version 1564736 (0.0011) [2023-12-27 02:47:36,558][105620] Updated weights for policy 1, policy_version 1568201 (0.0010) [2023-12-27 02:47:36,612][105620] Updated weights for policy 1, policy_version 1568211 (0.0010) [2023-12-27 02:47:36,671][105620] Updated weights for policy 1, policy_version 1568221 (0.0010) [2023-12-27 02:47:36,730][105620] Updated weights for policy 1, policy_version 1568231 (0.0011) [2023-12-27 02:47:37,075][105692] Updated weights for policy 0, policy_version 1564746 (0.0011) [2023-12-27 02:47:37,138][105692] Updated weights for policy 0, policy_version 1564756 (0.0010) [2023-12-27 02:47:37,193][105692] Updated weights for policy 0, policy_version 1564766 (0.0010) [2023-12-27 02:47:37,262][105692] Updated weights for policy 0, policy_version 1564776 (0.0010) [2023-12-27 02:47:37,462][105620] Updated weights for policy 1, policy_version 1568241 (0.0008) [2023-12-27 02:47:37,512][105620] Updated weights for policy 1, policy_version 1568251 (0.0009) [2023-12-27 02:47:37,565][105620] Updated weights for policy 1, policy_version 1568261 (0.0010) [2023-12-27 02:47:37,932][105692] Updated weights for policy 0, policy_version 1564786 (0.0005) [2023-12-27 02:47:37,978][105692] Updated weights for policy 0, policy_version 1564796 (0.0005) [2023-12-27 02:47:38,045][105692] Updated weights for policy 0, policy_version 1564806 (0.0005) [2023-12-27 02:47:38,272][105620] Updated weights for policy 1, policy_version 1568271 (0.0008) [2023-12-27 02:47:38,325][105620] Updated weights for policy 1, policy_version 1568281 (0.0008) [2023-12-27 02:47:38,390][105620] Updated weights for policy 1, policy_version 1568291 (0.0008) [2023-12-27 02:47:38,700][105692] Updated weights for policy 0, policy_version 1564816 (0.0010) [2023-12-27 02:47:38,762][105692] Updated weights for policy 0, policy_version 1564826 (0.0011) [2023-12-27 02:47:38,821][105692] Updated weights for policy 0, policy_version 1564836 (0.0010) [2023-12-27 02:47:39,050][105620] Updated weights for policy 1, policy_version 1568301 (0.0006) [2023-12-27 02:47:39,114][105620] Updated weights for policy 1, policy_version 1568311 (0.0010) [2023-12-27 02:47:39,181][105620] Updated weights for policy 1, policy_version 1568321 (0.0008) [2023-12-27 02:47:39,608][105692] Updated weights for policy 0, policy_version 1564846 (0.0010) [2023-12-27 02:47:39,662][105692] Updated weights for policy 0, policy_version 1564856 (0.0011) [2023-12-27 02:47:39,715][105692] Updated weights for policy 0, policy_version 1564866 (0.0011) [2023-12-27 02:47:39,881][105620] Updated weights for policy 1, policy_version 1568331 (0.0007) [2023-12-27 02:47:39,948][105620] Updated weights for policy 1, policy_version 1568341 (0.0007) [2023-12-27 02:47:40,021][105620] Updated weights for policy 1, policy_version 1568351 (0.0006) [2023-12-27 02:47:40,414][105692] Updated weights for policy 0, policy_version 1564876 (0.0011) [2023-12-27 02:47:40,480][105692] Updated weights for policy 0, policy_version 1564886 (0.0011) [2023-12-27 02:47:40,538][105692] Updated weights for policy 0, policy_version 1564896 (0.0007) [2023-12-27 02:47:40,669][105620] Updated weights for policy 1, policy_version 1568361 (0.0006) [2023-12-27 02:47:40,729][105620] Updated weights for policy 1, policy_version 1568371 (0.0008) [2023-12-27 02:47:40,790][105620] Updated weights for policy 1, policy_version 1568381 (0.0008) [2023-12-27 02:47:40,849][105620] Updated weights for policy 1, policy_version 1568391 (0.0008) [2023-12-27 02:47:41,062][104569] Fps is (10 sec: 21300.2, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 802234368. Throughput: 0: 9686.0, 1: 9883.1. Samples: 802241472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:47:41,062][104569] Avg episode reward: [(0, '8990.429'), (1, '8627.569')] [2023-12-27 02:47:41,325][105692] Updated weights for policy 0, policy_version 1564906 (0.0010) [2023-12-27 02:47:41,388][105692] Updated weights for policy 0, policy_version 1564916 (0.0011) [2023-12-27 02:47:41,446][105692] Updated weights for policy 0, policy_version 1564926 (0.0006) [2023-12-27 02:47:41,512][105692] Updated weights for policy 0, policy_version 1564936 (0.0009) [2023-12-27 02:47:41,659][105620] Updated weights for policy 1, policy_version 1568401 (0.0008) [2023-12-27 02:47:41,727][105620] Updated weights for policy 1, policy_version 1568411 (0.0009) [2023-12-27 02:47:41,792][105620] Updated weights for policy 1, policy_version 1568421 (0.0009) [2023-12-27 02:47:42,163][105692] Updated weights for policy 0, policy_version 1564946 (0.0006) [2023-12-27 02:47:42,223][105692] Updated weights for policy 0, policy_version 1564956 (0.0005) [2023-12-27 02:47:42,286][105692] Updated weights for policy 0, policy_version 1564966 (0.0007) [2023-12-27 02:47:42,613][105620] Updated weights for policy 1, policy_version 1568431 (0.0008) [2023-12-27 02:47:42,672][105620] Updated weights for policy 1, policy_version 1568441 (0.0008) [2023-12-27 02:47:42,731][105620] Updated weights for policy 1, policy_version 1568451 (0.0008) [2023-12-27 02:47:42,973][105692] Updated weights for policy 0, policy_version 1564976 (0.0010) [2023-12-27 02:47:43,022][105692] Updated weights for policy 0, policy_version 1564986 (0.0011) [2023-12-27 02:47:43,082][105692] Updated weights for policy 0, policy_version 1564996 (0.0011) [2023-12-27 02:47:43,418][105620] Updated weights for policy 1, policy_version 1568461 (0.0007) [2023-12-27 02:47:43,468][105620] Updated weights for policy 1, policy_version 1568471 (0.0005) [2023-12-27 02:47:43,523][105620] Updated weights for policy 1, policy_version 1568481 (0.0005) [2023-12-27 02:47:43,805][105692] Updated weights for policy 0, policy_version 1565006 (0.0009) [2023-12-27 02:47:43,864][105692] Updated weights for policy 0, policy_version 1565016 (0.0006) [2023-12-27 02:47:43,931][105692] Updated weights for policy 0, policy_version 1565026 (0.0005) [2023-12-27 02:47:44,190][105620] Updated weights for policy 1, policy_version 1568491 (0.0007) [2023-12-27 02:47:44,250][105620] Updated weights for policy 1, policy_version 1568501 (0.0009) [2023-12-27 02:47:44,315][105620] Updated weights for policy 1, policy_version 1568511 (0.0009) [2023-12-27 02:47:44,540][105692] Updated weights for policy 0, policy_version 1565036 (0.0010) [2023-12-27 02:47:44,606][105692] Updated weights for policy 0, policy_version 1565046 (0.0011) [2023-12-27 02:47:44,672][105692] Updated weights for policy 0, policy_version 1565056 (0.0011) [2023-12-27 02:47:45,119][105620] Updated weights for policy 1, policy_version 1568521 (0.0010) [2023-12-27 02:47:45,184][105620] Updated weights for policy 1, policy_version 1568531 (0.0009) [2023-12-27 02:47:45,250][105620] Updated weights for policy 1, policy_version 1568541 (0.0006) [2023-12-27 02:47:45,316][105620] Updated weights for policy 1, policy_version 1568551 (0.0006) [2023-12-27 02:47:45,433][105692] Updated weights for policy 0, policy_version 1565066 (0.0011) [2023-12-27 02:47:45,496][105692] Updated weights for policy 0, policy_version 1565076 (0.0011) [2023-12-27 02:47:45,550][105692] Updated weights for policy 0, policy_version 1565086 (0.0010) [2023-12-27 02:47:45,612][105692] Updated weights for policy 0, policy_version 1565096 (0.0008) [2023-12-27 02:47:45,944][105620] Updated weights for policy 1, policy_version 1568561 (0.0010) [2023-12-27 02:47:45,991][105620] Updated weights for policy 1, policy_version 1568571 (0.0005) [2023-12-27 02:47:46,040][105620] Updated weights for policy 1, policy_version 1568581 (0.0005) [2023-12-27 02:47:46,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 802332672. Throughput: 0: 9643.2, 1: 9905.9. Samples: 802299052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:47:46,062][104569] Avg episode reward: [(0, '8619.932'), (1, '8901.398')] [2023-12-27 02:47:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001565096_400719872.pth... [2023-12-27 02:47:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001568584_401612800.pth... [2023-12-27 02:47:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001563944_400424960.pth [2023-12-27 02:47:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001567432_401317888.pth [2023-12-27 02:47:46,267][105692] Updated weights for policy 0, policy_version 1565106 (0.0010) [2023-12-27 02:47:46,325][105692] Updated weights for policy 0, policy_version 1565116 (0.0010) [2023-12-27 02:47:46,374][105692] Updated weights for policy 0, policy_version 1565126 (0.0009) [2023-12-27 02:47:46,602][105620] Updated weights for policy 1, policy_version 1568591 (0.0006) [2023-12-27 02:47:46,657][105620] Updated weights for policy 1, policy_version 1568601 (0.0005) [2023-12-27 02:47:46,713][105620] Updated weights for policy 1, policy_version 1568611 (0.0005) [2023-12-27 02:47:47,132][105692] Updated weights for policy 0, policy_version 1565136 (0.0010) [2023-12-27 02:47:47,197][105692] Updated weights for policy 0, policy_version 1565146 (0.0010) [2023-12-27 02:47:47,258][105692] Updated weights for policy 0, policy_version 1565156 (0.0010) [2023-12-27 02:47:47,430][105620] Updated weights for policy 1, policy_version 1568621 (0.0008) [2023-12-27 02:47:47,490][105620] Updated weights for policy 1, policy_version 1568631 (0.0008) [2023-12-27 02:47:47,546][105620] Updated weights for policy 1, policy_version 1568641 (0.0008) [2023-12-27 02:47:47,986][105692] Updated weights for policy 0, policy_version 1565166 (0.0010) [2023-12-27 02:47:48,033][105692] Updated weights for policy 0, policy_version 1565176 (0.0008) [2023-12-27 02:47:48,080][105692] Updated weights for policy 0, policy_version 1565186 (0.0009) [2023-12-27 02:47:48,329][105620] Updated weights for policy 1, policy_version 1568651 (0.0008) [2023-12-27 02:47:48,398][105620] Updated weights for policy 1, policy_version 1568661 (0.0006) [2023-12-27 02:47:48,460][105620] Updated weights for policy 1, policy_version 1568671 (0.0009) [2023-12-27 02:47:48,761][105692] Updated weights for policy 0, policy_version 1565196 (0.0008) [2023-12-27 02:47:48,819][105692] Updated weights for policy 0, policy_version 1565206 (0.0010) [2023-12-27 02:47:48,867][105692] Updated weights for policy 0, policy_version 1565216 (0.0009) [2023-12-27 02:47:49,281][105620] Updated weights for policy 1, policy_version 1568681 (0.0010) [2023-12-27 02:47:49,344][105620] Updated weights for policy 1, policy_version 1568691 (0.0009) [2023-12-27 02:47:49,406][105620] Updated weights for policy 1, policy_version 1568701 (0.0008) [2023-12-27 02:47:49,466][105620] Updated weights for policy 1, policy_version 1568711 (0.0008) [2023-12-27 02:47:49,533][105692] Updated weights for policy 0, policy_version 1565226 (0.0009) [2023-12-27 02:47:49,597][105692] Updated weights for policy 0, policy_version 1565236 (0.0010) [2023-12-27 02:47:49,653][105692] Updated weights for policy 0, policy_version 1565246 (0.0009) [2023-12-27 02:47:49,704][105692] Updated weights for policy 0, policy_version 1565256 (0.0009) [2023-12-27 02:47:50,209][105620] Updated weights for policy 1, policy_version 1568721 (0.0008) [2023-12-27 02:47:50,265][105620] Updated weights for policy 1, policy_version 1568731 (0.0006) [2023-12-27 02:47:50,317][105620] Updated weights for policy 1, policy_version 1568741 (0.0005) [2023-12-27 02:47:50,509][105692] Updated weights for policy 0, policy_version 1565266 (0.0009) [2023-12-27 02:47:50,563][105692] Updated weights for policy 0, policy_version 1565276 (0.0008) [2023-12-27 02:47:50,632][105692] Updated weights for policy 0, policy_version 1565286 (0.0006) [2023-12-27 02:47:51,040][105620] Updated weights for policy 1, policy_version 1568751 (0.0007) [2023-12-27 02:47:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 802422784. Throughput: 0: 9695.6, 1: 9895.9. Samples: 802416216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:47:51,062][104569] Avg episode reward: [(0, '8528.622'), (1, '9264.789')] [2023-12-27 02:47:51,106][105620] Updated weights for policy 1, policy_version 1568761 (0.0009) [2023-12-27 02:47:51,164][105620] Updated weights for policy 1, policy_version 1568771 (0.0008) [2023-12-27 02:47:51,381][105692] Updated weights for policy 0, policy_version 1565296 (0.0009) [2023-12-27 02:47:51,444][105692] Updated weights for policy 0, policy_version 1565306 (0.0009) [2023-12-27 02:47:51,507][105692] Updated weights for policy 0, policy_version 1565316 (0.0010) [2023-12-27 02:47:51,905][105620] Updated weights for policy 1, policy_version 1568781 (0.0009) [2023-12-27 02:47:51,976][105620] Updated weights for policy 1, policy_version 1568791 (0.0010) [2023-12-27 02:47:52,035][105620] Updated weights for policy 1, policy_version 1568801 (0.0010) [2023-12-27 02:47:52,159][105692] Updated weights for policy 0, policy_version 1565326 (0.0009) [2023-12-27 02:47:52,211][105692] Updated weights for policy 0, policy_version 1565336 (0.0008) [2023-12-27 02:47:52,270][105692] Updated weights for policy 0, policy_version 1565346 (0.0009) [2023-12-27 02:47:52,782][105620] Updated weights for policy 1, policy_version 1568811 (0.0010) [2023-12-27 02:47:52,841][105620] Updated weights for policy 1, policy_version 1568821 (0.0010) [2023-12-27 02:47:52,887][105692] Updated weights for policy 0, policy_version 1565356 (0.0008) [2023-12-27 02:47:52,897][105620] Updated weights for policy 1, policy_version 1568831 (0.0010) [2023-12-27 02:47:52,939][105692] Updated weights for policy 0, policy_version 1565366 (0.0006) [2023-12-27 02:47:52,986][105692] Updated weights for policy 0, policy_version 1565376 (0.0006) [2023-12-27 02:47:53,612][105620] Updated weights for policy 1, policy_version 1568841 (0.0010) [2023-12-27 02:47:53,671][105620] Updated weights for policy 1, policy_version 1568851 (0.0007) [2023-12-27 02:47:53,706][105692] Updated weights for policy 0, policy_version 1565386 (0.0007) [2023-12-27 02:47:53,716][105620] Updated weights for policy 1, policy_version 1568861 (0.0009) [2023-12-27 02:47:53,759][105692] Updated weights for policy 0, policy_version 1565396 (0.0007) [2023-12-27 02:47:53,769][105620] Updated weights for policy 1, policy_version 1568871 (0.0006) [2023-12-27 02:47:53,814][105692] Updated weights for policy 0, policy_version 1565406 (0.0008) [2023-12-27 02:47:53,866][105692] Updated weights for policy 0, policy_version 1565416 (0.0009) [2023-12-27 02:47:54,458][105620] Updated weights for policy 1, policy_version 1568881 (0.0008) [2023-12-27 02:47:54,527][105620] Updated weights for policy 1, policy_version 1568891 (0.0008) [2023-12-27 02:47:54,588][105620] Updated weights for policy 1, policy_version 1568901 (0.0006) [2023-12-27 02:47:54,674][105692] Updated weights for policy 0, policy_version 1565426 (0.0009) [2023-12-27 02:47:54,729][105692] Updated weights for policy 0, policy_version 1565436 (0.0009) [2023-12-27 02:47:54,786][105692] Updated weights for policy 0, policy_version 1565446 (0.0009) [2023-12-27 02:47:55,310][105620] Updated weights for policy 1, policy_version 1568911 (0.0008) [2023-12-27 02:47:55,375][105620] Updated weights for policy 1, policy_version 1568921 (0.0009) [2023-12-27 02:47:55,430][105620] Updated weights for policy 1, policy_version 1568931 (0.0009) [2023-12-27 02:47:55,483][105692] Updated weights for policy 0, policy_version 1565456 (0.0007) [2023-12-27 02:47:55,538][105692] Updated weights for policy 0, policy_version 1565466 (0.0005) [2023-12-27 02:47:55,595][105692] Updated weights for policy 0, policy_version 1565476 (0.0009) [2023-12-27 02:47:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 802521088. Throughput: 0: 9790.4, 1: 9842.1. Samples: 802532380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:47:56,062][104569] Avg episode reward: [(0, '9079.926'), (1, '9356.963')] [2023-12-27 02:47:56,069][105620] Updated weights for policy 1, policy_version 1568941 (0.0009) [2023-12-27 02:47:56,122][105620] Updated weights for policy 1, policy_version 1568951 (0.0008) [2023-12-27 02:47:56,181][105620] Updated weights for policy 1, policy_version 1568961 (0.0007) [2023-12-27 02:47:56,194][105692] Updated weights for policy 0, policy_version 1565486 (0.0008) [2023-12-27 02:47:56,252][105692] Updated weights for policy 0, policy_version 1565496 (0.0006) [2023-12-27 02:47:56,304][105692] Updated weights for policy 0, policy_version 1565506 (0.0006) [2023-12-27 02:47:56,871][105692] Updated weights for policy 0, policy_version 1565516 (0.0007) [2023-12-27 02:47:56,910][105620] Updated weights for policy 1, policy_version 1568971 (0.0011) [2023-12-27 02:47:56,933][105692] Updated weights for policy 0, policy_version 1565526 (0.0007) [2023-12-27 02:47:56,977][105620] Updated weights for policy 1, policy_version 1568981 (0.0010) [2023-12-27 02:47:56,981][105692] Updated weights for policy 0, policy_version 1565536 (0.0005) [2023-12-27 02:47:57,035][105620] Updated weights for policy 1, policy_version 1568991 (0.0010) [2023-12-27 02:47:57,555][105692] Updated weights for policy 0, policy_version 1565546 (0.0005) [2023-12-27 02:47:57,603][105692] Updated weights for policy 0, policy_version 1565556 (0.0005) [2023-12-27 02:47:57,655][105692] Updated weights for policy 0, policy_version 1565566 (0.0005) [2023-12-27 02:47:57,710][105692] Updated weights for policy 0, policy_version 1565576 (0.0005) [2023-12-27 02:47:57,742][105620] Updated weights for policy 1, policy_version 1569001 (0.0010) [2023-12-27 02:47:57,793][105620] Updated weights for policy 1, policy_version 1569011 (0.0010) [2023-12-27 02:47:57,841][105620] Updated weights for policy 1, policy_version 1569021 (0.0010) [2023-12-27 02:47:57,889][105620] Updated weights for policy 1, policy_version 1569031 (0.0010) [2023-12-27 02:47:58,340][105692] Updated weights for policy 0, policy_version 1565586 (0.0008) [2023-12-27 02:47:58,401][105692] Updated weights for policy 0, policy_version 1565596 (0.0009) [2023-12-27 02:47:58,468][105692] Updated weights for policy 0, policy_version 1565606 (0.0009) [2023-12-27 02:47:58,609][105620] Updated weights for policy 1, policy_version 1569041 (0.0010) [2023-12-27 02:47:58,674][105620] Updated weights for policy 1, policy_version 1569051 (0.0011) [2023-12-27 02:47:58,742][105620] Updated weights for policy 1, policy_version 1569061 (0.0010) [2023-12-27 02:47:59,290][105692] Updated weights for policy 0, policy_version 1565616 (0.0011) [2023-12-27 02:47:59,350][105692] Updated weights for policy 0, policy_version 1565626 (0.0011) [2023-12-27 02:47:59,418][105692] Updated weights for policy 0, policy_version 1565636 (0.0009) [2023-12-27 02:47:59,530][105620] Updated weights for policy 1, policy_version 1569071 (0.0007) [2023-12-27 02:47:59,585][105620] Updated weights for policy 1, policy_version 1569081 (0.0008) [2023-12-27 02:47:59,649][105620] Updated weights for policy 1, policy_version 1569091 (0.0005) [2023-12-27 02:48:00,174][105692] Updated weights for policy 0, policy_version 1565646 (0.0008) [2023-12-27 02:48:00,230][105620] Updated weights for policy 1, policy_version 1569101 (0.0007) [2023-12-27 02:48:00,232][105692] Updated weights for policy 0, policy_version 1565656 (0.0008) [2023-12-27 02:48:00,278][105692] Updated weights for policy 0, policy_version 1565666 (0.0008) [2023-12-27 02:48:00,281][105620] Updated weights for policy 1, policy_version 1569111 (0.0007) [2023-12-27 02:48:00,331][105620] Updated weights for policy 1, policy_version 1569121 (0.0007) [2023-12-27 02:48:00,982][105620] Updated weights for policy 1, policy_version 1569131 (0.0009) [2023-12-27 02:48:01,043][105620] Updated weights for policy 1, policy_version 1569141 (0.0009) [2023-12-27 02:48:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 802619392. Throughput: 0: 9945.8, 1: 9820.4. Samples: 802594948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:48:01,062][104569] Avg episode reward: [(0, '9171.327'), (1, '9083.183')] [2023-12-27 02:48:01,068][105692] Updated weights for policy 0, policy_version 1565676 (0.0007) [2023-12-27 02:48:01,098][105620] Updated weights for policy 1, policy_version 1569151 (0.0007) [2023-12-27 02:48:01,125][105692] Updated weights for policy 0, policy_version 1565686 (0.0008) [2023-12-27 02:48:01,156][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001569160_401760256.pth... [2023-12-27 02:48:01,160][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001567976_401457152.pth [2023-12-27 02:48:01,189][105692] Updated weights for policy 0, policy_version 1565696 (0.0009) [2023-12-27 02:48:01,232][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001565704_400875520.pth... [2023-12-27 02:48:01,236][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001564520_400572416.pth [2023-12-27 02:48:01,786][105620] Updated weights for policy 1, policy_version 1569161 (0.0008) [2023-12-27 02:48:01,842][105620] Updated weights for policy 1, policy_version 1569171 (0.0008) [2023-12-27 02:48:01,899][105620] Updated weights for policy 1, policy_version 1569181 (0.0009) [2023-12-27 02:48:01,957][105620] Updated weights for policy 1, policy_version 1569191 (0.0009) [2023-12-27 02:48:01,992][105692] Updated weights for policy 0, policy_version 1565706 (0.0008) [2023-12-27 02:48:02,053][105692] Updated weights for policy 0, policy_version 1565716 (0.0007) [2023-12-27 02:48:02,110][105692] Updated weights for policy 0, policy_version 1565726 (0.0006) [2023-12-27 02:48:02,155][105692] Updated weights for policy 0, policy_version 1565736 (0.0006) [2023-12-27 02:48:02,722][105620] Updated weights for policy 1, policy_version 1569201 (0.0009) [2023-12-27 02:48:02,776][105620] Updated weights for policy 1, policy_version 1569211 (0.0009) [2023-12-27 02:48:02,822][105620] Updated weights for policy 1, policy_version 1569221 (0.0007) [2023-12-27 02:48:02,837][105692] Updated weights for policy 0, policy_version 1565746 (0.0007) [2023-12-27 02:48:02,884][105692] Updated weights for policy 0, policy_version 1565756 (0.0009) [2023-12-27 02:48:02,931][105692] Updated weights for policy 0, policy_version 1565766 (0.0009) [2023-12-27 02:48:03,583][105620] Updated weights for policy 1, policy_version 1569231 (0.0007) [2023-12-27 02:48:03,628][105692] Updated weights for policy 0, policy_version 1565776 (0.0009) [2023-12-27 02:48:03,631][105620] Updated weights for policy 1, policy_version 1569241 (0.0005) [2023-12-27 02:48:03,650][105586] KL-divergence is very high: 126.3214 [2023-12-27 02:48:03,673][105692] Updated weights for policy 0, policy_version 1565786 (0.0007) [2023-12-27 02:48:03,679][105620] Updated weights for policy 1, policy_version 1569251 (0.0006) [2023-12-27 02:48:03,688][105586] KL-divergence is very high: 175.7878 [2023-12-27 02:48:03,728][105692] Updated weights for policy 0, policy_version 1565796 (0.0007) [2023-12-27 02:48:04,411][105620] Updated weights for policy 1, policy_version 1569261 (0.0008) [2023-12-27 02:48:04,462][105620] Updated weights for policy 1, policy_version 1569271 (0.0009) [2023-12-27 02:48:04,509][105620] Updated weights for policy 1, policy_version 1569281 (0.0008) [2023-12-27 02:48:04,520][105692] Updated weights for policy 0, policy_version 1565806 (0.0007) [2023-12-27 02:48:04,581][105692] Updated weights for policy 0, policy_version 1565816 (0.0007) [2023-12-27 02:48:04,644][105692] Updated weights for policy 0, policy_version 1565826 (0.0010) [2023-12-27 02:48:05,280][105620] Updated weights for policy 1, policy_version 1569291 (0.0008) [2023-12-27 02:48:05,326][105620] Updated weights for policy 1, policy_version 1569301 (0.0008) [2023-12-27 02:48:05,372][105620] Updated weights for policy 1, policy_version 1569311 (0.0009) [2023-12-27 02:48:05,400][105692] Updated weights for policy 0, policy_version 1565836 (0.0008) [2023-12-27 02:48:05,460][105692] Updated weights for policy 0, policy_version 1565846 (0.0008) [2023-12-27 02:48:05,518][105692] Updated weights for policy 0, policy_version 1565856 (0.0009) [2023-12-27 02:48:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 802717696. Throughput: 0: 9947.1, 1: 9803.8. Samples: 802710388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:48:06,062][104569] Avg episode reward: [(0, '8530.725'), (1, '8814.690')] [2023-12-27 02:48:06,140][105620] Updated weights for policy 1, policy_version 1569321 (0.0008) [2023-12-27 02:48:06,204][105620] Updated weights for policy 1, policy_version 1569331 (0.0009) [2023-12-27 02:48:06,268][105692] Updated weights for policy 0, policy_version 1565866 (0.0008) [2023-12-27 02:48:06,269][105620] Updated weights for policy 1, policy_version 1569341 (0.0009) [2023-12-27 02:48:06,329][105692] Updated weights for policy 0, policy_version 1565876 (0.0009) [2023-12-27 02:48:06,331][105620] Updated weights for policy 1, policy_version 1569351 (0.0009) [2023-12-27 02:48:06,384][105692] Updated weights for policy 0, policy_version 1565886 (0.0008) [2023-12-27 02:48:06,443][105692] Updated weights for policy 0, policy_version 1565896 (0.0009) [2023-12-27 02:48:07,095][105620] Updated weights for policy 1, policy_version 1569361 (0.0009) [2023-12-27 02:48:07,151][105620] Updated weights for policy 1, policy_version 1569371 (0.0009) [2023-12-27 02:48:07,202][105620] Updated weights for policy 1, policy_version 1569381 (0.0007) [2023-12-27 02:48:07,223][105692] Updated weights for policy 0, policy_version 1565906 (0.0007) [2023-12-27 02:48:07,273][105692] Updated weights for policy 0, policy_version 1565916 (0.0008) [2023-12-27 02:48:07,329][105692] Updated weights for policy 0, policy_version 1565926 (0.0009) [2023-12-27 02:48:07,846][105620] Updated weights for policy 1, policy_version 1569391 (0.0005) [2023-12-27 02:48:07,907][105620] Updated weights for policy 1, policy_version 1569401 (0.0005) [2023-12-27 02:48:07,961][105620] Updated weights for policy 1, policy_version 1569411 (0.0005) [2023-12-27 02:48:08,219][105692] Updated weights for policy 0, policy_version 1565936 (0.0009) [2023-12-27 02:48:08,270][105692] Updated weights for policy 0, policy_version 1565947 (0.0008) [2023-12-27 02:48:08,327][105692] Updated weights for policy 0, policy_version 1565957 (0.0006) [2023-12-27 02:48:08,591][105620] Updated weights for policy 1, policy_version 1569421 (0.0008) [2023-12-27 02:48:08,653][105620] Updated weights for policy 1, policy_version 1569431 (0.0009) [2023-12-27 02:48:08,710][105620] Updated weights for policy 1, policy_version 1569441 (0.0009) [2023-12-27 02:48:09,007][105692] Updated weights for policy 0, policy_version 1565967 (0.0009) [2023-12-27 02:48:09,058][105692] Updated weights for policy 0, policy_version 1565977 (0.0008) [2023-12-27 02:48:09,114][105692] Updated weights for policy 0, policy_version 1565987 (0.0005) [2023-12-27 02:48:09,506][105620] Updated weights for policy 1, policy_version 1569451 (0.0009) [2023-12-27 02:48:09,569][105620] Updated weights for policy 1, policy_version 1569461 (0.0009) [2023-12-27 02:48:09,631][105620] Updated weights for policy 1, policy_version 1569471 (0.0009) [2023-12-27 02:48:09,859][105692] Updated weights for policy 0, policy_version 1565997 (0.0007) [2023-12-27 02:48:09,922][105692] Updated weights for policy 0, policy_version 1566007 (0.0010) [2023-12-27 02:48:09,989][105692] Updated weights for policy 0, policy_version 1566017 (0.0008) [2023-12-27 02:48:10,406][105620] Updated weights for policy 1, policy_version 1569481 (0.0009) [2023-12-27 02:48:10,459][105620] Updated weights for policy 1, policy_version 1569491 (0.0009) [2023-12-27 02:48:10,510][105620] Updated weights for policy 1, policy_version 1569501 (0.0009) [2023-12-27 02:48:10,572][105620] Updated weights for policy 1, policy_version 1569511 (0.0008) [2023-12-27 02:48:10,756][105692] Updated weights for policy 0, policy_version 1566027 (0.0009) [2023-12-27 02:48:10,812][105692] Updated weights for policy 0, policy_version 1566037 (0.0009) [2023-12-27 02:48:10,871][105692] Updated weights for policy 0, policy_version 1566047 (0.0009) [2023-12-27 02:48:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 802816000. Throughput: 0: 9869.6, 1: 9838.8. Samples: 802822404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:48:11,062][104569] Avg episode reward: [(0, '8170.368'), (1, '8905.508')] [2023-12-27 02:48:11,338][105620] Updated weights for policy 1, policy_version 1569521 (0.0009) [2023-12-27 02:48:11,405][105620] Updated weights for policy 1, policy_version 1569531 (0.0009) [2023-12-27 02:48:11,466][105620] Updated weights for policy 1, policy_version 1569541 (0.0010) [2023-12-27 02:48:11,693][105692] Updated weights for policy 0, policy_version 1566057 (0.0009) [2023-12-27 02:48:11,760][105692] Updated weights for policy 0, policy_version 1566067 (0.0009) [2023-12-27 02:48:11,812][105692] Updated weights for policy 0, policy_version 1566077 (0.0008) [2023-12-27 02:48:11,867][105692] Updated weights for policy 0, policy_version 1566087 (0.0009) [2023-12-27 02:48:12,208][105620] Updated weights for policy 1, policy_version 1569551 (0.0009) [2023-12-27 02:48:12,280][105620] Updated weights for policy 1, policy_version 1569561 (0.0009) [2023-12-27 02:48:12,347][105620] Updated weights for policy 1, policy_version 1569571 (0.0008) [2023-12-27 02:48:12,608][105692] Updated weights for policy 0, policy_version 1566097 (0.0006) [2023-12-27 02:48:12,667][105692] Updated weights for policy 0, policy_version 1566107 (0.0009) [2023-12-27 02:48:12,714][105692] Updated weights for policy 0, policy_version 1566117 (0.0009) [2023-12-27 02:48:13,013][105620] Updated weights for policy 1, policy_version 1569581 (0.0008) [2023-12-27 02:48:13,075][105620] Updated weights for policy 1, policy_version 1569591 (0.0008) [2023-12-27 02:48:13,137][105620] Updated weights for policy 1, policy_version 1569601 (0.0008) [2023-12-27 02:48:13,491][105692] Updated weights for policy 0, policy_version 1566127 (0.0010) [2023-12-27 02:48:13,550][105692] Updated weights for policy 0, policy_version 1566137 (0.0010) [2023-12-27 02:48:13,594][105692] Updated weights for policy 0, policy_version 1566147 (0.0009) [2023-12-27 02:48:13,773][105620] Updated weights for policy 1, policy_version 1569611 (0.0008) [2023-12-27 02:48:13,833][105620] Updated weights for policy 1, policy_version 1569621 (0.0008) [2023-12-27 02:48:13,895][105620] Updated weights for policy 1, policy_version 1569631 (0.0005) [2023-12-27 02:48:14,350][105692] Updated weights for policy 0, policy_version 1566157 (0.0010) [2023-12-27 02:48:14,404][105692] Updated weights for policy 0, policy_version 1566167 (0.0010) [2023-12-27 02:48:14,455][105692] Updated weights for policy 0, policy_version 1566177 (0.0010) [2023-12-27 02:48:14,502][105620] Updated weights for policy 1, policy_version 1569641 (0.0005) [2023-12-27 02:48:14,561][105620] Updated weights for policy 1, policy_version 1569651 (0.0007) [2023-12-27 02:48:14,615][105620] Updated weights for policy 1, policy_version 1569661 (0.0007) [2023-12-27 02:48:14,671][105620] Updated weights for policy 1, policy_version 1569671 (0.0009) [2023-12-27 02:48:15,119][105692] Updated weights for policy 0, policy_version 1566187 (0.0010) [2023-12-27 02:48:15,169][105692] Updated weights for policy 0, policy_version 1566197 (0.0010) [2023-12-27 02:48:15,237][105692] Updated weights for policy 0, policy_version 1566207 (0.0007) [2023-12-27 02:48:15,406][105620] Updated weights for policy 1, policy_version 1569681 (0.0010) [2023-12-27 02:48:15,472][105620] Updated weights for policy 1, policy_version 1569691 (0.0010) [2023-12-27 02:48:15,524][105620] Updated weights for policy 1, policy_version 1569701 (0.0010) [2023-12-27 02:48:15,893][105692] Updated weights for policy 0, policy_version 1566217 (0.0005) [2023-12-27 02:48:15,950][105692] Updated weights for policy 0, policy_version 1566227 (0.0005) [2023-12-27 02:48:16,016][105692] Updated weights for policy 0, policy_version 1566237 (0.0007) [2023-12-27 02:48:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 802906112. Throughput: 0: 9803.9, 1: 9842.5. Samples: 802879528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:48:16,062][104569] Avg episode reward: [(0, '8533.348'), (1, '9265.162')] [2023-12-27 02:48:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001569704_401899520.pth... [2023-12-27 02:48:16,072][105692] Updated weights for policy 0, policy_version 1566247 (0.0007) [2023-12-27 02:48:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001568584_401612800.pth [2023-12-27 02:48:16,075][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001566248_401014784.pth... [2023-12-27 02:48:16,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001565096_400719872.pth [2023-12-27 02:48:16,193][105620] Updated weights for policy 1, policy_version 1569711 (0.0011) [2023-12-27 02:48:16,262][105620] Updated weights for policy 1, policy_version 1569721 (0.0011) [2023-12-27 02:48:16,322][105620] Updated weights for policy 1, policy_version 1569731 (0.0008) [2023-12-27 02:48:16,606][105692] Updated weights for policy 0, policy_version 1566257 (0.0005) [2023-12-27 02:48:16,660][105692] Updated weights for policy 0, policy_version 1566267 (0.0005) [2023-12-27 02:48:16,714][105692] Updated weights for policy 0, policy_version 1566277 (0.0005) [2023-12-27 02:48:16,933][105620] Updated weights for policy 1, policy_version 1569741 (0.0008) [2023-12-27 02:48:16,995][105620] Updated weights for policy 1, policy_version 1569751 (0.0010) [2023-12-27 02:48:17,060][105620] Updated weights for policy 1, policy_version 1569761 (0.0010) [2023-12-27 02:48:17,261][105692] Updated weights for policy 0, policy_version 1566287 (0.0009) [2023-12-27 02:48:17,306][105692] Updated weights for policy 0, policy_version 1566297 (0.0010) [2023-12-27 02:48:17,360][105692] Updated weights for policy 0, policy_version 1566307 (0.0010) [2023-12-27 02:48:17,758][105620] Updated weights for policy 1, policy_version 1569771 (0.0010) [2023-12-27 02:48:17,814][105620] Updated weights for policy 1, policy_version 1569781 (0.0010) [2023-12-27 02:48:17,868][105620] Updated weights for policy 1, policy_version 1569791 (0.0010) [2023-12-27 02:48:18,140][105692] Updated weights for policy 0, policy_version 1566317 (0.0010) [2023-12-27 02:48:18,196][105692] Updated weights for policy 0, policy_version 1566327 (0.0011) [2023-12-27 02:48:18,252][105692] Updated weights for policy 0, policy_version 1566337 (0.0011) [2023-12-27 02:48:18,613][105620] Updated weights for policy 1, policy_version 1569801 (0.0010) [2023-12-27 02:48:18,677][105620] Updated weights for policy 1, policy_version 1569811 (0.0005) [2023-12-27 02:48:18,744][105620] Updated weights for policy 1, policy_version 1569821 (0.0005) [2023-12-27 02:48:18,802][105620] Updated weights for policy 1, policy_version 1569831 (0.0008) [2023-12-27 02:48:18,931][105692] Updated weights for policy 0, policy_version 1566347 (0.0009) [2023-12-27 02:48:18,990][105692] Updated weights for policy 0, policy_version 1566357 (0.0006) [2023-12-27 02:48:19,052][105692] Updated weights for policy 0, policy_version 1566367 (0.0005) [2023-12-27 02:48:19,518][105620] Updated weights for policy 1, policy_version 1569841 (0.0009) [2023-12-27 02:48:19,574][105620] Updated weights for policy 1, policy_version 1569851 (0.0008) [2023-12-27 02:48:19,632][105620] Updated weights for policy 1, policy_version 1569861 (0.0009) [2023-12-27 02:48:19,760][105692] Updated weights for policy 0, policy_version 1566377 (0.0009) [2023-12-27 02:48:19,824][105692] Updated weights for policy 0, policy_version 1566387 (0.0009) [2023-12-27 02:48:19,883][105692] Updated weights for policy 0, policy_version 1566397 (0.0008) [2023-12-27 02:48:19,954][105692] Updated weights for policy 0, policy_version 1566407 (0.0009) [2023-12-27 02:48:20,377][105620] Updated weights for policy 1, policy_version 1569871 (0.0008) [2023-12-27 02:48:20,448][105620] Updated weights for policy 1, policy_version 1569881 (0.0008) [2023-12-27 02:48:20,520][105620] Updated weights for policy 1, policy_version 1569891 (0.0008) [2023-12-27 02:48:20,668][105692] Updated weights for policy 0, policy_version 1566417 (0.0009) [2023-12-27 02:48:20,736][105692] Updated weights for policy 0, policy_version 1566427 (0.0009) [2023-12-27 02:48:20,795][105692] Updated weights for policy 0, policy_version 1566437 (0.0007) [2023-12-27 02:48:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 803012608. Throughput: 0: 9845.2, 1: 9733.3. Samples: 803001628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:48:21,063][104569] Avg episode reward: [(0, '8529.371'), (1, '9266.979')] [2023-12-27 02:48:21,231][105620] Updated weights for policy 1, policy_version 1569901 (0.0009) [2023-12-27 02:48:21,300][105620] Updated weights for policy 1, policy_version 1569911 (0.0010) [2023-12-27 02:48:21,364][105620] Updated weights for policy 1, policy_version 1569921 (0.0009) [2023-12-27 02:48:21,544][105692] Updated weights for policy 0, policy_version 1566447 (0.0006) [2023-12-27 02:48:21,603][105692] Updated weights for policy 0, policy_version 1566457 (0.0006) [2023-12-27 02:48:21,671][105692] Updated weights for policy 0, policy_version 1566467 (0.0009) [2023-12-27 02:48:22,179][105620] Updated weights for policy 1, policy_version 1569931 (0.0009) [2023-12-27 02:48:22,244][105620] Updated weights for policy 1, policy_version 1569941 (0.0008) [2023-12-27 02:48:22,302][105692] Updated weights for policy 0, policy_version 1566477 (0.0007) [2023-12-27 02:48:22,312][105620] Updated weights for policy 1, policy_version 1569951 (0.0008) [2023-12-27 02:48:22,364][105692] Updated weights for policy 0, policy_version 1566487 (0.0007) [2023-12-27 02:48:22,420][105692] Updated weights for policy 0, policy_version 1566497 (0.0008) [2023-12-27 02:48:23,068][105620] Updated weights for policy 1, policy_version 1569961 (0.0008) [2023-12-27 02:48:23,126][105620] Updated weights for policy 1, policy_version 1569971 (0.0009) [2023-12-27 02:48:23,177][105620] Updated weights for policy 1, policy_version 1569981 (0.0009) [2023-12-27 02:48:23,191][105692] Updated weights for policy 0, policy_version 1566507 (0.0008) [2023-12-27 02:48:23,231][105620] Updated weights for policy 1, policy_version 1569991 (0.0006) [2023-12-27 02:48:23,251][105692] Updated weights for policy 0, policy_version 1566517 (0.0008) [2023-12-27 02:48:23,311][105692] Updated weights for policy 0, policy_version 1566527 (0.0009) [2023-12-27 02:48:24,009][105692] Updated weights for policy 0, policy_version 1566537 (0.0009) [2023-12-27 02:48:24,014][105620] Updated weights for policy 1, policy_version 1570001 (0.0009) [2023-12-27 02:48:24,060][105620] Updated weights for policy 1, policy_version 1570011 (0.0007) [2023-12-27 02:48:24,070][105692] Updated weights for policy 0, policy_version 1566547 (0.0010) [2023-12-27 02:48:24,112][105620] Updated weights for policy 1, policy_version 1570021 (0.0008) [2023-12-27 02:48:24,135][105692] Updated weights for policy 0, policy_version 1566557 (0.0010) [2023-12-27 02:48:24,187][105692] Updated weights for policy 0, policy_version 1566567 (0.0010) [2023-12-27 02:48:24,879][105692] Updated weights for policy 0, policy_version 1566577 (0.0010) [2023-12-27 02:48:24,885][105620] Updated weights for policy 1, policy_version 1570031 (0.0007) [2023-12-27 02:48:24,941][105692] Updated weights for policy 0, policy_version 1566587 (0.0011) [2023-12-27 02:48:24,946][105620] Updated weights for policy 1, policy_version 1570041 (0.0009) [2023-12-27 02:48:25,004][105692] Updated weights for policy 0, policy_version 1566597 (0.0011) [2023-12-27 02:48:25,007][105620] Updated weights for policy 1, policy_version 1570051 (0.0005) [2023-12-27 02:48:25,672][105692] Updated weights for policy 0, policy_version 1566607 (0.0011) [2023-12-27 02:48:25,733][105620] Updated weights for policy 1, policy_version 1570061 (0.0007) [2023-12-27 02:48:25,737][105692] Updated weights for policy 0, policy_version 1566617 (0.0010) [2023-12-27 02:48:25,791][105620] Updated weights for policy 1, policy_version 1570071 (0.0007) [2023-12-27 02:48:25,796][105692] Updated weights for policy 0, policy_version 1566627 (0.0010) [2023-12-27 02:48:25,851][105620] Updated weights for policy 1, policy_version 1570081 (0.0007) [2023-12-27 02:48:26,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 803110912. Throughput: 0: 9782.7, 1: 9636.8. Samples: 803115352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:48:26,063][104569] Avg episode reward: [(0, '8710.697'), (1, '8816.414')] [2023-12-27 02:48:26,442][105620] Updated weights for policy 1, policy_version 1570091 (0.0007) [2023-12-27 02:48:26,453][105692] Updated weights for policy 0, policy_version 1566637 (0.0011) [2023-12-27 02:48:26,494][105620] Updated weights for policy 1, policy_version 1570101 (0.0005) [2023-12-27 02:48:26,497][105692] Updated weights for policy 0, policy_version 1566647 (0.0010) [2023-12-27 02:48:26,545][105692] Updated weights for policy 0, policy_version 1566657 (0.0010) [2023-12-27 02:48:26,555][105620] Updated weights for policy 1, policy_version 1570111 (0.0005) [2023-12-27 02:48:27,120][105692] Updated weights for policy 0, policy_version 1566667 (0.0009) [2023-12-27 02:48:27,154][105620] Updated weights for policy 1, policy_version 1570121 (0.0005) [2023-12-27 02:48:27,178][105692] Updated weights for policy 0, policy_version 1566677 (0.0006) [2023-12-27 02:48:27,211][105620] Updated weights for policy 1, policy_version 1570131 (0.0006) [2023-12-27 02:48:27,242][105692] Updated weights for policy 0, policy_version 1566687 (0.0008) [2023-12-27 02:48:27,264][105620] Updated weights for policy 1, policy_version 1570141 (0.0005) [2023-12-27 02:48:27,325][105620] Updated weights for policy 1, policy_version 1570151 (0.0006) [2023-12-27 02:48:27,805][105692] Updated weights for policy 0, policy_version 1566697 (0.0009) [2023-12-27 02:48:27,855][105692] Updated weights for policy 0, policy_version 1566707 (0.0010) [2023-12-27 02:48:27,913][105692] Updated weights for policy 0, policy_version 1566717 (0.0010) [2023-12-27 02:48:27,916][105620] Updated weights for policy 1, policy_version 1570161 (0.0008) [2023-12-27 02:48:27,970][105692] Updated weights for policy 0, policy_version 1566727 (0.0010) [2023-12-27 02:48:27,974][105620] Updated weights for policy 1, policy_version 1570171 (0.0005) [2023-12-27 02:48:28,039][105620] Updated weights for policy 1, policy_version 1570181 (0.0005) [2023-12-27 02:48:28,670][105620] Updated weights for policy 1, policy_version 1570191 (0.0009) [2023-12-27 02:48:28,682][105692] Updated weights for policy 0, policy_version 1566737 (0.0011) [2023-12-27 02:48:28,728][105620] Updated weights for policy 1, policy_version 1570201 (0.0010) [2023-12-27 02:48:28,734][105692] Updated weights for policy 0, policy_version 1566747 (0.0010) [2023-12-27 02:48:28,789][105692] Updated weights for policy 0, policy_version 1566757 (0.0010) [2023-12-27 02:48:28,790][105620] Updated weights for policy 1, policy_version 1570211 (0.0010) [2023-12-27 02:48:29,502][105620] Updated weights for policy 1, policy_version 1570221 (0.0009) [2023-12-27 02:48:29,547][105692] Updated weights for policy 0, policy_version 1566767 (0.0010) [2023-12-27 02:48:29,558][105620] Updated weights for policy 1, policy_version 1570231 (0.0006) [2023-12-27 02:48:29,606][105692] Updated weights for policy 0, policy_version 1566777 (0.0011) [2023-12-27 02:48:29,616][105620] Updated weights for policy 1, policy_version 1570241 (0.0005) [2023-12-27 02:48:29,664][105692] Updated weights for policy 0, policy_version 1566787 (0.0010) [2023-12-27 02:48:30,341][105692] Updated weights for policy 0, policy_version 1566797 (0.0010) [2023-12-27 02:48:30,342][105620] Updated weights for policy 1, policy_version 1570251 (0.0005) [2023-12-27 02:48:30,387][105692] Updated weights for policy 0, policy_version 1566807 (0.0010) [2023-12-27 02:48:30,400][105620] Updated weights for policy 1, policy_version 1570261 (0.0006) [2023-12-27 02:48:30,439][105692] Updated weights for policy 0, policy_version 1566817 (0.0011) [2023-12-27 02:48:30,463][105620] Updated weights for policy 1, policy_version 1570271 (0.0006) [2023-12-27 02:48:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.4, 300 sec: 19521.9). Total num frames: 803209216. Throughput: 0: 9853.8, 1: 9758.1. Samples: 803181588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:48:31,063][104569] Avg episode reward: [(0, '8620.780'), (1, '8814.642')] [2023-12-27 02:48:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001566824_401162240.pth... [2023-12-27 02:48:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001565704_400875520.pth [2023-12-27 02:48:31,084][105620] Updated weights for policy 1, policy_version 1570281 (0.0009) [2023-12-27 02:48:31,151][105620] Updated weights for policy 1, policy_version 1570291 (0.0009) [2023-12-27 02:48:31,175][105692] Updated weights for policy 0, policy_version 1566828 (0.0010) [2023-12-27 02:48:31,216][105620] Updated weights for policy 1, policy_version 1570301 (0.0008) [2023-12-27 02:48:31,231][105692] Updated weights for policy 0, policy_version 1566838 (0.0006) [2023-12-27 02:48:31,284][105620] Updated weights for policy 1, policy_version 1570311 (0.0008) [2023-12-27 02:48:31,289][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001570312_402055168.pth... [2023-12-27 02:48:31,294][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001569160_401760256.pth [2023-12-27 02:48:31,295][105692] Updated weights for policy 0, policy_version 1566848 (0.0008) [2023-12-27 02:48:31,944][105692] Updated weights for policy 0, policy_version 1566858 (0.0006) [2023-12-27 02:48:31,992][105692] Updated weights for policy 0, policy_version 1566868 (0.0005) [2023-12-27 02:48:32,045][105692] Updated weights for policy 0, policy_version 1566878 (0.0005) [2023-12-27 02:48:32,108][105692] Updated weights for policy 0, policy_version 1566888 (0.0005) [2023-12-27 02:48:32,114][105620] Updated weights for policy 1, policy_version 1570321 (0.0009) [2023-12-27 02:48:32,168][105620] Updated weights for policy 1, policy_version 1570331 (0.0010) [2023-12-27 02:48:32,229][105620] Updated weights for policy 1, policy_version 1570341 (0.0009) [2023-12-27 02:48:32,711][105692] Updated weights for policy 0, policy_version 1566898 (0.0010) [2023-12-27 02:48:32,767][105692] Updated weights for policy 0, policy_version 1566908 (0.0008) [2023-12-27 02:48:32,825][105692] Updated weights for policy 0, policy_version 1566918 (0.0007) [2023-12-27 02:48:33,070][105620] Updated weights for policy 1, policy_version 1570351 (0.0010) [2023-12-27 02:48:33,134][105620] Updated weights for policy 1, policy_version 1570361 (0.0010) [2023-12-27 02:48:33,183][105620] Updated weights for policy 1, policy_version 1570372 (0.0009) [2023-12-27 02:48:33,431][105692] Updated weights for policy 0, policy_version 1566928 (0.0006) [2023-12-27 02:48:33,489][105692] Updated weights for policy 0, policy_version 1566938 (0.0007) [2023-12-27 02:48:33,543][105692] Updated weights for policy 0, policy_version 1566948 (0.0010) [2023-12-27 02:48:33,958][105620] Updated weights for policy 1, policy_version 1570382 (0.0010) [2023-12-27 02:48:34,010][105620] Updated weights for policy 1, policy_version 1570392 (0.0010) [2023-12-27 02:48:34,063][105620] Updated weights for policy 1, policy_version 1570402 (0.0008) [2023-12-27 02:48:34,260][105692] Updated weights for policy 0, policy_version 1566958 (0.0007) [2023-12-27 02:48:34,321][105692] Updated weights for policy 0, policy_version 1566968 (0.0011) [2023-12-27 02:48:34,386][105692] Updated weights for policy 0, policy_version 1566978 (0.0009) [2023-12-27 02:48:34,816][105620] Updated weights for policy 1, policy_version 1570412 (0.0008) [2023-12-27 02:48:34,878][105620] Updated weights for policy 1, policy_version 1570422 (0.0008) [2023-12-27 02:48:34,948][105620] Updated weights for policy 1, policy_version 1570432 (0.0007) [2023-12-27 02:48:34,958][105692] Updated weights for policy 0, policy_version 1566988 (0.0006) [2023-12-27 02:48:35,027][105692] Updated weights for policy 0, policy_version 1566998 (0.0007) [2023-12-27 02:48:35,089][105692] Updated weights for policy 0, policy_version 1567008 (0.0009) [2023-12-27 02:48:35,638][105620] Updated weights for policy 1, policy_version 1570442 (0.0007) [2023-12-27 02:48:35,696][105620] Updated weights for policy 1, policy_version 1570452 (0.0010) [2023-12-27 02:48:35,736][105692] Updated weights for policy 0, policy_version 1567018 (0.0009) [2023-12-27 02:48:35,752][105620] Updated weights for policy 1, policy_version 1570462 (0.0010) [2023-12-27 02:48:35,789][105692] Updated weights for policy 0, policy_version 1567028 (0.0007) [2023-12-27 02:48:35,809][105620] Updated weights for policy 1, policy_version 1570472 (0.0009) [2023-12-27 02:48:35,848][105692] Updated weights for policy 0, policy_version 1567038 (0.0008) [2023-12-27 02:48:35,901][105692] Updated weights for policy 0, policy_version 1567048 (0.0009) [2023-12-27 02:48:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 803315712. Throughput: 0: 9898.8, 1: 9718.2. Samples: 803298984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:48:36,063][104569] Avg episode reward: [(0, '8444.241'), (1, '8995.234')] [2023-12-27 02:48:36,507][105620] Updated weights for policy 1, policy_version 1570482 (0.0011) [2023-12-27 02:48:36,581][105620] Updated weights for policy 1, policy_version 1570492 (0.0011) [2023-12-27 02:48:36,626][105692] Updated weights for policy 0, policy_version 1567058 (0.0006) [2023-12-27 02:48:36,634][105620] Updated weights for policy 1, policy_version 1570502 (0.0011) [2023-12-27 02:48:36,685][105692] Updated weights for policy 0, policy_version 1567068 (0.0007) [2023-12-27 02:48:36,737][105692] Updated weights for policy 0, policy_version 1567078 (0.0008) [2023-12-27 02:48:37,363][105620] Updated weights for policy 1, policy_version 1570512 (0.0008) [2023-12-27 02:48:37,413][105620] Updated weights for policy 1, policy_version 1570522 (0.0005) [2023-12-27 02:48:37,465][105692] Updated weights for policy 0, policy_version 1567088 (0.0009) [2023-12-27 02:48:37,467][105620] Updated weights for policy 1, policy_version 1570532 (0.0005) [2023-12-27 02:48:37,525][105692] Updated weights for policy 0, policy_version 1567098 (0.0009) [2023-12-27 02:48:37,594][105692] Updated weights for policy 0, policy_version 1567108 (0.0010) [2023-12-27 02:48:38,046][105620] Updated weights for policy 1, policy_version 1570542 (0.0006) [2023-12-27 02:48:38,108][105620] Updated weights for policy 1, policy_version 1570552 (0.0006) [2023-12-27 02:48:38,168][105620] Updated weights for policy 1, policy_version 1570562 (0.0008) [2023-12-27 02:48:38,272][105692] Updated weights for policy 0, policy_version 1567118 (0.0007) [2023-12-27 02:48:38,323][105692] Updated weights for policy 0, policy_version 1567128 (0.0006) [2023-12-27 02:48:38,382][105692] Updated weights for policy 0, policy_version 1567138 (0.0008) [2023-12-27 02:48:38,849][105620] Updated weights for policy 1, policy_version 1570572 (0.0007) [2023-12-27 02:48:38,897][105620] Updated weights for policy 1, policy_version 1570582 (0.0009) [2023-12-27 02:48:38,948][105620] Updated weights for policy 1, policy_version 1570592 (0.0009) [2023-12-27 02:48:39,105][105692] Updated weights for policy 0, policy_version 1567148 (0.0008) [2023-12-27 02:48:39,159][105692] Updated weights for policy 0, policy_version 1567158 (0.0009) [2023-12-27 02:48:39,218][105692] Updated weights for policy 0, policy_version 1567168 (0.0009) [2023-12-27 02:48:39,763][105620] Updated weights for policy 1, policy_version 1570602 (0.0008) [2023-12-27 02:48:39,830][105620] Updated weights for policy 1, policy_version 1570612 (0.0009) [2023-12-27 02:48:39,885][105620] Updated weights for policy 1, policy_version 1570622 (0.0009) [2023-12-27 02:48:39,928][105692] Updated weights for policy 0, policy_version 1567178 (0.0008) [2023-12-27 02:48:39,953][105620] Updated weights for policy 1, policy_version 1570632 (0.0009) [2023-12-27 02:48:39,988][105692] Updated weights for policy 0, policy_version 1567188 (0.0008) [2023-12-27 02:48:40,048][105692] Updated weights for policy 0, policy_version 1567198 (0.0009) [2023-12-27 02:48:40,108][105692] Updated weights for policy 0, policy_version 1567208 (0.0009) [2023-12-27 02:48:40,661][105620] Updated weights for policy 1, policy_version 1570642 (0.0010) [2023-12-27 02:48:40,707][105620] Updated weights for policy 1, policy_version 1570652 (0.0008) [2023-12-27 02:48:40,774][105620] Updated weights for policy 1, policy_version 1570662 (0.0008) [2023-12-27 02:48:40,885][105692] Updated weights for policy 0, policy_version 1567218 (0.0009) [2023-12-27 02:48:40,936][105692] Updated weights for policy 0, policy_version 1567228 (0.0009) [2023-12-27 02:48:40,983][105692] Updated weights for policy 0, policy_version 1567238 (0.0009) [2023-12-27 02:48:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 803414016. Throughput: 0: 9921.9, 1: 9732.9. Samples: 803416848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:48:41,063][104569] Avg episode reward: [(0, '8808.056'), (1, '8721.063')] [2023-12-27 02:48:41,522][105620] Updated weights for policy 1, policy_version 1570672 (0.0010) [2023-12-27 02:48:41,574][105620] Updated weights for policy 1, policy_version 1570682 (0.0009) [2023-12-27 02:48:41,636][105620] Updated weights for policy 1, policy_version 1570692 (0.0009) [2023-12-27 02:48:41,835][105692] Updated weights for policy 0, policy_version 1567248 (0.0009) [2023-12-27 02:48:41,889][105692] Updated weights for policy 0, policy_version 1567258 (0.0008) [2023-12-27 02:48:41,938][105692] Updated weights for policy 0, policy_version 1567268 (0.0010) [2023-12-27 02:48:42,442][105620] Updated weights for policy 1, policy_version 1570702 (0.0009) [2023-12-27 02:48:42,501][105620] Updated weights for policy 1, policy_version 1570712 (0.0009) [2023-12-27 02:48:42,553][105620] Updated weights for policy 1, policy_version 1570722 (0.0011) [2023-12-27 02:48:42,771][105692] Updated weights for policy 0, policy_version 1567278 (0.0009) [2023-12-27 02:48:42,840][105692] Updated weights for policy 0, policy_version 1567288 (0.0009) [2023-12-27 02:48:42,911][105692] Updated weights for policy 0, policy_version 1567298 (0.0010) [2023-12-27 02:48:43,184][105620] Updated weights for policy 1, policy_version 1570732 (0.0008) [2023-12-27 02:48:43,239][105620] Updated weights for policy 1, policy_version 1570742 (0.0005) [2023-12-27 02:48:43,296][105620] Updated weights for policy 1, policy_version 1570752 (0.0005) [2023-12-27 02:48:43,767][105692] Updated weights for policy 0, policy_version 1567308 (0.0009) [2023-12-27 02:48:43,811][105692] Updated weights for policy 0, policy_version 1567318 (0.0007) [2023-12-27 02:48:43,859][105692] Updated weights for policy 0, policy_version 1567328 (0.0008) [2023-12-27 02:48:43,887][105620] Updated weights for policy 1, policy_version 1570762 (0.0010) [2023-12-27 02:48:43,941][105620] Updated weights for policy 1, policy_version 1570772 (0.0010) [2023-12-27 02:48:43,992][105620] Updated weights for policy 1, policy_version 1570782 (0.0010) [2023-12-27 02:48:44,043][105620] Updated weights for policy 1, policy_version 1570792 (0.0010) [2023-12-27 02:48:44,612][105692] Updated weights for policy 0, policy_version 1567338 (0.0006) [2023-12-27 02:48:44,673][105692] Updated weights for policy 0, policy_version 1567348 (0.0005) [2023-12-27 02:48:44,733][105692] Updated weights for policy 0, policy_version 1567358 (0.0005) [2023-12-27 02:48:44,774][105620] Updated weights for policy 1, policy_version 1570802 (0.0011) [2023-12-27 02:48:44,794][105692] Updated weights for policy 0, policy_version 1567368 (0.0006) [2023-12-27 02:48:44,836][105620] Updated weights for policy 1, policy_version 1570812 (0.0012) [2023-12-27 02:48:44,890][105620] Updated weights for policy 1, policy_version 1570822 (0.0009) [2023-12-27 02:48:45,407][105692] Updated weights for policy 0, policy_version 1567378 (0.0005) [2023-12-27 02:48:45,468][105692] Updated weights for policy 0, policy_version 1567388 (0.0005) [2023-12-27 02:48:45,534][105692] Updated weights for policy 0, policy_version 1567398 (0.0005) [2023-12-27 02:48:45,612][105620] Updated weights for policy 1, policy_version 1570832 (0.0006) [2023-12-27 02:48:45,668][105620] Updated weights for policy 1, policy_version 1570842 (0.0005) [2023-12-27 02:48:45,730][105620] Updated weights for policy 1, policy_version 1570852 (0.0005) [2023-12-27 02:48:46,060][105692] Updated weights for policy 0, policy_version 1567408 (0.0009) [2023-12-27 02:48:46,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 803504128. Throughput: 0: 9744.4, 1: 9777.3. Samples: 803473424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:48:46,062][104569] Avg episode reward: [(0, '8900.434'), (1, '8714.709')] [2023-12-27 02:48:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001570856_402194432.pth... [2023-12-27 02:48:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001569704_401899520.pth [2023-12-27 02:48:46,108][105692] Updated weights for policy 0, policy_version 1567418 (0.0010) [2023-12-27 02:48:46,153][105692] Updated weights for policy 0, policy_version 1567428 (0.0010) [2023-12-27 02:48:46,170][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001567432_401317888.pth... [2023-12-27 02:48:46,173][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001566248_401014784.pth [2023-12-27 02:48:46,394][105620] Updated weights for policy 1, policy_version 1570862 (0.0006) [2023-12-27 02:48:46,457][105620] Updated weights for policy 1, policy_version 1570872 (0.0011) [2023-12-27 02:48:46,472][105586] KL-divergence is very high: 141.1537 [2023-12-27 02:48:46,512][105620] Updated weights for policy 1, policy_version 1570882 (0.0010) [2023-12-27 02:48:46,519][105586] KL-divergence is very high: 133.2977 [2023-12-27 02:48:46,922][105692] Updated weights for policy 0, policy_version 1567438 (0.0010) [2023-12-27 02:48:46,973][105692] Updated weights for policy 0, policy_version 1567448 (0.0010) [2023-12-27 02:48:47,028][105692] Updated weights for policy 0, policy_version 1567458 (0.0008) [2023-12-27 02:48:47,244][105620] Updated weights for policy 1, policy_version 1570892 (0.0010) [2023-12-27 02:48:47,291][105620] Updated weights for policy 1, policy_version 1570902 (0.0010) [2023-12-27 02:48:47,342][105620] Updated weights for policy 1, policy_version 1570912 (0.0010) [2023-12-27 02:48:47,667][105692] Updated weights for policy 0, policy_version 1567468 (0.0008) [2023-12-27 02:48:47,735][105692] Updated weights for policy 0, policy_version 1567478 (0.0006) [2023-12-27 02:48:47,797][105692] Updated weights for policy 0, policy_version 1567488 (0.0007) [2023-12-27 02:48:48,140][105620] Updated weights for policy 1, policy_version 1570922 (0.0010) [2023-12-27 02:48:48,203][105620] Updated weights for policy 1, policy_version 1570932 (0.0009) [2023-12-27 02:48:48,262][105620] Updated weights for policy 1, policy_version 1570942 (0.0011) [2023-12-27 02:48:48,329][105620] Updated weights for policy 1, policy_version 1570952 (0.0011) [2023-12-27 02:48:48,461][105692] Updated weights for policy 0, policy_version 1567498 (0.0007) [2023-12-27 02:48:48,521][105692] Updated weights for policy 0, policy_version 1567508 (0.0011) [2023-12-27 02:48:48,584][105692] Updated weights for policy 0, policy_version 1567518 (0.0011) [2023-12-27 02:48:48,643][105692] Updated weights for policy 0, policy_version 1567528 (0.0011) [2023-12-27 02:48:48,941][105620] Updated weights for policy 1, policy_version 1570962 (0.0005) [2023-12-27 02:48:48,988][105620] Updated weights for policy 1, policy_version 1570972 (0.0005) [2023-12-27 02:48:49,040][105620] Updated weights for policy 1, policy_version 1570982 (0.0005) [2023-12-27 02:48:49,350][105692] Updated weights for policy 0, policy_version 1567538 (0.0008) [2023-12-27 02:48:49,420][105692] Updated weights for policy 0, policy_version 1567548 (0.0011) [2023-12-27 02:48:49,488][105692] Updated weights for policy 0, policy_version 1567558 (0.0010) [2023-12-27 02:48:49,680][105620] Updated weights for policy 1, policy_version 1570992 (0.0005) [2023-12-27 02:48:49,750][105620] Updated weights for policy 1, policy_version 1571002 (0.0005) [2023-12-27 02:48:49,822][105620] Updated weights for policy 1, policy_version 1571012 (0.0006) [2023-12-27 02:48:50,193][105692] Updated weights for policy 0, policy_version 1567568 (0.0006) [2023-12-27 02:48:50,260][105692] Updated weights for policy 0, policy_version 1567578 (0.0005) [2023-12-27 02:48:50,324][105692] Updated weights for policy 0, policy_version 1567588 (0.0005) [2023-12-27 02:48:50,446][105620] Updated weights for policy 1, policy_version 1571022 (0.0011) [2023-12-27 02:48:50,495][105620] Updated weights for policy 1, policy_version 1571032 (0.0010) [2023-12-27 02:48:50,554][105620] Updated weights for policy 1, policy_version 1571042 (0.0010) [2023-12-27 02:48:50,872][105692] Updated weights for policy 0, policy_version 1567598 (0.0005) [2023-12-27 02:48:50,938][105692] Updated weights for policy 0, policy_version 1567608 (0.0009) [2023-12-27 02:48:51,007][105692] Updated weights for policy 0, policy_version 1567618 (0.0007) [2023-12-27 02:48:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 803610624. Throughput: 0: 9870.0, 1: 9787.0. Samples: 803594956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:48:51,062][104569] Avg episode reward: [(0, '8716.528'), (1, '8897.362')] [2023-12-27 02:48:51,343][105620] Updated weights for policy 1, policy_version 1571052 (0.0009) [2023-12-27 02:48:51,414][105620] Updated weights for policy 1, policy_version 1571062 (0.0010) [2023-12-27 02:48:51,467][105620] Updated weights for policy 1, policy_version 1571072 (0.0011) [2023-12-27 02:48:51,740][105692] Updated weights for policy 0, policy_version 1567628 (0.0008) [2023-12-27 02:48:51,800][105692] Updated weights for policy 0, policy_version 1567638 (0.0008) [2023-12-27 02:48:51,850][105692] Updated weights for policy 0, policy_version 1567648 (0.0008) [2023-12-27 02:48:52,241][105620] Updated weights for policy 1, policy_version 1571082 (0.0010) [2023-12-27 02:48:52,304][105620] Updated weights for policy 1, policy_version 1571092 (0.0011) [2023-12-27 02:48:52,369][105620] Updated weights for policy 1, policy_version 1571102 (0.0010) [2023-12-27 02:48:52,427][105620] Updated weights for policy 1, policy_version 1571112 (0.0011) [2023-12-27 02:48:52,604][105692] Updated weights for policy 0, policy_version 1567658 (0.0009) [2023-12-27 02:48:52,662][105692] Updated weights for policy 0, policy_version 1567668 (0.0009) [2023-12-27 02:48:52,719][105692] Updated weights for policy 0, policy_version 1567678 (0.0008) [2023-12-27 02:48:52,779][105692] Updated weights for policy 0, policy_version 1567688 (0.0008) [2023-12-27 02:48:53,101][105620] Updated weights for policy 1, policy_version 1571122 (0.0010) [2023-12-27 02:48:53,146][105620] Updated weights for policy 1, policy_version 1571132 (0.0010) [2023-12-27 02:48:53,195][105620] Updated weights for policy 1, policy_version 1571142 (0.0010) [2023-12-27 02:48:53,530][105692] Updated weights for policy 0, policy_version 1567698 (0.0009) [2023-12-27 02:48:53,584][105692] Updated weights for policy 0, policy_version 1567708 (0.0010) [2023-12-27 02:48:53,643][105692] Updated weights for policy 0, policy_version 1567718 (0.0010) [2023-12-27 02:48:53,839][105620] Updated weights for policy 1, policy_version 1571152 (0.0006) [2023-12-27 02:48:53,906][105620] Updated weights for policy 1, policy_version 1571162 (0.0005) [2023-12-27 02:48:53,970][105620] Updated weights for policy 1, policy_version 1571172 (0.0006) [2023-12-27 02:48:54,402][105692] Updated weights for policy 0, policy_version 1567728 (0.0009) [2023-12-27 02:48:54,460][105692] Updated weights for policy 0, policy_version 1567738 (0.0009) [2023-12-27 02:48:54,501][105620] Updated weights for policy 1, policy_version 1571182 (0.0007) [2023-12-27 02:48:54,514][105692] Updated weights for policy 0, policy_version 1567748 (0.0007) [2023-12-27 02:48:54,565][105620] Updated weights for policy 1, policy_version 1571192 (0.0005) [2023-12-27 02:48:54,626][105620] Updated weights for policy 1, policy_version 1571202 (0.0006) [2023-12-27 02:48:55,244][105692] Updated weights for policy 0, policy_version 1567758 (0.0007) [2023-12-27 02:48:55,310][105692] Updated weights for policy 0, policy_version 1567768 (0.0010) [2023-12-27 02:48:55,329][105620] Updated weights for policy 1, policy_version 1571212 (0.0007) [2023-12-27 02:48:55,373][105692] Updated weights for policy 0, policy_version 1567778 (0.0011) [2023-12-27 02:48:55,384][105620] Updated weights for policy 1, policy_version 1571222 (0.0010) [2023-12-27 02:48:55,444][105620] Updated weights for policy 1, policy_version 1571232 (0.0011) [2023-12-27 02:48:56,011][105692] Updated weights for policy 0, policy_version 1567788 (0.0010) [2023-12-27 02:48:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 803700736. Throughput: 0: 9948.8, 1: 9857.0. Samples: 803713664. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:48:56,062][104569] Avg episode reward: [(0, '8359.336'), (1, '9356.553')] [2023-12-27 02:48:56,076][105692] Updated weights for policy 0, policy_version 1567798 (0.0011) [2023-12-27 02:48:56,142][105692] Updated weights for policy 0, policy_version 1567808 (0.0011) [2023-12-27 02:48:56,156][105620] Updated weights for policy 1, policy_version 1571242 (0.0010) [2023-12-27 02:48:56,207][105620] Updated weights for policy 1, policy_version 1571252 (0.0010) [2023-12-27 02:48:56,258][105620] Updated weights for policy 1, policy_version 1571262 (0.0010) [2023-12-27 02:48:56,305][105620] Updated weights for policy 1, policy_version 1571272 (0.0010) [2023-12-27 02:48:56,856][105692] Updated weights for policy 0, policy_version 1567818 (0.0011) [2023-12-27 02:48:56,914][105692] Updated weights for policy 0, policy_version 1567828 (0.0010) [2023-12-27 02:48:56,962][105692] Updated weights for policy 0, policy_version 1567838 (0.0010) [2023-12-27 02:48:57,015][105692] Updated weights for policy 0, policy_version 1567848 (0.0011) [2023-12-27 02:48:57,050][105620] Updated weights for policy 1, policy_version 1571282 (0.0010) [2023-12-27 02:48:57,106][105620] Updated weights for policy 1, policy_version 1571292 (0.0010) [2023-12-27 02:48:57,162][105620] Updated weights for policy 1, policy_version 1571302 (0.0010) [2023-12-27 02:48:57,792][105692] Updated weights for policy 0, policy_version 1567858 (0.0011) [2023-12-27 02:48:57,841][105692] Updated weights for policy 0, policy_version 1567868 (0.0010) [2023-12-27 02:48:57,889][105692] Updated weights for policy 0, policy_version 1567878 (0.0010) [2023-12-27 02:48:57,913][105620] Updated weights for policy 1, policy_version 1571312 (0.0010) [2023-12-27 02:48:57,960][105620] Updated weights for policy 1, policy_version 1571322 (0.0010) [2023-12-27 02:48:58,004][105620] Updated weights for policy 1, policy_version 1571332 (0.0010) [2023-12-27 02:48:58,688][105692] Updated weights for policy 0, policy_version 1567888 (0.0009) [2023-12-27 02:48:58,757][105692] Updated weights for policy 0, policy_version 1567898 (0.0008) [2023-12-27 02:48:58,786][105620] Updated weights for policy 1, policy_version 1571342 (0.0009) [2023-12-27 02:48:58,821][105692] Updated weights for policy 0, policy_version 1567908 (0.0008) [2023-12-27 02:48:58,854][105620] Updated weights for policy 1, policy_version 1571352 (0.0007) [2023-12-27 02:48:58,921][105620] Updated weights for policy 1, policy_version 1571362 (0.0008) [2023-12-27 02:48:59,593][105692] Updated weights for policy 0, policy_version 1567918 (0.0009) [2023-12-27 02:48:59,651][105692] Updated weights for policy 0, policy_version 1567928 (0.0005) [2023-12-27 02:48:59,703][105692] Updated weights for policy 0, policy_version 1567938 (0.0005) [2023-12-27 02:48:59,750][105620] Updated weights for policy 1, policy_version 1571372 (0.0009) [2023-12-27 02:48:59,811][105620] Updated weights for policy 1, policy_version 1571382 (0.0009) [2023-12-27 02:48:59,867][105620] Updated weights for policy 1, policy_version 1571392 (0.0009) [2023-12-27 02:49:00,390][105692] Updated weights for policy 0, policy_version 1567948 (0.0007) [2023-12-27 02:49:00,447][105692] Updated weights for policy 0, policy_version 1567959 (0.0010) [2023-12-27 02:49:00,501][105692] Updated weights for policy 0, policy_version 1567971 (0.0010) [2023-12-27 02:49:00,579][105620] Updated weights for policy 1, policy_version 1571402 (0.0009) [2023-12-27 02:49:00,625][105620] Updated weights for policy 1, policy_version 1571412 (0.0008) [2023-12-27 02:49:00,670][105620] Updated weights for policy 1, policy_version 1571422 (0.0008) [2023-12-27 02:49:00,724][105620] Updated weights for policy 1, policy_version 1571432 (0.0008) [2023-12-27 02:49:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 803799040. Throughput: 0: 9973.4, 1: 9818.5. Samples: 803770164. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:49:01,062][104569] Avg episode reward: [(0, '8447.227'), (1, '9263.982')] [2023-12-27 02:49:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001567976_401457152.pth... [2023-12-27 02:49:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001571432_402341888.pth... [2023-12-27 02:49:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001566824_401162240.pth [2023-12-27 02:49:01,091][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001570312_402055168.pth [2023-12-27 02:49:01,284][105692] Updated weights for policy 0, policy_version 1567981 (0.0009) [2023-12-27 02:49:01,338][105692] Updated weights for policy 0, policy_version 1567991 (0.0009) [2023-12-27 02:49:01,405][105692] Updated weights for policy 0, policy_version 1568001 (0.0008) [2023-12-27 02:49:01,502][105620] Updated weights for policy 1, policy_version 1571442 (0.0008) [2023-12-27 02:49:01,557][105620] Updated weights for policy 1, policy_version 1571452 (0.0009) [2023-12-27 02:49:01,613][105620] Updated weights for policy 1, policy_version 1571462 (0.0009) [2023-12-27 02:49:02,153][105692] Updated weights for policy 0, policy_version 1568011 (0.0010) [2023-12-27 02:49:02,201][105692] Updated weights for policy 0, policy_version 1568021 (0.0010) [2023-12-27 02:49:02,248][105692] Updated weights for policy 0, policy_version 1568031 (0.0010) [2023-12-27 02:49:02,302][105620] Updated weights for policy 1, policy_version 1571472 (0.0006) [2023-12-27 02:49:02,360][105620] Updated weights for policy 1, policy_version 1571482 (0.0008) [2023-12-27 02:49:02,419][105620] Updated weights for policy 1, policy_version 1571492 (0.0008) [2023-12-27 02:49:03,031][105692] Updated weights for policy 0, policy_version 1568041 (0.0010) [2023-12-27 02:49:03,096][105692] Updated weights for policy 0, policy_version 1568051 (0.0006) [2023-12-27 02:49:03,137][105620] Updated weights for policy 1, policy_version 1571502 (0.0009) [2023-12-27 02:49:03,159][105692] Updated weights for policy 0, policy_version 1568061 (0.0006) [2023-12-27 02:49:03,189][105620] Updated weights for policy 1, policy_version 1571512 (0.0008) [2023-12-27 02:49:03,211][105692] Updated weights for policy 0, policy_version 1568071 (0.0006) [2023-12-27 02:49:03,248][105620] Updated weights for policy 1, policy_version 1571522 (0.0009) [2023-12-27 02:49:03,737][105692] Updated weights for policy 0, policy_version 1568081 (0.0006) [2023-12-27 02:49:03,784][105692] Updated weights for policy 0, policy_version 1568091 (0.0008) [2023-12-27 02:49:03,836][105692] Updated weights for policy 0, policy_version 1568101 (0.0006) [2023-12-27 02:49:03,968][105620] Updated weights for policy 1, policy_version 1571532 (0.0010) [2023-12-27 02:49:04,020][105620] Updated weights for policy 1, policy_version 1571542 (0.0010) [2023-12-27 02:49:04,076][105620] Updated weights for policy 1, policy_version 1571552 (0.0010) [2023-12-27 02:49:04,602][105692] Updated weights for policy 0, policy_version 1568111 (0.0007) [2023-12-27 02:49:04,657][105692] Updated weights for policy 0, policy_version 1568121 (0.0007) [2023-12-27 02:49:04,716][105692] Updated weights for policy 0, policy_version 1568131 (0.0008) [2023-12-27 02:49:04,795][105620] Updated weights for policy 1, policy_version 1571562 (0.0011) [2023-12-27 02:49:04,851][105620] Updated weights for policy 1, policy_version 1571572 (0.0009) [2023-12-27 02:49:04,910][105620] Updated weights for policy 1, policy_version 1571582 (0.0005) [2023-12-27 02:49:04,972][105620] Updated weights for policy 1, policy_version 1571592 (0.0005) [2023-12-27 02:49:05,350][105692] Updated weights for policy 0, policy_version 1568141 (0.0008) [2023-12-27 02:49:05,407][105692] Updated weights for policy 0, policy_version 1568152 (0.0010) [2023-12-27 02:49:05,467][105692] Updated weights for policy 0, policy_version 1568163 (0.0010) [2023-12-27 02:49:05,503][105620] Updated weights for policy 1, policy_version 1571602 (0.0005) [2023-12-27 02:49:05,559][105620] Updated weights for policy 1, policy_version 1571612 (0.0006) [2023-12-27 02:49:05,612][105620] Updated weights for policy 1, policy_version 1571622 (0.0006) [2023-12-27 02:49:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 803897344. Throughput: 0: 9878.0, 1: 9774.1. Samples: 803885976. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:49:06,063][104569] Avg episode reward: [(0, '8808.799'), (1, '9082.310')] [2023-12-27 02:49:06,244][105692] Updated weights for policy 0, policy_version 1568173 (0.0007) [2023-12-27 02:49:06,297][105620] Updated weights for policy 1, policy_version 1571632 (0.0008) [2023-12-27 02:49:06,297][105692] Updated weights for policy 0, policy_version 1568183 (0.0007) [2023-12-27 02:49:06,356][105692] Updated weights for policy 0, policy_version 1568193 (0.0006) [2023-12-27 02:49:06,358][105620] Updated weights for policy 1, policy_version 1571642 (0.0010) [2023-12-27 02:49:06,414][105620] Updated weights for policy 1, policy_version 1571652 (0.0011) [2023-12-27 02:49:07,080][105692] Updated weights for policy 0, policy_version 1568203 (0.0006) [2023-12-27 02:49:07,117][105620] Updated weights for policy 1, policy_version 1571662 (0.0011) [2023-12-27 02:49:07,140][105692] Updated weights for policy 0, policy_version 1568213 (0.0006) [2023-12-27 02:49:07,168][105620] Updated weights for policy 1, policy_version 1571672 (0.0009) [2023-12-27 02:49:07,199][105692] Updated weights for policy 0, policy_version 1568223 (0.0007) [2023-12-27 02:49:07,223][105620] Updated weights for policy 1, policy_version 1571682 (0.0005) [2023-12-27 02:49:07,865][105620] Updated weights for policy 1, policy_version 1571692 (0.0007) [2023-12-27 02:49:07,926][105620] Updated weights for policy 1, policy_version 1571702 (0.0009) [2023-12-27 02:49:07,986][105620] Updated weights for policy 1, policy_version 1571712 (0.0007) [2023-12-27 02:49:08,007][105692] Updated weights for policy 0, policy_version 1568233 (0.0009) [2023-12-27 02:49:08,069][105692] Updated weights for policy 0, policy_version 1568243 (0.0008) [2023-12-27 02:49:08,131][105692] Updated weights for policy 0, policy_version 1568253 (0.0009) [2023-12-27 02:49:08,195][105692] Updated weights for policy 0, policy_version 1568263 (0.0010) [2023-12-27 02:49:08,716][105620] Updated weights for policy 1, policy_version 1571722 (0.0008) [2023-12-27 02:49:08,775][105620] Updated weights for policy 1, policy_version 1571732 (0.0008) [2023-12-27 02:49:08,829][105620] Updated weights for policy 1, policy_version 1571742 (0.0010) [2023-12-27 02:49:08,883][105620] Updated weights for policy 1, policy_version 1571752 (0.0009) [2023-12-27 02:49:08,931][105692] Updated weights for policy 0, policy_version 1568273 (0.0007) [2023-12-27 02:49:08,981][105692] Updated weights for policy 0, policy_version 1568283 (0.0008) [2023-12-27 02:49:09,039][105692] Updated weights for policy 0, policy_version 1568293 (0.0005) [2023-12-27 02:49:09,632][105620] Updated weights for policy 1, policy_version 1571762 (0.0009) [2023-12-27 02:49:09,695][105620] Updated weights for policy 1, policy_version 1571772 (0.0009) [2023-12-27 02:49:09,758][105620] Updated weights for policy 1, policy_version 1571782 (0.0009) [2023-12-27 02:49:09,799][105692] Updated weights for policy 0, policy_version 1568303 (0.0008) [2023-12-27 02:49:09,867][105692] Updated weights for policy 0, policy_version 1568313 (0.0007) [2023-12-27 02:49:09,934][105692] Updated weights for policy 0, policy_version 1568323 (0.0008) [2023-12-27 02:49:10,508][105620] Updated weights for policy 1, policy_version 1571792 (0.0009) [2023-12-27 02:49:10,569][105620] Updated weights for policy 1, policy_version 1571802 (0.0009) [2023-12-27 02:49:10,636][105692] Updated weights for policy 0, policy_version 1568333 (0.0008) [2023-12-27 02:49:10,637][105620] Updated weights for policy 1, policy_version 1571812 (0.0009) [2023-12-27 02:49:10,699][105692] Updated weights for policy 0, policy_version 1568343 (0.0009) [2023-12-27 02:49:10,759][105692] Updated weights for policy 0, policy_version 1568353 (0.0009) [2023-12-27 02:49:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 803995648. Throughput: 0: 9835.3, 1: 9890.9. Samples: 804003028. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:49:11,062][104569] Avg episode reward: [(0, '8900.873'), (1, '8994.977')] [2023-12-27 02:49:11,335][105620] Updated weights for policy 1, policy_version 1571822 (0.0006) [2023-12-27 02:49:11,408][105620] Updated weights for policy 1, policy_version 1571832 (0.0008) [2023-12-27 02:49:11,446][105692] Updated weights for policy 0, policy_version 1568363 (0.0009) [2023-12-27 02:49:11,471][105620] Updated weights for policy 1, policy_version 1571842 (0.0010) [2023-12-27 02:49:11,507][105692] Updated weights for policy 0, policy_version 1568373 (0.0011) [2023-12-27 02:49:11,570][105692] Updated weights for policy 0, policy_version 1568383 (0.0010) [2023-12-27 02:49:12,235][105620] Updated weights for policy 1, policy_version 1571852 (0.0010) [2023-12-27 02:49:12,287][105620] Updated weights for policy 1, policy_version 1571862 (0.0010) [2023-12-27 02:49:12,343][105692] Updated weights for policy 0, policy_version 1568393 (0.0008) [2023-12-27 02:49:12,351][105620] Updated weights for policy 1, policy_version 1571873 (0.0011) [2023-12-27 02:49:12,411][105692] Updated weights for policy 0, policy_version 1568403 (0.0008) [2023-12-27 02:49:12,472][105692] Updated weights for policy 0, policy_version 1568413 (0.0010) [2023-12-27 02:49:12,534][105692] Updated weights for policy 0, policy_version 1568423 (0.0011) [2023-12-27 02:49:13,161][105620] Updated weights for policy 1, policy_version 1571883 (0.0009) [2023-12-27 02:49:13,225][105620] Updated weights for policy 1, policy_version 1571893 (0.0009) [2023-12-27 02:49:13,260][105692] Updated weights for policy 0, policy_version 1568433 (0.0006) [2023-12-27 02:49:13,283][105620] Updated weights for policy 1, policy_version 1571903 (0.0010) [2023-12-27 02:49:13,312][105692] Updated weights for policy 0, policy_version 1568443 (0.0006) [2023-12-27 02:49:13,371][105692] Updated weights for policy 0, policy_version 1568453 (0.0007) [2023-12-27 02:49:13,957][105692] Updated weights for policy 0, policy_version 1568463 (0.0007) [2023-12-27 02:49:14,003][105620] Updated weights for policy 1, policy_version 1571913 (0.0008) [2023-12-27 02:49:14,023][105692] Updated weights for policy 0, policy_version 1568473 (0.0007) [2023-12-27 02:49:14,064][105620] Updated weights for policy 1, policy_version 1571923 (0.0006) [2023-12-27 02:49:14,074][105692] Updated weights for policy 0, policy_version 1568483 (0.0007) [2023-12-27 02:49:14,123][105620] Updated weights for policy 1, policy_version 1571933 (0.0008) [2023-12-27 02:49:14,181][105620] Updated weights for policy 1, policy_version 1571943 (0.0009) [2023-12-27 02:49:14,663][105692] Updated weights for policy 0, policy_version 1568493 (0.0007) [2023-12-27 02:49:14,720][105692] Updated weights for policy 0, policy_version 1568503 (0.0010) [2023-12-27 02:49:14,777][105692] Updated weights for policy 0, policy_version 1568513 (0.0009) [2023-12-27 02:49:14,827][105620] Updated weights for policy 1, policy_version 1571953 (0.0008) [2023-12-27 02:49:14,889][105620] Updated weights for policy 1, policy_version 1571963 (0.0009) [2023-12-27 02:49:14,954][105620] Updated weights for policy 1, policy_version 1571973 (0.0008) [2023-12-27 02:49:15,504][105692] Updated weights for policy 0, policy_version 1568523 (0.0007) [2023-12-27 02:49:15,569][105692] Updated weights for policy 0, policy_version 1568533 (0.0009) [2023-12-27 02:49:15,628][105692] Updated weights for policy 0, policy_version 1568543 (0.0009) [2023-12-27 02:49:15,719][105620] Updated weights for policy 1, policy_version 1571983 (0.0009) [2023-12-27 02:49:15,773][105620] Updated weights for policy 1, policy_version 1571993 (0.0008) [2023-12-27 02:49:15,819][105620] Updated weights for policy 1, policy_version 1572003 (0.0008) [2023-12-27 02:49:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 804093952. Throughput: 0: 9755.4, 1: 9752.2. Samples: 804059428. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:49:16,062][104569] Avg episode reward: [(0, '8811.673'), (1, '8993.718')] [2023-12-27 02:49:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001568552_401604608.pth... [2023-12-27 02:49:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001572008_402489344.pth... [2023-12-27 02:49:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001567432_401317888.pth [2023-12-27 02:49:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001570856_402194432.pth [2023-12-27 02:49:16,381][105692] Updated weights for policy 0, policy_version 1568553 (0.0008) [2023-12-27 02:49:16,428][105692] Updated weights for policy 0, policy_version 1568563 (0.0009) [2023-12-27 02:49:16,476][105692] Updated weights for policy 0, policy_version 1568573 (0.0009) [2023-12-27 02:49:16,535][105692] Updated weights for policy 0, policy_version 1568584 (0.0009) [2023-12-27 02:49:16,576][105620] Updated weights for policy 1, policy_version 1572013 (0.0009) [2023-12-27 02:49:16,623][105620] Updated weights for policy 1, policy_version 1572023 (0.0009) [2023-12-27 02:49:16,669][105620] Updated weights for policy 1, policy_version 1572033 (0.0008) [2023-12-27 02:49:17,319][105692] Updated weights for policy 0, policy_version 1568594 (0.0008) [2023-12-27 02:49:17,377][105692] Updated weights for policy 0, policy_version 1568604 (0.0008) [2023-12-27 02:49:17,397][105620] Updated weights for policy 1, policy_version 1572043 (0.0008) [2023-12-27 02:49:17,429][105692] Updated weights for policy 0, policy_version 1568614 (0.0007) [2023-12-27 02:49:17,455][105620] Updated weights for policy 1, policy_version 1572053 (0.0005) [2023-12-27 02:49:17,523][105620] Updated weights for policy 1, policy_version 1572063 (0.0005) [2023-12-27 02:49:18,069][105620] Updated weights for policy 1, policy_version 1572073 (0.0006) [2023-12-27 02:49:18,107][105692] Updated weights for policy 0, policy_version 1568624 (0.0008) [2023-12-27 02:49:18,125][105620] Updated weights for policy 1, policy_version 1572083 (0.0009) [2023-12-27 02:49:18,158][105692] Updated weights for policy 0, policy_version 1568634 (0.0009) [2023-12-27 02:49:18,185][105620] Updated weights for policy 1, policy_version 1572093 (0.0011) [2023-12-27 02:49:18,217][105692] Updated weights for policy 0, policy_version 1568644 (0.0006) [2023-12-27 02:49:18,245][105620] Updated weights for policy 1, policy_version 1572103 (0.0010) [2023-12-27 02:49:18,912][105620] Updated weights for policy 1, policy_version 1572113 (0.0009) [2023-12-27 02:49:18,967][105620] Updated weights for policy 1, policy_version 1572123 (0.0010) [2023-12-27 02:49:19,016][105692] Updated weights for policy 0, policy_version 1568654 (0.0006) [2023-12-27 02:49:19,022][105620] Updated weights for policy 1, policy_version 1572133 (0.0008) [2023-12-27 02:49:19,071][105692] Updated weights for policy 0, policy_version 1568664 (0.0008) [2023-12-27 02:49:19,122][105692] Updated weights for policy 0, policy_version 1568674 (0.0009) [2023-12-27 02:49:19,847][105620] Updated weights for policy 1, policy_version 1572143 (0.0008) [2023-12-27 02:49:19,883][105692] Updated weights for policy 0, policy_version 1568684 (0.0009) [2023-12-27 02:49:19,912][105620] Updated weights for policy 1, policy_version 1572153 (0.0008) [2023-12-27 02:49:19,949][105692] Updated weights for policy 0, policy_version 1568694 (0.0007) [2023-12-27 02:49:19,967][105620] Updated weights for policy 1, policy_version 1572163 (0.0008) [2023-12-27 02:49:20,018][105692] Updated weights for policy 0, policy_version 1568704 (0.0007) [2023-12-27 02:49:20,709][105692] Updated weights for policy 0, policy_version 1568714 (0.0006) [2023-12-27 02:49:20,739][105620] Updated weights for policy 1, policy_version 1572173 (0.0008) [2023-12-27 02:49:20,774][105692] Updated weights for policy 0, policy_version 1568724 (0.0008) [2023-12-27 02:49:20,804][105620] Updated weights for policy 1, policy_version 1572183 (0.0010) [2023-12-27 02:49:20,835][105692] Updated weights for policy 0, policy_version 1568734 (0.0007) [2023-12-27 02:49:20,870][105620] Updated weights for policy 1, policy_version 1572193 (0.0009) [2023-12-27 02:49:20,898][105692] Updated weights for policy 0, policy_version 1568744 (0.0006) [2023-12-27 02:49:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 804192256. Throughput: 0: 9696.3, 1: 9828.6. Samples: 804177604. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:49:21,062][104569] Avg episode reward: [(0, '8718.776'), (1, '8807.550')] [2023-12-27 02:49:21,586][105692] Updated weights for policy 0, policy_version 1568754 (0.0011) [2023-12-27 02:49:21,652][105692] Updated weights for policy 0, policy_version 1568764 (0.0010) [2023-12-27 02:49:21,654][105620] Updated weights for policy 1, policy_version 1572203 (0.0008) [2023-12-27 02:49:21,713][105692] Updated weights for policy 0, policy_version 1568774 (0.0010) [2023-12-27 02:49:21,714][105620] Updated weights for policy 1, policy_version 1572213 (0.0006) [2023-12-27 02:49:21,772][105620] Updated weights for policy 1, policy_version 1572223 (0.0009) [2023-12-27 02:49:22,375][105620] Updated weights for policy 1, policy_version 1572233 (0.0006) [2023-12-27 02:49:22,447][105620] Updated weights for policy 1, policy_version 1572243 (0.0007) [2023-12-27 02:49:22,473][105692] Updated weights for policy 0, policy_version 1568784 (0.0006) [2023-12-27 02:49:22,505][105620] Updated weights for policy 1, policy_version 1572253 (0.0008) [2023-12-27 02:49:22,527][105692] Updated weights for policy 0, policy_version 1568794 (0.0006) [2023-12-27 02:49:22,567][105620] Updated weights for policy 1, policy_version 1572263 (0.0009) [2023-12-27 02:49:22,575][105692] Updated weights for policy 0, policy_version 1568804 (0.0006) [2023-12-27 02:49:23,129][105692] Updated weights for policy 0, policy_version 1568814 (0.0009) [2023-12-27 02:49:23,188][105692] Updated weights for policy 0, policy_version 1568824 (0.0010) [2023-12-27 02:49:23,243][105692] Updated weights for policy 0, policy_version 1568834 (0.0010) [2023-12-27 02:49:23,355][105620] Updated weights for policy 1, policy_version 1572273 (0.0010) [2023-12-27 02:49:23,402][105620] Updated weights for policy 1, policy_version 1572283 (0.0010) [2023-12-27 02:49:23,462][105620] Updated weights for policy 1, policy_version 1572293 (0.0010) [2023-12-27 02:49:23,842][105692] Updated weights for policy 0, policy_version 1568844 (0.0009) [2023-12-27 02:49:23,891][105692] Updated weights for policy 0, policy_version 1568854 (0.0005) [2023-12-27 02:49:23,948][105692] Updated weights for policy 0, policy_version 1568864 (0.0006) [2023-12-27 02:49:24,073][105620] Updated weights for policy 1, policy_version 1572303 (0.0007) [2023-12-27 02:49:24,142][105620] Updated weights for policy 1, policy_version 1572313 (0.0006) [2023-12-27 02:49:24,192][105620] Updated weights for policy 1, policy_version 1572323 (0.0007) [2023-12-27 02:49:24,534][105692] Updated weights for policy 0, policy_version 1568874 (0.0006) [2023-12-27 02:49:24,588][105692] Updated weights for policy 0, policy_version 1568884 (0.0009) [2023-12-27 02:49:24,657][105692] Updated weights for policy 0, policy_version 1568894 (0.0006) [2023-12-27 02:49:24,723][105692] Updated weights for policy 0, policy_version 1568904 (0.0011) [2023-12-27 02:49:24,846][105620] Updated weights for policy 1, policy_version 1572333 (0.0007) [2023-12-27 02:49:24,898][105620] Updated weights for policy 1, policy_version 1572343 (0.0006) [2023-12-27 02:49:24,953][105620] Updated weights for policy 1, policy_version 1572353 (0.0005) [2023-12-27 02:49:25,285][105692] Updated weights for policy 0, policy_version 1568914 (0.0005) [2023-12-27 02:49:25,333][105692] Updated weights for policy 0, policy_version 1568924 (0.0005) [2023-12-27 02:49:25,385][105692] Updated weights for policy 0, policy_version 1568934 (0.0005) [2023-12-27 02:49:25,587][105620] Updated weights for policy 1, policy_version 1572363 (0.0006) [2023-12-27 02:49:25,636][105620] Updated weights for policy 1, policy_version 1572373 (0.0008) [2023-12-27 02:49:25,683][105620] Updated weights for policy 1, policy_version 1572383 (0.0008) [2023-12-27 02:49:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 804290560. Throughput: 0: 9805.2, 1: 9837.7. Samples: 804300780. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:49:26,063][104569] Avg episode reward: [(0, '8622.186'), (1, '8989.846')] [2023-12-27 02:49:26,071][105692] Updated weights for policy 0, policy_version 1568944 (0.0006) [2023-12-27 02:49:26,126][105692] Updated weights for policy 0, policy_version 1568954 (0.0011) [2023-12-27 02:49:26,179][105692] Updated weights for policy 0, policy_version 1568964 (0.0010) [2023-12-27 02:49:26,384][105620] Updated weights for policy 1, policy_version 1572393 (0.0008) [2023-12-27 02:49:26,431][105620] Updated weights for policy 1, policy_version 1572403 (0.0006) [2023-12-27 02:49:26,475][105620] Updated weights for policy 1, policy_version 1572413 (0.0008) [2023-12-27 02:49:26,523][105620] Updated weights for policy 1, policy_version 1572423 (0.0008) [2023-12-27 02:49:26,898][105692] Updated weights for policy 0, policy_version 1568974 (0.0010) [2023-12-27 02:49:26,955][105692] Updated weights for policy 0, policy_version 1568984 (0.0010) [2023-12-27 02:49:27,019][105692] Updated weights for policy 0, policy_version 1568994 (0.0010) [2023-12-27 02:49:27,267][105620] Updated weights for policy 1, policy_version 1572433 (0.0005) [2023-12-27 02:49:27,321][105620] Updated weights for policy 1, policy_version 1572443 (0.0006) [2023-12-27 02:49:27,374][105620] Updated weights for policy 1, policy_version 1572453 (0.0008) [2023-12-27 02:49:27,708][105692] Updated weights for policy 0, policy_version 1569004 (0.0010) [2023-12-27 02:49:27,755][105692] Updated weights for policy 0, policy_version 1569014 (0.0010) [2023-12-27 02:49:27,803][105692] Updated weights for policy 0, policy_version 1569024 (0.0010) [2023-12-27 02:49:28,051][105620] Updated weights for policy 1, policy_version 1572463 (0.0006) [2023-12-27 02:49:28,109][105620] Updated weights for policy 1, policy_version 1572473 (0.0006) [2023-12-27 02:49:28,171][105620] Updated weights for policy 1, policy_version 1572483 (0.0005) [2023-12-27 02:49:28,562][105692] Updated weights for policy 0, policy_version 1569034 (0.0010) [2023-12-27 02:49:28,611][105692] Updated weights for policy 0, policy_version 1569044 (0.0011) [2023-12-27 02:49:28,656][105692] Updated weights for policy 0, policy_version 1569054 (0.0010) [2023-12-27 02:49:28,701][105692] Updated weights for policy 0, policy_version 1569064 (0.0009) [2023-12-27 02:49:28,802][105620] Updated weights for policy 1, policy_version 1572493 (0.0007) [2023-12-27 02:49:28,851][105620] Updated weights for policy 1, policy_version 1572503 (0.0008) [2023-12-27 02:49:28,913][105620] Updated weights for policy 1, policy_version 1572513 (0.0009) [2023-12-27 02:49:29,452][105692] Updated weights for policy 0, policy_version 1569074 (0.0011) [2023-12-27 02:49:29,512][105692] Updated weights for policy 0, policy_version 1569084 (0.0011) [2023-12-27 02:49:29,568][105692] Updated weights for policy 0, policy_version 1569094 (0.0009) [2023-12-27 02:49:29,638][105620] Updated weights for policy 1, policy_version 1572523 (0.0007) [2023-12-27 02:49:29,689][105620] Updated weights for policy 1, policy_version 1572533 (0.0005) [2023-12-27 02:49:29,751][105620] Updated weights for policy 1, policy_version 1572543 (0.0005) [2023-12-27 02:49:30,342][105692] Updated weights for policy 0, policy_version 1569104 (0.0009) [2023-12-27 02:49:30,404][105692] Updated weights for policy 0, policy_version 1569114 (0.0009) [2023-12-27 02:49:30,432][105620] Updated weights for policy 1, policy_version 1572553 (0.0006) [2023-12-27 02:49:30,457][105692] Updated weights for policy 0, policy_version 1569125 (0.0008) [2023-12-27 02:49:30,492][105620] Updated weights for policy 1, policy_version 1572563 (0.0007) [2023-12-27 02:49:30,536][105620] Updated weights for policy 1, policy_version 1572573 (0.0006) [2023-12-27 02:49:30,587][105620] Updated weights for policy 1, policy_version 1572583 (0.0005) [2023-12-27 02:49:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 804388864. Throughput: 0: 9875.5, 1: 9842.8. Samples: 804360752. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:49:31,063][104569] Avg episode reward: [(0, '8715.396'), (1, '8989.348')] [2023-12-27 02:49:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001572584_402636800.pth... [2023-12-27 02:49:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001571432_402341888.pth [2023-12-27 02:49:31,093][105692] Updated weights for policy 0, policy_version 1569135 (0.0009) [2023-12-27 02:49:31,160][105692] Updated weights for policy 0, policy_version 1569145 (0.0010) [2023-12-27 02:49:31,161][105620] Updated weights for policy 1, policy_version 1572593 (0.0007) [2023-12-27 02:49:31,213][105620] Updated weights for policy 1, policy_version 1572603 (0.0005) [2023-12-27 02:49:31,219][105692] Updated weights for policy 0, policy_version 1569155 (0.0011) [2023-12-27 02:49:31,249][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001569160_401760256.pth... [2023-12-27 02:49:31,254][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001567976_401457152.pth [2023-12-27 02:49:31,267][105620] Updated weights for policy 1, policy_version 1572613 (0.0006) [2023-12-27 02:49:31,932][105620] Updated weights for policy 1, policy_version 1572623 (0.0005) [2023-12-27 02:49:31,936][105692] Updated weights for policy 0, policy_version 1569165 (0.0008) [2023-12-27 02:49:31,991][105620] Updated weights for policy 1, policy_version 1572633 (0.0005) [2023-12-27 02:49:31,997][105692] Updated weights for policy 0, policy_version 1569175 (0.0006) [2023-12-27 02:49:32,042][105620] Updated weights for policy 1, policy_version 1572643 (0.0006) [2023-12-27 02:49:32,055][105692] Updated weights for policy 0, policy_version 1569185 (0.0006) [2023-12-27 02:49:32,660][105620] Updated weights for policy 1, policy_version 1572653 (0.0008) [2023-12-27 02:49:32,702][105692] Updated weights for policy 0, policy_version 1569195 (0.0009) [2023-12-27 02:49:32,716][105620] Updated weights for policy 1, policy_version 1572663 (0.0010) [2023-12-27 02:49:32,754][105692] Updated weights for policy 0, policy_version 1569205 (0.0010) [2023-12-27 02:49:32,773][105620] Updated weights for policy 1, policy_version 1572673 (0.0010) [2023-12-27 02:49:32,805][105692] Updated weights for policy 0, policy_version 1569215 (0.0010) [2023-12-27 02:49:33,399][105692] Updated weights for policy 0, policy_version 1569225 (0.0011) [2023-12-27 02:49:33,464][105692] Updated weights for policy 0, policy_version 1569235 (0.0011) [2023-12-27 02:49:33,515][105620] Updated weights for policy 1, policy_version 1572683 (0.0010) [2023-12-27 02:49:33,522][105692] Updated weights for policy 0, policy_version 1569245 (0.0010) [2023-12-27 02:49:33,569][105620] Updated weights for policy 1, policy_version 1572693 (0.0010) [2023-12-27 02:49:33,578][105692] Updated weights for policy 0, policy_version 1569255 (0.0009) [2023-12-27 02:49:33,630][105620] Updated weights for policy 1, policy_version 1572703 (0.0009) [2023-12-27 02:49:34,215][105692] Updated weights for policy 0, policy_version 1569265 (0.0007) [2023-12-27 02:49:34,264][105620] Updated weights for policy 1, policy_version 1572713 (0.0006) [2023-12-27 02:49:34,281][105692] Updated weights for policy 0, policy_version 1569275 (0.0010) [2023-12-27 02:49:34,326][105620] Updated weights for policy 1, policy_version 1572723 (0.0007) [2023-12-27 02:49:34,351][105692] Updated weights for policy 0, policy_version 1569285 (0.0007) [2023-12-27 02:49:34,390][105620] Updated weights for policy 1, policy_version 1572733 (0.0007) [2023-12-27 02:49:34,450][105620] Updated weights for policy 1, policy_version 1572743 (0.0005) [2023-12-27 02:49:34,952][105692] Updated weights for policy 0, policy_version 1569295 (0.0006) [2023-12-27 02:49:35,017][105692] Updated weights for policy 0, policy_version 1569305 (0.0005) [2023-12-27 02:49:35,027][105620] Updated weights for policy 1, policy_version 1572753 (0.0010) [2023-12-27 02:49:35,069][105692] Updated weights for policy 0, policy_version 1569315 (0.0005) [2023-12-27 02:49:35,082][105620] Updated weights for policy 1, policy_version 1572763 (0.0010) [2023-12-27 02:49:35,134][105620] Updated weights for policy 1, policy_version 1572773 (0.0010) [2023-12-27 02:49:35,634][105692] Updated weights for policy 0, policy_version 1569325 (0.0005) [2023-12-27 02:49:35,709][105692] Updated weights for policy 0, policy_version 1569335 (0.0010) [2023-12-27 02:49:35,770][105692] Updated weights for policy 0, policy_version 1569345 (0.0008) [2023-12-27 02:49:35,866][105620] Updated weights for policy 1, policy_version 1572783 (0.0009) [2023-12-27 02:49:35,924][105620] Updated weights for policy 1, policy_version 1572793 (0.0008) [2023-12-27 02:49:35,981][105620] Updated weights for policy 1, policy_version 1572803 (0.0007) [2023-12-27 02:49:36,062][104569] Fps is (10 sec: 21299.1, 60 sec: 19797.3, 300 sec: 19605.2). Total num frames: 804503552. Throughput: 0: 9869.6, 1: 9929.5. Samples: 804485916. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:49:36,063][104569] Avg episode reward: [(0, '8445.445'), (1, '8898.255')] [2023-12-27 02:49:36,393][105692] Updated weights for policy 0, policy_version 1569355 (0.0010) [2023-12-27 02:49:36,456][105692] Updated weights for policy 0, policy_version 1569365 (0.0011) [2023-12-27 02:49:36,515][105692] Updated weights for policy 0, policy_version 1569375 (0.0011) [2023-12-27 02:49:36,771][105620] Updated weights for policy 1, policy_version 1572813 (0.0007) [2023-12-27 02:49:36,831][105620] Updated weights for policy 1, policy_version 1572823 (0.0008) [2023-12-27 02:49:36,886][105620] Updated weights for policy 1, policy_version 1572833 (0.0008) [2023-12-27 02:49:37,211][105692] Updated weights for policy 0, policy_version 1569385 (0.0007) [2023-12-27 02:49:37,259][105692] Updated weights for policy 0, policy_version 1569395 (0.0010) [2023-12-27 02:49:37,309][105692] Updated weights for policy 0, policy_version 1569405 (0.0009) [2023-12-27 02:49:37,354][105692] Updated weights for policy 0, policy_version 1569415 (0.0010) [2023-12-27 02:49:37,568][105620] Updated weights for policy 1, policy_version 1572843 (0.0009) [2023-12-27 02:49:37,628][105620] Updated weights for policy 1, policy_version 1572853 (0.0008) [2023-12-27 02:49:37,683][105620] Updated weights for policy 1, policy_version 1572863 (0.0007) [2023-12-27 02:49:38,119][105692] Updated weights for policy 0, policy_version 1569425 (0.0006) [2023-12-27 02:49:38,179][105692] Updated weights for policy 0, policy_version 1569435 (0.0005) [2023-12-27 02:49:38,231][105692] Updated weights for policy 0, policy_version 1569445 (0.0005) [2023-12-27 02:49:38,273][105620] Updated weights for policy 1, policy_version 1572873 (0.0007) [2023-12-27 02:49:38,335][105620] Updated weights for policy 1, policy_version 1572883 (0.0009) [2023-12-27 02:49:38,395][105620] Updated weights for policy 1, policy_version 1572893 (0.0009) [2023-12-27 02:49:38,459][105620] Updated weights for policy 1, policy_version 1572903 (0.0008) [2023-12-27 02:49:38,933][105692] Updated weights for policy 0, policy_version 1569455 (0.0008) [2023-12-27 02:49:38,996][105692] Updated weights for policy 0, policy_version 1569465 (0.0007) [2023-12-27 02:49:39,061][105692] Updated weights for policy 0, policy_version 1569475 (0.0008) [2023-12-27 02:49:39,197][105620] Updated weights for policy 1, policy_version 1572913 (0.0009) [2023-12-27 02:49:39,259][105620] Updated weights for policy 1, policy_version 1572923 (0.0009) [2023-12-27 02:49:39,316][105620] Updated weights for policy 1, policy_version 1572933 (0.0010) [2023-12-27 02:49:39,735][105692] Updated weights for policy 0, policy_version 1569485 (0.0006) [2023-12-27 02:49:39,800][105692] Updated weights for policy 0, policy_version 1569495 (0.0008) [2023-12-27 02:49:39,865][105692] Updated weights for policy 0, policy_version 1569505 (0.0008) [2023-12-27 02:49:40,122][105620] Updated weights for policy 1, policy_version 1572943 (0.0008) [2023-12-27 02:49:40,173][105620] Updated weights for policy 1, policy_version 1572953 (0.0009) [2023-12-27 02:49:40,232][105620] Updated weights for policy 1, policy_version 1572963 (0.0009) [2023-12-27 02:49:40,237][105586] KL-divergence is very high: 180.9200 [2023-12-27 02:49:40,581][105692] Updated weights for policy 0, policy_version 1569515 (0.0009) [2023-12-27 02:49:40,638][105692] Updated weights for policy 0, policy_version 1569525 (0.0009) [2023-12-27 02:49:40,701][105692] Updated weights for policy 0, policy_version 1569535 (0.0009) [2023-12-27 02:49:40,962][105586] KL-divergence is very high: 120.6750 [2023-12-27 02:49:40,990][105620] Updated weights for policy 1, policy_version 1572973 (0.0008) [2023-12-27 02:49:41,009][105586] KL-divergence is very high: 115.6865 [2023-12-27 02:49:41,050][105620] Updated weights for policy 1, policy_version 1572983 (0.0009) [2023-12-27 02:49:41,057][105586] KL-divergence is very high: 100.5976 [2023-12-27 02:49:41,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 804593664. Throughput: 0: 9939.7, 1: 9871.0. Samples: 804605148. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:49:41,063][104569] Avg episode reward: [(0, '8810.693'), (1, '8806.315')] [2023-12-27 02:49:41,111][105620] Updated weights for policy 1, policy_version 1572993 (0.0009) [2023-12-27 02:49:41,446][105692] Updated weights for policy 0, policy_version 1569545 (0.0008) [2023-12-27 02:49:41,513][105692] Updated weights for policy 0, policy_version 1569555 (0.0009) [2023-12-27 02:49:41,566][105692] Updated weights for policy 0, policy_version 1569565 (0.0008) [2023-12-27 02:49:41,626][105692] Updated weights for policy 0, policy_version 1569575 (0.0008) [2023-12-27 02:49:41,904][105620] Updated weights for policy 1, policy_version 1573003 (0.0008) [2023-12-27 02:49:41,970][105620] Updated weights for policy 1, policy_version 1573013 (0.0007) [2023-12-27 02:49:42,033][105620] Updated weights for policy 1, policy_version 1573023 (0.0009) [2023-12-27 02:49:42,445][105692] Updated weights for policy 0, policy_version 1569585 (0.0010) [2023-12-27 02:49:42,517][105692] Updated weights for policy 0, policy_version 1569595 (0.0010) [2023-12-27 02:49:42,584][105692] Updated weights for policy 0, policy_version 1569605 (0.0010) [2023-12-27 02:49:42,713][105620] Updated weights for policy 1, policy_version 1573033 (0.0009) [2023-12-27 02:49:42,777][105620] Updated weights for policy 1, policy_version 1573043 (0.0008) [2023-12-27 02:49:42,826][105620] Updated weights for policy 1, policy_version 1573053 (0.0005) [2023-12-27 02:49:42,895][105620] Updated weights for policy 1, policy_version 1573063 (0.0005) [2023-12-27 02:49:43,444][105692] Updated weights for policy 0, policy_version 1569615 (0.0007) [2023-12-27 02:49:43,450][105620] Updated weights for policy 1, policy_version 1573073 (0.0007) [2023-12-27 02:49:43,504][105620] Updated weights for policy 1, policy_version 1573083 (0.0008) [2023-12-27 02:49:43,506][105692] Updated weights for policy 0, policy_version 1569625 (0.0006) [2023-12-27 02:49:43,560][105620] Updated weights for policy 1, policy_version 1573093 (0.0009) [2023-12-27 02:49:43,560][105692] Updated weights for policy 0, policy_version 1569635 (0.0005) [2023-12-27 02:49:44,148][105620] Updated weights for policy 1, policy_version 1573103 (0.0006) [2023-12-27 02:49:44,202][105620] Updated weights for policy 1, policy_version 1573113 (0.0005) [2023-12-27 02:49:44,236][105692] Updated weights for policy 0, policy_version 1569645 (0.0007) [2023-12-27 02:49:44,258][105620] Updated weights for policy 1, policy_version 1573123 (0.0009) [2023-12-27 02:49:44,298][105692] Updated weights for policy 0, policy_version 1569655 (0.0007) [2023-12-27 02:49:44,361][105692] Updated weights for policy 0, policy_version 1569665 (0.0009) [2023-12-27 02:49:44,848][105620] Updated weights for policy 1, policy_version 1573133 (0.0008) [2023-12-27 02:49:44,910][105620] Updated weights for policy 1, policy_version 1573143 (0.0008) [2023-12-27 02:49:44,982][105620] Updated weights for policy 1, policy_version 1573153 (0.0008) [2023-12-27 02:49:45,074][105692] Updated weights for policy 0, policy_version 1569675 (0.0010) [2023-12-27 02:49:45,141][105692] Updated weights for policy 0, policy_version 1569685 (0.0011) [2023-12-27 02:49:45,209][105692] Updated weights for policy 0, policy_version 1569695 (0.0011) [2023-12-27 02:49:45,593][105620] Updated weights for policy 1, policy_version 1573163 (0.0007) [2023-12-27 02:49:45,653][105620] Updated weights for policy 1, policy_version 1573173 (0.0006) [2023-12-27 02:49:45,717][105620] Updated weights for policy 1, policy_version 1573183 (0.0005) [2023-12-27 02:49:45,865][105692] Updated weights for policy 0, policy_version 1569705 (0.0010) [2023-12-27 02:49:45,922][105692] Updated weights for policy 0, policy_version 1569715 (0.0006) [2023-12-27 02:49:45,969][105692] Updated weights for policy 0, policy_version 1569725 (0.0005) [2023-12-27 02:49:46,015][105692] Updated weights for policy 0, policy_version 1569735 (0.0005) [2023-12-27 02:49:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.8, 300 sec: 19605.3). Total num frames: 804700160. Throughput: 0: 9890.5, 1: 9929.4. Samples: 804662060. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:49:46,063][104569] Avg episode reward: [(0, '8810.718'), (1, '8808.775')] [2023-12-27 02:49:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001569736_401907712.pth... [2023-12-27 02:49:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001573192_402792448.pth... [2023-12-27 02:49:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001568552_401604608.pth [2023-12-27 02:49:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001572008_402489344.pth [2023-12-27 02:49:46,250][105620] Updated weights for policy 1, policy_version 1573193 (0.0005) [2023-12-27 02:49:46,298][105620] Updated weights for policy 1, policy_version 1573203 (0.0008) [2023-12-27 02:49:46,355][105620] Updated weights for policy 1, policy_version 1573213 (0.0008) [2023-12-27 02:49:46,414][105620] Updated weights for policy 1, policy_version 1573223 (0.0008) [2023-12-27 02:49:46,670][105692] Updated weights for policy 0, policy_version 1569745 (0.0010) [2023-12-27 02:49:46,724][105692] Updated weights for policy 0, policy_version 1569755 (0.0010) [2023-12-27 02:49:46,781][105692] Updated weights for policy 0, policy_version 1569765 (0.0010) [2023-12-27 02:49:47,159][105620] Updated weights for policy 1, policy_version 1573233 (0.0006) [2023-12-27 02:49:47,220][105620] Updated weights for policy 1, policy_version 1573243 (0.0008) [2023-12-27 02:49:47,264][105620] Updated weights for policy 1, policy_version 1573253 (0.0008) [2023-12-27 02:49:47,431][105692] Updated weights for policy 0, policy_version 1569775 (0.0010) [2023-12-27 02:49:47,495][105692] Updated weights for policy 0, policy_version 1569785 (0.0010) [2023-12-27 02:49:47,550][105692] Updated weights for policy 0, policy_version 1569795 (0.0010) [2023-12-27 02:49:47,884][105620] Updated weights for policy 1, policy_version 1573263 (0.0006) [2023-12-27 02:49:47,941][105620] Updated weights for policy 1, policy_version 1573273 (0.0005) [2023-12-27 02:49:48,000][105620] Updated weights for policy 1, policy_version 1573283 (0.0005) [2023-12-27 02:49:48,171][105692] Updated weights for policy 0, policy_version 1569805 (0.0010) [2023-12-27 02:49:48,229][105692] Updated weights for policy 0, policy_version 1569815 (0.0010) [2023-12-27 02:49:48,284][105692] Updated weights for policy 0, policy_version 1569825 (0.0010) [2023-12-27 02:49:48,597][105620] Updated weights for policy 1, policy_version 1573293 (0.0006) [2023-12-27 02:49:48,651][105620] Updated weights for policy 1, policy_version 1573303 (0.0006) [2023-12-27 02:49:48,705][105620] Updated weights for policy 1, policy_version 1573313 (0.0006) [2023-12-27 02:49:49,073][105692] Updated weights for policy 0, policy_version 1569835 (0.0009) [2023-12-27 02:49:49,125][105692] Updated weights for policy 0, policy_version 1569845 (0.0005) [2023-12-27 02:49:49,194][105692] Updated weights for policy 0, policy_version 1569855 (0.0010) [2023-12-27 02:49:49,233][105620] Updated weights for policy 1, policy_version 1573323 (0.0006) [2023-12-27 02:49:49,296][105620] Updated weights for policy 1, policy_version 1573333 (0.0006) [2023-12-27 02:49:49,367][105620] Updated weights for policy 1, policy_version 1573343 (0.0008) [2023-12-27 02:49:49,933][105692] Updated weights for policy 0, policy_version 1569865 (0.0010) [2023-12-27 02:49:50,010][105692] Updated weights for policy 0, policy_version 1569875 (0.0011) [2023-12-27 02:49:50,079][105620] Updated weights for policy 1, policy_version 1573353 (0.0008) [2023-12-27 02:49:50,080][105692] Updated weights for policy 0, policy_version 1569885 (0.0011) [2023-12-27 02:49:50,140][105692] Updated weights for policy 0, policy_version 1569895 (0.0007) [2023-12-27 02:49:50,144][105620] Updated weights for policy 1, policy_version 1573363 (0.0008) [2023-12-27 02:49:50,204][105620] Updated weights for policy 1, policy_version 1573373 (0.0008) [2023-12-27 02:49:50,258][105620] Updated weights for policy 1, policy_version 1573383 (0.0010) [2023-12-27 02:49:50,793][105692] Updated weights for policy 0, policy_version 1569905 (0.0010) [2023-12-27 02:49:50,858][105692] Updated weights for policy 0, policy_version 1569915 (0.0011) [2023-12-27 02:49:50,925][105692] Updated weights for policy 0, policy_version 1569925 (0.0006) [2023-12-27 02:49:51,042][105620] Updated weights for policy 1, policy_version 1573393 (0.0006) [2023-12-27 02:49:51,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 804798464. Throughput: 0: 9939.9, 1: 10122.0. Samples: 804788760. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:49:51,062][104569] Avg episode reward: [(0, '8533.793'), (1, '8544.331')] [2023-12-27 02:49:51,111][105620] Updated weights for policy 1, policy_version 1573403 (0.0009) [2023-12-27 02:49:51,178][105620] Updated weights for policy 1, policy_version 1573413 (0.0008) [2023-12-27 02:49:51,619][105692] Updated weights for policy 0, policy_version 1569935 (0.0009) [2023-12-27 02:49:51,678][105692] Updated weights for policy 0, policy_version 1569945 (0.0011) [2023-12-27 02:49:51,746][105692] Updated weights for policy 0, policy_version 1569955 (0.0011) [2023-12-27 02:49:51,931][105620] Updated weights for policy 1, policy_version 1573423 (0.0008) [2023-12-27 02:49:51,983][105620] Updated weights for policy 1, policy_version 1573433 (0.0008) [2023-12-27 02:49:52,039][105620] Updated weights for policy 1, policy_version 1573443 (0.0008) [2023-12-27 02:49:52,504][105692] Updated weights for policy 0, policy_version 1569965 (0.0008) [2023-12-27 02:49:52,562][105692] Updated weights for policy 0, policy_version 1569975 (0.0010) [2023-12-27 02:49:52,618][105692] Updated weights for policy 0, policy_version 1569985 (0.0009) [2023-12-27 02:49:52,843][105620] Updated weights for policy 1, policy_version 1573453 (0.0009) [2023-12-27 02:49:52,898][105620] Updated weights for policy 1, policy_version 1573463 (0.0009) [2023-12-27 02:49:52,948][105620] Updated weights for policy 1, policy_version 1573473 (0.0009) [2023-12-27 02:49:53,346][105692] Updated weights for policy 0, policy_version 1569995 (0.0008) [2023-12-27 02:49:53,412][105692] Updated weights for policy 0, policy_version 1570005 (0.0005) [2023-12-27 02:49:53,478][105692] Updated weights for policy 0, policy_version 1570015 (0.0005) [2023-12-27 02:49:53,794][105620] Updated weights for policy 1, policy_version 1573483 (0.0009) [2023-12-27 02:49:53,848][105620] Updated weights for policy 1, policy_version 1573493 (0.0009) [2023-12-27 02:49:53,898][105620] Updated weights for policy 1, policy_version 1573503 (0.0009) [2023-12-27 02:49:54,041][105692] Updated weights for policy 0, policy_version 1570025 (0.0005) [2023-12-27 02:49:54,112][105692] Updated weights for policy 0, policy_version 1570035 (0.0006) [2023-12-27 02:49:54,178][105692] Updated weights for policy 0, policy_version 1570045 (0.0009) [2023-12-27 02:49:54,248][105692] Updated weights for policy 0, policy_version 1570055 (0.0009) [2023-12-27 02:49:54,718][105620] Updated weights for policy 1, policy_version 1573513 (0.0009) [2023-12-27 02:49:54,786][105620] Updated weights for policy 1, policy_version 1573523 (0.0008) [2023-12-27 02:49:54,851][105620] Updated weights for policy 1, policy_version 1573533 (0.0007) [2023-12-27 02:49:54,881][105692] Updated weights for policy 0, policy_version 1570065 (0.0010) [2023-12-27 02:49:54,911][105620] Updated weights for policy 1, policy_version 1573543 (0.0005) [2023-12-27 02:49:54,929][105692] Updated weights for policy 0, policy_version 1570075 (0.0010) [2023-12-27 02:49:54,981][105692] Updated weights for policy 0, policy_version 1570085 (0.0010) [2023-12-27 02:49:55,605][105692] Updated weights for policy 0, policy_version 1570095 (0.0005) [2023-12-27 02:49:55,669][105692] Updated weights for policy 0, policy_version 1570105 (0.0007) [2023-12-27 02:49:55,727][105692] Updated weights for policy 0, policy_version 1570115 (0.0010) [2023-12-27 02:49:55,738][105620] Updated weights for policy 1, policy_version 1573553 (0.0006) [2023-12-27 02:49:55,788][105620] Updated weights for policy 1, policy_version 1573563 (0.0008) [2023-12-27 02:49:55,854][105620] Updated weights for policy 1, policy_version 1573573 (0.0010) [2023-12-27 02:49:56,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19933.7, 300 sec: 19605.2). Total num frames: 804896768. Throughput: 0: 10031.3, 1: 9950.6. Samples: 804902220. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:49:56,063][104569] Avg episode reward: [(0, '8710.867'), (1, '8544.006')] [2023-12-27 02:49:56,277][105692] Updated weights for policy 0, policy_version 1570125 (0.0008) [2023-12-27 02:49:56,340][105692] Updated weights for policy 0, policy_version 1570135 (0.0007) [2023-12-27 02:49:56,403][105692] Updated weights for policy 0, policy_version 1570145 (0.0009) [2023-12-27 02:49:56,687][105620] Updated weights for policy 1, policy_version 1573583 (0.0009) [2023-12-27 02:49:56,739][105620] Updated weights for policy 1, policy_version 1573593 (0.0010) [2023-12-27 02:49:56,807][105620] Updated weights for policy 1, policy_version 1573605 (0.0012) [2023-12-27 02:49:57,054][105692] Updated weights for policy 0, policy_version 1570155 (0.0009) [2023-12-27 02:49:57,102][105692] Updated weights for policy 0, policy_version 1570165 (0.0008) [2023-12-27 02:49:57,153][105692] Updated weights for policy 0, policy_version 1570175 (0.0008) [2023-12-27 02:49:57,638][105620] Updated weights for policy 1, policy_version 1573615 (0.0010) [2023-12-27 02:49:57,695][105620] Updated weights for policy 1, policy_version 1573625 (0.0010) [2023-12-27 02:49:57,746][105620] Updated weights for policy 1, policy_version 1573635 (0.0010) [2023-12-27 02:49:57,763][105692] Updated weights for policy 0, policy_version 1570185 (0.0007) [2023-12-27 02:49:57,820][105692] Updated weights for policy 0, policy_version 1570195 (0.0005) [2023-12-27 02:49:57,889][105692] Updated weights for policy 0, policy_version 1570205 (0.0005) [2023-12-27 02:49:57,951][105692] Updated weights for policy 0, policy_version 1570215 (0.0006) [2023-12-27 02:49:58,453][105620] Updated weights for policy 1, policy_version 1573645 (0.0010) [2023-12-27 02:49:58,521][105620] Updated weights for policy 1, policy_version 1573655 (0.0011) [2023-12-27 02:49:58,589][105620] Updated weights for policy 1, policy_version 1573665 (0.0010) [2023-12-27 02:49:58,613][105692] Updated weights for policy 0, policy_version 1570225 (0.0007) [2023-12-27 02:49:58,677][105692] Updated weights for policy 0, policy_version 1570235 (0.0008) [2023-12-27 02:49:58,741][105692] Updated weights for policy 0, policy_version 1570245 (0.0010) [2023-12-27 02:49:59,481][105620] Updated weights for policy 1, policy_version 1573675 (0.0009) [2023-12-27 02:49:59,522][105692] Updated weights for policy 0, policy_version 1570255 (0.0010) [2023-12-27 02:49:59,540][105620] Updated weights for policy 1, policy_version 1573685 (0.0008) [2023-12-27 02:49:59,578][105692] Updated weights for policy 0, policy_version 1570265 (0.0007) [2023-12-27 02:49:59,593][105620] Updated weights for policy 1, policy_version 1573695 (0.0006) [2023-12-27 02:49:59,628][105692] Updated weights for policy 0, policy_version 1570275 (0.0006) [2023-12-27 02:50:00,306][105620] Updated weights for policy 1, policy_version 1573705 (0.0008) [2023-12-27 02:50:00,367][105620] Updated weights for policy 1, policy_version 1573715 (0.0006) [2023-12-27 02:50:00,400][105692] Updated weights for policy 0, policy_version 1570285 (0.0008) [2023-12-27 02:50:00,423][105620] Updated weights for policy 1, policy_version 1573725 (0.0008) [2023-12-27 02:50:00,465][105692] Updated weights for policy 0, policy_version 1570295 (0.0007) [2023-12-27 02:50:00,488][105620] Updated weights for policy 1, policy_version 1573735 (0.0010) [2023-12-27 02:50:00,528][105692] Updated weights for policy 0, policy_version 1570305 (0.0008) [2023-12-27 02:50:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 804986880. Throughput: 0: 10117.8, 1: 9935.3. Samples: 804961820. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:50:01,063][104569] Avg episode reward: [(0, '8715.135'), (1, '8995.287')] [2023-12-27 02:50:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001570312_402055168.pth... [2023-12-27 02:50:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001573736_402931712.pth... [2023-12-27 02:50:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001569160_401760256.pth [2023-12-27 02:50:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001572584_402636800.pth [2023-12-27 02:50:01,179][105620] Updated weights for policy 1, policy_version 1573745 (0.0010) [2023-12-27 02:50:01,226][105692] Updated weights for policy 0, policy_version 1570315 (0.0007) [2023-12-27 02:50:01,243][105620] Updated weights for policy 1, policy_version 1573755 (0.0009) [2023-12-27 02:50:01,291][105692] Updated weights for policy 0, policy_version 1570325 (0.0006) [2023-12-27 02:50:01,296][105620] Updated weights for policy 1, policy_version 1573765 (0.0009) [2023-12-27 02:50:01,362][105692] Updated weights for policy 0, policy_version 1570335 (0.0006) [2023-12-27 02:50:02,038][105620] Updated weights for policy 1, policy_version 1573775 (0.0009) [2023-12-27 02:50:02,047][105692] Updated weights for policy 0, policy_version 1570345 (0.0008) [2023-12-27 02:50:02,087][105620] Updated weights for policy 1, policy_version 1573785 (0.0009) [2023-12-27 02:50:02,101][105692] Updated weights for policy 0, policy_version 1570355 (0.0006) [2023-12-27 02:50:02,134][105620] Updated weights for policy 1, policy_version 1573795 (0.0009) [2023-12-27 02:50:02,157][105692] Updated weights for policy 0, policy_version 1570365 (0.0005) [2023-12-27 02:50:02,208][105692] Updated weights for policy 0, policy_version 1570375 (0.0006) [2023-12-27 02:50:02,856][105692] Updated weights for policy 0, policy_version 1570385 (0.0009) [2023-12-27 02:50:02,914][105692] Updated weights for policy 0, policy_version 1570395 (0.0009) [2023-12-27 02:50:02,939][105620] Updated weights for policy 1, policy_version 1573805 (0.0008) [2023-12-27 02:50:02,970][105692] Updated weights for policy 0, policy_version 1570405 (0.0008) [2023-12-27 02:50:02,993][105620] Updated weights for policy 1, policy_version 1573815 (0.0007) [2023-12-27 02:50:03,053][105620] Updated weights for policy 1, policy_version 1573825 (0.0009) [2023-12-27 02:50:03,724][105692] Updated weights for policy 0, policy_version 1570415 (0.0008) [2023-12-27 02:50:03,787][105692] Updated weights for policy 0, policy_version 1570425 (0.0010) [2023-12-27 02:50:03,814][105620] Updated weights for policy 1, policy_version 1573835 (0.0008) [2023-12-27 02:50:03,850][105692] Updated weights for policy 0, policy_version 1570435 (0.0007) [2023-12-27 02:50:03,880][105620] Updated weights for policy 1, policy_version 1573845 (0.0009) [2023-12-27 02:50:03,943][105620] Updated weights for policy 1, policy_version 1573855 (0.0008) [2023-12-27 02:50:04,482][105692] Updated weights for policy 0, policy_version 1570445 (0.0007) [2023-12-27 02:50:04,553][105692] Updated weights for policy 0, policy_version 1570455 (0.0009) [2023-12-27 02:50:04,615][105692] Updated weights for policy 0, policy_version 1570465 (0.0009) [2023-12-27 02:50:04,771][105620] Updated weights for policy 1, policy_version 1573865 (0.0008) [2023-12-27 02:50:04,838][105620] Updated weights for policy 1, policy_version 1573875 (0.0009) [2023-12-27 02:50:04,903][105620] Updated weights for policy 1, policy_version 1573886 (0.0009) [2023-12-27 02:50:04,957][105620] Updated weights for policy 1, policy_version 1573896 (0.0010) [2023-12-27 02:50:05,207][105692] Updated weights for policy 0, policy_version 1570475 (0.0007) [2023-12-27 02:50:05,259][105692] Updated weights for policy 0, policy_version 1570485 (0.0007) [2023-12-27 02:50:05,309][105692] Updated weights for policy 0, policy_version 1570495 (0.0005) [2023-12-27 02:50:05,813][105620] Updated weights for policy 1, policy_version 1573906 (0.0010) [2023-12-27 02:50:05,876][105692] Updated weights for policy 0, policy_version 1570505 (0.0006) [2023-12-27 02:50:05,877][105620] Updated weights for policy 1, policy_version 1573916 (0.0008) [2023-12-27 02:50:05,929][105692] Updated weights for policy 0, policy_version 1570515 (0.0006) [2023-12-27 02:50:05,931][105620] Updated weights for policy 1, policy_version 1573926 (0.0008) [2023-12-27 02:50:05,985][105692] Updated weights for policy 0, policy_version 1570525 (0.0008) [2023-12-27 02:50:06,039][105692] Updated weights for policy 0, policy_version 1570535 (0.0009) [2023-12-27 02:50:06,062][104569] Fps is (10 sec: 19661.6, 60 sec: 19933.9, 300 sec: 19549.7). Total num frames: 805093376. Throughput: 0: 10094.6, 1: 9830.9. Samples: 805074248. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:50:06,062][104569] Avg episode reward: [(0, '8716.781'), (1, '8998.646')] [2023-12-27 02:50:06,619][105620] Updated weights for policy 1, policy_version 1573936 (0.0009) [2023-12-27 02:50:06,690][105620] Updated weights for policy 1, policy_version 1573946 (0.0010) [2023-12-27 02:50:06,746][105620] Updated weights for policy 1, policy_version 1573956 (0.0008) [2023-12-27 02:50:06,779][105692] Updated weights for policy 0, policy_version 1570545 (0.0009) [2023-12-27 02:50:06,832][105692] Updated weights for policy 0, policy_version 1570555 (0.0011) [2023-12-27 02:50:06,888][105692] Updated weights for policy 0, policy_version 1570565 (0.0011) [2023-12-27 02:50:07,352][105620] Updated weights for policy 1, policy_version 1573966 (0.0006) [2023-12-27 02:50:07,407][105620] Updated weights for policy 1, policy_version 1573976 (0.0005) [2023-12-27 02:50:07,461][105620] Updated weights for policy 1, policy_version 1573986 (0.0009) [2023-12-27 02:50:07,612][105692] Updated weights for policy 0, policy_version 1570575 (0.0010) [2023-12-27 02:50:07,664][105692] Updated weights for policy 0, policy_version 1570585 (0.0010) [2023-12-27 02:50:07,719][105692] Updated weights for policy 0, policy_version 1570595 (0.0010) [2023-12-27 02:50:08,098][105620] Updated weights for policy 1, policy_version 1573996 (0.0008) [2023-12-27 02:50:08,155][105620] Updated weights for policy 1, policy_version 1574006 (0.0010) [2023-12-27 02:50:08,216][105620] Updated weights for policy 1, policy_version 1574016 (0.0005) [2023-12-27 02:50:08,332][105692] Updated weights for policy 0, policy_version 1570605 (0.0009) [2023-12-27 02:50:08,400][105692] Updated weights for policy 0, policy_version 1570615 (0.0007) [2023-12-27 02:50:08,462][105692] Updated weights for policy 0, policy_version 1570625 (0.0006) [2023-12-27 02:50:08,786][105620] Updated weights for policy 1, policy_version 1574026 (0.0005) [2023-12-27 02:50:08,849][105620] Updated weights for policy 1, policy_version 1574036 (0.0008) [2023-12-27 02:50:08,909][105620] Updated weights for policy 1, policy_version 1574046 (0.0008) [2023-12-27 02:50:08,964][105620] Updated weights for policy 1, policy_version 1574056 (0.0008) [2023-12-27 02:50:09,132][105692] Updated weights for policy 0, policy_version 1570635 (0.0011) [2023-12-27 02:50:09,180][105692] Updated weights for policy 0, policy_version 1570645 (0.0010) [2023-12-27 02:50:09,249][105692] Updated weights for policy 0, policy_version 1570655 (0.0011) [2023-12-27 02:50:09,686][105620] Updated weights for policy 1, policy_version 1574066 (0.0011) [2023-12-27 02:50:09,748][105620] Updated weights for policy 1, policy_version 1574076 (0.0010) [2023-12-27 02:50:09,811][105620] Updated weights for policy 1, policy_version 1574086 (0.0011) [2023-12-27 02:50:10,066][105692] Updated weights for policy 0, policy_version 1570665 (0.0009) [2023-12-27 02:50:10,121][105692] Updated weights for policy 0, policy_version 1570675 (0.0006) [2023-12-27 02:50:10,189][105692] Updated weights for policy 0, policy_version 1570685 (0.0009) [2023-12-27 02:50:10,257][105692] Updated weights for policy 0, policy_version 1570695 (0.0009) [2023-12-27 02:50:10,557][105620] Updated weights for policy 1, policy_version 1574096 (0.0010) [2023-12-27 02:50:10,605][105620] Updated weights for policy 1, policy_version 1574106 (0.0010) [2023-12-27 02:50:10,664][105620] Updated weights for policy 1, policy_version 1574116 (0.0010) [2023-12-27 02:50:11,018][105692] Updated weights for policy 0, policy_version 1570705 (0.0007) [2023-12-27 02:50:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 805183488. Throughput: 0: 10027.0, 1: 9857.0. Samples: 805195556. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:50:11,062][104569] Avg episode reward: [(0, '8625.322'), (1, '8991.596')] [2023-12-27 02:50:11,081][105692] Updated weights for policy 0, policy_version 1570715 (0.0009) [2023-12-27 02:50:11,144][105692] Updated weights for policy 0, policy_version 1570725 (0.0008) [2023-12-27 02:50:11,420][105620] Updated weights for policy 1, policy_version 1574126 (0.0008) [2023-12-27 02:50:11,476][105620] Updated weights for policy 1, policy_version 1574136 (0.0005) [2023-12-27 02:50:11,532][105620] Updated weights for policy 1, policy_version 1574146 (0.0009) [2023-12-27 02:50:11,910][105692] Updated weights for policy 0, policy_version 1570735 (0.0009) [2023-12-27 02:50:11,974][105692] Updated weights for policy 0, policy_version 1570745 (0.0008) [2023-12-27 02:50:12,039][105692] Updated weights for policy 0, policy_version 1570755 (0.0008) [2023-12-27 02:50:12,285][105620] Updated weights for policy 1, policy_version 1574156 (0.0006) [2023-12-27 02:50:12,351][105620] Updated weights for policy 1, policy_version 1574166 (0.0008) [2023-12-27 02:50:12,418][105620] Updated weights for policy 1, policy_version 1574176 (0.0009) [2023-12-27 02:50:12,844][105692] Updated weights for policy 0, policy_version 1570765 (0.0009) [2023-12-27 02:50:12,898][105692] Updated weights for policy 0, policy_version 1570775 (0.0010) [2023-12-27 02:50:12,957][105692] Updated weights for policy 0, policy_version 1570786 (0.0010) [2023-12-27 02:50:13,004][105620] Updated weights for policy 1, policy_version 1574186 (0.0009) [2023-12-27 02:50:13,075][105620] Updated weights for policy 1, policy_version 1574196 (0.0005) [2023-12-27 02:50:13,139][105620] Updated weights for policy 1, policy_version 1574206 (0.0005) [2023-12-27 02:50:13,204][105620] Updated weights for policy 1, policy_version 1574216 (0.0007) [2023-12-27 02:50:13,569][105692] Updated weights for policy 0, policy_version 1570796 (0.0007) [2023-12-27 02:50:13,628][105692] Updated weights for policy 0, policy_version 1570806 (0.0009) [2023-12-27 02:50:13,692][105692] Updated weights for policy 0, policy_version 1570816 (0.0008) [2023-12-27 02:50:13,723][105620] Updated weights for policy 1, policy_version 1574226 (0.0008) [2023-12-27 02:50:13,778][105620] Updated weights for policy 1, policy_version 1574236 (0.0010) [2023-12-27 02:50:13,829][105620] Updated weights for policy 1, policy_version 1574246 (0.0010) [2023-12-27 02:50:14,299][105692] Updated weights for policy 0, policy_version 1570826 (0.0010) [2023-12-27 02:50:14,343][105692] Updated weights for policy 0, policy_version 1570836 (0.0010) [2023-12-27 02:50:14,409][105692] Updated weights for policy 0, policy_version 1570846 (0.0010) [2023-12-27 02:50:14,477][105692] Updated weights for policy 0, policy_version 1570856 (0.0010) [2023-12-27 02:50:14,522][105620] Updated weights for policy 1, policy_version 1574256 (0.0007) [2023-12-27 02:50:14,589][105620] Updated weights for policy 1, policy_version 1574266 (0.0010) [2023-12-27 02:50:14,642][105620] Updated weights for policy 1, policy_version 1574276 (0.0011) [2023-12-27 02:50:15,133][105692] Updated weights for policy 0, policy_version 1570866 (0.0006) [2023-12-27 02:50:15,200][105692] Updated weights for policy 0, policy_version 1570876 (0.0005) [2023-12-27 02:50:15,251][105692] Updated weights for policy 0, policy_version 1570886 (0.0008) [2023-12-27 02:50:15,369][105620] Updated weights for policy 1, policy_version 1574286 (0.0011) [2023-12-27 02:50:15,425][105620] Updated weights for policy 1, policy_version 1574296 (0.0011) [2023-12-27 02:50:15,477][105620] Updated weights for policy 1, policy_version 1574306 (0.0011) [2023-12-27 02:50:15,851][105692] Updated weights for policy 0, policy_version 1570896 (0.0008) [2023-12-27 02:50:15,908][105692] Updated weights for policy 0, policy_version 1570906 (0.0010) [2023-12-27 02:50:15,964][105692] Updated weights for policy 0, policy_version 1570916 (0.0010) [2023-12-27 02:50:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 805289984. Throughput: 0: 10008.1, 1: 9871.8. Samples: 805255344. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:50:16,062][104569] Avg episode reward: [(0, '8351.377'), (1, '8988.972')] [2023-12-27 02:50:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001574312_403079168.pth... [2023-12-27 02:50:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001570920_402210816.pth... [2023-12-27 02:50:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001569736_401907712.pth [2023-12-27 02:50:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001573192_402792448.pth [2023-12-27 02:50:16,199][105620] Updated weights for policy 1, policy_version 1574316 (0.0008) [2023-12-27 02:50:16,260][105620] Updated weights for policy 1, policy_version 1574326 (0.0005) [2023-12-27 02:50:16,309][105620] Updated weights for policy 1, policy_version 1574336 (0.0005) [2023-12-27 02:50:16,627][105692] Updated weights for policy 0, policy_version 1570926 (0.0010) [2023-12-27 02:50:16,682][105692] Updated weights for policy 0, policy_version 1570936 (0.0010) [2023-12-27 02:50:16,743][105692] Updated weights for policy 0, policy_version 1570946 (0.0010) [2023-12-27 02:50:17,003][105620] Updated weights for policy 1, policy_version 1574346 (0.0007) [2023-12-27 02:50:17,066][105620] Updated weights for policy 1, policy_version 1574356 (0.0005) [2023-12-27 02:50:17,119][105620] Updated weights for policy 1, policy_version 1574366 (0.0007) [2023-12-27 02:50:17,170][105620] Updated weights for policy 1, policy_version 1574376 (0.0010) [2023-12-27 02:50:17,492][105692] Updated weights for policy 0, policy_version 1570956 (0.0010) [2023-12-27 02:50:17,549][105692] Updated weights for policy 0, policy_version 1570966 (0.0010) [2023-12-27 02:50:17,606][105692] Updated weights for policy 0, policy_version 1570976 (0.0010) [2023-12-27 02:50:17,866][105620] Updated weights for policy 1, policy_version 1574386 (0.0011) [2023-12-27 02:50:17,917][105620] Updated weights for policy 1, policy_version 1574396 (0.0010) [2023-12-27 02:50:17,968][105620] Updated weights for policy 1, policy_version 1574406 (0.0011) [2023-12-27 02:50:18,237][105692] Updated weights for policy 0, policy_version 1570986 (0.0008) [2023-12-27 02:50:18,289][105692] Updated weights for policy 0, policy_version 1570996 (0.0005) [2023-12-27 02:50:18,357][105692] Updated weights for policy 0, policy_version 1571006 (0.0007) [2023-12-27 02:50:18,417][105692] Updated weights for policy 0, policy_version 1571016 (0.0011) [2023-12-27 02:50:18,707][105620] Updated weights for policy 1, policy_version 1574416 (0.0011) [2023-12-27 02:50:18,771][105620] Updated weights for policy 1, policy_version 1574426 (0.0010) [2023-12-27 02:50:18,827][105620] Updated weights for policy 1, policy_version 1574436 (0.0008) [2023-12-27 02:50:19,087][105692] Updated weights for policy 0, policy_version 1571026 (0.0010) [2023-12-27 02:50:19,147][105692] Updated weights for policy 0, policy_version 1571036 (0.0009) [2023-12-27 02:50:19,212][105692] Updated weights for policy 0, policy_version 1571046 (0.0010) [2023-12-27 02:50:19,592][105620] Updated weights for policy 1, policy_version 1574446 (0.0007) [2023-12-27 02:50:19,658][105620] Updated weights for policy 1, policy_version 1574456 (0.0007) [2023-12-27 02:50:19,728][105620] Updated weights for policy 1, policy_version 1574466 (0.0008) [2023-12-27 02:50:19,908][105692] Updated weights for policy 0, policy_version 1571056 (0.0008) [2023-12-27 02:50:19,969][105692] Updated weights for policy 0, policy_version 1571066 (0.0008) [2023-12-27 02:50:20,028][105692] Updated weights for policy 0, policy_version 1571076 (0.0008) [2023-12-27 02:50:20,495][105620] Updated weights for policy 1, policy_version 1574476 (0.0008) [2023-12-27 02:50:20,556][105620] Updated weights for policy 1, policy_version 1574486 (0.0009) [2023-12-27 02:50:20,605][105692] Updated weights for policy 0, policy_version 1571086 (0.0006) [2023-12-27 02:50:20,625][105620] Updated weights for policy 1, policy_version 1574496 (0.0010) [2023-12-27 02:50:20,665][105692] Updated weights for policy 0, policy_version 1571096 (0.0008) [2023-12-27 02:50:20,717][105692] Updated weights for policy 0, policy_version 1571106 (0.0010) [2023-12-27 02:50:21,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19933.8, 300 sec: 19577.5). Total num frames: 805388288. Throughput: 0: 10033.4, 1: 9758.4. Samples: 805376544. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:50:21,063][104569] Avg episode reward: [(0, '8353.801'), (1, '9080.355')] [2023-12-27 02:50:21,408][105620] Updated weights for policy 1, policy_version 1574506 (0.0007) [2023-12-27 02:50:21,463][105620] Updated weights for policy 1, policy_version 1574516 (0.0005) [2023-12-27 02:50:21,522][105620] Updated weights for policy 1, policy_version 1574526 (0.0006) [2023-12-27 02:50:21,573][105692] Updated weights for policy 0, policy_version 1571116 (0.0010) [2023-12-27 02:50:21,583][105620] Updated weights for policy 1, policy_version 1574536 (0.0006) [2023-12-27 02:50:21,638][105692] Updated weights for policy 0, policy_version 1571126 (0.0010) [2023-12-27 02:50:21,700][105692] Updated weights for policy 0, policy_version 1571136 (0.0011) [2023-12-27 02:50:22,246][105620] Updated weights for policy 1, policy_version 1574546 (0.0007) [2023-12-27 02:50:22,307][105620] Updated weights for policy 1, policy_version 1574556 (0.0009) [2023-12-27 02:50:22,373][105620] Updated weights for policy 1, policy_version 1574566 (0.0009) [2023-12-27 02:50:22,489][105692] Updated weights for policy 0, policy_version 1571146 (0.0009) [2023-12-27 02:50:22,554][105692] Updated weights for policy 0, policy_version 1571156 (0.0010) [2023-12-27 02:50:22,613][105692] Updated weights for policy 0, policy_version 1571166 (0.0011) [2023-12-27 02:50:22,675][105692] Updated weights for policy 0, policy_version 1571176 (0.0008) [2023-12-27 02:50:23,139][105620] Updated weights for policy 1, policy_version 1574576 (0.0008) [2023-12-27 02:50:23,196][105620] Updated weights for policy 1, policy_version 1574586 (0.0008) [2023-12-27 02:50:23,261][105620] Updated weights for policy 1, policy_version 1574596 (0.0008) [2023-12-27 02:50:23,446][105692] Updated weights for policy 0, policy_version 1571186 (0.0010) [2023-12-27 02:50:23,506][105692] Updated weights for policy 0, policy_version 1571196 (0.0011) [2023-12-27 02:50:23,572][105692] Updated weights for policy 0, policy_version 1571206 (0.0011) [2023-12-27 02:50:24,083][105620] Updated weights for policy 1, policy_version 1574606 (0.0009) [2023-12-27 02:50:24,141][105620] Updated weights for policy 1, policy_version 1574616 (0.0007) [2023-12-27 02:50:24,158][105586] KL-divergence is very high: 153.2616 [2023-12-27 02:50:24,159][105692] Updated weights for policy 0, policy_version 1571216 (0.0011) [2023-12-27 02:50:24,170][105586] KL-divergence is very high: 120.4264 [2023-12-27 02:50:24,196][105620] Updated weights for policy 1, policy_version 1574626 (0.0006) [2023-12-27 02:50:24,204][105586] KL-divergence is very high: 150.0176 [2023-12-27 02:50:24,206][105692] Updated weights for policy 0, policy_version 1571226 (0.0006) [2023-12-27 02:50:24,215][105586] KL-divergence is very high: 108.2178 [2023-12-27 02:50:24,264][105692] Updated weights for policy 0, policy_version 1571236 (0.0005) [2023-12-27 02:50:24,844][105692] Updated weights for policy 0, policy_version 1571246 (0.0008) [2023-12-27 02:50:24,899][105692] Updated weights for policy 0, policy_version 1571256 (0.0010) [2023-12-27 02:50:24,947][105692] Updated weights for policy 0, policy_version 1571266 (0.0010) [2023-12-27 02:50:24,999][105586] KL-divergence is very high: 104.9130 [2023-12-27 02:50:25,017][105586] KL-divergence is very high: 106.7616 [2023-12-27 02:50:25,023][105620] Updated weights for policy 1, policy_version 1574636 (0.0009) [2023-12-27 02:50:25,074][105620] Updated weights for policy 1, policy_version 1574646 (0.0008) [2023-12-27 02:50:25,122][105620] Updated weights for policy 1, policy_version 1574656 (0.0008) [2023-12-27 02:50:25,152][105586] KL-divergence is very high: 133.2706 [2023-12-27 02:50:25,691][105692] Updated weights for policy 0, policy_version 1571276 (0.0010) [2023-12-27 02:50:25,747][105692] Updated weights for policy 0, policy_version 1571286 (0.0011) [2023-12-27 02:50:25,776][105620] Updated weights for policy 1, policy_version 1574666 (0.0007) [2023-12-27 02:50:25,802][105692] Updated weights for policy 0, policy_version 1571296 (0.0011) [2023-12-27 02:50:25,828][105620] Updated weights for policy 1, policy_version 1574676 (0.0006) [2023-12-27 02:50:25,882][105620] Updated weights for policy 1, policy_version 1574686 (0.0007) [2023-12-27 02:50:25,943][105620] Updated weights for policy 1, policy_version 1574696 (0.0008) [2023-12-27 02:50:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19933.9, 300 sec: 19549.7). Total num frames: 805486592. Throughput: 0: 9989.4, 1: 9692.0. Samples: 805490808. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:50:26,062][104569] Avg episode reward: [(0, '8624.070'), (1, '8721.064')] [2023-12-27 02:50:26,522][105692] Updated weights for policy 0, policy_version 1571306 (0.0009) [2023-12-27 02:50:26,582][105692] Updated weights for policy 0, policy_version 1571316 (0.0005) [2023-12-27 02:50:26,639][105692] Updated weights for policy 0, policy_version 1571326 (0.0005) [2023-12-27 02:50:26,682][105692] Updated weights for policy 0, policy_version 1571336 (0.0007) [2023-12-27 02:50:26,750][105620] Updated weights for policy 1, policy_version 1574706 (0.0008) [2023-12-27 02:50:26,801][105620] Updated weights for policy 1, policy_version 1574716 (0.0008) [2023-12-27 02:50:26,848][105620] Updated weights for policy 1, policy_version 1574726 (0.0008) [2023-12-27 02:50:27,278][105692] Updated weights for policy 0, policy_version 1571346 (0.0005) [2023-12-27 02:50:27,342][105692] Updated weights for policy 0, policy_version 1571356 (0.0006) [2023-12-27 02:50:27,411][105692] Updated weights for policy 0, policy_version 1571366 (0.0006) [2023-12-27 02:50:27,604][105620] Updated weights for policy 1, policy_version 1574736 (0.0007) [2023-12-27 02:50:27,662][105620] Updated weights for policy 1, policy_version 1574746 (0.0006) [2023-12-27 02:50:27,720][105620] Updated weights for policy 1, policy_version 1574756 (0.0006) [2023-12-27 02:50:28,081][105692] Updated weights for policy 0, policy_version 1571376 (0.0010) [2023-12-27 02:50:28,136][105692] Updated weights for policy 0, policy_version 1571386 (0.0010) [2023-12-27 02:50:28,184][105692] Updated weights for policy 0, policy_version 1571396 (0.0010) [2023-12-27 02:50:28,299][105620] Updated weights for policy 1, policy_version 1574766 (0.0007) [2023-12-27 02:50:28,356][105620] Updated weights for policy 1, policy_version 1574776 (0.0007) [2023-12-27 02:50:28,414][105620] Updated weights for policy 1, policy_version 1574786 (0.0009) [2023-12-27 02:50:28,847][105692] Updated weights for policy 0, policy_version 1571406 (0.0007) [2023-12-27 02:50:28,895][105692] Updated weights for policy 0, policy_version 1571416 (0.0005) [2023-12-27 02:50:28,955][105692] Updated weights for policy 0, policy_version 1571426 (0.0006) [2023-12-27 02:50:29,226][105620] Updated weights for policy 1, policy_version 1574796 (0.0008) [2023-12-27 02:50:29,290][105620] Updated weights for policy 1, policy_version 1574806 (0.0008) [2023-12-27 02:50:29,348][105620] Updated weights for policy 1, policy_version 1574816 (0.0006) [2023-12-27 02:50:29,666][105692] Updated weights for policy 0, policy_version 1571436 (0.0011) [2023-12-27 02:50:29,732][105692] Updated weights for policy 0, policy_version 1571446 (0.0011) [2023-12-27 02:50:29,789][105692] Updated weights for policy 0, policy_version 1571456 (0.0011) [2023-12-27 02:50:29,933][105620] Updated weights for policy 1, policy_version 1574826 (0.0007) [2023-12-27 02:50:29,991][105620] Updated weights for policy 1, policy_version 1574836 (0.0009) [2023-12-27 02:50:30,050][105620] Updated weights for policy 1, policy_version 1574846 (0.0010) [2023-12-27 02:50:30,434][105692] Updated weights for policy 0, policy_version 1571466 (0.0010) [2023-12-27 02:50:30,484][105692] Updated weights for policy 0, policy_version 1571476 (0.0008) [2023-12-27 02:50:30,527][105692] Updated weights for policy 0, policy_version 1571486 (0.0005) [2023-12-27 02:50:30,578][105692] Updated weights for policy 0, policy_version 1571496 (0.0005) [2023-12-27 02:50:30,819][105620] Updated weights for policy 1, policy_version 1574857 (0.0008) [2023-12-27 02:50:30,867][105620] Updated weights for policy 1, policy_version 1574867 (0.0005) [2023-12-27 02:50:30,918][105620] Updated weights for policy 1, policy_version 1574877 (0.0005) [2023-12-27 02:50:30,968][105620] Updated weights for policy 1, policy_version 1574887 (0.0008) [2023-12-27 02:50:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19933.9, 300 sec: 19549.7). Total num frames: 805584896. Throughput: 0: 10078.8, 1: 9652.1. Samples: 805549948. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:50:31,062][104569] Avg episode reward: [(0, '8805.862'), (1, '8447.103')] [2023-12-27 02:50:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001574888_403226624.pth... [2023-12-27 02:50:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001571496_402358272.pth... [2023-12-27 02:50:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001570312_402055168.pth [2023-12-27 02:50:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001573736_402931712.pth [2023-12-27 02:50:31,339][105692] Updated weights for policy 0, policy_version 1571506 (0.0010) [2023-12-27 02:50:31,409][105692] Updated weights for policy 0, policy_version 1571516 (0.0010) [2023-12-27 02:50:31,467][105692] Updated weights for policy 0, policy_version 1571526 (0.0009) [2023-12-27 02:50:31,731][105620] Updated weights for policy 1, policy_version 1574897 (0.0010) [2023-12-27 02:50:31,791][105620] Updated weights for policy 1, policy_version 1574907 (0.0009) [2023-12-27 02:50:31,847][105620] Updated weights for policy 1, policy_version 1574917 (0.0009) [2023-12-27 02:50:32,170][105692] Updated weights for policy 0, policy_version 1571536 (0.0008) [2023-12-27 02:50:32,235][105692] Updated weights for policy 0, policy_version 1571546 (0.0006) [2023-12-27 02:50:32,296][105692] Updated weights for policy 0, policy_version 1571556 (0.0009) [2023-12-27 02:50:32,649][105620] Updated weights for policy 1, policy_version 1574927 (0.0009) [2023-12-27 02:50:32,711][105620] Updated weights for policy 1, policy_version 1574937 (0.0009) [2023-12-27 02:50:32,776][105620] Updated weights for policy 1, policy_version 1574947 (0.0009) [2023-12-27 02:50:33,003][105692] Updated weights for policy 0, policy_version 1571566 (0.0009) [2023-12-27 02:50:33,066][105692] Updated weights for policy 0, policy_version 1571576 (0.0007) [2023-12-27 02:50:33,122][105692] Updated weights for policy 0, policy_version 1571586 (0.0005) [2023-12-27 02:50:33,600][105620] Updated weights for policy 1, policy_version 1574957 (0.0009) [2023-12-27 02:50:33,656][105620] Updated weights for policy 1, policy_version 1574967 (0.0007) [2023-12-27 02:50:33,670][105692] Updated weights for policy 0, policy_version 1571596 (0.0005) [2023-12-27 02:50:33,708][105620] Updated weights for policy 1, policy_version 1574977 (0.0008) [2023-12-27 02:50:33,728][105692] Updated weights for policy 0, policy_version 1571606 (0.0006) [2023-12-27 02:50:33,789][105692] Updated weights for policy 0, policy_version 1571616 (0.0007) [2023-12-27 02:50:34,481][105692] Updated weights for policy 0, policy_version 1571626 (0.0006) [2023-12-27 02:50:34,487][105620] Updated weights for policy 1, policy_version 1574987 (0.0008) [2023-12-27 02:50:34,535][105692] Updated weights for policy 0, policy_version 1571636 (0.0006) [2023-12-27 02:50:34,548][105620] Updated weights for policy 1, policy_version 1574997 (0.0008) [2023-12-27 02:50:34,586][105692] Updated weights for policy 0, policy_version 1571646 (0.0008) [2023-12-27 02:50:34,613][105620] Updated weights for policy 1, policy_version 1575007 (0.0008) [2023-12-27 02:50:34,640][105692] Updated weights for policy 0, policy_version 1571656 (0.0006) [2023-12-27 02:50:35,317][105692] Updated weights for policy 0, policy_version 1571666 (0.0008) [2023-12-27 02:50:35,367][105692] Updated weights for policy 0, policy_version 1571676 (0.0005) [2023-12-27 02:50:35,399][105620] Updated weights for policy 1, policy_version 1575017 (0.0008) [2023-12-27 02:50:35,413][105692] Updated weights for policy 0, policy_version 1571686 (0.0005) [2023-12-27 02:50:35,461][105620] Updated weights for policy 1, policy_version 1575027 (0.0009) [2023-12-27 02:50:35,514][105620] Updated weights for policy 1, policy_version 1575037 (0.0009) [2023-12-27 02:50:35,569][105620] Updated weights for policy 1, policy_version 1575047 (0.0008) [2023-12-27 02:50:36,021][105692] Updated weights for policy 0, policy_version 1571696 (0.0005) [2023-12-27 02:50:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 805675008. Throughput: 0: 10105.1, 1: 9426.5. Samples: 805667680. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:50:36,062][104569] Avg episode reward: [(0, '8442.117'), (1, '9080.972')] [2023-12-27 02:50:36,068][105692] Updated weights for policy 0, policy_version 1571706 (0.0008) [2023-12-27 02:50:36,123][105692] Updated weights for policy 0, policy_version 1571716 (0.0009) [2023-12-27 02:50:36,221][105620] Updated weights for policy 1, policy_version 1575057 (0.0009) [2023-12-27 02:50:36,273][105620] Updated weights for policy 1, policy_version 1575068 (0.0009) [2023-12-27 02:50:36,327][105620] Updated weights for policy 1, policy_version 1575078 (0.0009) [2023-12-27 02:50:36,842][105692] Updated weights for policy 0, policy_version 1571726 (0.0007) [2023-12-27 02:50:36,898][105692] Updated weights for policy 0, policy_version 1571736 (0.0006) [2023-12-27 02:50:36,952][105692] Updated weights for policy 0, policy_version 1571746 (0.0005) [2023-12-27 02:50:37,045][105620] Updated weights for policy 1, policy_version 1575088 (0.0009) [2023-12-27 02:50:37,100][105620] Updated weights for policy 1, policy_version 1575098 (0.0009) [2023-12-27 02:50:37,154][105620] Updated weights for policy 1, policy_version 1575108 (0.0009) [2023-12-27 02:50:37,541][105692] Updated weights for policy 0, policy_version 1571756 (0.0007) [2023-12-27 02:50:37,598][105692] Updated weights for policy 0, policy_version 1571766 (0.0009) [2023-12-27 02:50:37,649][105692] Updated weights for policy 0, policy_version 1571776 (0.0009) [2023-12-27 02:50:37,988][105620] Updated weights for policy 1, policy_version 1575118 (0.0009) [2023-12-27 02:50:38,051][105620] Updated weights for policy 1, policy_version 1575128 (0.0009) [2023-12-27 02:50:38,113][105620] Updated weights for policy 1, policy_version 1575138 (0.0009) [2023-12-27 02:50:38,355][105692] Updated weights for policy 0, policy_version 1571786 (0.0008) [2023-12-27 02:50:38,408][105692] Updated weights for policy 0, policy_version 1571796 (0.0008) [2023-12-27 02:50:38,476][105692] Updated weights for policy 0, policy_version 1571806 (0.0006) [2023-12-27 02:50:38,541][105692] Updated weights for policy 0, policy_version 1571816 (0.0007) [2023-12-27 02:50:38,924][105620] Updated weights for policy 1, policy_version 1575148 (0.0008) [2023-12-27 02:50:38,979][105620] Updated weights for policy 1, policy_version 1575158 (0.0009) [2023-12-27 02:50:39,044][105620] Updated weights for policy 1, policy_version 1575168 (0.0009) [2023-12-27 02:50:39,199][105692] Updated weights for policy 0, policy_version 1571826 (0.0009) [2023-12-27 02:50:39,258][105692] Updated weights for policy 0, policy_version 1571836 (0.0008) [2023-12-27 02:50:39,316][105692] Updated weights for policy 0, policy_version 1571846 (0.0008) [2023-12-27 02:50:39,767][105620] Updated weights for policy 1, policy_version 1575178 (0.0009) [2023-12-27 02:50:39,836][105620] Updated weights for policy 1, policy_version 1575188 (0.0008) [2023-12-27 02:50:39,903][105620] Updated weights for policy 1, policy_version 1575198 (0.0009) [2023-12-27 02:50:39,972][105620] Updated weights for policy 1, policy_version 1575208 (0.0008) [2023-12-27 02:50:40,140][105692] Updated weights for policy 0, policy_version 1571856 (0.0009) [2023-12-27 02:50:40,198][105692] Updated weights for policy 0, policy_version 1571866 (0.0009) [2023-12-27 02:50:40,248][105692] Updated weights for policy 0, policy_version 1571876 (0.0008) [2023-12-27 02:50:40,722][105620] Updated weights for policy 1, policy_version 1575218 (0.0009) [2023-12-27 02:50:40,778][105620] Updated weights for policy 1, policy_version 1575228 (0.0009) [2023-12-27 02:50:40,835][105620] Updated weights for policy 1, policy_version 1575238 (0.0008) [2023-12-27 02:50:41,016][105692] Updated weights for policy 0, policy_version 1571886 (0.0009) [2023-12-27 02:50:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 805773312. Throughput: 0: 10105.2, 1: 9490.2. Samples: 805784008. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:50:41,063][104569] Avg episode reward: [(0, '8988.166'), (1, '9177.115')] [2023-12-27 02:50:41,084][105692] Updated weights for policy 0, policy_version 1571896 (0.0009) [2023-12-27 02:50:41,149][105692] Updated weights for policy 0, policy_version 1571906 (0.0007) [2023-12-27 02:50:41,574][105620] Updated weights for policy 1, policy_version 1575248 (0.0009) [2023-12-27 02:50:41,635][105620] Updated weights for policy 1, policy_version 1575258 (0.0008) [2023-12-27 02:50:41,698][105620] Updated weights for policy 1, policy_version 1575268 (0.0009) [2023-12-27 02:50:42,005][105692] Updated weights for policy 0, policy_version 1571916 (0.0010) [2023-12-27 02:50:42,066][105692] Updated weights for policy 0, policy_version 1571926 (0.0006) [2023-12-27 02:50:42,134][105692] Updated weights for policy 0, policy_version 1571936 (0.0007) [2023-12-27 02:50:42,483][105620] Updated weights for policy 1, policy_version 1575278 (0.0010) [2023-12-27 02:50:42,532][105620] Updated weights for policy 1, policy_version 1575288 (0.0010) [2023-12-27 02:50:42,595][105620] Updated weights for policy 1, policy_version 1575298 (0.0011) [2023-12-27 02:50:42,861][105692] Updated weights for policy 0, policy_version 1571946 (0.0009) [2023-12-27 02:50:42,922][105692] Updated weights for policy 0, policy_version 1571956 (0.0008) [2023-12-27 02:50:42,982][105692] Updated weights for policy 0, policy_version 1571966 (0.0009) [2023-12-27 02:50:43,038][105692] Updated weights for policy 0, policy_version 1571976 (0.0008) [2023-12-27 02:50:43,346][105620] Updated weights for policy 1, policy_version 1575308 (0.0011) [2023-12-27 02:50:43,396][105620] Updated weights for policy 1, policy_version 1575318 (0.0010) [2023-12-27 02:50:43,440][105620] Updated weights for policy 1, policy_version 1575328 (0.0010) [2023-12-27 02:50:43,796][105692] Updated weights for policy 0, policy_version 1571986 (0.0008) [2023-12-27 02:50:43,844][105692] Updated weights for policy 0, policy_version 1571996 (0.0008) [2023-12-27 02:50:43,888][105692] Updated weights for policy 0, policy_version 1572006 (0.0007) [2023-12-27 02:50:44,183][105620] Updated weights for policy 1, policy_version 1575338 (0.0010) [2023-12-27 02:50:44,237][105620] Updated weights for policy 1, policy_version 1575348 (0.0008) [2023-12-27 02:50:44,292][105620] Updated weights for policy 1, policy_version 1575358 (0.0006) [2023-12-27 02:50:44,340][105620] Updated weights for policy 1, policy_version 1575368 (0.0006) [2023-12-27 02:50:44,603][105692] Updated weights for policy 0, policy_version 1572016 (0.0008) [2023-12-27 02:50:44,661][105692] Updated weights for policy 0, policy_version 1572026 (0.0008) [2023-12-27 02:50:44,720][105692] Updated weights for policy 0, policy_version 1572036 (0.0009) [2023-12-27 02:50:45,006][105620] Updated weights for policy 1, policy_version 1575378 (0.0009) [2023-12-27 02:50:45,063][105620] Updated weights for policy 1, policy_version 1575388 (0.0006) [2023-12-27 02:50:45,125][105620] Updated weights for policy 1, policy_version 1575398 (0.0006) [2023-12-27 02:50:45,568][105692] Updated weights for policy 0, policy_version 1572046 (0.0008) [2023-12-27 02:50:45,618][105692] Updated weights for policy 0, policy_version 1572056 (0.0008) [2023-12-27 02:50:45,670][105692] Updated weights for policy 0, policy_version 1572066 (0.0007) [2023-12-27 02:50:45,755][105620] Updated weights for policy 1, policy_version 1575408 (0.0010) [2023-12-27 02:50:45,814][105620] Updated weights for policy 1, policy_version 1575418 (0.0011) [2023-12-27 02:50:45,870][105620] Updated weights for policy 1, policy_version 1575428 (0.0011) [2023-12-27 02:50:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 805871616. Throughput: 0: 9984.9, 1: 9518.5. Samples: 805839472. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:50:46,063][104569] Avg episode reward: [(0, '8809.476'), (1, '8902.954')] [2023-12-27 02:50:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001572072_402505728.pth... [2023-12-27 02:50:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001575432_403365888.pth... [2023-12-27 02:50:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001570920_402210816.pth [2023-12-27 02:50:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001574312_403079168.pth [2023-12-27 02:50:46,299][105692] Updated weights for policy 0, policy_version 1572076 (0.0007) [2023-12-27 02:50:46,343][105692] Updated weights for policy 0, policy_version 1572086 (0.0005) [2023-12-27 02:50:46,393][105692] Updated weights for policy 0, policy_version 1572096 (0.0005) [2023-12-27 02:50:46,523][105620] Updated weights for policy 1, policy_version 1575438 (0.0010) [2023-12-27 02:50:46,580][105620] Updated weights for policy 1, policy_version 1575449 (0.0007) [2023-12-27 02:50:46,624][105620] Updated weights for policy 1, policy_version 1575459 (0.0005) [2023-12-27 02:50:47,052][105692] Updated weights for policy 0, policy_version 1572106 (0.0006) [2023-12-27 02:50:47,102][105692] Updated weights for policy 0, policy_version 1572116 (0.0009) [2023-12-27 02:50:47,156][105692] Updated weights for policy 0, policy_version 1572126 (0.0010) [2023-12-27 02:50:47,203][105692] Updated weights for policy 0, policy_version 1572136 (0.0008) [2023-12-27 02:50:47,336][105620] Updated weights for policy 1, policy_version 1575469 (0.0007) [2023-12-27 02:50:47,390][105620] Updated weights for policy 1, policy_version 1575479 (0.0008) [2023-12-27 02:50:47,454][105620] Updated weights for policy 1, policy_version 1575489 (0.0008) [2023-12-27 02:50:48,013][105692] Updated weights for policy 0, policy_version 1572146 (0.0009) [2023-12-27 02:50:48,071][105692] Updated weights for policy 0, policy_version 1572156 (0.0009) [2023-12-27 02:50:48,082][105620] Updated weights for policy 1, policy_version 1575499 (0.0009) [2023-12-27 02:50:48,129][105692] Updated weights for policy 0, policy_version 1572166 (0.0007) [2023-12-27 02:50:48,139][105620] Updated weights for policy 1, policy_version 1575509 (0.0006) [2023-12-27 02:50:48,202][105620] Updated weights for policy 1, policy_version 1575519 (0.0009) [2023-12-27 02:50:48,895][105620] Updated weights for policy 1, policy_version 1575529 (0.0009) [2023-12-27 02:50:48,922][105692] Updated weights for policy 0, policy_version 1572176 (0.0007) [2023-12-27 02:50:48,957][105620] Updated weights for policy 1, policy_version 1575539 (0.0007) [2023-12-27 02:50:48,977][105692] Updated weights for policy 0, policy_version 1572186 (0.0008) [2023-12-27 02:50:49,015][105620] Updated weights for policy 1, policy_version 1575549 (0.0007) [2023-12-27 02:50:49,040][105692] Updated weights for policy 0, policy_version 1572196 (0.0008) [2023-12-27 02:50:49,077][105620] Updated weights for policy 1, policy_version 1575559 (0.0008) [2023-12-27 02:50:49,802][105692] Updated weights for policy 0, policy_version 1572206 (0.0009) [2023-12-27 02:50:49,866][105692] Updated weights for policy 0, policy_version 1572216 (0.0008) [2023-12-27 02:50:49,879][105620] Updated weights for policy 1, policy_version 1575569 (0.0008) [2023-12-27 02:50:49,932][105692] Updated weights for policy 0, policy_version 1572226 (0.0008) [2023-12-27 02:50:49,943][105620] Updated weights for policy 1, policy_version 1575579 (0.0008) [2023-12-27 02:50:50,005][105620] Updated weights for policy 1, policy_version 1575589 (0.0008) [2023-12-27 02:50:50,642][105692] Updated weights for policy 0, policy_version 1572236 (0.0008) [2023-12-27 02:50:50,704][105692] Updated weights for policy 0, policy_version 1572246 (0.0009) [2023-12-27 02:50:50,727][105620] Updated weights for policy 1, policy_version 1575599 (0.0007) [2023-12-27 02:50:50,758][105692] Updated weights for policy 0, policy_version 1572256 (0.0008) [2023-12-27 02:50:50,789][105620] Updated weights for policy 1, policy_version 1575609 (0.0008) [2023-12-27 02:50:50,846][105620] Updated weights for policy 1, policy_version 1575619 (0.0005) [2023-12-27 02:50:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 805969920. Throughput: 0: 9965.2, 1: 9657.9. Samples: 805957288. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 02:50:51,062][104569] Avg episode reward: [(0, '8537.700'), (1, '8900.293')] [2023-12-27 02:50:51,549][105620] Updated weights for policy 1, policy_version 1575629 (0.0008) [2023-12-27 02:50:51,558][105692] Updated weights for policy 0, policy_version 1572266 (0.0009) [2023-12-27 02:50:51,606][105620] Updated weights for policy 1, policy_version 1575639 (0.0008) [2023-12-27 02:50:51,623][105692] Updated weights for policy 0, policy_version 1572276 (0.0009) [2023-12-27 02:50:51,670][105620] Updated weights for policy 1, policy_version 1575649 (0.0007) [2023-12-27 02:50:51,691][105692] Updated weights for policy 0, policy_version 1572286 (0.0008) [2023-12-27 02:50:51,756][105692] Updated weights for policy 0, policy_version 1572296 (0.0009) [2023-12-27 02:50:52,415][105620] Updated weights for policy 1, policy_version 1575659 (0.0006) [2023-12-27 02:50:52,470][105620] Updated weights for policy 1, policy_version 1575669 (0.0006) [2023-12-27 02:50:52,517][105692] Updated weights for policy 0, policy_version 1572306 (0.0011) [2023-12-27 02:50:52,538][105620] Updated weights for policy 1, policy_version 1575679 (0.0006) [2023-12-27 02:50:52,578][105692] Updated weights for policy 0, policy_version 1572316 (0.0011) [2023-12-27 02:50:52,627][105692] Updated weights for policy 0, policy_version 1572326 (0.0011) [2023-12-27 02:50:53,234][105620] Updated weights for policy 1, policy_version 1575689 (0.0011) [2023-12-27 02:50:53,291][105620] Updated weights for policy 1, policy_version 1575699 (0.0009) [2023-12-27 02:50:53,350][105620] Updated weights for policy 1, policy_version 1575709 (0.0008) [2023-12-27 02:50:53,373][105692] Updated weights for policy 0, policy_version 1572336 (0.0008) [2023-12-27 02:50:53,408][105620] Updated weights for policy 1, policy_version 1575719 (0.0010) [2023-12-27 02:50:53,424][105692] Updated weights for policy 0, policy_version 1572346 (0.0008) [2023-12-27 02:50:53,476][105692] Updated weights for policy 0, policy_version 1572356 (0.0009) [2023-12-27 02:50:54,063][105620] Updated weights for policy 1, policy_version 1575729 (0.0009) [2023-12-27 02:50:54,134][105620] Updated weights for policy 1, policy_version 1575739 (0.0006) [2023-12-27 02:50:54,194][105620] Updated weights for policy 1, policy_version 1575749 (0.0007) [2023-12-27 02:50:54,320][105692] Updated weights for policy 0, policy_version 1572366 (0.0009) [2023-12-27 02:50:54,379][105692] Updated weights for policy 0, policy_version 1572376 (0.0010) [2023-12-27 02:50:54,432][105692] Updated weights for policy 0, policy_version 1572386 (0.0009) [2023-12-27 02:50:54,752][105620] Updated weights for policy 1, policy_version 1575759 (0.0009) [2023-12-27 02:50:54,820][105620] Updated weights for policy 1, policy_version 1575769 (0.0008) [2023-12-27 02:50:54,879][105620] Updated weights for policy 1, policy_version 1575779 (0.0011) [2023-12-27 02:50:55,178][105692] Updated weights for policy 0, policy_version 1572397 (0.0007) [2023-12-27 02:50:55,227][105692] Updated weights for policy 0, policy_version 1572407 (0.0005) [2023-12-27 02:50:55,275][105692] Updated weights for policy 0, policy_version 1572417 (0.0005) [2023-12-27 02:50:55,546][105620] Updated weights for policy 1, policy_version 1575789 (0.0010) [2023-12-27 02:50:55,598][105620] Updated weights for policy 1, policy_version 1575799 (0.0009) [2023-12-27 02:50:55,650][105620] Updated weights for policy 1, policy_version 1575809 (0.0009) [2023-12-27 02:50:55,862][105692] Updated weights for policy 0, policy_version 1572427 (0.0006) [2023-12-27 02:50:55,916][105692] Updated weights for policy 0, policy_version 1572437 (0.0006) [2023-12-27 02:50:55,973][105692] Updated weights for policy 0, policy_version 1572447 (0.0006) [2023-12-27 02:50:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 806068224. Throughput: 0: 9859.5, 1: 9649.7. Samples: 806073476. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:50:56,063][104569] Avg episode reward: [(0, '8809.338'), (1, '8994.381')] [2023-12-27 02:50:56,496][105620] Updated weights for policy 1, policy_version 1575819 (0.0009) [2023-12-27 02:50:56,543][105620] Updated weights for policy 1, policy_version 1575829 (0.0008) [2023-12-27 02:50:56,590][105620] Updated weights for policy 1, policy_version 1575839 (0.0008) [2023-12-27 02:50:56,627][105692] Updated weights for policy 0, policy_version 1572457 (0.0006) [2023-12-27 02:50:56,677][105692] Updated weights for policy 0, policy_version 1572467 (0.0005) [2023-12-27 02:50:56,733][105692] Updated weights for policy 0, policy_version 1572477 (0.0007) [2023-12-27 02:50:56,776][105692] Updated weights for policy 0, policy_version 1572487 (0.0005) [2023-12-27 02:50:57,313][105692] Updated weights for policy 0, policy_version 1572497 (0.0006) [2023-12-27 02:50:57,359][105692] Updated weights for policy 0, policy_version 1572507 (0.0005) [2023-12-27 02:50:57,405][105692] Updated weights for policy 0, policy_version 1572517 (0.0005) [2023-12-27 02:50:57,443][105620] Updated weights for policy 1, policy_version 1575849 (0.0009) [2023-12-27 02:50:57,511][105620] Updated weights for policy 1, policy_version 1575859 (0.0005) [2023-12-27 02:50:57,584][105620] Updated weights for policy 1, policy_version 1575869 (0.0006) [2023-12-27 02:50:57,641][105620] Updated weights for policy 1, policy_version 1575879 (0.0009) [2023-12-27 02:50:57,998][105692] Updated weights for policy 0, policy_version 1572527 (0.0006) [2023-12-27 02:50:58,054][105692] Updated weights for policy 0, policy_version 1572537 (0.0005) [2023-12-27 02:50:58,102][105692] Updated weights for policy 0, policy_version 1572547 (0.0005) [2023-12-27 02:50:58,302][105620] Updated weights for policy 1, policy_version 1575889 (0.0008) [2023-12-27 02:50:58,371][105620] Updated weights for policy 1, policy_version 1575899 (0.0007) [2023-12-27 02:50:58,437][105620] Updated weights for policy 1, policy_version 1575909 (0.0008) [2023-12-27 02:50:58,881][105692] Updated weights for policy 0, policy_version 1572557 (0.0007) [2023-12-27 02:50:58,952][105692] Updated weights for policy 0, policy_version 1572567 (0.0009) [2023-12-27 02:50:59,016][105692] Updated weights for policy 0, policy_version 1572577 (0.0009) [2023-12-27 02:50:59,199][105620] Updated weights for policy 1, policy_version 1575919 (0.0006) [2023-12-27 02:50:59,266][105620] Updated weights for policy 1, policy_version 1575929 (0.0008) [2023-12-27 02:50:59,329][105620] Updated weights for policy 1, policy_version 1575939 (0.0008) [2023-12-27 02:50:59,799][105692] Updated weights for policy 0, policy_version 1572587 (0.0010) [2023-12-27 02:50:59,864][105692] Updated weights for policy 0, policy_version 1572598 (0.0009) [2023-12-27 02:50:59,921][105692] Updated weights for policy 0, policy_version 1572608 (0.0008) [2023-12-27 02:50:59,997][105620] Updated weights for policy 1, policy_version 1575949 (0.0007) [2023-12-27 02:51:00,069][105620] Updated weights for policy 1, policy_version 1575959 (0.0008) [2023-12-27 02:51:00,139][105620] Updated weights for policy 1, policy_version 1575969 (0.0008) [2023-12-27 02:51:00,719][105692] Updated weights for policy 0, policy_version 1572619 (0.0007) [2023-12-27 02:51:00,774][105620] Updated weights for policy 1, policy_version 1575979 (0.0007) [2023-12-27 02:51:00,778][105692] Updated weights for policy 0, policy_version 1572629 (0.0010) [2023-12-27 02:51:00,826][105620] Updated weights for policy 1, policy_version 1575989 (0.0005) [2023-12-27 02:51:00,834][105692] Updated weights for policy 0, policy_version 1572639 (0.0009) [2023-12-27 02:51:00,881][105620] Updated weights for policy 1, policy_version 1575999 (0.0005) [2023-12-27 02:51:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 806166528. Throughput: 0: 9976.6, 1: 9552.5. Samples: 806134156. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:51:01,063][104569] Avg episode reward: [(0, '8622.632'), (1, '9088.039')] [2023-12-27 02:51:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001572648_402653184.pth... [2023-12-27 02:51:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001576008_403513344.pth... [2023-12-27 02:51:01,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001574888_403226624.pth [2023-12-27 02:51:01,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001571496_402358272.pth [2023-12-27 02:51:01,536][105692] Updated weights for policy 0, policy_version 1572649 (0.0008) [2023-12-27 02:51:01,555][105620] Updated weights for policy 1, policy_version 1576009 (0.0008) [2023-12-27 02:51:01,590][105692] Updated weights for policy 0, policy_version 1572659 (0.0006) [2023-12-27 02:51:01,626][105620] Updated weights for policy 1, policy_version 1576019 (0.0007) [2023-12-27 02:51:01,660][105692] Updated weights for policy 0, policy_version 1572669 (0.0007) [2023-12-27 02:51:01,693][105620] Updated weights for policy 1, policy_version 1576029 (0.0008) [2023-12-27 02:51:01,719][105692] Updated weights for policy 0, policy_version 1572679 (0.0006) [2023-12-27 02:51:01,761][105620] Updated weights for policy 1, policy_version 1576040 (0.0008) [2023-12-27 02:51:02,424][105620] Updated weights for policy 1, policy_version 1576050 (0.0009) [2023-12-27 02:51:02,452][105692] Updated weights for policy 0, policy_version 1572689 (0.0008) [2023-12-27 02:51:02,488][105620] Updated weights for policy 1, policy_version 1576060 (0.0007) [2023-12-27 02:51:02,511][105692] Updated weights for policy 0, policy_version 1572699 (0.0006) [2023-12-27 02:51:02,554][105620] Updated weights for policy 1, policy_version 1576070 (0.0007) [2023-12-27 02:51:02,568][105692] Updated weights for policy 0, policy_version 1572709 (0.0007) [2023-12-27 02:51:03,298][105692] Updated weights for policy 0, policy_version 1572719 (0.0006) [2023-12-27 02:51:03,308][105620] Updated weights for policy 1, policy_version 1576080 (0.0009) [2023-12-27 02:51:03,343][105692] Updated weights for policy 0, policy_version 1572729 (0.0006) [2023-12-27 02:51:03,357][105620] Updated weights for policy 1, policy_version 1576090 (0.0007) [2023-12-27 02:51:03,396][105692] Updated weights for policy 0, policy_version 1572739 (0.0007) [2023-12-27 02:51:03,406][105620] Updated weights for policy 1, policy_version 1576100 (0.0009) [2023-12-27 02:51:04,178][105692] Updated weights for policy 0, policy_version 1572749 (0.0008) [2023-12-27 02:51:04,187][105620] Updated weights for policy 1, policy_version 1576110 (0.0008) [2023-12-27 02:51:04,243][105692] Updated weights for policy 0, policy_version 1572759 (0.0008) [2023-12-27 02:51:04,253][105620] Updated weights for policy 1, policy_version 1576120 (0.0008) [2023-12-27 02:51:04,307][105692] Updated weights for policy 0, policy_version 1572769 (0.0006) [2023-12-27 02:51:04,314][105620] Updated weights for policy 1, policy_version 1576130 (0.0010) [2023-12-27 02:51:04,965][105620] Updated weights for policy 1, policy_version 1576140 (0.0006) [2023-12-27 02:51:05,019][105620] Updated weights for policy 1, policy_version 1576150 (0.0005) [2023-12-27 02:51:05,068][105620] Updated weights for policy 1, policy_version 1576160 (0.0005) [2023-12-27 02:51:05,135][105692] Updated weights for policy 0, policy_version 1572779 (0.0008) [2023-12-27 02:51:05,193][105692] Updated weights for policy 0, policy_version 1572789 (0.0010) [2023-12-27 02:51:05,247][105692] Updated weights for policy 0, policy_version 1572799 (0.0010) [2023-12-27 02:51:05,578][105620] Updated weights for policy 1, policy_version 1576170 (0.0005) [2023-12-27 02:51:05,623][105620] Updated weights for policy 1, policy_version 1576180 (0.0005) [2023-12-27 02:51:05,672][105620] Updated weights for policy 1, policy_version 1576190 (0.0005) [2023-12-27 02:51:05,718][105620] Updated weights for policy 1, policy_version 1576200 (0.0005) [2023-12-27 02:51:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 806256640. Throughput: 0: 9809.3, 1: 9555.6. Samples: 806247968. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:51:06,063][104569] Avg episode reward: [(0, '8622.284'), (1, '8720.757')] [2023-12-27 02:51:06,161][105692] Updated weights for policy 0, policy_version 1572809 (0.0009) [2023-12-27 02:51:06,219][105692] Updated weights for policy 0, policy_version 1572819 (0.0008) [2023-12-27 02:51:06,280][105692] Updated weights for policy 0, policy_version 1572829 (0.0008) [2023-12-27 02:51:06,330][105620] Updated weights for policy 1, policy_version 1576210 (0.0011) [2023-12-27 02:51:06,344][105692] Updated weights for policy 0, policy_version 1572839 (0.0008) [2023-12-27 02:51:06,389][105620] Updated weights for policy 1, policy_version 1576220 (0.0011) [2023-12-27 02:51:06,442][105620] Updated weights for policy 1, policy_version 1576230 (0.0011) [2023-12-27 02:51:07,064][105620] Updated weights for policy 1, policy_version 1576240 (0.0009) [2023-12-27 02:51:07,114][105620] Updated weights for policy 1, policy_version 1576250 (0.0009) [2023-12-27 02:51:07,174][105620] Updated weights for policy 1, policy_version 1576260 (0.0010) [2023-12-27 02:51:07,181][105692] Updated weights for policy 0, policy_version 1572849 (0.0006) [2023-12-27 02:51:07,247][105692] Updated weights for policy 0, policy_version 1572859 (0.0007) [2023-12-27 02:51:07,304][105692] Updated weights for policy 0, policy_version 1572869 (0.0010) [2023-12-27 02:51:07,751][105620] Updated weights for policy 1, policy_version 1576270 (0.0007) [2023-12-27 02:51:07,796][105620] Updated weights for policy 1, policy_version 1576280 (0.0005) [2023-12-27 02:51:07,850][105620] Updated weights for policy 1, policy_version 1576290 (0.0006) [2023-12-27 02:51:08,127][105692] Updated weights for policy 0, policy_version 1572879 (0.0009) [2023-12-27 02:51:08,176][105692] Updated weights for policy 0, policy_version 1572889 (0.0010) [2023-12-27 02:51:08,231][105692] Updated weights for policy 0, policy_version 1572899 (0.0010) [2023-12-27 02:51:08,426][105620] Updated weights for policy 1, policy_version 1576300 (0.0010) [2023-12-27 02:51:08,486][105620] Updated weights for policy 1, policy_version 1576310 (0.0007) [2023-12-27 02:51:08,552][105620] Updated weights for policy 1, policy_version 1576320 (0.0008) [2023-12-27 02:51:08,976][105692] Updated weights for policy 0, policy_version 1572909 (0.0009) [2023-12-27 02:51:09,043][105692] Updated weights for policy 0, policy_version 1572919 (0.0007) [2023-12-27 02:51:09,107][105692] Updated weights for policy 0, policy_version 1572929 (0.0008) [2023-12-27 02:51:09,292][105620] Updated weights for policy 1, policy_version 1576330 (0.0007) [2023-12-27 02:51:09,359][105620] Updated weights for policy 1, policy_version 1576340 (0.0008) [2023-12-27 02:51:09,432][105620] Updated weights for policy 1, policy_version 1576350 (0.0009) [2023-12-27 02:51:09,499][105620] Updated weights for policy 1, policy_version 1576360 (0.0010) [2023-12-27 02:51:09,854][105692] Updated weights for policy 0, policy_version 1572939 (0.0010) [2023-12-27 02:51:09,917][105692] Updated weights for policy 0, policy_version 1572949 (0.0009) [2023-12-27 02:51:09,966][105692] Updated weights for policy 0, policy_version 1572959 (0.0008) [2023-12-27 02:51:10,239][105620] Updated weights for policy 1, policy_version 1576370 (0.0010) [2023-12-27 02:51:10,247][105586] KL-divergence is very high: 164.4679 [2023-12-27 02:51:10,291][105586] KL-divergence is very high: 301.7214 [2023-12-27 02:51:10,298][105620] Updated weights for policy 1, policy_version 1576380 (0.0010) [2023-12-27 02:51:10,330][105586] KL-divergence is very high: 331.2808 [2023-12-27 02:51:10,347][105620] Updated weights for policy 1, policy_version 1576390 (0.0010) [2023-12-27 02:51:10,695][105692] Updated weights for policy 0, policy_version 1572969 (0.0008) [2023-12-27 02:51:10,756][105692] Updated weights for policy 0, policy_version 1572979 (0.0005) [2023-12-27 02:51:10,801][105692] Updated weights for policy 0, policy_version 1572989 (0.0005) [2023-12-27 02:51:10,852][105692] Updated weights for policy 0, policy_version 1572999 (0.0005) [2023-12-27 02:51:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 806354944. Throughput: 0: 9643.4, 1: 9776.7. Samples: 806364712. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:51:11,062][104569] Avg episode reward: [(0, '8898.233'), (1, '8718.669')] [2023-12-27 02:51:11,104][105620] Updated weights for policy 1, policy_version 1576400 (0.0009) [2023-12-27 02:51:11,173][105620] Updated weights for policy 1, policy_version 1576410 (0.0009) [2023-12-27 02:51:11,232][105620] Updated weights for policy 1, policy_version 1576420 (0.0009) [2023-12-27 02:51:11,564][105692] Updated weights for policy 0, policy_version 1573009 (0.0006) [2023-12-27 02:51:11,621][105692] Updated weights for policy 0, policy_version 1573019 (0.0007) [2023-12-27 02:51:11,690][105692] Updated weights for policy 0, policy_version 1573029 (0.0009) [2023-12-27 02:51:11,966][105620] Updated weights for policy 1, policy_version 1576430 (0.0008) [2023-12-27 02:51:12,016][105620] Updated weights for policy 1, policy_version 1576440 (0.0010) [2023-12-27 02:51:12,072][105620] Updated weights for policy 1, policy_version 1576450 (0.0010) [2023-12-27 02:51:12,491][105692] Updated weights for policy 0, policy_version 1573039 (0.0008) [2023-12-27 02:51:12,548][105692] Updated weights for policy 0, policy_version 1573049 (0.0008) [2023-12-27 02:51:12,608][105692] Updated weights for policy 0, policy_version 1573059 (0.0008) [2023-12-27 02:51:12,823][105620] Updated weights for policy 1, policy_version 1576460 (0.0010) [2023-12-27 02:51:12,879][105620] Updated weights for policy 1, policy_version 1576470 (0.0010) [2023-12-27 02:51:12,934][105620] Updated weights for policy 1, policy_version 1576480 (0.0010) [2023-12-27 02:51:13,383][105692] Updated weights for policy 0, policy_version 1573069 (0.0008) [2023-12-27 02:51:13,434][105692] Updated weights for policy 0, policy_version 1573079 (0.0008) [2023-12-27 02:51:13,484][105692] Updated weights for policy 0, policy_version 1573089 (0.0008) [2023-12-27 02:51:13,684][105620] Updated weights for policy 1, policy_version 1576490 (0.0010) [2023-12-27 02:51:13,745][105620] Updated weights for policy 1, policy_version 1576500 (0.0010) [2023-12-27 02:51:13,803][105620] Updated weights for policy 1, policy_version 1576510 (0.0010) [2023-12-27 02:51:13,851][105620] Updated weights for policy 1, policy_version 1576520 (0.0010) [2023-12-27 02:51:14,209][105692] Updated weights for policy 0, policy_version 1573099 (0.0009) [2023-12-27 02:51:14,268][105692] Updated weights for policy 0, policy_version 1573109 (0.0010) [2023-12-27 02:51:14,323][105692] Updated weights for policy 0, policy_version 1573119 (0.0010) [2023-12-27 02:51:14,492][105620] Updated weights for policy 1, policy_version 1576530 (0.0006) [2023-12-27 02:51:14,552][105620] Updated weights for policy 1, policy_version 1576540 (0.0006) [2023-12-27 02:51:14,613][105620] Updated weights for policy 1, policy_version 1576550 (0.0006) [2023-12-27 02:51:15,179][105620] Updated weights for policy 1, policy_version 1576560 (0.0006) [2023-12-27 02:51:15,243][105620] Updated weights for policy 1, policy_version 1576570 (0.0007) [2023-12-27 02:51:15,246][105692] Updated weights for policy 0, policy_version 1573130 (0.0010) [2023-12-27 02:51:15,305][105620] Updated weights for policy 1, policy_version 1576580 (0.0009) [2023-12-27 02:51:15,307][105692] Updated weights for policy 0, policy_version 1573140 (0.0008) [2023-12-27 02:51:15,359][105692] Updated weights for policy 0, policy_version 1573150 (0.0008) [2023-12-27 02:51:15,414][105692] Updated weights for policy 0, policy_version 1573160 (0.0009) [2023-12-27 02:51:16,003][105620] Updated weights for policy 1, policy_version 1576590 (0.0010) [2023-12-27 02:51:16,051][105620] Updated weights for policy 1, policy_version 1576600 (0.0010) [2023-12-27 02:51:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.1, 300 sec: 19577.5). Total num frames: 806445056. Throughput: 0: 9584.8, 1: 9764.2. Samples: 806420656. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:51:16,063][104569] Avg episode reward: [(0, '8807.059'), (1, '8808.504')] [2023-12-27 02:51:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001573160_402784256.pth... [2023-12-27 02:51:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001572072_402505728.pth [2023-12-27 02:51:16,109][105620] Updated weights for policy 1, policy_version 1576610 (0.0010) [2023-12-27 02:51:16,137][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001576616_403668992.pth... [2023-12-27 02:51:16,140][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001575432_403365888.pth [2023-12-27 02:51:16,164][105692] Updated weights for policy 0, policy_version 1573170 (0.0008) [2023-12-27 02:51:16,209][105692] Updated weights for policy 0, policy_version 1573180 (0.0008) [2023-12-27 02:51:16,257][105692] Updated weights for policy 0, policy_version 1573190 (0.0008) [2023-12-27 02:51:16,789][105620] Updated weights for policy 1, policy_version 1576620 (0.0010) [2023-12-27 02:51:16,843][105620] Updated weights for policy 1, policy_version 1576630 (0.0010) [2023-12-27 02:51:16,898][105620] Updated weights for policy 1, policy_version 1576640 (0.0010) [2023-12-27 02:51:17,050][105692] Updated weights for policy 0, policy_version 1573200 (0.0009) [2023-12-27 02:51:17,109][105692] Updated weights for policy 0, policy_version 1573210 (0.0009) [2023-12-27 02:51:17,158][105692] Updated weights for policy 0, policy_version 1573220 (0.0008) [2023-12-27 02:51:17,600][105620] Updated weights for policy 1, policy_version 1576650 (0.0009) [2023-12-27 02:51:17,654][105620] Updated weights for policy 1, policy_version 1576660 (0.0005) [2023-12-27 02:51:17,712][105620] Updated weights for policy 1, policy_version 1576670 (0.0005) [2023-12-27 02:51:17,768][105620] Updated weights for policy 1, policy_version 1576680 (0.0005) [2023-12-27 02:51:18,018][105692] Updated weights for policy 0, policy_version 1573230 (0.0009) [2023-12-27 02:51:18,072][105692] Updated weights for policy 0, policy_version 1573240 (0.0010) [2023-12-27 02:51:18,130][105692] Updated weights for policy 0, policy_version 1573250 (0.0009) [2023-12-27 02:51:18,335][105620] Updated weights for policy 1, policy_version 1576690 (0.0006) [2023-12-27 02:51:18,394][105620] Updated weights for policy 1, policy_version 1576700 (0.0007) [2023-12-27 02:51:18,451][105620] Updated weights for policy 1, policy_version 1576710 (0.0009) [2023-12-27 02:51:18,919][105692] Updated weights for policy 0, policy_version 1573260 (0.0010) [2023-12-27 02:51:18,977][105692] Updated weights for policy 0, policy_version 1573270 (0.0009) [2023-12-27 02:51:19,029][105692] Updated weights for policy 0, policy_version 1573280 (0.0009) [2023-12-27 02:51:19,131][105620] Updated weights for policy 1, policy_version 1576720 (0.0009) [2023-12-27 02:51:19,195][105620] Updated weights for policy 1, policy_version 1576730 (0.0009) [2023-12-27 02:51:19,263][105620] Updated weights for policy 1, policy_version 1576740 (0.0009) [2023-12-27 02:51:19,814][105692] Updated weights for policy 0, policy_version 1573290 (0.0009) [2023-12-27 02:51:19,882][105692] Updated weights for policy 0, policy_version 1573300 (0.0008) [2023-12-27 02:51:19,951][105692] Updated weights for policy 0, policy_version 1573310 (0.0008) [2023-12-27 02:51:20,003][105692] Updated weights for policy 0, policy_version 1573320 (0.0009) [2023-12-27 02:51:20,062][105620] Updated weights for policy 1, policy_version 1576750 (0.0010) [2023-12-27 02:51:20,124][105620] Updated weights for policy 1, policy_version 1576760 (0.0009) [2023-12-27 02:51:20,182][105620] Updated weights for policy 1, policy_version 1576770 (0.0010) [2023-12-27 02:51:20,656][105692] Updated weights for policy 0, policy_version 1573330 (0.0009) [2023-12-27 02:51:20,722][105692] Updated weights for policy 0, policy_version 1573340 (0.0009) [2023-12-27 02:51:20,782][105692] Updated weights for policy 0, policy_version 1573350 (0.0009) [2023-12-27 02:51:20,998][105620] Updated weights for policy 1, policy_version 1576780 (0.0008) [2023-12-27 02:51:21,058][105620] Updated weights for policy 1, policy_version 1576790 (0.0008) [2023-12-27 02:51:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 806543360. Throughput: 0: 9401.3, 1: 9916.5. Samples: 806536984. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:51:21,062][104569] Avg episode reward: [(0, '8896.993'), (1, '8899.021')] [2023-12-27 02:51:21,114][105620] Updated weights for policy 1, policy_version 1576800 (0.0009) [2023-12-27 02:51:21,593][105692] Updated weights for policy 0, policy_version 1573360 (0.0010) [2023-12-27 02:51:21,662][105692] Updated weights for policy 0, policy_version 1573370 (0.0010) [2023-12-27 02:51:21,724][105692] Updated weights for policy 0, policy_version 1573380 (0.0010) [2023-12-27 02:51:21,832][105620] Updated weights for policy 1, policy_version 1576810 (0.0008) [2023-12-27 02:51:21,887][105620] Updated weights for policy 1, policy_version 1576820 (0.0009) [2023-12-27 02:51:21,951][105620] Updated weights for policy 1, policy_version 1576830 (0.0010) [2023-12-27 02:51:22,006][105620] Updated weights for policy 1, policy_version 1576840 (0.0009) [2023-12-27 02:51:22,473][105692] Updated weights for policy 0, policy_version 1573390 (0.0008) [2023-12-27 02:51:22,532][105692] Updated weights for policy 0, policy_version 1573400 (0.0008) [2023-12-27 02:51:22,597][105692] Updated weights for policy 0, policy_version 1573410 (0.0008) [2023-12-27 02:51:22,804][105620] Updated weights for policy 1, policy_version 1576850 (0.0010) [2023-12-27 02:51:22,866][105620] Updated weights for policy 1, policy_version 1576860 (0.0010) [2023-12-27 02:51:22,930][105620] Updated weights for policy 1, policy_version 1576870 (0.0010) [2023-12-27 02:51:23,307][105692] Updated weights for policy 0, policy_version 1573420 (0.0009) [2023-12-27 02:51:23,354][105692] Updated weights for policy 0, policy_version 1573430 (0.0008) [2023-12-27 02:51:23,426][105692] Updated weights for policy 0, policy_version 1573440 (0.0008) [2023-12-27 02:51:23,650][105586] KL-divergence is very high: 111.1500 [2023-12-27 02:51:23,651][105620] Updated weights for policy 1, policy_version 1576880 (0.0009) [2023-12-27 02:51:23,689][105586] KL-divergence is very high: 208.4153 [2023-12-27 02:51:23,699][105620] Updated weights for policy 1, policy_version 1576890 (0.0007) [2023-12-27 02:51:23,730][105586] KL-divergence is very high: 218.4597 [2023-12-27 02:51:23,750][105620] Updated weights for policy 1, policy_version 1576900 (0.0010) [2023-12-27 02:51:24,185][105692] Updated weights for policy 0, policy_version 1573450 (0.0011) [2023-12-27 02:51:24,240][105692] Updated weights for policy 0, policy_version 1573460 (0.0008) [2023-12-27 02:51:24,300][105692] Updated weights for policy 0, policy_version 1573470 (0.0008) [2023-12-27 02:51:24,353][105692] Updated weights for policy 0, policy_version 1573480 (0.0008) [2023-12-27 02:51:24,470][105586] KL-divergence is very high: 195.4834 [2023-12-27 02:51:24,496][105620] Updated weights for policy 1, policy_version 1576910 (0.0010) [2023-12-27 02:51:24,520][105586] KL-divergence is very high: 162.5493 [2023-12-27 02:51:24,551][105620] Updated weights for policy 1, policy_version 1576920 (0.0010) [2023-12-27 02:51:24,560][105586] KL-divergence is very high: 127.8823 [2023-12-27 02:51:24,606][105620] Updated weights for policy 1, policy_version 1576930 (0.0010) [2023-12-27 02:51:25,143][105692] Updated weights for policy 0, policy_version 1573490 (0.0008) [2023-12-27 02:51:25,200][105692] Updated weights for policy 0, policy_version 1573500 (0.0008) [2023-12-27 02:51:25,264][105692] Updated weights for policy 0, policy_version 1573510 (0.0008) [2023-12-27 02:51:25,353][105620] Updated weights for policy 1, policy_version 1576940 (0.0010) [2023-12-27 02:51:25,415][105620] Updated weights for policy 1, policy_version 1576950 (0.0010) [2023-12-27 02:51:25,473][105620] Updated weights for policy 1, policy_version 1576960 (0.0010) [2023-12-27 02:51:26,013][105692] Updated weights for policy 0, policy_version 1573520 (0.0009) [2023-12-27 02:51:26,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19114.6, 300 sec: 19549.7). Total num frames: 806633472. Throughput: 0: 9286.4, 1: 9909.2. Samples: 806647808. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:51:26,062][104569] Avg episode reward: [(0, '8804.461'), (1, '8898.876')] [2023-12-27 02:51:26,082][105692] Updated weights for policy 0, policy_version 1573530 (0.0010) [2023-12-27 02:51:26,144][105692] Updated weights for policy 0, policy_version 1573540 (0.0010) [2023-12-27 02:51:26,170][105620] Updated weights for policy 1, policy_version 1576970 (0.0010) [2023-12-27 02:51:26,235][105620] Updated weights for policy 1, policy_version 1576980 (0.0008) [2023-12-27 02:51:26,303][105620] Updated weights for policy 1, policy_version 1576990 (0.0007) [2023-12-27 02:51:26,372][105620] Updated weights for policy 1, policy_version 1577000 (0.0007) [2023-12-27 02:51:26,834][105692] Updated weights for policy 0, policy_version 1573550 (0.0007) [2023-12-27 02:51:26,885][105692] Updated weights for policy 0, policy_version 1573560 (0.0008) [2023-12-27 02:51:26,945][105692] Updated weights for policy 0, policy_version 1573570 (0.0006) [2023-12-27 02:51:26,953][105620] Updated weights for policy 1, policy_version 1577010 (0.0011) [2023-12-27 02:51:27,012][105620] Updated weights for policy 1, policy_version 1577020 (0.0010) [2023-12-27 02:51:27,064][105620] Updated weights for policy 1, policy_version 1577030 (0.0011) [2023-12-27 02:51:27,658][105692] Updated weights for policy 0, policy_version 1573580 (0.0009) [2023-12-27 02:51:27,716][105692] Updated weights for policy 0, policy_version 1573590 (0.0010) [2023-12-27 02:51:27,770][105692] Updated weights for policy 0, policy_version 1573600 (0.0010) [2023-12-27 02:51:27,813][105620] Updated weights for policy 1, policy_version 1577040 (0.0010) [2023-12-27 02:51:27,860][105620] Updated weights for policy 1, policy_version 1577050 (0.0010) [2023-12-27 02:51:27,911][105620] Updated weights for policy 1, policy_version 1577060 (0.0010) [2023-12-27 02:51:28,504][105692] Updated weights for policy 0, policy_version 1573610 (0.0009) [2023-12-27 02:51:28,563][105692] Updated weights for policy 0, policy_version 1573620 (0.0009) [2023-12-27 02:51:28,628][105692] Updated weights for policy 0, policy_version 1573630 (0.0010) [2023-12-27 02:51:28,676][105620] Updated weights for policy 1, policy_version 1577070 (0.0010) [2023-12-27 02:51:28,687][105692] Updated weights for policy 0, policy_version 1573640 (0.0010) [2023-12-27 02:51:28,741][105620] Updated weights for policy 1, policy_version 1577080 (0.0011) [2023-12-27 02:51:28,804][105620] Updated weights for policy 1, policy_version 1577090 (0.0008) [2023-12-27 02:51:29,422][105692] Updated weights for policy 0, policy_version 1573650 (0.0008) [2023-12-27 02:51:29,435][105620] Updated weights for policy 1, policy_version 1577100 (0.0007) [2023-12-27 02:51:29,472][105692] Updated weights for policy 0, policy_version 1573660 (0.0009) [2023-12-27 02:51:29,486][105620] Updated weights for policy 1, policy_version 1577110 (0.0010) [2023-12-27 02:51:29,520][105692] Updated weights for policy 0, policy_version 1573670 (0.0005) [2023-12-27 02:51:29,544][105620] Updated weights for policy 1, policy_version 1577120 (0.0010) [2023-12-27 02:51:30,209][105692] Updated weights for policy 0, policy_version 1573680 (0.0008) [2023-12-27 02:51:30,245][105620] Updated weights for policy 1, policy_version 1577130 (0.0009) [2023-12-27 02:51:30,275][105692] Updated weights for policy 0, policy_version 1573690 (0.0008) [2023-12-27 02:51:30,302][105620] Updated weights for policy 1, policy_version 1577140 (0.0006) [2023-12-27 02:51:30,331][105692] Updated weights for policy 0, policy_version 1573700 (0.0009) [2023-12-27 02:51:30,367][105620] Updated weights for policy 1, policy_version 1577150 (0.0007) [2023-12-27 02:51:30,421][105620] Updated weights for policy 1, policy_version 1577160 (0.0008) [2023-12-27 02:51:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19549.7). Total num frames: 806731776. Throughput: 0: 9329.3, 1: 9958.1. Samples: 806707400. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:51:31,062][104569] Avg episode reward: [(0, '8535.301'), (1, '8808.938')] [2023-12-27 02:51:31,076][105692] Updated weights for policy 0, policy_version 1573710 (0.0009) [2023-12-27 02:51:31,092][105620] Updated weights for policy 1, policy_version 1577170 (0.0007) [2023-12-27 02:51:31,141][105692] Updated weights for policy 0, policy_version 1573720 (0.0011) [2023-12-27 02:51:31,156][105620] Updated weights for policy 1, policy_version 1577180 (0.0007) [2023-12-27 02:51:31,204][105692] Updated weights for policy 0, policy_version 1573730 (0.0011) [2023-12-27 02:51:31,211][105620] Updated weights for policy 1, policy_version 1577190 (0.0006) [2023-12-27 02:51:31,221][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001577192_403816448.pth... [2023-12-27 02:51:31,226][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001576008_403513344.pth [2023-12-27 02:51:31,236][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001573736_402931712.pth... [2023-12-27 02:51:31,242][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001572648_402653184.pth [2023-12-27 02:51:31,919][105692] Updated weights for policy 0, policy_version 1573740 (0.0009) [2023-12-27 02:51:31,936][105620] Updated weights for policy 1, policy_version 1577200 (0.0008) [2023-12-27 02:51:31,978][105692] Updated weights for policy 0, policy_version 1573750 (0.0006) [2023-12-27 02:51:31,991][105620] Updated weights for policy 1, policy_version 1577211 (0.0009) [2023-12-27 02:51:32,037][105692] Updated weights for policy 0, policy_version 1573760 (0.0011) [2023-12-27 02:51:32,051][105620] Updated weights for policy 1, policy_version 1577221 (0.0007) [2023-12-27 02:51:32,744][105692] Updated weights for policy 0, policy_version 1573770 (0.0011) [2023-12-27 02:51:32,802][105692] Updated weights for policy 0, policy_version 1573780 (0.0010) [2023-12-27 02:51:32,832][105620] Updated weights for policy 1, policy_version 1577231 (0.0007) [2023-12-27 02:51:32,861][105692] Updated weights for policy 0, policy_version 1573790 (0.0011) [2023-12-27 02:51:32,894][105620] Updated weights for policy 1, policy_version 1577241 (0.0006) [2023-12-27 02:51:32,917][105692] Updated weights for policy 0, policy_version 1573800 (0.0008) [2023-12-27 02:51:32,949][105620] Updated weights for policy 1, policy_version 1577251 (0.0008) [2023-12-27 02:51:33,550][105692] Updated weights for policy 0, policy_version 1573810 (0.0009) [2023-12-27 02:51:33,605][105692] Updated weights for policy 0, policy_version 1573820 (0.0008) [2023-12-27 02:51:33,665][105692] Updated weights for policy 0, policy_version 1573830 (0.0009) [2023-12-27 02:51:33,728][105620] Updated weights for policy 1, policy_version 1577261 (0.0009) [2023-12-27 02:51:33,773][105620] Updated weights for policy 1, policy_version 1577271 (0.0008) [2023-12-27 02:51:33,818][105620] Updated weights for policy 1, policy_version 1577281 (0.0008) [2023-12-27 02:51:34,404][105692] Updated weights for policy 0, policy_version 1573840 (0.0010) [2023-12-27 02:51:34,461][105692] Updated weights for policy 0, policy_version 1573850 (0.0010) [2023-12-27 02:51:34,517][105692] Updated weights for policy 0, policy_version 1573860 (0.0010) [2023-12-27 02:51:34,645][105620] Updated weights for policy 1, policy_version 1577291 (0.0008) [2023-12-27 02:51:34,709][105620] Updated weights for policy 1, policy_version 1577301 (0.0008) [2023-12-27 02:51:34,776][105620] Updated weights for policy 1, policy_version 1577311 (0.0008) [2023-12-27 02:51:35,215][105692] Updated weights for policy 0, policy_version 1573870 (0.0007) [2023-12-27 02:51:35,266][105692] Updated weights for policy 0, policy_version 1573880 (0.0005) [2023-12-27 02:51:35,321][105692] Updated weights for policy 0, policy_version 1573890 (0.0005) [2023-12-27 02:51:35,640][105620] Updated weights for policy 1, policy_version 1577321 (0.0008) [2023-12-27 02:51:35,695][105620] Updated weights for policy 1, policy_version 1577331 (0.0010) [2023-12-27 02:51:35,750][105620] Updated weights for policy 1, policy_version 1577342 (0.0010) [2023-12-27 02:51:35,802][105620] Updated weights for policy 1, policy_version 1577352 (0.0008) [2023-12-27 02:51:35,855][105692] Updated weights for policy 0, policy_version 1573900 (0.0007) [2023-12-27 02:51:35,913][105692] Updated weights for policy 0, policy_version 1573910 (0.0010) [2023-12-27 02:51:35,971][105692] Updated weights for policy 0, policy_version 1573920 (0.0010) [2023-12-27 02:51:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 806838272. Throughput: 0: 9360.3, 1: 9865.9. Samples: 806822468. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:51:36,063][104569] Avg episode reward: [(0, '8719.049'), (1, '8991.996')] [2023-12-27 02:51:36,606][105692] Updated weights for policy 0, policy_version 1573930 (0.0008) [2023-12-27 02:51:36,646][105620] Updated weights for policy 1, policy_version 1577362 (0.0008) [2023-12-27 02:51:36,662][105692] Updated weights for policy 0, policy_version 1573940 (0.0006) [2023-12-27 02:51:36,703][105620] Updated weights for policy 1, policy_version 1577372 (0.0009) [2023-12-27 02:51:36,721][105692] Updated weights for policy 0, policy_version 1573950 (0.0006) [2023-12-27 02:51:36,760][105620] Updated weights for policy 1, policy_version 1577382 (0.0009) [2023-12-27 02:51:36,781][105692] Updated weights for policy 0, policy_version 1573960 (0.0006) [2023-12-27 02:51:37,476][105692] Updated weights for policy 0, policy_version 1573970 (0.0008) [2023-12-27 02:51:37,535][105692] Updated weights for policy 0, policy_version 1573980 (0.0010) [2023-12-27 02:51:37,554][105620] Updated weights for policy 1, policy_version 1577392 (0.0010) [2023-12-27 02:51:37,597][105692] Updated weights for policy 0, policy_version 1573990 (0.0007) [2023-12-27 02:51:37,615][105620] Updated weights for policy 1, policy_version 1577402 (0.0007) [2023-12-27 02:51:37,676][105620] Updated weights for policy 1, policy_version 1577412 (0.0009) [2023-12-27 02:51:38,317][105692] Updated weights for policy 0, policy_version 1574000 (0.0007) [2023-12-27 02:51:38,384][105692] Updated weights for policy 0, policy_version 1574010 (0.0009) [2023-12-27 02:51:38,441][105620] Updated weights for policy 1, policy_version 1577422 (0.0009) [2023-12-27 02:51:38,446][105692] Updated weights for policy 0, policy_version 1574020 (0.0011) [2023-12-27 02:51:38,497][105620] Updated weights for policy 1, policy_version 1577432 (0.0009) [2023-12-27 02:51:38,541][105620] Updated weights for policy 1, policy_version 1577442 (0.0008) [2023-12-27 02:51:39,149][105692] Updated weights for policy 0, policy_version 1574030 (0.0010) [2023-12-27 02:51:39,214][105692] Updated weights for policy 0, policy_version 1574040 (0.0011) [2023-12-27 02:51:39,277][105692] Updated weights for policy 0, policy_version 1574050 (0.0011) [2023-12-27 02:51:39,289][105620] Updated weights for policy 1, policy_version 1577452 (0.0007) [2023-12-27 02:51:39,360][105620] Updated weights for policy 1, policy_version 1577462 (0.0009) [2023-12-27 02:51:39,425][105620] Updated weights for policy 1, policy_version 1577472 (0.0011) [2023-12-27 02:51:40,014][105692] Updated weights for policy 0, policy_version 1574060 (0.0011) [2023-12-27 02:51:40,062][105692] Updated weights for policy 0, policy_version 1574070 (0.0011) [2023-12-27 02:51:40,112][105692] Updated weights for policy 0, policy_version 1574080 (0.0011) [2023-12-27 02:51:40,166][105620] Updated weights for policy 1, policy_version 1577482 (0.0009) [2023-12-27 02:51:40,228][105620] Updated weights for policy 1, policy_version 1577492 (0.0007) [2023-12-27 02:51:40,280][105620] Updated weights for policy 1, policy_version 1577502 (0.0007) [2023-12-27 02:51:40,324][105620] Updated weights for policy 1, policy_version 1577512 (0.0009) [2023-12-27 02:51:40,902][105692] Updated weights for policy 0, policy_version 1574090 (0.0011) [2023-12-27 02:51:40,941][105620] Updated weights for policy 1, policy_version 1577522 (0.0008) [2023-12-27 02:51:40,965][105692] Updated weights for policy 0, policy_version 1574100 (0.0011) [2023-12-27 02:51:41,004][105620] Updated weights for policy 1, policy_version 1577532 (0.0008) [2023-12-27 02:51:41,030][105692] Updated weights for policy 0, policy_version 1574110 (0.0011) [2023-12-27 02:51:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19549.7). Total num frames: 806920192. Throughput: 0: 9460.0, 1: 9760.3. Samples: 806938384. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:51:41,062][104569] Avg episode reward: [(0, '8988.477'), (1, '9081.025')] [2023-12-27 02:51:41,074][105620] Updated weights for policy 1, policy_version 1577543 (0.0009) [2023-12-27 02:51:41,092][105692] Updated weights for policy 0, policy_version 1574120 (0.0011) [2023-12-27 02:51:41,874][105620] Updated weights for policy 1, policy_version 1577553 (0.0008) [2023-12-27 02:51:41,897][105692] Updated weights for policy 0, policy_version 1574130 (0.0006) [2023-12-27 02:51:41,936][105620] Updated weights for policy 1, policy_version 1577563 (0.0006) [2023-12-27 02:51:41,959][105692] Updated weights for policy 0, policy_version 1574140 (0.0008) [2023-12-27 02:51:42,001][105620] Updated weights for policy 1, policy_version 1577573 (0.0008) [2023-12-27 02:51:42,020][105692] Updated weights for policy 0, policy_version 1574150 (0.0008) [2023-12-27 02:51:42,778][105692] Updated weights for policy 0, policy_version 1574160 (0.0009) [2023-12-27 02:51:42,781][105620] Updated weights for policy 1, policy_version 1577583 (0.0008) [2023-12-27 02:51:42,836][105692] Updated weights for policy 0, policy_version 1574170 (0.0008) [2023-12-27 02:51:42,839][105620] Updated weights for policy 1, policy_version 1577593 (0.0009) [2023-12-27 02:51:42,896][105620] Updated weights for policy 1, policy_version 1577603 (0.0007) [2023-12-27 02:51:42,901][105692] Updated weights for policy 0, policy_version 1574180 (0.0008) [2023-12-27 02:51:43,468][105620] Updated weights for policy 1, policy_version 1577613 (0.0006) [2023-12-27 02:51:43,528][105620] Updated weights for policy 1, policy_version 1577623 (0.0005) [2023-12-27 02:51:43,585][105620] Updated weights for policy 1, policy_version 1577633 (0.0008) [2023-12-27 02:51:43,758][105692] Updated weights for policy 0, policy_version 1574190 (0.0008) [2023-12-27 02:51:43,822][105692] Updated weights for policy 0, policy_version 1574200 (0.0010) [2023-12-27 02:51:43,879][105692] Updated weights for policy 0, policy_version 1574211 (0.0008) [2023-12-27 02:51:44,224][105620] Updated weights for policy 1, policy_version 1577643 (0.0009) [2023-12-27 02:51:44,292][105620] Updated weights for policy 1, policy_version 1577653 (0.0005) [2023-12-27 02:51:44,358][105620] Updated weights for policy 1, policy_version 1577663 (0.0006) [2023-12-27 02:51:44,634][105692] Updated weights for policy 0, policy_version 1574221 (0.0010) [2023-12-27 02:51:44,694][105692] Updated weights for policy 0, policy_version 1574231 (0.0008) [2023-12-27 02:51:44,757][105692] Updated weights for policy 0, policy_version 1574241 (0.0008) [2023-12-27 02:51:44,890][105620] Updated weights for policy 1, policy_version 1577673 (0.0005) [2023-12-27 02:51:44,955][105620] Updated weights for policy 1, policy_version 1577683 (0.0008) [2023-12-27 02:51:45,020][105620] Updated weights for policy 1, policy_version 1577693 (0.0007) [2023-12-27 02:51:45,074][105620] Updated weights for policy 1, policy_version 1577703 (0.0008) [2023-12-27 02:51:45,551][105692] Updated weights for policy 0, policy_version 1574251 (0.0008) [2023-12-27 02:51:45,605][105692] Updated weights for policy 0, policy_version 1574261 (0.0005) [2023-12-27 02:51:45,659][105692] Updated weights for policy 0, policy_version 1574271 (0.0007) [2023-12-27 02:51:45,709][105620] Updated weights for policy 1, policy_version 1577713 (0.0006) [2023-12-27 02:51:45,774][105620] Updated weights for policy 1, policy_version 1577723 (0.0009) [2023-12-27 02:51:45,827][105620] Updated weights for policy 1, policy_version 1577733 (0.0008) [2023-12-27 02:51:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 807026688. Throughput: 0: 9299.9, 1: 9811.7. Samples: 806994180. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:51:46,062][104569] Avg episode reward: [(0, '8806.661'), (1, '9080.599')] [2023-12-27 02:51:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001574280_403070976.pth... [2023-12-27 02:51:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001577736_403955712.pth... [2023-12-27 02:51:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001573160_402784256.pth [2023-12-27 02:51:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001576616_403668992.pth [2023-12-27 02:51:46,301][105692] Updated weights for policy 0, policy_version 1574281 (0.0010) [2023-12-27 02:51:46,356][105692] Updated weights for policy 0, policy_version 1574291 (0.0006) [2023-12-27 02:51:46,418][105692] Updated weights for policy 0, policy_version 1574301 (0.0006) [2023-12-27 02:51:46,475][105692] Updated weights for policy 0, policy_version 1574311 (0.0008) [2023-12-27 02:51:46,596][105620] Updated weights for policy 1, policy_version 1577743 (0.0009) [2023-12-27 02:51:46,649][105620] Updated weights for policy 1, policy_version 1577753 (0.0008) [2023-12-27 02:51:46,716][105620] Updated weights for policy 1, policy_version 1577763 (0.0007) [2023-12-27 02:51:47,190][105692] Updated weights for policy 0, policy_version 1574321 (0.0007) [2023-12-27 02:51:47,244][105692] Updated weights for policy 0, policy_version 1574331 (0.0008) [2023-12-27 02:51:47,303][105692] Updated weights for policy 0, policy_version 1574341 (0.0008) [2023-12-27 02:51:47,351][105620] Updated weights for policy 1, policy_version 1577773 (0.0010) [2023-12-27 02:51:47,402][105620] Updated weights for policy 1, policy_version 1577783 (0.0010) [2023-12-27 02:51:47,456][105620] Updated weights for policy 1, policy_version 1577793 (0.0010) [2023-12-27 02:51:48,080][105692] Updated weights for policy 0, policy_version 1574351 (0.0007) [2023-12-27 02:51:48,135][105692] Updated weights for policy 0, policy_version 1574361 (0.0006) [2023-12-27 02:51:48,169][105620] Updated weights for policy 1, policy_version 1577803 (0.0010) [2023-12-27 02:51:48,189][105692] Updated weights for policy 0, policy_version 1574371 (0.0005) [2023-12-27 02:51:48,221][105620] Updated weights for policy 1, policy_version 1577813 (0.0010) [2023-12-27 02:51:48,272][105620] Updated weights for policy 1, policy_version 1577823 (0.0010) [2023-12-27 02:51:48,896][105692] Updated weights for policy 0, policy_version 1574381 (0.0006) [2023-12-27 02:51:48,961][105692] Updated weights for policy 0, policy_version 1574391 (0.0008) [2023-12-27 02:51:49,014][105692] Updated weights for policy 0, policy_version 1574401 (0.0008) [2023-12-27 02:51:49,053][105620] Updated weights for policy 1, policy_version 1577833 (0.0009) [2023-12-27 02:51:49,110][105620] Updated weights for policy 1, policy_version 1577843 (0.0010) [2023-12-27 02:51:49,167][105620] Updated weights for policy 1, policy_version 1577853 (0.0009) [2023-12-27 02:51:49,238][105620] Updated weights for policy 1, policy_version 1577863 (0.0009) [2023-12-27 02:51:49,798][105692] Updated weights for policy 0, policy_version 1574411 (0.0009) [2023-12-27 02:51:49,858][105692] Updated weights for policy 0, policy_version 1574421 (0.0010) [2023-12-27 02:51:49,919][105692] Updated weights for policy 0, policy_version 1574431 (0.0010) [2023-12-27 02:51:49,995][105620] Updated weights for policy 1, policy_version 1577873 (0.0006) [2023-12-27 02:51:50,046][105620] Updated weights for policy 1, policy_version 1577883 (0.0005) [2023-12-27 02:51:50,105][105620] Updated weights for policy 1, policy_version 1577893 (0.0006) [2023-12-27 02:51:50,706][105692] Updated weights for policy 0, policy_version 1574441 (0.0008) [2023-12-27 02:51:50,772][105692] Updated weights for policy 0, policy_version 1574451 (0.0006) [2023-12-27 02:51:50,833][105692] Updated weights for policy 0, policy_version 1574461 (0.0006) [2023-12-27 02:51:50,855][105620] Updated weights for policy 1, policy_version 1577903 (0.0009) [2023-12-27 02:51:50,898][105692] Updated weights for policy 0, policy_version 1574471 (0.0006) [2023-12-27 02:51:50,920][105620] Updated weights for policy 1, policy_version 1577913 (0.0009) [2023-12-27 02:51:50,985][105620] Updated weights for policy 1, policy_version 1577923 (0.0008) [2023-12-27 02:51:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 807124992. Throughput: 0: 9346.4, 1: 9829.7. Samples: 807110888. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:51:51,062][104569] Avg episode reward: [(0, '8898.518'), (1, '9171.853')] [2023-12-27 02:51:51,579][105692] Updated weights for policy 0, policy_version 1574481 (0.0009) [2023-12-27 02:51:51,645][105692] Updated weights for policy 0, policy_version 1574491 (0.0008) [2023-12-27 02:51:51,706][105692] Updated weights for policy 0, policy_version 1574501 (0.0009) [2023-12-27 02:51:51,729][105620] Updated weights for policy 1, policy_version 1577933 (0.0009) [2023-12-27 02:51:51,793][105620] Updated weights for policy 1, policy_version 1577943 (0.0009) [2023-12-27 02:51:51,858][105620] Updated weights for policy 1, policy_version 1577953 (0.0009) [2023-12-27 02:51:52,459][105692] Updated weights for policy 0, policy_version 1574511 (0.0008) [2023-12-27 02:51:52,526][105692] Updated weights for policy 0, policy_version 1574521 (0.0006) [2023-12-27 02:51:52,595][105692] Updated weights for policy 0, policy_version 1574531 (0.0007) [2023-12-27 02:51:52,615][105620] Updated weights for policy 1, policy_version 1577963 (0.0007) [2023-12-27 02:51:52,668][105620] Updated weights for policy 1, policy_version 1577973 (0.0008) [2023-12-27 02:51:52,715][105620] Updated weights for policy 1, policy_version 1577983 (0.0009) [2023-12-27 02:51:53,257][105692] Updated weights for policy 0, policy_version 1574541 (0.0009) [2023-12-27 02:51:53,316][105692] Updated weights for policy 0, policy_version 1574551 (0.0009) [2023-12-27 02:51:53,363][105692] Updated weights for policy 0, policy_version 1574561 (0.0008) [2023-12-27 02:51:53,511][105620] Updated weights for policy 1, policy_version 1577993 (0.0009) [2023-12-27 02:51:53,557][105620] Updated weights for policy 1, policy_version 1578003 (0.0009) [2023-12-27 02:51:53,604][105620] Updated weights for policy 1, policy_version 1578013 (0.0009) [2023-12-27 02:51:53,649][105620] Updated weights for policy 1, policy_version 1578023 (0.0008) [2023-12-27 02:51:54,113][105692] Updated weights for policy 0, policy_version 1574571 (0.0009) [2023-12-27 02:51:54,179][105692] Updated weights for policy 0, policy_version 1574581 (0.0010) [2023-12-27 02:51:54,244][105692] Updated weights for policy 0, policy_version 1574591 (0.0009) [2023-12-27 02:51:54,426][105620] Updated weights for policy 1, policy_version 1578033 (0.0006) [2023-12-27 02:51:54,488][105620] Updated weights for policy 1, policy_version 1578043 (0.0008) [2023-12-27 02:51:54,542][105620] Updated weights for policy 1, policy_version 1578053 (0.0009) [2023-12-27 02:51:54,996][105692] Updated weights for policy 0, policy_version 1574601 (0.0009) [2023-12-27 02:51:55,051][105692] Updated weights for policy 0, policy_version 1574611 (0.0009) [2023-12-27 02:51:55,109][105692] Updated weights for policy 0, policy_version 1574621 (0.0009) [2023-12-27 02:51:55,176][105692] Updated weights for policy 0, policy_version 1574631 (0.0010) [2023-12-27 02:51:55,251][105620] Updated weights for policy 1, policy_version 1578063 (0.0008) [2023-12-27 02:51:55,304][105620] Updated weights for policy 1, policy_version 1578073 (0.0008) [2023-12-27 02:51:55,360][105620] Updated weights for policy 1, policy_version 1578083 (0.0009) [2023-12-27 02:51:55,872][105692] Updated weights for policy 0, policy_version 1574641 (0.0009) [2023-12-27 02:51:55,936][105692] Updated weights for policy 0, policy_version 1574651 (0.0009) [2023-12-27 02:51:55,983][105692] Updated weights for policy 0, policy_version 1574661 (0.0009) [2023-12-27 02:51:56,008][105620] Updated weights for policy 1, policy_version 1578093 (0.0008) [2023-12-27 02:51:56,053][105620] Updated weights for policy 1, policy_version 1578103 (0.0008) [2023-12-27 02:51:56,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19114.7, 300 sec: 19549.7). Total num frames: 807215104. Throughput: 0: 9433.2, 1: 9656.8. Samples: 807223764. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:51:56,062][104569] Avg episode reward: [(0, '8988.629'), (1, '9174.421')] [2023-12-27 02:51:56,109][105620] Updated weights for policy 1, policy_version 1578113 (0.0008) [2023-12-27 02:51:56,750][105692] Updated weights for policy 0, policy_version 1574671 (0.0008) [2023-12-27 02:51:56,797][105692] Updated weights for policy 0, policy_version 1574681 (0.0008) [2023-12-27 02:51:56,848][105692] Updated weights for policy 0, policy_version 1574691 (0.0008) [2023-12-27 02:51:56,874][105620] Updated weights for policy 1, policy_version 1578123 (0.0010) [2023-12-27 02:51:56,919][105620] Updated weights for policy 1, policy_version 1578133 (0.0008) [2023-12-27 02:51:56,966][105620] Updated weights for policy 1, policy_version 1578143 (0.0009) [2023-12-27 02:51:57,586][105692] Updated weights for policy 0, policy_version 1574701 (0.0007) [2023-12-27 02:51:57,650][105692] Updated weights for policy 0, policy_version 1574711 (0.0005) [2023-12-27 02:51:57,662][105620] Updated weights for policy 1, policy_version 1578153 (0.0008) [2023-12-27 02:51:57,718][105692] Updated weights for policy 0, policy_version 1574721 (0.0007) [2023-12-27 02:51:57,723][105620] Updated weights for policy 1, policy_version 1578163 (0.0007) [2023-12-27 02:51:57,782][105620] Updated weights for policy 1, policy_version 1578173 (0.0006) [2023-12-27 02:51:57,840][105620] Updated weights for policy 1, policy_version 1578183 (0.0006) [2023-12-27 02:51:58,396][105692] Updated weights for policy 0, policy_version 1574731 (0.0008) [2023-12-27 02:51:58,455][105692] Updated weights for policy 0, policy_version 1574741 (0.0008) [2023-12-27 02:51:58,513][105620] Updated weights for policy 1, policy_version 1578193 (0.0007) [2023-12-27 02:51:58,518][105692] Updated weights for policy 0, policy_version 1574751 (0.0008) [2023-12-27 02:51:58,585][105620] Updated weights for policy 1, policy_version 1578203 (0.0007) [2023-12-27 02:51:58,650][105620] Updated weights for policy 1, policy_version 1578213 (0.0010) [2023-12-27 02:51:59,359][105692] Updated weights for policy 0, policy_version 1574761 (0.0008) [2023-12-27 02:51:59,434][105692] Updated weights for policy 0, policy_version 1574771 (0.0008) [2023-12-27 02:51:59,485][105692] Updated weights for policy 0, policy_version 1574781 (0.0008) [2023-12-27 02:51:59,502][105620] Updated weights for policy 1, policy_version 1578223 (0.0010) [2023-12-27 02:51:59,534][105692] Updated weights for policy 0, policy_version 1574791 (0.0008) [2023-12-27 02:51:59,551][105620] Updated weights for policy 1, policy_version 1578233 (0.0010) [2023-12-27 02:51:59,601][105620] Updated weights for policy 1, policy_version 1578243 (0.0011) [2023-12-27 02:52:00,248][105692] Updated weights for policy 0, policy_version 1574801 (0.0010) [2023-12-27 02:52:00,306][105692] Updated weights for policy 0, policy_version 1574811 (0.0010) [2023-12-27 02:52:00,354][105620] Updated weights for policy 1, policy_version 1578253 (0.0011) [2023-12-27 02:52:00,358][105692] Updated weights for policy 0, policy_version 1574821 (0.0008) [2023-12-27 02:52:00,410][105620] Updated weights for policy 1, policy_version 1578263 (0.0011) [2023-12-27 02:52:00,481][105620] Updated weights for policy 1, policy_version 1578273 (0.0011) [2023-12-27 02:52:00,972][105692] Updated weights for policy 0, policy_version 1574831 (0.0006) [2023-12-27 02:52:01,027][105692] Updated weights for policy 0, policy_version 1574841 (0.0010) [2023-12-27 02:52:01,062][104569] Fps is (10 sec: 18022.3, 60 sec: 18978.1, 300 sec: 19521.9). Total num frames: 807305216. Throughput: 0: 9466.1, 1: 9660.6. Samples: 807281352. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:52:01,062][104569] Avg episode reward: [(0, '8715.963'), (1, '9082.895')] [2023-12-27 02:52:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001578280_404094976.pth... [2023-12-27 02:52:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001577192_403816448.pth [2023-12-27 02:52:01,086][105692] Updated weights for policy 0, policy_version 1574851 (0.0008) [2023-12-27 02:52:01,110][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001574856_403218432.pth... [2023-12-27 02:52:01,111][105620] Updated weights for policy 1, policy_version 1578283 (0.0010) [2023-12-27 02:52:01,115][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001573736_402931712.pth [2023-12-27 02:52:01,176][105620] Updated weights for policy 1, policy_version 1578293 (0.0010) [2023-12-27 02:52:01,229][105620] Updated weights for policy 1, policy_version 1578303 (0.0011) [2023-12-27 02:52:01,791][105692] Updated weights for policy 0, policy_version 1574861 (0.0007) [2023-12-27 02:52:01,850][105692] Updated weights for policy 0, policy_version 1574871 (0.0008) [2023-12-27 02:52:01,909][105692] Updated weights for policy 0, policy_version 1574881 (0.0008) [2023-12-27 02:52:01,971][105620] Updated weights for policy 1, policy_version 1578313 (0.0009) [2023-12-27 02:52:02,035][105620] Updated weights for policy 1, policy_version 1578323 (0.0011) [2023-12-27 02:52:02,094][105620] Updated weights for policy 1, policy_version 1578333 (0.0011) [2023-12-27 02:52:02,143][105620] Updated weights for policy 1, policy_version 1578343 (0.0011) [2023-12-27 02:52:02,662][105692] Updated weights for policy 0, policy_version 1574891 (0.0008) [2023-12-27 02:52:02,712][105692] Updated weights for policy 0, policy_version 1574901 (0.0006) [2023-12-27 02:52:02,764][105692] Updated weights for policy 0, policy_version 1574911 (0.0005) [2023-12-27 02:52:02,914][105620] Updated weights for policy 1, policy_version 1578353 (0.0010) [2023-12-27 02:52:02,958][105620] Updated weights for policy 1, policy_version 1578363 (0.0010) [2023-12-27 02:52:03,006][105620] Updated weights for policy 1, policy_version 1578373 (0.0010) [2023-12-27 02:52:03,337][105692] Updated weights for policy 0, policy_version 1574921 (0.0005) [2023-12-27 02:52:03,389][105692] Updated weights for policy 0, policy_version 1574931 (0.0006) [2023-12-27 02:52:03,432][105692] Updated weights for policy 0, policy_version 1574941 (0.0005) [2023-12-27 02:52:03,485][105692] Updated weights for policy 0, policy_version 1574951 (0.0005) [2023-12-27 02:52:03,757][105620] Updated weights for policy 1, policy_version 1578383 (0.0010) [2023-12-27 02:52:03,825][105620] Updated weights for policy 1, policy_version 1578393 (0.0010) [2023-12-27 02:52:03,883][105620] Updated weights for policy 1, policy_version 1578403 (0.0010) [2023-12-27 02:52:04,091][105692] Updated weights for policy 0, policy_version 1574961 (0.0008) [2023-12-27 02:52:04,154][105692] Updated weights for policy 0, policy_version 1574971 (0.0008) [2023-12-27 02:52:04,217][105692] Updated weights for policy 0, policy_version 1574981 (0.0006) [2023-12-27 02:52:04,638][105620] Updated weights for policy 1, policy_version 1578413 (0.0010) [2023-12-27 02:52:04,711][105620] Updated weights for policy 1, policy_version 1578423 (0.0009) [2023-12-27 02:52:04,767][105620] Updated weights for policy 1, policy_version 1578433 (0.0010) [2023-12-27 02:52:04,838][105692] Updated weights for policy 0, policy_version 1574991 (0.0008) [2023-12-27 02:52:04,896][105692] Updated weights for policy 0, policy_version 1575001 (0.0009) [2023-12-27 02:52:04,966][105692] Updated weights for policy 0, policy_version 1575011 (0.0010) [2023-12-27 02:52:05,509][105620] Updated weights for policy 1, policy_version 1578443 (0.0008) [2023-12-27 02:52:05,559][105620] Updated weights for policy 1, policy_version 1578453 (0.0009) [2023-12-27 02:52:05,613][105620] Updated weights for policy 1, policy_version 1578463 (0.0009) [2023-12-27 02:52:05,676][105692] Updated weights for policy 0, policy_version 1575021 (0.0009) [2023-12-27 02:52:05,729][105692] Updated weights for policy 0, policy_version 1575031 (0.0008) [2023-12-27 02:52:05,783][105692] Updated weights for policy 0, policy_version 1575041 (0.0010) [2023-12-27 02:52:06,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 807411712. Throughput: 0: 9636.8, 1: 9524.3. Samples: 807399240. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:52:06,063][104569] Avg episode reward: [(0, '8717.249'), (1, '8990.213')] [2023-12-27 02:52:06,391][105620] Updated weights for policy 1, policy_version 1578474 (0.0010) [2023-12-27 02:52:06,451][105620] Updated weights for policy 1, policy_version 1578484 (0.0011) [2023-12-27 02:52:06,469][105586] KL-divergence is very high: 147.3012 [2023-12-27 02:52:06,487][105692] Updated weights for policy 0, policy_version 1575051 (0.0008) [2023-12-27 02:52:06,497][105620] Updated weights for policy 1, policy_version 1578494 (0.0011) [2023-12-27 02:52:06,509][105586] KL-divergence is very high: 228.9196 [2023-12-27 02:52:06,537][105692] Updated weights for policy 0, policy_version 1575061 (0.0005) [2023-12-27 02:52:06,550][105620] Updated weights for policy 1, policy_version 1578504 (0.0011) [2023-12-27 02:52:06,584][105692] Updated weights for policy 0, policy_version 1575071 (0.0005) [2023-12-27 02:52:07,114][105692] Updated weights for policy 0, policy_version 1575081 (0.0005) [2023-12-27 02:52:07,171][105692] Updated weights for policy 0, policy_version 1575091 (0.0006) [2023-12-27 02:52:07,228][105692] Updated weights for policy 0, policy_version 1575101 (0.0008) [2023-12-27 02:52:07,241][105620] Updated weights for policy 1, policy_version 1578514 (0.0007) [2023-12-27 02:52:07,293][105692] Updated weights for policy 0, policy_version 1575111 (0.0007) [2023-12-27 02:52:07,296][105620] Updated weights for policy 1, policy_version 1578524 (0.0010) [2023-12-27 02:52:07,354][105620] Updated weights for policy 1, policy_version 1578534 (0.0010) [2023-12-27 02:52:07,827][105692] Updated weights for policy 0, policy_version 1575121 (0.0005) [2023-12-27 02:52:07,883][105692] Updated weights for policy 0, policy_version 1575131 (0.0005) [2023-12-27 02:52:07,910][105620] Updated weights for policy 1, policy_version 1578544 (0.0006) [2023-12-27 02:52:07,948][105692] Updated weights for policy 0, policy_version 1575141 (0.0006) [2023-12-27 02:52:07,972][105620] Updated weights for policy 1, policy_version 1578554 (0.0007) [2023-12-27 02:52:08,024][105620] Updated weights for policy 1, policy_version 1578564 (0.0005) [2023-12-27 02:52:08,620][105692] Updated weights for policy 0, policy_version 1575151 (0.0009) [2023-12-27 02:52:08,689][105692] Updated weights for policy 0, policy_version 1575161 (0.0010) [2023-12-27 02:52:08,734][105620] Updated weights for policy 1, policy_version 1578574 (0.0007) [2023-12-27 02:52:08,749][105692] Updated weights for policy 0, policy_version 1575171 (0.0008) [2023-12-27 02:52:08,791][105620] Updated weights for policy 1, policy_version 1578584 (0.0006) [2023-12-27 02:52:08,845][105620] Updated weights for policy 1, policy_version 1578594 (0.0006) [2023-12-27 02:52:09,379][105692] Updated weights for policy 0, policy_version 1575181 (0.0008) [2023-12-27 02:52:09,447][105692] Updated weights for policy 0, policy_version 1575191 (0.0008) [2023-12-27 02:52:09,451][105620] Updated weights for policy 1, policy_version 1578604 (0.0007) [2023-12-27 02:52:09,504][105692] Updated weights for policy 0, policy_version 1575201 (0.0008) [2023-12-27 02:52:09,518][105620] Updated weights for policy 1, policy_version 1578614 (0.0005) [2023-12-27 02:52:09,589][105620] Updated weights for policy 1, policy_version 1578624 (0.0006) [2023-12-27 02:52:10,253][105620] Updated weights for policy 1, policy_version 1578634 (0.0007) [2023-12-27 02:52:10,283][105692] Updated weights for policy 0, policy_version 1575211 (0.0007) [2023-12-27 02:52:10,314][105620] Updated weights for policy 1, policy_version 1578644 (0.0009) [2023-12-27 02:52:10,342][105692] Updated weights for policy 0, policy_version 1575221 (0.0006) [2023-12-27 02:52:10,368][105620] Updated weights for policy 1, policy_version 1578654 (0.0007) [2023-12-27 02:52:10,402][105692] Updated weights for policy 0, policy_version 1575231 (0.0011) [2023-12-27 02:52:10,420][105620] Updated weights for policy 1, policy_version 1578664 (0.0006) [2023-12-27 02:52:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 807510016. Throughput: 0: 9788.2, 1: 9661.8. Samples: 807523056. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:52:11,063][104569] Avg episode reward: [(0, '8894.366'), (1, '8995.736')] [2023-12-27 02:52:11,120][105620] Updated weights for policy 1, policy_version 1578674 (0.0008) [2023-12-27 02:52:11,162][105692] Updated weights for policy 0, policy_version 1575241 (0.0011) [2023-12-27 02:52:11,186][105620] Updated weights for policy 1, policy_version 1578684 (0.0008) [2023-12-27 02:52:11,222][105692] Updated weights for policy 0, policy_version 1575251 (0.0011) [2023-12-27 02:52:11,250][105620] Updated weights for policy 1, policy_version 1578694 (0.0008) [2023-12-27 02:52:11,280][105692] Updated weights for policy 0, policy_version 1575261 (0.0009) [2023-12-27 02:52:11,344][105692] Updated weights for policy 0, policy_version 1575271 (0.0009) [2023-12-27 02:52:11,958][105620] Updated weights for policy 1, policy_version 1578704 (0.0008) [2023-12-27 02:52:12,013][105620] Updated weights for policy 1, policy_version 1578714 (0.0009) [2023-12-27 02:52:12,077][105620] Updated weights for policy 1, policy_version 1578724 (0.0008) [2023-12-27 02:52:12,178][105692] Updated weights for policy 0, policy_version 1575281 (0.0010) [2023-12-27 02:52:12,273][105692] Updated weights for policy 0, policy_version 1575291 (0.0010) [2023-12-27 02:52:12,335][105692] Updated weights for policy 0, policy_version 1575301 (0.0010) [2023-12-27 02:52:12,786][105620] Updated weights for policy 1, policy_version 1578734 (0.0009) [2023-12-27 02:52:12,840][105620] Updated weights for policy 1, policy_version 1578744 (0.0009) [2023-12-27 02:52:12,901][105620] Updated weights for policy 1, policy_version 1578754 (0.0009) [2023-12-27 02:52:13,110][105692] Updated weights for policy 0, policy_version 1575311 (0.0010) [2023-12-27 02:52:13,164][105692] Updated weights for policy 0, policy_version 1575321 (0.0010) [2023-12-27 02:52:13,214][105692] Updated weights for policy 0, policy_version 1575331 (0.0006) [2023-12-27 02:52:13,583][105620] Updated weights for policy 1, policy_version 1578764 (0.0009) [2023-12-27 02:52:13,642][105620] Updated weights for policy 1, policy_version 1578774 (0.0010) [2023-12-27 02:52:13,696][105620] Updated weights for policy 1, policy_version 1578784 (0.0010) [2023-12-27 02:52:13,950][105692] Updated weights for policy 0, policy_version 1575341 (0.0005) [2023-12-27 02:52:14,002][105692] Updated weights for policy 0, policy_version 1575351 (0.0005) [2023-12-27 02:52:14,055][105692] Updated weights for policy 0, policy_version 1575361 (0.0005) [2023-12-27 02:52:14,382][105620] Updated weights for policy 1, policy_version 1578794 (0.0010) [2023-12-27 02:52:14,441][105620] Updated weights for policy 1, policy_version 1578804 (0.0009) [2023-12-27 02:52:14,487][105620] Updated weights for policy 1, policy_version 1578814 (0.0008) [2023-12-27 02:52:14,540][105620] Updated weights for policy 1, policy_version 1578824 (0.0010) [2023-12-27 02:52:14,681][105692] Updated weights for policy 0, policy_version 1575371 (0.0006) [2023-12-27 02:52:14,726][105692] Updated weights for policy 0, policy_version 1575381 (0.0005) [2023-12-27 02:52:14,779][105692] Updated weights for policy 0, policy_version 1575391 (0.0006) [2023-12-27 02:52:15,360][105620] Updated weights for policy 1, policy_version 1578834 (0.0009) [2023-12-27 02:52:15,423][105620] Updated weights for policy 1, policy_version 1578844 (0.0008) [2023-12-27 02:52:15,472][105620] Updated weights for policy 1, policy_version 1578854 (0.0005) [2023-12-27 02:52:15,535][105692] Updated weights for policy 0, policy_version 1575401 (0.0009) [2023-12-27 02:52:15,587][105692] Updated weights for policy 0, policy_version 1575411 (0.0009) [2023-12-27 02:52:15,638][105692] Updated weights for policy 0, policy_version 1575421 (0.0009) [2023-12-27 02:52:15,696][105692] Updated weights for policy 0, policy_version 1575431 (0.0009) [2023-12-27 02:52:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 807608320. Throughput: 0: 9726.6, 1: 9653.4. Samples: 807579504. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:52:16,063][104569] Avg episode reward: [(0, '8622.001'), (1, '8907.011')] [2023-12-27 02:52:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001575432_403365888.pth... [2023-12-27 02:52:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001578856_404242432.pth... [2023-12-27 02:52:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001577736_403955712.pth [2023-12-27 02:52:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001574280_403070976.pth [2023-12-27 02:52:16,159][105620] Updated weights for policy 1, policy_version 1578864 (0.0007) [2023-12-27 02:52:16,217][105620] Updated weights for policy 1, policy_version 1578874 (0.0009) [2023-12-27 02:52:16,270][105620] Updated weights for policy 1, policy_version 1578884 (0.0009) [2023-12-27 02:52:16,470][105692] Updated weights for policy 0, policy_version 1575441 (0.0009) [2023-12-27 02:52:16,521][105692] Updated weights for policy 0, policy_version 1575451 (0.0009) [2023-12-27 02:52:16,569][105692] Updated weights for policy 0, policy_version 1575461 (0.0008) [2023-12-27 02:52:17,026][105620] Updated weights for policy 1, policy_version 1578894 (0.0010) [2023-12-27 02:52:17,088][105620] Updated weights for policy 1, policy_version 1578904 (0.0009) [2023-12-27 02:52:17,145][105620] Updated weights for policy 1, policy_version 1578914 (0.0008) [2023-12-27 02:52:17,337][105692] Updated weights for policy 0, policy_version 1575471 (0.0009) [2023-12-27 02:52:17,399][105692] Updated weights for policy 0, policy_version 1575481 (0.0009) [2023-12-27 02:52:17,457][105692] Updated weights for policy 0, policy_version 1575491 (0.0009) [2023-12-27 02:52:17,876][105620] Updated weights for policy 1, policy_version 1578924 (0.0008) [2023-12-27 02:52:17,937][105620] Updated weights for policy 1, policy_version 1578934 (0.0008) [2023-12-27 02:52:17,998][105620] Updated weights for policy 1, policy_version 1578944 (0.0009) [2023-12-27 02:52:18,203][105692] Updated weights for policy 0, policy_version 1575501 (0.0009) [2023-12-27 02:52:18,261][105692] Updated weights for policy 0, policy_version 1575511 (0.0009) [2023-12-27 02:52:18,323][105692] Updated weights for policy 0, policy_version 1575521 (0.0009) [2023-12-27 02:52:18,686][105620] Updated weights for policy 1, policy_version 1578954 (0.0007) [2023-12-27 02:52:18,741][105620] Updated weights for policy 1, policy_version 1578964 (0.0005) [2023-12-27 02:52:18,808][105620] Updated weights for policy 1, policy_version 1578974 (0.0008) [2023-12-27 02:52:18,872][105620] Updated weights for policy 1, policy_version 1578984 (0.0009) [2023-12-27 02:52:19,072][105692] Updated weights for policy 0, policy_version 1575531 (0.0008) [2023-12-27 02:52:19,125][105692] Updated weights for policy 0, policy_version 1575541 (0.0009) [2023-12-27 02:52:19,173][105692] Updated weights for policy 0, policy_version 1575551 (0.0009) [2023-12-27 02:52:19,598][105620] Updated weights for policy 1, policy_version 1578994 (0.0009) [2023-12-27 02:52:19,648][105620] Updated weights for policy 1, policy_version 1579004 (0.0008) [2023-12-27 02:52:19,695][105620] Updated weights for policy 1, policy_version 1579014 (0.0006) [2023-12-27 02:52:20,031][105692] Updated weights for policy 0, policy_version 1575561 (0.0009) [2023-12-27 02:52:20,097][105692] Updated weights for policy 0, policy_version 1575571 (0.0009) [2023-12-27 02:52:20,159][105692] Updated weights for policy 0, policy_version 1575581 (0.0009) [2023-12-27 02:52:20,223][105692] Updated weights for policy 0, policy_version 1575591 (0.0010) [2023-12-27 02:52:20,388][105620] Updated weights for policy 1, policy_version 1579024 (0.0008) [2023-12-27 02:52:20,453][105620] Updated weights for policy 1, policy_version 1579034 (0.0008) [2023-12-27 02:52:20,513][105620] Updated weights for policy 1, policy_version 1579044 (0.0008) [2023-12-27 02:52:20,522][105586] KL-divergence is very high: 120.2383 [2023-12-27 02:52:20,535][105586] KL-divergence is very high: 137.5487 [2023-12-27 02:52:21,010][105692] Updated weights for policy 0, policy_version 1575601 (0.0009) [2023-12-27 02:52:21,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 807698432. Throughput: 0: 9690.2, 1: 9669.9. Samples: 807693668. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:52:21,062][104569] Avg episode reward: [(0, '8074.328'), (1, '8640.741')] [2023-12-27 02:52:21,068][105692] Updated weights for policy 0, policy_version 1575611 (0.0008) [2023-12-27 02:52:21,130][105692] Updated weights for policy 0, policy_version 1575621 (0.0006) [2023-12-27 02:52:21,270][105620] Updated weights for policy 1, policy_version 1579054 (0.0008) [2023-12-27 02:52:21,332][105620] Updated weights for policy 1, policy_version 1579064 (0.0008) [2023-12-27 02:52:21,406][105620] Updated weights for policy 1, policy_version 1579074 (0.0008) [2023-12-27 02:52:21,901][105692] Updated weights for policy 0, policy_version 1575631 (0.0007) [2023-12-27 02:52:21,960][105692] Updated weights for policy 0, policy_version 1575641 (0.0008) [2023-12-27 02:52:22,012][105692] Updated weights for policy 0, policy_version 1575651 (0.0008) [2023-12-27 02:52:22,162][105620] Updated weights for policy 1, policy_version 1579084 (0.0009) [2023-12-27 02:52:22,224][105620] Updated weights for policy 1, policy_version 1579094 (0.0009) [2023-12-27 02:52:22,289][105620] Updated weights for policy 1, policy_version 1579104 (0.0010) [2023-12-27 02:52:22,801][105692] Updated weights for policy 0, policy_version 1575661 (0.0009) [2023-12-27 02:52:22,868][105692] Updated weights for policy 0, policy_version 1575671 (0.0009) [2023-12-27 02:52:22,926][105692] Updated weights for policy 0, policy_version 1575681 (0.0010) [2023-12-27 02:52:22,984][105620] Updated weights for policy 1, policy_version 1579114 (0.0009) [2023-12-27 02:52:23,042][105620] Updated weights for policy 1, policy_version 1579124 (0.0006) [2023-12-27 02:52:23,108][105620] Updated weights for policy 1, policy_version 1579134 (0.0007) [2023-12-27 02:52:23,160][105620] Updated weights for policy 1, policy_version 1579144 (0.0005) [2023-12-27 02:52:23,724][105692] Updated weights for policy 0, policy_version 1575691 (0.0010) [2023-12-27 02:52:23,775][105692] Updated weights for policy 0, policy_version 1575701 (0.0009) [2023-12-27 02:52:23,825][105692] Updated weights for policy 0, policy_version 1575711 (0.0007) [2023-12-27 02:52:23,851][105620] Updated weights for policy 1, policy_version 1579154 (0.0008) [2023-12-27 02:52:23,907][105620] Updated weights for policy 1, policy_version 1579164 (0.0008) [2023-12-27 02:52:23,965][105620] Updated weights for policy 1, policy_version 1579174 (0.0008) [2023-12-27 02:52:24,599][105620] Updated weights for policy 1, policy_version 1579184 (0.0006) [2023-12-27 02:52:24,659][105620] Updated weights for policy 1, policy_version 1579194 (0.0005) [2023-12-27 02:52:24,663][105692] Updated weights for policy 0, policy_version 1575721 (0.0006) [2023-12-27 02:52:24,715][105620] Updated weights for policy 1, policy_version 1579204 (0.0010) [2023-12-27 02:52:24,722][105692] Updated weights for policy 0, policy_version 1575731 (0.0009) [2023-12-27 02:52:24,778][105692] Updated weights for policy 0, policy_version 1575741 (0.0008) [2023-12-27 02:52:24,839][105692] Updated weights for policy 0, policy_version 1575751 (0.0009) [2023-12-27 02:52:25,256][105620] Updated weights for policy 1, policy_version 1579214 (0.0009) [2023-12-27 02:52:25,315][105620] Updated weights for policy 1, policy_version 1579224 (0.0005) [2023-12-27 02:52:25,371][105620] Updated weights for policy 1, policy_version 1579234 (0.0005) [2023-12-27 02:52:25,707][105692] Updated weights for policy 0, policy_version 1575761 (0.0008) [2023-12-27 02:52:25,751][105692] Updated weights for policy 0, policy_version 1575771 (0.0008) [2023-12-27 02:52:25,795][105692] Updated weights for policy 0, policy_version 1575781 (0.0008) [2023-12-27 02:52:26,035][105620] Updated weights for policy 1, policy_version 1579244 (0.0007) [2023-12-27 02:52:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 807796736. Throughput: 0: 9506.6, 1: 9790.3. Samples: 807806744. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:52:26,062][104569] Avg episode reward: [(0, '8068.496'), (1, '8374.263')] [2023-12-27 02:52:26,086][105620] Updated weights for policy 1, policy_version 1579254 (0.0010) [2023-12-27 02:52:26,148][105620] Updated weights for policy 1, policy_version 1579264 (0.0010) [2023-12-27 02:52:26,606][105692] Updated weights for policy 0, policy_version 1575791 (0.0006) [2023-12-27 02:52:26,666][105692] Updated weights for policy 0, policy_version 1575801 (0.0005) [2023-12-27 02:52:26,723][105692] Updated weights for policy 0, policy_version 1575811 (0.0007) [2023-12-27 02:52:26,810][105620] Updated weights for policy 1, policy_version 1579274 (0.0010) [2023-12-27 02:52:26,871][105620] Updated weights for policy 1, policy_version 1579284 (0.0006) [2023-12-27 02:52:26,927][105620] Updated weights for policy 1, policy_version 1579294 (0.0005) [2023-12-27 02:52:26,982][105620] Updated weights for policy 1, policy_version 1579304 (0.0005) [2023-12-27 02:52:27,476][105692] Updated weights for policy 0, policy_version 1575821 (0.0008) [2023-12-27 02:52:27,523][105692] Updated weights for policy 0, policy_version 1575831 (0.0008) [2023-12-27 02:52:27,576][105692] Updated weights for policy 0, policy_version 1575841 (0.0007) [2023-12-27 02:52:27,602][105620] Updated weights for policy 1, policy_version 1579314 (0.0010) [2023-12-27 02:52:27,663][105620] Updated weights for policy 1, policy_version 1579324 (0.0010) [2023-12-27 02:52:27,713][105620] Updated weights for policy 1, policy_version 1579334 (0.0010) [2023-12-27 02:52:28,352][105692] Updated weights for policy 0, policy_version 1575851 (0.0006) [2023-12-27 02:52:28,410][105692] Updated weights for policy 0, policy_version 1575861 (0.0008) [2023-12-27 02:52:28,468][105692] Updated weights for policy 0, policy_version 1575871 (0.0007) [2023-12-27 02:52:28,469][105620] Updated weights for policy 1, policy_version 1579344 (0.0011) [2023-12-27 02:52:28,528][105620] Updated weights for policy 1, policy_version 1579354 (0.0010) [2023-12-27 02:52:28,586][105620] Updated weights for policy 1, policy_version 1579364 (0.0010) [2023-12-27 02:52:29,245][105692] Updated weights for policy 0, policy_version 1575881 (0.0007) [2023-12-27 02:52:29,290][105692] Updated weights for policy 0, policy_version 1575891 (0.0008) [2023-12-27 02:52:29,343][105620] Updated weights for policy 1, policy_version 1579374 (0.0010) [2023-12-27 02:52:29,353][105692] Updated weights for policy 0, policy_version 1575901 (0.0008) [2023-12-27 02:52:29,404][105620] Updated weights for policy 1, policy_version 1579384 (0.0008) [2023-12-27 02:52:29,416][105692] Updated weights for policy 0, policy_version 1575911 (0.0008) [2023-12-27 02:52:29,474][105620] Updated weights for policy 1, policy_version 1579394 (0.0006) [2023-12-27 02:52:30,213][105620] Updated weights for policy 1, policy_version 1579404 (0.0006) [2023-12-27 02:52:30,220][105692] Updated weights for policy 0, policy_version 1575921 (0.0008) [2023-12-27 02:52:30,265][105620] Updated weights for policy 1, policy_version 1579414 (0.0006) [2023-12-27 02:52:30,272][105692] Updated weights for policy 0, policy_version 1575932 (0.0008) [2023-12-27 02:52:30,319][105620] Updated weights for policy 1, policy_version 1579424 (0.0005) [2023-12-27 02:52:30,321][105692] Updated weights for policy 0, policy_version 1575942 (0.0008) [2023-12-27 02:52:30,870][105620] Updated weights for policy 1, policy_version 1579434 (0.0005) [2023-12-27 02:52:30,917][105620] Updated weights for policy 1, policy_version 1579444 (0.0005) [2023-12-27 02:52:30,965][105620] Updated weights for policy 1, policy_version 1579454 (0.0006) [2023-12-27 02:52:31,011][105620] Updated weights for policy 1, policy_version 1579464 (0.0009) [2023-12-27 02:52:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 807895040. Throughput: 0: 9527.2, 1: 9798.7. Samples: 807863844. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:52:31,062][104569] Avg episode reward: [(0, '8440.075'), (1, '8817.301')] [2023-12-27 02:52:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001575944_403496960.pth... [2023-12-27 02:52:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001579464_404398080.pth... [2023-12-27 02:52:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001578280_404094976.pth [2023-12-27 02:52:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001574856_403218432.pth [2023-12-27 02:52:31,203][105692] Updated weights for policy 0, policy_version 1575952 (0.0008) [2023-12-27 02:52:31,268][105692] Updated weights for policy 0, policy_version 1575962 (0.0007) [2023-12-27 02:52:31,328][105692] Updated weights for policy 0, policy_version 1575972 (0.0008) [2023-12-27 02:52:31,744][105620] Updated weights for policy 1, policy_version 1579474 (0.0011) [2023-12-27 02:52:31,802][105620] Updated weights for policy 1, policy_version 1579484 (0.0010) [2023-12-27 02:52:31,861][105620] Updated weights for policy 1, policy_version 1579494 (0.0011) [2023-12-27 02:52:32,041][105692] Updated weights for policy 0, policy_version 1575982 (0.0010) [2023-12-27 02:52:32,103][105692] Updated weights for policy 0, policy_version 1575992 (0.0010) [2023-12-27 02:52:32,165][105692] Updated weights for policy 0, policy_version 1576002 (0.0010) [2023-12-27 02:52:32,482][105620] Updated weights for policy 1, policy_version 1579504 (0.0006) [2023-12-27 02:52:32,536][105620] Updated weights for policy 1, policy_version 1579514 (0.0006) [2023-12-27 02:52:32,598][105620] Updated weights for policy 1, policy_version 1579524 (0.0007) [2023-12-27 02:52:32,966][105692] Updated weights for policy 0, policy_version 1576012 (0.0009) [2023-12-27 02:52:33,020][105692] Updated weights for policy 0, policy_version 1576024 (0.0010) [2023-12-27 02:52:33,071][105692] Updated weights for policy 0, policy_version 1576035 (0.0009) [2023-12-27 02:52:33,145][105620] Updated weights for policy 1, policy_version 1579534 (0.0007) [2023-12-27 02:52:33,200][105620] Updated weights for policy 1, policy_version 1579544 (0.0005) [2023-12-27 02:52:33,259][105620] Updated weights for policy 1, policy_version 1579554 (0.0005) [2023-12-27 02:52:33,833][105620] Updated weights for policy 1, policy_version 1579564 (0.0005) [2023-12-27 02:52:33,881][105620] Updated weights for policy 1, policy_version 1579574 (0.0005) [2023-12-27 02:52:33,934][105620] Updated weights for policy 1, policy_version 1579584 (0.0005) [2023-12-27 02:52:33,973][105692] Updated weights for policy 0, policy_version 1576046 (0.0009) [2023-12-27 02:52:34,024][105692] Updated weights for policy 0, policy_version 1576056 (0.0009) [2023-12-27 02:52:34,083][105692] Updated weights for policy 0, policy_version 1576066 (0.0008) [2023-12-27 02:52:34,611][105620] Updated weights for policy 1, policy_version 1579594 (0.0006) [2023-12-27 02:52:34,676][105620] Updated weights for policy 1, policy_version 1579604 (0.0011) [2023-12-27 02:52:34,739][105620] Updated weights for policy 1, policy_version 1579614 (0.0011) [2023-12-27 02:52:34,794][105620] Updated weights for policy 1, policy_version 1579624 (0.0011) [2023-12-27 02:52:34,870][105692] Updated weights for policy 0, policy_version 1576076 (0.0009) [2023-12-27 02:52:34,931][105692] Updated weights for policy 0, policy_version 1576087 (0.0009) [2023-12-27 02:52:34,991][105692] Updated weights for policy 0, policy_version 1576097 (0.0009) [2023-12-27 02:52:35,480][105620] Updated weights for policy 1, policy_version 1579634 (0.0009) [2023-12-27 02:52:35,535][105620] Updated weights for policy 1, policy_version 1579644 (0.0009) [2023-12-27 02:52:35,597][105620] Updated weights for policy 1, policy_version 1579654 (0.0009) [2023-12-27 02:52:35,804][105692] Updated weights for policy 0, policy_version 1576107 (0.0009) [2023-12-27 02:52:35,855][105692] Updated weights for policy 0, policy_version 1576117 (0.0009) [2023-12-27 02:52:35,902][105692] Updated weights for policy 0, policy_version 1576127 (0.0009) [2023-12-27 02:52:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 807993344. Throughput: 0: 9441.7, 1: 9888.9. Samples: 807980764. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:52:36,063][104569] Avg episode reward: [(0, '8354.721'), (1, '8903.054')] [2023-12-27 02:52:36,339][105620] Updated weights for policy 1, policy_version 1579664 (0.0009) [2023-12-27 02:52:36,402][105620] Updated weights for policy 1, policy_version 1579674 (0.0009) [2023-12-27 02:52:36,462][105620] Updated weights for policy 1, policy_version 1579684 (0.0008) [2023-12-27 02:52:36,683][105692] Updated weights for policy 0, policy_version 1576137 (0.0008) [2023-12-27 02:52:36,744][105692] Updated weights for policy 0, policy_version 1576147 (0.0006) [2023-12-27 02:52:36,807][105692] Updated weights for policy 0, policy_version 1576157 (0.0009) [2023-12-27 02:52:36,874][105692] Updated weights for policy 0, policy_version 1576167 (0.0006) [2023-12-27 02:52:37,246][105620] Updated weights for policy 1, policy_version 1579694 (0.0008) [2023-12-27 02:52:37,293][105620] Updated weights for policy 1, policy_version 1579704 (0.0008) [2023-12-27 02:52:37,340][105620] Updated weights for policy 1, policy_version 1579714 (0.0008) [2023-12-27 02:52:37,486][105692] Updated weights for policy 0, policy_version 1576178 (0.0010) [2023-12-27 02:52:37,534][105692] Updated weights for policy 0, policy_version 1576188 (0.0009) [2023-12-27 02:52:37,583][105692] Updated weights for policy 0, policy_version 1576198 (0.0009) [2023-12-27 02:52:38,007][105620] Updated weights for policy 1, policy_version 1579724 (0.0009) [2023-12-27 02:52:38,073][105620] Updated weights for policy 1, policy_version 1579734 (0.0009) [2023-12-27 02:52:38,132][105620] Updated weights for policy 1, policy_version 1579744 (0.0009) [2023-12-27 02:52:38,387][105692] Updated weights for policy 0, policy_version 1576208 (0.0008) [2023-12-27 02:52:38,452][105692] Updated weights for policy 0, policy_version 1576218 (0.0009) [2023-12-27 02:52:38,514][105692] Updated weights for policy 0, policy_version 1576228 (0.0009) [2023-12-27 02:52:38,795][105620] Updated weights for policy 1, policy_version 1579754 (0.0010) [2023-12-27 02:52:38,858][105620] Updated weights for policy 1, policy_version 1579764 (0.0009) [2023-12-27 02:52:38,926][105620] Updated weights for policy 1, policy_version 1579774 (0.0008) [2023-12-27 02:52:38,990][105620] Updated weights for policy 1, policy_version 1579784 (0.0006) [2023-12-27 02:52:39,236][105692] Updated weights for policy 0, policy_version 1576238 (0.0009) [2023-12-27 02:52:39,297][105692] Updated weights for policy 0, policy_version 1576248 (0.0009) [2023-12-27 02:52:39,348][105692] Updated weights for policy 0, policy_version 1576258 (0.0008) [2023-12-27 02:52:39,642][105620] Updated weights for policy 1, policy_version 1579794 (0.0008) [2023-12-27 02:52:39,701][105620] Updated weights for policy 1, policy_version 1579804 (0.0010) [2023-12-27 02:52:39,752][105620] Updated weights for policy 1, policy_version 1579814 (0.0009) [2023-12-27 02:52:40,103][105692] Updated weights for policy 0, policy_version 1576268 (0.0009) [2023-12-27 02:52:40,156][105692] Updated weights for policy 0, policy_version 1576278 (0.0006) [2023-12-27 02:52:40,212][105692] Updated weights for policy 0, policy_version 1576288 (0.0006) [2023-12-27 02:52:40,580][105620] Updated weights for policy 1, policy_version 1579824 (0.0010) [2023-12-27 02:52:40,639][105620] Updated weights for policy 1, policy_version 1579834 (0.0011) [2023-12-27 02:52:40,696][105620] Updated weights for policy 1, policy_version 1579844 (0.0009) [2023-12-27 02:52:40,907][105692] Updated weights for policy 0, policy_version 1576298 (0.0007) [2023-12-27 02:52:40,959][105692] Updated weights for policy 0, policy_version 1576308 (0.0010) [2023-12-27 02:52:41,029][105692] Updated weights for policy 0, policy_version 1576318 (0.0009) [2023-12-27 02:52:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 808083456. Throughput: 0: 9454.5, 1: 9925.1. Samples: 808095844. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:52:41,062][104569] Avg episode reward: [(0, '8534.769'), (1, '8903.528')] [2023-12-27 02:52:41,090][105692] Updated weights for policy 0, policy_version 1576328 (0.0008) [2023-12-27 02:52:41,455][105620] Updated weights for policy 1, policy_version 1579854 (0.0009) [2023-12-27 02:52:41,518][105620] Updated weights for policy 1, policy_version 1579864 (0.0008) [2023-12-27 02:52:41,567][105620] Updated weights for policy 1, policy_version 1579874 (0.0007) [2023-12-27 02:52:41,905][105692] Updated weights for policy 0, policy_version 1576338 (0.0008) [2023-12-27 02:52:41,963][105692] Updated weights for policy 0, policy_version 1576348 (0.0010) [2023-12-27 02:52:42,022][105692] Updated weights for policy 0, policy_version 1576358 (0.0008) [2023-12-27 02:52:42,290][105620] Updated weights for policy 1, policy_version 1579884 (0.0007) [2023-12-27 02:52:42,348][105620] Updated weights for policy 1, policy_version 1579894 (0.0008) [2023-12-27 02:52:42,411][105620] Updated weights for policy 1, policy_version 1579904 (0.0008) [2023-12-27 02:52:42,702][105692] Updated weights for policy 0, policy_version 1576368 (0.0008) [2023-12-27 02:52:42,755][105692] Updated weights for policy 0, policy_version 1576378 (0.0005) [2023-12-27 02:52:42,821][105692] Updated weights for policy 0, policy_version 1576388 (0.0006) [2023-12-27 02:52:43,125][105620] Updated weights for policy 1, policy_version 1579914 (0.0010) [2023-12-27 02:52:43,180][105620] Updated weights for policy 1, policy_version 1579924 (0.0009) [2023-12-27 02:52:43,235][105620] Updated weights for policy 1, policy_version 1579934 (0.0009) [2023-12-27 02:52:43,285][105620] Updated weights for policy 1, policy_version 1579944 (0.0008) [2023-12-27 02:52:43,486][105692] Updated weights for policy 0, policy_version 1576398 (0.0005) [2023-12-27 02:52:43,537][105692] Updated weights for policy 0, policy_version 1576408 (0.0006) [2023-12-27 02:52:43,598][105692] Updated weights for policy 0, policy_version 1576418 (0.0008) [2023-12-27 02:52:44,105][105620] Updated weights for policy 1, policy_version 1579954 (0.0008) [2023-12-27 02:52:44,168][105620] Updated weights for policy 1, policy_version 1579964 (0.0008) [2023-12-27 02:52:44,224][105620] Updated weights for policy 1, policy_version 1579974 (0.0005) [2023-12-27 02:52:44,322][105692] Updated weights for policy 0, policy_version 1576428 (0.0010) [2023-12-27 02:52:44,382][105692] Updated weights for policy 0, policy_version 1576438 (0.0010) [2023-12-27 02:52:44,433][105692] Updated weights for policy 0, policy_version 1576448 (0.0010) [2023-12-27 02:52:44,934][105620] Updated weights for policy 1, policy_version 1579984 (0.0011) [2023-12-27 02:52:45,004][105620] Updated weights for policy 1, policy_version 1579994 (0.0011) [2023-12-27 02:52:45,007][105692] Updated weights for policy 0, policy_version 1576458 (0.0006) [2023-12-27 02:52:45,067][105692] Updated weights for policy 0, policy_version 1576468 (0.0011) [2023-12-27 02:52:45,067][105620] Updated weights for policy 1, policy_version 1580004 (0.0010) [2023-12-27 02:52:45,128][105692] Updated weights for policy 0, policy_version 1576478 (0.0009) [2023-12-27 02:52:45,173][105692] Updated weights for policy 0, policy_version 1576488 (0.0011) [2023-12-27 02:52:45,710][105620] Updated weights for policy 1, policy_version 1580014 (0.0010) [2023-12-27 02:52:45,764][105620] Updated weights for policy 1, policy_version 1580024 (0.0010) [2023-12-27 02:52:45,816][105620] Updated weights for policy 1, policy_version 1580034 (0.0010) [2023-12-27 02:52:45,943][105692] Updated weights for policy 0, policy_version 1576498 (0.0008) [2023-12-27 02:52:45,992][105692] Updated weights for policy 0, policy_version 1576508 (0.0008) [2023-12-27 02:52:46,043][105692] Updated weights for policy 0, policy_version 1576518 (0.0008) [2023-12-27 02:52:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 808189952. Throughput: 0: 9447.9, 1: 9927.9. Samples: 808153264. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:52:46,063][104569] Avg episode reward: [(0, '8630.628'), (1, '9092.376')] [2023-12-27 02:52:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001576520_403644416.pth... [2023-12-27 02:52:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001580040_404545536.pth... [2023-12-27 02:52:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001575432_403365888.pth [2023-12-27 02:52:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001578856_404242432.pth [2023-12-27 02:52:46,517][105620] Updated weights for policy 1, policy_version 1580044 (0.0008) [2023-12-27 02:52:46,568][105620] Updated weights for policy 1, policy_version 1580054 (0.0006) [2023-12-27 02:52:46,617][105620] Updated weights for policy 1, policy_version 1580064 (0.0010) [2023-12-27 02:52:46,685][105692] Updated weights for policy 0, policy_version 1576528 (0.0009) [2023-12-27 02:52:46,745][105692] Updated weights for policy 0, policy_version 1576538 (0.0009) [2023-12-27 02:52:46,807][105692] Updated weights for policy 0, policy_version 1576548 (0.0009) [2023-12-27 02:52:47,322][105620] Updated weights for policy 1, policy_version 1580074 (0.0010) [2023-12-27 02:52:47,396][105620] Updated weights for policy 1, policy_version 1580084 (0.0010) [2023-12-27 02:52:47,466][105620] Updated weights for policy 1, policy_version 1580094 (0.0010) [2023-12-27 02:52:47,520][105620] Updated weights for policy 1, policy_version 1580104 (0.0010) [2023-12-27 02:52:47,573][105692] Updated weights for policy 0, policy_version 1576558 (0.0010) [2023-12-27 02:52:47,625][105692] Updated weights for policy 0, policy_version 1576568 (0.0006) [2023-12-27 02:52:47,694][105692] Updated weights for policy 0, policy_version 1576578 (0.0005) [2023-12-27 02:52:47,695][105585] KL-divergence is very high: 100.9963 [2023-12-27 02:52:48,217][105692] Updated weights for policy 0, policy_version 1576588 (0.0007) [2023-12-27 02:52:48,232][105620] Updated weights for policy 1, policy_version 1580114 (0.0010) [2023-12-27 02:52:48,259][105692] Updated weights for policy 0, policy_version 1576598 (0.0005) [2023-12-27 02:52:48,280][105620] Updated weights for policy 1, policy_version 1580124 (0.0010) [2023-12-27 02:52:48,306][105692] Updated weights for policy 0, policy_version 1576608 (0.0005) [2023-12-27 02:52:48,328][105620] Updated weights for policy 1, policy_version 1580134 (0.0010) [2023-12-27 02:52:48,904][105692] Updated weights for policy 0, policy_version 1576618 (0.0007) [2023-12-27 02:52:48,961][105692] Updated weights for policy 0, policy_version 1576628 (0.0006) [2023-12-27 02:52:49,020][105692] Updated weights for policy 0, policy_version 1576638 (0.0005) [2023-12-27 02:52:49,072][105692] Updated weights for policy 0, policy_version 1576648 (0.0005) [2023-12-27 02:52:49,094][105620] Updated weights for policy 1, policy_version 1580144 (0.0010) [2023-12-27 02:52:49,148][105620] Updated weights for policy 1, policy_version 1580154 (0.0010) [2023-12-27 02:52:49,213][105620] Updated weights for policy 1, policy_version 1580164 (0.0010) [2023-12-27 02:52:49,694][105692] Updated weights for policy 0, policy_version 1576658 (0.0007) [2023-12-27 02:52:49,758][105692] Updated weights for policy 0, policy_version 1576668 (0.0008) [2023-12-27 02:52:49,829][105692] Updated weights for policy 0, policy_version 1576678 (0.0008) [2023-12-27 02:52:49,965][105620] Updated weights for policy 1, policy_version 1580174 (0.0011) [2023-12-27 02:52:49,980][105586] KL-divergence is very high: 106.1028 [2023-12-27 02:52:50,024][105620] Updated weights for policy 1, policy_version 1580184 (0.0011) [2023-12-27 02:52:50,033][105586] KL-divergence is very high: 167.2697 [2023-12-27 02:52:50,089][105586] KL-divergence is very high: 146.2239 [2023-12-27 02:52:50,097][105620] Updated weights for policy 1, policy_version 1580194 (0.0009) [2023-12-27 02:52:50,431][105692] Updated weights for policy 0, policy_version 1576688 (0.0007) [2023-12-27 02:52:50,489][105692] Updated weights for policy 0, policy_version 1576698 (0.0005) [2023-12-27 02:52:50,544][105692] Updated weights for policy 0, policy_version 1576708 (0.0008) [2023-12-27 02:52:50,780][105620] Updated weights for policy 1, policy_version 1580204 (0.0008) [2023-12-27 02:52:50,839][105620] Updated weights for policy 1, policy_version 1580214 (0.0011) [2023-12-27 02:52:50,902][105620] Updated weights for policy 1, policy_version 1580224 (0.0011) [2023-12-27 02:52:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 808288256. Throughput: 0: 9494.9, 1: 9962.1. Samples: 808274800. Policy #0 lag: (min: 2.0, avg: 10.6, max: 34.0) [2023-12-27 02:52:51,062][104569] Avg episode reward: [(0, '8533.996'), (1, '8549.487')] [2023-12-27 02:52:51,327][105692] Updated weights for policy 0, policy_version 1576718 (0.0008) [2023-12-27 02:52:51,389][105692] Updated weights for policy 0, policy_version 1576728 (0.0008) [2023-12-27 02:52:51,449][105692] Updated weights for policy 0, policy_version 1576738 (0.0008) [2023-12-27 02:52:51,678][105620] Updated weights for policy 1, policy_version 1580234 (0.0011) [2023-12-27 02:52:51,741][105620] Updated weights for policy 1, policy_version 1580244 (0.0011) [2023-12-27 02:52:51,790][105620] Updated weights for policy 1, policy_version 1580254 (0.0010) [2023-12-27 02:52:51,839][105620] Updated weights for policy 1, policy_version 1580264 (0.0010) [2023-12-27 02:52:52,234][105692] Updated weights for policy 0, policy_version 1576748 (0.0008) [2023-12-27 02:52:52,298][105692] Updated weights for policy 0, policy_version 1576758 (0.0009) [2023-12-27 02:52:52,358][105692] Updated weights for policy 0, policy_version 1576768 (0.0009) [2023-12-27 02:52:52,591][105620] Updated weights for policy 1, policy_version 1580274 (0.0010) [2023-12-27 02:52:52,643][105620] Updated weights for policy 1, policy_version 1580284 (0.0010) [2023-12-27 02:52:52,688][105620] Updated weights for policy 1, policy_version 1580294 (0.0010) [2023-12-27 02:52:53,134][105692] Updated weights for policy 0, policy_version 1576778 (0.0008) [2023-12-27 02:52:53,194][105692] Updated weights for policy 0, policy_version 1576788 (0.0008) [2023-12-27 02:52:53,220][105585] KL-divergence is very high: 120.6940 [2023-12-27 02:52:53,246][105692] Updated weights for policy 0, policy_version 1576798 (0.0008) [2023-12-27 02:52:53,262][105585] KL-divergence is very high: 135.0666 [2023-12-27 02:52:53,297][105692] Updated weights for policy 0, policy_version 1576808 (0.0008) [2023-12-27 02:52:53,463][105620] Updated weights for policy 1, policy_version 1580304 (0.0010) [2023-12-27 02:52:53,525][105620] Updated weights for policy 1, policy_version 1580314 (0.0010) [2023-12-27 02:52:53,589][105620] Updated weights for policy 1, policy_version 1580324 (0.0010) [2023-12-27 02:52:54,074][105692] Updated weights for policy 0, policy_version 1576818 (0.0008) [2023-12-27 02:52:54,127][105692] Updated weights for policy 0, policy_version 1576828 (0.0009) [2023-12-27 02:52:54,179][105692] Updated weights for policy 0, policy_version 1576838 (0.0008) [2023-12-27 02:52:54,316][105620] Updated weights for policy 1, policy_version 1580334 (0.0010) [2023-12-27 02:52:54,378][105620] Updated weights for policy 1, policy_version 1580344 (0.0011) [2023-12-27 02:52:54,432][105620] Updated weights for policy 1, policy_version 1580354 (0.0010) [2023-12-27 02:52:54,919][105692] Updated weights for policy 0, policy_version 1576848 (0.0009) [2023-12-27 02:52:54,978][105692] Updated weights for policy 0, policy_version 1576858 (0.0009) [2023-12-27 02:52:55,031][105620] Updated weights for policy 1, policy_version 1580364 (0.0006) [2023-12-27 02:52:55,035][105692] Updated weights for policy 0, policy_version 1576868 (0.0010) [2023-12-27 02:52:55,084][105620] Updated weights for policy 1, policy_version 1580374 (0.0008) [2023-12-27 02:52:55,141][105620] Updated weights for policy 1, policy_version 1580384 (0.0008) [2023-12-27 02:52:55,794][105620] Updated weights for policy 1, policy_version 1580394 (0.0009) [2023-12-27 02:52:55,844][105692] Updated weights for policy 0, policy_version 1576878 (0.0008) [2023-12-27 02:52:55,847][105620] Updated weights for policy 1, policy_version 1580404 (0.0009) [2023-12-27 02:52:55,900][105692] Updated weights for policy 0, policy_version 1576888 (0.0008) [2023-12-27 02:52:55,906][105620] Updated weights for policy 1, policy_version 1580414 (0.0007) [2023-12-27 02:52:55,955][105692] Updated weights for policy 0, policy_version 1576898 (0.0007) [2023-12-27 02:52:55,958][105620] Updated weights for policy 1, policy_version 1580424 (0.0008) [2023-12-27 02:52:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 808386560. Throughput: 0: 9348.8, 1: 9901.8. Samples: 808389332. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:52:56,062][104569] Avg episode reward: [(0, '8532.390'), (1, '8367.317')] [2023-12-27 02:52:56,574][105620] Updated weights for policy 1, policy_version 1580434 (0.0009) [2023-12-27 02:52:56,624][105620] Updated weights for policy 1, policy_version 1580444 (0.0009) [2023-12-27 02:52:56,653][105586] KL-divergence is very high: 102.2214 [2023-12-27 02:52:56,672][105620] Updated weights for policy 1, policy_version 1580454 (0.0006) [2023-12-27 02:52:56,754][105692] Updated weights for policy 0, policy_version 1576908 (0.0009) [2023-12-27 02:52:56,818][105692] Updated weights for policy 0, policy_version 1576918 (0.0010) [2023-12-27 02:52:56,879][105692] Updated weights for policy 0, policy_version 1576928 (0.0010) [2023-12-27 02:52:57,237][105620] Updated weights for policy 1, policy_version 1580464 (0.0005) [2023-12-27 02:52:57,289][105620] Updated weights for policy 1, policy_version 1580474 (0.0005) [2023-12-27 02:52:57,349][105620] Updated weights for policy 1, policy_version 1580484 (0.0008) [2023-12-27 02:52:57,719][105692] Updated weights for policy 0, policy_version 1576938 (0.0010) [2023-12-27 02:52:57,783][105692] Updated weights for policy 0, policy_version 1576948 (0.0009) [2023-12-27 02:52:57,843][105692] Updated weights for policy 0, policy_version 1576958 (0.0010) [2023-12-27 02:52:57,899][105692] Updated weights for policy 0, policy_version 1576968 (0.0009) [2023-12-27 02:52:57,991][105620] Updated weights for policy 1, policy_version 1580494 (0.0009) [2023-12-27 02:52:58,049][105620] Updated weights for policy 1, policy_version 1580504 (0.0009) [2023-12-27 02:52:58,099][105620] Updated weights for policy 1, policy_version 1580514 (0.0009) [2023-12-27 02:52:58,741][105692] Updated weights for policy 0, policy_version 1576978 (0.0009) [2023-12-27 02:52:58,812][105692] Updated weights for policy 0, policy_version 1576988 (0.0010) [2023-12-27 02:52:58,838][105620] Updated weights for policy 1, policy_version 1580524 (0.0008) [2023-12-27 02:52:58,881][105692] Updated weights for policy 0, policy_version 1576998 (0.0009) [2023-12-27 02:52:58,908][105620] Updated weights for policy 1, policy_version 1580534 (0.0008) [2023-12-27 02:52:58,973][105620] Updated weights for policy 1, policy_version 1580544 (0.0006) [2023-12-27 02:52:59,620][105620] Updated weights for policy 1, policy_version 1580554 (0.0008) [2023-12-27 02:52:59,680][105620] Updated weights for policy 1, policy_version 1580564 (0.0009) [2023-12-27 02:52:59,729][105620] Updated weights for policy 1, policy_version 1580574 (0.0008) [2023-12-27 02:52:59,732][105692] Updated weights for policy 0, policy_version 1577008 (0.0008) [2023-12-27 02:52:59,787][105692] Updated weights for policy 0, policy_version 1577018 (0.0005) [2023-12-27 02:52:59,792][105620] Updated weights for policy 1, policy_version 1580584 (0.0008) [2023-12-27 02:52:59,844][105692] Updated weights for policy 0, policy_version 1577028 (0.0006) [2023-12-27 02:53:00,528][105692] Updated weights for policy 0, policy_version 1577038 (0.0008) [2023-12-27 02:53:00,593][105692] Updated weights for policy 0, policy_version 1577048 (0.0011) [2023-12-27 02:53:00,615][105620] Updated weights for policy 1, policy_version 1580594 (0.0006) [2023-12-27 02:53:00,649][105692] Updated weights for policy 0, policy_version 1577058 (0.0010) [2023-12-27 02:53:00,667][105620] Updated weights for policy 1, policy_version 1580604 (0.0005) [2023-12-27 02:53:00,727][105620] Updated weights for policy 1, policy_version 1580614 (0.0007) [2023-12-27 02:53:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 808476672. Throughput: 0: 9343.3, 1: 9945.0. Samples: 808447476. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:53:01,062][104569] Avg episode reward: [(0, '8621.789'), (1, '8819.461')] [2023-12-27 02:53:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001577064_403783680.pth... [2023-12-27 02:53:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001580616_404692992.pth... [2023-12-27 02:53:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001575944_403496960.pth [2023-12-27 02:53:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001579464_404398080.pth [2023-12-27 02:53:01,380][105692] Updated weights for policy 0, policy_version 1577068 (0.0010) [2023-12-27 02:53:01,444][105692] Updated weights for policy 0, policy_version 1577078 (0.0008) [2023-12-27 02:53:01,504][105692] Updated weights for policy 0, policy_version 1577088 (0.0009) [2023-12-27 02:53:01,514][105620] Updated weights for policy 1, policy_version 1580624 (0.0006) [2023-12-27 02:53:01,564][105620] Updated weights for policy 1, policy_version 1580634 (0.0007) [2023-12-27 02:53:01,629][105620] Updated weights for policy 1, policy_version 1580644 (0.0007) [2023-12-27 02:53:02,278][105692] Updated weights for policy 0, policy_version 1577098 (0.0009) [2023-12-27 02:53:02,301][105620] Updated weights for policy 1, policy_version 1580654 (0.0007) [2023-12-27 02:53:02,350][105692] Updated weights for policy 0, policy_version 1577108 (0.0008) [2023-12-27 02:53:02,363][105620] Updated weights for policy 1, policy_version 1580664 (0.0007) [2023-12-27 02:53:02,417][105692] Updated weights for policy 0, policy_version 1577118 (0.0007) [2023-12-27 02:53:02,428][105620] Updated weights for policy 1, policy_version 1580674 (0.0006) [2023-12-27 02:53:02,480][105692] Updated weights for policy 0, policy_version 1577128 (0.0009) [2023-12-27 02:53:03,077][105620] Updated weights for policy 1, policy_version 1580684 (0.0005) [2023-12-27 02:53:03,145][105620] Updated weights for policy 1, policy_version 1580694 (0.0006) [2023-12-27 02:53:03,169][105692] Updated weights for policy 0, policy_version 1577138 (0.0005) [2023-12-27 02:53:03,192][105620] Updated weights for policy 1, policy_version 1580704 (0.0005) [2023-12-27 02:53:03,217][105692] Updated weights for policy 0, policy_version 1577148 (0.0006) [2023-12-27 02:53:03,269][105692] Updated weights for policy 0, policy_version 1577158 (0.0010) [2023-12-27 02:53:03,709][105620] Updated weights for policy 1, policy_version 1580714 (0.0005) [2023-12-27 02:53:03,764][105620] Updated weights for policy 1, policy_version 1580724 (0.0005) [2023-12-27 02:53:03,821][105620] Updated weights for policy 1, policy_version 1580734 (0.0005) [2023-12-27 02:53:03,884][105620] Updated weights for policy 1, policy_version 1580744 (0.0006) [2023-12-27 02:53:03,890][105692] Updated weights for policy 0, policy_version 1577169 (0.0007) [2023-12-27 02:53:03,952][105692] Updated weights for policy 0, policy_version 1577179 (0.0005) [2023-12-27 02:53:04,008][105692] Updated weights for policy 0, policy_version 1577189 (0.0010) [2023-12-27 02:53:04,510][105620] Updated weights for policy 1, policy_version 1580754 (0.0009) [2023-12-27 02:53:04,563][105620] Updated weights for policy 1, policy_version 1580764 (0.0009) [2023-12-27 02:53:04,622][105620] Updated weights for policy 1, policy_version 1580774 (0.0009) [2023-12-27 02:53:04,656][105692] Updated weights for policy 0, policy_version 1577199 (0.0006) [2023-12-27 02:53:04,718][105692] Updated weights for policy 0, policy_version 1577209 (0.0010) [2023-12-27 02:53:04,773][105692] Updated weights for policy 0, policy_version 1577219 (0.0011) [2023-12-27 02:53:05,346][105620] Updated weights for policy 1, policy_version 1580784 (0.0008) [2023-12-27 02:53:05,398][105692] Updated weights for policy 0, policy_version 1577229 (0.0008) [2023-12-27 02:53:05,411][105620] Updated weights for policy 1, policy_version 1580794 (0.0011) [2023-12-27 02:53:05,453][105692] Updated weights for policy 0, policy_version 1577239 (0.0009) [2023-12-27 02:53:05,471][105620] Updated weights for policy 1, policy_version 1580804 (0.0011) [2023-12-27 02:53:05,511][105692] Updated weights for policy 0, policy_version 1577249 (0.0006) [2023-12-27 02:53:06,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.8, 300 sec: 19521.9). Total num frames: 808574976. Throughput: 0: 9361.6, 1: 10012.1. Samples: 808565484. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:53:06,063][104569] Avg episode reward: [(0, '8986.897'), (1, '8907.286')] [2023-12-27 02:53:06,143][105692] Updated weights for policy 0, policy_version 1577259 (0.0007) [2023-12-27 02:53:06,193][105620] Updated weights for policy 1, policy_version 1580814 (0.0010) [2023-12-27 02:53:06,203][105692] Updated weights for policy 0, policy_version 1577269 (0.0009) [2023-12-27 02:53:06,253][105620] Updated weights for policy 1, policy_version 1580824 (0.0008) [2023-12-27 02:53:06,262][105692] Updated weights for policy 0, policy_version 1577279 (0.0007) [2023-12-27 02:53:06,311][105620] Updated weights for policy 1, policy_version 1580834 (0.0008) [2023-12-27 02:53:07,029][105692] Updated weights for policy 0, policy_version 1577289 (0.0008) [2023-12-27 02:53:07,066][105620] Updated weights for policy 1, policy_version 1580844 (0.0011) [2023-12-27 02:53:07,084][105692] Updated weights for policy 0, policy_version 1577299 (0.0008) [2023-12-27 02:53:07,115][105620] Updated weights for policy 1, policy_version 1580854 (0.0010) [2023-12-27 02:53:07,129][105692] Updated weights for policy 0, policy_version 1577309 (0.0005) [2023-12-27 02:53:07,169][105620] Updated weights for policy 1, policy_version 1580864 (0.0011) [2023-12-27 02:53:07,183][105692] Updated weights for policy 0, policy_version 1577319 (0.0006) [2023-12-27 02:53:07,833][105620] Updated weights for policy 1, policy_version 1580874 (0.0009) [2023-12-27 02:53:07,881][105692] Updated weights for policy 0, policy_version 1577329 (0.0006) [2023-12-27 02:53:07,892][105620] Updated weights for policy 1, policy_version 1580884 (0.0005) [2023-12-27 02:53:07,946][105692] Updated weights for policy 0, policy_version 1577339 (0.0005) [2023-12-27 02:53:07,957][105620] Updated weights for policy 1, policy_version 1580894 (0.0008) [2023-12-27 02:53:08,011][105692] Updated weights for policy 0, policy_version 1577349 (0.0009) [2023-12-27 02:53:08,012][105620] Updated weights for policy 1, policy_version 1580904 (0.0010) [2023-12-27 02:53:08,610][105692] Updated weights for policy 0, policy_version 1577359 (0.0006) [2023-12-27 02:53:08,673][105692] Updated weights for policy 0, policy_version 1577369 (0.0008) [2023-12-27 02:53:08,694][105620] Updated weights for policy 1, policy_version 1580914 (0.0010) [2023-12-27 02:53:08,726][105692] Updated weights for policy 0, policy_version 1577379 (0.0007) [2023-12-27 02:53:08,752][105620] Updated weights for policy 1, policy_version 1580924 (0.0007) [2023-12-27 02:53:08,805][105620] Updated weights for policy 1, policy_version 1580934 (0.0009) [2023-12-27 02:53:09,452][105692] Updated weights for policy 0, policy_version 1577389 (0.0007) [2023-12-27 02:53:09,510][105692] Updated weights for policy 0, policy_version 1577399 (0.0008) [2023-12-27 02:53:09,570][105692] Updated weights for policy 0, policy_version 1577409 (0.0008) [2023-12-27 02:53:09,576][105620] Updated weights for policy 1, policy_version 1580944 (0.0006) [2023-12-27 02:53:09,637][105620] Updated weights for policy 1, policy_version 1580954 (0.0007) [2023-12-27 02:53:09,702][105620] Updated weights for policy 1, policy_version 1580964 (0.0009) [2023-12-27 02:53:10,405][105620] Updated weights for policy 1, policy_version 1580974 (0.0007) [2023-12-27 02:53:10,407][105692] Updated weights for policy 0, policy_version 1577419 (0.0008) [2023-12-27 02:53:10,464][105620] Updated weights for policy 1, policy_version 1580984 (0.0007) [2023-12-27 02:53:10,466][105692] Updated weights for policy 0, policy_version 1577429 (0.0006) [2023-12-27 02:53:10,527][105620] Updated weights for policy 1, policy_version 1580994 (0.0007) [2023-12-27 02:53:10,527][105692] Updated weights for policy 0, policy_version 1577439 (0.0010) [2023-12-27 02:53:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 808673280. Throughput: 0: 9515.4, 1: 9969.1. Samples: 808683544. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:53:11,062][104569] Avg episode reward: [(0, '8809.195'), (1, '8819.459')] [2023-12-27 02:53:11,207][105620] Updated weights for policy 1, policy_version 1581004 (0.0009) [2023-12-27 02:53:11,264][105620] Updated weights for policy 1, policy_version 1581014 (0.0008) [2023-12-27 02:53:11,318][105620] Updated weights for policy 1, policy_version 1581024 (0.0008) [2023-12-27 02:53:11,358][105692] Updated weights for policy 0, policy_version 1577449 (0.0009) [2023-12-27 02:53:11,424][105692] Updated weights for policy 0, policy_version 1577459 (0.0008) [2023-12-27 02:53:11,480][105692] Updated weights for policy 0, policy_version 1577469 (0.0008) [2023-12-27 02:53:11,531][105692] Updated weights for policy 0, policy_version 1577479 (0.0008) [2023-12-27 02:53:12,086][105620] Updated weights for policy 1, policy_version 1581034 (0.0009) [2023-12-27 02:53:12,151][105620] Updated weights for policy 1, policy_version 1581044 (0.0010) [2023-12-27 02:53:12,215][105620] Updated weights for policy 1, policy_version 1581054 (0.0009) [2023-12-27 02:53:12,269][105620] Updated weights for policy 1, policy_version 1581064 (0.0006) [2023-12-27 02:53:12,270][105692] Updated weights for policy 0, policy_version 1577489 (0.0009) [2023-12-27 02:53:12,331][105692] Updated weights for policy 0, policy_version 1577499 (0.0009) [2023-12-27 02:53:12,399][105692] Updated weights for policy 0, policy_version 1577509 (0.0009) [2023-12-27 02:53:12,925][105620] Updated weights for policy 1, policy_version 1581074 (0.0010) [2023-12-27 02:53:12,988][105620] Updated weights for policy 1, policy_version 1581084 (0.0009) [2023-12-27 02:53:13,052][105620] Updated weights for policy 1, policy_version 1581094 (0.0009) [2023-12-27 02:53:13,203][105692] Updated weights for policy 0, policy_version 1577519 (0.0010) [2023-12-27 02:53:13,255][105692] Updated weights for policy 0, policy_version 1577529 (0.0009) [2023-12-27 02:53:13,306][105692] Updated weights for policy 0, policy_version 1577539 (0.0007) [2023-12-27 02:53:13,839][105620] Updated weights for policy 1, policy_version 1581104 (0.0009) [2023-12-27 02:53:13,898][105620] Updated weights for policy 1, policy_version 1581114 (0.0008) [2023-12-27 02:53:13,931][105692] Updated weights for policy 0, policy_version 1577549 (0.0008) [2023-12-27 02:53:13,957][105620] Updated weights for policy 1, policy_version 1581124 (0.0008) [2023-12-27 02:53:13,983][105692] Updated weights for policy 0, policy_version 1577559 (0.0010) [2023-12-27 02:53:14,041][105692] Updated weights for policy 0, policy_version 1577569 (0.0010) [2023-12-27 02:53:14,682][105620] Updated weights for policy 1, policy_version 1581134 (0.0007) [2023-12-27 02:53:14,730][105620] Updated weights for policy 1, policy_version 1581144 (0.0008) [2023-12-27 02:53:14,788][105620] Updated weights for policy 1, policy_version 1581154 (0.0008) [2023-12-27 02:53:14,793][105692] Updated weights for policy 0, policy_version 1577580 (0.0010) [2023-12-27 02:53:14,854][105692] Updated weights for policy 0, policy_version 1577590 (0.0008) [2023-12-27 02:53:14,916][105692] Updated weights for policy 0, policy_version 1577600 (0.0010) [2023-12-27 02:53:15,552][105620] Updated weights for policy 1, policy_version 1581164 (0.0007) [2023-12-27 02:53:15,615][105620] Updated weights for policy 1, policy_version 1581174 (0.0008) [2023-12-27 02:53:15,625][105692] Updated weights for policy 0, policy_version 1577610 (0.0007) [2023-12-27 02:53:15,667][105620] Updated weights for policy 1, policy_version 1581184 (0.0006) [2023-12-27 02:53:15,687][105692] Updated weights for policy 0, policy_version 1577620 (0.0010) [2023-12-27 02:53:15,750][105692] Updated weights for policy 0, policy_version 1577630 (0.0011) [2023-12-27 02:53:15,808][105692] Updated weights for policy 0, policy_version 1577640 (0.0011) [2023-12-27 02:53:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 808771584. Throughput: 0: 9515.5, 1: 9944.3. Samples: 808739532. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:53:16,062][104569] Avg episode reward: [(0, '8723.025'), (1, '8723.679')] [2023-12-27 02:53:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001577640_403931136.pth... [2023-12-27 02:53:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001581192_404840448.pth... [2023-12-27 02:53:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001576520_403644416.pth [2023-12-27 02:53:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001580040_404545536.pth [2023-12-27 02:53:16,369][105620] Updated weights for policy 1, policy_version 1581194 (0.0008) [2023-12-27 02:53:16,423][105620] Updated weights for policy 1, policy_version 1581204 (0.0006) [2023-12-27 02:53:16,453][105692] Updated weights for policy 0, policy_version 1577650 (0.0005) [2023-12-27 02:53:16,475][105620] Updated weights for policy 1, policy_version 1581214 (0.0006) [2023-12-27 02:53:16,504][105692] Updated weights for policy 0, policy_version 1577660 (0.0005) [2023-12-27 02:53:16,526][105620] Updated weights for policy 1, policy_version 1581224 (0.0008) [2023-12-27 02:53:16,556][105692] Updated weights for policy 0, policy_version 1577670 (0.0010) [2023-12-27 02:53:17,119][105620] Updated weights for policy 1, policy_version 1581234 (0.0008) [2023-12-27 02:53:17,167][105620] Updated weights for policy 1, policy_version 1581244 (0.0009) [2023-12-27 02:53:17,235][105620] Updated weights for policy 1, policy_version 1581254 (0.0007) [2023-12-27 02:53:17,261][105692] Updated weights for policy 0, policy_version 1577680 (0.0009) [2023-12-27 02:53:17,331][105692] Updated weights for policy 0, policy_version 1577690 (0.0009) [2023-12-27 02:53:17,404][105692] Updated weights for policy 0, policy_version 1577700 (0.0010) [2023-12-27 02:53:17,810][105620] Updated weights for policy 1, policy_version 1581264 (0.0005) [2023-12-27 02:53:17,866][105620] Updated weights for policy 1, policy_version 1581274 (0.0005) [2023-12-27 02:53:17,921][105620] Updated weights for policy 1, policy_version 1581284 (0.0005) [2023-12-27 02:53:18,270][105692] Updated weights for policy 0, policy_version 1577710 (0.0010) [2023-12-27 02:53:18,333][105692] Updated weights for policy 0, policy_version 1577720 (0.0009) [2023-12-27 02:53:18,396][105692] Updated weights for policy 0, policy_version 1577730 (0.0009) [2023-12-27 02:53:18,446][105620] Updated weights for policy 1, policy_version 1581294 (0.0007) [2023-12-27 02:53:18,497][105620] Updated weights for policy 1, policy_version 1581304 (0.0009) [2023-12-27 02:53:18,545][105620] Updated weights for policy 1, policy_version 1581314 (0.0009) [2023-12-27 02:53:19,171][105692] Updated weights for policy 0, policy_version 1577740 (0.0008) [2023-12-27 02:53:19,236][105692] Updated weights for policy 0, policy_version 1577750 (0.0009) [2023-12-27 02:53:19,294][105692] Updated weights for policy 0, policy_version 1577760 (0.0009) [2023-12-27 02:53:19,308][105620] Updated weights for policy 1, policy_version 1581324 (0.0008) [2023-12-27 02:53:19,370][105620] Updated weights for policy 1, policy_version 1581334 (0.0008) [2023-12-27 02:53:19,441][105620] Updated weights for policy 1, policy_version 1581344 (0.0009) [2023-12-27 02:53:20,038][105692] Updated weights for policy 0, policy_version 1577770 (0.0006) [2023-12-27 02:53:20,104][105692] Updated weights for policy 0, policy_version 1577780 (0.0006) [2023-12-27 02:53:20,156][105692] Updated weights for policy 0, policy_version 1577790 (0.0006) [2023-12-27 02:53:20,224][105692] Updated weights for policy 0, policy_version 1577800 (0.0007) [2023-12-27 02:53:20,237][105620] Updated weights for policy 1, policy_version 1581354 (0.0009) [2023-12-27 02:53:20,299][105620] Updated weights for policy 1, policy_version 1581364 (0.0009) [2023-12-27 02:53:20,357][105620] Updated weights for policy 1, policy_version 1581374 (0.0008) [2023-12-27 02:53:20,416][105620] Updated weights for policy 1, policy_version 1581384 (0.0010) [2023-12-27 02:53:20,931][105692] Updated weights for policy 0, policy_version 1577810 (0.0008) [2023-12-27 02:53:20,999][105692] Updated weights for policy 0, policy_version 1577820 (0.0008) [2023-12-27 02:53:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 808861696. Throughput: 0: 9598.5, 1: 9894.2. Samples: 808857936. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:53:21,063][104569] Avg episode reward: [(0, '8623.367'), (1, '8716.652')] [2023-12-27 02:53:21,063][105692] Updated weights for policy 0, policy_version 1577830 (0.0009) [2023-12-27 02:53:21,257][105620] Updated weights for policy 1, policy_version 1581394 (0.0011) [2023-12-27 02:53:21,322][105620] Updated weights for policy 1, policy_version 1581404 (0.0011) [2023-12-27 02:53:21,393][105620] Updated weights for policy 1, policy_version 1581414 (0.0009) [2023-12-27 02:53:21,885][105692] Updated weights for policy 0, policy_version 1577840 (0.0008) [2023-12-27 02:53:21,937][105692] Updated weights for policy 0, policy_version 1577850 (0.0008) [2023-12-27 02:53:21,985][105692] Updated weights for policy 0, policy_version 1577860 (0.0008) [2023-12-27 02:53:22,099][105620] Updated weights for policy 1, policy_version 1581424 (0.0006) [2023-12-27 02:53:22,155][105620] Updated weights for policy 1, policy_version 1581434 (0.0006) [2023-12-27 02:53:22,214][105620] Updated weights for policy 1, policy_version 1581444 (0.0011) [2023-12-27 02:53:22,835][105620] Updated weights for policy 1, policy_version 1581454 (0.0008) [2023-12-27 02:53:22,890][105692] Updated weights for policy 0, policy_version 1577870 (0.0008) [2023-12-27 02:53:22,891][105620] Updated weights for policy 1, policy_version 1581464 (0.0005) [2023-12-27 02:53:22,947][105692] Updated weights for policy 0, policy_version 1577880 (0.0008) [2023-12-27 02:53:22,949][105620] Updated weights for policy 1, policy_version 1581474 (0.0010) [2023-12-27 02:53:23,004][105692] Updated weights for policy 0, policy_version 1577890 (0.0008) [2023-12-27 02:53:23,664][105620] Updated weights for policy 1, policy_version 1581484 (0.0011) [2023-12-27 02:53:23,722][105620] Updated weights for policy 1, policy_version 1581494 (0.0010) [2023-12-27 02:53:23,764][105692] Updated weights for policy 0, policy_version 1577900 (0.0008) [2023-12-27 02:53:23,767][105620] Updated weights for policy 1, policy_version 1581504 (0.0010) [2023-12-27 02:53:23,816][105692] Updated weights for policy 0, policy_version 1577910 (0.0006) [2023-12-27 02:53:23,874][105692] Updated weights for policy 0, policy_version 1577920 (0.0008) [2023-12-27 02:53:24,521][105620] Updated weights for policy 1, policy_version 1581514 (0.0010) [2023-12-27 02:53:24,566][105620] Updated weights for policy 1, policy_version 1581524 (0.0010) [2023-12-27 02:53:24,621][105620] Updated weights for policy 1, policy_version 1581534 (0.0010) [2023-12-27 02:53:24,630][105692] Updated weights for policy 0, policy_version 1577930 (0.0009) [2023-12-27 02:53:24,665][105620] Updated weights for policy 1, policy_version 1581544 (0.0010) [2023-12-27 02:53:24,686][105692] Updated weights for policy 0, policy_version 1577940 (0.0009) [2023-12-27 02:53:24,751][105692] Updated weights for policy 0, policy_version 1577950 (0.0008) [2023-12-27 02:53:24,817][105692] Updated weights for policy 0, policy_version 1577960 (0.0008) [2023-12-27 02:53:25,423][105620] Updated weights for policy 1, policy_version 1581554 (0.0010) [2023-12-27 02:53:25,437][105692] Updated weights for policy 0, policy_version 1577970 (0.0005) [2023-12-27 02:53:25,471][105620] Updated weights for policy 1, policy_version 1581564 (0.0010) [2023-12-27 02:53:25,494][105692] Updated weights for policy 0, policy_version 1577980 (0.0006) [2023-12-27 02:53:25,524][105620] Updated weights for policy 1, policy_version 1581574 (0.0011) [2023-12-27 02:53:25,543][105692] Updated weights for policy 0, policy_version 1577990 (0.0006) [2023-12-27 02:53:26,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 808960000. Throughput: 0: 9582.3, 1: 9868.0. Samples: 808971112. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:53:26,063][104569] Avg episode reward: [(0, '8531.148'), (1, '8717.039')] [2023-12-27 02:53:26,253][105692] Updated weights for policy 0, policy_version 1578000 (0.0008) [2023-12-27 02:53:26,308][105620] Updated weights for policy 1, policy_version 1581584 (0.0011) [2023-12-27 02:53:26,321][105692] Updated weights for policy 0, policy_version 1578010 (0.0010) [2023-12-27 02:53:26,367][105620] Updated weights for policy 1, policy_version 1581594 (0.0011) [2023-12-27 02:53:26,381][105692] Updated weights for policy 0, policy_version 1578020 (0.0006) [2023-12-27 02:53:26,423][105620] Updated weights for policy 1, policy_version 1581604 (0.0011) [2023-12-27 02:53:27,115][105692] Updated weights for policy 0, policy_version 1578030 (0.0007) [2023-12-27 02:53:27,147][105620] Updated weights for policy 1, policy_version 1581614 (0.0010) [2023-12-27 02:53:27,164][105692] Updated weights for policy 0, policy_version 1578040 (0.0009) [2023-12-27 02:53:27,195][105620] Updated weights for policy 1, policy_version 1581624 (0.0010) [2023-12-27 02:53:27,217][105692] Updated weights for policy 0, policy_version 1578050 (0.0005) [2023-12-27 02:53:27,250][105620] Updated weights for policy 1, policy_version 1581634 (0.0010) [2023-12-27 02:53:27,978][105692] Updated weights for policy 0, policy_version 1578060 (0.0007) [2023-12-27 02:53:28,000][105620] Updated weights for policy 1, policy_version 1581644 (0.0010) [2023-12-27 02:53:28,022][105692] Updated weights for policy 0, policy_version 1578070 (0.0005) [2023-12-27 02:53:28,047][105620] Updated weights for policy 1, policy_version 1581654 (0.0010) [2023-12-27 02:53:28,065][105692] Updated weights for policy 0, policy_version 1578080 (0.0005) [2023-12-27 02:53:28,095][105620] Updated weights for policy 1, policy_version 1581664 (0.0010) [2023-12-27 02:53:28,812][105692] Updated weights for policy 0, policy_version 1578090 (0.0006) [2023-12-27 02:53:28,852][105620] Updated weights for policy 1, policy_version 1581674 (0.0010) [2023-12-27 02:53:28,870][105692] Updated weights for policy 0, policy_version 1578100 (0.0007) [2023-12-27 02:53:28,907][105620] Updated weights for policy 1, policy_version 1581684 (0.0010) [2023-12-27 02:53:28,925][105692] Updated weights for policy 0, policy_version 1578110 (0.0006) [2023-12-27 02:53:28,966][105620] Updated weights for policy 1, policy_version 1581694 (0.0010) [2023-12-27 02:53:28,987][105692] Updated weights for policy 0, policy_version 1578120 (0.0005) [2023-12-27 02:53:29,016][105620] Updated weights for policy 1, policy_version 1581704 (0.0010) [2023-12-27 02:53:29,722][105620] Updated weights for policy 1, policy_version 1581714 (0.0011) [2023-12-27 02:53:29,762][105692] Updated weights for policy 0, policy_version 1578130 (0.0009) [2023-12-27 02:53:29,775][105620] Updated weights for policy 1, policy_version 1581724 (0.0011) [2023-12-27 02:53:29,823][105692] Updated weights for policy 0, policy_version 1578140 (0.0007) [2023-12-27 02:53:29,833][105620] Updated weights for policy 1, policy_version 1581734 (0.0011) [2023-12-27 02:53:29,886][105692] Updated weights for policy 0, policy_version 1578150 (0.0007) [2023-12-27 02:53:30,608][105620] Updated weights for policy 1, policy_version 1581744 (0.0010) [2023-12-27 02:53:30,666][105620] Updated weights for policy 1, policy_version 1581754 (0.0010) [2023-12-27 02:53:30,673][105692] Updated weights for policy 0, policy_version 1578160 (0.0007) [2023-12-27 02:53:30,722][105692] Updated weights for policy 0, policy_version 1578170 (0.0008) [2023-12-27 02:53:30,732][105620] Updated weights for policy 1, policy_version 1581764 (0.0010) [2023-12-27 02:53:30,778][105692] Updated weights for policy 0, policy_version 1578180 (0.0007) [2023-12-27 02:53:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 809058304. Throughput: 0: 9570.5, 1: 9863.3. Samples: 809027784. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:53:31,062][104569] Avg episode reward: [(0, '8261.105'), (1, '8902.867')] [2023-12-27 02:53:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001578184_404070400.pth... [2023-12-27 02:53:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001581768_404987904.pth... [2023-12-27 02:53:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001577064_403783680.pth [2023-12-27 02:53:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001580616_404692992.pth [2023-12-27 02:53:31,486][105620] Updated weights for policy 1, policy_version 1581774 (0.0011) [2023-12-27 02:53:31,530][105620] Updated weights for policy 1, policy_version 1581784 (0.0011) [2023-12-27 02:53:31,590][105620] Updated weights for policy 1, policy_version 1581794 (0.0011) [2023-12-27 02:53:31,611][105692] Updated weights for policy 0, policy_version 1578190 (0.0009) [2023-12-27 02:53:31,674][105692] Updated weights for policy 0, policy_version 1578200 (0.0008) [2023-12-27 02:53:31,749][105692] Updated weights for policy 0, policy_version 1578210 (0.0008) [2023-12-27 02:53:32,320][105620] Updated weights for policy 1, policy_version 1581804 (0.0008) [2023-12-27 02:53:32,382][105620] Updated weights for policy 1, policy_version 1581814 (0.0009) [2023-12-27 02:53:32,430][105620] Updated weights for policy 1, policy_version 1581824 (0.0009) [2023-12-27 02:53:32,538][105692] Updated weights for policy 0, policy_version 1578220 (0.0009) [2023-12-27 02:53:32,596][105692] Updated weights for policy 0, policy_version 1578230 (0.0008) [2023-12-27 02:53:32,654][105692] Updated weights for policy 0, policy_version 1578240 (0.0009) [2023-12-27 02:53:33,053][105620] Updated weights for policy 1, policy_version 1581834 (0.0008) [2023-12-27 02:53:33,112][105620] Updated weights for policy 1, policy_version 1581844 (0.0009) [2023-12-27 02:53:33,169][105620] Updated weights for policy 1, policy_version 1581854 (0.0006) [2023-12-27 02:53:33,227][105620] Updated weights for policy 1, policy_version 1581864 (0.0009) [2023-12-27 02:53:33,467][105692] Updated weights for policy 0, policy_version 1578250 (0.0009) [2023-12-27 02:53:33,515][105692] Updated weights for policy 0, policy_version 1578260 (0.0006) [2023-12-27 02:53:33,568][105692] Updated weights for policy 0, policy_version 1578270 (0.0008) [2023-12-27 02:53:33,621][105692] Updated weights for policy 0, policy_version 1578280 (0.0009) [2023-12-27 02:53:33,914][105620] Updated weights for policy 1, policy_version 1581874 (0.0010) [2023-12-27 02:53:33,966][105620] Updated weights for policy 1, policy_version 1581884 (0.0009) [2023-12-27 02:53:34,024][105620] Updated weights for policy 1, policy_version 1581895 (0.0010) [2023-12-27 02:53:34,211][105692] Updated weights for policy 0, policy_version 1578290 (0.0008) [2023-12-27 02:53:34,280][105692] Updated weights for policy 0, policy_version 1578300 (0.0008) [2023-12-27 02:53:34,341][105692] Updated weights for policy 0, policy_version 1578310 (0.0009) [2023-12-27 02:53:34,876][105620] Updated weights for policy 1, policy_version 1581905 (0.0009) [2023-12-27 02:53:34,937][105620] Updated weights for policy 1, policy_version 1581915 (0.0009) [2023-12-27 02:53:34,991][105620] Updated weights for policy 1, policy_version 1581925 (0.0007) [2023-12-27 02:53:34,998][105692] Updated weights for policy 0, policy_version 1578320 (0.0007) [2023-12-27 02:53:35,050][105692] Updated weights for policy 0, policy_version 1578330 (0.0008) [2023-12-27 02:53:35,112][105692] Updated weights for policy 0, policy_version 1578340 (0.0009) [2023-12-27 02:53:35,731][105620] Updated weights for policy 1, policy_version 1581935 (0.0008) [2023-12-27 02:53:35,796][105620] Updated weights for policy 1, policy_version 1581945 (0.0009) [2023-12-27 02:53:35,864][105620] Updated weights for policy 1, policy_version 1581955 (0.0008) [2023-12-27 02:53:35,887][105692] Updated weights for policy 0, policy_version 1578350 (0.0007) [2023-12-27 02:53:35,951][105692] Updated weights for policy 0, policy_version 1578360 (0.0009) [2023-12-27 02:53:36,004][105692] Updated weights for policy 0, policy_version 1578370 (0.0009) [2023-12-27 02:53:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 809156608. Throughput: 0: 9409.3, 1: 9840.1. Samples: 809141024. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:53:36,063][104569] Avg episode reward: [(0, '8166.337'), (1, '8902.496')] [2023-12-27 02:53:36,538][105620] Updated weights for policy 1, policy_version 1581965 (0.0008) [2023-12-27 02:53:36,599][105620] Updated weights for policy 1, policy_version 1581975 (0.0009) [2023-12-27 02:53:36,661][105620] Updated weights for policy 1, policy_version 1581985 (0.0008) [2023-12-27 02:53:36,802][105692] Updated weights for policy 0, policy_version 1578380 (0.0009) [2023-12-27 02:53:36,860][105692] Updated weights for policy 0, policy_version 1578390 (0.0009) [2023-12-27 02:53:36,915][105692] Updated weights for policy 0, policy_version 1578400 (0.0009) [2023-12-27 02:53:37,387][105620] Updated weights for policy 1, policy_version 1581995 (0.0009) [2023-12-27 02:53:37,442][105620] Updated weights for policy 1, policy_version 1582005 (0.0009) [2023-12-27 02:53:37,501][105620] Updated weights for policy 1, policy_version 1582015 (0.0009) [2023-12-27 02:53:37,672][105692] Updated weights for policy 0, policy_version 1578410 (0.0010) [2023-12-27 02:53:37,725][105692] Updated weights for policy 0, policy_version 1578420 (0.0009) [2023-12-27 02:53:37,790][105692] Updated weights for policy 0, policy_version 1578430 (0.0009) [2023-12-27 02:53:37,850][105692] Updated weights for policy 0, policy_version 1578440 (0.0008) [2023-12-27 02:53:38,263][105620] Updated weights for policy 1, policy_version 1582025 (0.0009) [2023-12-27 02:53:38,310][105620] Updated weights for policy 1, policy_version 1582035 (0.0008) [2023-12-27 02:53:38,380][105620] Updated weights for policy 1, policy_version 1582045 (0.0009) [2023-12-27 02:53:38,431][105620] Updated weights for policy 1, policy_version 1582055 (0.0008) [2023-12-27 02:53:38,597][105692] Updated weights for policy 0, policy_version 1578450 (0.0009) [2023-12-27 02:53:38,658][105692] Updated weights for policy 0, policy_version 1578460 (0.0009) [2023-12-27 02:53:38,728][105692] Updated weights for policy 0, policy_version 1578470 (0.0009) [2023-12-27 02:53:39,192][105620] Updated weights for policy 1, policy_version 1582065 (0.0006) [2023-12-27 02:53:39,263][105620] Updated weights for policy 1, policy_version 1582075 (0.0008) [2023-12-27 02:53:39,328][105620] Updated weights for policy 1, policy_version 1582085 (0.0008) [2023-12-27 02:53:39,467][105692] Updated weights for policy 0, policy_version 1578480 (0.0009) [2023-12-27 02:53:39,517][105692] Updated weights for policy 0, policy_version 1578490 (0.0008) [2023-12-27 02:53:39,566][105692] Updated weights for policy 0, policy_version 1578500 (0.0009) [2023-12-27 02:53:40,003][105620] Updated weights for policy 1, policy_version 1582095 (0.0008) [2023-12-27 02:53:40,068][105620] Updated weights for policy 1, policy_version 1582105 (0.0008) [2023-12-27 02:53:40,132][105620] Updated weights for policy 1, policy_version 1582115 (0.0007) [2023-12-27 02:53:40,407][105692] Updated weights for policy 0, policy_version 1578510 (0.0009) [2023-12-27 02:53:40,466][105692] Updated weights for policy 0, policy_version 1578520 (0.0009) [2023-12-27 02:53:40,531][105692] Updated weights for policy 0, policy_version 1578530 (0.0010) [2023-12-27 02:53:40,756][105620] Updated weights for policy 1, policy_version 1582125 (0.0009) [2023-12-27 02:53:40,810][105620] Updated weights for policy 1, policy_version 1582135 (0.0009) [2023-12-27 02:53:40,873][105620] Updated weights for policy 1, policy_version 1582145 (0.0009) [2023-12-27 02:53:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 809246720. Throughput: 0: 9385.7, 1: 9823.0. Samples: 809253724. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:53:41,063][104569] Avg episode reward: [(0, '8714.730'), (1, '8809.325')] [2023-12-27 02:53:41,286][105692] Updated weights for policy 0, policy_version 1578541 (0.0010) [2023-12-27 02:53:41,354][105692] Updated weights for policy 0, policy_version 1578551 (0.0010) [2023-12-27 02:53:41,417][105692] Updated weights for policy 0, policy_version 1578561 (0.0009) [2023-12-27 02:53:41,632][105620] Updated weights for policy 1, policy_version 1582155 (0.0009) [2023-12-27 02:53:41,690][105620] Updated weights for policy 1, policy_version 1582165 (0.0009) [2023-12-27 02:53:41,762][105620] Updated weights for policy 1, policy_version 1582175 (0.0008) [2023-12-27 02:53:42,220][105692] Updated weights for policy 0, policy_version 1578571 (0.0009) [2023-12-27 02:53:42,291][105692] Updated weights for policy 0, policy_version 1578581 (0.0009) [2023-12-27 02:53:42,362][105692] Updated weights for policy 0, policy_version 1578591 (0.0010) [2023-12-27 02:53:42,458][105620] Updated weights for policy 1, policy_version 1582185 (0.0008) [2023-12-27 02:53:42,527][105620] Updated weights for policy 1, policy_version 1582195 (0.0006) [2023-12-27 02:53:42,596][105620] Updated weights for policy 1, policy_version 1582205 (0.0006) [2023-12-27 02:53:42,664][105620] Updated weights for policy 1, policy_version 1582215 (0.0006) [2023-12-27 02:53:42,972][105692] Updated weights for policy 0, policy_version 1578601 (0.0011) [2023-12-27 02:53:43,042][105692] Updated weights for policy 0, policy_version 1578611 (0.0008) [2023-12-27 02:53:43,101][105692] Updated weights for policy 0, policy_version 1578621 (0.0005) [2023-12-27 02:53:43,153][105692] Updated weights for policy 0, policy_version 1578631 (0.0005) [2023-12-27 02:53:43,419][105620] Updated weights for policy 1, policy_version 1582225 (0.0009) [2023-12-27 02:53:43,477][105620] Updated weights for policy 1, policy_version 1582235 (0.0010) [2023-12-27 02:53:43,531][105620] Updated weights for policy 1, policy_version 1582246 (0.0010) [2023-12-27 02:53:43,675][105692] Updated weights for policy 0, policy_version 1578641 (0.0010) [2023-12-27 02:53:43,728][105692] Updated weights for policy 0, policy_version 1578651 (0.0006) [2023-12-27 02:53:43,787][105692] Updated weights for policy 0, policy_version 1578661 (0.0005) [2023-12-27 02:53:44,270][105620] Updated weights for policy 1, policy_version 1582256 (0.0009) [2023-12-27 02:53:44,330][105620] Updated weights for policy 1, policy_version 1582266 (0.0009) [2023-12-27 02:53:44,391][105620] Updated weights for policy 1, policy_version 1582276 (0.0009) [2023-12-27 02:53:44,433][105692] Updated weights for policy 0, policy_version 1578671 (0.0009) [2023-12-27 02:53:44,491][105692] Updated weights for policy 0, policy_version 1578681 (0.0010) [2023-12-27 02:53:44,550][105692] Updated weights for policy 0, policy_version 1578691 (0.0010) [2023-12-27 02:53:45,056][105620] Updated weights for policy 1, policy_version 1582286 (0.0008) [2023-12-27 02:53:45,121][105620] Updated weights for policy 1, policy_version 1582296 (0.0009) [2023-12-27 02:53:45,185][105620] Updated weights for policy 1, policy_version 1582306 (0.0008) [2023-12-27 02:53:45,292][105692] Updated weights for policy 0, policy_version 1578701 (0.0010) [2023-12-27 02:53:45,361][105692] Updated weights for policy 0, policy_version 1578711 (0.0010) [2023-12-27 02:53:45,432][105692] Updated weights for policy 0, policy_version 1578721 (0.0010) [2023-12-27 02:53:45,747][105620] Updated weights for policy 1, policy_version 1582316 (0.0007) [2023-12-27 02:53:45,806][105620] Updated weights for policy 1, policy_version 1582326 (0.0005) [2023-12-27 02:53:45,862][105620] Updated weights for policy 1, policy_version 1582336 (0.0005) [2023-12-27 02:53:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 809345024. Throughput: 0: 9486.5, 1: 9752.7. Samples: 809313240. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:53:46,063][104569] Avg episode reward: [(0, '8811.206'), (1, '8629.186')] [2023-12-27 02:53:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001582344_405135360.pth... [2023-12-27 02:53:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001578728_404209664.pth... [2023-12-27 02:53:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001577640_403931136.pth [2023-12-27 02:53:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001581192_404840448.pth [2023-12-27 02:53:46,143][105692] Updated weights for policy 0, policy_version 1578731 (0.0009) [2023-12-27 02:53:46,191][105692] Updated weights for policy 0, policy_version 1578741 (0.0005) [2023-12-27 02:53:46,247][105692] Updated weights for policy 0, policy_version 1578751 (0.0005) [2023-12-27 02:53:46,490][105620] Updated weights for policy 1, policy_version 1582346 (0.0006) [2023-12-27 02:53:46,550][105620] Updated weights for policy 1, policy_version 1582356 (0.0005) [2023-12-27 02:53:46,607][105620] Updated weights for policy 1, policy_version 1582366 (0.0007) [2023-12-27 02:53:46,664][105620] Updated weights for policy 1, policy_version 1582376 (0.0009) [2023-12-27 02:53:46,809][105692] Updated weights for policy 0, policy_version 1578761 (0.0006) [2023-12-27 02:53:46,860][105692] Updated weights for policy 0, policy_version 1578771 (0.0010) [2023-12-27 02:53:46,920][105692] Updated weights for policy 0, policy_version 1578781 (0.0010) [2023-12-27 02:53:46,982][105692] Updated weights for policy 0, policy_version 1578791 (0.0010) [2023-12-27 02:53:47,305][105620] Updated weights for policy 1, policy_version 1582386 (0.0009) [2023-12-27 02:53:47,350][105620] Updated weights for policy 1, policy_version 1582396 (0.0008) [2023-12-27 02:53:47,414][105620] Updated weights for policy 1, policy_version 1582406 (0.0007) [2023-12-27 02:53:47,709][105692] Updated weights for policy 0, policy_version 1578801 (0.0011) [2023-12-27 02:53:47,775][105692] Updated weights for policy 0, policy_version 1578811 (0.0010) [2023-12-27 02:53:47,840][105692] Updated weights for policy 0, policy_version 1578821 (0.0011) [2023-12-27 02:53:48,138][105620] Updated weights for policy 1, policy_version 1582416 (0.0006) [2023-12-27 02:53:48,183][105620] Updated weights for policy 1, policy_version 1582426 (0.0006) [2023-12-27 02:53:48,240][105620] Updated weights for policy 1, policy_version 1582436 (0.0007) [2023-12-27 02:53:48,525][105692] Updated weights for policy 0, policy_version 1578831 (0.0009) [2023-12-27 02:53:48,589][105692] Updated weights for policy 0, policy_version 1578841 (0.0009) [2023-12-27 02:53:48,643][105692] Updated weights for policy 0, policy_version 1578851 (0.0009) [2023-12-27 02:53:49,002][105620] Updated weights for policy 1, policy_version 1582447 (0.0011) [2023-12-27 02:53:49,043][105586] KL-divergence is very high: 127.9552 [2023-12-27 02:53:49,056][105620] Updated weights for policy 1, policy_version 1582458 (0.0010) [2023-12-27 02:53:49,082][105586] KL-divergence is very high: 127.7995 [2023-12-27 02:53:49,110][105620] Updated weights for policy 1, policy_version 1582470 (0.0011) [2023-12-27 02:53:49,276][105692] Updated weights for policy 0, policy_version 1578861 (0.0008) [2023-12-27 02:53:49,342][105692] Updated weights for policy 0, policy_version 1578871 (0.0009) [2023-12-27 02:53:49,408][105692] Updated weights for policy 0, policy_version 1578881 (0.0009) [2023-12-27 02:53:49,888][105620] Updated weights for policy 1, policy_version 1582480 (0.0009) [2023-12-27 02:53:49,956][105620] Updated weights for policy 1, policy_version 1582490 (0.0009) [2023-12-27 02:53:50,014][105620] Updated weights for policy 1, policy_version 1582500 (0.0008) [2023-12-27 02:53:50,165][105692] Updated weights for policy 0, policy_version 1578891 (0.0009) [2023-12-27 02:53:50,224][105692] Updated weights for policy 0, policy_version 1578901 (0.0009) [2023-12-27 02:53:50,282][105692] Updated weights for policy 0, policy_version 1578911 (0.0009) [2023-12-27 02:53:50,806][105620] Updated weights for policy 1, policy_version 1582510 (0.0008) [2023-12-27 02:53:50,864][105620] Updated weights for policy 1, policy_version 1582520 (0.0009) [2023-12-27 02:53:50,926][105620] Updated weights for policy 1, policy_version 1582530 (0.0006) [2023-12-27 02:53:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 809443328. Throughput: 0: 9543.0, 1: 9760.6. Samples: 809434148. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:53:51,062][104569] Avg episode reward: [(0, '8350.727'), (1, '8544.510')] [2023-12-27 02:53:51,106][105692] Updated weights for policy 0, policy_version 1578921 (0.0009) [2023-12-27 02:53:51,165][105692] Updated weights for policy 0, policy_version 1578931 (0.0010) [2023-12-27 02:53:51,222][105692] Updated weights for policy 0, policy_version 1578941 (0.0009) [2023-12-27 02:53:51,279][105692] Updated weights for policy 0, policy_version 1578951 (0.0008) [2023-12-27 02:53:51,593][105620] Updated weights for policy 1, policy_version 1582540 (0.0007) [2023-12-27 02:53:51,664][105620] Updated weights for policy 1, policy_version 1582550 (0.0007) [2023-12-27 02:53:51,721][105620] Updated weights for policy 1, policy_version 1582560 (0.0006) [2023-12-27 02:53:52,021][105692] Updated weights for policy 0, policy_version 1578961 (0.0008) [2023-12-27 02:53:52,083][105692] Updated weights for policy 0, policy_version 1578971 (0.0008) [2023-12-27 02:53:52,154][105692] Updated weights for policy 0, policy_version 1578981 (0.0009) [2023-12-27 02:53:52,373][105620] Updated weights for policy 1, policy_version 1582570 (0.0007) [2023-12-27 02:53:52,435][105620] Updated weights for policy 1, policy_version 1582580 (0.0007) [2023-12-27 02:53:52,501][105620] Updated weights for policy 1, policy_version 1582590 (0.0008) [2023-12-27 02:53:52,567][105620] Updated weights for policy 1, policy_version 1582600 (0.0008) [2023-12-27 02:53:52,998][105692] Updated weights for policy 0, policy_version 1578991 (0.0008) [2023-12-27 02:53:53,055][105692] Updated weights for policy 0, policy_version 1579001 (0.0008) [2023-12-27 02:53:53,114][105692] Updated weights for policy 0, policy_version 1579011 (0.0009) [2023-12-27 02:53:53,220][105620] Updated weights for policy 1, policy_version 1582610 (0.0009) [2023-12-27 02:53:53,274][105620] Updated weights for policy 1, policy_version 1582620 (0.0009) [2023-12-27 02:53:53,323][105620] Updated weights for policy 1, policy_version 1582630 (0.0008) [2023-12-27 02:53:53,922][105692] Updated weights for policy 0, policy_version 1579021 (0.0009) [2023-12-27 02:53:53,972][105620] Updated weights for policy 1, policy_version 1582640 (0.0006) [2023-12-27 02:53:53,974][105692] Updated weights for policy 0, policy_version 1579031 (0.0009) [2023-12-27 02:53:54,019][105692] Updated weights for policy 0, policy_version 1579041 (0.0006) [2023-12-27 02:53:54,021][105620] Updated weights for policy 1, policy_version 1582650 (0.0006) [2023-12-27 02:53:54,071][105620] Updated weights for policy 1, policy_version 1582660 (0.0007) [2023-12-27 02:53:54,743][105692] Updated weights for policy 0, policy_version 1579051 (0.0007) [2023-12-27 02:53:54,796][105692] Updated weights for policy 0, policy_version 1579061 (0.0008) [2023-12-27 02:53:54,847][105620] Updated weights for policy 1, policy_version 1582670 (0.0009) [2023-12-27 02:53:54,854][105692] Updated weights for policy 0, policy_version 1579071 (0.0009) [2023-12-27 02:53:54,900][105620] Updated weights for policy 1, policy_version 1582680 (0.0007) [2023-12-27 02:53:54,961][105620] Updated weights for policy 1, policy_version 1582690 (0.0008) [2023-12-27 02:53:55,585][105692] Updated weights for policy 0, policy_version 1579081 (0.0007) [2023-12-27 02:53:55,641][105692] Updated weights for policy 0, policy_version 1579091 (0.0010) [2023-12-27 02:53:55,691][105620] Updated weights for policy 1, policy_version 1582700 (0.0008) [2023-12-27 02:53:55,702][105692] Updated weights for policy 0, policy_version 1579101 (0.0009) [2023-12-27 02:53:55,748][105620] Updated weights for policy 1, policy_version 1582710 (0.0005) [2023-12-27 02:53:55,762][105692] Updated weights for policy 0, policy_version 1579111 (0.0009) [2023-12-27 02:53:55,804][105620] Updated weights for policy 1, policy_version 1582720 (0.0007) [2023-12-27 02:53:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.1, 300 sec: 19466.4). Total num frames: 809541632. Throughput: 0: 9447.8, 1: 9787.3. Samples: 809549128. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:53:56,063][104569] Avg episode reward: [(0, '8345.091'), (1, '8997.449')] [2023-12-27 02:53:56,448][105620] Updated weights for policy 1, policy_version 1582730 (0.0006) [2023-12-27 02:53:56,494][105620] Updated weights for policy 1, policy_version 1582740 (0.0009) [2023-12-27 02:53:56,547][105620] Updated weights for policy 1, policy_version 1582750 (0.0008) [2023-12-27 02:53:56,562][105692] Updated weights for policy 0, policy_version 1579121 (0.0008) [2023-12-27 02:53:56,594][105620] Updated weights for policy 1, policy_version 1582760 (0.0005) [2023-12-27 02:53:56,622][105692] Updated weights for policy 0, policy_version 1579131 (0.0009) [2023-12-27 02:53:56,691][105692] Updated weights for policy 0, policy_version 1579141 (0.0010) [2023-12-27 02:53:57,311][105620] Updated weights for policy 1, policy_version 1582770 (0.0008) [2023-12-27 02:53:57,372][105620] Updated weights for policy 1, policy_version 1582780 (0.0009) [2023-12-27 02:53:57,427][105692] Updated weights for policy 0, policy_version 1579151 (0.0007) [2023-12-27 02:53:57,433][105620] Updated weights for policy 1, policy_version 1582790 (0.0008) [2023-12-27 02:53:57,494][105692] Updated weights for policy 0, policy_version 1579161 (0.0006) [2023-12-27 02:53:57,544][105692] Updated weights for policy 0, policy_version 1579171 (0.0009) [2023-12-27 02:53:58,163][105620] Updated weights for policy 1, policy_version 1582800 (0.0007) [2023-12-27 02:53:58,232][105620] Updated weights for policy 1, policy_version 1582810 (0.0006) [2023-12-27 02:53:58,259][105692] Updated weights for policy 0, policy_version 1579181 (0.0007) [2023-12-27 02:53:58,291][105620] Updated weights for policy 1, policy_version 1582820 (0.0007) [2023-12-27 02:53:58,319][105692] Updated weights for policy 0, policy_version 1579191 (0.0007) [2023-12-27 02:53:58,387][105692] Updated weights for policy 0, policy_version 1579201 (0.0008) [2023-12-27 02:53:59,042][105620] Updated weights for policy 1, policy_version 1582830 (0.0008) [2023-12-27 02:53:59,092][105620] Updated weights for policy 1, policy_version 1582840 (0.0009) [2023-12-27 02:53:59,146][105620] Updated weights for policy 1, policy_version 1582850 (0.0008) [2023-12-27 02:53:59,241][105692] Updated weights for policy 0, policy_version 1579211 (0.0009) [2023-12-27 02:53:59,301][105692] Updated weights for policy 0, policy_version 1579221 (0.0008) [2023-12-27 02:53:59,366][105692] Updated weights for policy 0, policy_version 1579231 (0.0010) [2023-12-27 02:53:59,958][105620] Updated weights for policy 1, policy_version 1582860 (0.0010) [2023-12-27 02:54:00,014][105620] Updated weights for policy 1, policy_version 1582870 (0.0010) [2023-12-27 02:54:00,070][105620] Updated weights for policy 1, policy_version 1582880 (0.0010) [2023-12-27 02:54:00,121][105692] Updated weights for policy 0, policy_version 1579241 (0.0010) [2023-12-27 02:54:00,180][105692] Updated weights for policy 0, policy_version 1579251 (0.0010) [2023-12-27 02:54:00,239][105692] Updated weights for policy 0, policy_version 1579261 (0.0010) [2023-12-27 02:54:00,305][105692] Updated weights for policy 0, policy_version 1579271 (0.0010) [2023-12-27 02:54:00,830][105620] Updated weights for policy 1, policy_version 1582890 (0.0011) [2023-12-27 02:54:00,889][105620] Updated weights for policy 1, policy_version 1582900 (0.0010) [2023-12-27 02:54:00,937][105620] Updated weights for policy 1, policy_version 1582910 (0.0010) [2023-12-27 02:54:00,937][105586] KL-divergence is very high: 117.4186 [2023-12-27 02:54:00,977][105586] KL-divergence is very high: 222.0784 [2023-12-27 02:54:00,989][105620] Updated weights for policy 1, policy_version 1582920 (0.0010) [2023-12-27 02:54:00,990][105692] Updated weights for policy 0, policy_version 1579281 (0.0006) [2023-12-27 02:54:01,050][105692] Updated weights for policy 0, policy_version 1579291 (0.0009) [2023-12-27 02:54:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 809631744. Throughput: 0: 9444.3, 1: 9789.5. Samples: 809605052. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:54:01,062][104569] Avg episode reward: [(0, '8351.026'), (1, '9178.585')] [2023-12-27 02:54:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001582920_405282816.pth... [2023-12-27 02:54:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001581768_404987904.pth [2023-12-27 02:54:01,111][105692] Updated weights for policy 0, policy_version 1579301 (0.0010) [2023-12-27 02:54:01,128][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001579304_404357120.pth... [2023-12-27 02:54:01,133][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001578184_404070400.pth [2023-12-27 02:54:01,790][105620] Updated weights for policy 1, policy_version 1582930 (0.0010) [2023-12-27 02:54:01,816][105692] Updated weights for policy 0, policy_version 1579311 (0.0008) [2023-12-27 02:54:01,841][105620] Updated weights for policy 1, policy_version 1582940 (0.0006) [2023-12-27 02:54:01,869][105692] Updated weights for policy 0, policy_version 1579321 (0.0008) [2023-12-27 02:54:01,890][105620] Updated weights for policy 1, policy_version 1582950 (0.0005) [2023-12-27 02:54:01,921][105692] Updated weights for policy 0, policy_version 1579331 (0.0009) [2023-12-27 02:54:02,527][105620] Updated weights for policy 1, policy_version 1582960 (0.0008) [2023-12-27 02:54:02,573][105620] Updated weights for policy 1, policy_version 1582970 (0.0008) [2023-12-27 02:54:02,626][105620] Updated weights for policy 1, policy_version 1582981 (0.0009) [2023-12-27 02:54:02,632][105692] Updated weights for policy 0, policy_version 1579341 (0.0008) [2023-12-27 02:54:02,678][105692] Updated weights for policy 0, policy_version 1579351 (0.0008) [2023-12-27 02:54:02,744][105692] Updated weights for policy 0, policy_version 1579361 (0.0009) [2023-12-27 02:54:03,317][105620] Updated weights for policy 1, policy_version 1582991 (0.0006) [2023-12-27 02:54:03,389][105620] Updated weights for policy 1, policy_version 1583001 (0.0006) [2023-12-27 02:54:03,413][105692] Updated weights for policy 0, policy_version 1579371 (0.0008) [2023-12-27 02:54:03,452][105620] Updated weights for policy 1, policy_version 1583011 (0.0008) [2023-12-27 02:54:03,482][105692] Updated weights for policy 0, policy_version 1579381 (0.0005) [2023-12-27 02:54:03,552][105692] Updated weights for policy 0, policy_version 1579391 (0.0005) [2023-12-27 02:54:04,016][105620] Updated weights for policy 1, policy_version 1583021 (0.0007) [2023-12-27 02:54:04,073][105620] Updated weights for policy 1, policy_version 1583031 (0.0008) [2023-12-27 02:54:04,129][105620] Updated weights for policy 1, policy_version 1583041 (0.0008) [2023-12-27 02:54:04,215][105692] Updated weights for policy 0, policy_version 1579401 (0.0005) [2023-12-27 02:54:04,284][105692] Updated weights for policy 0, policy_version 1579411 (0.0008) [2023-12-27 02:54:04,348][105692] Updated weights for policy 0, policy_version 1579421 (0.0008) [2023-12-27 02:54:04,414][105692] Updated weights for policy 0, policy_version 1579431 (0.0010) [2023-12-27 02:54:04,805][105620] Updated weights for policy 1, policy_version 1583051 (0.0008) [2023-12-27 02:54:04,863][105620] Updated weights for policy 1, policy_version 1583061 (0.0010) [2023-12-27 02:54:04,919][105620] Updated weights for policy 1, policy_version 1583071 (0.0005) [2023-12-27 02:54:05,268][105692] Updated weights for policy 0, policy_version 1579441 (0.0009) [2023-12-27 02:54:05,314][105692] Updated weights for policy 0, policy_version 1579451 (0.0008) [2023-12-27 02:54:05,358][105692] Updated weights for policy 0, policy_version 1579461 (0.0008) [2023-12-27 02:54:05,533][105620] Updated weights for policy 1, policy_version 1583081 (0.0006) [2023-12-27 02:54:05,578][105620] Updated weights for policy 1, policy_version 1583091 (0.0010) [2023-12-27 02:54:05,641][105620] Updated weights for policy 1, policy_version 1583101 (0.0011) [2023-12-27 02:54:05,696][105620] Updated weights for policy 1, policy_version 1583111 (0.0010) [2023-12-27 02:54:06,034][105692] Updated weights for policy 0, policy_version 1579471 (0.0007) [2023-12-27 02:54:06,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 809730048. Throughput: 0: 9467.4, 1: 9737.4. Samples: 809722148. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:54:06,062][104569] Avg episode reward: [(0, '8078.226'), (1, '8996.653')] [2023-12-27 02:54:06,097][105692] Updated weights for policy 0, policy_version 1579481 (0.0008) [2023-12-27 02:54:06,163][105692] Updated weights for policy 0, policy_version 1579491 (0.0008) [2023-12-27 02:54:06,429][105620] Updated weights for policy 1, policy_version 1583121 (0.0006) [2023-12-27 02:54:06,493][105620] Updated weights for policy 1, policy_version 1583131 (0.0008) [2023-12-27 02:54:06,546][105620] Updated weights for policy 1, policy_version 1583141 (0.0008) [2023-12-27 02:54:07,014][105692] Updated weights for policy 0, policy_version 1579501 (0.0009) [2023-12-27 02:54:07,079][105692] Updated weights for policy 0, policy_version 1579511 (0.0010) [2023-12-27 02:54:07,143][105620] Updated weights for policy 1, policy_version 1583151 (0.0006) [2023-12-27 02:54:07,144][105692] Updated weights for policy 0, policy_version 1579521 (0.0008) [2023-12-27 02:54:07,200][105620] Updated weights for policy 1, policy_version 1583161 (0.0005) [2023-12-27 02:54:07,258][105620] Updated weights for policy 1, policy_version 1583171 (0.0008) [2023-12-27 02:54:07,744][105692] Updated weights for policy 0, policy_version 1579531 (0.0007) [2023-12-27 02:54:07,804][105692] Updated weights for policy 0, policy_version 1579541 (0.0009) [2023-12-27 02:54:07,851][105692] Updated weights for policy 0, policy_version 1579551 (0.0009) [2023-12-27 02:54:07,924][105620] Updated weights for policy 1, policy_version 1583181 (0.0009) [2023-12-27 02:54:07,979][105620] Updated weights for policy 1, policy_version 1583191 (0.0009) [2023-12-27 02:54:08,042][105620] Updated weights for policy 1, policy_version 1583201 (0.0009) [2023-12-27 02:54:08,505][105692] Updated weights for policy 0, policy_version 1579561 (0.0010) [2023-12-27 02:54:08,572][105692] Updated weights for policy 0, policy_version 1579571 (0.0009) [2023-12-27 02:54:08,641][105692] Updated weights for policy 0, policy_version 1579581 (0.0009) [2023-12-27 02:54:08,706][105692] Updated weights for policy 0, policy_version 1579591 (0.0007) [2023-12-27 02:54:08,857][105620] Updated weights for policy 1, policy_version 1583211 (0.0008) [2023-12-27 02:54:08,922][105620] Updated weights for policy 1, policy_version 1583221 (0.0009) [2023-12-27 02:54:08,976][105620] Updated weights for policy 1, policy_version 1583231 (0.0009) [2023-12-27 02:54:09,387][105692] Updated weights for policy 0, policy_version 1579601 (0.0009) [2023-12-27 02:54:09,457][105692] Updated weights for policy 0, policy_version 1579611 (0.0009) [2023-12-27 02:54:09,519][105692] Updated weights for policy 0, policy_version 1579621 (0.0009) [2023-12-27 02:54:09,776][105620] Updated weights for policy 1, policy_version 1583241 (0.0009) [2023-12-27 02:54:09,841][105620] Updated weights for policy 1, policy_version 1583251 (0.0007) [2023-12-27 02:54:09,908][105620] Updated weights for policy 1, policy_version 1583261 (0.0008) [2023-12-27 02:54:09,973][105620] Updated weights for policy 1, policy_version 1583271 (0.0006) [2023-12-27 02:54:10,284][105692] Updated weights for policy 0, policy_version 1579631 (0.0009) [2023-12-27 02:54:10,346][105692] Updated weights for policy 0, policy_version 1579641 (0.0009) [2023-12-27 02:54:10,404][105692] Updated weights for policy 0, policy_version 1579651 (0.0008) [2023-12-27 02:54:10,653][105620] Updated weights for policy 1, policy_version 1583281 (0.0006) [2023-12-27 02:54:10,716][105620] Updated weights for policy 1, policy_version 1583291 (0.0007) [2023-12-27 02:54:10,770][105620] Updated weights for policy 1, policy_version 1583301 (0.0009) [2023-12-27 02:54:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 809828352. Throughput: 0: 9493.7, 1: 9785.3. Samples: 809838664. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:54:11,063][104569] Avg episode reward: [(0, '7991.000'), (1, '8991.450')] [2023-12-27 02:54:11,094][105692] Updated weights for policy 0, policy_version 1579661 (0.0009) [2023-12-27 02:54:11,168][105692] Updated weights for policy 0, policy_version 1579671 (0.0007) [2023-12-27 02:54:11,231][105692] Updated weights for policy 0, policy_version 1579681 (0.0007) [2023-12-27 02:54:11,474][105620] Updated weights for policy 1, policy_version 1583311 (0.0007) [2023-12-27 02:54:11,536][105620] Updated weights for policy 1, policy_version 1583321 (0.0006) [2023-12-27 02:54:11,600][105620] Updated weights for policy 1, policy_version 1583331 (0.0008) [2023-12-27 02:54:11,955][105692] Updated weights for policy 0, policy_version 1579691 (0.0008) [2023-12-27 02:54:12,004][105692] Updated weights for policy 0, policy_version 1579701 (0.0010) [2023-12-27 02:54:12,060][105692] Updated weights for policy 0, policy_version 1579711 (0.0010) [2023-12-27 02:54:12,297][105620] Updated weights for policy 1, policy_version 1583341 (0.0008) [2023-12-27 02:54:12,365][105620] Updated weights for policy 1, policy_version 1583351 (0.0009) [2023-12-27 02:54:12,428][105620] Updated weights for policy 1, policy_version 1583361 (0.0007) [2023-12-27 02:54:12,842][105692] Updated weights for policy 0, policy_version 1579721 (0.0010) [2023-12-27 02:54:12,900][105692] Updated weights for policy 0, policy_version 1579731 (0.0010) [2023-12-27 02:54:12,962][105692] Updated weights for policy 0, policy_version 1579741 (0.0011) [2023-12-27 02:54:13,024][105692] Updated weights for policy 0, policy_version 1579751 (0.0011) [2023-12-27 02:54:13,139][105620] Updated weights for policy 1, policy_version 1583371 (0.0008) [2023-12-27 02:54:13,187][105620] Updated weights for policy 1, policy_version 1583381 (0.0008) [2023-12-27 02:54:13,240][105620] Updated weights for policy 1, policy_version 1583391 (0.0008) [2023-12-27 02:54:13,737][105692] Updated weights for policy 0, policy_version 1579761 (0.0011) [2023-12-27 02:54:13,803][105692] Updated weights for policy 0, policy_version 1579771 (0.0011) [2023-12-27 02:54:13,855][105692] Updated weights for policy 0, policy_version 1579781 (0.0008) [2023-12-27 02:54:13,987][105620] Updated weights for policy 1, policy_version 1583401 (0.0008) [2023-12-27 02:54:14,039][105620] Updated weights for policy 1, policy_version 1583411 (0.0009) [2023-12-27 02:54:14,091][105620] Updated weights for policy 1, policy_version 1583421 (0.0010) [2023-12-27 02:54:14,154][105620] Updated weights for policy 1, policy_version 1583431 (0.0011) [2023-12-27 02:54:14,462][105692] Updated weights for policy 0, policy_version 1579791 (0.0010) [2023-12-27 02:54:14,520][105692] Updated weights for policy 0, policy_version 1579801 (0.0010) [2023-12-27 02:54:14,585][105692] Updated weights for policy 0, policy_version 1579811 (0.0010) [2023-12-27 02:54:14,775][105620] Updated weights for policy 1, policy_version 1583441 (0.0010) [2023-12-27 02:54:14,837][105620] Updated weights for policy 1, policy_version 1583451 (0.0009) [2023-12-27 02:54:14,903][105620] Updated weights for policy 1, policy_version 1583461 (0.0005) [2023-12-27 02:54:15,333][105692] Updated weights for policy 0, policy_version 1579821 (0.0009) [2023-12-27 02:54:15,397][105692] Updated weights for policy 0, policy_version 1579831 (0.0008) [2023-12-27 02:54:15,452][105692] Updated weights for policy 0, policy_version 1579841 (0.0008) [2023-12-27 02:54:15,609][105620] Updated weights for policy 1, policy_version 1583471 (0.0005) [2023-12-27 02:54:15,665][105620] Updated weights for policy 1, policy_version 1583481 (0.0006) [2023-12-27 02:54:15,714][105620] Updated weights for policy 1, policy_version 1583491 (0.0005) [2023-12-27 02:54:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 809926656. Throughput: 0: 9501.5, 1: 9806.3. Samples: 809896636. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:54:16,063][104569] Avg episode reward: [(0, '8080.917'), (1, '8990.908')] [2023-12-27 02:54:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001583496_405430272.pth... [2023-12-27 02:54:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001579848_404496384.pth... [2023-12-27 02:54:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001582344_405135360.pth [2023-12-27 02:54:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001578728_404209664.pth [2023-12-27 02:54:16,245][105620] Updated weights for policy 1, policy_version 1583501 (0.0005) [2023-12-27 02:54:16,260][105692] Updated weights for policy 0, policy_version 1579851 (0.0009) [2023-12-27 02:54:16,311][105620] Updated weights for policy 1, policy_version 1583511 (0.0005) [2023-12-27 02:54:16,319][105692] Updated weights for policy 0, policy_version 1579861 (0.0010) [2023-12-27 02:54:16,370][105620] Updated weights for policy 1, policy_version 1583521 (0.0005) [2023-12-27 02:54:16,381][105692] Updated weights for policy 0, policy_version 1579871 (0.0010) [2023-12-27 02:54:16,859][105620] Updated weights for policy 1, policy_version 1583531 (0.0007) [2023-12-27 02:54:16,921][105620] Updated weights for policy 1, policy_version 1583541 (0.0010) [2023-12-27 02:54:16,971][105620] Updated weights for policy 1, policy_version 1583551 (0.0010) [2023-12-27 02:54:17,122][105692] Updated weights for policy 0, policy_version 1579881 (0.0010) [2023-12-27 02:54:17,183][105692] Updated weights for policy 0, policy_version 1579891 (0.0010) [2023-12-27 02:54:17,236][105692] Updated weights for policy 0, policy_version 1579901 (0.0006) [2023-12-27 02:54:17,290][105692] Updated weights for policy 0, policy_version 1579911 (0.0006) [2023-12-27 02:54:17,720][105620] Updated weights for policy 1, policy_version 1583561 (0.0010) [2023-12-27 02:54:17,774][105620] Updated weights for policy 1, policy_version 1583571 (0.0010) [2023-12-27 02:54:17,822][105620] Updated weights for policy 1, policy_version 1583581 (0.0010) [2023-12-27 02:54:17,874][105620] Updated weights for policy 1, policy_version 1583591 (0.0010) [2023-12-27 02:54:17,895][105692] Updated weights for policy 0, policy_version 1579921 (0.0006) [2023-12-27 02:54:17,949][105692] Updated weights for policy 0, policy_version 1579931 (0.0007) [2023-12-27 02:54:18,015][105692] Updated weights for policy 0, policy_version 1579941 (0.0007) [2023-12-27 02:54:18,515][105620] Updated weights for policy 1, policy_version 1583601 (0.0006) [2023-12-27 02:54:18,568][105620] Updated weights for policy 1, policy_version 1583611 (0.0006) [2023-12-27 02:54:18,629][105620] Updated weights for policy 1, policy_version 1583621 (0.0006) [2023-12-27 02:54:18,726][105692] Updated weights for policy 0, policy_version 1579951 (0.0011) [2023-12-27 02:54:18,781][105692] Updated weights for policy 0, policy_version 1579961 (0.0010) [2023-12-27 02:54:18,843][105692] Updated weights for policy 0, policy_version 1579971 (0.0010) [2023-12-27 02:54:19,353][105620] Updated weights for policy 1, policy_version 1583631 (0.0007) [2023-12-27 02:54:19,414][105620] Updated weights for policy 1, policy_version 1583641 (0.0008) [2023-12-27 02:54:19,467][105620] Updated weights for policy 1, policy_version 1583651 (0.0009) [2023-12-27 02:54:19,596][105692] Updated weights for policy 0, policy_version 1579981 (0.0009) [2023-12-27 02:54:19,651][105692] Updated weights for policy 0, policy_version 1579991 (0.0010) [2023-12-27 02:54:19,710][105692] Updated weights for policy 0, policy_version 1580001 (0.0009) [2023-12-27 02:54:20,171][105620] Updated weights for policy 1, policy_version 1583661 (0.0009) [2023-12-27 02:54:20,228][105620] Updated weights for policy 1, policy_version 1583671 (0.0009) [2023-12-27 02:54:20,282][105620] Updated weights for policy 1, policy_version 1583681 (0.0009) [2023-12-27 02:54:20,432][105692] Updated weights for policy 0, policy_version 1580011 (0.0009) [2023-12-27 02:54:20,484][105692] Updated weights for policy 0, policy_version 1580021 (0.0009) [2023-12-27 02:54:20,540][105692] Updated weights for policy 0, policy_version 1580031 (0.0009) [2023-12-27 02:54:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 810024960. Throughput: 0: 9555.8, 1: 9959.5. Samples: 810019212. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:54:21,063][104569] Avg episode reward: [(0, '8168.725'), (1, '8997.428')] [2023-12-27 02:54:21,082][105620] Updated weights for policy 1, policy_version 1583691 (0.0009) [2023-12-27 02:54:21,145][105620] Updated weights for policy 1, policy_version 1583701 (0.0008) [2023-12-27 02:54:21,204][105620] Updated weights for policy 1, policy_version 1583711 (0.0008) [2023-12-27 02:54:21,395][105692] Updated weights for policy 0, policy_version 1580041 (0.0007) [2023-12-27 02:54:21,460][105692] Updated weights for policy 0, policy_version 1580051 (0.0008) [2023-12-27 02:54:21,520][105692] Updated weights for policy 0, policy_version 1580061 (0.0008) [2023-12-27 02:54:21,581][105692] Updated weights for policy 0, policy_version 1580071 (0.0010) [2023-12-27 02:54:21,972][105620] Updated weights for policy 1, policy_version 1583721 (0.0008) [2023-12-27 02:54:22,028][105620] Updated weights for policy 1, policy_version 1583731 (0.0009) [2023-12-27 02:54:22,086][105620] Updated weights for policy 1, policy_version 1583741 (0.0008) [2023-12-27 02:54:22,143][105620] Updated weights for policy 1, policy_version 1583751 (0.0008) [2023-12-27 02:54:22,390][105692] Updated weights for policy 0, policy_version 1580081 (0.0009) [2023-12-27 02:54:22,447][105692] Updated weights for policy 0, policy_version 1580091 (0.0009) [2023-12-27 02:54:22,514][105692] Updated weights for policy 0, policy_version 1580101 (0.0010) [2023-12-27 02:54:22,853][105620] Updated weights for policy 1, policy_version 1583761 (0.0006) [2023-12-27 02:54:22,927][105620] Updated weights for policy 1, policy_version 1583771 (0.0006) [2023-12-27 02:54:22,989][105620] Updated weights for policy 1, policy_version 1583781 (0.0007) [2023-12-27 02:54:23,273][105692] Updated weights for policy 0, policy_version 1580112 (0.0009) [2023-12-27 02:54:23,340][105692] Updated weights for policy 0, policy_version 1580122 (0.0008) [2023-12-27 02:54:23,409][105692] Updated weights for policy 0, policy_version 1580132 (0.0009) [2023-12-27 02:54:23,585][105620] Updated weights for policy 1, policy_version 1583791 (0.0008) [2023-12-27 02:54:23,633][105620] Updated weights for policy 1, policy_version 1583801 (0.0009) [2023-12-27 02:54:23,677][105620] Updated weights for policy 1, policy_version 1583811 (0.0010) [2023-12-27 02:54:24,150][105692] Updated weights for policy 0, policy_version 1580142 (0.0007) [2023-12-27 02:54:24,213][105692] Updated weights for policy 0, policy_version 1580152 (0.0010) [2023-12-27 02:54:24,275][105692] Updated weights for policy 0, policy_version 1580162 (0.0010) [2023-12-27 02:54:24,345][105620] Updated weights for policy 1, policy_version 1583821 (0.0008) [2023-12-27 02:54:24,402][105620] Updated weights for policy 1, policy_version 1583831 (0.0006) [2023-12-27 02:54:24,457][105620] Updated weights for policy 1, policy_version 1583841 (0.0010) [2023-12-27 02:54:25,106][105620] Updated weights for policy 1, policy_version 1583851 (0.0009) [2023-12-27 02:54:25,117][105692] Updated weights for policy 0, policy_version 1580172 (0.0010) [2023-12-27 02:54:25,159][105620] Updated weights for policy 1, policy_version 1583861 (0.0006) [2023-12-27 02:54:25,169][105692] Updated weights for policy 0, policy_version 1580182 (0.0008) [2023-12-27 02:54:25,217][105620] Updated weights for policy 1, policy_version 1583871 (0.0008) [2023-12-27 02:54:25,233][105692] Updated weights for policy 0, policy_version 1580192 (0.0006) [2023-12-27 02:54:25,787][105620] Updated weights for policy 1, policy_version 1583881 (0.0006) [2023-12-27 02:54:25,851][105620] Updated weights for policy 1, policy_version 1583891 (0.0010) [2023-12-27 02:54:25,936][105620] Updated weights for policy 1, policy_version 1583901 (0.0010) [2023-12-27 02:54:25,999][105620] Updated weights for policy 1, policy_version 1583911 (0.0010) [2023-12-27 02:54:26,055][105692] Updated weights for policy 0, policy_version 1580202 (0.0009) [2023-12-27 02:54:26,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.8, 300 sec: 19438.7). Total num frames: 810123264. Throughput: 0: 9517.5, 1: 10018.2. Samples: 810132832. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:54:26,062][104569] Avg episode reward: [(0, '7806.116'), (1, '8997.543')] [2023-12-27 02:54:26,103][105692] Updated weights for policy 0, policy_version 1580212 (0.0008) [2023-12-27 02:54:26,150][105692] Updated weights for policy 0, policy_version 1580222 (0.0008) [2023-12-27 02:54:26,209][105692] Updated weights for policy 0, policy_version 1580232 (0.0008) [2023-12-27 02:54:26,653][105620] Updated weights for policy 1, policy_version 1583921 (0.0006) [2023-12-27 02:54:26,698][105620] Updated weights for policy 1, policy_version 1583931 (0.0005) [2023-12-27 02:54:26,743][105620] Updated weights for policy 1, policy_version 1583941 (0.0005) [2023-12-27 02:54:27,057][105692] Updated weights for policy 0, policy_version 1580242 (0.0009) [2023-12-27 02:54:27,105][105692] Updated weights for policy 0, policy_version 1580252 (0.0009) [2023-12-27 02:54:27,149][105692] Updated weights for policy 0, policy_version 1580262 (0.0008) [2023-12-27 02:54:27,310][105620] Updated weights for policy 1, policy_version 1583951 (0.0009) [2023-12-27 02:54:27,369][105620] Updated weights for policy 1, policy_version 1583961 (0.0010) [2023-12-27 02:54:27,422][105620] Updated weights for policy 1, policy_version 1583971 (0.0010) [2023-12-27 02:54:27,941][105692] Updated weights for policy 0, policy_version 1580272 (0.0008) [2023-12-27 02:54:27,995][105692] Updated weights for policy 0, policy_version 1580282 (0.0008) [2023-12-27 02:54:28,050][105692] Updated weights for policy 0, policy_version 1580292 (0.0008) [2023-12-27 02:54:28,166][105620] Updated weights for policy 1, policy_version 1583981 (0.0008) [2023-12-27 02:54:28,223][105620] Updated weights for policy 1, policy_version 1583991 (0.0010) [2023-12-27 02:54:28,279][105620] Updated weights for policy 1, policy_version 1584001 (0.0009) [2023-12-27 02:54:28,840][105692] Updated weights for policy 0, policy_version 1580302 (0.0009) [2023-12-27 02:54:28,896][105692] Updated weights for policy 0, policy_version 1580312 (0.0013) [2023-12-27 02:54:28,901][105620] Updated weights for policy 1, policy_version 1584011 (0.0009) [2023-12-27 02:54:28,947][105692] Updated weights for policy 0, policy_version 1580322 (0.0009) [2023-12-27 02:54:28,956][105620] Updated weights for policy 1, policy_version 1584021 (0.0005) [2023-12-27 02:54:29,003][105620] Updated weights for policy 1, policy_version 1584031 (0.0006) [2023-12-27 02:54:29,633][105620] Updated weights for policy 1, policy_version 1584041 (0.0006) [2023-12-27 02:54:29,696][105620] Updated weights for policy 1, policy_version 1584051 (0.0011) [2023-12-27 02:54:29,759][105620] Updated weights for policy 1, policy_version 1584061 (0.0011) [2023-12-27 02:54:29,806][105692] Updated weights for policy 0, policy_version 1580332 (0.0009) [2023-12-27 02:54:29,825][105620] Updated weights for policy 1, policy_version 1584071 (0.0011) [2023-12-27 02:54:29,865][105692] Updated weights for policy 0, policy_version 1580342 (0.0008) [2023-12-27 02:54:29,921][105692] Updated weights for policy 0, policy_version 1580352 (0.0008) [2023-12-27 02:54:30,579][105620] Updated weights for policy 1, policy_version 1584081 (0.0010) [2023-12-27 02:54:30,627][105620] Updated weights for policy 1, policy_version 1584091 (0.0009) [2023-12-27 02:54:30,639][105692] Updated weights for policy 0, policy_version 1580362 (0.0008) [2023-12-27 02:54:30,678][105620] Updated weights for policy 1, policy_version 1584101 (0.0008) [2023-12-27 02:54:30,691][105692] Updated weights for policy 0, policy_version 1580372 (0.0007) [2023-12-27 02:54:30,743][105692] Updated weights for policy 0, policy_version 1580382 (0.0009) [2023-12-27 02:54:30,804][105692] Updated weights for policy 0, policy_version 1580392 (0.0010) [2023-12-27 02:54:31,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.6, 300 sec: 19383.1). Total num frames: 810221568. Throughput: 0: 9431.2, 1: 10088.7. Samples: 810191636. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:54:31,063][104569] Avg episode reward: [(0, '8258.343'), (1, '8990.628')] [2023-12-27 02:54:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001580392_404635648.pth... [2023-12-27 02:54:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001584104_405585920.pth... [2023-12-27 02:54:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001579304_404357120.pth [2023-12-27 02:54:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001582920_405282816.pth [2023-12-27 02:54:31,328][105620] Updated weights for policy 1, policy_version 1584111 (0.0006) [2023-12-27 02:54:31,386][105620] Updated weights for policy 1, policy_version 1584121 (0.0011) [2023-12-27 02:54:31,448][105620] Updated weights for policy 1, policy_version 1584131 (0.0011) [2023-12-27 02:54:31,562][105692] Updated weights for policy 0, policy_version 1580402 (0.0008) [2023-12-27 02:54:31,618][105692] Updated weights for policy 0, policy_version 1580412 (0.0008) [2023-12-27 02:54:31,677][105692] Updated weights for policy 0, policy_version 1580422 (0.0008) [2023-12-27 02:54:32,235][105620] Updated weights for policy 1, policy_version 1584141 (0.0009) [2023-12-27 02:54:32,293][105620] Updated weights for policy 1, policy_version 1584151 (0.0008) [2023-12-27 02:54:32,349][105620] Updated weights for policy 1, policy_version 1584161 (0.0007) [2023-12-27 02:54:32,358][105692] Updated weights for policy 0, policy_version 1580432 (0.0007) [2023-12-27 02:54:32,415][105692] Updated weights for policy 0, policy_version 1580442 (0.0008) [2023-12-27 02:54:32,469][105692] Updated weights for policy 0, policy_version 1580452 (0.0006) [2023-12-27 02:54:33,081][105620] Updated weights for policy 1, policy_version 1584171 (0.0006) [2023-12-27 02:54:33,134][105620] Updated weights for policy 1, policy_version 1584181 (0.0006) [2023-12-27 02:54:33,181][105620] Updated weights for policy 1, policy_version 1584191 (0.0008) [2023-12-27 02:54:33,225][105692] Updated weights for policy 0, policy_version 1580462 (0.0008) [2023-12-27 02:54:33,276][105692] Updated weights for policy 0, policy_version 1580472 (0.0009) [2023-12-27 02:54:33,326][105692] Updated weights for policy 0, policy_version 1580482 (0.0009) [2023-12-27 02:54:33,798][105620] Updated weights for policy 1, policy_version 1584201 (0.0007) [2023-12-27 02:54:33,850][105620] Updated weights for policy 1, policy_version 1584211 (0.0005) [2023-12-27 02:54:33,896][105620] Updated weights for policy 1, policy_version 1584221 (0.0009) [2023-12-27 02:54:33,949][105620] Updated weights for policy 1, policy_version 1584231 (0.0009) [2023-12-27 02:54:34,137][105692] Updated weights for policy 0, policy_version 1580492 (0.0008) [2023-12-27 02:54:34,200][105692] Updated weights for policy 0, policy_version 1580502 (0.0007) [2023-12-27 02:54:34,262][105692] Updated weights for policy 0, policy_version 1580512 (0.0006) [2023-12-27 02:54:34,648][105620] Updated weights for policy 1, policy_version 1584241 (0.0010) [2023-12-27 02:54:34,712][105620] Updated weights for policy 1, policy_version 1584251 (0.0011) [2023-12-27 02:54:34,765][105620] Updated weights for policy 1, policy_version 1584261 (0.0011) [2023-12-27 02:54:34,955][105692] Updated weights for policy 0, policy_version 1580522 (0.0006) [2023-12-27 02:54:35,003][105692] Updated weights for policy 0, policy_version 1580532 (0.0007) [2023-12-27 02:54:35,066][105692] Updated weights for policy 0, policy_version 1580542 (0.0008) [2023-12-27 02:54:35,114][105692] Updated weights for policy 0, policy_version 1580552 (0.0008) [2023-12-27 02:54:35,520][105620] Updated weights for policy 1, policy_version 1584271 (0.0007) [2023-12-27 02:54:35,575][105620] Updated weights for policy 1, policy_version 1584281 (0.0007) [2023-12-27 02:54:35,627][105620] Updated weights for policy 1, policy_version 1584291 (0.0011) [2023-12-27 02:54:35,891][105692] Updated weights for policy 0, policy_version 1580562 (0.0006) [2023-12-27 02:54:35,943][105692] Updated weights for policy 0, policy_version 1580572 (0.0005) [2023-12-27 02:54:35,999][105692] Updated weights for policy 0, policy_version 1580582 (0.0009) [2023-12-27 02:54:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 810319872. Throughput: 0: 9337.7, 1: 10067.6. Samples: 810307388. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:54:36,062][104569] Avg episode reward: [(0, '8527.244'), (1, '9175.977')] [2023-12-27 02:54:36,373][105620] Updated weights for policy 1, policy_version 1584301 (0.0011) [2023-12-27 02:54:36,436][105620] Updated weights for policy 1, policy_version 1584311 (0.0011) [2023-12-27 02:54:36,502][105620] Updated weights for policy 1, policy_version 1584321 (0.0009) [2023-12-27 02:54:36,707][105692] Updated weights for policy 0, policy_version 1580592 (0.0008) [2023-12-27 02:54:36,768][105692] Updated weights for policy 0, policy_version 1580602 (0.0008) [2023-12-27 02:54:36,828][105692] Updated weights for policy 0, policy_version 1580612 (0.0008) [2023-12-27 02:54:37,252][105620] Updated weights for policy 1, policy_version 1584331 (0.0011) [2023-12-27 02:54:37,317][105620] Updated weights for policy 1, policy_version 1584341 (0.0011) [2023-12-27 02:54:37,379][105620] Updated weights for policy 1, policy_version 1584351 (0.0010) [2023-12-27 02:54:37,545][105692] Updated weights for policy 0, policy_version 1580622 (0.0009) [2023-12-27 02:54:37,611][105692] Updated weights for policy 0, policy_version 1580632 (0.0010) [2023-12-27 02:54:37,673][105692] Updated weights for policy 0, policy_version 1580642 (0.0010) [2023-12-27 02:54:38,049][105620] Updated weights for policy 1, policy_version 1584361 (0.0006) [2023-12-27 02:54:38,106][105620] Updated weights for policy 1, policy_version 1584371 (0.0010) [2023-12-27 02:54:38,167][105620] Updated weights for policy 1, policy_version 1584381 (0.0010) [2023-12-27 02:54:38,225][105620] Updated weights for policy 1, policy_version 1584391 (0.0010) [2023-12-27 02:54:38,406][105692] Updated weights for policy 0, policy_version 1580652 (0.0011) [2023-12-27 02:54:38,468][105692] Updated weights for policy 0, policy_version 1580662 (0.0010) [2023-12-27 02:54:38,541][105692] Updated weights for policy 0, policy_version 1580672 (0.0010) [2023-12-27 02:54:38,937][105620] Updated weights for policy 1, policy_version 1584401 (0.0007) [2023-12-27 02:54:38,991][105620] Updated weights for policy 1, policy_version 1584411 (0.0007) [2023-12-27 02:54:39,044][105620] Updated weights for policy 1, policy_version 1584421 (0.0006) [2023-12-27 02:54:39,146][105692] Updated weights for policy 0, policy_version 1580682 (0.0007) [2023-12-27 02:54:39,210][105692] Updated weights for policy 0, policy_version 1580692 (0.0006) [2023-12-27 02:54:39,275][105692] Updated weights for policy 0, policy_version 1580702 (0.0007) [2023-12-27 02:54:39,337][105692] Updated weights for policy 0, policy_version 1580712 (0.0009) [2023-12-27 02:54:39,780][105620] Updated weights for policy 1, policy_version 1584431 (0.0007) [2023-12-27 02:54:39,844][105620] Updated weights for policy 1, policy_version 1584441 (0.0008) [2023-12-27 02:54:39,908][105620] Updated weights for policy 1, policy_version 1584451 (0.0008) [2023-12-27 02:54:40,012][105692] Updated weights for policy 0, policy_version 1580722 (0.0010) [2023-12-27 02:54:40,080][105692] Updated weights for policy 0, policy_version 1580732 (0.0010) [2023-12-27 02:54:40,146][105692] Updated weights for policy 0, policy_version 1580742 (0.0011) [2023-12-27 02:54:40,697][105620] Updated weights for policy 1, policy_version 1584461 (0.0009) [2023-12-27 02:54:40,759][105620] Updated weights for policy 1, policy_version 1584471 (0.0010) [2023-12-27 02:54:40,820][105620] Updated weights for policy 1, policy_version 1584481 (0.0010) [2023-12-27 02:54:40,888][105692] Updated weights for policy 0, policy_version 1580752 (0.0010) [2023-12-27 02:54:40,939][105692] Updated weights for policy 0, policy_version 1580762 (0.0010) [2023-12-27 02:54:40,988][105692] Updated weights for policy 0, policy_version 1580772 (0.0010) [2023-12-27 02:54:41,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 810418176. Throughput: 0: 9409.3, 1: 9995.5. Samples: 810422340. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:54:41,063][104569] Avg episode reward: [(0, '8072.199'), (1, '9176.754')] [2023-12-27 02:54:41,512][105620] Updated weights for policy 1, policy_version 1584491 (0.0010) [2023-12-27 02:54:41,575][105620] Updated weights for policy 1, policy_version 1584501 (0.0010) [2023-12-27 02:54:41,645][105620] Updated weights for policy 1, policy_version 1584511 (0.0009) [2023-12-27 02:54:41,712][105692] Updated weights for policy 0, policy_version 1580782 (0.0009) [2023-12-27 02:54:41,778][105692] Updated weights for policy 0, policy_version 1580792 (0.0008) [2023-12-27 02:54:41,829][105692] Updated weights for policy 0, policy_version 1580802 (0.0010) [2023-12-27 02:54:42,365][105620] Updated weights for policy 1, policy_version 1584521 (0.0008) [2023-12-27 02:54:42,430][105620] Updated weights for policy 1, policy_version 1584531 (0.0006) [2023-12-27 02:54:42,488][105620] Updated weights for policy 1, policy_version 1584541 (0.0008) [2023-12-27 02:54:42,532][105692] Updated weights for policy 0, policy_version 1580812 (0.0009) [2023-12-27 02:54:42,544][105620] Updated weights for policy 1, policy_version 1584551 (0.0010) [2023-12-27 02:54:42,594][105692] Updated weights for policy 0, policy_version 1580822 (0.0006) [2023-12-27 02:54:42,659][105692] Updated weights for policy 0, policy_version 1580832 (0.0008) [2023-12-27 02:54:43,164][105620] Updated weights for policy 1, policy_version 1584561 (0.0006) [2023-12-27 02:54:43,215][105620] Updated weights for policy 1, policy_version 1584571 (0.0010) [2023-12-27 02:54:43,264][105620] Updated weights for policy 1, policy_version 1584581 (0.0010) [2023-12-27 02:54:43,290][105692] Updated weights for policy 0, policy_version 1580842 (0.0008) [2023-12-27 02:54:43,343][105692] Updated weights for policy 0, policy_version 1580852 (0.0005) [2023-12-27 02:54:43,397][105692] Updated weights for policy 0, policy_version 1580862 (0.0005) [2023-12-27 02:54:43,442][105692] Updated weights for policy 0, policy_version 1580872 (0.0005) [2023-12-27 02:54:43,991][105620] Updated weights for policy 1, policy_version 1584591 (0.0010) [2023-12-27 02:54:44,045][105620] Updated weights for policy 1, policy_version 1584601 (0.0010) [2023-12-27 02:54:44,056][105692] Updated weights for policy 0, policy_version 1580882 (0.0008) [2023-12-27 02:54:44,100][105620] Updated weights for policy 1, policy_version 1584611 (0.0010) [2023-12-27 02:54:44,112][105692] Updated weights for policy 0, policy_version 1580892 (0.0006) [2023-12-27 02:54:44,169][105692] Updated weights for policy 0, policy_version 1580902 (0.0010) [2023-12-27 02:54:44,764][105620] Updated weights for policy 1, policy_version 1584621 (0.0007) [2023-12-27 02:54:44,818][105620] Updated weights for policy 1, policy_version 1584631 (0.0007) [2023-12-27 02:54:44,863][105692] Updated weights for policy 0, policy_version 1580912 (0.0009) [2023-12-27 02:54:44,880][105620] Updated weights for policy 1, policy_version 1584641 (0.0011) [2023-12-27 02:54:44,923][105692] Updated weights for policy 0, policy_version 1580922 (0.0008) [2023-12-27 02:54:44,983][105692] Updated weights for policy 0, policy_version 1580932 (0.0008) [2023-12-27 02:54:45,532][105620] Updated weights for policy 1, policy_version 1584651 (0.0010) [2023-12-27 02:54:45,587][105620] Updated weights for policy 1, policy_version 1584661 (0.0009) [2023-12-27 02:54:45,641][105620] Updated weights for policy 1, policy_version 1584671 (0.0008) [2023-12-27 02:54:45,792][105692] Updated weights for policy 0, policy_version 1580942 (0.0009) [2023-12-27 02:54:45,852][105692] Updated weights for policy 0, policy_version 1580952 (0.0009) [2023-12-27 02:54:45,908][105692] Updated weights for policy 0, policy_version 1580962 (0.0008) [2023-12-27 02:54:46,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 810516480. Throughput: 0: 9484.3, 1: 10028.1. Samples: 810483108. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:54:46,062][104569] Avg episode reward: [(0, '8352.275'), (1, '8998.610')] [2023-12-27 02:54:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001584680_405733376.pth... [2023-12-27 02:54:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001580968_404783104.pth... [2023-12-27 02:54:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001583496_405430272.pth [2023-12-27 02:54:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001579848_404496384.pth [2023-12-27 02:54:46,365][105620] Updated weights for policy 1, policy_version 1584681 (0.0006) [2023-12-27 02:54:46,429][105620] Updated weights for policy 1, policy_version 1584691 (0.0009) [2023-12-27 02:54:46,491][105620] Updated weights for policy 1, policy_version 1584701 (0.0008) [2023-12-27 02:54:46,533][105692] Updated weights for policy 0, policy_version 1580972 (0.0007) [2023-12-27 02:54:46,542][105620] Updated weights for policy 1, policy_version 1584711 (0.0006) [2023-12-27 02:54:46,581][105692] Updated weights for policy 0, policy_version 1580982 (0.0007) [2023-12-27 02:54:46,634][105692] Updated weights for policy 0, policy_version 1580992 (0.0009) [2023-12-27 02:54:47,119][105620] Updated weights for policy 1, policy_version 1584721 (0.0005) [2023-12-27 02:54:47,178][105620] Updated weights for policy 1, policy_version 1584731 (0.0005) [2023-12-27 02:54:47,245][105620] Updated weights for policy 1, policy_version 1584741 (0.0005) [2023-12-27 02:54:47,339][105692] Updated weights for policy 0, policy_version 1581002 (0.0009) [2023-12-27 02:54:47,384][105692] Updated weights for policy 0, policy_version 1581012 (0.0006) [2023-12-27 02:54:47,444][105692] Updated weights for policy 0, policy_version 1581022 (0.0008) [2023-12-27 02:54:47,745][105620] Updated weights for policy 1, policy_version 1584751 (0.0005) [2023-12-27 02:54:47,796][105620] Updated weights for policy 1, policy_version 1584761 (0.0005) [2023-12-27 02:54:47,847][105620] Updated weights for policy 1, policy_version 1584771 (0.0005) [2023-12-27 02:54:48,080][105692] Updated weights for policy 0, policy_version 1581033 (0.0009) [2023-12-27 02:54:48,148][105692] Updated weights for policy 0, policy_version 1581043 (0.0010) [2023-12-27 02:54:48,212][105692] Updated weights for policy 0, policy_version 1581053 (0.0006) [2023-12-27 02:54:48,278][105692] Updated weights for policy 0, policy_version 1581063 (0.0005) [2023-12-27 02:54:48,448][105620] Updated weights for policy 1, policy_version 1584781 (0.0007) [2023-12-27 02:54:48,513][105620] Updated weights for policy 1, policy_version 1584791 (0.0008) [2023-12-27 02:54:48,577][105620] Updated weights for policy 1, policy_version 1584801 (0.0008) [2023-12-27 02:54:48,870][105692] Updated weights for policy 0, policy_version 1581073 (0.0010) [2023-12-27 02:54:48,919][105692] Updated weights for policy 0, policy_version 1581083 (0.0010) [2023-12-27 02:54:48,968][105692] Updated weights for policy 0, policy_version 1581093 (0.0010) [2023-12-27 02:54:49,270][105620] Updated weights for policy 1, policy_version 1584811 (0.0005) [2023-12-27 02:54:49,334][105620] Updated weights for policy 1, policy_version 1584821 (0.0007) [2023-12-27 02:54:49,404][105620] Updated weights for policy 1, policy_version 1584831 (0.0010) [2023-12-27 02:54:49,708][105692] Updated weights for policy 0, policy_version 1581103 (0.0009) [2023-12-27 02:54:49,775][105692] Updated weights for policy 0, policy_version 1581113 (0.0009) [2023-12-27 02:54:49,841][105692] Updated weights for policy 0, policy_version 1581123 (0.0008) [2023-12-27 02:54:50,157][105620] Updated weights for policy 1, policy_version 1584841 (0.0009) [2023-12-27 02:54:50,209][105620] Updated weights for policy 1, policy_version 1584851 (0.0009) [2023-12-27 02:54:50,264][105620] Updated weights for policy 1, policy_version 1584861 (0.0009) [2023-12-27 02:54:50,320][105620] Updated weights for policy 1, policy_version 1584871 (0.0009) [2023-12-27 02:54:50,514][105692] Updated weights for policy 0, policy_version 1581133 (0.0009) [2023-12-27 02:54:50,572][105692] Updated weights for policy 0, policy_version 1581143 (0.0008) [2023-12-27 02:54:50,635][105692] Updated weights for policy 0, policy_version 1581153 (0.0009) [2023-12-27 02:54:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 810614784. Throughput: 0: 9562.2, 1: 10122.3. Samples: 810607952. Policy #0 lag: (min: 7.0, avg: 8.8, max: 39.0) [2023-12-27 02:54:51,062][104569] Avg episode reward: [(0, '8532.361'), (1, '8998.729')] [2023-12-27 02:54:51,197][105620] Updated weights for policy 1, policy_version 1584881 (0.0007) [2023-12-27 02:54:51,259][105620] Updated weights for policy 1, policy_version 1584891 (0.0006) [2023-12-27 02:54:51,326][105620] Updated weights for policy 1, policy_version 1584901 (0.0006) [2023-12-27 02:54:51,341][105692] Updated weights for policy 0, policy_version 1581163 (0.0009) [2023-12-27 02:54:51,404][105692] Updated weights for policy 0, policy_version 1581173 (0.0008) [2023-12-27 02:54:51,471][105692] Updated weights for policy 0, policy_version 1581183 (0.0005) [2023-12-27 02:54:51,954][105620] Updated weights for policy 1, policy_version 1584911 (0.0007) [2023-12-27 02:54:52,024][105620] Updated weights for policy 1, policy_version 1584921 (0.0005) [2023-12-27 02:54:52,084][105620] Updated weights for policy 1, policy_version 1584931 (0.0005) [2023-12-27 02:54:52,174][105692] Updated weights for policy 0, policy_version 1581193 (0.0005) [2023-12-27 02:54:52,232][105692] Updated weights for policy 0, policy_version 1581203 (0.0005) [2023-12-27 02:54:52,291][105692] Updated weights for policy 0, policy_version 1581213 (0.0007) [2023-12-27 02:54:52,353][105692] Updated weights for policy 0, policy_version 1581223 (0.0006) [2023-12-27 02:54:52,664][105620] Updated weights for policy 1, policy_version 1584941 (0.0007) [2023-12-27 02:54:52,734][105620] Updated weights for policy 1, policy_version 1584951 (0.0008) [2023-12-27 02:54:52,787][105620] Updated weights for policy 1, policy_version 1584961 (0.0011) [2023-12-27 02:54:53,032][105692] Updated weights for policy 0, policy_version 1581233 (0.0006) [2023-12-27 02:54:53,079][105692] Updated weights for policy 0, policy_version 1581243 (0.0006) [2023-12-27 02:54:53,128][105692] Updated weights for policy 0, policy_version 1581253 (0.0008) [2023-12-27 02:54:53,481][105620] Updated weights for policy 1, policy_version 1584971 (0.0010) [2023-12-27 02:54:53,538][105620] Updated weights for policy 1, policy_version 1584981 (0.0010) [2023-12-27 02:54:53,593][105620] Updated weights for policy 1, policy_version 1584991 (0.0010) [2023-12-27 02:54:53,860][105692] Updated weights for policy 0, policy_version 1581263 (0.0008) [2023-12-27 02:54:53,909][105692] Updated weights for policy 0, policy_version 1581273 (0.0008) [2023-12-27 02:54:53,957][105692] Updated weights for policy 0, policy_version 1581283 (0.0008) [2023-12-27 02:54:54,335][105620] Updated weights for policy 1, policy_version 1585001 (0.0010) [2023-12-27 02:54:54,393][105620] Updated weights for policy 1, policy_version 1585011 (0.0010) [2023-12-27 02:54:54,459][105620] Updated weights for policy 1, policy_version 1585021 (0.0010) [2023-12-27 02:54:54,521][105620] Updated weights for policy 1, policy_version 1585031 (0.0008) [2023-12-27 02:54:54,689][105692] Updated weights for policy 0, policy_version 1581293 (0.0007) [2023-12-27 02:54:54,733][105692] Updated weights for policy 0, policy_version 1581303 (0.0005) [2023-12-27 02:54:54,786][105692] Updated weights for policy 0, policy_version 1581313 (0.0006) [2023-12-27 02:54:55,099][105620] Updated weights for policy 1, policy_version 1585041 (0.0010) [2023-12-27 02:54:55,147][105620] Updated weights for policy 1, policy_version 1585051 (0.0010) [2023-12-27 02:54:55,195][105620] Updated weights for policy 1, policy_version 1585061 (0.0010) [2023-12-27 02:54:55,466][105692] Updated weights for policy 0, policy_version 1581323 (0.0006) [2023-12-27 02:54:55,519][105692] Updated weights for policy 0, policy_version 1581333 (0.0008) [2023-12-27 02:54:55,573][105692] Updated weights for policy 0, policy_version 1581345 (0.0010) [2023-12-27 02:54:55,895][105620] Updated weights for policy 1, policy_version 1585071 (0.0010) [2023-12-27 02:54:55,948][105620] Updated weights for policy 1, policy_version 1585081 (0.0010) [2023-12-27 02:54:56,004][105620] Updated weights for policy 1, policy_version 1585091 (0.0010) [2023-12-27 02:54:56,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 810721280. Throughput: 0: 9601.8, 1: 10143.0. Samples: 810727180. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:54:56,062][104569] Avg episode reward: [(0, '8894.002'), (1, '8908.526')] [2023-12-27 02:54:56,228][105692] Updated weights for policy 0, policy_version 1581355 (0.0005) [2023-12-27 02:54:56,279][105692] Updated weights for policy 0, policy_version 1581365 (0.0005) [2023-12-27 02:54:56,331][105692] Updated weights for policy 0, policy_version 1581375 (0.0007) [2023-12-27 02:54:56,679][105620] Updated weights for policy 1, policy_version 1585101 (0.0008) [2023-12-27 02:54:56,731][105620] Updated weights for policy 1, policy_version 1585111 (0.0009) [2023-12-27 02:54:56,778][105620] Updated weights for policy 1, policy_version 1585121 (0.0010) [2023-12-27 02:54:56,985][105692] Updated weights for policy 0, policy_version 1581385 (0.0008) [2023-12-27 02:54:57,032][105692] Updated weights for policy 0, policy_version 1581395 (0.0010) [2023-12-27 02:54:57,076][105692] Updated weights for policy 0, policy_version 1581405 (0.0010) [2023-12-27 02:54:57,120][105692] Updated weights for policy 0, policy_version 1581415 (0.0006) [2023-12-27 02:54:57,504][105620] Updated weights for policy 1, policy_version 1585131 (0.0010) [2023-12-27 02:54:57,571][105620] Updated weights for policy 1, policy_version 1585141 (0.0005) [2023-12-27 02:54:57,636][105620] Updated weights for policy 1, policy_version 1585151 (0.0005) [2023-12-27 02:54:57,756][105692] Updated weights for policy 0, policy_version 1581425 (0.0006) [2023-12-27 02:54:57,811][105692] Updated weights for policy 0, policy_version 1581435 (0.0005) [2023-12-27 02:54:57,858][105692] Updated weights for policy 0, policy_version 1581445 (0.0006) [2023-12-27 02:54:58,159][105620] Updated weights for policy 1, policy_version 1585161 (0.0006) [2023-12-27 02:54:58,225][105620] Updated weights for policy 1, policy_version 1585171 (0.0010) [2023-12-27 02:54:58,285][105620] Updated weights for policy 1, policy_version 1585181 (0.0011) [2023-12-27 02:54:58,362][105620] Updated weights for policy 1, policy_version 1585191 (0.0010) [2023-12-27 02:54:58,565][105692] Updated weights for policy 0, policy_version 1581455 (0.0008) [2023-12-27 02:54:58,636][105692] Updated weights for policy 0, policy_version 1581465 (0.0009) [2023-12-27 02:54:58,704][105692] Updated weights for policy 0, policy_version 1581475 (0.0008) [2023-12-27 02:54:59,149][105620] Updated weights for policy 1, policy_version 1585201 (0.0006) [2023-12-27 02:54:59,214][105620] Updated weights for policy 1, policy_version 1585211 (0.0007) [2023-12-27 02:54:59,281][105620] Updated weights for policy 1, policy_version 1585221 (0.0007) [2023-12-27 02:54:59,550][105692] Updated weights for policy 0, policy_version 1581485 (0.0011) [2023-12-27 02:54:59,598][105692] Updated weights for policy 0, policy_version 1581495 (0.0008) [2023-12-27 02:54:59,651][105692] Updated weights for policy 0, policy_version 1581505 (0.0006) [2023-12-27 02:54:59,991][105620] Updated weights for policy 1, policy_version 1585231 (0.0007) [2023-12-27 02:55:00,051][105620] Updated weights for policy 1, policy_version 1585241 (0.0006) [2023-12-27 02:55:00,108][105620] Updated weights for policy 1, policy_version 1585251 (0.0005) [2023-12-27 02:55:00,307][105692] Updated weights for policy 0, policy_version 1581515 (0.0007) [2023-12-27 02:55:00,351][105692] Updated weights for policy 0, policy_version 1581525 (0.0010) [2023-12-27 02:55:00,396][105692] Updated weights for policy 0, policy_version 1581535 (0.0010) [2023-12-27 02:55:00,688][105620] Updated weights for policy 1, policy_version 1585261 (0.0005) [2023-12-27 02:55:00,749][105620] Updated weights for policy 1, policy_version 1585271 (0.0005) [2023-12-27 02:55:00,804][105620] Updated weights for policy 1, policy_version 1585281 (0.0005) [2023-12-27 02:55:00,972][105692] Updated weights for policy 0, policy_version 1581545 (0.0008) [2023-12-27 02:55:01,021][105692] Updated weights for policy 0, policy_version 1581555 (0.0006) [2023-12-27 02:55:01,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.3, 300 sec: 19410.9). Total num frames: 810819584. Throughput: 0: 9683.2, 1: 10163.2. Samples: 810789724. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:55:01,063][104569] Avg episode reward: [(0, '9170.036'), (1, '9090.937')] [2023-12-27 02:55:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001585288_405889024.pth... [2023-12-27 02:55:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001584104_405585920.pth [2023-12-27 02:55:01,083][105692] Updated weights for policy 0, policy_version 1581565 (0.0010) [2023-12-27 02:55:01,150][105692] Updated weights for policy 0, policy_version 1581575 (0.0006) [2023-12-27 02:55:01,155][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001581576_404938752.pth... [2023-12-27 02:55:01,160][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001580392_404635648.pth [2023-12-27 02:55:01,429][105620] Updated weights for policy 1, policy_version 1585291 (0.0006) [2023-12-27 02:55:01,488][105620] Updated weights for policy 1, policy_version 1585301 (0.0006) [2023-12-27 02:55:01,550][105620] Updated weights for policy 1, policy_version 1585311 (0.0007) [2023-12-27 02:55:01,862][105692] Updated weights for policy 0, policy_version 1581585 (0.0007) [2023-12-27 02:55:01,920][105692] Updated weights for policy 0, policy_version 1581595 (0.0008) [2023-12-27 02:55:01,983][105692] Updated weights for policy 0, policy_version 1581605 (0.0010) [2023-12-27 02:55:02,206][105620] Updated weights for policy 1, policy_version 1585321 (0.0008) [2023-12-27 02:55:02,256][105620] Updated weights for policy 1, policy_version 1585331 (0.0007) [2023-12-27 02:55:02,311][105620] Updated weights for policy 1, policy_version 1585341 (0.0009) [2023-12-27 02:55:02,377][105620] Updated weights for policy 1, policy_version 1585351 (0.0007) [2023-12-27 02:55:02,777][105692] Updated weights for policy 0, policy_version 1581615 (0.0009) [2023-12-27 02:55:02,831][105692] Updated weights for policy 0, policy_version 1581625 (0.0008) [2023-12-27 02:55:02,893][105692] Updated weights for policy 0, policy_version 1581635 (0.0007) [2023-12-27 02:55:03,068][105620] Updated weights for policy 1, policy_version 1585361 (0.0010) [2023-12-27 02:55:03,121][105620] Updated weights for policy 1, policy_version 1585371 (0.0010) [2023-12-27 02:55:03,179][105620] Updated weights for policy 1, policy_version 1585381 (0.0010) [2023-12-27 02:55:03,527][105692] Updated weights for policy 0, policy_version 1581645 (0.0006) [2023-12-27 02:55:03,580][105692] Updated weights for policy 0, policy_version 1581655 (0.0005) [2023-12-27 02:55:03,636][105692] Updated weights for policy 0, policy_version 1581665 (0.0005) [2023-12-27 02:55:03,898][105620] Updated weights for policy 1, policy_version 1585391 (0.0012) [2023-12-27 02:55:03,959][105620] Updated weights for policy 1, policy_version 1585401 (0.0009) [2023-12-27 02:55:04,008][105620] Updated weights for policy 1, policy_version 1585411 (0.0009) [2023-12-27 02:55:04,292][105692] Updated weights for policy 0, policy_version 1581675 (0.0006) [2023-12-27 02:55:04,349][105692] Updated weights for policy 0, policy_version 1581685 (0.0009) [2023-12-27 02:55:04,402][105692] Updated weights for policy 0, policy_version 1581696 (0.0010) [2023-12-27 02:55:04,725][105620] Updated weights for policy 1, policy_version 1585421 (0.0007) [2023-12-27 02:55:04,799][105620] Updated weights for policy 1, policy_version 1585431 (0.0005) [2023-12-27 02:55:04,861][105620] Updated weights for policy 1, policy_version 1585441 (0.0005) [2023-12-27 02:55:05,019][105692] Updated weights for policy 0, policy_version 1581706 (0.0006) [2023-12-27 02:55:05,071][105692] Updated weights for policy 0, policy_version 1581716 (0.0011) [2023-12-27 02:55:05,137][105692] Updated weights for policy 0, policy_version 1581726 (0.0011) [2023-12-27 02:55:05,197][105692] Updated weights for policy 0, policy_version 1581736 (0.0010) [2023-12-27 02:55:05,417][105620] Updated weights for policy 1, policy_version 1585451 (0.0007) [2023-12-27 02:55:05,464][105620] Updated weights for policy 1, policy_version 1585461 (0.0010) [2023-12-27 02:55:05,512][105620] Updated weights for policy 1, policy_version 1585471 (0.0010) [2023-12-27 02:55:05,829][105692] Updated weights for policy 0, policy_version 1581746 (0.0005) [2023-12-27 02:55:05,880][105692] Updated weights for policy 0, policy_version 1581756 (0.0005) [2023-12-27 02:55:05,940][105692] Updated weights for policy 0, policy_version 1581766 (0.0005) [2023-12-27 02:55:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19933.9, 300 sec: 19466.4). Total num frames: 810926080. Throughput: 0: 9708.6, 1: 10104.8. Samples: 810910816. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:55:06,062][104569] Avg episode reward: [(0, '8530.075'), (1, '9356.309')] [2023-12-27 02:55:06,298][105620] Updated weights for policy 1, policy_version 1585481 (0.0010) [2023-12-27 02:55:06,374][105620] Updated weights for policy 1, policy_version 1585491 (0.0005) [2023-12-27 02:55:06,437][105620] Updated weights for policy 1, policy_version 1585501 (0.0005) [2023-12-27 02:55:06,503][105620] Updated weights for policy 1, policy_version 1585511 (0.0009) [2023-12-27 02:55:06,594][105692] Updated weights for policy 0, policy_version 1581776 (0.0010) [2023-12-27 02:55:06,655][105692] Updated weights for policy 0, policy_version 1581786 (0.0011) [2023-12-27 02:55:06,720][105692] Updated weights for policy 0, policy_version 1581796 (0.0009) [2023-12-27 02:55:07,212][105620] Updated weights for policy 1, policy_version 1585521 (0.0008) [2023-12-27 02:55:07,273][105620] Updated weights for policy 1, policy_version 1585531 (0.0008) [2023-12-27 02:55:07,336][105620] Updated weights for policy 1, policy_version 1585541 (0.0008) [2023-12-27 02:55:07,463][105692] Updated weights for policy 0, policy_version 1581806 (0.0010) [2023-12-27 02:55:07,519][105692] Updated weights for policy 0, policy_version 1581816 (0.0011) [2023-12-27 02:55:07,581][105692] Updated weights for policy 0, policy_version 1581826 (0.0009) [2023-12-27 02:55:08,106][105620] Updated weights for policy 1, policy_version 1585551 (0.0008) [2023-12-27 02:55:08,154][105620] Updated weights for policy 1, policy_version 1585561 (0.0008) [2023-12-27 02:55:08,213][105620] Updated weights for policy 1, policy_version 1585571 (0.0008) [2023-12-27 02:55:08,335][105692] Updated weights for policy 0, policy_version 1581836 (0.0010) [2023-12-27 02:55:08,401][105692] Updated weights for policy 0, policy_version 1581846 (0.0007) [2023-12-27 02:55:08,466][105692] Updated weights for policy 0, policy_version 1581856 (0.0009) [2023-12-27 02:55:09,017][105620] Updated weights for policy 1, policy_version 1585581 (0.0009) [2023-12-27 02:55:09,072][105620] Updated weights for policy 1, policy_version 1585591 (0.0009) [2023-12-27 02:55:09,085][105692] Updated weights for policy 0, policy_version 1581866 (0.0008) [2023-12-27 02:55:09,126][105620] Updated weights for policy 1, policy_version 1585601 (0.0008) [2023-12-27 02:55:09,148][105692] Updated weights for policy 0, policy_version 1581876 (0.0007) [2023-12-27 02:55:09,214][105692] Updated weights for policy 0, policy_version 1581886 (0.0009) [2023-12-27 02:55:09,274][105692] Updated weights for policy 0, policy_version 1581896 (0.0010) [2023-12-27 02:55:09,806][105620] Updated weights for policy 1, policy_version 1585611 (0.0007) [2023-12-27 02:55:09,872][105620] Updated weights for policy 1, policy_version 1585621 (0.0009) [2023-12-27 02:55:09,942][105620] Updated weights for policy 1, policy_version 1585631 (0.0009) [2023-12-27 02:55:09,956][105692] Updated weights for policy 0, policy_version 1581906 (0.0008) [2023-12-27 02:55:10,020][105692] Updated weights for policy 0, policy_version 1581916 (0.0008) [2023-12-27 02:55:10,080][105692] Updated weights for policy 0, policy_version 1581926 (0.0009) [2023-12-27 02:55:10,689][105620] Updated weights for policy 1, policy_version 1585641 (0.0007) [2023-12-27 02:55:10,749][105620] Updated weights for policy 1, policy_version 1585651 (0.0011) [2023-12-27 02:55:10,784][105692] Updated weights for policy 0, policy_version 1581936 (0.0010) [2023-12-27 02:55:10,804][105620] Updated weights for policy 1, policy_version 1585661 (0.0011) [2023-12-27 02:55:10,846][105692] Updated weights for policy 0, policy_version 1581946 (0.0010) [2023-12-27 02:55:10,863][105620] Updated weights for policy 1, policy_version 1585671 (0.0010) [2023-12-27 02:55:10,903][105692] Updated weights for policy 0, policy_version 1581956 (0.0011) [2023-12-27 02:55:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19933.8, 300 sec: 19438.6). Total num frames: 811024384. Throughput: 0: 9892.1, 1: 10017.6. Samples: 811028772. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:55:11,063][104569] Avg episode reward: [(0, '8164.415'), (1, '9175.387')] [2023-12-27 02:55:11,630][105620] Updated weights for policy 1, policy_version 1585681 (0.0011) [2023-12-27 02:55:11,688][105620] Updated weights for policy 1, policy_version 1585691 (0.0011) [2023-12-27 02:55:11,699][105692] Updated weights for policy 0, policy_version 1581966 (0.0008) [2023-12-27 02:55:11,753][105620] Updated weights for policy 1, policy_version 1585701 (0.0009) [2023-12-27 02:55:11,761][105692] Updated weights for policy 0, policy_version 1581976 (0.0009) [2023-12-27 02:55:11,815][105692] Updated weights for policy 0, policy_version 1581986 (0.0010) [2023-12-27 02:55:12,556][105620] Updated weights for policy 1, policy_version 1585711 (0.0007) [2023-12-27 02:55:12,604][105692] Updated weights for policy 0, policy_version 1581996 (0.0009) [2023-12-27 02:55:12,611][105620] Updated weights for policy 1, policy_version 1585721 (0.0006) [2023-12-27 02:55:12,657][105692] Updated weights for policy 0, policy_version 1582006 (0.0009) [2023-12-27 02:55:12,673][105620] Updated weights for policy 1, policy_version 1585731 (0.0005) [2023-12-27 02:55:12,706][105692] Updated weights for policy 0, policy_version 1582016 (0.0009) [2023-12-27 02:55:13,244][105620] Updated weights for policy 1, policy_version 1585741 (0.0007) [2023-12-27 02:55:13,304][105620] Updated weights for policy 1, policy_version 1585751 (0.0008) [2023-12-27 02:55:13,368][105620] Updated weights for policy 1, policy_version 1585761 (0.0010) [2023-12-27 02:55:13,432][105692] Updated weights for policy 0, policy_version 1582026 (0.0008) [2023-12-27 02:55:13,476][105692] Updated weights for policy 0, policy_version 1582036 (0.0010) [2023-12-27 02:55:13,535][105692] Updated weights for policy 0, policy_version 1582046 (0.0010) [2023-12-27 02:55:13,597][105692] Updated weights for policy 0, policy_version 1582056 (0.0010) [2023-12-27 02:55:14,080][105620] Updated weights for policy 1, policy_version 1585771 (0.0008) [2023-12-27 02:55:14,145][105620] Updated weights for policy 1, policy_version 1585781 (0.0008) [2023-12-27 02:55:14,213][105620] Updated weights for policy 1, policy_version 1585791 (0.0008) [2023-12-27 02:55:14,221][105692] Updated weights for policy 0, policy_version 1582066 (0.0006) [2023-12-27 02:55:14,272][105692] Updated weights for policy 0, policy_version 1582076 (0.0006) [2023-12-27 02:55:14,324][105692] Updated weights for policy 0, policy_version 1582086 (0.0005) [2023-12-27 02:55:14,824][105620] Updated weights for policy 1, policy_version 1585801 (0.0007) [2023-12-27 02:55:14,882][105620] Updated weights for policy 1, policy_version 1585811 (0.0009) [2023-12-27 02:55:14,945][105620] Updated weights for policy 1, policy_version 1585821 (0.0007) [2023-12-27 02:55:14,955][105692] Updated weights for policy 0, policy_version 1582096 (0.0007) [2023-12-27 02:55:15,006][105620] Updated weights for policy 1, policy_version 1585831 (0.0008) [2023-12-27 02:55:15,013][105692] Updated weights for policy 0, policy_version 1582106 (0.0006) [2023-12-27 02:55:15,061][105692] Updated weights for policy 0, policy_version 1582116 (0.0008) [2023-12-27 02:55:15,747][105692] Updated weights for policy 0, policy_version 1582126 (0.0010) [2023-12-27 02:55:15,782][105620] Updated weights for policy 1, policy_version 1585841 (0.0007) [2023-12-27 02:55:15,812][105692] Updated weights for policy 0, policy_version 1582136 (0.0010) [2023-12-27 02:55:15,837][105620] Updated weights for policy 1, policy_version 1585851 (0.0005) [2023-12-27 02:55:15,873][105692] Updated weights for policy 0, policy_version 1582146 (0.0009) [2023-12-27 02:55:15,888][105620] Updated weights for policy 1, policy_version 1585861 (0.0009) [2023-12-27 02:55:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19933.9, 300 sec: 19438.6). Total num frames: 811122688. Throughput: 0: 9898.9, 1: 9964.1. Samples: 811085468. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:55:16,063][104569] Avg episode reward: [(0, '8529.312'), (1, '8992.096')] [2023-12-27 02:55:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001582152_405086208.pth... [2023-12-27 02:55:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001585864_406036480.pth... [2023-12-27 02:55:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001584680_405733376.pth [2023-12-27 02:55:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001580968_404783104.pth [2023-12-27 02:55:16,491][105692] Updated weights for policy 0, policy_version 1582156 (0.0007) [2023-12-27 02:55:16,550][105692] Updated weights for policy 0, policy_version 1582166 (0.0010) [2023-12-27 02:55:16,598][105692] Updated weights for policy 0, policy_version 1582176 (0.0010) [2023-12-27 02:55:16,636][105620] Updated weights for policy 1, policy_version 1585871 (0.0006) [2023-12-27 02:55:16,687][105620] Updated weights for policy 1, policy_version 1585881 (0.0005) [2023-12-27 02:55:16,737][105620] Updated weights for policy 1, policy_version 1585891 (0.0005) [2023-12-27 02:55:17,294][105620] Updated weights for policy 1, policy_version 1585901 (0.0005) [2023-12-27 02:55:17,301][105692] Updated weights for policy 0, policy_version 1582186 (0.0009) [2023-12-27 02:55:17,352][105620] Updated weights for policy 1, policy_version 1585911 (0.0007) [2023-12-27 02:55:17,362][105692] Updated weights for policy 0, policy_version 1582196 (0.0006) [2023-12-27 02:55:17,411][105620] Updated weights for policy 1, policy_version 1585921 (0.0010) [2023-12-27 02:55:17,415][105692] Updated weights for policy 0, policy_version 1582206 (0.0006) [2023-12-27 02:55:17,472][105692] Updated weights for policy 0, policy_version 1582216 (0.0005) [2023-12-27 02:55:18,048][105692] Updated weights for policy 0, policy_version 1582226 (0.0010) [2023-12-27 02:55:18,106][105692] Updated weights for policy 0, policy_version 1582236 (0.0010) [2023-12-27 02:55:18,108][105620] Updated weights for policy 1, policy_version 1585931 (0.0009) [2023-12-27 02:55:18,162][105620] Updated weights for policy 1, policy_version 1585941 (0.0005) [2023-12-27 02:55:18,172][105692] Updated weights for policy 0, policy_version 1582246 (0.0010) [2023-12-27 02:55:18,215][105620] Updated weights for policy 1, policy_version 1585951 (0.0008) [2023-12-27 02:55:18,933][105692] Updated weights for policy 0, policy_version 1582256 (0.0010) [2023-12-27 02:55:18,981][105620] Updated weights for policy 1, policy_version 1585961 (0.0007) [2023-12-27 02:55:18,988][105692] Updated weights for policy 0, policy_version 1582266 (0.0010) [2023-12-27 02:55:19,034][105620] Updated weights for policy 1, policy_version 1585971 (0.0006) [2023-12-27 02:55:19,047][105692] Updated weights for policy 0, policy_version 1582276 (0.0010) [2023-12-27 02:55:19,091][105620] Updated weights for policy 1, policy_version 1585981 (0.0007) [2023-12-27 02:55:19,150][105620] Updated weights for policy 1, policy_version 1585991 (0.0008) [2023-12-27 02:55:19,826][105692] Updated weights for policy 0, policy_version 1582286 (0.0011) [2023-12-27 02:55:19,885][105620] Updated weights for policy 1, policy_version 1586001 (0.0010) [2023-12-27 02:55:19,886][105692] Updated weights for policy 0, policy_version 1582296 (0.0011) [2023-12-27 02:55:19,952][105620] Updated weights for policy 1, policy_version 1586011 (0.0010) [2023-12-27 02:55:19,955][105692] Updated weights for policy 0, policy_version 1582306 (0.0010) [2023-12-27 02:55:20,017][105620] Updated weights for policy 1, policy_version 1586021 (0.0009) [2023-12-27 02:55:20,597][105620] Updated weights for policy 1, policy_version 1586031 (0.0009) [2023-12-27 02:55:20,671][105620] Updated weights for policy 1, policy_version 1586041 (0.0006) [2023-12-27 02:55:20,723][105692] Updated weights for policy 0, policy_version 1582316 (0.0011) [2023-12-27 02:55:20,735][105620] Updated weights for policy 1, policy_version 1586051 (0.0006) [2023-12-27 02:55:20,787][105692] Updated weights for policy 0, policy_version 1582326 (0.0011) [2023-12-27 02:55:20,845][105692] Updated weights for policy 0, policy_version 1582336 (0.0007) [2023-12-27 02:55:21,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19933.7, 300 sec: 19438.6). Total num frames: 811220992. Throughput: 0: 10047.9, 1: 9963.9. Samples: 811207928. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:55:21,063][104569] Avg episode reward: [(0, '8894.383'), (1, '8998.983')] [2023-12-27 02:55:21,419][105620] Updated weights for policy 1, policy_version 1586061 (0.0006) [2023-12-27 02:55:21,474][105620] Updated weights for policy 1, policy_version 1586071 (0.0005) [2023-12-27 02:55:21,531][105620] Updated weights for policy 1, policy_version 1586081 (0.0005) [2023-12-27 02:55:21,625][105692] Updated weights for policy 0, policy_version 1582346 (0.0010) [2023-12-27 02:55:21,692][105692] Updated weights for policy 0, policy_version 1582356 (0.0007) [2023-12-27 02:55:21,765][105692] Updated weights for policy 0, policy_version 1582366 (0.0008) [2023-12-27 02:55:21,823][105692] Updated weights for policy 0, policy_version 1582376 (0.0006) [2023-12-27 02:55:22,232][105620] Updated weights for policy 1, policy_version 1586091 (0.0007) [2023-12-27 02:55:22,293][105620] Updated weights for policy 1, policy_version 1586101 (0.0009) [2023-12-27 02:55:22,349][105620] Updated weights for policy 1, policy_version 1586111 (0.0007) [2023-12-27 02:55:22,521][105692] Updated weights for policy 0, policy_version 1582387 (0.0011) [2023-12-27 02:55:22,576][105692] Updated weights for policy 0, policy_version 1582398 (0.0009) [2023-12-27 02:55:22,635][105692] Updated weights for policy 0, policy_version 1582408 (0.0008) [2023-12-27 02:55:23,118][105620] Updated weights for policy 1, policy_version 1586121 (0.0009) [2023-12-27 02:55:23,177][105620] Updated weights for policy 1, policy_version 1586131 (0.0011) [2023-12-27 02:55:23,233][105620] Updated weights for policy 1, policy_version 1586141 (0.0010) [2023-12-27 02:55:23,298][105620] Updated weights for policy 1, policy_version 1586151 (0.0010) [2023-12-27 02:55:23,462][105692] Updated weights for policy 0, policy_version 1582418 (0.0008) [2023-12-27 02:55:23,514][105692] Updated weights for policy 0, policy_version 1582428 (0.0008) [2023-12-27 02:55:23,559][105692] Updated weights for policy 0, policy_version 1582438 (0.0008) [2023-12-27 02:55:24,041][105620] Updated weights for policy 1, policy_version 1586161 (0.0010) [2023-12-27 02:55:24,092][105620] Updated weights for policy 1, policy_version 1586171 (0.0010) [2023-12-27 02:55:24,142][105620] Updated weights for policy 1, policy_version 1586181 (0.0010) [2023-12-27 02:55:24,327][105692] Updated weights for policy 0, policy_version 1582448 (0.0008) [2023-12-27 02:55:24,390][105692] Updated weights for policy 0, policy_version 1582458 (0.0009) [2023-12-27 02:55:24,447][105692] Updated weights for policy 0, policy_version 1582468 (0.0008) [2023-12-27 02:55:24,812][105620] Updated weights for policy 1, policy_version 1586191 (0.0007) [2023-12-27 02:55:24,865][105620] Updated weights for policy 1, policy_version 1586201 (0.0006) [2023-12-27 02:55:24,923][105620] Updated weights for policy 1, policy_version 1586211 (0.0009) [2023-12-27 02:55:25,254][105692] Updated weights for policy 0, policy_version 1582478 (0.0009) [2023-12-27 02:55:25,317][105692] Updated weights for policy 0, policy_version 1582488 (0.0010) [2023-12-27 02:55:25,372][105692] Updated weights for policy 0, policy_version 1582498 (0.0010) [2023-12-27 02:55:25,512][105620] Updated weights for policy 1, policy_version 1586221 (0.0007) [2023-12-27 02:55:25,567][105620] Updated weights for policy 1, policy_version 1586231 (0.0009) [2023-12-27 02:55:25,618][105620] Updated weights for policy 1, policy_version 1586241 (0.0009) [2023-12-27 02:55:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19797.3, 300 sec: 19410.9). Total num frames: 811311104. Throughput: 0: 9950.7, 1: 10056.9. Samples: 811322680. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:55:26,062][104569] Avg episode reward: [(0, '8437.503'), (1, '8910.020')] [2023-12-27 02:55:26,195][105692] Updated weights for policy 0, policy_version 1582508 (0.0010) [2023-12-27 02:55:26,255][105692] Updated weights for policy 0, policy_version 1582518 (0.0006) [2023-12-27 02:55:26,309][105692] Updated weights for policy 0, policy_version 1582528 (0.0009) [2023-12-27 02:55:26,309][105620] Updated weights for policy 1, policy_version 1586251 (0.0009) [2023-12-27 02:55:26,358][105620] Updated weights for policy 1, policy_version 1586261 (0.0007) [2023-12-27 02:55:26,411][105620] Updated weights for policy 1, policy_version 1586271 (0.0008) [2023-12-27 02:55:27,014][105692] Updated weights for policy 0, policy_version 1582538 (0.0007) [2023-12-27 02:55:27,065][105692] Updated weights for policy 0, policy_version 1582548 (0.0005) [2023-12-27 02:55:27,101][105620] Updated weights for policy 1, policy_version 1586281 (0.0007) [2023-12-27 02:55:27,116][105692] Updated weights for policy 0, policy_version 1582558 (0.0005) [2023-12-27 02:55:27,164][105620] Updated weights for policy 1, policy_version 1586291 (0.0009) [2023-12-27 02:55:27,174][105692] Updated weights for policy 0, policy_version 1582568 (0.0006) [2023-12-27 02:55:27,224][105620] Updated weights for policy 1, policy_version 1586301 (0.0008) [2023-12-27 02:55:27,280][105620] Updated weights for policy 1, policy_version 1586311 (0.0009) [2023-12-27 02:55:27,862][105692] Updated weights for policy 0, policy_version 1582578 (0.0010) [2023-12-27 02:55:27,930][105692] Updated weights for policy 0, policy_version 1582588 (0.0010) [2023-12-27 02:55:27,992][105620] Updated weights for policy 1, policy_version 1586321 (0.0005) [2023-12-27 02:55:27,998][105692] Updated weights for policy 0, policy_version 1582598 (0.0010) [2023-12-27 02:55:28,051][105620] Updated weights for policy 1, policy_version 1586331 (0.0005) [2023-12-27 02:55:28,121][105620] Updated weights for policy 1, policy_version 1586341 (0.0005) [2023-12-27 02:55:28,612][105692] Updated weights for policy 0, policy_version 1582608 (0.0006) [2023-12-27 02:55:28,675][105692] Updated weights for policy 0, policy_version 1582618 (0.0006) [2023-12-27 02:55:28,682][105620] Updated weights for policy 1, policy_version 1586351 (0.0005) [2023-12-27 02:55:28,742][105620] Updated weights for policy 1, policy_version 1586361 (0.0005) [2023-12-27 02:55:28,742][105692] Updated weights for policy 0, policy_version 1582628 (0.0006) [2023-12-27 02:55:28,812][105620] Updated weights for policy 1, policy_version 1586371 (0.0009) [2023-12-27 02:55:29,358][105692] Updated weights for policy 0, policy_version 1582638 (0.0009) [2023-12-27 02:55:29,417][105692] Updated weights for policy 0, policy_version 1582648 (0.0011) [2023-12-27 02:55:29,472][105692] Updated weights for policy 0, policy_version 1582658 (0.0010) [2023-12-27 02:55:29,504][105620] Updated weights for policy 1, policy_version 1586381 (0.0008) [2023-12-27 02:55:29,567][105620] Updated weights for policy 1, policy_version 1586391 (0.0008) [2023-12-27 02:55:29,634][105620] Updated weights for policy 1, policy_version 1586401 (0.0009) [2023-12-27 02:55:30,237][105692] Updated weights for policy 0, policy_version 1582668 (0.0008) [2023-12-27 02:55:30,292][105620] Updated weights for policy 1, policy_version 1586411 (0.0009) [2023-12-27 02:55:30,301][105692] Updated weights for policy 0, policy_version 1582678 (0.0007) [2023-12-27 02:55:30,343][105620] Updated weights for policy 1, policy_version 1586421 (0.0007) [2023-12-27 02:55:30,360][105692] Updated weights for policy 0, policy_version 1582688 (0.0010) [2023-12-27 02:55:30,395][105620] Updated weights for policy 1, policy_version 1586431 (0.0006) [2023-12-27 02:55:30,953][105620] Updated weights for policy 1, policy_version 1586441 (0.0006) [2023-12-27 02:55:30,958][105692] Updated weights for policy 0, policy_version 1582698 (0.0008) [2023-12-27 02:55:31,006][105620] Updated weights for policy 1, policy_version 1586451 (0.0008) [2023-12-27 02:55:31,016][105692] Updated weights for policy 0, policy_version 1582708 (0.0006) [2023-12-27 02:55:31,062][104569] Fps is (10 sec: 18842.4, 60 sec: 19797.4, 300 sec: 19438.6). Total num frames: 811409408. Throughput: 0: 9941.2, 1: 10081.5. Samples: 811384132. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:55:31,062][104569] Avg episode reward: [(0, '8166.605'), (1, '8810.360')] [2023-12-27 02:55:31,064][105620] Updated weights for policy 1, policy_version 1586461 (0.0008) [2023-12-27 02:55:31,075][105692] Updated weights for policy 0, policy_version 1582718 (0.0008) [2023-12-27 02:55:31,121][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001582728_405233664.pth... [2023-12-27 02:55:31,122][105620] Updated weights for policy 1, policy_version 1586471 (0.0008) [2023-12-27 02:55:31,123][105692] Updated weights for policy 0, policy_version 1582728 (0.0005) [2023-12-27 02:55:31,125][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001586472_406192128.pth... [2023-12-27 02:55:31,124][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001581576_404938752.pth [2023-12-27 02:55:31,129][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001585288_405889024.pth [2023-12-27 02:55:31,841][105692] Updated weights for policy 0, policy_version 1582738 (0.0008) [2023-12-27 02:55:31,888][105620] Updated weights for policy 1, policy_version 1586481 (0.0008) [2023-12-27 02:55:31,894][105692] Updated weights for policy 0, policy_version 1582748 (0.0006) [2023-12-27 02:55:31,945][105620] Updated weights for policy 1, policy_version 1586491 (0.0006) [2023-12-27 02:55:31,947][105692] Updated weights for policy 0, policy_version 1582758 (0.0007) [2023-12-27 02:55:32,005][105620] Updated weights for policy 1, policy_version 1586501 (0.0008) [2023-12-27 02:55:32,623][105692] Updated weights for policy 0, policy_version 1582768 (0.0009) [2023-12-27 02:55:32,683][105692] Updated weights for policy 0, policy_version 1582778 (0.0010) [2023-12-27 02:55:32,720][105620] Updated weights for policy 1, policy_version 1586511 (0.0009) [2023-12-27 02:55:32,738][105692] Updated weights for policy 0, policy_version 1582788 (0.0006) [2023-12-27 02:55:32,785][105620] Updated weights for policy 1, policy_version 1586521 (0.0008) [2023-12-27 02:55:32,845][105620] Updated weights for policy 1, policy_version 1586531 (0.0007) [2023-12-27 02:55:33,465][105692] Updated weights for policy 0, policy_version 1582798 (0.0008) [2023-12-27 02:55:33,525][105692] Updated weights for policy 0, policy_version 1582808 (0.0008) [2023-12-27 02:55:33,570][105692] Updated weights for policy 0, policy_version 1582818 (0.0008) [2023-12-27 02:55:33,588][105620] Updated weights for policy 1, policy_version 1586541 (0.0008) [2023-12-27 02:55:33,647][105620] Updated weights for policy 1, policy_version 1586551 (0.0008) [2023-12-27 02:55:33,707][105620] Updated weights for policy 1, policy_version 1586561 (0.0009) [2023-12-27 02:55:34,193][105692] Updated weights for policy 0, policy_version 1582828 (0.0006) [2023-12-27 02:55:34,256][105692] Updated weights for policy 0, policy_version 1582838 (0.0006) [2023-12-27 02:55:34,315][105692] Updated weights for policy 0, policy_version 1582848 (0.0006) [2023-12-27 02:55:34,527][105620] Updated weights for policy 1, policy_version 1586571 (0.0008) [2023-12-27 02:55:34,588][105620] Updated weights for policy 1, policy_version 1586581 (0.0006) [2023-12-27 02:55:34,635][105620] Updated weights for policy 1, policy_version 1586591 (0.0008) [2023-12-27 02:55:35,098][105692] Updated weights for policy 0, policy_version 1582858 (0.0009) [2023-12-27 02:55:35,154][105692] Updated weights for policy 0, policy_version 1582868 (0.0009) [2023-12-27 02:55:35,216][105692] Updated weights for policy 0, policy_version 1582878 (0.0010) [2023-12-27 02:55:35,248][105620] Updated weights for policy 1, policy_version 1586601 (0.0008) [2023-12-27 02:55:35,262][105692] Updated weights for policy 0, policy_version 1582888 (0.0009) [2023-12-27 02:55:35,302][105620] Updated weights for policy 1, policy_version 1586611 (0.0008) [2023-12-27 02:55:35,355][105620] Updated weights for policy 1, policy_version 1586621 (0.0010) [2023-12-27 02:55:35,409][105620] Updated weights for policy 1, policy_version 1586631 (0.0010) [2023-12-27 02:55:35,915][105692] Updated weights for policy 0, policy_version 1582899 (0.0009) [2023-12-27 02:55:35,970][105692] Updated weights for policy 0, policy_version 1582909 (0.0008) [2023-12-27 02:55:36,024][105692] Updated weights for policy 0, policy_version 1582919 (0.0009) [2023-12-27 02:55:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19933.9, 300 sec: 19466.4). Total num frames: 811515904. Throughput: 0: 9935.9, 1: 9962.0. Samples: 811503360. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:55:36,063][104569] Avg episode reward: [(0, '8073.442'), (1, '8915.440')] [2023-12-27 02:55:36,117][105620] Updated weights for policy 1, policy_version 1586641 (0.0007) [2023-12-27 02:55:36,180][105620] Updated weights for policy 1, policy_version 1586651 (0.0006) [2023-12-27 02:55:36,247][105620] Updated weights for policy 1, policy_version 1586661 (0.0008) [2023-12-27 02:55:36,807][105692] Updated weights for policy 0, policy_version 1582929 (0.0008) [2023-12-27 02:55:36,873][105692] Updated weights for policy 0, policy_version 1582939 (0.0008) [2023-12-27 02:55:36,930][105692] Updated weights for policy 0, policy_version 1582949 (0.0008) [2023-12-27 02:55:36,974][105620] Updated weights for policy 1, policy_version 1586671 (0.0010) [2023-12-27 02:55:37,032][105620] Updated weights for policy 1, policy_version 1586681 (0.0010) [2023-12-27 02:55:37,088][105620] Updated weights for policy 1, policy_version 1586691 (0.0010) [2023-12-27 02:55:37,691][105692] Updated weights for policy 0, policy_version 1582959 (0.0009) [2023-12-27 02:55:37,754][105692] Updated weights for policy 0, policy_version 1582969 (0.0008) [2023-12-27 02:55:37,806][105620] Updated weights for policy 1, policy_version 1586701 (0.0008) [2023-12-27 02:55:37,812][105692] Updated weights for policy 0, policy_version 1582979 (0.0008) [2023-12-27 02:55:37,872][105620] Updated weights for policy 1, policy_version 1586711 (0.0008) [2023-12-27 02:55:37,934][105620] Updated weights for policy 1, policy_version 1586721 (0.0008) [2023-12-27 02:55:38,587][105620] Updated weights for policy 1, policy_version 1586731 (0.0007) [2023-12-27 02:55:38,604][105692] Updated weights for policy 0, policy_version 1582989 (0.0008) [2023-12-27 02:55:38,651][105620] Updated weights for policy 1, policy_version 1586741 (0.0008) [2023-12-27 02:55:38,674][105692] Updated weights for policy 0, policy_version 1582999 (0.0006) [2023-12-27 02:55:38,711][105620] Updated weights for policy 1, policy_version 1586751 (0.0007) [2023-12-27 02:55:38,740][105692] Updated weights for policy 0, policy_version 1583009 (0.0006) [2023-12-27 02:55:39,355][105620] Updated weights for policy 1, policy_version 1586761 (0.0008) [2023-12-27 02:55:39,421][105620] Updated weights for policy 1, policy_version 1586771 (0.0008) [2023-12-27 02:55:39,479][105692] Updated weights for policy 0, policy_version 1583019 (0.0005) [2023-12-27 02:55:39,481][105620] Updated weights for policy 1, policy_version 1586781 (0.0008) [2023-12-27 02:55:39,538][105620] Updated weights for policy 1, policy_version 1586791 (0.0008) [2023-12-27 02:55:39,543][105692] Updated weights for policy 0, policy_version 1583029 (0.0008) [2023-12-27 02:55:39,603][105692] Updated weights for policy 0, policy_version 1583039 (0.0008) [2023-12-27 02:55:40,241][105620] Updated weights for policy 1, policy_version 1586801 (0.0006) [2023-12-27 02:55:40,290][105620] Updated weights for policy 1, policy_version 1586811 (0.0008) [2023-12-27 02:55:40,348][105620] Updated weights for policy 1, policy_version 1586821 (0.0008) [2023-12-27 02:55:40,411][105692] Updated weights for policy 0, policy_version 1583049 (0.0008) [2023-12-27 02:55:40,466][105692] Updated weights for policy 0, policy_version 1583059 (0.0009) [2023-12-27 02:55:40,523][105692] Updated weights for policy 0, policy_version 1583069 (0.0009) [2023-12-27 02:55:40,591][105692] Updated weights for policy 0, policy_version 1583080 (0.0008) [2023-12-27 02:55:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19438.6). Total num frames: 811606016. Throughput: 0: 9841.7, 1: 9964.0. Samples: 811618436. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:55:41,063][104569] Avg episode reward: [(0, '8076.070'), (1, '9097.406')] [2023-12-27 02:55:41,087][105620] Updated weights for policy 1, policy_version 1586831 (0.0007) [2023-12-27 02:55:41,151][105620] Updated weights for policy 1, policy_version 1586841 (0.0008) [2023-12-27 02:55:41,220][105620] Updated weights for policy 1, policy_version 1586851 (0.0009) [2023-12-27 02:55:41,396][105692] Updated weights for policy 0, policy_version 1583090 (0.0009) [2023-12-27 02:55:41,457][105692] Updated weights for policy 0, policy_version 1583100 (0.0006) [2023-12-27 02:55:41,514][105692] Updated weights for policy 0, policy_version 1583110 (0.0005) [2023-12-27 02:55:42,024][105620] Updated weights for policy 1, policy_version 1586861 (0.0009) [2023-12-27 02:55:42,080][105620] Updated weights for policy 1, policy_version 1586871 (0.0008) [2023-12-27 02:55:42,105][105692] Updated weights for policy 0, policy_version 1583120 (0.0008) [2023-12-27 02:55:42,148][105620] Updated weights for policy 1, policy_version 1586881 (0.0007) [2023-12-27 02:55:42,167][105692] Updated weights for policy 0, policy_version 1583130 (0.0008) [2023-12-27 02:55:42,238][105692] Updated weights for policy 0, policy_version 1583140 (0.0007) [2023-12-27 02:55:42,889][105692] Updated weights for policy 0, policy_version 1583150 (0.0006) [2023-12-27 02:55:42,942][105692] Updated weights for policy 0, policy_version 1583160 (0.0006) [2023-12-27 02:55:42,966][105620] Updated weights for policy 1, policy_version 1586891 (0.0009) [2023-12-27 02:55:43,003][105692] Updated weights for policy 0, policy_version 1583170 (0.0006) [2023-12-27 02:55:43,023][105620] Updated weights for policy 1, policy_version 1586901 (0.0008) [2023-12-27 02:55:43,083][105620] Updated weights for policy 1, policy_version 1586911 (0.0005) [2023-12-27 02:55:43,643][105692] Updated weights for policy 0, policy_version 1583180 (0.0006) [2023-12-27 02:55:43,707][105692] Updated weights for policy 0, policy_version 1583190 (0.0005) [2023-12-27 02:55:43,767][105692] Updated weights for policy 0, policy_version 1583200 (0.0005) [2023-12-27 02:55:43,809][105620] Updated weights for policy 1, policy_version 1586921 (0.0006) [2023-12-27 02:55:43,862][105620] Updated weights for policy 1, policy_version 1586931 (0.0010) [2023-12-27 02:55:43,923][105620] Updated weights for policy 1, policy_version 1586941 (0.0010) [2023-12-27 02:55:43,970][105620] Updated weights for policy 1, policy_version 1586951 (0.0005) [2023-12-27 02:55:44,346][105692] Updated weights for policy 0, policy_version 1583210 (0.0005) [2023-12-27 02:55:44,410][105692] Updated weights for policy 0, policy_version 1583220 (0.0005) [2023-12-27 02:55:44,471][105692] Updated weights for policy 0, policy_version 1583230 (0.0008) [2023-12-27 02:55:44,534][105692] Updated weights for policy 0, policy_version 1583240 (0.0009) [2023-12-27 02:55:44,674][105620] Updated weights for policy 1, policy_version 1586961 (0.0009) [2023-12-27 02:55:44,719][105620] Updated weights for policy 1, policy_version 1586971 (0.0005) [2023-12-27 02:55:44,775][105620] Updated weights for policy 1, policy_version 1586981 (0.0005) [2023-12-27 02:55:45,210][105692] Updated weights for policy 0, policy_version 1583250 (0.0008) [2023-12-27 02:55:45,256][105692] Updated weights for policy 0, policy_version 1583260 (0.0005) [2023-12-27 02:55:45,306][105692] Updated weights for policy 0, policy_version 1583270 (0.0007) [2023-12-27 02:55:45,513][105620] Updated weights for policy 1, policy_version 1586991 (0.0010) [2023-12-27 02:55:45,558][105586] KL-divergence is very high: 183.4283 [2023-12-27 02:55:45,564][105620] Updated weights for policy 1, policy_version 1587001 (0.0011) [2023-12-27 02:55:45,602][105586] KL-divergence is very high: 202.3179 [2023-12-27 02:55:45,617][105620] Updated weights for policy 1, policy_version 1587011 (0.0011) [2023-12-27 02:55:46,056][105692] Updated weights for policy 0, policy_version 1583280 (0.0008) [2023-12-27 02:55:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19797.3, 300 sec: 19438.6). Total num frames: 811704320. Throughput: 0: 9809.8, 1: 9879.9. Samples: 811675756. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:55:46,062][104569] Avg episode reward: [(0, '8529.575'), (1, '9088.910')] [2023-12-27 02:55:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001587016_406331392.pth... [2023-12-27 02:55:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001585864_406036480.pth [2023-12-27 02:55:46,115][105692] Updated weights for policy 0, policy_version 1583290 (0.0010) [2023-12-27 02:55:46,163][105692] Updated weights for policy 0, policy_version 1583300 (0.0010) [2023-12-27 02:55:46,187][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001583304_405381120.pth... [2023-12-27 02:55:46,190][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001582152_405086208.pth [2023-12-27 02:55:46,358][105620] Updated weights for policy 1, policy_version 1587021 (0.0010) [2023-12-27 02:55:46,415][105620] Updated weights for policy 1, policy_version 1587031 (0.0008) [2023-12-27 02:55:46,478][105620] Updated weights for policy 1, policy_version 1587041 (0.0008) [2023-12-27 02:55:46,895][105692] Updated weights for policy 0, policy_version 1583310 (0.0010) [2023-12-27 02:55:46,950][105692] Updated weights for policy 0, policy_version 1583320 (0.0010) [2023-12-27 02:55:46,998][105692] Updated weights for policy 0, policy_version 1583330 (0.0010) [2023-12-27 02:55:47,114][105620] Updated weights for policy 1, policy_version 1587051 (0.0007) [2023-12-27 02:55:47,173][105620] Updated weights for policy 1, policy_version 1587061 (0.0005) [2023-12-27 02:55:47,230][105620] Updated weights for policy 1, policy_version 1587071 (0.0005) [2023-12-27 02:55:47,695][105692] Updated weights for policy 0, policy_version 1583340 (0.0009) [2023-12-27 02:55:47,747][105692] Updated weights for policy 0, policy_version 1583350 (0.0010) [2023-12-27 02:55:47,767][105620] Updated weights for policy 1, policy_version 1587081 (0.0006) [2023-12-27 02:55:47,798][105692] Updated weights for policy 0, policy_version 1583360 (0.0010) [2023-12-27 02:55:47,823][105620] Updated weights for policy 1, policy_version 1587091 (0.0005) [2023-12-27 02:55:47,874][105620] Updated weights for policy 1, policy_version 1587101 (0.0006) [2023-12-27 02:55:47,918][105620] Updated weights for policy 1, policy_version 1587111 (0.0005) [2023-12-27 02:55:48,470][105692] Updated weights for policy 0, policy_version 1583370 (0.0009) [2023-12-27 02:55:48,529][105692] Updated weights for policy 0, policy_version 1583380 (0.0009) [2023-12-27 02:55:48,544][105620] Updated weights for policy 1, policy_version 1587121 (0.0007) [2023-12-27 02:55:48,589][105692] Updated weights for policy 0, policy_version 1583390 (0.0009) [2023-12-27 02:55:48,600][105620] Updated weights for policy 1, policy_version 1587131 (0.0009) [2023-12-27 02:55:48,647][105692] Updated weights for policy 0, policy_version 1583400 (0.0008) [2023-12-27 02:55:48,661][105620] Updated weights for policy 1, policy_version 1587141 (0.0007) [2023-12-27 02:55:49,272][105692] Updated weights for policy 0, policy_version 1583410 (0.0008) [2023-12-27 02:55:49,340][105692] Updated weights for policy 0, policy_version 1583420 (0.0009) [2023-12-27 02:55:49,399][105692] Updated weights for policy 0, policy_version 1583430 (0.0009) [2023-12-27 02:55:49,445][105620] Updated weights for policy 1, policy_version 1587151 (0.0008) [2023-12-27 02:55:49,501][105620] Updated weights for policy 1, policy_version 1587161 (0.0008) [2023-12-27 02:55:49,557][105620] Updated weights for policy 1, policy_version 1587171 (0.0008) [2023-12-27 02:55:50,196][105620] Updated weights for policy 1, policy_version 1587181 (0.0008) [2023-12-27 02:55:50,251][105692] Updated weights for policy 0, policy_version 1583440 (0.0006) [2023-12-27 02:55:50,256][105620] Updated weights for policy 1, policy_version 1587191 (0.0011) [2023-12-27 02:55:50,303][105692] Updated weights for policy 0, policy_version 1583450 (0.0008) [2023-12-27 02:55:50,318][105620] Updated weights for policy 1, policy_version 1587201 (0.0011) [2023-12-27 02:55:50,356][105692] Updated weights for policy 0, policy_version 1583460 (0.0007) [2023-12-27 02:55:51,025][105620] Updated weights for policy 1, policy_version 1587211 (0.0010) [2023-12-27 02:55:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.4, 300 sec: 19438.7). Total num frames: 811802624. Throughput: 0: 9835.3, 1: 9907.2. Samples: 811799228. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:55:51,062][104569] Avg episode reward: [(0, '8532.217'), (1, '9086.872')] [2023-12-27 02:55:51,084][105620] Updated weights for policy 1, policy_version 1587221 (0.0009) [2023-12-27 02:55:51,137][105620] Updated weights for policy 1, policy_version 1587231 (0.0010) [2023-12-27 02:55:51,189][105692] Updated weights for policy 0, policy_version 1583470 (0.0007) [2023-12-27 02:55:51,242][105692] Updated weights for policy 0, policy_version 1583480 (0.0009) [2023-12-27 02:55:51,298][105692] Updated weights for policy 0, policy_version 1583490 (0.0008) [2023-12-27 02:55:51,857][105620] Updated weights for policy 1, policy_version 1587241 (0.0010) [2023-12-27 02:55:51,922][105620] Updated weights for policy 1, policy_version 1587251 (0.0007) [2023-12-27 02:55:51,976][105620] Updated weights for policy 1, policy_version 1587261 (0.0006) [2023-12-27 02:55:52,029][105620] Updated weights for policy 1, policy_version 1587271 (0.0011) [2023-12-27 02:55:52,046][105692] Updated weights for policy 0, policy_version 1583500 (0.0008) [2023-12-27 02:55:52,112][105692] Updated weights for policy 0, policy_version 1583510 (0.0011) [2023-12-27 02:55:52,174][105692] Updated weights for policy 0, policy_version 1583520 (0.0010) [2023-12-27 02:55:52,680][105620] Updated weights for policy 1, policy_version 1587281 (0.0011) [2023-12-27 02:55:52,740][105620] Updated weights for policy 1, policy_version 1587291 (0.0011) [2023-12-27 02:55:52,804][105620] Updated weights for policy 1, policy_version 1587301 (0.0010) [2023-12-27 02:55:52,888][105692] Updated weights for policy 0, policy_version 1583530 (0.0010) [2023-12-27 02:55:52,953][105692] Updated weights for policy 0, policy_version 1583540 (0.0011) [2023-12-27 02:55:53,021][105692] Updated weights for policy 0, policy_version 1583550 (0.0011) [2023-12-27 02:55:53,069][105692] Updated weights for policy 0, policy_version 1583560 (0.0010) [2023-12-27 02:55:53,535][105620] Updated weights for policy 1, policy_version 1587311 (0.0011) [2023-12-27 02:55:53,601][105620] Updated weights for policy 1, policy_version 1587321 (0.0011) [2023-12-27 02:55:53,660][105620] Updated weights for policy 1, policy_version 1587331 (0.0011) [2023-12-27 02:55:53,775][105692] Updated weights for policy 0, policy_version 1583570 (0.0008) [2023-12-27 02:55:53,829][105692] Updated weights for policy 0, policy_version 1583580 (0.0008) [2023-12-27 02:55:53,884][105692] Updated weights for policy 0, policy_version 1583590 (0.0008) [2023-12-27 02:55:54,396][105620] Updated weights for policy 1, policy_version 1587341 (0.0011) [2023-12-27 02:55:54,454][105620] Updated weights for policy 1, policy_version 1587351 (0.0011) [2023-12-27 02:55:54,509][105620] Updated weights for policy 1, policy_version 1587361 (0.0010) [2023-12-27 02:55:54,649][105692] Updated weights for policy 0, policy_version 1583600 (0.0008) [2023-12-27 02:55:54,708][105692] Updated weights for policy 0, policy_version 1583610 (0.0008) [2023-12-27 02:55:54,760][105692] Updated weights for policy 0, policy_version 1583620 (0.0009) [2023-12-27 02:55:55,231][105620] Updated weights for policy 1, policy_version 1587371 (0.0009) [2023-12-27 02:55:55,302][105620] Updated weights for policy 1, policy_version 1587381 (0.0005) [2023-12-27 02:55:55,359][105620] Updated weights for policy 1, policy_version 1587391 (0.0010) [2023-12-27 02:55:55,515][105692] Updated weights for policy 0, policy_version 1583630 (0.0008) [2023-12-27 02:55:55,570][105692] Updated weights for policy 0, policy_version 1583640 (0.0008) [2023-12-27 02:55:55,641][105692] Updated weights for policy 0, policy_version 1583650 (0.0005) [2023-12-27 02:55:56,020][105620] Updated weights for policy 1, policy_version 1587401 (0.0010) [2023-12-27 02:55:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 811900928. Throughput: 0: 9705.2, 1: 9945.7. Samples: 811913060. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:55:56,063][104569] Avg episode reward: [(0, '8534.155'), (1, '9264.967')] [2023-12-27 02:55:56,075][105620] Updated weights for policy 1, policy_version 1587411 (0.0006) [2023-12-27 02:55:56,139][105620] Updated weights for policy 1, policy_version 1587421 (0.0005) [2023-12-27 02:55:56,198][105620] Updated weights for policy 1, policy_version 1587431 (0.0007) [2023-12-27 02:55:56,280][105692] Updated weights for policy 0, policy_version 1583660 (0.0007) [2023-12-27 02:55:56,346][105692] Updated weights for policy 0, policy_version 1583670 (0.0008) [2023-12-27 02:55:56,410][105692] Updated weights for policy 0, policy_version 1583680 (0.0007) [2023-12-27 02:55:56,859][105620] Updated weights for policy 1, policy_version 1587441 (0.0010) [2023-12-27 02:55:56,912][105620] Updated weights for policy 1, policy_version 1587451 (0.0010) [2023-12-27 02:55:56,973][105620] Updated weights for policy 1, policy_version 1587461 (0.0010) [2023-12-27 02:55:57,083][105692] Updated weights for policy 0, policy_version 1583690 (0.0008) [2023-12-27 02:55:57,139][105692] Updated weights for policy 0, policy_version 1583700 (0.0005) [2023-12-27 02:55:57,195][105692] Updated weights for policy 0, policy_version 1583710 (0.0007) [2023-12-27 02:55:57,257][105692] Updated weights for policy 0, policy_version 1583720 (0.0008) [2023-12-27 02:55:57,708][105620] Updated weights for policy 1, policy_version 1587471 (0.0010) [2023-12-27 02:55:57,769][105620] Updated weights for policy 1, policy_version 1587481 (0.0011) [2023-12-27 02:55:57,831][105620] Updated weights for policy 1, policy_version 1587491 (0.0010) [2023-12-27 02:55:57,981][105692] Updated weights for policy 0, policy_version 1583731 (0.0010) [2023-12-27 02:55:58,038][105692] Updated weights for policy 0, policy_version 1583741 (0.0010) [2023-12-27 02:55:58,087][105692] Updated weights for policy 0, policy_version 1583752 (0.0008) [2023-12-27 02:55:58,473][105620] Updated weights for policy 1, policy_version 1587501 (0.0006) [2023-12-27 02:55:58,543][105620] Updated weights for policy 1, policy_version 1587511 (0.0008) [2023-12-27 02:55:58,609][105620] Updated weights for policy 1, policy_version 1587521 (0.0007) [2023-12-27 02:55:58,948][105692] Updated weights for policy 0, policy_version 1583762 (0.0009) [2023-12-27 02:55:59,004][105692] Updated weights for policy 0, policy_version 1583772 (0.0008) [2023-12-27 02:55:59,064][105692] Updated weights for policy 0, policy_version 1583782 (0.0008) [2023-12-27 02:55:59,422][105620] Updated weights for policy 1, policy_version 1587531 (0.0009) [2023-12-27 02:55:59,477][105620] Updated weights for policy 1, policy_version 1587541 (0.0010) [2023-12-27 02:55:59,530][105620] Updated weights for policy 1, policy_version 1587551 (0.0010) [2023-12-27 02:55:59,819][105692] Updated weights for policy 0, policy_version 1583792 (0.0006) [2023-12-27 02:55:59,880][105692] Updated weights for policy 0, policy_version 1583802 (0.0007) [2023-12-27 02:55:59,941][105692] Updated weights for policy 0, policy_version 1583812 (0.0009) [2023-12-27 02:56:00,281][105620] Updated weights for policy 1, policy_version 1587561 (0.0010) [2023-12-27 02:56:00,335][105620] Updated weights for policy 1, policy_version 1587571 (0.0010) [2023-12-27 02:56:00,384][105620] Updated weights for policy 1, policy_version 1587581 (0.0010) [2023-12-27 02:56:00,439][105620] Updated weights for policy 1, policy_version 1587591 (0.0010) [2023-12-27 02:56:00,670][105692] Updated weights for policy 0, policy_version 1583822 (0.0007) [2023-12-27 02:56:00,717][105692] Updated weights for policy 0, policy_version 1583832 (0.0008) [2023-12-27 02:56:00,771][105692] Updated weights for policy 0, policy_version 1583842 (0.0007) [2023-12-27 02:56:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.9, 300 sec: 19466.4). Total num frames: 811999232. Throughput: 0: 9746.4, 1: 9949.7. Samples: 811971792. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:56:01,063][104569] Avg episode reward: [(0, '8806.764'), (1, '9267.528')] [2023-12-27 02:56:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001583848_405520384.pth... [2023-12-27 02:56:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001587592_406478848.pth... [2023-12-27 02:56:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001582728_405233664.pth [2023-12-27 02:56:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001586472_406192128.pth [2023-12-27 02:56:01,202][105620] Updated weights for policy 1, policy_version 1587601 (0.0009) [2023-12-27 02:56:01,258][105620] Updated weights for policy 1, policy_version 1587611 (0.0009) [2023-12-27 02:56:01,307][105620] Updated weights for policy 1, policy_version 1587621 (0.0010) [2023-12-27 02:56:01,513][105692] Updated weights for policy 0, policy_version 1583852 (0.0008) [2023-12-27 02:56:01,570][105692] Updated weights for policy 0, policy_version 1583863 (0.0009) [2023-12-27 02:56:01,632][105692] Updated weights for policy 0, policy_version 1583873 (0.0009) [2023-12-27 02:56:02,020][105620] Updated weights for policy 1, policy_version 1587631 (0.0010) [2023-12-27 02:56:02,071][105620] Updated weights for policy 1, policy_version 1587641 (0.0010) [2023-12-27 02:56:02,129][105620] Updated weights for policy 1, policy_version 1587651 (0.0010) [2023-12-27 02:56:02,423][105692] Updated weights for policy 0, policy_version 1583883 (0.0009) [2023-12-27 02:56:02,490][105692] Updated weights for policy 0, policy_version 1583893 (0.0008) [2023-12-27 02:56:02,549][105692] Updated weights for policy 0, policy_version 1583903 (0.0008) [2023-12-27 02:56:02,884][105620] Updated weights for policy 1, policy_version 1587661 (0.0010) [2023-12-27 02:56:02,943][105620] Updated weights for policy 1, policy_version 1587671 (0.0010) [2023-12-27 02:56:02,998][105620] Updated weights for policy 1, policy_version 1587681 (0.0010) [2023-12-27 02:56:03,300][105692] Updated weights for policy 0, policy_version 1583913 (0.0008) [2023-12-27 02:56:03,352][105692] Updated weights for policy 0, policy_version 1583923 (0.0008) [2023-12-27 02:56:03,395][105692] Updated weights for policy 0, policy_version 1583933 (0.0008) [2023-12-27 02:56:03,440][105692] Updated weights for policy 0, policy_version 1583943 (0.0008) [2023-12-27 02:56:03,714][105620] Updated weights for policy 1, policy_version 1587691 (0.0010) [2023-12-27 02:56:03,774][105620] Updated weights for policy 1, policy_version 1587701 (0.0008) [2023-12-27 02:56:03,832][105620] Updated weights for policy 1, policy_version 1587711 (0.0008) [2023-12-27 02:56:04,234][105692] Updated weights for policy 0, policy_version 1583953 (0.0009) [2023-12-27 02:56:04,302][105692] Updated weights for policy 0, policy_version 1583963 (0.0009) [2023-12-27 02:56:04,365][105692] Updated weights for policy 0, policy_version 1583973 (0.0009) [2023-12-27 02:56:04,597][105620] Updated weights for policy 1, policy_version 1587721 (0.0009) [2023-12-27 02:56:04,662][105620] Updated weights for policy 1, policy_version 1587731 (0.0006) [2023-12-27 02:56:04,722][105620] Updated weights for policy 1, policy_version 1587741 (0.0006) [2023-12-27 02:56:04,782][105620] Updated weights for policy 1, policy_version 1587751 (0.0005) [2023-12-27 02:56:05,156][105692] Updated weights for policy 0, policy_version 1583983 (0.0007) [2023-12-27 02:56:05,213][105692] Updated weights for policy 0, policy_version 1583993 (0.0005) [2023-12-27 02:56:05,270][105692] Updated weights for policy 0, policy_version 1584003 (0.0005) [2023-12-27 02:56:05,371][105620] Updated weights for policy 1, policy_version 1587761 (0.0005) [2023-12-27 02:56:05,434][105620] Updated weights for policy 1, policy_version 1587771 (0.0005) [2023-12-27 02:56:05,483][105620] Updated weights for policy 1, policy_version 1587781 (0.0005) [2023-12-27 02:56:05,873][105692] Updated weights for policy 0, policy_version 1584013 (0.0008) [2023-12-27 02:56:05,938][105692] Updated weights for policy 0, policy_version 1584023 (0.0011) [2023-12-27 02:56:05,990][105692] Updated weights for policy 0, policy_version 1584033 (0.0010) [2023-12-27 02:56:05,995][105620] Updated weights for policy 1, policy_version 1587791 (0.0006) [2023-12-27 02:56:06,060][105620] Updated weights for policy 1, policy_version 1587801 (0.0007) [2023-12-27 02:56:06,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 812097536. Throughput: 0: 9579.1, 1: 9892.2. Samples: 812084128. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:56:06,062][104569] Avg episode reward: [(0, '8622.651'), (1, '8995.015')] [2023-12-27 02:56:06,123][105620] Updated weights for policy 1, policy_version 1587811 (0.0007) [2023-12-27 02:56:06,736][105692] Updated weights for policy 0, policy_version 1584043 (0.0009) [2023-12-27 02:56:06,798][105692] Updated weights for policy 0, policy_version 1584053 (0.0010) [2023-12-27 02:56:06,833][105620] Updated weights for policy 1, policy_version 1587821 (0.0006) [2023-12-27 02:56:06,867][105692] Updated weights for policy 0, policy_version 1584063 (0.0009) [2023-12-27 02:56:06,893][105620] Updated weights for policy 1, policy_version 1587831 (0.0006) [2023-12-27 02:56:06,943][105620] Updated weights for policy 1, policy_version 1587841 (0.0007) [2023-12-27 02:56:07,539][105692] Updated weights for policy 0, policy_version 1584073 (0.0011) [2023-12-27 02:56:07,585][105692] Updated weights for policy 0, policy_version 1584083 (0.0010) [2023-12-27 02:56:07,634][105692] Updated weights for policy 0, policy_version 1584093 (0.0011) [2023-12-27 02:56:07,641][105620] Updated weights for policy 1, policy_version 1587851 (0.0008) [2023-12-27 02:56:07,683][105692] Updated weights for policy 0, policy_version 1584103 (0.0010) [2023-12-27 02:56:07,697][105620] Updated weights for policy 1, policy_version 1587861 (0.0006) [2023-12-27 02:56:07,761][105620] Updated weights for policy 1, policy_version 1587871 (0.0008) [2023-12-27 02:56:08,373][105620] Updated weights for policy 1, policy_version 1587881 (0.0008) [2023-12-27 02:56:08,427][105620] Updated weights for policy 1, policy_version 1587891 (0.0008) [2023-12-27 02:56:08,461][105692] Updated weights for policy 0, policy_version 1584113 (0.0010) [2023-12-27 02:56:08,475][105620] Updated weights for policy 1, policy_version 1587901 (0.0006) [2023-12-27 02:56:08,522][105620] Updated weights for policy 1, policy_version 1587911 (0.0008) [2023-12-27 02:56:08,523][105692] Updated weights for policy 0, policy_version 1584123 (0.0010) [2023-12-27 02:56:08,582][105692] Updated weights for policy 0, policy_version 1584133 (0.0010) [2023-12-27 02:56:09,300][105620] Updated weights for policy 1, policy_version 1587921 (0.0008) [2023-12-27 02:56:09,330][105692] Updated weights for policy 0, policy_version 1584143 (0.0011) [2023-12-27 02:56:09,363][105620] Updated weights for policy 1, policy_version 1587931 (0.0006) [2023-12-27 02:56:09,398][105692] Updated weights for policy 0, policy_version 1584153 (0.0011) [2023-12-27 02:56:09,427][105620] Updated weights for policy 1, policy_version 1587941 (0.0007) [2023-12-27 02:56:09,463][105692] Updated weights for policy 0, policy_version 1584163 (0.0011) [2023-12-27 02:56:10,189][105620] Updated weights for policy 1, policy_version 1587951 (0.0009) [2023-12-27 02:56:10,250][105692] Updated weights for policy 0, policy_version 1584173 (0.0009) [2023-12-27 02:56:10,252][105620] Updated weights for policy 1, policy_version 1587961 (0.0008) [2023-12-27 02:56:10,315][105692] Updated weights for policy 0, policy_version 1584183 (0.0006) [2023-12-27 02:56:10,319][105620] Updated weights for policy 1, policy_version 1587971 (0.0011) [2023-12-27 02:56:10,378][105692] Updated weights for policy 0, policy_version 1584193 (0.0006) [2023-12-27 02:56:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 812187648. Throughput: 0: 9659.4, 1: 9874.0. Samples: 812201680. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:56:11,063][104569] Avg episode reward: [(0, '8623.497'), (1, '8542.834')] [2023-12-27 02:56:11,082][105692] Updated weights for policy 0, policy_version 1584203 (0.0010) [2023-12-27 02:56:11,147][105692] Updated weights for policy 0, policy_version 1584213 (0.0010) [2023-12-27 02:56:11,160][105620] Updated weights for policy 1, policy_version 1587981 (0.0010) [2023-12-27 02:56:11,212][105692] Updated weights for policy 0, policy_version 1584223 (0.0008) [2023-12-27 02:56:11,223][105620] Updated weights for policy 1, policy_version 1587991 (0.0007) [2023-12-27 02:56:11,291][105620] Updated weights for policy 1, policy_version 1588001 (0.0009) [2023-12-27 02:56:11,889][105692] Updated weights for policy 0, policy_version 1584233 (0.0008) [2023-12-27 02:56:11,951][105692] Updated weights for policy 0, policy_version 1584243 (0.0009) [2023-12-27 02:56:12,011][105692] Updated weights for policy 0, policy_version 1584253 (0.0010) [2023-12-27 02:56:12,061][105692] Updated weights for policy 0, policy_version 1584263 (0.0010) [2023-12-27 02:56:12,093][105620] Updated weights for policy 1, policy_version 1588011 (0.0008) [2023-12-27 02:56:12,156][105620] Updated weights for policy 1, policy_version 1588021 (0.0007) [2023-12-27 02:56:12,213][105620] Updated weights for policy 1, policy_version 1588031 (0.0005) [2023-12-27 02:56:12,896][105692] Updated weights for policy 0, policy_version 1584273 (0.0007) [2023-12-27 02:56:12,902][105620] Updated weights for policy 1, policy_version 1588041 (0.0008) [2023-12-27 02:56:12,957][105692] Updated weights for policy 0, policy_version 1584283 (0.0006) [2023-12-27 02:56:12,964][105620] Updated weights for policy 1, policy_version 1588051 (0.0008) [2023-12-27 02:56:13,012][105692] Updated weights for policy 0, policy_version 1584293 (0.0007) [2023-12-27 02:56:13,027][105620] Updated weights for policy 1, policy_version 1588061 (0.0007) [2023-12-27 02:56:13,089][105620] Updated weights for policy 1, policy_version 1588071 (0.0008) [2023-12-27 02:56:13,686][105692] Updated weights for policy 0, policy_version 1584303 (0.0007) [2023-12-27 02:56:13,737][105692] Updated weights for policy 0, policy_version 1584313 (0.0005) [2023-12-27 02:56:13,787][105692] Updated weights for policy 0, policy_version 1584323 (0.0005) [2023-12-27 02:56:13,885][105620] Updated weights for policy 1, policy_version 1588081 (0.0010) [2023-12-27 02:56:13,957][105620] Updated weights for policy 1, policy_version 1588091 (0.0010) [2023-12-27 02:56:14,023][105620] Updated weights for policy 1, policy_version 1588101 (0.0010) [2023-12-27 02:56:14,297][105692] Updated weights for policy 0, policy_version 1584333 (0.0005) [2023-12-27 02:56:14,358][105692] Updated weights for policy 0, policy_version 1584343 (0.0005) [2023-12-27 02:56:14,426][105692] Updated weights for policy 0, policy_version 1584353 (0.0005) [2023-12-27 02:56:14,934][105620] Updated weights for policy 1, policy_version 1588111 (0.0009) [2023-12-27 02:56:14,955][105692] Updated weights for policy 0, policy_version 1584363 (0.0007) [2023-12-27 02:56:14,996][105620] Updated weights for policy 1, policy_version 1588121 (0.0009) [2023-12-27 02:56:15,018][105692] Updated weights for policy 0, policy_version 1584373 (0.0011) [2023-12-27 02:56:15,060][105620] Updated weights for policy 1, policy_version 1588131 (0.0008) [2023-12-27 02:56:15,082][105692] Updated weights for policy 0, policy_version 1584383 (0.0011) [2023-12-27 02:56:15,702][105620] Updated weights for policy 1, policy_version 1588141 (0.0007) [2023-12-27 02:56:15,751][105620] Updated weights for policy 1, policy_version 1588151 (0.0008) [2023-12-27 02:56:15,813][105620] Updated weights for policy 1, policy_version 1588161 (0.0008) [2023-12-27 02:56:15,826][105692] Updated weights for policy 0, policy_version 1584393 (0.0011) [2023-12-27 02:56:15,892][105692] Updated weights for policy 0, policy_version 1584403 (0.0010) [2023-12-27 02:56:15,947][105692] Updated weights for policy 0, policy_version 1584413 (0.0008) [2023-12-27 02:56:16,002][105692] Updated weights for policy 0, policy_version 1584423 (0.0005) [2023-12-27 02:56:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 812294144. Throughput: 0: 9621.4, 1: 9787.0. Samples: 812257516. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:56:16,063][104569] Avg episode reward: [(0, '8259.439'), (1, '8641.547')] [2023-12-27 02:56:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001584424_405667840.pth... [2023-12-27 02:56:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001588168_406626304.pth... [2023-12-27 02:56:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001583304_405381120.pth [2023-12-27 02:56:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001587016_406331392.pth [2023-12-27 02:56:16,559][105620] Updated weights for policy 1, policy_version 1588171 (0.0007) [2023-12-27 02:56:16,611][105620] Updated weights for policy 1, policy_version 1588181 (0.0010) [2023-12-27 02:56:16,620][105692] Updated weights for policy 0, policy_version 1584433 (0.0006) [2023-12-27 02:56:16,662][105620] Updated weights for policy 1, policy_version 1588191 (0.0009) [2023-12-27 02:56:16,674][105692] Updated weights for policy 0, policy_version 1584443 (0.0005) [2023-12-27 02:56:16,734][105692] Updated weights for policy 0, policy_version 1584453 (0.0005) [2023-12-27 02:56:17,259][105692] Updated weights for policy 0, policy_version 1584463 (0.0005) [2023-12-27 02:56:17,314][105692] Updated weights for policy 0, policy_version 1584473 (0.0009) [2023-12-27 02:56:17,366][105692] Updated weights for policy 0, policy_version 1584483 (0.0010) [2023-12-27 02:56:17,544][105620] Updated weights for policy 1, policy_version 1588201 (0.0009) [2023-12-27 02:56:17,609][105620] Updated weights for policy 1, policy_version 1588211 (0.0010) [2023-12-27 02:56:17,663][105620] Updated weights for policy 1, policy_version 1588221 (0.0010) [2023-12-27 02:56:17,721][105620] Updated weights for policy 1, policy_version 1588231 (0.0010) [2023-12-27 02:56:18,100][105692] Updated weights for policy 0, policy_version 1584493 (0.0008) [2023-12-27 02:56:18,150][105692] Updated weights for policy 0, policy_version 1584503 (0.0009) [2023-12-27 02:56:18,205][105692] Updated weights for policy 0, policy_version 1584515 (0.0010) [2023-12-27 02:56:18,379][105620] Updated weights for policy 1, policy_version 1588241 (0.0010) [2023-12-27 02:56:18,437][105620] Updated weights for policy 1, policy_version 1588251 (0.0011) [2023-12-27 02:56:18,500][105620] Updated weights for policy 1, policy_version 1588261 (0.0010) [2023-12-27 02:56:18,979][105692] Updated weights for policy 0, policy_version 1584525 (0.0010) [2023-12-27 02:56:19,039][105692] Updated weights for policy 0, policy_version 1584535 (0.0010) [2023-12-27 02:56:19,099][105692] Updated weights for policy 0, policy_version 1584545 (0.0010) [2023-12-27 02:56:19,196][105620] Updated weights for policy 1, policy_version 1588271 (0.0010) [2023-12-27 02:56:19,260][105620] Updated weights for policy 1, policy_version 1588281 (0.0009) [2023-12-27 02:56:19,326][105620] Updated weights for policy 1, policy_version 1588291 (0.0011) [2023-12-27 02:56:19,748][105692] Updated weights for policy 0, policy_version 1584555 (0.0009) [2023-12-27 02:56:19,816][105692] Updated weights for policy 0, policy_version 1584565 (0.0007) [2023-12-27 02:56:19,884][105692] Updated weights for policy 0, policy_version 1584575 (0.0010) [2023-12-27 02:56:20,047][105620] Updated weights for policy 1, policy_version 1588301 (0.0008) [2023-12-27 02:56:20,102][105620] Updated weights for policy 1, policy_version 1588311 (0.0006) [2023-12-27 02:56:20,158][105620] Updated weights for policy 1, policy_version 1588321 (0.0007) [2023-12-27 02:56:20,627][105692] Updated weights for policy 0, policy_version 1584585 (0.0010) [2023-12-27 02:56:20,691][105692] Updated weights for policy 0, policy_version 1584595 (0.0007) [2023-12-27 02:56:20,758][105692] Updated weights for policy 0, policy_version 1584605 (0.0006) [2023-12-27 02:56:20,820][105692] Updated weights for policy 0, policy_version 1584615 (0.0006) [2023-12-27 02:56:20,881][105620] Updated weights for policy 1, policy_version 1588331 (0.0010) [2023-12-27 02:56:20,944][105620] Updated weights for policy 1, policy_version 1588341 (0.0008) [2023-12-27 02:56:21,004][105620] Updated weights for policy 1, policy_version 1588351 (0.0008) [2023-12-27 02:56:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.9, 300 sec: 19494.2). Total num frames: 812384256. Throughput: 0: 9705.9, 1: 9730.3. Samples: 812377988. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:56:21,063][104569] Avg episode reward: [(0, '7991.839'), (1, '9090.249')] [2023-12-27 02:56:21,532][105692] Updated weights for policy 0, policy_version 1584625 (0.0006) [2023-12-27 02:56:21,592][105692] Updated weights for policy 0, policy_version 1584635 (0.0008) [2023-12-27 02:56:21,653][105692] Updated weights for policy 0, policy_version 1584645 (0.0009) [2023-12-27 02:56:21,817][105620] Updated weights for policy 1, policy_version 1588361 (0.0009) [2023-12-27 02:56:21,877][105620] Updated weights for policy 1, policy_version 1588371 (0.0011) [2023-12-27 02:56:21,942][105620] Updated weights for policy 1, policy_version 1588381 (0.0011) [2023-12-27 02:56:21,995][105620] Updated weights for policy 1, policy_version 1588391 (0.0011) [2023-12-27 02:56:22,368][105692] Updated weights for policy 0, policy_version 1584655 (0.0008) [2023-12-27 02:56:22,432][105692] Updated weights for policy 0, policy_version 1584665 (0.0007) [2023-12-27 02:56:22,493][105692] Updated weights for policy 0, policy_version 1584675 (0.0008) [2023-12-27 02:56:22,774][105620] Updated weights for policy 1, policy_version 1588401 (0.0011) [2023-12-27 02:56:22,831][105620] Updated weights for policy 1, policy_version 1588411 (0.0011) [2023-12-27 02:56:22,889][105620] Updated weights for policy 1, policy_version 1588421 (0.0009) [2023-12-27 02:56:23,181][105692] Updated weights for policy 0, policy_version 1584685 (0.0009) [2023-12-27 02:56:23,247][105692] Updated weights for policy 0, policy_version 1584695 (0.0007) [2023-12-27 02:56:23,306][105692] Updated weights for policy 0, policy_version 1584705 (0.0005) [2023-12-27 02:56:23,631][105620] Updated weights for policy 1, policy_version 1588432 (0.0006) [2023-12-27 02:56:23,695][105620] Updated weights for policy 1, policy_version 1588442 (0.0005) [2023-12-27 02:56:23,765][105620] Updated weights for policy 1, policy_version 1588452 (0.0006) [2023-12-27 02:56:23,841][105692] Updated weights for policy 0, policy_version 1584715 (0.0005) [2023-12-27 02:56:23,909][105692] Updated weights for policy 0, policy_version 1584725 (0.0005) [2023-12-27 02:56:23,960][105692] Updated weights for policy 0, policy_version 1584735 (0.0005) [2023-12-27 02:56:24,395][105620] Updated weights for policy 1, policy_version 1588462 (0.0008) [2023-12-27 02:56:24,449][105620] Updated weights for policy 1, policy_version 1588472 (0.0010) [2023-12-27 02:56:24,475][105692] Updated weights for policy 0, policy_version 1584745 (0.0006) [2023-12-27 02:56:24,510][105620] Updated weights for policy 1, policy_version 1588482 (0.0009) [2023-12-27 02:56:24,530][105692] Updated weights for policy 0, policy_version 1584755 (0.0011) [2023-12-27 02:56:24,582][105692] Updated weights for policy 0, policy_version 1584765 (0.0011) [2023-12-27 02:56:24,644][105692] Updated weights for policy 0, policy_version 1584775 (0.0008) [2023-12-27 02:56:25,237][105692] Updated weights for policy 0, policy_version 1584785 (0.0005) [2023-12-27 02:56:25,263][105620] Updated weights for policy 1, policy_version 1588492 (0.0009) [2023-12-27 02:56:25,296][105692] Updated weights for policy 0, policy_version 1584795 (0.0009) [2023-12-27 02:56:25,322][105620] Updated weights for policy 1, policy_version 1588502 (0.0006) [2023-12-27 02:56:25,347][105692] Updated weights for policy 0, policy_version 1584805 (0.0010) [2023-12-27 02:56:25,381][105620] Updated weights for policy 1, policy_version 1588512 (0.0006) [2023-12-27 02:56:26,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 812482560. Throughput: 0: 9899.5, 1: 9645.4. Samples: 812497956. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:56:26,062][104569] Avg episode reward: [(0, '7892.319'), (1, '8912.051')] [2023-12-27 02:56:26,066][105620] Updated weights for policy 1, policy_version 1588522 (0.0010) [2023-12-27 02:56:26,069][105692] Updated weights for policy 0, policy_version 1584815 (0.0011) [2023-12-27 02:56:26,124][105620] Updated weights for policy 1, policy_version 1588532 (0.0005) [2023-12-27 02:56:26,128][105692] Updated weights for policy 0, policy_version 1584825 (0.0010) [2023-12-27 02:56:26,170][105620] Updated weights for policy 1, policy_version 1588542 (0.0008) [2023-12-27 02:56:26,186][105692] Updated weights for policy 0, policy_version 1584835 (0.0010) [2023-12-27 02:56:26,226][105620] Updated weights for policy 1, policy_version 1588552 (0.0010) [2023-12-27 02:56:26,839][105620] Updated weights for policy 1, policy_version 1588562 (0.0006) [2023-12-27 02:56:26,839][105692] Updated weights for policy 0, policy_version 1584845 (0.0008) [2023-12-27 02:56:26,884][105692] Updated weights for policy 0, policy_version 1584855 (0.0010) [2023-12-27 02:56:26,890][105620] Updated weights for policy 1, policy_version 1588572 (0.0005) [2023-12-27 02:56:26,935][105692] Updated weights for policy 0, policy_version 1584865 (0.0010) [2023-12-27 02:56:26,945][105620] Updated weights for policy 1, policy_version 1588582 (0.0005) [2023-12-27 02:56:27,445][105620] Updated weights for policy 1, policy_version 1588592 (0.0005) [2023-12-27 02:56:27,501][105620] Updated weights for policy 1, policy_version 1588602 (0.0005) [2023-12-27 02:56:27,565][105620] Updated weights for policy 1, policy_version 1588612 (0.0005) [2023-12-27 02:56:27,653][105692] Updated weights for policy 0, policy_version 1584875 (0.0009) [2023-12-27 02:56:27,722][105692] Updated weights for policy 0, policy_version 1584885 (0.0007) [2023-12-27 02:56:27,786][105692] Updated weights for policy 0, policy_version 1584895 (0.0008) [2023-12-27 02:56:28,080][105620] Updated weights for policy 1, policy_version 1588622 (0.0005) [2023-12-27 02:56:28,125][105620] Updated weights for policy 1, policy_version 1588632 (0.0005) [2023-12-27 02:56:28,175][105620] Updated weights for policy 1, policy_version 1588642 (0.0005) [2023-12-27 02:56:28,476][105692] Updated weights for policy 0, policy_version 1584905 (0.0011) [2023-12-27 02:56:28,528][105692] Updated weights for policy 0, policy_version 1584915 (0.0010) [2023-12-27 02:56:28,581][105692] Updated weights for policy 0, policy_version 1584925 (0.0010) [2023-12-27 02:56:28,643][105692] Updated weights for policy 0, policy_version 1584935 (0.0011) [2023-12-27 02:56:28,811][105620] Updated weights for policy 1, policy_version 1588652 (0.0006) [2023-12-27 02:56:28,878][105620] Updated weights for policy 1, policy_version 1588662 (0.0009) [2023-12-27 02:56:28,925][105620] Updated weights for policy 1, policy_version 1588672 (0.0008) [2023-12-27 02:56:29,380][105692] Updated weights for policy 0, policy_version 1584945 (0.0007) [2023-12-27 02:56:29,433][105692] Updated weights for policy 0, policy_version 1584955 (0.0009) [2023-12-27 02:56:29,488][105692] Updated weights for policy 0, policy_version 1584965 (0.0009) [2023-12-27 02:56:29,558][105620] Updated weights for policy 1, policy_version 1588682 (0.0007) [2023-12-27 02:56:29,613][105620] Updated weights for policy 1, policy_version 1588692 (0.0010) [2023-12-27 02:56:29,671][105620] Updated weights for policy 1, policy_version 1588702 (0.0010) [2023-12-27 02:56:29,726][105620] Updated weights for policy 1, policy_version 1588712 (0.0009) [2023-12-27 02:56:30,159][105692] Updated weights for policy 0, policy_version 1584975 (0.0010) [2023-12-27 02:56:30,214][105692] Updated weights for policy 0, policy_version 1584986 (0.0010) [2023-12-27 02:56:30,280][105692] Updated weights for policy 0, policy_version 1584996 (0.0010) [2023-12-27 02:56:30,451][105620] Updated weights for policy 1, policy_version 1588722 (0.0007) [2023-12-27 02:56:30,519][105620] Updated weights for policy 1, policy_version 1588732 (0.0008) [2023-12-27 02:56:30,580][105620] Updated weights for policy 1, policy_version 1588742 (0.0008) [2023-12-27 02:56:31,053][105692] Updated weights for policy 0, policy_version 1585006 (0.0008) [2023-12-27 02:56:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 812589056. Throughput: 0: 9873.4, 1: 9850.0. Samples: 812563308. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:56:31,062][104569] Avg episode reward: [(0, '7707.286'), (1, '8740.070')] [2023-12-27 02:56:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001588744_406773760.pth... [2023-12-27 02:56:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001587592_406478848.pth [2023-12-27 02:56:31,113][105692] Updated weights for policy 0, policy_version 1585016 (0.0006) [2023-12-27 02:56:31,136][105620] Updated weights for policy 1, policy_version 1588752 (0.0008) [2023-12-27 02:56:31,177][105692] Updated weights for policy 0, policy_version 1585026 (0.0007) [2023-12-27 02:56:31,206][105620] Updated weights for policy 1, policy_version 1588762 (0.0007) [2023-12-27 02:56:31,213][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001585032_405823488.pth... [2023-12-27 02:56:31,217][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001583848_405520384.pth [2023-12-27 02:56:31,268][105620] Updated weights for policy 1, policy_version 1588772 (0.0009) [2023-12-27 02:56:31,871][105692] Updated weights for policy 0, policy_version 1585036 (0.0009) [2023-12-27 02:56:31,917][105692] Updated weights for policy 0, policy_version 1585046 (0.0008) [2023-12-27 02:56:31,969][105692] Updated weights for policy 0, policy_version 1585056 (0.0008) [2023-12-27 02:56:31,994][105620] Updated weights for policy 1, policy_version 1588782 (0.0007) [2023-12-27 02:56:32,049][105620] Updated weights for policy 1, policy_version 1588792 (0.0008) [2023-12-27 02:56:32,096][105620] Updated weights for policy 1, policy_version 1588802 (0.0009) [2023-12-27 02:56:32,591][105692] Updated weights for policy 0, policy_version 1585066 (0.0007) [2023-12-27 02:56:32,648][105692] Updated weights for policy 0, policy_version 1585076 (0.0005) [2023-12-27 02:56:32,710][105692] Updated weights for policy 0, policy_version 1585086 (0.0006) [2023-12-27 02:56:32,759][105692] Updated weights for policy 0, policy_version 1585096 (0.0005) [2023-12-27 02:56:32,954][105620] Updated weights for policy 1, policy_version 1588812 (0.0008) [2023-12-27 02:56:33,006][105620] Updated weights for policy 1, policy_version 1588822 (0.0005) [2023-12-27 02:56:33,058][105620] Updated weights for policy 1, policy_version 1588832 (0.0006) [2023-12-27 02:56:33,376][105692] Updated weights for policy 0, policy_version 1585106 (0.0008) [2023-12-27 02:56:33,432][105692] Updated weights for policy 0, policy_version 1585116 (0.0009) [2023-12-27 02:56:33,498][105692] Updated weights for policy 0, policy_version 1585126 (0.0010) [2023-12-27 02:56:33,751][105620] Updated weights for policy 1, policy_version 1588842 (0.0008) [2023-12-27 02:56:33,801][105620] Updated weights for policy 1, policy_version 1588852 (0.0009) [2023-12-27 02:56:33,855][105620] Updated weights for policy 1, policy_version 1588862 (0.0008) [2023-12-27 02:56:33,913][105620] Updated weights for policy 1, policy_version 1588872 (0.0009) [2023-12-27 02:56:34,233][105692] Updated weights for policy 0, policy_version 1585136 (0.0008) [2023-12-27 02:56:34,287][105692] Updated weights for policy 0, policy_version 1585146 (0.0008) [2023-12-27 02:56:34,357][105692] Updated weights for policy 0, policy_version 1585156 (0.0009) [2023-12-27 02:56:34,633][105620] Updated weights for policy 1, policy_version 1588882 (0.0009) [2023-12-27 02:56:34,697][105620] Updated weights for policy 1, policy_version 1588892 (0.0009) [2023-12-27 02:56:34,764][105620] Updated weights for policy 1, policy_version 1588902 (0.0010) [2023-12-27 02:56:35,035][105692] Updated weights for policy 0, policy_version 1585166 (0.0007) [2023-12-27 02:56:35,100][105692] Updated weights for policy 0, policy_version 1585176 (0.0008) [2023-12-27 02:56:35,164][105692] Updated weights for policy 0, policy_version 1585186 (0.0009) [2023-12-27 02:56:35,424][105620] Updated weights for policy 1, policy_version 1588912 (0.0009) [2023-12-27 02:56:35,474][105620] Updated weights for policy 1, policy_version 1588922 (0.0008) [2023-12-27 02:56:35,537][105620] Updated weights for policy 1, policy_version 1588932 (0.0009) [2023-12-27 02:56:35,908][105692] Updated weights for policy 0, policy_version 1585196 (0.0010) [2023-12-27 02:56:35,956][105692] Updated weights for policy 0, policy_version 1585206 (0.0009) [2023-12-27 02:56:36,003][105692] Updated weights for policy 0, policy_version 1585216 (0.0009) [2023-12-27 02:56:36,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 812695552. Throughput: 0: 9844.2, 1: 9778.7. Samples: 812682256. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:56:36,062][104569] Avg episode reward: [(0, '7978.527'), (1, '9017.886')] [2023-12-27 02:56:36,307][105620] Updated weights for policy 1, policy_version 1588942 (0.0010) [2023-12-27 02:56:36,376][105620] Updated weights for policy 1, policy_version 1588952 (0.0010) [2023-12-27 02:56:36,436][105620] Updated weights for policy 1, policy_version 1588962 (0.0011) [2023-12-27 02:56:36,818][105692] Updated weights for policy 0, policy_version 1585226 (0.0009) [2023-12-27 02:56:36,873][105692] Updated weights for policy 0, policy_version 1585236 (0.0011) [2023-12-27 02:56:36,930][105692] Updated weights for policy 0, policy_version 1585246 (0.0011) [2023-12-27 02:56:36,988][105692] Updated weights for policy 0, policy_version 1585256 (0.0009) [2023-12-27 02:56:37,085][105620] Updated weights for policy 1, policy_version 1588972 (0.0011) [2023-12-27 02:56:37,148][105620] Updated weights for policy 1, policy_version 1588982 (0.0011) [2023-12-27 02:56:37,214][105620] Updated weights for policy 1, policy_version 1588992 (0.0010) [2023-12-27 02:56:37,701][105692] Updated weights for policy 0, policy_version 1585266 (0.0011) [2023-12-27 02:56:37,756][105692] Updated weights for policy 0, policy_version 1585276 (0.0010) [2023-12-27 02:56:37,809][105692] Updated weights for policy 0, policy_version 1585286 (0.0008) [2023-12-27 02:56:37,960][105620] Updated weights for policy 1, policy_version 1589002 (0.0011) [2023-12-27 02:56:38,016][105620] Updated weights for policy 1, policy_version 1589012 (0.0010) [2023-12-27 02:56:38,070][105620] Updated weights for policy 1, policy_version 1589022 (0.0010) [2023-12-27 02:56:38,119][105620] Updated weights for policy 1, policy_version 1589032 (0.0010) [2023-12-27 02:56:38,483][105692] Updated weights for policy 0, policy_version 1585296 (0.0007) [2023-12-27 02:56:38,541][105692] Updated weights for policy 0, policy_version 1585306 (0.0006) [2023-12-27 02:56:38,593][105692] Updated weights for policy 0, policy_version 1585316 (0.0010) [2023-12-27 02:56:38,854][105620] Updated weights for policy 1, policy_version 1589042 (0.0005) [2023-12-27 02:56:38,915][105620] Updated weights for policy 1, policy_version 1589052 (0.0008) [2023-12-27 02:56:38,978][105620] Updated weights for policy 1, policy_version 1589062 (0.0010) [2023-12-27 02:56:39,348][105692] Updated weights for policy 0, policy_version 1585326 (0.0011) [2023-12-27 02:56:39,420][105692] Updated weights for policy 0, policy_version 1585336 (0.0008) [2023-12-27 02:56:39,481][105692] Updated weights for policy 0, policy_version 1585346 (0.0010) [2023-12-27 02:56:39,642][105620] Updated weights for policy 1, policy_version 1589072 (0.0011) [2023-12-27 02:56:39,708][105620] Updated weights for policy 1, policy_version 1589082 (0.0010) [2023-12-27 02:56:39,770][105620] Updated weights for policy 1, policy_version 1589092 (0.0010) [2023-12-27 02:56:40,244][105692] Updated weights for policy 0, policy_version 1585356 (0.0009) [2023-12-27 02:56:40,305][105692] Updated weights for policy 0, policy_version 1585366 (0.0008) [2023-12-27 02:56:40,372][105692] Updated weights for policy 0, policy_version 1585376 (0.0009) [2023-12-27 02:56:40,418][105620] Updated weights for policy 1, policy_version 1589102 (0.0008) [2023-12-27 02:56:40,479][105620] Updated weights for policy 1, policy_version 1589112 (0.0005) [2023-12-27 02:56:40,541][105620] Updated weights for policy 1, policy_version 1589122 (0.0006) [2023-12-27 02:56:41,008][105692] Updated weights for policy 0, policy_version 1585386 (0.0008) [2023-12-27 02:56:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 812785664. Throughput: 0: 9874.9, 1: 9816.3. Samples: 812799160. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:56:41,063][104569] Avg episode reward: [(0, '8529.262'), (1, '8928.737')] [2023-12-27 02:56:41,082][105692] Updated weights for policy 0, policy_version 1585396 (0.0006) [2023-12-27 02:56:41,152][105692] Updated weights for policy 0, policy_version 1585406 (0.0007) [2023-12-27 02:56:41,206][105692] Updated weights for policy 0, policy_version 1585416 (0.0008) [2023-12-27 02:56:41,277][105620] Updated weights for policy 1, policy_version 1589132 (0.0009) [2023-12-27 02:56:41,341][105620] Updated weights for policy 1, policy_version 1589142 (0.0008) [2023-12-27 02:56:41,411][105620] Updated weights for policy 1, policy_version 1589152 (0.0008) [2023-12-27 02:56:41,982][105692] Updated weights for policy 0, policy_version 1585426 (0.0009) [2023-12-27 02:56:42,044][105692] Updated weights for policy 0, policy_version 1585436 (0.0008) [2023-12-27 02:56:42,114][105692] Updated weights for policy 0, policy_version 1585446 (0.0006) [2023-12-27 02:56:42,144][105620] Updated weights for policy 1, policy_version 1589162 (0.0009) [2023-12-27 02:56:42,198][105620] Updated weights for policy 1, policy_version 1589172 (0.0010) [2023-12-27 02:56:42,252][105620] Updated weights for policy 1, policy_version 1589182 (0.0009) [2023-12-27 02:56:42,317][105620] Updated weights for policy 1, policy_version 1589192 (0.0008) [2023-12-27 02:56:42,808][105692] Updated weights for policy 0, policy_version 1585456 (0.0008) [2023-12-27 02:56:42,869][105692] Updated weights for policy 0, policy_version 1585466 (0.0009) [2023-12-27 02:56:42,917][105692] Updated weights for policy 0, policy_version 1585476 (0.0008) [2023-12-27 02:56:43,093][105620] Updated weights for policy 1, policy_version 1589202 (0.0010) [2023-12-27 02:56:43,141][105620] Updated weights for policy 1, policy_version 1589212 (0.0010) [2023-12-27 02:56:43,194][105620] Updated weights for policy 1, policy_version 1589222 (0.0010) [2023-12-27 02:56:43,688][105692] Updated weights for policy 0, policy_version 1585486 (0.0009) [2023-12-27 02:56:43,735][105692] Updated weights for policy 0, policy_version 1585496 (0.0008) [2023-12-27 02:56:43,783][105692] Updated weights for policy 0, policy_version 1585506 (0.0007) [2023-12-27 02:56:43,968][105620] Updated weights for policy 1, policy_version 1589232 (0.0010) [2023-12-27 02:56:44,023][105620] Updated weights for policy 1, policy_version 1589242 (0.0010) [2023-12-27 02:56:44,082][105620] Updated weights for policy 1, policy_version 1589252 (0.0010) [2023-12-27 02:56:44,568][105692] Updated weights for policy 0, policy_version 1585516 (0.0008) [2023-12-27 02:56:44,616][105692] Updated weights for policy 0, policy_version 1585526 (0.0008) [2023-12-27 02:56:44,667][105692] Updated weights for policy 0, policy_version 1585536 (0.0008) [2023-12-27 02:56:44,822][105620] Updated weights for policy 1, policy_version 1589262 (0.0011) [2023-12-27 02:56:44,878][105620] Updated weights for policy 1, policy_version 1589272 (0.0011) [2023-12-27 02:56:44,930][105620] Updated weights for policy 1, policy_version 1589282 (0.0010) [2023-12-27 02:56:45,497][105692] Updated weights for policy 0, policy_version 1585546 (0.0008) [2023-12-27 02:56:45,559][105692] Updated weights for policy 0, policy_version 1585556 (0.0008) [2023-12-27 02:56:45,615][105692] Updated weights for policy 0, policy_version 1585566 (0.0008) [2023-12-27 02:56:45,667][105692] Updated weights for policy 0, policy_version 1585576 (0.0007) [2023-12-27 02:56:45,677][105620] Updated weights for policy 1, policy_version 1589292 (0.0010) [2023-12-27 02:56:45,734][105620] Updated weights for policy 1, policy_version 1589302 (0.0010) [2023-12-27 02:56:45,785][105620] Updated weights for policy 1, policy_version 1589312 (0.0010) [2023-12-27 02:56:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 812883968. Throughput: 0: 9871.0, 1: 9763.5. Samples: 812855344. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:56:46,062][104569] Avg episode reward: [(0, '8621.512'), (1, '8289.524')] [2023-12-27 02:56:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001585576_405962752.pth... [2023-12-27 02:56:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001589320_406921216.pth... [2023-12-27 02:56:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001588168_406626304.pth [2023-12-27 02:56:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001584424_405667840.pth [2023-12-27 02:56:46,410][105692] Updated weights for policy 0, policy_version 1585586 (0.0008) [2023-12-27 02:56:46,465][105692] Updated weights for policy 0, policy_version 1585596 (0.0008) [2023-12-27 02:56:46,524][105692] Updated weights for policy 0, policy_version 1585606 (0.0008) [2023-12-27 02:56:46,533][105620] Updated weights for policy 1, policy_version 1589322 (0.0010) [2023-12-27 02:56:46,589][105620] Updated weights for policy 1, policy_version 1589332 (0.0010) [2023-12-27 02:56:46,657][105620] Updated weights for policy 1, policy_version 1589342 (0.0010) [2023-12-27 02:56:46,712][105620] Updated weights for policy 1, policy_version 1589352 (0.0010) [2023-12-27 02:56:47,269][105692] Updated weights for policy 0, policy_version 1585616 (0.0008) [2023-12-27 02:56:47,337][105692] Updated weights for policy 0, policy_version 1585626 (0.0008) [2023-12-27 02:56:47,344][105620] Updated weights for policy 1, policy_version 1589362 (0.0006) [2023-12-27 02:56:47,398][105692] Updated weights for policy 0, policy_version 1585636 (0.0007) [2023-12-27 02:56:47,401][105620] Updated weights for policy 1, policy_version 1589372 (0.0007) [2023-12-27 02:56:47,460][105620] Updated weights for policy 1, policy_version 1589382 (0.0008) [2023-12-27 02:56:48,104][105692] Updated weights for policy 0, policy_version 1585646 (0.0008) [2023-12-27 02:56:48,152][105692] Updated weights for policy 0, policy_version 1585656 (0.0007) [2023-12-27 02:56:48,177][105620] Updated weights for policy 1, policy_version 1589392 (0.0009) [2023-12-27 02:56:48,198][105692] Updated weights for policy 0, policy_version 1585666 (0.0007) [2023-12-27 02:56:48,238][105620] Updated weights for policy 1, policy_version 1589402 (0.0008) [2023-12-27 02:56:48,300][105620] Updated weights for policy 1, policy_version 1589412 (0.0009) [2023-12-27 02:56:48,829][105692] Updated weights for policy 0, policy_version 1585676 (0.0006) [2023-12-27 02:56:48,891][105692] Updated weights for policy 0, policy_version 1585686 (0.0006) [2023-12-27 02:56:48,950][105692] Updated weights for policy 0, policy_version 1585696 (0.0008) [2023-12-27 02:56:49,098][105620] Updated weights for policy 1, policy_version 1589422 (0.0008) [2023-12-27 02:56:49,152][105620] Updated weights for policy 1, policy_version 1589432 (0.0009) [2023-12-27 02:56:49,208][105620] Updated weights for policy 1, policy_version 1589442 (0.0009) [2023-12-27 02:56:49,532][105692] Updated weights for policy 0, policy_version 1585706 (0.0006) [2023-12-27 02:56:49,587][105692] Updated weights for policy 0, policy_version 1585716 (0.0009) [2023-12-27 02:56:49,638][105692] Updated weights for policy 0, policy_version 1585726 (0.0009) [2023-12-27 02:56:49,692][105692] Updated weights for policy 0, policy_version 1585736 (0.0008) [2023-12-27 02:56:49,997][105620] Updated weights for policy 1, policy_version 1589452 (0.0009) [2023-12-27 02:56:50,067][105620] Updated weights for policy 1, policy_version 1589462 (0.0009) [2023-12-27 02:56:50,129][105620] Updated weights for policy 1, policy_version 1589472 (0.0009) [2023-12-27 02:56:50,445][105692] Updated weights for policy 0, policy_version 1585746 (0.0009) [2023-12-27 02:56:50,512][105692] Updated weights for policy 0, policy_version 1585756 (0.0008) [2023-12-27 02:56:50,575][105692] Updated weights for policy 0, policy_version 1585766 (0.0010) [2023-12-27 02:56:50,902][105620] Updated weights for policy 1, policy_version 1589482 (0.0009) [2023-12-27 02:56:50,961][105620] Updated weights for policy 1, policy_version 1589492 (0.0008) [2023-12-27 02:56:51,026][105620] Updated weights for policy 1, policy_version 1589502 (0.0009) [2023-12-27 02:56:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 812974080. Throughput: 0: 9949.5, 1: 9754.6. Samples: 812970812. Policy #0 lag: (min: 3.0, avg: 3.6, max: 16.0) [2023-12-27 02:56:51,062][104569] Avg episode reward: [(0, '8532.616'), (1, '8459.907')] [2023-12-27 02:56:51,090][105620] Updated weights for policy 1, policy_version 1589512 (0.0008) [2023-12-27 02:56:51,247][105692] Updated weights for policy 0, policy_version 1585776 (0.0009) [2023-12-27 02:56:51,311][105692] Updated weights for policy 0, policy_version 1585786 (0.0009) [2023-12-27 02:56:51,389][105692] Updated weights for policy 0, policy_version 1585796 (0.0009) [2023-12-27 02:56:51,818][105620] Updated weights for policy 1, policy_version 1589522 (0.0008) [2023-12-27 02:56:51,885][105620] Updated weights for policy 1, policy_version 1589532 (0.0008) [2023-12-27 02:56:51,947][105620] Updated weights for policy 1, policy_version 1589542 (0.0008) [2023-12-27 02:56:52,166][105692] Updated weights for policy 0, policy_version 1585806 (0.0007) [2023-12-27 02:56:52,215][105692] Updated weights for policy 0, policy_version 1585816 (0.0005) [2023-12-27 02:56:52,274][105692] Updated weights for policy 0, policy_version 1585826 (0.0007) [2023-12-27 02:56:52,701][105620] Updated weights for policy 1, policy_version 1589552 (0.0007) [2023-12-27 02:56:52,763][105620] Updated weights for policy 1, policy_version 1589562 (0.0006) [2023-12-27 02:56:52,821][105620] Updated weights for policy 1, policy_version 1589572 (0.0006) [2023-12-27 02:56:53,018][105692] Updated weights for policy 0, policy_version 1585836 (0.0007) [2023-12-27 02:56:53,076][105692] Updated weights for policy 0, policy_version 1585847 (0.0010) [2023-12-27 02:56:53,129][105692] Updated weights for policy 0, policy_version 1585858 (0.0010) [2023-12-27 02:56:53,354][105620] Updated weights for policy 1, policy_version 1589582 (0.0006) [2023-12-27 02:56:53,409][105620] Updated weights for policy 1, policy_version 1589592 (0.0006) [2023-12-27 02:56:53,464][105620] Updated weights for policy 1, policy_version 1589602 (0.0006) [2023-12-27 02:56:54,011][105620] Updated weights for policy 1, policy_version 1589612 (0.0005) [2023-12-27 02:56:54,060][105692] Updated weights for policy 0, policy_version 1585869 (0.0010) [2023-12-27 02:56:54,073][105620] Updated weights for policy 1, policy_version 1589622 (0.0005) [2023-12-27 02:56:54,122][105692] Updated weights for policy 0, policy_version 1585879 (0.0009) [2023-12-27 02:56:54,124][105620] Updated weights for policy 1, policy_version 1589632 (0.0006) [2023-12-27 02:56:54,184][105692] Updated weights for policy 0, policy_version 1585889 (0.0008) [2023-12-27 02:56:54,753][105692] Updated weights for policy 0, policy_version 1585899 (0.0006) [2023-12-27 02:56:54,813][105692] Updated weights for policy 0, policy_version 1585909 (0.0005) [2023-12-27 02:56:54,869][105620] Updated weights for policy 1, policy_version 1589642 (0.0009) [2023-12-27 02:56:54,870][105692] Updated weights for policy 0, policy_version 1585919 (0.0006) [2023-12-27 02:56:54,921][105620] Updated weights for policy 1, policy_version 1589652 (0.0006) [2023-12-27 02:56:54,969][105620] Updated weights for policy 1, policy_version 1589662 (0.0009) [2023-12-27 02:56:55,022][105620] Updated weights for policy 1, policy_version 1589672 (0.0008) [2023-12-27 02:56:55,485][105692] Updated weights for policy 0, policy_version 1585929 (0.0010) [2023-12-27 02:56:55,539][105692] Updated weights for policy 0, policy_version 1585939 (0.0006) [2023-12-27 02:56:55,595][105692] Updated weights for policy 0, policy_version 1585949 (0.0010) [2023-12-27 02:56:55,647][105620] Updated weights for policy 1, policy_version 1589682 (0.0010) [2023-12-27 02:56:55,653][105692] Updated weights for policy 0, policy_version 1585959 (0.0010) [2023-12-27 02:56:55,692][105620] Updated weights for policy 1, policy_version 1589692 (0.0010) [2023-12-27 02:56:55,736][105620] Updated weights for policy 1, policy_version 1589702 (0.0010) [2023-12-27 02:56:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 813080576. Throughput: 0: 9964.7, 1: 9761.1. Samples: 813089344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:56:56,063][104569] Avg episode reward: [(0, '8809.662'), (1, '9264.963')] [2023-12-27 02:56:56,282][105692] Updated weights for policy 0, policy_version 1585969 (0.0010) [2023-12-27 02:56:56,338][105692] Updated weights for policy 0, policy_version 1585979 (0.0011) [2023-12-27 02:56:56,380][105620] Updated weights for policy 1, policy_version 1589712 (0.0006) [2023-12-27 02:56:56,390][105692] Updated weights for policy 0, policy_version 1585989 (0.0008) [2023-12-27 02:56:56,445][105620] Updated weights for policy 1, policy_version 1589722 (0.0006) [2023-12-27 02:56:56,494][105620] Updated weights for policy 1, policy_version 1589732 (0.0006) [2023-12-27 02:56:56,982][105692] Updated weights for policy 0, policy_version 1585999 (0.0008) [2023-12-27 02:56:57,035][105692] Updated weights for policy 0, policy_version 1586009 (0.0010) [2023-12-27 02:56:57,102][105692] Updated weights for policy 0, policy_version 1586020 (0.0009) [2023-12-27 02:56:57,156][105620] Updated weights for policy 1, policy_version 1589742 (0.0007) [2023-12-27 02:56:57,200][105620] Updated weights for policy 1, policy_version 1589752 (0.0009) [2023-12-27 02:56:57,253][105620] Updated weights for policy 1, policy_version 1589762 (0.0005) [2023-12-27 02:56:57,792][105692] Updated weights for policy 0, policy_version 1586030 (0.0006) [2023-12-27 02:56:57,852][105692] Updated weights for policy 0, policy_version 1586040 (0.0005) [2023-12-27 02:56:57,908][105692] Updated weights for policy 0, policy_version 1586050 (0.0005) [2023-12-27 02:56:57,916][105620] Updated weights for policy 1, policy_version 1589772 (0.0007) [2023-12-27 02:56:57,979][105620] Updated weights for policy 1, policy_version 1589782 (0.0009) [2023-12-27 02:56:58,032][105620] Updated weights for policy 1, policy_version 1589792 (0.0010) [2023-12-27 02:56:58,486][105692] Updated weights for policy 0, policy_version 1586060 (0.0006) [2023-12-27 02:56:58,548][105692] Updated weights for policy 0, policy_version 1586070 (0.0008) [2023-12-27 02:56:58,616][105692] Updated weights for policy 0, policy_version 1586080 (0.0007) [2023-12-27 02:56:58,892][105620] Updated weights for policy 1, policy_version 1589802 (0.0009) [2023-12-27 02:56:58,951][105620] Updated weights for policy 1, policy_version 1589812 (0.0008) [2023-12-27 02:56:59,011][105620] Updated weights for policy 1, policy_version 1589822 (0.0008) [2023-12-27 02:56:59,091][105620] Updated weights for policy 1, policy_version 1589832 (0.0005) [2023-12-27 02:56:59,323][105692] Updated weights for policy 0, policy_version 1586090 (0.0008) [2023-12-27 02:56:59,386][105692] Updated weights for policy 0, policy_version 1586100 (0.0008) [2023-12-27 02:56:59,442][105692] Updated weights for policy 0, policy_version 1586110 (0.0006) [2023-12-27 02:56:59,497][105692] Updated weights for policy 0, policy_version 1586120 (0.0009) [2023-12-27 02:56:59,750][105620] Updated weights for policy 1, policy_version 1589842 (0.0010) [2023-12-27 02:56:59,805][105620] Updated weights for policy 1, policy_version 1589852 (0.0009) [2023-12-27 02:56:59,866][105620] Updated weights for policy 1, policy_version 1589862 (0.0008) [2023-12-27 02:57:00,189][105692] Updated weights for policy 0, policy_version 1586130 (0.0011) [2023-12-27 02:57:00,247][105692] Updated weights for policy 0, policy_version 1586140 (0.0010) [2023-12-27 02:57:00,315][105692] Updated weights for policy 0, policy_version 1586150 (0.0010) [2023-12-27 02:57:00,545][105620] Updated weights for policy 1, policy_version 1589872 (0.0005) [2023-12-27 02:57:00,603][105620] Updated weights for policy 1, policy_version 1589882 (0.0009) [2023-12-27 02:57:00,656][105620] Updated weights for policy 1, policy_version 1589892 (0.0007) [2023-12-27 02:57:01,004][105692] Updated weights for policy 0, policy_version 1586160 (0.0010) [2023-12-27 02:57:01,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 813178880. Throughput: 0: 10054.4, 1: 9824.5. Samples: 813152060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:57:01,062][104569] Avg episode reward: [(0, '8896.443'), (1, '8990.629')] [2023-12-27 02:57:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001589896_407068672.pth... [2023-12-27 02:57:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001588744_406773760.pth [2023-12-27 02:57:01,074][105692] Updated weights for policy 0, policy_version 1586170 (0.0010) [2023-12-27 02:57:01,137][105692] Updated weights for policy 0, policy_version 1586180 (0.0011) [2023-12-27 02:57:01,164][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001586184_406118400.pth... [2023-12-27 02:57:01,168][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001585032_405823488.pth [2023-12-27 02:57:01,294][105620] Updated weights for policy 1, policy_version 1589902 (0.0009) [2023-12-27 02:57:01,358][105620] Updated weights for policy 1, policy_version 1589912 (0.0010) [2023-12-27 02:57:01,417][105620] Updated weights for policy 1, policy_version 1589922 (0.0010) [2023-12-27 02:57:01,844][105692] Updated weights for policy 0, policy_version 1586190 (0.0009) [2023-12-27 02:57:01,892][105692] Updated weights for policy 0, policy_version 1586200 (0.0010) [2023-12-27 02:57:01,946][105692] Updated weights for policy 0, policy_version 1586210 (0.0010) [2023-12-27 02:57:02,065][105620] Updated weights for policy 1, policy_version 1589932 (0.0010) [2023-12-27 02:57:02,124][105620] Updated weights for policy 1, policy_version 1589942 (0.0010) [2023-12-27 02:57:02,188][105620] Updated weights for policy 1, policy_version 1589952 (0.0010) [2023-12-27 02:57:02,655][105692] Updated weights for policy 0, policy_version 1586220 (0.0010) [2023-12-27 02:57:02,714][105692] Updated weights for policy 0, policy_version 1586230 (0.0011) [2023-12-27 02:57:02,776][105692] Updated weights for policy 0, policy_version 1586240 (0.0009) [2023-12-27 02:57:02,895][105620] Updated weights for policy 1, policy_version 1589962 (0.0009) [2023-12-27 02:57:02,954][105620] Updated weights for policy 1, policy_version 1589972 (0.0007) [2023-12-27 02:57:03,003][105620] Updated weights for policy 1, policy_version 1589982 (0.0009) [2023-12-27 02:57:03,050][105620] Updated weights for policy 1, policy_version 1589992 (0.0008) [2023-12-27 02:57:03,521][105692] Updated weights for policy 0, policy_version 1586250 (0.0011) [2023-12-27 02:57:03,583][105692] Updated weights for policy 0, policy_version 1586260 (0.0010) [2023-12-27 02:57:03,641][105692] Updated weights for policy 0, policy_version 1586270 (0.0010) [2023-12-27 02:57:03,692][105692] Updated weights for policy 0, policy_version 1586280 (0.0010) [2023-12-27 02:57:03,802][105620] Updated weights for policy 1, policy_version 1590002 (0.0008) [2023-12-27 02:57:03,867][105620] Updated weights for policy 1, policy_version 1590012 (0.0008) [2023-12-27 02:57:03,921][105620] Updated weights for policy 1, policy_version 1590022 (0.0009) [2023-12-27 02:57:04,467][105692] Updated weights for policy 0, policy_version 1586290 (0.0010) [2023-12-27 02:57:04,524][105692] Updated weights for policy 0, policy_version 1586300 (0.0010) [2023-12-27 02:57:04,576][105692] Updated weights for policy 0, policy_version 1586310 (0.0010) [2023-12-27 02:57:04,675][105620] Updated weights for policy 1, policy_version 1590032 (0.0006) [2023-12-27 02:57:04,731][105620] Updated weights for policy 1, policy_version 1590042 (0.0005) [2023-12-27 02:57:04,789][105620] Updated weights for policy 1, policy_version 1590052 (0.0008) [2023-12-27 02:57:05,283][105692] Updated weights for policy 0, policy_version 1586320 (0.0010) [2023-12-27 02:57:05,343][105692] Updated weights for policy 0, policy_version 1586330 (0.0009) [2023-12-27 02:57:05,407][105692] Updated weights for policy 0, policy_version 1586340 (0.0006) [2023-12-27 02:57:05,534][105620] Updated weights for policy 1, policy_version 1590062 (0.0009) [2023-12-27 02:57:05,587][105620] Updated weights for policy 1, policy_version 1590072 (0.0010) [2023-12-27 02:57:05,646][105620] Updated weights for policy 1, policy_version 1590082 (0.0009) [2023-12-27 02:57:06,048][105692] Updated weights for policy 0, policy_version 1586350 (0.0008) [2023-12-27 02:57:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 813277184. Throughput: 0: 9909.4, 1: 9917.5. Samples: 813270200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:57:06,062][104569] Avg episode reward: [(0, '8710.867'), (1, '9081.064')] [2023-12-27 02:57:06,100][105692] Updated weights for policy 0, policy_version 1586360 (0.0007) [2023-12-27 02:57:06,162][105692] Updated weights for policy 0, policy_version 1586370 (0.0009) [2023-12-27 02:57:06,445][105620] Updated weights for policy 1, policy_version 1590092 (0.0009) [2023-12-27 02:57:06,503][105620] Updated weights for policy 1, policy_version 1590102 (0.0009) [2023-12-27 02:57:06,568][105620] Updated weights for policy 1, policy_version 1590112 (0.0009) [2023-12-27 02:57:06,947][105692] Updated weights for policy 0, policy_version 1586380 (0.0009) [2023-12-27 02:57:07,017][105692] Updated weights for policy 0, policy_version 1586390 (0.0008) [2023-12-27 02:57:07,069][105692] Updated weights for policy 0, policy_version 1586400 (0.0009) [2023-12-27 02:57:07,312][105620] Updated weights for policy 1, policy_version 1590122 (0.0009) [2023-12-27 02:57:07,379][105620] Updated weights for policy 1, policy_version 1590132 (0.0009) [2023-12-27 02:57:07,433][105620] Updated weights for policy 1, policy_version 1590142 (0.0008) [2023-12-27 02:57:07,484][105620] Updated weights for policy 1, policy_version 1590152 (0.0009) [2023-12-27 02:57:07,870][105692] Updated weights for policy 0, policy_version 1586410 (0.0008) [2023-12-27 02:57:07,939][105692] Updated weights for policy 0, policy_version 1586420 (0.0009) [2023-12-27 02:57:07,995][105692] Updated weights for policy 0, policy_version 1586430 (0.0009) [2023-12-27 02:57:08,053][105692] Updated weights for policy 0, policy_version 1586440 (0.0007) [2023-12-27 02:57:08,169][105620] Updated weights for policy 1, policy_version 1590162 (0.0008) [2023-12-27 02:57:08,238][105620] Updated weights for policy 1, policy_version 1590172 (0.0011) [2023-12-27 02:57:08,298][105620] Updated weights for policy 1, policy_version 1590182 (0.0011) [2023-12-27 02:57:08,707][105692] Updated weights for policy 0, policy_version 1586450 (0.0008) [2023-12-27 02:57:08,760][105692] Updated weights for policy 0, policy_version 1586460 (0.0008) [2023-12-27 02:57:08,814][105692] Updated weights for policy 0, policy_version 1586470 (0.0008) [2023-12-27 02:57:09,017][105620] Updated weights for policy 1, policy_version 1590192 (0.0009) [2023-12-27 02:57:09,066][105620] Updated weights for policy 1, policy_version 1590202 (0.0009) [2023-12-27 02:57:09,113][105620] Updated weights for policy 1, policy_version 1590212 (0.0008) [2023-12-27 02:57:09,517][105692] Updated weights for policy 0, policy_version 1586480 (0.0011) [2023-12-27 02:57:09,587][105692] Updated weights for policy 0, policy_version 1586490 (0.0011) [2023-12-27 02:57:09,653][105692] Updated weights for policy 0, policy_version 1586500 (0.0008) [2023-12-27 02:57:09,972][105620] Updated weights for policy 1, policy_version 1590222 (0.0010) [2023-12-27 02:57:10,035][105620] Updated weights for policy 1, policy_version 1590232 (0.0010) [2023-12-27 02:57:10,091][105620] Updated weights for policy 1, policy_version 1590242 (0.0010) [2023-12-27 02:57:10,315][105692] Updated weights for policy 0, policy_version 1586510 (0.0008) [2023-12-27 02:57:10,381][105692] Updated weights for policy 0, policy_version 1586520 (0.0011) [2023-12-27 02:57:10,444][105692] Updated weights for policy 0, policy_version 1586530 (0.0011) [2023-12-27 02:57:10,759][105620] Updated weights for policy 1, policy_version 1590252 (0.0009) [2023-12-27 02:57:10,816][105620] Updated weights for policy 1, policy_version 1590262 (0.0008) [2023-12-27 02:57:10,873][105620] Updated weights for policy 1, policy_version 1590272 (0.0008) [2023-12-27 02:57:11,049][105692] Updated weights for policy 0, policy_version 1586540 (0.0011) [2023-12-27 02:57:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 813375488. Throughput: 0: 9794.0, 1: 9919.4. Samples: 813385060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:57:11,063][104569] Avg episode reward: [(0, '8436.866'), (1, '8991.256')] [2023-12-27 02:57:11,112][105692] Updated weights for policy 0, policy_version 1586550 (0.0011) [2023-12-27 02:57:11,180][105692] Updated weights for policy 0, policy_version 1586560 (0.0008) [2023-12-27 02:57:11,618][105620] Updated weights for policy 1, policy_version 1590282 (0.0006) [2023-12-27 02:57:11,685][105620] Updated weights for policy 1, policy_version 1590292 (0.0009) [2023-12-27 02:57:11,757][105620] Updated weights for policy 1, policy_version 1590302 (0.0009) [2023-12-27 02:57:11,821][105620] Updated weights for policy 1, policy_version 1590312 (0.0008) [2023-12-27 02:57:11,866][105692] Updated weights for policy 0, policy_version 1586570 (0.0006) [2023-12-27 02:57:11,920][105692] Updated weights for policy 0, policy_version 1586580 (0.0009) [2023-12-27 02:57:11,986][105692] Updated weights for policy 0, policy_version 1586590 (0.0007) [2023-12-27 02:57:12,048][105692] Updated weights for policy 0, policy_version 1586600 (0.0006) [2023-12-27 02:57:12,604][105620] Updated weights for policy 1, policy_version 1590322 (0.0010) [2023-12-27 02:57:12,664][105620] Updated weights for policy 1, policy_version 1590332 (0.0010) [2023-12-27 02:57:12,719][105620] Updated weights for policy 1, policy_version 1590342 (0.0010) [2023-12-27 02:57:12,722][105692] Updated weights for policy 0, policy_version 1586610 (0.0006) [2023-12-27 02:57:12,789][105692] Updated weights for policy 0, policy_version 1586620 (0.0006) [2023-12-27 02:57:12,854][105692] Updated weights for policy 0, policy_version 1586630 (0.0009) [2023-12-27 02:57:13,391][105620] Updated weights for policy 1, policy_version 1590352 (0.0006) [2023-12-27 02:57:13,420][105692] Updated weights for policy 0, policy_version 1586640 (0.0008) [2023-12-27 02:57:13,458][105620] Updated weights for policy 1, policy_version 1590362 (0.0005) [2023-12-27 02:57:13,469][105692] Updated weights for policy 0, policy_version 1586650 (0.0008) [2023-12-27 02:57:13,522][105620] Updated weights for policy 1, policy_version 1590372 (0.0007) [2023-12-27 02:57:13,524][105692] Updated weights for policy 0, policy_version 1586660 (0.0008) [2023-12-27 02:57:14,094][105620] Updated weights for policy 1, policy_version 1590382 (0.0007) [2023-12-27 02:57:14,159][105620] Updated weights for policy 1, policy_version 1590392 (0.0008) [2023-12-27 02:57:14,179][105692] Updated weights for policy 0, policy_version 1586670 (0.0007) [2023-12-27 02:57:14,226][105620] Updated weights for policy 1, policy_version 1590402 (0.0006) [2023-12-27 02:57:14,243][105692] Updated weights for policy 0, policy_version 1586680 (0.0006) [2023-12-27 02:57:14,292][105692] Updated weights for policy 0, policy_version 1586690 (0.0006) [2023-12-27 02:57:14,889][105692] Updated weights for policy 0, policy_version 1586700 (0.0010) [2023-12-27 02:57:14,951][105692] Updated weights for policy 0, policy_version 1586710 (0.0009) [2023-12-27 02:57:14,979][105620] Updated weights for policy 1, policy_version 1590412 (0.0007) [2023-12-27 02:57:15,016][105692] Updated weights for policy 0, policy_version 1586720 (0.0007) [2023-12-27 02:57:15,044][105620] Updated weights for policy 1, policy_version 1590422 (0.0008) [2023-12-27 02:57:15,107][105620] Updated weights for policy 1, policy_version 1590432 (0.0008) [2023-12-27 02:57:15,673][105692] Updated weights for policy 0, policy_version 1586730 (0.0008) [2023-12-27 02:57:15,725][105692] Updated weights for policy 0, policy_version 1586740 (0.0008) [2023-12-27 02:57:15,790][105692] Updated weights for policy 0, policy_version 1586750 (0.0008) [2023-12-27 02:57:15,843][105620] Updated weights for policy 1, policy_version 1590442 (0.0010) [2023-12-27 02:57:15,851][105692] Updated weights for policy 0, policy_version 1586760 (0.0009) [2023-12-27 02:57:15,898][105620] Updated weights for policy 1, policy_version 1590452 (0.0010) [2023-12-27 02:57:15,942][105620] Updated weights for policy 1, policy_version 1590462 (0.0010) [2023-12-27 02:57:15,994][105620] Updated weights for policy 1, policy_version 1590472 (0.0010) [2023-12-27 02:57:16,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 813481984. Throughput: 0: 9835.3, 1: 9789.0. Samples: 813446408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:57:16,063][104569] Avg episode reward: [(0, '8624.715'), (1, '9083.646')] [2023-12-27 02:57:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001586760_406265856.pth... [2023-12-27 02:57:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001590472_407216128.pth... [2023-12-27 02:57:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001589320_406921216.pth [2023-12-27 02:57:16,091][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001585576_405962752.pth [2023-12-27 02:57:16,617][105692] Updated weights for policy 0, policy_version 1586770 (0.0008) [2023-12-27 02:57:16,666][105692] Updated weights for policy 0, policy_version 1586780 (0.0008) [2023-12-27 02:57:16,714][105692] Updated weights for policy 0, policy_version 1586790 (0.0008) [2023-12-27 02:57:16,780][105620] Updated weights for policy 1, policy_version 1590482 (0.0011) [2023-12-27 02:57:16,832][105620] Updated weights for policy 1, policy_version 1590492 (0.0008) [2023-12-27 02:57:16,880][105620] Updated weights for policy 1, policy_version 1590502 (0.0007) [2023-12-27 02:57:17,518][105620] Updated weights for policy 1, policy_version 1590512 (0.0008) [2023-12-27 02:57:17,529][105692] Updated weights for policy 0, policy_version 1586800 (0.0006) [2023-12-27 02:57:17,586][105620] Updated weights for policy 1, policy_version 1590522 (0.0008) [2023-12-27 02:57:17,590][105692] Updated weights for policy 0, policy_version 1586810 (0.0005) [2023-12-27 02:57:17,638][105692] Updated weights for policy 0, policy_version 1586820 (0.0005) [2023-12-27 02:57:17,653][105620] Updated weights for policy 1, policy_version 1590532 (0.0009) [2023-12-27 02:57:18,186][105692] Updated weights for policy 0, policy_version 1586830 (0.0005) [2023-12-27 02:57:18,238][105692] Updated weights for policy 0, policy_version 1586840 (0.0005) [2023-12-27 02:57:18,287][105692] Updated weights for policy 0, policy_version 1586850 (0.0009) [2023-12-27 02:57:18,489][105620] Updated weights for policy 1, policy_version 1590543 (0.0010) [2023-12-27 02:57:18,543][105620] Updated weights for policy 1, policy_version 1590553 (0.0009) [2023-12-27 02:57:18,596][105620] Updated weights for policy 1, policy_version 1590563 (0.0009) [2023-12-27 02:57:19,004][105692] Updated weights for policy 0, policy_version 1586860 (0.0009) [2023-12-27 02:57:19,053][105692] Updated weights for policy 0, policy_version 1586870 (0.0009) [2023-12-27 02:57:19,102][105692] Updated weights for policy 0, policy_version 1586880 (0.0009) [2023-12-27 02:57:19,372][105620] Updated weights for policy 1, policy_version 1590573 (0.0008) [2023-12-27 02:57:19,432][105620] Updated weights for policy 1, policy_version 1590583 (0.0009) [2023-12-27 02:57:19,499][105620] Updated weights for policy 1, policy_version 1590593 (0.0008) [2023-12-27 02:57:19,873][105692] Updated weights for policy 0, policy_version 1586890 (0.0009) [2023-12-27 02:57:19,934][105692] Updated weights for policy 0, policy_version 1586900 (0.0008) [2023-12-27 02:57:20,004][105692] Updated weights for policy 0, policy_version 1586910 (0.0008) [2023-12-27 02:57:20,065][105692] Updated weights for policy 0, policy_version 1586920 (0.0008) [2023-12-27 02:57:20,264][105620] Updated weights for policy 1, policy_version 1590603 (0.0008) [2023-12-27 02:57:20,315][105620] Updated weights for policy 1, policy_version 1590613 (0.0009) [2023-12-27 02:57:20,371][105620] Updated weights for policy 1, policy_version 1590623 (0.0009) [2023-12-27 02:57:20,820][105692] Updated weights for policy 0, policy_version 1586930 (0.0009) [2023-12-27 02:57:20,879][105692] Updated weights for policy 0, policy_version 1586940 (0.0009) [2023-12-27 02:57:20,944][105692] Updated weights for policy 0, policy_version 1586950 (0.0009) [2023-12-27 02:57:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 813572096. Throughput: 0: 9867.9, 1: 9700.4. Samples: 813562832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:57:21,062][104569] Avg episode reward: [(0, '8806.582'), (1, '8992.682')] [2023-12-27 02:57:21,182][105620] Updated weights for policy 1, policy_version 1590633 (0.0009) [2023-12-27 02:57:21,244][105620] Updated weights for policy 1, policy_version 1590643 (0.0009) [2023-12-27 02:57:21,306][105620] Updated weights for policy 1, policy_version 1590653 (0.0009) [2023-12-27 02:57:21,372][105620] Updated weights for policy 1, policy_version 1590663 (0.0008) [2023-12-27 02:57:21,729][105692] Updated weights for policy 0, policy_version 1586960 (0.0009) [2023-12-27 02:57:21,789][105692] Updated weights for policy 0, policy_version 1586970 (0.0009) [2023-12-27 02:57:21,836][105692] Updated weights for policy 0, policy_version 1586980 (0.0008) [2023-12-27 02:57:22,155][105620] Updated weights for policy 1, policy_version 1590673 (0.0009) [2023-12-27 02:57:22,213][105620] Updated weights for policy 1, policy_version 1590683 (0.0008) [2023-12-27 02:57:22,275][105620] Updated weights for policy 1, policy_version 1590693 (0.0008) [2023-12-27 02:57:22,593][105692] Updated weights for policy 0, policy_version 1586990 (0.0009) [2023-12-27 02:57:22,648][105692] Updated weights for policy 0, policy_version 1587000 (0.0009) [2023-12-27 02:57:22,700][105692] Updated weights for policy 0, policy_version 1587010 (0.0008) [2023-12-27 02:57:23,073][105620] Updated weights for policy 1, policy_version 1590703 (0.0008) [2023-12-27 02:57:23,135][105620] Updated weights for policy 1, policy_version 1590713 (0.0009) [2023-12-27 02:57:23,185][105620] Updated weights for policy 1, policy_version 1590723 (0.0009) [2023-12-27 02:57:23,333][105692] Updated weights for policy 0, policy_version 1587020 (0.0006) [2023-12-27 02:57:23,384][105692] Updated weights for policy 0, policy_version 1587030 (0.0009) [2023-12-27 02:57:23,430][105692] Updated weights for policy 0, policy_version 1587040 (0.0008) [2023-12-27 02:57:24,018][105620] Updated weights for policy 1, policy_version 1590733 (0.0010) [2023-12-27 02:57:24,067][105692] Updated weights for policy 0, policy_version 1587050 (0.0008) [2023-12-27 02:57:24,070][105620] Updated weights for policy 1, policy_version 1590743 (0.0010) [2023-12-27 02:57:24,120][105692] Updated weights for policy 0, policy_version 1587060 (0.0006) [2023-12-27 02:57:24,122][105620] Updated weights for policy 1, policy_version 1590753 (0.0007) [2023-12-27 02:57:24,183][105692] Updated weights for policy 0, policy_version 1587070 (0.0007) [2023-12-27 02:57:24,229][105692] Updated weights for policy 0, policy_version 1587080 (0.0008) [2023-12-27 02:57:24,862][105620] Updated weights for policy 1, policy_version 1590763 (0.0009) [2023-12-27 02:57:24,915][105620] Updated weights for policy 1, policy_version 1590773 (0.0009) [2023-12-27 02:57:24,965][105620] Updated weights for policy 1, policy_version 1590783 (0.0008) [2023-12-27 02:57:24,972][105692] Updated weights for policy 0, policy_version 1587090 (0.0008) [2023-12-27 02:57:25,031][105692] Updated weights for policy 0, policy_version 1587100 (0.0008) [2023-12-27 02:57:25,078][105692] Updated weights for policy 0, policy_version 1587110 (0.0005) [2023-12-27 02:57:25,640][105692] Updated weights for policy 0, policy_version 1587120 (0.0008) [2023-12-27 02:57:25,693][105620] Updated weights for policy 1, policy_version 1590793 (0.0008) [2023-12-27 02:57:25,706][105692] Updated weights for policy 0, policy_version 1587130 (0.0007) [2023-12-27 02:57:25,748][105620] Updated weights for policy 1, policy_version 1590803 (0.0008) [2023-12-27 02:57:25,756][105692] Updated weights for policy 0, policy_version 1587140 (0.0006) [2023-12-27 02:57:25,805][105620] Updated weights for policy 1, policy_version 1590813 (0.0009) [2023-12-27 02:57:25,863][105620] Updated weights for policy 1, policy_version 1590823 (0.0008) [2023-12-27 02:57:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 813670400. Throughput: 0: 9932.0, 1: 9565.3. Samples: 813676540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:57:26,063][104569] Avg episode reward: [(0, '8437.232'), (1, '8900.519')] [2023-12-27 02:57:26,323][105692] Updated weights for policy 0, policy_version 1587150 (0.0008) [2023-12-27 02:57:26,377][105692] Updated weights for policy 0, policy_version 1587160 (0.0005) [2023-12-27 02:57:26,433][105692] Updated weights for policy 0, policy_version 1587170 (0.0005) [2023-12-27 02:57:26,572][105620] Updated weights for policy 1, policy_version 1590833 (0.0009) [2023-12-27 02:57:26,624][105620] Updated weights for policy 1, policy_version 1590843 (0.0008) [2023-12-27 02:57:26,683][105620] Updated weights for policy 1, policy_version 1590853 (0.0008) [2023-12-27 02:57:27,028][105692] Updated weights for policy 0, policy_version 1587180 (0.0005) [2023-12-27 02:57:27,084][105692] Updated weights for policy 0, policy_version 1587190 (0.0009) [2023-12-27 02:57:27,131][105692] Updated weights for policy 0, policy_version 1587200 (0.0010) [2023-12-27 02:57:27,384][105620] Updated weights for policy 1, policy_version 1590863 (0.0006) [2023-12-27 02:57:27,436][105620] Updated weights for policy 1, policy_version 1590873 (0.0005) [2023-12-27 02:57:27,499][105620] Updated weights for policy 1, policy_version 1590883 (0.0005) [2023-12-27 02:57:27,825][105692] Updated weights for policy 0, policy_version 1587210 (0.0010) [2023-12-27 02:57:27,873][105692] Updated weights for policy 0, policy_version 1587220 (0.0010) [2023-12-27 02:57:27,917][105692] Updated weights for policy 0, policy_version 1587230 (0.0010) [2023-12-27 02:57:27,968][105692] Updated weights for policy 0, policy_version 1587240 (0.0010) [2023-12-27 02:57:28,120][105620] Updated weights for policy 1, policy_version 1590893 (0.0007) [2023-12-27 02:57:28,172][105620] Updated weights for policy 1, policy_version 1590904 (0.0009) [2023-12-27 02:57:28,225][105620] Updated weights for policy 1, policy_version 1590915 (0.0010) [2023-12-27 02:57:28,694][105692] Updated weights for policy 0, policy_version 1587250 (0.0010) [2023-12-27 02:57:28,745][105692] Updated weights for policy 0, policy_version 1587260 (0.0010) [2023-12-27 02:57:28,803][105692] Updated weights for policy 0, policy_version 1587270 (0.0010) [2023-12-27 02:57:29,002][105620] Updated weights for policy 1, policy_version 1590926 (0.0007) [2023-12-27 02:57:29,061][105620] Updated weights for policy 1, policy_version 1590936 (0.0009) [2023-12-27 02:57:29,112][105620] Updated weights for policy 1, policy_version 1590946 (0.0010) [2023-12-27 02:57:29,574][105692] Updated weights for policy 0, policy_version 1587280 (0.0009) [2023-12-27 02:57:29,631][105692] Updated weights for policy 0, policy_version 1587290 (0.0006) [2023-12-27 02:57:29,698][105692] Updated weights for policy 0, policy_version 1587300 (0.0005) [2023-12-27 02:57:29,802][105620] Updated weights for policy 1, policy_version 1590956 (0.0009) [2023-12-27 02:57:29,866][105620] Updated weights for policy 1, policy_version 1590966 (0.0011) [2023-12-27 02:57:29,932][105620] Updated weights for policy 1, policy_version 1590976 (0.0010) [2023-12-27 02:57:30,382][105692] Updated weights for policy 0, policy_version 1587310 (0.0009) [2023-12-27 02:57:30,438][105692] Updated weights for policy 0, policy_version 1587320 (0.0008) [2023-12-27 02:57:30,493][105692] Updated weights for policy 0, policy_version 1587330 (0.0008) [2023-12-27 02:57:30,685][105620] Updated weights for policy 1, policy_version 1590986 (0.0010) [2023-12-27 02:57:30,744][105620] Updated weights for policy 1, policy_version 1590996 (0.0010) [2023-12-27 02:57:30,803][105620] Updated weights for policy 1, policy_version 1591006 (0.0010) [2023-12-27 02:57:30,862][105620] Updated weights for policy 1, policy_version 1591016 (0.0011) [2023-12-27 02:57:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 813768704. Throughput: 0: 10014.8, 1: 9633.8. Samples: 813739528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:57:31,062][104569] Avg episode reward: [(0, '8261.267'), (1, '9083.169')] [2023-12-27 02:57:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001591016_407355392.pth... [2023-12-27 02:57:31,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001587336_406413312.pth... [2023-12-27 02:57:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001589896_407068672.pth [2023-12-27 02:57:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001586184_406118400.pth [2023-12-27 02:57:31,110][105692] Updated weights for policy 0, policy_version 1587340 (0.0008) [2023-12-27 02:57:31,178][105692] Updated weights for policy 0, policy_version 1587350 (0.0008) [2023-12-27 02:57:31,230][105692] Updated weights for policy 0, policy_version 1587360 (0.0008) [2023-12-27 02:57:31,611][105620] Updated weights for policy 1, policy_version 1591026 (0.0007) [2023-12-27 02:57:31,677][105620] Updated weights for policy 1, policy_version 1591036 (0.0011) [2023-12-27 02:57:31,747][105620] Updated weights for policy 1, policy_version 1591046 (0.0009) [2023-12-27 02:57:31,941][105692] Updated weights for policy 0, policy_version 1587370 (0.0008) [2023-12-27 02:57:32,006][105692] Updated weights for policy 0, policy_version 1587380 (0.0008) [2023-12-27 02:57:32,071][105692] Updated weights for policy 0, policy_version 1587390 (0.0008) [2023-12-27 02:57:32,133][105692] Updated weights for policy 0, policy_version 1587400 (0.0009) [2023-12-27 02:57:32,305][105620] Updated weights for policy 1, policy_version 1591056 (0.0006) [2023-12-27 02:57:32,370][105620] Updated weights for policy 1, policy_version 1591066 (0.0007) [2023-12-27 02:57:32,438][105620] Updated weights for policy 1, policy_version 1591076 (0.0007) [2023-12-27 02:57:32,918][105692] Updated weights for policy 0, policy_version 1587410 (0.0010) [2023-12-27 02:57:32,973][105620] Updated weights for policy 1, policy_version 1591086 (0.0008) [2023-12-27 02:57:32,977][105692] Updated weights for policy 0, policy_version 1587420 (0.0007) [2023-12-27 02:57:33,027][105692] Updated weights for policy 0, policy_version 1587430 (0.0007) [2023-12-27 02:57:33,040][105620] Updated weights for policy 1, policy_version 1591096 (0.0009) [2023-12-27 02:57:33,094][105620] Updated weights for policy 1, policy_version 1591106 (0.0010) [2023-12-27 02:57:33,720][105692] Updated weights for policy 0, policy_version 1587440 (0.0009) [2023-12-27 02:57:33,775][105692] Updated weights for policy 0, policy_version 1587450 (0.0005) [2023-12-27 02:57:33,791][105620] Updated weights for policy 1, policy_version 1591116 (0.0007) [2023-12-27 02:57:33,831][105692] Updated weights for policy 0, policy_version 1587460 (0.0006) [2023-12-27 02:57:33,859][105620] Updated weights for policy 1, policy_version 1591126 (0.0006) [2023-12-27 02:57:33,906][105620] Updated weights for policy 1, policy_version 1591136 (0.0006) [2023-12-27 02:57:34,499][105692] Updated weights for policy 0, policy_version 1587470 (0.0007) [2023-12-27 02:57:34,548][105620] Updated weights for policy 1, policy_version 1591146 (0.0006) [2023-12-27 02:57:34,557][105692] Updated weights for policy 0, policy_version 1587480 (0.0009) [2023-12-27 02:57:34,609][105620] Updated weights for policy 1, policy_version 1591156 (0.0007) [2023-12-27 02:57:34,619][105692] Updated weights for policy 0, policy_version 1587490 (0.0006) [2023-12-27 02:57:34,667][105620] Updated weights for policy 1, policy_version 1591166 (0.0007) [2023-12-27 02:57:34,719][105620] Updated weights for policy 1, policy_version 1591176 (0.0009) [2023-12-27 02:57:35,358][105692] Updated weights for policy 0, policy_version 1587500 (0.0007) [2023-12-27 02:57:35,405][105692] Updated weights for policy 0, policy_version 1587510 (0.0005) [2023-12-27 02:57:35,461][105692] Updated weights for policy 0, policy_version 1587520 (0.0005) [2023-12-27 02:57:35,475][105620] Updated weights for policy 1, policy_version 1591186 (0.0011) [2023-12-27 02:57:35,538][105620] Updated weights for policy 1, policy_version 1591196 (0.0011) [2023-12-27 02:57:35,599][105620] Updated weights for policy 1, policy_version 1591206 (0.0010) [2023-12-27 02:57:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 813867008. Throughput: 0: 10018.5, 1: 9728.2. Samples: 813859416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:57:36,062][104569] Avg episode reward: [(0, '8447.242'), (1, '9175.333')] [2023-12-27 02:57:36,186][105620] Updated weights for policy 1, policy_version 1591216 (0.0010) [2023-12-27 02:57:36,187][105692] Updated weights for policy 0, policy_version 1587530 (0.0009) [2023-12-27 02:57:36,243][105620] Updated weights for policy 1, policy_version 1591226 (0.0010) [2023-12-27 02:57:36,244][105692] Updated weights for policy 0, policy_version 1587540 (0.0011) [2023-12-27 02:57:36,304][105692] Updated weights for policy 0, policy_version 1587550 (0.0011) [2023-12-27 02:57:36,307][105620] Updated weights for policy 1, policy_version 1591236 (0.0010) [2023-12-27 02:57:36,366][105692] Updated weights for policy 0, policy_version 1587560 (0.0011) [2023-12-27 02:57:36,968][105620] Updated weights for policy 1, policy_version 1591246 (0.0009) [2023-12-27 02:57:37,039][105620] Updated weights for policy 1, policy_version 1591256 (0.0008) [2023-12-27 02:57:37,098][105620] Updated weights for policy 1, policy_version 1591266 (0.0010) [2023-12-27 02:57:37,109][105692] Updated weights for policy 0, policy_version 1587570 (0.0010) [2023-12-27 02:57:37,165][105692] Updated weights for policy 0, policy_version 1587580 (0.0010) [2023-12-27 02:57:37,220][105692] Updated weights for policy 0, policy_version 1587590 (0.0010) [2023-12-27 02:57:37,774][105620] Updated weights for policy 1, policy_version 1591276 (0.0011) [2023-12-27 02:57:37,824][105620] Updated weights for policy 1, policy_version 1591286 (0.0010) [2023-12-27 02:57:37,873][105620] Updated weights for policy 1, policy_version 1591296 (0.0010) [2023-12-27 02:57:38,002][105692] Updated weights for policy 0, policy_version 1587600 (0.0011) [2023-12-27 02:57:38,062][105692] Updated weights for policy 0, policy_version 1587610 (0.0011) [2023-12-27 02:57:38,114][105692] Updated weights for policy 0, policy_version 1587620 (0.0011) [2023-12-27 02:57:38,641][105620] Updated weights for policy 1, policy_version 1591306 (0.0010) [2023-12-27 02:57:38,707][105620] Updated weights for policy 1, policy_version 1591316 (0.0010) [2023-12-27 02:57:38,762][105620] Updated weights for policy 1, policy_version 1591326 (0.0010) [2023-12-27 02:57:38,821][105620] Updated weights for policy 1, policy_version 1591336 (0.0010) [2023-12-27 02:57:38,860][105692] Updated weights for policy 0, policy_version 1587630 (0.0010) [2023-12-27 02:57:38,920][105692] Updated weights for policy 0, policy_version 1587640 (0.0011) [2023-12-27 02:57:38,982][105692] Updated weights for policy 0, policy_version 1587650 (0.0011) [2023-12-27 02:57:39,554][105620] Updated weights for policy 1, policy_version 1591346 (0.0009) [2023-12-27 02:57:39,611][105620] Updated weights for policy 1, policy_version 1591356 (0.0010) [2023-12-27 02:57:39,671][105620] Updated weights for policy 1, policy_version 1591366 (0.0011) [2023-12-27 02:57:39,720][105692] Updated weights for policy 0, policy_version 1587660 (0.0009) [2023-12-27 02:57:39,777][105692] Updated weights for policy 0, policy_version 1587670 (0.0005) [2023-12-27 02:57:39,839][105692] Updated weights for policy 0, policy_version 1587680 (0.0006) [2023-12-27 02:57:40,450][105692] Updated weights for policy 0, policy_version 1587690 (0.0006) [2023-12-27 02:57:40,459][105620] Updated weights for policy 1, policy_version 1591376 (0.0011) [2023-12-27 02:57:40,510][105692] Updated weights for policy 0, policy_version 1587700 (0.0006) [2023-12-27 02:57:40,516][105620] Updated weights for policy 1, policy_version 1591386 (0.0011) [2023-12-27 02:57:40,568][105620] Updated weights for policy 1, policy_version 1591396 (0.0010) [2023-12-27 02:57:40,571][105692] Updated weights for policy 0, policy_version 1587710 (0.0006) [2023-12-27 02:57:40,634][105692] Updated weights for policy 0, policy_version 1587720 (0.0005) [2023-12-27 02:57:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 813965312. Throughput: 0: 10007.2, 1: 9689.8. Samples: 813975708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:57:41,063][104569] Avg episode reward: [(0, '8167.105'), (1, '9265.432')] [2023-12-27 02:57:41,296][105692] Updated weights for policy 0, policy_version 1587730 (0.0010) [2023-12-27 02:57:41,353][105620] Updated weights for policy 1, policy_version 1591406 (0.0009) [2023-12-27 02:57:41,367][105692] Updated weights for policy 0, policy_version 1587740 (0.0008) [2023-12-27 02:57:41,414][105620] Updated weights for policy 1, policy_version 1591416 (0.0008) [2023-12-27 02:57:41,436][105692] Updated weights for policy 0, policy_version 1587750 (0.0008) [2023-12-27 02:57:41,486][105620] Updated weights for policy 1, policy_version 1591426 (0.0008) [2023-12-27 02:57:42,120][105692] Updated weights for policy 0, policy_version 1587760 (0.0006) [2023-12-27 02:57:42,181][105692] Updated weights for policy 0, policy_version 1587770 (0.0006) [2023-12-27 02:57:42,243][105692] Updated weights for policy 0, policy_version 1587780 (0.0005) [2023-12-27 02:57:42,299][105620] Updated weights for policy 1, policy_version 1591436 (0.0009) [2023-12-27 02:57:42,366][105620] Updated weights for policy 1, policy_version 1591446 (0.0009) [2023-12-27 02:57:42,432][105620] Updated weights for policy 1, policy_version 1591456 (0.0009) [2023-12-27 02:57:42,864][105692] Updated weights for policy 0, policy_version 1587790 (0.0008) [2023-12-27 02:57:42,911][105692] Updated weights for policy 0, policy_version 1587800 (0.0005) [2023-12-27 02:57:42,962][105692] Updated weights for policy 0, policy_version 1587810 (0.0007) [2023-12-27 02:57:43,152][105620] Updated weights for policy 1, policy_version 1591466 (0.0009) [2023-12-27 02:57:43,207][105620] Updated weights for policy 1, policy_version 1591476 (0.0009) [2023-12-27 02:57:43,263][105620] Updated weights for policy 1, policy_version 1591486 (0.0009) [2023-12-27 02:57:43,309][105620] Updated weights for policy 1, policy_version 1591496 (0.0006) [2023-12-27 02:57:43,731][105692] Updated weights for policy 0, policy_version 1587820 (0.0008) [2023-12-27 02:57:43,778][105692] Updated weights for policy 0, policy_version 1587830 (0.0005) [2023-12-27 02:57:43,836][105692] Updated weights for policy 0, policy_version 1587840 (0.0008) [2023-12-27 02:57:43,903][105620] Updated weights for policy 1, policy_version 1591506 (0.0006) [2023-12-27 02:57:43,959][105620] Updated weights for policy 1, policy_version 1591516 (0.0005) [2023-12-27 02:57:44,018][105620] Updated weights for policy 1, policy_version 1591526 (0.0009) [2023-12-27 02:57:44,585][105692] Updated weights for policy 0, policy_version 1587850 (0.0008) [2023-12-27 02:57:44,647][105692] Updated weights for policy 0, policy_version 1587860 (0.0008) [2023-12-27 02:57:44,707][105692] Updated weights for policy 0, policy_version 1587870 (0.0006) [2023-12-27 02:57:44,709][105620] Updated weights for policy 1, policy_version 1591536 (0.0010) [2023-12-27 02:57:44,768][105620] Updated weights for policy 1, policy_version 1591546 (0.0008) [2023-12-27 02:57:44,770][105692] Updated weights for policy 0, policy_version 1587880 (0.0008) [2023-12-27 02:57:44,831][105620] Updated weights for policy 1, policy_version 1591556 (0.0008) [2023-12-27 02:57:45,565][105692] Updated weights for policy 0, policy_version 1587890 (0.0006) [2023-12-27 02:57:45,579][105620] Updated weights for policy 1, policy_version 1591566 (0.0008) [2023-12-27 02:57:45,629][105692] Updated weights for policy 0, policy_version 1587900 (0.0008) [2023-12-27 02:57:45,633][105620] Updated weights for policy 1, policy_version 1591576 (0.0007) [2023-12-27 02:57:45,688][105692] Updated weights for policy 0, policy_version 1587910 (0.0010) [2023-12-27 02:57:45,690][105620] Updated weights for policy 1, policy_version 1591586 (0.0006) [2023-12-27 02:57:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 814063616. Throughput: 0: 9955.6, 1: 9659.3. Samples: 814034736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:57:46,063][104569] Avg episode reward: [(0, '8343.415'), (1, '9081.990')] [2023-12-27 02:57:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001587912_406560768.pth... [2023-12-27 02:57:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001591592_407502848.pth... [2023-12-27 02:57:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001586760_406265856.pth [2023-12-27 02:57:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001590472_407216128.pth [2023-12-27 02:57:46,361][105620] Updated weights for policy 1, policy_version 1591596 (0.0010) [2023-12-27 02:57:46,413][105620] Updated weights for policy 1, policy_version 1591606 (0.0010) [2023-12-27 02:57:46,448][105692] Updated weights for policy 0, policy_version 1587920 (0.0009) [2023-12-27 02:57:46,475][105620] Updated weights for policy 1, policy_version 1591616 (0.0010) [2023-12-27 02:57:46,505][105692] Updated weights for policy 0, policy_version 1587930 (0.0006) [2023-12-27 02:57:46,564][105692] Updated weights for policy 0, policy_version 1587940 (0.0007) [2023-12-27 02:57:47,164][105620] Updated weights for policy 1, policy_version 1591626 (0.0010) [2023-12-27 02:57:47,217][105620] Updated weights for policy 1, policy_version 1591636 (0.0005) [2023-12-27 02:57:47,265][105620] Updated weights for policy 1, policy_version 1591646 (0.0006) [2023-12-27 02:57:47,315][105620] Updated weights for policy 1, policy_version 1591656 (0.0005) [2023-12-27 02:57:47,339][105692] Updated weights for policy 0, policy_version 1587950 (0.0006) [2023-12-27 02:57:47,394][105692] Updated weights for policy 0, policy_version 1587960 (0.0009) [2023-12-27 02:57:47,453][105692] Updated weights for policy 0, policy_version 1587970 (0.0013) [2023-12-27 02:57:47,932][105620] Updated weights for policy 1, policy_version 1591666 (0.0009) [2023-12-27 02:57:47,987][105620] Updated weights for policy 1, policy_version 1591676 (0.0009) [2023-12-27 02:57:48,038][105620] Updated weights for policy 1, policy_version 1591686 (0.0009) [2023-12-27 02:57:48,220][105692] Updated weights for policy 0, policy_version 1587980 (0.0010) [2023-12-27 02:57:48,273][105692] Updated weights for policy 0, policy_version 1587990 (0.0010) [2023-12-27 02:57:48,333][105692] Updated weights for policy 0, policy_version 1588000 (0.0009) [2023-12-27 02:57:48,817][105620] Updated weights for policy 1, policy_version 1591696 (0.0009) [2023-12-27 02:57:48,875][105620] Updated weights for policy 1, policy_version 1591706 (0.0009) [2023-12-27 02:57:48,923][105620] Updated weights for policy 1, policy_version 1591716 (0.0009) [2023-12-27 02:57:49,052][105692] Updated weights for policy 0, policy_version 1588010 (0.0008) [2023-12-27 02:57:49,103][105692] Updated weights for policy 0, policy_version 1588020 (0.0009) [2023-12-27 02:57:49,156][105692] Updated weights for policy 0, policy_version 1588030 (0.0010) [2023-12-27 02:57:49,670][105620] Updated weights for policy 1, policy_version 1591726 (0.0010) [2023-12-27 02:57:49,737][105620] Updated weights for policy 1, policy_version 1591736 (0.0010) [2023-12-27 02:57:49,795][105620] Updated weights for policy 1, policy_version 1591746 (0.0010) [2023-12-27 02:57:49,902][105692] Updated weights for policy 0, policy_version 1588041 (0.0009) [2023-12-27 02:57:49,970][105692] Updated weights for policy 0, policy_version 1588051 (0.0009) [2023-12-27 02:57:50,035][105692] Updated weights for policy 0, policy_version 1588061 (0.0009) [2023-12-27 02:57:50,096][105692] Updated weights for policy 0, policy_version 1588071 (0.0009) [2023-12-27 02:57:50,465][105620] Updated weights for policy 1, policy_version 1591756 (0.0009) [2023-12-27 02:57:50,520][105620] Updated weights for policy 1, policy_version 1591766 (0.0008) [2023-12-27 02:57:50,585][105620] Updated weights for policy 1, policy_version 1591776 (0.0008) [2023-12-27 02:57:50,889][105692] Updated weights for policy 0, policy_version 1588081 (0.0008) [2023-12-27 02:57:50,955][105692] Updated weights for policy 0, policy_version 1588091 (0.0008) [2023-12-27 02:57:51,006][105692] Updated weights for policy 0, policy_version 1588101 (0.0009) [2023-12-27 02:57:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 814161920. Throughput: 0: 9885.0, 1: 9633.8. Samples: 814148552. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:57:51,063][104569] Avg episode reward: [(0, '8618.780'), (1, '8902.429')] [2023-12-27 02:57:51,312][105620] Updated weights for policy 1, policy_version 1591786 (0.0008) [2023-12-27 02:57:51,379][105620] Updated weights for policy 1, policy_version 1591796 (0.0009) [2023-12-27 02:57:51,447][105620] Updated weights for policy 1, policy_version 1591806 (0.0007) [2023-12-27 02:57:51,500][105620] Updated weights for policy 1, policy_version 1591816 (0.0006) [2023-12-27 02:57:51,808][105692] Updated weights for policy 0, policy_version 1588111 (0.0009) [2023-12-27 02:57:51,861][105692] Updated weights for policy 0, policy_version 1588121 (0.0009) [2023-12-27 02:57:51,923][105692] Updated weights for policy 0, policy_version 1588131 (0.0009) [2023-12-27 02:57:52,197][105620] Updated weights for policy 1, policy_version 1591826 (0.0010) [2023-12-27 02:57:52,252][105620] Updated weights for policy 1, policy_version 1591836 (0.0010) [2023-12-27 02:57:52,315][105620] Updated weights for policy 1, policy_version 1591846 (0.0011) [2023-12-27 02:57:52,724][105692] Updated weights for policy 0, policy_version 1588141 (0.0008) [2023-12-27 02:57:52,785][105692] Updated weights for policy 0, policy_version 1588151 (0.0008) [2023-12-27 02:57:52,837][105692] Updated weights for policy 0, policy_version 1588161 (0.0008) [2023-12-27 02:57:53,057][105620] Updated weights for policy 1, policy_version 1591856 (0.0011) [2023-12-27 02:57:53,105][105620] Updated weights for policy 1, policy_version 1591866 (0.0010) [2023-12-27 02:57:53,164][105620] Updated weights for policy 1, policy_version 1591876 (0.0010) [2023-12-27 02:57:53,537][105692] Updated weights for policy 0, policy_version 1588171 (0.0007) [2023-12-27 02:57:53,588][105692] Updated weights for policy 0, policy_version 1588181 (0.0010) [2023-12-27 02:57:53,650][105692] Updated weights for policy 0, policy_version 1588191 (0.0010) [2023-12-27 02:57:53,872][105620] Updated weights for policy 1, policy_version 1591886 (0.0011) [2023-12-27 02:57:53,926][105620] Updated weights for policy 1, policy_version 1591896 (0.0010) [2023-12-27 02:57:53,988][105620] Updated weights for policy 1, policy_version 1591906 (0.0010) [2023-12-27 02:57:54,375][105692] Updated weights for policy 0, policy_version 1588201 (0.0007) [2023-12-27 02:57:54,439][105692] Updated weights for policy 0, policy_version 1588211 (0.0010) [2023-12-27 02:57:54,491][105692] Updated weights for policy 0, policy_version 1588221 (0.0010) [2023-12-27 02:57:54,543][105692] Updated weights for policy 0, policy_version 1588231 (0.0010) [2023-12-27 02:57:54,705][105620] Updated weights for policy 1, policy_version 1591916 (0.0010) [2023-12-27 02:57:54,756][105620] Updated weights for policy 1, policy_version 1591926 (0.0010) [2023-12-27 02:57:54,814][105620] Updated weights for policy 1, policy_version 1591936 (0.0010) [2023-12-27 02:57:55,243][105692] Updated weights for policy 0, policy_version 1588241 (0.0008) [2023-12-27 02:57:55,301][105692] Updated weights for policy 0, policy_version 1588251 (0.0007) [2023-12-27 02:57:55,356][105692] Updated weights for policy 0, policy_version 1588261 (0.0007) [2023-12-27 02:57:55,550][105620] Updated weights for policy 1, policy_version 1591946 (0.0010) [2023-12-27 02:57:55,609][105620] Updated weights for policy 1, policy_version 1591956 (0.0011) [2023-12-27 02:57:55,667][105620] Updated weights for policy 1, policy_version 1591966 (0.0010) [2023-12-27 02:57:55,722][105620] Updated weights for policy 1, policy_version 1591976 (0.0010) [2023-12-27 02:57:56,061][105692] Updated weights for policy 0, policy_version 1588271 (0.0007) [2023-12-27 02:57:56,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 814252032. Throughput: 0: 9852.1, 1: 9680.4. Samples: 814264020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:57:56,062][104569] Avg episode reward: [(0, '8621.242'), (1, '8910.396')] [2023-12-27 02:57:56,119][105692] Updated weights for policy 0, policy_version 1588281 (0.0006) [2023-12-27 02:57:56,194][105692] Updated weights for policy 0, policy_version 1588291 (0.0008) [2023-12-27 02:57:56,409][105620] Updated weights for policy 1, policy_version 1591986 (0.0010) [2023-12-27 02:57:56,454][105620] Updated weights for policy 1, policy_version 1591996 (0.0010) [2023-12-27 02:57:56,498][105620] Updated weights for policy 1, policy_version 1592006 (0.0010) [2023-12-27 02:57:56,849][105692] Updated weights for policy 0, policy_version 1588301 (0.0009) [2023-12-27 02:57:56,907][105692] Updated weights for policy 0, policy_version 1588311 (0.0007) [2023-12-27 02:57:56,964][105692] Updated weights for policy 0, policy_version 1588321 (0.0008) [2023-12-27 02:57:57,237][105620] Updated weights for policy 1, policy_version 1592016 (0.0011) [2023-12-27 02:57:57,282][105620] Updated weights for policy 1, policy_version 1592026 (0.0010) [2023-12-27 02:57:57,330][105620] Updated weights for policy 1, policy_version 1592036 (0.0009) [2023-12-27 02:57:57,555][105692] Updated weights for policy 0, policy_version 1588331 (0.0005) [2023-12-27 02:57:57,613][105692] Updated weights for policy 0, policy_version 1588341 (0.0005) [2023-12-27 02:57:57,673][105692] Updated weights for policy 0, policy_version 1588351 (0.0006) [2023-12-27 02:57:58,088][105620] Updated weights for policy 1, policy_version 1592046 (0.0011) [2023-12-27 02:57:58,155][105620] Updated weights for policy 1, policy_version 1592056 (0.0010) [2023-12-27 02:57:58,225][105620] Updated weights for policy 1, policy_version 1592066 (0.0007) [2023-12-27 02:57:58,300][105692] Updated weights for policy 0, policy_version 1588361 (0.0007) [2023-12-27 02:57:58,374][105692] Updated weights for policy 0, policy_version 1588371 (0.0009) [2023-12-27 02:57:58,446][105692] Updated weights for policy 0, policy_version 1588381 (0.0008) [2023-12-27 02:57:58,514][105692] Updated weights for policy 0, policy_version 1588391 (0.0008) [2023-12-27 02:57:58,995][105620] Updated weights for policy 1, policy_version 1592076 (0.0009) [2023-12-27 02:57:59,044][105620] Updated weights for policy 1, policy_version 1592086 (0.0010) [2023-12-27 02:57:59,090][105620] Updated weights for policy 1, policy_version 1592096 (0.0009) [2023-12-27 02:57:59,340][105692] Updated weights for policy 0, policy_version 1588401 (0.0008) [2023-12-27 02:57:59,403][105692] Updated weights for policy 0, policy_version 1588411 (0.0007) [2023-12-27 02:57:59,463][105692] Updated weights for policy 0, policy_version 1588421 (0.0008) [2023-12-27 02:57:59,786][105620] Updated weights for policy 1, policy_version 1592106 (0.0006) [2023-12-27 02:57:59,853][105620] Updated weights for policy 1, policy_version 1592116 (0.0007) [2023-12-27 02:57:59,918][105620] Updated weights for policy 1, policy_version 1592126 (0.0006) [2023-12-27 02:57:59,981][105620] Updated weights for policy 1, policy_version 1592136 (0.0006) [2023-12-27 02:58:00,211][105692] Updated weights for policy 0, policy_version 1588431 (0.0009) [2023-12-27 02:58:00,267][105692] Updated weights for policy 0, policy_version 1588441 (0.0009) [2023-12-27 02:58:00,325][105692] Updated weights for policy 0, policy_version 1588451 (0.0009) [2023-12-27 02:58:00,705][105620] Updated weights for policy 1, policy_version 1592146 (0.0009) [2023-12-27 02:58:00,769][105620] Updated weights for policy 1, policy_version 1592156 (0.0007) [2023-12-27 02:58:00,831][105620] Updated weights for policy 1, policy_version 1592166 (0.0008) [2023-12-27 02:58:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 814350336. Throughput: 0: 9839.8, 1: 9654.0. Samples: 814323624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:58:01,063][104569] Avg episode reward: [(0, '8623.068'), (1, '9180.333')] [2023-12-27 02:58:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001592168_407650304.pth... [2023-12-27 02:58:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001591016_407355392.pth [2023-12-27 02:58:01,082][105692] Updated weights for policy 0, policy_version 1588461 (0.0009) [2023-12-27 02:58:01,151][105692] Updated weights for policy 0, policy_version 1588471 (0.0009) [2023-12-27 02:58:01,201][105692] Updated weights for policy 0, policy_version 1588481 (0.0009) [2023-12-27 02:58:01,237][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001588488_406708224.pth... [2023-12-27 02:58:01,242][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001587336_406413312.pth [2023-12-27 02:58:01,578][105620] Updated weights for policy 1, policy_version 1592176 (0.0008) [2023-12-27 02:58:01,642][105620] Updated weights for policy 1, policy_version 1592186 (0.0008) [2023-12-27 02:58:01,713][105620] Updated weights for policy 1, policy_version 1592196 (0.0008) [2023-12-27 02:58:01,951][105692] Updated weights for policy 0, policy_version 1588491 (0.0008) [2023-12-27 02:58:02,007][105692] Updated weights for policy 0, policy_version 1588501 (0.0005) [2023-12-27 02:58:02,055][105692] Updated weights for policy 0, policy_version 1588511 (0.0005) [2023-12-27 02:58:02,546][105620] Updated weights for policy 1, policy_version 1592206 (0.0009) [2023-12-27 02:58:02,608][105620] Updated weights for policy 1, policy_version 1592216 (0.0010) [2023-12-27 02:58:02,628][105692] Updated weights for policy 0, policy_version 1588521 (0.0006) [2023-12-27 02:58:02,664][105620] Updated weights for policy 1, policy_version 1592226 (0.0007) [2023-12-27 02:58:02,686][105692] Updated weights for policy 0, policy_version 1588531 (0.0007) [2023-12-27 02:58:02,742][105692] Updated weights for policy 0, policy_version 1588541 (0.0006) [2023-12-27 02:58:02,787][105692] Updated weights for policy 0, policy_version 1588551 (0.0010) [2023-12-27 02:58:03,346][105692] Updated weights for policy 0, policy_version 1588561 (0.0008) [2023-12-27 02:58:03,393][105692] Updated weights for policy 0, policy_version 1588571 (0.0009) [2023-12-27 02:58:03,447][105692] Updated weights for policy 0, policy_version 1588581 (0.0009) [2023-12-27 02:58:03,500][105620] Updated weights for policy 1, policy_version 1592236 (0.0009) [2023-12-27 02:58:03,553][105620] Updated weights for policy 1, policy_version 1592246 (0.0010) [2023-12-27 02:58:03,605][105620] Updated weights for policy 1, policy_version 1592256 (0.0009) [2023-12-27 02:58:04,045][105692] Updated weights for policy 0, policy_version 1588591 (0.0007) [2023-12-27 02:58:04,114][105692] Updated weights for policy 0, policy_version 1588601 (0.0006) [2023-12-27 02:58:04,175][105692] Updated weights for policy 0, policy_version 1588611 (0.0010) [2023-12-27 02:58:04,460][105620] Updated weights for policy 1, policy_version 1592266 (0.0009) [2023-12-27 02:58:04,512][105620] Updated weights for policy 1, policy_version 1592276 (0.0008) [2023-12-27 02:58:04,575][105620] Updated weights for policy 1, policy_version 1592286 (0.0008) [2023-12-27 02:58:04,643][105620] Updated weights for policy 1, policy_version 1592296 (0.0008) [2023-12-27 02:58:04,813][105692] Updated weights for policy 0, policy_version 1588621 (0.0011) [2023-12-27 02:58:04,874][105692] Updated weights for policy 0, policy_version 1588631 (0.0010) [2023-12-27 02:58:04,925][105692] Updated weights for policy 0, policy_version 1588641 (0.0011) [2023-12-27 02:58:05,361][105620] Updated weights for policy 1, policy_version 1592306 (0.0005) [2023-12-27 02:58:05,417][105620] Updated weights for policy 1, policy_version 1592316 (0.0005) [2023-12-27 02:58:05,473][105620] Updated weights for policy 1, policy_version 1592326 (0.0005) [2023-12-27 02:58:05,576][105692] Updated weights for policy 0, policy_version 1588651 (0.0009) [2023-12-27 02:58:05,624][105692] Updated weights for policy 0, policy_version 1588661 (0.0005) [2023-12-27 02:58:05,684][105692] Updated weights for policy 0, policy_version 1588671 (0.0005) [2023-12-27 02:58:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 814448640. Throughput: 0: 9850.3, 1: 9614.0. Samples: 814438728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:58:06,063][104569] Avg episode reward: [(0, '8898.081'), (1, '8989.260')] [2023-12-27 02:58:06,219][105620] Updated weights for policy 1, policy_version 1592336 (0.0008) [2023-12-27 02:58:06,288][105620] Updated weights for policy 1, policy_version 1592346 (0.0008) [2023-12-27 02:58:06,307][105692] Updated weights for policy 0, policy_version 1588681 (0.0006) [2023-12-27 02:58:06,351][105620] Updated weights for policy 1, policy_version 1592356 (0.0007) [2023-12-27 02:58:06,361][105692] Updated weights for policy 0, policy_version 1588691 (0.0011) [2023-12-27 02:58:06,421][105692] Updated weights for policy 0, policy_version 1588701 (0.0010) [2023-12-27 02:58:06,484][105692] Updated weights for policy 0, policy_version 1588711 (0.0008) [2023-12-27 02:58:07,113][105620] Updated weights for policy 1, policy_version 1592366 (0.0008) [2023-12-27 02:58:07,178][105620] Updated weights for policy 1, policy_version 1592376 (0.0009) [2023-12-27 02:58:07,226][105692] Updated weights for policy 0, policy_version 1588721 (0.0006) [2023-12-27 02:58:07,229][105620] Updated weights for policy 1, policy_version 1592386 (0.0007) [2023-12-27 02:58:07,272][105692] Updated weights for policy 0, policy_version 1588731 (0.0006) [2023-12-27 02:58:07,319][105692] Updated weights for policy 0, policy_version 1588741 (0.0008) [2023-12-27 02:58:07,917][105620] Updated weights for policy 1, policy_version 1592396 (0.0010) [2023-12-27 02:58:07,982][105620] Updated weights for policy 1, policy_version 1592406 (0.0011) [2023-12-27 02:58:08,049][105620] Updated weights for policy 1, policy_version 1592416 (0.0011) [2023-12-27 02:58:08,125][105692] Updated weights for policy 0, policy_version 1588752 (0.0011) [2023-12-27 02:58:08,188][105692] Updated weights for policy 0, policy_version 1588762 (0.0011) [2023-12-27 02:58:08,250][105692] Updated weights for policy 0, policy_version 1588772 (0.0011) [2023-12-27 02:58:08,712][105620] Updated weights for policy 1, policy_version 1592426 (0.0011) [2023-12-27 02:58:08,760][105620] Updated weights for policy 1, policy_version 1592436 (0.0010) [2023-12-27 02:58:08,811][105620] Updated weights for policy 1, policy_version 1592446 (0.0010) [2023-12-27 02:58:08,860][105620] Updated weights for policy 1, policy_version 1592456 (0.0010) [2023-12-27 02:58:08,905][105692] Updated weights for policy 0, policy_version 1588782 (0.0007) [2023-12-27 02:58:08,953][105692] Updated weights for policy 0, policy_version 1588792 (0.0008) [2023-12-27 02:58:08,997][105692] Updated weights for policy 0, policy_version 1588802 (0.0008) [2023-12-27 02:58:09,655][105620] Updated weights for policy 1, policy_version 1592466 (0.0010) [2023-12-27 02:58:09,716][105620] Updated weights for policy 1, policy_version 1592476 (0.0011) [2023-12-27 02:58:09,776][105620] Updated weights for policy 1, policy_version 1592486 (0.0010) [2023-12-27 02:58:09,805][105692] Updated weights for policy 0, policy_version 1588812 (0.0008) [2023-12-27 02:58:09,866][105692] Updated weights for policy 0, policy_version 1588822 (0.0008) [2023-12-27 02:58:09,919][105692] Updated weights for policy 0, policy_version 1588832 (0.0009) [2023-12-27 02:58:10,462][105620] Updated weights for policy 1, policy_version 1592496 (0.0011) [2023-12-27 02:58:10,518][105620] Updated weights for policy 1, policy_version 1592506 (0.0011) [2023-12-27 02:58:10,574][105620] Updated weights for policy 1, policy_version 1592516 (0.0011) [2023-12-27 02:58:10,701][105692] Updated weights for policy 0, policy_version 1588842 (0.0008) [2023-12-27 02:58:10,761][105692] Updated weights for policy 0, policy_version 1588852 (0.0008) [2023-12-27 02:58:10,806][105692] Updated weights for policy 0, policy_version 1588862 (0.0008) [2023-12-27 02:58:10,858][105692] Updated weights for policy 0, policy_version 1588872 (0.0009) [2023-12-27 02:58:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 814546944. Throughput: 0: 9818.5, 1: 9695.7. Samples: 814554676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:58:11,062][104569] Avg episode reward: [(0, '8804.166'), (1, '8625.923')] [2023-12-27 02:58:11,342][105620] Updated weights for policy 1, policy_version 1592526 (0.0010) [2023-12-27 02:58:11,414][105620] Updated weights for policy 1, policy_version 1592536 (0.0009) [2023-12-27 02:58:11,471][105620] Updated weights for policy 1, policy_version 1592546 (0.0011) [2023-12-27 02:58:11,682][105692] Updated weights for policy 0, policy_version 1588882 (0.0008) [2023-12-27 02:58:11,748][105692] Updated weights for policy 0, policy_version 1588892 (0.0009) [2023-12-27 02:58:11,805][105692] Updated weights for policy 0, policy_version 1588902 (0.0008) [2023-12-27 02:58:12,254][105620] Updated weights for policy 1, policy_version 1592556 (0.0009) [2023-12-27 02:58:12,318][105620] Updated weights for policy 1, policy_version 1592566 (0.0007) [2023-12-27 02:58:12,388][105620] Updated weights for policy 1, policy_version 1592576 (0.0007) [2023-12-27 02:58:12,599][105692] Updated weights for policy 0, policy_version 1588912 (0.0009) [2023-12-27 02:58:12,665][105692] Updated weights for policy 0, policy_version 1588922 (0.0009) [2023-12-27 02:58:12,723][105692] Updated weights for policy 0, policy_version 1588932 (0.0008) [2023-12-27 02:58:13,070][105620] Updated weights for policy 1, policy_version 1592586 (0.0008) [2023-12-27 02:58:13,125][105620] Updated weights for policy 1, policy_version 1592596 (0.0005) [2023-12-27 02:58:13,190][105620] Updated weights for policy 1, policy_version 1592606 (0.0005) [2023-12-27 02:58:13,251][105620] Updated weights for policy 1, policy_version 1592616 (0.0006) [2023-12-27 02:58:13,581][105692] Updated weights for policy 0, policy_version 1588942 (0.0009) [2023-12-27 02:58:13,642][105692] Updated weights for policy 0, policy_version 1588952 (0.0010) [2023-12-27 02:58:13,699][105692] Updated weights for policy 0, policy_version 1588962 (0.0009) [2023-12-27 02:58:13,778][105620] Updated weights for policy 1, policy_version 1592626 (0.0005) [2023-12-27 02:58:13,844][105620] Updated weights for policy 1, policy_version 1592636 (0.0010) [2023-12-27 02:58:13,909][105620] Updated weights for policy 1, policy_version 1592646 (0.0010) [2023-12-27 02:58:14,492][105692] Updated weights for policy 0, policy_version 1588972 (0.0010) [2023-12-27 02:58:14,547][105692] Updated weights for policy 0, policy_version 1588982 (0.0008) [2023-12-27 02:58:14,574][105620] Updated weights for policy 1, policy_version 1592656 (0.0010) [2023-12-27 02:58:14,609][105692] Updated weights for policy 0, policy_version 1588992 (0.0010) [2023-12-27 02:58:14,632][105620] Updated weights for policy 1, policy_version 1592666 (0.0010) [2023-12-27 02:58:14,697][105620] Updated weights for policy 1, policy_version 1592676 (0.0010) [2023-12-27 02:58:15,353][105692] Updated weights for policy 0, policy_version 1589002 (0.0007) [2023-12-27 02:58:15,416][105692] Updated weights for policy 0, policy_version 1589012 (0.0008) [2023-12-27 02:58:15,427][105620] Updated weights for policy 1, policy_version 1592686 (0.0009) [2023-12-27 02:58:15,480][105692] Updated weights for policy 0, policy_version 1589022 (0.0009) [2023-12-27 02:58:15,480][105620] Updated weights for policy 1, policy_version 1592696 (0.0005) [2023-12-27 02:58:15,532][105620] Updated weights for policy 1, policy_version 1592706 (0.0005) [2023-12-27 02:58:15,543][105692] Updated weights for policy 0, policy_version 1589032 (0.0009) [2023-12-27 02:58:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 814637056. Throughput: 0: 9665.4, 1: 9696.3. Samples: 814610804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:58:16,062][104569] Avg episode reward: [(0, '8710.237'), (1, '8719.314')] [2023-12-27 02:58:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001589032_406847488.pth... [2023-12-27 02:58:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001592712_407789568.pth... [2023-12-27 02:58:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001591592_407502848.pth [2023-12-27 02:58:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001587912_406560768.pth [2023-12-27 02:58:16,250][105620] Updated weights for policy 1, policy_version 1592716 (0.0007) [2023-12-27 02:58:16,295][105692] Updated weights for policy 0, policy_version 1589042 (0.0011) [2023-12-27 02:58:16,309][105620] Updated weights for policy 1, policy_version 1592726 (0.0006) [2023-12-27 02:58:16,344][105692] Updated weights for policy 0, policy_version 1589052 (0.0011) [2023-12-27 02:58:16,369][105620] Updated weights for policy 1, policy_version 1592736 (0.0009) [2023-12-27 02:58:16,403][105692] Updated weights for policy 0, policy_version 1589062 (0.0010) [2023-12-27 02:58:17,049][105620] Updated weights for policy 1, policy_version 1592746 (0.0007) [2023-12-27 02:58:17,061][105692] Updated weights for policy 0, policy_version 1589072 (0.0007) [2023-12-27 02:58:17,108][105620] Updated weights for policy 1, policy_version 1592756 (0.0008) [2023-12-27 02:58:17,129][105692] Updated weights for policy 0, policy_version 1589082 (0.0008) [2023-12-27 02:58:17,168][105620] Updated weights for policy 1, policy_version 1592766 (0.0007) [2023-12-27 02:58:17,179][105692] Updated weights for policy 0, policy_version 1589092 (0.0006) [2023-12-27 02:58:17,226][105620] Updated weights for policy 1, policy_version 1592776 (0.0008) [2023-12-27 02:58:17,931][105620] Updated weights for policy 1, policy_version 1592786 (0.0006) [2023-12-27 02:58:17,935][105692] Updated weights for policy 0, policy_version 1589102 (0.0010) [2023-12-27 02:58:17,978][105620] Updated weights for policy 1, policy_version 1592796 (0.0006) [2023-12-27 02:58:17,988][105692] Updated weights for policy 0, policy_version 1589112 (0.0008) [2023-12-27 02:58:18,033][105692] Updated weights for policy 0, policy_version 1589122 (0.0005) [2023-12-27 02:58:18,036][105620] Updated weights for policy 1, policy_version 1592806 (0.0005) [2023-12-27 02:58:18,611][105620] Updated weights for policy 1, policy_version 1592816 (0.0009) [2023-12-27 02:58:18,669][105620] Updated weights for policy 1, policy_version 1592826 (0.0009) [2023-12-27 02:58:18,725][105620] Updated weights for policy 1, policy_version 1592836 (0.0009) [2023-12-27 02:58:18,776][105692] Updated weights for policy 0, policy_version 1589132 (0.0008) [2023-12-27 02:58:18,827][105692] Updated weights for policy 0, policy_version 1589142 (0.0009) [2023-12-27 02:58:18,876][105692] Updated weights for policy 0, policy_version 1589152 (0.0008) [2023-12-27 02:58:19,520][105620] Updated weights for policy 1, policy_version 1592846 (0.0009) [2023-12-27 02:58:19,580][105620] Updated weights for policy 1, policy_version 1592856 (0.0009) [2023-12-27 02:58:19,635][105620] Updated weights for policy 1, policy_version 1592866 (0.0009) [2023-12-27 02:58:19,645][105692] Updated weights for policy 0, policy_version 1589162 (0.0009) [2023-12-27 02:58:19,706][105692] Updated weights for policy 0, policy_version 1589172 (0.0005) [2023-12-27 02:58:19,765][105692] Updated weights for policy 0, policy_version 1589182 (0.0005) [2023-12-27 02:58:19,829][105692] Updated weights for policy 0, policy_version 1589192 (0.0006) [2023-12-27 02:58:20,475][105692] Updated weights for policy 0, policy_version 1589202 (0.0006) [2023-12-27 02:58:20,481][105620] Updated weights for policy 1, policy_version 1592876 (0.0009) [2023-12-27 02:58:20,531][105692] Updated weights for policy 0, policy_version 1589212 (0.0008) [2023-12-27 02:58:20,540][105620] Updated weights for policy 1, policy_version 1592886 (0.0008) [2023-12-27 02:58:20,586][105692] Updated weights for policy 0, policy_version 1589222 (0.0008) [2023-12-27 02:58:20,604][105620] Updated weights for policy 1, policy_version 1592896 (0.0007) [2023-12-27 02:58:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 814735360. Throughput: 0: 9619.8, 1: 9643.4. Samples: 814726256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:58:21,062][104569] Avg episode reward: [(0, '8623.990'), (1, '8901.058')] [2023-12-27 02:58:21,285][105692] Updated weights for policy 0, policy_version 1589232 (0.0007) [2023-12-27 02:58:21,342][105692] Updated weights for policy 0, policy_version 1589242 (0.0008) [2023-12-27 02:58:21,403][105620] Updated weights for policy 1, policy_version 1592906 (0.0008) [2023-12-27 02:58:21,410][105692] Updated weights for policy 0, policy_version 1589252 (0.0008) [2023-12-27 02:58:21,465][105620] Updated weights for policy 1, policy_version 1592916 (0.0008) [2023-12-27 02:58:21,521][105620] Updated weights for policy 1, policy_version 1592926 (0.0009) [2023-12-27 02:58:21,585][105620] Updated weights for policy 1, policy_version 1592936 (0.0008) [2023-12-27 02:58:22,208][105692] Updated weights for policy 0, policy_version 1589262 (0.0007) [2023-12-27 02:58:22,227][105620] Updated weights for policy 1, policy_version 1592946 (0.0007) [2023-12-27 02:58:22,275][105692] Updated weights for policy 0, policy_version 1589272 (0.0009) [2023-12-27 02:58:22,291][105620] Updated weights for policy 1, policy_version 1592956 (0.0009) [2023-12-27 02:58:22,334][105692] Updated weights for policy 0, policy_version 1589282 (0.0009) [2023-12-27 02:58:22,354][105620] Updated weights for policy 1, policy_version 1592966 (0.0006) [2023-12-27 02:58:23,046][105692] Updated weights for policy 0, policy_version 1589292 (0.0008) [2023-12-27 02:58:23,102][105692] Updated weights for policy 0, policy_version 1589302 (0.0008) [2023-12-27 02:58:23,158][105692] Updated weights for policy 0, policy_version 1589312 (0.0008) [2023-12-27 02:58:23,216][105620] Updated weights for policy 1, policy_version 1592976 (0.0008) [2023-12-27 02:58:23,281][105620] Updated weights for policy 1, policy_version 1592986 (0.0008) [2023-12-27 02:58:23,331][105620] Updated weights for policy 1, policy_version 1592996 (0.0005) [2023-12-27 02:58:23,933][105692] Updated weights for policy 0, policy_version 1589322 (0.0006) [2023-12-27 02:58:23,984][105692] Updated weights for policy 0, policy_version 1589332 (0.0007) [2023-12-27 02:58:24,006][105620] Updated weights for policy 1, policy_version 1593006 (0.0007) [2023-12-27 02:58:24,039][105692] Updated weights for policy 0, policy_version 1589342 (0.0008) [2023-12-27 02:58:24,053][105620] Updated weights for policy 1, policy_version 1593016 (0.0008) [2023-12-27 02:58:24,092][105692] Updated weights for policy 0, policy_version 1589352 (0.0006) [2023-12-27 02:58:24,107][105620] Updated weights for policy 1, policy_version 1593026 (0.0007) [2023-12-27 02:58:24,850][105620] Updated weights for policy 1, policy_version 1593036 (0.0007) [2023-12-27 02:58:24,853][105692] Updated weights for policy 0, policy_version 1589362 (0.0005) [2023-12-27 02:58:24,904][105692] Updated weights for policy 0, policy_version 1589372 (0.0005) [2023-12-27 02:58:24,909][105620] Updated weights for policy 1, policy_version 1593046 (0.0007) [2023-12-27 02:58:24,950][105692] Updated weights for policy 0, policy_version 1589382 (0.0006) [2023-12-27 02:58:24,970][105620] Updated weights for policy 1, policy_version 1593056 (0.0008) [2023-12-27 02:58:25,560][105692] Updated weights for policy 0, policy_version 1589392 (0.0005) [2023-12-27 02:58:25,626][105692] Updated weights for policy 0, policy_version 1589402 (0.0005) [2023-12-27 02:58:25,689][105692] Updated weights for policy 0, policy_version 1589412 (0.0005) [2023-12-27 02:58:25,757][105620] Updated weights for policy 1, policy_version 1593066 (0.0009) [2023-12-27 02:58:25,812][105620] Updated weights for policy 1, policy_version 1593076 (0.0010) [2023-12-27 02:58:25,870][105620] Updated weights for policy 1, policy_version 1593086 (0.0009) [2023-12-27 02:58:25,927][105620] Updated weights for policy 1, policy_version 1593096 (0.0009) [2023-12-27 02:58:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 814833664. Throughput: 0: 9642.4, 1: 9565.3. Samples: 814840056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:58:26,063][104569] Avg episode reward: [(0, '8810.131'), (1, '8992.346')] [2023-12-27 02:58:26,214][105692] Updated weights for policy 0, policy_version 1589422 (0.0007) [2023-12-27 02:58:26,262][105692] Updated weights for policy 0, policy_version 1589432 (0.0009) [2023-12-27 02:58:26,320][105692] Updated weights for policy 0, policy_version 1589442 (0.0009) [2023-12-27 02:58:26,728][105620] Updated weights for policy 1, policy_version 1593106 (0.0009) [2023-12-27 02:58:26,773][105620] Updated weights for policy 1, policy_version 1593116 (0.0008) [2023-12-27 02:58:26,831][105620] Updated weights for policy 1, policy_version 1593126 (0.0007) [2023-12-27 02:58:27,094][105692] Updated weights for policy 0, policy_version 1589452 (0.0009) [2023-12-27 02:58:27,140][105692] Updated weights for policy 0, policy_version 1589462 (0.0009) [2023-12-27 02:58:27,187][105692] Updated weights for policy 0, policy_version 1589472 (0.0009) [2023-12-27 02:58:27,581][105620] Updated weights for policy 1, policy_version 1593136 (0.0009) [2023-12-27 02:58:27,636][105620] Updated weights for policy 1, policy_version 1593146 (0.0009) [2023-12-27 02:58:27,690][105620] Updated weights for policy 1, policy_version 1593156 (0.0009) [2023-12-27 02:58:27,950][105692] Updated weights for policy 0, policy_version 1589482 (0.0008) [2023-12-27 02:58:27,997][105692] Updated weights for policy 0, policy_version 1589492 (0.0009) [2023-12-27 02:58:28,046][105692] Updated weights for policy 0, policy_version 1589502 (0.0009) [2023-12-27 02:58:28,104][105692] Updated weights for policy 0, policy_version 1589512 (0.0009) [2023-12-27 02:58:28,430][105620] Updated weights for policy 1, policy_version 1593166 (0.0009) [2023-12-27 02:58:28,477][105620] Updated weights for policy 1, policy_version 1593176 (0.0009) [2023-12-27 02:58:28,525][105620] Updated weights for policy 1, policy_version 1593186 (0.0009) [2023-12-27 02:58:28,874][105692] Updated weights for policy 0, policy_version 1589522 (0.0009) [2023-12-27 02:58:28,923][105692] Updated weights for policy 0, policy_version 1589532 (0.0009) [2023-12-27 02:58:28,969][105692] Updated weights for policy 0, policy_version 1589542 (0.0009) [2023-12-27 02:58:29,242][105620] Updated weights for policy 1, policy_version 1593196 (0.0010) [2023-12-27 02:58:29,297][105620] Updated weights for policy 1, policy_version 1593206 (0.0011) [2023-12-27 02:58:29,357][105620] Updated weights for policy 1, policy_version 1593216 (0.0011) [2023-12-27 02:58:29,827][105692] Updated weights for policy 0, policy_version 1589552 (0.0009) [2023-12-27 02:58:29,887][105692] Updated weights for policy 0, policy_version 1589562 (0.0007) [2023-12-27 02:58:29,948][105692] Updated weights for policy 0, policy_version 1589572 (0.0006) [2023-12-27 02:58:30,030][105620] Updated weights for policy 1, policy_version 1593226 (0.0009) [2023-12-27 02:58:30,081][105620] Updated weights for policy 1, policy_version 1593236 (0.0005) [2023-12-27 02:58:30,133][105620] Updated weights for policy 1, policy_version 1593246 (0.0006) [2023-12-27 02:58:30,187][105620] Updated weights for policy 1, policy_version 1593256 (0.0008) [2023-12-27 02:58:30,682][105692] Updated weights for policy 0, policy_version 1589582 (0.0007) [2023-12-27 02:58:30,734][105692] Updated weights for policy 0, policy_version 1589592 (0.0008) [2023-12-27 02:58:30,790][105692] Updated weights for policy 0, policy_version 1589603 (0.0010) [2023-12-27 02:58:30,869][105620] Updated weights for policy 1, policy_version 1593266 (0.0006) [2023-12-27 02:58:30,921][105620] Updated weights for policy 1, policy_version 1593276 (0.0005) [2023-12-27 02:58:30,971][105620] Updated weights for policy 1, policy_version 1593286 (0.0005) [2023-12-27 02:58:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 814931968. Throughput: 0: 9635.9, 1: 9550.0. Samples: 814898100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:58:31,063][104569] Avg episode reward: [(0, '8990.483'), (1, '8903.466')] [2023-12-27 02:58:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001589608_406994944.pth... [2023-12-27 02:58:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001593288_407937024.pth... [2023-12-27 02:58:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001592168_407650304.pth [2023-12-27 02:58:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001588488_406708224.pth [2023-12-27 02:58:31,587][105692] Updated weights for policy 0, policy_version 1589613 (0.0009) [2023-12-27 02:58:31,653][105692] Updated weights for policy 0, policy_version 1589623 (0.0009) [2023-12-27 02:58:31,698][105620] Updated weights for policy 1, policy_version 1593296 (0.0009) [2023-12-27 02:58:31,723][105692] Updated weights for policy 0, policy_version 1589633 (0.0008) [2023-12-27 02:58:31,764][105620] Updated weights for policy 1, policy_version 1593306 (0.0009) [2023-12-27 02:58:31,820][105620] Updated weights for policy 1, policy_version 1593316 (0.0010) [2023-12-27 02:58:32,468][105692] Updated weights for policy 0, policy_version 1589643 (0.0008) [2023-12-27 02:58:32,518][105692] Updated weights for policy 0, policy_version 1589653 (0.0009) [2023-12-27 02:58:32,520][105620] Updated weights for policy 1, policy_version 1593326 (0.0007) [2023-12-27 02:58:32,571][105692] Updated weights for policy 0, policy_version 1589663 (0.0007) [2023-12-27 02:58:32,577][105620] Updated weights for policy 1, policy_version 1593336 (0.0006) [2023-12-27 02:58:32,631][105620] Updated weights for policy 1, policy_version 1593346 (0.0007) [2023-12-27 02:58:33,317][105692] Updated weights for policy 0, policy_version 1589673 (0.0006) [2023-12-27 02:58:33,361][105620] Updated weights for policy 1, policy_version 1593356 (0.0008) [2023-12-27 02:58:33,371][105692] Updated weights for policy 0, policy_version 1589683 (0.0006) [2023-12-27 02:58:33,414][105620] Updated weights for policy 1, policy_version 1593366 (0.0006) [2023-12-27 02:58:33,420][105692] Updated weights for policy 0, policy_version 1589693 (0.0007) [2023-12-27 02:58:33,456][105620] Updated weights for policy 1, policy_version 1593376 (0.0005) [2023-12-27 02:58:33,470][105692] Updated weights for policy 0, policy_version 1589703 (0.0007) [2023-12-27 02:58:34,107][105620] Updated weights for policy 1, policy_version 1593386 (0.0007) [2023-12-27 02:58:34,167][105620] Updated weights for policy 1, policy_version 1593396 (0.0007) [2023-12-27 02:58:34,169][105692] Updated weights for policy 0, policy_version 1589713 (0.0007) [2023-12-27 02:58:34,227][105620] Updated weights for policy 1, policy_version 1593406 (0.0007) [2023-12-27 02:58:34,235][105692] Updated weights for policy 0, policy_version 1589723 (0.0006) [2023-12-27 02:58:34,279][105620] Updated weights for policy 1, policy_version 1593416 (0.0008) [2023-12-27 02:58:34,287][105692] Updated weights for policy 0, policy_version 1589733 (0.0007) [2023-12-27 02:58:34,985][105692] Updated weights for policy 0, policy_version 1589743 (0.0009) [2023-12-27 02:58:34,994][105620] Updated weights for policy 1, policy_version 1593426 (0.0006) [2023-12-27 02:58:35,036][105692] Updated weights for policy 0, policy_version 1589753 (0.0007) [2023-12-27 02:58:35,050][105620] Updated weights for policy 1, policy_version 1593436 (0.0011) [2023-12-27 02:58:35,085][105692] Updated weights for policy 0, policy_version 1589763 (0.0005) [2023-12-27 02:58:35,106][105620] Updated weights for policy 1, policy_version 1593446 (0.0011) [2023-12-27 02:58:35,765][105620] Updated weights for policy 1, policy_version 1593456 (0.0010) [2023-12-27 02:58:35,827][105620] Updated weights for policy 1, policy_version 1593466 (0.0008) [2023-12-27 02:58:35,843][105692] Updated weights for policy 0, policy_version 1589773 (0.0007) [2023-12-27 02:58:35,880][105620] Updated weights for policy 1, policy_version 1593476 (0.0007) [2023-12-27 02:58:35,898][105692] Updated weights for policy 0, policy_version 1589783 (0.0008) [2023-12-27 02:58:35,954][105692] Updated weights for policy 0, policy_version 1589793 (0.0009) [2023-12-27 02:58:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 815030272. Throughput: 0: 9644.6, 1: 9592.1. Samples: 815014204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:58:36,063][104569] Avg episode reward: [(0, '8537.220'), (1, '8815.392')] [2023-12-27 02:58:36,619][105620] Updated weights for policy 1, policy_version 1593486 (0.0006) [2023-12-27 02:58:36,680][105620] Updated weights for policy 1, policy_version 1593496 (0.0008) [2023-12-27 02:58:36,708][105692] Updated weights for policy 0, policy_version 1589803 (0.0009) [2023-12-27 02:58:36,742][105620] Updated weights for policy 1, policy_version 1593506 (0.0008) [2023-12-27 02:58:36,764][105692] Updated weights for policy 0, policy_version 1589813 (0.0006) [2023-12-27 02:58:36,826][105692] Updated weights for policy 0, policy_version 1589823 (0.0006) [2023-12-27 02:58:37,431][105620] Updated weights for policy 1, policy_version 1593516 (0.0008) [2023-12-27 02:58:37,478][105692] Updated weights for policy 0, policy_version 1589833 (0.0009) [2023-12-27 02:58:37,480][105620] Updated weights for policy 1, policy_version 1593527 (0.0009) [2023-12-27 02:58:37,540][105620] Updated weights for policy 1, policy_version 1593537 (0.0008) [2023-12-27 02:58:37,545][105692] Updated weights for policy 0, policy_version 1589843 (0.0005) [2023-12-27 02:58:37,605][105692] Updated weights for policy 0, policy_version 1589853 (0.0008) [2023-12-27 02:58:37,665][105692] Updated weights for policy 0, policy_version 1589863 (0.0010) [2023-12-27 02:58:38,267][105620] Updated weights for policy 1, policy_version 1593547 (0.0008) [2023-12-27 02:58:38,330][105692] Updated weights for policy 0, policy_version 1589873 (0.0008) [2023-12-27 02:58:38,335][105620] Updated weights for policy 1, policy_version 1593557 (0.0007) [2023-12-27 02:58:38,386][105692] Updated weights for policy 0, policy_version 1589883 (0.0007) [2023-12-27 02:58:38,400][105620] Updated weights for policy 1, policy_version 1593567 (0.0008) [2023-12-27 02:58:38,436][105586] KL-divergence is very high: 119.3211 [2023-12-27 02:58:38,441][105692] Updated weights for policy 0, policy_version 1589893 (0.0007) [2023-12-27 02:58:39,113][105620] Updated weights for policy 1, policy_version 1593577 (0.0008) [2023-12-27 02:58:39,125][105586] KL-divergence is very high: 149.4856 [2023-12-27 02:58:39,166][105692] Updated weights for policy 0, policy_version 1589903 (0.0006) [2023-12-27 02:58:39,166][105586] KL-divergence is very high: 131.4555 [2023-12-27 02:58:39,166][105620] Updated weights for policy 1, policy_version 1593587 (0.0009) [2023-12-27 02:58:39,211][105586] KL-divergence is very high: 114.1189 [2023-12-27 02:58:39,225][105620] Updated weights for policy 1, policy_version 1593597 (0.0008) [2023-12-27 02:58:39,226][105692] Updated weights for policy 0, policy_version 1589913 (0.0006) [2023-12-27 02:58:39,284][105620] Updated weights for policy 1, policy_version 1593607 (0.0009) [2023-12-27 02:58:39,288][105692] Updated weights for policy 0, policy_version 1589923 (0.0007) [2023-12-27 02:58:39,955][105620] Updated weights for policy 1, policy_version 1593617 (0.0011) [2023-12-27 02:58:40,011][105692] Updated weights for policy 0, policy_version 1589933 (0.0008) [2023-12-27 02:58:40,018][105620] Updated weights for policy 1, policy_version 1593627 (0.0011) [2023-12-27 02:58:40,075][105692] Updated weights for policy 0, policy_version 1589943 (0.0006) [2023-12-27 02:58:40,077][105620] Updated weights for policy 1, policy_version 1593637 (0.0011) [2023-12-27 02:58:40,141][105692] Updated weights for policy 0, policy_version 1589953 (0.0010) [2023-12-27 02:58:40,649][105620] Updated weights for policy 1, policy_version 1593647 (0.0007) [2023-12-27 02:58:40,702][105620] Updated weights for policy 1, policy_version 1593657 (0.0011) [2023-12-27 02:58:40,752][105620] Updated weights for policy 1, policy_version 1593667 (0.0010) [2023-12-27 02:58:40,909][105692] Updated weights for policy 0, policy_version 1589963 (0.0008) [2023-12-27 02:58:40,965][105692] Updated weights for policy 0, policy_version 1589973 (0.0005) [2023-12-27 02:58:41,021][105692] Updated weights for policy 0, policy_version 1589983 (0.0010) [2023-12-27 02:58:41,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 815120384. Throughput: 0: 9657.8, 1: 9631.2. Samples: 815132024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:58:41,062][104569] Avg episode reward: [(0, '8441.619'), (1, '8816.675')] [2023-12-27 02:58:41,447][105620] Updated weights for policy 1, policy_version 1593677 (0.0008) [2023-12-27 02:58:41,499][105620] Updated weights for policy 1, policy_version 1593687 (0.0005) [2023-12-27 02:58:41,556][105620] Updated weights for policy 1, policy_version 1593697 (0.0005) [2023-12-27 02:58:41,768][105692] Updated weights for policy 0, policy_version 1589993 (0.0009) [2023-12-27 02:58:41,822][105692] Updated weights for policy 0, policy_version 1590003 (0.0006) [2023-12-27 02:58:41,876][105692] Updated weights for policy 0, policy_version 1590013 (0.0009) [2023-12-27 02:58:41,931][105692] Updated weights for policy 0, policy_version 1590023 (0.0005) [2023-12-27 02:58:42,285][105620] Updated weights for policy 1, policy_version 1593707 (0.0006) [2023-12-27 02:58:42,353][105620] Updated weights for policy 1, policy_version 1593717 (0.0007) [2023-12-27 02:58:42,419][105620] Updated weights for policy 1, policy_version 1593727 (0.0006) [2023-12-27 02:58:42,644][105692] Updated weights for policy 0, policy_version 1590033 (0.0010) [2023-12-27 02:58:42,693][105692] Updated weights for policy 0, policy_version 1590043 (0.0011) [2023-12-27 02:58:42,745][105692] Updated weights for policy 0, policy_version 1590053 (0.0010) [2023-12-27 02:58:43,060][105620] Updated weights for policy 1, policy_version 1593737 (0.0006) [2023-12-27 02:58:43,121][105620] Updated weights for policy 1, policy_version 1593747 (0.0010) [2023-12-27 02:58:43,179][105620] Updated weights for policy 1, policy_version 1593757 (0.0010) [2023-12-27 02:58:43,244][105620] Updated weights for policy 1, policy_version 1593767 (0.0010) [2023-12-27 02:58:43,501][105692] Updated weights for policy 0, policy_version 1590063 (0.0010) [2023-12-27 02:58:43,564][105692] Updated weights for policy 0, policy_version 1590073 (0.0009) [2023-12-27 02:58:43,629][105692] Updated weights for policy 0, policy_version 1590083 (0.0006) [2023-12-27 02:58:43,912][105620] Updated weights for policy 1, policy_version 1593777 (0.0006) [2023-12-27 02:58:43,977][105620] Updated weights for policy 1, policy_version 1593787 (0.0009) [2023-12-27 02:58:44,044][105620] Updated weights for policy 1, policy_version 1593797 (0.0006) [2023-12-27 02:58:44,187][105692] Updated weights for policy 0, policy_version 1590093 (0.0007) [2023-12-27 02:58:44,243][105692] Updated weights for policy 0, policy_version 1590103 (0.0010) [2023-12-27 02:58:44,302][105692] Updated weights for policy 0, policy_version 1590113 (0.0009) [2023-12-27 02:58:44,676][105620] Updated weights for policy 1, policy_version 1593807 (0.0005) [2023-12-27 02:58:44,719][105620] Updated weights for policy 1, policy_version 1593817 (0.0005) [2023-12-27 02:58:44,775][105620] Updated weights for policy 1, policy_version 1593827 (0.0006) [2023-12-27 02:58:45,138][105692] Updated weights for policy 0, policy_version 1590123 (0.0010) [2023-12-27 02:58:45,200][105692] Updated weights for policy 0, policy_version 1590133 (0.0009) [2023-12-27 02:58:45,259][105692] Updated weights for policy 0, policy_version 1590143 (0.0010) [2023-12-27 02:58:45,445][105620] Updated weights for policy 1, policy_version 1593837 (0.0008) [2023-12-27 02:58:45,504][105620] Updated weights for policy 1, policy_version 1593847 (0.0007) [2023-12-27 02:58:45,550][105620] Updated weights for policy 1, policy_version 1593857 (0.0008) [2023-12-27 02:58:45,917][105692] Updated weights for policy 0, policy_version 1590153 (0.0009) [2023-12-27 02:58:45,969][105692] Updated weights for policy 0, policy_version 1590163 (0.0005) [2023-12-27 02:58:46,012][105692] Updated weights for policy 0, policy_version 1590173 (0.0005) [2023-12-27 02:58:46,058][105692] Updated weights for policy 0, policy_version 1590183 (0.0005) [2023-12-27 02:58:46,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 815226880. Throughput: 0: 9613.5, 1: 9669.7. Samples: 815191368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:58:46,063][104569] Avg episode reward: [(0, '8437.785'), (1, '8723.281')] [2023-12-27 02:58:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001590184_407142400.pth... [2023-12-27 02:58:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001593864_408084480.pth... [2023-12-27 02:58:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001592712_407789568.pth [2023-12-27 02:58:46,084][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001589032_406847488.pth [2023-12-27 02:58:46,295][105620] Updated weights for policy 1, policy_version 1593867 (0.0008) [2023-12-27 02:58:46,353][105620] Updated weights for policy 1, policy_version 1593877 (0.0008) [2023-12-27 02:58:46,415][105620] Updated weights for policy 1, policy_version 1593887 (0.0006) [2023-12-27 02:58:46,774][105692] Updated weights for policy 0, policy_version 1590193 (0.0005) [2023-12-27 02:58:46,832][105692] Updated weights for policy 0, policy_version 1590203 (0.0005) [2023-12-27 02:58:46,894][105692] Updated weights for policy 0, policy_version 1590213 (0.0005) [2023-12-27 02:58:46,949][105620] Updated weights for policy 1, policy_version 1593897 (0.0005) [2023-12-27 02:58:47,006][105620] Updated weights for policy 1, policy_version 1593907 (0.0010) [2023-12-27 02:58:47,070][105620] Updated weights for policy 1, policy_version 1593917 (0.0006) [2023-12-27 02:58:47,133][105620] Updated weights for policy 1, policy_version 1593927 (0.0005) [2023-12-27 02:58:47,425][105692] Updated weights for policy 0, policy_version 1590223 (0.0005) [2023-12-27 02:58:47,490][105692] Updated weights for policy 0, policy_version 1590233 (0.0007) [2023-12-27 02:58:47,542][105692] Updated weights for policy 0, policy_version 1590243 (0.0007) [2023-12-27 02:58:47,741][105620] Updated weights for policy 1, policy_version 1593937 (0.0008) [2023-12-27 02:58:47,809][105620] Updated weights for policy 1, policy_version 1593947 (0.0007) [2023-12-27 02:58:47,877][105620] Updated weights for policy 1, policy_version 1593957 (0.0010) [2023-12-27 02:58:48,152][105692] Updated weights for policy 0, policy_version 1590253 (0.0009) [2023-12-27 02:58:48,202][105692] Updated weights for policy 0, policy_version 1590263 (0.0011) [2023-12-27 02:58:48,255][105692] Updated weights for policy 0, policy_version 1590273 (0.0008) [2023-12-27 02:58:48,465][105620] Updated weights for policy 1, policy_version 1593967 (0.0010) [2023-12-27 02:58:48,521][105620] Updated weights for policy 1, policy_version 1593977 (0.0010) [2023-12-27 02:58:48,570][105620] Updated weights for policy 1, policy_version 1593987 (0.0010) [2023-12-27 02:58:48,947][105692] Updated weights for policy 0, policy_version 1590283 (0.0007) [2023-12-27 02:58:48,995][105692] Updated weights for policy 0, policy_version 1590293 (0.0010) [2023-12-27 02:58:49,043][105692] Updated weights for policy 0, policy_version 1590303 (0.0010) [2023-12-27 02:58:49,271][105620] Updated weights for policy 1, policy_version 1593997 (0.0009) [2023-12-27 02:58:49,327][105620] Updated weights for policy 1, policy_version 1594007 (0.0008) [2023-12-27 02:58:49,387][105620] Updated weights for policy 1, policy_version 1594017 (0.0009) [2023-12-27 02:58:49,656][105692] Updated weights for policy 0, policy_version 1590313 (0.0007) [2023-12-27 02:58:49,721][105692] Updated weights for policy 0, policy_version 1590323 (0.0010) [2023-12-27 02:58:49,786][105692] Updated weights for policy 0, policy_version 1590333 (0.0010) [2023-12-27 02:58:49,851][105692] Updated weights for policy 0, policy_version 1590343 (0.0011) [2023-12-27 02:58:50,167][105620] Updated weights for policy 1, policy_version 1594027 (0.0010) [2023-12-27 02:58:50,222][105620] Updated weights for policy 1, policy_version 1594037 (0.0009) [2023-12-27 02:58:50,273][105620] Updated weights for policy 1, policy_version 1594047 (0.0009) [2023-12-27 02:58:50,533][105692] Updated weights for policy 0, policy_version 1590353 (0.0010) [2023-12-27 02:58:50,593][105692] Updated weights for policy 0, policy_version 1590363 (0.0011) [2023-12-27 02:58:50,646][105692] Updated weights for policy 0, policy_version 1590373 (0.0010) [2023-12-27 02:58:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 815325184. Throughput: 0: 9666.1, 1: 9869.8. Samples: 815317840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 02:58:51,062][104569] Avg episode reward: [(0, '8165.809'), (1, '9082.208')] [2023-12-27 02:58:51,067][105620] Updated weights for policy 1, policy_version 1594057 (0.0008) [2023-12-27 02:58:51,127][105620] Updated weights for policy 1, policy_version 1594067 (0.0009) [2023-12-27 02:58:51,180][105620] Updated weights for policy 1, policy_version 1594077 (0.0008) [2023-12-27 02:58:51,229][105620] Updated weights for policy 1, policy_version 1594087 (0.0008) [2023-12-27 02:58:51,441][105692] Updated weights for policy 0, policy_version 1590383 (0.0011) [2023-12-27 02:58:51,491][105692] Updated weights for policy 0, policy_version 1590393 (0.0010) [2023-12-27 02:58:51,550][105692] Updated weights for policy 0, policy_version 1590403 (0.0011) [2023-12-27 02:58:52,040][105620] Updated weights for policy 1, policy_version 1594097 (0.0010) [2023-12-27 02:58:52,090][105620] Updated weights for policy 1, policy_version 1594107 (0.0008) [2023-12-27 02:58:52,146][105620] Updated weights for policy 1, policy_version 1594117 (0.0008) [2023-12-27 02:58:52,294][105692] Updated weights for policy 0, policy_version 1590413 (0.0008) [2023-12-27 02:58:52,340][105692] Updated weights for policy 0, policy_version 1590423 (0.0006) [2023-12-27 02:58:52,417][105692] Updated weights for policy 0, policy_version 1590433 (0.0008) [2023-12-27 02:58:52,959][105620] Updated weights for policy 1, policy_version 1594127 (0.0008) [2023-12-27 02:58:53,017][105620] Updated weights for policy 1, policy_version 1594137 (0.0007) [2023-12-27 02:58:53,078][105620] Updated weights for policy 1, policy_version 1594147 (0.0008) [2023-12-27 02:58:53,084][105692] Updated weights for policy 0, policy_version 1590443 (0.0010) [2023-12-27 02:58:53,147][105692] Updated weights for policy 0, policy_version 1590453 (0.0011) [2023-12-27 02:58:53,217][105692] Updated weights for policy 0, policy_version 1590463 (0.0011) [2023-12-27 02:58:53,799][105692] Updated weights for policy 0, policy_version 1590473 (0.0010) [2023-12-27 02:58:53,833][105620] Updated weights for policy 1, policy_version 1594157 (0.0010) [2023-12-27 02:58:53,859][105692] Updated weights for policy 0, policy_version 1590483 (0.0008) [2023-12-27 02:58:53,893][105620] Updated weights for policy 1, policy_version 1594167 (0.0011) [2023-12-27 02:58:53,922][105692] Updated weights for policy 0, policy_version 1590493 (0.0005) [2023-12-27 02:58:53,946][105620] Updated weights for policy 1, policy_version 1594177 (0.0011) [2023-12-27 02:58:53,980][105692] Updated weights for policy 0, policy_version 1590503 (0.0006) [2023-12-27 02:58:54,600][105692] Updated weights for policy 0, policy_version 1590513 (0.0005) [2023-12-27 02:58:54,652][105620] Updated weights for policy 1, policy_version 1594187 (0.0009) [2023-12-27 02:58:54,663][105692] Updated weights for policy 0, policy_version 1590523 (0.0005) [2023-12-27 02:58:54,722][105620] Updated weights for policy 1, policy_version 1594197 (0.0005) [2023-12-27 02:58:54,732][105692] Updated weights for policy 0, policy_version 1590533 (0.0005) [2023-12-27 02:58:54,787][105620] Updated weights for policy 1, policy_version 1594207 (0.0005) [2023-12-27 02:58:55,266][105692] Updated weights for policy 0, policy_version 1590543 (0.0008) [2023-12-27 02:58:55,323][105620] Updated weights for policy 1, policy_version 1594217 (0.0006) [2023-12-27 02:58:55,326][105692] Updated weights for policy 0, policy_version 1590553 (0.0006) [2023-12-27 02:58:55,379][105620] Updated weights for policy 1, policy_version 1594227 (0.0010) [2023-12-27 02:58:55,386][105692] Updated weights for policy 0, policy_version 1590563 (0.0005) [2023-12-27 02:58:55,434][105620] Updated weights for policy 1, policy_version 1594237 (0.0010) [2023-12-27 02:58:55,502][105620] Updated weights for policy 1, policy_version 1594247 (0.0010) [2023-12-27 02:58:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 815423488. Throughput: 0: 9744.5, 1: 9855.1. Samples: 815436660. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 02:58:56,063][104569] Avg episode reward: [(0, '8355.406'), (1, '9264.260')] [2023-12-27 02:58:56,140][105692] Updated weights for policy 0, policy_version 1590573 (0.0008) [2023-12-27 02:58:56,143][105620] Updated weights for policy 1, policy_version 1594257 (0.0006) [2023-12-27 02:58:56,197][105692] Updated weights for policy 0, policy_version 1590583 (0.0008) [2023-12-27 02:58:56,205][105620] Updated weights for policy 1, policy_version 1594267 (0.0006) [2023-12-27 02:58:56,255][105692] Updated weights for policy 0, policy_version 1590593 (0.0007) [2023-12-27 02:58:56,260][105620] Updated weights for policy 1, policy_version 1594277 (0.0008) [2023-12-27 02:58:56,782][105620] Updated weights for policy 1, policy_version 1594287 (0.0009) [2023-12-27 02:58:56,840][105620] Updated weights for policy 1, policy_version 1594297 (0.0010) [2023-12-27 02:58:56,897][105620] Updated weights for policy 1, policy_version 1594307 (0.0010) [2023-12-27 02:58:57,099][105692] Updated weights for policy 0, policy_version 1590603 (0.0008) [2023-12-27 02:58:57,162][105692] Updated weights for policy 0, policy_version 1590613 (0.0009) [2023-12-27 02:58:57,224][105692] Updated weights for policy 0, policy_version 1590623 (0.0009) [2023-12-27 02:58:57,483][105620] Updated weights for policy 1, policy_version 1594317 (0.0010) [2023-12-27 02:58:57,551][105620] Updated weights for policy 1, policy_version 1594327 (0.0010) [2023-12-27 02:58:57,598][105620] Updated weights for policy 1, policy_version 1594337 (0.0010) [2023-12-27 02:58:57,888][105692] Updated weights for policy 0, policy_version 1590633 (0.0009) [2023-12-27 02:58:57,941][105692] Updated weights for policy 0, policy_version 1590643 (0.0010) [2023-12-27 02:58:57,996][105692] Updated weights for policy 0, policy_version 1590653 (0.0008) [2023-12-27 02:58:58,051][105692] Updated weights for policy 0, policy_version 1590663 (0.0009) [2023-12-27 02:58:58,179][105620] Updated weights for policy 1, policy_version 1594347 (0.0009) [2023-12-27 02:58:58,240][105620] Updated weights for policy 1, policy_version 1594357 (0.0006) [2023-12-27 02:58:58,291][105620] Updated weights for policy 1, policy_version 1594367 (0.0006) [2023-12-27 02:58:58,907][105692] Updated weights for policy 0, policy_version 1590673 (0.0008) [2023-12-27 02:58:58,967][105692] Updated weights for policy 0, policy_version 1590683 (0.0008) [2023-12-27 02:58:59,035][105692] Updated weights for policy 0, policy_version 1590693 (0.0008) [2023-12-27 02:58:59,083][105620] Updated weights for policy 1, policy_version 1594377 (0.0007) [2023-12-27 02:58:59,148][105620] Updated weights for policy 1, policy_version 1594387 (0.0007) [2023-12-27 02:58:59,218][105620] Updated weights for policy 1, policy_version 1594397 (0.0006) [2023-12-27 02:58:59,281][105620] Updated weights for policy 1, policy_version 1594407 (0.0009) [2023-12-27 02:58:59,796][105692] Updated weights for policy 0, policy_version 1590703 (0.0007) [2023-12-27 02:58:59,860][105692] Updated weights for policy 0, policy_version 1590713 (0.0008) [2023-12-27 02:58:59,920][105692] Updated weights for policy 0, policy_version 1590723 (0.0006) [2023-12-27 02:58:59,955][105620] Updated weights for policy 1, policy_version 1594417 (0.0008) [2023-12-27 02:59:00,012][105620] Updated weights for policy 1, policy_version 1594427 (0.0008) [2023-12-27 02:59:00,059][105620] Updated weights for policy 1, policy_version 1594437 (0.0009) [2023-12-27 02:59:00,643][105692] Updated weights for policy 0, policy_version 1590733 (0.0009) [2023-12-27 02:59:00,701][105692] Updated weights for policy 0, policy_version 1590743 (0.0009) [2023-12-27 02:59:00,762][105692] Updated weights for policy 0, policy_version 1590753 (0.0009) [2023-12-27 02:59:00,793][105620] Updated weights for policy 1, policy_version 1594447 (0.0008) [2023-12-27 02:59:00,846][105620] Updated weights for policy 1, policy_version 1594457 (0.0009) [2023-12-27 02:59:00,899][105620] Updated weights for policy 1, policy_version 1594467 (0.0009) [2023-12-27 02:59:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 815529984. Throughput: 0: 9765.4, 1: 9923.3. Samples: 815496796. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 02:59:01,062][104569] Avg episode reward: [(0, '8811.530'), (1, '9085.234')] [2023-12-27 02:59:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001590760_407289856.pth... [2023-12-27 02:59:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001594472_408240128.pth... [2023-12-27 02:59:01,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001589608_406994944.pth [2023-12-27 02:59:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001593288_407937024.pth [2023-12-27 02:59:01,448][105692] Updated weights for policy 0, policy_version 1590763 (0.0007) [2023-12-27 02:59:01,493][105692] Updated weights for policy 0, policy_version 1590773 (0.0009) [2023-12-27 02:59:01,558][105692] Updated weights for policy 0, policy_version 1590783 (0.0006) [2023-12-27 02:59:01,673][105620] Updated weights for policy 1, policy_version 1594477 (0.0009) [2023-12-27 02:59:01,737][105620] Updated weights for policy 1, policy_version 1594487 (0.0008) [2023-12-27 02:59:01,793][105620] Updated weights for policy 1, policy_version 1594497 (0.0009) [2023-12-27 02:59:02,219][105692] Updated weights for policy 0, policy_version 1590793 (0.0007) [2023-12-27 02:59:02,278][105692] Updated weights for policy 0, policy_version 1590803 (0.0006) [2023-12-27 02:59:02,337][105692] Updated weights for policy 0, policy_version 1590813 (0.0008) [2023-12-27 02:59:02,399][105692] Updated weights for policy 0, policy_version 1590823 (0.0009) [2023-12-27 02:59:02,528][105620] Updated weights for policy 1, policy_version 1594507 (0.0009) [2023-12-27 02:59:02,590][105620] Updated weights for policy 1, policy_version 1594517 (0.0009) [2023-12-27 02:59:02,648][105620] Updated weights for policy 1, policy_version 1594527 (0.0010) [2023-12-27 02:59:03,051][105692] Updated weights for policy 0, policy_version 1590833 (0.0009) [2023-12-27 02:59:03,106][105692] Updated weights for policy 0, policy_version 1590843 (0.0005) [2023-12-27 02:59:03,158][105692] Updated weights for policy 0, policy_version 1590853 (0.0005) [2023-12-27 02:59:03,514][105620] Updated weights for policy 1, policy_version 1594537 (0.0010) [2023-12-27 02:59:03,560][105620] Updated weights for policy 1, policy_version 1594547 (0.0006) [2023-12-27 02:59:03,611][105620] Updated weights for policy 1, policy_version 1594557 (0.0008) [2023-12-27 02:59:03,669][105620] Updated weights for policy 1, policy_version 1594567 (0.0010) [2023-12-27 02:59:03,703][105692] Updated weights for policy 0, policy_version 1590863 (0.0005) [2023-12-27 02:59:03,758][105692] Updated weights for policy 0, policy_version 1590873 (0.0005) [2023-12-27 02:59:03,811][105692] Updated weights for policy 0, policy_version 1590883 (0.0005) [2023-12-27 02:59:04,443][105620] Updated weights for policy 1, policy_version 1594577 (0.0009) [2023-12-27 02:59:04,497][105620] Updated weights for policy 1, policy_version 1594587 (0.0009) [2023-12-27 02:59:04,499][105692] Updated weights for policy 0, policy_version 1590893 (0.0006) [2023-12-27 02:59:04,558][105692] Updated weights for policy 0, policy_version 1590903 (0.0006) [2023-12-27 02:59:04,559][105620] Updated weights for policy 1, policy_version 1594597 (0.0008) [2023-12-27 02:59:04,621][105692] Updated weights for policy 0, policy_version 1590913 (0.0009) [2023-12-27 02:59:05,211][105620] Updated weights for policy 1, policy_version 1594608 (0.0009) [2023-12-27 02:59:05,270][105620] Updated weights for policy 1, policy_version 1594619 (0.0010) [2023-12-27 02:59:05,328][105620] Updated weights for policy 1, policy_version 1594629 (0.0009) [2023-12-27 02:59:05,386][105692] Updated weights for policy 0, policy_version 1590923 (0.0010) [2023-12-27 02:59:05,443][105692] Updated weights for policy 0, policy_version 1590933 (0.0009) [2023-12-27 02:59:05,489][105692] Updated weights for policy 0, policy_version 1590943 (0.0008) [2023-12-27 02:59:05,977][105620] Updated weights for policy 1, policy_version 1594639 (0.0009) [2023-12-27 02:59:06,024][105620] Updated weights for policy 1, policy_version 1594649 (0.0009) [2023-12-27 02:59:06,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 815620096. Throughput: 0: 9859.6, 1: 9873.9. Samples: 815614260. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 02:59:06,062][104569] Avg episode reward: [(0, '8346.396'), (1, '8903.037')] [2023-12-27 02:59:06,082][105620] Updated weights for policy 1, policy_version 1594659 (0.0009) [2023-12-27 02:59:06,315][105692] Updated weights for policy 0, policy_version 1590953 (0.0008) [2023-12-27 02:59:06,380][105692] Updated weights for policy 0, policy_version 1590963 (0.0007) [2023-12-27 02:59:06,440][105692] Updated weights for policy 0, policy_version 1590973 (0.0007) [2023-12-27 02:59:06,497][105692] Updated weights for policy 0, policy_version 1590983 (0.0009) [2023-12-27 02:59:06,849][105620] Updated weights for policy 1, policy_version 1594669 (0.0008) [2023-12-27 02:59:06,908][105620] Updated weights for policy 1, policy_version 1594679 (0.0005) [2023-12-27 02:59:06,970][105620] Updated weights for policy 1, policy_version 1594689 (0.0005) [2023-12-27 02:59:07,325][105692] Updated weights for policy 0, policy_version 1590993 (0.0009) [2023-12-27 02:59:07,392][105692] Updated weights for policy 0, policy_version 1591003 (0.0009) [2023-12-27 02:59:07,457][105692] Updated weights for policy 0, policy_version 1591013 (0.0009) [2023-12-27 02:59:07,512][105620] Updated weights for policy 1, policy_version 1594699 (0.0006) [2023-12-27 02:59:07,564][105620] Updated weights for policy 1, policy_version 1594709 (0.0005) [2023-12-27 02:59:07,623][105620] Updated weights for policy 1, policy_version 1594719 (0.0006) [2023-12-27 02:59:08,219][105620] Updated weights for policy 1, policy_version 1594729 (0.0005) [2023-12-27 02:59:08,275][105692] Updated weights for policy 0, policy_version 1591023 (0.0007) [2023-12-27 02:59:08,277][105620] Updated weights for policy 1, policy_version 1594739 (0.0007) [2023-12-27 02:59:08,331][105692] Updated weights for policy 0, policy_version 1591033 (0.0008) [2023-12-27 02:59:08,340][105620] Updated weights for policy 1, policy_version 1594749 (0.0007) [2023-12-27 02:59:08,391][105692] Updated weights for policy 0, policy_version 1591043 (0.0007) [2023-12-27 02:59:08,395][105620] Updated weights for policy 1, policy_version 1594759 (0.0006) [2023-12-27 02:59:09,124][105692] Updated weights for policy 0, policy_version 1591053 (0.0008) [2023-12-27 02:59:09,149][105620] Updated weights for policy 1, policy_version 1594769 (0.0008) [2023-12-27 02:59:09,187][105692] Updated weights for policy 0, policy_version 1591063 (0.0007) [2023-12-27 02:59:09,210][105620] Updated weights for policy 1, policy_version 1594779 (0.0007) [2023-12-27 02:59:09,252][105692] Updated weights for policy 0, policy_version 1591073 (0.0007) [2023-12-27 02:59:09,274][105620] Updated weights for policy 1, policy_version 1594789 (0.0009) [2023-12-27 02:59:10,015][105692] Updated weights for policy 0, policy_version 1591083 (0.0008) [2023-12-27 02:59:10,049][105620] Updated weights for policy 1, policy_version 1594799 (0.0007) [2023-12-27 02:59:10,072][105692] Updated weights for policy 0, policy_version 1591093 (0.0008) [2023-12-27 02:59:10,108][105620] Updated weights for policy 1, policy_version 1594809 (0.0007) [2023-12-27 02:59:10,133][105692] Updated weights for policy 0, policy_version 1591103 (0.0007) [2023-12-27 02:59:10,158][105620] Updated weights for policy 1, policy_version 1594819 (0.0009) [2023-12-27 02:59:10,802][105692] Updated weights for policy 0, policy_version 1591113 (0.0008) [2023-12-27 02:59:10,858][105692] Updated weights for policy 0, policy_version 1591123 (0.0011) [2023-12-27 02:59:10,861][105620] Updated weights for policy 1, policy_version 1594829 (0.0008) [2023-12-27 02:59:10,908][105692] Updated weights for policy 0, policy_version 1591133 (0.0011) [2023-12-27 02:59:10,915][105620] Updated weights for policy 1, policy_version 1594839 (0.0005) [2023-12-27 02:59:10,954][105692] Updated weights for policy 0, policy_version 1591143 (0.0007) [2023-12-27 02:59:10,968][105620] Updated weights for policy 1, policy_version 1594849 (0.0008) [2023-12-27 02:59:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 815726592. Throughput: 0: 9742.7, 1: 9992.3. Samples: 815728128. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 02:59:11,063][104569] Avg episode reward: [(0, '8166.112'), (1, '9173.603')] [2023-12-27 02:59:11,708][105620] Updated weights for policy 1, policy_version 1594859 (0.0009) [2023-12-27 02:59:11,771][105620] Updated weights for policy 1, policy_version 1594869 (0.0007) [2023-12-27 02:59:11,788][105692] Updated weights for policy 0, policy_version 1591153 (0.0008) [2023-12-27 02:59:11,833][105620] Updated weights for policy 1, policy_version 1594879 (0.0005) [2023-12-27 02:59:11,844][105692] Updated weights for policy 0, policy_version 1591163 (0.0008) [2023-12-27 02:59:11,907][105692] Updated weights for policy 0, policy_version 1591173 (0.0009) [2023-12-27 02:59:12,455][105620] Updated weights for policy 1, policy_version 1594889 (0.0005) [2023-12-27 02:59:12,515][105620] Updated weights for policy 1, policy_version 1594899 (0.0008) [2023-12-27 02:59:12,571][105620] Updated weights for policy 1, policy_version 1594909 (0.0011) [2023-12-27 02:59:12,629][105620] Updated weights for policy 1, policy_version 1594919 (0.0011) [2023-12-27 02:59:12,699][105692] Updated weights for policy 0, policy_version 1591183 (0.0009) [2023-12-27 02:59:12,770][105692] Updated weights for policy 0, policy_version 1591193 (0.0009) [2023-12-27 02:59:12,838][105692] Updated weights for policy 0, policy_version 1591203 (0.0009) [2023-12-27 02:59:13,257][105620] Updated weights for policy 1, policy_version 1594929 (0.0006) [2023-12-27 02:59:13,313][105620] Updated weights for policy 1, policy_version 1594939 (0.0006) [2023-12-27 02:59:13,367][105620] Updated weights for policy 1, policy_version 1594949 (0.0005) [2023-12-27 02:59:13,678][105692] Updated weights for policy 0, policy_version 1591213 (0.0009) [2023-12-27 02:59:13,753][105692] Updated weights for policy 0, policy_version 1591223 (0.0010) [2023-12-27 02:59:13,820][105692] Updated weights for policy 0, policy_version 1591233 (0.0009) [2023-12-27 02:59:13,936][105620] Updated weights for policy 1, policy_version 1594959 (0.0007) [2023-12-27 02:59:13,999][105620] Updated weights for policy 1, policy_version 1594969 (0.0009) [2023-12-27 02:59:14,070][105620] Updated weights for policy 1, policy_version 1594979 (0.0009) [2023-12-27 02:59:14,503][105692] Updated weights for policy 0, policy_version 1591243 (0.0008) [2023-12-27 02:59:14,564][105692] Updated weights for policy 0, policy_version 1591253 (0.0008) [2023-12-27 02:59:14,630][105692] Updated weights for policy 0, policy_version 1591263 (0.0008) [2023-12-27 02:59:14,822][105620] Updated weights for policy 1, policy_version 1594989 (0.0009) [2023-12-27 02:59:14,887][105620] Updated weights for policy 1, policy_version 1594999 (0.0007) [2023-12-27 02:59:14,892][105586] KL-divergence is very high: 126.7461 [2023-12-27 02:59:14,945][105586] KL-divergence is very high: 239.2981 [2023-12-27 02:59:14,952][105620] Updated weights for policy 1, policy_version 1595009 (0.0009) [2023-12-27 02:59:15,381][105692] Updated weights for policy 0, policy_version 1591273 (0.0008) [2023-12-27 02:59:15,435][105692] Updated weights for policy 0, policy_version 1591283 (0.0008) [2023-12-27 02:59:15,485][105692] Updated weights for policy 0, policy_version 1591293 (0.0008) [2023-12-27 02:59:15,532][105692] Updated weights for policy 0, policy_version 1591303 (0.0009) [2023-12-27 02:59:15,668][105620] Updated weights for policy 1, policy_version 1595019 (0.0009) [2023-12-27 02:59:15,718][105620] Updated weights for policy 1, policy_version 1595029 (0.0008) [2023-12-27 02:59:15,766][105620] Updated weights for policy 1, policy_version 1595039 (0.0010) [2023-12-27 02:59:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 815816704. Throughput: 0: 9677.0, 1: 10087.8. Samples: 815787516. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 02:59:16,063][104569] Avg episode reward: [(0, '8534.517'), (1, '9264.932')] [2023-12-27 02:59:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001591304_407429120.pth... [2023-12-27 02:59:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001595048_408387584.pth... [2023-12-27 02:59:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001590184_407142400.pth [2023-12-27 02:59:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001593864_408084480.pth [2023-12-27 02:59:16,321][105692] Updated weights for policy 0, policy_version 1591313 (0.0008) [2023-12-27 02:59:16,369][105692] Updated weights for policy 0, policy_version 1591323 (0.0008) [2023-12-27 02:59:16,418][105692] Updated weights for policy 0, policy_version 1591333 (0.0008) [2023-12-27 02:59:16,516][105620] Updated weights for policy 1, policy_version 1595049 (0.0010) [2023-12-27 02:59:16,581][105620] Updated weights for policy 1, policy_version 1595059 (0.0010) [2023-12-27 02:59:16,635][105620] Updated weights for policy 1, policy_version 1595069 (0.0010) [2023-12-27 02:59:16,682][105620] Updated weights for policy 1, policy_version 1595079 (0.0010) [2023-12-27 02:59:17,186][105692] Updated weights for policy 0, policy_version 1591343 (0.0008) [2023-12-27 02:59:17,234][105692] Updated weights for policy 0, policy_version 1591353 (0.0008) [2023-12-27 02:59:17,279][105692] Updated weights for policy 0, policy_version 1591363 (0.0008) [2023-12-27 02:59:17,436][105620] Updated weights for policy 1, policy_version 1595089 (0.0010) [2023-12-27 02:59:17,494][105620] Updated weights for policy 1, policy_version 1595099 (0.0010) [2023-12-27 02:59:17,558][105620] Updated weights for policy 1, policy_version 1595109 (0.0010) [2023-12-27 02:59:18,015][105692] Updated weights for policy 0, policy_version 1591373 (0.0007) [2023-12-27 02:59:18,069][105692] Updated weights for policy 0, policy_version 1591383 (0.0006) [2023-12-27 02:59:18,133][105692] Updated weights for policy 0, policy_version 1591393 (0.0009) [2023-12-27 02:59:18,284][105620] Updated weights for policy 1, policy_version 1595119 (0.0010) [2023-12-27 02:59:18,345][105620] Updated weights for policy 1, policy_version 1595129 (0.0010) [2023-12-27 02:59:18,414][105620] Updated weights for policy 1, policy_version 1595139 (0.0011) [2023-12-27 02:59:18,774][105692] Updated weights for policy 0, policy_version 1591403 (0.0011) [2023-12-27 02:59:18,833][105692] Updated weights for policy 0, policy_version 1591413 (0.0011) [2023-12-27 02:59:18,896][105692] Updated weights for policy 0, policy_version 1591423 (0.0011) [2023-12-27 02:59:19,136][105620] Updated weights for policy 1, policy_version 1595149 (0.0010) [2023-12-27 02:59:19,188][105620] Updated weights for policy 1, policy_version 1595159 (0.0010) [2023-12-27 02:59:19,252][105620] Updated weights for policy 1, policy_version 1595169 (0.0010) [2023-12-27 02:59:19,690][105692] Updated weights for policy 0, policy_version 1591433 (0.0011) [2023-12-27 02:59:19,743][105692] Updated weights for policy 0, policy_version 1591443 (0.0011) [2023-12-27 02:59:19,800][105692] Updated weights for policy 0, policy_version 1591453 (0.0011) [2023-12-27 02:59:19,864][105692] Updated weights for policy 0, policy_version 1591463 (0.0010) [2023-12-27 02:59:20,076][105620] Updated weights for policy 1, policy_version 1595179 (0.0010) [2023-12-27 02:59:20,125][105620] Updated weights for policy 1, policy_version 1595189 (0.0010) [2023-12-27 02:59:20,181][105620] Updated weights for policy 1, policy_version 1595199 (0.0010) [2023-12-27 02:59:20,647][105692] Updated weights for policy 0, policy_version 1591473 (0.0009) [2023-12-27 02:59:20,708][105692] Updated weights for policy 0, policy_version 1591483 (0.0011) [2023-12-27 02:59:20,761][105692] Updated weights for policy 0, policy_version 1591493 (0.0011) [2023-12-27 02:59:20,945][105620] Updated weights for policy 1, policy_version 1595209 (0.0010) [2023-12-27 02:59:21,002][105620] Updated weights for policy 1, policy_version 1595219 (0.0011) [2023-12-27 02:59:21,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 815906816. Throughput: 0: 9708.0, 1: 9981.0. Samples: 815900208. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 02:59:21,063][104569] Avg episode reward: [(0, '8626.678'), (1, '8992.080')] [2023-12-27 02:59:21,071][105620] Updated weights for policy 1, policy_version 1595229 (0.0011) [2023-12-27 02:59:21,139][105620] Updated weights for policy 1, policy_version 1595239 (0.0011) [2023-12-27 02:59:21,585][105692] Updated weights for policy 0, policy_version 1591503 (0.0011) [2023-12-27 02:59:21,652][105692] Updated weights for policy 0, policy_version 1591513 (0.0009) [2023-12-27 02:59:21,722][105692] Updated weights for policy 0, policy_version 1591523 (0.0009) [2023-12-27 02:59:21,899][105620] Updated weights for policy 1, policy_version 1595249 (0.0010) [2023-12-27 02:59:21,958][105620] Updated weights for policy 1, policy_version 1595259 (0.0010) [2023-12-27 02:59:22,007][105620] Updated weights for policy 1, policy_version 1595269 (0.0010) [2023-12-27 02:59:22,495][105692] Updated weights for policy 0, policy_version 1591533 (0.0010) [2023-12-27 02:59:22,551][105692] Updated weights for policy 0, policy_version 1591543 (0.0006) [2023-12-27 02:59:22,621][105692] Updated weights for policy 0, policy_version 1591553 (0.0006) [2023-12-27 02:59:22,728][105620] Updated weights for policy 1, policy_version 1595279 (0.0008) [2023-12-27 02:59:22,791][105620] Updated weights for policy 1, policy_version 1595289 (0.0010) [2023-12-27 02:59:22,849][105620] Updated weights for policy 1, policy_version 1595299 (0.0009) [2023-12-27 02:59:23,174][105692] Updated weights for policy 0, policy_version 1591563 (0.0005) [2023-12-27 02:59:23,228][105692] Updated weights for policy 0, policy_version 1591573 (0.0005) [2023-12-27 02:59:23,285][105692] Updated weights for policy 0, policy_version 1591583 (0.0005) [2023-12-27 02:59:23,501][105620] Updated weights for policy 1, policy_version 1595309 (0.0007) [2023-12-27 02:59:23,561][105620] Updated weights for policy 1, policy_version 1595319 (0.0006) [2023-12-27 02:59:23,620][105620] Updated weights for policy 1, policy_version 1595329 (0.0006) [2023-12-27 02:59:23,884][105692] Updated weights for policy 0, policy_version 1591593 (0.0006) [2023-12-27 02:59:23,945][105692] Updated weights for policy 0, policy_version 1591603 (0.0009) [2023-12-27 02:59:24,001][105692] Updated weights for policy 0, policy_version 1591613 (0.0007) [2023-12-27 02:59:24,047][105692] Updated weights for policy 0, policy_version 1591623 (0.0005) [2023-12-27 02:59:24,273][105620] Updated weights for policy 1, policy_version 1595339 (0.0007) [2023-12-27 02:59:24,331][105620] Updated weights for policy 1, policy_version 1595349 (0.0009) [2023-12-27 02:59:24,389][105620] Updated weights for policy 1, policy_version 1595359 (0.0009) [2023-12-27 02:59:24,706][105692] Updated weights for policy 0, policy_version 1591633 (0.0005) [2023-12-27 02:59:24,757][105692] Updated weights for policy 0, policy_version 1591643 (0.0005) [2023-12-27 02:59:24,811][105692] Updated weights for policy 0, policy_version 1591653 (0.0007) [2023-12-27 02:59:25,186][105620] Updated weights for policy 1, policy_version 1595369 (0.0008) [2023-12-27 02:59:25,233][105620] Updated weights for policy 1, policy_version 1595379 (0.0008) [2023-12-27 02:59:25,284][105620] Updated weights for policy 1, policy_version 1595389 (0.0009) [2023-12-27 02:59:25,334][105620] Updated weights for policy 1, policy_version 1595399 (0.0009) [2023-12-27 02:59:25,453][105692] Updated weights for policy 0, policy_version 1591663 (0.0009) [2023-12-27 02:59:25,508][105692] Updated weights for policy 0, policy_version 1591673 (0.0009) [2023-12-27 02:59:25,571][105692] Updated weights for policy 0, policy_version 1591683 (0.0010) [2023-12-27 02:59:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 816005120. Throughput: 0: 9757.2, 1: 9906.1. Samples: 816016876. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 02:59:26,062][104569] Avg episode reward: [(0, '8442.485'), (1, '8724.559')] [2023-12-27 02:59:26,122][105620] Updated weights for policy 1, policy_version 1595409 (0.0010) [2023-12-27 02:59:26,191][105620] Updated weights for policy 1, policy_version 1595419 (0.0009) [2023-12-27 02:59:26,255][105620] Updated weights for policy 1, policy_version 1595429 (0.0008) [2023-12-27 02:59:26,275][105692] Updated weights for policy 0, policy_version 1591693 (0.0008) [2023-12-27 02:59:26,324][105692] Updated weights for policy 0, policy_version 1591703 (0.0005) [2023-12-27 02:59:26,385][105692] Updated weights for policy 0, policy_version 1591713 (0.0005) [2023-12-27 02:59:27,031][105620] Updated weights for policy 1, policy_version 1595439 (0.0008) [2023-12-27 02:59:27,064][105692] Updated weights for policy 0, policy_version 1591723 (0.0007) [2023-12-27 02:59:27,078][105620] Updated weights for policy 1, policy_version 1595449 (0.0007) [2023-12-27 02:59:27,120][105692] Updated weights for policy 0, policy_version 1591733 (0.0005) [2023-12-27 02:59:27,134][105620] Updated weights for policy 1, policy_version 1595459 (0.0008) [2023-12-27 02:59:27,184][105692] Updated weights for policy 0, policy_version 1591743 (0.0007) [2023-12-27 02:59:27,745][105692] Updated weights for policy 0, policy_version 1591753 (0.0008) [2023-12-27 02:59:27,810][105692] Updated weights for policy 0, policy_version 1591763 (0.0006) [2023-12-27 02:59:27,871][105692] Updated weights for policy 0, policy_version 1591773 (0.0005) [2023-12-27 02:59:27,918][105692] Updated weights for policy 0, policy_version 1591783 (0.0005) [2023-12-27 02:59:27,997][105620] Updated weights for policy 1, policy_version 1595469 (0.0008) [2023-12-27 02:59:28,049][105620] Updated weights for policy 1, policy_version 1595479 (0.0009) [2023-12-27 02:59:28,106][105620] Updated weights for policy 1, policy_version 1595489 (0.0010) [2023-12-27 02:59:28,457][105692] Updated weights for policy 0, policy_version 1591793 (0.0008) [2023-12-27 02:59:28,507][105692] Updated weights for policy 0, policy_version 1591803 (0.0008) [2023-12-27 02:59:28,562][105692] Updated weights for policy 0, policy_version 1591813 (0.0008) [2023-12-27 02:59:28,962][105620] Updated weights for policy 1, policy_version 1595499 (0.0009) [2023-12-27 02:59:29,017][105620] Updated weights for policy 1, policy_version 1595509 (0.0008) [2023-12-27 02:59:29,077][105620] Updated weights for policy 1, policy_version 1595519 (0.0008) [2023-12-27 02:59:29,231][105692] Updated weights for policy 0, policy_version 1591823 (0.0008) [2023-12-27 02:59:29,296][105692] Updated weights for policy 0, policy_version 1591833 (0.0011) [2023-12-27 02:59:29,364][105692] Updated weights for policy 0, policy_version 1591843 (0.0009) [2023-12-27 02:59:29,916][105620] Updated weights for policy 1, policy_version 1595529 (0.0009) [2023-12-27 02:59:29,979][105620] Updated weights for policy 1, policy_version 1595539 (0.0008) [2023-12-27 02:59:30,027][105620] Updated weights for policy 1, policy_version 1595549 (0.0008) [2023-12-27 02:59:30,053][105692] Updated weights for policy 0, policy_version 1591853 (0.0007) [2023-12-27 02:59:30,077][105620] Updated weights for policy 1, policy_version 1595559 (0.0008) [2023-12-27 02:59:30,106][105692] Updated weights for policy 0, policy_version 1591863 (0.0007) [2023-12-27 02:59:30,154][105692] Updated weights for policy 0, policy_version 1591873 (0.0010) [2023-12-27 02:59:30,715][105620] Updated weights for policy 1, policy_version 1595569 (0.0006) [2023-12-27 02:59:30,767][105692] Updated weights for policy 0, policy_version 1591883 (0.0008) [2023-12-27 02:59:30,771][105620] Updated weights for policy 1, policy_version 1595579 (0.0008) [2023-12-27 02:59:30,818][105692] Updated weights for policy 0, policy_version 1591893 (0.0005) [2023-12-27 02:59:30,835][105620] Updated weights for policy 1, policy_version 1595589 (0.0009) [2023-12-27 02:59:30,866][105692] Updated weights for policy 0, policy_version 1591903 (0.0005) [2023-12-27 02:59:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 816111616. Throughput: 0: 9839.4, 1: 9804.7. Samples: 816075352. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 02:59:31,062][104569] Avg episode reward: [(0, '8254.755'), (1, '8635.727')] [2023-12-27 02:59:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001595592_408526848.pth... [2023-12-27 02:59:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001591912_407584768.pth... [2023-12-27 02:59:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001594472_408240128.pth [2023-12-27 02:59:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001590760_407289856.pth [2023-12-27 02:59:31,557][105692] Updated weights for policy 0, policy_version 1591913 (0.0007) [2023-12-27 02:59:31,603][105620] Updated weights for policy 1, policy_version 1595599 (0.0007) [2023-12-27 02:59:31,613][105692] Updated weights for policy 0, policy_version 1591923 (0.0007) [2023-12-27 02:59:31,656][105620] Updated weights for policy 1, policy_version 1595609 (0.0006) [2023-12-27 02:59:31,675][105692] Updated weights for policy 0, policy_version 1591933 (0.0008) [2023-12-27 02:59:31,711][105620] Updated weights for policy 1, policy_version 1595619 (0.0007) [2023-12-27 02:59:31,735][105692] Updated weights for policy 0, policy_version 1591943 (0.0007) [2023-12-27 02:59:32,457][105692] Updated weights for policy 0, policy_version 1591953 (0.0009) [2023-12-27 02:59:32,478][105620] Updated weights for policy 1, policy_version 1595629 (0.0009) [2023-12-27 02:59:32,518][105692] Updated weights for policy 0, policy_version 1591963 (0.0007) [2023-12-27 02:59:32,528][105620] Updated weights for policy 1, policy_version 1595639 (0.0007) [2023-12-27 02:59:32,577][105692] Updated weights for policy 0, policy_version 1591973 (0.0008) [2023-12-27 02:59:32,587][105620] Updated weights for policy 1, policy_version 1595649 (0.0007) [2023-12-27 02:59:33,144][105692] Updated weights for policy 0, policy_version 1591983 (0.0006) [2023-12-27 02:59:33,201][105692] Updated weights for policy 0, policy_version 1591993 (0.0006) [2023-12-27 02:59:33,253][105692] Updated weights for policy 0, policy_version 1592003 (0.0006) [2023-12-27 02:59:33,414][105620] Updated weights for policy 1, policy_version 1595659 (0.0010) [2023-12-27 02:59:33,472][105620] Updated weights for policy 1, policy_version 1595669 (0.0010) [2023-12-27 02:59:33,528][105620] Updated weights for policy 1, policy_version 1595679 (0.0010) [2023-12-27 02:59:33,829][105692] Updated weights for policy 0, policy_version 1592013 (0.0007) [2023-12-27 02:59:33,891][105692] Updated weights for policy 0, policy_version 1592023 (0.0009) [2023-12-27 02:59:33,949][105692] Updated weights for policy 0, policy_version 1592033 (0.0009) [2023-12-27 02:59:34,363][105620] Updated weights for policy 1, policy_version 1595690 (0.0010) [2023-12-27 02:59:34,421][105620] Updated weights for policy 1, policy_version 1595700 (0.0009) [2023-12-27 02:59:34,483][105620] Updated weights for policy 1, policy_version 1595710 (0.0009) [2023-12-27 02:59:34,544][105620] Updated weights for policy 1, policy_version 1595720 (0.0008) [2023-12-27 02:59:34,713][105692] Updated weights for policy 0, policy_version 1592043 (0.0009) [2023-12-27 02:59:34,784][105692] Updated weights for policy 0, policy_version 1592053 (0.0006) [2023-12-27 02:59:34,847][105692] Updated weights for policy 0, policy_version 1592063 (0.0007) [2023-12-27 02:59:35,277][105620] Updated weights for policy 1, policy_version 1595730 (0.0009) [2023-12-27 02:59:35,326][105620] Updated weights for policy 1, policy_version 1595740 (0.0009) [2023-12-27 02:59:35,381][105620] Updated weights for policy 1, policy_version 1595750 (0.0009) [2023-12-27 02:59:35,459][105692] Updated weights for policy 0, policy_version 1592073 (0.0006) [2023-12-27 02:59:35,516][105692] Updated weights for policy 0, policy_version 1592083 (0.0009) [2023-12-27 02:59:35,577][105692] Updated weights for policy 0, policy_version 1592093 (0.0009) [2023-12-27 02:59:35,642][105692] Updated weights for policy 0, policy_version 1592103 (0.0009) [2023-12-27 02:59:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 816201728. Throughput: 0: 9834.8, 1: 9624.7. Samples: 816193516. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 02:59:36,062][104569] Avg episode reward: [(0, '8343.838'), (1, '8905.495')] [2023-12-27 02:59:36,087][105620] Updated weights for policy 1, policy_version 1595760 (0.0008) [2023-12-27 02:59:36,181][105620] Updated weights for policy 1, policy_version 1595770 (0.0008) [2023-12-27 02:59:36,232][105620] Updated weights for policy 1, policy_version 1595780 (0.0008) [2023-12-27 02:59:36,451][105692] Updated weights for policy 0, policy_version 1592113 (0.0011) [2023-12-27 02:59:36,511][105692] Updated weights for policy 0, policy_version 1592123 (0.0010) [2023-12-27 02:59:36,577][105692] Updated weights for policy 0, policy_version 1592133 (0.0011) [2023-12-27 02:59:36,994][105620] Updated weights for policy 1, policy_version 1595790 (0.0009) [2023-12-27 02:59:37,054][105620] Updated weights for policy 1, policy_version 1595800 (0.0008) [2023-12-27 02:59:37,115][105620] Updated weights for policy 1, policy_version 1595810 (0.0008) [2023-12-27 02:59:37,324][105692] Updated weights for policy 0, policy_version 1592143 (0.0010) [2023-12-27 02:59:37,383][105692] Updated weights for policy 0, policy_version 1592153 (0.0010) [2023-12-27 02:59:37,438][105692] Updated weights for policy 0, policy_version 1592163 (0.0010) [2023-12-27 02:59:37,878][105620] Updated weights for policy 1, policy_version 1595820 (0.0009) [2023-12-27 02:59:37,936][105620] Updated weights for policy 1, policy_version 1595831 (0.0010) [2023-12-27 02:59:38,004][105620] Updated weights for policy 1, policy_version 1595841 (0.0010) [2023-12-27 02:59:38,126][105692] Updated weights for policy 0, policy_version 1592173 (0.0008) [2023-12-27 02:59:38,192][105692] Updated weights for policy 0, policy_version 1592183 (0.0010) [2023-12-27 02:59:38,258][105692] Updated weights for policy 0, policy_version 1592193 (0.0005) [2023-12-27 02:59:38,787][105620] Updated weights for policy 1, policy_version 1595851 (0.0008) [2023-12-27 02:59:38,849][105620] Updated weights for policy 1, policy_version 1595861 (0.0007) [2023-12-27 02:59:38,900][105692] Updated weights for policy 0, policy_version 1592203 (0.0006) [2023-12-27 02:59:38,903][105620] Updated weights for policy 1, policy_version 1595871 (0.0008) [2023-12-27 02:59:38,963][105692] Updated weights for policy 0, policy_version 1592213 (0.0005) [2023-12-27 02:59:39,025][105692] Updated weights for policy 0, policy_version 1592223 (0.0005) [2023-12-27 02:59:39,637][105692] Updated weights for policy 0, policy_version 1592233 (0.0007) [2023-12-27 02:59:39,696][105692] Updated weights for policy 0, policy_version 1592243 (0.0010) [2023-12-27 02:59:39,703][105620] Updated weights for policy 1, policy_version 1595881 (0.0008) [2023-12-27 02:59:39,745][105692] Updated weights for policy 0, policy_version 1592253 (0.0011) [2023-12-27 02:59:39,760][105620] Updated weights for policy 1, policy_version 1595891 (0.0006) [2023-12-27 02:59:39,802][105692] Updated weights for policy 0, policy_version 1592263 (0.0011) [2023-12-27 02:59:39,824][105620] Updated weights for policy 1, policy_version 1595901 (0.0006) [2023-12-27 02:59:39,887][105620] Updated weights for policy 1, policy_version 1595911 (0.0006) [2023-12-27 02:59:40,578][105692] Updated weights for policy 0, policy_version 1592273 (0.0010) [2023-12-27 02:59:40,616][105620] Updated weights for policy 1, policy_version 1595921 (0.0006) [2023-12-27 02:59:40,638][105692] Updated weights for policy 0, policy_version 1592283 (0.0011) [2023-12-27 02:59:40,676][105620] Updated weights for policy 1, policy_version 1595931 (0.0006) [2023-12-27 02:59:40,690][105692] Updated weights for policy 0, policy_version 1592293 (0.0011) [2023-12-27 02:59:40,741][105620] Updated weights for policy 1, policy_version 1595941 (0.0007) [2023-12-27 02:59:41,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19660.7, 300 sec: 19605.3). Total num frames: 816300032. Throughput: 0: 9769.2, 1: 9581.8. Samples: 816307452. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 02:59:41,063][104569] Avg episode reward: [(0, '8625.538'), (1, '9175.956')] [2023-12-27 02:59:41,406][105692] Updated weights for policy 0, policy_version 1592303 (0.0011) [2023-12-27 02:59:41,465][105692] Updated weights for policy 0, policy_version 1592313 (0.0008) [2023-12-27 02:59:41,531][105692] Updated weights for policy 0, policy_version 1592323 (0.0008) [2023-12-27 02:59:41,538][105620] Updated weights for policy 1, policy_version 1595951 (0.0009) [2023-12-27 02:59:41,594][105620] Updated weights for policy 1, policy_version 1595961 (0.0009) [2023-12-27 02:59:41,656][105620] Updated weights for policy 1, policy_version 1595971 (0.0008) [2023-12-27 02:59:42,271][105692] Updated weights for policy 0, policy_version 1592333 (0.0007) [2023-12-27 02:59:42,335][105692] Updated weights for policy 0, policy_version 1592343 (0.0009) [2023-12-27 02:59:42,397][105692] Updated weights for policy 0, policy_version 1592353 (0.0012) [2023-12-27 02:59:42,442][105620] Updated weights for policy 1, policy_version 1595981 (0.0008) [2023-12-27 02:59:42,499][105620] Updated weights for policy 1, policy_version 1595991 (0.0008) [2023-12-27 02:59:42,546][105586] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000003 [2023-12-27 02:59:43,035][105692] Updated weights for policy 0, policy_version 1592363 (0.0009) [2023-12-27 02:59:43,091][105692] Updated weights for policy 0, policy_version 1592373 (0.0005) [2023-12-27 02:59:43,153][105692] Updated weights for policy 0, policy_version 1592383 (0.0010) [2023-12-27 02:59:43,351][105620] Updated weights for policy 1, policy_version 1596001 (0.0008) [2023-12-27 02:59:43,410][105620] Updated weights for policy 1, policy_version 1596011 (0.0008) [2023-12-27 02:59:43,462][105620] Updated weights for policy 1, policy_version 1596021 (0.0008) [2023-12-27 02:59:43,510][105620] Updated weights for policy 1, policy_version 1596031 (0.0007) [2023-12-27 02:59:43,868][105692] Updated weights for policy 0, policy_version 1592393 (0.0010) [2023-12-27 02:59:43,923][105692] Updated weights for policy 0, policy_version 1592403 (0.0010) [2023-12-27 02:59:43,975][105692] Updated weights for policy 0, policy_version 1592413 (0.0006) [2023-12-27 02:59:44,028][105692] Updated weights for policy 0, policy_version 1592423 (0.0010) [2023-12-27 02:59:44,290][105620] Updated weights for policy 1, policy_version 1596041 (0.0007) [2023-12-27 02:59:44,352][105620] Updated weights for policy 1, policy_version 1596051 (0.0006) [2023-12-27 02:59:44,415][105620] Updated weights for policy 1, policy_version 1596061 (0.0005) [2023-12-27 02:59:44,711][105692] Updated weights for policy 0, policy_version 1592433 (0.0006) [2023-12-27 02:59:44,771][105692] Updated weights for policy 0, policy_version 1592443 (0.0008) [2023-12-27 02:59:44,833][105692] Updated weights for policy 0, policy_version 1592453 (0.0011) [2023-12-27 02:59:45,132][105620] Updated weights for policy 1, policy_version 1596071 (0.0007) [2023-12-27 02:59:45,201][105620] Updated weights for policy 1, policy_version 1596081 (0.0008) [2023-12-27 02:59:45,261][105620] Updated weights for policy 1, policy_version 1596091 (0.0008) [2023-12-27 02:59:45,518][105692] Updated weights for policy 0, policy_version 1592463 (0.0011) [2023-12-27 02:59:45,577][105692] Updated weights for policy 0, policy_version 1592473 (0.0010) [2023-12-27 02:59:45,636][105692] Updated weights for policy 0, policy_version 1592483 (0.0010) [2023-12-27 02:59:46,035][105620] Updated weights for policy 1, policy_version 1596101 (0.0008) [2023-12-27 02:59:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 816390144. Throughput: 0: 9838.6, 1: 9430.7. Samples: 816363912. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 02:59:46,062][104569] Avg episode reward: [(0, '8532.589'), (1, '9264.772')] [2023-12-27 02:59:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001592488_407732224.pth... [2023-12-27 02:59:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001591304_407429120.pth [2023-12-27 02:59:46,087][105620] Updated weights for policy 1, policy_version 1596111 (0.0008) [2023-12-27 02:59:46,145][105620] Updated weights for policy 1, policy_version 1596121 (0.0008) [2023-12-27 02:59:46,184][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001596128_408666112.pth... [2023-12-27 02:59:46,189][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001595048_408387584.pth [2023-12-27 02:59:46,348][105692] Updated weights for policy 0, policy_version 1592493 (0.0010) [2023-12-27 02:59:46,403][105692] Updated weights for policy 0, policy_version 1592503 (0.0010) [2023-12-27 02:59:46,459][105692] Updated weights for policy 0, policy_version 1592513 (0.0011) [2023-12-27 02:59:46,884][105620] Updated weights for policy 1, policy_version 1596131 (0.0008) [2023-12-27 02:59:46,930][105620] Updated weights for policy 1, policy_version 1596141 (0.0007) [2023-12-27 02:59:46,993][105620] Updated weights for policy 1, policy_version 1596151 (0.0008) [2023-12-27 02:59:47,152][105692] Updated weights for policy 0, policy_version 1592523 (0.0011) [2023-12-27 02:59:47,213][105692] Updated weights for policy 0, policy_version 1592533 (0.0011) [2023-12-27 02:59:47,276][105692] Updated weights for policy 0, policy_version 1592543 (0.0010) [2023-12-27 02:59:47,570][105620] Updated weights for policy 1, policy_version 1596161 (0.0005) [2023-12-27 02:59:47,616][105620] Updated weights for policy 1, policy_version 1596171 (0.0005) [2023-12-27 02:59:47,662][105620] Updated weights for policy 1, policy_version 1596181 (0.0005) [2023-12-27 02:59:47,710][105620] Updated weights for policy 1, policy_version 1596191 (0.0005) [2023-12-27 02:59:47,988][105692] Updated weights for policy 0, policy_version 1592553 (0.0010) [2023-12-27 02:59:48,043][105692] Updated weights for policy 0, policy_version 1592563 (0.0010) [2023-12-27 02:59:48,097][105692] Updated weights for policy 0, policy_version 1592573 (0.0008) [2023-12-27 02:59:48,145][105692] Updated weights for policy 0, policy_version 1592583 (0.0005) [2023-12-27 02:59:48,257][105620] Updated weights for policy 1, policy_version 1596201 (0.0005) [2023-12-27 02:59:48,325][105620] Updated weights for policy 1, policy_version 1596211 (0.0007) [2023-12-27 02:59:48,387][105620] Updated weights for policy 1, policy_version 1596221 (0.0007) [2023-12-27 02:59:48,874][105692] Updated weights for policy 0, policy_version 1592593 (0.0006) [2023-12-27 02:59:48,942][105692] Updated weights for policy 0, policy_version 1592603 (0.0006) [2023-12-27 02:59:48,992][105620] Updated weights for policy 1, policy_version 1596231 (0.0006) [2023-12-27 02:59:49,006][105692] Updated weights for policy 0, policy_version 1592613 (0.0006) [2023-12-27 02:59:49,042][105620] Updated weights for policy 1, policy_version 1596241 (0.0005) [2023-12-27 02:59:49,107][105620] Updated weights for policy 1, policy_version 1596251 (0.0005) [2023-12-27 02:59:49,694][105692] Updated weights for policy 0, policy_version 1592623 (0.0008) [2023-12-27 02:59:49,761][105692] Updated weights for policy 0, policy_version 1592633 (0.0009) [2023-12-27 02:59:49,785][105620] Updated weights for policy 1, policy_version 1596261 (0.0006) [2023-12-27 02:59:49,817][105692] Updated weights for policy 0, policy_version 1592643 (0.0008) [2023-12-27 02:59:49,848][105620] Updated weights for policy 1, policy_version 1596271 (0.0007) [2023-12-27 02:59:49,873][105586] KL-divergence is very high: 142.1105 [2023-12-27 02:59:49,908][105586] KL-divergence is very high: 108.4296 [2023-12-27 02:59:49,916][105620] Updated weights for policy 1, policy_version 1596281 (0.0007) [2023-12-27 02:59:49,929][105586] KL-divergence is very high: 153.8441 [2023-12-27 02:59:50,602][105620] Updated weights for policy 1, policy_version 1596291 (0.0007) [2023-12-27 02:59:50,667][105620] Updated weights for policy 1, policy_version 1596301 (0.0007) [2023-12-27 02:59:50,674][105692] Updated weights for policy 0, policy_version 1592653 (0.0006) [2023-12-27 02:59:50,725][105620] Updated weights for policy 1, policy_version 1596311 (0.0009) [2023-12-27 02:59:50,731][105692] Updated weights for policy 0, policy_version 1592663 (0.0006) [2023-12-27 02:59:50,795][105692] Updated weights for policy 0, policy_version 1592673 (0.0007) [2023-12-27 02:59:51,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 816496640. Throughput: 0: 9799.0, 1: 9549.4. Samples: 816484940. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 02:59:51,062][104569] Avg episode reward: [(0, '8253.766'), (1, '8995.638')] [2023-12-27 02:59:51,452][105620] Updated weights for policy 1, policy_version 1596321 (0.0007) [2023-12-27 02:59:51,517][105620] Updated weights for policy 1, policy_version 1596331 (0.0008) [2023-12-27 02:59:51,569][105620] Updated weights for policy 1, policy_version 1596341 (0.0007) [2023-12-27 02:59:51,617][105692] Updated weights for policy 0, policy_version 1592683 (0.0008) [2023-12-27 02:59:51,624][105620] Updated weights for policy 1, policy_version 1596351 (0.0006) [2023-12-27 02:59:51,679][105692] Updated weights for policy 0, policy_version 1592693 (0.0010) [2023-12-27 02:59:51,752][105692] Updated weights for policy 0, policy_version 1592703 (0.0010) [2023-12-27 02:59:52,369][105620] Updated weights for policy 1, policy_version 1596361 (0.0009) [2023-12-27 02:59:52,433][105620] Updated weights for policy 1, policy_version 1596371 (0.0008) [2023-12-27 02:59:52,490][105620] Updated weights for policy 1, policy_version 1596381 (0.0008) [2023-12-27 02:59:52,521][105692] Updated weights for policy 0, policy_version 1592713 (0.0009) [2023-12-27 02:59:52,584][105692] Updated weights for policy 0, policy_version 1592723 (0.0007) [2023-12-27 02:59:52,656][105692] Updated weights for policy 0, policy_version 1592733 (0.0010) [2023-12-27 02:59:52,718][105692] Updated weights for policy 0, policy_version 1592743 (0.0009) [2023-12-27 02:59:53,255][105620] Updated weights for policy 1, policy_version 1596391 (0.0009) [2023-12-27 02:59:53,301][105620] Updated weights for policy 1, policy_version 1596401 (0.0009) [2023-12-27 02:59:53,355][105620] Updated weights for policy 1, policy_version 1596411 (0.0007) [2023-12-27 02:59:53,370][105692] Updated weights for policy 0, policy_version 1592753 (0.0007) [2023-12-27 02:59:53,422][105692] Updated weights for policy 0, policy_version 1592763 (0.0008) [2023-12-27 02:59:53,480][105692] Updated weights for policy 0, policy_version 1592773 (0.0009) [2023-12-27 02:59:54,098][105620] Updated weights for policy 1, policy_version 1596421 (0.0007) [2023-12-27 02:59:54,158][105620] Updated weights for policy 1, policy_version 1596431 (0.0006) [2023-12-27 02:59:54,221][105620] Updated weights for policy 1, policy_version 1596441 (0.0006) [2023-12-27 02:59:54,260][105692] Updated weights for policy 0, policy_version 1592783 (0.0007) [2023-12-27 02:59:54,317][105692] Updated weights for policy 0, policy_version 1592793 (0.0009) [2023-12-27 02:59:54,375][105692] Updated weights for policy 0, policy_version 1592803 (0.0007) [2023-12-27 02:59:54,791][105620] Updated weights for policy 1, policy_version 1596451 (0.0009) [2023-12-27 02:59:54,842][105620] Updated weights for policy 1, policy_version 1596461 (0.0009) [2023-12-27 02:59:54,895][105620] Updated weights for policy 1, policy_version 1596471 (0.0009) [2023-12-27 02:59:55,080][105692] Updated weights for policy 0, policy_version 1592813 (0.0009) [2023-12-27 02:59:55,134][105692] Updated weights for policy 0, policy_version 1592823 (0.0009) [2023-12-27 02:59:55,197][105692] Updated weights for policy 0, policy_version 1592833 (0.0006) [2023-12-27 02:59:55,646][105620] Updated weights for policy 1, policy_version 1596482 (0.0008) [2023-12-27 02:59:55,693][105620] Updated weights for policy 1, policy_version 1596492 (0.0009) [2023-12-27 02:59:55,746][105620] Updated weights for policy 1, policy_version 1596503 (0.0010) [2023-12-27 02:59:55,839][105692] Updated weights for policy 0, policy_version 1592843 (0.0005) [2023-12-27 02:59:55,906][105692] Updated weights for policy 0, policy_version 1592853 (0.0007) [2023-12-27 02:59:55,958][105585] KL-divergence is very high: 129.3225 [2023-12-27 02:59:55,967][105692] Updated weights for policy 0, policy_version 1592863 (0.0009) [2023-12-27 02:59:56,004][105585] KL-divergence is very high: 133.7192 [2023-12-27 02:59:56,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 816594944. Throughput: 0: 9843.3, 1: 9500.5. Samples: 816598600. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 02:59:56,063][104569] Avg episode reward: [(0, '8075.105'), (1, '8904.855')] [2023-12-27 02:59:56,425][105620] Updated weights for policy 1, policy_version 1596513 (0.0008) [2023-12-27 02:59:56,483][105620] Updated weights for policy 1, policy_version 1596523 (0.0005) [2023-12-27 02:59:56,544][105620] Updated weights for policy 1, policy_version 1596533 (0.0005) [2023-12-27 02:59:56,579][105692] Updated weights for policy 0, policy_version 1592873 (0.0009) [2023-12-27 02:59:56,614][105620] Updated weights for policy 1, policy_version 1596543 (0.0005) [2023-12-27 02:59:56,632][105692] Updated weights for policy 0, policy_version 1592883 (0.0009) [2023-12-27 02:59:56,683][105692] Updated weights for policy 0, policy_version 1592893 (0.0009) [2023-12-27 02:59:56,741][105692] Updated weights for policy 0, policy_version 1592903 (0.0009) [2023-12-27 02:59:57,289][105620] Updated weights for policy 1, policy_version 1596553 (0.0006) [2023-12-27 02:59:57,348][105620] Updated weights for policy 1, policy_version 1596563 (0.0008) [2023-12-27 02:59:57,403][105620] Updated weights for policy 1, policy_version 1596573 (0.0006) [2023-12-27 02:59:57,426][105692] Updated weights for policy 0, policy_version 1592913 (0.0009) [2023-12-27 02:59:57,479][105692] Updated weights for policy 0, policy_version 1592923 (0.0010) [2023-12-27 02:59:57,532][105692] Updated weights for policy 0, policy_version 1592933 (0.0009) [2023-12-27 02:59:58,075][105620] Updated weights for policy 1, policy_version 1596583 (0.0008) [2023-12-27 02:59:58,128][105620] Updated weights for policy 1, policy_version 1596593 (0.0009) [2023-12-27 02:59:58,140][105692] Updated weights for policy 0, policy_version 1592943 (0.0007) [2023-12-27 02:59:58,191][105620] Updated weights for policy 1, policy_version 1596603 (0.0008) [2023-12-27 02:59:58,206][105692] Updated weights for policy 0, policy_version 1592953 (0.0008) [2023-12-27 02:59:58,258][105692] Updated weights for policy 0, policy_version 1592963 (0.0008) [2023-12-27 02:59:58,955][105692] Updated weights for policy 0, policy_version 1592973 (0.0006) [2023-12-27 02:59:59,018][105692] Updated weights for policy 0, policy_version 1592983 (0.0007) [2023-12-27 02:59:59,055][105620] Updated weights for policy 1, policy_version 1596613 (0.0010) [2023-12-27 02:59:59,077][105692] Updated weights for policy 0, policy_version 1592993 (0.0009) [2023-12-27 02:59:59,112][105620] Updated weights for policy 1, policy_version 1596623 (0.0005) [2023-12-27 02:59:59,175][105620] Updated weights for policy 1, policy_version 1596633 (0.0008) [2023-12-27 02:59:59,709][105692] Updated weights for policy 0, policy_version 1593003 (0.0009) [2023-12-27 02:59:59,768][105692] Updated weights for policy 0, policy_version 1593013 (0.0006) [2023-12-27 02:59:59,828][105692] Updated weights for policy 0, policy_version 1593023 (0.0008) [2023-12-27 02:59:59,970][105620] Updated weights for policy 1, policy_version 1596643 (0.0008) [2023-12-27 03:00:00,031][105620] Updated weights for policy 1, policy_version 1596653 (0.0007) [2023-12-27 03:00:00,090][105620] Updated weights for policy 1, policy_version 1596663 (0.0011) [2023-12-27 03:00:00,513][105692] Updated weights for policy 0, policy_version 1593033 (0.0010) [2023-12-27 03:00:00,576][105692] Updated weights for policy 0, policy_version 1593043 (0.0006) [2023-12-27 03:00:00,642][105692] Updated weights for policy 0, policy_version 1593053 (0.0005) [2023-12-27 03:00:00,697][105692] Updated weights for policy 0, policy_version 1593063 (0.0007) [2023-12-27 03:00:00,741][105620] Updated weights for policy 1, policy_version 1596673 (0.0011) [2023-12-27 03:00:00,790][105620] Updated weights for policy 1, policy_version 1596683 (0.0011) [2023-12-27 03:00:00,851][105620] Updated weights for policy 1, policy_version 1596693 (0.0011) [2023-12-27 03:00:00,910][105620] Updated weights for policy 1, policy_version 1596703 (0.0011) [2023-12-27 03:00:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 816693248. Throughput: 0: 9947.2, 1: 9432.1. Samples: 816659584. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 03:00:01,063][104569] Avg episode reward: [(0, '8163.877'), (1, '8993.131')] [2023-12-27 03:00:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001593064_407879680.pth... [2023-12-27 03:00:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001596704_408813568.pth... [2023-12-27 03:00:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001595592_408526848.pth [2023-12-27 03:00:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001591912_407584768.pth [2023-12-27 03:00:01,487][105692] Updated weights for policy 0, policy_version 1593073 (0.0008) [2023-12-27 03:00:01,549][105692] Updated weights for policy 0, policy_version 1593083 (0.0009) [2023-12-27 03:00:01,614][105692] Updated weights for policy 0, policy_version 1593093 (0.0007) [2023-12-27 03:00:01,647][105620] Updated weights for policy 1, policy_version 1596713 (0.0012) [2023-12-27 03:00:01,702][105620] Updated weights for policy 1, policy_version 1596723 (0.0010) [2023-12-27 03:00:01,772][105620] Updated weights for policy 1, policy_version 1596733 (0.0011) [2023-12-27 03:00:02,285][105692] Updated weights for policy 0, policy_version 1593103 (0.0008) [2023-12-27 03:00:02,336][105692] Updated weights for policy 0, policy_version 1593113 (0.0009) [2023-12-27 03:00:02,399][105620] Updated weights for policy 1, policy_version 1596743 (0.0008) [2023-12-27 03:00:02,401][105692] Updated weights for policy 0, policy_version 1593123 (0.0008) [2023-12-27 03:00:02,468][105620] Updated weights for policy 1, policy_version 1596753 (0.0008) [2023-12-27 03:00:02,532][105620] Updated weights for policy 1, policy_version 1596763 (0.0006) [2023-12-27 03:00:03,175][105692] Updated weights for policy 0, policy_version 1593133 (0.0008) [2023-12-27 03:00:03,214][105620] Updated weights for policy 1, policy_version 1596773 (0.0008) [2023-12-27 03:00:03,226][105692] Updated weights for policy 0, policy_version 1593143 (0.0006) [2023-12-27 03:00:03,276][105692] Updated weights for policy 0, policy_version 1593153 (0.0007) [2023-12-27 03:00:03,279][105620] Updated weights for policy 1, policy_version 1596783 (0.0006) [2023-12-27 03:00:03,344][105620] Updated weights for policy 1, policy_version 1596793 (0.0008) [2023-12-27 03:00:03,852][105692] Updated weights for policy 0, policy_version 1593163 (0.0007) [2023-12-27 03:00:03,905][105692] Updated weights for policy 0, policy_version 1593173 (0.0007) [2023-12-27 03:00:03,956][105692] Updated weights for policy 0, policy_version 1593183 (0.0008) [2023-12-27 03:00:04,161][105620] Updated weights for policy 1, policy_version 1596803 (0.0009) [2023-12-27 03:00:04,222][105620] Updated weights for policy 1, policy_version 1596813 (0.0008) [2023-12-27 03:00:04,287][105620] Updated weights for policy 1, policy_version 1596823 (0.0009) [2023-12-27 03:00:04,621][105692] Updated weights for policy 0, policy_version 1593193 (0.0009) [2023-12-27 03:00:04,673][105692] Updated weights for policy 0, policy_version 1593203 (0.0008) [2023-12-27 03:00:04,730][105692] Updated weights for policy 0, policy_version 1593213 (0.0008) [2023-12-27 03:00:04,792][105692] Updated weights for policy 0, policy_version 1593223 (0.0009) [2023-12-27 03:00:05,072][105620] Updated weights for policy 1, policy_version 1596833 (0.0009) [2023-12-27 03:00:05,133][105620] Updated weights for policy 1, policy_version 1596843 (0.0009) [2023-12-27 03:00:05,188][105620] Updated weights for policy 1, policy_version 1596853 (0.0009) [2023-12-27 03:00:05,238][105620] Updated weights for policy 1, policy_version 1596863 (0.0008) [2023-12-27 03:00:05,510][105692] Updated weights for policy 0, policy_version 1593233 (0.0007) [2023-12-27 03:00:05,559][105692] Updated weights for policy 0, policy_version 1593243 (0.0006) [2023-12-27 03:00:05,611][105692] Updated weights for policy 0, policy_version 1593253 (0.0006) [2023-12-27 03:00:06,003][105620] Updated weights for policy 1, policy_version 1596873 (0.0010) [2023-12-27 03:00:06,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 816783360. Throughput: 0: 10022.6, 1: 9448.7. Samples: 816776416. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 03:00:06,062][104569] Avg episode reward: [(0, '8345.749'), (1, '8549.856')] [2023-12-27 03:00:06,064][105620] Updated weights for policy 1, policy_version 1596883 (0.0010) [2023-12-27 03:00:06,128][105620] Updated weights for policy 1, policy_version 1596893 (0.0010) [2023-12-27 03:00:06,191][105692] Updated weights for policy 0, policy_version 1593263 (0.0008) [2023-12-27 03:00:06,258][105692] Updated weights for policy 0, policy_version 1593273 (0.0009) [2023-12-27 03:00:06,329][105692] Updated weights for policy 0, policy_version 1593283 (0.0009) [2023-12-27 03:00:06,917][105620] Updated weights for policy 1, policy_version 1596903 (0.0009) [2023-12-27 03:00:06,975][105620] Updated weights for policy 1, policy_version 1596913 (0.0009) [2023-12-27 03:00:07,040][105620] Updated weights for policy 1, policy_version 1596923 (0.0009) [2023-12-27 03:00:07,063][105692] Updated weights for policy 0, policy_version 1593293 (0.0007) [2023-12-27 03:00:07,116][105692] Updated weights for policy 0, policy_version 1593303 (0.0008) [2023-12-27 03:00:07,167][105692] Updated weights for policy 0, policy_version 1593313 (0.0008) [2023-12-27 03:00:07,829][105620] Updated weights for policy 1, policy_version 1596933 (0.0008) [2023-12-27 03:00:07,882][105620] Updated weights for policy 1, policy_version 1596943 (0.0008) [2023-12-27 03:00:07,893][105692] Updated weights for policy 0, policy_version 1593323 (0.0008) [2023-12-27 03:00:07,940][105620] Updated weights for policy 1, policy_version 1596953 (0.0007) [2023-12-27 03:00:07,942][105692] Updated weights for policy 0, policy_version 1593333 (0.0006) [2023-12-27 03:00:07,999][105692] Updated weights for policy 0, policy_version 1593343 (0.0006) [2023-12-27 03:00:08,668][105620] Updated weights for policy 1, policy_version 1596963 (0.0009) [2023-12-27 03:00:08,727][105620] Updated weights for policy 1, policy_version 1596973 (0.0009) [2023-12-27 03:00:08,789][105620] Updated weights for policy 1, policy_version 1596983 (0.0009) [2023-12-27 03:00:08,798][105692] Updated weights for policy 0, policy_version 1593353 (0.0009) [2023-12-27 03:00:08,865][105692] Updated weights for policy 0, policy_version 1593363 (0.0007) [2023-12-27 03:00:08,929][105692] Updated weights for policy 0, policy_version 1593373 (0.0008) [2023-12-27 03:00:08,990][105692] Updated weights for policy 0, policy_version 1593383 (0.0005) [2023-12-27 03:00:09,587][105620] Updated weights for policy 1, policy_version 1596993 (0.0009) [2023-12-27 03:00:09,639][105692] Updated weights for policy 0, policy_version 1593393 (0.0006) [2023-12-27 03:00:09,649][105620] Updated weights for policy 1, policy_version 1597003 (0.0007) [2023-12-27 03:00:09,704][105692] Updated weights for policy 0, policy_version 1593403 (0.0006) [2023-12-27 03:00:09,710][105620] Updated weights for policy 1, policy_version 1597013 (0.0007) [2023-12-27 03:00:09,762][105692] Updated weights for policy 0, policy_version 1593413 (0.0007) [2023-12-27 03:00:09,771][105620] Updated weights for policy 1, policy_version 1597023 (0.0006) [2023-12-27 03:00:10,511][105692] Updated weights for policy 0, policy_version 1593423 (0.0005) [2023-12-27 03:00:10,512][105620] Updated weights for policy 1, policy_version 1597033 (0.0008) [2023-12-27 03:00:10,562][105692] Updated weights for policy 0, policy_version 1593433 (0.0006) [2023-12-27 03:00:10,568][105620] Updated weights for policy 1, policy_version 1597043 (0.0007) [2023-12-27 03:00:10,612][105692] Updated weights for policy 0, policy_version 1593443 (0.0006) [2023-12-27 03:00:10,624][105620] Updated weights for policy 1, policy_version 1597053 (0.0008) [2023-12-27 03:00:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 816881664. Throughput: 0: 10002.0, 1: 9396.9. Samples: 816889828. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 03:00:11,063][104569] Avg episode reward: [(0, '8621.208'), (1, '8638.206')] [2023-12-27 03:00:11,401][105620] Updated weights for policy 1, policy_version 1597063 (0.0007) [2023-12-27 03:00:11,420][105692] Updated weights for policy 0, policy_version 1593453 (0.0007) [2023-12-27 03:00:11,463][105620] Updated weights for policy 1, policy_version 1597073 (0.0007) [2023-12-27 03:00:11,482][105692] Updated weights for policy 0, policy_version 1593463 (0.0006) [2023-12-27 03:00:11,521][105620] Updated weights for policy 1, policy_version 1597083 (0.0009) [2023-12-27 03:00:11,536][105692] Updated weights for policy 0, policy_version 1593473 (0.0007) [2023-12-27 03:00:12,263][105620] Updated weights for policy 1, policy_version 1597093 (0.0009) [2023-12-27 03:00:12,327][105692] Updated weights for policy 0, policy_version 1593483 (0.0008) [2023-12-27 03:00:12,334][105620] Updated weights for policy 1, policy_version 1597103 (0.0008) [2023-12-27 03:00:12,398][105692] Updated weights for policy 0, policy_version 1593493 (0.0009) [2023-12-27 03:00:12,400][105620] Updated weights for policy 1, policy_version 1597113 (0.0008) [2023-12-27 03:00:12,461][105692] Updated weights for policy 0, policy_version 1593503 (0.0007) [2023-12-27 03:00:13,152][105620] Updated weights for policy 1, policy_version 1597123 (0.0008) [2023-12-27 03:00:13,163][105692] Updated weights for policy 0, policy_version 1593513 (0.0009) [2023-12-27 03:00:13,206][105620] Updated weights for policy 1, policy_version 1597133 (0.0007) [2023-12-27 03:00:13,213][105692] Updated weights for policy 0, policy_version 1593523 (0.0006) [2023-12-27 03:00:13,256][105620] Updated weights for policy 1, policy_version 1597143 (0.0007) [2023-12-27 03:00:13,262][105692] Updated weights for policy 0, policy_version 1593533 (0.0007) [2023-12-27 03:00:13,323][105692] Updated weights for policy 0, policy_version 1593543 (0.0006) [2023-12-27 03:00:14,032][105620] Updated weights for policy 1, policy_version 1597153 (0.0009) [2023-12-27 03:00:14,055][105692] Updated weights for policy 0, policy_version 1593553 (0.0008) [2023-12-27 03:00:14,078][105620] Updated weights for policy 1, policy_version 1597163 (0.0009) [2023-12-27 03:00:14,108][105692] Updated weights for policy 0, policy_version 1593563 (0.0006) [2023-12-27 03:00:14,130][105620] Updated weights for policy 1, policy_version 1597173 (0.0007) [2023-12-27 03:00:14,166][105692] Updated weights for policy 0, policy_version 1593573 (0.0006) [2023-12-27 03:00:14,182][105620] Updated weights for policy 1, policy_version 1597183 (0.0008) [2023-12-27 03:00:14,768][105692] Updated weights for policy 0, policy_version 1593583 (0.0006) [2023-12-27 03:00:14,834][105692] Updated weights for policy 0, policy_version 1593593 (0.0009) [2023-12-27 03:00:14,892][105692] Updated weights for policy 0, policy_version 1593603 (0.0009) [2023-12-27 03:00:15,043][105620] Updated weights for policy 1, policy_version 1597193 (0.0009) [2023-12-27 03:00:15,108][105620] Updated weights for policy 1, policy_version 1597203 (0.0010) [2023-12-27 03:00:15,172][105620] Updated weights for policy 1, policy_version 1597213 (0.0009) [2023-12-27 03:00:15,563][105692] Updated weights for policy 0, policy_version 1593613 (0.0007) [2023-12-27 03:00:15,622][105692] Updated weights for policy 0, policy_version 1593623 (0.0006) [2023-12-27 03:00:15,677][105692] Updated weights for policy 0, policy_version 1593633 (0.0009) [2023-12-27 03:00:15,987][105620] Updated weights for policy 1, policy_version 1597223 (0.0009) [2023-12-27 03:00:16,038][105620] Updated weights for policy 1, policy_version 1597233 (0.0009) [2023-12-27 03:00:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.3, 300 sec: 19494.2). Total num frames: 816971776. Throughput: 0: 9876.2, 1: 9439.8. Samples: 816944572. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 03:00:16,062][104569] Avg episode reward: [(0, '8895.855'), (1, '9080.662')] [2023-12-27 03:00:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001593640_408027136.pth... [2023-12-27 03:00:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001592488_407732224.pth [2023-12-27 03:00:16,095][105620] Updated weights for policy 1, policy_version 1597244 (0.0010) [2023-12-27 03:00:16,112][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001597248_408952832.pth... [2023-12-27 03:00:16,116][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001596128_408666112.pth [2023-12-27 03:00:16,297][105692] Updated weights for policy 0, policy_version 1593643 (0.0010) [2023-12-27 03:00:16,353][105692] Updated weights for policy 0, policy_version 1593653 (0.0011) [2023-12-27 03:00:16,403][105692] Updated weights for policy 0, policy_version 1593663 (0.0005) [2023-12-27 03:00:16,978][105620] Updated weights for policy 1, policy_version 1597254 (0.0009) [2023-12-27 03:00:16,993][105692] Updated weights for policy 0, policy_version 1593673 (0.0005) [2023-12-27 03:00:17,032][105620] Updated weights for policy 1, policy_version 1597264 (0.0009) [2023-12-27 03:00:17,051][105692] Updated weights for policy 0, policy_version 1593683 (0.0006) [2023-12-27 03:00:17,081][105620] Updated weights for policy 1, policy_version 1597274 (0.0006) [2023-12-27 03:00:17,109][105692] Updated weights for policy 0, policy_version 1593693 (0.0008) [2023-12-27 03:00:17,168][105692] Updated weights for policy 0, policy_version 1593703 (0.0008) [2023-12-27 03:00:17,779][105692] Updated weights for policy 0, policy_version 1593713 (0.0008) [2023-12-27 03:00:17,839][105692] Updated weights for policy 0, policy_version 1593723 (0.0005) [2023-12-27 03:00:17,901][105692] Updated weights for policy 0, policy_version 1593733 (0.0007) [2023-12-27 03:00:17,916][105620] Updated weights for policy 1, policy_version 1597284 (0.0008) [2023-12-27 03:00:17,967][105620] Updated weights for policy 1, policy_version 1597294 (0.0008) [2023-12-27 03:00:18,021][105620] Updated weights for policy 1, policy_version 1597304 (0.0009) [2023-12-27 03:00:18,616][105692] Updated weights for policy 0, policy_version 1593743 (0.0009) [2023-12-27 03:00:18,675][105692] Updated weights for policy 0, policy_version 1593753 (0.0009) [2023-12-27 03:00:18,724][105692] Updated weights for policy 0, policy_version 1593763 (0.0009) [2023-12-27 03:00:18,787][105620] Updated weights for policy 1, policy_version 1597314 (0.0009) [2023-12-27 03:00:18,835][105620] Updated weights for policy 1, policy_version 1597324 (0.0010) [2023-12-27 03:00:18,883][105620] Updated weights for policy 1, policy_version 1597334 (0.0009) [2023-12-27 03:00:18,933][105620] Updated weights for policy 1, policy_version 1597344 (0.0009) [2023-12-27 03:00:19,442][105692] Updated weights for policy 0, policy_version 1593773 (0.0007) [2023-12-27 03:00:19,510][105692] Updated weights for policy 0, policy_version 1593783 (0.0007) [2023-12-27 03:00:19,571][105692] Updated weights for policy 0, policy_version 1593793 (0.0009) [2023-12-27 03:00:19,786][105620] Updated weights for policy 1, policy_version 1597354 (0.0008) [2023-12-27 03:00:19,865][105620] Updated weights for policy 1, policy_version 1597364 (0.0008) [2023-12-27 03:00:19,932][105620] Updated weights for policy 1, policy_version 1597374 (0.0009) [2023-12-27 03:00:20,253][105692] Updated weights for policy 0, policy_version 1593803 (0.0008) [2023-12-27 03:00:20,306][105692] Updated weights for policy 0, policy_version 1593813 (0.0005) [2023-12-27 03:00:20,370][105692] Updated weights for policy 0, policy_version 1593823 (0.0006) [2023-12-27 03:00:20,741][105620] Updated weights for policy 1, policy_version 1597384 (0.0009) [2023-12-27 03:00:20,811][105620] Updated weights for policy 1, policy_version 1597394 (0.0010) [2023-12-27 03:00:20,869][105620] Updated weights for policy 1, policy_version 1597404 (0.0008) [2023-12-27 03:00:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 817070080. Throughput: 0: 9871.1, 1: 9373.1. Samples: 817059508. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 03:00:21,063][104569] Avg episode reward: [(0, '8260.814'), (1, '9083.441')] [2023-12-27 03:00:21,067][105692] Updated weights for policy 0, policy_version 1593833 (0.0007) [2023-12-27 03:00:21,132][105692] Updated weights for policy 0, policy_version 1593843 (0.0008) [2023-12-27 03:00:21,198][105692] Updated weights for policy 0, policy_version 1593853 (0.0006) [2023-12-27 03:00:21,265][105692] Updated weights for policy 0, policy_version 1593863 (0.0007) [2023-12-27 03:00:21,652][105620] Updated weights for policy 1, policy_version 1597414 (0.0009) [2023-12-27 03:00:21,710][105620] Updated weights for policy 1, policy_version 1597424 (0.0009) [2023-12-27 03:00:21,791][105620] Updated weights for policy 1, policy_version 1597434 (0.0009) [2023-12-27 03:00:22,005][105692] Updated weights for policy 0, policy_version 1593873 (0.0009) [2023-12-27 03:00:22,058][105692] Updated weights for policy 0, policy_version 1593883 (0.0009) [2023-12-27 03:00:22,121][105692] Updated weights for policy 0, policy_version 1593893 (0.0009) [2023-12-27 03:00:22,605][105620] Updated weights for policy 1, policy_version 1597444 (0.0008) [2023-12-27 03:00:22,671][105620] Updated weights for policy 1, policy_version 1597454 (0.0009) [2023-12-27 03:00:22,734][105620] Updated weights for policy 1, policy_version 1597464 (0.0009) [2023-12-27 03:00:22,826][105692] Updated weights for policy 0, policy_version 1593903 (0.0009) [2023-12-27 03:00:22,882][105692] Updated weights for policy 0, policy_version 1593913 (0.0009) [2023-12-27 03:00:22,947][105692] Updated weights for policy 0, policy_version 1593923 (0.0009) [2023-12-27 03:00:23,487][105620] Updated weights for policy 1, policy_version 1597474 (0.0009) [2023-12-27 03:00:23,545][105620] Updated weights for policy 1, policy_version 1597484 (0.0009) [2023-12-27 03:00:23,606][105620] Updated weights for policy 1, policy_version 1597494 (0.0008) [2023-12-27 03:00:23,669][105620] Updated weights for policy 1, policy_version 1597504 (0.0008) [2023-12-27 03:00:23,714][105692] Updated weights for policy 0, policy_version 1593933 (0.0010) [2023-12-27 03:00:23,781][105692] Updated weights for policy 0, policy_version 1593943 (0.0010) [2023-12-27 03:00:23,840][105692] Updated weights for policy 0, policy_version 1593953 (0.0009) [2023-12-27 03:00:24,346][105620] Updated weights for policy 1, policy_version 1597514 (0.0008) [2023-12-27 03:00:24,405][105620] Updated weights for policy 1, policy_version 1597524 (0.0007) [2023-12-27 03:00:24,461][105620] Updated weights for policy 1, policy_version 1597534 (0.0005) [2023-12-27 03:00:24,657][105692] Updated weights for policy 0, policy_version 1593963 (0.0010) [2023-12-27 03:00:24,701][105692] Updated weights for policy 0, policy_version 1593973 (0.0010) [2023-12-27 03:00:24,750][105692] Updated weights for policy 0, policy_version 1593983 (0.0010) [2023-12-27 03:00:25,135][105620] Updated weights for policy 1, policy_version 1597544 (0.0008) [2023-12-27 03:00:25,187][105620] Updated weights for policy 1, policy_version 1597555 (0.0009) [2023-12-27 03:00:25,244][105620] Updated weights for policy 1, policy_version 1597566 (0.0010) [2023-12-27 03:00:25,330][105692] Updated weights for policy 0, policy_version 1593993 (0.0010) [2023-12-27 03:00:25,387][105692] Updated weights for policy 0, policy_version 1594003 (0.0005) [2023-12-27 03:00:25,443][105692] Updated weights for policy 0, policy_version 1594013 (0.0006) [2023-12-27 03:00:25,500][105692] Updated weights for policy 0, policy_version 1594023 (0.0005) [2023-12-27 03:00:26,036][105692] Updated weights for policy 0, policy_version 1594033 (0.0006) [2023-12-27 03:00:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 817160192. Throughput: 0: 9875.3, 1: 9360.0. Samples: 817173036. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 03:00:26,062][104569] Avg episode reward: [(0, '8260.354'), (1, '8994.457')] [2023-12-27 03:00:26,099][105692] Updated weights for policy 0, policy_version 1594043 (0.0006) [2023-12-27 03:00:26,128][105620] Updated weights for policy 1, policy_version 1597576 (0.0009) [2023-12-27 03:00:26,156][105692] Updated weights for policy 0, policy_version 1594053 (0.0005) [2023-12-27 03:00:26,181][105620] Updated weights for policy 1, policy_version 1597586 (0.0008) [2023-12-27 03:00:26,243][105620] Updated weights for policy 1, policy_version 1597596 (0.0010) [2023-12-27 03:00:26,685][105692] Updated weights for policy 0, policy_version 1594063 (0.0009) [2023-12-27 03:00:26,735][105692] Updated weights for policy 0, policy_version 1594073 (0.0005) [2023-12-27 03:00:26,792][105692] Updated weights for policy 0, policy_version 1594083 (0.0006) [2023-12-27 03:00:27,082][105620] Updated weights for policy 1, policy_version 1597606 (0.0010) [2023-12-27 03:00:27,134][105620] Updated weights for policy 1, policy_version 1597616 (0.0009) [2023-12-27 03:00:27,187][105620] Updated weights for policy 1, policy_version 1597627 (0.0009) [2023-12-27 03:00:27,313][105692] Updated weights for policy 0, policy_version 1594093 (0.0009) [2023-12-27 03:00:27,368][105692] Updated weights for policy 0, policy_version 1594103 (0.0010) [2023-12-27 03:00:27,415][105692] Updated weights for policy 0, policy_version 1594113 (0.0010) [2023-12-27 03:00:27,982][105692] Updated weights for policy 0, policy_version 1594123 (0.0009) [2023-12-27 03:00:27,987][105620] Updated weights for policy 1, policy_version 1597637 (0.0007) [2023-12-27 03:00:28,033][105692] Updated weights for policy 0, policy_version 1594133 (0.0008) [2023-12-27 03:00:28,035][105620] Updated weights for policy 1, policy_version 1597647 (0.0008) [2023-12-27 03:00:28,077][105692] Updated weights for policy 0, policy_version 1594143 (0.0010) [2023-12-27 03:00:28,086][105620] Updated weights for policy 1, policy_version 1597657 (0.0006) [2023-12-27 03:00:28,733][105692] Updated weights for policy 0, policy_version 1594153 (0.0010) [2023-12-27 03:00:28,760][105620] Updated weights for policy 1, policy_version 1597667 (0.0007) [2023-12-27 03:00:28,793][105692] Updated weights for policy 0, policy_version 1594163 (0.0007) [2023-12-27 03:00:28,818][105620] Updated weights for policy 1, policy_version 1597677 (0.0007) [2023-12-27 03:00:28,853][105692] Updated weights for policy 0, policy_version 1594173 (0.0008) [2023-12-27 03:00:28,876][105620] Updated weights for policy 1, policy_version 1597687 (0.0007) [2023-12-27 03:00:28,913][105692] Updated weights for policy 0, policy_version 1594183 (0.0009) [2023-12-27 03:00:29,579][105620] Updated weights for policy 1, policy_version 1597697 (0.0006) [2023-12-27 03:00:29,631][105620] Updated weights for policy 1, policy_version 1597707 (0.0009) [2023-12-27 03:00:29,676][105620] Updated weights for policy 1, policy_version 1597717 (0.0006) [2023-12-27 03:00:29,679][105692] Updated weights for policy 0, policy_version 1594193 (0.0008) [2023-12-27 03:00:29,726][105620] Updated weights for policy 1, policy_version 1597727 (0.0008) [2023-12-27 03:00:29,739][105692] Updated weights for policy 0, policy_version 1594203 (0.0007) [2023-12-27 03:00:29,805][105692] Updated weights for policy 0, policy_version 1594213 (0.0009) [2023-12-27 03:00:30,512][105692] Updated weights for policy 0, policy_version 1594223 (0.0006) [2023-12-27 03:00:30,575][105692] Updated weights for policy 0, policy_version 1594233 (0.0006) [2023-12-27 03:00:30,581][105620] Updated weights for policy 1, policy_version 1597737 (0.0008) [2023-12-27 03:00:30,632][105692] Updated weights for policy 0, policy_version 1594243 (0.0006) [2023-12-27 03:00:30,639][105620] Updated weights for policy 1, policy_version 1597747 (0.0008) [2023-12-27 03:00:30,699][105620] Updated weights for policy 1, policy_version 1597757 (0.0009) [2023-12-27 03:00:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 817266688. Throughput: 0: 10021.3, 1: 9380.2. Samples: 817236980. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 03:00:31,062][104569] Avg episode reward: [(0, '8437.555'), (1, '8995.917')] [2023-12-27 03:00:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001594248_408182784.pth... [2023-12-27 03:00:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001597760_409083904.pth... [2023-12-27 03:00:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001593064_407879680.pth [2023-12-27 03:00:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001596704_408813568.pth [2023-12-27 03:00:31,224][105692] Updated weights for policy 0, policy_version 1594253 (0.0006) [2023-12-27 03:00:31,290][105692] Updated weights for policy 0, policy_version 1594263 (0.0006) [2023-12-27 03:00:31,356][105692] Updated weights for policy 0, policy_version 1594273 (0.0008) [2023-12-27 03:00:31,538][105620] Updated weights for policy 1, policy_version 1597767 (0.0008) [2023-12-27 03:00:31,592][105620] Updated weights for policy 1, policy_version 1597777 (0.0008) [2023-12-27 03:00:31,660][105620] Updated weights for policy 1, policy_version 1597787 (0.0008) [2023-12-27 03:00:31,999][105692] Updated weights for policy 0, policy_version 1594283 (0.0008) [2023-12-27 03:00:32,050][105692] Updated weights for policy 0, policy_version 1594293 (0.0006) [2023-12-27 03:00:32,105][105692] Updated weights for policy 0, policy_version 1594303 (0.0006) [2023-12-27 03:00:32,441][105620] Updated weights for policy 1, policy_version 1597797 (0.0009) [2023-12-27 03:00:32,493][105620] Updated weights for policy 1, policy_version 1597807 (0.0009) [2023-12-27 03:00:32,549][105620] Updated weights for policy 1, policy_version 1597818 (0.0009) [2023-12-27 03:00:32,708][105692] Updated weights for policy 0, policy_version 1594313 (0.0008) [2023-12-27 03:00:32,766][105692] Updated weights for policy 0, policy_version 1594323 (0.0006) [2023-12-27 03:00:32,824][105692] Updated weights for policy 0, policy_version 1594333 (0.0010) [2023-12-27 03:00:32,880][105692] Updated weights for policy 0, policy_version 1594343 (0.0011) [2023-12-27 03:00:33,333][105620] Updated weights for policy 1, policy_version 1597828 (0.0010) [2023-12-27 03:00:33,394][105620] Updated weights for policy 1, policy_version 1597838 (0.0009) [2023-12-27 03:00:33,449][105620] Updated weights for policy 1, policy_version 1597848 (0.0007) [2023-12-27 03:00:33,464][105692] Updated weights for policy 0, policy_version 1594353 (0.0008) [2023-12-27 03:00:33,514][105692] Updated weights for policy 0, policy_version 1594363 (0.0007) [2023-12-27 03:00:33,562][105692] Updated weights for policy 0, policy_version 1594373 (0.0009) [2023-12-27 03:00:34,192][105692] Updated weights for policy 0, policy_version 1594383 (0.0010) [2023-12-27 03:00:34,256][105692] Updated weights for policy 0, policy_version 1594393 (0.0011) [2023-12-27 03:00:34,262][105620] Updated weights for policy 1, policy_version 1597858 (0.0008) [2023-12-27 03:00:34,313][105692] Updated weights for policy 0, policy_version 1594403 (0.0011) [2023-12-27 03:00:34,328][105620] Updated weights for policy 1, policy_version 1597868 (0.0006) [2023-12-27 03:00:34,393][105620] Updated weights for policy 1, policy_version 1597878 (0.0007) [2023-12-27 03:00:34,461][105620] Updated weights for policy 1, policy_version 1597888 (0.0010) [2023-12-27 03:00:34,931][105692] Updated weights for policy 0, policy_version 1594413 (0.0010) [2023-12-27 03:00:34,993][105692] Updated weights for policy 0, policy_version 1594423 (0.0009) [2023-12-27 03:00:35,058][105692] Updated weights for policy 0, policy_version 1594433 (0.0005) [2023-12-27 03:00:35,234][105620] Updated weights for policy 1, policy_version 1597898 (0.0008) [2023-12-27 03:00:35,281][105620] Updated weights for policy 1, policy_version 1597908 (0.0008) [2023-12-27 03:00:35,339][105620] Updated weights for policy 1, policy_version 1597918 (0.0007) [2023-12-27 03:00:35,650][105692] Updated weights for policy 0, policy_version 1594443 (0.0006) [2023-12-27 03:00:35,698][105692] Updated weights for policy 0, policy_version 1594453 (0.0009) [2023-12-27 03:00:35,753][105692] Updated weights for policy 0, policy_version 1594463 (0.0009) [2023-12-27 03:00:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 817364992. Throughput: 0: 10115.6, 1: 9198.4. Samples: 817354068. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 03:00:36,062][104569] Avg episode reward: [(0, '8082.091'), (1, '8905.633')] [2023-12-27 03:00:36,205][105620] Updated weights for policy 1, policy_version 1597928 (0.0008) [2023-12-27 03:00:36,275][105620] Updated weights for policy 1, policy_version 1597938 (0.0008) [2023-12-27 03:00:36,346][105620] Updated weights for policy 1, policy_version 1597948 (0.0008) [2023-12-27 03:00:36,415][105692] Updated weights for policy 0, policy_version 1594473 (0.0005) [2023-12-27 03:00:36,488][105692] Updated weights for policy 0, policy_version 1594483 (0.0008) [2023-12-27 03:00:36,559][105692] Updated weights for policy 0, policy_version 1594493 (0.0008) [2023-12-27 03:00:36,618][105692] Updated weights for policy 0, policy_version 1594503 (0.0009) [2023-12-27 03:00:37,096][105620] Updated weights for policy 1, policy_version 1597958 (0.0008) [2023-12-27 03:00:37,154][105620] Updated weights for policy 1, policy_version 1597968 (0.0010) [2023-12-27 03:00:37,215][105620] Updated weights for policy 1, policy_version 1597978 (0.0007) [2023-12-27 03:00:37,233][105692] Updated weights for policy 0, policy_version 1594513 (0.0008) [2023-12-27 03:00:37,299][105692] Updated weights for policy 0, policy_version 1594523 (0.0009) [2023-12-27 03:00:37,367][105692] Updated weights for policy 0, policy_version 1594533 (0.0008) [2023-12-27 03:00:38,010][105620] Updated weights for policy 1, policy_version 1597988 (0.0007) [2023-12-27 03:00:38,016][105692] Updated weights for policy 0, policy_version 1594543 (0.0006) [2023-12-27 03:00:38,066][105620] Updated weights for policy 1, policy_version 1597998 (0.0008) [2023-12-27 03:00:38,074][105692] Updated weights for policy 0, policy_version 1594553 (0.0006) [2023-12-27 03:00:38,127][105620] Updated weights for policy 1, policy_version 1598008 (0.0006) [2023-12-27 03:00:38,133][105692] Updated weights for policy 0, policy_version 1594563 (0.0010) [2023-12-27 03:00:38,783][105692] Updated weights for policy 0, policy_version 1594573 (0.0008) [2023-12-27 03:00:38,840][105692] Updated weights for policy 0, policy_version 1594583 (0.0008) [2023-12-27 03:00:38,898][105620] Updated weights for policy 1, policy_version 1598018 (0.0007) [2023-12-27 03:00:38,912][105692] Updated weights for policy 0, policy_version 1594593 (0.0007) [2023-12-27 03:00:38,954][105620] Updated weights for policy 1, policy_version 1598028 (0.0006) [2023-12-27 03:00:39,018][105620] Updated weights for policy 1, policy_version 1598038 (0.0009) [2023-12-27 03:00:39,083][105620] Updated weights for policy 1, policy_version 1598048 (0.0009) [2023-12-27 03:00:39,658][105692] Updated weights for policy 0, policy_version 1594603 (0.0008) [2023-12-27 03:00:39,715][105692] Updated weights for policy 0, policy_version 1594613 (0.0011) [2023-12-27 03:00:39,771][105692] Updated weights for policy 0, policy_version 1594623 (0.0010) [2023-12-27 03:00:39,889][105620] Updated weights for policy 1, policy_version 1598058 (0.0008) [2023-12-27 03:00:39,951][105620] Updated weights for policy 1, policy_version 1598068 (0.0009) [2023-12-27 03:00:40,012][105620] Updated weights for policy 1, policy_version 1598078 (0.0008) [2023-12-27 03:00:40,440][105692] Updated weights for policy 0, policy_version 1594633 (0.0010) [2023-12-27 03:00:40,501][105692] Updated weights for policy 0, policy_version 1594643 (0.0007) [2023-12-27 03:00:40,568][105692] Updated weights for policy 0, policy_version 1594653 (0.0007) [2023-12-27 03:00:40,617][105692] Updated weights for policy 0, policy_version 1594663 (0.0006) [2023-12-27 03:00:40,876][105620] Updated weights for policy 1, policy_version 1598088 (0.0009) [2023-12-27 03:00:40,929][105620] Updated weights for policy 1, policy_version 1598098 (0.0008) [2023-12-27 03:00:40,980][105620] Updated weights for policy 1, policy_version 1598108 (0.0008) [2023-12-27 03:00:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 817463296. Throughput: 0: 10265.1, 1: 9059.2. Samples: 817468192. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 03:00:41,062][104569] Avg episode reward: [(0, '8170.577'), (1, '8902.699')] [2023-12-27 03:00:41,247][105692] Updated weights for policy 0, policy_version 1594673 (0.0008) [2023-12-27 03:00:41,309][105692] Updated weights for policy 0, policy_version 1594683 (0.0006) [2023-12-27 03:00:41,380][105692] Updated weights for policy 0, policy_version 1594693 (0.0009) [2023-12-27 03:00:41,876][105620] Updated weights for policy 1, policy_version 1598118 (0.0007) [2023-12-27 03:00:41,936][105620] Updated weights for policy 1, policy_version 1598128 (0.0009) [2023-12-27 03:00:42,001][105620] Updated weights for policy 1, policy_version 1598138 (0.0008) [2023-12-27 03:00:42,008][105692] Updated weights for policy 0, policy_version 1594703 (0.0008) [2023-12-27 03:00:42,059][105692] Updated weights for policy 0, policy_version 1594713 (0.0009) [2023-12-27 03:00:42,112][105692] Updated weights for policy 0, policy_version 1594723 (0.0009) [2023-12-27 03:00:42,783][105620] Updated weights for policy 1, policy_version 1598148 (0.0008) [2023-12-27 03:00:42,809][105692] Updated weights for policy 0, policy_version 1594733 (0.0007) [2023-12-27 03:00:42,840][105620] Updated weights for policy 1, policy_version 1598159 (0.0007) [2023-12-27 03:00:42,867][105692] Updated weights for policy 0, policy_version 1594743 (0.0006) [2023-12-27 03:00:42,901][105620] Updated weights for policy 1, policy_version 1598169 (0.0008) [2023-12-27 03:00:42,919][105692] Updated weights for policy 0, policy_version 1594753 (0.0010) [2023-12-27 03:00:43,646][105692] Updated weights for policy 0, policy_version 1594763 (0.0010) [2023-12-27 03:00:43,659][105620] Updated weights for policy 1, policy_version 1598179 (0.0006) [2023-12-27 03:00:43,700][105692] Updated weights for policy 0, policy_version 1594773 (0.0010) [2023-12-27 03:00:43,714][105620] Updated weights for policy 1, policy_version 1598189 (0.0005) [2023-12-27 03:00:43,748][105692] Updated weights for policy 0, policy_version 1594783 (0.0010) [2023-12-27 03:00:43,766][105620] Updated weights for policy 1, policy_version 1598199 (0.0005) [2023-12-27 03:00:44,478][105620] Updated weights for policy 1, policy_version 1598209 (0.0006) [2023-12-27 03:00:44,503][105692] Updated weights for policy 0, policy_version 1594793 (0.0010) [2023-12-27 03:00:44,542][105620] Updated weights for policy 1, policy_version 1598219 (0.0006) [2023-12-27 03:00:44,564][105692] Updated weights for policy 0, policy_version 1594803 (0.0008) [2023-12-27 03:00:44,598][105620] Updated weights for policy 1, policy_version 1598229 (0.0009) [2023-12-27 03:00:44,617][105692] Updated weights for policy 0, policy_version 1594813 (0.0006) [2023-12-27 03:00:44,656][105620] Updated weights for policy 1, policy_version 1598239 (0.0008) [2023-12-27 03:00:44,676][105692] Updated weights for policy 0, policy_version 1594823 (0.0007) [2023-12-27 03:00:45,356][105620] Updated weights for policy 1, policy_version 1598249 (0.0006) [2023-12-27 03:00:45,404][105620] Updated weights for policy 1, policy_version 1598259 (0.0006) [2023-12-27 03:00:45,412][105692] Updated weights for policy 0, policy_version 1594833 (0.0009) [2023-12-27 03:00:45,451][105620] Updated weights for policy 1, policy_version 1598269 (0.0005) [2023-12-27 03:00:45,467][105692] Updated weights for policy 0, policy_version 1594843 (0.0009) [2023-12-27 03:00:45,521][105692] Updated weights for policy 0, policy_version 1594853 (0.0008) [2023-12-27 03:00:46,038][105620] Updated weights for policy 1, policy_version 1598279 (0.0009) [2023-12-27 03:00:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 817553408. Throughput: 0: 10247.2, 1: 8995.4. Samples: 817525500. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 03:00:46,063][104569] Avg episode reward: [(0, '8345.010'), (1, '8811.527')] [2023-12-27 03:00:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001594856_408338432.pth... [2023-12-27 03:00:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001593640_408027136.pth [2023-12-27 03:00:46,090][105620] Updated weights for policy 1, policy_version 1598289 (0.0009) [2023-12-27 03:00:46,144][105620] Updated weights for policy 1, policy_version 1598299 (0.0007) [2023-12-27 03:00:46,166][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001598304_409223168.pth... [2023-12-27 03:00:46,175][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001597248_408952832.pth [2023-12-27 03:00:46,217][105692] Updated weights for policy 0, policy_version 1594863 (0.0007) [2023-12-27 03:00:46,261][105692] Updated weights for policy 0, policy_version 1594873 (0.0005) [2023-12-27 03:00:46,311][105692] Updated weights for policy 0, policy_version 1594883 (0.0006) [2023-12-27 03:00:46,859][105692] Updated weights for policy 0, policy_version 1594893 (0.0006) [2023-12-27 03:00:46,885][105620] Updated weights for policy 1, policy_version 1598309 (0.0010) [2023-12-27 03:00:46,901][105692] Updated weights for policy 0, policy_version 1594903 (0.0005) [2023-12-27 03:00:46,941][105620] Updated weights for policy 1, policy_version 1598319 (0.0010) [2023-12-27 03:00:46,947][105692] Updated weights for policy 0, policy_version 1594913 (0.0005) [2023-12-27 03:00:47,000][105620] Updated weights for policy 1, policy_version 1598329 (0.0010) [2023-12-27 03:00:47,505][105692] Updated weights for policy 0, policy_version 1594923 (0.0006) [2023-12-27 03:00:47,565][105692] Updated weights for policy 0, policy_version 1594933 (0.0006) [2023-12-27 03:00:47,616][105692] Updated weights for policy 0, policy_version 1594943 (0.0005) [2023-12-27 03:00:47,751][105620] Updated weights for policy 1, policy_version 1598339 (0.0010) [2023-12-27 03:00:47,808][105620] Updated weights for policy 1, policy_version 1598349 (0.0009) [2023-12-27 03:00:47,862][105620] Updated weights for policy 1, policy_version 1598359 (0.0005) [2023-12-27 03:00:48,196][105692] Updated weights for policy 0, policy_version 1594953 (0.0005) [2023-12-27 03:00:48,262][105692] Updated weights for policy 0, policy_version 1594963 (0.0007) [2023-12-27 03:00:48,321][105692] Updated weights for policy 0, policy_version 1594973 (0.0010) [2023-12-27 03:00:48,375][105692] Updated weights for policy 0, policy_version 1594983 (0.0010) [2023-12-27 03:00:48,560][105620] Updated weights for policy 1, policy_version 1598369 (0.0005) [2023-12-27 03:00:48,621][105620] Updated weights for policy 1, policy_version 1598379 (0.0010) [2023-12-27 03:00:48,686][105620] Updated weights for policy 1, policy_version 1598389 (0.0010) [2023-12-27 03:00:48,745][105620] Updated weights for policy 1, policy_version 1598399 (0.0011) [2023-12-27 03:00:49,032][105692] Updated weights for policy 0, policy_version 1594993 (0.0006) [2023-12-27 03:00:49,095][105692] Updated weights for policy 0, policy_version 1595003 (0.0005) [2023-12-27 03:00:49,154][105692] Updated weights for policy 0, policy_version 1595013 (0.0009) [2023-12-27 03:00:49,504][105620] Updated weights for policy 1, policy_version 1598409 (0.0010) [2023-12-27 03:00:49,572][105620] Updated weights for policy 1, policy_version 1598419 (0.0011) [2023-12-27 03:00:49,638][105620] Updated weights for policy 1, policy_version 1598429 (0.0011) [2023-12-27 03:00:49,813][105692] Updated weights for policy 0, policy_version 1595023 (0.0008) [2023-12-27 03:00:49,877][105692] Updated weights for policy 0, policy_version 1595033 (0.0010) [2023-12-27 03:00:49,945][105692] Updated weights for policy 0, policy_version 1595043 (0.0009) [2023-12-27 03:00:50,379][105620] Updated weights for policy 1, policy_version 1598439 (0.0009) [2023-12-27 03:00:50,435][105620] Updated weights for policy 1, policy_version 1598449 (0.0008) [2023-12-27 03:00:50,500][105620] Updated weights for policy 1, policy_version 1598459 (0.0009) [2023-12-27 03:00:50,655][105692] Updated weights for policy 0, policy_version 1595053 (0.0009) [2023-12-27 03:00:50,709][105692] Updated weights for policy 0, policy_version 1595063 (0.0009) [2023-12-27 03:00:50,765][105692] Updated weights for policy 0, policy_version 1595073 (0.0009) [2023-12-27 03:00:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 817659904. Throughput: 0: 10324.3, 1: 9053.6. Samples: 817648420. Policy #0 lag: (min: 27.0, avg: 38.5, max: 59.0) [2023-12-27 03:00:51,062][104569] Avg episode reward: [(0, '8436.971'), (1, '8813.718')] [2023-12-27 03:00:51,219][105620] Updated weights for policy 1, policy_version 1598469 (0.0007) [2023-12-27 03:00:51,287][105620] Updated weights for policy 1, policy_version 1598479 (0.0008) [2023-12-27 03:00:51,356][105620] Updated weights for policy 1, policy_version 1598489 (0.0008) [2023-12-27 03:00:51,601][105692] Updated weights for policy 0, policy_version 1595083 (0.0009) [2023-12-27 03:00:51,679][105692] Updated weights for policy 0, policy_version 1595093 (0.0010) [2023-12-27 03:00:51,746][105692] Updated weights for policy 0, policy_version 1595103 (0.0009) [2023-12-27 03:00:52,010][105620] Updated weights for policy 1, policy_version 1598499 (0.0007) [2023-12-27 03:00:52,060][105620] Updated weights for policy 1, policy_version 1598509 (0.0005) [2023-12-27 03:00:52,107][105620] Updated weights for policy 1, policy_version 1598519 (0.0005) [2023-12-27 03:00:52,585][105692] Updated weights for policy 0, policy_version 1595113 (0.0008) [2023-12-27 03:00:52,652][105692] Updated weights for policy 0, policy_version 1595123 (0.0011) [2023-12-27 03:00:52,719][105692] Updated weights for policy 0, policy_version 1595133 (0.0010) [2023-12-27 03:00:52,761][105620] Updated weights for policy 1, policy_version 1598529 (0.0007) [2023-12-27 03:00:52,775][105692] Updated weights for policy 0, policy_version 1595143 (0.0010) [2023-12-27 03:00:52,816][105620] Updated weights for policy 1, policy_version 1598539 (0.0007) [2023-12-27 03:00:52,874][105620] Updated weights for policy 1, policy_version 1598549 (0.0008) [2023-12-27 03:00:52,924][105620] Updated weights for policy 1, policy_version 1598559 (0.0008) [2023-12-27 03:00:53,512][105692] Updated weights for policy 0, policy_version 1595153 (0.0010) [2023-12-27 03:00:53,556][105692] Updated weights for policy 0, policy_version 1595163 (0.0010) [2023-12-27 03:00:53,601][105692] Updated weights for policy 0, policy_version 1595173 (0.0010) [2023-12-27 03:00:53,704][105620] Updated weights for policy 1, policy_version 1598569 (0.0009) [2023-12-27 03:00:53,765][105620] Updated weights for policy 1, policy_version 1598579 (0.0008) [2023-12-27 03:00:53,831][105620] Updated weights for policy 1, policy_version 1598589 (0.0009) [2023-12-27 03:00:54,406][105692] Updated weights for policy 0, policy_version 1595183 (0.0009) [2023-12-27 03:00:54,468][105692] Updated weights for policy 0, policy_version 1595193 (0.0008) [2023-12-27 03:00:54,539][105692] Updated weights for policy 0, policy_version 1595203 (0.0008) [2023-12-27 03:00:54,586][105620] Updated weights for policy 1, policy_version 1598599 (0.0009) [2023-12-27 03:00:54,636][105620] Updated weights for policy 1, policy_version 1598609 (0.0008) [2023-12-27 03:00:54,692][105620] Updated weights for policy 1, policy_version 1598619 (0.0009) [2023-12-27 03:00:55,171][105692] Updated weights for policy 0, policy_version 1595213 (0.0008) [2023-12-27 03:00:55,227][105692] Updated weights for policy 0, policy_version 1595223 (0.0009) [2023-12-27 03:00:55,275][105692] Updated weights for policy 0, policy_version 1595233 (0.0008) [2023-12-27 03:00:55,521][105620] Updated weights for policy 1, policy_version 1598629 (0.0009) [2023-12-27 03:00:55,591][105620] Updated weights for policy 1, policy_version 1598639 (0.0009) [2023-12-27 03:00:55,659][105620] Updated weights for policy 1, policy_version 1598649 (0.0010) [2023-12-27 03:00:56,034][105692] Updated weights for policy 0, policy_version 1595243 (0.0009) [2023-12-27 03:00:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19251.3, 300 sec: 19494.2). Total num frames: 817750016. Throughput: 0: 10252.6, 1: 9109.7. Samples: 817761128. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:00:56,062][104569] Avg episode reward: [(0, '8805.848'), (1, '9084.397')] [2023-12-27 03:00:56,096][105692] Updated weights for policy 0, policy_version 1595253 (0.0010) [2023-12-27 03:00:56,151][105692] Updated weights for policy 0, policy_version 1595263 (0.0010) [2023-12-27 03:00:56,333][105620] Updated weights for policy 1, policy_version 1598659 (0.0010) [2023-12-27 03:00:56,399][105620] Updated weights for policy 1, policy_version 1598669 (0.0010) [2023-12-27 03:00:56,458][105620] Updated weights for policy 1, policy_version 1598681 (0.0010) [2023-12-27 03:00:56,746][105692] Updated weights for policy 0, policy_version 1595273 (0.0010) [2023-12-27 03:00:56,791][105692] Updated weights for policy 0, policy_version 1595283 (0.0005) [2023-12-27 03:00:56,834][105692] Updated weights for policy 0, policy_version 1595293 (0.0005) [2023-12-27 03:00:56,886][105692] Updated weights for policy 0, policy_version 1595303 (0.0005) [2023-12-27 03:00:57,091][105620] Updated weights for policy 1, policy_version 1598691 (0.0009) [2023-12-27 03:00:57,139][105620] Updated weights for policy 1, policy_version 1598701 (0.0005) [2023-12-27 03:00:57,191][105620] Updated weights for policy 1, policy_version 1598711 (0.0005) [2023-12-27 03:00:57,473][105692] Updated weights for policy 0, policy_version 1595313 (0.0010) [2023-12-27 03:00:57,536][105692] Updated weights for policy 0, policy_version 1595323 (0.0010) [2023-12-27 03:00:57,597][105692] Updated weights for policy 0, policy_version 1595333 (0.0010) [2023-12-27 03:00:57,804][105620] Updated weights for policy 1, policy_version 1598721 (0.0006) [2023-12-27 03:00:57,848][105620] Updated weights for policy 1, policy_version 1598731 (0.0008) [2023-12-27 03:00:57,895][105620] Updated weights for policy 1, policy_version 1598741 (0.0008) [2023-12-27 03:00:57,943][105620] Updated weights for policy 1, policy_version 1598751 (0.0007) [2023-12-27 03:00:58,284][105692] Updated weights for policy 0, policy_version 1595343 (0.0009) [2023-12-27 03:00:58,344][105692] Updated weights for policy 0, policy_version 1595353 (0.0010) [2023-12-27 03:00:58,406][105692] Updated weights for policy 0, policy_version 1595363 (0.0010) [2023-12-27 03:00:58,723][105620] Updated weights for policy 1, policy_version 1598761 (0.0010) [2023-12-27 03:00:58,786][105620] Updated weights for policy 1, policy_version 1598771 (0.0010) [2023-12-27 03:00:58,853][105620] Updated weights for policy 1, policy_version 1598781 (0.0009) [2023-12-27 03:00:59,268][105692] Updated weights for policy 0, policy_version 1595373 (0.0009) [2023-12-27 03:00:59,333][105692] Updated weights for policy 0, policy_version 1595383 (0.0009) [2023-12-27 03:00:59,405][105692] Updated weights for policy 0, policy_version 1595393 (0.0008) [2023-12-27 03:00:59,595][105620] Updated weights for policy 1, policy_version 1598791 (0.0009) [2023-12-27 03:00:59,655][105620] Updated weights for policy 1, policy_version 1598801 (0.0009) [2023-12-27 03:00:59,713][105620] Updated weights for policy 1, policy_version 1598811 (0.0009) [2023-12-27 03:01:00,132][105692] Updated weights for policy 0, policy_version 1595403 (0.0008) [2023-12-27 03:01:00,190][105692] Updated weights for policy 0, policy_version 1595413 (0.0005) [2023-12-27 03:01:00,242][105692] Updated weights for policy 0, policy_version 1595423 (0.0005) [2023-12-27 03:01:00,478][105620] Updated weights for policy 1, policy_version 1598821 (0.0008) [2023-12-27 03:01:00,539][105620] Updated weights for policy 1, policy_version 1598831 (0.0009) [2023-12-27 03:01:00,604][105620] Updated weights for policy 1, policy_version 1598841 (0.0009) [2023-12-27 03:01:00,816][105692] Updated weights for policy 0, policy_version 1595433 (0.0005) [2023-12-27 03:01:00,871][105692] Updated weights for policy 0, policy_version 1595443 (0.0006) [2023-12-27 03:01:00,929][105692] Updated weights for policy 0, policy_version 1595453 (0.0010) [2023-12-27 03:01:00,993][105692] Updated weights for policy 0, policy_version 1595463 (0.0009) [2023-12-27 03:01:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 817856512. Throughput: 0: 10354.5, 1: 9173.8. Samples: 817823344. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:01:01,062][104569] Avg episode reward: [(0, '8534.698'), (1, '9177.642')] [2023-12-27 03:01:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001595464_408494080.pth... [2023-12-27 03:01:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001598848_409362432.pth... [2023-12-27 03:01:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001594248_408182784.pth [2023-12-27 03:01:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001597760_409083904.pth [2023-12-27 03:01:01,430][105620] Updated weights for policy 1, policy_version 1598851 (0.0009) [2023-12-27 03:01:01,491][105620] Updated weights for policy 1, policy_version 1598861 (0.0009) [2023-12-27 03:01:01,556][105620] Updated weights for policy 1, policy_version 1598871 (0.0008) [2023-12-27 03:01:01,722][105692] Updated weights for policy 0, policy_version 1595473 (0.0009) [2023-12-27 03:01:01,790][105692] Updated weights for policy 0, policy_version 1595483 (0.0009) [2023-12-27 03:01:01,844][105692] Updated weights for policy 0, policy_version 1595493 (0.0009) [2023-12-27 03:01:02,318][105620] Updated weights for policy 1, policy_version 1598881 (0.0009) [2023-12-27 03:01:02,386][105620] Updated weights for policy 1, policy_version 1598891 (0.0010) [2023-12-27 03:01:02,450][105620] Updated weights for policy 1, policy_version 1598901 (0.0009) [2023-12-27 03:01:02,514][105620] Updated weights for policy 1, policy_version 1598911 (0.0009) [2023-12-27 03:01:02,678][105692] Updated weights for policy 0, policy_version 1595503 (0.0008) [2023-12-27 03:01:02,731][105692] Updated weights for policy 0, policy_version 1595513 (0.0009) [2023-12-27 03:01:02,792][105692] Updated weights for policy 0, policy_version 1595523 (0.0010) [2023-12-27 03:01:03,135][105620] Updated weights for policy 1, policy_version 1598921 (0.0006) [2023-12-27 03:01:03,201][105620] Updated weights for policy 1, policy_version 1598931 (0.0006) [2023-12-27 03:01:03,262][105620] Updated weights for policy 1, policy_version 1598941 (0.0005) [2023-12-27 03:01:03,650][105692] Updated weights for policy 0, policy_version 1595533 (0.0007) [2023-12-27 03:01:03,702][105692] Updated weights for policy 0, policy_version 1595543 (0.0006) [2023-12-27 03:01:03,751][105692] Updated weights for policy 0, policy_version 1595553 (0.0006) [2023-12-27 03:01:03,757][105620] Updated weights for policy 1, policy_version 1598951 (0.0005) [2023-12-27 03:01:03,820][105620] Updated weights for policy 1, policy_version 1598961 (0.0005) [2023-12-27 03:01:03,884][105620] Updated weights for policy 1, policy_version 1598971 (0.0008) [2023-12-27 03:01:04,518][105692] Updated weights for policy 0, policy_version 1595563 (0.0007) [2023-12-27 03:01:04,552][105620] Updated weights for policy 1, policy_version 1598981 (0.0007) [2023-12-27 03:01:04,577][105692] Updated weights for policy 0, policy_version 1595573 (0.0006) [2023-12-27 03:01:04,621][105620] Updated weights for policy 1, policy_version 1598991 (0.0008) [2023-12-27 03:01:04,628][105692] Updated weights for policy 0, policy_version 1595583 (0.0007) [2023-12-27 03:01:04,687][105620] Updated weights for policy 1, policy_version 1599001 (0.0008) [2023-12-27 03:01:05,360][105692] Updated weights for policy 0, policy_version 1595593 (0.0007) [2023-12-27 03:01:05,418][105692] Updated weights for policy 0, policy_version 1595603 (0.0005) [2023-12-27 03:01:05,427][105620] Updated weights for policy 1, policy_version 1599011 (0.0010) [2023-12-27 03:01:05,464][105692] Updated weights for policy 0, policy_version 1595613 (0.0005) [2023-12-27 03:01:05,475][105620] Updated weights for policy 1, policy_version 1599021 (0.0010) [2023-12-27 03:01:05,512][105692] Updated weights for policy 0, policy_version 1595623 (0.0005) [2023-12-27 03:01:05,527][105620] Updated weights for policy 1, policy_version 1599031 (0.0010) [2023-12-27 03:01:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 817946624. Throughput: 0: 10185.2, 1: 9329.3. Samples: 817937660. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:01:06,062][104569] Avg episode reward: [(0, '8438.847'), (1, '9087.435')] [2023-12-27 03:01:06,272][105692] Updated weights for policy 0, policy_version 1595633 (0.0007) [2023-12-27 03:01:06,281][105620] Updated weights for policy 1, policy_version 1599041 (0.0010) [2023-12-27 03:01:06,337][105692] Updated weights for policy 0, policy_version 1595643 (0.0007) [2023-12-27 03:01:06,345][105620] Updated weights for policy 1, policy_version 1599051 (0.0008) [2023-12-27 03:01:06,402][105692] Updated weights for policy 0, policy_version 1595653 (0.0007) [2023-12-27 03:01:06,407][105620] Updated weights for policy 1, policy_version 1599061 (0.0008) [2023-12-27 03:01:06,474][105620] Updated weights for policy 1, policy_version 1599071 (0.0008) [2023-12-27 03:01:07,170][105692] Updated weights for policy 0, policy_version 1595663 (0.0007) [2023-12-27 03:01:07,207][105620] Updated weights for policy 1, policy_version 1599081 (0.0010) [2023-12-27 03:01:07,226][105692] Updated weights for policy 0, policy_version 1595673 (0.0006) [2023-12-27 03:01:07,263][105620] Updated weights for policy 1, policy_version 1599091 (0.0011) [2023-12-27 03:01:07,275][105692] Updated weights for policy 0, policy_version 1595683 (0.0008) [2023-12-27 03:01:07,329][105620] Updated weights for policy 1, policy_version 1599101 (0.0011) [2023-12-27 03:01:08,054][105692] Updated weights for policy 0, policy_version 1595693 (0.0007) [2023-12-27 03:01:08,065][105620] Updated weights for policy 1, policy_version 1599111 (0.0008) [2023-12-27 03:01:08,111][105692] Updated weights for policy 0, policy_version 1595703 (0.0009) [2023-12-27 03:01:08,121][105620] Updated weights for policy 1, policy_version 1599121 (0.0007) [2023-12-27 03:01:08,170][105692] Updated weights for policy 0, policy_version 1595713 (0.0007) [2023-12-27 03:01:08,182][105620] Updated weights for policy 1, policy_version 1599131 (0.0006) [2023-12-27 03:01:08,913][105620] Updated weights for policy 1, policy_version 1599141 (0.0008) [2023-12-27 03:01:08,977][105620] Updated weights for policy 1, policy_version 1599151 (0.0009) [2023-12-27 03:01:09,002][105692] Updated weights for policy 0, policy_version 1595723 (0.0008) [2023-12-27 03:01:09,032][105620] Updated weights for policy 1, policy_version 1599161 (0.0008) [2023-12-27 03:01:09,055][105692] Updated weights for policy 0, policy_version 1595733 (0.0007) [2023-12-27 03:01:09,109][105692] Updated weights for policy 0, policy_version 1595743 (0.0008) [2023-12-27 03:01:09,789][105620] Updated weights for policy 1, policy_version 1599171 (0.0007) [2023-12-27 03:01:09,863][105620] Updated weights for policy 1, policy_version 1599181 (0.0009) [2023-12-27 03:01:09,930][105620] Updated weights for policy 1, policy_version 1599191 (0.0008) [2023-12-27 03:01:09,982][105692] Updated weights for policy 0, policy_version 1595753 (0.0009) [2023-12-27 03:01:10,045][105692] Updated weights for policy 0, policy_version 1595763 (0.0009) [2023-12-27 03:01:10,108][105692] Updated weights for policy 0, policy_version 1595773 (0.0009) [2023-12-27 03:01:10,175][105692] Updated weights for policy 0, policy_version 1595783 (0.0008) [2023-12-27 03:01:10,687][105620] Updated weights for policy 1, policy_version 1599201 (0.0009) [2023-12-27 03:01:10,744][105620] Updated weights for policy 1, policy_version 1599211 (0.0009) [2023-12-27 03:01:10,792][105620] Updated weights for policy 1, policy_version 1599221 (0.0008) [2023-12-27 03:01:10,840][105620] Updated weights for policy 1, policy_version 1599231 (0.0009) [2023-12-27 03:01:10,941][105692] Updated weights for policy 0, policy_version 1595793 (0.0009) [2023-12-27 03:01:10,995][105692] Updated weights for policy 0, policy_version 1595803 (0.0009) [2023-12-27 03:01:11,060][105692] Updated weights for policy 0, policy_version 1595813 (0.0008) [2023-12-27 03:01:11,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 818036736. Throughput: 0: 10075.5, 1: 9359.7. Samples: 818047624. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:01:11,062][104569] Avg episode reward: [(0, '8442.450'), (1, '9175.138')] [2023-12-27 03:01:11,719][105620] Updated weights for policy 1, policy_version 1599241 (0.0006) [2023-12-27 03:01:11,788][105620] Updated weights for policy 1, policy_version 1599251 (0.0009) [2023-12-27 03:01:11,854][105620] Updated weights for policy 1, policy_version 1599261 (0.0009) [2023-12-27 03:01:11,904][105692] Updated weights for policy 0, policy_version 1595823 (0.0009) [2023-12-27 03:01:11,972][105692] Updated weights for policy 0, policy_version 1595833 (0.0007) [2023-12-27 03:01:12,041][105692] Updated weights for policy 0, policy_version 1595843 (0.0006) [2023-12-27 03:01:12,637][105620] Updated weights for policy 1, policy_version 1599271 (0.0008) [2023-12-27 03:01:12,693][105620] Updated weights for policy 1, policy_version 1599281 (0.0008) [2023-12-27 03:01:12,703][105692] Updated weights for policy 0, policy_version 1595853 (0.0009) [2023-12-27 03:01:12,752][105620] Updated weights for policy 1, policy_version 1599291 (0.0007) [2023-12-27 03:01:12,757][105692] Updated weights for policy 0, policy_version 1595863 (0.0008) [2023-12-27 03:01:12,809][105692] Updated weights for policy 0, policy_version 1595873 (0.0005) [2023-12-27 03:01:13,488][105620] Updated weights for policy 1, policy_version 1599301 (0.0007) [2023-12-27 03:01:13,545][105692] Updated weights for policy 0, policy_version 1595883 (0.0007) [2023-12-27 03:01:13,550][105620] Updated weights for policy 1, policy_version 1599311 (0.0006) [2023-12-27 03:01:13,604][105692] Updated weights for policy 0, policy_version 1595893 (0.0007) [2023-12-27 03:01:13,609][105620] Updated weights for policy 1, policy_version 1599321 (0.0006) [2023-12-27 03:01:13,663][105692] Updated weights for policy 0, policy_version 1595903 (0.0006) [2023-12-27 03:01:14,175][105620] Updated weights for policy 1, policy_version 1599331 (0.0006) [2023-12-27 03:01:14,238][105620] Updated weights for policy 1, policy_version 1599341 (0.0008) [2023-12-27 03:01:14,305][105620] Updated weights for policy 1, policy_version 1599351 (0.0008) [2023-12-27 03:01:14,306][105692] Updated weights for policy 0, policy_version 1595913 (0.0006) [2023-12-27 03:01:14,354][105692] Updated weights for policy 0, policy_version 1595923 (0.0010) [2023-12-27 03:01:14,402][105692] Updated weights for policy 0, policy_version 1595933 (0.0010) [2023-12-27 03:01:14,460][105692] Updated weights for policy 0, policy_version 1595943 (0.0010) [2023-12-27 03:01:15,034][105620] Updated weights for policy 1, policy_version 1599361 (0.0006) [2023-12-27 03:01:15,092][105620] Updated weights for policy 1, policy_version 1599371 (0.0009) [2023-12-27 03:01:15,155][105620] Updated weights for policy 1, policy_version 1599381 (0.0009) [2023-12-27 03:01:15,162][105692] Updated weights for policy 0, policy_version 1595953 (0.0007) [2023-12-27 03:01:15,220][105620] Updated weights for policy 1, policy_version 1599391 (0.0008) [2023-12-27 03:01:15,226][105692] Updated weights for policy 0, policy_version 1595963 (0.0009) [2023-12-27 03:01:15,288][105692] Updated weights for policy 0, policy_version 1595973 (0.0011) [2023-12-27 03:01:15,990][105692] Updated weights for policy 0, policy_version 1595983 (0.0010) [2023-12-27 03:01:15,999][105620] Updated weights for policy 1, policy_version 1599401 (0.0008) [2023-12-27 03:01:16,035][105692] Updated weights for policy 0, policy_version 1595993 (0.0010) [2023-12-27 03:01:16,049][105620] Updated weights for policy 1, policy_version 1599411 (0.0005) [2023-12-27 03:01:16,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19251.1, 300 sec: 19466.4). Total num frames: 818126848. Throughput: 0: 9893.3, 1: 9389.1. Samples: 818104692. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:01:16,063][104569] Avg episode reward: [(0, '8443.809'), (1, '9085.720')] [2023-12-27 03:01:16,086][105692] Updated weights for policy 0, policy_version 1596003 (0.0010) [2023-12-27 03:01:16,104][105620] Updated weights for policy 1, policy_version 1599421 (0.0005) [2023-12-27 03:01:16,108][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001596008_408633344.pth... [2023-12-27 03:01:16,111][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001594856_408338432.pth [2023-12-27 03:01:16,119][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001599424_409509888.pth... [2023-12-27 03:01:16,124][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001598304_409223168.pth [2023-12-27 03:01:16,848][105692] Updated weights for policy 0, policy_version 1596013 (0.0010) [2023-12-27 03:01:16,882][105620] Updated weights for policy 1, policy_version 1599431 (0.0006) [2023-12-27 03:01:16,892][105692] Updated weights for policy 0, policy_version 1596023 (0.0010) [2023-12-27 03:01:16,939][105620] Updated weights for policy 1, policy_version 1599441 (0.0008) [2023-12-27 03:01:16,950][105692] Updated weights for policy 0, policy_version 1596033 (0.0010) [2023-12-27 03:01:16,991][105620] Updated weights for policy 1, policy_version 1599451 (0.0006) [2023-12-27 03:01:17,661][105692] Updated weights for policy 0, policy_version 1596043 (0.0010) [2023-12-27 03:01:17,722][105692] Updated weights for policy 0, policy_version 1596053 (0.0008) [2023-12-27 03:01:17,760][105620] Updated weights for policy 1, policy_version 1599461 (0.0007) [2023-12-27 03:01:17,782][105692] Updated weights for policy 0, policy_version 1596063 (0.0008) [2023-12-27 03:01:17,821][105620] Updated weights for policy 1, policy_version 1599471 (0.0006) [2023-12-27 03:01:17,871][105620] Updated weights for policy 1, policy_version 1599481 (0.0009) [2023-12-27 03:01:18,458][105692] Updated weights for policy 0, policy_version 1596073 (0.0008) [2023-12-27 03:01:18,510][105692] Updated weights for policy 0, policy_version 1596083 (0.0006) [2023-12-27 03:01:18,561][105692] Updated weights for policy 0, policy_version 1596093 (0.0005) [2023-12-27 03:01:18,615][105692] Updated weights for policy 0, policy_version 1596103 (0.0006) [2023-12-27 03:01:18,625][105620] Updated weights for policy 1, policy_version 1599491 (0.0009) [2023-12-27 03:01:18,688][105620] Updated weights for policy 1, policy_version 1599501 (0.0010) [2023-12-27 03:01:18,743][105620] Updated weights for policy 1, policy_version 1599511 (0.0010) [2023-12-27 03:01:19,248][105692] Updated weights for policy 0, policy_version 1596113 (0.0006) [2023-12-27 03:01:19,314][105692] Updated weights for policy 0, policy_version 1596123 (0.0007) [2023-12-27 03:01:19,405][105692] Updated weights for policy 0, policy_version 1596133 (0.0007) [2023-12-27 03:01:19,497][105620] Updated weights for policy 1, policy_version 1599521 (0.0010) [2023-12-27 03:01:19,553][105620] Updated weights for policy 1, policy_version 1599531 (0.0011) [2023-12-27 03:01:19,606][105620] Updated weights for policy 1, policy_version 1599541 (0.0011) [2023-12-27 03:01:19,654][105620] Updated weights for policy 1, policy_version 1599551 (0.0010) [2023-12-27 03:01:20,061][105692] Updated weights for policy 0, policy_version 1596143 (0.0005) [2023-12-27 03:01:20,118][105692] Updated weights for policy 0, policy_version 1596153 (0.0006) [2023-12-27 03:01:20,174][105692] Updated weights for policy 0, policy_version 1596163 (0.0005) [2023-12-27 03:01:20,494][105620] Updated weights for policy 1, policy_version 1599561 (0.0009) [2023-12-27 03:01:20,545][105620] Updated weights for policy 1, policy_version 1599571 (0.0009) [2023-12-27 03:01:20,609][105620] Updated weights for policy 1, policy_version 1599581 (0.0014) [2023-12-27 03:01:20,740][105692] Updated weights for policy 0, policy_version 1596173 (0.0007) [2023-12-27 03:01:20,805][105692] Updated weights for policy 0, policy_version 1596183 (0.0008) [2023-12-27 03:01:20,868][105692] Updated weights for policy 0, policy_version 1596193 (0.0009) [2023-12-27 03:01:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 818233344. Throughput: 0: 9829.8, 1: 9431.9. Samples: 818220844. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:01:21,063][104569] Avg episode reward: [(0, '8625.309'), (1, '9085.725')] [2023-12-27 03:01:21,406][105620] Updated weights for policy 1, policy_version 1599591 (0.0008) [2023-12-27 03:01:21,467][105620] Updated weights for policy 1, policy_version 1599601 (0.0006) [2023-12-27 03:01:21,527][105620] Updated weights for policy 1, policy_version 1599611 (0.0005) [2023-12-27 03:01:21,665][105692] Updated weights for policy 0, policy_version 1596203 (0.0007) [2023-12-27 03:01:21,730][105692] Updated weights for policy 0, policy_version 1596213 (0.0007) [2023-12-27 03:01:21,793][105692] Updated weights for policy 0, policy_version 1596223 (0.0007) [2023-12-27 03:01:22,264][105620] Updated weights for policy 1, policy_version 1599621 (0.0009) [2023-12-27 03:01:22,324][105620] Updated weights for policy 1, policy_version 1599631 (0.0010) [2023-12-27 03:01:22,395][105620] Updated weights for policy 1, policy_version 1599641 (0.0010) [2023-12-27 03:01:22,442][105692] Updated weights for policy 0, policy_version 1596233 (0.0006) [2023-12-27 03:01:22,509][105692] Updated weights for policy 0, policy_version 1596243 (0.0006) [2023-12-27 03:01:22,575][105692] Updated weights for policy 0, policy_version 1596253 (0.0008) [2023-12-27 03:01:22,644][105692] Updated weights for policy 0, policy_version 1596263 (0.0008) [2023-12-27 03:01:23,104][105620] Updated weights for policy 1, policy_version 1599651 (0.0010) [2023-12-27 03:01:23,160][105620] Updated weights for policy 1, policy_version 1599661 (0.0010) [2023-12-27 03:01:23,210][105620] Updated weights for policy 1, policy_version 1599671 (0.0010) [2023-12-27 03:01:23,213][105692] Updated weights for policy 0, policy_version 1596273 (0.0010) [2023-12-27 03:01:23,273][105692] Updated weights for policy 0, policy_version 1596283 (0.0011) [2023-12-27 03:01:23,339][105692] Updated weights for policy 0, policy_version 1596293 (0.0010) [2023-12-27 03:01:23,911][105620] Updated weights for policy 1, policy_version 1599681 (0.0007) [2023-12-27 03:01:23,935][105692] Updated weights for policy 0, policy_version 1596303 (0.0007) [2023-12-27 03:01:23,968][105620] Updated weights for policy 1, policy_version 1599691 (0.0006) [2023-12-27 03:01:23,991][105692] Updated weights for policy 0, policy_version 1596313 (0.0005) [2023-12-27 03:01:24,027][105620] Updated weights for policy 1, policy_version 1599701 (0.0006) [2023-12-27 03:01:24,048][105692] Updated weights for policy 0, policy_version 1596323 (0.0005) [2023-12-27 03:01:24,081][105620] Updated weights for policy 1, policy_version 1599711 (0.0009) [2023-12-27 03:01:24,683][105692] Updated weights for policy 0, policy_version 1596333 (0.0008) [2023-12-27 03:01:24,748][105692] Updated weights for policy 0, policy_version 1596343 (0.0009) [2023-12-27 03:01:24,808][105692] Updated weights for policy 0, policy_version 1596353 (0.0011) [2023-12-27 03:01:24,870][105620] Updated weights for policy 1, policy_version 1599721 (0.0010) [2023-12-27 03:01:24,932][105620] Updated weights for policy 1, policy_version 1599731 (0.0010) [2023-12-27 03:01:24,987][105620] Updated weights for policy 1, policy_version 1599741 (0.0010) [2023-12-27 03:01:25,450][105692] Updated weights for policy 0, policy_version 1596363 (0.0011) [2023-12-27 03:01:25,498][105692] Updated weights for policy 0, policy_version 1596373 (0.0010) [2023-12-27 03:01:25,542][105692] Updated weights for policy 0, policy_version 1596383 (0.0010) [2023-12-27 03:01:25,678][105620] Updated weights for policy 1, policy_version 1599751 (0.0007) [2023-12-27 03:01:25,741][105620] Updated weights for policy 1, policy_version 1599761 (0.0006) [2023-12-27 03:01:25,802][105620] Updated weights for policy 1, policy_version 1599771 (0.0008) [2023-12-27 03:01:26,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 818331648. Throughput: 0: 9854.4, 1: 9531.2. Samples: 818340544. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:01:26,063][104569] Avg episode reward: [(0, '8441.734'), (1, '9265.591')] [2023-12-27 03:01:26,274][105692] Updated weights for policy 0, policy_version 1596393 (0.0010) [2023-12-27 03:01:26,325][105692] Updated weights for policy 0, policy_version 1596403 (0.0010) [2023-12-27 03:01:26,370][105692] Updated weights for policy 0, policy_version 1596413 (0.0010) [2023-12-27 03:01:26,417][105692] Updated weights for policy 0, policy_version 1596423 (0.0010) [2023-12-27 03:01:26,462][105620] Updated weights for policy 1, policy_version 1599781 (0.0011) [2023-12-27 03:01:26,513][105620] Updated weights for policy 1, policy_version 1599791 (0.0010) [2023-12-27 03:01:26,584][105620] Updated weights for policy 1, policy_version 1599801 (0.0011) [2023-12-27 03:01:27,006][105692] Updated weights for policy 0, policy_version 1596433 (0.0006) [2023-12-27 03:01:27,061][105692] Updated weights for policy 0, policy_version 1596443 (0.0005) [2023-12-27 03:01:27,115][105692] Updated weights for policy 0, policy_version 1596453 (0.0005) [2023-12-27 03:01:27,323][105620] Updated weights for policy 1, policy_version 1599811 (0.0010) [2023-12-27 03:01:27,381][105620] Updated weights for policy 1, policy_version 1599821 (0.0010) [2023-12-27 03:01:27,445][105620] Updated weights for policy 1, policy_version 1599831 (0.0010) [2023-12-27 03:01:27,643][105692] Updated weights for policy 0, policy_version 1596463 (0.0009) [2023-12-27 03:01:27,690][105692] Updated weights for policy 0, policy_version 1596473 (0.0010) [2023-12-27 03:01:27,746][105692] Updated weights for policy 0, policy_version 1596483 (0.0010) [2023-12-27 03:01:28,186][105620] Updated weights for policy 1, policy_version 1599841 (0.0010) [2023-12-27 03:01:28,251][105620] Updated weights for policy 1, policy_version 1599851 (0.0010) [2023-12-27 03:01:28,299][105620] Updated weights for policy 1, policy_version 1599861 (0.0010) [2023-12-27 03:01:28,370][105620] Updated weights for policy 1, policy_version 1599871 (0.0010) [2023-12-27 03:01:28,374][105692] Updated weights for policy 0, policy_version 1596493 (0.0008) [2023-12-27 03:01:28,437][105692] Updated weights for policy 0, policy_version 1596503 (0.0006) [2023-12-27 03:01:28,500][105692] Updated weights for policy 0, policy_version 1596513 (0.0009) [2023-12-27 03:01:29,077][105620] Updated weights for policy 1, policy_version 1599881 (0.0010) [2023-12-27 03:01:29,141][105620] Updated weights for policy 1, policy_version 1599891 (0.0010) [2023-12-27 03:01:29,170][105692] Updated weights for policy 0, policy_version 1596523 (0.0008) [2023-12-27 03:01:29,203][105620] Updated weights for policy 1, policy_version 1599901 (0.0010) [2023-12-27 03:01:29,231][105692] Updated weights for policy 0, policy_version 1596533 (0.0008) [2023-12-27 03:01:29,286][105692] Updated weights for policy 0, policy_version 1596543 (0.0011) [2023-12-27 03:01:29,846][105620] Updated weights for policy 1, policy_version 1599911 (0.0008) [2023-12-27 03:01:29,902][105692] Updated weights for policy 0, policy_version 1596553 (0.0009) [2023-12-27 03:01:29,909][105620] Updated weights for policy 1, policy_version 1599921 (0.0008) [2023-12-27 03:01:29,964][105692] Updated weights for policy 0, policy_version 1596563 (0.0008) [2023-12-27 03:01:29,969][105620] Updated weights for policy 1, policy_version 1599931 (0.0011) [2023-12-27 03:01:30,018][105692] Updated weights for policy 0, policy_version 1596573 (0.0007) [2023-12-27 03:01:30,077][105692] Updated weights for policy 0, policy_version 1596583 (0.0008) [2023-12-27 03:01:30,632][105620] Updated weights for policy 1, policy_version 1599941 (0.0010) [2023-12-27 03:01:30,686][105620] Updated weights for policy 1, policy_version 1599951 (0.0009) [2023-12-27 03:01:30,739][105620] Updated weights for policy 1, policy_version 1599961 (0.0007) [2023-12-27 03:01:30,739][105692] Updated weights for policy 0, policy_version 1596593 (0.0009) [2023-12-27 03:01:30,789][105692] Updated weights for policy 0, policy_version 1596604 (0.0009) [2023-12-27 03:01:30,840][105692] Updated weights for policy 0, policy_version 1596616 (0.0007) [2023-12-27 03:01:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 818438144. Throughput: 0: 9919.3, 1: 9576.2. Samples: 818402796. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:01:31,062][104569] Avg episode reward: [(0, '8623.396'), (1, '9176.021')] [2023-12-27 03:01:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001596616_408788992.pth... [2023-12-27 03:01:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001599968_409649152.pth... [2023-12-27 03:01:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001595464_408494080.pth [2023-12-27 03:01:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001598848_409362432.pth [2023-12-27 03:01:31,447][105692] Updated weights for policy 0, policy_version 1596626 (0.0009) [2023-12-27 03:01:31,499][105692] Updated weights for policy 0, policy_version 1596636 (0.0010) [2023-12-27 03:01:31,530][105620] Updated weights for policy 1, policy_version 1599971 (0.0005) [2023-12-27 03:01:31,557][105692] Updated weights for policy 0, policy_version 1596646 (0.0008) [2023-12-27 03:01:31,586][105620] Updated weights for policy 1, policy_version 1599981 (0.0007) [2023-12-27 03:01:31,649][105620] Updated weights for policy 1, policy_version 1599991 (0.0008) [2023-12-27 03:01:32,236][105692] Updated weights for policy 0, policy_version 1596656 (0.0009) [2023-12-27 03:01:32,294][105692] Updated weights for policy 0, policy_version 1596666 (0.0009) [2023-12-27 03:01:32,351][105692] Updated weights for policy 0, policy_version 1596676 (0.0009) [2023-12-27 03:01:32,402][105620] Updated weights for policy 1, policy_version 1600001 (0.0007) [2023-12-27 03:01:32,468][105620] Updated weights for policy 1, policy_version 1600011 (0.0009) [2023-12-27 03:01:32,531][105620] Updated weights for policy 1, policy_version 1600021 (0.0010) [2023-12-27 03:01:32,591][105620] Updated weights for policy 1, policy_version 1600031 (0.0007) [2023-12-27 03:01:32,996][105692] Updated weights for policy 0, policy_version 1596686 (0.0008) [2023-12-27 03:01:33,046][105692] Updated weights for policy 0, policy_version 1596696 (0.0007) [2023-12-27 03:01:33,094][105692] Updated weights for policy 0, policy_version 1596706 (0.0005) [2023-12-27 03:01:33,338][105620] Updated weights for policy 1, policy_version 1600041 (0.0010) [2023-12-27 03:01:33,395][105620] Updated weights for policy 1, policy_version 1600051 (0.0010) [2023-12-27 03:01:33,454][105620] Updated weights for policy 1, policy_version 1600061 (0.0010) [2023-12-27 03:01:33,671][105692] Updated weights for policy 0, policy_version 1596716 (0.0005) [2023-12-27 03:01:33,736][105692] Updated weights for policy 0, policy_version 1596726 (0.0005) [2023-12-27 03:01:33,802][105692] Updated weights for policy 0, policy_version 1596736 (0.0005) [2023-12-27 03:01:34,071][105620] Updated weights for policy 1, policy_version 1600071 (0.0007) [2023-12-27 03:01:34,122][105620] Updated weights for policy 1, policy_version 1600081 (0.0005) [2023-12-27 03:01:34,187][105620] Updated weights for policy 1, policy_version 1600091 (0.0008) [2023-12-27 03:01:34,464][105692] Updated weights for policy 0, policy_version 1596746 (0.0007) [2023-12-27 03:01:34,535][105692] Updated weights for policy 0, policy_version 1596756 (0.0008) [2023-12-27 03:01:34,584][105692] Updated weights for policy 0, policy_version 1596766 (0.0007) [2023-12-27 03:01:34,638][105692] Updated weights for policy 0, policy_version 1596776 (0.0008) [2023-12-27 03:01:34,898][105620] Updated weights for policy 1, policy_version 1600101 (0.0008) [2023-12-27 03:01:34,954][105620] Updated weights for policy 1, policy_version 1600111 (0.0010) [2023-12-27 03:01:35,012][105620] Updated weights for policy 1, policy_version 1600121 (0.0008) [2023-12-27 03:01:35,338][105692] Updated weights for policy 0, policy_version 1596786 (0.0005) [2023-12-27 03:01:35,397][105692] Updated weights for policy 0, policy_version 1596796 (0.0005) [2023-12-27 03:01:35,452][105692] Updated weights for policy 0, policy_version 1596806 (0.0005) [2023-12-27 03:01:35,929][105620] Updated weights for policy 1, policy_version 1600131 (0.0008) [2023-12-27 03:01:35,952][105692] Updated weights for policy 0, policy_version 1596816 (0.0005) [2023-12-27 03:01:35,978][105620] Updated weights for policy 1, policy_version 1600141 (0.0008) [2023-12-27 03:01:36,009][105692] Updated weights for policy 0, policy_version 1596826 (0.0008) [2023-12-27 03:01:36,032][105620] Updated weights for policy 1, policy_version 1600151 (0.0005) [2023-12-27 03:01:36,062][105692] Updated weights for policy 0, policy_version 1596836 (0.0009) [2023-12-27 03:01:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 818528256. Throughput: 0: 9930.1, 1: 9576.0. Samples: 818526196. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:01:36,063][104569] Avg episode reward: [(0, '7985.694'), (1, '9083.517')] [2023-12-27 03:01:36,591][105620] Updated weights for policy 1, policy_version 1600161 (0.0006) [2023-12-27 03:01:36,648][105620] Updated weights for policy 1, policy_version 1600171 (0.0009) [2023-12-27 03:01:36,699][105620] Updated weights for policy 1, policy_version 1600181 (0.0008) [2023-12-27 03:01:36,750][105620] Updated weights for policy 1, policy_version 1600191 (0.0009) [2023-12-27 03:01:36,872][105692] Updated weights for policy 0, policy_version 1596846 (0.0009) [2023-12-27 03:01:36,924][105692] Updated weights for policy 0, policy_version 1596856 (0.0009) [2023-12-27 03:01:36,983][105692] Updated weights for policy 0, policy_version 1596866 (0.0009) [2023-12-27 03:01:37,522][105620] Updated weights for policy 1, policy_version 1600201 (0.0009) [2023-12-27 03:01:37,582][105620] Updated weights for policy 1, policy_version 1600211 (0.0009) [2023-12-27 03:01:37,633][105620] Updated weights for policy 1, policy_version 1600221 (0.0008) [2023-12-27 03:01:37,746][105692] Updated weights for policy 0, policy_version 1596876 (0.0010) [2023-12-27 03:01:37,805][105692] Updated weights for policy 0, policy_version 1596886 (0.0010) [2023-12-27 03:01:37,873][105692] Updated weights for policy 0, policy_version 1596896 (0.0009) [2023-12-27 03:01:38,250][105620] Updated weights for policy 1, policy_version 1600231 (0.0008) [2023-12-27 03:01:38,307][105620] Updated weights for policy 1, policy_version 1600241 (0.0005) [2023-12-27 03:01:38,373][105620] Updated weights for policy 1, policy_version 1600251 (0.0009) [2023-12-27 03:01:38,717][105692] Updated weights for policy 0, policy_version 1596906 (0.0009) [2023-12-27 03:01:38,770][105692] Updated weights for policy 0, policy_version 1596916 (0.0009) [2023-12-27 03:01:38,832][105692] Updated weights for policy 0, policy_version 1596927 (0.0010) [2023-12-27 03:01:39,021][105620] Updated weights for policy 1, policy_version 1600261 (0.0010) [2023-12-27 03:01:39,081][105620] Updated weights for policy 1, policy_version 1600272 (0.0011) [2023-12-27 03:01:39,137][105620] Updated weights for policy 1, policy_version 1600282 (0.0009) [2023-12-27 03:01:39,528][105692] Updated weights for policy 0, policy_version 1596937 (0.0009) [2023-12-27 03:01:39,596][105692] Updated weights for policy 0, policy_version 1596947 (0.0009) [2023-12-27 03:01:39,656][105692] Updated weights for policy 0, policy_version 1596957 (0.0008) [2023-12-27 03:01:39,713][105692] Updated weights for policy 0, policy_version 1596967 (0.0008) [2023-12-27 03:01:39,908][105620] Updated weights for policy 1, policy_version 1600292 (0.0008) [2023-12-27 03:01:39,975][105620] Updated weights for policy 1, policy_version 1600302 (0.0010) [2023-12-27 03:01:40,031][105620] Updated weights for policy 1, policy_version 1600312 (0.0011) [2023-12-27 03:01:40,546][105692] Updated weights for policy 0, policy_version 1596977 (0.0008) [2023-12-27 03:01:40,614][105692] Updated weights for policy 0, policy_version 1596987 (0.0009) [2023-12-27 03:01:40,681][105692] Updated weights for policy 0, policy_version 1596997 (0.0009) [2023-12-27 03:01:40,745][105620] Updated weights for policy 1, policy_version 1600322 (0.0010) [2023-12-27 03:01:40,809][105620] Updated weights for policy 1, policy_version 1600332 (0.0009) [2023-12-27 03:01:40,874][105620] Updated weights for policy 1, policy_version 1600342 (0.0011) [2023-12-27 03:01:40,928][105620] Updated weights for policy 1, policy_version 1600352 (0.0010) [2023-12-27 03:01:41,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 818634752. Throughput: 0: 9961.4, 1: 9603.0. Samples: 818641528. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:01:41,063][104569] Avg episode reward: [(0, '7438.933'), (1, '8900.789')] [2023-12-27 03:01:41,385][105692] Updated weights for policy 0, policy_version 1597007 (0.0008) [2023-12-27 03:01:41,453][105692] Updated weights for policy 0, policy_version 1597017 (0.0008) [2023-12-27 03:01:41,513][105692] Updated weights for policy 0, policy_version 1597027 (0.0008) [2023-12-27 03:01:41,668][105620] Updated weights for policy 1, policy_version 1600362 (0.0011) [2023-12-27 03:01:41,734][105620] Updated weights for policy 1, policy_version 1600372 (0.0011) [2023-12-27 03:01:41,794][105620] Updated weights for policy 1, policy_version 1600382 (0.0011) [2023-12-27 03:01:42,326][105692] Updated weights for policy 0, policy_version 1597037 (0.0008) [2023-12-27 03:01:42,390][105692] Updated weights for policy 0, policy_version 1597047 (0.0008) [2023-12-27 03:01:42,456][105692] Updated weights for policy 0, policy_version 1597057 (0.0008) [2023-12-27 03:01:42,547][105620] Updated weights for policy 1, policy_version 1600392 (0.0010) [2023-12-27 03:01:42,598][105620] Updated weights for policy 1, policy_version 1600402 (0.0010) [2023-12-27 03:01:42,654][105620] Updated weights for policy 1, policy_version 1600412 (0.0010) [2023-12-27 03:01:43,202][105692] Updated weights for policy 0, policy_version 1597067 (0.0008) [2023-12-27 03:01:43,254][105692] Updated weights for policy 0, policy_version 1597077 (0.0008) [2023-12-27 03:01:43,306][105692] Updated weights for policy 0, policy_version 1597087 (0.0008) [2023-12-27 03:01:43,404][105620] Updated weights for policy 1, policy_version 1600422 (0.0010) [2023-12-27 03:01:43,456][105620] Updated weights for policy 1, policy_version 1600432 (0.0010) [2023-12-27 03:01:43,504][105620] Updated weights for policy 1, policy_version 1600442 (0.0010) [2023-12-27 03:01:44,113][105692] Updated weights for policy 0, policy_version 1597097 (0.0008) [2023-12-27 03:01:44,169][105692] Updated weights for policy 0, policy_version 1597107 (0.0008) [2023-12-27 03:01:44,184][105620] Updated weights for policy 1, policy_version 1600452 (0.0009) [2023-12-27 03:01:44,227][105692] Updated weights for policy 0, policy_version 1597117 (0.0007) [2023-12-27 03:01:44,237][105620] Updated weights for policy 1, policy_version 1600462 (0.0009) [2023-12-27 03:01:44,278][105692] Updated weights for policy 0, policy_version 1597127 (0.0005) [2023-12-27 03:01:44,303][105620] Updated weights for policy 1, policy_version 1600472 (0.0008) [2023-12-27 03:01:45,019][105620] Updated weights for policy 1, policy_version 1600482 (0.0008) [2023-12-27 03:01:45,084][105620] Updated weights for policy 1, policy_version 1600492 (0.0006) [2023-12-27 03:01:45,110][105692] Updated weights for policy 0, policy_version 1597137 (0.0007) [2023-12-27 03:01:45,145][105620] Updated weights for policy 1, policy_version 1600502 (0.0007) [2023-12-27 03:01:45,171][105692] Updated weights for policy 0, policy_version 1597147 (0.0008) [2023-12-27 03:01:45,206][105620] Updated weights for policy 1, policy_version 1600512 (0.0006) [2023-12-27 03:01:45,237][105692] Updated weights for policy 0, policy_version 1597157 (0.0009) [2023-12-27 03:01:45,800][105620] Updated weights for policy 1, policy_version 1600522 (0.0011) [2023-12-27 03:01:45,866][105620] Updated weights for policy 1, policy_version 1600532 (0.0010) [2023-12-27 03:01:45,924][105620] Updated weights for policy 1, policy_version 1600542 (0.0010) [2023-12-27 03:01:46,056][105692] Updated weights for policy 0, policy_version 1597167 (0.0008) [2023-12-27 03:01:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 818724864. Throughput: 0: 9864.2, 1: 9559.4. Samples: 818697404. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:01:46,062][104569] Avg episode reward: [(0, '7711.590'), (1, '8901.568')] [2023-12-27 03:01:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001600544_409796608.pth... [2023-12-27 03:01:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001599424_409509888.pth [2023-12-27 03:01:46,074][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_001600544_409796608.pth [2023-12-27 03:01:46,108][105692] Updated weights for policy 0, policy_version 1597177 (0.0007) [2023-12-27 03:01:46,159][105692] Updated weights for policy 0, policy_version 1597187 (0.0008) [2023-12-27 03:01:46,189][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001597192_408936448.pth... [2023-12-27 03:01:46,194][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001596008_408633344.pth [2023-12-27 03:01:46,194][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_001597192_408936448.pth [2023-12-27 03:01:46,564][105620] Updated weights for policy 1, policy_version 1600552 (0.0010) [2023-12-27 03:01:46,614][105620] Updated weights for policy 1, policy_version 1600562 (0.0008) [2023-12-27 03:01:46,668][105620] Updated weights for policy 1, policy_version 1600572 (0.0010) [2023-12-27 03:01:46,825][105692] Updated weights for policy 0, policy_version 1597197 (0.0007) [2023-12-27 03:01:46,887][105692] Updated weights for policy 0, policy_version 1597207 (0.0007) [2023-12-27 03:01:46,949][105692] Updated weights for policy 0, policy_version 1597217 (0.0008) [2023-12-27 03:01:47,423][105620] Updated weights for policy 1, policy_version 1600582 (0.0010) [2023-12-27 03:01:47,478][105620] Updated weights for policy 1, policy_version 1600592 (0.0010) [2023-12-27 03:01:47,545][105620] Updated weights for policy 1, policy_version 1600602 (0.0010) [2023-12-27 03:01:47,594][105692] Updated weights for policy 0, policy_version 1597227 (0.0008) [2023-12-27 03:01:47,645][105692] Updated weights for policy 0, policy_version 1597237 (0.0008) [2023-12-27 03:01:47,704][105692] Updated weights for policy 0, policy_version 1597247 (0.0008) [2023-12-27 03:01:48,258][105620] Updated weights for policy 1, policy_version 1600612 (0.0010) [2023-12-27 03:01:48,318][105620] Updated weights for policy 1, policy_version 1600622 (0.0010) [2023-12-27 03:01:48,348][105692] Updated weights for policy 0, policy_version 1597257 (0.0008) [2023-12-27 03:01:48,378][105620] Updated weights for policy 1, policy_version 1600632 (0.0007) [2023-12-27 03:01:48,405][105692] Updated weights for policy 0, policy_version 1597267 (0.0007) [2023-12-27 03:01:48,464][105692] Updated weights for policy 0, policy_version 1597277 (0.0007) [2023-12-27 03:01:48,516][105692] Updated weights for policy 0, policy_version 1597287 (0.0008) [2023-12-27 03:01:49,121][105620] Updated weights for policy 1, policy_version 1600642 (0.0010) [2023-12-27 03:01:49,177][105620] Updated weights for policy 1, policy_version 1600652 (0.0009) [2023-12-27 03:01:49,199][105692] Updated weights for policy 0, policy_version 1597297 (0.0007) [2023-12-27 03:01:49,245][105620] Updated weights for policy 1, policy_version 1600662 (0.0010) [2023-12-27 03:01:49,265][105692] Updated weights for policy 0, policy_version 1597307 (0.0008) [2023-12-27 03:01:49,308][105620] Updated weights for policy 1, policy_version 1600672 (0.0008) [2023-12-27 03:01:49,324][105692] Updated weights for policy 0, policy_version 1597317 (0.0007) [2023-12-27 03:01:49,975][105620] Updated weights for policy 1, policy_version 1600682 (0.0007) [2023-12-27 03:01:50,040][105620] Updated weights for policy 1, policy_version 1600692 (0.0009) [2023-12-27 03:01:50,088][105620] Updated weights for policy 1, policy_version 1600702 (0.0007) [2023-12-27 03:01:50,090][105692] Updated weights for policy 0, policy_version 1597327 (0.0007) [2023-12-27 03:01:50,151][105692] Updated weights for policy 0, policy_version 1597337 (0.0007) [2023-12-27 03:01:50,210][105692] Updated weights for policy 0, policy_version 1597347 (0.0009) [2023-12-27 03:01:50,787][105620] Updated weights for policy 1, policy_version 1600712 (0.0008) [2023-12-27 03:01:50,837][105620] Updated weights for policy 1, policy_version 1600722 (0.0009) [2023-12-27 03:01:50,889][105620] Updated weights for policy 1, policy_version 1600732 (0.0006) [2023-12-27 03:01:51,005][105692] Updated weights for policy 0, policy_version 1597357 (0.0009) [2023-12-27 03:01:51,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 818823168. Throughput: 0: 9919.6, 1: 9584.0. Samples: 818815324. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:01:51,063][104569] Avg episode reward: [(0, '8254.315'), (1, '8989.396')] [2023-12-27 03:01:51,071][105692] Updated weights for policy 0, policy_version 1597367 (0.0009) [2023-12-27 03:01:51,117][105692] Updated weights for policy 0, policy_version 1597377 (0.0008) [2023-12-27 03:01:51,676][105620] Updated weights for policy 1, policy_version 1600742 (0.0006) [2023-12-27 03:01:51,736][105620] Updated weights for policy 1, policy_version 1600752 (0.0008) [2023-12-27 03:01:51,794][105620] Updated weights for policy 1, policy_version 1600762 (0.0009) [2023-12-27 03:01:51,810][105692] Updated weights for policy 0, policy_version 1597387 (0.0008) [2023-12-27 03:01:51,863][105692] Updated weights for policy 0, policy_version 1597397 (0.0008) [2023-12-27 03:01:51,914][105692] Updated weights for policy 0, policy_version 1597407 (0.0009) [2023-12-27 03:01:52,435][105620] Updated weights for policy 1, policy_version 1600772 (0.0007) [2023-12-27 03:01:52,499][105620] Updated weights for policy 1, policy_version 1600782 (0.0005) [2023-12-27 03:01:52,549][105620] Updated weights for policy 1, policy_version 1600792 (0.0007) [2023-12-27 03:01:52,785][105692] Updated weights for policy 0, policy_version 1597417 (0.0009) [2023-12-27 03:01:52,848][105692] Updated weights for policy 0, policy_version 1597427 (0.0010) [2023-12-27 03:01:52,908][105692] Updated weights for policy 0, policy_version 1597437 (0.0009) [2023-12-27 03:01:52,965][105692] Updated weights for policy 0, policy_version 1597447 (0.0009) [2023-12-27 03:01:53,207][105620] Updated weights for policy 1, policy_version 1600802 (0.0008) [2023-12-27 03:01:53,259][105620] Updated weights for policy 1, policy_version 1600812 (0.0008) [2023-12-27 03:01:53,315][105620] Updated weights for policy 1, policy_version 1600822 (0.0008) [2023-12-27 03:01:53,368][105620] Updated weights for policy 1, policy_version 1600832 (0.0009) [2023-12-27 03:01:53,774][105692] Updated weights for policy 0, policy_version 1597457 (0.0010) [2023-12-27 03:01:53,828][105692] Updated weights for policy 0, policy_version 1597467 (0.0009) [2023-12-27 03:01:53,880][105692] Updated weights for policy 0, policy_version 1597477 (0.0009) [2023-12-27 03:01:53,994][105620] Updated weights for policy 1, policy_version 1600842 (0.0010) [2023-12-27 03:01:54,055][105620] Updated weights for policy 1, policy_version 1600852 (0.0010) [2023-12-27 03:01:54,118][105620] Updated weights for policy 1, policy_version 1600862 (0.0010) [2023-12-27 03:01:54,739][105692] Updated weights for policy 0, policy_version 1597487 (0.0006) [2023-12-27 03:01:54,741][105620] Updated weights for policy 1, policy_version 1600872 (0.0011) [2023-12-27 03:01:54,805][105620] Updated weights for policy 1, policy_version 1600882 (0.0008) [2023-12-27 03:01:54,811][105692] Updated weights for policy 0, policy_version 1597497 (0.0008) [2023-12-27 03:01:54,871][105620] Updated weights for policy 1, policy_version 1600892 (0.0007) [2023-12-27 03:01:54,877][105692] Updated weights for policy 0, policy_version 1597507 (0.0008) [2023-12-27 03:01:55,489][105692] Updated weights for policy 0, policy_version 1597517 (0.0008) [2023-12-27 03:01:55,559][105692] Updated weights for policy 0, policy_version 1597527 (0.0008) [2023-12-27 03:01:55,601][105620] Updated weights for policy 1, policy_version 1600902 (0.0007) [2023-12-27 03:01:55,627][105692] Updated weights for policy 0, policy_version 1597537 (0.0008) [2023-12-27 03:01:55,652][105620] Updated weights for policy 1, policy_version 1600912 (0.0007) [2023-12-27 03:01:55,711][105620] Updated weights for policy 1, policy_version 1600922 (0.0009) [2023-12-27 03:01:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 818921472. Throughput: 0: 9926.2, 1: 9691.7. Samples: 818930432. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:01:56,063][104569] Avg episode reward: [(0, '8164.170'), (1, '8904.674')] [2023-12-27 03:01:56,170][105692] Updated weights for policy 0, policy_version 1597547 (0.0006) [2023-12-27 03:01:56,229][105692] Updated weights for policy 0, policy_version 1597557 (0.0005) [2023-12-27 03:01:56,290][105692] Updated weights for policy 0, policy_version 1597567 (0.0005) [2023-12-27 03:01:56,431][105620] Updated weights for policy 1, policy_version 1600932 (0.0010) [2023-12-27 03:01:56,490][105620] Updated weights for policy 1, policy_version 1600942 (0.0010) [2023-12-27 03:01:56,545][105620] Updated weights for policy 1, policy_version 1600952 (0.0010) [2023-12-27 03:01:56,791][105692] Updated weights for policy 0, policy_version 1597577 (0.0005) [2023-12-27 03:01:56,850][105692] Updated weights for policy 0, policy_version 1597587 (0.0005) [2023-12-27 03:01:56,902][105692] Updated weights for policy 0, policy_version 1597597 (0.0006) [2023-12-27 03:01:56,953][105692] Updated weights for policy 0, policy_version 1597607 (0.0010) [2023-12-27 03:01:57,191][105620] Updated weights for policy 1, policy_version 1600962 (0.0009) [2023-12-27 03:01:57,238][105620] Updated weights for policy 1, policy_version 1600972 (0.0005) [2023-12-27 03:01:57,293][105620] Updated weights for policy 1, policy_version 1600982 (0.0006) [2023-12-27 03:01:57,351][105620] Updated weights for policy 1, policy_version 1600992 (0.0009) [2023-12-27 03:01:57,650][105692] Updated weights for policy 0, policy_version 1597617 (0.0011) [2023-12-27 03:01:57,716][105692] Updated weights for policy 0, policy_version 1597627 (0.0011) [2023-12-27 03:01:57,780][105692] Updated weights for policy 0, policy_version 1597637 (0.0010) [2023-12-27 03:01:57,950][105620] Updated weights for policy 1, policy_version 1601002 (0.0010) [2023-12-27 03:01:57,993][105620] Updated weights for policy 1, policy_version 1601012 (0.0010) [2023-12-27 03:01:58,052][105620] Updated weights for policy 1, policy_version 1601022 (0.0008) [2023-12-27 03:01:58,540][105692] Updated weights for policy 0, policy_version 1597647 (0.0008) [2023-12-27 03:01:58,604][105692] Updated weights for policy 0, policy_version 1597657 (0.0007) [2023-12-27 03:01:58,665][105692] Updated weights for policy 0, policy_version 1597667 (0.0010) [2023-12-27 03:01:58,839][105620] Updated weights for policy 1, policy_version 1601032 (0.0007) [2023-12-27 03:01:58,909][105620] Updated weights for policy 1, policy_version 1601042 (0.0009) [2023-12-27 03:01:58,979][105620] Updated weights for policy 1, policy_version 1601052 (0.0008) [2023-12-27 03:01:59,486][105692] Updated weights for policy 0, policy_version 1597677 (0.0008) [2023-12-27 03:01:59,535][105692] Updated weights for policy 0, policy_version 1597687 (0.0005) [2023-12-27 03:01:59,592][105692] Updated weights for policy 0, policy_version 1597697 (0.0006) [2023-12-27 03:01:59,816][105620] Updated weights for policy 1, policy_version 1601062 (0.0009) [2023-12-27 03:01:59,883][105620] Updated weights for policy 1, policy_version 1601072 (0.0008) [2023-12-27 03:01:59,939][105620] Updated weights for policy 1, policy_version 1601082 (0.0009) [2023-12-27 03:02:00,190][105692] Updated weights for policy 0, policy_version 1597707 (0.0009) [2023-12-27 03:02:00,249][105692] Updated weights for policy 0, policy_version 1597717 (0.0011) [2023-12-27 03:02:00,311][105692] Updated weights for policy 0, policy_version 1597727 (0.0010) [2023-12-27 03:02:00,746][105620] Updated weights for policy 1, policy_version 1601092 (0.0007) [2023-12-27 03:02:00,806][105620] Updated weights for policy 1, policy_version 1601102 (0.0005) [2023-12-27 03:02:00,852][105620] Updated weights for policy 1, policy_version 1601112 (0.0005) [2023-12-27 03:02:00,985][105692] Updated weights for policy 0, policy_version 1597737 (0.0007) [2023-12-27 03:02:01,043][105692] Updated weights for policy 0, policy_version 1597747 (0.0008) [2023-12-27 03:02:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 819019776. Throughput: 0: 10022.2, 1: 9722.6. Samples: 818993208. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:02:01,063][104569] Avg episode reward: [(0, '8351.313'), (1, '9085.150')] [2023-12-27 03:02:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001601120_409944064.pth... [2023-12-27 03:02:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001599968_409649152.pth [2023-12-27 03:02:01,101][105692] Updated weights for policy 0, policy_version 1597757 (0.0010) [2023-12-27 03:02:01,160][105692] Updated weights for policy 0, policy_version 1597767 (0.0009) [2023-12-27 03:02:01,164][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001597768_409083904.pth... [2023-12-27 03:02:01,168][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001596616_408788992.pth [2023-12-27 03:02:01,574][105620] Updated weights for policy 1, policy_version 1601122 (0.0008) [2023-12-27 03:02:01,635][105620] Updated weights for policy 1, policy_version 1601132 (0.0008) [2023-12-27 03:02:01,693][105620] Updated weights for policy 1, policy_version 1601142 (0.0005) [2023-12-27 03:02:01,754][105620] Updated weights for policy 1, policy_version 1601152 (0.0009) [2023-12-27 03:02:01,869][105692] Updated weights for policy 0, policy_version 1597777 (0.0009) [2023-12-27 03:02:01,934][105692] Updated weights for policy 0, policy_version 1597787 (0.0009) [2023-12-27 03:02:01,989][105692] Updated weights for policy 0, policy_version 1597797 (0.0010) [2023-12-27 03:02:02,365][105620] Updated weights for policy 1, policy_version 1601162 (0.0007) [2023-12-27 03:02:02,427][105620] Updated weights for policy 1, policy_version 1601172 (0.0006) [2023-12-27 03:02:02,484][105620] Updated weights for policy 1, policy_version 1601182 (0.0005) [2023-12-27 03:02:02,811][105692] Updated weights for policy 0, policy_version 1597807 (0.0009) [2023-12-27 03:02:02,862][105692] Updated weights for policy 0, policy_version 1597817 (0.0007) [2023-12-27 03:02:02,929][105692] Updated weights for policy 0, policy_version 1597827 (0.0009) [2023-12-27 03:02:03,049][105620] Updated weights for policy 1, policy_version 1601192 (0.0009) [2023-12-27 03:02:03,100][105620] Updated weights for policy 1, policy_version 1601202 (0.0010) [2023-12-27 03:02:03,158][105620] Updated weights for policy 1, policy_version 1601212 (0.0010) [2023-12-27 03:02:03,537][105692] Updated weights for policy 0, policy_version 1597837 (0.0006) [2023-12-27 03:02:03,596][105692] Updated weights for policy 0, policy_version 1597847 (0.0005) [2023-12-27 03:02:03,666][105692] Updated weights for policy 0, policy_version 1597857 (0.0005) [2023-12-27 03:02:03,793][105620] Updated weights for policy 1, policy_version 1601222 (0.0008) [2023-12-27 03:02:03,861][105620] Updated weights for policy 1, policy_version 1601232 (0.0008) [2023-12-27 03:02:03,930][105620] Updated weights for policy 1, policy_version 1601242 (0.0008) [2023-12-27 03:02:04,210][105692] Updated weights for policy 0, policy_version 1597867 (0.0006) [2023-12-27 03:02:04,271][105692] Updated weights for policy 0, policy_version 1597877 (0.0005) [2023-12-27 03:02:04,338][105692] Updated weights for policy 0, policy_version 1597887 (0.0005) [2023-12-27 03:02:04,629][105620] Updated weights for policy 1, policy_version 1601252 (0.0009) [2023-12-27 03:02:04,692][105620] Updated weights for policy 1, policy_version 1601262 (0.0010) [2023-12-27 03:02:04,757][105620] Updated weights for policy 1, policy_version 1601272 (0.0010) [2023-12-27 03:02:04,995][105692] Updated weights for policy 0, policy_version 1597897 (0.0006) [2023-12-27 03:02:05,050][105692] Updated weights for policy 0, policy_version 1597907 (0.0010) [2023-12-27 03:02:05,103][105692] Updated weights for policy 0, policy_version 1597917 (0.0009) [2023-12-27 03:02:05,157][105692] Updated weights for policy 0, policy_version 1597928 (0.0010) [2023-12-27 03:02:05,346][105620] Updated weights for policy 1, policy_version 1601282 (0.0009) [2023-12-27 03:02:05,392][105620] Updated weights for policy 1, policy_version 1601292 (0.0005) [2023-12-27 03:02:05,442][105620] Updated weights for policy 1, policy_version 1601302 (0.0006) [2023-12-27 03:02:05,487][105620] Updated weights for policy 1, policy_version 1601312 (0.0005) [2023-12-27 03:02:06,009][105692] Updated weights for policy 0, policy_version 1597938 (0.0008) [2023-12-27 03:02:06,057][105692] Updated weights for policy 0, policy_version 1597948 (0.0008) [2023-12-27 03:02:06,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 819118080. Throughput: 0: 10025.0, 1: 9808.7. Samples: 819113360. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:02:06,062][104569] Avg episode reward: [(0, '8716.983'), (1, '9079.820')] [2023-12-27 03:02:06,105][105692] Updated weights for policy 0, policy_version 1597958 (0.0008) [2023-12-27 03:02:06,219][105620] Updated weights for policy 1, policy_version 1601322 (0.0011) [2023-12-27 03:02:06,285][105620] Updated weights for policy 1, policy_version 1601332 (0.0011) [2023-12-27 03:02:06,351][105620] Updated weights for policy 1, policy_version 1601342 (0.0011) [2023-12-27 03:02:06,900][105692] Updated weights for policy 0, policy_version 1597968 (0.0008) [2023-12-27 03:02:06,960][105692] Updated weights for policy 0, policy_version 1597978 (0.0008) [2023-12-27 03:02:07,021][105692] Updated weights for policy 0, policy_version 1597988 (0.0008) [2023-12-27 03:02:07,091][105620] Updated weights for policy 1, policy_version 1601352 (0.0011) [2023-12-27 03:02:07,140][105620] Updated weights for policy 1, policy_version 1601362 (0.0010) [2023-12-27 03:02:07,188][105620] Updated weights for policy 1, policy_version 1601372 (0.0010) [2023-12-27 03:02:07,714][105692] Updated weights for policy 0, policy_version 1597998 (0.0006) [2023-12-27 03:02:07,772][105692] Updated weights for policy 0, policy_version 1598008 (0.0005) [2023-12-27 03:02:07,830][105692] Updated weights for policy 0, policy_version 1598018 (0.0005) [2023-12-27 03:02:07,845][105620] Updated weights for policy 1, policy_version 1601382 (0.0007) [2023-12-27 03:02:07,916][105620] Updated weights for policy 1, policy_version 1601392 (0.0005) [2023-12-27 03:02:07,987][105620] Updated weights for policy 1, policy_version 1601402 (0.0005) [2023-12-27 03:02:08,424][105692] Updated weights for policy 0, policy_version 1598028 (0.0006) [2023-12-27 03:02:08,484][105692] Updated weights for policy 0, policy_version 1598038 (0.0008) [2023-12-27 03:02:08,539][105692] Updated weights for policy 0, policy_version 1598048 (0.0009) [2023-12-27 03:02:08,615][105620] Updated weights for policy 1, policy_version 1601412 (0.0007) [2023-12-27 03:02:08,684][105620] Updated weights for policy 1, policy_version 1601422 (0.0005) [2023-12-27 03:02:08,739][105620] Updated weights for policy 1, policy_version 1601432 (0.0006) [2023-12-27 03:02:09,300][105620] Updated weights for policy 1, policy_version 1601442 (0.0006) [2023-12-27 03:02:09,375][105620] Updated weights for policy 1, policy_version 1601452 (0.0010) [2023-12-27 03:02:09,388][105692] Updated weights for policy 0, policy_version 1598058 (0.0010) [2023-12-27 03:02:09,450][105620] Updated weights for policy 1, policy_version 1601462 (0.0008) [2023-12-27 03:02:09,460][105692] Updated weights for policy 0, policy_version 1598068 (0.0008) [2023-12-27 03:02:09,510][105620] Updated weights for policy 1, policy_version 1601472 (0.0008) [2023-12-27 03:02:09,521][105692] Updated weights for policy 0, policy_version 1598078 (0.0007) [2023-12-27 03:02:09,582][105692] Updated weights for policy 0, policy_version 1598088 (0.0009) [2023-12-27 03:02:10,313][105620] Updated weights for policy 1, policy_version 1601482 (0.0008) [2023-12-27 03:02:10,341][105692] Updated weights for policy 0, policy_version 1598098 (0.0006) [2023-12-27 03:02:10,381][105620] Updated weights for policy 1, policy_version 1601492 (0.0008) [2023-12-27 03:02:10,406][105692] Updated weights for policy 0, policy_version 1598108 (0.0009) [2023-12-27 03:02:10,441][105620] Updated weights for policy 1, policy_version 1601502 (0.0006) [2023-12-27 03:02:10,471][105692] Updated weights for policy 0, policy_version 1598118 (0.0009) [2023-12-27 03:02:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 819216384. Throughput: 0: 9853.7, 1: 9913.2. Samples: 819230056. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:02:11,063][104569] Avg episode reward: [(0, '8711.041'), (1, '8989.963')] [2023-12-27 03:02:11,166][105620] Updated weights for policy 1, policy_version 1601512 (0.0006) [2023-12-27 03:02:11,167][105692] Updated weights for policy 0, policy_version 1598128 (0.0009) [2023-12-27 03:02:11,225][105692] Updated weights for policy 0, policy_version 1598138 (0.0007) [2023-12-27 03:02:11,231][105620] Updated weights for policy 1, policy_version 1601522 (0.0007) [2023-12-27 03:02:11,290][105692] Updated weights for policy 0, policy_version 1598148 (0.0008) [2023-12-27 03:02:11,294][105620] Updated weights for policy 1, policy_version 1601532 (0.0008) [2023-12-27 03:02:11,953][105620] Updated weights for policy 1, policy_version 1601542 (0.0007) [2023-12-27 03:02:12,018][105620] Updated weights for policy 1, policy_version 1601552 (0.0006) [2023-12-27 03:02:12,050][105692] Updated weights for policy 0, policy_version 1598158 (0.0008) [2023-12-27 03:02:12,072][105620] Updated weights for policy 1, policy_version 1601562 (0.0005) [2023-12-27 03:02:12,103][105692] Updated weights for policy 0, policy_version 1598168 (0.0008) [2023-12-27 03:02:12,165][105692] Updated weights for policy 0, policy_version 1598178 (0.0010) [2023-12-27 03:02:12,662][105620] Updated weights for policy 1, policy_version 1601572 (0.0005) [2023-12-27 03:02:12,720][105620] Updated weights for policy 1, policy_version 1601582 (0.0005) [2023-12-27 03:02:12,776][105620] Updated weights for policy 1, policy_version 1601592 (0.0005) [2023-12-27 03:02:12,836][105692] Updated weights for policy 0, policy_version 1598188 (0.0008) [2023-12-27 03:02:12,887][105692] Updated weights for policy 0, policy_version 1598198 (0.0005) [2023-12-27 03:02:12,937][105692] Updated weights for policy 0, policy_version 1598208 (0.0005) [2023-12-27 03:02:13,331][105620] Updated weights for policy 1, policy_version 1601602 (0.0008) [2023-12-27 03:02:13,392][105620] Updated weights for policy 1, policy_version 1601612 (0.0005) [2023-12-27 03:02:13,462][105620] Updated weights for policy 1, policy_version 1601622 (0.0006) [2023-12-27 03:02:13,527][105620] Updated weights for policy 1, policy_version 1601632 (0.0006) [2023-12-27 03:02:13,533][105692] Updated weights for policy 0, policy_version 1598218 (0.0006) [2023-12-27 03:02:13,602][105692] Updated weights for policy 0, policy_version 1598228 (0.0009) [2023-12-27 03:02:13,660][105692] Updated weights for policy 0, policy_version 1598238 (0.0006) [2023-12-27 03:02:13,724][105692] Updated weights for policy 0, policy_version 1598248 (0.0006) [2023-12-27 03:02:14,172][105620] Updated weights for policy 1, policy_version 1601642 (0.0008) [2023-12-27 03:02:14,235][105620] Updated weights for policy 1, policy_version 1601652 (0.0009) [2023-12-27 03:02:14,297][105620] Updated weights for policy 1, policy_version 1601662 (0.0009) [2023-12-27 03:02:14,340][105692] Updated weights for policy 0, policy_version 1598258 (0.0005) [2023-12-27 03:02:14,396][105692] Updated weights for policy 0, policy_version 1598268 (0.0005) [2023-12-27 03:02:14,456][105692] Updated weights for policy 0, policy_version 1598278 (0.0006) [2023-12-27 03:02:15,065][105620] Updated weights for policy 1, policy_version 1601672 (0.0009) [2023-12-27 03:02:15,119][105620] Updated weights for policy 1, policy_version 1601682 (0.0009) [2023-12-27 03:02:15,186][105620] Updated weights for policy 1, policy_version 1601692 (0.0006) [2023-12-27 03:02:15,187][105692] Updated weights for policy 0, policy_version 1598288 (0.0008) [2023-12-27 03:02:15,241][105692] Updated weights for policy 0, policy_version 1598298 (0.0010) [2023-12-27 03:02:15,290][105692] Updated weights for policy 0, policy_version 1598308 (0.0010) [2023-12-27 03:02:15,855][105620] Updated weights for policy 1, policy_version 1601702 (0.0006) [2023-12-27 03:02:15,926][105692] Updated weights for policy 0, policy_version 1598318 (0.0007) [2023-12-27 03:02:15,940][105620] Updated weights for policy 1, policy_version 1601712 (0.0010) [2023-12-27 03:02:15,992][105692] Updated weights for policy 0, policy_version 1598328 (0.0005) [2023-12-27 03:02:15,999][105620] Updated weights for policy 1, policy_version 1601722 (0.0008) [2023-12-27 03:02:16,051][105692] Updated weights for policy 0, policy_version 1598338 (0.0007) [2023-12-27 03:02:16,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19933.9, 300 sec: 19494.2). Total num frames: 819322880. Throughput: 0: 9782.9, 1: 10003.7. Samples: 819293196. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:02:16,063][104569] Avg episode reward: [(0, '8618.603'), (1, '9081.834')] [2023-12-27 03:02:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001601728_410099712.pth... [2023-12-27 03:02:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001600544_409796608.pth [2023-12-27 03:02:16,079][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001598344_409231360.pth... [2023-12-27 03:02:16,081][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001597192_408936448.pth [2023-12-27 03:02:16,583][105620] Updated weights for policy 1, policy_version 1601732 (0.0007) [2023-12-27 03:02:16,616][105692] Updated weights for policy 0, policy_version 1598348 (0.0008) [2023-12-27 03:02:16,646][105620] Updated weights for policy 1, policy_version 1601742 (0.0005) [2023-12-27 03:02:16,681][105692] Updated weights for policy 0, policy_version 1598358 (0.0006) [2023-12-27 03:02:16,695][105620] Updated weights for policy 1, policy_version 1601752 (0.0005) [2023-12-27 03:02:16,743][105692] Updated weights for policy 0, policy_version 1598368 (0.0010) [2023-12-27 03:02:17,237][105620] Updated weights for policy 1, policy_version 1601762 (0.0005) [2023-12-27 03:02:17,290][105692] Updated weights for policy 0, policy_version 1598378 (0.0009) [2023-12-27 03:02:17,298][105620] Updated weights for policy 1, policy_version 1601772 (0.0005) [2023-12-27 03:02:17,343][105620] Updated weights for policy 1, policy_version 1601782 (0.0006) [2023-12-27 03:02:17,353][105692] Updated weights for policy 0, policy_version 1598388 (0.0005) [2023-12-27 03:02:17,407][105620] Updated weights for policy 1, policy_version 1601792 (0.0005) [2023-12-27 03:02:17,414][105692] Updated weights for policy 0, policy_version 1598398 (0.0008) [2023-12-27 03:02:17,463][105692] Updated weights for policy 0, policy_version 1598408 (0.0010) [2023-12-27 03:02:17,922][105620] Updated weights for policy 1, policy_version 1601802 (0.0005) [2023-12-27 03:02:17,989][105620] Updated weights for policy 1, policy_version 1601812 (0.0005) [2023-12-27 03:02:18,004][105692] Updated weights for policy 0, policy_version 1598418 (0.0005) [2023-12-27 03:02:18,049][105692] Updated weights for policy 0, policy_version 1598428 (0.0005) [2023-12-27 03:02:18,049][105620] Updated weights for policy 1, policy_version 1601822 (0.0006) [2023-12-27 03:02:18,110][105692] Updated weights for policy 0, policy_version 1598438 (0.0006) [2023-12-27 03:02:18,643][105620] Updated weights for policy 1, policy_version 1601832 (0.0010) [2023-12-27 03:02:18,703][105692] Updated weights for policy 0, policy_version 1598448 (0.0010) [2023-12-27 03:02:18,710][105620] Updated weights for policy 1, policy_version 1601842 (0.0009) [2023-12-27 03:02:18,765][105692] Updated weights for policy 0, policy_version 1598458 (0.0011) [2023-12-27 03:02:18,770][105620] Updated weights for policy 1, policy_version 1601852 (0.0011) [2023-12-27 03:02:18,825][105692] Updated weights for policy 0, policy_version 1598468 (0.0011) [2023-12-27 03:02:19,403][105620] Updated weights for policy 1, policy_version 1601862 (0.0011) [2023-12-27 03:02:19,459][105620] Updated weights for policy 1, policy_version 1601872 (0.0011) [2023-12-27 03:02:19,520][105620] Updated weights for policy 1, policy_version 1601882 (0.0011) [2023-12-27 03:02:19,561][105692] Updated weights for policy 0, policy_version 1598478 (0.0010) [2023-12-27 03:02:19,621][105692] Updated weights for policy 0, policy_version 1598488 (0.0008) [2023-12-27 03:02:19,673][105692] Updated weights for policy 0, policy_version 1598498 (0.0008) [2023-12-27 03:02:20,236][105620] Updated weights for policy 1, policy_version 1601892 (0.0008) [2023-12-27 03:02:20,300][105620] Updated weights for policy 1, policy_version 1601902 (0.0006) [2023-12-27 03:02:20,359][105620] Updated weights for policy 1, policy_version 1601912 (0.0007) [2023-12-27 03:02:20,465][105692] Updated weights for policy 0, policy_version 1598508 (0.0009) [2023-12-27 03:02:20,521][105692] Updated weights for policy 0, policy_version 1598518 (0.0006) [2023-12-27 03:02:20,597][105692] Updated weights for policy 0, policy_version 1598528 (0.0008) [2023-12-27 03:02:21,025][105620] Updated weights for policy 1, policy_version 1601922 (0.0008) [2023-12-27 03:02:21,062][104569] Fps is (10 sec: 21299.1, 60 sec: 19933.8, 300 sec: 19522.0). Total num frames: 819429376. Throughput: 0: 9815.3, 1: 10114.3. Samples: 819423028. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:02:21,063][104569] Avg episode reward: [(0, '8531.428'), (1, '9267.130')] [2023-12-27 03:02:21,100][105620] Updated weights for policy 1, policy_version 1601932 (0.0010) [2023-12-27 03:02:21,168][105620] Updated weights for policy 1, policy_version 1601942 (0.0009) [2023-12-27 03:02:21,230][105620] Updated weights for policy 1, policy_version 1601952 (0.0009) [2023-12-27 03:02:21,327][105692] Updated weights for policy 0, policy_version 1598538 (0.0008) [2023-12-27 03:02:21,392][105692] Updated weights for policy 0, policy_version 1598548 (0.0009) [2023-12-27 03:02:21,450][105692] Updated weights for policy 0, policy_version 1598558 (0.0010) [2023-12-27 03:02:21,513][105692] Updated weights for policy 0, policy_version 1598568 (0.0010) [2023-12-27 03:02:21,986][105620] Updated weights for policy 1, policy_version 1601962 (0.0008) [2023-12-27 03:02:22,044][105620] Updated weights for policy 1, policy_version 1601972 (0.0008) [2023-12-27 03:02:22,106][105620] Updated weights for policy 1, policy_version 1601982 (0.0009) [2023-12-27 03:02:22,278][105692] Updated weights for policy 0, policy_version 1598578 (0.0010) [2023-12-27 03:02:22,336][105692] Updated weights for policy 0, policy_version 1598588 (0.0009) [2023-12-27 03:02:22,402][105692] Updated weights for policy 0, policy_version 1598598 (0.0007) [2023-12-27 03:02:22,808][105620] Updated weights for policy 1, policy_version 1601992 (0.0006) [2023-12-27 03:02:22,878][105620] Updated weights for policy 1, policy_version 1602002 (0.0006) [2023-12-27 03:02:22,943][105620] Updated weights for policy 1, policy_version 1602012 (0.0006) [2023-12-27 03:02:23,250][105692] Updated weights for policy 0, policy_version 1598608 (0.0009) [2023-12-27 03:02:23,312][105692] Updated weights for policy 0, policy_version 1598619 (0.0010) [2023-12-27 03:02:23,374][105692] Updated weights for policy 0, policy_version 1598629 (0.0010) [2023-12-27 03:02:23,440][105620] Updated weights for policy 1, policy_version 1602022 (0.0007) [2023-12-27 03:02:23,498][105620] Updated weights for policy 1, policy_version 1602032 (0.0009) [2023-12-27 03:02:23,552][105620] Updated weights for policy 1, policy_version 1602042 (0.0009) [2023-12-27 03:02:24,201][105692] Updated weights for policy 0, policy_version 1598639 (0.0007) [2023-12-27 03:02:24,237][105620] Updated weights for policy 1, policy_version 1602052 (0.0007) [2023-12-27 03:02:24,255][105692] Updated weights for policy 0, policy_version 1598649 (0.0008) [2023-12-27 03:02:24,292][105620] Updated weights for policy 1, policy_version 1602062 (0.0007) [2023-12-27 03:02:24,298][105692] Updated weights for policy 0, policy_version 1598659 (0.0008) [2023-12-27 03:02:24,349][105620] Updated weights for policy 1, policy_version 1602072 (0.0009) [2023-12-27 03:02:25,088][105692] Updated weights for policy 0, policy_version 1598669 (0.0007) [2023-12-27 03:02:25,093][105620] Updated weights for policy 1, policy_version 1602082 (0.0009) [2023-12-27 03:02:25,148][105692] Updated weights for policy 0, policy_version 1598679 (0.0008) [2023-12-27 03:02:25,154][105620] Updated weights for policy 1, policy_version 1602092 (0.0008) [2023-12-27 03:02:25,207][105692] Updated weights for policy 0, policy_version 1598689 (0.0006) [2023-12-27 03:02:25,214][105620] Updated weights for policy 1, policy_version 1602102 (0.0008) [2023-12-27 03:02:25,274][105620] Updated weights for policy 1, policy_version 1602112 (0.0008) [2023-12-27 03:02:25,931][105692] Updated weights for policy 0, policy_version 1598699 (0.0007) [2023-12-27 03:02:25,988][105692] Updated weights for policy 0, policy_version 1598709 (0.0009) [2023-12-27 03:02:26,034][105692] Updated weights for policy 0, policy_version 1598719 (0.0007) [2023-12-27 03:02:26,048][105620] Updated weights for policy 1, policy_version 1602122 (0.0007) [2023-12-27 03:02:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 819519488. Throughput: 0: 9734.6, 1: 10146.0. Samples: 819536152. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:02:26,062][104569] Avg episode reward: [(0, '8531.746'), (1, '9174.875')] [2023-12-27 03:02:26,101][105620] Updated weights for policy 1, policy_version 1602132 (0.0006) [2023-12-27 03:02:26,158][105620] Updated weights for policy 1, policy_version 1602142 (0.0008) [2023-12-27 03:02:26,720][105692] Updated weights for policy 0, policy_version 1598729 (0.0007) [2023-12-27 03:02:26,782][105692] Updated weights for policy 0, policy_version 1598739 (0.0006) [2023-12-27 03:02:26,838][105692] Updated weights for policy 0, policy_version 1598749 (0.0007) [2023-12-27 03:02:26,896][105692] Updated weights for policy 0, policy_version 1598759 (0.0005) [2023-12-27 03:02:26,967][105620] Updated weights for policy 1, policy_version 1602152 (0.0010) [2023-12-27 03:02:27,012][105620] Updated weights for policy 1, policy_version 1602162 (0.0008) [2023-12-27 03:02:27,059][105620] Updated weights for policy 1, policy_version 1602172 (0.0009) [2023-12-27 03:02:27,473][105692] Updated weights for policy 0, policy_version 1598769 (0.0008) [2023-12-27 03:02:27,520][105692] Updated weights for policy 0, policy_version 1598779 (0.0009) [2023-12-27 03:02:27,566][105692] Updated weights for policy 0, policy_version 1598789 (0.0009) [2023-12-27 03:02:27,810][105620] Updated weights for policy 1, policy_version 1602182 (0.0007) [2023-12-27 03:02:27,842][105586] KL-divergence is very high: 138.4905 [2023-12-27 03:02:27,856][105620] Updated weights for policy 1, policy_version 1602192 (0.0008) [2023-12-27 03:02:27,881][105586] KL-divergence is very high: 250.6314 [2023-12-27 03:02:27,906][105620] Updated weights for policy 1, policy_version 1602202 (0.0008) [2023-12-27 03:02:27,919][105586] KL-divergence is very high: 270.8903 [2023-12-27 03:02:28,377][105692] Updated weights for policy 0, policy_version 1598799 (0.0008) [2023-12-27 03:02:28,424][105692] Updated weights for policy 0, policy_version 1598809 (0.0009) [2023-12-27 03:02:28,475][105692] Updated weights for policy 0, policy_version 1598819 (0.0009) [2023-12-27 03:02:28,657][105620] Updated weights for policy 1, policy_version 1602212 (0.0008) [2023-12-27 03:02:28,728][105620] Updated weights for policy 1, policy_version 1602222 (0.0005) [2023-12-27 03:02:28,797][105620] Updated weights for policy 1, policy_version 1602232 (0.0006) [2023-12-27 03:02:29,244][105692] Updated weights for policy 0, policy_version 1598829 (0.0007) [2023-12-27 03:02:29,310][105692] Updated weights for policy 0, policy_version 1598839 (0.0010) [2023-12-27 03:02:29,376][105692] Updated weights for policy 0, policy_version 1598849 (0.0009) [2023-12-27 03:02:29,459][105620] Updated weights for policy 1, policy_version 1602242 (0.0006) [2023-12-27 03:02:29,515][105620] Updated weights for policy 1, policy_version 1602252 (0.0008) [2023-12-27 03:02:29,573][105620] Updated weights for policy 1, policy_version 1602262 (0.0005) [2023-12-27 03:02:29,633][105620] Updated weights for policy 1, policy_version 1602272 (0.0008) [2023-12-27 03:02:30,140][105692] Updated weights for policy 0, policy_version 1598859 (0.0009) [2023-12-27 03:02:30,188][105692] Updated weights for policy 0, policy_version 1598869 (0.0006) [2023-12-27 03:02:30,249][105692] Updated weights for policy 0, policy_version 1598879 (0.0005) [2023-12-27 03:02:30,396][105620] Updated weights for policy 1, policy_version 1602282 (0.0010) [2023-12-27 03:02:30,444][105620] Updated weights for policy 1, policy_version 1602292 (0.0010) [2023-12-27 03:02:30,493][105620] Updated weights for policy 1, policy_version 1602302 (0.0010) [2023-12-27 03:02:30,791][105692] Updated weights for policy 0, policy_version 1598889 (0.0005) [2023-12-27 03:02:30,838][105692] Updated weights for policy 0, policy_version 1598899 (0.0005) [2023-12-27 03:02:30,893][105692] Updated weights for policy 0, policy_version 1598909 (0.0005) [2023-12-27 03:02:30,936][105692] Updated weights for policy 0, policy_version 1598919 (0.0005) [2023-12-27 03:02:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 819625984. Throughput: 0: 9801.4, 1: 10141.9. Samples: 819594852. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:02:31,062][104569] Avg episode reward: [(0, '8440.979'), (1, '8992.644')] [2023-12-27 03:02:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001598920_409378816.pth... [2023-12-27 03:02:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001602304_410247168.pth... [2023-12-27 03:02:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001597768_409083904.pth [2023-12-27 03:02:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001601120_409944064.pth [2023-12-27 03:02:31,239][105620] Updated weights for policy 1, policy_version 1602312 (0.0010) [2023-12-27 03:02:31,290][105620] Updated weights for policy 1, policy_version 1602322 (0.0008) [2023-12-27 03:02:31,351][105620] Updated weights for policy 1, policy_version 1602332 (0.0009) [2023-12-27 03:02:31,583][105692] Updated weights for policy 0, policy_version 1598929 (0.0010) [2023-12-27 03:02:31,650][105692] Updated weights for policy 0, policy_version 1598939 (0.0011) [2023-12-27 03:02:31,702][105692] Updated weights for policy 0, policy_version 1598949 (0.0011) [2023-12-27 03:02:32,109][105620] Updated weights for policy 1, policy_version 1602342 (0.0010) [2023-12-27 03:02:32,166][105620] Updated weights for policy 1, policy_version 1602352 (0.0007) [2023-12-27 03:02:32,232][105620] Updated weights for policy 1, policy_version 1602362 (0.0008) [2023-12-27 03:02:32,471][105692] Updated weights for policy 0, policy_version 1598959 (0.0011) [2023-12-27 03:02:32,529][105692] Updated weights for policy 0, policy_version 1598969 (0.0010) [2023-12-27 03:02:32,584][105692] Updated weights for policy 0, policy_version 1598979 (0.0010) [2023-12-27 03:02:32,924][105620] Updated weights for policy 1, policy_version 1602372 (0.0009) [2023-12-27 03:02:32,985][105620] Updated weights for policy 1, policy_version 1602382 (0.0010) [2023-12-27 03:02:33,049][105620] Updated weights for policy 1, policy_version 1602392 (0.0010) [2023-12-27 03:02:33,194][105692] Updated weights for policy 0, policy_version 1598989 (0.0008) [2023-12-27 03:02:33,248][105692] Updated weights for policy 0, policy_version 1598999 (0.0005) [2023-12-27 03:02:33,302][105692] Updated weights for policy 0, policy_version 1599009 (0.0005) [2023-12-27 03:02:33,757][105620] Updated weights for policy 1, policy_version 1602402 (0.0009) [2023-12-27 03:02:33,805][105620] Updated weights for policy 1, policy_version 1602412 (0.0005) [2023-12-27 03:02:33,847][105692] Updated weights for policy 0, policy_version 1599019 (0.0005) [2023-12-27 03:02:33,859][105620] Updated weights for policy 1, policy_version 1602422 (0.0005) [2023-12-27 03:02:33,905][105692] Updated weights for policy 0, policy_version 1599029 (0.0006) [2023-12-27 03:02:33,911][105620] Updated weights for policy 1, policy_version 1602432 (0.0006) [2023-12-27 03:02:33,950][105692] Updated weights for policy 0, policy_version 1599039 (0.0006) [2023-12-27 03:02:34,528][105620] Updated weights for policy 1, policy_version 1602442 (0.0009) [2023-12-27 03:02:34,578][105620] Updated weights for policy 1, policy_version 1602452 (0.0009) [2023-12-27 03:02:34,602][105692] Updated weights for policy 0, policy_version 1599049 (0.0006) [2023-12-27 03:02:34,633][105620] Updated weights for policy 1, policy_version 1602462 (0.0007) [2023-12-27 03:02:34,662][105692] Updated weights for policy 0, policy_version 1599059 (0.0011) [2023-12-27 03:02:34,724][105692] Updated weights for policy 0, policy_version 1599069 (0.0011) [2023-12-27 03:02:34,789][105692] Updated weights for policy 0, policy_version 1599079 (0.0006) [2023-12-27 03:02:35,391][105692] Updated weights for policy 0, policy_version 1599089 (0.0009) [2023-12-27 03:02:35,446][105692] Updated weights for policy 0, policy_version 1599099 (0.0008) [2023-12-27 03:02:35,485][105620] Updated weights for policy 1, policy_version 1602472 (0.0009) [2023-12-27 03:02:35,504][105692] Updated weights for policy 0, policy_version 1599109 (0.0008) [2023-12-27 03:02:35,535][105620] Updated weights for policy 1, policy_version 1602482 (0.0010) [2023-12-27 03:02:35,590][105620] Updated weights for policy 1, policy_version 1602492 (0.0010) [2023-12-27 03:02:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19933.9, 300 sec: 19522.0). Total num frames: 819724288. Throughput: 0: 9917.1, 1: 10097.5. Samples: 819715980. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:02:36,063][104569] Avg episode reward: [(0, '8623.454'), (1, '9085.296')] [2023-12-27 03:02:36,216][105692] Updated weights for policy 0, policy_version 1599119 (0.0009) [2023-12-27 03:02:36,270][105692] Updated weights for policy 0, policy_version 1599129 (0.0011) [2023-12-27 03:02:36,278][105620] Updated weights for policy 1, policy_version 1602502 (0.0010) [2023-12-27 03:02:36,326][105692] Updated weights for policy 0, policy_version 1599139 (0.0011) [2023-12-27 03:02:36,335][105620] Updated weights for policy 1, policy_version 1602512 (0.0011) [2023-12-27 03:02:36,387][105620] Updated weights for policy 1, policy_version 1602522 (0.0010) [2023-12-27 03:02:37,047][105692] Updated weights for policy 0, policy_version 1599149 (0.0008) [2023-12-27 03:02:37,076][105620] Updated weights for policy 1, policy_version 1602532 (0.0010) [2023-12-27 03:02:37,111][105692] Updated weights for policy 0, policy_version 1599159 (0.0006) [2023-12-27 03:02:37,124][105620] Updated weights for policy 1, policy_version 1602542 (0.0010) [2023-12-27 03:02:37,173][105692] Updated weights for policy 0, policy_version 1599169 (0.0006) [2023-12-27 03:02:37,176][105620] Updated weights for policy 1, policy_version 1602552 (0.0010) [2023-12-27 03:02:37,835][105692] Updated weights for policy 0, policy_version 1599179 (0.0007) [2023-12-27 03:02:37,846][105620] Updated weights for policy 1, policy_version 1602562 (0.0009) [2023-12-27 03:02:37,896][105692] Updated weights for policy 0, policy_version 1599189 (0.0009) [2023-12-27 03:02:37,907][105620] Updated weights for policy 1, policy_version 1602572 (0.0006) [2023-12-27 03:02:37,946][105692] Updated weights for policy 0, policy_version 1599199 (0.0006) [2023-12-27 03:02:37,964][105620] Updated weights for policy 1, policy_version 1602582 (0.0010) [2023-12-27 03:02:38,026][105620] Updated weights for policy 1, policy_version 1602592 (0.0010) [2023-12-27 03:02:38,657][105620] Updated weights for policy 1, policy_version 1602602 (0.0008) [2023-12-27 03:02:38,666][105692] Updated weights for policy 0, policy_version 1599209 (0.0006) [2023-12-27 03:02:38,713][105692] Updated weights for policy 0, policy_version 1599219 (0.0007) [2023-12-27 03:02:38,719][105620] Updated weights for policy 1, policy_version 1602612 (0.0010) [2023-12-27 03:02:38,773][105692] Updated weights for policy 0, policy_version 1599229 (0.0006) [2023-12-27 03:02:38,778][105620] Updated weights for policy 1, policy_version 1602622 (0.0010) [2023-12-27 03:02:38,832][105692] Updated weights for policy 0, policy_version 1599239 (0.0008) [2023-12-27 03:02:39,465][105620] Updated weights for policy 1, policy_version 1602632 (0.0007) [2023-12-27 03:02:39,534][105620] Updated weights for policy 1, policy_version 1602642 (0.0008) [2023-12-27 03:02:39,595][105620] Updated weights for policy 1, policy_version 1602652 (0.0008) [2023-12-27 03:02:39,621][105692] Updated weights for policy 0, policy_version 1599249 (0.0007) [2023-12-27 03:02:39,679][105692] Updated weights for policy 0, policy_version 1599259 (0.0006) [2023-12-27 03:02:39,743][105692] Updated weights for policy 0, policy_version 1599269 (0.0007) [2023-12-27 03:02:40,257][105620] Updated weights for policy 1, policy_version 1602662 (0.0006) [2023-12-27 03:02:40,318][105620] Updated weights for policy 1, policy_version 1602672 (0.0008) [2023-12-27 03:02:40,372][105620] Updated weights for policy 1, policy_version 1602682 (0.0008) [2023-12-27 03:02:40,444][105692] Updated weights for policy 0, policy_version 1599279 (0.0007) [2023-12-27 03:02:40,509][105692] Updated weights for policy 0, policy_version 1599289 (0.0008) [2023-12-27 03:02:40,576][105692] Updated weights for policy 0, policy_version 1599299 (0.0006) [2023-12-27 03:02:41,010][105620] Updated weights for policy 1, policy_version 1602692 (0.0010) [2023-12-27 03:02:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19522.0). Total num frames: 819822592. Throughput: 0: 10023.8, 1: 10106.2. Samples: 819836280. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:02:41,062][104569] Avg episode reward: [(0, '8987.274'), (1, '8992.525')] [2023-12-27 03:02:41,075][105620] Updated weights for policy 1, policy_version 1602702 (0.0010) [2023-12-27 03:02:41,137][105620] Updated weights for policy 1, policy_version 1602712 (0.0009) [2023-12-27 03:02:41,357][105692] Updated weights for policy 0, policy_version 1599309 (0.0009) [2023-12-27 03:02:41,423][105692] Updated weights for policy 0, policy_version 1599319 (0.0009) [2023-12-27 03:02:41,484][105692] Updated weights for policy 0, policy_version 1599329 (0.0009) [2023-12-27 03:02:41,987][105620] Updated weights for policy 1, policy_version 1602722 (0.0009) [2023-12-27 03:02:42,049][105620] Updated weights for policy 1, policy_version 1602732 (0.0008) [2023-12-27 03:02:42,098][105620] Updated weights for policy 1, policy_version 1602742 (0.0011) [2023-12-27 03:02:42,162][105620] Updated weights for policy 1, policy_version 1602752 (0.0006) [2023-12-27 03:02:42,246][105692] Updated weights for policy 0, policy_version 1599339 (0.0010) [2023-12-27 03:02:42,314][105692] Updated weights for policy 0, policy_version 1599349 (0.0008) [2023-12-27 03:02:42,383][105692] Updated weights for policy 0, policy_version 1599359 (0.0008) [2023-12-27 03:02:42,899][105620] Updated weights for policy 1, policy_version 1602762 (0.0010) [2023-12-27 03:02:42,940][105586] KL-divergence is very high: 140.6028 [2023-12-27 03:02:42,950][105620] Updated weights for policy 1, policy_version 1602772 (0.0010) [2023-12-27 03:02:42,985][105586] KL-divergence is very high: 272.2520 [2023-12-27 03:02:43,006][105620] Updated weights for policy 1, policy_version 1602782 (0.0009) [2023-12-27 03:02:43,113][105692] Updated weights for policy 0, policy_version 1599369 (0.0010) [2023-12-27 03:02:43,165][105692] Updated weights for policy 0, policy_version 1599379 (0.0010) [2023-12-27 03:02:43,221][105692] Updated weights for policy 0, policy_version 1599389 (0.0010) [2023-12-27 03:02:43,276][105692] Updated weights for policy 0, policy_version 1599399 (0.0010) [2023-12-27 03:02:43,605][105620] Updated weights for policy 1, policy_version 1602792 (0.0008) [2023-12-27 03:02:43,649][105620] Updated weights for policy 1, policy_version 1602802 (0.0008) [2023-12-27 03:02:43,693][105620] Updated weights for policy 1, policy_version 1602812 (0.0006) [2023-12-27 03:02:43,975][105692] Updated weights for policy 0, policy_version 1599409 (0.0010) [2023-12-27 03:02:44,029][105692] Updated weights for policy 0, policy_version 1599419 (0.0010) [2023-12-27 03:02:44,080][105692] Updated weights for policy 0, policy_version 1599429 (0.0010) [2023-12-27 03:02:44,475][105620] Updated weights for policy 1, policy_version 1602822 (0.0010) [2023-12-27 03:02:44,541][105620] Updated weights for policy 1, policy_version 1602832 (0.0010) [2023-12-27 03:02:44,602][105620] Updated weights for policy 1, policy_version 1602842 (0.0010) [2023-12-27 03:02:44,800][105692] Updated weights for policy 0, policy_version 1599439 (0.0008) [2023-12-27 03:02:44,861][105692] Updated weights for policy 0, policy_version 1599449 (0.0008) [2023-12-27 03:02:44,918][105692] Updated weights for policy 0, policy_version 1599459 (0.0008) [2023-12-27 03:02:45,349][105620] Updated weights for policy 1, policy_version 1602852 (0.0011) [2023-12-27 03:02:45,413][105620] Updated weights for policy 1, policy_version 1602862 (0.0011) [2023-12-27 03:02:45,483][105620] Updated weights for policy 1, policy_version 1602872 (0.0010) [2023-12-27 03:02:45,599][105692] Updated weights for policy 0, policy_version 1599469 (0.0008) [2023-12-27 03:02:45,665][105692] Updated weights for policy 0, policy_version 1599479 (0.0006) [2023-12-27 03:02:45,732][105692] Updated weights for policy 0, policy_version 1599489 (0.0005) [2023-12-27 03:02:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19933.8, 300 sec: 19521.9). Total num frames: 819920896. Throughput: 0: 9903.2, 1: 10071.1. Samples: 819892056. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:02:46,063][104569] Avg episode reward: [(0, '8720.175'), (1, '8992.627')] [2023-12-27 03:02:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001599496_409526272.pth... [2023-12-27 03:02:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001602880_410394624.pth... [2023-12-27 03:02:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001598344_409231360.pth [2023-12-27 03:02:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001601728_410099712.pth [2023-12-27 03:02:46,166][105620] Updated weights for policy 1, policy_version 1602882 (0.0011) [2023-12-27 03:02:46,218][105620] Updated weights for policy 1, policy_version 1602892 (0.0010) [2023-12-27 03:02:46,262][105620] Updated weights for policy 1, policy_version 1602902 (0.0010) [2023-12-27 03:02:46,289][105692] Updated weights for policy 0, policy_version 1599499 (0.0005) [2023-12-27 03:02:46,317][105620] Updated weights for policy 1, policy_version 1602912 (0.0010) [2023-12-27 03:02:46,351][105692] Updated weights for policy 0, policy_version 1599509 (0.0006) [2023-12-27 03:02:46,412][105692] Updated weights for policy 0, policy_version 1599519 (0.0008) [2023-12-27 03:02:47,073][105692] Updated weights for policy 0, policy_version 1599529 (0.0006) [2023-12-27 03:02:47,102][105620] Updated weights for policy 1, policy_version 1602922 (0.0011) [2023-12-27 03:02:47,130][105692] Updated weights for policy 0, policy_version 1599539 (0.0007) [2023-12-27 03:02:47,154][105620] Updated weights for policy 1, policy_version 1602932 (0.0011) [2023-12-27 03:02:47,187][105692] Updated weights for policy 0, policy_version 1599549 (0.0007) [2023-12-27 03:02:47,209][105620] Updated weights for policy 1, policy_version 1602942 (0.0010) [2023-12-27 03:02:47,244][105692] Updated weights for policy 0, policy_version 1599559 (0.0006) [2023-12-27 03:02:47,842][105692] Updated weights for policy 0, policy_version 1599569 (0.0006) [2023-12-27 03:02:47,908][105692] Updated weights for policy 0, policy_version 1599579 (0.0005) [2023-12-27 03:02:47,947][105620] Updated weights for policy 1, policy_version 1602952 (0.0011) [2023-12-27 03:02:47,962][105692] Updated weights for policy 0, policy_version 1599589 (0.0007) [2023-12-27 03:02:48,003][105620] Updated weights for policy 1, policy_version 1602962 (0.0011) [2023-12-27 03:02:48,062][105620] Updated weights for policy 1, policy_version 1602972 (0.0011) [2023-12-27 03:02:48,651][105692] Updated weights for policy 0, policy_version 1599599 (0.0007) [2023-12-27 03:02:48,714][105692] Updated weights for policy 0, policy_version 1599609 (0.0005) [2023-12-27 03:02:48,779][105692] Updated weights for policy 0, policy_version 1599619 (0.0005) [2023-12-27 03:02:48,817][105620] Updated weights for policy 1, policy_version 1602982 (0.0011) [2023-12-27 03:02:48,879][105620] Updated weights for policy 1, policy_version 1602992 (0.0011) [2023-12-27 03:02:48,930][105620] Updated weights for policy 1, policy_version 1603002 (0.0010) [2023-12-27 03:02:49,452][105692] Updated weights for policy 0, policy_version 1599629 (0.0008) [2023-12-27 03:02:49,516][105692] Updated weights for policy 0, policy_version 1599639 (0.0009) [2023-12-27 03:02:49,568][105692] Updated weights for policy 0, policy_version 1599650 (0.0010) [2023-12-27 03:02:49,661][105620] Updated weights for policy 1, policy_version 1603012 (0.0010) [2023-12-27 03:02:49,724][105620] Updated weights for policy 1, policy_version 1603022 (0.0009) [2023-12-27 03:02:49,786][105620] Updated weights for policy 1, policy_version 1603032 (0.0009) [2023-12-27 03:02:50,338][105692] Updated weights for policy 0, policy_version 1599660 (0.0009) [2023-12-27 03:02:50,410][105692] Updated weights for policy 0, policy_version 1599670 (0.0007) [2023-12-27 03:02:50,475][105692] Updated weights for policy 0, policy_version 1599680 (0.0009) [2023-12-27 03:02:50,475][105620] Updated weights for policy 1, policy_version 1603042 (0.0009) [2023-12-27 03:02:50,527][105620] Updated weights for policy 1, policy_version 1603052 (0.0009) [2023-12-27 03:02:50,596][105620] Updated weights for policy 1, policy_version 1603062 (0.0010) [2023-12-27 03:02:50,648][105620] Updated weights for policy 1, policy_version 1603072 (0.0010) [2023-12-27 03:02:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19549.7). Total num frames: 820019200. Throughput: 0: 9940.9, 1: 10000.3. Samples: 820010716. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:02:51,063][104569] Avg episode reward: [(0, '8718.684'), (1, '8901.471')] [2023-12-27 03:02:51,102][105692] Updated weights for policy 0, policy_version 1599690 (0.0007) [2023-12-27 03:02:51,167][105692] Updated weights for policy 0, policy_version 1599700 (0.0009) [2023-12-27 03:02:51,229][105692] Updated weights for policy 0, policy_version 1599710 (0.0009) [2023-12-27 03:02:51,291][105692] Updated weights for policy 0, policy_version 1599720 (0.0009) [2023-12-27 03:02:51,509][105620] Updated weights for policy 1, policy_version 1603082 (0.0009) [2023-12-27 03:02:51,567][105620] Updated weights for policy 1, policy_version 1603092 (0.0009) [2023-12-27 03:02:51,620][105620] Updated weights for policy 1, policy_version 1603102 (0.0010) [2023-12-27 03:02:52,074][105692] Updated weights for policy 0, policy_version 1599730 (0.0008) [2023-12-27 03:02:52,134][105692] Updated weights for policy 0, policy_version 1599740 (0.0008) [2023-12-27 03:02:52,190][105692] Updated weights for policy 0, policy_version 1599750 (0.0008) [2023-12-27 03:02:52,385][105620] Updated weights for policy 1, policy_version 1603112 (0.0009) [2023-12-27 03:02:52,435][105620] Updated weights for policy 1, policy_version 1603122 (0.0008) [2023-12-27 03:02:52,499][105620] Updated weights for policy 1, policy_version 1603132 (0.0006) [2023-12-27 03:02:52,970][105692] Updated weights for policy 0, policy_version 1599760 (0.0009) [2023-12-27 03:02:53,021][105692] Updated weights for policy 0, policy_version 1599770 (0.0008) [2023-12-27 03:02:53,080][105692] Updated weights for policy 0, policy_version 1599780 (0.0009) [2023-12-27 03:02:53,111][105620] Updated weights for policy 1, policy_version 1603142 (0.0007) [2023-12-27 03:02:53,161][105620] Updated weights for policy 1, policy_version 1603152 (0.0007) [2023-12-27 03:02:53,221][105620] Updated weights for policy 1, policy_version 1603162 (0.0008) [2023-12-27 03:02:53,794][105692] Updated weights for policy 0, policy_version 1599790 (0.0008) [2023-12-27 03:02:53,853][105692] Updated weights for policy 0, policy_version 1599800 (0.0005) [2023-12-27 03:02:53,910][105692] Updated weights for policy 0, policy_version 1599810 (0.0009) [2023-12-27 03:02:53,911][105620] Updated weights for policy 1, policy_version 1603172 (0.0008) [2023-12-27 03:02:53,971][105620] Updated weights for policy 1, policy_version 1603182 (0.0009) [2023-12-27 03:02:54,029][105620] Updated weights for policy 1, policy_version 1603192 (0.0010) [2023-12-27 03:02:54,507][105692] Updated weights for policy 0, policy_version 1599820 (0.0008) [2023-12-27 03:02:54,578][105692] Updated weights for policy 0, policy_version 1599830 (0.0008) [2023-12-27 03:02:54,624][105620] Updated weights for policy 1, policy_version 1603202 (0.0010) [2023-12-27 03:02:54,638][105692] Updated weights for policy 0, policy_version 1599840 (0.0006) [2023-12-27 03:02:54,683][105620] Updated weights for policy 1, policy_version 1603212 (0.0009) [2023-12-27 03:02:54,748][105620] Updated weights for policy 1, policy_version 1603222 (0.0011) [2023-12-27 03:02:54,816][105620] Updated weights for policy 1, policy_version 1603232 (0.0011) [2023-12-27 03:02:55,190][105692] Updated weights for policy 0, policy_version 1599850 (0.0005) [2023-12-27 03:02:55,247][105692] Updated weights for policy 0, policy_version 1599860 (0.0005) [2023-12-27 03:02:55,305][105692] Updated weights for policy 0, policy_version 1599870 (0.0011) [2023-12-27 03:02:55,354][105692] Updated weights for policy 0, policy_version 1599880 (0.0010) [2023-12-27 03:02:55,487][105620] Updated weights for policy 1, policy_version 1603242 (0.0011) [2023-12-27 03:02:55,549][105620] Updated weights for policy 1, policy_version 1603252 (0.0011) [2023-12-27 03:02:55,616][105620] Updated weights for policy 1, policy_version 1603262 (0.0011) [2023-12-27 03:02:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19933.9, 300 sec: 19549.7). Total num frames: 820117504. Throughput: 0: 10033.4, 1: 9959.7. Samples: 820129748. Policy #0 lag: (min: 25.0, avg: 41.4, max: 57.0) [2023-12-27 03:02:56,063][104569] Avg episode reward: [(0, '8804.632'), (1, '8991.487')] [2023-12-27 03:02:56,093][105692] Updated weights for policy 0, policy_version 1599890 (0.0010) [2023-12-27 03:02:56,152][105692] Updated weights for policy 0, policy_version 1599900 (0.0010) [2023-12-27 03:02:56,200][105692] Updated weights for policy 0, policy_version 1599910 (0.0010) [2023-12-27 03:02:56,322][105620] Updated weights for policy 1, policy_version 1603272 (0.0007) [2023-12-27 03:02:56,377][105620] Updated weights for policy 1, policy_version 1603282 (0.0010) [2023-12-27 03:02:56,432][105620] Updated weights for policy 1, policy_version 1603292 (0.0010) [2023-12-27 03:02:56,969][105692] Updated weights for policy 0, policy_version 1599920 (0.0010) [2023-12-27 03:02:57,022][105692] Updated weights for policy 0, policy_version 1599930 (0.0011) [2023-12-27 03:02:57,054][105620] Updated weights for policy 1, policy_version 1603302 (0.0007) [2023-12-27 03:02:57,080][105692] Updated weights for policy 0, policy_version 1599940 (0.0007) [2023-12-27 03:02:57,111][105620] Updated weights for policy 1, policy_version 1603312 (0.0008) [2023-12-27 03:02:57,177][105620] Updated weights for policy 1, policy_version 1603322 (0.0010) [2023-12-27 03:02:57,634][105692] Updated weights for policy 0, policy_version 1599950 (0.0005) [2023-12-27 03:02:57,685][105692] Updated weights for policy 0, policy_version 1599960 (0.0005) [2023-12-27 03:02:57,742][105692] Updated weights for policy 0, policy_version 1599970 (0.0006) [2023-12-27 03:02:57,751][105620] Updated weights for policy 1, policy_version 1603332 (0.0010) [2023-12-27 03:02:57,806][105620] Updated weights for policy 1, policy_version 1603342 (0.0010) [2023-12-27 03:02:57,863][105620] Updated weights for policy 1, policy_version 1603352 (0.0010) [2023-12-27 03:02:58,324][105692] Updated weights for policy 0, policy_version 1599980 (0.0007) [2023-12-27 03:02:58,390][105692] Updated weights for policy 0, policy_version 1599990 (0.0007) [2023-12-27 03:02:58,451][105692] Updated weights for policy 0, policy_version 1600000 (0.0009) [2023-12-27 03:02:58,636][105620] Updated weights for policy 1, policy_version 1603362 (0.0010) [2023-12-27 03:02:58,705][105620] Updated weights for policy 1, policy_version 1603372 (0.0010) [2023-12-27 03:02:58,775][105620] Updated weights for policy 1, policy_version 1603382 (0.0010) [2023-12-27 03:02:58,846][105620] Updated weights for policy 1, policy_version 1603392 (0.0008) [2023-12-27 03:02:59,240][105692] Updated weights for policy 0, policy_version 1600010 (0.0010) [2023-12-27 03:02:59,307][105692] Updated weights for policy 0, policy_version 1600020 (0.0009) [2023-12-27 03:02:59,368][105692] Updated weights for policy 0, policy_version 1600030 (0.0008) [2023-12-27 03:02:59,440][105692] Updated weights for policy 0, policy_version 1600040 (0.0008) [2023-12-27 03:02:59,625][105620] Updated weights for policy 1, policy_version 1603402 (0.0010) [2023-12-27 03:02:59,686][105620] Updated weights for policy 1, policy_version 1603412 (0.0009) [2023-12-27 03:02:59,741][105620] Updated weights for policy 1, policy_version 1603422 (0.0006) [2023-12-27 03:03:00,154][105692] Updated weights for policy 0, policy_version 1600050 (0.0006) [2023-12-27 03:03:00,210][105692] Updated weights for policy 0, policy_version 1600060 (0.0010) [2023-12-27 03:03:00,272][105692] Updated weights for policy 0, policy_version 1600070 (0.0010) [2023-12-27 03:03:00,508][105620] Updated weights for policy 1, policy_version 1603432 (0.0010) [2023-12-27 03:03:00,566][105620] Updated weights for policy 1, policy_version 1603442 (0.0010) [2023-12-27 03:03:00,622][105620] Updated weights for policy 1, policy_version 1603452 (0.0010) [2023-12-27 03:03:00,969][105692] Updated weights for policy 0, policy_version 1600080 (0.0010) [2023-12-27 03:03:01,012][105692] Updated weights for policy 0, policy_version 1600090 (0.0010) [2023-12-27 03:03:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19933.9, 300 sec: 19549.7). Total num frames: 820215808. Throughput: 0: 10054.2, 1: 9914.9. Samples: 820191808. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:03:01,062][104569] Avg episode reward: [(0, '8806.016'), (1, '9090.888')] [2023-12-27 03:03:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001603456_410542080.pth... [2023-12-27 03:03:01,073][105692] Updated weights for policy 0, policy_version 1600100 (0.0010) [2023-12-27 03:03:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001602304_410247168.pth [2023-12-27 03:03:01,096][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001600104_409681920.pth... [2023-12-27 03:03:01,100][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001598920_409378816.pth [2023-12-27 03:03:01,317][105620] Updated weights for policy 1, policy_version 1603462 (0.0008) [2023-12-27 03:03:01,387][105620] Updated weights for policy 1, policy_version 1603472 (0.0008) [2023-12-27 03:03:01,444][105620] Updated weights for policy 1, policy_version 1603482 (0.0009) [2023-12-27 03:03:01,772][105692] Updated weights for policy 0, policy_version 1600110 (0.0011) [2023-12-27 03:03:01,833][105692] Updated weights for policy 0, policy_version 1600120 (0.0010) [2023-12-27 03:03:01,888][105692] Updated weights for policy 0, policy_version 1600130 (0.0010) [2023-12-27 03:03:02,198][105620] Updated weights for policy 1, policy_version 1603492 (0.0010) [2023-12-27 03:03:02,261][105620] Updated weights for policy 1, policy_version 1603502 (0.0010) [2023-12-27 03:03:02,314][105620] Updated weights for policy 1, policy_version 1603512 (0.0007) [2023-12-27 03:03:02,569][105692] Updated weights for policy 0, policy_version 1600140 (0.0011) [2023-12-27 03:03:02,637][105692] Updated weights for policy 0, policy_version 1600150 (0.0011) [2023-12-27 03:03:02,702][105692] Updated weights for policy 0, policy_version 1600160 (0.0011) [2023-12-27 03:03:02,918][105620] Updated weights for policy 1, policy_version 1603522 (0.0010) [2023-12-27 03:03:02,978][105620] Updated weights for policy 1, policy_version 1603532 (0.0009) [2023-12-27 03:03:03,037][105620] Updated weights for policy 1, policy_version 1603542 (0.0010) [2023-12-27 03:03:03,084][105620] Updated weights for policy 1, policy_version 1603552 (0.0009) [2023-12-27 03:03:03,290][105692] Updated weights for policy 0, policy_version 1600170 (0.0010) [2023-12-27 03:03:03,347][105692] Updated weights for policy 0, policy_version 1600180 (0.0006) [2023-12-27 03:03:03,411][105692] Updated weights for policy 0, policy_version 1600190 (0.0007) [2023-12-27 03:03:03,472][105692] Updated weights for policy 0, policy_version 1600200 (0.0008) [2023-12-27 03:03:03,740][105620] Updated weights for policy 1, policy_version 1603562 (0.0010) [2023-12-27 03:03:03,784][105620] Updated weights for policy 1, policy_version 1603572 (0.0010) [2023-12-27 03:03:03,848][105620] Updated weights for policy 1, policy_version 1603582 (0.0006) [2023-12-27 03:03:04,026][105692] Updated weights for policy 0, policy_version 1600210 (0.0009) [2023-12-27 03:03:04,081][105692] Updated weights for policy 0, policy_version 1600220 (0.0009) [2023-12-27 03:03:04,137][105692] Updated weights for policy 0, policy_version 1600230 (0.0009) [2023-12-27 03:03:04,527][105620] Updated weights for policy 1, policy_version 1603592 (0.0006) [2023-12-27 03:03:04,585][105620] Updated weights for policy 1, policy_version 1603602 (0.0006) [2023-12-27 03:03:04,641][105620] Updated weights for policy 1, policy_version 1603612 (0.0008) [2023-12-27 03:03:04,850][105692] Updated weights for policy 0, policy_version 1600240 (0.0008) [2023-12-27 03:03:04,899][105692] Updated weights for policy 0, policy_version 1600250 (0.0008) [2023-12-27 03:03:04,947][105692] Updated weights for policy 0, policy_version 1600260 (0.0008) [2023-12-27 03:03:05,369][105620] Updated weights for policy 1, policy_version 1603622 (0.0010) [2023-12-27 03:03:05,421][105620] Updated weights for policy 1, policy_version 1603632 (0.0010) [2023-12-27 03:03:05,472][105620] Updated weights for policy 1, policy_version 1603642 (0.0010) [2023-12-27 03:03:05,718][105692] Updated weights for policy 0, policy_version 1600270 (0.0008) [2023-12-27 03:03:05,769][105692] Updated weights for policy 0, policy_version 1600280 (0.0008) [2023-12-27 03:03:05,821][105692] Updated weights for policy 0, policy_version 1600290 (0.0008) [2023-12-27 03:03:06,062][104569] Fps is (10 sec: 20480.3, 60 sec: 20070.4, 300 sec: 19577.5). Total num frames: 820322304. Throughput: 0: 9925.0, 1: 9806.3. Samples: 820310936. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:03:06,062][104569] Avg episode reward: [(0, '8712.825'), (1, '9001.184')] [2023-12-27 03:03:06,229][105620] Updated weights for policy 1, policy_version 1603652 (0.0010) [2023-12-27 03:03:06,292][105620] Updated weights for policy 1, policy_version 1603662 (0.0010) [2023-12-27 03:03:06,354][105620] Updated weights for policy 1, policy_version 1603672 (0.0010) [2023-12-27 03:03:06,597][105692] Updated weights for policy 0, policy_version 1600300 (0.0008) [2023-12-27 03:03:06,658][105692] Updated weights for policy 0, policy_version 1600310 (0.0008) [2023-12-27 03:03:06,725][105692] Updated weights for policy 0, policy_version 1600320 (0.0008) [2023-12-27 03:03:07,096][105620] Updated weights for policy 1, policy_version 1603682 (0.0010) [2023-12-27 03:03:07,161][105620] Updated weights for policy 1, policy_version 1603692 (0.0010) [2023-12-27 03:03:07,224][105620] Updated weights for policy 1, policy_version 1603702 (0.0010) [2023-12-27 03:03:07,283][105620] Updated weights for policy 1, policy_version 1603712 (0.0010) [2023-12-27 03:03:07,489][105692] Updated weights for policy 0, policy_version 1600330 (0.0008) [2023-12-27 03:03:07,539][105692] Updated weights for policy 0, policy_version 1600340 (0.0008) [2023-12-27 03:03:07,593][105692] Updated weights for policy 0, policy_version 1600350 (0.0007) [2023-12-27 03:03:07,649][105692] Updated weights for policy 0, policy_version 1600360 (0.0009) [2023-12-27 03:03:08,010][105620] Updated weights for policy 1, policy_version 1603722 (0.0010) [2023-12-27 03:03:08,057][105620] Updated weights for policy 1, policy_version 1603732 (0.0010) [2023-12-27 03:03:08,102][105620] Updated weights for policy 1, policy_version 1603742 (0.0010) [2023-12-27 03:03:08,442][105692] Updated weights for policy 0, policy_version 1600370 (0.0008) [2023-12-27 03:03:08,492][105692] Updated weights for policy 0, policy_version 1600380 (0.0008) [2023-12-27 03:03:08,544][105692] Updated weights for policy 0, policy_version 1600390 (0.0008) [2023-12-27 03:03:08,879][105620] Updated weights for policy 1, policy_version 1603752 (0.0010) [2023-12-27 03:03:08,930][105620] Updated weights for policy 1, policy_version 1603762 (0.0010) [2023-12-27 03:03:08,987][105620] Updated weights for policy 1, policy_version 1603772 (0.0010) [2023-12-27 03:03:09,337][105692] Updated weights for policy 0, policy_version 1600400 (0.0009) [2023-12-27 03:03:09,406][105692] Updated weights for policy 0, policy_version 1600410 (0.0009) [2023-12-27 03:03:09,463][105692] Updated weights for policy 0, policy_version 1600420 (0.0008) [2023-12-27 03:03:09,775][105620] Updated weights for policy 1, policy_version 1603782 (0.0010) [2023-12-27 03:03:09,838][105620] Updated weights for policy 1, policy_version 1603792 (0.0011) [2023-12-27 03:03:09,905][105620] Updated weights for policy 1, policy_version 1603802 (0.0011) [2023-12-27 03:03:10,317][105692] Updated weights for policy 0, policy_version 1600430 (0.0008) [2023-12-27 03:03:10,369][105692] Updated weights for policy 0, policy_version 1600440 (0.0007) [2023-12-27 03:03:10,436][105692] Updated weights for policy 0, policy_version 1600450 (0.0008) [2023-12-27 03:03:10,670][105620] Updated weights for policy 1, policy_version 1603812 (0.0011) [2023-12-27 03:03:10,729][105620] Updated weights for policy 1, policy_version 1603822 (0.0010) [2023-12-27 03:03:10,788][105620] Updated weights for policy 1, policy_version 1603832 (0.0010) [2023-12-27 03:03:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 820412416. Throughput: 0: 9963.4, 1: 9716.1. Samples: 820421728. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:03:11,063][104569] Avg episode reward: [(0, '8530.831'), (1, '8902.440')] [2023-12-27 03:03:11,238][105692] Updated weights for policy 0, policy_version 1600460 (0.0008) [2023-12-27 03:03:11,297][105692] Updated weights for policy 0, policy_version 1600470 (0.0009) [2023-12-27 03:03:11,356][105692] Updated weights for policy 0, policy_version 1600480 (0.0008) [2023-12-27 03:03:11,543][105620] Updated weights for policy 1, policy_version 1603842 (0.0008) [2023-12-27 03:03:11,610][105620] Updated weights for policy 1, policy_version 1603852 (0.0011) [2023-12-27 03:03:11,679][105620] Updated weights for policy 1, policy_version 1603862 (0.0007) [2023-12-27 03:03:11,753][105620] Updated weights for policy 1, policy_version 1603872 (0.0008) [2023-12-27 03:03:12,143][105692] Updated weights for policy 0, policy_version 1600490 (0.0008) [2023-12-27 03:03:12,209][105692] Updated weights for policy 0, policy_version 1600500 (0.0010) [2023-12-27 03:03:12,268][105692] Updated weights for policy 0, policy_version 1600511 (0.0010) [2023-12-27 03:03:12,445][105620] Updated weights for policy 1, policy_version 1603882 (0.0009) [2023-12-27 03:03:12,502][105620] Updated weights for policy 1, policy_version 1603892 (0.0009) [2023-12-27 03:03:12,550][105620] Updated weights for policy 1, policy_version 1603902 (0.0009) [2023-12-27 03:03:13,039][105692] Updated weights for policy 0, policy_version 1600521 (0.0010) [2023-12-27 03:03:13,094][105692] Updated weights for policy 0, policy_version 1600531 (0.0008) [2023-12-27 03:03:13,150][105692] Updated weights for policy 0, policy_version 1600541 (0.0009) [2023-12-27 03:03:13,201][105692] Updated weights for policy 0, policy_version 1600551 (0.0006) [2023-12-27 03:03:13,298][105620] Updated weights for policy 1, policy_version 1603912 (0.0006) [2023-12-27 03:03:13,354][105620] Updated weights for policy 1, policy_version 1603922 (0.0007) [2023-12-27 03:03:13,404][105620] Updated weights for policy 1, policy_version 1603932 (0.0007) [2023-12-27 03:03:13,926][105692] Updated weights for policy 0, policy_version 1600561 (0.0006) [2023-12-27 03:03:13,977][105692] Updated weights for policy 0, policy_version 1600571 (0.0005) [2023-12-27 03:03:14,028][105692] Updated weights for policy 0, policy_version 1600581 (0.0005) [2023-12-27 03:03:14,136][105620] Updated weights for policy 1, policy_version 1603942 (0.0008) [2023-12-27 03:03:14,190][105620] Updated weights for policy 1, policy_version 1603952 (0.0009) [2023-12-27 03:03:14,237][105620] Updated weights for policy 1, policy_version 1603962 (0.0008) [2023-12-27 03:03:14,654][105692] Updated weights for policy 0, policy_version 1600591 (0.0008) [2023-12-27 03:03:14,712][105692] Updated weights for policy 0, policy_version 1600601 (0.0007) [2023-12-27 03:03:14,778][105692] Updated weights for policy 0, policy_version 1600611 (0.0006) [2023-12-27 03:03:15,064][105620] Updated weights for policy 1, policy_version 1603972 (0.0009) [2023-12-27 03:03:15,125][105620] Updated weights for policy 1, policy_version 1603982 (0.0009) [2023-12-27 03:03:15,189][105620] Updated weights for policy 1, policy_version 1603992 (0.0009) [2023-12-27 03:03:15,472][105692] Updated weights for policy 0, policy_version 1600621 (0.0008) [2023-12-27 03:03:15,541][105692] Updated weights for policy 0, policy_version 1600631 (0.0009) [2023-12-27 03:03:15,607][105692] Updated weights for policy 0, policy_version 1600641 (0.0010) [2023-12-27 03:03:15,913][105620] Updated weights for policy 1, policy_version 1604002 (0.0008) [2023-12-27 03:03:15,972][105620] Updated weights for policy 1, policy_version 1604012 (0.0008) [2023-12-27 03:03:16,021][105620] Updated weights for policy 1, policy_version 1604022 (0.0008) [2023-12-27 03:03:16,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 820502528. Throughput: 0: 9890.8, 1: 9717.5. Samples: 820477224. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:03:16,062][104569] Avg episode reward: [(0, '8713.473'), (1, '8993.133')] [2023-12-27 03:03:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001600648_409821184.pth... [2023-12-27 03:03:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001599496_409526272.pth [2023-12-27 03:03:16,084][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001604032_410689536.pth... [2023-12-27 03:03:16,085][105620] Updated weights for policy 1, policy_version 1604032 (0.0008) [2023-12-27 03:03:16,088][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001602880_410394624.pth [2023-12-27 03:03:16,400][105692] Updated weights for policy 0, policy_version 1600651 (0.0009) [2023-12-27 03:03:16,455][105692] Updated weights for policy 0, policy_version 1600661 (0.0005) [2023-12-27 03:03:16,507][105692] Updated weights for policy 0, policy_version 1600671 (0.0007) [2023-12-27 03:03:16,853][105620] Updated weights for policy 1, policy_version 1604042 (0.0010) [2023-12-27 03:03:16,924][105620] Updated weights for policy 1, policy_version 1604052 (0.0010) [2023-12-27 03:03:16,994][105620] Updated weights for policy 1, policy_version 1604062 (0.0009) [2023-12-27 03:03:17,039][105692] Updated weights for policy 0, policy_version 1600681 (0.0007) [2023-12-27 03:03:17,094][105692] Updated weights for policy 0, policy_version 1600691 (0.0006) [2023-12-27 03:03:17,160][105692] Updated weights for policy 0, policy_version 1600701 (0.0005) [2023-12-27 03:03:17,223][105692] Updated weights for policy 0, policy_version 1600711 (0.0009) [2023-12-27 03:03:17,745][105692] Updated weights for policy 0, policy_version 1600721 (0.0009) [2023-12-27 03:03:17,803][105692] Updated weights for policy 0, policy_version 1600731 (0.0010) [2023-12-27 03:03:17,851][105620] Updated weights for policy 1, policy_version 1604072 (0.0008) [2023-12-27 03:03:17,869][105692] Updated weights for policy 0, policy_version 1600741 (0.0007) [2023-12-27 03:03:17,902][105620] Updated weights for policy 1, policy_version 1604082 (0.0006) [2023-12-27 03:03:17,964][105620] Updated weights for policy 1, policy_version 1604092 (0.0010) [2023-12-27 03:03:18,506][105692] Updated weights for policy 0, policy_version 1600751 (0.0009) [2023-12-27 03:03:18,562][105692] Updated weights for policy 0, policy_version 1600761 (0.0011) [2023-12-27 03:03:18,624][105692] Updated weights for policy 0, policy_version 1600771 (0.0011) [2023-12-27 03:03:18,743][105620] Updated weights for policy 1, policy_version 1604102 (0.0007) [2023-12-27 03:03:18,806][105620] Updated weights for policy 1, policy_version 1604112 (0.0008) [2023-12-27 03:03:18,869][105620] Updated weights for policy 1, policy_version 1604122 (0.0008) [2023-12-27 03:03:19,379][105692] Updated weights for policy 0, policy_version 1600781 (0.0011) [2023-12-27 03:03:19,437][105692] Updated weights for policy 0, policy_version 1600791 (0.0010) [2023-12-27 03:03:19,496][105692] Updated weights for policy 0, policy_version 1600801 (0.0010) [2023-12-27 03:03:19,605][105620] Updated weights for policy 1, policy_version 1604132 (0.0007) [2023-12-27 03:03:19,672][105620] Updated weights for policy 1, policy_version 1604142 (0.0008) [2023-12-27 03:03:19,731][105620] Updated weights for policy 1, policy_version 1604152 (0.0009) [2023-12-27 03:03:20,231][105692] Updated weights for policy 0, policy_version 1600811 (0.0009) [2023-12-27 03:03:20,276][105692] Updated weights for policy 0, policy_version 1600821 (0.0010) [2023-12-27 03:03:20,325][105692] Updated weights for policy 0, policy_version 1600831 (0.0010) [2023-12-27 03:03:20,484][105620] Updated weights for policy 1, policy_version 1604162 (0.0010) [2023-12-27 03:03:20,552][105620] Updated weights for policy 1, policy_version 1604172 (0.0008) [2023-12-27 03:03:20,619][105620] Updated weights for policy 1, policy_version 1604182 (0.0008) [2023-12-27 03:03:20,674][105620] Updated weights for policy 1, policy_version 1604192 (0.0010) [2023-12-27 03:03:21,005][105692] Updated weights for policy 0, policy_version 1600841 (0.0011) [2023-12-27 03:03:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 820600832. Throughput: 0: 9862.1, 1: 9623.8. Samples: 820592844. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:03:21,062][104569] Avg episode reward: [(0, '8620.279'), (1, '8990.370')] [2023-12-27 03:03:21,068][105692] Updated weights for policy 0, policy_version 1600851 (0.0011) [2023-12-27 03:03:21,128][105692] Updated weights for policy 0, policy_version 1600861 (0.0010) [2023-12-27 03:03:21,191][105692] Updated weights for policy 0, policy_version 1600871 (0.0007) [2023-12-27 03:03:21,528][105620] Updated weights for policy 1, policy_version 1604202 (0.0009) [2023-12-27 03:03:21,583][105620] Updated weights for policy 1, policy_version 1604212 (0.0008) [2023-12-27 03:03:21,645][105620] Updated weights for policy 1, policy_version 1604222 (0.0010) [2023-12-27 03:03:21,933][105692] Updated weights for policy 0, policy_version 1600881 (0.0010) [2023-12-27 03:03:21,996][105692] Updated weights for policy 0, policy_version 1600891 (0.0011) [2023-12-27 03:03:22,060][105692] Updated weights for policy 0, policy_version 1600901 (0.0011) [2023-12-27 03:03:22,417][105620] Updated weights for policy 1, policy_version 1604232 (0.0009) [2023-12-27 03:03:22,478][105620] Updated weights for policy 1, policy_version 1604242 (0.0008) [2023-12-27 03:03:22,538][105620] Updated weights for policy 1, policy_version 1604252 (0.0008) [2023-12-27 03:03:22,800][105692] Updated weights for policy 0, policy_version 1600911 (0.0011) [2023-12-27 03:03:22,849][105692] Updated weights for policy 0, policy_version 1600921 (0.0010) [2023-12-27 03:03:22,898][105692] Updated weights for policy 0, policy_version 1600931 (0.0010) [2023-12-27 03:03:23,220][105620] Updated weights for policy 1, policy_version 1604262 (0.0008) [2023-12-27 03:03:23,272][105620] Updated weights for policy 1, policy_version 1604272 (0.0008) [2023-12-27 03:03:23,317][105620] Updated weights for policy 1, policy_version 1604282 (0.0008) [2023-12-27 03:03:23,629][105692] Updated weights for policy 0, policy_version 1600941 (0.0009) [2023-12-27 03:03:23,680][105692] Updated weights for policy 0, policy_version 1600951 (0.0008) [2023-12-27 03:03:23,723][105692] Updated weights for policy 0, policy_version 1600961 (0.0005) [2023-12-27 03:03:24,041][105620] Updated weights for policy 1, policy_version 1604292 (0.0008) [2023-12-27 03:03:24,094][105620] Updated weights for policy 1, policy_version 1604302 (0.0009) [2023-12-27 03:03:24,148][105620] Updated weights for policy 1, policy_version 1604312 (0.0010) [2023-12-27 03:03:24,309][105692] Updated weights for policy 0, policy_version 1600971 (0.0005) [2023-12-27 03:03:24,361][105692] Updated weights for policy 0, policy_version 1600981 (0.0005) [2023-12-27 03:03:24,429][105692] Updated weights for policy 0, policy_version 1600991 (0.0005) [2023-12-27 03:03:24,933][105620] Updated weights for policy 1, policy_version 1604322 (0.0009) [2023-12-27 03:03:24,960][105692] Updated weights for policy 0, policy_version 1601001 (0.0006) [2023-12-27 03:03:24,981][105620] Updated weights for policy 1, policy_version 1604332 (0.0005) [2023-12-27 03:03:25,004][105692] Updated weights for policy 0, policy_version 1601011 (0.0010) [2023-12-27 03:03:25,032][105620] Updated weights for policy 1, policy_version 1604342 (0.0005) [2023-12-27 03:03:25,049][105692] Updated weights for policy 0, policy_version 1601021 (0.0010) [2023-12-27 03:03:25,082][105620] Updated weights for policy 1, policy_version 1604352 (0.0005) [2023-12-27 03:03:25,745][105692] Updated weights for policy 0, policy_version 1601033 (0.0010) [2023-12-27 03:03:25,806][105692] Updated weights for policy 0, policy_version 1601043 (0.0009) [2023-12-27 03:03:25,868][105692] Updated weights for policy 0, policy_version 1601053 (0.0009) [2023-12-27 03:03:25,879][105620] Updated weights for policy 1, policy_version 1604362 (0.0009) [2023-12-27 03:03:25,935][105692] Updated weights for policy 0, policy_version 1601063 (0.0006) [2023-12-27 03:03:25,945][105620] Updated weights for policy 1, policy_version 1604372 (0.0009) [2023-12-27 03:03:25,996][105620] Updated weights for policy 1, policy_version 1604382 (0.0009) [2023-12-27 03:03:26,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 820707328. Throughput: 0: 9909.8, 1: 9499.1. Samples: 820709680. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:03:26,062][104569] Avg episode reward: [(0, '8255.528'), (1, '9081.543')] [2023-12-27 03:03:26,600][105692] Updated weights for policy 0, policy_version 1601073 (0.0008) [2023-12-27 03:03:26,651][105692] Updated weights for policy 0, policy_version 1601083 (0.0009) [2023-12-27 03:03:26,707][105692] Updated weights for policy 0, policy_version 1601093 (0.0006) [2023-12-27 03:03:26,797][105620] Updated weights for policy 1, policy_version 1604392 (0.0010) [2023-12-27 03:03:26,854][105620] Updated weights for policy 1, policy_version 1604402 (0.0010) [2023-12-27 03:03:26,908][105620] Updated weights for policy 1, policy_version 1604412 (0.0010) [2023-12-27 03:03:27,323][105692] Updated weights for policy 0, policy_version 1601103 (0.0009) [2023-12-27 03:03:27,381][105692] Updated weights for policy 0, policy_version 1601113 (0.0010) [2023-12-27 03:03:27,425][105692] Updated weights for policy 0, policy_version 1601123 (0.0005) [2023-12-27 03:03:27,608][105620] Updated weights for policy 1, policy_version 1604422 (0.0011) [2023-12-27 03:03:27,662][105620] Updated weights for policy 1, policy_version 1604433 (0.0010) [2023-12-27 03:03:27,714][105620] Updated weights for policy 1, policy_version 1604444 (0.0007) [2023-12-27 03:03:28,101][105692] Updated weights for policy 0, policy_version 1601133 (0.0006) [2023-12-27 03:03:28,146][105692] Updated weights for policy 0, policy_version 1601143 (0.0008) [2023-12-27 03:03:28,190][105692] Updated weights for policy 0, policy_version 1601153 (0.0008) [2023-12-27 03:03:28,368][105620] Updated weights for policy 1, policy_version 1604454 (0.0010) [2023-12-27 03:03:28,420][105620] Updated weights for policy 1, policy_version 1604464 (0.0010) [2023-12-27 03:03:28,475][105620] Updated weights for policy 1, policy_version 1604474 (0.0010) [2023-12-27 03:03:29,005][105692] Updated weights for policy 0, policy_version 1601163 (0.0007) [2023-12-27 03:03:29,054][105692] Updated weights for policy 0, policy_version 1601173 (0.0005) [2023-12-27 03:03:29,101][105692] Updated weights for policy 0, policy_version 1601183 (0.0005) [2023-12-27 03:03:29,115][105620] Updated weights for policy 1, policy_version 1604484 (0.0007) [2023-12-27 03:03:29,170][105620] Updated weights for policy 1, policy_version 1604494 (0.0010) [2023-12-27 03:03:29,233][105620] Updated weights for policy 1, policy_version 1604504 (0.0010) [2023-12-27 03:03:29,831][105692] Updated weights for policy 0, policy_version 1601193 (0.0006) [2023-12-27 03:03:29,887][105692] Updated weights for policy 0, policy_version 1601203 (0.0012) [2023-12-27 03:03:29,948][105692] Updated weights for policy 0, policy_version 1601213 (0.0010) [2023-12-27 03:03:29,960][105620] Updated weights for policy 1, policy_version 1604514 (0.0006) [2023-12-27 03:03:29,997][105692] Updated weights for policy 0, policy_version 1601223 (0.0010) [2023-12-27 03:03:30,021][105620] Updated weights for policy 1, policy_version 1604524 (0.0006) [2023-12-27 03:03:30,078][105620] Updated weights for policy 1, policy_version 1604534 (0.0008) [2023-12-27 03:03:30,130][105620] Updated weights for policy 1, policy_version 1604544 (0.0008) [2023-12-27 03:03:30,744][105620] Updated weights for policy 1, policy_version 1604554 (0.0007) [2023-12-27 03:03:30,745][105692] Updated weights for policy 0, policy_version 1601233 (0.0007) [2023-12-27 03:03:30,803][105692] Updated weights for policy 0, policy_version 1601243 (0.0005) [2023-12-27 03:03:30,804][105620] Updated weights for policy 1, policy_version 1604564 (0.0007) [2023-12-27 03:03:30,866][105692] Updated weights for policy 0, policy_version 1601253 (0.0005) [2023-12-27 03:03:30,866][105620] Updated weights for policy 1, policy_version 1604574 (0.0006) [2023-12-27 03:03:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 820805632. Throughput: 0: 9994.0, 1: 9524.1. Samples: 820770368. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:03:31,063][104569] Avg episode reward: [(0, '8438.238'), (1, '9082.423')] [2023-12-27 03:03:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001601256_409976832.pth... [2023-12-27 03:03:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001604576_410828800.pth... [2023-12-27 03:03:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001600104_409681920.pth [2023-12-27 03:03:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001603456_410542080.pth [2023-12-27 03:03:31,448][105692] Updated weights for policy 0, policy_version 1601263 (0.0008) [2023-12-27 03:03:31,504][105692] Updated weights for policy 0, policy_version 1601273 (0.0010) [2023-12-27 03:03:31,518][105620] Updated weights for policy 1, policy_version 1604584 (0.0006) [2023-12-27 03:03:31,560][105692] Updated weights for policy 0, policy_version 1601283 (0.0010) [2023-12-27 03:03:31,574][105620] Updated weights for policy 1, policy_version 1604594 (0.0006) [2023-12-27 03:03:31,634][105620] Updated weights for policy 1, policy_version 1604604 (0.0008) [2023-12-27 03:03:32,208][105692] Updated weights for policy 0, policy_version 1601293 (0.0008) [2023-12-27 03:03:32,270][105692] Updated weights for policy 0, policy_version 1601303 (0.0006) [2023-12-27 03:03:32,334][105692] Updated weights for policy 0, policy_version 1601313 (0.0011) [2023-12-27 03:03:32,381][105620] Updated weights for policy 1, policy_version 1604614 (0.0009) [2023-12-27 03:03:32,452][105620] Updated weights for policy 1, policy_version 1604624 (0.0009) [2023-12-27 03:03:32,523][105620] Updated weights for policy 1, policy_version 1604634 (0.0009) [2023-12-27 03:03:32,934][105692] Updated weights for policy 0, policy_version 1601323 (0.0011) [2023-12-27 03:03:32,986][105692] Updated weights for policy 0, policy_version 1601333 (0.0007) [2023-12-27 03:03:33,042][105692] Updated weights for policy 0, policy_version 1601343 (0.0005) [2023-12-27 03:03:33,276][105620] Updated weights for policy 1, policy_version 1604644 (0.0008) [2023-12-27 03:03:33,327][105620] Updated weights for policy 1, policy_version 1604654 (0.0005) [2023-12-27 03:03:33,372][105620] Updated weights for policy 1, policy_version 1604664 (0.0005) [2023-12-27 03:03:33,746][105692] Updated weights for policy 0, policy_version 1601353 (0.0006) [2023-12-27 03:03:33,807][105692] Updated weights for policy 0, policy_version 1601363 (0.0010) [2023-12-27 03:03:33,864][105692] Updated weights for policy 0, policy_version 1601373 (0.0010) [2023-12-27 03:03:33,926][105692] Updated weights for policy 0, policy_version 1601383 (0.0011) [2023-12-27 03:03:34,016][105620] Updated weights for policy 1, policy_version 1604674 (0.0008) [2023-12-27 03:03:34,077][105620] Updated weights for policy 1, policy_version 1604684 (0.0005) [2023-12-27 03:03:34,129][105620] Updated weights for policy 1, policy_version 1604694 (0.0007) [2023-12-27 03:03:34,191][105620] Updated weights for policy 1, policy_version 1604704 (0.0007) [2023-12-27 03:03:34,674][105692] Updated weights for policy 0, policy_version 1601393 (0.0009) [2023-12-27 03:03:34,723][105692] Updated weights for policy 0, policy_version 1601403 (0.0008) [2023-12-27 03:03:34,772][105692] Updated weights for policy 0, policy_version 1601413 (0.0008) [2023-12-27 03:03:34,877][105620] Updated weights for policy 1, policy_version 1604714 (0.0010) [2023-12-27 03:03:34,929][105620] Updated weights for policy 1, policy_version 1604724 (0.0010) [2023-12-27 03:03:34,977][105620] Updated weights for policy 1, policy_version 1604734 (0.0010) [2023-12-27 03:03:35,534][105692] Updated weights for policy 0, policy_version 1601423 (0.0007) [2023-12-27 03:03:35,586][105692] Updated weights for policy 0, policy_version 1601433 (0.0007) [2023-12-27 03:03:35,641][105692] Updated weights for policy 0, policy_version 1601443 (0.0006) [2023-12-27 03:03:35,642][105620] Updated weights for policy 1, policy_version 1604744 (0.0010) [2023-12-27 03:03:35,707][105620] Updated weights for policy 1, policy_version 1604754 (0.0010) [2023-12-27 03:03:35,772][105620] Updated weights for policy 1, policy_version 1604764 (0.0010) [2023-12-27 03:03:36,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19660.8, 300 sec: 19605.2). Total num frames: 820903936. Throughput: 0: 9961.0, 1: 9618.6. Samples: 820891804. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:03:36,063][104569] Avg episode reward: [(0, '8442.521'), (1, '9082.083')] [2023-12-27 03:03:36,249][105692] Updated weights for policy 0, policy_version 1601453 (0.0007) [2023-12-27 03:03:36,295][105692] Updated weights for policy 0, policy_version 1601463 (0.0008) [2023-12-27 03:03:36,347][105692] Updated weights for policy 0, policy_version 1601473 (0.0009) [2023-12-27 03:03:36,514][105620] Updated weights for policy 1, policy_version 1604774 (0.0011) [2023-12-27 03:03:36,583][105620] Updated weights for policy 1, policy_version 1604784 (0.0010) [2023-12-27 03:03:36,642][105620] Updated weights for policy 1, policy_version 1604794 (0.0010) [2023-12-27 03:03:37,155][105692] Updated weights for policy 0, policy_version 1601483 (0.0009) [2023-12-27 03:03:37,210][105692] Updated weights for policy 0, policy_version 1601493 (0.0006) [2023-12-27 03:03:37,255][105692] Updated weights for policy 0, policy_version 1601503 (0.0005) [2023-12-27 03:03:37,356][105620] Updated weights for policy 1, policy_version 1604804 (0.0009) [2023-12-27 03:03:37,403][105620] Updated weights for policy 1, policy_version 1604814 (0.0010) [2023-12-27 03:03:37,455][105620] Updated weights for policy 1, policy_version 1604824 (0.0010) [2023-12-27 03:03:37,847][105692] Updated weights for policy 0, policy_version 1601513 (0.0006) [2023-12-27 03:03:37,902][105692] Updated weights for policy 0, policy_version 1601523 (0.0006) [2023-12-27 03:03:37,950][105692] Updated weights for policy 0, policy_version 1601533 (0.0005) [2023-12-27 03:03:37,995][105692] Updated weights for policy 0, policy_version 1601543 (0.0005) [2023-12-27 03:03:38,213][105620] Updated weights for policy 1, policy_version 1604834 (0.0011) [2023-12-27 03:03:38,274][105620] Updated weights for policy 1, policy_version 1604844 (0.0010) [2023-12-27 03:03:38,330][105620] Updated weights for policy 1, policy_version 1604854 (0.0010) [2023-12-27 03:03:38,384][105620] Updated weights for policy 1, policy_version 1604864 (0.0008) [2023-12-27 03:03:38,719][105692] Updated weights for policy 0, policy_version 1601553 (0.0009) [2023-12-27 03:03:38,785][105692] Updated weights for policy 0, policy_version 1601563 (0.0009) [2023-12-27 03:03:38,850][105692] Updated weights for policy 0, policy_version 1601573 (0.0007) [2023-12-27 03:03:39,130][105620] Updated weights for policy 1, policy_version 1604874 (0.0008) [2023-12-27 03:03:39,189][105620] Updated weights for policy 1, policy_version 1604884 (0.0010) [2023-12-27 03:03:39,258][105620] Updated weights for policy 1, policy_version 1604894 (0.0010) [2023-12-27 03:03:39,578][105692] Updated weights for policy 0, policy_version 1601583 (0.0008) [2023-12-27 03:03:39,634][105692] Updated weights for policy 0, policy_version 1601593 (0.0008) [2023-12-27 03:03:39,682][105692] Updated weights for policy 0, policy_version 1601603 (0.0008) [2023-12-27 03:03:40,032][105620] Updated weights for policy 1, policy_version 1604904 (0.0011) [2023-12-27 03:03:40,094][105620] Updated weights for policy 1, policy_version 1604914 (0.0011) [2023-12-27 03:03:40,147][105620] Updated weights for policy 1, policy_version 1604924 (0.0010) [2023-12-27 03:03:40,496][105692] Updated weights for policy 0, policy_version 1601613 (0.0008) [2023-12-27 03:03:40,554][105692] Updated weights for policy 0, policy_version 1601623 (0.0008) [2023-12-27 03:03:40,599][105692] Updated weights for policy 0, policy_version 1601633 (0.0008) [2023-12-27 03:03:40,920][105620] Updated weights for policy 1, policy_version 1604934 (0.0010) [2023-12-27 03:03:40,981][105620] Updated weights for policy 1, policy_version 1604944 (0.0011) [2023-12-27 03:03:41,049][105620] Updated weights for policy 1, policy_version 1604954 (0.0011) [2023-12-27 03:03:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 820994048. Throughput: 0: 9936.6, 1: 9558.3. Samples: 821007016. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:03:41,062][104569] Avg episode reward: [(0, '7985.279'), (1, '9263.430')] [2023-12-27 03:03:41,415][105692] Updated weights for policy 0, policy_version 1601643 (0.0009) [2023-12-27 03:03:41,469][105692] Updated weights for policy 0, policy_version 1601653 (0.0009) [2023-12-27 03:03:41,523][105692] Updated weights for policy 0, policy_version 1601663 (0.0009) [2023-12-27 03:03:41,807][105620] Updated weights for policy 1, policy_version 1604964 (0.0010) [2023-12-27 03:03:41,875][105620] Updated weights for policy 1, policy_version 1604974 (0.0009) [2023-12-27 03:03:41,938][105620] Updated weights for policy 1, policy_version 1604984 (0.0009) [2023-12-27 03:03:42,335][105692] Updated weights for policy 0, policy_version 1601674 (0.0010) [2023-12-27 03:03:42,399][105692] Updated weights for policy 0, policy_version 1601684 (0.0009) [2023-12-27 03:03:42,454][105692] Updated weights for policy 0, policy_version 1601695 (0.0011) [2023-12-27 03:03:42,629][105620] Updated weights for policy 1, policy_version 1604994 (0.0008) [2023-12-27 03:03:42,686][105620] Updated weights for policy 1, policy_version 1605004 (0.0007) [2023-12-27 03:03:42,747][105620] Updated weights for policy 1, policy_version 1605014 (0.0010) [2023-12-27 03:03:42,807][105620] Updated weights for policy 1, policy_version 1605024 (0.0008) [2023-12-27 03:03:43,367][105692] Updated weights for policy 0, policy_version 1601706 (0.0010) [2023-12-27 03:03:43,412][105620] Updated weights for policy 1, policy_version 1605034 (0.0009) [2023-12-27 03:03:43,420][105692] Updated weights for policy 0, policy_version 1601716 (0.0007) [2023-12-27 03:03:43,464][105620] Updated weights for policy 1, policy_version 1605044 (0.0005) [2023-12-27 03:03:43,468][105692] Updated weights for policy 0, policy_version 1601726 (0.0009) [2023-12-27 03:03:43,516][105692] Updated weights for policy 0, policy_version 1601736 (0.0009) [2023-12-27 03:03:43,522][105620] Updated weights for policy 1, policy_version 1605054 (0.0009) [2023-12-27 03:03:44,258][105620] Updated weights for policy 1, policy_version 1605064 (0.0008) [2023-12-27 03:03:44,312][105620] Updated weights for policy 1, policy_version 1605074 (0.0010) [2023-12-27 03:03:44,318][105692] Updated weights for policy 0, policy_version 1601746 (0.0005) [2023-12-27 03:03:44,367][105620] Updated weights for policy 1, policy_version 1605084 (0.0010) [2023-12-27 03:03:44,373][105692] Updated weights for policy 0, policy_version 1601756 (0.0007) [2023-12-27 03:03:44,427][105692] Updated weights for policy 0, policy_version 1601766 (0.0008) [2023-12-27 03:03:45,110][105620] Updated weights for policy 1, policy_version 1605094 (0.0009) [2023-12-27 03:03:45,168][105692] Updated weights for policy 0, policy_version 1601776 (0.0006) [2023-12-27 03:03:45,174][105620] Updated weights for policy 1, policy_version 1605104 (0.0008) [2023-12-27 03:03:45,223][105692] Updated weights for policy 0, policy_version 1601786 (0.0006) [2023-12-27 03:03:45,236][105620] Updated weights for policy 1, policy_version 1605114 (0.0008) [2023-12-27 03:03:45,276][105692] Updated weights for policy 0, policy_version 1601796 (0.0005) [2023-12-27 03:03:45,833][105620] Updated weights for policy 1, policy_version 1605124 (0.0009) [2023-12-27 03:03:45,880][105620] Updated weights for policy 1, policy_version 1605134 (0.0010) [2023-12-27 03:03:45,925][105620] Updated weights for policy 1, policy_version 1605144 (0.0010) [2023-12-27 03:03:46,061][105692] Updated weights for policy 0, policy_version 1601806 (0.0007) [2023-12-27 03:03:46,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.4, 300 sec: 19549.7). Total num frames: 821092352. Throughput: 0: 9804.1, 1: 9531.6. Samples: 821061912. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:03:46,062][104569] Avg episode reward: [(0, '8076.694'), (1, '9080.781')] [2023-12-27 03:03:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001605152_410976256.pth... [2023-12-27 03:03:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001604032_410689536.pth [2023-12-27 03:03:46,112][105692] Updated weights for policy 0, policy_version 1601816 (0.0008) [2023-12-27 03:03:46,163][105692] Updated weights for policy 0, policy_version 1601826 (0.0009) [2023-12-27 03:03:46,192][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001601832_410124288.pth... [2023-12-27 03:03:46,195][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001600648_409821184.pth [2023-12-27 03:03:46,607][105620] Updated weights for policy 1, policy_version 1605154 (0.0009) [2023-12-27 03:03:46,667][105620] Updated weights for policy 1, policy_version 1605164 (0.0005) [2023-12-27 03:03:46,720][105620] Updated weights for policy 1, policy_version 1605174 (0.0005) [2023-12-27 03:03:46,766][105620] Updated weights for policy 1, policy_version 1605184 (0.0006) [2023-12-27 03:03:47,010][105692] Updated weights for policy 0, policy_version 1601836 (0.0009) [2023-12-27 03:03:47,064][105692] Updated weights for policy 0, policy_version 1601846 (0.0010) [2023-12-27 03:03:47,118][105692] Updated weights for policy 0, policy_version 1601857 (0.0011) [2023-12-27 03:03:47,311][105620] Updated weights for policy 1, policy_version 1605194 (0.0005) [2023-12-27 03:03:47,361][105620] Updated weights for policy 1, policy_version 1605204 (0.0005) [2023-12-27 03:03:47,409][105620] Updated weights for policy 1, policy_version 1605214 (0.0005) [2023-12-27 03:03:47,944][105620] Updated weights for policy 1, policy_version 1605224 (0.0008) [2023-12-27 03:03:48,005][105620] Updated weights for policy 1, policy_version 1605234 (0.0009) [2023-12-27 03:03:48,037][105692] Updated weights for policy 0, policy_version 1601868 (0.0009) [2023-12-27 03:03:48,063][105620] Updated weights for policy 1, policy_version 1605244 (0.0009) [2023-12-27 03:03:48,089][105692] Updated weights for policy 0, policy_version 1601878 (0.0006) [2023-12-27 03:03:48,144][105692] Updated weights for policy 0, policy_version 1601888 (0.0009) [2023-12-27 03:03:48,795][105692] Updated weights for policy 0, policy_version 1601898 (0.0008) [2023-12-27 03:03:48,843][105692] Updated weights for policy 0, policy_version 1601908 (0.0009) [2023-12-27 03:03:48,883][105620] Updated weights for policy 1, policy_version 1605254 (0.0007) [2023-12-27 03:03:48,890][105692] Updated weights for policy 0, policy_version 1601918 (0.0007) [2023-12-27 03:03:48,941][105692] Updated weights for policy 0, policy_version 1601928 (0.0008) [2023-12-27 03:03:48,941][105620] Updated weights for policy 1, policy_version 1605264 (0.0009) [2023-12-27 03:03:49,001][105620] Updated weights for policy 1, policy_version 1605274 (0.0009) [2023-12-27 03:03:49,597][105692] Updated weights for policy 0, policy_version 1601938 (0.0005) [2023-12-27 03:03:49,655][105692] Updated weights for policy 0, policy_version 1601948 (0.0005) [2023-12-27 03:03:49,714][105692] Updated weights for policy 0, policy_version 1601958 (0.0006) [2023-12-27 03:03:49,848][105620] Updated weights for policy 1, policy_version 1605284 (0.0008) [2023-12-27 03:03:49,907][105620] Updated weights for policy 1, policy_version 1605294 (0.0008) [2023-12-27 03:03:49,974][105620] Updated weights for policy 1, policy_version 1605304 (0.0010) [2023-12-27 03:03:50,405][105692] Updated weights for policy 0, policy_version 1601968 (0.0008) [2023-12-27 03:03:50,468][105692] Updated weights for policy 0, policy_version 1601978 (0.0009) [2023-12-27 03:03:50,532][105692] Updated weights for policy 0, policy_version 1601988 (0.0008) [2023-12-27 03:03:50,760][105620] Updated weights for policy 1, policy_version 1605314 (0.0009) [2023-12-27 03:03:50,825][105620] Updated weights for policy 1, policy_version 1605324 (0.0010) [2023-12-27 03:03:50,885][105620] Updated weights for policy 1, policy_version 1605334 (0.0011) [2023-12-27 03:03:50,938][105620] Updated weights for policy 1, policy_version 1605344 (0.0010) [2023-12-27 03:03:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 821190656. Throughput: 0: 9717.3, 1: 9568.3. Samples: 821178788. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:03:51,062][104569] Avg episode reward: [(0, '8169.646'), (1, '9172.994')] [2023-12-27 03:03:51,291][105692] Updated weights for policy 0, policy_version 1601998 (0.0009) [2023-12-27 03:03:51,351][105692] Updated weights for policy 0, policy_version 1602008 (0.0008) [2023-12-27 03:03:51,416][105692] Updated weights for policy 0, policy_version 1602018 (0.0008) [2023-12-27 03:03:51,705][105620] Updated weights for policy 1, policy_version 1605354 (0.0006) [2023-12-27 03:03:51,770][105620] Updated weights for policy 1, policy_version 1605364 (0.0011) [2023-12-27 03:03:51,829][105620] Updated weights for policy 1, policy_version 1605374 (0.0011) [2023-12-27 03:03:52,136][105692] Updated weights for policy 0, policy_version 1602028 (0.0009) [2023-12-27 03:03:52,198][105692] Updated weights for policy 0, policy_version 1602038 (0.0007) [2023-12-27 03:03:52,258][105692] Updated weights for policy 0, policy_version 1602048 (0.0007) [2023-12-27 03:03:52,527][105620] Updated weights for policy 1, policy_version 1605384 (0.0009) [2023-12-27 03:03:52,578][105620] Updated weights for policy 1, policy_version 1605394 (0.0008) [2023-12-27 03:03:52,631][105620] Updated weights for policy 1, policy_version 1605404 (0.0008) [2023-12-27 03:03:52,975][105692] Updated weights for policy 0, policy_version 1602058 (0.0007) [2023-12-27 03:03:53,024][105692] Updated weights for policy 0, policy_version 1602068 (0.0010) [2023-12-27 03:03:53,080][105692] Updated weights for policy 0, policy_version 1602078 (0.0010) [2023-12-27 03:03:53,386][105620] Updated weights for policy 1, policy_version 1605414 (0.0007) [2023-12-27 03:03:53,436][105620] Updated weights for policy 1, policy_version 1605424 (0.0006) [2023-12-27 03:03:53,490][105620] Updated weights for policy 1, policy_version 1605434 (0.0005) [2023-12-27 03:03:53,786][105692] Updated weights for policy 0, policy_version 1602089 (0.0008) [2023-12-27 03:03:53,838][105692] Updated weights for policy 0, policy_version 1602099 (0.0005) [2023-12-27 03:03:53,890][105692] Updated weights for policy 0, policy_version 1602109 (0.0005) [2023-12-27 03:03:53,943][105692] Updated weights for policy 0, policy_version 1602119 (0.0005) [2023-12-27 03:03:54,156][105620] Updated weights for policy 1, policy_version 1605444 (0.0006) [2023-12-27 03:03:54,217][105620] Updated weights for policy 1, policy_version 1605454 (0.0009) [2023-12-27 03:03:54,277][105620] Updated weights for policy 1, policy_version 1605464 (0.0010) [2023-12-27 03:03:54,515][105692] Updated weights for policy 0, policy_version 1602129 (0.0010) [2023-12-27 03:03:54,571][105692] Updated weights for policy 0, policy_version 1602139 (0.0009) [2023-12-27 03:03:54,635][105692] Updated weights for policy 0, policy_version 1602149 (0.0009) [2023-12-27 03:03:55,019][105620] Updated weights for policy 1, policy_version 1605474 (0.0010) [2023-12-27 03:03:55,080][105620] Updated weights for policy 1, policy_version 1605484 (0.0009) [2023-12-27 03:03:55,145][105620] Updated weights for policy 1, policy_version 1605494 (0.0009) [2023-12-27 03:03:55,204][105620] Updated weights for policy 1, policy_version 1605504 (0.0010) [2023-12-27 03:03:55,400][105692] Updated weights for policy 0, policy_version 1602159 (0.0009) [2023-12-27 03:03:55,450][105692] Updated weights for policy 0, policy_version 1602169 (0.0009) [2023-12-27 03:03:55,503][105692] Updated weights for policy 0, policy_version 1602179 (0.0010) [2023-12-27 03:03:55,900][105620] Updated weights for policy 1, policy_version 1605514 (0.0005) [2023-12-27 03:03:55,949][105620] Updated weights for policy 1, policy_version 1605524 (0.0007) [2023-12-27 03:03:55,996][105620] Updated weights for policy 1, policy_version 1605534 (0.0008) [2023-12-27 03:03:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 821288960. Throughput: 0: 9793.3, 1: 9601.4. Samples: 821294484. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:03:56,062][104569] Avg episode reward: [(0, '8260.105'), (1, '9355.628')] [2023-12-27 03:03:56,326][105692] Updated weights for policy 0, policy_version 1602189 (0.0009) [2023-12-27 03:03:56,379][105692] Updated weights for policy 0, policy_version 1602199 (0.0009) [2023-12-27 03:03:56,435][105692] Updated weights for policy 0, policy_version 1602209 (0.0009) [2023-12-27 03:03:56,763][105620] Updated weights for policy 1, policy_version 1605544 (0.0009) [2023-12-27 03:03:56,826][105620] Updated weights for policy 1, policy_version 1605554 (0.0009) [2023-12-27 03:03:56,896][105620] Updated weights for policy 1, policy_version 1605564 (0.0008) [2023-12-27 03:03:57,108][105692] Updated weights for policy 0, policy_version 1602219 (0.0008) [2023-12-27 03:03:57,159][105692] Updated weights for policy 0, policy_version 1602229 (0.0005) [2023-12-27 03:03:57,210][105692] Updated weights for policy 0, policy_version 1602239 (0.0005) [2023-12-27 03:03:57,458][105620] Updated weights for policy 1, policy_version 1605574 (0.0005) [2023-12-27 03:03:57,511][105620] Updated weights for policy 1, policy_version 1605584 (0.0005) [2023-12-27 03:03:57,569][105620] Updated weights for policy 1, policy_version 1605594 (0.0005) [2023-12-27 03:03:57,874][105692] Updated weights for policy 0, policy_version 1602249 (0.0006) [2023-12-27 03:03:57,933][105692] Updated weights for policy 0, policy_version 1602259 (0.0005) [2023-12-27 03:03:57,995][105692] Updated weights for policy 0, policy_version 1602269 (0.0005) [2023-12-27 03:03:58,061][105692] Updated weights for policy 0, policy_version 1602279 (0.0005) [2023-12-27 03:03:58,102][105620] Updated weights for policy 1, policy_version 1605604 (0.0005) [2023-12-27 03:03:58,170][105620] Updated weights for policy 1, policy_version 1605614 (0.0007) [2023-12-27 03:03:58,232][105620] Updated weights for policy 1, policy_version 1605624 (0.0008) [2023-12-27 03:03:58,734][105692] Updated weights for policy 0, policy_version 1602289 (0.0008) [2023-12-27 03:03:58,808][105692] Updated weights for policy 0, policy_version 1602299 (0.0008) [2023-12-27 03:03:58,872][105692] Updated weights for policy 0, policy_version 1602309 (0.0008) [2023-12-27 03:03:59,087][105620] Updated weights for policy 1, policy_version 1605634 (0.0008) [2023-12-27 03:03:59,150][105620] Updated weights for policy 1, policy_version 1605644 (0.0010) [2023-12-27 03:03:59,210][105620] Updated weights for policy 1, policy_version 1605654 (0.0010) [2023-12-27 03:03:59,295][105620] Updated weights for policy 1, policy_version 1605664 (0.0012) [2023-12-27 03:03:59,707][105692] Updated weights for policy 0, policy_version 1602319 (0.0009) [2023-12-27 03:03:59,767][105692] Updated weights for policy 0, policy_version 1602329 (0.0009) [2023-12-27 03:03:59,830][105692] Updated weights for policy 0, policy_version 1602339 (0.0009) [2023-12-27 03:03:59,922][105620] Updated weights for policy 1, policy_version 1605674 (0.0009) [2023-12-27 03:04:00,000][105620] Updated weights for policy 1, policy_version 1605684 (0.0009) [2023-12-27 03:04:00,050][105620] Updated weights for policy 1, policy_version 1605694 (0.0007) [2023-12-27 03:04:00,536][105692] Updated weights for policy 0, policy_version 1602349 (0.0007) [2023-12-27 03:04:00,592][105692] Updated weights for policy 0, policy_version 1602359 (0.0005) [2023-12-27 03:04:00,644][105692] Updated weights for policy 0, policy_version 1602369 (0.0005) [2023-12-27 03:04:00,649][105620] Updated weights for policy 1, policy_version 1605704 (0.0005) [2023-12-27 03:04:00,704][105620] Updated weights for policy 1, policy_version 1605714 (0.0005) [2023-12-27 03:04:00,755][105620] Updated weights for policy 1, policy_version 1605724 (0.0005) [2023-12-27 03:04:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 821387264. Throughput: 0: 9866.6, 1: 9656.7. Samples: 821355776. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:04:01,063][104569] Avg episode reward: [(0, '8076.354'), (1, '9265.574')] [2023-12-27 03:04:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001602376_410263552.pth... [2023-12-27 03:04:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001605728_411123712.pth... [2023-12-27 03:04:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001601256_409976832.pth [2023-12-27 03:04:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001604576_410828800.pth [2023-12-27 03:04:01,316][105692] Updated weights for policy 0, policy_version 1602379 (0.0007) [2023-12-27 03:04:01,336][105620] Updated weights for policy 1, policy_version 1605734 (0.0009) [2023-12-27 03:04:01,379][105692] Updated weights for policy 0, policy_version 1602389 (0.0012) [2023-12-27 03:04:01,396][105620] Updated weights for policy 1, policy_version 1605744 (0.0008) [2023-12-27 03:04:01,435][105692] Updated weights for policy 0, policy_version 1602399 (0.0011) [2023-12-27 03:04:01,448][105620] Updated weights for policy 1, policy_version 1605754 (0.0005) [2023-12-27 03:04:02,071][105692] Updated weights for policy 0, policy_version 1602409 (0.0010) [2023-12-27 03:04:02,136][105692] Updated weights for policy 0, policy_version 1602419 (0.0005) [2023-12-27 03:04:02,198][105620] Updated weights for policy 1, policy_version 1605764 (0.0007) [2023-12-27 03:04:02,200][105692] Updated weights for policy 0, policy_version 1602429 (0.0005) [2023-12-27 03:04:02,254][105620] Updated weights for policy 1, policy_version 1605774 (0.0007) [2023-12-27 03:04:02,254][105692] Updated weights for policy 0, policy_version 1602439 (0.0008) [2023-12-27 03:04:02,308][105620] Updated weights for policy 1, policy_version 1605784 (0.0008) [2023-12-27 03:04:02,850][105692] Updated weights for policy 0, policy_version 1602449 (0.0007) [2023-12-27 03:04:02,893][105692] Updated weights for policy 0, policy_version 1602459 (0.0005) [2023-12-27 03:04:02,939][105692] Updated weights for policy 0, policy_version 1602469 (0.0005) [2023-12-27 03:04:03,055][105620] Updated weights for policy 1, policy_version 1605794 (0.0008) [2023-12-27 03:04:03,117][105620] Updated weights for policy 1, policy_version 1605804 (0.0010) [2023-12-27 03:04:03,175][105620] Updated weights for policy 1, policy_version 1605814 (0.0010) [2023-12-27 03:04:03,236][105620] Updated weights for policy 1, policy_version 1605824 (0.0010) [2023-12-27 03:04:03,483][105692] Updated weights for policy 0, policy_version 1602479 (0.0005) [2023-12-27 03:04:03,538][105692] Updated weights for policy 0, policy_version 1602489 (0.0005) [2023-12-27 03:04:03,596][105692] Updated weights for policy 0, policy_version 1602499 (0.0005) [2023-12-27 03:04:03,923][105620] Updated weights for policy 1, policy_version 1605834 (0.0010) [2023-12-27 03:04:03,985][105620] Updated weights for policy 1, policy_version 1605844 (0.0011) [2023-12-27 03:04:04,047][105620] Updated weights for policy 1, policy_version 1605854 (0.0010) [2023-12-27 03:04:04,155][105692] Updated weights for policy 0, policy_version 1602509 (0.0008) [2023-12-27 03:04:04,214][105692] Updated weights for policy 0, policy_version 1602519 (0.0011) [2023-12-27 03:04:04,274][105692] Updated weights for policy 0, policy_version 1602529 (0.0011) [2023-12-27 03:04:04,762][105620] Updated weights for policy 1, policy_version 1605864 (0.0010) [2023-12-27 03:04:04,830][105620] Updated weights for policy 1, policy_version 1605874 (0.0010) [2023-12-27 03:04:04,894][105620] Updated weights for policy 1, policy_version 1605884 (0.0010) [2023-12-27 03:04:04,969][105692] Updated weights for policy 0, policy_version 1602539 (0.0010) [2023-12-27 03:04:05,014][105692] Updated weights for policy 0, policy_version 1602549 (0.0008) [2023-12-27 03:04:05,062][105692] Updated weights for policy 0, policy_version 1602559 (0.0008) [2023-12-27 03:04:05,491][105620] Updated weights for policy 1, policy_version 1605895 (0.0006) [2023-12-27 03:04:05,545][105620] Updated weights for policy 1, policy_version 1605905 (0.0006) [2023-12-27 03:04:05,608][105620] Updated weights for policy 1, policy_version 1605915 (0.0009) [2023-12-27 03:04:05,901][105692] Updated weights for policy 0, policy_version 1602569 (0.0008) [2023-12-27 03:04:05,947][105692] Updated weights for policy 0, policy_version 1602579 (0.0008) [2023-12-27 03:04:05,997][105692] Updated weights for policy 0, policy_version 1602589 (0.0007) [2023-12-27 03:04:06,041][105692] Updated weights for policy 0, policy_version 1602599 (0.0005) [2023-12-27 03:04:06,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 821493760. Throughput: 0: 9876.1, 1: 9807.9. Samples: 821478624. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:04:06,063][104569] Avg episode reward: [(0, '8714.026'), (1, '9086.904')] [2023-12-27 03:04:06,380][105620] Updated weights for policy 1, policy_version 1605925 (0.0009) [2023-12-27 03:04:06,431][105620] Updated weights for policy 1, policy_version 1605935 (0.0008) [2023-12-27 03:04:06,485][105620] Updated weights for policy 1, policy_version 1605945 (0.0009) [2023-12-27 03:04:06,716][105692] Updated weights for policy 0, policy_version 1602609 (0.0007) [2023-12-27 03:04:06,762][105692] Updated weights for policy 0, policy_version 1602619 (0.0008) [2023-12-27 03:04:06,826][105692] Updated weights for policy 0, policy_version 1602629 (0.0006) [2023-12-27 03:04:07,231][105620] Updated weights for policy 1, policy_version 1605955 (0.0009) [2023-12-27 03:04:07,293][105620] Updated weights for policy 1, policy_version 1605965 (0.0008) [2023-12-27 03:04:07,354][105620] Updated weights for policy 1, policy_version 1605975 (0.0008) [2023-12-27 03:04:07,578][105692] Updated weights for policy 0, policy_version 1602639 (0.0008) [2023-12-27 03:04:07,628][105692] Updated weights for policy 0, policy_version 1602649 (0.0009) [2023-12-27 03:04:07,687][105692] Updated weights for policy 0, policy_version 1602660 (0.0010) [2023-12-27 03:04:08,003][105620] Updated weights for policy 1, policy_version 1605985 (0.0005) [2023-12-27 03:04:08,065][105620] Updated weights for policy 1, policy_version 1605995 (0.0005) [2023-12-27 03:04:08,131][105620] Updated weights for policy 1, policy_version 1606005 (0.0006) [2023-12-27 03:04:08,180][105620] Updated weights for policy 1, policy_version 1606015 (0.0008) [2023-12-27 03:04:08,501][105692] Updated weights for policy 0, policy_version 1602670 (0.0008) [2023-12-27 03:04:08,559][105692] Updated weights for policy 0, policy_version 1602680 (0.0009) [2023-12-27 03:04:08,615][105692] Updated weights for policy 0, policy_version 1602690 (0.0009) [2023-12-27 03:04:08,857][105620] Updated weights for policy 1, policy_version 1606025 (0.0009) [2023-12-27 03:04:08,923][105620] Updated weights for policy 1, policy_version 1606035 (0.0010) [2023-12-27 03:04:08,977][105620] Updated weights for policy 1, policy_version 1606045 (0.0010) [2023-12-27 03:04:09,442][105692] Updated weights for policy 0, policy_version 1602700 (0.0009) [2023-12-27 03:04:09,497][105692] Updated weights for policy 0, policy_version 1602710 (0.0010) [2023-12-27 03:04:09,554][105692] Updated weights for policy 0, policy_version 1602720 (0.0008) [2023-12-27 03:04:09,631][105620] Updated weights for policy 1, policy_version 1606055 (0.0010) [2023-12-27 03:04:09,684][105620] Updated weights for policy 1, policy_version 1606065 (0.0010) [2023-12-27 03:04:09,746][105620] Updated weights for policy 1, policy_version 1606075 (0.0010) [2023-12-27 03:04:10,401][105692] Updated weights for policy 0, policy_version 1602730 (0.0008) [2023-12-27 03:04:10,461][105692] Updated weights for policy 0, policy_version 1602740 (0.0009) [2023-12-27 03:04:10,478][105620] Updated weights for policy 1, policy_version 1606085 (0.0010) [2023-12-27 03:04:10,520][105692] Updated weights for policy 0, policy_version 1602750 (0.0006) [2023-12-27 03:04:10,526][105620] Updated weights for policy 1, policy_version 1606095 (0.0010) [2023-12-27 03:04:10,577][105620] Updated weights for policy 1, policy_version 1606105 (0.0010) [2023-12-27 03:04:10,580][105692] Updated weights for policy 0, policy_version 1602760 (0.0007) [2023-12-27 03:04:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 821583872. Throughput: 0: 9723.3, 1: 9907.9. Samples: 821593088. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:04:11,063][104569] Avg episode reward: [(0, '8989.581'), (1, '9085.020')] [2023-12-27 03:04:11,365][105692] Updated weights for policy 0, policy_version 1602770 (0.0008) [2023-12-27 03:04:11,387][105620] Updated weights for policy 1, policy_version 1606115 (0.0010) [2023-12-27 03:04:11,430][105692] Updated weights for policy 0, policy_version 1602780 (0.0011) [2023-12-27 03:04:11,452][105620] Updated weights for policy 1, policy_version 1606125 (0.0008) [2023-12-27 03:04:11,487][105692] Updated weights for policy 0, policy_version 1602790 (0.0011) [2023-12-27 03:04:11,511][105620] Updated weights for policy 1, policy_version 1606135 (0.0010) [2023-12-27 03:04:12,231][105692] Updated weights for policy 0, policy_version 1602800 (0.0009) [2023-12-27 03:04:12,278][105620] Updated weights for policy 1, policy_version 1606145 (0.0010) [2023-12-27 03:04:12,292][105692] Updated weights for policy 0, policy_version 1602810 (0.0008) [2023-12-27 03:04:12,346][105620] Updated weights for policy 1, policy_version 1606155 (0.0010) [2023-12-27 03:04:12,360][105692] Updated weights for policy 0, policy_version 1602820 (0.0007) [2023-12-27 03:04:12,407][105620] Updated weights for policy 1, policy_version 1606165 (0.0011) [2023-12-27 03:04:12,463][105620] Updated weights for policy 1, policy_version 1606175 (0.0010) [2023-12-27 03:04:13,073][105692] Updated weights for policy 0, policy_version 1602830 (0.0006) [2023-12-27 03:04:13,084][105620] Updated weights for policy 1, policy_version 1606185 (0.0006) [2023-12-27 03:04:13,123][105692] Updated weights for policy 0, policy_version 1602840 (0.0005) [2023-12-27 03:04:13,139][105620] Updated weights for policy 1, policy_version 1606195 (0.0005) [2023-12-27 03:04:13,173][105692] Updated weights for policy 0, policy_version 1602850 (0.0010) [2023-12-27 03:04:13,193][105620] Updated weights for policy 1, policy_version 1606205 (0.0007) [2023-12-27 03:04:13,785][105620] Updated weights for policy 1, policy_version 1606215 (0.0010) [2023-12-27 03:04:13,862][105620] Updated weights for policy 1, policy_version 1606225 (0.0010) [2023-12-27 03:04:13,909][105692] Updated weights for policy 0, policy_version 1602860 (0.0006) [2023-12-27 03:04:13,921][105620] Updated weights for policy 1, policy_version 1606235 (0.0010) [2023-12-27 03:04:13,968][105692] Updated weights for policy 0, policy_version 1602870 (0.0007) [2023-12-27 03:04:14,020][105692] Updated weights for policy 0, policy_version 1602880 (0.0010) [2023-12-27 03:04:14,559][105620] Updated weights for policy 1, policy_version 1606245 (0.0008) [2023-12-27 03:04:14,616][105620] Updated weights for policy 1, policy_version 1606255 (0.0009) [2023-12-27 03:04:14,664][105620] Updated weights for policy 1, policy_version 1606265 (0.0009) [2023-12-27 03:04:14,697][105692] Updated weights for policy 0, policy_version 1602890 (0.0008) [2023-12-27 03:04:14,758][105692] Updated weights for policy 0, policy_version 1602900 (0.0009) [2023-12-27 03:04:14,826][105692] Updated weights for policy 0, policy_version 1602910 (0.0009) [2023-12-27 03:04:14,891][105692] Updated weights for policy 0, policy_version 1602920 (0.0006) [2023-12-27 03:04:15,361][105620] Updated weights for policy 1, policy_version 1606275 (0.0008) [2023-12-27 03:04:15,419][105620] Updated weights for policy 1, policy_version 1606285 (0.0008) [2023-12-27 03:04:15,471][105620] Updated weights for policy 1, policy_version 1606295 (0.0010) [2023-12-27 03:04:15,573][105692] Updated weights for policy 0, policy_version 1602930 (0.0010) [2023-12-27 03:04:15,640][105692] Updated weights for policy 0, policy_version 1602940 (0.0010) [2023-12-27 03:04:15,704][105692] Updated weights for policy 0, policy_version 1602950 (0.0009) [2023-12-27 03:04:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 821682176. Throughput: 0: 9651.4, 1: 9915.1. Samples: 821650860. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:04:16,062][104569] Avg episode reward: [(0, '8803.376'), (1, '8992.805')] [2023-12-27 03:04:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001606304_411271168.pth... [2023-12-27 03:04:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001602952_410411008.pth... [2023-12-27 03:04:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001605152_410976256.pth [2023-12-27 03:04:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001601832_410124288.pth [2023-12-27 03:04:16,089][105620] Updated weights for policy 1, policy_version 1606305 (0.0005) [2023-12-27 03:04:16,142][105620] Updated weights for policy 1, policy_version 1606315 (0.0006) [2023-12-27 03:04:16,191][105620] Updated weights for policy 1, policy_version 1606325 (0.0005) [2023-12-27 03:04:16,243][105620] Updated weights for policy 1, policy_version 1606335 (0.0005) [2023-12-27 03:04:16,532][105692] Updated weights for policy 0, policy_version 1602960 (0.0010) [2023-12-27 03:04:16,585][105692] Updated weights for policy 0, policy_version 1602970 (0.0008) [2023-12-27 03:04:16,649][105692] Updated weights for policy 0, policy_version 1602980 (0.0009) [2023-12-27 03:04:16,848][105620] Updated weights for policy 1, policy_version 1606345 (0.0007) [2023-12-27 03:04:16,903][105620] Updated weights for policy 1, policy_version 1606355 (0.0007) [2023-12-27 03:04:16,966][105620] Updated weights for policy 1, policy_version 1606365 (0.0008) [2023-12-27 03:04:17,431][105692] Updated weights for policy 0, policy_version 1602990 (0.0010) [2023-12-27 03:04:17,488][105692] Updated weights for policy 0, policy_version 1603000 (0.0010) [2023-12-27 03:04:17,549][105692] Updated weights for policy 0, policy_version 1603010 (0.0010) [2023-12-27 03:04:17,657][105620] Updated weights for policy 1, policy_version 1606375 (0.0010) [2023-12-27 03:04:17,718][105620] Updated weights for policy 1, policy_version 1606385 (0.0010) [2023-12-27 03:04:17,768][105620] Updated weights for policy 1, policy_version 1606395 (0.0010) [2023-12-27 03:04:18,238][105692] Updated weights for policy 0, policy_version 1603020 (0.0010) [2023-12-27 03:04:18,292][105692] Updated weights for policy 0, policy_version 1603030 (0.0010) [2023-12-27 03:04:18,354][105692] Updated weights for policy 0, policy_version 1603040 (0.0012) [2023-12-27 03:04:18,461][105620] Updated weights for policy 1, policy_version 1606405 (0.0008) [2023-12-27 03:04:18,529][105620] Updated weights for policy 1, policy_version 1606415 (0.0007) [2023-12-27 03:04:18,584][105620] Updated weights for policy 1, policy_version 1606425 (0.0010) [2023-12-27 03:04:19,027][105692] Updated weights for policy 0, policy_version 1603050 (0.0007) [2023-12-27 03:04:19,092][105692] Updated weights for policy 0, policy_version 1603060 (0.0010) [2023-12-27 03:04:19,151][105692] Updated weights for policy 0, policy_version 1603070 (0.0011) [2023-12-27 03:04:19,214][105692] Updated weights for policy 0, policy_version 1603080 (0.0011) [2023-12-27 03:04:19,236][105620] Updated weights for policy 1, policy_version 1606435 (0.0008) [2023-12-27 03:04:19,301][105620] Updated weights for policy 1, policy_version 1606445 (0.0011) [2023-12-27 03:04:19,366][105620] Updated weights for policy 1, policy_version 1606455 (0.0009) [2023-12-27 03:04:19,969][105692] Updated weights for policy 0, policy_version 1603090 (0.0010) [2023-12-27 03:04:20,029][105692] Updated weights for policy 0, policy_version 1603100 (0.0010) [2023-12-27 03:04:20,048][105620] Updated weights for policy 1, policy_version 1606465 (0.0007) [2023-12-27 03:04:20,089][105692] Updated weights for policy 0, policy_version 1603110 (0.0011) [2023-12-27 03:04:20,108][105620] Updated weights for policy 1, policy_version 1606475 (0.0006) [2023-12-27 03:04:20,172][105620] Updated weights for policy 1, policy_version 1606485 (0.0008) [2023-12-27 03:04:20,237][105620] Updated weights for policy 1, policy_version 1606495 (0.0009) [2023-12-27 03:04:20,738][105692] Updated weights for policy 0, policy_version 1603120 (0.0007) [2023-12-27 03:04:20,799][105692] Updated weights for policy 0, policy_version 1603130 (0.0010) [2023-12-27 03:04:20,862][105692] Updated weights for policy 0, policy_version 1603140 (0.0011) [2023-12-27 03:04:21,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 821780480. Throughput: 0: 9583.6, 1: 9967.1. Samples: 821771580. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:04:21,062][104569] Avg episode reward: [(0, '8342.457'), (1, '8814.562')] [2023-12-27 03:04:21,076][105620] Updated weights for policy 1, policy_version 1606505 (0.0010) [2023-12-27 03:04:21,138][105620] Updated weights for policy 1, policy_version 1606515 (0.0011) [2023-12-27 03:04:21,191][105620] Updated weights for policy 1, policy_version 1606525 (0.0011) [2023-12-27 03:04:21,648][105692] Updated weights for policy 0, policy_version 1603150 (0.0009) [2023-12-27 03:04:21,715][105692] Updated weights for policy 0, policy_version 1603160 (0.0008) [2023-12-27 03:04:21,776][105692] Updated weights for policy 0, policy_version 1603170 (0.0008) [2023-12-27 03:04:21,899][105620] Updated weights for policy 1, policy_version 1606535 (0.0008) [2023-12-27 03:04:21,956][105620] Updated weights for policy 1, policy_version 1606545 (0.0010) [2023-12-27 03:04:22,015][105620] Updated weights for policy 1, policy_version 1606555 (0.0009) [2023-12-27 03:04:22,514][105692] Updated weights for policy 0, policy_version 1603180 (0.0008) [2023-12-27 03:04:22,564][105692] Updated weights for policy 0, policy_version 1603190 (0.0008) [2023-12-27 03:04:22,622][105692] Updated weights for policy 0, policy_version 1603200 (0.0009) [2023-12-27 03:04:22,771][105620] Updated weights for policy 1, policy_version 1606565 (0.0009) [2023-12-27 03:04:22,838][105620] Updated weights for policy 1, policy_version 1606575 (0.0010) [2023-12-27 03:04:22,902][105620] Updated weights for policy 1, policy_version 1606585 (0.0009) [2023-12-27 03:04:23,274][105692] Updated weights for policy 0, policy_version 1603210 (0.0009) [2023-12-27 03:04:23,338][105692] Updated weights for policy 0, policy_version 1603220 (0.0009) [2023-12-27 03:04:23,404][105692] Updated weights for policy 0, policy_version 1603230 (0.0006) [2023-12-27 03:04:23,473][105692] Updated weights for policy 0, policy_version 1603240 (0.0006) [2023-12-27 03:04:23,718][105620] Updated weights for policy 1, policy_version 1606595 (0.0010) [2023-12-27 03:04:23,783][105620] Updated weights for policy 1, policy_version 1606605 (0.0009) [2023-12-27 03:04:23,844][105620] Updated weights for policy 1, policy_version 1606615 (0.0009) [2023-12-27 03:04:24,076][105692] Updated weights for policy 0, policy_version 1603250 (0.0009) [2023-12-27 03:04:24,128][105692] Updated weights for policy 0, policy_version 1603260 (0.0006) [2023-12-27 03:04:24,184][105692] Updated weights for policy 0, policy_version 1603270 (0.0005) [2023-12-27 03:04:24,675][105620] Updated weights for policy 1, policy_version 1606625 (0.0009) [2023-12-27 03:04:24,734][105620] Updated weights for policy 1, policy_version 1606635 (0.0009) [2023-12-27 03:04:24,754][105692] Updated weights for policy 0, policy_version 1603280 (0.0005) [2023-12-27 03:04:24,795][105620] Updated weights for policy 1, policy_version 1606645 (0.0009) [2023-12-27 03:04:24,807][105692] Updated weights for policy 0, policy_version 1603290 (0.0005) [2023-12-27 03:04:24,854][105620] Updated weights for policy 1, policy_version 1606655 (0.0006) [2023-12-27 03:04:24,857][105692] Updated weights for policy 0, policy_version 1603300 (0.0007) [2023-12-27 03:04:25,483][105692] Updated weights for policy 0, policy_version 1603310 (0.0006) [2023-12-27 03:04:25,532][105692] Updated weights for policy 0, policy_version 1603320 (0.0005) [2023-12-27 03:04:25,560][105620] Updated weights for policy 1, policy_version 1606665 (0.0010) [2023-12-27 03:04:25,591][105692] Updated weights for policy 0, policy_version 1603330 (0.0008) [2023-12-27 03:04:25,613][105620] Updated weights for policy 1, policy_version 1606675 (0.0010) [2023-12-27 03:04:25,672][105620] Updated weights for policy 1, policy_version 1606685 (0.0010) [2023-12-27 03:04:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 821878784. Throughput: 0: 9662.4, 1: 9920.3. Samples: 821888236. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:04:26,062][104569] Avg episode reward: [(0, '8437.269'), (1, '8901.220')] [2023-12-27 03:04:26,235][105692] Updated weights for policy 0, policy_version 1603340 (0.0010) [2023-12-27 03:04:26,295][105692] Updated weights for policy 0, policy_version 1603350 (0.0011) [2023-12-27 03:04:26,345][105620] Updated weights for policy 1, policy_version 1606695 (0.0009) [2023-12-27 03:04:26,350][105692] Updated weights for policy 0, policy_version 1603360 (0.0007) [2023-12-27 03:04:26,413][105620] Updated weights for policy 1, policy_version 1606705 (0.0009) [2023-12-27 03:04:26,480][105620] Updated weights for policy 1, policy_version 1606715 (0.0008) [2023-12-27 03:04:26,957][105692] Updated weights for policy 0, policy_version 1603370 (0.0005) [2023-12-27 03:04:27,006][105692] Updated weights for policy 0, policy_version 1603380 (0.0006) [2023-12-27 03:04:27,051][105620] Updated weights for policy 1, policy_version 1606725 (0.0009) [2023-12-27 03:04:27,056][105692] Updated weights for policy 0, policy_version 1603390 (0.0008) [2023-12-27 03:04:27,111][105620] Updated weights for policy 1, policy_version 1606735 (0.0005) [2023-12-27 03:04:27,116][105692] Updated weights for policy 0, policy_version 1603400 (0.0008) [2023-12-27 03:04:27,164][105620] Updated weights for policy 1, policy_version 1606745 (0.0005) [2023-12-27 03:04:27,800][105620] Updated weights for policy 1, policy_version 1606755 (0.0007) [2023-12-27 03:04:27,842][105692] Updated weights for policy 0, policy_version 1603410 (0.0010) [2023-12-27 03:04:27,855][105620] Updated weights for policy 1, policy_version 1606765 (0.0010) [2023-12-27 03:04:27,892][105692] Updated weights for policy 0, policy_version 1603420 (0.0010) [2023-12-27 03:04:27,906][105620] Updated weights for policy 1, policy_version 1606775 (0.0010) [2023-12-27 03:04:27,943][105692] Updated weights for policy 0, policy_version 1603430 (0.0010) [2023-12-27 03:04:28,625][105620] Updated weights for policy 1, policy_version 1606785 (0.0010) [2023-12-27 03:04:28,687][105620] Updated weights for policy 1, policy_version 1606795 (0.0009) [2023-12-27 03:04:28,691][105692] Updated weights for policy 0, policy_version 1603440 (0.0010) [2023-12-27 03:04:28,736][105692] Updated weights for policy 0, policy_version 1603450 (0.0010) [2023-12-27 03:04:28,746][105620] Updated weights for policy 1, policy_version 1606805 (0.0006) [2023-12-27 03:04:28,787][105692] Updated weights for policy 0, policy_version 1603460 (0.0010) [2023-12-27 03:04:28,801][105620] Updated weights for policy 1, policy_version 1606815 (0.0006) [2023-12-27 03:04:29,421][105620] Updated weights for policy 1, policy_version 1606825 (0.0008) [2023-12-27 03:04:29,477][105620] Updated weights for policy 1, policy_version 1606835 (0.0008) [2023-12-27 03:04:29,544][105620] Updated weights for policy 1, policy_version 1606845 (0.0008) [2023-12-27 03:04:29,573][105692] Updated weights for policy 0, policy_version 1603470 (0.0008) [2023-12-27 03:04:29,627][105692] Updated weights for policy 0, policy_version 1603480 (0.0010) [2023-12-27 03:04:29,685][105692] Updated weights for policy 0, policy_version 1603490 (0.0010) [2023-12-27 03:04:30,304][105620] Updated weights for policy 1, policy_version 1606855 (0.0009) [2023-12-27 03:04:30,363][105620] Updated weights for policy 1, policy_version 1606865 (0.0007) [2023-12-27 03:04:30,409][105692] Updated weights for policy 0, policy_version 1603500 (0.0010) [2023-12-27 03:04:30,420][105620] Updated weights for policy 1, policy_version 1606875 (0.0006) [2023-12-27 03:04:30,464][105692] Updated weights for policy 0, policy_version 1603510 (0.0008) [2023-12-27 03:04:30,511][105692] Updated weights for policy 0, policy_version 1603520 (0.0007) [2023-12-27 03:04:31,023][105620] Updated weights for policy 1, policy_version 1606885 (0.0007) [2023-12-27 03:04:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 821977088. Throughput: 0: 9772.8, 1: 9966.4. Samples: 821950176. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:04:31,063][104569] Avg episode reward: [(0, '8357.211'), (1, '9264.009')] [2023-12-27 03:04:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001603528_410558464.pth... [2023-12-27 03:04:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001602376_410263552.pth [2023-12-27 03:04:31,093][105620] Updated weights for policy 1, policy_version 1606895 (0.0008) [2023-12-27 03:04:31,163][105620] Updated weights for policy 1, policy_version 1606905 (0.0009) [2023-12-27 03:04:31,197][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001606912_411426816.pth... [2023-12-27 03:04:31,200][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001605728_411123712.pth [2023-12-27 03:04:31,293][105692] Updated weights for policy 0, policy_version 1603530 (0.0007) [2023-12-27 03:04:31,346][105692] Updated weights for policy 0, policy_version 1603540 (0.0008) [2023-12-27 03:04:31,412][105692] Updated weights for policy 0, policy_version 1603550 (0.0007) [2023-12-27 03:04:31,471][105692] Updated weights for policy 0, policy_version 1603560 (0.0008) [2023-12-27 03:04:31,865][105620] Updated weights for policy 1, policy_version 1606915 (0.0008) [2023-12-27 03:04:31,923][105620] Updated weights for policy 1, policy_version 1606925 (0.0007) [2023-12-27 03:04:31,978][105620] Updated weights for policy 1, policy_version 1606935 (0.0008) [2023-12-27 03:04:32,271][105692] Updated weights for policy 0, policy_version 1603570 (0.0009) [2023-12-27 03:04:32,323][105692] Updated weights for policy 0, policy_version 1603580 (0.0009) [2023-12-27 03:04:32,381][105692] Updated weights for policy 0, policy_version 1603590 (0.0009) [2023-12-27 03:04:32,643][105620] Updated weights for policy 1, policy_version 1606945 (0.0009) [2023-12-27 03:04:32,697][105620] Updated weights for policy 1, policy_version 1606955 (0.0009) [2023-12-27 03:04:32,756][105620] Updated weights for policy 1, policy_version 1606965 (0.0008) [2023-12-27 03:04:32,814][105620] Updated weights for policy 1, policy_version 1606975 (0.0009) [2023-12-27 03:04:33,171][105692] Updated weights for policy 0, policy_version 1603600 (0.0009) [2023-12-27 03:04:33,217][105692] Updated weights for policy 0, policy_version 1603610 (0.0009) [2023-12-27 03:04:33,264][105692] Updated weights for policy 0, policy_version 1603620 (0.0009) [2023-12-27 03:04:33,606][105620] Updated weights for policy 1, policy_version 1606985 (0.0009) [2023-12-27 03:04:33,668][105620] Updated weights for policy 1, policy_version 1606995 (0.0008) [2023-12-27 03:04:33,725][105620] Updated weights for policy 1, policy_version 1607005 (0.0008) [2023-12-27 03:04:33,916][105692] Updated weights for policy 0, policy_version 1603630 (0.0008) [2023-12-27 03:04:33,968][105692] Updated weights for policy 0, policy_version 1603640 (0.0009) [2023-12-27 03:04:34,021][105692] Updated weights for policy 0, policy_version 1603651 (0.0010) [2023-12-27 03:04:34,416][105620] Updated weights for policy 1, policy_version 1607015 (0.0009) [2023-12-27 03:04:34,472][105620] Updated weights for policy 1, policy_version 1607025 (0.0009) [2023-12-27 03:04:34,527][105620] Updated weights for policy 1, policy_version 1607035 (0.0009) [2023-12-27 03:04:34,809][105692] Updated weights for policy 0, policy_version 1603662 (0.0010) [2023-12-27 03:04:34,860][105692] Updated weights for policy 0, policy_version 1603672 (0.0009) [2023-12-27 03:04:34,907][105692] Updated weights for policy 0, policy_version 1603682 (0.0009) [2023-12-27 03:04:35,298][105620] Updated weights for policy 1, policy_version 1607045 (0.0009) [2023-12-27 03:04:35,353][105620] Updated weights for policy 1, policy_version 1607055 (0.0009) [2023-12-27 03:04:35,416][105620] Updated weights for policy 1, policy_version 1607065 (0.0009) [2023-12-27 03:04:35,676][105692] Updated weights for policy 0, policy_version 1603692 (0.0008) [2023-12-27 03:04:35,731][105692] Updated weights for policy 0, policy_version 1603702 (0.0009) [2023-12-27 03:04:35,782][105692] Updated weights for policy 0, policy_version 1603712 (0.0009) [2023-12-27 03:04:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.4, 300 sec: 19577.5). Total num frames: 822075392. Throughput: 0: 9775.6, 1: 9934.9. Samples: 822065764. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:04:36,062][104569] Avg episode reward: [(0, '7988.582'), (1, '8990.803')] [2023-12-27 03:04:36,160][105620] Updated weights for policy 1, policy_version 1607075 (0.0008) [2023-12-27 03:04:36,215][105620] Updated weights for policy 1, policy_version 1607085 (0.0009) [2023-12-27 03:04:36,271][105620] Updated weights for policy 1, policy_version 1607095 (0.0009) [2023-12-27 03:04:36,566][105692] Updated weights for policy 0, policy_version 1603722 (0.0008) [2023-12-27 03:04:36,626][105692] Updated weights for policy 0, policy_version 1603732 (0.0006) [2023-12-27 03:04:36,680][105692] Updated weights for policy 0, policy_version 1603742 (0.0010) [2023-12-27 03:04:36,740][105692] Updated weights for policy 0, policy_version 1603752 (0.0010) [2023-12-27 03:04:36,940][105620] Updated weights for policy 1, policy_version 1607105 (0.0008) [2023-12-27 03:04:37,007][105620] Updated weights for policy 1, policy_version 1607115 (0.0011) [2023-12-27 03:04:37,067][105620] Updated weights for policy 1, policy_version 1607125 (0.0011) [2023-12-27 03:04:37,122][105620] Updated weights for policy 1, policy_version 1607135 (0.0010) [2023-12-27 03:04:37,458][105692] Updated weights for policy 0, policy_version 1603762 (0.0010) [2023-12-27 03:04:37,517][105692] Updated weights for policy 0, policy_version 1603772 (0.0010) [2023-12-27 03:04:37,572][105692] Updated weights for policy 0, policy_version 1603783 (0.0010) [2023-12-27 03:04:37,775][105620] Updated weights for policy 1, policy_version 1607145 (0.0010) [2023-12-27 03:04:37,831][105620] Updated weights for policy 1, policy_version 1607155 (0.0010) [2023-12-27 03:04:37,882][105620] Updated weights for policy 1, policy_version 1607165 (0.0009) [2023-12-27 03:04:38,341][105692] Updated weights for policy 0, policy_version 1603793 (0.0009) [2023-12-27 03:04:38,406][105692] Updated weights for policy 0, policy_version 1603803 (0.0008) [2023-12-27 03:04:38,472][105692] Updated weights for policy 0, policy_version 1603813 (0.0009) [2023-12-27 03:04:38,591][105620] Updated weights for policy 1, policy_version 1607175 (0.0006) [2023-12-27 03:04:38,648][105620] Updated weights for policy 1, policy_version 1607185 (0.0005) [2023-12-27 03:04:38,706][105620] Updated weights for policy 1, policy_version 1607195 (0.0007) [2023-12-27 03:04:39,209][105692] Updated weights for policy 0, policy_version 1603823 (0.0010) [2023-12-27 03:04:39,271][105692] Updated weights for policy 0, policy_version 1603833 (0.0007) [2023-12-27 03:04:39,330][105692] Updated weights for policy 0, policy_version 1603843 (0.0009) [2023-12-27 03:04:39,410][105620] Updated weights for policy 1, policy_version 1607205 (0.0009) [2023-12-27 03:04:39,469][105620] Updated weights for policy 1, policy_version 1607215 (0.0006) [2023-12-27 03:04:39,523][105620] Updated weights for policy 1, policy_version 1607225 (0.0005) [2023-12-27 03:04:40,142][105692] Updated weights for policy 0, policy_version 1603853 (0.0009) [2023-12-27 03:04:40,203][105692] Updated weights for policy 0, policy_version 1603863 (0.0010) [2023-12-27 03:04:40,209][105620] Updated weights for policy 1, policy_version 1607235 (0.0007) [2023-12-27 03:04:40,261][105692] Updated weights for policy 0, policy_version 1603873 (0.0006) [2023-12-27 03:04:40,270][105620] Updated weights for policy 1, policy_version 1607245 (0.0007) [2023-12-27 03:04:40,326][105620] Updated weights for policy 1, policy_version 1607255 (0.0008) [2023-12-27 03:04:41,036][105692] Updated weights for policy 0, policy_version 1603883 (0.0007) [2023-12-27 03:04:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 822165504. Throughput: 0: 9697.4, 1: 9971.6. Samples: 822179592. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:04:41,063][104569] Avg episode reward: [(0, '8070.475'), (1, '8899.447')] [2023-12-27 03:04:41,088][105620] Updated weights for policy 1, policy_version 1607265 (0.0010) [2023-12-27 03:04:41,099][105692] Updated weights for policy 0, policy_version 1603893 (0.0009) [2023-12-27 03:04:41,153][105620] Updated weights for policy 1, policy_version 1607275 (0.0009) [2023-12-27 03:04:41,165][105692] Updated weights for policy 0, policy_version 1603903 (0.0009) [2023-12-27 03:04:41,215][105620] Updated weights for policy 1, policy_version 1607285 (0.0008) [2023-12-27 03:04:41,282][105620] Updated weights for policy 1, policy_version 1607295 (0.0008) [2023-12-27 03:04:41,906][105692] Updated weights for policy 0, policy_version 1603913 (0.0008) [2023-12-27 03:04:41,966][105692] Updated weights for policy 0, policy_version 1603923 (0.0009) [2023-12-27 03:04:42,029][105692] Updated weights for policy 0, policy_version 1603933 (0.0009) [2023-12-27 03:04:42,072][105620] Updated weights for policy 1, policy_version 1607305 (0.0008) [2023-12-27 03:04:42,090][105692] Updated weights for policy 0, policy_version 1603943 (0.0008) [2023-12-27 03:04:42,129][105620] Updated weights for policy 1, policy_version 1607315 (0.0008) [2023-12-27 03:04:42,142][105586] KL-divergence is very high: 103.9131 [2023-12-27 03:04:42,195][105620] Updated weights for policy 1, policy_version 1607325 (0.0009) [2023-12-27 03:04:42,196][105586] KL-divergence is very high: 182.1346 [2023-12-27 03:04:42,795][105692] Updated weights for policy 0, policy_version 1603953 (0.0010) [2023-12-27 03:04:42,849][105692] Updated weights for policy 0, policy_version 1603963 (0.0010) [2023-12-27 03:04:42,906][105692] Updated weights for policy 0, policy_version 1603973 (0.0007) [2023-12-27 03:04:42,935][105620] Updated weights for policy 1, policy_version 1607335 (0.0009) [2023-12-27 03:04:42,993][105620] Updated weights for policy 1, policy_version 1607345 (0.0009) [2023-12-27 03:04:43,050][105620] Updated weights for policy 1, policy_version 1607355 (0.0009) [2023-12-27 03:04:43,608][105620] Updated weights for policy 1, policy_version 1607365 (0.0008) [2023-12-27 03:04:43,655][105620] Updated weights for policy 1, policy_version 1607375 (0.0008) [2023-12-27 03:04:43,673][105692] Updated weights for policy 0, policy_version 1603983 (0.0010) [2023-12-27 03:04:43,712][105620] Updated weights for policy 1, policy_version 1607385 (0.0005) [2023-12-27 03:04:43,726][105692] Updated weights for policy 0, policy_version 1603993 (0.0006) [2023-12-27 03:04:43,778][105692] Updated weights for policy 0, policy_version 1604003 (0.0005) [2023-12-27 03:04:44,295][105620] Updated weights for policy 1, policy_version 1607395 (0.0005) [2023-12-27 03:04:44,349][105620] Updated weights for policy 1, policy_version 1607405 (0.0006) [2023-12-27 03:04:44,385][105692] Updated weights for policy 0, policy_version 1604013 (0.0007) [2023-12-27 03:04:44,399][105620] Updated weights for policy 1, policy_version 1607415 (0.0006) [2023-12-27 03:04:44,443][105692] Updated weights for policy 0, policy_version 1604023 (0.0007) [2023-12-27 03:04:44,503][105692] Updated weights for policy 0, policy_version 1604033 (0.0008) [2023-12-27 03:04:45,118][105620] Updated weights for policy 1, policy_version 1607425 (0.0006) [2023-12-27 03:04:45,174][105620] Updated weights for policy 1, policy_version 1607435 (0.0005) [2023-12-27 03:04:45,224][105692] Updated weights for policy 0, policy_version 1604043 (0.0009) [2023-12-27 03:04:45,233][105620] Updated weights for policy 1, policy_version 1607445 (0.0006) [2023-12-27 03:04:45,277][105692] Updated weights for policy 0, policy_version 1604053 (0.0009) [2023-12-27 03:04:45,292][105620] Updated weights for policy 1, policy_version 1607455 (0.0007) [2023-12-27 03:04:45,336][105692] Updated weights for policy 0, policy_version 1604063 (0.0008) [2023-12-27 03:04:45,988][105620] Updated weights for policy 1, policy_version 1607465 (0.0009) [2023-12-27 03:04:45,993][105692] Updated weights for policy 0, policy_version 1604073 (0.0006) [2023-12-27 03:04:46,037][105620] Updated weights for policy 1, policy_version 1607475 (0.0007) [2023-12-27 03:04:46,050][105692] Updated weights for policy 0, policy_version 1604083 (0.0008) [2023-12-27 03:04:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 822263808. Throughput: 0: 9649.5, 1: 9948.6. Samples: 822237688. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:04:46,062][104569] Avg episode reward: [(0, '8348.423'), (1, '8991.176')] [2023-12-27 03:04:46,096][105620] Updated weights for policy 1, policy_version 1607485 (0.0008) [2023-12-27 03:04:46,100][105692] Updated weights for policy 0, policy_version 1604093 (0.0007) [2023-12-27 03:04:46,109][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001607488_411574272.pth... [2023-12-27 03:04:46,114][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001606304_411271168.pth [2023-12-27 03:04:46,154][105692] Updated weights for policy 0, policy_version 1604103 (0.0008) [2023-12-27 03:04:46,155][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001604104_410705920.pth... [2023-12-27 03:04:46,158][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001602952_410411008.pth [2023-12-27 03:04:46,824][105692] Updated weights for policy 0, policy_version 1604113 (0.0006) [2023-12-27 03:04:46,870][105620] Updated weights for policy 1, policy_version 1607495 (0.0007) [2023-12-27 03:04:46,871][105692] Updated weights for policy 0, policy_version 1604123 (0.0010) [2023-12-27 03:04:46,918][105620] Updated weights for policy 1, policy_version 1607505 (0.0007) [2023-12-27 03:04:46,921][105692] Updated weights for policy 0, policy_version 1604133 (0.0007) [2023-12-27 03:04:46,968][105620] Updated weights for policy 1, policy_version 1607515 (0.0009) [2023-12-27 03:04:47,490][105692] Updated weights for policy 0, policy_version 1604143 (0.0006) [2023-12-27 03:04:47,538][105692] Updated weights for policy 0, policy_version 1604153 (0.0009) [2023-12-27 03:04:47,592][105692] Updated weights for policy 0, policy_version 1604163 (0.0009) [2023-12-27 03:04:47,831][105620] Updated weights for policy 1, policy_version 1607526 (0.0010) [2023-12-27 03:04:47,893][105620] Updated weights for policy 1, policy_version 1607536 (0.0008) [2023-12-27 03:04:47,955][105620] Updated weights for policy 1, policy_version 1607546 (0.0010) [2023-12-27 03:04:48,355][105692] Updated weights for policy 0, policy_version 1604173 (0.0008) [2023-12-27 03:04:48,408][105692] Updated weights for policy 0, policy_version 1604183 (0.0009) [2023-12-27 03:04:48,460][105692] Updated weights for policy 0, policy_version 1604193 (0.0009) [2023-12-27 03:04:48,594][105620] Updated weights for policy 1, policy_version 1607556 (0.0008) [2023-12-27 03:04:48,658][105620] Updated weights for policy 1, policy_version 1607566 (0.0005) [2023-12-27 03:04:48,729][105620] Updated weights for policy 1, policy_version 1607576 (0.0005) [2023-12-27 03:04:49,228][105692] Updated weights for policy 0, policy_version 1604203 (0.0009) [2023-12-27 03:04:49,290][105692] Updated weights for policy 0, policy_version 1604213 (0.0008) [2023-12-27 03:04:49,357][105692] Updated weights for policy 0, policy_version 1604223 (0.0010) [2023-12-27 03:04:49,437][105620] Updated weights for policy 1, policy_version 1607586 (0.0008) [2023-12-27 03:04:49,495][105620] Updated weights for policy 1, policy_version 1607596 (0.0007) [2023-12-27 03:04:49,554][105620] Updated weights for policy 1, policy_version 1607606 (0.0008) [2023-12-27 03:04:49,614][105620] Updated weights for policy 1, policy_version 1607616 (0.0008) [2023-12-27 03:04:50,049][105692] Updated weights for policy 0, policy_version 1604233 (0.0009) [2023-12-27 03:04:50,098][105692] Updated weights for policy 0, policy_version 1604243 (0.0010) [2023-12-27 03:04:50,143][105692] Updated weights for policy 0, policy_version 1604253 (0.0010) [2023-12-27 03:04:50,199][105692] Updated weights for policy 0, policy_version 1604263 (0.0011) [2023-12-27 03:04:50,359][105620] Updated weights for policy 1, policy_version 1607626 (0.0007) [2023-12-27 03:04:50,413][105620] Updated weights for policy 1, policy_version 1607636 (0.0008) [2023-12-27 03:04:50,472][105620] Updated weights for policy 1, policy_version 1607646 (0.0008) [2023-12-27 03:04:50,976][105692] Updated weights for policy 0, policy_version 1604273 (0.0010) [2023-12-27 03:04:51,040][105692] Updated weights for policy 0, policy_version 1604283 (0.0009) [2023-12-27 03:04:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 822362112. Throughput: 0: 9617.1, 1: 9892.7. Samples: 822356568. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:04:51,063][104569] Avg episode reward: [(0, '8443.346'), (1, '9174.204')] [2023-12-27 03:04:51,097][105692] Updated weights for policy 0, policy_version 1604293 (0.0008) [2023-12-27 03:04:51,134][105620] Updated weights for policy 1, policy_version 1607656 (0.0008) [2023-12-27 03:04:51,192][105620] Updated weights for policy 1, policy_version 1607666 (0.0008) [2023-12-27 03:04:51,247][105620] Updated weights for policy 1, policy_version 1607676 (0.0009) [2023-12-27 03:04:51,817][105692] Updated weights for policy 0, policy_version 1604303 (0.0006) [2023-12-27 03:04:51,871][105692] Updated weights for policy 0, policy_version 1604313 (0.0005) [2023-12-27 03:04:51,925][105692] Updated weights for policy 0, policy_version 1604323 (0.0009) [2023-12-27 03:04:52,102][105620] Updated weights for policy 1, policy_version 1607686 (0.0009) [2023-12-27 03:04:52,149][105620] Updated weights for policy 1, policy_version 1607696 (0.0009) [2023-12-27 03:04:52,204][105620] Updated weights for policy 1, policy_version 1607706 (0.0008) [2023-12-27 03:04:52,577][105692] Updated weights for policy 0, policy_version 1604333 (0.0009) [2023-12-27 03:04:52,628][105692] Updated weights for policy 0, policy_version 1604343 (0.0009) [2023-12-27 03:04:52,691][105692] Updated weights for policy 0, policy_version 1604353 (0.0009) [2023-12-27 03:04:53,023][105620] Updated weights for policy 1, policy_version 1607716 (0.0009) [2023-12-27 03:04:53,084][105620] Updated weights for policy 1, policy_version 1607726 (0.0010) [2023-12-27 03:04:53,144][105620] Updated weights for policy 1, policy_version 1607736 (0.0008) [2023-12-27 03:04:53,442][105692] Updated weights for policy 0, policy_version 1604363 (0.0009) [2023-12-27 03:04:53,503][105692] Updated weights for policy 0, policy_version 1604373 (0.0010) [2023-12-27 03:04:53,551][105692] Updated weights for policy 0, policy_version 1604383 (0.0010) [2023-12-27 03:04:53,825][105620] Updated weights for policy 1, policy_version 1607746 (0.0008) [2023-12-27 03:04:53,877][105620] Updated weights for policy 1, policy_version 1607756 (0.0008) [2023-12-27 03:04:53,933][105620] Updated weights for policy 1, policy_version 1607766 (0.0009) [2023-12-27 03:04:53,986][105620] Updated weights for policy 1, policy_version 1607776 (0.0009) [2023-12-27 03:04:54,149][105692] Updated weights for policy 0, policy_version 1604393 (0.0009) [2023-12-27 03:04:54,215][105692] Updated weights for policy 0, policy_version 1604403 (0.0009) [2023-12-27 03:04:54,268][105692] Updated weights for policy 0, policy_version 1604413 (0.0005) [2023-12-27 03:04:54,319][105692] Updated weights for policy 0, policy_version 1604423 (0.0005) [2023-12-27 03:04:54,849][105620] Updated weights for policy 1, policy_version 1607786 (0.0008) [2023-12-27 03:04:54,902][105620] Updated weights for policy 1, policy_version 1607796 (0.0008) [2023-12-27 03:04:54,925][105692] Updated weights for policy 0, policy_version 1604433 (0.0006) [2023-12-27 03:04:54,959][105620] Updated weights for policy 1, policy_version 1607806 (0.0006) [2023-12-27 03:04:54,983][105692] Updated weights for policy 0, policy_version 1604443 (0.0009) [2023-12-27 03:04:55,044][105692] Updated weights for policy 0, policy_version 1604453 (0.0007) [2023-12-27 03:04:55,640][105620] Updated weights for policy 1, policy_version 1607816 (0.0009) [2023-12-27 03:04:55,688][105620] Updated weights for policy 1, policy_version 1607826 (0.0009) [2023-12-27 03:04:55,748][105620] Updated weights for policy 1, policy_version 1607836 (0.0008) [2023-12-27 03:04:55,785][105692] Updated weights for policy 0, policy_version 1604463 (0.0007) [2023-12-27 03:04:55,833][105692] Updated weights for policy 0, policy_version 1604473 (0.0008) [2023-12-27 03:04:55,883][105692] Updated weights for policy 0, policy_version 1604483 (0.0006) [2023-12-27 03:04:56,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 822468608. Throughput: 0: 9747.8, 1: 9810.8. Samples: 822473224. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-12-27 03:04:56,063][104569] Avg episode reward: [(0, '8626.723'), (1, '9172.936')] [2023-12-27 03:04:56,534][105692] Updated weights for policy 0, policy_version 1604493 (0.0007) [2023-12-27 03:04:56,565][105620] Updated weights for policy 1, policy_version 1607846 (0.0008) [2023-12-27 03:04:56,591][105692] Updated weights for policy 0, policy_version 1604503 (0.0007) [2023-12-27 03:04:56,622][105620] Updated weights for policy 1, policy_version 1607856 (0.0007) [2023-12-27 03:04:56,653][105692] Updated weights for policy 0, policy_version 1604513 (0.0006) [2023-12-27 03:04:56,679][105620] Updated weights for policy 1, policy_version 1607866 (0.0007) [2023-12-27 03:04:57,287][105692] Updated weights for policy 0, policy_version 1604523 (0.0006) [2023-12-27 03:04:57,340][105692] Updated weights for policy 0, policy_version 1604533 (0.0006) [2023-12-27 03:04:57,413][105692] Updated weights for policy 0, policy_version 1604543 (0.0009) [2023-12-27 03:04:57,470][105620] Updated weights for policy 1, policy_version 1607876 (0.0006) [2023-12-27 03:04:57,524][105620] Updated weights for policy 1, policy_version 1607886 (0.0006) [2023-12-27 03:04:57,578][105620] Updated weights for policy 1, policy_version 1607896 (0.0010) [2023-12-27 03:04:58,120][105692] Updated weights for policy 0, policy_version 1604553 (0.0010) [2023-12-27 03:04:58,190][105692] Updated weights for policy 0, policy_version 1604563 (0.0011) [2023-12-27 03:04:58,253][105692] Updated weights for policy 0, policy_version 1604573 (0.0010) [2023-12-27 03:04:58,302][105620] Updated weights for policy 1, policy_version 1607906 (0.0009) [2023-12-27 03:04:58,319][105692] Updated weights for policy 0, policy_version 1604583 (0.0010) [2023-12-27 03:04:58,365][105620] Updated weights for policy 1, policy_version 1607916 (0.0009) [2023-12-27 03:04:58,428][105620] Updated weights for policy 1, policy_version 1607926 (0.0009) [2023-12-27 03:04:58,494][105620] Updated weights for policy 1, policy_version 1607936 (0.0007) [2023-12-27 03:04:59,110][105620] Updated weights for policy 1, policy_version 1607946 (0.0005) [2023-12-27 03:04:59,142][105692] Updated weights for policy 0, policy_version 1604593 (0.0009) [2023-12-27 03:04:59,167][105620] Updated weights for policy 1, policy_version 1607956 (0.0006) [2023-12-27 03:04:59,188][105692] Updated weights for policy 0, policy_version 1604603 (0.0009) [2023-12-27 03:04:59,221][105620] Updated weights for policy 1, policy_version 1607966 (0.0006) [2023-12-27 03:04:59,252][105692] Updated weights for policy 0, policy_version 1604613 (0.0007) [2023-12-27 03:04:59,816][105620] Updated weights for policy 1, policy_version 1607976 (0.0009) [2023-12-27 03:04:59,883][105620] Updated weights for policy 1, policy_version 1607986 (0.0007) [2023-12-27 03:04:59,949][105620] Updated weights for policy 1, policy_version 1607996 (0.0008) [2023-12-27 03:05:00,103][105692] Updated weights for policy 0, policy_version 1604623 (0.0009) [2023-12-27 03:05:00,169][105692] Updated weights for policy 0, policy_version 1604633 (0.0009) [2023-12-27 03:05:00,235][105692] Updated weights for policy 0, policy_version 1604643 (0.0010) [2023-12-27 03:05:00,669][105620] Updated weights for policy 1, policy_version 1608006 (0.0009) [2023-12-27 03:05:00,718][105620] Updated weights for policy 1, policy_version 1608016 (0.0009) [2023-12-27 03:05:00,765][105620] Updated weights for policy 1, policy_version 1608026 (0.0009) [2023-12-27 03:05:00,936][105692] Updated weights for policy 0, policy_version 1604653 (0.0008) [2023-12-27 03:05:00,986][105692] Updated weights for policy 0, policy_version 1604663 (0.0009) [2023-12-27 03:05:01,041][105692] Updated weights for policy 0, policy_version 1604673 (0.0009) [2023-12-27 03:05:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 822558720. Throughput: 0: 9806.4, 1: 9767.9. Samples: 822531704. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:05:01,063][104569] Avg episode reward: [(0, '8717.238'), (1, '9082.432')] [2023-12-27 03:05:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001608032_411713536.pth... [2023-12-27 03:05:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001606912_411426816.pth [2023-12-27 03:05:01,077][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001604680_410853376.pth... [2023-12-27 03:05:01,080][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001603528_410558464.pth [2023-12-27 03:05:01,573][105620] Updated weights for policy 1, policy_version 1608036 (0.0009) [2023-12-27 03:05:01,633][105620] Updated weights for policy 1, policy_version 1608046 (0.0009) [2023-12-27 03:05:01,695][105620] Updated weights for policy 1, policy_version 1608056 (0.0010) [2023-12-27 03:05:01,784][105692] Updated weights for policy 0, policy_version 1604683 (0.0009) [2023-12-27 03:05:01,844][105692] Updated weights for policy 0, policy_version 1604693 (0.0009) [2023-12-27 03:05:01,909][105692] Updated weights for policy 0, policy_version 1604703 (0.0008) [2023-12-27 03:05:02,506][105620] Updated weights for policy 1, policy_version 1608066 (0.0009) [2023-12-27 03:05:02,560][105620] Updated weights for policy 1, policy_version 1608076 (0.0008) [2023-12-27 03:05:02,578][105692] Updated weights for policy 0, policy_version 1604713 (0.0008) [2023-12-27 03:05:02,621][105620] Updated weights for policy 1, policy_version 1608086 (0.0009) [2023-12-27 03:05:02,636][105692] Updated weights for policy 0, policy_version 1604723 (0.0006) [2023-12-27 03:05:02,673][105620] Updated weights for policy 1, policy_version 1608096 (0.0008) [2023-12-27 03:05:02,687][105692] Updated weights for policy 0, policy_version 1604733 (0.0006) [2023-12-27 03:05:02,739][105692] Updated weights for policy 0, policy_version 1604743 (0.0005) [2023-12-27 03:05:03,360][105692] Updated weights for policy 0, policy_version 1604753 (0.0006) [2023-12-27 03:05:03,421][105692] Updated weights for policy 0, policy_version 1604763 (0.0008) [2023-12-27 03:05:03,434][105620] Updated weights for policy 1, policy_version 1608106 (0.0005) [2023-12-27 03:05:03,468][105692] Updated weights for policy 0, policy_version 1604773 (0.0008) [2023-12-27 03:05:03,480][105620] Updated weights for policy 1, policy_version 1608116 (0.0005) [2023-12-27 03:05:03,534][105620] Updated weights for policy 1, policy_version 1608126 (0.0009) [2023-12-27 03:05:04,115][105692] Updated weights for policy 0, policy_version 1604783 (0.0009) [2023-12-27 03:05:04,178][105692] Updated weights for policy 0, policy_version 1604793 (0.0009) [2023-12-27 03:05:04,240][105692] Updated weights for policy 0, policy_version 1604803 (0.0009) [2023-12-27 03:05:04,329][105620] Updated weights for policy 1, policy_version 1608136 (0.0009) [2023-12-27 03:05:04,382][105620] Updated weights for policy 1, policy_version 1608146 (0.0009) [2023-12-27 03:05:04,436][105620] Updated weights for policy 1, policy_version 1608156 (0.0009) [2023-12-27 03:05:04,967][105692] Updated weights for policy 0, policy_version 1604813 (0.0009) [2023-12-27 03:05:05,023][105692] Updated weights for policy 0, policy_version 1604823 (0.0009) [2023-12-27 03:05:05,070][105692] Updated weights for policy 0, policy_version 1604833 (0.0009) [2023-12-27 03:05:05,203][105620] Updated weights for policy 1, policy_version 1608166 (0.0009) [2023-12-27 03:05:05,267][105620] Updated weights for policy 1, policy_version 1608176 (0.0009) [2023-12-27 03:05:05,319][105620] Updated weights for policy 1, policy_version 1608186 (0.0009) [2023-12-27 03:05:05,826][105692] Updated weights for policy 0, policy_version 1604843 (0.0009) [2023-12-27 03:05:05,884][105692] Updated weights for policy 0, policy_version 1604853 (0.0009) [2023-12-27 03:05:05,938][105692] Updated weights for policy 0, policy_version 1604863 (0.0010) [2023-12-27 03:05:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 822657024. Throughput: 0: 9809.2, 1: 9623.7. Samples: 822646064. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:05:06,063][104569] Avg episode reward: [(0, '8444.979'), (1, '9080.845')] [2023-12-27 03:05:06,085][105620] Updated weights for policy 1, policy_version 1608196 (0.0008) [2023-12-27 03:05:06,160][105620] Updated weights for policy 1, policy_version 1608206 (0.0009) [2023-12-27 03:05:06,223][105620] Updated weights for policy 1, policy_version 1608216 (0.0009) [2023-12-27 03:05:06,741][105692] Updated weights for policy 0, policy_version 1604873 (0.0011) [2023-12-27 03:05:06,806][105692] Updated weights for policy 0, policy_version 1604883 (0.0008) [2023-12-27 03:05:06,865][105692] Updated weights for policy 0, policy_version 1604893 (0.0008) [2023-12-27 03:05:06,936][105692] Updated weights for policy 0, policy_version 1604903 (0.0008) [2023-12-27 03:05:06,984][105620] Updated weights for policy 1, policy_version 1608226 (0.0008) [2023-12-27 03:05:07,048][105620] Updated weights for policy 1, policy_version 1608236 (0.0008) [2023-12-27 03:05:07,094][105620] Updated weights for policy 1, policy_version 1608246 (0.0008) [2023-12-27 03:05:07,146][105620] Updated weights for policy 1, policy_version 1608256 (0.0008) [2023-12-27 03:05:07,668][105692] Updated weights for policy 0, policy_version 1604913 (0.0010) [2023-12-27 03:05:07,730][105692] Updated weights for policy 0, policy_version 1604923 (0.0010) [2023-12-27 03:05:07,787][105692] Updated weights for policy 0, policy_version 1604933 (0.0010) [2023-12-27 03:05:07,926][105620] Updated weights for policy 1, policy_version 1608266 (0.0009) [2023-12-27 03:05:07,989][105620] Updated weights for policy 1, policy_version 1608276 (0.0009) [2023-12-27 03:05:08,049][105620] Updated weights for policy 1, policy_version 1608286 (0.0007) [2023-12-27 03:05:08,509][105692] Updated weights for policy 0, policy_version 1604943 (0.0010) [2023-12-27 03:05:08,561][105692] Updated weights for policy 0, policy_version 1604953 (0.0010) [2023-12-27 03:05:08,623][105692] Updated weights for policy 0, policy_version 1604963 (0.0010) [2023-12-27 03:05:08,702][105620] Updated weights for policy 1, policy_version 1608296 (0.0007) [2023-12-27 03:05:08,753][105620] Updated weights for policy 1, policy_version 1608306 (0.0008) [2023-12-27 03:05:08,805][105620] Updated weights for policy 1, policy_version 1608316 (0.0007) [2023-12-27 03:05:09,375][105692] Updated weights for policy 0, policy_version 1604973 (0.0010) [2023-12-27 03:05:09,447][105692] Updated weights for policy 0, policy_version 1604983 (0.0011) [2023-12-27 03:05:09,509][105692] Updated weights for policy 0, policy_version 1604993 (0.0010) [2023-12-27 03:05:09,583][105620] Updated weights for policy 1, policy_version 1608326 (0.0006) [2023-12-27 03:05:09,640][105620] Updated weights for policy 1, policy_version 1608336 (0.0006) [2023-12-27 03:05:09,693][105620] Updated weights for policy 1, policy_version 1608346 (0.0006) [2023-12-27 03:05:10,260][105692] Updated weights for policy 0, policy_version 1605003 (0.0010) [2023-12-27 03:05:10,320][105692] Updated weights for policy 0, policy_version 1605013 (0.0010) [2023-12-27 03:05:10,379][105692] Updated weights for policy 0, policy_version 1605023 (0.0010) [2023-12-27 03:05:10,416][105620] Updated weights for policy 1, policy_version 1608356 (0.0007) [2023-12-27 03:05:10,464][105620] Updated weights for policy 1, policy_version 1608366 (0.0008) [2023-12-27 03:05:10,513][105620] Updated weights for policy 1, policy_version 1608376 (0.0008) [2023-12-27 03:05:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 822747136. Throughput: 0: 9670.9, 1: 9661.3. Samples: 822758184. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:05:11,062][104569] Avg episode reward: [(0, '8079.246'), (1, '8993.631')] [2023-12-27 03:05:11,175][105692] Updated weights for policy 0, policy_version 1605033 (0.0010) [2023-12-27 03:05:11,241][105692] Updated weights for policy 0, policy_version 1605043 (0.0008) [2023-12-27 03:05:11,278][105620] Updated weights for policy 1, policy_version 1608386 (0.0008) [2023-12-27 03:05:11,303][105692] Updated weights for policy 0, policy_version 1605053 (0.0009) [2023-12-27 03:05:11,337][105620] Updated weights for policy 1, policy_version 1608396 (0.0007) [2023-12-27 03:05:11,371][105692] Updated weights for policy 0, policy_version 1605063 (0.0009) [2023-12-27 03:05:11,405][105620] Updated weights for policy 1, policy_version 1608406 (0.0008) [2023-12-27 03:05:11,453][105620] Updated weights for policy 1, policy_version 1608416 (0.0009) [2023-12-27 03:05:12,146][105692] Updated weights for policy 0, policy_version 1605073 (0.0010) [2023-12-27 03:05:12,168][105620] Updated weights for policy 1, policy_version 1608426 (0.0007) [2023-12-27 03:05:12,206][105692] Updated weights for policy 0, policy_version 1605083 (0.0011) [2023-12-27 03:05:12,234][105620] Updated weights for policy 1, policy_version 1608436 (0.0008) [2023-12-27 03:05:12,258][105692] Updated weights for policy 0, policy_version 1605093 (0.0010) [2023-12-27 03:05:12,296][105620] Updated weights for policy 1, policy_version 1608446 (0.0008) [2023-12-27 03:05:12,889][105620] Updated weights for policy 1, policy_version 1608456 (0.0008) [2023-12-27 03:05:12,940][105620] Updated weights for policy 1, policy_version 1608466 (0.0008) [2023-12-27 03:05:12,997][105620] Updated weights for policy 1, policy_version 1608476 (0.0008) [2023-12-27 03:05:13,018][105692] Updated weights for policy 0, policy_version 1605103 (0.0008) [2023-12-27 03:05:13,067][105692] Updated weights for policy 0, policy_version 1605113 (0.0005) [2023-12-27 03:05:13,128][105692] Updated weights for policy 0, policy_version 1605123 (0.0008) [2023-12-27 03:05:13,660][105620] Updated weights for policy 1, policy_version 1608486 (0.0006) [2023-12-27 03:05:13,724][105620] Updated weights for policy 1, policy_version 1608496 (0.0005) [2023-12-27 03:05:13,783][105620] Updated weights for policy 1, policy_version 1608506 (0.0005) [2023-12-27 03:05:13,796][105692] Updated weights for policy 0, policy_version 1605133 (0.0010) [2023-12-27 03:05:13,855][105692] Updated weights for policy 0, policy_version 1605143 (0.0007) [2023-12-27 03:05:13,900][105692] Updated weights for policy 0, policy_version 1605153 (0.0010) [2023-12-27 03:05:14,280][105620] Updated weights for policy 1, policy_version 1608516 (0.0005) [2023-12-27 03:05:14,337][105620] Updated weights for policy 1, policy_version 1608526 (0.0006) [2023-12-27 03:05:14,395][105620] Updated weights for policy 1, policy_version 1608536 (0.0010) [2023-12-27 03:05:14,586][105692] Updated weights for policy 0, policy_version 1605163 (0.0009) [2023-12-27 03:05:14,650][105692] Updated weights for policy 0, policy_version 1605173 (0.0005) [2023-12-27 03:05:14,717][105692] Updated weights for policy 0, policy_version 1605183 (0.0005) [2023-12-27 03:05:15,029][105620] Updated weights for policy 1, policy_version 1608546 (0.0006) [2023-12-27 03:05:15,103][105620] Updated weights for policy 1, policy_version 1608556 (0.0008) [2023-12-27 03:05:15,173][105620] Updated weights for policy 1, policy_version 1608566 (0.0008) [2023-12-27 03:05:15,241][105620] Updated weights for policy 1, policy_version 1608576 (0.0008) [2023-12-27 03:05:15,325][105692] Updated weights for policy 0, policy_version 1605193 (0.0008) [2023-12-27 03:05:15,379][105692] Updated weights for policy 0, policy_version 1605203 (0.0011) [2023-12-27 03:05:15,449][105692] Updated weights for policy 0, policy_version 1605213 (0.0011) [2023-12-27 03:05:15,508][105692] Updated weights for policy 0, policy_version 1605223 (0.0011) [2023-12-27 03:05:15,972][105620] Updated weights for policy 1, policy_version 1608586 (0.0008) [2023-12-27 03:05:16,028][105620] Updated weights for policy 1, policy_version 1608596 (0.0008) [2023-12-27 03:05:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 822845440. Throughput: 0: 9605.5, 1: 9668.9. Samples: 822817524. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:05:16,062][104569] Avg episode reward: [(0, '7990.084'), (1, '8995.372')] [2023-12-27 03:05:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001605224_410992640.pth... [2023-12-27 03:05:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001604104_410705920.pth [2023-12-27 03:05:16,080][105620] Updated weights for policy 1, policy_version 1608606 (0.0008) [2023-12-27 03:05:16,089][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001608608_411860992.pth... [2023-12-27 03:05:16,093][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001607488_411574272.pth [2023-12-27 03:05:16,238][105692] Updated weights for policy 0, policy_version 1605233 (0.0010) [2023-12-27 03:05:16,282][105692] Updated weights for policy 0, policy_version 1605243 (0.0010) [2023-12-27 03:05:16,340][105692] Updated weights for policy 0, policy_version 1605253 (0.0010) [2023-12-27 03:05:16,840][105620] Updated weights for policy 1, policy_version 1608616 (0.0010) [2023-12-27 03:05:16,893][105620] Updated weights for policy 1, policy_version 1608626 (0.0009) [2023-12-27 03:05:16,946][105620] Updated weights for policy 1, policy_version 1608636 (0.0010) [2023-12-27 03:05:16,953][105586] KL-divergence is very high: 102.2058 [2023-12-27 03:05:16,969][105692] Updated weights for policy 0, policy_version 1605263 (0.0007) [2023-12-27 03:05:17,026][105692] Updated weights for policy 0, policy_version 1605273 (0.0005) [2023-12-27 03:05:17,085][105692] Updated weights for policy 0, policy_version 1605283 (0.0009) [2023-12-27 03:05:17,714][105620] Updated weights for policy 1, policy_version 1608646 (0.0009) [2023-12-27 03:05:17,766][105620] Updated weights for policy 1, policy_version 1608656 (0.0008) [2023-12-27 03:05:17,784][105692] Updated weights for policy 0, policy_version 1605293 (0.0011) [2023-12-27 03:05:17,817][105620] Updated weights for policy 1, policy_version 1608666 (0.0006) [2023-12-27 03:05:17,842][105692] Updated weights for policy 0, policy_version 1605303 (0.0010) [2023-12-27 03:05:17,890][105692] Updated weights for policy 0, policy_version 1605313 (0.0010) [2023-12-27 03:05:18,581][105620] Updated weights for policy 1, policy_version 1608676 (0.0006) [2023-12-27 03:05:18,620][105692] Updated weights for policy 0, policy_version 1605323 (0.0010) [2023-12-27 03:05:18,645][105620] Updated weights for policy 1, policy_version 1608686 (0.0005) [2023-12-27 03:05:18,675][105692] Updated weights for policy 0, policy_version 1605333 (0.0010) [2023-12-27 03:05:18,713][105620] Updated weights for policy 1, policy_version 1608696 (0.0005) [2023-12-27 03:05:18,734][105692] Updated weights for policy 0, policy_version 1605343 (0.0011) [2023-12-27 03:05:19,427][105620] Updated weights for policy 1, policy_version 1608706 (0.0007) [2023-12-27 03:05:19,482][105692] Updated weights for policy 0, policy_version 1605353 (0.0011) [2023-12-27 03:05:19,487][105620] Updated weights for policy 1, policy_version 1608716 (0.0008) [2023-12-27 03:05:19,547][105692] Updated weights for policy 0, policy_version 1605363 (0.0011) [2023-12-27 03:05:19,558][105620] Updated weights for policy 1, policy_version 1608726 (0.0008) [2023-12-27 03:05:19,608][105692] Updated weights for policy 0, policy_version 1605373 (0.0009) [2023-12-27 03:05:19,622][105620] Updated weights for policy 1, policy_version 1608736 (0.0007) [2023-12-27 03:05:19,671][105692] Updated weights for policy 0, policy_version 1605383 (0.0011) [2023-12-27 03:05:20,291][105620] Updated weights for policy 1, policy_version 1608746 (0.0008) [2023-12-27 03:05:20,350][105620] Updated weights for policy 1, policy_version 1608756 (0.0008) [2023-12-27 03:05:20,410][105620] Updated weights for policy 1, policy_version 1608766 (0.0008) [2023-12-27 03:05:20,456][105692] Updated weights for policy 0, policy_version 1605393 (0.0006) [2023-12-27 03:05:20,502][105692] Updated weights for policy 0, policy_version 1605403 (0.0005) [2023-12-27 03:05:20,567][105692] Updated weights for policy 0, policy_version 1605413 (0.0006) [2023-12-27 03:05:21,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 822943744. Throughput: 0: 9706.5, 1: 9653.5. Samples: 822936964. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:05:21,063][104569] Avg episode reward: [(0, '8626.675'), (1, '8718.767')] [2023-12-27 03:05:21,172][105620] Updated weights for policy 1, policy_version 1608776 (0.0008) [2023-12-27 03:05:21,203][105692] Updated weights for policy 0, policy_version 1605423 (0.0010) [2023-12-27 03:05:21,235][105620] Updated weights for policy 1, policy_version 1608786 (0.0008) [2023-12-27 03:05:21,267][105692] Updated weights for policy 0, policy_version 1605433 (0.0010) [2023-12-27 03:05:21,303][105620] Updated weights for policy 1, policy_version 1608796 (0.0008) [2023-12-27 03:05:21,327][105692] Updated weights for policy 0, policy_version 1605443 (0.0011) [2023-12-27 03:05:22,044][105692] Updated weights for policy 0, policy_version 1605453 (0.0008) [2023-12-27 03:05:22,072][105620] Updated weights for policy 1, policy_version 1608806 (0.0009) [2023-12-27 03:05:22,105][105692] Updated weights for policy 0, policy_version 1605463 (0.0006) [2023-12-27 03:05:22,133][105620] Updated weights for policy 1, policy_version 1608816 (0.0011) [2023-12-27 03:05:22,169][105692] Updated weights for policy 0, policy_version 1605473 (0.0011) [2023-12-27 03:05:22,193][105620] Updated weights for policy 1, policy_version 1608826 (0.0011) [2023-12-27 03:05:22,861][105692] Updated weights for policy 0, policy_version 1605483 (0.0011) [2023-12-27 03:05:22,920][105692] Updated weights for policy 0, policy_version 1605493 (0.0011) [2023-12-27 03:05:22,984][105620] Updated weights for policy 1, policy_version 1608836 (0.0011) [2023-12-27 03:05:22,984][105692] Updated weights for policy 0, policy_version 1605503 (0.0006) [2023-12-27 03:05:23,041][105620] Updated weights for policy 1, policy_version 1608846 (0.0011) [2023-12-27 03:05:23,105][105620] Updated weights for policy 1, policy_version 1608856 (0.0011) [2023-12-27 03:05:23,591][105692] Updated weights for policy 0, policy_version 1605513 (0.0005) [2023-12-27 03:05:23,637][105692] Updated weights for policy 0, policy_version 1605523 (0.0005) [2023-12-27 03:05:23,686][105692] Updated weights for policy 0, policy_version 1605533 (0.0005) [2023-12-27 03:05:23,741][105620] Updated weights for policy 1, policy_version 1608866 (0.0009) [2023-12-27 03:05:23,743][105692] Updated weights for policy 0, policy_version 1605543 (0.0007) [2023-12-27 03:05:23,801][105620] Updated weights for policy 1, policy_version 1608876 (0.0006) [2023-12-27 03:05:23,861][105620] Updated weights for policy 1, policy_version 1608886 (0.0005) [2023-12-27 03:05:23,911][105620] Updated weights for policy 1, policy_version 1608896 (0.0008) [2023-12-27 03:05:24,481][105692] Updated weights for policy 0, policy_version 1605553 (0.0009) [2023-12-27 03:05:24,535][105692] Updated weights for policy 0, policy_version 1605563 (0.0008) [2023-12-27 03:05:24,587][105692] Updated weights for policy 0, policy_version 1605573 (0.0008) [2023-12-27 03:05:24,603][105620] Updated weights for policy 1, policy_version 1608906 (0.0006) [2023-12-27 03:05:24,667][105620] Updated weights for policy 1, policy_version 1608916 (0.0008) [2023-12-27 03:05:24,718][105620] Updated weights for policy 1, policy_version 1608926 (0.0009) [2023-12-27 03:05:25,349][105692] Updated weights for policy 0, policy_version 1605583 (0.0009) [2023-12-27 03:05:25,406][105692] Updated weights for policy 0, policy_version 1605593 (0.0009) [2023-12-27 03:05:25,460][105620] Updated weights for policy 1, policy_version 1608936 (0.0006) [2023-12-27 03:05:25,469][105692] Updated weights for policy 0, policy_version 1605603 (0.0008) [2023-12-27 03:05:25,508][105620] Updated weights for policy 1, policy_version 1608946 (0.0006) [2023-12-27 03:05:25,555][105620] Updated weights for policy 1, policy_version 1608956 (0.0009) [2023-12-27 03:05:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 823042048. Throughput: 0: 9793.4, 1: 9603.1. Samples: 823052436. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:05:26,063][104569] Avg episode reward: [(0, '8533.534'), (1, '8809.870')] [2023-12-27 03:05:26,208][105692] Updated weights for policy 0, policy_version 1605613 (0.0007) [2023-12-27 03:05:26,251][105692] Updated weights for policy 0, policy_version 1605623 (0.0008) [2023-12-27 03:05:26,306][105692] Updated weights for policy 0, policy_version 1605633 (0.0005) [2023-12-27 03:05:26,333][105620] Updated weights for policy 1, policy_version 1608966 (0.0010) [2023-12-27 03:05:26,385][105620] Updated weights for policy 1, policy_version 1608976 (0.0010) [2023-12-27 03:05:26,440][105620] Updated weights for policy 1, policy_version 1608986 (0.0010) [2023-12-27 03:05:26,985][105692] Updated weights for policy 0, policy_version 1605643 (0.0007) [2023-12-27 03:05:27,032][105692] Updated weights for policy 0, policy_version 1605653 (0.0008) [2023-12-27 03:05:27,077][105692] Updated weights for policy 0, policy_version 1605663 (0.0007) [2023-12-27 03:05:27,194][105620] Updated weights for policy 1, policy_version 1608996 (0.0010) [2023-12-27 03:05:27,243][105620] Updated weights for policy 1, policy_version 1609006 (0.0008) [2023-12-27 03:05:27,298][105620] Updated weights for policy 1, policy_version 1609016 (0.0010) [2023-12-27 03:05:27,905][105692] Updated weights for policy 0, policy_version 1605673 (0.0007) [2023-12-27 03:05:27,936][105620] Updated weights for policy 1, policy_version 1609026 (0.0010) [2023-12-27 03:05:27,954][105692] Updated weights for policy 0, policy_version 1605683 (0.0009) [2023-12-27 03:05:27,997][105620] Updated weights for policy 1, policy_version 1609036 (0.0005) [2023-12-27 03:05:28,011][105692] Updated weights for policy 0, policy_version 1605693 (0.0009) [2023-12-27 03:05:28,068][105620] Updated weights for policy 1, policy_version 1609046 (0.0005) [2023-12-27 03:05:28,074][105692] Updated weights for policy 0, policy_version 1605703 (0.0008) [2023-12-27 03:05:28,135][105620] Updated weights for policy 1, policy_version 1609056 (0.0007) [2023-12-27 03:05:28,641][105620] Updated weights for policy 1, policy_version 1609066 (0.0006) [2023-12-27 03:05:28,700][105620] Updated weights for policy 1, policy_version 1609076 (0.0009) [2023-12-27 03:05:28,766][105620] Updated weights for policy 1, policy_version 1609086 (0.0009) [2023-12-27 03:05:28,915][105692] Updated weights for policy 0, policy_version 1605713 (0.0010) [2023-12-27 03:05:28,973][105692] Updated weights for policy 0, policy_version 1605723 (0.0009) [2023-12-27 03:05:29,032][105692] Updated weights for policy 0, policy_version 1605733 (0.0005) [2023-12-27 03:05:29,434][105620] Updated weights for policy 1, policy_version 1609096 (0.0009) [2023-12-27 03:05:29,483][105620] Updated weights for policy 1, policy_version 1609106 (0.0008) [2023-12-27 03:05:29,529][105620] Updated weights for policy 1, policy_version 1609116 (0.0009) [2023-12-27 03:05:29,766][105692] Updated weights for policy 0, policy_version 1605743 (0.0009) [2023-12-27 03:05:29,824][105692] Updated weights for policy 0, policy_version 1605753 (0.0008) [2023-12-27 03:05:29,882][105692] Updated weights for policy 0, policy_version 1605763 (0.0009) [2023-12-27 03:05:30,266][105620] Updated weights for policy 1, policy_version 1609126 (0.0009) [2023-12-27 03:05:30,326][105620] Updated weights for policy 1, policy_version 1609136 (0.0009) [2023-12-27 03:05:30,389][105620] Updated weights for policy 1, policy_version 1609146 (0.0009) [2023-12-27 03:05:30,592][105692] Updated weights for policy 0, policy_version 1605773 (0.0009) [2023-12-27 03:05:30,643][105692] Updated weights for policy 0, policy_version 1605783 (0.0009) [2023-12-27 03:05:30,690][105692] Updated weights for policy 0, policy_version 1605793 (0.0008) [2023-12-27 03:05:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 823140352. Throughput: 0: 9781.4, 1: 9633.3. Samples: 823111352. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:05:31,062][104569] Avg episode reward: [(0, '8261.743'), (1, '9171.975')] [2023-12-27 03:05:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001605800_411140096.pth... [2023-12-27 03:05:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001609152_412000256.pth... [2023-12-27 03:05:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001604680_410853376.pth [2023-12-27 03:05:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001608032_411713536.pth [2023-12-27 03:05:31,166][105620] Updated weights for policy 1, policy_version 1609156 (0.0009) [2023-12-27 03:05:31,227][105620] Updated weights for policy 1, policy_version 1609166 (0.0009) [2023-12-27 03:05:31,292][105620] Updated weights for policy 1, policy_version 1609176 (0.0008) [2023-12-27 03:05:31,471][105692] Updated weights for policy 0, policy_version 1605803 (0.0008) [2023-12-27 03:05:31,526][105692] Updated weights for policy 0, policy_version 1605813 (0.0009) [2023-12-27 03:05:31,579][105692] Updated weights for policy 0, policy_version 1605823 (0.0009) [2023-12-27 03:05:32,106][105620] Updated weights for policy 1, policy_version 1609186 (0.0009) [2023-12-27 03:05:32,159][105620] Updated weights for policy 1, policy_version 1609196 (0.0010) [2023-12-27 03:05:32,199][105692] Updated weights for policy 0, policy_version 1605833 (0.0009) [2023-12-27 03:05:32,210][105620] Updated weights for policy 1, policy_version 1609206 (0.0009) [2023-12-27 03:05:32,253][105692] Updated weights for policy 0, policy_version 1605843 (0.0006) [2023-12-27 03:05:32,260][105620] Updated weights for policy 1, policy_version 1609216 (0.0009) [2023-12-27 03:05:32,316][105692] Updated weights for policy 0, policy_version 1605853 (0.0006) [2023-12-27 03:05:32,373][105692] Updated weights for policy 0, policy_version 1605863 (0.0007) [2023-12-27 03:05:33,063][105620] Updated weights for policy 1, policy_version 1609226 (0.0009) [2023-12-27 03:05:33,073][105692] Updated weights for policy 0, policy_version 1605873 (0.0007) [2023-12-27 03:05:33,120][105620] Updated weights for policy 1, policy_version 1609236 (0.0007) [2023-12-27 03:05:33,126][105692] Updated weights for policy 0, policy_version 1605883 (0.0006) [2023-12-27 03:05:33,176][105620] Updated weights for policy 1, policy_version 1609246 (0.0006) [2023-12-27 03:05:33,190][105692] Updated weights for policy 0, policy_version 1605893 (0.0008) [2023-12-27 03:05:33,925][105620] Updated weights for policy 1, policy_version 1609256 (0.0008) [2023-12-27 03:05:33,931][105692] Updated weights for policy 0, policy_version 1605903 (0.0007) [2023-12-27 03:05:33,977][105620] Updated weights for policy 1, policy_version 1609266 (0.0007) [2023-12-27 03:05:33,979][105692] Updated weights for policy 0, policy_version 1605913 (0.0006) [2023-12-27 03:05:34,029][105620] Updated weights for policy 1, policy_version 1609276 (0.0007) [2023-12-27 03:05:34,038][105692] Updated weights for policy 0, policy_version 1605923 (0.0008) [2023-12-27 03:05:34,691][105620] Updated weights for policy 1, policy_version 1609286 (0.0008) [2023-12-27 03:05:34,752][105620] Updated weights for policy 1, policy_version 1609296 (0.0009) [2023-12-27 03:05:34,805][105620] Updated weights for policy 1, policy_version 1609306 (0.0009) [2023-12-27 03:05:34,847][105692] Updated weights for policy 0, policy_version 1605933 (0.0009) [2023-12-27 03:05:34,905][105692] Updated weights for policy 0, policy_version 1605943 (0.0009) [2023-12-27 03:05:34,961][105692] Updated weights for policy 0, policy_version 1605953 (0.0009) [2023-12-27 03:05:35,573][105620] Updated weights for policy 1, policy_version 1609316 (0.0007) [2023-12-27 03:05:35,634][105620] Updated weights for policy 1, policy_version 1609326 (0.0009) [2023-12-27 03:05:35,693][105620] Updated weights for policy 1, policy_version 1609336 (0.0007) [2023-12-27 03:05:35,719][105692] Updated weights for policy 0, policy_version 1605963 (0.0009) [2023-12-27 03:05:35,767][105692] Updated weights for policy 0, policy_version 1605973 (0.0007) [2023-12-27 03:05:35,816][105692] Updated weights for policy 0, policy_version 1605983 (0.0009) [2023-12-27 03:05:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 823238656. Throughput: 0: 9715.1, 1: 9604.5. Samples: 823225944. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:05:36,062][104569] Avg episode reward: [(0, '8623.564'), (1, '8987.953')] [2023-12-27 03:05:36,378][105620] Updated weights for policy 1, policy_version 1609346 (0.0007) [2023-12-27 03:05:36,441][105620] Updated weights for policy 1, policy_version 1609356 (0.0009) [2023-12-27 03:05:36,496][105620] Updated weights for policy 1, policy_version 1609366 (0.0009) [2023-12-27 03:05:36,547][105620] Updated weights for policy 1, policy_version 1609376 (0.0009) [2023-12-27 03:05:36,643][105692] Updated weights for policy 0, policy_version 1605993 (0.0009) [2023-12-27 03:05:36,700][105692] Updated weights for policy 0, policy_version 1606003 (0.0009) [2023-12-27 03:05:36,755][105692] Updated weights for policy 0, policy_version 1606013 (0.0009) [2023-12-27 03:05:36,814][105692] Updated weights for policy 0, policy_version 1606023 (0.0009) [2023-12-27 03:05:37,280][105620] Updated weights for policy 1, policy_version 1609386 (0.0008) [2023-12-27 03:05:37,339][105620] Updated weights for policy 1, policy_version 1609396 (0.0009) [2023-12-27 03:05:37,395][105620] Updated weights for policy 1, policy_version 1609406 (0.0008) [2023-12-27 03:05:37,609][105692] Updated weights for policy 0, policy_version 1606033 (0.0009) [2023-12-27 03:05:37,675][105692] Updated weights for policy 0, policy_version 1606043 (0.0009) [2023-12-27 03:05:37,736][105692] Updated weights for policy 0, policy_version 1606053 (0.0008) [2023-12-27 03:05:38,184][105620] Updated weights for policy 1, policy_version 1609416 (0.0008) [2023-12-27 03:05:38,243][105620] Updated weights for policy 1, policy_version 1609426 (0.0006) [2023-12-27 03:05:38,310][105620] Updated weights for policy 1, policy_version 1609436 (0.0009) [2023-12-27 03:05:38,479][105692] Updated weights for policy 0, policy_version 1606063 (0.0008) [2023-12-27 03:05:38,535][105692] Updated weights for policy 0, policy_version 1606073 (0.0008) [2023-12-27 03:05:38,588][105692] Updated weights for policy 0, policy_version 1606083 (0.0008) [2023-12-27 03:05:39,033][105620] Updated weights for policy 1, policy_version 1609446 (0.0009) [2023-12-27 03:05:39,080][105620] Updated weights for policy 1, policy_version 1609456 (0.0009) [2023-12-27 03:05:39,127][105620] Updated weights for policy 1, policy_version 1609466 (0.0009) [2023-12-27 03:05:39,362][105692] Updated weights for policy 0, policy_version 1606093 (0.0010) [2023-12-27 03:05:39,416][105692] Updated weights for policy 0, policy_version 1606103 (0.0009) [2023-12-27 03:05:39,470][105692] Updated weights for policy 0, policy_version 1606113 (0.0008) [2023-12-27 03:05:39,918][105620] Updated weights for policy 1, policy_version 1609476 (0.0009) [2023-12-27 03:05:39,982][105620] Updated weights for policy 1, policy_version 1609486 (0.0009) [2023-12-27 03:05:40,042][105620] Updated weights for policy 1, policy_version 1609496 (0.0010) [2023-12-27 03:05:40,277][105692] Updated weights for policy 0, policy_version 1606123 (0.0009) [2023-12-27 03:05:40,328][105692] Updated weights for policy 0, policy_version 1606133 (0.0008) [2023-12-27 03:05:40,379][105692] Updated weights for policy 0, policy_version 1606143 (0.0009) [2023-12-27 03:05:40,820][105620] Updated weights for policy 1, policy_version 1609506 (0.0010) [2023-12-27 03:05:40,875][105620] Updated weights for policy 1, policy_version 1609516 (0.0009) [2023-12-27 03:05:40,931][105620] Updated weights for policy 1, policy_version 1609526 (0.0009) [2023-12-27 03:05:40,981][105620] Updated weights for policy 1, policy_version 1609536 (0.0009) [2023-12-27 03:05:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 823328768. Throughput: 0: 9575.8, 1: 9603.7. Samples: 823336296. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:05:41,063][104569] Avg episode reward: [(0, '8534.916'), (1, '8899.865')] [2023-12-27 03:05:41,169][105692] Updated weights for policy 0, policy_version 1606153 (0.0010) [2023-12-27 03:05:41,219][105692] Updated weights for policy 0, policy_version 1606163 (0.0009) [2023-12-27 03:05:41,270][105692] Updated weights for policy 0, policy_version 1606173 (0.0008) [2023-12-27 03:05:41,334][105692] Updated weights for policy 0, policy_version 1606183 (0.0007) [2023-12-27 03:05:41,778][105620] Updated weights for policy 1, policy_version 1609546 (0.0009) [2023-12-27 03:05:41,847][105620] Updated weights for policy 1, policy_version 1609556 (0.0008) [2023-12-27 03:05:41,903][105620] Updated weights for policy 1, policy_version 1609566 (0.0010) [2023-12-27 03:05:42,029][105692] Updated weights for policy 0, policy_version 1606193 (0.0010) [2023-12-27 03:05:42,088][105692] Updated weights for policy 0, policy_version 1606203 (0.0011) [2023-12-27 03:05:42,151][105692] Updated weights for policy 0, policy_version 1606213 (0.0010) [2023-12-27 03:05:42,667][105620] Updated weights for policy 1, policy_version 1609576 (0.0006) [2023-12-27 03:05:42,737][105620] Updated weights for policy 1, policy_version 1609586 (0.0005) [2023-12-27 03:05:42,799][105620] Updated weights for policy 1, policy_version 1609596 (0.0006) [2023-12-27 03:05:42,892][105692] Updated weights for policy 0, policy_version 1606223 (0.0007) [2023-12-27 03:05:42,962][105692] Updated weights for policy 0, policy_version 1606233 (0.0006) [2023-12-27 03:05:43,018][105692] Updated weights for policy 0, policy_version 1606243 (0.0007) [2023-12-27 03:05:43,283][105620] Updated weights for policy 1, policy_version 1609606 (0.0005) [2023-12-27 03:05:43,328][105620] Updated weights for policy 1, policy_version 1609616 (0.0005) [2023-12-27 03:05:43,371][105620] Updated weights for policy 1, policy_version 1609626 (0.0005) [2023-12-27 03:05:43,699][105692] Updated weights for policy 0, policy_version 1606253 (0.0008) [2023-12-27 03:05:43,758][105692] Updated weights for policy 0, policy_version 1606263 (0.0010) [2023-12-27 03:05:43,816][105692] Updated weights for policy 0, policy_version 1606273 (0.0010) [2023-12-27 03:05:44,067][105620] Updated weights for policy 1, policy_version 1609636 (0.0006) [2023-12-27 03:05:44,123][105620] Updated weights for policy 1, policy_version 1609646 (0.0007) [2023-12-27 03:05:44,182][105620] Updated weights for policy 1, policy_version 1609656 (0.0008) [2023-12-27 03:05:44,532][105692] Updated weights for policy 0, policy_version 1606283 (0.0010) [2023-12-27 03:05:44,579][105692] Updated weights for policy 0, policy_version 1606293 (0.0010) [2023-12-27 03:05:44,624][105692] Updated weights for policy 0, policy_version 1606303 (0.0010) [2023-12-27 03:05:44,897][105620] Updated weights for policy 1, policy_version 1609666 (0.0008) [2023-12-27 03:05:44,961][105620] Updated weights for policy 1, policy_version 1609676 (0.0010) [2023-12-27 03:05:45,028][105620] Updated weights for policy 1, policy_version 1609686 (0.0011) [2023-12-27 03:05:45,095][105620] Updated weights for policy 1, policy_version 1609696 (0.0010) [2023-12-27 03:05:45,397][105692] Updated weights for policy 0, policy_version 1606313 (0.0010) [2023-12-27 03:05:45,459][105692] Updated weights for policy 0, policy_version 1606323 (0.0010) [2023-12-27 03:05:45,521][105692] Updated weights for policy 0, policy_version 1606333 (0.0010) [2023-12-27 03:05:45,583][105692] Updated weights for policy 0, policy_version 1606343 (0.0010) [2023-12-27 03:05:45,808][105620] Updated weights for policy 1, policy_version 1609706 (0.0010) [2023-12-27 03:05:45,870][105620] Updated weights for policy 1, policy_version 1609716 (0.0010) [2023-12-27 03:05:45,929][105620] Updated weights for policy 1, policy_version 1609726 (0.0010) [2023-12-27 03:05:46,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 823427072. Throughput: 0: 9552.6, 1: 9649.5. Samples: 823395800. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:05:46,063][104569] Avg episode reward: [(0, '8627.248'), (1, '8812.766')] [2023-12-27 03:05:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001606344_411279360.pth... [2023-12-27 03:05:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001609728_412147712.pth... [2023-12-27 03:05:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001605224_410992640.pth [2023-12-27 03:05:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001608608_411860992.pth [2023-12-27 03:05:46,322][105692] Updated weights for policy 0, policy_version 1606353 (0.0010) [2023-12-27 03:05:46,383][105692] Updated weights for policy 0, policy_version 1606363 (0.0010) [2023-12-27 03:05:46,448][105692] Updated weights for policy 0, policy_version 1606373 (0.0011) [2023-12-27 03:05:46,619][105620] Updated weights for policy 1, policy_version 1609736 (0.0010) [2023-12-27 03:05:46,681][105620] Updated weights for policy 1, policy_version 1609746 (0.0010) [2023-12-27 03:05:46,747][105620] Updated weights for policy 1, policy_version 1609756 (0.0010) [2023-12-27 03:05:47,129][105692] Updated weights for policy 0, policy_version 1606383 (0.0011) [2023-12-27 03:05:47,173][105692] Updated weights for policy 0, policy_version 1606393 (0.0010) [2023-12-27 03:05:47,221][105692] Updated weights for policy 0, policy_version 1606403 (0.0010) [2023-12-27 03:05:47,451][105620] Updated weights for policy 1, policy_version 1609766 (0.0010) [2023-12-27 03:05:47,505][105620] Updated weights for policy 1, policy_version 1609776 (0.0010) [2023-12-27 03:05:47,562][105620] Updated weights for policy 1, policy_version 1609786 (0.0010) [2023-12-27 03:05:47,928][105692] Updated weights for policy 0, policy_version 1606413 (0.0008) [2023-12-27 03:05:47,992][105692] Updated weights for policy 0, policy_version 1606423 (0.0007) [2023-12-27 03:05:48,051][105692] Updated weights for policy 0, policy_version 1606433 (0.0010) [2023-12-27 03:05:48,293][105620] Updated weights for policy 1, policy_version 1609796 (0.0010) [2023-12-27 03:05:48,351][105620] Updated weights for policy 1, policy_version 1609806 (0.0011) [2023-12-27 03:05:48,414][105620] Updated weights for policy 1, policy_version 1609816 (0.0011) [2023-12-27 03:05:48,763][105692] Updated weights for policy 0, policy_version 1606443 (0.0010) [2023-12-27 03:05:48,826][105692] Updated weights for policy 0, policy_version 1606453 (0.0007) [2023-12-27 03:05:48,895][105692] Updated weights for policy 0, policy_version 1606463 (0.0006) [2023-12-27 03:05:49,173][105620] Updated weights for policy 1, policy_version 1609826 (0.0010) [2023-12-27 03:05:49,228][105620] Updated weights for policy 1, policy_version 1609836 (0.0010) [2023-12-27 03:05:49,286][105620] Updated weights for policy 1, policy_version 1609846 (0.0010) [2023-12-27 03:05:49,350][105620] Updated weights for policy 1, policy_version 1609856 (0.0011) [2023-12-27 03:05:49,478][105692] Updated weights for policy 0, policy_version 1606473 (0.0005) [2023-12-27 03:05:49,548][105692] Updated weights for policy 0, policy_version 1606483 (0.0005) [2023-12-27 03:05:49,612][105692] Updated weights for policy 0, policy_version 1606493 (0.0008) [2023-12-27 03:05:49,664][105692] Updated weights for policy 0, policy_version 1606503 (0.0008) [2023-12-27 03:05:50,050][105620] Updated weights for policy 1, policy_version 1609866 (0.0009) [2023-12-27 03:05:50,099][105620] Updated weights for policy 1, policy_version 1609876 (0.0009) [2023-12-27 03:05:50,157][105620] Updated weights for policy 1, policy_version 1609886 (0.0010) [2023-12-27 03:05:50,332][105692] Updated weights for policy 0, policy_version 1606513 (0.0005) [2023-12-27 03:05:50,397][105692] Updated weights for policy 0, policy_version 1606523 (0.0005) [2023-12-27 03:05:50,467][105692] Updated weights for policy 0, policy_version 1606533 (0.0006) [2023-12-27 03:05:50,904][105620] Updated weights for policy 1, policy_version 1609896 (0.0009) [2023-12-27 03:05:50,952][105620] Updated weights for policy 1, policy_version 1609906 (0.0009) [2023-12-27 03:05:50,999][105620] Updated weights for policy 1, policy_version 1609916 (0.0009) [2023-12-27 03:05:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 823525376. Throughput: 0: 9578.0, 1: 9681.6. Samples: 823512744. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:05:51,063][104569] Avg episode reward: [(0, '8712.290'), (1, '8995.271')] [2023-12-27 03:05:51,131][105692] Updated weights for policy 0, policy_version 1606543 (0.0008) [2023-12-27 03:05:51,194][105692] Updated weights for policy 0, policy_version 1606553 (0.0009) [2023-12-27 03:05:51,260][105692] Updated weights for policy 0, policy_version 1606563 (0.0009) [2023-12-27 03:05:51,820][105620] Updated weights for policy 1, policy_version 1609926 (0.0009) [2023-12-27 03:05:51,878][105620] Updated weights for policy 1, policy_version 1609936 (0.0009) [2023-12-27 03:05:51,944][105620] Updated weights for policy 1, policy_version 1609946 (0.0009) [2023-12-27 03:05:52,028][105692] Updated weights for policy 0, policy_version 1606573 (0.0007) [2023-12-27 03:05:52,095][105692] Updated weights for policy 0, policy_version 1606583 (0.0006) [2023-12-27 03:05:52,156][105692] Updated weights for policy 0, policy_version 1606593 (0.0007) [2023-12-27 03:05:52,756][105620] Updated weights for policy 1, policy_version 1609956 (0.0008) [2023-12-27 03:05:52,807][105692] Updated weights for policy 0, policy_version 1606603 (0.0008) [2023-12-27 03:05:52,815][105620] Updated weights for policy 1, policy_version 1609966 (0.0006) [2023-12-27 03:05:52,872][105692] Updated weights for policy 0, policy_version 1606613 (0.0008) [2023-12-27 03:05:52,880][105620] Updated weights for policy 1, policy_version 1609976 (0.0005) [2023-12-27 03:05:52,927][105692] Updated weights for policy 0, policy_version 1606623 (0.0009) [2023-12-27 03:05:53,519][105620] Updated weights for policy 1, policy_version 1609986 (0.0006) [2023-12-27 03:05:53,568][105620] Updated weights for policy 1, policy_version 1609996 (0.0005) [2023-12-27 03:05:53,634][105620] Updated weights for policy 1, policy_version 1610006 (0.0005) [2023-12-27 03:05:53,645][105692] Updated weights for policy 0, policy_version 1606633 (0.0008) [2023-12-27 03:05:53,701][105692] Updated weights for policy 0, policy_version 1606643 (0.0008) [2023-12-27 03:05:53,702][105620] Updated weights for policy 1, policy_version 1610016 (0.0006) [2023-12-27 03:05:53,754][105692] Updated weights for policy 0, policy_version 1606653 (0.0009) [2023-12-27 03:05:53,804][105692] Updated weights for policy 0, policy_version 1606663 (0.0009) [2023-12-27 03:05:54,386][105620] Updated weights for policy 1, policy_version 1610026 (0.0010) [2023-12-27 03:05:54,440][105620] Updated weights for policy 1, policy_version 1610036 (0.0008) [2023-12-27 03:05:54,504][105620] Updated weights for policy 1, policy_version 1610046 (0.0009) [2023-12-27 03:05:54,565][105692] Updated weights for policy 0, policy_version 1606673 (0.0006) [2023-12-27 03:05:54,634][105692] Updated weights for policy 0, policy_version 1606683 (0.0005) [2023-12-27 03:05:54,702][105692] Updated weights for policy 0, policy_version 1606693 (0.0005) [2023-12-27 03:05:55,207][105620] Updated weights for policy 1, policy_version 1610056 (0.0006) [2023-12-27 03:05:55,271][105620] Updated weights for policy 1, policy_version 1610066 (0.0008) [2023-12-27 03:05:55,316][105692] Updated weights for policy 0, policy_version 1606703 (0.0007) [2023-12-27 03:05:55,329][105620] Updated weights for policy 1, policy_version 1610076 (0.0010) [2023-12-27 03:05:55,370][105692] Updated weights for policy 0, policy_version 1606713 (0.0008) [2023-12-27 03:05:55,416][105692] Updated weights for policy 0, policy_version 1606723 (0.0009) [2023-12-27 03:05:56,060][105620] Updated weights for policy 1, policy_version 1610086 (0.0008) [2023-12-27 03:05:56,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19114.7, 300 sec: 19522.0). Total num frames: 823615488. Throughput: 0: 9657.3, 1: 9698.7. Samples: 823629204. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:05:56,062][104569] Avg episode reward: [(0, '8620.098'), (1, '9084.143')] [2023-12-27 03:05:56,106][105620] Updated weights for policy 1, policy_version 1610096 (0.0008) [2023-12-27 03:05:56,152][105620] Updated weights for policy 1, policy_version 1610106 (0.0008) [2023-12-27 03:05:56,162][105692] Updated weights for policy 0, policy_version 1606733 (0.0007) [2023-12-27 03:05:56,217][105692] Updated weights for policy 0, policy_version 1606743 (0.0008) [2023-12-27 03:05:56,278][105692] Updated weights for policy 0, policy_version 1606753 (0.0009) [2023-12-27 03:05:56,934][105620] Updated weights for policy 1, policy_version 1610116 (0.0007) [2023-12-27 03:05:56,995][105620] Updated weights for policy 1, policy_version 1610126 (0.0008) [2023-12-27 03:05:57,004][105692] Updated weights for policy 0, policy_version 1606763 (0.0008) [2023-12-27 03:05:57,054][105620] Updated weights for policy 1, policy_version 1610136 (0.0007) [2023-12-27 03:05:57,057][105692] Updated weights for policy 0, policy_version 1606773 (0.0006) [2023-12-27 03:05:57,114][105692] Updated weights for policy 0, policy_version 1606783 (0.0006) [2023-12-27 03:05:57,685][105620] Updated weights for policy 1, policy_version 1610146 (0.0006) [2023-12-27 03:05:57,733][105620] Updated weights for policy 1, policy_version 1610156 (0.0007) [2023-12-27 03:05:57,778][105620] Updated weights for policy 1, policy_version 1610166 (0.0008) [2023-12-27 03:05:57,830][105620] Updated weights for policy 1, policy_version 1610176 (0.0009) [2023-12-27 03:05:57,870][105692] Updated weights for policy 0, policy_version 1606793 (0.0009) [2023-12-27 03:05:57,926][105692] Updated weights for policy 0, policy_version 1606803 (0.0007) [2023-12-27 03:05:57,982][105692] Updated weights for policy 0, policy_version 1606813 (0.0008) [2023-12-27 03:05:58,039][105692] Updated weights for policy 0, policy_version 1606823 (0.0009) [2023-12-27 03:05:58,553][105620] Updated weights for policy 1, policy_version 1610186 (0.0008) [2023-12-27 03:05:58,617][105620] Updated weights for policy 1, policy_version 1610196 (0.0007) [2023-12-27 03:05:58,680][105620] Updated weights for policy 1, policy_version 1610206 (0.0007) [2023-12-27 03:05:58,881][105692] Updated weights for policy 0, policy_version 1606833 (0.0008) [2023-12-27 03:05:58,951][105692] Updated weights for policy 0, policy_version 1606843 (0.0007) [2023-12-27 03:05:59,021][105692] Updated weights for policy 0, policy_version 1606853 (0.0009) [2023-12-27 03:05:59,470][105620] Updated weights for policy 1, policy_version 1610216 (0.0010) [2023-12-27 03:05:59,529][105620] Updated weights for policy 1, policy_version 1610226 (0.0011) [2023-12-27 03:05:59,584][105620] Updated weights for policy 1, policy_version 1610236 (0.0010) [2023-12-27 03:05:59,799][105692] Updated weights for policy 0, policy_version 1606863 (0.0009) [2023-12-27 03:05:59,860][105692] Updated weights for policy 0, policy_version 1606873 (0.0009) [2023-12-27 03:05:59,919][105692] Updated weights for policy 0, policy_version 1606883 (0.0008) [2023-12-27 03:06:00,339][105620] Updated weights for policy 1, policy_version 1610246 (0.0011) [2023-12-27 03:06:00,397][105620] Updated weights for policy 1, policy_version 1610256 (0.0011) [2023-12-27 03:06:00,455][105620] Updated weights for policy 1, policy_version 1610266 (0.0010) [2023-12-27 03:06:00,658][105692] Updated weights for policy 0, policy_version 1606893 (0.0006) [2023-12-27 03:06:00,705][105692] Updated weights for policy 0, policy_version 1606903 (0.0008) [2023-12-27 03:06:00,752][105692] Updated weights for policy 0, policy_version 1606913 (0.0010) [2023-12-27 03:06:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 823713792. Throughput: 0: 9668.5, 1: 9638.7. Samples: 823686352. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:06:01,062][104569] Avg episode reward: [(0, '8621.515'), (1, '9172.419')] [2023-12-27 03:06:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001606920_411426816.pth... [2023-12-27 03:06:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001610272_412286976.pth... [2023-12-27 03:06:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001605800_411140096.pth [2023-12-27 03:06:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001609152_412000256.pth [2023-12-27 03:06:01,192][105620] Updated weights for policy 1, policy_version 1610276 (0.0011) [2023-12-27 03:06:01,263][105620] Updated weights for policy 1, policy_version 1610286 (0.0011) [2023-12-27 03:06:01,319][105620] Updated weights for policy 1, policy_version 1610296 (0.0011) [2023-12-27 03:06:01,418][105692] Updated weights for policy 0, policy_version 1606923 (0.0008) [2023-12-27 03:06:01,474][105692] Updated weights for policy 0, policy_version 1606933 (0.0008) [2023-12-27 03:06:01,531][105692] Updated weights for policy 0, policy_version 1606943 (0.0009) [2023-12-27 03:06:02,111][105620] Updated weights for policy 1, policy_version 1610306 (0.0010) [2023-12-27 03:06:02,163][105620] Updated weights for policy 1, policy_version 1610316 (0.0009) [2023-12-27 03:06:02,219][105692] Updated weights for policy 0, policy_version 1606953 (0.0008) [2023-12-27 03:06:02,224][105620] Updated weights for policy 1, policy_version 1610326 (0.0009) [2023-12-27 03:06:02,278][105692] Updated weights for policy 0, policy_version 1606963 (0.0006) [2023-12-27 03:06:02,287][105620] Updated weights for policy 1, policy_version 1610336 (0.0009) [2023-12-27 03:06:02,335][105692] Updated weights for policy 0, policy_version 1606973 (0.0008) [2023-12-27 03:06:02,405][105692] Updated weights for policy 0, policy_version 1606983 (0.0008) [2023-12-27 03:06:03,062][105620] Updated weights for policy 1, policy_version 1610346 (0.0009) [2023-12-27 03:06:03,110][105692] Updated weights for policy 0, policy_version 1606993 (0.0009) [2023-12-27 03:06:03,112][105620] Updated weights for policy 1, policy_version 1610356 (0.0006) [2023-12-27 03:06:03,163][105692] Updated weights for policy 0, policy_version 1607003 (0.0007) [2023-12-27 03:06:03,169][105620] Updated weights for policy 1, policy_version 1610366 (0.0006) [2023-12-27 03:06:03,207][105692] Updated weights for policy 0, policy_version 1607013 (0.0007) [2023-12-27 03:06:03,885][105692] Updated weights for policy 0, policy_version 1607023 (0.0009) [2023-12-27 03:06:03,952][105692] Updated weights for policy 0, policy_version 1607033 (0.0009) [2023-12-27 03:06:03,984][105620] Updated weights for policy 1, policy_version 1610376 (0.0006) [2023-12-27 03:06:04,014][105692] Updated weights for policy 0, policy_version 1607043 (0.0007) [2023-12-27 03:06:04,041][105620] Updated weights for policy 1, policy_version 1610386 (0.0008) [2023-12-27 03:06:04,106][105620] Updated weights for policy 1, policy_version 1610396 (0.0008) [2023-12-27 03:06:04,775][105692] Updated weights for policy 0, policy_version 1607053 (0.0007) [2023-12-27 03:06:04,823][105692] Updated weights for policy 0, policy_version 1607063 (0.0005) [2023-12-27 03:06:04,836][105620] Updated weights for policy 1, policy_version 1610406 (0.0008) [2023-12-27 03:06:04,886][105620] Updated weights for policy 1, policy_version 1610416 (0.0008) [2023-12-27 03:06:04,886][105692] Updated weights for policy 0, policy_version 1607073 (0.0008) [2023-12-27 03:06:04,939][105620] Updated weights for policy 1, policy_version 1610426 (0.0008) [2023-12-27 03:06:05,600][105692] Updated weights for policy 0, policy_version 1607083 (0.0010) [2023-12-27 03:06:05,657][105692] Updated weights for policy 0, policy_version 1607093 (0.0009) [2023-12-27 03:06:05,708][105692] Updated weights for policy 0, policy_version 1607103 (0.0007) [2023-12-27 03:06:05,718][105620] Updated weights for policy 1, policy_version 1610436 (0.0009) [2023-12-27 03:06:05,769][105620] Updated weights for policy 1, policy_version 1610446 (0.0008) [2023-12-27 03:06:05,822][105620] Updated weights for policy 1, policy_version 1610456 (0.0010) [2023-12-27 03:06:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 823812096. Throughput: 0: 9598.9, 1: 9551.5. Samples: 823798732. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:06:06,063][104569] Avg episode reward: [(0, '8445.505'), (1, '9172.640')] [2023-12-27 03:06:06,327][105692] Updated weights for policy 0, policy_version 1607113 (0.0006) [2023-12-27 03:06:06,398][105692] Updated weights for policy 0, policy_version 1607123 (0.0009) [2023-12-27 03:06:06,456][105692] Updated weights for policy 0, policy_version 1607133 (0.0009) [2023-12-27 03:06:06,522][105692] Updated weights for policy 0, policy_version 1607143 (0.0008) [2023-12-27 03:06:06,632][105620] Updated weights for policy 1, policy_version 1610466 (0.0010) [2023-12-27 03:06:06,690][105620] Updated weights for policy 1, policy_version 1610476 (0.0009) [2023-12-27 03:06:06,752][105620] Updated weights for policy 1, policy_version 1610486 (0.0009) [2023-12-27 03:06:06,817][105620] Updated weights for policy 1, policy_version 1610496 (0.0008) [2023-12-27 03:06:07,284][105692] Updated weights for policy 0, policy_version 1607153 (0.0009) [2023-12-27 03:06:07,334][105692] Updated weights for policy 0, policy_version 1607163 (0.0009) [2023-12-27 03:06:07,382][105692] Updated weights for policy 0, policy_version 1607173 (0.0009) [2023-12-27 03:06:07,569][105620] Updated weights for policy 1, policy_version 1610506 (0.0009) [2023-12-27 03:06:07,626][105620] Updated weights for policy 1, policy_version 1610516 (0.0009) [2023-12-27 03:06:07,686][105620] Updated weights for policy 1, policy_version 1610526 (0.0008) [2023-12-27 03:06:08,035][105692] Updated weights for policy 0, policy_version 1607183 (0.0007) [2023-12-27 03:06:08,093][105692] Updated weights for policy 0, policy_version 1607193 (0.0006) [2023-12-27 03:06:08,147][105692] Updated weights for policy 0, policy_version 1607203 (0.0006) [2023-12-27 03:06:08,502][105620] Updated weights for policy 1, policy_version 1610536 (0.0005) [2023-12-27 03:06:08,557][105620] Updated weights for policy 1, policy_version 1610546 (0.0006) [2023-12-27 03:06:08,625][105620] Updated weights for policy 1, policy_version 1610556 (0.0008) [2023-12-27 03:06:08,768][105692] Updated weights for policy 0, policy_version 1607213 (0.0007) [2023-12-27 03:06:08,824][105692] Updated weights for policy 0, policy_version 1607223 (0.0008) [2023-12-27 03:06:08,885][105692] Updated weights for policy 0, policy_version 1607233 (0.0006) [2023-12-27 03:06:09,352][105620] Updated weights for policy 1, policy_version 1610566 (0.0008) [2023-12-27 03:06:09,420][105620] Updated weights for policy 1, policy_version 1610576 (0.0008) [2023-12-27 03:06:09,491][105620] Updated weights for policy 1, policy_version 1610586 (0.0008) [2023-12-27 03:06:09,541][105692] Updated weights for policy 0, policy_version 1607243 (0.0008) [2023-12-27 03:06:09,601][105692] Updated weights for policy 0, policy_version 1607253 (0.0008) [2023-12-27 03:06:09,663][105692] Updated weights for policy 0, policy_version 1607263 (0.0009) [2023-12-27 03:06:10,241][105620] Updated weights for policy 1, policy_version 1610596 (0.0008) [2023-12-27 03:06:10,311][105620] Updated weights for policy 1, policy_version 1610606 (0.0010) [2023-12-27 03:06:10,358][105620] Updated weights for policy 1, policy_version 1610616 (0.0009) [2023-12-27 03:06:10,446][105692] Updated weights for policy 0, policy_version 1607273 (0.0009) [2023-12-27 03:06:10,509][105692] Updated weights for policy 0, policy_version 1607283 (0.0009) [2023-12-27 03:06:10,574][105692] Updated weights for policy 0, policy_version 1607293 (0.0009) [2023-12-27 03:06:10,629][105692] Updated weights for policy 0, policy_version 1607303 (0.0010) [2023-12-27 03:06:11,038][105620] Updated weights for policy 1, policy_version 1610626 (0.0009) [2023-12-27 03:06:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 823902208. Throughput: 0: 9631.8, 1: 9521.2. Samples: 823914320. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:06:11,063][104569] Avg episode reward: [(0, '8350.062'), (1, '9263.434')] [2023-12-27 03:06:11,107][105620] Updated weights for policy 1, policy_version 1610636 (0.0009) [2023-12-27 03:06:11,174][105620] Updated weights for policy 1, policy_version 1610646 (0.0009) [2023-12-27 03:06:11,243][105620] Updated weights for policy 1, policy_version 1610656 (0.0009) [2023-12-27 03:06:11,384][105692] Updated weights for policy 0, policy_version 1607313 (0.0008) [2023-12-27 03:06:11,450][105692] Updated weights for policy 0, policy_version 1607323 (0.0011) [2023-12-27 03:06:11,513][105692] Updated weights for policy 0, policy_version 1607333 (0.0010) [2023-12-27 03:06:12,015][105620] Updated weights for policy 1, policy_version 1610666 (0.0009) [2023-12-27 03:06:12,072][105620] Updated weights for policy 1, policy_version 1610676 (0.0009) [2023-12-27 03:06:12,135][105620] Updated weights for policy 1, policy_version 1610686 (0.0008) [2023-12-27 03:06:12,337][105692] Updated weights for policy 0, policy_version 1607343 (0.0011) [2023-12-27 03:06:12,411][105692] Updated weights for policy 0, policy_version 1607353 (0.0010) [2023-12-27 03:06:12,480][105692] Updated weights for policy 0, policy_version 1607363 (0.0009) [2023-12-27 03:06:12,928][105620] Updated weights for policy 1, policy_version 1610696 (0.0009) [2023-12-27 03:06:12,982][105620] Updated weights for policy 1, policy_version 1610706 (0.0009) [2023-12-27 03:06:13,032][105620] Updated weights for policy 1, policy_version 1610716 (0.0009) [2023-12-27 03:06:13,164][105692] Updated weights for policy 0, policy_version 1607373 (0.0009) [2023-12-27 03:06:13,214][105692] Updated weights for policy 0, policy_version 1607383 (0.0008) [2023-12-27 03:06:13,274][105692] Updated weights for policy 0, policy_version 1607393 (0.0009) [2023-12-27 03:06:13,671][105620] Updated weights for policy 1, policy_version 1610726 (0.0007) [2023-12-27 03:06:13,722][105620] Updated weights for policy 1, policy_version 1610736 (0.0007) [2023-12-27 03:06:13,775][105620] Updated weights for policy 1, policy_version 1610746 (0.0007) [2023-12-27 03:06:14,005][105692] Updated weights for policy 0, policy_version 1607403 (0.0010) [2023-12-27 03:06:14,055][105692] Updated weights for policy 0, policy_version 1607413 (0.0010) [2023-12-27 03:06:14,106][105692] Updated weights for policy 0, policy_version 1607423 (0.0010) [2023-12-27 03:06:14,481][105620] Updated weights for policy 1, policy_version 1610756 (0.0007) [2023-12-27 03:06:14,543][105620] Updated weights for policy 1, policy_version 1610766 (0.0008) [2023-12-27 03:06:14,596][105620] Updated weights for policy 1, policy_version 1610776 (0.0009) [2023-12-27 03:06:14,777][105692] Updated weights for policy 0, policy_version 1607433 (0.0010) [2023-12-27 03:06:14,844][105692] Updated weights for policy 0, policy_version 1607443 (0.0010) [2023-12-27 03:06:14,916][105692] Updated weights for policy 0, policy_version 1607453 (0.0010) [2023-12-27 03:06:14,980][105692] Updated weights for policy 0, policy_version 1607463 (0.0011) [2023-12-27 03:06:15,338][105620] Updated weights for policy 1, policy_version 1610786 (0.0009) [2023-12-27 03:06:15,398][105620] Updated weights for policy 1, policy_version 1610796 (0.0008) [2023-12-27 03:06:15,457][105620] Updated weights for policy 1, policy_version 1610806 (0.0008) [2023-12-27 03:06:15,517][105620] Updated weights for policy 1, policy_version 1610816 (0.0008) [2023-12-27 03:06:15,726][105692] Updated weights for policy 0, policy_version 1607473 (0.0010) [2023-12-27 03:06:15,789][105692] Updated weights for policy 0, policy_version 1607483 (0.0006) [2023-12-27 03:06:15,856][105692] Updated weights for policy 0, policy_version 1607493 (0.0009) [2023-12-27 03:06:16,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 824000512. Throughput: 0: 9636.8, 1: 9463.8. Samples: 823970880. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:06:16,062][104569] Avg episode reward: [(0, '8530.036'), (1, '9263.604')] [2023-12-27 03:06:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001607496_411574272.pth... [2023-12-27 03:06:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001610816_412426240.pth... [2023-12-27 03:06:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001606344_411279360.pth [2023-12-27 03:06:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001609728_412147712.pth [2023-12-27 03:06:16,303][105620] Updated weights for policy 1, policy_version 1610826 (0.0005) [2023-12-27 03:06:16,369][105620] Updated weights for policy 1, policy_version 1610836 (0.0006) [2023-12-27 03:06:16,420][105620] Updated weights for policy 1, policy_version 1610846 (0.0006) [2023-12-27 03:06:16,602][105692] Updated weights for policy 0, policy_version 1607503 (0.0009) [2023-12-27 03:06:16,656][105692] Updated weights for policy 0, policy_version 1607513 (0.0009) [2023-12-27 03:06:16,703][105692] Updated weights for policy 0, policy_version 1607523 (0.0009) [2023-12-27 03:06:17,103][105620] Updated weights for policy 1, policy_version 1610856 (0.0008) [2023-12-27 03:06:17,160][105620] Updated weights for policy 1, policy_version 1610866 (0.0009) [2023-12-27 03:06:17,225][105620] Updated weights for policy 1, policy_version 1610877 (0.0010) [2023-12-27 03:06:17,357][105692] Updated weights for policy 0, policy_version 1607533 (0.0009) [2023-12-27 03:06:17,416][105692] Updated weights for policy 0, policy_version 1607543 (0.0009) [2023-12-27 03:06:17,468][105692] Updated weights for policy 0, policy_version 1607553 (0.0009) [2023-12-27 03:06:17,919][105620] Updated weights for policy 1, policy_version 1610887 (0.0007) [2023-12-27 03:06:17,979][105620] Updated weights for policy 1, policy_version 1610897 (0.0005) [2023-12-27 03:06:18,040][105620] Updated weights for policy 1, policy_version 1610907 (0.0006) [2023-12-27 03:06:18,219][105692] Updated weights for policy 0, policy_version 1607563 (0.0009) [2023-12-27 03:06:18,270][105692] Updated weights for policy 0, policy_version 1607573 (0.0009) [2023-12-27 03:06:18,317][105692] Updated weights for policy 0, policy_version 1607583 (0.0008) [2023-12-27 03:06:18,656][105620] Updated weights for policy 1, policy_version 1610917 (0.0008) [2023-12-27 03:06:18,724][105620] Updated weights for policy 1, policy_version 1610927 (0.0008) [2023-12-27 03:06:18,783][105620] Updated weights for policy 1, policy_version 1610937 (0.0009) [2023-12-27 03:06:18,982][105692] Updated weights for policy 0, policy_version 1607593 (0.0007) [2023-12-27 03:06:19,044][105692] Updated weights for policy 0, policy_version 1607603 (0.0009) [2023-12-27 03:06:19,113][105692] Updated weights for policy 0, policy_version 1607613 (0.0009) [2023-12-27 03:06:19,168][105692] Updated weights for policy 0, policy_version 1607623 (0.0009) [2023-12-27 03:06:19,590][105620] Updated weights for policy 1, policy_version 1610947 (0.0009) [2023-12-27 03:06:19,649][105620] Updated weights for policy 1, policy_version 1610957 (0.0009) [2023-12-27 03:06:19,707][105620] Updated weights for policy 1, policy_version 1610967 (0.0008) [2023-12-27 03:06:19,920][105692] Updated weights for policy 0, policy_version 1607633 (0.0009) [2023-12-27 03:06:19,986][105692] Updated weights for policy 0, policy_version 1607643 (0.0008) [2023-12-27 03:06:20,046][105692] Updated weights for policy 0, policy_version 1607653 (0.0009) [2023-12-27 03:06:20,482][105620] Updated weights for policy 1, policy_version 1610977 (0.0009) [2023-12-27 03:06:20,533][105620] Updated weights for policy 1, policy_version 1610987 (0.0009) [2023-12-27 03:06:20,602][105620] Updated weights for policy 1, policy_version 1610997 (0.0008) [2023-12-27 03:06:20,659][105620] Updated weights for policy 1, policy_version 1611007 (0.0009) [2023-12-27 03:06:20,803][105692] Updated weights for policy 0, policy_version 1607663 (0.0007) [2023-12-27 03:06:20,864][105692] Updated weights for policy 0, policy_version 1607673 (0.0008) [2023-12-27 03:06:20,912][105692] Updated weights for policy 0, policy_version 1607683 (0.0009) [2023-12-27 03:06:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 824098816. Throughput: 0: 9652.0, 1: 9482.2. Samples: 824086984. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:06:21,063][104569] Avg episode reward: [(0, '8265.442'), (1, '9170.824')] [2023-12-27 03:06:21,450][105620] Updated weights for policy 1, policy_version 1611017 (0.0006) [2023-12-27 03:06:21,510][105620] Updated weights for policy 1, policy_version 1611027 (0.0008) [2023-12-27 03:06:21,565][105620] Updated weights for policy 1, policy_version 1611037 (0.0009) [2023-12-27 03:06:21,672][105692] Updated weights for policy 0, policy_version 1607693 (0.0009) [2023-12-27 03:06:21,744][105692] Updated weights for policy 0, policy_version 1607703 (0.0009) [2023-12-27 03:06:21,803][105692] Updated weights for policy 0, policy_version 1607713 (0.0009) [2023-12-27 03:06:22,356][105620] Updated weights for policy 1, policy_version 1611047 (0.0009) [2023-12-27 03:06:22,423][105620] Updated weights for policy 1, policy_version 1611057 (0.0008) [2023-12-27 03:06:22,490][105620] Updated weights for policy 1, policy_version 1611067 (0.0010) [2023-12-27 03:06:22,515][105692] Updated weights for policy 0, policy_version 1607723 (0.0008) [2023-12-27 03:06:22,562][105692] Updated weights for policy 0, policy_version 1607733 (0.0008) [2023-12-27 03:06:22,615][105692] Updated weights for policy 0, policy_version 1607743 (0.0009) [2023-12-27 03:06:23,170][105620] Updated weights for policy 1, policy_version 1611077 (0.0009) [2023-12-27 03:06:23,232][105620] Updated weights for policy 1, policy_version 1611087 (0.0009) [2023-12-27 03:06:23,298][105620] Updated weights for policy 1, policy_version 1611097 (0.0005) [2023-12-27 03:06:23,432][105692] Updated weights for policy 0, policy_version 1607753 (0.0010) [2023-12-27 03:06:23,480][105692] Updated weights for policy 0, policy_version 1607763 (0.0008) [2023-12-27 03:06:23,527][105692] Updated weights for policy 0, policy_version 1607773 (0.0009) [2023-12-27 03:06:23,573][105692] Updated weights for policy 0, policy_version 1607783 (0.0008) [2023-12-27 03:06:23,866][105620] Updated weights for policy 1, policy_version 1611107 (0.0005) [2023-12-27 03:06:23,928][105620] Updated weights for policy 1, policy_version 1611117 (0.0005) [2023-12-27 03:06:23,980][105620] Updated weights for policy 1, policy_version 1611127 (0.0008) [2023-12-27 03:06:24,411][105692] Updated weights for policy 0, policy_version 1607793 (0.0009) [2023-12-27 03:06:24,465][105692] Updated weights for policy 0, policy_version 1607803 (0.0009) [2023-12-27 03:06:24,519][105692] Updated weights for policy 0, policy_version 1607813 (0.0009) [2023-12-27 03:06:24,688][105620] Updated weights for policy 1, policy_version 1611137 (0.0008) [2023-12-27 03:06:24,751][105620] Updated weights for policy 1, policy_version 1611147 (0.0005) [2023-12-27 03:06:24,812][105620] Updated weights for policy 1, policy_version 1611157 (0.0005) [2023-12-27 03:06:24,870][105620] Updated weights for policy 1, policy_version 1611167 (0.0005) [2023-12-27 03:06:25,386][105620] Updated weights for policy 1, policy_version 1611177 (0.0010) [2023-12-27 03:06:25,387][105692] Updated weights for policy 0, policy_version 1607823 (0.0007) [2023-12-27 03:06:25,433][105692] Updated weights for policy 0, policy_version 1607833 (0.0005) [2023-12-27 03:06:25,435][105620] Updated weights for policy 1, policy_version 1611187 (0.0010) [2023-12-27 03:06:25,481][105692] Updated weights for policy 0, policy_version 1607843 (0.0005) [2023-12-27 03:06:25,491][105620] Updated weights for policy 1, policy_version 1611197 (0.0011) [2023-12-27 03:06:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19494.2). Total num frames: 824188928. Throughput: 0: 9653.8, 1: 9581.4. Samples: 824201880. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:06:26,062][104569] Avg episode reward: [(0, '8074.649'), (1, '9170.722')] [2023-12-27 03:06:26,090][105620] Updated weights for policy 1, policy_version 1611207 (0.0011) [2023-12-27 03:06:26,136][105620] Updated weights for policy 1, policy_version 1611217 (0.0010) [2023-12-27 03:06:26,182][105620] Updated weights for policy 1, policy_version 1611227 (0.0010) [2023-12-27 03:06:26,333][105692] Updated weights for policy 0, policy_version 1607853 (0.0007) [2023-12-27 03:06:26,387][105692] Updated weights for policy 0, policy_version 1607865 (0.0010) [2023-12-27 03:06:26,440][105692] Updated weights for policy 0, policy_version 1607876 (0.0010) [2023-12-27 03:06:26,750][105620] Updated weights for policy 1, policy_version 1611237 (0.0006) [2023-12-27 03:06:26,807][105620] Updated weights for policy 1, policy_version 1611247 (0.0008) [2023-12-27 03:06:26,862][105620] Updated weights for policy 1, policy_version 1611257 (0.0011) [2023-12-27 03:06:27,332][105692] Updated weights for policy 0, policy_version 1607886 (0.0009) [2023-12-27 03:06:27,391][105692] Updated weights for policy 0, policy_version 1607896 (0.0008) [2023-12-27 03:06:27,448][105692] Updated weights for policy 0, policy_version 1607906 (0.0009) [2023-12-27 03:06:27,517][105620] Updated weights for policy 1, policy_version 1611267 (0.0009) [2023-12-27 03:06:27,574][105620] Updated weights for policy 1, policy_version 1611277 (0.0005) [2023-12-27 03:06:27,629][105620] Updated weights for policy 1, policy_version 1611287 (0.0010) [2023-12-27 03:06:28,199][105620] Updated weights for policy 1, policy_version 1611297 (0.0009) [2023-12-27 03:06:28,262][105620] Updated weights for policy 1, policy_version 1611307 (0.0007) [2023-12-27 03:06:28,301][105692] Updated weights for policy 0, policy_version 1607916 (0.0009) [2023-12-27 03:06:28,321][105620] Updated weights for policy 1, policy_version 1611317 (0.0006) [2023-12-27 03:06:28,358][105692] Updated weights for policy 0, policy_version 1607926 (0.0008) [2023-12-27 03:06:28,382][105620] Updated weights for policy 1, policy_version 1611327 (0.0007) [2023-12-27 03:06:28,430][105692] Updated weights for policy 0, policy_version 1607936 (0.0009) [2023-12-27 03:06:28,993][105620] Updated weights for policy 1, policy_version 1611337 (0.0005) [2023-12-27 03:06:29,044][105620] Updated weights for policy 1, policy_version 1611347 (0.0008) [2023-12-27 03:06:29,095][105620] Updated weights for policy 1, policy_version 1611357 (0.0010) [2023-12-27 03:06:29,235][105692] Updated weights for policy 0, policy_version 1607946 (0.0009) [2023-12-27 03:06:29,293][105692] Updated weights for policy 0, policy_version 1607956 (0.0008) [2023-12-27 03:06:29,356][105692] Updated weights for policy 0, policy_version 1607966 (0.0009) [2023-12-27 03:06:29,413][105692] Updated weights for policy 0, policy_version 1607976 (0.0008) [2023-12-27 03:06:29,856][105620] Updated weights for policy 1, policy_version 1611367 (0.0011) [2023-12-27 03:06:29,910][105620] Updated weights for policy 1, policy_version 1611377 (0.0011) [2023-12-27 03:06:29,973][105620] Updated weights for policy 1, policy_version 1611387 (0.0011) [2023-12-27 03:06:30,197][105692] Updated weights for policy 0, policy_version 1607986 (0.0008) [2023-12-27 03:06:30,253][105692] Updated weights for policy 0, policy_version 1607996 (0.0008) [2023-12-27 03:06:30,314][105692] Updated weights for policy 0, policy_version 1608006 (0.0008) [2023-12-27 03:06:30,717][105620] Updated weights for policy 1, policy_version 1611397 (0.0011) [2023-12-27 03:06:30,782][105620] Updated weights for policy 1, policy_version 1611407 (0.0011) [2023-12-27 03:06:30,847][105620] Updated weights for policy 1, policy_version 1611417 (0.0011) [2023-12-27 03:06:31,050][105692] Updated weights for policy 0, policy_version 1608016 (0.0009) [2023-12-27 03:06:31,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19114.7, 300 sec: 19522.0). Total num frames: 824287232. Throughput: 0: 9557.9, 1: 9663.6. Samples: 824260764. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:06:31,062][104569] Avg episode reward: [(0, '8525.382'), (1, '9263.544')] [2023-12-27 03:06:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001611424_412581888.pth... [2023-12-27 03:06:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001610272_412286976.pth [2023-12-27 03:06:31,096][105692] Updated weights for policy 0, policy_version 1608026 (0.0008) [2023-12-27 03:06:31,155][105692] Updated weights for policy 0, policy_version 1608036 (0.0009) [2023-12-27 03:06:31,175][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001608040_411713536.pth... [2023-12-27 03:06:31,180][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001606920_411426816.pth [2023-12-27 03:06:31,535][105620] Updated weights for policy 1, policy_version 1611427 (0.0009) [2023-12-27 03:06:31,600][105620] Updated weights for policy 1, policy_version 1611437 (0.0006) [2023-12-27 03:06:31,664][105620] Updated weights for policy 1, policy_version 1611447 (0.0008) [2023-12-27 03:06:31,972][105692] Updated weights for policy 0, policy_version 1608046 (0.0010) [2023-12-27 03:06:32,022][105692] Updated weights for policy 0, policy_version 1608056 (0.0009) [2023-12-27 03:06:32,078][105692] Updated weights for policy 0, policy_version 1608066 (0.0007) [2023-12-27 03:06:32,364][105620] Updated weights for policy 1, policy_version 1611457 (0.0009) [2023-12-27 03:06:32,423][105620] Updated weights for policy 1, policy_version 1611467 (0.0009) [2023-12-27 03:06:32,477][105620] Updated weights for policy 1, policy_version 1611477 (0.0010) [2023-12-27 03:06:32,528][105620] Updated weights for policy 1, policy_version 1611487 (0.0008) [2023-12-27 03:06:32,735][105692] Updated weights for policy 0, policy_version 1608076 (0.0007) [2023-12-27 03:06:32,788][105692] Updated weights for policy 0, policy_version 1608086 (0.0005) [2023-12-27 03:06:32,848][105692] Updated weights for policy 0, policy_version 1608096 (0.0006) [2023-12-27 03:06:33,187][105620] Updated weights for policy 1, policy_version 1611497 (0.0005) [2023-12-27 03:06:33,241][105620] Updated weights for policy 1, policy_version 1611507 (0.0007) [2023-12-27 03:06:33,302][105620] Updated weights for policy 1, policy_version 1611517 (0.0009) [2023-12-27 03:06:33,525][105692] Updated weights for policy 0, policy_version 1608106 (0.0008) [2023-12-27 03:06:33,580][105692] Updated weights for policy 0, policy_version 1608116 (0.0007) [2023-12-27 03:06:33,632][105692] Updated weights for policy 0, policy_version 1608126 (0.0010) [2023-12-27 03:06:33,684][105692] Updated weights for policy 0, policy_version 1608136 (0.0006) [2023-12-27 03:06:33,965][105620] Updated weights for policy 1, policy_version 1611528 (0.0010) [2023-12-27 03:06:34,009][105620] Updated weights for policy 1, policy_version 1611538 (0.0008) [2023-12-27 03:06:34,058][105620] Updated weights for policy 1, policy_version 1611548 (0.0008) [2023-12-27 03:06:34,298][105692] Updated weights for policy 0, policy_version 1608146 (0.0005) [2023-12-27 03:06:34,354][105692] Updated weights for policy 0, policy_version 1608156 (0.0006) [2023-12-27 03:06:34,403][105692] Updated weights for policy 0, policy_version 1608166 (0.0006) [2023-12-27 03:06:34,765][105620] Updated weights for policy 1, policy_version 1611558 (0.0005) [2023-12-27 03:06:34,820][105620] Updated weights for policy 1, policy_version 1611568 (0.0006) [2023-12-27 03:06:34,867][105620] Updated weights for policy 1, policy_version 1611578 (0.0005) [2023-12-27 03:06:35,121][105692] Updated weights for policy 0, policy_version 1608176 (0.0010) [2023-12-27 03:06:35,170][105692] Updated weights for policy 0, policy_version 1608186 (0.0010) [2023-12-27 03:06:35,222][105692] Updated weights for policy 0, policy_version 1608196 (0.0010) [2023-12-27 03:06:35,442][105620] Updated weights for policy 1, policy_version 1611588 (0.0006) [2023-12-27 03:06:35,505][105620] Updated weights for policy 1, policy_version 1611598 (0.0010) [2023-12-27 03:06:35,558][105620] Updated weights for policy 1, policy_version 1611608 (0.0010) [2023-12-27 03:06:35,790][105692] Updated weights for policy 0, policy_version 1608206 (0.0009) [2023-12-27 03:06:35,850][105692] Updated weights for policy 0, policy_version 1608216 (0.0007) [2023-12-27 03:06:35,903][105692] Updated weights for policy 0, policy_version 1608226 (0.0008) [2023-12-27 03:06:36,062][104569] Fps is (10 sec: 20479.3, 60 sec: 19251.1, 300 sec: 19521.9). Total num frames: 824393728. Throughput: 0: 9542.5, 1: 9705.2. Samples: 824378896. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:06:36,063][104569] Avg episode reward: [(0, '8347.053'), (1, '8990.476')] [2023-12-27 03:06:36,430][105620] Updated weights for policy 1, policy_version 1611618 (0.0010) [2023-12-27 03:06:36,491][105620] Updated weights for policy 1, policy_version 1611628 (0.0008) [2023-12-27 03:06:36,502][105692] Updated weights for policy 0, policy_version 1608236 (0.0006) [2023-12-27 03:06:36,552][105620] Updated weights for policy 1, policy_version 1611638 (0.0009) [2023-12-27 03:06:36,565][105692] Updated weights for policy 0, policy_version 1608246 (0.0006) [2023-12-27 03:06:36,616][105620] Updated weights for policy 1, policy_version 1611648 (0.0009) [2023-12-27 03:06:36,625][105692] Updated weights for policy 0, policy_version 1608256 (0.0007) [2023-12-27 03:06:37,287][105692] Updated weights for policy 0, policy_version 1608266 (0.0009) [2023-12-27 03:06:37,339][105692] Updated weights for policy 0, policy_version 1608276 (0.0006) [2023-12-27 03:06:37,387][105620] Updated weights for policy 1, policy_version 1611658 (0.0007) [2023-12-27 03:06:37,389][105692] Updated weights for policy 0, policy_version 1608286 (0.0006) [2023-12-27 03:06:37,436][105692] Updated weights for policy 0, policy_version 1608296 (0.0007) [2023-12-27 03:06:37,442][105620] Updated weights for policy 1, policy_version 1611668 (0.0007) [2023-12-27 03:06:37,489][105620] Updated weights for policy 1, policy_version 1611678 (0.0008) [2023-12-27 03:06:38,069][105692] Updated weights for policy 0, policy_version 1608306 (0.0010) [2023-12-27 03:06:38,121][105692] Updated weights for policy 0, policy_version 1608316 (0.0010) [2023-12-27 03:06:38,174][105692] Updated weights for policy 0, policy_version 1608326 (0.0010) [2023-12-27 03:06:38,276][105620] Updated weights for policy 1, policy_version 1611688 (0.0010) [2023-12-27 03:06:38,324][105620] Updated weights for policy 1, policy_version 1611698 (0.0009) [2023-12-27 03:06:38,384][105620] Updated weights for policy 1, policy_version 1611708 (0.0007) [2023-12-27 03:06:38,777][105692] Updated weights for policy 0, policy_version 1608336 (0.0011) [2023-12-27 03:06:38,841][105692] Updated weights for policy 0, policy_version 1608346 (0.0011) [2023-12-27 03:06:38,900][105692] Updated weights for policy 0, policy_version 1608356 (0.0011) [2023-12-27 03:06:39,124][105620] Updated weights for policy 1, policy_version 1611718 (0.0010) [2023-12-27 03:06:39,188][105620] Updated weights for policy 1, policy_version 1611728 (0.0010) [2023-12-27 03:06:39,256][105620] Updated weights for policy 1, policy_version 1611738 (0.0010) [2023-12-27 03:06:39,580][105692] Updated weights for policy 0, policy_version 1608366 (0.0010) [2023-12-27 03:06:39,639][105692] Updated weights for policy 0, policy_version 1608376 (0.0010) [2023-12-27 03:06:39,695][105692] Updated weights for policy 0, policy_version 1608386 (0.0010) [2023-12-27 03:06:40,011][105620] Updated weights for policy 1, policy_version 1611748 (0.0010) [2023-12-27 03:06:40,078][105620] Updated weights for policy 1, policy_version 1611758 (0.0008) [2023-12-27 03:06:40,142][105620] Updated weights for policy 1, policy_version 1611768 (0.0007) [2023-12-27 03:06:40,480][105692] Updated weights for policy 0, policy_version 1608396 (0.0008) [2023-12-27 03:06:40,543][105692] Updated weights for policy 0, policy_version 1608406 (0.0006) [2023-12-27 03:06:40,599][105692] Updated weights for policy 0, policy_version 1608416 (0.0008) [2023-12-27 03:06:40,830][105620] Updated weights for policy 1, policy_version 1611778 (0.0006) [2023-12-27 03:06:40,899][105620] Updated weights for policy 1, policy_version 1611788 (0.0007) [2023-12-27 03:06:40,963][105620] Updated weights for policy 1, policy_version 1611798 (0.0010) [2023-12-27 03:06:41,031][105620] Updated weights for policy 1, policy_version 1611808 (0.0011) [2023-12-27 03:06:41,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 824492032. Throughput: 0: 9632.1, 1: 9687.7. Samples: 824498596. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:06:41,063][104569] Avg episode reward: [(0, '8440.021'), (1, '8901.296')] [2023-12-27 03:06:41,319][105692] Updated weights for policy 0, policy_version 1608426 (0.0007) [2023-12-27 03:06:41,390][105692] Updated weights for policy 0, policy_version 1608436 (0.0009) [2023-12-27 03:06:41,451][105692] Updated weights for policy 0, policy_version 1608446 (0.0008) [2023-12-27 03:06:41,512][105692] Updated weights for policy 0, policy_version 1608456 (0.0008) [2023-12-27 03:06:41,800][105620] Updated weights for policy 1, policy_version 1611818 (0.0009) [2023-12-27 03:06:41,860][105620] Updated weights for policy 1, policy_version 1611828 (0.0011) [2023-12-27 03:06:41,924][105620] Updated weights for policy 1, policy_version 1611838 (0.0011) [2023-12-27 03:06:42,298][105692] Updated weights for policy 0, policy_version 1608466 (0.0009) [2023-12-27 03:06:42,363][105692] Updated weights for policy 0, policy_version 1608476 (0.0008) [2023-12-27 03:06:42,415][105692] Updated weights for policy 0, policy_version 1608486 (0.0008) [2023-12-27 03:06:42,666][105620] Updated weights for policy 1, policy_version 1611848 (0.0008) [2023-12-27 03:06:42,716][105620] Updated weights for policy 1, policy_version 1611859 (0.0010) [2023-12-27 03:06:42,768][105620] Updated weights for policy 1, policy_version 1611869 (0.0010) [2023-12-27 03:06:43,114][105692] Updated weights for policy 0, policy_version 1608496 (0.0008) [2023-12-27 03:06:43,161][105692] Updated weights for policy 0, policy_version 1608506 (0.0007) [2023-12-27 03:06:43,206][105692] Updated weights for policy 0, policy_version 1608516 (0.0008) [2023-12-27 03:06:43,540][105620] Updated weights for policy 1, policy_version 1611879 (0.0010) [2023-12-27 03:06:43,601][105620] Updated weights for policy 1, policy_version 1611889 (0.0010) [2023-12-27 03:06:43,659][105620] Updated weights for policy 1, policy_version 1611899 (0.0010) [2023-12-27 03:06:43,931][105692] Updated weights for policy 0, policy_version 1608526 (0.0007) [2023-12-27 03:06:43,986][105692] Updated weights for policy 0, policy_version 1608536 (0.0005) [2023-12-27 03:06:44,050][105692] Updated weights for policy 0, policy_version 1608546 (0.0005) [2023-12-27 03:06:44,319][105620] Updated weights for policy 1, policy_version 1611909 (0.0008) [2023-12-27 03:06:44,377][105620] Updated weights for policy 1, policy_version 1611919 (0.0005) [2023-12-27 03:06:44,430][105620] Updated weights for policy 1, policy_version 1611929 (0.0005) [2023-12-27 03:06:44,763][105692] Updated weights for policy 0, policy_version 1608556 (0.0007) [2023-12-27 03:06:44,819][105692] Updated weights for policy 0, policy_version 1608566 (0.0010) [2023-12-27 03:06:44,881][105692] Updated weights for policy 0, policy_version 1608576 (0.0008) [2023-12-27 03:06:45,070][105620] Updated weights for policy 1, policy_version 1611939 (0.0007) [2023-12-27 03:06:45,126][105620] Updated weights for policy 1, policy_version 1611949 (0.0011) [2023-12-27 03:06:45,186][105620] Updated weights for policy 1, policy_version 1611959 (0.0011) [2023-12-27 03:06:45,699][105692] Updated weights for policy 0, policy_version 1608586 (0.0008) [2023-12-27 03:06:45,751][105692] Updated weights for policy 0, policy_version 1608596 (0.0009) [2023-12-27 03:06:45,800][105692] Updated weights for policy 0, policy_version 1608606 (0.0008) [2023-12-27 03:06:45,848][105692] Updated weights for policy 0, policy_version 1608616 (0.0008) [2023-12-27 03:06:45,875][105620] Updated weights for policy 1, policy_version 1611969 (0.0011) [2023-12-27 03:06:45,926][105620] Updated weights for policy 1, policy_version 1611979 (0.0011) [2023-12-27 03:06:45,989][105620] Updated weights for policy 1, policy_version 1611989 (0.0008) [2023-12-27 03:06:46,044][105620] Updated weights for policy 1, policy_version 1611999 (0.0007) [2023-12-27 03:06:46,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 824590336. Throughput: 0: 9625.6, 1: 9659.2. Samples: 824554168. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:06:46,063][104569] Avg episode reward: [(0, '9171.744'), (1, '8996.275')] [2023-12-27 03:06:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001608616_411860992.pth... [2023-12-27 03:06:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001612000_412729344.pth... [2023-12-27 03:06:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001610816_412426240.pth [2023-12-27 03:06:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001607496_411574272.pth [2023-12-27 03:06:46,644][105692] Updated weights for policy 0, policy_version 1608626 (0.0009) [2023-12-27 03:06:46,707][105692] Updated weights for policy 0, policy_version 1608636 (0.0009) [2023-12-27 03:06:46,759][105692] Updated weights for policy 0, policy_version 1608646 (0.0006) [2023-12-27 03:06:46,764][105620] Updated weights for policy 1, policy_version 1612009 (0.0007) [2023-12-27 03:06:46,815][105620] Updated weights for policy 1, policy_version 1612019 (0.0006) [2023-12-27 03:06:46,866][105620] Updated weights for policy 1, policy_version 1612029 (0.0005) [2023-12-27 03:06:47,498][105620] Updated weights for policy 1, policy_version 1612039 (0.0006) [2023-12-27 03:06:47,550][105620] Updated weights for policy 1, policy_version 1612049 (0.0005) [2023-12-27 03:06:47,585][105692] Updated weights for policy 0, policy_version 1608656 (0.0007) [2023-12-27 03:06:47,599][105620] Updated weights for policy 1, policy_version 1612059 (0.0005) [2023-12-27 03:06:47,648][105692] Updated weights for policy 0, policy_version 1608666 (0.0009) [2023-12-27 03:06:47,708][105692] Updated weights for policy 0, policy_version 1608676 (0.0009) [2023-12-27 03:06:48,219][105620] Updated weights for policy 1, policy_version 1612069 (0.0008) [2023-12-27 03:06:48,267][105620] Updated weights for policy 1, policy_version 1612079 (0.0010) [2023-12-27 03:06:48,330][105620] Updated weights for policy 1, policy_version 1612089 (0.0011) [2023-12-27 03:06:48,502][105692] Updated weights for policy 0, policy_version 1608686 (0.0009) [2023-12-27 03:06:48,570][105692] Updated weights for policy 0, policy_version 1608696 (0.0010) [2023-12-27 03:06:48,630][105692] Updated weights for policy 0, policy_version 1608706 (0.0007) [2023-12-27 03:06:49,017][105620] Updated weights for policy 1, policy_version 1612099 (0.0008) [2023-12-27 03:06:49,068][105620] Updated weights for policy 1, policy_version 1612109 (0.0010) [2023-12-27 03:06:49,134][105620] Updated weights for policy 1, policy_version 1612119 (0.0010) [2023-12-27 03:06:49,417][105692] Updated weights for policy 0, policy_version 1608716 (0.0008) [2023-12-27 03:06:49,484][105692] Updated weights for policy 0, policy_version 1608726 (0.0010) [2023-12-27 03:06:49,551][105692] Updated weights for policy 0, policy_version 1608736 (0.0010) [2023-12-27 03:06:49,917][105620] Updated weights for policy 1, policy_version 1612129 (0.0010) [2023-12-27 03:06:49,989][105620] Updated weights for policy 1, policy_version 1612139 (0.0007) [2023-12-27 03:06:50,057][105620] Updated weights for policy 1, policy_version 1612149 (0.0008) [2023-12-27 03:06:50,125][105620] Updated weights for policy 1, policy_version 1612159 (0.0009) [2023-12-27 03:06:50,315][105692] Updated weights for policy 0, policy_version 1608746 (0.0010) [2023-12-27 03:06:50,372][105692] Updated weights for policy 0, policy_version 1608756 (0.0010) [2023-12-27 03:06:50,428][105692] Updated weights for policy 0, policy_version 1608766 (0.0009) [2023-12-27 03:06:50,486][105692] Updated weights for policy 0, policy_version 1608776 (0.0007) [2023-12-27 03:06:50,869][105620] Updated weights for policy 1, policy_version 1612169 (0.0005) [2023-12-27 03:06:50,938][105620] Updated weights for policy 1, policy_version 1612179 (0.0007) [2023-12-27 03:06:51,004][105620] Updated weights for policy 1, policy_version 1612189 (0.0007) [2023-12-27 03:06:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 824680448. Throughput: 0: 9555.9, 1: 9809.4. Samples: 824670172. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:06:51,062][104569] Avg episode reward: [(0, '8895.895'), (1, '9086.268')] [2023-12-27 03:06:51,129][105692] Updated weights for policy 0, policy_version 1608786 (0.0009) [2023-12-27 03:06:51,190][105692] Updated weights for policy 0, policy_version 1608796 (0.0006) [2023-12-27 03:06:51,249][105692] Updated weights for policy 0, policy_version 1608806 (0.0007) [2023-12-27 03:06:51,614][105620] Updated weights for policy 1, policy_version 1612199 (0.0008) [2023-12-27 03:06:51,672][105620] Updated weights for policy 1, policy_version 1612209 (0.0009) [2023-12-27 03:06:51,729][105620] Updated weights for policy 1, policy_version 1612219 (0.0008) [2023-12-27 03:06:51,902][105692] Updated weights for policy 0, policy_version 1608816 (0.0009) [2023-12-27 03:06:51,963][105692] Updated weights for policy 0, policy_version 1608826 (0.0006) [2023-12-27 03:06:52,023][105692] Updated weights for policy 0, policy_version 1608836 (0.0005) [2023-12-27 03:06:52,534][105620] Updated weights for policy 1, policy_version 1612229 (0.0009) [2023-12-27 03:06:52,594][105620] Updated weights for policy 1, policy_version 1612239 (0.0008) [2023-12-27 03:06:52,659][105620] Updated weights for policy 1, policy_version 1612249 (0.0009) [2023-12-27 03:06:52,683][105692] Updated weights for policy 0, policy_version 1608846 (0.0007) [2023-12-27 03:06:52,746][105692] Updated weights for policy 0, policy_version 1608856 (0.0006) [2023-12-27 03:06:52,805][105692] Updated weights for policy 0, policy_version 1608866 (0.0005) [2023-12-27 03:06:53,316][105620] Updated weights for policy 1, policy_version 1612259 (0.0008) [2023-12-27 03:06:53,369][105620] Updated weights for policy 1, policy_version 1612269 (0.0009) [2023-12-27 03:06:53,399][105692] Updated weights for policy 0, policy_version 1608876 (0.0005) [2023-12-27 03:06:53,422][105620] Updated weights for policy 1, policy_version 1612279 (0.0010) [2023-12-27 03:06:53,457][105692] Updated weights for policy 0, policy_version 1608886 (0.0005) [2023-12-27 03:06:53,505][105692] Updated weights for policy 0, policy_version 1608896 (0.0005) [2023-12-27 03:06:54,156][105692] Updated weights for policy 0, policy_version 1608906 (0.0006) [2023-12-27 03:06:54,206][105620] Updated weights for policy 1, policy_version 1612289 (0.0008) [2023-12-27 03:06:54,210][105692] Updated weights for policy 0, policy_version 1608916 (0.0009) [2023-12-27 03:06:54,261][105620] Updated weights for policy 1, policy_version 1612299 (0.0006) [2023-12-27 03:06:54,271][105692] Updated weights for policy 0, policy_version 1608926 (0.0007) [2023-12-27 03:06:54,317][105620] Updated weights for policy 1, policy_version 1612309 (0.0007) [2023-12-27 03:06:54,326][105692] Updated weights for policy 0, policy_version 1608936 (0.0005) [2023-12-27 03:06:54,378][105620] Updated weights for policy 1, policy_version 1612319 (0.0009) [2023-12-27 03:06:54,955][105692] Updated weights for policy 0, policy_version 1608946 (0.0006) [2023-12-27 03:06:55,016][105692] Updated weights for policy 0, policy_version 1608956 (0.0005) [2023-12-27 03:06:55,072][105692] Updated weights for policy 0, policy_version 1608966 (0.0008) [2023-12-27 03:06:55,223][105620] Updated weights for policy 1, policy_version 1612329 (0.0010) [2023-12-27 03:06:55,276][105620] Updated weights for policy 1, policy_version 1612339 (0.0010) [2023-12-27 03:06:55,328][105620] Updated weights for policy 1, policy_version 1612349 (0.0009) [2023-12-27 03:06:55,720][105692] Updated weights for policy 0, policy_version 1608976 (0.0008) [2023-12-27 03:06:55,781][105692] Updated weights for policy 0, policy_version 1608986 (0.0008) [2023-12-27 03:06:55,837][105692] Updated weights for policy 0, policy_version 1608996 (0.0006) [2023-12-27 03:06:56,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 824778752. Throughput: 0: 9622.6, 1: 9820.4. Samples: 824789252. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-12-27 03:06:56,062][104569] Avg episode reward: [(0, '8535.182'), (1, '9080.575')] [2023-12-27 03:06:56,091][105620] Updated weights for policy 1, policy_version 1612359 (0.0010) [2023-12-27 03:06:56,142][105620] Updated weights for policy 1, policy_version 1612369 (0.0006) [2023-12-27 03:06:56,194][105620] Updated weights for policy 1, policy_version 1612379 (0.0008) [2023-12-27 03:06:56,535][105692] Updated weights for policy 0, policy_version 1609006 (0.0009) [2023-12-27 03:06:56,593][105692] Updated weights for policy 0, policy_version 1609016 (0.0011) [2023-12-27 03:06:56,645][105692] Updated weights for policy 0, policy_version 1609026 (0.0011) [2023-12-27 03:06:56,893][105620] Updated weights for policy 1, policy_version 1612389 (0.0010) [2023-12-27 03:06:56,949][105620] Updated weights for policy 1, policy_version 1612399 (0.0010) [2023-12-27 03:06:57,016][105620] Updated weights for policy 1, policy_version 1612409 (0.0010) [2023-12-27 03:06:57,309][105692] Updated weights for policy 0, policy_version 1609036 (0.0010) [2023-12-27 03:06:57,361][105692] Updated weights for policy 0, policy_version 1609046 (0.0008) [2023-12-27 03:06:57,420][105692] Updated weights for policy 0, policy_version 1609056 (0.0011) [2023-12-27 03:06:57,702][105620] Updated weights for policy 1, policy_version 1612419 (0.0010) [2023-12-27 03:06:57,757][105620] Updated weights for policy 1, policy_version 1612429 (0.0010) [2023-12-27 03:06:57,804][105620] Updated weights for policy 1, policy_version 1612439 (0.0010) [2023-12-27 03:06:58,161][105692] Updated weights for policy 0, policy_version 1609066 (0.0010) [2023-12-27 03:06:58,235][105692] Updated weights for policy 0, policy_version 1609076 (0.0011) [2023-12-27 03:06:58,305][105692] Updated weights for policy 0, policy_version 1609086 (0.0011) [2023-12-27 03:06:58,374][105692] Updated weights for policy 0, policy_version 1609096 (0.0010) [2023-12-27 03:06:58,518][105620] Updated weights for policy 1, policy_version 1612449 (0.0010) [2023-12-27 03:06:58,586][105620] Updated weights for policy 1, policy_version 1612459 (0.0009) [2023-12-27 03:06:58,649][105620] Updated weights for policy 1, policy_version 1612469 (0.0010) [2023-12-27 03:06:58,700][105620] Updated weights for policy 1, policy_version 1612479 (0.0011) [2023-12-27 03:06:59,151][105692] Updated weights for policy 0, policy_version 1609106 (0.0010) [2023-12-27 03:06:59,205][105692] Updated weights for policy 0, policy_version 1609116 (0.0010) [2023-12-27 03:06:59,267][105692] Updated weights for policy 0, policy_version 1609126 (0.0009) [2023-12-27 03:06:59,404][105620] Updated weights for policy 1, policy_version 1612489 (0.0010) [2023-12-27 03:06:59,455][105620] Updated weights for policy 1, policy_version 1612499 (0.0010) [2023-12-27 03:06:59,513][105620] Updated weights for policy 1, policy_version 1612509 (0.0010) [2023-12-27 03:07:00,037][105692] Updated weights for policy 0, policy_version 1609136 (0.0008) [2023-12-27 03:07:00,094][105692] Updated weights for policy 0, policy_version 1609146 (0.0008) [2023-12-27 03:07:00,158][105692] Updated weights for policy 0, policy_version 1609156 (0.0008) [2023-12-27 03:07:00,279][105620] Updated weights for policy 1, policy_version 1612519 (0.0010) [2023-12-27 03:07:00,339][105620] Updated weights for policy 1, policy_version 1612529 (0.0008) [2023-12-27 03:07:00,397][105620] Updated weights for policy 1, policy_version 1612539 (0.0005) [2023-12-27 03:07:00,936][105620] Updated weights for policy 1, policy_version 1612549 (0.0006) [2023-12-27 03:07:00,989][105620] Updated weights for policy 1, policy_version 1612559 (0.0005) [2023-12-27 03:07:01,004][105692] Updated weights for policy 0, policy_version 1609166 (0.0008) [2023-12-27 03:07:01,043][105620] Updated weights for policy 1, policy_version 1612569 (0.0007) [2023-12-27 03:07:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 824868864. Throughput: 0: 9650.4, 1: 9822.3. Samples: 824847152. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:07:01,062][105692] Updated weights for policy 0, policy_version 1609176 (0.0007) [2023-12-27 03:07:01,063][104569] Avg episode reward: [(0, '8626.881'), (1, '8899.414')] [2023-12-27 03:07:01,075][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001612576_412876800.pth... [2023-12-27 03:07:01,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001611424_412581888.pth [2023-12-27 03:07:01,121][105692] Updated weights for policy 0, policy_version 1609186 (0.0007) [2023-12-27 03:07:01,151][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001609192_412008448.pth... [2023-12-27 03:07:01,154][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001608040_411713536.pth [2023-12-27 03:07:01,748][105620] Updated weights for policy 1, policy_version 1612579 (0.0009) [2023-12-27 03:07:01,809][105620] Updated weights for policy 1, policy_version 1612589 (0.0010) [2023-12-27 03:07:01,863][105620] Updated weights for policy 1, policy_version 1612599 (0.0009) [2023-12-27 03:07:01,897][105692] Updated weights for policy 0, policy_version 1609196 (0.0008) [2023-12-27 03:07:01,946][105692] Updated weights for policy 0, policy_version 1609206 (0.0005) [2023-12-27 03:07:01,994][105692] Updated weights for policy 0, policy_version 1609216 (0.0007) [2023-12-27 03:07:02,548][105620] Updated weights for policy 1, policy_version 1612609 (0.0010) [2023-12-27 03:07:02,599][105620] Updated weights for policy 1, policy_version 1612619 (0.0010) [2023-12-27 03:07:02,653][105620] Updated weights for policy 1, policy_version 1612629 (0.0010) [2023-12-27 03:07:02,708][105620] Updated weights for policy 1, policy_version 1612639 (0.0010) [2023-12-27 03:07:02,774][105692] Updated weights for policy 0, policy_version 1609226 (0.0007) [2023-12-27 03:07:02,826][105692] Updated weights for policy 0, policy_version 1609236 (0.0007) [2023-12-27 03:07:02,878][105692] Updated weights for policy 0, policy_version 1609246 (0.0008) [2023-12-27 03:07:02,937][105692] Updated weights for policy 0, policy_version 1609256 (0.0008) [2023-12-27 03:07:03,434][105620] Updated weights for policy 1, policy_version 1612649 (0.0006) [2023-12-27 03:07:03,488][105620] Updated weights for policy 1, policy_version 1612659 (0.0005) [2023-12-27 03:07:03,541][105620] Updated weights for policy 1, policy_version 1612669 (0.0006) [2023-12-27 03:07:03,626][105692] Updated weights for policy 0, policy_version 1609267 (0.0009) [2023-12-27 03:07:03,676][105692] Updated weights for policy 0, policy_version 1609278 (0.0010) [2023-12-27 03:07:04,077][105620] Updated weights for policy 1, policy_version 1612679 (0.0011) [2023-12-27 03:07:04,144][105620] Updated weights for policy 1, policy_version 1612689 (0.0007) [2023-12-27 03:07:04,206][105620] Updated weights for policy 1, policy_version 1612699 (0.0008) [2023-12-27 03:07:04,535][105692] Updated weights for policy 0, policy_version 1609289 (0.0010) [2023-12-27 03:07:04,605][105692] Updated weights for policy 0, policy_version 1609299 (0.0010) [2023-12-27 03:07:04,675][105692] Updated weights for policy 0, policy_version 1609309 (0.0011) [2023-12-27 03:07:04,727][105692] Updated weights for policy 0, policy_version 1609319 (0.0010) [2023-12-27 03:07:04,854][105620] Updated weights for policy 1, policy_version 1612709 (0.0008) [2023-12-27 03:07:04,919][105620] Updated weights for policy 1, policy_version 1612719 (0.0005) [2023-12-27 03:07:04,973][105620] Updated weights for policy 1, policy_version 1612729 (0.0006) [2023-12-27 03:07:05,365][105692] Updated weights for policy 0, policy_version 1609329 (0.0006) [2023-12-27 03:07:05,421][105692] Updated weights for policy 0, policy_version 1609339 (0.0005) [2023-12-27 03:07:05,483][105692] Updated weights for policy 0, policy_version 1609349 (0.0005) [2023-12-27 03:07:05,564][105620] Updated weights for policy 1, policy_version 1612739 (0.0009) [2023-12-27 03:07:05,636][105620] Updated weights for policy 1, policy_version 1612749 (0.0005) [2023-12-27 03:07:05,708][105620] Updated weights for policy 1, policy_version 1612759 (0.0008) [2023-12-27 03:07:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 824975360. Throughput: 0: 9541.1, 1: 9960.7. Samples: 824964564. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:07:06,062][104569] Avg episode reward: [(0, '8805.336'), (1, '8899.182')] [2023-12-27 03:07:06,083][105692] Updated weights for policy 0, policy_version 1609359 (0.0005) [2023-12-27 03:07:06,143][105692] Updated weights for policy 0, policy_version 1609369 (0.0008) [2023-12-27 03:07:06,207][105692] Updated weights for policy 0, policy_version 1609379 (0.0006) [2023-12-27 03:07:06,273][105620] Updated weights for policy 1, policy_version 1612769 (0.0010) [2023-12-27 03:07:06,335][105620] Updated weights for policy 1, policy_version 1612779 (0.0007) [2023-12-27 03:07:06,399][105620] Updated weights for policy 1, policy_version 1612789 (0.0011) [2023-12-27 03:07:06,459][105620] Updated weights for policy 1, policy_version 1612799 (0.0011) [2023-12-27 03:07:06,810][105692] Updated weights for policy 0, policy_version 1609389 (0.0007) [2023-12-27 03:07:06,861][105692] Updated weights for policy 0, policy_version 1609399 (0.0009) [2023-12-27 03:07:06,924][105692] Updated weights for policy 0, policy_version 1609409 (0.0006) [2023-12-27 03:07:07,177][105620] Updated weights for policy 1, policy_version 1612809 (0.0006) [2023-12-27 03:07:07,233][105620] Updated weights for policy 1, policy_version 1612819 (0.0011) [2023-12-27 03:07:07,293][105620] Updated weights for policy 1, policy_version 1612829 (0.0011) [2023-12-27 03:07:07,641][105692] Updated weights for policy 0, policy_version 1609419 (0.0010) [2023-12-27 03:07:07,703][105692] Updated weights for policy 0, policy_version 1609429 (0.0010) [2023-12-27 03:07:07,767][105692] Updated weights for policy 0, policy_version 1609439 (0.0010) [2023-12-27 03:07:07,875][105620] Updated weights for policy 1, policy_version 1612839 (0.0007) [2023-12-27 03:07:07,935][105620] Updated weights for policy 1, policy_version 1612849 (0.0006) [2023-12-27 03:07:08,000][105620] Updated weights for policy 1, policy_version 1612859 (0.0005) [2023-12-27 03:07:08,507][105692] Updated weights for policy 0, policy_version 1609449 (0.0010) [2023-12-27 03:07:08,579][105692] Updated weights for policy 0, policy_version 1609459 (0.0010) [2023-12-27 03:07:08,641][105692] Updated weights for policy 0, policy_version 1609469 (0.0010) [2023-12-27 03:07:08,663][105620] Updated weights for policy 1, policy_version 1612869 (0.0006) [2023-12-27 03:07:08,696][105692] Updated weights for policy 0, policy_version 1609479 (0.0010) [2023-12-27 03:07:08,719][105620] Updated weights for policy 1, policy_version 1612879 (0.0006) [2023-12-27 03:07:08,776][105620] Updated weights for policy 1, policy_version 1612889 (0.0009) [2023-12-27 03:07:09,455][105692] Updated weights for policy 0, policy_version 1609489 (0.0008) [2023-12-27 03:07:09,512][105620] Updated weights for policy 1, policy_version 1612899 (0.0008) [2023-12-27 03:07:09,514][105692] Updated weights for policy 0, policy_version 1609499 (0.0006) [2023-12-27 03:07:09,573][105692] Updated weights for policy 0, policy_version 1609509 (0.0006) [2023-12-27 03:07:09,575][105620] Updated weights for policy 1, policy_version 1612909 (0.0007) [2023-12-27 03:07:09,632][105620] Updated weights for policy 1, policy_version 1612919 (0.0010) [2023-12-27 03:07:10,246][105692] Updated weights for policy 0, policy_version 1609519 (0.0007) [2023-12-27 03:07:10,310][105692] Updated weights for policy 0, policy_version 1609529 (0.0008) [2023-12-27 03:07:10,370][105692] Updated weights for policy 0, policy_version 1609539 (0.0008) [2023-12-27 03:07:10,446][105620] Updated weights for policy 1, policy_version 1612929 (0.0010) [2023-12-27 03:07:10,504][105620] Updated weights for policy 1, policy_version 1612939 (0.0011) [2023-12-27 03:07:10,560][105620] Updated weights for policy 1, policy_version 1612949 (0.0011) [2023-12-27 03:07:10,617][105620] Updated weights for policy 1, policy_version 1612959 (0.0010) [2023-12-27 03:07:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 825073664. Throughput: 0: 9671.5, 1: 9953.1. Samples: 825084984. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:07:11,062][104569] Avg episode reward: [(0, '8624.523'), (1, '9172.310')] [2023-12-27 03:07:11,211][105692] Updated weights for policy 0, policy_version 1609549 (0.0008) [2023-12-27 03:07:11,273][105692] Updated weights for policy 0, policy_version 1609559 (0.0008) [2023-12-27 03:07:11,326][105692] Updated weights for policy 0, policy_version 1609569 (0.0008) [2023-12-27 03:07:11,341][105620] Updated weights for policy 1, policy_version 1612969 (0.0007) [2023-12-27 03:07:11,408][105620] Updated weights for policy 1, policy_version 1612979 (0.0009) [2023-12-27 03:07:11,456][105620] Updated weights for policy 1, policy_version 1612989 (0.0009) [2023-12-27 03:07:11,998][105692] Updated weights for policy 0, policy_version 1609579 (0.0007) [2023-12-27 03:07:12,057][105692] Updated weights for policy 0, policy_version 1609589 (0.0005) [2023-12-27 03:07:12,123][105692] Updated weights for policy 0, policy_version 1609599 (0.0007) [2023-12-27 03:07:12,267][105620] Updated weights for policy 1, policy_version 1612999 (0.0010) [2023-12-27 03:07:12,338][105620] Updated weights for policy 1, policy_version 1613009 (0.0009) [2023-12-27 03:07:12,401][105620] Updated weights for policy 1, policy_version 1613019 (0.0009) [2023-12-27 03:07:12,766][105692] Updated weights for policy 0, policy_version 1609609 (0.0008) [2023-12-27 03:07:12,836][105692] Updated weights for policy 0, policy_version 1609619 (0.0011) [2023-12-27 03:07:12,903][105692] Updated weights for policy 0, policy_version 1609629 (0.0011) [2023-12-27 03:07:12,969][105692] Updated weights for policy 0, policy_version 1609639 (0.0011) [2023-12-27 03:07:13,151][105620] Updated weights for policy 1, policy_version 1613030 (0.0008) [2023-12-27 03:07:13,208][105620] Updated weights for policy 1, policy_version 1613040 (0.0009) [2023-12-27 03:07:13,269][105620] Updated weights for policy 1, policy_version 1613050 (0.0007) [2023-12-27 03:07:13,672][105692] Updated weights for policy 0, policy_version 1609649 (0.0006) [2023-12-27 03:07:13,729][105692] Updated weights for policy 0, policy_version 1609659 (0.0010) [2023-12-27 03:07:13,792][105692] Updated weights for policy 0, policy_version 1609669 (0.0011) [2023-12-27 03:07:14,002][105620] Updated weights for policy 1, policy_version 1613060 (0.0008) [2023-12-27 03:07:14,047][105620] Updated weights for policy 1, policy_version 1613070 (0.0010) [2023-12-27 03:07:14,094][105620] Updated weights for policy 1, policy_version 1613080 (0.0010) [2023-12-27 03:07:14,504][105692] Updated weights for policy 0, policy_version 1609679 (0.0010) [2023-12-27 03:07:14,562][105692] Updated weights for policy 0, policy_version 1609689 (0.0011) [2023-12-27 03:07:14,620][105692] Updated weights for policy 0, policy_version 1609699 (0.0010) [2023-12-27 03:07:14,875][105620] Updated weights for policy 1, policy_version 1613090 (0.0010) [2023-12-27 03:07:14,935][105620] Updated weights for policy 1, policy_version 1613100 (0.0008) [2023-12-27 03:07:14,992][105620] Updated weights for policy 1, policy_version 1613110 (0.0008) [2023-12-27 03:07:15,044][105620] Updated weights for policy 1, policy_version 1613120 (0.0008) [2023-12-27 03:07:15,402][105692] Updated weights for policy 0, policy_version 1609709 (0.0011) [2023-12-27 03:07:15,451][105692] Updated weights for policy 0, policy_version 1609719 (0.0010) [2023-12-27 03:07:15,495][105692] Updated weights for policy 0, policy_version 1609729 (0.0010) [2023-12-27 03:07:15,810][105620] Updated weights for policy 1, policy_version 1613130 (0.0007) [2023-12-27 03:07:15,863][105620] Updated weights for policy 1, policy_version 1613140 (0.0005) [2023-12-27 03:07:15,918][105620] Updated weights for policy 1, policy_version 1613150 (0.0005) [2023-12-27 03:07:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 825171968. Throughput: 0: 9772.3, 1: 9815.4. Samples: 825142212. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:07:16,062][104569] Avg episode reward: [(0, '8440.168'), (1, '8992.303')] [2023-12-27 03:07:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001609736_412147712.pth... [2023-12-27 03:07:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001613152_413024256.pth... [2023-12-27 03:07:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001612000_412729344.pth [2023-12-27 03:07:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001608616_411860992.pth [2023-12-27 03:07:16,180][105692] Updated weights for policy 0, policy_version 1609739 (0.0009) [2023-12-27 03:07:16,244][105692] Updated weights for policy 0, policy_version 1609749 (0.0008) [2023-12-27 03:07:16,300][105692] Updated weights for policy 0, policy_version 1609759 (0.0008) [2023-12-27 03:07:16,584][105620] Updated weights for policy 1, policy_version 1613160 (0.0010) [2023-12-27 03:07:16,650][105620] Updated weights for policy 1, policy_version 1613170 (0.0011) [2023-12-27 03:07:16,719][105620] Updated weights for policy 1, policy_version 1613180 (0.0011) [2023-12-27 03:07:16,869][105692] Updated weights for policy 0, policy_version 1609769 (0.0010) [2023-12-27 03:07:16,919][105692] Updated weights for policy 0, policy_version 1609779 (0.0010) [2023-12-27 03:07:16,971][105692] Updated weights for policy 0, policy_version 1609789 (0.0010) [2023-12-27 03:07:17,023][105692] Updated weights for policy 0, policy_version 1609799 (0.0010) [2023-12-27 03:07:17,401][105620] Updated weights for policy 1, policy_version 1613190 (0.0011) [2023-12-27 03:07:17,462][105620] Updated weights for policy 1, policy_version 1613200 (0.0010) [2023-12-27 03:07:17,521][105620] Updated weights for policy 1, policy_version 1613210 (0.0010) [2023-12-27 03:07:17,636][105692] Updated weights for policy 0, policy_version 1609809 (0.0006) [2023-12-27 03:07:17,695][105692] Updated weights for policy 0, policy_version 1609819 (0.0006) [2023-12-27 03:07:17,756][105692] Updated weights for policy 0, policy_version 1609829 (0.0005) [2023-12-27 03:07:18,261][105620] Updated weights for policy 1, policy_version 1613220 (0.0011) [2023-12-27 03:07:18,317][105620] Updated weights for policy 1, policy_version 1613230 (0.0010) [2023-12-27 03:07:18,342][105692] Updated weights for policy 0, policy_version 1609839 (0.0009) [2023-12-27 03:07:18,382][105620] Updated weights for policy 1, policy_version 1613240 (0.0008) [2023-12-27 03:07:18,401][105692] Updated weights for policy 0, policy_version 1609849 (0.0006) [2023-12-27 03:07:18,451][105692] Updated weights for policy 0, policy_version 1609859 (0.0009) [2023-12-27 03:07:19,103][105620] Updated weights for policy 1, policy_version 1613250 (0.0011) [2023-12-27 03:07:19,143][105692] Updated weights for policy 0, policy_version 1609869 (0.0010) [2023-12-27 03:07:19,152][105620] Updated weights for policy 1, policy_version 1613260 (0.0010) [2023-12-27 03:07:19,200][105620] Updated weights for policy 1, policy_version 1613270 (0.0010) [2023-12-27 03:07:19,201][105692] Updated weights for policy 0, policy_version 1609879 (0.0010) [2023-12-27 03:07:19,264][105620] Updated weights for policy 1, policy_version 1613280 (0.0008) [2023-12-27 03:07:19,268][105692] Updated weights for policy 0, policy_version 1609889 (0.0010) [2023-12-27 03:07:19,971][105692] Updated weights for policy 0, policy_version 1609899 (0.0009) [2023-12-27 03:07:20,012][105620] Updated weights for policy 1, policy_version 1613290 (0.0011) [2023-12-27 03:07:20,032][105692] Updated weights for policy 0, policy_version 1609909 (0.0011) [2023-12-27 03:07:20,077][105620] Updated weights for policy 1, policy_version 1613300 (0.0011) [2023-12-27 03:07:20,096][105692] Updated weights for policy 0, policy_version 1609919 (0.0011) [2023-12-27 03:07:20,142][105620] Updated weights for policy 1, policy_version 1613310 (0.0011) [2023-12-27 03:07:20,792][105692] Updated weights for policy 0, policy_version 1609929 (0.0011) [2023-12-27 03:07:20,853][105692] Updated weights for policy 0, policy_version 1609939 (0.0008) [2023-12-27 03:07:20,896][105620] Updated weights for policy 1, policy_version 1613320 (0.0011) [2023-12-27 03:07:20,910][105692] Updated weights for policy 0, policy_version 1609949 (0.0008) [2023-12-27 03:07:20,959][105620] Updated weights for policy 1, policy_version 1613330 (0.0011) [2023-12-27 03:07:20,961][105692] Updated weights for policy 0, policy_version 1609959 (0.0006) [2023-12-27 03:07:21,035][105620] Updated weights for policy 1, policy_version 1613340 (0.0009) [2023-12-27 03:07:21,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 825278464. Throughput: 0: 9864.4, 1: 9773.4. Samples: 825262592. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:07:21,062][104569] Avg episode reward: [(0, '8438.128'), (1, '8990.928')] [2023-12-27 03:07:21,761][105692] Updated weights for policy 0, policy_version 1609969 (0.0009) [2023-12-27 03:07:21,774][105620] Updated weights for policy 1, policy_version 1613350 (0.0008) [2023-12-27 03:07:21,816][105692] Updated weights for policy 0, policy_version 1609979 (0.0008) [2023-12-27 03:07:21,831][105620] Updated weights for policy 1, policy_version 1613360 (0.0005) [2023-12-27 03:07:21,880][105692] Updated weights for policy 0, policy_version 1609989 (0.0008) [2023-12-27 03:07:21,895][105620] Updated weights for policy 1, policy_version 1613370 (0.0005) [2023-12-27 03:07:22,497][105620] Updated weights for policy 1, policy_version 1613380 (0.0007) [2023-12-27 03:07:22,547][105620] Updated weights for policy 1, policy_version 1613390 (0.0010) [2023-12-27 03:07:22,593][105620] Updated weights for policy 1, policy_version 1613400 (0.0009) [2023-12-27 03:07:22,723][105692] Updated weights for policy 0, policy_version 1609999 (0.0010) [2023-12-27 03:07:22,777][105692] Updated weights for policy 0, policy_version 1610009 (0.0010) [2023-12-27 03:07:22,826][105692] Updated weights for policy 0, policy_version 1610019 (0.0009) [2023-12-27 03:07:23,201][105620] Updated weights for policy 1, policy_version 1613410 (0.0007) [2023-12-27 03:07:23,256][105620] Updated weights for policy 1, policy_version 1613420 (0.0007) [2023-12-27 03:07:23,307][105620] Updated weights for policy 1, policy_version 1613430 (0.0008) [2023-12-27 03:07:23,365][105620] Updated weights for policy 1, policy_version 1613440 (0.0006) [2023-12-27 03:07:23,746][105692] Updated weights for policy 0, policy_version 1610029 (0.0009) [2023-12-27 03:07:23,796][105692] Updated weights for policy 0, policy_version 1610039 (0.0009) [2023-12-27 03:07:23,856][105692] Updated weights for policy 0, policy_version 1610049 (0.0009) [2023-12-27 03:07:23,951][105620] Updated weights for policy 1, policy_version 1613450 (0.0008) [2023-12-27 03:07:24,008][105620] Updated weights for policy 1, policy_version 1613460 (0.0010) [2023-12-27 03:07:24,062][105620] Updated weights for policy 1, policy_version 1613470 (0.0008) [2023-12-27 03:07:24,567][105692] Updated weights for policy 0, policy_version 1610059 (0.0009) [2023-12-27 03:07:24,632][105692] Updated weights for policy 0, policy_version 1610069 (0.0006) [2023-12-27 03:07:24,686][105692] Updated weights for policy 0, policy_version 1610079 (0.0005) [2023-12-27 03:07:24,799][105620] Updated weights for policy 1, policy_version 1613480 (0.0009) [2023-12-27 03:07:24,853][105620] Updated weights for policy 1, policy_version 1613490 (0.0009) [2023-12-27 03:07:24,906][105620] Updated weights for policy 1, policy_version 1613500 (0.0010) [2023-12-27 03:07:25,191][105692] Updated weights for policy 0, policy_version 1610089 (0.0005) [2023-12-27 03:07:25,247][105692] Updated weights for policy 0, policy_version 1610099 (0.0005) [2023-12-27 03:07:25,305][105692] Updated weights for policy 0, policy_version 1610109 (0.0005) [2023-12-27 03:07:25,352][105692] Updated weights for policy 0, policy_version 1610119 (0.0005) [2023-12-27 03:07:25,803][105620] Updated weights for policy 1, policy_version 1613510 (0.0009) [2023-12-27 03:07:25,859][105620] Updated weights for policy 1, policy_version 1613520 (0.0008) [2023-12-27 03:07:25,918][105620] Updated weights for policy 1, policy_version 1613530 (0.0007) [2023-12-27 03:07:25,935][105692] Updated weights for policy 0, policy_version 1610129 (0.0007) [2023-12-27 03:07:25,988][105692] Updated weights for policy 0, policy_version 1610139 (0.0006) [2023-12-27 03:07:26,039][105692] Updated weights for policy 0, policy_version 1610149 (0.0007) [2023-12-27 03:07:26,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 825376768. Throughput: 0: 9726.2, 1: 9822.2. Samples: 825378276. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:07:26,063][104569] Avg episode reward: [(0, '8621.031'), (1, '9172.923')] [2023-12-27 03:07:26,706][105692] Updated weights for policy 0, policy_version 1610160 (0.0010) [2023-12-27 03:07:26,743][105620] Updated weights for policy 1, policy_version 1613540 (0.0007) [2023-12-27 03:07:26,756][105692] Updated weights for policy 0, policy_version 1610170 (0.0006) [2023-12-27 03:07:26,792][105620] Updated weights for policy 1, policy_version 1613550 (0.0009) [2023-12-27 03:07:26,813][105692] Updated weights for policy 0, policy_version 1610180 (0.0006) [2023-12-27 03:07:26,840][105620] Updated weights for policy 1, policy_version 1613560 (0.0007) [2023-12-27 03:07:27,442][105692] Updated weights for policy 0, policy_version 1610190 (0.0008) [2023-12-27 03:07:27,495][105692] Updated weights for policy 0, policy_version 1610201 (0.0006) [2023-12-27 03:07:27,543][105692] Updated weights for policy 0, policy_version 1610211 (0.0005) [2023-12-27 03:07:27,689][105620] Updated weights for policy 1, policy_version 1613570 (0.0008) [2023-12-27 03:07:27,744][105620] Updated weights for policy 1, policy_version 1613580 (0.0009) [2023-12-27 03:07:27,806][105620] Updated weights for policy 1, policy_version 1613590 (0.0007) [2023-12-27 03:07:27,866][105620] Updated weights for policy 1, policy_version 1613600 (0.0008) [2023-12-27 03:07:28,294][105692] Updated weights for policy 0, policy_version 1610221 (0.0007) [2023-12-27 03:07:28,350][105692] Updated weights for policy 0, policy_version 1610232 (0.0009) [2023-12-27 03:07:28,408][105692] Updated weights for policy 0, policy_version 1610242 (0.0010) [2023-12-27 03:07:28,453][105620] Updated weights for policy 1, policy_version 1613610 (0.0007) [2023-12-27 03:07:28,505][105620] Updated weights for policy 1, policy_version 1613621 (0.0009) [2023-12-27 03:07:28,557][105620] Updated weights for policy 1, policy_version 1613631 (0.0009) [2023-12-27 03:07:29,081][105692] Updated weights for policy 0, policy_version 1610252 (0.0006) [2023-12-27 03:07:29,132][105692] Updated weights for policy 0, policy_version 1610262 (0.0005) [2023-12-27 03:07:29,187][105692] Updated weights for policy 0, policy_version 1610272 (0.0005) [2023-12-27 03:07:29,239][105620] Updated weights for policy 1, policy_version 1613641 (0.0009) [2023-12-27 03:07:29,304][105620] Updated weights for policy 1, policy_version 1613651 (0.0009) [2023-12-27 03:07:29,370][105620] Updated weights for policy 1, policy_version 1613661 (0.0008) [2023-12-27 03:07:29,849][105692] Updated weights for policy 0, policy_version 1610282 (0.0007) [2023-12-27 03:07:29,915][105692] Updated weights for policy 0, policy_version 1610292 (0.0011) [2023-12-27 03:07:29,974][105692] Updated weights for policy 0, policy_version 1610302 (0.0011) [2023-12-27 03:07:30,026][105692] Updated weights for policy 0, policy_version 1610312 (0.0011) [2023-12-27 03:07:30,164][105620] Updated weights for policy 1, policy_version 1613671 (0.0008) [2023-12-27 03:07:30,218][105620] Updated weights for policy 1, policy_version 1613681 (0.0007) [2023-12-27 03:07:30,273][105620] Updated weights for policy 1, policy_version 1613691 (0.0008) [2023-12-27 03:07:30,787][105692] Updated weights for policy 0, policy_version 1610322 (0.0005) [2023-12-27 03:07:30,841][105692] Updated weights for policy 0, policy_version 1610332 (0.0006) [2023-12-27 03:07:30,893][105692] Updated weights for policy 0, policy_version 1610342 (0.0006) [2023-12-27 03:07:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 825466880. Throughput: 0: 9795.3, 1: 9832.4. Samples: 825437412. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:07:31,063][104569] Avg episode reward: [(0, '8530.551'), (1, '9264.892')] [2023-12-27 03:07:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001610344_412303360.pth... [2023-12-27 03:07:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001609192_412008448.pth [2023-12-27 03:07:31,078][105620] Updated weights for policy 1, policy_version 1613701 (0.0009) [2023-12-27 03:07:31,138][105620] Updated weights for policy 1, policy_version 1613711 (0.0011) [2023-12-27 03:07:31,194][105620] Updated weights for policy 1, policy_version 1613721 (0.0010) [2023-12-27 03:07:31,235][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001613728_413171712.pth... [2023-12-27 03:07:31,238][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001612576_412876800.pth [2023-12-27 03:07:31,577][105692] Updated weights for policy 0, policy_version 1610352 (0.0007) [2023-12-27 03:07:31,639][105692] Updated weights for policy 0, policy_version 1610362 (0.0007) [2023-12-27 03:07:31,703][105692] Updated weights for policy 0, policy_version 1610372 (0.0006) [2023-12-27 03:07:31,918][105620] Updated weights for policy 1, policy_version 1613731 (0.0010) [2023-12-27 03:07:31,961][105620] Updated weights for policy 1, policy_version 1613741 (0.0006) [2023-12-27 03:07:32,009][105620] Updated weights for policy 1, policy_version 1613751 (0.0006) [2023-12-27 03:07:32,466][105692] Updated weights for policy 0, policy_version 1610382 (0.0010) [2023-12-27 03:07:32,521][105692] Updated weights for policy 0, policy_version 1610392 (0.0010) [2023-12-27 03:07:32,579][105692] Updated weights for policy 0, policy_version 1610402 (0.0010) [2023-12-27 03:07:32,662][105620] Updated weights for policy 1, policy_version 1613761 (0.0008) [2023-12-27 03:07:32,710][105620] Updated weights for policy 1, policy_version 1613771 (0.0005) [2023-12-27 03:07:32,766][105620] Updated weights for policy 1, policy_version 1613781 (0.0005) [2023-12-27 03:07:32,810][105620] Updated weights for policy 1, policy_version 1613791 (0.0005) [2023-12-27 03:07:33,221][105692] Updated weights for policy 0, policy_version 1610412 (0.0011) [2023-12-27 03:07:33,272][105692] Updated weights for policy 0, policy_version 1610422 (0.0010) [2023-12-27 03:07:33,321][105692] Updated weights for policy 0, policy_version 1610432 (0.0008) [2023-12-27 03:07:33,556][105620] Updated weights for policy 1, policy_version 1613801 (0.0008) [2023-12-27 03:07:33,617][105620] Updated weights for policy 1, policy_version 1613812 (0.0011) [2023-12-27 03:07:33,663][105620] Updated weights for policy 1, policy_version 1613822 (0.0008) [2023-12-27 03:07:33,902][105692] Updated weights for policy 0, policy_version 1610442 (0.0006) [2023-12-27 03:07:33,953][105692] Updated weights for policy 0, policy_version 1610452 (0.0010) [2023-12-27 03:07:34,003][105692] Updated weights for policy 0, policy_version 1610462 (0.0010) [2023-12-27 03:07:34,057][105692] Updated weights for policy 0, policy_version 1610472 (0.0010) [2023-12-27 03:07:34,292][105620] Updated weights for policy 1, policy_version 1613832 (0.0009) [2023-12-27 03:07:34,356][105620] Updated weights for policy 1, policy_version 1613842 (0.0005) [2023-12-27 03:07:34,420][105620] Updated weights for policy 1, policy_version 1613852 (0.0008) [2023-12-27 03:07:34,842][105692] Updated weights for policy 0, policy_version 1610482 (0.0008) [2023-12-27 03:07:34,902][105692] Updated weights for policy 0, policy_version 1610492 (0.0008) [2023-12-27 03:07:34,959][105692] Updated weights for policy 0, policy_version 1610502 (0.0009) [2023-12-27 03:07:35,143][105620] Updated weights for policy 1, policy_version 1613862 (0.0009) [2023-12-27 03:07:35,204][105620] Updated weights for policy 1, policy_version 1613872 (0.0010) [2023-12-27 03:07:35,253][105620] Updated weights for policy 1, policy_version 1613882 (0.0008) [2023-12-27 03:07:35,796][105692] Updated weights for policy 0, policy_version 1610512 (0.0010) [2023-12-27 03:07:35,815][105620] Updated weights for policy 1, policy_version 1613892 (0.0006) [2023-12-27 03:07:35,860][105692] Updated weights for policy 0, policy_version 1610522 (0.0008) [2023-12-27 03:07:35,864][105620] Updated weights for policy 1, policy_version 1613902 (0.0005) [2023-12-27 03:07:35,918][105692] Updated weights for policy 0, policy_version 1610532 (0.0008) [2023-12-27 03:07:35,926][105620] Updated weights for policy 1, policy_version 1613912 (0.0007) [2023-12-27 03:07:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 825573376. Throughput: 0: 9938.7, 1: 9777.5. Samples: 825557404. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:07:36,063][104569] Avg episode reward: [(0, '8259.737'), (1, '9080.897')] [2023-12-27 03:07:36,666][105620] Updated weights for policy 1, policy_version 1613922 (0.0008) [2023-12-27 03:07:36,717][105692] Updated weights for policy 0, policy_version 1610542 (0.0008) [2023-12-27 03:07:36,727][105620] Updated weights for policy 1, policy_version 1613932 (0.0008) [2023-12-27 03:07:36,764][105692] Updated weights for policy 0, policy_version 1610552 (0.0006) [2023-12-27 03:07:36,787][105620] Updated weights for policy 1, policy_version 1613942 (0.0008) [2023-12-27 03:07:36,821][105692] Updated weights for policy 0, policy_version 1610562 (0.0009) [2023-12-27 03:07:36,853][105620] Updated weights for policy 1, policy_version 1613952 (0.0010) [2023-12-27 03:07:37,614][105620] Updated weights for policy 1, policy_version 1613962 (0.0007) [2023-12-27 03:07:37,616][105692] Updated weights for policy 0, policy_version 1610572 (0.0009) [2023-12-27 03:07:37,671][105620] Updated weights for policy 1, policy_version 1613972 (0.0007) [2023-12-27 03:07:37,673][105692] Updated weights for policy 0, policy_version 1610582 (0.0006) [2023-12-27 03:07:37,723][105692] Updated weights for policy 0, policy_version 1610592 (0.0006) [2023-12-27 03:07:37,727][105620] Updated weights for policy 1, policy_version 1613982 (0.0008) [2023-12-27 03:07:38,468][105692] Updated weights for policy 0, policy_version 1610602 (0.0008) [2023-12-27 03:07:38,478][105620] Updated weights for policy 1, policy_version 1613992 (0.0008) [2023-12-27 03:07:38,528][105692] Updated weights for policy 0, policy_version 1610612 (0.0006) [2023-12-27 03:07:38,534][105620] Updated weights for policy 1, policy_version 1614002 (0.0007) [2023-12-27 03:07:38,589][105692] Updated weights for policy 0, policy_version 1610622 (0.0006) [2023-12-27 03:07:38,592][105620] Updated weights for policy 1, policy_version 1614012 (0.0007) [2023-12-27 03:07:38,646][105692] Updated weights for policy 0, policy_version 1610632 (0.0006) [2023-12-27 03:07:39,347][105692] Updated weights for policy 0, policy_version 1610642 (0.0008) [2023-12-27 03:07:39,357][105620] Updated weights for policy 1, policy_version 1614022 (0.0008) [2023-12-27 03:07:39,418][105692] Updated weights for policy 0, policy_version 1610652 (0.0008) [2023-12-27 03:07:39,423][105620] Updated weights for policy 1, policy_version 1614032 (0.0008) [2023-12-27 03:07:39,481][105692] Updated weights for policy 0, policy_version 1610662 (0.0006) [2023-12-27 03:07:39,484][105620] Updated weights for policy 1, policy_version 1614042 (0.0008) [2023-12-27 03:07:40,267][105692] Updated weights for policy 0, policy_version 1610672 (0.0008) [2023-12-27 03:07:40,269][105620] Updated weights for policy 1, policy_version 1614052 (0.0010) [2023-12-27 03:07:40,321][105620] Updated weights for policy 1, policy_version 1614062 (0.0011) [2023-12-27 03:07:40,331][105692] Updated weights for policy 0, policy_version 1610682 (0.0006) [2023-12-27 03:07:40,369][105620] Updated weights for policy 1, policy_version 1614072 (0.0009) [2023-12-27 03:07:40,394][105692] Updated weights for policy 0, policy_version 1610692 (0.0007) [2023-12-27 03:07:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19438.7). Total num frames: 825655296. Throughput: 0: 9734.7, 1: 9816.1. Samples: 825669036. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:07:41,062][104569] Avg episode reward: [(0, '8715.241'), (1, '9081.205')] [2023-12-27 03:07:41,113][105620] Updated weights for policy 1, policy_version 1614082 (0.0006) [2023-12-27 03:07:41,148][105692] Updated weights for policy 0, policy_version 1610702 (0.0007) [2023-12-27 03:07:41,177][105620] Updated weights for policy 1, policy_version 1614092 (0.0009) [2023-12-27 03:07:41,197][105692] Updated weights for policy 0, policy_version 1610712 (0.0008) [2023-12-27 03:07:41,242][105620] Updated weights for policy 1, policy_version 1614102 (0.0008) [2023-12-27 03:07:41,256][105692] Updated weights for policy 0, policy_version 1610722 (0.0007) [2023-12-27 03:07:41,312][105620] Updated weights for policy 1, policy_version 1614112 (0.0009) [2023-12-27 03:07:41,879][105692] Updated weights for policy 0, policy_version 1610732 (0.0007) [2023-12-27 03:07:41,939][105692] Updated weights for policy 0, policy_version 1610742 (0.0009) [2023-12-27 03:07:41,996][105692] Updated weights for policy 0, policy_version 1610752 (0.0010) [2023-12-27 03:07:42,153][105620] Updated weights for policy 1, policy_version 1614122 (0.0008) [2023-12-27 03:07:42,211][105620] Updated weights for policy 1, policy_version 1614132 (0.0009) [2023-12-27 03:07:42,273][105620] Updated weights for policy 1, policy_version 1614142 (0.0009) [2023-12-27 03:07:42,775][105692] Updated weights for policy 0, policy_version 1610762 (0.0009) [2023-12-27 03:07:42,835][105692] Updated weights for policy 0, policy_version 1610772 (0.0009) [2023-12-27 03:07:42,884][105692] Updated weights for policy 0, policy_version 1610782 (0.0009) [2023-12-27 03:07:42,939][105692] Updated weights for policy 0, policy_version 1610792 (0.0008) [2023-12-27 03:07:43,035][105620] Updated weights for policy 1, policy_version 1614152 (0.0009) [2023-12-27 03:07:43,102][105620] Updated weights for policy 1, policy_version 1614162 (0.0008) [2023-12-27 03:07:43,171][105620] Updated weights for policy 1, policy_version 1614172 (0.0008) [2023-12-27 03:07:43,725][105692] Updated weights for policy 0, policy_version 1610802 (0.0009) [2023-12-27 03:07:43,746][105620] Updated weights for policy 1, policy_version 1614182 (0.0006) [2023-12-27 03:07:43,785][105692] Updated weights for policy 0, policy_version 1610812 (0.0006) [2023-12-27 03:07:43,812][105620] Updated weights for policy 1, policy_version 1614192 (0.0005) [2023-12-27 03:07:43,843][105692] Updated weights for policy 0, policy_version 1610822 (0.0006) [2023-12-27 03:07:43,876][105620] Updated weights for policy 1, policy_version 1614202 (0.0005) [2023-12-27 03:07:44,386][105620] Updated weights for policy 1, policy_version 1614212 (0.0006) [2023-12-27 03:07:44,442][105620] Updated weights for policy 1, policy_version 1614222 (0.0008) [2023-12-27 03:07:44,493][105620] Updated weights for policy 1, policy_version 1614232 (0.0005) [2023-12-27 03:07:44,692][105692] Updated weights for policy 0, policy_version 1610832 (0.0007) [2023-12-27 03:07:44,741][105692] Updated weights for policy 0, policy_version 1610842 (0.0009) [2023-12-27 03:07:44,801][105692] Updated weights for policy 0, policy_version 1610852 (0.0009) [2023-12-27 03:07:45,139][105620] Updated weights for policy 1, policy_version 1614242 (0.0007) [2023-12-27 03:07:45,202][105620] Updated weights for policy 1, policy_version 1614252 (0.0006) [2023-12-27 03:07:45,257][105620] Updated weights for policy 1, policy_version 1614262 (0.0005) [2023-12-27 03:07:45,320][105620] Updated weights for policy 1, policy_version 1614272 (0.0006) [2023-12-27 03:07:45,691][105692] Updated weights for policy 0, policy_version 1610862 (0.0009) [2023-12-27 03:07:45,740][105692] Updated weights for policy 0, policy_version 1610872 (0.0008) [2023-12-27 03:07:45,789][105692] Updated weights for policy 0, policy_version 1610882 (0.0008) [2023-12-27 03:07:45,966][105620] Updated weights for policy 1, policy_version 1614282 (0.0010) [2023-12-27 03:07:46,030][105620] Updated weights for policy 1, policy_version 1614292 (0.0005) [2023-12-27 03:07:46,062][104569] Fps is (10 sec: 18022.7, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 825753600. Throughput: 0: 9721.8, 1: 9817.5. Samples: 825726420. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:07:46,062][104569] Avg episode reward: [(0, '8808.669'), (1, '9171.601')] [2023-12-27 03:07:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001610888_412442624.pth... [2023-12-27 03:07:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001609736_412147712.pth [2023-12-27 03:07:46,095][105620] Updated weights for policy 1, policy_version 1614302 (0.0009) [2023-12-27 03:07:46,105][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001614304_413319168.pth... [2023-12-27 03:07:46,109][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001613152_413024256.pth [2023-12-27 03:07:46,539][105692] Updated weights for policy 0, policy_version 1610892 (0.0007) [2023-12-27 03:07:46,583][105692] Updated weights for policy 0, policy_version 1610902 (0.0005) [2023-12-27 03:07:46,632][105692] Updated weights for policy 0, policy_version 1610912 (0.0005) [2023-12-27 03:07:46,760][105620] Updated weights for policy 1, policy_version 1614312 (0.0009) [2023-12-27 03:07:46,813][105620] Updated weights for policy 1, policy_version 1614322 (0.0008) [2023-12-27 03:07:46,871][105620] Updated weights for policy 1, policy_version 1614332 (0.0007) [2023-12-27 03:07:47,411][105692] Updated weights for policy 0, policy_version 1610922 (0.0006) [2023-12-27 03:07:47,464][105692] Updated weights for policy 0, policy_version 1610932 (0.0008) [2023-12-27 03:07:47,507][105620] Updated weights for policy 1, policy_version 1614342 (0.0008) [2023-12-27 03:07:47,521][105692] Updated weights for policy 0, policy_version 1610942 (0.0009) [2023-12-27 03:07:47,571][105620] Updated weights for policy 1, policy_version 1614352 (0.0006) [2023-12-27 03:07:47,575][105692] Updated weights for policy 0, policy_version 1610952 (0.0009) [2023-12-27 03:07:47,633][105620] Updated weights for policy 1, policy_version 1614362 (0.0005) [2023-12-27 03:07:48,140][105620] Updated weights for policy 1, policy_version 1614372 (0.0006) [2023-12-27 03:07:48,189][105620] Updated weights for policy 1, policy_version 1614382 (0.0005) [2023-12-27 03:07:48,246][105620] Updated weights for policy 1, policy_version 1614392 (0.0005) [2023-12-27 03:07:48,251][105692] Updated weights for policy 0, policy_version 1610962 (0.0005) [2023-12-27 03:07:48,306][105692] Updated weights for policy 0, policy_version 1610972 (0.0008) [2023-12-27 03:07:48,363][105692] Updated weights for policy 0, policy_version 1610982 (0.0009) [2023-12-27 03:07:48,835][105620] Updated weights for policy 1, policy_version 1614402 (0.0006) [2023-12-27 03:07:48,881][105620] Updated weights for policy 1, policy_version 1614412 (0.0008) [2023-12-27 03:07:48,933][105620] Updated weights for policy 1, policy_version 1614422 (0.0010) [2023-12-27 03:07:48,986][105620] Updated weights for policy 1, policy_version 1614432 (0.0010) [2023-12-27 03:07:49,146][105692] Updated weights for policy 0, policy_version 1610992 (0.0010) [2023-12-27 03:07:49,208][105692] Updated weights for policy 0, policy_version 1611002 (0.0010) [2023-12-27 03:07:49,274][105692] Updated weights for policy 0, policy_version 1611012 (0.0011) [2023-12-27 03:07:49,734][105620] Updated weights for policy 1, policy_version 1614442 (0.0011) [2023-12-27 03:07:49,802][105620] Updated weights for policy 1, policy_version 1614452 (0.0010) [2023-12-27 03:07:49,865][105620] Updated weights for policy 1, policy_version 1614462 (0.0011) [2023-12-27 03:07:50,014][105692] Updated weights for policy 0, policy_version 1611022 (0.0011) [2023-12-27 03:07:50,059][105692] Updated weights for policy 0, policy_version 1611032 (0.0010) [2023-12-27 03:07:50,114][105692] Updated weights for policy 0, policy_version 1611042 (0.0010) [2023-12-27 03:07:50,620][105620] Updated weights for policy 1, policy_version 1614472 (0.0011) [2023-12-27 03:07:50,683][105620] Updated weights for policy 1, policy_version 1614482 (0.0011) [2023-12-27 03:07:50,749][105620] Updated weights for policy 1, policy_version 1614492 (0.0011) [2023-12-27 03:07:50,891][105692] Updated weights for policy 0, policy_version 1611052 (0.0011) [2023-12-27 03:07:50,945][105692] Updated weights for policy 0, policy_version 1611062 (0.0007) [2023-12-27 03:07:51,000][105692] Updated weights for policy 0, policy_version 1611072 (0.0006) [2023-12-27 03:07:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 825860096. Throughput: 0: 9743.9, 1: 9854.0. Samples: 825846468. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:07:51,062][104569] Avg episode reward: [(0, '8625.833'), (1, '9263.156')] [2023-12-27 03:07:51,516][105620] Updated weights for policy 1, policy_version 1614502 (0.0009) [2023-12-27 03:07:51,576][105620] Updated weights for policy 1, policy_version 1614512 (0.0005) [2023-12-27 03:07:51,648][105620] Updated weights for policy 1, policy_version 1614522 (0.0007) [2023-12-27 03:07:51,807][105692] Updated weights for policy 0, policy_version 1611082 (0.0009) [2023-12-27 03:07:51,873][105692] Updated weights for policy 0, policy_version 1611092 (0.0006) [2023-12-27 03:07:51,941][105692] Updated weights for policy 0, policy_version 1611102 (0.0005) [2023-12-27 03:07:52,007][105692] Updated weights for policy 0, policy_version 1611112 (0.0006) [2023-12-27 03:07:52,390][105620] Updated weights for policy 1, policy_version 1614532 (0.0007) [2023-12-27 03:07:52,444][105620] Updated weights for policy 1, policy_version 1614542 (0.0008) [2023-12-27 03:07:52,500][105620] Updated weights for policy 1, policy_version 1614552 (0.0008) [2023-12-27 03:07:52,650][105692] Updated weights for policy 0, policy_version 1611122 (0.0011) [2023-12-27 03:07:52,702][105692] Updated weights for policy 0, policy_version 1611132 (0.0010) [2023-12-27 03:07:52,758][105692] Updated weights for policy 0, policy_version 1611142 (0.0010) [2023-12-27 03:07:53,260][105620] Updated weights for policy 1, policy_version 1614562 (0.0008) [2023-12-27 03:07:53,309][105620] Updated weights for policy 1, policy_version 1614572 (0.0008) [2023-12-27 03:07:53,360][105620] Updated weights for policy 1, policy_version 1614582 (0.0007) [2023-12-27 03:07:53,405][105620] Updated weights for policy 1, policy_version 1614592 (0.0008) [2023-12-27 03:07:53,524][105692] Updated weights for policy 0, policy_version 1611152 (0.0010) [2023-12-27 03:07:53,589][105692] Updated weights for policy 0, policy_version 1611162 (0.0010) [2023-12-27 03:07:53,656][105692] Updated weights for policy 0, policy_version 1611172 (0.0010) [2023-12-27 03:07:54,037][105620] Updated weights for policy 1, policy_version 1614602 (0.0010) [2023-12-27 03:07:54,092][105620] Updated weights for policy 1, policy_version 1614612 (0.0011) [2023-12-27 03:07:54,156][105620] Updated weights for policy 1, policy_version 1614622 (0.0009) [2023-12-27 03:07:54,292][105692] Updated weights for policy 0, policy_version 1611182 (0.0008) [2023-12-27 03:07:54,338][105692] Updated weights for policy 0, policy_version 1611192 (0.0007) [2023-12-27 03:07:54,387][105692] Updated weights for policy 0, policy_version 1611202 (0.0008) [2023-12-27 03:07:55,003][105620] Updated weights for policy 1, policy_version 1614632 (0.0009) [2023-12-27 03:07:55,022][105692] Updated weights for policy 0, policy_version 1611212 (0.0007) [2023-12-27 03:07:55,055][105620] Updated weights for policy 1, policy_version 1614642 (0.0009) [2023-12-27 03:07:55,081][105692] Updated weights for policy 0, policy_version 1611222 (0.0008) [2023-12-27 03:07:55,107][105620] Updated weights for policy 1, policy_version 1614652 (0.0005) [2023-12-27 03:07:55,129][105692] Updated weights for policy 0, policy_version 1611232 (0.0010) [2023-12-27 03:07:55,844][105692] Updated weights for policy 0, policy_version 1611242 (0.0009) [2023-12-27 03:07:55,896][105692] Updated weights for policy 0, policy_version 1611252 (0.0010) [2023-12-27 03:07:55,904][105620] Updated weights for policy 1, policy_version 1614662 (0.0008) [2023-12-27 03:07:55,951][105692] Updated weights for policy 0, policy_version 1611262 (0.0008) [2023-12-27 03:07:55,958][105620] Updated weights for policy 1, policy_version 1614672 (0.0007) [2023-12-27 03:07:56,007][105692] Updated weights for policy 0, policy_version 1611272 (0.0005) [2023-12-27 03:07:56,007][105620] Updated weights for policy 1, policy_version 1614682 (0.0009) [2023-12-27 03:07:56,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.7, 300 sec: 19466.4). Total num frames: 825958400. Throughput: 0: 9714.4, 1: 9748.9. Samples: 825960836. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:07:56,063][104569] Avg episode reward: [(0, '8803.472'), (1, '9264.012')] [2023-12-27 03:07:56,565][105692] Updated weights for policy 0, policy_version 1611282 (0.0005) [2023-12-27 03:07:56,620][105692] Updated weights for policy 0, policy_version 1611292 (0.0005) [2023-12-27 03:07:56,675][105692] Updated weights for policy 0, policy_version 1611302 (0.0005) [2023-12-27 03:07:56,817][105620] Updated weights for policy 1, policy_version 1614692 (0.0007) [2023-12-27 03:07:56,886][105620] Updated weights for policy 1, policy_version 1614702 (0.0005) [2023-12-27 03:07:56,951][105620] Updated weights for policy 1, policy_version 1614712 (0.0006) [2023-12-27 03:07:57,183][105692] Updated weights for policy 0, policy_version 1611312 (0.0005) [2023-12-27 03:07:57,248][105692] Updated weights for policy 0, policy_version 1611322 (0.0005) [2023-12-27 03:07:57,315][105692] Updated weights for policy 0, policy_version 1611332 (0.0007) [2023-12-27 03:07:57,457][105620] Updated weights for policy 1, policy_version 1614722 (0.0005) [2023-12-27 03:07:57,523][105620] Updated weights for policy 1, policy_version 1614732 (0.0005) [2023-12-27 03:07:57,588][105620] Updated weights for policy 1, policy_version 1614742 (0.0006) [2023-12-27 03:07:57,648][105620] Updated weights for policy 1, policy_version 1614752 (0.0010) [2023-12-27 03:07:57,860][105692] Updated weights for policy 0, policy_version 1611342 (0.0007) [2023-12-27 03:07:57,931][105692] Updated weights for policy 0, policy_version 1611352 (0.0009) [2023-12-27 03:07:57,998][105692] Updated weights for policy 0, policy_version 1611362 (0.0011) [2023-12-27 03:07:58,248][105620] Updated weights for policy 1, policy_version 1614762 (0.0008) [2023-12-27 03:07:58,313][105620] Updated weights for policy 1, policy_version 1614772 (0.0008) [2023-12-27 03:07:58,393][105620] Updated weights for policy 1, policy_version 1614782 (0.0007) [2023-12-27 03:07:58,715][105692] Updated weights for policy 0, policy_version 1611372 (0.0009) [2023-12-27 03:07:58,778][105692] Updated weights for policy 0, policy_version 1611382 (0.0008) [2023-12-27 03:07:58,839][105692] Updated weights for policy 0, policy_version 1611392 (0.0008) [2023-12-27 03:07:59,120][105620] Updated weights for policy 1, policy_version 1614792 (0.0008) [2023-12-27 03:07:59,165][105620] Updated weights for policy 1, policy_version 1614802 (0.0008) [2023-12-27 03:07:59,213][105620] Updated weights for policy 1, policy_version 1614812 (0.0005) [2023-12-27 03:07:59,526][105692] Updated weights for policy 0, policy_version 1611402 (0.0008) [2023-12-27 03:07:59,575][105692] Updated weights for policy 0, policy_version 1611412 (0.0006) [2023-12-27 03:07:59,629][105692] Updated weights for policy 0, policy_version 1611422 (0.0009) [2023-12-27 03:07:59,677][105692] Updated weights for policy 0, policy_version 1611432 (0.0010) [2023-12-27 03:07:59,902][105620] Updated weights for policy 1, policy_version 1614822 (0.0009) [2023-12-27 03:07:59,969][105620] Updated weights for policy 1, policy_version 1614832 (0.0008) [2023-12-27 03:08:00,029][105620] Updated weights for policy 1, policy_version 1614842 (0.0007) [2023-12-27 03:08:00,317][105692] Updated weights for policy 0, policy_version 1611442 (0.0005) [2023-12-27 03:08:00,366][105692] Updated weights for policy 0, policy_version 1611452 (0.0009) [2023-12-27 03:08:00,408][105692] Updated weights for policy 0, policy_version 1611462 (0.0008) [2023-12-27 03:08:00,576][105620] Updated weights for policy 1, policy_version 1614852 (0.0005) [2023-12-27 03:08:00,631][105620] Updated weights for policy 1, policy_version 1614862 (0.0006) [2023-12-27 03:08:00,683][105620] Updated weights for policy 1, policy_version 1614872 (0.0009) [2023-12-27 03:08:00,970][105585] KL-divergence is very high: 100.8286 [2023-12-27 03:08:00,997][105692] Updated weights for policy 0, policy_version 1611472 (0.0008) [2023-12-27 03:08:01,040][105585] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000003 [2023-12-27 03:08:01,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19933.8, 300 sec: 19466.4). Total num frames: 826064896. Throughput: 0: 9826.1, 1: 9810.9. Samples: 826025880. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:08:01,063][104569] Avg episode reward: [(0, '8531.091'), (1, '9174.196')] [2023-12-27 03:08:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001611480_412598272.pth... [2023-12-27 03:08:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001614880_413466624.pth... [2023-12-27 03:08:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001610344_412303360.pth [2023-12-27 03:08:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001613728_413171712.pth [2023-12-27 03:08:01,392][105620] Updated weights for policy 1, policy_version 1614884 (0.0010) [2023-12-27 03:08:01,440][105620] Updated weights for policy 1, policy_version 1614894 (0.0009) [2023-12-27 03:08:01,496][105620] Updated weights for policy 1, policy_version 1614904 (0.0008) [2023-12-27 03:08:01,871][105692] Updated weights for policy 0, policy_version 1611482 (0.0008) [2023-12-27 03:08:01,931][105692] Updated weights for policy 0, policy_version 1611492 (0.0006) [2023-12-27 03:08:01,989][105692] Updated weights for policy 0, policy_version 1611502 (0.0011) [2023-12-27 03:08:02,041][105692] Updated weights for policy 0, policy_version 1611512 (0.0011) [2023-12-27 03:08:02,296][105620] Updated weights for policy 1, policy_version 1614914 (0.0008) [2023-12-27 03:08:02,363][105620] Updated weights for policy 1, policy_version 1614924 (0.0008) [2023-12-27 03:08:02,413][105620] Updated weights for policy 1, policy_version 1614934 (0.0008) [2023-12-27 03:08:02,470][105620] Updated weights for policy 1, policy_version 1614944 (0.0008) [2023-12-27 03:08:02,772][105692] Updated weights for policy 0, policy_version 1611522 (0.0010) [2023-12-27 03:08:02,831][105692] Updated weights for policy 0, policy_version 1611532 (0.0007) [2023-12-27 03:08:02,893][105692] Updated weights for policy 0, policy_version 1611542 (0.0009) [2023-12-27 03:08:03,171][105620] Updated weights for policy 1, policy_version 1614955 (0.0010) [2023-12-27 03:08:03,223][105620] Updated weights for policy 1, policy_version 1614966 (0.0010) [2023-12-27 03:08:03,473][105692] Updated weights for policy 0, policy_version 1611552 (0.0007) [2023-12-27 03:08:03,529][105692] Updated weights for policy 0, policy_version 1611562 (0.0006) [2023-12-27 03:08:03,585][105692] Updated weights for policy 0, policy_version 1611572 (0.0006) [2023-12-27 03:08:04,068][105620] Updated weights for policy 1, policy_version 1614977 (0.0010) [2023-12-27 03:08:04,128][105620] Updated weights for policy 1, policy_version 1614987 (0.0008) [2023-12-27 03:08:04,184][105620] Updated weights for policy 1, policy_version 1614997 (0.0007) [2023-12-27 03:08:04,204][105692] Updated weights for policy 0, policy_version 1611582 (0.0008) [2023-12-27 03:08:04,233][105620] Updated weights for policy 1, policy_version 1615007 (0.0008) [2023-12-27 03:08:04,253][105692] Updated weights for policy 0, policy_version 1611592 (0.0010) [2023-12-27 03:08:04,301][105692] Updated weights for policy 0, policy_version 1611602 (0.0010) [2023-12-27 03:08:05,014][105620] Updated weights for policy 1, policy_version 1615017 (0.0008) [2023-12-27 03:08:05,058][105620] Updated weights for policy 1, policy_version 1615027 (0.0007) [2023-12-27 03:08:05,064][105692] Updated weights for policy 0, policy_version 1611612 (0.0011) [2023-12-27 03:08:05,110][105620] Updated weights for policy 1, policy_version 1615037 (0.0006) [2023-12-27 03:08:05,119][105692] Updated weights for policy 0, policy_version 1611622 (0.0010) [2023-12-27 03:08:05,171][105692] Updated weights for policy 0, policy_version 1611632 (0.0010) [2023-12-27 03:08:05,856][105692] Updated weights for policy 0, policy_version 1611642 (0.0009) [2023-12-27 03:08:05,894][105620] Updated weights for policy 1, policy_version 1615047 (0.0008) [2023-12-27 03:08:05,907][105692] Updated weights for policy 0, policy_version 1611652 (0.0005) [2023-12-27 03:08:05,945][105620] Updated weights for policy 1, policy_version 1615057 (0.0007) [2023-12-27 03:08:05,961][105692] Updated weights for policy 0, policy_version 1611662 (0.0005) [2023-12-27 03:08:05,998][105620] Updated weights for policy 1, policy_version 1615067 (0.0009) [2023-12-27 03:08:06,028][105692] Updated weights for policy 0, policy_version 1611672 (0.0006) [2023-12-27 03:08:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 826163200. Throughput: 0: 9810.0, 1: 9828.1. Samples: 826146312. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:08:06,063][104569] Avg episode reward: [(0, '7893.260'), (1, '9083.340')] [2023-12-27 03:08:06,646][105692] Updated weights for policy 0, policy_version 1611682 (0.0011) [2023-12-27 03:08:06,713][105692] Updated weights for policy 0, policy_version 1611692 (0.0011) [2023-12-27 03:08:06,759][105620] Updated weights for policy 1, policy_version 1615077 (0.0008) [2023-12-27 03:08:06,773][105692] Updated weights for policy 0, policy_version 1611702 (0.0011) [2023-12-27 03:08:06,816][105620] Updated weights for policy 1, policy_version 1615087 (0.0007) [2023-12-27 03:08:06,866][105620] Updated weights for policy 1, policy_version 1615097 (0.0008) [2023-12-27 03:08:07,485][105692] Updated weights for policy 0, policy_version 1611712 (0.0011) [2023-12-27 03:08:07,544][105692] Updated weights for policy 0, policy_version 1611722 (0.0010) [2023-12-27 03:08:07,607][105692] Updated weights for policy 0, policy_version 1611732 (0.0010) [2023-12-27 03:08:07,616][105620] Updated weights for policy 1, policy_version 1615107 (0.0008) [2023-12-27 03:08:07,670][105620] Updated weights for policy 1, policy_version 1615117 (0.0009) [2023-12-27 03:08:07,729][105620] Updated weights for policy 1, policy_version 1615128 (0.0009) [2023-12-27 03:08:08,210][105692] Updated weights for policy 0, policy_version 1611742 (0.0007) [2023-12-27 03:08:08,263][105692] Updated weights for policy 0, policy_version 1611752 (0.0005) [2023-12-27 03:08:08,318][105692] Updated weights for policy 0, policy_version 1611762 (0.0011) [2023-12-27 03:08:08,592][105620] Updated weights for policy 1, policy_version 1615138 (0.0008) [2023-12-27 03:08:08,652][105620] Updated weights for policy 1, policy_version 1615148 (0.0008) [2023-12-27 03:08:08,708][105620] Updated weights for policy 1, policy_version 1615158 (0.0008) [2023-12-27 03:08:08,763][105620] Updated weights for policy 1, policy_version 1615168 (0.0008) [2023-12-27 03:08:08,999][105692] Updated weights for policy 0, policy_version 1611772 (0.0011) [2023-12-27 03:08:09,047][105692] Updated weights for policy 0, policy_version 1611782 (0.0010) [2023-12-27 03:08:09,106][105692] Updated weights for policy 0, policy_version 1611792 (0.0010) [2023-12-27 03:08:09,602][105620] Updated weights for policy 1, policy_version 1615178 (0.0008) [2023-12-27 03:08:09,666][105620] Updated weights for policy 1, policy_version 1615188 (0.0008) [2023-12-27 03:08:09,721][105620] Updated weights for policy 1, policy_version 1615198 (0.0009) [2023-12-27 03:08:09,755][105692] Updated weights for policy 0, policy_version 1611802 (0.0009) [2023-12-27 03:08:09,816][105692] Updated weights for policy 0, policy_version 1611812 (0.0009) [2023-12-27 03:08:09,888][105692] Updated weights for policy 0, policy_version 1611822 (0.0007) [2023-12-27 03:08:09,956][105692] Updated weights for policy 0, policy_version 1611832 (0.0008) [2023-12-27 03:08:10,530][105620] Updated weights for policy 1, policy_version 1615208 (0.0008) [2023-12-27 03:08:10,586][105620] Updated weights for policy 1, policy_version 1615218 (0.0007) [2023-12-27 03:08:10,596][105692] Updated weights for policy 0, policy_version 1611842 (0.0008) [2023-12-27 03:08:10,648][105620] Updated weights for policy 1, policy_version 1615228 (0.0006) [2023-12-27 03:08:10,654][105692] Updated weights for policy 0, policy_version 1611852 (0.0007) [2023-12-27 03:08:10,713][105692] Updated weights for policy 0, policy_version 1611862 (0.0008) [2023-12-27 03:08:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 826253312. Throughput: 0: 9920.7, 1: 9710.4. Samples: 826261676. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:08:11,062][104569] Avg episode reward: [(0, '8626.612'), (1, '8988.444')] [2023-12-27 03:08:11,399][105692] Updated weights for policy 0, policy_version 1611872 (0.0007) [2023-12-27 03:08:11,454][105620] Updated weights for policy 1, policy_version 1615238 (0.0006) [2023-12-27 03:08:11,470][105692] Updated weights for policy 0, policy_version 1611882 (0.0010) [2023-12-27 03:08:11,518][105620] Updated weights for policy 1, policy_version 1615248 (0.0010) [2023-12-27 03:08:11,528][105692] Updated weights for policy 0, policy_version 1611892 (0.0009) [2023-12-27 03:08:11,575][105620] Updated weights for policy 1, policy_version 1615258 (0.0009) [2023-12-27 03:08:12,192][105692] Updated weights for policy 0, policy_version 1611902 (0.0009) [2023-12-27 03:08:12,259][105692] Updated weights for policy 0, policy_version 1611912 (0.0007) [2023-12-27 03:08:12,323][105692] Updated weights for policy 0, policy_version 1611922 (0.0008) [2023-12-27 03:08:12,359][105620] Updated weights for policy 1, policy_version 1615268 (0.0008) [2023-12-27 03:08:12,423][105620] Updated weights for policy 1, policy_version 1615278 (0.0010) [2023-12-27 03:08:12,484][105620] Updated weights for policy 1, policy_version 1615288 (0.0010) [2023-12-27 03:08:12,976][105692] Updated weights for policy 0, policy_version 1611932 (0.0009) [2023-12-27 03:08:13,030][105692] Updated weights for policy 0, policy_version 1611942 (0.0010) [2023-12-27 03:08:13,082][105692] Updated weights for policy 0, policy_version 1611952 (0.0011) [2023-12-27 03:08:13,257][105620] Updated weights for policy 1, policy_version 1615298 (0.0009) [2023-12-27 03:08:13,309][105620] Updated weights for policy 1, policy_version 1615308 (0.0008) [2023-12-27 03:08:13,371][105620] Updated weights for policy 1, policy_version 1615318 (0.0007) [2023-12-27 03:08:13,436][105620] Updated weights for policy 1, policy_version 1615328 (0.0008) [2023-12-27 03:08:13,858][105692] Updated weights for policy 0, policy_version 1611962 (0.0010) [2023-12-27 03:08:13,921][105692] Updated weights for policy 0, policy_version 1611972 (0.0011) [2023-12-27 03:08:13,977][105692] Updated weights for policy 0, policy_version 1611982 (0.0010) [2023-12-27 03:08:14,034][105692] Updated weights for policy 0, policy_version 1611992 (0.0011) [2023-12-27 03:08:14,178][105620] Updated weights for policy 1, policy_version 1615338 (0.0009) [2023-12-27 03:08:14,237][105620] Updated weights for policy 1, policy_version 1615348 (0.0005) [2023-12-27 03:08:14,283][105620] Updated weights for policy 1, policy_version 1615358 (0.0005) [2023-12-27 03:08:14,676][105692] Updated weights for policy 0, policy_version 1612002 (0.0005) [2023-12-27 03:08:14,728][105692] Updated weights for policy 0, policy_version 1612012 (0.0005) [2023-12-27 03:08:14,790][105692] Updated weights for policy 0, policy_version 1612022 (0.0011) [2023-12-27 03:08:14,870][105620] Updated weights for policy 1, policy_version 1615368 (0.0005) [2023-12-27 03:08:14,935][105620] Updated weights for policy 1, policy_version 1615378 (0.0005) [2023-12-27 03:08:14,995][105620] Updated weights for policy 1, policy_version 1615388 (0.0006) [2023-12-27 03:08:15,432][105692] Updated weights for policy 0, policy_version 1612032 (0.0006) [2023-12-27 03:08:15,483][105692] Updated weights for policy 0, policy_version 1612042 (0.0010) [2023-12-27 03:08:15,546][105692] Updated weights for policy 0, policy_version 1612052 (0.0011) [2023-12-27 03:08:15,640][105620] Updated weights for policy 1, policy_version 1615398 (0.0011) [2023-12-27 03:08:15,701][105620] Updated weights for policy 1, policy_version 1615408 (0.0009) [2023-12-27 03:08:15,767][105620] Updated weights for policy 1, policy_version 1615418 (0.0010) [2023-12-27 03:08:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 826351616. Throughput: 0: 9896.4, 1: 9685.8. Samples: 826318608. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:08:16,063][104569] Avg episode reward: [(0, '8530.273'), (1, '8986.710')] [2023-12-27 03:08:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001612056_412745728.pth... [2023-12-27 03:08:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001615424_413605888.pth... [2023-12-27 03:08:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001610888_412442624.pth [2023-12-27 03:08:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001614304_413319168.pth [2023-12-27 03:08:16,305][105692] Updated weights for policy 0, policy_version 1612062 (0.0010) [2023-12-27 03:08:16,360][105692] Updated weights for policy 0, policy_version 1612072 (0.0010) [2023-12-27 03:08:16,426][105692] Updated weights for policy 0, policy_version 1612082 (0.0008) [2023-12-27 03:08:16,506][105620] Updated weights for policy 1, policy_version 1615428 (0.0010) [2023-12-27 03:08:16,566][105620] Updated weights for policy 1, policy_version 1615438 (0.0011) [2023-12-27 03:08:16,626][105620] Updated weights for policy 1, policy_version 1615448 (0.0011) [2023-12-27 03:08:17,080][105692] Updated weights for policy 0, policy_version 1612092 (0.0007) [2023-12-27 03:08:17,125][105692] Updated weights for policy 0, policy_version 1612102 (0.0009) [2023-12-27 03:08:17,174][105692] Updated weights for policy 0, policy_version 1612112 (0.0008) [2023-12-27 03:08:17,317][105620] Updated weights for policy 1, policy_version 1615458 (0.0010) [2023-12-27 03:08:17,365][105620] Updated weights for policy 1, policy_version 1615468 (0.0005) [2023-12-27 03:08:17,425][105620] Updated weights for policy 1, policy_version 1615478 (0.0005) [2023-12-27 03:08:17,492][105620] Updated weights for policy 1, policy_version 1615488 (0.0006) [2023-12-27 03:08:17,916][105692] Updated weights for policy 0, policy_version 1612122 (0.0008) [2023-12-27 03:08:17,964][105692] Updated weights for policy 0, policy_version 1612132 (0.0005) [2023-12-27 03:08:18,020][105692] Updated weights for policy 0, policy_version 1612142 (0.0006) [2023-12-27 03:08:18,075][105692] Updated weights for policy 0, policy_version 1612152 (0.0009) [2023-12-27 03:08:18,105][105620] Updated weights for policy 1, policy_version 1615498 (0.0008) [2023-12-27 03:08:18,161][105620] Updated weights for policy 1, policy_version 1615508 (0.0010) [2023-12-27 03:08:18,221][105620] Updated weights for policy 1, policy_version 1615518 (0.0011) [2023-12-27 03:08:18,700][105692] Updated weights for policy 0, policy_version 1612162 (0.0005) [2023-12-27 03:08:18,760][105692] Updated weights for policy 0, policy_version 1612172 (0.0006) [2023-12-27 03:08:18,815][105692] Updated weights for policy 0, policy_version 1612182 (0.0005) [2023-12-27 03:08:18,951][105620] Updated weights for policy 1, policy_version 1615528 (0.0010) [2023-12-27 03:08:19,008][105620] Updated weights for policy 1, policy_version 1615538 (0.0010) [2023-12-27 03:08:19,063][105620] Updated weights for policy 1, policy_version 1615548 (0.0010) [2023-12-27 03:08:19,471][105692] Updated weights for policy 0, policy_version 1612192 (0.0006) [2023-12-27 03:08:19,553][105692] Updated weights for policy 0, policy_version 1612202 (0.0009) [2023-12-27 03:08:19,616][105692] Updated weights for policy 0, policy_version 1612212 (0.0011) [2023-12-27 03:08:19,846][105620] Updated weights for policy 1, policy_version 1615558 (0.0011) [2023-12-27 03:08:19,908][105620] Updated weights for policy 1, policy_version 1615568 (0.0011) [2023-12-27 03:08:19,977][105620] Updated weights for policy 1, policy_version 1615578 (0.0007) [2023-12-27 03:08:20,349][105692] Updated weights for policy 0, policy_version 1612222 (0.0010) [2023-12-27 03:08:20,403][105692] Updated weights for policy 0, policy_version 1612232 (0.0006) [2023-12-27 03:08:20,455][105692] Updated weights for policy 0, policy_version 1612242 (0.0005) [2023-12-27 03:08:20,640][105620] Updated weights for policy 1, policy_version 1615588 (0.0007) [2023-12-27 03:08:20,702][105620] Updated weights for policy 1, policy_version 1615598 (0.0008) [2023-12-27 03:08:20,769][105620] Updated weights for policy 1, policy_version 1615608 (0.0010) [2023-12-27 03:08:21,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 826449920. Throughput: 0: 9903.6, 1: 9703.5. Samples: 826439720. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:08:21,063][104569] Avg episode reward: [(0, '8620.482'), (1, '9080.037')] [2023-12-27 03:08:21,207][105692] Updated weights for policy 0, policy_version 1612252 (0.0010) [2023-12-27 03:08:21,268][105692] Updated weights for policy 0, policy_version 1612262 (0.0009) [2023-12-27 03:08:21,317][105692] Updated weights for policy 0, policy_version 1612272 (0.0009) [2023-12-27 03:08:21,511][105620] Updated weights for policy 1, policy_version 1615618 (0.0011) [2023-12-27 03:08:21,577][105620] Updated weights for policy 1, policy_version 1615628 (0.0011) [2023-12-27 03:08:21,641][105620] Updated weights for policy 1, policy_version 1615638 (0.0009) [2023-12-27 03:08:21,707][105620] Updated weights for policy 1, policy_version 1615648 (0.0008) [2023-12-27 03:08:22,122][105692] Updated weights for policy 0, policy_version 1612282 (0.0008) [2023-12-27 03:08:22,193][105692] Updated weights for policy 0, policy_version 1612292 (0.0008) [2023-12-27 03:08:22,257][105692] Updated weights for policy 0, policy_version 1612302 (0.0008) [2023-12-27 03:08:22,322][105692] Updated weights for policy 0, policy_version 1612312 (0.0008) [2023-12-27 03:08:22,499][105620] Updated weights for policy 1, policy_version 1615658 (0.0011) [2023-12-27 03:08:22,559][105620] Updated weights for policy 1, policy_version 1615668 (0.0011) [2023-12-27 03:08:22,622][105620] Updated weights for policy 1, policy_version 1615678 (0.0011) [2023-12-27 03:08:23,087][105692] Updated weights for policy 0, policy_version 1612322 (0.0008) [2023-12-27 03:08:23,150][105692] Updated weights for policy 0, policy_version 1612332 (0.0010) [2023-12-27 03:08:23,203][105692] Updated weights for policy 0, policy_version 1612342 (0.0008) [2023-12-27 03:08:23,292][105620] Updated weights for policy 1, policy_version 1615688 (0.0011) [2023-12-27 03:08:23,350][105620] Updated weights for policy 1, policy_version 1615698 (0.0009) [2023-12-27 03:08:23,422][105620] Updated weights for policy 1, policy_version 1615708 (0.0006) [2023-12-27 03:08:23,870][105692] Updated weights for policy 0, policy_version 1612352 (0.0010) [2023-12-27 03:08:23,924][105692] Updated weights for policy 0, policy_version 1612363 (0.0010) [2023-12-27 03:08:23,974][105620] Updated weights for policy 1, policy_version 1615718 (0.0006) [2023-12-27 03:08:23,974][105692] Updated weights for policy 0, policy_version 1612373 (0.0009) [2023-12-27 03:08:24,032][105620] Updated weights for policy 1, policy_version 1615728 (0.0005) [2023-12-27 03:08:24,083][105620] Updated weights for policy 1, policy_version 1615738 (0.0005) [2023-12-27 03:08:24,772][105620] Updated weights for policy 1, policy_version 1615748 (0.0007) [2023-12-27 03:08:24,811][105692] Updated weights for policy 0, policy_version 1612383 (0.0007) [2023-12-27 03:08:24,820][105620] Updated weights for policy 1, policy_version 1615758 (0.0010) [2023-12-27 03:08:24,855][105692] Updated weights for policy 0, policy_version 1612393 (0.0007) [2023-12-27 03:08:24,868][105620] Updated weights for policy 1, policy_version 1615768 (0.0010) [2023-12-27 03:08:24,910][105692] Updated weights for policy 0, policy_version 1612403 (0.0006) [2023-12-27 03:08:25,636][105620] Updated weights for policy 1, policy_version 1615778 (0.0010) [2023-12-27 03:08:25,697][105692] Updated weights for policy 0, policy_version 1612413 (0.0007) [2023-12-27 03:08:25,697][105620] Updated weights for policy 1, policy_version 1615788 (0.0010) [2023-12-27 03:08:25,748][105620] Updated weights for policy 1, policy_version 1615798 (0.0010) [2023-12-27 03:08:25,752][105692] Updated weights for policy 0, policy_version 1612423 (0.0006) [2023-12-27 03:08:25,799][105620] Updated weights for policy 1, policy_version 1615808 (0.0010) [2023-12-27 03:08:25,811][105692] Updated weights for policy 0, policy_version 1612433 (0.0006) [2023-12-27 03:08:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 826548224. Throughput: 0: 9928.0, 1: 9743.3. Samples: 826554244. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:08:26,062][104569] Avg episode reward: [(0, '8896.012'), (1, '9172.794')] [2023-12-27 03:08:26,370][105692] Updated weights for policy 0, policy_version 1612443 (0.0005) [2023-12-27 03:08:26,424][105692] Updated weights for policy 0, policy_version 1612453 (0.0005) [2023-12-27 03:08:26,487][105692] Updated weights for policy 0, policy_version 1612463 (0.0008) [2023-12-27 03:08:26,615][105620] Updated weights for policy 1, policy_version 1615818 (0.0009) [2023-12-27 03:08:26,667][105620] Updated weights for policy 1, policy_version 1615828 (0.0006) [2023-12-27 03:08:26,714][105620] Updated weights for policy 1, policy_version 1615838 (0.0005) [2023-12-27 03:08:27,047][105692] Updated weights for policy 0, policy_version 1612473 (0.0010) [2023-12-27 03:08:27,102][105692] Updated weights for policy 0, policy_version 1612483 (0.0005) [2023-12-27 03:08:27,159][105692] Updated weights for policy 0, policy_version 1612493 (0.0005) [2023-12-27 03:08:27,218][105692] Updated weights for policy 0, policy_version 1612503 (0.0010) [2023-12-27 03:08:27,419][105620] Updated weights for policy 1, policy_version 1615848 (0.0009) [2023-12-27 03:08:27,488][105620] Updated weights for policy 1, policy_version 1615858 (0.0009) [2023-12-27 03:08:27,539][105620] Updated weights for policy 1, policy_version 1615869 (0.0009) [2023-12-27 03:08:27,779][105692] Updated weights for policy 0, policy_version 1612513 (0.0006) [2023-12-27 03:08:27,848][105692] Updated weights for policy 0, policy_version 1612523 (0.0005) [2023-12-27 03:08:27,905][105692] Updated weights for policy 0, policy_version 1612533 (0.0008) [2023-12-27 03:08:28,382][105620] Updated weights for policy 1, policy_version 1615879 (0.0007) [2023-12-27 03:08:28,442][105620] Updated weights for policy 1, policy_version 1615890 (0.0010) [2023-12-27 03:08:28,492][105620] Updated weights for policy 1, policy_version 1615900 (0.0008) [2023-12-27 03:08:28,505][105692] Updated weights for policy 0, policy_version 1612543 (0.0007) [2023-12-27 03:08:28,565][105692] Updated weights for policy 0, policy_version 1612553 (0.0005) [2023-12-27 03:08:28,626][105692] Updated weights for policy 0, policy_version 1612563 (0.0006) [2023-12-27 03:08:29,266][105620] Updated weights for policy 1, policy_version 1615910 (0.0008) [2023-12-27 03:08:29,316][105692] Updated weights for policy 0, policy_version 1612573 (0.0011) [2023-12-27 03:08:29,319][105620] Updated weights for policy 1, policy_version 1615920 (0.0008) [2023-12-27 03:08:29,381][105692] Updated weights for policy 0, policy_version 1612583 (0.0011) [2023-12-27 03:08:29,387][105620] Updated weights for policy 1, policy_version 1615930 (0.0007) [2023-12-27 03:08:29,443][105692] Updated weights for policy 0, policy_version 1612593 (0.0010) [2023-12-27 03:08:30,132][105620] Updated weights for policy 1, policy_version 1615940 (0.0007) [2023-12-27 03:08:30,183][105692] Updated weights for policy 0, policy_version 1612603 (0.0012) [2023-12-27 03:08:30,191][105620] Updated weights for policy 1, policy_version 1615950 (0.0008) [2023-12-27 03:08:30,235][105692] Updated weights for policy 0, policy_version 1612613 (0.0010) [2023-12-27 03:08:30,248][105620] Updated weights for policy 1, policy_version 1615960 (0.0007) [2023-12-27 03:08:30,255][105586] KL-divergence is very high: 121.2697 [2023-12-27 03:08:30,289][105692] Updated weights for policy 0, policy_version 1612623 (0.0010) [2023-12-27 03:08:30,934][105620] Updated weights for policy 1, policy_version 1615970 (0.0007) [2023-12-27 03:08:30,986][105620] Updated weights for policy 1, policy_version 1615980 (0.0005) [2023-12-27 03:08:31,048][105692] Updated weights for policy 0, policy_version 1612633 (0.0010) [2023-12-27 03:08:31,052][105620] Updated weights for policy 1, policy_version 1615990 (0.0006) [2023-12-27 03:08:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 826638336. Throughput: 0: 10061.4, 1: 9703.9. Samples: 826615860. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:08:31,063][104569] Avg episode reward: [(0, '8715.200'), (1, '8993.450')] [2023-12-27 03:08:31,105][105692] Updated weights for policy 0, policy_version 1612643 (0.0010) [2023-12-27 03:08:31,112][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001616000_413753344.pth... [2023-12-27 03:08:31,114][105620] Updated weights for policy 1, policy_version 1616000 (0.0007) [2023-12-27 03:08:31,117][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001614880_413466624.pth [2023-12-27 03:08:31,172][105692] Updated weights for policy 0, policy_version 1612653 (0.0011) [2023-12-27 03:08:31,238][105692] Updated weights for policy 0, policy_version 1612663 (0.0008) [2023-12-27 03:08:31,242][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001612664_412901376.pth... [2023-12-27 03:08:31,247][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001611480_412598272.pth [2023-12-27 03:08:31,703][105620] Updated weights for policy 1, policy_version 1616010 (0.0008) [2023-12-27 03:08:31,764][105620] Updated weights for policy 1, policy_version 1616020 (0.0009) [2023-12-27 03:08:31,818][105620] Updated weights for policy 1, policy_version 1616030 (0.0008) [2023-12-27 03:08:32,052][105692] Updated weights for policy 0, policy_version 1612673 (0.0008) [2023-12-27 03:08:32,110][105692] Updated weights for policy 0, policy_version 1612683 (0.0008) [2023-12-27 03:08:32,172][105692] Updated weights for policy 0, policy_version 1612693 (0.0008) [2023-12-27 03:08:32,545][105620] Updated weights for policy 1, policy_version 1616040 (0.0009) [2023-12-27 03:08:32,600][105620] Updated weights for policy 1, policy_version 1616050 (0.0005) [2023-12-27 03:08:32,660][105620] Updated weights for policy 1, policy_version 1616060 (0.0008) [2023-12-27 03:08:32,857][105692] Updated weights for policy 0, policy_version 1612703 (0.0009) [2023-12-27 03:08:32,907][105692] Updated weights for policy 0, policy_version 1612713 (0.0011) [2023-12-27 03:08:32,963][105692] Updated weights for policy 0, policy_version 1612723 (0.0011) [2023-12-27 03:08:33,400][105620] Updated weights for policy 1, policy_version 1616070 (0.0007) [2023-12-27 03:08:33,458][105620] Updated weights for policy 1, policy_version 1616080 (0.0010) [2023-12-27 03:08:33,516][105620] Updated weights for policy 1, policy_version 1616090 (0.0010) [2023-12-27 03:08:33,664][105692] Updated weights for policy 0, policy_version 1612733 (0.0008) [2023-12-27 03:08:33,729][105692] Updated weights for policy 0, policy_version 1612743 (0.0006) [2023-12-27 03:08:33,794][105692] Updated weights for policy 0, policy_version 1612753 (0.0007) [2023-12-27 03:08:34,131][105620] Updated weights for policy 1, policy_version 1616100 (0.0008) [2023-12-27 03:08:34,207][105620] Updated weights for policy 1, policy_version 1616110 (0.0009) [2023-12-27 03:08:34,264][105620] Updated weights for policy 1, policy_version 1616120 (0.0009) [2023-12-27 03:08:34,456][105692] Updated weights for policy 0, policy_version 1612763 (0.0008) [2023-12-27 03:08:34,515][105692] Updated weights for policy 0, policy_version 1612773 (0.0009) [2023-12-27 03:08:34,573][105692] Updated weights for policy 0, policy_version 1612783 (0.0009) [2023-12-27 03:08:35,012][105620] Updated weights for policy 1, policy_version 1616130 (0.0009) [2023-12-27 03:08:35,063][105620] Updated weights for policy 1, policy_version 1616140 (0.0009) [2023-12-27 03:08:35,122][105620] Updated weights for policy 1, policy_version 1616150 (0.0009) [2023-12-27 03:08:35,183][105620] Updated weights for policy 1, policy_version 1616160 (0.0008) [2023-12-27 03:08:35,237][105692] Updated weights for policy 0, policy_version 1612793 (0.0008) [2023-12-27 03:08:35,285][105692] Updated weights for policy 0, policy_version 1612803 (0.0009) [2023-12-27 03:08:35,333][105692] Updated weights for policy 0, policy_version 1612813 (0.0009) [2023-12-27 03:08:35,381][105692] Updated weights for policy 0, policy_version 1612823 (0.0009) [2023-12-27 03:08:35,888][105620] Updated weights for policy 1, policy_version 1616170 (0.0005) [2023-12-27 03:08:35,941][105620] Updated weights for policy 1, policy_version 1616180 (0.0005) [2023-12-27 03:08:35,992][105620] Updated weights for policy 1, policy_version 1616190 (0.0005) [2023-12-27 03:08:36,049][105692] Updated weights for policy 0, policy_version 1612833 (0.0010) [2023-12-27 03:08:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 826744832. Throughput: 0: 10102.5, 1: 9585.5. Samples: 826732428. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:08:36,062][104569] Avg episode reward: [(0, '8805.291'), (1, '8995.687')] [2023-12-27 03:08:36,101][105692] Updated weights for policy 0, policy_version 1612843 (0.0009) [2023-12-27 03:08:36,162][105692] Updated weights for policy 0, policy_version 1612853 (0.0009) [2023-12-27 03:08:36,563][105620] Updated weights for policy 1, policy_version 1616200 (0.0008) [2023-12-27 03:08:36,628][105620] Updated weights for policy 1, policy_version 1616210 (0.0007) [2023-12-27 03:08:36,702][105620] Updated weights for policy 1, policy_version 1616220 (0.0010) [2023-12-27 03:08:36,906][105692] Updated weights for policy 0, policy_version 1612863 (0.0008) [2023-12-27 03:08:36,962][105692] Updated weights for policy 0, policy_version 1612873 (0.0008) [2023-12-27 03:08:37,028][105692] Updated weights for policy 0, policy_version 1612883 (0.0010) [2023-12-27 03:08:37,447][105620] Updated weights for policy 1, policy_version 1616230 (0.0011) [2023-12-27 03:08:37,515][105620] Updated weights for policy 1, policy_version 1616240 (0.0011) [2023-12-27 03:08:37,578][105620] Updated weights for policy 1, policy_version 1616250 (0.0011) [2023-12-27 03:08:37,807][105692] Updated weights for policy 0, policy_version 1612893 (0.0008) [2023-12-27 03:08:37,874][105692] Updated weights for policy 0, policy_version 1612903 (0.0011) [2023-12-27 03:08:37,939][105692] Updated weights for policy 0, policy_version 1612913 (0.0010) [2023-12-27 03:08:38,213][105620] Updated weights for policy 1, policy_version 1616260 (0.0009) [2023-12-27 03:08:38,271][105620] Updated weights for policy 1, policy_version 1616270 (0.0010) [2023-12-27 03:08:38,319][105620] Updated weights for policy 1, policy_version 1616280 (0.0010) [2023-12-27 03:08:38,687][105692] Updated weights for policy 0, policy_version 1612923 (0.0009) [2023-12-27 03:08:38,743][105692] Updated weights for policy 0, policy_version 1612933 (0.0007) [2023-12-27 03:08:38,803][105692] Updated weights for policy 0, policy_version 1612943 (0.0009) [2023-12-27 03:08:38,987][105620] Updated weights for policy 1, policy_version 1616290 (0.0008) [2023-12-27 03:08:39,036][105620] Updated weights for policy 1, policy_version 1616300 (0.0008) [2023-12-27 03:08:39,087][105620] Updated weights for policy 1, policy_version 1616310 (0.0010) [2023-12-27 03:08:39,137][105620] Updated weights for policy 1, policy_version 1616320 (0.0009) [2023-12-27 03:08:39,628][105692] Updated weights for policy 0, policy_version 1612953 (0.0009) [2023-12-27 03:08:39,690][105692] Updated weights for policy 0, policy_version 1612963 (0.0009) [2023-12-27 03:08:39,751][105692] Updated weights for policy 0, policy_version 1612973 (0.0008) [2023-12-27 03:08:39,808][105692] Updated weights for policy 0, policy_version 1612983 (0.0007) [2023-12-27 03:08:39,933][105620] Updated weights for policy 1, policy_version 1616330 (0.0008) [2023-12-27 03:08:40,000][105620] Updated weights for policy 1, policy_version 1616340 (0.0008) [2023-12-27 03:08:40,057][105620] Updated weights for policy 1, policy_version 1616350 (0.0008) [2023-12-27 03:08:40,629][105692] Updated weights for policy 0, policy_version 1612993 (0.0009) [2023-12-27 03:08:40,680][105692] Updated weights for policy 0, policy_version 1613003 (0.0008) [2023-12-27 03:08:40,737][105620] Updated weights for policy 1, policy_version 1616360 (0.0007) [2023-12-27 03:08:40,740][105692] Updated weights for policy 0, policy_version 1613013 (0.0008) [2023-12-27 03:08:40,786][105620] Updated weights for policy 1, policy_version 1616370 (0.0009) [2023-12-27 03:08:40,839][105620] Updated weights for policy 1, policy_version 1616380 (0.0009) [2023-12-27 03:08:41,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.4, 300 sec: 19494.2). Total num frames: 826843136. Throughput: 0: 10053.2, 1: 9677.8. Samples: 826848728. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:08:41,062][104569] Avg episode reward: [(0, '8801.576'), (1, '8903.815')] [2023-12-27 03:08:41,514][105692] Updated weights for policy 0, policy_version 1613023 (0.0008) [2023-12-27 03:08:41,576][105692] Updated weights for policy 0, policy_version 1613033 (0.0009) [2023-12-27 03:08:41,640][105620] Updated weights for policy 1, policy_version 1616390 (0.0010) [2023-12-27 03:08:41,644][105692] Updated weights for policy 0, policy_version 1613043 (0.0008) [2023-12-27 03:08:41,697][105620] Updated weights for policy 1, policy_version 1616400 (0.0011) [2023-12-27 03:08:41,770][105620] Updated weights for policy 1, policy_version 1616410 (0.0010) [2023-12-27 03:08:42,365][105692] Updated weights for policy 0, policy_version 1613053 (0.0008) [2023-12-27 03:08:42,431][105692] Updated weights for policy 0, policy_version 1613063 (0.0008) [2023-12-27 03:08:42,434][105620] Updated weights for policy 1, policy_version 1616420 (0.0009) [2023-12-27 03:08:42,484][105620] Updated weights for policy 1, policy_version 1616430 (0.0007) [2023-12-27 03:08:42,493][105692] Updated weights for policy 0, policy_version 1613073 (0.0009) [2023-12-27 03:08:42,538][105586] KL-divergence is very high: 141.4078 [2023-12-27 03:08:42,545][105620] Updated weights for policy 1, policy_version 1616440 (0.0007) [2023-12-27 03:08:42,587][105586] KL-divergence is very high: 138.7161 [2023-12-27 03:08:43,232][105692] Updated weights for policy 0, policy_version 1613083 (0.0009) [2023-12-27 03:08:43,297][105692] Updated weights for policy 0, policy_version 1613093 (0.0009) [2023-12-27 03:08:43,320][105620] Updated weights for policy 1, policy_version 1616450 (0.0009) [2023-12-27 03:08:43,362][105692] Updated weights for policy 0, policy_version 1613103 (0.0010) [2023-12-27 03:08:43,373][105620] Updated weights for policy 1, policy_version 1616460 (0.0006) [2023-12-27 03:08:43,438][105620] Updated weights for policy 1, policy_version 1616470 (0.0008) [2023-12-27 03:08:43,501][105620] Updated weights for policy 1, policy_version 1616480 (0.0009) [2023-12-27 03:08:44,090][105620] Updated weights for policy 1, policy_version 1616490 (0.0005) [2023-12-27 03:08:44,144][105620] Updated weights for policy 1, policy_version 1616500 (0.0009) [2023-12-27 03:08:44,184][105692] Updated weights for policy 0, policy_version 1613113 (0.0008) [2023-12-27 03:08:44,195][105620] Updated weights for policy 1, policy_version 1616510 (0.0010) [2023-12-27 03:08:44,243][105692] Updated weights for policy 0, policy_version 1613123 (0.0008) [2023-12-27 03:08:44,301][105692] Updated weights for policy 0, policy_version 1613133 (0.0008) [2023-12-27 03:08:44,361][105692] Updated weights for policy 0, policy_version 1613143 (0.0008) [2023-12-27 03:08:44,933][105620] Updated weights for policy 1, policy_version 1616520 (0.0006) [2023-12-27 03:08:45,003][105620] Updated weights for policy 1, policy_version 1616530 (0.0006) [2023-12-27 03:08:45,072][105620] Updated weights for policy 1, policy_version 1616540 (0.0010) [2023-12-27 03:08:45,122][105692] Updated weights for policy 0, policy_version 1613153 (0.0011) [2023-12-27 03:08:45,183][105692] Updated weights for policy 0, policy_version 1613163 (0.0006) [2023-12-27 03:08:45,253][105692] Updated weights for policy 0, policy_version 1613173 (0.0008) [2023-12-27 03:08:45,724][105620] Updated weights for policy 1, policy_version 1616550 (0.0008) [2023-12-27 03:08:45,776][105620] Updated weights for policy 1, policy_version 1616560 (0.0007) [2023-12-27 03:08:45,828][105620] Updated weights for policy 1, policy_version 1616570 (0.0006) [2023-12-27 03:08:45,942][105692] Updated weights for policy 0, policy_version 1613183 (0.0010) [2023-12-27 03:08:45,994][105692] Updated weights for policy 0, policy_version 1613193 (0.0010) [2023-12-27 03:08:46,038][105692] Updated weights for policy 0, policy_version 1613203 (0.0010) [2023-12-27 03:08:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 826941440. Throughput: 0: 9903.4, 1: 9651.3. Samples: 826905840. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:08:46,062][104569] Avg episode reward: [(0, '8167.891'), (1, '8991.760')] [2023-12-27 03:08:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001613208_413040640.pth... [2023-12-27 03:08:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001616576_413900800.pth... [2023-12-27 03:08:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001612056_412745728.pth [2023-12-27 03:08:46,082][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001615424_413605888.pth [2023-12-27 03:08:46,449][105620] Updated weights for policy 1, policy_version 1616580 (0.0007) [2023-12-27 03:08:46,492][105620] Updated weights for policy 1, policy_version 1616590 (0.0010) [2023-12-27 03:08:46,547][105620] Updated weights for policy 1, policy_version 1616600 (0.0006) [2023-12-27 03:08:46,813][105692] Updated weights for policy 0, policy_version 1613213 (0.0010) [2023-12-27 03:08:46,875][105692] Updated weights for policy 0, policy_version 1613223 (0.0010) [2023-12-27 03:08:46,930][105692] Updated weights for policy 0, policy_version 1613233 (0.0010) [2023-12-27 03:08:47,205][105620] Updated weights for policy 1, policy_version 1616610 (0.0005) [2023-12-27 03:08:47,277][105620] Updated weights for policy 1, policy_version 1616620 (0.0006) [2023-12-27 03:08:47,350][105620] Updated weights for policy 1, policy_version 1616630 (0.0007) [2023-12-27 03:08:47,407][105620] Updated weights for policy 1, policy_version 1616640 (0.0007) [2023-12-27 03:08:47,583][105692] Updated weights for policy 0, policy_version 1613243 (0.0009) [2023-12-27 03:08:47,636][105692] Updated weights for policy 0, policy_version 1613253 (0.0008) [2023-12-27 03:08:47,698][105692] Updated weights for policy 0, policy_version 1613263 (0.0008) [2023-12-27 03:08:47,949][105620] Updated weights for policy 1, policy_version 1616650 (0.0009) [2023-12-27 03:08:47,997][105620] Updated weights for policy 1, policy_version 1616660 (0.0009) [2023-12-27 03:08:48,045][105620] Updated weights for policy 1, policy_version 1616670 (0.0009) [2023-12-27 03:08:48,480][105692] Updated weights for policy 0, policy_version 1613273 (0.0010) [2023-12-27 03:08:48,528][105692] Updated weights for policy 0, policy_version 1613283 (0.0008) [2023-12-27 03:08:48,585][105692] Updated weights for policy 0, policy_version 1613293 (0.0009) [2023-12-27 03:08:48,640][105692] Updated weights for policy 0, policy_version 1613303 (0.0010) [2023-12-27 03:08:48,785][105620] Updated weights for policy 1, policy_version 1616680 (0.0010) [2023-12-27 03:08:48,846][105620] Updated weights for policy 1, policy_version 1616690 (0.0008) [2023-12-27 03:08:48,897][105620] Updated weights for policy 1, policy_version 1616700 (0.0009) [2023-12-27 03:08:49,441][105692] Updated weights for policy 0, policy_version 1613313 (0.0009) [2023-12-27 03:08:49,491][105692] Updated weights for policy 0, policy_version 1613323 (0.0009) [2023-12-27 03:08:49,551][105692] Updated weights for policy 0, policy_version 1613333 (0.0009) [2023-12-27 03:08:49,676][105620] Updated weights for policy 1, policy_version 1616710 (0.0007) [2023-12-27 03:08:49,732][105620] Updated weights for policy 1, policy_version 1616720 (0.0009) [2023-12-27 03:08:49,790][105620] Updated weights for policy 1, policy_version 1616730 (0.0009) [2023-12-27 03:08:50,288][105692] Updated weights for policy 0, policy_version 1613343 (0.0010) [2023-12-27 03:08:50,347][105692] Updated weights for policy 0, policy_version 1613354 (0.0010) [2023-12-27 03:08:50,401][105692] Updated weights for policy 0, policy_version 1613365 (0.0010) [2023-12-27 03:08:50,512][105620] Updated weights for policy 1, policy_version 1616740 (0.0007) [2023-12-27 03:08:50,573][105620] Updated weights for policy 1, policy_version 1616750 (0.0006) [2023-12-27 03:08:50,634][105620] Updated weights for policy 1, policy_version 1616760 (0.0009) [2023-12-27 03:08:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 827031552. Throughput: 0: 9758.2, 1: 9706.3. Samples: 827022208. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:08:51,062][104569] Avg episode reward: [(0, '8263.592'), (1, '9080.860')] [2023-12-27 03:08:51,263][105692] Updated weights for policy 0, policy_version 1613375 (0.0010) [2023-12-27 03:08:51,299][105620] Updated weights for policy 1, policy_version 1616770 (0.0008) [2023-12-27 03:08:51,322][105692] Updated weights for policy 0, policy_version 1613385 (0.0008) [2023-12-27 03:08:51,362][105620] Updated weights for policy 1, policy_version 1616780 (0.0009) [2023-12-27 03:08:51,385][105692] Updated weights for policy 0, policy_version 1613395 (0.0008) [2023-12-27 03:08:51,424][105620] Updated weights for policy 1, policy_version 1616790 (0.0009) [2023-12-27 03:08:51,485][105620] Updated weights for policy 1, policy_version 1616800 (0.0009) [2023-12-27 03:08:52,153][105692] Updated weights for policy 0, policy_version 1613405 (0.0007) [2023-12-27 03:08:52,218][105692] Updated weights for policy 0, policy_version 1613415 (0.0009) [2023-12-27 03:08:52,262][105620] Updated weights for policy 1, policy_version 1616810 (0.0007) [2023-12-27 03:08:52,280][105692] Updated weights for policy 0, policy_version 1613425 (0.0007) [2023-12-27 03:08:52,315][105620] Updated weights for policy 1, policy_version 1616820 (0.0008) [2023-12-27 03:08:52,373][105620] Updated weights for policy 1, policy_version 1616830 (0.0009) [2023-12-27 03:08:52,938][105692] Updated weights for policy 0, policy_version 1613435 (0.0007) [2023-12-27 03:08:53,001][105692] Updated weights for policy 0, policy_version 1613445 (0.0008) [2023-12-27 03:08:53,065][105692] Updated weights for policy 0, policy_version 1613455 (0.0009) [2023-12-27 03:08:53,187][105620] Updated weights for policy 1, policy_version 1616840 (0.0009) [2023-12-27 03:08:53,239][105620] Updated weights for policy 1, policy_version 1616850 (0.0010) [2023-12-27 03:08:53,296][105620] Updated weights for policy 1, policy_version 1616860 (0.0010) [2023-12-27 03:08:53,715][105692] Updated weights for policy 0, policy_version 1613465 (0.0008) [2023-12-27 03:08:53,766][105692] Updated weights for policy 0, policy_version 1613475 (0.0005) [2023-12-27 03:08:53,810][105692] Updated weights for policy 0, policy_version 1613485 (0.0005) [2023-12-27 03:08:53,861][105692] Updated weights for policy 0, policy_version 1613495 (0.0005) [2023-12-27 03:08:54,084][105620] Updated weights for policy 1, policy_version 1616871 (0.0010) [2023-12-27 03:08:54,145][105620] Updated weights for policy 1, policy_version 1616881 (0.0008) [2023-12-27 03:08:54,199][105620] Updated weights for policy 1, policy_version 1616891 (0.0009) [2023-12-27 03:08:54,517][105692] Updated weights for policy 0, policy_version 1613505 (0.0009) [2023-12-27 03:08:54,576][105692] Updated weights for policy 0, policy_version 1613516 (0.0010) [2023-12-27 03:08:54,641][105692] Updated weights for policy 0, policy_version 1613526 (0.0009) [2023-12-27 03:08:54,916][105620] Updated weights for policy 1, policy_version 1616901 (0.0009) [2023-12-27 03:08:54,966][105620] Updated weights for policy 1, policy_version 1616911 (0.0009) [2023-12-27 03:08:55,017][105620] Updated weights for policy 1, policy_version 1616921 (0.0008) [2023-12-27 03:08:55,426][105692] Updated weights for policy 0, policy_version 1613536 (0.0008) [2023-12-27 03:08:55,492][105692] Updated weights for policy 0, policy_version 1613546 (0.0008) [2023-12-27 03:08:55,553][105692] Updated weights for policy 0, policy_version 1613556 (0.0009) [2023-12-27 03:08:55,775][105620] Updated weights for policy 1, policy_version 1616931 (0.0009) [2023-12-27 03:08:55,837][105620] Updated weights for policy 1, policy_version 1616941 (0.0009) [2023-12-27 03:08:55,898][105620] Updated weights for policy 1, policy_version 1616951 (0.0009) [2023-12-27 03:08:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 827129856. Throughput: 0: 9651.1, 1: 9775.6. Samples: 827135880. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-12-27 03:08:56,062][104569] Avg episode reward: [(0, '8713.947'), (1, '8807.929')] [2023-12-27 03:08:56,285][105692] Updated weights for policy 0, policy_version 1613566 (0.0010) [2023-12-27 03:08:56,345][105692] Updated weights for policy 0, policy_version 1613576 (0.0008) [2023-12-27 03:08:56,405][105692] Updated weights for policy 0, policy_version 1613586 (0.0009) [2023-12-27 03:08:56,639][105620] Updated weights for policy 1, policy_version 1616962 (0.0009) [2023-12-27 03:08:56,691][105620] Updated weights for policy 1, policy_version 1616972 (0.0005) [2023-12-27 03:08:56,748][105620] Updated weights for policy 1, policy_version 1616982 (0.0007) [2023-12-27 03:08:56,810][105620] Updated weights for policy 1, policy_version 1616992 (0.0010) [2023-12-27 03:08:57,182][105692] Updated weights for policy 0, policy_version 1613596 (0.0010) [2023-12-27 03:08:57,236][105692] Updated weights for policy 0, policy_version 1613606 (0.0010) [2023-12-27 03:08:57,295][105692] Updated weights for policy 0, policy_version 1613616 (0.0010) [2023-12-27 03:08:57,446][105620] Updated weights for policy 1, policy_version 1617002 (0.0008) [2023-12-27 03:08:57,492][105620] Updated weights for policy 1, policy_version 1617012 (0.0007) [2023-12-27 03:08:57,540][105620] Updated weights for policy 1, policy_version 1617022 (0.0008) [2023-12-27 03:08:57,966][105692] Updated weights for policy 0, policy_version 1613626 (0.0008) [2023-12-27 03:08:58,024][105692] Updated weights for policy 0, policy_version 1613636 (0.0006) [2023-12-27 03:08:58,070][105692] Updated weights for policy 0, policy_version 1613646 (0.0005) [2023-12-27 03:08:58,119][105692] Updated weights for policy 0, policy_version 1613656 (0.0005) [2023-12-27 03:08:58,206][105620] Updated weights for policy 1, policy_version 1617032 (0.0010) [2023-12-27 03:08:58,268][105620] Updated weights for policy 1, policy_version 1617042 (0.0011) [2023-12-27 03:08:58,331][105620] Updated weights for policy 1, policy_version 1617052 (0.0010) [2023-12-27 03:08:58,918][105692] Updated weights for policy 0, policy_version 1613666 (0.0009) [2023-12-27 03:08:58,985][105692] Updated weights for policy 0, policy_version 1613676 (0.0007) [2023-12-27 03:08:59,054][105692] Updated weights for policy 0, policy_version 1613686 (0.0009) [2023-12-27 03:08:59,100][105620] Updated weights for policy 1, policy_version 1617062 (0.0008) [2023-12-27 03:08:59,155][105620] Updated weights for policy 1, policy_version 1617072 (0.0007) [2023-12-27 03:08:59,216][105620] Updated weights for policy 1, policy_version 1617082 (0.0007) [2023-12-27 03:08:59,846][105620] Updated weights for policy 1, policy_version 1617092 (0.0008) [2023-12-27 03:08:59,900][105620] Updated weights for policy 1, policy_version 1617102 (0.0008) [2023-12-27 03:08:59,921][105692] Updated weights for policy 0, policy_version 1613696 (0.0008) [2023-12-27 03:08:59,963][105620] Updated weights for policy 1, policy_version 1617112 (0.0009) [2023-12-27 03:08:59,977][105692] Updated weights for policy 0, policy_version 1613706 (0.0006) [2023-12-27 03:09:00,041][105692] Updated weights for policy 0, policy_version 1613716 (0.0010) [2023-12-27 03:09:00,659][105620] Updated weights for policy 1, policy_version 1617122 (0.0007) [2023-12-27 03:09:00,708][105620] Updated weights for policy 1, policy_version 1617132 (0.0009) [2023-12-27 03:09:00,758][105620] Updated weights for policy 1, policy_version 1617142 (0.0009) [2023-12-27 03:09:00,773][105692] Updated weights for policy 0, policy_version 1613726 (0.0007) [2023-12-27 03:09:00,804][105620] Updated weights for policy 1, policy_version 1617152 (0.0007) [2023-12-27 03:09:00,819][105692] Updated weights for policy 0, policy_version 1613736 (0.0005) [2023-12-27 03:09:00,872][105692] Updated weights for policy 0, policy_version 1613746 (0.0007) [2023-12-27 03:09:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 827228160. Throughput: 0: 9637.4, 1: 9825.9. Samples: 827194456. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:09:01,063][104569] Avg episode reward: [(0, '8437.960'), (1, '9170.564')] [2023-12-27 03:09:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001613752_413179904.pth... [2023-12-27 03:09:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001617152_414048256.pth... [2023-12-27 03:09:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001612664_412901376.pth [2023-12-27 03:09:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001616000_413753344.pth [2023-12-27 03:09:01,514][105620] Updated weights for policy 1, policy_version 1617162 (0.0008) [2023-12-27 03:09:01,574][105620] Updated weights for policy 1, policy_version 1617172 (0.0008) [2023-12-27 03:09:01,641][105620] Updated weights for policy 1, policy_version 1617182 (0.0008) [2023-12-27 03:09:01,681][105692] Updated weights for policy 0, policy_version 1613756 (0.0009) [2023-12-27 03:09:01,754][105692] Updated weights for policy 0, policy_version 1613766 (0.0010) [2023-12-27 03:09:01,812][105692] Updated weights for policy 0, policy_version 1613776 (0.0009) [2023-12-27 03:09:02,314][105620] Updated weights for policy 1, policy_version 1617192 (0.0008) [2023-12-27 03:09:02,379][105620] Updated weights for policy 1, policy_version 1617202 (0.0009) [2023-12-27 03:09:02,444][105620] Updated weights for policy 1, policy_version 1617212 (0.0006) [2023-12-27 03:09:02,649][105692] Updated weights for policy 0, policy_version 1613786 (0.0009) [2023-12-27 03:09:02,716][105692] Updated weights for policy 0, policy_version 1613796 (0.0009) [2023-12-27 03:09:02,782][105692] Updated weights for policy 0, policy_version 1613806 (0.0009) [2023-12-27 03:09:02,841][105692] Updated weights for policy 0, policy_version 1613816 (0.0009) [2023-12-27 03:09:03,069][105620] Updated weights for policy 1, policy_version 1617222 (0.0008) [2023-12-27 03:09:03,132][105620] Updated weights for policy 1, policy_version 1617232 (0.0008) [2023-12-27 03:09:03,188][105620] Updated weights for policy 1, policy_version 1617242 (0.0006) [2023-12-27 03:09:03,615][105692] Updated weights for policy 0, policy_version 1613826 (0.0005) [2023-12-27 03:09:03,673][105692] Updated weights for policy 0, policy_version 1613836 (0.0005) [2023-12-27 03:09:03,713][105620] Updated weights for policy 1, policy_version 1617252 (0.0005) [2023-12-27 03:09:03,735][105692] Updated weights for policy 0, policy_version 1613846 (0.0005) [2023-12-27 03:09:03,767][105620] Updated weights for policy 1, policy_version 1617262 (0.0008) [2023-12-27 03:09:03,832][105620] Updated weights for policy 1, policy_version 1617272 (0.0009) [2023-12-27 03:09:04,366][105692] Updated weights for policy 0, policy_version 1613856 (0.0008) [2023-12-27 03:09:04,419][105692] Updated weights for policy 0, policy_version 1613866 (0.0009) [2023-12-27 03:09:04,478][105692] Updated weights for policy 0, policy_version 1613876 (0.0011) [2023-12-27 03:09:04,584][105620] Updated weights for policy 1, policy_version 1617282 (0.0009) [2023-12-27 03:09:04,636][105620] Updated weights for policy 1, policy_version 1617292 (0.0009) [2023-12-27 03:09:04,690][105620] Updated weights for policy 1, policy_version 1617303 (0.0010) [2023-12-27 03:09:05,200][105692] Updated weights for policy 0, policy_version 1613886 (0.0011) [2023-12-27 03:09:05,260][105692] Updated weights for policy 0, policy_version 1613896 (0.0011) [2023-12-27 03:09:05,322][105692] Updated weights for policy 0, policy_version 1613906 (0.0011) [2023-12-27 03:09:05,408][105620] Updated weights for policy 1, policy_version 1617313 (0.0008) [2023-12-27 03:09:05,465][105620] Updated weights for policy 1, policy_version 1617323 (0.0006) [2023-12-27 03:09:05,523][105620] Updated weights for policy 1, policy_version 1617333 (0.0005) [2023-12-27 03:09:05,584][105620] Updated weights for policy 1, policy_version 1617343 (0.0005) [2023-12-27 03:09:05,925][105692] Updated weights for policy 0, policy_version 1613916 (0.0008) [2023-12-27 03:09:05,983][105692] Updated weights for policy 0, policy_version 1613926 (0.0005) [2023-12-27 03:09:06,060][105692] Updated weights for policy 0, policy_version 1613936 (0.0006) [2023-12-27 03:09:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.3, 300 sec: 19438.7). Total num frames: 827318272. Throughput: 0: 9466.4, 1: 9861.9. Samples: 827309492. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:09:06,062][104569] Avg episode reward: [(0, '8437.755'), (1, '9174.739')] [2023-12-27 03:09:06,200][105620] Updated weights for policy 1, policy_version 1617353 (0.0007) [2023-12-27 03:09:06,253][105620] Updated weights for policy 1, policy_version 1617363 (0.0005) [2023-12-27 03:09:06,316][105620] Updated weights for policy 1, policy_version 1617373 (0.0007) [2023-12-27 03:09:06,742][105692] Updated weights for policy 0, policy_version 1613946 (0.0007) [2023-12-27 03:09:06,798][105692] Updated weights for policy 0, policy_version 1613956 (0.0009) [2023-12-27 03:09:06,849][105692] Updated weights for policy 0, policy_version 1613966 (0.0008) [2023-12-27 03:09:06,919][105692] Updated weights for policy 0, policy_version 1613976 (0.0009) [2023-12-27 03:09:07,016][105620] Updated weights for policy 1, policy_version 1617383 (0.0005) [2023-12-27 03:09:07,064][105620] Updated weights for policy 1, policy_version 1617393 (0.0008) [2023-12-27 03:09:07,112][105620] Updated weights for policy 1, policy_version 1617403 (0.0009) [2023-12-27 03:09:07,663][105692] Updated weights for policy 0, policy_version 1613986 (0.0005) [2023-12-27 03:09:07,724][105692] Updated weights for policy 0, policy_version 1613996 (0.0005) [2023-12-27 03:09:07,787][105692] Updated weights for policy 0, policy_version 1614006 (0.0007) [2023-12-27 03:09:07,794][105620] Updated weights for policy 1, policy_version 1617413 (0.0006) [2023-12-27 03:09:07,849][105620] Updated weights for policy 1, policy_version 1617423 (0.0008) [2023-12-27 03:09:07,896][105620] Updated weights for policy 1, policy_version 1617433 (0.0008) [2023-12-27 03:09:08,479][105692] Updated weights for policy 0, policy_version 1614016 (0.0009) [2023-12-27 03:09:08,528][105692] Updated weights for policy 0, policy_version 1614026 (0.0009) [2023-12-27 03:09:08,580][105692] Updated weights for policy 0, policy_version 1614036 (0.0009) [2023-12-27 03:09:08,646][105620] Updated weights for policy 1, policy_version 1617443 (0.0007) [2023-12-27 03:09:08,710][105620] Updated weights for policy 1, policy_version 1617453 (0.0008) [2023-12-27 03:09:08,766][105620] Updated weights for policy 1, policy_version 1617463 (0.0007) [2023-12-27 03:09:09,316][105692] Updated weights for policy 0, policy_version 1614046 (0.0006) [2023-12-27 03:09:09,377][105692] Updated weights for policy 0, policy_version 1614056 (0.0008) [2023-12-27 03:09:09,436][105620] Updated weights for policy 1, policy_version 1617473 (0.0005) [2023-12-27 03:09:09,442][105692] Updated weights for policy 0, policy_version 1614066 (0.0008) [2023-12-27 03:09:09,495][105620] Updated weights for policy 1, policy_version 1617483 (0.0007) [2023-12-27 03:09:09,553][105620] Updated weights for policy 1, policy_version 1617493 (0.0008) [2023-12-27 03:09:09,618][105620] Updated weights for policy 1, policy_version 1617503 (0.0008) [2023-12-27 03:09:10,134][105692] Updated weights for policy 0, policy_version 1614076 (0.0008) [2023-12-27 03:09:10,191][105692] Updated weights for policy 0, policy_version 1614086 (0.0009) [2023-12-27 03:09:10,252][105692] Updated weights for policy 0, policy_version 1614096 (0.0009) [2023-12-27 03:09:10,403][105620] Updated weights for policy 1, policy_version 1617513 (0.0008) [2023-12-27 03:09:10,472][105620] Updated weights for policy 1, policy_version 1617523 (0.0005) [2023-12-27 03:09:10,540][105620] Updated weights for policy 1, policy_version 1617533 (0.0008) [2023-12-27 03:09:11,023][105692] Updated weights for policy 0, policy_version 1614106 (0.0010) [2023-12-27 03:09:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 827416576. Throughput: 0: 9555.5, 1: 9872.8. Samples: 827428520. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:09:11,063][104569] Avg episode reward: [(0, '8805.413'), (1, '8991.778')] [2023-12-27 03:09:11,086][105692] Updated weights for policy 0, policy_version 1614116 (0.0009) [2023-12-27 03:09:11,148][105692] Updated weights for policy 0, policy_version 1614126 (0.0010) [2023-12-27 03:09:11,212][105692] Updated weights for policy 0, policy_version 1614136 (0.0008) [2023-12-27 03:09:11,247][105620] Updated weights for policy 1, policy_version 1617543 (0.0009) [2023-12-27 03:09:11,308][105620] Updated weights for policy 1, policy_version 1617553 (0.0009) [2023-12-27 03:09:11,363][105620] Updated weights for policy 1, policy_version 1617563 (0.0008) [2023-12-27 03:09:11,987][105692] Updated weights for policy 0, policy_version 1614146 (0.0010) [2023-12-27 03:09:12,047][105692] Updated weights for policy 0, policy_version 1614156 (0.0009) [2023-12-27 03:09:12,100][105692] Updated weights for policy 0, policy_version 1614166 (0.0007) [2023-12-27 03:09:12,162][105620] Updated weights for policy 1, policy_version 1617573 (0.0008) [2023-12-27 03:09:12,229][105620] Updated weights for policy 1, policy_version 1617583 (0.0009) [2023-12-27 03:09:12,298][105620] Updated weights for policy 1, policy_version 1617593 (0.0009) [2023-12-27 03:09:12,836][105692] Updated weights for policy 0, policy_version 1614176 (0.0009) [2023-12-27 03:09:12,884][105692] Updated weights for policy 0, policy_version 1614186 (0.0009) [2023-12-27 03:09:12,903][105620] Updated weights for policy 1, policy_version 1617603 (0.0007) [2023-12-27 03:09:12,938][105692] Updated weights for policy 0, policy_version 1614196 (0.0008) [2023-12-27 03:09:12,958][105620] Updated weights for policy 1, policy_version 1617613 (0.0007) [2023-12-27 03:09:13,004][105620] Updated weights for policy 1, policy_version 1617623 (0.0009) [2023-12-27 03:09:13,703][105620] Updated weights for policy 1, policy_version 1617633 (0.0009) [2023-12-27 03:09:13,744][105692] Updated weights for policy 0, policy_version 1614206 (0.0009) [2023-12-27 03:09:13,767][105620] Updated weights for policy 1, policy_version 1617643 (0.0006) [2023-12-27 03:09:13,802][105692] Updated weights for policy 0, policy_version 1614216 (0.0009) [2023-12-27 03:09:13,829][105620] Updated weights for policy 1, policy_version 1617653 (0.0006) [2023-12-27 03:09:13,856][105692] Updated weights for policy 0, policy_version 1614226 (0.0007) [2023-12-27 03:09:13,878][105620] Updated weights for policy 1, policy_version 1617663 (0.0006) [2023-12-27 03:09:14,610][105620] Updated weights for policy 1, policy_version 1617673 (0.0006) [2023-12-27 03:09:14,625][105692] Updated weights for policy 0, policy_version 1614236 (0.0007) [2023-12-27 03:09:14,675][105620] Updated weights for policy 1, policy_version 1617683 (0.0006) [2023-12-27 03:09:14,680][105692] Updated weights for policy 0, policy_version 1614246 (0.0009) [2023-12-27 03:09:14,740][105692] Updated weights for policy 0, policy_version 1614256 (0.0008) [2023-12-27 03:09:14,743][105620] Updated weights for policy 1, policy_version 1617693 (0.0008) [2023-12-27 03:09:15,401][105692] Updated weights for policy 0, policy_version 1614266 (0.0008) [2023-12-27 03:09:15,456][105692] Updated weights for policy 0, policy_version 1614276 (0.0005) [2023-12-27 03:09:15,506][105692] Updated weights for policy 0, policy_version 1614286 (0.0005) [2023-12-27 03:09:15,525][105620] Updated weights for policy 1, policy_version 1617703 (0.0009) [2023-12-27 03:09:15,551][105692] Updated weights for policy 0, policy_version 1614296 (0.0005) [2023-12-27 03:09:15,581][105620] Updated weights for policy 1, policy_version 1617713 (0.0007) [2023-12-27 03:09:15,628][105620] Updated weights for policy 1, policy_version 1617723 (0.0005) [2023-12-27 03:09:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 827514880. Throughput: 0: 9404.6, 1: 9919.4. Samples: 827485440. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:09:16,063][104569] Avg episode reward: [(0, '8991.282'), (1, '8901.269')] [2023-12-27 03:09:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001617728_414195712.pth... [2023-12-27 03:09:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001616576_413900800.pth [2023-12-27 03:09:16,095][105692] Updated weights for policy 0, policy_version 1614306 (0.0005) [2023-12-27 03:09:16,148][105692] Updated weights for policy 0, policy_version 1614316 (0.0005) [2023-12-27 03:09:16,200][105692] Updated weights for policy 0, policy_version 1614326 (0.0005) [2023-12-27 03:09:16,210][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001614328_413327360.pth... [2023-12-27 03:09:16,214][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001613208_413040640.pth [2023-12-27 03:09:16,407][105620] Updated weights for policy 1, policy_version 1617733 (0.0007) [2023-12-27 03:09:16,458][105620] Updated weights for policy 1, policy_version 1617743 (0.0009) [2023-12-27 03:09:16,512][105620] Updated weights for policy 1, policy_version 1617753 (0.0009) [2023-12-27 03:09:16,867][105692] Updated weights for policy 0, policy_version 1614336 (0.0010) [2023-12-27 03:09:16,932][105692] Updated weights for policy 0, policy_version 1614346 (0.0010) [2023-12-27 03:09:16,991][105692] Updated weights for policy 0, policy_version 1614356 (0.0010) [2023-12-27 03:09:17,282][105620] Updated weights for policy 1, policy_version 1617763 (0.0008) [2023-12-27 03:09:17,345][105620] Updated weights for policy 1, policy_version 1617773 (0.0008) [2023-12-27 03:09:17,400][105620] Updated weights for policy 1, policy_version 1617783 (0.0008) [2023-12-27 03:09:17,727][105692] Updated weights for policy 0, policy_version 1614366 (0.0010) [2023-12-27 03:09:17,782][105692] Updated weights for policy 0, policy_version 1614376 (0.0010) [2023-12-27 03:09:17,841][105692] Updated weights for policy 0, policy_version 1614386 (0.0009) [2023-12-27 03:09:18,191][105620] Updated weights for policy 1, policy_version 1617793 (0.0008) [2023-12-27 03:09:18,245][105620] Updated weights for policy 1, policy_version 1617803 (0.0009) [2023-12-27 03:09:18,306][105620] Updated weights for policy 1, policy_version 1617813 (0.0009) [2023-12-27 03:09:18,370][105620] Updated weights for policy 1, policy_version 1617823 (0.0009) [2023-12-27 03:09:18,507][105692] Updated weights for policy 0, policy_version 1614396 (0.0008) [2023-12-27 03:09:18,562][105692] Updated weights for policy 0, policy_version 1614406 (0.0006) [2023-12-27 03:09:18,620][105692] Updated weights for policy 0, policy_version 1614416 (0.0009) [2023-12-27 03:09:19,190][105620] Updated weights for policy 1, policy_version 1617833 (0.0011) [2023-12-27 03:09:19,264][105620] Updated weights for policy 1, policy_version 1617843 (0.0011) [2023-12-27 03:09:19,331][105620] Updated weights for policy 1, policy_version 1617853 (0.0011) [2023-12-27 03:09:19,371][105692] Updated weights for policy 0, policy_version 1614426 (0.0009) [2023-12-27 03:09:19,431][105692] Updated weights for policy 0, policy_version 1614436 (0.0008) [2023-12-27 03:09:19,493][105692] Updated weights for policy 0, policy_version 1614446 (0.0008) [2023-12-27 03:09:19,561][105692] Updated weights for policy 0, policy_version 1614456 (0.0008) [2023-12-27 03:09:20,088][105620] Updated weights for policy 1, policy_version 1617863 (0.0011) [2023-12-27 03:09:20,144][105620] Updated weights for policy 1, policy_version 1617873 (0.0010) [2023-12-27 03:09:20,206][105620] Updated weights for policy 1, policy_version 1617883 (0.0009) [2023-12-27 03:09:20,281][105692] Updated weights for policy 0, policy_version 1614466 (0.0009) [2023-12-27 03:09:20,339][105692] Updated weights for policy 0, policy_version 1614476 (0.0009) [2023-12-27 03:09:20,394][105692] Updated weights for policy 0, policy_version 1614486 (0.0010) [2023-12-27 03:09:20,912][105620] Updated weights for policy 1, policy_version 1617893 (0.0009) [2023-12-27 03:09:20,975][105620] Updated weights for policy 1, policy_version 1617903 (0.0010) [2023-12-27 03:09:21,036][105620] Updated weights for policy 1, policy_version 1617913 (0.0009) [2023-12-27 03:09:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.3, 300 sec: 19410.9). Total num frames: 827604992. Throughput: 0: 9485.4, 1: 9800.0. Samples: 827600272. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:09:21,062][104569] Avg episode reward: [(0, '8529.072'), (1, '8717.910')] [2023-12-27 03:09:21,115][105692] Updated weights for policy 0, policy_version 1614496 (0.0007) [2023-12-27 03:09:21,185][105692] Updated weights for policy 0, policy_version 1614506 (0.0007) [2023-12-27 03:09:21,247][105692] Updated weights for policy 0, policy_version 1614516 (0.0006) [2023-12-27 03:09:21,879][105620] Updated weights for policy 1, policy_version 1617923 (0.0008) [2023-12-27 03:09:21,939][105620] Updated weights for policy 1, policy_version 1617933 (0.0008) [2023-12-27 03:09:21,957][105692] Updated weights for policy 0, policy_version 1614526 (0.0008) [2023-12-27 03:09:22,000][105620] Updated weights for policy 1, policy_version 1617943 (0.0006) [2023-12-27 03:09:22,018][105692] Updated weights for policy 0, policy_version 1614536 (0.0009) [2023-12-27 03:09:22,081][105692] Updated weights for policy 0, policy_version 1614546 (0.0006) [2023-12-27 03:09:22,756][105692] Updated weights for policy 0, policy_version 1614556 (0.0008) [2023-12-27 03:09:22,811][105692] Updated weights for policy 0, policy_version 1614566 (0.0009) [2023-12-27 03:09:22,827][105620] Updated weights for policy 1, policy_version 1617953 (0.0007) [2023-12-27 03:09:22,870][105692] Updated weights for policy 0, policy_version 1614576 (0.0007) [2023-12-27 03:09:22,890][105620] Updated weights for policy 1, policy_version 1617963 (0.0009) [2023-12-27 03:09:22,948][105620] Updated weights for policy 1, policy_version 1617973 (0.0009) [2023-12-27 03:09:23,005][105620] Updated weights for policy 1, policy_version 1617984 (0.0009) [2023-12-27 03:09:23,522][105692] Updated weights for policy 0, policy_version 1614586 (0.0006) [2023-12-27 03:09:23,572][105692] Updated weights for policy 0, policy_version 1614596 (0.0005) [2023-12-27 03:09:23,624][105692] Updated weights for policy 0, policy_version 1614606 (0.0007) [2023-12-27 03:09:23,676][105692] Updated weights for policy 0, policy_version 1614616 (0.0009) [2023-12-27 03:09:23,832][105620] Updated weights for policy 1, policy_version 1617994 (0.0010) [2023-12-27 03:09:23,899][105620] Updated weights for policy 1, policy_version 1618004 (0.0010) [2023-12-27 03:09:23,966][105620] Updated weights for policy 1, policy_version 1618014 (0.0010) [2023-12-27 03:09:24,293][105692] Updated weights for policy 0, policy_version 1614626 (0.0011) [2023-12-27 03:09:24,353][105692] Updated weights for policy 0, policy_version 1614636 (0.0010) [2023-12-27 03:09:24,415][105692] Updated weights for policy 0, policy_version 1614646 (0.0010) [2023-12-27 03:09:24,815][105620] Updated weights for policy 1, policy_version 1618024 (0.0010) [2023-12-27 03:09:24,868][105620] Updated weights for policy 1, policy_version 1618034 (0.0010) [2023-12-27 03:09:24,922][105620] Updated weights for policy 1, policy_version 1618044 (0.0009) [2023-12-27 03:09:25,002][105692] Updated weights for policy 0, policy_version 1614656 (0.0005) [2023-12-27 03:09:25,068][105692] Updated weights for policy 0, policy_version 1614666 (0.0007) [2023-12-27 03:09:25,125][105692] Updated weights for policy 0, policy_version 1614676 (0.0009) [2023-12-27 03:09:25,744][105620] Updated weights for policy 1, policy_version 1618054 (0.0009) [2023-12-27 03:09:25,814][105692] Updated weights for policy 0, policy_version 1614686 (0.0007) [2023-12-27 03:09:25,815][105620] Updated weights for policy 1, policy_version 1618064 (0.0009) [2023-12-27 03:09:25,863][105692] Updated weights for policy 0, policy_version 1614696 (0.0005) [2023-12-27 03:09:25,879][105620] Updated weights for policy 1, policy_version 1618074 (0.0009) [2023-12-27 03:09:25,915][105692] Updated weights for policy 0, policy_version 1614706 (0.0005) [2023-12-27 03:09:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 827711488. Throughput: 0: 9608.0, 1: 9618.1. Samples: 827713904. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:09:26,062][104569] Avg episode reward: [(0, '8438.965'), (1, '8716.261')] [2023-12-27 03:09:26,471][105692] Updated weights for policy 0, policy_version 1614716 (0.0007) [2023-12-27 03:09:26,519][105692] Updated weights for policy 0, policy_version 1614726 (0.0008) [2023-12-27 03:09:26,563][105692] Updated weights for policy 0, policy_version 1614736 (0.0005) [2023-12-27 03:09:26,729][105620] Updated weights for policy 1, policy_version 1618084 (0.0010) [2023-12-27 03:09:26,796][105620] Updated weights for policy 1, policy_version 1618094 (0.0010) [2023-12-27 03:09:26,865][105620] Updated weights for policy 1, policy_version 1618104 (0.0009) [2023-12-27 03:09:27,167][105692] Updated weights for policy 0, policy_version 1614746 (0.0006) [2023-12-27 03:09:27,215][105692] Updated weights for policy 0, policy_version 1614756 (0.0010) [2023-12-27 03:09:27,259][105692] Updated weights for policy 0, policy_version 1614766 (0.0010) [2023-12-27 03:09:27,307][105692] Updated weights for policy 0, policy_version 1614776 (0.0010) [2023-12-27 03:09:27,613][105620] Updated weights for policy 1, policy_version 1618114 (0.0008) [2023-12-27 03:09:27,673][105620] Updated weights for policy 1, policy_version 1618124 (0.0006) [2023-12-27 03:09:27,739][105620] Updated weights for policy 1, policy_version 1618134 (0.0008) [2023-12-27 03:09:27,953][105692] Updated weights for policy 0, policy_version 1614786 (0.0008) [2023-12-27 03:09:28,008][105692] Updated weights for policy 0, policy_version 1614796 (0.0007) [2023-12-27 03:09:28,053][105692] Updated weights for policy 0, policy_version 1614806 (0.0005) [2023-12-27 03:09:28,514][105620] Updated weights for policy 1, policy_version 1618145 (0.0009) [2023-12-27 03:09:28,571][105620] Updated weights for policy 1, policy_version 1618155 (0.0009) [2023-12-27 03:09:28,632][105620] Updated weights for policy 1, policy_version 1618165 (0.0009) [2023-12-27 03:09:28,678][105692] Updated weights for policy 0, policy_version 1614816 (0.0005) [2023-12-27 03:09:28,689][105620] Updated weights for policy 1, policy_version 1618175 (0.0008) [2023-12-27 03:09:28,723][105692] Updated weights for policy 0, policy_version 1614826 (0.0005) [2023-12-27 03:09:28,773][105692] Updated weights for policy 0, policy_version 1614836 (0.0009) [2023-12-27 03:09:29,452][105620] Updated weights for policy 1, policy_version 1618185 (0.0006) [2023-12-27 03:09:29,503][105620] Updated weights for policy 1, policy_version 1618195 (0.0009) [2023-12-27 03:09:29,557][105692] Updated weights for policy 0, policy_version 1614846 (0.0009) [2023-12-27 03:09:29,559][105620] Updated weights for policy 1, policy_version 1618205 (0.0006) [2023-12-27 03:09:29,614][105692] Updated weights for policy 0, policy_version 1614856 (0.0010) [2023-12-27 03:09:29,668][105692] Updated weights for policy 0, policy_version 1614867 (0.0011) [2023-12-27 03:09:30,282][105620] Updated weights for policy 1, policy_version 1618215 (0.0007) [2023-12-27 03:09:30,342][105620] Updated weights for policy 1, policy_version 1618225 (0.0008) [2023-12-27 03:09:30,401][105620] Updated weights for policy 1, policy_version 1618235 (0.0008) [2023-12-27 03:09:30,435][105692] Updated weights for policy 0, policy_version 1614877 (0.0010) [2023-12-27 03:09:30,482][105692] Updated weights for policy 0, policy_version 1614887 (0.0010) [2023-12-27 03:09:30,530][105692] Updated weights for policy 0, policy_version 1614897 (0.0010) [2023-12-27 03:09:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 827801600. Throughput: 0: 9744.4, 1: 9555.6. Samples: 827774344. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:09:31,062][104569] Avg episode reward: [(0, '8533.906'), (1, '9080.467')] [2023-12-27 03:09:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001614904_413474816.pth... [2023-12-27 03:09:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001618240_414326784.pth... [2023-12-27 03:09:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001613752_413179904.pth [2023-12-27 03:09:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001617152_414048256.pth [2023-12-27 03:09:31,194][105620] Updated weights for policy 1, policy_version 1618245 (0.0008) [2023-12-27 03:09:31,262][105620] Updated weights for policy 1, policy_version 1618255 (0.0009) [2023-12-27 03:09:31,309][105692] Updated weights for policy 0, policy_version 1614907 (0.0010) [2023-12-27 03:09:31,322][105620] Updated weights for policy 1, policy_version 1618265 (0.0007) [2023-12-27 03:09:31,374][105692] Updated weights for policy 0, policy_version 1614917 (0.0009) [2023-12-27 03:09:31,431][105692] Updated weights for policy 0, policy_version 1614927 (0.0005) [2023-12-27 03:09:32,051][105620] Updated weights for policy 1, policy_version 1618275 (0.0007) [2023-12-27 03:09:32,094][105692] Updated weights for policy 0, policy_version 1614937 (0.0007) [2023-12-27 03:09:32,102][105620] Updated weights for policy 1, policy_version 1618285 (0.0006) [2023-12-27 03:09:32,150][105692] Updated weights for policy 0, policy_version 1614947 (0.0008) [2023-12-27 03:09:32,156][105620] Updated weights for policy 1, policy_version 1618295 (0.0006) [2023-12-27 03:09:32,209][105692] Updated weights for policy 0, policy_version 1614957 (0.0006) [2023-12-27 03:09:32,270][105692] Updated weights for policy 0, policy_version 1614967 (0.0007) [2023-12-27 03:09:32,854][105620] Updated weights for policy 1, policy_version 1618305 (0.0008) [2023-12-27 03:09:32,889][105692] Updated weights for policy 0, policy_version 1614977 (0.0008) [2023-12-27 03:09:32,899][105620] Updated weights for policy 1, policy_version 1618315 (0.0008) [2023-12-27 03:09:32,940][105692] Updated weights for policy 0, policy_version 1614987 (0.0005) [2023-12-27 03:09:32,952][105620] Updated weights for policy 1, policy_version 1618325 (0.0009) [2023-12-27 03:09:32,998][105620] Updated weights for policy 1, policy_version 1618335 (0.0009) [2023-12-27 03:09:33,004][105692] Updated weights for policy 0, policy_version 1614997 (0.0007) [2023-12-27 03:09:33,697][105692] Updated weights for policy 0, policy_version 1615007 (0.0008) [2023-12-27 03:09:33,759][105692] Updated weights for policy 0, policy_version 1615017 (0.0007) [2023-12-27 03:09:33,788][105620] Updated weights for policy 1, policy_version 1618345 (0.0007) [2023-12-27 03:09:33,814][105692] Updated weights for policy 0, policy_version 1615027 (0.0007) [2023-12-27 03:09:33,843][105620] Updated weights for policy 1, policy_version 1618355 (0.0007) [2023-12-27 03:09:33,904][105620] Updated weights for policy 1, policy_version 1618365 (0.0009) [2023-12-27 03:09:34,590][105620] Updated weights for policy 1, policy_version 1618375 (0.0008) [2023-12-27 03:09:34,636][105692] Updated weights for policy 0, policy_version 1615037 (0.0007) [2023-12-27 03:09:34,652][105620] Updated weights for policy 1, policy_version 1618385 (0.0008) [2023-12-27 03:09:34,683][105692] Updated weights for policy 0, policy_version 1615047 (0.0008) [2023-12-27 03:09:34,709][105620] Updated weights for policy 1, policy_version 1618395 (0.0008) [2023-12-27 03:09:34,732][105692] Updated weights for policy 0, policy_version 1615057 (0.0006) [2023-12-27 03:09:35,449][105692] Updated weights for policy 0, policy_version 1615067 (0.0009) [2023-12-27 03:09:35,468][105620] Updated weights for policy 1, policy_version 1618405 (0.0007) [2023-12-27 03:09:35,498][105692] Updated weights for policy 0, policy_version 1615077 (0.0010) [2023-12-27 03:09:35,516][105620] Updated weights for policy 1, policy_version 1618415 (0.0006) [2023-12-27 03:09:35,552][105692] Updated weights for policy 0, policy_version 1615087 (0.0010) [2023-12-27 03:09:35,570][105620] Updated weights for policy 1, policy_version 1618425 (0.0006) [2023-12-27 03:09:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 827899904. Throughput: 0: 9808.8, 1: 9471.4. Samples: 827889820. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:09:36,062][104569] Avg episode reward: [(0, '8441.446'), (1, '9263.521')] [2023-12-27 03:09:36,239][105692] Updated weights for policy 0, policy_version 1615097 (0.0011) [2023-12-27 03:09:36,288][105692] Updated weights for policy 0, policy_version 1615107 (0.0010) [2023-12-27 03:09:36,337][105692] Updated weights for policy 0, policy_version 1615117 (0.0010) [2023-12-27 03:09:36,363][105620] Updated weights for policy 1, policy_version 1618435 (0.0006) [2023-12-27 03:09:36,390][105692] Updated weights for policy 0, policy_version 1615127 (0.0011) [2023-12-27 03:09:36,422][105620] Updated weights for policy 1, policy_version 1618445 (0.0007) [2023-12-27 03:09:36,471][105620] Updated weights for policy 1, policy_version 1618455 (0.0007) [2023-12-27 03:09:37,174][105692] Updated weights for policy 0, policy_version 1615137 (0.0011) [2023-12-27 03:09:37,195][105620] Updated weights for policy 1, policy_version 1618465 (0.0008) [2023-12-27 03:09:37,236][105692] Updated weights for policy 0, policy_version 1615147 (0.0009) [2023-12-27 03:09:37,251][105620] Updated weights for policy 1, policy_version 1618475 (0.0008) [2023-12-27 03:09:37,286][105692] Updated weights for policy 0, policy_version 1615157 (0.0005) [2023-12-27 03:09:37,299][105620] Updated weights for policy 1, policy_version 1618485 (0.0008) [2023-12-27 03:09:37,351][105620] Updated weights for policy 1, policy_version 1618495 (0.0010) [2023-12-27 03:09:37,905][105692] Updated weights for policy 0, policy_version 1615167 (0.0009) [2023-12-27 03:09:37,960][105692] Updated weights for policy 0, policy_version 1615177 (0.0007) [2023-12-27 03:09:38,016][105692] Updated weights for policy 0, policy_version 1615187 (0.0009) [2023-12-27 03:09:38,045][105620] Updated weights for policy 1, policy_version 1618505 (0.0008) [2023-12-27 03:09:38,114][105620] Updated weights for policy 1, policy_version 1618515 (0.0008) [2023-12-27 03:09:38,176][105620] Updated weights for policy 1, policy_version 1618525 (0.0008) [2023-12-27 03:09:38,741][105692] Updated weights for policy 0, policy_version 1615197 (0.0008) [2023-12-27 03:09:38,788][105692] Updated weights for policy 0, policy_version 1615207 (0.0007) [2023-12-27 03:09:38,827][105620] Updated weights for policy 1, policy_version 1618535 (0.0009) [2023-12-27 03:09:38,839][105692] Updated weights for policy 0, policy_version 1615217 (0.0006) [2023-12-27 03:09:38,894][105620] Updated weights for policy 1, policy_version 1618545 (0.0006) [2023-12-27 03:09:38,956][105620] Updated weights for policy 1, policy_version 1618555 (0.0008) [2023-12-27 03:09:39,535][105692] Updated weights for policy 0, policy_version 1615227 (0.0009) [2023-12-27 03:09:39,585][105692] Updated weights for policy 0, policy_version 1615237 (0.0008) [2023-12-27 03:09:39,642][105620] Updated weights for policy 1, policy_version 1618565 (0.0008) [2023-12-27 03:09:39,643][105692] Updated weights for policy 0, policy_version 1615247 (0.0006) [2023-12-27 03:09:39,703][105620] Updated weights for policy 1, policy_version 1618575 (0.0008) [2023-12-27 03:09:39,761][105620] Updated weights for policy 1, policy_version 1618585 (0.0010) [2023-12-27 03:09:40,349][105692] Updated weights for policy 0, policy_version 1615257 (0.0006) [2023-12-27 03:09:40,388][105620] Updated weights for policy 1, policy_version 1618595 (0.0006) [2023-12-27 03:09:40,401][105692] Updated weights for policy 0, policy_version 1615267 (0.0009) [2023-12-27 03:09:40,440][105620] Updated weights for policy 1, policy_version 1618605 (0.0005) [2023-12-27 03:09:40,459][105692] Updated weights for policy 0, policy_version 1615277 (0.0009) [2023-12-27 03:09:40,497][105620] Updated weights for policy 1, policy_version 1618615 (0.0005) [2023-12-27 03:09:40,509][105692] Updated weights for policy 0, policy_version 1615287 (0.0009) [2023-12-27 03:09:41,059][105620] Updated weights for policy 1, policy_version 1618625 (0.0006) [2023-12-27 03:09:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 827998208. Throughput: 0: 9844.4, 1: 9574.6. Samples: 828009732. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:09:41,062][104569] Avg episode reward: [(0, '8714.176'), (1, '8993.043')] [2023-12-27 03:09:41,132][105620] Updated weights for policy 1, policy_version 1618635 (0.0009) [2023-12-27 03:09:41,197][105620] Updated weights for policy 1, policy_version 1618645 (0.0011) [2023-12-27 03:09:41,260][105620] Updated weights for policy 1, policy_version 1618655 (0.0011) [2023-12-27 03:09:41,335][105692] Updated weights for policy 0, policy_version 1615297 (0.0009) [2023-12-27 03:09:41,405][105692] Updated weights for policy 0, policy_version 1615307 (0.0008) [2023-12-27 03:09:41,460][105692] Updated weights for policy 0, policy_version 1615317 (0.0008) [2023-12-27 03:09:42,036][105620] Updated weights for policy 1, policy_version 1618665 (0.0011) [2023-12-27 03:09:42,095][105620] Updated weights for policy 1, policy_version 1618675 (0.0010) [2023-12-27 03:09:42,150][105620] Updated weights for policy 1, policy_version 1618685 (0.0010) [2023-12-27 03:09:42,271][105692] Updated weights for policy 0, policy_version 1615327 (0.0008) [2023-12-27 03:09:42,329][105692] Updated weights for policy 0, policy_version 1615337 (0.0008) [2023-12-27 03:09:42,391][105692] Updated weights for policy 0, policy_version 1615347 (0.0008) [2023-12-27 03:09:42,910][105620] Updated weights for policy 1, policy_version 1618695 (0.0010) [2023-12-27 03:09:42,968][105620] Updated weights for policy 1, policy_version 1618705 (0.0010) [2023-12-27 03:09:43,029][105620] Updated weights for policy 1, policy_version 1618715 (0.0010) [2023-12-27 03:09:43,042][105692] Updated weights for policy 0, policy_version 1615357 (0.0006) [2023-12-27 03:09:43,102][105692] Updated weights for policy 0, policy_version 1615367 (0.0008) [2023-12-27 03:09:43,154][105692] Updated weights for policy 0, policy_version 1615377 (0.0008) [2023-12-27 03:09:43,743][105620] Updated weights for policy 1, policy_version 1618725 (0.0010) [2023-12-27 03:09:43,794][105620] Updated weights for policy 1, policy_version 1618735 (0.0009) [2023-12-27 03:09:43,848][105620] Updated weights for policy 1, policy_version 1618745 (0.0009) [2023-12-27 03:09:43,910][105692] Updated weights for policy 0, policy_version 1615387 (0.0008) [2023-12-27 03:09:43,970][105692] Updated weights for policy 0, policy_version 1615397 (0.0008) [2023-12-27 03:09:44,034][105692] Updated weights for policy 0, policy_version 1615407 (0.0008) [2023-12-27 03:09:44,629][105620] Updated weights for policy 1, policy_version 1618755 (0.0009) [2023-12-27 03:09:44,692][105620] Updated weights for policy 1, policy_version 1618765 (0.0010) [2023-12-27 03:09:44,751][105620] Updated weights for policy 1, policy_version 1618775 (0.0010) [2023-12-27 03:09:44,769][105586] KL-divergence is very high: 155.1006 [2023-12-27 03:09:44,804][105692] Updated weights for policy 0, policy_version 1615417 (0.0008) [2023-12-27 03:09:44,854][105692] Updated weights for policy 0, policy_version 1615427 (0.0008) [2023-12-27 03:09:44,911][105692] Updated weights for policy 0, policy_version 1615437 (0.0008) [2023-12-27 03:09:44,971][105692] Updated weights for policy 0, policy_version 1615447 (0.0009) [2023-12-27 03:09:45,470][105620] Updated weights for policy 1, policy_version 1618785 (0.0010) [2023-12-27 03:09:45,528][105620] Updated weights for policy 1, policy_version 1618795 (0.0009) [2023-12-27 03:09:45,583][105620] Updated weights for policy 1, policy_version 1618805 (0.0010) [2023-12-27 03:09:45,652][105620] Updated weights for policy 1, policy_version 1618815 (0.0011) [2023-12-27 03:09:45,699][105692] Updated weights for policy 0, policy_version 1615457 (0.0006) [2023-12-27 03:09:45,763][105692] Updated weights for policy 0, policy_version 1615467 (0.0005) [2023-12-27 03:09:45,828][105692] Updated weights for policy 0, policy_version 1615477 (0.0005) [2023-12-27 03:09:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 828096512. Throughput: 0: 9813.7, 1: 9529.9. Samples: 828064916. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:09:46,062][104569] Avg episode reward: [(0, '8987.396'), (1, '8726.812')] [2023-12-27 03:09:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001618816_414474240.pth... [2023-12-27 03:09:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001615480_413622272.pth... [2023-12-27 03:09:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001614328_413327360.pth [2023-12-27 03:09:46,081][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001617728_414195712.pth [2023-12-27 03:09:46,316][105620] Updated weights for policy 1, policy_version 1618825 (0.0006) [2023-12-27 03:09:46,370][105620] Updated weights for policy 1, policy_version 1618835 (0.0005) [2023-12-27 03:09:46,386][105692] Updated weights for policy 0, policy_version 1615487 (0.0005) [2023-12-27 03:09:46,416][105620] Updated weights for policy 1, policy_version 1618845 (0.0005) [2023-12-27 03:09:46,440][105692] Updated weights for policy 0, policy_version 1615497 (0.0010) [2023-12-27 03:09:46,492][105692] Updated weights for policy 0, policy_version 1615507 (0.0010) [2023-12-27 03:09:47,003][105620] Updated weights for policy 1, policy_version 1618855 (0.0005) [2023-12-27 03:09:47,051][105620] Updated weights for policy 1, policy_version 1618865 (0.0006) [2023-12-27 03:09:47,069][105692] Updated weights for policy 0, policy_version 1615517 (0.0008) [2023-12-27 03:09:47,112][105620] Updated weights for policy 1, policy_version 1618875 (0.0010) [2023-12-27 03:09:47,124][105692] Updated weights for policy 0, policy_version 1615527 (0.0005) [2023-12-27 03:09:47,191][105692] Updated weights for policy 0, policy_version 1615537 (0.0010) [2023-12-27 03:09:47,677][105620] Updated weights for policy 1, policy_version 1618885 (0.0005) [2023-12-27 03:09:47,735][105620] Updated weights for policy 1, policy_version 1618895 (0.0005) [2023-12-27 03:09:47,792][105620] Updated weights for policy 1, policy_version 1618905 (0.0005) [2023-12-27 03:09:47,886][105692] Updated weights for policy 0, policy_version 1615547 (0.0010) [2023-12-27 03:09:47,952][105692] Updated weights for policy 0, policy_version 1615557 (0.0009) [2023-12-27 03:09:48,008][105692] Updated weights for policy 0, policy_version 1615567 (0.0006) [2023-12-27 03:09:48,304][105620] Updated weights for policy 1, policy_version 1618915 (0.0006) [2023-12-27 03:09:48,368][105620] Updated weights for policy 1, policy_version 1618925 (0.0007) [2023-12-27 03:09:48,432][105620] Updated weights for policy 1, policy_version 1618935 (0.0007) [2023-12-27 03:09:48,712][105692] Updated weights for policy 0, policy_version 1615577 (0.0009) [2023-12-27 03:09:48,781][105692] Updated weights for policy 0, policy_version 1615587 (0.0011) [2023-12-27 03:09:48,836][105692] Updated weights for policy 0, policy_version 1615597 (0.0010) [2023-12-27 03:09:48,896][105692] Updated weights for policy 0, policy_version 1615607 (0.0011) [2023-12-27 03:09:49,041][105620] Updated weights for policy 1, policy_version 1618945 (0.0007) [2023-12-27 03:09:49,104][105620] Updated weights for policy 1, policy_version 1618955 (0.0008) [2023-12-27 03:09:49,153][105620] Updated weights for policy 1, policy_version 1618965 (0.0010) [2023-12-27 03:09:49,209][105620] Updated weights for policy 1, policy_version 1618975 (0.0010) [2023-12-27 03:09:49,635][105692] Updated weights for policy 0, policy_version 1615617 (0.0010) [2023-12-27 03:09:49,689][105692] Updated weights for policy 0, policy_version 1615627 (0.0011) [2023-12-27 03:09:49,738][105692] Updated weights for policy 0, policy_version 1615637 (0.0010) [2023-12-27 03:09:49,972][105620] Updated weights for policy 1, policy_version 1618985 (0.0008) [2023-12-27 03:09:50,032][105620] Updated weights for policy 1, policy_version 1618995 (0.0008) [2023-12-27 03:09:50,097][105620] Updated weights for policy 1, policy_version 1619005 (0.0008) [2023-12-27 03:09:50,477][105692] Updated weights for policy 0, policy_version 1615647 (0.0009) [2023-12-27 03:09:50,533][105692] Updated weights for policy 0, policy_version 1615657 (0.0008) [2023-12-27 03:09:50,601][105692] Updated weights for policy 0, policy_version 1615667 (0.0009) [2023-12-27 03:09:50,802][105620] Updated weights for policy 1, policy_version 1619015 (0.0010) [2023-12-27 03:09:50,854][105620] Updated weights for policy 1, policy_version 1619025 (0.0010) [2023-12-27 03:09:50,903][105620] Updated weights for policy 1, policy_version 1619035 (0.0010) [2023-12-27 03:09:51,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.2, 300 sec: 19438.7). Total num frames: 828203008. Throughput: 0: 9969.9, 1: 9585.2. Samples: 828189476. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:09:51,062][104569] Avg episode reward: [(0, '8535.258'), (1, '8816.068')] [2023-12-27 03:09:51,391][105692] Updated weights for policy 0, policy_version 1615677 (0.0009) [2023-12-27 03:09:51,446][105692] Updated weights for policy 0, policy_version 1615687 (0.0009) [2023-12-27 03:09:51,496][105692] Updated weights for policy 0, policy_version 1615697 (0.0008) [2023-12-27 03:09:51,650][105620] Updated weights for policy 1, policy_version 1619045 (0.0010) [2023-12-27 03:09:51,704][105620] Updated weights for policy 1, policy_version 1619055 (0.0008) [2023-12-27 03:09:51,769][105620] Updated weights for policy 1, policy_version 1619065 (0.0007) [2023-12-27 03:09:52,371][105692] Updated weights for policy 0, policy_version 1615707 (0.0010) [2023-12-27 03:09:52,418][105620] Updated weights for policy 1, policy_version 1619075 (0.0009) [2023-12-27 03:09:52,427][105692] Updated weights for policy 0, policy_version 1615717 (0.0008) [2023-12-27 03:09:52,471][105620] Updated weights for policy 1, policy_version 1619085 (0.0007) [2023-12-27 03:09:52,473][105692] Updated weights for policy 0, policy_version 1615727 (0.0005) [2023-12-27 03:09:52,526][105620] Updated weights for policy 1, policy_version 1619095 (0.0007) [2023-12-27 03:09:53,211][105692] Updated weights for policy 0, policy_version 1615737 (0.0006) [2023-12-27 03:09:53,230][105620] Updated weights for policy 1, policy_version 1619105 (0.0009) [2023-12-27 03:09:53,274][105692] Updated weights for policy 0, policy_version 1615747 (0.0006) [2023-12-27 03:09:53,285][105620] Updated weights for policy 1, policy_version 1619115 (0.0005) [2023-12-27 03:09:53,337][105692] Updated weights for policy 0, policy_version 1615757 (0.0007) [2023-12-27 03:09:53,341][105620] Updated weights for policy 1, policy_version 1619125 (0.0005) [2023-12-27 03:09:53,390][105692] Updated weights for policy 0, policy_version 1615767 (0.0005) [2023-12-27 03:09:53,396][105620] Updated weights for policy 1, policy_version 1619135 (0.0010) [2023-12-27 03:09:53,922][105620] Updated weights for policy 1, policy_version 1619145 (0.0010) [2023-12-27 03:09:53,925][105692] Updated weights for policy 0, policy_version 1615777 (0.0007) [2023-12-27 03:09:53,982][105620] Updated weights for policy 1, policy_version 1619155 (0.0011) [2023-12-27 03:09:53,984][105692] Updated weights for policy 0, policy_version 1615787 (0.0006) [2023-12-27 03:09:54,034][105620] Updated weights for policy 1, policy_version 1619165 (0.0011) [2023-12-27 03:09:54,044][105692] Updated weights for policy 0, policy_version 1615797 (0.0005) [2023-12-27 03:09:54,758][105620] Updated weights for policy 1, policy_version 1619175 (0.0007) [2023-12-27 03:09:54,794][105692] Updated weights for policy 0, policy_version 1615807 (0.0009) [2023-12-27 03:09:54,814][105620] Updated weights for policy 1, policy_version 1619185 (0.0005) [2023-12-27 03:09:54,855][105692] Updated weights for policy 0, policy_version 1615817 (0.0009) [2023-12-27 03:09:54,874][105620] Updated weights for policy 1, policy_version 1619195 (0.0005) [2023-12-27 03:09:54,919][105692] Updated weights for policy 0, policy_version 1615827 (0.0008) [2023-12-27 03:09:55,456][105620] Updated weights for policy 1, policy_version 1619205 (0.0008) [2023-12-27 03:09:55,508][105620] Updated weights for policy 1, policy_version 1619215 (0.0011) [2023-12-27 03:09:55,568][105692] Updated weights for policy 0, policy_version 1615837 (0.0008) [2023-12-27 03:09:55,571][105620] Updated weights for policy 1, policy_version 1619225 (0.0008) [2023-12-27 03:09:55,628][105692] Updated weights for policy 0, policy_version 1615847 (0.0009) [2023-12-27 03:09:55,683][105692] Updated weights for policy 0, policy_version 1615857 (0.0006) [2023-12-27 03:09:56,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 828301312. Throughput: 0: 9936.1, 1: 9623.7. Samples: 828308712. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:09:56,063][104569] Avg episode reward: [(0, '8264.632'), (1, '9172.902')] [2023-12-27 03:09:56,363][105620] Updated weights for policy 1, policy_version 1619235 (0.0009) [2023-12-27 03:09:56,390][105692] Updated weights for policy 0, policy_version 1615867 (0.0009) [2023-12-27 03:09:56,420][105620] Updated weights for policy 1, policy_version 1619245 (0.0008) [2023-12-27 03:09:56,448][105692] Updated weights for policy 0, policy_version 1615877 (0.0006) [2023-12-27 03:09:56,476][105620] Updated weights for policy 1, policy_version 1619255 (0.0009) [2023-12-27 03:09:56,505][105692] Updated weights for policy 0, policy_version 1615887 (0.0007) [2023-12-27 03:09:57,187][105692] Updated weights for policy 0, policy_version 1615897 (0.0010) [2023-12-27 03:09:57,211][105620] Updated weights for policy 1, policy_version 1619265 (0.0006) [2023-12-27 03:09:57,238][105692] Updated weights for policy 0, policy_version 1615907 (0.0010) [2023-12-27 03:09:57,264][105620] Updated weights for policy 1, policy_version 1619275 (0.0005) [2023-12-27 03:09:57,292][105692] Updated weights for policy 0, policy_version 1615917 (0.0010) [2023-12-27 03:09:57,321][105620] Updated weights for policy 1, policy_version 1619285 (0.0008) [2023-12-27 03:09:57,351][105692] Updated weights for policy 0, policy_version 1615927 (0.0010) [2023-12-27 03:09:57,386][105620] Updated weights for policy 1, policy_version 1619295 (0.0006) [2023-12-27 03:09:57,983][105620] Updated weights for policy 1, policy_version 1619305 (0.0009) [2023-12-27 03:09:58,045][105620] Updated weights for policy 1, policy_version 1619315 (0.0008) [2023-12-27 03:09:58,066][105692] Updated weights for policy 0, policy_version 1615937 (0.0007) [2023-12-27 03:09:58,100][105620] Updated weights for policy 1, policy_version 1619325 (0.0008) [2023-12-27 03:09:58,122][105692] Updated weights for policy 0, policy_version 1615947 (0.0007) [2023-12-27 03:09:58,178][105692] Updated weights for policy 0, policy_version 1615957 (0.0009) [2023-12-27 03:09:58,824][105620] Updated weights for policy 1, policy_version 1619335 (0.0008) [2023-12-27 03:09:58,896][105620] Updated weights for policy 1, policy_version 1619345 (0.0009) [2023-12-27 03:09:58,967][105620] Updated weights for policy 1, policy_version 1619355 (0.0009) [2023-12-27 03:09:59,033][105692] Updated weights for policy 0, policy_version 1615967 (0.0008) [2023-12-27 03:09:59,094][105692] Updated weights for policy 0, policy_version 1615977 (0.0009) [2023-12-27 03:09:59,154][105692] Updated weights for policy 0, policy_version 1615987 (0.0009) [2023-12-27 03:09:59,751][105620] Updated weights for policy 1, policy_version 1619365 (0.0009) [2023-12-27 03:09:59,820][105620] Updated weights for policy 1, policy_version 1619375 (0.0010) [2023-12-27 03:09:59,890][105620] Updated weights for policy 1, policy_version 1619385 (0.0010) [2023-12-27 03:09:59,991][105692] Updated weights for policy 0, policy_version 1615997 (0.0009) [2023-12-27 03:10:00,048][105692] Updated weights for policy 0, policy_version 1616007 (0.0008) [2023-12-27 03:10:00,108][105692] Updated weights for policy 0, policy_version 1616017 (0.0008) [2023-12-27 03:10:00,610][105620] Updated weights for policy 1, policy_version 1619395 (0.0009) [2023-12-27 03:10:00,670][105620] Updated weights for policy 1, policy_version 1619405 (0.0010) [2023-12-27 03:10:00,728][105620] Updated weights for policy 1, policy_version 1619415 (0.0010) [2023-12-27 03:10:00,768][105692] Updated weights for policy 0, policy_version 1616027 (0.0009) [2023-12-27 03:10:00,820][105692] Updated weights for policy 0, policy_version 1616037 (0.0010) [2023-12-27 03:10:00,870][105692] Updated weights for policy 0, policy_version 1616047 (0.0010) [2023-12-27 03:10:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 828399616. Throughput: 0: 9961.8, 1: 9625.5. Samples: 828366864. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:10:01,062][104569] Avg episode reward: [(0, '8351.686'), (1, '9174.743')] [2023-12-27 03:10:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001616056_413769728.pth... [2023-12-27 03:10:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001619424_414629888.pth... [2023-12-27 03:10:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001618240_414326784.pth [2023-12-27 03:10:01,084][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001614904_413474816.pth [2023-12-27 03:10:01,447][105620] Updated weights for policy 1, policy_version 1619425 (0.0010) [2023-12-27 03:10:01,508][105620] Updated weights for policy 1, policy_version 1619435 (0.0010) [2023-12-27 03:10:01,561][105692] Updated weights for policy 0, policy_version 1616057 (0.0010) [2023-12-27 03:10:01,567][105620] Updated weights for policy 1, policy_version 1619445 (0.0010) [2023-12-27 03:10:01,621][105692] Updated weights for policy 0, policy_version 1616067 (0.0008) [2023-12-27 03:10:01,631][105620] Updated weights for policy 1, policy_version 1619455 (0.0009) [2023-12-27 03:10:01,684][105692] Updated weights for policy 0, policy_version 1616077 (0.0011) [2023-12-27 03:10:01,741][105692] Updated weights for policy 0, policy_version 1616087 (0.0009) [2023-12-27 03:10:02,316][105620] Updated weights for policy 1, policy_version 1619465 (0.0009) [2023-12-27 03:10:02,377][105620] Updated weights for policy 1, policy_version 1619475 (0.0009) [2023-12-27 03:10:02,431][105620] Updated weights for policy 1, policy_version 1619485 (0.0008) [2023-12-27 03:10:02,469][105692] Updated weights for policy 0, policy_version 1616097 (0.0008) [2023-12-27 03:10:02,524][105692] Updated weights for policy 0, policy_version 1616107 (0.0005) [2023-12-27 03:10:02,573][105692] Updated weights for policy 0, policy_version 1616117 (0.0007) [2023-12-27 03:10:03,215][105620] Updated weights for policy 1, policy_version 1619495 (0.0009) [2023-12-27 03:10:03,255][105692] Updated weights for policy 0, policy_version 1616127 (0.0007) [2023-12-27 03:10:03,269][105620] Updated weights for policy 1, policy_version 1619505 (0.0008) [2023-12-27 03:10:03,305][105692] Updated weights for policy 0, policy_version 1616137 (0.0007) [2023-12-27 03:10:03,331][105620] Updated weights for policy 1, policy_version 1619515 (0.0008) [2023-12-27 03:10:03,356][105692] Updated weights for policy 0, policy_version 1616147 (0.0006) [2023-12-27 03:10:04,049][105692] Updated weights for policy 0, policy_version 1616157 (0.0009) [2023-12-27 03:10:04,097][105620] Updated weights for policy 1, policy_version 1619525 (0.0008) [2023-12-27 03:10:04,115][105692] Updated weights for policy 0, policy_version 1616167 (0.0008) [2023-12-27 03:10:04,150][105620] Updated weights for policy 1, policy_version 1619535 (0.0007) [2023-12-27 03:10:04,179][105692] Updated weights for policy 0, policy_version 1616177 (0.0008) [2023-12-27 03:10:04,209][105620] Updated weights for policy 1, policy_version 1619545 (0.0009) [2023-12-27 03:10:04,769][105692] Updated weights for policy 0, policy_version 1616187 (0.0005) [2023-12-27 03:10:04,830][105692] Updated weights for policy 0, policy_version 1616197 (0.0006) [2023-12-27 03:10:04,888][105692] Updated weights for policy 0, policy_version 1616207 (0.0009) [2023-12-27 03:10:05,047][105620] Updated weights for policy 1, policy_version 1619555 (0.0010) [2023-12-27 03:10:05,112][105620] Updated weights for policy 1, policy_version 1619565 (0.0008) [2023-12-27 03:10:05,166][105620] Updated weights for policy 1, policy_version 1619575 (0.0005) [2023-12-27 03:10:05,218][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000001 [2023-12-27 03:10:05,559][105692] Updated weights for policy 0, policy_version 1616217 (0.0008) [2023-12-27 03:10:05,618][105692] Updated weights for policy 0, policy_version 1616227 (0.0005) [2023-12-27 03:10:05,677][105692] Updated weights for policy 0, policy_version 1616237 (0.0005) [2023-12-27 03:10:05,743][105692] Updated weights for policy 0, policy_version 1616247 (0.0005) [2023-12-27 03:10:05,833][105620] Updated weights for policy 1, policy_version 1619585 (0.0009) [2023-12-27 03:10:05,880][105620] Updated weights for policy 1, policy_version 1619595 (0.0010) [2023-12-27 03:10:05,937][105620] Updated weights for policy 1, policy_version 1619605 (0.0009) [2023-12-27 03:10:05,984][105620] Updated weights for policy 1, policy_version 1619615 (0.0008) [2023-12-27 03:10:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 828497920. Throughput: 0: 9945.0, 1: 9650.5. Samples: 828482072. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:10:06,063][104569] Avg episode reward: [(0, '8438.408'), (1, '9083.465')] [2023-12-27 03:10:06,336][105692] Updated weights for policy 0, policy_version 1616257 (0.0008) [2023-12-27 03:10:06,392][105692] Updated weights for policy 0, policy_version 1616267 (0.0006) [2023-12-27 03:10:06,441][105692] Updated weights for policy 0, policy_version 1616277 (0.0006) [2023-12-27 03:10:06,816][105620] Updated weights for policy 1, policy_version 1619625 (0.0008) [2023-12-27 03:10:06,868][105620] Updated weights for policy 1, policy_version 1619635 (0.0008) [2023-12-27 03:10:06,921][105620] Updated weights for policy 1, policy_version 1619645 (0.0008) [2023-12-27 03:10:07,114][105692] Updated weights for policy 0, policy_version 1616287 (0.0009) [2023-12-27 03:10:07,180][105692] Updated weights for policy 0, policy_version 1616297 (0.0011) [2023-12-27 03:10:07,245][105692] Updated weights for policy 0, policy_version 1616307 (0.0011) [2023-12-27 03:10:07,700][105620] Updated weights for policy 1, policy_version 1619655 (0.0009) [2023-12-27 03:10:07,752][105620] Updated weights for policy 1, policy_version 1619665 (0.0009) [2023-12-27 03:10:07,803][105620] Updated weights for policy 1, policy_version 1619676 (0.0009) [2023-12-27 03:10:07,848][105692] Updated weights for policy 0, policy_version 1616317 (0.0008) [2023-12-27 03:10:07,901][105692] Updated weights for policy 0, policy_version 1616327 (0.0008) [2023-12-27 03:10:07,949][105692] Updated weights for policy 0, policy_version 1616337 (0.0010) [2023-12-27 03:10:08,538][105692] Updated weights for policy 0, policy_version 1616347 (0.0008) [2023-12-27 03:10:08,590][105692] Updated weights for policy 0, policy_version 1616357 (0.0009) [2023-12-27 03:10:08,641][105620] Updated weights for policy 1, policy_version 1619686 (0.0010) [2023-12-27 03:10:08,652][105692] Updated weights for policy 0, policy_version 1616367 (0.0009) [2023-12-27 03:10:08,699][105620] Updated weights for policy 1, policy_version 1619696 (0.0010) [2023-12-27 03:10:08,758][105620] Updated weights for policy 1, policy_version 1619706 (0.0010) [2023-12-27 03:10:09,412][105692] Updated weights for policy 0, policy_version 1616377 (0.0009) [2023-12-27 03:10:09,482][105692] Updated weights for policy 0, policy_version 1616387 (0.0011) [2023-12-27 03:10:09,530][105620] Updated weights for policy 1, policy_version 1619716 (0.0010) [2023-12-27 03:10:09,542][105692] Updated weights for policy 0, policy_version 1616397 (0.0011) [2023-12-27 03:10:09,592][105620] Updated weights for policy 1, policy_version 1619726 (0.0007) [2023-12-27 03:10:09,605][105692] Updated weights for policy 0, policy_version 1616407 (0.0011) [2023-12-27 03:10:09,653][105620] Updated weights for policy 1, policy_version 1619736 (0.0007) [2023-12-27 03:10:10,365][105692] Updated weights for policy 0, policy_version 1616417 (0.0010) [2023-12-27 03:10:10,389][105620] Updated weights for policy 1, policy_version 1619746 (0.0009) [2023-12-27 03:10:10,424][105692] Updated weights for policy 0, policy_version 1616427 (0.0010) [2023-12-27 03:10:10,445][105620] Updated weights for policy 1, policy_version 1619756 (0.0011) [2023-12-27 03:10:10,484][105692] Updated weights for policy 0, policy_version 1616437 (0.0010) [2023-12-27 03:10:10,499][105620] Updated weights for policy 1, policy_version 1619766 (0.0008) [2023-12-27 03:10:10,556][105620] Updated weights for policy 1, policy_version 1619776 (0.0006) [2023-12-27 03:10:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 828588032. Throughput: 0: 9935.5, 1: 9738.4. Samples: 828599228. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:10:11,062][104569] Avg episode reward: [(0, '8438.895'), (1, '9082.888')] [2023-12-27 03:10:11,247][105692] Updated weights for policy 0, policy_version 1616447 (0.0009) [2023-12-27 03:10:11,290][105620] Updated weights for policy 1, policy_version 1619786 (0.0011) [2023-12-27 03:10:11,312][105692] Updated weights for policy 0, policy_version 1616457 (0.0010) [2023-12-27 03:10:11,355][105620] Updated weights for policy 1, policy_version 1619796 (0.0009) [2023-12-27 03:10:11,377][105692] Updated weights for policy 0, policy_version 1616467 (0.0008) [2023-12-27 03:10:11,422][105620] Updated weights for policy 1, policy_version 1619806 (0.0007) [2023-12-27 03:10:12,137][105692] Updated weights for policy 0, policy_version 1616477 (0.0010) [2023-12-27 03:10:12,144][105620] Updated weights for policy 1, policy_version 1619816 (0.0009) [2023-12-27 03:10:12,193][105692] Updated weights for policy 0, policy_version 1616487 (0.0011) [2023-12-27 03:10:12,200][105620] Updated weights for policy 1, policy_version 1619826 (0.0006) [2023-12-27 03:10:12,253][105620] Updated weights for policy 1, policy_version 1619836 (0.0006) [2023-12-27 03:10:12,255][105692] Updated weights for policy 0, policy_version 1616497 (0.0010) [2023-12-27 03:10:12,886][105692] Updated weights for policy 0, policy_version 1616507 (0.0009) [2023-12-27 03:10:12,944][105692] Updated weights for policy 0, policy_version 1616517 (0.0010) [2023-12-27 03:10:13,009][105692] Updated weights for policy 0, policy_version 1616527 (0.0011) [2023-12-27 03:10:13,043][105620] Updated weights for policy 1, policy_version 1619846 (0.0007) [2023-12-27 03:10:13,097][105620] Updated weights for policy 1, policy_version 1619856 (0.0008) [2023-12-27 03:10:13,156][105620] Updated weights for policy 1, policy_version 1619866 (0.0008) [2023-12-27 03:10:13,740][105692] Updated weights for policy 0, policy_version 1616537 (0.0010) [2023-12-27 03:10:13,801][105692] Updated weights for policy 0, policy_version 1616547 (0.0005) [2023-12-27 03:10:13,849][105692] Updated weights for policy 0, policy_version 1616557 (0.0006) [2023-12-27 03:10:13,894][105692] Updated weights for policy 0, policy_version 1616567 (0.0005) [2023-12-27 03:10:13,933][105620] Updated weights for policy 1, policy_version 1619876 (0.0009) [2023-12-27 03:10:13,986][105620] Updated weights for policy 1, policy_version 1619886 (0.0008) [2023-12-27 03:10:14,052][105620] Updated weights for policy 1, policy_version 1619896 (0.0010) [2023-12-27 03:10:14,558][105692] Updated weights for policy 0, policy_version 1616577 (0.0008) [2023-12-27 03:10:14,612][105692] Updated weights for policy 0, policy_version 1616587 (0.0009) [2023-12-27 03:10:14,666][105692] Updated weights for policy 0, policy_version 1616597 (0.0009) [2023-12-27 03:10:14,844][105620] Updated weights for policy 1, policy_version 1619906 (0.0008) [2023-12-27 03:10:14,914][105620] Updated weights for policy 1, policy_version 1619916 (0.0006) [2023-12-27 03:10:14,978][105620] Updated weights for policy 1, policy_version 1619926 (0.0006) [2023-12-27 03:10:15,039][105620] Updated weights for policy 1, policy_version 1619936 (0.0008) [2023-12-27 03:10:15,500][105692] Updated weights for policy 0, policy_version 1616607 (0.0009) [2023-12-27 03:10:15,551][105692] Updated weights for policy 0, policy_version 1616617 (0.0008) [2023-12-27 03:10:15,597][105692] Updated weights for policy 0, policy_version 1616627 (0.0007) [2023-12-27 03:10:15,604][105620] Updated weights for policy 1, policy_version 1619946 (0.0008) [2023-12-27 03:10:15,663][105620] Updated weights for policy 1, policy_version 1619956 (0.0007) [2023-12-27 03:10:15,728][105620] Updated weights for policy 1, policy_version 1619966 (0.0007) [2023-12-27 03:10:16,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 828686336. Throughput: 0: 9825.2, 1: 9761.4. Samples: 828655740. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:10:16,062][104569] Avg episode reward: [(0, '8531.871'), (1, '9173.981')] [2023-12-27 03:10:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001616632_413917184.pth... [2023-12-27 03:10:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001619968_414769152.pth... [2023-12-27 03:10:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001615480_413622272.pth [2023-12-27 03:10:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001618816_414474240.pth [2023-12-27 03:10:16,297][105620] Updated weights for policy 1, policy_version 1619976 (0.0005) [2023-12-27 03:10:16,363][105620] Updated weights for policy 1, policy_version 1619987 (0.0007) [2023-12-27 03:10:16,418][105620] Updated weights for policy 1, policy_version 1619997 (0.0005) [2023-12-27 03:10:16,469][105692] Updated weights for policy 0, policy_version 1616637 (0.0007) [2023-12-27 03:10:16,524][105692] Updated weights for policy 0, policy_version 1616647 (0.0008) [2023-12-27 03:10:16,581][105692] Updated weights for policy 0, policy_version 1616657 (0.0008) [2023-12-27 03:10:17,004][105620] Updated weights for policy 1, policy_version 1620007 (0.0006) [2023-12-27 03:10:17,072][105620] Updated weights for policy 1, policy_version 1620017 (0.0005) [2023-12-27 03:10:17,138][105620] Updated weights for policy 1, policy_version 1620027 (0.0008) [2023-12-27 03:10:17,436][105692] Updated weights for policy 0, policy_version 1616667 (0.0009) [2023-12-27 03:10:17,491][105692] Updated weights for policy 0, policy_version 1616677 (0.0009) [2023-12-27 03:10:17,550][105692] Updated weights for policy 0, policy_version 1616688 (0.0011) [2023-12-27 03:10:17,711][105620] Updated weights for policy 1, policy_version 1620037 (0.0009) [2023-12-27 03:10:17,766][105620] Updated weights for policy 1, policy_version 1620047 (0.0009) [2023-12-27 03:10:17,816][105620] Updated weights for policy 1, policy_version 1620057 (0.0008) [2023-12-27 03:10:18,396][105620] Updated weights for policy 1, policy_version 1620067 (0.0008) [2023-12-27 03:10:18,400][105692] Updated weights for policy 0, policy_version 1616698 (0.0008) [2023-12-27 03:10:18,459][105620] Updated weights for policy 1, policy_version 1620077 (0.0007) [2023-12-27 03:10:18,461][105692] Updated weights for policy 0, policy_version 1616708 (0.0008) [2023-12-27 03:10:18,512][105620] Updated weights for policy 1, policy_version 1620087 (0.0006) [2023-12-27 03:10:18,518][105692] Updated weights for policy 0, policy_version 1616718 (0.0008) [2023-12-27 03:10:18,579][105692] Updated weights for policy 0, policy_version 1616728 (0.0009) [2023-12-27 03:10:19,256][105620] Updated weights for policy 1, policy_version 1620097 (0.0006) [2023-12-27 03:10:19,318][105692] Updated weights for policy 0, policy_version 1616738 (0.0008) [2023-12-27 03:10:19,320][105620] Updated weights for policy 1, policy_version 1620107 (0.0005) [2023-12-27 03:10:19,383][105692] Updated weights for policy 0, policy_version 1616748 (0.0008) [2023-12-27 03:10:19,389][105620] Updated weights for policy 1, policy_version 1620117 (0.0009) [2023-12-27 03:10:19,435][105620] Updated weights for policy 1, policy_version 1620127 (0.0008) [2023-12-27 03:10:19,442][105692] Updated weights for policy 0, policy_version 1616758 (0.0005) [2023-12-27 03:10:20,125][105620] Updated weights for policy 1, policy_version 1620137 (0.0007) [2023-12-27 03:10:20,188][105620] Updated weights for policy 1, policy_version 1620147 (0.0008) [2023-12-27 03:10:20,223][105692] Updated weights for policy 0, policy_version 1616768 (0.0007) [2023-12-27 03:10:20,250][105620] Updated weights for policy 1, policy_version 1620157 (0.0007) [2023-12-27 03:10:20,282][105692] Updated weights for policy 0, policy_version 1616778 (0.0006) [2023-12-27 03:10:20,346][105692] Updated weights for policy 0, policy_version 1616788 (0.0007) [2023-12-27 03:10:21,038][105620] Updated weights for policy 1, policy_version 1620167 (0.0008) [2023-12-27 03:10:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 828776448. Throughput: 0: 9719.5, 1: 9910.6. Samples: 828773172. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:10:21,062][104569] Avg episode reward: [(0, '8534.439'), (1, '8991.937')] [2023-12-27 03:10:21,101][105620] Updated weights for policy 1, policy_version 1620177 (0.0007) [2023-12-27 03:10:21,129][105692] Updated weights for policy 0, policy_version 1616798 (0.0008) [2023-12-27 03:10:21,173][105620] Updated weights for policy 1, policy_version 1620187 (0.0008) [2023-12-27 03:10:21,196][105692] Updated weights for policy 0, policy_version 1616808 (0.0007) [2023-12-27 03:10:21,260][105692] Updated weights for policy 0, policy_version 1616818 (0.0008) [2023-12-27 03:10:21,929][105692] Updated weights for policy 0, policy_version 1616828 (0.0006) [2023-12-27 03:10:21,980][105620] Updated weights for policy 1, policy_version 1620197 (0.0009) [2023-12-27 03:10:21,987][105692] Updated weights for policy 0, policy_version 1616838 (0.0008) [2023-12-27 03:10:22,040][105620] Updated weights for policy 1, policy_version 1620207 (0.0009) [2023-12-27 03:10:22,043][105692] Updated weights for policy 0, policy_version 1616848 (0.0006) [2023-12-27 03:10:22,099][105620] Updated weights for policy 1, policy_version 1620217 (0.0006) [2023-12-27 03:10:22,765][105620] Updated weights for policy 1, policy_version 1620227 (0.0009) [2023-12-27 03:10:22,816][105620] Updated weights for policy 1, policy_version 1620237 (0.0007) [2023-12-27 03:10:22,866][105692] Updated weights for policy 0, policy_version 1616858 (0.0008) [2023-12-27 03:10:22,878][105620] Updated weights for policy 1, policy_version 1620247 (0.0007) [2023-12-27 03:10:22,919][105692] Updated weights for policy 0, policy_version 1616868 (0.0009) [2023-12-27 03:10:22,969][105692] Updated weights for policy 0, policy_version 1616878 (0.0008) [2023-12-27 03:10:23,028][105692] Updated weights for policy 0, policy_version 1616888 (0.0009) [2023-12-27 03:10:23,616][105620] Updated weights for policy 1, policy_version 1620257 (0.0006) [2023-12-27 03:10:23,671][105620] Updated weights for policy 1, policy_version 1620267 (0.0009) [2023-12-27 03:10:23,723][105620] Updated weights for policy 1, policy_version 1620277 (0.0007) [2023-12-27 03:10:23,774][105620] Updated weights for policy 1, policy_version 1620287 (0.0007) [2023-12-27 03:10:23,807][105692] Updated weights for policy 0, policy_version 1616898 (0.0007) [2023-12-27 03:10:23,850][105692] Updated weights for policy 0, policy_version 1616908 (0.0005) [2023-12-27 03:10:23,905][105692] Updated weights for policy 0, policy_version 1616918 (0.0005) [2023-12-27 03:10:24,540][105692] Updated weights for policy 0, policy_version 1616928 (0.0005) [2023-12-27 03:10:24,564][105620] Updated weights for policy 1, policy_version 1620297 (0.0008) [2023-12-27 03:10:24,601][105692] Updated weights for policy 0, policy_version 1616938 (0.0005) [2023-12-27 03:10:24,632][105620] Updated weights for policy 1, policy_version 1620307 (0.0008) [2023-12-27 03:10:24,660][105692] Updated weights for policy 0, policy_version 1616948 (0.0006) [2023-12-27 03:10:24,688][105620] Updated weights for policy 1, policy_version 1620317 (0.0007) [2023-12-27 03:10:25,253][105692] Updated weights for policy 0, policy_version 1616958 (0.0007) [2023-12-27 03:10:25,299][105692] Updated weights for policy 0, policy_version 1616968 (0.0005) [2023-12-27 03:10:25,344][105692] Updated weights for policy 0, policy_version 1616978 (0.0005) [2023-12-27 03:10:25,512][105620] Updated weights for policy 1, policy_version 1620327 (0.0007) [2023-12-27 03:10:25,575][105620] Updated weights for policy 1, policy_version 1620337 (0.0008) [2023-12-27 03:10:25,640][105620] Updated weights for policy 1, policy_version 1620347 (0.0008) [2023-12-27 03:10:26,007][105692] Updated weights for policy 0, policy_version 1616988 (0.0007) [2023-12-27 03:10:26,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 828874752. Throughput: 0: 9698.1, 1: 9771.8. Samples: 828885880. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:10:26,063][104569] Avg episode reward: [(0, '8622.187'), (1, '8989.993')] [2023-12-27 03:10:26,065][105692] Updated weights for policy 0, policy_version 1616998 (0.0009) [2023-12-27 03:10:26,116][105692] Updated weights for policy 0, policy_version 1617008 (0.0009) [2023-12-27 03:10:26,357][105620] Updated weights for policy 1, policy_version 1620357 (0.0010) [2023-12-27 03:10:26,411][105620] Updated weights for policy 1, policy_version 1620367 (0.0010) [2023-12-27 03:10:26,468][105620] Updated weights for policy 1, policy_version 1620377 (0.0010) [2023-12-27 03:10:26,754][105692] Updated weights for policy 0, policy_version 1617018 (0.0008) [2023-12-27 03:10:26,809][105692] Updated weights for policy 0, policy_version 1617028 (0.0005) [2023-12-27 03:10:26,866][105692] Updated weights for policy 0, policy_version 1617038 (0.0005) [2023-12-27 03:10:26,927][105692] Updated weights for policy 0, policy_version 1617048 (0.0006) [2023-12-27 03:10:27,268][105620] Updated weights for policy 1, policy_version 1620387 (0.0009) [2023-12-27 03:10:27,328][105620] Updated weights for policy 1, policy_version 1620397 (0.0008) [2023-12-27 03:10:27,375][105620] Updated weights for policy 1, policy_version 1620407 (0.0008) [2023-12-27 03:10:27,630][105692] Updated weights for policy 0, policy_version 1617058 (0.0009) [2023-12-27 03:10:27,677][105692] Updated weights for policy 0, policy_version 1617068 (0.0008) [2023-12-27 03:10:27,728][105692] Updated weights for policy 0, policy_version 1617078 (0.0009) [2023-12-27 03:10:28,129][105620] Updated weights for policy 1, policy_version 1620417 (0.0009) [2023-12-27 03:10:28,175][105620] Updated weights for policy 1, policy_version 1620427 (0.0008) [2023-12-27 03:10:28,225][105620] Updated weights for policy 1, policy_version 1620437 (0.0007) [2023-12-27 03:10:28,290][105620] Updated weights for policy 1, policy_version 1620447 (0.0005) [2023-12-27 03:10:28,509][105692] Updated weights for policy 0, policy_version 1617088 (0.0008) [2023-12-27 03:10:28,558][105692] Updated weights for policy 0, policy_version 1617098 (0.0005) [2023-12-27 03:10:28,612][105692] Updated weights for policy 0, policy_version 1617108 (0.0005) [2023-12-27 03:10:29,070][105620] Updated weights for policy 1, policy_version 1620457 (0.0008) [2023-12-27 03:10:29,121][105620] Updated weights for policy 1, policy_version 1620467 (0.0009) [2023-12-27 03:10:29,181][105620] Updated weights for policy 1, policy_version 1620477 (0.0006) [2023-12-27 03:10:29,235][105692] Updated weights for policy 0, policy_version 1617118 (0.0008) [2023-12-27 03:10:29,301][105692] Updated weights for policy 0, policy_version 1617128 (0.0008) [2023-12-27 03:10:29,359][105692] Updated weights for policy 0, policy_version 1617138 (0.0008) [2023-12-27 03:10:29,891][105620] Updated weights for policy 1, policy_version 1620487 (0.0008) [2023-12-27 03:10:29,942][105620] Updated weights for policy 1, policy_version 1620497 (0.0009) [2023-12-27 03:10:29,993][105620] Updated weights for policy 1, policy_version 1620507 (0.0009) [2023-12-27 03:10:30,120][105692] Updated weights for policy 0, policy_version 1617148 (0.0009) [2023-12-27 03:10:30,182][105692] Updated weights for policy 0, policy_version 1617158 (0.0009) [2023-12-27 03:10:30,247][105692] Updated weights for policy 0, policy_version 1617168 (0.0009) [2023-12-27 03:10:30,720][105620] Updated weights for policy 1, policy_version 1620517 (0.0007) [2023-12-27 03:10:30,770][105620] Updated weights for policy 1, policy_version 1620527 (0.0005) [2023-12-27 03:10:30,814][105620] Updated weights for policy 1, policy_version 1620537 (0.0005) [2023-12-27 03:10:30,890][105692] Updated weights for policy 0, policy_version 1617178 (0.0008) [2023-12-27 03:10:30,937][105692] Updated weights for policy 0, policy_version 1617188 (0.0008) [2023-12-27 03:10:30,984][105692] Updated weights for policy 0, policy_version 1617198 (0.0008) [2023-12-27 03:10:31,041][105692] Updated weights for policy 0, policy_version 1617208 (0.0007) [2023-12-27 03:10:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 828981248. Throughput: 0: 9749.2, 1: 9780.5. Samples: 828943756. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:10:31,062][104569] Avg episode reward: [(0, '8803.995'), (1, '8898.691')] [2023-12-27 03:10:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001617208_414064640.pth... [2023-12-27 03:10:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001620544_414916608.pth... [2023-12-27 03:10:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001616056_413769728.pth [2023-12-27 03:10:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001619424_414629888.pth [2023-12-27 03:10:31,438][105620] Updated weights for policy 1, policy_version 1620547 (0.0006) [2023-12-27 03:10:31,486][105620] Updated weights for policy 1, policy_version 1620557 (0.0008) [2023-12-27 03:10:31,538][105620] Updated weights for policy 1, policy_version 1620567 (0.0008) [2023-12-27 03:10:31,844][105692] Updated weights for policy 0, policy_version 1617218 (0.0005) [2023-12-27 03:10:31,893][105692] Updated weights for policy 0, policy_version 1617228 (0.0005) [2023-12-27 03:10:31,946][105692] Updated weights for policy 0, policy_version 1617238 (0.0009) [2023-12-27 03:10:32,277][105620] Updated weights for policy 1, policy_version 1620577 (0.0008) [2023-12-27 03:10:32,338][105620] Updated weights for policy 1, policy_version 1620587 (0.0009) [2023-12-27 03:10:32,401][105620] Updated weights for policy 1, policy_version 1620597 (0.0009) [2023-12-27 03:10:32,459][105620] Updated weights for policy 1, policy_version 1620607 (0.0009) [2023-12-27 03:10:32,686][105692] Updated weights for policy 0, policy_version 1617248 (0.0009) [2023-12-27 03:10:32,751][105692] Updated weights for policy 0, policy_version 1617258 (0.0009) [2023-12-27 03:10:32,801][105692] Updated weights for policy 0, policy_version 1617268 (0.0009) [2023-12-27 03:10:33,135][105620] Updated weights for policy 1, policy_version 1620617 (0.0010) [2023-12-27 03:10:33,192][105620] Updated weights for policy 1, policy_version 1620627 (0.0010) [2023-12-27 03:10:33,250][105620] Updated weights for policy 1, policy_version 1620637 (0.0010) [2023-12-27 03:10:33,559][105692] Updated weights for policy 0, policy_version 1617278 (0.0008) [2023-12-27 03:10:33,617][105692] Updated weights for policy 0, policy_version 1617288 (0.0008) [2023-12-27 03:10:33,672][105692] Updated weights for policy 0, policy_version 1617298 (0.0008) [2023-12-27 03:10:33,927][105620] Updated weights for policy 1, policy_version 1620647 (0.0007) [2023-12-27 03:10:33,978][105620] Updated weights for policy 1, policy_version 1620657 (0.0005) [2023-12-27 03:10:34,040][105620] Updated weights for policy 1, policy_version 1620667 (0.0010) [2023-12-27 03:10:34,508][105692] Updated weights for policy 0, policy_version 1617308 (0.0008) [2023-12-27 03:10:34,564][105692] Updated weights for policy 0, policy_version 1617318 (0.0009) [2023-12-27 03:10:34,627][105692] Updated weights for policy 0, policy_version 1617328 (0.0009) [2023-12-27 03:10:34,724][105620] Updated weights for policy 1, policy_version 1620677 (0.0009) [2023-12-27 03:10:34,790][105620] Updated weights for policy 1, policy_version 1620687 (0.0009) [2023-12-27 03:10:34,848][105620] Updated weights for policy 1, policy_version 1620697 (0.0009) [2023-12-27 03:10:35,416][105692] Updated weights for policy 0, policy_version 1617338 (0.0008) [2023-12-27 03:10:35,472][105692] Updated weights for policy 0, policy_version 1617348 (0.0008) [2023-12-27 03:10:35,530][105692] Updated weights for policy 0, policy_version 1617358 (0.0008) [2023-12-27 03:10:35,559][105620] Updated weights for policy 1, policy_version 1620707 (0.0009) [2023-12-27 03:10:35,585][105692] Updated weights for policy 0, policy_version 1617368 (0.0007) [2023-12-27 03:10:35,607][105620] Updated weights for policy 1, policy_version 1620717 (0.0010) [2023-12-27 03:10:35,668][105620] Updated weights for policy 1, policy_version 1620727 (0.0010) [2023-12-27 03:10:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 829071360. Throughput: 0: 9676.2, 1: 9695.1. Samples: 829061188. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:10:36,063][104569] Avg episode reward: [(0, '8805.166'), (1, '8991.357')] [2023-12-27 03:10:36,348][105620] Updated weights for policy 1, policy_version 1620737 (0.0010) [2023-12-27 03:10:36,400][105692] Updated weights for policy 0, policy_version 1617378 (0.0007) [2023-12-27 03:10:36,413][105620] Updated weights for policy 1, policy_version 1620747 (0.0009) [2023-12-27 03:10:36,466][105692] Updated weights for policy 0, policy_version 1617388 (0.0008) [2023-12-27 03:10:36,472][105620] Updated weights for policy 1, policy_version 1620757 (0.0008) [2023-12-27 03:10:36,527][105692] Updated weights for policy 0, policy_version 1617398 (0.0007) [2023-12-27 03:10:36,533][105620] Updated weights for policy 1, policy_version 1620767 (0.0006) [2023-12-27 03:10:37,216][105620] Updated weights for policy 1, policy_version 1620777 (0.0005) [2023-12-27 03:10:37,275][105620] Updated weights for policy 1, policy_version 1620787 (0.0005) [2023-12-27 03:10:37,324][105692] Updated weights for policy 0, policy_version 1617408 (0.0008) [2023-12-27 03:10:37,331][105620] Updated weights for policy 1, policy_version 1620797 (0.0005) [2023-12-27 03:10:37,385][105692] Updated weights for policy 0, policy_version 1617418 (0.0010) [2023-12-27 03:10:37,433][105692] Updated weights for policy 0, policy_version 1617428 (0.0009) [2023-12-27 03:10:37,865][105620] Updated weights for policy 1, policy_version 1620807 (0.0007) [2023-12-27 03:10:37,934][105620] Updated weights for policy 1, policy_version 1620817 (0.0009) [2023-12-27 03:10:37,988][105620] Updated weights for policy 1, policy_version 1620827 (0.0009) [2023-12-27 03:10:38,260][105692] Updated weights for policy 0, policy_version 1617438 (0.0007) [2023-12-27 03:10:38,319][105692] Updated weights for policy 0, policy_version 1617448 (0.0006) [2023-12-27 03:10:38,378][105692] Updated weights for policy 0, policy_version 1617458 (0.0007) [2023-12-27 03:10:38,775][105620] Updated weights for policy 1, policy_version 1620837 (0.0008) [2023-12-27 03:10:38,834][105620] Updated weights for policy 1, policy_version 1620847 (0.0008) [2023-12-27 03:10:38,884][105620] Updated weights for policy 1, policy_version 1620857 (0.0009) [2023-12-27 03:10:39,076][105692] Updated weights for policy 0, policy_version 1617468 (0.0011) [2023-12-27 03:10:39,141][105692] Updated weights for policy 0, policy_version 1617478 (0.0011) [2023-12-27 03:10:39,198][105692] Updated weights for policy 0, policy_version 1617488 (0.0010) [2023-12-27 03:10:39,621][105620] Updated weights for policy 1, policy_version 1620867 (0.0007) [2023-12-27 03:10:39,683][105620] Updated weights for policy 1, policy_version 1620877 (0.0006) [2023-12-27 03:10:39,751][105620] Updated weights for policy 1, policy_version 1620887 (0.0008) [2023-12-27 03:10:39,905][105692] Updated weights for policy 0, policy_version 1617498 (0.0008) [2023-12-27 03:10:39,973][105692] Updated weights for policy 0, policy_version 1617508 (0.0007) [2023-12-27 03:10:40,031][105692] Updated weights for policy 0, policy_version 1617518 (0.0008) [2023-12-27 03:10:40,094][105692] Updated weights for policy 0, policy_version 1617528 (0.0009) [2023-12-27 03:10:40,498][105620] Updated weights for policy 1, policy_version 1620897 (0.0010) [2023-12-27 03:10:40,564][105620] Updated weights for policy 1, policy_version 1620907 (0.0005) [2023-12-27 03:10:40,630][105620] Updated weights for policy 1, policy_version 1620917 (0.0005) [2023-12-27 03:10:40,690][105620] Updated weights for policy 1, policy_version 1620927 (0.0005) [2023-12-27 03:10:40,863][105692] Updated weights for policy 0, policy_version 1617538 (0.0007) [2023-12-27 03:10:40,929][105692] Updated weights for policy 0, policy_version 1617548 (0.0006) [2023-12-27 03:10:40,987][105692] Updated weights for policy 0, policy_version 1617558 (0.0010) [2023-12-27 03:10:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 829169664. Throughput: 0: 9611.2, 1: 9675.8. Samples: 829176628. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:10:41,062][104569] Avg episode reward: [(0, '9078.938'), (1, '8987.593')] [2023-12-27 03:10:41,263][105620] Updated weights for policy 1, policy_version 1620937 (0.0007) [2023-12-27 03:10:41,328][105620] Updated weights for policy 1, policy_version 1620947 (0.0006) [2023-12-27 03:10:41,394][105620] Updated weights for policy 1, policy_version 1620957 (0.0009) [2023-12-27 03:10:41,787][105692] Updated weights for policy 0, policy_version 1617568 (0.0008) [2023-12-27 03:10:41,845][105692] Updated weights for policy 0, policy_version 1617578 (0.0010) [2023-12-27 03:10:41,901][105692] Updated weights for policy 0, policy_version 1617588 (0.0011) [2023-12-27 03:10:42,078][105620] Updated weights for policy 1, policy_version 1620967 (0.0006) [2023-12-27 03:10:42,137][105620] Updated weights for policy 1, policy_version 1620977 (0.0006) [2023-12-27 03:10:42,196][105620] Updated weights for policy 1, policy_version 1620987 (0.0008) [2023-12-27 03:10:42,723][105692] Updated weights for policy 0, policy_version 1617598 (0.0009) [2023-12-27 03:10:42,779][105692] Updated weights for policy 0, policy_version 1617608 (0.0009) [2023-12-27 03:10:42,843][105692] Updated weights for policy 0, policy_version 1617618 (0.0009) [2023-12-27 03:10:42,976][105620] Updated weights for policy 1, policy_version 1620997 (0.0009) [2023-12-27 03:10:43,022][105620] Updated weights for policy 1, policy_version 1621007 (0.0009) [2023-12-27 03:10:43,082][105620] Updated weights for policy 1, policy_version 1621017 (0.0008) [2023-12-27 03:10:43,681][105620] Updated weights for policy 1, policy_version 1621027 (0.0006) [2023-12-27 03:10:43,682][105692] Updated weights for policy 0, policy_version 1617628 (0.0009) [2023-12-27 03:10:43,729][105692] Updated weights for policy 0, policy_version 1617638 (0.0007) [2023-12-27 03:10:43,738][105620] Updated weights for policy 1, policy_version 1621037 (0.0007) [2023-12-27 03:10:43,775][105692] Updated weights for policy 0, policy_version 1617648 (0.0007) [2023-12-27 03:10:43,786][105620] Updated weights for policy 1, policy_version 1621047 (0.0007) [2023-12-27 03:10:44,383][105692] Updated weights for policy 0, policy_version 1617658 (0.0007) [2023-12-27 03:10:44,444][105692] Updated weights for policy 0, policy_version 1617668 (0.0005) [2023-12-27 03:10:44,494][105692] Updated weights for policy 0, policy_version 1617678 (0.0006) [2023-12-27 03:10:44,545][105692] Updated weights for policy 0, policy_version 1617688 (0.0006) [2023-12-27 03:10:44,637][105620] Updated weights for policy 1, policy_version 1621057 (0.0006) [2023-12-27 03:10:44,698][105620] Updated weights for policy 1, policy_version 1621067 (0.0009) [2023-12-27 03:10:44,765][105620] Updated weights for policy 1, policy_version 1621077 (0.0008) [2023-12-27 03:10:44,824][105620] Updated weights for policy 1, policy_version 1621087 (0.0009) [2023-12-27 03:10:45,217][105692] Updated weights for policy 0, policy_version 1617698 (0.0009) [2023-12-27 03:10:45,286][105692] Updated weights for policy 0, policy_version 1617708 (0.0009) [2023-12-27 03:10:45,349][105692] Updated weights for policy 0, policy_version 1617718 (0.0009) [2023-12-27 03:10:45,552][105620] Updated weights for policy 1, policy_version 1621097 (0.0006) [2023-12-27 03:10:45,617][105620] Updated weights for policy 1, policy_version 1621107 (0.0006) [2023-12-27 03:10:45,679][105620] Updated weights for policy 1, policy_version 1621117 (0.0005) [2023-12-27 03:10:46,001][105692] Updated weights for policy 0, policy_version 1617728 (0.0009) [2023-12-27 03:10:46,056][105692] Updated weights for policy 0, policy_version 1617738 (0.0010) [2023-12-27 03:10:46,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 829259776. Throughput: 0: 9542.1, 1: 9701.7. Samples: 829232836. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:10:46,062][104569] Avg episode reward: [(0, '9079.445'), (1, '8989.202')] [2023-12-27 03:10:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001621120_415064064.pth... [2023-12-27 03:10:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001619968_414769152.pth [2023-12-27 03:10:46,111][105692] Updated weights for policy 0, policy_version 1617748 (0.0010) [2023-12-27 03:10:46,128][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001617752_414203904.pth... [2023-12-27 03:10:46,131][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001616632_413917184.pth [2023-12-27 03:10:46,257][105620] Updated weights for policy 1, policy_version 1621127 (0.0006) [2023-12-27 03:10:46,304][105620] Updated weights for policy 1, policy_version 1621137 (0.0008) [2023-12-27 03:10:46,349][105620] Updated weights for policy 1, policy_version 1621147 (0.0008) [2023-12-27 03:10:46,820][105692] Updated weights for policy 0, policy_version 1617758 (0.0009) [2023-12-27 03:10:46,888][105692] Updated weights for policy 0, policy_version 1617768 (0.0010) [2023-12-27 03:10:46,940][105620] Updated weights for policy 1, policy_version 1621157 (0.0005) [2023-12-27 03:10:46,950][105692] Updated weights for policy 0, policy_version 1617778 (0.0010) [2023-12-27 03:10:47,000][105620] Updated weights for policy 1, policy_version 1621167 (0.0006) [2023-12-27 03:10:47,056][105620] Updated weights for policy 1, policy_version 1621177 (0.0005) [2023-12-27 03:10:47,570][105692] Updated weights for policy 0, policy_version 1617788 (0.0009) [2023-12-27 03:10:47,605][105620] Updated weights for policy 1, policy_version 1621187 (0.0007) [2023-12-27 03:10:47,631][105692] Updated weights for policy 0, policy_version 1617798 (0.0007) [2023-12-27 03:10:47,666][105620] Updated weights for policy 1, policy_version 1621197 (0.0010) [2023-12-27 03:10:47,684][105692] Updated weights for policy 0, policy_version 1617808 (0.0007) [2023-12-27 03:10:47,731][105620] Updated weights for policy 1, policy_version 1621207 (0.0010) [2023-12-27 03:10:48,314][105692] Updated weights for policy 0, policy_version 1617818 (0.0008) [2023-12-27 03:10:48,383][105692] Updated weights for policy 0, policy_version 1617828 (0.0009) [2023-12-27 03:10:48,408][105620] Updated weights for policy 1, policy_version 1621217 (0.0010) [2023-12-27 03:10:48,438][105692] Updated weights for policy 0, policy_version 1617838 (0.0008) [2023-12-27 03:10:48,477][105620] Updated weights for policy 1, policy_version 1621227 (0.0005) [2023-12-27 03:10:48,493][105692] Updated weights for policy 0, policy_version 1617848 (0.0009) [2023-12-27 03:10:48,537][105620] Updated weights for policy 1, policy_version 1621237 (0.0006) [2023-12-27 03:10:48,593][105620] Updated weights for policy 1, policy_version 1621247 (0.0010) [2023-12-27 03:10:49,234][105692] Updated weights for policy 0, policy_version 1617858 (0.0006) [2023-12-27 03:10:49,251][105620] Updated weights for policy 1, policy_version 1621257 (0.0009) [2023-12-27 03:10:49,297][105692] Updated weights for policy 0, policy_version 1617868 (0.0011) [2023-12-27 03:10:49,314][105620] Updated weights for policy 1, policy_version 1621267 (0.0010) [2023-12-27 03:10:49,364][105692] Updated weights for policy 0, policy_version 1617878 (0.0012) [2023-12-27 03:10:49,393][105620] Updated weights for policy 1, policy_version 1621277 (0.0009) [2023-12-27 03:10:50,001][105620] Updated weights for policy 1, policy_version 1621287 (0.0007) [2023-12-27 03:10:50,061][105620] Updated weights for policy 1, policy_version 1621297 (0.0011) [2023-12-27 03:10:50,109][105692] Updated weights for policy 0, policy_version 1617888 (0.0008) [2023-12-27 03:10:50,117][105620] Updated weights for policy 1, policy_version 1621307 (0.0010) [2023-12-27 03:10:50,173][105692] Updated weights for policy 0, policy_version 1617898 (0.0009) [2023-12-27 03:10:50,222][105692] Updated weights for policy 0, policy_version 1617908 (0.0010) [2023-12-27 03:10:50,863][105692] Updated weights for policy 0, policy_version 1617918 (0.0010) [2023-12-27 03:10:50,870][105620] Updated weights for policy 1, policy_version 1621317 (0.0009) [2023-12-27 03:10:50,917][105692] Updated weights for policy 0, policy_version 1617928 (0.0010) [2023-12-27 03:10:50,931][105620] Updated weights for policy 1, policy_version 1621327 (0.0007) [2023-12-27 03:10:50,968][105692] Updated weights for policy 0, policy_version 1617938 (0.0011) [2023-12-27 03:10:50,993][105620] Updated weights for policy 1, policy_version 1621337 (0.0007) [2023-12-27 03:10:51,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 829374464. Throughput: 0: 9591.9, 1: 9857.0. Samples: 829357272. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:10:51,063][104569] Avg episode reward: [(0, '8451.367'), (1, '8897.270')] [2023-12-27 03:10:51,678][105620] Updated weights for policy 1, policy_version 1621347 (0.0008) [2023-12-27 03:10:51,679][105692] Updated weights for policy 0, policy_version 1617948 (0.0011) [2023-12-27 03:10:51,747][105620] Updated weights for policy 1, policy_version 1621357 (0.0008) [2023-12-27 03:10:51,748][105692] Updated weights for policy 0, policy_version 1617958 (0.0011) [2023-12-27 03:10:51,807][105620] Updated weights for policy 1, policy_version 1621367 (0.0006) [2023-12-27 03:10:51,808][105692] Updated weights for policy 0, policy_version 1617968 (0.0011) [2023-12-27 03:10:52,520][105620] Updated weights for policy 1, policy_version 1621377 (0.0006) [2023-12-27 03:10:52,555][105692] Updated weights for policy 0, policy_version 1617978 (0.0010) [2023-12-27 03:10:52,578][105620] Updated weights for policy 1, policy_version 1621387 (0.0008) [2023-12-27 03:10:52,611][105692] Updated weights for policy 0, policy_version 1617988 (0.0008) [2023-12-27 03:10:52,638][105620] Updated weights for policy 1, policy_version 1621397 (0.0008) [2023-12-27 03:10:52,658][105692] Updated weights for policy 0, policy_version 1617998 (0.0008) [2023-12-27 03:10:52,690][105620] Updated weights for policy 1, policy_version 1621407 (0.0007) [2023-12-27 03:10:52,713][105692] Updated weights for policy 0, policy_version 1618008 (0.0008) [2023-12-27 03:10:53,382][105620] Updated weights for policy 1, policy_version 1621417 (0.0009) [2023-12-27 03:10:53,437][105620] Updated weights for policy 1, policy_version 1621427 (0.0009) [2023-12-27 03:10:53,488][105620] Updated weights for policy 1, policy_version 1621437 (0.0007) [2023-12-27 03:10:53,519][105692] Updated weights for policy 0, policy_version 1618018 (0.0008) [2023-12-27 03:10:53,581][105692] Updated weights for policy 0, policy_version 1618028 (0.0010) [2023-12-27 03:10:53,631][105692] Updated weights for policy 0, policy_version 1618038 (0.0009) [2023-12-27 03:10:54,200][105620] Updated weights for policy 1, policy_version 1621447 (0.0008) [2023-12-27 03:10:54,256][105620] Updated weights for policy 1, policy_version 1621457 (0.0006) [2023-12-27 03:10:54,319][105620] Updated weights for policy 1, policy_version 1621467 (0.0008) [2023-12-27 03:10:54,383][105692] Updated weights for policy 0, policy_version 1618048 (0.0008) [2023-12-27 03:10:54,442][105692] Updated weights for policy 0, policy_version 1618058 (0.0009) [2023-12-27 03:10:54,507][105692] Updated weights for policy 0, policy_version 1618068 (0.0008) [2023-12-27 03:10:55,015][105620] Updated weights for policy 1, policy_version 1621477 (0.0007) [2023-12-27 03:10:55,073][105620] Updated weights for policy 1, policy_version 1621487 (0.0005) [2023-12-27 03:10:55,131][105620] Updated weights for policy 1, policy_version 1621497 (0.0005) [2023-12-27 03:10:55,158][105692] Updated weights for policy 0, policy_version 1618078 (0.0009) [2023-12-27 03:10:55,219][105692] Updated weights for policy 0, policy_version 1618088 (0.0010) [2023-12-27 03:10:55,274][105692] Updated weights for policy 0, policy_version 1618098 (0.0009) [2023-12-27 03:10:55,710][105620] Updated weights for policy 1, policy_version 1621507 (0.0005) [2023-12-27 03:10:55,757][105620] Updated weights for policy 1, policy_version 1621517 (0.0009) [2023-12-27 03:10:55,813][105620] Updated weights for policy 1, policy_version 1621527 (0.0005) [2023-12-27 03:10:56,025][105692] Updated weights for policy 0, policy_version 1618108 (0.0008) [2023-12-27 03:10:56,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 829464576. Throughput: 0: 9505.1, 1: 9953.2. Samples: 829474856. Policy #0 lag: (min: 12.0, avg: 31.6, max: 44.0) [2023-12-27 03:10:56,063][104569] Avg episode reward: [(0, '8271.526'), (1, '8713.543')] [2023-12-27 03:10:56,098][105692] Updated weights for policy 0, policy_version 1618118 (0.0005) [2023-12-27 03:10:56,165][105692] Updated weights for policy 0, policy_version 1618128 (0.0010) [2023-12-27 03:10:56,434][105620] Updated weights for policy 1, policy_version 1621537 (0.0005) [2023-12-27 03:10:56,484][105620] Updated weights for policy 1, policy_version 1621547 (0.0006) [2023-12-27 03:10:56,536][105620] Updated weights for policy 1, policy_version 1621557 (0.0010) [2023-12-27 03:10:56,588][105620] Updated weights for policy 1, policy_version 1621567 (0.0009) [2023-12-27 03:10:56,825][105692] Updated weights for policy 0, policy_version 1618138 (0.0010) [2023-12-27 03:10:56,877][105692] Updated weights for policy 0, policy_version 1618148 (0.0010) [2023-12-27 03:10:56,931][105692] Updated weights for policy 0, policy_version 1618158 (0.0010) [2023-12-27 03:10:56,986][105692] Updated weights for policy 0, policy_version 1618168 (0.0010) [2023-12-27 03:10:57,193][105620] Updated weights for policy 1, policy_version 1621577 (0.0009) [2023-12-27 03:10:57,246][105620] Updated weights for policy 1, policy_version 1621588 (0.0010) [2023-12-27 03:10:57,300][105620] Updated weights for policy 1, policy_version 1621599 (0.0010) [2023-12-27 03:10:57,659][105692] Updated weights for policy 0, policy_version 1618178 (0.0010) [2023-12-27 03:10:57,709][105692] Updated weights for policy 0, policy_version 1618188 (0.0010) [2023-12-27 03:10:57,764][105692] Updated weights for policy 0, policy_version 1618198 (0.0005) [2023-12-27 03:10:58,086][105620] Updated weights for policy 1, policy_version 1621609 (0.0007) [2023-12-27 03:10:58,150][105620] Updated weights for policy 1, policy_version 1621619 (0.0010) [2023-12-27 03:10:58,219][105620] Updated weights for policy 1, policy_version 1621629 (0.0008) [2023-12-27 03:10:58,430][105692] Updated weights for policy 0, policy_version 1618208 (0.0010) [2023-12-27 03:10:58,492][105692] Updated weights for policy 0, policy_version 1618218 (0.0011) [2023-12-27 03:10:58,554][105692] Updated weights for policy 0, policy_version 1618228 (0.0010) [2023-12-27 03:10:58,982][105620] Updated weights for policy 1, policy_version 1621639 (0.0008) [2023-12-27 03:10:59,044][105620] Updated weights for policy 1, policy_version 1621649 (0.0008) [2023-12-27 03:10:59,106][105620] Updated weights for policy 1, policy_version 1621659 (0.0009) [2023-12-27 03:10:59,348][105692] Updated weights for policy 0, policy_version 1618238 (0.0009) [2023-12-27 03:10:59,406][105692] Updated weights for policy 0, policy_version 1618248 (0.0007) [2023-12-27 03:10:59,465][105692] Updated weights for policy 0, policy_version 1618258 (0.0007) [2023-12-27 03:10:59,952][105620] Updated weights for policy 1, policy_version 1621669 (0.0009) [2023-12-27 03:11:00,006][105620] Updated weights for policy 1, policy_version 1621679 (0.0010) [2023-12-27 03:11:00,059][105620] Updated weights for policy 1, policy_version 1621689 (0.0010) [2023-12-27 03:11:00,118][105692] Updated weights for policy 0, policy_version 1618268 (0.0007) [2023-12-27 03:11:00,184][105692] Updated weights for policy 0, policy_version 1618278 (0.0005) [2023-12-27 03:11:00,237][105692] Updated weights for policy 0, policy_version 1618288 (0.0005) [2023-12-27 03:11:00,762][105692] Updated weights for policy 0, policy_version 1618298 (0.0006) [2023-12-27 03:11:00,823][105692] Updated weights for policy 0, policy_version 1618308 (0.0009) [2023-12-27 03:11:00,887][105692] Updated weights for policy 0, policy_version 1618318 (0.0009) [2023-12-27 03:11:00,921][105620] Updated weights for policy 1, policy_version 1621699 (0.0010) [2023-12-27 03:11:00,943][105692] Updated weights for policy 0, policy_version 1618328 (0.0007) [2023-12-27 03:11:00,980][105620] Updated weights for policy 1, policy_version 1621709 (0.0008) [2023-12-27 03:11:01,036][105620] Updated weights for policy 1, policy_version 1621719 (0.0009) [2023-12-27 03:11:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 829562880. Throughput: 0: 9534.8, 1: 10003.0. Samples: 829534944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:11:01,063][104569] Avg episode reward: [(0, '8169.843'), (1, '8715.275')] [2023-12-27 03:11:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001618328_414351360.pth... [2023-12-27 03:11:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001617208_414064640.pth [2023-12-27 03:11:01,089][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001621728_415219712.pth... [2023-12-27 03:11:01,093][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001620544_414916608.pth [2023-12-27 03:11:01,716][105692] Updated weights for policy 0, policy_version 1618338 (0.0009) [2023-12-27 03:11:01,737][105620] Updated weights for policy 1, policy_version 1621729 (0.0008) [2023-12-27 03:11:01,782][105692] Updated weights for policy 0, policy_version 1618348 (0.0007) [2023-12-27 03:11:01,799][105620] Updated weights for policy 1, policy_version 1621739 (0.0010) [2023-12-27 03:11:01,837][105692] Updated weights for policy 0, policy_version 1618358 (0.0006) [2023-12-27 03:11:01,859][105620] Updated weights for policy 1, policy_version 1621749 (0.0008) [2023-12-27 03:11:01,920][105620] Updated weights for policy 1, policy_version 1621759 (0.0009) [2023-12-27 03:11:02,594][105692] Updated weights for policy 0, policy_version 1618368 (0.0007) [2023-12-27 03:11:02,646][105692] Updated weights for policy 0, policy_version 1618378 (0.0005) [2023-12-27 03:11:02,662][105620] Updated weights for policy 1, policy_version 1621769 (0.0008) [2023-12-27 03:11:02,698][105692] Updated weights for policy 0, policy_version 1618388 (0.0006) [2023-12-27 03:11:02,719][105620] Updated weights for policy 1, policy_version 1621779 (0.0009) [2023-12-27 03:11:02,776][105620] Updated weights for policy 1, policy_version 1621789 (0.0006) [2023-12-27 03:11:03,382][105692] Updated weights for policy 0, policy_version 1618398 (0.0009) [2023-12-27 03:11:03,438][105692] Updated weights for policy 0, policy_version 1618408 (0.0010) [2023-12-27 03:11:03,468][105620] Updated weights for policy 1, policy_version 1621799 (0.0006) [2023-12-27 03:11:03,490][105692] Updated weights for policy 0, policy_version 1618418 (0.0010) [2023-12-27 03:11:03,526][105620] Updated weights for policy 1, policy_version 1621809 (0.0005) [2023-12-27 03:11:03,576][105620] Updated weights for policy 1, policy_version 1621819 (0.0005) [2023-12-27 03:11:04,220][105692] Updated weights for policy 0, policy_version 1618428 (0.0009) [2023-12-27 03:11:04,282][105692] Updated weights for policy 0, policy_version 1618438 (0.0009) [2023-12-27 03:11:04,305][105620] Updated weights for policy 1, policy_version 1621829 (0.0008) [2023-12-27 03:11:04,346][105692] Updated weights for policy 0, policy_version 1618448 (0.0006) [2023-12-27 03:11:04,368][105620] Updated weights for policy 1, policy_version 1621839 (0.0008) [2023-12-27 03:11:04,428][105620] Updated weights for policy 1, policy_version 1621849 (0.0008) [2023-12-27 03:11:05,067][105692] Updated weights for policy 0, policy_version 1618458 (0.0006) [2023-12-27 03:11:05,115][105692] Updated weights for policy 0, policy_version 1618468 (0.0008) [2023-12-27 03:11:05,163][105692] Updated weights for policy 0, policy_version 1618478 (0.0009) [2023-12-27 03:11:05,210][105620] Updated weights for policy 1, policy_version 1621859 (0.0009) [2023-12-27 03:11:05,222][105692] Updated weights for policy 0, policy_version 1618488 (0.0009) [2023-12-27 03:11:05,271][105620] Updated weights for policy 1, policy_version 1621869 (0.0008) [2023-12-27 03:11:05,325][105620] Updated weights for policy 1, policy_version 1621879 (0.0009) [2023-12-27 03:11:05,903][105692] Updated weights for policy 0, policy_version 1618498 (0.0008) [2023-12-27 03:11:05,956][105692] Updated weights for policy 0, policy_version 1618508 (0.0008) [2023-12-27 03:11:06,008][105692] Updated weights for policy 0, policy_version 1618518 (0.0008) [2023-12-27 03:11:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 829661184. Throughput: 0: 9657.6, 1: 9803.1. Samples: 829648908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:11:06,063][104569] Avg episode reward: [(0, '8536.070'), (1, '9170.828')] [2023-12-27 03:11:06,103][105620] Updated weights for policy 1, policy_version 1621889 (0.0008) [2023-12-27 03:11:06,169][105620] Updated weights for policy 1, policy_version 1621899 (0.0011) [2023-12-27 03:11:06,226][105620] Updated weights for policy 1, policy_version 1621909 (0.0007) [2023-12-27 03:11:06,287][105620] Updated weights for policy 1, policy_version 1621919 (0.0008) [2023-12-27 03:11:06,820][105692] Updated weights for policy 0, policy_version 1618528 (0.0009) [2023-12-27 03:11:06,876][105692] Updated weights for policy 0, policy_version 1618538 (0.0009) [2023-12-27 03:11:06,929][105692] Updated weights for policy 0, policy_version 1618548 (0.0009) [2023-12-27 03:11:07,035][105620] Updated weights for policy 1, policy_version 1621929 (0.0009) [2023-12-27 03:11:07,096][105620] Updated weights for policy 1, policy_version 1621939 (0.0009) [2023-12-27 03:11:07,162][105620] Updated weights for policy 1, policy_version 1621949 (0.0010) [2023-12-27 03:11:07,532][105692] Updated weights for policy 0, policy_version 1618558 (0.0010) [2023-12-27 03:11:07,594][105692] Updated weights for policy 0, policy_version 1618568 (0.0010) [2023-12-27 03:11:07,652][105692] Updated weights for policy 0, policy_version 1618578 (0.0010) [2023-12-27 03:11:08,012][105620] Updated weights for policy 1, policy_version 1621959 (0.0009) [2023-12-27 03:11:08,068][105620] Updated weights for policy 1, policy_version 1621970 (0.0009) [2023-12-27 03:11:08,124][105620] Updated weights for policy 1, policy_version 1621980 (0.0009) [2023-12-27 03:11:08,214][105692] Updated weights for policy 0, policy_version 1618588 (0.0007) [2023-12-27 03:11:08,275][105692] Updated weights for policy 0, policy_version 1618598 (0.0005) [2023-12-27 03:11:08,341][105692] Updated weights for policy 0, policy_version 1618608 (0.0007) [2023-12-27 03:11:08,907][105692] Updated weights for policy 0, policy_version 1618618 (0.0007) [2023-12-27 03:11:08,965][105692] Updated weights for policy 0, policy_version 1618628 (0.0005) [2023-12-27 03:11:09,018][105692] Updated weights for policy 0, policy_version 1618638 (0.0006) [2023-12-27 03:11:09,035][105620] Updated weights for policy 1, policy_version 1621991 (0.0010) [2023-12-27 03:11:09,066][105692] Updated weights for policy 0, policy_version 1618648 (0.0011) [2023-12-27 03:11:09,086][105620] Updated weights for policy 1, policy_version 1622001 (0.0007) [2023-12-27 03:11:09,138][105620] Updated weights for policy 1, policy_version 1622011 (0.0007) [2023-12-27 03:11:09,688][105692] Updated weights for policy 0, policy_version 1618658 (0.0010) [2023-12-27 03:11:09,752][105692] Updated weights for policy 0, policy_version 1618668 (0.0010) [2023-12-27 03:11:09,813][105692] Updated weights for policy 0, policy_version 1618678 (0.0006) [2023-12-27 03:11:09,999][105620] Updated weights for policy 1, policy_version 1622021 (0.0007) [2023-12-27 03:11:10,055][105620] Updated weights for policy 1, policy_version 1622031 (0.0008) [2023-12-27 03:11:10,113][105620] Updated weights for policy 1, policy_version 1622041 (0.0008) [2023-12-27 03:11:10,526][105692] Updated weights for policy 0, policy_version 1618688 (0.0008) [2023-12-27 03:11:10,583][105692] Updated weights for policy 0, policy_version 1618698 (0.0009) [2023-12-27 03:11:10,642][105692] Updated weights for policy 0, policy_version 1618708 (0.0009) [2023-12-27 03:11:10,954][105620] Updated weights for policy 1, policy_version 1622051 (0.0008) [2023-12-27 03:11:11,002][105620] Updated weights for policy 1, policy_version 1622061 (0.0008) [2023-12-27 03:11:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 829751296. Throughput: 0: 9769.4, 1: 9743.5. Samples: 829763956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:11:11,062][104569] Avg episode reward: [(0, '8897.947'), (1, '9261.135')] [2023-12-27 03:11:11,062][105620] Updated weights for policy 1, policy_version 1622071 (0.0008) [2023-12-27 03:11:11,330][105692] Updated weights for policy 0, policy_version 1618718 (0.0007) [2023-12-27 03:11:11,395][105692] Updated weights for policy 0, policy_version 1618728 (0.0007) [2023-12-27 03:11:11,454][105692] Updated weights for policy 0, policy_version 1618738 (0.0007) [2023-12-27 03:11:11,881][105620] Updated weights for policy 1, policy_version 1622081 (0.0009) [2023-12-27 03:11:11,948][105620] Updated weights for policy 1, policy_version 1622091 (0.0005) [2023-12-27 03:11:12,012][105620] Updated weights for policy 1, policy_version 1622101 (0.0005) [2023-12-27 03:11:12,074][105620] Updated weights for policy 1, policy_version 1622111 (0.0008) [2023-12-27 03:11:12,214][105692] Updated weights for policy 0, policy_version 1618748 (0.0011) [2023-12-27 03:11:12,278][105692] Updated weights for policy 0, policy_version 1618758 (0.0011) [2023-12-27 03:11:12,337][105692] Updated weights for policy 0, policy_version 1618768 (0.0007) [2023-12-27 03:11:12,716][105620] Updated weights for policy 1, policy_version 1622121 (0.0006) [2023-12-27 03:11:12,771][105620] Updated weights for policy 1, policy_version 1622131 (0.0008) [2023-12-27 03:11:12,826][105620] Updated weights for policy 1, policy_version 1622141 (0.0008) [2023-12-27 03:11:13,035][105692] Updated weights for policy 0, policy_version 1618778 (0.0009) [2023-12-27 03:11:13,087][105692] Updated weights for policy 0, policy_version 1618788 (0.0006) [2023-12-27 03:11:13,140][105692] Updated weights for policy 0, policy_version 1618798 (0.0005) [2023-12-27 03:11:13,199][105692] Updated weights for policy 0, policy_version 1618808 (0.0007) [2023-12-27 03:11:13,563][105620] Updated weights for policy 1, policy_version 1622151 (0.0008) [2023-12-27 03:11:13,611][105620] Updated weights for policy 1, policy_version 1622161 (0.0009) [2023-12-27 03:11:13,663][105620] Updated weights for policy 1, policy_version 1622171 (0.0009) [2023-12-27 03:11:13,896][105692] Updated weights for policy 0, policy_version 1618818 (0.0008) [2023-12-27 03:11:13,943][105692] Updated weights for policy 0, policy_version 1618828 (0.0009) [2023-12-27 03:11:13,996][105692] Updated weights for policy 0, policy_version 1618838 (0.0010) [2023-12-27 03:11:14,513][105620] Updated weights for policy 1, policy_version 1622181 (0.0009) [2023-12-27 03:11:14,567][105620] Updated weights for policy 1, policy_version 1622191 (0.0010) [2023-12-27 03:11:14,602][105692] Updated weights for policy 0, policy_version 1618848 (0.0006) [2023-12-27 03:11:14,624][105620] Updated weights for policy 1, policy_version 1622201 (0.0007) [2023-12-27 03:11:14,660][105692] Updated weights for policy 0, policy_version 1618858 (0.0005) [2023-12-27 03:11:14,719][105692] Updated weights for policy 0, policy_version 1618868 (0.0006) [2023-12-27 03:11:15,339][105620] Updated weights for policy 1, policy_version 1622211 (0.0008) [2023-12-27 03:11:15,403][105620] Updated weights for policy 1, policy_version 1622221 (0.0007) [2023-12-27 03:11:15,468][105692] Updated weights for policy 0, policy_version 1618878 (0.0007) [2023-12-27 03:11:15,468][105620] Updated weights for policy 1, policy_version 1622231 (0.0009) [2023-12-27 03:11:15,514][105692] Updated weights for policy 0, policy_version 1618888 (0.0008) [2023-12-27 03:11:15,570][105692] Updated weights for policy 0, policy_version 1618898 (0.0010) [2023-12-27 03:11:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.6, 300 sec: 19494.2). Total num frames: 829849600. Throughput: 0: 9747.9, 1: 9743.7. Samples: 829820884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:11:16,063][104569] Avg episode reward: [(0, '8986.715'), (1, '9077.740')] [2023-12-27 03:11:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001622240_415350784.pth... [2023-12-27 03:11:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001618904_414498816.pth... [2023-12-27 03:11:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001621120_415064064.pth [2023-12-27 03:11:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001617752_414203904.pth [2023-12-27 03:11:16,188][105620] Updated weights for policy 1, policy_version 1622241 (0.0008) [2023-12-27 03:11:16,240][105620] Updated weights for policy 1, policy_version 1622251 (0.0008) [2023-12-27 03:11:16,291][105620] Updated weights for policy 1, policy_version 1622261 (0.0008) [2023-12-27 03:11:16,334][105692] Updated weights for policy 0, policy_version 1618908 (0.0011) [2023-12-27 03:11:16,337][105620] Updated weights for policy 1, policy_version 1622271 (0.0007) [2023-12-27 03:11:16,382][105692] Updated weights for policy 0, policy_version 1618918 (0.0011) [2023-12-27 03:11:16,434][105692] Updated weights for policy 0, policy_version 1618928 (0.0010) [2023-12-27 03:11:17,121][105620] Updated weights for policy 1, policy_version 1622281 (0.0008) [2023-12-27 03:11:17,183][105620] Updated weights for policy 1, policy_version 1622291 (0.0008) [2023-12-27 03:11:17,194][105692] Updated weights for policy 0, policy_version 1618938 (0.0011) [2023-12-27 03:11:17,247][105620] Updated weights for policy 1, policy_version 1622301 (0.0006) [2023-12-27 03:11:17,249][105692] Updated weights for policy 0, policy_version 1618948 (0.0011) [2023-12-27 03:11:17,307][105692] Updated weights for policy 0, policy_version 1618958 (0.0010) [2023-12-27 03:11:17,367][105692] Updated weights for policy 0, policy_version 1618968 (0.0010) [2023-12-27 03:11:17,990][105620] Updated weights for policy 1, policy_version 1622311 (0.0007) [2023-12-27 03:11:18,052][105620] Updated weights for policy 1, policy_version 1622321 (0.0008) [2023-12-27 03:11:18,093][105692] Updated weights for policy 0, policy_version 1618978 (0.0010) [2023-12-27 03:11:18,114][105620] Updated weights for policy 1, policy_version 1622331 (0.0006) [2023-12-27 03:11:18,145][105692] Updated weights for policy 0, policy_version 1618988 (0.0010) [2023-12-27 03:11:18,198][105692] Updated weights for policy 0, policy_version 1618998 (0.0011) [2023-12-27 03:11:18,896][105620] Updated weights for policy 1, policy_version 1622341 (0.0006) [2023-12-27 03:11:18,952][105620] Updated weights for policy 1, policy_version 1622351 (0.0008) [2023-12-27 03:11:18,981][105692] Updated weights for policy 0, policy_version 1619008 (0.0011) [2023-12-27 03:11:19,007][105620] Updated weights for policy 1, policy_version 1622361 (0.0005) [2023-12-27 03:11:19,039][105692] Updated weights for policy 0, policy_version 1619018 (0.0010) [2023-12-27 03:11:19,098][105692] Updated weights for policy 0, policy_version 1619028 (0.0010) [2023-12-27 03:11:19,753][105620] Updated weights for policy 1, policy_version 1622371 (0.0007) [2023-12-27 03:11:19,813][105620] Updated weights for policy 1, policy_version 1622381 (0.0011) [2023-12-27 03:11:19,827][105692] Updated weights for policy 0, policy_version 1619038 (0.0010) [2023-12-27 03:11:19,876][105620] Updated weights for policy 1, policy_version 1622391 (0.0007) [2023-12-27 03:11:19,895][105692] Updated weights for policy 0, policy_version 1619048 (0.0011) [2023-12-27 03:11:19,965][105692] Updated weights for policy 0, policy_version 1619058 (0.0010) [2023-12-27 03:11:20,528][105620] Updated weights for policy 1, policy_version 1622401 (0.0007) [2023-12-27 03:11:20,593][105620] Updated weights for policy 1, policy_version 1622411 (0.0006) [2023-12-27 03:11:20,652][105620] Updated weights for policy 1, policy_version 1622421 (0.0006) [2023-12-27 03:11:20,717][105620] Updated weights for policy 1, policy_version 1622431 (0.0006) [2023-12-27 03:11:20,809][105692] Updated weights for policy 0, policy_version 1619068 (0.0009) [2023-12-27 03:11:20,862][105692] Updated weights for policy 0, policy_version 1619078 (0.0009) [2023-12-27 03:11:20,921][105692] Updated weights for policy 0, policy_version 1619088 (0.0005) [2023-12-27 03:11:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 829947904. Throughput: 0: 9766.0, 1: 9649.9. Samples: 829934900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:11:21,063][104569] Avg episode reward: [(0, '9078.044'), (1, '9169.313')] [2023-12-27 03:11:21,439][105620] Updated weights for policy 1, policy_version 1622441 (0.0008) [2023-12-27 03:11:21,500][105620] Updated weights for policy 1, policy_version 1622451 (0.0009) [2023-12-27 03:11:21,560][105620] Updated weights for policy 1, policy_version 1622461 (0.0008) [2023-12-27 03:11:21,679][105692] Updated weights for policy 0, policy_version 1619098 (0.0005) [2023-12-27 03:11:21,747][105692] Updated weights for policy 0, policy_version 1619108 (0.0008) [2023-12-27 03:11:21,819][105692] Updated weights for policy 0, policy_version 1619118 (0.0009) [2023-12-27 03:11:21,885][105692] Updated weights for policy 0, policy_version 1619128 (0.0009) [2023-12-27 03:11:22,397][105620] Updated weights for policy 1, policy_version 1622471 (0.0007) [2023-12-27 03:11:22,464][105620] Updated weights for policy 1, policy_version 1622481 (0.0010) [2023-12-27 03:11:22,525][105620] Updated weights for policy 1, policy_version 1622491 (0.0008) [2023-12-27 03:11:22,537][105692] Updated weights for policy 0, policy_version 1619138 (0.0008) [2023-12-27 03:11:22,599][105692] Updated weights for policy 0, policy_version 1619148 (0.0007) [2023-12-27 03:11:22,661][105692] Updated weights for policy 0, policy_version 1619158 (0.0010) [2023-12-27 03:11:23,264][105620] Updated weights for policy 1, policy_version 1622501 (0.0009) [2023-12-27 03:11:23,314][105620] Updated weights for policy 1, policy_version 1622511 (0.0009) [2023-12-27 03:11:23,370][105620] Updated weights for policy 1, policy_version 1622521 (0.0010) [2023-12-27 03:11:23,387][105692] Updated weights for policy 0, policy_version 1619168 (0.0008) [2023-12-27 03:11:23,435][105692] Updated weights for policy 0, policy_version 1619178 (0.0008) [2023-12-27 03:11:23,493][105692] Updated weights for policy 0, policy_version 1619188 (0.0009) [2023-12-27 03:11:23,990][105620] Updated weights for policy 1, policy_version 1622531 (0.0006) [2023-12-27 03:11:24,050][105620] Updated weights for policy 1, policy_version 1622541 (0.0005) [2023-12-27 03:11:24,102][105620] Updated weights for policy 1, policy_version 1622551 (0.0005) [2023-12-27 03:11:24,229][105692] Updated weights for policy 0, policy_version 1619198 (0.0007) [2023-12-27 03:11:24,286][105692] Updated weights for policy 0, policy_version 1619208 (0.0006) [2023-12-27 03:11:24,348][105692] Updated weights for policy 0, policy_version 1619218 (0.0008) [2023-12-27 03:11:24,786][105620] Updated weights for policy 1, policy_version 1622561 (0.0009) [2023-12-27 03:11:24,850][105620] Updated weights for policy 1, policy_version 1622571 (0.0009) [2023-12-27 03:11:24,910][105620] Updated weights for policy 1, policy_version 1622581 (0.0009) [2023-12-27 03:11:24,960][105620] Updated weights for policy 1, policy_version 1622591 (0.0009) [2023-12-27 03:11:25,055][105692] Updated weights for policy 0, policy_version 1619228 (0.0008) [2023-12-27 03:11:25,106][105692] Updated weights for policy 0, policy_version 1619238 (0.0009) [2023-12-27 03:11:25,161][105692] Updated weights for policy 0, policy_version 1619248 (0.0009) [2023-12-27 03:11:25,559][105620] Updated weights for policy 1, policy_version 1622601 (0.0006) [2023-12-27 03:11:25,618][105620] Updated weights for policy 1, policy_version 1622611 (0.0007) [2023-12-27 03:11:25,673][105620] Updated weights for policy 1, policy_version 1622621 (0.0010) [2023-12-27 03:11:26,015][105692] Updated weights for policy 0, policy_version 1619258 (0.0009) [2023-12-27 03:11:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 830038016. Throughput: 0: 9778.7, 1: 9632.6. Samples: 830050140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:11:26,063][104569] Avg episode reward: [(0, '8802.029'), (1, '9260.294')] [2023-12-27 03:11:26,071][105692] Updated weights for policy 0, policy_version 1619268 (0.0007) [2023-12-27 03:11:26,129][105692] Updated weights for policy 0, policy_version 1619278 (0.0007) [2023-12-27 03:11:26,193][105692] Updated weights for policy 0, policy_version 1619288 (0.0009) [2023-12-27 03:11:26,376][105620] Updated weights for policy 1, policy_version 1622631 (0.0009) [2023-12-27 03:11:26,438][105620] Updated weights for policy 1, policy_version 1622641 (0.0009) [2023-12-27 03:11:26,498][105620] Updated weights for policy 1, policy_version 1622651 (0.0008) [2023-12-27 03:11:26,945][105692] Updated weights for policy 0, policy_version 1619298 (0.0009) [2023-12-27 03:11:27,000][105692] Updated weights for policy 0, policy_version 1619308 (0.0009) [2023-12-27 03:11:27,050][105692] Updated weights for policy 0, policy_version 1619318 (0.0008) [2023-12-27 03:11:27,112][105620] Updated weights for policy 1, policy_version 1622661 (0.0007) [2023-12-27 03:11:27,172][105620] Updated weights for policy 1, policy_version 1622671 (0.0009) [2023-12-27 03:11:27,232][105620] Updated weights for policy 1, policy_version 1622682 (0.0009) [2023-12-27 03:11:27,749][105692] Updated weights for policy 0, policy_version 1619328 (0.0010) [2023-12-27 03:11:27,810][105692] Updated weights for policy 0, policy_version 1619338 (0.0010) [2023-12-27 03:11:27,858][105692] Updated weights for policy 0, policy_version 1619348 (0.0010) [2023-12-27 03:11:27,878][105620] Updated weights for policy 1, policy_version 1622692 (0.0007) [2023-12-27 03:11:27,931][105620] Updated weights for policy 1, policy_version 1622702 (0.0005) [2023-12-27 03:11:27,986][105620] Updated weights for policy 1, policy_version 1622712 (0.0005) [2023-12-27 03:11:28,510][105692] Updated weights for policy 0, policy_version 1619358 (0.0009) [2023-12-27 03:11:28,561][105692] Updated weights for policy 0, policy_version 1619368 (0.0008) [2023-12-27 03:11:28,613][105692] Updated weights for policy 0, policy_version 1619378 (0.0008) [2023-12-27 03:11:28,664][105620] Updated weights for policy 1, policy_version 1622722 (0.0007) [2023-12-27 03:11:28,728][105620] Updated weights for policy 1, policy_version 1622732 (0.0010) [2023-12-27 03:11:28,793][105620] Updated weights for policy 1, policy_version 1622742 (0.0007) [2023-12-27 03:11:28,848][105620] Updated weights for policy 1, policy_version 1622752 (0.0009) [2023-12-27 03:11:29,301][105692] Updated weights for policy 0, policy_version 1619388 (0.0008) [2023-12-27 03:11:29,369][105692] Updated weights for policy 0, policy_version 1619398 (0.0010) [2023-12-27 03:11:29,436][105692] Updated weights for policy 0, policy_version 1619408 (0.0010) [2023-12-27 03:11:29,517][105620] Updated weights for policy 1, policy_version 1622762 (0.0006) [2023-12-27 03:11:29,578][105620] Updated weights for policy 1, policy_version 1622772 (0.0005) [2023-12-27 03:11:29,640][105620] Updated weights for policy 1, policy_version 1622782 (0.0005) [2023-12-27 03:11:30,157][105692] Updated weights for policy 0, policy_version 1619418 (0.0010) [2023-12-27 03:11:30,214][105692] Updated weights for policy 0, policy_version 1619428 (0.0011) [2023-12-27 03:11:30,269][105692] Updated weights for policy 0, policy_version 1619438 (0.0010) [2023-12-27 03:11:30,317][105620] Updated weights for policy 1, policy_version 1622792 (0.0009) [2023-12-27 03:11:30,329][105692] Updated weights for policy 0, policy_version 1619448 (0.0007) [2023-12-27 03:11:30,381][105620] Updated weights for policy 1, policy_version 1622802 (0.0010) [2023-12-27 03:11:30,446][105620] Updated weights for policy 1, policy_version 1622812 (0.0011) [2023-12-27 03:11:30,938][105692] Updated weights for policy 0, policy_version 1619458 (0.0006) [2023-12-27 03:11:30,995][105692] Updated weights for policy 0, policy_version 1619468 (0.0007) [2023-12-27 03:11:31,052][105692] Updated weights for policy 0, policy_version 1619478 (0.0008) [2023-12-27 03:11:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 830136320. Throughput: 0: 9858.7, 1: 9652.2. Samples: 830110832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:11:31,063][104569] Avg episode reward: [(0, '8801.361'), (1, '9077.537')] [2023-12-27 03:11:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001619480_414646272.pth... [2023-12-27 03:11:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001622816_415498240.pth... [2023-12-27 03:11:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001618328_414351360.pth [2023-12-27 03:11:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001621728_415219712.pth [2023-12-27 03:11:31,156][105620] Updated weights for policy 1, policy_version 1622822 (0.0008) [2023-12-27 03:11:31,216][105620] Updated weights for policy 1, policy_version 1622832 (0.0006) [2023-12-27 03:11:31,274][105620] Updated weights for policy 1, policy_version 1622842 (0.0008) [2023-12-27 03:11:31,748][105692] Updated weights for policy 0, policy_version 1619488 (0.0008) [2023-12-27 03:11:31,803][105692] Updated weights for policy 0, policy_version 1619498 (0.0006) [2023-12-27 03:11:31,863][105692] Updated weights for policy 0, policy_version 1619508 (0.0005) [2023-12-27 03:11:31,976][105620] Updated weights for policy 1, policy_version 1622852 (0.0007) [2023-12-27 03:11:32,030][105620] Updated weights for policy 1, policy_version 1622862 (0.0010) [2023-12-27 03:11:32,084][105620] Updated weights for policy 1, policy_version 1622872 (0.0006) [2023-12-27 03:11:32,442][105692] Updated weights for policy 0, policy_version 1619518 (0.0007) [2023-12-27 03:11:32,501][105692] Updated weights for policy 0, policy_version 1619528 (0.0005) [2023-12-27 03:11:32,562][105692] Updated weights for policy 0, policy_version 1619538 (0.0005) [2023-12-27 03:11:32,913][105620] Updated weights for policy 1, policy_version 1622882 (0.0007) [2023-12-27 03:11:32,980][105620] Updated weights for policy 1, policy_version 1622892 (0.0010) [2023-12-27 03:11:33,042][105620] Updated weights for policy 1, policy_version 1622902 (0.0010) [2023-12-27 03:11:33,086][105620] Updated weights for policy 1, policy_version 1622912 (0.0008) [2023-12-27 03:11:33,096][105692] Updated weights for policy 0, policy_version 1619548 (0.0006) [2023-12-27 03:11:33,149][105692] Updated weights for policy 0, policy_version 1619558 (0.0006) [2023-12-27 03:11:33,199][105692] Updated weights for policy 0, policy_version 1619568 (0.0006) [2023-12-27 03:11:33,860][105692] Updated weights for policy 0, policy_version 1619578 (0.0006) [2023-12-27 03:11:33,862][105620] Updated weights for policy 1, policy_version 1622922 (0.0008) [2023-12-27 03:11:33,919][105620] Updated weights for policy 1, policy_version 1622932 (0.0008) [2023-12-27 03:11:33,922][105692] Updated weights for policy 0, policy_version 1619588 (0.0005) [2023-12-27 03:11:33,975][105620] Updated weights for policy 1, policy_version 1622942 (0.0007) [2023-12-27 03:11:33,985][105692] Updated weights for policy 0, policy_version 1619598 (0.0005) [2023-12-27 03:11:34,040][105692] Updated weights for policy 0, policy_version 1619608 (0.0005) [2023-12-27 03:11:34,641][105620] Updated weights for policy 1, policy_version 1622952 (0.0007) [2023-12-27 03:11:34,641][105692] Updated weights for policy 0, policy_version 1619618 (0.0009) [2023-12-27 03:11:34,701][105692] Updated weights for policy 0, policy_version 1619628 (0.0008) [2023-12-27 03:11:34,705][105620] Updated weights for policy 1, policy_version 1622962 (0.0006) [2023-12-27 03:11:34,766][105692] Updated weights for policy 0, policy_version 1619638 (0.0010) [2023-12-27 03:11:34,772][105620] Updated weights for policy 1, policy_version 1622972 (0.0006) [2023-12-27 03:11:35,313][105620] Updated weights for policy 1, policy_version 1622982 (0.0005) [2023-12-27 03:11:35,379][105620] Updated weights for policy 1, policy_version 1622992 (0.0005) [2023-12-27 03:11:35,447][105620] Updated weights for policy 1, policy_version 1623002 (0.0008) [2023-12-27 03:11:35,634][105692] Updated weights for policy 0, policy_version 1619648 (0.0006) [2023-12-27 03:11:35,682][105692] Updated weights for policy 0, policy_version 1619658 (0.0005) [2023-12-27 03:11:35,739][105692] Updated weights for policy 0, policy_version 1619668 (0.0005) [2023-12-27 03:11:36,011][105620] Updated weights for policy 1, policy_version 1623012 (0.0009) [2023-12-27 03:11:36,058][105620] Updated weights for policy 1, policy_version 1623022 (0.0009) [2023-12-27 03:11:36,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 830242816. Throughput: 0: 9921.8, 1: 9555.6. Samples: 830233756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:11:36,062][104569] Avg episode reward: [(0, '8711.894'), (1, '9170.443')] [2023-12-27 03:11:36,108][105620] Updated weights for policy 1, policy_version 1623032 (0.0009) [2023-12-27 03:11:36,326][105692] Updated weights for policy 0, policy_version 1619678 (0.0005) [2023-12-27 03:11:36,381][105692] Updated weights for policy 0, policy_version 1619688 (0.0007) [2023-12-27 03:11:36,436][105692] Updated weights for policy 0, policy_version 1619698 (0.0005) [2023-12-27 03:11:36,772][105620] Updated weights for policy 1, policy_version 1623042 (0.0009) [2023-12-27 03:11:36,825][105620] Updated weights for policy 1, policy_version 1623052 (0.0006) [2023-12-27 03:11:36,882][105620] Updated weights for policy 1, policy_version 1623062 (0.0005) [2023-12-27 03:11:36,946][105620] Updated weights for policy 1, policy_version 1623072 (0.0006) [2023-12-27 03:11:37,210][105692] Updated weights for policy 0, policy_version 1619708 (0.0007) [2023-12-27 03:11:37,267][105692] Updated weights for policy 0, policy_version 1619718 (0.0009) [2023-12-27 03:11:37,322][105692] Updated weights for policy 0, policy_version 1619728 (0.0010) [2023-12-27 03:11:37,515][105620] Updated weights for policy 1, policy_version 1623082 (0.0008) [2023-12-27 03:11:37,565][105620] Updated weights for policy 1, policy_version 1623092 (0.0009) [2023-12-27 03:11:37,616][105620] Updated weights for policy 1, policy_version 1623102 (0.0010) [2023-12-27 03:11:38,125][105692] Updated weights for policy 0, policy_version 1619738 (0.0009) [2023-12-27 03:11:38,188][105692] Updated weights for policy 0, policy_version 1619748 (0.0008) [2023-12-27 03:11:38,236][105692] Updated weights for policy 0, policy_version 1619758 (0.0008) [2023-12-27 03:11:38,299][105692] Updated weights for policy 0, policy_version 1619768 (0.0008) [2023-12-27 03:11:38,380][105620] Updated weights for policy 1, policy_version 1623112 (0.0011) [2023-12-27 03:11:38,439][105620] Updated weights for policy 1, policy_version 1623122 (0.0008) [2023-12-27 03:11:38,503][105620] Updated weights for policy 1, policy_version 1623132 (0.0006) [2023-12-27 03:11:39,125][105620] Updated weights for policy 1, policy_version 1623142 (0.0008) [2023-12-27 03:11:39,144][105692] Updated weights for policy 0, policy_version 1619778 (0.0005) [2023-12-27 03:11:39,180][105620] Updated weights for policy 1, policy_version 1623152 (0.0010) [2023-12-27 03:11:39,199][105692] Updated weights for policy 0, policy_version 1619788 (0.0009) [2023-12-27 03:11:39,243][105620] Updated weights for policy 1, policy_version 1623162 (0.0010) [2023-12-27 03:11:39,276][105692] Updated weights for policy 0, policy_version 1619798 (0.0008) [2023-12-27 03:11:39,987][105620] Updated weights for policy 1, policy_version 1623172 (0.0009) [2023-12-27 03:11:40,039][105692] Updated weights for policy 0, policy_version 1619808 (0.0010) [2023-12-27 03:11:40,047][105620] Updated weights for policy 1, policy_version 1623182 (0.0007) [2023-12-27 03:11:40,092][105692] Updated weights for policy 0, policy_version 1619818 (0.0009) [2023-12-27 03:11:40,099][105620] Updated weights for policy 1, policy_version 1623192 (0.0007) [2023-12-27 03:11:40,142][105692] Updated weights for policy 0, policy_version 1619828 (0.0008) [2023-12-27 03:11:40,864][105620] Updated weights for policy 1, policy_version 1623202 (0.0007) [2023-12-27 03:11:40,918][105620] Updated weights for policy 1, policy_version 1623212 (0.0007) [2023-12-27 03:11:40,974][105620] Updated weights for policy 1, policy_version 1623222 (0.0007) [2023-12-27 03:11:40,984][105692] Updated weights for policy 0, policy_version 1619838 (0.0009) [2023-12-27 03:11:41,033][105620] Updated weights for policy 1, policy_version 1623232 (0.0007) [2023-12-27 03:11:41,048][105692] Updated weights for policy 0, policy_version 1619848 (0.0007) [2023-12-27 03:11:41,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 830341120. Throughput: 0: 9845.1, 1: 9617.6. Samples: 830350676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:11:41,062][104569] Avg episode reward: [(0, '8624.751'), (1, '9262.166')] [2023-12-27 03:11:41,112][105692] Updated weights for policy 0, policy_version 1619858 (0.0010) [2023-12-27 03:11:41,827][105620] Updated weights for policy 1, policy_version 1623242 (0.0008) [2023-12-27 03:11:41,885][105620] Updated weights for policy 1, policy_version 1623252 (0.0008) [2023-12-27 03:11:41,931][105692] Updated weights for policy 0, policy_version 1619868 (0.0008) [2023-12-27 03:11:41,944][105620] Updated weights for policy 1, policy_version 1623262 (0.0009) [2023-12-27 03:11:41,995][105692] Updated weights for policy 0, policy_version 1619878 (0.0008) [2023-12-27 03:11:42,060][105692] Updated weights for policy 0, policy_version 1619888 (0.0010) [2023-12-27 03:11:42,639][105620] Updated weights for policy 1, policy_version 1623272 (0.0006) [2023-12-27 03:11:42,690][105620] Updated weights for policy 1, policy_version 1623282 (0.0005) [2023-12-27 03:11:42,738][105620] Updated weights for policy 1, policy_version 1623292 (0.0006) [2023-12-27 03:11:42,906][105692] Updated weights for policy 0, policy_version 1619898 (0.0009) [2023-12-27 03:11:42,977][105692] Updated weights for policy 0, policy_version 1619908 (0.0009) [2023-12-27 03:11:43,045][105692] Updated weights for policy 0, policy_version 1619918 (0.0009) [2023-12-27 03:11:43,098][105692] Updated weights for policy 0, policy_version 1619928 (0.0010) [2023-12-27 03:11:43,300][105620] Updated weights for policy 1, policy_version 1623302 (0.0008) [2023-12-27 03:11:43,353][105620] Updated weights for policy 1, policy_version 1623312 (0.0009) [2023-12-27 03:11:43,418][105620] Updated weights for policy 1, policy_version 1623322 (0.0010) [2023-12-27 03:11:43,879][105692] Updated weights for policy 0, policy_version 1619938 (0.0008) [2023-12-27 03:11:43,942][105692] Updated weights for policy 0, policy_version 1619948 (0.0008) [2023-12-27 03:11:44,008][105692] Updated weights for policy 0, policy_version 1619958 (0.0009) [2023-12-27 03:11:44,120][105620] Updated weights for policy 1, policy_version 1623332 (0.0008) [2023-12-27 03:11:44,188][105620] Updated weights for policy 1, policy_version 1623342 (0.0007) [2023-12-27 03:11:44,251][105620] Updated weights for policy 1, policy_version 1623352 (0.0010) [2023-12-27 03:11:44,848][105620] Updated weights for policy 1, policy_version 1623362 (0.0007) [2023-12-27 03:11:44,852][105692] Updated weights for policy 0, policy_version 1619968 (0.0009) [2023-12-27 03:11:44,910][105620] Updated weights for policy 1, policy_version 1623372 (0.0007) [2023-12-27 03:11:44,920][105692] Updated weights for policy 0, policy_version 1619978 (0.0007) [2023-12-27 03:11:44,974][105620] Updated weights for policy 1, policy_version 1623382 (0.0007) [2023-12-27 03:11:44,989][105692] Updated weights for policy 0, policy_version 1619988 (0.0008) [2023-12-27 03:11:45,049][105620] Updated weights for policy 1, policy_version 1623392 (0.0009) [2023-12-27 03:11:45,684][105620] Updated weights for policy 1, policy_version 1623402 (0.0005) [2023-12-27 03:11:45,733][105620] Updated weights for policy 1, policy_version 1623412 (0.0005) [2023-12-27 03:11:45,792][105620] Updated weights for policy 1, policy_version 1623422 (0.0006) [2023-12-27 03:11:45,838][105692] Updated weights for policy 0, policy_version 1619998 (0.0009) [2023-12-27 03:11:45,903][105692] Updated weights for policy 0, policy_version 1620008 (0.0009) [2023-12-27 03:11:45,961][105692] Updated weights for policy 0, policy_version 1620018 (0.0007) [2023-12-27 03:11:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 830439424. Throughput: 0: 9747.4, 1: 9620.4. Samples: 830406492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:11:46,062][104569] Avg episode reward: [(0, '8805.429'), (1, '9079.771')] [2023-12-27 03:11:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001620024_414785536.pth... [2023-12-27 03:11:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001623424_415653888.pth... [2023-12-27 03:11:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001622240_415350784.pth [2023-12-27 03:11:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001618904_414498816.pth [2023-12-27 03:11:46,372][105620] Updated weights for policy 1, policy_version 1623432 (0.0008) [2023-12-27 03:11:46,426][105620] Updated weights for policy 1, policy_version 1623442 (0.0010) [2023-12-27 03:11:46,484][105620] Updated weights for policy 1, policy_version 1623452 (0.0010) [2023-12-27 03:11:46,772][105692] Updated weights for policy 0, policy_version 1620028 (0.0008) [2023-12-27 03:11:46,834][105692] Updated weights for policy 0, policy_version 1620038 (0.0006) [2023-12-27 03:11:46,893][105692] Updated weights for policy 0, policy_version 1620048 (0.0009) [2023-12-27 03:11:47,127][105620] Updated weights for policy 1, policy_version 1623462 (0.0007) [2023-12-27 03:11:47,182][105620] Updated weights for policy 1, policy_version 1623472 (0.0008) [2023-12-27 03:11:47,227][105620] Updated weights for policy 1, policy_version 1623482 (0.0010) [2023-12-27 03:11:47,578][105692] Updated weights for policy 0, policy_version 1620058 (0.0009) [2023-12-27 03:11:47,626][105692] Updated weights for policy 0, policy_version 1620068 (0.0006) [2023-12-27 03:11:47,682][105692] Updated weights for policy 0, policy_version 1620078 (0.0007) [2023-12-27 03:11:47,733][105692] Updated weights for policy 0, policy_version 1620088 (0.0008) [2023-12-27 03:11:47,940][105620] Updated weights for policy 1, policy_version 1623492 (0.0008) [2023-12-27 03:11:47,990][105620] Updated weights for policy 1, policy_version 1623502 (0.0005) [2023-12-27 03:11:48,049][105620] Updated weights for policy 1, policy_version 1623512 (0.0005) [2023-12-27 03:11:48,476][105692] Updated weights for policy 0, policy_version 1620098 (0.0009) [2023-12-27 03:11:48,535][105692] Updated weights for policy 0, policy_version 1620108 (0.0010) [2023-12-27 03:11:48,591][105692] Updated weights for policy 0, policy_version 1620118 (0.0010) [2023-12-27 03:11:48,653][105620] Updated weights for policy 1, policy_version 1623522 (0.0006) [2023-12-27 03:11:48,713][105620] Updated weights for policy 1, policy_version 1623532 (0.0008) [2023-12-27 03:11:48,774][105620] Updated weights for policy 1, policy_version 1623542 (0.0006) [2023-12-27 03:11:48,833][105620] Updated weights for policy 1, policy_version 1623552 (0.0005) [2023-12-27 03:11:49,306][105692] Updated weights for policy 0, policy_version 1620128 (0.0008) [2023-12-27 03:11:49,373][105692] Updated weights for policy 0, policy_version 1620138 (0.0009) [2023-12-27 03:11:49,435][105692] Updated weights for policy 0, policy_version 1620148 (0.0009) [2023-12-27 03:11:49,523][105620] Updated weights for policy 1, policy_version 1623562 (0.0010) [2023-12-27 03:11:49,588][105620] Updated weights for policy 1, policy_version 1623572 (0.0010) [2023-12-27 03:11:49,644][105620] Updated weights for policy 1, policy_version 1623582 (0.0010) [2023-12-27 03:11:50,137][105692] Updated weights for policy 0, policy_version 1620158 (0.0011) [2023-12-27 03:11:50,196][105692] Updated weights for policy 0, policy_version 1620168 (0.0010) [2023-12-27 03:11:50,254][105692] Updated weights for policy 0, policy_version 1620178 (0.0011) [2023-12-27 03:11:50,362][105620] Updated weights for policy 1, policy_version 1623592 (0.0011) [2023-12-27 03:11:50,425][105620] Updated weights for policy 1, policy_version 1623602 (0.0010) [2023-12-27 03:11:50,482][105620] Updated weights for policy 1, policy_version 1623612 (0.0011) [2023-12-27 03:11:50,868][105692] Updated weights for policy 0, policy_version 1620188 (0.0009) [2023-12-27 03:11:50,935][105692] Updated weights for policy 0, policy_version 1620198 (0.0006) [2023-12-27 03:11:51,001][105692] Updated weights for policy 0, policy_version 1620208 (0.0009) [2023-12-27 03:11:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 830537728. Throughput: 0: 9642.2, 1: 9815.9. Samples: 830524520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:11:51,063][104569] Avg episode reward: [(0, '8986.132'), (1, '9079.531')] [2023-12-27 03:11:51,197][105620] Updated weights for policy 1, policy_version 1623622 (0.0009) [2023-12-27 03:11:51,260][105620] Updated weights for policy 1, policy_version 1623632 (0.0011) [2023-12-27 03:11:51,321][105620] Updated weights for policy 1, policy_version 1623642 (0.0009) [2023-12-27 03:11:51,684][105692] Updated weights for policy 0, policy_version 1620218 (0.0010) [2023-12-27 03:11:51,749][105692] Updated weights for policy 0, policy_version 1620228 (0.0011) [2023-12-27 03:11:51,811][105692] Updated weights for policy 0, policy_version 1620238 (0.0010) [2023-12-27 03:11:51,874][105692] Updated weights for policy 0, policy_version 1620248 (0.0011) [2023-12-27 03:11:52,057][105620] Updated weights for policy 1, policy_version 1623652 (0.0008) [2023-12-27 03:11:52,112][105620] Updated weights for policy 1, policy_version 1623662 (0.0006) [2023-12-27 03:11:52,165][105620] Updated weights for policy 1, policy_version 1623672 (0.0006) [2023-12-27 03:11:52,626][105692] Updated weights for policy 0, policy_version 1620258 (0.0010) [2023-12-27 03:11:52,685][105692] Updated weights for policy 0, policy_version 1620268 (0.0009) [2023-12-27 03:11:52,742][105692] Updated weights for policy 0, policy_version 1620278 (0.0006) [2023-12-27 03:11:52,813][105620] Updated weights for policy 1, policy_version 1623682 (0.0007) [2023-12-27 03:11:52,884][105620] Updated weights for policy 1, policy_version 1623692 (0.0009) [2023-12-27 03:11:52,945][105620] Updated weights for policy 1, policy_version 1623702 (0.0011) [2023-12-27 03:11:53,016][105620] Updated weights for policy 1, policy_version 1623712 (0.0011) [2023-12-27 03:11:53,304][105692] Updated weights for policy 0, policy_version 1620288 (0.0007) [2023-12-27 03:11:53,349][105692] Updated weights for policy 0, policy_version 1620298 (0.0009) [2023-12-27 03:11:53,402][105692] Updated weights for policy 0, policy_version 1620308 (0.0005) [2023-12-27 03:11:53,660][105620] Updated weights for policy 1, policy_version 1623722 (0.0010) [2023-12-27 03:11:53,727][105620] Updated weights for policy 1, policy_version 1623732 (0.0010) [2023-12-27 03:11:53,784][105620] Updated weights for policy 1, policy_version 1623742 (0.0010) [2023-12-27 03:11:54,075][105692] Updated weights for policy 0, policy_version 1620318 (0.0008) [2023-12-27 03:11:54,134][105692] Updated weights for policy 0, policy_version 1620328 (0.0010) [2023-12-27 03:11:54,207][105692] Updated weights for policy 0, policy_version 1620338 (0.0010) [2023-12-27 03:11:54,532][105620] Updated weights for policy 1, policy_version 1623752 (0.0010) [2023-12-27 03:11:54,593][105620] Updated weights for policy 1, policy_version 1623762 (0.0010) [2023-12-27 03:11:54,654][105620] Updated weights for policy 1, policy_version 1623772 (0.0010) [2023-12-27 03:11:54,980][105692] Updated weights for policy 0, policy_version 1620348 (0.0008) [2023-12-27 03:11:55,040][105692] Updated weights for policy 0, policy_version 1620358 (0.0005) [2023-12-27 03:11:55,095][105692] Updated weights for policy 0, policy_version 1620368 (0.0005) [2023-12-27 03:11:55,395][105620] Updated weights for policy 1, policy_version 1623782 (0.0007) [2023-12-27 03:11:55,448][105620] Updated weights for policy 1, policy_version 1623792 (0.0005) [2023-12-27 03:11:55,494][105620] Updated weights for policy 1, policy_version 1623802 (0.0005) [2023-12-27 03:11:55,712][105692] Updated weights for policy 0, policy_version 1620378 (0.0007) [2023-12-27 03:11:55,771][105692] Updated weights for policy 0, policy_version 1620388 (0.0011) [2023-12-27 03:11:55,823][105692] Updated weights for policy 0, policy_version 1620398 (0.0010) [2023-12-27 03:11:55,875][105692] Updated weights for policy 0, policy_version 1620408 (0.0009) [2023-12-27 03:11:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 830636032. Throughput: 0: 9594.0, 1: 9962.4. Samples: 830643992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:11:56,062][104569] Avg episode reward: [(0, '9077.907'), (1, '8985.017')] [2023-12-27 03:11:56,274][105620] Updated weights for policy 1, policy_version 1623812 (0.0007) [2023-12-27 03:11:56,329][105620] Updated weights for policy 1, policy_version 1623822 (0.0008) [2023-12-27 03:11:56,379][105620] Updated weights for policy 1, policy_version 1623832 (0.0006) [2023-12-27 03:11:56,457][105692] Updated weights for policy 0, policy_version 1620418 (0.0005) [2023-12-27 03:11:56,503][105692] Updated weights for policy 0, policy_version 1620428 (0.0005) [2023-12-27 03:11:56,551][105692] Updated weights for policy 0, policy_version 1620438 (0.0005) [2023-12-27 03:11:56,950][105620] Updated weights for policy 1, policy_version 1623842 (0.0006) [2023-12-27 03:11:57,001][105620] Updated weights for policy 1, policy_version 1623852 (0.0008) [2023-12-27 03:11:57,045][105620] Updated weights for policy 1, policy_version 1623862 (0.0008) [2023-12-27 03:11:57,094][105620] Updated weights for policy 1, policy_version 1623872 (0.0008) [2023-12-27 03:11:57,265][105692] Updated weights for policy 0, policy_version 1620448 (0.0009) [2023-12-27 03:11:57,321][105692] Updated weights for policy 0, policy_version 1620458 (0.0008) [2023-12-27 03:11:57,377][105692] Updated weights for policy 0, policy_version 1620468 (0.0008) [2023-12-27 03:11:57,813][105620] Updated weights for policy 1, policy_version 1623883 (0.0009) [2023-12-27 03:11:57,860][105620] Updated weights for policy 1, policy_version 1623894 (0.0007) [2023-12-27 03:11:57,905][105620] Updated weights for policy 1, policy_version 1623904 (0.0005) [2023-12-27 03:11:57,964][105692] Updated weights for policy 0, policy_version 1620478 (0.0010) [2023-12-27 03:11:58,019][105692] Updated weights for policy 0, policy_version 1620488 (0.0010) [2023-12-27 03:11:58,073][105692] Updated weights for policy 0, policy_version 1620498 (0.0010) [2023-12-27 03:11:58,622][105620] Updated weights for policy 1, policy_version 1623914 (0.0008) [2023-12-27 03:11:58,677][105620] Updated weights for policy 1, policy_version 1623924 (0.0008) [2023-12-27 03:11:58,738][105620] Updated weights for policy 1, policy_version 1623934 (0.0008) [2023-12-27 03:11:58,852][105692] Updated weights for policy 0, policy_version 1620508 (0.0009) [2023-12-27 03:11:58,924][105692] Updated weights for policy 0, policy_version 1620518 (0.0008) [2023-12-27 03:11:58,985][105692] Updated weights for policy 0, policy_version 1620528 (0.0008) [2023-12-27 03:11:59,594][105620] Updated weights for policy 1, policy_version 1623944 (0.0006) [2023-12-27 03:11:59,645][105620] Updated weights for policy 1, policy_version 1623954 (0.0005) [2023-12-27 03:11:59,701][105620] Updated weights for policy 1, policy_version 1623964 (0.0006) [2023-12-27 03:11:59,806][105692] Updated weights for policy 0, policy_version 1620538 (0.0009) [2023-12-27 03:11:59,864][105692] Updated weights for policy 0, policy_version 1620548 (0.0010) [2023-12-27 03:11:59,931][105692] Updated weights for policy 0, policy_version 1620558 (0.0010) [2023-12-27 03:11:59,994][105692] Updated weights for policy 0, policy_version 1620568 (0.0010) [2023-12-27 03:12:00,456][105620] Updated weights for policy 1, policy_version 1623974 (0.0009) [2023-12-27 03:12:00,518][105620] Updated weights for policy 1, policy_version 1623984 (0.0010) [2023-12-27 03:12:00,578][105620] Updated weights for policy 1, policy_version 1623994 (0.0009) [2023-12-27 03:12:00,630][105692] Updated weights for policy 0, policy_version 1620578 (0.0010) [2023-12-27 03:12:00,684][105692] Updated weights for policy 0, policy_version 1620588 (0.0010) [2023-12-27 03:12:00,746][105692] Updated weights for policy 0, policy_version 1620598 (0.0010) [2023-12-27 03:12:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 830734336. Throughput: 0: 9662.6, 1: 10003.9. Samples: 830705872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:12:01,062][104569] Avg episode reward: [(0, '8716.916'), (1, '8892.401')] [2023-12-27 03:12:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001620600_414932992.pth... [2023-12-27 03:12:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001624000_415801344.pth... [2023-12-27 03:12:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001622816_415498240.pth [2023-12-27 03:12:01,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001619480_414646272.pth [2023-12-27 03:12:01,352][105620] Updated weights for policy 1, policy_version 1624004 (0.0008) [2023-12-27 03:12:01,421][105620] Updated weights for policy 1, policy_version 1624014 (0.0012) [2023-12-27 03:12:01,448][105692] Updated weights for policy 0, policy_version 1620608 (0.0010) [2023-12-27 03:12:01,481][105620] Updated weights for policy 1, policy_version 1624024 (0.0011) [2023-12-27 03:12:01,500][105692] Updated weights for policy 0, policy_version 1620618 (0.0010) [2023-12-27 03:12:01,552][105692] Updated weights for policy 0, policy_version 1620628 (0.0010) [2023-12-27 03:12:02,120][105620] Updated weights for policy 1, policy_version 1624034 (0.0008) [2023-12-27 03:12:02,167][105620] Updated weights for policy 1, policy_version 1624044 (0.0008) [2023-12-27 03:12:02,219][105620] Updated weights for policy 1, policy_version 1624054 (0.0007) [2023-12-27 03:12:02,277][105620] Updated weights for policy 1, policy_version 1624064 (0.0006) [2023-12-27 03:12:02,283][105692] Updated weights for policy 0, policy_version 1620638 (0.0010) [2023-12-27 03:12:02,341][105692] Updated weights for policy 0, policy_version 1620648 (0.0010) [2023-12-27 03:12:02,403][105692] Updated weights for policy 0, policy_version 1620658 (0.0010) [2023-12-27 03:12:02,913][105620] Updated weights for policy 1, policy_version 1624074 (0.0005) [2023-12-27 03:12:02,966][105620] Updated weights for policy 1, policy_version 1624084 (0.0005) [2023-12-27 03:12:03,019][105620] Updated weights for policy 1, policy_version 1624094 (0.0005) [2023-12-27 03:12:03,188][105692] Updated weights for policy 0, policy_version 1620668 (0.0008) [2023-12-27 03:12:03,255][105692] Updated weights for policy 0, policy_version 1620678 (0.0005) [2023-12-27 03:12:03,322][105692] Updated weights for policy 0, policy_version 1620689 (0.0008) [2023-12-27 03:12:03,546][105620] Updated weights for policy 1, policy_version 1624104 (0.0005) [2023-12-27 03:12:03,597][105620] Updated weights for policy 1, policy_version 1624114 (0.0005) [2023-12-27 03:12:03,642][105620] Updated weights for policy 1, policy_version 1624124 (0.0005) [2023-12-27 03:12:04,035][105692] Updated weights for policy 0, policy_version 1620699 (0.0007) [2023-12-27 03:12:04,084][105692] Updated weights for policy 0, policy_version 1620709 (0.0005) [2023-12-27 03:12:04,135][105692] Updated weights for policy 0, policy_version 1620719 (0.0005) [2023-12-27 03:12:04,299][105620] Updated weights for policy 1, policy_version 1624134 (0.0008) [2023-12-27 03:12:04,358][105620] Updated weights for policy 1, policy_version 1624144 (0.0011) [2023-12-27 03:12:04,420][105620] Updated weights for policy 1, policy_version 1624154 (0.0010) [2023-12-27 03:12:04,827][105692] Updated weights for policy 0, policy_version 1620729 (0.0007) [2023-12-27 03:12:04,894][105692] Updated weights for policy 0, policy_version 1620739 (0.0005) [2023-12-27 03:12:04,954][105692] Updated weights for policy 0, policy_version 1620749 (0.0006) [2023-12-27 03:12:05,013][105692] Updated weights for policy 0, policy_version 1620759 (0.0010) [2023-12-27 03:12:05,110][105620] Updated weights for policy 1, policy_version 1624164 (0.0008) [2023-12-27 03:12:05,159][105620] Updated weights for policy 1, policy_version 1624174 (0.0005) [2023-12-27 03:12:05,227][105620] Updated weights for policy 1, policy_version 1624184 (0.0005) [2023-12-27 03:12:05,561][105692] Updated weights for policy 0, policy_version 1620769 (0.0010) [2023-12-27 03:12:05,616][105692] Updated weights for policy 0, policy_version 1620779 (0.0010) [2023-12-27 03:12:05,681][105692] Updated weights for policy 0, policy_version 1620789 (0.0010) [2023-12-27 03:12:05,739][105620] Updated weights for policy 1, policy_version 1624194 (0.0005) [2023-12-27 03:12:05,795][105620] Updated weights for policy 1, policy_version 1624204 (0.0005) [2023-12-27 03:12:05,851][105620] Updated weights for policy 1, policy_version 1624214 (0.0005) [2023-12-27 03:12:05,904][105620] Updated weights for policy 1, policy_version 1624224 (0.0009) [2023-12-27 03:12:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 830840832. Throughput: 0: 9644.9, 1: 10119.6. Samples: 830824300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:12:06,062][104569] Avg episode reward: [(0, '8804.480'), (1, '8988.506')] [2023-12-27 03:12:06,438][105692] Updated weights for policy 0, policy_version 1620799 (0.0011) [2023-12-27 03:12:06,506][105692] Updated weights for policy 0, policy_version 1620809 (0.0011) [2023-12-27 03:12:06,550][105620] Updated weights for policy 1, policy_version 1624234 (0.0011) [2023-12-27 03:12:06,563][105692] Updated weights for policy 0, policy_version 1620819 (0.0011) [2023-12-27 03:12:06,611][105620] Updated weights for policy 1, policy_version 1624244 (0.0011) [2023-12-27 03:12:06,668][105620] Updated weights for policy 1, policy_version 1624254 (0.0011) [2023-12-27 03:12:07,307][105692] Updated weights for policy 0, policy_version 1620829 (0.0008) [2023-12-27 03:12:07,360][105692] Updated weights for policy 0, policy_version 1620839 (0.0006) [2023-12-27 03:12:07,416][105692] Updated weights for policy 0, policy_version 1620849 (0.0007) [2023-12-27 03:12:07,425][105620] Updated weights for policy 1, policy_version 1624264 (0.0008) [2023-12-27 03:12:07,478][105620] Updated weights for policy 1, policy_version 1624274 (0.0007) [2023-12-27 03:12:07,533][105620] Updated weights for policy 1, policy_version 1624284 (0.0005) [2023-12-27 03:12:08,098][105692] Updated weights for policy 0, policy_version 1620859 (0.0007) [2023-12-27 03:12:08,151][105692] Updated weights for policy 0, policy_version 1620869 (0.0008) [2023-12-27 03:12:08,200][105620] Updated weights for policy 1, policy_version 1624294 (0.0005) [2023-12-27 03:12:08,216][105692] Updated weights for policy 0, policy_version 1620879 (0.0009) [2023-12-27 03:12:08,248][105620] Updated weights for policy 1, policy_version 1624304 (0.0007) [2023-12-27 03:12:08,295][105620] Updated weights for policy 1, policy_version 1624314 (0.0010) [2023-12-27 03:12:08,995][105692] Updated weights for policy 0, policy_version 1620889 (0.0007) [2023-12-27 03:12:09,043][105620] Updated weights for policy 1, policy_version 1624324 (0.0010) [2023-12-27 03:12:09,050][105692] Updated weights for policy 0, policy_version 1620899 (0.0010) [2023-12-27 03:12:09,103][105620] Updated weights for policy 1, policy_version 1624334 (0.0011) [2023-12-27 03:12:09,110][105692] Updated weights for policy 0, policy_version 1620909 (0.0011) [2023-12-27 03:12:09,163][105620] Updated weights for policy 1, policy_version 1624344 (0.0011) [2023-12-27 03:12:09,174][105692] Updated weights for policy 0, policy_version 1620919 (0.0011) [2023-12-27 03:12:09,863][105692] Updated weights for policy 0, policy_version 1620929 (0.0007) [2023-12-27 03:12:09,935][105692] Updated weights for policy 0, policy_version 1620939 (0.0009) [2023-12-27 03:12:09,942][105620] Updated weights for policy 1, policy_version 1624354 (0.0010) [2023-12-27 03:12:10,001][105692] Updated weights for policy 0, policy_version 1620949 (0.0006) [2023-12-27 03:12:10,003][105620] Updated weights for policy 1, policy_version 1624364 (0.0008) [2023-12-27 03:12:10,072][105620] Updated weights for policy 1, policy_version 1624374 (0.0008) [2023-12-27 03:12:10,131][105620] Updated weights for policy 1, policy_version 1624384 (0.0010) [2023-12-27 03:12:10,577][105692] Updated weights for policy 0, policy_version 1620959 (0.0007) [2023-12-27 03:12:10,642][105692] Updated weights for policy 0, policy_version 1620969 (0.0008) [2023-12-27 03:12:10,710][105692] Updated weights for policy 0, policy_version 1620979 (0.0009) [2023-12-27 03:12:10,905][105620] Updated weights for policy 1, policy_version 1624394 (0.0010) [2023-12-27 03:12:10,965][105620] Updated weights for policy 1, policy_version 1624405 (0.0010) [2023-12-27 03:12:11,017][105620] Updated weights for policy 1, policy_version 1624415 (0.0008) [2023-12-27 03:12:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 830939136. Throughput: 0: 9750.6, 1: 10116.5. Samples: 830944156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:12:11,062][104569] Avg episode reward: [(0, '8894.591'), (1, '9083.421')] [2023-12-27 03:12:11,289][105692] Updated weights for policy 0, policy_version 1620989 (0.0007) [2023-12-27 03:12:11,358][105692] Updated weights for policy 0, policy_version 1620999 (0.0008) [2023-12-27 03:12:11,424][105692] Updated weights for policy 0, policy_version 1621009 (0.0006) [2023-12-27 03:12:11,884][105620] Updated weights for policy 1, policy_version 1624425 (0.0008) [2023-12-27 03:12:11,951][105620] Updated weights for policy 1, policy_version 1624435 (0.0006) [2023-12-27 03:12:12,013][105620] Updated weights for policy 1, policy_version 1624445 (0.0005) [2023-12-27 03:12:12,224][105692] Updated weights for policy 0, policy_version 1621019 (0.0007) [2023-12-27 03:12:12,283][105692] Updated weights for policy 0, policy_version 1621029 (0.0010) [2023-12-27 03:12:12,338][105692] Updated weights for policy 0, policy_version 1621039 (0.0009) [2023-12-27 03:12:12,594][105620] Updated weights for policy 1, policy_version 1624455 (0.0008) [2023-12-27 03:12:12,645][105620] Updated weights for policy 1, policy_version 1624465 (0.0009) [2023-12-27 03:12:12,700][105620] Updated weights for policy 1, policy_version 1624475 (0.0009) [2023-12-27 03:12:13,174][105692] Updated weights for policy 0, policy_version 1621049 (0.0009) [2023-12-27 03:12:13,225][105692] Updated weights for policy 0, policy_version 1621059 (0.0009) [2023-12-27 03:12:13,272][105692] Updated weights for policy 0, policy_version 1621069 (0.0009) [2023-12-27 03:12:13,318][105692] Updated weights for policy 0, policy_version 1621079 (0.0008) [2023-12-27 03:12:13,366][105620] Updated weights for policy 1, policy_version 1624485 (0.0009) [2023-12-27 03:12:13,417][105620] Updated weights for policy 1, policy_version 1624495 (0.0009) [2023-12-27 03:12:13,471][105620] Updated weights for policy 1, policy_version 1624505 (0.0009) [2023-12-27 03:12:14,091][105692] Updated weights for policy 0, policy_version 1621089 (0.0009) [2023-12-27 03:12:14,153][105692] Updated weights for policy 0, policy_version 1621099 (0.0008) [2023-12-27 03:12:14,213][105692] Updated weights for policy 0, policy_version 1621109 (0.0008) [2023-12-27 03:12:14,245][105620] Updated weights for policy 1, policy_version 1624515 (0.0008) [2023-12-27 03:12:14,308][105620] Updated weights for policy 1, policy_version 1624525 (0.0008) [2023-12-27 03:12:14,376][105620] Updated weights for policy 1, policy_version 1624535 (0.0009) [2023-12-27 03:12:14,820][105692] Updated weights for policy 0, policy_version 1621119 (0.0008) [2023-12-27 03:12:14,888][105692] Updated weights for policy 0, policy_version 1621129 (0.0006) [2023-12-27 03:12:14,956][105692] Updated weights for policy 0, policy_version 1621139 (0.0006) [2023-12-27 03:12:15,211][105620] Updated weights for policy 1, policy_version 1624545 (0.0009) [2023-12-27 03:12:15,278][105620] Updated weights for policy 1, policy_version 1624555 (0.0008) [2023-12-27 03:12:15,343][105620] Updated weights for policy 1, policy_version 1624565 (0.0009) [2023-12-27 03:12:15,411][105620] Updated weights for policy 1, policy_version 1624575 (0.0008) [2023-12-27 03:12:15,593][105692] Updated weights for policy 0, policy_version 1621149 (0.0008) [2023-12-27 03:12:15,653][105692] Updated weights for policy 0, policy_version 1621159 (0.0011) [2023-12-27 03:12:15,719][105692] Updated weights for policy 0, policy_version 1621169 (0.0011) [2023-12-27 03:12:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.9, 300 sec: 19494.2). Total num frames: 831029248. Throughput: 0: 9710.9, 1: 10079.8. Samples: 831001408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:12:16,062][104569] Avg episode reward: [(0, '8894.588'), (1, '9081.441')] [2023-12-27 03:12:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001624576_415948800.pth... [2023-12-27 03:12:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001621176_415080448.pth... [2023-12-27 03:12:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001620024_414785536.pth [2023-12-27 03:12:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001623424_415653888.pth [2023-12-27 03:12:16,212][105620] Updated weights for policy 1, policy_version 1624585 (0.0010) [2023-12-27 03:12:16,275][105620] Updated weights for policy 1, policy_version 1624595 (0.0008) [2023-12-27 03:12:16,304][105692] Updated weights for policy 0, policy_version 1621179 (0.0009) [2023-12-27 03:12:16,339][105620] Updated weights for policy 1, policy_version 1624605 (0.0006) [2023-12-27 03:12:16,353][105692] Updated weights for policy 0, policy_version 1621189 (0.0008) [2023-12-27 03:12:16,420][105692] Updated weights for policy 0, policy_version 1621199 (0.0006) [2023-12-27 03:12:17,060][105692] Updated weights for policy 0, policy_version 1621209 (0.0006) [2023-12-27 03:12:17,122][105692] Updated weights for policy 0, policy_version 1621219 (0.0010) [2023-12-27 03:12:17,144][105620] Updated weights for policy 1, policy_version 1624615 (0.0006) [2023-12-27 03:12:17,184][105692] Updated weights for policy 0, policy_version 1621229 (0.0010) [2023-12-27 03:12:17,188][105620] Updated weights for policy 1, policy_version 1624625 (0.0008) [2023-12-27 03:12:17,233][105620] Updated weights for policy 1, policy_version 1624635 (0.0007) [2023-12-27 03:12:17,246][105692] Updated weights for policy 0, policy_version 1621239 (0.0010) [2023-12-27 03:12:17,933][105620] Updated weights for policy 1, policy_version 1624645 (0.0008) [2023-12-27 03:12:17,936][105692] Updated weights for policy 0, policy_version 1621249 (0.0006) [2023-12-27 03:12:17,981][105620] Updated weights for policy 1, policy_version 1624655 (0.0008) [2023-12-27 03:12:17,987][105692] Updated weights for policy 0, policy_version 1621259 (0.0005) [2023-12-27 03:12:18,046][105620] Updated weights for policy 1, policy_version 1624665 (0.0009) [2023-12-27 03:12:18,047][105692] Updated weights for policy 0, policy_version 1621269 (0.0005) [2023-12-27 03:12:18,655][105692] Updated weights for policy 0, policy_version 1621279 (0.0009) [2023-12-27 03:12:18,706][105692] Updated weights for policy 0, policy_version 1621289 (0.0010) [2023-12-27 03:12:18,766][105692] Updated weights for policy 0, policy_version 1621299 (0.0011) [2023-12-27 03:12:18,831][105620] Updated weights for policy 1, policy_version 1624675 (0.0010) [2023-12-27 03:12:18,893][105620] Updated weights for policy 1, policy_version 1624685 (0.0009) [2023-12-27 03:12:18,957][105620] Updated weights for policy 1, policy_version 1624695 (0.0010) [2023-12-27 03:12:19,517][105692] Updated weights for policy 0, policy_version 1621309 (0.0011) [2023-12-27 03:12:19,570][105692] Updated weights for policy 0, policy_version 1621319 (0.0011) [2023-12-27 03:12:19,621][105620] Updated weights for policy 1, policy_version 1624705 (0.0009) [2023-12-27 03:12:19,633][105692] Updated weights for policy 0, policy_version 1621329 (0.0009) [2023-12-27 03:12:19,683][105620] Updated weights for policy 1, policy_version 1624715 (0.0010) [2023-12-27 03:12:19,745][105620] Updated weights for policy 1, policy_version 1624725 (0.0011) [2023-12-27 03:12:19,813][105620] Updated weights for policy 1, policy_version 1624735 (0.0010) [2023-12-27 03:12:20,371][105692] Updated weights for policy 0, policy_version 1621339 (0.0007) [2023-12-27 03:12:20,440][105692] Updated weights for policy 0, policy_version 1621349 (0.0008) [2023-12-27 03:12:20,468][105620] Updated weights for policy 1, policy_version 1624745 (0.0010) [2023-12-27 03:12:20,499][105692] Updated weights for policy 0, policy_version 1621359 (0.0009) [2023-12-27 03:12:20,521][105620] Updated weights for policy 1, policy_version 1624755 (0.0010) [2023-12-27 03:12:20,610][105620] Updated weights for policy 1, policy_version 1624765 (0.0011) [2023-12-27 03:12:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 831127552. Throughput: 0: 9659.4, 1: 9998.6. Samples: 831118372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:12:21,063][104569] Avg episode reward: [(0, '8804.496'), (1, '8892.991')] [2023-12-27 03:12:21,295][105692] Updated weights for policy 0, policy_version 1621369 (0.0006) [2023-12-27 03:12:21,336][105620] Updated weights for policy 1, policy_version 1624775 (0.0010) [2023-12-27 03:12:21,360][105692] Updated weights for policy 0, policy_version 1621379 (0.0009) [2023-12-27 03:12:21,403][105620] Updated weights for policy 1, policy_version 1624785 (0.0009) [2023-12-27 03:12:21,423][105692] Updated weights for policy 0, policy_version 1621389 (0.0009) [2023-12-27 03:12:21,460][105620] Updated weights for policy 1, policy_version 1624795 (0.0011) [2023-12-27 03:12:21,480][105692] Updated weights for policy 0, policy_version 1621399 (0.0009) [2023-12-27 03:12:22,259][105620] Updated weights for policy 1, policy_version 1624805 (0.0008) [2023-12-27 03:12:22,261][105692] Updated weights for policy 0, policy_version 1621409 (0.0007) [2023-12-27 03:12:22,318][105620] Updated weights for policy 1, policy_version 1624815 (0.0008) [2023-12-27 03:12:22,323][105692] Updated weights for policy 0, policy_version 1621419 (0.0007) [2023-12-27 03:12:22,381][105620] Updated weights for policy 1, policy_version 1624825 (0.0009) [2023-12-27 03:12:22,390][105692] Updated weights for policy 0, policy_version 1621429 (0.0007) [2023-12-27 03:12:23,090][105620] Updated weights for policy 1, policy_version 1624835 (0.0007) [2023-12-27 03:12:23,159][105620] Updated weights for policy 1, policy_version 1624845 (0.0008) [2023-12-27 03:12:23,193][105692] Updated weights for policy 0, policy_version 1621439 (0.0007) [2023-12-27 03:12:23,222][105620] Updated weights for policy 1, policy_version 1624855 (0.0006) [2023-12-27 03:12:23,255][105692] Updated weights for policy 0, policy_version 1621449 (0.0007) [2023-12-27 03:12:23,318][105692] Updated weights for policy 0, policy_version 1621459 (0.0008) [2023-12-27 03:12:23,781][105620] Updated weights for policy 1, policy_version 1624865 (0.0007) [2023-12-27 03:12:23,837][105620] Updated weights for policy 1, policy_version 1624875 (0.0006) [2023-12-27 03:12:23,893][105620] Updated weights for policy 1, policy_version 1624885 (0.0010) [2023-12-27 03:12:23,938][105620] Updated weights for policy 1, policy_version 1624895 (0.0010) [2023-12-27 03:12:24,073][105692] Updated weights for policy 0, policy_version 1621469 (0.0007) [2023-12-27 03:12:24,120][105692] Updated weights for policy 0, policy_version 1621479 (0.0007) [2023-12-27 03:12:24,181][105692] Updated weights for policy 0, policy_version 1621489 (0.0006) [2023-12-27 03:12:24,574][105620] Updated weights for policy 1, policy_version 1624905 (0.0009) [2023-12-27 03:12:24,631][105620] Updated weights for policy 1, policy_version 1624915 (0.0006) [2023-12-27 03:12:24,682][105620] Updated weights for policy 1, policy_version 1624925 (0.0006) [2023-12-27 03:12:24,729][105692] Updated weights for policy 0, policy_version 1621499 (0.0007) [2023-12-27 03:12:24,775][105692] Updated weights for policy 0, policy_version 1621509 (0.0007) [2023-12-27 03:12:24,821][105692] Updated weights for policy 0, policy_version 1621519 (0.0005) [2023-12-27 03:12:25,287][105620] Updated weights for policy 1, policy_version 1624935 (0.0008) [2023-12-27 03:12:25,358][105620] Updated weights for policy 1, policy_version 1624945 (0.0009) [2023-12-27 03:12:25,424][105620] Updated weights for policy 1, policy_version 1624955 (0.0007) [2023-12-27 03:12:25,437][105692] Updated weights for policy 0, policy_version 1621530 (0.0008) [2023-12-27 03:12:25,489][105692] Updated weights for policy 0, policy_version 1621540 (0.0010) [2023-12-27 03:12:25,540][105692] Updated weights for policy 0, policy_version 1621550 (0.0010) [2023-12-27 03:12:25,591][105692] Updated weights for policy 0, policy_version 1621560 (0.0010) [2023-12-27 03:12:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.4, 300 sec: 19522.0). Total num frames: 831225856. Throughput: 0: 9738.8, 1: 9963.7. Samples: 831237292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:12:26,063][104569] Avg episode reward: [(0, '8253.731'), (1, '8983.890')] [2023-12-27 03:12:26,111][105620] Updated weights for policy 1, policy_version 1624965 (0.0005) [2023-12-27 03:12:26,164][105620] Updated weights for policy 1, policy_version 1624975 (0.0005) [2023-12-27 03:12:26,224][105620] Updated weights for policy 1, policy_version 1624985 (0.0008) [2023-12-27 03:12:26,225][105692] Updated weights for policy 0, policy_version 1621570 (0.0011) [2023-12-27 03:12:26,283][105692] Updated weights for policy 0, policy_version 1621580 (0.0009) [2023-12-27 03:12:26,338][105692] Updated weights for policy 0, policy_version 1621590 (0.0005) [2023-12-27 03:12:26,780][105620] Updated weights for policy 1, policy_version 1624995 (0.0008) [2023-12-27 03:12:26,826][105620] Updated weights for policy 1, policy_version 1625005 (0.0008) [2023-12-27 03:12:26,871][105620] Updated weights for policy 1, policy_version 1625015 (0.0010) [2023-12-27 03:12:26,940][105692] Updated weights for policy 0, policy_version 1621600 (0.0010) [2023-12-27 03:12:26,991][105692] Updated weights for policy 0, policy_version 1621610 (0.0010) [2023-12-27 03:12:27,042][105692] Updated weights for policy 0, policy_version 1621620 (0.0010) [2023-12-27 03:12:27,475][105620] Updated weights for policy 1, policy_version 1625025 (0.0010) [2023-12-27 03:12:27,552][105620] Updated weights for policy 1, policy_version 1625035 (0.0005) [2023-12-27 03:12:27,615][105620] Updated weights for policy 1, policy_version 1625045 (0.0005) [2023-12-27 03:12:27,688][105620] Updated weights for policy 1, policy_version 1625055 (0.0005) [2023-12-27 03:12:27,719][105692] Updated weights for policy 0, policy_version 1621630 (0.0008) [2023-12-27 03:12:27,765][105692] Updated weights for policy 0, policy_version 1621640 (0.0009) [2023-12-27 03:12:27,822][105692] Updated weights for policy 0, policy_version 1621650 (0.0006) [2023-12-27 03:12:28,243][105620] Updated weights for policy 1, policy_version 1625065 (0.0009) [2023-12-27 03:12:28,303][105620] Updated weights for policy 1, policy_version 1625075 (0.0009) [2023-12-27 03:12:28,366][105620] Updated weights for policy 1, policy_version 1625085 (0.0008) [2023-12-27 03:12:28,407][105692] Updated weights for policy 0, policy_version 1621660 (0.0009) [2023-12-27 03:12:28,466][105692] Updated weights for policy 0, policy_version 1621670 (0.0011) [2023-12-27 03:12:28,519][105585] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000003 [2023-12-27 03:12:28,521][105692] Updated weights for policy 0, policy_version 1621680 (0.0010) [2023-12-27 03:12:29,130][105620] Updated weights for policy 1, policy_version 1625095 (0.0007) [2023-12-27 03:12:29,190][105620] Updated weights for policy 1, policy_version 1625105 (0.0006) [2023-12-27 03:12:29,245][105620] Updated weights for policy 1, policy_version 1625115 (0.0009) [2023-12-27 03:12:29,312][105692] Updated weights for policy 0, policy_version 1621690 (0.0008) [2023-12-27 03:12:29,374][105692] Updated weights for policy 0, policy_version 1621700 (0.0008) [2023-12-27 03:12:29,425][105692] Updated weights for policy 0, policy_version 1621710 (0.0008) [2023-12-27 03:12:29,917][105620] Updated weights for policy 1, policy_version 1625125 (0.0007) [2023-12-27 03:12:29,981][105620] Updated weights for policy 1, policy_version 1625135 (0.0008) [2023-12-27 03:12:30,038][105620] Updated weights for policy 1, policy_version 1625145 (0.0009) [2023-12-27 03:12:30,147][105692] Updated weights for policy 0, policy_version 1621720 (0.0010) [2023-12-27 03:12:30,192][105692] Updated weights for policy 0, policy_version 1621730 (0.0010) [2023-12-27 03:12:30,243][105692] Updated weights for policy 0, policy_version 1621740 (0.0010) [2023-12-27 03:12:30,795][105620] Updated weights for policy 1, policy_version 1625155 (0.0009) [2023-12-27 03:12:30,856][105620] Updated weights for policy 1, policy_version 1625165 (0.0008) [2023-12-27 03:12:30,910][105620] Updated weights for policy 1, policy_version 1625175 (0.0010) [2023-12-27 03:12:30,970][105692] Updated weights for policy 0, policy_version 1621750 (0.0007) [2023-12-27 03:12:31,032][105692] Updated weights for policy 0, policy_version 1621760 (0.0008) [2023-12-27 03:12:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19933.9, 300 sec: 19522.0). Total num frames: 831332352. Throughput: 0: 9903.7, 1: 10022.5. Samples: 831303172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:12:31,063][104569] Avg episode reward: [(0, '7983.221'), (1, '9079.166')] [2023-12-27 03:12:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001625184_416104448.pth... [2023-12-27 03:12:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001624000_415801344.pth [2023-12-27 03:12:31,096][105692] Updated weights for policy 0, policy_version 1621770 (0.0010) [2023-12-27 03:12:31,126][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001621776_415236096.pth... [2023-12-27 03:12:31,131][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001620600_414932992.pth [2023-12-27 03:12:31,665][105620] Updated weights for policy 1, policy_version 1625185 (0.0009) [2023-12-27 03:12:31,719][105620] Updated weights for policy 1, policy_version 1625195 (0.0008) [2023-12-27 03:12:31,784][105620] Updated weights for policy 1, policy_version 1625205 (0.0007) [2023-12-27 03:12:31,825][105692] Updated weights for policy 0, policy_version 1621780 (0.0009) [2023-12-27 03:12:31,844][105620] Updated weights for policy 1, policy_version 1625215 (0.0006) [2023-12-27 03:12:31,884][105692] Updated weights for policy 0, policy_version 1621790 (0.0010) [2023-12-27 03:12:31,944][105692] Updated weights for policy 0, policy_version 1621800 (0.0010) [2023-12-27 03:12:32,635][105620] Updated weights for policy 1, policy_version 1625225 (0.0009) [2023-12-27 03:12:32,648][105692] Updated weights for policy 0, policy_version 1621810 (0.0008) [2023-12-27 03:12:32,696][105620] Updated weights for policy 1, policy_version 1625235 (0.0009) [2023-12-27 03:12:32,701][105692] Updated weights for policy 0, policy_version 1621820 (0.0005) [2023-12-27 03:12:32,753][105620] Updated weights for policy 1, policy_version 1625245 (0.0009) [2023-12-27 03:12:32,767][105692] Updated weights for policy 0, policy_version 1621830 (0.0005) [2023-12-27 03:12:32,829][105692] Updated weights for policy 0, policy_version 1621840 (0.0006) [2023-12-27 03:12:33,369][105692] Updated weights for policy 0, policy_version 1621850 (0.0005) [2023-12-27 03:12:33,434][105692] Updated weights for policy 0, policy_version 1621860 (0.0005) [2023-12-27 03:12:33,499][105692] Updated weights for policy 0, policy_version 1621870 (0.0008) [2023-12-27 03:12:33,521][105620] Updated weights for policy 1, policy_version 1625255 (0.0010) [2023-12-27 03:12:33,578][105620] Updated weights for policy 1, policy_version 1625265 (0.0010) [2023-12-27 03:12:33,631][105620] Updated weights for policy 1, policy_version 1625275 (0.0010) [2023-12-27 03:12:34,075][105692] Updated weights for policy 0, policy_version 1621880 (0.0010) [2023-12-27 03:12:34,130][105692] Updated weights for policy 0, policy_version 1621890 (0.0010) [2023-12-27 03:12:34,196][105692] Updated weights for policy 0, policy_version 1621900 (0.0011) [2023-12-27 03:12:34,362][105620] Updated weights for policy 1, policy_version 1625285 (0.0010) [2023-12-27 03:12:34,428][105620] Updated weights for policy 1, policy_version 1625295 (0.0011) [2023-12-27 03:12:34,487][105620] Updated weights for policy 1, policy_version 1625305 (0.0011) [2023-12-27 03:12:34,896][105692] Updated weights for policy 0, policy_version 1621910 (0.0009) [2023-12-27 03:12:34,952][105692] Updated weights for policy 0, policy_version 1621920 (0.0011) [2023-12-27 03:12:35,006][105692] Updated weights for policy 0, policy_version 1621930 (0.0010) [2023-12-27 03:12:35,207][105620] Updated weights for policy 1, policy_version 1625315 (0.0009) [2023-12-27 03:12:35,253][105620] Updated weights for policy 1, policy_version 1625325 (0.0005) [2023-12-27 03:12:35,309][105620] Updated weights for policy 1, policy_version 1625335 (0.0005) [2023-12-27 03:12:35,759][105692] Updated weights for policy 0, policy_version 1621940 (0.0010) [2023-12-27 03:12:35,804][105692] Updated weights for policy 0, policy_version 1621950 (0.0010) [2023-12-27 03:12:35,823][105620] Updated weights for policy 1, policy_version 1625345 (0.0005) [2023-12-27 03:12:35,859][105692] Updated weights for policy 0, policy_version 1621960 (0.0010) [2023-12-27 03:12:35,886][105620] Updated weights for policy 1, policy_version 1625355 (0.0008) [2023-12-27 03:12:35,941][105620] Updated weights for policy 1, policy_version 1625365 (0.0010) [2023-12-27 03:12:35,990][105620] Updated weights for policy 1, policy_version 1625375 (0.0010) [2023-12-27 03:12:36,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19933.8, 300 sec: 19605.3). Total num frames: 831438848. Throughput: 0: 10057.0, 1: 9864.2. Samples: 831420972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:12:36,062][104569] Avg episode reward: [(0, '8257.375'), (1, '9170.642')] [2023-12-27 03:12:36,446][105692] Updated weights for policy 0, policy_version 1621970 (0.0009) [2023-12-27 03:12:36,506][105692] Updated weights for policy 0, policy_version 1621980 (0.0006) [2023-12-27 03:12:36,561][105692] Updated weights for policy 0, policy_version 1621990 (0.0006) [2023-12-27 03:12:36,612][105692] Updated weights for policy 0, policy_version 1622000 (0.0009) [2023-12-27 03:12:36,736][105620] Updated weights for policy 1, policy_version 1625385 (0.0011) [2023-12-27 03:12:36,791][105620] Updated weights for policy 1, policy_version 1625395 (0.0010) [2023-12-27 03:12:36,857][105620] Updated weights for policy 1, policy_version 1625405 (0.0011) [2023-12-27 03:12:37,346][105692] Updated weights for policy 0, policy_version 1622010 (0.0011) [2023-12-27 03:12:37,402][105692] Updated weights for policy 0, policy_version 1622020 (0.0011) [2023-12-27 03:12:37,454][105692] Updated weights for policy 0, policy_version 1622030 (0.0010) [2023-12-27 03:12:37,617][105620] Updated weights for policy 1, policy_version 1625415 (0.0011) [2023-12-27 03:12:37,678][105620] Updated weights for policy 1, policy_version 1625425 (0.0010) [2023-12-27 03:12:37,741][105620] Updated weights for policy 1, policy_version 1625435 (0.0005) [2023-12-27 03:12:38,195][105692] Updated weights for policy 0, policy_version 1622040 (0.0010) [2023-12-27 03:12:38,253][105692] Updated weights for policy 0, policy_version 1622050 (0.0011) [2023-12-27 03:12:38,298][105620] Updated weights for policy 1, policy_version 1625445 (0.0008) [2023-12-27 03:12:38,305][105692] Updated weights for policy 0, policy_version 1622060 (0.0010) [2023-12-27 03:12:38,364][105620] Updated weights for policy 1, policy_version 1625455 (0.0011) [2023-12-27 03:12:38,421][105620] Updated weights for policy 1, policy_version 1625465 (0.0010) [2023-12-27 03:12:39,087][105692] Updated weights for policy 0, policy_version 1622070 (0.0010) [2023-12-27 03:12:39,091][105620] Updated weights for policy 1, policy_version 1625475 (0.0007) [2023-12-27 03:12:39,147][105620] Updated weights for policy 1, policy_version 1625485 (0.0010) [2023-12-27 03:12:39,149][105692] Updated weights for policy 0, policy_version 1622080 (0.0011) [2023-12-27 03:12:39,205][105692] Updated weights for policy 0, policy_version 1622090 (0.0010) [2023-12-27 03:12:39,206][105620] Updated weights for policy 1, policy_version 1625495 (0.0011) [2023-12-27 03:12:39,962][105692] Updated weights for policy 0, policy_version 1622100 (0.0010) [2023-12-27 03:12:39,992][105620] Updated weights for policy 1, policy_version 1625505 (0.0008) [2023-12-27 03:12:40,022][105692] Updated weights for policy 0, policy_version 1622110 (0.0011) [2023-12-27 03:12:40,052][105620] Updated weights for policy 1, policy_version 1625515 (0.0006) [2023-12-27 03:12:40,075][105692] Updated weights for policy 0, policy_version 1622120 (0.0011) [2023-12-27 03:12:40,115][105620] Updated weights for policy 1, policy_version 1625525 (0.0006) [2023-12-27 03:12:40,179][105620] Updated weights for policy 1, policy_version 1625535 (0.0008) [2023-12-27 03:12:40,882][105692] Updated weights for policy 0, policy_version 1622130 (0.0011) [2023-12-27 03:12:40,919][105620] Updated weights for policy 1, policy_version 1625545 (0.0011) [2023-12-27 03:12:40,942][105692] Updated weights for policy 0, policy_version 1622140 (0.0011) [2023-12-27 03:12:40,976][105620] Updated weights for policy 1, policy_version 1625555 (0.0011) [2023-12-27 03:12:41,001][105692] Updated weights for policy 0, policy_version 1622150 (0.0010) [2023-12-27 03:12:41,037][105620] Updated weights for policy 1, policy_version 1625565 (0.0010) [2023-12-27 03:12:41,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 831528960. Throughput: 0: 9977.9, 1: 9907.2. Samples: 831538820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:12:41,062][104569] Avg episode reward: [(0, '9077.082'), (1, '8894.873')] [2023-12-27 03:12:41,068][105692] Updated weights for policy 0, policy_version 1622160 (0.0010) [2023-12-27 03:12:41,817][105620] Updated weights for policy 1, policy_version 1625575 (0.0010) [2023-12-27 03:12:41,871][105692] Updated weights for policy 0, policy_version 1622170 (0.0009) [2023-12-27 03:12:41,875][105620] Updated weights for policy 1, policy_version 1625585 (0.0010) [2023-12-27 03:12:41,932][105692] Updated weights for policy 0, policy_version 1622180 (0.0009) [2023-12-27 03:12:41,945][105620] Updated weights for policy 1, policy_version 1625595 (0.0008) [2023-12-27 03:12:41,992][105692] Updated weights for policy 0, policy_version 1622190 (0.0009) [2023-12-27 03:12:42,693][105620] Updated weights for policy 1, policy_version 1625605 (0.0011) [2023-12-27 03:12:42,756][105620] Updated weights for policy 1, policy_version 1625615 (0.0011) [2023-12-27 03:12:42,819][105620] Updated weights for policy 1, policy_version 1625625 (0.0011) [2023-12-27 03:12:42,833][105692] Updated weights for policy 0, policy_version 1622200 (0.0007) [2023-12-27 03:12:42,898][105692] Updated weights for policy 0, policy_version 1622210 (0.0010) [2023-12-27 03:12:42,964][105692] Updated weights for policy 0, policy_version 1622220 (0.0009) [2023-12-27 03:12:43,440][105620] Updated weights for policy 1, policy_version 1625635 (0.0008) [2023-12-27 03:12:43,498][105620] Updated weights for policy 1, policy_version 1625645 (0.0011) [2023-12-27 03:12:43,553][105620] Updated weights for policy 1, policy_version 1625655 (0.0011) [2023-12-27 03:12:43,664][105692] Updated weights for policy 0, policy_version 1622230 (0.0009) [2023-12-27 03:12:43,722][105692] Updated weights for policy 0, policy_version 1622240 (0.0010) [2023-12-27 03:12:43,775][105692] Updated weights for policy 0, policy_version 1622250 (0.0009) [2023-12-27 03:12:44,247][105620] Updated weights for policy 1, policy_version 1625665 (0.0007) [2023-12-27 03:12:44,305][105620] Updated weights for policy 1, policy_version 1625675 (0.0010) [2023-12-27 03:12:44,372][105620] Updated weights for policy 1, policy_version 1625685 (0.0010) [2023-12-27 03:12:44,432][105620] Updated weights for policy 1, policy_version 1625695 (0.0010) [2023-12-27 03:12:44,553][105692] Updated weights for policy 0, policy_version 1622260 (0.0008) [2023-12-27 03:12:44,610][105692] Updated weights for policy 0, policy_version 1622270 (0.0008) [2023-12-27 03:12:44,667][105692] Updated weights for policy 0, policy_version 1622280 (0.0009) [2023-12-27 03:12:45,155][105620] Updated weights for policy 1, policy_version 1625705 (0.0006) [2023-12-27 03:12:45,221][105620] Updated weights for policy 1, policy_version 1625715 (0.0005) [2023-12-27 03:12:45,285][105620] Updated weights for policy 1, policy_version 1625725 (0.0008) [2023-12-27 03:12:45,484][105692] Updated weights for policy 0, policy_version 1622290 (0.0008) [2023-12-27 03:12:45,557][105692] Updated weights for policy 0, policy_version 1622300 (0.0010) [2023-12-27 03:12:45,619][105692] Updated weights for policy 0, policy_version 1622310 (0.0009) [2023-12-27 03:12:45,679][105692] Updated weights for policy 0, policy_version 1622320 (0.0010) [2023-12-27 03:12:45,907][105620] Updated weights for policy 1, policy_version 1625735 (0.0009) [2023-12-27 03:12:45,963][105620] Updated weights for policy 1, policy_version 1625745 (0.0009) [2023-12-27 03:12:46,019][105620] Updated weights for policy 1, policy_version 1625755 (0.0009) [2023-12-27 03:12:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 831627264. Throughput: 0: 9866.4, 1: 9893.0. Samples: 831595044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:12:46,062][104569] Avg episode reward: [(0, '8712.584'), (1, '8895.483')] [2023-12-27 03:12:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001625760_416251904.pth... [2023-12-27 03:12:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001622320_415375360.pth... [2023-12-27 03:12:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001624576_415948800.pth [2023-12-27 03:12:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001621176_415080448.pth [2023-12-27 03:12:46,532][105692] Updated weights for policy 0, policy_version 1622330 (0.0008) [2023-12-27 03:12:46,600][105692] Updated weights for policy 0, policy_version 1622340 (0.0008) [2023-12-27 03:12:46,646][105620] Updated weights for policy 1, policy_version 1625765 (0.0008) [2023-12-27 03:12:46,657][105692] Updated weights for policy 0, policy_version 1622350 (0.0009) [2023-12-27 03:12:46,697][105620] Updated weights for policy 1, policy_version 1625775 (0.0006) [2023-12-27 03:12:46,752][105620] Updated weights for policy 1, policy_version 1625785 (0.0008) [2023-12-27 03:12:47,431][105620] Updated weights for policy 1, policy_version 1625795 (0.0008) [2023-12-27 03:12:47,458][105692] Updated weights for policy 0, policy_version 1622360 (0.0010) [2023-12-27 03:12:47,494][105620] Updated weights for policy 1, policy_version 1625805 (0.0005) [2023-12-27 03:12:47,518][105692] Updated weights for policy 0, policy_version 1622370 (0.0011) [2023-12-27 03:12:47,561][105620] Updated weights for policy 1, policy_version 1625815 (0.0006) [2023-12-27 03:12:47,574][105692] Updated weights for policy 0, policy_version 1622380 (0.0011) [2023-12-27 03:12:48,120][105620] Updated weights for policy 1, policy_version 1625825 (0.0010) [2023-12-27 03:12:48,180][105620] Updated weights for policy 1, policy_version 1625835 (0.0010) [2023-12-27 03:12:48,222][105692] Updated weights for policy 0, policy_version 1622390 (0.0011) [2023-12-27 03:12:48,239][105620] Updated weights for policy 1, policy_version 1625845 (0.0010) [2023-12-27 03:12:48,281][105692] Updated weights for policy 0, policy_version 1622400 (0.0011) [2023-12-27 03:12:48,298][105620] Updated weights for policy 1, policy_version 1625855 (0.0010) [2023-12-27 03:12:48,342][105692] Updated weights for policy 0, policy_version 1622410 (0.0011) [2023-12-27 03:12:49,012][105692] Updated weights for policy 0, policy_version 1622420 (0.0009) [2023-12-27 03:12:49,054][105620] Updated weights for policy 1, policy_version 1625865 (0.0011) [2023-12-27 03:12:49,075][105692] Updated weights for policy 0, policy_version 1622430 (0.0009) [2023-12-27 03:12:49,115][105620] Updated weights for policy 1, policy_version 1625875 (0.0011) [2023-12-27 03:12:49,134][105692] Updated weights for policy 0, policy_version 1622440 (0.0010) [2023-12-27 03:12:49,171][105620] Updated weights for policy 1, policy_version 1625885 (0.0010) [2023-12-27 03:12:49,858][105692] Updated weights for policy 0, policy_version 1622450 (0.0010) [2023-12-27 03:12:49,921][105692] Updated weights for policy 0, policy_version 1622460 (0.0007) [2023-12-27 03:12:49,936][105620] Updated weights for policy 1, policy_version 1625895 (0.0009) [2023-12-27 03:12:49,987][105692] Updated weights for policy 0, policy_version 1622470 (0.0008) [2023-12-27 03:12:50,000][105620] Updated weights for policy 1, policy_version 1625905 (0.0010) [2023-12-27 03:12:50,044][105692] Updated weights for policy 0, policy_version 1622480 (0.0008) [2023-12-27 03:12:50,052][105620] Updated weights for policy 1, policy_version 1625915 (0.0010) [2023-12-27 03:12:50,804][105692] Updated weights for policy 0, policy_version 1622490 (0.0011) [2023-12-27 03:12:50,805][105620] Updated weights for policy 1, policy_version 1625925 (0.0009) [2023-12-27 03:12:50,865][105692] Updated weights for policy 0, policy_version 1622500 (0.0010) [2023-12-27 03:12:50,868][105620] Updated weights for policy 1, policy_version 1625935 (0.0005) [2023-12-27 03:12:50,921][105692] Updated weights for policy 0, policy_version 1622510 (0.0011) [2023-12-27 03:12:50,929][105620] Updated weights for policy 1, policy_version 1625945 (0.0006) [2023-12-27 03:12:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 831725568. Throughput: 0: 9821.2, 1: 9879.8. Samples: 831710844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:12:51,062][104569] Avg episode reward: [(0, '8528.934'), (1, '9169.544')] [2023-12-27 03:12:51,618][105620] Updated weights for policy 1, policy_version 1625955 (0.0006) [2023-12-27 03:12:51,672][105692] Updated weights for policy 0, policy_version 1622520 (0.0007) [2023-12-27 03:12:51,682][105620] Updated weights for policy 1, policy_version 1625965 (0.0009) [2023-12-27 03:12:51,740][105692] Updated weights for policy 0, policy_version 1622530 (0.0007) [2023-12-27 03:12:51,752][105620] Updated weights for policy 1, policy_version 1625975 (0.0009) [2023-12-27 03:12:51,801][105692] Updated weights for policy 0, policy_version 1622540 (0.0007) [2023-12-27 03:12:52,395][105692] Updated weights for policy 0, policy_version 1622550 (0.0008) [2023-12-27 03:12:52,452][105692] Updated weights for policy 0, policy_version 1622560 (0.0008) [2023-12-27 03:12:52,509][105692] Updated weights for policy 0, policy_version 1622570 (0.0006) [2023-12-27 03:12:52,618][105620] Updated weights for policy 1, policy_version 1625985 (0.0008) [2023-12-27 03:12:52,675][105620] Updated weights for policy 1, policy_version 1625995 (0.0009) [2023-12-27 03:12:52,741][105620] Updated weights for policy 1, policy_version 1626005 (0.0009) [2023-12-27 03:12:52,802][105620] Updated weights for policy 1, policy_version 1626015 (0.0009) [2023-12-27 03:12:53,255][105692] Updated weights for policy 0, policy_version 1622580 (0.0009) [2023-12-27 03:12:53,313][105692] Updated weights for policy 0, policy_version 1622590 (0.0010) [2023-12-27 03:12:53,369][105692] Updated weights for policy 0, policy_version 1622600 (0.0009) [2023-12-27 03:12:53,561][105620] Updated weights for policy 1, policy_version 1626025 (0.0009) [2023-12-27 03:12:53,619][105620] Updated weights for policy 1, policy_version 1626035 (0.0009) [2023-12-27 03:12:53,675][105620] Updated weights for policy 1, policy_version 1626045 (0.0009) [2023-12-27 03:12:54,133][105692] Updated weights for policy 0, policy_version 1622610 (0.0009) [2023-12-27 03:12:54,189][105692] Updated weights for policy 0, policy_version 1622620 (0.0009) [2023-12-27 03:12:54,241][105692] Updated weights for policy 0, policy_version 1622630 (0.0009) [2023-12-27 03:12:54,309][105692] Updated weights for policy 0, policy_version 1622640 (0.0009) [2023-12-27 03:12:54,436][105620] Updated weights for policy 1, policy_version 1626055 (0.0008) [2023-12-27 03:12:54,484][105620] Updated weights for policy 1, policy_version 1626065 (0.0007) [2023-12-27 03:12:54,539][105620] Updated weights for policy 1, policy_version 1626075 (0.0008) [2023-12-27 03:12:55,099][105692] Updated weights for policy 0, policy_version 1622650 (0.0010) [2023-12-27 03:12:55,164][105692] Updated weights for policy 0, policy_version 1622660 (0.0010) [2023-12-27 03:12:55,226][105620] Updated weights for policy 1, policy_version 1626085 (0.0008) [2023-12-27 03:12:55,227][105692] Updated weights for policy 0, policy_version 1622670 (0.0009) [2023-12-27 03:12:55,278][105620] Updated weights for policy 1, policy_version 1626095 (0.0008) [2023-12-27 03:12:55,343][105620] Updated weights for policy 1, policy_version 1626105 (0.0010) [2023-12-27 03:12:55,882][105692] Updated weights for policy 0, policy_version 1622680 (0.0007) [2023-12-27 03:12:55,942][105692] Updated weights for policy 0, policy_version 1622690 (0.0005) [2023-12-27 03:12:55,997][105692] Updated weights for policy 0, policy_version 1622700 (0.0006) [2023-12-27 03:12:56,058][105620] Updated weights for policy 1, policy_version 1626115 (0.0008) [2023-12-27 03:12:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 831815680. Throughput: 0: 9743.2, 1: 9787.4. Samples: 831823032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:12:56,063][104569] Avg episode reward: [(0, '8440.606'), (1, '9079.135')] [2023-12-27 03:12:56,113][105620] Updated weights for policy 1, policy_version 1626125 (0.0006) [2023-12-27 03:12:56,167][105620] Updated weights for policy 1, policy_version 1626135 (0.0009) [2023-12-27 03:12:56,607][105692] Updated weights for policy 0, policy_version 1622710 (0.0006) [2023-12-27 03:12:56,664][105692] Updated weights for policy 0, policy_version 1622720 (0.0007) [2023-12-27 03:12:56,730][105692] Updated weights for policy 0, policy_version 1622730 (0.0009) [2023-12-27 03:12:56,923][105620] Updated weights for policy 1, policy_version 1626145 (0.0009) [2023-12-27 03:12:56,979][105620] Updated weights for policy 1, policy_version 1626155 (0.0010) [2023-12-27 03:12:57,035][105620] Updated weights for policy 1, policy_version 1626165 (0.0010) [2023-12-27 03:12:57,102][105620] Updated weights for policy 1, policy_version 1626175 (0.0008) [2023-12-27 03:12:57,342][105692] Updated weights for policy 0, policy_version 1622740 (0.0007) [2023-12-27 03:12:57,398][105692] Updated weights for policy 0, policy_version 1622750 (0.0005) [2023-12-27 03:12:57,458][105692] Updated weights for policy 0, policy_version 1622760 (0.0005) [2023-12-27 03:12:57,861][105620] Updated weights for policy 1, policy_version 1626185 (0.0009) [2023-12-27 03:12:57,916][105620] Updated weights for policy 1, policy_version 1626195 (0.0009) [2023-12-27 03:12:57,959][105620] Updated weights for policy 1, policy_version 1626205 (0.0006) [2023-12-27 03:12:58,194][105692] Updated weights for policy 0, policy_version 1622770 (0.0008) [2023-12-27 03:12:58,260][105692] Updated weights for policy 0, policy_version 1622780 (0.0008) [2023-12-27 03:12:58,323][105692] Updated weights for policy 0, policy_version 1622790 (0.0008) [2023-12-27 03:12:58,393][105692] Updated weights for policy 0, policy_version 1622800 (0.0007) [2023-12-27 03:12:58,730][105620] Updated weights for policy 1, policy_version 1626215 (0.0007) [2023-12-27 03:12:58,797][105620] Updated weights for policy 1, policy_version 1626225 (0.0008) [2023-12-27 03:12:58,860][105620] Updated weights for policy 1, policy_version 1626235 (0.0009) [2023-12-27 03:12:59,207][105692] Updated weights for policy 0, policy_version 1622810 (0.0009) [2023-12-27 03:12:59,274][105692] Updated weights for policy 0, policy_version 1622820 (0.0009) [2023-12-27 03:12:59,337][105692] Updated weights for policy 0, policy_version 1622830 (0.0008) [2023-12-27 03:12:59,726][105620] Updated weights for policy 1, policy_version 1626245 (0.0007) [2023-12-27 03:12:59,795][105620] Updated weights for policy 1, policy_version 1626255 (0.0006) [2023-12-27 03:12:59,863][105620] Updated weights for policy 1, policy_version 1626265 (0.0008) [2023-12-27 03:13:00,183][105692] Updated weights for policy 0, policy_version 1622840 (0.0008) [2023-12-27 03:13:00,247][105692] Updated weights for policy 0, policy_version 1622850 (0.0009) [2023-12-27 03:13:00,305][105692] Updated weights for policy 0, policy_version 1622860 (0.0008) [2023-12-27 03:13:00,544][105620] Updated weights for policy 1, policy_version 1626275 (0.0007) [2023-12-27 03:13:00,600][105620] Updated weights for policy 1, policy_version 1626285 (0.0009) [2023-12-27 03:13:00,661][105620] Updated weights for policy 1, policy_version 1626295 (0.0009) [2023-12-27 03:13:01,057][105692] Updated weights for policy 0, policy_version 1622870 (0.0009) [2023-12-27 03:13:01,062][104569] Fps is (10 sec: 18022.1, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 831905792. Throughput: 0: 9822.6, 1: 9746.2. Samples: 831882008. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:13:01,062][104569] Avg episode reward: [(0, '8258.968'), (1, '9079.289')] [2023-12-27 03:13:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001626304_416391168.pth... [2023-12-27 03:13:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001625184_416104448.pth [2023-12-27 03:13:01,126][105692] Updated weights for policy 0, policy_version 1622880 (0.0010) [2023-12-27 03:13:01,186][105692] Updated weights for policy 0, policy_version 1622890 (0.0009) [2023-12-27 03:13:01,218][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001622896_415522816.pth... [2023-12-27 03:13:01,222][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001621776_415236096.pth [2023-12-27 03:13:01,418][105620] Updated weights for policy 1, policy_version 1626305 (0.0009) [2023-12-27 03:13:01,473][105620] Updated weights for policy 1, policy_version 1626315 (0.0009) [2023-12-27 03:13:01,531][105620] Updated weights for policy 1, policy_version 1626325 (0.0009) [2023-12-27 03:13:01,587][105620] Updated weights for policy 1, policy_version 1626335 (0.0009) [2023-12-27 03:13:01,964][105692] Updated weights for policy 0, policy_version 1622900 (0.0009) [2023-12-27 03:13:02,020][105692] Updated weights for policy 0, policy_version 1622910 (0.0010) [2023-12-27 03:13:02,083][105692] Updated weights for policy 0, policy_version 1622920 (0.0008) [2023-12-27 03:13:02,294][105620] Updated weights for policy 1, policy_version 1626345 (0.0008) [2023-12-27 03:13:02,353][105620] Updated weights for policy 1, policy_version 1626355 (0.0009) [2023-12-27 03:13:02,413][105620] Updated weights for policy 1, policy_version 1626365 (0.0010) [2023-12-27 03:13:02,821][105692] Updated weights for policy 0, policy_version 1622930 (0.0009) [2023-12-27 03:13:02,889][105692] Updated weights for policy 0, policy_version 1622940 (0.0010) [2023-12-27 03:13:02,955][105692] Updated weights for policy 0, policy_version 1622950 (0.0009) [2023-12-27 03:13:03,006][105692] Updated weights for policy 0, policy_version 1622960 (0.0009) [2023-12-27 03:13:03,135][105620] Updated weights for policy 1, policy_version 1626375 (0.0008) [2023-12-27 03:13:03,188][105620] Updated weights for policy 1, policy_version 1626385 (0.0007) [2023-12-27 03:13:03,236][105620] Updated weights for policy 1, policy_version 1626395 (0.0009) [2023-12-27 03:13:03,782][105692] Updated weights for policy 0, policy_version 1622970 (0.0008) [2023-12-27 03:13:03,849][105692] Updated weights for policy 0, policy_version 1622980 (0.0008) [2023-12-27 03:13:03,915][105692] Updated weights for policy 0, policy_version 1622990 (0.0006) [2023-12-27 03:13:04,014][105620] Updated weights for policy 1, policy_version 1626405 (0.0009) [2023-12-27 03:13:04,070][105620] Updated weights for policy 1, policy_version 1626415 (0.0009) [2023-12-27 03:13:04,127][105620] Updated weights for policy 1, policy_version 1626425 (0.0009) [2023-12-27 03:13:04,577][105692] Updated weights for policy 0, policy_version 1623000 (0.0010) [2023-12-27 03:13:04,642][105692] Updated weights for policy 0, policy_version 1623010 (0.0011) [2023-12-27 03:13:04,702][105692] Updated weights for policy 0, policy_version 1623020 (0.0010) [2023-12-27 03:13:04,925][105620] Updated weights for policy 1, policy_version 1626435 (0.0009) [2023-12-27 03:13:04,980][105620] Updated weights for policy 1, policy_version 1626445 (0.0008) [2023-12-27 03:13:05,046][105620] Updated weights for policy 1, policy_version 1626455 (0.0010) [2023-12-27 03:13:05,418][105692] Updated weights for policy 0, policy_version 1623030 (0.0010) [2023-12-27 03:13:05,472][105692] Updated weights for policy 0, policy_version 1623040 (0.0007) [2023-12-27 03:13:05,537][105692] Updated weights for policy 0, policy_version 1623050 (0.0005) [2023-12-27 03:13:05,836][105620] Updated weights for policy 1, policy_version 1626465 (0.0007) [2023-12-27 03:13:05,899][105620] Updated weights for policy 1, policy_version 1626475 (0.0009) [2023-12-27 03:13:05,969][105620] Updated weights for policy 1, policy_version 1626485 (0.0009) [2023-12-27 03:13:06,044][105620] Updated weights for policy 1, policy_version 1626495 (0.0010) [2023-12-27 03:13:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 832004096. Throughput: 0: 9649.2, 1: 9777.7. Samples: 831992580. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:13:06,063][104569] Avg episode reward: [(0, '8711.867'), (1, '8898.574')] [2023-12-27 03:13:06,156][105692] Updated weights for policy 0, policy_version 1623060 (0.0007) [2023-12-27 03:13:06,217][105692] Updated weights for policy 0, policy_version 1623070 (0.0008) [2023-12-27 03:13:06,288][105692] Updated weights for policy 0, policy_version 1623080 (0.0005) [2023-12-27 03:13:06,741][105620] Updated weights for policy 1, policy_version 1626505 (0.0010) [2023-12-27 03:13:06,794][105620] Updated weights for policy 1, policy_version 1626515 (0.0011) [2023-12-27 03:13:06,845][105692] Updated weights for policy 0, policy_version 1623090 (0.0006) [2023-12-27 03:13:06,853][105620] Updated weights for policy 1, policy_version 1626525 (0.0010) [2023-12-27 03:13:06,901][105692] Updated weights for policy 0, policy_version 1623100 (0.0010) [2023-12-27 03:13:06,949][105692] Updated weights for policy 0, policy_version 1623110 (0.0010) [2023-12-27 03:13:06,998][105692] Updated weights for policy 0, policy_version 1623120 (0.0010) [2023-12-27 03:13:07,482][105620] Updated weights for policy 1, policy_version 1626535 (0.0007) [2023-12-27 03:13:07,540][105620] Updated weights for policy 1, policy_version 1626545 (0.0005) [2023-12-27 03:13:07,604][105620] Updated weights for policy 1, policy_version 1626555 (0.0006) [2023-12-27 03:13:07,610][105692] Updated weights for policy 0, policy_version 1623130 (0.0010) [2023-12-27 03:13:07,665][105692] Updated weights for policy 0, policy_version 1623140 (0.0010) [2023-12-27 03:13:07,727][105692] Updated weights for policy 0, policy_version 1623150 (0.0010) [2023-12-27 03:13:08,254][105620] Updated weights for policy 1, policy_version 1626565 (0.0007) [2023-12-27 03:13:08,312][105620] Updated weights for policy 1, policy_version 1626575 (0.0008) [2023-12-27 03:13:08,376][105620] Updated weights for policy 1, policy_version 1626585 (0.0008) [2023-12-27 03:13:08,506][105692] Updated weights for policy 0, policy_version 1623160 (0.0010) [2023-12-27 03:13:08,568][105692] Updated weights for policy 0, policy_version 1623170 (0.0011) [2023-12-27 03:13:08,631][105692] Updated weights for policy 0, policy_version 1623180 (0.0010) [2023-12-27 03:13:09,088][105620] Updated weights for policy 1, policy_version 1626595 (0.0007) [2023-12-27 03:13:09,151][105620] Updated weights for policy 1, policy_version 1626605 (0.0008) [2023-12-27 03:13:09,203][105620] Updated weights for policy 1, policy_version 1626615 (0.0008) [2023-12-27 03:13:09,351][105692] Updated weights for policy 0, policy_version 1623190 (0.0010) [2023-12-27 03:13:09,417][105692] Updated weights for policy 0, policy_version 1623200 (0.0009) [2023-12-27 03:13:09,483][105692] Updated weights for policy 0, policy_version 1623210 (0.0006) [2023-12-27 03:13:10,086][105620] Updated weights for policy 1, policy_version 1626625 (0.0008) [2023-12-27 03:13:10,098][105692] Updated weights for policy 0, policy_version 1623220 (0.0006) [2023-12-27 03:13:10,151][105620] Updated weights for policy 1, policy_version 1626635 (0.0010) [2023-12-27 03:13:10,154][105692] Updated weights for policy 0, policy_version 1623230 (0.0006) [2023-12-27 03:13:10,213][105620] Updated weights for policy 1, policy_version 1626645 (0.0008) [2023-12-27 03:13:10,219][105692] Updated weights for policy 0, policy_version 1623240 (0.0007) [2023-12-27 03:13:10,277][105620] Updated weights for policy 1, policy_version 1626655 (0.0008) [2023-12-27 03:13:10,867][105620] Updated weights for policy 1, policy_version 1626665 (0.0009) [2023-12-27 03:13:10,926][105620] Updated weights for policy 1, policy_version 1626675 (0.0008) [2023-12-27 03:13:10,974][105692] Updated weights for policy 0, policy_version 1623250 (0.0007) [2023-12-27 03:13:10,983][105620] Updated weights for policy 1, policy_version 1626685 (0.0005) [2023-12-27 03:13:11,045][105692] Updated weights for policy 0, policy_version 1623260 (0.0008) [2023-12-27 03:13:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 832102400. Throughput: 0: 9723.6, 1: 9717.0. Samples: 832112116. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:13:11,062][104569] Avg episode reward: [(0, '8625.169'), (1, '8897.238')] [2023-12-27 03:13:11,116][105692] Updated weights for policy 0, policy_version 1623270 (0.0009) [2023-12-27 03:13:11,174][105692] Updated weights for policy 0, policy_version 1623280 (0.0008) [2023-12-27 03:13:11,669][105620] Updated weights for policy 1, policy_version 1626695 (0.0008) [2023-12-27 03:13:11,731][105620] Updated weights for policy 1, policy_version 1626705 (0.0009) [2023-12-27 03:13:11,795][105620] Updated weights for policy 1, policy_version 1626715 (0.0007) [2023-12-27 03:13:11,931][105692] Updated weights for policy 0, policy_version 1623290 (0.0010) [2023-12-27 03:13:11,986][105692] Updated weights for policy 0, policy_version 1623300 (0.0009) [2023-12-27 03:13:12,033][105692] Updated weights for policy 0, policy_version 1623310 (0.0008) [2023-12-27 03:13:12,588][105620] Updated weights for policy 1, policy_version 1626725 (0.0008) [2023-12-27 03:13:12,649][105620] Updated weights for policy 1, policy_version 1626735 (0.0009) [2023-12-27 03:13:12,720][105620] Updated weights for policy 1, policy_version 1626745 (0.0009) [2023-12-27 03:13:12,721][105692] Updated weights for policy 0, policy_version 1623320 (0.0007) [2023-12-27 03:13:12,782][105692] Updated weights for policy 0, policy_version 1623330 (0.0005) [2023-12-27 03:13:12,830][105692] Updated weights for policy 0, policy_version 1623340 (0.0005) [2023-12-27 03:13:13,399][105692] Updated weights for policy 0, policy_version 1623350 (0.0007) [2023-12-27 03:13:13,454][105692] Updated weights for policy 0, policy_version 1623360 (0.0008) [2023-12-27 03:13:13,505][105692] Updated weights for policy 0, policy_version 1623370 (0.0009) [2023-12-27 03:13:13,564][105620] Updated weights for policy 1, policy_version 1626755 (0.0008) [2023-12-27 03:13:13,624][105620] Updated weights for policy 1, policy_version 1626766 (0.0010) [2023-12-27 03:13:13,684][105620] Updated weights for policy 1, policy_version 1626776 (0.0008) [2023-12-27 03:13:14,134][105692] Updated weights for policy 0, policy_version 1623380 (0.0007) [2023-12-27 03:13:14,188][105692] Updated weights for policy 0, policy_version 1623390 (0.0005) [2023-12-27 03:13:14,236][105692] Updated weights for policy 0, policy_version 1623400 (0.0005) [2023-12-27 03:13:14,427][105620] Updated weights for policy 1, policy_version 1626786 (0.0009) [2023-12-27 03:13:14,490][105620] Updated weights for policy 1, policy_version 1626796 (0.0006) [2023-12-27 03:13:14,559][105620] Updated weights for policy 1, policy_version 1626806 (0.0006) [2023-12-27 03:13:14,621][105620] Updated weights for policy 1, policy_version 1626816 (0.0005) [2023-12-27 03:13:14,877][105692] Updated weights for policy 0, policy_version 1623410 (0.0006) [2023-12-27 03:13:14,936][105692] Updated weights for policy 0, policy_version 1623420 (0.0009) [2023-12-27 03:13:14,999][105692] Updated weights for policy 0, policy_version 1623430 (0.0009) [2023-12-27 03:13:15,065][105692] Updated weights for policy 0, policy_version 1623440 (0.0009) [2023-12-27 03:13:15,257][105620] Updated weights for policy 1, policy_version 1626826 (0.0010) [2023-12-27 03:13:15,307][105620] Updated weights for policy 1, policy_version 1626836 (0.0008) [2023-12-27 03:13:15,360][105620] Updated weights for policy 1, policy_version 1626846 (0.0009) [2023-12-27 03:13:15,832][105692] Updated weights for policy 0, policy_version 1623450 (0.0009) [2023-12-27 03:13:15,894][105692] Updated weights for policy 0, policy_version 1623460 (0.0009) [2023-12-27 03:13:15,948][105692] Updated weights for policy 0, policy_version 1623470 (0.0009) [2023-12-27 03:13:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 832200704. Throughput: 0: 9657.9, 1: 9578.2. Samples: 832168796. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:13:16,062][104569] Avg episode reward: [(0, '8442.751'), (1, '9261.955')] [2023-12-27 03:13:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001623472_415670272.pth... [2023-12-27 03:13:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001626848_416530432.pth... [2023-12-27 03:13:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001622320_415375360.pth [2023-12-27 03:13:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001625760_416251904.pth [2023-12-27 03:13:16,126][105620] Updated weights for policy 1, policy_version 1626856 (0.0009) [2023-12-27 03:13:16,180][105620] Updated weights for policy 1, policy_version 1626866 (0.0009) [2023-12-27 03:13:16,230][105620] Updated weights for policy 1, policy_version 1626876 (0.0009) [2023-12-27 03:13:16,616][105692] Updated weights for policy 0, policy_version 1623480 (0.0007) [2023-12-27 03:13:16,674][105692] Updated weights for policy 0, policy_version 1623490 (0.0006) [2023-12-27 03:13:16,732][105692] Updated weights for policy 0, policy_version 1623500 (0.0006) [2023-12-27 03:13:17,040][105620] Updated weights for policy 1, policy_version 1626886 (0.0008) [2023-12-27 03:13:17,097][105620] Updated weights for policy 1, policy_version 1626896 (0.0009) [2023-12-27 03:13:17,147][105620] Updated weights for policy 1, policy_version 1626906 (0.0009) [2023-12-27 03:13:17,446][105692] Updated weights for policy 0, policy_version 1623510 (0.0008) [2023-12-27 03:13:17,499][105692] Updated weights for policy 0, policy_version 1623520 (0.0006) [2023-12-27 03:13:17,547][105692] Updated weights for policy 0, policy_version 1623530 (0.0006) [2023-12-27 03:13:17,942][105620] Updated weights for policy 1, policy_version 1626916 (0.0009) [2023-12-27 03:13:17,988][105620] Updated weights for policy 1, policy_version 1626926 (0.0008) [2023-12-27 03:13:18,042][105620] Updated weights for policy 1, policy_version 1626936 (0.0009) [2023-12-27 03:13:18,170][105692] Updated weights for policy 0, policy_version 1623540 (0.0008) [2023-12-27 03:13:18,229][105692] Updated weights for policy 0, policy_version 1623550 (0.0008) [2023-12-27 03:13:18,281][105692] Updated weights for policy 0, policy_version 1623560 (0.0009) [2023-12-27 03:13:18,752][105620] Updated weights for policy 1, policy_version 1626946 (0.0009) [2023-12-27 03:13:18,807][105620] Updated weights for policy 1, policy_version 1626956 (0.0010) [2023-12-27 03:13:18,856][105620] Updated weights for policy 1, policy_version 1626966 (0.0010) [2023-12-27 03:13:18,915][105620] Updated weights for policy 1, policy_version 1626976 (0.0010) [2023-12-27 03:13:19,103][105692] Updated weights for policy 0, policy_version 1623570 (0.0009) [2023-12-27 03:13:19,165][105692] Updated weights for policy 0, policy_version 1623580 (0.0010) [2023-12-27 03:13:19,232][105692] Updated weights for policy 0, policy_version 1623590 (0.0011) [2023-12-27 03:13:19,295][105692] Updated weights for policy 0, policy_version 1623600 (0.0010) [2023-12-27 03:13:19,692][105620] Updated weights for policy 1, policy_version 1626986 (0.0010) [2023-12-27 03:13:19,751][105620] Updated weights for policy 1, policy_version 1626996 (0.0010) [2023-12-27 03:13:19,811][105620] Updated weights for policy 1, policy_version 1627006 (0.0011) [2023-12-27 03:13:20,040][105692] Updated weights for policy 0, policy_version 1623610 (0.0011) [2023-12-27 03:13:20,097][105692] Updated weights for policy 0, policy_version 1623620 (0.0011) [2023-12-27 03:13:20,151][105692] Updated weights for policy 0, policy_version 1623630 (0.0011) [2023-12-27 03:13:20,583][105620] Updated weights for policy 1, policy_version 1627016 (0.0009) [2023-12-27 03:13:20,648][105620] Updated weights for policy 1, policy_version 1627026 (0.0008) [2023-12-27 03:13:20,708][105620] Updated weights for policy 1, policy_version 1627036 (0.0011) [2023-12-27 03:13:20,912][105692] Updated weights for policy 0, policy_version 1623640 (0.0010) [2023-12-27 03:13:20,966][105692] Updated weights for policy 0, policy_version 1623650 (0.0010) [2023-12-27 03:13:21,022][105692] Updated weights for policy 0, policy_version 1623660 (0.0010) [2023-12-27 03:13:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 832299008. Throughput: 0: 9630.7, 1: 9580.8. Samples: 832285488. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:13:21,065][104569] Avg episode reward: [(0, '8438.925'), (1, '9261.246')] [2023-12-27 03:13:21,418][105620] Updated weights for policy 1, policy_version 1627046 (0.0008) [2023-12-27 03:13:21,487][105620] Updated weights for policy 1, policy_version 1627056 (0.0009) [2023-12-27 03:13:21,550][105620] Updated weights for policy 1, policy_version 1627066 (0.0008) [2023-12-27 03:13:21,795][105692] Updated weights for policy 0, policy_version 1623670 (0.0007) [2023-12-27 03:13:21,858][105692] Updated weights for policy 0, policy_version 1623680 (0.0006) [2023-12-27 03:13:21,924][105692] Updated weights for policy 0, policy_version 1623690 (0.0009) [2023-12-27 03:13:22,326][105620] Updated weights for policy 1, policy_version 1627076 (0.0009) [2023-12-27 03:13:22,394][105620] Updated weights for policy 1, policy_version 1627086 (0.0009) [2023-12-27 03:13:22,453][105620] Updated weights for policy 1, policy_version 1627096 (0.0009) [2023-12-27 03:13:22,654][105692] Updated weights for policy 0, policy_version 1623700 (0.0008) [2023-12-27 03:13:22,710][105692] Updated weights for policy 0, policy_version 1623710 (0.0006) [2023-12-27 03:13:22,768][105692] Updated weights for policy 0, policy_version 1623720 (0.0008) [2023-12-27 03:13:23,244][105620] Updated weights for policy 1, policy_version 1627106 (0.0010) [2023-12-27 03:13:23,307][105620] Updated weights for policy 1, policy_version 1627116 (0.0005) [2023-12-27 03:13:23,367][105620] Updated weights for policy 1, policy_version 1627126 (0.0008) [2023-12-27 03:13:23,427][105620] Updated weights for policy 1, policy_version 1627136 (0.0009) [2023-12-27 03:13:23,509][105692] Updated weights for policy 0, policy_version 1623730 (0.0009) [2023-12-27 03:13:23,561][105692] Updated weights for policy 0, policy_version 1623740 (0.0008) [2023-12-27 03:13:23,615][105692] Updated weights for policy 0, policy_version 1623750 (0.0006) [2023-12-27 03:13:23,664][105692] Updated weights for policy 0, policy_version 1623760 (0.0005) [2023-12-27 03:13:24,080][105620] Updated weights for policy 1, policy_version 1627146 (0.0010) [2023-12-27 03:13:24,146][105620] Updated weights for policy 1, policy_version 1627156 (0.0011) [2023-12-27 03:13:24,198][105620] Updated weights for policy 1, policy_version 1627166 (0.0010) [2023-12-27 03:13:24,381][105692] Updated weights for policy 0, policy_version 1623770 (0.0008) [2023-12-27 03:13:24,434][105692] Updated weights for policy 0, policy_version 1623780 (0.0008) [2023-12-27 03:13:24,487][105692] Updated weights for policy 0, policy_version 1623790 (0.0008) [2023-12-27 03:13:24,954][105620] Updated weights for policy 1, policy_version 1627176 (0.0009) [2023-12-27 03:13:25,009][105620] Updated weights for policy 1, policy_version 1627186 (0.0010) [2023-12-27 03:13:25,057][105620] Updated weights for policy 1, policy_version 1627196 (0.0010) [2023-12-27 03:13:25,186][105692] Updated weights for policy 0, policy_version 1623800 (0.0010) [2023-12-27 03:13:25,250][105692] Updated weights for policy 0, policy_version 1623810 (0.0010) [2023-12-27 03:13:25,315][105692] Updated weights for policy 0, policy_version 1623820 (0.0010) [2023-12-27 03:13:25,810][105620] Updated weights for policy 1, policy_version 1627206 (0.0010) [2023-12-27 03:13:25,878][105620] Updated weights for policy 1, policy_version 1627216 (0.0010) [2023-12-27 03:13:25,937][105620] Updated weights for policy 1, policy_version 1627226 (0.0010) [2023-12-27 03:13:25,938][105692] Updated weights for policy 0, policy_version 1623830 (0.0007) [2023-12-27 03:13:25,988][105692] Updated weights for policy 0, policy_version 1623840 (0.0006) [2023-12-27 03:13:26,044][105692] Updated weights for policy 0, policy_version 1623850 (0.0008) [2023-12-27 03:13:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 832389120. Throughput: 0: 9627.5, 1: 9486.5. Samples: 832398948. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:13:26,062][104569] Avg episode reward: [(0, '8535.532'), (1, '9261.510')] [2023-12-27 03:13:26,661][105620] Updated weights for policy 1, policy_version 1627236 (0.0008) [2023-12-27 03:13:26,680][105692] Updated weights for policy 0, policy_version 1623860 (0.0008) [2023-12-27 03:13:26,722][105620] Updated weights for policy 1, policy_version 1627246 (0.0005) [2023-12-27 03:13:26,733][105692] Updated weights for policy 0, policy_version 1623870 (0.0007) [2023-12-27 03:13:26,778][105620] Updated weights for policy 1, policy_version 1627256 (0.0005) [2023-12-27 03:13:26,792][105692] Updated weights for policy 0, policy_version 1623880 (0.0007) [2023-12-27 03:13:27,278][105620] Updated weights for policy 1, policy_version 1627266 (0.0005) [2023-12-27 03:13:27,326][105620] Updated weights for policy 1, policy_version 1627276 (0.0005) [2023-12-27 03:13:27,334][105692] Updated weights for policy 0, policy_version 1623890 (0.0005) [2023-12-27 03:13:27,381][105620] Updated weights for policy 1, policy_version 1627286 (0.0006) [2023-12-27 03:13:27,391][105692] Updated weights for policy 0, policy_version 1623900 (0.0005) [2023-12-27 03:13:27,435][105620] Updated weights for policy 1, policy_version 1627296 (0.0010) [2023-12-27 03:13:27,442][105692] Updated weights for policy 0, policy_version 1623910 (0.0005) [2023-12-27 03:13:27,497][105692] Updated weights for policy 0, policy_version 1623920 (0.0005) [2023-12-27 03:13:28,007][105620] Updated weights for policy 1, policy_version 1627306 (0.0010) [2023-12-27 03:13:28,064][105620] Updated weights for policy 1, policy_version 1627316 (0.0010) [2023-12-27 03:13:28,101][105692] Updated weights for policy 0, policy_version 1623930 (0.0010) [2023-12-27 03:13:28,124][105620] Updated weights for policy 1, policy_version 1627326 (0.0010) [2023-12-27 03:13:28,148][105692] Updated weights for policy 0, policy_version 1623940 (0.0010) [2023-12-27 03:13:28,219][105692] Updated weights for policy 0, policy_version 1623950 (0.0006) [2023-12-27 03:13:28,816][105620] Updated weights for policy 1, policy_version 1627336 (0.0006) [2023-12-27 03:13:28,876][105692] Updated weights for policy 0, policy_version 1623960 (0.0006) [2023-12-27 03:13:28,879][105620] Updated weights for policy 1, policy_version 1627346 (0.0006) [2023-12-27 03:13:28,937][105692] Updated weights for policy 0, policy_version 1623970 (0.0007) [2023-12-27 03:13:28,943][105620] Updated weights for policy 1, policy_version 1627356 (0.0006) [2023-12-27 03:13:28,989][105692] Updated weights for policy 0, policy_version 1623980 (0.0007) [2023-12-27 03:13:29,498][105620] Updated weights for policy 1, policy_version 1627366 (0.0006) [2023-12-27 03:13:29,553][105620] Updated weights for policy 1, policy_version 1627376 (0.0009) [2023-12-27 03:13:29,603][105620] Updated weights for policy 1, policy_version 1627386 (0.0009) [2023-12-27 03:13:29,725][105692] Updated weights for policy 0, policy_version 1623990 (0.0005) [2023-12-27 03:13:29,776][105692] Updated weights for policy 0, policy_version 1624000 (0.0005) [2023-12-27 03:13:29,830][105692] Updated weights for policy 0, policy_version 1624010 (0.0006) [2023-12-27 03:13:30,273][105620] Updated weights for policy 1, policy_version 1627396 (0.0007) [2023-12-27 03:13:30,338][105620] Updated weights for policy 1, policy_version 1627406 (0.0007) [2023-12-27 03:13:30,404][105620] Updated weights for policy 1, policy_version 1627416 (0.0009) [2023-12-27 03:13:30,524][105692] Updated weights for policy 0, policy_version 1624020 (0.0007) [2023-12-27 03:13:30,572][105692] Updated weights for policy 0, policy_version 1624030 (0.0009) [2023-12-27 03:13:30,632][105692] Updated weights for policy 0, policy_version 1624040 (0.0007) [2023-12-27 03:13:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 832495616. Throughput: 0: 9784.4, 1: 9578.5. Samples: 832466380. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:13:31,063][104569] Avg episode reward: [(0, '8624.135'), (1, '9262.269')] [2023-12-27 03:13:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001624048_415817728.pth... [2023-12-27 03:13:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001627424_416677888.pth... [2023-12-27 03:13:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001626304_416391168.pth [2023-12-27 03:13:31,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001622896_415522816.pth [2023-12-27 03:13:31,168][105620] Updated weights for policy 1, policy_version 1627426 (0.0009) [2023-12-27 03:13:31,215][105620] Updated weights for policy 1, policy_version 1627436 (0.0009) [2023-12-27 03:13:31,265][105620] Updated weights for policy 1, policy_version 1627446 (0.0008) [2023-12-27 03:13:31,324][105620] Updated weights for policy 1, policy_version 1627456 (0.0009) [2023-12-27 03:13:31,397][105692] Updated weights for policy 0, policy_version 1624050 (0.0009) [2023-12-27 03:13:31,461][105692] Updated weights for policy 0, policy_version 1624060 (0.0009) [2023-12-27 03:13:31,523][105692] Updated weights for policy 0, policy_version 1624070 (0.0009) [2023-12-27 03:13:31,591][105692] Updated weights for policy 0, policy_version 1624080 (0.0010) [2023-12-27 03:13:32,072][105620] Updated weights for policy 1, policy_version 1627466 (0.0009) [2023-12-27 03:13:32,131][105620] Updated weights for policy 1, policy_version 1627476 (0.0009) [2023-12-27 03:13:32,179][105620] Updated weights for policy 1, policy_version 1627486 (0.0009) [2023-12-27 03:13:32,324][105692] Updated weights for policy 0, policy_version 1624090 (0.0007) [2023-12-27 03:13:32,393][105692] Updated weights for policy 0, policy_version 1624100 (0.0008) [2023-12-27 03:13:32,457][105692] Updated weights for policy 0, policy_version 1624110 (0.0009) [2023-12-27 03:13:32,977][105620] Updated weights for policy 1, policy_version 1627496 (0.0009) [2023-12-27 03:13:33,038][105620] Updated weights for policy 1, policy_version 1627506 (0.0009) [2023-12-27 03:13:33,101][105620] Updated weights for policy 1, policy_version 1627516 (0.0008) [2023-12-27 03:13:33,110][105692] Updated weights for policy 0, policy_version 1624120 (0.0006) [2023-12-27 03:13:33,163][105692] Updated weights for policy 0, policy_version 1624130 (0.0005) [2023-12-27 03:13:33,217][105692] Updated weights for policy 0, policy_version 1624140 (0.0006) [2023-12-27 03:13:33,759][105692] Updated weights for policy 0, policy_version 1624150 (0.0005) [2023-12-27 03:13:33,801][105692] Updated weights for policy 0, policy_version 1624160 (0.0005) [2023-12-27 03:13:33,847][105692] Updated weights for policy 0, policy_version 1624170 (0.0005) [2023-12-27 03:13:33,964][105620] Updated weights for policy 1, policy_version 1627526 (0.0009) [2023-12-27 03:13:34,018][105620] Updated weights for policy 1, policy_version 1627536 (0.0009) [2023-12-27 03:13:34,065][105620] Updated weights for policy 1, policy_version 1627546 (0.0009) [2023-12-27 03:13:34,561][105692] Updated weights for policy 0, policy_version 1624180 (0.0007) [2023-12-27 03:13:34,609][105692] Updated weights for policy 0, policy_version 1624190 (0.0009) [2023-12-27 03:13:34,660][105692] Updated weights for policy 0, policy_version 1624200 (0.0008) [2023-12-27 03:13:34,840][105620] Updated weights for policy 1, policy_version 1627556 (0.0009) [2023-12-27 03:13:34,899][105620] Updated weights for policy 1, policy_version 1627566 (0.0009) [2023-12-27 03:13:34,959][105620] Updated weights for policy 1, policy_version 1627576 (0.0009) [2023-12-27 03:13:35,406][105692] Updated weights for policy 0, policy_version 1624210 (0.0009) [2023-12-27 03:13:35,466][105692] Updated weights for policy 0, policy_version 1624220 (0.0007) [2023-12-27 03:13:35,533][105692] Updated weights for policy 0, policy_version 1624230 (0.0005) [2023-12-27 03:13:35,589][105692] Updated weights for policy 0, policy_version 1624240 (0.0008) [2023-12-27 03:13:35,745][105620] Updated weights for policy 1, policy_version 1627586 (0.0010) [2023-12-27 03:13:35,793][105620] Updated weights for policy 1, policy_version 1627596 (0.0009) [2023-12-27 03:13:35,839][105620] Updated weights for policy 1, policy_version 1627606 (0.0008) [2023-12-27 03:13:35,886][105620] Updated weights for policy 1, policy_version 1627616 (0.0009) [2023-12-27 03:13:36,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 832593920. Throughput: 0: 9891.9, 1: 9478.1. Samples: 832582496. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:13:36,063][104569] Avg episode reward: [(0, '8893.682'), (1, '9262.203')] [2023-12-27 03:13:36,286][105692] Updated weights for policy 0, policy_version 1624250 (0.0009) [2023-12-27 03:13:36,349][105692] Updated weights for policy 0, policy_version 1624260 (0.0009) [2023-12-27 03:13:36,411][105692] Updated weights for policy 0, policy_version 1624270 (0.0008) [2023-12-27 03:13:36,698][105620] Updated weights for policy 1, policy_version 1627626 (0.0009) [2023-12-27 03:13:36,762][105620] Updated weights for policy 1, policy_version 1627636 (0.0006) [2023-12-27 03:13:36,827][105620] Updated weights for policy 1, policy_version 1627646 (0.0007) [2023-12-27 03:13:37,229][105692] Updated weights for policy 0, policy_version 1624280 (0.0009) [2023-12-27 03:13:37,288][105692] Updated weights for policy 0, policy_version 1624290 (0.0005) [2023-12-27 03:13:37,347][105692] Updated weights for policy 0, policy_version 1624300 (0.0007) [2023-12-27 03:13:37,463][105620] Updated weights for policy 1, policy_version 1627656 (0.0010) [2023-12-27 03:13:37,528][105620] Updated weights for policy 1, policy_version 1627666 (0.0011) [2023-12-27 03:13:37,603][105620] Updated weights for policy 1, policy_version 1627676 (0.0011) [2023-12-27 03:13:37,932][105692] Updated weights for policy 0, policy_version 1624310 (0.0006) [2023-12-27 03:13:37,989][105692] Updated weights for policy 0, policy_version 1624320 (0.0006) [2023-12-27 03:13:38,054][105692] Updated weights for policy 0, policy_version 1624330 (0.0007) [2023-12-27 03:13:38,229][105620] Updated weights for policy 1, policy_version 1627686 (0.0011) [2023-12-27 03:13:38,288][105620] Updated weights for policy 1, policy_version 1627696 (0.0010) [2023-12-27 03:13:38,360][105620] Updated weights for policy 1, policy_version 1627707 (0.0009) [2023-12-27 03:13:38,742][105692] Updated weights for policy 0, policy_version 1624340 (0.0010) [2023-12-27 03:13:38,804][105692] Updated weights for policy 0, policy_version 1624350 (0.0009) [2023-12-27 03:13:38,864][105692] Updated weights for policy 0, policy_version 1624360 (0.0009) [2023-12-27 03:13:39,025][105620] Updated weights for policy 1, policy_version 1627717 (0.0008) [2023-12-27 03:13:39,083][105620] Updated weights for policy 1, policy_version 1627727 (0.0009) [2023-12-27 03:13:39,134][105620] Updated weights for policy 1, policy_version 1627737 (0.0009) [2023-12-27 03:13:39,714][105692] Updated weights for policy 0, policy_version 1624370 (0.0010) [2023-12-27 03:13:39,780][105692] Updated weights for policy 0, policy_version 1624380 (0.0009) [2023-12-27 03:13:39,842][105692] Updated weights for policy 0, policy_version 1624390 (0.0008) [2023-12-27 03:13:39,891][105620] Updated weights for policy 1, policy_version 1627747 (0.0008) [2023-12-27 03:13:39,906][105692] Updated weights for policy 0, policy_version 1624400 (0.0008) [2023-12-27 03:13:39,954][105620] Updated weights for policy 1, policy_version 1627757 (0.0008) [2023-12-27 03:13:40,013][105620] Updated weights for policy 1, policy_version 1627767 (0.0010) [2023-12-27 03:13:40,671][105692] Updated weights for policy 0, policy_version 1624410 (0.0009) [2023-12-27 03:13:40,730][105692] Updated weights for policy 0, policy_version 1624420 (0.0009) [2023-12-27 03:13:40,789][105620] Updated weights for policy 1, policy_version 1627777 (0.0009) [2023-12-27 03:13:40,791][105692] Updated weights for policy 0, policy_version 1624430 (0.0008) [2023-12-27 03:13:40,846][105620] Updated weights for policy 1, policy_version 1627787 (0.0008) [2023-12-27 03:13:40,898][105620] Updated weights for policy 1, policy_version 1627797 (0.0008) [2023-12-27 03:13:40,944][105620] Updated weights for policy 1, policy_version 1627807 (0.0008) [2023-12-27 03:13:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 832692224. Throughput: 0: 9899.7, 1: 9529.1. Samples: 832697328. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:13:41,062][104569] Avg episode reward: [(0, '8805.438'), (1, '9172.714')] [2023-12-27 03:13:41,549][105692] Updated weights for policy 0, policy_version 1624440 (0.0005) [2023-12-27 03:13:41,617][105692] Updated weights for policy 0, policy_version 1624450 (0.0008) [2023-12-27 03:13:41,680][105692] Updated weights for policy 0, policy_version 1624460 (0.0009) [2023-12-27 03:13:41,769][105620] Updated weights for policy 1, policy_version 1627817 (0.0009) [2023-12-27 03:13:41,833][105620] Updated weights for policy 1, policy_version 1627827 (0.0008) [2023-12-27 03:13:41,895][105620] Updated weights for policy 1, policy_version 1627837 (0.0006) [2023-12-27 03:13:42,374][105692] Updated weights for policy 0, policy_version 1624470 (0.0009) [2023-12-27 03:13:42,433][105692] Updated weights for policy 0, policy_version 1624480 (0.0008) [2023-12-27 03:13:42,491][105692] Updated weights for policy 0, policy_version 1624490 (0.0008) [2023-12-27 03:13:42,625][105620] Updated weights for policy 1, policy_version 1627847 (0.0008) [2023-12-27 03:13:42,682][105620] Updated weights for policy 1, policy_version 1627857 (0.0010) [2023-12-27 03:13:42,740][105620] Updated weights for policy 1, policy_version 1627867 (0.0009) [2023-12-27 03:13:43,178][105692] Updated weights for policy 0, policy_version 1624500 (0.0009) [2023-12-27 03:13:43,233][105692] Updated weights for policy 0, policy_version 1624510 (0.0006) [2023-12-27 03:13:43,289][105692] Updated weights for policy 0, policy_version 1624520 (0.0008) [2023-12-27 03:13:43,453][105620] Updated weights for policy 1, policy_version 1627877 (0.0008) [2023-12-27 03:13:43,504][105620] Updated weights for policy 1, policy_version 1627887 (0.0009) [2023-12-27 03:13:43,558][105620] Updated weights for policy 1, policy_version 1627897 (0.0008) [2023-12-27 03:13:43,942][105692] Updated weights for policy 0, policy_version 1624530 (0.0009) [2023-12-27 03:13:43,990][105692] Updated weights for policy 0, policy_version 1624540 (0.0006) [2023-12-27 03:13:44,044][105692] Updated weights for policy 0, policy_version 1624550 (0.0005) [2023-12-27 03:13:44,095][105692] Updated weights for policy 0, policy_version 1624560 (0.0005) [2023-12-27 03:13:44,362][105620] Updated weights for policy 1, policy_version 1627907 (0.0009) [2023-12-27 03:13:44,420][105620] Updated weights for policy 1, policy_version 1627917 (0.0010) [2023-12-27 03:13:44,475][105620] Updated weights for policy 1, policy_version 1627927 (0.0010) [2023-12-27 03:13:44,753][105692] Updated weights for policy 0, policy_version 1624570 (0.0010) [2023-12-27 03:13:44,818][105692] Updated weights for policy 0, policy_version 1624580 (0.0009) [2023-12-27 03:13:44,874][105692] Updated weights for policy 0, policy_version 1624590 (0.0008) [2023-12-27 03:13:45,167][105620] Updated weights for policy 1, policy_version 1627937 (0.0010) [2023-12-27 03:13:45,225][105620] Updated weights for policy 1, policy_version 1627947 (0.0005) [2023-12-27 03:13:45,283][105620] Updated weights for policy 1, policy_version 1627957 (0.0006) [2023-12-27 03:13:45,341][105620] Updated weights for policy 1, policy_version 1627967 (0.0005) [2023-12-27 03:13:45,620][105692] Updated weights for policy 0, policy_version 1624600 (0.0007) [2023-12-27 03:13:45,671][105692] Updated weights for policy 0, policy_version 1624610 (0.0005) [2023-12-27 03:13:45,725][105692] Updated weights for policy 0, policy_version 1624620 (0.0005) [2023-12-27 03:13:45,981][105620] Updated weights for policy 1, policy_version 1627977 (0.0007) [2023-12-27 03:13:46,046][105620] Updated weights for policy 1, policy_version 1627987 (0.0010) [2023-12-27 03:13:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.1, 300 sec: 19494.2). Total num frames: 832782336. Throughput: 0: 9844.2, 1: 9520.3. Samples: 832753412. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:13:46,063][104569] Avg episode reward: [(0, '8349.513'), (1, '9089.969')] [2023-12-27 03:13:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001624624_415965184.pth... [2023-12-27 03:13:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001623472_415670272.pth [2023-12-27 03:13:46,109][105620] Updated weights for policy 1, policy_version 1627997 (0.0008) [2023-12-27 03:13:46,126][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001628000_416825344.pth... [2023-12-27 03:13:46,129][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001626848_416530432.pth [2023-12-27 03:13:46,367][105692] Updated weights for policy 0, policy_version 1624630 (0.0008) [2023-12-27 03:13:46,415][105692] Updated weights for policy 0, policy_version 1624640 (0.0010) [2023-12-27 03:13:46,478][105692] Updated weights for policy 0, policy_version 1624650 (0.0010) [2023-12-27 03:13:46,777][105620] Updated weights for policy 1, policy_version 1628007 (0.0007) [2023-12-27 03:13:46,830][105620] Updated weights for policy 1, policy_version 1628017 (0.0009) [2023-12-27 03:13:46,893][105620] Updated weights for policy 1, policy_version 1628027 (0.0010) [2023-12-27 03:13:47,154][105692] Updated weights for policy 0, policy_version 1624660 (0.0011) [2023-12-27 03:13:47,209][105692] Updated weights for policy 0, policy_version 1624670 (0.0010) [2023-12-27 03:13:47,265][105692] Updated weights for policy 0, policy_version 1624680 (0.0011) [2023-12-27 03:13:47,703][105620] Updated weights for policy 1, policy_version 1628037 (0.0010) [2023-12-27 03:13:47,752][105620] Updated weights for policy 1, policy_version 1628047 (0.0008) [2023-12-27 03:13:47,804][105620] Updated weights for policy 1, policy_version 1628057 (0.0009) [2023-12-27 03:13:47,928][105692] Updated weights for policy 0, policy_version 1624690 (0.0009) [2023-12-27 03:13:47,976][105692] Updated weights for policy 0, policy_version 1624700 (0.0010) [2023-12-27 03:13:48,027][105692] Updated weights for policy 0, policy_version 1624710 (0.0010) [2023-12-27 03:13:48,082][105692] Updated weights for policy 0, policy_version 1624720 (0.0010) [2023-12-27 03:13:48,580][105620] Updated weights for policy 1, policy_version 1628067 (0.0008) [2023-12-27 03:13:48,635][105620] Updated weights for policy 1, policy_version 1628077 (0.0008) [2023-12-27 03:13:48,695][105620] Updated weights for policy 1, policy_version 1628087 (0.0008) [2023-12-27 03:13:48,865][105692] Updated weights for policy 0, policy_version 1624730 (0.0010) [2023-12-27 03:13:48,924][105692] Updated weights for policy 0, policy_version 1624740 (0.0010) [2023-12-27 03:13:48,987][105692] Updated weights for policy 0, policy_version 1624750 (0.0011) [2023-12-27 03:13:49,496][105620] Updated weights for policy 1, policy_version 1628097 (0.0009) [2023-12-27 03:13:49,552][105620] Updated weights for policy 1, policy_version 1628107 (0.0007) [2023-12-27 03:13:49,611][105620] Updated weights for policy 1, policy_version 1628117 (0.0008) [2023-12-27 03:13:49,679][105620] Updated weights for policy 1, policy_version 1628127 (0.0008) [2023-12-27 03:13:49,766][105692] Updated weights for policy 0, policy_version 1624760 (0.0011) [2023-12-27 03:13:49,820][105692] Updated weights for policy 0, policy_version 1624770 (0.0010) [2023-12-27 03:13:49,888][105692] Updated weights for policy 0, policy_version 1624780 (0.0007) [2023-12-27 03:13:50,380][105620] Updated weights for policy 1, policy_version 1628137 (0.0007) [2023-12-27 03:13:50,437][105620] Updated weights for policy 1, policy_version 1628147 (0.0006) [2023-12-27 03:13:50,501][105620] Updated weights for policy 1, policy_version 1628157 (0.0005) [2023-12-27 03:13:50,691][105692] Updated weights for policy 0, policy_version 1624790 (0.0010) [2023-12-27 03:13:50,746][105692] Updated weights for policy 0, policy_version 1624800 (0.0010) [2023-12-27 03:13:50,814][105692] Updated weights for policy 0, policy_version 1624810 (0.0010) [2023-12-27 03:13:51,059][105620] Updated weights for policy 1, policy_version 1628167 (0.0007) [2023-12-27 03:13:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 832880640. Throughput: 0: 9969.6, 1: 9548.9. Samples: 832870912. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:13:51,062][104569] Avg episode reward: [(0, '8533.271'), (1, '9270.477')] [2023-12-27 03:13:51,125][105620] Updated weights for policy 1, policy_version 1628177 (0.0008) [2023-12-27 03:13:51,187][105620] Updated weights for policy 1, policy_version 1628187 (0.0008) [2023-12-27 03:13:51,626][105692] Updated weights for policy 0, policy_version 1624820 (0.0009) [2023-12-27 03:13:51,683][105692] Updated weights for policy 0, policy_version 1624830 (0.0009) [2023-12-27 03:13:51,750][105692] Updated weights for policy 0, policy_version 1624840 (0.0008) [2023-12-27 03:13:51,864][105620] Updated weights for policy 1, policy_version 1628197 (0.0008) [2023-12-27 03:13:51,925][105620] Updated weights for policy 1, policy_version 1628207 (0.0005) [2023-12-27 03:13:51,983][105620] Updated weights for policy 1, policy_version 1628217 (0.0005) [2023-12-27 03:13:52,552][105692] Updated weights for policy 0, policy_version 1624850 (0.0008) [2023-12-27 03:13:52,608][105692] Updated weights for policy 0, policy_version 1624861 (0.0012) [2023-12-27 03:13:52,655][105692] Updated weights for policy 0, policy_version 1624871 (0.0009) [2023-12-27 03:13:52,669][105620] Updated weights for policy 1, policy_version 1628227 (0.0006) [2023-12-27 03:13:52,718][105620] Updated weights for policy 1, policy_version 1628237 (0.0008) [2023-12-27 03:13:52,767][105620] Updated weights for policy 1, policy_version 1628248 (0.0010) [2023-12-27 03:13:53,272][105692] Updated weights for policy 0, policy_version 1624881 (0.0005) [2023-12-27 03:13:53,321][105692] Updated weights for policy 0, policy_version 1624891 (0.0005) [2023-12-27 03:13:53,371][105692] Updated weights for policy 0, policy_version 1624901 (0.0005) [2023-12-27 03:13:53,378][105620] Updated weights for policy 1, policy_version 1628258 (0.0009) [2023-12-27 03:13:53,433][105692] Updated weights for policy 0, policy_version 1624911 (0.0005) [2023-12-27 03:13:53,442][105620] Updated weights for policy 1, policy_version 1628268 (0.0010) [2023-12-27 03:13:53,493][105620] Updated weights for policy 1, policy_version 1628278 (0.0010) [2023-12-27 03:13:53,545][105620] Updated weights for policy 1, policy_version 1628288 (0.0009) [2023-12-27 03:13:54,056][105692] Updated weights for policy 0, policy_version 1624921 (0.0009) [2023-12-27 03:13:54,104][105692] Updated weights for policy 0, policy_version 1624931 (0.0009) [2023-12-27 03:13:54,158][105692] Updated weights for policy 0, policy_version 1624941 (0.0009) [2023-12-27 03:13:54,316][105620] Updated weights for policy 1, policy_version 1628298 (0.0009) [2023-12-27 03:13:54,370][105620] Updated weights for policy 1, policy_version 1628308 (0.0009) [2023-12-27 03:13:54,424][105620] Updated weights for policy 1, policy_version 1628318 (0.0008) [2023-12-27 03:13:54,887][105692] Updated weights for policy 0, policy_version 1624951 (0.0006) [2023-12-27 03:13:54,942][105692] Updated weights for policy 0, policy_version 1624961 (0.0008) [2023-12-27 03:13:55,002][105692] Updated weights for policy 0, policy_version 1624971 (0.0007) [2023-12-27 03:13:55,259][105620] Updated weights for policy 1, policy_version 1628328 (0.0009) [2023-12-27 03:13:55,307][105620] Updated weights for policy 1, policy_version 1628338 (0.0008) [2023-12-27 03:13:55,354][105620] Updated weights for policy 1, policy_version 1628348 (0.0009) [2023-12-27 03:13:55,699][105692] Updated weights for policy 0, policy_version 1624981 (0.0008) [2023-12-27 03:13:55,747][105692] Updated weights for policy 0, policy_version 1624991 (0.0007) [2023-12-27 03:13:55,792][105692] Updated weights for policy 0, policy_version 1625001 (0.0008) [2023-12-27 03:13:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 832978944. Throughput: 0: 9901.8, 1: 9592.4. Samples: 832989360. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:13:56,063][104569] Avg episode reward: [(0, '9081.146'), (1, '8989.133')] [2023-12-27 03:13:56,095][105620] Updated weights for policy 1, policy_version 1628358 (0.0009) [2023-12-27 03:13:56,146][105620] Updated weights for policy 1, policy_version 1628368 (0.0009) [2023-12-27 03:13:56,206][105620] Updated weights for policy 1, policy_version 1628378 (0.0008) [2023-12-27 03:13:56,239][105586] KL-divergence is very high: 101.8132 [2023-12-27 03:13:56,518][105692] Updated weights for policy 0, policy_version 1625011 (0.0008) [2023-12-27 03:13:56,574][105692] Updated weights for policy 0, policy_version 1625021 (0.0005) [2023-12-27 03:13:56,623][105692] Updated weights for policy 0, policy_version 1625031 (0.0005) [2023-12-27 03:13:57,008][105620] Updated weights for policy 1, policy_version 1628388 (0.0010) [2023-12-27 03:13:57,071][105620] Updated weights for policy 1, policy_version 1628398 (0.0010) [2023-12-27 03:13:57,136][105620] Updated weights for policy 1, policy_version 1628408 (0.0010) [2023-12-27 03:13:57,260][105692] Updated weights for policy 0, policy_version 1625041 (0.0006) [2023-12-27 03:13:57,328][105692] Updated weights for policy 0, policy_version 1625051 (0.0009) [2023-12-27 03:13:57,385][105692] Updated weights for policy 0, policy_version 1625061 (0.0006) [2023-12-27 03:13:57,441][105692] Updated weights for policy 0, policy_version 1625071 (0.0007) [2023-12-27 03:13:57,759][105620] Updated weights for policy 1, policy_version 1628418 (0.0009) [2023-12-27 03:13:57,810][105620] Updated weights for policy 1, policy_version 1628428 (0.0005) [2023-12-27 03:13:57,861][105620] Updated weights for policy 1, policy_version 1628438 (0.0005) [2023-12-27 03:13:57,909][105620] Updated weights for policy 1, policy_version 1628448 (0.0005) [2023-12-27 03:13:58,021][105692] Updated weights for policy 0, policy_version 1625081 (0.0006) [2023-12-27 03:13:58,070][105692] Updated weights for policy 0, policy_version 1625091 (0.0007) [2023-12-27 03:13:58,128][105692] Updated weights for policy 0, policy_version 1625101 (0.0010) [2023-12-27 03:13:58,588][105620] Updated weights for policy 1, policy_version 1628458 (0.0007) [2023-12-27 03:13:58,649][105620] Updated weights for policy 1, policy_version 1628468 (0.0008) [2023-12-27 03:13:58,713][105620] Updated weights for policy 1, policy_version 1628478 (0.0008) [2023-12-27 03:13:58,939][105692] Updated weights for policy 0, policy_version 1625111 (0.0009) [2023-12-27 03:13:58,998][105692] Updated weights for policy 0, policy_version 1625121 (0.0009) [2023-12-27 03:13:59,063][105692] Updated weights for policy 0, policy_version 1625131 (0.0009) [2023-12-27 03:13:59,599][105620] Updated weights for policy 1, policy_version 1628488 (0.0010) [2023-12-27 03:13:59,660][105620] Updated weights for policy 1, policy_version 1628498 (0.0011) [2023-12-27 03:13:59,714][105620] Updated weights for policy 1, policy_version 1628508 (0.0010) [2023-12-27 03:13:59,881][105692] Updated weights for policy 0, policy_version 1625141 (0.0008) [2023-12-27 03:13:59,944][105692] Updated weights for policy 0, policy_version 1625151 (0.0009) [2023-12-27 03:14:00,008][105692] Updated weights for policy 0, policy_version 1625161 (0.0010) [2023-12-27 03:14:00,425][105620] Updated weights for policy 1, policy_version 1628518 (0.0010) [2023-12-27 03:14:00,483][105620] Updated weights for policy 1, policy_version 1628528 (0.0010) [2023-12-27 03:14:00,538][105620] Updated weights for policy 1, policy_version 1628538 (0.0010) [2023-12-27 03:14:00,741][105692] Updated weights for policy 0, policy_version 1625171 (0.0010) [2023-12-27 03:14:00,797][105692] Updated weights for policy 0, policy_version 1625181 (0.0010) [2023-12-27 03:14:00,851][105692] Updated weights for policy 0, policy_version 1625191 (0.0010) [2023-12-27 03:14:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 833077248. Throughput: 0: 9917.3, 1: 9647.8. Samples: 833049228. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:14:01,063][104569] Avg episode reward: [(0, '9263.842'), (1, '8988.987')] [2023-12-27 03:14:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001625200_416112640.pth... [2023-12-27 03:14:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001628544_416964608.pth... [2023-12-27 03:14:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001624048_415817728.pth [2023-12-27 03:14:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001627424_416677888.pth [2023-12-27 03:14:01,269][105620] Updated weights for policy 1, policy_version 1628548 (0.0009) [2023-12-27 03:14:01,320][105620] Updated weights for policy 1, policy_version 1628558 (0.0008) [2023-12-27 03:14:01,382][105620] Updated weights for policy 1, policy_version 1628568 (0.0008) [2023-12-27 03:14:01,566][105692] Updated weights for policy 0, policy_version 1625201 (0.0008) [2023-12-27 03:14:01,634][105692] Updated weights for policy 0, policy_version 1625211 (0.0008) [2023-12-27 03:14:01,701][105692] Updated weights for policy 0, policy_version 1625221 (0.0006) [2023-12-27 03:14:01,766][105692] Updated weights for policy 0, policy_version 1625231 (0.0006) [2023-12-27 03:14:02,117][105620] Updated weights for policy 1, policy_version 1628578 (0.0008) [2023-12-27 03:14:02,174][105620] Updated weights for policy 1, policy_version 1628588 (0.0011) [2023-12-27 03:14:02,235][105620] Updated weights for policy 1, policy_version 1628598 (0.0011) [2023-12-27 03:14:02,295][105620] Updated weights for policy 1, policy_version 1628608 (0.0011) [2023-12-27 03:14:02,306][105692] Updated weights for policy 0, policy_version 1625241 (0.0009) [2023-12-27 03:14:02,368][105692] Updated weights for policy 0, policy_version 1625251 (0.0009) [2023-12-27 03:14:02,424][105692] Updated weights for policy 0, policy_version 1625261 (0.0008) [2023-12-27 03:14:02,944][105620] Updated weights for policy 1, policy_version 1628618 (0.0006) [2023-12-27 03:14:02,997][105620] Updated weights for policy 1, policy_version 1628628 (0.0005) [2023-12-27 03:14:03,046][105620] Updated weights for policy 1, policy_version 1628638 (0.0005) [2023-12-27 03:14:03,130][105692] Updated weights for policy 0, policy_version 1625271 (0.0006) [2023-12-27 03:14:03,185][105692] Updated weights for policy 0, policy_version 1625281 (0.0006) [2023-12-27 03:14:03,242][105692] Updated weights for policy 0, policy_version 1625291 (0.0010) [2023-12-27 03:14:03,599][105620] Updated weights for policy 1, policy_version 1628648 (0.0005) [2023-12-27 03:14:03,655][105620] Updated weights for policy 1, policy_version 1628658 (0.0005) [2023-12-27 03:14:03,709][105620] Updated weights for policy 1, policy_version 1628668 (0.0005) [2023-12-27 03:14:03,970][105692] Updated weights for policy 0, policy_version 1625301 (0.0009) [2023-12-27 03:14:04,023][105692] Updated weights for policy 0, policy_version 1625311 (0.0006) [2023-12-27 03:14:04,082][105692] Updated weights for policy 0, policy_version 1625321 (0.0008) [2023-12-27 03:14:04,378][105620] Updated weights for policy 1, policy_version 1628678 (0.0008) [2023-12-27 03:14:04,427][105620] Updated weights for policy 1, policy_version 1628688 (0.0010) [2023-12-27 03:14:04,490][105620] Updated weights for policy 1, policy_version 1628698 (0.0009) [2023-12-27 03:14:04,813][105692] Updated weights for policy 0, policy_version 1625331 (0.0006) [2023-12-27 03:14:04,863][105692] Updated weights for policy 0, policy_version 1625341 (0.0005) [2023-12-27 03:14:04,913][105692] Updated weights for policy 0, policy_version 1625351 (0.0005) [2023-12-27 03:14:05,196][105620] Updated weights for policy 1, policy_version 1628708 (0.0009) [2023-12-27 03:14:05,254][105620] Updated weights for policy 1, policy_version 1628718 (0.0008) [2023-12-27 03:14:05,317][105620] Updated weights for policy 1, policy_version 1628728 (0.0009) [2023-12-27 03:14:05,670][105692] Updated weights for policy 0, policy_version 1625361 (0.0006) [2023-12-27 03:14:05,722][105692] Updated weights for policy 0, policy_version 1625371 (0.0009) [2023-12-27 03:14:05,769][105692] Updated weights for policy 0, policy_version 1625381 (0.0008) [2023-12-27 03:14:05,837][105692] Updated weights for policy 0, policy_version 1625391 (0.0005) [2023-12-27 03:14:05,989][105620] Updated weights for policy 1, policy_version 1628738 (0.0009) [2023-12-27 03:14:06,050][105620] Updated weights for policy 1, policy_version 1628748 (0.0009) [2023-12-27 03:14:06,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 833175552. Throughput: 0: 9864.8, 1: 9708.6. Samples: 833166288. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:14:06,062][104569] Avg episode reward: [(0, '9081.900'), (1, '9352.728')] [2023-12-27 03:14:06,118][105620] Updated weights for policy 1, policy_version 1628758 (0.0008) [2023-12-27 03:14:06,177][105620] Updated weights for policy 1, policy_version 1628768 (0.0008) [2023-12-27 03:14:06,583][105692] Updated weights for policy 0, policy_version 1625401 (0.0009) [2023-12-27 03:14:06,649][105692] Updated weights for policy 0, policy_version 1625411 (0.0009) [2023-12-27 03:14:06,711][105692] Updated weights for policy 0, policy_version 1625421 (0.0009) [2023-12-27 03:14:06,957][105620] Updated weights for policy 1, policy_version 1628778 (0.0010) [2023-12-27 03:14:07,021][105620] Updated weights for policy 1, policy_version 1628788 (0.0010) [2023-12-27 03:14:07,074][105620] Updated weights for policy 1, policy_version 1628798 (0.0009) [2023-12-27 03:14:07,283][105692] Updated weights for policy 0, policy_version 1625431 (0.0006) [2023-12-27 03:14:07,343][105692] Updated weights for policy 0, policy_version 1625441 (0.0005) [2023-12-27 03:14:07,411][105692] Updated weights for policy 0, policy_version 1625451 (0.0005) [2023-12-27 03:14:07,764][105620] Updated weights for policy 1, policy_version 1628808 (0.0007) [2023-12-27 03:14:07,811][105620] Updated weights for policy 1, policy_version 1628818 (0.0009) [2023-12-27 03:14:07,856][105620] Updated weights for policy 1, policy_version 1628828 (0.0008) [2023-12-27 03:14:07,936][105692] Updated weights for policy 0, policy_version 1625461 (0.0008) [2023-12-27 03:14:07,989][105692] Updated weights for policy 0, policy_version 1625472 (0.0010) [2023-12-27 03:14:08,042][105692] Updated weights for policy 0, policy_version 1625482 (0.0006) [2023-12-27 03:14:08,606][105620] Updated weights for policy 1, policy_version 1628838 (0.0007) [2023-12-27 03:14:08,662][105620] Updated weights for policy 1, policy_version 1628848 (0.0008) [2023-12-27 03:14:08,721][105620] Updated weights for policy 1, policy_version 1628858 (0.0008) [2023-12-27 03:14:08,825][105692] Updated weights for policy 0, policy_version 1625492 (0.0007) [2023-12-27 03:14:08,887][105692] Updated weights for policy 0, policy_version 1625502 (0.0010) [2023-12-27 03:14:08,951][105692] Updated weights for policy 0, policy_version 1625512 (0.0011) [2023-12-27 03:14:09,448][105620] Updated weights for policy 1, policy_version 1628868 (0.0008) [2023-12-27 03:14:09,509][105620] Updated weights for policy 1, policy_version 1628878 (0.0009) [2023-12-27 03:14:09,564][105620] Updated weights for policy 1, policy_version 1628888 (0.0009) [2023-12-27 03:14:09,628][105692] Updated weights for policy 0, policy_version 1625522 (0.0007) [2023-12-27 03:14:09,687][105692] Updated weights for policy 0, policy_version 1625532 (0.0009) [2023-12-27 03:14:09,745][105692] Updated weights for policy 0, policy_version 1625542 (0.0009) [2023-12-27 03:14:09,809][105692] Updated weights for policy 0, policy_version 1625552 (0.0007) [2023-12-27 03:14:10,366][105620] Updated weights for policy 1, policy_version 1628898 (0.0008) [2023-12-27 03:14:10,416][105620] Updated weights for policy 1, policy_version 1628908 (0.0006) [2023-12-27 03:14:10,476][105620] Updated weights for policy 1, policy_version 1628918 (0.0006) [2023-12-27 03:14:10,537][105620] Updated weights for policy 1, policy_version 1628928 (0.0008) [2023-12-27 03:14:10,589][105692] Updated weights for policy 0, policy_version 1625562 (0.0007) [2023-12-27 03:14:10,648][105692] Updated weights for policy 0, policy_version 1625572 (0.0009) [2023-12-27 03:14:10,708][105692] Updated weights for policy 0, policy_version 1625582 (0.0009) [2023-12-27 03:14:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 833273856. Throughput: 0: 9922.4, 1: 9768.4. Samples: 833285036. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:14:11,062][104569] Avg episode reward: [(0, '8529.372'), (1, '9262.847')] [2023-12-27 03:14:11,173][105620] Updated weights for policy 1, policy_version 1628938 (0.0010) [2023-12-27 03:14:11,236][105620] Updated weights for policy 1, policy_version 1628948 (0.0011) [2023-12-27 03:14:11,296][105620] Updated weights for policy 1, policy_version 1628958 (0.0011) [2023-12-27 03:14:11,507][105692] Updated weights for policy 0, policy_version 1625592 (0.0008) [2023-12-27 03:14:11,566][105692] Updated weights for policy 0, policy_version 1625602 (0.0008) [2023-12-27 03:14:11,622][105692] Updated weights for policy 0, policy_version 1625612 (0.0008) [2023-12-27 03:14:12,066][105620] Updated weights for policy 1, policy_version 1628968 (0.0011) [2023-12-27 03:14:12,118][105620] Updated weights for policy 1, policy_version 1628978 (0.0010) [2023-12-27 03:14:12,180][105620] Updated weights for policy 1, policy_version 1628988 (0.0010) [2023-12-27 03:14:12,424][105692] Updated weights for policy 0, policy_version 1625622 (0.0008) [2023-12-27 03:14:12,473][105692] Updated weights for policy 0, policy_version 1625632 (0.0008) [2023-12-27 03:14:12,519][105692] Updated weights for policy 0, policy_version 1625642 (0.0008) [2023-12-27 03:14:12,940][105620] Updated weights for policy 1, policy_version 1628998 (0.0010) [2023-12-27 03:14:12,999][105620] Updated weights for policy 1, policy_version 1629008 (0.0010) [2023-12-27 03:14:13,048][105620] Updated weights for policy 1, policy_version 1629018 (0.0010) [2023-12-27 03:14:13,328][105692] Updated weights for policy 0, policy_version 1625652 (0.0008) [2023-12-27 03:14:13,376][105692] Updated weights for policy 0, policy_version 1625662 (0.0008) [2023-12-27 03:14:13,428][105692] Updated weights for policy 0, policy_version 1625672 (0.0007) [2023-12-27 03:14:13,812][105620] Updated weights for policy 1, policy_version 1629028 (0.0010) [2023-12-27 03:14:13,870][105620] Updated weights for policy 1, policy_version 1629038 (0.0009) [2023-12-27 03:14:13,926][105620] Updated weights for policy 1, policy_version 1629048 (0.0008) [2023-12-27 03:14:14,177][105692] Updated weights for policy 0, policy_version 1625682 (0.0008) [2023-12-27 03:14:14,224][105692] Updated weights for policy 0, policy_version 1625692 (0.0009) [2023-12-27 03:14:14,276][105692] Updated weights for policy 0, policy_version 1625702 (0.0009) [2023-12-27 03:14:14,336][105692] Updated weights for policy 0, policy_version 1625712 (0.0010) [2023-12-27 03:14:14,642][105620] Updated weights for policy 1, policy_version 1629058 (0.0008) [2023-12-27 03:14:14,706][105620] Updated weights for policy 1, policy_version 1629068 (0.0008) [2023-12-27 03:14:14,768][105620] Updated weights for policy 1, policy_version 1629078 (0.0008) [2023-12-27 03:14:14,816][105620] Updated weights for policy 1, policy_version 1629088 (0.0008) [2023-12-27 03:14:15,140][105692] Updated weights for policy 0, policy_version 1625722 (0.0009) [2023-12-27 03:14:15,205][105692] Updated weights for policy 0, policy_version 1625732 (0.0008) [2023-12-27 03:14:15,271][105692] Updated weights for policy 0, policy_version 1625742 (0.0009) [2023-12-27 03:14:15,537][105620] Updated weights for policy 1, policy_version 1629098 (0.0005) [2023-12-27 03:14:15,603][105620] Updated weights for policy 1, policy_version 1629108 (0.0005) [2023-12-27 03:14:15,659][105620] Updated weights for policy 1, policy_version 1629118 (0.0005) [2023-12-27 03:14:16,024][105692] Updated weights for policy 0, policy_version 1625752 (0.0009) [2023-12-27 03:14:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 833363968. Throughput: 0: 9749.2, 1: 9635.3. Samples: 833338680. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:14:16,063][104569] Avg episode reward: [(0, '8527.854'), (1, '9172.115')] [2023-12-27 03:14:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001629120_417112064.pth... [2023-12-27 03:14:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001628000_416825344.pth [2023-12-27 03:14:16,078][105692] Updated weights for policy 0, policy_version 1625762 (0.0009) [2023-12-27 03:14:16,139][105692] Updated weights for policy 0, policy_version 1625772 (0.0009) [2023-12-27 03:14:16,156][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001625776_416260096.pth... [2023-12-27 03:14:16,159][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001624624_415965184.pth [2023-12-27 03:14:16,341][105620] Updated weights for policy 1, policy_version 1629128 (0.0008) [2023-12-27 03:14:16,396][105620] Updated weights for policy 1, policy_version 1629138 (0.0009) [2023-12-27 03:14:16,459][105620] Updated weights for policy 1, policy_version 1629148 (0.0009) [2023-12-27 03:14:16,926][105692] Updated weights for policy 0, policy_version 1625782 (0.0009) [2023-12-27 03:14:16,983][105692] Updated weights for policy 0, policy_version 1625792 (0.0009) [2023-12-27 03:14:17,044][105692] Updated weights for policy 0, policy_version 1625802 (0.0008) [2023-12-27 03:14:17,171][105620] Updated weights for policy 1, policy_version 1629158 (0.0007) [2023-12-27 03:14:17,217][105620] Updated weights for policy 1, policy_version 1629168 (0.0005) [2023-12-27 03:14:17,273][105620] Updated weights for policy 1, policy_version 1629178 (0.0006) [2023-12-27 03:14:17,796][105692] Updated weights for policy 0, policy_version 1625812 (0.0008) [2023-12-27 03:14:17,804][105620] Updated weights for policy 1, policy_version 1629188 (0.0005) [2023-12-27 03:14:17,845][105692] Updated weights for policy 0, policy_version 1625822 (0.0007) [2023-12-27 03:14:17,867][105620] Updated weights for policy 1, policy_version 1629198 (0.0006) [2023-12-27 03:14:17,907][105692] Updated weights for policy 0, policy_version 1625832 (0.0007) [2023-12-27 03:14:17,925][105620] Updated weights for policy 1, policy_version 1629208 (0.0005) [2023-12-27 03:14:18,461][105620] Updated weights for policy 1, policy_version 1629218 (0.0006) [2023-12-27 03:14:18,520][105620] Updated weights for policy 1, policy_version 1629228 (0.0011) [2023-12-27 03:14:18,578][105620] Updated weights for policy 1, policy_version 1629238 (0.0010) [2023-12-27 03:14:18,631][105620] Updated weights for policy 1, policy_version 1629248 (0.0010) [2023-12-27 03:14:18,732][105692] Updated weights for policy 0, policy_version 1625842 (0.0008) [2023-12-27 03:14:18,792][105692] Updated weights for policy 0, policy_version 1625852 (0.0008) [2023-12-27 03:14:18,845][105692] Updated weights for policy 0, policy_version 1625862 (0.0009) [2023-12-27 03:14:18,890][105692] Updated weights for policy 0, policy_version 1625872 (0.0008) [2023-12-27 03:14:19,298][105620] Updated weights for policy 1, policy_version 1629258 (0.0009) [2023-12-27 03:14:19,363][105620] Updated weights for policy 1, policy_version 1629268 (0.0008) [2023-12-27 03:14:19,426][105620] Updated weights for policy 1, policy_version 1629278 (0.0011) [2023-12-27 03:14:19,737][105692] Updated weights for policy 0, policy_version 1625882 (0.0008) [2023-12-27 03:14:19,804][105692] Updated weights for policy 0, policy_version 1625892 (0.0009) [2023-12-27 03:14:19,869][105692] Updated weights for policy 0, policy_version 1625902 (0.0008) [2023-12-27 03:14:20,178][105620] Updated weights for policy 1, policy_version 1629288 (0.0010) [2023-12-27 03:14:20,238][105620] Updated weights for policy 1, policy_version 1629298 (0.0010) [2023-12-27 03:14:20,296][105620] Updated weights for policy 1, policy_version 1629308 (0.0010) [2023-12-27 03:14:20,707][105692] Updated weights for policy 0, policy_version 1625912 (0.0008) [2023-12-27 03:14:20,770][105692] Updated weights for policy 0, policy_version 1625922 (0.0006) [2023-12-27 03:14:20,835][105692] Updated weights for policy 0, policy_version 1625932 (0.0006) [2023-12-27 03:14:21,000][105620] Updated weights for policy 1, policy_version 1629318 (0.0010) [2023-12-27 03:14:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 833462272. Throughput: 0: 9614.4, 1: 9777.3. Samples: 833455120. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:14:21,062][104569] Avg episode reward: [(0, '8807.372'), (1, '9262.336')] [2023-12-27 03:14:21,067][105620] Updated weights for policy 1, policy_version 1629328 (0.0010) [2023-12-27 03:14:21,128][105620] Updated weights for policy 1, policy_version 1629338 (0.0010) [2023-12-27 03:14:21,581][105692] Updated weights for policy 0, policy_version 1625942 (0.0007) [2023-12-27 03:14:21,646][105692] Updated weights for policy 0, policy_version 1625952 (0.0008) [2023-12-27 03:14:21,707][105692] Updated weights for policy 0, policy_version 1625962 (0.0008) [2023-12-27 03:14:21,885][105620] Updated weights for policy 1, policy_version 1629348 (0.0009) [2023-12-27 03:14:21,931][105620] Updated weights for policy 1, policy_version 1629358 (0.0010) [2023-12-27 03:14:21,983][105620] Updated weights for policy 1, policy_version 1629368 (0.0010) [2023-12-27 03:14:22,433][105692] Updated weights for policy 0, policy_version 1625972 (0.0010) [2023-12-27 03:14:22,501][105692] Updated weights for policy 0, policy_version 1625982 (0.0011) [2023-12-27 03:14:22,561][105692] Updated weights for policy 0, policy_version 1625992 (0.0010) [2023-12-27 03:14:22,767][105620] Updated weights for policy 1, policy_version 1629378 (0.0010) [2023-12-27 03:14:22,829][105620] Updated weights for policy 1, policy_version 1629388 (0.0010) [2023-12-27 03:14:22,888][105620] Updated weights for policy 1, policy_version 1629398 (0.0010) [2023-12-27 03:14:22,949][105620] Updated weights for policy 1, policy_version 1629408 (0.0010) [2023-12-27 03:14:23,287][105692] Updated weights for policy 0, policy_version 1626002 (0.0010) [2023-12-27 03:14:23,348][105692] Updated weights for policy 0, policy_version 1626012 (0.0010) [2023-12-27 03:14:23,392][105692] Updated weights for policy 0, policy_version 1626022 (0.0010) [2023-12-27 03:14:23,437][105692] Updated weights for policy 0, policy_version 1626032 (0.0010) [2023-12-27 03:14:23,690][105620] Updated weights for policy 1, policy_version 1629418 (0.0010) [2023-12-27 03:14:23,751][105620] Updated weights for policy 1, policy_version 1629428 (0.0010) [2023-12-27 03:14:23,802][105620] Updated weights for policy 1, policy_version 1629438 (0.0010) [2023-12-27 03:14:24,114][105692] Updated weights for policy 0, policy_version 1626042 (0.0010) [2023-12-27 03:14:24,176][105692] Updated weights for policy 0, policy_version 1626052 (0.0010) [2023-12-27 03:14:24,230][105692] Updated weights for policy 0, policy_version 1626062 (0.0010) [2023-12-27 03:14:24,549][105620] Updated weights for policy 1, policy_version 1629448 (0.0006) [2023-12-27 03:14:24,608][105620] Updated weights for policy 1, policy_version 1629458 (0.0005) [2023-12-27 03:14:24,666][105620] Updated weights for policy 1, policy_version 1629468 (0.0005) [2023-12-27 03:14:24,847][105692] Updated weights for policy 0, policy_version 1626072 (0.0007) [2023-12-27 03:14:24,909][105692] Updated weights for policy 0, policy_version 1626082 (0.0005) [2023-12-27 03:14:24,969][105692] Updated weights for policy 0, policy_version 1626092 (0.0005) [2023-12-27 03:14:25,332][105620] Updated weights for policy 1, policy_version 1629478 (0.0008) [2023-12-27 03:14:25,379][105620] Updated weights for policy 1, policy_version 1629488 (0.0010) [2023-12-27 03:14:25,436][105620] Updated weights for policy 1, policy_version 1629498 (0.0009) [2023-12-27 03:14:25,590][105692] Updated weights for policy 0, policy_version 1626102 (0.0006) [2023-12-27 03:14:25,652][105692] Updated weights for policy 0, policy_version 1626112 (0.0010) [2023-12-27 03:14:25,714][105692] Updated weights for policy 0, policy_version 1626122 (0.0009) [2023-12-27 03:14:26,000][105620] Updated weights for policy 1, policy_version 1629508 (0.0005) [2023-12-27 03:14:26,056][105620] Updated weights for policy 1, policy_version 1629518 (0.0009) [2023-12-27 03:14:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 833560576. Throughput: 0: 9648.8, 1: 9795.1. Samples: 833572304. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:14:26,062][104569] Avg episode reward: [(0, '8714.887'), (1, '9260.934')] [2023-12-27 03:14:26,114][105620] Updated weights for policy 1, policy_version 1629528 (0.0010) [2023-12-27 03:14:26,410][105692] Updated weights for policy 0, policy_version 1626132 (0.0007) [2023-12-27 03:14:26,454][105692] Updated weights for policy 0, policy_version 1626142 (0.0008) [2023-12-27 03:14:26,499][105692] Updated weights for policy 0, policy_version 1626152 (0.0008) [2023-12-27 03:14:26,837][105620] Updated weights for policy 1, policy_version 1629538 (0.0010) [2023-12-27 03:14:26,901][105620] Updated weights for policy 1, policy_version 1629548 (0.0010) [2023-12-27 03:14:26,965][105620] Updated weights for policy 1, policy_version 1629558 (0.0010) [2023-12-27 03:14:27,019][105620] Updated weights for policy 1, policy_version 1629568 (0.0010) [2023-12-27 03:14:27,185][105692] Updated weights for policy 0, policy_version 1626162 (0.0005) [2023-12-27 03:14:27,244][105692] Updated weights for policy 0, policy_version 1626172 (0.0005) [2023-12-27 03:14:27,293][105692] Updated weights for policy 0, policy_version 1626182 (0.0005) [2023-12-27 03:14:27,355][105692] Updated weights for policy 0, policy_version 1626192 (0.0006) [2023-12-27 03:14:27,746][105620] Updated weights for policy 1, policy_version 1629578 (0.0010) [2023-12-27 03:14:27,768][105586] KL-divergence is very high: 219.3291 [2023-12-27 03:14:27,795][105586] KL-divergence is very high: 288.7682 [2023-12-27 03:14:27,797][105620] Updated weights for policy 1, policy_version 1629588 (0.0010) [2023-12-27 03:14:27,810][105586] KL-divergence is very high: 425.1554 [2023-12-27 03:14:27,835][105586] KL-divergence is very high: 363.0530 [2023-12-27 03:14:27,847][105620] Updated weights for policy 1, policy_version 1629598 (0.0010) [2023-12-27 03:14:27,850][105586] KL-divergence is very high: 495.5892 [2023-12-27 03:14:28,016][105692] Updated weights for policy 0, policy_version 1626202 (0.0008) [2023-12-27 03:14:28,067][105692] Updated weights for policy 0, policy_version 1626212 (0.0010) [2023-12-27 03:14:28,118][105692] Updated weights for policy 0, policy_version 1626222 (0.0010) [2023-12-27 03:14:28,588][105620] Updated weights for policy 1, policy_version 1629608 (0.0008) [2023-12-27 03:14:28,642][105620] Updated weights for policy 1, policy_version 1629618 (0.0005) [2023-12-27 03:14:28,691][105620] Updated weights for policy 1, policy_version 1629628 (0.0005) [2023-12-27 03:14:28,816][105692] Updated weights for policy 0, policy_version 1626232 (0.0006) [2023-12-27 03:14:28,878][105692] Updated weights for policy 0, policy_version 1626242 (0.0005) [2023-12-27 03:14:28,943][105692] Updated weights for policy 0, policy_version 1626252 (0.0005) [2023-12-27 03:14:29,395][105620] Updated weights for policy 1, policy_version 1629638 (0.0008) [2023-12-27 03:14:29,459][105620] Updated weights for policy 1, policy_version 1629648 (0.0006) [2023-12-27 03:14:29,526][105620] Updated weights for policy 1, policy_version 1629658 (0.0007) [2023-12-27 03:14:29,586][105692] Updated weights for policy 0, policy_version 1626262 (0.0008) [2023-12-27 03:14:29,636][105692] Updated weights for policy 0, policy_version 1626272 (0.0007) [2023-12-27 03:14:29,692][105692] Updated weights for policy 0, policy_version 1626282 (0.0005) [2023-12-27 03:14:30,280][105620] Updated weights for policy 1, policy_version 1629668 (0.0008) [2023-12-27 03:14:30,337][105620] Updated weights for policy 1, policy_version 1629678 (0.0007) [2023-12-27 03:14:30,394][105692] Updated weights for policy 0, policy_version 1626292 (0.0006) [2023-12-27 03:14:30,395][105620] Updated weights for policy 1, policy_version 1629688 (0.0009) [2023-12-27 03:14:30,448][105692] Updated weights for policy 0, policy_version 1626302 (0.0007) [2023-12-27 03:14:30,503][105692] Updated weights for policy 0, policy_version 1626312 (0.0005) [2023-12-27 03:14:30,977][105620] Updated weights for policy 1, policy_version 1629698 (0.0007) [2023-12-27 03:14:31,027][105620] Updated weights for policy 1, policy_version 1629708 (0.0007) [2023-12-27 03:14:31,057][105692] Updated weights for policy 0, policy_version 1626322 (0.0006) [2023-12-27 03:14:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 833658880. Throughput: 0: 9690.1, 1: 9831.2. Samples: 833631872. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:14:31,063][104569] Avg episode reward: [(0, '8711.475'), (1, '9077.247')] [2023-12-27 03:14:31,094][105620] Updated weights for policy 1, policy_version 1629718 (0.0007) [2023-12-27 03:14:31,121][105692] Updated weights for policy 0, policy_version 1626332 (0.0011) [2023-12-27 03:14:31,158][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001629728_417267712.pth... [2023-12-27 03:14:31,158][105620] Updated weights for policy 1, policy_version 1629728 (0.0008) [2023-12-27 03:14:31,161][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001628544_416964608.pth [2023-12-27 03:14:31,184][105692] Updated weights for policy 0, policy_version 1626342 (0.0010) [2023-12-27 03:14:31,250][105692] Updated weights for policy 0, policy_version 1626352 (0.0011) [2023-12-27 03:14:31,250][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001626352_416407552.pth... [2023-12-27 03:14:31,254][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001625200_416112640.pth [2023-12-27 03:14:31,885][105620] Updated weights for policy 1, policy_version 1629738 (0.0008) [2023-12-27 03:14:31,950][105620] Updated weights for policy 1, policy_version 1629748 (0.0006) [2023-12-27 03:14:32,001][105692] Updated weights for policy 0, policy_version 1626362 (0.0008) [2023-12-27 03:14:32,001][105620] Updated weights for policy 1, policy_version 1629758 (0.0005) [2023-12-27 03:14:32,059][105692] Updated weights for policy 0, policy_version 1626372 (0.0008) [2023-12-27 03:14:32,108][105692] Updated weights for policy 0, policy_version 1626382 (0.0007) [2023-12-27 03:14:32,628][105620] Updated weights for policy 1, policy_version 1629768 (0.0007) [2023-12-27 03:14:32,682][105620] Updated weights for policy 1, policy_version 1629778 (0.0007) [2023-12-27 03:14:32,704][105692] Updated weights for policy 0, policy_version 1626392 (0.0009) [2023-12-27 03:14:32,737][105620] Updated weights for policy 1, policy_version 1629788 (0.0005) [2023-12-27 03:14:32,755][105692] Updated weights for policy 0, policy_version 1626402 (0.0010) [2023-12-27 03:14:32,816][105692] Updated weights for policy 0, policy_version 1626412 (0.0010) [2023-12-27 03:14:33,418][105692] Updated weights for policy 0, policy_version 1626422 (0.0007) [2023-12-27 03:14:33,474][105692] Updated weights for policy 0, policy_version 1626432 (0.0005) [2023-12-27 03:14:33,523][105692] Updated weights for policy 0, policy_version 1626442 (0.0005) [2023-12-27 03:14:33,557][105620] Updated weights for policy 1, policy_version 1629798 (0.0007) [2023-12-27 03:14:33,613][105620] Updated weights for policy 1, policy_version 1629808 (0.0007) [2023-12-27 03:14:33,674][105620] Updated weights for policy 1, policy_version 1629818 (0.0008) [2023-12-27 03:14:34,053][105692] Updated weights for policy 0, policy_version 1626452 (0.0005) [2023-12-27 03:14:34,113][105692] Updated weights for policy 0, policy_version 1626462 (0.0005) [2023-12-27 03:14:34,180][105692] Updated weights for policy 0, policy_version 1626472 (0.0007) [2023-12-27 03:14:34,485][105620] Updated weights for policy 1, policy_version 1629828 (0.0009) [2023-12-27 03:14:34,548][105620] Updated weights for policy 1, policy_version 1629838 (0.0008) [2023-12-27 03:14:34,611][105620] Updated weights for policy 1, policy_version 1629848 (0.0009) [2023-12-27 03:14:34,921][105692] Updated weights for policy 0, policy_version 1626482 (0.0010) [2023-12-27 03:14:34,972][105692] Updated weights for policy 0, policy_version 1626492 (0.0010) [2023-12-27 03:14:35,021][105692] Updated weights for policy 0, policy_version 1626502 (0.0010) [2023-12-27 03:14:35,069][105692] Updated weights for policy 0, policy_version 1626512 (0.0010) [2023-12-27 03:14:35,410][105620] Updated weights for policy 1, policy_version 1629858 (0.0009) [2023-12-27 03:14:35,477][105620] Updated weights for policy 1, policy_version 1629868 (0.0010) [2023-12-27 03:14:35,536][105620] Updated weights for policy 1, policy_version 1629878 (0.0010) [2023-12-27 03:14:35,599][105620] Updated weights for policy 1, policy_version 1629888 (0.0009) [2023-12-27 03:14:35,645][105692] Updated weights for policy 0, policy_version 1626522 (0.0006) [2023-12-27 03:14:35,702][105692] Updated weights for policy 0, policy_version 1626532 (0.0005) [2023-12-27 03:14:35,761][105692] Updated weights for policy 0, policy_version 1626542 (0.0005) [2023-12-27 03:14:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 833765376. Throughput: 0: 9787.5, 1: 9845.6. Samples: 833754400. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:14:36,063][104569] Avg episode reward: [(0, '8715.741'), (1, '9169.146')] [2023-12-27 03:14:36,375][105620] Updated weights for policy 1, policy_version 1629898 (0.0009) [2023-12-27 03:14:36,430][105692] Updated weights for policy 0, policy_version 1626552 (0.0007) [2023-12-27 03:14:36,435][105620] Updated weights for policy 1, policy_version 1629908 (0.0006) [2023-12-27 03:14:36,492][105692] Updated weights for policy 0, policy_version 1626562 (0.0006) [2023-12-27 03:14:36,497][105620] Updated weights for policy 1, policy_version 1629918 (0.0006) [2023-12-27 03:14:36,556][105692] Updated weights for policy 0, policy_version 1626572 (0.0006) [2023-12-27 03:14:37,168][105620] Updated weights for policy 1, policy_version 1629928 (0.0008) [2023-12-27 03:14:37,208][105692] Updated weights for policy 0, policy_version 1626582 (0.0009) [2023-12-27 03:14:37,225][105620] Updated weights for policy 1, policy_version 1629938 (0.0008) [2023-12-27 03:14:37,262][105692] Updated weights for policy 0, policy_version 1626592 (0.0010) [2023-12-27 03:14:37,274][105620] Updated weights for policy 1, policy_version 1629948 (0.0007) [2023-12-27 03:14:37,311][105692] Updated weights for policy 0, policy_version 1626602 (0.0010) [2023-12-27 03:14:37,994][105620] Updated weights for policy 1, policy_version 1629958 (0.0007) [2023-12-27 03:14:38,017][105692] Updated weights for policy 0, policy_version 1626612 (0.0010) [2023-12-27 03:14:38,049][105620] Updated weights for policy 1, policy_version 1629968 (0.0006) [2023-12-27 03:14:38,073][105692] Updated weights for policy 0, policy_version 1626622 (0.0005) [2023-12-27 03:14:38,112][105620] Updated weights for policy 1, policy_version 1629978 (0.0005) [2023-12-27 03:14:38,135][105692] Updated weights for policy 0, policy_version 1626632 (0.0005) [2023-12-27 03:14:38,800][105620] Updated weights for policy 1, policy_version 1629988 (0.0007) [2023-12-27 03:14:38,813][105692] Updated weights for policy 0, policy_version 1626642 (0.0006) [2023-12-27 03:14:38,862][105620] Updated weights for policy 1, policy_version 1629998 (0.0006) [2023-12-27 03:14:38,872][105692] Updated weights for policy 0, policy_version 1626652 (0.0010) [2023-12-27 03:14:38,918][105620] Updated weights for policy 1, policy_version 1630008 (0.0006) [2023-12-27 03:14:38,931][105692] Updated weights for policy 0, policy_version 1626662 (0.0010) [2023-12-27 03:14:38,983][105692] Updated weights for policy 0, policy_version 1626672 (0.0010) [2023-12-27 03:14:39,649][105620] Updated weights for policy 1, policy_version 1630018 (0.0007) [2023-12-27 03:14:39,705][105692] Updated weights for policy 0, policy_version 1626682 (0.0006) [2023-12-27 03:14:39,710][105620] Updated weights for policy 1, policy_version 1630028 (0.0008) [2023-12-27 03:14:39,768][105620] Updated weights for policy 1, policy_version 1630038 (0.0010) [2023-12-27 03:14:39,770][105692] Updated weights for policy 0, policy_version 1626692 (0.0008) [2023-12-27 03:14:39,835][105692] Updated weights for policy 0, policy_version 1626702 (0.0009) [2023-12-27 03:14:39,836][105620] Updated weights for policy 1, policy_version 1630048 (0.0010) [2023-12-27 03:14:40,555][105620] Updated weights for policy 1, policy_version 1630058 (0.0009) [2023-12-27 03:14:40,560][105692] Updated weights for policy 0, policy_version 1626712 (0.0011) [2023-12-27 03:14:40,611][105620] Updated weights for policy 1, policy_version 1630068 (0.0011) [2023-12-27 03:14:40,623][105692] Updated weights for policy 0, policy_version 1626722 (0.0011) [2023-12-27 03:14:40,671][105620] Updated weights for policy 1, policy_version 1630078 (0.0011) [2023-12-27 03:14:40,685][105692] Updated weights for policy 0, policy_version 1626732 (0.0010) [2023-12-27 03:14:41,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 833863680. Throughput: 0: 9865.9, 1: 9780.6. Samples: 833873452. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:14:41,063][104569] Avg episode reward: [(0, '8348.057'), (1, '9076.989')] [2023-12-27 03:14:41,329][105620] Updated weights for policy 1, policy_version 1630088 (0.0006) [2023-12-27 03:14:41,399][105620] Updated weights for policy 1, policy_version 1630098 (0.0008) [2023-12-27 03:14:41,451][105692] Updated weights for policy 0, policy_version 1626742 (0.0010) [2023-12-27 03:14:41,459][105620] Updated weights for policy 1, policy_version 1630108 (0.0011) [2023-12-27 03:14:41,507][105692] Updated weights for policy 0, policy_version 1626752 (0.0010) [2023-12-27 03:14:41,564][105692] Updated weights for policy 0, policy_version 1626762 (0.0006) [2023-12-27 03:14:42,207][105692] Updated weights for policy 0, policy_version 1626772 (0.0007) [2023-12-27 03:14:42,217][105620] Updated weights for policy 1, policy_version 1630118 (0.0010) [2023-12-27 03:14:42,269][105692] Updated weights for policy 0, policy_version 1626782 (0.0007) [2023-12-27 03:14:42,280][105620] Updated weights for policy 1, policy_version 1630128 (0.0011) [2023-12-27 03:14:42,335][105692] Updated weights for policy 0, policy_version 1626792 (0.0008) [2023-12-27 03:14:42,345][105620] Updated weights for policy 1, policy_version 1630138 (0.0010) [2023-12-27 03:14:43,007][105692] Updated weights for policy 0, policy_version 1626802 (0.0011) [2023-12-27 03:14:43,058][105620] Updated weights for policy 1, policy_version 1630148 (0.0006) [2023-12-27 03:14:43,074][105692] Updated weights for policy 0, policy_version 1626812 (0.0009) [2023-12-27 03:14:43,112][105620] Updated weights for policy 1, policy_version 1630158 (0.0005) [2023-12-27 03:14:43,132][105692] Updated weights for policy 0, policy_version 1626822 (0.0010) [2023-12-27 03:14:43,158][105620] Updated weights for policy 1, policy_version 1630168 (0.0005) [2023-12-27 03:14:43,188][105692] Updated weights for policy 0, policy_version 1626832 (0.0008) [2023-12-27 03:14:43,853][105692] Updated weights for policy 0, policy_version 1626842 (0.0009) [2023-12-27 03:14:43,868][105620] Updated weights for policy 1, policy_version 1630178 (0.0009) [2023-12-27 03:14:43,910][105692] Updated weights for policy 0, policy_version 1626852 (0.0008) [2023-12-27 03:14:43,929][105620] Updated weights for policy 1, policy_version 1630188 (0.0006) [2023-12-27 03:14:43,971][105692] Updated weights for policy 0, policy_version 1626862 (0.0007) [2023-12-27 03:14:43,981][105620] Updated weights for policy 1, policy_version 1630198 (0.0007) [2023-12-27 03:14:44,034][105620] Updated weights for policy 1, policy_version 1630208 (0.0008) [2023-12-27 03:14:44,722][105692] Updated weights for policy 0, policy_version 1626872 (0.0009) [2023-12-27 03:14:44,769][105620] Updated weights for policy 1, policy_version 1630218 (0.0011) [2023-12-27 03:14:44,788][105692] Updated weights for policy 0, policy_version 1626882 (0.0007) [2023-12-27 03:14:44,833][105620] Updated weights for policy 1, policy_version 1630228 (0.0010) [2023-12-27 03:14:44,840][105692] Updated weights for policy 0, policy_version 1626892 (0.0006) [2023-12-27 03:14:44,898][105620] Updated weights for policy 1, policy_version 1630238 (0.0011) [2023-12-27 03:14:45,626][105692] Updated weights for policy 0, policy_version 1626902 (0.0008) [2023-12-27 03:14:45,637][105620] Updated weights for policy 1, policy_version 1630248 (0.0010) [2023-12-27 03:14:45,684][105692] Updated weights for policy 0, policy_version 1626912 (0.0006) [2023-12-27 03:14:45,690][105620] Updated weights for policy 1, policy_version 1630258 (0.0010) [2023-12-27 03:14:45,735][105620] Updated weights for policy 1, policy_version 1630268 (0.0010) [2023-12-27 03:14:45,741][105692] Updated weights for policy 0, policy_version 1626922 (0.0007) [2023-12-27 03:14:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 833961984. Throughput: 0: 9842.0, 1: 9782.1. Samples: 833932316. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:14:46,063][104569] Avg episode reward: [(0, '8441.816'), (1, '8893.593')] [2023-12-27 03:14:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001626928_416555008.pth... [2023-12-27 03:14:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001630272_417406976.pth... [2023-12-27 03:14:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001625776_416260096.pth [2023-12-27 03:14:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001629120_417112064.pth [2023-12-27 03:14:46,306][105692] Updated weights for policy 0, policy_version 1626932 (0.0005) [2023-12-27 03:14:46,351][105692] Updated weights for policy 0, policy_version 1626942 (0.0005) [2023-12-27 03:14:46,396][105692] Updated weights for policy 0, policy_version 1626952 (0.0005) [2023-12-27 03:14:46,467][105620] Updated weights for policy 1, policy_version 1630278 (0.0010) [2023-12-27 03:14:46,527][105620] Updated weights for policy 1, policy_version 1630288 (0.0011) [2023-12-27 03:14:46,576][105620] Updated weights for policy 1, policy_version 1630298 (0.0010) [2023-12-27 03:14:47,037][105692] Updated weights for policy 0, policy_version 1626962 (0.0008) [2023-12-27 03:14:47,097][105692] Updated weights for policy 0, policy_version 1626972 (0.0008) [2023-12-27 03:14:47,161][105692] Updated weights for policy 0, policy_version 1626982 (0.0009) [2023-12-27 03:14:47,229][105692] Updated weights for policy 0, policy_version 1626992 (0.0010) [2023-12-27 03:14:47,306][105620] Updated weights for policy 1, policy_version 1630308 (0.0011) [2023-12-27 03:14:47,368][105620] Updated weights for policy 1, policy_version 1630318 (0.0010) [2023-12-27 03:14:47,425][105620] Updated weights for policy 1, policy_version 1630328 (0.0010) [2023-12-27 03:14:47,906][105692] Updated weights for policy 0, policy_version 1627002 (0.0006) [2023-12-27 03:14:47,958][105692] Updated weights for policy 0, policy_version 1627012 (0.0005) [2023-12-27 03:14:48,022][105692] Updated weights for policy 0, policy_version 1627022 (0.0006) [2023-12-27 03:14:48,153][105620] Updated weights for policy 1, policy_version 1630338 (0.0010) [2023-12-27 03:14:48,204][105620] Updated weights for policy 1, policy_version 1630348 (0.0008) [2023-12-27 03:14:48,256][105620] Updated weights for policy 1, policy_version 1630358 (0.0010) [2023-12-27 03:14:48,306][105620] Updated weights for policy 1, policy_version 1630368 (0.0007) [2023-12-27 03:14:48,766][105692] Updated weights for policy 0, policy_version 1627032 (0.0010) [2023-12-27 03:14:48,820][105692] Updated weights for policy 0, policy_version 1627042 (0.0010) [2023-12-27 03:14:48,870][105692] Updated weights for policy 0, policy_version 1627052 (0.0009) [2023-12-27 03:14:48,892][105620] Updated weights for policy 1, policy_version 1630378 (0.0005) [2023-12-27 03:14:48,956][105620] Updated weights for policy 1, policy_version 1630388 (0.0008) [2023-12-27 03:14:49,015][105620] Updated weights for policy 1, policy_version 1630398 (0.0010) [2023-12-27 03:14:49,697][105620] Updated weights for policy 1, policy_version 1630408 (0.0009) [2023-12-27 03:14:49,716][105692] Updated weights for policy 0, policy_version 1627062 (0.0008) [2023-12-27 03:14:49,746][105620] Updated weights for policy 1, policy_version 1630418 (0.0010) [2023-12-27 03:14:49,776][105692] Updated weights for policy 0, policy_version 1627072 (0.0007) [2023-12-27 03:14:49,790][105620] Updated weights for policy 1, policy_version 1630428 (0.0008) [2023-12-27 03:14:49,838][105692] Updated weights for policy 0, policy_version 1627082 (0.0008) [2023-12-27 03:14:50,542][105620] Updated weights for policy 1, policy_version 1630438 (0.0007) [2023-12-27 03:14:50,612][105620] Updated weights for policy 1, policy_version 1630448 (0.0008) [2023-12-27 03:14:50,644][105692] Updated weights for policy 0, policy_version 1627092 (0.0008) [2023-12-27 03:14:50,683][105620] Updated weights for policy 1, policy_version 1630458 (0.0009) [2023-12-27 03:14:50,709][105692] Updated weights for policy 0, policy_version 1627102 (0.0006) [2023-12-27 03:14:50,776][105692] Updated weights for policy 0, policy_version 1627112 (0.0005) [2023-12-27 03:14:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 834060288. Throughput: 0: 9859.1, 1: 9778.4. Samples: 834049976. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:14:51,063][104569] Avg episode reward: [(0, '8535.201'), (1, '8986.049')] [2023-12-27 03:14:51,402][105692] Updated weights for policy 0, policy_version 1627122 (0.0007) [2023-12-27 03:14:51,448][105620] Updated weights for policy 1, policy_version 1630468 (0.0007) [2023-12-27 03:14:51,458][105692] Updated weights for policy 0, policy_version 1627132 (0.0011) [2023-12-27 03:14:51,507][105620] Updated weights for policy 1, policy_version 1630478 (0.0006) [2023-12-27 03:14:51,507][105692] Updated weights for policy 0, policy_version 1627142 (0.0008) [2023-12-27 03:14:51,566][105692] Updated weights for policy 0, policy_version 1627152 (0.0009) [2023-12-27 03:14:51,567][105620] Updated weights for policy 1, policy_version 1630488 (0.0008) [2023-12-27 03:14:52,297][105692] Updated weights for policy 0, policy_version 1627162 (0.0009) [2023-12-27 03:14:52,345][105620] Updated weights for policy 1, policy_version 1630498 (0.0008) [2023-12-27 03:14:52,366][105692] Updated weights for policy 0, policy_version 1627172 (0.0009) [2023-12-27 03:14:52,415][105620] Updated weights for policy 1, policy_version 1630508 (0.0007) [2023-12-27 03:14:52,426][105692] Updated weights for policy 0, policy_version 1627182 (0.0007) [2023-12-27 03:14:52,475][105620] Updated weights for policy 1, policy_version 1630518 (0.0009) [2023-12-27 03:14:52,536][105620] Updated weights for policy 1, policy_version 1630528 (0.0009) [2023-12-27 03:14:53,177][105620] Updated weights for policy 1, policy_version 1630538 (0.0010) [2023-12-27 03:14:53,183][105692] Updated weights for policy 0, policy_version 1627192 (0.0006) [2023-12-27 03:14:53,230][105692] Updated weights for policy 0, policy_version 1627202 (0.0007) [2023-12-27 03:14:53,232][105620] Updated weights for policy 1, policy_version 1630548 (0.0007) [2023-12-27 03:14:53,286][105692] Updated weights for policy 0, policy_version 1627212 (0.0008) [2023-12-27 03:14:53,297][105620] Updated weights for policy 1, policy_version 1630558 (0.0008) [2023-12-27 03:14:53,941][105692] Updated weights for policy 0, policy_version 1627222 (0.0008) [2023-12-27 03:14:53,950][105620] Updated weights for policy 1, policy_version 1630568 (0.0008) [2023-12-27 03:14:53,990][105692] Updated weights for policy 0, policy_version 1627232 (0.0007) [2023-12-27 03:14:54,013][105620] Updated weights for policy 1, policy_version 1630578 (0.0008) [2023-12-27 03:14:54,040][105692] Updated weights for policy 0, policy_version 1627242 (0.0007) [2023-12-27 03:14:54,066][105620] Updated weights for policy 1, policy_version 1630588 (0.0008) [2023-12-27 03:14:54,683][105692] Updated weights for policy 0, policy_version 1627252 (0.0006) [2023-12-27 03:14:54,741][105692] Updated weights for policy 0, policy_version 1627262 (0.0005) [2023-12-27 03:14:54,792][105692] Updated weights for policy 0, policy_version 1627272 (0.0009) [2023-12-27 03:14:54,867][105620] Updated weights for policy 1, policy_version 1630598 (0.0009) [2023-12-27 03:14:54,921][105620] Updated weights for policy 1, policy_version 1630608 (0.0009) [2023-12-27 03:14:54,975][105620] Updated weights for policy 1, policy_version 1630618 (0.0009) [2023-12-27 03:14:55,478][105692] Updated weights for policy 0, policy_version 1627282 (0.0010) [2023-12-27 03:14:55,537][105692] Updated weights for policy 0, policy_version 1627292 (0.0009) [2023-12-27 03:14:55,596][105692] Updated weights for policy 0, policy_version 1627302 (0.0009) [2023-12-27 03:14:55,655][105692] Updated weights for policy 0, policy_version 1627312 (0.0009) [2023-12-27 03:14:55,760][105620] Updated weights for policy 1, policy_version 1630628 (0.0009) [2023-12-27 03:14:55,818][105620] Updated weights for policy 1, policy_version 1630638 (0.0009) [2023-12-27 03:14:55,879][105620] Updated weights for policy 1, policy_version 1630648 (0.0008) [2023-12-27 03:14:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.9, 300 sec: 19522.0). Total num frames: 834158592. Throughput: 0: 9839.2, 1: 9728.8. Samples: 834165596. Policy #0 lag: (min: 2.0, avg: 13.3, max: 34.0) [2023-12-27 03:14:56,062][104569] Avg episode reward: [(0, '8715.100'), (1, '9079.085')] [2023-12-27 03:14:56,471][105620] Updated weights for policy 1, policy_version 1630658 (0.0009) [2023-12-27 03:14:56,478][105692] Updated weights for policy 0, policy_version 1627322 (0.0009) [2023-12-27 03:14:56,531][105620] Updated weights for policy 1, policy_version 1630668 (0.0007) [2023-12-27 03:14:56,537][105692] Updated weights for policy 0, policy_version 1627332 (0.0007) [2023-12-27 03:14:56,581][105620] Updated weights for policy 1, policy_version 1630678 (0.0006) [2023-12-27 03:14:56,599][105692] Updated weights for policy 0, policy_version 1627342 (0.0007) [2023-12-27 03:14:56,635][105620] Updated weights for policy 1, policy_version 1630688 (0.0008) [2023-12-27 03:14:57,285][105692] Updated weights for policy 0, policy_version 1627352 (0.0008) [2023-12-27 03:14:57,336][105692] Updated weights for policy 0, policy_version 1627362 (0.0008) [2023-12-27 03:14:57,352][105620] Updated weights for policy 1, policy_version 1630698 (0.0007) [2023-12-27 03:14:57,393][105692] Updated weights for policy 0, policy_version 1627372 (0.0006) [2023-12-27 03:14:57,427][105620] Updated weights for policy 1, policy_version 1630708 (0.0009) [2023-12-27 03:14:57,484][105620] Updated weights for policy 1, policy_version 1630718 (0.0009) [2023-12-27 03:14:58,043][105692] Updated weights for policy 0, policy_version 1627382 (0.0005) [2023-12-27 03:14:58,093][105692] Updated weights for policy 0, policy_version 1627392 (0.0005) [2023-12-27 03:14:58,155][105692] Updated weights for policy 0, policy_version 1627402 (0.0006) [2023-12-27 03:14:58,254][105620] Updated weights for policy 1, policy_version 1630728 (0.0009) [2023-12-27 03:14:58,318][105620] Updated weights for policy 1, policy_version 1630738 (0.0008) [2023-12-27 03:14:58,390][105620] Updated weights for policy 1, policy_version 1630748 (0.0008) [2023-12-27 03:14:58,849][105692] Updated weights for policy 0, policy_version 1627412 (0.0008) [2023-12-27 03:14:58,914][105692] Updated weights for policy 0, policy_version 1627422 (0.0009) [2023-12-27 03:14:58,997][105692] Updated weights for policy 0, policy_version 1627432 (0.0008) [2023-12-27 03:14:59,198][105620] Updated weights for policy 1, policy_version 1630758 (0.0009) [2023-12-27 03:14:59,267][105620] Updated weights for policy 1, policy_version 1630768 (0.0008) [2023-12-27 03:14:59,332][105620] Updated weights for policy 1, policy_version 1630778 (0.0008) [2023-12-27 03:14:59,670][105692] Updated weights for policy 0, policy_version 1627442 (0.0007) [2023-12-27 03:14:59,730][105692] Updated weights for policy 0, policy_version 1627452 (0.0009) [2023-12-27 03:14:59,783][105692] Updated weights for policy 0, policy_version 1627462 (0.0009) [2023-12-27 03:14:59,842][105692] Updated weights for policy 0, policy_version 1627472 (0.0010) [2023-12-27 03:15:00,148][105620] Updated weights for policy 1, policy_version 1630788 (0.0009) [2023-12-27 03:15:00,196][105620] Updated weights for policy 1, policy_version 1630798 (0.0008) [2023-12-27 03:15:00,255][105620] Updated weights for policy 1, policy_version 1630808 (0.0008) [2023-12-27 03:15:00,587][105692] Updated weights for policy 0, policy_version 1627482 (0.0010) [2023-12-27 03:15:00,644][105692] Updated weights for policy 0, policy_version 1627492 (0.0011) [2023-12-27 03:15:00,701][105692] Updated weights for policy 0, policy_version 1627502 (0.0010) [2023-12-27 03:15:00,951][105620] Updated weights for policy 1, policy_version 1630818 (0.0007) [2023-12-27 03:15:01,006][105620] Updated weights for policy 1, policy_version 1630828 (0.0011) [2023-12-27 03:15:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 834248704. Throughput: 0: 9903.9, 1: 9763.7. Samples: 834223724. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:15:01,063][104569] Avg episode reward: [(0, '8806.779'), (1, '9171.790')] [2023-12-27 03:15:01,067][105620] Updated weights for policy 1, policy_version 1630838 (0.0009) [2023-12-27 03:15:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001627504_416702464.pth... [2023-12-27 03:15:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001626352_416407552.pth [2023-12-27 03:15:01,134][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001630848_417554432.pth... [2023-12-27 03:15:01,134][105620] Updated weights for policy 1, policy_version 1630848 (0.0010) [2023-12-27 03:15:01,139][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001629728_417267712.pth [2023-12-27 03:15:01,332][105692] Updated weights for policy 0, policy_version 1627512 (0.0010) [2023-12-27 03:15:01,396][105692] Updated weights for policy 0, policy_version 1627522 (0.0009) [2023-12-27 03:15:01,457][105692] Updated weights for policy 0, policy_version 1627532 (0.0007) [2023-12-27 03:15:02,005][105620] Updated weights for policy 1, policy_version 1630858 (0.0009) [2023-12-27 03:15:02,063][105620] Updated weights for policy 1, policy_version 1630868 (0.0008) [2023-12-27 03:15:02,064][105692] Updated weights for policy 0, policy_version 1627542 (0.0005) [2023-12-27 03:15:02,112][105692] Updated weights for policy 0, policy_version 1627552 (0.0005) [2023-12-27 03:15:02,114][105620] Updated weights for policy 1, policy_version 1630878 (0.0008) [2023-12-27 03:15:02,170][105692] Updated weights for policy 0, policy_version 1627562 (0.0005) [2023-12-27 03:15:02,799][105692] Updated weights for policy 0, policy_version 1627572 (0.0007) [2023-12-27 03:15:02,857][105692] Updated weights for policy 0, policy_version 1627582 (0.0009) [2023-12-27 03:15:02,910][105692] Updated weights for policy 0, policy_version 1627592 (0.0007) [2023-12-27 03:15:02,946][105620] Updated weights for policy 1, policy_version 1630888 (0.0010) [2023-12-27 03:15:03,006][105620] Updated weights for policy 1, policy_version 1630898 (0.0008) [2023-12-27 03:15:03,066][105620] Updated weights for policy 1, policy_version 1630908 (0.0008) [2023-12-27 03:15:03,526][105692] Updated weights for policy 0, policy_version 1627602 (0.0006) [2023-12-27 03:15:03,571][105692] Updated weights for policy 0, policy_version 1627612 (0.0005) [2023-12-27 03:15:03,618][105692] Updated weights for policy 0, policy_version 1627622 (0.0005) [2023-12-27 03:15:03,675][105692] Updated weights for policy 0, policy_version 1627632 (0.0005) [2023-12-27 03:15:03,912][105620] Updated weights for policy 1, policy_version 1630918 (0.0009) [2023-12-27 03:15:03,974][105620] Updated weights for policy 1, policy_version 1630928 (0.0008) [2023-12-27 03:15:04,032][105620] Updated weights for policy 1, policy_version 1630938 (0.0006) [2023-12-27 03:15:04,343][105692] Updated weights for policy 0, policy_version 1627642 (0.0008) [2023-12-27 03:15:04,404][105692] Updated weights for policy 0, policy_version 1627652 (0.0008) [2023-12-27 03:15:04,464][105692] Updated weights for policy 0, policy_version 1627662 (0.0008) [2023-12-27 03:15:04,749][105620] Updated weights for policy 1, policy_version 1630948 (0.0008) [2023-12-27 03:15:04,799][105620] Updated weights for policy 1, policy_version 1630958 (0.0010) [2023-12-27 03:15:04,850][105620] Updated weights for policy 1, policy_version 1630968 (0.0010) [2023-12-27 03:15:05,221][105692] Updated weights for policy 0, policy_version 1627672 (0.0008) [2023-12-27 03:15:05,271][105692] Updated weights for policy 0, policy_version 1627682 (0.0007) [2023-12-27 03:15:05,320][105692] Updated weights for policy 0, policy_version 1627692 (0.0008) [2023-12-27 03:15:05,601][105620] Updated weights for policy 1, policy_version 1630978 (0.0010) [2023-12-27 03:15:05,658][105620] Updated weights for policy 1, policy_version 1630988 (0.0010) [2023-12-27 03:15:05,716][105620] Updated weights for policy 1, policy_version 1630998 (0.0010) [2023-12-27 03:15:05,785][105620] Updated weights for policy 1, policy_version 1631008 (0.0010) [2023-12-27 03:15:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 834347008. Throughput: 0: 10071.7, 1: 9550.0. Samples: 834338100. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:15:06,063][104569] Avg episode reward: [(0, '8439.579'), (1, '9086.812')] [2023-12-27 03:15:06,101][105692] Updated weights for policy 0, policy_version 1627702 (0.0008) [2023-12-27 03:15:06,167][105692] Updated weights for policy 0, policy_version 1627712 (0.0008) [2023-12-27 03:15:06,220][105692] Updated weights for policy 0, policy_version 1627722 (0.0008) [2023-12-27 03:15:06,534][105620] Updated weights for policy 1, policy_version 1631018 (0.0011) [2023-12-27 03:15:06,595][105620] Updated weights for policy 1, policy_version 1631028 (0.0011) [2023-12-27 03:15:06,658][105620] Updated weights for policy 1, policy_version 1631038 (0.0010) [2023-12-27 03:15:07,015][105692] Updated weights for policy 0, policy_version 1627732 (0.0008) [2023-12-27 03:15:07,072][105692] Updated weights for policy 0, policy_version 1627742 (0.0008) [2023-12-27 03:15:07,121][105692] Updated weights for policy 0, policy_version 1627752 (0.0008) [2023-12-27 03:15:07,401][105620] Updated weights for policy 1, policy_version 1631048 (0.0010) [2023-12-27 03:15:07,461][105620] Updated weights for policy 1, policy_version 1631058 (0.0008) [2023-12-27 03:15:07,520][105620] Updated weights for policy 1, policy_version 1631068 (0.0007) [2023-12-27 03:15:07,875][105692] Updated weights for policy 0, policy_version 1627762 (0.0008) [2023-12-27 03:15:07,930][105692] Updated weights for policy 0, policy_version 1627772 (0.0009) [2023-12-27 03:15:07,985][105692] Updated weights for policy 0, policy_version 1627782 (0.0009) [2023-12-27 03:15:08,039][105692] Updated weights for policy 0, policy_version 1627792 (0.0009) [2023-12-27 03:15:08,207][105620] Updated weights for policy 1, policy_version 1631078 (0.0007) [2023-12-27 03:15:08,254][105620] Updated weights for policy 1, policy_version 1631088 (0.0009) [2023-12-27 03:15:08,312][105620] Updated weights for policy 1, policy_version 1631098 (0.0009) [2023-12-27 03:15:08,857][105692] Updated weights for policy 0, policy_version 1627802 (0.0009) [2023-12-27 03:15:08,907][105692] Updated weights for policy 0, policy_version 1627812 (0.0008) [2023-12-27 03:15:08,961][105692] Updated weights for policy 0, policy_version 1627822 (0.0009) [2023-12-27 03:15:08,990][105620] Updated weights for policy 1, policy_version 1631108 (0.0009) [2023-12-27 03:15:09,035][105620] Updated weights for policy 1, policy_version 1631118 (0.0010) [2023-12-27 03:15:09,087][105620] Updated weights for policy 1, policy_version 1631128 (0.0010) [2023-12-27 03:15:09,719][105692] Updated weights for policy 0, policy_version 1627832 (0.0008) [2023-12-27 03:15:09,786][105692] Updated weights for policy 0, policy_version 1627842 (0.0009) [2023-12-27 03:15:09,851][105692] Updated weights for policy 0, policy_version 1627852 (0.0007) [2023-12-27 03:15:09,895][105620] Updated weights for policy 1, policy_version 1631138 (0.0010) [2023-12-27 03:15:09,964][105620] Updated weights for policy 1, policy_version 1631148 (0.0009) [2023-12-27 03:15:10,026][105620] Updated weights for policy 1, policy_version 1631158 (0.0009) [2023-12-27 03:15:10,091][105620] Updated weights for policy 1, policy_version 1631168 (0.0008) [2023-12-27 03:15:10,562][105692] Updated weights for policy 0, policy_version 1627862 (0.0009) [2023-12-27 03:15:10,624][105692] Updated weights for policy 0, policy_version 1627872 (0.0009) [2023-12-27 03:15:10,680][105692] Updated weights for policy 0, policy_version 1627882 (0.0009) [2023-12-27 03:15:10,788][105620] Updated weights for policy 1, policy_version 1631178 (0.0009) [2023-12-27 03:15:10,845][105620] Updated weights for policy 1, policy_version 1631188 (0.0008) [2023-12-27 03:15:10,902][105620] Updated weights for policy 1, policy_version 1631198 (0.0009) [2023-12-27 03:15:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 834445312. Throughput: 0: 10006.2, 1: 9513.8. Samples: 834450704. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:15:11,062][104569] Avg episode reward: [(0, '8444.055'), (1, '9088.531')] [2023-12-27 03:15:11,506][105692] Updated weights for policy 0, policy_version 1627892 (0.0008) [2023-12-27 03:15:11,561][105692] Updated weights for policy 0, policy_version 1627902 (0.0009) [2023-12-27 03:15:11,618][105692] Updated weights for policy 0, policy_version 1627912 (0.0009) [2023-12-27 03:15:11,685][105620] Updated weights for policy 1, policy_version 1631208 (0.0008) [2023-12-27 03:15:11,746][105620] Updated weights for policy 1, policy_version 1631218 (0.0009) [2023-12-27 03:15:11,805][105620] Updated weights for policy 1, policy_version 1631228 (0.0010) [2023-12-27 03:15:12,403][105692] Updated weights for policy 0, policy_version 1627922 (0.0009) [2023-12-27 03:15:12,460][105692] Updated weights for policy 0, policy_version 1627932 (0.0007) [2023-12-27 03:15:12,513][105692] Updated weights for policy 0, policy_version 1627942 (0.0007) [2023-12-27 03:15:12,568][105692] Updated weights for policy 0, policy_version 1627952 (0.0006) [2023-12-27 03:15:12,573][105620] Updated weights for policy 1, policy_version 1631238 (0.0009) [2023-12-27 03:15:12,642][105620] Updated weights for policy 1, policy_version 1631248 (0.0009) [2023-12-27 03:15:12,710][105620] Updated weights for policy 1, policy_version 1631258 (0.0009) [2023-12-27 03:15:13,198][105692] Updated weights for policy 0, policy_version 1627962 (0.0006) [2023-12-27 03:15:13,254][105692] Updated weights for policy 0, policy_version 1627972 (0.0008) [2023-12-27 03:15:13,316][105692] Updated weights for policy 0, policy_version 1627982 (0.0006) [2023-12-27 03:15:13,495][105620] Updated weights for policy 1, policy_version 1631268 (0.0010) [2023-12-27 03:15:13,542][105620] Updated weights for policy 1, policy_version 1631278 (0.0010) [2023-12-27 03:15:13,587][105620] Updated weights for policy 1, policy_version 1631288 (0.0010) [2023-12-27 03:15:14,014][105692] Updated weights for policy 0, policy_version 1627992 (0.0009) [2023-12-27 03:15:14,067][105692] Updated weights for policy 0, policy_version 1628002 (0.0010) [2023-12-27 03:15:14,121][105692] Updated weights for policy 0, policy_version 1628013 (0.0010) [2023-12-27 03:15:14,244][105620] Updated weights for policy 1, policy_version 1631298 (0.0010) [2023-12-27 03:15:14,302][105620] Updated weights for policy 1, policy_version 1631308 (0.0009) [2023-12-27 03:15:14,355][105620] Updated weights for policy 1, policy_version 1631318 (0.0008) [2023-12-27 03:15:14,403][105620] Updated weights for policy 1, policy_version 1631328 (0.0007) [2023-12-27 03:15:14,959][105692] Updated weights for policy 0, policy_version 1628023 (0.0009) [2023-12-27 03:15:15,012][105692] Updated weights for policy 0, policy_version 1628033 (0.0008) [2023-12-27 03:15:15,068][105692] Updated weights for policy 0, policy_version 1628043 (0.0007) [2023-12-27 03:15:15,105][105620] Updated weights for policy 1, policy_version 1631338 (0.0011) [2023-12-27 03:15:15,168][105620] Updated weights for policy 1, policy_version 1631348 (0.0011) [2023-12-27 03:15:15,224][105620] Updated weights for policy 1, policy_version 1631358 (0.0010) [2023-12-27 03:15:15,817][105692] Updated weights for policy 0, policy_version 1628053 (0.0008) [2023-12-27 03:15:15,872][105692] Updated weights for policy 0, policy_version 1628063 (0.0008) [2023-12-27 03:15:15,930][105692] Updated weights for policy 0, policy_version 1628073 (0.0008) [2023-12-27 03:15:15,975][105620] Updated weights for policy 1, policy_version 1631368 (0.0010) [2023-12-27 03:15:16,030][105620] Updated weights for policy 1, policy_version 1631378 (0.0010) [2023-12-27 03:15:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 834535424. Throughput: 0: 9964.5, 1: 9467.0. Samples: 834506284. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:15:16,062][104569] Avg episode reward: [(0, '8625.443'), (1, '9082.402')] [2023-12-27 03:15:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001628080_416849920.pth... [2023-12-27 03:15:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001626928_416555008.pth [2023-12-27 03:15:16,088][105620] Updated weights for policy 1, policy_version 1631388 (0.0010) [2023-12-27 03:15:16,108][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001631392_417693696.pth... [2023-12-27 03:15:16,111][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001630272_417406976.pth [2023-12-27 03:15:16,682][105692] Updated weights for policy 0, policy_version 1628083 (0.0007) [2023-12-27 03:15:16,750][105692] Updated weights for policy 0, policy_version 1628093 (0.0009) [2023-12-27 03:15:16,790][105620] Updated weights for policy 1, policy_version 1631398 (0.0007) [2023-12-27 03:15:16,811][105692] Updated weights for policy 0, policy_version 1628103 (0.0009) [2023-12-27 03:15:16,844][105620] Updated weights for policy 1, policy_version 1631408 (0.0005) [2023-12-27 03:15:16,905][105620] Updated weights for policy 1, policy_version 1631418 (0.0006) [2023-12-27 03:15:17,499][105692] Updated weights for policy 0, policy_version 1628113 (0.0010) [2023-12-27 03:15:17,566][105692] Updated weights for policy 0, policy_version 1628123 (0.0008) [2023-12-27 03:15:17,573][105620] Updated weights for policy 1, policy_version 1631428 (0.0006) [2023-12-27 03:15:17,611][105692] Updated weights for policy 0, policy_version 1628133 (0.0007) [2023-12-27 03:15:17,618][105620] Updated weights for policy 1, policy_version 1631438 (0.0006) [2023-12-27 03:15:17,657][105692] Updated weights for policy 0, policy_version 1628143 (0.0007) [2023-12-27 03:15:17,667][105620] Updated weights for policy 1, policy_version 1631448 (0.0006) [2023-12-27 03:15:18,311][105620] Updated weights for policy 1, policy_version 1631458 (0.0009) [2023-12-27 03:15:18,373][105620] Updated weights for policy 1, policy_version 1631468 (0.0008) [2023-12-27 03:15:18,424][105620] Updated weights for policy 1, policy_version 1631478 (0.0009) [2023-12-27 03:15:18,475][105620] Updated weights for policy 1, policy_version 1631488 (0.0008) [2023-12-27 03:15:18,485][105692] Updated weights for policy 0, policy_version 1628153 (0.0008) [2023-12-27 03:15:18,543][105692] Updated weights for policy 0, policy_version 1628163 (0.0009) [2023-12-27 03:15:18,597][105692] Updated weights for policy 0, policy_version 1628173 (0.0009) [2023-12-27 03:15:19,209][105620] Updated weights for policy 1, policy_version 1631498 (0.0009) [2023-12-27 03:15:19,273][105620] Updated weights for policy 1, policy_version 1631508 (0.0010) [2023-12-27 03:15:19,324][105620] Updated weights for policy 1, policy_version 1631518 (0.0009) [2023-12-27 03:15:19,356][105692] Updated weights for policy 0, policy_version 1628183 (0.0009) [2023-12-27 03:15:19,410][105692] Updated weights for policy 0, policy_version 1628193 (0.0009) [2023-12-27 03:15:19,460][105692] Updated weights for policy 0, policy_version 1628203 (0.0008) [2023-12-27 03:15:20,098][105620] Updated weights for policy 1, policy_version 1631528 (0.0009) [2023-12-27 03:15:20,170][105620] Updated weights for policy 1, policy_version 1631538 (0.0006) [2023-12-27 03:15:20,233][105620] Updated weights for policy 1, policy_version 1631548 (0.0009) [2023-12-27 03:15:20,264][105692] Updated weights for policy 0, policy_version 1628213 (0.0009) [2023-12-27 03:15:20,329][105692] Updated weights for policy 0, policy_version 1628223 (0.0009) [2023-12-27 03:15:20,385][105692] Updated weights for policy 0, policy_version 1628233 (0.0009) [2023-12-27 03:15:20,924][105620] Updated weights for policy 1, policy_version 1631558 (0.0008) [2023-12-27 03:15:20,978][105620] Updated weights for policy 1, policy_version 1631568 (0.0008) [2023-12-27 03:15:21,024][105620] Updated weights for policy 1, policy_version 1631578 (0.0009) [2023-12-27 03:15:21,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 834625536. Throughput: 0: 9752.3, 1: 9514.4. Samples: 834621400. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:15:21,062][104569] Avg episode reward: [(0, '8713.543'), (1, '8987.998')] [2023-12-27 03:15:21,235][105692] Updated weights for policy 0, policy_version 1628243 (0.0009) [2023-12-27 03:15:21,301][105692] Updated weights for policy 0, policy_version 1628253 (0.0010) [2023-12-27 03:15:21,370][105692] Updated weights for policy 0, policy_version 1628263 (0.0008) [2023-12-27 03:15:21,774][105620] Updated weights for policy 1, policy_version 1631588 (0.0009) [2023-12-27 03:15:21,833][105620] Updated weights for policy 1, policy_version 1631598 (0.0007) [2023-12-27 03:15:21,898][105620] Updated weights for policy 1, policy_version 1631608 (0.0006) [2023-12-27 03:15:22,214][105692] Updated weights for policy 0, policy_version 1628273 (0.0009) [2023-12-27 03:15:22,287][105692] Updated weights for policy 0, policy_version 1628283 (0.0010) [2023-12-27 03:15:22,350][105692] Updated weights for policy 0, policy_version 1628293 (0.0009) [2023-12-27 03:15:22,416][105692] Updated weights for policy 0, policy_version 1628303 (0.0009) [2023-12-27 03:15:22,548][105620] Updated weights for policy 1, policy_version 1631618 (0.0008) [2023-12-27 03:15:22,614][105620] Updated weights for policy 1, policy_version 1631628 (0.0010) [2023-12-27 03:15:22,688][105620] Updated weights for policy 1, policy_version 1631638 (0.0009) [2023-12-27 03:15:22,755][105620] Updated weights for policy 1, policy_version 1631648 (0.0010) [2023-12-27 03:15:23,095][105692] Updated weights for policy 0, policy_version 1628313 (0.0011) [2023-12-27 03:15:23,154][105692] Updated weights for policy 0, policy_version 1628323 (0.0009) [2023-12-27 03:15:23,204][105692] Updated weights for policy 0, policy_version 1628333 (0.0008) [2023-12-27 03:15:23,560][105620] Updated weights for policy 1, policy_version 1631658 (0.0008) [2023-12-27 03:15:23,623][105620] Updated weights for policy 1, policy_version 1631668 (0.0008) [2023-12-27 03:15:23,679][105620] Updated weights for policy 1, policy_version 1631678 (0.0008) [2023-12-27 03:15:23,953][105692] Updated weights for policy 0, policy_version 1628343 (0.0011) [2023-12-27 03:15:24,017][105692] Updated weights for policy 0, policy_version 1628353 (0.0010) [2023-12-27 03:15:24,083][105692] Updated weights for policy 0, policy_version 1628363 (0.0011) [2023-12-27 03:15:24,291][105620] Updated weights for policy 1, policy_version 1631688 (0.0006) [2023-12-27 03:15:24,345][105620] Updated weights for policy 1, policy_version 1631698 (0.0005) [2023-12-27 03:15:24,396][105620] Updated weights for policy 1, policy_version 1631708 (0.0005) [2023-12-27 03:15:24,745][105692] Updated weights for policy 0, policy_version 1628373 (0.0011) [2023-12-27 03:15:24,806][105692] Updated weights for policy 0, policy_version 1628383 (0.0011) [2023-12-27 03:15:24,868][105692] Updated weights for policy 0, policy_version 1628393 (0.0008) [2023-12-27 03:15:24,972][105620] Updated weights for policy 1, policy_version 1631718 (0.0005) [2023-12-27 03:15:25,030][105620] Updated weights for policy 1, policy_version 1631728 (0.0006) [2023-12-27 03:15:25,081][105620] Updated weights for policy 1, policy_version 1631738 (0.0006) [2023-12-27 03:15:25,585][105692] Updated weights for policy 0, policy_version 1628403 (0.0006) [2023-12-27 03:15:25,632][105692] Updated weights for policy 0, policy_version 1628413 (0.0006) [2023-12-27 03:15:25,686][105692] Updated weights for policy 0, policy_version 1628423 (0.0009) [2023-12-27 03:15:25,729][105620] Updated weights for policy 1, policy_version 1631748 (0.0006) [2023-12-27 03:15:25,788][105620] Updated weights for policy 1, policy_version 1631758 (0.0008) [2023-12-27 03:15:25,853][105620] Updated weights for policy 1, policy_version 1631768 (0.0008) [2023-12-27 03:15:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 834732032. Throughput: 0: 9608.1, 1: 9568.7. Samples: 834736408. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:15:26,063][104569] Avg episode reward: [(0, '8717.152'), (1, '8989.487')] [2023-12-27 03:15:26,341][105692] Updated weights for policy 0, policy_version 1628433 (0.0011) [2023-12-27 03:15:26,401][105692] Updated weights for policy 0, policy_version 1628443 (0.0011) [2023-12-27 03:15:26,451][105692] Updated weights for policy 0, policy_version 1628453 (0.0011) [2023-12-27 03:15:26,511][105692] Updated weights for policy 0, policy_version 1628463 (0.0011) [2023-12-27 03:15:26,611][105620] Updated weights for policy 1, policy_version 1631778 (0.0010) [2023-12-27 03:15:26,671][105620] Updated weights for policy 1, policy_version 1631788 (0.0009) [2023-12-27 03:15:26,732][105620] Updated weights for policy 1, policy_version 1631798 (0.0008) [2023-12-27 03:15:26,792][105620] Updated weights for policy 1, policy_version 1631808 (0.0005) [2023-12-27 03:15:27,279][105692] Updated weights for policy 0, policy_version 1628473 (0.0010) [2023-12-27 03:15:27,339][105692] Updated weights for policy 0, policy_version 1628483 (0.0011) [2023-12-27 03:15:27,390][105692] Updated weights for policy 0, policy_version 1628493 (0.0011) [2023-12-27 03:15:27,463][105620] Updated weights for policy 1, policy_version 1631818 (0.0010) [2023-12-27 03:15:27,514][105620] Updated weights for policy 1, policy_version 1631828 (0.0010) [2023-12-27 03:15:27,563][105620] Updated weights for policy 1, policy_version 1631838 (0.0010) [2023-12-27 03:15:28,141][105692] Updated weights for policy 0, policy_version 1628503 (0.0010) [2023-12-27 03:15:28,206][105692] Updated weights for policy 0, policy_version 1628513 (0.0011) [2023-12-27 03:15:28,267][105692] Updated weights for policy 0, policy_version 1628523 (0.0010) [2023-12-27 03:15:28,342][105620] Updated weights for policy 1, policy_version 1631848 (0.0009) [2023-12-27 03:15:28,405][105620] Updated weights for policy 1, policy_version 1631858 (0.0010) [2023-12-27 03:15:28,464][105620] Updated weights for policy 1, policy_version 1631868 (0.0010) [2023-12-27 03:15:29,015][105692] Updated weights for policy 0, policy_version 1628533 (0.0011) [2023-12-27 03:15:29,066][105692] Updated weights for policy 0, policy_version 1628543 (0.0010) [2023-12-27 03:15:29,118][105692] Updated weights for policy 0, policy_version 1628553 (0.0010) [2023-12-27 03:15:29,235][105620] Updated weights for policy 1, policy_version 1631878 (0.0011) [2023-12-27 03:15:29,297][105620] Updated weights for policy 1, policy_version 1631888 (0.0010) [2023-12-27 03:15:29,363][105620] Updated weights for policy 1, policy_version 1631898 (0.0011) [2023-12-27 03:15:29,850][105692] Updated weights for policy 0, policy_version 1628563 (0.0010) [2023-12-27 03:15:29,912][105692] Updated weights for policy 0, policy_version 1628573 (0.0008) [2023-12-27 03:15:29,979][105692] Updated weights for policy 0, policy_version 1628583 (0.0008) [2023-12-27 03:15:30,113][105620] Updated weights for policy 1, policy_version 1631908 (0.0010) [2023-12-27 03:15:30,172][105620] Updated weights for policy 1, policy_version 1631918 (0.0010) [2023-12-27 03:15:30,229][105620] Updated weights for policy 1, policy_version 1631928 (0.0010) [2023-12-27 03:15:30,733][105692] Updated weights for policy 0, policy_version 1628593 (0.0008) [2023-12-27 03:15:30,794][105692] Updated weights for policy 0, policy_version 1628603 (0.0010) [2023-12-27 03:15:30,846][105692] Updated weights for policy 0, policy_version 1628613 (0.0010) [2023-12-27 03:15:30,898][105692] Updated weights for policy 0, policy_version 1628623 (0.0011) [2023-12-27 03:15:30,910][105620] Updated weights for policy 1, policy_version 1631938 (0.0009) [2023-12-27 03:15:30,960][105620] Updated weights for policy 1, policy_version 1631948 (0.0005) [2023-12-27 03:15:31,007][105620] Updated weights for policy 1, policy_version 1631958 (0.0006) [2023-12-27 03:15:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 834822144. Throughput: 0: 9581.0, 1: 9563.0. Samples: 834793792. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:15:31,063][104569] Avg episode reward: [(0, '8625.730'), (1, '9173.052')] [2023-12-27 03:15:31,067][105620] Updated weights for policy 1, policy_version 1631968 (0.0010) [2023-12-27 03:15:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001631968_417841152.pth... [2023-12-27 03:15:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001628624_416989184.pth... [2023-12-27 03:15:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001630848_417554432.pth [2023-12-27 03:15:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001627504_416702464.pth [2023-12-27 03:15:31,597][105692] Updated weights for policy 0, policy_version 1628633 (0.0007) [2023-12-27 03:15:31,662][105692] Updated weights for policy 0, policy_version 1628643 (0.0009) [2023-12-27 03:15:31,725][105692] Updated weights for policy 0, policy_version 1628653 (0.0008) [2023-12-27 03:15:31,863][105620] Updated weights for policy 1, policy_version 1631978 (0.0008) [2023-12-27 03:15:31,927][105620] Updated weights for policy 1, policy_version 1631988 (0.0008) [2023-12-27 03:15:31,990][105620] Updated weights for policy 1, policy_version 1631998 (0.0007) [2023-12-27 03:15:32,473][105692] Updated weights for policy 0, policy_version 1628663 (0.0007) [2023-12-27 03:15:32,532][105692] Updated weights for policy 0, policy_version 1628673 (0.0009) [2023-12-27 03:15:32,588][105692] Updated weights for policy 0, policy_version 1628683 (0.0010) [2023-12-27 03:15:32,651][105620] Updated weights for policy 1, policy_version 1632008 (0.0008) [2023-12-27 03:15:32,705][105620] Updated weights for policy 1, policy_version 1632018 (0.0009) [2023-12-27 03:15:32,755][105620] Updated weights for policy 1, policy_version 1632028 (0.0008) [2023-12-27 03:15:33,394][105620] Updated weights for policy 1, policy_version 1632038 (0.0008) [2023-12-27 03:15:33,417][105692] Updated weights for policy 0, policy_version 1628693 (0.0008) [2023-12-27 03:15:33,443][105620] Updated weights for policy 1, policy_version 1632048 (0.0008) [2023-12-27 03:15:33,474][105692] Updated weights for policy 0, policy_version 1628703 (0.0008) [2023-12-27 03:15:33,499][105620] Updated weights for policy 1, policy_version 1632058 (0.0007) [2023-12-27 03:15:33,525][105692] Updated weights for policy 0, policy_version 1628713 (0.0006) [2023-12-27 03:15:34,096][105692] Updated weights for policy 0, policy_version 1628723 (0.0009) [2023-12-27 03:15:34,150][105692] Updated weights for policy 0, policy_version 1628733 (0.0006) [2023-12-27 03:15:34,162][105620] Updated weights for policy 1, policy_version 1632068 (0.0008) [2023-12-27 03:15:34,216][105692] Updated weights for policy 0, policy_version 1628743 (0.0008) [2023-12-27 03:15:34,228][105620] Updated weights for policy 1, policy_version 1632078 (0.0006) [2023-12-27 03:15:34,292][105620] Updated weights for policy 1, policy_version 1632088 (0.0006) [2023-12-27 03:15:34,949][105692] Updated weights for policy 0, policy_version 1628753 (0.0010) [2023-12-27 03:15:34,967][105620] Updated weights for policy 1, policy_version 1632098 (0.0006) [2023-12-27 03:15:35,006][105692] Updated weights for policy 0, policy_version 1628763 (0.0007) [2023-12-27 03:15:35,024][105620] Updated weights for policy 1, policy_version 1632108 (0.0008) [2023-12-27 03:15:35,060][105692] Updated weights for policy 0, policy_version 1628773 (0.0007) [2023-12-27 03:15:35,078][105620] Updated weights for policy 1, policy_version 1632118 (0.0006) [2023-12-27 03:15:35,112][105692] Updated weights for policy 0, policy_version 1628783 (0.0007) [2023-12-27 03:15:35,126][105620] Updated weights for policy 1, policy_version 1632128 (0.0006) [2023-12-27 03:15:35,809][105620] Updated weights for policy 1, policy_version 1632138 (0.0009) [2023-12-27 03:15:35,856][105620] Updated weights for policy 1, policy_version 1632148 (0.0008) [2023-12-27 03:15:35,916][105620] Updated weights for policy 1, policy_version 1632158 (0.0008) [2023-12-27 03:15:35,917][105692] Updated weights for policy 0, policy_version 1628793 (0.0008) [2023-12-27 03:15:35,967][105692] Updated weights for policy 0, policy_version 1628803 (0.0009) [2023-12-27 03:15:36,015][105692] Updated weights for policy 0, policy_version 1628813 (0.0008) [2023-12-27 03:15:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 834928640. Throughput: 0: 9570.7, 1: 9563.3. Samples: 834911008. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:15:36,063][104569] Avg episode reward: [(0, '8806.865'), (1, '9264.329')] [2023-12-27 03:15:36,651][105620] Updated weights for policy 1, policy_version 1632168 (0.0010) [2023-12-27 03:15:36,711][105620] Updated weights for policy 1, policy_version 1632178 (0.0011) [2023-12-27 03:15:36,770][105620] Updated weights for policy 1, policy_version 1632188 (0.0011) [2023-12-27 03:15:36,813][105692] Updated weights for policy 0, policy_version 1628823 (0.0007) [2023-12-27 03:15:36,870][105692] Updated weights for policy 0, policy_version 1628833 (0.0009) [2023-12-27 03:15:36,925][105692] Updated weights for policy 0, policy_version 1628843 (0.0008) [2023-12-27 03:15:37,517][105620] Updated weights for policy 1, policy_version 1632198 (0.0008) [2023-12-27 03:15:37,569][105620] Updated weights for policy 1, policy_version 1632208 (0.0006) [2023-12-27 03:15:37,628][105620] Updated weights for policy 1, policy_version 1632218 (0.0008) [2023-12-27 03:15:37,667][105692] Updated weights for policy 0, policy_version 1628853 (0.0009) [2023-12-27 03:15:37,724][105692] Updated weights for policy 0, policy_version 1628863 (0.0008) [2023-12-27 03:15:37,778][105692] Updated weights for policy 0, policy_version 1628873 (0.0009) [2023-12-27 03:15:38,315][105620] Updated weights for policy 1, policy_version 1632228 (0.0011) [2023-12-27 03:15:38,382][105620] Updated weights for policy 1, policy_version 1632238 (0.0011) [2023-12-27 03:15:38,445][105620] Updated weights for policy 1, policy_version 1632248 (0.0011) [2023-12-27 03:15:38,610][105692] Updated weights for policy 0, policy_version 1628883 (0.0008) [2023-12-27 03:15:38,672][105692] Updated weights for policy 0, policy_version 1628893 (0.0008) [2023-12-27 03:15:38,732][105692] Updated weights for policy 0, policy_version 1628903 (0.0008) [2023-12-27 03:15:39,197][105620] Updated weights for policy 1, policy_version 1632258 (0.0011) [2023-12-27 03:15:39,263][105620] Updated weights for policy 1, policy_version 1632268 (0.0011) [2023-12-27 03:15:39,329][105620] Updated weights for policy 1, policy_version 1632278 (0.0011) [2023-12-27 03:15:39,395][105620] Updated weights for policy 1, policy_version 1632288 (0.0009) [2023-12-27 03:15:39,526][105692] Updated weights for policy 0, policy_version 1628913 (0.0009) [2023-12-27 03:15:39,588][105692] Updated weights for policy 0, policy_version 1628923 (0.0008) [2023-12-27 03:15:39,648][105692] Updated weights for policy 0, policy_version 1628933 (0.0008) [2023-12-27 03:15:39,708][105692] Updated weights for policy 0, policy_version 1628943 (0.0008) [2023-12-27 03:15:40,141][105620] Updated weights for policy 1, policy_version 1632298 (0.0007) [2023-12-27 03:15:40,204][105620] Updated weights for policy 1, policy_version 1632308 (0.0006) [2023-12-27 03:15:40,275][105620] Updated weights for policy 1, policy_version 1632318 (0.0008) [2023-12-27 03:15:40,468][105692] Updated weights for policy 0, policy_version 1628953 (0.0008) [2023-12-27 03:15:40,536][105692] Updated weights for policy 0, policy_version 1628963 (0.0009) [2023-12-27 03:15:40,607][105692] Updated weights for policy 0, policy_version 1628973 (0.0007) [2023-12-27 03:15:41,006][105620] Updated weights for policy 1, policy_version 1632328 (0.0007) [2023-12-27 03:15:41,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19114.7, 300 sec: 19494.2). Total num frames: 835010560. Throughput: 0: 9464.0, 1: 9586.9. Samples: 835022888. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:15:41,063][104569] Avg episode reward: [(0, '8626.594'), (1, '8989.848')] [2023-12-27 03:15:41,078][105620] Updated weights for policy 1, policy_version 1632338 (0.0010) [2023-12-27 03:15:41,151][105620] Updated weights for policy 1, policy_version 1632348 (0.0010) [2023-12-27 03:15:41,352][105692] Updated weights for policy 0, policy_version 1628983 (0.0009) [2023-12-27 03:15:41,421][105692] Updated weights for policy 0, policy_version 1628993 (0.0010) [2023-12-27 03:15:41,477][105692] Updated weights for policy 0, policy_version 1629003 (0.0009) [2023-12-27 03:15:41,938][105620] Updated weights for policy 1, policy_version 1632358 (0.0010) [2023-12-27 03:15:41,998][105620] Updated weights for policy 1, policy_version 1632368 (0.0009) [2023-12-27 03:15:42,062][105620] Updated weights for policy 1, policy_version 1632378 (0.0010) [2023-12-27 03:15:42,255][105692] Updated weights for policy 0, policy_version 1629013 (0.0009) [2023-12-27 03:15:42,321][105692] Updated weights for policy 0, policy_version 1629023 (0.0008) [2023-12-27 03:15:42,386][105692] Updated weights for policy 0, policy_version 1629033 (0.0007) [2023-12-27 03:15:42,872][105620] Updated weights for policy 1, policy_version 1632388 (0.0009) [2023-12-27 03:15:42,936][105620] Updated weights for policy 1, policy_version 1632398 (0.0008) [2023-12-27 03:15:42,995][105620] Updated weights for policy 1, policy_version 1632408 (0.0009) [2023-12-27 03:15:43,111][105692] Updated weights for policy 0, policy_version 1629043 (0.0007) [2023-12-27 03:15:43,171][105692] Updated weights for policy 0, policy_version 1629053 (0.0008) [2023-12-27 03:15:43,230][105692] Updated weights for policy 0, policy_version 1629063 (0.0009) [2023-12-27 03:15:43,772][105620] Updated weights for policy 1, policy_version 1632418 (0.0009) [2023-12-27 03:15:43,820][105620] Updated weights for policy 1, policy_version 1632428 (0.0009) [2023-12-27 03:15:43,874][105620] Updated weights for policy 1, policy_version 1632438 (0.0009) [2023-12-27 03:15:43,923][105620] Updated weights for policy 1, policy_version 1632448 (0.0009) [2023-12-27 03:15:43,969][105692] Updated weights for policy 0, policy_version 1629073 (0.0009) [2023-12-27 03:15:44,033][105692] Updated weights for policy 0, policy_version 1629083 (0.0008) [2023-12-27 03:15:44,084][105692] Updated weights for policy 0, policy_version 1629093 (0.0006) [2023-12-27 03:15:44,143][105692] Updated weights for policy 0, policy_version 1629103 (0.0009) [2023-12-27 03:15:44,681][105620] Updated weights for policy 1, policy_version 1632458 (0.0010) [2023-12-27 03:15:44,738][105620] Updated weights for policy 1, policy_version 1632468 (0.0009) [2023-12-27 03:15:44,804][105620] Updated weights for policy 1, policy_version 1632478 (0.0009) [2023-12-27 03:15:44,833][105692] Updated weights for policy 0, policy_version 1629113 (0.0008) [2023-12-27 03:15:44,892][105692] Updated weights for policy 0, policy_version 1629123 (0.0011) [2023-12-27 03:15:44,945][105692] Updated weights for policy 0, policy_version 1629133 (0.0009) [2023-12-27 03:15:45,547][105620] Updated weights for policy 1, policy_version 1632488 (0.0009) [2023-12-27 03:15:45,598][105620] Updated weights for policy 1, policy_version 1632498 (0.0009) [2023-12-27 03:15:45,658][105620] Updated weights for policy 1, policy_version 1632508 (0.0009) [2023-12-27 03:15:45,709][105692] Updated weights for policy 0, policy_version 1629143 (0.0009) [2023-12-27 03:15:45,757][105692] Updated weights for policy 0, policy_version 1629153 (0.0008) [2023-12-27 03:15:45,805][105692] Updated weights for policy 0, policy_version 1629163 (0.0009) [2023-12-27 03:15:46,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19114.7, 300 sec: 19438.6). Total num frames: 835108864. Throughput: 0: 9424.6, 1: 9532.3. Samples: 835076780. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:15:46,062][104569] Avg episode reward: [(0, '8625.190'), (1, '9081.413')] [2023-12-27 03:15:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001629168_417128448.pth... [2023-12-27 03:15:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001632512_417980416.pth... [2023-12-27 03:15:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001631392_417693696.pth [2023-12-27 03:15:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001628080_416849920.pth [2023-12-27 03:15:46,421][105620] Updated weights for policy 1, policy_version 1632518 (0.0008) [2023-12-27 03:15:46,469][105620] Updated weights for policy 1, policy_version 1632528 (0.0009) [2023-12-27 03:15:46,516][105620] Updated weights for policy 1, policy_version 1632538 (0.0008) [2023-12-27 03:15:46,585][105692] Updated weights for policy 0, policy_version 1629173 (0.0009) [2023-12-27 03:15:46,648][105692] Updated weights for policy 0, policy_version 1629183 (0.0009) [2023-12-27 03:15:46,707][105692] Updated weights for policy 0, policy_version 1629193 (0.0009) [2023-12-27 03:15:47,284][105620] Updated weights for policy 1, policy_version 1632548 (0.0009) [2023-12-27 03:15:47,342][105620] Updated weights for policy 1, policy_version 1632558 (0.0009) [2023-12-27 03:15:47,401][105620] Updated weights for policy 1, policy_version 1632568 (0.0009) [2023-12-27 03:15:47,469][105692] Updated weights for policy 0, policy_version 1629203 (0.0008) [2023-12-27 03:15:47,530][105692] Updated weights for policy 0, policy_version 1629213 (0.0009) [2023-12-27 03:15:47,589][105692] Updated weights for policy 0, policy_version 1629223 (0.0008) [2023-12-27 03:15:48,168][105620] Updated weights for policy 1, policy_version 1632578 (0.0009) [2023-12-27 03:15:48,224][105620] Updated weights for policy 1, policy_version 1632588 (0.0010) [2023-12-27 03:15:48,274][105620] Updated weights for policy 1, policy_version 1632598 (0.0009) [2023-12-27 03:15:48,286][105692] Updated weights for policy 0, policy_version 1629233 (0.0007) [2023-12-27 03:15:48,328][105620] Updated weights for policy 1, policy_version 1632608 (0.0009) [2023-12-27 03:15:48,348][105692] Updated weights for policy 0, policy_version 1629243 (0.0007) [2023-12-27 03:15:48,412][105692] Updated weights for policy 0, policy_version 1629253 (0.0008) [2023-12-27 03:15:48,468][105692] Updated weights for policy 0, policy_version 1629263 (0.0011) [2023-12-27 03:15:49,034][105620] Updated weights for policy 1, policy_version 1632618 (0.0005) [2023-12-27 03:15:49,102][105620] Updated weights for policy 1, policy_version 1632628 (0.0007) [2023-12-27 03:15:49,160][105620] Updated weights for policy 1, policy_version 1632638 (0.0009) [2023-12-27 03:15:49,183][105692] Updated weights for policy 0, policy_version 1629273 (0.0010) [2023-12-27 03:15:49,242][105692] Updated weights for policy 0, policy_version 1629283 (0.0010) [2023-12-27 03:15:49,295][105692] Updated weights for policy 0, policy_version 1629293 (0.0008) [2023-12-27 03:15:49,890][105620] Updated weights for policy 1, policy_version 1632648 (0.0008) [2023-12-27 03:15:49,955][105620] Updated weights for policy 1, policy_version 1632658 (0.0008) [2023-12-27 03:15:50,026][105620] Updated weights for policy 1, policy_version 1632668 (0.0008) [2023-12-27 03:15:50,089][105692] Updated weights for policy 0, policy_version 1629303 (0.0011) [2023-12-27 03:15:50,148][105692] Updated weights for policy 0, policy_version 1629313 (0.0011) [2023-12-27 03:15:50,208][105692] Updated weights for policy 0, policy_version 1629323 (0.0011) [2023-12-27 03:15:50,715][105620] Updated weights for policy 1, policy_version 1632678 (0.0008) [2023-12-27 03:15:50,770][105620] Updated weights for policy 1, policy_version 1632688 (0.0009) [2023-12-27 03:15:50,831][105620] Updated weights for policy 1, policy_version 1632698 (0.0007) [2023-12-27 03:15:50,999][105692] Updated weights for policy 0, policy_version 1629333 (0.0010) [2023-12-27 03:15:51,062][104569] Fps is (10 sec: 18841.9, 60 sec: 18978.1, 300 sec: 19438.7). Total num frames: 835198976. Throughput: 0: 9319.8, 1: 9612.5. Samples: 835190048. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:15:51,062][104569] Avg episode reward: [(0, '8713.562'), (1, '9263.907')] [2023-12-27 03:15:51,063][105692] Updated weights for policy 0, policy_version 1629343 (0.0009) [2023-12-27 03:15:51,125][105692] Updated weights for policy 0, policy_version 1629353 (0.0009) [2023-12-27 03:15:51,597][105620] Updated weights for policy 1, policy_version 1632708 (0.0009) [2023-12-27 03:15:51,659][105620] Updated weights for policy 1, policy_version 1632718 (0.0009) [2023-12-27 03:15:51,709][105620] Updated weights for policy 1, policy_version 1632728 (0.0009) [2023-12-27 03:15:51,874][105692] Updated weights for policy 0, policy_version 1629363 (0.0009) [2023-12-27 03:15:51,937][105692] Updated weights for policy 0, policy_version 1629373 (0.0009) [2023-12-27 03:15:52,000][105692] Updated weights for policy 0, policy_version 1629383 (0.0009) [2023-12-27 03:15:52,483][105620] Updated weights for policy 1, policy_version 1632738 (0.0008) [2023-12-27 03:15:52,544][105620] Updated weights for policy 1, policy_version 1632748 (0.0010) [2023-12-27 03:15:52,599][105620] Updated weights for policy 1, policy_version 1632758 (0.0009) [2023-12-27 03:15:52,657][105620] Updated weights for policy 1, policy_version 1632768 (0.0009) [2023-12-27 03:15:52,704][105692] Updated weights for policy 0, policy_version 1629393 (0.0010) [2023-12-27 03:15:52,763][105692] Updated weights for policy 0, policy_version 1629403 (0.0009) [2023-12-27 03:15:52,816][105692] Updated weights for policy 0, policy_version 1629413 (0.0010) [2023-12-27 03:15:52,880][105692] Updated weights for policy 0, policy_version 1629423 (0.0010) [2023-12-27 03:15:53,300][105620] Updated weights for policy 1, policy_version 1632778 (0.0009) [2023-12-27 03:15:53,355][105620] Updated weights for policy 1, policy_version 1632788 (0.0009) [2023-12-27 03:15:53,413][105620] Updated weights for policy 1, policy_version 1632798 (0.0008) [2023-12-27 03:15:53,575][105692] Updated weights for policy 0, policy_version 1629433 (0.0009) [2023-12-27 03:15:53,626][105692] Updated weights for policy 0, policy_version 1629443 (0.0009) [2023-12-27 03:15:53,681][105692] Updated weights for policy 0, policy_version 1629453 (0.0009) [2023-12-27 03:15:54,200][105620] Updated weights for policy 1, policy_version 1632808 (0.0009) [2023-12-27 03:15:54,265][105620] Updated weights for policy 1, policy_version 1632818 (0.0008) [2023-12-27 03:15:54,330][105620] Updated weights for policy 1, policy_version 1632828 (0.0009) [2023-12-27 03:15:54,344][105692] Updated weights for policy 0, policy_version 1629463 (0.0007) [2023-12-27 03:15:54,417][105692] Updated weights for policy 0, policy_version 1629473 (0.0009) [2023-12-27 03:15:54,482][105692] Updated weights for policy 0, policy_version 1629483 (0.0009) [2023-12-27 03:15:55,058][105620] Updated weights for policy 1, policy_version 1632838 (0.0009) [2023-12-27 03:15:55,093][105692] Updated weights for policy 0, policy_version 1629493 (0.0009) [2023-12-27 03:15:55,108][105620] Updated weights for policy 1, policy_version 1632848 (0.0006) [2023-12-27 03:15:55,145][105692] Updated weights for policy 0, policy_version 1629503 (0.0010) [2023-12-27 03:15:55,165][105620] Updated weights for policy 1, policy_version 1632858 (0.0010) [2023-12-27 03:15:55,197][105692] Updated weights for policy 0, policy_version 1629513 (0.0010) [2023-12-27 03:15:55,799][105692] Updated weights for policy 0, policy_version 1629523 (0.0005) [2023-12-27 03:15:55,859][105692] Updated weights for policy 0, policy_version 1629533 (0.0005) [2023-12-27 03:15:55,874][105620] Updated weights for policy 1, policy_version 1632868 (0.0011) [2023-12-27 03:15:55,918][105692] Updated weights for policy 0, policy_version 1629543 (0.0005) [2023-12-27 03:15:55,930][105620] Updated weights for policy 1, policy_version 1632878 (0.0011) [2023-12-27 03:15:55,985][105620] Updated weights for policy 1, policy_version 1632888 (0.0007) [2023-12-27 03:15:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 835305472. Throughput: 0: 9407.0, 1: 9621.8. Samples: 835307000. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:15:56,062][104569] Avg episode reward: [(0, '8528.560'), (1, '9172.468')] [2023-12-27 03:15:56,491][105692] Updated weights for policy 0, policy_version 1629553 (0.0005) [2023-12-27 03:15:56,547][105692] Updated weights for policy 0, policy_version 1629563 (0.0005) [2023-12-27 03:15:56,569][105620] Updated weights for policy 1, policy_version 1632898 (0.0009) [2023-12-27 03:15:56,608][105692] Updated weights for policy 0, policy_version 1629573 (0.0005) [2023-12-27 03:15:56,624][105620] Updated weights for policy 1, policy_version 1632908 (0.0007) [2023-12-27 03:15:56,663][105692] Updated weights for policy 0, policy_version 1629583 (0.0005) [2023-12-27 03:15:56,674][105620] Updated weights for policy 1, policy_version 1632918 (0.0009) [2023-12-27 03:15:56,723][105620] Updated weights for policy 1, policy_version 1632928 (0.0010) [2023-12-27 03:15:57,210][105692] Updated weights for policy 0, policy_version 1629593 (0.0006) [2023-12-27 03:15:57,269][105692] Updated weights for policy 0, policy_version 1629603 (0.0011) [2023-12-27 03:15:57,323][105692] Updated weights for policy 0, policy_version 1629613 (0.0011) [2023-12-27 03:15:57,402][105620] Updated weights for policy 1, policy_version 1632938 (0.0006) [2023-12-27 03:15:57,461][105620] Updated weights for policy 1, policy_version 1632948 (0.0010) [2023-12-27 03:15:57,515][105620] Updated weights for policy 1, policy_version 1632958 (0.0010) [2023-12-27 03:15:57,895][105692] Updated weights for policy 0, policy_version 1629623 (0.0007) [2023-12-27 03:15:57,944][105692] Updated weights for policy 0, policy_version 1629633 (0.0008) [2023-12-27 03:15:57,992][105692] Updated weights for policy 0, policy_version 1629643 (0.0010) [2023-12-27 03:15:58,187][105620] Updated weights for policy 1, policy_version 1632968 (0.0008) [2023-12-27 03:15:58,235][105620] Updated weights for policy 1, policy_version 1632978 (0.0008) [2023-12-27 03:15:58,292][105620] Updated weights for policy 1, policy_version 1632988 (0.0008) [2023-12-27 03:15:58,741][105692] Updated weights for policy 0, policy_version 1629653 (0.0010) [2023-12-27 03:15:58,814][105692] Updated weights for policy 0, policy_version 1629663 (0.0010) [2023-12-27 03:15:58,885][105692] Updated weights for policy 0, policy_version 1629673 (0.0011) [2023-12-27 03:15:59,121][105620] Updated weights for policy 1, policy_version 1632998 (0.0009) [2023-12-27 03:15:59,181][105620] Updated weights for policy 1, policy_version 1633008 (0.0008) [2023-12-27 03:15:59,245][105620] Updated weights for policy 1, policy_version 1633018 (0.0009) [2023-12-27 03:15:59,652][105692] Updated weights for policy 0, policy_version 1629683 (0.0012) [2023-12-27 03:15:59,714][105692] Updated weights for policy 0, policy_version 1629693 (0.0010) [2023-12-27 03:15:59,772][105692] Updated weights for policy 0, policy_version 1629703 (0.0010) [2023-12-27 03:16:00,066][105620] Updated weights for policy 1, policy_version 1633028 (0.0009) [2023-12-27 03:16:00,124][105620] Updated weights for policy 1, policy_version 1633038 (0.0009) [2023-12-27 03:16:00,177][105620] Updated weights for policy 1, policy_version 1633048 (0.0007) [2023-12-27 03:16:00,465][105692] Updated weights for policy 0, policy_version 1629713 (0.0010) [2023-12-27 03:16:00,531][105692] Updated weights for policy 0, policy_version 1629723 (0.0006) [2023-12-27 03:16:00,600][105692] Updated weights for policy 0, policy_version 1629733 (0.0009) [2023-12-27 03:16:00,649][105692] Updated weights for policy 0, policy_version 1629743 (0.0006) [2023-12-27 03:16:00,977][105620] Updated weights for policy 1, policy_version 1633058 (0.0006) [2023-12-27 03:16:01,037][105620] Updated weights for policy 1, policy_version 1633068 (0.0009) [2023-12-27 03:16:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19114.7, 300 sec: 19438.6). Total num frames: 835395584. Throughput: 0: 9519.8, 1: 9701.0. Samples: 835371224. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:16:01,063][104569] Avg episode reward: [(0, '8528.071'), (1, '9264.631')] [2023-12-27 03:16:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001629744_417275904.pth... [2023-12-27 03:16:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001628624_416989184.pth [2023-12-27 03:16:01,103][105620] Updated weights for policy 1, policy_version 1633078 (0.0009) [2023-12-27 03:16:01,164][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001633088_418127872.pth... [2023-12-27 03:16:01,166][105620] Updated weights for policy 1, policy_version 1633088 (0.0008) [2023-12-27 03:16:01,168][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001631968_417841152.pth [2023-12-27 03:16:01,199][105692] Updated weights for policy 0, policy_version 1629753 (0.0012) [2023-12-27 03:16:01,260][105692] Updated weights for policy 0, policy_version 1629763 (0.0012) [2023-12-27 03:16:01,326][105692] Updated weights for policy 0, policy_version 1629773 (0.0010) [2023-12-27 03:16:01,926][105620] Updated weights for policy 1, policy_version 1633098 (0.0007) [2023-12-27 03:16:01,990][105620] Updated weights for policy 1, policy_version 1633108 (0.0009) [2023-12-27 03:16:02,047][105620] Updated weights for policy 1, policy_version 1633118 (0.0009) [2023-12-27 03:16:02,132][105692] Updated weights for policy 0, policy_version 1629783 (0.0011) [2023-12-27 03:16:02,185][105692] Updated weights for policy 0, policy_version 1629793 (0.0010) [2023-12-27 03:16:02,235][105692] Updated weights for policy 0, policy_version 1629803 (0.0010) [2023-12-27 03:16:02,833][105620] Updated weights for policy 1, policy_version 1633128 (0.0009) [2023-12-27 03:16:02,877][105692] Updated weights for policy 0, policy_version 1629813 (0.0008) [2023-12-27 03:16:02,891][105620] Updated weights for policy 1, policy_version 1633138 (0.0008) [2023-12-27 03:16:02,936][105692] Updated weights for policy 0, policy_version 1629823 (0.0009) [2023-12-27 03:16:02,944][105620] Updated weights for policy 1, policy_version 1633148 (0.0008) [2023-12-27 03:16:02,998][105692] Updated weights for policy 0, policy_version 1629833 (0.0011) [2023-12-27 03:16:03,720][105620] Updated weights for policy 1, policy_version 1633158 (0.0009) [2023-12-27 03:16:03,743][105692] Updated weights for policy 0, policy_version 1629843 (0.0011) [2023-12-27 03:16:03,784][105620] Updated weights for policy 1, policy_version 1633168 (0.0007) [2023-12-27 03:16:03,794][105692] Updated weights for policy 0, policy_version 1629853 (0.0011) [2023-12-27 03:16:03,846][105620] Updated weights for policy 1, policy_version 1633178 (0.0007) [2023-12-27 03:16:03,863][105692] Updated weights for policy 0, policy_version 1629863 (0.0012) [2023-12-27 03:16:04,592][105620] Updated weights for policy 1, policy_version 1633188 (0.0008) [2023-12-27 03:16:04,653][105620] Updated weights for policy 1, policy_version 1633198 (0.0009) [2023-12-27 03:16:04,672][105692] Updated weights for policy 0, policy_version 1629873 (0.0010) [2023-12-27 03:16:04,713][105620] Updated weights for policy 1, policy_version 1633208 (0.0007) [2023-12-27 03:16:04,733][105692] Updated weights for policy 0, policy_version 1629883 (0.0008) [2023-12-27 03:16:04,802][105692] Updated weights for policy 0, policy_version 1629893 (0.0006) [2023-12-27 03:16:04,872][105692] Updated weights for policy 0, policy_version 1629903 (0.0008) [2023-12-27 03:16:05,527][105620] Updated weights for policy 1, policy_version 1633218 (0.0008) [2023-12-27 03:16:05,546][105692] Updated weights for policy 0, policy_version 1629913 (0.0006) [2023-12-27 03:16:05,585][105620] Updated weights for policy 1, policy_version 1633228 (0.0008) [2023-12-27 03:16:05,603][105692] Updated weights for policy 0, policy_version 1629923 (0.0010) [2023-12-27 03:16:05,651][105620] Updated weights for policy 1, policy_version 1633238 (0.0008) [2023-12-27 03:16:05,669][105692] Updated weights for policy 0, policy_version 1629933 (0.0010) [2023-12-27 03:16:05,713][105620] Updated weights for policy 1, policy_version 1633248 (0.0006) [2023-12-27 03:16:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 835493888. Throughput: 0: 9581.8, 1: 9560.9. Samples: 835482820. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:16:06,062][104569] Avg episode reward: [(0, '8621.587'), (1, '9174.876')] [2023-12-27 03:16:06,327][105692] Updated weights for policy 0, policy_version 1629943 (0.0007) [2023-12-27 03:16:06,380][105620] Updated weights for policy 1, policy_version 1633258 (0.0008) [2023-12-27 03:16:06,394][105692] Updated weights for policy 0, policy_version 1629953 (0.0007) [2023-12-27 03:16:06,447][105620] Updated weights for policy 1, policy_version 1633268 (0.0007) [2023-12-27 03:16:06,461][105692] Updated weights for policy 0, policy_version 1629963 (0.0008) [2023-12-27 03:16:06,509][105620] Updated weights for policy 1, policy_version 1633278 (0.0007) [2023-12-27 03:16:07,121][105692] Updated weights for policy 0, policy_version 1629973 (0.0008) [2023-12-27 03:16:07,181][105692] Updated weights for policy 0, policy_version 1629983 (0.0008) [2023-12-27 03:16:07,226][105620] Updated weights for policy 1, policy_version 1633288 (0.0007) [2023-12-27 03:16:07,236][105692] Updated weights for policy 0, policy_version 1629993 (0.0009) [2023-12-27 03:16:07,272][105620] Updated weights for policy 1, policy_version 1633298 (0.0007) [2023-12-27 03:16:07,322][105620] Updated weights for policy 1, policy_version 1633308 (0.0008) [2023-12-27 03:16:07,934][105692] Updated weights for policy 0, policy_version 1630003 (0.0010) [2023-12-27 03:16:07,994][105692] Updated weights for policy 0, policy_version 1630013 (0.0011) [2023-12-27 03:16:08,060][105692] Updated weights for policy 0, policy_version 1630023 (0.0011) [2023-12-27 03:16:08,085][105620] Updated weights for policy 1, policy_version 1633318 (0.0009) [2023-12-27 03:16:08,136][105620] Updated weights for policy 1, policy_version 1633328 (0.0007) [2023-12-27 03:16:08,193][105620] Updated weights for policy 1, policy_version 1633338 (0.0008) [2023-12-27 03:16:08,818][105692] Updated weights for policy 0, policy_version 1630033 (0.0011) [2023-12-27 03:16:08,889][105692] Updated weights for policy 0, policy_version 1630043 (0.0011) [2023-12-27 03:16:08,955][105692] Updated weights for policy 0, policy_version 1630053 (0.0009) [2023-12-27 03:16:08,985][105620] Updated weights for policy 1, policy_version 1633348 (0.0008) [2023-12-27 03:16:09,022][105692] Updated weights for policy 0, policy_version 1630063 (0.0009) [2023-12-27 03:16:09,051][105620] Updated weights for policy 1, policy_version 1633358 (0.0009) [2023-12-27 03:16:09,110][105620] Updated weights for policy 1, policy_version 1633368 (0.0009) [2023-12-27 03:16:09,795][105692] Updated weights for policy 0, policy_version 1630073 (0.0009) [2023-12-27 03:16:09,869][105692] Updated weights for policy 0, policy_version 1630083 (0.0009) [2023-12-27 03:16:09,940][105692] Updated weights for policy 0, policy_version 1630093 (0.0009) [2023-12-27 03:16:09,946][105620] Updated weights for policy 1, policy_version 1633378 (0.0009) [2023-12-27 03:16:10,013][105620] Updated weights for policy 1, policy_version 1633388 (0.0008) [2023-12-27 03:16:10,072][105620] Updated weights for policy 1, policy_version 1633398 (0.0008) [2023-12-27 03:16:10,132][105620] Updated weights for policy 1, policy_version 1633408 (0.0009) [2023-12-27 03:16:10,625][105692] Updated weights for policy 0, policy_version 1630103 (0.0008) [2023-12-27 03:16:10,682][105692] Updated weights for policy 0, policy_version 1630113 (0.0006) [2023-12-27 03:16:10,736][105692] Updated weights for policy 0, policy_version 1630123 (0.0006) [2023-12-27 03:16:10,949][105620] Updated weights for policy 1, policy_version 1633418 (0.0010) [2023-12-27 03:16:11,010][105620] Updated weights for policy 1, policy_version 1633428 (0.0009) [2023-12-27 03:16:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 18978.1, 300 sec: 19438.7). Total num frames: 835584000. Throughput: 0: 9651.9, 1: 9430.1. Samples: 835595096. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:16:11,062][104569] Avg episode reward: [(0, '8895.426'), (1, '8990.950')] [2023-12-27 03:16:11,075][105620] Updated weights for policy 1, policy_version 1633438 (0.0009) [2023-12-27 03:16:11,425][105692] Updated weights for policy 0, policy_version 1630133 (0.0007) [2023-12-27 03:16:11,480][105692] Updated weights for policy 0, policy_version 1630143 (0.0008) [2023-12-27 03:16:11,540][105692] Updated weights for policy 0, policy_version 1630153 (0.0008) [2023-12-27 03:16:11,889][105620] Updated weights for policy 1, policy_version 1633448 (0.0009) [2023-12-27 03:16:11,956][105620] Updated weights for policy 1, policy_version 1633458 (0.0009) [2023-12-27 03:16:12,010][105620] Updated weights for policy 1, policy_version 1633468 (0.0008) [2023-12-27 03:16:12,363][105692] Updated weights for policy 0, policy_version 1630163 (0.0009) [2023-12-27 03:16:12,422][105692] Updated weights for policy 0, policy_version 1630173 (0.0009) [2023-12-27 03:16:12,480][105692] Updated weights for policy 0, policy_version 1630183 (0.0010) [2023-12-27 03:16:12,765][105620] Updated weights for policy 1, policy_version 1633478 (0.0009) [2023-12-27 03:16:12,825][105620] Updated weights for policy 1, policy_version 1633488 (0.0010) [2023-12-27 03:16:12,880][105620] Updated weights for policy 1, policy_version 1633498 (0.0009) [2023-12-27 03:16:13,170][105692] Updated weights for policy 0, policy_version 1630193 (0.0009) [2023-12-27 03:16:13,227][105692] Updated weights for policy 0, policy_version 1630203 (0.0008) [2023-12-27 03:16:13,284][105692] Updated weights for policy 0, policy_version 1630213 (0.0005) [2023-12-27 03:16:13,349][105692] Updated weights for policy 0, policy_version 1630223 (0.0005) [2023-12-27 03:16:13,694][105620] Updated weights for policy 1, policy_version 1633508 (0.0009) [2023-12-27 03:16:13,748][105620] Updated weights for policy 1, policy_version 1633519 (0.0010) [2023-12-27 03:16:13,801][105620] Updated weights for policy 1, policy_version 1633529 (0.0010) [2023-12-27 03:16:13,888][105692] Updated weights for policy 0, policy_version 1630233 (0.0005) [2023-12-27 03:16:13,941][105692] Updated weights for policy 0, policy_version 1630243 (0.0005) [2023-12-27 03:16:13,988][105692] Updated weights for policy 0, policy_version 1630253 (0.0009) [2023-12-27 03:16:14,523][105620] Updated weights for policy 1, policy_version 1633539 (0.0010) [2023-12-27 03:16:14,583][105620] Updated weights for policy 1, policy_version 1633549 (0.0009) [2023-12-27 03:16:14,630][105620] Updated weights for policy 1, policy_version 1633559 (0.0008) [2023-12-27 03:16:14,698][105692] Updated weights for policy 0, policy_version 1630263 (0.0009) [2023-12-27 03:16:14,754][105692] Updated weights for policy 0, policy_version 1630273 (0.0009) [2023-12-27 03:16:14,823][105692] Updated weights for policy 0, policy_version 1630283 (0.0010) [2023-12-27 03:16:15,436][105692] Updated weights for policy 0, policy_version 1630293 (0.0009) [2023-12-27 03:16:15,480][105620] Updated weights for policy 1, policy_version 1633569 (0.0008) [2023-12-27 03:16:15,486][105692] Updated weights for policy 0, policy_version 1630303 (0.0008) [2023-12-27 03:16:15,530][105620] Updated weights for policy 1, policy_version 1633579 (0.0006) [2023-12-27 03:16:15,540][105692] Updated weights for policy 0, policy_version 1630313 (0.0007) [2023-12-27 03:16:15,588][105620] Updated weights for policy 1, policy_version 1633589 (0.0006) [2023-12-27 03:16:15,655][105620] Updated weights for policy 1, policy_version 1633599 (0.0005) [2023-12-27 03:16:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.7, 300 sec: 19438.6). Total num frames: 835682304. Throughput: 0: 9678.2, 1: 9376.2. Samples: 835651240. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:16:16,062][104569] Avg episode reward: [(0, '8711.899'), (1, '8988.086')] [2023-12-27 03:16:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001630320_417423360.pth... [2023-12-27 03:16:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001633600_418258944.pth... [2023-12-27 03:16:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001632512_417980416.pth [2023-12-27 03:16:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001629168_417128448.pth [2023-12-27 03:16:16,332][105692] Updated weights for policy 0, policy_version 1630323 (0.0008) [2023-12-27 03:16:16,359][105620] Updated weights for policy 1, policy_version 1633609 (0.0007) [2023-12-27 03:16:16,391][105692] Updated weights for policy 0, policy_version 1630333 (0.0007) [2023-12-27 03:16:16,427][105620] Updated weights for policy 1, policy_version 1633619 (0.0008) [2023-12-27 03:16:16,459][105692] Updated weights for policy 0, policy_version 1630343 (0.0009) [2023-12-27 03:16:16,491][105620] Updated weights for policy 1, policy_version 1633629 (0.0008) [2023-12-27 03:16:17,142][105620] Updated weights for policy 1, policy_version 1633639 (0.0007) [2023-12-27 03:16:17,206][105620] Updated weights for policy 1, policy_version 1633649 (0.0006) [2023-12-27 03:16:17,266][105620] Updated weights for policy 1, policy_version 1633659 (0.0009) [2023-12-27 03:16:17,268][105692] Updated weights for policy 0, policy_version 1630353 (0.0007) [2023-12-27 03:16:17,330][105692] Updated weights for policy 0, policy_version 1630363 (0.0009) [2023-12-27 03:16:17,378][105692] Updated weights for policy 0, policy_version 1630373 (0.0009) [2023-12-27 03:16:17,426][105692] Updated weights for policy 0, policy_version 1630383 (0.0009) [2023-12-27 03:16:17,968][105620] Updated weights for policy 1, policy_version 1633669 (0.0008) [2023-12-27 03:16:18,033][105620] Updated weights for policy 1, policy_version 1633679 (0.0010) [2023-12-27 03:16:18,098][105620] Updated weights for policy 1, policy_version 1633689 (0.0010) [2023-12-27 03:16:18,244][105692] Updated weights for policy 0, policy_version 1630393 (0.0008) [2023-12-27 03:16:18,297][105585] KL-divergence is very high: 154.6678 [2023-12-27 03:16:18,314][105692] Updated weights for policy 0, policy_version 1630403 (0.0008) [2023-12-27 03:16:18,345][105585] KL-divergence is very high: 179.0247 [2023-12-27 03:16:18,373][105692] Updated weights for policy 0, policy_version 1630413 (0.0009) [2023-12-27 03:16:18,718][105620] Updated weights for policy 1, policy_version 1633699 (0.0009) [2023-12-27 03:16:18,780][105620] Updated weights for policy 1, policy_version 1633709 (0.0005) [2023-12-27 03:16:18,842][105620] Updated weights for policy 1, policy_version 1633719 (0.0005) [2023-12-27 03:16:19,228][105692] Updated weights for policy 0, policy_version 1630423 (0.0010) [2023-12-27 03:16:19,284][105692] Updated weights for policy 0, policy_version 1630433 (0.0009) [2023-12-27 03:16:19,338][105692] Updated weights for policy 0, policy_version 1630443 (0.0009) [2023-12-27 03:16:19,413][105620] Updated weights for policy 1, policy_version 1633729 (0.0006) [2023-12-27 03:16:19,469][105620] Updated weights for policy 1, policy_version 1633739 (0.0005) [2023-12-27 03:16:19,528][105620] Updated weights for policy 1, policy_version 1633749 (0.0008) [2023-12-27 03:16:19,587][105620] Updated weights for policy 1, policy_version 1633759 (0.0008) [2023-12-27 03:16:20,194][105692] Updated weights for policy 0, policy_version 1630453 (0.0007) [2023-12-27 03:16:20,221][105620] Updated weights for policy 1, policy_version 1633769 (0.0007) [2023-12-27 03:16:20,252][105692] Updated weights for policy 0, policy_version 1630463 (0.0006) [2023-12-27 03:16:20,283][105620] Updated weights for policy 1, policy_version 1633779 (0.0007) [2023-12-27 03:16:20,320][105692] Updated weights for policy 0, policy_version 1630473 (0.0009) [2023-12-27 03:16:20,335][105620] Updated weights for policy 1, policy_version 1633789 (0.0006) [2023-12-27 03:16:21,062][105692] Updated weights for policy 0, policy_version 1630483 (0.0008) [2023-12-27 03:16:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19114.6, 300 sec: 19438.6). Total num frames: 835772416. Throughput: 0: 9635.9, 1: 9417.3. Samples: 835768404. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:16:21,063][104569] Avg episode reward: [(0, '8803.771'), (1, '8895.745')] [2023-12-27 03:16:21,119][105692] Updated weights for policy 0, policy_version 1630493 (0.0006) [2023-12-27 03:16:21,137][105620] Updated weights for policy 1, policy_version 1633799 (0.0008) [2023-12-27 03:16:21,181][105692] Updated weights for policy 0, policy_version 1630503 (0.0009) [2023-12-27 03:16:21,195][105620] Updated weights for policy 1, policy_version 1633809 (0.0007) [2023-12-27 03:16:21,255][105620] Updated weights for policy 1, policy_version 1633819 (0.0007) [2023-12-27 03:16:21,960][105692] Updated weights for policy 0, policy_version 1630513 (0.0009) [2023-12-27 03:16:22,015][105692] Updated weights for policy 0, policy_version 1630523 (0.0009) [2023-12-27 03:16:22,079][105692] Updated weights for policy 0, policy_version 1630533 (0.0009) [2023-12-27 03:16:22,081][105620] Updated weights for policy 1, policy_version 1633829 (0.0008) [2023-12-27 03:16:22,134][105620] Updated weights for policy 1, policy_version 1633839 (0.0006) [2023-12-27 03:16:22,147][105692] Updated weights for policy 0, policy_version 1630543 (0.0009) [2023-12-27 03:16:22,187][105620] Updated weights for policy 1, policy_version 1633849 (0.0008) [2023-12-27 03:16:22,823][105620] Updated weights for policy 1, policy_version 1633859 (0.0005) [2023-12-27 03:16:22,883][105620] Updated weights for policy 1, policy_version 1633869 (0.0008) [2023-12-27 03:16:22,938][105620] Updated weights for policy 1, policy_version 1633879 (0.0008) [2023-12-27 03:16:22,983][105692] Updated weights for policy 0, policy_version 1630553 (0.0009) [2023-12-27 03:16:23,042][105692] Updated weights for policy 0, policy_version 1630563 (0.0006) [2023-12-27 03:16:23,111][105692] Updated weights for policy 0, policy_version 1630573 (0.0007) [2023-12-27 03:16:23,631][105620] Updated weights for policy 1, policy_version 1633889 (0.0006) [2023-12-27 03:16:23,683][105620] Updated weights for policy 1, policy_version 1633899 (0.0011) [2023-12-27 03:16:23,707][105692] Updated weights for policy 0, policy_version 1630583 (0.0006) [2023-12-27 03:16:23,749][105620] Updated weights for policy 1, policy_version 1633909 (0.0011) [2023-12-27 03:16:23,761][105692] Updated weights for policy 0, policy_version 1630593 (0.0005) [2023-12-27 03:16:23,797][105620] Updated weights for policy 1, policy_version 1633919 (0.0010) [2023-12-27 03:16:23,818][105692] Updated weights for policy 0, policy_version 1630603 (0.0008) [2023-12-27 03:16:24,358][105692] Updated weights for policy 0, policy_version 1630613 (0.0008) [2023-12-27 03:16:24,418][105692] Updated weights for policy 0, policy_version 1630623 (0.0008) [2023-12-27 03:16:24,480][105692] Updated weights for policy 0, policy_version 1630633 (0.0005) [2023-12-27 03:16:24,545][105620] Updated weights for policy 1, policy_version 1633929 (0.0010) [2023-12-27 03:16:24,611][105620] Updated weights for policy 1, policy_version 1633939 (0.0011) [2023-12-27 03:16:24,656][105620] Updated weights for policy 1, policy_version 1633949 (0.0010) [2023-12-27 03:16:25,149][105692] Updated weights for policy 0, policy_version 1630643 (0.0007) [2023-12-27 03:16:25,209][105692] Updated weights for policy 0, policy_version 1630653 (0.0011) [2023-12-27 03:16:25,272][105692] Updated weights for policy 0, policy_version 1630663 (0.0011) [2023-12-27 03:16:25,369][105620] Updated weights for policy 1, policy_version 1633959 (0.0011) [2023-12-27 03:16:25,428][105620] Updated weights for policy 1, policy_version 1633969 (0.0011) [2023-12-27 03:16:25,487][105620] Updated weights for policy 1, policy_version 1633979 (0.0010) [2023-12-27 03:16:25,944][105692] Updated weights for policy 0, policy_version 1630673 (0.0011) [2023-12-27 03:16:26,009][105692] Updated weights for policy 0, policy_version 1630683 (0.0011) [2023-12-27 03:16:26,062][104569] Fps is (10 sec: 18841.2, 60 sec: 18978.1, 300 sec: 19438.6). Total num frames: 835870720. Throughput: 0: 9722.4, 1: 9418.9. Samples: 835884248. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:16:26,063][104569] Avg episode reward: [(0, '9077.709'), (1, '9082.587')] [2023-12-27 03:16:26,067][105692] Updated weights for policy 0, policy_version 1630693 (0.0010) [2023-12-27 03:16:26,113][105620] Updated weights for policy 1, policy_version 1633989 (0.0008) [2023-12-27 03:16:26,119][105692] Updated weights for policy 0, policy_version 1630703 (0.0011) [2023-12-27 03:16:26,177][105620] Updated weights for policy 1, policy_version 1633999 (0.0005) [2023-12-27 03:16:26,228][105620] Updated weights for policy 1, policy_version 1634009 (0.0005) [2023-12-27 03:16:26,756][105692] Updated weights for policy 0, policy_version 1630713 (0.0011) [2023-12-27 03:16:26,777][105620] Updated weights for policy 1, policy_version 1634019 (0.0005) [2023-12-27 03:16:26,815][105692] Updated weights for policy 0, policy_version 1630723 (0.0011) [2023-12-27 03:16:26,842][105620] Updated weights for policy 1, policy_version 1634029 (0.0009) [2023-12-27 03:16:26,871][105692] Updated weights for policy 0, policy_version 1630733 (0.0009) [2023-12-27 03:16:26,897][105620] Updated weights for policy 1, policy_version 1634039 (0.0010) [2023-12-27 03:16:27,445][105692] Updated weights for policy 0, policy_version 1630743 (0.0005) [2023-12-27 03:16:27,500][105692] Updated weights for policy 0, policy_version 1630753 (0.0005) [2023-12-27 03:16:27,560][105692] Updated weights for policy 0, policy_version 1630763 (0.0010) [2023-12-27 03:16:27,575][105620] Updated weights for policy 1, policy_version 1634049 (0.0010) [2023-12-27 03:16:27,629][105620] Updated weights for policy 1, policy_version 1634059 (0.0005) [2023-12-27 03:16:27,683][105620] Updated weights for policy 1, policy_version 1634069 (0.0005) [2023-12-27 03:16:27,734][105620] Updated weights for policy 1, policy_version 1634079 (0.0005) [2023-12-27 03:16:28,254][105692] Updated weights for policy 0, policy_version 1630773 (0.0011) [2023-12-27 03:16:28,290][105620] Updated weights for policy 1, policy_version 1634089 (0.0005) [2023-12-27 03:16:28,306][105692] Updated weights for policy 0, policy_version 1630783 (0.0011) [2023-12-27 03:16:28,346][105620] Updated weights for policy 1, policy_version 1634099 (0.0006) [2023-12-27 03:16:28,366][105692] Updated weights for policy 0, policy_version 1630793 (0.0010) [2023-12-27 03:16:28,407][105620] Updated weights for policy 1, policy_version 1634109 (0.0008) [2023-12-27 03:16:29,074][105692] Updated weights for policy 0, policy_version 1630803 (0.0010) [2023-12-27 03:16:29,121][105620] Updated weights for policy 1, policy_version 1634119 (0.0010) [2023-12-27 03:16:29,134][105692] Updated weights for policy 0, policy_version 1630813 (0.0010) [2023-12-27 03:16:29,168][105620] Updated weights for policy 1, policy_version 1634129 (0.0009) [2023-12-27 03:16:29,181][105692] Updated weights for policy 0, policy_version 1630823 (0.0010) [2023-12-27 03:16:29,233][105620] Updated weights for policy 1, policy_version 1634139 (0.0007) [2023-12-27 03:16:29,856][105692] Updated weights for policy 0, policy_version 1630833 (0.0010) [2023-12-27 03:16:29,920][105692] Updated weights for policy 0, policy_version 1630843 (0.0006) [2023-12-27 03:16:29,978][105692] Updated weights for policy 0, policy_version 1630853 (0.0008) [2023-12-27 03:16:30,032][105620] Updated weights for policy 1, policy_version 1634149 (0.0007) [2023-12-27 03:16:30,034][105692] Updated weights for policy 0, policy_version 1630863 (0.0007) [2023-12-27 03:16:30,086][105620] Updated weights for policy 1, policy_version 1634159 (0.0009) [2023-12-27 03:16:30,142][105620] Updated weights for policy 1, policy_version 1634169 (0.0007) [2023-12-27 03:16:30,648][105692] Updated weights for policy 0, policy_version 1630873 (0.0008) [2023-12-27 03:16:30,710][105692] Updated weights for policy 0, policy_version 1630883 (0.0008) [2023-12-27 03:16:30,763][105692] Updated weights for policy 0, policy_version 1630893 (0.0005) [2023-12-27 03:16:30,764][105620] Updated weights for policy 1, policy_version 1634179 (0.0007) [2023-12-27 03:16:30,831][105620] Updated weights for policy 1, policy_version 1634189 (0.0010) [2023-12-27 03:16:30,890][105620] Updated weights for policy 1, policy_version 1634199 (0.0010) [2023-12-27 03:16:31,062][104569] Fps is (10 sec: 21299.4, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 835985408. Throughput: 0: 9815.7, 1: 9574.2. Samples: 835949328. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:16:31,063][104569] Avg episode reward: [(0, '8985.275'), (1, '9081.778')] [2023-12-27 03:16:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001634208_418414592.pth... [2023-12-27 03:16:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001630896_417570816.pth... [2023-12-27 03:16:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001633088_418127872.pth [2023-12-27 03:16:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001629744_417275904.pth [2023-12-27 03:16:31,400][105692] Updated weights for policy 0, policy_version 1630903 (0.0007) [2023-12-27 03:16:31,453][105692] Updated weights for policy 0, policy_version 1630913 (0.0005) [2023-12-27 03:16:31,525][105692] Updated weights for policy 0, policy_version 1630923 (0.0005) [2023-12-27 03:16:31,615][105620] Updated weights for policy 1, policy_version 1634210 (0.0010) [2023-12-27 03:16:31,678][105620] Updated weights for policy 1, policy_version 1634220 (0.0009) [2023-12-27 03:16:31,748][105620] Updated weights for policy 1, policy_version 1634230 (0.0008) [2023-12-27 03:16:31,795][105620] Updated weights for policy 1, policy_version 1634240 (0.0005) [2023-12-27 03:16:32,156][105692] Updated weights for policy 0, policy_version 1630933 (0.0008) [2023-12-27 03:16:32,219][105692] Updated weights for policy 0, policy_version 1630943 (0.0011) [2023-12-27 03:16:32,285][105692] Updated weights for policy 0, policy_version 1630953 (0.0009) [2023-12-27 03:16:32,416][105620] Updated weights for policy 1, policy_version 1634250 (0.0008) [2023-12-27 03:16:32,470][105620] Updated weights for policy 1, policy_version 1634260 (0.0006) [2023-12-27 03:16:32,524][105620] Updated weights for policy 1, policy_version 1634270 (0.0006) [2023-12-27 03:16:32,882][105692] Updated weights for policy 0, policy_version 1630963 (0.0007) [2023-12-27 03:16:32,938][105692] Updated weights for policy 0, policy_version 1630973 (0.0008) [2023-12-27 03:16:32,992][105692] Updated weights for policy 0, policy_version 1630983 (0.0010) [2023-12-27 03:16:33,145][105620] Updated weights for policy 1, policy_version 1634280 (0.0007) [2023-12-27 03:16:33,208][105620] Updated weights for policy 1, policy_version 1634290 (0.0006) [2023-12-27 03:16:33,266][105620] Updated weights for policy 1, policy_version 1634300 (0.0005) [2023-12-27 03:16:33,719][105692] Updated weights for policy 0, policy_version 1630993 (0.0010) [2023-12-27 03:16:33,767][105620] Updated weights for policy 1, policy_version 1634310 (0.0006) [2023-12-27 03:16:33,787][105692] Updated weights for policy 0, policy_version 1631003 (0.0008) [2023-12-27 03:16:33,822][105620] Updated weights for policy 1, policy_version 1634320 (0.0006) [2023-12-27 03:16:33,844][105692] Updated weights for policy 0, policy_version 1631013 (0.0009) [2023-12-27 03:16:33,871][105620] Updated weights for policy 1, policy_version 1634330 (0.0005) [2023-12-27 03:16:33,896][105692] Updated weights for policy 0, policy_version 1631023 (0.0008) [2023-12-27 03:16:34,445][105620] Updated weights for policy 1, policy_version 1634340 (0.0006) [2023-12-27 03:16:34,513][105620] Updated weights for policy 1, policy_version 1634350 (0.0006) [2023-12-27 03:16:34,570][105692] Updated weights for policy 0, policy_version 1631033 (0.0006) [2023-12-27 03:16:34,575][105620] Updated weights for policy 1, policy_version 1634360 (0.0006) [2023-12-27 03:16:34,634][105692] Updated weights for policy 0, policy_version 1631043 (0.0007) [2023-12-27 03:16:34,702][105692] Updated weights for policy 0, policy_version 1631053 (0.0009) [2023-12-27 03:16:35,150][105620] Updated weights for policy 1, policy_version 1634370 (0.0008) [2023-12-27 03:16:35,200][105620] Updated weights for policy 1, policy_version 1634380 (0.0006) [2023-12-27 03:16:35,254][105620] Updated weights for policy 1, policy_version 1634390 (0.0005) [2023-12-27 03:16:35,292][105692] Updated weights for policy 0, policy_version 1631063 (0.0006) [2023-12-27 03:16:35,318][105620] Updated weights for policy 1, policy_version 1634400 (0.0005) [2023-12-27 03:16:35,339][105692] Updated weights for policy 0, policy_version 1631073 (0.0010) [2023-12-27 03:16:35,383][105692] Updated weights for policy 0, policy_version 1631083 (0.0010) [2023-12-27 03:16:35,865][105620] Updated weights for policy 1, policy_version 1634410 (0.0005) [2023-12-27 03:16:35,918][105620] Updated weights for policy 1, policy_version 1634420 (0.0006) [2023-12-27 03:16:35,977][105620] Updated weights for policy 1, policy_version 1634430 (0.0005) [2023-12-27 03:16:36,062][104569] Fps is (10 sec: 22118.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 836091904. Throughput: 0: 9954.3, 1: 9751.9. Samples: 836076828. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:16:36,063][104569] Avg episode reward: [(0, '8711.237'), (1, '9081.800')] [2023-12-27 03:16:36,117][105692] Updated weights for policy 0, policy_version 1631093 (0.0010) [2023-12-27 03:16:36,176][105692] Updated weights for policy 0, policy_version 1631103 (0.0008) [2023-12-27 03:16:36,234][105692] Updated weights for policy 0, policy_version 1631113 (0.0010) [2023-12-27 03:16:36,569][105620] Updated weights for policy 1, policy_version 1634440 (0.0005) [2023-12-27 03:16:36,632][105620] Updated weights for policy 1, policy_version 1634450 (0.0007) [2023-12-27 03:16:36,700][105620] Updated weights for policy 1, policy_version 1634460 (0.0009) [2023-12-27 03:16:36,984][105692] Updated weights for policy 0, policy_version 1631123 (0.0008) [2023-12-27 03:16:37,043][105692] Updated weights for policy 0, policy_version 1631133 (0.0008) [2023-12-27 03:16:37,098][105692] Updated weights for policy 0, policy_version 1631143 (0.0009) [2023-12-27 03:16:37,453][105620] Updated weights for policy 1, policy_version 1634470 (0.0009) [2023-12-27 03:16:37,504][105620] Updated weights for policy 1, policy_version 1634480 (0.0008) [2023-12-27 03:16:37,554][105620] Updated weights for policy 1, policy_version 1634490 (0.0009) [2023-12-27 03:16:37,788][105692] Updated weights for policy 0, policy_version 1631153 (0.0009) [2023-12-27 03:16:37,846][105692] Updated weights for policy 0, policy_version 1631163 (0.0009) [2023-12-27 03:16:37,904][105692] Updated weights for policy 0, policy_version 1631173 (0.0009) [2023-12-27 03:16:37,965][105692] Updated weights for policy 0, policy_version 1631183 (0.0010) [2023-12-27 03:16:38,245][105620] Updated weights for policy 1, policy_version 1634500 (0.0009) [2023-12-27 03:16:38,303][105620] Updated weights for policy 1, policy_version 1634510 (0.0009) [2023-12-27 03:16:38,356][105620] Updated weights for policy 1, policy_version 1634520 (0.0009) [2023-12-27 03:16:38,669][105692] Updated weights for policy 0, policy_version 1631193 (0.0007) [2023-12-27 03:16:38,728][105692] Updated weights for policy 0, policy_version 1631203 (0.0009) [2023-12-27 03:16:38,783][105692] Updated weights for policy 0, policy_version 1631213 (0.0008) [2023-12-27 03:16:39,061][105620] Updated weights for policy 1, policy_version 1634530 (0.0007) [2023-12-27 03:16:39,107][105620] Updated weights for policy 1, policy_version 1634540 (0.0006) [2023-12-27 03:16:39,160][105620] Updated weights for policy 1, policy_version 1634550 (0.0005) [2023-12-27 03:16:39,223][105620] Updated weights for policy 1, policy_version 1634560 (0.0007) [2023-12-27 03:16:39,578][105692] Updated weights for policy 0, policy_version 1631223 (0.0010) [2023-12-27 03:16:39,631][105692] Updated weights for policy 0, policy_version 1631233 (0.0011) [2023-12-27 03:16:39,693][105692] Updated weights for policy 0, policy_version 1631243 (0.0007) [2023-12-27 03:16:40,005][105620] Updated weights for policy 1, policy_version 1634570 (0.0006) [2023-12-27 03:16:40,072][105620] Updated weights for policy 1, policy_version 1634580 (0.0011) [2023-12-27 03:16:40,139][105620] Updated weights for policy 1, policy_version 1634590 (0.0011) [2023-12-27 03:16:40,439][105692] Updated weights for policy 0, policy_version 1631253 (0.0009) [2023-12-27 03:16:40,498][105692] Updated weights for policy 0, policy_version 1631263 (0.0005) [2023-12-27 03:16:40,556][105692] Updated weights for policy 0, policy_version 1631273 (0.0006) [2023-12-27 03:16:40,910][105620] Updated weights for policy 1, policy_version 1634600 (0.0008) [2023-12-27 03:16:40,961][105620] Updated weights for policy 1, policy_version 1634610 (0.0008) [2023-12-27 03:16:41,020][105620] Updated weights for policy 1, policy_version 1634620 (0.0009) [2023-12-27 03:16:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 836190208. Throughput: 0: 9935.6, 1: 9828.6. Samples: 836196388. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:16:41,062][104569] Avg episode reward: [(0, '8255.574'), (1, '9174.280')] [2023-12-27 03:16:41,280][105692] Updated weights for policy 0, policy_version 1631283 (0.0008) [2023-12-27 03:16:41,340][105692] Updated weights for policy 0, policy_version 1631293 (0.0011) [2023-12-27 03:16:41,411][105692] Updated weights for policy 0, policy_version 1631303 (0.0011) [2023-12-27 03:16:41,830][105620] Updated weights for policy 1, policy_version 1634630 (0.0007) [2023-12-27 03:16:41,894][105620] Updated weights for policy 1, policy_version 1634640 (0.0010) [2023-12-27 03:16:41,953][105620] Updated weights for policy 1, policy_version 1634650 (0.0009) [2023-12-27 03:16:42,173][105692] Updated weights for policy 0, policy_version 1631313 (0.0009) [2023-12-27 03:16:42,239][105692] Updated weights for policy 0, policy_version 1631323 (0.0005) [2023-12-27 03:16:42,300][105692] Updated weights for policy 0, policy_version 1631333 (0.0006) [2023-12-27 03:16:42,370][105692] Updated weights for policy 0, policy_version 1631343 (0.0007) [2023-12-27 03:16:42,748][105620] Updated weights for policy 1, policy_version 1634660 (0.0010) [2023-12-27 03:16:42,807][105620] Updated weights for policy 1, policy_version 1634670 (0.0007) [2023-12-27 03:16:42,870][105620] Updated weights for policy 1, policy_version 1634680 (0.0006) [2023-12-27 03:16:42,934][105692] Updated weights for policy 0, policy_version 1631353 (0.0009) [2023-12-27 03:16:42,997][105692] Updated weights for policy 0, policy_version 1631363 (0.0007) [2023-12-27 03:16:43,055][105692] Updated weights for policy 0, policy_version 1631373 (0.0007) [2023-12-27 03:16:43,622][105620] Updated weights for policy 1, policy_version 1634690 (0.0006) [2023-12-27 03:16:43,671][105620] Updated weights for policy 1, policy_version 1634700 (0.0007) [2023-12-27 03:16:43,679][105692] Updated weights for policy 0, policy_version 1631383 (0.0009) [2023-12-27 03:16:43,734][105692] Updated weights for policy 0, policy_version 1631393 (0.0010) [2023-12-27 03:16:43,736][105620] Updated weights for policy 1, policy_version 1634710 (0.0006) [2023-12-27 03:16:43,782][105620] Updated weights for policy 1, policy_version 1634720 (0.0007) [2023-12-27 03:16:43,792][105692] Updated weights for policy 0, policy_version 1631403 (0.0010) [2023-12-27 03:16:44,498][105620] Updated weights for policy 1, policy_version 1634730 (0.0008) [2023-12-27 03:16:44,512][105692] Updated weights for policy 0, policy_version 1631413 (0.0009) [2023-12-27 03:16:44,548][105620] Updated weights for policy 1, policy_version 1634740 (0.0008) [2023-12-27 03:16:44,574][105692] Updated weights for policy 0, policy_version 1631423 (0.0005) [2023-12-27 03:16:44,599][105620] Updated weights for policy 1, policy_version 1634750 (0.0007) [2023-12-27 03:16:44,642][105692] Updated weights for policy 0, policy_version 1631433 (0.0006) [2023-12-27 03:16:45,324][105692] Updated weights for policy 0, policy_version 1631443 (0.0006) [2023-12-27 03:16:45,378][105692] Updated weights for policy 0, policy_version 1631453 (0.0008) [2023-12-27 03:16:45,389][105620] Updated weights for policy 1, policy_version 1634760 (0.0010) [2023-12-27 03:16:45,430][105692] Updated weights for policy 0, policy_version 1631463 (0.0006) [2023-12-27 03:16:45,450][105620] Updated weights for policy 1, policy_version 1634770 (0.0010) [2023-12-27 03:16:45,512][105620] Updated weights for policy 1, policy_version 1634780 (0.0010) [2023-12-27 03:16:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 836280320. Throughput: 0: 9858.6, 1: 9748.2. Samples: 836253528. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:16:46,063][104569] Avg episode reward: [(0, '8530.240'), (1, '9172.949')] [2023-12-27 03:16:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001631472_417718272.pth... [2023-12-27 03:16:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001634784_418562048.pth... [2023-12-27 03:16:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001633600_418258944.pth [2023-12-27 03:16:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001630320_417423360.pth [2023-12-27 03:16:46,173][105620] Updated weights for policy 1, policy_version 1634790 (0.0010) [2023-12-27 03:16:46,197][105692] Updated weights for policy 0, policy_version 1631473 (0.0006) [2023-12-27 03:16:46,228][105620] Updated weights for policy 1, policy_version 1634800 (0.0010) [2023-12-27 03:16:46,255][105692] Updated weights for policy 0, policy_version 1631483 (0.0006) [2023-12-27 03:16:46,275][105620] Updated weights for policy 1, policy_version 1634810 (0.0007) [2023-12-27 03:16:46,306][105692] Updated weights for policy 0, policy_version 1631493 (0.0009) [2023-12-27 03:16:46,361][105692] Updated weights for policy 0, policy_version 1631503 (0.0011) [2023-12-27 03:16:46,909][105620] Updated weights for policy 1, policy_version 1634820 (0.0006) [2023-12-27 03:16:46,965][105620] Updated weights for policy 1, policy_version 1634830 (0.0009) [2023-12-27 03:16:47,030][105620] Updated weights for policy 1, policy_version 1634840 (0.0009) [2023-12-27 03:16:47,137][105692] Updated weights for policy 0, policy_version 1631513 (0.0007) [2023-12-27 03:16:47,204][105692] Updated weights for policy 0, policy_version 1631523 (0.0005) [2023-12-27 03:16:47,267][105692] Updated weights for policy 0, policy_version 1631533 (0.0005) [2023-12-27 03:16:47,672][105620] Updated weights for policy 1, policy_version 1634850 (0.0009) [2023-12-27 03:16:47,727][105620] Updated weights for policy 1, policy_version 1634860 (0.0008) [2023-12-27 03:16:47,779][105620] Updated weights for policy 1, policy_version 1634870 (0.0009) [2023-12-27 03:16:47,837][105620] Updated weights for policy 1, policy_version 1634880 (0.0010) [2023-12-27 03:16:47,893][105692] Updated weights for policy 0, policy_version 1631543 (0.0009) [2023-12-27 03:16:47,956][105692] Updated weights for policy 0, policy_version 1631553 (0.0010) [2023-12-27 03:16:48,007][105692] Updated weights for policy 0, policy_version 1631563 (0.0010) [2023-12-27 03:16:48,557][105620] Updated weights for policy 1, policy_version 1634890 (0.0011) [2023-12-27 03:16:48,591][105692] Updated weights for policy 0, policy_version 1631573 (0.0008) [2023-12-27 03:16:48,606][105620] Updated weights for policy 1, policy_version 1634900 (0.0010) [2023-12-27 03:16:48,643][105692] Updated weights for policy 0, policy_version 1631583 (0.0005) [2023-12-27 03:16:48,662][105620] Updated weights for policy 1, policy_version 1634910 (0.0011) [2023-12-27 03:16:48,690][105692] Updated weights for policy 0, policy_version 1631593 (0.0005) [2023-12-27 03:16:49,233][105692] Updated weights for policy 0, policy_version 1631603 (0.0007) [2023-12-27 03:16:49,295][105692] Updated weights for policy 0, policy_version 1631613 (0.0010) [2023-12-27 03:16:49,363][105692] Updated weights for policy 0, policy_version 1631623 (0.0011) [2023-12-27 03:16:49,370][105620] Updated weights for policy 1, policy_version 1634920 (0.0009) [2023-12-27 03:16:49,428][105620] Updated weights for policy 1, policy_version 1634930 (0.0006) [2023-12-27 03:16:49,481][105620] Updated weights for policy 1, policy_version 1634940 (0.0006) [2023-12-27 03:16:49,971][105692] Updated weights for policy 0, policy_version 1631633 (0.0008) [2023-12-27 03:16:50,030][105692] Updated weights for policy 0, policy_version 1631643 (0.0008) [2023-12-27 03:16:50,087][105692] Updated weights for policy 0, policy_version 1631653 (0.0007) [2023-12-27 03:16:50,118][105620] Updated weights for policy 1, policy_version 1634950 (0.0009) [2023-12-27 03:16:50,149][105692] Updated weights for policy 0, policy_version 1631663 (0.0006) [2023-12-27 03:16:50,171][105620] Updated weights for policy 1, policy_version 1634960 (0.0010) [2023-12-27 03:16:50,220][105620] Updated weights for policy 1, policy_version 1634970 (0.0010) [2023-12-27 03:16:50,784][105692] Updated weights for policy 0, policy_version 1631673 (0.0005) [2023-12-27 03:16:50,844][105692] Updated weights for policy 0, policy_version 1631683 (0.0006) [2023-12-27 03:16:50,910][105692] Updated weights for policy 0, policy_version 1631693 (0.0006) [2023-12-27 03:16:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 836386816. Throughput: 0: 9933.3, 1: 9913.0. Samples: 836375904. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:16:51,063][104569] Avg episode reward: [(0, '8716.652'), (1, '9081.608')] [2023-12-27 03:16:51,091][105620] Updated weights for policy 1, policy_version 1634981 (0.0010) [2023-12-27 03:16:51,157][105620] Updated weights for policy 1, policy_version 1634991 (0.0009) [2023-12-27 03:16:51,220][105620] Updated weights for policy 1, policy_version 1635001 (0.0009) [2023-12-27 03:16:51,555][105692] Updated weights for policy 0, policy_version 1631703 (0.0007) [2023-12-27 03:16:51,623][105692] Updated weights for policy 0, policy_version 1631713 (0.0011) [2023-12-27 03:16:51,684][105692] Updated weights for policy 0, policy_version 1631723 (0.0011) [2023-12-27 03:16:51,987][105620] Updated weights for policy 1, policy_version 1635011 (0.0009) [2023-12-27 03:16:52,046][105620] Updated weights for policy 1, policy_version 1635021 (0.0008) [2023-12-27 03:16:52,106][105620] Updated weights for policy 1, policy_version 1635031 (0.0006) [2023-12-27 03:16:52,378][105692] Updated weights for policy 0, policy_version 1631733 (0.0009) [2023-12-27 03:16:52,440][105692] Updated weights for policy 0, policy_version 1631743 (0.0007) [2023-12-27 03:16:52,499][105692] Updated weights for policy 0, policy_version 1631753 (0.0009) [2023-12-27 03:16:52,791][105620] Updated weights for policy 1, policy_version 1635041 (0.0006) [2023-12-27 03:16:52,860][105620] Updated weights for policy 1, policy_version 1635051 (0.0010) [2023-12-27 03:16:52,927][105620] Updated weights for policy 1, policy_version 1635061 (0.0010) [2023-12-27 03:16:52,976][105620] Updated weights for policy 1, policy_version 1635071 (0.0009) [2023-12-27 03:16:53,072][105692] Updated weights for policy 0, policy_version 1631763 (0.0009) [2023-12-27 03:16:53,132][105692] Updated weights for policy 0, policy_version 1631773 (0.0006) [2023-12-27 03:16:53,181][105692] Updated weights for policy 0, policy_version 1631783 (0.0008) [2023-12-27 03:16:53,687][105620] Updated weights for policy 1, policy_version 1635081 (0.0009) [2023-12-27 03:16:53,750][105620] Updated weights for policy 1, policy_version 1635091 (0.0008) [2023-12-27 03:16:53,782][105692] Updated weights for policy 0, policy_version 1631793 (0.0010) [2023-12-27 03:16:53,803][105620] Updated weights for policy 1, policy_version 1635101 (0.0008) [2023-12-27 03:16:53,842][105692] Updated weights for policy 0, policy_version 1631803 (0.0006) [2023-12-27 03:16:53,904][105692] Updated weights for policy 0, policy_version 1631813 (0.0005) [2023-12-27 03:16:53,973][105692] Updated weights for policy 0, policy_version 1631823 (0.0005) [2023-12-27 03:16:54,460][105620] Updated weights for policy 1, policy_version 1635111 (0.0011) [2023-12-27 03:16:54,526][105620] Updated weights for policy 1, policy_version 1635121 (0.0011) [2023-12-27 03:16:54,536][105692] Updated weights for policy 0, policy_version 1631833 (0.0007) [2023-12-27 03:16:54,594][105620] Updated weights for policy 1, policy_version 1635131 (0.0006) [2023-12-27 03:16:54,595][105692] Updated weights for policy 0, policy_version 1631843 (0.0006) [2023-12-27 03:16:54,655][105692] Updated weights for policy 0, policy_version 1631853 (0.0007) [2023-12-27 03:16:55,184][105620] Updated weights for policy 1, policy_version 1635141 (0.0007) [2023-12-27 03:16:55,248][105620] Updated weights for policy 1, policy_version 1635151 (0.0009) [2023-12-27 03:16:55,301][105620] Updated weights for policy 1, policy_version 1635161 (0.0010) [2023-12-27 03:16:55,325][105586] KL-divergence is very high: 140.7672 [2023-12-27 03:16:55,410][105692] Updated weights for policy 0, policy_version 1631864 (0.0009) [2023-12-27 03:16:55,457][105692] Updated weights for policy 0, policy_version 1631874 (0.0009) [2023-12-27 03:16:55,514][105692] Updated weights for policy 0, policy_version 1631884 (0.0009) [2023-12-27 03:16:56,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 836485120. Throughput: 0: 10053.4, 1: 9991.4. Samples: 836497112. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-12-27 03:16:56,062][104569] Avg episode reward: [(0, '8623.376'), (1, '9084.202')] [2023-12-27 03:16:56,070][105620] Updated weights for policy 1, policy_version 1635171 (0.0011) [2023-12-27 03:16:56,134][105620] Updated weights for policy 1, policy_version 1635181 (0.0010) [2023-12-27 03:16:56,197][105620] Updated weights for policy 1, policy_version 1635191 (0.0010) [2023-12-27 03:16:56,213][105692] Updated weights for policy 0, policy_version 1631894 (0.0007) [2023-12-27 03:16:56,260][105692] Updated weights for policy 0, policy_version 1631904 (0.0005) [2023-12-27 03:16:56,304][105692] Updated weights for policy 0, policy_version 1631914 (0.0005) [2023-12-27 03:16:56,857][105620] Updated weights for policy 1, policy_version 1635201 (0.0010) [2023-12-27 03:16:56,859][105692] Updated weights for policy 0, policy_version 1631924 (0.0005) [2023-12-27 03:16:56,912][105620] Updated weights for policy 1, policy_version 1635211 (0.0010) [2023-12-27 03:16:56,918][105692] Updated weights for policy 0, policy_version 1631934 (0.0005) [2023-12-27 03:16:56,968][105620] Updated weights for policy 1, policy_version 1635221 (0.0010) [2023-12-27 03:16:56,970][105692] Updated weights for policy 0, policy_version 1631944 (0.0009) [2023-12-27 03:16:57,026][105620] Updated weights for policy 1, policy_version 1635231 (0.0010) [2023-12-27 03:16:57,601][105692] Updated weights for policy 0, policy_version 1631954 (0.0006) [2023-12-27 03:16:57,657][105692] Updated weights for policy 0, policy_version 1631964 (0.0005) [2023-12-27 03:16:57,723][105692] Updated weights for policy 0, policy_version 1631974 (0.0005) [2023-12-27 03:16:57,745][105620] Updated weights for policy 1, policy_version 1635241 (0.0006) [2023-12-27 03:16:57,781][105692] Updated weights for policy 0, policy_version 1631984 (0.0006) [2023-12-27 03:16:57,793][105620] Updated weights for policy 1, policy_version 1635251 (0.0010) [2023-12-27 03:16:57,850][105620] Updated weights for policy 1, policy_version 1635261 (0.0010) [2023-12-27 03:16:58,359][105692] Updated weights for policy 0, policy_version 1631994 (0.0010) [2023-12-27 03:16:58,426][105692] Updated weights for policy 0, policy_version 1632004 (0.0011) [2023-12-27 03:16:58,494][105692] Updated weights for policy 0, policy_version 1632014 (0.0011) [2023-12-27 03:16:58,637][105620] Updated weights for policy 1, policy_version 1635271 (0.0008) [2023-12-27 03:16:58,699][105620] Updated weights for policy 1, policy_version 1635281 (0.0007) [2023-12-27 03:16:58,764][105620] Updated weights for policy 1, policy_version 1635291 (0.0007) [2023-12-27 03:16:59,289][105692] Updated weights for policy 0, policy_version 1632024 (0.0009) [2023-12-27 03:16:59,358][105692] Updated weights for policy 0, policy_version 1632034 (0.0008) [2023-12-27 03:16:59,408][105692] Updated weights for policy 0, policy_version 1632044 (0.0007) [2023-12-27 03:16:59,604][105620] Updated weights for policy 1, policy_version 1635301 (0.0007) [2023-12-27 03:16:59,656][105620] Updated weights for policy 1, policy_version 1635311 (0.0005) [2023-12-27 03:16:59,707][105620] Updated weights for policy 1, policy_version 1635321 (0.0005) [2023-12-27 03:17:00,180][105692] Updated weights for policy 0, policy_version 1632054 (0.0009) [2023-12-27 03:17:00,235][105692] Updated weights for policy 0, policy_version 1632065 (0.0010) [2023-12-27 03:17:00,290][105692] Updated weights for policy 0, policy_version 1632077 (0.0010) [2023-12-27 03:17:00,362][105620] Updated weights for policy 1, policy_version 1635331 (0.0005) [2023-12-27 03:17:00,428][105620] Updated weights for policy 1, policy_version 1635341 (0.0005) [2023-12-27 03:17:00,486][105620] Updated weights for policy 1, policy_version 1635351 (0.0005) [2023-12-27 03:17:00,993][105620] Updated weights for policy 1, policy_version 1635361 (0.0006) [2023-12-27 03:17:01,052][105620] Updated weights for policy 1, policy_version 1635371 (0.0008) [2023-12-27 03:17:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19466.4). Total num frames: 836583424. Throughput: 0: 10133.6, 1: 10035.7. Samples: 836558860. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:17:01,062][104569] Avg episode reward: [(0, '8524.233'), (1, '8899.378')] [2023-12-27 03:17:01,078][105692] Updated weights for policy 0, policy_version 1632087 (0.0010) [2023-12-27 03:17:01,115][105620] Updated weights for policy 1, policy_version 1635381 (0.0007) [2023-12-27 03:17:01,141][105692] Updated weights for policy 0, policy_version 1632097 (0.0010) [2023-12-27 03:17:01,169][105620] Updated weights for policy 1, policy_version 1635391 (0.0007) [2023-12-27 03:17:01,173][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001635392_418717696.pth... [2023-12-27 03:17:01,176][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001634208_418414592.pth [2023-12-27 03:17:01,197][105692] Updated weights for policy 0, policy_version 1632107 (0.0009) [2023-12-27 03:17:01,226][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001632112_417882112.pth... [2023-12-27 03:17:01,231][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001630896_417570816.pth [2023-12-27 03:17:01,861][105692] Updated weights for policy 0, policy_version 1632117 (0.0009) [2023-12-27 03:17:01,906][105692] Updated weights for policy 0, policy_version 1632127 (0.0008) [2023-12-27 03:17:01,938][105620] Updated weights for policy 1, policy_version 1635401 (0.0009) [2023-12-27 03:17:01,956][105692] Updated weights for policy 0, policy_version 1632137 (0.0008) [2023-12-27 03:17:01,994][105620] Updated weights for policy 1, policy_version 1635411 (0.0008) [2023-12-27 03:17:02,058][105620] Updated weights for policy 1, policy_version 1635421 (0.0009) [2023-12-27 03:17:02,661][105692] Updated weights for policy 0, policy_version 1632147 (0.0005) [2023-12-27 03:17:02,729][105692] Updated weights for policy 0, policy_version 1632157 (0.0005) [2023-12-27 03:17:02,788][105692] Updated weights for policy 0, policy_version 1632167 (0.0007) [2023-12-27 03:17:02,803][105620] Updated weights for policy 1, policy_version 1635431 (0.0008) [2023-12-27 03:17:02,850][105620] Updated weights for policy 1, policy_version 1635441 (0.0008) [2023-12-27 03:17:02,900][105620] Updated weights for policy 1, policy_version 1635451 (0.0006) [2023-12-27 03:17:03,460][105620] Updated weights for policy 1, policy_version 1635461 (0.0005) [2023-12-27 03:17:03,503][105692] Updated weights for policy 0, policy_version 1632177 (0.0007) [2023-12-27 03:17:03,517][105620] Updated weights for policy 1, policy_version 1635471 (0.0005) [2023-12-27 03:17:03,553][105692] Updated weights for policy 0, policy_version 1632187 (0.0006) [2023-12-27 03:17:03,572][105620] Updated weights for policy 1, policy_version 1635481 (0.0005) [2023-12-27 03:17:03,615][105692] Updated weights for policy 0, policy_version 1632197 (0.0006) [2023-12-27 03:17:03,672][105692] Updated weights for policy 0, policy_version 1632207 (0.0006) [2023-12-27 03:17:04,211][105692] Updated weights for policy 0, policy_version 1632217 (0.0008) [2023-12-27 03:17:04,263][105692] Updated weights for policy 0, policy_version 1632227 (0.0008) [2023-12-27 03:17:04,285][105620] Updated weights for policy 1, policy_version 1635491 (0.0008) [2023-12-27 03:17:04,322][105692] Updated weights for policy 0, policy_version 1632237 (0.0008) [2023-12-27 03:17:04,337][105620] Updated weights for policy 1, policy_version 1635501 (0.0008) [2023-12-27 03:17:04,393][105620] Updated weights for policy 1, policy_version 1635511 (0.0008) [2023-12-27 03:17:04,925][105692] Updated weights for policy 0, policy_version 1632247 (0.0007) [2023-12-27 03:17:04,978][105692] Updated weights for policy 0, policy_version 1632257 (0.0009) [2023-12-27 03:17:05,036][105692] Updated weights for policy 0, policy_version 1632267 (0.0009) [2023-12-27 03:17:05,222][105620] Updated weights for policy 1, policy_version 1635521 (0.0009) [2023-12-27 03:17:05,277][105620] Updated weights for policy 1, policy_version 1635531 (0.0009) [2023-12-27 03:17:05,340][105620] Updated weights for policy 1, policy_version 1635541 (0.0005) [2023-12-27 03:17:05,398][105620] Updated weights for policy 1, policy_version 1635551 (0.0007) [2023-12-27 03:17:05,615][105692] Updated weights for policy 0, policy_version 1632277 (0.0008) [2023-12-27 03:17:05,676][105692] Updated weights for policy 0, policy_version 1632287 (0.0009) [2023-12-27 03:17:05,727][105692] Updated weights for policy 0, policy_version 1632297 (0.0009) [2023-12-27 03:17:06,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19933.8, 300 sec: 19494.2). Total num frames: 836689920. Throughput: 0: 10190.8, 1: 10021.7. Samples: 836677964. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:17:06,063][104569] Avg episode reward: [(0, '8801.301'), (1, '8897.376')] [2023-12-27 03:17:06,154][105620] Updated weights for policy 1, policy_version 1635561 (0.0009) [2023-12-27 03:17:06,209][105620] Updated weights for policy 1, policy_version 1635571 (0.0009) [2023-12-27 03:17:06,265][105620] Updated weights for policy 1, policy_version 1635581 (0.0008) [2023-12-27 03:17:06,456][105692] Updated weights for policy 0, policy_version 1632307 (0.0008) [2023-12-27 03:17:06,519][105692] Updated weights for policy 0, policy_version 1632317 (0.0009) [2023-12-27 03:17:06,583][105692] Updated weights for policy 0, policy_version 1632327 (0.0009) [2023-12-27 03:17:07,035][105620] Updated weights for policy 1, policy_version 1635591 (0.0009) [2023-12-27 03:17:07,081][105620] Updated weights for policy 1, policy_version 1635601 (0.0009) [2023-12-27 03:17:07,134][105620] Updated weights for policy 1, policy_version 1635611 (0.0008) [2023-12-27 03:17:07,353][105692] Updated weights for policy 0, policy_version 1632337 (0.0009) [2023-12-27 03:17:07,406][105692] Updated weights for policy 0, policy_version 1632347 (0.0009) [2023-12-27 03:17:07,461][105692] Updated weights for policy 0, policy_version 1632357 (0.0009) [2023-12-27 03:17:07,511][105692] Updated weights for policy 0, policy_version 1632367 (0.0009) [2023-12-27 03:17:07,922][105620] Updated weights for policy 1, policy_version 1635621 (0.0008) [2023-12-27 03:17:07,981][105620] Updated weights for policy 1, policy_version 1635631 (0.0009) [2023-12-27 03:17:08,041][105620] Updated weights for policy 1, policy_version 1635641 (0.0009) [2023-12-27 03:17:08,239][105692] Updated weights for policy 0, policy_version 1632377 (0.0009) [2023-12-27 03:17:08,304][105692] Updated weights for policy 0, policy_version 1632387 (0.0009) [2023-12-27 03:17:08,368][105692] Updated weights for policy 0, policy_version 1632397 (0.0009) [2023-12-27 03:17:08,807][105620] Updated weights for policy 1, policy_version 1635651 (0.0009) [2023-12-27 03:17:08,867][105620] Updated weights for policy 1, policy_version 1635661 (0.0010) [2023-12-27 03:17:08,923][105620] Updated weights for policy 1, policy_version 1635671 (0.0009) [2023-12-27 03:17:09,107][105692] Updated weights for policy 0, policy_version 1632407 (0.0007) [2023-12-27 03:17:09,165][105692] Updated weights for policy 0, policy_version 1632417 (0.0008) [2023-12-27 03:17:09,227][105692] Updated weights for policy 0, policy_version 1632427 (0.0009) [2023-12-27 03:17:09,738][105620] Updated weights for policy 1, policy_version 1635681 (0.0009) [2023-12-27 03:17:09,793][105620] Updated weights for policy 1, policy_version 1635691 (0.0009) [2023-12-27 03:17:09,862][105620] Updated weights for policy 1, policy_version 1635701 (0.0010) [2023-12-27 03:17:09,916][105620] Updated weights for policy 1, policy_version 1635711 (0.0008) [2023-12-27 03:17:09,985][105692] Updated weights for policy 0, policy_version 1632437 (0.0008) [2023-12-27 03:17:10,040][105692] Updated weights for policy 0, policy_version 1632447 (0.0006) [2023-12-27 03:17:10,100][105692] Updated weights for policy 0, policy_version 1632457 (0.0006) [2023-12-27 03:17:10,697][105620] Updated weights for policy 1, policy_version 1635721 (0.0009) [2023-12-27 03:17:10,751][105620] Updated weights for policy 1, policy_version 1635731 (0.0008) [2023-12-27 03:17:10,797][105620] Updated weights for policy 1, policy_version 1635741 (0.0009) [2023-12-27 03:17:10,824][105692] Updated weights for policy 0, policy_version 1632467 (0.0009) [2023-12-27 03:17:10,883][105692] Updated weights for policy 0, policy_version 1632477 (0.0009) [2023-12-27 03:17:10,951][105692] Updated weights for policy 0, policy_version 1632487 (0.0009) [2023-12-27 03:17:11,062][104569] Fps is (10 sec: 20479.8, 60 sec: 20070.4, 300 sec: 19521.9). Total num frames: 836788224. Throughput: 0: 10225.8, 1: 9943.7. Samples: 836791872. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:17:11,063][104569] Avg episode reward: [(0, '9264.448'), (1, '9082.242')] [2023-12-27 03:17:11,627][105620] Updated weights for policy 1, policy_version 1635751 (0.0008) [2023-12-27 03:17:11,689][105620] Updated weights for policy 1, policy_version 1635761 (0.0008) [2023-12-27 03:17:11,717][105692] Updated weights for policy 0, policy_version 1632497 (0.0009) [2023-12-27 03:17:11,753][105620] Updated weights for policy 1, policy_version 1635771 (0.0008) [2023-12-27 03:17:11,788][105692] Updated weights for policy 0, policy_version 1632507 (0.0008) [2023-12-27 03:17:11,844][105692] Updated weights for policy 0, policy_version 1632517 (0.0008) [2023-12-27 03:17:11,905][105692] Updated weights for policy 0, policy_version 1632527 (0.0007) [2023-12-27 03:17:12,575][105620] Updated weights for policy 1, policy_version 1635781 (0.0009) [2023-12-27 03:17:12,629][105620] Updated weights for policy 1, policy_version 1635791 (0.0009) [2023-12-27 03:17:12,666][105692] Updated weights for policy 0, policy_version 1632537 (0.0006) [2023-12-27 03:17:12,690][105620] Updated weights for policy 1, policy_version 1635801 (0.0009) [2023-12-27 03:17:12,731][105692] Updated weights for policy 0, policy_version 1632547 (0.0006) [2023-12-27 03:17:12,794][105692] Updated weights for policy 0, policy_version 1632557 (0.0007) [2023-12-27 03:17:13,467][105620] Updated weights for policy 1, policy_version 1635811 (0.0009) [2023-12-27 03:17:13,495][105692] Updated weights for policy 0, policy_version 1632567 (0.0007) [2023-12-27 03:17:13,528][105620] Updated weights for policy 1, policy_version 1635821 (0.0008) [2023-12-27 03:17:13,547][105692] Updated weights for policy 0, policy_version 1632577 (0.0008) [2023-12-27 03:17:13,587][105620] Updated weights for policy 1, policy_version 1635831 (0.0007) [2023-12-27 03:17:13,612][105692] Updated weights for policy 0, policy_version 1632588 (0.0009) [2023-12-27 03:17:14,323][105620] Updated weights for policy 1, policy_version 1635841 (0.0008) [2023-12-27 03:17:14,353][105692] Updated weights for policy 0, policy_version 1632598 (0.0006) [2023-12-27 03:17:14,383][105620] Updated weights for policy 1, policy_version 1635851 (0.0008) [2023-12-27 03:17:14,406][105692] Updated weights for policy 0, policy_version 1632608 (0.0005) [2023-12-27 03:17:14,436][105620] Updated weights for policy 1, policy_version 1635861 (0.0006) [2023-12-27 03:17:14,459][105692] Updated weights for policy 0, policy_version 1632618 (0.0007) [2023-12-27 03:17:14,492][105620] Updated weights for policy 1, policy_version 1635871 (0.0009) [2023-12-27 03:17:15,225][105692] Updated weights for policy 0, policy_version 1632628 (0.0007) [2023-12-27 03:17:15,238][105620] Updated weights for policy 1, policy_version 1635881 (0.0009) [2023-12-27 03:17:15,285][105692] Updated weights for policy 0, policy_version 1632638 (0.0007) [2023-12-27 03:17:15,299][105620] Updated weights for policy 1, policy_version 1635891 (0.0006) [2023-12-27 03:17:15,339][105692] Updated weights for policy 0, policy_version 1632648 (0.0006) [2023-12-27 03:17:15,353][105620] Updated weights for policy 1, policy_version 1635901 (0.0007) [2023-12-27 03:17:16,062][104569] Fps is (10 sec: 18022.0, 60 sec: 19797.2, 300 sec: 19466.4). Total num frames: 836870144. Throughput: 0: 10135.6, 1: 9802.9. Samples: 836846564. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:17:16,063][104569] Avg episode reward: [(0, '8987.770'), (1, '9265.046')] [2023-12-27 03:17:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001632656_418021376.pth... [2023-12-27 03:17:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001631472_417718272.pth [2023-12-27 03:17:16,097][105620] Updated weights for policy 1, policy_version 1635911 (0.0008) [2023-12-27 03:17:16,099][105692] Updated weights for policy 0, policy_version 1632658 (0.0007) [2023-12-27 03:17:16,155][105692] Updated weights for policy 0, policy_version 1632668 (0.0008) [2023-12-27 03:17:16,157][105620] Updated weights for policy 1, policy_version 1635921 (0.0006) [2023-12-27 03:17:16,212][105620] Updated weights for policy 1, policy_version 1635931 (0.0006) [2023-12-27 03:17:16,218][105692] Updated weights for policy 0, policy_version 1632678 (0.0008) [2023-12-27 03:17:16,242][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001635936_418856960.pth... [2023-12-27 03:17:16,245][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001634784_418562048.pth [2023-12-27 03:17:16,282][105692] Updated weights for policy 0, policy_version 1632688 (0.0007) [2023-12-27 03:17:16,933][105692] Updated weights for policy 0, policy_version 1632698 (0.0009) [2023-12-27 03:17:16,974][105620] Updated weights for policy 1, policy_version 1635941 (0.0007) [2023-12-27 03:17:16,988][105692] Updated weights for policy 0, policy_version 1632708 (0.0008) [2023-12-27 03:17:17,035][105620] Updated weights for policy 1, policy_version 1635951 (0.0009) [2023-12-27 03:17:17,045][105692] Updated weights for policy 0, policy_version 1632718 (0.0009) [2023-12-27 03:17:17,093][105620] Updated weights for policy 1, policy_version 1635961 (0.0008) [2023-12-27 03:17:17,782][105692] Updated weights for policy 0, policy_version 1632728 (0.0008) [2023-12-27 03:17:17,832][105692] Updated weights for policy 0, policy_version 1632738 (0.0008) [2023-12-27 03:17:17,872][105620] Updated weights for policy 1, policy_version 1635971 (0.0008) [2023-12-27 03:17:17,887][105692] Updated weights for policy 0, policy_version 1632748 (0.0009) [2023-12-27 03:17:17,932][105620] Updated weights for policy 1, policy_version 1635981 (0.0008) [2023-12-27 03:17:17,991][105620] Updated weights for policy 1, policy_version 1635991 (0.0009) [2023-12-27 03:17:18,658][105692] Updated weights for policy 0, policy_version 1632758 (0.0008) [2023-12-27 03:17:18,705][105692] Updated weights for policy 0, policy_version 1632768 (0.0009) [2023-12-27 03:17:18,752][105692] Updated weights for policy 0, policy_version 1632778 (0.0009) [2023-12-27 03:17:18,760][105620] Updated weights for policy 1, policy_version 1636001 (0.0009) [2023-12-27 03:17:18,816][105620] Updated weights for policy 1, policy_version 1636011 (0.0008) [2023-12-27 03:17:18,865][105620] Updated weights for policy 1, policy_version 1636021 (0.0008) [2023-12-27 03:17:18,912][105620] Updated weights for policy 1, policy_version 1636031 (0.0009) [2023-12-27 03:17:19,585][105692] Updated weights for policy 0, policy_version 1632788 (0.0008) [2023-12-27 03:17:19,645][105692] Updated weights for policy 0, policy_version 1632798 (0.0011) [2023-12-27 03:17:19,647][105620] Updated weights for policy 1, policy_version 1636041 (0.0006) [2023-12-27 03:17:19,705][105692] Updated weights for policy 0, policy_version 1632808 (0.0011) [2023-12-27 03:17:19,711][105620] Updated weights for policy 1, policy_version 1636051 (0.0005) [2023-12-27 03:17:19,775][105620] Updated weights for policy 1, policy_version 1636061 (0.0006) [2023-12-27 03:17:20,479][105692] Updated weights for policy 0, policy_version 1632818 (0.0010) [2023-12-27 03:17:20,487][105620] Updated weights for policy 1, policy_version 1636071 (0.0006) [2023-12-27 03:17:20,540][105692] Updated weights for policy 0, policy_version 1632828 (0.0010) [2023-12-27 03:17:20,554][105620] Updated weights for policy 1, policy_version 1636081 (0.0006) [2023-12-27 03:17:20,604][105692] Updated weights for policy 0, policy_version 1632838 (0.0010) [2023-12-27 03:17:20,623][105620] Updated weights for policy 1, policy_version 1636091 (0.0006) [2023-12-27 03:17:20,669][105692] Updated weights for policy 0, policy_version 1632848 (0.0008) [2023-12-27 03:17:21,062][104569] Fps is (10 sec: 18022.7, 60 sec: 19933.9, 300 sec: 19466.4). Total num frames: 836968448. Throughput: 0: 9985.3, 1: 9608.4. Samples: 836958540. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:17:21,062][104569] Avg episode reward: [(0, '8437.650'), (1, '9080.536')] [2023-12-27 03:17:21,339][105620] Updated weights for policy 1, policy_version 1636101 (0.0007) [2023-12-27 03:17:21,408][105692] Updated weights for policy 0, policy_version 1632858 (0.0007) [2023-12-27 03:17:21,410][105620] Updated weights for policy 1, policy_version 1636111 (0.0008) [2023-12-27 03:17:21,480][105692] Updated weights for policy 0, policy_version 1632868 (0.0006) [2023-12-27 03:17:21,481][105620] Updated weights for policy 1, policy_version 1636121 (0.0010) [2023-12-27 03:17:21,547][105692] Updated weights for policy 0, policy_version 1632878 (0.0007) [2023-12-27 03:17:22,210][105692] Updated weights for policy 0, policy_version 1632888 (0.0009) [2023-12-27 03:17:22,274][105692] Updated weights for policy 0, policy_version 1632898 (0.0009) [2023-12-27 03:17:22,289][105620] Updated weights for policy 1, policy_version 1636131 (0.0008) [2023-12-27 03:17:22,333][105692] Updated weights for policy 0, policy_version 1632908 (0.0008) [2023-12-27 03:17:22,348][105620] Updated weights for policy 1, policy_version 1636141 (0.0006) [2023-12-27 03:17:22,354][105585] KL-divergence is very high: 114.9541 [2023-12-27 03:17:22,415][105620] Updated weights for policy 1, policy_version 1636151 (0.0009) [2023-12-27 03:17:23,080][105692] Updated weights for policy 0, policy_version 1632918 (0.0009) [2023-12-27 03:17:23,129][105620] Updated weights for policy 1, policy_version 1636161 (0.0008) [2023-12-27 03:17:23,131][105692] Updated weights for policy 0, policy_version 1632928 (0.0008) [2023-12-27 03:17:23,179][105620] Updated weights for policy 1, policy_version 1636171 (0.0007) [2023-12-27 03:17:23,187][105692] Updated weights for policy 0, policy_version 1632938 (0.0008) [2023-12-27 03:17:23,235][105620] Updated weights for policy 1, policy_version 1636181 (0.0009) [2023-12-27 03:17:23,298][105620] Updated weights for policy 1, policy_version 1636191 (0.0009) [2023-12-27 03:17:23,926][105692] Updated weights for policy 0, policy_version 1632948 (0.0006) [2023-12-27 03:17:23,985][105692] Updated weights for policy 0, policy_version 1632958 (0.0005) [2023-12-27 03:17:23,998][105620] Updated weights for policy 1, policy_version 1636201 (0.0006) [2023-12-27 03:17:24,035][105692] Updated weights for policy 0, policy_version 1632968 (0.0005) [2023-12-27 03:17:24,042][105620] Updated weights for policy 1, policy_version 1636211 (0.0005) [2023-12-27 03:17:24,090][105620] Updated weights for policy 1, policy_version 1636221 (0.0006) [2023-12-27 03:17:24,612][105692] Updated weights for policy 0, policy_version 1632978 (0.0006) [2023-12-27 03:17:24,658][105692] Updated weights for policy 0, policy_version 1632988 (0.0008) [2023-12-27 03:17:24,705][105692] Updated weights for policy 0, policy_version 1632998 (0.0009) [2023-12-27 03:17:24,766][105692] Updated weights for policy 0, policy_version 1633008 (0.0009) [2023-12-27 03:17:24,844][105620] Updated weights for policy 1, policy_version 1636231 (0.0010) [2023-12-27 03:17:24,897][105620] Updated weights for policy 1, policy_version 1636242 (0.0010) [2023-12-27 03:17:24,955][105620] Updated weights for policy 1, policy_version 1636252 (0.0010) [2023-12-27 03:17:25,369][105692] Updated weights for policy 0, policy_version 1633018 (0.0006) [2023-12-27 03:17:25,422][105692] Updated weights for policy 0, policy_version 1633028 (0.0009) [2023-12-27 03:17:25,469][105692] Updated weights for policy 0, policy_version 1633038 (0.0009) [2023-12-27 03:17:25,841][105620] Updated weights for policy 1, policy_version 1636262 (0.0009) [2023-12-27 03:17:25,898][105620] Updated weights for policy 1, policy_version 1636272 (0.0009) [2023-12-27 03:17:25,962][105620] Updated weights for policy 1, policy_version 1636283 (0.0008) [2023-12-27 03:17:26,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19933.9, 300 sec: 19438.6). Total num frames: 837066752. Throughput: 0: 10007.8, 1: 9474.9. Samples: 837073112. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:17:26,063][104569] Avg episode reward: [(0, '8170.027'), (1, '9080.655')] [2023-12-27 03:17:26,070][105692] Updated weights for policy 0, policy_version 1633048 (0.0009) [2023-12-27 03:17:26,127][105692] Updated weights for policy 0, policy_version 1633058 (0.0009) [2023-12-27 03:17:26,173][105692] Updated weights for policy 0, policy_version 1633068 (0.0009) [2023-12-27 03:17:26,798][105692] Updated weights for policy 0, policy_version 1633078 (0.0008) [2023-12-27 03:17:26,816][105620] Updated weights for policy 1, policy_version 1636293 (0.0008) [2023-12-27 03:17:26,866][105692] Updated weights for policy 0, policy_version 1633088 (0.0009) [2023-12-27 03:17:26,869][105620] Updated weights for policy 1, policy_version 1636303 (0.0006) [2023-12-27 03:17:26,928][105620] Updated weights for policy 1, policy_version 1636313 (0.0006) [2023-12-27 03:17:26,930][105692] Updated weights for policy 0, policy_version 1633098 (0.0010) [2023-12-27 03:17:27,533][105620] Updated weights for policy 1, policy_version 1636323 (0.0006) [2023-12-27 03:17:27,593][105620] Updated weights for policy 1, policy_version 1636333 (0.0009) [2023-12-27 03:17:27,647][105620] Updated weights for policy 1, policy_version 1636343 (0.0009) [2023-12-27 03:17:27,723][105692] Updated weights for policy 0, policy_version 1633108 (0.0008) [2023-12-27 03:17:27,769][105692] Updated weights for policy 0, policy_version 1633118 (0.0008) [2023-12-27 03:17:27,823][105692] Updated weights for policy 0, policy_version 1633128 (0.0009) [2023-12-27 03:17:28,412][105620] Updated weights for policy 1, policy_version 1636353 (0.0009) [2023-12-27 03:17:28,473][105620] Updated weights for policy 1, policy_version 1636363 (0.0009) [2023-12-27 03:17:28,528][105620] Updated weights for policy 1, policy_version 1636373 (0.0008) [2023-12-27 03:17:28,529][105692] Updated weights for policy 0, policy_version 1633138 (0.0009) [2023-12-27 03:17:28,586][105620] Updated weights for policy 1, policy_version 1636383 (0.0006) [2023-12-27 03:17:28,591][105692] Updated weights for policy 0, policy_version 1633148 (0.0008) [2023-12-27 03:17:28,654][105692] Updated weights for policy 0, policy_version 1633158 (0.0010) [2023-12-27 03:17:28,708][105692] Updated weights for policy 0, policy_version 1633168 (0.0009) [2023-12-27 03:17:29,395][105620] Updated weights for policy 1, policy_version 1636393 (0.0010) [2023-12-27 03:17:29,453][105692] Updated weights for policy 0, policy_version 1633178 (0.0006) [2023-12-27 03:17:29,459][105620] Updated weights for policy 1, policy_version 1636403 (0.0011) [2023-12-27 03:17:29,513][105692] Updated weights for policy 0, policy_version 1633188 (0.0005) [2023-12-27 03:17:29,519][105620] Updated weights for policy 1, policy_version 1636413 (0.0011) [2023-12-27 03:17:29,567][105692] Updated weights for policy 0, policy_version 1633198 (0.0007) [2023-12-27 03:17:30,195][105620] Updated weights for policy 1, policy_version 1636423 (0.0009) [2023-12-27 03:17:30,257][105620] Updated weights for policy 1, policy_version 1636433 (0.0009) [2023-12-27 03:17:30,318][105620] Updated weights for policy 1, policy_version 1636443 (0.0006) [2023-12-27 03:17:30,421][105692] Updated weights for policy 0, policy_version 1633208 (0.0009) [2023-12-27 03:17:30,476][105692] Updated weights for policy 0, policy_version 1633218 (0.0008) [2023-12-27 03:17:30,528][105692] Updated weights for policy 0, policy_version 1633228 (0.0007) [2023-12-27 03:17:30,946][105620] Updated weights for policy 1, policy_version 1636453 (0.0007) [2023-12-27 03:17:31,011][105620] Updated weights for policy 1, policy_version 1636463 (0.0005) [2023-12-27 03:17:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 837156864. Throughput: 0: 10014.4, 1: 9509.0. Samples: 837132080. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:17:31,062][104569] Avg episode reward: [(0, '8805.711'), (1, '9081.155')] [2023-12-27 03:17:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001633232_418168832.pth... [2023-12-27 03:17:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001632112_417882112.pth [2023-12-27 03:17:31,080][105620] Updated weights for policy 1, policy_version 1636473 (0.0006) [2023-12-27 03:17:31,126][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001636480_418996224.pth... [2023-12-27 03:17:31,131][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001635392_418717696.pth [2023-12-27 03:17:31,284][105692] Updated weights for policy 0, policy_version 1633238 (0.0007) [2023-12-27 03:17:31,337][105692] Updated weights for policy 0, policy_version 1633248 (0.0008) [2023-12-27 03:17:31,398][105692] Updated weights for policy 0, policy_version 1633258 (0.0008) [2023-12-27 03:17:31,739][105620] Updated weights for policy 1, policy_version 1636483 (0.0006) [2023-12-27 03:17:31,795][105620] Updated weights for policy 1, policy_version 1636493 (0.0009) [2023-12-27 03:17:31,840][105620] Updated weights for policy 1, policy_version 1636503 (0.0010) [2023-12-27 03:17:32,202][105692] Updated weights for policy 0, policy_version 1633268 (0.0009) [2023-12-27 03:17:32,258][105692] Updated weights for policy 0, policy_version 1633278 (0.0008) [2023-12-27 03:17:32,307][105692] Updated weights for policy 0, policy_version 1633288 (0.0008) [2023-12-27 03:17:32,585][105620] Updated weights for policy 1, policy_version 1636513 (0.0010) [2023-12-27 03:17:32,643][105620] Updated weights for policy 1, policy_version 1636523 (0.0010) [2023-12-27 03:17:32,704][105620] Updated weights for policy 1, policy_version 1636533 (0.0010) [2023-12-27 03:17:32,762][105620] Updated weights for policy 1, policy_version 1636543 (0.0010) [2023-12-27 03:17:33,121][105692] Updated weights for policy 0, policy_version 1633298 (0.0007) [2023-12-27 03:17:33,186][105692] Updated weights for policy 0, policy_version 1633308 (0.0009) [2023-12-27 03:17:33,238][105692] Updated weights for policy 0, policy_version 1633318 (0.0011) [2023-12-27 03:17:33,287][105692] Updated weights for policy 0, policy_version 1633328 (0.0010) [2023-12-27 03:17:33,452][105620] Updated weights for policy 1, policy_version 1636553 (0.0010) [2023-12-27 03:17:33,500][105620] Updated weights for policy 1, policy_version 1636563 (0.0010) [2023-12-27 03:17:33,547][105620] Updated weights for policy 1, policy_version 1636573 (0.0010) [2023-12-27 03:17:33,976][105692] Updated weights for policy 0, policy_version 1633338 (0.0011) [2023-12-27 03:17:34,031][105692] Updated weights for policy 0, policy_version 1633348 (0.0011) [2023-12-27 03:17:34,075][105692] Updated weights for policy 0, policy_version 1633358 (0.0010) [2023-12-27 03:17:34,307][105620] Updated weights for policy 1, policy_version 1636583 (0.0010) [2023-12-27 03:17:34,359][105620] Updated weights for policy 1, policy_version 1636593 (0.0010) [2023-12-27 03:17:34,414][105620] Updated weights for policy 1, policy_version 1636603 (0.0010) [2023-12-27 03:17:34,808][105692] Updated weights for policy 0, policy_version 1633368 (0.0011) [2023-12-27 03:17:34,859][105692] Updated weights for policy 0, policy_version 1633378 (0.0010) [2023-12-27 03:17:34,913][105692] Updated weights for policy 0, policy_version 1633388 (0.0005) [2023-12-27 03:17:35,167][105620] Updated weights for policy 1, policy_version 1636613 (0.0010) [2023-12-27 03:17:35,226][105620] Updated weights for policy 1, policy_version 1636623 (0.0008) [2023-12-27 03:17:35,287][105620] Updated weights for policy 1, policy_version 1636633 (0.0010) [2023-12-27 03:17:35,450][105692] Updated weights for policy 0, policy_version 1633398 (0.0005) [2023-12-27 03:17:35,493][105692] Updated weights for policy 0, policy_version 1633408 (0.0005) [2023-12-27 03:17:35,555][105692] Updated weights for policy 0, policy_version 1633418 (0.0005) [2023-12-27 03:17:36,009][105620] Updated weights for policy 1, policy_version 1636643 (0.0010) [2023-12-27 03:17:36,057][105620] Updated weights for policy 1, policy_version 1636653 (0.0010) [2023-12-27 03:17:36,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 837255168. Throughput: 0: 9886.2, 1: 9457.9. Samples: 837246384. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:17:36,062][104569] Avg episode reward: [(0, '8895.340'), (1, '9081.073')] [2023-12-27 03:17:36,108][105620] Updated weights for policy 1, policy_version 1636663 (0.0010) [2023-12-27 03:17:36,184][105692] Updated weights for policy 0, policy_version 1633428 (0.0008) [2023-12-27 03:17:36,247][105692] Updated weights for policy 0, policy_version 1633438 (0.0011) [2023-12-27 03:17:36,317][105692] Updated weights for policy 0, policy_version 1633448 (0.0007) [2023-12-27 03:17:36,876][105620] Updated weights for policy 1, policy_version 1636673 (0.0010) [2023-12-27 03:17:36,931][105620] Updated weights for policy 1, policy_version 1636683 (0.0010) [2023-12-27 03:17:36,932][105692] Updated weights for policy 0, policy_version 1633458 (0.0007) [2023-12-27 03:17:36,984][105692] Updated weights for policy 0, policy_version 1633468 (0.0010) [2023-12-27 03:17:36,989][105620] Updated weights for policy 1, policy_version 1636693 (0.0010) [2023-12-27 03:17:37,034][105692] Updated weights for policy 0, policy_version 1633478 (0.0011) [2023-12-27 03:17:37,041][105620] Updated weights for policy 1, policy_version 1636703 (0.0010) [2023-12-27 03:17:37,082][105692] Updated weights for policy 0, policy_version 1633488 (0.0010) [2023-12-27 03:17:37,672][105620] Updated weights for policy 1, policy_version 1636713 (0.0008) [2023-12-27 03:17:37,710][105692] Updated weights for policy 0, policy_version 1633498 (0.0010) [2023-12-27 03:17:37,730][105620] Updated weights for policy 1, policy_version 1636723 (0.0009) [2023-12-27 03:17:37,762][105692] Updated weights for policy 0, policy_version 1633508 (0.0010) [2023-12-27 03:17:37,781][105620] Updated weights for policy 1, policy_version 1636733 (0.0008) [2023-12-27 03:17:37,811][105692] Updated weights for policy 0, policy_version 1633518 (0.0010) [2023-12-27 03:17:38,532][105620] Updated weights for policy 1, policy_version 1636743 (0.0008) [2023-12-27 03:17:38,563][105692] Updated weights for policy 0, policy_version 1633528 (0.0009) [2023-12-27 03:17:38,589][105620] Updated weights for policy 1, policy_version 1636753 (0.0006) [2023-12-27 03:17:38,622][105692] Updated weights for policy 0, policy_version 1633538 (0.0008) [2023-12-27 03:17:38,656][105620] Updated weights for policy 1, policy_version 1636763 (0.0007) [2023-12-27 03:17:38,675][105692] Updated weights for policy 0, policy_version 1633548 (0.0007) [2023-12-27 03:17:39,341][105692] Updated weights for policy 0, policy_version 1633558 (0.0009) [2023-12-27 03:17:39,413][105692] Updated weights for policy 0, policy_version 1633568 (0.0009) [2023-12-27 03:17:39,477][105620] Updated weights for policy 1, policy_version 1636773 (0.0008) [2023-12-27 03:17:39,478][105692] Updated weights for policy 0, policy_version 1633578 (0.0008) [2023-12-27 03:17:39,524][105620] Updated weights for policy 1, policy_version 1636783 (0.0007) [2023-12-27 03:17:39,577][105620] Updated weights for policy 1, policy_version 1636793 (0.0010) [2023-12-27 03:17:40,116][105692] Updated weights for policy 0, policy_version 1633588 (0.0010) [2023-12-27 03:17:40,168][105692] Updated weights for policy 0, policy_version 1633598 (0.0009) [2023-12-27 03:17:40,227][105692] Updated weights for policy 0, policy_version 1633608 (0.0009) [2023-12-27 03:17:40,388][105620] Updated weights for policy 1, policy_version 1636803 (0.0007) [2023-12-27 03:17:40,442][105620] Updated weights for policy 1, policy_version 1636813 (0.0006) [2023-12-27 03:17:40,501][105620] Updated weights for policy 1, policy_version 1636823 (0.0006) [2023-12-27 03:17:40,999][105692] Updated weights for policy 0, policy_version 1633618 (0.0009) [2023-12-27 03:17:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 837353472. Throughput: 0: 9871.3, 1: 9465.0. Samples: 837367244. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:17:41,063][104569] Avg episode reward: [(0, '8805.668'), (1, '8991.801')] [2023-12-27 03:17:41,067][105692] Updated weights for policy 0, policy_version 1633628 (0.0010) [2023-12-27 03:17:41,135][105692] Updated weights for policy 0, policy_version 1633638 (0.0006) [2023-12-27 03:17:41,142][105620] Updated weights for policy 1, policy_version 1636833 (0.0005) [2023-12-27 03:17:41,194][105692] Updated weights for policy 0, policy_version 1633648 (0.0008) [2023-12-27 03:17:41,206][105620] Updated weights for policy 1, policy_version 1636843 (0.0008) [2023-12-27 03:17:41,270][105620] Updated weights for policy 1, policy_version 1636853 (0.0007) [2023-12-27 03:17:41,320][105620] Updated weights for policy 1, policy_version 1636863 (0.0008) [2023-12-27 03:17:42,001][105692] Updated weights for policy 0, policy_version 1633658 (0.0008) [2023-12-27 03:17:42,066][105692] Updated weights for policy 0, policy_version 1633668 (0.0008) [2023-12-27 03:17:42,069][105620] Updated weights for policy 1, policy_version 1636873 (0.0006) [2023-12-27 03:17:42,120][105692] Updated weights for policy 0, policy_version 1633678 (0.0007) [2023-12-27 03:17:42,137][105620] Updated weights for policy 1, policy_version 1636883 (0.0007) [2023-12-27 03:17:42,206][105620] Updated weights for policy 1, policy_version 1636893 (0.0009) [2023-12-27 03:17:42,861][105620] Updated weights for policy 1, policy_version 1636903 (0.0007) [2023-12-27 03:17:42,909][105692] Updated weights for policy 0, policy_version 1633688 (0.0009) [2023-12-27 03:17:42,926][105620] Updated weights for policy 1, policy_version 1636913 (0.0006) [2023-12-27 03:17:42,969][105692] Updated weights for policy 0, policy_version 1633698 (0.0008) [2023-12-27 03:17:42,991][105620] Updated weights for policy 1, policy_version 1636923 (0.0006) [2023-12-27 03:17:43,028][105692] Updated weights for policy 0, policy_version 1633708 (0.0008) [2023-12-27 03:17:43,556][105620] Updated weights for policy 1, policy_version 1636933 (0.0008) [2023-12-27 03:17:43,615][105620] Updated weights for policy 1, policy_version 1636943 (0.0008) [2023-12-27 03:17:43,666][105620] Updated weights for policy 1, policy_version 1636953 (0.0008) [2023-12-27 03:17:43,847][105692] Updated weights for policy 0, policy_version 1633718 (0.0010) [2023-12-27 03:17:43,895][105692] Updated weights for policy 0, policy_version 1633728 (0.0010) [2023-12-27 03:17:43,940][105692] Updated weights for policy 0, policy_version 1633738 (0.0010) [2023-12-27 03:17:44,469][105620] Updated weights for policy 1, policy_version 1636963 (0.0008) [2023-12-27 03:17:44,523][105620] Updated weights for policy 1, policy_version 1636973 (0.0008) [2023-12-27 03:17:44,577][105620] Updated weights for policy 1, policy_version 1636983 (0.0008) [2023-12-27 03:17:44,620][105692] Updated weights for policy 0, policy_version 1633748 (0.0008) [2023-12-27 03:17:44,677][105692] Updated weights for policy 0, policy_version 1633758 (0.0007) [2023-12-27 03:17:44,736][105692] Updated weights for policy 0, policy_version 1633768 (0.0011) [2023-12-27 03:17:45,345][105620] Updated weights for policy 1, policy_version 1636993 (0.0008) [2023-12-27 03:17:45,408][105620] Updated weights for policy 1, policy_version 1637003 (0.0008) [2023-12-27 03:17:45,468][105620] Updated weights for policy 1, policy_version 1637013 (0.0008) [2023-12-27 03:17:45,479][105692] Updated weights for policy 0, policy_version 1633778 (0.0011) [2023-12-27 03:17:45,528][105620] Updated weights for policy 1, policy_version 1637023 (0.0006) [2023-12-27 03:17:45,534][105692] Updated weights for policy 0, policy_version 1633788 (0.0011) [2023-12-27 03:17:45,583][105692] Updated weights for policy 0, policy_version 1633798 (0.0010) [2023-12-27 03:17:45,641][105692] Updated weights for policy 0, policy_version 1633808 (0.0007) [2023-12-27 03:17:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 837451776. Throughput: 0: 9715.2, 1: 9502.7. Samples: 837423664. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:17:46,062][104569] Avg episode reward: [(0, '8531.494'), (1, '8899.801')] [2023-12-27 03:17:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001633808_418316288.pth... [2023-12-27 03:17:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001637024_419135488.pth... [2023-12-27 03:17:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001635936_418856960.pth [2023-12-27 03:17:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001632656_418021376.pth [2023-12-27 03:17:46,241][105692] Updated weights for policy 0, policy_version 1633818 (0.0005) [2023-12-27 03:17:46,300][105692] Updated weights for policy 0, policy_version 1633828 (0.0005) [2023-12-27 03:17:46,350][105692] Updated weights for policy 0, policy_version 1633838 (0.0005) [2023-12-27 03:17:46,356][105620] Updated weights for policy 1, policy_version 1637033 (0.0008) [2023-12-27 03:17:46,412][105620] Updated weights for policy 1, policy_version 1637043 (0.0010) [2023-12-27 03:17:46,466][105620] Updated weights for policy 1, policy_version 1637055 (0.0010) [2023-12-27 03:17:46,870][105692] Updated weights for policy 0, policy_version 1633848 (0.0005) [2023-12-27 03:17:46,926][105692] Updated weights for policy 0, policy_version 1633858 (0.0005) [2023-12-27 03:17:46,974][105692] Updated weights for policy 0, policy_version 1633868 (0.0005) [2023-12-27 03:17:47,384][105620] Updated weights for policy 1, policy_version 1637065 (0.0010) [2023-12-27 03:17:47,443][105620] Updated weights for policy 1, policy_version 1637077 (0.0010) [2023-12-27 03:17:47,495][105620] Updated weights for policy 1, policy_version 1637087 (0.0010) [2023-12-27 03:17:47,532][105692] Updated weights for policy 0, policy_version 1633878 (0.0005) [2023-12-27 03:17:47,587][105692] Updated weights for policy 0, policy_version 1633888 (0.0005) [2023-12-27 03:17:47,646][105692] Updated weights for policy 0, policy_version 1633898 (0.0005) [2023-12-27 03:17:48,232][105692] Updated weights for policy 0, policy_version 1633908 (0.0006) [2023-12-27 03:17:48,282][105620] Updated weights for policy 1, policy_version 1637097 (0.0006) [2023-12-27 03:17:48,296][105692] Updated weights for policy 0, policy_version 1633918 (0.0008) [2023-12-27 03:17:48,352][105620] Updated weights for policy 1, policy_version 1637107 (0.0007) [2023-12-27 03:17:48,363][105692] Updated weights for policy 0, policy_version 1633928 (0.0008) [2023-12-27 03:17:48,417][105620] Updated weights for policy 1, policy_version 1637117 (0.0007) [2023-12-27 03:17:49,045][105620] Updated weights for policy 1, policy_version 1637127 (0.0007) [2023-12-27 03:17:49,091][105692] Updated weights for policy 0, policy_version 1633938 (0.0009) [2023-12-27 03:17:49,107][105620] Updated weights for policy 1, policy_version 1637138 (0.0007) [2023-12-27 03:17:49,140][105692] Updated weights for policy 0, policy_version 1633948 (0.0010) [2023-12-27 03:17:49,155][105620] Updated weights for policy 1, policy_version 1637148 (0.0008) [2023-12-27 03:17:49,188][105692] Updated weights for policy 0, policy_version 1633958 (0.0010) [2023-12-27 03:17:49,256][105692] Updated weights for policy 0, policy_version 1633968 (0.0009) [2023-12-27 03:17:49,869][105620] Updated weights for policy 1, policy_version 1637158 (0.0008) [2023-12-27 03:17:49,933][105620] Updated weights for policy 1, policy_version 1637168 (0.0010) [2023-12-27 03:17:49,960][105692] Updated weights for policy 0, policy_version 1633978 (0.0011) [2023-12-27 03:17:49,987][105620] Updated weights for policy 1, policy_version 1637178 (0.0011) [2023-12-27 03:17:50,017][105692] Updated weights for policy 0, policy_version 1633988 (0.0008) [2023-12-27 03:17:50,074][105692] Updated weights for policy 0, policy_version 1633998 (0.0005) [2023-12-27 03:17:50,695][105692] Updated weights for policy 0, policy_version 1634008 (0.0010) [2023-12-27 03:17:50,717][105620] Updated weights for policy 1, policy_version 1637188 (0.0009) [2023-12-27 03:17:50,751][105692] Updated weights for policy 0, policy_version 1634018 (0.0010) [2023-12-27 03:17:50,777][105620] Updated weights for policy 1, policy_version 1637198 (0.0011) [2023-12-27 03:17:50,811][105692] Updated weights for policy 0, policy_version 1634028 (0.0010) [2023-12-27 03:17:50,837][105620] Updated weights for policy 1, policy_version 1637208 (0.0011) [2023-12-27 03:17:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 837558272. Throughput: 0: 9844.0, 1: 9370.6. Samples: 837542620. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:17:51,063][104569] Avg episode reward: [(0, '8445.255'), (1, '9171.521')] [2023-12-27 03:17:51,528][105620] Updated weights for policy 1, policy_version 1637218 (0.0010) [2023-12-27 03:17:51,560][105692] Updated weights for policy 0, policy_version 1634038 (0.0010) [2023-12-27 03:17:51,583][105620] Updated weights for policy 1, policy_version 1637228 (0.0005) [2023-12-27 03:17:51,623][105692] Updated weights for policy 0, policy_version 1634048 (0.0010) [2023-12-27 03:17:51,645][105620] Updated weights for policy 1, policy_version 1637238 (0.0008) [2023-12-27 03:17:51,681][105692] Updated weights for policy 0, policy_version 1634058 (0.0009) [2023-12-27 03:17:51,700][105620] Updated weights for policy 1, policy_version 1637248 (0.0006) [2023-12-27 03:17:52,365][105692] Updated weights for policy 0, policy_version 1634068 (0.0009) [2023-12-27 03:17:52,425][105692] Updated weights for policy 0, policy_version 1634078 (0.0009) [2023-12-27 03:17:52,463][105620] Updated weights for policy 1, policy_version 1637258 (0.0009) [2023-12-27 03:17:52,488][105692] Updated weights for policy 0, policy_version 1634088 (0.0008) [2023-12-27 03:17:52,510][105620] Updated weights for policy 1, policy_version 1637268 (0.0007) [2023-12-27 03:17:52,564][105620] Updated weights for policy 1, policy_version 1637278 (0.0008) [2023-12-27 03:17:53,225][105692] Updated weights for policy 0, policy_version 1634098 (0.0007) [2023-12-27 03:17:53,267][105620] Updated weights for policy 1, policy_version 1637288 (0.0006) [2023-12-27 03:17:53,282][105692] Updated weights for policy 0, policy_version 1634108 (0.0005) [2023-12-27 03:17:53,320][105620] Updated weights for policy 1, policy_version 1637298 (0.0007) [2023-12-27 03:17:53,345][105692] Updated weights for policy 0, policy_version 1634118 (0.0005) [2023-12-27 03:17:53,381][105620] Updated weights for policy 1, policy_version 1637308 (0.0009) [2023-12-27 03:17:53,396][105692] Updated weights for policy 0, policy_version 1634128 (0.0005) [2023-12-27 03:17:53,961][105620] Updated weights for policy 1, policy_version 1637318 (0.0007) [2023-12-27 03:17:54,016][105620] Updated weights for policy 1, policy_version 1637328 (0.0007) [2023-12-27 03:17:54,042][105692] Updated weights for policy 0, policy_version 1634138 (0.0007) [2023-12-27 03:17:54,076][105620] Updated weights for policy 1, policy_version 1637338 (0.0008) [2023-12-27 03:17:54,094][105692] Updated weights for policy 0, policy_version 1634148 (0.0006) [2023-12-27 03:17:54,156][105692] Updated weights for policy 0, policy_version 1634158 (0.0008) [2023-12-27 03:17:54,804][105620] Updated weights for policy 1, policy_version 1637348 (0.0006) [2023-12-27 03:17:54,811][105692] Updated weights for policy 0, policy_version 1634168 (0.0008) [2023-12-27 03:17:54,855][105620] Updated weights for policy 1, policy_version 1637358 (0.0006) [2023-12-27 03:17:54,874][105692] Updated weights for policy 0, policy_version 1634178 (0.0008) [2023-12-27 03:17:54,927][105620] Updated weights for policy 1, policy_version 1637368 (0.0007) [2023-12-27 03:17:54,940][105692] Updated weights for policy 0, policy_version 1634188 (0.0008) [2023-12-27 03:17:55,607][105620] Updated weights for policy 1, policy_version 1637378 (0.0011) [2023-12-27 03:17:55,651][105620] Updated weights for policy 1, policy_version 1637388 (0.0010) [2023-12-27 03:17:55,669][105692] Updated weights for policy 0, policy_version 1634198 (0.0007) [2023-12-27 03:17:55,703][105620] Updated weights for policy 1, policy_version 1637398 (0.0010) [2023-12-27 03:17:55,727][105692] Updated weights for policy 0, policy_version 1634208 (0.0006) [2023-12-27 03:17:55,768][105620] Updated weights for policy 1, policy_version 1637408 (0.0010) [2023-12-27 03:17:55,785][105692] Updated weights for policy 0, policy_version 1634218 (0.0006) [2023-12-27 03:17:56,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 837656576. Throughput: 0: 9857.7, 1: 9492.9. Samples: 837662648. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:17:56,063][104569] Avg episode reward: [(0, '8718.830'), (1, '9263.083')] [2023-12-27 03:17:56,403][105620] Updated weights for policy 1, policy_version 1637418 (0.0006) [2023-12-27 03:17:56,447][105620] Updated weights for policy 1, policy_version 1637428 (0.0009) [2023-12-27 03:17:56,505][105620] Updated weights for policy 1, policy_version 1637438 (0.0011) [2023-12-27 03:17:56,513][105692] Updated weights for policy 0, policy_version 1634228 (0.0009) [2023-12-27 03:17:56,571][105692] Updated weights for policy 0, policy_version 1634238 (0.0010) [2023-12-27 03:17:56,632][105692] Updated weights for policy 0, policy_version 1634248 (0.0010) [2023-12-27 03:17:57,208][105620] Updated weights for policy 1, policy_version 1637448 (0.0008) [2023-12-27 03:17:57,273][105620] Updated weights for policy 1, policy_version 1637458 (0.0008) [2023-12-27 03:17:57,279][105692] Updated weights for policy 0, policy_version 1634258 (0.0008) [2023-12-27 03:17:57,332][105692] Updated weights for policy 0, policy_version 1634268 (0.0009) [2023-12-27 03:17:57,334][105620] Updated weights for policy 1, policy_version 1637468 (0.0006) [2023-12-27 03:17:57,376][105692] Updated weights for policy 0, policy_version 1634278 (0.0010) [2023-12-27 03:17:57,420][105692] Updated weights for policy 0, policy_version 1634288 (0.0010) [2023-12-27 03:17:57,953][105620] Updated weights for policy 1, policy_version 1637478 (0.0007) [2023-12-27 03:17:58,018][105620] Updated weights for policy 1, policy_version 1637488 (0.0010) [2023-12-27 03:17:58,076][105620] Updated weights for policy 1, policy_version 1637498 (0.0008) [2023-12-27 03:17:58,118][105692] Updated weights for policy 0, policy_version 1634298 (0.0005) [2023-12-27 03:17:58,178][105692] Updated weights for policy 0, policy_version 1634308 (0.0006) [2023-12-27 03:17:58,240][105692] Updated weights for policy 0, policy_version 1634318 (0.0008) [2023-12-27 03:17:58,842][105620] Updated weights for policy 1, policy_version 1637508 (0.0006) [2023-12-27 03:17:58,903][105620] Updated weights for policy 1, policy_version 1637518 (0.0009) [2023-12-27 03:17:58,974][105620] Updated weights for policy 1, policy_version 1637528 (0.0008) [2023-12-27 03:17:59,035][105692] Updated weights for policy 0, policy_version 1634328 (0.0006) [2023-12-27 03:17:59,100][105692] Updated weights for policy 0, policy_version 1634338 (0.0007) [2023-12-27 03:17:59,159][105692] Updated weights for policy 0, policy_version 1634348 (0.0009) [2023-12-27 03:17:59,744][105620] Updated weights for policy 1, policy_version 1637538 (0.0008) [2023-12-27 03:17:59,811][105620] Updated weights for policy 1, policy_version 1637548 (0.0010) [2023-12-27 03:17:59,878][105620] Updated weights for policy 1, policy_version 1637558 (0.0007) [2023-12-27 03:17:59,902][105692] Updated weights for policy 0, policy_version 1634358 (0.0008) [2023-12-27 03:17:59,943][105620] Updated weights for policy 1, policy_version 1637568 (0.0006) [2023-12-27 03:17:59,963][105692] Updated weights for policy 0, policy_version 1634368 (0.0007) [2023-12-27 03:18:00,021][105692] Updated weights for policy 0, policy_version 1634378 (0.0007) [2023-12-27 03:18:00,644][105620] Updated weights for policy 1, policy_version 1637578 (0.0009) [2023-12-27 03:18:00,698][105620] Updated weights for policy 1, policy_version 1637588 (0.0009) [2023-12-27 03:18:00,730][105692] Updated weights for policy 0, policy_version 1634388 (0.0009) [2023-12-27 03:18:00,750][105620] Updated weights for policy 1, policy_version 1637598 (0.0008) [2023-12-27 03:18:00,787][105692] Updated weights for policy 0, policy_version 1634398 (0.0008) [2023-12-27 03:18:00,844][105692] Updated weights for policy 0, policy_version 1634408 (0.0009) [2023-12-27 03:18:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 837754880. Throughput: 0: 9906.6, 1: 9563.4. Samples: 837722712. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:18:01,063][104569] Avg episode reward: [(0, '8713.812'), (1, '9355.639')] [2023-12-27 03:18:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001637600_419282944.pth... [2023-12-27 03:18:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001634416_418471936.pth... [2023-12-27 03:18:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001636480_418996224.pth [2023-12-27 03:18:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001633232_418168832.pth [2023-12-27 03:18:01,450][105620] Updated weights for policy 1, policy_version 1637608 (0.0012) [2023-12-27 03:18:01,509][105620] Updated weights for policy 1, policy_version 1637618 (0.0006) [2023-12-27 03:18:01,569][105620] Updated weights for policy 1, policy_version 1637628 (0.0005) [2023-12-27 03:18:01,652][105692] Updated weights for policy 0, policy_version 1634419 (0.0009) [2023-12-27 03:18:01,707][105692] Updated weights for policy 0, policy_version 1634429 (0.0008) [2023-12-27 03:18:01,766][105692] Updated weights for policy 0, policy_version 1634439 (0.0008) [2023-12-27 03:18:02,228][105620] Updated weights for policy 1, policy_version 1637638 (0.0010) [2023-12-27 03:18:02,290][105620] Updated weights for policy 1, policy_version 1637648 (0.0009) [2023-12-27 03:18:02,358][105620] Updated weights for policy 1, policy_version 1637658 (0.0011) [2023-12-27 03:18:02,492][105692] Updated weights for policy 0, policy_version 1634449 (0.0008) [2023-12-27 03:18:02,554][105692] Updated weights for policy 0, policy_version 1634459 (0.0008) [2023-12-27 03:18:02,617][105692] Updated weights for policy 0, policy_version 1634469 (0.0008) [2023-12-27 03:18:02,682][105692] Updated weights for policy 0, policy_version 1634479 (0.0010) [2023-12-27 03:18:02,950][105620] Updated weights for policy 1, policy_version 1637668 (0.0008) [2023-12-27 03:18:02,995][105620] Updated weights for policy 1, policy_version 1637678 (0.0005) [2023-12-27 03:18:03,041][105620] Updated weights for policy 1, policy_version 1637688 (0.0005) [2023-12-27 03:18:03,561][105692] Updated weights for policy 0, policy_version 1634489 (0.0010) [2023-12-27 03:18:03,607][105620] Updated weights for policy 1, policy_version 1637698 (0.0005) [2023-12-27 03:18:03,618][105692] Updated weights for policy 0, policy_version 1634499 (0.0008) [2023-12-27 03:18:03,658][105620] Updated weights for policy 1, policy_version 1637708 (0.0005) [2023-12-27 03:18:03,674][105692] Updated weights for policy 0, policy_version 1634509 (0.0009) [2023-12-27 03:18:03,712][105620] Updated weights for policy 1, policy_version 1637718 (0.0005) [2023-12-27 03:18:03,766][105620] Updated weights for policy 1, policy_version 1637728 (0.0005) [2023-12-27 03:18:04,370][105620] Updated weights for policy 1, policy_version 1637738 (0.0005) [2023-12-27 03:18:04,429][105620] Updated weights for policy 1, policy_version 1637748 (0.0008) [2023-12-27 03:18:04,490][105620] Updated weights for policy 1, policy_version 1637758 (0.0008) [2023-12-27 03:18:04,503][105692] Updated weights for policy 0, policy_version 1634519 (0.0007) [2023-12-27 03:18:04,558][105692] Updated weights for policy 0, policy_version 1634529 (0.0008) [2023-12-27 03:18:04,619][105692] Updated weights for policy 0, policy_version 1634539 (0.0008) [2023-12-27 03:18:05,211][105620] Updated weights for policy 1, policy_version 1637768 (0.0010) [2023-12-27 03:18:05,262][105620] Updated weights for policy 1, policy_version 1637778 (0.0010) [2023-12-27 03:18:05,318][105620] Updated weights for policy 1, policy_version 1637788 (0.0011) [2023-12-27 03:18:05,379][105692] Updated weights for policy 0, policy_version 1634549 (0.0008) [2023-12-27 03:18:05,427][105692] Updated weights for policy 0, policy_version 1634559 (0.0008) [2023-12-27 03:18:05,483][105692] Updated weights for policy 0, policy_version 1634569 (0.0008) [2023-12-27 03:18:06,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 837844992. Throughput: 0: 9846.8, 1: 9705.3. Samples: 837838388. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:18:06,062][104569] Avg episode reward: [(0, '8346.406'), (1, '9178.510')] [2023-12-27 03:18:06,075][105692] Updated weights for policy 0, policy_version 1634579 (0.0008) [2023-12-27 03:18:06,077][105620] Updated weights for policy 1, policy_version 1637798 (0.0010) [2023-12-27 03:18:06,133][105692] Updated weights for policy 0, policy_version 1634589 (0.0009) [2023-12-27 03:18:06,133][105620] Updated weights for policy 1, policy_version 1637808 (0.0011) [2023-12-27 03:18:06,192][105692] Updated weights for policy 0, policy_version 1634599 (0.0010) [2023-12-27 03:18:06,194][105620] Updated weights for policy 1, policy_version 1637818 (0.0010) [2023-12-27 03:18:06,828][105620] Updated weights for policy 1, policy_version 1637828 (0.0010) [2023-12-27 03:18:06,882][105620] Updated weights for policy 1, policy_version 1637838 (0.0009) [2023-12-27 03:18:06,934][105620] Updated weights for policy 1, policy_version 1637848 (0.0009) [2023-12-27 03:18:07,018][105692] Updated weights for policy 0, policy_version 1634609 (0.0007) [2023-12-27 03:18:07,081][105692] Updated weights for policy 0, policy_version 1634619 (0.0009) [2023-12-27 03:18:07,144][105692] Updated weights for policy 0, policy_version 1634629 (0.0010) [2023-12-27 03:18:07,197][105692] Updated weights for policy 0, policy_version 1634639 (0.0010) [2023-12-27 03:18:07,605][105620] Updated weights for policy 1, policy_version 1637858 (0.0009) [2023-12-27 03:18:07,663][105620] Updated weights for policy 1, policy_version 1637868 (0.0009) [2023-12-27 03:18:07,725][105620] Updated weights for policy 1, policy_version 1637878 (0.0009) [2023-12-27 03:18:07,772][105620] Updated weights for policy 1, policy_version 1637888 (0.0008) [2023-12-27 03:18:07,958][105692] Updated weights for policy 0, policy_version 1634649 (0.0009) [2023-12-27 03:18:08,013][105692] Updated weights for policy 0, policy_version 1634659 (0.0009) [2023-12-27 03:18:08,071][105692] Updated weights for policy 0, policy_version 1634669 (0.0009) [2023-12-27 03:18:08,451][105620] Updated weights for policy 1, policy_version 1637898 (0.0008) [2023-12-27 03:18:08,509][105620] Updated weights for policy 1, policy_version 1637908 (0.0007) [2023-12-27 03:18:08,571][105620] Updated weights for policy 1, policy_version 1637918 (0.0008) [2023-12-27 03:18:08,850][105692] Updated weights for policy 0, policy_version 1634679 (0.0010) [2023-12-27 03:18:08,909][105692] Updated weights for policy 0, policy_version 1634689 (0.0011) [2023-12-27 03:18:08,974][105692] Updated weights for policy 0, policy_version 1634699 (0.0009) [2023-12-27 03:18:09,316][105620] Updated weights for policy 1, policy_version 1637928 (0.0007) [2023-12-27 03:18:09,381][105620] Updated weights for policy 1, policy_version 1637938 (0.0008) [2023-12-27 03:18:09,447][105620] Updated weights for policy 1, policy_version 1637948 (0.0009) [2023-12-27 03:18:09,663][105692] Updated weights for policy 0, policy_version 1634709 (0.0005) [2023-12-27 03:18:09,710][105692] Updated weights for policy 0, policy_version 1634719 (0.0006) [2023-12-27 03:18:09,760][105692] Updated weights for policy 0, policy_version 1634729 (0.0010) [2023-12-27 03:18:10,206][105620] Updated weights for policy 1, policy_version 1637958 (0.0010) [2023-12-27 03:18:10,252][105620] Updated weights for policy 1, policy_version 1637968 (0.0010) [2023-12-27 03:18:10,302][105620] Updated weights for policy 1, policy_version 1637978 (0.0010) [2023-12-27 03:18:10,462][105692] Updated weights for policy 0, policy_version 1634739 (0.0011) [2023-12-27 03:18:10,524][105692] Updated weights for policy 0, policy_version 1634749 (0.0008) [2023-12-27 03:18:10,585][105692] Updated weights for policy 0, policy_version 1634759 (0.0008) [2023-12-27 03:18:11,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19251.3, 300 sec: 19466.4). Total num frames: 837943296. Throughput: 0: 9813.6, 1: 9777.0. Samples: 837954688. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:18:11,062][104569] Avg episode reward: [(0, '8346.885'), (1, '9086.784')] [2023-12-27 03:18:11,083][105620] Updated weights for policy 1, policy_version 1637988 (0.0008) [2023-12-27 03:18:11,152][105620] Updated weights for policy 1, policy_version 1637998 (0.0008) [2023-12-27 03:18:11,221][105620] Updated weights for policy 1, policy_version 1638008 (0.0008) [2023-12-27 03:18:11,286][105692] Updated weights for policy 0, policy_version 1634769 (0.0011) [2023-12-27 03:18:11,351][105692] Updated weights for policy 0, policy_version 1634779 (0.0009) [2023-12-27 03:18:11,407][105692] Updated weights for policy 0, policy_version 1634789 (0.0009) [2023-12-27 03:18:11,465][105692] Updated weights for policy 0, policy_version 1634799 (0.0009) [2023-12-27 03:18:11,913][105620] Updated weights for policy 1, policy_version 1638018 (0.0008) [2023-12-27 03:18:11,978][105620] Updated weights for policy 1, policy_version 1638028 (0.0009) [2023-12-27 03:18:12,042][105620] Updated weights for policy 1, policy_version 1638038 (0.0008) [2023-12-27 03:18:12,093][105620] Updated weights for policy 1, policy_version 1638048 (0.0008) [2023-12-27 03:18:12,255][105692] Updated weights for policy 0, policy_version 1634809 (0.0008) [2023-12-27 03:18:12,325][105692] Updated weights for policy 0, policy_version 1634819 (0.0009) [2023-12-27 03:18:12,392][105692] Updated weights for policy 0, policy_version 1634829 (0.0008) [2023-12-27 03:18:12,918][105620] Updated weights for policy 1, policy_version 1638058 (0.0009) [2023-12-27 03:18:12,981][105620] Updated weights for policy 1, policy_version 1638068 (0.0008) [2023-12-27 03:18:13,047][105620] Updated weights for policy 1, policy_version 1638078 (0.0009) [2023-12-27 03:18:13,070][105692] Updated weights for policy 0, policy_version 1634839 (0.0006) [2023-12-27 03:18:13,135][105692] Updated weights for policy 0, policy_version 1634849 (0.0009) [2023-12-27 03:18:13,194][105692] Updated weights for policy 0, policy_version 1634859 (0.0009) [2023-12-27 03:18:13,769][105620] Updated weights for policy 1, policy_version 1638088 (0.0006) [2023-12-27 03:18:13,820][105620] Updated weights for policy 1, policy_version 1638098 (0.0005) [2023-12-27 03:18:13,872][105620] Updated weights for policy 1, policy_version 1638108 (0.0006) [2023-12-27 03:18:13,922][105692] Updated weights for policy 0, policy_version 1634869 (0.0009) [2023-12-27 03:18:13,981][105692] Updated weights for policy 0, policy_version 1634879 (0.0009) [2023-12-27 03:18:14,029][105692] Updated weights for policy 0, policy_version 1634889 (0.0009) [2023-12-27 03:18:14,591][105620] Updated weights for policy 1, policy_version 1638118 (0.0009) [2023-12-27 03:18:14,642][105620] Updated weights for policy 1, policy_version 1638128 (0.0009) [2023-12-27 03:18:14,701][105620] Updated weights for policy 1, policy_version 1638138 (0.0010) [2023-12-27 03:18:14,706][105692] Updated weights for policy 0, policy_version 1634899 (0.0008) [2023-12-27 03:18:14,782][105692] Updated weights for policy 0, policy_version 1634909 (0.0007) [2023-12-27 03:18:14,833][105692] Updated weights for policy 0, policy_version 1634919 (0.0009) [2023-12-27 03:18:15,525][105620] Updated weights for policy 1, policy_version 1638148 (0.0009) [2023-12-27 03:18:15,550][105692] Updated weights for policy 0, policy_version 1634929 (0.0008) [2023-12-27 03:18:15,581][105620] Updated weights for policy 1, policy_version 1638158 (0.0008) [2023-12-27 03:18:15,606][105692] Updated weights for policy 0, policy_version 1634939 (0.0009) [2023-12-27 03:18:15,630][105620] Updated weights for policy 1, policy_version 1638168 (0.0008) [2023-12-27 03:18:15,657][105692] Updated weights for policy 0, policy_version 1634949 (0.0007) [2023-12-27 03:18:15,722][105692] Updated weights for policy 0, policy_version 1634959 (0.0008) [2023-12-27 03:18:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.4, 300 sec: 19466.4). Total num frames: 838041600. Throughput: 0: 9762.7, 1: 9770.2. Samples: 838011060. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:18:16,062][104569] Avg episode reward: [(0, '8898.693'), (1, '8989.855')] [2023-12-27 03:18:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001638176_419430400.pth... [2023-12-27 03:18:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001634960_418611200.pth... [2023-12-27 03:18:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001633808_418316288.pth [2023-12-27 03:18:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001637024_419135488.pth [2023-12-27 03:18:16,305][105692] Updated weights for policy 0, policy_version 1634969 (0.0009) [2023-12-27 03:18:16,357][105692] Updated weights for policy 0, policy_version 1634979 (0.0009) [2023-12-27 03:18:16,418][105692] Updated weights for policy 0, policy_version 1634989 (0.0008) [2023-12-27 03:18:16,469][105620] Updated weights for policy 1, policy_version 1638178 (0.0008) [2023-12-27 03:18:16,516][105620] Updated weights for policy 1, policy_version 1638188 (0.0007) [2023-12-27 03:18:16,567][105620] Updated weights for policy 1, policy_version 1638198 (0.0007) [2023-12-27 03:18:16,623][105620] Updated weights for policy 1, policy_version 1638208 (0.0010) [2023-12-27 03:18:17,247][105692] Updated weights for policy 0, policy_version 1634999 (0.0006) [2023-12-27 03:18:17,257][105620] Updated weights for policy 1, policy_version 1638218 (0.0010) [2023-12-27 03:18:17,303][105692] Updated weights for policy 0, policy_version 1635009 (0.0008) [2023-12-27 03:18:17,322][105620] Updated weights for policy 1, policy_version 1638228 (0.0010) [2023-12-27 03:18:17,356][105692] Updated weights for policy 0, policy_version 1635019 (0.0006) [2023-12-27 03:18:17,373][105620] Updated weights for policy 1, policy_version 1638238 (0.0010) [2023-12-27 03:18:17,965][105692] Updated weights for policy 0, policy_version 1635029 (0.0006) [2023-12-27 03:18:18,019][105692] Updated weights for policy 0, policy_version 1635039 (0.0005) [2023-12-27 03:18:18,067][105692] Updated weights for policy 0, policy_version 1635049 (0.0005) [2023-12-27 03:18:18,074][105620] Updated weights for policy 1, policy_version 1638248 (0.0010) [2023-12-27 03:18:18,126][105620] Updated weights for policy 1, policy_version 1638258 (0.0010) [2023-12-27 03:18:18,188][105620] Updated weights for policy 1, policy_version 1638268 (0.0010) [2023-12-27 03:18:18,739][105692] Updated weights for policy 0, policy_version 1635059 (0.0006) [2023-12-27 03:18:18,808][105692] Updated weights for policy 0, policy_version 1635069 (0.0008) [2023-12-27 03:18:18,863][105620] Updated weights for policy 1, policy_version 1638278 (0.0011) [2023-12-27 03:18:18,876][105692] Updated weights for policy 0, policy_version 1635079 (0.0008) [2023-12-27 03:18:18,926][105620] Updated weights for policy 1, policy_version 1638288 (0.0011) [2023-12-27 03:18:18,986][105620] Updated weights for policy 1, policy_version 1638298 (0.0011) [2023-12-27 03:18:19,590][105692] Updated weights for policy 0, policy_version 1635089 (0.0008) [2023-12-27 03:18:19,655][105692] Updated weights for policy 0, policy_version 1635099 (0.0008) [2023-12-27 03:18:19,669][105620] Updated weights for policy 1, policy_version 1638308 (0.0011) [2023-12-27 03:18:19,715][105692] Updated weights for policy 0, policy_version 1635109 (0.0006) [2023-12-27 03:18:19,728][105620] Updated weights for policy 1, policy_version 1638318 (0.0010) [2023-12-27 03:18:19,775][105692] Updated weights for policy 0, policy_version 1635119 (0.0006) [2023-12-27 03:18:19,780][105620] Updated weights for policy 1, policy_version 1638328 (0.0010) [2023-12-27 03:18:20,509][105692] Updated weights for policy 0, policy_version 1635129 (0.0010) [2023-12-27 03:18:20,520][105620] Updated weights for policy 1, policy_version 1638338 (0.0007) [2023-12-27 03:18:20,576][105692] Updated weights for policy 0, policy_version 1635139 (0.0011) [2023-12-27 03:18:20,590][105620] Updated weights for policy 1, policy_version 1638348 (0.0007) [2023-12-27 03:18:20,640][105692] Updated weights for policy 0, policy_version 1635149 (0.0008) [2023-12-27 03:18:20,656][105620] Updated weights for policy 1, policy_version 1638358 (0.0007) [2023-12-27 03:18:20,712][105620] Updated weights for policy 1, policy_version 1638368 (0.0005) [2023-12-27 03:18:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 838139904. Throughput: 0: 9874.0, 1: 9748.7. Samples: 838129404. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:18:21,062][104569] Avg episode reward: [(0, '8988.035'), (1, '8989.516')] [2023-12-27 03:18:21,360][105692] Updated weights for policy 0, policy_version 1635159 (0.0007) [2023-12-27 03:18:21,418][105620] Updated weights for policy 1, policy_version 1638378 (0.0009) [2023-12-27 03:18:21,423][105692] Updated weights for policy 0, policy_version 1635169 (0.0007) [2023-12-27 03:18:21,474][105692] Updated weights for policy 0, policy_version 1635179 (0.0008) [2023-12-27 03:18:21,485][105620] Updated weights for policy 1, policy_version 1638388 (0.0006) [2023-12-27 03:18:21,556][105620] Updated weights for policy 1, policy_version 1638398 (0.0008) [2023-12-27 03:18:22,155][105692] Updated weights for policy 0, policy_version 1635189 (0.0006) [2023-12-27 03:18:22,217][105692] Updated weights for policy 0, policy_version 1635199 (0.0010) [2023-12-27 03:18:22,281][105692] Updated weights for policy 0, policy_version 1635209 (0.0008) [2023-12-27 03:18:22,294][105620] Updated weights for policy 1, policy_version 1638408 (0.0007) [2023-12-27 03:18:22,361][105620] Updated weights for policy 1, policy_version 1638418 (0.0008) [2023-12-27 03:18:22,420][105620] Updated weights for policy 1, policy_version 1638428 (0.0009) [2023-12-27 03:18:23,047][105692] Updated weights for policy 0, policy_version 1635219 (0.0009) [2023-12-27 03:18:23,106][105692] Updated weights for policy 0, policy_version 1635229 (0.0009) [2023-12-27 03:18:23,165][105692] Updated weights for policy 0, policy_version 1635239 (0.0007) [2023-12-27 03:18:23,175][105620] Updated weights for policy 1, policy_version 1638438 (0.0009) [2023-12-27 03:18:23,231][105620] Updated weights for policy 1, policy_version 1638448 (0.0009) [2023-12-27 03:18:23,284][105620] Updated weights for policy 1, policy_version 1638458 (0.0010) [2023-12-27 03:18:23,725][105692] Updated weights for policy 0, policy_version 1635249 (0.0005) [2023-12-27 03:18:23,787][105692] Updated weights for policy 0, policy_version 1635259 (0.0006) [2023-12-27 03:18:23,844][105692] Updated weights for policy 0, policy_version 1635269 (0.0009) [2023-12-27 03:18:23,897][105692] Updated weights for policy 0, policy_version 1635279 (0.0009) [2023-12-27 03:18:24,109][105620] Updated weights for policy 1, policy_version 1638468 (0.0009) [2023-12-27 03:18:24,167][105620] Updated weights for policy 1, policy_version 1638478 (0.0009) [2023-12-27 03:18:24,216][105620] Updated weights for policy 1, policy_version 1638488 (0.0008) [2023-12-27 03:18:24,630][105692] Updated weights for policy 0, policy_version 1635289 (0.0006) [2023-12-27 03:18:24,684][105692] Updated weights for policy 0, policy_version 1635299 (0.0005) [2023-12-27 03:18:24,738][105692] Updated weights for policy 0, policy_version 1635309 (0.0009) [2023-12-27 03:18:24,993][105620] Updated weights for policy 1, policy_version 1638498 (0.0009) [2023-12-27 03:18:25,053][105620] Updated weights for policy 1, policy_version 1638508 (0.0010) [2023-12-27 03:18:25,115][105620] Updated weights for policy 1, policy_version 1638518 (0.0010) [2023-12-27 03:18:25,176][105620] Updated weights for policy 1, policy_version 1638528 (0.0006) [2023-12-27 03:18:25,468][105692] Updated weights for policy 0, policy_version 1635319 (0.0010) [2023-12-27 03:18:25,529][105692] Updated weights for policy 0, policy_version 1635329 (0.0010) [2023-12-27 03:18:25,594][105692] Updated weights for policy 0, policy_version 1635339 (0.0008) [2023-12-27 03:18:25,740][105620] Updated weights for policy 1, policy_version 1638538 (0.0005) [2023-12-27 03:18:25,796][105620] Updated weights for policy 1, policy_version 1638548 (0.0005) [2023-12-27 03:18:25,852][105620] Updated weights for policy 1, policy_version 1638558 (0.0005) [2023-12-27 03:18:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 838238208. Throughput: 0: 9778.0, 1: 9736.7. Samples: 838245404. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:18:26,062][104569] Avg episode reward: [(0, '8803.733'), (1, '8994.464')] [2023-12-27 03:18:26,379][105620] Updated weights for policy 1, policy_version 1638568 (0.0007) [2023-12-27 03:18:26,410][105692] Updated weights for policy 0, policy_version 1635349 (0.0007) [2023-12-27 03:18:26,434][105620] Updated weights for policy 1, policy_version 1638578 (0.0006) [2023-12-27 03:18:26,470][105692] Updated weights for policy 0, policy_version 1635359 (0.0009) [2023-12-27 03:18:26,492][105620] Updated weights for policy 1, policy_version 1638588 (0.0005) [2023-12-27 03:18:26,527][105692] Updated weights for policy 0, policy_version 1635369 (0.0009) [2023-12-27 03:18:27,147][105620] Updated weights for policy 1, policy_version 1638598 (0.0008) [2023-12-27 03:18:27,199][105620] Updated weights for policy 1, policy_version 1638609 (0.0007) [2023-12-27 03:18:27,231][105692] Updated weights for policy 0, policy_version 1635379 (0.0008) [2023-12-27 03:18:27,246][105620] Updated weights for policy 1, policy_version 1638619 (0.0005) [2023-12-27 03:18:27,289][105692] Updated weights for policy 0, policy_version 1635389 (0.0005) [2023-12-27 03:18:27,350][105692] Updated weights for policy 0, policy_version 1635400 (0.0007) [2023-12-27 03:18:27,868][105620] Updated weights for policy 1, policy_version 1638629 (0.0005) [2023-12-27 03:18:27,916][105620] Updated weights for policy 1, policy_version 1638639 (0.0005) [2023-12-27 03:18:27,974][105620] Updated weights for policy 1, policy_version 1638649 (0.0005) [2023-12-27 03:18:28,115][105692] Updated weights for policy 0, policy_version 1635410 (0.0010) [2023-12-27 03:18:28,166][105692] Updated weights for policy 0, policy_version 1635420 (0.0008) [2023-12-27 03:18:28,213][105692] Updated weights for policy 0, policy_version 1635430 (0.0009) [2023-12-27 03:18:28,259][105692] Updated weights for policy 0, policy_version 1635440 (0.0009) [2023-12-27 03:18:28,613][105620] Updated weights for policy 1, policy_version 1638659 (0.0007) [2023-12-27 03:18:28,669][105620] Updated weights for policy 1, policy_version 1638669 (0.0009) [2023-12-27 03:18:28,721][105620] Updated weights for policy 1, policy_version 1638679 (0.0009) [2023-12-27 03:18:29,094][105692] Updated weights for policy 0, policy_version 1635450 (0.0009) [2023-12-27 03:18:29,146][105692] Updated weights for policy 0, policy_version 1635460 (0.0008) [2023-12-27 03:18:29,198][105692] Updated weights for policy 0, policy_version 1635470 (0.0008) [2023-12-27 03:18:29,397][105620] Updated weights for policy 1, policy_version 1638689 (0.0009) [2023-12-27 03:18:29,459][105620] Updated weights for policy 1, policy_version 1638699 (0.0005) [2023-12-27 03:18:29,528][105620] Updated weights for policy 1, policy_version 1638709 (0.0006) [2023-12-27 03:18:29,599][105620] Updated weights for policy 1, policy_version 1638719 (0.0007) [2023-12-27 03:18:29,996][105692] Updated weights for policy 0, policy_version 1635480 (0.0007) [2023-12-27 03:18:30,058][105692] Updated weights for policy 0, policy_version 1635490 (0.0009) [2023-12-27 03:18:30,124][105692] Updated weights for policy 0, policy_version 1635500 (0.0009) [2023-12-27 03:18:30,300][105620] Updated weights for policy 1, policy_version 1638729 (0.0008) [2023-12-27 03:18:30,354][105620] Updated weights for policy 1, policy_version 1638739 (0.0008) [2023-12-27 03:18:30,409][105620] Updated weights for policy 1, policy_version 1638749 (0.0009) [2023-12-27 03:18:30,943][105692] Updated weights for policy 0, policy_version 1635510 (0.0010) [2023-12-27 03:18:30,980][105620] Updated weights for policy 1, policy_version 1638759 (0.0008) [2023-12-27 03:18:30,990][105692] Updated weights for policy 0, policy_version 1635520 (0.0007) [2023-12-27 03:18:31,046][105620] Updated weights for policy 1, policy_version 1638769 (0.0009) [2023-12-27 03:18:31,048][105692] Updated weights for policy 0, policy_version 1635530 (0.0006) [2023-12-27 03:18:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 838328320. Throughput: 0: 9800.9, 1: 9807.0. Samples: 838306020. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:18:31,062][104569] Avg episode reward: [(0, '8806.064'), (1, '8903.215')] [2023-12-27 03:18:31,084][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001635536_418758656.pth... [2023-12-27 03:18:31,087][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001634416_418471936.pth [2023-12-27 03:18:31,100][105620] Updated weights for policy 1, policy_version 1638779 (0.0007) [2023-12-27 03:18:31,126][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001638784_419586048.pth... [2023-12-27 03:18:31,130][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001637600_419282944.pth [2023-12-27 03:18:31,798][105692] Updated weights for policy 0, policy_version 1635540 (0.0008) [2023-12-27 03:18:31,851][105692] Updated weights for policy 0, policy_version 1635550 (0.0010) [2023-12-27 03:18:31,881][105620] Updated weights for policy 1, policy_version 1638789 (0.0008) [2023-12-27 03:18:31,903][105692] Updated weights for policy 0, policy_version 1635560 (0.0010) [2023-12-27 03:18:31,938][105620] Updated weights for policy 1, policy_version 1638799 (0.0006) [2023-12-27 03:18:31,999][105620] Updated weights for policy 1, policy_version 1638809 (0.0007) [2023-12-27 03:18:32,568][105692] Updated weights for policy 0, policy_version 1635570 (0.0010) [2023-12-27 03:18:32,632][105692] Updated weights for policy 0, policy_version 1635580 (0.0010) [2023-12-27 03:18:32,683][105692] Updated weights for policy 0, policy_version 1635590 (0.0007) [2023-12-27 03:18:32,742][105692] Updated weights for policy 0, policy_version 1635600 (0.0005) [2023-12-27 03:18:32,760][105620] Updated weights for policy 1, policy_version 1638819 (0.0009) [2023-12-27 03:18:32,818][105620] Updated weights for policy 1, policy_version 1638829 (0.0010) [2023-12-27 03:18:32,880][105620] Updated weights for policy 1, policy_version 1638840 (0.0010) [2023-12-27 03:18:33,393][105692] Updated weights for policy 0, policy_version 1635610 (0.0011) [2023-12-27 03:18:33,452][105692] Updated weights for policy 0, policy_version 1635620 (0.0010) [2023-12-27 03:18:33,514][105692] Updated weights for policy 0, policy_version 1635630 (0.0011) [2023-12-27 03:18:33,604][105620] Updated weights for policy 1, policy_version 1638850 (0.0009) [2023-12-27 03:18:33,673][105620] Updated weights for policy 1, policy_version 1638860 (0.0009) [2023-12-27 03:18:33,729][105620] Updated weights for policy 1, policy_version 1638870 (0.0010) [2023-12-27 03:18:33,787][105620] Updated weights for policy 1, policy_version 1638880 (0.0010) [2023-12-27 03:18:34,123][105692] Updated weights for policy 0, policy_version 1635640 (0.0009) [2023-12-27 03:18:34,190][105692] Updated weights for policy 0, policy_version 1635650 (0.0009) [2023-12-27 03:18:34,253][105692] Updated weights for policy 0, policy_version 1635660 (0.0005) [2023-12-27 03:18:34,627][105620] Updated weights for policy 1, policy_version 1638890 (0.0008) [2023-12-27 03:18:34,679][105620] Updated weights for policy 1, policy_version 1638900 (0.0008) [2023-12-27 03:18:34,731][105620] Updated weights for policy 1, policy_version 1638910 (0.0008) [2023-12-27 03:18:34,995][105692] Updated weights for policy 0, policy_version 1635670 (0.0009) [2023-12-27 03:18:35,054][105692] Updated weights for policy 0, policy_version 1635680 (0.0011) [2023-12-27 03:18:35,099][105692] Updated weights for policy 0, policy_version 1635690 (0.0010) [2023-12-27 03:18:35,483][105620] Updated weights for policy 1, policy_version 1638920 (0.0006) [2023-12-27 03:18:35,539][105620] Updated weights for policy 1, policy_version 1638930 (0.0005) [2023-12-27 03:18:35,601][105620] Updated weights for policy 1, policy_version 1638940 (0.0005) [2023-12-27 03:18:35,771][105692] Updated weights for policy 0, policy_version 1635700 (0.0010) [2023-12-27 03:18:35,826][105692] Updated weights for policy 0, policy_version 1635710 (0.0010) [2023-12-27 03:18:35,888][105692] Updated weights for policy 0, policy_version 1635720 (0.0010) [2023-12-27 03:18:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 838434816. Throughput: 0: 9660.3, 1: 9871.3. Samples: 838421540. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:18:36,062][104569] Avg episode reward: [(0, '8717.889'), (1, '8989.740')] [2023-12-27 03:18:36,183][105620] Updated weights for policy 1, policy_version 1638950 (0.0008) [2023-12-27 03:18:36,239][105620] Updated weights for policy 1, policy_version 1638960 (0.0010) [2023-12-27 03:18:36,287][105620] Updated weights for policy 1, policy_version 1638970 (0.0010) [2023-12-27 03:18:36,614][105692] Updated weights for policy 0, policy_version 1635730 (0.0010) [2023-12-27 03:18:36,665][105692] Updated weights for policy 0, policy_version 1635740 (0.0011) [2023-12-27 03:18:36,721][105692] Updated weights for policy 0, policy_version 1635750 (0.0011) [2023-12-27 03:18:36,778][105692] Updated weights for policy 0, policy_version 1635760 (0.0011) [2023-12-27 03:18:36,983][105620] Updated weights for policy 1, policy_version 1638980 (0.0009) [2023-12-27 03:18:37,044][105620] Updated weights for policy 1, policy_version 1638990 (0.0011) [2023-12-27 03:18:37,110][105620] Updated weights for policy 1, policy_version 1639000 (0.0010) [2023-12-27 03:18:37,445][105692] Updated weights for policy 0, policy_version 1635770 (0.0006) [2023-12-27 03:18:37,497][105692] Updated weights for policy 0, policy_version 1635780 (0.0006) [2023-12-27 03:18:37,554][105692] Updated weights for policy 0, policy_version 1635790 (0.0008) [2023-12-27 03:18:37,754][105620] Updated weights for policy 1, policy_version 1639010 (0.0008) [2023-12-27 03:18:37,807][105620] Updated weights for policy 1, policy_version 1639020 (0.0008) [2023-12-27 03:18:37,868][105620] Updated weights for policy 1, policy_version 1639030 (0.0008) [2023-12-27 03:18:37,927][105620] Updated weights for policy 1, policy_version 1639040 (0.0007) [2023-12-27 03:18:38,203][105692] Updated weights for policy 0, policy_version 1635800 (0.0011) [2023-12-27 03:18:38,251][105692] Updated weights for policy 0, policy_version 1635810 (0.0010) [2023-12-27 03:18:38,299][105692] Updated weights for policy 0, policy_version 1635820 (0.0010) [2023-12-27 03:18:38,611][105620] Updated weights for policy 1, policy_version 1639050 (0.0008) [2023-12-27 03:18:38,674][105620] Updated weights for policy 1, policy_version 1639060 (0.0008) [2023-12-27 03:18:38,734][105620] Updated weights for policy 1, policy_version 1639070 (0.0008) [2023-12-27 03:18:39,013][105692] Updated weights for policy 0, policy_version 1635830 (0.0009) [2023-12-27 03:18:39,070][105692] Updated weights for policy 0, policy_version 1635840 (0.0008) [2023-12-27 03:18:39,126][105692] Updated weights for policy 0, policy_version 1635850 (0.0011) [2023-12-27 03:18:39,465][105620] Updated weights for policy 1, policy_version 1639080 (0.0007) [2023-12-27 03:18:39,529][105620] Updated weights for policy 1, policy_version 1639090 (0.0008) [2023-12-27 03:18:39,587][105620] Updated weights for policy 1, policy_version 1639100 (0.0007) [2023-12-27 03:18:39,864][105692] Updated weights for policy 0, policy_version 1635860 (0.0010) [2023-12-27 03:18:39,932][105692] Updated weights for policy 0, policy_version 1635870 (0.0010) [2023-12-27 03:18:39,993][105692] Updated weights for policy 0, policy_version 1635880 (0.0009) [2023-12-27 03:18:40,385][105620] Updated weights for policy 1, policy_version 1639110 (0.0007) [2023-12-27 03:18:40,439][105620] Updated weights for policy 1, policy_version 1639120 (0.0005) [2023-12-27 03:18:40,485][105620] Updated weights for policy 1, policy_version 1639130 (0.0005) [2023-12-27 03:18:40,676][105692] Updated weights for policy 0, policy_version 1635890 (0.0007) [2023-12-27 03:18:40,737][105692] Updated weights for policy 0, policy_version 1635900 (0.0010) [2023-12-27 03:18:40,800][105692] Updated weights for policy 0, policy_version 1635910 (0.0008) [2023-12-27 03:18:40,859][105692] Updated weights for policy 0, policy_version 1635920 (0.0009) [2023-12-27 03:18:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 838533120. Throughput: 0: 9662.0, 1: 9900.9. Samples: 838542972. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:18:41,062][104569] Avg episode reward: [(0, '8900.159'), (1, '8989.488')] [2023-12-27 03:18:41,156][105620] Updated weights for policy 1, policy_version 1639140 (0.0007) [2023-12-27 03:18:41,216][105620] Updated weights for policy 1, policy_version 1639150 (0.0009) [2023-12-27 03:18:41,281][105620] Updated weights for policy 1, policy_version 1639160 (0.0010) [2023-12-27 03:18:41,523][105692] Updated weights for policy 0, policy_version 1635930 (0.0009) [2023-12-27 03:18:41,585][105692] Updated weights for policy 0, policy_version 1635940 (0.0008) [2023-12-27 03:18:41,650][105692] Updated weights for policy 0, policy_version 1635950 (0.0008) [2023-12-27 03:18:42,064][105620] Updated weights for policy 1, policy_version 1639170 (0.0009) [2023-12-27 03:18:42,120][105620] Updated weights for policy 1, policy_version 1639180 (0.0008) [2023-12-27 03:18:42,188][105620] Updated weights for policy 1, policy_version 1639190 (0.0008) [2023-12-27 03:18:42,251][105620] Updated weights for policy 1, policy_version 1639200 (0.0008) [2023-12-27 03:18:42,410][105692] Updated weights for policy 0, policy_version 1635960 (0.0009) [2023-12-27 03:18:42,474][105692] Updated weights for policy 0, policy_version 1635970 (0.0009) [2023-12-27 03:18:42,538][105692] Updated weights for policy 0, policy_version 1635980 (0.0008) [2023-12-27 03:18:42,931][105620] Updated weights for policy 1, policy_version 1639210 (0.0009) [2023-12-27 03:18:42,981][105620] Updated weights for policy 1, policy_version 1639221 (0.0008) [2023-12-27 03:18:43,041][105620] Updated weights for policy 1, policy_version 1639231 (0.0008) [2023-12-27 03:18:43,319][105692] Updated weights for policy 0, policy_version 1635990 (0.0010) [2023-12-27 03:18:43,361][105692] Updated weights for policy 0, policy_version 1636000 (0.0010) [2023-12-27 03:18:43,413][105692] Updated weights for policy 0, policy_version 1636010 (0.0010) [2023-12-27 03:18:43,790][105620] Updated weights for policy 1, policy_version 1639241 (0.0008) [2023-12-27 03:18:43,841][105620] Updated weights for policy 1, policy_version 1639251 (0.0008) [2023-12-27 03:18:43,894][105620] Updated weights for policy 1, policy_version 1639261 (0.0008) [2023-12-27 03:18:44,109][105692] Updated weights for policy 0, policy_version 1636020 (0.0010) [2023-12-27 03:18:44,162][105692] Updated weights for policy 0, policy_version 1636030 (0.0010) [2023-12-27 03:18:44,217][105692] Updated weights for policy 0, policy_version 1636040 (0.0010) [2023-12-27 03:18:44,702][105620] Updated weights for policy 1, policy_version 1639271 (0.0009) [2023-12-27 03:18:44,752][105620] Updated weights for policy 1, policy_version 1639281 (0.0009) [2023-12-27 03:18:44,815][105620] Updated weights for policy 1, policy_version 1639291 (0.0009) [2023-12-27 03:18:44,964][105692] Updated weights for policy 0, policy_version 1636050 (0.0009) [2023-12-27 03:18:45,024][105692] Updated weights for policy 0, policy_version 1636060 (0.0007) [2023-12-27 03:18:45,085][105692] Updated weights for policy 0, policy_version 1636070 (0.0006) [2023-12-27 03:18:45,142][105692] Updated weights for policy 0, policy_version 1636080 (0.0006) [2023-12-27 03:18:45,610][105620] Updated weights for policy 1, policy_version 1639301 (0.0009) [2023-12-27 03:18:45,657][105620] Updated weights for policy 1, policy_version 1639311 (0.0007) [2023-12-27 03:18:45,725][105620] Updated weights for policy 1, policy_version 1639321 (0.0010) [2023-12-27 03:18:45,797][105692] Updated weights for policy 0, policy_version 1636090 (0.0005) [2023-12-27 03:18:45,846][105692] Updated weights for policy 0, policy_version 1636100 (0.0005) [2023-12-27 03:18:45,899][105692] Updated weights for policy 0, policy_version 1636110 (0.0005) [2023-12-27 03:18:46,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19660.7, 300 sec: 19494.2). Total num frames: 838631424. Throughput: 0: 9621.7, 1: 9839.0. Samples: 838598448. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:18:46,063][104569] Avg episode reward: [(0, '9082.804'), (1, '8988.863')] [2023-12-27 03:18:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001636112_418906112.pth... [2023-12-27 03:18:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001639328_419725312.pth... [2023-12-27 03:18:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001634960_418611200.pth [2023-12-27 03:18:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001638176_419430400.pth [2023-12-27 03:18:46,529][105620] Updated weights for policy 1, policy_version 1639331 (0.0009) [2023-12-27 03:18:46,544][105692] Updated weights for policy 0, policy_version 1636120 (0.0006) [2023-12-27 03:18:46,584][105620] Updated weights for policy 1, policy_version 1639341 (0.0007) [2023-12-27 03:18:46,602][105692] Updated weights for policy 0, policy_version 1636130 (0.0008) [2023-12-27 03:18:46,644][105620] Updated weights for policy 1, policy_version 1639351 (0.0007) [2023-12-27 03:18:46,662][105692] Updated weights for policy 0, policy_version 1636140 (0.0006) [2023-12-27 03:18:47,380][105620] Updated weights for policy 1, policy_version 1639361 (0.0007) [2023-12-27 03:18:47,416][105692] Updated weights for policy 0, policy_version 1636150 (0.0006) [2023-12-27 03:18:47,436][105620] Updated weights for policy 1, policy_version 1639371 (0.0009) [2023-12-27 03:18:47,468][105692] Updated weights for policy 0, policy_version 1636160 (0.0005) [2023-12-27 03:18:47,495][105620] Updated weights for policy 1, policy_version 1639381 (0.0005) [2023-12-27 03:18:47,525][105692] Updated weights for policy 0, policy_version 1636170 (0.0005) [2023-12-27 03:18:47,552][105620] Updated weights for policy 1, policy_version 1639391 (0.0006) [2023-12-27 03:18:48,243][105692] Updated weights for policy 0, policy_version 1636180 (0.0007) [2023-12-27 03:18:48,280][105620] Updated weights for policy 1, policy_version 1639401 (0.0006) [2023-12-27 03:18:48,290][105692] Updated weights for policy 0, policy_version 1636190 (0.0007) [2023-12-27 03:18:48,332][105620] Updated weights for policy 1, policy_version 1639411 (0.0007) [2023-12-27 03:18:48,345][105692] Updated weights for policy 0, policy_version 1636200 (0.0007) [2023-12-27 03:18:48,395][105620] Updated weights for policy 1, policy_version 1639421 (0.0008) [2023-12-27 03:18:49,056][105620] Updated weights for policy 1, policy_version 1639431 (0.0009) [2023-12-27 03:18:49,119][105620] Updated weights for policy 1, policy_version 1639441 (0.0009) [2023-12-27 03:18:49,158][105692] Updated weights for policy 0, policy_version 1636210 (0.0009) [2023-12-27 03:18:49,181][105620] Updated weights for policy 1, policy_version 1639451 (0.0009) [2023-12-27 03:18:49,226][105692] Updated weights for policy 0, policy_version 1636220 (0.0006) [2023-12-27 03:18:49,285][105692] Updated weights for policy 0, policy_version 1636230 (0.0010) [2023-12-27 03:18:49,350][105692] Updated weights for policy 0, policy_version 1636240 (0.0008) [2023-12-27 03:18:49,943][105620] Updated weights for policy 1, policy_version 1639461 (0.0008) [2023-12-27 03:18:50,008][105620] Updated weights for policy 1, policy_version 1639471 (0.0009) [2023-12-27 03:18:50,067][105620] Updated weights for policy 1, policy_version 1639481 (0.0009) [2023-12-27 03:18:50,105][105692] Updated weights for policy 0, policy_version 1636250 (0.0006) [2023-12-27 03:18:50,162][105692] Updated weights for policy 0, policy_version 1636260 (0.0009) [2023-12-27 03:18:50,224][105692] Updated weights for policy 0, policy_version 1636270 (0.0009) [2023-12-27 03:18:50,823][105620] Updated weights for policy 1, policy_version 1639491 (0.0007) [2023-12-27 03:18:50,882][105620] Updated weights for policy 1, policy_version 1639501 (0.0009) [2023-12-27 03:18:50,938][105620] Updated weights for policy 1, policy_version 1639511 (0.0009) [2023-12-27 03:18:50,973][105692] Updated weights for policy 0, policy_version 1636280 (0.0006) [2023-12-27 03:18:51,039][105692] Updated weights for policy 0, policy_version 1636290 (0.0009) [2023-12-27 03:18:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 838721536. Throughput: 0: 9727.4, 1: 9703.5. Samples: 838712784. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:18:51,063][104569] Avg episode reward: [(0, '8627.604'), (1, '8896.065')] [2023-12-27 03:18:51,098][105692] Updated weights for policy 0, policy_version 1636300 (0.0009) [2023-12-27 03:18:51,644][105620] Updated weights for policy 1, policy_version 1639521 (0.0006) [2023-12-27 03:18:51,707][105620] Updated weights for policy 1, policy_version 1639531 (0.0009) [2023-12-27 03:18:51,780][105620] Updated weights for policy 1, policy_version 1639541 (0.0010) [2023-12-27 03:18:51,833][105620] Updated weights for policy 1, policy_version 1639551 (0.0009) [2023-12-27 03:18:51,833][105586] KL-divergence is very high: 107.3709 [2023-12-27 03:18:51,906][105692] Updated weights for policy 0, policy_version 1636310 (0.0009) [2023-12-27 03:18:51,964][105692] Updated weights for policy 0, policy_version 1636320 (0.0009) [2023-12-27 03:18:52,020][105692] Updated weights for policy 0, policy_version 1636330 (0.0010) [2023-12-27 03:18:52,564][105620] Updated weights for policy 1, policy_version 1639561 (0.0006) [2023-12-27 03:18:52,616][105620] Updated weights for policy 1, policy_version 1639571 (0.0006) [2023-12-27 03:18:52,661][105620] Updated weights for policy 1, policy_version 1639581 (0.0010) [2023-12-27 03:18:52,832][105692] Updated weights for policy 0, policy_version 1636340 (0.0008) [2023-12-27 03:18:52,890][105692] Updated weights for policy 0, policy_version 1636350 (0.0007) [2023-12-27 03:18:52,951][105692] Updated weights for policy 0, policy_version 1636360 (0.0008) [2023-12-27 03:18:53,398][105620] Updated weights for policy 1, policy_version 1639591 (0.0007) [2023-12-27 03:18:53,462][105620] Updated weights for policy 1, policy_version 1639601 (0.0005) [2023-12-27 03:18:53,526][105620] Updated weights for policy 1, policy_version 1639611 (0.0006) [2023-12-27 03:18:53,686][105692] Updated weights for policy 0, policy_version 1636370 (0.0007) [2023-12-27 03:18:53,748][105692] Updated weights for policy 0, policy_version 1636380 (0.0005) [2023-12-27 03:18:53,805][105692] Updated weights for policy 0, policy_version 1636390 (0.0005) [2023-12-27 03:18:53,866][105692] Updated weights for policy 0, policy_version 1636400 (0.0005) [2023-12-27 03:18:54,172][105620] Updated weights for policy 1, policy_version 1639621 (0.0008) [2023-12-27 03:18:54,237][105620] Updated weights for policy 1, policy_version 1639631 (0.0010) [2023-12-27 03:18:54,303][105620] Updated weights for policy 1, policy_version 1639641 (0.0007) [2023-12-27 03:18:54,402][105692] Updated weights for policy 0, policy_version 1636410 (0.0011) [2023-12-27 03:18:54,461][105692] Updated weights for policy 0, policy_version 1636420 (0.0010) [2023-12-27 03:18:54,518][105692] Updated weights for policy 0, policy_version 1636430 (0.0010) [2023-12-27 03:18:54,947][105620] Updated weights for policy 1, policy_version 1639651 (0.0008) [2023-12-27 03:18:55,004][105620] Updated weights for policy 1, policy_version 1639661 (0.0010) [2023-12-27 03:18:55,057][105620] Updated weights for policy 1, policy_version 1639671 (0.0009) [2023-12-27 03:18:55,188][105692] Updated weights for policy 0, policy_version 1636440 (0.0010) [2023-12-27 03:18:55,232][105692] Updated weights for policy 0, policy_version 1636450 (0.0010) [2023-12-27 03:18:55,280][105692] Updated weights for policy 0, policy_version 1636460 (0.0010) [2023-12-27 03:18:55,799][105620] Updated weights for policy 1, policy_version 1639681 (0.0009) [2023-12-27 03:18:55,861][105620] Updated weights for policy 1, policy_version 1639691 (0.0010) [2023-12-27 03:18:55,865][105692] Updated weights for policy 0, policy_version 1636470 (0.0007) [2023-12-27 03:18:55,914][105620] Updated weights for policy 1, policy_version 1639701 (0.0010) [2023-12-27 03:18:55,915][105692] Updated weights for policy 0, policy_version 1636480 (0.0005) [2023-12-27 03:18:55,962][105620] Updated weights for policy 1, policy_version 1639711 (0.0010) [2023-12-27 03:18:55,979][105692] Updated weights for policy 0, policy_version 1636490 (0.0005) [2023-12-27 03:18:56,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 838828032. Throughput: 0: 9729.6, 1: 9702.4. Samples: 838829128. Policy #0 lag: (min: 31.0, avg: 32.0, max: 58.0) [2023-12-27 03:18:56,062][104569] Avg episode reward: [(0, '8259.826'), (1, '8990.075')] [2023-12-27 03:18:56,565][105692] Updated weights for policy 0, policy_version 1636500 (0.0006) [2023-12-27 03:18:56,611][105692] Updated weights for policy 0, policy_version 1636510 (0.0005) [2023-12-27 03:18:56,654][105620] Updated weights for policy 1, policy_version 1639721 (0.0010) [2023-12-27 03:18:56,655][105692] Updated weights for policy 0, policy_version 1636520 (0.0006) [2023-12-27 03:18:56,716][105620] Updated weights for policy 1, policy_version 1639731 (0.0010) [2023-12-27 03:18:56,776][105620] Updated weights for policy 1, policy_version 1639741 (0.0010) [2023-12-27 03:18:57,308][105692] Updated weights for policy 0, policy_version 1636530 (0.0007) [2023-12-27 03:18:57,372][105692] Updated weights for policy 0, policy_version 1636540 (0.0008) [2023-12-27 03:18:57,438][105692] Updated weights for policy 0, policy_version 1636550 (0.0008) [2023-12-27 03:18:57,478][105620] Updated weights for policy 1, policy_version 1639751 (0.0010) [2023-12-27 03:18:57,501][105692] Updated weights for policy 0, policy_version 1636560 (0.0009) [2023-12-27 03:18:57,539][105620] Updated weights for policy 1, policy_version 1639761 (0.0010) [2023-12-27 03:18:57,602][105620] Updated weights for policy 1, policy_version 1639771 (0.0011) [2023-12-27 03:18:58,088][105692] Updated weights for policy 0, policy_version 1636570 (0.0007) [2023-12-27 03:18:58,152][105692] Updated weights for policy 0, policy_version 1636580 (0.0006) [2023-12-27 03:18:58,214][105692] Updated weights for policy 0, policy_version 1636590 (0.0008) [2023-12-27 03:18:58,304][105620] Updated weights for policy 1, policy_version 1639781 (0.0010) [2023-12-27 03:18:58,366][105620] Updated weights for policy 1, policy_version 1639791 (0.0009) [2023-12-27 03:18:58,431][105620] Updated weights for policy 1, policy_version 1639801 (0.0008) [2023-12-27 03:18:58,989][105692] Updated weights for policy 0, policy_version 1636600 (0.0007) [2023-12-27 03:18:59,049][105692] Updated weights for policy 0, policy_version 1636610 (0.0011) [2023-12-27 03:18:59,102][105692] Updated weights for policy 0, policy_version 1636620 (0.0011) [2023-12-27 03:18:59,216][105620] Updated weights for policy 1, policy_version 1639811 (0.0008) [2023-12-27 03:18:59,283][105620] Updated weights for policy 1, policy_version 1639821 (0.0010) [2023-12-27 03:18:59,361][105620] Updated weights for policy 1, policy_version 1639831 (0.0010) [2023-12-27 03:18:59,845][105692] Updated weights for policy 0, policy_version 1636630 (0.0009) [2023-12-27 03:18:59,906][105692] Updated weights for policy 0, policy_version 1636640 (0.0008) [2023-12-27 03:18:59,974][105692] Updated weights for policy 0, policy_version 1636650 (0.0008) [2023-12-27 03:19:00,152][105620] Updated weights for policy 1, policy_version 1639841 (0.0010) [2023-12-27 03:19:00,214][105620] Updated weights for policy 1, policy_version 1639851 (0.0006) [2023-12-27 03:19:00,269][105620] Updated weights for policy 1, policy_version 1639861 (0.0010) [2023-12-27 03:19:00,329][105620] Updated weights for policy 1, policy_version 1639871 (0.0008) [2023-12-27 03:19:00,708][105692] Updated weights for policy 0, policy_version 1636660 (0.0009) [2023-12-27 03:19:00,767][105692] Updated weights for policy 0, policy_version 1636670 (0.0010) [2023-12-27 03:19:00,828][105692] Updated weights for policy 0, policy_version 1636680 (0.0010) [2023-12-27 03:19:00,934][105620] Updated weights for policy 1, policy_version 1639881 (0.0010) [2023-12-27 03:19:00,994][105620] Updated weights for policy 1, policy_version 1639891 (0.0011) [2023-12-27 03:19:01,058][105620] Updated weights for policy 1, policy_version 1639901 (0.0011) [2023-12-27 03:19:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 838918144. Throughput: 0: 9843.4, 1: 9733.3. Samples: 838892012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:19:01,063][104569] Avg episode reward: [(0, '8254.537'), (1, '9355.969')] [2023-12-27 03:19:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001636688_419053568.pth... [2023-12-27 03:19:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001635536_418758656.pth [2023-12-27 03:19:01,076][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001639904_419872768.pth... [2023-12-27 03:19:01,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001638784_419586048.pth [2023-12-27 03:19:01,538][105692] Updated weights for policy 0, policy_version 1636690 (0.0010) [2023-12-27 03:19:01,583][105692] Updated weights for policy 0, policy_version 1636700 (0.0010) [2023-12-27 03:19:01,640][105692] Updated weights for policy 0, policy_version 1636710 (0.0010) [2023-12-27 03:19:01,703][105692] Updated weights for policy 0, policy_version 1636720 (0.0011) [2023-12-27 03:19:01,830][105620] Updated weights for policy 1, policy_version 1639911 (0.0008) [2023-12-27 03:19:01,895][105620] Updated weights for policy 1, policy_version 1639921 (0.0011) [2023-12-27 03:19:01,958][105620] Updated weights for policy 1, policy_version 1639931 (0.0010) [2023-12-27 03:19:02,449][105692] Updated weights for policy 0, policy_version 1636730 (0.0005) [2023-12-27 03:19:02,503][105692] Updated weights for policy 0, policy_version 1636740 (0.0005) [2023-12-27 03:19:02,561][105692] Updated weights for policy 0, policy_version 1636750 (0.0010) [2023-12-27 03:19:02,692][105620] Updated weights for policy 1, policy_version 1639941 (0.0010) [2023-12-27 03:19:02,746][105620] Updated weights for policy 1, policy_version 1639951 (0.0010) [2023-12-27 03:19:02,804][105620] Updated weights for policy 1, policy_version 1639961 (0.0010) [2023-12-27 03:19:03,261][105692] Updated weights for policy 0, policy_version 1636760 (0.0010) [2023-12-27 03:19:03,308][105692] Updated weights for policy 0, policy_version 1636770 (0.0010) [2023-12-27 03:19:03,360][105692] Updated weights for policy 0, policy_version 1636780 (0.0010) [2023-12-27 03:19:03,547][105620] Updated weights for policy 1, policy_version 1639971 (0.0010) [2023-12-27 03:19:03,608][105620] Updated weights for policy 1, policy_version 1639981 (0.0011) [2023-12-27 03:19:03,660][105620] Updated weights for policy 1, policy_version 1639991 (0.0010) [2023-12-27 03:19:04,142][105692] Updated weights for policy 0, policy_version 1636790 (0.0010) [2023-12-27 03:19:04,199][105692] Updated weights for policy 0, policy_version 1636800 (0.0010) [2023-12-27 03:19:04,268][105692] Updated weights for policy 0, policy_version 1636810 (0.0008) [2023-12-27 03:19:04,323][105620] Updated weights for policy 1, policy_version 1640001 (0.0010) [2023-12-27 03:19:04,382][105620] Updated weights for policy 1, policy_version 1640011 (0.0011) [2023-12-27 03:19:04,448][105620] Updated weights for policy 1, policy_version 1640021 (0.0010) [2023-12-27 03:19:04,503][105620] Updated weights for policy 1, policy_version 1640031 (0.0010) [2023-12-27 03:19:05,086][105692] Updated weights for policy 0, policy_version 1636820 (0.0008) [2023-12-27 03:19:05,138][105692] Updated weights for policy 0, policy_version 1636830 (0.0009) [2023-12-27 03:19:05,171][105620] Updated weights for policy 1, policy_version 1640041 (0.0006) [2023-12-27 03:19:05,204][105692] Updated weights for policy 0, policy_version 1636840 (0.0008) [2023-12-27 03:19:05,234][105620] Updated weights for policy 1, policy_version 1640051 (0.0007) [2023-12-27 03:19:05,300][105620] Updated weights for policy 1, policy_version 1640061 (0.0006) [2023-12-27 03:19:05,913][105620] Updated weights for policy 1, policy_version 1640071 (0.0009) [2023-12-27 03:19:05,914][105692] Updated weights for policy 0, policy_version 1636850 (0.0007) [2023-12-27 03:19:05,973][105692] Updated weights for policy 0, policy_version 1636860 (0.0005) [2023-12-27 03:19:05,975][105620] Updated weights for policy 1, policy_version 1640081 (0.0010) [2023-12-27 03:19:06,030][105692] Updated weights for policy 0, policy_version 1636870 (0.0005) [2023-12-27 03:19:06,040][105620] Updated weights for policy 1, policy_version 1640091 (0.0010) [2023-12-27 03:19:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 839016448. Throughput: 0: 9755.6, 1: 9722.5. Samples: 839005916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:19:06,062][104569] Avg episode reward: [(0, '7894.611'), (1, '9175.062')] [2023-12-27 03:19:06,082][105692] Updated weights for policy 0, policy_version 1636880 (0.0005) [2023-12-27 03:19:06,764][105692] Updated weights for policy 0, policy_version 1636890 (0.0009) [2023-12-27 03:19:06,764][105620] Updated weights for policy 1, policy_version 1640101 (0.0010) [2023-12-27 03:19:06,826][105692] Updated weights for policy 0, policy_version 1636900 (0.0011) [2023-12-27 03:19:06,827][105620] Updated weights for policy 1, policy_version 1640111 (0.0010) [2023-12-27 03:19:06,878][105692] Updated weights for policy 0, policy_version 1636910 (0.0011) [2023-12-27 03:19:06,888][105620] Updated weights for policy 1, policy_version 1640121 (0.0006) [2023-12-27 03:19:07,551][105620] Updated weights for policy 1, policy_version 1640131 (0.0007) [2023-12-27 03:19:07,617][105620] Updated weights for policy 1, policy_version 1640141 (0.0005) [2023-12-27 03:19:07,624][105692] Updated weights for policy 0, policy_version 1636920 (0.0010) [2023-12-27 03:19:07,666][105620] Updated weights for policy 1, policy_version 1640151 (0.0005) [2023-12-27 03:19:07,685][105692] Updated weights for policy 0, policy_version 1636930 (0.0009) [2023-12-27 03:19:07,748][105692] Updated weights for policy 0, policy_version 1636940 (0.0009) [2023-12-27 03:19:08,217][105620] Updated weights for policy 1, policy_version 1640161 (0.0006) [2023-12-27 03:19:08,268][105620] Updated weights for policy 1, policy_version 1640171 (0.0009) [2023-12-27 03:19:08,318][105620] Updated weights for policy 1, policy_version 1640181 (0.0006) [2023-12-27 03:19:08,382][105620] Updated weights for policy 1, policy_version 1640191 (0.0006) [2023-12-27 03:19:08,521][105692] Updated weights for policy 0, policy_version 1636950 (0.0010) [2023-12-27 03:19:08,572][105692] Updated weights for policy 0, policy_version 1636960 (0.0009) [2023-12-27 03:19:08,619][105692] Updated weights for policy 0, policy_version 1636970 (0.0008) [2023-12-27 03:19:09,073][105620] Updated weights for policy 1, policy_version 1640201 (0.0008) [2023-12-27 03:19:09,120][105620] Updated weights for policy 1, policy_version 1640211 (0.0009) [2023-12-27 03:19:09,168][105620] Updated weights for policy 1, policy_version 1640221 (0.0009) [2023-12-27 03:19:09,424][105692] Updated weights for policy 0, policy_version 1636980 (0.0007) [2023-12-27 03:19:09,480][105692] Updated weights for policy 0, policy_version 1636990 (0.0009) [2023-12-27 03:19:09,545][105692] Updated weights for policy 0, policy_version 1637000 (0.0007) [2023-12-27 03:19:09,950][105620] Updated weights for policy 1, policy_version 1640231 (0.0010) [2023-12-27 03:19:10,020][105620] Updated weights for policy 1, policy_version 1640241 (0.0010) [2023-12-27 03:19:10,088][105620] Updated weights for policy 1, policy_version 1640251 (0.0011) [2023-12-27 03:19:10,168][105692] Updated weights for policy 0, policy_version 1637010 (0.0006) [2023-12-27 03:19:10,226][105692] Updated weights for policy 0, policy_version 1637020 (0.0006) [2023-12-27 03:19:10,294][105692] Updated weights for policy 0, policy_version 1637030 (0.0008) [2023-12-27 03:19:10,364][105692] Updated weights for policy 0, policy_version 1637040 (0.0010) [2023-12-27 03:19:10,700][105620] Updated weights for policy 1, policy_version 1640261 (0.0009) [2023-12-27 03:19:10,750][105620] Updated weights for policy 1, policy_version 1640271 (0.0008) [2023-12-27 03:19:10,798][105620] Updated weights for policy 1, policy_version 1640281 (0.0008) [2023-12-27 03:19:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 839114752. Throughput: 0: 9715.9, 1: 9817.4. Samples: 839124404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:19:11,063][104569] Avg episode reward: [(0, '8272.435'), (1, '9082.709')] [2023-12-27 03:19:11,105][105692] Updated weights for policy 0, policy_version 1637050 (0.0010) [2023-12-27 03:19:11,171][105692] Updated weights for policy 0, policy_version 1637060 (0.0008) [2023-12-27 03:19:11,237][105692] Updated weights for policy 0, policy_version 1637070 (0.0008) [2023-12-27 03:19:11,518][105620] Updated weights for policy 1, policy_version 1640291 (0.0008) [2023-12-27 03:19:11,579][105620] Updated weights for policy 1, policy_version 1640301 (0.0007) [2023-12-27 03:19:11,648][105620] Updated weights for policy 1, policy_version 1640311 (0.0009) [2023-12-27 03:19:11,950][105692] Updated weights for policy 0, policy_version 1637080 (0.0006) [2023-12-27 03:19:12,014][105692] Updated weights for policy 0, policy_version 1637090 (0.0005) [2023-12-27 03:19:12,081][105692] Updated weights for policy 0, policy_version 1637100 (0.0007) [2023-12-27 03:19:12,351][105620] Updated weights for policy 1, policy_version 1640321 (0.0010) [2023-12-27 03:19:12,416][105620] Updated weights for policy 1, policy_version 1640331 (0.0009) [2023-12-27 03:19:12,473][105620] Updated weights for policy 1, policy_version 1640341 (0.0009) [2023-12-27 03:19:12,527][105620] Updated weights for policy 1, policy_version 1640351 (0.0009) [2023-12-27 03:19:12,779][105692] Updated weights for policy 0, policy_version 1637110 (0.0009) [2023-12-27 03:19:12,841][105692] Updated weights for policy 0, policy_version 1637120 (0.0009) [2023-12-27 03:19:12,903][105692] Updated weights for policy 0, policy_version 1637130 (0.0010) [2023-12-27 03:19:13,190][105620] Updated weights for policy 1, policy_version 1640361 (0.0009) [2023-12-27 03:19:13,253][105620] Updated weights for policy 1, policy_version 1640371 (0.0009) [2023-12-27 03:19:13,319][105620] Updated weights for policy 1, policy_version 1640381 (0.0010) [2023-12-27 03:19:13,639][105692] Updated weights for policy 0, policy_version 1637140 (0.0008) [2023-12-27 03:19:13,693][105692] Updated weights for policy 0, policy_version 1637150 (0.0006) [2023-12-27 03:19:13,741][105692] Updated weights for policy 0, policy_version 1637160 (0.0008) [2023-12-27 03:19:13,996][105620] Updated weights for policy 1, policy_version 1640391 (0.0009) [2023-12-27 03:19:14,053][105620] Updated weights for policy 1, policy_version 1640401 (0.0005) [2023-12-27 03:19:14,109][105620] Updated weights for policy 1, policy_version 1640411 (0.0005) [2023-12-27 03:19:14,600][105692] Updated weights for policy 0, policy_version 1637170 (0.0009) [2023-12-27 03:19:14,654][105692] Updated weights for policy 0, policy_version 1637180 (0.0008) [2023-12-27 03:19:14,661][105620] Updated weights for policy 1, policy_version 1640421 (0.0008) [2023-12-27 03:19:14,706][105620] Updated weights for policy 1, policy_version 1640431 (0.0009) [2023-12-27 03:19:14,708][105692] Updated weights for policy 0, policy_version 1637190 (0.0006) [2023-12-27 03:19:14,752][105620] Updated weights for policy 1, policy_version 1640441 (0.0007) [2023-12-27 03:19:14,765][105692] Updated weights for policy 0, policy_version 1637200 (0.0007) [2023-12-27 03:19:15,487][105692] Updated weights for policy 0, policy_version 1637210 (0.0009) [2023-12-27 03:19:15,538][105620] Updated weights for policy 1, policy_version 1640451 (0.0009) [2023-12-27 03:19:15,545][105692] Updated weights for policy 0, policy_version 1637220 (0.0008) [2023-12-27 03:19:15,590][105620] Updated weights for policy 1, policy_version 1640461 (0.0005) [2023-12-27 03:19:15,609][105692] Updated weights for policy 0, policy_version 1637230 (0.0008) [2023-12-27 03:19:15,641][105620] Updated weights for policy 1, policy_version 1640471 (0.0005) [2023-12-27 03:19:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 839213056. Throughput: 0: 9740.7, 1: 9752.5. Samples: 839183216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:19:16,063][104569] Avg episode reward: [(0, '8632.242'), (1, '8989.548')] [2023-12-27 03:19:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001637232_419192832.pth... [2023-12-27 03:19:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001640480_420020224.pth... [2023-12-27 03:19:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001636112_418906112.pth [2023-12-27 03:19:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001639328_419725312.pth [2023-12-27 03:19:16,301][105692] Updated weights for policy 0, policy_version 1637240 (0.0007) [2023-12-27 03:19:16,308][105620] Updated weights for policy 1, policy_version 1640481 (0.0005) [2023-12-27 03:19:16,358][105692] Updated weights for policy 0, policy_version 1637250 (0.0009) [2023-12-27 03:19:16,362][105620] Updated weights for policy 1, policy_version 1640491 (0.0005) [2023-12-27 03:19:16,411][105692] Updated weights for policy 0, policy_version 1637260 (0.0005) [2023-12-27 03:19:16,412][105620] Updated weights for policy 1, policy_version 1640501 (0.0005) [2023-12-27 03:19:16,465][105620] Updated weights for policy 1, policy_version 1640511 (0.0006) [2023-12-27 03:19:17,000][105620] Updated weights for policy 1, policy_version 1640521 (0.0008) [2023-12-27 03:19:17,059][105620] Updated weights for policy 1, policy_version 1640531 (0.0009) [2023-12-27 03:19:17,117][105620] Updated weights for policy 1, policy_version 1640541 (0.0009) [2023-12-27 03:19:17,181][105692] Updated weights for policy 0, policy_version 1637270 (0.0008) [2023-12-27 03:19:17,235][105692] Updated weights for policy 0, policy_version 1637280 (0.0009) [2023-12-27 03:19:17,288][105692] Updated weights for policy 0, policy_version 1637290 (0.0009) [2023-12-27 03:19:17,838][105620] Updated weights for policy 1, policy_version 1640551 (0.0006) [2023-12-27 03:19:17,889][105620] Updated weights for policy 1, policy_version 1640561 (0.0007) [2023-12-27 03:19:17,937][105620] Updated weights for policy 1, policy_version 1640571 (0.0009) [2023-12-27 03:19:17,996][105692] Updated weights for policy 0, policy_version 1637300 (0.0009) [2023-12-27 03:19:18,053][105692] Updated weights for policy 0, policy_version 1637310 (0.0010) [2023-12-27 03:19:18,112][105692] Updated weights for policy 0, policy_version 1637320 (0.0010) [2023-12-27 03:19:18,649][105620] Updated weights for policy 1, policy_version 1640581 (0.0009) [2023-12-27 03:19:18,697][105620] Updated weights for policy 1, policy_version 1640591 (0.0009) [2023-12-27 03:19:18,760][105620] Updated weights for policy 1, policy_version 1640601 (0.0009) [2023-12-27 03:19:18,877][105692] Updated weights for policy 0, policy_version 1637331 (0.0009) [2023-12-27 03:19:18,935][105692] Updated weights for policy 0, policy_version 1637341 (0.0008) [2023-12-27 03:19:19,005][105692] Updated weights for policy 0, policy_version 1637351 (0.0005) [2023-12-27 03:19:19,484][105620] Updated weights for policy 1, policy_version 1640611 (0.0009) [2023-12-27 03:19:19,550][105620] Updated weights for policy 1, policy_version 1640621 (0.0007) [2023-12-27 03:19:19,613][105620] Updated weights for policy 1, policy_version 1640631 (0.0008) [2023-12-27 03:19:19,752][105692] Updated weights for policy 0, policy_version 1637361 (0.0006) [2023-12-27 03:19:19,805][105692] Updated weights for policy 0, policy_version 1637371 (0.0010) [2023-12-27 03:19:19,869][105692] Updated weights for policy 0, policy_version 1637381 (0.0008) [2023-12-27 03:19:19,930][105692] Updated weights for policy 0, policy_version 1637391 (0.0008) [2023-12-27 03:19:20,374][105620] Updated weights for policy 1, policy_version 1640641 (0.0009) [2023-12-27 03:19:20,435][105620] Updated weights for policy 1, policy_version 1640651 (0.0009) [2023-12-27 03:19:20,497][105620] Updated weights for policy 1, policy_version 1640661 (0.0009) [2023-12-27 03:19:20,559][105620] Updated weights for policy 1, policy_version 1640671 (0.0009) [2023-12-27 03:19:20,661][105692] Updated weights for policy 0, policy_version 1637401 (0.0005) [2023-12-27 03:19:20,713][105692] Updated weights for policy 0, policy_version 1637411 (0.0005) [2023-12-27 03:19:20,766][105692] Updated weights for policy 0, policy_version 1637421 (0.0005) [2023-12-27 03:19:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 839311360. Throughput: 0: 9696.8, 1: 9849.2. Samples: 839301112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:19:21,062][104569] Avg episode reward: [(0, '9078.873'), (1, '8898.510')] [2023-12-27 03:19:21,390][105620] Updated weights for policy 1, policy_version 1640681 (0.0009) [2023-12-27 03:19:21,447][105692] Updated weights for policy 0, policy_version 1637431 (0.0007) [2023-12-27 03:19:21,458][105620] Updated weights for policy 1, policy_version 1640691 (0.0009) [2023-12-27 03:19:21,510][105692] Updated weights for policy 0, policy_version 1637441 (0.0009) [2023-12-27 03:19:21,518][105620] Updated weights for policy 1, policy_version 1640701 (0.0007) [2023-12-27 03:19:21,569][105692] Updated weights for policy 0, policy_version 1637451 (0.0009) [2023-12-27 03:19:22,198][105620] Updated weights for policy 1, policy_version 1640711 (0.0009) [2023-12-27 03:19:22,254][105620] Updated weights for policy 1, policy_version 1640721 (0.0009) [2023-12-27 03:19:22,310][105620] Updated weights for policy 1, policy_version 1640731 (0.0010) [2023-12-27 03:19:22,367][105692] Updated weights for policy 0, policy_version 1637461 (0.0008) [2023-12-27 03:19:22,427][105692] Updated weights for policy 0, policy_version 1637471 (0.0008) [2023-12-27 03:19:22,499][105692] Updated weights for policy 0, policy_version 1637481 (0.0009) [2023-12-27 03:19:23,003][105620] Updated weights for policy 1, policy_version 1640741 (0.0009) [2023-12-27 03:19:23,051][105620] Updated weights for policy 1, policy_version 1640751 (0.0007) [2023-12-27 03:19:23,098][105620] Updated weights for policy 1, policy_version 1640761 (0.0009) [2023-12-27 03:19:23,302][105692] Updated weights for policy 0, policy_version 1637491 (0.0007) [2023-12-27 03:19:23,361][105692] Updated weights for policy 0, policy_version 1637501 (0.0009) [2023-12-27 03:19:23,419][105692] Updated weights for policy 0, policy_version 1637511 (0.0009) [2023-12-27 03:19:23,722][105620] Updated weights for policy 1, policy_version 1640771 (0.0006) [2023-12-27 03:19:23,780][105620] Updated weights for policy 1, policy_version 1640781 (0.0009) [2023-12-27 03:19:23,826][105620] Updated weights for policy 1, policy_version 1640791 (0.0008) [2023-12-27 03:19:24,285][105692] Updated weights for policy 0, policy_version 1637521 (0.0008) [2023-12-27 03:19:24,333][105692] Updated weights for policy 0, policy_version 1637531 (0.0009) [2023-12-27 03:19:24,391][105692] Updated weights for policy 0, policy_version 1637541 (0.0010) [2023-12-27 03:19:24,393][105620] Updated weights for policy 1, policy_version 1640801 (0.0007) [2023-12-27 03:19:24,447][105692] Updated weights for policy 0, policy_version 1637551 (0.0009) [2023-12-27 03:19:24,450][105620] Updated weights for policy 1, policy_version 1640811 (0.0006) [2023-12-27 03:19:24,504][105620] Updated weights for policy 1, policy_version 1640821 (0.0008) [2023-12-27 03:19:24,572][105620] Updated weights for policy 1, policy_version 1640831 (0.0005) [2023-12-27 03:19:25,155][105620] Updated weights for policy 1, policy_version 1640841 (0.0008) [2023-12-27 03:19:25,216][105620] Updated weights for policy 1, policy_version 1640851 (0.0009) [2023-12-27 03:19:25,271][105620] Updated weights for policy 1, policy_version 1640861 (0.0008) [2023-12-27 03:19:25,294][105692] Updated weights for policy 0, policy_version 1637561 (0.0008) [2023-12-27 03:19:25,341][105692] Updated weights for policy 0, policy_version 1637571 (0.0009) [2023-12-27 03:19:25,392][105692] Updated weights for policy 0, policy_version 1637581 (0.0009) [2023-12-27 03:19:25,972][105620] Updated weights for policy 1, policy_version 1640871 (0.0008) [2023-12-27 03:19:26,028][105620] Updated weights for policy 1, policy_version 1640881 (0.0007) [2023-12-27 03:19:26,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 839401472. Throughput: 0: 9545.6, 1: 9850.3. Samples: 839415788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:19:26,063][104569] Avg episode reward: [(0, '8532.508'), (1, '8714.363')] [2023-12-27 03:19:26,089][105620] Updated weights for policy 1, policy_version 1640891 (0.0006) [2023-12-27 03:19:26,183][105692] Updated weights for policy 0, policy_version 1637591 (0.0009) [2023-12-27 03:19:26,237][105692] Updated weights for policy 0, policy_version 1637601 (0.0009) [2023-12-27 03:19:26,291][105692] Updated weights for policy 0, policy_version 1637611 (0.0009) [2023-12-27 03:19:26,811][105620] Updated weights for policy 1, policy_version 1640901 (0.0009) [2023-12-27 03:19:26,857][105620] Updated weights for policy 1, policy_version 1640911 (0.0008) [2023-12-27 03:19:26,908][105620] Updated weights for policy 1, policy_version 1640921 (0.0009) [2023-12-27 03:19:27,040][105692] Updated weights for policy 0, policy_version 1637621 (0.0009) [2023-12-27 03:19:27,089][105692] Updated weights for policy 0, policy_version 1637631 (0.0009) [2023-12-27 03:19:27,150][105692] Updated weights for policy 0, policy_version 1637642 (0.0009) [2023-12-27 03:19:27,592][105620] Updated weights for policy 1, policy_version 1640931 (0.0009) [2023-12-27 03:19:27,646][105620] Updated weights for policy 1, policy_version 1640941 (0.0009) [2023-12-27 03:19:27,703][105620] Updated weights for policy 1, policy_version 1640951 (0.0009) [2023-12-27 03:19:27,989][105692] Updated weights for policy 0, policy_version 1637652 (0.0009) [2023-12-27 03:19:28,042][105692] Updated weights for policy 0, policy_version 1637662 (0.0010) [2023-12-27 03:19:28,093][105692] Updated weights for policy 0, policy_version 1637672 (0.0010) [2023-12-27 03:19:28,305][105620] Updated weights for policy 1, policy_version 1640961 (0.0008) [2023-12-27 03:19:28,366][105620] Updated weights for policy 1, policy_version 1640971 (0.0011) [2023-12-27 03:19:28,425][105620] Updated weights for policy 1, policy_version 1640981 (0.0006) [2023-12-27 03:19:28,476][105620] Updated weights for policy 1, policy_version 1640991 (0.0005) [2023-12-27 03:19:28,959][105692] Updated weights for policy 0, policy_version 1637682 (0.0009) [2023-12-27 03:19:29,012][105692] Updated weights for policy 0, policy_version 1637692 (0.0009) [2023-12-27 03:19:29,064][105620] Updated weights for policy 1, policy_version 1641001 (0.0006) [2023-12-27 03:19:29,067][105692] Updated weights for policy 0, policy_version 1637702 (0.0008) [2023-12-27 03:19:29,118][105620] Updated weights for policy 1, policy_version 1641011 (0.0005) [2023-12-27 03:19:29,123][105692] Updated weights for policy 0, policy_version 1637712 (0.0009) [2023-12-27 03:19:29,169][105620] Updated weights for policy 1, policy_version 1641021 (0.0006) [2023-12-27 03:19:29,864][105620] Updated weights for policy 1, policy_version 1641031 (0.0009) [2023-12-27 03:19:29,918][105620] Updated weights for policy 1, policy_version 1641041 (0.0009) [2023-12-27 03:19:29,948][105692] Updated weights for policy 0, policy_version 1637722 (0.0008) [2023-12-27 03:19:29,987][105620] Updated weights for policy 1, policy_version 1641051 (0.0011) [2023-12-27 03:19:30,014][105692] Updated weights for policy 0, policy_version 1637732 (0.0009) [2023-12-27 03:19:30,072][105692] Updated weights for policy 0, policy_version 1637742 (0.0008) [2023-12-27 03:19:30,741][105620] Updated weights for policy 1, policy_version 1641061 (0.0011) [2023-12-27 03:19:30,785][105620] Updated weights for policy 1, policy_version 1641071 (0.0010) [2023-12-27 03:19:30,846][105692] Updated weights for policy 0, policy_version 1637752 (0.0008) [2023-12-27 03:19:30,846][105620] Updated weights for policy 1, policy_version 1641081 (0.0010) [2023-12-27 03:19:30,900][105692] Updated weights for policy 0, policy_version 1637762 (0.0007) [2023-12-27 03:19:30,959][105692] Updated weights for policy 0, policy_version 1637772 (0.0008) [2023-12-27 03:19:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 839507968. Throughput: 0: 9518.9, 1: 9943.1. Samples: 839474232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:19:31,063][104569] Avg episode reward: [(0, '8535.225'), (1, '8895.298')] [2023-12-27 03:19:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001637776_419332096.pth... [2023-12-27 03:19:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001641088_420175872.pth... [2023-12-27 03:19:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001636688_419053568.pth [2023-12-27 03:19:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001639904_419872768.pth [2023-12-27 03:19:31,566][105620] Updated weights for policy 1, policy_version 1641091 (0.0010) [2023-12-27 03:19:31,631][105620] Updated weights for policy 1, policy_version 1641101 (0.0011) [2023-12-27 03:19:31,696][105620] Updated weights for policy 1, policy_version 1641111 (0.0008) [2023-12-27 03:19:31,758][105692] Updated weights for policy 0, policy_version 1637782 (0.0009) [2023-12-27 03:19:31,814][105692] Updated weights for policy 0, policy_version 1637792 (0.0010) [2023-12-27 03:19:31,870][105692] Updated weights for policy 0, policy_version 1637802 (0.0009) [2023-12-27 03:19:32,350][105620] Updated weights for policy 1, policy_version 1641121 (0.0008) [2023-12-27 03:19:32,417][105620] Updated weights for policy 1, policy_version 1641131 (0.0010) [2023-12-27 03:19:32,476][105620] Updated weights for policy 1, policy_version 1641141 (0.0010) [2023-12-27 03:19:32,530][105620] Updated weights for policy 1, policy_version 1641151 (0.0008) [2023-12-27 03:19:32,668][105692] Updated weights for policy 0, policy_version 1637812 (0.0007) [2023-12-27 03:19:32,721][105692] Updated weights for policy 0, policy_version 1637822 (0.0008) [2023-12-27 03:19:32,780][105692] Updated weights for policy 0, policy_version 1637832 (0.0008) [2023-12-27 03:19:33,244][105620] Updated weights for policy 1, policy_version 1641161 (0.0011) [2023-12-27 03:19:33,302][105620] Updated weights for policy 1, policy_version 1641171 (0.0010) [2023-12-27 03:19:33,361][105620] Updated weights for policy 1, policy_version 1641181 (0.0010) [2023-12-27 03:19:33,458][105692] Updated weights for policy 0, policy_version 1637842 (0.0008) [2023-12-27 03:19:33,522][105692] Updated weights for policy 0, policy_version 1637852 (0.0005) [2023-12-27 03:19:33,574][105692] Updated weights for policy 0, policy_version 1637862 (0.0005) [2023-12-27 03:19:33,619][105692] Updated weights for policy 0, policy_version 1637872 (0.0005) [2023-12-27 03:19:34,049][105620] Updated weights for policy 1, policy_version 1641191 (0.0010) [2023-12-27 03:19:34,106][105620] Updated weights for policy 1, policy_version 1641201 (0.0008) [2023-12-27 03:19:34,166][105620] Updated weights for policy 1, policy_version 1641211 (0.0009) [2023-12-27 03:19:34,181][105692] Updated weights for policy 0, policy_version 1637882 (0.0007) [2023-12-27 03:19:34,247][105692] Updated weights for policy 0, policy_version 1637892 (0.0009) [2023-12-27 03:19:34,320][105692] Updated weights for policy 0, policy_version 1637902 (0.0009) [2023-12-27 03:19:34,790][105620] Updated weights for policy 1, policy_version 1641221 (0.0009) [2023-12-27 03:19:34,846][105620] Updated weights for policy 1, policy_version 1641231 (0.0010) [2023-12-27 03:19:34,909][105620] Updated weights for policy 1, policy_version 1641241 (0.0011) [2023-12-27 03:19:35,106][105692] Updated weights for policy 0, policy_version 1637912 (0.0006) [2023-12-27 03:19:35,165][105692] Updated weights for policy 0, policy_version 1637922 (0.0005) [2023-12-27 03:19:35,223][105692] Updated weights for policy 0, policy_version 1637932 (0.0006) [2023-12-27 03:19:35,586][105620] Updated weights for policy 1, policy_version 1641251 (0.0009) [2023-12-27 03:19:35,640][105620] Updated weights for policy 1, policy_version 1641261 (0.0006) [2023-12-27 03:19:35,691][105620] Updated weights for policy 1, policy_version 1641271 (0.0006) [2023-12-27 03:19:35,790][105692] Updated weights for policy 0, policy_version 1637942 (0.0006) [2023-12-27 03:19:35,846][105692] Updated weights for policy 0, policy_version 1637952 (0.0005) [2023-12-27 03:19:35,907][105692] Updated weights for policy 0, policy_version 1637962 (0.0005) [2023-12-27 03:19:36,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 839606272. Throughput: 0: 9473.0, 1: 10026.2. Samples: 839590244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:19:36,062][104569] Avg episode reward: [(0, '8901.325'), (1, '9171.225')] [2023-12-27 03:19:36,327][105620] Updated weights for policy 1, policy_version 1641281 (0.0005) [2023-12-27 03:19:36,387][105620] Updated weights for policy 1, policy_version 1641291 (0.0007) [2023-12-27 03:19:36,449][105620] Updated weights for policy 1, policy_version 1641301 (0.0006) [2023-12-27 03:19:36,512][105620] Updated weights for policy 1, policy_version 1641311 (0.0005) [2023-12-27 03:19:36,659][105692] Updated weights for policy 0, policy_version 1637972 (0.0007) [2023-12-27 03:19:36,726][105692] Updated weights for policy 0, policy_version 1637982 (0.0010) [2023-12-27 03:19:36,787][105692] Updated weights for policy 0, policy_version 1637992 (0.0009) [2023-12-27 03:19:37,110][105620] Updated weights for policy 1, policy_version 1641321 (0.0008) [2023-12-27 03:19:37,175][105620] Updated weights for policy 1, policy_version 1641331 (0.0006) [2023-12-27 03:19:37,241][105620] Updated weights for policy 1, policy_version 1641341 (0.0005) [2023-12-27 03:19:37,562][105692] Updated weights for policy 0, policy_version 1638002 (0.0009) [2023-12-27 03:19:37,627][105692] Updated weights for policy 0, policy_version 1638012 (0.0011) [2023-12-27 03:19:37,683][105692] Updated weights for policy 0, policy_version 1638022 (0.0006) [2023-12-27 03:19:37,747][105692] Updated weights for policy 0, policy_version 1638032 (0.0006) [2023-12-27 03:19:37,884][105620] Updated weights for policy 1, policy_version 1641352 (0.0009) [2023-12-27 03:19:37,947][105620] Updated weights for policy 1, policy_version 1641362 (0.0006) [2023-12-27 03:19:38,007][105620] Updated weights for policy 1, policy_version 1641372 (0.0008) [2023-12-27 03:19:38,420][105692] Updated weights for policy 0, policy_version 1638042 (0.0011) [2023-12-27 03:19:38,476][105692] Updated weights for policy 0, policy_version 1638052 (0.0011) [2023-12-27 03:19:38,533][105692] Updated weights for policy 0, policy_version 1638062 (0.0011) [2023-12-27 03:19:38,765][105620] Updated weights for policy 1, policy_version 1641382 (0.0008) [2023-12-27 03:19:38,823][105620] Updated weights for policy 1, policy_version 1641392 (0.0008) [2023-12-27 03:19:38,882][105620] Updated weights for policy 1, policy_version 1641402 (0.0009) [2023-12-27 03:19:39,196][105692] Updated weights for policy 0, policy_version 1638072 (0.0006) [2023-12-27 03:19:39,266][105692] Updated weights for policy 0, policy_version 1638082 (0.0010) [2023-12-27 03:19:39,323][105692] Updated weights for policy 0, policy_version 1638092 (0.0010) [2023-12-27 03:19:39,639][105620] Updated weights for policy 1, policy_version 1641412 (0.0010) [2023-12-27 03:19:39,707][105620] Updated weights for policy 1, policy_version 1641422 (0.0008) [2023-12-27 03:19:39,763][105620] Updated weights for policy 1, policy_version 1641432 (0.0008) [2023-12-27 03:19:40,038][105692] Updated weights for policy 0, policy_version 1638102 (0.0011) [2023-12-27 03:19:40,098][105692] Updated weights for policy 0, policy_version 1638112 (0.0009) [2023-12-27 03:19:40,164][105692] Updated weights for policy 0, policy_version 1638122 (0.0006) [2023-12-27 03:19:40,539][105620] Updated weights for policy 1, policy_version 1641442 (0.0008) [2023-12-27 03:19:40,600][105620] Updated weights for policy 1, policy_version 1641452 (0.0009) [2023-12-27 03:19:40,650][105620] Updated weights for policy 1, policy_version 1641462 (0.0008) [2023-12-27 03:19:40,703][105620] Updated weights for policy 1, policy_version 1641472 (0.0008) [2023-12-27 03:19:40,872][105692] Updated weights for policy 0, policy_version 1638132 (0.0011) [2023-12-27 03:19:40,930][105692] Updated weights for policy 0, policy_version 1638142 (0.0011) [2023-12-27 03:19:40,996][105692] Updated weights for policy 0, policy_version 1638152 (0.0010) [2023-12-27 03:19:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 839704576. Throughput: 0: 9497.7, 1: 10058.8. Samples: 839709172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:19:41,063][104569] Avg episode reward: [(0, '8990.100'), (1, '8995.637')] [2023-12-27 03:19:41,497][105620] Updated weights for policy 1, policy_version 1641482 (0.0009) [2023-12-27 03:19:41,559][105620] Updated weights for policy 1, policy_version 1641492 (0.0009) [2023-12-27 03:19:41,611][105620] Updated weights for policy 1, policy_version 1641502 (0.0008) [2023-12-27 03:19:41,732][105692] Updated weights for policy 0, policy_version 1638162 (0.0010) [2023-12-27 03:19:41,794][105692] Updated weights for policy 0, policy_version 1638172 (0.0009) [2023-12-27 03:19:41,865][105692] Updated weights for policy 0, policy_version 1638182 (0.0010) [2023-12-27 03:19:41,928][105692] Updated weights for policy 0, policy_version 1638192 (0.0009) [2023-12-27 03:19:42,387][105620] Updated weights for policy 1, policy_version 1641512 (0.0009) [2023-12-27 03:19:42,449][105620] Updated weights for policy 1, policy_version 1641522 (0.0008) [2023-12-27 03:19:42,516][105620] Updated weights for policy 1, policy_version 1641532 (0.0007) [2023-12-27 03:19:42,684][105692] Updated weights for policy 0, policy_version 1638202 (0.0006) [2023-12-27 03:19:42,747][105692] Updated weights for policy 0, policy_version 1638212 (0.0005) [2023-12-27 03:19:42,813][105692] Updated weights for policy 0, policy_version 1638222 (0.0006) [2023-12-27 03:19:43,328][105620] Updated weights for policy 1, policy_version 1641542 (0.0008) [2023-12-27 03:19:43,331][105692] Updated weights for policy 0, policy_version 1638232 (0.0006) [2023-12-27 03:19:43,379][105692] Updated weights for policy 0, policy_version 1638242 (0.0006) [2023-12-27 03:19:43,382][105620] Updated weights for policy 1, policy_version 1641552 (0.0007) [2023-12-27 03:19:43,436][105692] Updated weights for policy 0, policy_version 1638252 (0.0006) [2023-12-27 03:19:43,438][105620] Updated weights for policy 1, policy_version 1641562 (0.0006) [2023-12-27 03:19:44,142][105692] Updated weights for policy 0, policy_version 1638262 (0.0008) [2023-12-27 03:19:44,164][105620] Updated weights for policy 1, policy_version 1641572 (0.0008) [2023-12-27 03:19:44,192][105692] Updated weights for policy 0, policy_version 1638272 (0.0006) [2023-12-27 03:19:44,210][105620] Updated weights for policy 1, policy_version 1641582 (0.0006) [2023-12-27 03:19:44,245][105692] Updated weights for policy 0, policy_version 1638282 (0.0008) [2023-12-27 03:19:44,255][105620] Updated weights for policy 1, policy_version 1641592 (0.0006) [2023-12-27 03:19:44,968][105620] Updated weights for policy 1, policy_version 1641602 (0.0007) [2023-12-27 03:19:45,035][105620] Updated weights for policy 1, policy_version 1641612 (0.0006) [2023-12-27 03:19:45,075][105692] Updated weights for policy 0, policy_version 1638292 (0.0007) [2023-12-27 03:19:45,090][105620] Updated weights for policy 1, policy_version 1641622 (0.0005) [2023-12-27 03:19:45,139][105692] Updated weights for policy 0, policy_version 1638302 (0.0008) [2023-12-27 03:19:45,146][105620] Updated weights for policy 1, policy_version 1641632 (0.0005) [2023-12-27 03:19:45,208][105692] Updated weights for policy 0, policy_version 1638312 (0.0009) [2023-12-27 03:19:45,684][105620] Updated weights for policy 1, policy_version 1641642 (0.0009) [2023-12-27 03:19:45,731][105620] Updated weights for policy 1, policy_version 1641652 (0.0009) [2023-12-27 03:19:45,782][105620] Updated weights for policy 1, policy_version 1641662 (0.0009) [2023-12-27 03:19:46,011][105692] Updated weights for policy 0, policy_version 1638322 (0.0009) [2023-12-27 03:19:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 839794688. Throughput: 0: 9412.2, 1: 9987.1. Samples: 839764980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:19:46,062][104569] Avg episode reward: [(0, '8631.669'), (1, '8993.684')] [2023-12-27 03:19:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001641664_420323328.pth... [2023-12-27 03:19:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001640480_420020224.pth [2023-12-27 03:19:46,073][105692] Updated weights for policy 0, policy_version 1638332 (0.0009) [2023-12-27 03:19:46,132][105692] Updated weights for policy 0, policy_version 1638342 (0.0009) [2023-12-27 03:19:46,191][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001638352_419479552.pth... [2023-12-27 03:19:46,194][105692] Updated weights for policy 0, policy_version 1638352 (0.0008) [2023-12-27 03:19:46,195][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001637232_419192832.pth [2023-12-27 03:19:46,576][105620] Updated weights for policy 1, policy_version 1641672 (0.0009) [2023-12-27 03:19:46,631][105620] Updated weights for policy 1, policy_version 1641682 (0.0009) [2023-12-27 03:19:46,677][105620] Updated weights for policy 1, policy_version 1641692 (0.0008) [2023-12-27 03:19:46,948][105692] Updated weights for policy 0, policy_version 1638362 (0.0009) [2023-12-27 03:19:47,018][105692] Updated weights for policy 0, policy_version 1638372 (0.0009) [2023-12-27 03:19:47,080][105692] Updated weights for policy 0, policy_version 1638382 (0.0009) [2023-12-27 03:19:47,440][105620] Updated weights for policy 1, policy_version 1641702 (0.0009) [2023-12-27 03:19:47,491][105620] Updated weights for policy 1, policy_version 1641712 (0.0008) [2023-12-27 03:19:47,551][105620] Updated weights for policy 1, policy_version 1641722 (0.0009) [2023-12-27 03:19:47,803][105692] Updated weights for policy 0, policy_version 1638392 (0.0009) [2023-12-27 03:19:47,854][105692] Updated weights for policy 0, policy_version 1638402 (0.0008) [2023-12-27 03:19:47,900][105692] Updated weights for policy 0, policy_version 1638412 (0.0008) [2023-12-27 03:19:48,366][105620] Updated weights for policy 1, policy_version 1641732 (0.0009) [2023-12-27 03:19:48,431][105620] Updated weights for policy 1, policy_version 1641742 (0.0008) [2023-12-27 03:19:48,478][105620] Updated weights for policy 1, policy_version 1641752 (0.0009) [2023-12-27 03:19:48,572][105692] Updated weights for policy 0, policy_version 1638422 (0.0009) [2023-12-27 03:19:48,658][105692] Updated weights for policy 0, policy_version 1638432 (0.0009) [2023-12-27 03:19:48,716][105692] Updated weights for policy 0, policy_version 1638442 (0.0009) [2023-12-27 03:19:49,245][105620] Updated weights for policy 1, policy_version 1641762 (0.0008) [2023-12-27 03:19:49,289][105620] Updated weights for policy 1, policy_version 1641772 (0.0008) [2023-12-27 03:19:49,351][105620] Updated weights for policy 1, policy_version 1641782 (0.0007) [2023-12-27 03:19:49,418][105620] Updated weights for policy 1, policy_version 1641792 (0.0006) [2023-12-27 03:19:49,485][105692] Updated weights for policy 0, policy_version 1638452 (0.0008) [2023-12-27 03:19:49,543][105692] Updated weights for policy 0, policy_version 1638462 (0.0009) [2023-12-27 03:19:49,599][105692] Updated weights for policy 0, policy_version 1638472 (0.0009) [2023-12-27 03:19:50,111][105620] Updated weights for policy 1, policy_version 1641802 (0.0009) [2023-12-27 03:19:50,172][105620] Updated weights for policy 1, policy_version 1641812 (0.0008) [2023-12-27 03:19:50,236][105620] Updated weights for policy 1, policy_version 1641822 (0.0008) [2023-12-27 03:19:50,424][105692] Updated weights for policy 0, policy_version 1638482 (0.0009) [2023-12-27 03:19:50,486][105692] Updated weights for policy 0, policy_version 1638492 (0.0010) [2023-12-27 03:19:50,538][105692] Updated weights for policy 0, policy_version 1638502 (0.0008) [2023-12-27 03:19:50,600][105692] Updated weights for policy 0, policy_version 1638512 (0.0009) [2023-12-27 03:19:50,894][105620] Updated weights for policy 1, policy_version 1641832 (0.0006) [2023-12-27 03:19:50,953][105620] Updated weights for policy 1, policy_version 1641842 (0.0006) [2023-12-27 03:19:51,005][105620] Updated weights for policy 1, policy_version 1641852 (0.0006) [2023-12-27 03:19:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 839892992. Throughput: 0: 9400.9, 1: 10028.3. Samples: 839880228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:19:51,063][104569] Avg episode reward: [(0, '8271.450'), (1, '9264.559')] [2023-12-27 03:19:51,419][105692] Updated weights for policy 0, policy_version 1638522 (0.0010) [2023-12-27 03:19:51,485][105692] Updated weights for policy 0, policy_version 1638532 (0.0009) [2023-12-27 03:19:51,543][105692] Updated weights for policy 0, policy_version 1638542 (0.0009) [2023-12-27 03:19:51,788][105620] Updated weights for policy 1, policy_version 1641862 (0.0007) [2023-12-27 03:19:51,851][105620] Updated weights for policy 1, policy_version 1641872 (0.0008) [2023-12-27 03:19:51,918][105620] Updated weights for policy 1, policy_version 1641882 (0.0008) [2023-12-27 03:19:52,313][105692] Updated weights for policy 0, policy_version 1638552 (0.0010) [2023-12-27 03:19:52,377][105692] Updated weights for policy 0, policy_version 1638562 (0.0009) [2023-12-27 03:19:52,439][105692] Updated weights for policy 0, policy_version 1638572 (0.0010) [2023-12-27 03:19:52,548][105620] Updated weights for policy 1, policy_version 1641892 (0.0007) [2023-12-27 03:19:52,601][105620] Updated weights for policy 1, policy_version 1641902 (0.0007) [2023-12-27 03:19:52,653][105620] Updated weights for policy 1, policy_version 1641912 (0.0009) [2023-12-27 03:19:53,240][105692] Updated weights for policy 0, policy_version 1638582 (0.0009) [2023-12-27 03:19:53,286][105692] Updated weights for policy 0, policy_version 1638592 (0.0008) [2023-12-27 03:19:53,344][105692] Updated weights for policy 0, policy_version 1638602 (0.0009) [2023-12-27 03:19:53,385][105620] Updated weights for policy 1, policy_version 1641922 (0.0008) [2023-12-27 03:19:53,447][105620] Updated weights for policy 1, policy_version 1641932 (0.0010) [2023-12-27 03:19:53,497][105620] Updated weights for policy 1, policy_version 1641942 (0.0009) [2023-12-27 03:19:53,551][105620] Updated weights for policy 1, policy_version 1641952 (0.0009) [2023-12-27 03:19:54,056][105692] Updated weights for policy 0, policy_version 1638612 (0.0009) [2023-12-27 03:19:54,115][105692] Updated weights for policy 0, policy_version 1638622 (0.0006) [2023-12-27 03:19:54,172][105692] Updated weights for policy 0, policy_version 1638632 (0.0009) [2023-12-27 03:19:54,337][105620] Updated weights for policy 1, policy_version 1641962 (0.0008) [2023-12-27 03:19:54,393][105620] Updated weights for policy 1, policy_version 1641972 (0.0009) [2023-12-27 03:19:54,445][105620] Updated weights for policy 1, policy_version 1641983 (0.0008) [2023-12-27 03:19:54,873][105692] Updated weights for policy 0, policy_version 1638642 (0.0009) [2023-12-27 03:19:54,921][105692] Updated weights for policy 0, policy_version 1638652 (0.0009) [2023-12-27 03:19:54,975][105692] Updated weights for policy 0, policy_version 1638662 (0.0006) [2023-12-27 03:19:55,033][105692] Updated weights for policy 0, policy_version 1638672 (0.0006) [2023-12-27 03:19:55,281][105620] Updated weights for policy 1, policy_version 1641993 (0.0009) [2023-12-27 03:19:55,328][105620] Updated weights for policy 1, policy_version 1642003 (0.0009) [2023-12-27 03:19:55,382][105620] Updated weights for policy 1, policy_version 1642013 (0.0009) [2023-12-27 03:19:55,663][105692] Updated weights for policy 0, policy_version 1638682 (0.0009) [2023-12-27 03:19:55,714][105692] Updated weights for policy 0, policy_version 1638692 (0.0009) [2023-12-27 03:19:55,774][105692] Updated weights for policy 0, policy_version 1638702 (0.0009) [2023-12-27 03:19:56,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19438.7). Total num frames: 839983104. Throughput: 0: 9375.4, 1: 9940.2. Samples: 839993604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:19:56,062][104569] Avg episode reward: [(0, '8534.938'), (1, '8992.770')] [2023-12-27 03:19:56,149][105620] Updated weights for policy 1, policy_version 1642023 (0.0009) [2023-12-27 03:19:56,201][105620] Updated weights for policy 1, policy_version 1642033 (0.0009) [2023-12-27 03:19:56,255][105620] Updated weights for policy 1, policy_version 1642043 (0.0009) [2023-12-27 03:19:56,535][105692] Updated weights for policy 0, policy_version 1638712 (0.0010) [2023-12-27 03:19:56,591][105692] Updated weights for policy 0, policy_version 1638722 (0.0009) [2023-12-27 03:19:56,638][105692] Updated weights for policy 0, policy_version 1638732 (0.0008) [2023-12-27 03:19:57,024][105620] Updated weights for policy 1, policy_version 1642053 (0.0009) [2023-12-27 03:19:57,080][105620] Updated weights for policy 1, policy_version 1642063 (0.0009) [2023-12-27 03:19:57,138][105620] Updated weights for policy 1, policy_version 1642074 (0.0010) [2023-12-27 03:19:57,304][105692] Updated weights for policy 0, policy_version 1638742 (0.0008) [2023-12-27 03:19:57,350][105692] Updated weights for policy 0, policy_version 1638752 (0.0009) [2023-12-27 03:19:57,397][105692] Updated weights for policy 0, policy_version 1638762 (0.0008) [2023-12-27 03:19:57,920][105620] Updated weights for policy 1, policy_version 1642084 (0.0010) [2023-12-27 03:19:57,979][105620] Updated weights for policy 1, policy_version 1642094 (0.0009) [2023-12-27 03:19:58,040][105620] Updated weights for policy 1, policy_version 1642104 (0.0009) [2023-12-27 03:19:58,132][105692] Updated weights for policy 0, policy_version 1638772 (0.0009) [2023-12-27 03:19:58,198][105692] Updated weights for policy 0, policy_version 1638782 (0.0010) [2023-12-27 03:19:58,260][105692] Updated weights for policy 0, policy_version 1638792 (0.0009) [2023-12-27 03:19:58,853][105620] Updated weights for policy 1, policy_version 1642114 (0.0009) [2023-12-27 03:19:58,913][105620] Updated weights for policy 1, policy_version 1642124 (0.0008) [2023-12-27 03:19:58,972][105620] Updated weights for policy 1, policy_version 1642134 (0.0008) [2023-12-27 03:19:59,030][105620] Updated weights for policy 1, policy_version 1642144 (0.0008) [2023-12-27 03:19:59,077][105692] Updated weights for policy 0, policy_version 1638802 (0.0009) [2023-12-27 03:19:59,134][105692] Updated weights for policy 0, policy_version 1638812 (0.0009) [2023-12-27 03:19:59,183][105692] Updated weights for policy 0, policy_version 1638822 (0.0009) [2023-12-27 03:19:59,245][105692] Updated weights for policy 0, policy_version 1638832 (0.0009) [2023-12-27 03:19:59,768][105620] Updated weights for policy 1, policy_version 1642154 (0.0005) [2023-12-27 03:19:59,817][105620] Updated weights for policy 1, policy_version 1642164 (0.0006) [2023-12-27 03:19:59,875][105620] Updated weights for policy 1, policy_version 1642174 (0.0007) [2023-12-27 03:20:00,022][105692] Updated weights for policy 0, policy_version 1638842 (0.0011) [2023-12-27 03:20:00,081][105692] Updated weights for policy 0, policy_version 1638852 (0.0009) [2023-12-27 03:20:00,141][105692] Updated weights for policy 0, policy_version 1638862 (0.0011) [2023-12-27 03:20:00,574][105620] Updated weights for policy 1, policy_version 1642184 (0.0009) [2023-12-27 03:20:00,626][105620] Updated weights for policy 1, policy_version 1642194 (0.0010) [2023-12-27 03:20:00,694][105620] Updated weights for policy 1, policy_version 1642204 (0.0010) [2023-12-27 03:20:00,873][105692] Updated weights for policy 0, policy_version 1638872 (0.0009) [2023-12-27 03:20:00,937][105692] Updated weights for policy 0, policy_version 1638882 (0.0008) [2023-12-27 03:20:00,994][105692] Updated weights for policy 0, policy_version 1638892 (0.0010) [2023-12-27 03:20:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 840081408. Throughput: 0: 9374.8, 1: 9860.5. Samples: 840048804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:20:01,062][104569] Avg episode reward: [(0, '8532.527'), (1, '8993.145')] [2023-12-27 03:20:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001642208_420462592.pth... [2023-12-27 03:20:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001638896_419618816.pth... [2023-12-27 03:20:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001641088_420175872.pth [2023-12-27 03:20:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001637776_419332096.pth [2023-12-27 03:20:01,391][105620] Updated weights for policy 1, policy_version 1642214 (0.0009) [2023-12-27 03:20:01,445][105620] Updated weights for policy 1, policy_version 1642224 (0.0008) [2023-12-27 03:20:01,496][105620] Updated weights for policy 1, policy_version 1642234 (0.0008) [2023-12-27 03:20:01,803][105692] Updated weights for policy 0, policy_version 1638902 (0.0011) [2023-12-27 03:20:01,868][105692] Updated weights for policy 0, policy_version 1638912 (0.0011) [2023-12-27 03:20:01,937][105692] Updated weights for policy 0, policy_version 1638922 (0.0011) [2023-12-27 03:20:02,229][105620] Updated weights for policy 1, policy_version 1642246 (0.0007) [2023-12-27 03:20:02,293][105620] Updated weights for policy 1, policy_version 1642256 (0.0006) [2023-12-27 03:20:02,354][105620] Updated weights for policy 1, policy_version 1642266 (0.0007) [2023-12-27 03:20:02,611][105692] Updated weights for policy 0, policy_version 1638932 (0.0009) [2023-12-27 03:20:02,663][105692] Updated weights for policy 0, policy_version 1638942 (0.0010) [2023-12-27 03:20:02,724][105692] Updated weights for policy 0, policy_version 1638952 (0.0010) [2023-12-27 03:20:03,058][105620] Updated weights for policy 1, policy_version 1642276 (0.0009) [2023-12-27 03:20:03,116][105620] Updated weights for policy 1, policy_version 1642286 (0.0010) [2023-12-27 03:20:03,169][105620] Updated weights for policy 1, policy_version 1642296 (0.0010) [2023-12-27 03:20:03,346][105692] Updated weights for policy 0, policy_version 1638962 (0.0006) [2023-12-27 03:20:03,417][105692] Updated weights for policy 0, policy_version 1638972 (0.0011) [2023-12-27 03:20:03,474][105692] Updated weights for policy 0, policy_version 1638982 (0.0005) [2023-12-27 03:20:03,522][105692] Updated weights for policy 0, policy_version 1638992 (0.0005) [2023-12-27 03:20:03,806][105620] Updated weights for policy 1, policy_version 1642307 (0.0009) [2023-12-27 03:20:03,867][105620] Updated weights for policy 1, policy_version 1642317 (0.0006) [2023-12-27 03:20:03,929][105620] Updated weights for policy 1, policy_version 1642327 (0.0008) [2023-12-27 03:20:04,103][105692] Updated weights for policy 0, policy_version 1639002 (0.0011) [2023-12-27 03:20:04,164][105692] Updated weights for policy 0, policy_version 1639012 (0.0010) [2023-12-27 03:20:04,212][105692] Updated weights for policy 0, policy_version 1639022 (0.0009) [2023-12-27 03:20:04,630][105620] Updated weights for policy 1, policy_version 1642337 (0.0008) [2023-12-27 03:20:04,683][105620] Updated weights for policy 1, policy_version 1642347 (0.0007) [2023-12-27 03:20:04,735][105620] Updated weights for policy 1, policy_version 1642357 (0.0010) [2023-12-27 03:20:04,785][105620] Updated weights for policy 1, policy_version 1642367 (0.0010) [2023-12-27 03:20:04,908][105692] Updated weights for policy 0, policy_version 1639032 (0.0007) [2023-12-27 03:20:04,960][105692] Updated weights for policy 0, policy_version 1639042 (0.0008) [2023-12-27 03:20:05,016][105692] Updated weights for policy 0, policy_version 1639052 (0.0008) [2023-12-27 03:20:05,516][105620] Updated weights for policy 1, policy_version 1642377 (0.0010) [2023-12-27 03:20:05,567][105620] Updated weights for policy 1, policy_version 1642387 (0.0010) [2023-12-27 03:20:05,622][105620] Updated weights for policy 1, policy_version 1642397 (0.0010) [2023-12-27 03:20:05,628][105692] Updated weights for policy 0, policy_version 1639062 (0.0008) [2023-12-27 03:20:05,691][105692] Updated weights for policy 0, policy_version 1639072 (0.0008) [2023-12-27 03:20:05,755][105692] Updated weights for policy 0, policy_version 1639082 (0.0008) [2023-12-27 03:20:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 840179712. Throughput: 0: 9445.3, 1: 9804.0. Samples: 840167332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:20:06,062][104569] Avg episode reward: [(0, '8715.079'), (1, '9265.283')] [2023-12-27 03:20:06,338][105620] Updated weights for policy 1, policy_version 1642407 (0.0009) [2023-12-27 03:20:06,375][105692] Updated weights for policy 0, policy_version 1639092 (0.0008) [2023-12-27 03:20:06,401][105620] Updated weights for policy 1, policy_version 1642417 (0.0007) [2023-12-27 03:20:06,441][105692] Updated weights for policy 0, policy_version 1639102 (0.0011) [2023-12-27 03:20:06,466][105620] Updated weights for policy 1, policy_version 1642427 (0.0006) [2023-12-27 03:20:06,508][105692] Updated weights for policy 0, policy_version 1639112 (0.0011) [2023-12-27 03:20:07,084][105620] Updated weights for policy 1, policy_version 1642437 (0.0008) [2023-12-27 03:20:07,144][105620] Updated weights for policy 1, policy_version 1642447 (0.0011) [2023-12-27 03:20:07,209][105620] Updated weights for policy 1, policy_version 1642457 (0.0007) [2023-12-27 03:20:07,250][105692] Updated weights for policy 0, policy_version 1639122 (0.0011) [2023-12-27 03:20:07,299][105692] Updated weights for policy 0, policy_version 1639132 (0.0009) [2023-12-27 03:20:07,358][105692] Updated weights for policy 0, policy_version 1639142 (0.0011) [2023-12-27 03:20:07,417][105692] Updated weights for policy 0, policy_version 1639152 (0.0011) [2023-12-27 03:20:07,741][105620] Updated weights for policy 1, policy_version 1642467 (0.0006) [2023-12-27 03:20:07,788][105620] Updated weights for policy 1, policy_version 1642477 (0.0005) [2023-12-27 03:20:07,836][105620] Updated weights for policy 1, policy_version 1642487 (0.0005) [2023-12-27 03:20:08,197][105692] Updated weights for policy 0, policy_version 1639162 (0.0009) [2023-12-27 03:20:08,248][105692] Updated weights for policy 0, policy_version 1639172 (0.0008) [2023-12-27 03:20:08,311][105692] Updated weights for policy 0, policy_version 1639182 (0.0008) [2023-12-27 03:20:08,491][105620] Updated weights for policy 1, policy_version 1642497 (0.0005) [2023-12-27 03:20:08,538][105620] Updated weights for policy 1, policy_version 1642507 (0.0005) [2023-12-27 03:20:08,587][105620] Updated weights for policy 1, policy_version 1642517 (0.0005) [2023-12-27 03:20:08,649][105620] Updated weights for policy 1, policy_version 1642527 (0.0008) [2023-12-27 03:20:09,176][105692] Updated weights for policy 0, policy_version 1639192 (0.0009) [2023-12-27 03:20:09,228][105620] Updated weights for policy 1, policy_version 1642537 (0.0007) [2023-12-27 03:20:09,229][105692] Updated weights for policy 0, policy_version 1639203 (0.0009) [2023-12-27 03:20:09,288][105692] Updated weights for policy 0, policy_version 1639213 (0.0009) [2023-12-27 03:20:09,293][105620] Updated weights for policy 1, policy_version 1642547 (0.0007) [2023-12-27 03:20:09,359][105620] Updated weights for policy 1, policy_version 1642557 (0.0008) [2023-12-27 03:20:10,044][105692] Updated weights for policy 0, policy_version 1639223 (0.0010) [2023-12-27 03:20:10,092][105620] Updated weights for policy 1, policy_version 1642567 (0.0008) [2023-12-27 03:20:10,101][105692] Updated weights for policy 0, policy_version 1639233 (0.0011) [2023-12-27 03:20:10,152][105620] Updated weights for policy 1, policy_version 1642577 (0.0008) [2023-12-27 03:20:10,161][105692] Updated weights for policy 0, policy_version 1639243 (0.0011) [2023-12-27 03:20:10,215][105620] Updated weights for policy 1, policy_version 1642587 (0.0008) [2023-12-27 03:20:10,899][105692] Updated weights for policy 0, policy_version 1639253 (0.0011) [2023-12-27 03:20:10,951][105692] Updated weights for policy 0, policy_version 1639263 (0.0010) [2023-12-27 03:20:10,956][105620] Updated weights for policy 1, policy_version 1642597 (0.0007) [2023-12-27 03:20:11,006][105692] Updated weights for policy 0, policy_version 1639273 (0.0011) [2023-12-27 03:20:11,008][105620] Updated weights for policy 1, policy_version 1642607 (0.0007) [2023-12-27 03:20:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 840278016. Throughput: 0: 9524.7, 1: 9839.3. Samples: 840287164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:20:11,062][104569] Avg episode reward: [(0, '8987.881'), (1, '9265.358')] [2023-12-27 03:20:11,071][105620] Updated weights for policy 1, policy_version 1642617 (0.0008) [2023-12-27 03:20:11,778][105620] Updated weights for policy 1, policy_version 1642627 (0.0007) [2023-12-27 03:20:11,838][105620] Updated weights for policy 1, policy_version 1642637 (0.0006) [2023-12-27 03:20:11,866][105692] Updated weights for policy 0, policy_version 1639283 (0.0008) [2023-12-27 03:20:11,889][105620] Updated weights for policy 1, policy_version 1642647 (0.0007) [2023-12-27 03:20:11,924][105692] Updated weights for policy 0, policy_version 1639293 (0.0008) [2023-12-27 03:20:11,984][105692] Updated weights for policy 0, policy_version 1639303 (0.0008) [2023-12-27 03:20:12,649][105692] Updated weights for policy 0, policy_version 1639313 (0.0007) [2023-12-27 03:20:12,661][105620] Updated weights for policy 1, policy_version 1642657 (0.0007) [2023-12-27 03:20:12,705][105692] Updated weights for policy 0, policy_version 1639323 (0.0007) [2023-12-27 03:20:12,715][105620] Updated weights for policy 1, policy_version 1642667 (0.0006) [2023-12-27 03:20:12,765][105692] Updated weights for policy 0, policy_version 1639333 (0.0008) [2023-12-27 03:20:12,772][105620] Updated weights for policy 1, policy_version 1642677 (0.0006) [2023-12-27 03:20:12,825][105620] Updated weights for policy 1, policy_version 1642687 (0.0007) [2023-12-27 03:20:12,825][105692] Updated weights for policy 0, policy_version 1639343 (0.0009) [2023-12-27 03:20:13,479][105620] Updated weights for policy 1, policy_version 1642697 (0.0007) [2023-12-27 03:20:13,541][105620] Updated weights for policy 1, policy_version 1642707 (0.0005) [2023-12-27 03:20:13,596][105620] Updated weights for policy 1, policy_version 1642717 (0.0005) [2023-12-27 03:20:13,647][105692] Updated weights for policy 0, policy_version 1639353 (0.0009) [2023-12-27 03:20:13,704][105692] Updated weights for policy 0, policy_version 1639363 (0.0009) [2023-12-27 03:20:13,766][105692] Updated weights for policy 0, policy_version 1639373 (0.0009) [2023-12-27 03:20:14,284][105620] Updated weights for policy 1, policy_version 1642727 (0.0007) [2023-12-27 03:20:14,343][105620] Updated weights for policy 1, policy_version 1642737 (0.0005) [2023-12-27 03:20:14,392][105620] Updated weights for policy 1, policy_version 1642747 (0.0006) [2023-12-27 03:20:14,431][105692] Updated weights for policy 0, policy_version 1639383 (0.0007) [2023-12-27 03:20:14,477][105692] Updated weights for policy 0, policy_version 1639393 (0.0005) [2023-12-27 03:20:14,534][105692] Updated weights for policy 0, policy_version 1639403 (0.0005) [2023-12-27 03:20:15,127][105620] Updated weights for policy 1, policy_version 1642757 (0.0006) [2023-12-27 03:20:15,179][105692] Updated weights for policy 0, policy_version 1639413 (0.0007) [2023-12-27 03:20:15,192][105620] Updated weights for policy 1, policy_version 1642767 (0.0006) [2023-12-27 03:20:15,247][105692] Updated weights for policy 0, policy_version 1639423 (0.0008) [2023-12-27 03:20:15,257][105620] Updated weights for policy 1, policy_version 1642777 (0.0006) [2023-12-27 03:20:15,315][105692] Updated weights for policy 0, policy_version 1639433 (0.0009) [2023-12-27 03:20:15,880][105620] Updated weights for policy 1, policy_version 1642787 (0.0008) [2023-12-27 03:20:15,948][105620] Updated weights for policy 1, policy_version 1642797 (0.0009) [2023-12-27 03:20:16,005][105620] Updated weights for policy 1, policy_version 1642807 (0.0008) [2023-12-27 03:20:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 840376320. Throughput: 0: 9537.1, 1: 9806.4. Samples: 840344688. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:20:16,062][104569] Avg episode reward: [(0, '9077.742'), (1, '9356.877')] [2023-12-27 03:20:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001642816_420618240.pth... [2023-12-27 03:20:16,071][105692] Updated weights for policy 0, policy_version 1639443 (0.0006) [2023-12-27 03:20:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001641664_420323328.pth [2023-12-27 03:20:16,127][105692] Updated weights for policy 0, policy_version 1639453 (0.0009) [2023-12-27 03:20:16,175][105692] Updated weights for policy 0, policy_version 1639463 (0.0009) [2023-12-27 03:20:16,226][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001639472_419766272.pth... [2023-12-27 03:20:16,230][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001638352_419479552.pth [2023-12-27 03:20:16,694][105620] Updated weights for policy 1, policy_version 1642817 (0.0009) [2023-12-27 03:20:16,743][105620] Updated weights for policy 1, policy_version 1642827 (0.0005) [2023-12-27 03:20:16,791][105620] Updated weights for policy 1, policy_version 1642837 (0.0010) [2023-12-27 03:20:16,842][105620] Updated weights for policy 1, policy_version 1642847 (0.0010) [2023-12-27 03:20:16,858][105692] Updated weights for policy 0, policy_version 1639473 (0.0009) [2023-12-27 03:20:16,914][105692] Updated weights for policy 0, policy_version 1639483 (0.0008) [2023-12-27 03:20:16,962][105692] Updated weights for policy 0, policy_version 1639493 (0.0008) [2023-12-27 03:20:17,010][105692] Updated weights for policy 0, policy_version 1639503 (0.0008) [2023-12-27 03:20:17,578][105620] Updated weights for policy 1, policy_version 1642857 (0.0006) [2023-12-27 03:20:17,633][105620] Updated weights for policy 1, policy_version 1642867 (0.0005) [2023-12-27 03:20:17,689][105620] Updated weights for policy 1, policy_version 1642877 (0.0009) [2023-12-27 03:20:17,827][105692] Updated weights for policy 0, policy_version 1639513 (0.0010) [2023-12-27 03:20:17,883][105692] Updated weights for policy 0, policy_version 1639523 (0.0011) [2023-12-27 03:20:17,948][105692] Updated weights for policy 0, policy_version 1639533 (0.0010) [2023-12-27 03:20:18,217][105620] Updated weights for policy 1, policy_version 1642887 (0.0007) [2023-12-27 03:20:18,279][105620] Updated weights for policy 1, policy_version 1642897 (0.0005) [2023-12-27 03:20:18,344][105620] Updated weights for policy 1, policy_version 1642907 (0.0008) [2023-12-27 03:20:18,659][105692] Updated weights for policy 0, policy_version 1639543 (0.0011) [2023-12-27 03:20:18,721][105692] Updated weights for policy 0, policy_version 1639553 (0.0011) [2023-12-27 03:20:18,780][105692] Updated weights for policy 0, policy_version 1639563 (0.0011) [2023-12-27 03:20:19,028][105620] Updated weights for policy 1, policy_version 1642917 (0.0010) [2023-12-27 03:20:19,096][105620] Updated weights for policy 1, policy_version 1642927 (0.0010) [2023-12-27 03:20:19,151][105620] Updated weights for policy 1, policy_version 1642937 (0.0010) [2023-12-27 03:20:19,552][105692] Updated weights for policy 0, policy_version 1639573 (0.0008) [2023-12-27 03:20:19,618][105692] Updated weights for policy 0, policy_version 1639583 (0.0007) [2023-12-27 03:20:19,678][105692] Updated weights for policy 0, policy_version 1639593 (0.0011) [2023-12-27 03:20:19,800][105620] Updated weights for policy 1, policy_version 1642947 (0.0010) [2023-12-27 03:20:19,871][105620] Updated weights for policy 1, policy_version 1642957 (0.0008) [2023-12-27 03:20:19,931][105620] Updated weights for policy 1, policy_version 1642967 (0.0008) [2023-12-27 03:20:20,348][105692] Updated weights for policy 0, policy_version 1639603 (0.0008) [2023-12-27 03:20:20,407][105692] Updated weights for policy 0, policy_version 1639613 (0.0011) [2023-12-27 03:20:20,464][105692] Updated weights for policy 0, policy_version 1639623 (0.0011) [2023-12-27 03:20:20,656][105620] Updated weights for policy 1, policy_version 1642977 (0.0008) [2023-12-27 03:20:20,722][105620] Updated weights for policy 1, policy_version 1642987 (0.0006) [2023-12-27 03:20:20,781][105620] Updated weights for policy 1, policy_version 1642997 (0.0006) [2023-12-27 03:20:20,847][105620] Updated weights for policy 1, policy_version 1643007 (0.0006) [2023-12-27 03:20:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 840474624. Throughput: 0: 9574.5, 1: 9845.5. Samples: 840464144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:20:21,062][104569] Avg episode reward: [(0, '9262.953'), (1, '9174.310')] [2023-12-27 03:20:21,135][105692] Updated weights for policy 0, policy_version 1639633 (0.0010) [2023-12-27 03:20:21,198][105692] Updated weights for policy 0, policy_version 1639643 (0.0008) [2023-12-27 03:20:21,259][105692] Updated weights for policy 0, policy_version 1639653 (0.0009) [2023-12-27 03:20:21,328][105692] Updated weights for policy 0, policy_version 1639663 (0.0008) [2023-12-27 03:20:21,544][105620] Updated weights for policy 1, policy_version 1643017 (0.0010) [2023-12-27 03:20:21,606][105620] Updated weights for policy 1, policy_version 1643027 (0.0009) [2023-12-27 03:20:21,679][105620] Updated weights for policy 1, policy_version 1643037 (0.0008) [2023-12-27 03:20:22,217][105692] Updated weights for policy 0, policy_version 1639673 (0.0006) [2023-12-27 03:20:22,285][105692] Updated weights for policy 0, policy_version 1639683 (0.0007) [2023-12-27 03:20:22,356][105692] Updated weights for policy 0, policy_version 1639693 (0.0006) [2023-12-27 03:20:22,385][105620] Updated weights for policy 1, policy_version 1643047 (0.0009) [2023-12-27 03:20:22,450][105620] Updated weights for policy 1, policy_version 1643057 (0.0009) [2023-12-27 03:20:22,523][105620] Updated weights for policy 1, policy_version 1643067 (0.0009) [2023-12-27 03:20:23,054][105692] Updated weights for policy 0, policy_version 1639703 (0.0009) [2023-12-27 03:20:23,106][105692] Updated weights for policy 0, policy_version 1639713 (0.0009) [2023-12-27 03:20:23,158][105692] Updated weights for policy 0, policy_version 1639723 (0.0009) [2023-12-27 03:20:23,229][105620] Updated weights for policy 1, policy_version 1643077 (0.0009) [2023-12-27 03:20:23,285][105620] Updated weights for policy 1, policy_version 1643087 (0.0009) [2023-12-27 03:20:23,343][105620] Updated weights for policy 1, policy_version 1643097 (0.0008) [2023-12-27 03:20:23,820][105692] Updated weights for policy 0, policy_version 1639733 (0.0010) [2023-12-27 03:20:23,886][105692] Updated weights for policy 0, policy_version 1639743 (0.0009) [2023-12-27 03:20:23,947][105692] Updated weights for policy 0, policy_version 1639753 (0.0009) [2023-12-27 03:20:24,169][105620] Updated weights for policy 1, policy_version 1643107 (0.0007) [2023-12-27 03:20:24,236][105620] Updated weights for policy 1, policy_version 1643117 (0.0007) [2023-12-27 03:20:24,298][105620] Updated weights for policy 1, policy_version 1643127 (0.0007) [2023-12-27 03:20:24,596][105692] Updated weights for policy 0, policy_version 1639763 (0.0009) [2023-12-27 03:20:24,650][105692] Updated weights for policy 0, policy_version 1639774 (0.0010) [2023-12-27 03:20:24,710][105692] Updated weights for policy 0, policy_version 1639785 (0.0010) [2023-12-27 03:20:24,905][105620] Updated weights for policy 1, policy_version 1643137 (0.0006) [2023-12-27 03:20:24,951][105620] Updated weights for policy 1, policy_version 1643147 (0.0008) [2023-12-27 03:20:25,001][105620] Updated weights for policy 1, policy_version 1643157 (0.0009) [2023-12-27 03:20:25,060][105620] Updated weights for policy 1, policy_version 1643167 (0.0009) [2023-12-27 03:20:25,552][105692] Updated weights for policy 0, policy_version 1639795 (0.0009) [2023-12-27 03:20:25,614][105692] Updated weights for policy 0, policy_version 1639805 (0.0009) [2023-12-27 03:20:25,673][105692] Updated weights for policy 0, policy_version 1639815 (0.0011) [2023-12-27 03:20:25,682][105620] Updated weights for policy 1, policy_version 1643177 (0.0006) [2023-12-27 03:20:25,747][105620] Updated weights for policy 1, policy_version 1643187 (0.0006) [2023-12-27 03:20:25,805][105620] Updated weights for policy 1, policy_version 1643197 (0.0010) [2023-12-27 03:20:26,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 840572928. Throughput: 0: 9534.2, 1: 9830.4. Samples: 840580584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:20:26,063][104569] Avg episode reward: [(0, '8896.433'), (1, '9174.347')] [2023-12-27 03:20:26,304][105692] Updated weights for policy 0, policy_version 1639825 (0.0011) [2023-12-27 03:20:26,368][105692] Updated weights for policy 0, policy_version 1639835 (0.0010) [2023-12-27 03:20:26,426][105620] Updated weights for policy 1, policy_version 1643207 (0.0010) [2023-12-27 03:20:26,430][105692] Updated weights for policy 0, policy_version 1639845 (0.0011) [2023-12-27 03:20:26,482][105620] Updated weights for policy 1, policy_version 1643217 (0.0010) [2023-12-27 03:20:26,485][105692] Updated weights for policy 0, policy_version 1639855 (0.0010) [2023-12-27 03:20:26,552][105620] Updated weights for policy 1, policy_version 1643227 (0.0006) [2023-12-27 03:20:27,098][105692] Updated weights for policy 0, policy_version 1639865 (0.0006) [2023-12-27 03:20:27,110][105620] Updated weights for policy 1, policy_version 1643237 (0.0008) [2023-12-27 03:20:27,154][105692] Updated weights for policy 0, policy_version 1639875 (0.0009) [2023-12-27 03:20:27,165][105620] Updated weights for policy 1, policy_version 1643247 (0.0005) [2023-12-27 03:20:27,219][105692] Updated weights for policy 0, policy_version 1639885 (0.0006) [2023-12-27 03:20:27,229][105620] Updated weights for policy 1, policy_version 1643257 (0.0007) [2023-12-27 03:20:27,853][105692] Updated weights for policy 0, policy_version 1639895 (0.0005) [2023-12-27 03:20:27,896][105692] Updated weights for policy 0, policy_version 1639905 (0.0005) [2023-12-27 03:20:27,931][105620] Updated weights for policy 1, policy_version 1643267 (0.0010) [2023-12-27 03:20:27,954][105692] Updated weights for policy 0, policy_version 1639915 (0.0005) [2023-12-27 03:20:27,982][105620] Updated weights for policy 1, policy_version 1643277 (0.0010) [2023-12-27 03:20:28,032][105620] Updated weights for policy 1, policy_version 1643287 (0.0010) [2023-12-27 03:20:28,563][105692] Updated weights for policy 0, policy_version 1639925 (0.0007) [2023-12-27 03:20:28,613][105692] Updated weights for policy 0, policy_version 1639935 (0.0010) [2023-12-27 03:20:28,667][105692] Updated weights for policy 0, policy_version 1639945 (0.0010) [2023-12-27 03:20:28,769][105620] Updated weights for policy 1, policy_version 1643297 (0.0010) [2023-12-27 03:20:28,833][105620] Updated weights for policy 1, policy_version 1643307 (0.0005) [2023-12-27 03:20:28,895][105620] Updated weights for policy 1, policy_version 1643317 (0.0005) [2023-12-27 03:20:28,951][105620] Updated weights for policy 1, policy_version 1643327 (0.0005) [2023-12-27 03:20:29,301][105692] Updated weights for policy 0, policy_version 1639955 (0.0010) [2023-12-27 03:20:29,361][105692] Updated weights for policy 0, policy_version 1639965 (0.0010) [2023-12-27 03:20:29,422][105692] Updated weights for policy 0, policy_version 1639975 (0.0010) [2023-12-27 03:20:29,594][105620] Updated weights for policy 1, policy_version 1643337 (0.0010) [2023-12-27 03:20:29,650][105620] Updated weights for policy 1, policy_version 1643347 (0.0010) [2023-12-27 03:20:29,719][105620] Updated weights for policy 1, policy_version 1643357 (0.0010) [2023-12-27 03:20:30,076][105692] Updated weights for policy 0, policy_version 1639985 (0.0007) [2023-12-27 03:20:30,125][105692] Updated weights for policy 0, policy_version 1639995 (0.0010) [2023-12-27 03:20:30,190][105692] Updated weights for policy 0, policy_version 1640005 (0.0010) [2023-12-27 03:20:30,246][105692] Updated weights for policy 0, policy_version 1640015 (0.0010) [2023-12-27 03:20:30,433][105620] Updated weights for policy 1, policy_version 1643367 (0.0007) [2023-12-27 03:20:30,491][105620] Updated weights for policy 1, policy_version 1643377 (0.0006) [2023-12-27 03:20:30,537][105620] Updated weights for policy 1, policy_version 1643387 (0.0010) [2023-12-27 03:20:30,863][105692] Updated weights for policy 0, policy_version 1640025 (0.0010) [2023-12-27 03:20:30,917][105692] Updated weights for policy 0, policy_version 1640035 (0.0010) [2023-12-27 03:20:30,964][105692] Updated weights for policy 0, policy_version 1640045 (0.0010) [2023-12-27 03:20:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 840679424. Throughput: 0: 9609.0, 1: 9951.5. Samples: 840645200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:20:31,062][104569] Avg episode reward: [(0, '8444.091'), (1, '9081.586')] [2023-12-27 03:20:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001640048_419913728.pth... [2023-12-27 03:20:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001643392_420765696.pth... [2023-12-27 03:20:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001638896_419618816.pth [2023-12-27 03:20:31,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001642208_420462592.pth [2023-12-27 03:20:31,250][105620] Updated weights for policy 1, policy_version 1643397 (0.0010) [2023-12-27 03:20:31,306][105620] Updated weights for policy 1, policy_version 1643407 (0.0010) [2023-12-27 03:20:31,362][105620] Updated weights for policy 1, policy_version 1643417 (0.0010) [2023-12-27 03:20:31,754][105692] Updated weights for policy 0, policy_version 1640055 (0.0009) [2023-12-27 03:20:31,810][105692] Updated weights for policy 0, policy_version 1640065 (0.0008) [2023-12-27 03:20:31,868][105692] Updated weights for policy 0, policy_version 1640075 (0.0008) [2023-12-27 03:20:32,120][105620] Updated weights for policy 1, policy_version 1643427 (0.0009) [2023-12-27 03:20:32,194][105620] Updated weights for policy 1, policy_version 1643437 (0.0008) [2023-12-27 03:20:32,253][105620] Updated weights for policy 1, policy_version 1643447 (0.0010) [2023-12-27 03:20:32,591][105692] Updated weights for policy 0, policy_version 1640085 (0.0008) [2023-12-27 03:20:32,657][105692] Updated weights for policy 0, policy_version 1640095 (0.0008) [2023-12-27 03:20:32,721][105692] Updated weights for policy 0, policy_version 1640105 (0.0008) [2023-12-27 03:20:32,954][105620] Updated weights for policy 1, policy_version 1643457 (0.0011) [2023-12-27 03:20:33,014][105620] Updated weights for policy 1, policy_version 1643467 (0.0007) [2023-12-27 03:20:33,060][105620] Updated weights for policy 1, policy_version 1643477 (0.0005) [2023-12-27 03:20:33,116][105620] Updated weights for policy 1, policy_version 1643487 (0.0005) [2023-12-27 03:20:33,279][105692] Updated weights for policy 0, policy_version 1640115 (0.0008) [2023-12-27 03:20:33,337][105692] Updated weights for policy 0, policy_version 1640125 (0.0010) [2023-12-27 03:20:33,395][105692] Updated weights for policy 0, policy_version 1640135 (0.0010) [2023-12-27 03:20:33,779][105620] Updated weights for policy 1, policy_version 1643497 (0.0006) [2023-12-27 03:20:33,830][105620] Updated weights for policy 1, policy_version 1643507 (0.0005) [2023-12-27 03:20:33,875][105620] Updated weights for policy 1, policy_version 1643517 (0.0007) [2023-12-27 03:20:34,131][105692] Updated weights for policy 0, policy_version 1640145 (0.0010) [2023-12-27 03:20:34,199][105692] Updated weights for policy 0, policy_version 1640155 (0.0009) [2023-12-27 03:20:34,267][105692] Updated weights for policy 0, policy_version 1640165 (0.0005) [2023-12-27 03:20:34,328][105692] Updated weights for policy 0, policy_version 1640175 (0.0006) [2023-12-27 03:20:34,605][105620] Updated weights for policy 1, policy_version 1643527 (0.0007) [2023-12-27 03:20:34,679][105620] Updated weights for policy 1, policy_version 1643537 (0.0006) [2023-12-27 03:20:34,737][105620] Updated weights for policy 1, policy_version 1643547 (0.0009) [2023-12-27 03:20:34,891][105692] Updated weights for policy 0, policy_version 1640185 (0.0006) [2023-12-27 03:20:34,960][105692] Updated weights for policy 0, policy_version 1640195 (0.0010) [2023-12-27 03:20:35,029][105692] Updated weights for policy 0, policy_version 1640205 (0.0009) [2023-12-27 03:20:35,384][105620] Updated weights for policy 1, policy_version 1643557 (0.0009) [2023-12-27 03:20:35,447][105620] Updated weights for policy 1, policy_version 1643567 (0.0010) [2023-12-27 03:20:35,499][105620] Updated weights for policy 1, policy_version 1643577 (0.0009) [2023-12-27 03:20:35,554][105692] Updated weights for policy 0, policy_version 1640215 (0.0005) [2023-12-27 03:20:35,603][105692] Updated weights for policy 0, policy_version 1640225 (0.0005) [2023-12-27 03:20:35,648][105692] Updated weights for policy 0, policy_version 1640235 (0.0005) [2023-12-27 03:20:36,062][104569] Fps is (10 sec: 20480.6, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 840777728. Throughput: 0: 9736.1, 1: 9939.5. Samples: 840765632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:20:36,063][104569] Avg episode reward: [(0, '8630.053'), (1, '8989.965')] [2023-12-27 03:20:36,301][105692] Updated weights for policy 0, policy_version 1640245 (0.0006) [2023-12-27 03:20:36,313][105620] Updated weights for policy 1, policy_version 1643587 (0.0010) [2023-12-27 03:20:36,371][105692] Updated weights for policy 0, policy_version 1640255 (0.0005) [2023-12-27 03:20:36,379][105620] Updated weights for policy 1, policy_version 1643597 (0.0009) [2023-12-27 03:20:36,438][105692] Updated weights for policy 0, policy_version 1640265 (0.0009) [2023-12-27 03:20:36,440][105620] Updated weights for policy 1, policy_version 1643607 (0.0006) [2023-12-27 03:20:37,124][105692] Updated weights for policy 0, policy_version 1640275 (0.0008) [2023-12-27 03:20:37,167][105620] Updated weights for policy 1, policy_version 1643617 (0.0007) [2023-12-27 03:20:37,177][105692] Updated weights for policy 0, policy_version 1640285 (0.0011) [2023-12-27 03:20:37,230][105692] Updated weights for policy 0, policy_version 1640295 (0.0011) [2023-12-27 03:20:37,231][105620] Updated weights for policy 1, policy_version 1643627 (0.0006) [2023-12-27 03:20:37,293][105620] Updated weights for policy 1, policy_version 1643637 (0.0006) [2023-12-27 03:20:37,349][105620] Updated weights for policy 1, policy_version 1643647 (0.0008) [2023-12-27 03:20:37,895][105692] Updated weights for policy 0, policy_version 1640305 (0.0011) [2023-12-27 03:20:37,941][105692] Updated weights for policy 0, policy_version 1640315 (0.0010) [2023-12-27 03:20:37,985][105692] Updated weights for policy 0, policy_version 1640325 (0.0010) [2023-12-27 03:20:38,034][105692] Updated weights for policy 0, policy_version 1640335 (0.0011) [2023-12-27 03:20:38,104][105620] Updated weights for policy 1, policy_version 1643657 (0.0006) [2023-12-27 03:20:38,165][105620] Updated weights for policy 1, policy_version 1643667 (0.0008) [2023-12-27 03:20:38,222][105620] Updated weights for policy 1, policy_version 1643677 (0.0009) [2023-12-27 03:20:38,712][105692] Updated weights for policy 0, policy_version 1640345 (0.0006) [2023-12-27 03:20:38,759][105692] Updated weights for policy 0, policy_version 1640355 (0.0005) [2023-12-27 03:20:38,826][105692] Updated weights for policy 0, policy_version 1640365 (0.0009) [2023-12-27 03:20:38,912][105620] Updated weights for policy 1, policy_version 1643688 (0.0009) [2023-12-27 03:20:38,977][105620] Updated weights for policy 1, policy_version 1643698 (0.0010) [2023-12-27 03:20:39,046][105620] Updated weights for policy 1, policy_version 1643708 (0.0010) [2023-12-27 03:20:39,537][105692] Updated weights for policy 0, policy_version 1640375 (0.0009) [2023-12-27 03:20:39,593][105692] Updated weights for policy 0, policy_version 1640385 (0.0008) [2023-12-27 03:20:39,658][105692] Updated weights for policy 0, policy_version 1640395 (0.0009) [2023-12-27 03:20:39,780][105620] Updated weights for policy 1, policy_version 1643718 (0.0007) [2023-12-27 03:20:39,844][105620] Updated weights for policy 1, policy_version 1643728 (0.0009) [2023-12-27 03:20:39,916][105620] Updated weights for policy 1, policy_version 1643738 (0.0010) [2023-12-27 03:20:40,420][105692] Updated weights for policy 0, policy_version 1640405 (0.0010) [2023-12-27 03:20:40,482][105692] Updated weights for policy 0, policy_version 1640415 (0.0008) [2023-12-27 03:20:40,535][105692] Updated weights for policy 0, policy_version 1640425 (0.0008) [2023-12-27 03:20:40,660][105620] Updated weights for policy 1, policy_version 1643748 (0.0011) [2023-12-27 03:20:40,713][105620] Updated weights for policy 1, policy_version 1643758 (0.0011) [2023-12-27 03:20:40,758][105620] Updated weights for policy 1, policy_version 1643768 (0.0011) [2023-12-27 03:20:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 840876032. Throughput: 0: 9878.5, 1: 9908.2. Samples: 840884008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:20:41,062][104569] Avg episode reward: [(0, '8624.136'), (1, '8993.925')] [2023-12-27 03:20:41,186][105692] Updated weights for policy 0, policy_version 1640435 (0.0009) [2023-12-27 03:20:41,252][105692] Updated weights for policy 0, policy_version 1640445 (0.0010) [2023-12-27 03:20:41,325][105692] Updated weights for policy 0, policy_version 1640455 (0.0011) [2023-12-27 03:20:41,500][105620] Updated weights for policy 1, policy_version 1643778 (0.0010) [2023-12-27 03:20:41,557][105620] Updated weights for policy 1, policy_version 1643788 (0.0006) [2023-12-27 03:20:41,614][105620] Updated weights for policy 1, policy_version 1643798 (0.0008) [2023-12-27 03:20:41,680][105620] Updated weights for policy 1, policy_version 1643808 (0.0008) [2023-12-27 03:20:42,028][105692] Updated weights for policy 0, policy_version 1640465 (0.0010) [2023-12-27 03:20:42,083][105692] Updated weights for policy 0, policy_version 1640475 (0.0005) [2023-12-27 03:20:42,136][105692] Updated weights for policy 0, policy_version 1640485 (0.0005) [2023-12-27 03:20:42,192][105692] Updated weights for policy 0, policy_version 1640495 (0.0007) [2023-12-27 03:20:42,458][105620] Updated weights for policy 1, policy_version 1643818 (0.0010) [2023-12-27 03:20:42,524][105620] Updated weights for policy 1, policy_version 1643828 (0.0010) [2023-12-27 03:20:42,577][105620] Updated weights for policy 1, policy_version 1643838 (0.0009) [2023-12-27 03:20:42,832][105692] Updated weights for policy 0, policy_version 1640505 (0.0009) [2023-12-27 03:20:42,886][105692] Updated weights for policy 0, policy_version 1640515 (0.0009) [2023-12-27 03:20:42,953][105692] Updated weights for policy 0, policy_version 1640525 (0.0009) [2023-12-27 03:20:43,332][105620] Updated weights for policy 1, policy_version 1643848 (0.0006) [2023-12-27 03:20:43,393][105620] Updated weights for policy 1, policy_version 1643858 (0.0006) [2023-12-27 03:20:43,457][105620] Updated weights for policy 1, policy_version 1643868 (0.0005) [2023-12-27 03:20:43,749][105692] Updated weights for policy 0, policy_version 1640535 (0.0008) [2023-12-27 03:20:43,808][105692] Updated weights for policy 0, policy_version 1640545 (0.0006) [2023-12-27 03:20:43,867][105692] Updated weights for policy 0, policy_version 1640555 (0.0008) [2023-12-27 03:20:44,048][105620] Updated weights for policy 1, policy_version 1643878 (0.0007) [2023-12-27 03:20:44,107][105620] Updated weights for policy 1, policy_version 1643888 (0.0011) [2023-12-27 03:20:44,156][105620] Updated weights for policy 1, policy_version 1643898 (0.0010) [2023-12-27 03:20:44,615][105692] Updated weights for policy 0, policy_version 1640565 (0.0009) [2023-12-27 03:20:44,660][105692] Updated weights for policy 0, policy_version 1640575 (0.0010) [2023-12-27 03:20:44,722][105692] Updated weights for policy 0, policy_version 1640585 (0.0011) [2023-12-27 03:20:44,757][105620] Updated weights for policy 1, policy_version 1643908 (0.0010) [2023-12-27 03:20:44,825][105620] Updated weights for policy 1, policy_version 1643918 (0.0009) [2023-12-27 03:20:44,888][105620] Updated weights for policy 1, policy_version 1643928 (0.0010) [2023-12-27 03:20:45,402][105692] Updated weights for policy 0, policy_version 1640595 (0.0009) [2023-12-27 03:20:45,475][105692] Updated weights for policy 0, policy_version 1640605 (0.0006) [2023-12-27 03:20:45,510][105620] Updated weights for policy 1, policy_version 1643938 (0.0011) [2023-12-27 03:20:45,531][105692] Updated weights for policy 0, policy_version 1640615 (0.0006) [2023-12-27 03:20:45,562][105620] Updated weights for policy 1, policy_version 1643948 (0.0008) [2023-12-27 03:20:45,619][105620] Updated weights for policy 1, policy_version 1643958 (0.0008) [2023-12-27 03:20:45,675][105620] Updated weights for policy 1, policy_version 1643968 (0.0011) [2023-12-27 03:20:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 840974336. Throughput: 0: 9905.6, 1: 9965.4. Samples: 840943000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:20:46,062][104569] Avg episode reward: [(0, '8717.831'), (1, '9085.215')] [2023-12-27 03:20:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001643968_420913152.pth... [2023-12-27 03:20:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001640624_420061184.pth... [2023-12-27 03:20:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001642816_420618240.pth [2023-12-27 03:20:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001639472_419766272.pth [2023-12-27 03:20:46,135][105692] Updated weights for policy 0, policy_version 1640625 (0.0009) [2023-12-27 03:20:46,190][105692] Updated weights for policy 0, policy_version 1640635 (0.0011) [2023-12-27 03:20:46,248][105692] Updated weights for policy 0, policy_version 1640645 (0.0010) [2023-12-27 03:20:46,306][105692] Updated weights for policy 0, policy_version 1640655 (0.0010) [2023-12-27 03:20:46,420][105620] Updated weights for policy 1, policy_version 1643978 (0.0010) [2023-12-27 03:20:46,473][105620] Updated weights for policy 1, policy_version 1643988 (0.0010) [2023-12-27 03:20:46,532][105620] Updated weights for policy 1, policy_version 1643998 (0.0010) [2023-12-27 03:20:46,972][105692] Updated weights for policy 0, policy_version 1640665 (0.0008) [2023-12-27 03:20:47,018][105692] Updated weights for policy 0, policy_version 1640675 (0.0008) [2023-12-27 03:20:47,070][105692] Updated weights for policy 0, policy_version 1640685 (0.0008) [2023-12-27 03:20:47,206][105620] Updated weights for policy 1, policy_version 1644008 (0.0011) [2023-12-27 03:20:47,264][105620] Updated weights for policy 1, policy_version 1644018 (0.0010) [2023-12-27 03:20:47,329][105620] Updated weights for policy 1, policy_version 1644028 (0.0010) [2023-12-27 03:20:47,702][105692] Updated weights for policy 0, policy_version 1640695 (0.0007) [2023-12-27 03:20:47,751][105692] Updated weights for policy 0, policy_version 1640705 (0.0008) [2023-12-27 03:20:47,800][105692] Updated weights for policy 0, policy_version 1640715 (0.0010) [2023-12-27 03:20:47,942][105620] Updated weights for policy 1, policy_version 1644038 (0.0007) [2023-12-27 03:20:48,001][105620] Updated weights for policy 1, policy_version 1644048 (0.0008) [2023-12-27 03:20:48,057][105620] Updated weights for policy 1, policy_version 1644058 (0.0009) [2023-12-27 03:20:48,522][105692] Updated weights for policy 0, policy_version 1640725 (0.0011) [2023-12-27 03:20:48,584][105692] Updated weights for policy 0, policy_version 1640735 (0.0008) [2023-12-27 03:20:48,641][105692] Updated weights for policy 0, policy_version 1640745 (0.0006) [2023-12-27 03:20:48,731][105620] Updated weights for policy 1, policy_version 1644068 (0.0007) [2023-12-27 03:20:48,801][105620] Updated weights for policy 1, policy_version 1644078 (0.0011) [2023-12-27 03:20:48,866][105620] Updated weights for policy 1, policy_version 1644088 (0.0010) [2023-12-27 03:20:49,312][105692] Updated weights for policy 0, policy_version 1640755 (0.0006) [2023-12-27 03:20:49,376][105692] Updated weights for policy 0, policy_version 1640765 (0.0008) [2023-12-27 03:20:49,440][105692] Updated weights for policy 0, policy_version 1640775 (0.0008) [2023-12-27 03:20:49,632][105620] Updated weights for policy 1, policy_version 1644098 (0.0010) [2023-12-27 03:20:49,691][105620] Updated weights for policy 1, policy_version 1644108 (0.0010) [2023-12-27 03:20:49,757][105620] Updated weights for policy 1, policy_version 1644118 (0.0009) [2023-12-27 03:20:49,809][105620] Updated weights for policy 1, policy_version 1644128 (0.0010) [2023-12-27 03:20:50,239][105692] Updated weights for policy 0, policy_version 1640785 (0.0008) [2023-12-27 03:20:50,298][105692] Updated weights for policy 0, policy_version 1640795 (0.0008) [2023-12-27 03:20:50,357][105692] Updated weights for policy 0, policy_version 1640805 (0.0008) [2023-12-27 03:20:50,414][105692] Updated weights for policy 0, policy_version 1640815 (0.0006) [2023-12-27 03:20:50,540][105620] Updated weights for policy 1, policy_version 1644138 (0.0010) [2023-12-27 03:20:50,603][105620] Updated weights for policy 1, policy_version 1644148 (0.0011) [2023-12-27 03:20:50,656][105620] Updated weights for policy 1, policy_version 1644158 (0.0010) [2023-12-27 03:20:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 841072640. Throughput: 0: 9938.9, 1: 10018.5. Samples: 841065416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:20:51,062][104569] Avg episode reward: [(0, '8810.778'), (1, '9085.494')] [2023-12-27 03:20:51,136][105692] Updated weights for policy 0, policy_version 1640825 (0.0009) [2023-12-27 03:20:51,189][105692] Updated weights for policy 0, policy_version 1640835 (0.0010) [2023-12-27 03:20:51,243][105692] Updated weights for policy 0, policy_version 1640845 (0.0010) [2023-12-27 03:20:51,334][105620] Updated weights for policy 1, policy_version 1644168 (0.0007) [2023-12-27 03:20:51,403][105620] Updated weights for policy 1, policy_version 1644178 (0.0009) [2023-12-27 03:20:51,464][105620] Updated weights for policy 1, policy_version 1644188 (0.0009) [2023-12-27 03:20:52,024][105692] Updated weights for policy 0, policy_version 1640855 (0.0010) [2023-12-27 03:20:52,089][105692] Updated weights for policy 0, policy_version 1640865 (0.0007) [2023-12-27 03:20:52,156][105620] Updated weights for policy 1, policy_version 1644198 (0.0007) [2023-12-27 03:20:52,160][105692] Updated weights for policy 0, policy_version 1640875 (0.0009) [2023-12-27 03:20:52,208][105620] Updated weights for policy 1, policy_version 1644208 (0.0006) [2023-12-27 03:20:52,274][105620] Updated weights for policy 1, policy_version 1644218 (0.0008) [2023-12-27 03:20:52,867][105692] Updated weights for policy 0, policy_version 1640885 (0.0007) [2023-12-27 03:20:52,929][105692] Updated weights for policy 0, policy_version 1640895 (0.0005) [2023-12-27 03:20:52,971][105620] Updated weights for policy 1, policy_version 1644228 (0.0008) [2023-12-27 03:20:52,987][105692] Updated weights for policy 0, policy_version 1640905 (0.0006) [2023-12-27 03:20:53,030][105620] Updated weights for policy 1, policy_version 1644238 (0.0009) [2023-12-27 03:20:53,092][105620] Updated weights for policy 1, policy_version 1644248 (0.0008) [2023-12-27 03:20:53,602][105692] Updated weights for policy 0, policy_version 1640915 (0.0005) [2023-12-27 03:20:53,663][105692] Updated weights for policy 0, policy_version 1640925 (0.0007) [2023-12-27 03:20:53,725][105692] Updated weights for policy 0, policy_version 1640935 (0.0011) [2023-12-27 03:20:53,901][105620] Updated weights for policy 1, policy_version 1644258 (0.0009) [2023-12-27 03:20:53,959][105620] Updated weights for policy 1, policy_version 1644268 (0.0010) [2023-12-27 03:20:54,025][105620] Updated weights for policy 1, policy_version 1644278 (0.0010) [2023-12-27 03:20:54,086][105620] Updated weights for policy 1, policy_version 1644288 (0.0008) [2023-12-27 03:20:54,320][105692] Updated weights for policy 0, policy_version 1640945 (0.0006) [2023-12-27 03:20:54,386][105692] Updated weights for policy 0, policy_version 1640955 (0.0005) [2023-12-27 03:20:54,439][105692] Updated weights for policy 0, policy_version 1640965 (0.0005) [2023-12-27 03:20:54,501][105692] Updated weights for policy 0, policy_version 1640975 (0.0005) [2023-12-27 03:20:54,810][105620] Updated weights for policy 1, policy_version 1644298 (0.0009) [2023-12-27 03:20:54,864][105620] Updated weights for policy 1, policy_version 1644308 (0.0008) [2023-12-27 03:20:54,921][105620] Updated weights for policy 1, policy_version 1644318 (0.0008) [2023-12-27 03:20:55,136][105692] Updated weights for policy 0, policy_version 1640985 (0.0009) [2023-12-27 03:20:55,195][105692] Updated weights for policy 0, policy_version 1640995 (0.0009) [2023-12-27 03:20:55,248][105692] Updated weights for policy 0, policy_version 1641005 (0.0009) [2023-12-27 03:20:55,674][105620] Updated weights for policy 1, policy_version 1644328 (0.0009) [2023-12-27 03:20:55,721][105620] Updated weights for policy 1, policy_version 1644338 (0.0009) [2023-12-27 03:20:55,767][105620] Updated weights for policy 1, policy_version 1644348 (0.0008) [2023-12-27 03:20:55,993][105692] Updated weights for policy 0, policy_version 1641015 (0.0009) [2023-12-27 03:20:56,042][105692] Updated weights for policy 0, policy_version 1641025 (0.0009) [2023-12-27 03:20:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 841170944. Throughput: 0: 9990.1, 1: 9886.4. Samples: 841181608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:20:56,063][104569] Avg episode reward: [(0, '8986.525'), (1, '9177.021')] [2023-12-27 03:20:56,089][105692] Updated weights for policy 0, policy_version 1641035 (0.0009) [2023-12-27 03:20:56,555][105620] Updated weights for policy 1, policy_version 1644358 (0.0009) [2023-12-27 03:20:56,612][105620] Updated weights for policy 1, policy_version 1644368 (0.0009) [2023-12-27 03:20:56,671][105620] Updated weights for policy 1, policy_version 1644378 (0.0009) [2023-12-27 03:20:56,787][105692] Updated weights for policy 0, policy_version 1641045 (0.0010) [2023-12-27 03:20:56,835][105692] Updated weights for policy 0, policy_version 1641056 (0.0009) [2023-12-27 03:20:56,883][105692] Updated weights for policy 0, policy_version 1641066 (0.0009) [2023-12-27 03:20:57,422][105620] Updated weights for policy 1, policy_version 1644388 (0.0008) [2023-12-27 03:20:57,469][105620] Updated weights for policy 1, policy_version 1644398 (0.0005) [2023-12-27 03:20:57,519][105620] Updated weights for policy 1, policy_version 1644408 (0.0005) [2023-12-27 03:20:57,630][105692] Updated weights for policy 0, policy_version 1641076 (0.0007) [2023-12-27 03:20:57,684][105692] Updated weights for policy 0, policy_version 1641086 (0.0005) [2023-12-27 03:20:57,748][105692] Updated weights for policy 0, policy_version 1641096 (0.0010) [2023-12-27 03:20:58,078][105620] Updated weights for policy 1, policy_version 1644418 (0.0006) [2023-12-27 03:20:58,137][105620] Updated weights for policy 1, policy_version 1644428 (0.0009) [2023-12-27 03:20:58,212][105620] Updated weights for policy 1, policy_version 1644438 (0.0009) [2023-12-27 03:20:58,270][105620] Updated weights for policy 1, policy_version 1644448 (0.0010) [2023-12-27 03:20:58,485][105692] Updated weights for policy 0, policy_version 1641106 (0.0010) [2023-12-27 03:20:58,554][105692] Updated weights for policy 0, policy_version 1641116 (0.0006) [2023-12-27 03:20:58,620][105692] Updated weights for policy 0, policy_version 1641126 (0.0008) [2023-12-27 03:20:58,687][105692] Updated weights for policy 0, policy_version 1641136 (0.0009) [2023-12-27 03:20:59,142][105620] Updated weights for policy 1, policy_version 1644458 (0.0009) [2023-12-27 03:20:59,203][105620] Updated weights for policy 1, policy_version 1644468 (0.0009) [2023-12-27 03:20:59,264][105620] Updated weights for policy 1, policy_version 1644478 (0.0008) [2023-12-27 03:20:59,463][105692] Updated weights for policy 0, policy_version 1641146 (0.0008) [2023-12-27 03:20:59,510][105692] Updated weights for policy 0, policy_version 1641156 (0.0008) [2023-12-27 03:20:59,564][105692] Updated weights for policy 0, policy_version 1641166 (0.0006) [2023-12-27 03:20:59,982][105620] Updated weights for policy 1, policy_version 1644488 (0.0006) [2023-12-27 03:21:00,049][105620] Updated weights for policy 1, policy_version 1644498 (0.0006) [2023-12-27 03:21:00,112][105620] Updated weights for policy 1, policy_version 1644508 (0.0005) [2023-12-27 03:21:00,320][105692] Updated weights for policy 0, policy_version 1641176 (0.0009) [2023-12-27 03:21:00,384][105692] Updated weights for policy 0, policy_version 1641186 (0.0010) [2023-12-27 03:21:00,452][105692] Updated weights for policy 0, policy_version 1641196 (0.0010) [2023-12-27 03:21:00,639][105620] Updated weights for policy 1, policy_version 1644518 (0.0006) [2023-12-27 03:21:00,713][105620] Updated weights for policy 1, policy_version 1644528 (0.0006) [2023-12-27 03:21:00,779][105620] Updated weights for policy 1, policy_version 1644538 (0.0006) [2023-12-27 03:21:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 841269248. Throughput: 0: 10029.3, 1: 9869.7. Samples: 841240144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:21:01,063][104569] Avg episode reward: [(0, '8805.952'), (1, '9175.604')] [2023-12-27 03:21:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001644544_421060608.pth... [2023-12-27 03:21:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001643392_420765696.pth [2023-12-27 03:21:01,111][105692] Updated weights for policy 0, policy_version 1641206 (0.0009) [2023-12-27 03:21:01,173][105692] Updated weights for policy 0, policy_version 1641216 (0.0008) [2023-12-27 03:21:01,226][105692] Updated weights for policy 0, policy_version 1641226 (0.0009) [2023-12-27 03:21:01,266][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001641232_420216832.pth... [2023-12-27 03:21:01,269][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001640048_419913728.pth [2023-12-27 03:21:01,373][105620] Updated weights for policy 1, policy_version 1644548 (0.0008) [2023-12-27 03:21:01,425][105620] Updated weights for policy 1, policy_version 1644558 (0.0005) [2023-12-27 03:21:01,477][105620] Updated weights for policy 1, policy_version 1644568 (0.0006) [2023-12-27 03:21:02,068][105620] Updated weights for policy 1, policy_version 1644578 (0.0009) [2023-12-27 03:21:02,114][105692] Updated weights for policy 0, policy_version 1641236 (0.0010) [2023-12-27 03:21:02,117][105620] Updated weights for policy 1, policy_version 1644588 (0.0005) [2023-12-27 03:21:02,170][105620] Updated weights for policy 1, policy_version 1644598 (0.0006) [2023-12-27 03:21:02,175][105692] Updated weights for policy 0, policy_version 1641246 (0.0009) [2023-12-27 03:21:02,228][105620] Updated weights for policy 1, policy_version 1644608 (0.0010) [2023-12-27 03:21:02,237][105692] Updated weights for policy 0, policy_version 1641256 (0.0008) [2023-12-27 03:21:02,839][105620] Updated weights for policy 1, policy_version 1644618 (0.0010) [2023-12-27 03:21:02,887][105692] Updated weights for policy 0, policy_version 1641266 (0.0008) [2023-12-27 03:21:02,897][105620] Updated weights for policy 1, policy_version 1644628 (0.0009) [2023-12-27 03:21:02,947][105692] Updated weights for policy 0, policy_version 1641276 (0.0009) [2023-12-27 03:21:02,953][105620] Updated weights for policy 1, policy_version 1644638 (0.0007) [2023-12-27 03:21:03,009][105692] Updated weights for policy 0, policy_version 1641286 (0.0008) [2023-12-27 03:21:03,064][105692] Updated weights for policy 0, policy_version 1641296 (0.0010) [2023-12-27 03:21:03,578][105620] Updated weights for policy 1, policy_version 1644648 (0.0005) [2023-12-27 03:21:03,632][105620] Updated weights for policy 1, policy_version 1644658 (0.0006) [2023-12-27 03:21:03,683][105620] Updated weights for policy 1, policy_version 1644668 (0.0005) [2023-12-27 03:21:03,940][105692] Updated weights for policy 0, policy_version 1641306 (0.0008) [2023-12-27 03:21:04,001][105692] Updated weights for policy 0, policy_version 1641316 (0.0008) [2023-12-27 03:21:04,067][105692] Updated weights for policy 0, policy_version 1641326 (0.0008) [2023-12-27 03:21:04,384][105620] Updated weights for policy 1, policy_version 1644678 (0.0009) [2023-12-27 03:21:04,447][105620] Updated weights for policy 1, policy_version 1644688 (0.0010) [2023-12-27 03:21:04,503][105620] Updated weights for policy 1, policy_version 1644698 (0.0009) [2023-12-27 03:21:04,758][105692] Updated weights for policy 0, policy_version 1641336 (0.0006) [2023-12-27 03:21:04,825][105692] Updated weights for policy 0, policy_version 1641346 (0.0006) [2023-12-27 03:21:04,891][105692] Updated weights for policy 0, policy_version 1641356 (0.0006) [2023-12-27 03:21:05,241][105620] Updated weights for policy 1, policy_version 1644708 (0.0010) [2023-12-27 03:21:05,300][105620] Updated weights for policy 1, policy_version 1644718 (0.0010) [2023-12-27 03:21:05,358][105620] Updated weights for policy 1, policy_version 1644728 (0.0010) [2023-12-27 03:21:05,519][105692] Updated weights for policy 0, policy_version 1641366 (0.0007) [2023-12-27 03:21:05,564][105692] Updated weights for policy 0, policy_version 1641376 (0.0008) [2023-12-27 03:21:05,617][105692] Updated weights for policy 0, policy_version 1641386 (0.0008) [2023-12-27 03:21:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 841367552. Throughput: 0: 9976.3, 1: 9906.8. Samples: 841358880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:21:06,062][104569] Avg episode reward: [(0, '8623.347'), (1, '9175.488')] [2023-12-27 03:21:06,064][105620] Updated weights for policy 1, policy_version 1644738 (0.0010) [2023-12-27 03:21:06,121][105620] Updated weights for policy 1, policy_version 1644748 (0.0011) [2023-12-27 03:21:06,186][105620] Updated weights for policy 1, policy_version 1644758 (0.0011) [2023-12-27 03:21:06,249][105620] Updated weights for policy 1, policy_version 1644768 (0.0011) [2023-12-27 03:21:06,313][105692] Updated weights for policy 0, policy_version 1641396 (0.0009) [2023-12-27 03:21:06,370][105692] Updated weights for policy 0, policy_version 1641406 (0.0009) [2023-12-27 03:21:06,429][105692] Updated weights for policy 0, policy_version 1641416 (0.0009) [2023-12-27 03:21:06,997][105620] Updated weights for policy 1, policy_version 1644778 (0.0011) [2023-12-27 03:21:07,038][105692] Updated weights for policy 0, policy_version 1641426 (0.0008) [2023-12-27 03:21:07,049][105620] Updated weights for policy 1, policy_version 1644788 (0.0010) [2023-12-27 03:21:07,098][105692] Updated weights for policy 0, policy_version 1641436 (0.0011) [2023-12-27 03:21:07,106][105620] Updated weights for policy 1, policy_version 1644798 (0.0011) [2023-12-27 03:21:07,164][105692] Updated weights for policy 0, policy_version 1641446 (0.0010) [2023-12-27 03:21:07,225][105692] Updated weights for policy 0, policy_version 1641456 (0.0010) [2023-12-27 03:21:07,773][105620] Updated weights for policy 1, policy_version 1644808 (0.0006) [2023-12-27 03:21:07,827][105620] Updated weights for policy 1, policy_version 1644818 (0.0005) [2023-12-27 03:21:07,880][105620] Updated weights for policy 1, policy_version 1644828 (0.0005) [2023-12-27 03:21:07,926][105692] Updated weights for policy 0, policy_version 1641466 (0.0010) [2023-12-27 03:21:07,984][105692] Updated weights for policy 0, policy_version 1641476 (0.0010) [2023-12-27 03:21:08,043][105692] Updated weights for policy 0, policy_version 1641486 (0.0010) [2023-12-27 03:21:08,472][105620] Updated weights for policy 1, policy_version 1644838 (0.0005) [2023-12-27 03:21:08,533][105620] Updated weights for policy 1, policy_version 1644848 (0.0007) [2023-12-27 03:21:08,592][105620] Updated weights for policy 1, policy_version 1644858 (0.0008) [2023-12-27 03:21:08,779][105692] Updated weights for policy 0, policy_version 1641496 (0.0011) [2023-12-27 03:21:08,830][105692] Updated weights for policy 0, policy_version 1641506 (0.0011) [2023-12-27 03:21:08,892][105692] Updated weights for policy 0, policy_version 1641516 (0.0010) [2023-12-27 03:21:09,347][105620] Updated weights for policy 1, policy_version 1644868 (0.0009) [2023-12-27 03:21:09,415][105620] Updated weights for policy 1, policy_version 1644878 (0.0011) [2023-12-27 03:21:09,475][105620] Updated weights for policy 1, policy_version 1644888 (0.0011) [2023-12-27 03:21:09,679][105692] Updated weights for policy 0, policy_version 1641526 (0.0009) [2023-12-27 03:21:09,732][105692] Updated weights for policy 0, policy_version 1641536 (0.0007) [2023-12-27 03:21:09,784][105692] Updated weights for policy 0, policy_version 1641546 (0.0010) [2023-12-27 03:21:10,266][105620] Updated weights for policy 1, policy_version 1644898 (0.0013) [2023-12-27 03:21:10,325][105620] Updated weights for policy 1, policy_version 1644908 (0.0008) [2023-12-27 03:21:10,386][105620] Updated weights for policy 1, policy_version 1644918 (0.0010) [2023-12-27 03:21:10,448][105620] Updated weights for policy 1, policy_version 1644928 (0.0011) [2023-12-27 03:21:10,551][105692] Updated weights for policy 0, policy_version 1641556 (0.0011) [2023-12-27 03:21:10,608][105692] Updated weights for policy 0, policy_version 1641566 (0.0011) [2023-12-27 03:21:10,657][105692] Updated weights for policy 0, policy_version 1641576 (0.0010) [2023-12-27 03:21:11,043][105620] Updated weights for policy 1, policy_version 1644938 (0.0007) [2023-12-27 03:21:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 841465856. Throughput: 0: 10009.6, 1: 9931.0. Samples: 841477904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:21:11,062][104569] Avg episode reward: [(0, '8895.007'), (1, '9263.869')] [2023-12-27 03:21:11,106][105620] Updated weights for policy 1, policy_version 1644948 (0.0007) [2023-12-27 03:21:11,171][105620] Updated weights for policy 1, policy_version 1644958 (0.0009) [2023-12-27 03:21:11,417][105692] Updated weights for policy 0, policy_version 1641586 (0.0010) [2023-12-27 03:21:11,474][105692] Updated weights for policy 0, policy_version 1641596 (0.0009) [2023-12-27 03:21:11,526][105692] Updated weights for policy 0, policy_version 1641606 (0.0009) [2023-12-27 03:21:11,585][105692] Updated weights for policy 0, policy_version 1641616 (0.0008) [2023-12-27 03:21:11,870][105620] Updated weights for policy 1, policy_version 1644968 (0.0007) [2023-12-27 03:21:11,933][105620] Updated weights for policy 1, policy_version 1644978 (0.0005) [2023-12-27 03:21:11,995][105620] Updated weights for policy 1, policy_version 1644988 (0.0008) [2023-12-27 03:21:12,376][105692] Updated weights for policy 0, policy_version 1641626 (0.0010) [2023-12-27 03:21:12,439][105692] Updated weights for policy 0, policy_version 1641636 (0.0010) [2023-12-27 03:21:12,501][105692] Updated weights for policy 0, policy_version 1641646 (0.0010) [2023-12-27 03:21:12,694][105620] Updated weights for policy 1, policy_version 1644998 (0.0008) [2023-12-27 03:21:12,752][105620] Updated weights for policy 1, policy_version 1645008 (0.0008) [2023-12-27 03:21:12,801][105620] Updated weights for policy 1, policy_version 1645018 (0.0006) [2023-12-27 03:21:13,184][105692] Updated weights for policy 0, policy_version 1641656 (0.0009) [2023-12-27 03:21:13,228][105692] Updated weights for policy 0, policy_version 1641666 (0.0005) [2023-12-27 03:21:13,272][105692] Updated weights for policy 0, policy_version 1641676 (0.0005) [2023-12-27 03:21:13,419][105620] Updated weights for policy 1, policy_version 1645028 (0.0008) [2023-12-27 03:21:13,474][105620] Updated weights for policy 1, policy_version 1645038 (0.0008) [2023-12-27 03:21:13,539][105620] Updated weights for policy 1, policy_version 1645048 (0.0010) [2023-12-27 03:21:13,947][105692] Updated weights for policy 0, policy_version 1641686 (0.0005) [2023-12-27 03:21:14,000][105692] Updated weights for policy 0, policy_version 1641696 (0.0006) [2023-12-27 03:21:14,054][105692] Updated weights for policy 0, policy_version 1641706 (0.0006) [2023-12-27 03:21:14,309][105620] Updated weights for policy 1, policy_version 1645058 (0.0008) [2023-12-27 03:21:14,368][105620] Updated weights for policy 1, policy_version 1645068 (0.0008) [2023-12-27 03:21:14,426][105620] Updated weights for policy 1, policy_version 1645078 (0.0008) [2023-12-27 03:21:14,487][105620] Updated weights for policy 1, policy_version 1645088 (0.0008) [2023-12-27 03:21:14,731][105692] Updated weights for policy 0, policy_version 1641716 (0.0007) [2023-12-27 03:21:14,790][105692] Updated weights for policy 0, policy_version 1641726 (0.0007) [2023-12-27 03:21:14,854][105692] Updated weights for policy 0, policy_version 1641736 (0.0006) [2023-12-27 03:21:15,260][105620] Updated weights for policy 1, policy_version 1645098 (0.0009) [2023-12-27 03:21:15,318][105620] Updated weights for policy 1, policy_version 1645108 (0.0009) [2023-12-27 03:21:15,377][105620] Updated weights for policy 1, policy_version 1645118 (0.0009) [2023-12-27 03:21:15,560][105692] Updated weights for policy 0, policy_version 1641746 (0.0006) [2023-12-27 03:21:15,606][105692] Updated weights for policy 0, policy_version 1641756 (0.0008) [2023-12-27 03:21:15,628][105585] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 03:21:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 841564160. Throughput: 0: 9910.1, 1: 9892.0. Samples: 841536292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:21:16,062][104569] Avg episode reward: [(0, '8898.019'), (1, '9171.115')] [2023-12-27 03:21:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001641760_420356096.pth... [2023-12-27 03:21:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001645120_421208064.pth... [2023-12-27 03:21:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001640624_420061184.pth [2023-12-27 03:21:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001643968_420913152.pth [2023-12-27 03:21:16,138][105620] Updated weights for policy 1, policy_version 1645128 (0.0009) [2023-12-27 03:21:16,197][105620] Updated weights for policy 1, policy_version 1645138 (0.0009) [2023-12-27 03:21:16,247][105620] Updated weights for policy 1, policy_version 1645148 (0.0009) [2023-12-27 03:21:16,425][105692] Updated weights for policy 0, policy_version 1641766 (0.0009) [2023-12-27 03:21:16,487][105692] Updated weights for policy 0, policy_version 1641776 (0.0005) [2023-12-27 03:21:16,551][105692] Updated weights for policy 0, policy_version 1641786 (0.0005) [2023-12-27 03:21:17,037][105620] Updated weights for policy 1, policy_version 1645158 (0.0009) [2023-12-27 03:21:17,083][105692] Updated weights for policy 0, policy_version 1641796 (0.0005) [2023-12-27 03:21:17,089][105620] Updated weights for policy 1, policy_version 1645168 (0.0008) [2023-12-27 03:21:17,143][105692] Updated weights for policy 0, policy_version 1641806 (0.0007) [2023-12-27 03:21:17,149][105620] Updated weights for policy 1, policy_version 1645178 (0.0007) [2023-12-27 03:21:17,202][105692] Updated weights for policy 0, policy_version 1641816 (0.0008) [2023-12-27 03:21:17,731][105620] Updated weights for policy 1, policy_version 1645188 (0.0007) [2023-12-27 03:21:17,785][105620] Updated weights for policy 1, policy_version 1645199 (0.0010) [2023-12-27 03:21:17,836][105620] Updated weights for policy 1, policy_version 1645210 (0.0010) [2023-12-27 03:21:17,974][105692] Updated weights for policy 0, policy_version 1641826 (0.0009) [2023-12-27 03:21:18,028][105692] Updated weights for policy 0, policy_version 1641836 (0.0005) [2023-12-27 03:21:18,085][105692] Updated weights for policy 0, policy_version 1641846 (0.0008) [2023-12-27 03:21:18,138][105692] Updated weights for policy 0, policy_version 1641856 (0.0007) [2023-12-27 03:21:18,591][105620] Updated weights for policy 1, policy_version 1645220 (0.0010) [2023-12-27 03:21:18,639][105620] Updated weights for policy 1, policy_version 1645230 (0.0009) [2023-12-27 03:21:18,696][105620] Updated weights for policy 1, policy_version 1645240 (0.0007) [2023-12-27 03:21:18,714][105692] Updated weights for policy 0, policy_version 1641866 (0.0007) [2023-12-27 03:21:18,771][105692] Updated weights for policy 0, policy_version 1641876 (0.0007) [2023-12-27 03:21:18,833][105692] Updated weights for policy 0, policy_version 1641886 (0.0007) [2023-12-27 03:21:19,492][105620] Updated weights for policy 1, policy_version 1645250 (0.0007) [2023-12-27 03:21:19,528][105692] Updated weights for policy 0, policy_version 1641896 (0.0006) [2023-12-27 03:21:19,556][105620] Updated weights for policy 1, policy_version 1645260 (0.0008) [2023-12-27 03:21:19,586][105692] Updated weights for policy 0, policy_version 1641906 (0.0008) [2023-12-27 03:21:19,620][105620] Updated weights for policy 1, policy_version 1645270 (0.0006) [2023-12-27 03:21:19,648][105692] Updated weights for policy 0, policy_version 1641916 (0.0009) [2023-12-27 03:21:19,680][105620] Updated weights for policy 1, policy_version 1645280 (0.0005) [2023-12-27 03:21:20,372][105620] Updated weights for policy 1, policy_version 1645290 (0.0006) [2023-12-27 03:21:20,412][105692] Updated weights for policy 0, policy_version 1641926 (0.0007) [2023-12-27 03:21:20,436][105620] Updated weights for policy 1, policy_version 1645300 (0.0008) [2023-12-27 03:21:20,480][105692] Updated weights for policy 0, policy_version 1641936 (0.0007) [2023-12-27 03:21:20,497][105620] Updated weights for policy 1, policy_version 1645310 (0.0007) [2023-12-27 03:21:20,548][105692] Updated weights for policy 0, policy_version 1641946 (0.0009) [2023-12-27 03:21:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 841662464. Throughput: 0: 9914.5, 1: 9865.8. Samples: 841655744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:21:21,062][104569] Avg episode reward: [(0, '8443.917'), (1, '8808.144')] [2023-12-27 03:21:21,165][105620] Updated weights for policy 1, policy_version 1645320 (0.0008) [2023-12-27 03:21:21,227][105620] Updated weights for policy 1, policy_version 1645330 (0.0008) [2023-12-27 03:21:21,261][105692] Updated weights for policy 0, policy_version 1641956 (0.0008) [2023-12-27 03:21:21,289][105620] Updated weights for policy 1, policy_version 1645340 (0.0008) [2023-12-27 03:21:21,320][105692] Updated weights for policy 0, policy_version 1641966 (0.0009) [2023-12-27 03:21:21,387][105692] Updated weights for policy 0, policy_version 1641976 (0.0010) [2023-12-27 03:21:22,081][105620] Updated weights for policy 1, policy_version 1645350 (0.0008) [2023-12-27 03:21:22,146][105620] Updated weights for policy 1, policy_version 1645360 (0.0009) [2023-12-27 03:21:22,156][105692] Updated weights for policy 0, policy_version 1641986 (0.0007) [2023-12-27 03:21:22,211][105692] Updated weights for policy 0, policy_version 1641996 (0.0006) [2023-12-27 03:21:22,213][105620] Updated weights for policy 1, policy_version 1645370 (0.0009) [2023-12-27 03:21:22,268][105692] Updated weights for policy 0, policy_version 1642006 (0.0008) [2023-12-27 03:21:22,327][105692] Updated weights for policy 0, policy_version 1642016 (0.0009) [2023-12-27 03:21:22,907][105620] Updated weights for policy 1, policy_version 1645380 (0.0009) [2023-12-27 03:21:22,963][105620] Updated weights for policy 1, policy_version 1645390 (0.0008) [2023-12-27 03:21:23,012][105620] Updated weights for policy 1, policy_version 1645400 (0.0008) [2023-12-27 03:21:23,121][105692] Updated weights for policy 0, policy_version 1642026 (0.0009) [2023-12-27 03:21:23,184][105692] Updated weights for policy 0, policy_version 1642036 (0.0009) [2023-12-27 03:21:23,246][105692] Updated weights for policy 0, policy_version 1642046 (0.0009) [2023-12-27 03:21:23,772][105620] Updated weights for policy 1, policy_version 1645410 (0.0009) [2023-12-27 03:21:23,822][105620] Updated weights for policy 1, policy_version 1645420 (0.0010) [2023-12-27 03:21:23,871][105620] Updated weights for policy 1, policy_version 1645430 (0.0005) [2023-12-27 03:21:23,914][105620] Updated weights for policy 1, policy_version 1645440 (0.0005) [2023-12-27 03:21:24,002][105692] Updated weights for policy 0, policy_version 1642056 (0.0010) [2023-12-27 03:21:24,060][105692] Updated weights for policy 0, policy_version 1642067 (0.0009) [2023-12-27 03:21:24,121][105692] Updated weights for policy 0, policy_version 1642077 (0.0009) [2023-12-27 03:21:24,528][105620] Updated weights for policy 1, policy_version 1645450 (0.0010) [2023-12-27 03:21:24,596][105620] Updated weights for policy 1, policy_version 1645460 (0.0010) [2023-12-27 03:21:24,654][105620] Updated weights for policy 1, policy_version 1645470 (0.0010) [2023-12-27 03:21:24,937][105692] Updated weights for policy 0, policy_version 1642087 (0.0009) [2023-12-27 03:21:24,990][105692] Updated weights for policy 0, policy_version 1642097 (0.0010) [2023-12-27 03:21:25,048][105692] Updated weights for policy 0, policy_version 1642107 (0.0010) [2023-12-27 03:21:25,243][105620] Updated weights for policy 1, policy_version 1645480 (0.0006) [2023-12-27 03:21:25,306][105620] Updated weights for policy 1, policy_version 1645490 (0.0007) [2023-12-27 03:21:25,362][105620] Updated weights for policy 1, policy_version 1645500 (0.0005) [2023-12-27 03:21:25,772][105692] Updated weights for policy 0, policy_version 1642117 (0.0008) [2023-12-27 03:21:25,835][105692] Updated weights for policy 0, policy_version 1642127 (0.0006) [2023-12-27 03:21:25,905][105692] Updated weights for policy 0, policy_version 1642137 (0.0006) [2023-12-27 03:21:25,972][105620] Updated weights for policy 1, policy_version 1645510 (0.0005) [2023-12-27 03:21:26,023][105620] Updated weights for policy 1, policy_version 1645520 (0.0006) [2023-12-27 03:21:26,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 841760768. Throughput: 0: 9741.9, 1: 9954.2. Samples: 841770336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:21:26,063][104569] Avg episode reward: [(0, '8442.838'), (1, '8624.464')] [2023-12-27 03:21:26,075][105620] Updated weights for policy 1, policy_version 1645530 (0.0010) [2023-12-27 03:21:26,473][105692] Updated weights for policy 0, policy_version 1642147 (0.0006) [2023-12-27 03:21:26,537][105692] Updated weights for policy 0, policy_version 1642157 (0.0009) [2023-12-27 03:21:26,599][105692] Updated weights for policy 0, policy_version 1642167 (0.0008) [2023-12-27 03:21:26,710][105620] Updated weights for policy 1, policy_version 1645540 (0.0009) [2023-12-27 03:21:26,761][105620] Updated weights for policy 1, policy_version 1645550 (0.0005) [2023-12-27 03:21:26,821][105620] Updated weights for policy 1, policy_version 1645560 (0.0009) [2023-12-27 03:21:27,111][105692] Updated weights for policy 0, policy_version 1642177 (0.0006) [2023-12-27 03:21:27,159][105692] Updated weights for policy 0, policy_version 1642187 (0.0010) [2023-12-27 03:21:27,203][105692] Updated weights for policy 0, policy_version 1642197 (0.0010) [2023-12-27 03:21:27,246][105692] Updated weights for policy 0, policy_version 1642207 (0.0010) [2023-12-27 03:21:27,566][105620] Updated weights for policy 1, policy_version 1645570 (0.0009) [2023-12-27 03:21:27,612][105620] Updated weights for policy 1, policy_version 1645580 (0.0008) [2023-12-27 03:21:27,658][105620] Updated weights for policy 1, policy_version 1645590 (0.0005) [2023-12-27 03:21:27,703][105620] Updated weights for policy 1, policy_version 1645600 (0.0005) [2023-12-27 03:21:27,897][105692] Updated weights for policy 0, policy_version 1642217 (0.0006) [2023-12-27 03:21:27,950][105692] Updated weights for policy 0, policy_version 1642227 (0.0010) [2023-12-27 03:21:28,003][105692] Updated weights for policy 0, policy_version 1642237 (0.0010) [2023-12-27 03:21:28,310][105620] Updated weights for policy 1, policy_version 1645610 (0.0007) [2023-12-27 03:21:28,371][105620] Updated weights for policy 1, policy_version 1645620 (0.0009) [2023-12-27 03:21:28,437][105620] Updated weights for policy 1, policy_version 1645630 (0.0009) [2023-12-27 03:21:28,781][105692] Updated weights for policy 0, policy_version 1642247 (0.0007) [2023-12-27 03:21:28,828][105692] Updated weights for policy 0, policy_version 1642257 (0.0008) [2023-12-27 03:21:28,875][105692] Updated weights for policy 0, policy_version 1642267 (0.0009) [2023-12-27 03:21:29,112][105620] Updated weights for policy 1, policy_version 1645640 (0.0007) [2023-12-27 03:21:29,162][105620] Updated weights for policy 1, policy_version 1645650 (0.0008) [2023-12-27 03:21:29,217][105620] Updated weights for policy 1, policy_version 1645660 (0.0008) [2023-12-27 03:21:29,642][105692] Updated weights for policy 0, policy_version 1642277 (0.0007) [2023-12-27 03:21:29,700][105692] Updated weights for policy 0, policy_version 1642287 (0.0007) [2023-12-27 03:21:29,760][105692] Updated weights for policy 0, policy_version 1642297 (0.0007) [2023-12-27 03:21:30,005][105620] Updated weights for policy 1, policy_version 1645670 (0.0009) [2023-12-27 03:21:30,070][105620] Updated weights for policy 1, policy_version 1645680 (0.0008) [2023-12-27 03:21:30,137][105620] Updated weights for policy 1, policy_version 1645690 (0.0006) [2023-12-27 03:21:30,559][105692] Updated weights for policy 0, policy_version 1642307 (0.0007) [2023-12-27 03:21:30,615][105692] Updated weights for policy 0, policy_version 1642318 (0.0009) [2023-12-27 03:21:30,659][105692] Updated weights for policy 0, policy_version 1642328 (0.0008) [2023-12-27 03:21:30,723][105620] Updated weights for policy 1, policy_version 1645700 (0.0007) [2023-12-27 03:21:30,773][105620] Updated weights for policy 1, policy_version 1645710 (0.0010) [2023-12-27 03:21:30,816][105620] Updated weights for policy 1, policy_version 1645720 (0.0008) [2023-12-27 03:21:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 841867264. Throughput: 0: 9826.2, 1: 10009.0. Samples: 841835588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:21:31,062][104569] Avg episode reward: [(0, '8626.105'), (1, '9079.893')] [2023-12-27 03:21:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001642336_420503552.pth... [2023-12-27 03:21:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001645728_421363712.pth... [2023-12-27 03:21:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001644544_421060608.pth [2023-12-27 03:21:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001641232_420216832.pth [2023-12-27 03:21:31,398][105692] Updated weights for policy 0, policy_version 1642338 (0.0008) [2023-12-27 03:21:31,461][105692] Updated weights for policy 0, policy_version 1642348 (0.0008) [2023-12-27 03:21:31,523][105692] Updated weights for policy 0, policy_version 1642358 (0.0008) [2023-12-27 03:21:31,525][105620] Updated weights for policy 1, policy_version 1645730 (0.0008) [2023-12-27 03:21:31,579][105692] Updated weights for policy 0, policy_version 1642368 (0.0007) [2023-12-27 03:21:31,588][105620] Updated weights for policy 1, policy_version 1645740 (0.0010) [2023-12-27 03:21:31,658][105620] Updated weights for policy 1, policy_version 1645750 (0.0011) [2023-12-27 03:21:31,729][105620] Updated weights for policy 1, policy_version 1645760 (0.0009) [2023-12-27 03:21:32,274][105692] Updated weights for policy 0, policy_version 1642378 (0.0011) [2023-12-27 03:21:32,335][105692] Updated weights for policy 0, policy_version 1642388 (0.0010) [2023-12-27 03:21:32,396][105692] Updated weights for policy 0, policy_version 1642398 (0.0008) [2023-12-27 03:21:32,479][105620] Updated weights for policy 1, policy_version 1645770 (0.0010) [2023-12-27 03:21:32,540][105620] Updated weights for policy 1, policy_version 1645780 (0.0010) [2023-12-27 03:21:32,605][105620] Updated weights for policy 1, policy_version 1645790 (0.0010) [2023-12-27 03:21:33,006][105692] Updated weights for policy 0, policy_version 1642408 (0.0005) [2023-12-27 03:21:33,056][105692] Updated weights for policy 0, policy_version 1642418 (0.0005) [2023-12-27 03:21:33,112][105692] Updated weights for policy 0, policy_version 1642428 (0.0007) [2023-12-27 03:21:33,327][105620] Updated weights for policy 1, policy_version 1645800 (0.0010) [2023-12-27 03:21:33,381][105620] Updated weights for policy 1, policy_version 1645810 (0.0010) [2023-12-27 03:21:33,443][105620] Updated weights for policy 1, policy_version 1645820 (0.0010) [2023-12-27 03:21:33,733][105692] Updated weights for policy 0, policy_version 1642438 (0.0007) [2023-12-27 03:21:33,800][105692] Updated weights for policy 0, policy_version 1642448 (0.0009) [2023-12-27 03:21:33,866][105692] Updated weights for policy 0, policy_version 1642458 (0.0008) [2023-12-27 03:21:34,189][105620] Updated weights for policy 1, policy_version 1645830 (0.0010) [2023-12-27 03:21:34,251][105620] Updated weights for policy 1, policy_version 1645840 (0.0008) [2023-12-27 03:21:34,319][105620] Updated weights for policy 1, policy_version 1645850 (0.0009) [2023-12-27 03:21:34,610][105692] Updated weights for policy 0, policy_version 1642468 (0.0008) [2023-12-27 03:21:34,674][105692] Updated weights for policy 0, policy_version 1642478 (0.0008) [2023-12-27 03:21:34,726][105692] Updated weights for policy 0, policy_version 1642488 (0.0008) [2023-12-27 03:21:35,058][105620] Updated weights for policy 1, policy_version 1645860 (0.0010) [2023-12-27 03:21:35,120][105620] Updated weights for policy 1, policy_version 1645870 (0.0006) [2023-12-27 03:21:35,186][105620] Updated weights for policy 1, policy_version 1645880 (0.0009) [2023-12-27 03:21:35,531][105692] Updated weights for policy 0, policy_version 1642498 (0.0008) [2023-12-27 03:21:35,590][105692] Updated weights for policy 0, policy_version 1642508 (0.0008) [2023-12-27 03:21:35,643][105692] Updated weights for policy 0, policy_version 1642518 (0.0008) [2023-12-27 03:21:35,699][105692] Updated weights for policy 0, policy_version 1642528 (0.0008) [2023-12-27 03:21:35,901][105620] Updated weights for policy 1, policy_version 1645890 (0.0009) [2023-12-27 03:21:35,958][105620] Updated weights for policy 1, policy_version 1645900 (0.0005) [2023-12-27 03:21:36,020][105620] Updated weights for policy 1, policy_version 1645910 (0.0005) [2023-12-27 03:21:36,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 841957376. Throughput: 0: 9775.6, 1: 9923.5. Samples: 841951872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:21:36,062][104569] Avg episode reward: [(0, '8716.214'), (1, '8894.752')] [2023-12-27 03:21:36,071][105620] Updated weights for policy 1, policy_version 1645920 (0.0009) [2023-12-27 03:21:36,510][105692] Updated weights for policy 0, policy_version 1642538 (0.0009) [2023-12-27 03:21:36,569][105692] Updated weights for policy 0, policy_version 1642548 (0.0009) [2023-12-27 03:21:36,628][105692] Updated weights for policy 0, policy_version 1642558 (0.0008) [2023-12-27 03:21:36,786][105620] Updated weights for policy 1, policy_version 1645930 (0.0009) [2023-12-27 03:21:36,852][105620] Updated weights for policy 1, policy_version 1645940 (0.0008) [2023-12-27 03:21:36,916][105620] Updated weights for policy 1, policy_version 1645950 (0.0008) [2023-12-27 03:21:37,402][105692] Updated weights for policy 0, policy_version 1642568 (0.0009) [2023-12-27 03:21:37,460][105692] Updated weights for policy 0, policy_version 1642578 (0.0009) [2023-12-27 03:21:37,507][105692] Updated weights for policy 0, policy_version 1642588 (0.0008) [2023-12-27 03:21:37,642][105620] Updated weights for policy 1, policy_version 1645960 (0.0006) [2023-12-27 03:21:37,707][105620] Updated weights for policy 1, policy_version 1645970 (0.0005) [2023-12-27 03:21:37,769][105620] Updated weights for policy 1, policy_version 1645980 (0.0005) [2023-12-27 03:21:38,350][105620] Updated weights for policy 1, policy_version 1645990 (0.0008) [2023-12-27 03:21:38,378][105692] Updated weights for policy 0, policy_version 1642598 (0.0008) [2023-12-27 03:21:38,409][105620] Updated weights for policy 1, policy_version 1646000 (0.0008) [2023-12-27 03:21:38,439][105692] Updated weights for policy 0, policy_version 1642608 (0.0008) [2023-12-27 03:21:38,466][105620] Updated weights for policy 1, policy_version 1646010 (0.0006) [2023-12-27 03:21:38,500][105692] Updated weights for policy 0, policy_version 1642618 (0.0009) [2023-12-27 03:21:39,177][105620] Updated weights for policy 1, policy_version 1646020 (0.0008) [2023-12-27 03:21:39,240][105620] Updated weights for policy 1, policy_version 1646030 (0.0009) [2023-12-27 03:21:39,272][105692] Updated weights for policy 0, policy_version 1642628 (0.0009) [2023-12-27 03:21:39,302][105620] Updated weights for policy 1, policy_version 1646040 (0.0008) [2023-12-27 03:21:39,329][105692] Updated weights for policy 0, policy_version 1642638 (0.0007) [2023-12-27 03:21:39,395][105692] Updated weights for policy 0, policy_version 1642648 (0.0009) [2023-12-27 03:21:40,030][105620] Updated weights for policy 1, policy_version 1646050 (0.0007) [2023-12-27 03:21:40,098][105620] Updated weights for policy 1, policy_version 1646060 (0.0008) [2023-12-27 03:21:40,154][105620] Updated weights for policy 1, policy_version 1646070 (0.0009) [2023-12-27 03:21:40,168][105692] Updated weights for policy 0, policy_version 1642658 (0.0010) [2023-12-27 03:21:40,218][105620] Updated weights for policy 1, policy_version 1646080 (0.0006) [2023-12-27 03:21:40,221][105692] Updated weights for policy 0, policy_version 1642668 (0.0011) [2023-12-27 03:21:40,277][105692] Updated weights for policy 0, policy_version 1642678 (0.0010) [2023-12-27 03:21:40,334][105692] Updated weights for policy 0, policy_version 1642688 (0.0011) [2023-12-27 03:21:40,999][105620] Updated weights for policy 1, policy_version 1646090 (0.0008) [2023-12-27 03:21:41,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 842047488. Throughput: 0: 9630.7, 1: 9953.7. Samples: 842062904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:21:41,063][104569] Avg episode reward: [(0, '8622.142'), (1, '8803.057')] [2023-12-27 03:21:41,063][105692] Updated weights for policy 0, policy_version 1642699 (0.0009) [2023-12-27 03:21:41,064][105620] Updated weights for policy 1, policy_version 1646100 (0.0007) [2023-12-27 03:21:41,120][105620] Updated weights for policy 1, policy_version 1646110 (0.0007) [2023-12-27 03:21:41,122][105692] Updated weights for policy 0, policy_version 1642709 (0.0008) [2023-12-27 03:21:41,195][105692] Updated weights for policy 0, policy_version 1642719 (0.0006) [2023-12-27 03:21:41,847][105620] Updated weights for policy 1, policy_version 1646120 (0.0008) [2023-12-27 03:21:41,902][105620] Updated weights for policy 1, policy_version 1646130 (0.0007) [2023-12-27 03:21:41,921][105692] Updated weights for policy 0, policy_version 1642729 (0.0008) [2023-12-27 03:21:41,953][105620] Updated weights for policy 1, policy_version 1646140 (0.0006) [2023-12-27 03:21:41,983][105692] Updated weights for policy 0, policy_version 1642739 (0.0009) [2023-12-27 03:21:42,046][105692] Updated weights for policy 0, policy_version 1642749 (0.0009) [2023-12-27 03:21:42,726][105620] Updated weights for policy 1, policy_version 1646150 (0.0006) [2023-12-27 03:21:42,790][105620] Updated weights for policy 1, policy_version 1646160 (0.0009) [2023-12-27 03:21:42,820][105692] Updated weights for policy 0, policy_version 1642759 (0.0008) [2023-12-27 03:21:42,850][105620] Updated weights for policy 1, policy_version 1646170 (0.0011) [2023-12-27 03:21:42,876][105692] Updated weights for policy 0, policy_version 1642769 (0.0006) [2023-12-27 03:21:42,940][105692] Updated weights for policy 0, policy_version 1642779 (0.0007) [2023-12-27 03:21:43,423][105620] Updated weights for policy 1, policy_version 1646180 (0.0009) [2023-12-27 03:21:43,475][105620] Updated weights for policy 1, policy_version 1646190 (0.0008) [2023-12-27 03:21:43,527][105620] Updated weights for policy 1, policy_version 1646200 (0.0010) [2023-12-27 03:21:43,663][105692] Updated weights for policy 0, policy_version 1642789 (0.0008) [2023-12-27 03:21:43,711][105692] Updated weights for policy 0, policy_version 1642799 (0.0008) [2023-12-27 03:21:43,762][105692] Updated weights for policy 0, policy_version 1642809 (0.0008) [2023-12-27 03:21:44,263][105620] Updated weights for policy 1, policy_version 1646210 (0.0011) [2023-12-27 03:21:44,319][105620] Updated weights for policy 1, policy_version 1646220 (0.0011) [2023-12-27 03:21:44,376][105620] Updated weights for policy 1, policy_version 1646230 (0.0010) [2023-12-27 03:21:44,435][105620] Updated weights for policy 1, policy_version 1646240 (0.0010) [2023-12-27 03:21:44,437][105692] Updated weights for policy 0, policy_version 1642819 (0.0007) [2023-12-27 03:21:44,482][105692] Updated weights for policy 0, policy_version 1642829 (0.0006) [2023-12-27 03:21:44,535][105692] Updated weights for policy 0, policy_version 1642839 (0.0005) [2023-12-27 03:21:45,249][105692] Updated weights for policy 0, policy_version 1642849 (0.0006) [2023-12-27 03:21:45,287][105620] Updated weights for policy 1, policy_version 1646250 (0.0008) [2023-12-27 03:21:45,319][105692] Updated weights for policy 0, policy_version 1642859 (0.0007) [2023-12-27 03:21:45,350][105620] Updated weights for policy 1, policy_version 1646260 (0.0007) [2023-12-27 03:21:45,382][105692] Updated weights for policy 0, policy_version 1642869 (0.0011) [2023-12-27 03:21:45,412][105620] Updated weights for policy 1, policy_version 1646270 (0.0007) [2023-12-27 03:21:45,445][105692] Updated weights for policy 0, policy_version 1642879 (0.0011) [2023-12-27 03:21:46,059][105620] Updated weights for policy 1, policy_version 1646280 (0.0006) [2023-12-27 03:21:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 842145792. Throughput: 0: 9617.2, 1: 9960.2. Samples: 842121124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:21:46,062][104569] Avg episode reward: [(0, '8350.930'), (1, '8987.073')] [2023-12-27 03:21:46,092][105692] Updated weights for policy 0, policy_version 1642889 (0.0009) [2023-12-27 03:21:46,118][105620] Updated weights for policy 1, policy_version 1646290 (0.0006) [2023-12-27 03:21:46,153][105692] Updated weights for policy 0, policy_version 1642899 (0.0008) [2023-12-27 03:21:46,170][105620] Updated weights for policy 1, policy_version 1646300 (0.0006) [2023-12-27 03:21:46,196][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001646304_421511168.pth... [2023-12-27 03:21:46,200][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001645120_421208064.pth [2023-12-27 03:21:46,202][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_001646304_421511168.pth [2023-12-27 03:21:46,215][105692] Updated weights for policy 0, policy_version 1642909 (0.0011) [2023-12-27 03:21:46,236][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001642912_420651008.pth... [2023-12-27 03:21:46,239][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001641760_420356096.pth [2023-12-27 03:21:46,239][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_001642912_420651008.pth [2023-12-27 03:21:46,811][105620] Updated weights for policy 1, policy_version 1646310 (0.0005) [2023-12-27 03:21:46,879][105620] Updated weights for policy 1, policy_version 1646320 (0.0005) [2023-12-27 03:21:46,930][105692] Updated weights for policy 0, policy_version 1642919 (0.0009) [2023-12-27 03:21:46,941][105620] Updated weights for policy 1, policy_version 1646330 (0.0005) [2023-12-27 03:21:46,987][105692] Updated weights for policy 0, policy_version 1642929 (0.0009) [2023-12-27 03:21:47,051][105692] Updated weights for policy 0, policy_version 1642939 (0.0009) [2023-12-27 03:21:47,561][105620] Updated weights for policy 1, policy_version 1646340 (0.0007) [2023-12-27 03:21:47,625][105620] Updated weights for policy 1, policy_version 1646350 (0.0010) [2023-12-27 03:21:47,683][105620] Updated weights for policy 1, policy_version 1646360 (0.0008) [2023-12-27 03:21:47,744][105692] Updated weights for policy 0, policy_version 1642949 (0.0010) [2023-12-27 03:21:47,797][105692] Updated weights for policy 0, policy_version 1642959 (0.0006) [2023-12-27 03:21:47,846][105692] Updated weights for policy 0, policy_version 1642969 (0.0008) [2023-12-27 03:21:48,482][105692] Updated weights for policy 0, policy_version 1642979 (0.0007) [2023-12-27 03:21:48,516][105620] Updated weights for policy 1, policy_version 1646370 (0.0009) [2023-12-27 03:21:48,542][105692] Updated weights for policy 0, policy_version 1642989 (0.0009) [2023-12-27 03:21:48,566][105620] Updated weights for policy 1, policy_version 1646380 (0.0006) [2023-12-27 03:21:48,599][105692] Updated weights for policy 0, policy_version 1642999 (0.0008) [2023-12-27 03:21:48,620][105620] Updated weights for policy 1, policy_version 1646390 (0.0008) [2023-12-27 03:21:48,675][105620] Updated weights for policy 1, policy_version 1646400 (0.0009) [2023-12-27 03:21:49,164][105692] Updated weights for policy 0, policy_version 1643009 (0.0006) [2023-12-27 03:21:49,217][105692] Updated weights for policy 0, policy_version 1643019 (0.0006) [2023-12-27 03:21:49,280][105692] Updated weights for policy 0, policy_version 1643029 (0.0008) [2023-12-27 03:21:49,340][105692] Updated weights for policy 0, policy_version 1643039 (0.0008) [2023-12-27 03:21:49,498][105620] Updated weights for policy 1, policy_version 1646410 (0.0005) [2023-12-27 03:21:49,553][105620] Updated weights for policy 1, policy_version 1646420 (0.0005) [2023-12-27 03:21:49,603][105620] Updated weights for policy 1, policy_version 1646430 (0.0008) [2023-12-27 03:21:49,991][105692] Updated weights for policy 0, policy_version 1643049 (0.0006) [2023-12-27 03:21:50,048][105692] Updated weights for policy 0, policy_version 1643059 (0.0006) [2023-12-27 03:21:50,109][105692] Updated weights for policy 0, policy_version 1643069 (0.0007) [2023-12-27 03:21:50,390][105620] Updated weights for policy 1, policy_version 1646440 (0.0009) [2023-12-27 03:21:50,445][105620] Updated weights for policy 1, policy_version 1646450 (0.0009) [2023-12-27 03:21:50,492][105620] Updated weights for policy 1, policy_version 1646460 (0.0008) [2023-12-27 03:21:50,802][105692] Updated weights for policy 0, policy_version 1643079 (0.0009) [2023-12-27 03:21:50,857][105692] Updated weights for policy 0, policy_version 1643089 (0.0009) [2023-12-27 03:21:50,912][105692] Updated weights for policy 0, policy_version 1643099 (0.0009) [2023-12-27 03:21:51,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 842252288. Throughput: 0: 9752.6, 1: 9831.3. Samples: 842240156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:21:51,062][104569] Avg episode reward: [(0, '8717.642'), (1, '9263.185')] [2023-12-27 03:21:51,270][105620] Updated weights for policy 1, policy_version 1646470 (0.0009) [2023-12-27 03:21:51,333][105620] Updated weights for policy 1, policy_version 1646480 (0.0010) [2023-12-27 03:21:51,405][105620] Updated weights for policy 1, policy_version 1646490 (0.0008) [2023-12-27 03:21:51,709][105692] Updated weights for policy 0, policy_version 1643109 (0.0009) [2023-12-27 03:21:51,777][105692] Updated weights for policy 0, policy_version 1643119 (0.0009) [2023-12-27 03:21:51,828][105692] Updated weights for policy 0, policy_version 1643129 (0.0009) [2023-12-27 03:21:52,164][105620] Updated weights for policy 1, policy_version 1646500 (0.0009) [2023-12-27 03:21:52,229][105620] Updated weights for policy 1, policy_version 1646510 (0.0008) [2023-12-27 03:21:52,292][105620] Updated weights for policy 1, policy_version 1646520 (0.0009) [2023-12-27 03:21:52,626][105692] Updated weights for policy 0, policy_version 1643139 (0.0009) [2023-12-27 03:21:52,688][105692] Updated weights for policy 0, policy_version 1643149 (0.0009) [2023-12-27 03:21:52,755][105692] Updated weights for policy 0, policy_version 1643159 (0.0009) [2023-12-27 03:21:52,999][105620] Updated weights for policy 1, policy_version 1646530 (0.0009) [2023-12-27 03:21:53,049][105620] Updated weights for policy 1, policy_version 1646540 (0.0008) [2023-12-27 03:21:53,095][105620] Updated weights for policy 1, policy_version 1646550 (0.0008) [2023-12-27 03:21:53,151][105620] Updated weights for policy 1, policy_version 1646560 (0.0009) [2023-12-27 03:21:53,455][105692] Updated weights for policy 0, policy_version 1643169 (0.0009) [2023-12-27 03:21:53,508][105692] Updated weights for policy 0, policy_version 1643179 (0.0005) [2023-12-27 03:21:53,564][105692] Updated weights for policy 0, policy_version 1643189 (0.0007) [2023-12-27 03:21:53,618][105692] Updated weights for policy 0, policy_version 1643199 (0.0010) [2023-12-27 03:21:53,983][105620] Updated weights for policy 1, policy_version 1646570 (0.0005) [2023-12-27 03:21:54,035][105620] Updated weights for policy 1, policy_version 1646580 (0.0005) [2023-12-27 03:21:54,089][105620] Updated weights for policy 1, policy_version 1646590 (0.0005) [2023-12-27 03:21:54,192][105692] Updated weights for policy 0, policy_version 1643210 (0.0009) [2023-12-27 03:21:54,240][105692] Updated weights for policy 0, policy_version 1643220 (0.0008) [2023-12-27 03:21:54,295][105692] Updated weights for policy 0, policy_version 1643230 (0.0008) [2023-12-27 03:21:54,729][105620] Updated weights for policy 1, policy_version 1646600 (0.0009) [2023-12-27 03:21:54,795][105620] Updated weights for policy 1, policy_version 1646610 (0.0010) [2023-12-27 03:21:54,851][105586] KL-divergence is very high: 146.9160 [2023-12-27 03:21:54,853][105620] Updated weights for policy 1, policy_version 1646620 (0.0009) [2023-12-27 03:21:54,992][105692] Updated weights for policy 0, policy_version 1643240 (0.0008) [2023-12-27 03:21:55,066][105692] Updated weights for policy 0, policy_version 1643250 (0.0009) [2023-12-27 03:21:55,132][105692] Updated weights for policy 0, policy_version 1643260 (0.0008) [2023-12-27 03:21:55,669][105620] Updated weights for policy 1, policy_version 1646630 (0.0010) [2023-12-27 03:21:55,700][105692] Updated weights for policy 0, policy_version 1643270 (0.0006) [2023-12-27 03:21:55,722][105620] Updated weights for policy 1, policy_version 1646640 (0.0007) [2023-12-27 03:21:55,758][105692] Updated weights for policy 0, policy_version 1643280 (0.0011) [2023-12-27 03:21:55,780][105620] Updated weights for policy 1, policy_version 1646650 (0.0005) [2023-12-27 03:21:55,807][105692] Updated weights for policy 0, policy_version 1643290 (0.0010) [2023-12-27 03:21:56,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 842350592. Throughput: 0: 9780.9, 1: 9717.3. Samples: 842355328. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:21:56,063][104569] Avg episode reward: [(0, '8900.046'), (1, '8898.271')] [2023-12-27 03:21:56,519][105620] Updated weights for policy 1, policy_version 1646660 (0.0008) [2023-12-27 03:21:56,534][105692] Updated weights for policy 0, policy_version 1643300 (0.0009) [2023-12-27 03:21:56,569][105620] Updated weights for policy 1, policy_version 1646670 (0.0005) [2023-12-27 03:21:56,586][105692] Updated weights for policy 0, policy_version 1643310 (0.0011) [2023-12-27 03:21:56,628][105620] Updated weights for policy 1, policy_version 1646680 (0.0006) [2023-12-27 03:21:56,646][105692] Updated weights for policy 0, policy_version 1643320 (0.0011) [2023-12-27 03:21:57,331][105620] Updated weights for policy 1, policy_version 1646690 (0.0006) [2023-12-27 03:21:57,364][105692] Updated weights for policy 0, policy_version 1643330 (0.0010) [2023-12-27 03:21:57,389][105620] Updated weights for policy 1, policy_version 1646700 (0.0006) [2023-12-27 03:21:57,424][105692] Updated weights for policy 0, policy_version 1643340 (0.0005) [2023-12-27 03:21:57,441][105620] Updated weights for policy 1, policy_version 1646710 (0.0005) [2023-12-27 03:21:57,476][105692] Updated weights for policy 0, policy_version 1643350 (0.0010) [2023-12-27 03:21:57,497][105620] Updated weights for policy 1, policy_version 1646720 (0.0006) [2023-12-27 03:21:57,523][105692] Updated weights for policy 0, policy_version 1643360 (0.0010) [2023-12-27 03:21:58,200][105620] Updated weights for policy 1, policy_version 1646730 (0.0009) [2023-12-27 03:21:58,258][105692] Updated weights for policy 0, policy_version 1643370 (0.0011) [2023-12-27 03:21:58,265][105620] Updated weights for policy 1, policy_version 1646740 (0.0007) [2023-12-27 03:21:58,318][105692] Updated weights for policy 0, policy_version 1643380 (0.0011) [2023-12-27 03:21:58,321][105620] Updated weights for policy 1, policy_version 1646750 (0.0006) [2023-12-27 03:21:58,390][105692] Updated weights for policy 0, policy_version 1643390 (0.0010) [2023-12-27 03:21:59,071][105620] Updated weights for policy 1, policy_version 1646760 (0.0008) [2023-12-27 03:21:59,136][105620] Updated weights for policy 1, policy_version 1646770 (0.0007) [2023-12-27 03:21:59,205][105620] Updated weights for policy 1, policy_version 1646780 (0.0006) [2023-12-27 03:21:59,225][105692] Updated weights for policy 0, policy_version 1643400 (0.0009) [2023-12-27 03:21:59,297][105692] Updated weights for policy 0, policy_version 1643410 (0.0008) [2023-12-27 03:21:59,362][105692] Updated weights for policy 0, policy_version 1643420 (0.0008) [2023-12-27 03:21:59,911][105620] Updated weights for policy 1, policy_version 1646790 (0.0007) [2023-12-27 03:21:59,969][105692] Updated weights for policy 0, policy_version 1643430 (0.0007) [2023-12-27 03:21:59,975][105620] Updated weights for policy 1, policy_version 1646800 (0.0007) [2023-12-27 03:22:00,025][105692] Updated weights for policy 0, policy_version 1643440 (0.0007) [2023-12-27 03:22:00,035][105620] Updated weights for policy 1, policy_version 1646810 (0.0008) [2023-12-27 03:22:00,082][105692] Updated weights for policy 0, policy_version 1643450 (0.0006) [2023-12-27 03:22:00,729][105620] Updated weights for policy 1, policy_version 1646820 (0.0006) [2023-12-27 03:22:00,783][105692] Updated weights for policy 0, policy_version 1643460 (0.0007) [2023-12-27 03:22:00,789][105620] Updated weights for policy 1, policy_version 1646830 (0.0008) [2023-12-27 03:22:00,838][105692] Updated weights for policy 0, policy_version 1643470 (0.0009) [2023-12-27 03:22:00,844][105620] Updated weights for policy 1, policy_version 1646840 (0.0005) [2023-12-27 03:22:00,893][105692] Updated weights for policy 0, policy_version 1643480 (0.0010) [2023-12-27 03:22:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 842448896. Throughput: 0: 9793.3, 1: 9707.2. Samples: 842413816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:22:01,062][104569] Avg episode reward: [(0, '8632.059'), (1, '8714.924')] [2023-12-27 03:22:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001643488_420798464.pth... [2023-12-27 03:22:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001646848_421650432.pth... [2023-12-27 03:22:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001642336_420503552.pth [2023-12-27 03:22:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001645728_421363712.pth [2023-12-27 03:22:01,568][105692] Updated weights for policy 0, policy_version 1643490 (0.0009) [2023-12-27 03:22:01,595][105620] Updated weights for policy 1, policy_version 1646850 (0.0008) [2023-12-27 03:22:01,631][105692] Updated weights for policy 0, policy_version 1643500 (0.0007) [2023-12-27 03:22:01,672][105620] Updated weights for policy 1, policy_version 1646860 (0.0009) [2023-12-27 03:22:01,701][105692] Updated weights for policy 0, policy_version 1643510 (0.0010) [2023-12-27 03:22:01,735][105620] Updated weights for policy 1, policy_version 1646870 (0.0007) [2023-12-27 03:22:01,764][105692] Updated weights for policy 0, policy_version 1643520 (0.0010) [2023-12-27 03:22:01,790][105620] Updated weights for policy 1, policy_version 1646880 (0.0007) [2023-12-27 03:22:02,476][105692] Updated weights for policy 0, policy_version 1643530 (0.0009) [2023-12-27 03:22:02,522][105620] Updated weights for policy 1, policy_version 1646890 (0.0008) [2023-12-27 03:22:02,524][105692] Updated weights for policy 0, policy_version 1643540 (0.0007) [2023-12-27 03:22:02,581][105692] Updated weights for policy 0, policy_version 1643550 (0.0006) [2023-12-27 03:22:02,582][105620] Updated weights for policy 1, policy_version 1646900 (0.0007) [2023-12-27 03:22:02,638][105620] Updated weights for policy 1, policy_version 1646910 (0.0009) [2023-12-27 03:22:03,329][105620] Updated weights for policy 1, policy_version 1646920 (0.0009) [2023-12-27 03:22:03,334][105692] Updated weights for policy 0, policy_version 1643560 (0.0006) [2023-12-27 03:22:03,381][105620] Updated weights for policy 1, policy_version 1646930 (0.0009) [2023-12-27 03:22:03,387][105692] Updated weights for policy 0, policy_version 1643570 (0.0006) [2023-12-27 03:22:03,429][105620] Updated weights for policy 1, policy_version 1646940 (0.0008) [2023-12-27 03:22:03,445][105692] Updated weights for policy 0, policy_version 1643580 (0.0005) [2023-12-27 03:22:03,986][105692] Updated weights for policy 0, policy_version 1643590 (0.0006) [2023-12-27 03:22:04,038][105692] Updated weights for policy 0, policy_version 1643600 (0.0009) [2023-12-27 03:22:04,098][105692] Updated weights for policy 0, policy_version 1643610 (0.0010) [2023-12-27 03:22:04,287][105620] Updated weights for policy 1, policy_version 1646950 (0.0009) [2023-12-27 03:22:04,346][105620] Updated weights for policy 1, policy_version 1646960 (0.0008) [2023-12-27 03:22:04,416][105620] Updated weights for policy 1, policy_version 1646970 (0.0006) [2023-12-27 03:22:04,821][105692] Updated weights for policy 0, policy_version 1643620 (0.0009) [2023-12-27 03:22:04,887][105692] Updated weights for policy 0, policy_version 1643630 (0.0006) [2023-12-27 03:22:04,956][105692] Updated weights for policy 0, policy_version 1643640 (0.0006) [2023-12-27 03:22:05,085][105620] Updated weights for policy 1, policy_version 1646980 (0.0007) [2023-12-27 03:22:05,142][105620] Updated weights for policy 1, policy_version 1646990 (0.0009) [2023-12-27 03:22:05,206][105620] Updated weights for policy 1, policy_version 1647000 (0.0008) [2023-12-27 03:22:05,626][105692] Updated weights for policy 0, policy_version 1643650 (0.0009) [2023-12-27 03:22:05,692][105692] Updated weights for policy 0, policy_version 1643660 (0.0010) [2023-12-27 03:22:05,748][105692] Updated weights for policy 0, policy_version 1643670 (0.0010) [2023-12-27 03:22:05,808][105692] Updated weights for policy 0, policy_version 1643680 (0.0010) [2023-12-27 03:22:05,917][105620] Updated weights for policy 1, policy_version 1647010 (0.0009) [2023-12-27 03:22:05,973][105620] Updated weights for policy 1, policy_version 1647020 (0.0010) [2023-12-27 03:22:06,028][105620] Updated weights for policy 1, policy_version 1647030 (0.0010) [2023-12-27 03:22:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 842539008. Throughput: 0: 9767.3, 1: 9684.2. Samples: 842531060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:22:06,062][104569] Avg episode reward: [(0, '8540.620'), (1, '8806.730')] [2023-12-27 03:22:06,079][105620] Updated weights for policy 1, policy_version 1647040 (0.0009) [2023-12-27 03:22:06,480][105692] Updated weights for policy 0, policy_version 1643690 (0.0010) [2023-12-27 03:22:06,545][105692] Updated weights for policy 0, policy_version 1643700 (0.0007) [2023-12-27 03:22:06,606][105692] Updated weights for policy 0, policy_version 1643710 (0.0005) [2023-12-27 03:22:06,940][105620] Updated weights for policy 1, policy_version 1647050 (0.0007) [2023-12-27 03:22:07,010][105620] Updated weights for policy 1, policy_version 1647060 (0.0008) [2023-12-27 03:22:07,071][105620] Updated weights for policy 1, policy_version 1647070 (0.0008) [2023-12-27 03:22:07,290][105692] Updated weights for policy 0, policy_version 1643720 (0.0008) [2023-12-27 03:22:07,342][105692] Updated weights for policy 0, policy_version 1643730 (0.0009) [2023-12-27 03:22:07,400][105692] Updated weights for policy 0, policy_version 1643740 (0.0010) [2023-12-27 03:22:07,776][105620] Updated weights for policy 1, policy_version 1647080 (0.0009) [2023-12-27 03:22:07,824][105620] Updated weights for policy 1, policy_version 1647090 (0.0007) [2023-12-27 03:22:07,874][105620] Updated weights for policy 1, policy_version 1647100 (0.0007) [2023-12-27 03:22:08,103][105692] Updated weights for policy 0, policy_version 1643750 (0.0009) [2023-12-27 03:22:08,153][105692] Updated weights for policy 0, policy_version 1643760 (0.0008) [2023-12-27 03:22:08,208][105692] Updated weights for policy 0, policy_version 1643770 (0.0009) [2023-12-27 03:22:08,580][105620] Updated weights for policy 1, policy_version 1647110 (0.0008) [2023-12-27 03:22:08,639][105620] Updated weights for policy 1, policy_version 1647120 (0.0009) [2023-12-27 03:22:08,705][105620] Updated weights for policy 1, policy_version 1647130 (0.0009) [2023-12-27 03:22:08,984][105692] Updated weights for policy 0, policy_version 1643780 (0.0009) [2023-12-27 03:22:09,052][105692] Updated weights for policy 0, policy_version 1643790 (0.0009) [2023-12-27 03:22:09,111][105692] Updated weights for policy 0, policy_version 1643800 (0.0009) [2023-12-27 03:22:09,386][105620] Updated weights for policy 1, policy_version 1647140 (0.0009) [2023-12-27 03:22:09,452][105620] Updated weights for policy 1, policy_version 1647150 (0.0006) [2023-12-27 03:22:09,509][105620] Updated weights for policy 1, policy_version 1647160 (0.0005) [2023-12-27 03:22:09,956][105692] Updated weights for policy 0, policy_version 1643810 (0.0009) [2023-12-27 03:22:10,006][105692] Updated weights for policy 0, policy_version 1643820 (0.0008) [2023-12-27 03:22:10,054][105692] Updated weights for policy 0, policy_version 1643830 (0.0008) [2023-12-27 03:22:10,111][105692] Updated weights for policy 0, policy_version 1643840 (0.0007) [2023-12-27 03:22:10,222][105620] Updated weights for policy 1, policy_version 1647170 (0.0006) [2023-12-27 03:22:10,289][105620] Updated weights for policy 1, policy_version 1647180 (0.0009) [2023-12-27 03:22:10,314][105586] KL-divergence is very high: 219.7133 [2023-12-27 03:22:10,357][105620] Updated weights for policy 1, policy_version 1647190 (0.0009) [2023-12-27 03:22:10,370][105586] KL-divergence is very high: 361.8753 [2023-12-27 03:22:10,426][105620] Updated weights for policy 1, policy_version 1647200 (0.0010) [2023-12-27 03:22:10,733][105692] Updated weights for policy 0, policy_version 1643850 (0.0005) [2023-12-27 03:22:10,787][105692] Updated weights for policy 0, policy_version 1643860 (0.0005) [2023-12-27 03:22:10,836][105692] Updated weights for policy 0, policy_version 1643870 (0.0005) [2023-12-27 03:22:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 842637312. Throughput: 0: 9855.0, 1: 9590.5. Samples: 842645380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:22:11,062][104569] Avg episode reward: [(0, '8714.928'), (1, '9082.303')] [2023-12-27 03:22:11,276][105620] Updated weights for policy 1, policy_version 1647210 (0.0007) [2023-12-27 03:22:11,340][105620] Updated weights for policy 1, policy_version 1647220 (0.0007) [2023-12-27 03:22:11,409][105620] Updated weights for policy 1, policy_version 1647230 (0.0007) [2023-12-27 03:22:11,565][105692] Updated weights for policy 0, policy_version 1643880 (0.0005) [2023-12-27 03:22:11,628][105692] Updated weights for policy 0, policy_version 1643890 (0.0009) [2023-12-27 03:22:11,691][105692] Updated weights for policy 0, policy_version 1643900 (0.0009) [2023-12-27 03:22:12,134][105620] Updated weights for policy 1, policy_version 1647240 (0.0009) [2023-12-27 03:22:12,181][105620] Updated weights for policy 1, policy_version 1647250 (0.0009) [2023-12-27 03:22:12,231][105620] Updated weights for policy 1, policy_version 1647260 (0.0008) [2023-12-27 03:22:12,433][105692] Updated weights for policy 0, policy_version 1643910 (0.0010) [2023-12-27 03:22:12,495][105692] Updated weights for policy 0, policy_version 1643920 (0.0009) [2023-12-27 03:22:12,557][105692] Updated weights for policy 0, policy_version 1643930 (0.0009) [2023-12-27 03:22:13,019][105620] Updated weights for policy 1, policy_version 1647270 (0.0009) [2023-12-27 03:22:13,069][105620] Updated weights for policy 1, policy_version 1647280 (0.0008) [2023-12-27 03:22:13,119][105620] Updated weights for policy 1, policy_version 1647290 (0.0006) [2023-12-27 03:22:13,347][105692] Updated weights for policy 0, policy_version 1643940 (0.0008) [2023-12-27 03:22:13,408][105692] Updated weights for policy 0, policy_version 1643950 (0.0010) [2023-12-27 03:22:13,467][105692] Updated weights for policy 0, policy_version 1643960 (0.0010) [2023-12-27 03:22:13,743][105620] Updated weights for policy 1, policy_version 1647300 (0.0007) [2023-12-27 03:22:13,799][105620] Updated weights for policy 1, policy_version 1647310 (0.0009) [2023-12-27 03:22:13,857][105620] Updated weights for policy 1, policy_version 1647320 (0.0009) [2023-12-27 03:22:14,185][105692] Updated weights for policy 0, policy_version 1643970 (0.0009) [2023-12-27 03:22:14,251][105692] Updated weights for policy 0, policy_version 1643980 (0.0005) [2023-12-27 03:22:14,310][105692] Updated weights for policy 0, policy_version 1643990 (0.0009) [2023-12-27 03:22:14,371][105692] Updated weights for policy 0, policy_version 1644000 (0.0008) [2023-12-27 03:22:14,672][105620] Updated weights for policy 1, policy_version 1647330 (0.0009) [2023-12-27 03:22:14,729][105620] Updated weights for policy 1, policy_version 1647340 (0.0009) [2023-12-27 03:22:14,794][105620] Updated weights for policy 1, policy_version 1647350 (0.0008) [2023-12-27 03:22:14,849][105620] Updated weights for policy 1, policy_version 1647360 (0.0009) [2023-12-27 03:22:15,024][105692] Updated weights for policy 0, policy_version 1644010 (0.0008) [2023-12-27 03:22:15,072][105692] Updated weights for policy 0, policy_version 1644020 (0.0005) [2023-12-27 03:22:15,122][105692] Updated weights for policy 0, policy_version 1644030 (0.0006) [2023-12-27 03:22:15,644][105620] Updated weights for policy 1, policy_version 1647370 (0.0009) [2023-12-27 03:22:15,695][105620] Updated weights for policy 1, policy_version 1647380 (0.0008) [2023-12-27 03:22:15,747][105620] Updated weights for policy 1, policy_version 1647390 (0.0009) [2023-12-27 03:22:15,837][105692] Updated weights for policy 0, policy_version 1644040 (0.0005) [2023-12-27 03:22:15,892][105692] Updated weights for policy 0, policy_version 1644050 (0.0007) [2023-12-27 03:22:15,948][105692] Updated weights for policy 0, policy_version 1644060 (0.0007) [2023-12-27 03:22:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 842735616. Throughput: 0: 9722.3, 1: 9527.2. Samples: 842701816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:22:16,063][104569] Avg episode reward: [(0, '8538.457'), (1, '8991.112')] [2023-12-27 03:22:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001644064_420945920.pth... [2023-12-27 03:22:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001647392_421789696.pth... [2023-12-27 03:22:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001642912_420651008.pth [2023-12-27 03:22:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001646304_421511168.pth [2023-12-27 03:22:16,594][105620] Updated weights for policy 1, policy_version 1647400 (0.0009) [2023-12-27 03:22:16,606][105692] Updated weights for policy 0, policy_version 1644070 (0.0008) [2023-12-27 03:22:16,652][105692] Updated weights for policy 0, policy_version 1644080 (0.0008) [2023-12-27 03:22:16,655][105620] Updated weights for policy 1, policy_version 1647410 (0.0009) [2023-12-27 03:22:16,699][105692] Updated weights for policy 0, policy_version 1644090 (0.0008) [2023-12-27 03:22:16,715][105620] Updated weights for policy 1, policy_version 1647420 (0.0008) [2023-12-27 03:22:17,368][105692] Updated weights for policy 0, policy_version 1644100 (0.0007) [2023-12-27 03:22:17,403][105620] Updated weights for policy 1, policy_version 1647430 (0.0008) [2023-12-27 03:22:17,420][105692] Updated weights for policy 0, policy_version 1644110 (0.0005) [2023-12-27 03:22:17,466][105620] Updated weights for policy 1, policy_version 1647440 (0.0008) [2023-12-27 03:22:17,476][105692] Updated weights for policy 0, policy_version 1644120 (0.0006) [2023-12-27 03:22:17,531][105620] Updated weights for policy 1, policy_version 1647450 (0.0007) [2023-12-27 03:22:18,205][105692] Updated weights for policy 0, policy_version 1644130 (0.0006) [2023-12-27 03:22:18,224][105620] Updated weights for policy 1, policy_version 1647460 (0.0008) [2023-12-27 03:22:18,270][105692] Updated weights for policy 0, policy_version 1644140 (0.0005) [2023-12-27 03:22:18,277][105620] Updated weights for policy 1, policy_version 1647470 (0.0007) [2023-12-27 03:22:18,316][105692] Updated weights for policy 0, policy_version 1644150 (0.0007) [2023-12-27 03:22:18,333][105620] Updated weights for policy 1, policy_version 1647480 (0.0008) [2023-12-27 03:22:18,379][105692] Updated weights for policy 0, policy_version 1644160 (0.0007) [2023-12-27 03:22:19,100][105692] Updated weights for policy 0, policy_version 1644170 (0.0009) [2023-12-27 03:22:19,102][105620] Updated weights for policy 1, policy_version 1647490 (0.0007) [2023-12-27 03:22:19,154][105692] Updated weights for policy 0, policy_version 1644180 (0.0009) [2023-12-27 03:22:19,160][105620] Updated weights for policy 1, policy_version 1647500 (0.0008) [2023-12-27 03:22:19,205][105620] Updated weights for policy 1, policy_version 1647510 (0.0005) [2023-12-27 03:22:19,211][105692] Updated weights for policy 0, policy_version 1644190 (0.0008) [2023-12-27 03:22:19,272][105620] Updated weights for policy 1, policy_version 1647520 (0.0008) [2023-12-27 03:22:20,027][105692] Updated weights for policy 0, policy_version 1644200 (0.0009) [2023-12-27 03:22:20,062][105620] Updated weights for policy 1, policy_version 1647530 (0.0007) [2023-12-27 03:22:20,089][105692] Updated weights for policy 0, policy_version 1644210 (0.0007) [2023-12-27 03:22:20,123][105620] Updated weights for policy 1, policy_version 1647540 (0.0010) [2023-12-27 03:22:20,147][105692] Updated weights for policy 0, policy_version 1644220 (0.0006) [2023-12-27 03:22:20,182][105620] Updated weights for policy 1, policy_version 1647550 (0.0008) [2023-12-27 03:22:20,908][105692] Updated weights for policy 0, policy_version 1644230 (0.0008) [2023-12-27 03:22:20,964][105620] Updated weights for policy 1, policy_version 1647560 (0.0008) [2023-12-27 03:22:20,970][105692] Updated weights for policy 0, policy_version 1644240 (0.0008) [2023-12-27 03:22:21,027][105620] Updated weights for policy 1, policy_version 1647570 (0.0007) [2023-12-27 03:22:21,030][105692] Updated weights for policy 0, policy_version 1644250 (0.0006) [2023-12-27 03:22:21,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 842817536. Throughput: 0: 9758.2, 1: 9467.5. Samples: 842817032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:22:21,062][104569] Avg episode reward: [(0, '8631.324'), (1, '8715.908')] [2023-12-27 03:22:21,090][105620] Updated weights for policy 1, policy_version 1647580 (0.0008) [2023-12-27 03:22:21,827][105692] Updated weights for policy 0, policy_version 1644260 (0.0008) [2023-12-27 03:22:21,892][105692] Updated weights for policy 0, policy_version 1644270 (0.0009) [2023-12-27 03:22:21,909][105620] Updated weights for policy 1, policy_version 1647590 (0.0010) [2023-12-27 03:22:21,946][105692] Updated weights for policy 0, policy_version 1644280 (0.0008) [2023-12-27 03:22:21,970][105620] Updated weights for policy 1, policy_version 1647600 (0.0008) [2023-12-27 03:22:22,033][105620] Updated weights for policy 1, policy_version 1647610 (0.0009) [2023-12-27 03:22:22,714][105620] Updated weights for policy 1, policy_version 1647620 (0.0007) [2023-12-27 03:22:22,770][105620] Updated weights for policy 1, policy_version 1647630 (0.0006) [2023-12-27 03:22:22,787][105692] Updated weights for policy 0, policy_version 1644290 (0.0008) [2023-12-27 03:22:22,829][105620] Updated weights for policy 1, policy_version 1647640 (0.0007) [2023-12-27 03:22:22,849][105692] Updated weights for policy 0, policy_version 1644300 (0.0008) [2023-12-27 03:22:22,908][105692] Updated weights for policy 0, policy_version 1644310 (0.0009) [2023-12-27 03:22:22,961][105692] Updated weights for policy 0, policy_version 1644320 (0.0009) [2023-12-27 03:22:23,401][105620] Updated weights for policy 1, policy_version 1647650 (0.0008) [2023-12-27 03:22:23,462][105620] Updated weights for policy 1, policy_version 1647660 (0.0005) [2023-12-27 03:22:23,518][105620] Updated weights for policy 1, policy_version 1647670 (0.0008) [2023-12-27 03:22:23,564][105620] Updated weights for policy 1, policy_version 1647680 (0.0008) [2023-12-27 03:22:23,821][105692] Updated weights for policy 0, policy_version 1644330 (0.0009) [2023-12-27 03:22:23,873][105692] Updated weights for policy 0, policy_version 1644340 (0.0009) [2023-12-27 03:22:23,931][105692] Updated weights for policy 0, policy_version 1644350 (0.0009) [2023-12-27 03:22:24,302][105620] Updated weights for policy 1, policy_version 1647690 (0.0009) [2023-12-27 03:22:24,368][105620] Updated weights for policy 1, policy_version 1647700 (0.0008) [2023-12-27 03:22:24,422][105620] Updated weights for policy 1, policy_version 1647710 (0.0007) [2023-12-27 03:22:24,683][105692] Updated weights for policy 0, policy_version 1644360 (0.0009) [2023-12-27 03:22:24,739][105692] Updated weights for policy 0, policy_version 1644370 (0.0010) [2023-12-27 03:22:24,829][105692] Updated weights for policy 0, policy_version 1644380 (0.0010) [2023-12-27 03:22:25,070][105620] Updated weights for policy 1, policy_version 1647720 (0.0009) [2023-12-27 03:22:25,121][105620] Updated weights for policy 1, policy_version 1647730 (0.0009) [2023-12-27 03:22:25,172][105620] Updated weights for policy 1, policy_version 1647740 (0.0009) [2023-12-27 03:22:25,554][105692] Updated weights for policy 0, policy_version 1644390 (0.0007) [2023-12-27 03:22:25,613][105692] Updated weights for policy 0, policy_version 1644400 (0.0005) [2023-12-27 03:22:25,679][105692] Updated weights for policy 0, policy_version 1644410 (0.0005) [2023-12-27 03:22:25,989][105620] Updated weights for policy 1, policy_version 1647750 (0.0009) [2023-12-27 03:22:26,039][105620] Updated weights for policy 1, policy_version 1647760 (0.0008) [2023-12-27 03:22:26,062][104569] Fps is (10 sec: 18022.6, 60 sec: 19251.3, 300 sec: 19522.0). Total num frames: 842915840. Throughput: 0: 9770.6, 1: 9463.1. Samples: 842928420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:22:26,062][104569] Avg episode reward: [(0, '8632.528'), (1, '8898.665')] [2023-12-27 03:22:26,096][105620] Updated weights for policy 1, policy_version 1647770 (0.0009) [2023-12-27 03:22:26,286][105692] Updated weights for policy 0, policy_version 1644420 (0.0005) [2023-12-27 03:22:26,350][105692] Updated weights for policy 0, policy_version 1644430 (0.0008) [2023-12-27 03:22:26,404][105692] Updated weights for policy 0, policy_version 1644440 (0.0011) [2023-12-27 03:22:26,934][105620] Updated weights for policy 1, policy_version 1647780 (0.0008) [2023-12-27 03:22:26,948][105692] Updated weights for policy 0, policy_version 1644450 (0.0009) [2023-12-27 03:22:26,982][105620] Updated weights for policy 1, policy_version 1647790 (0.0009) [2023-12-27 03:22:26,999][105692] Updated weights for policy 0, policy_version 1644460 (0.0005) [2023-12-27 03:22:27,030][105620] Updated weights for policy 1, policy_version 1647800 (0.0008) [2023-12-27 03:22:27,049][105692] Updated weights for policy 0, policy_version 1644470 (0.0005) [2023-12-27 03:22:27,095][105692] Updated weights for policy 0, policy_version 1644480 (0.0006) [2023-12-27 03:22:27,685][105692] Updated weights for policy 0, policy_version 1644490 (0.0006) [2023-12-27 03:22:27,751][105692] Updated weights for policy 0, policy_version 1644500 (0.0006) [2023-12-27 03:22:27,809][105692] Updated weights for policy 0, policy_version 1644510 (0.0008) [2023-12-27 03:22:27,864][105620] Updated weights for policy 1, policy_version 1647811 (0.0009) [2023-12-27 03:22:27,919][105620] Updated weights for policy 1, policy_version 1647821 (0.0008) [2023-12-27 03:22:27,988][105620] Updated weights for policy 1, policy_version 1647831 (0.0008) [2023-12-27 03:22:28,382][105692] Updated weights for policy 0, policy_version 1644520 (0.0008) [2023-12-27 03:22:28,438][105692] Updated weights for policy 0, policy_version 1644530 (0.0006) [2023-12-27 03:22:28,497][105692] Updated weights for policy 0, policy_version 1644540 (0.0005) [2023-12-27 03:22:28,898][105620] Updated weights for policy 1, policy_version 1647841 (0.0009) [2023-12-27 03:22:28,961][105620] Updated weights for policy 1, policy_version 1647852 (0.0011) [2023-12-27 03:22:29,018][105620] Updated weights for policy 1, policy_version 1647862 (0.0009) [2023-12-27 03:22:29,026][105692] Updated weights for policy 0, policy_version 1644550 (0.0006) [2023-12-27 03:22:29,066][105620] Updated weights for policy 1, policy_version 1647872 (0.0009) [2023-12-27 03:22:29,075][105692] Updated weights for policy 0, policy_version 1644560 (0.0005) [2023-12-27 03:22:29,134][105692] Updated weights for policy 0, policy_version 1644570 (0.0005) [2023-12-27 03:22:29,760][105692] Updated weights for policy 0, policy_version 1644580 (0.0009) [2023-12-27 03:22:29,791][105620] Updated weights for policy 1, policy_version 1647882 (0.0005) [2023-12-27 03:22:29,816][105692] Updated weights for policy 0, policy_version 1644590 (0.0011) [2023-12-27 03:22:29,857][105620] Updated weights for policy 1, policy_version 1647892 (0.0007) [2023-12-27 03:22:29,875][105692] Updated weights for policy 0, policy_version 1644600 (0.0010) [2023-12-27 03:22:29,918][105620] Updated weights for policy 1, policy_version 1647902 (0.0006) [2023-12-27 03:22:30,475][105620] Updated weights for policy 1, policy_version 1647912 (0.0008) [2023-12-27 03:22:30,531][105620] Updated weights for policy 1, policy_version 1647922 (0.0008) [2023-12-27 03:22:30,579][105620] Updated weights for policy 1, policy_version 1647932 (0.0005) [2023-12-27 03:22:30,622][105692] Updated weights for policy 0, policy_version 1644610 (0.0010) [2023-12-27 03:22:30,668][105692] Updated weights for policy 0, policy_version 1644620 (0.0006) [2023-12-27 03:22:30,712][105692] Updated weights for policy 0, policy_version 1644630 (0.0010) [2023-12-27 03:22:30,759][105692] Updated weights for policy 0, policy_version 1644640 (0.0010) [2023-12-27 03:22:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19251.3, 300 sec: 19549.7). Total num frames: 843022336. Throughput: 0: 9908.3, 1: 9363.4. Samples: 842988348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:22:31,062][104569] Avg episode reward: [(0, '8633.193'), (1, '9172.539')] [2023-12-27 03:22:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001644640_421093376.pth... [2023-12-27 03:22:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001647936_421928960.pth... [2023-12-27 03:22:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001646848_421650432.pth [2023-12-27 03:22:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001643488_420798464.pth [2023-12-27 03:22:31,364][105620] Updated weights for policy 1, policy_version 1647942 (0.0008) [2023-12-27 03:22:31,426][105620] Updated weights for policy 1, policy_version 1647952 (0.0006) [2023-12-27 03:22:31,480][105692] Updated weights for policy 0, policy_version 1644650 (0.0011) [2023-12-27 03:22:31,493][105620] Updated weights for policy 1, policy_version 1647962 (0.0008) [2023-12-27 03:22:31,539][105692] Updated weights for policy 0, policy_version 1644660 (0.0010) [2023-12-27 03:22:31,605][105692] Updated weights for policy 0, policy_version 1644670 (0.0010) [2023-12-27 03:22:32,239][105620] Updated weights for policy 1, policy_version 1647972 (0.0008) [2023-12-27 03:22:32,280][105692] Updated weights for policy 0, policy_version 1644680 (0.0010) [2023-12-27 03:22:32,299][105620] Updated weights for policy 1, policy_version 1647982 (0.0005) [2023-12-27 03:22:32,335][105692] Updated weights for policy 0, policy_version 1644690 (0.0009) [2023-12-27 03:22:32,355][105620] Updated weights for policy 1, policy_version 1647992 (0.0007) [2023-12-27 03:22:32,397][105692] Updated weights for policy 0, policy_version 1644700 (0.0008) [2023-12-27 03:22:32,963][105692] Updated weights for policy 0, policy_version 1644710 (0.0006) [2023-12-27 03:22:33,008][105692] Updated weights for policy 0, policy_version 1644720 (0.0008) [2023-12-27 03:22:33,054][105692] Updated weights for policy 0, policy_version 1644730 (0.0008) [2023-12-27 03:22:33,197][105620] Updated weights for policy 1, policy_version 1648002 (0.0007) [2023-12-27 03:22:33,250][105620] Updated weights for policy 1, policy_version 1648012 (0.0009) [2023-12-27 03:22:33,304][105620] Updated weights for policy 1, policy_version 1648022 (0.0009) [2023-12-27 03:22:33,358][105620] Updated weights for policy 1, policy_version 1648032 (0.0009) [2023-12-27 03:22:33,787][105692] Updated weights for policy 0, policy_version 1644740 (0.0009) [2023-12-27 03:22:33,837][105692] Updated weights for policy 0, policy_version 1644750 (0.0009) [2023-12-27 03:22:33,887][105692] Updated weights for policy 0, policy_version 1644760 (0.0009) [2023-12-27 03:22:34,127][105620] Updated weights for policy 1, policy_version 1648042 (0.0009) [2023-12-27 03:22:34,186][105620] Updated weights for policy 1, policy_version 1648052 (0.0008) [2023-12-27 03:22:34,245][105620] Updated weights for policy 1, policy_version 1648062 (0.0008) [2023-12-27 03:22:34,587][105692] Updated weights for policy 0, policy_version 1644770 (0.0009) [2023-12-27 03:22:34,645][105692] Updated weights for policy 0, policy_version 1644780 (0.0005) [2023-12-27 03:22:34,704][105692] Updated weights for policy 0, policy_version 1644790 (0.0007) [2023-12-27 03:22:34,762][105692] Updated weights for policy 0, policy_version 1644800 (0.0006) [2023-12-27 03:22:34,921][105620] Updated weights for policy 1, policy_version 1648072 (0.0009) [2023-12-27 03:22:34,991][105620] Updated weights for policy 1, policy_version 1648082 (0.0005) [2023-12-27 03:22:35,055][105620] Updated weights for policy 1, policy_version 1648092 (0.0010) [2023-12-27 03:22:35,460][105692] Updated weights for policy 0, policy_version 1644810 (0.0008) [2023-12-27 03:22:35,515][105692] Updated weights for policy 0, policy_version 1644820 (0.0009) [2023-12-27 03:22:35,580][105692] Updated weights for policy 0, policy_version 1644830 (0.0008) [2023-12-27 03:22:35,754][105620] Updated weights for policy 1, policy_version 1648102 (0.0010) [2023-12-27 03:22:35,816][105620] Updated weights for policy 1, policy_version 1648112 (0.0010) [2023-12-27 03:22:35,875][105620] Updated weights for policy 1, policy_version 1648122 (0.0010) [2023-12-27 03:22:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 843120640. Throughput: 0: 9927.9, 1: 9366.9. Samples: 843108424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:22:36,062][104569] Avg episode reward: [(0, '8899.003'), (1, '9172.404')] [2023-12-27 03:22:36,315][105692] Updated weights for policy 0, policy_version 1644840 (0.0010) [2023-12-27 03:22:36,364][105692] Updated weights for policy 0, policy_version 1644850 (0.0011) [2023-12-27 03:22:36,431][105692] Updated weights for policy 0, policy_version 1644860 (0.0011) [2023-12-27 03:22:36,622][105620] Updated weights for policy 1, policy_version 1648132 (0.0009) [2023-12-27 03:22:36,677][105620] Updated weights for policy 1, policy_version 1648142 (0.0010) [2023-12-27 03:22:36,731][105620] Updated weights for policy 1, policy_version 1648152 (0.0010) [2023-12-27 03:22:37,133][105692] Updated weights for policy 0, policy_version 1644870 (0.0010) [2023-12-27 03:22:37,191][105692] Updated weights for policy 0, policy_version 1644880 (0.0010) [2023-12-27 03:22:37,254][105692] Updated weights for policy 0, policy_version 1644890 (0.0011) [2023-12-27 03:22:37,456][105620] Updated weights for policy 1, policy_version 1648162 (0.0010) [2023-12-27 03:22:37,519][105620] Updated weights for policy 1, policy_version 1648172 (0.0010) [2023-12-27 03:22:37,584][105620] Updated weights for policy 1, policy_version 1648182 (0.0010) [2023-12-27 03:22:37,646][105620] Updated weights for policy 1, policy_version 1648192 (0.0007) [2023-12-27 03:22:37,892][105692] Updated weights for policy 0, policy_version 1644900 (0.0010) [2023-12-27 03:22:37,951][105692] Updated weights for policy 0, policy_version 1644910 (0.0011) [2023-12-27 03:22:38,000][105692] Updated weights for policy 0, policy_version 1644920 (0.0010) [2023-12-27 03:22:38,376][105620] Updated weights for policy 1, policy_version 1648202 (0.0010) [2023-12-27 03:22:38,434][105620] Updated weights for policy 1, policy_version 1648212 (0.0010) [2023-12-27 03:22:38,499][105620] Updated weights for policy 1, policy_version 1648222 (0.0010) [2023-12-27 03:22:38,775][105692] Updated weights for policy 0, policy_version 1644930 (0.0011) [2023-12-27 03:22:38,837][105692] Updated weights for policy 0, policy_version 1644940 (0.0011) [2023-12-27 03:22:38,892][105692] Updated weights for policy 0, policy_version 1644950 (0.0010) [2023-12-27 03:22:38,955][105692] Updated weights for policy 0, policy_version 1644960 (0.0011) [2023-12-27 03:22:39,224][105620] Updated weights for policy 1, policy_version 1648232 (0.0010) [2023-12-27 03:22:39,283][105620] Updated weights for policy 1, policy_version 1648242 (0.0011) [2023-12-27 03:22:39,347][105620] Updated weights for policy 1, policy_version 1648252 (0.0011) [2023-12-27 03:22:39,679][105692] Updated weights for policy 0, policy_version 1644970 (0.0009) [2023-12-27 03:22:39,739][105692] Updated weights for policy 0, policy_version 1644980 (0.0011) [2023-12-27 03:22:39,792][105692] Updated weights for policy 0, policy_version 1644990 (0.0011) [2023-12-27 03:22:40,124][105620] Updated weights for policy 1, policy_version 1648262 (0.0008) [2023-12-27 03:22:40,179][105620] Updated weights for policy 1, policy_version 1648272 (0.0009) [2023-12-27 03:22:40,232][105620] Updated weights for policy 1, policy_version 1648282 (0.0008) [2023-12-27 03:22:40,553][105692] Updated weights for policy 0, policy_version 1645000 (0.0010) [2023-12-27 03:22:40,616][105692] Updated weights for policy 0, policy_version 1645010 (0.0010) [2023-12-27 03:22:40,682][105692] Updated weights for policy 0, policy_version 1645020 (0.0011) [2023-12-27 03:22:40,977][105620] Updated weights for policy 1, policy_version 1648292 (0.0009) [2023-12-27 03:22:41,036][105620] Updated weights for policy 1, policy_version 1648302 (0.0008) [2023-12-27 03:22:41,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 843210752. Throughput: 0: 9878.4, 1: 9404.1. Samples: 843223040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:22:41,063][104569] Avg episode reward: [(0, '8806.562'), (1, '8990.847')] [2023-12-27 03:22:41,097][105620] Updated weights for policy 1, policy_version 1648312 (0.0008) [2023-12-27 03:22:41,404][105692] Updated weights for policy 0, policy_version 1645030 (0.0008) [2023-12-27 03:22:41,468][105692] Updated weights for policy 0, policy_version 1645040 (0.0006) [2023-12-27 03:22:41,531][105692] Updated weights for policy 0, policy_version 1645050 (0.0009) [2023-12-27 03:22:41,917][105620] Updated weights for policy 1, policy_version 1648322 (0.0007) [2023-12-27 03:22:41,981][105620] Updated weights for policy 1, policy_version 1648332 (0.0008) [2023-12-27 03:22:42,044][105620] Updated weights for policy 1, policy_version 1648342 (0.0009) [2023-12-27 03:22:42,100][105620] Updated weights for policy 1, policy_version 1648352 (0.0010) [2023-12-27 03:22:42,212][105692] Updated weights for policy 0, policy_version 1645060 (0.0008) [2023-12-27 03:22:42,272][105692] Updated weights for policy 0, policy_version 1645070 (0.0007) [2023-12-27 03:22:42,333][105692] Updated weights for policy 0, policy_version 1645080 (0.0008) [2023-12-27 03:22:42,913][105620] Updated weights for policy 1, policy_version 1648362 (0.0008) [2023-12-27 03:22:42,972][105620] Updated weights for policy 1, policy_version 1648372 (0.0005) [2023-12-27 03:22:42,993][105692] Updated weights for policy 0, policy_version 1645090 (0.0008) [2023-12-27 03:22:43,029][105620] Updated weights for policy 1, policy_version 1648382 (0.0005) [2023-12-27 03:22:43,054][105692] Updated weights for policy 0, policy_version 1645100 (0.0009) [2023-12-27 03:22:43,107][105692] Updated weights for policy 0, policy_version 1645110 (0.0009) [2023-12-27 03:22:43,163][105692] Updated weights for policy 0, policy_version 1645120 (0.0005) [2023-12-27 03:22:43,634][105620] Updated weights for policy 1, policy_version 1648392 (0.0008) [2023-12-27 03:22:43,683][105620] Updated weights for policy 1, policy_version 1648402 (0.0008) [2023-12-27 03:22:43,733][105620] Updated weights for policy 1, policy_version 1648412 (0.0008) [2023-12-27 03:22:43,939][105692] Updated weights for policy 0, policy_version 1645130 (0.0010) [2023-12-27 03:22:43,990][105692] Updated weights for policy 0, policy_version 1645140 (0.0010) [2023-12-27 03:22:44,049][105692] Updated weights for policy 0, policy_version 1645150 (0.0010) [2023-12-27 03:22:44,434][105620] Updated weights for policy 1, policy_version 1648422 (0.0010) [2023-12-27 03:22:44,484][105620] Updated weights for policy 1, policy_version 1648432 (0.0010) [2023-12-27 03:22:44,535][105620] Updated weights for policy 1, policy_version 1648442 (0.0010) [2023-12-27 03:22:44,811][105692] Updated weights for policy 0, policy_version 1645160 (0.0008) [2023-12-27 03:22:44,873][105692] Updated weights for policy 0, policy_version 1645170 (0.0011) [2023-12-27 03:22:44,938][105692] Updated weights for policy 0, policy_version 1645180 (0.0010) [2023-12-27 03:22:45,205][105620] Updated weights for policy 1, policy_version 1648452 (0.0007) [2023-12-27 03:22:45,258][105620] Updated weights for policy 1, policy_version 1648462 (0.0008) [2023-12-27 03:22:45,312][105620] Updated weights for policy 1, policy_version 1648472 (0.0008) [2023-12-27 03:22:45,688][105692] Updated weights for policy 0, policy_version 1645190 (0.0007) [2023-12-27 03:22:45,748][105692] Updated weights for policy 0, policy_version 1645200 (0.0009) [2023-12-27 03:22:45,809][105692] Updated weights for policy 0, policy_version 1645210 (0.0008) [2023-12-27 03:22:45,979][105620] Updated weights for policy 1, policy_version 1648482 (0.0008) [2023-12-27 03:22:46,046][105620] Updated weights for policy 1, policy_version 1648492 (0.0005) [2023-12-27 03:22:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 843309056. Throughput: 0: 9883.6, 1: 9369.9. Samples: 843280228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:22:46,063][104569] Avg episode reward: [(0, '8628.259'), (1, '9082.432')] [2023-12-27 03:22:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001645216_421240832.pth... [2023-12-27 03:22:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001644064_420945920.pth [2023-12-27 03:22:46,115][105620] Updated weights for policy 1, policy_version 1648502 (0.0005) [2023-12-27 03:22:46,169][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001648512_422076416.pth... [2023-12-27 03:22:46,170][105620] Updated weights for policy 1, policy_version 1648512 (0.0005) [2023-12-27 03:22:46,173][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001647392_421789696.pth [2023-12-27 03:22:46,486][105692] Updated weights for policy 0, policy_version 1645220 (0.0005) [2023-12-27 03:22:46,551][105692] Updated weights for policy 0, policy_version 1645230 (0.0005) [2023-12-27 03:22:46,606][105692] Updated weights for policy 0, policy_version 1645240 (0.0005) [2023-12-27 03:22:46,679][105620] Updated weights for policy 1, policy_version 1648522 (0.0009) [2023-12-27 03:22:46,738][105620] Updated weights for policy 1, policy_version 1648532 (0.0010) [2023-12-27 03:22:46,802][105620] Updated weights for policy 1, policy_version 1648542 (0.0010) [2023-12-27 03:22:47,285][105692] Updated weights for policy 0, policy_version 1645250 (0.0006) [2023-12-27 03:22:47,331][105692] Updated weights for policy 0, policy_version 1645260 (0.0009) [2023-12-27 03:22:47,388][105692] Updated weights for policy 0, policy_version 1645270 (0.0010) [2023-12-27 03:22:47,406][105620] Updated weights for policy 1, policy_version 1648552 (0.0007) [2023-12-27 03:22:47,449][105692] Updated weights for policy 0, policy_version 1645280 (0.0008) [2023-12-27 03:22:47,460][105620] Updated weights for policy 1, policy_version 1648562 (0.0009) [2023-12-27 03:22:47,518][105620] Updated weights for policy 1, policy_version 1648572 (0.0009) [2023-12-27 03:22:48,111][105692] Updated weights for policy 0, policy_version 1645290 (0.0008) [2023-12-27 03:22:48,163][105692] Updated weights for policy 0, policy_version 1645300 (0.0008) [2023-12-27 03:22:48,178][105620] Updated weights for policy 1, policy_version 1648582 (0.0007) [2023-12-27 03:22:48,224][105692] Updated weights for policy 0, policy_version 1645310 (0.0007) [2023-12-27 03:22:48,230][105620] Updated weights for policy 1, policy_version 1648592 (0.0006) [2023-12-27 03:22:48,290][105620] Updated weights for policy 1, policy_version 1648602 (0.0008) [2023-12-27 03:22:48,993][105692] Updated weights for policy 0, policy_version 1645320 (0.0009) [2023-12-27 03:22:49,031][105620] Updated weights for policy 1, policy_version 1648612 (0.0008) [2023-12-27 03:22:49,054][105692] Updated weights for policy 0, policy_version 1645330 (0.0008) [2023-12-27 03:22:49,086][105620] Updated weights for policy 1, policy_version 1648622 (0.0008) [2023-12-27 03:22:49,108][105692] Updated weights for policy 0, policy_version 1645340 (0.0006) [2023-12-27 03:22:49,140][105620] Updated weights for policy 1, policy_version 1648632 (0.0006) [2023-12-27 03:22:49,813][105620] Updated weights for policy 1, policy_version 1648642 (0.0008) [2023-12-27 03:22:49,876][105620] Updated weights for policy 1, policy_version 1648652 (0.0008) [2023-12-27 03:22:49,930][105620] Updated weights for policy 1, policy_version 1648662 (0.0010) [2023-12-27 03:22:49,938][105692] Updated weights for policy 0, policy_version 1645350 (0.0007) [2023-12-27 03:22:49,987][105620] Updated weights for policy 1, policy_version 1648672 (0.0009) [2023-12-27 03:22:50,000][105692] Updated weights for policy 0, policy_version 1645360 (0.0007) [2023-12-27 03:22:50,057][105692] Updated weights for policy 0, policy_version 1645370 (0.0007) [2023-12-27 03:22:50,620][105620] Updated weights for policy 1, policy_version 1648682 (0.0011) [2023-12-27 03:22:50,685][105620] Updated weights for policy 1, policy_version 1648692 (0.0009) [2023-12-27 03:22:50,747][105620] Updated weights for policy 1, policy_version 1648702 (0.0011) [2023-12-27 03:22:50,855][105692] Updated weights for policy 0, policy_version 1645380 (0.0006) [2023-12-27 03:22:50,916][105692] Updated weights for policy 0, policy_version 1645390 (0.0010) [2023-12-27 03:22:50,974][105692] Updated weights for policy 0, policy_version 1645400 (0.0007) [2023-12-27 03:22:51,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 843415552. Throughput: 0: 9816.7, 1: 9527.9. Samples: 843401568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:22:51,063][104569] Avg episode reward: [(0, '8630.438'), (1, '8899.295')] [2023-12-27 03:22:51,493][105620] Updated weights for policy 1, policy_version 1648712 (0.0011) [2023-12-27 03:22:51,549][105620] Updated weights for policy 1, policy_version 1648722 (0.0008) [2023-12-27 03:22:51,613][105620] Updated weights for policy 1, policy_version 1648732 (0.0007) [2023-12-27 03:22:51,714][105692] Updated weights for policy 0, policy_version 1645410 (0.0007) [2023-12-27 03:22:51,777][105692] Updated weights for policy 0, policy_version 1645420 (0.0009) [2023-12-27 03:22:51,826][105692] Updated weights for policy 0, policy_version 1645430 (0.0009) [2023-12-27 03:22:51,875][105692] Updated weights for policy 0, policy_version 1645440 (0.0009) [2023-12-27 03:22:52,370][105620] Updated weights for policy 1, policy_version 1648742 (0.0010) [2023-12-27 03:22:52,436][105620] Updated weights for policy 1, policy_version 1648752 (0.0009) [2023-12-27 03:22:52,495][105620] Updated weights for policy 1, policy_version 1648762 (0.0007) [2023-12-27 03:22:52,645][105692] Updated weights for policy 0, policy_version 1645450 (0.0009) [2023-12-27 03:22:52,708][105692] Updated weights for policy 0, policy_version 1645460 (0.0009) [2023-12-27 03:22:52,767][105692] Updated weights for policy 0, policy_version 1645470 (0.0008) [2023-12-27 03:22:53,264][105620] Updated weights for policy 1, policy_version 1648772 (0.0010) [2023-12-27 03:22:53,316][105620] Updated weights for policy 1, policy_version 1648782 (0.0009) [2023-12-27 03:22:53,368][105620] Updated weights for policy 1, policy_version 1648793 (0.0010) [2023-12-27 03:22:53,381][105692] Updated weights for policy 0, policy_version 1645480 (0.0005) [2023-12-27 03:22:53,443][105692] Updated weights for policy 0, policy_version 1645490 (0.0007) [2023-12-27 03:22:53,508][105692] Updated weights for policy 0, policy_version 1645500 (0.0009) [2023-12-27 03:22:54,165][105620] Updated weights for policy 1, policy_version 1648803 (0.0008) [2023-12-27 03:22:54,221][105692] Updated weights for policy 0, policy_version 1645510 (0.0007) [2023-12-27 03:22:54,223][105620] Updated weights for policy 1, policy_version 1648813 (0.0008) [2023-12-27 03:22:54,281][105692] Updated weights for policy 0, policy_version 1645520 (0.0007) [2023-12-27 03:22:54,283][105620] Updated weights for policy 1, policy_version 1648823 (0.0009) [2023-12-27 03:22:54,329][105692] Updated weights for policy 0, policy_version 1645530 (0.0006) [2023-12-27 03:22:55,028][105620] Updated weights for policy 1, policy_version 1648833 (0.0008) [2023-12-27 03:22:55,061][105692] Updated weights for policy 0, policy_version 1645540 (0.0008) [2023-12-27 03:22:55,085][105620] Updated weights for policy 1, policy_version 1648843 (0.0010) [2023-12-27 03:22:55,117][105692] Updated weights for policy 0, policy_version 1645550 (0.0008) [2023-12-27 03:22:55,140][105620] Updated weights for policy 1, policy_version 1648853 (0.0008) [2023-12-27 03:22:55,175][105692] Updated weights for policy 0, policy_version 1645560 (0.0009) [2023-12-27 03:22:55,190][105620] Updated weights for policy 1, policy_version 1648863 (0.0008) [2023-12-27 03:22:55,930][105692] Updated weights for policy 0, policy_version 1645570 (0.0008) [2023-12-27 03:22:55,962][105620] Updated weights for policy 1, policy_version 1648873 (0.0008) [2023-12-27 03:22:55,981][105692] Updated weights for policy 0, policy_version 1645580 (0.0006) [2023-12-27 03:22:56,014][105620] Updated weights for policy 1, policy_version 1648883 (0.0007) [2023-12-27 03:22:56,043][105692] Updated weights for policy 0, policy_version 1645590 (0.0008) [2023-12-27 03:22:56,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 843497472. Throughput: 0: 9780.0, 1: 9541.2. Samples: 843514832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:22:56,063][104569] Avg episode reward: [(0, '8811.941'), (1, '8624.461')] [2023-12-27 03:22:56,064][105620] Updated weights for policy 1, policy_version 1648893 (0.0010) [2023-12-27 03:22:56,107][105692] Updated weights for policy 0, policy_version 1645600 (0.0008) [2023-12-27 03:22:56,648][105692] Updated weights for policy 0, policy_version 1645610 (0.0010) [2023-12-27 03:22:56,701][105692] Updated weights for policy 0, policy_version 1645620 (0.0007) [2023-12-27 03:22:56,749][105692] Updated weights for policy 0, policy_version 1645630 (0.0005) [2023-12-27 03:22:56,956][105620] Updated weights for policy 1, policy_version 1648903 (0.0008) [2023-12-27 03:22:57,021][105620] Updated weights for policy 1, policy_version 1648913 (0.0010) [2023-12-27 03:22:57,090][105620] Updated weights for policy 1, policy_version 1648923 (0.0009) [2023-12-27 03:22:57,293][105692] Updated weights for policy 0, policy_version 1645640 (0.0005) [2023-12-27 03:22:57,350][105692] Updated weights for policy 0, policy_version 1645650 (0.0006) [2023-12-27 03:22:57,408][105692] Updated weights for policy 0, policy_version 1645660 (0.0010) [2023-12-27 03:22:57,827][105620] Updated weights for policy 1, policy_version 1648933 (0.0010) [2023-12-27 03:22:57,852][105586] KL-divergence is very high: 102.7642 [2023-12-27 03:22:57,885][105620] Updated weights for policy 1, policy_version 1648943 (0.0010) [2023-12-27 03:22:57,894][105586] KL-divergence is very high: 151.3924 [2023-12-27 03:22:57,941][105586] KL-divergence is very high: 152.8270 [2023-12-27 03:22:57,942][105620] Updated weights for policy 1, policy_version 1648953 (0.0009) [2023-12-27 03:22:57,952][105692] Updated weights for policy 0, policy_version 1645670 (0.0009) [2023-12-27 03:22:58,004][105692] Updated weights for policy 0, policy_version 1645680 (0.0009) [2023-12-27 03:22:58,070][105692] Updated weights for policy 0, policy_version 1645690 (0.0005) [2023-12-27 03:22:58,722][105692] Updated weights for policy 0, policy_version 1645700 (0.0008) [2023-12-27 03:22:58,780][105620] Updated weights for policy 1, policy_version 1648963 (0.0006) [2023-12-27 03:22:58,785][105692] Updated weights for policy 0, policy_version 1645710 (0.0010) [2023-12-27 03:22:58,841][105620] Updated weights for policy 1, policy_version 1648973 (0.0008) [2023-12-27 03:22:58,850][105692] Updated weights for policy 0, policy_version 1645720 (0.0010) [2023-12-27 03:22:58,906][105620] Updated weights for policy 1, policy_version 1648983 (0.0007) [2023-12-27 03:22:59,680][105692] Updated weights for policy 0, policy_version 1645730 (0.0009) [2023-12-27 03:22:59,728][105692] Updated weights for policy 0, policy_version 1645740 (0.0005) [2023-12-27 03:22:59,730][105620] Updated weights for policy 1, policy_version 1648993 (0.0009) [2023-12-27 03:22:59,783][105620] Updated weights for policy 1, policy_version 1649003 (0.0011) [2023-12-27 03:22:59,785][105692] Updated weights for policy 0, policy_version 1645750 (0.0006) [2023-12-27 03:22:59,843][105620] Updated weights for policy 1, policy_version 1649013 (0.0009) [2023-12-27 03:22:59,846][105692] Updated weights for policy 0, policy_version 1645760 (0.0007) [2023-12-27 03:22:59,915][105620] Updated weights for policy 1, policy_version 1649023 (0.0007) [2023-12-27 03:23:00,584][105692] Updated weights for policy 0, policy_version 1645770 (0.0010) [2023-12-27 03:23:00,594][105620] Updated weights for policy 1, policy_version 1649033 (0.0006) [2023-12-27 03:23:00,642][105692] Updated weights for policy 0, policy_version 1645780 (0.0010) [2023-12-27 03:23:00,652][105620] Updated weights for policy 1, policy_version 1649043 (0.0005) [2023-12-27 03:23:00,683][105692] Updated weights for policy 0, policy_version 1645790 (0.0010) [2023-12-27 03:23:00,701][105620] Updated weights for policy 1, policy_version 1649053 (0.0006) [2023-12-27 03:23:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 843603968. Throughput: 0: 9940.3, 1: 9464.1. Samples: 843575016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:23:01,063][104569] Avg episode reward: [(0, '8624.379'), (1, '8713.125')] [2023-12-27 03:23:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001645792_421388288.pth... [2023-12-27 03:23:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001649056_422215680.pth... [2023-12-27 03:23:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001644640_421093376.pth [2023-12-27 03:23:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001647936_421928960.pth [2023-12-27 03:23:01,428][105620] Updated weights for policy 1, policy_version 1649063 (0.0007) [2023-12-27 03:23:01,445][105692] Updated weights for policy 0, policy_version 1645800 (0.0009) [2023-12-27 03:23:01,482][105620] Updated weights for policy 1, policy_version 1649073 (0.0009) [2023-12-27 03:23:01,500][105692] Updated weights for policy 0, policy_version 1645810 (0.0008) [2023-12-27 03:23:01,534][105620] Updated weights for policy 1, policy_version 1649083 (0.0006) [2023-12-27 03:23:01,564][105692] Updated weights for policy 0, policy_version 1645820 (0.0008) [2023-12-27 03:23:02,286][105620] Updated weights for policy 1, policy_version 1649093 (0.0006) [2023-12-27 03:23:02,290][105692] Updated weights for policy 0, policy_version 1645830 (0.0009) [2023-12-27 03:23:02,347][105692] Updated weights for policy 0, policy_version 1645840 (0.0007) [2023-12-27 03:23:02,353][105620] Updated weights for policy 1, policy_version 1649103 (0.0008) [2023-12-27 03:23:02,411][105692] Updated weights for policy 0, policy_version 1645850 (0.0006) [2023-12-27 03:23:02,414][105620] Updated weights for policy 1, policy_version 1649113 (0.0008) [2023-12-27 03:23:03,070][105692] Updated weights for policy 0, policy_version 1645860 (0.0006) [2023-12-27 03:23:03,118][105620] Updated weights for policy 1, policy_version 1649123 (0.0007) [2023-12-27 03:23:03,129][105692] Updated weights for policy 0, policy_version 1645870 (0.0006) [2023-12-27 03:23:03,171][105620] Updated weights for policy 1, policy_version 1649133 (0.0008) [2023-12-27 03:23:03,176][105692] Updated weights for policy 0, policy_version 1645880 (0.0006) [2023-12-27 03:23:03,219][105620] Updated weights for policy 1, policy_version 1649143 (0.0008) [2023-12-27 03:23:03,729][105692] Updated weights for policy 0, policy_version 1645890 (0.0006) [2023-12-27 03:23:03,778][105692] Updated weights for policy 0, policy_version 1645901 (0.0008) [2023-12-27 03:23:03,794][105620] Updated weights for policy 1, policy_version 1649153 (0.0009) [2023-12-27 03:23:03,824][105692] Updated weights for policy 0, policy_version 1645911 (0.0007) [2023-12-27 03:23:03,854][105620] Updated weights for policy 1, policy_version 1649163 (0.0008) [2023-12-27 03:23:03,912][105620] Updated weights for policy 1, policy_version 1649173 (0.0008) [2023-12-27 03:23:03,976][105620] Updated weights for policy 1, policy_version 1649183 (0.0008) [2023-12-27 03:23:04,501][105692] Updated weights for policy 0, policy_version 1645921 (0.0008) [2023-12-27 03:23:04,561][105692] Updated weights for policy 0, policy_version 1645931 (0.0010) [2023-12-27 03:23:04,615][105692] Updated weights for policy 0, policy_version 1645941 (0.0010) [2023-12-27 03:23:04,664][105692] Updated weights for policy 0, policy_version 1645951 (0.0010) [2023-12-27 03:23:04,780][105620] Updated weights for policy 1, policy_version 1649193 (0.0008) [2023-12-27 03:23:04,827][105620] Updated weights for policy 1, policy_version 1649203 (0.0007) [2023-12-27 03:23:04,872][105620] Updated weights for policy 1, policy_version 1649213 (0.0008) [2023-12-27 03:23:05,428][105692] Updated weights for policy 0, policy_version 1645961 (0.0010) [2023-12-27 03:23:05,493][105692] Updated weights for policy 0, policy_version 1645971 (0.0011) [2023-12-27 03:23:05,549][105620] Updated weights for policy 1, policy_version 1649223 (0.0006) [2023-12-27 03:23:05,552][105692] Updated weights for policy 0, policy_version 1645981 (0.0011) [2023-12-27 03:23:05,602][105620] Updated weights for policy 1, policy_version 1649233 (0.0005) [2023-12-27 03:23:05,649][105620] Updated weights for policy 1, policy_version 1649243 (0.0005) [2023-12-27 03:23:06,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 843702272. Throughput: 0: 9928.6, 1: 9521.6. Samples: 843692292. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:23:06,063][104569] Avg episode reward: [(0, '8353.414'), (1, '8988.977')] [2023-12-27 03:23:06,218][105692] Updated weights for policy 0, policy_version 1645991 (0.0011) [2023-12-27 03:23:06,287][105692] Updated weights for policy 0, policy_version 1646001 (0.0011) [2023-12-27 03:23:06,322][105620] Updated weights for policy 1, policy_version 1649253 (0.0006) [2023-12-27 03:23:06,344][105692] Updated weights for policy 0, policy_version 1646011 (0.0010) [2023-12-27 03:23:06,370][105620] Updated weights for policy 1, policy_version 1649263 (0.0006) [2023-12-27 03:23:06,418][105620] Updated weights for policy 1, policy_version 1649273 (0.0007) [2023-12-27 03:23:06,982][105692] Updated weights for policy 0, policy_version 1646021 (0.0008) [2023-12-27 03:23:07,036][105692] Updated weights for policy 0, policy_version 1646031 (0.0006) [2023-12-27 03:23:07,087][105692] Updated weights for policy 0, policy_version 1646041 (0.0006) [2023-12-27 03:23:07,282][105620] Updated weights for policy 1, policy_version 1649283 (0.0009) [2023-12-27 03:23:07,341][105620] Updated weights for policy 1, policy_version 1649293 (0.0010) [2023-12-27 03:23:07,400][105620] Updated weights for policy 1, policy_version 1649303 (0.0010) [2023-12-27 03:23:07,700][105692] Updated weights for policy 0, policy_version 1646051 (0.0007) [2023-12-27 03:23:07,749][105692] Updated weights for policy 0, policy_version 1646061 (0.0010) [2023-12-27 03:23:07,796][105692] Updated weights for policy 0, policy_version 1646071 (0.0010) [2023-12-27 03:23:08,159][105620] Updated weights for policy 1, policy_version 1649313 (0.0009) [2023-12-27 03:23:08,222][105620] Updated weights for policy 1, policy_version 1649323 (0.0008) [2023-12-27 03:23:08,288][105620] Updated weights for policy 1, policy_version 1649333 (0.0008) [2023-12-27 03:23:08,379][105620] Updated weights for policy 1, policy_version 1649343 (0.0010) [2023-12-27 03:23:08,556][105692] Updated weights for policy 0, policy_version 1646081 (0.0010) [2023-12-27 03:23:08,616][105692] Updated weights for policy 0, policy_version 1646091 (0.0010) [2023-12-27 03:23:08,677][105692] Updated weights for policy 0, policy_version 1646101 (0.0009) [2023-12-27 03:23:08,735][105692] Updated weights for policy 0, policy_version 1646111 (0.0009) [2023-12-27 03:23:09,014][105620] Updated weights for policy 1, policy_version 1649353 (0.0009) [2023-12-27 03:23:09,066][105620] Updated weights for policy 1, policy_version 1649363 (0.0009) [2023-12-27 03:23:09,122][105620] Updated weights for policy 1, policy_version 1649373 (0.0008) [2023-12-27 03:23:09,501][105692] Updated weights for policy 0, policy_version 1646121 (0.0009) [2023-12-27 03:23:09,554][105692] Updated weights for policy 0, policy_version 1646131 (0.0008) [2023-12-27 03:23:09,607][105692] Updated weights for policy 0, policy_version 1646141 (0.0008) [2023-12-27 03:23:09,861][105620] Updated weights for policy 1, policy_version 1649383 (0.0008) [2023-12-27 03:23:09,918][105620] Updated weights for policy 1, policy_version 1649393 (0.0011) [2023-12-27 03:23:09,981][105620] Updated weights for policy 1, policy_version 1649403 (0.0008) [2023-12-27 03:23:10,456][105692] Updated weights for policy 0, policy_version 1646151 (0.0009) [2023-12-27 03:23:10,506][105692] Updated weights for policy 0, policy_version 1646161 (0.0008) [2023-12-27 03:23:10,565][105692] Updated weights for policy 0, policy_version 1646171 (0.0009) [2023-12-27 03:23:10,606][105620] Updated weights for policy 1, policy_version 1649413 (0.0006) [2023-12-27 03:23:10,671][105620] Updated weights for policy 1, policy_version 1649423 (0.0005) [2023-12-27 03:23:10,740][105620] Updated weights for policy 1, policy_version 1649433 (0.0006) [2023-12-27 03:23:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 843800576. Throughput: 0: 10018.9, 1: 9564.1. Samples: 843809656. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:23:11,062][104569] Avg episode reward: [(0, '8533.491'), (1, '9173.873')] [2023-12-27 03:23:11,344][105620] Updated weights for policy 1, policy_version 1649443 (0.0006) [2023-12-27 03:23:11,380][105692] Updated weights for policy 0, policy_version 1646181 (0.0009) [2023-12-27 03:23:11,415][105620] Updated weights for policy 1, policy_version 1649453 (0.0008) [2023-12-27 03:23:11,447][105692] Updated weights for policy 0, policy_version 1646191 (0.0008) [2023-12-27 03:23:11,477][105620] Updated weights for policy 1, policy_version 1649463 (0.0007) [2023-12-27 03:23:11,510][105692] Updated weights for policy 0, policy_version 1646201 (0.0008) [2023-12-27 03:23:12,272][105692] Updated weights for policy 0, policy_version 1646211 (0.0008) [2023-12-27 03:23:12,301][105620] Updated weights for policy 1, policy_version 1649473 (0.0008) [2023-12-27 03:23:12,329][105692] Updated weights for policy 0, policy_version 1646221 (0.0007) [2023-12-27 03:23:12,366][105620] Updated weights for policy 1, policy_version 1649483 (0.0008) [2023-12-27 03:23:12,390][105692] Updated weights for policy 0, policy_version 1646231 (0.0008) [2023-12-27 03:23:12,429][105620] Updated weights for policy 1, policy_version 1649493 (0.0008) [2023-12-27 03:23:12,502][105620] Updated weights for policy 1, policy_version 1649503 (0.0009) [2023-12-27 03:23:12,997][105692] Updated weights for policy 0, policy_version 1646241 (0.0007) [2023-12-27 03:23:13,044][105692] Updated weights for policy 0, policy_version 1646251 (0.0008) [2023-12-27 03:23:13,109][105692] Updated weights for policy 0, policy_version 1646261 (0.0009) [2023-12-27 03:23:13,164][105692] Updated weights for policy 0, policy_version 1646271 (0.0010) [2023-12-27 03:23:13,258][105620] Updated weights for policy 1, policy_version 1649513 (0.0006) [2023-12-27 03:23:13,321][105620] Updated weights for policy 1, policy_version 1649523 (0.0005) [2023-12-27 03:23:13,381][105620] Updated weights for policy 1, policy_version 1649533 (0.0008) [2023-12-27 03:23:13,880][105692] Updated weights for policy 0, policy_version 1646281 (0.0009) [2023-12-27 03:23:13,930][105692] Updated weights for policy 0, policy_version 1646291 (0.0008) [2023-12-27 03:23:13,982][105692] Updated weights for policy 0, policy_version 1646301 (0.0008) [2023-12-27 03:23:14,060][105620] Updated weights for policy 1, policy_version 1649543 (0.0008) [2023-12-27 03:23:14,121][105620] Updated weights for policy 1, policy_version 1649553 (0.0010) [2023-12-27 03:23:14,183][105620] Updated weights for policy 1, policy_version 1649563 (0.0009) [2023-12-27 03:23:14,653][105692] Updated weights for policy 0, policy_version 1646311 (0.0006) [2023-12-27 03:23:14,712][105692] Updated weights for policy 0, policy_version 1646321 (0.0005) [2023-12-27 03:23:14,772][105692] Updated weights for policy 0, policy_version 1646331 (0.0007) [2023-12-27 03:23:15,001][105620] Updated weights for policy 1, policy_version 1649573 (0.0009) [2023-12-27 03:23:15,064][105620] Updated weights for policy 1, policy_version 1649583 (0.0008) [2023-12-27 03:23:15,124][105620] Updated weights for policy 1, policy_version 1649593 (0.0008) [2023-12-27 03:23:15,424][105692] Updated weights for policy 0, policy_version 1646341 (0.0008) [2023-12-27 03:23:15,480][105692] Updated weights for policy 0, policy_version 1646351 (0.0005) [2023-12-27 03:23:15,537][105692] Updated weights for policy 0, policy_version 1646361 (0.0006) [2023-12-27 03:23:15,918][105620] Updated weights for policy 1, policy_version 1649603 (0.0008) [2023-12-27 03:23:15,964][105620] Updated weights for policy 1, policy_version 1649613 (0.0007) [2023-12-27 03:23:16,013][105620] Updated weights for policy 1, policy_version 1649623 (0.0008) [2023-12-27 03:23:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 843898880. Throughput: 0: 9898.8, 1: 9620.0. Samples: 843866700. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:23:16,063][104569] Avg episode reward: [(0, '8806.493'), (1, '9173.738')] [2023-12-27 03:23:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001646368_421535744.pth... [2023-12-27 03:23:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001649632_422363136.pth... [2023-12-27 03:23:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001645216_421240832.pth [2023-12-27 03:23:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001648512_422076416.pth [2023-12-27 03:23:16,222][105692] Updated weights for policy 0, policy_version 1646371 (0.0011) [2023-12-27 03:23:16,280][105692] Updated weights for policy 0, policy_version 1646381 (0.0010) [2023-12-27 03:23:16,344][105692] Updated weights for policy 0, policy_version 1646391 (0.0010) [2023-12-27 03:23:16,759][105620] Updated weights for policy 1, policy_version 1649633 (0.0008) [2023-12-27 03:23:16,809][105620] Updated weights for policy 1, policy_version 1649643 (0.0008) [2023-12-27 03:23:16,858][105620] Updated weights for policy 1, policy_version 1649653 (0.0008) [2023-12-27 03:23:16,905][105620] Updated weights for policy 1, policy_version 1649663 (0.0007) [2023-12-27 03:23:17,068][105692] Updated weights for policy 0, policy_version 1646401 (0.0010) [2023-12-27 03:23:17,124][105692] Updated weights for policy 0, policy_version 1646411 (0.0010) [2023-12-27 03:23:17,176][105692] Updated weights for policy 0, policy_version 1646421 (0.0010) [2023-12-27 03:23:17,224][105692] Updated weights for policy 0, policy_version 1646431 (0.0010) [2023-12-27 03:23:17,666][105620] Updated weights for policy 1, policy_version 1649673 (0.0008) [2023-12-27 03:23:17,709][105620] Updated weights for policy 1, policy_version 1649683 (0.0008) [2023-12-27 03:23:17,761][105620] Updated weights for policy 1, policy_version 1649693 (0.0008) [2023-12-27 03:23:17,982][105692] Updated weights for policy 0, policy_version 1646441 (0.0008) [2023-12-27 03:23:18,037][105692] Updated weights for policy 0, policy_version 1646451 (0.0005) [2023-12-27 03:23:18,083][105692] Updated weights for policy 0, policy_version 1646461 (0.0005) [2023-12-27 03:23:18,586][105620] Updated weights for policy 1, policy_version 1649703 (0.0008) [2023-12-27 03:23:18,651][105620] Updated weights for policy 1, policy_version 1649713 (0.0009) [2023-12-27 03:23:18,713][105620] Updated weights for policy 1, policy_version 1649723 (0.0010) [2023-12-27 03:23:18,760][105692] Updated weights for policy 0, policy_version 1646471 (0.0009) [2023-12-27 03:23:18,819][105692] Updated weights for policy 0, policy_version 1646481 (0.0010) [2023-12-27 03:23:18,878][105692] Updated weights for policy 0, policy_version 1646491 (0.0011) [2023-12-27 03:23:19,412][105620] Updated weights for policy 1, policy_version 1649733 (0.0010) [2023-12-27 03:23:19,457][105620] Updated weights for policy 1, policy_version 1649743 (0.0010) [2023-12-27 03:23:19,524][105620] Updated weights for policy 1, policy_version 1649753 (0.0011) [2023-12-27 03:23:19,637][105692] Updated weights for policy 0, policy_version 1646501 (0.0011) [2023-12-27 03:23:19,696][105692] Updated weights for policy 0, policy_version 1646511 (0.0011) [2023-12-27 03:23:19,756][105692] Updated weights for policy 0, policy_version 1646521 (0.0011) [2023-12-27 03:23:20,266][105620] Updated weights for policy 1, policy_version 1649763 (0.0010) [2023-12-27 03:23:20,322][105620] Updated weights for policy 1, policy_version 1649773 (0.0009) [2023-12-27 03:23:20,378][105620] Updated weights for policy 1, policy_version 1649783 (0.0010) [2023-12-27 03:23:20,473][105692] Updated weights for policy 0, policy_version 1646531 (0.0011) [2023-12-27 03:23:20,528][105692] Updated weights for policy 0, policy_version 1646541 (0.0010) [2023-12-27 03:23:20,589][105692] Updated weights for policy 0, policy_version 1646551 (0.0009) [2023-12-27 03:23:21,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 843988992. Throughput: 0: 9831.4, 1: 9562.4. Samples: 843981152. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:23:21,063][104569] Avg episode reward: [(0, '8898.263'), (1, '9172.202')] [2023-12-27 03:23:21,148][105620] Updated weights for policy 1, policy_version 1649793 (0.0009) [2023-12-27 03:23:21,208][105620] Updated weights for policy 1, policy_version 1649803 (0.0008) [2023-12-27 03:23:21,279][105620] Updated weights for policy 1, policy_version 1649813 (0.0007) [2023-12-27 03:23:21,332][105692] Updated weights for policy 0, policy_version 1646561 (0.0010) [2023-12-27 03:23:21,349][105620] Updated weights for policy 1, policy_version 1649823 (0.0008) [2023-12-27 03:23:21,405][105692] Updated weights for policy 0, policy_version 1646571 (0.0009) [2023-12-27 03:23:21,463][105692] Updated weights for policy 0, policy_version 1646581 (0.0008) [2023-12-27 03:23:21,519][105692] Updated weights for policy 0, policy_version 1646591 (0.0009) [2023-12-27 03:23:22,115][105620] Updated weights for policy 1, policy_version 1649833 (0.0010) [2023-12-27 03:23:22,154][105586] KL-divergence is very high: 155.9626 [2023-12-27 03:23:22,172][105620] Updated weights for policy 1, policy_version 1649843 (0.0008) [2023-12-27 03:23:22,201][105586] KL-divergence is very high: 173.2483 [2023-12-27 03:23:22,210][105692] Updated weights for policy 0, policy_version 1646601 (0.0010) [2023-12-27 03:23:22,233][105620] Updated weights for policy 1, policy_version 1649853 (0.0007) [2023-12-27 03:23:22,266][105692] Updated weights for policy 0, policy_version 1646611 (0.0008) [2023-12-27 03:23:22,341][105692] Updated weights for policy 0, policy_version 1646621 (0.0010) [2023-12-27 03:23:22,964][105620] Updated weights for policy 1, policy_version 1649863 (0.0009) [2023-12-27 03:23:23,026][105620] Updated weights for policy 1, policy_version 1649873 (0.0009) [2023-12-27 03:23:23,076][105620] Updated weights for policy 1, policy_version 1649883 (0.0009) [2023-12-27 03:23:23,107][105692] Updated weights for policy 0, policy_version 1646631 (0.0007) [2023-12-27 03:23:23,171][105692] Updated weights for policy 0, policy_version 1646641 (0.0009) [2023-12-27 03:23:23,230][105692] Updated weights for policy 0, policy_version 1646651 (0.0009) [2023-12-27 03:23:23,855][105620] Updated weights for policy 1, policy_version 1649893 (0.0008) [2023-12-27 03:23:23,920][105620] Updated weights for policy 1, policy_version 1649903 (0.0008) [2023-12-27 03:23:23,935][105692] Updated weights for policy 0, policy_version 1646661 (0.0010) [2023-12-27 03:23:23,970][105620] Updated weights for policy 1, policy_version 1649913 (0.0006) [2023-12-27 03:23:23,987][105692] Updated weights for policy 0, policy_version 1646671 (0.0010) [2023-12-27 03:23:24,050][105692] Updated weights for policy 0, policy_version 1646681 (0.0008) [2023-12-27 03:23:24,702][105692] Updated weights for policy 0, policy_version 1646691 (0.0007) [2023-12-27 03:23:24,721][105620] Updated weights for policy 1, policy_version 1649923 (0.0006) [2023-12-27 03:23:24,764][105692] Updated weights for policy 0, policy_version 1646701 (0.0010) [2023-12-27 03:23:24,783][105620] Updated weights for policy 1, policy_version 1649933 (0.0005) [2023-12-27 03:23:24,816][105692] Updated weights for policy 0, policy_version 1646711 (0.0010) [2023-12-27 03:23:24,840][105620] Updated weights for policy 1, policy_version 1649943 (0.0005) [2023-12-27 03:23:25,372][105620] Updated weights for policy 1, policy_version 1649953 (0.0005) [2023-12-27 03:23:25,431][105620] Updated weights for policy 1, policy_version 1649963 (0.0005) [2023-12-27 03:23:25,473][105692] Updated weights for policy 0, policy_version 1646721 (0.0011) [2023-12-27 03:23:25,477][105620] Updated weights for policy 1, policy_version 1649973 (0.0005) [2023-12-27 03:23:25,525][105692] Updated weights for policy 0, policy_version 1646731 (0.0010) [2023-12-27 03:23:25,530][105620] Updated weights for policy 1, policy_version 1649983 (0.0007) [2023-12-27 03:23:25,571][105692] Updated weights for policy 0, policy_version 1646741 (0.0010) [2023-12-27 03:23:25,629][105692] Updated weights for policy 0, policy_version 1646751 (0.0010) [2023-12-27 03:23:26,062][104569] Fps is (10 sec: 18842.3, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 844087296. Throughput: 0: 9834.4, 1: 9596.1. Samples: 844097412. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:23:26,062][104569] Avg episode reward: [(0, '8625.815'), (1, '9079.758')] [2023-12-27 03:23:26,230][105620] Updated weights for policy 1, policy_version 1649993 (0.0008) [2023-12-27 03:23:26,286][105620] Updated weights for policy 1, policy_version 1650003 (0.0008) [2023-12-27 03:23:26,351][105620] Updated weights for policy 1, policy_version 1650013 (0.0010) [2023-12-27 03:23:26,377][105692] Updated weights for policy 0, policy_version 1646761 (0.0006) [2023-12-27 03:23:26,433][105692] Updated weights for policy 0, policy_version 1646771 (0.0005) [2023-12-27 03:23:26,483][105692] Updated weights for policy 0, policy_version 1646781 (0.0005) [2023-12-27 03:23:27,067][105692] Updated weights for policy 0, policy_version 1646791 (0.0009) [2023-12-27 03:23:27,126][105692] Updated weights for policy 0, policy_version 1646801 (0.0006) [2023-12-27 03:23:27,174][105692] Updated weights for policy 0, policy_version 1646811 (0.0005) [2023-12-27 03:23:27,203][105620] Updated weights for policy 1, policy_version 1650023 (0.0008) [2023-12-27 03:23:27,274][105620] Updated weights for policy 1, policy_version 1650033 (0.0010) [2023-12-27 03:23:27,339][105620] Updated weights for policy 1, policy_version 1650043 (0.0009) [2023-12-27 03:23:27,748][105692] Updated weights for policy 0, policy_version 1646821 (0.0006) [2023-12-27 03:23:27,794][105692] Updated weights for policy 0, policy_version 1646831 (0.0005) [2023-12-27 03:23:27,838][105692] Updated weights for policy 0, policy_version 1646841 (0.0005) [2023-12-27 03:23:28,171][105620] Updated weights for policy 1, policy_version 1650053 (0.0009) [2023-12-27 03:23:28,225][105620] Updated weights for policy 1, policy_version 1650063 (0.0010) [2023-12-27 03:23:28,286][105620] Updated weights for policy 1, policy_version 1650073 (0.0010) [2023-12-27 03:23:28,419][105692] Updated weights for policy 0, policy_version 1646851 (0.0005) [2023-12-27 03:23:28,467][105692] Updated weights for policy 0, policy_version 1646861 (0.0010) [2023-12-27 03:23:28,515][105692] Updated weights for policy 0, policy_version 1646871 (0.0010) [2023-12-27 03:23:29,152][105692] Updated weights for policy 0, policy_version 1646881 (0.0010) [2023-12-27 03:23:29,186][105620] Updated weights for policy 1, policy_version 1650083 (0.0009) [2023-12-27 03:23:29,208][105692] Updated weights for policy 0, policy_version 1646891 (0.0006) [2023-12-27 03:23:29,251][105620] Updated weights for policy 1, policy_version 1650093 (0.0009) [2023-12-27 03:23:29,269][105692] Updated weights for policy 0, policy_version 1646901 (0.0007) [2023-12-27 03:23:29,309][105620] Updated weights for policy 1, policy_version 1650103 (0.0006) [2023-12-27 03:23:29,329][105692] Updated weights for policy 0, policy_version 1646911 (0.0010) [2023-12-27 03:23:30,013][105620] Updated weights for policy 1, policy_version 1650113 (0.0008) [2023-12-27 03:23:30,058][105692] Updated weights for policy 0, policy_version 1646921 (0.0006) [2023-12-27 03:23:30,071][105620] Updated weights for policy 1, policy_version 1650124 (0.0008) [2023-12-27 03:23:30,112][105692] Updated weights for policy 0, policy_version 1646931 (0.0011) [2023-12-27 03:23:30,119][105620] Updated weights for policy 1, policy_version 1650134 (0.0006) [2023-12-27 03:23:30,170][105620] Updated weights for policy 1, policy_version 1650144 (0.0006) [2023-12-27 03:23:30,171][105692] Updated weights for policy 0, policy_version 1646941 (0.0010) [2023-12-27 03:23:30,866][105692] Updated weights for policy 0, policy_version 1646951 (0.0008) [2023-12-27 03:23:30,873][105620] Updated weights for policy 1, policy_version 1650154 (0.0008) [2023-12-27 03:23:30,915][105692] Updated weights for policy 0, policy_version 1646961 (0.0007) [2023-12-27 03:23:30,930][105620] Updated weights for policy 1, policy_version 1650164 (0.0006) [2023-12-27 03:23:30,961][105692] Updated weights for policy 0, policy_version 1646971 (0.0005) [2023-12-27 03:23:30,978][105620] Updated weights for policy 1, policy_version 1650174 (0.0007) [2023-12-27 03:23:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 844193792. Throughput: 0: 9925.5, 1: 9534.0. Samples: 844155904. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:23:31,063][104569] Avg episode reward: [(0, '8445.527'), (1, '9173.863')] [2023-12-27 03:23:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001646976_421691392.pth... [2023-12-27 03:23:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001650176_422502400.pth... [2023-12-27 03:23:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001649056_422215680.pth [2023-12-27 03:23:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001645792_421388288.pth [2023-12-27 03:23:31,686][105692] Updated weights for policy 0, policy_version 1646981 (0.0006) [2023-12-27 03:23:31,758][105692] Updated weights for policy 0, policy_version 1646991 (0.0009) [2023-12-27 03:23:31,792][105620] Updated weights for policy 1, policy_version 1650184 (0.0007) [2023-12-27 03:23:31,817][105692] Updated weights for policy 0, policy_version 1647001 (0.0011) [2023-12-27 03:23:31,851][105620] Updated weights for policy 1, policy_version 1650194 (0.0006) [2023-12-27 03:23:31,915][105620] Updated weights for policy 1, policy_version 1650204 (0.0008) [2023-12-27 03:23:32,573][105692] Updated weights for policy 0, policy_version 1647011 (0.0011) [2023-12-27 03:23:32,624][105692] Updated weights for policy 0, policy_version 1647021 (0.0010) [2023-12-27 03:23:32,646][105620] Updated weights for policy 1, policy_version 1650214 (0.0007) [2023-12-27 03:23:32,669][105692] Updated weights for policy 0, policy_version 1647031 (0.0010) [2023-12-27 03:23:32,705][105620] Updated weights for policy 1, policy_version 1650224 (0.0008) [2023-12-27 03:23:32,771][105620] Updated weights for policy 1, policy_version 1650234 (0.0008) [2023-12-27 03:23:33,369][105692] Updated weights for policy 0, policy_version 1647041 (0.0010) [2023-12-27 03:23:33,420][105692] Updated weights for policy 0, policy_version 1647051 (0.0010) [2023-12-27 03:23:33,477][105692] Updated weights for policy 0, policy_version 1647061 (0.0010) [2023-12-27 03:23:33,531][105692] Updated weights for policy 0, policy_version 1647071 (0.0010) [2023-12-27 03:23:33,534][105620] Updated weights for policy 1, policy_version 1650244 (0.0007) [2023-12-27 03:23:33,594][105620] Updated weights for policy 1, policy_version 1650254 (0.0007) [2023-12-27 03:23:33,647][105620] Updated weights for policy 1, policy_version 1650264 (0.0008) [2023-12-27 03:23:34,110][105692] Updated weights for policy 0, policy_version 1647081 (0.0006) [2023-12-27 03:23:34,171][105692] Updated weights for policy 0, policy_version 1647091 (0.0007) [2023-12-27 03:23:34,228][105692] Updated weights for policy 0, policy_version 1647101 (0.0006) [2023-12-27 03:23:34,526][105620] Updated weights for policy 1, policy_version 1650275 (0.0010) [2023-12-27 03:23:34,589][105620] Updated weights for policy 1, policy_version 1650285 (0.0007) [2023-12-27 03:23:34,651][105620] Updated weights for policy 1, policy_version 1650295 (0.0008) [2023-12-27 03:23:34,872][105692] Updated weights for policy 0, policy_version 1647111 (0.0009) [2023-12-27 03:23:34,922][105692] Updated weights for policy 0, policy_version 1647121 (0.0010) [2023-12-27 03:23:34,984][105692] Updated weights for policy 0, policy_version 1647131 (0.0010) [2023-12-27 03:23:35,421][105620] Updated weights for policy 1, policy_version 1650305 (0.0008) [2023-12-27 03:23:35,478][105620] Updated weights for policy 1, policy_version 1650315 (0.0010) [2023-12-27 03:23:35,529][105620] Updated weights for policy 1, policy_version 1650325 (0.0008) [2023-12-27 03:23:35,585][105620] Updated weights for policy 1, policy_version 1650335 (0.0008) [2023-12-27 03:23:35,670][105692] Updated weights for policy 0, policy_version 1647141 (0.0008) [2023-12-27 03:23:35,731][105692] Updated weights for policy 0, policy_version 1647151 (0.0005) [2023-12-27 03:23:35,788][105692] Updated weights for policy 0, policy_version 1647161 (0.0005) [2023-12-27 03:23:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 844283904. Throughput: 0: 10024.7, 1: 9342.8. Samples: 844273108. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:23:36,063][104569] Avg episode reward: [(0, '8711.635'), (1, '9173.720')] [2023-12-27 03:23:36,414][105692] Updated weights for policy 0, policy_version 1647171 (0.0006) [2023-12-27 03:23:36,466][105692] Updated weights for policy 0, policy_version 1647181 (0.0008) [2023-12-27 03:23:36,482][105620] Updated weights for policy 1, policy_version 1650345 (0.0006) [2023-12-27 03:23:36,518][105692] Updated weights for policy 0, policy_version 1647191 (0.0009) [2023-12-27 03:23:36,553][105620] Updated weights for policy 1, policy_version 1650355 (0.0005) [2023-12-27 03:23:36,609][105620] Updated weights for policy 1, policy_version 1650365 (0.0007) [2023-12-27 03:23:37,206][105692] Updated weights for policy 0, policy_version 1647201 (0.0008) [2023-12-27 03:23:37,256][105692] Updated weights for policy 0, policy_version 1647211 (0.0009) [2023-12-27 03:23:37,277][105620] Updated weights for policy 1, policy_version 1650375 (0.0010) [2023-12-27 03:23:37,311][105692] Updated weights for policy 0, policy_version 1647221 (0.0007) [2023-12-27 03:23:37,326][105620] Updated weights for policy 1, policy_version 1650385 (0.0006) [2023-12-27 03:23:37,376][105692] Updated weights for policy 0, policy_version 1647231 (0.0010) [2023-12-27 03:23:37,387][105620] Updated weights for policy 1, policy_version 1650395 (0.0007) [2023-12-27 03:23:37,991][105620] Updated weights for policy 1, policy_version 1650405 (0.0009) [2023-12-27 03:23:38,048][105620] Updated weights for policy 1, policy_version 1650415 (0.0008) [2023-12-27 03:23:38,108][105620] Updated weights for policy 1, policy_version 1650425 (0.0008) [2023-12-27 03:23:38,219][105692] Updated weights for policy 0, policy_version 1647241 (0.0009) [2023-12-27 03:23:38,272][105692] Updated weights for policy 0, policy_version 1647252 (0.0009) [2023-12-27 03:23:38,329][105692] Updated weights for policy 0, policy_version 1647263 (0.0009) [2023-12-27 03:23:38,689][105620] Updated weights for policy 1, policy_version 1650435 (0.0008) [2023-12-27 03:23:38,741][105620] Updated weights for policy 1, policy_version 1650445 (0.0005) [2023-12-27 03:23:38,806][105620] Updated weights for policy 1, policy_version 1650455 (0.0006) [2023-12-27 03:23:39,227][105692] Updated weights for policy 0, policy_version 1647273 (0.0008) [2023-12-27 03:23:39,287][105692] Updated weights for policy 0, policy_version 1647283 (0.0008) [2023-12-27 03:23:39,354][105692] Updated weights for policy 0, policy_version 1647293 (0.0009) [2023-12-27 03:23:39,462][105620] Updated weights for policy 1, policy_version 1650465 (0.0006) [2023-12-27 03:23:39,523][105620] Updated weights for policy 1, policy_version 1650475 (0.0011) [2023-12-27 03:23:39,587][105620] Updated weights for policy 1, policy_version 1650485 (0.0011) [2023-12-27 03:23:39,646][105620] Updated weights for policy 1, policy_version 1650495 (0.0011) [2023-12-27 03:23:40,165][105692] Updated weights for policy 0, policy_version 1647303 (0.0009) [2023-12-27 03:23:40,230][105692] Updated weights for policy 0, policy_version 1647313 (0.0008) [2023-12-27 03:23:40,293][105692] Updated weights for policy 0, policy_version 1647323 (0.0008) [2023-12-27 03:23:40,417][105620] Updated weights for policy 1, policy_version 1650505 (0.0011) [2023-12-27 03:23:40,478][105620] Updated weights for policy 1, policy_version 1650515 (0.0011) [2023-12-27 03:23:40,548][105620] Updated weights for policy 1, policy_version 1650525 (0.0010) [2023-12-27 03:23:41,062][104569] Fps is (10 sec: 18022.6, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 844374016. Throughput: 0: 9997.0, 1: 9392.5. Samples: 844387360. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:23:41,062][104569] Avg episode reward: [(0, '8803.657'), (1, '8990.451')] [2023-12-27 03:23:41,072][105692] Updated weights for policy 0, policy_version 1647333 (0.0008) [2023-12-27 03:23:41,132][105692] Updated weights for policy 0, policy_version 1647343 (0.0008) [2023-12-27 03:23:41,191][105692] Updated weights for policy 0, policy_version 1647353 (0.0008) [2023-12-27 03:23:41,306][105620] Updated weights for policy 1, policy_version 1650535 (0.0011) [2023-12-27 03:23:41,376][105620] Updated weights for policy 1, policy_version 1650545 (0.0010) [2023-12-27 03:23:41,440][105620] Updated weights for policy 1, policy_version 1650555 (0.0011) [2023-12-27 03:23:41,982][105692] Updated weights for policy 0, policy_version 1647363 (0.0009) [2023-12-27 03:23:42,029][105692] Updated weights for policy 0, policy_version 1647373 (0.0008) [2023-12-27 03:23:42,082][105692] Updated weights for policy 0, policy_version 1647383 (0.0008) [2023-12-27 03:23:42,230][105620] Updated weights for policy 1, policy_version 1650565 (0.0011) [2023-12-27 03:23:42,285][105620] Updated weights for policy 1, policy_version 1650575 (0.0009) [2023-12-27 03:23:42,335][105620] Updated weights for policy 1, policy_version 1650585 (0.0011) [2023-12-27 03:23:42,805][105692] Updated weights for policy 0, policy_version 1647393 (0.0009) [2023-12-27 03:23:42,866][105692] Updated weights for policy 0, policy_version 1647403 (0.0010) [2023-12-27 03:23:42,923][105692] Updated weights for policy 0, policy_version 1647413 (0.0008) [2023-12-27 03:23:42,978][105692] Updated weights for policy 0, policy_version 1647423 (0.0010) [2023-12-27 03:23:43,053][105620] Updated weights for policy 1, policy_version 1650595 (0.0010) [2023-12-27 03:23:43,105][105620] Updated weights for policy 1, policy_version 1650605 (0.0007) [2023-12-27 03:23:43,162][105620] Updated weights for policy 1, policy_version 1650615 (0.0009) [2023-12-27 03:23:43,685][105692] Updated weights for policy 0, policy_version 1647433 (0.0009) [2023-12-27 03:23:43,740][105692] Updated weights for policy 0, policy_version 1647443 (0.0009) [2023-12-27 03:23:43,799][105692] Updated weights for policy 0, policy_version 1647453 (0.0008) [2023-12-27 03:23:43,911][105620] Updated weights for policy 1, policy_version 1650625 (0.0009) [2023-12-27 03:23:43,956][105620] Updated weights for policy 1, policy_version 1650635 (0.0005) [2023-12-27 03:23:44,005][105620] Updated weights for policy 1, policy_version 1650645 (0.0008) [2023-12-27 03:23:44,055][105620] Updated weights for policy 1, policy_version 1650655 (0.0008) [2023-12-27 03:23:44,623][105692] Updated weights for policy 0, policy_version 1647463 (0.0006) [2023-12-27 03:23:44,677][105692] Updated weights for policy 0, policy_version 1647473 (0.0006) [2023-12-27 03:23:44,695][105620] Updated weights for policy 1, policy_version 1650665 (0.0009) [2023-12-27 03:23:44,738][105692] Updated weights for policy 0, policy_version 1647483 (0.0008) [2023-12-27 03:23:44,744][105620] Updated weights for policy 1, policy_version 1650675 (0.0006) [2023-12-27 03:23:44,801][105620] Updated weights for policy 1, policy_version 1650685 (0.0008) [2023-12-27 03:23:45,449][105620] Updated weights for policy 1, policy_version 1650695 (0.0007) [2023-12-27 03:23:45,498][105620] Updated weights for policy 1, policy_version 1650705 (0.0005) [2023-12-27 03:23:45,545][105620] Updated weights for policy 1, policy_version 1650715 (0.0005) [2023-12-27 03:23:45,587][105692] Updated weights for policy 0, policy_version 1647493 (0.0006) [2023-12-27 03:23:45,652][105692] Updated weights for policy 0, policy_version 1647503 (0.0008) [2023-12-27 03:23:45,716][105692] Updated weights for policy 0, policy_version 1647513 (0.0008) [2023-12-27 03:23:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 844472320. Throughput: 0: 9850.1, 1: 9452.7. Samples: 844443644. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:23:46,062][104569] Avg episode reward: [(0, '8712.624'), (1, '8902.939')] [2023-12-27 03:23:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001647520_421830656.pth... [2023-12-27 03:23:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001650720_422641664.pth... [2023-12-27 03:23:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001646368_421535744.pth [2023-12-27 03:23:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001649632_422363136.pth [2023-12-27 03:23:46,287][105692] Updated weights for policy 0, policy_version 1647523 (0.0009) [2023-12-27 03:23:46,318][105620] Updated weights for policy 1, policy_version 1650725 (0.0007) [2023-12-27 03:23:46,334][105692] Updated weights for policy 0, policy_version 1647533 (0.0008) [2023-12-27 03:23:46,380][105692] Updated weights for policy 0, policy_version 1647543 (0.0005) [2023-12-27 03:23:46,386][105620] Updated weights for policy 1, policy_version 1650735 (0.0008) [2023-12-27 03:23:46,452][105620] Updated weights for policy 1, policy_version 1650745 (0.0008) [2023-12-27 03:23:46,987][105620] Updated weights for policy 1, policy_version 1650755 (0.0008) [2023-12-27 03:23:47,008][105692] Updated weights for policy 0, policy_version 1647553 (0.0005) [2023-12-27 03:23:47,050][105620] Updated weights for policy 1, policy_version 1650765 (0.0009) [2023-12-27 03:23:47,068][105692] Updated weights for policy 0, policy_version 1647563 (0.0005) [2023-12-27 03:23:47,109][105620] Updated weights for policy 1, policy_version 1650775 (0.0011) [2023-12-27 03:23:47,129][105692] Updated weights for policy 0, policy_version 1647573 (0.0006) [2023-12-27 03:23:47,180][105692] Updated weights for policy 0, policy_version 1647583 (0.0005) [2023-12-27 03:23:47,787][105692] Updated weights for policy 0, policy_version 1647593 (0.0005) [2023-12-27 03:23:47,816][105620] Updated weights for policy 1, policy_version 1650785 (0.0010) [2023-12-27 03:23:47,847][105692] Updated weights for policy 0, policy_version 1647603 (0.0010) [2023-12-27 03:23:47,877][105620] Updated weights for policy 1, policy_version 1650795 (0.0006) [2023-12-27 03:23:47,913][105692] Updated weights for policy 0, policy_version 1647613 (0.0011) [2023-12-27 03:23:47,941][105620] Updated weights for policy 1, policy_version 1650805 (0.0005) [2023-12-27 03:23:48,009][105620] Updated weights for policy 1, policy_version 1650815 (0.0007) [2023-12-27 03:23:48,561][105692] Updated weights for policy 0, policy_version 1647623 (0.0011) [2023-12-27 03:23:48,617][105692] Updated weights for policy 0, policy_version 1647633 (0.0011) [2023-12-27 03:23:48,626][105620] Updated weights for policy 1, policy_version 1650825 (0.0006) [2023-12-27 03:23:48,674][105692] Updated weights for policy 0, policy_version 1647643 (0.0009) [2023-12-27 03:23:48,689][105620] Updated weights for policy 1, policy_version 1650835 (0.0006) [2023-12-27 03:23:48,755][105620] Updated weights for policy 1, policy_version 1650845 (0.0007) [2023-12-27 03:23:49,405][105620] Updated weights for policy 1, policy_version 1650855 (0.0009) [2023-12-27 03:23:49,431][105692] Updated weights for policy 0, policy_version 1647653 (0.0009) [2023-12-27 03:23:49,464][105620] Updated weights for policy 1, policy_version 1650865 (0.0010) [2023-12-27 03:23:49,486][105692] Updated weights for policy 0, policy_version 1647663 (0.0010) [2023-12-27 03:23:49,523][105620] Updated weights for policy 1, policy_version 1650875 (0.0010) [2023-12-27 03:23:49,542][105692] Updated weights for policy 0, policy_version 1647673 (0.0010) [2023-12-27 03:23:50,198][105620] Updated weights for policy 1, policy_version 1650885 (0.0008) [2023-12-27 03:23:50,260][105620] Updated weights for policy 1, policy_version 1650895 (0.0007) [2023-12-27 03:23:50,303][105692] Updated weights for policy 0, policy_version 1647683 (0.0010) [2023-12-27 03:23:50,322][105620] Updated weights for policy 1, policy_version 1650905 (0.0006) [2023-12-27 03:23:50,366][105692] Updated weights for policy 0, policy_version 1647693 (0.0011) [2023-12-27 03:23:50,421][105692] Updated weights for policy 0, policy_version 1647703 (0.0010) [2023-12-27 03:23:51,051][105620] Updated weights for policy 1, policy_version 1650915 (0.0008) [2023-12-27 03:23:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 844570624. Throughput: 0: 9846.6, 1: 9554.0. Samples: 844565316. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:23:51,062][104569] Avg episode reward: [(0, '8804.490'), (1, '9356.521')] [2023-12-27 03:23:51,120][105620] Updated weights for policy 1, policy_version 1650925 (0.0007) [2023-12-27 03:23:51,179][105692] Updated weights for policy 0, policy_version 1647713 (0.0010) [2023-12-27 03:23:51,186][105620] Updated weights for policy 1, policy_version 1650935 (0.0008) [2023-12-27 03:23:51,238][105692] Updated weights for policy 0, policy_version 1647723 (0.0010) [2023-12-27 03:23:51,299][105692] Updated weights for policy 0, policy_version 1647733 (0.0010) [2023-12-27 03:23:51,363][105692] Updated weights for policy 0, policy_version 1647743 (0.0010) [2023-12-27 03:23:51,928][105620] Updated weights for policy 1, policy_version 1650945 (0.0006) [2023-12-27 03:23:51,990][105620] Updated weights for policy 1, policy_version 1650955 (0.0006) [2023-12-27 03:23:52,045][105620] Updated weights for policy 1, policy_version 1650965 (0.0008) [2023-12-27 03:23:52,099][105620] Updated weights for policy 1, policy_version 1650975 (0.0008) [2023-12-27 03:23:52,119][105692] Updated weights for policy 0, policy_version 1647753 (0.0008) [2023-12-27 03:23:52,171][105692] Updated weights for policy 0, policy_version 1647763 (0.0010) [2023-12-27 03:23:52,215][105692] Updated weights for policy 0, policy_version 1647773 (0.0007) [2023-12-27 03:23:52,730][105620] Updated weights for policy 1, policy_version 1650985 (0.0008) [2023-12-27 03:23:52,787][105620] Updated weights for policy 1, policy_version 1650995 (0.0008) [2023-12-27 03:23:52,845][105620] Updated weights for policy 1, policy_version 1651005 (0.0007) [2023-12-27 03:23:52,955][105692] Updated weights for policy 0, policy_version 1647783 (0.0009) [2023-12-27 03:23:53,003][105692] Updated weights for policy 0, policy_version 1647793 (0.0010) [2023-12-27 03:23:53,055][105692] Updated weights for policy 0, policy_version 1647803 (0.0010) [2023-12-27 03:23:53,431][105620] Updated weights for policy 1, policy_version 1651015 (0.0006) [2023-12-27 03:23:53,497][105620] Updated weights for policy 1, policy_version 1651025 (0.0008) [2023-12-27 03:23:53,551][105620] Updated weights for policy 1, policy_version 1651035 (0.0010) [2023-12-27 03:23:53,750][105692] Updated weights for policy 0, policy_version 1647813 (0.0010) [2023-12-27 03:23:53,811][105692] Updated weights for policy 0, policy_version 1647823 (0.0010) [2023-12-27 03:23:53,858][105692] Updated weights for policy 0, policy_version 1647833 (0.0010) [2023-12-27 03:23:54,123][105620] Updated weights for policy 1, policy_version 1651045 (0.0011) [2023-12-27 03:23:54,187][105620] Updated weights for policy 1, policy_version 1651055 (0.0011) [2023-12-27 03:23:54,250][105620] Updated weights for policy 1, policy_version 1651065 (0.0011) [2023-12-27 03:23:54,604][105692] Updated weights for policy 0, policy_version 1647843 (0.0010) [2023-12-27 03:23:54,656][105692] Updated weights for policy 0, policy_version 1647853 (0.0010) [2023-12-27 03:23:54,708][105692] Updated weights for policy 0, policy_version 1647863 (0.0010) [2023-12-27 03:23:54,980][105620] Updated weights for policy 1, policy_version 1651075 (0.0010) [2023-12-27 03:23:55,037][105620] Updated weights for policy 1, policy_version 1651085 (0.0008) [2023-12-27 03:23:55,093][105620] Updated weights for policy 1, policy_version 1651095 (0.0008) [2023-12-27 03:23:55,412][105692] Updated weights for policy 0, policy_version 1647873 (0.0010) [2023-12-27 03:23:55,462][105692] Updated weights for policy 0, policy_version 1647883 (0.0010) [2023-12-27 03:23:55,511][105692] Updated weights for policy 0, policy_version 1647894 (0.0009) [2023-12-27 03:23:55,567][105692] Updated weights for policy 0, policy_version 1647904 (0.0009) [2023-12-27 03:23:55,876][105620] Updated weights for policy 1, policy_version 1651105 (0.0009) [2023-12-27 03:23:55,925][105620] Updated weights for policy 1, policy_version 1651115 (0.0010) [2023-12-27 03:23:55,976][105620] Updated weights for policy 1, policy_version 1651125 (0.0010) [2023-12-27 03:23:56,033][105620] Updated weights for policy 1, policy_version 1651135 (0.0010) [2023-12-27 03:23:56,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 844677120. Throughput: 0: 9843.7, 1: 9577.4. Samples: 844683604. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:23:56,062][104569] Avg episode reward: [(0, '8803.047'), (1, '9265.200')] [2023-12-27 03:23:56,288][105692] Updated weights for policy 0, policy_version 1647914 (0.0005) [2023-12-27 03:23:56,344][105692] Updated weights for policy 0, policy_version 1647924 (0.0006) [2023-12-27 03:23:56,395][105692] Updated weights for policy 0, policy_version 1647934 (0.0009) [2023-12-27 03:23:56,890][105620] Updated weights for policy 1, policy_version 1651145 (0.0010) [2023-12-27 03:23:56,927][105692] Updated weights for policy 0, policy_version 1647944 (0.0007) [2023-12-27 03:23:56,938][105620] Updated weights for policy 1, policy_version 1651155 (0.0010) [2023-12-27 03:23:56,972][105692] Updated weights for policy 0, policy_version 1647954 (0.0005) [2023-12-27 03:23:56,986][105620] Updated weights for policy 1, policy_version 1651165 (0.0010) [2023-12-27 03:23:57,017][105692] Updated weights for policy 0, policy_version 1647964 (0.0006) [2023-12-27 03:23:57,758][105692] Updated weights for policy 0, policy_version 1647974 (0.0006) [2023-12-27 03:23:57,761][105620] Updated weights for policy 1, policy_version 1651175 (0.0010) [2023-12-27 03:23:57,805][105692] Updated weights for policy 0, policy_version 1647984 (0.0009) [2023-12-27 03:23:57,805][105620] Updated weights for policy 1, policy_version 1651185 (0.0010) [2023-12-27 03:23:57,849][105620] Updated weights for policy 1, policy_version 1651195 (0.0010) [2023-12-27 03:23:57,859][105692] Updated weights for policy 0, policy_version 1647994 (0.0010) [2023-12-27 03:23:58,518][105620] Updated weights for policy 1, policy_version 1651205 (0.0009) [2023-12-27 03:23:58,595][105620] Updated weights for policy 1, policy_version 1651215 (0.0011) [2023-12-27 03:23:58,630][105692] Updated weights for policy 0, policy_version 1648004 (0.0008) [2023-12-27 03:23:58,658][105620] Updated weights for policy 1, policy_version 1651225 (0.0010) [2023-12-27 03:23:58,694][105692] Updated weights for policy 0, policy_version 1648014 (0.0010) [2023-12-27 03:23:58,756][105692] Updated weights for policy 0, policy_version 1648024 (0.0008) [2023-12-27 03:23:59,432][105620] Updated weights for policy 1, policy_version 1651235 (0.0009) [2023-12-27 03:23:59,487][105620] Updated weights for policy 1, policy_version 1651245 (0.0006) [2023-12-27 03:23:59,540][105620] Updated weights for policy 1, policy_version 1651255 (0.0005) [2023-12-27 03:23:59,584][105692] Updated weights for policy 0, policy_version 1648034 (0.0009) [2023-12-27 03:23:59,636][105692] Updated weights for policy 0, policy_version 1648044 (0.0009) [2023-12-27 03:23:59,684][105692] Updated weights for policy 0, policy_version 1648054 (0.0009) [2023-12-27 03:23:59,735][105692] Updated weights for policy 0, policy_version 1648064 (0.0009) [2023-12-27 03:24:00,281][105620] Updated weights for policy 1, policy_version 1651265 (0.0008) [2023-12-27 03:24:00,334][105620] Updated weights for policy 1, policy_version 1651275 (0.0007) [2023-12-27 03:24:00,392][105620] Updated weights for policy 1, policy_version 1651285 (0.0008) [2023-12-27 03:24:00,403][105692] Updated weights for policy 0, policy_version 1648074 (0.0006) [2023-12-27 03:24:00,449][105620] Updated weights for policy 1, policy_version 1651295 (0.0008) [2023-12-27 03:24:00,455][105692] Updated weights for policy 0, policy_version 1648084 (0.0007) [2023-12-27 03:24:00,513][105692] Updated weights for policy 0, policy_version 1648094 (0.0008) [2023-12-27 03:24:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 844767232. Throughput: 0: 9884.5, 1: 9576.8. Samples: 844742456. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:24:01,063][104569] Avg episode reward: [(0, '8803.422'), (1, '9177.271')] [2023-12-27 03:24:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001648096_421978112.pth... [2023-12-27 03:24:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001646976_421691392.pth [2023-12-27 03:24:01,086][105620] Updated weights for policy 1, policy_version 1651305 (0.0005) [2023-12-27 03:24:01,152][105620] Updated weights for policy 1, policy_version 1651315 (0.0008) [2023-12-27 03:24:01,215][105620] Updated weights for policy 1, policy_version 1651325 (0.0011) [2023-12-27 03:24:01,233][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001651328_422797312.pth... [2023-12-27 03:24:01,236][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001650176_422502400.pth [2023-12-27 03:24:01,304][105692] Updated weights for policy 0, policy_version 1648104 (0.0008) [2023-12-27 03:24:01,365][105692] Updated weights for policy 0, policy_version 1648114 (0.0008) [2023-12-27 03:24:01,419][105692] Updated weights for policy 0, policy_version 1648124 (0.0008) [2023-12-27 03:24:01,972][105620] Updated weights for policy 1, policy_version 1651335 (0.0011) [2023-12-27 03:24:02,038][105620] Updated weights for policy 1, policy_version 1651345 (0.0010) [2023-12-27 03:24:02,100][105620] Updated weights for policy 1, policy_version 1651355 (0.0010) [2023-12-27 03:24:02,207][105692] Updated weights for policy 0, policy_version 1648134 (0.0008) [2023-12-27 03:24:02,262][105692] Updated weights for policy 0, policy_version 1648144 (0.0008) [2023-12-27 03:24:02,318][105692] Updated weights for policy 0, policy_version 1648154 (0.0008) [2023-12-27 03:24:02,845][105620] Updated weights for policy 1, policy_version 1651365 (0.0010) [2023-12-27 03:24:02,908][105620] Updated weights for policy 1, policy_version 1651375 (0.0010) [2023-12-27 03:24:02,970][105620] Updated weights for policy 1, policy_version 1651385 (0.0011) [2023-12-27 03:24:03,102][105692] Updated weights for policy 0, policy_version 1648164 (0.0008) [2023-12-27 03:24:03,150][105692] Updated weights for policy 0, policy_version 1648174 (0.0008) [2023-12-27 03:24:03,198][105692] Updated weights for policy 0, policy_version 1648184 (0.0008) [2023-12-27 03:24:03,709][105620] Updated weights for policy 1, policy_version 1651395 (0.0010) [2023-12-27 03:24:03,767][105620] Updated weights for policy 1, policy_version 1651405 (0.0010) [2023-12-27 03:24:03,818][105620] Updated weights for policy 1, policy_version 1651415 (0.0010) [2023-12-27 03:24:03,982][105692] Updated weights for policy 0, policy_version 1648194 (0.0008) [2023-12-27 03:24:04,035][105692] Updated weights for policy 0, policy_version 1648204 (0.0008) [2023-12-27 03:24:04,087][105692] Updated weights for policy 0, policy_version 1648214 (0.0008) [2023-12-27 03:24:04,140][105692] Updated weights for policy 0, policy_version 1648224 (0.0008) [2023-12-27 03:24:04,575][105620] Updated weights for policy 1, policy_version 1651425 (0.0010) [2023-12-27 03:24:04,631][105620] Updated weights for policy 1, policy_version 1651435 (0.0011) [2023-12-27 03:24:04,694][105620] Updated weights for policy 1, policy_version 1651445 (0.0011) [2023-12-27 03:24:04,753][105620] Updated weights for policy 1, policy_version 1651455 (0.0011) [2023-12-27 03:24:04,916][105692] Updated weights for policy 0, policy_version 1648234 (0.0009) [2023-12-27 03:24:04,982][105692] Updated weights for policy 0, policy_version 1648244 (0.0009) [2023-12-27 03:24:05,039][105692] Updated weights for policy 0, policy_version 1648254 (0.0009) [2023-12-27 03:24:05,483][105620] Updated weights for policy 1, policy_version 1651465 (0.0009) [2023-12-27 03:24:05,529][105620] Updated weights for policy 1, policy_version 1651475 (0.0008) [2023-12-27 03:24:05,576][105620] Updated weights for policy 1, policy_version 1651485 (0.0009) [2023-12-27 03:24:05,699][105692] Updated weights for policy 0, policy_version 1648264 (0.0006) [2023-12-27 03:24:05,757][105692] Updated weights for policy 0, policy_version 1648274 (0.0009) [2023-12-27 03:24:05,810][105692] Updated weights for policy 0, policy_version 1648284 (0.0008) [2023-12-27 03:24:06,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 844865536. Throughput: 0: 9780.5, 1: 9613.5. Samples: 844853880. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:24:06,063][104569] Avg episode reward: [(0, '8988.178'), (1, '8992.646')] [2023-12-27 03:24:06,314][105620] Updated weights for policy 1, policy_version 1651495 (0.0007) [2023-12-27 03:24:06,382][105620] Updated weights for policy 1, policy_version 1651505 (0.0005) [2023-12-27 03:24:06,449][105620] Updated weights for policy 1, policy_version 1651515 (0.0005) [2023-12-27 03:24:06,535][105692] Updated weights for policy 0, policy_version 1648294 (0.0008) [2023-12-27 03:24:06,595][105692] Updated weights for policy 0, policy_version 1648304 (0.0007) [2023-12-27 03:24:06,665][105692] Updated weights for policy 0, policy_version 1648314 (0.0009) [2023-12-27 03:24:07,107][105620] Updated weights for policy 1, policy_version 1651525 (0.0009) [2023-12-27 03:24:07,155][105620] Updated weights for policy 1, policy_version 1651535 (0.0010) [2023-12-27 03:24:07,204][105620] Updated weights for policy 1, policy_version 1651545 (0.0010) [2023-12-27 03:24:07,405][105692] Updated weights for policy 0, policy_version 1648324 (0.0009) [2023-12-27 03:24:07,459][105692] Updated weights for policy 0, policy_version 1648334 (0.0010) [2023-12-27 03:24:07,514][105692] Updated weights for policy 0, policy_version 1648344 (0.0010) [2023-12-27 03:24:07,882][105620] Updated weights for policy 1, policy_version 1651555 (0.0009) [2023-12-27 03:24:07,934][105620] Updated weights for policy 1, policy_version 1651565 (0.0005) [2023-12-27 03:24:07,994][105620] Updated weights for policy 1, policy_version 1651575 (0.0005) [2023-12-27 03:24:08,248][105692] Updated weights for policy 0, policy_version 1648354 (0.0009) [2023-12-27 03:24:08,313][105692] Updated weights for policy 0, policy_version 1648364 (0.0008) [2023-12-27 03:24:08,377][105692] Updated weights for policy 0, policy_version 1648374 (0.0008) [2023-12-27 03:24:08,429][105692] Updated weights for policy 0, policy_version 1648384 (0.0009) [2023-12-27 03:24:08,642][105620] Updated weights for policy 1, policy_version 1651585 (0.0009) [2023-12-27 03:24:08,702][105620] Updated weights for policy 1, policy_version 1651595 (0.0008) [2023-12-27 03:24:08,754][105620] Updated weights for policy 1, policy_version 1651605 (0.0010) [2023-12-27 03:24:08,802][105620] Updated weights for policy 1, policy_version 1651615 (0.0010) [2023-12-27 03:24:09,174][105692] Updated weights for policy 0, policy_version 1648394 (0.0011) [2023-12-27 03:24:09,242][105692] Updated weights for policy 0, policy_version 1648404 (0.0011) [2023-12-27 03:24:09,296][105692] Updated weights for policy 0, policy_version 1648414 (0.0011) [2023-12-27 03:24:09,548][105620] Updated weights for policy 1, policy_version 1651625 (0.0006) [2023-12-27 03:24:09,609][105620] Updated weights for policy 1, policy_version 1651635 (0.0006) [2023-12-27 03:24:09,667][105620] Updated weights for policy 1, policy_version 1651645 (0.0006) [2023-12-27 03:24:10,026][105692] Updated weights for policy 0, policy_version 1648424 (0.0011) [2023-12-27 03:24:10,093][105692] Updated weights for policy 0, policy_version 1648434 (0.0011) [2023-12-27 03:24:10,157][105692] Updated weights for policy 0, policy_version 1648444 (0.0011) [2023-12-27 03:24:10,319][105620] Updated weights for policy 1, policy_version 1651655 (0.0009) [2023-12-27 03:24:10,386][105620] Updated weights for policy 1, policy_version 1651665 (0.0011) [2023-12-27 03:24:10,454][105620] Updated weights for policy 1, policy_version 1651675 (0.0011) [2023-12-27 03:24:10,863][105692] Updated weights for policy 0, policy_version 1648454 (0.0010) [2023-12-27 03:24:10,913][105692] Updated weights for policy 0, policy_version 1648464 (0.0008) [2023-12-27 03:24:10,978][105692] Updated weights for policy 0, policy_version 1648474 (0.0005) [2023-12-27 03:24:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 844963840. Throughput: 0: 9759.5, 1: 9647.6. Samples: 844970732. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:24:11,063][104569] Avg episode reward: [(0, '8894.981'), (1, '8987.635')] [2023-12-27 03:24:11,297][105620] Updated weights for policy 1, policy_version 1651685 (0.0010) [2023-12-27 03:24:11,351][105620] Updated weights for policy 1, policy_version 1651695 (0.0008) [2023-12-27 03:24:11,426][105620] Updated weights for policy 1, policy_version 1651705 (0.0009) [2023-12-27 03:24:11,713][105692] Updated weights for policy 0, policy_version 1648484 (0.0007) [2023-12-27 03:24:11,782][105692] Updated weights for policy 0, policy_version 1648494 (0.0008) [2023-12-27 03:24:11,852][105692] Updated weights for policy 0, policy_version 1648504 (0.0006) [2023-12-27 03:24:12,288][105620] Updated weights for policy 1, policy_version 1651715 (0.0008) [2023-12-27 03:24:12,349][105620] Updated weights for policy 1, policy_version 1651725 (0.0010) [2023-12-27 03:24:12,408][105620] Updated weights for policy 1, policy_version 1651735 (0.0011) [2023-12-27 03:24:12,471][105692] Updated weights for policy 0, policy_version 1648514 (0.0007) [2023-12-27 03:24:12,528][105692] Updated weights for policy 0, policy_version 1648524 (0.0007) [2023-12-27 03:24:12,597][105692] Updated weights for policy 0, policy_version 1648534 (0.0006) [2023-12-27 03:24:12,665][105692] Updated weights for policy 0, policy_version 1648544 (0.0005) [2023-12-27 03:24:13,183][105620] Updated weights for policy 1, policy_version 1651745 (0.0006) [2023-12-27 03:24:13,206][105692] Updated weights for policy 0, policy_version 1648554 (0.0010) [2023-12-27 03:24:13,249][105620] Updated weights for policy 1, policy_version 1651755 (0.0007) [2023-12-27 03:24:13,256][105692] Updated weights for policy 0, policy_version 1648564 (0.0006) [2023-12-27 03:24:13,310][105692] Updated weights for policy 0, policy_version 1648574 (0.0007) [2023-12-27 03:24:13,313][105620] Updated weights for policy 1, policy_version 1651765 (0.0005) [2023-12-27 03:24:13,375][105620] Updated weights for policy 1, policy_version 1651775 (0.0006) [2023-12-27 03:24:13,869][105692] Updated weights for policy 0, policy_version 1648584 (0.0006) [2023-12-27 03:24:13,915][105692] Updated weights for policy 0, policy_version 1648594 (0.0005) [2023-12-27 03:24:13,969][105692] Updated weights for policy 0, policy_version 1648604 (0.0005) [2023-12-27 03:24:14,039][105620] Updated weights for policy 1, policy_version 1651785 (0.0006) [2023-12-27 03:24:14,098][105620] Updated weights for policy 1, policy_version 1651795 (0.0006) [2023-12-27 03:24:14,159][105620] Updated weights for policy 1, policy_version 1651805 (0.0010) [2023-12-27 03:24:14,589][105692] Updated weights for policy 0, policy_version 1648614 (0.0006) [2023-12-27 03:24:14,651][105692] Updated weights for policy 0, policy_version 1648624 (0.0006) [2023-12-27 03:24:14,714][105692] Updated weights for policy 0, policy_version 1648634 (0.0005) [2023-12-27 03:24:14,880][105620] Updated weights for policy 1, policy_version 1651815 (0.0008) [2023-12-27 03:24:14,947][105620] Updated weights for policy 1, policy_version 1651825 (0.0008) [2023-12-27 03:24:15,011][105620] Updated weights for policy 1, policy_version 1651835 (0.0008) [2023-12-27 03:24:15,410][105692] Updated weights for policy 0, policy_version 1648644 (0.0008) [2023-12-27 03:24:15,468][105692] Updated weights for policy 0, policy_version 1648654 (0.0007) [2023-12-27 03:24:15,523][105692] Updated weights for policy 0, policy_version 1648664 (0.0008) [2023-12-27 03:24:15,791][105620] Updated weights for policy 1, policy_version 1651845 (0.0008) [2023-12-27 03:24:15,845][105620] Updated weights for policy 1, policy_version 1651855 (0.0005) [2023-12-27 03:24:15,896][105620] Updated weights for policy 1, policy_version 1651865 (0.0005) [2023-12-27 03:24:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 845062144. Throughput: 0: 9730.2, 1: 9686.1. Samples: 845029640. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:24:16,063][104569] Avg episode reward: [(0, '8894.801'), (1, '8810.847')] [2023-12-27 03:24:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001648672_422125568.pth... [2023-12-27 03:24:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001651872_422936576.pth... [2023-12-27 03:24:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001647520_421830656.pth [2023-12-27 03:24:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001650720_422641664.pth [2023-12-27 03:24:16,264][105692] Updated weights for policy 0, policy_version 1648674 (0.0011) [2023-12-27 03:24:16,329][105692] Updated weights for policy 0, policy_version 1648684 (0.0010) [2023-12-27 03:24:16,384][105692] Updated weights for policy 0, policy_version 1648694 (0.0010) [2023-12-27 03:24:16,453][105692] Updated weights for policy 0, policy_version 1648704 (0.0010) [2023-12-27 03:24:16,531][105620] Updated weights for policy 1, policy_version 1651875 (0.0006) [2023-12-27 03:24:16,584][105620] Updated weights for policy 1, policy_version 1651885 (0.0007) [2023-12-27 03:24:16,643][105620] Updated weights for policy 1, policy_version 1651895 (0.0008) [2023-12-27 03:24:17,165][105692] Updated weights for policy 0, policy_version 1648714 (0.0010) [2023-12-27 03:24:17,217][105692] Updated weights for policy 0, policy_version 1648724 (0.0010) [2023-12-27 03:24:17,260][105692] Updated weights for policy 0, policy_version 1648734 (0.0010) [2023-12-27 03:24:17,358][105620] Updated weights for policy 1, policy_version 1651905 (0.0008) [2023-12-27 03:24:17,416][105620] Updated weights for policy 1, policy_version 1651915 (0.0008) [2023-12-27 03:24:17,475][105620] Updated weights for policy 1, policy_version 1651925 (0.0008) [2023-12-27 03:24:17,536][105620] Updated weights for policy 1, policy_version 1651935 (0.0008) [2023-12-27 03:24:17,938][105692] Updated weights for policy 0, policy_version 1648744 (0.0006) [2023-12-27 03:24:17,999][105692] Updated weights for policy 0, policy_version 1648754 (0.0008) [2023-12-27 03:24:18,054][105692] Updated weights for policy 0, policy_version 1648764 (0.0010) [2023-12-27 03:24:18,340][105620] Updated weights for policy 1, policy_version 1651945 (0.0008) [2023-12-27 03:24:18,400][105620] Updated weights for policy 1, policy_version 1651955 (0.0009) [2023-12-27 03:24:18,461][105620] Updated weights for policy 1, policy_version 1651965 (0.0006) [2023-12-27 03:24:18,772][105692] Updated weights for policy 0, policy_version 1648774 (0.0010) [2023-12-27 03:24:18,837][105692] Updated weights for policy 0, policy_version 1648784 (0.0010) [2023-12-27 03:24:18,896][105692] Updated weights for policy 0, policy_version 1648794 (0.0005) [2023-12-27 03:24:19,245][105620] Updated weights for policy 1, policy_version 1651975 (0.0008) [2023-12-27 03:24:19,317][105620] Updated weights for policy 1, policy_version 1651985 (0.0009) [2023-12-27 03:24:19,384][105620] Updated weights for policy 1, policy_version 1651995 (0.0008) [2023-12-27 03:24:19,537][105692] Updated weights for policy 0, policy_version 1648804 (0.0008) [2023-12-27 03:24:19,590][105692] Updated weights for policy 0, policy_version 1648814 (0.0010) [2023-12-27 03:24:19,646][105692] Updated weights for policy 0, policy_version 1648824 (0.0010) [2023-12-27 03:24:20,166][105620] Updated weights for policy 1, policy_version 1652005 (0.0008) [2023-12-27 03:24:20,229][105620] Updated weights for policy 1, policy_version 1652015 (0.0010) [2023-12-27 03:24:20,287][105620] Updated weights for policy 1, policy_version 1652025 (0.0010) [2023-12-27 03:24:20,349][105692] Updated weights for policy 0, policy_version 1648834 (0.0010) [2023-12-27 03:24:20,409][105692] Updated weights for policy 0, policy_version 1648844 (0.0008) [2023-12-27 03:24:20,470][105692] Updated weights for policy 0, policy_version 1648854 (0.0007) [2023-12-27 03:24:20,534][105692] Updated weights for policy 0, policy_version 1648864 (0.0009) [2023-12-27 03:24:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 845152256. Throughput: 0: 9716.3, 1: 9707.0. Samples: 845147156. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:24:21,063][104569] Avg episode reward: [(0, '8988.993'), (1, '8903.890')] [2023-12-27 03:24:21,142][105620] Updated weights for policy 1, policy_version 1652035 (0.0010) [2023-12-27 03:24:21,205][105620] Updated weights for policy 1, policy_version 1652045 (0.0008) [2023-12-27 03:24:21,272][105620] Updated weights for policy 1, policy_version 1652055 (0.0009) [2023-12-27 03:24:21,272][105692] Updated weights for policy 0, policy_version 1648874 (0.0008) [2023-12-27 03:24:21,336][105692] Updated weights for policy 0, policy_version 1648884 (0.0006) [2023-12-27 03:24:21,412][105692] Updated weights for policy 0, policy_version 1648894 (0.0009) [2023-12-27 03:24:22,069][105620] Updated weights for policy 1, policy_version 1652065 (0.0009) [2023-12-27 03:24:22,122][105620] Updated weights for policy 1, policy_version 1652075 (0.0011) [2023-12-27 03:24:22,182][105620] Updated weights for policy 1, policy_version 1652085 (0.0011) [2023-12-27 03:24:22,184][105692] Updated weights for policy 0, policy_version 1648904 (0.0006) [2023-12-27 03:24:22,238][105620] Updated weights for policy 1, policy_version 1652095 (0.0010) [2023-12-27 03:24:22,245][105692] Updated weights for policy 0, policy_version 1648914 (0.0006) [2023-12-27 03:24:22,312][105692] Updated weights for policy 0, policy_version 1648924 (0.0009) [2023-12-27 03:24:22,976][105620] Updated weights for policy 1, policy_version 1652105 (0.0008) [2023-12-27 03:24:23,040][105620] Updated weights for policy 1, policy_version 1652115 (0.0009) [2023-12-27 03:24:23,082][105692] Updated weights for policy 0, policy_version 1648934 (0.0006) [2023-12-27 03:24:23,096][105620] Updated weights for policy 1, policy_version 1652125 (0.0009) [2023-12-27 03:24:23,133][105692] Updated weights for policy 0, policy_version 1648944 (0.0009) [2023-12-27 03:24:23,189][105692] Updated weights for policy 0, policy_version 1648954 (0.0009) [2023-12-27 03:24:23,833][105620] Updated weights for policy 1, policy_version 1652135 (0.0006) [2023-12-27 03:24:23,882][105620] Updated weights for policy 1, policy_version 1652145 (0.0005) [2023-12-27 03:24:23,930][105620] Updated weights for policy 1, policy_version 1652155 (0.0006) [2023-12-27 03:24:23,958][105692] Updated weights for policy 0, policy_version 1648964 (0.0008) [2023-12-27 03:24:24,023][105692] Updated weights for policy 0, policy_version 1648974 (0.0009) [2023-12-27 03:24:24,077][105692] Updated weights for policy 0, policy_version 1648984 (0.0010) [2023-12-27 03:24:24,550][105620] Updated weights for policy 1, policy_version 1652165 (0.0008) [2023-12-27 03:24:24,611][105620] Updated weights for policy 1, policy_version 1652175 (0.0008) [2023-12-27 03:24:24,660][105620] Updated weights for policy 1, policy_version 1652185 (0.0006) [2023-12-27 03:24:24,956][105692] Updated weights for policy 0, policy_version 1648994 (0.0009) [2023-12-27 03:24:25,024][105692] Updated weights for policy 0, policy_version 1649004 (0.0010) [2023-12-27 03:24:25,092][105692] Updated weights for policy 0, policy_version 1649014 (0.0009) [2023-12-27 03:24:25,158][105692] Updated weights for policy 0, policy_version 1649024 (0.0010) [2023-12-27 03:24:25,236][105620] Updated weights for policy 1, policy_version 1652195 (0.0006) [2023-12-27 03:24:25,295][105620] Updated weights for policy 1, policy_version 1652205 (0.0007) [2023-12-27 03:24:25,348][105620] Updated weights for policy 1, policy_version 1652215 (0.0010) [2023-12-27 03:24:25,933][105620] Updated weights for policy 1, policy_version 1652225 (0.0008) [2023-12-27 03:24:25,933][105692] Updated weights for policy 0, policy_version 1649034 (0.0011) [2023-12-27 03:24:25,993][105620] Updated weights for policy 1, policy_version 1652235 (0.0011) [2023-12-27 03:24:25,993][105692] Updated weights for policy 0, policy_version 1649044 (0.0011) [2023-12-27 03:24:26,046][105620] Updated weights for policy 1, policy_version 1652245 (0.0011) [2023-12-27 03:24:26,049][105692] Updated weights for policy 0, policy_version 1649054 (0.0011) [2023-12-27 03:24:26,062][104569] Fps is (10 sec: 18842.3, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 845250560. Throughput: 0: 9673.7, 1: 9721.4. Samples: 845260140. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:24:26,062][104569] Avg episode reward: [(0, '8713.426'), (1, '8903.116')] [2023-12-27 03:24:26,101][105620] Updated weights for policy 1, policy_version 1652255 (0.0011) [2023-12-27 03:24:26,796][105692] Updated weights for policy 0, policy_version 1649064 (0.0010) [2023-12-27 03:24:26,807][105620] Updated weights for policy 1, policy_version 1652265 (0.0006) [2023-12-27 03:24:26,848][105692] Updated weights for policy 0, policy_version 1649074 (0.0010) [2023-12-27 03:24:26,879][105620] Updated weights for policy 1, policy_version 1652275 (0.0006) [2023-12-27 03:24:26,900][105692] Updated weights for policy 0, policy_version 1649084 (0.0010) [2023-12-27 03:24:26,932][105620] Updated weights for policy 1, policy_version 1652285 (0.0005) [2023-12-27 03:24:27,489][105620] Updated weights for policy 1, policy_version 1652295 (0.0007) [2023-12-27 03:24:27,547][105620] Updated weights for policy 1, policy_version 1652305 (0.0008) [2023-12-27 03:24:27,601][105620] Updated weights for policy 1, policy_version 1652315 (0.0007) [2023-12-27 03:24:27,665][105692] Updated weights for policy 0, policy_version 1649094 (0.0010) [2023-12-27 03:24:27,715][105692] Updated weights for policy 0, policy_version 1649104 (0.0009) [2023-12-27 03:24:27,783][105692] Updated weights for policy 0, policy_version 1649114 (0.0010) [2023-12-27 03:24:28,250][105620] Updated weights for policy 1, policy_version 1652325 (0.0007) [2023-12-27 03:24:28,310][105620] Updated weights for policy 1, policy_version 1652335 (0.0005) [2023-12-27 03:24:28,371][105620] Updated weights for policy 1, policy_version 1652345 (0.0010) [2023-12-27 03:24:28,538][105692] Updated weights for policy 0, policy_version 1649124 (0.0010) [2023-12-27 03:24:28,594][105692] Updated weights for policy 0, policy_version 1649134 (0.0010) [2023-12-27 03:24:28,655][105692] Updated weights for policy 0, policy_version 1649144 (0.0010) [2023-12-27 03:24:28,986][105620] Updated weights for policy 1, policy_version 1652355 (0.0011) [2023-12-27 03:24:29,034][105620] Updated weights for policy 1, policy_version 1652365 (0.0010) [2023-12-27 03:24:29,092][105620] Updated weights for policy 1, policy_version 1652375 (0.0010) [2023-12-27 03:24:29,370][105692] Updated weights for policy 0, policy_version 1649154 (0.0011) [2023-12-27 03:24:29,433][105692] Updated weights for policy 0, policy_version 1649164 (0.0007) [2023-12-27 03:24:29,493][105692] Updated weights for policy 0, policy_version 1649174 (0.0005) [2023-12-27 03:24:29,554][105692] Updated weights for policy 0, policy_version 1649184 (0.0005) [2023-12-27 03:24:29,908][105620] Updated weights for policy 1, policy_version 1652385 (0.0010) [2023-12-27 03:24:30,004][105620] Updated weights for policy 1, policy_version 1652395 (0.0006) [2023-12-27 03:24:30,060][105620] Updated weights for policy 1, policy_version 1652405 (0.0009) [2023-12-27 03:24:30,106][105692] Updated weights for policy 0, policy_version 1649194 (0.0007) [2023-12-27 03:24:30,113][105620] Updated weights for policy 1, policy_version 1652415 (0.0007) [2023-12-27 03:24:30,172][105692] Updated weights for policy 0, policy_version 1649204 (0.0006) [2023-12-27 03:24:30,232][105692] Updated weights for policy 0, policy_version 1649214 (0.0009) [2023-12-27 03:24:30,830][105620] Updated weights for policy 1, policy_version 1652425 (0.0009) [2023-12-27 03:24:30,872][105692] Updated weights for policy 0, policy_version 1649224 (0.0007) [2023-12-27 03:24:30,878][105620] Updated weights for policy 1, policy_version 1652435 (0.0007) [2023-12-27 03:24:30,926][105620] Updated weights for policy 1, policy_version 1652445 (0.0006) [2023-12-27 03:24:30,930][105692] Updated weights for policy 0, policy_version 1649234 (0.0008) [2023-12-27 03:24:30,990][105692] Updated weights for policy 0, policy_version 1649244 (0.0009) [2023-12-27 03:24:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 845357056. Throughput: 0: 9672.3, 1: 9815.0. Samples: 845320572. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:24:31,062][104569] Avg episode reward: [(0, '8162.522'), (1, '8902.892')] [2023-12-27 03:24:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001649248_422273024.pth... [2023-12-27 03:24:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001652448_423084032.pth... [2023-12-27 03:24:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001651328_422797312.pth [2023-12-27 03:24:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001648096_421978112.pth [2023-12-27 03:24:31,723][105692] Updated weights for policy 0, policy_version 1649254 (0.0008) [2023-12-27 03:24:31,756][105620] Updated weights for policy 1, policy_version 1652455 (0.0007) [2023-12-27 03:24:31,783][105692] Updated weights for policy 0, policy_version 1649264 (0.0006) [2023-12-27 03:24:31,816][105620] Updated weights for policy 1, policy_version 1652465 (0.0010) [2023-12-27 03:24:31,829][105692] Updated weights for policy 0, policy_version 1649274 (0.0009) [2023-12-27 03:24:31,874][105620] Updated weights for policy 1, policy_version 1652475 (0.0006) [2023-12-27 03:24:32,555][105620] Updated weights for policy 1, policy_version 1652485 (0.0007) [2023-12-27 03:24:32,619][105692] Updated weights for policy 0, policy_version 1649284 (0.0008) [2023-12-27 03:24:32,622][105620] Updated weights for policy 1, policy_version 1652495 (0.0007) [2023-12-27 03:24:32,678][105692] Updated weights for policy 0, policy_version 1649294 (0.0011) [2023-12-27 03:24:32,681][105620] Updated weights for policy 1, policy_version 1652505 (0.0005) [2023-12-27 03:24:32,733][105692] Updated weights for policy 0, policy_version 1649304 (0.0011) [2023-12-27 03:24:33,379][105692] Updated weights for policy 0, policy_version 1649314 (0.0009) [2023-12-27 03:24:33,406][105620] Updated weights for policy 1, policy_version 1652515 (0.0005) [2023-12-27 03:24:33,437][105692] Updated weights for policy 0, policy_version 1649324 (0.0010) [2023-12-27 03:24:33,455][105620] Updated weights for policy 1, policy_version 1652525 (0.0006) [2023-12-27 03:24:33,482][105692] Updated weights for policy 0, policy_version 1649334 (0.0010) [2023-12-27 03:24:33,502][105620] Updated weights for policy 1, policy_version 1652535 (0.0005) [2023-12-27 03:24:33,537][105692] Updated weights for policy 0, policy_version 1649344 (0.0010) [2023-12-27 03:24:34,045][105620] Updated weights for policy 1, policy_version 1652545 (0.0005) [2023-12-27 03:24:34,100][105620] Updated weights for policy 1, policy_version 1652555 (0.0005) [2023-12-27 03:24:34,161][105620] Updated weights for policy 1, policy_version 1652565 (0.0007) [2023-12-27 03:24:34,221][105620] Updated weights for policy 1, policy_version 1652575 (0.0008) [2023-12-27 03:24:34,275][105692] Updated weights for policy 0, policy_version 1649354 (0.0011) [2023-12-27 03:24:34,343][105692] Updated weights for policy 0, policy_version 1649364 (0.0011) [2023-12-27 03:24:34,406][105692] Updated weights for policy 0, policy_version 1649374 (0.0011) [2023-12-27 03:24:34,911][105620] Updated weights for policy 1, policy_version 1652585 (0.0008) [2023-12-27 03:24:34,974][105620] Updated weights for policy 1, policy_version 1652595 (0.0008) [2023-12-27 03:24:35,039][105620] Updated weights for policy 1, policy_version 1652605 (0.0008) [2023-12-27 03:24:35,170][105692] Updated weights for policy 0, policy_version 1649384 (0.0011) [2023-12-27 03:24:35,232][105692] Updated weights for policy 0, policy_version 1649394 (0.0010) [2023-12-27 03:24:35,297][105692] Updated weights for policy 0, policy_version 1649404 (0.0011) [2023-12-27 03:24:35,690][105620] Updated weights for policy 1, policy_version 1652615 (0.0009) [2023-12-27 03:24:35,750][105620] Updated weights for policy 1, policy_version 1652625 (0.0008) [2023-12-27 03:24:35,805][105620] Updated weights for policy 1, policy_version 1652635 (0.0008) [2023-12-27 03:24:36,047][105692] Updated weights for policy 0, policy_version 1649414 (0.0011) [2023-12-27 03:24:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 845447168. Throughput: 0: 9681.1, 1: 9733.6. Samples: 845438976. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:24:36,062][104569] Avg episode reward: [(0, '8618.021'), (1, '9265.289')] [2023-12-27 03:24:36,101][105692] Updated weights for policy 0, policy_version 1649424 (0.0010) [2023-12-27 03:24:36,161][105692] Updated weights for policy 0, policy_version 1649434 (0.0009) [2023-12-27 03:24:36,411][105620] Updated weights for policy 1, policy_version 1652645 (0.0008) [2023-12-27 03:24:36,480][105620] Updated weights for policy 1, policy_version 1652655 (0.0008) [2023-12-27 03:24:36,565][105620] Updated weights for policy 1, policy_version 1652665 (0.0007) [2023-12-27 03:24:36,939][105692] Updated weights for policy 0, policy_version 1649444 (0.0008) [2023-12-27 03:24:36,997][105692] Updated weights for policy 0, policy_version 1649454 (0.0010) [2023-12-27 03:24:37,056][105692] Updated weights for policy 0, policy_version 1649464 (0.0010) [2023-12-27 03:24:37,255][105620] Updated weights for policy 1, policy_version 1652675 (0.0008) [2023-12-27 03:24:37,321][105620] Updated weights for policy 1, policy_version 1652685 (0.0009) [2023-12-27 03:24:37,377][105620] Updated weights for policy 1, policy_version 1652695 (0.0009) [2023-12-27 03:24:37,785][105692] Updated weights for policy 0, policy_version 1649474 (0.0010) [2023-12-27 03:24:37,847][105692] Updated weights for policy 0, policy_version 1649484 (0.0010) [2023-12-27 03:24:37,900][105692] Updated weights for policy 0, policy_version 1649494 (0.0006) [2023-12-27 03:24:37,949][105692] Updated weights for policy 0, policy_version 1649504 (0.0010) [2023-12-27 03:24:38,121][105620] Updated weights for policy 1, policy_version 1652705 (0.0008) [2023-12-27 03:24:38,182][105620] Updated weights for policy 1, policy_version 1652715 (0.0005) [2023-12-27 03:24:38,243][105620] Updated weights for policy 1, policy_version 1652725 (0.0005) [2023-12-27 03:24:38,294][105620] Updated weights for policy 1, policy_version 1652735 (0.0005) [2023-12-27 03:24:38,590][105692] Updated weights for policy 0, policy_version 1649514 (0.0010) [2023-12-27 03:24:38,654][105692] Updated weights for policy 0, policy_version 1649524 (0.0009) [2023-12-27 03:24:38,721][105692] Updated weights for policy 0, policy_version 1649534 (0.0011) [2023-12-27 03:24:39,017][105620] Updated weights for policy 1, policy_version 1652745 (0.0009) [2023-12-27 03:24:39,070][105620] Updated weights for policy 1, policy_version 1652755 (0.0009) [2023-12-27 03:24:39,124][105620] Updated weights for policy 1, policy_version 1652766 (0.0009) [2023-12-27 03:24:39,281][105692] Updated weights for policy 0, policy_version 1649544 (0.0010) [2023-12-27 03:24:39,337][105692] Updated weights for policy 0, policy_version 1649554 (0.0010) [2023-12-27 03:24:39,404][105692] Updated weights for policy 0, policy_version 1649564 (0.0011) [2023-12-27 03:24:40,007][105620] Updated weights for policy 1, policy_version 1652776 (0.0009) [2023-12-27 03:24:40,079][105620] Updated weights for policy 1, policy_version 1652786 (0.0008) [2023-12-27 03:24:40,135][105692] Updated weights for policy 0, policy_version 1649574 (0.0007) [2023-12-27 03:24:40,143][105620] Updated weights for policy 1, policy_version 1652796 (0.0007) [2023-12-27 03:24:40,203][105692] Updated weights for policy 0, policy_version 1649584 (0.0006) [2023-12-27 03:24:40,265][105692] Updated weights for policy 0, policy_version 1649594 (0.0009) [2023-12-27 03:24:40,831][105620] Updated weights for policy 1, policy_version 1652806 (0.0006) [2023-12-27 03:24:40,897][105620] Updated weights for policy 1, policy_version 1652816 (0.0006) [2023-12-27 03:24:40,969][105620] Updated weights for policy 1, policy_version 1652826 (0.0009) [2023-12-27 03:24:41,000][105692] Updated weights for policy 0, policy_version 1649604 (0.0008) [2023-12-27 03:24:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 845545472. Throughput: 0: 9709.2, 1: 9662.2. Samples: 845555320. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:24:41,062][104569] Avg episode reward: [(0, '8985.396'), (1, '9265.059')] [2023-12-27 03:24:41,064][105692] Updated weights for policy 0, policy_version 1649614 (0.0007) [2023-12-27 03:24:41,120][105692] Updated weights for policy 0, policy_version 1649624 (0.0009) [2023-12-27 03:24:41,777][105620] Updated weights for policy 1, policy_version 1652836 (0.0008) [2023-12-27 03:24:41,841][105620] Updated weights for policy 1, policy_version 1652846 (0.0009) [2023-12-27 03:24:41,888][105692] Updated weights for policy 0, policy_version 1649634 (0.0008) [2023-12-27 03:24:41,907][105620] Updated weights for policy 1, policy_version 1652856 (0.0009) [2023-12-27 03:24:41,956][105692] Updated weights for policy 0, policy_version 1649644 (0.0008) [2023-12-27 03:24:42,021][105692] Updated weights for policy 0, policy_version 1649654 (0.0008) [2023-12-27 03:24:42,073][105692] Updated weights for policy 0, policy_version 1649664 (0.0008) [2023-12-27 03:24:42,713][105620] Updated weights for policy 1, policy_version 1652866 (0.0009) [2023-12-27 03:24:42,776][105620] Updated weights for policy 1, policy_version 1652876 (0.0009) [2023-12-27 03:24:42,817][105692] Updated weights for policy 0, policy_version 1649674 (0.0006) [2023-12-27 03:24:42,835][105620] Updated weights for policy 1, policy_version 1652886 (0.0007) [2023-12-27 03:24:42,880][105692] Updated weights for policy 0, policy_version 1649684 (0.0007) [2023-12-27 03:24:42,892][105620] Updated weights for policy 1, policy_version 1652896 (0.0006) [2023-12-27 03:24:42,944][105692] Updated weights for policy 0, policy_version 1649694 (0.0005) [2023-12-27 03:24:43,559][105692] Updated weights for policy 0, policy_version 1649704 (0.0008) [2023-12-27 03:24:43,619][105692] Updated weights for policy 0, policy_version 1649714 (0.0009) [2023-12-27 03:24:43,673][105620] Updated weights for policy 1, policy_version 1652906 (0.0005) [2023-12-27 03:24:43,678][105692] Updated weights for policy 0, policy_version 1649724 (0.0010) [2023-12-27 03:24:43,726][105620] Updated weights for policy 1, policy_version 1652916 (0.0008) [2023-12-27 03:24:43,770][105620] Updated weights for policy 1, policy_version 1652926 (0.0007) [2023-12-27 03:24:44,315][105692] Updated weights for policy 0, policy_version 1649734 (0.0010) [2023-12-27 03:24:44,370][105692] Updated weights for policy 0, policy_version 1649744 (0.0010) [2023-12-27 03:24:44,442][105692] Updated weights for policy 0, policy_version 1649754 (0.0010) [2023-12-27 03:24:44,574][105620] Updated weights for policy 1, policy_version 1652936 (0.0008) [2023-12-27 03:24:44,628][105620] Updated weights for policy 1, policy_version 1652946 (0.0010) [2023-12-27 03:24:44,683][105620] Updated weights for policy 1, policy_version 1652956 (0.0010) [2023-12-27 03:24:45,142][105692] Updated weights for policy 0, policy_version 1649764 (0.0011) [2023-12-27 03:24:45,203][105692] Updated weights for policy 0, policy_version 1649774 (0.0011) [2023-12-27 03:24:45,266][105692] Updated weights for policy 0, policy_version 1649784 (0.0011) [2023-12-27 03:24:45,353][105620] Updated weights for policy 1, policy_version 1652966 (0.0010) [2023-12-27 03:24:45,422][105620] Updated weights for policy 1, policy_version 1652976 (0.0010) [2023-12-27 03:24:45,488][105620] Updated weights for policy 1, policy_version 1652986 (0.0010) [2023-12-27 03:24:46,002][105692] Updated weights for policy 0, policy_version 1649794 (0.0011) [2023-12-27 03:24:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 845635584. Throughput: 0: 9663.2, 1: 9609.4. Samples: 845609720. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:24:46,062][104569] Avg episode reward: [(0, '8808.098'), (1, '9265.031')] [2023-12-27 03:24:46,072][105692] Updated weights for policy 0, policy_version 1649804 (0.0011) [2023-12-27 03:24:46,102][105620] Updated weights for policy 1, policy_version 1652996 (0.0008) [2023-12-27 03:24:46,134][105692] Updated weights for policy 0, policy_version 1649814 (0.0011) [2023-12-27 03:24:46,151][105620] Updated weights for policy 1, policy_version 1653006 (0.0005) [2023-12-27 03:24:46,185][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001649824_422420480.pth... [2023-12-27 03:24:46,186][105692] Updated weights for policy 0, policy_version 1649824 (0.0011) [2023-12-27 03:24:46,188][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001648672_422125568.pth [2023-12-27 03:24:46,216][105620] Updated weights for policy 1, policy_version 1653016 (0.0005) [2023-12-27 03:24:46,270][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001653024_423231488.pth... [2023-12-27 03:24:46,274][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001651872_422936576.pth [2023-12-27 03:24:46,710][105620] Updated weights for policy 1, policy_version 1653026 (0.0005) [2023-12-27 03:24:46,778][105620] Updated weights for policy 1, policy_version 1653036 (0.0006) [2023-12-27 03:24:46,842][105620] Updated weights for policy 1, policy_version 1653046 (0.0010) [2023-12-27 03:24:46,891][105620] Updated weights for policy 1, policy_version 1653056 (0.0010) [2023-12-27 03:24:46,934][105692] Updated weights for policy 0, policy_version 1649834 (0.0005) [2023-12-27 03:24:46,988][105692] Updated weights for policy 0, policy_version 1649844 (0.0006) [2023-12-27 03:24:47,035][105692] Updated weights for policy 0, policy_version 1649854 (0.0007) [2023-12-27 03:24:47,545][105620] Updated weights for policy 1, policy_version 1653066 (0.0005) [2023-12-27 03:24:47,589][105692] Updated weights for policy 0, policy_version 1649864 (0.0007) [2023-12-27 03:24:47,591][105620] Updated weights for policy 1, policy_version 1653076 (0.0005) [2023-12-27 03:24:47,634][105692] Updated weights for policy 0, policy_version 1649874 (0.0006) [2023-12-27 03:24:47,637][105620] Updated weights for policy 1, policy_version 1653086 (0.0006) [2023-12-27 03:24:47,679][105692] Updated weights for policy 0, policy_version 1649884 (0.0007) [2023-12-27 03:24:48,338][105692] Updated weights for policy 0, policy_version 1649894 (0.0009) [2023-12-27 03:24:48,350][105620] Updated weights for policy 1, policy_version 1653096 (0.0009) [2023-12-27 03:24:48,402][105692] Updated weights for policy 0, policy_version 1649904 (0.0007) [2023-12-27 03:24:48,411][105620] Updated weights for policy 1, policy_version 1653106 (0.0009) [2023-12-27 03:24:48,462][105692] Updated weights for policy 0, policy_version 1649914 (0.0006) [2023-12-27 03:24:48,466][105620] Updated weights for policy 1, policy_version 1653116 (0.0010) [2023-12-27 03:24:49,139][105620] Updated weights for policy 1, policy_version 1653126 (0.0010) [2023-12-27 03:24:49,186][105620] Updated weights for policy 1, policy_version 1653136 (0.0006) [2023-12-27 03:24:49,191][105692] Updated weights for policy 0, policy_version 1649924 (0.0007) [2023-12-27 03:24:49,249][105620] Updated weights for policy 1, policy_version 1653146 (0.0008) [2023-12-27 03:24:49,258][105692] Updated weights for policy 0, policy_version 1649934 (0.0007) [2023-12-27 03:24:49,321][105692] Updated weights for policy 0, policy_version 1649944 (0.0008) [2023-12-27 03:24:49,994][105620] Updated weights for policy 1, policy_version 1653156 (0.0008) [2023-12-27 03:24:50,046][105620] Updated weights for policy 1, policy_version 1653166 (0.0009) [2023-12-27 03:24:50,096][105620] Updated weights for policy 1, policy_version 1653176 (0.0008) [2023-12-27 03:24:50,097][105692] Updated weights for policy 0, policy_version 1649954 (0.0010) [2023-12-27 03:24:50,151][105692] Updated weights for policy 0, policy_version 1649964 (0.0008) [2023-12-27 03:24:50,216][105692] Updated weights for policy 0, policy_version 1649974 (0.0009) [2023-12-27 03:24:50,277][105692] Updated weights for policy 0, policy_version 1649984 (0.0008) [2023-12-27 03:24:50,877][105620] Updated weights for policy 1, policy_version 1653186 (0.0006) [2023-12-27 03:24:50,944][105620] Updated weights for policy 1, policy_version 1653196 (0.0009) [2023-12-27 03:24:50,991][105620] Updated weights for policy 1, policy_version 1653206 (0.0008) [2023-12-27 03:24:51,053][105692] Updated weights for policy 0, policy_version 1649994 (0.0007) [2023-12-27 03:24:51,058][105620] Updated weights for policy 1, policy_version 1653216 (0.0008) [2023-12-27 03:24:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 845742080. Throughput: 0: 9791.5, 1: 9744.7. Samples: 845733004. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:24:51,062][104569] Avg episode reward: [(0, '8719.075'), (1, '9264.016')] [2023-12-27 03:24:51,116][105692] Updated weights for policy 0, policy_version 1650004 (0.0009) [2023-12-27 03:24:51,184][105692] Updated weights for policy 0, policy_version 1650014 (0.0009) [2023-12-27 03:24:51,906][105620] Updated weights for policy 1, policy_version 1653226 (0.0010) [2023-12-27 03:24:51,911][105692] Updated weights for policy 0, policy_version 1650024 (0.0006) [2023-12-27 03:24:51,970][105620] Updated weights for policy 1, policy_version 1653236 (0.0009) [2023-12-27 03:24:51,973][105692] Updated weights for policy 0, policy_version 1650034 (0.0005) [2023-12-27 03:24:52,026][105692] Updated weights for policy 0, policy_version 1650044 (0.0006) [2023-12-27 03:24:52,028][105620] Updated weights for policy 1, policy_version 1653246 (0.0008) [2023-12-27 03:24:52,718][105692] Updated weights for policy 0, policy_version 1650054 (0.0008) [2023-12-27 03:24:52,771][105692] Updated weights for policy 0, policy_version 1650064 (0.0009) [2023-12-27 03:24:52,818][105620] Updated weights for policy 1, policy_version 1653256 (0.0006) [2023-12-27 03:24:52,831][105692] Updated weights for policy 0, policy_version 1650074 (0.0007) [2023-12-27 03:24:52,879][105620] Updated weights for policy 1, policy_version 1653266 (0.0007) [2023-12-27 03:24:52,935][105620] Updated weights for policy 1, policy_version 1653276 (0.0009) [2023-12-27 03:24:53,571][105692] Updated weights for policy 0, policy_version 1650084 (0.0009) [2023-12-27 03:24:53,625][105692] Updated weights for policy 0, policy_version 1650094 (0.0009) [2023-12-27 03:24:53,676][105692] Updated weights for policy 0, policy_version 1650104 (0.0008) [2023-12-27 03:24:53,686][105620] Updated weights for policy 1, policy_version 1653286 (0.0007) [2023-12-27 03:24:53,744][105620] Updated weights for policy 1, policy_version 1653296 (0.0008) [2023-12-27 03:24:53,809][105620] Updated weights for policy 1, policy_version 1653306 (0.0009) [2023-12-27 03:24:54,441][105692] Updated weights for policy 0, policy_version 1650114 (0.0007) [2023-12-27 03:24:54,498][105692] Updated weights for policy 0, policy_version 1650124 (0.0009) [2023-12-27 03:24:54,509][105620] Updated weights for policy 1, policy_version 1653316 (0.0007) [2023-12-27 03:24:54,544][105692] Updated weights for policy 0, policy_version 1650134 (0.0007) [2023-12-27 03:24:54,562][105620] Updated weights for policy 1, policy_version 1653326 (0.0006) [2023-12-27 03:24:54,596][105692] Updated weights for policy 0, policy_version 1650144 (0.0006) [2023-12-27 03:24:54,613][105620] Updated weights for policy 1, policy_version 1653336 (0.0007) [2023-12-27 03:24:55,325][105692] Updated weights for policy 0, policy_version 1650154 (0.0005) [2023-12-27 03:24:55,370][105692] Updated weights for policy 0, policy_version 1650164 (0.0005) [2023-12-27 03:24:55,413][105692] Updated weights for policy 0, policy_version 1650174 (0.0005) [2023-12-27 03:24:55,422][105620] Updated weights for policy 1, policy_version 1653346 (0.0009) [2023-12-27 03:24:55,478][105620] Updated weights for policy 1, policy_version 1653356 (0.0009) [2023-12-27 03:24:55,526][105620] Updated weights for policy 1, policy_version 1653366 (0.0005) [2023-12-27 03:24:55,573][105620] Updated weights for policy 1, policy_version 1653376 (0.0009) [2023-12-27 03:24:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 845832192. Throughput: 0: 9792.4, 1: 9632.2. Samples: 845844836. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:24:56,062][104569] Avg episode reward: [(0, '8991.093'), (1, '9172.206')] [2023-12-27 03:24:56,131][105692] Updated weights for policy 0, policy_version 1650184 (0.0008) [2023-12-27 03:24:56,197][105692] Updated weights for policy 0, policy_version 1650194 (0.0008) [2023-12-27 03:24:56,261][105692] Updated weights for policy 0, policy_version 1650204 (0.0009) [2023-12-27 03:24:56,323][105620] Updated weights for policy 1, policy_version 1653386 (0.0009) [2023-12-27 03:24:56,385][105620] Updated weights for policy 1, policy_version 1653396 (0.0009) [2023-12-27 03:24:56,446][105620] Updated weights for policy 1, policy_version 1653406 (0.0009) [2023-12-27 03:24:56,963][105692] Updated weights for policy 0, policy_version 1650214 (0.0008) [2023-12-27 03:24:57,013][105692] Updated weights for policy 0, policy_version 1650224 (0.0008) [2023-12-27 03:24:57,063][105692] Updated weights for policy 0, policy_version 1650234 (0.0009) [2023-12-27 03:24:57,193][105620] Updated weights for policy 1, policy_version 1653416 (0.0009) [2023-12-27 03:24:57,250][105620] Updated weights for policy 1, policy_version 1653426 (0.0009) [2023-12-27 03:24:57,315][105620] Updated weights for policy 1, policy_version 1653436 (0.0008) [2023-12-27 03:24:57,810][105692] Updated weights for policy 0, policy_version 1650244 (0.0009) [2023-12-27 03:24:57,861][105692] Updated weights for policy 0, policy_version 1650254 (0.0008) [2023-12-27 03:24:57,908][105692] Updated weights for policy 0, policy_version 1650264 (0.0009) [2023-12-27 03:24:58,076][105620] Updated weights for policy 1, policy_version 1653446 (0.0009) [2023-12-27 03:24:58,131][105620] Updated weights for policy 1, policy_version 1653456 (0.0009) [2023-12-27 03:24:58,189][105620] Updated weights for policy 1, policy_version 1653466 (0.0009) [2023-12-27 03:24:58,722][105692] Updated weights for policy 0, policy_version 1650274 (0.0009) [2023-12-27 03:24:58,791][105692] Updated weights for policy 0, policy_version 1650284 (0.0009) [2023-12-27 03:24:58,857][105692] Updated weights for policy 0, policy_version 1650294 (0.0009) [2023-12-27 03:24:58,913][105620] Updated weights for policy 1, policy_version 1653476 (0.0008) [2023-12-27 03:24:58,925][105692] Updated weights for policy 0, policy_version 1650304 (0.0012) [2023-12-27 03:24:58,974][105620] Updated weights for policy 1, policy_version 1653486 (0.0006) [2023-12-27 03:24:59,040][105620] Updated weights for policy 1, policy_version 1653496 (0.0008) [2023-12-27 03:24:59,615][105692] Updated weights for policy 0, policy_version 1650314 (0.0007) [2023-12-27 03:24:59,694][105692] Updated weights for policy 0, policy_version 1650324 (0.0007) [2023-12-27 03:24:59,753][105692] Updated weights for policy 0, policy_version 1650334 (0.0006) [2023-12-27 03:24:59,806][105620] Updated weights for policy 1, policy_version 1653506 (0.0008) [2023-12-27 03:24:59,864][105620] Updated weights for policy 1, policy_version 1653516 (0.0009) [2023-12-27 03:24:59,926][105620] Updated weights for policy 1, policy_version 1653526 (0.0010) [2023-12-27 03:24:59,979][105620] Updated weights for policy 1, policy_version 1653536 (0.0009) [2023-12-27 03:25:00,384][105692] Updated weights for policy 0, policy_version 1650344 (0.0006) [2023-12-27 03:25:00,445][105692] Updated weights for policy 0, policy_version 1650354 (0.0005) [2023-12-27 03:25:00,501][105692] Updated weights for policy 0, policy_version 1650364 (0.0005) [2023-12-27 03:25:00,855][105620] Updated weights for policy 1, policy_version 1653546 (0.0009) [2023-12-27 03:25:00,916][105620] Updated weights for policy 1, policy_version 1653556 (0.0008) [2023-12-27 03:25:00,975][105620] Updated weights for policy 1, policy_version 1653566 (0.0008) [2023-12-27 03:25:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 845930496. Throughput: 0: 9705.7, 1: 9650.5. Samples: 845900660. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 03:25:01,062][104569] Avg episode reward: [(0, '8992.146'), (1, '9081.363')] [2023-12-27 03:25:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001653568_423370752.pth... [2023-12-27 03:25:01,069][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001652448_423084032.pth [2023-12-27 03:25:01,070][105692] Updated weights for policy 0, policy_version 1650374 (0.0007) [2023-12-27 03:25:01,139][105692] Updated weights for policy 0, policy_version 1650384 (0.0005) [2023-12-27 03:25:01,207][105692] Updated weights for policy 0, policy_version 1650394 (0.0010) [2023-12-27 03:25:01,243][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001650400_422567936.pth... [2023-12-27 03:25:01,248][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001649248_422273024.pth [2023-12-27 03:25:01,784][105620] Updated weights for policy 1, policy_version 1653576 (0.0006) [2023-12-27 03:25:01,807][105692] Updated weights for policy 0, policy_version 1650404 (0.0010) [2023-12-27 03:25:01,845][105620] Updated weights for policy 1, policy_version 1653586 (0.0006) [2023-12-27 03:25:01,867][105692] Updated weights for policy 0, policy_version 1650414 (0.0011) [2023-12-27 03:25:01,913][105620] Updated weights for policy 1, policy_version 1653596 (0.0008) [2023-12-27 03:25:01,925][105692] Updated weights for policy 0, policy_version 1650424 (0.0011) [2023-12-27 03:25:02,580][105620] Updated weights for policy 1, policy_version 1653606 (0.0008) [2023-12-27 03:25:02,630][105692] Updated weights for policy 0, policy_version 1650434 (0.0010) [2023-12-27 03:25:02,631][105620] Updated weights for policy 1, policy_version 1653616 (0.0009) [2023-12-27 03:25:02,674][105692] Updated weights for policy 0, policy_version 1650444 (0.0005) [2023-12-27 03:25:02,682][105620] Updated weights for policy 1, policy_version 1653626 (0.0008) [2023-12-27 03:25:02,729][105692] Updated weights for policy 0, policy_version 1650454 (0.0005) [2023-12-27 03:25:02,773][105692] Updated weights for policy 0, policy_version 1650464 (0.0005) [2023-12-27 03:25:03,349][105620] Updated weights for policy 1, policy_version 1653636 (0.0007) [2023-12-27 03:25:03,393][105692] Updated weights for policy 0, policy_version 1650474 (0.0008) [2023-12-27 03:25:03,415][105620] Updated weights for policy 1, policy_version 1653646 (0.0005) [2023-12-27 03:25:03,438][105692] Updated weights for policy 0, policy_version 1650484 (0.0010) [2023-12-27 03:25:03,471][105620] Updated weights for policy 1, policy_version 1653656 (0.0005) [2023-12-27 03:25:03,489][105692] Updated weights for policy 0, policy_version 1650494 (0.0010) [2023-12-27 03:25:04,011][105620] Updated weights for policy 1, policy_version 1653666 (0.0005) [2023-12-27 03:25:04,066][105620] Updated weights for policy 1, policy_version 1653676 (0.0006) [2023-12-27 03:25:04,137][105620] Updated weights for policy 1, policy_version 1653686 (0.0006) [2023-12-27 03:25:04,198][105620] Updated weights for policy 1, policy_version 1653696 (0.0008) [2023-12-27 03:25:04,256][105692] Updated weights for policy 0, policy_version 1650504 (0.0010) [2023-12-27 03:25:04,315][105692] Updated weights for policy 0, policy_version 1650514 (0.0010) [2023-12-27 03:25:04,374][105692] Updated weights for policy 0, policy_version 1650524 (0.0011) [2023-12-27 03:25:04,820][105620] Updated weights for policy 1, policy_version 1653706 (0.0006) [2023-12-27 03:25:04,872][105620] Updated weights for policy 1, policy_version 1653716 (0.0006) [2023-12-27 03:25:04,920][105620] Updated weights for policy 1, policy_version 1653726 (0.0008) [2023-12-27 03:25:05,116][105692] Updated weights for policy 0, policy_version 1650534 (0.0011) [2023-12-27 03:25:05,163][105692] Updated weights for policy 0, policy_version 1650544 (0.0010) [2023-12-27 03:25:05,216][105692] Updated weights for policy 0, policy_version 1650554 (0.0010) [2023-12-27 03:25:05,491][105620] Updated weights for policy 1, policy_version 1653736 (0.0005) [2023-12-27 03:25:05,541][105620] Updated weights for policy 1, policy_version 1653746 (0.0010) [2023-12-27 03:25:05,596][105620] Updated weights for policy 1, policy_version 1653756 (0.0010) [2023-12-27 03:25:05,981][105692] Updated weights for policy 0, policy_version 1650564 (0.0010) [2023-12-27 03:25:06,050][105692] Updated weights for policy 0, policy_version 1650574 (0.0010) [2023-12-27 03:25:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 846028800. Throughput: 0: 9711.6, 1: 9714.4. Samples: 846021324. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:25:06,062][104569] Avg episode reward: [(0, '8900.101'), (1, '8811.544')] [2023-12-27 03:25:06,119][105692] Updated weights for policy 0, policy_version 1650584 (0.0010) [2023-12-27 03:25:06,272][105620] Updated weights for policy 1, policy_version 1653766 (0.0007) [2023-12-27 03:25:06,327][105620] Updated weights for policy 1, policy_version 1653776 (0.0006) [2023-12-27 03:25:06,387][105620] Updated weights for policy 1, policy_version 1653786 (0.0006) [2023-12-27 03:25:06,869][105692] Updated weights for policy 0, policy_version 1650594 (0.0011) [2023-12-27 03:25:06,932][105692] Updated weights for policy 0, policy_version 1650604 (0.0010) [2023-12-27 03:25:06,990][105692] Updated weights for policy 0, policy_version 1650614 (0.0010) [2023-12-27 03:25:07,048][105692] Updated weights for policy 0, policy_version 1650624 (0.0007) [2023-12-27 03:25:07,096][105620] Updated weights for policy 1, policy_version 1653796 (0.0007) [2023-12-27 03:25:07,162][105620] Updated weights for policy 1, policy_version 1653806 (0.0010) [2023-12-27 03:25:07,227][105620] Updated weights for policy 1, policy_version 1653816 (0.0010) [2023-12-27 03:25:07,650][105692] Updated weights for policy 0, policy_version 1650634 (0.0005) [2023-12-27 03:25:07,710][105692] Updated weights for policy 0, policy_version 1650644 (0.0009) [2023-12-27 03:25:07,772][105692] Updated weights for policy 0, policy_version 1650654 (0.0009) [2023-12-27 03:25:08,097][105620] Updated weights for policy 1, policy_version 1653826 (0.0009) [2023-12-27 03:25:08,164][105620] Updated weights for policy 1, policy_version 1653836 (0.0009) [2023-12-27 03:25:08,220][105620] Updated weights for policy 1, policy_version 1653846 (0.0009) [2023-12-27 03:25:08,275][105620] Updated weights for policy 1, policy_version 1653856 (0.0009) [2023-12-27 03:25:08,450][105692] Updated weights for policy 0, policy_version 1650664 (0.0008) [2023-12-27 03:25:08,513][105692] Updated weights for policy 0, policy_version 1650674 (0.0008) [2023-12-27 03:25:08,581][105692] Updated weights for policy 0, policy_version 1650684 (0.0008) [2023-12-27 03:25:08,996][105620] Updated weights for policy 1, policy_version 1653866 (0.0010) [2023-12-27 03:25:09,060][105620] Updated weights for policy 1, policy_version 1653876 (0.0008) [2023-12-27 03:25:09,120][105620] Updated weights for policy 1, policy_version 1653886 (0.0005) [2023-12-27 03:25:09,181][105692] Updated weights for policy 0, policy_version 1650694 (0.0009) [2023-12-27 03:25:09,253][105692] Updated weights for policy 0, policy_version 1650704 (0.0010) [2023-12-27 03:25:09,336][105692] Updated weights for policy 0, policy_version 1650714 (0.0009) [2023-12-27 03:25:09,751][105620] Updated weights for policy 1, policy_version 1653896 (0.0010) [2023-12-27 03:25:09,819][105620] Updated weights for policy 1, policy_version 1653906 (0.0012) [2023-12-27 03:25:09,881][105620] Updated weights for policy 1, policy_version 1653916 (0.0008) [2023-12-27 03:25:10,095][105692] Updated weights for policy 0, policy_version 1650724 (0.0008) [2023-12-27 03:25:10,160][105692] Updated weights for policy 0, policy_version 1650734 (0.0008) [2023-12-27 03:25:10,221][105692] Updated weights for policy 0, policy_version 1650744 (0.0008) [2023-12-27 03:25:10,605][105620] Updated weights for policy 1, policy_version 1653926 (0.0009) [2023-12-27 03:25:10,661][105620] Updated weights for policy 1, policy_version 1653936 (0.0009) [2023-12-27 03:25:10,723][105620] Updated weights for policy 1, policy_version 1653946 (0.0008) [2023-12-27 03:25:10,980][105692] Updated weights for policy 0, policy_version 1650754 (0.0009) [2023-12-27 03:25:11,037][105692] Updated weights for policy 0, policy_version 1650764 (0.0008) [2023-12-27 03:25:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 846127104. Throughput: 0: 9812.2, 1: 9691.9. Samples: 846137824. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:25:11,062][104569] Avg episode reward: [(0, '8988.216'), (1, '8903.808')] [2023-12-27 03:25:11,106][105692] Updated weights for policy 0, policy_version 1650774 (0.0007) [2023-12-27 03:25:11,178][105692] Updated weights for policy 0, policy_version 1650784 (0.0009) [2023-12-27 03:25:11,526][105620] Updated weights for policy 1, policy_version 1653956 (0.0008) [2023-12-27 03:25:11,584][105620] Updated weights for policy 1, policy_version 1653966 (0.0005) [2023-12-27 03:25:11,648][105620] Updated weights for policy 1, policy_version 1653976 (0.0007) [2023-12-27 03:25:11,978][105692] Updated weights for policy 0, policy_version 1650794 (0.0009) [2023-12-27 03:25:12,045][105692] Updated weights for policy 0, policy_version 1650804 (0.0009) [2023-12-27 03:25:12,100][105692] Updated weights for policy 0, policy_version 1650814 (0.0009) [2023-12-27 03:25:12,373][105620] Updated weights for policy 1, policy_version 1653986 (0.0008) [2023-12-27 03:25:12,439][105620] Updated weights for policy 1, policy_version 1653996 (0.0009) [2023-12-27 03:25:12,500][105620] Updated weights for policy 1, policy_version 1654006 (0.0009) [2023-12-27 03:25:12,563][105620] Updated weights for policy 1, policy_version 1654016 (0.0008) [2023-12-27 03:25:12,825][105692] Updated weights for policy 0, policy_version 1650824 (0.0006) [2023-12-27 03:25:12,878][105692] Updated weights for policy 0, policy_version 1650834 (0.0007) [2023-12-27 03:25:12,927][105692] Updated weights for policy 0, policy_version 1650844 (0.0009) [2023-12-27 03:25:13,295][105620] Updated weights for policy 1, policy_version 1654026 (0.0009) [2023-12-27 03:25:13,347][105620] Updated weights for policy 1, policy_version 1654036 (0.0007) [2023-12-27 03:25:13,395][105620] Updated weights for policy 1, policy_version 1654046 (0.0005) [2023-12-27 03:25:13,695][105692] Updated weights for policy 0, policy_version 1650855 (0.0010) [2023-12-27 03:25:13,747][105692] Updated weights for policy 0, policy_version 1650865 (0.0009) [2023-12-27 03:25:13,805][105692] Updated weights for policy 0, policy_version 1650875 (0.0009) [2023-12-27 03:25:14,074][105620] Updated weights for policy 1, policy_version 1654056 (0.0008) [2023-12-27 03:25:14,142][105620] Updated weights for policy 1, policy_version 1654066 (0.0009) [2023-12-27 03:25:14,202][105620] Updated weights for policy 1, policy_version 1654076 (0.0007) [2023-12-27 03:25:14,572][105692] Updated weights for policy 0, policy_version 1650885 (0.0009) [2023-12-27 03:25:14,622][105692] Updated weights for policy 0, policy_version 1650895 (0.0009) [2023-12-27 03:25:14,681][105692] Updated weights for policy 0, policy_version 1650906 (0.0010) [2023-12-27 03:25:14,885][105620] Updated weights for policy 1, policy_version 1654086 (0.0009) [2023-12-27 03:25:14,946][105620] Updated weights for policy 1, policy_version 1654096 (0.0009) [2023-12-27 03:25:15,009][105620] Updated weights for policy 1, policy_version 1654106 (0.0009) [2023-12-27 03:25:15,043][105586] KL-divergence is very high: 129.9949 [2023-12-27 03:25:15,561][105692] Updated weights for policy 0, policy_version 1650916 (0.0010) [2023-12-27 03:25:15,610][105692] Updated weights for policy 0, policy_version 1650926 (0.0009) [2023-12-27 03:25:15,662][105692] Updated weights for policy 0, policy_version 1650936 (0.0009) [2023-12-27 03:25:15,736][105620] Updated weights for policy 1, policy_version 1654116 (0.0008) [2023-12-27 03:25:15,790][105620] Updated weights for policy 1, policy_version 1654126 (0.0008) [2023-12-27 03:25:15,837][105620] Updated weights for policy 1, policy_version 1654136 (0.0009) [2023-12-27 03:25:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 846225408. Throughput: 0: 9798.6, 1: 9618.8. Samples: 846194360. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:25:16,063][104569] Avg episode reward: [(0, '8807.770'), (1, '8987.909')] [2023-12-27 03:25:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001654144_423518208.pth... [2023-12-27 03:25:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001650944_422707200.pth... [2023-12-27 03:25:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001649824_422420480.pth [2023-12-27 03:25:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001653024_423231488.pth [2023-12-27 03:25:16,458][105692] Updated weights for policy 0, policy_version 1650946 (0.0009) [2023-12-27 03:25:16,508][105692] Updated weights for policy 0, policy_version 1650956 (0.0009) [2023-12-27 03:25:16,557][105692] Updated weights for policy 0, policy_version 1650967 (0.0009) [2023-12-27 03:25:16,595][105620] Updated weights for policy 1, policy_version 1654146 (0.0008) [2023-12-27 03:25:16,647][105620] Updated weights for policy 1, policy_version 1654156 (0.0008) [2023-12-27 03:25:16,708][105620] Updated weights for policy 1, policy_version 1654166 (0.0007) [2023-12-27 03:25:16,778][105620] Updated weights for policy 1, policy_version 1654176 (0.0009) [2023-12-27 03:25:17,365][105692] Updated weights for policy 0, policy_version 1650977 (0.0008) [2023-12-27 03:25:17,411][105620] Updated weights for policy 1, policy_version 1654186 (0.0008) [2023-12-27 03:25:17,425][105692] Updated weights for policy 0, policy_version 1650987 (0.0006) [2023-12-27 03:25:17,460][105620] Updated weights for policy 1, policy_version 1654196 (0.0006) [2023-12-27 03:25:17,486][105692] Updated weights for policy 0, policy_version 1650997 (0.0008) [2023-12-27 03:25:17,505][105620] Updated weights for policy 1, policy_version 1654206 (0.0006) [2023-12-27 03:25:17,549][105692] Updated weights for policy 0, policy_version 1651007 (0.0008) [2023-12-27 03:25:18,156][105620] Updated weights for policy 1, policy_version 1654216 (0.0008) [2023-12-27 03:25:18,216][105620] Updated weights for policy 1, policy_version 1654226 (0.0008) [2023-12-27 03:25:18,277][105620] Updated weights for policy 1, policy_version 1654236 (0.0009) [2023-12-27 03:25:18,338][105692] Updated weights for policy 0, policy_version 1651017 (0.0008) [2023-12-27 03:25:18,402][105692] Updated weights for policy 0, policy_version 1651027 (0.0009) [2023-12-27 03:25:18,466][105692] Updated weights for policy 0, policy_version 1651037 (0.0008) [2023-12-27 03:25:19,016][105620] Updated weights for policy 1, policy_version 1654246 (0.0008) [2023-12-27 03:25:19,070][105620] Updated weights for policy 1, policy_version 1654256 (0.0009) [2023-12-27 03:25:19,124][105620] Updated weights for policy 1, policy_version 1654266 (0.0009) [2023-12-27 03:25:19,233][105692] Updated weights for policy 0, policy_version 1651047 (0.0008) [2023-12-27 03:25:19,293][105692] Updated weights for policy 0, policy_version 1651057 (0.0009) [2023-12-27 03:25:19,355][105692] Updated weights for policy 0, policy_version 1651067 (0.0009) [2023-12-27 03:25:19,960][105620] Updated weights for policy 1, policy_version 1654276 (0.0009) [2023-12-27 03:25:20,015][105692] Updated weights for policy 0, policy_version 1651077 (0.0009) [2023-12-27 03:25:20,022][105620] Updated weights for policy 1, policy_version 1654286 (0.0009) [2023-12-27 03:25:20,067][105692] Updated weights for policy 0, policy_version 1651087 (0.0007) [2023-12-27 03:25:20,091][105620] Updated weights for policy 1, policy_version 1654296 (0.0008) [2023-12-27 03:25:20,116][105692] Updated weights for policy 0, policy_version 1651097 (0.0008) [2023-12-27 03:25:20,670][105620] Updated weights for policy 1, policy_version 1654306 (0.0006) [2023-12-27 03:25:20,731][105620] Updated weights for policy 1, policy_version 1654316 (0.0006) [2023-12-27 03:25:20,793][105620] Updated weights for policy 1, policy_version 1654326 (0.0006) [2023-12-27 03:25:20,853][105620] Updated weights for policy 1, policy_version 1654336 (0.0009) [2023-12-27 03:25:20,959][105692] Updated weights for policy 0, policy_version 1651107 (0.0008) [2023-12-27 03:25:21,029][105692] Updated weights for policy 0, policy_version 1651117 (0.0007) [2023-12-27 03:25:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 846315520. Throughput: 0: 9665.6, 1: 9607.0. Samples: 846306244. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:25:21,063][104569] Avg episode reward: [(0, '8538.150'), (1, '8986.717')] [2023-12-27 03:25:21,099][105692] Updated weights for policy 0, policy_version 1651127 (0.0008) [2023-12-27 03:25:21,607][105620] Updated weights for policy 1, policy_version 1654346 (0.0008) [2023-12-27 03:25:21,678][105620] Updated weights for policy 1, policy_version 1654356 (0.0008) [2023-12-27 03:25:21,747][105620] Updated weights for policy 1, policy_version 1654366 (0.0008) [2023-12-27 03:25:21,764][105692] Updated weights for policy 0, policy_version 1651137 (0.0007) [2023-12-27 03:25:21,829][105692] Updated weights for policy 0, policy_version 1651147 (0.0009) [2023-12-27 03:25:21,888][105692] Updated weights for policy 0, policy_version 1651157 (0.0009) [2023-12-27 03:25:21,944][105692] Updated weights for policy 0, policy_version 1651167 (0.0008) [2023-12-27 03:25:22,503][105620] Updated weights for policy 1, policy_version 1654376 (0.0006) [2023-12-27 03:25:22,571][105620] Updated weights for policy 1, policy_version 1654386 (0.0006) [2023-12-27 03:25:22,625][105620] Updated weights for policy 1, policy_version 1654396 (0.0007) [2023-12-27 03:25:22,657][105692] Updated weights for policy 0, policy_version 1651177 (0.0007) [2023-12-27 03:25:22,722][105692] Updated weights for policy 0, policy_version 1651187 (0.0008) [2023-12-27 03:25:22,784][105692] Updated weights for policy 0, policy_version 1651197 (0.0008) [2023-12-27 03:25:23,324][105620] Updated weights for policy 1, policy_version 1654406 (0.0009) [2023-12-27 03:25:23,384][105620] Updated weights for policy 1, policy_version 1654416 (0.0009) [2023-12-27 03:25:23,428][105692] Updated weights for policy 0, policy_version 1651207 (0.0006) [2023-12-27 03:25:23,441][105620] Updated weights for policy 1, policy_version 1654426 (0.0007) [2023-12-27 03:25:23,484][105692] Updated weights for policy 0, policy_version 1651217 (0.0005) [2023-12-27 03:25:23,534][105692] Updated weights for policy 0, policy_version 1651227 (0.0008) [2023-12-27 03:25:24,033][105620] Updated weights for policy 1, policy_version 1654436 (0.0007) [2023-12-27 03:25:24,089][105620] Updated weights for policy 1, policy_version 1654446 (0.0010) [2023-12-27 03:25:24,119][105692] Updated weights for policy 0, policy_version 1651237 (0.0007) [2023-12-27 03:25:24,148][105620] Updated weights for policy 1, policy_version 1654456 (0.0011) [2023-12-27 03:25:24,180][105692] Updated weights for policy 0, policy_version 1651247 (0.0008) [2023-12-27 03:25:24,235][105692] Updated weights for policy 0, policy_version 1651257 (0.0006) [2023-12-27 03:25:24,881][105620] Updated weights for policy 1, policy_version 1654466 (0.0011) [2023-12-27 03:25:24,934][105692] Updated weights for policy 0, policy_version 1651267 (0.0005) [2023-12-27 03:25:24,935][105620] Updated weights for policy 1, policy_version 1654476 (0.0010) [2023-12-27 03:25:24,982][105692] Updated weights for policy 0, policy_version 1651277 (0.0005) [2023-12-27 03:25:24,987][105620] Updated weights for policy 1, policy_version 1654486 (0.0010) [2023-12-27 03:25:25,033][105692] Updated weights for policy 0, policy_version 1651287 (0.0006) [2023-12-27 03:25:25,043][105620] Updated weights for policy 1, policy_version 1654496 (0.0010) [2023-12-27 03:25:25,799][105692] Updated weights for policy 0, policy_version 1651297 (0.0007) [2023-12-27 03:25:25,800][105620] Updated weights for policy 1, policy_version 1654506 (0.0010) [2023-12-27 03:25:25,850][105692] Updated weights for policy 0, policy_version 1651307 (0.0006) [2023-12-27 03:25:25,855][105620] Updated weights for policy 1, policy_version 1654516 (0.0010) [2023-12-27 03:25:25,900][105692] Updated weights for policy 0, policy_version 1651317 (0.0008) [2023-12-27 03:25:25,903][105620] Updated weights for policy 1, policy_version 1654526 (0.0010) [2023-12-27 03:25:25,956][105692] Updated weights for policy 0, policy_version 1651327 (0.0008) [2023-12-27 03:25:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 846422016. Throughput: 0: 9693.2, 1: 9626.8. Samples: 846424716. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:25:26,062][104569] Avg episode reward: [(0, '8355.293'), (1, '9171.849')] [2023-12-27 03:25:26,651][105620] Updated weights for policy 1, policy_version 1654536 (0.0010) [2023-12-27 03:25:26,699][105620] Updated weights for policy 1, policy_version 1654546 (0.0010) [2023-12-27 03:25:26,719][105692] Updated weights for policy 0, policy_version 1651337 (0.0010) [2023-12-27 03:25:26,744][105620] Updated weights for policy 1, policy_version 1654556 (0.0010) [2023-12-27 03:25:26,768][105692] Updated weights for policy 0, policy_version 1651347 (0.0010) [2023-12-27 03:25:26,819][105692] Updated weights for policy 0, policy_version 1651357 (0.0010) [2023-12-27 03:25:27,510][105620] Updated weights for policy 1, policy_version 1654566 (0.0010) [2023-12-27 03:25:27,538][105692] Updated weights for policy 0, policy_version 1651367 (0.0007) [2023-12-27 03:25:27,568][105620] Updated weights for policy 1, policy_version 1654576 (0.0010) [2023-12-27 03:25:27,588][105692] Updated weights for policy 0, policy_version 1651377 (0.0005) [2023-12-27 03:25:27,626][105620] Updated weights for policy 1, policy_version 1654586 (0.0010) [2023-12-27 03:25:27,636][105692] Updated weights for policy 0, policy_version 1651387 (0.0005) [2023-12-27 03:25:28,275][105692] Updated weights for policy 0, policy_version 1651397 (0.0007) [2023-12-27 03:25:28,337][105692] Updated weights for policy 0, policy_version 1651407 (0.0008) [2023-12-27 03:25:28,368][105620] Updated weights for policy 1, policy_version 1654596 (0.0009) [2023-12-27 03:25:28,394][105692] Updated weights for policy 0, policy_version 1651417 (0.0009) [2023-12-27 03:25:28,425][105620] Updated weights for policy 1, policy_version 1654606 (0.0010) [2023-12-27 03:25:28,477][105620] Updated weights for policy 1, policy_version 1654616 (0.0010) [2023-12-27 03:25:29,120][105692] Updated weights for policy 0, policy_version 1651427 (0.0008) [2023-12-27 03:25:29,175][105692] Updated weights for policy 0, policy_version 1651437 (0.0009) [2023-12-27 03:25:29,232][105692] Updated weights for policy 0, policy_version 1651447 (0.0009) [2023-12-27 03:25:29,244][105620] Updated weights for policy 1, policy_version 1654626 (0.0009) [2023-12-27 03:25:29,296][105620] Updated weights for policy 1, policy_version 1654636 (0.0009) [2023-12-27 03:25:29,359][105620] Updated weights for policy 1, policy_version 1654646 (0.0008) [2023-12-27 03:25:29,414][105620] Updated weights for policy 1, policy_version 1654656 (0.0009) [2023-12-27 03:25:29,996][105692] Updated weights for policy 0, policy_version 1651457 (0.0007) [2023-12-27 03:25:30,054][105692] Updated weights for policy 0, policy_version 1651467 (0.0008) [2023-12-27 03:25:30,106][105692] Updated weights for policy 0, policy_version 1651477 (0.0008) [2023-12-27 03:25:30,167][105692] Updated weights for policy 0, policy_version 1651487 (0.0008) [2023-12-27 03:25:30,203][105620] Updated weights for policy 1, policy_version 1654666 (0.0011) [2023-12-27 03:25:30,261][105620] Updated weights for policy 1, policy_version 1654676 (0.0010) [2023-12-27 03:25:30,319][105620] Updated weights for policy 1, policy_version 1654686 (0.0010) [2023-12-27 03:25:30,879][105692] Updated weights for policy 0, policy_version 1651497 (0.0008) [2023-12-27 03:25:30,934][105692] Updated weights for policy 0, policy_version 1651507 (0.0008) [2023-12-27 03:25:30,981][105692] Updated weights for policy 0, policy_version 1651517 (0.0007) [2023-12-27 03:25:31,058][105620] Updated weights for policy 1, policy_version 1654696 (0.0010) [2023-12-27 03:25:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.1, 300 sec: 19438.6). Total num frames: 846512128. Throughput: 0: 9699.7, 1: 9694.4. Samples: 846482456. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:25:31,063][104569] Avg episode reward: [(0, '8353.644'), (1, '9356.295')] [2023-12-27 03:25:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001651520_422854656.pth... [2023-12-27 03:25:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001650400_422567936.pth [2023-12-27 03:25:31,115][105620] Updated weights for policy 1, policy_version 1654706 (0.0011) [2023-12-27 03:25:31,176][105620] Updated weights for policy 1, policy_version 1654716 (0.0010) [2023-12-27 03:25:31,201][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001654720_423665664.pth... [2023-12-27 03:25:31,204][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001653568_423370752.pth [2023-12-27 03:25:31,755][105692] Updated weights for policy 0, policy_version 1651527 (0.0009) [2023-12-27 03:25:31,822][105692] Updated weights for policy 0, policy_version 1651537 (0.0009) [2023-12-27 03:25:31,834][105620] Updated weights for policy 1, policy_version 1654726 (0.0010) [2023-12-27 03:25:31,880][105692] Updated weights for policy 0, policy_version 1651547 (0.0008) [2023-12-27 03:25:31,889][105620] Updated weights for policy 1, policy_version 1654736 (0.0011) [2023-12-27 03:25:31,952][105620] Updated weights for policy 1, policy_version 1654746 (0.0009) [2023-12-27 03:25:32,643][105620] Updated weights for policy 1, policy_version 1654756 (0.0008) [2023-12-27 03:25:32,659][105692] Updated weights for policy 0, policy_version 1651557 (0.0007) [2023-12-27 03:25:32,702][105620] Updated weights for policy 1, policy_version 1654766 (0.0005) [2023-12-27 03:25:32,718][105692] Updated weights for policy 0, policy_version 1651567 (0.0009) [2023-12-27 03:25:32,772][105620] Updated weights for policy 1, policy_version 1654776 (0.0005) [2023-12-27 03:25:32,774][105692] Updated weights for policy 0, policy_version 1651577 (0.0009) [2023-12-27 03:25:33,416][105620] Updated weights for policy 1, policy_version 1654786 (0.0007) [2023-12-27 03:25:33,463][105620] Updated weights for policy 1, policy_version 1654796 (0.0010) [2023-12-27 03:25:33,518][105620] Updated weights for policy 1, policy_version 1654806 (0.0010) [2023-12-27 03:25:33,547][105692] Updated weights for policy 0, policy_version 1651587 (0.0008) [2023-12-27 03:25:33,569][105620] Updated weights for policy 1, policy_version 1654816 (0.0010) [2023-12-27 03:25:33,600][105692] Updated weights for policy 0, policy_version 1651597 (0.0007) [2023-12-27 03:25:33,650][105692] Updated weights for policy 0, policy_version 1651607 (0.0008) [2023-12-27 03:25:34,310][105620] Updated weights for policy 1, policy_version 1654826 (0.0010) [2023-12-27 03:25:34,372][105620] Updated weights for policy 1, policy_version 1654836 (0.0010) [2023-12-27 03:25:34,431][105692] Updated weights for policy 0, policy_version 1651617 (0.0007) [2023-12-27 03:25:34,435][105620] Updated weights for policy 1, policy_version 1654846 (0.0010) [2023-12-27 03:25:34,491][105692] Updated weights for policy 0, policy_version 1651627 (0.0008) [2023-12-27 03:25:34,548][105692] Updated weights for policy 0, policy_version 1651637 (0.0008) [2023-12-27 03:25:34,605][105692] Updated weights for policy 0, policy_version 1651647 (0.0008) [2023-12-27 03:25:35,173][105620] Updated weights for policy 1, policy_version 1654856 (0.0010) [2023-12-27 03:25:35,224][105620] Updated weights for policy 1, policy_version 1654866 (0.0010) [2023-12-27 03:25:35,273][105620] Updated weights for policy 1, policy_version 1654876 (0.0010) [2023-12-27 03:25:35,381][105692] Updated weights for policy 0, policy_version 1651657 (0.0008) [2023-12-27 03:25:35,440][105692] Updated weights for policy 0, policy_version 1651667 (0.0008) [2023-12-27 03:25:35,484][105692] Updated weights for policy 0, policy_version 1651677 (0.0008) [2023-12-27 03:25:36,025][105620] Updated weights for policy 1, policy_version 1654886 (0.0010) [2023-12-27 03:25:36,062][104569] Fps is (10 sec: 18021.9, 60 sec: 19251.1, 300 sec: 19410.9). Total num frames: 846602240. Throughput: 0: 9594.1, 1: 9602.7. Samples: 846596864. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:25:36,063][104569] Avg episode reward: [(0, '8267.979'), (1, '9173.037')] [2023-12-27 03:25:36,073][105620] Updated weights for policy 1, policy_version 1654896 (0.0010) [2023-12-27 03:25:36,139][105620] Updated weights for policy 1, policy_version 1654906 (0.0010) [2023-12-27 03:25:36,237][105692] Updated weights for policy 0, policy_version 1651687 (0.0008) [2023-12-27 03:25:36,297][105692] Updated weights for policy 0, policy_version 1651697 (0.0008) [2023-12-27 03:25:36,357][105692] Updated weights for policy 0, policy_version 1651707 (0.0009) [2023-12-27 03:25:36,934][105620] Updated weights for policy 1, policy_version 1654916 (0.0011) [2023-12-27 03:25:36,993][105620] Updated weights for policy 1, policy_version 1654926 (0.0011) [2023-12-27 03:25:37,049][105620] Updated weights for policy 1, policy_version 1654936 (0.0010) [2023-12-27 03:25:37,146][105692] Updated weights for policy 0, policy_version 1651717 (0.0008) [2023-12-27 03:25:37,202][105692] Updated weights for policy 0, policy_version 1651727 (0.0009) [2023-12-27 03:25:37,266][105692] Updated weights for policy 0, policy_version 1651737 (0.0008) [2023-12-27 03:25:37,814][105620] Updated weights for policy 1, policy_version 1654946 (0.0010) [2023-12-27 03:25:37,876][105620] Updated weights for policy 1, policy_version 1654956 (0.0010) [2023-12-27 03:25:37,937][105620] Updated weights for policy 1, policy_version 1654966 (0.0011) [2023-12-27 03:25:37,970][105692] Updated weights for policy 0, policy_version 1651747 (0.0009) [2023-12-27 03:25:37,986][105620] Updated weights for policy 1, policy_version 1654976 (0.0011) [2023-12-27 03:25:38,029][105692] Updated weights for policy 0, policy_version 1651757 (0.0009) [2023-12-27 03:25:38,079][105692] Updated weights for policy 0, policy_version 1651767 (0.0008) [2023-12-27 03:25:38,774][105620] Updated weights for policy 1, policy_version 1654986 (0.0009) [2023-12-27 03:25:38,827][105620] Updated weights for policy 1, policy_version 1654996 (0.0007) [2023-12-27 03:25:38,874][105620] Updated weights for policy 1, policy_version 1655006 (0.0009) [2023-12-27 03:25:38,877][105692] Updated weights for policy 0, policy_version 1651777 (0.0008) [2023-12-27 03:25:38,926][105692] Updated weights for policy 0, policy_version 1651787 (0.0008) [2023-12-27 03:25:38,972][105692] Updated weights for policy 0, policy_version 1651797 (0.0008) [2023-12-27 03:25:39,020][105692] Updated weights for policy 0, policy_version 1651807 (0.0009) [2023-12-27 03:25:39,651][105620] Updated weights for policy 1, policy_version 1655016 (0.0009) [2023-12-27 03:25:39,711][105620] Updated weights for policy 1, policy_version 1655026 (0.0009) [2023-12-27 03:25:39,762][105620] Updated weights for policy 1, policy_version 1655036 (0.0009) [2023-12-27 03:25:39,820][105692] Updated weights for policy 0, policy_version 1651817 (0.0008) [2023-12-27 03:25:39,892][105692] Updated weights for policy 0, policy_version 1651827 (0.0009) [2023-12-27 03:25:39,963][105692] Updated weights for policy 0, policy_version 1651837 (0.0009) [2023-12-27 03:25:40,554][105620] Updated weights for policy 1, policy_version 1655046 (0.0006) [2023-12-27 03:25:40,624][105620] Updated weights for policy 1, policy_version 1655056 (0.0006) [2023-12-27 03:25:40,687][105620] Updated weights for policy 1, policy_version 1655066 (0.0009) [2023-12-27 03:25:40,698][105692] Updated weights for policy 0, policy_version 1651847 (0.0008) [2023-12-27 03:25:40,758][105692] Updated weights for policy 0, policy_version 1651857 (0.0006) [2023-12-27 03:25:40,824][105692] Updated weights for policy 0, policy_version 1651867 (0.0007) [2023-12-27 03:25:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 846700544. Throughput: 0: 9555.9, 1: 9620.0. Samples: 846707752. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:25:41,063][104569] Avg episode reward: [(0, '7623.480'), (1, '9173.000')] [2023-12-27 03:25:41,396][105620] Updated weights for policy 1, policy_version 1655076 (0.0009) [2023-12-27 03:25:41,449][105620] Updated weights for policy 1, policy_version 1655086 (0.0009) [2023-12-27 03:25:41,506][105620] Updated weights for policy 1, policy_version 1655096 (0.0009) [2023-12-27 03:25:41,542][105692] Updated weights for policy 0, policy_version 1651877 (0.0008) [2023-12-27 03:25:41,594][105692] Updated weights for policy 0, policy_version 1651887 (0.0008) [2023-12-27 03:25:41,665][105692] Updated weights for policy 0, policy_version 1651897 (0.0009) [2023-12-27 03:25:42,297][105620] Updated weights for policy 1, policy_version 1655106 (0.0008) [2023-12-27 03:25:42,360][105620] Updated weights for policy 1, policy_version 1655116 (0.0009) [2023-12-27 03:25:42,392][105692] Updated weights for policy 0, policy_version 1651907 (0.0008) [2023-12-27 03:25:42,425][105620] Updated weights for policy 1, policy_version 1655126 (0.0010) [2023-12-27 03:25:42,448][105692] Updated weights for policy 0, policy_version 1651917 (0.0008) [2023-12-27 03:25:42,479][105620] Updated weights for policy 1, policy_version 1655136 (0.0006) [2023-12-27 03:25:42,510][105692] Updated weights for policy 0, policy_version 1651927 (0.0009) [2023-12-27 03:25:43,118][105692] Updated weights for policy 0, policy_version 1651937 (0.0007) [2023-12-27 03:25:43,178][105692] Updated weights for policy 0, policy_version 1651947 (0.0011) [2023-12-27 03:25:43,213][105620] Updated weights for policy 1, policy_version 1655146 (0.0011) [2023-12-27 03:25:43,238][105692] Updated weights for policy 0, policy_version 1651957 (0.0011) [2023-12-27 03:25:43,246][105586] KL-divergence is very high: 110.3793 [2023-12-27 03:25:43,269][105620] Updated weights for policy 1, policy_version 1655156 (0.0011) [2023-12-27 03:25:43,288][105692] Updated weights for policy 0, policy_version 1651967 (0.0007) [2023-12-27 03:25:43,292][105586] KL-divergence is very high: 108.4594 [2023-12-27 03:25:43,325][105620] Updated weights for policy 1, policy_version 1655166 (0.0011) [2023-12-27 03:25:43,912][105692] Updated weights for policy 0, policy_version 1651977 (0.0007) [2023-12-27 03:25:43,972][105692] Updated weights for policy 0, policy_version 1651987 (0.0007) [2023-12-27 03:25:44,026][105692] Updated weights for policy 0, policy_version 1651997 (0.0008) [2023-12-27 03:25:44,098][105620] Updated weights for policy 1, policy_version 1655176 (0.0010) [2023-12-27 03:25:44,159][105620] Updated weights for policy 1, policy_version 1655186 (0.0010) [2023-12-27 03:25:44,231][105620] Updated weights for policy 1, policy_version 1655196 (0.0010) [2023-12-27 03:25:44,724][105692] Updated weights for policy 0, policy_version 1652007 (0.0008) [2023-12-27 03:25:44,772][105692] Updated weights for policy 0, policy_version 1652017 (0.0008) [2023-12-27 03:25:44,831][105692] Updated weights for policy 0, policy_version 1652027 (0.0008) [2023-12-27 03:25:44,942][105620] Updated weights for policy 1, policy_version 1655206 (0.0008) [2023-12-27 03:25:45,004][105620] Updated weights for policy 1, policy_version 1655216 (0.0008) [2023-12-27 03:25:45,071][105620] Updated weights for policy 1, policy_version 1655226 (0.0011) [2023-12-27 03:25:45,669][105620] Updated weights for policy 1, policy_version 1655236 (0.0009) [2023-12-27 03:25:45,689][105692] Updated weights for policy 0, policy_version 1652037 (0.0008) [2023-12-27 03:25:45,730][105620] Updated weights for policy 1, policy_version 1655246 (0.0010) [2023-12-27 03:25:45,748][105692] Updated weights for policy 0, policy_version 1652047 (0.0007) [2023-12-27 03:25:45,791][105620] Updated weights for policy 1, policy_version 1655256 (0.0010) [2023-12-27 03:25:45,807][105692] Updated weights for policy 0, policy_version 1652057 (0.0009) [2023-12-27 03:25:46,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 846798848. Throughput: 0: 9610.4, 1: 9616.4. Samples: 846765864. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:25:46,062][104569] Avg episode reward: [(0, '8442.248'), (1, '9263.616')] [2023-12-27 03:25:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001652064_422993920.pth... [2023-12-27 03:25:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001655264_423804928.pth... [2023-12-27 03:25:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001650944_422707200.pth [2023-12-27 03:25:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001654144_423518208.pth [2023-12-27 03:25:46,400][105620] Updated weights for policy 1, policy_version 1655266 (0.0009) [2023-12-27 03:25:46,458][105620] Updated weights for policy 1, policy_version 1655276 (0.0005) [2023-12-27 03:25:46,516][105620] Updated weights for policy 1, policy_version 1655286 (0.0005) [2023-12-27 03:25:46,570][105620] Updated weights for policy 1, policy_version 1655296 (0.0005) [2023-12-27 03:25:46,647][105692] Updated weights for policy 0, policy_version 1652067 (0.0010) [2023-12-27 03:25:46,717][105692] Updated weights for policy 0, policy_version 1652077 (0.0009) [2023-12-27 03:25:46,774][105692] Updated weights for policy 0, policy_version 1652088 (0.0010) [2023-12-27 03:25:47,073][105620] Updated weights for policy 1, policy_version 1655306 (0.0005) [2023-12-27 03:25:47,119][105620] Updated weights for policy 1, policy_version 1655316 (0.0006) [2023-12-27 03:25:47,169][105620] Updated weights for policy 1, policy_version 1655326 (0.0006) [2023-12-27 03:25:47,694][105620] Updated weights for policy 1, policy_version 1655336 (0.0005) [2023-12-27 03:25:47,701][105692] Updated weights for policy 0, policy_version 1652099 (0.0010) [2023-12-27 03:25:47,748][105620] Updated weights for policy 1, policy_version 1655346 (0.0006) [2023-12-27 03:25:47,764][105692] Updated weights for policy 0, policy_version 1652109 (0.0008) [2023-12-27 03:25:47,799][105620] Updated weights for policy 1, policy_version 1655356 (0.0006) [2023-12-27 03:25:47,833][105692] Updated weights for policy 0, policy_version 1652119 (0.0007) [2023-12-27 03:25:48,352][105620] Updated weights for policy 1, policy_version 1655366 (0.0009) [2023-12-27 03:25:48,412][105620] Updated weights for policy 1, policy_version 1655376 (0.0007) [2023-12-27 03:25:48,474][105620] Updated weights for policy 1, policy_version 1655386 (0.0010) [2023-12-27 03:25:48,669][105692] Updated weights for policy 0, policy_version 1652129 (0.0008) [2023-12-27 03:25:48,732][105692] Updated weights for policy 0, policy_version 1652139 (0.0010) [2023-12-27 03:25:48,794][105692] Updated weights for policy 0, policy_version 1652149 (0.0006) [2023-12-27 03:25:48,849][105692] Updated weights for policy 0, policy_version 1652159 (0.0007) [2023-12-27 03:25:49,128][105620] Updated weights for policy 1, policy_version 1655396 (0.0009) [2023-12-27 03:25:49,184][105620] Updated weights for policy 1, policy_version 1655406 (0.0006) [2023-12-27 03:25:49,257][105620] Updated weights for policy 1, policy_version 1655416 (0.0011) [2023-12-27 03:25:49,519][105692] Updated weights for policy 0, policy_version 1652169 (0.0006) [2023-12-27 03:25:49,582][105692] Updated weights for policy 0, policy_version 1652179 (0.0006) [2023-12-27 03:25:49,648][105692] Updated weights for policy 0, policy_version 1652189 (0.0010) [2023-12-27 03:25:49,912][105620] Updated weights for policy 1, policy_version 1655426 (0.0010) [2023-12-27 03:25:49,975][105620] Updated weights for policy 1, policy_version 1655436 (0.0008) [2023-12-27 03:25:50,045][105620] Updated weights for policy 1, policy_version 1655446 (0.0008) [2023-12-27 03:25:50,114][105620] Updated weights for policy 1, policy_version 1655456 (0.0008) [2023-12-27 03:25:50,272][105692] Updated weights for policy 0, policy_version 1652199 (0.0010) [2023-12-27 03:25:50,336][105692] Updated weights for policy 0, policy_version 1652209 (0.0009) [2023-12-27 03:25:50,387][105692] Updated weights for policy 0, policy_version 1652219 (0.0009) [2023-12-27 03:25:50,823][105620] Updated weights for policy 1, policy_version 1655466 (0.0010) [2023-12-27 03:25:50,886][105620] Updated weights for policy 1, policy_version 1655476 (0.0010) [2023-12-27 03:25:50,949][105620] Updated weights for policy 1, policy_version 1655487 (0.0010) [2023-12-27 03:25:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 846897152. Throughput: 0: 9420.4, 1: 9793.5. Samples: 846885948. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:25:51,062][104569] Avg episode reward: [(0, '8900.258'), (1, '8896.854')] [2023-12-27 03:25:51,176][105692] Updated weights for policy 0, policy_version 1652229 (0.0009) [2023-12-27 03:25:51,227][105692] Updated weights for policy 0, policy_version 1652239 (0.0009) [2023-12-27 03:25:51,281][105692] Updated weights for policy 0, policy_version 1652249 (0.0008) [2023-12-27 03:25:51,727][105620] Updated weights for policy 1, policy_version 1655497 (0.0008) [2023-12-27 03:25:51,790][105620] Updated weights for policy 1, policy_version 1655507 (0.0008) [2023-12-27 03:25:51,840][105620] Updated weights for policy 1, policy_version 1655517 (0.0009) [2023-12-27 03:25:52,049][105692] Updated weights for policy 0, policy_version 1652259 (0.0007) [2023-12-27 03:25:52,102][105692] Updated weights for policy 0, policy_version 1652269 (0.0009) [2023-12-27 03:25:52,164][105692] Updated weights for policy 0, policy_version 1652279 (0.0009) [2023-12-27 03:25:52,608][105620] Updated weights for policy 1, policy_version 1655527 (0.0010) [2023-12-27 03:25:52,666][105620] Updated weights for policy 1, policy_version 1655537 (0.0010) [2023-12-27 03:25:52,729][105620] Updated weights for policy 1, policy_version 1655547 (0.0011) [2023-12-27 03:25:52,931][105692] Updated weights for policy 0, policy_version 1652289 (0.0009) [2023-12-27 03:25:52,982][105692] Updated weights for policy 0, policy_version 1652299 (0.0007) [2023-12-27 03:25:53,031][105692] Updated weights for policy 0, policy_version 1652309 (0.0008) [2023-12-27 03:25:53,075][105692] Updated weights for policy 0, policy_version 1652319 (0.0008) [2023-12-27 03:25:53,456][105620] Updated weights for policy 1, policy_version 1655557 (0.0009) [2023-12-27 03:25:53,519][105620] Updated weights for policy 1, policy_version 1655567 (0.0006) [2023-12-27 03:25:53,563][105620] Updated weights for policy 1, policy_version 1655577 (0.0010) [2023-12-27 03:25:53,851][105692] Updated weights for policy 0, policy_version 1652329 (0.0007) [2023-12-27 03:25:53,897][105692] Updated weights for policy 0, policy_version 1652339 (0.0008) [2023-12-27 03:25:53,944][105692] Updated weights for policy 0, policy_version 1652349 (0.0008) [2023-12-27 03:25:54,304][105620] Updated weights for policy 1, policy_version 1655587 (0.0010) [2023-12-27 03:25:54,372][105620] Updated weights for policy 1, policy_version 1655597 (0.0010) [2023-12-27 03:25:54,419][105620] Updated weights for policy 1, policy_version 1655607 (0.0010) [2023-12-27 03:25:54,627][105692] Updated weights for policy 0, policy_version 1652359 (0.0008) [2023-12-27 03:25:54,686][105692] Updated weights for policy 0, policy_version 1652369 (0.0009) [2023-12-27 03:25:54,740][105692] Updated weights for policy 0, policy_version 1652379 (0.0006) [2023-12-27 03:25:55,095][105620] Updated weights for policy 1, policy_version 1655617 (0.0010) [2023-12-27 03:25:55,153][105620] Updated weights for policy 1, policy_version 1655627 (0.0010) [2023-12-27 03:25:55,208][105620] Updated weights for policy 1, policy_version 1655637 (0.0010) [2023-12-27 03:25:55,262][105620] Updated weights for policy 1, policy_version 1655647 (0.0010) [2023-12-27 03:25:55,364][105692] Updated weights for policy 0, policy_version 1652389 (0.0005) [2023-12-27 03:25:55,407][105692] Updated weights for policy 0, policy_version 1652399 (0.0005) [2023-12-27 03:25:55,457][105692] Updated weights for policy 0, policy_version 1652409 (0.0005) [2023-12-27 03:25:55,940][105620] Updated weights for policy 1, policy_version 1655657 (0.0007) [2023-12-27 03:25:55,994][105620] Updated weights for policy 1, policy_version 1655667 (0.0009) [2023-12-27 03:25:56,012][105692] Updated weights for policy 0, policy_version 1652419 (0.0006) [2023-12-27 03:25:56,039][105620] Updated weights for policy 1, policy_version 1655677 (0.0010) [2023-12-27 03:25:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 846995456. Throughput: 0: 9438.2, 1: 9772.7. Samples: 847002316. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:25:56,062][104569] Avg episode reward: [(0, '8896.660'), (1, '8805.814')] [2023-12-27 03:25:56,068][105692] Updated weights for policy 0, policy_version 1652429 (0.0010) [2023-12-27 03:25:56,129][105692] Updated weights for policy 0, policy_version 1652439 (0.0010) [2023-12-27 03:25:56,701][105620] Updated weights for policy 1, policy_version 1655687 (0.0007) [2023-12-27 03:25:56,747][105620] Updated weights for policy 1, policy_version 1655697 (0.0005) [2023-12-27 03:25:56,801][105620] Updated weights for policy 1, policy_version 1655707 (0.0005) [2023-12-27 03:25:56,878][105692] Updated weights for policy 0, policy_version 1652449 (0.0010) [2023-12-27 03:25:56,929][105692] Updated weights for policy 0, policy_version 1652459 (0.0008) [2023-12-27 03:25:56,990][105692] Updated weights for policy 0, policy_version 1652469 (0.0008) [2023-12-27 03:25:57,053][105692] Updated weights for policy 0, policy_version 1652479 (0.0005) [2023-12-27 03:25:57,578][105692] Updated weights for policy 0, policy_version 1652489 (0.0005) [2023-12-27 03:25:57,587][105620] Updated weights for policy 1, policy_version 1655717 (0.0008) [2023-12-27 03:25:57,633][105692] Updated weights for policy 0, policy_version 1652499 (0.0006) [2023-12-27 03:25:57,639][105620] Updated weights for policy 1, policy_version 1655727 (0.0007) [2023-12-27 03:25:57,684][105692] Updated weights for policy 0, policy_version 1652509 (0.0010) [2023-12-27 03:25:57,694][105620] Updated weights for policy 1, policy_version 1655737 (0.0006) [2023-12-27 03:25:58,398][105692] Updated weights for policy 0, policy_version 1652519 (0.0008) [2023-12-27 03:25:58,450][105620] Updated weights for policy 1, policy_version 1655747 (0.0009) [2023-12-27 03:25:58,461][105692] Updated weights for policy 0, policy_version 1652529 (0.0008) [2023-12-27 03:25:58,512][105620] Updated weights for policy 1, policy_version 1655757 (0.0009) [2023-12-27 03:25:58,523][105692] Updated weights for policy 0, policy_version 1652539 (0.0008) [2023-12-27 03:25:58,578][105620] Updated weights for policy 1, policy_version 1655767 (0.0008) [2023-12-27 03:25:59,303][105692] Updated weights for policy 0, policy_version 1652549 (0.0010) [2023-12-27 03:25:59,367][105692] Updated weights for policy 0, policy_version 1652559 (0.0009) [2023-12-27 03:25:59,421][105692] Updated weights for policy 0, policy_version 1652569 (0.0007) [2023-12-27 03:25:59,427][105620] Updated weights for policy 1, policy_version 1655777 (0.0008) [2023-12-27 03:25:59,486][105620] Updated weights for policy 1, policy_version 1655787 (0.0006) [2023-12-27 03:25:59,543][105620] Updated weights for policy 1, policy_version 1655797 (0.0006) [2023-12-27 03:25:59,604][105620] Updated weights for policy 1, policy_version 1655807 (0.0006) [2023-12-27 03:26:00,064][105692] Updated weights for policy 0, policy_version 1652579 (0.0006) [2023-12-27 03:26:00,138][105692] Updated weights for policy 0, policy_version 1652589 (0.0006) [2023-12-27 03:26:00,212][105692] Updated weights for policy 0, policy_version 1652599 (0.0006) [2023-12-27 03:26:00,247][105620] Updated weights for policy 1, policy_version 1655817 (0.0008) [2023-12-27 03:26:00,315][105620] Updated weights for policy 1, policy_version 1655827 (0.0006) [2023-12-27 03:26:00,387][105620] Updated weights for policy 1, policy_version 1655837 (0.0006) [2023-12-27 03:26:00,723][105692] Updated weights for policy 0, policy_version 1652609 (0.0005) [2023-12-27 03:26:00,782][105692] Updated weights for policy 0, policy_version 1652619 (0.0005) [2023-12-27 03:26:00,853][105692] Updated weights for policy 0, policy_version 1652629 (0.0006) [2023-12-27 03:26:00,908][105692] Updated weights for policy 0, policy_version 1652639 (0.0010) [2023-12-27 03:26:00,943][105620] Updated weights for policy 1, policy_version 1655847 (0.0007) [2023-12-27 03:26:01,000][105620] Updated weights for policy 1, policy_version 1655857 (0.0006) [2023-12-27 03:26:01,056][105620] Updated weights for policy 1, policy_version 1655867 (0.0009) [2023-12-27 03:26:01,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 847093760. Throughput: 0: 9533.3, 1: 9745.0. Samples: 847061884. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:26:01,063][104569] Avg episode reward: [(0, '8803.508'), (1, '8897.740')] [2023-12-27 03:26:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001652640_423141376.pth... [2023-12-27 03:26:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001651520_422854656.pth [2023-12-27 03:26:01,082][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001655872_423960576.pth... [2023-12-27 03:26:01,085][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001654720_423665664.pth [2023-12-27 03:26:01,571][105692] Updated weights for policy 0, policy_version 1652649 (0.0009) [2023-12-27 03:26:01,627][105692] Updated weights for policy 0, policy_version 1652659 (0.0011) [2023-12-27 03:26:01,693][105692] Updated weights for policy 0, policy_version 1652669 (0.0011) [2023-12-27 03:26:01,846][105620] Updated weights for policy 1, policy_version 1655877 (0.0008) [2023-12-27 03:26:01,894][105620] Updated weights for policy 1, policy_version 1655887 (0.0008) [2023-12-27 03:26:01,946][105620] Updated weights for policy 1, policy_version 1655897 (0.0008) [2023-12-27 03:26:02,428][105692] Updated weights for policy 0, policy_version 1652679 (0.0011) [2023-12-27 03:26:02,493][105692] Updated weights for policy 0, policy_version 1652689 (0.0010) [2023-12-27 03:26:02,547][105692] Updated weights for policy 0, policy_version 1652699 (0.0005) [2023-12-27 03:26:02,650][105620] Updated weights for policy 1, policy_version 1655907 (0.0008) [2023-12-27 03:26:02,712][105620] Updated weights for policy 1, policy_version 1655917 (0.0010) [2023-12-27 03:26:02,778][105620] Updated weights for policy 1, policy_version 1655927 (0.0009) [2023-12-27 03:26:03,100][105692] Updated weights for policy 0, policy_version 1652709 (0.0007) [2023-12-27 03:26:03,161][105692] Updated weights for policy 0, policy_version 1652719 (0.0010) [2023-12-27 03:26:03,224][105692] Updated weights for policy 0, policy_version 1652729 (0.0007) [2023-12-27 03:26:03,425][105620] Updated weights for policy 1, policy_version 1655937 (0.0010) [2023-12-27 03:26:03,480][105620] Updated weights for policy 1, policy_version 1655947 (0.0008) [2023-12-27 03:26:03,536][105620] Updated weights for policy 1, policy_version 1655957 (0.0008) [2023-12-27 03:26:03,583][105620] Updated weights for policy 1, policy_version 1655967 (0.0009) [2023-12-27 03:26:03,831][105692] Updated weights for policy 0, policy_version 1652739 (0.0008) [2023-12-27 03:26:03,896][105692] Updated weights for policy 0, policy_version 1652749 (0.0008) [2023-12-27 03:26:03,962][105692] Updated weights for policy 0, policy_version 1652759 (0.0006) [2023-12-27 03:26:04,372][105620] Updated weights for policy 1, policy_version 1655977 (0.0006) [2023-12-27 03:26:04,434][105620] Updated weights for policy 1, policy_version 1655987 (0.0009) [2023-12-27 03:26:04,487][105620] Updated weights for policy 1, policy_version 1655997 (0.0008) [2023-12-27 03:26:04,713][105692] Updated weights for policy 0, policy_version 1652769 (0.0010) [2023-12-27 03:26:04,775][105692] Updated weights for policy 0, policy_version 1652779 (0.0010) [2023-12-27 03:26:04,840][105692] Updated weights for policy 0, policy_version 1652789 (0.0010) [2023-12-27 03:26:04,897][105692] Updated weights for policy 0, policy_version 1652799 (0.0010) [2023-12-27 03:26:05,180][105620] Updated weights for policy 1, policy_version 1656007 (0.0007) [2023-12-27 03:26:05,227][105620] Updated weights for policy 1, policy_version 1656017 (0.0005) [2023-12-27 03:26:05,284][105620] Updated weights for policy 1, policy_version 1656027 (0.0005) [2023-12-27 03:26:05,593][105692] Updated weights for policy 0, policy_version 1652809 (0.0010) [2023-12-27 03:26:05,641][105692] Updated weights for policy 0, policy_version 1652819 (0.0010) [2023-12-27 03:26:05,697][105692] Updated weights for policy 0, policy_version 1652829 (0.0007) [2023-12-27 03:26:05,954][105620] Updated weights for policy 1, policy_version 1656037 (0.0008) [2023-12-27 03:26:06,013][105620] Updated weights for policy 1, policy_version 1656047 (0.0010) [2023-12-27 03:26:06,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 847192064. Throughput: 0: 9727.1, 1: 9767.8. Samples: 847183516. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:26:06,063][104569] Avg episode reward: [(0, '8808.817'), (1, '8804.816')] [2023-12-27 03:26:06,072][105620] Updated weights for policy 1, policy_version 1656057 (0.0010) [2023-12-27 03:26:06,335][105692] Updated weights for policy 0, policy_version 1652839 (0.0007) [2023-12-27 03:26:06,399][105692] Updated weights for policy 0, policy_version 1652849 (0.0007) [2023-12-27 03:26:06,467][105692] Updated weights for policy 0, policy_version 1652859 (0.0006) [2023-12-27 03:26:06,787][105620] Updated weights for policy 1, policy_version 1656067 (0.0009) [2023-12-27 03:26:06,844][105620] Updated weights for policy 1, policy_version 1656077 (0.0005) [2023-12-27 03:26:06,908][105620] Updated weights for policy 1, policy_version 1656087 (0.0005) [2023-12-27 03:26:07,164][105692] Updated weights for policy 0, policy_version 1652869 (0.0009) [2023-12-27 03:26:07,224][105692] Updated weights for policy 0, policy_version 1652879 (0.0011) [2023-12-27 03:26:07,286][105692] Updated weights for policy 0, policy_version 1652889 (0.0011) [2023-12-27 03:26:07,554][105620] Updated weights for policy 1, policy_version 1656097 (0.0006) [2023-12-27 03:26:07,618][105620] Updated weights for policy 1, policy_version 1656107 (0.0011) [2023-12-27 03:26:07,680][105620] Updated weights for policy 1, policy_version 1656117 (0.0010) [2023-12-27 03:26:07,736][105620] Updated weights for policy 1, policy_version 1656127 (0.0010) [2023-12-27 03:26:08,037][105692] Updated weights for policy 0, policy_version 1652899 (0.0011) [2023-12-27 03:26:08,098][105692] Updated weights for policy 0, policy_version 1652909 (0.0007) [2023-12-27 03:26:08,152][105692] Updated weights for policy 0, policy_version 1652919 (0.0010) [2023-12-27 03:26:08,488][105620] Updated weights for policy 1, policy_version 1656137 (0.0011) [2023-12-27 03:26:08,550][105620] Updated weights for policy 1, policy_version 1656147 (0.0011) [2023-12-27 03:26:08,616][105620] Updated weights for policy 1, policy_version 1656157 (0.0011) [2023-12-27 03:26:08,847][105692] Updated weights for policy 0, policy_version 1652929 (0.0010) [2023-12-27 03:26:08,900][105692] Updated weights for policy 0, policy_version 1652939 (0.0010) [2023-12-27 03:26:08,950][105692] Updated weights for policy 0, policy_version 1652949 (0.0011) [2023-12-27 03:26:09,006][105692] Updated weights for policy 0, policy_version 1652959 (0.0011) [2023-12-27 03:26:09,378][105620] Updated weights for policy 1, policy_version 1656167 (0.0009) [2023-12-27 03:26:09,445][105620] Updated weights for policy 1, policy_version 1656177 (0.0008) [2023-12-27 03:26:09,513][105620] Updated weights for policy 1, policy_version 1656187 (0.0006) [2023-12-27 03:26:09,820][105692] Updated weights for policy 0, policy_version 1652969 (0.0011) [2023-12-27 03:26:09,891][105692] Updated weights for policy 0, policy_version 1652979 (0.0010) [2023-12-27 03:26:09,956][105692] Updated weights for policy 0, policy_version 1652989 (0.0010) [2023-12-27 03:26:10,186][105620] Updated weights for policy 1, policy_version 1656197 (0.0006) [2023-12-27 03:26:10,238][105620] Updated weights for policy 1, policy_version 1656207 (0.0009) [2023-12-27 03:26:10,298][105620] Updated weights for policy 1, policy_version 1656217 (0.0011) [2023-12-27 03:26:10,724][105692] Updated weights for policy 0, policy_version 1652999 (0.0008) [2023-12-27 03:26:10,773][105692] Updated weights for policy 0, policy_version 1653009 (0.0008) [2023-12-27 03:26:10,818][105692] Updated weights for policy 0, policy_version 1653019 (0.0008) [2023-12-27 03:26:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 847290368. Throughput: 0: 9673.0, 1: 9781.3. Samples: 847300164. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:26:11,063][104569] Avg episode reward: [(0, '8993.067'), (1, '8988.296')] [2023-12-27 03:26:11,068][105620] Updated weights for policy 1, policy_version 1656227 (0.0011) [2023-12-27 03:26:11,132][105620] Updated weights for policy 1, policy_version 1656237 (0.0011) [2023-12-27 03:26:11,197][105620] Updated weights for policy 1, policy_version 1656247 (0.0011) [2023-12-27 03:26:11,612][105692] Updated weights for policy 0, policy_version 1653029 (0.0008) [2023-12-27 03:26:11,673][105692] Updated weights for policy 0, policy_version 1653039 (0.0009) [2023-12-27 03:26:11,734][105692] Updated weights for policy 0, policy_version 1653049 (0.0008) [2023-12-27 03:26:11,970][105620] Updated weights for policy 1, policy_version 1656257 (0.0011) [2023-12-27 03:26:12,023][105620] Updated weights for policy 1, policy_version 1656267 (0.0011) [2023-12-27 03:26:12,087][105620] Updated weights for policy 1, policy_version 1656277 (0.0011) [2023-12-27 03:26:12,151][105620] Updated weights for policy 1, policy_version 1656287 (0.0011) [2023-12-27 03:26:12,502][105692] Updated weights for policy 0, policy_version 1653059 (0.0009) [2023-12-27 03:26:12,553][105692] Updated weights for policy 0, policy_version 1653069 (0.0009) [2023-12-27 03:26:12,604][105692] Updated weights for policy 0, policy_version 1653079 (0.0008) [2023-12-27 03:26:12,922][105620] Updated weights for policy 1, policy_version 1656297 (0.0010) [2023-12-27 03:26:12,981][105620] Updated weights for policy 1, policy_version 1656307 (0.0010) [2023-12-27 03:26:13,039][105620] Updated weights for policy 1, policy_version 1656317 (0.0010) [2023-12-27 03:26:13,369][105692] Updated weights for policy 0, policy_version 1653089 (0.0010) [2023-12-27 03:26:13,428][105692] Updated weights for policy 0, policy_version 1653099 (0.0010) [2023-12-27 03:26:13,490][105692] Updated weights for policy 0, policy_version 1653109 (0.0010) [2023-12-27 03:26:13,535][105692] Updated weights for policy 0, policy_version 1653119 (0.0010) [2023-12-27 03:26:13,778][105620] Updated weights for policy 1, policy_version 1656327 (0.0009) [2023-12-27 03:26:13,846][105620] Updated weights for policy 1, policy_version 1656337 (0.0008) [2023-12-27 03:26:13,916][105620] Updated weights for policy 1, policy_version 1656347 (0.0007) [2023-12-27 03:26:14,252][105692] Updated weights for policy 0, policy_version 1653129 (0.0006) [2023-12-27 03:26:14,311][105692] Updated weights for policy 0, policy_version 1653139 (0.0009) [2023-12-27 03:26:14,361][105692] Updated weights for policy 0, policy_version 1653149 (0.0008) [2023-12-27 03:26:14,557][105620] Updated weights for policy 1, policy_version 1656357 (0.0008) [2023-12-27 03:26:14,605][105620] Updated weights for policy 1, policy_version 1656367 (0.0010) [2023-12-27 03:26:14,650][105620] Updated weights for policy 1, policy_version 1656377 (0.0008) [2023-12-27 03:26:15,021][105692] Updated weights for policy 0, policy_version 1653159 (0.0008) [2023-12-27 03:26:15,084][105692] Updated weights for policy 0, policy_version 1653169 (0.0009) [2023-12-27 03:26:15,150][105692] Updated weights for policy 0, policy_version 1653179 (0.0009) [2023-12-27 03:26:15,450][105620] Updated weights for policy 1, policy_version 1656387 (0.0009) [2023-12-27 03:26:15,511][105620] Updated weights for policy 1, policy_version 1656397 (0.0008) [2023-12-27 03:26:15,578][105620] Updated weights for policy 1, policy_version 1656407 (0.0009) [2023-12-27 03:26:15,831][105692] Updated weights for policy 0, policy_version 1653189 (0.0009) [2023-12-27 03:26:15,899][105692] Updated weights for policy 0, policy_version 1653199 (0.0008) [2023-12-27 03:26:15,962][105692] Updated weights for policy 0, policy_version 1653209 (0.0008) [2023-12-27 03:26:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.6, 300 sec: 19410.9). Total num frames: 847388672. Throughput: 0: 9621.9, 1: 9754.8. Samples: 847354412. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:26:16,064][104569] Avg episode reward: [(0, '8899.070'), (1, '9172.363')] [2023-12-27 03:26:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001653216_423288832.pth... [2023-12-27 03:26:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001656416_424099840.pth... [2023-12-27 03:26:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001655264_423804928.pth [2023-12-27 03:26:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001652064_422993920.pth [2023-12-27 03:26:16,339][105620] Updated weights for policy 1, policy_version 1656417 (0.0009) [2023-12-27 03:26:16,389][105620] Updated weights for policy 1, policy_version 1656427 (0.0005) [2023-12-27 03:26:16,437][105620] Updated weights for policy 1, policy_version 1656437 (0.0005) [2023-12-27 03:26:16,483][105620] Updated weights for policy 1, policy_version 1656447 (0.0005) [2023-12-27 03:26:16,676][105692] Updated weights for policy 0, policy_version 1653219 (0.0009) [2023-12-27 03:26:16,730][105692] Updated weights for policy 0, policy_version 1653229 (0.0010) [2023-12-27 03:26:16,784][105692] Updated weights for policy 0, policy_version 1653239 (0.0010) [2023-12-27 03:26:17,046][105620] Updated weights for policy 1, policy_version 1656457 (0.0008) [2023-12-27 03:26:17,092][105620] Updated weights for policy 1, policy_version 1656467 (0.0006) [2023-12-27 03:26:17,143][105620] Updated weights for policy 1, policy_version 1656477 (0.0005) [2023-12-27 03:26:17,553][105692] Updated weights for policy 0, policy_version 1653249 (0.0010) [2023-12-27 03:26:17,612][105692] Updated weights for policy 0, policy_version 1653259 (0.0007) [2023-12-27 03:26:17,673][105692] Updated weights for policy 0, policy_version 1653269 (0.0005) [2023-12-27 03:26:17,745][105692] Updated weights for policy 0, policy_version 1653279 (0.0008) [2023-12-27 03:26:17,771][105620] Updated weights for policy 1, policy_version 1656487 (0.0006) [2023-12-27 03:26:17,824][105620] Updated weights for policy 1, policy_version 1656497 (0.0006) [2023-12-27 03:26:17,886][105620] Updated weights for policy 1, policy_version 1656507 (0.0005) [2023-12-27 03:26:18,265][105692] Updated weights for policy 0, policy_version 1653289 (0.0006) [2023-12-27 03:26:18,317][105692] Updated weights for policy 0, policy_version 1653299 (0.0006) [2023-12-27 03:26:18,383][105692] Updated weights for policy 0, policy_version 1653309 (0.0011) [2023-12-27 03:26:18,469][105620] Updated weights for policy 1, policy_version 1656517 (0.0008) [2023-12-27 03:26:18,531][105620] Updated weights for policy 1, policy_version 1656527 (0.0010) [2023-12-27 03:26:18,593][105620] Updated weights for policy 1, policy_version 1656537 (0.0010) [2023-12-27 03:26:19,052][105692] Updated weights for policy 0, policy_version 1653319 (0.0007) [2023-12-27 03:26:19,114][105692] Updated weights for policy 0, policy_version 1653329 (0.0006) [2023-12-27 03:26:19,173][105692] Updated weights for policy 0, policy_version 1653339 (0.0010) [2023-12-27 03:26:19,397][105620] Updated weights for policy 1, policy_version 1656547 (0.0010) [2023-12-27 03:26:19,466][105620] Updated weights for policy 1, policy_version 1656557 (0.0010) [2023-12-27 03:26:19,538][105620] Updated weights for policy 1, policy_version 1656567 (0.0009) [2023-12-27 03:26:19,865][105692] Updated weights for policy 0, policy_version 1653349 (0.0010) [2023-12-27 03:26:19,930][105692] Updated weights for policy 0, policy_version 1653359 (0.0009) [2023-12-27 03:26:19,996][105692] Updated weights for policy 0, policy_version 1653369 (0.0008) [2023-12-27 03:26:20,315][105620] Updated weights for policy 1, policy_version 1656577 (0.0009) [2023-12-27 03:26:20,375][105620] Updated weights for policy 1, policy_version 1656587 (0.0011) [2023-12-27 03:26:20,425][105620] Updated weights for policy 1, policy_version 1656597 (0.0006) [2023-12-27 03:26:20,480][105620] Updated weights for policy 1, policy_version 1656607 (0.0005) [2023-12-27 03:26:20,764][105692] Updated weights for policy 0, policy_version 1653379 (0.0007) [2023-12-27 03:26:20,822][105692] Updated weights for policy 0, policy_version 1653389 (0.0006) [2023-12-27 03:26:20,889][105692] Updated weights for policy 0, policy_version 1653399 (0.0006) [2023-12-27 03:26:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 847486976. Throughput: 0: 9734.4, 1: 9801.0. Samples: 847475956. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:26:21,062][104569] Avg episode reward: [(0, '8440.958'), (1, '9172.380')] [2023-12-27 03:26:21,194][105620] Updated weights for policy 1, policy_version 1656617 (0.0010) [2023-12-27 03:26:21,255][105620] Updated weights for policy 1, policy_version 1656627 (0.0009) [2023-12-27 03:26:21,306][105620] Updated weights for policy 1, policy_version 1656637 (0.0007) [2023-12-27 03:26:21,637][105692] Updated weights for policy 0, policy_version 1653409 (0.0006) [2023-12-27 03:26:21,700][105692] Updated weights for policy 0, policy_version 1653419 (0.0010) [2023-12-27 03:26:21,767][105692] Updated weights for policy 0, policy_version 1653429 (0.0009) [2023-12-27 03:26:21,826][105692] Updated weights for policy 0, policy_version 1653439 (0.0011) [2023-12-27 03:26:21,958][105620] Updated weights for policy 1, policy_version 1656647 (0.0006) [2023-12-27 03:26:22,023][105620] Updated weights for policy 1, policy_version 1656657 (0.0006) [2023-12-27 03:26:22,091][105620] Updated weights for policy 1, policy_version 1656667 (0.0008) [2023-12-27 03:26:22,592][105692] Updated weights for policy 0, policy_version 1653449 (0.0006) [2023-12-27 03:26:22,661][105692] Updated weights for policy 0, policy_version 1653459 (0.0009) [2023-12-27 03:26:22,721][105692] Updated weights for policy 0, policy_version 1653469 (0.0009) [2023-12-27 03:26:22,852][105620] Updated weights for policy 1, policy_version 1656677 (0.0009) [2023-12-27 03:26:22,915][105620] Updated weights for policy 1, policy_version 1656687 (0.0008) [2023-12-27 03:26:22,975][105620] Updated weights for policy 1, policy_version 1656697 (0.0007) [2023-12-27 03:26:23,426][105692] Updated weights for policy 0, policy_version 1653479 (0.0009) [2023-12-27 03:26:23,480][105692] Updated weights for policy 0, policy_version 1653489 (0.0008) [2023-12-27 03:26:23,524][105692] Updated weights for policy 0, policy_version 1653499 (0.0007) [2023-12-27 03:26:23,685][105620] Updated weights for policy 1, policy_version 1656707 (0.0007) [2023-12-27 03:26:23,737][105620] Updated weights for policy 1, policy_version 1656717 (0.0010) [2023-12-27 03:26:23,791][105620] Updated weights for policy 1, policy_version 1656727 (0.0010) [2023-12-27 03:26:24,297][105692] Updated weights for policy 0, policy_version 1653509 (0.0007) [2023-12-27 03:26:24,345][105692] Updated weights for policy 0, policy_version 1653519 (0.0005) [2023-12-27 03:26:24,367][105620] Updated weights for policy 1, policy_version 1656737 (0.0008) [2023-12-27 03:26:24,404][105692] Updated weights for policy 0, policy_version 1653529 (0.0005) [2023-12-27 03:26:24,422][105620] Updated weights for policy 1, policy_version 1656747 (0.0006) [2023-12-27 03:26:24,482][105620] Updated weights for policy 1, policy_version 1656757 (0.0006) [2023-12-27 03:26:24,539][105620] Updated weights for policy 1, policy_version 1656767 (0.0005) [2023-12-27 03:26:25,027][105692] Updated weights for policy 0, policy_version 1653539 (0.0007) [2023-12-27 03:26:25,089][105692] Updated weights for policy 0, policy_version 1653549 (0.0011) [2023-12-27 03:26:25,092][105620] Updated weights for policy 1, policy_version 1656777 (0.0006) [2023-12-27 03:26:25,137][105620] Updated weights for policy 1, policy_version 1656787 (0.0007) [2023-12-27 03:26:25,157][105692] Updated weights for policy 0, policy_version 1653559 (0.0006) [2023-12-27 03:26:25,188][105620] Updated weights for policy 1, policy_version 1656797 (0.0009) [2023-12-27 03:26:25,830][105620] Updated weights for policy 1, policy_version 1656807 (0.0007) [2023-12-27 03:26:25,844][105692] Updated weights for policy 0, policy_version 1653569 (0.0006) [2023-12-27 03:26:25,881][105620] Updated weights for policy 1, policy_version 1656817 (0.0005) [2023-12-27 03:26:25,891][105692] Updated weights for policy 0, policy_version 1653579 (0.0010) [2023-12-27 03:26:25,939][105620] Updated weights for policy 1, policy_version 1656827 (0.0005) [2023-12-27 03:26:25,952][105692] Updated weights for policy 0, policy_version 1653589 (0.0008) [2023-12-27 03:26:26,007][105692] Updated weights for policy 0, policy_version 1653599 (0.0006) [2023-12-27 03:26:26,062][104569] Fps is (10 sec: 20480.9, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 847593472. Throughput: 0: 9772.8, 1: 9963.0. Samples: 847595860. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:26:26,062][104569] Avg episode reward: [(0, '8348.162'), (1, '8897.959')] [2023-12-27 03:26:26,548][105620] Updated weights for policy 1, policy_version 1656837 (0.0008) [2023-12-27 03:26:26,607][105620] Updated weights for policy 1, policy_version 1656847 (0.0010) [2023-12-27 03:26:26,657][105692] Updated weights for policy 0, policy_version 1653609 (0.0010) [2023-12-27 03:26:26,659][105620] Updated weights for policy 1, policy_version 1656857 (0.0010) [2023-12-27 03:26:26,711][105692] Updated weights for policy 0, policy_version 1653619 (0.0010) [2023-12-27 03:26:26,759][105692] Updated weights for policy 0, policy_version 1653629 (0.0010) [2023-12-27 03:26:27,408][105620] Updated weights for policy 1, policy_version 1656867 (0.0010) [2023-12-27 03:26:27,441][105692] Updated weights for policy 0, policy_version 1653639 (0.0007) [2023-12-27 03:26:27,463][105620] Updated weights for policy 1, policy_version 1656877 (0.0010) [2023-12-27 03:26:27,498][105692] Updated weights for policy 0, policy_version 1653649 (0.0005) [2023-12-27 03:26:27,514][105620] Updated weights for policy 1, policy_version 1656887 (0.0010) [2023-12-27 03:26:27,551][105692] Updated weights for policy 0, policy_version 1653659 (0.0010) [2023-12-27 03:26:28,129][105620] Updated weights for policy 1, policy_version 1656897 (0.0007) [2023-12-27 03:26:28,178][105620] Updated weights for policy 1, policy_version 1656907 (0.0010) [2023-12-27 03:26:28,236][105620] Updated weights for policy 1, policy_version 1656917 (0.0007) [2023-12-27 03:26:28,252][105692] Updated weights for policy 0, policy_version 1653669 (0.0011) [2023-12-27 03:26:28,286][105620] Updated weights for policy 1, policy_version 1656927 (0.0005) [2023-12-27 03:26:28,310][105692] Updated weights for policy 0, policy_version 1653679 (0.0010) [2023-12-27 03:26:28,380][105692] Updated weights for policy 0, policy_version 1653689 (0.0011) [2023-12-27 03:26:29,011][105620] Updated weights for policy 1, policy_version 1656937 (0.0010) [2023-12-27 03:26:29,066][105620] Updated weights for policy 1, policy_version 1656947 (0.0010) [2023-12-27 03:26:29,070][105692] Updated weights for policy 0, policy_version 1653699 (0.0010) [2023-12-27 03:26:29,124][105620] Updated weights for policy 1, policy_version 1656957 (0.0010) [2023-12-27 03:26:29,124][105692] Updated weights for policy 0, policy_version 1653709 (0.0010) [2023-12-27 03:26:29,171][105692] Updated weights for policy 0, policy_version 1653719 (0.0010) [2023-12-27 03:26:29,763][105620] Updated weights for policy 1, policy_version 1656967 (0.0007) [2023-12-27 03:26:29,814][105620] Updated weights for policy 1, policy_version 1656977 (0.0005) [2023-12-27 03:26:29,874][105620] Updated weights for policy 1, policy_version 1656987 (0.0007) [2023-12-27 03:26:29,881][105692] Updated weights for policy 0, policy_version 1653729 (0.0010) [2023-12-27 03:26:29,932][105692] Updated weights for policy 0, policy_version 1653739 (0.0008) [2023-12-27 03:26:29,994][105692] Updated weights for policy 0, policy_version 1653749 (0.0008) [2023-12-27 03:26:30,051][105692] Updated weights for policy 0, policy_version 1653759 (0.0010) [2023-12-27 03:26:30,603][105620] Updated weights for policy 1, policy_version 1656997 (0.0011) [2023-12-27 03:26:30,665][105620] Updated weights for policy 1, policy_version 1657007 (0.0011) [2023-12-27 03:26:30,672][105692] Updated weights for policy 0, policy_version 1653769 (0.0006) [2023-12-27 03:26:30,721][105620] Updated weights for policy 1, policy_version 1657017 (0.0010) [2023-12-27 03:26:30,734][105692] Updated weights for policy 0, policy_version 1653779 (0.0006) [2023-12-27 03:26:30,795][105692] Updated weights for policy 0, policy_version 1653789 (0.0007) [2023-12-27 03:26:31,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 847691776. Throughput: 0: 9768.1, 1: 10038.9. Samples: 847657180. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:26:31,063][104569] Avg episode reward: [(0, '8625.146'), (1, '8989.270')] [2023-12-27 03:26:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001653792_423436288.pth... [2023-12-27 03:26:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001657024_424255488.pth... [2023-12-27 03:26:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001655872_423960576.pth [2023-12-27 03:26:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001652640_423141376.pth [2023-12-27 03:26:31,409][105620] Updated weights for policy 1, policy_version 1657027 (0.0010) [2023-12-27 03:26:31,477][105620] Updated weights for policy 1, policy_version 1657037 (0.0006) [2023-12-27 03:26:31,541][105620] Updated weights for policy 1, policy_version 1657047 (0.0008) [2023-12-27 03:26:31,544][105692] Updated weights for policy 0, policy_version 1653799 (0.0006) [2023-12-27 03:26:31,599][105692] Updated weights for policy 0, policy_version 1653809 (0.0009) [2023-12-27 03:26:31,655][105692] Updated weights for policy 0, policy_version 1653819 (0.0008) [2023-12-27 03:26:32,140][105620] Updated weights for policy 1, policy_version 1657057 (0.0007) [2023-12-27 03:26:32,194][105620] Updated weights for policy 1, policy_version 1657067 (0.0009) [2023-12-27 03:26:32,257][105620] Updated weights for policy 1, policy_version 1657077 (0.0009) [2023-12-27 03:26:32,315][105620] Updated weights for policy 1, policy_version 1657087 (0.0009) [2023-12-27 03:26:32,464][105692] Updated weights for policy 0, policy_version 1653829 (0.0008) [2023-12-27 03:26:32,512][105692] Updated weights for policy 0, policy_version 1653839 (0.0009) [2023-12-27 03:26:32,563][105692] Updated weights for policy 0, policy_version 1653849 (0.0009) [2023-12-27 03:26:32,990][105620] Updated weights for policy 1, policy_version 1657097 (0.0010) [2023-12-27 03:26:33,049][105620] Updated weights for policy 1, policy_version 1657107 (0.0009) [2023-12-27 03:26:33,110][105620] Updated weights for policy 1, policy_version 1657117 (0.0009) [2023-12-27 03:26:33,367][105692] Updated weights for policy 0, policy_version 1653859 (0.0009) [2023-12-27 03:26:33,422][105692] Updated weights for policy 0, policy_version 1653869 (0.0008) [2023-12-27 03:26:33,468][105692] Updated weights for policy 0, policy_version 1653879 (0.0007) [2023-12-27 03:26:33,861][105620] Updated weights for policy 1, policy_version 1657127 (0.0008) [2023-12-27 03:26:33,914][105620] Updated weights for policy 1, policy_version 1657137 (0.0007) [2023-12-27 03:26:33,922][105586] KL-divergence is very high: 105.7373 [2023-12-27 03:26:33,966][105586] KL-divergence is very high: 124.2810 [2023-12-27 03:26:33,972][105620] Updated weights for policy 1, policy_version 1657147 (0.0005) [2023-12-27 03:26:34,247][105692] Updated weights for policy 0, policy_version 1653889 (0.0009) [2023-12-27 03:26:34,302][105692] Updated weights for policy 0, policy_version 1653899 (0.0009) [2023-12-27 03:26:34,357][105692] Updated weights for policy 0, policy_version 1653909 (0.0009) [2023-12-27 03:26:34,412][105692] Updated weights for policy 0, policy_version 1653919 (0.0008) [2023-12-27 03:26:34,644][105620] Updated weights for policy 1, policy_version 1657157 (0.0007) [2023-12-27 03:26:34,697][105620] Updated weights for policy 1, policy_version 1657167 (0.0008) [2023-12-27 03:26:34,754][105620] Updated weights for policy 1, policy_version 1657177 (0.0009) [2023-12-27 03:26:35,143][105692] Updated weights for policy 0, policy_version 1653929 (0.0009) [2023-12-27 03:26:35,198][105692] Updated weights for policy 0, policy_version 1653939 (0.0006) [2023-12-27 03:26:35,254][105692] Updated weights for policy 0, policy_version 1653949 (0.0009) [2023-12-27 03:26:35,501][105620] Updated weights for policy 1, policy_version 1657187 (0.0008) [2023-12-27 03:26:35,558][105620] Updated weights for policy 1, policy_version 1657197 (0.0009) [2023-12-27 03:26:35,617][105620] Updated weights for policy 1, policy_version 1657207 (0.0009) [2023-12-27 03:26:35,982][105692] Updated weights for policy 0, policy_version 1653959 (0.0007) [2023-12-27 03:26:36,056][105692] Updated weights for policy 0, policy_version 1653969 (0.0007) [2023-12-27 03:26:36,062][104569] Fps is (10 sec: 18840.6, 60 sec: 19660.7, 300 sec: 19438.6). Total num frames: 847781888. Throughput: 0: 9848.6, 1: 9890.1. Samples: 847774200. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:26:36,063][104569] Avg episode reward: [(0, '8356.361'), (1, '8897.754')] [2023-12-27 03:26:36,118][105692] Updated weights for policy 0, policy_version 1653979 (0.0009) [2023-12-27 03:26:36,314][105620] Updated weights for policy 1, policy_version 1657217 (0.0008) [2023-12-27 03:26:36,377][105620] Updated weights for policy 1, policy_version 1657227 (0.0009) [2023-12-27 03:26:36,424][105620] Updated weights for policy 1, policy_version 1657237 (0.0008) [2023-12-27 03:26:36,475][105620] Updated weights for policy 1, policy_version 1657247 (0.0009) [2023-12-27 03:26:36,785][105692] Updated weights for policy 0, policy_version 1653989 (0.0008) [2023-12-27 03:26:36,848][105692] Updated weights for policy 0, policy_version 1653999 (0.0005) [2023-12-27 03:26:36,894][105692] Updated weights for policy 0, policy_version 1654009 (0.0005) [2023-12-27 03:26:37,196][105620] Updated weights for policy 1, policy_version 1657257 (0.0006) [2023-12-27 03:26:37,250][105620] Updated weights for policy 1, policy_version 1657267 (0.0007) [2023-12-27 03:26:37,305][105620] Updated weights for policy 1, policy_version 1657277 (0.0009) [2023-12-27 03:26:37,559][105692] Updated weights for policy 0, policy_version 1654019 (0.0009) [2023-12-27 03:26:37,611][105692] Updated weights for policy 0, policy_version 1654029 (0.0008) [2023-12-27 03:26:37,666][105692] Updated weights for policy 0, policy_version 1654039 (0.0008) [2023-12-27 03:26:37,995][105620] Updated weights for policy 1, policy_version 1657287 (0.0010) [2023-12-27 03:26:38,051][105620] Updated weights for policy 1, policy_version 1657297 (0.0007) [2023-12-27 03:26:38,103][105620] Updated weights for policy 1, policy_version 1657307 (0.0008) [2023-12-27 03:26:38,413][105692] Updated weights for policy 0, policy_version 1654049 (0.0009) [2023-12-27 03:26:38,466][105692] Updated weights for policy 0, policy_version 1654059 (0.0008) [2023-12-27 03:26:38,517][105692] Updated weights for policy 0, policy_version 1654069 (0.0009) [2023-12-27 03:26:38,583][105692] Updated weights for policy 0, policy_version 1654079 (0.0008) [2023-12-27 03:26:38,845][105620] Updated weights for policy 1, policy_version 1657317 (0.0010) [2023-12-27 03:26:38,894][105620] Updated weights for policy 1, policy_version 1657327 (0.0010) [2023-12-27 03:26:38,956][105620] Updated weights for policy 1, policy_version 1657337 (0.0011) [2023-12-27 03:26:38,975][105586] KL-divergence is very high: 161.1715 [2023-12-27 03:26:39,386][105692] Updated weights for policy 0, policy_version 1654089 (0.0008) [2023-12-27 03:26:39,451][105692] Updated weights for policy 0, policy_version 1654099 (0.0008) [2023-12-27 03:26:39,515][105692] Updated weights for policy 0, policy_version 1654109 (0.0010) [2023-12-27 03:26:39,722][105620] Updated weights for policy 1, policy_version 1657347 (0.0010) [2023-12-27 03:26:39,772][105620] Updated weights for policy 1, policy_version 1657357 (0.0009) [2023-12-27 03:26:39,835][105620] Updated weights for policy 1, policy_version 1657367 (0.0009) [2023-12-27 03:26:40,274][105692] Updated weights for policy 0, policy_version 1654119 (0.0008) [2023-12-27 03:26:40,341][105692] Updated weights for policy 0, policy_version 1654129 (0.0008) [2023-12-27 03:26:40,404][105692] Updated weights for policy 0, policy_version 1654139 (0.0010) [2023-12-27 03:26:40,572][105620] Updated weights for policy 1, policy_version 1657377 (0.0009) [2023-12-27 03:26:40,626][105620] Updated weights for policy 1, policy_version 1657387 (0.0006) [2023-12-27 03:26:40,680][105620] Updated weights for policy 1, policy_version 1657397 (0.0006) [2023-12-27 03:26:40,733][105620] Updated weights for policy 1, policy_version 1657407 (0.0006) [2023-12-27 03:26:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 847880192. Throughput: 0: 9810.6, 1: 9903.6. Samples: 847889456. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:26:41,063][104569] Avg episode reward: [(0, '8538.602'), (1, '8806.244')] [2023-12-27 03:26:41,138][105692] Updated weights for policy 0, policy_version 1654149 (0.0008) [2023-12-27 03:26:41,206][105692] Updated weights for policy 0, policy_version 1654159 (0.0009) [2023-12-27 03:26:41,268][105692] Updated weights for policy 0, policy_version 1654169 (0.0009) [2023-12-27 03:26:41,512][105620] Updated weights for policy 1, policy_version 1657417 (0.0009) [2023-12-27 03:26:41,569][105620] Updated weights for policy 1, policy_version 1657427 (0.0011) [2023-12-27 03:26:41,634][105620] Updated weights for policy 1, policy_version 1657437 (0.0009) [2023-12-27 03:26:42,032][105692] Updated weights for policy 0, policy_version 1654179 (0.0007) [2023-12-27 03:26:42,083][105692] Updated weights for policy 0, policy_version 1654189 (0.0009) [2023-12-27 03:26:42,139][105692] Updated weights for policy 0, policy_version 1654199 (0.0008) [2023-12-27 03:26:42,411][105620] Updated weights for policy 1, policy_version 1657447 (0.0006) [2023-12-27 03:26:42,471][105620] Updated weights for policy 1, policy_version 1657457 (0.0007) [2023-12-27 03:26:42,522][105620] Updated weights for policy 1, policy_version 1657467 (0.0007) [2023-12-27 03:26:43,064][105692] Updated weights for policy 0, policy_version 1654209 (0.0009) [2023-12-27 03:26:43,111][105692] Updated weights for policy 0, policy_version 1654219 (0.0008) [2023-12-27 03:26:43,118][105620] Updated weights for policy 1, policy_version 1657477 (0.0009) [2023-12-27 03:26:43,160][105692] Updated weights for policy 0, policy_version 1654229 (0.0008) [2023-12-27 03:26:43,183][105620] Updated weights for policy 1, policy_version 1657487 (0.0005) [2023-12-27 03:26:43,219][105692] Updated weights for policy 0, policy_version 1654239 (0.0007) [2023-12-27 03:26:43,246][105620] Updated weights for policy 1, policy_version 1657497 (0.0008) [2023-12-27 03:26:43,869][105620] Updated weights for policy 1, policy_version 1657507 (0.0005) [2023-12-27 03:26:43,927][105620] Updated weights for policy 1, policy_version 1657517 (0.0008) [2023-12-27 03:26:43,975][105620] Updated weights for policy 1, policy_version 1657527 (0.0005) [2023-12-27 03:26:44,037][105692] Updated weights for policy 0, policy_version 1654249 (0.0007) [2023-12-27 03:26:44,083][105692] Updated weights for policy 0, policy_version 1654259 (0.0008) [2023-12-27 03:26:44,138][105692] Updated weights for policy 0, policy_version 1654269 (0.0009) [2023-12-27 03:26:44,607][105620] Updated weights for policy 1, policy_version 1657537 (0.0006) [2023-12-27 03:26:44,659][105620] Updated weights for policy 1, policy_version 1657547 (0.0009) [2023-12-27 03:26:44,709][105620] Updated weights for policy 1, policy_version 1657557 (0.0005) [2023-12-27 03:26:44,761][105620] Updated weights for policy 1, policy_version 1657567 (0.0006) [2023-12-27 03:26:44,978][105692] Updated weights for policy 0, policy_version 1654280 (0.0009) [2023-12-27 03:26:45,037][105692] Updated weights for policy 0, policy_version 1654290 (0.0009) [2023-12-27 03:26:45,093][105692] Updated weights for policy 0, policy_version 1654300 (0.0009) [2023-12-27 03:26:45,516][105620] Updated weights for policy 1, policy_version 1657577 (0.0009) [2023-12-27 03:26:45,579][105620] Updated weights for policy 1, policy_version 1657587 (0.0009) [2023-12-27 03:26:45,635][105620] Updated weights for policy 1, policy_version 1657597 (0.0009) [2023-12-27 03:26:45,836][105692] Updated weights for policy 0, policy_version 1654310 (0.0007) [2023-12-27 03:26:45,900][105692] Updated weights for policy 0, policy_version 1654320 (0.0005) [2023-12-27 03:26:45,963][105692] Updated weights for policy 0, policy_version 1654330 (0.0009) [2023-12-27 03:26:46,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.7, 300 sec: 19410.9). Total num frames: 847978496. Throughput: 0: 9697.0, 1: 9959.8. Samples: 847946444. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:26:46,063][104569] Avg episode reward: [(0, '8806.460'), (1, '8896.035')] [2023-12-27 03:26:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001654336_423575552.pth... [2023-12-27 03:26:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001657600_424402944.pth... [2023-12-27 03:26:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001653216_423288832.pth [2023-12-27 03:26:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001656416_424099840.pth [2023-12-27 03:26:46,392][105620] Updated weights for policy 1, policy_version 1657607 (0.0009) [2023-12-27 03:26:46,442][105620] Updated weights for policy 1, policy_version 1657617 (0.0008) [2023-12-27 03:26:46,489][105620] Updated weights for policy 1, policy_version 1657627 (0.0009) [2023-12-27 03:26:46,694][105692] Updated weights for policy 0, policy_version 1654340 (0.0010) [2023-12-27 03:26:46,756][105692] Updated weights for policy 0, policy_version 1654350 (0.0007) [2023-12-27 03:26:46,825][105692] Updated weights for policy 0, policy_version 1654360 (0.0007) [2023-12-27 03:26:47,104][105620] Updated weights for policy 1, policy_version 1657637 (0.0007) [2023-12-27 03:26:47,163][105620] Updated weights for policy 1, policy_version 1657647 (0.0005) [2023-12-27 03:26:47,224][105620] Updated weights for policy 1, policy_version 1657657 (0.0005) [2023-12-27 03:26:47,683][105692] Updated weights for policy 0, policy_version 1654370 (0.0009) [2023-12-27 03:26:47,723][105620] Updated weights for policy 1, policy_version 1657667 (0.0005) [2023-12-27 03:26:47,732][105692] Updated weights for policy 0, policy_version 1654380 (0.0009) [2023-12-27 03:26:47,777][105620] Updated weights for policy 1, policy_version 1657677 (0.0005) [2023-12-27 03:26:47,788][105692] Updated weights for policy 0, policy_version 1654390 (0.0009) [2023-12-27 03:26:47,828][105620] Updated weights for policy 1, policy_version 1657687 (0.0005) [2023-12-27 03:26:47,845][105692] Updated weights for policy 0, policy_version 1654400 (0.0008) [2023-12-27 03:26:48,375][105620] Updated weights for policy 1, policy_version 1657697 (0.0005) [2023-12-27 03:26:48,424][105620] Updated weights for policy 1, policy_version 1657707 (0.0006) [2023-12-27 03:26:48,475][105620] Updated weights for policy 1, policy_version 1657717 (0.0006) [2023-12-27 03:26:48,527][105620] Updated weights for policy 1, policy_version 1657727 (0.0005) [2023-12-27 03:26:48,758][105692] Updated weights for policy 0, policy_version 1654410 (0.0009) [2023-12-27 03:26:48,817][105692] Updated weights for policy 0, policy_version 1654420 (0.0010) [2023-12-27 03:26:48,883][105692] Updated weights for policy 0, policy_version 1654430 (0.0009) [2023-12-27 03:26:49,130][105620] Updated weights for policy 1, policy_version 1657737 (0.0005) [2023-12-27 03:26:49,179][105620] Updated weights for policy 1, policy_version 1657747 (0.0006) [2023-12-27 03:26:49,244][105620] Updated weights for policy 1, policy_version 1657757 (0.0007) [2023-12-27 03:26:49,753][105692] Updated weights for policy 0, policy_version 1654440 (0.0010) [2023-12-27 03:26:49,825][105692] Updated weights for policy 0, policy_version 1654450 (0.0010) [2023-12-27 03:26:49,841][105620] Updated weights for policy 1, policy_version 1657767 (0.0007) [2023-12-27 03:26:49,889][105692] Updated weights for policy 0, policy_version 1654460 (0.0007) [2023-12-27 03:26:49,901][105620] Updated weights for policy 1, policy_version 1657777 (0.0006) [2023-12-27 03:26:49,963][105620] Updated weights for policy 1, policy_version 1657787 (0.0010) [2023-12-27 03:26:50,497][105620] Updated weights for policy 1, policy_version 1657797 (0.0005) [2023-12-27 03:26:50,555][105620] Updated weights for policy 1, policy_version 1657807 (0.0005) [2023-12-27 03:26:50,614][105620] Updated weights for policy 1, policy_version 1657817 (0.0008) [2023-12-27 03:26:50,727][105692] Updated weights for policy 0, policy_version 1654470 (0.0008) [2023-12-27 03:26:50,790][105692] Updated weights for policy 0, policy_version 1654480 (0.0009) [2023-12-27 03:26:50,850][105692] Updated weights for policy 0, policy_version 1654490 (0.0008) [2023-12-27 03:26:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 848076800. Throughput: 0: 9465.1, 1: 10113.6. Samples: 848064556. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:26:51,062][104569] Avg episode reward: [(0, '8619.256'), (1, '8990.944')] [2023-12-27 03:26:51,343][105620] Updated weights for policy 1, policy_version 1657827 (0.0009) [2023-12-27 03:26:51,412][105620] Updated weights for policy 1, policy_version 1657837 (0.0010) [2023-12-27 03:26:51,485][105620] Updated weights for policy 1, policy_version 1657847 (0.0008) [2023-12-27 03:26:51,555][105692] Updated weights for policy 0, policy_version 1654500 (0.0007) [2023-12-27 03:26:51,616][105692] Updated weights for policy 0, policy_version 1654510 (0.0008) [2023-12-27 03:26:51,682][105692] Updated weights for policy 0, policy_version 1654520 (0.0008) [2023-12-27 03:26:52,160][105620] Updated weights for policy 1, policy_version 1657857 (0.0006) [2023-12-27 03:26:52,213][105620] Updated weights for policy 1, policy_version 1657867 (0.0010) [2023-12-27 03:26:52,273][105620] Updated weights for policy 1, policy_version 1657877 (0.0011) [2023-12-27 03:26:52,336][105620] Updated weights for policy 1, policy_version 1657887 (0.0010) [2023-12-27 03:26:52,381][105692] Updated weights for policy 0, policy_version 1654530 (0.0007) [2023-12-27 03:26:52,439][105692] Updated weights for policy 0, policy_version 1654540 (0.0007) [2023-12-27 03:26:52,506][105692] Updated weights for policy 0, policy_version 1654550 (0.0005) [2023-12-27 03:26:52,575][105692] Updated weights for policy 0, policy_version 1654560 (0.0006) [2023-12-27 03:26:53,100][105620] Updated weights for policy 1, policy_version 1657897 (0.0011) [2023-12-27 03:26:53,142][105692] Updated weights for policy 0, policy_version 1654570 (0.0006) [2023-12-27 03:26:53,160][105620] Updated weights for policy 1, policy_version 1657907 (0.0010) [2023-12-27 03:26:53,198][105692] Updated weights for policy 0, policy_version 1654580 (0.0006) [2023-12-27 03:26:53,215][105620] Updated weights for policy 1, policy_version 1657917 (0.0010) [2023-12-27 03:26:53,253][105692] Updated weights for policy 0, policy_version 1654590 (0.0006) [2023-12-27 03:26:53,893][105620] Updated weights for policy 1, policy_version 1657927 (0.0007) [2023-12-27 03:26:53,949][105620] Updated weights for policy 1, policy_version 1657937 (0.0005) [2023-12-27 03:26:54,004][105620] Updated weights for policy 1, policy_version 1657947 (0.0006) [2023-12-27 03:26:54,063][105692] Updated weights for policy 0, policy_version 1654600 (0.0009) [2023-12-27 03:26:54,124][105692] Updated weights for policy 0, policy_version 1654610 (0.0009) [2023-12-27 03:26:54,196][105692] Updated weights for policy 0, policy_version 1654620 (0.0010) [2023-12-27 03:26:54,526][105620] Updated weights for policy 1, policy_version 1657957 (0.0005) [2023-12-27 03:26:54,581][105620] Updated weights for policy 1, policy_version 1657967 (0.0005) [2023-12-27 03:26:54,640][105620] Updated weights for policy 1, policy_version 1657977 (0.0008) [2023-12-27 03:26:55,028][105692] Updated weights for policy 0, policy_version 1654630 (0.0008) [2023-12-27 03:26:55,093][105692] Updated weights for policy 0, policy_version 1654640 (0.0007) [2023-12-27 03:26:55,149][105692] Updated weights for policy 0, policy_version 1654650 (0.0009) [2023-12-27 03:26:55,236][105620] Updated weights for policy 1, policy_version 1657987 (0.0007) [2023-12-27 03:26:55,297][105620] Updated weights for policy 1, policy_version 1657997 (0.0010) [2023-12-27 03:26:55,342][105620] Updated weights for policy 1, policy_version 1658007 (0.0010) [2023-12-27 03:26:55,915][105692] Updated weights for policy 0, policy_version 1654660 (0.0009) [2023-12-27 03:26:55,977][105692] Updated weights for policy 0, policy_version 1654670 (0.0008) [2023-12-27 03:26:56,021][105692] Updated weights for policy 0, policy_version 1654680 (0.0007) [2023-12-27 03:26:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 848175104. Throughput: 0: 9408.7, 1: 10193.5. Samples: 848182264. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:26:56,063][104569] Avg episode reward: [(0, '8621.347'), (1, '9082.997')] [2023-12-27 03:26:56,081][105620] Updated weights for policy 1, policy_version 1658017 (0.0010) [2023-12-27 03:26:56,141][105620] Updated weights for policy 1, policy_version 1658027 (0.0007) [2023-12-27 03:26:56,197][105620] Updated weights for policy 1, policy_version 1658037 (0.0008) [2023-12-27 03:26:56,255][105620] Updated weights for policy 1, policy_version 1658047 (0.0009) [2023-12-27 03:26:56,837][105692] Updated weights for policy 0, policy_version 1654690 (0.0009) [2023-12-27 03:26:56,872][105620] Updated weights for policy 1, policy_version 1658057 (0.0008) [2023-12-27 03:26:56,889][105692] Updated weights for policy 0, policy_version 1654700 (0.0005) [2023-12-27 03:26:56,916][105620] Updated weights for policy 1, policy_version 1658067 (0.0006) [2023-12-27 03:26:56,932][105692] Updated weights for policy 0, policy_version 1654710 (0.0005) [2023-12-27 03:26:56,972][105620] Updated weights for policy 1, policy_version 1658077 (0.0005) [2023-12-27 03:26:56,982][105692] Updated weights for policy 0, policy_version 1654720 (0.0008) [2023-12-27 03:26:57,626][105620] Updated weights for policy 1, policy_version 1658087 (0.0005) [2023-12-27 03:26:57,674][105620] Updated weights for policy 1, policy_version 1658097 (0.0005) [2023-12-27 03:26:57,727][105692] Updated weights for policy 0, policy_version 1654730 (0.0010) [2023-12-27 03:26:57,727][105620] Updated weights for policy 1, policy_version 1658107 (0.0005) [2023-12-27 03:26:57,777][105692] Updated weights for policy 0, policy_version 1654741 (0.0010) [2023-12-27 03:26:57,831][105692] Updated weights for policy 0, policy_version 1654752 (0.0011) [2023-12-27 03:26:58,335][105620] Updated weights for policy 1, policy_version 1658117 (0.0007) [2023-12-27 03:26:58,393][105620] Updated weights for policy 1, policy_version 1658127 (0.0011) [2023-12-27 03:26:58,461][105620] Updated weights for policy 1, policy_version 1658137 (0.0011) [2023-12-27 03:26:58,674][105692] Updated weights for policy 0, policy_version 1654762 (0.0011) [2023-12-27 03:26:58,736][105692] Updated weights for policy 0, policy_version 1654772 (0.0009) [2023-12-27 03:26:58,813][105692] Updated weights for policy 0, policy_version 1654782 (0.0013) [2023-12-27 03:26:59,230][105620] Updated weights for policy 1, policy_version 1658147 (0.0010) [2023-12-27 03:26:59,295][105620] Updated weights for policy 1, policy_version 1658157 (0.0008) [2023-12-27 03:26:59,362][105620] Updated weights for policy 1, policy_version 1658167 (0.0008) [2023-12-27 03:26:59,638][105692] Updated weights for policy 0, policy_version 1654792 (0.0009) [2023-12-27 03:26:59,699][105692] Updated weights for policy 0, policy_version 1654802 (0.0009) [2023-12-27 03:26:59,756][105692] Updated weights for policy 0, policy_version 1654812 (0.0009) [2023-12-27 03:27:00,044][105620] Updated weights for policy 1, policy_version 1658177 (0.0009) [2023-12-27 03:27:00,103][105620] Updated weights for policy 1, policy_version 1658187 (0.0010) [2023-12-27 03:27:00,156][105620] Updated weights for policy 1, policy_version 1658197 (0.0010) [2023-12-27 03:27:00,216][105620] Updated weights for policy 1, policy_version 1658207 (0.0009) [2023-12-27 03:27:00,444][105692] Updated weights for policy 0, policy_version 1654822 (0.0007) [2023-12-27 03:27:00,494][105692] Updated weights for policy 0, policy_version 1654832 (0.0005) [2023-12-27 03:27:00,545][105692] Updated weights for policy 0, policy_version 1654842 (0.0005) [2023-12-27 03:27:00,994][105620] Updated weights for policy 1, policy_version 1658217 (0.0009) [2023-12-27 03:27:01,045][105620] Updated weights for policy 1, policy_version 1658227 (0.0009) [2023-12-27 03:27:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 848265216. Throughput: 0: 9411.4, 1: 10282.7. Samples: 848240636. Policy #0 lag: (min: 17.0, avg: 43.6, max: 49.0) [2023-12-27 03:27:01,062][104569] Avg episode reward: [(0, '8713.367'), (1, '8805.122')] [2023-12-27 03:27:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001654848_423706624.pth... [2023-12-27 03:27:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001653792_423436288.pth [2023-12-27 03:27:01,109][105620] Updated weights for policy 1, policy_version 1658237 (0.0009) [2023-12-27 03:27:01,127][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001658240_424566784.pth... [2023-12-27 03:27:01,132][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001657024_424255488.pth [2023-12-27 03:27:01,234][105692] Updated weights for policy 0, policy_version 1654852 (0.0005) [2023-12-27 03:27:01,296][105692] Updated weights for policy 0, policy_version 1654862 (0.0009) [2023-12-27 03:27:01,356][105692] Updated weights for policy 0, policy_version 1654872 (0.0010) [2023-12-27 03:27:01,927][105620] Updated weights for policy 1, policy_version 1658247 (0.0010) [2023-12-27 03:27:01,978][105620] Updated weights for policy 1, policy_version 1658257 (0.0009) [2023-12-27 03:27:01,999][105692] Updated weights for policy 0, policy_version 1654882 (0.0009) [2023-12-27 03:27:02,034][105620] Updated weights for policy 1, policy_version 1658267 (0.0008) [2023-12-27 03:27:02,049][105692] Updated weights for policy 0, policy_version 1654892 (0.0006) [2023-12-27 03:27:02,101][105692] Updated weights for policy 0, policy_version 1654902 (0.0008) [2023-12-27 03:27:02,160][105692] Updated weights for policy 0, policy_version 1654912 (0.0009) [2023-12-27 03:27:02,747][105620] Updated weights for policy 1, policy_version 1658277 (0.0008) [2023-12-27 03:27:02,801][105620] Updated weights for policy 1, policy_version 1658287 (0.0009) [2023-12-27 03:27:02,858][105620] Updated weights for policy 1, policy_version 1658297 (0.0009) [2023-12-27 03:27:02,918][105692] Updated weights for policy 0, policy_version 1654922 (0.0008) [2023-12-27 03:27:02,972][105692] Updated weights for policy 0, policy_version 1654932 (0.0009) [2023-12-27 03:27:03,033][105692] Updated weights for policy 0, policy_version 1654942 (0.0008) [2023-12-27 03:27:03,561][105620] Updated weights for policy 1, policy_version 1658307 (0.0008) [2023-12-27 03:27:03,610][105620] Updated weights for policy 1, policy_version 1658317 (0.0010) [2023-12-27 03:27:03,654][105620] Updated weights for policy 1, policy_version 1658327 (0.0010) [2023-12-27 03:27:03,681][105692] Updated weights for policy 0, policy_version 1654952 (0.0006) [2023-12-27 03:27:03,734][105692] Updated weights for policy 0, policy_version 1654962 (0.0005) [2023-12-27 03:27:03,778][105692] Updated weights for policy 0, policy_version 1654972 (0.0007) [2023-12-27 03:27:04,390][105692] Updated weights for policy 0, policy_version 1654982 (0.0010) [2023-12-27 03:27:04,412][105620] Updated weights for policy 1, policy_version 1658337 (0.0010) [2023-12-27 03:27:04,457][105692] Updated weights for policy 0, policy_version 1654992 (0.0008) [2023-12-27 03:27:04,470][105620] Updated weights for policy 1, policy_version 1658347 (0.0005) [2023-12-27 03:27:04,519][105692] Updated weights for policy 0, policy_version 1655002 (0.0011) [2023-12-27 03:27:04,533][105620] Updated weights for policy 1, policy_version 1658357 (0.0008) [2023-12-27 03:27:04,589][105620] Updated weights for policy 1, policy_version 1658367 (0.0011) [2023-12-27 03:27:05,088][105692] Updated weights for policy 0, policy_version 1655012 (0.0008) [2023-12-27 03:27:05,146][105692] Updated weights for policy 0, policy_version 1655022 (0.0005) [2023-12-27 03:27:05,200][105692] Updated weights for policy 0, policy_version 1655032 (0.0009) [2023-12-27 03:27:05,287][105620] Updated weights for policy 1, policy_version 1658377 (0.0009) [2023-12-27 03:27:05,344][105620] Updated weights for policy 1, policy_version 1658388 (0.0010) [2023-12-27 03:27:05,404][105620] Updated weights for policy 1, policy_version 1658398 (0.0009) [2023-12-27 03:27:05,781][105692] Updated weights for policy 0, policy_version 1655042 (0.0007) [2023-12-27 03:27:05,832][105692] Updated weights for policy 0, policy_version 1655052 (0.0005) [2023-12-27 03:27:05,883][105692] Updated weights for policy 0, policy_version 1655062 (0.0005) [2023-12-27 03:27:05,926][105692] Updated weights for policy 0, policy_version 1655072 (0.0005) [2023-12-27 03:27:06,012][105620] Updated weights for policy 1, policy_version 1658408 (0.0007) [2023-12-27 03:27:06,058][105620] Updated weights for policy 1, policy_version 1658418 (0.0008) [2023-12-27 03:27:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.9, 300 sec: 19438.6). Total num frames: 848371712. Throughput: 0: 9380.7, 1: 10217.1. Samples: 848357856. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:27:06,062][104569] Avg episode reward: [(0, '8261.840'), (1, '8804.211')] [2023-12-27 03:27:06,114][105620] Updated weights for policy 1, policy_version 1658428 (0.0009) [2023-12-27 03:27:06,565][105692] Updated weights for policy 0, policy_version 1655082 (0.0006) [2023-12-27 03:27:06,621][105692] Updated weights for policy 0, policy_version 1655092 (0.0006) [2023-12-27 03:27:06,680][105692] Updated weights for policy 0, policy_version 1655102 (0.0005) [2023-12-27 03:27:06,943][105620] Updated weights for policy 1, policy_version 1658438 (0.0009) [2023-12-27 03:27:07,000][105620] Updated weights for policy 1, policy_version 1658448 (0.0009) [2023-12-27 03:27:07,061][105620] Updated weights for policy 1, policy_version 1658458 (0.0009) [2023-12-27 03:27:07,210][105692] Updated weights for policy 0, policy_version 1655112 (0.0010) [2023-12-27 03:27:07,262][105692] Updated weights for policy 0, policy_version 1655122 (0.0010) [2023-12-27 03:27:07,318][105692] Updated weights for policy 0, policy_version 1655132 (0.0011) [2023-12-27 03:27:07,894][105620] Updated weights for policy 1, policy_version 1658468 (0.0009) [2023-12-27 03:27:07,951][105620] Updated weights for policy 1, policy_version 1658478 (0.0008) [2023-12-27 03:27:08,015][105620] Updated weights for policy 1, policy_version 1658488 (0.0007) [2023-12-27 03:27:08,026][105692] Updated weights for policy 0, policy_version 1655142 (0.0011) [2023-12-27 03:27:08,089][105692] Updated weights for policy 0, policy_version 1655152 (0.0011) [2023-12-27 03:27:08,148][105692] Updated weights for policy 0, policy_version 1655162 (0.0010) [2023-12-27 03:27:08,671][105620] Updated weights for policy 1, policy_version 1658498 (0.0006) [2023-12-27 03:27:08,725][105620] Updated weights for policy 1, policy_version 1658508 (0.0010) [2023-12-27 03:27:08,782][105620] Updated weights for policy 1, policy_version 1658518 (0.0010) [2023-12-27 03:27:08,803][105692] Updated weights for policy 0, policy_version 1655172 (0.0009) [2023-12-27 03:27:08,831][105620] Updated weights for policy 1, policy_version 1658528 (0.0009) [2023-12-27 03:27:08,855][105692] Updated weights for policy 0, policy_version 1655182 (0.0005) [2023-12-27 03:27:08,909][105692] Updated weights for policy 0, policy_version 1655192 (0.0005) [2023-12-27 03:27:09,553][105620] Updated weights for policy 1, policy_version 1658538 (0.0009) [2023-12-27 03:27:09,573][105692] Updated weights for policy 0, policy_version 1655202 (0.0007) [2023-12-27 03:27:09,612][105620] Updated weights for policy 1, policy_version 1658548 (0.0008) [2023-12-27 03:27:09,623][105692] Updated weights for policy 0, policy_version 1655212 (0.0011) [2023-12-27 03:27:09,674][105620] Updated weights for policy 1, policy_version 1658558 (0.0006) [2023-12-27 03:27:09,678][105692] Updated weights for policy 0, policy_version 1655222 (0.0010) [2023-12-27 03:27:09,729][105692] Updated weights for policy 0, policy_version 1655232 (0.0010) [2023-12-27 03:27:10,451][105620] Updated weights for policy 1, policy_version 1658568 (0.0008) [2023-12-27 03:27:10,509][105620] Updated weights for policy 1, policy_version 1658578 (0.0008) [2023-12-27 03:27:10,539][105692] Updated weights for policy 0, policy_version 1655242 (0.0007) [2023-12-27 03:27:10,564][105620] Updated weights for policy 1, policy_version 1658588 (0.0007) [2023-12-27 03:27:10,606][105692] Updated weights for policy 0, policy_version 1655252 (0.0009) [2023-12-27 03:27:10,660][105692] Updated weights for policy 0, policy_version 1655262 (0.0010) [2023-12-27 03:27:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 848470016. Throughput: 0: 9539.2, 1: 10108.3. Samples: 848479996. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:27:11,063][104569] Avg episode reward: [(0, '8533.479'), (1, '8991.528')] [2023-12-27 03:27:11,183][105620] Updated weights for policy 1, policy_version 1658598 (0.0008) [2023-12-27 03:27:11,245][105620] Updated weights for policy 1, policy_version 1658608 (0.0008) [2023-12-27 03:27:11,315][105620] Updated weights for policy 1, policy_version 1658618 (0.0006) [2023-12-27 03:27:11,496][105692] Updated weights for policy 0, policy_version 1655272 (0.0009) [2023-12-27 03:27:11,550][105692] Updated weights for policy 0, policy_version 1655282 (0.0008) [2023-12-27 03:27:11,607][105692] Updated weights for policy 0, policy_version 1655292 (0.0009) [2023-12-27 03:27:12,038][105620] Updated weights for policy 1, policy_version 1658628 (0.0010) [2023-12-27 03:27:12,097][105620] Updated weights for policy 1, policy_version 1658638 (0.0009) [2023-12-27 03:27:12,152][105620] Updated weights for policy 1, policy_version 1658648 (0.0010) [2023-12-27 03:27:12,307][105692] Updated weights for policy 0, policy_version 1655302 (0.0009) [2023-12-27 03:27:12,369][105692] Updated weights for policy 0, policy_version 1655312 (0.0009) [2023-12-27 03:27:12,443][105692] Updated weights for policy 0, policy_version 1655322 (0.0009) [2023-12-27 03:27:12,863][105620] Updated weights for policy 1, policy_version 1658658 (0.0009) [2023-12-27 03:27:12,919][105620] Updated weights for policy 1, policy_version 1658668 (0.0006) [2023-12-27 03:27:12,979][105620] Updated weights for policy 1, policy_version 1658678 (0.0006) [2023-12-27 03:27:13,037][105620] Updated weights for policy 1, policy_version 1658688 (0.0006) [2023-12-27 03:27:13,319][105692] Updated weights for policy 0, policy_version 1655332 (0.0008) [2023-12-27 03:27:13,383][105692] Updated weights for policy 0, policy_version 1655342 (0.0005) [2023-12-27 03:27:13,451][105692] Updated weights for policy 0, policy_version 1655352 (0.0005) [2023-12-27 03:27:13,649][105620] Updated weights for policy 1, policy_version 1658699 (0.0010) [2023-12-27 03:27:13,704][105620] Updated weights for policy 1, policy_version 1658710 (0.0010) [2023-12-27 03:27:14,024][105692] Updated weights for policy 0, policy_version 1655362 (0.0006) [2023-12-27 03:27:14,077][105692] Updated weights for policy 0, policy_version 1655372 (0.0008) [2023-12-27 03:27:14,127][105692] Updated weights for policy 0, policy_version 1655382 (0.0005) [2023-12-27 03:27:14,185][105692] Updated weights for policy 0, policy_version 1655392 (0.0005) [2023-12-27 03:27:14,493][105620] Updated weights for policy 1, policy_version 1658721 (0.0010) [2023-12-27 03:27:14,551][105620] Updated weights for policy 1, policy_version 1658731 (0.0008) [2023-12-27 03:27:14,613][105620] Updated weights for policy 1, policy_version 1658741 (0.0008) [2023-12-27 03:27:14,677][105620] Updated weights for policy 1, policy_version 1658751 (0.0008) [2023-12-27 03:27:14,810][105692] Updated weights for policy 0, policy_version 1655402 (0.0011) [2023-12-27 03:27:14,871][105692] Updated weights for policy 0, policy_version 1655412 (0.0008) [2023-12-27 03:27:14,925][105692] Updated weights for policy 0, policy_version 1655422 (0.0006) [2023-12-27 03:27:15,462][105620] Updated weights for policy 1, policy_version 1658761 (0.0008) [2023-12-27 03:27:15,514][105620] Updated weights for policy 1, policy_version 1658771 (0.0008) [2023-12-27 03:27:15,571][105620] Updated weights for policy 1, policy_version 1658781 (0.0008) [2023-12-27 03:27:15,642][105692] Updated weights for policy 0, policy_version 1655432 (0.0010) [2023-12-27 03:27:15,690][105692] Updated weights for policy 0, policy_version 1655442 (0.0010) [2023-12-27 03:27:15,739][105692] Updated weights for policy 0, policy_version 1655452 (0.0010) [2023-12-27 03:27:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.9, 300 sec: 19494.2). Total num frames: 848568320. Throughput: 0: 9459.0, 1: 10087.0. Samples: 848536752. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:27:16,063][104569] Avg episode reward: [(0, '8895.889'), (1, '9175.019')] [2023-12-27 03:27:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001655456_423862272.pth... [2023-12-27 03:27:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001658784_424706048.pth... [2023-12-27 03:27:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001654336_423575552.pth [2023-12-27 03:27:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001657600_424402944.pth [2023-12-27 03:27:16,260][105620] Updated weights for policy 1, policy_version 1658791 (0.0006) [2023-12-27 03:27:16,308][105620] Updated weights for policy 1, policy_version 1658801 (0.0005) [2023-12-27 03:27:16,361][105620] Updated weights for policy 1, policy_version 1658811 (0.0005) [2023-12-27 03:27:16,501][105692] Updated weights for policy 0, policy_version 1655462 (0.0010) [2023-12-27 03:27:16,563][105692] Updated weights for policy 0, policy_version 1655472 (0.0010) [2023-12-27 03:27:16,625][105692] Updated weights for policy 0, policy_version 1655482 (0.0011) [2023-12-27 03:27:17,025][105620] Updated weights for policy 1, policy_version 1658821 (0.0007) [2023-12-27 03:27:17,078][105620] Updated weights for policy 1, policy_version 1658831 (0.0008) [2023-12-27 03:27:17,139][105620] Updated weights for policy 1, policy_version 1658841 (0.0008) [2023-12-27 03:27:17,222][105692] Updated weights for policy 0, policy_version 1655492 (0.0011) [2023-12-27 03:27:17,287][105692] Updated weights for policy 0, policy_version 1655502 (0.0011) [2023-12-27 03:27:17,352][105692] Updated weights for policy 0, policy_version 1655512 (0.0011) [2023-12-27 03:27:17,735][105620] Updated weights for policy 1, policy_version 1658851 (0.0007) [2023-12-27 03:27:17,786][105620] Updated weights for policy 1, policy_version 1658861 (0.0005) [2023-12-27 03:27:17,836][105620] Updated weights for policy 1, policy_version 1658871 (0.0007) [2023-12-27 03:27:18,087][105692] Updated weights for policy 0, policy_version 1655522 (0.0010) [2023-12-27 03:27:18,142][105692] Updated weights for policy 0, policy_version 1655532 (0.0010) [2023-12-27 03:27:18,200][105692] Updated weights for policy 0, policy_version 1655542 (0.0010) [2023-12-27 03:27:18,253][105692] Updated weights for policy 0, policy_version 1655552 (0.0010) [2023-12-27 03:27:18,509][105620] Updated weights for policy 1, policy_version 1658881 (0.0006) [2023-12-27 03:27:18,562][105620] Updated weights for policy 1, policy_version 1658891 (0.0009) [2023-12-27 03:27:18,618][105620] Updated weights for policy 1, policy_version 1658901 (0.0007) [2023-12-27 03:27:18,672][105620] Updated weights for policy 1, policy_version 1658911 (0.0005) [2023-12-27 03:27:18,975][105692] Updated weights for policy 0, policy_version 1655562 (0.0010) [2023-12-27 03:27:19,033][105692] Updated weights for policy 0, policy_version 1655572 (0.0010) [2023-12-27 03:27:19,094][105692] Updated weights for policy 0, policy_version 1655582 (0.0010) [2023-12-27 03:27:19,386][105620] Updated weights for policy 1, policy_version 1658921 (0.0008) [2023-12-27 03:27:19,441][105620] Updated weights for policy 1, policy_version 1658931 (0.0008) [2023-12-27 03:27:19,500][105620] Updated weights for policy 1, policy_version 1658941 (0.0010) [2023-12-27 03:27:19,819][105692] Updated weights for policy 0, policy_version 1655592 (0.0007) [2023-12-27 03:27:19,880][105692] Updated weights for policy 0, policy_version 1655602 (0.0010) [2023-12-27 03:27:19,934][105692] Updated weights for policy 0, policy_version 1655612 (0.0011) [2023-12-27 03:27:20,196][105620] Updated weights for policy 1, policy_version 1658951 (0.0011) [2023-12-27 03:27:20,253][105620] Updated weights for policy 1, policy_version 1658961 (0.0010) [2023-12-27 03:27:20,309][105620] Updated weights for policy 1, policy_version 1658971 (0.0009) [2023-12-27 03:27:20,676][105692] Updated weights for policy 0, policy_version 1655622 (0.0009) [2023-12-27 03:27:20,739][105692] Updated weights for policy 0, policy_version 1655632 (0.0011) [2023-12-27 03:27:20,806][105692] Updated weights for policy 0, policy_version 1655642 (0.0011) [2023-12-27 03:27:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 848666624. Throughput: 0: 9531.5, 1: 10095.4. Samples: 848657400. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:27:21,063][104569] Avg episode reward: [(0, '8619.819'), (1, '8807.827')] [2023-12-27 03:27:21,102][105620] Updated weights for policy 1, policy_version 1658981 (0.0010) [2023-12-27 03:27:21,160][105620] Updated weights for policy 1, policy_version 1658991 (0.0008) [2023-12-27 03:27:21,219][105620] Updated weights for policy 1, policy_version 1659001 (0.0009) [2023-12-27 03:27:21,594][105692] Updated weights for policy 0, policy_version 1655652 (0.0010) [2023-12-27 03:27:21,660][105692] Updated weights for policy 0, policy_version 1655662 (0.0009) [2023-12-27 03:27:21,726][105692] Updated weights for policy 0, policy_version 1655672 (0.0010) [2023-12-27 03:27:21,927][105620] Updated weights for policy 1, policy_version 1659011 (0.0008) [2023-12-27 03:27:21,985][105620] Updated weights for policy 1, policy_version 1659021 (0.0005) [2023-12-27 03:27:22,048][105620] Updated weights for policy 1, policy_version 1659031 (0.0005) [2023-12-27 03:27:22,592][105692] Updated weights for policy 0, policy_version 1655682 (0.0009) [2023-12-27 03:27:22,651][105692] Updated weights for policy 0, policy_version 1655692 (0.0008) [2023-12-27 03:27:22,704][105692] Updated weights for policy 0, policy_version 1655702 (0.0008) [2023-12-27 03:27:22,717][105620] Updated weights for policy 1, policy_version 1659041 (0.0008) [2023-12-27 03:27:22,761][105692] Updated weights for policy 0, policy_version 1655712 (0.0006) [2023-12-27 03:27:22,780][105620] Updated weights for policy 1, policy_version 1659051 (0.0008) [2023-12-27 03:27:22,842][105620] Updated weights for policy 1, policy_version 1659061 (0.0008) [2023-12-27 03:27:22,894][105620] Updated weights for policy 1, policy_version 1659071 (0.0008) [2023-12-27 03:27:23,562][105620] Updated weights for policy 1, policy_version 1659081 (0.0006) [2023-12-27 03:27:23,568][105692] Updated weights for policy 0, policy_version 1655722 (0.0005) [2023-12-27 03:27:23,619][105620] Updated weights for policy 1, policy_version 1659091 (0.0005) [2023-12-27 03:27:23,627][105692] Updated weights for policy 0, policy_version 1655732 (0.0010) [2023-12-27 03:27:23,673][105620] Updated weights for policy 1, policy_version 1659101 (0.0005) [2023-12-27 03:27:23,686][105692] Updated weights for policy 0, policy_version 1655742 (0.0008) [2023-12-27 03:27:24,234][105692] Updated weights for policy 0, policy_version 1655752 (0.0008) [2023-12-27 03:27:24,267][105620] Updated weights for policy 1, policy_version 1659111 (0.0008) [2023-12-27 03:27:24,288][105692] Updated weights for policy 0, policy_version 1655762 (0.0006) [2023-12-27 03:27:24,330][105620] Updated weights for policy 1, policy_version 1659121 (0.0007) [2023-12-27 03:27:24,345][105692] Updated weights for policy 0, policy_version 1655772 (0.0006) [2023-12-27 03:27:24,390][105620] Updated weights for policy 1, policy_version 1659131 (0.0008) [2023-12-27 03:27:25,069][105692] Updated weights for policy 0, policy_version 1655782 (0.0006) [2023-12-27 03:27:25,096][105620] Updated weights for policy 1, policy_version 1659141 (0.0007) [2023-12-27 03:27:25,122][105692] Updated weights for policy 0, policy_version 1655792 (0.0007) [2023-12-27 03:27:25,144][105620] Updated weights for policy 1, policy_version 1659151 (0.0007) [2023-12-27 03:27:25,173][105692] Updated weights for policy 0, policy_version 1655802 (0.0006) [2023-12-27 03:27:25,196][105620] Updated weights for policy 1, policy_version 1659161 (0.0006) [2023-12-27 03:27:25,778][105692] Updated weights for policy 0, policy_version 1655812 (0.0007) [2023-12-27 03:27:25,809][105620] Updated weights for policy 1, policy_version 1659171 (0.0006) [2023-12-27 03:27:25,840][105692] Updated weights for policy 0, policy_version 1655822 (0.0007) [2023-12-27 03:27:25,873][105620] Updated weights for policy 1, policy_version 1659181 (0.0007) [2023-12-27 03:27:25,911][105692] Updated weights for policy 0, policy_version 1655832 (0.0008) [2023-12-27 03:27:25,929][105620] Updated weights for policy 1, policy_version 1659191 (0.0007) [2023-12-27 03:27:26,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.7, 300 sec: 19494.2). Total num frames: 848773120. Throughput: 0: 9555.8, 1: 10158.0. Samples: 848776580. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:27:26,063][104569] Avg episode reward: [(0, '8438.587'), (1, '8629.121')] [2023-12-27 03:27:26,652][105692] Updated weights for policy 0, policy_version 1655842 (0.0009) [2023-12-27 03:27:26,670][105620] Updated weights for policy 1, policy_version 1659201 (0.0007) [2023-12-27 03:27:26,705][105692] Updated weights for policy 0, policy_version 1655852 (0.0006) [2023-12-27 03:27:26,732][105620] Updated weights for policy 1, policy_version 1659211 (0.0008) [2023-12-27 03:27:26,746][105692] Updated weights for policy 0, policy_version 1655862 (0.0007) [2023-12-27 03:27:26,789][105620] Updated weights for policy 1, policy_version 1659221 (0.0007) [2023-12-27 03:27:26,837][105620] Updated weights for policy 1, policy_version 1659231 (0.0005) [2023-12-27 03:27:27,378][105620] Updated weights for policy 1, policy_version 1659241 (0.0008) [2023-12-27 03:27:27,436][105620] Updated weights for policy 1, policy_version 1659251 (0.0010) [2023-12-27 03:27:27,491][105620] Updated weights for policy 1, policy_version 1659261 (0.0010) [2023-12-27 03:27:27,596][105692] Updated weights for policy 0, policy_version 1655873 (0.0008) [2023-12-27 03:27:27,648][105692] Updated weights for policy 0, policy_version 1655883 (0.0007) [2023-12-27 03:27:27,699][105692] Updated weights for policy 0, policy_version 1655893 (0.0005) [2023-12-27 03:27:27,745][105692] Updated weights for policy 0, policy_version 1655903 (0.0005) [2023-12-27 03:27:28,212][105620] Updated weights for policy 1, policy_version 1659271 (0.0010) [2023-12-27 03:27:28,263][105620] Updated weights for policy 1, policy_version 1659281 (0.0010) [2023-12-27 03:27:28,311][105620] Updated weights for policy 1, policy_version 1659291 (0.0010) [2023-12-27 03:27:28,375][105692] Updated weights for policy 0, policy_version 1655913 (0.0007) [2023-12-27 03:27:28,431][105692] Updated weights for policy 0, policy_version 1655923 (0.0005) [2023-12-27 03:27:28,488][105692] Updated weights for policy 0, policy_version 1655933 (0.0005) [2023-12-27 03:27:29,011][105620] Updated weights for policy 1, policy_version 1659301 (0.0009) [2023-12-27 03:27:29,059][105620] Updated weights for policy 1, policy_version 1659311 (0.0010) [2023-12-27 03:27:29,085][105692] Updated weights for policy 0, policy_version 1655943 (0.0005) [2023-12-27 03:27:29,103][105620] Updated weights for policy 1, policy_version 1659321 (0.0010) [2023-12-27 03:27:29,129][105692] Updated weights for policy 0, policy_version 1655953 (0.0005) [2023-12-27 03:27:29,176][105692] Updated weights for policy 0, policy_version 1655963 (0.0008) [2023-12-27 03:27:29,857][105620] Updated weights for policy 1, policy_version 1659331 (0.0010) [2023-12-27 03:27:29,923][105620] Updated weights for policy 1, policy_version 1659341 (0.0006) [2023-12-27 03:27:29,959][105692] Updated weights for policy 0, policy_version 1655973 (0.0007) [2023-12-27 03:27:29,983][105620] Updated weights for policy 1, policy_version 1659351 (0.0007) [2023-12-27 03:27:30,022][105692] Updated weights for policy 0, policy_version 1655983 (0.0009) [2023-12-27 03:27:30,078][105692] Updated weights for policy 0, policy_version 1655993 (0.0009) [2023-12-27 03:27:30,609][105620] Updated weights for policy 1, policy_version 1659361 (0.0006) [2023-12-27 03:27:30,654][105620] Updated weights for policy 1, policy_version 1659371 (0.0010) [2023-12-27 03:27:30,713][105620] Updated weights for policy 1, policy_version 1659381 (0.0009) [2023-12-27 03:27:30,762][105620] Updated weights for policy 1, policy_version 1659391 (0.0010) [2023-12-27 03:27:30,894][105692] Updated weights for policy 0, policy_version 1656003 (0.0009) [2023-12-27 03:27:30,946][105692] Updated weights for policy 0, policy_version 1656013 (0.0008) [2023-12-27 03:27:31,012][105692] Updated weights for policy 0, policy_version 1656023 (0.0008) [2023-12-27 03:27:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 848863232. Throughput: 0: 9611.7, 1: 10177.4. Samples: 848836948. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:27:31,063][104569] Avg episode reward: [(0, '8252.540'), (1, '8901.847')] [2023-12-27 03:27:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001659392_424861696.pth... [2023-12-27 03:27:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001656032_424009728.pth... [2023-12-27 03:27:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001658240_424566784.pth [2023-12-27 03:27:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001654848_423706624.pth [2023-12-27 03:27:31,482][105620] Updated weights for policy 1, policy_version 1659401 (0.0006) [2023-12-27 03:27:31,542][105620] Updated weights for policy 1, policy_version 1659411 (0.0005) [2023-12-27 03:27:31,604][105620] Updated weights for policy 1, policy_version 1659421 (0.0009) [2023-12-27 03:27:31,830][105692] Updated weights for policy 0, policy_version 1656033 (0.0009) [2023-12-27 03:27:31,880][105692] Updated weights for policy 0, policy_version 1656043 (0.0007) [2023-12-27 03:27:31,931][105692] Updated weights for policy 0, policy_version 1656053 (0.0007) [2023-12-27 03:27:31,978][105692] Updated weights for policy 0, policy_version 1656063 (0.0008) [2023-12-27 03:27:32,261][105620] Updated weights for policy 1, policy_version 1659431 (0.0009) [2023-12-27 03:27:32,317][105620] Updated weights for policy 1, policy_version 1659441 (0.0011) [2023-12-27 03:27:32,370][105620] Updated weights for policy 1, policy_version 1659451 (0.0011) [2023-12-27 03:27:32,822][105692] Updated weights for policy 0, policy_version 1656073 (0.0008) [2023-12-27 03:27:32,880][105692] Updated weights for policy 0, policy_version 1656083 (0.0008) [2023-12-27 03:27:32,947][105692] Updated weights for policy 0, policy_version 1656093 (0.0009) [2023-12-27 03:27:33,003][105620] Updated weights for policy 1, policy_version 1659461 (0.0008) [2023-12-27 03:27:33,046][105620] Updated weights for policy 1, policy_version 1659471 (0.0009) [2023-12-27 03:27:33,094][105620] Updated weights for policy 1, policy_version 1659481 (0.0010) [2023-12-27 03:27:33,674][105692] Updated weights for policy 0, policy_version 1656103 (0.0009) [2023-12-27 03:27:33,728][105692] Updated weights for policy 0, policy_version 1656113 (0.0008) [2023-12-27 03:27:33,778][105692] Updated weights for policy 0, policy_version 1656123 (0.0008) [2023-12-27 03:27:33,836][105620] Updated weights for policy 1, policy_version 1659491 (0.0010) [2023-12-27 03:27:33,893][105620] Updated weights for policy 1, policy_version 1659501 (0.0010) [2023-12-27 03:27:33,944][105620] Updated weights for policy 1, policy_version 1659511 (0.0010) [2023-12-27 03:27:34,423][105692] Updated weights for policy 0, policy_version 1656133 (0.0007) [2023-12-27 03:27:34,478][105692] Updated weights for policy 0, policy_version 1656143 (0.0009) [2023-12-27 03:27:34,535][105692] Updated weights for policy 0, policy_version 1656153 (0.0010) [2023-12-27 03:27:34,639][105620] Updated weights for policy 1, policy_version 1659521 (0.0010) [2023-12-27 03:27:34,695][105620] Updated weights for policy 1, policy_version 1659531 (0.0011) [2023-12-27 03:27:34,756][105620] Updated weights for policy 1, policy_version 1659541 (0.0011) [2023-12-27 03:27:34,814][105620] Updated weights for policy 1, policy_version 1659551 (0.0011) [2023-12-27 03:27:35,244][105692] Updated weights for policy 0, policy_version 1656163 (0.0008) [2023-12-27 03:27:35,294][105692] Updated weights for policy 0, policy_version 1656173 (0.0005) [2023-12-27 03:27:35,348][105692] Updated weights for policy 0, policy_version 1656183 (0.0008) [2023-12-27 03:27:35,560][105620] Updated weights for policy 1, policy_version 1659561 (0.0010) [2023-12-27 03:27:35,612][105620] Updated weights for policy 1, policy_version 1659572 (0.0009) [2023-12-27 03:27:35,674][105620] Updated weights for policy 1, policy_version 1659583 (0.0010) [2023-12-27 03:27:35,895][105692] Updated weights for policy 0, policy_version 1656193 (0.0009) [2023-12-27 03:27:35,959][105692] Updated weights for policy 0, policy_version 1656203 (0.0006) [2023-12-27 03:27:36,022][105692] Updated weights for policy 0, policy_version 1656213 (0.0009) [2023-12-27 03:27:36,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19661.0, 300 sec: 19494.2). Total num frames: 848961536. Throughput: 0: 9697.6, 1: 10050.4. Samples: 848953216. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:27:36,062][104569] Avg episode reward: [(0, '8254.591'), (1, '8805.577')] [2023-12-27 03:27:36,091][105692] Updated weights for policy 0, policy_version 1656223 (0.0009) [2023-12-27 03:27:36,391][105620] Updated weights for policy 1, policy_version 1659593 (0.0009) [2023-12-27 03:27:36,447][105620] Updated weights for policy 1, policy_version 1659603 (0.0009) [2023-12-27 03:27:36,510][105620] Updated weights for policy 1, policy_version 1659613 (0.0010) [2023-12-27 03:27:36,803][105692] Updated weights for policy 0, policy_version 1656233 (0.0009) [2023-12-27 03:27:36,876][105692] Updated weights for policy 0, policy_version 1656243 (0.0009) [2023-12-27 03:27:36,941][105692] Updated weights for policy 0, policy_version 1656253 (0.0010) [2023-12-27 03:27:37,224][105620] Updated weights for policy 1, policy_version 1659623 (0.0009) [2023-12-27 03:27:37,271][105620] Updated weights for policy 1, policy_version 1659633 (0.0009) [2023-12-27 03:27:37,319][105620] Updated weights for policy 1, policy_version 1659644 (0.0009) [2023-12-27 03:27:37,701][105692] Updated weights for policy 0, policy_version 1656263 (0.0009) [2023-12-27 03:27:37,800][105692] Updated weights for policy 0, policy_version 1656273 (0.0009) [2023-12-27 03:27:37,858][105692] Updated weights for policy 0, policy_version 1656283 (0.0009) [2023-12-27 03:27:38,095][105620] Updated weights for policy 1, policy_version 1659654 (0.0008) [2023-12-27 03:27:38,161][105620] Updated weights for policy 1, policy_version 1659664 (0.0008) [2023-12-27 03:27:38,218][105620] Updated weights for policy 1, policy_version 1659674 (0.0010) [2023-12-27 03:27:38,555][105692] Updated weights for policy 0, policy_version 1656293 (0.0008) [2023-12-27 03:27:38,616][105692] Updated weights for policy 0, policy_version 1656303 (0.0009) [2023-12-27 03:27:38,671][105692] Updated weights for policy 0, policy_version 1656313 (0.0009) [2023-12-27 03:27:38,956][105620] Updated weights for policy 1, policy_version 1659684 (0.0009) [2023-12-27 03:27:39,018][105620] Updated weights for policy 1, policy_version 1659694 (0.0009) [2023-12-27 03:27:39,080][105620] Updated weights for policy 1, policy_version 1659704 (0.0009) [2023-12-27 03:27:39,449][105692] Updated weights for policy 0, policy_version 1656323 (0.0009) [2023-12-27 03:27:39,516][105692] Updated weights for policy 0, policy_version 1656333 (0.0009) [2023-12-27 03:27:39,581][105692] Updated weights for policy 0, policy_version 1656343 (0.0009) [2023-12-27 03:27:39,816][105620] Updated weights for policy 1, policy_version 1659714 (0.0008) [2023-12-27 03:27:39,884][105620] Updated weights for policy 1, policy_version 1659724 (0.0008) [2023-12-27 03:27:39,953][105620] Updated weights for policy 1, policy_version 1659734 (0.0008) [2023-12-27 03:27:40,018][105620] Updated weights for policy 1, policy_version 1659744 (0.0008) [2023-12-27 03:27:40,309][105692] Updated weights for policy 0, policy_version 1656353 (0.0009) [2023-12-27 03:27:40,373][105692] Updated weights for policy 0, policy_version 1656363 (0.0008) [2023-12-27 03:27:40,436][105692] Updated weights for policy 0, policy_version 1656373 (0.0009) [2023-12-27 03:27:40,495][105692] Updated weights for policy 0, policy_version 1656383 (0.0009) [2023-12-27 03:27:40,759][105620] Updated weights for policy 1, policy_version 1659754 (0.0006) [2023-12-27 03:27:40,820][105620] Updated weights for policy 1, policy_version 1659764 (0.0007) [2023-12-27 03:27:40,878][105620] Updated weights for policy 1, policy_version 1659774 (0.0008) [2023-12-27 03:27:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 849059840. Throughput: 0: 9766.0, 1: 9916.3. Samples: 849067964. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:27:41,062][104569] Avg episode reward: [(0, '7892.906'), (1, '8806.502')] [2023-12-27 03:27:41,275][105692] Updated weights for policy 0, policy_version 1656393 (0.0009) [2023-12-27 03:27:41,340][105692] Updated weights for policy 0, policy_version 1656403 (0.0009) [2023-12-27 03:27:41,404][105692] Updated weights for policy 0, policy_version 1656413 (0.0010) [2023-12-27 03:27:41,653][105620] Updated weights for policy 1, policy_version 1659784 (0.0009) [2023-12-27 03:27:41,713][105620] Updated weights for policy 1, policy_version 1659794 (0.0008) [2023-12-27 03:27:41,774][105620] Updated weights for policy 1, policy_version 1659804 (0.0008) [2023-12-27 03:27:42,168][105692] Updated weights for policy 0, policy_version 1656423 (0.0010) [2023-12-27 03:27:42,219][105692] Updated weights for policy 0, policy_version 1656433 (0.0009) [2023-12-27 03:27:42,268][105692] Updated weights for policy 0, policy_version 1656443 (0.0008) [2023-12-27 03:27:42,542][105620] Updated weights for policy 1, policy_version 1659814 (0.0010) [2023-12-27 03:27:42,606][105620] Updated weights for policy 1, policy_version 1659824 (0.0010) [2023-12-27 03:27:42,676][105620] Updated weights for policy 1, policy_version 1659834 (0.0005) [2023-12-27 03:27:43,067][105692] Updated weights for policy 0, policy_version 1656453 (0.0008) [2023-12-27 03:27:43,113][105692] Updated weights for policy 0, policy_version 1656463 (0.0005) [2023-12-27 03:27:43,166][105692] Updated weights for policy 0, policy_version 1656473 (0.0006) [2023-12-27 03:27:43,236][105620] Updated weights for policy 1, policy_version 1659844 (0.0007) [2023-12-27 03:27:43,297][105620] Updated weights for policy 1, policy_version 1659854 (0.0009) [2023-12-27 03:27:43,352][105620] Updated weights for policy 1, policy_version 1659864 (0.0009) [2023-12-27 03:27:43,895][105692] Updated weights for policy 0, policy_version 1656484 (0.0009) [2023-12-27 03:27:43,950][105692] Updated weights for policy 0, policy_version 1656494 (0.0006) [2023-12-27 03:27:43,999][105692] Updated weights for policy 0, policy_version 1656504 (0.0005) [2023-12-27 03:27:44,152][105620] Updated weights for policy 1, policy_version 1659874 (0.0009) [2023-12-27 03:27:44,203][105620] Updated weights for policy 1, policy_version 1659884 (0.0010) [2023-12-27 03:27:44,256][105620] Updated weights for policy 1, policy_version 1659894 (0.0010) [2023-12-27 03:27:44,293][105586] KL-divergence is very high: 118.2297 [2023-12-27 03:27:44,307][105620] Updated weights for policy 1, policy_version 1659904 (0.0010) [2023-12-27 03:27:44,738][105692] Updated weights for policy 0, policy_version 1656514 (0.0008) [2023-12-27 03:27:44,803][105692] Updated weights for policy 0, policy_version 1656524 (0.0009) [2023-12-27 03:27:44,860][105692] Updated weights for policy 0, policy_version 1656534 (0.0011) [2023-12-27 03:27:44,919][105692] Updated weights for policy 0, policy_version 1656544 (0.0010) [2023-12-27 03:27:45,069][105620] Updated weights for policy 1, policy_version 1659914 (0.0008) [2023-12-27 03:27:45,136][105620] Updated weights for policy 1, policy_version 1659924 (0.0008) [2023-12-27 03:27:45,201][105620] Updated weights for policy 1, policy_version 1659934 (0.0008) [2023-12-27 03:27:45,601][105692] Updated weights for policy 0, policy_version 1656554 (0.0005) [2023-12-27 03:27:45,663][105692] Updated weights for policy 0, policy_version 1656564 (0.0006) [2023-12-27 03:27:45,726][105692] Updated weights for policy 0, policy_version 1656574 (0.0005) [2023-12-27 03:27:45,993][105620] Updated weights for policy 1, policy_version 1659944 (0.0006) [2023-12-27 03:27:46,060][105620] Updated weights for policy 1, policy_version 1659954 (0.0005) [2023-12-27 03:27:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 849149952. Throughput: 0: 9772.7, 1: 9865.3. Samples: 849124348. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:27:46,063][104569] Avg episode reward: [(0, '8167.466'), (1, '8987.656')] [2023-12-27 03:27:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001656576_424148992.pth... [2023-12-27 03:27:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001655456_423862272.pth [2023-12-27 03:27:46,124][105620] Updated weights for policy 1, policy_version 1659964 (0.0005) [2023-12-27 03:27:46,147][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001659968_425009152.pth... [2023-12-27 03:27:46,152][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001658784_424706048.pth [2023-12-27 03:27:46,353][105692] Updated weights for policy 0, policy_version 1656584 (0.0005) [2023-12-27 03:27:46,410][105692] Updated weights for policy 0, policy_version 1656594 (0.0005) [2023-12-27 03:27:46,469][105692] Updated weights for policy 0, policy_version 1656604 (0.0005) [2023-12-27 03:27:46,767][105620] Updated weights for policy 1, policy_version 1659974 (0.0007) [2023-12-27 03:27:46,818][105620] Updated weights for policy 1, policy_version 1659984 (0.0008) [2023-12-27 03:27:46,862][105620] Updated weights for policy 1, policy_version 1659994 (0.0008) [2023-12-27 03:27:47,126][105692] Updated weights for policy 0, policy_version 1656614 (0.0009) [2023-12-27 03:27:47,184][105692] Updated weights for policy 0, policy_version 1656624 (0.0010) [2023-12-27 03:27:47,243][105692] Updated weights for policy 0, policy_version 1656634 (0.0010) [2023-12-27 03:27:47,610][105620] Updated weights for policy 1, policy_version 1660004 (0.0009) [2023-12-27 03:27:47,668][105620] Updated weights for policy 1, policy_version 1660014 (0.0007) [2023-12-27 03:27:47,727][105620] Updated weights for policy 1, policy_version 1660024 (0.0010) [2023-12-27 03:27:47,913][105692] Updated weights for policy 0, policy_version 1656644 (0.0010) [2023-12-27 03:27:47,980][105692] Updated weights for policy 0, policy_version 1656654 (0.0006) [2023-12-27 03:27:48,043][105692] Updated weights for policy 0, policy_version 1656664 (0.0008) [2023-12-27 03:27:48,497][105620] Updated weights for policy 1, policy_version 1660034 (0.0010) [2023-12-27 03:27:48,555][105620] Updated weights for policy 1, policy_version 1660044 (0.0010) [2023-12-27 03:27:48,603][105620] Updated weights for policy 1, policy_version 1660054 (0.0010) [2023-12-27 03:27:48,654][105620] Updated weights for policy 1, policy_version 1660064 (0.0010) [2023-12-27 03:27:48,729][105692] Updated weights for policy 0, policy_version 1656674 (0.0008) [2023-12-27 03:27:48,780][105692] Updated weights for policy 0, policy_version 1656684 (0.0006) [2023-12-27 03:27:48,828][105692] Updated weights for policy 0, policy_version 1656694 (0.0005) [2023-12-27 03:27:48,891][105692] Updated weights for policy 0, policy_version 1656704 (0.0005) [2023-12-27 03:27:49,438][105620] Updated weights for policy 1, policy_version 1660074 (0.0009) [2023-12-27 03:27:49,461][105692] Updated weights for policy 0, policy_version 1656714 (0.0008) [2023-12-27 03:27:49,501][105620] Updated weights for policy 1, policy_version 1660084 (0.0008) [2023-12-27 03:27:49,516][105692] Updated weights for policy 0, policy_version 1656724 (0.0008) [2023-12-27 03:27:49,556][105620] Updated weights for policy 1, policy_version 1660094 (0.0010) [2023-12-27 03:27:49,575][105692] Updated weights for policy 0, policy_version 1656734 (0.0007) [2023-12-27 03:27:50,229][105620] Updated weights for policy 1, policy_version 1660104 (0.0007) [2023-12-27 03:27:50,294][105620] Updated weights for policy 1, policy_version 1660114 (0.0009) [2023-12-27 03:27:50,329][105692] Updated weights for policy 0, policy_version 1656744 (0.0006) [2023-12-27 03:27:50,357][105620] Updated weights for policy 1, policy_version 1660124 (0.0008) [2023-12-27 03:27:50,381][105692] Updated weights for policy 0, policy_version 1656754 (0.0008) [2023-12-27 03:27:50,433][105692] Updated weights for policy 0, policy_version 1656764 (0.0010) [2023-12-27 03:27:51,019][105620] Updated weights for policy 1, policy_version 1660134 (0.0008) [2023-12-27 03:27:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 849248256. Throughput: 0: 9813.0, 1: 9836.6. Samples: 849242088. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:27:51,063][104569] Avg episode reward: [(0, '8713.138'), (1, '8989.374')] [2023-12-27 03:27:51,089][105620] Updated weights for policy 1, policy_version 1660144 (0.0007) [2023-12-27 03:27:51,154][105620] Updated weights for policy 1, policy_version 1660154 (0.0006) [2023-12-27 03:27:51,287][105692] Updated weights for policy 0, policy_version 1656774 (0.0009) [2023-12-27 03:27:51,341][105692] Updated weights for policy 0, policy_version 1656784 (0.0009) [2023-12-27 03:27:51,410][105692] Updated weights for policy 0, policy_version 1656794 (0.0007) [2023-12-27 03:27:51,858][105620] Updated weights for policy 1, policy_version 1660164 (0.0006) [2023-12-27 03:27:51,922][105620] Updated weights for policy 1, policy_version 1660174 (0.0008) [2023-12-27 03:27:51,987][105620] Updated weights for policy 1, policy_version 1660184 (0.0008) [2023-12-27 03:27:52,131][105692] Updated weights for policy 0, policy_version 1656804 (0.0007) [2023-12-27 03:27:52,186][105692] Updated weights for policy 0, policy_version 1656814 (0.0009) [2023-12-27 03:27:52,245][105692] Updated weights for policy 0, policy_version 1656824 (0.0009) [2023-12-27 03:27:52,610][105620] Updated weights for policy 1, policy_version 1660194 (0.0008) [2023-12-27 03:27:52,662][105620] Updated weights for policy 1, policy_version 1660204 (0.0008) [2023-12-27 03:27:52,709][105620] Updated weights for policy 1, policy_version 1660214 (0.0008) [2023-12-27 03:27:52,758][105620] Updated weights for policy 1, policy_version 1660224 (0.0008) [2023-12-27 03:27:53,061][105692] Updated weights for policy 0, policy_version 1656834 (0.0009) [2023-12-27 03:27:53,131][105692] Updated weights for policy 0, policy_version 1656844 (0.0009) [2023-12-27 03:27:53,187][105692] Updated weights for policy 0, policy_version 1656854 (0.0009) [2023-12-27 03:27:53,242][105692] Updated weights for policy 0, policy_version 1656864 (0.0009) [2023-12-27 03:27:53,520][105620] Updated weights for policy 1, policy_version 1660234 (0.0009) [2023-12-27 03:27:53,582][105620] Updated weights for policy 1, policy_version 1660244 (0.0009) [2023-12-27 03:27:53,637][105620] Updated weights for policy 1, policy_version 1660254 (0.0009) [2023-12-27 03:27:53,919][105692] Updated weights for policy 0, policy_version 1656874 (0.0010) [2023-12-27 03:27:53,971][105692] Updated weights for policy 0, policy_version 1656886 (0.0010) [2023-12-27 03:27:54,017][105692] Updated weights for policy 0, policy_version 1656896 (0.0005) [2023-12-27 03:27:54,280][105620] Updated weights for policy 1, policy_version 1660264 (0.0010) [2023-12-27 03:27:54,345][105620] Updated weights for policy 1, policy_version 1660274 (0.0010) [2023-12-27 03:27:54,403][105620] Updated weights for policy 1, policy_version 1660284 (0.0010) [2023-12-27 03:27:54,651][105692] Updated weights for policy 0, policy_version 1656906 (0.0009) [2023-12-27 03:27:54,695][105692] Updated weights for policy 0, policy_version 1656916 (0.0008) [2023-12-27 03:27:54,740][105692] Updated weights for policy 0, policy_version 1656926 (0.0009) [2023-12-27 03:27:55,139][105620] Updated weights for policy 1, policy_version 1660294 (0.0010) [2023-12-27 03:27:55,197][105620] Updated weights for policy 1, policy_version 1660304 (0.0010) [2023-12-27 03:27:55,251][105620] Updated weights for policy 1, policy_version 1660314 (0.0010) [2023-12-27 03:27:55,426][105692] Updated weights for policy 0, policy_version 1656936 (0.0006) [2023-12-27 03:27:55,472][105692] Updated weights for policy 0, policy_version 1656946 (0.0005) [2023-12-27 03:27:55,517][105692] Updated weights for policy 0, policy_version 1656956 (0.0007) [2023-12-27 03:27:55,959][105620] Updated weights for policy 1, policy_version 1660324 (0.0008) [2023-12-27 03:27:56,007][105620] Updated weights for policy 1, policy_version 1660334 (0.0005) [2023-12-27 03:27:56,059][105692] Updated weights for policy 0, policy_version 1656966 (0.0007) [2023-12-27 03:27:56,061][105620] Updated weights for policy 1, policy_version 1660344 (0.0010) [2023-12-27 03:27:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 849346560. Throughput: 0: 9666.3, 1: 9887.6. Samples: 849359920. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:27:56,062][104569] Avg episode reward: [(0, '8620.169'), (1, '8626.900')] [2023-12-27 03:27:56,114][105692] Updated weights for policy 0, policy_version 1656976 (0.0006) [2023-12-27 03:27:56,183][105692] Updated weights for policy 0, policy_version 1656986 (0.0008) [2023-12-27 03:27:56,777][105620] Updated weights for policy 1, policy_version 1660354 (0.0010) [2023-12-27 03:27:56,828][105620] Updated weights for policy 1, policy_version 1660364 (0.0010) [2023-12-27 03:27:56,881][105620] Updated weights for policy 1, policy_version 1660374 (0.0010) [2023-12-27 03:27:56,897][105692] Updated weights for policy 0, policy_version 1656996 (0.0006) [2023-12-27 03:27:56,939][105620] Updated weights for policy 1, policy_version 1660384 (0.0010) [2023-12-27 03:27:56,946][105692] Updated weights for policy 0, policy_version 1657006 (0.0005) [2023-12-27 03:27:56,997][105692] Updated weights for policy 0, policy_version 1657016 (0.0006) [2023-12-27 03:27:57,629][105620] Updated weights for policy 1, policy_version 1660394 (0.0010) [2023-12-27 03:27:57,677][105620] Updated weights for policy 1, policy_version 1660404 (0.0010) [2023-12-27 03:27:57,715][105692] Updated weights for policy 0, policy_version 1657026 (0.0009) [2023-12-27 03:27:57,729][105620] Updated weights for policy 1, policy_version 1660414 (0.0010) [2023-12-27 03:27:57,774][105692] Updated weights for policy 0, policy_version 1657036 (0.0007) [2023-12-27 03:27:57,826][105692] Updated weights for policy 0, policy_version 1657046 (0.0010) [2023-12-27 03:27:58,343][105620] Updated weights for policy 1, policy_version 1660424 (0.0008) [2023-12-27 03:27:58,406][105620] Updated weights for policy 1, policy_version 1660434 (0.0007) [2023-12-27 03:27:58,472][105620] Updated weights for policy 1, policy_version 1660444 (0.0008) [2023-12-27 03:27:58,749][105692] Updated weights for policy 0, policy_version 1657058 (0.0011) [2023-12-27 03:27:58,815][105692] Updated weights for policy 0, policy_version 1657068 (0.0009) [2023-12-27 03:27:58,884][105692] Updated weights for policy 0, policy_version 1657078 (0.0009) [2023-12-27 03:27:58,947][105692] Updated weights for policy 0, policy_version 1657088 (0.0010) [2023-12-27 03:27:59,217][105620] Updated weights for policy 1, policy_version 1660454 (0.0010) [2023-12-27 03:27:59,278][105620] Updated weights for policy 1, policy_version 1660464 (0.0008) [2023-12-27 03:27:59,337][105620] Updated weights for policy 1, policy_version 1660474 (0.0008) [2023-12-27 03:27:59,756][105692] Updated weights for policy 0, policy_version 1657098 (0.0005) [2023-12-27 03:27:59,816][105692] Updated weights for policy 0, policy_version 1657108 (0.0006) [2023-12-27 03:27:59,878][105692] Updated weights for policy 0, policy_version 1657118 (0.0007) [2023-12-27 03:28:00,101][105620] Updated weights for policy 1, policy_version 1660484 (0.0009) [2023-12-27 03:28:00,157][105620] Updated weights for policy 1, policy_version 1660494 (0.0010) [2023-12-27 03:28:00,205][105620] Updated weights for policy 1, policy_version 1660504 (0.0010) [2023-12-27 03:28:00,442][105692] Updated weights for policy 0, policy_version 1657128 (0.0005) [2023-12-27 03:28:00,497][105692] Updated weights for policy 0, policy_version 1657138 (0.0005) [2023-12-27 03:28:00,552][105692] Updated weights for policy 0, policy_version 1657148 (0.0005) [2023-12-27 03:28:00,968][105620] Updated weights for policy 1, policy_version 1660514 (0.0010) [2023-12-27 03:28:01,034][105620] Updated weights for policy 1, policy_version 1660524 (0.0006) [2023-12-27 03:28:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 849444864. Throughput: 0: 9732.4, 1: 9892.4. Samples: 849419864. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:28:01,063][104569] Avg episode reward: [(0, '8347.405'), (1, '8626.015')] [2023-12-27 03:28:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001657152_424296448.pth... [2023-12-27 03:28:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001656032_424009728.pth [2023-12-27 03:28:01,098][105620] Updated weights for policy 1, policy_version 1660534 (0.0008) [2023-12-27 03:28:01,141][105692] Updated weights for policy 0, policy_version 1657158 (0.0007) [2023-12-27 03:28:01,167][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001660544_425156608.pth... [2023-12-27 03:28:01,167][105620] Updated weights for policy 1, policy_version 1660544 (0.0009) [2023-12-27 03:28:01,173][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001659392_424861696.pth [2023-12-27 03:28:01,207][105692] Updated weights for policy 0, policy_version 1657168 (0.0006) [2023-12-27 03:28:01,272][105692] Updated weights for policy 0, policy_version 1657178 (0.0007) [2023-12-27 03:28:01,855][105620] Updated weights for policy 1, policy_version 1660554 (0.0005) [2023-12-27 03:28:01,903][105620] Updated weights for policy 1, policy_version 1660564 (0.0008) [2023-12-27 03:28:01,955][105620] Updated weights for policy 1, policy_version 1660574 (0.0005) [2023-12-27 03:28:02,025][105692] Updated weights for policy 0, policy_version 1657188 (0.0009) [2023-12-27 03:28:02,079][105692] Updated weights for policy 0, policy_version 1657199 (0.0010) [2023-12-27 03:28:02,135][105692] Updated weights for policy 0, policy_version 1657209 (0.0010) [2023-12-27 03:28:02,629][105620] Updated weights for policy 1, policy_version 1660584 (0.0008) [2023-12-27 03:28:02,687][105620] Updated weights for policy 1, policy_version 1660594 (0.0009) [2023-12-27 03:28:02,737][105620] Updated weights for policy 1, policy_version 1660604 (0.0009) [2023-12-27 03:28:02,850][105692] Updated weights for policy 0, policy_version 1657219 (0.0009) [2023-12-27 03:28:02,915][105692] Updated weights for policy 0, policy_version 1657229 (0.0005) [2023-12-27 03:28:02,988][105692] Updated weights for policy 0, policy_version 1657239 (0.0005) [2023-12-27 03:28:03,418][105620] Updated weights for policy 1, policy_version 1660614 (0.0008) [2023-12-27 03:28:03,463][105620] Updated weights for policy 1, policy_version 1660624 (0.0009) [2023-12-27 03:28:03,510][105620] Updated weights for policy 1, policy_version 1660634 (0.0008) [2023-12-27 03:28:03,667][105692] Updated weights for policy 0, policy_version 1657249 (0.0007) [2023-12-27 03:28:03,714][105692] Updated weights for policy 0, policy_version 1657259 (0.0009) [2023-12-27 03:28:03,761][105692] Updated weights for policy 0, policy_version 1657269 (0.0009) [2023-12-27 03:28:03,809][105692] Updated weights for policy 0, policy_version 1657279 (0.0009) [2023-12-27 03:28:04,216][105620] Updated weights for policy 1, policy_version 1660644 (0.0009) [2023-12-27 03:28:04,272][105620] Updated weights for policy 1, policy_version 1660654 (0.0010) [2023-12-27 03:28:04,329][105620] Updated weights for policy 1, policy_version 1660664 (0.0009) [2023-12-27 03:28:04,652][105692] Updated weights for policy 0, policy_version 1657289 (0.0009) [2023-12-27 03:28:04,717][105692] Updated weights for policy 0, policy_version 1657299 (0.0009) [2023-12-27 03:28:04,776][105692] Updated weights for policy 0, policy_version 1657309 (0.0009) [2023-12-27 03:28:05,033][105620] Updated weights for policy 1, policy_version 1660674 (0.0008) [2023-12-27 03:28:05,093][105620] Updated weights for policy 1, policy_version 1660684 (0.0005) [2023-12-27 03:28:05,151][105620] Updated weights for policy 1, policy_version 1660694 (0.0005) [2023-12-27 03:28:05,214][105620] Updated weights for policy 1, policy_version 1660704 (0.0007) [2023-12-27 03:28:05,534][105692] Updated weights for policy 0, policy_version 1657319 (0.0008) [2023-12-27 03:28:05,591][105692] Updated weights for policy 0, policy_version 1657329 (0.0007) [2023-12-27 03:28:05,647][105692] Updated weights for policy 0, policy_version 1657339 (0.0005) [2023-12-27 03:28:05,827][105620] Updated weights for policy 1, policy_version 1660714 (0.0010) [2023-12-27 03:28:05,883][105620] Updated weights for policy 1, policy_version 1660724 (0.0010) [2023-12-27 03:28:05,931][105620] Updated weights for policy 1, policy_version 1660734 (0.0010) [2023-12-27 03:28:06,062][104569] Fps is (10 sec: 20479.4, 60 sec: 19660.7, 300 sec: 19494.2). Total num frames: 849551360. Throughput: 0: 9697.5, 1: 9854.1. Samples: 849537224. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:28:06,063][104569] Avg episode reward: [(0, '8621.160'), (1, '8991.750')] [2023-12-27 03:28:06,356][105692] Updated weights for policy 0, policy_version 1657349 (0.0005) [2023-12-27 03:28:06,426][105692] Updated weights for policy 0, policy_version 1657359 (0.0008) [2023-12-27 03:28:06,492][105692] Updated weights for policy 0, policy_version 1657369 (0.0008) [2023-12-27 03:28:06,622][105620] Updated weights for policy 1, policy_version 1660744 (0.0009) [2023-12-27 03:28:06,675][105620] Updated weights for policy 1, policy_version 1660754 (0.0009) [2023-12-27 03:28:06,726][105620] Updated weights for policy 1, policy_version 1660764 (0.0009) [2023-12-27 03:28:07,168][105692] Updated weights for policy 0, policy_version 1657379 (0.0008) [2023-12-27 03:28:07,239][105692] Updated weights for policy 0, policy_version 1657389 (0.0007) [2023-12-27 03:28:07,305][105692] Updated weights for policy 0, policy_version 1657399 (0.0009) [2023-12-27 03:28:07,516][105620] Updated weights for policy 1, policy_version 1660774 (0.0009) [2023-12-27 03:28:07,582][105620] Updated weights for policy 1, policy_version 1660784 (0.0010) [2023-12-27 03:28:07,639][105620] Updated weights for policy 1, policy_version 1660794 (0.0009) [2023-12-27 03:28:07,861][105692] Updated weights for policy 0, policy_version 1657409 (0.0008) [2023-12-27 03:28:07,918][105692] Updated weights for policy 0, policy_version 1657419 (0.0005) [2023-12-27 03:28:07,973][105692] Updated weights for policy 0, policy_version 1657429 (0.0005) [2023-12-27 03:28:08,035][105692] Updated weights for policy 0, policy_version 1657439 (0.0007) [2023-12-27 03:28:08,468][105620] Updated weights for policy 1, policy_version 1660804 (0.0009) [2023-12-27 03:28:08,527][105620] Updated weights for policy 1, policy_version 1660814 (0.0009) [2023-12-27 03:28:08,578][105620] Updated weights for policy 1, policy_version 1660824 (0.0009) [2023-12-27 03:28:08,749][105692] Updated weights for policy 0, policy_version 1657449 (0.0009) [2023-12-27 03:28:08,810][105692] Updated weights for policy 0, policy_version 1657459 (0.0006) [2023-12-27 03:28:08,870][105692] Updated weights for policy 0, policy_version 1657469 (0.0005) [2023-12-27 03:28:09,389][105620] Updated weights for policy 1, policy_version 1660834 (0.0008) [2023-12-27 03:28:09,454][105620] Updated weights for policy 1, policy_version 1660844 (0.0008) [2023-12-27 03:28:09,501][105620] Updated weights for policy 1, policy_version 1660854 (0.0008) [2023-12-27 03:28:09,535][105692] Updated weights for policy 0, policy_version 1657479 (0.0007) [2023-12-27 03:28:09,558][105620] Updated weights for policy 1, policy_version 1660864 (0.0007) [2023-12-27 03:28:09,591][105692] Updated weights for policy 0, policy_version 1657489 (0.0008) [2023-12-27 03:28:09,646][105692] Updated weights for policy 0, policy_version 1657499 (0.0009) [2023-12-27 03:28:10,312][105620] Updated weights for policy 1, policy_version 1660874 (0.0009) [2023-12-27 03:28:10,375][105620] Updated weights for policy 1, policy_version 1660884 (0.0010) [2023-12-27 03:28:10,431][105620] Updated weights for policy 1, policy_version 1660894 (0.0008) [2023-12-27 03:28:10,449][105692] Updated weights for policy 0, policy_version 1657509 (0.0009) [2023-12-27 03:28:10,508][105692] Updated weights for policy 0, policy_version 1657519 (0.0009) [2023-12-27 03:28:10,572][105692] Updated weights for policy 0, policy_version 1657529 (0.0009) [2023-12-27 03:28:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 849641472. Throughput: 0: 9711.7, 1: 9756.2. Samples: 849652632. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:28:11,062][104569] Avg episode reward: [(0, '9169.700'), (1, '8991.963')] [2023-12-27 03:28:11,147][105620] Updated weights for policy 1, policy_version 1660904 (0.0008) [2023-12-27 03:28:11,208][105620] Updated weights for policy 1, policy_version 1660914 (0.0009) [2023-12-27 03:28:11,272][105620] Updated weights for policy 1, policy_version 1660924 (0.0009) [2023-12-27 03:28:11,373][105692] Updated weights for policy 0, policy_version 1657539 (0.0008) [2023-12-27 03:28:11,442][105692] Updated weights for policy 0, policy_version 1657549 (0.0009) [2023-12-27 03:28:11,494][105692] Updated weights for policy 0, policy_version 1657559 (0.0010) [2023-12-27 03:28:12,068][105620] Updated weights for policy 1, policy_version 1660934 (0.0007) [2023-12-27 03:28:12,125][105620] Updated weights for policy 1, policy_version 1660944 (0.0006) [2023-12-27 03:28:12,186][105620] Updated weights for policy 1, policy_version 1660954 (0.0009) [2023-12-27 03:28:12,315][105692] Updated weights for policy 0, policy_version 1657569 (0.0009) [2023-12-27 03:28:12,378][105692] Updated weights for policy 0, policy_version 1657579 (0.0010) [2023-12-27 03:28:12,426][105692] Updated weights for policy 0, policy_version 1657589 (0.0008) [2023-12-27 03:28:12,474][105692] Updated weights for policy 0, policy_version 1657599 (0.0008) [2023-12-27 03:28:12,853][105620] Updated weights for policy 1, policy_version 1660964 (0.0006) [2023-12-27 03:28:12,924][105620] Updated weights for policy 1, policy_version 1660974 (0.0008) [2023-12-27 03:28:12,979][105620] Updated weights for policy 1, policy_version 1660984 (0.0010) [2023-12-27 03:28:13,334][105692] Updated weights for policy 0, policy_version 1657609 (0.0010) [2023-12-27 03:28:13,392][105692] Updated weights for policy 0, policy_version 1657619 (0.0010) [2023-12-27 03:28:13,446][105692] Updated weights for policy 0, policy_version 1657629 (0.0010) [2023-12-27 03:28:13,548][105620] Updated weights for policy 1, policy_version 1660994 (0.0010) [2023-12-27 03:28:13,612][105620] Updated weights for policy 1, policy_version 1661004 (0.0008) [2023-12-27 03:28:13,674][105620] Updated weights for policy 1, policy_version 1661014 (0.0009) [2023-12-27 03:28:13,732][105620] Updated weights for policy 1, policy_version 1661024 (0.0010) [2023-12-27 03:28:14,246][105692] Updated weights for policy 0, policy_version 1657639 (0.0009) [2023-12-27 03:28:14,296][105692] Updated weights for policy 0, policy_version 1657649 (0.0009) [2023-12-27 03:28:14,330][105620] Updated weights for policy 1, policy_version 1661034 (0.0005) [2023-12-27 03:28:14,347][105692] Updated weights for policy 0, policy_version 1657659 (0.0007) [2023-12-27 03:28:14,378][105620] Updated weights for policy 1, policy_version 1661044 (0.0005) [2023-12-27 03:28:14,427][105620] Updated weights for policy 1, policy_version 1661054 (0.0006) [2023-12-27 03:28:15,050][105620] Updated weights for policy 1, policy_version 1661064 (0.0009) [2023-12-27 03:28:15,117][105620] Updated weights for policy 1, policy_version 1661074 (0.0006) [2023-12-27 03:28:15,180][105692] Updated weights for policy 0, policy_version 1657669 (0.0008) [2023-12-27 03:28:15,184][105620] Updated weights for policy 1, policy_version 1661084 (0.0008) [2023-12-27 03:28:15,242][105692] Updated weights for policy 0, policy_version 1657679 (0.0007) [2023-12-27 03:28:15,299][105692] Updated weights for policy 0, policy_version 1657689 (0.0009) [2023-12-27 03:28:15,825][105620] Updated weights for policy 1, policy_version 1661094 (0.0010) [2023-12-27 03:28:15,887][105620] Updated weights for policy 1, policy_version 1661104 (0.0010) [2023-12-27 03:28:15,961][105620] Updated weights for policy 1, policy_version 1661114 (0.0009) [2023-12-27 03:28:15,983][105692] Updated weights for policy 0, policy_version 1657699 (0.0009) [2023-12-27 03:28:16,048][105692] Updated weights for policy 0, policy_version 1657709 (0.0009) [2023-12-27 03:28:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 849739776. Throughput: 0: 9640.6, 1: 9739.3. Samples: 849709044. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:28:16,063][104569] Avg episode reward: [(0, '8987.762'), (1, '8899.066')] [2023-12-27 03:28:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001661120_425304064.pth... [2023-12-27 03:28:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001659968_425009152.pth [2023-12-27 03:28:16,119][105692] Updated weights for policy 0, policy_version 1657719 (0.0010) [2023-12-27 03:28:16,170][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001657728_424443904.pth... [2023-12-27 03:28:16,173][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001656576_424148992.pth [2023-12-27 03:28:16,610][105620] Updated weights for policy 1, policy_version 1661124 (0.0009) [2023-12-27 03:28:16,660][105620] Updated weights for policy 1, policy_version 1661134 (0.0005) [2023-12-27 03:28:16,727][105620] Updated weights for policy 1, policy_version 1661144 (0.0007) [2023-12-27 03:28:16,738][105692] Updated weights for policy 0, policy_version 1657729 (0.0010) [2023-12-27 03:28:16,782][105692] Updated weights for policy 0, policy_version 1657739 (0.0005) [2023-12-27 03:28:16,843][105692] Updated weights for policy 0, policy_version 1657749 (0.0008) [2023-12-27 03:28:16,914][105692] Updated weights for policy 0, policy_version 1657759 (0.0010) [2023-12-27 03:28:17,311][105620] Updated weights for policy 1, policy_version 1661154 (0.0008) [2023-12-27 03:28:17,356][105620] Updated weights for policy 1, policy_version 1661164 (0.0005) [2023-12-27 03:28:17,403][105620] Updated weights for policy 1, policy_version 1661174 (0.0005) [2023-12-27 03:28:17,447][105620] Updated weights for policy 1, policy_version 1661184 (0.0010) [2023-12-27 03:28:17,507][105692] Updated weights for policy 0, policy_version 1657769 (0.0007) [2023-12-27 03:28:17,562][105692] Updated weights for policy 0, policy_version 1657779 (0.0009) [2023-12-27 03:28:17,621][105692] Updated weights for policy 0, policy_version 1657789 (0.0010) [2023-12-27 03:28:18,138][105620] Updated weights for policy 1, policy_version 1661194 (0.0010) [2023-12-27 03:28:18,200][105620] Updated weights for policy 1, policy_version 1661204 (0.0008) [2023-12-27 03:28:18,265][105620] Updated weights for policy 1, policy_version 1661214 (0.0005) [2023-12-27 03:28:18,267][105692] Updated weights for policy 0, policy_version 1657799 (0.0007) [2023-12-27 03:28:18,323][105692] Updated weights for policy 0, policy_version 1657809 (0.0006) [2023-12-27 03:28:18,385][105692] Updated weights for policy 0, policy_version 1657819 (0.0008) [2023-12-27 03:28:18,837][105620] Updated weights for policy 1, policy_version 1661224 (0.0010) [2023-12-27 03:28:18,900][105620] Updated weights for policy 1, policy_version 1661234 (0.0011) [2023-12-27 03:28:18,966][105620] Updated weights for policy 1, policy_version 1661244 (0.0011) [2023-12-27 03:28:19,066][105692] Updated weights for policy 0, policy_version 1657829 (0.0007) [2023-12-27 03:28:19,121][105692] Updated weights for policy 0, policy_version 1657839 (0.0006) [2023-12-27 03:28:19,184][105692] Updated weights for policy 0, policy_version 1657849 (0.0007) [2023-12-27 03:28:19,724][105620] Updated weights for policy 1, policy_version 1661254 (0.0011) [2023-12-27 03:28:19,787][105620] Updated weights for policy 1, policy_version 1661264 (0.0011) [2023-12-27 03:28:19,850][105620] Updated weights for policy 1, policy_version 1661274 (0.0009) [2023-12-27 03:28:19,874][105692] Updated weights for policy 0, policy_version 1657859 (0.0008) [2023-12-27 03:28:19,935][105692] Updated weights for policy 0, policy_version 1657869 (0.0007) [2023-12-27 03:28:19,999][105692] Updated weights for policy 0, policy_version 1657879 (0.0008) [2023-12-27 03:28:20,613][105620] Updated weights for policy 1, policy_version 1661284 (0.0011) [2023-12-27 03:28:20,675][105620] Updated weights for policy 1, policy_version 1661294 (0.0011) [2023-12-27 03:28:20,732][105620] Updated weights for policy 1, policy_version 1661304 (0.0011) [2023-12-27 03:28:20,774][105692] Updated weights for policy 0, policy_version 1657889 (0.0010) [2023-12-27 03:28:20,844][105692] Updated weights for policy 0, policy_version 1657899 (0.0009) [2023-12-27 03:28:20,896][105692] Updated weights for policy 0, policy_version 1657909 (0.0008) [2023-12-27 03:28:20,946][105692] Updated weights for policy 0, policy_version 1657919 (0.0008) [2023-12-27 03:28:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 849846272. Throughput: 0: 9743.3, 1: 9811.8. Samples: 849833196. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:28:21,062][104569] Avg episode reward: [(0, '8807.753'), (1, '9081.993')] [2023-12-27 03:28:21,437][105620] Updated weights for policy 1, policy_version 1661314 (0.0009) [2023-12-27 03:28:21,496][105620] Updated weights for policy 1, policy_version 1661324 (0.0006) [2023-12-27 03:28:21,564][105620] Updated weights for policy 1, policy_version 1661334 (0.0006) [2023-12-27 03:28:21,632][105620] Updated weights for policy 1, policy_version 1661344 (0.0008) [2023-12-27 03:28:21,808][105692] Updated weights for policy 0, policy_version 1657929 (0.0009) [2023-12-27 03:28:21,868][105692] Updated weights for policy 0, policy_version 1657939 (0.0009) [2023-12-27 03:28:21,922][105692] Updated weights for policy 0, policy_version 1657949 (0.0009) [2023-12-27 03:28:22,231][105620] Updated weights for policy 1, policy_version 1661354 (0.0009) [2023-12-27 03:28:22,295][105620] Updated weights for policy 1, policy_version 1661364 (0.0011) [2023-12-27 03:28:22,360][105620] Updated weights for policy 1, policy_version 1661374 (0.0010) [2023-12-27 03:28:22,819][105692] Updated weights for policy 0, policy_version 1657959 (0.0009) [2023-12-27 03:28:22,870][105692] Updated weights for policy 0, policy_version 1657969 (0.0008) [2023-12-27 03:28:22,922][105692] Updated weights for policy 0, policy_version 1657979 (0.0007) [2023-12-27 03:28:22,951][105620] Updated weights for policy 1, policy_version 1661384 (0.0008) [2023-12-27 03:28:23,012][105620] Updated weights for policy 1, policy_version 1661394 (0.0007) [2023-12-27 03:28:23,070][105620] Updated weights for policy 1, policy_version 1661404 (0.0007) [2023-12-27 03:28:23,736][105692] Updated weights for policy 0, policy_version 1657989 (0.0007) [2023-12-27 03:28:23,780][105692] Updated weights for policy 0, policy_version 1657999 (0.0006) [2023-12-27 03:28:23,790][105620] Updated weights for policy 1, policy_version 1661414 (0.0008) [2023-12-27 03:28:23,828][105692] Updated weights for policy 0, policy_version 1658009 (0.0006) [2023-12-27 03:28:23,851][105620] Updated weights for policy 1, policy_version 1661424 (0.0007) [2023-12-27 03:28:23,924][105620] Updated weights for policy 1, policy_version 1661434 (0.0009) [2023-12-27 03:28:24,471][105692] Updated weights for policy 0, policy_version 1658019 (0.0006) [2023-12-27 03:28:24,534][105692] Updated weights for policy 0, policy_version 1658029 (0.0008) [2023-12-27 03:28:24,593][105692] Updated weights for policy 0, policy_version 1658039 (0.0005) [2023-12-27 03:28:24,650][105620] Updated weights for policy 1, policy_version 1661444 (0.0008) [2023-12-27 03:28:24,719][105620] Updated weights for policy 1, policy_version 1661454 (0.0006) [2023-12-27 03:28:24,785][105620] Updated weights for policy 1, policy_version 1661464 (0.0007) [2023-12-27 03:28:25,176][105692] Updated weights for policy 0, policy_version 1658049 (0.0006) [2023-12-27 03:28:25,230][105692] Updated weights for policy 0, policy_version 1658059 (0.0005) [2023-12-27 03:28:25,288][105692] Updated weights for policy 0, policy_version 1658069 (0.0009) [2023-12-27 03:28:25,341][105692] Updated weights for policy 0, policy_version 1658079 (0.0009) [2023-12-27 03:28:25,478][105620] Updated weights for policy 1, policy_version 1661474 (0.0008) [2023-12-27 03:28:25,539][105620] Updated weights for policy 1, policy_version 1661484 (0.0009) [2023-12-27 03:28:25,605][105620] Updated weights for policy 1, policy_version 1661494 (0.0009) [2023-12-27 03:28:25,656][105620] Updated weights for policy 1, policy_version 1661504 (0.0009) [2023-12-27 03:28:25,956][105692] Updated weights for policy 0, policy_version 1658089 (0.0009) [2023-12-27 03:28:26,017][105692] Updated weights for policy 0, policy_version 1658099 (0.0009) [2023-12-27 03:28:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 849936384. Throughput: 0: 9688.7, 1: 9853.3. Samples: 849947356. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:28:26,063][104569] Avg episode reward: [(0, '8808.904'), (1, '9266.131')] [2023-12-27 03:28:26,078][105692] Updated weights for policy 0, policy_version 1658109 (0.0009) [2023-12-27 03:28:26,453][105620] Updated weights for policy 1, policy_version 1661514 (0.0006) [2023-12-27 03:28:26,511][105620] Updated weights for policy 1, policy_version 1661524 (0.0008) [2023-12-27 03:28:26,561][105620] Updated weights for policy 1, policy_version 1661534 (0.0008) [2023-12-27 03:28:26,787][105692] Updated weights for policy 0, policy_version 1658119 (0.0009) [2023-12-27 03:28:26,840][105692] Updated weights for policy 0, policy_version 1658129 (0.0010) [2023-12-27 03:28:26,900][105692] Updated weights for policy 0, policy_version 1658139 (0.0010) [2023-12-27 03:28:27,181][105620] Updated weights for policy 1, policy_version 1661544 (0.0006) [2023-12-27 03:28:27,229][105620] Updated weights for policy 1, policy_version 1661554 (0.0005) [2023-12-27 03:28:27,292][105620] Updated weights for policy 1, policy_version 1661564 (0.0005) [2023-12-27 03:28:27,546][105692] Updated weights for policy 0, policy_version 1658149 (0.0009) [2023-12-27 03:28:27,599][105692] Updated weights for policy 0, policy_version 1658159 (0.0008) [2023-12-27 03:28:27,645][105692] Updated weights for policy 0, policy_version 1658169 (0.0006) [2023-12-27 03:28:27,834][105620] Updated weights for policy 1, policy_version 1661574 (0.0008) [2023-12-27 03:28:27,881][105620] Updated weights for policy 1, policy_version 1661584 (0.0010) [2023-12-27 03:28:27,937][105620] Updated weights for policy 1, policy_version 1661594 (0.0009) [2023-12-27 03:28:28,403][105692] Updated weights for policy 0, policy_version 1658179 (0.0009) [2023-12-27 03:28:28,461][105692] Updated weights for policy 0, policy_version 1658189 (0.0009) [2023-12-27 03:28:28,520][105692] Updated weights for policy 0, policy_version 1658199 (0.0009) [2023-12-27 03:28:28,639][105620] Updated weights for policy 1, policy_version 1661604 (0.0009) [2023-12-27 03:28:28,693][105620] Updated weights for policy 1, policy_version 1661614 (0.0009) [2023-12-27 03:28:28,757][105620] Updated weights for policy 1, policy_version 1661624 (0.0009) [2023-12-27 03:28:29,286][105692] Updated weights for policy 0, policy_version 1658209 (0.0009) [2023-12-27 03:28:29,356][105692] Updated weights for policy 0, policy_version 1658219 (0.0008) [2023-12-27 03:28:29,417][105692] Updated weights for policy 0, policy_version 1658229 (0.0008) [2023-12-27 03:28:29,469][105692] Updated weights for policy 0, policy_version 1658239 (0.0007) [2023-12-27 03:28:29,471][105620] Updated weights for policy 1, policy_version 1661634 (0.0009) [2023-12-27 03:28:29,541][105620] Updated weights for policy 1, policy_version 1661644 (0.0008) [2023-12-27 03:28:29,598][105620] Updated weights for policy 1, policy_version 1661654 (0.0006) [2023-12-27 03:28:29,668][105620] Updated weights for policy 1, policy_version 1661664 (0.0005) [2023-12-27 03:28:30,248][105692] Updated weights for policy 0, policy_version 1658249 (0.0009) [2023-12-27 03:28:30,304][105692] Updated weights for policy 0, policy_version 1658259 (0.0007) [2023-12-27 03:28:30,307][105620] Updated weights for policy 1, policy_version 1661674 (0.0010) [2023-12-27 03:28:30,358][105620] Updated weights for policy 1, policy_version 1661684 (0.0009) [2023-12-27 03:28:30,362][105692] Updated weights for policy 0, policy_version 1658269 (0.0007) [2023-12-27 03:28:30,408][105620] Updated weights for policy 1, policy_version 1661694 (0.0005) [2023-12-27 03:28:30,967][105620] Updated weights for policy 1, policy_version 1661704 (0.0009) [2023-12-27 03:28:31,016][105620] Updated weights for policy 1, policy_version 1661714 (0.0005) [2023-12-27 03:28:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 850034688. Throughput: 0: 9749.1, 1: 9920.8. Samples: 850009492. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:28:31,062][104569] Avg episode reward: [(0, '8807.322'), (1, '8899.755')] [2023-12-27 03:28:31,074][105620] Updated weights for policy 1, policy_version 1661724 (0.0009) [2023-12-27 03:28:31,089][105692] Updated weights for policy 0, policy_version 1658279 (0.0008) [2023-12-27 03:28:31,097][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001661728_425459712.pth... [2023-12-27 03:28:31,103][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001660544_425156608.pth [2023-12-27 03:28:31,159][105692] Updated weights for policy 0, policy_version 1658289 (0.0008) [2023-12-27 03:28:31,208][105692] Updated weights for policy 0, policy_version 1658299 (0.0008) [2023-12-27 03:28:31,228][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001658304_424591360.pth... [2023-12-27 03:28:31,232][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001657152_424296448.pth [2023-12-27 03:28:31,753][105620] Updated weights for policy 1, policy_version 1661734 (0.0009) [2023-12-27 03:28:31,805][105620] Updated weights for policy 1, policy_version 1661744 (0.0009) [2023-12-27 03:28:31,851][105620] Updated weights for policy 1, policy_version 1661754 (0.0009) [2023-12-27 03:28:32,027][105692] Updated weights for policy 0, policy_version 1658309 (0.0009) [2023-12-27 03:28:32,081][105692] Updated weights for policy 0, policy_version 1658319 (0.0009) [2023-12-27 03:28:32,127][105692] Updated weights for policy 0, policy_version 1658329 (0.0008) [2023-12-27 03:28:32,572][105620] Updated weights for policy 1, policy_version 1661764 (0.0007) [2023-12-27 03:28:32,628][105620] Updated weights for policy 1, policy_version 1661774 (0.0005) [2023-12-27 03:28:32,687][105620] Updated weights for policy 1, policy_version 1661784 (0.0005) [2023-12-27 03:28:32,978][105692] Updated weights for policy 0, policy_version 1658339 (0.0009) [2023-12-27 03:28:33,036][105692] Updated weights for policy 0, policy_version 1658349 (0.0009) [2023-12-27 03:28:33,089][105692] Updated weights for policy 0, policy_version 1658359 (0.0009) [2023-12-27 03:28:33,260][105620] Updated weights for policy 1, policy_version 1661794 (0.0006) [2023-12-27 03:28:33,311][105620] Updated weights for policy 1, policy_version 1661804 (0.0009) [2023-12-27 03:28:33,358][105620] Updated weights for policy 1, policy_version 1661814 (0.0009) [2023-12-27 03:28:33,418][105620] Updated weights for policy 1, policy_version 1661824 (0.0009) [2023-12-27 03:28:33,798][105692] Updated weights for policy 0, policy_version 1658369 (0.0009) [2023-12-27 03:28:33,848][105692] Updated weights for policy 0, policy_version 1658379 (0.0005) [2023-12-27 03:28:33,878][105585] KL-divergence is very high: 103.7672 [2023-12-27 03:28:33,890][105692] Updated weights for policy 0, policy_version 1658389 (0.0005) [2023-12-27 03:28:33,915][105585] KL-divergence is very high: 135.3519 [2023-12-27 03:28:33,938][105692] Updated weights for policy 0, policy_version 1658399 (0.0005) [2023-12-27 03:28:34,322][105620] Updated weights for policy 1, policy_version 1661834 (0.0008) [2023-12-27 03:28:34,370][105620] Updated weights for policy 1, policy_version 1661844 (0.0009) [2023-12-27 03:28:34,426][105620] Updated weights for policy 1, policy_version 1661854 (0.0009) [2023-12-27 03:28:34,596][105692] Updated weights for policy 0, policy_version 1658409 (0.0008) [2023-12-27 03:28:34,659][105692] Updated weights for policy 0, policy_version 1658419 (0.0007) [2023-12-27 03:28:34,723][105692] Updated weights for policy 0, policy_version 1658429 (0.0008) [2023-12-27 03:28:35,240][105620] Updated weights for policy 1, policy_version 1661864 (0.0009) [2023-12-27 03:28:35,290][105620] Updated weights for policy 1, policy_version 1661874 (0.0008) [2023-12-27 03:28:35,340][105620] Updated weights for policy 1, policy_version 1661884 (0.0006) [2023-12-27 03:28:35,410][105692] Updated weights for policy 0, policy_version 1658439 (0.0009) [2023-12-27 03:28:35,465][105692] Updated weights for policy 0, policy_version 1658449 (0.0010) [2023-12-27 03:28:35,513][105692] Updated weights for policy 0, policy_version 1658459 (0.0005) [2023-12-27 03:28:36,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 850132992. Throughput: 0: 9635.9, 1: 10021.4. Samples: 850126668. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:28:36,062][104569] Avg episode reward: [(0, '8437.251'), (1, '8990.372')] [2023-12-27 03:28:36,088][105620] Updated weights for policy 1, policy_version 1661894 (0.0008) [2023-12-27 03:28:36,157][105620] Updated weights for policy 1, policy_version 1661904 (0.0008) [2023-12-27 03:28:36,196][105692] Updated weights for policy 0, policy_version 1658469 (0.0007) [2023-12-27 03:28:36,214][105620] Updated weights for policy 1, policy_version 1661914 (0.0009) [2023-12-27 03:28:36,258][105692] Updated weights for policy 0, policy_version 1658479 (0.0006) [2023-12-27 03:28:36,320][105692] Updated weights for policy 0, policy_version 1658489 (0.0009) [2023-12-27 03:28:37,009][105620] Updated weights for policy 1, policy_version 1661924 (0.0007) [2023-12-27 03:28:37,015][105692] Updated weights for policy 0, policy_version 1658499 (0.0008) [2023-12-27 03:28:37,061][105620] Updated weights for policy 1, policy_version 1661934 (0.0007) [2023-12-27 03:28:37,075][105692] Updated weights for policy 0, policy_version 1658509 (0.0007) [2023-12-27 03:28:37,122][105620] Updated weights for policy 1, policy_version 1661944 (0.0007) [2023-12-27 03:28:37,132][105692] Updated weights for policy 0, policy_version 1658519 (0.0006) [2023-12-27 03:28:37,862][105620] Updated weights for policy 1, policy_version 1661954 (0.0007) [2023-12-27 03:28:37,917][105620] Updated weights for policy 1, policy_version 1661964 (0.0006) [2023-12-27 03:28:37,922][105692] Updated weights for policy 0, policy_version 1658529 (0.0007) [2023-12-27 03:28:37,967][105620] Updated weights for policy 1, policy_version 1661974 (0.0007) [2023-12-27 03:28:37,977][105692] Updated weights for policy 0, policy_version 1658539 (0.0006) [2023-12-27 03:28:38,025][105620] Updated weights for policy 1, policy_version 1661984 (0.0007) [2023-12-27 03:28:38,030][105692] Updated weights for policy 0, policy_version 1658549 (0.0010) [2023-12-27 03:28:38,077][105692] Updated weights for policy 0, policy_version 1658559 (0.0008) [2023-12-27 03:28:38,782][105692] Updated weights for policy 0, policy_version 1658569 (0.0010) [2023-12-27 03:28:38,801][105620] Updated weights for policy 1, policy_version 1661994 (0.0007) [2023-12-27 03:28:38,839][105692] Updated weights for policy 0, policy_version 1658579 (0.0008) [2023-12-27 03:28:38,860][105620] Updated weights for policy 1, policy_version 1662004 (0.0007) [2023-12-27 03:28:38,886][105692] Updated weights for policy 0, policy_version 1658589 (0.0008) [2023-12-27 03:28:38,915][105620] Updated weights for policy 1, policy_version 1662014 (0.0008) [2023-12-27 03:28:39,556][105692] Updated weights for policy 0, policy_version 1658599 (0.0009) [2023-12-27 03:28:39,611][105692] Updated weights for policy 0, policy_version 1658609 (0.0009) [2023-12-27 03:28:39,663][105692] Updated weights for policy 0, policy_version 1658619 (0.0009) [2023-12-27 03:28:39,684][105620] Updated weights for policy 1, policy_version 1662024 (0.0010) [2023-12-27 03:28:39,743][105620] Updated weights for policy 1, policy_version 1662034 (0.0009) [2023-12-27 03:28:39,796][105620] Updated weights for policy 1, policy_version 1662044 (0.0009) [2023-12-27 03:28:40,420][105692] Updated weights for policy 0, policy_version 1658629 (0.0007) [2023-12-27 03:28:40,491][105692] Updated weights for policy 0, policy_version 1658639 (0.0007) [2023-12-27 03:28:40,548][105692] Updated weights for policy 0, policy_version 1658649 (0.0005) [2023-12-27 03:28:40,633][105620] Updated weights for policy 1, policy_version 1662054 (0.0010) [2023-12-27 03:28:40,703][105620] Updated weights for policy 1, policy_version 1662064 (0.0009) [2023-12-27 03:28:40,778][105620] Updated weights for policy 1, policy_version 1662074 (0.0010) [2023-12-27 03:28:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 850231296. Throughput: 0: 9658.5, 1: 9888.2. Samples: 850239528. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:28:41,063][104569] Avg episode reward: [(0, '8528.118'), (1, '9171.814')] [2023-12-27 03:28:41,140][105692] Updated weights for policy 0, policy_version 1658659 (0.0006) [2023-12-27 03:28:41,209][105692] Updated weights for policy 0, policy_version 1658669 (0.0006) [2023-12-27 03:28:41,279][105692] Updated weights for policy 0, policy_version 1658679 (0.0010) [2023-12-27 03:28:41,469][105620] Updated weights for policy 1, policy_version 1662084 (0.0007) [2023-12-27 03:28:41,536][105620] Updated weights for policy 1, policy_version 1662094 (0.0008) [2023-12-27 03:28:41,606][105620] Updated weights for policy 1, policy_version 1662104 (0.0008) [2023-12-27 03:28:41,950][105692] Updated weights for policy 0, policy_version 1658689 (0.0010) [2023-12-27 03:28:42,014][105692] Updated weights for policy 0, policy_version 1658699 (0.0011) [2023-12-27 03:28:42,065][105692] Updated weights for policy 0, policy_version 1658709 (0.0010) [2023-12-27 03:28:42,122][105692] Updated weights for policy 0, policy_version 1658719 (0.0011) [2023-12-27 03:28:42,232][105620] Updated weights for policy 1, policy_version 1662114 (0.0008) [2023-12-27 03:28:42,294][105620] Updated weights for policy 1, policy_version 1662124 (0.0008) [2023-12-27 03:28:42,367][105620] Updated weights for policy 1, policy_version 1662134 (0.0008) [2023-12-27 03:28:42,433][105620] Updated weights for policy 1, policy_version 1662144 (0.0006) [2023-12-27 03:28:42,933][105692] Updated weights for policy 0, policy_version 1658729 (0.0009) [2023-12-27 03:28:42,984][105692] Updated weights for policy 0, policy_version 1658739 (0.0005) [2023-12-27 03:28:43,042][105692] Updated weights for policy 0, policy_version 1658749 (0.0008) [2023-12-27 03:28:43,043][105620] Updated weights for policy 1, policy_version 1662154 (0.0006) [2023-12-27 03:28:43,097][105620] Updated weights for policy 1, policy_version 1662164 (0.0005) [2023-12-27 03:28:43,154][105620] Updated weights for policy 1, policy_version 1662174 (0.0006) [2023-12-27 03:28:43,613][105692] Updated weights for policy 0, policy_version 1658759 (0.0005) [2023-12-27 03:28:43,678][105692] Updated weights for policy 0, policy_version 1658769 (0.0010) [2023-12-27 03:28:43,730][105692] Updated weights for policy 0, policy_version 1658779 (0.0010) [2023-12-27 03:28:43,932][105620] Updated weights for policy 1, policy_version 1662184 (0.0009) [2023-12-27 03:28:43,976][105620] Updated weights for policy 1, policy_version 1662194 (0.0008) [2023-12-27 03:28:44,028][105620] Updated weights for policy 1, policy_version 1662204 (0.0008) [2023-12-27 03:28:44,437][105692] Updated weights for policy 0, policy_version 1658789 (0.0010) [2023-12-27 03:28:44,481][105692] Updated weights for policy 0, policy_version 1658799 (0.0010) [2023-12-27 03:28:44,540][105692] Updated weights for policy 0, policy_version 1658809 (0.0010) [2023-12-27 03:28:44,811][105620] Updated weights for policy 1, policy_version 1662214 (0.0008) [2023-12-27 03:28:44,872][105620] Updated weights for policy 1, policy_version 1662224 (0.0007) [2023-12-27 03:28:44,937][105620] Updated weights for policy 1, policy_version 1662234 (0.0006) [2023-12-27 03:28:45,296][105692] Updated weights for policy 0, policy_version 1658819 (0.0009) [2023-12-27 03:28:45,355][105692] Updated weights for policy 0, policy_version 1658829 (0.0005) [2023-12-27 03:28:45,413][105692] Updated weights for policy 0, policy_version 1658839 (0.0005) [2023-12-27 03:28:45,661][105620] Updated weights for policy 1, policy_version 1662244 (0.0010) [2023-12-27 03:28:45,723][105620] Updated weights for policy 1, policy_version 1662254 (0.0009) [2023-12-27 03:28:45,784][105620] Updated weights for policy 1, policy_version 1662264 (0.0009) [2023-12-27 03:28:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 850329600. Throughput: 0: 9688.8, 1: 9898.2. Samples: 850301280. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:28:46,063][104569] Avg episode reward: [(0, '9080.785'), (1, '8806.212')] [2023-12-27 03:28:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001662272_425598976.pth... [2023-12-27 03:28:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001661120_425304064.pth [2023-12-27 03:28:46,077][105692] Updated weights for policy 0, policy_version 1658849 (0.0006) [2023-12-27 03:28:46,139][105692] Updated weights for policy 0, policy_version 1658859 (0.0010) [2023-12-27 03:28:46,199][105692] Updated weights for policy 0, policy_version 1658869 (0.0011) [2023-12-27 03:28:46,251][105692] Updated weights for policy 0, policy_version 1658879 (0.0010) [2023-12-27 03:28:46,255][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001658880_424738816.pth... [2023-12-27 03:28:46,258][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001657728_424443904.pth [2023-12-27 03:28:46,402][105620] Updated weights for policy 1, policy_version 1662274 (0.0008) [2023-12-27 03:28:46,451][105620] Updated weights for policy 1, policy_version 1662284 (0.0009) [2023-12-27 03:28:46,505][105620] Updated weights for policy 1, policy_version 1662294 (0.0006) [2023-12-27 03:28:46,570][105620] Updated weights for policy 1, policy_version 1662304 (0.0005) [2023-12-27 03:28:46,972][105692] Updated weights for policy 0, policy_version 1658889 (0.0011) [2023-12-27 03:28:47,024][105692] Updated weights for policy 0, policy_version 1658899 (0.0011) [2023-12-27 03:28:47,084][105692] Updated weights for policy 0, policy_version 1658909 (0.0010) [2023-12-27 03:28:47,181][105620] Updated weights for policy 1, policy_version 1662314 (0.0011) [2023-12-27 03:28:47,233][105620] Updated weights for policy 1, policy_version 1662324 (0.0010) [2023-12-27 03:28:47,281][105620] Updated weights for policy 1, policy_version 1662334 (0.0010) [2023-12-27 03:28:47,840][105692] Updated weights for policy 0, policy_version 1658919 (0.0007) [2023-12-27 03:28:47,892][105692] Updated weights for policy 0, policy_version 1658929 (0.0005) [2023-12-27 03:28:47,937][105692] Updated weights for policy 0, policy_version 1658939 (0.0005) [2023-12-27 03:28:48,050][105620] Updated weights for policy 1, policy_version 1662344 (0.0009) [2023-12-27 03:28:48,105][105620] Updated weights for policy 1, policy_version 1662354 (0.0009) [2023-12-27 03:28:48,164][105620] Updated weights for policy 1, policy_version 1662364 (0.0010) [2023-12-27 03:28:48,576][105692] Updated weights for policy 0, policy_version 1658949 (0.0010) [2023-12-27 03:28:48,637][105692] Updated weights for policy 0, policy_version 1658959 (0.0008) [2023-12-27 03:28:48,717][105692] Updated weights for policy 0, policy_version 1658969 (0.0009) [2023-12-27 03:28:48,883][105620] Updated weights for policy 1, policy_version 1662374 (0.0009) [2023-12-27 03:28:48,944][105620] Updated weights for policy 1, policy_version 1662384 (0.0007) [2023-12-27 03:28:49,007][105620] Updated weights for policy 1, policy_version 1662394 (0.0005) [2023-12-27 03:28:49,507][105692] Updated weights for policy 0, policy_version 1658979 (0.0009) [2023-12-27 03:28:49,567][105692] Updated weights for policy 0, policy_version 1658989 (0.0006) [2023-12-27 03:28:49,628][105692] Updated weights for policy 0, policy_version 1658999 (0.0009) [2023-12-27 03:28:49,689][105620] Updated weights for policy 1, policy_version 1662404 (0.0007) [2023-12-27 03:28:49,755][105620] Updated weights for policy 1, policy_version 1662414 (0.0005) [2023-12-27 03:28:49,821][105620] Updated weights for policy 1, policy_version 1662424 (0.0006) [2023-12-27 03:28:50,382][105692] Updated weights for policy 0, policy_version 1659009 (0.0009) [2023-12-27 03:28:50,439][105692] Updated weights for policy 0, policy_version 1659019 (0.0010) [2023-12-27 03:28:50,485][105620] Updated weights for policy 1, policy_version 1662434 (0.0010) [2023-12-27 03:28:50,500][105692] Updated weights for policy 0, policy_version 1659029 (0.0011) [2023-12-27 03:28:50,535][105620] Updated weights for policy 1, policy_version 1662444 (0.0006) [2023-12-27 03:28:50,549][105692] Updated weights for policy 0, policy_version 1659039 (0.0010) [2023-12-27 03:28:50,602][105620] Updated weights for policy 1, policy_version 1662454 (0.0008) [2023-12-27 03:28:50,671][105620] Updated weights for policy 1, policy_version 1662464 (0.0009) [2023-12-27 03:28:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 850427904. Throughput: 0: 9670.2, 1: 9913.6. Samples: 850418496. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:28:51,063][104569] Avg episode reward: [(0, '8620.989'), (1, '8809.400')] [2023-12-27 03:28:51,225][105692] Updated weights for policy 0, policy_version 1659049 (0.0009) [2023-12-27 03:28:51,294][105692] Updated weights for policy 0, policy_version 1659059 (0.0010) [2023-12-27 03:28:51,363][105692] Updated weights for policy 0, policy_version 1659069 (0.0009) [2023-12-27 03:28:51,403][105620] Updated weights for policy 1, policy_version 1662474 (0.0007) [2023-12-27 03:28:51,452][105620] Updated weights for policy 1, policy_version 1662484 (0.0007) [2023-12-27 03:28:51,517][105620] Updated weights for policy 1, policy_version 1662494 (0.0008) [2023-12-27 03:28:52,089][105692] Updated weights for policy 0, policy_version 1659079 (0.0008) [2023-12-27 03:28:52,140][105692] Updated weights for policy 0, policy_version 1659089 (0.0009) [2023-12-27 03:28:52,188][105692] Updated weights for policy 0, policy_version 1659099 (0.0009) [2023-12-27 03:28:52,270][105620] Updated weights for policy 1, policy_version 1662504 (0.0008) [2023-12-27 03:28:52,337][105620] Updated weights for policy 1, policy_version 1662514 (0.0010) [2023-12-27 03:28:52,402][105620] Updated weights for policy 1, policy_version 1662524 (0.0010) [2023-12-27 03:28:52,938][105692] Updated weights for policy 0, policy_version 1659109 (0.0009) [2023-12-27 03:28:52,998][105692] Updated weights for policy 0, policy_version 1659119 (0.0009) [2023-12-27 03:28:53,057][105692] Updated weights for policy 0, policy_version 1659129 (0.0007) [2023-12-27 03:28:53,157][105620] Updated weights for policy 1, policy_version 1662534 (0.0009) [2023-12-27 03:28:53,214][105620] Updated weights for policy 1, policy_version 1662544 (0.0010) [2023-12-27 03:28:53,261][105620] Updated weights for policy 1, policy_version 1662554 (0.0008) [2023-12-27 03:28:53,740][105692] Updated weights for policy 0, policy_version 1659139 (0.0007) [2023-12-27 03:28:53,798][105692] Updated weights for policy 0, policy_version 1659149 (0.0008) [2023-12-27 03:28:53,859][105692] Updated weights for policy 0, policy_version 1659159 (0.0005) [2023-12-27 03:28:53,888][105620] Updated weights for policy 1, policy_version 1662564 (0.0007) [2023-12-27 03:28:53,949][105620] Updated weights for policy 1, policy_version 1662574 (0.0008) [2023-12-27 03:28:53,994][105620] Updated weights for policy 1, policy_version 1662584 (0.0006) [2023-12-27 03:28:54,435][105692] Updated weights for policy 0, policy_version 1659169 (0.0006) [2023-12-27 03:28:54,481][105692] Updated weights for policy 0, policy_version 1659179 (0.0005) [2023-12-27 03:28:54,534][105692] Updated weights for policy 0, policy_version 1659189 (0.0005) [2023-12-27 03:28:54,586][105692] Updated weights for policy 0, policy_version 1659199 (0.0005) [2023-12-27 03:28:54,732][105620] Updated weights for policy 1, policy_version 1662594 (0.0006) [2023-12-27 03:28:54,805][105620] Updated weights for policy 1, policy_version 1662604 (0.0009) [2023-12-27 03:28:54,860][105620] Updated weights for policy 1, policy_version 1662614 (0.0009) [2023-12-27 03:28:54,918][105620] Updated weights for policy 1, policy_version 1662624 (0.0008) [2023-12-27 03:28:55,197][105692] Updated weights for policy 0, policy_version 1659209 (0.0008) [2023-12-27 03:28:55,257][105692] Updated weights for policy 0, policy_version 1659219 (0.0009) [2023-12-27 03:28:55,328][105692] Updated weights for policy 0, policy_version 1659229 (0.0010) [2023-12-27 03:28:55,587][105620] Updated weights for policy 1, policy_version 1662634 (0.0005) [2023-12-27 03:28:55,642][105620] Updated weights for policy 1, policy_version 1662644 (0.0005) [2023-12-27 03:28:55,709][105620] Updated weights for policy 1, policy_version 1662654 (0.0005) [2023-12-27 03:28:56,022][105692] Updated weights for policy 0, policy_version 1659239 (0.0007) [2023-12-27 03:28:56,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 850526208. Throughput: 0: 9689.3, 1: 9965.9. Samples: 850537120. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:28:56,063][104569] Avg episode reward: [(0, '8253.257'), (1, '8625.121')] [2023-12-27 03:28:56,080][105692] Updated weights for policy 0, policy_version 1659249 (0.0005) [2023-12-27 03:28:56,150][105692] Updated weights for policy 0, policy_version 1659259 (0.0006) [2023-12-27 03:28:56,370][105620] Updated weights for policy 1, policy_version 1662664 (0.0007) [2023-12-27 03:28:56,436][105620] Updated weights for policy 1, policy_version 1662674 (0.0008) [2023-12-27 03:28:56,490][105620] Updated weights for policy 1, policy_version 1662684 (0.0005) [2023-12-27 03:28:56,690][105692] Updated weights for policy 0, policy_version 1659269 (0.0006) [2023-12-27 03:28:56,748][105692] Updated weights for policy 0, policy_version 1659279 (0.0005) [2023-12-27 03:28:56,800][105692] Updated weights for policy 0, policy_version 1659289 (0.0005) [2023-12-27 03:28:57,061][105620] Updated weights for policy 1, policy_version 1662694 (0.0005) [2023-12-27 03:28:57,104][105620] Updated weights for policy 1, policy_version 1662704 (0.0005) [2023-12-27 03:28:57,155][105620] Updated weights for policy 1, policy_version 1662714 (0.0005) [2023-12-27 03:28:57,406][105692] Updated weights for policy 0, policy_version 1659299 (0.0005) [2023-12-27 03:28:57,449][105692] Updated weights for policy 0, policy_version 1659309 (0.0005) [2023-12-27 03:28:57,502][105692] Updated weights for policy 0, policy_version 1659319 (0.0005) [2023-12-27 03:28:57,719][105620] Updated weights for policy 1, policy_version 1662724 (0.0005) [2023-12-27 03:28:57,776][105620] Updated weights for policy 1, policy_version 1662734 (0.0005) [2023-12-27 03:28:57,830][105620] Updated weights for policy 1, policy_version 1662744 (0.0005) [2023-12-27 03:28:58,014][105692] Updated weights for policy 0, policy_version 1659329 (0.0005) [2023-12-27 03:28:58,078][105692] Updated weights for policy 0, policy_version 1659339 (0.0005) [2023-12-27 03:28:58,128][105692] Updated weights for policy 0, policy_version 1659349 (0.0006) [2023-12-27 03:28:58,192][105692] Updated weights for policy 0, policy_version 1659359 (0.0008) [2023-12-27 03:28:58,440][105620] Updated weights for policy 1, policy_version 1662754 (0.0006) [2023-12-27 03:28:58,503][105620] Updated weights for policy 1, policy_version 1662764 (0.0011) [2023-12-27 03:28:58,572][105620] Updated weights for policy 1, policy_version 1662774 (0.0010) [2023-12-27 03:28:58,632][105620] Updated weights for policy 1, policy_version 1662784 (0.0011) [2023-12-27 03:28:58,991][105692] Updated weights for policy 0, policy_version 1659369 (0.0008) [2023-12-27 03:28:59,056][105692] Updated weights for policy 0, policy_version 1659379 (0.0006) [2023-12-27 03:28:59,122][105692] Updated weights for policy 0, policy_version 1659389 (0.0006) [2023-12-27 03:28:59,480][105620] Updated weights for policy 1, policy_version 1662794 (0.0010) [2023-12-27 03:28:59,542][105620] Updated weights for policy 1, policy_version 1662804 (0.0010) [2023-12-27 03:28:59,594][105620] Updated weights for policy 1, policy_version 1662814 (0.0010) [2023-12-27 03:28:59,804][105692] Updated weights for policy 0, policy_version 1659399 (0.0007) [2023-12-27 03:28:59,864][105692] Updated weights for policy 0, policy_version 1659409 (0.0009) [2023-12-27 03:28:59,928][105692] Updated weights for policy 0, policy_version 1659419 (0.0008) [2023-12-27 03:29:00,215][105620] Updated weights for policy 1, policy_version 1662824 (0.0006) [2023-12-27 03:29:00,280][105620] Updated weights for policy 1, policy_version 1662834 (0.0005) [2023-12-27 03:29:00,349][105620] Updated weights for policy 1, policy_version 1662844 (0.0005) [2023-12-27 03:29:00,612][105692] Updated weights for policy 0, policy_version 1659429 (0.0007) [2023-12-27 03:29:00,682][105692] Updated weights for policy 0, policy_version 1659439 (0.0005) [2023-12-27 03:29:00,739][105692] Updated weights for policy 0, policy_version 1659449 (0.0005) [2023-12-27 03:29:00,842][105620] Updated weights for policy 1, policy_version 1662854 (0.0007) [2023-12-27 03:29:00,892][105620] Updated weights for policy 1, policy_version 1662864 (0.0010) [2023-12-27 03:29:00,951][105620] Updated weights for policy 1, policy_version 1662874 (0.0010) [2023-12-27 03:29:01,062][104569] Fps is (10 sec: 21299.3, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 850640896. Throughput: 0: 9887.3, 1: 10015.5. Samples: 850604668. Policy #0 lag: (min: 19.0, avg: 23.6, max: 51.0) [2023-12-27 03:29:01,062][104569] Avg episode reward: [(0, '8803.996'), (1, '8990.990')] [2023-12-27 03:29:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001659456_424886272.pth... [2023-12-27 03:29:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001662880_425754624.pth... [2023-12-27 03:29:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001658304_424591360.pth [2023-12-27 03:29:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001661728_425459712.pth [2023-12-27 03:29:01,272][105692] Updated weights for policy 0, policy_version 1659459 (0.0006) [2023-12-27 03:29:01,338][105692] Updated weights for policy 0, policy_version 1659469 (0.0009) [2023-12-27 03:29:01,408][105692] Updated weights for policy 0, policy_version 1659479 (0.0007) [2023-12-27 03:29:01,681][105620] Updated weights for policy 1, policy_version 1662884 (0.0008) [2023-12-27 03:29:01,746][105620] Updated weights for policy 1, policy_version 1662894 (0.0008) [2023-12-27 03:29:01,794][105620] Updated weights for policy 1, policy_version 1662904 (0.0007) [2023-12-27 03:29:02,223][105692] Updated weights for policy 0, policy_version 1659489 (0.0009) [2023-12-27 03:29:02,293][105692] Updated weights for policy 0, policy_version 1659499 (0.0008) [2023-12-27 03:29:02,353][105692] Updated weights for policy 0, policy_version 1659509 (0.0006) [2023-12-27 03:29:02,387][105620] Updated weights for policy 1, policy_version 1662914 (0.0005) [2023-12-27 03:29:02,412][105692] Updated weights for policy 0, policy_version 1659519 (0.0007) [2023-12-27 03:29:02,458][105620] Updated weights for policy 1, policy_version 1662924 (0.0008) [2023-12-27 03:29:02,520][105620] Updated weights for policy 1, policy_version 1662934 (0.0008) [2023-12-27 03:29:02,586][105620] Updated weights for policy 1, policy_version 1662944 (0.0009) [2023-12-27 03:29:03,003][105692] Updated weights for policy 0, policy_version 1659529 (0.0006) [2023-12-27 03:29:03,052][105692] Updated weights for policy 0, policy_version 1659539 (0.0006) [2023-12-27 03:29:03,099][105692] Updated weights for policy 0, policy_version 1659549 (0.0008) [2023-12-27 03:29:03,209][105620] Updated weights for policy 1, policy_version 1662954 (0.0010) [2023-12-27 03:29:03,258][105620] Updated weights for policy 1, policy_version 1662964 (0.0010) [2023-12-27 03:29:03,301][105620] Updated weights for policy 1, policy_version 1662974 (0.0007) [2023-12-27 03:29:03,654][105692] Updated weights for policy 0, policy_version 1659559 (0.0006) [2023-12-27 03:29:03,713][105692] Updated weights for policy 0, policy_version 1659569 (0.0005) [2023-12-27 03:29:03,767][105692] Updated weights for policy 0, policy_version 1659579 (0.0005) [2023-12-27 03:29:03,994][105620] Updated weights for policy 1, policy_version 1662984 (0.0006) [2023-12-27 03:29:04,046][105620] Updated weights for policy 1, policy_version 1662994 (0.0010) [2023-12-27 03:29:04,095][105620] Updated weights for policy 1, policy_version 1663004 (0.0010) [2023-12-27 03:29:04,395][105692] Updated weights for policy 0, policy_version 1659589 (0.0005) [2023-12-27 03:29:04,457][105692] Updated weights for policy 0, policy_version 1659599 (0.0006) [2023-12-27 03:29:04,517][105692] Updated weights for policy 0, policy_version 1659609 (0.0009) [2023-12-27 03:29:04,763][105620] Updated weights for policy 1, policy_version 1663014 (0.0007) [2023-12-27 03:29:04,812][105620] Updated weights for policy 1, policy_version 1663024 (0.0005) [2023-12-27 03:29:04,859][105620] Updated weights for policy 1, policy_version 1663034 (0.0005) [2023-12-27 03:29:05,112][105692] Updated weights for policy 0, policy_version 1659619 (0.0008) [2023-12-27 03:29:05,159][105692] Updated weights for policy 0, policy_version 1659629 (0.0008) [2023-12-27 03:29:05,213][105692] Updated weights for policy 0, policy_version 1659639 (0.0006) [2023-12-27 03:29:05,537][105620] Updated weights for policy 1, policy_version 1663044 (0.0010) [2023-12-27 03:29:05,585][105620] Updated weights for policy 1, policy_version 1663054 (0.0010) [2023-12-27 03:29:05,629][105620] Updated weights for policy 1, policy_version 1663064 (0.0010) [2023-12-27 03:29:05,877][105692] Updated weights for policy 0, policy_version 1659649 (0.0007) [2023-12-27 03:29:05,936][105692] Updated weights for policy 0, policy_version 1659659 (0.0006) [2023-12-27 03:29:05,984][105692] Updated weights for policy 0, policy_version 1659669 (0.0005) [2023-12-27 03:29:06,038][105692] Updated weights for policy 0, policy_version 1659679 (0.0008) [2023-12-27 03:29:06,062][104569] Fps is (10 sec: 22118.8, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 850747392. Throughput: 0: 9924.5, 1: 10020.0. Samples: 850730700. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:29:06,062][104569] Avg episode reward: [(0, '8803.874'), (1, '8898.196')] [2023-12-27 03:29:06,410][105620] Updated weights for policy 1, policy_version 1663074 (0.0010) [2023-12-27 03:29:06,475][105620] Updated weights for policy 1, policy_version 1663084 (0.0008) [2023-12-27 03:29:06,532][105620] Updated weights for policy 1, policy_version 1663094 (0.0008) [2023-12-27 03:29:06,593][105620] Updated weights for policy 1, policy_version 1663104 (0.0008) [2023-12-27 03:29:06,772][105692] Updated weights for policy 0, policy_version 1659689 (0.0011) [2023-12-27 03:29:06,832][105692] Updated weights for policy 0, policy_version 1659699 (0.0010) [2023-12-27 03:29:06,891][105692] Updated weights for policy 0, policy_version 1659709 (0.0010) [2023-12-27 03:29:07,378][105620] Updated weights for policy 1, policy_version 1663114 (0.0008) [2023-12-27 03:29:07,432][105620] Updated weights for policy 1, policy_version 1663124 (0.0007) [2023-12-27 03:29:07,490][105620] Updated weights for policy 1, policy_version 1663134 (0.0008) [2023-12-27 03:29:07,640][105692] Updated weights for policy 0, policy_version 1659719 (0.0009) [2023-12-27 03:29:07,688][105692] Updated weights for policy 0, policy_version 1659729 (0.0009) [2023-12-27 03:29:07,740][105692] Updated weights for policy 0, policy_version 1659739 (0.0009) [2023-12-27 03:29:08,133][105620] Updated weights for policy 1, policy_version 1663144 (0.0010) [2023-12-27 03:29:08,195][105620] Updated weights for policy 1, policy_version 1663154 (0.0010) [2023-12-27 03:29:08,254][105620] Updated weights for policy 1, policy_version 1663164 (0.0010) [2023-12-27 03:29:08,378][105692] Updated weights for policy 0, policy_version 1659749 (0.0009) [2023-12-27 03:29:08,427][105692] Updated weights for policy 0, policy_version 1659759 (0.0010) [2023-12-27 03:29:08,472][105692] Updated weights for policy 0, policy_version 1659769 (0.0010) [2023-12-27 03:29:08,968][105620] Updated weights for policy 1, policy_version 1663174 (0.0007) [2023-12-27 03:29:09,023][105620] Updated weights for policy 1, policy_version 1663184 (0.0006) [2023-12-27 03:29:09,079][105620] Updated weights for policy 1, policy_version 1663194 (0.0005) [2023-12-27 03:29:09,256][105692] Updated weights for policy 0, policy_version 1659779 (0.0008) [2023-12-27 03:29:09,318][105692] Updated weights for policy 0, policy_version 1659789 (0.0009) [2023-12-27 03:29:09,384][105692] Updated weights for policy 0, policy_version 1659799 (0.0008) [2023-12-27 03:29:09,672][105620] Updated weights for policy 1, policy_version 1663204 (0.0007) [2023-12-27 03:29:09,735][105620] Updated weights for policy 1, policy_version 1663214 (0.0010) [2023-12-27 03:29:09,791][105620] Updated weights for policy 1, policy_version 1663224 (0.0011) [2023-12-27 03:29:10,192][105692] Updated weights for policy 0, policy_version 1659809 (0.0010) [2023-12-27 03:29:10,243][105692] Updated weights for policy 0, policy_version 1659819 (0.0007) [2023-12-27 03:29:10,297][105692] Updated weights for policy 0, policy_version 1659829 (0.0005) [2023-12-27 03:29:10,358][105692] Updated weights for policy 0, policy_version 1659839 (0.0006) [2023-12-27 03:29:10,539][105620] Updated weights for policy 1, policy_version 1663234 (0.0010) [2023-12-27 03:29:10,601][105620] Updated weights for policy 1, policy_version 1663244 (0.0011) [2023-12-27 03:29:10,663][105620] Updated weights for policy 1, policy_version 1663254 (0.0010) [2023-12-27 03:29:10,718][105620] Updated weights for policy 1, policy_version 1663264 (0.0010) [2023-12-27 03:29:10,913][105692] Updated weights for policy 0, policy_version 1659849 (0.0006) [2023-12-27 03:29:10,958][105692] Updated weights for policy 0, policy_version 1659859 (0.0007) [2023-12-27 03:29:11,015][105692] Updated weights for policy 0, policy_version 1659870 (0.0009) [2023-12-27 03:29:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 20070.4, 300 sec: 19605.3). Total num frames: 850845696. Throughput: 0: 10014.1, 1: 10021.8. Samples: 850848968. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:29:11,063][104569] Avg episode reward: [(0, '8712.077'), (1, '8989.442')] [2023-12-27 03:29:11,447][105620] Updated weights for policy 1, policy_version 1663274 (0.0008) [2023-12-27 03:29:11,517][105620] Updated weights for policy 1, policy_version 1663284 (0.0008) [2023-12-27 03:29:11,580][105620] Updated weights for policy 1, policy_version 1663294 (0.0009) [2023-12-27 03:29:11,844][105692] Updated weights for policy 0, policy_version 1659880 (0.0007) [2023-12-27 03:29:11,893][105692] Updated weights for policy 0, policy_version 1659890 (0.0006) [2023-12-27 03:29:11,956][105692] Updated weights for policy 0, policy_version 1659900 (0.0009) [2023-12-27 03:29:12,355][105620] Updated weights for policy 1, policy_version 1663304 (0.0008) [2023-12-27 03:29:12,421][105620] Updated weights for policy 1, policy_version 1663314 (0.0008) [2023-12-27 03:29:12,479][105620] Updated weights for policy 1, policy_version 1663324 (0.0008) [2023-12-27 03:29:12,711][105692] Updated weights for policy 0, policy_version 1659910 (0.0008) [2023-12-27 03:29:12,779][105692] Updated weights for policy 0, policy_version 1659920 (0.0009) [2023-12-27 03:29:12,839][105692] Updated weights for policy 0, policy_version 1659930 (0.0010) [2023-12-27 03:29:13,158][105620] Updated weights for policy 1, policy_version 1663334 (0.0008) [2023-12-27 03:29:13,221][105620] Updated weights for policy 1, policy_version 1663344 (0.0011) [2023-12-27 03:29:13,281][105620] Updated weights for policy 1, policy_version 1663354 (0.0011) [2023-12-27 03:29:13,420][105692] Updated weights for policy 0, policy_version 1659940 (0.0008) [2023-12-27 03:29:13,490][105692] Updated weights for policy 0, policy_version 1659950 (0.0005) [2023-12-27 03:29:13,550][105692] Updated weights for policy 0, policy_version 1659960 (0.0005) [2023-12-27 03:29:13,941][105620] Updated weights for policy 1, policy_version 1663364 (0.0010) [2023-12-27 03:29:14,003][105620] Updated weights for policy 1, policy_version 1663374 (0.0010) [2023-12-27 03:29:14,060][105620] Updated weights for policy 1, policy_version 1663384 (0.0010) [2023-12-27 03:29:14,105][105692] Updated weights for policy 0, policy_version 1659970 (0.0008) [2023-12-27 03:29:14,172][105692] Updated weights for policy 0, policy_version 1659980 (0.0006) [2023-12-27 03:29:14,221][105692] Updated weights for policy 0, policy_version 1659990 (0.0009) [2023-12-27 03:29:14,270][105692] Updated weights for policy 0, policy_version 1660000 (0.0006) [2023-12-27 03:29:14,777][105620] Updated weights for policy 1, policy_version 1663394 (0.0011) [2023-12-27 03:29:14,848][105620] Updated weights for policy 1, policy_version 1663404 (0.0011) [2023-12-27 03:29:14,911][105620] Updated weights for policy 1, policy_version 1663414 (0.0011) [2023-12-27 03:29:14,941][105692] Updated weights for policy 0, policy_version 1660010 (0.0005) [2023-12-27 03:29:14,967][105620] Updated weights for policy 1, policy_version 1663424 (0.0011) [2023-12-27 03:29:14,999][105692] Updated weights for policy 0, policy_version 1660020 (0.0007) [2023-12-27 03:29:15,059][105692] Updated weights for policy 0, policy_version 1660030 (0.0008) [2023-12-27 03:29:15,696][105620] Updated weights for policy 1, policy_version 1663434 (0.0010) [2023-12-27 03:29:15,744][105620] Updated weights for policy 1, policy_version 1663444 (0.0010) [2023-12-27 03:29:15,790][105692] Updated weights for policy 0, policy_version 1660040 (0.0006) [2023-12-27 03:29:15,796][105620] Updated weights for policy 1, policy_version 1663454 (0.0010) [2023-12-27 03:29:15,850][105692] Updated weights for policy 0, policy_version 1660050 (0.0005) [2023-12-27 03:29:15,909][105692] Updated weights for policy 0, policy_version 1660060 (0.0008) [2023-12-27 03:29:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 20070.5, 300 sec: 19633.0). Total num frames: 850944000. Throughput: 0: 10015.4, 1: 9972.3. Samples: 850908936. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:29:16,062][104569] Avg episode reward: [(0, '8713.008'), (1, '9356.785')] [2023-12-27 03:29:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001663456_425902080.pth... [2023-12-27 03:29:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001660064_425041920.pth... [2023-12-27 03:29:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001662272_425598976.pth [2023-12-27 03:29:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001658880_424738816.pth [2023-12-27 03:29:16,427][105620] Updated weights for policy 1, policy_version 1663464 (0.0006) [2023-12-27 03:29:16,490][105620] Updated weights for policy 1, policy_version 1663474 (0.0011) [2023-12-27 03:29:16,515][105692] Updated weights for policy 0, policy_version 1660070 (0.0007) [2023-12-27 03:29:16,545][105620] Updated weights for policy 1, policy_version 1663484 (0.0009) [2023-12-27 03:29:16,579][105692] Updated weights for policy 0, policy_version 1660080 (0.0005) [2023-12-27 03:29:16,645][105692] Updated weights for policy 0, policy_version 1660090 (0.0005) [2023-12-27 03:29:17,139][105620] Updated weights for policy 1, policy_version 1663494 (0.0009) [2023-12-27 03:29:17,194][105692] Updated weights for policy 0, policy_version 1660100 (0.0005) [2023-12-27 03:29:17,200][105620] Updated weights for policy 1, policy_version 1663504 (0.0009) [2023-12-27 03:29:17,266][105692] Updated weights for policy 0, policy_version 1660110 (0.0005) [2023-12-27 03:29:17,266][105620] Updated weights for policy 1, policy_version 1663514 (0.0006) [2023-12-27 03:29:17,326][105692] Updated weights for policy 0, policy_version 1660120 (0.0005) [2023-12-27 03:29:17,854][105692] Updated weights for policy 0, policy_version 1660130 (0.0006) [2023-12-27 03:29:17,905][105692] Updated weights for policy 0, policy_version 1660140 (0.0008) [2023-12-27 03:29:17,961][105692] Updated weights for policy 0, policy_version 1660150 (0.0005) [2023-12-27 03:29:17,980][105620] Updated weights for policy 1, policy_version 1663524 (0.0011) [2023-12-27 03:29:18,019][105692] Updated weights for policy 0, policy_version 1660160 (0.0005) [2023-12-27 03:29:18,033][105620] Updated weights for policy 1, policy_version 1663534 (0.0011) [2023-12-27 03:29:18,083][105620] Updated weights for policy 1, policy_version 1663544 (0.0011) [2023-12-27 03:29:18,627][105692] Updated weights for policy 0, policy_version 1660170 (0.0005) [2023-12-27 03:29:18,686][105692] Updated weights for policy 0, policy_version 1660180 (0.0007) [2023-12-27 03:29:18,746][105692] Updated weights for policy 0, policy_version 1660190 (0.0010) [2023-12-27 03:29:18,846][105620] Updated weights for policy 1, policy_version 1663554 (0.0010) [2023-12-27 03:29:18,903][105620] Updated weights for policy 1, policy_version 1663564 (0.0011) [2023-12-27 03:29:18,962][105620] Updated weights for policy 1, policy_version 1663574 (0.0011) [2023-12-27 03:29:19,029][105620] Updated weights for policy 1, policy_version 1663584 (0.0011) [2023-12-27 03:29:19,462][105692] Updated weights for policy 0, policy_version 1660200 (0.0011) [2023-12-27 03:29:19,523][105692] Updated weights for policy 0, policy_version 1660210 (0.0010) [2023-12-27 03:29:19,583][105692] Updated weights for policy 0, policy_version 1660220 (0.0010) [2023-12-27 03:29:19,802][105620] Updated weights for policy 1, policy_version 1663594 (0.0008) [2023-12-27 03:29:19,869][105620] Updated weights for policy 1, policy_version 1663604 (0.0008) [2023-12-27 03:29:19,939][105620] Updated weights for policy 1, policy_version 1663614 (0.0008) [2023-12-27 03:29:20,323][105692] Updated weights for policy 0, policy_version 1660230 (0.0008) [2023-12-27 03:29:20,387][105692] Updated weights for policy 0, policy_version 1660240 (0.0006) [2023-12-27 03:29:20,454][105692] Updated weights for policy 0, policy_version 1660250 (0.0006) [2023-12-27 03:29:20,678][105620] Updated weights for policy 1, policy_version 1663624 (0.0010) [2023-12-27 03:29:20,748][105620] Updated weights for policy 1, policy_version 1663634 (0.0011) [2023-12-27 03:29:20,811][105620] Updated weights for policy 1, policy_version 1663644 (0.0011) [2023-12-27 03:29:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 851042304. Throughput: 0: 10205.2, 1: 9915.6. Samples: 851032108. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:29:21,063][104569] Avg episode reward: [(0, '8710.022'), (1, '9083.318')] [2023-12-27 03:29:21,080][105692] Updated weights for policy 0, policy_version 1660260 (0.0010) [2023-12-27 03:29:21,145][105692] Updated weights for policy 0, policy_version 1660270 (0.0011) [2023-12-27 03:29:21,210][105692] Updated weights for policy 0, policy_version 1660280 (0.0011) [2023-12-27 03:29:21,558][105620] Updated weights for policy 1, policy_version 1663654 (0.0010) [2023-12-27 03:29:21,626][105620] Updated weights for policy 1, policy_version 1663664 (0.0010) [2023-12-27 03:29:21,694][105620] Updated weights for policy 1, policy_version 1663674 (0.0007) [2023-12-27 03:29:21,945][105692] Updated weights for policy 0, policy_version 1660290 (0.0009) [2023-12-27 03:29:21,998][105692] Updated weights for policy 0, policy_version 1660300 (0.0010) [2023-12-27 03:29:22,055][105692] Updated weights for policy 0, policy_version 1660310 (0.0009) [2023-12-27 03:29:22,126][105692] Updated weights for policy 0, policy_version 1660320 (0.0009) [2023-12-27 03:29:22,415][105620] Updated weights for policy 1, policy_version 1663684 (0.0009) [2023-12-27 03:29:22,485][105620] Updated weights for policy 1, policy_version 1663694 (0.0009) [2023-12-27 03:29:22,552][105620] Updated weights for policy 1, policy_version 1663704 (0.0009) [2023-12-27 03:29:22,960][105692] Updated weights for policy 0, policy_version 1660330 (0.0011) [2023-12-27 03:29:23,022][105692] Updated weights for policy 0, policy_version 1660340 (0.0011) [2023-12-27 03:29:23,085][105692] Updated weights for policy 0, policy_version 1660350 (0.0007) [2023-12-27 03:29:23,223][105620] Updated weights for policy 1, policy_version 1663714 (0.0009) [2023-12-27 03:29:23,290][105620] Updated weights for policy 1, policy_version 1663724 (0.0010) [2023-12-27 03:29:23,341][105620] Updated weights for policy 1, policy_version 1663734 (0.0007) [2023-12-27 03:29:23,617][105692] Updated weights for policy 0, policy_version 1660360 (0.0005) [2023-12-27 03:29:23,682][105692] Updated weights for policy 0, policy_version 1660370 (0.0005) [2023-12-27 03:29:23,736][105692] Updated weights for policy 0, policy_version 1660380 (0.0006) [2023-12-27 03:29:24,167][105620] Updated weights for policy 1, policy_version 1663746 (0.0010) [2023-12-27 03:29:24,222][105620] Updated weights for policy 1, policy_version 1663756 (0.0008) [2023-12-27 03:29:24,282][105620] Updated weights for policy 1, policy_version 1663766 (0.0008) [2023-12-27 03:29:24,347][105620] Updated weights for policy 1, policy_version 1663776 (0.0007) [2023-12-27 03:29:24,356][105692] Updated weights for policy 0, policy_version 1660390 (0.0008) [2023-12-27 03:29:24,412][105692] Updated weights for policy 0, policy_version 1660400 (0.0011) [2023-12-27 03:29:24,472][105692] Updated weights for policy 0, policy_version 1660410 (0.0010) [2023-12-27 03:29:25,084][105692] Updated weights for policy 0, policy_version 1660420 (0.0010) [2023-12-27 03:29:25,145][105692] Updated weights for policy 0, policy_version 1660430 (0.0010) [2023-12-27 03:29:25,168][105620] Updated weights for policy 1, policy_version 1663786 (0.0010) [2023-12-27 03:29:25,207][105692] Updated weights for policy 0, policy_version 1660440 (0.0011) [2023-12-27 03:29:25,232][105620] Updated weights for policy 1, policy_version 1663796 (0.0005) [2023-12-27 03:29:25,289][105620] Updated weights for policy 1, policy_version 1663806 (0.0007) [2023-12-27 03:29:25,907][105620] Updated weights for policy 1, policy_version 1663816 (0.0006) [2023-12-27 03:29:25,940][105692] Updated weights for policy 0, policy_version 1660450 (0.0010) [2023-12-27 03:29:25,967][105620] Updated weights for policy 1, policy_version 1663826 (0.0006) [2023-12-27 03:29:25,988][105692] Updated weights for policy 0, policy_version 1660460 (0.0010) [2023-12-27 03:29:26,016][105620] Updated weights for policy 1, policy_version 1663836 (0.0005) [2023-12-27 03:29:26,037][105692] Updated weights for policy 0, policy_version 1660470 (0.0010) [2023-12-27 03:29:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 20070.5, 300 sec: 19605.3). Total num frames: 851140608. Throughput: 0: 10251.8, 1: 9978.9. Samples: 851149904. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:29:26,062][104569] Avg episode reward: [(0, '8620.273'), (1, '8807.499')] [2023-12-27 03:29:26,101][105692] Updated weights for policy 0, policy_version 1660480 (0.0010) [2023-12-27 03:29:26,650][105620] Updated weights for policy 1, policy_version 1663846 (0.0007) [2023-12-27 03:29:26,708][105620] Updated weights for policy 1, policy_version 1663856 (0.0006) [2023-12-27 03:29:26,762][105620] Updated weights for policy 1, policy_version 1663866 (0.0010) [2023-12-27 03:29:26,854][105692] Updated weights for policy 0, policy_version 1660490 (0.0010) [2023-12-27 03:29:26,916][105692] Updated weights for policy 0, policy_version 1660500 (0.0010) [2023-12-27 03:29:26,973][105692] Updated weights for policy 0, policy_version 1660510 (0.0010) [2023-12-27 03:29:27,422][105620] Updated weights for policy 1, policy_version 1663876 (0.0008) [2023-12-27 03:29:27,489][105620] Updated weights for policy 1, policy_version 1663886 (0.0006) [2023-12-27 03:29:27,557][105620] Updated weights for policy 1, policy_version 1663896 (0.0006) [2023-12-27 03:29:27,601][105692] Updated weights for policy 0, policy_version 1660520 (0.0006) [2023-12-27 03:29:27,649][105692] Updated weights for policy 0, policy_version 1660530 (0.0005) [2023-12-27 03:29:27,704][105692] Updated weights for policy 0, policy_version 1660540 (0.0005) [2023-12-27 03:29:28,131][105620] Updated weights for policy 1, policy_version 1663906 (0.0007) [2023-12-27 03:29:28,186][105620] Updated weights for policy 1, policy_version 1663916 (0.0006) [2023-12-27 03:29:28,237][105620] Updated weights for policy 1, policy_version 1663926 (0.0008) [2023-12-27 03:29:28,269][105692] Updated weights for policy 0, policy_version 1660550 (0.0007) [2023-12-27 03:29:28,330][105692] Updated weights for policy 0, policy_version 1660560 (0.0006) [2023-12-27 03:29:28,389][105692] Updated weights for policy 0, policy_version 1660570 (0.0011) [2023-12-27 03:29:28,940][105620] Updated weights for policy 1, policy_version 1663937 (0.0008) [2023-12-27 03:29:28,991][105620] Updated weights for policy 1, policy_version 1663947 (0.0008) [2023-12-27 03:29:29,040][105620] Updated weights for policy 1, policy_version 1663957 (0.0008) [2023-12-27 03:29:29,087][105620] Updated weights for policy 1, policy_version 1663967 (0.0008) [2023-12-27 03:29:29,097][105692] Updated weights for policy 0, policy_version 1660580 (0.0010) [2023-12-27 03:29:29,141][105692] Updated weights for policy 0, policy_version 1660590 (0.0010) [2023-12-27 03:29:29,191][105692] Updated weights for policy 0, policy_version 1660600 (0.0010) [2023-12-27 03:29:29,756][105620] Updated weights for policy 1, policy_version 1663977 (0.0005) [2023-12-27 03:29:29,808][105620] Updated weights for policy 1, policy_version 1663987 (0.0006) [2023-12-27 03:29:29,877][105620] Updated weights for policy 1, policy_version 1663997 (0.0008) [2023-12-27 03:29:29,899][105692] Updated weights for policy 0, policy_version 1660610 (0.0011) [2023-12-27 03:29:29,962][105692] Updated weights for policy 0, policy_version 1660620 (0.0010) [2023-12-27 03:29:30,024][105692] Updated weights for policy 0, policy_version 1660630 (0.0010) [2023-12-27 03:29:30,079][105692] Updated weights for policy 0, policy_version 1660640 (0.0010) [2023-12-27 03:29:30,442][105620] Updated weights for policy 1, policy_version 1664007 (0.0006) [2023-12-27 03:29:30,488][105620] Updated weights for policy 1, policy_version 1664017 (0.0005) [2023-12-27 03:29:30,521][105586] KL-divergence is very high: 232.4608 [2023-12-27 03:29:30,535][105620] Updated weights for policy 1, policy_version 1664027 (0.0008) [2023-12-27 03:29:30,760][105692] Updated weights for policy 0, policy_version 1660650 (0.0005) [2023-12-27 03:29:30,806][105692] Updated weights for policy 0, policy_version 1660660 (0.0005) [2023-12-27 03:29:30,855][105692] Updated weights for policy 0, policy_version 1660670 (0.0005) [2023-12-27 03:29:31,062][104569] Fps is (10 sec: 20479.7, 60 sec: 20206.9, 300 sec: 19660.8). Total num frames: 851247104. Throughput: 0: 10252.6, 1: 10011.0. Samples: 851213144. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:29:31,063][104569] Avg episode reward: [(0, '8714.060'), (1, '8805.529')] [2023-12-27 03:29:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001660672_425197568.pth... [2023-12-27 03:29:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001664032_426049536.pth... [2023-12-27 03:29:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001662880_425754624.pth [2023-12-27 03:29:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001659456_424886272.pth [2023-12-27 03:29:31,198][105620] Updated weights for policy 1, policy_version 1664037 (0.0009) [2023-12-27 03:29:31,252][105620] Updated weights for policy 1, policy_version 1664048 (0.0010) [2023-12-27 03:29:31,309][105620] Updated weights for policy 1, policy_version 1664058 (0.0010) [2023-12-27 03:29:31,412][105692] Updated weights for policy 0, policy_version 1660680 (0.0009) [2023-12-27 03:29:31,460][105692] Updated weights for policy 0, policy_version 1660690 (0.0010) [2023-12-27 03:29:31,512][105692] Updated weights for policy 0, policy_version 1660700 (0.0010) [2023-12-27 03:29:32,163][105620] Updated weights for policy 1, policy_version 1664069 (0.0009) [2023-12-27 03:29:32,219][105620] Updated weights for policy 1, policy_version 1664079 (0.0009) [2023-12-27 03:29:32,232][105692] Updated weights for policy 0, policy_version 1660710 (0.0010) [2023-12-27 03:29:32,286][105620] Updated weights for policy 1, policy_version 1664089 (0.0007) [2023-12-27 03:29:32,289][105692] Updated weights for policy 0, policy_version 1660720 (0.0008) [2023-12-27 03:29:32,347][105692] Updated weights for policy 0, policy_version 1660730 (0.0007) [2023-12-27 03:29:33,055][105620] Updated weights for policy 1, policy_version 1664099 (0.0006) [2023-12-27 03:29:33,056][105692] Updated weights for policy 0, policy_version 1660740 (0.0010) [2023-12-27 03:29:33,106][105620] Updated weights for policy 1, policy_version 1664109 (0.0005) [2023-12-27 03:29:33,120][105692] Updated weights for policy 0, policy_version 1660750 (0.0008) [2023-12-27 03:29:33,159][105620] Updated weights for policy 1, policy_version 1664119 (0.0005) [2023-12-27 03:29:33,175][105692] Updated weights for policy 0, policy_version 1660760 (0.0008) [2023-12-27 03:29:33,843][105692] Updated weights for policy 0, policy_version 1660770 (0.0007) [2023-12-27 03:29:33,899][105620] Updated weights for policy 1, policy_version 1664129 (0.0007) [2023-12-27 03:29:33,901][105692] Updated weights for policy 0, policy_version 1660780 (0.0010) [2023-12-27 03:29:33,953][105620] Updated weights for policy 1, policy_version 1664139 (0.0006) [2023-12-27 03:29:33,959][105692] Updated weights for policy 0, policy_version 1660790 (0.0009) [2023-12-27 03:29:34,011][105692] Updated weights for policy 0, policy_version 1660800 (0.0010) [2023-12-27 03:29:34,017][105620] Updated weights for policy 1, policy_version 1664149 (0.0006) [2023-12-27 03:29:34,070][105620] Updated weights for policy 1, policy_version 1664159 (0.0006) [2023-12-27 03:29:34,711][105692] Updated weights for policy 0, policy_version 1660810 (0.0010) [2023-12-27 03:29:34,762][105692] Updated weights for policy 0, policy_version 1660820 (0.0010) [2023-12-27 03:29:34,777][105620] Updated weights for policy 1, policy_version 1664169 (0.0006) [2023-12-27 03:29:34,809][105692] Updated weights for policy 0, policy_version 1660830 (0.0005) [2023-12-27 03:29:34,843][105620] Updated weights for policy 1, policy_version 1664179 (0.0010) [2023-12-27 03:29:34,909][105620] Updated weights for policy 1, policy_version 1664189 (0.0010) [2023-12-27 03:29:35,508][105620] Updated weights for policy 1, policy_version 1664199 (0.0007) [2023-12-27 03:29:35,560][105620] Updated weights for policy 1, policy_version 1664209 (0.0005) [2023-12-27 03:29:35,614][105620] Updated weights for policy 1, policy_version 1664219 (0.0006) [2023-12-27 03:29:35,617][105692] Updated weights for policy 0, policy_version 1660840 (0.0006) [2023-12-27 03:29:35,667][105692] Updated weights for policy 0, policy_version 1660850 (0.0009) [2023-12-27 03:29:35,719][105692] Updated weights for policy 0, policy_version 1660861 (0.0010) [2023-12-27 03:29:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 20206.9, 300 sec: 19660.8). Total num frames: 851345408. Throughput: 0: 10335.4, 1: 10001.3. Samples: 851333648. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:29:36,063][104569] Avg episode reward: [(0, '8530.444'), (1, '8990.654')] [2023-12-27 03:29:36,218][105620] Updated weights for policy 1, policy_version 1664229 (0.0008) [2023-12-27 03:29:36,285][105620] Updated weights for policy 1, policy_version 1664239 (0.0011) [2023-12-27 03:29:36,345][105620] Updated weights for policy 1, policy_version 1664249 (0.0010) [2023-12-27 03:29:36,565][105692] Updated weights for policy 0, policy_version 1660871 (0.0008) [2023-12-27 03:29:36,632][105692] Updated weights for policy 0, policy_version 1660881 (0.0007) [2023-12-27 03:29:36,694][105692] Updated weights for policy 0, policy_version 1660891 (0.0009) [2023-12-27 03:29:36,995][105620] Updated weights for policy 1, policy_version 1664259 (0.0010) [2023-12-27 03:29:37,044][105620] Updated weights for policy 1, policy_version 1664269 (0.0010) [2023-12-27 03:29:37,093][105620] Updated weights for policy 1, policy_version 1664279 (0.0010) [2023-12-27 03:29:37,438][105692] Updated weights for policy 0, policy_version 1660901 (0.0009) [2023-12-27 03:29:37,485][105692] Updated weights for policy 0, policy_version 1660911 (0.0009) [2023-12-27 03:29:37,537][105692] Updated weights for policy 0, policy_version 1660921 (0.0009) [2023-12-27 03:29:37,811][105620] Updated weights for policy 1, policy_version 1664289 (0.0010) [2023-12-27 03:29:37,859][105620] Updated weights for policy 1, policy_version 1664299 (0.0009) [2023-12-27 03:29:37,915][105620] Updated weights for policy 1, policy_version 1664309 (0.0010) [2023-12-27 03:29:37,972][105620] Updated weights for policy 1, policy_version 1664319 (0.0006) [2023-12-27 03:29:38,355][105692] Updated weights for policy 0, policy_version 1660931 (0.0008) [2023-12-27 03:29:38,413][105692] Updated weights for policy 0, policy_version 1660941 (0.0007) [2023-12-27 03:29:38,476][105692] Updated weights for policy 0, policy_version 1660951 (0.0005) [2023-12-27 03:29:38,728][105620] Updated weights for policy 1, policy_version 1664329 (0.0011) [2023-12-27 03:29:38,794][105620] Updated weights for policy 1, policy_version 1664339 (0.0011) [2023-12-27 03:29:38,846][105620] Updated weights for policy 1, policy_version 1664349 (0.0010) [2023-12-27 03:29:39,096][105692] Updated weights for policy 0, policy_version 1660961 (0.0006) [2023-12-27 03:29:39,151][105692] Updated weights for policy 0, policy_version 1660971 (0.0008) [2023-12-27 03:29:39,205][105692] Updated weights for policy 0, policy_version 1660981 (0.0008) [2023-12-27 03:29:39,261][105692] Updated weights for policy 0, policy_version 1660991 (0.0009) [2023-12-27 03:29:39,575][105620] Updated weights for policy 1, policy_version 1664359 (0.0010) [2023-12-27 03:29:39,641][105620] Updated weights for policy 1, policy_version 1664369 (0.0010) [2023-12-27 03:29:39,700][105620] Updated weights for policy 1, policy_version 1664379 (0.0010) [2023-12-27 03:29:40,102][105692] Updated weights for policy 0, policy_version 1661001 (0.0009) [2023-12-27 03:29:40,160][105692] Updated weights for policy 0, policy_version 1661011 (0.0010) [2023-12-27 03:29:40,222][105692] Updated weights for policy 0, policy_version 1661021 (0.0009) [2023-12-27 03:29:40,341][105620] Updated weights for policy 1, policy_version 1664389 (0.0007) [2023-12-27 03:29:40,408][105620] Updated weights for policy 1, policy_version 1664399 (0.0005) [2023-12-27 03:29:40,475][105620] Updated weights for policy 1, policy_version 1664409 (0.0007) [2023-12-27 03:29:41,016][105620] Updated weights for policy 1, policy_version 1664419 (0.0008) [2023-12-27 03:29:41,062][104569] Fps is (10 sec: 18842.0, 60 sec: 20070.4, 300 sec: 19660.8). Total num frames: 851435520. Throughput: 0: 10214.5, 1: 10087.3. Samples: 851450692. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:29:41,062][104569] Avg episode reward: [(0, '8804.125'), (1, '9172.273')] [2023-12-27 03:29:41,088][105620] Updated weights for policy 1, policy_version 1664429 (0.0007) [2023-12-27 03:29:41,123][105692] Updated weights for policy 0, policy_version 1661031 (0.0009) [2023-12-27 03:29:41,158][105620] Updated weights for policy 1, policy_version 1664439 (0.0007) [2023-12-27 03:29:41,193][105692] Updated weights for policy 0, policy_version 1661041 (0.0008) [2023-12-27 03:29:41,265][105692] Updated weights for policy 0, policy_version 1661051 (0.0008) [2023-12-27 03:29:41,921][105620] Updated weights for policy 1, policy_version 1664449 (0.0008) [2023-12-27 03:29:41,987][105620] Updated weights for policy 1, policy_version 1664459 (0.0009) [2023-12-27 03:29:42,005][105692] Updated weights for policy 0, policy_version 1661061 (0.0010) [2023-12-27 03:29:42,045][105620] Updated weights for policy 1, policy_version 1664469 (0.0008) [2023-12-27 03:29:42,060][105692] Updated weights for policy 0, policy_version 1661071 (0.0007) [2023-12-27 03:29:42,103][105620] Updated weights for policy 1, policy_version 1664479 (0.0007) [2023-12-27 03:29:42,117][105692] Updated weights for policy 0, policy_version 1661081 (0.0006) [2023-12-27 03:29:42,807][105692] Updated weights for policy 0, policy_version 1661091 (0.0009) [2023-12-27 03:29:42,858][105620] Updated weights for policy 1, policy_version 1664489 (0.0008) [2023-12-27 03:29:42,864][105692] Updated weights for policy 0, policy_version 1661101 (0.0009) [2023-12-27 03:29:42,921][105620] Updated weights for policy 1, policy_version 1664499 (0.0008) [2023-12-27 03:29:42,924][105692] Updated weights for policy 0, policy_version 1661111 (0.0006) [2023-12-27 03:29:42,981][105620] Updated weights for policy 1, policy_version 1664509 (0.0007) [2023-12-27 03:29:43,642][105692] Updated weights for policy 0, policy_version 1661121 (0.0006) [2023-12-27 03:29:43,698][105692] Updated weights for policy 0, policy_version 1661131 (0.0007) [2023-12-27 03:29:43,704][105620] Updated weights for policy 1, policy_version 1664519 (0.0008) [2023-12-27 03:29:43,758][105692] Updated weights for policy 0, policy_version 1661141 (0.0007) [2023-12-27 03:29:43,759][105620] Updated weights for policy 1, policy_version 1664529 (0.0006) [2023-12-27 03:29:43,814][105692] Updated weights for policy 0, policy_version 1661151 (0.0006) [2023-12-27 03:29:43,816][105620] Updated weights for policy 1, policy_version 1664539 (0.0008) [2023-12-27 03:29:44,530][105620] Updated weights for policy 1, policy_version 1664549 (0.0009) [2023-12-27 03:29:44,543][105692] Updated weights for policy 0, policy_version 1661161 (0.0005) [2023-12-27 03:29:44,579][105620] Updated weights for policy 1, policy_version 1664559 (0.0008) [2023-12-27 03:29:44,599][105692] Updated weights for policy 0, policy_version 1661171 (0.0007) [2023-12-27 03:29:44,623][105620] Updated weights for policy 1, policy_version 1664569 (0.0007) [2023-12-27 03:29:44,651][105692] Updated weights for policy 0, policy_version 1661181 (0.0005) [2023-12-27 03:29:45,323][105692] Updated weights for policy 0, policy_version 1661191 (0.0009) [2023-12-27 03:29:45,347][105620] Updated weights for policy 1, policy_version 1664579 (0.0005) [2023-12-27 03:29:45,385][105692] Updated weights for policy 0, policy_version 1661201 (0.0008) [2023-12-27 03:29:45,402][105620] Updated weights for policy 1, policy_version 1664589 (0.0005) [2023-12-27 03:29:45,434][105692] Updated weights for policy 0, policy_version 1661211 (0.0009) [2023-12-27 03:29:45,456][105620] Updated weights for policy 1, policy_version 1664599 (0.0007) [2023-12-27 03:29:46,062][104569] Fps is (10 sec: 18841.1, 60 sec: 20070.3, 300 sec: 19633.0). Total num frames: 851533824. Throughput: 0: 10044.6, 1: 9978.2. Samples: 851505696. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:29:46,063][104569] Avg episode reward: [(0, '8986.778'), (1, '9080.685')] [2023-12-27 03:29:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001661216_425336832.pth... [2023-12-27 03:29:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001664608_426196992.pth... [2023-12-27 03:29:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001660064_425041920.pth [2023-12-27 03:29:46,082][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001663456_425902080.pth [2023-12-27 03:29:46,127][105692] Updated weights for policy 0, policy_version 1661221 (0.0010) [2023-12-27 03:29:46,180][105692] Updated weights for policy 0, policy_version 1661231 (0.0008) [2023-12-27 03:29:46,209][105620] Updated weights for policy 1, policy_version 1664609 (0.0009) [2023-12-27 03:29:46,232][105692] Updated weights for policy 0, policy_version 1661241 (0.0006) [2023-12-27 03:29:46,262][105620] Updated weights for policy 1, policy_version 1664619 (0.0008) [2023-12-27 03:29:46,322][105620] Updated weights for policy 1, policy_version 1664629 (0.0009) [2023-12-27 03:29:46,374][105620] Updated weights for policy 1, policy_version 1664639 (0.0008) [2023-12-27 03:29:46,923][105692] Updated weights for policy 0, policy_version 1661251 (0.0008) [2023-12-27 03:29:46,971][105692] Updated weights for policy 0, policy_version 1661261 (0.0009) [2023-12-27 03:29:47,018][105692] Updated weights for policy 0, policy_version 1661271 (0.0006) [2023-12-27 03:29:47,176][105620] Updated weights for policy 1, policy_version 1664649 (0.0009) [2023-12-27 03:29:47,223][105620] Updated weights for policy 1, policy_version 1664659 (0.0009) [2023-12-27 03:29:47,272][105620] Updated weights for policy 1, policy_version 1664669 (0.0006) [2023-12-27 03:29:47,794][105692] Updated weights for policy 0, policy_version 1661281 (0.0007) [2023-12-27 03:29:47,841][105692] Updated weights for policy 0, policy_version 1661291 (0.0010) [2023-12-27 03:29:47,890][105692] Updated weights for policy 0, policy_version 1661301 (0.0010) [2023-12-27 03:29:47,898][105620] Updated weights for policy 1, policy_version 1664679 (0.0005) [2023-12-27 03:29:47,944][105692] Updated weights for policy 0, policy_version 1661311 (0.0010) [2023-12-27 03:29:47,946][105620] Updated weights for policy 1, policy_version 1664689 (0.0006) [2023-12-27 03:29:47,998][105620] Updated weights for policy 1, policy_version 1664699 (0.0006) [2023-12-27 03:29:48,609][105692] Updated weights for policy 0, policy_version 1661321 (0.0011) [2023-12-27 03:29:48,662][105692] Updated weights for policy 0, policy_version 1661331 (0.0011) [2023-12-27 03:29:48,711][105692] Updated weights for policy 0, policy_version 1661341 (0.0010) [2023-12-27 03:29:48,726][105620] Updated weights for policy 1, policy_version 1664709 (0.0008) [2023-12-27 03:29:48,787][105620] Updated weights for policy 1, policy_version 1664719 (0.0007) [2023-12-27 03:29:48,851][105620] Updated weights for policy 1, policy_version 1664729 (0.0005) [2023-12-27 03:29:49,382][105692] Updated weights for policy 0, policy_version 1661351 (0.0010) [2023-12-27 03:29:49,442][105692] Updated weights for policy 0, policy_version 1661361 (0.0010) [2023-12-27 03:29:49,503][105692] Updated weights for policy 0, policy_version 1661371 (0.0010) [2023-12-27 03:29:49,556][105620] Updated weights for policy 1, policy_version 1664739 (0.0007) [2023-12-27 03:29:49,610][105620] Updated weights for policy 1, policy_version 1664749 (0.0010) [2023-12-27 03:29:49,662][105620] Updated weights for policy 1, policy_version 1664759 (0.0009) [2023-12-27 03:29:50,114][105692] Updated weights for policy 0, policy_version 1661381 (0.0011) [2023-12-27 03:29:50,172][105692] Updated weights for policy 0, policy_version 1661391 (0.0008) [2023-12-27 03:29:50,240][105692] Updated weights for policy 0, policy_version 1661401 (0.0006) [2023-12-27 03:29:50,495][105620] Updated weights for policy 1, policy_version 1664769 (0.0010) [2023-12-27 03:29:50,563][105620] Updated weights for policy 1, policy_version 1664779 (0.0011) [2023-12-27 03:29:50,627][105620] Updated weights for policy 1, policy_version 1664789 (0.0012) [2023-12-27 03:29:50,687][105620] Updated weights for policy 1, policy_version 1664799 (0.0009) [2023-12-27 03:29:50,911][105692] Updated weights for policy 0, policy_version 1661411 (0.0009) [2023-12-27 03:29:50,969][105692] Updated weights for policy 0, policy_version 1661421 (0.0009) [2023-12-27 03:29:51,036][105692] Updated weights for policy 0, policy_version 1661431 (0.0009) [2023-12-27 03:29:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 20070.4, 300 sec: 19660.8). Total num frames: 851632128. Throughput: 0: 9997.3, 1: 9846.4. Samples: 851623664. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:29:51,062][104569] Avg episode reward: [(0, '8712.333'), (1, '8898.031')] [2023-12-27 03:29:51,461][105620] Updated weights for policy 1, policy_version 1664809 (0.0007) [2023-12-27 03:29:51,512][105620] Updated weights for policy 1, policy_version 1664819 (0.0010) [2023-12-27 03:29:51,564][105620] Updated weights for policy 1, policy_version 1664829 (0.0005) [2023-12-27 03:29:51,951][105692] Updated weights for policy 0, policy_version 1661441 (0.0009) [2023-12-27 03:29:52,006][105692] Updated weights for policy 0, policy_version 1661451 (0.0010) [2023-12-27 03:29:52,072][105692] Updated weights for policy 0, policy_version 1661461 (0.0008) [2023-12-27 03:29:52,125][105692] Updated weights for policy 0, policy_version 1661471 (0.0007) [2023-12-27 03:29:52,233][105620] Updated weights for policy 1, policy_version 1664839 (0.0007) [2023-12-27 03:29:52,287][105620] Updated weights for policy 1, policy_version 1664849 (0.0009) [2023-12-27 03:29:52,339][105620] Updated weights for policy 1, policy_version 1664859 (0.0009) [2023-12-27 03:29:52,841][105692] Updated weights for policy 0, policy_version 1661481 (0.0009) [2023-12-27 03:29:52,896][105692] Updated weights for policy 0, policy_version 1661491 (0.0009) [2023-12-27 03:29:52,955][105692] Updated weights for policy 0, policy_version 1661501 (0.0010) [2023-12-27 03:29:53,181][105620] Updated weights for policy 1, policy_version 1664869 (0.0008) [2023-12-27 03:29:53,233][105620] Updated weights for policy 1, policy_version 1664879 (0.0008) [2023-12-27 03:29:53,285][105620] Updated weights for policy 1, policy_version 1664889 (0.0008) [2023-12-27 03:29:53,614][105692] Updated weights for policy 0, policy_version 1661511 (0.0007) [2023-12-27 03:29:53,670][105692] Updated weights for policy 0, policy_version 1661521 (0.0009) [2023-12-27 03:29:53,724][105692] Updated weights for policy 0, policy_version 1661531 (0.0008) [2023-12-27 03:29:54,041][105620] Updated weights for policy 1, policy_version 1664899 (0.0008) [2023-12-27 03:29:54,102][105620] Updated weights for policy 1, policy_version 1664909 (0.0010) [2023-12-27 03:29:54,158][105620] Updated weights for policy 1, policy_version 1664919 (0.0009) [2023-12-27 03:29:54,341][105692] Updated weights for policy 0, policy_version 1661541 (0.0007) [2023-12-27 03:29:54,401][105692] Updated weights for policy 0, policy_version 1661551 (0.0008) [2023-12-27 03:29:54,457][105692] Updated weights for policy 0, policy_version 1661561 (0.0009) [2023-12-27 03:29:54,872][105620] Updated weights for policy 1, policy_version 1664929 (0.0008) [2023-12-27 03:29:54,924][105620] Updated weights for policy 1, policy_version 1664939 (0.0008) [2023-12-27 03:29:54,987][105620] Updated weights for policy 1, policy_version 1664949 (0.0010) [2023-12-27 03:29:55,046][105620] Updated weights for policy 1, policy_version 1664960 (0.0010) [2023-12-27 03:29:55,164][105692] Updated weights for policy 0, policy_version 1661571 (0.0009) [2023-12-27 03:29:55,222][105692] Updated weights for policy 0, policy_version 1661581 (0.0007) [2023-12-27 03:29:55,270][105692] Updated weights for policy 0, policy_version 1661591 (0.0010) [2023-12-27 03:29:55,852][105620] Updated weights for policy 1, policy_version 1664970 (0.0009) [2023-12-27 03:29:55,914][105620] Updated weights for policy 1, policy_version 1664980 (0.0008) [2023-12-27 03:29:55,958][105620] Updated weights for policy 1, policy_version 1664990 (0.0008) [2023-12-27 03:29:55,990][105692] Updated weights for policy 0, policy_version 1661601 (0.0008) [2023-12-27 03:29:56,046][105692] Updated weights for policy 0, policy_version 1661611 (0.0009) [2023-12-27 03:29:56,062][104569] Fps is (10 sec: 19661.4, 60 sec: 20070.5, 300 sec: 19660.8). Total num frames: 851730432. Throughput: 0: 10004.9, 1: 9788.5. Samples: 851739672. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:29:56,063][104569] Avg episode reward: [(0, '8895.906'), (1, '8717.288')] [2023-12-27 03:29:56,105][105692] Updated weights for policy 0, policy_version 1661621 (0.0010) [2023-12-27 03:29:56,170][105692] Updated weights for policy 0, policy_version 1661631 (0.0011) [2023-12-27 03:29:56,688][105620] Updated weights for policy 1, policy_version 1665000 (0.0009) [2023-12-27 03:29:56,752][105620] Updated weights for policy 1, policy_version 1665010 (0.0007) [2023-12-27 03:29:56,760][105692] Updated weights for policy 0, policy_version 1661641 (0.0010) [2023-12-27 03:29:56,805][105620] Updated weights for policy 1, policy_version 1665020 (0.0006) [2023-12-27 03:29:56,807][105692] Updated weights for policy 0, policy_version 1661651 (0.0007) [2023-12-27 03:29:56,854][105692] Updated weights for policy 0, policy_version 1661661 (0.0005) [2023-12-27 03:29:57,443][105692] Updated weights for policy 0, policy_version 1661671 (0.0005) [2023-12-27 03:29:57,487][105692] Updated weights for policy 0, policy_version 1661681 (0.0005) [2023-12-27 03:29:57,548][105692] Updated weights for policy 0, policy_version 1661691 (0.0005) [2023-12-27 03:29:57,588][105620] Updated weights for policy 1, policy_version 1665030 (0.0008) [2023-12-27 03:29:57,655][105620] Updated weights for policy 1, policy_version 1665040 (0.0009) [2023-12-27 03:29:57,719][105620] Updated weights for policy 1, policy_version 1665050 (0.0009) [2023-12-27 03:29:58,248][105692] Updated weights for policy 0, policy_version 1661701 (0.0008) [2023-12-27 03:29:58,306][105692] Updated weights for policy 0, policy_version 1661711 (0.0009) [2023-12-27 03:29:58,378][105692] Updated weights for policy 0, policy_version 1661721 (0.0008) [2023-12-27 03:29:58,462][105620] Updated weights for policy 1, policy_version 1665060 (0.0009) [2023-12-27 03:29:58,529][105620] Updated weights for policy 1, policy_version 1665070 (0.0008) [2023-12-27 03:29:58,599][105620] Updated weights for policy 1, policy_version 1665080 (0.0006) [2023-12-27 03:29:59,102][105692] Updated weights for policy 0, policy_version 1661731 (0.0008) [2023-12-27 03:29:59,156][105692] Updated weights for policy 0, policy_version 1661741 (0.0007) [2023-12-27 03:29:59,218][105692] Updated weights for policy 0, policy_version 1661751 (0.0008) [2023-12-27 03:29:59,412][105620] Updated weights for policy 1, policy_version 1665090 (0.0008) [2023-12-27 03:29:59,473][105620] Updated weights for policy 1, policy_version 1665100 (0.0009) [2023-12-27 03:29:59,538][105620] Updated weights for policy 1, policy_version 1665110 (0.0009) [2023-12-27 03:29:59,598][105620] Updated weights for policy 1, policy_version 1665120 (0.0009) [2023-12-27 03:29:59,936][105692] Updated weights for policy 0, policy_version 1661761 (0.0008) [2023-12-27 03:29:59,999][105692] Updated weights for policy 0, policy_version 1661771 (0.0009) [2023-12-27 03:30:00,050][105692] Updated weights for policy 0, policy_version 1661781 (0.0009) [2023-12-27 03:30:00,105][105692] Updated weights for policy 0, policy_version 1661791 (0.0009) [2023-12-27 03:30:00,318][105620] Updated weights for policy 1, policy_version 1665130 (0.0009) [2023-12-27 03:30:00,372][105620] Updated weights for policy 1, policy_version 1665140 (0.0008) [2023-12-27 03:30:00,429][105620] Updated weights for policy 1, policy_version 1665150 (0.0007) [2023-12-27 03:30:00,850][105692] Updated weights for policy 0, policy_version 1661801 (0.0009) [2023-12-27 03:30:00,904][105692] Updated weights for policy 0, policy_version 1661811 (0.0009) [2023-12-27 03:30:00,951][105692] Updated weights for policy 0, policy_version 1661821 (0.0009) [2023-12-27 03:30:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 851828736. Throughput: 0: 10028.9, 1: 9733.0. Samples: 851798228. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:30:01,062][104569] Avg episode reward: [(0, '8623.932'), (1, '8717.064')] [2023-12-27 03:30:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001661824_425492480.pth... [2023-12-27 03:30:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001665152_426336256.pth... [2023-12-27 03:30:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001660672_425197568.pth [2023-12-27 03:30:01,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001664032_426049536.pth [2023-12-27 03:30:01,145][105620] Updated weights for policy 1, policy_version 1665160 (0.0009) [2023-12-27 03:30:01,202][105620] Updated weights for policy 1, policy_version 1665170 (0.0008) [2023-12-27 03:30:01,269][105620] Updated weights for policy 1, policy_version 1665180 (0.0009) [2023-12-27 03:30:01,687][105692] Updated weights for policy 0, policy_version 1661831 (0.0009) [2023-12-27 03:30:01,753][105692] Updated weights for policy 0, policy_version 1661841 (0.0008) [2023-12-27 03:30:01,806][105692] Updated weights for policy 0, policy_version 1661851 (0.0008) [2023-12-27 03:30:02,110][105620] Updated weights for policy 1, policy_version 1665190 (0.0010) [2023-12-27 03:30:02,168][105620] Updated weights for policy 1, policy_version 1665200 (0.0009) [2023-12-27 03:30:02,231][105620] Updated weights for policy 1, policy_version 1665210 (0.0008) [2023-12-27 03:30:02,424][105692] Updated weights for policy 0, policy_version 1661861 (0.0008) [2023-12-27 03:30:02,475][105692] Updated weights for policy 0, policy_version 1661871 (0.0009) [2023-12-27 03:30:02,525][105692] Updated weights for policy 0, policy_version 1661881 (0.0009) [2023-12-27 03:30:03,012][105620] Updated weights for policy 1, policy_version 1665220 (0.0009) [2023-12-27 03:30:03,066][105620] Updated weights for policy 1, policy_version 1665230 (0.0009) [2023-12-27 03:30:03,119][105620] Updated weights for policy 1, policy_version 1665240 (0.0009) [2023-12-27 03:30:03,183][105692] Updated weights for policy 0, policy_version 1661891 (0.0010) [2023-12-27 03:30:03,243][105692] Updated weights for policy 0, policy_version 1661901 (0.0009) [2023-12-27 03:30:03,303][105692] Updated weights for policy 0, policy_version 1661911 (0.0009) [2023-12-27 03:30:03,837][105620] Updated weights for policy 1, policy_version 1665250 (0.0010) [2023-12-27 03:30:03,896][105620] Updated weights for policy 1, policy_version 1665260 (0.0009) [2023-12-27 03:30:03,955][105620] Updated weights for policy 1, policy_version 1665270 (0.0009) [2023-12-27 03:30:04,012][105620] Updated weights for policy 1, policy_version 1665280 (0.0009) [2023-12-27 03:30:04,064][105692] Updated weights for policy 0, policy_version 1661921 (0.0009) [2023-12-27 03:30:04,155][105692] Updated weights for policy 0, policy_version 1661931 (0.0009) [2023-12-27 03:30:04,215][105692] Updated weights for policy 0, policy_version 1661941 (0.0009) [2023-12-27 03:30:04,270][105692] Updated weights for policy 0, policy_version 1661951 (0.0009) [2023-12-27 03:30:04,770][105620] Updated weights for policy 1, policy_version 1665290 (0.0005) [2023-12-27 03:30:04,831][105620] Updated weights for policy 1, policy_version 1665300 (0.0005) [2023-12-27 03:30:04,892][105620] Updated weights for policy 1, policy_version 1665310 (0.0005) [2023-12-27 03:30:05,067][105692] Updated weights for policy 0, policy_version 1661961 (0.0010) [2023-12-27 03:30:05,126][105692] Updated weights for policy 0, policy_version 1661971 (0.0010) [2023-12-27 03:30:05,183][105692] Updated weights for policy 0, policy_version 1661982 (0.0009) [2023-12-27 03:30:05,384][105620] Updated weights for policy 1, policy_version 1665320 (0.0005) [2023-12-27 03:30:05,431][105620] Updated weights for policy 1, policy_version 1665330 (0.0006) [2023-12-27 03:30:05,436][105586] KL-divergence is very high: 161.1257 [2023-12-27 03:30:05,472][105586] KL-divergence is very high: 191.3189 [2023-12-27 03:30:05,476][105620] Updated weights for policy 1, policy_version 1665340 (0.0006) [2023-12-27 03:30:06,003][105620] Updated weights for policy 1, policy_version 1665350 (0.0008) [2023-12-27 03:30:06,057][105620] Updated weights for policy 1, policy_version 1665360 (0.0008) [2023-12-27 03:30:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 851918848. Throughput: 0: 9891.9, 1: 9676.4. Samples: 851912680. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:30:06,062][104569] Avg episode reward: [(0, '8440.535'), (1, '8804.841')] [2023-12-27 03:30:06,104][105692] Updated weights for policy 0, policy_version 1661992 (0.0008) [2023-12-27 03:30:06,106][105620] Updated weights for policy 1, policy_version 1665370 (0.0006) [2023-12-27 03:30:06,171][105692] Updated weights for policy 0, policy_version 1662002 (0.0009) [2023-12-27 03:30:06,228][105692] Updated weights for policy 0, policy_version 1662012 (0.0009) [2023-12-27 03:30:06,776][105620] Updated weights for policy 1, policy_version 1665380 (0.0007) [2023-12-27 03:30:06,826][105620] Updated weights for policy 1, policy_version 1665390 (0.0009) [2023-12-27 03:30:06,878][105620] Updated weights for policy 1, policy_version 1665400 (0.0009) [2023-12-27 03:30:07,026][105692] Updated weights for policy 0, policy_version 1662022 (0.0009) [2023-12-27 03:30:07,076][105692] Updated weights for policy 0, policy_version 1662032 (0.0009) [2023-12-27 03:30:07,124][105692] Updated weights for policy 0, policy_version 1662042 (0.0009) [2023-12-27 03:30:07,661][105620] Updated weights for policy 1, policy_version 1665410 (0.0009) [2023-12-27 03:30:07,722][105620] Updated weights for policy 1, policy_version 1665420 (0.0009) [2023-12-27 03:30:07,784][105620] Updated weights for policy 1, policy_version 1665430 (0.0009) [2023-12-27 03:30:07,842][105620] Updated weights for policy 1, policy_version 1665440 (0.0009) [2023-12-27 03:30:07,887][105692] Updated weights for policy 0, policy_version 1662052 (0.0009) [2023-12-27 03:30:07,934][105692] Updated weights for policy 0, policy_version 1662062 (0.0008) [2023-12-27 03:30:07,985][105692] Updated weights for policy 0, policy_version 1662072 (0.0009) [2023-12-27 03:30:08,609][105620] Updated weights for policy 1, policy_version 1665450 (0.0009) [2023-12-27 03:30:08,660][105620] Updated weights for policy 1, policy_version 1665460 (0.0009) [2023-12-27 03:30:08,714][105620] Updated weights for policy 1, policy_version 1665470 (0.0008) [2023-12-27 03:30:08,784][105692] Updated weights for policy 0, policy_version 1662082 (0.0009) [2023-12-27 03:30:08,839][105692] Updated weights for policy 0, policy_version 1662092 (0.0009) [2023-12-27 03:30:08,885][105692] Updated weights for policy 0, policy_version 1662102 (0.0008) [2023-12-27 03:30:08,933][105692] Updated weights for policy 0, policy_version 1662112 (0.0009) [2023-12-27 03:30:09,523][105620] Updated weights for policy 1, policy_version 1665480 (0.0008) [2023-12-27 03:30:09,575][105620] Updated weights for policy 1, policy_version 1665490 (0.0009) [2023-12-27 03:30:09,632][105620] Updated weights for policy 1, policy_version 1665500 (0.0009) [2023-12-27 03:30:09,703][105692] Updated weights for policy 0, policy_version 1662122 (0.0006) [2023-12-27 03:30:09,769][105692] Updated weights for policy 0, policy_version 1662132 (0.0006) [2023-12-27 03:30:09,843][105692] Updated weights for policy 0, policy_version 1662142 (0.0009) [2023-12-27 03:30:10,408][105620] Updated weights for policy 1, policy_version 1665510 (0.0010) [2023-12-27 03:30:10,480][105620] Updated weights for policy 1, policy_version 1665520 (0.0008) [2023-12-27 03:30:10,503][105692] Updated weights for policy 0, policy_version 1662152 (0.0006) [2023-12-27 03:30:10,541][105620] Updated weights for policy 1, policy_version 1665530 (0.0009) [2023-12-27 03:30:10,561][105692] Updated weights for policy 0, policy_version 1662162 (0.0007) [2023-12-27 03:30:10,623][105692] Updated weights for policy 0, policy_version 1662172 (0.0009) [2023-12-27 03:30:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 852017152. Throughput: 0: 9749.7, 1: 9759.5. Samples: 852027816. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:30:11,062][104569] Avg episode reward: [(0, '8712.053'), (1, '8989.492')] [2023-12-27 03:30:11,138][105620] Updated weights for policy 1, policy_version 1665540 (0.0008) [2023-12-27 03:30:11,199][105620] Updated weights for policy 1, policy_version 1665550 (0.0009) [2023-12-27 03:30:11,263][105620] Updated weights for policy 1, policy_version 1665560 (0.0010) [2023-12-27 03:30:11,420][105692] Updated weights for policy 0, policy_version 1662182 (0.0007) [2023-12-27 03:30:11,477][105692] Updated weights for policy 0, policy_version 1662192 (0.0007) [2023-12-27 03:30:11,530][105692] Updated weights for policy 0, policy_version 1662202 (0.0009) [2023-12-27 03:30:12,049][105620] Updated weights for policy 1, policy_version 1665570 (0.0007) [2023-12-27 03:30:12,109][105620] Updated weights for policy 1, policy_version 1665580 (0.0009) [2023-12-27 03:30:12,165][105620] Updated weights for policy 1, policy_version 1665590 (0.0009) [2023-12-27 03:30:12,223][105620] Updated weights for policy 1, policy_version 1665600 (0.0009) [2023-12-27 03:30:12,307][105692] Updated weights for policy 0, policy_version 1662212 (0.0009) [2023-12-27 03:30:12,369][105692] Updated weights for policy 0, policy_version 1662222 (0.0008) [2023-12-27 03:30:12,420][105692] Updated weights for policy 0, policy_version 1662232 (0.0009) [2023-12-27 03:30:13,007][105620] Updated weights for policy 1, policy_version 1665610 (0.0007) [2023-12-27 03:30:13,055][105620] Updated weights for policy 1, policy_version 1665620 (0.0009) [2023-12-27 03:30:13,108][105620] Updated weights for policy 1, policy_version 1665630 (0.0010) [2023-12-27 03:30:13,156][105692] Updated weights for policy 0, policy_version 1662242 (0.0008) [2023-12-27 03:30:13,220][105692] Updated weights for policy 0, policy_version 1662252 (0.0007) [2023-12-27 03:30:13,281][105692] Updated weights for policy 0, policy_version 1662262 (0.0009) [2023-12-27 03:30:13,345][105692] Updated weights for policy 0, policy_version 1662272 (0.0009) [2023-12-27 03:30:13,913][105620] Updated weights for policy 1, policy_version 1665640 (0.0010) [2023-12-27 03:30:13,968][105620] Updated weights for policy 1, policy_version 1665650 (0.0010) [2023-12-27 03:30:14,013][105692] Updated weights for policy 0, policy_version 1662282 (0.0006) [2023-12-27 03:30:14,023][105620] Updated weights for policy 1, policy_version 1665660 (0.0010) [2023-12-27 03:30:14,069][105692] Updated weights for policy 0, policy_version 1662292 (0.0007) [2023-12-27 03:30:14,130][105692] Updated weights for policy 0, policy_version 1662302 (0.0008) [2023-12-27 03:30:14,787][105620] Updated weights for policy 1, policy_version 1665670 (0.0009) [2023-12-27 03:30:14,840][105620] Updated weights for policy 1, policy_version 1665680 (0.0010) [2023-12-27 03:30:14,907][105692] Updated weights for policy 0, policy_version 1662312 (0.0006) [2023-12-27 03:30:14,908][105620] Updated weights for policy 1, policy_version 1665690 (0.0011) [2023-12-27 03:30:14,963][105692] Updated weights for policy 0, policy_version 1662322 (0.0009) [2023-12-27 03:30:15,027][105692] Updated weights for policy 0, policy_version 1662332 (0.0008) [2023-12-27 03:30:15,685][105620] Updated weights for policy 1, policy_version 1665700 (0.0010) [2023-12-27 03:30:15,740][105620] Updated weights for policy 1, policy_version 1665710 (0.0010) [2023-12-27 03:30:15,790][105692] Updated weights for policy 0, policy_version 1662342 (0.0008) [2023-12-27 03:30:15,795][105620] Updated weights for policy 1, policy_version 1665720 (0.0010) [2023-12-27 03:30:15,845][105692] Updated weights for policy 0, policy_version 1662352 (0.0006) [2023-12-27 03:30:15,890][105692] Updated weights for policy 0, policy_version 1662362 (0.0008) [2023-12-27 03:30:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 852115456. Throughput: 0: 9666.4, 1: 9657.6. Samples: 852082724. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:30:16,062][104569] Avg episode reward: [(0, '8805.419'), (1, '9081.696')] [2023-12-27 03:30:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001665728_426483712.pth... [2023-12-27 03:30:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001662368_425631744.pth... [2023-12-27 03:30:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001664608_426196992.pth [2023-12-27 03:30:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001661216_425336832.pth [2023-12-27 03:30:16,551][105620] Updated weights for policy 1, policy_version 1665730 (0.0010) [2023-12-27 03:30:16,599][105620] Updated weights for policy 1, policy_version 1665740 (0.0010) [2023-12-27 03:30:16,660][105620] Updated weights for policy 1, policy_version 1665750 (0.0010) [2023-12-27 03:30:16,666][105692] Updated weights for policy 0, policy_version 1662372 (0.0007) [2023-12-27 03:30:16,714][105620] Updated weights for policy 1, policy_version 1665760 (0.0010) [2023-12-27 03:30:16,716][105692] Updated weights for policy 0, policy_version 1662382 (0.0005) [2023-12-27 03:30:16,775][105692] Updated weights for policy 0, policy_version 1662392 (0.0008) [2023-12-27 03:30:17,459][105620] Updated weights for policy 1, policy_version 1665770 (0.0010) [2023-12-27 03:30:17,516][105620] Updated weights for policy 1, policy_version 1665780 (0.0010) [2023-12-27 03:30:17,534][105692] Updated weights for policy 0, policy_version 1662402 (0.0008) [2023-12-27 03:30:17,568][105620] Updated weights for policy 1, policy_version 1665790 (0.0010) [2023-12-27 03:30:17,586][105692] Updated weights for policy 0, policy_version 1662412 (0.0006) [2023-12-27 03:30:17,643][105692] Updated weights for policy 0, policy_version 1662422 (0.0008) [2023-12-27 03:30:17,691][105692] Updated weights for policy 0, policy_version 1662432 (0.0008) [2023-12-27 03:30:18,314][105620] Updated weights for policy 1, policy_version 1665800 (0.0010) [2023-12-27 03:30:18,374][105620] Updated weights for policy 1, policy_version 1665810 (0.0011) [2023-12-27 03:30:18,429][105620] Updated weights for policy 1, policy_version 1665820 (0.0010) [2023-12-27 03:30:18,490][105692] Updated weights for policy 0, policy_version 1662442 (0.0009) [2023-12-27 03:30:18,557][105692] Updated weights for policy 0, policy_version 1662452 (0.0008) [2023-12-27 03:30:18,617][105692] Updated weights for policy 0, policy_version 1662462 (0.0008) [2023-12-27 03:30:19,192][105620] Updated weights for policy 1, policy_version 1665830 (0.0010) [2023-12-27 03:30:19,258][105620] Updated weights for policy 1, policy_version 1665840 (0.0008) [2023-12-27 03:30:19,323][105620] Updated weights for policy 1, policy_version 1665850 (0.0008) [2023-12-27 03:30:19,347][105692] Updated weights for policy 0, policy_version 1662472 (0.0007) [2023-12-27 03:30:19,408][105692] Updated weights for policy 0, policy_version 1662482 (0.0006) [2023-12-27 03:30:19,472][105692] Updated weights for policy 0, policy_version 1662492 (0.0006) [2023-12-27 03:30:20,112][105620] Updated weights for policy 1, policy_version 1665860 (0.0010) [2023-12-27 03:30:20,176][105620] Updated weights for policy 1, policy_version 1665870 (0.0008) [2023-12-27 03:30:20,202][105692] Updated weights for policy 0, policy_version 1662502 (0.0009) [2023-12-27 03:30:20,240][105620] Updated weights for policy 1, policy_version 1665880 (0.0006) [2023-12-27 03:30:20,255][105692] Updated weights for policy 0, policy_version 1662512 (0.0007) [2023-12-27 03:30:20,318][105692] Updated weights for policy 0, policy_version 1662522 (0.0008) [2023-12-27 03:30:20,956][105620] Updated weights for policy 1, policy_version 1665890 (0.0006) [2023-12-27 03:30:21,009][105620] Updated weights for policy 1, policy_version 1665900 (0.0009) [2023-12-27 03:30:21,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 852197376. Throughput: 0: 9536.9, 1: 9577.4. Samples: 852193792. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:30:21,062][104569] Avg episode reward: [(0, '8716.956'), (1, '8989.074')] [2023-12-27 03:30:21,082][105620] Updated weights for policy 1, policy_version 1665910 (0.0009) [2023-12-27 03:30:21,137][105692] Updated weights for policy 0, policy_version 1662532 (0.0009) [2023-12-27 03:30:21,144][105620] Updated weights for policy 1, policy_version 1665920 (0.0009) [2023-12-27 03:30:21,200][105692] Updated weights for policy 0, policy_version 1662542 (0.0009) [2023-12-27 03:30:21,254][105692] Updated weights for policy 0, policy_version 1662552 (0.0008) [2023-12-27 03:30:21,872][105620] Updated weights for policy 1, policy_version 1665930 (0.0006) [2023-12-27 03:30:21,931][105620] Updated weights for policy 1, policy_version 1665940 (0.0006) [2023-12-27 03:30:22,006][105620] Updated weights for policy 1, policy_version 1665950 (0.0008) [2023-12-27 03:30:22,114][105692] Updated weights for policy 0, policy_version 1662562 (0.0008) [2023-12-27 03:30:22,173][105692] Updated weights for policy 0, policy_version 1662572 (0.0009) [2023-12-27 03:30:22,229][105692] Updated weights for policy 0, policy_version 1662582 (0.0010) [2023-12-27 03:30:22,288][105692] Updated weights for policy 0, policy_version 1662592 (0.0009) [2023-12-27 03:30:22,690][105620] Updated weights for policy 1, policy_version 1665960 (0.0008) [2023-12-27 03:30:22,745][105620] Updated weights for policy 1, policy_version 1665970 (0.0009) [2023-12-27 03:30:22,800][105620] Updated weights for policy 1, policy_version 1665980 (0.0008) [2023-12-27 03:30:23,106][105692] Updated weights for policy 0, policy_version 1662602 (0.0009) [2023-12-27 03:30:23,166][105692] Updated weights for policy 0, policy_version 1662613 (0.0010) [2023-12-27 03:30:23,220][105692] Updated weights for policy 0, policy_version 1662624 (0.0010) [2023-12-27 03:30:23,460][105620] Updated weights for policy 1, policy_version 1665990 (0.0007) [2023-12-27 03:30:23,510][105620] Updated weights for policy 1, policy_version 1666000 (0.0009) [2023-12-27 03:30:23,573][105620] Updated weights for policy 1, policy_version 1666010 (0.0011) [2023-12-27 03:30:24,068][105692] Updated weights for policy 0, policy_version 1662634 (0.0010) [2023-12-27 03:30:24,122][105692] Updated weights for policy 0, policy_version 1662644 (0.0010) [2023-12-27 03:30:24,180][105692] Updated weights for policy 0, policy_version 1662654 (0.0009) [2023-12-27 03:30:24,182][105620] Updated weights for policy 1, policy_version 1666020 (0.0009) [2023-12-27 03:30:24,239][105620] Updated weights for policy 1, policy_version 1666030 (0.0005) [2023-12-27 03:30:24,300][105620] Updated weights for policy 1, policy_version 1666040 (0.0005) [2023-12-27 03:30:24,916][105620] Updated weights for policy 1, policy_version 1666050 (0.0006) [2023-12-27 03:30:24,976][105620] Updated weights for policy 1, policy_version 1666060 (0.0011) [2023-12-27 03:30:25,043][105620] Updated weights for policy 1, policy_version 1666070 (0.0011) [2023-12-27 03:30:25,088][105692] Updated weights for policy 0, policy_version 1662664 (0.0008) [2023-12-27 03:30:25,098][105620] Updated weights for policy 1, policy_version 1666080 (0.0008) [2023-12-27 03:30:25,157][105692] Updated weights for policy 0, policy_version 1662674 (0.0010) [2023-12-27 03:30:25,219][105692] Updated weights for policy 0, policy_version 1662684 (0.0010) [2023-12-27 03:30:25,748][105620] Updated weights for policy 1, policy_version 1666090 (0.0011) [2023-12-27 03:30:25,809][105620] Updated weights for policy 1, policy_version 1666100 (0.0010) [2023-12-27 03:30:25,873][105620] Updated weights for policy 1, policy_version 1666110 (0.0010) [2023-12-27 03:30:26,006][105692] Updated weights for policy 0, policy_version 1662694 (0.0009) [2023-12-27 03:30:26,062][105692] Updated weights for policy 0, policy_version 1662704 (0.0008) [2023-12-27 03:30:26,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 852295680. Throughput: 0: 9468.2, 1: 9537.1. Samples: 852305932. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:30:26,063][104569] Avg episode reward: [(0, '8624.960'), (1, '9081.239')] [2023-12-27 03:30:26,114][105692] Updated weights for policy 0, policy_version 1662714 (0.0008) [2023-12-27 03:30:26,599][105620] Updated weights for policy 1, policy_version 1666120 (0.0010) [2023-12-27 03:30:26,659][105620] Updated weights for policy 1, policy_version 1666130 (0.0010) [2023-12-27 03:30:26,726][105620] Updated weights for policy 1, policy_version 1666140 (0.0010) [2023-12-27 03:30:26,819][105692] Updated weights for policy 0, policy_version 1662724 (0.0007) [2023-12-27 03:30:26,868][105692] Updated weights for policy 0, policy_version 1662734 (0.0008) [2023-12-27 03:30:26,927][105692] Updated weights for policy 0, policy_version 1662744 (0.0008) [2023-12-27 03:30:27,462][105620] Updated weights for policy 1, policy_version 1666150 (0.0010) [2023-12-27 03:30:27,528][105620] Updated weights for policy 1, policy_version 1666160 (0.0010) [2023-12-27 03:30:27,592][105620] Updated weights for policy 1, policy_version 1666170 (0.0010) [2023-12-27 03:30:27,606][105692] Updated weights for policy 0, policy_version 1662754 (0.0007) [2023-12-27 03:30:27,649][105692] Updated weights for policy 0, policy_version 1662764 (0.0005) [2023-12-27 03:30:27,692][105692] Updated weights for policy 0, policy_version 1662774 (0.0005) [2023-12-27 03:30:27,739][105692] Updated weights for policy 0, policy_version 1662784 (0.0005) [2023-12-27 03:30:28,304][105620] Updated weights for policy 1, policy_version 1666180 (0.0010) [2023-12-27 03:30:28,366][105620] Updated weights for policy 1, policy_version 1666190 (0.0011) [2023-12-27 03:30:28,389][105692] Updated weights for policy 0, policy_version 1662794 (0.0010) [2023-12-27 03:30:28,428][105620] Updated weights for policy 1, policy_version 1666200 (0.0010) [2023-12-27 03:30:28,453][105692] Updated weights for policy 0, policy_version 1662804 (0.0008) [2023-12-27 03:30:28,512][105692] Updated weights for policy 0, policy_version 1662814 (0.0011) [2023-12-27 03:30:29,161][105620] Updated weights for policy 1, policy_version 1666210 (0.0010) [2023-12-27 03:30:29,168][105692] Updated weights for policy 0, policy_version 1662824 (0.0007) [2023-12-27 03:30:29,219][105620] Updated weights for policy 1, policy_version 1666220 (0.0011) [2023-12-27 03:30:29,225][105692] Updated weights for policy 0, policy_version 1662834 (0.0008) [2023-12-27 03:30:29,278][105620] Updated weights for policy 1, policy_version 1666230 (0.0011) [2023-12-27 03:30:29,287][105692] Updated weights for policy 0, policy_version 1662844 (0.0007) [2023-12-27 03:30:29,348][105620] Updated weights for policy 1, policy_version 1666240 (0.0010) [2023-12-27 03:30:29,883][105692] Updated weights for policy 0, policy_version 1662854 (0.0006) [2023-12-27 03:30:29,899][105620] Updated weights for policy 1, policy_version 1666250 (0.0009) [2023-12-27 03:30:29,944][105692] Updated weights for policy 0, policy_version 1662864 (0.0007) [2023-12-27 03:30:29,963][105620] Updated weights for policy 1, policy_version 1666260 (0.0011) [2023-12-27 03:30:30,002][105692] Updated weights for policy 0, policy_version 1662874 (0.0007) [2023-12-27 03:30:30,019][105620] Updated weights for policy 1, policy_version 1666270 (0.0010) [2023-12-27 03:30:30,688][105620] Updated weights for policy 1, policy_version 1666280 (0.0010) [2023-12-27 03:30:30,704][105692] Updated weights for policy 0, policy_version 1662884 (0.0006) [2023-12-27 03:30:30,746][105620] Updated weights for policy 1, policy_version 1666290 (0.0011) [2023-12-27 03:30:30,758][105692] Updated weights for policy 0, policy_version 1662894 (0.0010) [2023-12-27 03:30:30,794][105620] Updated weights for policy 1, policy_version 1666300 (0.0010) [2023-12-27 03:30:30,804][105692] Updated weights for policy 0, policy_version 1662904 (0.0008) [2023-12-27 03:30:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19251.2, 300 sec: 19660.8). Total num frames: 852402176. Throughput: 0: 9538.9, 1: 9556.4. Samples: 852364980. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:30:31,063][104569] Avg episode reward: [(0, '8712.081'), (1, '8989.280')] [2023-12-27 03:30:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001662912_425771008.pth... [2023-12-27 03:30:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001666304_426631168.pth... [2023-12-27 03:30:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001665152_426336256.pth [2023-12-27 03:30:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001661824_425492480.pth [2023-12-27 03:30:31,527][105620] Updated weights for policy 1, policy_version 1666310 (0.0010) [2023-12-27 03:30:31,534][105692] Updated weights for policy 0, policy_version 1662914 (0.0007) [2023-12-27 03:30:31,579][105620] Updated weights for policy 1, policy_version 1666320 (0.0010) [2023-12-27 03:30:31,593][105692] Updated weights for policy 0, policy_version 1662924 (0.0006) [2023-12-27 03:30:31,647][105620] Updated weights for policy 1, policy_version 1666330 (0.0011) [2023-12-27 03:30:31,658][105692] Updated weights for policy 0, policy_version 1662934 (0.0009) [2023-12-27 03:30:31,714][105692] Updated weights for policy 0, policy_version 1662944 (0.0011) [2023-12-27 03:30:32,255][105620] Updated weights for policy 1, policy_version 1666340 (0.0008) [2023-12-27 03:30:32,325][105620] Updated weights for policy 1, policy_version 1666350 (0.0008) [2023-12-27 03:30:32,388][105620] Updated weights for policy 1, policy_version 1666360 (0.0007) [2023-12-27 03:30:32,446][105692] Updated weights for policy 0, policy_version 1662954 (0.0011) [2023-12-27 03:30:32,509][105692] Updated weights for policy 0, policy_version 1662964 (0.0011) [2023-12-27 03:30:32,564][105692] Updated weights for policy 0, policy_version 1662974 (0.0011) [2023-12-27 03:30:33,086][105620] Updated weights for policy 1, policy_version 1666370 (0.0006) [2023-12-27 03:30:33,144][105620] Updated weights for policy 1, policy_version 1666380 (0.0005) [2023-12-27 03:30:33,202][105620] Updated weights for policy 1, policy_version 1666390 (0.0008) [2023-12-27 03:30:33,260][105620] Updated weights for policy 1, policy_version 1666400 (0.0008) [2023-12-27 03:30:33,294][105692] Updated weights for policy 0, policy_version 1662984 (0.0011) [2023-12-27 03:30:33,345][105692] Updated weights for policy 0, policy_version 1662994 (0.0008) [2023-12-27 03:30:33,392][105692] Updated weights for policy 0, policy_version 1663004 (0.0010) [2023-12-27 03:30:33,927][105620] Updated weights for policy 1, policy_version 1666410 (0.0009) [2023-12-27 03:30:33,982][105620] Updated weights for policy 1, policy_version 1666420 (0.0008) [2023-12-27 03:30:34,040][105620] Updated weights for policy 1, policy_version 1666430 (0.0008) [2023-12-27 03:30:34,150][105692] Updated weights for policy 0, policy_version 1663014 (0.0011) [2023-12-27 03:30:34,216][105692] Updated weights for policy 0, policy_version 1663024 (0.0011) [2023-12-27 03:30:34,277][105692] Updated weights for policy 0, policy_version 1663034 (0.0011) [2023-12-27 03:30:34,829][105620] Updated weights for policy 1, policy_version 1666440 (0.0008) [2023-12-27 03:30:34,891][105620] Updated weights for policy 1, policy_version 1666450 (0.0006) [2023-12-27 03:30:34,954][105620] Updated weights for policy 1, policy_version 1666460 (0.0007) [2023-12-27 03:30:35,008][105692] Updated weights for policy 0, policy_version 1663044 (0.0009) [2023-12-27 03:30:35,064][105692] Updated weights for policy 0, policy_version 1663054 (0.0009) [2023-12-27 03:30:35,118][105692] Updated weights for policy 0, policy_version 1663066 (0.0010) [2023-12-27 03:30:35,580][105620] Updated weights for policy 1, policy_version 1666470 (0.0009) [2023-12-27 03:30:35,631][105620] Updated weights for policy 1, policy_version 1666480 (0.0010) [2023-12-27 03:30:35,674][105620] Updated weights for policy 1, policy_version 1666490 (0.0009) [2023-12-27 03:30:35,836][105692] Updated weights for policy 0, policy_version 1663077 (0.0009) [2023-12-27 03:30:35,898][105692] Updated weights for policy 0, policy_version 1663087 (0.0007) [2023-12-27 03:30:35,959][105692] Updated weights for policy 0, policy_version 1663097 (0.0009) [2023-12-27 03:30:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19251.2, 300 sec: 19660.8). Total num frames: 852500480. Throughput: 0: 9515.9, 1: 9616.0. Samples: 852484600. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:30:36,062][104569] Avg episode reward: [(0, '8437.253'), (1, '8806.867')] [2023-12-27 03:30:36,351][105620] Updated weights for policy 1, policy_version 1666500 (0.0010) [2023-12-27 03:30:36,412][105620] Updated weights for policy 1, policy_version 1666510 (0.0008) [2023-12-27 03:30:36,470][105620] Updated weights for policy 1, policy_version 1666520 (0.0009) [2023-12-27 03:30:36,782][105692] Updated weights for policy 0, policy_version 1663107 (0.0009) [2023-12-27 03:30:36,841][105692] Updated weights for policy 0, policy_version 1663117 (0.0010) [2023-12-27 03:30:36,895][105692] Updated weights for policy 0, policy_version 1663127 (0.0009) [2023-12-27 03:30:37,079][105620] Updated weights for policy 1, policy_version 1666530 (0.0009) [2023-12-27 03:30:37,132][105620] Updated weights for policy 1, policy_version 1666540 (0.0009) [2023-12-27 03:30:37,183][105620] Updated weights for policy 1, policy_version 1666550 (0.0008) [2023-12-27 03:30:37,230][105620] Updated weights for policy 1, policy_version 1666560 (0.0005) [2023-12-27 03:30:37,592][105692] Updated weights for policy 0, policy_version 1663137 (0.0009) [2023-12-27 03:30:37,645][105692] Updated weights for policy 0, policy_version 1663147 (0.0009) [2023-12-27 03:30:37,704][105692] Updated weights for policy 0, policy_version 1663157 (0.0008) [2023-12-27 03:30:37,774][105692] Updated weights for policy 0, policy_version 1663167 (0.0006) [2023-12-27 03:30:37,847][105620] Updated weights for policy 1, policy_version 1666570 (0.0006) [2023-12-27 03:30:37,912][105620] Updated weights for policy 1, policy_version 1666580 (0.0005) [2023-12-27 03:30:37,977][105620] Updated weights for policy 1, policy_version 1666590 (0.0005) [2023-12-27 03:30:38,409][105692] Updated weights for policy 0, policy_version 1663177 (0.0009) [2023-12-27 03:30:38,466][105692] Updated weights for policy 0, policy_version 1663187 (0.0009) [2023-12-27 03:30:38,520][105692] Updated weights for policy 0, policy_version 1663197 (0.0010) [2023-12-27 03:30:38,527][105620] Updated weights for policy 1, policy_version 1666600 (0.0006) [2023-12-27 03:30:38,583][105620] Updated weights for policy 1, policy_version 1666610 (0.0006) [2023-12-27 03:30:38,631][105620] Updated weights for policy 1, policy_version 1666620 (0.0005) [2023-12-27 03:30:39,120][105692] Updated weights for policy 0, policy_version 1663207 (0.0007) [2023-12-27 03:30:39,177][105692] Updated weights for policy 0, policy_version 1663217 (0.0009) [2023-12-27 03:30:39,246][105692] Updated weights for policy 0, policy_version 1663227 (0.0011) [2023-12-27 03:30:39,266][105620] Updated weights for policy 1, policy_version 1666630 (0.0007) [2023-12-27 03:30:39,318][105620] Updated weights for policy 1, policy_version 1666640 (0.0009) [2023-12-27 03:30:39,374][105620] Updated weights for policy 1, policy_version 1666650 (0.0008) [2023-12-27 03:30:39,998][105692] Updated weights for policy 0, policy_version 1663237 (0.0011) [2023-12-27 03:30:40,051][105692] Updated weights for policy 0, policy_version 1663247 (0.0011) [2023-12-27 03:30:40,111][105692] Updated weights for policy 0, policy_version 1663257 (0.0011) [2023-12-27 03:30:40,155][105620] Updated weights for policy 1, policy_version 1666660 (0.0007) [2023-12-27 03:30:40,208][105620] Updated weights for policy 1, policy_version 1666670 (0.0008) [2023-12-27 03:30:40,258][105620] Updated weights for policy 1, policy_version 1666680 (0.0008) [2023-12-27 03:30:40,879][105692] Updated weights for policy 0, policy_version 1663267 (0.0011) [2023-12-27 03:30:40,931][105692] Updated weights for policy 0, policy_version 1663277 (0.0010) [2023-12-27 03:30:40,982][105692] Updated weights for policy 0, policy_version 1663287 (0.0010) [2023-12-27 03:30:41,031][105620] Updated weights for policy 1, policy_version 1666690 (0.0008) [2023-12-27 03:30:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 852598784. Throughput: 0: 9489.9, 1: 9768.6. Samples: 852606308. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:30:41,063][104569] Avg episode reward: [(0, '8712.711'), (1, '9265.115')] [2023-12-27 03:30:41,098][105620] Updated weights for policy 1, policy_version 1666700 (0.0007) [2023-12-27 03:30:41,163][105620] Updated weights for policy 1, policy_version 1666710 (0.0008) [2023-12-27 03:30:41,231][105620] Updated weights for policy 1, policy_version 1666720 (0.0008) [2023-12-27 03:30:41,793][105692] Updated weights for policy 0, policy_version 1663297 (0.0011) [2023-12-27 03:30:41,856][105692] Updated weights for policy 0, policy_version 1663307 (0.0011) [2023-12-27 03:30:41,923][105692] Updated weights for policy 0, policy_version 1663317 (0.0011) [2023-12-27 03:30:41,983][105692] Updated weights for policy 0, policy_version 1663327 (0.0011) [2023-12-27 03:30:41,994][105620] Updated weights for policy 1, policy_version 1666730 (0.0007) [2023-12-27 03:30:42,047][105620] Updated weights for policy 1, policy_version 1666740 (0.0008) [2023-12-27 03:30:42,099][105620] Updated weights for policy 1, policy_version 1666750 (0.0008) [2023-12-27 03:30:42,724][105692] Updated weights for policy 0, policy_version 1663337 (0.0006) [2023-12-27 03:30:42,782][105692] Updated weights for policy 0, policy_version 1663347 (0.0005) [2023-12-27 03:30:42,839][105692] Updated weights for policy 0, policy_version 1663357 (0.0006) [2023-12-27 03:30:42,858][105620] Updated weights for policy 1, policy_version 1666760 (0.0008) [2023-12-27 03:30:42,919][105620] Updated weights for policy 1, policy_version 1666770 (0.0010) [2023-12-27 03:30:42,978][105620] Updated weights for policy 1, policy_version 1666780 (0.0010) [2023-12-27 03:30:43,506][105692] Updated weights for policy 0, policy_version 1663367 (0.0007) [2023-12-27 03:30:43,560][105692] Updated weights for policy 0, policy_version 1663377 (0.0009) [2023-12-27 03:30:43,608][105692] Updated weights for policy 0, policy_version 1663387 (0.0007) [2023-12-27 03:30:43,624][105620] Updated weights for policy 1, policy_version 1666790 (0.0010) [2023-12-27 03:30:43,675][105620] Updated weights for policy 1, policy_version 1666800 (0.0005) [2023-12-27 03:30:43,726][105620] Updated weights for policy 1, policy_version 1666810 (0.0005) [2023-12-27 03:30:44,305][105620] Updated weights for policy 1, policy_version 1666820 (0.0007) [2023-12-27 03:30:44,316][105692] Updated weights for policy 0, policy_version 1663397 (0.0008) [2023-12-27 03:30:44,356][105620] Updated weights for policy 1, policy_version 1666830 (0.0010) [2023-12-27 03:30:44,370][105692] Updated weights for policy 0, policy_version 1663407 (0.0006) [2023-12-27 03:30:44,414][105620] Updated weights for policy 1, policy_version 1666840 (0.0010) [2023-12-27 03:30:44,431][105692] Updated weights for policy 0, policy_version 1663417 (0.0006) [2023-12-27 03:30:44,986][105692] Updated weights for policy 0, policy_version 1663427 (0.0006) [2023-12-27 03:30:45,060][105692] Updated weights for policy 0, policy_version 1663437 (0.0006) [2023-12-27 03:30:45,129][105692] Updated weights for policy 0, policy_version 1663447 (0.0006) [2023-12-27 03:30:45,170][105620] Updated weights for policy 1, policy_version 1666850 (0.0010) [2023-12-27 03:30:45,234][105620] Updated weights for policy 1, policy_version 1666860 (0.0006) [2023-12-27 03:30:45,307][105620] Updated weights for policy 1, policy_version 1666870 (0.0007) [2023-12-27 03:30:45,368][105620] Updated weights for policy 1, policy_version 1666880 (0.0011) [2023-12-27 03:30:45,792][105692] Updated weights for policy 0, policy_version 1663457 (0.0008) [2023-12-27 03:30:45,851][105692] Updated weights for policy 0, policy_version 1663467 (0.0005) [2023-12-27 03:30:45,907][105692] Updated weights for policy 0, policy_version 1663477 (0.0006) [2023-12-27 03:30:45,970][105692] Updated weights for policy 0, policy_version 1663487 (0.0010) [2023-12-27 03:30:46,056][105620] Updated weights for policy 1, policy_version 1666890 (0.0010) [2023-12-27 03:30:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19660.8). Total num frames: 852697088. Throughput: 0: 9418.7, 1: 9813.0. Samples: 852663652. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:30:46,062][104569] Avg episode reward: [(0, '9169.979'), (1, '9355.048')] [2023-12-27 03:30:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001663488_425918464.pth... [2023-12-27 03:30:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001662368_425631744.pth [2023-12-27 03:30:46,114][105620] Updated weights for policy 1, policy_version 1666900 (0.0010) [2023-12-27 03:30:46,172][105620] Updated weights for policy 1, policy_version 1666910 (0.0010) [2023-12-27 03:30:46,184][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001666912_426786816.pth... [2023-12-27 03:30:46,187][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001665728_426483712.pth [2023-12-27 03:30:46,666][105692] Updated weights for policy 0, policy_version 1663497 (0.0010) [2023-12-27 03:30:46,728][105692] Updated weights for policy 0, policy_version 1663507 (0.0010) [2023-12-27 03:30:46,783][105692] Updated weights for policy 0, policy_version 1663517 (0.0010) [2023-12-27 03:30:46,900][105620] Updated weights for policy 1, policy_version 1666920 (0.0010) [2023-12-27 03:30:46,955][105620] Updated weights for policy 1, policy_version 1666930 (0.0010) [2023-12-27 03:30:47,010][105620] Updated weights for policy 1, policy_version 1666940 (0.0010) [2023-12-27 03:30:47,527][105692] Updated weights for policy 0, policy_version 1663527 (0.0007) [2023-12-27 03:30:47,578][105692] Updated weights for policy 0, policy_version 1663537 (0.0005) [2023-12-27 03:30:47,629][105692] Updated weights for policy 0, policy_version 1663547 (0.0005) [2023-12-27 03:30:47,716][105620] Updated weights for policy 1, policy_version 1666950 (0.0010) [2023-12-27 03:30:47,763][105620] Updated weights for policy 1, policy_version 1666960 (0.0010) [2023-12-27 03:30:47,808][105620] Updated weights for policy 1, policy_version 1666970 (0.0010) [2023-12-27 03:30:48,202][105692] Updated weights for policy 0, policy_version 1663557 (0.0005) [2023-12-27 03:30:48,255][105692] Updated weights for policy 0, policy_version 1663567 (0.0005) [2023-12-27 03:30:48,313][105692] Updated weights for policy 0, policy_version 1663577 (0.0008) [2023-12-27 03:30:48,621][105620] Updated weights for policy 1, policy_version 1666980 (0.0010) [2023-12-27 03:30:48,666][105620] Updated weights for policy 1, policy_version 1666990 (0.0010) [2023-12-27 03:30:48,718][105620] Updated weights for policy 1, policy_version 1667000 (0.0010) [2023-12-27 03:30:48,960][105692] Updated weights for policy 0, policy_version 1663587 (0.0009) [2023-12-27 03:30:49,006][105692] Updated weights for policy 0, policy_version 1663597 (0.0005) [2023-12-27 03:30:49,061][105692] Updated weights for policy 0, policy_version 1663607 (0.0005) [2023-12-27 03:30:49,494][105620] Updated weights for policy 1, policy_version 1667010 (0.0010) [2023-12-27 03:30:49,551][105620] Updated weights for policy 1, policy_version 1667020 (0.0007) [2023-12-27 03:30:49,613][105620] Updated weights for policy 1, policy_version 1667030 (0.0005) [2023-12-27 03:30:49,681][105620] Updated weights for policy 1, policy_version 1667040 (0.0007) [2023-12-27 03:30:49,710][105692] Updated weights for policy 0, policy_version 1663617 (0.0006) [2023-12-27 03:30:49,777][105692] Updated weights for policy 0, policy_version 1663627 (0.0007) [2023-12-27 03:30:49,841][105692] Updated weights for policy 0, policy_version 1663637 (0.0007) [2023-12-27 03:30:49,904][105692] Updated weights for policy 0, policy_version 1663647 (0.0006) [2023-12-27 03:30:50,470][105692] Updated weights for policy 0, policy_version 1663657 (0.0005) [2023-12-27 03:30:50,487][105620] Updated weights for policy 1, policy_version 1667050 (0.0008) [2023-12-27 03:30:50,519][105692] Updated weights for policy 0, policy_version 1663667 (0.0005) [2023-12-27 03:30:50,536][105620] Updated weights for policy 1, policy_version 1667060 (0.0007) [2023-12-27 03:30:50,576][105692] Updated weights for policy 0, policy_version 1663677 (0.0007) [2023-12-27 03:30:50,597][105620] Updated weights for policy 1, policy_version 1667070 (0.0008) [2023-12-27 03:30:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 852795392. Throughput: 0: 9528.1, 1: 9857.2. Samples: 852785020. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:30:51,062][104569] Avg episode reward: [(0, '8709.863'), (1, '9170.249')] [2023-12-27 03:30:51,180][105692] Updated weights for policy 0, policy_version 1663687 (0.0009) [2023-12-27 03:30:51,227][105692] Updated weights for policy 0, policy_version 1663697 (0.0009) [2023-12-27 03:30:51,286][105620] Updated weights for policy 1, policy_version 1667080 (0.0009) [2023-12-27 03:30:51,293][105692] Updated weights for policy 0, policy_version 1663707 (0.0006) [2023-12-27 03:30:51,354][105620] Updated weights for policy 1, policy_version 1667090 (0.0008) [2023-12-27 03:30:51,417][105620] Updated weights for policy 1, policy_version 1667100 (0.0008) [2023-12-27 03:30:52,026][105692] Updated weights for policy 0, policy_version 1663717 (0.0009) [2023-12-27 03:30:52,088][105692] Updated weights for policy 0, policy_version 1663727 (0.0010) [2023-12-27 03:30:52,148][105692] Updated weights for policy 0, policy_version 1663737 (0.0010) [2023-12-27 03:30:52,197][105620] Updated weights for policy 1, policy_version 1667110 (0.0010) [2023-12-27 03:30:52,253][105620] Updated weights for policy 1, policy_version 1667120 (0.0009) [2023-12-27 03:30:52,320][105620] Updated weights for policy 1, policy_version 1667130 (0.0007) [2023-12-27 03:30:52,903][105692] Updated weights for policy 0, policy_version 1663747 (0.0010) [2023-12-27 03:30:52,942][105620] Updated weights for policy 1, policy_version 1667140 (0.0007) [2023-12-27 03:30:52,956][105692] Updated weights for policy 0, policy_version 1663757 (0.0010) [2023-12-27 03:30:52,998][105620] Updated weights for policy 1, policy_version 1667150 (0.0006) [2023-12-27 03:30:53,007][105692] Updated weights for policy 0, policy_version 1663767 (0.0010) [2023-12-27 03:30:53,054][105620] Updated weights for policy 1, policy_version 1667160 (0.0006) [2023-12-27 03:30:53,745][105692] Updated weights for policy 0, policy_version 1663777 (0.0010) [2023-12-27 03:30:53,803][105692] Updated weights for policy 0, policy_version 1663787 (0.0010) [2023-12-27 03:30:53,815][105620] Updated weights for policy 1, policy_version 1667170 (0.0008) [2023-12-27 03:30:53,855][105692] Updated weights for policy 0, policy_version 1663797 (0.0010) [2023-12-27 03:30:53,872][105620] Updated weights for policy 1, policy_version 1667180 (0.0005) [2023-12-27 03:30:53,917][105692] Updated weights for policy 0, policy_version 1663807 (0.0010) [2023-12-27 03:30:53,927][105620] Updated weights for policy 1, policy_version 1667190 (0.0005) [2023-12-27 03:30:53,987][105620] Updated weights for policy 1, policy_version 1667200 (0.0006) [2023-12-27 03:30:54,689][105692] Updated weights for policy 0, policy_version 1663817 (0.0008) [2023-12-27 03:30:54,722][105620] Updated weights for policy 1, policy_version 1667210 (0.0010) [2023-12-27 03:30:54,750][105692] Updated weights for policy 0, policy_version 1663827 (0.0008) [2023-12-27 03:30:54,784][105620] Updated weights for policy 1, policy_version 1667220 (0.0008) [2023-12-27 03:30:54,795][105692] Updated weights for policy 0, policy_version 1663837 (0.0008) [2023-12-27 03:30:54,842][105620] Updated weights for policy 1, policy_version 1667230 (0.0008) [2023-12-27 03:30:55,481][105692] Updated weights for policy 0, policy_version 1663847 (0.0008) [2023-12-27 03:30:55,528][105692] Updated weights for policy 0, policy_version 1663857 (0.0009) [2023-12-27 03:30:55,586][105692] Updated weights for policy 0, policy_version 1663867 (0.0008) [2023-12-27 03:30:55,651][105620] Updated weights for policy 1, policy_version 1667240 (0.0008) [2023-12-27 03:30:55,713][105620] Updated weights for policy 1, policy_version 1667250 (0.0009) [2023-12-27 03:30:55,774][105620] Updated weights for policy 1, policy_version 1667260 (0.0009) [2023-12-27 03:30:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 852893696. Throughput: 0: 9642.8, 1: 9759.1. Samples: 852900900. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:30:56,062][104569] Avg episode reward: [(0, '8441.517'), (1, '8988.065')] [2023-12-27 03:30:56,398][105692] Updated weights for policy 0, policy_version 1663877 (0.0009) [2023-12-27 03:30:56,406][105620] Updated weights for policy 1, policy_version 1667270 (0.0008) [2023-12-27 03:30:56,454][105692] Updated weights for policy 0, policy_version 1663887 (0.0011) [2023-12-27 03:30:56,461][105620] Updated weights for policy 1, policy_version 1667280 (0.0011) [2023-12-27 03:30:56,489][105586] KL-divergence is very high: 121.3211 [2023-12-27 03:30:56,509][105692] Updated weights for policy 0, policy_version 1663897 (0.0011) [2023-12-27 03:30:56,515][105620] Updated weights for policy 1, policy_version 1667290 (0.0009) [2023-12-27 03:30:56,531][105586] KL-divergence is very high: 114.6305 [2023-12-27 03:30:57,213][105620] Updated weights for policy 1, policy_version 1667300 (0.0007) [2023-12-27 03:30:57,220][105692] Updated weights for policy 0, policy_version 1663907 (0.0010) [2023-12-27 03:30:57,269][105620] Updated weights for policy 1, policy_version 1667310 (0.0011) [2023-12-27 03:30:57,269][105692] Updated weights for policy 0, policy_version 1663917 (0.0009) [2023-12-27 03:30:57,320][105620] Updated weights for policy 1, policy_version 1667320 (0.0010) [2023-12-27 03:30:57,322][105692] Updated weights for policy 0, policy_version 1663927 (0.0006) [2023-12-27 03:30:57,990][105692] Updated weights for policy 0, policy_version 1663937 (0.0010) [2023-12-27 03:30:58,017][105620] Updated weights for policy 1, policy_version 1667330 (0.0010) [2023-12-27 03:30:58,046][105692] Updated weights for policy 0, policy_version 1663947 (0.0006) [2023-12-27 03:30:58,075][105620] Updated weights for policy 1, policy_version 1667340 (0.0010) [2023-12-27 03:30:58,097][105692] Updated weights for policy 0, policy_version 1663957 (0.0006) [2023-12-27 03:30:58,129][105620] Updated weights for policy 1, policy_version 1667350 (0.0010) [2023-12-27 03:30:58,153][105692] Updated weights for policy 0, policy_version 1663967 (0.0006) [2023-12-27 03:30:58,200][105620] Updated weights for policy 1, policy_version 1667360 (0.0007) [2023-12-27 03:30:58,940][105692] Updated weights for policy 0, policy_version 1663977 (0.0009) [2023-12-27 03:30:59,000][105692] Updated weights for policy 0, policy_version 1663987 (0.0008) [2023-12-27 03:30:59,018][105620] Updated weights for policy 1, policy_version 1667370 (0.0006) [2023-12-27 03:30:59,062][105692] Updated weights for policy 0, policy_version 1663997 (0.0007) [2023-12-27 03:30:59,086][105620] Updated weights for policy 1, policy_version 1667380 (0.0008) [2023-12-27 03:30:59,146][105620] Updated weights for policy 1, policy_version 1667390 (0.0008) [2023-12-27 03:30:59,836][105692] Updated weights for policy 0, policy_version 1664007 (0.0008) [2023-12-27 03:30:59,898][105692] Updated weights for policy 0, policy_version 1664017 (0.0009) [2023-12-27 03:30:59,955][105620] Updated weights for policy 1, policy_version 1667400 (0.0007) [2023-12-27 03:30:59,965][105692] Updated weights for policy 0, policy_version 1664027 (0.0009) [2023-12-27 03:31:00,013][105620] Updated weights for policy 1, policy_version 1667410 (0.0007) [2023-12-27 03:31:00,063][105620] Updated weights for policy 1, policy_version 1667420 (0.0009) [2023-12-27 03:31:00,635][105692] Updated weights for policy 0, policy_version 1664037 (0.0007) [2023-12-27 03:31:00,686][105692] Updated weights for policy 0, policy_version 1664047 (0.0008) [2023-12-27 03:31:00,744][105692] Updated weights for policy 0, policy_version 1664057 (0.0009) [2023-12-27 03:31:00,846][105620] Updated weights for policy 1, policy_version 1667430 (0.0007) [2023-12-27 03:31:00,902][105620] Updated weights for policy 1, policy_version 1667440 (0.0006) [2023-12-27 03:31:00,955][105620] Updated weights for policy 1, policy_version 1667450 (0.0006) [2023-12-27 03:31:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19660.8). Total num frames: 852992000. Throughput: 0: 9685.4, 1: 9806.8. Samples: 852959876. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-12-27 03:31:01,062][104569] Avg episode reward: [(0, '8535.568'), (1, '8627.240')] [2023-12-27 03:31:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001664064_426065920.pth... [2023-12-27 03:31:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001667456_426926080.pth... [2023-12-27 03:31:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001662912_425771008.pth [2023-12-27 03:31:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001666304_426631168.pth [2023-12-27 03:31:01,565][105692] Updated weights for policy 0, policy_version 1664067 (0.0010) [2023-12-27 03:31:01,629][105692] Updated weights for policy 0, policy_version 1664077 (0.0009) [2023-12-27 03:31:01,668][105620] Updated weights for policy 1, policy_version 1667460 (0.0008) [2023-12-27 03:31:01,686][105692] Updated weights for policy 0, policy_version 1664087 (0.0008) [2023-12-27 03:31:01,730][105620] Updated weights for policy 1, policy_version 1667470 (0.0007) [2023-12-27 03:31:01,797][105620] Updated weights for policy 1, policy_version 1667480 (0.0007) [2023-12-27 03:31:02,424][105692] Updated weights for policy 0, policy_version 1664097 (0.0008) [2023-12-27 03:31:02,488][105692] Updated weights for policy 0, policy_version 1664107 (0.0011) [2023-12-27 03:31:02,545][105692] Updated weights for policy 0, policy_version 1664117 (0.0011) [2023-12-27 03:31:02,550][105620] Updated weights for policy 1, policy_version 1667490 (0.0010) [2023-12-27 03:31:02,601][105692] Updated weights for policy 0, policy_version 1664127 (0.0011) [2023-12-27 03:31:02,609][105620] Updated weights for policy 1, policy_version 1667500 (0.0008) [2023-12-27 03:31:02,668][105620] Updated weights for policy 1, policy_version 1667510 (0.0008) [2023-12-27 03:31:02,715][105620] Updated weights for policy 1, policy_version 1667520 (0.0008) [2023-12-27 03:31:03,254][105692] Updated weights for policy 0, policy_version 1664137 (0.0011) [2023-12-27 03:31:03,306][105692] Updated weights for policy 0, policy_version 1664147 (0.0007) [2023-12-27 03:31:03,364][105692] Updated weights for policy 0, policy_version 1664157 (0.0007) [2023-12-27 03:31:03,556][105620] Updated weights for policy 1, policy_version 1667530 (0.0009) [2023-12-27 03:31:03,613][105620] Updated weights for policy 1, policy_version 1667540 (0.0008) [2023-12-27 03:31:03,667][105620] Updated weights for policy 1, policy_version 1667550 (0.0009) [2023-12-27 03:31:03,956][105692] Updated weights for policy 0, policy_version 1664167 (0.0006) [2023-12-27 03:31:04,022][105692] Updated weights for policy 0, policy_version 1664177 (0.0008) [2023-12-27 03:31:04,079][105692] Updated weights for policy 0, policy_version 1664187 (0.0011) [2023-12-27 03:31:04,516][105620] Updated weights for policy 1, policy_version 1667560 (0.0008) [2023-12-27 03:31:04,584][105620] Updated weights for policy 1, policy_version 1667570 (0.0007) [2023-12-27 03:31:04,639][105620] Updated weights for policy 1, policy_version 1667580 (0.0005) [2023-12-27 03:31:04,818][105692] Updated weights for policy 0, policy_version 1664197 (0.0009) [2023-12-27 03:31:04,882][105692] Updated weights for policy 0, policy_version 1664207 (0.0011) [2023-12-27 03:31:04,936][105692] Updated weights for policy 0, policy_version 1664217 (0.0011) [2023-12-27 03:31:05,260][105620] Updated weights for policy 1, policy_version 1667590 (0.0007) [2023-12-27 03:31:05,327][105620] Updated weights for policy 1, policy_version 1667600 (0.0007) [2023-12-27 03:31:05,392][105620] Updated weights for policy 1, policy_version 1667610 (0.0007) [2023-12-27 03:31:05,578][105692] Updated weights for policy 0, policy_version 1664227 (0.0009) [2023-12-27 03:31:05,637][105692] Updated weights for policy 0, policy_version 1664237 (0.0010) [2023-12-27 03:31:05,700][105692] Updated weights for policy 0, policy_version 1664247 (0.0011) [2023-12-27 03:31:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 853082112. Throughput: 0: 9735.7, 1: 9771.7. Samples: 853071624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:31:06,062][104569] Avg episode reward: [(0, '8623.907'), (1, '8718.396')] [2023-12-27 03:31:06,087][105620] Updated weights for policy 1, policy_version 1667620 (0.0009) [2023-12-27 03:31:06,158][105620] Updated weights for policy 1, policy_version 1667630 (0.0008) [2023-12-27 03:31:06,227][105620] Updated weights for policy 1, policy_version 1667640 (0.0008) [2023-12-27 03:31:06,439][105692] Updated weights for policy 0, policy_version 1664257 (0.0011) [2023-12-27 03:31:06,500][105692] Updated weights for policy 0, policy_version 1664267 (0.0011) [2023-12-27 03:31:06,568][105692] Updated weights for policy 0, policy_version 1664277 (0.0011) [2023-12-27 03:31:06,632][105692] Updated weights for policy 0, policy_version 1664287 (0.0011) [2023-12-27 03:31:06,959][105620] Updated weights for policy 1, policy_version 1667650 (0.0008) [2023-12-27 03:31:07,018][105620] Updated weights for policy 1, policy_version 1667660 (0.0008) [2023-12-27 03:31:07,069][105620] Updated weights for policy 1, policy_version 1667670 (0.0008) [2023-12-27 03:31:07,121][105620] Updated weights for policy 1, policy_version 1667680 (0.0008) [2023-12-27 03:31:07,392][105692] Updated weights for policy 0, policy_version 1664297 (0.0011) [2023-12-27 03:31:07,455][105692] Updated weights for policy 0, policy_version 1664307 (0.0011) [2023-12-27 03:31:07,522][105692] Updated weights for policy 0, policy_version 1664317 (0.0011) [2023-12-27 03:31:07,855][105620] Updated weights for policy 1, policy_version 1667690 (0.0008) [2023-12-27 03:31:07,922][105620] Updated weights for policy 1, policy_version 1667700 (0.0009) [2023-12-27 03:31:07,980][105620] Updated weights for policy 1, policy_version 1667710 (0.0008) [2023-12-27 03:31:08,261][105692] Updated weights for policy 0, policy_version 1664327 (0.0011) [2023-12-27 03:31:08,313][105692] Updated weights for policy 0, policy_version 1664337 (0.0010) [2023-12-27 03:31:08,377][105692] Updated weights for policy 0, policy_version 1664347 (0.0011) [2023-12-27 03:31:08,732][105620] Updated weights for policy 1, policy_version 1667720 (0.0006) [2023-12-27 03:31:08,797][105620] Updated weights for policy 1, policy_version 1667730 (0.0006) [2023-12-27 03:31:08,849][105620] Updated weights for policy 1, policy_version 1667740 (0.0006) [2023-12-27 03:31:09,116][105692] Updated weights for policy 0, policy_version 1664357 (0.0011) [2023-12-27 03:31:09,168][105692] Updated weights for policy 0, policy_version 1664367 (0.0010) [2023-12-27 03:31:09,229][105692] Updated weights for policy 0, policy_version 1664377 (0.0010) [2023-12-27 03:31:09,511][105620] Updated weights for policy 1, policy_version 1667750 (0.0009) [2023-12-27 03:31:09,562][105620] Updated weights for policy 1, policy_version 1667760 (0.0009) [2023-12-27 03:31:09,623][105620] Updated weights for policy 1, policy_version 1667770 (0.0009) [2023-12-27 03:31:09,947][105692] Updated weights for policy 0, policy_version 1664387 (0.0008) [2023-12-27 03:31:10,007][105692] Updated weights for policy 0, policy_version 1664397 (0.0009) [2023-12-27 03:31:10,068][105692] Updated weights for policy 0, policy_version 1664407 (0.0010) [2023-12-27 03:31:10,486][105620] Updated weights for policy 1, policy_version 1667780 (0.0008) [2023-12-27 03:31:10,539][105620] Updated weights for policy 1, policy_version 1667790 (0.0009) [2023-12-27 03:31:10,599][105620] Updated weights for policy 1, policy_version 1667800 (0.0006) [2023-12-27 03:31:10,798][105692] Updated weights for policy 0, policy_version 1664417 (0.0009) [2023-12-27 03:31:10,852][105692] Updated weights for policy 0, policy_version 1664427 (0.0005) [2023-12-27 03:31:10,901][105692] Updated weights for policy 0, policy_version 1664437 (0.0006) [2023-12-27 03:31:10,960][105692] Updated weights for policy 0, policy_version 1664447 (0.0008) [2023-12-27 03:31:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19633.1). Total num frames: 853180416. Throughput: 0: 9863.5, 1: 9680.4. Samples: 853185404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:31:11,063][104569] Avg episode reward: [(0, '8621.197'), (1, '8896.458')] [2023-12-27 03:31:11,383][105620] Updated weights for policy 1, policy_version 1667810 (0.0008) [2023-12-27 03:31:11,445][105620] Updated weights for policy 1, policy_version 1667820 (0.0010) [2023-12-27 03:31:11,505][105620] Updated weights for policy 1, policy_version 1667830 (0.0011) [2023-12-27 03:31:11,567][105620] Updated weights for policy 1, policy_version 1667840 (0.0011) [2023-12-27 03:31:11,674][105692] Updated weights for policy 0, policy_version 1664457 (0.0007) [2023-12-27 03:31:11,737][105692] Updated weights for policy 0, policy_version 1664467 (0.0007) [2023-12-27 03:31:11,807][105692] Updated weights for policy 0, policy_version 1664477 (0.0008) [2023-12-27 03:31:12,352][105620] Updated weights for policy 1, policy_version 1667850 (0.0011) [2023-12-27 03:31:12,386][105692] Updated weights for policy 0, policy_version 1664487 (0.0007) [2023-12-27 03:31:12,420][105620] Updated weights for policy 1, policy_version 1667860 (0.0011) [2023-12-27 03:31:12,444][105692] Updated weights for policy 0, policy_version 1664497 (0.0006) [2023-12-27 03:31:12,478][105620] Updated weights for policy 1, policy_version 1667870 (0.0010) [2023-12-27 03:31:12,505][105692] Updated weights for policy 0, policy_version 1664507 (0.0007) [2023-12-27 03:31:13,175][105620] Updated weights for policy 1, policy_version 1667880 (0.0006) [2023-12-27 03:31:13,224][105692] Updated weights for policy 0, policy_version 1664517 (0.0007) [2023-12-27 03:31:13,224][105620] Updated weights for policy 1, policy_version 1667890 (0.0009) [2023-12-27 03:31:13,272][105620] Updated weights for policy 1, policy_version 1667900 (0.0010) [2023-12-27 03:31:13,273][105692] Updated weights for policy 0, policy_version 1664527 (0.0005) [2023-12-27 03:31:13,321][105692] Updated weights for policy 0, policy_version 1664537 (0.0005) [2023-12-27 03:31:13,959][105692] Updated weights for policy 0, policy_version 1664547 (0.0007) [2023-12-27 03:31:14,008][105692] Updated weights for policy 0, policy_version 1664557 (0.0007) [2023-12-27 03:31:14,010][105620] Updated weights for policy 1, policy_version 1667910 (0.0010) [2023-12-27 03:31:14,058][105620] Updated weights for policy 1, policy_version 1667920 (0.0010) [2023-12-27 03:31:14,060][105692] Updated weights for policy 0, policy_version 1664567 (0.0005) [2023-12-27 03:31:14,113][105620] Updated weights for policy 1, policy_version 1667930 (0.0010) [2023-12-27 03:31:14,668][105620] Updated weights for policy 1, policy_version 1667940 (0.0006) [2023-12-27 03:31:14,689][105692] Updated weights for policy 0, policy_version 1664577 (0.0006) [2023-12-27 03:31:14,719][105620] Updated weights for policy 1, policy_version 1667950 (0.0005) [2023-12-27 03:31:14,738][105692] Updated weights for policy 0, policy_version 1664587 (0.0010) [2023-12-27 03:31:14,773][105620] Updated weights for policy 1, policy_version 1667960 (0.0006) [2023-12-27 03:31:14,798][105692] Updated weights for policy 0, policy_version 1664597 (0.0007) [2023-12-27 03:31:14,858][105692] Updated weights for policy 0, policy_version 1664607 (0.0009) [2023-12-27 03:31:15,370][105620] Updated weights for policy 1, policy_version 1667970 (0.0008) [2023-12-27 03:31:15,435][105620] Updated weights for policy 1, policy_version 1667980 (0.0007) [2023-12-27 03:31:15,498][105620] Updated weights for policy 1, policy_version 1667990 (0.0009) [2023-12-27 03:31:15,558][105620] Updated weights for policy 1, policy_version 1668000 (0.0009) [2023-12-27 03:31:15,726][105692] Updated weights for policy 0, policy_version 1664617 (0.0009) [2023-12-27 03:31:15,782][105692] Updated weights for policy 0, policy_version 1664627 (0.0008) [2023-12-27 03:31:15,839][105692] Updated weights for policy 0, policy_version 1664637 (0.0008) [2023-12-27 03:31:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 853278720. Throughput: 0: 9868.9, 1: 9676.3. Samples: 853244512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:31:16,063][104569] Avg episode reward: [(0, '8709.320'), (1, '8808.809')] [2023-12-27 03:31:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001664640_426213376.pth... [2023-12-27 03:31:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001668000_427065344.pth... [2023-12-27 03:31:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001666912_426786816.pth [2023-12-27 03:31:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001663488_425918464.pth [2023-12-27 03:31:16,322][105620] Updated weights for policy 1, policy_version 1668010 (0.0005) [2023-12-27 03:31:16,383][105620] Updated weights for policy 1, policy_version 1668020 (0.0005) [2023-12-27 03:31:16,445][105620] Updated weights for policy 1, policy_version 1668030 (0.0006) [2023-12-27 03:31:16,480][105692] Updated weights for policy 0, policy_version 1664647 (0.0007) [2023-12-27 03:31:16,533][105692] Updated weights for policy 0, policy_version 1664657 (0.0010) [2023-12-27 03:31:16,585][105692] Updated weights for policy 0, policy_version 1664667 (0.0010) [2023-12-27 03:31:17,026][105620] Updated weights for policy 1, policy_version 1668040 (0.0005) [2023-12-27 03:31:17,084][105620] Updated weights for policy 1, policy_version 1668050 (0.0006) [2023-12-27 03:31:17,144][105620] Updated weights for policy 1, policy_version 1668060 (0.0005) [2023-12-27 03:31:17,293][105692] Updated weights for policy 0, policy_version 1664677 (0.0007) [2023-12-27 03:31:17,347][105692] Updated weights for policy 0, policy_version 1664687 (0.0009) [2023-12-27 03:31:17,412][105692] Updated weights for policy 0, policy_version 1664697 (0.0008) [2023-12-27 03:31:17,753][105620] Updated weights for policy 1, policy_version 1668070 (0.0008) [2023-12-27 03:31:17,812][105620] Updated weights for policy 1, policy_version 1668080 (0.0011) [2023-12-27 03:31:17,868][105620] Updated weights for policy 1, policy_version 1668090 (0.0010) [2023-12-27 03:31:18,054][105692] Updated weights for policy 0, policy_version 1664707 (0.0008) [2023-12-27 03:31:18,116][105692] Updated weights for policy 0, policy_version 1664717 (0.0005) [2023-12-27 03:31:18,181][105692] Updated weights for policy 0, policy_version 1664727 (0.0005) [2023-12-27 03:31:18,563][105620] Updated weights for policy 1, policy_version 1668100 (0.0010) [2023-12-27 03:31:18,622][105620] Updated weights for policy 1, policy_version 1668110 (0.0011) [2023-12-27 03:31:18,686][105620] Updated weights for policy 1, policy_version 1668120 (0.0008) [2023-12-27 03:31:18,740][105692] Updated weights for policy 0, policy_version 1664737 (0.0006) [2023-12-27 03:31:18,810][105692] Updated weights for policy 0, policy_version 1664747 (0.0008) [2023-12-27 03:31:18,878][105692] Updated weights for policy 0, policy_version 1664757 (0.0008) [2023-12-27 03:31:18,932][105692] Updated weights for policy 0, policy_version 1664767 (0.0006) [2023-12-27 03:31:19,296][105620] Updated weights for policy 1, policy_version 1668130 (0.0006) [2023-12-27 03:31:19,366][105620] Updated weights for policy 1, policy_version 1668140 (0.0010) [2023-12-27 03:31:19,426][105620] Updated weights for policy 1, policy_version 1668150 (0.0006) [2023-12-27 03:31:19,484][105620] Updated weights for policy 1, policy_version 1668160 (0.0006) [2023-12-27 03:31:19,577][105692] Updated weights for policy 0, policy_version 1664777 (0.0008) [2023-12-27 03:31:19,636][105692] Updated weights for policy 0, policy_version 1664787 (0.0008) [2023-12-27 03:31:19,699][105692] Updated weights for policy 0, policy_version 1664797 (0.0009) [2023-12-27 03:31:20,230][105620] Updated weights for policy 1, policy_version 1668170 (0.0006) [2023-12-27 03:31:20,298][105620] Updated weights for policy 1, policy_version 1668180 (0.0007) [2023-12-27 03:31:20,364][105620] Updated weights for policy 1, policy_version 1668190 (0.0007) [2023-12-27 03:31:20,499][105692] Updated weights for policy 0, policy_version 1664807 (0.0010) [2023-12-27 03:31:20,559][105692] Updated weights for policy 0, policy_version 1664817 (0.0011) [2023-12-27 03:31:20,632][105692] Updated weights for policy 0, policy_version 1664827 (0.0011) [2023-12-27 03:31:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 853377024. Throughput: 0: 9939.7, 1: 9743.9. Samples: 853370364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:31:21,063][104569] Avg episode reward: [(0, '8901.332'), (1, '8807.809')] [2023-12-27 03:31:21,067][105620] Updated weights for policy 1, policy_version 1668200 (0.0010) [2023-12-27 03:31:21,125][105620] Updated weights for policy 1, policy_version 1668210 (0.0009) [2023-12-27 03:31:21,193][105620] Updated weights for policy 1, policy_version 1668220 (0.0009) [2023-12-27 03:31:21,366][105692] Updated weights for policy 0, policy_version 1664837 (0.0009) [2023-12-27 03:31:21,435][105692] Updated weights for policy 0, policy_version 1664847 (0.0006) [2023-12-27 03:31:21,493][105692] Updated weights for policy 0, policy_version 1664857 (0.0010) [2023-12-27 03:31:21,973][105620] Updated weights for policy 1, policy_version 1668230 (0.0008) [2023-12-27 03:31:22,044][105620] Updated weights for policy 1, policy_version 1668240 (0.0006) [2023-12-27 03:31:22,112][105620] Updated weights for policy 1, policy_version 1668250 (0.0008) [2023-12-27 03:31:22,214][105692] Updated weights for policy 0, policy_version 1664867 (0.0010) [2023-12-27 03:31:22,283][105692] Updated weights for policy 0, policy_version 1664877 (0.0008) [2023-12-27 03:31:22,352][105692] Updated weights for policy 0, policy_version 1664887 (0.0007) [2023-12-27 03:31:22,830][105620] Updated weights for policy 1, policy_version 1668260 (0.0009) [2023-12-27 03:31:22,894][105620] Updated weights for policy 1, policy_version 1668270 (0.0007) [2023-12-27 03:31:22,968][105620] Updated weights for policy 1, policy_version 1668280 (0.0007) [2023-12-27 03:31:23,128][105692] Updated weights for policy 0, policy_version 1664897 (0.0009) [2023-12-27 03:31:23,190][105692] Updated weights for policy 0, policy_version 1664907 (0.0010) [2023-12-27 03:31:23,252][105692] Updated weights for policy 0, policy_version 1664917 (0.0010) [2023-12-27 03:31:23,311][105692] Updated weights for policy 0, policy_version 1664927 (0.0009) [2023-12-27 03:31:23,641][105620] Updated weights for policy 1, policy_version 1668290 (0.0009) [2023-12-27 03:31:23,688][105620] Updated weights for policy 1, policy_version 1668300 (0.0009) [2023-12-27 03:31:23,740][105620] Updated weights for policy 1, policy_version 1668310 (0.0009) [2023-12-27 03:31:23,789][105620] Updated weights for policy 1, policy_version 1668320 (0.0008) [2023-12-27 03:31:24,063][105692] Updated weights for policy 0, policy_version 1664937 (0.0008) [2023-12-27 03:31:24,121][105692] Updated weights for policy 0, policy_version 1664947 (0.0009) [2023-12-27 03:31:24,179][105692] Updated weights for policy 0, policy_version 1664957 (0.0009) [2023-12-27 03:31:24,560][105620] Updated weights for policy 1, policy_version 1668330 (0.0009) [2023-12-27 03:31:24,611][105620] Updated weights for policy 1, policy_version 1668340 (0.0009) [2023-12-27 03:31:24,657][105620] Updated weights for policy 1, policy_version 1668350 (0.0008) [2023-12-27 03:31:24,945][105692] Updated weights for policy 0, policy_version 1664967 (0.0009) [2023-12-27 03:31:25,012][105692] Updated weights for policy 0, policy_version 1664977 (0.0009) [2023-12-27 03:31:25,075][105692] Updated weights for policy 0, policy_version 1664987 (0.0009) [2023-12-27 03:31:25,455][105620] Updated weights for policy 1, policy_version 1668360 (0.0010) [2023-12-27 03:31:25,518][105620] Updated weights for policy 1, policy_version 1668370 (0.0009) [2023-12-27 03:31:25,577][105620] Updated weights for policy 1, policy_version 1668380 (0.0009) [2023-12-27 03:31:25,818][105692] Updated weights for policy 0, policy_version 1664997 (0.0008) [2023-12-27 03:31:25,874][105692] Updated weights for policy 0, policy_version 1665007 (0.0009) [2023-12-27 03:31:25,925][105692] Updated weights for policy 0, policy_version 1665017 (0.0008) [2023-12-27 03:31:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 853475328. Throughput: 0: 9849.0, 1: 9591.3. Samples: 853481120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:31:26,062][104569] Avg episode reward: [(0, '8715.891'), (1, '8715.310')] [2023-12-27 03:31:26,323][105620] Updated weights for policy 1, policy_version 1668390 (0.0008) [2023-12-27 03:31:26,377][105620] Updated weights for policy 1, policy_version 1668400 (0.0009) [2023-12-27 03:31:26,423][105620] Updated weights for policy 1, policy_version 1668410 (0.0008) [2023-12-27 03:31:26,668][105692] Updated weights for policy 0, policy_version 1665027 (0.0007) [2023-12-27 03:31:26,719][105692] Updated weights for policy 0, policy_version 1665037 (0.0005) [2023-12-27 03:31:26,772][105692] Updated weights for policy 0, policy_version 1665047 (0.0005) [2023-12-27 03:31:27,260][105620] Updated weights for policy 1, policy_version 1668420 (0.0010) [2023-12-27 03:31:27,314][105620] Updated weights for policy 1, policy_version 1668430 (0.0009) [2023-12-27 03:31:27,365][105620] Updated weights for policy 1, policy_version 1668440 (0.0009) [2023-12-27 03:31:27,403][105692] Updated weights for policy 0, policy_version 1665057 (0.0005) [2023-12-27 03:31:27,464][105692] Updated weights for policy 0, policy_version 1665067 (0.0007) [2023-12-27 03:31:27,523][105692] Updated weights for policy 0, policy_version 1665077 (0.0005) [2023-12-27 03:31:27,581][105692] Updated weights for policy 0, policy_version 1665087 (0.0005) [2023-12-27 03:31:28,134][105620] Updated weights for policy 1, policy_version 1668450 (0.0008) [2023-12-27 03:31:28,182][105620] Updated weights for policy 1, policy_version 1668460 (0.0005) [2023-12-27 03:31:28,207][105692] Updated weights for policy 0, policy_version 1665097 (0.0009) [2023-12-27 03:31:28,228][105620] Updated weights for policy 1, policy_version 1668470 (0.0007) [2023-12-27 03:31:28,255][105692] Updated weights for policy 0, policy_version 1665107 (0.0009) [2023-12-27 03:31:28,277][105620] Updated weights for policy 1, policy_version 1668480 (0.0005) [2023-12-27 03:31:28,310][105692] Updated weights for policy 0, policy_version 1665117 (0.0005) [2023-12-27 03:31:28,981][105692] Updated weights for policy 0, policy_version 1665127 (0.0006) [2023-12-27 03:31:28,991][105620] Updated weights for policy 1, policy_version 1668490 (0.0011) [2023-12-27 03:31:29,039][105692] Updated weights for policy 0, policy_version 1665137 (0.0005) [2023-12-27 03:31:29,052][105620] Updated weights for policy 1, policy_version 1668500 (0.0010) [2023-12-27 03:31:29,102][105692] Updated weights for policy 0, policy_version 1665147 (0.0007) [2023-12-27 03:31:29,108][105620] Updated weights for policy 1, policy_version 1668510 (0.0008) [2023-12-27 03:31:29,799][105620] Updated weights for policy 1, policy_version 1668520 (0.0010) [2023-12-27 03:31:29,862][105620] Updated weights for policy 1, policy_version 1668530 (0.0011) [2023-12-27 03:31:29,867][105692] Updated weights for policy 0, policy_version 1665157 (0.0008) [2023-12-27 03:31:29,911][105620] Updated weights for policy 1, policy_version 1668540 (0.0010) [2023-12-27 03:31:29,924][105692] Updated weights for policy 0, policy_version 1665167 (0.0007) [2023-12-27 03:31:29,991][105692] Updated weights for policy 0, policy_version 1665177 (0.0008) [2023-12-27 03:31:30,619][105620] Updated weights for policy 1, policy_version 1668550 (0.0008) [2023-12-27 03:31:30,676][105620] Updated weights for policy 1, policy_version 1668560 (0.0010) [2023-12-27 03:31:30,720][105620] Updated weights for policy 1, policy_version 1668570 (0.0010) [2023-12-27 03:31:30,742][105692] Updated weights for policy 0, policy_version 1665187 (0.0007) [2023-12-27 03:31:30,788][105692] Updated weights for policy 0, policy_version 1665197 (0.0007) [2023-12-27 03:31:30,843][105692] Updated weights for policy 0, policy_version 1665207 (0.0008) [2023-12-27 03:31:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19633.1). Total num frames: 853573632. Throughput: 0: 9904.4, 1: 9570.0. Samples: 853539996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:31:31,062][104569] Avg episode reward: [(0, '8533.961'), (1, '9081.560')] [2023-12-27 03:31:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001668576_427212800.pth... [2023-12-27 03:31:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001665216_426360832.pth... [2023-12-27 03:31:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001664064_426065920.pth [2023-12-27 03:31:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001667456_426926080.pth [2023-12-27 03:31:31,436][105620] Updated weights for policy 1, policy_version 1668580 (0.0008) [2023-12-27 03:31:31,504][105620] Updated weights for policy 1, policy_version 1668590 (0.0007) [2023-12-27 03:31:31,563][105692] Updated weights for policy 0, policy_version 1665217 (0.0005) [2023-12-27 03:31:31,569][105620] Updated weights for policy 1, policy_version 1668600 (0.0009) [2023-12-27 03:31:31,629][105692] Updated weights for policy 0, policy_version 1665227 (0.0008) [2023-12-27 03:31:31,692][105692] Updated weights for policy 0, policy_version 1665237 (0.0009) [2023-12-27 03:31:31,757][105692] Updated weights for policy 0, policy_version 1665247 (0.0009) [2023-12-27 03:31:32,234][105620] Updated weights for policy 1, policy_version 1668610 (0.0008) [2023-12-27 03:31:32,292][105620] Updated weights for policy 1, policy_version 1668620 (0.0009) [2023-12-27 03:31:32,348][105620] Updated weights for policy 1, policy_version 1668630 (0.0007) [2023-12-27 03:31:32,409][105620] Updated weights for policy 1, policy_version 1668640 (0.0009) [2023-12-27 03:31:32,486][105692] Updated weights for policy 0, policy_version 1665257 (0.0010) [2023-12-27 03:31:32,540][105692] Updated weights for policy 0, policy_version 1665267 (0.0009) [2023-12-27 03:31:32,590][105692] Updated weights for policy 0, policy_version 1665277 (0.0008) [2023-12-27 03:31:33,078][105620] Updated weights for policy 1, policy_version 1668650 (0.0005) [2023-12-27 03:31:33,133][105620] Updated weights for policy 1, policy_version 1668660 (0.0006) [2023-12-27 03:31:33,190][105620] Updated weights for policy 1, policy_version 1668670 (0.0009) [2023-12-27 03:31:33,413][105692] Updated weights for policy 0, policy_version 1665287 (0.0009) [2023-12-27 03:31:33,461][105692] Updated weights for policy 0, policy_version 1665297 (0.0009) [2023-12-27 03:31:33,514][105692] Updated weights for policy 0, policy_version 1665307 (0.0009) [2023-12-27 03:31:33,902][105620] Updated weights for policy 1, policy_version 1668680 (0.0009) [2023-12-27 03:31:33,955][105620] Updated weights for policy 1, policy_version 1668691 (0.0010) [2023-12-27 03:31:34,008][105620] Updated weights for policy 1, policy_version 1668702 (0.0009) [2023-12-27 03:31:34,279][105692] Updated weights for policy 0, policy_version 1665317 (0.0009) [2023-12-27 03:31:34,344][105692] Updated weights for policy 0, policy_version 1665327 (0.0008) [2023-12-27 03:31:34,405][105692] Updated weights for policy 0, policy_version 1665337 (0.0008) [2023-12-27 03:31:34,723][105620] Updated weights for policy 1, policy_version 1668712 (0.0007) [2023-12-27 03:31:34,785][105620] Updated weights for policy 1, policy_version 1668722 (0.0009) [2023-12-27 03:31:34,857][105620] Updated weights for policy 1, policy_version 1668732 (0.0010) [2023-12-27 03:31:35,090][105692] Updated weights for policy 0, policy_version 1665347 (0.0008) [2023-12-27 03:31:35,144][105692] Updated weights for policy 0, policy_version 1665357 (0.0005) [2023-12-27 03:31:35,210][105692] Updated weights for policy 0, policy_version 1665367 (0.0006) [2023-12-27 03:31:35,394][105620] Updated weights for policy 1, policy_version 1668742 (0.0008) [2023-12-27 03:31:35,449][105620] Updated weights for policy 1, policy_version 1668752 (0.0010) [2023-12-27 03:31:35,517][105620] Updated weights for policy 1, policy_version 1668762 (0.0010) [2023-12-27 03:31:35,734][105692] Updated weights for policy 0, policy_version 1665377 (0.0006) [2023-12-27 03:31:35,786][105692] Updated weights for policy 0, policy_version 1665387 (0.0011) [2023-12-27 03:31:35,835][105692] Updated weights for policy 0, policy_version 1665397 (0.0010) [2023-12-27 03:31:35,888][105692] Updated weights for policy 0, policy_version 1665407 (0.0009) [2023-12-27 03:31:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 853671936. Throughput: 0: 9741.8, 1: 9612.6. Samples: 853655964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:31:36,062][104569] Avg episode reward: [(0, '8717.577'), (1, '9173.235')] [2023-12-27 03:31:36,262][105620] Updated weights for policy 1, policy_version 1668772 (0.0009) [2023-12-27 03:31:36,323][105620] Updated weights for policy 1, policy_version 1668782 (0.0008) [2023-12-27 03:31:36,386][105620] Updated weights for policy 1, policy_version 1668792 (0.0006) [2023-12-27 03:31:36,649][105692] Updated weights for policy 0, policy_version 1665417 (0.0010) [2023-12-27 03:31:36,712][105692] Updated weights for policy 0, policy_version 1665427 (0.0011) [2023-12-27 03:31:36,774][105692] Updated weights for policy 0, policy_version 1665437 (0.0010) [2023-12-27 03:31:37,119][105620] Updated weights for policy 1, policy_version 1668802 (0.0008) [2023-12-27 03:31:37,181][105620] Updated weights for policy 1, policy_version 1668812 (0.0009) [2023-12-27 03:31:37,238][105620] Updated weights for policy 1, policy_version 1668822 (0.0008) [2023-12-27 03:31:37,287][105620] Updated weights for policy 1, policy_version 1668832 (0.0009) [2023-12-27 03:31:37,499][105692] Updated weights for policy 0, policy_version 1665447 (0.0007) [2023-12-27 03:31:37,554][105692] Updated weights for policy 0, policy_version 1665457 (0.0010) [2023-12-27 03:31:37,609][105692] Updated weights for policy 0, policy_version 1665467 (0.0010) [2023-12-27 03:31:38,070][105620] Updated weights for policy 1, policy_version 1668842 (0.0006) [2023-12-27 03:31:38,114][105620] Updated weights for policy 1, policy_version 1668852 (0.0007) [2023-12-27 03:31:38,178][105620] Updated weights for policy 1, policy_version 1668862 (0.0006) [2023-12-27 03:31:38,324][105692] Updated weights for policy 0, policy_version 1665477 (0.0010) [2023-12-27 03:31:38,387][105692] Updated weights for policy 0, policy_version 1665487 (0.0010) [2023-12-27 03:31:38,444][105692] Updated weights for policy 0, policy_version 1665497 (0.0011) [2023-12-27 03:31:38,812][105620] Updated weights for policy 1, policy_version 1668872 (0.0007) [2023-12-27 03:31:38,872][105620] Updated weights for policy 1, policy_version 1668882 (0.0008) [2023-12-27 03:31:38,936][105620] Updated weights for policy 1, policy_version 1668892 (0.0007) [2023-12-27 03:31:39,186][105692] Updated weights for policy 0, policy_version 1665507 (0.0011) [2023-12-27 03:31:39,244][105692] Updated weights for policy 0, policy_version 1665517 (0.0010) [2023-12-27 03:31:39,295][105692] Updated weights for policy 0, policy_version 1665527 (0.0010) [2023-12-27 03:31:39,676][105620] Updated weights for policy 1, policy_version 1668902 (0.0007) [2023-12-27 03:31:39,736][105620] Updated weights for policy 1, policy_version 1668912 (0.0008) [2023-12-27 03:31:39,788][105620] Updated weights for policy 1, policy_version 1668922 (0.0008) [2023-12-27 03:31:40,108][105692] Updated weights for policy 0, policy_version 1665537 (0.0015) [2023-12-27 03:31:40,165][105692] Updated weights for policy 0, policy_version 1665547 (0.0010) [2023-12-27 03:31:40,215][105692] Updated weights for policy 0, policy_version 1665557 (0.0011) [2023-12-27 03:31:40,267][105692] Updated weights for policy 0, policy_version 1665567 (0.0011) [2023-12-27 03:31:40,596][105620] Updated weights for policy 1, policy_version 1668932 (0.0009) [2023-12-27 03:31:40,657][105620] Updated weights for policy 1, policy_version 1668942 (0.0009) [2023-12-27 03:31:40,709][105620] Updated weights for policy 1, policy_version 1668952 (0.0009) [2023-12-27 03:31:40,909][105692] Updated weights for policy 0, policy_version 1665577 (0.0006) [2023-12-27 03:31:40,972][105692] Updated weights for policy 0, policy_version 1665587 (0.0010) [2023-12-27 03:31:41,025][105692] Updated weights for policy 0, policy_version 1665597 (0.0011) [2023-12-27 03:31:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 853770240. Throughput: 0: 9729.5, 1: 9642.0. Samples: 853772620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:31:41,062][104569] Avg episode reward: [(0, '8898.893'), (1, '8990.077')] [2023-12-27 03:31:41,548][105620] Updated weights for policy 1, policy_version 1668962 (0.0009) [2023-12-27 03:31:41,597][105620] Updated weights for policy 1, policy_version 1668972 (0.0008) [2023-12-27 03:31:41,660][105620] Updated weights for policy 1, policy_version 1668982 (0.0008) [2023-12-27 03:31:41,724][105620] Updated weights for policy 1, policy_version 1668992 (0.0008) [2023-12-27 03:31:41,769][105692] Updated weights for policy 0, policy_version 1665607 (0.0009) [2023-12-27 03:31:41,825][105692] Updated weights for policy 0, policy_version 1665617 (0.0009) [2023-12-27 03:31:41,884][105692] Updated weights for policy 0, policy_version 1665627 (0.0010) [2023-12-27 03:31:42,556][105620] Updated weights for policy 1, policy_version 1669002 (0.0009) [2023-12-27 03:31:42,563][105692] Updated weights for policy 0, policy_version 1665637 (0.0008) [2023-12-27 03:31:42,615][105620] Updated weights for policy 1, policy_version 1669012 (0.0007) [2023-12-27 03:31:42,617][105692] Updated weights for policy 0, policy_version 1665647 (0.0008) [2023-12-27 03:31:42,672][105620] Updated weights for policy 1, policy_version 1669022 (0.0006) [2023-12-27 03:31:42,678][105692] Updated weights for policy 0, policy_version 1665657 (0.0007) [2023-12-27 03:31:43,368][105620] Updated weights for policy 1, policy_version 1669032 (0.0005) [2023-12-27 03:31:43,422][105620] Updated weights for policy 1, policy_version 1669042 (0.0005) [2023-12-27 03:31:43,473][105620] Updated weights for policy 1, policy_version 1669052 (0.0008) [2023-12-27 03:31:43,486][105692] Updated weights for policy 0, policy_version 1665667 (0.0009) [2023-12-27 03:31:43,530][105692] Updated weights for policy 0, policy_version 1665677 (0.0005) [2023-12-27 03:31:43,576][105692] Updated weights for policy 0, policy_version 1665687 (0.0005) [2023-12-27 03:31:44,196][105692] Updated weights for policy 0, policy_version 1665697 (0.0006) [2023-12-27 03:31:44,239][105620] Updated weights for policy 1, policy_version 1669062 (0.0009) [2023-12-27 03:31:44,256][105692] Updated weights for policy 0, policy_version 1665707 (0.0008) [2023-12-27 03:31:44,299][105620] Updated weights for policy 1, policy_version 1669072 (0.0007) [2023-12-27 03:31:44,305][105692] Updated weights for policy 0, policy_version 1665717 (0.0008) [2023-12-27 03:31:44,351][105692] Updated weights for policy 0, policy_version 1665727 (0.0008) [2023-12-27 03:31:44,357][105620] Updated weights for policy 1, policy_version 1669082 (0.0008) [2023-12-27 03:31:45,034][105692] Updated weights for policy 0, policy_version 1665737 (0.0008) [2023-12-27 03:31:45,094][105692] Updated weights for policy 0, policy_version 1665747 (0.0010) [2023-12-27 03:31:45,127][105620] Updated weights for policy 1, policy_version 1669092 (0.0008) [2023-12-27 03:31:45,154][105692] Updated weights for policy 0, policy_version 1665757 (0.0008) [2023-12-27 03:31:45,190][105620] Updated weights for policy 1, policy_version 1669102 (0.0007) [2023-12-27 03:31:45,252][105620] Updated weights for policy 1, policy_version 1669112 (0.0009) [2023-12-27 03:31:45,831][105692] Updated weights for policy 0, policy_version 1665767 (0.0006) [2023-12-27 03:31:45,879][105692] Updated weights for policy 0, policy_version 1665777 (0.0005) [2023-12-27 03:31:45,927][105692] Updated weights for policy 0, policy_version 1665787 (0.0005) [2023-12-27 03:31:46,030][105620] Updated weights for policy 1, policy_version 1669122 (0.0006) [2023-12-27 03:31:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 853860352. Throughput: 0: 9729.4, 1: 9597.4. Samples: 853829584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:31:46,062][104569] Avg episode reward: [(0, '8717.281'), (1, '8805.505')] [2023-12-27 03:31:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001665792_426508288.pth... [2023-12-27 03:31:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001664640_426213376.pth [2023-12-27 03:31:46,089][105620] Updated weights for policy 1, policy_version 1669132 (0.0008) [2023-12-27 03:31:46,141][105620] Updated weights for policy 1, policy_version 1669142 (0.0009) [2023-12-27 03:31:46,192][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001669152_427360256.pth... [2023-12-27 03:31:46,195][105620] Updated weights for policy 1, policy_version 1669152 (0.0010) [2023-12-27 03:31:46,197][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001668000_427065344.pth [2023-12-27 03:31:46,505][105692] Updated weights for policy 0, policy_version 1665797 (0.0007) [2023-12-27 03:31:46,564][105692] Updated weights for policy 0, policy_version 1665807 (0.0009) [2023-12-27 03:31:46,627][105692] Updated weights for policy 0, policy_version 1665817 (0.0009) [2023-12-27 03:31:47,005][105620] Updated weights for policy 1, policy_version 1669162 (0.0006) [2023-12-27 03:31:47,066][105620] Updated weights for policy 1, policy_version 1669172 (0.0008) [2023-12-27 03:31:47,128][105620] Updated weights for policy 1, policy_version 1669182 (0.0010) [2023-12-27 03:31:47,394][105692] Updated weights for policy 0, policy_version 1665827 (0.0009) [2023-12-27 03:31:47,461][105692] Updated weights for policy 0, policy_version 1665837 (0.0009) [2023-12-27 03:31:47,523][105692] Updated weights for policy 0, policy_version 1665847 (0.0009) [2023-12-27 03:31:47,695][105620] Updated weights for policy 1, policy_version 1669192 (0.0006) [2023-12-27 03:31:47,756][105620] Updated weights for policy 1, policy_version 1669202 (0.0005) [2023-12-27 03:31:47,810][105620] Updated weights for policy 1, policy_version 1669212 (0.0007) [2023-12-27 03:31:48,169][105692] Updated weights for policy 0, policy_version 1665857 (0.0006) [2023-12-27 03:31:48,219][105692] Updated weights for policy 0, policy_version 1665867 (0.0006) [2023-12-27 03:31:48,273][105692] Updated weights for policy 0, policy_version 1665877 (0.0008) [2023-12-27 03:31:48,338][105692] Updated weights for policy 0, policy_version 1665887 (0.0008) [2023-12-27 03:31:48,527][105620] Updated weights for policy 1, policy_version 1669222 (0.0010) [2023-12-27 03:31:48,579][105620] Updated weights for policy 1, policy_version 1669232 (0.0010) [2023-12-27 03:31:48,630][105620] Updated weights for policy 1, policy_version 1669242 (0.0007) [2023-12-27 03:31:49,079][105692] Updated weights for policy 0, policy_version 1665897 (0.0008) [2023-12-27 03:31:49,134][105692] Updated weights for policy 0, policy_version 1665907 (0.0008) [2023-12-27 03:31:49,196][105692] Updated weights for policy 0, policy_version 1665917 (0.0008) [2023-12-27 03:31:49,353][105620] Updated weights for policy 1, policy_version 1669252 (0.0010) [2023-12-27 03:31:49,411][105620] Updated weights for policy 1, policy_version 1669262 (0.0010) [2023-12-27 03:31:49,471][105620] Updated weights for policy 1, policy_version 1669272 (0.0009) [2023-12-27 03:31:50,070][105692] Updated weights for policy 0, policy_version 1665927 (0.0009) [2023-12-27 03:31:50,092][105620] Updated weights for policy 1, policy_version 1669282 (0.0007) [2023-12-27 03:31:50,126][105692] Updated weights for policy 0, policy_version 1665937 (0.0008) [2023-12-27 03:31:50,151][105620] Updated weights for policy 1, policy_version 1669292 (0.0011) [2023-12-27 03:31:50,185][105692] Updated weights for policy 0, policy_version 1665947 (0.0008) [2023-12-27 03:31:50,211][105620] Updated weights for policy 1, policy_version 1669302 (0.0011) [2023-12-27 03:31:50,273][105620] Updated weights for policy 1, policy_version 1669312 (0.0010) [2023-12-27 03:31:50,937][105692] Updated weights for policy 0, policy_version 1665957 (0.0008) [2023-12-27 03:31:50,989][105692] Updated weights for policy 0, policy_version 1665967 (0.0007) [2023-12-27 03:31:51,011][105620] Updated weights for policy 1, policy_version 1669322 (0.0011) [2023-12-27 03:31:51,056][105692] Updated weights for policy 0, policy_version 1665977 (0.0007) [2023-12-27 03:31:51,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 853950464. Throughput: 0: 9796.5, 1: 9704.8. Samples: 853949184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:31:51,062][104569] Avg episode reward: [(0, '8536.338'), (1, '8802.216')] [2023-12-27 03:31:51,082][105620] Updated weights for policy 1, policy_version 1669332 (0.0010) [2023-12-27 03:31:51,151][105620] Updated weights for policy 1, policy_version 1669342 (0.0011) [2023-12-27 03:31:51,817][105692] Updated weights for policy 0, policy_version 1665987 (0.0009) [2023-12-27 03:31:51,878][105692] Updated weights for policy 0, policy_version 1665997 (0.0008) [2023-12-27 03:31:51,907][105620] Updated weights for policy 1, policy_version 1669352 (0.0010) [2023-12-27 03:31:51,942][105692] Updated weights for policy 0, policy_version 1666007 (0.0009) [2023-12-27 03:31:51,975][105620] Updated weights for policy 1, policy_version 1669362 (0.0008) [2023-12-27 03:31:52,037][105620] Updated weights for policy 1, policy_version 1669372 (0.0006) [2023-12-27 03:31:52,700][105692] Updated weights for policy 0, policy_version 1666017 (0.0007) [2023-12-27 03:31:52,728][105620] Updated weights for policy 1, policy_version 1669382 (0.0008) [2023-12-27 03:31:52,754][105692] Updated weights for policy 0, policy_version 1666027 (0.0005) [2023-12-27 03:31:52,781][105620] Updated weights for policy 1, policy_version 1669392 (0.0009) [2023-12-27 03:31:52,812][105692] Updated weights for policy 0, policy_version 1666037 (0.0005) [2023-12-27 03:31:52,834][105620] Updated weights for policy 1, policy_version 1669402 (0.0009) [2023-12-27 03:31:52,876][105692] Updated weights for policy 0, policy_version 1666047 (0.0005) [2023-12-27 03:31:53,560][105692] Updated weights for policy 0, policy_version 1666057 (0.0006) [2023-12-27 03:31:53,588][105620] Updated weights for policy 1, policy_version 1669412 (0.0008) [2023-12-27 03:31:53,616][105692] Updated weights for policy 0, policy_version 1666067 (0.0005) [2023-12-27 03:31:53,637][105620] Updated weights for policy 1, policy_version 1669422 (0.0009) [2023-12-27 03:31:53,669][105692] Updated weights for policy 0, policy_version 1666077 (0.0005) [2023-12-27 03:31:53,687][105620] Updated weights for policy 1, policy_version 1669432 (0.0008) [2023-12-27 03:31:54,182][105692] Updated weights for policy 0, policy_version 1666087 (0.0005) [2023-12-27 03:31:54,241][105692] Updated weights for policy 0, policy_version 1666097 (0.0006) [2023-12-27 03:31:54,284][105692] Updated weights for policy 0, policy_version 1666107 (0.0007) [2023-12-27 03:31:54,610][105620] Updated weights for policy 1, policy_version 1669443 (0.0009) [2023-12-27 03:31:54,662][105620] Updated weights for policy 1, policy_version 1669453 (0.0008) [2023-12-27 03:31:54,709][105620] Updated weights for policy 1, policy_version 1669463 (0.0009) [2023-12-27 03:31:54,870][105692] Updated weights for policy 0, policy_version 1666117 (0.0005) [2023-12-27 03:31:54,928][105692] Updated weights for policy 0, policy_version 1666127 (0.0008) [2023-12-27 03:31:54,995][105692] Updated weights for policy 0, policy_version 1666137 (0.0011) [2023-12-27 03:31:55,406][105620] Updated weights for policy 1, policy_version 1669473 (0.0010) [2023-12-27 03:31:55,458][105620] Updated weights for policy 1, policy_version 1669483 (0.0011) [2023-12-27 03:31:55,514][105620] Updated weights for policy 1, policy_version 1669493 (0.0011) [2023-12-27 03:31:55,574][105620] Updated weights for policy 1, policy_version 1669503 (0.0011) [2023-12-27 03:31:55,734][105692] Updated weights for policy 0, policy_version 1666147 (0.0010) [2023-12-27 03:31:55,799][105692] Updated weights for policy 0, policy_version 1666157 (0.0010) [2023-12-27 03:31:55,870][105692] Updated weights for policy 0, policy_version 1666167 (0.0010) [2023-12-27 03:31:56,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19387.6, 300 sec: 19633.0). Total num frames: 854056960. Throughput: 0: 9859.6, 1: 9687.0. Samples: 854065008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:31:56,063][104569] Avg episode reward: [(0, '8621.726'), (1, '9077.499')] [2023-12-27 03:31:56,194][105620] Updated weights for policy 1, policy_version 1669513 (0.0008) [2023-12-27 03:31:56,250][105620] Updated weights for policy 1, policy_version 1669523 (0.0011) [2023-12-27 03:31:56,295][105620] Updated weights for policy 1, policy_version 1669533 (0.0010) [2023-12-27 03:31:56,566][105692] Updated weights for policy 0, policy_version 1666177 (0.0010) [2023-12-27 03:31:56,631][105692] Updated weights for policy 0, policy_version 1666187 (0.0010) [2023-12-27 03:31:56,689][105692] Updated weights for policy 0, policy_version 1666197 (0.0010) [2023-12-27 03:31:56,746][105692] Updated weights for policy 0, policy_version 1666207 (0.0010) [2023-12-27 03:31:57,045][105620] Updated weights for policy 1, policy_version 1669543 (0.0011) [2023-12-27 03:31:57,111][105620] Updated weights for policy 1, policy_version 1669553 (0.0010) [2023-12-27 03:31:57,159][105620] Updated weights for policy 1, policy_version 1669563 (0.0010) [2023-12-27 03:31:57,421][105692] Updated weights for policy 0, policy_version 1666217 (0.0006) [2023-12-27 03:31:57,481][105692] Updated weights for policy 0, policy_version 1666227 (0.0005) [2023-12-27 03:31:57,551][105692] Updated weights for policy 0, policy_version 1666237 (0.0005) [2023-12-27 03:31:57,889][105620] Updated weights for policy 1, policy_version 1669573 (0.0010) [2023-12-27 03:31:57,936][105620] Updated weights for policy 1, policy_version 1669583 (0.0010) [2023-12-27 03:31:57,989][105620] Updated weights for policy 1, policy_version 1669593 (0.0005) [2023-12-27 03:31:58,059][105692] Updated weights for policy 0, policy_version 1666247 (0.0009) [2023-12-27 03:31:58,107][105692] Updated weights for policy 0, policy_version 1666257 (0.0010) [2023-12-27 03:31:58,156][105692] Updated weights for policy 0, policy_version 1666267 (0.0010) [2023-12-27 03:31:58,762][105620] Updated weights for policy 1, policy_version 1669603 (0.0006) [2023-12-27 03:31:58,831][105620] Updated weights for policy 1, policy_version 1669613 (0.0009) [2023-12-27 03:31:58,899][105620] Updated weights for policy 1, policy_version 1669623 (0.0009) [2023-12-27 03:31:58,995][105692] Updated weights for policy 0, policy_version 1666277 (0.0010) [2023-12-27 03:31:59,056][105692] Updated weights for policy 0, policy_version 1666287 (0.0009) [2023-12-27 03:31:59,114][105692] Updated weights for policy 0, policy_version 1666297 (0.0009) [2023-12-27 03:31:59,638][105620] Updated weights for policy 1, policy_version 1669633 (0.0009) [2023-12-27 03:31:59,705][105620] Updated weights for policy 1, policy_version 1669643 (0.0007) [2023-12-27 03:31:59,766][105620] Updated weights for policy 1, policy_version 1669653 (0.0008) [2023-12-27 03:31:59,833][105620] Updated weights for policy 1, policy_version 1669663 (0.0008) [2023-12-27 03:31:59,985][105692] Updated weights for policy 0, policy_version 1666307 (0.0009) [2023-12-27 03:32:00,042][105692] Updated weights for policy 0, policy_version 1666317 (0.0009) [2023-12-27 03:32:00,096][105692] Updated weights for policy 0, policy_version 1666328 (0.0008) [2023-12-27 03:32:00,444][105620] Updated weights for policy 1, policy_version 1669673 (0.0005) [2023-12-27 03:32:00,510][105620] Updated weights for policy 1, policy_version 1669683 (0.0007) [2023-12-27 03:32:00,568][105620] Updated weights for policy 1, policy_version 1669693 (0.0006) [2023-12-27 03:32:00,877][105692] Updated weights for policy 0, policy_version 1666338 (0.0008) [2023-12-27 03:32:00,931][105692] Updated weights for policy 0, policy_version 1666349 (0.0009) [2023-12-27 03:32:00,987][105692] Updated weights for policy 0, policy_version 1666359 (0.0005) [2023-12-27 03:32:01,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 854155264. Throughput: 0: 9860.4, 1: 9685.4. Samples: 854124072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:32:01,062][104569] Avg episode reward: [(0, '8257.646'), (1, '9170.101')] [2023-12-27 03:32:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001666368_426655744.pth... [2023-12-27 03:32:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001669696_427499520.pth... [2023-12-27 03:32:01,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001665216_426360832.pth [2023-12-27 03:32:01,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001668576_427212800.pth [2023-12-27 03:32:01,138][105620] Updated weights for policy 1, policy_version 1669703 (0.0009) [2023-12-27 03:32:01,198][105620] Updated weights for policy 1, policy_version 1669713 (0.0008) [2023-12-27 03:32:01,254][105620] Updated weights for policy 1, policy_version 1669723 (0.0008) [2023-12-27 03:32:01,770][105692] Updated weights for policy 0, policy_version 1666369 (0.0007) [2023-12-27 03:32:01,830][105692] Updated weights for policy 0, policy_version 1666379 (0.0009) [2023-12-27 03:32:01,887][105692] Updated weights for policy 0, policy_version 1666389 (0.0009) [2023-12-27 03:32:01,939][105692] Updated weights for policy 0, policy_version 1666399 (0.0008) [2023-12-27 03:32:01,948][105620] Updated weights for policy 1, policy_version 1669733 (0.0009) [2023-12-27 03:32:01,997][105620] Updated weights for policy 1, policy_version 1669743 (0.0007) [2023-12-27 03:32:02,045][105620] Updated weights for policy 1, policy_version 1669753 (0.0006) [2023-12-27 03:32:02,661][105620] Updated weights for policy 1, policy_version 1669763 (0.0005) [2023-12-27 03:32:02,728][105620] Updated weights for policy 1, policy_version 1669773 (0.0007) [2023-12-27 03:32:02,774][105692] Updated weights for policy 0, policy_version 1666409 (0.0007) [2023-12-27 03:32:02,789][105620] Updated weights for policy 1, policy_version 1669783 (0.0005) [2023-12-27 03:32:02,826][105692] Updated weights for policy 0, policy_version 1666419 (0.0008) [2023-12-27 03:32:02,878][105692] Updated weights for policy 0, policy_version 1666430 (0.0010) [2023-12-27 03:32:03,432][105620] Updated weights for policy 1, policy_version 1669793 (0.0006) [2023-12-27 03:32:03,490][105620] Updated weights for policy 1, policy_version 1669803 (0.0010) [2023-12-27 03:32:03,506][105692] Updated weights for policy 0, policy_version 1666440 (0.0006) [2023-12-27 03:32:03,546][105620] Updated weights for policy 1, policy_version 1669813 (0.0009) [2023-12-27 03:32:03,562][105692] Updated weights for policy 0, policy_version 1666450 (0.0005) [2023-12-27 03:32:03,601][105620] Updated weights for policy 1, policy_version 1669823 (0.0008) [2023-12-27 03:32:03,615][105692] Updated weights for policy 0, policy_version 1666460 (0.0005) [2023-12-27 03:32:04,198][105692] Updated weights for policy 0, policy_version 1666470 (0.0007) [2023-12-27 03:32:04,257][105692] Updated weights for policy 0, policy_version 1666480 (0.0009) [2023-12-27 03:32:04,316][105692] Updated weights for policy 0, policy_version 1666490 (0.0009) [2023-12-27 03:32:04,421][105620] Updated weights for policy 1, policy_version 1669833 (0.0008) [2023-12-27 03:32:04,473][105620] Updated weights for policy 1, policy_version 1669843 (0.0009) [2023-12-27 03:32:04,521][105620] Updated weights for policy 1, policy_version 1669853 (0.0008) [2023-12-27 03:32:05,015][105692] Updated weights for policy 0, policy_version 1666500 (0.0008) [2023-12-27 03:32:05,080][105692] Updated weights for policy 0, policy_version 1666510 (0.0007) [2023-12-27 03:32:05,143][105692] Updated weights for policy 0, policy_version 1666520 (0.0008) [2023-12-27 03:32:05,326][105620] Updated weights for policy 1, policy_version 1669863 (0.0007) [2023-12-27 03:32:05,379][105620] Updated weights for policy 1, policy_version 1669873 (0.0005) [2023-12-27 03:32:05,437][105620] Updated weights for policy 1, policy_version 1669883 (0.0008) [2023-12-27 03:32:05,896][105692] Updated weights for policy 0, policy_version 1666530 (0.0009) [2023-12-27 03:32:05,959][105692] Updated weights for policy 0, policy_version 1666540 (0.0010) [2023-12-27 03:32:06,021][105692] Updated weights for policy 0, policy_version 1666550 (0.0010) [2023-12-27 03:32:06,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 854245376. Throughput: 0: 9725.9, 1: 9638.1. Samples: 854241740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:32:06,063][104569] Avg episode reward: [(0, '8534.258'), (1, '9172.278')] [2023-12-27 03:32:06,071][105692] Updated weights for policy 0, policy_version 1666560 (0.0010) [2023-12-27 03:32:06,089][105620] Updated weights for policy 1, policy_version 1669893 (0.0008) [2023-12-27 03:32:06,159][105620] Updated weights for policy 1, policy_version 1669903 (0.0009) [2023-12-27 03:32:06,219][105620] Updated weights for policy 1, policy_version 1669913 (0.0009) [2023-12-27 03:32:06,921][105692] Updated weights for policy 0, policy_version 1666570 (0.0010) [2023-12-27 03:32:06,923][105620] Updated weights for policy 1, policy_version 1669923 (0.0008) [2023-12-27 03:32:06,986][105620] Updated weights for policy 1, policy_version 1669933 (0.0006) [2023-12-27 03:32:06,988][105692] Updated weights for policy 0, policy_version 1666580 (0.0007) [2023-12-27 03:32:07,047][105692] Updated weights for policy 0, policy_version 1666590 (0.0009) [2023-12-27 03:32:07,049][105620] Updated weights for policy 1, policy_version 1669943 (0.0006) [2023-12-27 03:32:07,661][105620] Updated weights for policy 1, policy_version 1669953 (0.0009) [2023-12-27 03:32:07,720][105620] Updated weights for policy 1, policy_version 1669963 (0.0010) [2023-12-27 03:32:07,774][105620] Updated weights for policy 1, policy_version 1669973 (0.0010) [2023-12-27 03:32:07,836][105620] Updated weights for policy 1, policy_version 1669983 (0.0010) [2023-12-27 03:32:07,850][105692] Updated weights for policy 0, policy_version 1666600 (0.0007) [2023-12-27 03:32:07,905][105692] Updated weights for policy 0, policy_version 1666610 (0.0008) [2023-12-27 03:32:07,959][105692] Updated weights for policy 0, policy_version 1666620 (0.0008) [2023-12-27 03:32:08,521][105620] Updated weights for policy 1, policy_version 1669993 (0.0006) [2023-12-27 03:32:08,579][105620] Updated weights for policy 1, policy_version 1670003 (0.0008) [2023-12-27 03:32:08,642][105620] Updated weights for policy 1, policy_version 1670013 (0.0011) [2023-12-27 03:32:08,708][105692] Updated weights for policy 0, policy_version 1666630 (0.0007) [2023-12-27 03:32:08,769][105692] Updated weights for policy 0, policy_version 1666640 (0.0005) [2023-12-27 03:32:08,828][105692] Updated weights for policy 0, policy_version 1666650 (0.0006) [2023-12-27 03:32:09,226][105620] Updated weights for policy 1, policy_version 1670023 (0.0009) [2023-12-27 03:32:09,286][105620] Updated weights for policy 1, policy_version 1670033 (0.0009) [2023-12-27 03:32:09,339][105620] Updated weights for policy 1, policy_version 1670043 (0.0008) [2023-12-27 03:32:09,656][105692] Updated weights for policy 0, policy_version 1666660 (0.0010) [2023-12-27 03:32:09,713][105692] Updated weights for policy 0, policy_version 1666670 (0.0008) [2023-12-27 03:32:09,772][105692] Updated weights for policy 0, policy_version 1666680 (0.0009) [2023-12-27 03:32:10,015][105620] Updated weights for policy 1, policy_version 1670053 (0.0009) [2023-12-27 03:32:10,077][105620] Updated weights for policy 1, policy_version 1670063 (0.0009) [2023-12-27 03:32:10,138][105620] Updated weights for policy 1, policy_version 1670073 (0.0009) [2023-12-27 03:32:10,576][105692] Updated weights for policy 0, policy_version 1666690 (0.0009) [2023-12-27 03:32:10,629][105692] Updated weights for policy 0, policy_version 1666700 (0.0009) [2023-12-27 03:32:10,681][105692] Updated weights for policy 0, policy_version 1666710 (0.0010) [2023-12-27 03:32:10,731][105692] Updated weights for policy 0, policy_version 1666720 (0.0008) [2023-12-27 03:32:10,881][105620] Updated weights for policy 1, policy_version 1670083 (0.0009) [2023-12-27 03:32:10,930][105620] Updated weights for policy 1, policy_version 1670093 (0.0008) [2023-12-27 03:32:10,975][105620] Updated weights for policy 1, policy_version 1670103 (0.0008) [2023-12-27 03:32:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 854351872. Throughput: 0: 9709.0, 1: 9753.2. Samples: 854356924. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:32:11,063][104569] Avg episode reward: [(0, '8715.024'), (1, '9172.191')] [2023-12-27 03:32:11,475][105692] Updated weights for policy 0, policy_version 1666730 (0.0007) [2023-12-27 03:32:11,538][105692] Updated weights for policy 0, policy_version 1666740 (0.0006) [2023-12-27 03:32:11,604][105692] Updated weights for policy 0, policy_version 1666750 (0.0006) [2023-12-27 03:32:11,814][105620] Updated weights for policy 1, policy_version 1670113 (0.0008) [2023-12-27 03:32:11,872][105620] Updated weights for policy 1, policy_version 1670123 (0.0008) [2023-12-27 03:32:11,939][105620] Updated weights for policy 1, policy_version 1670133 (0.0008) [2023-12-27 03:32:11,997][105620] Updated weights for policy 1, policy_version 1670143 (0.0009) [2023-12-27 03:32:12,302][105692] Updated weights for policy 0, policy_version 1666760 (0.0011) [2023-12-27 03:32:12,369][105692] Updated weights for policy 0, policy_version 1666770 (0.0011) [2023-12-27 03:32:12,435][105692] Updated weights for policy 0, policy_version 1666780 (0.0011) [2023-12-27 03:32:12,714][105620] Updated weights for policy 1, policy_version 1670153 (0.0008) [2023-12-27 03:32:12,766][105620] Updated weights for policy 1, policy_version 1670163 (0.0010) [2023-12-27 03:32:12,835][105620] Updated weights for policy 1, policy_version 1670173 (0.0006) [2023-12-27 03:32:13,152][105692] Updated weights for policy 0, policy_version 1666790 (0.0008) [2023-12-27 03:32:13,213][105692] Updated weights for policy 0, policy_version 1666800 (0.0006) [2023-12-27 03:32:13,273][105692] Updated weights for policy 0, policy_version 1666810 (0.0005) [2023-12-27 03:32:13,523][105620] Updated weights for policy 1, policy_version 1670183 (0.0008) [2023-12-27 03:32:13,580][105620] Updated weights for policy 1, policy_version 1670193 (0.0010) [2023-12-27 03:32:13,629][105620] Updated weights for policy 1, policy_version 1670203 (0.0010) [2023-12-27 03:32:13,870][105692] Updated weights for policy 0, policy_version 1666820 (0.0007) [2023-12-27 03:32:13,929][105692] Updated weights for policy 0, policy_version 1666830 (0.0010) [2023-12-27 03:32:13,987][105692] Updated weights for policy 0, policy_version 1666840 (0.0010) [2023-12-27 03:32:14,335][105620] Updated weights for policy 1, policy_version 1670213 (0.0010) [2023-12-27 03:32:14,383][105620] Updated weights for policy 1, policy_version 1670223 (0.0010) [2023-12-27 03:32:14,433][105620] Updated weights for policy 1, policy_version 1670233 (0.0010) [2023-12-27 03:32:14,682][105692] Updated weights for policy 0, policy_version 1666850 (0.0011) [2023-12-27 03:32:14,736][105692] Updated weights for policy 0, policy_version 1666860 (0.0010) [2023-12-27 03:32:14,797][105692] Updated weights for policy 0, policy_version 1666870 (0.0011) [2023-12-27 03:32:14,860][105692] Updated weights for policy 0, policy_version 1666880 (0.0010) [2023-12-27 03:32:15,136][105620] Updated weights for policy 1, policy_version 1670243 (0.0009) [2023-12-27 03:32:15,187][105620] Updated weights for policy 1, policy_version 1670253 (0.0006) [2023-12-27 03:32:15,246][105620] Updated weights for policy 1, policy_version 1670263 (0.0005) [2023-12-27 03:32:15,635][105692] Updated weights for policy 0, policy_version 1666890 (0.0010) [2023-12-27 03:32:15,695][105692] Updated weights for policy 0, policy_version 1666900 (0.0011) [2023-12-27 03:32:15,754][105692] Updated weights for policy 0, policy_version 1666910 (0.0011) [2023-12-27 03:32:15,793][105620] Updated weights for policy 1, policy_version 1670273 (0.0005) [2023-12-27 03:32:15,849][105620] Updated weights for policy 1, policy_version 1670283 (0.0005) [2023-12-27 03:32:15,915][105620] Updated weights for policy 1, policy_version 1670293 (0.0005) [2023-12-27 03:32:15,978][105620] Updated weights for policy 1, policy_version 1670303 (0.0005) [2023-12-27 03:32:16,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 854450176. Throughput: 0: 9689.9, 1: 9757.1. Samples: 854415112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:32:16,062][104569] Avg episode reward: [(0, '8625.341'), (1, '9263.063')] [2023-12-27 03:32:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001670304_427655168.pth... [2023-12-27 03:32:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001666912_426795008.pth... [2023-12-27 03:32:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001669152_427360256.pth [2023-12-27 03:32:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001665792_426508288.pth [2023-12-27 03:32:16,481][105620] Updated weights for policy 1, policy_version 1670313 (0.0006) [2023-12-27 03:32:16,483][105692] Updated weights for policy 0, policy_version 1666920 (0.0007) [2023-12-27 03:32:16,534][105620] Updated weights for policy 1, policy_version 1670323 (0.0008) [2023-12-27 03:32:16,538][105692] Updated weights for policy 0, policy_version 1666930 (0.0006) [2023-12-27 03:32:16,587][105620] Updated weights for policy 1, policy_version 1670334 (0.0007) [2023-12-27 03:32:16,595][105692] Updated weights for policy 0, policy_version 1666940 (0.0008) [2023-12-27 03:32:17,153][105620] Updated weights for policy 1, policy_version 1670344 (0.0006) [2023-12-27 03:32:17,162][105692] Updated weights for policy 0, policy_version 1666950 (0.0008) [2023-12-27 03:32:17,213][105620] Updated weights for policy 1, policy_version 1670354 (0.0010) [2023-12-27 03:32:17,223][105692] Updated weights for policy 0, policy_version 1666960 (0.0007) [2023-12-27 03:32:17,276][105620] Updated weights for policy 1, policy_version 1670364 (0.0010) [2023-12-27 03:32:17,281][105692] Updated weights for policy 0, policy_version 1666970 (0.0005) [2023-12-27 03:32:17,895][105620] Updated weights for policy 1, policy_version 1670374 (0.0010) [2023-12-27 03:32:17,942][105620] Updated weights for policy 1, policy_version 1670384 (0.0010) [2023-12-27 03:32:17,989][105620] Updated weights for policy 1, policy_version 1670394 (0.0010) [2023-12-27 03:32:18,004][105692] Updated weights for policy 0, policy_version 1666980 (0.0007) [2023-12-27 03:32:18,063][105692] Updated weights for policy 0, policy_version 1666990 (0.0009) [2023-12-27 03:32:18,117][105692] Updated weights for policy 0, policy_version 1667000 (0.0008) [2023-12-27 03:32:18,739][105620] Updated weights for policy 1, policy_version 1670404 (0.0009) [2023-12-27 03:32:18,797][105620] Updated weights for policy 1, policy_version 1670414 (0.0010) [2023-12-27 03:32:18,833][105692] Updated weights for policy 0, policy_version 1667010 (0.0008) [2023-12-27 03:32:18,856][105620] Updated weights for policy 1, policy_version 1670424 (0.0010) [2023-12-27 03:32:18,893][105692] Updated weights for policy 0, policy_version 1667020 (0.0010) [2023-12-27 03:32:18,955][105692] Updated weights for policy 0, policy_version 1667030 (0.0011) [2023-12-27 03:32:19,010][105692] Updated weights for policy 0, policy_version 1667040 (0.0010) [2023-12-27 03:32:19,593][105620] Updated weights for policy 1, policy_version 1670434 (0.0009) [2023-12-27 03:32:19,647][105620] Updated weights for policy 1, policy_version 1670444 (0.0006) [2023-12-27 03:32:19,705][105620] Updated weights for policy 1, policy_version 1670454 (0.0006) [2023-12-27 03:32:19,721][105692] Updated weights for policy 0, policy_version 1667050 (0.0008) [2023-12-27 03:32:19,765][105620] Updated weights for policy 1, policy_version 1670464 (0.0006) [2023-12-27 03:32:19,793][105692] Updated weights for policy 0, policy_version 1667060 (0.0009) [2023-12-27 03:32:19,859][105692] Updated weights for policy 0, policy_version 1667070 (0.0009) [2023-12-27 03:32:20,505][105620] Updated weights for policy 1, policy_version 1670474 (0.0009) [2023-12-27 03:32:20,555][105620] Updated weights for policy 1, policy_version 1670484 (0.0008) [2023-12-27 03:32:20,618][105692] Updated weights for policy 0, policy_version 1667080 (0.0008) [2023-12-27 03:32:20,619][105620] Updated weights for policy 1, policy_version 1670494 (0.0008) [2023-12-27 03:32:20,675][105692] Updated weights for policy 0, policy_version 1667090 (0.0009) [2023-12-27 03:32:20,734][105692] Updated weights for policy 0, policy_version 1667100 (0.0009) [2023-12-27 03:32:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 854548480. Throughput: 0: 9761.3, 1: 9850.0. Samples: 854538476. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:32:21,062][104569] Avg episode reward: [(0, '8898.889'), (1, '9263.353')] [2023-12-27 03:32:21,403][105620] Updated weights for policy 1, policy_version 1670504 (0.0009) [2023-12-27 03:32:21,457][105620] Updated weights for policy 1, policy_version 1670514 (0.0009) [2023-12-27 03:32:21,515][105620] Updated weights for policy 1, policy_version 1670524 (0.0009) [2023-12-27 03:32:21,588][105692] Updated weights for policy 0, policy_version 1667110 (0.0008) [2023-12-27 03:32:21,653][105692] Updated weights for policy 0, policy_version 1667120 (0.0008) [2023-12-27 03:32:21,718][105692] Updated weights for policy 0, policy_version 1667130 (0.0008) [2023-12-27 03:32:22,362][105620] Updated weights for policy 1, policy_version 1670534 (0.0009) [2023-12-27 03:32:22,419][105692] Updated weights for policy 0, policy_version 1667140 (0.0007) [2023-12-27 03:32:22,430][105620] Updated weights for policy 1, policy_version 1670544 (0.0009) [2023-12-27 03:32:22,487][105692] Updated weights for policy 0, policy_version 1667150 (0.0006) [2023-12-27 03:32:22,497][105620] Updated weights for policy 1, policy_version 1670554 (0.0009) [2023-12-27 03:32:22,549][105692] Updated weights for policy 0, policy_version 1667160 (0.0007) [2023-12-27 03:32:23,234][105620] Updated weights for policy 1, policy_version 1670564 (0.0007) [2023-12-27 03:32:23,274][105692] Updated weights for policy 0, policy_version 1667170 (0.0010) [2023-12-27 03:32:23,299][105620] Updated weights for policy 1, policy_version 1670574 (0.0009) [2023-12-27 03:32:23,337][105692] Updated weights for policy 0, policy_version 1667180 (0.0010) [2023-12-27 03:32:23,352][105620] Updated weights for policy 1, policy_version 1670584 (0.0006) [2023-12-27 03:32:23,389][105692] Updated weights for policy 0, policy_version 1667190 (0.0010) [2023-12-27 03:32:23,441][105692] Updated weights for policy 0, policy_version 1667200 (0.0010) [2023-12-27 03:32:23,966][105620] Updated weights for policy 1, policy_version 1670594 (0.0005) [2023-12-27 03:32:24,038][105620] Updated weights for policy 1, policy_version 1670604 (0.0005) [2023-12-27 03:32:24,092][105620] Updated weights for policy 1, policy_version 1670614 (0.0008) [2023-12-27 03:32:24,139][105692] Updated weights for policy 0, policy_version 1667210 (0.0008) [2023-12-27 03:32:24,151][105620] Updated weights for policy 1, policy_version 1670624 (0.0010) [2023-12-27 03:32:24,185][105692] Updated weights for policy 0, policy_version 1667220 (0.0008) [2023-12-27 03:32:24,239][105692] Updated weights for policy 0, policy_version 1667230 (0.0008) [2023-12-27 03:32:24,765][105620] Updated weights for policy 1, policy_version 1670634 (0.0007) [2023-12-27 03:32:24,818][105620] Updated weights for policy 1, policy_version 1670644 (0.0005) [2023-12-27 03:32:24,885][105620] Updated weights for policy 1, policy_version 1670654 (0.0005) [2023-12-27 03:32:24,885][105692] Updated weights for policy 0, policy_version 1667240 (0.0006) [2023-12-27 03:32:24,945][105692] Updated weights for policy 0, policy_version 1667250 (0.0005) [2023-12-27 03:32:24,992][105692] Updated weights for policy 0, policy_version 1667260 (0.0005) [2023-12-27 03:32:25,580][105620] Updated weights for policy 1, policy_version 1670664 (0.0009) [2023-12-27 03:32:25,632][105620] Updated weights for policy 1, policy_version 1670674 (0.0010) [2023-12-27 03:32:25,659][105692] Updated weights for policy 0, policy_version 1667270 (0.0006) [2023-12-27 03:32:25,681][105620] Updated weights for policy 1, policy_version 1670684 (0.0010) [2023-12-27 03:32:25,720][105692] Updated weights for policy 0, policy_version 1667280 (0.0007) [2023-12-27 03:32:25,772][105692] Updated weights for policy 0, policy_version 1667290 (0.0008) [2023-12-27 03:32:26,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 854646784. Throughput: 0: 9725.8, 1: 9851.3. Samples: 854653596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:32:26,063][104569] Avg episode reward: [(0, '9078.234'), (1, '9083.074')] [2023-12-27 03:32:26,436][105620] Updated weights for policy 1, policy_version 1670694 (0.0010) [2023-12-27 03:32:26,487][105620] Updated weights for policy 1, policy_version 1670704 (0.0010) [2023-12-27 03:32:26,546][105620] Updated weights for policy 1, policy_version 1670714 (0.0010) [2023-12-27 03:32:26,546][105692] Updated weights for policy 0, policy_version 1667300 (0.0009) [2023-12-27 03:32:26,609][105692] Updated weights for policy 0, policy_version 1667310 (0.0009) [2023-12-27 03:32:26,654][105692] Updated weights for policy 0, policy_version 1667320 (0.0008) [2023-12-27 03:32:27,295][105620] Updated weights for policy 1, policy_version 1670724 (0.0010) [2023-12-27 03:32:27,352][105620] Updated weights for policy 1, policy_version 1670734 (0.0010) [2023-12-27 03:32:27,409][105620] Updated weights for policy 1, policy_version 1670744 (0.0010) [2023-12-27 03:32:27,423][105692] Updated weights for policy 0, policy_version 1667330 (0.0007) [2023-12-27 03:32:27,475][105692] Updated weights for policy 0, policy_version 1667340 (0.0008) [2023-12-27 03:32:27,538][105692] Updated weights for policy 0, policy_version 1667350 (0.0008) [2023-12-27 03:32:27,593][105692] Updated weights for policy 0, policy_version 1667360 (0.0008) [2023-12-27 03:32:28,079][105620] Updated weights for policy 1, policy_version 1670754 (0.0009) [2023-12-27 03:32:28,134][105620] Updated weights for policy 1, policy_version 1670764 (0.0006) [2023-12-27 03:32:28,185][105620] Updated weights for policy 1, policy_version 1670774 (0.0005) [2023-12-27 03:32:28,238][105620] Updated weights for policy 1, policy_version 1670784 (0.0006) [2023-12-27 03:32:28,297][105692] Updated weights for policy 0, policy_version 1667370 (0.0006) [2023-12-27 03:32:28,360][105692] Updated weights for policy 0, policy_version 1667380 (0.0006) [2023-12-27 03:32:28,420][105692] Updated weights for policy 0, policy_version 1667390 (0.0008) [2023-12-27 03:32:28,940][105620] Updated weights for policy 1, policy_version 1670794 (0.0009) [2023-12-27 03:32:28,996][105620] Updated weights for policy 1, policy_version 1670804 (0.0007) [2023-12-27 03:32:29,046][105620] Updated weights for policy 1, policy_version 1670814 (0.0005) [2023-12-27 03:32:29,129][105692] Updated weights for policy 0, policy_version 1667400 (0.0009) [2023-12-27 03:32:29,196][105692] Updated weights for policy 0, policy_version 1667410 (0.0009) [2023-12-27 03:32:29,265][105692] Updated weights for policy 0, policy_version 1667420 (0.0010) [2023-12-27 03:32:29,769][105620] Updated weights for policy 1, policy_version 1670824 (0.0006) [2023-12-27 03:32:29,829][105620] Updated weights for policy 1, policy_version 1670834 (0.0006) [2023-12-27 03:32:29,881][105620] Updated weights for policy 1, policy_version 1670844 (0.0010) [2023-12-27 03:32:29,997][105692] Updated weights for policy 0, policy_version 1667430 (0.0010) [2023-12-27 03:32:30,056][105692] Updated weights for policy 0, policy_version 1667440 (0.0010) [2023-12-27 03:32:30,115][105692] Updated weights for policy 0, policy_version 1667450 (0.0009) [2023-12-27 03:32:30,603][105620] Updated weights for policy 1, policy_version 1670854 (0.0007) [2023-12-27 03:32:30,668][105620] Updated weights for policy 1, policy_version 1670864 (0.0008) [2023-12-27 03:32:30,731][105620] Updated weights for policy 1, policy_version 1670874 (0.0005) [2023-12-27 03:32:30,805][105692] Updated weights for policy 0, policy_version 1667460 (0.0009) [2023-12-27 03:32:30,850][105692] Updated weights for policy 0, policy_version 1667470 (0.0005) [2023-12-27 03:32:30,895][105692] Updated weights for policy 0, policy_version 1667480 (0.0008) [2023-12-27 03:32:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 854745088. Throughput: 0: 9698.3, 1: 9894.6. Samples: 854711264. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:32:31,063][104569] Avg episode reward: [(0, '8801.854'), (1, '8809.336')] [2023-12-27 03:32:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001670880_427802624.pth... [2023-12-27 03:32:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001667488_426942464.pth... [2023-12-27 03:32:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001669696_427499520.pth [2023-12-27 03:32:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001666368_426655744.pth [2023-12-27 03:32:31,408][105620] Updated weights for policy 1, policy_version 1670884 (0.0007) [2023-12-27 03:32:31,468][105620] Updated weights for policy 1, policy_version 1670894 (0.0009) [2023-12-27 03:32:31,529][105620] Updated weights for policy 1, policy_version 1670904 (0.0010) [2023-12-27 03:32:31,550][105692] Updated weights for policy 0, policy_version 1667490 (0.0008) [2023-12-27 03:32:31,613][105692] Updated weights for policy 0, policy_version 1667500 (0.0006) [2023-12-27 03:32:31,673][105692] Updated weights for policy 0, policy_version 1667510 (0.0007) [2023-12-27 03:32:31,735][105692] Updated weights for policy 0, policy_version 1667520 (0.0008) [2023-12-27 03:32:32,328][105620] Updated weights for policy 1, policy_version 1670914 (0.0009) [2023-12-27 03:32:32,393][105620] Updated weights for policy 1, policy_version 1670924 (0.0009) [2023-12-27 03:32:32,414][105692] Updated weights for policy 0, policy_version 1667530 (0.0006) [2023-12-27 03:32:32,459][105620] Updated weights for policy 1, policy_version 1670934 (0.0008) [2023-12-27 03:32:32,474][105692] Updated weights for policy 0, policy_version 1667540 (0.0006) [2023-12-27 03:32:32,524][105620] Updated weights for policy 1, policy_version 1670944 (0.0006) [2023-12-27 03:32:32,539][105692] Updated weights for policy 0, policy_version 1667550 (0.0009) [2023-12-27 03:32:33,099][105620] Updated weights for policy 1, policy_version 1670954 (0.0007) [2023-12-27 03:32:33,162][105620] Updated weights for policy 1, policy_version 1670964 (0.0007) [2023-12-27 03:32:33,225][105620] Updated weights for policy 1, policy_version 1670974 (0.0008) [2023-12-27 03:32:33,327][105692] Updated weights for policy 0, policy_version 1667560 (0.0008) [2023-12-27 03:32:33,390][105692] Updated weights for policy 0, policy_version 1667570 (0.0009) [2023-12-27 03:32:33,450][105692] Updated weights for policy 0, policy_version 1667580 (0.0009) [2023-12-27 03:32:33,899][105620] Updated weights for policy 1, policy_version 1670984 (0.0008) [2023-12-27 03:32:33,956][105620] Updated weights for policy 1, policy_version 1670994 (0.0009) [2023-12-27 03:32:34,007][105620] Updated weights for policy 1, policy_version 1671004 (0.0009) [2023-12-27 03:32:34,185][105692] Updated weights for policy 0, policy_version 1667590 (0.0009) [2023-12-27 03:32:34,235][105692] Updated weights for policy 0, policy_version 1667600 (0.0009) [2023-12-27 03:32:34,293][105692] Updated weights for policy 0, policy_version 1667610 (0.0009) [2023-12-27 03:32:34,795][105620] Updated weights for policy 1, policy_version 1671015 (0.0010) [2023-12-27 03:32:34,841][105620] Updated weights for policy 1, policy_version 1671025 (0.0008) [2023-12-27 03:32:34,892][105620] Updated weights for policy 1, policy_version 1671035 (0.0005) [2023-12-27 03:32:35,030][105692] Updated weights for policy 0, policy_version 1667620 (0.0009) [2023-12-27 03:32:35,077][105692] Updated weights for policy 0, policy_version 1667630 (0.0009) [2023-12-27 03:32:35,127][105692] Updated weights for policy 0, policy_version 1667640 (0.0008) [2023-12-27 03:32:35,512][105620] Updated weights for policy 1, policy_version 1671045 (0.0005) [2023-12-27 03:32:35,566][105620] Updated weights for policy 1, policy_version 1671055 (0.0008) [2023-12-27 03:32:35,624][105620] Updated weights for policy 1, policy_version 1671065 (0.0009) [2023-12-27 03:32:36,015][105692] Updated weights for policy 0, policy_version 1667650 (0.0008) [2023-12-27 03:32:36,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 854835200. Throughput: 0: 9628.0, 1: 9886.5. Samples: 854827340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:32:36,063][104569] Avg episode reward: [(0, '8532.184'), (1, '8807.312')] [2023-12-27 03:32:36,073][105692] Updated weights for policy 0, policy_version 1667660 (0.0009) [2023-12-27 03:32:36,139][105692] Updated weights for policy 0, policy_version 1667670 (0.0008) [2023-12-27 03:32:36,198][105692] Updated weights for policy 0, policy_version 1667680 (0.0007) [2023-12-27 03:32:36,244][105620] Updated weights for policy 1, policy_version 1671075 (0.0009) [2023-12-27 03:32:36,307][105620] Updated weights for policy 1, policy_version 1671085 (0.0009) [2023-12-27 03:32:36,369][105620] Updated weights for policy 1, policy_version 1671095 (0.0009) [2023-12-27 03:32:36,891][105692] Updated weights for policy 0, policy_version 1667690 (0.0010) [2023-12-27 03:32:36,949][105692] Updated weights for policy 0, policy_version 1667700 (0.0009) [2023-12-27 03:32:37,009][105692] Updated weights for policy 0, policy_version 1667710 (0.0009) [2023-12-27 03:32:37,072][105620] Updated weights for policy 1, policy_version 1671105 (0.0009) [2023-12-27 03:32:37,135][105620] Updated weights for policy 1, policy_version 1671115 (0.0010) [2023-12-27 03:32:37,205][105620] Updated weights for policy 1, policy_version 1671125 (0.0010) [2023-12-27 03:32:37,273][105620] Updated weights for policy 1, policy_version 1671135 (0.0010) [2023-12-27 03:32:37,610][105692] Updated weights for policy 0, policy_version 1667720 (0.0007) [2023-12-27 03:32:37,664][105692] Updated weights for policy 0, policy_version 1667730 (0.0006) [2023-12-27 03:32:37,724][105692] Updated weights for policy 0, policy_version 1667740 (0.0006) [2023-12-27 03:32:38,071][105620] Updated weights for policy 1, policy_version 1671145 (0.0006) [2023-12-27 03:32:38,124][105620] Updated weights for policy 1, policy_version 1671155 (0.0005) [2023-12-27 03:32:38,179][105620] Updated weights for policy 1, policy_version 1671165 (0.0005) [2023-12-27 03:32:38,368][105692] Updated weights for policy 0, policy_version 1667750 (0.0008) [2023-12-27 03:32:38,419][105692] Updated weights for policy 0, policy_version 1667760 (0.0008) [2023-12-27 03:32:38,480][105692] Updated weights for policy 0, policy_version 1667770 (0.0009) [2023-12-27 03:32:38,936][105620] Updated weights for policy 1, policy_version 1671175 (0.0008) [2023-12-27 03:32:38,992][105620] Updated weights for policy 1, policy_version 1671185 (0.0009) [2023-12-27 03:32:39,054][105620] Updated weights for policy 1, policy_version 1671195 (0.0009) [2023-12-27 03:32:39,110][105692] Updated weights for policy 0, policy_version 1667780 (0.0009) [2023-12-27 03:32:39,155][105692] Updated weights for policy 0, policy_version 1667790 (0.0008) [2023-12-27 03:32:39,208][105692] Updated weights for policy 0, policy_version 1667800 (0.0007) [2023-12-27 03:32:39,841][105692] Updated weights for policy 0, policy_version 1667810 (0.0008) [2023-12-27 03:32:39,896][105692] Updated weights for policy 0, policy_version 1667820 (0.0008) [2023-12-27 03:32:39,936][105620] Updated weights for policy 1, policy_version 1671205 (0.0009) [2023-12-27 03:32:39,959][105692] Updated weights for policy 0, policy_version 1667830 (0.0007) [2023-12-27 03:32:39,994][105620] Updated weights for policy 1, policy_version 1671215 (0.0008) [2023-12-27 03:32:40,018][105692] Updated weights for policy 0, policy_version 1667840 (0.0007) [2023-12-27 03:32:40,053][105620] Updated weights for policy 1, policy_version 1671225 (0.0010) [2023-12-27 03:32:40,723][105692] Updated weights for policy 0, policy_version 1667850 (0.0009) [2023-12-27 03:32:40,790][105692] Updated weights for policy 0, policy_version 1667860 (0.0006) [2023-12-27 03:32:40,850][105692] Updated weights for policy 0, policy_version 1667870 (0.0005) [2023-12-27 03:32:40,913][105620] Updated weights for policy 1, policy_version 1671235 (0.0009) [2023-12-27 03:32:40,976][105620] Updated weights for policy 1, policy_version 1671245 (0.0009) [2023-12-27 03:32:41,040][105620] Updated weights for policy 1, policy_version 1671255 (0.0009) [2023-12-27 03:32:41,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 854933504. Throughput: 0: 9650.7, 1: 9902.1. Samples: 854944880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:32:41,062][104569] Avg episode reward: [(0, '8628.098'), (1, '8897.170')] [2023-12-27 03:32:41,616][105692] Updated weights for policy 0, policy_version 1667880 (0.0008) [2023-12-27 03:32:41,679][105692] Updated weights for policy 0, policy_version 1667890 (0.0009) [2023-12-27 03:32:41,734][105692] Updated weights for policy 0, policy_version 1667900 (0.0009) [2023-12-27 03:32:41,751][105620] Updated weights for policy 1, policy_version 1671265 (0.0007) [2023-12-27 03:32:41,824][105620] Updated weights for policy 1, policy_version 1671275 (0.0007) [2023-12-27 03:32:41,898][105620] Updated weights for policy 1, policy_version 1671285 (0.0006) [2023-12-27 03:32:41,963][105620] Updated weights for policy 1, policy_version 1671295 (0.0006) [2023-12-27 03:32:42,513][105620] Updated weights for policy 1, policy_version 1671305 (0.0010) [2023-12-27 03:32:42,554][105692] Updated weights for policy 0, policy_version 1667910 (0.0007) [2023-12-27 03:32:42,576][105620] Updated weights for policy 1, policy_version 1671315 (0.0011) [2023-12-27 03:32:42,611][105692] Updated weights for policy 0, policy_version 1667920 (0.0006) [2023-12-27 03:32:42,632][105620] Updated weights for policy 1, policy_version 1671325 (0.0008) [2023-12-27 03:32:42,674][105692] Updated weights for policy 0, policy_version 1667930 (0.0009) [2023-12-27 03:32:43,241][105620] Updated weights for policy 1, policy_version 1671335 (0.0005) [2023-12-27 03:32:43,290][105620] Updated weights for policy 1, policy_version 1671345 (0.0005) [2023-12-27 03:32:43,297][105692] Updated weights for policy 0, policy_version 1667940 (0.0008) [2023-12-27 03:32:43,348][105620] Updated weights for policy 1, policy_version 1671355 (0.0005) [2023-12-27 03:32:43,359][105692] Updated weights for policy 0, policy_version 1667950 (0.0006) [2023-12-27 03:32:43,416][105692] Updated weights for policy 0, policy_version 1667960 (0.0005) [2023-12-27 03:32:43,874][105620] Updated weights for policy 1, policy_version 1671365 (0.0005) [2023-12-27 03:32:43,933][105620] Updated weights for policy 1, policy_version 1671375 (0.0005) [2023-12-27 03:32:43,982][105620] Updated weights for policy 1, policy_version 1671385 (0.0005) [2023-12-27 03:32:44,070][105692] Updated weights for policy 0, policy_version 1667970 (0.0006) [2023-12-27 03:32:44,124][105692] Updated weights for policy 0, policy_version 1667980 (0.0010) [2023-12-27 03:32:44,182][105692] Updated weights for policy 0, policy_version 1667990 (0.0010) [2023-12-27 03:32:44,244][105692] Updated weights for policy 0, policy_version 1668000 (0.0011) [2023-12-27 03:32:44,618][105620] Updated weights for policy 1, policy_version 1671395 (0.0007) [2023-12-27 03:32:44,683][105620] Updated weights for policy 1, policy_version 1671405 (0.0010) [2023-12-27 03:32:44,738][105620] Updated weights for policy 1, policy_version 1671415 (0.0008) [2023-12-27 03:32:44,965][105692] Updated weights for policy 0, policy_version 1668010 (0.0011) [2023-12-27 03:32:45,032][105692] Updated weights for policy 0, policy_version 1668020 (0.0011) [2023-12-27 03:32:45,091][105692] Updated weights for policy 0, policy_version 1668030 (0.0010) [2023-12-27 03:32:45,505][105620] Updated weights for policy 1, policy_version 1671425 (0.0008) [2023-12-27 03:32:45,553][105620] Updated weights for policy 1, policy_version 1671435 (0.0008) [2023-12-27 03:32:45,610][105620] Updated weights for policy 1, policy_version 1671445 (0.0009) [2023-12-27 03:32:45,672][105620] Updated weights for policy 1, policy_version 1671455 (0.0010) [2023-12-27 03:32:45,783][105692] Updated weights for policy 0, policy_version 1668040 (0.0010) [2023-12-27 03:32:45,834][105692] Updated weights for policy 0, policy_version 1668050 (0.0010) [2023-12-27 03:32:45,892][105692] Updated weights for policy 0, policy_version 1668060 (0.0010) [2023-12-27 03:32:46,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 855040000. Throughput: 0: 9616.5, 1: 10015.3. Samples: 855007504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:32:46,063][104569] Avg episode reward: [(0, '8349.644'), (1, '9081.495')] [2023-12-27 03:32:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001668064_427089920.pth... [2023-12-27 03:32:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001671456_427950080.pth... [2023-12-27 03:32:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001670304_427655168.pth [2023-12-27 03:32:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001666912_426795008.pth [2023-12-27 03:32:46,400][105620] Updated weights for policy 1, policy_version 1671465 (0.0008) [2023-12-27 03:32:46,445][105620] Updated weights for policy 1, policy_version 1671475 (0.0008) [2023-12-27 03:32:46,488][105620] Updated weights for policy 1, policy_version 1671485 (0.0007) [2023-12-27 03:32:46,528][105692] Updated weights for policy 0, policy_version 1668070 (0.0010) [2023-12-27 03:32:46,576][105692] Updated weights for policy 0, policy_version 1668080 (0.0010) [2023-12-27 03:32:46,638][105692] Updated weights for policy 0, policy_version 1668090 (0.0010) [2023-12-27 03:32:47,264][105620] Updated weights for policy 1, policy_version 1671495 (0.0008) [2023-12-27 03:32:47,319][105620] Updated weights for policy 1, policy_version 1671505 (0.0009) [2023-12-27 03:32:47,333][105692] Updated weights for policy 0, policy_version 1668100 (0.0010) [2023-12-27 03:32:47,368][105620] Updated weights for policy 1, policy_version 1671515 (0.0006) [2023-12-27 03:32:47,386][105692] Updated weights for policy 0, policy_version 1668110 (0.0007) [2023-12-27 03:32:47,434][105692] Updated weights for policy 0, policy_version 1668120 (0.0006) [2023-12-27 03:32:48,027][105620] Updated weights for policy 1, policy_version 1671525 (0.0007) [2023-12-27 03:32:48,095][105620] Updated weights for policy 1, policy_version 1671535 (0.0005) [2023-12-27 03:32:48,151][105692] Updated weights for policy 0, policy_version 1668130 (0.0009) [2023-12-27 03:32:48,156][105620] Updated weights for policy 1, policy_version 1671545 (0.0009) [2023-12-27 03:32:48,208][105692] Updated weights for policy 0, policy_version 1668140 (0.0005) [2023-12-27 03:32:48,273][105692] Updated weights for policy 0, policy_version 1668150 (0.0007) [2023-12-27 03:32:48,347][105692] Updated weights for policy 0, policy_version 1668160 (0.0007) [2023-12-27 03:32:48,853][105620] Updated weights for policy 1, policy_version 1671555 (0.0010) [2023-12-27 03:32:48,919][105620] Updated weights for policy 1, policy_version 1671565 (0.0010) [2023-12-27 03:32:48,979][105620] Updated weights for policy 1, policy_version 1671575 (0.0010) [2023-12-27 03:32:49,008][105692] Updated weights for policy 0, policy_version 1668170 (0.0010) [2023-12-27 03:32:49,072][105692] Updated weights for policy 0, policy_version 1668180 (0.0009) [2023-12-27 03:32:49,137][105692] Updated weights for policy 0, policy_version 1668190 (0.0011) [2023-12-27 03:32:49,707][105620] Updated weights for policy 1, policy_version 1671585 (0.0010) [2023-12-27 03:32:49,751][105620] Updated weights for policy 1, policy_version 1671595 (0.0009) [2023-12-27 03:32:49,799][105620] Updated weights for policy 1, policy_version 1671605 (0.0009) [2023-12-27 03:32:49,856][105620] Updated weights for policy 1, policy_version 1671615 (0.0009) [2023-12-27 03:32:49,880][105692] Updated weights for policy 0, policy_version 1668200 (0.0013) [2023-12-27 03:32:49,927][105692] Updated weights for policy 0, policy_version 1668210 (0.0006) [2023-12-27 03:32:49,991][105692] Updated weights for policy 0, policy_version 1668220 (0.0009) [2023-12-27 03:32:50,619][105620] Updated weights for policy 1, policy_version 1671625 (0.0007) [2023-12-27 03:32:50,679][105620] Updated weights for policy 1, policy_version 1671635 (0.0005) [2023-12-27 03:32:50,731][105620] Updated weights for policy 1, policy_version 1671645 (0.0005) [2023-12-27 03:32:50,764][105692] Updated weights for policy 0, policy_version 1668230 (0.0009) [2023-12-27 03:32:50,836][105692] Updated weights for policy 0, policy_version 1668240 (0.0008) [2023-12-27 03:32:50,899][105692] Updated weights for policy 0, policy_version 1668250 (0.0007) [2023-12-27 03:32:51,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 855138304. Throughput: 0: 9692.5, 1: 9939.1. Samples: 855125164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:32:51,063][104569] Avg episode reward: [(0, '8251.720'), (1, '8991.578')] [2023-12-27 03:32:51,462][105620] Updated weights for policy 1, policy_version 1671655 (0.0007) [2023-12-27 03:32:51,523][105620] Updated weights for policy 1, policy_version 1671665 (0.0005) [2023-12-27 03:32:51,560][105692] Updated weights for policy 0, policy_version 1668260 (0.0009) [2023-12-27 03:32:51,585][105620] Updated weights for policy 1, policy_version 1671675 (0.0010) [2023-12-27 03:32:51,622][105692] Updated weights for policy 0, policy_version 1668270 (0.0007) [2023-12-27 03:32:51,691][105692] Updated weights for policy 0, policy_version 1668280 (0.0009) [2023-12-27 03:32:52,243][105620] Updated weights for policy 1, policy_version 1671685 (0.0009) [2023-12-27 03:32:52,313][105620] Updated weights for policy 1, policy_version 1671695 (0.0009) [2023-12-27 03:32:52,381][105620] Updated weights for policy 1, policy_version 1671705 (0.0010) [2023-12-27 03:32:52,489][105692] Updated weights for policy 0, policy_version 1668290 (0.0009) [2023-12-27 03:32:52,541][105692] Updated weights for policy 0, policy_version 1668300 (0.0009) [2023-12-27 03:32:52,603][105692] Updated weights for policy 0, policy_version 1668311 (0.0009) [2023-12-27 03:32:53,110][105620] Updated weights for policy 1, policy_version 1671715 (0.0009) [2023-12-27 03:32:53,172][105620] Updated weights for policy 1, policy_version 1671725 (0.0008) [2023-12-27 03:32:53,223][105620] Updated weights for policy 1, policy_version 1671735 (0.0007) [2023-12-27 03:32:53,360][105692] Updated weights for policy 0, policy_version 1668321 (0.0009) [2023-12-27 03:32:53,407][105692] Updated weights for policy 0, policy_version 1668331 (0.0008) [2023-12-27 03:32:53,464][105692] Updated weights for policy 0, policy_version 1668341 (0.0010) [2023-12-27 03:32:53,527][105692] Updated weights for policy 0, policy_version 1668351 (0.0009) [2023-12-27 03:32:53,892][105620] Updated weights for policy 1, policy_version 1671745 (0.0005) [2023-12-27 03:32:53,947][105620] Updated weights for policy 1, policy_version 1671755 (0.0009) [2023-12-27 03:32:53,999][105620] Updated weights for policy 1, policy_version 1671766 (0.0010) [2023-12-27 03:32:54,045][105620] Updated weights for policy 1, policy_version 1671776 (0.0010) [2023-12-27 03:32:54,296][105692] Updated weights for policy 0, policy_version 1668361 (0.0009) [2023-12-27 03:32:54,359][105692] Updated weights for policy 0, policy_version 1668371 (0.0006) [2023-12-27 03:32:54,416][105692] Updated weights for policy 0, policy_version 1668381 (0.0007) [2023-12-27 03:32:54,734][105620] Updated weights for policy 1, policy_version 1671786 (0.0011) [2023-12-27 03:32:54,793][105620] Updated weights for policy 1, policy_version 1671796 (0.0008) [2023-12-27 03:32:54,844][105620] Updated weights for policy 1, policy_version 1671806 (0.0010) [2023-12-27 03:32:55,179][105692] Updated weights for policy 0, policy_version 1668391 (0.0008) [2023-12-27 03:32:55,243][105692] Updated weights for policy 0, policy_version 1668401 (0.0009) [2023-12-27 03:32:55,305][105692] Updated weights for policy 0, policy_version 1668411 (0.0009) [2023-12-27 03:32:55,499][105620] Updated weights for policy 1, policy_version 1671816 (0.0006) [2023-12-27 03:32:55,556][105620] Updated weights for policy 1, policy_version 1671826 (0.0007) [2023-12-27 03:32:55,610][105620] Updated weights for policy 1, policy_version 1671836 (0.0008) [2023-12-27 03:32:56,055][105692] Updated weights for policy 0, policy_version 1668421 (0.0009) [2023-12-27 03:32:56,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.4, 300 sec: 19605.3). Total num frames: 855228416. Throughput: 0: 9730.0, 1: 9907.8. Samples: 855240624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:32:56,062][104569] Avg episode reward: [(0, '8437.778'), (1, '8531.682')] [2023-12-27 03:32:56,110][105692] Updated weights for policy 0, policy_version 1668431 (0.0009) [2023-12-27 03:32:56,169][105692] Updated weights for policy 0, policy_version 1668441 (0.0009) [2023-12-27 03:32:56,376][105620] Updated weights for policy 1, policy_version 1671846 (0.0007) [2023-12-27 03:32:56,428][105620] Updated weights for policy 1, policy_version 1671856 (0.0005) [2023-12-27 03:32:56,493][105620] Updated weights for policy 1, policy_version 1671866 (0.0005) [2023-12-27 03:32:56,909][105692] Updated weights for policy 0, policy_version 1668451 (0.0009) [2023-12-27 03:32:56,963][105692] Updated weights for policy 0, policy_version 1668461 (0.0010) [2023-12-27 03:32:57,027][105692] Updated weights for policy 0, policy_version 1668471 (0.0010) [2023-12-27 03:32:57,054][105620] Updated weights for policy 1, policy_version 1671876 (0.0005) [2023-12-27 03:32:57,107][105620] Updated weights for policy 1, policy_version 1671886 (0.0009) [2023-12-27 03:32:57,164][105620] Updated weights for policy 1, policy_version 1671896 (0.0010) [2023-12-27 03:32:57,639][105692] Updated weights for policy 0, policy_version 1668481 (0.0010) [2023-12-27 03:32:57,698][105692] Updated weights for policy 0, policy_version 1668491 (0.0009) [2023-12-27 03:32:57,757][105692] Updated weights for policy 0, policy_version 1668501 (0.0009) [2023-12-27 03:32:57,759][105620] Updated weights for policy 1, policy_version 1671906 (0.0009) [2023-12-27 03:32:57,816][105620] Updated weights for policy 1, policy_version 1671916 (0.0005) [2023-12-27 03:32:57,817][105692] Updated weights for policy 0, policy_version 1668511 (0.0009) [2023-12-27 03:32:57,869][105620] Updated weights for policy 1, policy_version 1671926 (0.0005) [2023-12-27 03:32:57,921][105620] Updated weights for policy 1, policy_version 1671936 (0.0005) [2023-12-27 03:32:58,523][105620] Updated weights for policy 1, policy_version 1671946 (0.0008) [2023-12-27 03:32:58,575][105692] Updated weights for policy 0, policy_version 1668521 (0.0008) [2023-12-27 03:32:58,587][105620] Updated weights for policy 1, policy_version 1671956 (0.0007) [2023-12-27 03:32:58,638][105692] Updated weights for policy 0, policy_version 1668531 (0.0007) [2023-12-27 03:32:58,648][105620] Updated weights for policy 1, policy_version 1671966 (0.0009) [2023-12-27 03:32:58,702][105692] Updated weights for policy 0, policy_version 1668541 (0.0008) [2023-12-27 03:32:59,475][105620] Updated weights for policy 1, policy_version 1671976 (0.0009) [2023-12-27 03:32:59,510][105692] Updated weights for policy 0, policy_version 1668551 (0.0008) [2023-12-27 03:32:59,537][105620] Updated weights for policy 1, policy_version 1671986 (0.0009) [2023-12-27 03:32:59,561][105692] Updated weights for policy 0, policy_version 1668561 (0.0007) [2023-12-27 03:32:59,601][105620] Updated weights for policy 1, policy_version 1671996 (0.0009) [2023-12-27 03:32:59,616][105692] Updated weights for policy 0, policy_version 1668571 (0.0007) [2023-12-27 03:33:00,385][105692] Updated weights for policy 0, policy_version 1668581 (0.0008) [2023-12-27 03:33:00,442][105692] Updated weights for policy 0, policy_version 1668591 (0.0008) [2023-12-27 03:33:00,464][105620] Updated weights for policy 1, policy_version 1672006 (0.0010) [2023-12-27 03:33:00,494][105692] Updated weights for policy 0, policy_version 1668601 (0.0006) [2023-12-27 03:33:00,520][105620] Updated weights for policy 1, policy_version 1672016 (0.0010) [2023-12-27 03:33:00,575][105620] Updated weights for policy 1, policy_version 1672026 (0.0007) [2023-12-27 03:33:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 855326720. Throughput: 0: 9729.6, 1: 10000.8. Samples: 855302980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:33:01,062][104569] Avg episode reward: [(0, '8532.921'), (1, '8715.813')] [2023-12-27 03:33:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001668608_427229184.pth... [2023-12-27 03:33:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001672032_428097536.pth... [2023-12-27 03:33:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001667488_426942464.pth [2023-12-27 03:33:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001670880_427802624.pth [2023-12-27 03:33:01,104][105692] Updated weights for policy 0, policy_version 1668611 (0.0006) [2023-12-27 03:33:01,156][105620] Updated weights for policy 1, policy_version 1672036 (0.0008) [2023-12-27 03:33:01,166][105692] Updated weights for policy 0, policy_version 1668621 (0.0007) [2023-12-27 03:33:01,213][105620] Updated weights for policy 1, policy_version 1672046 (0.0006) [2023-12-27 03:33:01,219][105692] Updated weights for policy 0, policy_version 1668631 (0.0009) [2023-12-27 03:33:01,276][105620] Updated weights for policy 1, policy_version 1672056 (0.0007) [2023-12-27 03:33:01,936][105620] Updated weights for policy 1, policy_version 1672066 (0.0005) [2023-12-27 03:33:02,003][105620] Updated weights for policy 1, policy_version 1672076 (0.0005) [2023-12-27 03:33:02,006][105692] Updated weights for policy 0, policy_version 1668641 (0.0008) [2023-12-27 03:33:02,062][105620] Updated weights for policy 1, policy_version 1672086 (0.0007) [2023-12-27 03:33:02,064][105692] Updated weights for policy 0, policy_version 1668651 (0.0006) [2023-12-27 03:33:02,123][105620] Updated weights for policy 1, policy_version 1672096 (0.0007) [2023-12-27 03:33:02,129][105692] Updated weights for policy 0, policy_version 1668661 (0.0007) [2023-12-27 03:33:02,182][105692] Updated weights for policy 0, policy_version 1668671 (0.0009) [2023-12-27 03:33:02,744][105620] Updated weights for policy 1, policy_version 1672106 (0.0005) [2023-12-27 03:33:02,788][105620] Updated weights for policy 1, policy_version 1672116 (0.0008) [2023-12-27 03:33:02,838][105620] Updated weights for policy 1, policy_version 1672126 (0.0008) [2023-12-27 03:33:02,933][105692] Updated weights for policy 0, policy_version 1668681 (0.0009) [2023-12-27 03:33:02,979][105692] Updated weights for policy 0, policy_version 1668691 (0.0008) [2023-12-27 03:33:03,029][105692] Updated weights for policy 0, policy_version 1668701 (0.0009) [2023-12-27 03:33:03,590][105620] Updated weights for policy 1, policy_version 1672136 (0.0009) [2023-12-27 03:33:03,652][105620] Updated weights for policy 1, policy_version 1672147 (0.0010) [2023-12-27 03:33:03,713][105620] Updated weights for policy 1, policy_version 1672157 (0.0008) [2023-12-27 03:33:03,713][105692] Updated weights for policy 0, policy_version 1668711 (0.0006) [2023-12-27 03:33:03,761][105692] Updated weights for policy 0, policy_version 1668721 (0.0007) [2023-12-27 03:33:03,808][105692] Updated weights for policy 0, policy_version 1668731 (0.0006) [2023-12-27 03:33:04,495][105620] Updated weights for policy 1, policy_version 1672167 (0.0009) [2023-12-27 03:33:04,540][105692] Updated weights for policy 0, policy_version 1668741 (0.0006) [2023-12-27 03:33:04,559][105620] Updated weights for policy 1, policy_version 1672177 (0.0008) [2023-12-27 03:33:04,601][105692] Updated weights for policy 0, policy_version 1668751 (0.0007) [2023-12-27 03:33:04,608][105620] Updated weights for policy 1, policy_version 1672187 (0.0007) [2023-12-27 03:33:04,655][105692] Updated weights for policy 0, policy_version 1668761 (0.0008) [2023-12-27 03:33:05,180][105620] Updated weights for policy 1, policy_version 1672197 (0.0007) [2023-12-27 03:33:05,232][105620] Updated weights for policy 1, policy_version 1672207 (0.0005) [2023-12-27 03:33:05,283][105620] Updated weights for policy 1, policy_version 1672217 (0.0005) [2023-12-27 03:33:05,520][105692] Updated weights for policy 0, policy_version 1668771 (0.0008) [2023-12-27 03:33:05,575][105692] Updated weights for policy 0, policy_version 1668781 (0.0005) [2023-12-27 03:33:05,632][105692] Updated weights for policy 0, policy_version 1668791 (0.0005) [2023-12-27 03:33:05,908][105620] Updated weights for policy 1, policy_version 1672227 (0.0007) [2023-12-27 03:33:05,964][105620] Updated weights for policy 1, policy_version 1672237 (0.0005) [2023-12-27 03:33:06,018][105620] Updated weights for policy 1, policy_version 1672247 (0.0005) [2023-12-27 03:33:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 855425024. Throughput: 0: 9679.9, 1: 9879.4. Samples: 855418648. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:33:06,063][104569] Avg episode reward: [(0, '8715.461'), (1, '8990.662')] [2023-12-27 03:33:06,152][105692] Updated weights for policy 0, policy_version 1668801 (0.0006) [2023-12-27 03:33:06,210][105692] Updated weights for policy 0, policy_version 1668811 (0.0008) [2023-12-27 03:33:06,280][105692] Updated weights for policy 0, policy_version 1668821 (0.0008) [2023-12-27 03:33:06,339][105692] Updated weights for policy 0, policy_version 1668831 (0.0008) [2023-12-27 03:33:06,698][105620] Updated weights for policy 1, policy_version 1672257 (0.0006) [2023-12-27 03:33:06,751][105620] Updated weights for policy 1, policy_version 1672267 (0.0009) [2023-12-27 03:33:06,798][105620] Updated weights for policy 1, policy_version 1672277 (0.0008) [2023-12-27 03:33:06,852][105620] Updated weights for policy 1, policy_version 1672287 (0.0006) [2023-12-27 03:33:07,013][105692] Updated weights for policy 0, policy_version 1668841 (0.0009) [2023-12-27 03:33:07,072][105692] Updated weights for policy 0, policy_version 1668852 (0.0010) [2023-12-27 03:33:07,145][105692] Updated weights for policy 0, policy_version 1668862 (0.0010) [2023-12-27 03:33:07,586][105620] Updated weights for policy 1, policy_version 1672297 (0.0010) [2023-12-27 03:33:07,640][105620] Updated weights for policy 1, policy_version 1672307 (0.0010) [2023-12-27 03:33:07,698][105620] Updated weights for policy 1, policy_version 1672317 (0.0006) [2023-12-27 03:33:07,760][105692] Updated weights for policy 0, policy_version 1668873 (0.0008) [2023-12-27 03:33:07,822][105692] Updated weights for policy 0, policy_version 1668884 (0.0010) [2023-12-27 03:33:07,876][105692] Updated weights for policy 0, policy_version 1668895 (0.0010) [2023-12-27 03:33:08,314][105620] Updated weights for policy 1, policy_version 1672327 (0.0006) [2023-12-27 03:33:08,388][105620] Updated weights for policy 1, policy_version 1672337 (0.0007) [2023-12-27 03:33:08,456][105620] Updated weights for policy 1, policy_version 1672347 (0.0006) [2023-12-27 03:33:08,634][105692] Updated weights for policy 0, policy_version 1668905 (0.0010) [2023-12-27 03:33:08,691][105692] Updated weights for policy 0, policy_version 1668915 (0.0009) [2023-12-27 03:33:08,744][105692] Updated weights for policy 0, policy_version 1668925 (0.0009) [2023-12-27 03:33:09,044][105620] Updated weights for policy 1, policy_version 1672357 (0.0008) [2023-12-27 03:33:09,100][105620] Updated weights for policy 1, policy_version 1672367 (0.0006) [2023-12-27 03:33:09,154][105620] Updated weights for policy 1, policy_version 1672377 (0.0008) [2023-12-27 03:33:09,554][105692] Updated weights for policy 0, policy_version 1668935 (0.0009) [2023-12-27 03:33:09,623][105692] Updated weights for policy 0, policy_version 1668945 (0.0010) [2023-12-27 03:33:09,691][105692] Updated weights for policy 0, policy_version 1668955 (0.0011) [2023-12-27 03:33:09,866][105620] Updated weights for policy 1, policy_version 1672387 (0.0010) [2023-12-27 03:33:09,925][105620] Updated weights for policy 1, policy_version 1672397 (0.0009) [2023-12-27 03:33:09,975][105620] Updated weights for policy 1, policy_version 1672407 (0.0008) [2023-12-27 03:33:10,426][105692] Updated weights for policy 0, policy_version 1668965 (0.0011) [2023-12-27 03:33:10,491][105692] Updated weights for policy 0, policy_version 1668975 (0.0010) [2023-12-27 03:33:10,558][105692] Updated weights for policy 0, policy_version 1668985 (0.0010) [2023-12-27 03:33:10,707][105620] Updated weights for policy 1, policy_version 1672417 (0.0008) [2023-12-27 03:33:10,767][105620] Updated weights for policy 1, policy_version 1672427 (0.0006) [2023-12-27 03:33:10,827][105620] Updated weights for policy 1, policy_version 1672437 (0.0006) [2023-12-27 03:33:10,887][105620] Updated weights for policy 1, policy_version 1672447 (0.0008) [2023-12-27 03:33:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 855531520. Throughput: 0: 9703.6, 1: 9992.8. Samples: 855539928. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:33:11,062][104569] Avg episode reward: [(0, '8804.123'), (1, '8896.740')] [2023-12-27 03:33:11,287][105692] Updated weights for policy 0, policy_version 1668995 (0.0011) [2023-12-27 03:33:11,348][105692] Updated weights for policy 0, policy_version 1669005 (0.0013) [2023-12-27 03:33:11,415][105692] Updated weights for policy 0, policy_version 1669015 (0.0009) [2023-12-27 03:33:11,584][105620] Updated weights for policy 1, policy_version 1672457 (0.0008) [2023-12-27 03:33:11,657][105620] Updated weights for policy 1, policy_version 1672467 (0.0009) [2023-12-27 03:33:11,725][105620] Updated weights for policy 1, policy_version 1672477 (0.0010) [2023-12-27 03:33:12,183][105692] Updated weights for policy 0, policy_version 1669025 (0.0008) [2023-12-27 03:33:12,242][105692] Updated weights for policy 0, policy_version 1669035 (0.0009) [2023-12-27 03:33:12,302][105692] Updated weights for policy 0, policy_version 1669045 (0.0009) [2023-12-27 03:33:12,365][105692] Updated weights for policy 0, policy_version 1669055 (0.0009) [2023-12-27 03:33:12,458][105620] Updated weights for policy 1, policy_version 1672487 (0.0008) [2023-12-27 03:33:12,505][105620] Updated weights for policy 1, policy_version 1672497 (0.0008) [2023-12-27 03:33:12,562][105620] Updated weights for policy 1, policy_version 1672507 (0.0005) [2023-12-27 03:33:13,160][105692] Updated weights for policy 0, policy_version 1669065 (0.0010) [2023-12-27 03:33:13,224][105692] Updated weights for policy 0, policy_version 1669075 (0.0009) [2023-12-27 03:33:13,253][105620] Updated weights for policy 1, policy_version 1672517 (0.0007) [2023-12-27 03:33:13,280][105692] Updated weights for policy 0, policy_version 1669085 (0.0008) [2023-12-27 03:33:13,306][105620] Updated weights for policy 1, policy_version 1672527 (0.0008) [2023-12-27 03:33:13,368][105620] Updated weights for policy 1, policy_version 1672537 (0.0009) [2023-12-27 03:33:13,949][105692] Updated weights for policy 0, policy_version 1669095 (0.0005) [2023-12-27 03:33:14,001][105692] Updated weights for policy 0, policy_version 1669105 (0.0008) [2023-12-27 03:33:14,057][105692] Updated weights for policy 0, policy_version 1669115 (0.0009) [2023-12-27 03:33:14,150][105620] Updated weights for policy 1, policy_version 1672547 (0.0009) [2023-12-27 03:33:14,208][105620] Updated weights for policy 1, policy_version 1672557 (0.0009) [2023-12-27 03:33:14,273][105620] Updated weights for policy 1, policy_version 1672567 (0.0008) [2023-12-27 03:33:14,745][105692] Updated weights for policy 0, policy_version 1669125 (0.0011) [2023-12-27 03:33:14,812][105692] Updated weights for policy 0, policy_version 1669135 (0.0009) [2023-12-27 03:33:14,868][105692] Updated weights for policy 0, policy_version 1669145 (0.0010) [2023-12-27 03:33:15,041][105620] Updated weights for policy 1, policy_version 1672577 (0.0009) [2023-12-27 03:33:15,103][105620] Updated weights for policy 1, policy_version 1672587 (0.0009) [2023-12-27 03:33:15,161][105620] Updated weights for policy 1, policy_version 1672597 (0.0008) [2023-12-27 03:33:15,225][105620] Updated weights for policy 1, policy_version 1672607 (0.0006) [2023-12-27 03:33:15,657][105692] Updated weights for policy 0, policy_version 1669155 (0.0011) [2023-12-27 03:33:15,727][105692] Updated weights for policy 0, policy_version 1669165 (0.0011) [2023-12-27 03:33:15,790][105692] Updated weights for policy 0, policy_version 1669175 (0.0011) [2023-12-27 03:33:15,813][105620] Updated weights for policy 1, policy_version 1672617 (0.0006) [2023-12-27 03:33:15,873][105620] Updated weights for policy 1, policy_version 1672627 (0.0006) [2023-12-27 03:33:15,921][105620] Updated weights for policy 1, policy_version 1672637 (0.0005) [2023-12-27 03:33:16,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 855629824. Throughput: 0: 9678.8, 1: 9969.0. Samples: 855595416. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:33:16,063][104569] Avg episode reward: [(0, '8710.619'), (1, '8901.658')] [2023-12-27 03:33:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001669184_427376640.pth... [2023-12-27 03:33:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001672640_428253184.pth... [2023-12-27 03:33:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001668064_427089920.pth [2023-12-27 03:33:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001671456_427950080.pth [2023-12-27 03:33:16,404][105692] Updated weights for policy 0, policy_version 1669185 (0.0011) [2023-12-27 03:33:16,455][105692] Updated weights for policy 0, policy_version 1669195 (0.0010) [2023-12-27 03:33:16,499][105692] Updated weights for policy 0, policy_version 1669205 (0.0010) [2023-12-27 03:33:16,547][105692] Updated weights for policy 0, policy_version 1669215 (0.0010) [2023-12-27 03:33:16,670][105620] Updated weights for policy 1, policy_version 1672647 (0.0007) [2023-12-27 03:33:16,729][105620] Updated weights for policy 1, policy_version 1672657 (0.0007) [2023-12-27 03:33:16,787][105620] Updated weights for policy 1, policy_version 1672667 (0.0008) [2023-12-27 03:33:17,305][105692] Updated weights for policy 0, policy_version 1669225 (0.0010) [2023-12-27 03:33:17,352][105692] Updated weights for policy 0, policy_version 1669235 (0.0009) [2023-12-27 03:33:17,405][105692] Updated weights for policy 0, policy_version 1669245 (0.0010) [2023-12-27 03:33:17,558][105620] Updated weights for policy 1, policy_version 1672677 (0.0008) [2023-12-27 03:33:17,618][105620] Updated weights for policy 1, policy_version 1672687 (0.0007) [2023-12-27 03:33:17,671][105620] Updated weights for policy 1, policy_version 1672697 (0.0005) [2023-12-27 03:33:18,131][105692] Updated weights for policy 0, policy_version 1669255 (0.0009) [2023-12-27 03:33:18,181][105692] Updated weights for policy 0, policy_version 1669265 (0.0010) [2023-12-27 03:33:18,233][105692] Updated weights for policy 0, policy_version 1669275 (0.0010) [2023-12-27 03:33:18,337][105620] Updated weights for policy 1, policy_version 1672707 (0.0007) [2023-12-27 03:33:18,403][105620] Updated weights for policy 1, policy_version 1672717 (0.0007) [2023-12-27 03:33:18,470][105620] Updated weights for policy 1, policy_version 1672727 (0.0006) [2023-12-27 03:33:18,956][105692] Updated weights for policy 0, policy_version 1669285 (0.0009) [2023-12-27 03:33:19,008][105692] Updated weights for policy 0, policy_version 1669295 (0.0011) [2023-12-27 03:33:19,046][105620] Updated weights for policy 1, policy_version 1672737 (0.0006) [2023-12-27 03:33:19,061][105692] Updated weights for policy 0, policy_version 1669305 (0.0010) [2023-12-27 03:33:19,104][105620] Updated weights for policy 1, policy_version 1672747 (0.0005) [2023-12-27 03:33:19,160][105620] Updated weights for policy 1, policy_version 1672757 (0.0008) [2023-12-27 03:33:19,217][105620] Updated weights for policy 1, policy_version 1672767 (0.0006) [2023-12-27 03:33:19,831][105692] Updated weights for policy 0, policy_version 1669315 (0.0010) [2023-12-27 03:33:19,905][105692] Updated weights for policy 0, policy_version 1669325 (0.0010) [2023-12-27 03:33:19,965][105620] Updated weights for policy 1, policy_version 1672777 (0.0007) [2023-12-27 03:33:19,974][105692] Updated weights for policy 0, policy_version 1669335 (0.0009) [2023-12-27 03:33:20,032][105620] Updated weights for policy 1, policy_version 1672787 (0.0011) [2023-12-27 03:33:20,101][105620] Updated weights for policy 1, policy_version 1672797 (0.0011) [2023-12-27 03:33:20,706][105692] Updated weights for policy 0, policy_version 1669345 (0.0009) [2023-12-27 03:33:20,766][105692] Updated weights for policy 0, policy_version 1669355 (0.0011) [2023-12-27 03:33:20,831][105692] Updated weights for policy 0, policy_version 1669365 (0.0011) [2023-12-27 03:33:20,863][105620] Updated weights for policy 1, policy_version 1672807 (0.0010) [2023-12-27 03:33:20,895][105692] Updated weights for policy 0, policy_version 1669375 (0.0011) [2023-12-27 03:33:20,921][105620] Updated weights for policy 1, policy_version 1672817 (0.0009) [2023-12-27 03:33:20,987][105620] Updated weights for policy 1, policy_version 1672827 (0.0005) [2023-12-27 03:33:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 855728128. Throughput: 0: 9705.1, 1: 9996.6. Samples: 855713916. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:33:21,062][104569] Avg episode reward: [(0, '8893.800'), (1, '9083.642')] [2023-12-27 03:33:21,622][105620] Updated weights for policy 1, policy_version 1672837 (0.0008) [2023-12-27 03:33:21,649][105692] Updated weights for policy 0, policy_version 1669385 (0.0011) [2023-12-27 03:33:21,685][105620] Updated weights for policy 1, policy_version 1672847 (0.0010) [2023-12-27 03:33:21,710][105692] Updated weights for policy 0, policy_version 1669395 (0.0011) [2023-12-27 03:33:21,753][105620] Updated weights for policy 1, policy_version 1672857 (0.0009) [2023-12-27 03:33:21,781][105692] Updated weights for policy 0, policy_version 1669405 (0.0008) [2023-12-27 03:33:22,510][105620] Updated weights for policy 1, policy_version 1672867 (0.0011) [2023-12-27 03:33:22,554][105692] Updated weights for policy 0, policy_version 1669415 (0.0007) [2023-12-27 03:33:22,577][105620] Updated weights for policy 1, policy_version 1672877 (0.0011) [2023-12-27 03:33:22,618][105692] Updated weights for policy 0, policy_version 1669425 (0.0009) [2023-12-27 03:33:22,636][105620] Updated weights for policy 1, policy_version 1672887 (0.0010) [2023-12-27 03:33:22,682][105692] Updated weights for policy 0, policy_version 1669435 (0.0011) [2023-12-27 03:33:23,384][105692] Updated weights for policy 0, policy_version 1669445 (0.0010) [2023-12-27 03:33:23,385][105620] Updated weights for policy 1, policy_version 1672897 (0.0011) [2023-12-27 03:33:23,432][105692] Updated weights for policy 0, policy_version 1669455 (0.0010) [2023-12-27 03:33:23,436][105620] Updated weights for policy 1, policy_version 1672907 (0.0010) [2023-12-27 03:33:23,484][105692] Updated weights for policy 0, policy_version 1669465 (0.0010) [2023-12-27 03:33:23,488][105620] Updated weights for policy 1, policy_version 1672917 (0.0010) [2023-12-27 03:33:23,546][105620] Updated weights for policy 1, policy_version 1672927 (0.0010) [2023-12-27 03:33:24,172][105692] Updated weights for policy 0, policy_version 1669475 (0.0008) [2023-12-27 03:33:24,223][105692] Updated weights for policy 0, policy_version 1669485 (0.0010) [2023-12-27 03:33:24,253][105620] Updated weights for policy 1, policy_version 1672937 (0.0007) [2023-12-27 03:33:24,272][105692] Updated weights for policy 0, policy_version 1669495 (0.0009) [2023-12-27 03:33:24,306][105620] Updated weights for policy 1, policy_version 1672947 (0.0005) [2023-12-27 03:33:24,365][105620] Updated weights for policy 1, policy_version 1672957 (0.0008) [2023-12-27 03:33:24,963][105692] Updated weights for policy 0, policy_version 1669505 (0.0008) [2023-12-27 03:33:25,015][105692] Updated weights for policy 0, policy_version 1669515 (0.0007) [2023-12-27 03:33:25,058][105692] Updated weights for policy 0, policy_version 1669525 (0.0007) [2023-12-27 03:33:25,104][105692] Updated weights for policy 0, policy_version 1669535 (0.0006) [2023-12-27 03:33:25,127][105620] Updated weights for policy 1, policy_version 1672967 (0.0008) [2023-12-27 03:33:25,171][105620] Updated weights for policy 1, policy_version 1672977 (0.0008) [2023-12-27 03:33:25,234][105620] Updated weights for policy 1, policy_version 1672987 (0.0006) [2023-12-27 03:33:25,864][105692] Updated weights for policy 0, policy_version 1669545 (0.0010) [2023-12-27 03:33:25,888][105620] Updated weights for policy 1, policy_version 1672997 (0.0007) [2023-12-27 03:33:25,911][105692] Updated weights for policy 0, policy_version 1669555 (0.0010) [2023-12-27 03:33:25,945][105620] Updated weights for policy 1, policy_version 1673007 (0.0005) [2023-12-27 03:33:25,957][105692] Updated weights for policy 0, policy_version 1669565 (0.0007) [2023-12-27 03:33:26,006][105620] Updated weights for policy 1, policy_version 1673017 (0.0007) [2023-12-27 03:33:26,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.9, 300 sec: 19633.0). Total num frames: 855826432. Throughput: 0: 9628.4, 1: 10019.7. Samples: 855829044. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:33:26,062][104569] Avg episode reward: [(0, '8713.221'), (1, '9263.595')] [2023-12-27 03:33:26,715][105620] Updated weights for policy 1, policy_version 1673027 (0.0009) [2023-12-27 03:33:26,747][105692] Updated weights for policy 0, policy_version 1669575 (0.0010) [2023-12-27 03:33:26,781][105620] Updated weights for policy 1, policy_version 1673037 (0.0006) [2023-12-27 03:33:26,792][105692] Updated weights for policy 0, policy_version 1669585 (0.0009) [2023-12-27 03:33:26,836][105692] Updated weights for policy 0, policy_version 1669595 (0.0005) [2023-12-27 03:33:26,838][105620] Updated weights for policy 1, policy_version 1673047 (0.0006) [2023-12-27 03:33:27,481][105692] Updated weights for policy 0, policy_version 1669605 (0.0005) [2023-12-27 03:33:27,528][105620] Updated weights for policy 1, policy_version 1673057 (0.0005) [2023-12-27 03:33:27,545][105692] Updated weights for policy 0, policy_version 1669615 (0.0005) [2023-12-27 03:33:27,576][105620] Updated weights for policy 1, policy_version 1673067 (0.0006) [2023-12-27 03:33:27,601][105692] Updated weights for policy 0, policy_version 1669625 (0.0008) [2023-12-27 03:33:27,632][105620] Updated weights for policy 1, policy_version 1673077 (0.0007) [2023-12-27 03:33:27,692][105620] Updated weights for policy 1, policy_version 1673087 (0.0007) [2023-12-27 03:33:28,156][105692] Updated weights for policy 0, policy_version 1669635 (0.0009) [2023-12-27 03:33:28,199][105692] Updated weights for policy 0, policy_version 1669645 (0.0005) [2023-12-27 03:33:28,244][105692] Updated weights for policy 0, policy_version 1669655 (0.0006) [2023-12-27 03:33:28,527][105620] Updated weights for policy 1, policy_version 1673097 (0.0007) [2023-12-27 03:33:28,592][105620] Updated weights for policy 1, policy_version 1673107 (0.0005) [2023-12-27 03:33:28,654][105620] Updated weights for policy 1, policy_version 1673117 (0.0006) [2023-12-27 03:33:28,837][105692] Updated weights for policy 0, policy_version 1669665 (0.0008) [2023-12-27 03:33:28,902][105692] Updated weights for policy 0, policy_version 1669675 (0.0007) [2023-12-27 03:33:28,966][105692] Updated weights for policy 0, policy_version 1669685 (0.0010) [2023-12-27 03:33:29,027][105692] Updated weights for policy 0, policy_version 1669695 (0.0010) [2023-12-27 03:33:29,292][105620] Updated weights for policy 1, policy_version 1673127 (0.0007) [2023-12-27 03:33:29,360][105620] Updated weights for policy 1, policy_version 1673137 (0.0008) [2023-12-27 03:33:29,421][105620] Updated weights for policy 1, policy_version 1673147 (0.0005) [2023-12-27 03:33:29,644][105692] Updated weights for policy 0, policy_version 1669705 (0.0006) [2023-12-27 03:33:29,703][105692] Updated weights for policy 0, policy_version 1669715 (0.0010) [2023-12-27 03:33:29,757][105692] Updated weights for policy 0, policy_version 1669725 (0.0010) [2023-12-27 03:33:30,160][105620] Updated weights for policy 1, policy_version 1673157 (0.0007) [2023-12-27 03:33:30,213][105620] Updated weights for policy 1, policy_version 1673167 (0.0008) [2023-12-27 03:33:30,296][105620] Updated weights for policy 1, policy_version 1673177 (0.0008) [2023-12-27 03:33:30,498][105692] Updated weights for policy 0, policy_version 1669735 (0.0009) [2023-12-27 03:33:30,561][105692] Updated weights for policy 0, policy_version 1669745 (0.0011) [2023-12-27 03:33:30,621][105692] Updated weights for policy 0, policy_version 1669755 (0.0010) [2023-12-27 03:33:31,012][105620] Updated weights for policy 1, policy_version 1673187 (0.0008) [2023-12-27 03:33:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 855916544. Throughput: 0: 9692.2, 1: 9915.9. Samples: 855889864. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:33:31,062][104569] Avg episode reward: [(0, '8528.662'), (1, '9356.191')] [2023-12-27 03:33:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001669760_427524096.pth... [2023-12-27 03:33:31,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001668608_427229184.pth [2023-12-27 03:33:31,084][105620] Updated weights for policy 1, policy_version 1673197 (0.0008) [2023-12-27 03:33:31,149][105620] Updated weights for policy 1, policy_version 1673207 (0.0008) [2023-12-27 03:33:31,205][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001673216_428400640.pth... [2023-12-27 03:33:31,210][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001672032_428097536.pth [2023-12-27 03:33:31,374][105692] Updated weights for policy 0, policy_version 1669765 (0.0010) [2023-12-27 03:33:31,439][105692] Updated weights for policy 0, policy_version 1669775 (0.0007) [2023-12-27 03:33:31,496][105692] Updated weights for policy 0, policy_version 1669785 (0.0006) [2023-12-27 03:33:31,870][105620] Updated weights for policy 1, policy_version 1673217 (0.0008) [2023-12-27 03:33:31,922][105620] Updated weights for policy 1, policy_version 1673227 (0.0005) [2023-12-27 03:33:31,969][105620] Updated weights for policy 1, policy_version 1673237 (0.0005) [2023-12-27 03:33:32,022][105620] Updated weights for policy 1, policy_version 1673247 (0.0005) [2023-12-27 03:33:32,170][105692] Updated weights for policy 0, policy_version 1669795 (0.0005) [2023-12-27 03:33:32,232][105692] Updated weights for policy 0, policy_version 1669805 (0.0006) [2023-12-27 03:33:32,299][105692] Updated weights for policy 0, policy_version 1669815 (0.0007) [2023-12-27 03:33:32,684][105620] Updated weights for policy 1, policy_version 1673257 (0.0010) [2023-12-27 03:33:32,746][105620] Updated weights for policy 1, policy_version 1673267 (0.0010) [2023-12-27 03:33:32,801][105620] Updated weights for policy 1, policy_version 1673277 (0.0010) [2023-12-27 03:33:32,918][105692] Updated weights for policy 0, policy_version 1669825 (0.0007) [2023-12-27 03:33:32,970][105692] Updated weights for policy 0, policy_version 1669835 (0.0008) [2023-12-27 03:33:33,027][105692] Updated weights for policy 0, policy_version 1669845 (0.0007) [2023-12-27 03:33:33,082][105692] Updated weights for policy 0, policy_version 1669855 (0.0008) [2023-12-27 03:33:33,527][105620] Updated weights for policy 1, policy_version 1673287 (0.0007) [2023-12-27 03:33:33,586][105620] Updated weights for policy 1, policy_version 1673297 (0.0007) [2023-12-27 03:33:33,646][105620] Updated weights for policy 1, policy_version 1673307 (0.0009) [2023-12-27 03:33:33,687][105692] Updated weights for policy 0, policy_version 1669865 (0.0006) [2023-12-27 03:33:33,733][105692] Updated weights for policy 0, policy_version 1669875 (0.0005) [2023-12-27 03:33:33,798][105692] Updated weights for policy 0, policy_version 1669885 (0.0009) [2023-12-27 03:33:34,255][105620] Updated weights for policy 1, policy_version 1673317 (0.0011) [2023-12-27 03:33:34,317][105620] Updated weights for policy 1, policy_version 1673327 (0.0011) [2023-12-27 03:33:34,370][105620] Updated weights for policy 1, policy_version 1673337 (0.0011) [2023-12-27 03:33:34,498][105692] Updated weights for policy 0, policy_version 1669895 (0.0007) [2023-12-27 03:33:34,548][105692] Updated weights for policy 0, policy_version 1669905 (0.0005) [2023-12-27 03:33:34,601][105692] Updated weights for policy 0, policy_version 1669915 (0.0007) [2023-12-27 03:33:35,134][105620] Updated weights for policy 1, policy_version 1673347 (0.0011) [2023-12-27 03:33:35,193][105620] Updated weights for policy 1, policy_version 1673357 (0.0011) [2023-12-27 03:33:35,244][105620] Updated weights for policy 1, policy_version 1673367 (0.0010) [2023-12-27 03:33:35,276][105692] Updated weights for policy 0, policy_version 1669925 (0.0007) [2023-12-27 03:33:35,330][105692] Updated weights for policy 0, policy_version 1669935 (0.0008) [2023-12-27 03:33:35,379][105692] Updated weights for policy 0, policy_version 1669945 (0.0011) [2023-12-27 03:33:35,967][105692] Updated weights for policy 0, policy_version 1669956 (0.0006) [2023-12-27 03:33:35,989][105620] Updated weights for policy 1, policy_version 1673377 (0.0010) [2023-12-27 03:33:36,023][105692] Updated weights for policy 0, policy_version 1669966 (0.0005) [2023-12-27 03:33:36,045][105620] Updated weights for policy 1, policy_version 1673387 (0.0010) [2023-12-27 03:33:36,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19660.7, 300 sec: 19605.2). Total num frames: 856014848. Throughput: 0: 9735.2, 1: 9931.4. Samples: 856010168. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:33:36,064][104569] Avg episode reward: [(0, '8896.362'), (1, '9173.952')] [2023-12-27 03:33:36,078][105692] Updated weights for policy 0, policy_version 1669976 (0.0006) [2023-12-27 03:33:36,105][105620] Updated weights for policy 1, policy_version 1673397 (0.0010) [2023-12-27 03:33:36,165][105620] Updated weights for policy 1, policy_version 1673407 (0.0011) [2023-12-27 03:33:36,643][105692] Updated weights for policy 0, policy_version 1669986 (0.0008) [2023-12-27 03:33:36,707][105692] Updated weights for policy 0, policy_version 1669996 (0.0011) [2023-12-27 03:33:36,776][105692] Updated weights for policy 0, policy_version 1670006 (0.0010) [2023-12-27 03:33:36,841][105692] Updated weights for policy 0, policy_version 1670016 (0.0008) [2023-12-27 03:33:36,920][105620] Updated weights for policy 1, policy_version 1673417 (0.0010) [2023-12-27 03:33:36,986][105620] Updated weights for policy 1, policy_version 1673427 (0.0011) [2023-12-27 03:33:37,056][105620] Updated weights for policy 1, policy_version 1673437 (0.0011) [2023-12-27 03:33:37,466][105692] Updated weights for policy 0, policy_version 1670026 (0.0011) [2023-12-27 03:33:37,524][105692] Updated weights for policy 0, policy_version 1670036 (0.0010) [2023-12-27 03:33:37,584][105692] Updated weights for policy 0, policy_version 1670046 (0.0010) [2023-12-27 03:33:37,796][105620] Updated weights for policy 1, policy_version 1673447 (0.0011) [2023-12-27 03:33:37,866][105620] Updated weights for policy 1, policy_version 1673457 (0.0011) [2023-12-27 03:33:37,926][105620] Updated weights for policy 1, policy_version 1673467 (0.0011) [2023-12-27 03:33:38,338][105692] Updated weights for policy 0, policy_version 1670056 (0.0008) [2023-12-27 03:33:38,395][105692] Updated weights for policy 0, policy_version 1670067 (0.0009) [2023-12-27 03:33:38,444][105692] Updated weights for policy 0, policy_version 1670077 (0.0008) [2023-12-27 03:33:38,612][105620] Updated weights for policy 1, policy_version 1673477 (0.0011) [2023-12-27 03:33:38,682][105620] Updated weights for policy 1, policy_version 1673487 (0.0011) [2023-12-27 03:33:38,734][105620] Updated weights for policy 1, policy_version 1673497 (0.0011) [2023-12-27 03:33:39,234][105692] Updated weights for policy 0, policy_version 1670087 (0.0008) [2023-12-27 03:33:39,297][105692] Updated weights for policy 0, policy_version 1670097 (0.0008) [2023-12-27 03:33:39,354][105692] Updated weights for policy 0, policy_version 1670107 (0.0008) [2023-12-27 03:33:39,523][105620] Updated weights for policy 1, policy_version 1673507 (0.0011) [2023-12-27 03:33:39,586][105620] Updated weights for policy 1, policy_version 1673517 (0.0011) [2023-12-27 03:33:39,649][105620] Updated weights for policy 1, policy_version 1673527 (0.0011) [2023-12-27 03:33:40,072][105692] Updated weights for policy 0, policy_version 1670117 (0.0009) [2023-12-27 03:33:40,138][105692] Updated weights for policy 0, policy_version 1670127 (0.0008) [2023-12-27 03:33:40,202][105692] Updated weights for policy 0, policy_version 1670137 (0.0007) [2023-12-27 03:33:40,333][105620] Updated weights for policy 1, policy_version 1673537 (0.0009) [2023-12-27 03:33:40,388][105620] Updated weights for policy 1, policy_version 1673547 (0.0011) [2023-12-27 03:33:40,451][105620] Updated weights for policy 1, policy_version 1673557 (0.0010) [2023-12-27 03:33:40,506][105620] Updated weights for policy 1, policy_version 1673567 (0.0010) [2023-12-27 03:33:40,867][105692] Updated weights for policy 0, policy_version 1670147 (0.0007) [2023-12-27 03:33:40,932][105692] Updated weights for policy 0, policy_version 1670157 (0.0011) [2023-12-27 03:33:40,995][105692] Updated weights for policy 0, policy_version 1670167 (0.0011) [2023-12-27 03:33:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 856121344. Throughput: 0: 9873.1, 1: 9864.1. Samples: 856128796. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:33:41,062][104569] Avg episode reward: [(0, '8896.526'), (1, '8898.113')] [2023-12-27 03:33:41,202][105620] Updated weights for policy 1, policy_version 1673577 (0.0011) [2023-12-27 03:33:41,262][105620] Updated weights for policy 1, policy_version 1673587 (0.0010) [2023-12-27 03:33:41,326][105620] Updated weights for policy 1, policy_version 1673597 (0.0009) [2023-12-27 03:33:41,715][105692] Updated weights for policy 0, policy_version 1670177 (0.0008) [2023-12-27 03:33:41,782][105692] Updated weights for policy 0, policy_version 1670187 (0.0008) [2023-12-27 03:33:41,836][105692] Updated weights for policy 0, policy_version 1670197 (0.0010) [2023-12-27 03:33:41,899][105692] Updated weights for policy 0, policy_version 1670207 (0.0010) [2023-12-27 03:33:42,043][105620] Updated weights for policy 1, policy_version 1673607 (0.0009) [2023-12-27 03:33:42,097][105620] Updated weights for policy 1, policy_version 1673617 (0.0010) [2023-12-27 03:33:42,153][105620] Updated weights for policy 1, policy_version 1673627 (0.0009) [2023-12-27 03:33:42,657][105692] Updated weights for policy 0, policy_version 1670217 (0.0007) [2023-12-27 03:33:42,713][105692] Updated weights for policy 0, policy_version 1670227 (0.0006) [2023-12-27 03:33:42,770][105692] Updated weights for policy 0, policy_version 1670237 (0.0006) [2023-12-27 03:33:42,937][105620] Updated weights for policy 1, policy_version 1673637 (0.0009) [2023-12-27 03:33:42,994][105620] Updated weights for policy 1, policy_version 1673647 (0.0010) [2023-12-27 03:33:43,058][105620] Updated weights for policy 1, policy_version 1673657 (0.0009) [2023-12-27 03:33:43,366][105692] Updated weights for policy 0, policy_version 1670247 (0.0008) [2023-12-27 03:33:43,424][105692] Updated weights for policy 0, policy_version 1670257 (0.0008) [2023-12-27 03:33:43,486][105692] Updated weights for policy 0, policy_version 1670267 (0.0008) [2023-12-27 03:33:43,844][105620] Updated weights for policy 1, policy_version 1673668 (0.0010) [2023-12-27 03:33:43,892][105620] Updated weights for policy 1, policy_version 1673678 (0.0010) [2023-12-27 03:33:43,941][105620] Updated weights for policy 1, policy_version 1673688 (0.0010) [2023-12-27 03:33:44,247][105692] Updated weights for policy 0, policy_version 1670277 (0.0008) [2023-12-27 03:33:44,302][105692] Updated weights for policy 0, policy_version 1670287 (0.0008) [2023-12-27 03:33:44,353][105692] Updated weights for policy 0, policy_version 1670297 (0.0008) [2023-12-27 03:33:44,700][105620] Updated weights for policy 1, policy_version 1673698 (0.0010) [2023-12-27 03:33:44,745][105620] Updated weights for policy 1, policy_version 1673708 (0.0010) [2023-12-27 03:33:44,803][105620] Updated weights for policy 1, policy_version 1673718 (0.0009) [2023-12-27 03:33:44,856][105620] Updated weights for policy 1, policy_version 1673728 (0.0010) [2023-12-27 03:33:45,139][105692] Updated weights for policy 0, policy_version 1670307 (0.0008) [2023-12-27 03:33:45,195][105692] Updated weights for policy 0, policy_version 1670317 (0.0008) [2023-12-27 03:33:45,255][105692] Updated weights for policy 0, policy_version 1670327 (0.0008) [2023-12-27 03:33:45,641][105620] Updated weights for policy 1, policy_version 1673738 (0.0011) [2023-12-27 03:33:45,716][105620] Updated weights for policy 1, policy_version 1673748 (0.0011) [2023-12-27 03:33:45,783][105620] Updated weights for policy 1, policy_version 1673758 (0.0011) [2023-12-27 03:33:46,005][105692] Updated weights for policy 0, policy_version 1670337 (0.0008) [2023-12-27 03:33:46,061][105692] Updated weights for policy 0, policy_version 1670347 (0.0008) [2023-12-27 03:33:46,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 856211456. Throughput: 0: 9853.7, 1: 9751.3. Samples: 856185204. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:33:46,063][104569] Avg episode reward: [(0, '8623.631'), (1, '8718.019')] [2023-12-27 03:33:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001673760_428539904.pth... [2023-12-27 03:33:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001672640_428253184.pth [2023-12-27 03:33:46,121][105692] Updated weights for policy 0, policy_version 1670357 (0.0008) [2023-12-27 03:33:46,181][105692] Updated weights for policy 0, policy_version 1670367 (0.0007) [2023-12-27 03:33:46,188][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001670368_427679744.pth... [2023-12-27 03:33:46,193][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001669184_427376640.pth [2023-12-27 03:33:46,447][105620] Updated weights for policy 1, policy_version 1673768 (0.0009) [2023-12-27 03:33:46,510][105620] Updated weights for policy 1, policy_version 1673778 (0.0009) [2023-12-27 03:33:46,575][105620] Updated weights for policy 1, policy_version 1673788 (0.0011) [2023-12-27 03:33:46,929][105692] Updated weights for policy 0, policy_version 1670377 (0.0008) [2023-12-27 03:33:46,990][105692] Updated weights for policy 0, policy_version 1670387 (0.0009) [2023-12-27 03:33:47,036][105692] Updated weights for policy 0, policy_version 1670397 (0.0008) [2023-12-27 03:33:47,309][105620] Updated weights for policy 1, policy_version 1673798 (0.0007) [2023-12-27 03:33:47,377][105620] Updated weights for policy 1, policy_version 1673808 (0.0006) [2023-12-27 03:33:47,440][105620] Updated weights for policy 1, policy_version 1673818 (0.0009) [2023-12-27 03:33:47,840][105692] Updated weights for policy 0, policy_version 1670407 (0.0008) [2023-12-27 03:33:47,891][105692] Updated weights for policy 0, policy_version 1670417 (0.0008) [2023-12-27 03:33:47,954][105692] Updated weights for policy 0, policy_version 1670427 (0.0008) [2023-12-27 03:33:48,094][105620] Updated weights for policy 1, policy_version 1673828 (0.0007) [2023-12-27 03:33:48,158][105620] Updated weights for policy 1, policy_version 1673838 (0.0010) [2023-12-27 03:33:48,215][105620] Updated weights for policy 1, policy_version 1673848 (0.0010) [2023-12-27 03:33:48,707][105692] Updated weights for policy 0, policy_version 1670437 (0.0010) [2023-12-27 03:33:48,761][105692] Updated weights for policy 0, policy_version 1670447 (0.0010) [2023-12-27 03:33:48,818][105692] Updated weights for policy 0, policy_version 1670457 (0.0009) [2023-12-27 03:33:48,901][105620] Updated weights for policy 1, policy_version 1673858 (0.0010) [2023-12-27 03:33:48,967][105620] Updated weights for policy 1, policy_version 1673868 (0.0010) [2023-12-27 03:33:49,017][105586] KL-divergence is very high: 132.5832 [2023-12-27 03:33:49,030][105620] Updated weights for policy 1, policy_version 1673878 (0.0010) [2023-12-27 03:33:49,070][105586] KL-divergence is very high: 211.0710 [2023-12-27 03:33:49,096][105620] Updated weights for policy 1, policy_version 1673888 (0.0011) [2023-12-27 03:33:49,645][105692] Updated weights for policy 0, policy_version 1670467 (0.0008) [2023-12-27 03:33:49,710][105692] Updated weights for policy 0, policy_version 1670477 (0.0008) [2023-12-27 03:33:49,748][105620] Updated weights for policy 1, policy_version 1673898 (0.0005) [2023-12-27 03:33:49,775][105692] Updated weights for policy 0, policy_version 1670487 (0.0008) [2023-12-27 03:33:49,804][105620] Updated weights for policy 1, policy_version 1673908 (0.0008) [2023-12-27 03:33:49,869][105620] Updated weights for policy 1, policy_version 1673918 (0.0009) [2023-12-27 03:33:50,452][105692] Updated weights for policy 0, policy_version 1670497 (0.0009) [2023-12-27 03:33:50,515][105692] Updated weights for policy 0, policy_version 1670508 (0.0009) [2023-12-27 03:33:50,550][105620] Updated weights for policy 1, policy_version 1673928 (0.0006) [2023-12-27 03:33:50,583][105692] Updated weights for policy 0, policy_version 1670518 (0.0006) [2023-12-27 03:33:50,615][105620] Updated weights for policy 1, policy_version 1673938 (0.0009) [2023-12-27 03:33:50,656][105692] Updated weights for policy 0, policy_version 1670528 (0.0006) [2023-12-27 03:33:50,683][105620] Updated weights for policy 1, policy_version 1673948 (0.0011) [2023-12-27 03:33:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 856309760. Throughput: 0: 9801.1, 1: 9766.4. Samples: 856299184. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:33:51,062][104569] Avg episode reward: [(0, '8626.119'), (1, '8627.769')] [2023-12-27 03:33:51,326][105620] Updated weights for policy 1, policy_version 1673958 (0.0011) [2023-12-27 03:33:51,387][105692] Updated weights for policy 0, policy_version 1670538 (0.0008) [2023-12-27 03:33:51,394][105620] Updated weights for policy 1, policy_version 1673968 (0.0011) [2023-12-27 03:33:51,448][105692] Updated weights for policy 0, policy_version 1670548 (0.0009) [2023-12-27 03:33:51,454][105620] Updated weights for policy 1, policy_version 1673978 (0.0009) [2023-12-27 03:33:51,506][105692] Updated weights for policy 0, policy_version 1670558 (0.0008) [2023-12-27 03:33:52,150][105620] Updated weights for policy 1, policy_version 1673988 (0.0009) [2023-12-27 03:33:52,203][105620] Updated weights for policy 1, policy_version 1673998 (0.0010) [2023-12-27 03:33:52,249][105620] Updated weights for policy 1, policy_version 1674008 (0.0010) [2023-12-27 03:33:52,305][105692] Updated weights for policy 0, policy_version 1670568 (0.0008) [2023-12-27 03:33:52,361][105692] Updated weights for policy 0, policy_version 1670578 (0.0008) [2023-12-27 03:33:52,422][105692] Updated weights for policy 0, policy_version 1670588 (0.0008) [2023-12-27 03:33:52,991][105620] Updated weights for policy 1, policy_version 1674018 (0.0008) [2023-12-27 03:33:53,051][105620] Updated weights for policy 1, policy_version 1674028 (0.0005) [2023-12-27 03:33:53,117][105620] Updated weights for policy 1, policy_version 1674038 (0.0006) [2023-12-27 03:33:53,182][105620] Updated weights for policy 1, policy_version 1674048 (0.0010) [2023-12-27 03:33:53,208][105692] Updated weights for policy 0, policy_version 1670598 (0.0008) [2023-12-27 03:33:53,260][105692] Updated weights for policy 0, policy_version 1670608 (0.0008) [2023-12-27 03:33:53,309][105692] Updated weights for policy 0, policy_version 1670618 (0.0008) [2023-12-27 03:33:53,830][105620] Updated weights for policy 1, policy_version 1674058 (0.0005) [2023-12-27 03:33:53,893][105620] Updated weights for policy 1, policy_version 1674068 (0.0005) [2023-12-27 03:33:53,954][105620] Updated weights for policy 1, policy_version 1674078 (0.0005) [2023-12-27 03:33:54,133][105692] Updated weights for policy 0, policy_version 1670628 (0.0008) [2023-12-27 03:33:54,196][105692] Updated weights for policy 0, policy_version 1670638 (0.0008) [2023-12-27 03:33:54,256][105692] Updated weights for policy 0, policy_version 1670648 (0.0009) [2023-12-27 03:33:54,597][105620] Updated weights for policy 1, policy_version 1674088 (0.0008) [2023-12-27 03:33:54,644][105620] Updated weights for policy 1, policy_version 1674098 (0.0009) [2023-12-27 03:33:54,691][105620] Updated weights for policy 1, policy_version 1674108 (0.0009) [2023-12-27 03:33:55,001][105692] Updated weights for policy 0, policy_version 1670658 (0.0008) [2023-12-27 03:33:55,057][105692] Updated weights for policy 0, policy_version 1670668 (0.0005) [2023-12-27 03:33:55,118][105692] Updated weights for policy 0, policy_version 1670678 (0.0010) [2023-12-27 03:33:55,184][105692] Updated weights for policy 0, policy_version 1670688 (0.0011) [2023-12-27 03:33:55,493][105620] Updated weights for policy 1, policy_version 1674118 (0.0008) [2023-12-27 03:33:55,544][105620] Updated weights for policy 1, policy_version 1674128 (0.0007) [2023-12-27 03:33:55,596][105620] Updated weights for policy 1, policy_version 1674138 (0.0008) [2023-12-27 03:33:55,879][105692] Updated weights for policy 0, policy_version 1670698 (0.0008) [2023-12-27 03:33:55,943][105692] Updated weights for policy 0, policy_version 1670708 (0.0008) [2023-12-27 03:33:56,005][105692] Updated weights for policy 0, policy_version 1670718 (0.0008) [2023-12-27 03:33:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 856408064. Throughput: 0: 9729.6, 1: 9688.4. Samples: 856413740. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:33:56,062][104569] Avg episode reward: [(0, '8718.069'), (1, '8806.720')] [2023-12-27 03:33:56,364][105620] Updated weights for policy 1, policy_version 1674148 (0.0009) [2023-12-27 03:33:56,409][105620] Updated weights for policy 1, policy_version 1674158 (0.0010) [2023-12-27 03:33:56,466][105620] Updated weights for policy 1, policy_version 1674168 (0.0011) [2023-12-27 03:33:56,787][105692] Updated weights for policy 0, policy_version 1670728 (0.0008) [2023-12-27 03:33:56,849][105692] Updated weights for policy 0, policy_version 1670738 (0.0008) [2023-12-27 03:33:56,914][105692] Updated weights for policy 0, policy_version 1670748 (0.0008) [2023-12-27 03:33:57,233][105620] Updated weights for policy 1, policy_version 1674178 (0.0011) [2023-12-27 03:33:57,278][105620] Updated weights for policy 1, policy_version 1674188 (0.0010) [2023-12-27 03:33:57,334][105620] Updated weights for policy 1, policy_version 1674198 (0.0011) [2023-12-27 03:33:57,386][105620] Updated weights for policy 1, policy_version 1674208 (0.0011) [2023-12-27 03:33:57,688][105692] Updated weights for policy 0, policy_version 1670758 (0.0009) [2023-12-27 03:33:57,756][105692] Updated weights for policy 0, policy_version 1670768 (0.0010) [2023-12-27 03:33:57,814][105692] Updated weights for policy 0, policy_version 1670778 (0.0010) [2023-12-27 03:33:57,992][105620] Updated weights for policy 1, policy_version 1674218 (0.0005) [2023-12-27 03:33:58,041][105620] Updated weights for policy 1, policy_version 1674228 (0.0006) [2023-12-27 03:33:58,091][105620] Updated weights for policy 1, policy_version 1674238 (0.0006) [2023-12-27 03:33:58,556][105692] Updated weights for policy 0, policy_version 1670788 (0.0009) [2023-12-27 03:33:58,616][105692] Updated weights for policy 0, policy_version 1670798 (0.0007) [2023-12-27 03:33:58,680][105692] Updated weights for policy 0, policy_version 1670808 (0.0009) [2023-12-27 03:33:58,833][105620] Updated weights for policy 1, policy_version 1674248 (0.0008) [2023-12-27 03:33:58,899][105620] Updated weights for policy 1, policy_version 1674258 (0.0009) [2023-12-27 03:33:58,964][105620] Updated weights for policy 1, policy_version 1674268 (0.0008) [2023-12-27 03:33:59,325][105692] Updated weights for policy 0, policy_version 1670818 (0.0009) [2023-12-27 03:33:59,390][105692] Updated weights for policy 0, policy_version 1670828 (0.0008) [2023-12-27 03:33:59,445][105692] Updated weights for policy 0, policy_version 1670838 (0.0007) [2023-12-27 03:33:59,680][105620] Updated weights for policy 1, policy_version 1674278 (0.0009) [2023-12-27 03:33:59,736][105620] Updated weights for policy 1, policy_version 1674288 (0.0009) [2023-12-27 03:33:59,794][105620] Updated weights for policy 1, policy_version 1674298 (0.0010) [2023-12-27 03:34:00,112][105692] Updated weights for policy 0, policy_version 1670849 (0.0009) [2023-12-27 03:34:00,159][105692] Updated weights for policy 0, policy_version 1670859 (0.0005) [2023-12-27 03:34:00,218][105692] Updated weights for policy 0, policy_version 1670870 (0.0010) [2023-12-27 03:34:00,274][105692] Updated weights for policy 0, policy_version 1670880 (0.0009) [2023-12-27 03:34:00,465][105620] Updated weights for policy 1, policy_version 1674308 (0.0008) [2023-12-27 03:34:00,531][105620] Updated weights for policy 1, policy_version 1674318 (0.0006) [2023-12-27 03:34:00,584][105620] Updated weights for policy 1, policy_version 1674328 (0.0006) [2023-12-27 03:34:00,919][105692] Updated weights for policy 0, policy_version 1670890 (0.0005) [2023-12-27 03:34:00,977][105692] Updated weights for policy 0, policy_version 1670900 (0.0005) [2023-12-27 03:34:01,034][105692] Updated weights for policy 0, policy_version 1670910 (0.0006) [2023-12-27 03:34:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 856506368. Throughput: 0: 9747.4, 1: 9715.4. Samples: 856471240. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:34:01,063][104569] Avg episode reward: [(0, '8719.603'), (1, '8988.180')] [2023-12-27 03:34:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001670912_427819008.pth... [2023-12-27 03:34:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001674336_428687360.pth... [2023-12-27 03:34:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001673216_428400640.pth [2023-12-27 03:34:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001669760_427524096.pth [2023-12-27 03:34:01,199][105620] Updated weights for policy 1, policy_version 1674338 (0.0006) [2023-12-27 03:34:01,270][105620] Updated weights for policy 1, policy_version 1674348 (0.0008) [2023-12-27 03:34:01,341][105620] Updated weights for policy 1, policy_version 1674358 (0.0008) [2023-12-27 03:34:01,405][105620] Updated weights for policy 1, policy_version 1674368 (0.0009) [2023-12-27 03:34:01,687][105692] Updated weights for policy 0, policy_version 1670920 (0.0008) [2023-12-27 03:34:01,756][105692] Updated weights for policy 0, policy_version 1670930 (0.0009) [2023-12-27 03:34:01,814][105692] Updated weights for policy 0, policy_version 1670940 (0.0008) [2023-12-27 03:34:02,170][105620] Updated weights for policy 1, policy_version 1674378 (0.0009) [2023-12-27 03:34:02,223][105620] Updated weights for policy 1, policy_version 1674388 (0.0009) [2023-12-27 03:34:02,282][105620] Updated weights for policy 1, policy_version 1674398 (0.0008) [2023-12-27 03:34:02,525][105692] Updated weights for policy 0, policy_version 1670950 (0.0009) [2023-12-27 03:34:02,587][105692] Updated weights for policy 0, policy_version 1670960 (0.0007) [2023-12-27 03:34:02,648][105692] Updated weights for policy 0, policy_version 1670970 (0.0005) [2023-12-27 03:34:03,095][105620] Updated weights for policy 1, policy_version 1674408 (0.0006) [2023-12-27 03:34:03,151][105620] Updated weights for policy 1, policy_version 1674418 (0.0005) [2023-12-27 03:34:03,214][105620] Updated weights for policy 1, policy_version 1674428 (0.0009) [2023-12-27 03:34:03,283][105692] Updated weights for policy 0, policy_version 1670980 (0.0007) [2023-12-27 03:34:03,345][105692] Updated weights for policy 0, policy_version 1670990 (0.0008) [2023-12-27 03:34:03,406][105692] Updated weights for policy 0, policy_version 1671000 (0.0006) [2023-12-27 03:34:03,932][105620] Updated weights for policy 1, policy_version 1674438 (0.0007) [2023-12-27 03:34:03,981][105692] Updated weights for policy 0, policy_version 1671010 (0.0006) [2023-12-27 03:34:04,001][105620] Updated weights for policy 1, policy_version 1674448 (0.0006) [2023-12-27 03:34:04,037][105692] Updated weights for policy 0, policy_version 1671020 (0.0008) [2023-12-27 03:34:04,062][105620] Updated weights for policy 1, policy_version 1674458 (0.0006) [2023-12-27 03:34:04,094][105692] Updated weights for policy 0, policy_version 1671030 (0.0008) [2023-12-27 03:34:04,162][105692] Updated weights for policy 0, policy_version 1671040 (0.0010) [2023-12-27 03:34:04,678][105620] Updated weights for policy 1, policy_version 1674468 (0.0007) [2023-12-27 03:34:04,723][105620] Updated weights for policy 1, policy_version 1674478 (0.0008) [2023-12-27 03:34:04,773][105620] Updated weights for policy 1, policy_version 1674488 (0.0008) [2023-12-27 03:34:04,965][105692] Updated weights for policy 0, policy_version 1671050 (0.0009) [2023-12-27 03:34:05,024][105692] Updated weights for policy 0, policy_version 1671060 (0.0009) [2023-12-27 03:34:05,084][105692] Updated weights for policy 0, policy_version 1671070 (0.0009) [2023-12-27 03:34:05,548][105620] Updated weights for policy 1, policy_version 1674498 (0.0009) [2023-12-27 03:34:05,611][105620] Updated weights for policy 1, policy_version 1674508 (0.0009) [2023-12-27 03:34:05,674][105620] Updated weights for policy 1, policy_version 1674518 (0.0009) [2023-12-27 03:34:05,734][105620] Updated weights for policy 1, policy_version 1674528 (0.0009) [2023-12-27 03:34:05,754][105692] Updated weights for policy 0, policy_version 1671080 (0.0006) [2023-12-27 03:34:05,802][105692] Updated weights for policy 0, policy_version 1671090 (0.0005) [2023-12-27 03:34:05,848][105692] Updated weights for policy 0, policy_version 1671100 (0.0005) [2023-12-27 03:34:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 856604672. Throughput: 0: 9809.9, 1: 9687.1. Samples: 856591280. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:34:06,062][104569] Avg episode reward: [(0, '8440.633'), (1, '8896.537')] [2023-12-27 03:34:06,473][105692] Updated weights for policy 0, policy_version 1671110 (0.0009) [2023-12-27 03:34:06,536][105692] Updated weights for policy 0, policy_version 1671120 (0.0011) [2023-12-27 03:34:06,582][105620] Updated weights for policy 1, policy_version 1674538 (0.0006) [2023-12-27 03:34:06,589][105692] Updated weights for policy 0, policy_version 1671130 (0.0011) [2023-12-27 03:34:06,643][105620] Updated weights for policy 1, policy_version 1674548 (0.0006) [2023-12-27 03:34:06,709][105620] Updated weights for policy 1, policy_version 1674558 (0.0009) [2023-12-27 03:34:07,285][105620] Updated weights for policy 1, policy_version 1674568 (0.0009) [2023-12-27 03:34:07,330][105692] Updated weights for policy 0, policy_version 1671140 (0.0009) [2023-12-27 03:34:07,349][105620] Updated weights for policy 1, policy_version 1674578 (0.0008) [2023-12-27 03:34:07,393][105692] Updated weights for policy 0, policy_version 1671150 (0.0008) [2023-12-27 03:34:07,398][105620] Updated weights for policy 1, policy_version 1674588 (0.0009) [2023-12-27 03:34:07,449][105692] Updated weights for policy 0, policy_version 1671160 (0.0009) [2023-12-27 03:34:08,093][105620] Updated weights for policy 1, policy_version 1674598 (0.0005) [2023-12-27 03:34:08,113][105692] Updated weights for policy 0, policy_version 1671170 (0.0008) [2023-12-27 03:34:08,145][105620] Updated weights for policy 1, policy_version 1674608 (0.0005) [2023-12-27 03:34:08,166][105692] Updated weights for policy 0, policy_version 1671180 (0.0009) [2023-12-27 03:34:08,194][105620] Updated weights for policy 1, policy_version 1674618 (0.0007) [2023-12-27 03:34:08,217][105692] Updated weights for policy 0, policy_version 1671190 (0.0008) [2023-12-27 03:34:08,272][105692] Updated weights for policy 0, policy_version 1671200 (0.0010) [2023-12-27 03:34:08,789][105620] Updated weights for policy 1, policy_version 1674628 (0.0007) [2023-12-27 03:34:08,845][105620] Updated weights for policy 1, policy_version 1674638 (0.0008) [2023-12-27 03:34:08,896][105620] Updated weights for policy 1, policy_version 1674648 (0.0007) [2023-12-27 03:34:09,001][105692] Updated weights for policy 0, policy_version 1671210 (0.0010) [2023-12-27 03:34:09,059][105692] Updated weights for policy 0, policy_version 1671220 (0.0010) [2023-12-27 03:34:09,108][105692] Updated weights for policy 0, policy_version 1671230 (0.0006) [2023-12-27 03:34:09,718][105620] Updated weights for policy 1, policy_version 1674658 (0.0009) [2023-12-27 03:34:09,776][105620] Updated weights for policy 1, policy_version 1674668 (0.0009) [2023-12-27 03:34:09,839][105620] Updated weights for policy 1, policy_version 1674678 (0.0008) [2023-12-27 03:34:09,864][105692] Updated weights for policy 0, policy_version 1671240 (0.0008) [2023-12-27 03:34:09,899][105620] Updated weights for policy 1, policy_version 1674688 (0.0008) [2023-12-27 03:34:09,921][105692] Updated weights for policy 0, policy_version 1671250 (0.0007) [2023-12-27 03:34:09,986][105692] Updated weights for policy 0, policy_version 1671260 (0.0008) [2023-12-27 03:34:10,662][105620] Updated weights for policy 1, policy_version 1674698 (0.0009) [2023-12-27 03:34:10,722][105620] Updated weights for policy 1, policy_version 1674708 (0.0009) [2023-12-27 03:34:10,744][105692] Updated weights for policy 0, policy_version 1671270 (0.0007) [2023-12-27 03:34:10,781][105620] Updated weights for policy 1, policy_version 1674718 (0.0005) [2023-12-27 03:34:10,794][105692] Updated weights for policy 0, policy_version 1671280 (0.0005) [2023-12-27 03:34:10,845][105692] Updated weights for policy 0, policy_version 1671290 (0.0005) [2023-12-27 03:34:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 856702976. Throughput: 0: 9853.6, 1: 9700.9. Samples: 856708996. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:34:11,062][104569] Avg episode reward: [(0, '8805.229'), (1, '8530.402')] [2023-12-27 03:34:11,495][105620] Updated weights for policy 1, policy_version 1674728 (0.0010) [2023-12-27 03:34:11,548][105620] Updated weights for policy 1, policy_version 1674738 (0.0011) [2023-12-27 03:34:11,603][105692] Updated weights for policy 0, policy_version 1671300 (0.0007) [2023-12-27 03:34:11,605][105620] Updated weights for policy 1, policy_version 1674748 (0.0010) [2023-12-27 03:34:11,668][105692] Updated weights for policy 0, policy_version 1671310 (0.0009) [2023-12-27 03:34:11,738][105692] Updated weights for policy 0, policy_version 1671320 (0.0008) [2023-12-27 03:34:12,414][105620] Updated weights for policy 1, policy_version 1674758 (0.0008) [2023-12-27 03:34:12,479][105620] Updated weights for policy 1, policy_version 1674768 (0.0008) [2023-12-27 03:34:12,505][105692] Updated weights for policy 0, policy_version 1671330 (0.0008) [2023-12-27 03:34:12,540][105620] Updated weights for policy 1, policy_version 1674778 (0.0009) [2023-12-27 03:34:12,566][105692] Updated weights for policy 0, policy_version 1671340 (0.0009) [2023-12-27 03:34:12,627][105692] Updated weights for policy 0, policy_version 1671350 (0.0008) [2023-12-27 03:34:12,689][105692] Updated weights for policy 0, policy_version 1671360 (0.0006) [2023-12-27 03:34:13,178][105620] Updated weights for policy 1, policy_version 1674788 (0.0006) [2023-12-27 03:34:13,247][105620] Updated weights for policy 1, policy_version 1674798 (0.0006) [2023-12-27 03:34:13,316][105620] Updated weights for policy 1, policy_version 1674808 (0.0006) [2023-12-27 03:34:13,354][105692] Updated weights for policy 0, policy_version 1671370 (0.0005) [2023-12-27 03:34:13,411][105692] Updated weights for policy 0, policy_version 1671380 (0.0006) [2023-12-27 03:34:13,466][105692] Updated weights for policy 0, policy_version 1671390 (0.0008) [2023-12-27 03:34:13,901][105620] Updated weights for policy 1, policy_version 1674818 (0.0008) [2023-12-27 03:34:13,950][105620] Updated weights for policy 1, policy_version 1674828 (0.0005) [2023-12-27 03:34:14,003][105620] Updated weights for policy 1, policy_version 1674838 (0.0005) [2023-12-27 03:34:14,056][105620] Updated weights for policy 1, policy_version 1674848 (0.0005) [2023-12-27 03:34:14,135][105692] Updated weights for policy 0, policy_version 1671400 (0.0010) [2023-12-27 03:34:14,194][105692] Updated weights for policy 0, policy_version 1671410 (0.0011) [2023-12-27 03:34:14,249][105692] Updated weights for policy 0, policy_version 1671420 (0.0011) [2023-12-27 03:34:14,716][105620] Updated weights for policy 1, policy_version 1674858 (0.0008) [2023-12-27 03:34:14,770][105620] Updated weights for policy 1, policy_version 1674868 (0.0008) [2023-12-27 03:34:14,827][105620] Updated weights for policy 1, policy_version 1674878 (0.0008) [2023-12-27 03:34:14,996][105692] Updated weights for policy 0, policy_version 1671430 (0.0010) [2023-12-27 03:34:15,053][105692] Updated weights for policy 0, policy_version 1671440 (0.0011) [2023-12-27 03:34:15,115][105692] Updated weights for policy 0, policy_version 1671450 (0.0010) [2023-12-27 03:34:15,564][105620] Updated weights for policy 1, policy_version 1674888 (0.0008) [2023-12-27 03:34:15,624][105620] Updated weights for policy 1, policy_version 1674898 (0.0008) [2023-12-27 03:34:15,684][105620] Updated weights for policy 1, policy_version 1674908 (0.0008) [2023-12-27 03:34:15,885][105692] Updated weights for policy 0, policy_version 1671460 (0.0011) [2023-12-27 03:34:15,933][105692] Updated weights for policy 0, policy_version 1671470 (0.0010) [2023-12-27 03:34:15,984][105692] Updated weights for policy 0, policy_version 1671480 (0.0010) [2023-12-27 03:34:16,062][104569] Fps is (10 sec: 19660.0, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 856801280. Throughput: 0: 9777.3, 1: 9731.7. Samples: 856767780. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:34:16,063][104569] Avg episode reward: [(0, '9078.876'), (1, '8716.082')] [2023-12-27 03:34:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001671488_427966464.pth... [2023-12-27 03:34:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001674912_428834816.pth... [2023-12-27 03:34:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001670368_427679744.pth [2023-12-27 03:34:16,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001673760_428539904.pth [2023-12-27 03:34:16,327][105620] Updated weights for policy 1, policy_version 1674918 (0.0008) [2023-12-27 03:34:16,398][105620] Updated weights for policy 1, policy_version 1674928 (0.0009) [2023-12-27 03:34:16,465][105620] Updated weights for policy 1, policy_version 1674938 (0.0010) [2023-12-27 03:34:16,614][105692] Updated weights for policy 0, policy_version 1671490 (0.0009) [2023-12-27 03:34:16,670][105692] Updated weights for policy 0, policy_version 1671500 (0.0005) [2023-12-27 03:34:16,728][105692] Updated weights for policy 0, policy_version 1671510 (0.0007) [2023-12-27 03:34:16,779][105692] Updated weights for policy 0, policy_version 1671520 (0.0009) [2023-12-27 03:34:17,234][105620] Updated weights for policy 1, policy_version 1674948 (0.0009) [2023-12-27 03:34:17,304][105620] Updated weights for policy 1, policy_version 1674958 (0.0009) [2023-12-27 03:34:17,375][105620] Updated weights for policy 1, policy_version 1674968 (0.0009) [2023-12-27 03:34:17,459][105692] Updated weights for policy 0, policy_version 1671530 (0.0006) [2023-12-27 03:34:17,526][105692] Updated weights for policy 0, policy_version 1671540 (0.0006) [2023-12-27 03:34:17,588][105692] Updated weights for policy 0, policy_version 1671550 (0.0008) [2023-12-27 03:34:18,076][105620] Updated weights for policy 1, policy_version 1674978 (0.0009) [2023-12-27 03:34:18,127][105620] Updated weights for policy 1, policy_version 1674988 (0.0009) [2023-12-27 03:34:18,178][105692] Updated weights for policy 0, policy_version 1671560 (0.0009) [2023-12-27 03:34:18,184][105620] Updated weights for policy 1, policy_version 1674998 (0.0006) [2023-12-27 03:34:18,241][105692] Updated weights for policy 0, policy_version 1671570 (0.0009) [2023-12-27 03:34:18,245][105620] Updated weights for policy 1, policy_version 1675008 (0.0006) [2023-12-27 03:34:18,299][105692] Updated weights for policy 0, policy_version 1671580 (0.0006) [2023-12-27 03:34:18,989][105692] Updated weights for policy 0, policy_version 1671590 (0.0009) [2023-12-27 03:34:19,012][105620] Updated weights for policy 1, policy_version 1675018 (0.0010) [2023-12-27 03:34:19,050][105692] Updated weights for policy 0, policy_version 1671600 (0.0009) [2023-12-27 03:34:19,072][105620] Updated weights for policy 1, policy_version 1675028 (0.0007) [2023-12-27 03:34:19,111][105692] Updated weights for policy 0, policy_version 1671610 (0.0007) [2023-12-27 03:34:19,133][105620] Updated weights for policy 1, policy_version 1675038 (0.0007) [2023-12-27 03:34:19,853][105692] Updated weights for policy 0, policy_version 1671620 (0.0008) [2023-12-27 03:34:19,913][105692] Updated weights for policy 0, policy_version 1671630 (0.0007) [2023-12-27 03:34:19,940][105620] Updated weights for policy 1, policy_version 1675048 (0.0008) [2023-12-27 03:34:19,984][105692] Updated weights for policy 0, policy_version 1671640 (0.0007) [2023-12-27 03:34:20,002][105620] Updated weights for policy 1, policy_version 1675058 (0.0007) [2023-12-27 03:34:20,062][105620] Updated weights for policy 1, policy_version 1675068 (0.0009) [2023-12-27 03:34:20,598][105692] Updated weights for policy 0, policy_version 1671650 (0.0007) [2023-12-27 03:34:20,665][105692] Updated weights for policy 0, policy_version 1671660 (0.0006) [2023-12-27 03:34:20,738][105692] Updated weights for policy 0, policy_version 1671670 (0.0010) [2023-12-27 03:34:20,807][105692] Updated weights for policy 0, policy_version 1671680 (0.0008) [2023-12-27 03:34:20,830][105620] Updated weights for policy 1, policy_version 1675078 (0.0009) [2023-12-27 03:34:20,895][105620] Updated weights for policy 1, policy_version 1675088 (0.0008) [2023-12-27 03:34:20,961][105620] Updated weights for policy 1, policy_version 1675098 (0.0008) [2023-12-27 03:34:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 856899584. Throughput: 0: 9751.4, 1: 9696.3. Samples: 856885308. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:34:21,063][104569] Avg episode reward: [(0, '8348.173'), (1, '8901.364')] [2023-12-27 03:34:21,543][105692] Updated weights for policy 0, policy_version 1671690 (0.0009) [2023-12-27 03:34:21,605][105692] Updated weights for policy 0, policy_version 1671700 (0.0009) [2023-12-27 03:34:21,671][105692] Updated weights for policy 0, policy_version 1671710 (0.0008) [2023-12-27 03:34:21,725][105620] Updated weights for policy 1, policy_version 1675108 (0.0009) [2023-12-27 03:34:21,796][105620] Updated weights for policy 1, policy_version 1675118 (0.0009) [2023-12-27 03:34:21,858][105620] Updated weights for policy 1, policy_version 1675128 (0.0009) [2023-12-27 03:34:22,539][105692] Updated weights for policy 0, policy_version 1671720 (0.0008) [2023-12-27 03:34:22,580][105620] Updated weights for policy 1, policy_version 1675138 (0.0009) [2023-12-27 03:34:22,594][105692] Updated weights for policy 0, policy_version 1671730 (0.0008) [2023-12-27 03:34:22,639][105620] Updated weights for policy 1, policy_version 1675148 (0.0011) [2023-12-27 03:34:22,649][105692] Updated weights for policy 0, policy_version 1671740 (0.0006) [2023-12-27 03:34:22,698][105620] Updated weights for policy 1, policy_version 1675158 (0.0008) [2023-12-27 03:34:22,763][105620] Updated weights for policy 1, policy_version 1675168 (0.0005) [2023-12-27 03:34:23,368][105620] Updated weights for policy 1, policy_version 1675178 (0.0008) [2023-12-27 03:34:23,441][105620] Updated weights for policy 1, policy_version 1675188 (0.0005) [2023-12-27 03:34:23,501][105620] Updated weights for policy 1, policy_version 1675198 (0.0006) [2023-12-27 03:34:23,513][105692] Updated weights for policy 0, policy_version 1671750 (0.0008) [2023-12-27 03:34:23,568][105692] Updated weights for policy 0, policy_version 1671760 (0.0009) [2023-12-27 03:34:23,626][105692] Updated weights for policy 0, policy_version 1671770 (0.0010) [2023-12-27 03:34:23,999][105620] Updated weights for policy 1, policy_version 1675208 (0.0005) [2023-12-27 03:34:24,047][105620] Updated weights for policy 1, policy_version 1675218 (0.0005) [2023-12-27 03:34:24,094][105620] Updated weights for policy 1, policy_version 1675228 (0.0005) [2023-12-27 03:34:24,551][105692] Updated weights for policy 0, policy_version 1671780 (0.0010) [2023-12-27 03:34:24,600][105692] Updated weights for policy 0, policy_version 1671790 (0.0008) [2023-12-27 03:34:24,652][105692] Updated weights for policy 0, policy_version 1671800 (0.0009) [2023-12-27 03:34:24,680][105620] Updated weights for policy 1, policy_version 1675238 (0.0007) [2023-12-27 03:34:24,739][105620] Updated weights for policy 1, policy_version 1675248 (0.0008) [2023-12-27 03:34:24,800][105620] Updated weights for policy 1, policy_version 1675258 (0.0009) [2023-12-27 03:34:25,301][105692] Updated weights for policy 0, policy_version 1671810 (0.0007) [2023-12-27 03:34:25,352][105692] Updated weights for policy 0, policy_version 1671820 (0.0005) [2023-12-27 03:34:25,412][105692] Updated weights for policy 0, policy_version 1671830 (0.0005) [2023-12-27 03:34:25,459][105692] Updated weights for policy 0, policy_version 1671840 (0.0008) [2023-12-27 03:34:25,620][105620] Updated weights for policy 1, policy_version 1675268 (0.0009) [2023-12-27 03:34:25,665][105620] Updated weights for policy 1, policy_version 1675278 (0.0007) [2023-12-27 03:34:25,716][105620] Updated weights for policy 1, policy_version 1675288 (0.0006) [2023-12-27 03:34:26,062][104569] Fps is (10 sec: 18842.4, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 856989696. Throughput: 0: 9585.5, 1: 9780.5. Samples: 857000264. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:34:26,062][104569] Avg episode reward: [(0, '8257.661'), (1, '8897.833')] [2023-12-27 03:34:26,142][105692] Updated weights for policy 0, policy_version 1671850 (0.0011) [2023-12-27 03:34:26,194][105692] Updated weights for policy 0, policy_version 1671860 (0.0010) [2023-12-27 03:34:26,242][105692] Updated weights for policy 0, policy_version 1671870 (0.0010) [2023-12-27 03:34:26,406][105620] Updated weights for policy 1, policy_version 1675298 (0.0008) [2023-12-27 03:34:26,473][105620] Updated weights for policy 1, policy_version 1675308 (0.0009) [2023-12-27 03:34:26,532][105620] Updated weights for policy 1, policy_version 1675318 (0.0008) [2023-12-27 03:34:26,590][105620] Updated weights for policy 1, policy_version 1675328 (0.0005) [2023-12-27 03:34:26,915][105692] Updated weights for policy 0, policy_version 1671880 (0.0010) [2023-12-27 03:34:26,977][105692] Updated weights for policy 0, policy_version 1671890 (0.0011) [2023-12-27 03:34:27,035][105692] Updated weights for policy 0, policy_version 1671900 (0.0010) [2023-12-27 03:34:27,264][105620] Updated weights for policy 1, policy_version 1675338 (0.0008) [2023-12-27 03:34:27,323][105620] Updated weights for policy 1, policy_version 1675348 (0.0007) [2023-12-27 03:34:27,384][105620] Updated weights for policy 1, policy_version 1675358 (0.0007) [2023-12-27 03:34:27,759][105692] Updated weights for policy 0, policy_version 1671910 (0.0010) [2023-12-27 03:34:27,820][105692] Updated weights for policy 0, policy_version 1671920 (0.0010) [2023-12-27 03:34:27,875][105692] Updated weights for policy 0, policy_version 1671930 (0.0010) [2023-12-27 03:34:27,940][105620] Updated weights for policy 1, policy_version 1675368 (0.0005) [2023-12-27 03:34:27,988][105620] Updated weights for policy 1, policy_version 1675378 (0.0005) [2023-12-27 03:34:28,040][105620] Updated weights for policy 1, policy_version 1675388 (0.0007) [2023-12-27 03:34:28,620][105692] Updated weights for policy 0, policy_version 1671940 (0.0012) [2023-12-27 03:34:28,672][105692] Updated weights for policy 0, policy_version 1671950 (0.0010) [2023-12-27 03:34:28,707][105620] Updated weights for policy 1, policy_version 1675398 (0.0006) [2023-12-27 03:34:28,717][105692] Updated weights for policy 0, policy_version 1671960 (0.0009) [2023-12-27 03:34:28,761][105620] Updated weights for policy 1, policy_version 1675408 (0.0008) [2023-12-27 03:34:28,813][105620] Updated weights for policy 1, policy_version 1675418 (0.0008) [2023-12-27 03:34:29,468][105692] Updated weights for policy 0, policy_version 1671970 (0.0007) [2023-12-27 03:34:29,513][105692] Updated weights for policy 0, policy_version 1671980 (0.0010) [2023-12-27 03:34:29,565][105692] Updated weights for policy 0, policy_version 1671990 (0.0010) [2023-12-27 03:34:29,589][105620] Updated weights for policy 1, policy_version 1675428 (0.0009) [2023-12-27 03:34:29,623][105692] Updated weights for policy 0, policy_version 1672000 (0.0010) [2023-12-27 03:34:29,642][105620] Updated weights for policy 1, policy_version 1675438 (0.0007) [2023-12-27 03:34:29,703][105620] Updated weights for policy 1, policy_version 1675448 (0.0008) [2023-12-27 03:34:30,374][105692] Updated weights for policy 0, policy_version 1672010 (0.0010) [2023-12-27 03:34:30,422][105692] Updated weights for policy 0, policy_version 1672020 (0.0010) [2023-12-27 03:34:30,444][105620] Updated weights for policy 1, policy_version 1675458 (0.0008) [2023-12-27 03:34:30,474][105692] Updated weights for policy 0, policy_version 1672030 (0.0010) [2023-12-27 03:34:30,497][105620] Updated weights for policy 1, policy_version 1675468 (0.0006) [2023-12-27 03:34:30,549][105620] Updated weights for policy 1, policy_version 1675478 (0.0008) [2023-12-27 03:34:30,600][105620] Updated weights for policy 1, policy_version 1675488 (0.0008) [2023-12-27 03:34:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 857088000. Throughput: 0: 9592.9, 1: 9863.9. Samples: 857060756. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:34:31,062][104569] Avg episode reward: [(0, '8711.546'), (1, '8988.086')] [2023-12-27 03:34:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001675488_428982272.pth... [2023-12-27 03:34:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001672032_428105728.pth... [2023-12-27 03:34:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001674336_428687360.pth [2023-12-27 03:34:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001670912_427819008.pth [2023-12-27 03:34:31,181][105692] Updated weights for policy 0, policy_version 1672040 (0.0007) [2023-12-27 03:34:31,240][105692] Updated weights for policy 0, policy_version 1672050 (0.0008) [2023-12-27 03:34:31,291][105692] Updated weights for policy 0, policy_version 1672060 (0.0010) [2023-12-27 03:34:31,430][105620] Updated weights for policy 1, policy_version 1675498 (0.0009) [2023-12-27 03:34:31,486][105620] Updated weights for policy 1, policy_version 1675508 (0.0011) [2023-12-27 03:34:31,541][105620] Updated weights for policy 1, policy_version 1675518 (0.0010) [2023-12-27 03:34:32,094][105692] Updated weights for policy 0, policy_version 1672070 (0.0007) [2023-12-27 03:34:32,154][105692] Updated weights for policy 0, policy_version 1672080 (0.0008) [2023-12-27 03:34:32,201][105692] Updated weights for policy 0, policy_version 1672090 (0.0009) [2023-12-27 03:34:32,290][105620] Updated weights for policy 1, policy_version 1675528 (0.0009) [2023-12-27 03:34:32,355][105620] Updated weights for policy 1, policy_version 1675538 (0.0009) [2023-12-27 03:34:32,423][105620] Updated weights for policy 1, policy_version 1675548 (0.0007) [2023-12-27 03:34:32,897][105692] Updated weights for policy 0, policy_version 1672100 (0.0008) [2023-12-27 03:34:32,962][105692] Updated weights for policy 0, policy_version 1672110 (0.0008) [2023-12-27 03:34:33,027][105692] Updated weights for policy 0, policy_version 1672120 (0.0008) [2023-12-27 03:34:33,117][105620] Updated weights for policy 1, policy_version 1675558 (0.0008) [2023-12-27 03:34:33,166][105620] Updated weights for policy 1, policy_version 1675568 (0.0008) [2023-12-27 03:34:33,226][105620] Updated weights for policy 1, policy_version 1675578 (0.0009) [2023-12-27 03:34:33,750][105692] Updated weights for policy 0, policy_version 1672130 (0.0008) [2023-12-27 03:34:33,797][105692] Updated weights for policy 0, policy_version 1672140 (0.0009) [2023-12-27 03:34:33,847][105692] Updated weights for policy 0, policy_version 1672150 (0.0009) [2023-12-27 03:34:33,895][105692] Updated weights for policy 0, policy_version 1672160 (0.0008) [2023-12-27 03:34:33,972][105620] Updated weights for policy 1, policy_version 1675588 (0.0007) [2023-12-27 03:34:34,023][105620] Updated weights for policy 1, policy_version 1675598 (0.0005) [2023-12-27 03:34:34,081][105620] Updated weights for policy 1, policy_version 1675608 (0.0007) [2023-12-27 03:34:34,677][105692] Updated weights for policy 0, policy_version 1672170 (0.0011) [2023-12-27 03:34:34,743][105692] Updated weights for policy 0, policy_version 1672180 (0.0011) [2023-12-27 03:34:34,802][105692] Updated weights for policy 0, policy_version 1672190 (0.0010) [2023-12-27 03:34:34,869][105620] Updated weights for policy 1, policy_version 1675618 (0.0009) [2023-12-27 03:34:34,918][105620] Updated weights for policy 1, policy_version 1675628 (0.0010) [2023-12-27 03:34:34,970][105620] Updated weights for policy 1, policy_version 1675638 (0.0010) [2023-12-27 03:34:35,028][105620] Updated weights for policy 1, policy_version 1675648 (0.0010) [2023-12-27 03:34:35,466][105692] Updated weights for policy 0, policy_version 1672200 (0.0006) [2023-12-27 03:34:35,525][105692] Updated weights for policy 0, policy_version 1672210 (0.0005) [2023-12-27 03:34:35,590][105692] Updated weights for policy 0, policy_version 1672220 (0.0010) [2023-12-27 03:34:35,687][105620] Updated weights for policy 1, policy_version 1675658 (0.0007) [2023-12-27 03:34:35,740][105620] Updated weights for policy 1, policy_version 1675668 (0.0005) [2023-12-27 03:34:35,796][105620] Updated weights for policy 1, policy_version 1675678 (0.0008) [2023-12-27 03:34:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.4, 300 sec: 19494.2). Total num frames: 857186304. Throughput: 0: 9648.9, 1: 9798.1. Samples: 857174300. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:34:36,062][104569] Avg episode reward: [(0, '9080.383'), (1, '8987.646')] [2023-12-27 03:34:36,331][105692] Updated weights for policy 0, policy_version 1672230 (0.0010) [2023-12-27 03:34:36,374][105620] Updated weights for policy 1, policy_version 1675688 (0.0008) [2023-12-27 03:34:36,391][105692] Updated weights for policy 0, policy_version 1672240 (0.0007) [2023-12-27 03:34:36,436][105620] Updated weights for policy 1, policy_version 1675698 (0.0006) [2023-12-27 03:34:36,441][105692] Updated weights for policy 0, policy_version 1672250 (0.0009) [2023-12-27 03:34:36,502][105620] Updated weights for policy 1, policy_version 1675708 (0.0010) [2023-12-27 03:34:37,074][105620] Updated weights for policy 1, policy_version 1675718 (0.0008) [2023-12-27 03:34:37,129][105620] Updated weights for policy 1, policy_version 1675728 (0.0005) [2023-12-27 03:34:37,183][105692] Updated weights for policy 0, policy_version 1672260 (0.0007) [2023-12-27 03:34:37,188][105620] Updated weights for policy 1, policy_version 1675738 (0.0005) [2023-12-27 03:34:37,234][105692] Updated weights for policy 0, policy_version 1672270 (0.0005) [2023-12-27 03:34:37,292][105692] Updated weights for policy 0, policy_version 1672280 (0.0005) [2023-12-27 03:34:37,849][105692] Updated weights for policy 0, policy_version 1672290 (0.0007) [2023-12-27 03:34:37,889][105620] Updated weights for policy 1, policy_version 1675748 (0.0010) [2023-12-27 03:34:37,905][105692] Updated weights for policy 0, policy_version 1672300 (0.0011) [2023-12-27 03:34:37,941][105620] Updated weights for policy 1, policy_version 1675758 (0.0010) [2023-12-27 03:34:37,950][105692] Updated weights for policy 0, policy_version 1672310 (0.0010) [2023-12-27 03:34:38,000][105620] Updated weights for policy 1, policy_version 1675768 (0.0010) [2023-12-27 03:34:38,004][105692] Updated weights for policy 0, policy_version 1672320 (0.0010) [2023-12-27 03:34:38,695][105620] Updated weights for policy 1, policy_version 1675778 (0.0011) [2023-12-27 03:34:38,764][105620] Updated weights for policy 1, policy_version 1675788 (0.0010) [2023-12-27 03:34:38,791][105692] Updated weights for policy 0, policy_version 1672330 (0.0007) [2023-12-27 03:34:38,829][105620] Updated weights for policy 1, policy_version 1675798 (0.0009) [2023-12-27 03:34:38,850][105692] Updated weights for policy 0, policy_version 1672340 (0.0007) [2023-12-27 03:34:38,892][105620] Updated weights for policy 1, policy_version 1675808 (0.0010) [2023-12-27 03:34:38,906][105692] Updated weights for policy 0, policy_version 1672350 (0.0007) [2023-12-27 03:34:39,549][105620] Updated weights for policy 1, policy_version 1675818 (0.0011) [2023-12-27 03:34:39,580][105692] Updated weights for policy 0, policy_version 1672360 (0.0005) [2023-12-27 03:34:39,609][105620] Updated weights for policy 1, policy_version 1675828 (0.0011) [2023-12-27 03:34:39,639][105692] Updated weights for policy 0, policy_version 1672370 (0.0006) [2023-12-27 03:34:39,669][105620] Updated weights for policy 1, policy_version 1675838 (0.0011) [2023-12-27 03:34:39,692][105692] Updated weights for policy 0, policy_version 1672380 (0.0007) [2023-12-27 03:34:40,403][105692] Updated weights for policy 0, policy_version 1672390 (0.0009) [2023-12-27 03:34:40,452][105692] Updated weights for policy 0, policy_version 1672400 (0.0010) [2023-12-27 03:34:40,476][105620] Updated weights for policy 1, policy_version 1675848 (0.0009) [2023-12-27 03:34:40,497][105692] Updated weights for policy 0, policy_version 1672410 (0.0010) [2023-12-27 03:34:40,534][105620] Updated weights for policy 1, policy_version 1675858 (0.0010) [2023-12-27 03:34:40,590][105620] Updated weights for policy 1, policy_version 1675868 (0.0011) [2023-12-27 03:34:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 857284608. Throughput: 0: 9732.7, 1: 9859.4. Samples: 857295384. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:34:41,063][104569] Avg episode reward: [(0, '9083.804'), (1, '8803.348')] [2023-12-27 03:34:41,226][105620] Updated weights for policy 1, policy_version 1675878 (0.0008) [2023-12-27 03:34:41,247][105692] Updated weights for policy 0, policy_version 1672420 (0.0010) [2023-12-27 03:34:41,293][105620] Updated weights for policy 1, policy_version 1675888 (0.0007) [2023-12-27 03:34:41,300][105692] Updated weights for policy 0, policy_version 1672430 (0.0007) [2023-12-27 03:34:41,362][105620] Updated weights for policy 1, policy_version 1675898 (0.0007) [2023-12-27 03:34:41,376][105692] Updated weights for policy 0, policy_version 1672440 (0.0008) [2023-12-27 03:34:42,037][105620] Updated weights for policy 1, policy_version 1675908 (0.0009) [2023-12-27 03:34:42,100][105620] Updated weights for policy 1, policy_version 1675918 (0.0011) [2023-12-27 03:34:42,163][105620] Updated weights for policy 1, policy_version 1675928 (0.0011) [2023-12-27 03:34:42,164][105692] Updated weights for policy 0, policy_version 1672450 (0.0008) [2023-12-27 03:34:42,214][105692] Updated weights for policy 0, policy_version 1672460 (0.0006) [2023-12-27 03:34:42,267][105692] Updated weights for policy 0, policy_version 1672470 (0.0008) [2023-12-27 03:34:42,328][105692] Updated weights for policy 0, policy_version 1672480 (0.0008) [2023-12-27 03:34:42,825][105620] Updated weights for policy 1, policy_version 1675938 (0.0010) [2023-12-27 03:34:42,879][105620] Updated weights for policy 1, policy_version 1675948 (0.0008) [2023-12-27 03:34:42,940][105620] Updated weights for policy 1, policy_version 1675958 (0.0009) [2023-12-27 03:34:42,992][105620] Updated weights for policy 1, policy_version 1675968 (0.0005) [2023-12-27 03:34:43,174][105692] Updated weights for policy 0, policy_version 1672490 (0.0009) [2023-12-27 03:34:43,234][105692] Updated weights for policy 0, policy_version 1672500 (0.0008) [2023-12-27 03:34:43,291][105692] Updated weights for policy 0, policy_version 1672510 (0.0010) [2023-12-27 03:34:43,594][105620] Updated weights for policy 1, policy_version 1675978 (0.0005) [2023-12-27 03:34:43,646][105620] Updated weights for policy 1, policy_version 1675988 (0.0005) [2023-12-27 03:34:43,692][105620] Updated weights for policy 1, policy_version 1675998 (0.0005) [2023-12-27 03:34:44,035][105692] Updated weights for policy 0, policy_version 1672520 (0.0010) [2023-12-27 03:34:44,089][105692] Updated weights for policy 0, policy_version 1672530 (0.0009) [2023-12-27 03:34:44,141][105692] Updated weights for policy 0, policy_version 1672540 (0.0010) [2023-12-27 03:34:44,242][105620] Updated weights for policy 1, policy_version 1676008 (0.0005) [2023-12-27 03:34:44,288][105620] Updated weights for policy 1, policy_version 1676018 (0.0005) [2023-12-27 03:34:44,342][105620] Updated weights for policy 1, policy_version 1676028 (0.0005) [2023-12-27 03:34:44,886][105692] Updated weights for policy 0, policy_version 1672550 (0.0010) [2023-12-27 03:34:44,944][105692] Updated weights for policy 0, policy_version 1672560 (0.0010) [2023-12-27 03:34:45,000][105620] Updated weights for policy 1, policy_version 1676038 (0.0007) [2023-12-27 03:34:45,007][105692] Updated weights for policy 0, policy_version 1672570 (0.0009) [2023-12-27 03:34:45,054][105620] Updated weights for policy 1, policy_version 1676048 (0.0010) [2023-12-27 03:34:45,103][105620] Updated weights for policy 1, policy_version 1676058 (0.0006) [2023-12-27 03:34:45,805][105620] Updated weights for policy 1, policy_version 1676068 (0.0009) [2023-12-27 03:34:45,817][105692] Updated weights for policy 0, policy_version 1672580 (0.0007) [2023-12-27 03:34:45,863][105620] Updated weights for policy 1, policy_version 1676078 (0.0005) [2023-12-27 03:34:45,869][105692] Updated weights for policy 0, policy_version 1672590 (0.0008) [2023-12-27 03:34:45,906][105620] Updated weights for policy 1, policy_version 1676088 (0.0005) [2023-12-27 03:34:45,917][105692] Updated weights for policy 0, policy_version 1672601 (0.0009) [2023-12-27 03:34:46,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 857391104. Throughput: 0: 9707.2, 1: 9920.4. Samples: 857354484. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:34:46,063][104569] Avg episode reward: [(0, '8988.429'), (1, '8343.494')] [2023-12-27 03:34:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001672608_428253184.pth... [2023-12-27 03:34:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001676096_429137920.pth... [2023-12-27 03:34:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001671488_427966464.pth [2023-12-27 03:34:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001674912_428834816.pth [2023-12-27 03:34:46,434][105620] Updated weights for policy 1, policy_version 1676098 (0.0005) [2023-12-27 03:34:46,482][105620] Updated weights for policy 1, policy_version 1676108 (0.0010) [2023-12-27 03:34:46,517][105692] Updated weights for policy 0, policy_version 1672612 (0.0008) [2023-12-27 03:34:46,544][105620] Updated weights for policy 1, policy_version 1676118 (0.0010) [2023-12-27 03:34:46,577][105692] Updated weights for policy 0, policy_version 1672622 (0.0007) [2023-12-27 03:34:46,602][105620] Updated weights for policy 1, policy_version 1676128 (0.0009) [2023-12-27 03:34:46,634][105692] Updated weights for policy 0, policy_version 1672632 (0.0007) [2023-12-27 03:34:47,163][105692] Updated weights for policy 0, policy_version 1672642 (0.0007) [2023-12-27 03:34:47,228][105692] Updated weights for policy 0, policy_version 1672652 (0.0006) [2023-12-27 03:34:47,270][105620] Updated weights for policy 1, policy_version 1676138 (0.0005) [2023-12-27 03:34:47,288][105692] Updated weights for policy 0, policy_version 1672662 (0.0008) [2023-12-27 03:34:47,337][105620] Updated weights for policy 1, policy_version 1676148 (0.0008) [2023-12-27 03:34:47,350][105692] Updated weights for policy 0, policy_version 1672672 (0.0008) [2023-12-27 03:34:47,397][105620] Updated weights for policy 1, policy_version 1676158 (0.0008) [2023-12-27 03:34:47,919][105620] Updated weights for policy 1, policy_version 1676168 (0.0009) [2023-12-27 03:34:47,966][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000007 [2023-12-27 03:34:48,102][105692] Updated weights for policy 0, policy_version 1672682 (0.0008) [2023-12-27 03:34:48,162][105692] Updated weights for policy 0, policy_version 1672692 (0.0008) [2023-12-27 03:34:48,230][105692] Updated weights for policy 0, policy_version 1672702 (0.0008) [2023-12-27 03:34:48,755][105620] Updated weights for policy 1, policy_version 1676178 (0.0010) [2023-12-27 03:34:48,808][105620] Updated weights for policy 1, policy_version 1676188 (0.0007) [2023-12-27 03:34:48,866][105620] Updated weights for policy 1, policy_version 1676198 (0.0007) [2023-12-27 03:34:48,921][105620] Updated weights for policy 1, policy_version 1676208 (0.0010) [2023-12-27 03:34:48,952][105692] Updated weights for policy 0, policy_version 1672712 (0.0007) [2023-12-27 03:34:49,000][105692] Updated weights for policy 0, policy_version 1672722 (0.0005) [2023-12-27 03:34:49,046][105692] Updated weights for policy 0, policy_version 1672732 (0.0005) [2023-12-27 03:34:49,626][105620] Updated weights for policy 1, policy_version 1676218 (0.0008) [2023-12-27 03:34:49,693][105620] Updated weights for policy 1, policy_version 1676228 (0.0008) [2023-12-27 03:34:49,758][105620] Updated weights for policy 1, policy_version 1676238 (0.0007) [2023-12-27 03:34:49,768][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000009 [2023-12-27 03:34:49,769][105692] Updated weights for policy 0, policy_version 1672742 (0.0006) [2023-12-27 03:34:49,834][105692] Updated weights for policy 0, policy_version 1672752 (0.0010) [2023-12-27 03:34:49,897][105692] Updated weights for policy 0, policy_version 1672762 (0.0009) [2023-12-27 03:34:50,428][105620] Updated weights for policy 1, policy_version 1676248 (0.0009) [2023-12-27 03:34:50,485][105620] Updated weights for policy 1, policy_version 1676258 (0.0010) [2023-12-27 03:34:50,552][105620] Updated weights for policy 1, policy_version 1676268 (0.0010) [2023-12-27 03:34:50,630][105692] Updated weights for policy 0, policy_version 1672772 (0.0009) [2023-12-27 03:34:50,692][105692] Updated weights for policy 0, policy_version 1672782 (0.0009) [2023-12-27 03:34:50,764][105692] Updated weights for policy 0, policy_version 1672792 (0.0009) [2023-12-27 03:34:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 857489408. Throughput: 0: 9665.5, 1: 10053.9. Samples: 857478652. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:34:51,062][104569] Avg episode reward: [(0, '8717.256'), (1, '8713.178')] [2023-12-27 03:34:51,308][105620] Updated weights for policy 1, policy_version 1676278 (0.0009) [2023-12-27 03:34:51,379][105620] Updated weights for policy 1, policy_version 1676288 (0.0008) [2023-12-27 03:34:51,440][105620] Updated weights for policy 1, policy_version 1676298 (0.0006) [2023-12-27 03:34:51,553][105692] Updated weights for policy 0, policy_version 1672802 (0.0009) [2023-12-27 03:34:51,612][105692] Updated weights for policy 0, policy_version 1672812 (0.0009) [2023-12-27 03:34:51,674][105692] Updated weights for policy 0, policy_version 1672822 (0.0009) [2023-12-27 03:34:51,731][105692] Updated weights for policy 0, policy_version 1672832 (0.0009) [2023-12-27 03:34:52,148][105620] Updated weights for policy 1, policy_version 1676308 (0.0007) [2023-12-27 03:34:52,203][105620] Updated weights for policy 1, policy_version 1676318 (0.0010) [2023-12-27 03:34:52,253][105620] Updated weights for policy 1, policy_version 1676328 (0.0009) [2023-12-27 03:34:52,495][105692] Updated weights for policy 0, policy_version 1672842 (0.0005) [2023-12-27 03:34:52,555][105692] Updated weights for policy 0, policy_version 1672852 (0.0008) [2023-12-27 03:34:52,615][105692] Updated weights for policy 0, policy_version 1672862 (0.0009) [2023-12-27 03:34:53,043][105620] Updated weights for policy 1, policy_version 1676338 (0.0009) [2023-12-27 03:34:53,091][105620] Updated weights for policy 1, policy_version 1676348 (0.0010) [2023-12-27 03:34:53,144][105620] Updated weights for policy 1, policy_version 1676358 (0.0010) [2023-12-27 03:34:53,192][105620] Updated weights for policy 1, policy_version 1676368 (0.0010) [2023-12-27 03:34:53,366][105692] Updated weights for policy 0, policy_version 1672872 (0.0009) [2023-12-27 03:34:53,423][105692] Updated weights for policy 0, policy_version 1672882 (0.0010) [2023-12-27 03:34:53,480][105692] Updated weights for policy 0, policy_version 1672892 (0.0010) [2023-12-27 03:34:53,847][105620] Updated weights for policy 1, policy_version 1676378 (0.0005) [2023-12-27 03:34:53,895][105620] Updated weights for policy 1, policy_version 1676388 (0.0005) [2023-12-27 03:34:53,945][105620] Updated weights for policy 1, policy_version 1676398 (0.0005) [2023-12-27 03:34:54,282][105692] Updated weights for policy 0, policy_version 1672902 (0.0007) [2023-12-27 03:34:54,327][105692] Updated weights for policy 0, policy_version 1672912 (0.0005) [2023-12-27 03:34:54,391][105692] Updated weights for policy 0, policy_version 1672922 (0.0007) [2023-12-27 03:34:54,472][105620] Updated weights for policy 1, policy_version 1676408 (0.0007) [2023-12-27 03:34:54,516][105620] Updated weights for policy 1, policy_version 1676418 (0.0010) [2023-12-27 03:34:54,563][105620] Updated weights for policy 1, policy_version 1676428 (0.0009) [2023-12-27 03:34:55,005][105692] Updated weights for policy 0, policy_version 1672932 (0.0010) [2023-12-27 03:34:55,069][105692] Updated weights for policy 0, policy_version 1672942 (0.0011) [2023-12-27 03:34:55,139][105692] Updated weights for policy 0, policy_version 1672952 (0.0010) [2023-12-27 03:34:55,305][105620] Updated weights for policy 1, policy_version 1676438 (0.0011) [2023-12-27 03:34:55,364][105620] Updated weights for policy 1, policy_version 1676448 (0.0010) [2023-12-27 03:34:55,424][105620] Updated weights for policy 1, policy_version 1676458 (0.0007) [2023-12-27 03:34:55,857][105692] Updated weights for policy 0, policy_version 1672962 (0.0007) [2023-12-27 03:34:55,912][105692] Updated weights for policy 0, policy_version 1672972 (0.0010) [2023-12-27 03:34:55,974][105692] Updated weights for policy 0, policy_version 1672982 (0.0010) [2023-12-27 03:34:55,985][105620] Updated weights for policy 1, policy_version 1676468 (0.0006) [2023-12-27 03:34:56,032][105692] Updated weights for policy 0, policy_version 1672992 (0.0010) [2023-12-27 03:34:56,044][105620] Updated weights for policy 1, policy_version 1676478 (0.0005) [2023-12-27 03:34:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 857587712. Throughput: 0: 9603.2, 1: 10118.3. Samples: 857596468. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:34:56,063][104569] Avg episode reward: [(0, '8451.403'), (1, '8621.262')] [2023-12-27 03:34:56,109][105620] Updated weights for policy 1, policy_version 1676488 (0.0006) [2023-12-27 03:34:56,626][105620] Updated weights for policy 1, policy_version 1676498 (0.0007) [2023-12-27 03:34:56,647][105692] Updated weights for policy 0, policy_version 1673002 (0.0005) [2023-12-27 03:34:56,687][105620] Updated weights for policy 1, policy_version 1676508 (0.0010) [2023-12-27 03:34:56,707][105692] Updated weights for policy 0, policy_version 1673012 (0.0006) [2023-12-27 03:34:56,747][105620] Updated weights for policy 1, policy_version 1676518 (0.0011) [2023-12-27 03:34:56,760][105692] Updated weights for policy 0, policy_version 1673022 (0.0010) [2023-12-27 03:34:56,805][105620] Updated weights for policy 1, policy_version 1676528 (0.0010) [2023-12-27 03:34:57,419][105692] Updated weights for policy 0, policy_version 1673032 (0.0010) [2023-12-27 03:34:57,466][105692] Updated weights for policy 0, policy_version 1673042 (0.0010) [2023-12-27 03:34:57,495][105620] Updated weights for policy 1, policy_version 1676538 (0.0005) [2023-12-27 03:34:57,513][105692] Updated weights for policy 0, policy_version 1673052 (0.0010) [2023-12-27 03:34:57,553][105620] Updated weights for policy 1, policy_version 1676548 (0.0005) [2023-12-27 03:34:57,610][105620] Updated weights for policy 1, policy_version 1676558 (0.0006) [2023-12-27 03:34:58,198][105692] Updated weights for policy 0, policy_version 1673062 (0.0008) [2023-12-27 03:34:58,218][105620] Updated weights for policy 1, policy_version 1676568 (0.0007) [2023-12-27 03:34:58,264][105692] Updated weights for policy 0, policy_version 1673072 (0.0007) [2023-12-27 03:34:58,271][105620] Updated weights for policy 1, policy_version 1676578 (0.0008) [2023-12-27 03:34:58,335][105692] Updated weights for policy 0, policy_version 1673082 (0.0008) [2023-12-27 03:34:58,336][105620] Updated weights for policy 1, policy_version 1676588 (0.0007) [2023-12-27 03:34:59,114][105620] Updated weights for policy 1, policy_version 1676598 (0.0009) [2023-12-27 03:34:59,147][105692] Updated weights for policy 0, policy_version 1673092 (0.0010) [2023-12-27 03:34:59,178][105620] Updated weights for policy 1, policy_version 1676608 (0.0007) [2023-12-27 03:34:59,201][105692] Updated weights for policy 0, policy_version 1673102 (0.0010) [2023-12-27 03:34:59,234][105620] Updated weights for policy 1, policy_version 1676618 (0.0008) [2023-12-27 03:34:59,273][105692] Updated weights for policy 0, policy_version 1673112 (0.0009) [2023-12-27 03:34:59,962][105620] Updated weights for policy 1, policy_version 1676628 (0.0008) [2023-12-27 03:35:00,024][105620] Updated weights for policy 1, policy_version 1676638 (0.0008) [2023-12-27 03:35:00,060][105692] Updated weights for policy 0, policy_version 1673122 (0.0008) [2023-12-27 03:35:00,081][105620] Updated weights for policy 1, policy_version 1676648 (0.0008) [2023-12-27 03:35:00,111][105692] Updated weights for policy 0, policy_version 1673132 (0.0007) [2023-12-27 03:35:00,163][105692] Updated weights for policy 0, policy_version 1673142 (0.0006) [2023-12-27 03:35:00,216][105692] Updated weights for policy 0, policy_version 1673152 (0.0009) [2023-12-27 03:35:00,738][105620] Updated weights for policy 1, policy_version 1676658 (0.0008) [2023-12-27 03:35:00,793][105620] Updated weights for policy 1, policy_version 1676668 (0.0005) [2023-12-27 03:35:00,843][105620] Updated weights for policy 1, policy_version 1676678 (0.0005) [2023-12-27 03:35:00,889][105620] Updated weights for policy 1, policy_version 1676688 (0.0008) [2023-12-27 03:35:00,994][105692] Updated weights for policy 0, policy_version 1673162 (0.0009) [2023-12-27 03:35:01,056][105692] Updated weights for policy 0, policy_version 1673172 (0.0009) [2023-12-27 03:35:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 857686016. Throughput: 0: 9639.0, 1: 10148.7. Samples: 857658220. Policy #0 lag: (min: 26.0, avg: 48.4, max: 58.0) [2023-12-27 03:35:01,063][104569] Avg episode reward: [(0, '8629.028'), (1, '8533.235')] [2023-12-27 03:35:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001676688_429293568.pth... [2023-12-27 03:35:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001675488_428982272.pth [2023-12-27 03:35:01,116][105692] Updated weights for policy 0, policy_version 1673182 (0.0009) [2023-12-27 03:35:01,128][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001673184_428400640.pth... [2023-12-27 03:35:01,133][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001672032_428105728.pth [2023-12-27 03:35:01,603][105620] Updated weights for policy 1, policy_version 1676698 (0.0008) [2023-12-27 03:35:01,667][105620] Updated weights for policy 1, policy_version 1676708 (0.0008) [2023-12-27 03:35:01,734][105620] Updated weights for policy 1, policy_version 1676718 (0.0007) [2023-12-27 03:35:01,933][105692] Updated weights for policy 0, policy_version 1673192 (0.0008) [2023-12-27 03:35:01,981][105692] Updated weights for policy 0, policy_version 1673202 (0.0008) [2023-12-27 03:35:02,032][105692] Updated weights for policy 0, policy_version 1673212 (0.0008) [2023-12-27 03:35:02,481][105620] Updated weights for policy 1, policy_version 1676728 (0.0006) [2023-12-27 03:35:02,546][105620] Updated weights for policy 1, policy_version 1676738 (0.0010) [2023-12-27 03:35:02,594][105620] Updated weights for policy 1, policy_version 1676748 (0.0010) [2023-12-27 03:35:02,820][105692] Updated weights for policy 0, policy_version 1673222 (0.0008) [2023-12-27 03:35:02,877][105692] Updated weights for policy 0, policy_version 1673233 (0.0010) [2023-12-27 03:35:02,929][105692] Updated weights for policy 0, policy_version 1673243 (0.0009) [2023-12-27 03:35:03,194][105620] Updated weights for policy 1, policy_version 1676758 (0.0007) [2023-12-27 03:35:03,255][105620] Updated weights for policy 1, policy_version 1676768 (0.0010) [2023-12-27 03:35:03,300][105620] Updated weights for policy 1, policy_version 1676778 (0.0010) [2023-12-27 03:35:03,775][105692] Updated weights for policy 0, policy_version 1673253 (0.0010) [2023-12-27 03:35:03,833][105692] Updated weights for policy 0, policy_version 1673263 (0.0010) [2023-12-27 03:35:03,903][105692] Updated weights for policy 0, policy_version 1673273 (0.0007) [2023-12-27 03:35:03,964][105620] Updated weights for policy 1, policy_version 1676789 (0.0010) [2023-12-27 03:35:04,014][105620] Updated weights for policy 1, policy_version 1676799 (0.0011) [2023-12-27 03:35:04,067][105620] Updated weights for policy 1, policy_version 1676809 (0.0011) [2023-12-27 03:35:04,617][105692] Updated weights for policy 0, policy_version 1673283 (0.0007) [2023-12-27 03:35:04,688][105692] Updated weights for policy 0, policy_version 1673293 (0.0006) [2023-12-27 03:35:04,754][105692] Updated weights for policy 0, policy_version 1673303 (0.0009) [2023-12-27 03:35:04,780][105620] Updated weights for policy 1, policy_version 1676819 (0.0011) [2023-12-27 03:35:04,835][105620] Updated weights for policy 1, policy_version 1676829 (0.0010) [2023-12-27 03:35:04,890][105620] Updated weights for policy 1, policy_version 1676839 (0.0010) [2023-12-27 03:35:05,383][105692] Updated weights for policy 0, policy_version 1673313 (0.0010) [2023-12-27 03:35:05,442][105692] Updated weights for policy 0, policy_version 1673323 (0.0005) [2023-12-27 03:35:05,499][105692] Updated weights for policy 0, policy_version 1673333 (0.0009) [2023-12-27 03:35:05,561][105692] Updated weights for policy 0, policy_version 1673343 (0.0010) [2023-12-27 03:35:05,648][105620] Updated weights for policy 1, policy_version 1676849 (0.0011) [2023-12-27 03:35:05,706][105620] Updated weights for policy 1, policy_version 1676859 (0.0010) [2023-12-27 03:35:05,764][105620] Updated weights for policy 1, policy_version 1676869 (0.0010) [2023-12-27 03:35:05,824][105620] Updated weights for policy 1, policy_version 1676879 (0.0011) [2023-12-27 03:35:06,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 857784320. Throughput: 0: 9503.0, 1: 10208.5. Samples: 857772324. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:35:06,062][104569] Avg episode reward: [(0, '8894.208'), (1, '8719.899')] [2023-12-27 03:35:06,286][105692] Updated weights for policy 0, policy_version 1673353 (0.0011) [2023-12-27 03:35:06,356][105692] Updated weights for policy 0, policy_version 1673363 (0.0011) [2023-12-27 03:35:06,416][105692] Updated weights for policy 0, policy_version 1673373 (0.0011) [2023-12-27 03:35:06,556][105620] Updated weights for policy 1, policy_version 1676889 (0.0008) [2023-12-27 03:35:06,619][105620] Updated weights for policy 1, policy_version 1676899 (0.0007) [2023-12-27 03:35:06,681][105620] Updated weights for policy 1, policy_version 1676909 (0.0007) [2023-12-27 03:35:07,151][105692] Updated weights for policy 0, policy_version 1673383 (0.0011) [2023-12-27 03:35:07,201][105692] Updated weights for policy 0, policy_version 1673393 (0.0009) [2023-12-27 03:35:07,258][105692] Updated weights for policy 0, policy_version 1673403 (0.0010) [2023-12-27 03:35:07,292][105620] Updated weights for policy 1, policy_version 1676919 (0.0007) [2023-12-27 03:35:07,351][105620] Updated weights for policy 1, policy_version 1676929 (0.0008) [2023-12-27 03:35:07,407][105620] Updated weights for policy 1, policy_version 1676939 (0.0008) [2023-12-27 03:35:07,981][105692] Updated weights for policy 0, policy_version 1673413 (0.0008) [2023-12-27 03:35:08,035][105692] Updated weights for policy 0, policy_version 1673423 (0.0009) [2023-12-27 03:35:08,093][105692] Updated weights for policy 0, policy_version 1673433 (0.0006) [2023-12-27 03:35:08,188][105620] Updated weights for policy 1, policy_version 1676949 (0.0009) [2023-12-27 03:35:08,240][105620] Updated weights for policy 1, policy_version 1676959 (0.0009) [2023-12-27 03:35:08,297][105620] Updated weights for policy 1, policy_version 1676970 (0.0010) [2023-12-27 03:35:08,720][105692] Updated weights for policy 0, policy_version 1673443 (0.0007) [2023-12-27 03:35:08,786][105692] Updated weights for policy 0, policy_version 1673453 (0.0008) [2023-12-27 03:35:08,846][105692] Updated weights for policy 0, policy_version 1673463 (0.0009) [2023-12-27 03:35:09,108][105620] Updated weights for policy 1, policy_version 1676980 (0.0009) [2023-12-27 03:35:09,161][105620] Updated weights for policy 1, policy_version 1676990 (0.0009) [2023-12-27 03:35:09,218][105620] Updated weights for policy 1, policy_version 1677000 (0.0009) [2023-12-27 03:35:09,526][105692] Updated weights for policy 0, policy_version 1673473 (0.0009) [2023-12-27 03:35:09,580][105692] Updated weights for policy 0, policy_version 1673483 (0.0005) [2023-12-27 03:35:09,627][105692] Updated weights for policy 0, policy_version 1673493 (0.0005) [2023-12-27 03:35:09,677][105692] Updated weights for policy 0, policy_version 1673503 (0.0006) [2023-12-27 03:35:10,056][105620] Updated weights for policy 1, policy_version 1677010 (0.0009) [2023-12-27 03:35:10,116][105620] Updated weights for policy 1, policy_version 1677020 (0.0008) [2023-12-27 03:35:10,173][105620] Updated weights for policy 1, policy_version 1677030 (0.0008) [2023-12-27 03:35:10,241][105620] Updated weights for policy 1, policy_version 1677040 (0.0008) [2023-12-27 03:35:10,358][105692] Updated weights for policy 0, policy_version 1673513 (0.0010) [2023-12-27 03:35:10,417][105692] Updated weights for policy 0, policy_version 1673523 (0.0011) [2023-12-27 03:35:10,477][105692] Updated weights for policy 0, policy_version 1673533 (0.0009) [2023-12-27 03:35:10,927][105620] Updated weights for policy 1, policy_version 1677050 (0.0008) [2023-12-27 03:35:10,993][105620] Updated weights for policy 1, policy_version 1677060 (0.0005) [2023-12-27 03:35:11,057][105620] Updated weights for policy 1, policy_version 1677070 (0.0008) [2023-12-27 03:35:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 857874432. Throughput: 0: 9631.0, 1: 10109.9. Samples: 857888608. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:35:11,062][104569] Avg episode reward: [(0, '8618.574'), (1, '8905.978')] [2023-12-27 03:35:11,161][105692] Updated weights for policy 0, policy_version 1673543 (0.0008) [2023-12-27 03:35:11,227][105692] Updated weights for policy 0, policy_version 1673553 (0.0010) [2023-12-27 03:35:11,293][105692] Updated weights for policy 0, policy_version 1673563 (0.0009) [2023-12-27 03:35:11,783][105620] Updated weights for policy 1, policy_version 1677080 (0.0008) [2023-12-27 03:35:11,846][105620] Updated weights for policy 1, policy_version 1677090 (0.0009) [2023-12-27 03:35:11,909][105620] Updated weights for policy 1, policy_version 1677100 (0.0007) [2023-12-27 03:35:12,065][105692] Updated weights for policy 0, policy_version 1673573 (0.0008) [2023-12-27 03:35:12,126][105692] Updated weights for policy 0, policy_version 1673583 (0.0007) [2023-12-27 03:35:12,175][105692] Updated weights for policy 0, policy_version 1673593 (0.0008) [2023-12-27 03:35:12,601][105620] Updated weights for policy 1, policy_version 1677110 (0.0010) [2023-12-27 03:35:12,653][105620] Updated weights for policy 1, policy_version 1677120 (0.0010) [2023-12-27 03:35:12,705][105620] Updated weights for policy 1, policy_version 1677130 (0.0010) [2023-12-27 03:35:13,023][105692] Updated weights for policy 0, policy_version 1673603 (0.0009) [2023-12-27 03:35:13,087][105692] Updated weights for policy 0, policy_version 1673613 (0.0008) [2023-12-27 03:35:13,148][105692] Updated weights for policy 0, policy_version 1673623 (0.0007) [2023-12-27 03:35:13,339][105620] Updated weights for policy 1, policy_version 1677140 (0.0009) [2023-12-27 03:35:13,383][105620] Updated weights for policy 1, policy_version 1677150 (0.0008) [2023-12-27 03:35:13,428][105620] Updated weights for policy 1, policy_version 1677160 (0.0005) [2023-12-27 03:35:13,867][105692] Updated weights for policy 0, policy_version 1673633 (0.0006) [2023-12-27 03:35:13,923][105692] Updated weights for policy 0, policy_version 1673643 (0.0010) [2023-12-27 03:35:13,979][105692] Updated weights for policy 0, policy_version 1673654 (0.0010) [2023-12-27 03:35:14,037][105692] Updated weights for policy 0, policy_version 1673664 (0.0008) [2023-12-27 03:35:14,056][105620] Updated weights for policy 1, policy_version 1677170 (0.0005) [2023-12-27 03:35:14,107][105620] Updated weights for policy 1, policy_version 1677180 (0.0005) [2023-12-27 03:35:14,164][105620] Updated weights for policy 1, policy_version 1677190 (0.0005) [2023-12-27 03:35:14,221][105620] Updated weights for policy 1, policy_version 1677200 (0.0005) [2023-12-27 03:35:14,730][105692] Updated weights for policy 0, policy_version 1673674 (0.0005) [2023-12-27 03:35:14,735][105620] Updated weights for policy 1, policy_version 1677210 (0.0008) [2023-12-27 03:35:14,789][105692] Updated weights for policy 0, policy_version 1673684 (0.0008) [2023-12-27 03:35:14,800][105620] Updated weights for policy 1, policy_version 1677220 (0.0007) [2023-12-27 03:35:14,846][105692] Updated weights for policy 0, policy_version 1673694 (0.0011) [2023-12-27 03:35:14,860][105620] Updated weights for policy 1, policy_version 1677230 (0.0006) [2023-12-27 03:35:15,472][105620] Updated weights for policy 1, policy_version 1677240 (0.0010) [2023-12-27 03:35:15,524][105620] Updated weights for policy 1, policy_version 1677250 (0.0008) [2023-12-27 03:35:15,527][105692] Updated weights for policy 0, policy_version 1673704 (0.0006) [2023-12-27 03:35:15,577][105620] Updated weights for policy 1, policy_version 1677260 (0.0008) [2023-12-27 03:35:15,584][105692] Updated weights for policy 0, policy_version 1673714 (0.0005) [2023-12-27 03:35:15,644][105692] Updated weights for policy 0, policy_version 1673724 (0.0007) [2023-12-27 03:35:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 857980928. Throughput: 0: 9594.5, 1: 10095.2. Samples: 857946792. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:35:16,063][104569] Avg episode reward: [(0, '8076.600'), (1, '8902.354')] [2023-12-27 03:35:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001677264_429441024.pth... [2023-12-27 03:35:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001673728_428539904.pth... [2023-12-27 03:35:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001676096_429137920.pth [2023-12-27 03:35:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001672608_428253184.pth [2023-12-27 03:35:16,248][105620] Updated weights for policy 1, policy_version 1677270 (0.0009) [2023-12-27 03:35:16,303][105620] Updated weights for policy 1, policy_version 1677280 (0.0007) [2023-12-27 03:35:16,362][105620] Updated weights for policy 1, policy_version 1677290 (0.0005) [2023-12-27 03:35:16,410][105692] Updated weights for policy 0, policy_version 1673734 (0.0007) [2023-12-27 03:35:16,465][105692] Updated weights for policy 0, policy_version 1673744 (0.0009) [2023-12-27 03:35:16,520][105692] Updated weights for policy 0, policy_version 1673754 (0.0009) [2023-12-27 03:35:17,007][105620] Updated weights for policy 1, policy_version 1677300 (0.0006) [2023-12-27 03:35:17,064][105620] Updated weights for policy 1, policy_version 1677310 (0.0009) [2023-12-27 03:35:17,112][105620] Updated weights for policy 1, policy_version 1677320 (0.0005) [2023-12-27 03:35:17,333][105692] Updated weights for policy 0, policy_version 1673764 (0.0009) [2023-12-27 03:35:17,391][105692] Updated weights for policy 0, policy_version 1673774 (0.0005) [2023-12-27 03:35:17,444][105692] Updated weights for policy 0, policy_version 1673784 (0.0005) [2023-12-27 03:35:17,856][105620] Updated weights for policy 1, policy_version 1677330 (0.0006) [2023-12-27 03:35:17,914][105620] Updated weights for policy 1, policy_version 1677341 (0.0010) [2023-12-27 03:35:17,972][105620] Updated weights for policy 1, policy_version 1677352 (0.0011) [2023-12-27 03:35:17,989][105692] Updated weights for policy 0, policy_version 1673794 (0.0006) [2023-12-27 03:35:18,052][105692] Updated weights for policy 0, policy_version 1673804 (0.0005) [2023-12-27 03:35:18,117][105692] Updated weights for policy 0, policy_version 1673814 (0.0009) [2023-12-27 03:35:18,194][105692] Updated weights for policy 0, policy_version 1673824 (0.0007) [2023-12-27 03:35:18,797][105620] Updated weights for policy 1, policy_version 1677362 (0.0009) [2023-12-27 03:35:18,824][105692] Updated weights for policy 0, policy_version 1673834 (0.0006) [2023-12-27 03:35:18,856][105620] Updated weights for policy 1, policy_version 1677372 (0.0006) [2023-12-27 03:35:18,887][105692] Updated weights for policy 0, policy_version 1673844 (0.0007) [2023-12-27 03:35:18,914][105620] Updated weights for policy 1, policy_version 1677382 (0.0007) [2023-12-27 03:35:18,949][105692] Updated weights for policy 0, policy_version 1673854 (0.0007) [2023-12-27 03:35:18,976][105620] Updated weights for policy 1, policy_version 1677392 (0.0009) [2023-12-27 03:35:19,690][105692] Updated weights for policy 0, policy_version 1673864 (0.0007) [2023-12-27 03:35:19,754][105692] Updated weights for policy 0, policy_version 1673874 (0.0006) [2023-12-27 03:35:19,765][105620] Updated weights for policy 1, policy_version 1677402 (0.0007) [2023-12-27 03:35:19,822][105620] Updated weights for policy 1, policy_version 1677412 (0.0008) [2023-12-27 03:35:19,827][105692] Updated weights for policy 0, policy_version 1673884 (0.0006) [2023-12-27 03:35:19,884][105620] Updated weights for policy 1, policy_version 1677422 (0.0007) [2023-12-27 03:35:20,543][105620] Updated weights for policy 1, policy_version 1677432 (0.0008) [2023-12-27 03:35:20,578][105692] Updated weights for policy 0, policy_version 1673894 (0.0010) [2023-12-27 03:35:20,602][105620] Updated weights for policy 1, policy_version 1677442 (0.0008) [2023-12-27 03:35:20,639][105692] Updated weights for policy 0, policy_version 1673904 (0.0011) [2023-12-27 03:35:20,657][105620] Updated weights for policy 1, policy_version 1677452 (0.0008) [2023-12-27 03:35:20,696][105692] Updated weights for policy 0, policy_version 1673914 (0.0008) [2023-12-27 03:35:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 858079232. Throughput: 0: 9648.4, 1: 10193.6. Samples: 858067192. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:35:21,063][104569] Avg episode reward: [(0, '8355.027'), (1, '9080.490')] [2023-12-27 03:35:21,477][105620] Updated weights for policy 1, policy_version 1677462 (0.0007) [2023-12-27 03:35:21,529][105620] Updated weights for policy 1, policy_version 1677472 (0.0008) [2023-12-27 03:35:21,542][105692] Updated weights for policy 0, policy_version 1673924 (0.0010) [2023-12-27 03:35:21,580][105620] Updated weights for policy 1, policy_version 1677482 (0.0008) [2023-12-27 03:35:21,590][105692] Updated weights for policy 0, policy_version 1673934 (0.0007) [2023-12-27 03:35:21,658][105692] Updated weights for policy 0, policy_version 1673944 (0.0008) [2023-12-27 03:35:22,265][105620] Updated weights for policy 1, policy_version 1677492 (0.0009) [2023-12-27 03:35:22,335][105620] Updated weights for policy 1, policy_version 1677502 (0.0010) [2023-12-27 03:35:22,406][105620] Updated weights for policy 1, policy_version 1677512 (0.0008) [2023-12-27 03:35:22,543][105692] Updated weights for policy 0, policy_version 1673954 (0.0009) [2023-12-27 03:35:22,601][105692] Updated weights for policy 0, policy_version 1673964 (0.0009) [2023-12-27 03:35:22,658][105692] Updated weights for policy 0, policy_version 1673974 (0.0009) [2023-12-27 03:35:22,720][105692] Updated weights for policy 0, policy_version 1673984 (0.0009) [2023-12-27 03:35:23,104][105620] Updated weights for policy 1, policy_version 1677522 (0.0008) [2023-12-27 03:35:23,164][105620] Updated weights for policy 1, policy_version 1677532 (0.0009) [2023-12-27 03:35:23,220][105620] Updated weights for policy 1, policy_version 1677542 (0.0006) [2023-12-27 03:35:23,278][105620] Updated weights for policy 1, policy_version 1677552 (0.0007) [2023-12-27 03:35:23,564][105692] Updated weights for policy 0, policy_version 1673994 (0.0007) [2023-12-27 03:35:23,630][105692] Updated weights for policy 0, policy_version 1674004 (0.0006) [2023-12-27 03:35:23,699][105692] Updated weights for policy 0, policy_version 1674014 (0.0009) [2023-12-27 03:35:23,867][105620] Updated weights for policy 1, policy_version 1677562 (0.0009) [2023-12-27 03:35:23,920][105620] Updated weights for policy 1, policy_version 1677572 (0.0008) [2023-12-27 03:35:23,970][105620] Updated weights for policy 1, policy_version 1677582 (0.0009) [2023-12-27 03:35:24,351][105692] Updated weights for policy 0, policy_version 1674024 (0.0009) [2023-12-27 03:35:24,399][105692] Updated weights for policy 0, policy_version 1674034 (0.0009) [2023-12-27 03:35:24,454][105692] Updated weights for policy 0, policy_version 1674044 (0.0009) [2023-12-27 03:35:24,731][105620] Updated weights for policy 1, policy_version 1677592 (0.0009) [2023-12-27 03:35:24,794][105620] Updated weights for policy 1, policy_version 1677602 (0.0010) [2023-12-27 03:35:24,860][105620] Updated weights for policy 1, policy_version 1677612 (0.0009) [2023-12-27 03:35:25,100][105692] Updated weights for policy 0, policy_version 1674054 (0.0009) [2023-12-27 03:35:25,152][105692] Updated weights for policy 0, policy_version 1674064 (0.0010) [2023-12-27 03:35:25,201][105692] Updated weights for policy 0, policy_version 1674074 (0.0009) [2023-12-27 03:35:25,520][105620] Updated weights for policy 1, policy_version 1677622 (0.0006) [2023-12-27 03:35:25,580][105620] Updated weights for policy 1, policy_version 1677632 (0.0006) [2023-12-27 03:35:25,632][105620] Updated weights for policy 1, policy_version 1677642 (0.0005) [2023-12-27 03:35:25,941][105692] Updated weights for policy 0, policy_version 1674084 (0.0010) [2023-12-27 03:35:25,987][105692] Updated weights for policy 0, policy_version 1674094 (0.0007) [2023-12-27 03:35:26,036][105692] Updated weights for policy 0, policy_version 1674104 (0.0010) [2023-12-27 03:35:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 858169344. Throughput: 0: 9553.3, 1: 10182.0. Samples: 858183472. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:35:26,062][104569] Avg episode reward: [(0, '8808.040'), (1, '9080.383')] [2023-12-27 03:35:26,232][105620] Updated weights for policy 1, policy_version 1677652 (0.0005) [2023-12-27 03:35:26,297][105620] Updated weights for policy 1, policy_version 1677662 (0.0005) [2023-12-27 03:35:26,360][105620] Updated weights for policy 1, policy_version 1677672 (0.0005) [2023-12-27 03:35:26,706][105692] Updated weights for policy 0, policy_version 1674114 (0.0010) [2023-12-27 03:35:26,762][105692] Updated weights for policy 0, policy_version 1674124 (0.0008) [2023-12-27 03:35:26,826][105692] Updated weights for policy 0, policy_version 1674134 (0.0009) [2023-12-27 03:35:26,882][105620] Updated weights for policy 1, policy_version 1677682 (0.0005) [2023-12-27 03:35:26,886][105692] Updated weights for policy 0, policy_version 1674144 (0.0009) [2023-12-27 03:35:26,941][105620] Updated weights for policy 1, policy_version 1677692 (0.0009) [2023-12-27 03:35:27,002][105620] Updated weights for policy 1, policy_version 1677702 (0.0007) [2023-12-27 03:35:27,059][105620] Updated weights for policy 1, policy_version 1677712 (0.0007) [2023-12-27 03:35:27,598][105692] Updated weights for policy 0, policy_version 1674154 (0.0005) [2023-12-27 03:35:27,642][105692] Updated weights for policy 0, policy_version 1674164 (0.0005) [2023-12-27 03:35:27,695][105692] Updated weights for policy 0, policy_version 1674174 (0.0005) [2023-12-27 03:35:27,746][105620] Updated weights for policy 1, policy_version 1677722 (0.0009) [2023-12-27 03:35:27,800][105620] Updated weights for policy 1, policy_version 1677732 (0.0009) [2023-12-27 03:35:27,853][105620] Updated weights for policy 1, policy_version 1677742 (0.0009) [2023-12-27 03:35:28,255][105692] Updated weights for policy 0, policy_version 1674184 (0.0008) [2023-12-27 03:35:28,307][105692] Updated weights for policy 0, policy_version 1674194 (0.0009) [2023-12-27 03:35:28,366][105692] Updated weights for policy 0, policy_version 1674204 (0.0009) [2023-12-27 03:35:28,642][105620] Updated weights for policy 1, policy_version 1677752 (0.0006) [2023-12-27 03:35:28,698][105620] Updated weights for policy 1, policy_version 1677762 (0.0005) [2023-12-27 03:35:28,750][105620] Updated weights for policy 1, policy_version 1677772 (0.0007) [2023-12-27 03:35:29,069][105692] Updated weights for policy 0, policy_version 1674214 (0.0009) [2023-12-27 03:35:29,117][105692] Updated weights for policy 0, policy_version 1674224 (0.0009) [2023-12-27 03:35:29,165][105692] Updated weights for policy 0, policy_version 1674234 (0.0007) [2023-12-27 03:35:29,502][105620] Updated weights for policy 1, policy_version 1677782 (0.0009) [2023-12-27 03:35:29,549][105620] Updated weights for policy 1, policy_version 1677792 (0.0009) [2023-12-27 03:35:29,595][105620] Updated weights for policy 1, policy_version 1677802 (0.0008) [2023-12-27 03:35:29,953][105692] Updated weights for policy 0, policy_version 1674244 (0.0007) [2023-12-27 03:35:30,005][105692] Updated weights for policy 0, policy_version 1674254 (0.0009) [2023-12-27 03:35:30,060][105692] Updated weights for policy 0, policy_version 1674264 (0.0009) [2023-12-27 03:35:30,372][105620] Updated weights for policy 1, policy_version 1677812 (0.0009) [2023-12-27 03:35:30,418][105620] Updated weights for policy 1, policy_version 1677822 (0.0009) [2023-12-27 03:35:30,463][105620] Updated weights for policy 1, policy_version 1677832 (0.0009) [2023-12-27 03:35:30,815][105692] Updated weights for policy 0, policy_version 1674274 (0.0009) [2023-12-27 03:35:30,874][105692] Updated weights for policy 0, policy_version 1674284 (0.0011) [2023-12-27 03:35:30,930][105692] Updated weights for policy 0, policy_version 1674294 (0.0009) [2023-12-27 03:35:30,983][105692] Updated weights for policy 0, policy_version 1674304 (0.0009) [2023-12-27 03:35:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 858275840. Throughput: 0: 9651.9, 1: 10138.3. Samples: 858245036. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:35:31,062][104569] Avg episode reward: [(0, '8895.336'), (1, '9263.910')] [2023-12-27 03:35:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001674304_428687360.pth... [2023-12-27 03:35:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001677840_429588480.pth... [2023-12-27 03:35:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001673184_428400640.pth [2023-12-27 03:35:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001676688_429293568.pth [2023-12-27 03:35:31,240][105620] Updated weights for policy 1, policy_version 1677842 (0.0008) [2023-12-27 03:35:31,303][105620] Updated weights for policy 1, policy_version 1677852 (0.0008) [2023-12-27 03:35:31,364][105620] Updated weights for policy 1, policy_version 1677862 (0.0009) [2023-12-27 03:35:31,429][105620] Updated weights for policy 1, policy_version 1677872 (0.0008) [2023-12-27 03:35:31,695][105692] Updated weights for policy 0, policy_version 1674314 (0.0008) [2023-12-27 03:35:31,755][105692] Updated weights for policy 0, policy_version 1674325 (0.0009) [2023-12-27 03:35:31,819][105692] Updated weights for policy 0, policy_version 1674335 (0.0007) [2023-12-27 03:35:32,175][105620] Updated weights for policy 1, policy_version 1677882 (0.0005) [2023-12-27 03:35:32,235][105620] Updated weights for policy 1, policy_version 1677892 (0.0009) [2023-12-27 03:35:32,298][105620] Updated weights for policy 1, policy_version 1677902 (0.0009) [2023-12-27 03:35:32,551][105692] Updated weights for policy 0, policy_version 1674345 (0.0009) [2023-12-27 03:35:32,603][105692] Updated weights for policy 0, policy_version 1674355 (0.0009) [2023-12-27 03:35:32,657][105692] Updated weights for policy 0, policy_version 1674365 (0.0009) [2023-12-27 03:35:33,033][105620] Updated weights for policy 1, policy_version 1677912 (0.0009) [2023-12-27 03:35:33,088][105620] Updated weights for policy 1, policy_version 1677922 (0.0009) [2023-12-27 03:35:33,145][105620] Updated weights for policy 1, policy_version 1677932 (0.0009) [2023-12-27 03:35:33,350][105692] Updated weights for policy 0, policy_version 1674375 (0.0006) [2023-12-27 03:35:33,395][105692] Updated weights for policy 0, policy_version 1674385 (0.0005) [2023-12-27 03:35:33,447][105692] Updated weights for policy 0, policy_version 1674395 (0.0005) [2023-12-27 03:35:33,977][105692] Updated weights for policy 0, policy_version 1674405 (0.0008) [2023-12-27 03:35:34,021][105692] Updated weights for policy 0, policy_version 1674415 (0.0010) [2023-12-27 03:35:34,035][105620] Updated weights for policy 1, policy_version 1677942 (0.0008) [2023-12-27 03:35:34,064][105692] Updated weights for policy 0, policy_version 1674425 (0.0008) [2023-12-27 03:35:34,086][105620] Updated weights for policy 1, policy_version 1677952 (0.0006) [2023-12-27 03:35:34,136][105620] Updated weights for policy 1, policy_version 1677962 (0.0009) [2023-12-27 03:35:34,720][105692] Updated weights for policy 0, policy_version 1674435 (0.0005) [2023-12-27 03:35:34,786][105692] Updated weights for policy 0, policy_version 1674445 (0.0006) [2023-12-27 03:35:34,858][105692] Updated weights for policy 0, policy_version 1674455 (0.0006) [2023-12-27 03:35:34,901][105620] Updated weights for policy 1, policy_version 1677972 (0.0007) [2023-12-27 03:35:34,960][105620] Updated weights for policy 1, policy_version 1677982 (0.0009) [2023-12-27 03:35:35,018][105620] Updated weights for policy 1, policy_version 1677992 (0.0009) [2023-12-27 03:35:35,456][105692] Updated weights for policy 0, policy_version 1674465 (0.0007) [2023-12-27 03:35:35,521][105692] Updated weights for policy 0, policy_version 1674475 (0.0010) [2023-12-27 03:35:35,582][105692] Updated weights for policy 0, policy_version 1674485 (0.0010) [2023-12-27 03:35:35,646][105692] Updated weights for policy 0, policy_version 1674495 (0.0010) [2023-12-27 03:35:35,759][105620] Updated weights for policy 1, policy_version 1678002 (0.0008) [2023-12-27 03:35:35,825][105620] Updated weights for policy 1, policy_version 1678012 (0.0005) [2023-12-27 03:35:35,887][105620] Updated weights for policy 1, policy_version 1678022 (0.0008) [2023-12-27 03:35:35,953][105620] Updated weights for policy 1, policy_version 1678032 (0.0010) [2023-12-27 03:35:36,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 858374144. Throughput: 0: 9683.1, 1: 9930.6. Samples: 858361272. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:35:36,063][104569] Avg episode reward: [(0, '8438.149'), (1, '9356.120')] [2023-12-27 03:35:36,338][105692] Updated weights for policy 0, policy_version 1674505 (0.0006) [2023-12-27 03:35:36,407][105692] Updated weights for policy 0, policy_version 1674515 (0.0006) [2023-12-27 03:35:36,464][105692] Updated weights for policy 0, policy_version 1674525 (0.0010) [2023-12-27 03:35:36,530][105620] Updated weights for policy 1, policy_version 1678042 (0.0011) [2023-12-27 03:35:36,590][105620] Updated weights for policy 1, policy_version 1678052 (0.0011) [2023-12-27 03:35:36,653][105620] Updated weights for policy 1, policy_version 1678062 (0.0011) [2023-12-27 03:35:37,085][105692] Updated weights for policy 0, policy_version 1674535 (0.0010) [2023-12-27 03:35:37,150][105692] Updated weights for policy 0, policy_version 1674545 (0.0010) [2023-12-27 03:35:37,216][105692] Updated weights for policy 0, policy_version 1674555 (0.0010) [2023-12-27 03:35:37,403][105620] Updated weights for policy 1, policy_version 1678072 (0.0006) [2023-12-27 03:35:37,469][105620] Updated weights for policy 1, policy_version 1678082 (0.0005) [2023-12-27 03:35:37,520][105620] Updated weights for policy 1, policy_version 1678092 (0.0005) [2023-12-27 03:35:37,844][105692] Updated weights for policy 0, policy_version 1674565 (0.0010) [2023-12-27 03:35:37,894][105692] Updated weights for policy 0, policy_version 1674575 (0.0010) [2023-12-27 03:35:37,947][105692] Updated weights for policy 0, policy_version 1674585 (0.0011) [2023-12-27 03:35:38,157][105620] Updated weights for policy 1, policy_version 1678102 (0.0008) [2023-12-27 03:35:38,205][105620] Updated weights for policy 1, policy_version 1678112 (0.0010) [2023-12-27 03:35:38,247][105620] Updated weights for policy 1, policy_version 1678122 (0.0010) [2023-12-27 03:35:38,735][105692] Updated weights for policy 0, policy_version 1674595 (0.0009) [2023-12-27 03:35:38,781][105692] Updated weights for policy 0, policy_version 1674605 (0.0005) [2023-12-27 03:35:38,838][105692] Updated weights for policy 0, policy_version 1674615 (0.0007) [2023-12-27 03:35:39,062][105620] Updated weights for policy 1, policy_version 1678132 (0.0010) [2023-12-27 03:35:39,111][105620] Updated weights for policy 1, policy_version 1678142 (0.0010) [2023-12-27 03:35:39,174][105620] Updated weights for policy 1, policy_version 1678152 (0.0011) [2023-12-27 03:35:39,466][105692] Updated weights for policy 0, policy_version 1674625 (0.0011) [2023-12-27 03:35:39,526][105692] Updated weights for policy 0, policy_version 1674635 (0.0011) [2023-12-27 03:35:39,585][105692] Updated weights for policy 0, policy_version 1674645 (0.0011) [2023-12-27 03:35:39,641][105692] Updated weights for policy 0, policy_version 1674655 (0.0011) [2023-12-27 03:35:39,804][105620] Updated weights for policy 1, policy_version 1678162 (0.0009) [2023-12-27 03:35:39,871][105620] Updated weights for policy 1, policy_version 1678172 (0.0007) [2023-12-27 03:35:39,939][105620] Updated weights for policy 1, policy_version 1678182 (0.0009) [2023-12-27 03:35:40,011][105620] Updated weights for policy 1, policy_version 1678192 (0.0011) [2023-12-27 03:35:40,419][105692] Updated weights for policy 0, policy_version 1674665 (0.0009) [2023-12-27 03:35:40,486][105692] Updated weights for policy 0, policy_version 1674675 (0.0008) [2023-12-27 03:35:40,552][105692] Updated weights for policy 0, policy_version 1674685 (0.0008) [2023-12-27 03:35:40,665][105620] Updated weights for policy 1, policy_version 1678202 (0.0005) [2023-12-27 03:35:40,725][105620] Updated weights for policy 1, policy_version 1678212 (0.0006) [2023-12-27 03:35:40,791][105620] Updated weights for policy 1, policy_version 1678222 (0.0011) [2023-12-27 03:35:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 858472448. Throughput: 0: 9773.0, 1: 9904.2. Samples: 858481940. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:35:41,062][104569] Avg episode reward: [(0, '8529.544'), (1, '9173.461')] [2023-12-27 03:35:41,158][105692] Updated weights for policy 0, policy_version 1674695 (0.0008) [2023-12-27 03:35:41,229][105692] Updated weights for policy 0, policy_version 1674706 (0.0009) [2023-12-27 03:35:41,288][105692] Updated weights for policy 0, policy_version 1674716 (0.0009) [2023-12-27 03:35:41,500][105620] Updated weights for policy 1, policy_version 1678232 (0.0010) [2023-12-27 03:35:41,560][105620] Updated weights for policy 1, policy_version 1678242 (0.0011) [2023-12-27 03:35:41,628][105620] Updated weights for policy 1, policy_version 1678252 (0.0011) [2023-12-27 03:35:42,071][105692] Updated weights for policy 0, policy_version 1674726 (0.0006) [2023-12-27 03:35:42,137][105692] Updated weights for policy 0, policy_version 1674736 (0.0008) [2023-12-27 03:35:42,197][105692] Updated weights for policy 0, policy_version 1674746 (0.0008) [2023-12-27 03:35:42,375][105620] Updated weights for policy 1, policy_version 1678262 (0.0010) [2023-12-27 03:35:42,438][105620] Updated weights for policy 1, policy_version 1678272 (0.0007) [2023-12-27 03:35:42,494][105620] Updated weights for policy 1, policy_version 1678282 (0.0006) [2023-12-27 03:35:42,954][105692] Updated weights for policy 0, policy_version 1674756 (0.0007) [2023-12-27 03:35:43,012][105692] Updated weights for policy 0, policy_version 1674766 (0.0010) [2023-12-27 03:35:43,036][105620] Updated weights for policy 1, policy_version 1678292 (0.0006) [2023-12-27 03:35:43,067][105692] Updated weights for policy 0, policy_version 1674776 (0.0006) [2023-12-27 03:35:43,083][105620] Updated weights for policy 1, policy_version 1678302 (0.0008) [2023-12-27 03:35:43,134][105620] Updated weights for policy 1, policy_version 1678312 (0.0007) [2023-12-27 03:35:43,170][105586] KL-divergence is very high: 102.5365 [2023-12-27 03:35:43,827][105620] Updated weights for policy 1, policy_version 1678322 (0.0010) [2023-12-27 03:35:43,863][105692] Updated weights for policy 0, policy_version 1674786 (0.0009) [2023-12-27 03:35:43,875][105620] Updated weights for policy 1, policy_version 1678332 (0.0010) [2023-12-27 03:35:43,918][105692] Updated weights for policy 0, policy_version 1674796 (0.0010) [2023-12-27 03:35:43,924][105620] Updated weights for policy 1, policy_version 1678342 (0.0010) [2023-12-27 03:35:43,976][105620] Updated weights for policy 1, policy_version 1678352 (0.0010) [2023-12-27 03:35:43,976][105692] Updated weights for policy 0, policy_version 1674806 (0.0010) [2023-12-27 03:35:44,041][105692] Updated weights for policy 0, policy_version 1674816 (0.0010) [2023-12-27 03:35:44,579][105620] Updated weights for policy 1, policy_version 1678362 (0.0005) [2023-12-27 03:35:44,630][105620] Updated weights for policy 1, policy_version 1678372 (0.0005) [2023-12-27 03:35:44,686][105692] Updated weights for policy 0, policy_version 1674826 (0.0009) [2023-12-27 03:35:44,689][105620] Updated weights for policy 1, policy_version 1678382 (0.0005) [2023-12-27 03:35:44,754][105692] Updated weights for policy 0, policy_version 1674836 (0.0005) [2023-12-27 03:35:44,821][105692] Updated weights for policy 0, policy_version 1674846 (0.0007) [2023-12-27 03:35:45,292][105620] Updated weights for policy 1, policy_version 1678392 (0.0006) [2023-12-27 03:35:45,359][105620] Updated weights for policy 1, policy_version 1678402 (0.0006) [2023-12-27 03:35:45,425][105620] Updated weights for policy 1, policy_version 1678412 (0.0006) [2023-12-27 03:35:45,513][105692] Updated weights for policy 0, policy_version 1674856 (0.0010) [2023-12-27 03:35:45,582][105692] Updated weights for policy 0, policy_version 1674866 (0.0010) [2023-12-27 03:35:45,638][105692] Updated weights for policy 0, policy_version 1674876 (0.0011) [2023-12-27 03:35:45,976][105620] Updated weights for policy 1, policy_version 1678422 (0.0006) [2023-12-27 03:35:46,043][105620] Updated weights for policy 1, policy_version 1678432 (0.0005) [2023-12-27 03:35:46,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 858570752. Throughput: 0: 9726.0, 1: 9876.0. Samples: 858540308. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:35:46,063][104569] Avg episode reward: [(0, '9079.789'), (1, '8805.378')] [2023-12-27 03:35:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001674880_428834816.pth... [2023-12-27 03:35:46,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001673728_428539904.pth [2023-12-27 03:35:46,107][105620] Updated weights for policy 1, policy_version 1678442 (0.0006) [2023-12-27 03:35:46,149][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001678448_429744128.pth... [2023-12-27 03:35:46,154][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001677264_429441024.pth [2023-12-27 03:35:46,371][105692] Updated weights for policy 0, policy_version 1674886 (0.0010) [2023-12-27 03:35:46,428][105692] Updated weights for policy 0, policy_version 1674896 (0.0010) [2023-12-27 03:35:46,489][105692] Updated weights for policy 0, policy_version 1674906 (0.0010) [2023-12-27 03:35:46,670][105620] Updated weights for policy 1, policy_version 1678452 (0.0007) [2023-12-27 03:35:46,728][105620] Updated weights for policy 1, policy_version 1678462 (0.0010) [2023-12-27 03:35:46,780][105620] Updated weights for policy 1, policy_version 1678472 (0.0010) [2023-12-27 03:35:47,231][105692] Updated weights for policy 0, policy_version 1674916 (0.0010) [2023-12-27 03:35:47,285][105692] Updated weights for policy 0, policy_version 1674926 (0.0010) [2023-12-27 03:35:47,346][105692] Updated weights for policy 0, policy_version 1674936 (0.0010) [2023-12-27 03:35:47,484][105620] Updated weights for policy 1, policy_version 1678482 (0.0010) [2023-12-27 03:35:47,535][105620] Updated weights for policy 1, policy_version 1678492 (0.0008) [2023-12-27 03:35:47,589][105620] Updated weights for policy 1, policy_version 1678502 (0.0007) [2023-12-27 03:35:47,642][105620] Updated weights for policy 1, policy_version 1678512 (0.0005) [2023-12-27 03:35:48,075][105692] Updated weights for policy 0, policy_version 1674946 (0.0010) [2023-12-27 03:35:48,145][105692] Updated weights for policy 0, policy_version 1674956 (0.0011) [2023-12-27 03:35:48,202][105692] Updated weights for policy 0, policy_version 1674966 (0.0009) [2023-12-27 03:35:48,260][105692] Updated weights for policy 0, policy_version 1674976 (0.0008) [2023-12-27 03:35:48,323][105620] Updated weights for policy 1, policy_version 1678522 (0.0008) [2023-12-27 03:35:48,388][105620] Updated weights for policy 1, policy_version 1678532 (0.0008) [2023-12-27 03:35:48,450][105620] Updated weights for policy 1, policy_version 1678542 (0.0010) [2023-12-27 03:35:48,985][105692] Updated weights for policy 0, policy_version 1674986 (0.0011) [2023-12-27 03:35:49,053][105692] Updated weights for policy 0, policy_version 1674996 (0.0010) [2023-12-27 03:35:49,120][105692] Updated weights for policy 0, policy_version 1675006 (0.0010) [2023-12-27 03:35:49,189][105620] Updated weights for policy 1, policy_version 1678552 (0.0010) [2023-12-27 03:35:49,253][105620] Updated weights for policy 1, policy_version 1678562 (0.0008) [2023-12-27 03:35:49,321][105620] Updated weights for policy 1, policy_version 1678572 (0.0011) [2023-12-27 03:35:49,860][105692] Updated weights for policy 0, policy_version 1675016 (0.0009) [2023-12-27 03:35:49,932][105692] Updated weights for policy 0, policy_version 1675026 (0.0008) [2023-12-27 03:35:50,003][105692] Updated weights for policy 0, policy_version 1675036 (0.0011) [2023-12-27 03:35:50,006][105620] Updated weights for policy 1, policy_version 1678582 (0.0008) [2023-12-27 03:35:50,071][105620] Updated weights for policy 1, policy_version 1678592 (0.0006) [2023-12-27 03:35:50,140][105620] Updated weights for policy 1, policy_version 1678602 (0.0008) [2023-12-27 03:35:50,695][105692] Updated weights for policy 0, policy_version 1675046 (0.0008) [2023-12-27 03:35:50,759][105692] Updated weights for policy 0, policy_version 1675056 (0.0008) [2023-12-27 03:35:50,806][105620] Updated weights for policy 1, policy_version 1678612 (0.0009) [2023-12-27 03:35:50,819][105692] Updated weights for policy 0, policy_version 1675066 (0.0010) [2023-12-27 03:35:50,871][105620] Updated weights for policy 1, policy_version 1678622 (0.0009) [2023-12-27 03:35:50,938][105620] Updated weights for policy 1, policy_version 1678632 (0.0007) [2023-12-27 03:35:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 858677248. Throughput: 0: 9804.4, 1: 9972.8. Samples: 858662296. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:35:51,062][104569] Avg episode reward: [(0, '9170.924'), (1, '8804.434')] [2023-12-27 03:35:51,560][105692] Updated weights for policy 0, policy_version 1675076 (0.0007) [2023-12-27 03:35:51,630][105692] Updated weights for policy 0, policy_version 1675086 (0.0008) [2023-12-27 03:35:51,632][105620] Updated weights for policy 1, policy_version 1678642 (0.0008) [2023-12-27 03:35:51,687][105620] Updated weights for policy 1, policy_version 1678652 (0.0011) [2023-12-27 03:35:51,689][105692] Updated weights for policy 0, policy_version 1675096 (0.0007) [2023-12-27 03:35:51,770][105620] Updated weights for policy 1, policy_version 1678662 (0.0010) [2023-12-27 03:35:51,838][105620] Updated weights for policy 1, policy_version 1678672 (0.0011) [2023-12-27 03:35:52,450][105692] Updated weights for policy 0, policy_version 1675106 (0.0009) [2023-12-27 03:35:52,515][105692] Updated weights for policy 0, policy_version 1675116 (0.0007) [2023-12-27 03:35:52,574][105692] Updated weights for policy 0, policy_version 1675126 (0.0008) [2023-12-27 03:35:52,607][105620] Updated weights for policy 1, policy_version 1678682 (0.0010) [2023-12-27 03:35:52,627][105692] Updated weights for policy 0, policy_version 1675136 (0.0009) [2023-12-27 03:35:52,670][105620] Updated weights for policy 1, policy_version 1678692 (0.0010) [2023-12-27 03:35:52,732][105620] Updated weights for policy 1, policy_version 1678702 (0.0010) [2023-12-27 03:35:53,339][105692] Updated weights for policy 0, policy_version 1675146 (0.0010) [2023-12-27 03:35:53,386][105620] Updated weights for policy 1, policy_version 1678712 (0.0010) [2023-12-27 03:35:53,394][105692] Updated weights for policy 0, policy_version 1675156 (0.0010) [2023-12-27 03:35:53,445][105620] Updated weights for policy 1, policy_version 1678722 (0.0006) [2023-12-27 03:35:53,453][105692] Updated weights for policy 0, policy_version 1675166 (0.0010) [2023-12-27 03:35:53,500][105620] Updated weights for policy 1, policy_version 1678732 (0.0009) [2023-12-27 03:35:54,108][105620] Updated weights for policy 1, policy_version 1678742 (0.0007) [2023-12-27 03:35:54,167][105620] Updated weights for policy 1, policy_version 1678752 (0.0006) [2023-12-27 03:35:54,191][105692] Updated weights for policy 0, policy_version 1675176 (0.0010) [2023-12-27 03:35:54,225][105620] Updated weights for policy 1, policy_version 1678762 (0.0007) [2023-12-27 03:35:54,250][105692] Updated weights for policy 0, policy_version 1675186 (0.0010) [2023-12-27 03:35:54,311][105692] Updated weights for policy 0, policy_version 1675196 (0.0010) [2023-12-27 03:35:54,931][105620] Updated weights for policy 1, policy_version 1678772 (0.0006) [2023-12-27 03:35:54,985][105620] Updated weights for policy 1, policy_version 1678782 (0.0007) [2023-12-27 03:35:55,040][105620] Updated weights for policy 1, policy_version 1678792 (0.0008) [2023-12-27 03:35:55,044][105692] Updated weights for policy 0, policy_version 1675206 (0.0009) [2023-12-27 03:35:55,098][105692] Updated weights for policy 0, policy_version 1675216 (0.0006) [2023-12-27 03:35:55,153][105692] Updated weights for policy 0, policy_version 1675226 (0.0009) [2023-12-27 03:35:55,761][105692] Updated weights for policy 0, policy_version 1675236 (0.0009) [2023-12-27 03:35:55,817][105692] Updated weights for policy 0, policy_version 1675246 (0.0011) [2023-12-27 03:35:55,826][105620] Updated weights for policy 1, policy_version 1678802 (0.0007) [2023-12-27 03:35:55,875][105692] Updated weights for policy 0, policy_version 1675256 (0.0010) [2023-12-27 03:35:55,881][105620] Updated weights for policy 1, policy_version 1678812 (0.0006) [2023-12-27 03:35:55,939][105620] Updated weights for policy 1, policy_version 1678822 (0.0009) [2023-12-27 03:35:55,991][105620] Updated weights for policy 1, policy_version 1678832 (0.0009) [2023-12-27 03:35:56,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 858775552. Throughput: 0: 9740.5, 1: 10029.0. Samples: 858778232. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:35:56,062][104569] Avg episode reward: [(0, '8897.738'), (1, '9083.854')] [2023-12-27 03:35:56,532][105692] Updated weights for policy 0, policy_version 1675266 (0.0007) [2023-12-27 03:35:56,591][105692] Updated weights for policy 0, policy_version 1675276 (0.0010) [2023-12-27 03:35:56,651][105692] Updated weights for policy 0, policy_version 1675286 (0.0010) [2023-12-27 03:35:56,708][105692] Updated weights for policy 0, policy_version 1675296 (0.0008) [2023-12-27 03:35:56,725][105620] Updated weights for policy 1, policy_version 1678842 (0.0008) [2023-12-27 03:35:56,785][105620] Updated weights for policy 1, policy_version 1678853 (0.0009) [2023-12-27 03:35:57,330][105692] Updated weights for policy 0, policy_version 1675306 (0.0010) [2023-12-27 03:35:57,385][105692] Updated weights for policy 0, policy_version 1675316 (0.0011) [2023-12-27 03:35:57,440][105692] Updated weights for policy 0, policy_version 1675326 (0.0010) [2023-12-27 03:35:57,474][105620] Updated weights for policy 1, policy_version 1678865 (0.0010) [2023-12-27 03:35:57,525][105620] Updated weights for policy 1, policy_version 1678875 (0.0010) [2023-12-27 03:35:57,582][105620] Updated weights for policy 1, policy_version 1678885 (0.0011) [2023-12-27 03:35:57,644][105620] Updated weights for policy 1, policy_version 1678895 (0.0010) [2023-12-27 03:35:58,195][105692] Updated weights for policy 0, policy_version 1675336 (0.0011) [2023-12-27 03:35:58,254][105692] Updated weights for policy 0, policy_version 1675346 (0.0011) [2023-12-27 03:35:58,316][105692] Updated weights for policy 0, policy_version 1675356 (0.0011) [2023-12-27 03:35:58,320][105620] Updated weights for policy 1, policy_version 1678905 (0.0007) [2023-12-27 03:35:58,387][105620] Updated weights for policy 1, policy_version 1678915 (0.0008) [2023-12-27 03:35:58,455][105620] Updated weights for policy 1, policy_version 1678925 (0.0008) [2023-12-27 03:35:59,096][105692] Updated weights for policy 0, policy_version 1675366 (0.0010) [2023-12-27 03:35:59,140][105692] Updated weights for policy 0, policy_version 1675376 (0.0010) [2023-12-27 03:35:59,201][105692] Updated weights for policy 0, policy_version 1675386 (0.0006) [2023-12-27 03:35:59,220][105620] Updated weights for policy 1, policy_version 1678935 (0.0008) [2023-12-27 03:35:59,283][105620] Updated weights for policy 1, policy_version 1678945 (0.0008) [2023-12-27 03:35:59,345][105620] Updated weights for policy 1, policy_version 1678955 (0.0008) [2023-12-27 03:35:59,939][105692] Updated weights for policy 0, policy_version 1675396 (0.0009) [2023-12-27 03:36:00,000][105692] Updated weights for policy 0, policy_version 1675406 (0.0010) [2023-12-27 03:36:00,062][105692] Updated weights for policy 0, policy_version 1675416 (0.0010) [2023-12-27 03:36:00,087][105620] Updated weights for policy 1, policy_version 1678965 (0.0008) [2023-12-27 03:36:00,145][105620] Updated weights for policy 1, policy_version 1678975 (0.0007) [2023-12-27 03:36:00,201][105620] Updated weights for policy 1, policy_version 1678985 (0.0008) [2023-12-27 03:36:00,736][105692] Updated weights for policy 0, policy_version 1675426 (0.0009) [2023-12-27 03:36:00,786][105692] Updated weights for policy 0, policy_version 1675436 (0.0005) [2023-12-27 03:36:00,843][105692] Updated weights for policy 0, policy_version 1675446 (0.0005) [2023-12-27 03:36:00,908][105692] Updated weights for policy 0, policy_version 1675456 (0.0006) [2023-12-27 03:36:00,988][105620] Updated weights for policy 1, policy_version 1678995 (0.0007) [2023-12-27 03:36:01,040][105620] Updated weights for policy 1, policy_version 1679005 (0.0007) [2023-12-27 03:36:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 858865664. Throughput: 0: 9801.2, 1: 10003.5. Samples: 858838004. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:36:01,063][104569] Avg episode reward: [(0, '8530.588'), (1, '9084.312')] [2023-12-27 03:36:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001675456_428982272.pth... [2023-12-27 03:36:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001674304_428687360.pth [2023-12-27 03:36:01,107][105620] Updated weights for policy 1, policy_version 1679015 (0.0009) [2023-12-27 03:36:01,171][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001679024_429891584.pth... [2023-12-27 03:36:01,175][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001677840_429588480.pth [2023-12-27 03:36:01,546][105692] Updated weights for policy 0, policy_version 1675466 (0.0010) [2023-12-27 03:36:01,595][105692] Updated weights for policy 0, policy_version 1675476 (0.0008) [2023-12-27 03:36:01,665][105692] Updated weights for policy 0, policy_version 1675486 (0.0010) [2023-12-27 03:36:01,824][105620] Updated weights for policy 1, policy_version 1679025 (0.0009) [2023-12-27 03:36:01,877][105620] Updated weights for policy 1, policy_version 1679035 (0.0009) [2023-12-27 03:36:01,925][105620] Updated weights for policy 1, policy_version 1679045 (0.0006) [2023-12-27 03:36:01,987][105620] Updated weights for policy 1, policy_version 1679055 (0.0007) [2023-12-27 03:36:02,395][105692] Updated weights for policy 0, policy_version 1675496 (0.0009) [2023-12-27 03:36:02,446][105692] Updated weights for policy 0, policy_version 1675506 (0.0009) [2023-12-27 03:36:02,493][105692] Updated weights for policy 0, policy_version 1675516 (0.0009) [2023-12-27 03:36:02,742][105620] Updated weights for policy 1, policy_version 1679065 (0.0009) [2023-12-27 03:36:02,803][105620] Updated weights for policy 1, policy_version 1679075 (0.0009) [2023-12-27 03:36:02,864][105620] Updated weights for policy 1, policy_version 1679085 (0.0009) [2023-12-27 03:36:03,247][105692] Updated weights for policy 0, policy_version 1675526 (0.0007) [2023-12-27 03:36:03,293][105692] Updated weights for policy 0, policy_version 1675536 (0.0006) [2023-12-27 03:36:03,342][105692] Updated weights for policy 0, policy_version 1675546 (0.0008) [2023-12-27 03:36:03,546][105620] Updated weights for policy 1, policy_version 1679095 (0.0007) [2023-12-27 03:36:03,593][105620] Updated weights for policy 1, policy_version 1679105 (0.0005) [2023-12-27 03:36:03,649][105620] Updated weights for policy 1, policy_version 1679115 (0.0005) [2023-12-27 03:36:04,174][105692] Updated weights for policy 0, policy_version 1675556 (0.0008) [2023-12-27 03:36:04,237][105692] Updated weights for policy 0, policy_version 1675566 (0.0009) [2023-12-27 03:36:04,261][105620] Updated weights for policy 1, policy_version 1679125 (0.0007) [2023-12-27 03:36:04,299][105692] Updated weights for policy 0, policy_version 1675576 (0.0007) [2023-12-27 03:36:04,320][105620] Updated weights for policy 1, policy_version 1679135 (0.0008) [2023-12-27 03:36:04,375][105620] Updated weights for policy 1, policy_version 1679145 (0.0007) [2023-12-27 03:36:04,922][105692] Updated weights for policy 0, policy_version 1675586 (0.0007) [2023-12-27 03:36:04,976][105692] Updated weights for policy 0, policy_version 1675596 (0.0006) [2023-12-27 03:36:05,036][105692] Updated weights for policy 0, policy_version 1675606 (0.0009) [2023-12-27 03:36:05,084][105692] Updated weights for policy 0, policy_version 1675616 (0.0008) [2023-12-27 03:36:05,161][105620] Updated weights for policy 1, policy_version 1679155 (0.0009) [2023-12-27 03:36:05,217][105620] Updated weights for policy 1, policy_version 1679165 (0.0010) [2023-12-27 03:36:05,279][105620] Updated weights for policy 1, policy_version 1679175 (0.0010) [2023-12-27 03:36:05,840][105692] Updated weights for policy 0, policy_version 1675626 (0.0008) [2023-12-27 03:36:05,899][105692] Updated weights for policy 0, policy_version 1675636 (0.0008) [2023-12-27 03:36:05,955][105692] Updated weights for policy 0, policy_version 1675646 (0.0008) [2023-12-27 03:36:05,995][105620] Updated weights for policy 1, policy_version 1679185 (0.0010) [2023-12-27 03:36:06,056][105620] Updated weights for policy 1, policy_version 1679195 (0.0005) [2023-12-27 03:36:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 858963968. Throughput: 0: 9742.7, 1: 9926.6. Samples: 858952308. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:36:06,062][104569] Avg episode reward: [(0, '8346.511'), (1, '8805.104')] [2023-12-27 03:36:06,121][105620] Updated weights for policy 1, policy_version 1679205 (0.0008) [2023-12-27 03:36:06,186][105620] Updated weights for policy 1, policy_version 1679215 (0.0009) [2023-12-27 03:36:06,662][105692] Updated weights for policy 0, policy_version 1675656 (0.0010) [2023-12-27 03:36:06,722][105692] Updated weights for policy 0, policy_version 1675666 (0.0011) [2023-12-27 03:36:06,781][105692] Updated weights for policy 0, policy_version 1675676 (0.0011) [2023-12-27 03:36:06,921][105620] Updated weights for policy 1, policy_version 1679225 (0.0008) [2023-12-27 03:36:06,985][105620] Updated weights for policy 1, policy_version 1679235 (0.0009) [2023-12-27 03:36:07,037][105620] Updated weights for policy 1, policy_version 1679245 (0.0006) [2023-12-27 03:36:07,519][105692] Updated weights for policy 0, policy_version 1675686 (0.0007) [2023-12-27 03:36:07,587][105692] Updated weights for policy 0, policy_version 1675696 (0.0005) [2023-12-27 03:36:07,644][105692] Updated weights for policy 0, policy_version 1675706 (0.0005) [2023-12-27 03:36:07,727][105620] Updated weights for policy 1, policy_version 1679255 (0.0005) [2023-12-27 03:36:07,793][105620] Updated weights for policy 1, policy_version 1679265 (0.0010) [2023-12-27 03:36:07,858][105620] Updated weights for policy 1, policy_version 1679275 (0.0010) [2023-12-27 03:36:08,146][105692] Updated weights for policy 0, policy_version 1675716 (0.0006) [2023-12-27 03:36:08,218][105692] Updated weights for policy 0, policy_version 1675726 (0.0008) [2023-12-27 03:36:08,285][105692] Updated weights for policy 0, policy_version 1675736 (0.0007) [2023-12-27 03:36:08,468][105620] Updated weights for policy 1, policy_version 1679285 (0.0008) [2023-12-27 03:36:08,537][105620] Updated weights for policy 1, policy_version 1679295 (0.0009) [2023-12-27 03:36:08,602][105620] Updated weights for policy 1, policy_version 1679305 (0.0010) [2023-12-27 03:36:08,938][105692] Updated weights for policy 0, policy_version 1675746 (0.0007) [2023-12-27 03:36:09,001][105692] Updated weights for policy 0, policy_version 1675756 (0.0011) [2023-12-27 03:36:09,053][105692] Updated weights for policy 0, policy_version 1675766 (0.0010) [2023-12-27 03:36:09,106][105692] Updated weights for policy 0, policy_version 1675776 (0.0011) [2023-12-27 03:36:09,315][105620] Updated weights for policy 1, policy_version 1679315 (0.0011) [2023-12-27 03:36:09,383][105620] Updated weights for policy 1, policy_version 1679325 (0.0010) [2023-12-27 03:36:09,454][105620] Updated weights for policy 1, policy_version 1679335 (0.0008) [2023-12-27 03:36:09,896][105692] Updated weights for policy 0, policy_version 1675786 (0.0008) [2023-12-27 03:36:09,963][105692] Updated weights for policy 0, policy_version 1675796 (0.0008) [2023-12-27 03:36:10,031][105692] Updated weights for policy 0, policy_version 1675806 (0.0008) [2023-12-27 03:36:10,124][105620] Updated weights for policy 1, policy_version 1679345 (0.0008) [2023-12-27 03:36:10,188][105620] Updated weights for policy 1, policy_version 1679355 (0.0010) [2023-12-27 03:36:10,255][105620] Updated weights for policy 1, policy_version 1679365 (0.0011) [2023-12-27 03:36:10,318][105620] Updated weights for policy 1, policy_version 1679375 (0.0011) [2023-12-27 03:36:10,818][105692] Updated weights for policy 0, policy_version 1675816 (0.0009) [2023-12-27 03:36:10,884][105692] Updated weights for policy 0, policy_version 1675826 (0.0008) [2023-12-27 03:36:10,945][105692] Updated weights for policy 0, policy_version 1675836 (0.0009) [2023-12-27 03:36:11,060][105620] Updated weights for policy 1, policy_version 1679385 (0.0007) [2023-12-27 03:36:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 859062272. Throughput: 0: 9859.4, 1: 9871.8. Samples: 859071380. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:36:11,063][104569] Avg episode reward: [(0, '8713.699'), (1, '8988.271')] [2023-12-27 03:36:11,125][105620] Updated weights for policy 1, policy_version 1679395 (0.0008) [2023-12-27 03:36:11,189][105620] Updated weights for policy 1, policy_version 1679405 (0.0008) [2023-12-27 03:36:11,705][105692] Updated weights for policy 0, policy_version 1675846 (0.0010) [2023-12-27 03:36:11,778][105692] Updated weights for policy 0, policy_version 1675856 (0.0008) [2023-12-27 03:36:11,838][105692] Updated weights for policy 0, policy_version 1675866 (0.0009) [2023-12-27 03:36:11,968][105620] Updated weights for policy 1, policy_version 1679415 (0.0008) [2023-12-27 03:36:12,024][105620] Updated weights for policy 1, policy_version 1679425 (0.0008) [2023-12-27 03:36:12,072][105620] Updated weights for policy 1, policy_version 1679435 (0.0008) [2023-12-27 03:36:12,593][105692] Updated weights for policy 0, policy_version 1675876 (0.0009) [2023-12-27 03:36:12,645][105692] Updated weights for policy 0, policy_version 1675886 (0.0009) [2023-12-27 03:36:12,704][105692] Updated weights for policy 0, policy_version 1675896 (0.0008) [2023-12-27 03:36:12,831][105620] Updated weights for policy 1, policy_version 1679445 (0.0008) [2023-12-27 03:36:12,892][105620] Updated weights for policy 1, policy_version 1679455 (0.0005) [2023-12-27 03:36:12,951][105620] Updated weights for policy 1, policy_version 1679465 (0.0005) [2023-12-27 03:36:13,508][105692] Updated weights for policy 0, policy_version 1675906 (0.0008) [2023-12-27 03:36:13,565][105692] Updated weights for policy 0, policy_version 1675916 (0.0009) [2023-12-27 03:36:13,617][105620] Updated weights for policy 1, policy_version 1679475 (0.0008) [2023-12-27 03:36:13,630][105692] Updated weights for policy 0, policy_version 1675926 (0.0008) [2023-12-27 03:36:13,666][105620] Updated weights for policy 1, policy_version 1679485 (0.0007) [2023-12-27 03:36:13,684][105692] Updated weights for policy 0, policy_version 1675936 (0.0007) [2023-12-27 03:36:13,718][105620] Updated weights for policy 1, policy_version 1679495 (0.0008) [2023-12-27 03:36:14,316][105620] Updated weights for policy 1, policy_version 1679505 (0.0008) [2023-12-27 03:36:14,370][105620] Updated weights for policy 1, policy_version 1679515 (0.0008) [2023-12-27 03:36:14,432][105620] Updated weights for policy 1, policy_version 1679525 (0.0009) [2023-12-27 03:36:14,496][105620] Updated weights for policy 1, policy_version 1679535 (0.0009) [2023-12-27 03:36:14,501][105692] Updated weights for policy 0, policy_version 1675946 (0.0009) [2023-12-27 03:36:14,562][105692] Updated weights for policy 0, policy_version 1675956 (0.0009) [2023-12-27 03:36:14,622][105692] Updated weights for policy 0, policy_version 1675966 (0.0009) [2023-12-27 03:36:15,202][105620] Updated weights for policy 1, policy_version 1679545 (0.0009) [2023-12-27 03:36:15,256][105620] Updated weights for policy 1, policy_version 1679555 (0.0007) [2023-12-27 03:36:15,312][105620] Updated weights for policy 1, policy_version 1679565 (0.0005) [2023-12-27 03:36:15,417][105692] Updated weights for policy 0, policy_version 1675976 (0.0009) [2023-12-27 03:36:15,467][105692] Updated weights for policy 0, policy_version 1675986 (0.0009) [2023-12-27 03:36:15,520][105692] Updated weights for policy 0, policy_version 1675996 (0.0008) [2023-12-27 03:36:16,015][105620] Updated weights for policy 1, policy_version 1679575 (0.0008) [2023-12-27 03:36:16,061][105620] Updated weights for policy 1, policy_version 1679585 (0.0008) [2023-12-27 03:36:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 859152384. Throughput: 0: 9769.4, 1: 9829.0. Samples: 859126964. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:36:16,062][104569] Avg episode reward: [(0, '9170.617'), (1, '9356.201')] [2023-12-27 03:36:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001676000_429121536.pth... [2023-12-27 03:36:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001674880_428834816.pth [2023-12-27 03:36:16,115][105620] Updated weights for policy 1, policy_version 1679595 (0.0009) [2023-12-27 03:36:16,145][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001679600_430039040.pth... [2023-12-27 03:36:16,150][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001678448_429744128.pth [2023-12-27 03:36:16,277][105692] Updated weights for policy 0, policy_version 1676006 (0.0009) [2023-12-27 03:36:16,334][105692] Updated weights for policy 0, policy_version 1676016 (0.0009) [2023-12-27 03:36:16,396][105692] Updated weights for policy 0, policy_version 1676026 (0.0009) [2023-12-27 03:36:16,829][105620] Updated weights for policy 1, policy_version 1679605 (0.0007) [2023-12-27 03:36:16,892][105620] Updated weights for policy 1, policy_version 1679615 (0.0005) [2023-12-27 03:36:16,946][105620] Updated weights for policy 1, policy_version 1679625 (0.0006) [2023-12-27 03:36:17,117][105692] Updated weights for policy 0, policy_version 1676036 (0.0009) [2023-12-27 03:36:17,182][105692] Updated weights for policy 0, policy_version 1676046 (0.0007) [2023-12-27 03:36:17,246][105692] Updated weights for policy 0, policy_version 1676056 (0.0005) [2023-12-27 03:36:17,578][105620] Updated weights for policy 1, policy_version 1679635 (0.0010) [2023-12-27 03:36:17,643][105620] Updated weights for policy 1, policy_version 1679645 (0.0007) [2023-12-27 03:36:17,708][105620] Updated weights for policy 1, policy_version 1679655 (0.0005) [2023-12-27 03:36:17,883][105692] Updated weights for policy 0, policy_version 1676066 (0.0008) [2023-12-27 03:36:17,948][105692] Updated weights for policy 0, policy_version 1676076 (0.0011) [2023-12-27 03:36:18,016][105692] Updated weights for policy 0, policy_version 1676086 (0.0011) [2023-12-27 03:36:18,082][105692] Updated weights for policy 0, policy_version 1676096 (0.0011) [2023-12-27 03:36:18,266][105620] Updated weights for policy 1, policy_version 1679665 (0.0006) [2023-12-27 03:36:18,323][105620] Updated weights for policy 1, policy_version 1679675 (0.0010) [2023-12-27 03:36:18,385][105620] Updated weights for policy 1, policy_version 1679685 (0.0009) [2023-12-27 03:36:18,445][105620] Updated weights for policy 1, policy_version 1679695 (0.0011) [2023-12-27 03:36:18,754][105692] Updated weights for policy 0, policy_version 1676106 (0.0011) [2023-12-27 03:36:18,820][105692] Updated weights for policy 0, policy_version 1676116 (0.0011) [2023-12-27 03:36:18,882][105692] Updated weights for policy 0, policy_version 1676126 (0.0010) [2023-12-27 03:36:19,197][105620] Updated weights for policy 1, policy_version 1679705 (0.0009) [2023-12-27 03:36:19,261][105620] Updated weights for policy 1, policy_version 1679715 (0.0007) [2023-12-27 03:36:19,337][105620] Updated weights for policy 1, policy_version 1679725 (0.0008) [2023-12-27 03:36:19,582][105692] Updated weights for policy 0, policy_version 1676136 (0.0009) [2023-12-27 03:36:19,649][105692] Updated weights for policy 0, policy_version 1676146 (0.0011) [2023-12-27 03:36:19,712][105692] Updated weights for policy 0, policy_version 1676156 (0.0006) [2023-12-27 03:36:20,056][105620] Updated weights for policy 1, policy_version 1679735 (0.0008) [2023-12-27 03:36:20,117][105620] Updated weights for policy 1, policy_version 1679745 (0.0009) [2023-12-27 03:36:20,172][105620] Updated weights for policy 1, policy_version 1679755 (0.0009) [2023-12-27 03:36:20,403][105692] Updated weights for policy 0, policy_version 1676166 (0.0007) [2023-12-27 03:36:20,463][105692] Updated weights for policy 0, policy_version 1676176 (0.0005) [2023-12-27 03:36:20,529][105692] Updated weights for policy 0, policy_version 1676186 (0.0005) [2023-12-27 03:36:20,914][105620] Updated weights for policy 1, policy_version 1679765 (0.0008) [2023-12-27 03:36:20,981][105620] Updated weights for policy 1, policy_version 1679775 (0.0009) [2023-12-27 03:36:21,055][105620] Updated weights for policy 1, policy_version 1679785 (0.0009) [2023-12-27 03:36:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 859250688. Throughput: 0: 9683.7, 1: 9975.5. Samples: 859245932. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:36:21,062][104569] Avg episode reward: [(0, '8898.530'), (1, '9263.864')] [2023-12-27 03:36:21,129][105692] Updated weights for policy 0, policy_version 1676196 (0.0009) [2023-12-27 03:36:21,191][105692] Updated weights for policy 0, policy_version 1676206 (0.0011) [2023-12-27 03:36:21,252][105692] Updated weights for policy 0, policy_version 1676216 (0.0010) [2023-12-27 03:36:21,831][105620] Updated weights for policy 1, policy_version 1679795 (0.0008) [2023-12-27 03:36:21,894][105620] Updated weights for policy 1, policy_version 1679805 (0.0008) [2023-12-27 03:36:21,946][105620] Updated weights for policy 1, policy_version 1679815 (0.0008) [2023-12-27 03:36:22,045][105692] Updated weights for policy 0, policy_version 1676226 (0.0011) [2023-12-27 03:36:22,109][105692] Updated weights for policy 0, policy_version 1676236 (0.0011) [2023-12-27 03:36:22,171][105692] Updated weights for policy 0, policy_version 1676246 (0.0011) [2023-12-27 03:36:22,224][105692] Updated weights for policy 0, policy_version 1676256 (0.0010) [2023-12-27 03:36:22,678][105620] Updated weights for policy 1, policy_version 1679825 (0.0008) [2023-12-27 03:36:22,736][105620] Updated weights for policy 1, policy_version 1679835 (0.0008) [2023-12-27 03:36:22,789][105620] Updated weights for policy 1, policy_version 1679845 (0.0008) [2023-12-27 03:36:22,846][105620] Updated weights for policy 1, policy_version 1679855 (0.0008) [2023-12-27 03:36:22,994][105692] Updated weights for policy 0, policy_version 1676266 (0.0011) [2023-12-27 03:36:23,049][105692] Updated weights for policy 0, policy_version 1676276 (0.0011) [2023-12-27 03:36:23,094][105692] Updated weights for policy 0, policy_version 1676286 (0.0010) [2023-12-27 03:36:23,615][105620] Updated weights for policy 1, policy_version 1679865 (0.0009) [2023-12-27 03:36:23,679][105620] Updated weights for policy 1, policy_version 1679875 (0.0006) [2023-12-27 03:36:23,737][105620] Updated weights for policy 1, policy_version 1679885 (0.0008) [2023-12-27 03:36:23,821][105692] Updated weights for policy 0, policy_version 1676296 (0.0006) [2023-12-27 03:36:23,868][105692] Updated weights for policy 0, policy_version 1676306 (0.0009) [2023-12-27 03:36:23,919][105692] Updated weights for policy 0, policy_version 1676316 (0.0009) [2023-12-27 03:36:24,466][105692] Updated weights for policy 0, policy_version 1676326 (0.0006) [2023-12-27 03:36:24,514][105692] Updated weights for policy 0, policy_version 1676336 (0.0005) [2023-12-27 03:36:24,566][105692] Updated weights for policy 0, policy_version 1676346 (0.0011) [2023-12-27 03:36:24,592][105620] Updated weights for policy 1, policy_version 1679895 (0.0007) [2023-12-27 03:36:24,649][105620] Updated weights for policy 1, policy_version 1679905 (0.0008) [2023-12-27 03:36:24,712][105620] Updated weights for policy 1, policy_version 1679915 (0.0009) [2023-12-27 03:36:25,135][105692] Updated weights for policy 0, policy_version 1676356 (0.0008) [2023-12-27 03:36:25,191][105692] Updated weights for policy 0, policy_version 1676366 (0.0006) [2023-12-27 03:36:25,244][105692] Updated weights for policy 0, policy_version 1676376 (0.0011) [2023-12-27 03:36:25,533][105620] Updated weights for policy 1, policy_version 1679925 (0.0008) [2023-12-27 03:36:25,593][105620] Updated weights for policy 1, policy_version 1679935 (0.0006) [2023-12-27 03:36:25,658][105620] Updated weights for policy 1, policy_version 1679945 (0.0008) [2023-12-27 03:36:25,942][105692] Updated weights for policy 0, policy_version 1676386 (0.0011) [2023-12-27 03:36:25,990][105692] Updated weights for policy 0, policy_version 1676396 (0.0011) [2023-12-27 03:36:26,039][105692] Updated weights for policy 0, policy_version 1676406 (0.0010) [2023-12-27 03:36:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 859348992. Throughput: 0: 9699.7, 1: 9851.4. Samples: 859361740. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:36:26,062][104569] Avg episode reward: [(0, '8629.358'), (1, '9173.175')] [2023-12-27 03:36:26,087][105692] Updated weights for policy 0, policy_version 1676416 (0.0010) [2023-12-27 03:36:26,355][105620] Updated weights for policy 1, policy_version 1679955 (0.0008) [2023-12-27 03:36:26,407][105620] Updated weights for policy 1, policy_version 1679965 (0.0008) [2023-12-27 03:36:26,455][105620] Updated weights for policy 1, policy_version 1679975 (0.0008) [2023-12-27 03:36:26,841][105692] Updated weights for policy 0, policy_version 1676426 (0.0011) [2023-12-27 03:36:26,898][105692] Updated weights for policy 0, policy_version 1676436 (0.0010) [2023-12-27 03:36:26,958][105692] Updated weights for policy 0, policy_version 1676446 (0.0010) [2023-12-27 03:36:27,235][105620] Updated weights for policy 1, policy_version 1679985 (0.0008) [2023-12-27 03:36:27,289][105620] Updated weights for policy 1, policy_version 1679995 (0.0008) [2023-12-27 03:36:27,337][105620] Updated weights for policy 1, policy_version 1680005 (0.0008) [2023-12-27 03:36:27,384][105620] Updated weights for policy 1, policy_version 1680015 (0.0008) [2023-12-27 03:36:27,669][105692] Updated weights for policy 0, policy_version 1676456 (0.0011) [2023-12-27 03:36:27,735][105692] Updated weights for policy 0, policy_version 1676466 (0.0011) [2023-12-27 03:36:27,799][105692] Updated weights for policy 0, policy_version 1676476 (0.0010) [2023-12-27 03:36:28,182][105620] Updated weights for policy 1, policy_version 1680025 (0.0007) [2023-12-27 03:36:28,236][105620] Updated weights for policy 1, policy_version 1680035 (0.0008) [2023-12-27 03:36:28,290][105620] Updated weights for policy 1, policy_version 1680045 (0.0008) [2023-12-27 03:36:28,484][105692] Updated weights for policy 0, policy_version 1676486 (0.0011) [2023-12-27 03:36:28,539][105692] Updated weights for policy 0, policy_version 1676496 (0.0011) [2023-12-27 03:36:28,588][105692] Updated weights for policy 0, policy_version 1676506 (0.0010) [2023-12-27 03:36:29,066][105620] Updated weights for policy 1, policy_version 1680055 (0.0008) [2023-12-27 03:36:29,113][105620] Updated weights for policy 1, policy_version 1680065 (0.0007) [2023-12-27 03:36:29,161][105620] Updated weights for policy 1, policy_version 1680075 (0.0008) [2023-12-27 03:36:29,346][105692] Updated weights for policy 0, policy_version 1676516 (0.0011) [2023-12-27 03:36:29,412][105692] Updated weights for policy 0, policy_version 1676526 (0.0011) [2023-12-27 03:36:29,474][105692] Updated weights for policy 0, policy_version 1676536 (0.0011) [2023-12-27 03:36:29,951][105620] Updated weights for policy 1, policy_version 1680085 (0.0007) [2023-12-27 03:36:30,011][105620] Updated weights for policy 1, policy_version 1680095 (0.0006) [2023-12-27 03:36:30,078][105620] Updated weights for policy 1, policy_version 1680105 (0.0008) [2023-12-27 03:36:30,135][105692] Updated weights for policy 0, policy_version 1676546 (0.0010) [2023-12-27 03:36:30,203][105692] Updated weights for policy 0, policy_version 1676556 (0.0010) [2023-12-27 03:36:30,262][105692] Updated weights for policy 0, policy_version 1676566 (0.0010) [2023-12-27 03:36:30,320][105692] Updated weights for policy 0, policy_version 1676576 (0.0010) [2023-12-27 03:36:30,761][105620] Updated weights for policy 1, policy_version 1680115 (0.0007) [2023-12-27 03:36:30,821][105620] Updated weights for policy 1, policy_version 1680125 (0.0006) [2023-12-27 03:36:30,866][105620] Updated weights for policy 1, policy_version 1680135 (0.0005) [2023-12-27 03:36:30,962][105692] Updated weights for policy 0, policy_version 1676586 (0.0005) [2023-12-27 03:36:31,016][105692] Updated weights for policy 0, policy_version 1676596 (0.0005) [2023-12-27 03:36:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 859447296. Throughput: 0: 9724.4, 1: 9795.9. Samples: 859418724. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:36:31,062][104569] Avg episode reward: [(0, '8442.729'), (1, '8901.104')] [2023-12-27 03:36:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001680144_430178304.pth... [2023-12-27 03:36:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001679024_429891584.pth [2023-12-27 03:36:31,078][105692] Updated weights for policy 0, policy_version 1676606 (0.0007) [2023-12-27 03:36:31,087][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001676608_429277184.pth... [2023-12-27 03:36:31,090][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001675456_428982272.pth [2023-12-27 03:36:31,591][105620] Updated weights for policy 1, policy_version 1680145 (0.0009) [2023-12-27 03:36:31,656][105620] Updated weights for policy 1, policy_version 1680155 (0.0008) [2023-12-27 03:36:31,719][105620] Updated weights for policy 1, policy_version 1680165 (0.0009) [2023-12-27 03:36:31,780][105620] Updated weights for policy 1, policy_version 1680175 (0.0007) [2023-12-27 03:36:31,788][105692] Updated weights for policy 0, policy_version 1676616 (0.0006) [2023-12-27 03:36:31,852][105692] Updated weights for policy 0, policy_version 1676626 (0.0005) [2023-12-27 03:36:31,912][105692] Updated weights for policy 0, policy_version 1676636 (0.0005) [2023-12-27 03:36:32,434][105620] Updated weights for policy 1, policy_version 1680185 (0.0008) [2023-12-27 03:36:32,494][105620] Updated weights for policy 1, policy_version 1680195 (0.0007) [2023-12-27 03:36:32,539][105620] Updated weights for policy 1, policy_version 1680205 (0.0008) [2023-12-27 03:36:32,600][105692] Updated weights for policy 0, policy_version 1676646 (0.0008) [2023-12-27 03:36:32,657][105692] Updated weights for policy 0, policy_version 1676656 (0.0007) [2023-12-27 03:36:32,723][105692] Updated weights for policy 0, policy_version 1676666 (0.0005) [2023-12-27 03:36:33,311][105620] Updated weights for policy 1, policy_version 1680215 (0.0008) [2023-12-27 03:36:33,367][105620] Updated weights for policy 1, policy_version 1680225 (0.0008) [2023-12-27 03:36:33,389][105692] Updated weights for policy 0, policy_version 1676676 (0.0008) [2023-12-27 03:36:33,412][105620] Updated weights for policy 1, policy_version 1680235 (0.0005) [2023-12-27 03:36:33,437][105692] Updated weights for policy 0, policy_version 1676686 (0.0010) [2023-12-27 03:36:33,491][105692] Updated weights for policy 0, policy_version 1676696 (0.0010) [2023-12-27 03:36:34,137][105692] Updated weights for policy 0, policy_version 1676706 (0.0010) [2023-12-27 03:36:34,150][105620] Updated weights for policy 1, policy_version 1680245 (0.0008) [2023-12-27 03:36:34,198][105692] Updated weights for policy 0, policy_version 1676716 (0.0010) [2023-12-27 03:36:34,217][105620] Updated weights for policy 1, policy_version 1680255 (0.0010) [2023-12-27 03:36:34,261][105692] Updated weights for policy 0, policy_version 1676726 (0.0011) [2023-12-27 03:36:34,280][105620] Updated weights for policy 1, policy_version 1680265 (0.0010) [2023-12-27 03:36:34,323][105692] Updated weights for policy 0, policy_version 1676736 (0.0010) [2023-12-27 03:36:34,987][105620] Updated weights for policy 1, policy_version 1680275 (0.0009) [2023-12-27 03:36:35,047][105620] Updated weights for policy 1, policy_version 1680285 (0.0005) [2023-12-27 03:36:35,049][105692] Updated weights for policy 0, policy_version 1676746 (0.0011) [2023-12-27 03:36:35,102][105620] Updated weights for policy 1, policy_version 1680295 (0.0005) [2023-12-27 03:36:35,103][105692] Updated weights for policy 0, policy_version 1676756 (0.0010) [2023-12-27 03:36:35,159][105692] Updated weights for policy 0, policy_version 1676766 (0.0010) [2023-12-27 03:36:35,789][105620] Updated weights for policy 1, policy_version 1680305 (0.0005) [2023-12-27 03:36:35,854][105620] Updated weights for policy 1, policy_version 1680315 (0.0005) [2023-12-27 03:36:35,909][105692] Updated weights for policy 0, policy_version 1676776 (0.0006) [2023-12-27 03:36:35,915][105620] Updated weights for policy 1, policy_version 1680325 (0.0008) [2023-12-27 03:36:35,967][105692] Updated weights for policy 0, policy_version 1676786 (0.0006) [2023-12-27 03:36:35,969][105620] Updated weights for policy 1, policy_version 1680335 (0.0008) [2023-12-27 03:36:36,025][105692] Updated weights for policy 0, policy_version 1676796 (0.0008) [2023-12-27 03:36:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 859553792. Throughput: 0: 9780.5, 1: 9652.0. Samples: 859536760. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:36:36,062][104569] Avg episode reward: [(0, '8439.496'), (1, '8717.565')] [2023-12-27 03:36:36,663][105620] Updated weights for policy 1, policy_version 1680345 (0.0008) [2023-12-27 03:36:36,731][105620] Updated weights for policy 1, policy_version 1680355 (0.0007) [2023-12-27 03:36:36,749][105692] Updated weights for policy 0, policy_version 1676806 (0.0011) [2023-12-27 03:36:36,801][105620] Updated weights for policy 1, policy_version 1680365 (0.0008) [2023-12-27 03:36:36,803][105692] Updated weights for policy 0, policy_version 1676816 (0.0011) [2023-12-27 03:36:36,874][105692] Updated weights for policy 0, policy_version 1676826 (0.0011) [2023-12-27 03:36:37,471][105620] Updated weights for policy 1, policy_version 1680375 (0.0005) [2023-12-27 03:36:37,521][105620] Updated weights for policy 1, policy_version 1680385 (0.0005) [2023-12-27 03:36:37,566][105620] Updated weights for policy 1, policy_version 1680395 (0.0006) [2023-12-27 03:36:37,604][105692] Updated weights for policy 0, policy_version 1676836 (0.0011) [2023-12-27 03:36:37,663][105692] Updated weights for policy 0, policy_version 1676846 (0.0011) [2023-12-27 03:36:37,716][105692] Updated weights for policy 0, policy_version 1676856 (0.0011) [2023-12-27 03:36:38,298][105620] Updated weights for policy 1, policy_version 1680405 (0.0008) [2023-12-27 03:36:38,361][105620] Updated weights for policy 1, policy_version 1680415 (0.0007) [2023-12-27 03:36:38,419][105620] Updated weights for policy 1, policy_version 1680425 (0.0006) [2023-12-27 03:36:38,462][105692] Updated weights for policy 0, policy_version 1676866 (0.0010) [2023-12-27 03:36:38,521][105692] Updated weights for policy 0, policy_version 1676876 (0.0008) [2023-12-27 03:36:38,579][105692] Updated weights for policy 0, policy_version 1676886 (0.0010) [2023-12-27 03:36:38,631][105692] Updated weights for policy 0, policy_version 1676896 (0.0011) [2023-12-27 03:36:39,044][105620] Updated weights for policy 1, policy_version 1680435 (0.0007) [2023-12-27 03:36:39,103][105620] Updated weights for policy 1, policy_version 1680445 (0.0008) [2023-12-27 03:36:39,163][105620] Updated weights for policy 1, policy_version 1680455 (0.0007) [2023-12-27 03:36:39,402][105692] Updated weights for policy 0, policy_version 1676906 (0.0010) [2023-12-27 03:36:39,469][105692] Updated weights for policy 0, policy_version 1676916 (0.0010) [2023-12-27 03:36:39,538][105692] Updated weights for policy 0, policy_version 1676926 (0.0011) [2023-12-27 03:36:39,924][105620] Updated weights for policy 1, policy_version 1680465 (0.0008) [2023-12-27 03:36:39,990][105620] Updated weights for policy 1, policy_version 1680475 (0.0008) [2023-12-27 03:36:40,048][105620] Updated weights for policy 1, policy_version 1680485 (0.0008) [2023-12-27 03:36:40,107][105620] Updated weights for policy 1, policy_version 1680495 (0.0008) [2023-12-27 03:36:40,311][105692] Updated weights for policy 0, policy_version 1676936 (0.0011) [2023-12-27 03:36:40,369][105692] Updated weights for policy 0, policy_version 1676946 (0.0010) [2023-12-27 03:36:40,418][105692] Updated weights for policy 0, policy_version 1676956 (0.0010) [2023-12-27 03:36:40,829][105620] Updated weights for policy 1, policy_version 1680505 (0.0006) [2023-12-27 03:36:40,899][105620] Updated weights for policy 1, policy_version 1680515 (0.0005) [2023-12-27 03:36:40,968][105620] Updated weights for policy 1, policy_version 1680525 (0.0006) [2023-12-27 03:36:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 859643904. Throughput: 0: 9760.4, 1: 9677.9. Samples: 859652960. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:36:41,063][104569] Avg episode reward: [(0, '8625.594'), (1, '8991.083')] [2023-12-27 03:36:41,209][105692] Updated weights for policy 0, policy_version 1676966 (0.0009) [2023-12-27 03:36:41,290][105692] Updated weights for policy 0, policy_version 1676976 (0.0009) [2023-12-27 03:36:41,349][105692] Updated weights for policy 0, policy_version 1676986 (0.0009) [2023-12-27 03:36:41,692][105620] Updated weights for policy 1, policy_version 1680535 (0.0007) [2023-12-27 03:36:41,756][105620] Updated weights for policy 1, policy_version 1680545 (0.0008) [2023-12-27 03:36:41,819][105620] Updated weights for policy 1, policy_version 1680555 (0.0006) [2023-12-27 03:36:42,138][105692] Updated weights for policy 0, policy_version 1676996 (0.0009) [2023-12-27 03:36:42,191][105692] Updated weights for policy 0, policy_version 1677006 (0.0010) [2023-12-27 03:36:42,245][105692] Updated weights for policy 0, policy_version 1677016 (0.0008) [2023-12-27 03:36:42,469][105620] Updated weights for policy 1, policy_version 1680565 (0.0008) [2023-12-27 03:36:42,528][105620] Updated weights for policy 1, policy_version 1680575 (0.0008) [2023-12-27 03:36:42,593][105620] Updated weights for policy 1, policy_version 1680585 (0.0008) [2023-12-27 03:36:42,963][105692] Updated weights for policy 0, policy_version 1677026 (0.0007) [2023-12-27 03:36:43,022][105692] Updated weights for policy 0, policy_version 1677036 (0.0008) [2023-12-27 03:36:43,089][105692] Updated weights for policy 0, policy_version 1677046 (0.0006) [2023-12-27 03:36:43,147][105692] Updated weights for policy 0, policy_version 1677056 (0.0006) [2023-12-27 03:36:43,299][105620] Updated weights for policy 1, policy_version 1680595 (0.0008) [2023-12-27 03:36:43,355][105620] Updated weights for policy 1, policy_version 1680605 (0.0009) [2023-12-27 03:36:43,408][105620] Updated weights for policy 1, policy_version 1680615 (0.0010) [2023-12-27 03:36:43,766][105692] Updated weights for policy 0, policy_version 1677066 (0.0010) [2023-12-27 03:36:43,817][105692] Updated weights for policy 0, policy_version 1677076 (0.0009) [2023-12-27 03:36:43,870][105692] Updated weights for policy 0, policy_version 1677087 (0.0009) [2023-12-27 03:36:44,018][105620] Updated weights for policy 1, policy_version 1680625 (0.0010) [2023-12-27 03:36:44,078][105620] Updated weights for policy 1, policy_version 1680635 (0.0006) [2023-12-27 03:36:44,133][105620] Updated weights for policy 1, policy_version 1680645 (0.0010) [2023-12-27 03:36:44,186][105620] Updated weights for policy 1, policy_version 1680655 (0.0009) [2023-12-27 03:36:44,691][105692] Updated weights for policy 0, policy_version 1677097 (0.0009) [2023-12-27 03:36:44,752][105692] Updated weights for policy 0, policy_version 1677107 (0.0009) [2023-12-27 03:36:44,817][105692] Updated weights for policy 0, policy_version 1677117 (0.0010) [2023-12-27 03:36:44,895][105620] Updated weights for policy 1, policy_version 1680665 (0.0009) [2023-12-27 03:36:44,959][105620] Updated weights for policy 1, policy_version 1680675 (0.0008) [2023-12-27 03:36:45,020][105620] Updated weights for policy 1, policy_version 1680685 (0.0006) [2023-12-27 03:36:45,649][105620] Updated weights for policy 1, policy_version 1680695 (0.0007) [2023-12-27 03:36:45,668][105692] Updated weights for policy 0, policy_version 1677127 (0.0008) [2023-12-27 03:36:45,703][105620] Updated weights for policy 1, policy_version 1680705 (0.0007) [2023-12-27 03:36:45,727][105692] Updated weights for policy 0, policy_version 1677137 (0.0007) [2023-12-27 03:36:45,748][105620] Updated weights for policy 1, policy_version 1680715 (0.0007) [2023-12-27 03:36:45,771][105692] Updated weights for policy 0, policy_version 1677147 (0.0007) [2023-12-27 03:36:46,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 859742208. Throughput: 0: 9719.1, 1: 9688.6. Samples: 859711352. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:36:46,063][104569] Avg episode reward: [(0, '8537.013'), (1, '9264.890')] [2023-12-27 03:36:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001677152_429416448.pth... [2023-12-27 03:36:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001680720_430325760.pth... [2023-12-27 03:36:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001676000_429121536.pth [2023-12-27 03:36:46,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001679600_430039040.pth [2023-12-27 03:36:46,361][105620] Updated weights for policy 1, policy_version 1680725 (0.0006) [2023-12-27 03:36:46,411][105620] Updated weights for policy 1, policy_version 1680735 (0.0006) [2023-12-27 03:36:46,469][105620] Updated weights for policy 1, policy_version 1680745 (0.0009) [2023-12-27 03:36:46,593][105692] Updated weights for policy 0, policy_version 1677157 (0.0009) [2023-12-27 03:36:46,651][105692] Updated weights for policy 0, policy_version 1677167 (0.0009) [2023-12-27 03:36:46,703][105692] Updated weights for policy 0, policy_version 1677177 (0.0010) [2023-12-27 03:36:47,161][105620] Updated weights for policy 1, policy_version 1680755 (0.0009) [2023-12-27 03:36:47,216][105620] Updated weights for policy 1, policy_version 1680765 (0.0009) [2023-12-27 03:36:47,266][105620] Updated weights for policy 1, policy_version 1680775 (0.0006) [2023-12-27 03:36:47,505][105692] Updated weights for policy 0, policy_version 1677188 (0.0010) [2023-12-27 03:36:47,564][105692] Updated weights for policy 0, policy_version 1677198 (0.0009) [2023-12-27 03:36:47,621][105692] Updated weights for policy 0, policy_version 1677208 (0.0010) [2023-12-27 03:36:47,953][105620] Updated weights for policy 1, policy_version 1680785 (0.0006) [2023-12-27 03:36:48,005][105620] Updated weights for policy 1, policy_version 1680795 (0.0008) [2023-12-27 03:36:48,072][105620] Updated weights for policy 1, policy_version 1680805 (0.0008) [2023-12-27 03:36:48,127][105620] Updated weights for policy 1, policy_version 1680815 (0.0010) [2023-12-27 03:36:48,423][105692] Updated weights for policy 0, policy_version 1677218 (0.0009) [2023-12-27 03:36:48,488][105692] Updated weights for policy 0, policy_version 1677228 (0.0010) [2023-12-27 03:36:48,553][105692] Updated weights for policy 0, policy_version 1677238 (0.0009) [2023-12-27 03:36:48,616][105692] Updated weights for policy 0, policy_version 1677248 (0.0008) [2023-12-27 03:36:48,774][105620] Updated weights for policy 1, policy_version 1680825 (0.0010) [2023-12-27 03:36:48,841][105620] Updated weights for policy 1, policy_version 1680835 (0.0008) [2023-12-27 03:36:48,908][105620] Updated weights for policy 1, policy_version 1680845 (0.0006) [2023-12-27 03:36:49,305][105692] Updated weights for policy 0, policy_version 1677258 (0.0009) [2023-12-27 03:36:49,370][105692] Updated weights for policy 0, policy_version 1677268 (0.0008) [2023-12-27 03:36:49,433][105692] Updated weights for policy 0, policy_version 1677278 (0.0009) [2023-12-27 03:36:49,535][105620] Updated weights for policy 1, policy_version 1680855 (0.0008) [2023-12-27 03:36:49,596][105620] Updated weights for policy 1, policy_version 1680865 (0.0008) [2023-12-27 03:36:49,664][105620] Updated weights for policy 1, policy_version 1680875 (0.0009) [2023-12-27 03:36:50,146][105692] Updated weights for policy 0, policy_version 1677288 (0.0008) [2023-12-27 03:36:50,194][105692] Updated weights for policy 0, policy_version 1677298 (0.0009) [2023-12-27 03:36:50,245][105692] Updated weights for policy 0, policy_version 1677308 (0.0009) [2023-12-27 03:36:50,415][105620] Updated weights for policy 1, policy_version 1680885 (0.0007) [2023-12-27 03:36:50,478][105620] Updated weights for policy 1, policy_version 1680895 (0.0005) [2023-12-27 03:36:50,539][105620] Updated weights for policy 1, policy_version 1680905 (0.0006) [2023-12-27 03:36:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 859832320. Throughput: 0: 9625.1, 1: 9798.8. Samples: 859826380. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:36:51,062][104569] Avg episode reward: [(0, '8625.463'), (1, '9173.539')] [2023-12-27 03:36:51,138][105620] Updated weights for policy 1, policy_version 1680915 (0.0007) [2023-12-27 03:36:51,171][105692] Updated weights for policy 0, policy_version 1677318 (0.0007) [2023-12-27 03:36:51,202][105620] Updated weights for policy 1, policy_version 1680925 (0.0008) [2023-12-27 03:36:51,228][105692] Updated weights for policy 0, policy_version 1677328 (0.0007) [2023-12-27 03:36:51,260][105620] Updated weights for policy 1, policy_version 1680935 (0.0007) [2023-12-27 03:36:51,288][105692] Updated weights for policy 0, policy_version 1677338 (0.0008) [2023-12-27 03:36:51,928][105620] Updated weights for policy 1, policy_version 1680945 (0.0006) [2023-12-27 03:36:51,983][105620] Updated weights for policy 1, policy_version 1680955 (0.0006) [2023-12-27 03:36:52,030][105692] Updated weights for policy 0, policy_version 1677348 (0.0009) [2023-12-27 03:36:52,040][105620] Updated weights for policy 1, policy_version 1680965 (0.0006) [2023-12-27 03:36:52,083][105692] Updated weights for policy 0, policy_version 1677358 (0.0008) [2023-12-27 03:36:52,098][105620] Updated weights for policy 1, policy_version 1680975 (0.0006) [2023-12-27 03:36:52,142][105692] Updated weights for policy 0, policy_version 1677368 (0.0009) [2023-12-27 03:36:52,728][105620] Updated weights for policy 1, policy_version 1680985 (0.0007) [2023-12-27 03:36:52,784][105620] Updated weights for policy 1, policy_version 1680995 (0.0011) [2023-12-27 03:36:52,846][105620] Updated weights for policy 1, policy_version 1681005 (0.0010) [2023-12-27 03:36:52,932][105692] Updated weights for policy 0, policy_version 1677378 (0.0007) [2023-12-27 03:36:52,981][105692] Updated weights for policy 0, policy_version 1677388 (0.0008) [2023-12-27 03:36:53,030][105692] Updated weights for policy 0, policy_version 1677398 (0.0008) [2023-12-27 03:36:53,084][105692] Updated weights for policy 0, policy_version 1677408 (0.0008) [2023-12-27 03:36:53,534][105620] Updated weights for policy 1, policy_version 1681015 (0.0009) [2023-12-27 03:36:53,588][105620] Updated weights for policy 1, policy_version 1681025 (0.0005) [2023-12-27 03:36:53,640][105620] Updated weights for policy 1, policy_version 1681035 (0.0008) [2023-12-27 03:36:53,887][105692] Updated weights for policy 0, policy_version 1677418 (0.0005) [2023-12-27 03:36:53,943][105692] Updated weights for policy 0, policy_version 1677428 (0.0005) [2023-12-27 03:36:54,000][105692] Updated weights for policy 0, policy_version 1677438 (0.0005) [2023-12-27 03:36:54,232][105620] Updated weights for policy 1, policy_version 1681045 (0.0006) [2023-12-27 03:36:54,279][105620] Updated weights for policy 1, policy_version 1681055 (0.0005) [2023-12-27 03:36:54,336][105620] Updated weights for policy 1, policy_version 1681066 (0.0008) [2023-12-27 03:36:54,593][105692] Updated weights for policy 0, policy_version 1677448 (0.0009) [2023-12-27 03:36:54,659][105692] Updated weights for policy 0, policy_version 1677458 (0.0009) [2023-12-27 03:36:54,724][105692] Updated weights for policy 0, policy_version 1677468 (0.0009) [2023-12-27 03:36:55,034][105620] Updated weights for policy 1, policy_version 1681076 (0.0006) [2023-12-27 03:36:55,081][105620] Updated weights for policy 1, policy_version 1681086 (0.0005) [2023-12-27 03:36:55,135][105620] Updated weights for policy 1, policy_version 1681096 (0.0006) [2023-12-27 03:36:55,430][105692] Updated weights for policy 0, policy_version 1677478 (0.0009) [2023-12-27 03:36:55,493][105692] Updated weights for policy 0, policy_version 1677488 (0.0009) [2023-12-27 03:36:55,558][105692] Updated weights for policy 0, policy_version 1677498 (0.0009) [2023-12-27 03:36:55,673][105620] Updated weights for policy 1, policy_version 1681106 (0.0007) [2023-12-27 03:36:55,724][105620] Updated weights for policy 1, policy_version 1681116 (0.0009) [2023-12-27 03:36:55,774][105620] Updated weights for policy 1, policy_version 1681126 (0.0007) [2023-12-27 03:36:55,823][105620] Updated weights for policy 1, policy_version 1681136 (0.0008) [2023-12-27 03:36:56,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 859938816. Throughput: 0: 9530.7, 1: 9909.7. Samples: 859946200. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:36:56,062][104569] Avg episode reward: [(0, '8895.871'), (1, '9080.081')] [2023-12-27 03:36:56,388][105692] Updated weights for policy 0, policy_version 1677508 (0.0009) [2023-12-27 03:36:56,440][105692] Updated weights for policy 0, policy_version 1677518 (0.0006) [2023-12-27 03:36:56,442][105620] Updated weights for policy 1, policy_version 1681146 (0.0008) [2023-12-27 03:36:56,495][105620] Updated weights for policy 1, policy_version 1681156 (0.0006) [2023-12-27 03:36:56,500][105692] Updated weights for policy 0, policy_version 1677528 (0.0008) [2023-12-27 03:36:56,551][105620] Updated weights for policy 1, policy_version 1681166 (0.0008) [2023-12-27 03:36:57,262][105692] Updated weights for policy 0, policy_version 1677538 (0.0006) [2023-12-27 03:36:57,310][105620] Updated weights for policy 1, policy_version 1681176 (0.0009) [2023-12-27 03:36:57,324][105692] Updated weights for policy 0, policy_version 1677548 (0.0009) [2023-12-27 03:36:57,367][105620] Updated weights for policy 1, policy_version 1681186 (0.0006) [2023-12-27 03:36:57,386][105692] Updated weights for policy 0, policy_version 1677558 (0.0007) [2023-12-27 03:36:57,425][105620] Updated weights for policy 1, policy_version 1681196 (0.0005) [2023-12-27 03:36:57,438][105692] Updated weights for policy 0, policy_version 1677568 (0.0008) [2023-12-27 03:36:58,133][105620] Updated weights for policy 1, policy_version 1681206 (0.0008) [2023-12-27 03:36:58,190][105692] Updated weights for policy 0, policy_version 1677578 (0.0009) [2023-12-27 03:36:58,208][105620] Updated weights for policy 1, policy_version 1681216 (0.0008) [2023-12-27 03:36:58,247][105692] Updated weights for policy 0, policy_version 1677588 (0.0007) [2023-12-27 03:36:58,271][105620] Updated weights for policy 1, policy_version 1681226 (0.0008) [2023-12-27 03:36:58,310][105692] Updated weights for policy 0, policy_version 1677598 (0.0009) [2023-12-27 03:36:59,107][105692] Updated weights for policy 0, policy_version 1677608 (0.0010) [2023-12-27 03:36:59,136][105620] Updated weights for policy 1, policy_version 1681236 (0.0010) [2023-12-27 03:36:59,167][105692] Updated weights for policy 0, policy_version 1677618 (0.0008) [2023-12-27 03:36:59,197][105620] Updated weights for policy 1, policy_version 1681246 (0.0008) [2023-12-27 03:36:59,233][105692] Updated weights for policy 0, policy_version 1677628 (0.0008) [2023-12-27 03:36:59,267][105620] Updated weights for policy 1, policy_version 1681256 (0.0008) [2023-12-27 03:36:59,914][105692] Updated weights for policy 0, policy_version 1677638 (0.0008) [2023-12-27 03:36:59,977][105692] Updated weights for policy 0, policy_version 1677648 (0.0008) [2023-12-27 03:37:00,039][105692] Updated weights for policy 0, policy_version 1677658 (0.0008) [2023-12-27 03:37:00,111][105620] Updated weights for policy 1, policy_version 1681266 (0.0010) [2023-12-27 03:37:00,176][105620] Updated weights for policy 1, policy_version 1681276 (0.0009) [2023-12-27 03:37:00,238][105620] Updated weights for policy 1, policy_version 1681286 (0.0009) [2023-12-27 03:37:00,288][105620] Updated weights for policy 1, policy_version 1681296 (0.0008) [2023-12-27 03:37:00,770][105692] Updated weights for policy 0, policy_version 1677668 (0.0008) [2023-12-27 03:37:00,824][105692] Updated weights for policy 0, policy_version 1677678 (0.0007) [2023-12-27 03:37:00,875][105692] Updated weights for policy 0, policy_version 1677688 (0.0005) [2023-12-27 03:37:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 860028928. Throughput: 0: 9530.5, 1: 9923.6. Samples: 860002396. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) [2023-12-27 03:37:01,063][104569] Avg episode reward: [(0, '8804.124'), (1, '9171.133')] [2023-12-27 03:37:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001677696_429555712.pth... [2023-12-27 03:37:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001676608_429277184.pth [2023-12-27 03:37:01,082][105620] Updated weights for policy 1, policy_version 1681306 (0.0009) [2023-12-27 03:37:01,142][105620] Updated weights for policy 1, policy_version 1681316 (0.0008) [2023-12-27 03:37:01,200][105620] Updated weights for policy 1, policy_version 1681326 (0.0008) [2023-12-27 03:37:01,211][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001681328_430481408.pth... [2023-12-27 03:37:01,215][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001680144_430178304.pth [2023-12-27 03:37:01,537][105692] Updated weights for policy 0, policy_version 1677698 (0.0005) [2023-12-27 03:37:01,589][105692] Updated weights for policy 0, policy_version 1677708 (0.0005) [2023-12-27 03:37:01,655][105692] Updated weights for policy 0, policy_version 1677718 (0.0008) [2023-12-27 03:37:01,703][105692] Updated weights for policy 0, policy_version 1677728 (0.0009) [2023-12-27 03:37:01,961][105620] Updated weights for policy 1, policy_version 1681336 (0.0008) [2023-12-27 03:37:02,018][105620] Updated weights for policy 1, policy_version 1681346 (0.0008) [2023-12-27 03:37:02,072][105620] Updated weights for policy 1, policy_version 1681356 (0.0009) [2023-12-27 03:37:02,426][105692] Updated weights for policy 0, policy_version 1677738 (0.0009) [2023-12-27 03:37:02,494][105692] Updated weights for policy 0, policy_version 1677748 (0.0010) [2023-12-27 03:37:02,558][105692] Updated weights for policy 0, policy_version 1677758 (0.0010) [2023-12-27 03:37:02,767][105620] Updated weights for policy 1, policy_version 1681366 (0.0009) [2023-12-27 03:37:02,825][105620] Updated weights for policy 1, policy_version 1681376 (0.0005) [2023-12-27 03:37:02,891][105620] Updated weights for policy 1, policy_version 1681386 (0.0005) [2023-12-27 03:37:03,279][105692] Updated weights for policy 0, policy_version 1677768 (0.0006) [2023-12-27 03:37:03,344][105692] Updated weights for policy 0, policy_version 1677778 (0.0005) [2023-12-27 03:37:03,398][105620] Updated weights for policy 1, policy_version 1681396 (0.0007) [2023-12-27 03:37:03,409][105692] Updated weights for policy 0, policy_version 1677788 (0.0005) [2023-12-27 03:37:03,446][105620] Updated weights for policy 1, policy_version 1681406 (0.0010) [2023-12-27 03:37:03,490][105620] Updated weights for policy 1, policy_version 1681416 (0.0010) [2023-12-27 03:37:03,875][105692] Updated weights for policy 0, policy_version 1677798 (0.0006) [2023-12-27 03:37:03,934][105692] Updated weights for policy 0, policy_version 1677808 (0.0008) [2023-12-27 03:37:03,998][105692] Updated weights for policy 0, policy_version 1677818 (0.0008) [2023-12-27 03:37:04,255][105620] Updated weights for policy 1, policy_version 1681426 (0.0010) [2023-12-27 03:37:04,307][105620] Updated weights for policy 1, policy_version 1681436 (0.0010) [2023-12-27 03:37:04,365][105620] Updated weights for policy 1, policy_version 1681446 (0.0011) [2023-12-27 03:37:04,424][105620] Updated weights for policy 1, policy_version 1681456 (0.0010) [2023-12-27 03:37:04,802][105692] Updated weights for policy 0, policy_version 1677828 (0.0009) [2023-12-27 03:37:04,856][105692] Updated weights for policy 0, policy_version 1677838 (0.0007) [2023-12-27 03:37:04,914][105692] Updated weights for policy 0, policy_version 1677848 (0.0007) [2023-12-27 03:37:05,059][105620] Updated weights for policy 1, policy_version 1681466 (0.0008) [2023-12-27 03:37:05,103][105620] Updated weights for policy 1, policy_version 1681476 (0.0005) [2023-12-27 03:37:05,150][105620] Updated weights for policy 1, policy_version 1681486 (0.0005) [2023-12-27 03:37:05,648][105692] Updated weights for policy 0, policy_version 1677858 (0.0006) [2023-12-27 03:37:05,697][105692] Updated weights for policy 0, policy_version 1677868 (0.0010) [2023-12-27 03:37:05,755][105692] Updated weights for policy 0, policy_version 1677878 (0.0010) [2023-12-27 03:37:05,803][105692] Updated weights for policy 0, policy_version 1677888 (0.0010) [2023-12-27 03:37:05,806][105620] Updated weights for policy 1, policy_version 1681496 (0.0006) [2023-12-27 03:37:05,867][105620] Updated weights for policy 1, policy_version 1681506 (0.0008) [2023-12-27 03:37:05,921][105620] Updated weights for policy 1, policy_version 1681516 (0.0007) [2023-12-27 03:37:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 860135424. Throughput: 0: 9583.7, 1: 9847.9. Samples: 860120356. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:37:06,062][104569] Avg episode reward: [(0, '8714.910'), (1, '9355.104')] [2023-12-27 03:37:06,591][105692] Updated weights for policy 0, policy_version 1677898 (0.0010) [2023-12-27 03:37:06,640][105692] Updated weights for policy 0, policy_version 1677908 (0.0010) [2023-12-27 03:37:06,656][105620] Updated weights for policy 1, policy_version 1681526 (0.0009) [2023-12-27 03:37:06,692][105692] Updated weights for policy 0, policy_version 1677918 (0.0010) [2023-12-27 03:37:06,716][105620] Updated weights for policy 1, policy_version 1681536 (0.0010) [2023-12-27 03:37:06,775][105620] Updated weights for policy 1, policy_version 1681546 (0.0010) [2023-12-27 03:37:07,475][105620] Updated weights for policy 1, policy_version 1681556 (0.0009) [2023-12-27 03:37:07,481][105692] Updated weights for policy 0, policy_version 1677928 (0.0011) [2023-12-27 03:37:07,536][105620] Updated weights for policy 1, policy_version 1681566 (0.0005) [2023-12-27 03:37:07,538][105692] Updated weights for policy 0, policy_version 1677938 (0.0010) [2023-12-27 03:37:07,591][105620] Updated weights for policy 1, policy_version 1681576 (0.0005) [2023-12-27 03:37:07,594][105692] Updated weights for policy 0, policy_version 1677948 (0.0011) [2023-12-27 03:37:08,302][105620] Updated weights for policy 1, policy_version 1681586 (0.0010) [2023-12-27 03:37:08,341][105692] Updated weights for policy 0, policy_version 1677958 (0.0009) [2023-12-27 03:37:08,371][105620] Updated weights for policy 1, policy_version 1681596 (0.0011) [2023-12-27 03:37:08,409][105692] Updated weights for policy 0, policy_version 1677968 (0.0008) [2023-12-27 03:37:08,432][105620] Updated weights for policy 1, policy_version 1681606 (0.0006) [2023-12-27 03:37:08,474][105692] Updated weights for policy 0, policy_version 1677978 (0.0008) [2023-12-27 03:37:08,489][105620] Updated weights for policy 1, policy_version 1681616 (0.0006) [2023-12-27 03:37:09,183][105620] Updated weights for policy 1, policy_version 1681626 (0.0010) [2023-12-27 03:37:09,249][105620] Updated weights for policy 1, policy_version 1681636 (0.0009) [2023-12-27 03:37:09,254][105692] Updated weights for policy 0, policy_version 1677988 (0.0009) [2023-12-27 03:37:09,309][105620] Updated weights for policy 1, policy_version 1681646 (0.0009) [2023-12-27 03:37:09,316][105692] Updated weights for policy 0, policy_version 1677998 (0.0006) [2023-12-27 03:37:09,382][105692] Updated weights for policy 0, policy_version 1678008 (0.0007) [2023-12-27 03:37:10,050][105620] Updated weights for policy 1, policy_version 1681656 (0.0010) [2023-12-27 03:37:10,102][105620] Updated weights for policy 1, policy_version 1681666 (0.0010) [2023-12-27 03:37:10,147][105620] Updated weights for policy 1, policy_version 1681676 (0.0010) [2023-12-27 03:37:10,170][105692] Updated weights for policy 0, policy_version 1678018 (0.0009) [2023-12-27 03:37:10,230][105692] Updated weights for policy 0, policy_version 1678028 (0.0009) [2023-12-27 03:37:10,292][105692] Updated weights for policy 0, policy_version 1678038 (0.0008) [2023-12-27 03:37:10,355][105692] Updated weights for policy 0, policy_version 1678048 (0.0008) [2023-12-27 03:37:10,920][105620] Updated weights for policy 1, policy_version 1681686 (0.0009) [2023-12-27 03:37:10,990][105620] Updated weights for policy 1, policy_version 1681696 (0.0011) [2023-12-27 03:37:11,060][105620] Updated weights for policy 1, policy_version 1681706 (0.0011) [2023-12-27 03:37:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 860217344. Throughput: 0: 9432.1, 1: 9938.5. Samples: 860233416. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:37:11,063][104569] Avg episode reward: [(0, '8714.000'), (1, '9194.778')] [2023-12-27 03:37:11,091][105692] Updated weights for policy 0, policy_version 1678058 (0.0009) [2023-12-27 03:37:11,159][105692] Updated weights for policy 0, policy_version 1678068 (0.0008) [2023-12-27 03:37:11,229][105692] Updated weights for policy 0, policy_version 1678078 (0.0010) [2023-12-27 03:37:11,693][105620] Updated weights for policy 1, policy_version 1681716 (0.0009) [2023-12-27 03:37:11,759][105620] Updated weights for policy 1, policy_version 1681726 (0.0008) [2023-12-27 03:37:11,813][105620] Updated weights for policy 1, policy_version 1681736 (0.0010) [2023-12-27 03:37:12,059][105692] Updated weights for policy 0, policy_version 1678088 (0.0009) [2023-12-27 03:37:12,123][105692] Updated weights for policy 0, policy_version 1678098 (0.0008) [2023-12-27 03:37:12,186][105692] Updated weights for policy 0, policy_version 1678108 (0.0008) [2023-12-27 03:37:12,549][105620] Updated weights for policy 1, policy_version 1681746 (0.0009) [2023-12-27 03:37:12,610][105620] Updated weights for policy 1, policy_version 1681756 (0.0007) [2023-12-27 03:37:12,666][105620] Updated weights for policy 1, policy_version 1681766 (0.0009) [2023-12-27 03:37:12,715][105620] Updated weights for policy 1, policy_version 1681776 (0.0009) [2023-12-27 03:37:12,957][105692] Updated weights for policy 0, policy_version 1678118 (0.0009) [2023-12-27 03:37:13,017][105692] Updated weights for policy 0, policy_version 1678128 (0.0009) [2023-12-27 03:37:13,077][105692] Updated weights for policy 0, policy_version 1678138 (0.0006) [2023-12-27 03:37:13,476][105620] Updated weights for policy 1, policy_version 1681786 (0.0009) [2023-12-27 03:37:13,534][105620] Updated weights for policy 1, policy_version 1681796 (0.0009) [2023-12-27 03:37:13,581][105620] Updated weights for policy 1, policy_version 1681806 (0.0009) [2023-12-27 03:37:13,818][105692] Updated weights for policy 0, policy_version 1678148 (0.0007) [2023-12-27 03:37:13,879][105692] Updated weights for policy 0, policy_version 1678158 (0.0008) [2023-12-27 03:37:13,938][105692] Updated weights for policy 0, policy_version 1678168 (0.0009) [2023-12-27 03:37:14,304][105620] Updated weights for policy 1, policy_version 1681816 (0.0008) [2023-12-27 03:37:14,365][105620] Updated weights for policy 1, policy_version 1681826 (0.0009) [2023-12-27 03:37:14,414][105620] Updated weights for policy 1, policy_version 1681836 (0.0008) [2023-12-27 03:37:14,608][105692] Updated weights for policy 0, policy_version 1678178 (0.0009) [2023-12-27 03:37:14,660][105692] Updated weights for policy 0, policy_version 1678188 (0.0008) [2023-12-27 03:37:14,717][105692] Updated weights for policy 0, policy_version 1678198 (0.0009) [2023-12-27 03:37:14,769][105692] Updated weights for policy 0, policy_version 1678208 (0.0010) [2023-12-27 03:37:15,103][105620] Updated weights for policy 1, policy_version 1681846 (0.0010) [2023-12-27 03:37:15,168][105620] Updated weights for policy 1, policy_version 1681856 (0.0010) [2023-12-27 03:37:15,232][105620] Updated weights for policy 1, policy_version 1681866 (0.0011) [2023-12-27 03:37:15,526][105692] Updated weights for policy 0, policy_version 1678218 (0.0010) [2023-12-27 03:37:15,586][105692] Updated weights for policy 0, policy_version 1678228 (0.0008) [2023-12-27 03:37:15,655][105692] Updated weights for policy 0, policy_version 1678238 (0.0009) [2023-12-27 03:37:15,887][105620] Updated weights for policy 1, policy_version 1681876 (0.0011) [2023-12-27 03:37:15,956][105620] Updated weights for policy 1, policy_version 1681886 (0.0008) [2023-12-27 03:37:16,022][105620] Updated weights for policy 1, policy_version 1681896 (0.0005) [2023-12-27 03:37:16,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 860315648. Throughput: 0: 9379.5, 1: 9971.3. Samples: 860289508. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:37:16,062][104569] Avg episode reward: [(0, '8621.271'), (1, '8548.060')] [2023-12-27 03:37:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001678240_429694976.pth... [2023-12-27 03:37:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001677152_429416448.pth [2023-12-27 03:37:16,075][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001681904_430628864.pth... [2023-12-27 03:37:16,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001680720_430325760.pth [2023-12-27 03:37:16,251][105692] Updated weights for policy 0, policy_version 1678248 (0.0006) [2023-12-27 03:37:16,316][105692] Updated weights for policy 0, policy_version 1678258 (0.0005) [2023-12-27 03:37:16,377][105692] Updated weights for policy 0, policy_version 1678268 (0.0006) [2023-12-27 03:37:16,530][105620] Updated weights for policy 1, policy_version 1681906 (0.0006) [2023-12-27 03:37:16,593][105620] Updated weights for policy 1, policy_version 1681916 (0.0006) [2023-12-27 03:37:16,656][105620] Updated weights for policy 1, policy_version 1681926 (0.0008) [2023-12-27 03:37:16,706][105586] KL-divergence is very high: 156.9676 [2023-12-27 03:37:16,716][105620] Updated weights for policy 1, policy_version 1681936 (0.0008) [2023-12-27 03:37:16,975][105692] Updated weights for policy 0, policy_version 1678278 (0.0006) [2023-12-27 03:37:17,036][105692] Updated weights for policy 0, policy_version 1678288 (0.0005) [2023-12-27 03:37:17,095][105692] Updated weights for policy 0, policy_version 1678298 (0.0005) [2023-12-27 03:37:17,269][105620] Updated weights for policy 1, policy_version 1681947 (0.0009) [2023-12-27 03:37:17,322][105620] Updated weights for policy 1, policy_version 1681957 (0.0009) [2023-12-27 03:37:17,384][105620] Updated weights for policy 1, policy_version 1681967 (0.0010) [2023-12-27 03:37:17,607][105692] Updated weights for policy 0, policy_version 1678308 (0.0006) [2023-12-27 03:37:17,663][105692] Updated weights for policy 0, policy_version 1678318 (0.0006) [2023-12-27 03:37:17,718][105692] Updated weights for policy 0, policy_version 1678328 (0.0006) [2023-12-27 03:37:18,082][105620] Updated weights for policy 1, policy_version 1681977 (0.0010) [2023-12-27 03:37:18,130][105620] Updated weights for policy 1, policy_version 1681987 (0.0010) [2023-12-27 03:37:18,177][105620] Updated weights for policy 1, policy_version 1681997 (0.0010) [2023-12-27 03:37:18,403][105692] Updated weights for policy 0, policy_version 1678338 (0.0007) [2023-12-27 03:37:18,462][105692] Updated weights for policy 0, policy_version 1678348 (0.0010) [2023-12-27 03:37:18,511][105692] Updated weights for policy 0, policy_version 1678358 (0.0010) [2023-12-27 03:37:18,570][105692] Updated weights for policy 0, policy_version 1678368 (0.0010) [2023-12-27 03:37:18,941][105620] Updated weights for policy 1, policy_version 1682007 (0.0011) [2023-12-27 03:37:18,993][105620] Updated weights for policy 1, policy_version 1682017 (0.0010) [2023-12-27 03:37:19,045][105620] Updated weights for policy 1, policy_version 1682027 (0.0010) [2023-12-27 03:37:19,267][105692] Updated weights for policy 0, policy_version 1678378 (0.0009) [2023-12-27 03:37:19,330][105692] Updated weights for policy 0, policy_version 1678388 (0.0010) [2023-12-27 03:37:19,401][105692] Updated weights for policy 0, policy_version 1678398 (0.0009) [2023-12-27 03:37:19,747][105620] Updated weights for policy 1, policy_version 1682037 (0.0011) [2023-12-27 03:37:19,812][105620] Updated weights for policy 1, policy_version 1682047 (0.0011) [2023-12-27 03:37:19,885][105620] Updated weights for policy 1, policy_version 1682057 (0.0011) [2023-12-27 03:37:20,071][105692] Updated weights for policy 0, policy_version 1678408 (0.0011) [2023-12-27 03:37:20,135][105692] Updated weights for policy 0, policy_version 1678418 (0.0008) [2023-12-27 03:37:20,198][105692] Updated weights for policy 0, policy_version 1678428 (0.0006) [2023-12-27 03:37:20,616][105620] Updated weights for policy 1, policy_version 1682067 (0.0010) [2023-12-27 03:37:20,672][105620] Updated weights for policy 1, policy_version 1682077 (0.0009) [2023-12-27 03:37:20,729][105620] Updated weights for policy 1, policy_version 1682087 (0.0009) [2023-12-27 03:37:20,781][105692] Updated weights for policy 0, policy_version 1678438 (0.0007) [2023-12-27 03:37:20,830][105692] Updated weights for policy 0, policy_version 1678448 (0.0009) [2023-12-27 03:37:20,883][105692] Updated weights for policy 0, policy_version 1678458 (0.0009) [2023-12-27 03:37:21,062][104569] Fps is (10 sec: 21299.3, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 860430336. Throughput: 0: 9440.6, 1: 10063.6. Samples: 860414452. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:37:21,062][104569] Avg episode reward: [(0, '8445.832'), (1, '8547.000')] [2023-12-27 03:37:21,578][105620] Updated weights for policy 1, policy_version 1682097 (0.0009) [2023-12-27 03:37:21,641][105620] Updated weights for policy 1, policy_version 1682107 (0.0009) [2023-12-27 03:37:21,699][105620] Updated weights for policy 1, policy_version 1682117 (0.0008) [2023-12-27 03:37:21,744][105692] Updated weights for policy 0, policy_version 1678468 (0.0009) [2023-12-27 03:37:21,772][105620] Updated weights for policy 1, policy_version 1682127 (0.0009) [2023-12-27 03:37:21,799][105692] Updated weights for policy 0, policy_version 1678478 (0.0008) [2023-12-27 03:37:21,847][105692] Updated weights for policy 0, policy_version 1678488 (0.0008) [2023-12-27 03:37:22,455][105620] Updated weights for policy 1, policy_version 1682137 (0.0008) [2023-12-27 03:37:22,513][105620] Updated weights for policy 1, policy_version 1682147 (0.0008) [2023-12-27 03:37:22,576][105620] Updated weights for policy 1, policy_version 1682157 (0.0008) [2023-12-27 03:37:22,585][105692] Updated weights for policy 0, policy_version 1678498 (0.0008) [2023-12-27 03:37:22,643][105692] Updated weights for policy 0, policy_version 1678508 (0.0008) [2023-12-27 03:37:22,694][105692] Updated weights for policy 0, policy_version 1678518 (0.0008) [2023-12-27 03:37:22,743][105692] Updated weights for policy 0, policy_version 1678528 (0.0008) [2023-12-27 03:37:23,223][105620] Updated weights for policy 1, policy_version 1682167 (0.0007) [2023-12-27 03:37:23,286][105620] Updated weights for policy 1, policy_version 1682177 (0.0009) [2023-12-27 03:37:23,340][105620] Updated weights for policy 1, policy_version 1682187 (0.0008) [2023-12-27 03:37:23,445][105692] Updated weights for policy 0, policy_version 1678538 (0.0010) [2023-12-27 03:37:23,500][105692] Updated weights for policy 0, policy_version 1678548 (0.0010) [2023-12-27 03:37:23,555][105692] Updated weights for policy 0, policy_version 1678558 (0.0010) [2023-12-27 03:37:24,041][105620] Updated weights for policy 1, policy_version 1682197 (0.0009) [2023-12-27 03:37:24,093][105620] Updated weights for policy 1, policy_version 1682207 (0.0009) [2023-12-27 03:37:24,152][105620] Updated weights for policy 1, policy_version 1682217 (0.0009) [2023-12-27 03:37:24,319][105692] Updated weights for policy 0, policy_version 1678568 (0.0009) [2023-12-27 03:37:24,375][105692] Updated weights for policy 0, policy_version 1678578 (0.0010) [2023-12-27 03:37:24,430][105692] Updated weights for policy 0, policy_version 1678588 (0.0010) [2023-12-27 03:37:24,975][105620] Updated weights for policy 1, policy_version 1682227 (0.0009) [2023-12-27 03:37:25,025][105620] Updated weights for policy 1, policy_version 1682237 (0.0008) [2023-12-27 03:37:25,074][105692] Updated weights for policy 0, policy_version 1678598 (0.0007) [2023-12-27 03:37:25,086][105620] Updated weights for policy 1, policy_version 1682247 (0.0008) [2023-12-27 03:37:25,129][105692] Updated weights for policy 0, policy_version 1678608 (0.0005) [2023-12-27 03:37:25,181][105692] Updated weights for policy 0, policy_version 1678618 (0.0006) [2023-12-27 03:37:25,816][105620] Updated weights for policy 1, policy_version 1682257 (0.0009) [2023-12-27 03:37:25,844][105692] Updated weights for policy 0, policy_version 1678628 (0.0010) [2023-12-27 03:37:25,870][105620] Updated weights for policy 1, policy_version 1682267 (0.0005) [2023-12-27 03:37:25,903][105692] Updated weights for policy 0, policy_version 1678638 (0.0011) [2023-12-27 03:37:25,926][105620] Updated weights for policy 1, policy_version 1682277 (0.0005) [2023-12-27 03:37:25,958][105692] Updated weights for policy 0, policy_version 1678648 (0.0010) [2023-12-27 03:37:25,981][105620] Updated weights for policy 1, policy_version 1682287 (0.0005) [2023-12-27 03:37:26,062][104569] Fps is (10 sec: 21299.5, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 860528640. Throughput: 0: 9515.9, 1: 10003.1. Samples: 860531316. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:37:26,062][104569] Avg episode reward: [(0, '8806.868'), (1, '8803.973')] [2023-12-27 03:37:26,586][105620] Updated weights for policy 1, policy_version 1682297 (0.0005) [2023-12-27 03:37:26,643][105620] Updated weights for policy 1, policy_version 1682307 (0.0005) [2023-12-27 03:37:26,650][105692] Updated weights for policy 0, policy_version 1678658 (0.0010) [2023-12-27 03:37:26,696][105620] Updated weights for policy 1, policy_version 1682317 (0.0005) [2023-12-27 03:37:26,706][105692] Updated weights for policy 0, policy_version 1678668 (0.0005) [2023-12-27 03:37:26,760][105692] Updated weights for policy 0, policy_version 1678678 (0.0009) [2023-12-27 03:37:26,815][105692] Updated weights for policy 0, policy_version 1678688 (0.0010) [2023-12-27 03:37:27,328][105620] Updated weights for policy 1, policy_version 1682327 (0.0009) [2023-12-27 03:37:27,386][105620] Updated weights for policy 1, policy_version 1682337 (0.0010) [2023-12-27 03:37:27,444][105620] Updated weights for policy 1, policy_version 1682347 (0.0010) [2023-12-27 03:37:27,525][105692] Updated weights for policy 0, policy_version 1678698 (0.0005) [2023-12-27 03:37:27,574][105692] Updated weights for policy 0, policy_version 1678708 (0.0005) [2023-12-27 03:37:27,641][105692] Updated weights for policy 0, policy_version 1678718 (0.0008) [2023-12-27 03:37:28,133][105620] Updated weights for policy 1, policy_version 1682357 (0.0010) [2023-12-27 03:37:28,190][105620] Updated weights for policy 1, policy_version 1682367 (0.0009) [2023-12-27 03:37:28,242][105620] Updated weights for policy 1, policy_version 1682377 (0.0010) [2023-12-27 03:37:28,309][105692] Updated weights for policy 0, policy_version 1678728 (0.0006) [2023-12-27 03:37:28,381][105692] Updated weights for policy 0, policy_version 1678738 (0.0006) [2023-12-27 03:37:28,452][105692] Updated weights for policy 0, policy_version 1678748 (0.0007) [2023-12-27 03:37:29,014][105692] Updated weights for policy 0, policy_version 1678758 (0.0005) [2023-12-27 03:37:29,027][105620] Updated weights for policy 1, policy_version 1682387 (0.0009) [2023-12-27 03:37:29,063][105692] Updated weights for policy 0, policy_version 1678768 (0.0005) [2023-12-27 03:37:29,090][105620] Updated weights for policy 1, policy_version 1682397 (0.0008) [2023-12-27 03:37:29,115][105692] Updated weights for policy 0, policy_version 1678778 (0.0006) [2023-12-27 03:37:29,144][105620] Updated weights for policy 1, policy_version 1682407 (0.0008) [2023-12-27 03:37:29,744][105692] Updated weights for policy 0, policy_version 1678788 (0.0006) [2023-12-27 03:37:29,792][105692] Updated weights for policy 0, policy_version 1678798 (0.0006) [2023-12-27 03:37:29,846][105692] Updated weights for policy 0, policy_version 1678808 (0.0009) [2023-12-27 03:37:29,904][105620] Updated weights for policy 1, policy_version 1682417 (0.0007) [2023-12-27 03:37:29,968][105620] Updated weights for policy 1, policy_version 1682427 (0.0007) [2023-12-27 03:37:30,038][105620] Updated weights for policy 1, policy_version 1682437 (0.0005) [2023-12-27 03:37:30,105][105620] Updated weights for policy 1, policy_version 1682447 (0.0005) [2023-12-27 03:37:30,627][105620] Updated weights for policy 1, policy_version 1682457 (0.0005) [2023-12-27 03:37:30,675][105620] Updated weights for policy 1, policy_version 1682467 (0.0005) [2023-12-27 03:37:30,699][105692] Updated weights for policy 0, policy_version 1678818 (0.0008) [2023-12-27 03:37:30,728][105620] Updated weights for policy 1, policy_version 1682477 (0.0009) [2023-12-27 03:37:30,749][105692] Updated weights for policy 0, policy_version 1678828 (0.0005) [2023-12-27 03:37:30,800][105692] Updated weights for policy 0, policy_version 1678839 (0.0008) [2023-12-27 03:37:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 860626944. Throughput: 0: 9555.9, 1: 10015.1. Samples: 860592040. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:37:31,062][104569] Avg episode reward: [(0, '9078.794'), (1, '8803.082')] [2023-12-27 03:37:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001682480_430776320.pth... [2023-12-27 03:37:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001678848_429850624.pth... [2023-12-27 03:37:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001681328_430481408.pth [2023-12-27 03:37:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001677696_429555712.pth [2023-12-27 03:37:31,300][105620] Updated weights for policy 1, policy_version 1682487 (0.0009) [2023-12-27 03:37:31,353][105620] Updated weights for policy 1, policy_version 1682497 (0.0010) [2023-12-27 03:37:31,416][105620] Updated weights for policy 1, policy_version 1682507 (0.0008) [2023-12-27 03:37:31,587][105692] Updated weights for policy 0, policy_version 1678850 (0.0009) [2023-12-27 03:37:31,651][105692] Updated weights for policy 0, policy_version 1678860 (0.0009) [2023-12-27 03:37:31,705][105692] Updated weights for policy 0, policy_version 1678870 (0.0007) [2023-12-27 03:37:31,763][105692] Updated weights for policy 0, policy_version 1678880 (0.0009) [2023-12-27 03:37:32,087][105620] Updated weights for policy 1, policy_version 1682517 (0.0007) [2023-12-27 03:37:32,135][105620] Updated weights for policy 1, policy_version 1682527 (0.0005) [2023-12-27 03:37:32,180][105620] Updated weights for policy 1, policy_version 1682537 (0.0005) [2023-12-27 03:37:32,596][105692] Updated weights for policy 0, policy_version 1678890 (0.0007) [2023-12-27 03:37:32,656][105692] Updated weights for policy 0, policy_version 1678900 (0.0010) [2023-12-27 03:37:32,722][105692] Updated weights for policy 0, policy_version 1678910 (0.0009) [2023-12-27 03:37:32,841][105620] Updated weights for policy 1, policy_version 1682547 (0.0008) [2023-12-27 03:37:32,909][105620] Updated weights for policy 1, policy_version 1682557 (0.0009) [2023-12-27 03:37:32,969][105620] Updated weights for policy 1, policy_version 1682567 (0.0008) [2023-12-27 03:37:33,406][105692] Updated weights for policy 0, policy_version 1678920 (0.0008) [2023-12-27 03:37:33,456][105692] Updated weights for policy 0, policy_version 1678930 (0.0009) [2023-12-27 03:37:33,510][105692] Updated weights for policy 0, policy_version 1678940 (0.0009) [2023-12-27 03:37:33,697][105620] Updated weights for policy 1, policy_version 1682577 (0.0009) [2023-12-27 03:37:33,751][105620] Updated weights for policy 1, policy_version 1682587 (0.0010) [2023-12-27 03:37:33,805][105620] Updated weights for policy 1, policy_version 1682597 (0.0009) [2023-12-27 03:37:33,863][105620] Updated weights for policy 1, policy_version 1682607 (0.0010) [2023-12-27 03:37:34,230][105692] Updated weights for policy 0, policy_version 1678950 (0.0008) [2023-12-27 03:37:34,290][105692] Updated weights for policy 0, policy_version 1678960 (0.0008) [2023-12-27 03:37:34,344][105692] Updated weights for policy 0, policy_version 1678970 (0.0008) [2023-12-27 03:37:34,658][105620] Updated weights for policy 1, policy_version 1682617 (0.0010) [2023-12-27 03:37:34,720][105620] Updated weights for policy 1, policy_version 1682627 (0.0010) [2023-12-27 03:37:34,783][105620] Updated weights for policy 1, policy_version 1682637 (0.0010) [2023-12-27 03:37:35,044][105692] Updated weights for policy 0, policy_version 1678980 (0.0008) [2023-12-27 03:37:35,096][105692] Updated weights for policy 0, policy_version 1678990 (0.0008) [2023-12-27 03:37:35,147][105692] Updated weights for policy 0, policy_version 1679000 (0.0009) [2023-12-27 03:37:35,445][105620] Updated weights for policy 1, policy_version 1682647 (0.0007) [2023-12-27 03:37:35,495][105620] Updated weights for policy 1, policy_version 1682657 (0.0005) [2023-12-27 03:37:35,546][105620] Updated weights for policy 1, policy_version 1682667 (0.0005) [2023-12-27 03:37:36,039][105692] Updated weights for policy 0, policy_version 1679010 (0.0010) [2023-12-27 03:37:36,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 860717056. Throughput: 0: 9653.4, 1: 9987.5. Samples: 860710220. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:37:36,062][104569] Avg episode reward: [(0, '8804.563'), (1, '8489.137')] [2023-12-27 03:37:36,079][105620] Updated weights for policy 1, policy_version 1682677 (0.0008) [2023-12-27 03:37:36,096][105692] Updated weights for policy 0, policy_version 1679020 (0.0009) [2023-12-27 03:37:36,141][105620] Updated weights for policy 1, policy_version 1682687 (0.0008) [2023-12-27 03:37:36,161][105692] Updated weights for policy 0, policy_version 1679030 (0.0008) [2023-12-27 03:37:36,204][105620] Updated weights for policy 1, policy_version 1682697 (0.0009) [2023-12-27 03:37:36,214][105692] Updated weights for policy 0, policy_version 1679040 (0.0006) [2023-12-27 03:37:36,952][105620] Updated weights for policy 1, policy_version 1682707 (0.0009) [2023-12-27 03:37:36,962][105692] Updated weights for policy 0, policy_version 1679050 (0.0009) [2023-12-27 03:37:36,999][105620] Updated weights for policy 1, policy_version 1682717 (0.0005) [2023-12-27 03:37:37,027][105692] Updated weights for policy 0, policy_version 1679060 (0.0008) [2023-12-27 03:37:37,048][105620] Updated weights for policy 1, policy_version 1682727 (0.0005) [2023-12-27 03:37:37,095][105692] Updated weights for policy 0, policy_version 1679070 (0.0010) [2023-12-27 03:37:37,610][105620] Updated weights for policy 1, policy_version 1682737 (0.0005) [2023-12-27 03:37:37,676][105620] Updated weights for policy 1, policy_version 1682747 (0.0006) [2023-12-27 03:37:37,748][105620] Updated weights for policy 1, policy_version 1682757 (0.0006) [2023-12-27 03:37:37,812][105620] Updated weights for policy 1, policy_version 1682767 (0.0005) [2023-12-27 03:37:37,866][105692] Updated weights for policy 0, policy_version 1679080 (0.0007) [2023-12-27 03:37:37,936][105692] Updated weights for policy 0, policy_version 1679090 (0.0006) [2023-12-27 03:37:37,992][105692] Updated weights for policy 0, policy_version 1679100 (0.0006) [2023-12-27 03:37:38,444][105620] Updated weights for policy 1, policy_version 1682777 (0.0009) [2023-12-27 03:37:38,509][105620] Updated weights for policy 1, policy_version 1682787 (0.0009) [2023-12-27 03:37:38,572][105620] Updated weights for policy 1, policy_version 1682797 (0.0011) [2023-12-27 03:37:38,654][105692] Updated weights for policy 0, policy_version 1679110 (0.0008) [2023-12-27 03:37:38,721][105692] Updated weights for policy 0, policy_version 1679120 (0.0007) [2023-12-27 03:37:38,790][105692] Updated weights for policy 0, policy_version 1679130 (0.0008) [2023-12-27 03:37:39,318][105620] Updated weights for policy 1, policy_version 1682807 (0.0009) [2023-12-27 03:37:39,390][105620] Updated weights for policy 1, policy_version 1682817 (0.0008) [2023-12-27 03:37:39,460][105620] Updated weights for policy 1, policy_version 1682827 (0.0011) [2023-12-27 03:37:39,528][105692] Updated weights for policy 0, policy_version 1679140 (0.0008) [2023-12-27 03:37:39,595][105692] Updated weights for policy 0, policy_version 1679150 (0.0007) [2023-12-27 03:37:39,660][105692] Updated weights for policy 0, policy_version 1679160 (0.0008) [2023-12-27 03:37:40,147][105620] Updated weights for policy 1, policy_version 1682837 (0.0010) [2023-12-27 03:37:40,211][105620] Updated weights for policy 1, policy_version 1682847 (0.0011) [2023-12-27 03:37:40,277][105620] Updated weights for policy 1, policy_version 1682857 (0.0010) [2023-12-27 03:37:40,375][105692] Updated weights for policy 0, policy_version 1679170 (0.0008) [2023-12-27 03:37:40,426][105692] Updated weights for policy 0, policy_version 1679180 (0.0009) [2023-12-27 03:37:40,482][105692] Updated weights for policy 0, policy_version 1679190 (0.0008) [2023-12-27 03:37:40,541][105692] Updated weights for policy 0, policy_version 1679200 (0.0008) [2023-12-27 03:37:40,944][105620] Updated weights for policy 1, policy_version 1682867 (0.0007) [2023-12-27 03:37:41,006][105620] Updated weights for policy 1, policy_version 1682877 (0.0005) [2023-12-27 03:37:41,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 860815360. Throughput: 0: 9661.9, 1: 9961.5. Samples: 860829252. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:37:41,063][104569] Avg episode reward: [(0, '8534.202'), (1, '8397.123')] [2023-12-27 03:37:41,074][105620] Updated weights for policy 1, policy_version 1682887 (0.0008) [2023-12-27 03:37:41,218][105692] Updated weights for policy 0, policy_version 1679210 (0.0010) [2023-12-27 03:37:41,276][105692] Updated weights for policy 0, policy_version 1679220 (0.0008) [2023-12-27 03:37:41,338][105692] Updated weights for policy 0, policy_version 1679230 (0.0008) [2023-12-27 03:37:41,801][105620] Updated weights for policy 1, policy_version 1682897 (0.0006) [2023-12-27 03:37:41,865][105620] Updated weights for policy 1, policy_version 1682907 (0.0011) [2023-12-27 03:37:41,927][105620] Updated weights for policy 1, policy_version 1682917 (0.0010) [2023-12-27 03:37:41,990][105620] Updated weights for policy 1, policy_version 1682927 (0.0010) [2023-12-27 03:37:42,097][105692] Updated weights for policy 0, policy_version 1679240 (0.0006) [2023-12-27 03:37:42,156][105692] Updated weights for policy 0, policy_version 1679250 (0.0010) [2023-12-27 03:37:42,208][105692] Updated weights for policy 0, policy_version 1679260 (0.0010) [2023-12-27 03:37:42,670][105620] Updated weights for policy 1, policy_version 1682937 (0.0010) [2023-12-27 03:37:42,736][105620] Updated weights for policy 1, policy_version 1682947 (0.0010) [2023-12-27 03:37:42,797][105620] Updated weights for policy 1, policy_version 1682957 (0.0010) [2023-12-27 03:37:42,952][105692] Updated weights for policy 0, policy_version 1679270 (0.0007) [2023-12-27 03:37:43,020][105692] Updated weights for policy 0, policy_version 1679280 (0.0008) [2023-12-27 03:37:43,087][105692] Updated weights for policy 0, policy_version 1679290 (0.0005) [2023-12-27 03:37:43,435][105620] Updated weights for policy 1, policy_version 1682967 (0.0010) [2023-12-27 03:37:43,498][105620] Updated weights for policy 1, policy_version 1682977 (0.0010) [2023-12-27 03:37:43,559][105620] Updated weights for policy 1, policy_version 1682987 (0.0010) [2023-12-27 03:37:43,712][105692] Updated weights for policy 0, policy_version 1679300 (0.0007) [2023-12-27 03:37:43,769][105692] Updated weights for policy 0, policy_version 1679310 (0.0008) [2023-12-27 03:37:43,829][105692] Updated weights for policy 0, policy_version 1679320 (0.0008) [2023-12-27 03:37:44,209][105620] Updated weights for policy 1, policy_version 1682997 (0.0010) [2023-12-27 03:37:44,268][105620] Updated weights for policy 1, policy_version 1683007 (0.0009) [2023-12-27 03:37:44,335][105620] Updated weights for policy 1, policy_version 1683017 (0.0009) [2023-12-27 03:37:44,632][105692] Updated weights for policy 0, policy_version 1679330 (0.0009) [2023-12-27 03:37:44,691][105692] Updated weights for policy 0, policy_version 1679340 (0.0010) [2023-12-27 03:37:44,750][105692] Updated weights for policy 0, policy_version 1679350 (0.0009) [2023-12-27 03:37:44,817][105692] Updated weights for policy 0, policy_version 1679360 (0.0009) [2023-12-27 03:37:45,021][105620] Updated weights for policy 1, policy_version 1683027 (0.0009) [2023-12-27 03:37:45,083][105620] Updated weights for policy 1, policy_version 1683037 (0.0009) [2023-12-27 03:37:45,138][105620] Updated weights for policy 1, policy_version 1683047 (0.0009) [2023-12-27 03:37:45,665][105692] Updated weights for policy 0, policy_version 1679370 (0.0005) [2023-12-27 03:37:45,713][105692] Updated weights for policy 0, policy_version 1679380 (0.0006) [2023-12-27 03:37:45,764][105692] Updated weights for policy 0, policy_version 1679390 (0.0006) [2023-12-27 03:37:45,825][105620] Updated weights for policy 1, policy_version 1683057 (0.0009) [2023-12-27 03:37:45,878][105620] Updated weights for policy 1, policy_version 1683067 (0.0010) [2023-12-27 03:37:45,931][105620] Updated weights for policy 1, policy_version 1683077 (0.0009) [2023-12-27 03:37:45,985][105620] Updated weights for policy 1, policy_version 1683087 (0.0009) [2023-12-27 03:37:46,062][104569] Fps is (10 sec: 20479.3, 60 sec: 19660.8, 300 sec: 19605.2). Total num frames: 860921856. Throughput: 0: 9715.0, 1: 9970.2. Samples: 860888232. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:37:46,063][104569] Avg episode reward: [(0, '8536.973'), (1, '8986.516')] [2023-12-27 03:37:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001679392_429989888.pth... [2023-12-27 03:37:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001683088_430931968.pth... [2023-12-27 03:37:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001681904_430628864.pth [2023-12-27 03:37:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001678240_429694976.pth [2023-12-27 03:37:46,380][105692] Updated weights for policy 0, policy_version 1679400 (0.0008) [2023-12-27 03:37:46,431][105692] Updated weights for policy 0, policy_version 1679410 (0.0009) [2023-12-27 03:37:46,484][105692] Updated weights for policy 0, policy_version 1679420 (0.0010) [2023-12-27 03:37:46,694][105620] Updated weights for policy 1, policy_version 1683097 (0.0008) [2023-12-27 03:37:46,744][105620] Updated weights for policy 1, policy_version 1683107 (0.0008) [2023-12-27 03:37:46,808][105620] Updated weights for policy 1, policy_version 1683117 (0.0009) [2023-12-27 03:37:47,220][105692] Updated weights for policy 0, policy_version 1679430 (0.0007) [2023-12-27 03:37:47,272][105692] Updated weights for policy 0, policy_version 1679440 (0.0006) [2023-12-27 03:37:47,337][105692] Updated weights for policy 0, policy_version 1679450 (0.0005) [2023-12-27 03:37:47,646][105620] Updated weights for policy 1, policy_version 1683127 (0.0009) [2023-12-27 03:37:47,697][105620] Updated weights for policy 1, policy_version 1683138 (0.0009) [2023-12-27 03:37:47,744][105620] Updated weights for policy 1, policy_version 1683148 (0.0009) [2023-12-27 03:37:47,948][105692] Updated weights for policy 0, policy_version 1679460 (0.0007) [2023-12-27 03:37:48,002][105692] Updated weights for policy 0, policy_version 1679470 (0.0007) [2023-12-27 03:37:48,064][105692] Updated weights for policy 0, policy_version 1679480 (0.0008) [2023-12-27 03:37:48,464][105620] Updated weights for policy 1, policy_version 1683158 (0.0009) [2023-12-27 03:37:48,532][105620] Updated weights for policy 1, policy_version 1683168 (0.0010) [2023-12-27 03:37:48,595][105620] Updated weights for policy 1, policy_version 1683178 (0.0010) [2023-12-27 03:37:48,781][105692] Updated weights for policy 0, policy_version 1679491 (0.0009) [2023-12-27 03:37:48,841][105692] Updated weights for policy 0, policy_version 1679501 (0.0008) [2023-12-27 03:37:48,886][105692] Updated weights for policy 0, policy_version 1679511 (0.0008) [2023-12-27 03:37:49,367][105620] Updated weights for policy 1, policy_version 1683188 (0.0009) [2023-12-27 03:37:49,432][105620] Updated weights for policy 1, policy_version 1683198 (0.0010) [2023-12-27 03:37:49,498][105620] Updated weights for policy 1, policy_version 1683208 (0.0010) [2023-12-27 03:37:49,681][105692] Updated weights for policy 0, policy_version 1679521 (0.0008) [2023-12-27 03:37:49,747][105692] Updated weights for policy 0, policy_version 1679531 (0.0008) [2023-12-27 03:37:49,802][105692] Updated weights for policy 0, policy_version 1679541 (0.0008) [2023-12-27 03:37:49,866][105692] Updated weights for policy 0, policy_version 1679551 (0.0009) [2023-12-27 03:37:50,247][105620] Updated weights for policy 1, policy_version 1683218 (0.0009) [2023-12-27 03:37:50,316][105620] Updated weights for policy 1, policy_version 1683228 (0.0007) [2023-12-27 03:37:50,392][105620] Updated weights for policy 1, policy_version 1683238 (0.0007) [2023-12-27 03:37:50,453][105620] Updated weights for policy 1, policy_version 1683248 (0.0006) [2023-12-27 03:37:50,587][105692] Updated weights for policy 0, policy_version 1679561 (0.0007) [2023-12-27 03:37:50,659][105692] Updated weights for policy 0, policy_version 1679571 (0.0006) [2023-12-27 03:37:50,721][105692] Updated weights for policy 0, policy_version 1679581 (0.0009) [2023-12-27 03:37:51,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 861011968. Throughput: 0: 9670.4, 1: 9955.2. Samples: 861003508. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:37:51,062][104569] Avg episode reward: [(0, '8715.430'), (1, '9080.057')] [2023-12-27 03:37:51,080][105620] Updated weights for policy 1, policy_version 1683258 (0.0008) [2023-12-27 03:37:51,143][105620] Updated weights for policy 1, policy_version 1683268 (0.0007) [2023-12-27 03:37:51,208][105620] Updated weights for policy 1, policy_version 1683278 (0.0010) [2023-12-27 03:37:51,470][105692] Updated weights for policy 0, policy_version 1679591 (0.0009) [2023-12-27 03:37:51,533][105692] Updated weights for policy 0, policy_version 1679601 (0.0008) [2023-12-27 03:37:51,599][105692] Updated weights for policy 0, policy_version 1679611 (0.0009) [2023-12-27 03:37:51,895][105620] Updated weights for policy 1, policy_version 1683288 (0.0007) [2023-12-27 03:37:51,953][105620] Updated weights for policy 1, policy_version 1683298 (0.0007) [2023-12-27 03:37:52,018][105620] Updated weights for policy 1, policy_version 1683308 (0.0010) [2023-12-27 03:37:52,400][105692] Updated weights for policy 0, policy_version 1679621 (0.0009) [2023-12-27 03:37:52,462][105692] Updated weights for policy 0, policy_version 1679631 (0.0010) [2023-12-27 03:37:52,523][105692] Updated weights for policy 0, policy_version 1679641 (0.0009) [2023-12-27 03:37:52,663][105620] Updated weights for policy 1, policy_version 1683318 (0.0010) [2023-12-27 03:37:52,716][105620] Updated weights for policy 1, policy_version 1683328 (0.0007) [2023-12-27 03:37:52,777][105620] Updated weights for policy 1, policy_version 1683338 (0.0008) [2023-12-27 03:37:53,326][105692] Updated weights for policy 0, policy_version 1679651 (0.0009) [2023-12-27 03:37:53,375][105692] Updated weights for policy 0, policy_version 1679662 (0.0009) [2023-12-27 03:37:53,421][105620] Updated weights for policy 1, policy_version 1683348 (0.0005) [2023-12-27 03:37:53,426][105692] Updated weights for policy 0, policy_version 1679672 (0.0007) [2023-12-27 03:37:53,483][105620] Updated weights for policy 1, policy_version 1683358 (0.0005) [2023-12-27 03:37:53,529][105620] Updated weights for policy 1, policy_version 1683368 (0.0007) [2023-12-27 03:37:54,135][105692] Updated weights for policy 0, policy_version 1679682 (0.0006) [2023-12-27 03:37:54,174][105620] Updated weights for policy 1, policy_version 1683378 (0.0008) [2023-12-27 03:37:54,193][105692] Updated weights for policy 0, policy_version 1679692 (0.0007) [2023-12-27 03:37:54,236][105620] Updated weights for policy 1, policy_version 1683388 (0.0008) [2023-12-27 03:37:54,242][105692] Updated weights for policy 0, policy_version 1679702 (0.0008) [2023-12-27 03:37:54,290][105692] Updated weights for policy 0, policy_version 1679712 (0.0008) [2023-12-27 03:37:54,299][105620] Updated weights for policy 1, policy_version 1683398 (0.0006) [2023-12-27 03:37:54,362][105620] Updated weights for policy 1, policy_version 1683408 (0.0005) [2023-12-27 03:37:54,924][105620] Updated weights for policy 1, policy_version 1683418 (0.0005) [2023-12-27 03:37:54,986][105620] Updated weights for policy 1, policy_version 1683428 (0.0006) [2023-12-27 03:37:55,057][105620] Updated weights for policy 1, policy_version 1683438 (0.0008) [2023-12-27 03:37:55,166][105692] Updated weights for policy 0, policy_version 1679722 (0.0009) [2023-12-27 03:37:55,221][105692] Updated weights for policy 0, policy_version 1679732 (0.0009) [2023-12-27 03:37:55,278][105692] Updated weights for policy 0, policy_version 1679742 (0.0009) [2023-12-27 03:37:55,780][105620] Updated weights for policy 1, policy_version 1683448 (0.0010) [2023-12-27 03:37:55,842][105620] Updated weights for policy 1, policy_version 1683458 (0.0011) [2023-12-27 03:37:55,897][105620] Updated weights for policy 1, policy_version 1683468 (0.0010) [2023-12-27 03:37:55,998][105692] Updated weights for policy 0, policy_version 1679752 (0.0010) [2023-12-27 03:37:56,060][105692] Updated weights for policy 0, policy_version 1679762 (0.0010) [2023-12-27 03:37:56,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 861110272. Throughput: 0: 9679.7, 1: 10018.4. Samples: 861119832. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:37:56,063][104569] Avg episode reward: [(0, '8807.841'), (1, '8805.108')] [2023-12-27 03:37:56,138][105692] Updated weights for policy 0, policy_version 1679772 (0.0010) [2023-12-27 03:37:56,549][105620] Updated weights for policy 1, policy_version 1683478 (0.0007) [2023-12-27 03:37:56,598][105620] Updated weights for policy 1, policy_version 1683488 (0.0006) [2023-12-27 03:37:56,646][105620] Updated weights for policy 1, policy_version 1683498 (0.0008) [2023-12-27 03:37:56,845][105692] Updated weights for policy 0, policy_version 1679782 (0.0008) [2023-12-27 03:37:56,898][105692] Updated weights for policy 0, policy_version 1679792 (0.0005) [2023-12-27 03:37:56,965][105692] Updated weights for policy 0, policy_version 1679802 (0.0007) [2023-12-27 03:37:57,402][105620] Updated weights for policy 1, policy_version 1683508 (0.0008) [2023-12-27 03:37:57,459][105620] Updated weights for policy 1, policy_version 1683518 (0.0010) [2023-12-27 03:37:57,512][105620] Updated weights for policy 1, policy_version 1683528 (0.0010) [2023-12-27 03:37:57,577][105692] Updated weights for policy 0, policy_version 1679812 (0.0008) [2023-12-27 03:37:57,630][105692] Updated weights for policy 0, policy_version 1679822 (0.0005) [2023-12-27 03:37:57,680][105692] Updated weights for policy 0, policy_version 1679832 (0.0007) [2023-12-27 03:37:58,332][105620] Updated weights for policy 1, policy_version 1683539 (0.0009) [2023-12-27 03:37:58,397][105620] Updated weights for policy 1, policy_version 1683549 (0.0009) [2023-12-27 03:37:58,407][105692] Updated weights for policy 0, policy_version 1679842 (0.0010) [2023-12-27 03:37:58,459][105620] Updated weights for policy 1, policy_version 1683559 (0.0007) [2023-12-27 03:37:58,472][105692] Updated weights for policy 0, policy_version 1679852 (0.0008) [2023-12-27 03:37:58,535][105692] Updated weights for policy 0, policy_version 1679862 (0.0008) [2023-12-27 03:37:58,597][105692] Updated weights for policy 0, policy_version 1679872 (0.0007) [2023-12-27 03:37:59,324][105620] Updated weights for policy 1, policy_version 1683569 (0.0007) [2023-12-27 03:37:59,392][105620] Updated weights for policy 1, policy_version 1683579 (0.0007) [2023-12-27 03:37:59,453][105620] Updated weights for policy 1, policy_version 1683589 (0.0010) [2023-12-27 03:37:59,475][105692] Updated weights for policy 0, policy_version 1679882 (0.0006) [2023-12-27 03:37:59,517][105620] Updated weights for policy 1, policy_version 1683599 (0.0008) [2023-12-27 03:37:59,531][105692] Updated weights for policy 0, policy_version 1679892 (0.0005) [2023-12-27 03:37:59,587][105692] Updated weights for policy 0, policy_version 1679902 (0.0005) [2023-12-27 03:38:00,151][105692] Updated weights for policy 0, policy_version 1679912 (0.0008) [2023-12-27 03:38:00,208][105692] Updated weights for policy 0, policy_version 1679922 (0.0010) [2023-12-27 03:38:00,270][105692] Updated weights for policy 0, policy_version 1679932 (0.0011) [2023-12-27 03:38:00,354][105620] Updated weights for policy 1, policy_version 1683609 (0.0008) [2023-12-27 03:38:00,411][105620] Updated weights for policy 1, policy_version 1683619 (0.0009) [2023-12-27 03:38:00,467][105620] Updated weights for policy 1, policy_version 1683629 (0.0005) [2023-12-27 03:38:00,958][105692] Updated weights for policy 0, policy_version 1679942 (0.0007) [2023-12-27 03:38:01,023][105692] Updated weights for policy 0, policy_version 1679952 (0.0006) [2023-12-27 03:38:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 861200384. Throughput: 0: 9744.1, 1: 9999.4. Samples: 861177964. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:38:01,063][104569] Avg episode reward: [(0, '8718.483'), (1, '8987.890')] [2023-12-27 03:38:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001683632_431071232.pth... [2023-12-27 03:38:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001682480_430776320.pth [2023-12-27 03:38:01,087][105692] Updated weights for policy 0, policy_version 1679962 (0.0009) [2023-12-27 03:38:01,117][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001679968_430137344.pth... [2023-12-27 03:38:01,122][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001678848_429850624.pth [2023-12-27 03:38:01,208][105620] Updated weights for policy 1, policy_version 1683639 (0.0008) [2023-12-27 03:38:01,272][105620] Updated weights for policy 1, policy_version 1683649 (0.0008) [2023-12-27 03:38:01,324][105620] Updated weights for policy 1, policy_version 1683659 (0.0009) [2023-12-27 03:38:01,741][105692] Updated weights for policy 0, policy_version 1679972 (0.0008) [2023-12-27 03:38:01,796][105692] Updated weights for policy 0, policy_version 1679982 (0.0008) [2023-12-27 03:38:01,854][105692] Updated weights for policy 0, policy_version 1679992 (0.0009) [2023-12-27 03:38:02,109][105620] Updated weights for policy 1, policy_version 1683670 (0.0009) [2023-12-27 03:38:02,171][105620] Updated weights for policy 1, policy_version 1683680 (0.0008) [2023-12-27 03:38:02,236][105620] Updated weights for policy 1, policy_version 1683690 (0.0009) [2023-12-27 03:38:02,601][105692] Updated weights for policy 0, policy_version 1680002 (0.0008) [2023-12-27 03:38:02,650][105692] Updated weights for policy 0, policy_version 1680012 (0.0005) [2023-12-27 03:38:02,704][105692] Updated weights for policy 0, policy_version 1680022 (0.0007) [2023-12-27 03:38:02,753][105692] Updated weights for policy 0, policy_version 1680032 (0.0006) [2023-12-27 03:38:03,003][105620] Updated weights for policy 1, policy_version 1683700 (0.0007) [2023-12-27 03:38:03,062][105620] Updated weights for policy 1, policy_version 1683710 (0.0010) [2023-12-27 03:38:03,126][105620] Updated weights for policy 1, policy_version 1683720 (0.0009) [2023-12-27 03:38:03,318][105692] Updated weights for policy 0, policy_version 1680042 (0.0011) [2023-12-27 03:38:03,376][105692] Updated weights for policy 0, policy_version 1680052 (0.0010) [2023-12-27 03:38:03,436][105692] Updated weights for policy 0, policy_version 1680062 (0.0009) [2023-12-27 03:38:03,802][105620] Updated weights for policy 1, policy_version 1683730 (0.0009) [2023-12-27 03:38:03,870][105620] Updated weights for policy 1, policy_version 1683740 (0.0009) [2023-12-27 03:38:03,937][105620] Updated weights for policy 1, policy_version 1683750 (0.0008) [2023-12-27 03:38:04,005][105620] Updated weights for policy 1, policy_version 1683760 (0.0007) [2023-12-27 03:38:04,046][105692] Updated weights for policy 0, policy_version 1680072 (0.0008) [2023-12-27 03:38:04,114][105692] Updated weights for policy 0, policy_version 1680082 (0.0010) [2023-12-27 03:38:04,177][105692] Updated weights for policy 0, policy_version 1680092 (0.0010) [2023-12-27 03:38:04,653][105620] Updated weights for policy 1, policy_version 1683770 (0.0005) [2023-12-27 03:38:04,708][105620] Updated weights for policy 1, policy_version 1683780 (0.0005) [2023-12-27 03:38:04,765][105620] Updated weights for policy 1, policy_version 1683790 (0.0008) [2023-12-27 03:38:04,978][105692] Updated weights for policy 0, policy_version 1680102 (0.0008) [2023-12-27 03:38:05,046][105692] Updated weights for policy 0, policy_version 1680112 (0.0006) [2023-12-27 03:38:05,104][105692] Updated weights for policy 0, policy_version 1680122 (0.0009) [2023-12-27 03:38:05,552][105620] Updated weights for policy 1, policy_version 1683800 (0.0009) [2023-12-27 03:38:05,612][105620] Updated weights for policy 1, policy_version 1683810 (0.0008) [2023-12-27 03:38:05,667][105620] Updated weights for policy 1, policy_version 1683820 (0.0008) [2023-12-27 03:38:05,689][105692] Updated weights for policy 0, policy_version 1680132 (0.0007) [2023-12-27 03:38:05,756][105692] Updated weights for policy 0, policy_version 1680142 (0.0007) [2023-12-27 03:38:05,814][105692] Updated weights for policy 0, policy_version 1680152 (0.0008) [2023-12-27 03:38:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 861306880. Throughput: 0: 9699.0, 1: 9863.8. Samples: 861294780. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:38:06,063][104569] Avg episode reward: [(0, '8901.465'), (1, '8713.134')] [2023-12-27 03:38:06,411][105692] Updated weights for policy 0, policy_version 1680162 (0.0006) [2023-12-27 03:38:06,475][105692] Updated weights for policy 0, policy_version 1680172 (0.0009) [2023-12-27 03:38:06,523][105620] Updated weights for policy 1, policy_version 1683830 (0.0009) [2023-12-27 03:38:06,535][105692] Updated weights for policy 0, policy_version 1680182 (0.0010) [2023-12-27 03:38:06,580][105620] Updated weights for policy 1, policy_version 1683840 (0.0009) [2023-12-27 03:38:06,594][105692] Updated weights for policy 0, policy_version 1680192 (0.0011) [2023-12-27 03:38:06,628][105620] Updated weights for policy 1, policy_version 1683850 (0.0007) [2023-12-27 03:38:07,160][105692] Updated weights for policy 0, policy_version 1680202 (0.0009) [2023-12-27 03:38:07,229][105692] Updated weights for policy 0, policy_version 1680212 (0.0009) [2023-12-27 03:38:07,281][105692] Updated weights for policy 0, policy_version 1680222 (0.0010) [2023-12-27 03:38:07,487][105620] Updated weights for policy 1, policy_version 1683860 (0.0008) [2023-12-27 03:38:07,545][105620] Updated weights for policy 1, policy_version 1683870 (0.0008) [2023-12-27 03:38:07,604][105620] Updated weights for policy 1, policy_version 1683880 (0.0008) [2023-12-27 03:38:07,969][105692] Updated weights for policy 0, policy_version 1680232 (0.0006) [2023-12-27 03:38:08,030][105692] Updated weights for policy 0, policy_version 1680242 (0.0008) [2023-12-27 03:38:08,099][105692] Updated weights for policy 0, policy_version 1680252 (0.0009) [2023-12-27 03:38:08,379][105620] Updated weights for policy 1, policy_version 1683890 (0.0008) [2023-12-27 03:38:08,454][105620] Updated weights for policy 1, policy_version 1683900 (0.0005) [2023-12-27 03:38:08,509][105620] Updated weights for policy 1, policy_version 1683910 (0.0005) [2023-12-27 03:38:08,573][105620] Updated weights for policy 1, policy_version 1683920 (0.0007) [2023-12-27 03:38:08,835][105692] Updated weights for policy 0, policy_version 1680262 (0.0007) [2023-12-27 03:38:08,887][105692] Updated weights for policy 0, policy_version 1680272 (0.0005) [2023-12-27 03:38:08,941][105692] Updated weights for policy 0, policy_version 1680282 (0.0006) [2023-12-27 03:38:09,299][105620] Updated weights for policy 1, policy_version 1683930 (0.0009) [2023-12-27 03:38:09,362][105620] Updated weights for policy 1, policy_version 1683940 (0.0008) [2023-12-27 03:38:09,433][105620] Updated weights for policy 1, policy_version 1683950 (0.0009) [2023-12-27 03:38:09,638][105692] Updated weights for policy 0, policy_version 1680292 (0.0009) [2023-12-27 03:38:09,696][105692] Updated weights for policy 0, policy_version 1680302 (0.0005) [2023-12-27 03:38:09,751][105692] Updated weights for policy 0, policy_version 1680312 (0.0006) [2023-12-27 03:38:10,230][105620] Updated weights for policy 1, policy_version 1683960 (0.0009) [2023-12-27 03:38:10,295][105620] Updated weights for policy 1, policy_version 1683970 (0.0009) [2023-12-27 03:38:10,314][105586] KL-divergence is very high: 116.9767 [2023-12-27 03:38:10,354][105620] Updated weights for policy 1, policy_version 1683980 (0.0009) [2023-12-27 03:38:10,362][105586] KL-divergence is very high: 157.0540 [2023-12-27 03:38:10,484][105692] Updated weights for policy 0, policy_version 1680322 (0.0009) [2023-12-27 03:38:10,546][105692] Updated weights for policy 0, policy_version 1680332 (0.0009) [2023-12-27 03:38:10,605][105692] Updated weights for policy 0, policy_version 1680342 (0.0009) [2023-12-27 03:38:10,664][105692] Updated weights for policy 0, policy_version 1680352 (0.0008) [2023-12-27 03:38:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 861396992. Throughput: 0: 9730.7, 1: 9760.0. Samples: 861408396. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:38:11,062][104569] Avg episode reward: [(0, '8898.350'), (1, '8533.777')] [2023-12-27 03:38:11,082][105620] Updated weights for policy 1, policy_version 1683990 (0.0009) [2023-12-27 03:38:11,154][105620] Updated weights for policy 1, policy_version 1684000 (0.0008) [2023-12-27 03:38:11,217][105620] Updated weights for policy 1, policy_version 1684010 (0.0008) [2023-12-27 03:38:11,497][105692] Updated weights for policy 0, policy_version 1680362 (0.0009) [2023-12-27 03:38:11,563][105692] Updated weights for policy 0, policy_version 1680372 (0.0009) [2023-12-27 03:38:11,624][105692] Updated weights for policy 0, policy_version 1680382 (0.0009) [2023-12-27 03:38:11,945][105620] Updated weights for policy 1, policy_version 1684020 (0.0008) [2023-12-27 03:38:12,004][105620] Updated weights for policy 1, policy_version 1684030 (0.0008) [2023-12-27 03:38:12,050][105620] Updated weights for policy 1, policy_version 1684040 (0.0007) [2023-12-27 03:38:12,418][105692] Updated weights for policy 0, policy_version 1680392 (0.0008) [2023-12-27 03:38:12,480][105692] Updated weights for policy 0, policy_version 1680402 (0.0010) [2023-12-27 03:38:12,538][105692] Updated weights for policy 0, policy_version 1680412 (0.0010) [2023-12-27 03:38:12,722][105620] Updated weights for policy 1, policy_version 1684050 (0.0006) [2023-12-27 03:38:12,786][105620] Updated weights for policy 1, policy_version 1684060 (0.0008) [2023-12-27 03:38:12,848][105620] Updated weights for policy 1, policy_version 1684070 (0.0009) [2023-12-27 03:38:12,895][105620] Updated weights for policy 1, policy_version 1684080 (0.0008) [2023-12-27 03:38:13,309][105692] Updated weights for policy 0, policy_version 1680422 (0.0006) [2023-12-27 03:38:13,373][105692] Updated weights for policy 0, policy_version 1680432 (0.0005) [2023-12-27 03:38:13,433][105692] Updated weights for policy 0, policy_version 1680442 (0.0005) [2023-12-27 03:38:13,725][105620] Updated weights for policy 1, policy_version 1684090 (0.0009) [2023-12-27 03:38:13,787][105620] Updated weights for policy 1, policy_version 1684100 (0.0009) [2023-12-27 03:38:13,842][105620] Updated weights for policy 1, policy_version 1684110 (0.0009) [2023-12-27 03:38:14,058][105692] Updated weights for policy 0, policy_version 1680452 (0.0007) [2023-12-27 03:38:14,115][105692] Updated weights for policy 0, policy_version 1680462 (0.0008) [2023-12-27 03:38:14,161][105692] Updated weights for policy 0, policy_version 1680472 (0.0008) [2023-12-27 03:38:14,589][105620] Updated weights for policy 1, policy_version 1684120 (0.0009) [2023-12-27 03:38:14,634][105620] Updated weights for policy 1, policy_version 1684130 (0.0008) [2023-12-27 03:38:14,682][105620] Updated weights for policy 1, policy_version 1684140 (0.0009) [2023-12-27 03:38:14,923][105692] Updated weights for policy 0, policy_version 1680482 (0.0009) [2023-12-27 03:38:14,983][105692] Updated weights for policy 0, policy_version 1680492 (0.0011) [2023-12-27 03:38:15,051][105692] Updated weights for policy 0, policy_version 1680502 (0.0009) [2023-12-27 03:38:15,116][105692] Updated weights for policy 0, policy_version 1680512 (0.0005) [2023-12-27 03:38:15,487][105620] Updated weights for policy 1, policy_version 1684150 (0.0008) [2023-12-27 03:38:15,536][105620] Updated weights for policy 1, policy_version 1684160 (0.0008) [2023-12-27 03:38:15,600][105620] Updated weights for policy 1, policy_version 1684170 (0.0009) [2023-12-27 03:38:15,768][105692] Updated weights for policy 0, policy_version 1680522 (0.0006) [2023-12-27 03:38:15,836][105692] Updated weights for policy 0, policy_version 1680532 (0.0010) [2023-12-27 03:38:15,891][105692] Updated weights for policy 0, policy_version 1680542 (0.0010) [2023-12-27 03:38:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 861495296. Throughput: 0: 9668.2, 1: 9700.1. Samples: 861463612. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:38:16,063][104569] Avg episode reward: [(0, '8713.026'), (1, '8443.394')] [2023-12-27 03:38:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001680544_430284800.pth... [2023-12-27 03:38:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001684176_431210496.pth... [2023-12-27 03:38:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001679392_429989888.pth [2023-12-27 03:38:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001683088_430931968.pth [2023-12-27 03:38:16,400][105620] Updated weights for policy 1, policy_version 1684180 (0.0009) [2023-12-27 03:38:16,466][105620] Updated weights for policy 1, policy_version 1684190 (0.0008) [2023-12-27 03:38:16,529][105620] Updated weights for policy 1, policy_version 1684200 (0.0008) [2023-12-27 03:38:16,567][105692] Updated weights for policy 0, policy_version 1680552 (0.0010) [2023-12-27 03:38:16,629][105692] Updated weights for policy 0, policy_version 1680562 (0.0010) [2023-12-27 03:38:16,686][105692] Updated weights for policy 0, policy_version 1680572 (0.0010) [2023-12-27 03:38:17,269][105620] Updated weights for policy 1, policy_version 1684210 (0.0007) [2023-12-27 03:38:17,324][105620] Updated weights for policy 1, policy_version 1684220 (0.0005) [2023-12-27 03:38:17,373][105692] Updated weights for policy 0, policy_version 1680582 (0.0009) [2023-12-27 03:38:17,377][105620] Updated weights for policy 1, policy_version 1684230 (0.0005) [2023-12-27 03:38:17,433][105620] Updated weights for policy 1, policy_version 1684240 (0.0007) [2023-12-27 03:38:17,435][105692] Updated weights for policy 0, policy_version 1680592 (0.0010) [2023-12-27 03:38:17,487][105692] Updated weights for policy 0, policy_version 1680602 (0.0010) [2023-12-27 03:38:18,080][105692] Updated weights for policy 0, policy_version 1680612 (0.0009) [2023-12-27 03:38:18,132][105692] Updated weights for policy 0, policy_version 1680622 (0.0010) [2023-12-27 03:38:18,184][105692] Updated weights for policy 0, policy_version 1680632 (0.0010) [2023-12-27 03:38:18,194][105620] Updated weights for policy 1, policy_version 1684250 (0.0006) [2023-12-27 03:38:18,242][105620] Updated weights for policy 1, policy_version 1684260 (0.0006) [2023-12-27 03:38:18,294][105620] Updated weights for policy 1, policy_version 1684270 (0.0008) [2023-12-27 03:38:18,937][105692] Updated weights for policy 0, policy_version 1680642 (0.0009) [2023-12-27 03:38:18,991][105692] Updated weights for policy 0, policy_version 1680652 (0.0009) [2023-12-27 03:38:19,050][105692] Updated weights for policy 0, policy_version 1680662 (0.0010) [2023-12-27 03:38:19,079][105620] Updated weights for policy 1, policy_version 1684280 (0.0007) [2023-12-27 03:38:19,105][105692] Updated weights for policy 0, policy_version 1680672 (0.0010) [2023-12-27 03:38:19,130][105620] Updated weights for policy 1, policy_version 1684290 (0.0007) [2023-12-27 03:38:19,182][105620] Updated weights for policy 1, policy_version 1684300 (0.0008) [2023-12-27 03:38:19,877][105692] Updated weights for policy 0, policy_version 1680682 (0.0011) [2023-12-27 03:38:19,946][105692] Updated weights for policy 0, policy_version 1680692 (0.0011) [2023-12-27 03:38:20,004][105620] Updated weights for policy 1, policy_version 1684310 (0.0007) [2023-12-27 03:38:20,010][105692] Updated weights for policy 0, policy_version 1680702 (0.0011) [2023-12-27 03:38:20,056][105620] Updated weights for policy 1, policy_version 1684320 (0.0008) [2023-12-27 03:38:20,113][105620] Updated weights for policy 1, policy_version 1684330 (0.0008) [2023-12-27 03:38:20,753][105692] Updated weights for policy 0, policy_version 1680712 (0.0011) [2023-12-27 03:38:20,812][105692] Updated weights for policy 0, policy_version 1680722 (0.0010) [2023-12-27 03:38:20,875][105692] Updated weights for policy 0, policy_version 1680732 (0.0011) [2023-12-27 03:38:20,905][105620] Updated weights for policy 1, policy_version 1684340 (0.0009) [2023-12-27 03:38:20,966][105620] Updated weights for policy 1, policy_version 1684350 (0.0008) [2023-12-27 03:38:21,025][105620] Updated weights for policy 1, policy_version 1684360 (0.0008) [2023-12-27 03:38:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 861585408. Throughput: 0: 9715.8, 1: 9572.6. Samples: 861578200. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:38:21,062][104569] Avg episode reward: [(0, '8715.635'), (1, '8346.560')] [2023-12-27 03:38:21,573][105692] Updated weights for policy 0, policy_version 1680742 (0.0010) [2023-12-27 03:38:21,635][105692] Updated weights for policy 0, policy_version 1680752 (0.0007) [2023-12-27 03:38:21,701][105692] Updated weights for policy 0, policy_version 1680762 (0.0008) [2023-12-27 03:38:21,847][105620] Updated weights for policy 1, policy_version 1684370 (0.0009) [2023-12-27 03:38:21,911][105620] Updated weights for policy 1, policy_version 1684380 (0.0009) [2023-12-27 03:38:21,981][105620] Updated weights for policy 1, policy_version 1684390 (0.0008) [2023-12-27 03:38:22,047][105620] Updated weights for policy 1, policy_version 1684400 (0.0009) [2023-12-27 03:38:22,351][105692] Updated weights for policy 0, policy_version 1680772 (0.0008) [2023-12-27 03:38:22,416][105692] Updated weights for policy 0, policy_version 1680782 (0.0008) [2023-12-27 03:38:22,480][105692] Updated weights for policy 0, policy_version 1680792 (0.0008) [2023-12-27 03:38:22,755][105620] Updated weights for policy 1, policy_version 1684410 (0.0009) [2023-12-27 03:38:22,809][105620] Updated weights for policy 1, policy_version 1684420 (0.0009) [2023-12-27 03:38:22,857][105620] Updated weights for policy 1, policy_version 1684430 (0.0009) [2023-12-27 03:38:23,230][105692] Updated weights for policy 0, policy_version 1680802 (0.0008) [2023-12-27 03:38:23,281][105692] Updated weights for policy 0, policy_version 1680812 (0.0005) [2023-12-27 03:38:23,337][105692] Updated weights for policy 0, policy_version 1680822 (0.0005) [2023-12-27 03:38:23,392][105692] Updated weights for policy 0, policy_version 1680832 (0.0005) [2023-12-27 03:38:23,660][105620] Updated weights for policy 1, policy_version 1684440 (0.0009) [2023-12-27 03:38:23,713][105620] Updated weights for policy 1, policy_version 1684450 (0.0009) [2023-12-27 03:38:23,766][105620] Updated weights for policy 1, policy_version 1684460 (0.0009) [2023-12-27 03:38:24,077][105692] Updated weights for policy 0, policy_version 1680842 (0.0006) [2023-12-27 03:38:24,138][105692] Updated weights for policy 0, policy_version 1680852 (0.0005) [2023-12-27 03:38:24,197][105692] Updated weights for policy 0, policy_version 1680862 (0.0007) [2023-12-27 03:38:24,518][105620] Updated weights for policy 1, policy_version 1684470 (0.0007) [2023-12-27 03:38:24,575][105620] Updated weights for policy 1, policy_version 1684480 (0.0006) [2023-12-27 03:38:24,642][105620] Updated weights for policy 1, policy_version 1684490 (0.0006) [2023-12-27 03:38:24,929][105692] Updated weights for policy 0, policy_version 1680872 (0.0009) [2023-12-27 03:38:24,983][105692] Updated weights for policy 0, policy_version 1680882 (0.0008) [2023-12-27 03:38:25,045][105692] Updated weights for policy 0, policy_version 1680892 (0.0009) [2023-12-27 03:38:25,270][105620] Updated weights for policy 1, policy_version 1684500 (0.0010) [2023-12-27 03:38:25,325][105620] Updated weights for policy 1, policy_version 1684510 (0.0010) [2023-12-27 03:38:25,379][105620] Updated weights for policy 1, policy_version 1684520 (0.0010) [2023-12-27 03:38:25,856][105692] Updated weights for policy 0, policy_version 1680902 (0.0009) [2023-12-27 03:38:25,915][105692] Updated weights for policy 0, policy_version 1680912 (0.0008) [2023-12-27 03:38:25,971][105692] Updated weights for policy 0, policy_version 1680922 (0.0008) [2023-12-27 03:38:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 861683712. Throughput: 0: 9737.2, 1: 9416.5. Samples: 861691164. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:38:26,062][104569] Avg episode reward: [(0, '8806.679'), (1, '8623.207')] [2023-12-27 03:38:26,117][105620] Updated weights for policy 1, policy_version 1684530 (0.0010) [2023-12-27 03:38:26,172][105620] Updated weights for policy 1, policy_version 1684540 (0.0008) [2023-12-27 03:38:26,227][105620] Updated weights for policy 1, policy_version 1684550 (0.0009) [2023-12-27 03:38:26,274][105620] Updated weights for policy 1, policy_version 1684560 (0.0009) [2023-12-27 03:38:26,743][105692] Updated weights for policy 0, policy_version 1680932 (0.0007) [2023-12-27 03:38:26,811][105692] Updated weights for policy 0, policy_version 1680942 (0.0005) [2023-12-27 03:38:26,860][105692] Updated weights for policy 0, policy_version 1680952 (0.0005) [2023-12-27 03:38:26,905][105620] Updated weights for policy 1, policy_version 1684570 (0.0005) [2023-12-27 03:38:26,963][105620] Updated weights for policy 1, policy_version 1684580 (0.0005) [2023-12-27 03:38:27,023][105620] Updated weights for policy 1, policy_version 1684590 (0.0006) [2023-12-27 03:38:27,374][105692] Updated weights for policy 0, policy_version 1680962 (0.0007) [2023-12-27 03:38:27,442][105692] Updated weights for policy 0, policy_version 1680972 (0.0010) [2023-12-27 03:38:27,506][105692] Updated weights for policy 0, policy_version 1680982 (0.0010) [2023-12-27 03:38:27,541][105620] Updated weights for policy 1, policy_version 1684600 (0.0006) [2023-12-27 03:38:27,560][105692] Updated weights for policy 0, policy_version 1680992 (0.0010) [2023-12-27 03:38:27,602][105620] Updated weights for policy 1, policy_version 1684610 (0.0008) [2023-12-27 03:38:27,665][105620] Updated weights for policy 1, policy_version 1684620 (0.0008) [2023-12-27 03:38:28,278][105692] Updated weights for policy 0, policy_version 1681002 (0.0010) [2023-12-27 03:38:28,338][105620] Updated weights for policy 1, policy_version 1684630 (0.0008) [2023-12-27 03:38:28,340][105692] Updated weights for policy 0, policy_version 1681012 (0.0010) [2023-12-27 03:38:28,399][105692] Updated weights for policy 0, policy_version 1681022 (0.0011) [2023-12-27 03:38:28,406][105620] Updated weights for policy 1, policy_version 1684640 (0.0006) [2023-12-27 03:38:28,468][105620] Updated weights for policy 1, policy_version 1684650 (0.0008) [2023-12-27 03:38:29,118][105692] Updated weights for policy 0, policy_version 1681032 (0.0007) [2023-12-27 03:38:29,170][105692] Updated weights for policy 0, policy_version 1681042 (0.0006) [2023-12-27 03:38:29,215][105620] Updated weights for policy 1, policy_version 1684660 (0.0008) [2023-12-27 03:38:29,228][105692] Updated weights for policy 0, policy_version 1681052 (0.0007) [2023-12-27 03:38:29,274][105620] Updated weights for policy 1, policy_version 1684670 (0.0008) [2023-12-27 03:38:29,333][105620] Updated weights for policy 1, policy_version 1684680 (0.0008) [2023-12-27 03:38:29,940][105692] Updated weights for policy 0, policy_version 1681062 (0.0011) [2023-12-27 03:38:30,001][105692] Updated weights for policy 0, policy_version 1681072 (0.0010) [2023-12-27 03:38:30,043][105620] Updated weights for policy 1, policy_version 1684690 (0.0008) [2023-12-27 03:38:30,067][105692] Updated weights for policy 0, policy_version 1681082 (0.0010) [2023-12-27 03:38:30,099][105620] Updated weights for policy 1, policy_version 1684700 (0.0006) [2023-12-27 03:38:30,166][105620] Updated weights for policy 1, policy_version 1684710 (0.0008) [2023-12-27 03:38:30,219][105620] Updated weights for policy 1, policy_version 1684720 (0.0008) [2023-12-27 03:38:30,713][105692] Updated weights for policy 0, policy_version 1681092 (0.0010) [2023-12-27 03:38:30,761][105692] Updated weights for policy 0, policy_version 1681102 (0.0010) [2023-12-27 03:38:30,794][105620] Updated weights for policy 1, policy_version 1684730 (0.0006) [2023-12-27 03:38:30,818][105692] Updated weights for policy 0, policy_version 1681112 (0.0010) [2023-12-27 03:38:30,844][105620] Updated weights for policy 1, policy_version 1684740 (0.0007) [2023-12-27 03:38:30,902][105620] Updated weights for policy 1, policy_version 1684750 (0.0007) [2023-12-27 03:38:31,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 861790208. Throughput: 0: 9747.7, 1: 9472.8. Samples: 861753148. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:38:31,063][104569] Avg episode reward: [(0, '8715.631'), (1, '9084.074')] [2023-12-27 03:38:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001681120_430432256.pth... [2023-12-27 03:38:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001684752_431357952.pth... [2023-12-27 03:38:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001683632_431071232.pth [2023-12-27 03:38:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001679968_430137344.pth [2023-12-27 03:38:31,540][105692] Updated weights for policy 0, policy_version 1681122 (0.0010) [2023-12-27 03:38:31,601][105692] Updated weights for policy 0, policy_version 1681132 (0.0009) [2023-12-27 03:38:31,605][105620] Updated weights for policy 1, policy_version 1684760 (0.0006) [2023-12-27 03:38:31,661][105692] Updated weights for policy 0, policy_version 1681142 (0.0010) [2023-12-27 03:38:31,675][105620] Updated weights for policy 1, policy_version 1684770 (0.0007) [2023-12-27 03:38:31,725][105692] Updated weights for policy 0, policy_version 1681152 (0.0010) [2023-12-27 03:38:31,744][105620] Updated weights for policy 1, policy_version 1684780 (0.0007) [2023-12-27 03:38:32,394][105620] Updated weights for policy 1, policy_version 1684790 (0.0008) [2023-12-27 03:38:32,450][105620] Updated weights for policy 1, policy_version 1684800 (0.0008) [2023-12-27 03:38:32,463][105692] Updated weights for policy 0, policy_version 1681162 (0.0011) [2023-12-27 03:38:32,514][105620] Updated weights for policy 1, policy_version 1684810 (0.0007) [2023-12-27 03:38:32,520][105692] Updated weights for policy 0, policy_version 1681172 (0.0011) [2023-12-27 03:38:32,571][105692] Updated weights for policy 0, policy_version 1681182 (0.0010) [2023-12-27 03:38:33,245][105620] Updated weights for policy 1, policy_version 1684820 (0.0006) [2023-12-27 03:38:33,301][105620] Updated weights for policy 1, policy_version 1684830 (0.0008) [2023-12-27 03:38:33,320][105692] Updated weights for policy 0, policy_version 1681192 (0.0008) [2023-12-27 03:38:33,352][105620] Updated weights for policy 1, policy_version 1684840 (0.0008) [2023-12-27 03:38:33,365][105692] Updated weights for policy 0, policy_version 1681202 (0.0009) [2023-12-27 03:38:33,413][105692] Updated weights for policy 0, policy_version 1681212 (0.0005) [2023-12-27 03:38:34,114][105692] Updated weights for policy 0, policy_version 1681222 (0.0008) [2023-12-27 03:38:34,115][105620] Updated weights for policy 1, policy_version 1684850 (0.0009) [2023-12-27 03:38:34,175][105620] Updated weights for policy 1, policy_version 1684860 (0.0008) [2023-12-27 03:38:34,177][105692] Updated weights for policy 0, policy_version 1681232 (0.0008) [2023-12-27 03:38:34,234][105620] Updated weights for policy 1, policy_version 1684870 (0.0008) [2023-12-27 03:38:34,236][105692] Updated weights for policy 0, policy_version 1681242 (0.0008) [2023-12-27 03:38:34,292][105620] Updated weights for policy 1, policy_version 1684880 (0.0008) [2023-12-27 03:38:35,000][105692] Updated weights for policy 0, policy_version 1681252 (0.0006) [2023-12-27 03:38:35,030][105620] Updated weights for policy 1, policy_version 1684890 (0.0007) [2023-12-27 03:38:35,059][105692] Updated weights for policy 0, policy_version 1681262 (0.0007) [2023-12-27 03:38:35,089][105620] Updated weights for policy 1, policy_version 1684900 (0.0008) [2023-12-27 03:38:35,115][105692] Updated weights for policy 0, policy_version 1681272 (0.0007) [2023-12-27 03:38:35,148][105620] Updated weights for policy 1, policy_version 1684910 (0.0007) [2023-12-27 03:38:35,787][105692] Updated weights for policy 0, policy_version 1681282 (0.0006) [2023-12-27 03:38:35,841][105692] Updated weights for policy 0, policy_version 1681292 (0.0005) [2023-12-27 03:38:35,901][105692] Updated weights for policy 0, policy_version 1681302 (0.0007) [2023-12-27 03:38:35,936][105620] Updated weights for policy 1, policy_version 1684920 (0.0008) [2023-12-27 03:38:35,955][105692] Updated weights for policy 0, policy_version 1681312 (0.0006) [2023-12-27 03:38:35,999][105620] Updated weights for policy 1, policy_version 1684930 (0.0008) [2023-12-27 03:38:36,061][105620] Updated weights for policy 1, policy_version 1684940 (0.0009) [2023-12-27 03:38:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 861880320. Throughput: 0: 9761.9, 1: 9516.1. Samples: 861871020. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:38:36,062][104569] Avg episode reward: [(0, '8625.455'), (1, '8989.979')] [2023-12-27 03:38:36,658][105692] Updated weights for policy 0, policy_version 1681322 (0.0006) [2023-12-27 03:38:36,722][105692] Updated weights for policy 0, policy_version 1681332 (0.0007) [2023-12-27 03:38:36,774][105692] Updated weights for policy 0, policy_version 1681342 (0.0006) [2023-12-27 03:38:36,827][105620] Updated weights for policy 1, policy_version 1684950 (0.0010) [2023-12-27 03:38:36,875][105620] Updated weights for policy 1, policy_version 1684960 (0.0010) [2023-12-27 03:38:36,927][105620] Updated weights for policy 1, policy_version 1684970 (0.0010) [2023-12-27 03:38:37,413][105692] Updated weights for policy 0, policy_version 1681352 (0.0007) [2023-12-27 03:38:37,465][105692] Updated weights for policy 0, policy_version 1681362 (0.0008) [2023-12-27 03:38:37,518][105692] Updated weights for policy 0, policy_version 1681372 (0.0007) [2023-12-27 03:38:37,705][105620] Updated weights for policy 1, policy_version 1684980 (0.0011) [2023-12-27 03:38:37,765][105620] Updated weights for policy 1, policy_version 1684990 (0.0010) [2023-12-27 03:38:37,831][105620] Updated weights for policy 1, policy_version 1685000 (0.0010) [2023-12-27 03:38:38,233][105692] Updated weights for policy 0, policy_version 1681382 (0.0008) [2023-12-27 03:38:38,293][105692] Updated weights for policy 0, policy_version 1681392 (0.0010) [2023-12-27 03:38:38,361][105692] Updated weights for policy 0, policy_version 1681402 (0.0009) [2023-12-27 03:38:38,501][105620] Updated weights for policy 1, policy_version 1685010 (0.0010) [2023-12-27 03:38:38,574][105620] Updated weights for policy 1, policy_version 1685020 (0.0011) [2023-12-27 03:38:38,643][105620] Updated weights for policy 1, policy_version 1685030 (0.0010) [2023-12-27 03:38:38,699][105620] Updated weights for policy 1, policy_version 1685040 (0.0006) [2023-12-27 03:38:39,147][105692] Updated weights for policy 0, policy_version 1681412 (0.0008) [2023-12-27 03:38:39,201][105692] Updated weights for policy 0, policy_version 1681422 (0.0008) [2023-12-27 03:38:39,267][105692] Updated weights for policy 0, policy_version 1681432 (0.0008) [2023-12-27 03:38:39,378][105620] Updated weights for policy 1, policy_version 1685050 (0.0011) [2023-12-27 03:38:39,450][105620] Updated weights for policy 1, policy_version 1685061 (0.0010) [2023-12-27 03:38:39,513][105620] Updated weights for policy 1, policy_version 1685071 (0.0009) [2023-12-27 03:38:40,071][105692] Updated weights for policy 0, policy_version 1681442 (0.0008) [2023-12-27 03:38:40,145][105692] Updated weights for policy 0, policy_version 1681452 (0.0006) [2023-12-27 03:38:40,202][105692] Updated weights for policy 0, policy_version 1681462 (0.0008) [2023-12-27 03:38:40,258][105692] Updated weights for policy 0, policy_version 1681472 (0.0008) [2023-12-27 03:38:40,270][105620] Updated weights for policy 1, policy_version 1685081 (0.0007) [2023-12-27 03:38:40,338][105620] Updated weights for policy 1, policy_version 1685091 (0.0007) [2023-12-27 03:38:40,404][105620] Updated weights for policy 1, policy_version 1685101 (0.0007) [2023-12-27 03:38:41,023][105692] Updated weights for policy 0, policy_version 1681482 (0.0006) [2023-12-27 03:38:41,054][105620] Updated weights for policy 1, policy_version 1685111 (0.0008) [2023-12-27 03:38:41,062][104569] Fps is (10 sec: 18021.5, 60 sec: 19251.0, 300 sec: 19521.9). Total num frames: 861970432. Throughput: 0: 9834.0, 1: 9420.0. Samples: 861986272. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:38:41,063][104569] Avg episode reward: [(0, '8714.579'), (1, '8992.360')] [2023-12-27 03:38:41,092][105692] Updated weights for policy 0, policy_version 1681492 (0.0008) [2023-12-27 03:38:41,120][105620] Updated weights for policy 1, policy_version 1685121 (0.0008) [2023-12-27 03:38:41,153][105692] Updated weights for policy 0, policy_version 1681502 (0.0008) [2023-12-27 03:38:41,185][105620] Updated weights for policy 1, policy_version 1685131 (0.0008) [2023-12-27 03:38:41,945][105692] Updated weights for policy 0, policy_version 1681512 (0.0009) [2023-12-27 03:38:42,002][105692] Updated weights for policy 0, policy_version 1681522 (0.0008) [2023-12-27 03:38:42,004][105620] Updated weights for policy 1, policy_version 1685141 (0.0009) [2023-12-27 03:38:42,058][105620] Updated weights for policy 1, policy_version 1685151 (0.0007) [2023-12-27 03:38:42,060][105692] Updated weights for policy 0, policy_version 1681532 (0.0006) [2023-12-27 03:38:42,106][105620] Updated weights for policy 1, policy_version 1685161 (0.0007) [2023-12-27 03:38:42,731][105692] Updated weights for policy 0, policy_version 1681542 (0.0008) [2023-12-27 03:38:42,794][105692] Updated weights for policy 0, policy_version 1681552 (0.0008) [2023-12-27 03:38:42,850][105692] Updated weights for policy 0, policy_version 1681562 (0.0008) [2023-12-27 03:38:42,897][105620] Updated weights for policy 1, policy_version 1685171 (0.0010) [2023-12-27 03:38:42,960][105620] Updated weights for policy 1, policy_version 1685181 (0.0011) [2023-12-27 03:38:43,015][105620] Updated weights for policy 1, policy_version 1685191 (0.0010) [2023-12-27 03:38:43,577][105692] Updated weights for policy 0, policy_version 1681572 (0.0009) [2023-12-27 03:38:43,628][105692] Updated weights for policy 0, policy_version 1681582 (0.0009) [2023-12-27 03:38:43,681][105620] Updated weights for policy 1, policy_version 1685201 (0.0010) [2023-12-27 03:38:43,685][105692] Updated weights for policy 0, policy_version 1681592 (0.0009) [2023-12-27 03:38:43,730][105620] Updated weights for policy 1, policy_version 1685211 (0.0007) [2023-12-27 03:38:43,789][105620] Updated weights for policy 1, policy_version 1685221 (0.0009) [2023-12-27 03:38:43,849][105620] Updated weights for policy 1, policy_version 1685231 (0.0009) [2023-12-27 03:38:44,404][105692] Updated weights for policy 0, policy_version 1681602 (0.0008) [2023-12-27 03:38:44,465][105692] Updated weights for policy 0, policy_version 1681612 (0.0009) [2023-12-27 03:38:44,521][105692] Updated weights for policy 0, policy_version 1681622 (0.0009) [2023-12-27 03:38:44,570][105620] Updated weights for policy 1, policy_version 1685241 (0.0008) [2023-12-27 03:38:44,583][105692] Updated weights for policy 0, policy_version 1681632 (0.0009) [2023-12-27 03:38:44,628][105620] Updated weights for policy 1, policy_version 1685251 (0.0008) [2023-12-27 03:38:44,690][105620] Updated weights for policy 1, policy_version 1685261 (0.0009) [2023-12-27 03:38:45,361][105620] Updated weights for policy 1, policy_version 1685271 (0.0008) [2023-12-27 03:38:45,395][105692] Updated weights for policy 0, policy_version 1681642 (0.0006) [2023-12-27 03:38:45,414][105620] Updated weights for policy 1, policy_version 1685281 (0.0007) [2023-12-27 03:38:45,460][105692] Updated weights for policy 0, policy_version 1681652 (0.0008) [2023-12-27 03:38:45,469][105620] Updated weights for policy 1, policy_version 1685291 (0.0006) [2023-12-27 03:38:45,523][105692] Updated weights for policy 0, policy_version 1681662 (0.0008) [2023-12-27 03:38:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.8, 300 sec: 19522.0). Total num frames: 862068736. Throughput: 0: 9797.7, 1: 9394.6. Samples: 862041616. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:38:46,062][104569] Avg episode reward: [(0, '8621.905'), (1, '8994.014')] [2023-12-27 03:38:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001681664_430571520.pth... [2023-12-27 03:38:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001680544_430284800.pth [2023-12-27 03:38:46,087][105620] Updated weights for policy 1, policy_version 1685301 (0.0008) [2023-12-27 03:38:46,134][105620] Updated weights for policy 1, policy_version 1685311 (0.0009) [2023-12-27 03:38:46,179][105620] Updated weights for policy 1, policy_version 1685321 (0.0006) [2023-12-27 03:38:46,209][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001685328_431505408.pth... [2023-12-27 03:38:46,212][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001684176_431210496.pth [2023-12-27 03:38:46,321][105692] Updated weights for policy 0, policy_version 1681672 (0.0006) [2023-12-27 03:38:46,385][105692] Updated weights for policy 0, policy_version 1681682 (0.0006) [2023-12-27 03:38:46,443][105692] Updated weights for policy 0, policy_version 1681692 (0.0010) [2023-12-27 03:38:46,819][105620] Updated weights for policy 1, policy_version 1685331 (0.0007) [2023-12-27 03:38:46,879][105620] Updated weights for policy 1, policy_version 1685341 (0.0006) [2023-12-27 03:38:46,935][105620] Updated weights for policy 1, policy_version 1685351 (0.0006) [2023-12-27 03:38:47,074][105692] Updated weights for policy 0, policy_version 1681702 (0.0009) [2023-12-27 03:38:47,137][105692] Updated weights for policy 0, policy_version 1681712 (0.0008) [2023-12-27 03:38:47,188][105692] Updated weights for policy 0, policy_version 1681722 (0.0009) [2023-12-27 03:38:47,621][105620] Updated weights for policy 1, policy_version 1685361 (0.0007) [2023-12-27 03:38:47,687][105620] Updated weights for policy 1, policy_version 1685371 (0.0006) [2023-12-27 03:38:47,743][105620] Updated weights for policy 1, policy_version 1685381 (0.0006) [2023-12-27 03:38:47,798][105620] Updated weights for policy 1, policy_version 1685391 (0.0006) [2023-12-27 03:38:47,844][105692] Updated weights for policy 0, policy_version 1681732 (0.0007) [2023-12-27 03:38:47,904][105692] Updated weights for policy 0, policy_version 1681742 (0.0007) [2023-12-27 03:38:47,966][105692] Updated weights for policy 0, policy_version 1681752 (0.0010) [2023-12-27 03:38:48,461][105620] Updated weights for policy 1, policy_version 1685401 (0.0007) [2023-12-27 03:38:48,519][105620] Updated weights for policy 1, policy_version 1685411 (0.0005) [2023-12-27 03:38:48,573][105620] Updated weights for policy 1, policy_version 1685421 (0.0006) [2023-12-27 03:38:48,579][105692] Updated weights for policy 0, policy_version 1681763 (0.0008) [2023-12-27 03:38:48,633][105692] Updated weights for policy 0, policy_version 1681773 (0.0005) [2023-12-27 03:38:48,695][105692] Updated weights for policy 0, policy_version 1681783 (0.0005) [2023-12-27 03:38:49,264][105692] Updated weights for policy 0, policy_version 1681793 (0.0006) [2023-12-27 03:38:49,326][105692] Updated weights for policy 0, policy_version 1681803 (0.0007) [2023-12-27 03:38:49,336][105620] Updated weights for policy 1, policy_version 1685431 (0.0007) [2023-12-27 03:38:49,389][105692] Updated weights for policy 0, policy_version 1681813 (0.0007) [2023-12-27 03:38:49,401][105620] Updated weights for policy 1, policy_version 1685441 (0.0008) [2023-12-27 03:38:49,448][105692] Updated weights for policy 0, policy_version 1681823 (0.0005) [2023-12-27 03:38:49,458][105620] Updated weights for policy 1, policy_version 1685451 (0.0008) [2023-12-27 03:38:50,094][105692] Updated weights for policy 0, policy_version 1681833 (0.0010) [2023-12-27 03:38:50,143][105692] Updated weights for policy 0, policy_version 1681843 (0.0010) [2023-12-27 03:38:50,195][105692] Updated weights for policy 0, policy_version 1681853 (0.0010) [2023-12-27 03:38:50,220][105620] Updated weights for policy 1, policy_version 1685461 (0.0009) [2023-12-27 03:38:50,280][105620] Updated weights for policy 1, policy_version 1685471 (0.0008) [2023-12-27 03:38:50,334][105620] Updated weights for policy 1, policy_version 1685481 (0.0010) [2023-12-27 03:38:50,872][105692] Updated weights for policy 0, policy_version 1681863 (0.0009) [2023-12-27 03:38:50,936][105692] Updated weights for policy 0, policy_version 1681873 (0.0008) [2023-12-27 03:38:50,999][105692] Updated weights for policy 0, policy_version 1681883 (0.0009) [2023-12-27 03:38:51,062][104569] Fps is (10 sec: 20481.3, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 862175232. Throughput: 0: 9781.8, 1: 9504.8. Samples: 862162676. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:38:51,063][104569] Avg episode reward: [(0, '8897.456'), (1, '9081.267')] [2023-12-27 03:38:51,166][105620] Updated weights for policy 1, policy_version 1685491 (0.0010) [2023-12-27 03:38:51,220][105620] Updated weights for policy 1, policy_version 1685501 (0.0008) [2023-12-27 03:38:51,289][105620] Updated weights for policy 1, policy_version 1685511 (0.0009) [2023-12-27 03:38:51,677][105692] Updated weights for policy 0, policy_version 1681893 (0.0009) [2023-12-27 03:38:51,734][105692] Updated weights for policy 0, policy_version 1681903 (0.0009) [2023-12-27 03:38:51,794][105692] Updated weights for policy 0, policy_version 1681913 (0.0009) [2023-12-27 03:38:52,012][105620] Updated weights for policy 1, policy_version 1685521 (0.0009) [2023-12-27 03:38:52,066][105620] Updated weights for policy 1, policy_version 1685531 (0.0008) [2023-12-27 03:38:52,121][105620] Updated weights for policy 1, policy_version 1685541 (0.0009) [2023-12-27 03:38:52,181][105620] Updated weights for policy 1, policy_version 1685551 (0.0009) [2023-12-27 03:38:52,561][105692] Updated weights for policy 0, policy_version 1681923 (0.0009) [2023-12-27 03:38:52,620][105692] Updated weights for policy 0, policy_version 1681933 (0.0008) [2023-12-27 03:38:52,678][105692] Updated weights for policy 0, policy_version 1681943 (0.0010) [2023-12-27 03:38:52,964][105620] Updated weights for policy 1, policy_version 1685561 (0.0010) [2023-12-27 03:38:53,016][105620] Updated weights for policy 1, policy_version 1685571 (0.0010) [2023-12-27 03:38:53,075][105620] Updated weights for policy 1, policy_version 1685581 (0.0010) [2023-12-27 03:38:53,409][105692] Updated weights for policy 0, policy_version 1681953 (0.0010) [2023-12-27 03:38:53,471][105692] Updated weights for policy 0, policy_version 1681963 (0.0010) [2023-12-27 03:38:53,521][105692] Updated weights for policy 0, policy_version 1681973 (0.0010) [2023-12-27 03:38:53,573][105692] Updated weights for policy 0, policy_version 1681983 (0.0010) [2023-12-27 03:38:53,817][105620] Updated weights for policy 1, policy_version 1685591 (0.0007) [2023-12-27 03:38:53,871][105620] Updated weights for policy 1, policy_version 1685601 (0.0007) [2023-12-27 03:38:53,916][105620] Updated weights for policy 1, policy_version 1685611 (0.0006) [2023-12-27 03:38:54,296][105692] Updated weights for policy 0, policy_version 1681993 (0.0007) [2023-12-27 03:38:54,354][105692] Updated weights for policy 0, policy_version 1682003 (0.0005) [2023-12-27 03:38:54,401][105692] Updated weights for policy 0, policy_version 1682013 (0.0005) [2023-12-27 03:38:54,692][105620] Updated weights for policy 1, policy_version 1685621 (0.0007) [2023-12-27 03:38:54,743][105620] Updated weights for policy 1, policy_version 1685631 (0.0008) [2023-12-27 03:38:54,804][105620] Updated weights for policy 1, policy_version 1685641 (0.0008) [2023-12-27 03:38:55,072][105692] Updated weights for policy 0, policy_version 1682023 (0.0009) [2023-12-27 03:38:55,128][105692] Updated weights for policy 0, policy_version 1682033 (0.0010) [2023-12-27 03:38:55,187][105692] Updated weights for policy 0, policy_version 1682043 (0.0010) [2023-12-27 03:38:55,499][105620] Updated weights for policy 1, policy_version 1685651 (0.0008) [2023-12-27 03:38:55,558][105620] Updated weights for policy 1, policy_version 1685661 (0.0009) [2023-12-27 03:38:55,615][105620] Updated weights for policy 1, policy_version 1685671 (0.0007) [2023-12-27 03:38:55,937][105692] Updated weights for policy 0, policy_version 1682053 (0.0011) [2023-12-27 03:38:55,999][105692] Updated weights for policy 0, policy_version 1682063 (0.0010) [2023-12-27 03:38:56,057][105692] Updated weights for policy 0, policy_version 1682073 (0.0010) [2023-12-27 03:38:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 862265344. Throughput: 0: 9742.8, 1: 9585.7. Samples: 862278180. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:38:56,062][104569] Avg episode reward: [(0, '9077.226'), (1, '9090.159')] [2023-12-27 03:38:56,349][105620] Updated weights for policy 1, policy_version 1685681 (0.0008) [2023-12-27 03:38:56,396][105620] Updated weights for policy 1, policy_version 1685691 (0.0010) [2023-12-27 03:38:56,444][105620] Updated weights for policy 1, policy_version 1685701 (0.0010) [2023-12-27 03:38:56,502][105620] Updated weights for policy 1, policy_version 1685711 (0.0010) [2023-12-27 03:38:56,718][105692] Updated weights for policy 0, policy_version 1682083 (0.0011) [2023-12-27 03:38:56,766][105692] Updated weights for policy 0, policy_version 1682093 (0.0010) [2023-12-27 03:38:56,814][105692] Updated weights for policy 0, policy_version 1682103 (0.0010) [2023-12-27 03:38:57,246][105620] Updated weights for policy 1, policy_version 1685721 (0.0010) [2023-12-27 03:38:57,306][105620] Updated weights for policy 1, policy_version 1685731 (0.0011) [2023-12-27 03:38:57,359][105620] Updated weights for policy 1, policy_version 1685741 (0.0010) [2023-12-27 03:38:57,364][105692] Updated weights for policy 0, policy_version 1682113 (0.0005) [2023-12-27 03:38:57,421][105692] Updated weights for policy 0, policy_version 1682123 (0.0005) [2023-12-27 03:38:57,483][105692] Updated weights for policy 0, policy_version 1682133 (0.0005) [2023-12-27 03:38:57,534][105692] Updated weights for policy 0, policy_version 1682143 (0.0005) [2023-12-27 03:38:57,930][105620] Updated weights for policy 1, policy_version 1685751 (0.0007) [2023-12-27 03:38:57,981][105620] Updated weights for policy 1, policy_version 1685761 (0.0005) [2023-12-27 03:38:58,038][105620] Updated weights for policy 1, policy_version 1685771 (0.0005) [2023-12-27 03:38:58,197][105692] Updated weights for policy 0, policy_version 1682153 (0.0010) [2023-12-27 03:38:58,249][105692] Updated weights for policy 0, policy_version 1682163 (0.0011) [2023-12-27 03:38:58,312][105692] Updated weights for policy 0, policy_version 1682173 (0.0011) [2023-12-27 03:38:58,781][105620] Updated weights for policy 1, policy_version 1685781 (0.0007) [2023-12-27 03:38:58,847][105620] Updated weights for policy 1, policy_version 1685791 (0.0009) [2023-12-27 03:38:58,913][105620] Updated weights for policy 1, policy_version 1685801 (0.0007) [2023-12-27 03:38:59,169][105692] Updated weights for policy 0, policy_version 1682183 (0.0010) [2023-12-27 03:38:59,237][105692] Updated weights for policy 0, policy_version 1682193 (0.0008) [2023-12-27 03:38:59,296][105692] Updated weights for policy 0, policy_version 1682203 (0.0010) [2023-12-27 03:38:59,655][105620] Updated weights for policy 1, policy_version 1685811 (0.0009) [2023-12-27 03:38:59,707][105620] Updated weights for policy 1, policy_version 1685821 (0.0008) [2023-12-27 03:38:59,756][105620] Updated weights for policy 1, policy_version 1685831 (0.0009) [2023-12-27 03:39:00,033][105692] Updated weights for policy 0, policy_version 1682213 (0.0009) [2023-12-27 03:39:00,077][105692] Updated weights for policy 0, policy_version 1682223 (0.0008) [2023-12-27 03:39:00,131][105692] Updated weights for policy 0, policy_version 1682233 (0.0008) [2023-12-27 03:39:00,481][105620] Updated weights for policy 1, policy_version 1685841 (0.0007) [2023-12-27 03:39:00,529][105620] Updated weights for policy 1, policy_version 1685851 (0.0005) [2023-12-27 03:39:00,588][105620] Updated weights for policy 1, policy_version 1685861 (0.0010) [2023-12-27 03:39:00,649][105620] Updated weights for policy 1, policy_version 1685871 (0.0010) [2023-12-27 03:39:00,917][105692] Updated weights for policy 0, policy_version 1682243 (0.0009) [2023-12-27 03:39:00,961][105692] Updated weights for policy 0, policy_version 1682253 (0.0008) [2023-12-27 03:39:01,010][105692] Updated weights for policy 0, policy_version 1682263 (0.0009) [2023-12-27 03:39:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 862371840. Throughput: 0: 9826.3, 1: 9636.1. Samples: 862339420. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-12-27 03:39:01,062][104569] Avg episode reward: [(0, '8804.371'), (1, '8816.966')] [2023-12-27 03:39:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001682272_430727168.pth... [2023-12-27 03:39:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001685872_431644672.pth... [2023-12-27 03:39:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001681120_430432256.pth [2023-12-27 03:39:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001684752_431357952.pth [2023-12-27 03:39:01,285][105620] Updated weights for policy 1, policy_version 1685881 (0.0011) [2023-12-27 03:39:01,353][105620] Updated weights for policy 1, policy_version 1685891 (0.0011) [2023-12-27 03:39:01,422][105620] Updated weights for policy 1, policy_version 1685901 (0.0008) [2023-12-27 03:39:01,824][105692] Updated weights for policy 0, policy_version 1682273 (0.0011) [2023-12-27 03:39:01,888][105692] Updated weights for policy 0, policy_version 1682283 (0.0010) [2023-12-27 03:39:01,947][105692] Updated weights for policy 0, policy_version 1682293 (0.0008) [2023-12-27 03:39:02,005][105692] Updated weights for policy 0, policy_version 1682303 (0.0005) [2023-12-27 03:39:02,043][105620] Updated weights for policy 1, policy_version 1685911 (0.0008) [2023-12-27 03:39:02,101][105620] Updated weights for policy 1, policy_version 1685921 (0.0010) [2023-12-27 03:39:02,156][105620] Updated weights for policy 1, policy_version 1685931 (0.0009) [2023-12-27 03:39:02,660][105692] Updated weights for policy 0, policy_version 1682313 (0.0009) [2023-12-27 03:39:02,708][105692] Updated weights for policy 0, policy_version 1682323 (0.0009) [2023-12-27 03:39:02,758][105692] Updated weights for policy 0, policy_version 1682333 (0.0009) [2023-12-27 03:39:02,970][105620] Updated weights for policy 1, policy_version 1685941 (0.0009) [2023-12-27 03:39:03,027][105620] Updated weights for policy 1, policy_version 1685951 (0.0008) [2023-12-27 03:39:03,073][105620] Updated weights for policy 1, policy_version 1685961 (0.0009) [2023-12-27 03:39:03,453][105692] Updated weights for policy 0, policy_version 1682343 (0.0007) [2023-12-27 03:39:03,503][105692] Updated weights for policy 0, policy_version 1682353 (0.0005) [2023-12-27 03:39:03,555][105692] Updated weights for policy 0, policy_version 1682363 (0.0005) [2023-12-27 03:39:03,906][105620] Updated weights for policy 1, policy_version 1685971 (0.0009) [2023-12-27 03:39:03,966][105620] Updated weights for policy 1, policy_version 1685981 (0.0009) [2023-12-27 03:39:04,023][105620] Updated weights for policy 1, policy_version 1685991 (0.0009) [2023-12-27 03:39:04,181][105692] Updated weights for policy 0, policy_version 1682373 (0.0007) [2023-12-27 03:39:04,248][105692] Updated weights for policy 0, policy_version 1682383 (0.0009) [2023-12-27 03:39:04,310][105692] Updated weights for policy 0, policy_version 1682393 (0.0009) [2023-12-27 03:39:04,868][105620] Updated weights for policy 1, policy_version 1686001 (0.0009) [2023-12-27 03:39:04,934][105620] Updated weights for policy 1, policy_version 1686011 (0.0009) [2023-12-27 03:39:04,959][105692] Updated weights for policy 0, policy_version 1682403 (0.0008) [2023-12-27 03:39:04,992][105620] Updated weights for policy 1, policy_version 1686021 (0.0007) [2023-12-27 03:39:05,022][105692] Updated weights for policy 0, policy_version 1682413 (0.0006) [2023-12-27 03:39:05,057][105620] Updated weights for policy 1, policy_version 1686031 (0.0006) [2023-12-27 03:39:05,077][105692] Updated weights for policy 0, policy_version 1682423 (0.0006) [2023-12-27 03:39:05,651][105692] Updated weights for policy 0, policy_version 1682433 (0.0006) [2023-12-27 03:39:05,698][105692] Updated weights for policy 0, policy_version 1682443 (0.0005) [2023-12-27 03:39:05,751][105692] Updated weights for policy 0, policy_version 1682453 (0.0008) [2023-12-27 03:39:05,797][105692] Updated weights for policy 0, policy_version 1682463 (0.0009) [2023-12-27 03:39:05,859][105620] Updated weights for policy 1, policy_version 1686042 (0.0010) [2023-12-27 03:39:05,916][105620] Updated weights for policy 1, policy_version 1686052 (0.0010) [2023-12-27 03:39:05,964][105620] Updated weights for policy 1, policy_version 1686062 (0.0008) [2023-12-27 03:39:06,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 862470144. Throughput: 0: 9790.9, 1: 9670.1. Samples: 862453944. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:39:06,062][104569] Avg episode reward: [(0, '8715.990'), (1, '8990.973')] [2023-12-27 03:39:06,478][105692] Updated weights for policy 0, policy_version 1682473 (0.0005) [2023-12-27 03:39:06,545][105692] Updated weights for policy 0, policy_version 1682483 (0.0006) [2023-12-27 03:39:06,603][105692] Updated weights for policy 0, policy_version 1682493 (0.0007) [2023-12-27 03:39:06,789][105620] Updated weights for policy 1, policy_version 1686072 (0.0008) [2023-12-27 03:39:06,847][105620] Updated weights for policy 1, policy_version 1686082 (0.0008) [2023-12-27 03:39:06,912][105620] Updated weights for policy 1, policy_version 1686092 (0.0010) [2023-12-27 03:39:07,196][105692] Updated weights for policy 0, policy_version 1682503 (0.0007) [2023-12-27 03:39:07,242][105692] Updated weights for policy 0, policy_version 1682513 (0.0005) [2023-12-27 03:39:07,294][105692] Updated weights for policy 0, policy_version 1682523 (0.0005) [2023-12-27 03:39:07,822][105692] Updated weights for policy 0, policy_version 1682533 (0.0005) [2023-12-27 03:39:07,824][105620] Updated weights for policy 1, policy_version 1686102 (0.0009) [2023-12-27 03:39:07,872][105620] Updated weights for policy 1, policy_version 1686112 (0.0009) [2023-12-27 03:39:07,873][105692] Updated weights for policy 0, policy_version 1682543 (0.0005) [2023-12-27 03:39:07,927][105620] Updated weights for policy 1, policy_version 1686122 (0.0006) [2023-12-27 03:39:07,935][105692] Updated weights for policy 0, policy_version 1682553 (0.0005) [2023-12-27 03:39:08,584][105692] Updated weights for policy 0, policy_version 1682563 (0.0007) [2023-12-27 03:39:08,586][105620] Updated weights for policy 1, policy_version 1686132 (0.0007) [2023-12-27 03:39:08,632][105620] Updated weights for policy 1, policy_version 1686142 (0.0005) [2023-12-27 03:39:08,636][105692] Updated weights for policy 0, policy_version 1682573 (0.0010) [2023-12-27 03:39:08,682][105620] Updated weights for policy 1, policy_version 1686152 (0.0006) [2023-12-27 03:39:08,691][105692] Updated weights for policy 0, policy_version 1682583 (0.0010) [2023-12-27 03:39:09,282][105620] Updated weights for policy 1, policy_version 1686162 (0.0006) [2023-12-27 03:39:09,334][105620] Updated weights for policy 1, policy_version 1686172 (0.0008) [2023-12-27 03:39:09,406][105620] Updated weights for policy 1, policy_version 1686182 (0.0009) [2023-12-27 03:39:09,464][105692] Updated weights for policy 0, policy_version 1682593 (0.0010) [2023-12-27 03:39:09,466][105620] Updated weights for policy 1, policy_version 1686192 (0.0008) [2023-12-27 03:39:09,527][105692] Updated weights for policy 0, policy_version 1682603 (0.0008) [2023-12-27 03:39:09,590][105692] Updated weights for policy 0, policy_version 1682613 (0.0008) [2023-12-27 03:39:09,637][105692] Updated weights for policy 0, policy_version 1682623 (0.0010) [2023-12-27 03:39:10,231][105620] Updated weights for policy 1, policy_version 1686202 (0.0007) [2023-12-27 03:39:10,288][105620] Updated weights for policy 1, policy_version 1686212 (0.0008) [2023-12-27 03:39:10,355][105620] Updated weights for policy 1, policy_version 1686222 (0.0008) [2023-12-27 03:39:10,387][105692] Updated weights for policy 0, policy_version 1682633 (0.0011) [2023-12-27 03:39:10,445][105692] Updated weights for policy 0, policy_version 1682643 (0.0011) [2023-12-27 03:39:10,498][105692] Updated weights for policy 0, policy_version 1682653 (0.0011) [2023-12-27 03:39:11,062][105620] Updated weights for policy 1, policy_version 1686232 (0.0008) [2023-12-27 03:39:11,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 862560256. Throughput: 0: 9935.2, 1: 9671.8. Samples: 862573480. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:39:11,063][104569] Avg episode reward: [(0, '8989.985'), (1, '9079.086')] [2023-12-27 03:39:11,128][105620] Updated weights for policy 1, policy_version 1686242 (0.0008) [2023-12-27 03:39:11,193][105620] Updated weights for policy 1, policy_version 1686252 (0.0007) [2023-12-27 03:39:11,262][105692] Updated weights for policy 0, policy_version 1682663 (0.0009) [2023-12-27 03:39:11,327][105692] Updated weights for policy 0, policy_version 1682673 (0.0009) [2023-12-27 03:39:11,393][105692] Updated weights for policy 0, policy_version 1682683 (0.0010) [2023-12-27 03:39:11,964][105620] Updated weights for policy 1, policy_version 1686262 (0.0008) [2023-12-27 03:39:12,031][105620] Updated weights for policy 1, policy_version 1686272 (0.0008) [2023-12-27 03:39:12,044][105692] Updated weights for policy 0, policy_version 1682693 (0.0008) [2023-12-27 03:39:12,099][105620] Updated weights for policy 1, policy_version 1686282 (0.0007) [2023-12-27 03:39:12,112][105692] Updated weights for policy 0, policy_version 1682703 (0.0010) [2023-12-27 03:39:12,182][105692] Updated weights for policy 0, policy_version 1682713 (0.0006) [2023-12-27 03:39:12,749][105620] Updated weights for policy 1, policy_version 1686292 (0.0005) [2023-12-27 03:39:12,810][105620] Updated weights for policy 1, policy_version 1686302 (0.0007) [2023-12-27 03:39:12,873][105620] Updated weights for policy 1, policy_version 1686312 (0.0007) [2023-12-27 03:39:12,873][105692] Updated weights for policy 0, policy_version 1682723 (0.0007) [2023-12-27 03:39:12,936][105692] Updated weights for policy 0, policy_version 1682733 (0.0011) [2023-12-27 03:39:13,001][105692] Updated weights for policy 0, policy_version 1682743 (0.0011) [2023-12-27 03:39:13,552][105620] Updated weights for policy 1, policy_version 1686322 (0.0006) [2023-12-27 03:39:13,614][105620] Updated weights for policy 1, policy_version 1686332 (0.0008) [2023-12-27 03:39:13,676][105620] Updated weights for policy 1, policy_version 1686342 (0.0008) [2023-12-27 03:39:13,687][105692] Updated weights for policy 0, policy_version 1682753 (0.0011) [2023-12-27 03:39:13,732][105620] Updated weights for policy 1, policy_version 1686352 (0.0005) [2023-12-27 03:39:13,749][105692] Updated weights for policy 0, policy_version 1682763 (0.0010) [2023-12-27 03:39:13,813][105692] Updated weights for policy 0, policy_version 1682773 (0.0010) [2023-12-27 03:39:13,874][105692] Updated weights for policy 0, policy_version 1682783 (0.0010) [2023-12-27 03:39:14,487][105692] Updated weights for policy 0, policy_version 1682793 (0.0007) [2023-12-27 03:39:14,547][105692] Updated weights for policy 0, policy_version 1682803 (0.0006) [2023-12-27 03:39:14,550][105620] Updated weights for policy 1, policy_version 1686362 (0.0006) [2023-12-27 03:39:14,608][105692] Updated weights for policy 0, policy_version 1682813 (0.0008) [2023-12-27 03:39:14,622][105620] Updated weights for policy 1, policy_version 1686372 (0.0006) [2023-12-27 03:39:14,685][105620] Updated weights for policy 1, policy_version 1686382 (0.0007) [2023-12-27 03:39:15,358][105620] Updated weights for policy 1, policy_version 1686392 (0.0009) [2023-12-27 03:39:15,394][105692] Updated weights for policy 0, policy_version 1682823 (0.0007) [2023-12-27 03:39:15,413][105620] Updated weights for policy 1, policy_version 1686402 (0.0007) [2023-12-27 03:39:15,448][105692] Updated weights for policy 0, policy_version 1682833 (0.0006) [2023-12-27 03:39:15,473][105620] Updated weights for policy 1, policy_version 1686412 (0.0007) [2023-12-27 03:39:15,495][105692] Updated weights for policy 0, policy_version 1682843 (0.0008) [2023-12-27 03:39:16,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 862658560. Throughput: 0: 9929.7, 1: 9599.1. Samples: 862631948. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:39:16,063][104569] Avg episode reward: [(0, '8986.446'), (1, '8817.858')] [2023-12-27 03:39:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001686416_431783936.pth... [2023-12-27 03:39:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001682848_430874624.pth... [2023-12-27 03:39:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001685328_431505408.pth [2023-12-27 03:39:16,080][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001681664_430571520.pth [2023-12-27 03:39:16,194][105620] Updated weights for policy 1, policy_version 1686422 (0.0008) [2023-12-27 03:39:16,214][105692] Updated weights for policy 0, policy_version 1682853 (0.0007) [2023-12-27 03:39:16,247][105620] Updated weights for policy 1, policy_version 1686432 (0.0009) [2023-12-27 03:39:16,272][105692] Updated weights for policy 0, policy_version 1682863 (0.0005) [2023-12-27 03:39:16,305][105620] Updated weights for policy 1, policy_version 1686442 (0.0007) [2023-12-27 03:39:16,333][105692] Updated weights for policy 0, policy_version 1682873 (0.0008) [2023-12-27 03:39:17,036][105692] Updated weights for policy 0, policy_version 1682883 (0.0007) [2023-12-27 03:39:17,046][105620] Updated weights for policy 1, policy_version 1686452 (0.0008) [2023-12-27 03:39:17,087][105692] Updated weights for policy 0, policy_version 1682893 (0.0005) [2023-12-27 03:39:17,097][105620] Updated weights for policy 1, policy_version 1686462 (0.0009) [2023-12-27 03:39:17,137][105692] Updated weights for policy 0, policy_version 1682903 (0.0006) [2023-12-27 03:39:17,143][105620] Updated weights for policy 1, policy_version 1686472 (0.0007) [2023-12-27 03:39:17,688][105692] Updated weights for policy 0, policy_version 1682913 (0.0005) [2023-12-27 03:39:17,750][105692] Updated weights for policy 0, policy_version 1682923 (0.0009) [2023-12-27 03:39:17,811][105692] Updated weights for policy 0, policy_version 1682933 (0.0010) [2023-12-27 03:39:17,860][105692] Updated weights for policy 0, policy_version 1682943 (0.0007) [2023-12-27 03:39:18,013][105620] Updated weights for policy 1, policy_version 1686482 (0.0007) [2023-12-27 03:39:18,067][105620] Updated weights for policy 1, policy_version 1686492 (0.0008) [2023-12-27 03:39:18,121][105620] Updated weights for policy 1, policy_version 1686502 (0.0009) [2023-12-27 03:39:18,473][105692] Updated weights for policy 0, policy_version 1682953 (0.0006) [2023-12-27 03:39:18,531][105692] Updated weights for policy 0, policy_version 1682963 (0.0005) [2023-12-27 03:39:18,592][105692] Updated weights for policy 0, policy_version 1682973 (0.0006) [2023-12-27 03:39:18,928][105620] Updated weights for policy 1, policy_version 1686513 (0.0010) [2023-12-27 03:39:18,992][105620] Updated weights for policy 1, policy_version 1686523 (0.0008) [2023-12-27 03:39:19,054][105620] Updated weights for policy 1, policy_version 1686533 (0.0009) [2023-12-27 03:39:19,111][105620] Updated weights for policy 1, policy_version 1686543 (0.0009) [2023-12-27 03:39:19,211][105692] Updated weights for policy 0, policy_version 1682983 (0.0007) [2023-12-27 03:39:19,288][105692] Updated weights for policy 0, policy_version 1682993 (0.0008) [2023-12-27 03:39:19,352][105692] Updated weights for policy 0, policy_version 1683003 (0.0010) [2023-12-27 03:39:19,849][105620] Updated weights for policy 1, policy_version 1686553 (0.0008) [2023-12-27 03:39:19,902][105620] Updated weights for policy 1, policy_version 1686564 (0.0010) [2023-12-27 03:39:19,970][105620] Updated weights for policy 1, policy_version 1686574 (0.0009) [2023-12-27 03:39:20,026][105692] Updated weights for policy 0, policy_version 1683013 (0.0007) [2023-12-27 03:39:20,083][105692] Updated weights for policy 0, policy_version 1683023 (0.0006) [2023-12-27 03:39:20,143][105692] Updated weights for policy 0, policy_version 1683033 (0.0006) [2023-12-27 03:39:20,720][105692] Updated weights for policy 0, policy_version 1683043 (0.0006) [2023-12-27 03:39:20,772][105692] Updated weights for policy 0, policy_version 1683053 (0.0005) [2023-12-27 03:39:20,832][105692] Updated weights for policy 0, policy_version 1683063 (0.0006) [2023-12-27 03:39:20,880][105620] Updated weights for policy 1, policy_version 1686584 (0.0007) [2023-12-27 03:39:20,939][105620] Updated weights for policy 1, policy_version 1686594 (0.0009) [2023-12-27 03:39:20,990][105620] Updated weights for policy 1, policy_version 1686604 (0.0009) [2023-12-27 03:39:21,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 862765056. Throughput: 0: 10014.3, 1: 9498.7. Samples: 862749104. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:39:21,062][104569] Avg episode reward: [(0, '9078.385'), (1, '8657.521')] [2023-12-27 03:39:21,489][105692] Updated weights for policy 0, policy_version 1683073 (0.0007) [2023-12-27 03:39:21,553][105692] Updated weights for policy 0, policy_version 1683083 (0.0005) [2023-12-27 03:39:21,618][105692] Updated weights for policy 0, policy_version 1683093 (0.0007) [2023-12-27 03:39:21,679][105692] Updated weights for policy 0, policy_version 1683103 (0.0009) [2023-12-27 03:39:21,830][105620] Updated weights for policy 1, policy_version 1686614 (0.0008) [2023-12-27 03:39:21,890][105620] Updated weights for policy 1, policy_version 1686624 (0.0009) [2023-12-27 03:39:21,952][105620] Updated weights for policy 1, policy_version 1686634 (0.0009) [2023-12-27 03:39:22,324][105692] Updated weights for policy 0, policy_version 1683113 (0.0009) [2023-12-27 03:39:22,390][105692] Updated weights for policy 0, policy_version 1683123 (0.0009) [2023-12-27 03:39:22,453][105692] Updated weights for policy 0, policy_version 1683133 (0.0009) [2023-12-27 03:39:22,746][105620] Updated weights for policy 1, policy_version 1686644 (0.0009) [2023-12-27 03:39:22,796][105620] Updated weights for policy 1, policy_version 1686654 (0.0008) [2023-12-27 03:39:22,846][105620] Updated weights for policy 1, policy_version 1686664 (0.0008) [2023-12-27 03:39:23,158][105692] Updated weights for policy 0, policy_version 1683143 (0.0009) [2023-12-27 03:39:23,231][105692] Updated weights for policy 0, policy_version 1683153 (0.0010) [2023-12-27 03:39:23,303][105692] Updated weights for policy 0, policy_version 1683163 (0.0010) [2023-12-27 03:39:23,480][105620] Updated weights for policy 1, policy_version 1686674 (0.0008) [2023-12-27 03:39:23,545][105620] Updated weights for policy 1, policy_version 1686684 (0.0007) [2023-12-27 03:39:23,607][105620] Updated weights for policy 1, policy_version 1686694 (0.0005) [2023-12-27 03:39:23,658][105620] Updated weights for policy 1, policy_version 1686704 (0.0005) [2023-12-27 03:39:23,921][105692] Updated weights for policy 0, policy_version 1683173 (0.0010) [2023-12-27 03:39:23,965][105692] Updated weights for policy 0, policy_version 1683183 (0.0010) [2023-12-27 03:39:24,023][105692] Updated weights for policy 0, policy_version 1683193 (0.0010) [2023-12-27 03:39:24,355][105620] Updated weights for policy 1, policy_version 1686714 (0.0010) [2023-12-27 03:39:24,408][105620] Updated weights for policy 1, policy_version 1686724 (0.0010) [2023-12-27 03:39:24,461][105620] Updated weights for policy 1, policy_version 1686734 (0.0009) [2023-12-27 03:39:24,610][105692] Updated weights for policy 0, policy_version 1683203 (0.0010) [2023-12-27 03:39:24,665][105692] Updated weights for policy 0, policy_version 1683213 (0.0010) [2023-12-27 03:39:24,723][105692] Updated weights for policy 0, policy_version 1683223 (0.0009) [2023-12-27 03:39:25,207][105620] Updated weights for policy 1, policy_version 1686744 (0.0008) [2023-12-27 03:39:25,271][105620] Updated weights for policy 1, policy_version 1686754 (0.0007) [2023-12-27 03:39:25,337][105620] Updated weights for policy 1, policy_version 1686764 (0.0005) [2023-12-27 03:39:25,407][105692] Updated weights for policy 0, policy_version 1683233 (0.0006) [2023-12-27 03:39:25,473][105692] Updated weights for policy 0, policy_version 1683243 (0.0010) [2023-12-27 03:39:25,524][105692] Updated weights for policy 0, policy_version 1683253 (0.0010) [2023-12-27 03:39:25,579][105692] Updated weights for policy 0, policy_version 1683263 (0.0010) [2023-12-27 03:39:25,954][105620] Updated weights for policy 1, policy_version 1686774 (0.0006) [2023-12-27 03:39:26,016][105620] Updated weights for policy 1, policy_version 1686784 (0.0009) [2023-12-27 03:39:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 862855168. Throughput: 0: 10118.1, 1: 9480.8. Samples: 862868212. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:39:26,063][104569] Avg episode reward: [(0, '8716.603'), (1, '8638.547')] [2023-12-27 03:39:26,081][105620] Updated weights for policy 1, policy_version 1686794 (0.0008) [2023-12-27 03:39:26,280][105692] Updated weights for policy 0, policy_version 1683273 (0.0009) [2023-12-27 03:39:26,352][105692] Updated weights for policy 0, policy_version 1683283 (0.0008) [2023-12-27 03:39:26,417][105692] Updated weights for policy 0, policy_version 1683293 (0.0005) [2023-12-27 03:39:26,886][105620] Updated weights for policy 1, policy_version 1686804 (0.0007) [2023-12-27 03:39:26,926][105692] Updated weights for policy 0, policy_version 1683303 (0.0005) [2023-12-27 03:39:26,939][105620] Updated weights for policy 1, policy_version 1686814 (0.0005) [2023-12-27 03:39:26,972][105692] Updated weights for policy 0, policy_version 1683313 (0.0005) [2023-12-27 03:39:26,989][105620] Updated weights for policy 1, policy_version 1686824 (0.0005) [2023-12-27 03:39:27,019][105692] Updated weights for policy 0, policy_version 1683323 (0.0005) [2023-12-27 03:39:27,528][105620] Updated weights for policy 1, policy_version 1686834 (0.0006) [2023-12-27 03:39:27,545][105692] Updated weights for policy 0, policy_version 1683333 (0.0005) [2023-12-27 03:39:27,593][105620] Updated weights for policy 1, policy_version 1686844 (0.0008) [2023-12-27 03:39:27,597][105692] Updated weights for policy 0, policy_version 1683343 (0.0009) [2023-12-27 03:39:27,652][105692] Updated weights for policy 0, policy_version 1683353 (0.0010) [2023-12-27 03:39:27,654][105620] Updated weights for policy 1, policy_version 1686854 (0.0006) [2023-12-27 03:39:27,716][105620] Updated weights for policy 1, policy_version 1686864 (0.0007) [2023-12-27 03:39:28,305][105692] Updated weights for policy 0, policy_version 1683363 (0.0010) [2023-12-27 03:39:28,364][105692] Updated weights for policy 0, policy_version 1683373 (0.0007) [2023-12-27 03:39:28,389][105620] Updated weights for policy 1, policy_version 1686874 (0.0008) [2023-12-27 03:39:28,425][105692] Updated weights for policy 0, policy_version 1683383 (0.0006) [2023-12-27 03:39:28,445][105620] Updated weights for policy 1, policy_version 1686884 (0.0008) [2023-12-27 03:39:28,500][105620] Updated weights for policy 1, policy_version 1686894 (0.0008) [2023-12-27 03:39:29,077][105620] Updated weights for policy 1, policy_version 1686904 (0.0008) [2023-12-27 03:39:29,102][105692] Updated weights for policy 0, policy_version 1683393 (0.0006) [2023-12-27 03:39:29,133][105620] Updated weights for policy 1, policy_version 1686914 (0.0008) [2023-12-27 03:39:29,153][105692] Updated weights for policy 0, policy_version 1683403 (0.0010) [2023-12-27 03:39:29,193][105620] Updated weights for policy 1, policy_version 1686924 (0.0007) [2023-12-27 03:39:29,221][105692] Updated weights for policy 0, policy_version 1683413 (0.0010) [2023-12-27 03:39:29,281][105692] Updated weights for policy 0, policy_version 1683423 (0.0010) [2023-12-27 03:39:29,802][105620] Updated weights for policy 1, policy_version 1686934 (0.0008) [2023-12-27 03:39:29,869][105620] Updated weights for policy 1, policy_version 1686944 (0.0007) [2023-12-27 03:39:29,939][105620] Updated weights for policy 1, policy_version 1686954 (0.0007) [2023-12-27 03:39:29,990][105692] Updated weights for policy 0, policy_version 1683433 (0.0008) [2023-12-27 03:39:30,047][105692] Updated weights for policy 0, policy_version 1683443 (0.0009) [2023-12-27 03:39:30,115][105692] Updated weights for policy 0, policy_version 1683453 (0.0008) [2023-12-27 03:39:30,657][105620] Updated weights for policy 1, policy_version 1686964 (0.0008) [2023-12-27 03:39:30,708][105620] Updated weights for policy 1, policy_version 1686974 (0.0008) [2023-12-27 03:39:30,761][105620] Updated weights for policy 1, policy_version 1686984 (0.0006) [2023-12-27 03:39:30,766][105692] Updated weights for policy 0, policy_version 1683463 (0.0009) [2023-12-27 03:39:30,820][105692] Updated weights for policy 0, policy_version 1683473 (0.0009) [2023-12-27 03:39:30,883][105692] Updated weights for policy 0, policy_version 1683483 (0.0005) [2023-12-27 03:39:31,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.8, 300 sec: 19605.2). Total num frames: 862969856. Throughput: 0: 10252.3, 1: 9570.4. Samples: 862933640. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:39:31,063][104569] Avg episode reward: [(0, '8809.244'), (1, '8822.847')] [2023-12-27 03:39:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001683488_431038464.pth... [2023-12-27 03:39:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001686992_431931392.pth... [2023-12-27 03:39:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001682272_430727168.pth [2023-12-27 03:39:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001685872_431644672.pth [2023-12-27 03:39:31,509][105692] Updated weights for policy 0, policy_version 1683493 (0.0007) [2023-12-27 03:39:31,560][105692] Updated weights for policy 0, policy_version 1683503 (0.0008) [2023-12-27 03:39:31,580][105620] Updated weights for policy 1, policy_version 1686994 (0.0007) [2023-12-27 03:39:31,617][105692] Updated weights for policy 0, policy_version 1683513 (0.0007) [2023-12-27 03:39:31,636][105620] Updated weights for policy 1, policy_version 1687004 (0.0009) [2023-12-27 03:39:31,685][105620] Updated weights for policy 1, policy_version 1687014 (0.0006) [2023-12-27 03:39:31,744][105620] Updated weights for policy 1, policy_version 1687024 (0.0007) [2023-12-27 03:39:32,279][105692] Updated weights for policy 0, policy_version 1683523 (0.0007) [2023-12-27 03:39:32,331][105692] Updated weights for policy 0, policy_version 1683533 (0.0007) [2023-12-27 03:39:32,399][105692] Updated weights for policy 0, policy_version 1683543 (0.0007) [2023-12-27 03:39:32,493][105620] Updated weights for policy 1, policy_version 1687034 (0.0006) [2023-12-27 03:39:32,547][105620] Updated weights for policy 1, policy_version 1687044 (0.0005) [2023-12-27 03:39:32,606][105620] Updated weights for policy 1, policy_version 1687054 (0.0008) [2023-12-27 03:39:33,127][105692] Updated weights for policy 0, policy_version 1683553 (0.0008) [2023-12-27 03:39:33,179][105692] Updated weights for policy 0, policy_version 1683563 (0.0009) [2023-12-27 03:39:33,228][105692] Updated weights for policy 0, policy_version 1683573 (0.0008) [2023-12-27 03:39:33,234][105620] Updated weights for policy 1, policy_version 1687064 (0.0006) [2023-12-27 03:39:33,280][105620] Updated weights for policy 1, policy_version 1687074 (0.0006) [2023-12-27 03:39:33,282][105692] Updated weights for policy 0, policy_version 1683583 (0.0008) [2023-12-27 03:39:33,325][105620] Updated weights for policy 1, policy_version 1687084 (0.0008) [2023-12-27 03:39:33,975][105692] Updated weights for policy 0, policy_version 1683593 (0.0005) [2023-12-27 03:39:34,026][105692] Updated weights for policy 0, policy_version 1683603 (0.0005) [2023-12-27 03:39:34,036][105620] Updated weights for policy 1, policy_version 1687094 (0.0010) [2023-12-27 03:39:34,074][105692] Updated weights for policy 0, policy_version 1683613 (0.0005) [2023-12-27 03:39:34,094][105620] Updated weights for policy 1, policy_version 1687104 (0.0010) [2023-12-27 03:39:34,153][105620] Updated weights for policy 1, policy_version 1687114 (0.0010) [2023-12-27 03:39:34,760][105692] Updated weights for policy 0, policy_version 1683623 (0.0007) [2023-12-27 03:39:34,823][105692] Updated weights for policy 0, policy_version 1683633 (0.0009) [2023-12-27 03:39:34,883][105692] Updated weights for policy 0, policy_version 1683643 (0.0009) [2023-12-27 03:39:34,898][105620] Updated weights for policy 1, policy_version 1687124 (0.0009) [2023-12-27 03:39:34,953][105620] Updated weights for policy 1, policy_version 1687134 (0.0008) [2023-12-27 03:39:35,021][105620] Updated weights for policy 1, policy_version 1687144 (0.0009) [2023-12-27 03:39:35,502][105692] Updated weights for policy 0, policy_version 1683653 (0.0006) [2023-12-27 03:39:35,555][105692] Updated weights for policy 0, policy_version 1683663 (0.0005) [2023-12-27 03:39:35,602][105692] Updated weights for policy 0, policy_version 1683673 (0.0006) [2023-12-27 03:39:35,825][105620] Updated weights for policy 1, policy_version 1687154 (0.0009) [2023-12-27 03:39:35,880][105620] Updated weights for policy 1, policy_version 1687164 (0.0009) [2023-12-27 03:39:35,934][105620] Updated weights for policy 1, policy_version 1687175 (0.0009) [2023-12-27 03:39:36,062][104569] Fps is (10 sec: 21298.5, 60 sec: 19797.2, 300 sec: 19605.2). Total num frames: 863068160. Throughput: 0: 10273.6, 1: 9535.0. Samples: 863054068. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:39:36,063][104569] Avg episode reward: [(0, '8988.448'), (1, '9080.508')] [2023-12-27 03:39:36,157][105692] Updated weights for policy 0, policy_version 1683683 (0.0005) [2023-12-27 03:39:36,209][105692] Updated weights for policy 0, policy_version 1683693 (0.0008) [2023-12-27 03:39:36,274][105692] Updated weights for policy 0, policy_version 1683703 (0.0006) [2023-12-27 03:39:36,644][105620] Updated weights for policy 1, policy_version 1687186 (0.0010) [2023-12-27 03:39:36,706][105620] Updated weights for policy 1, policy_version 1687196 (0.0011) [2023-12-27 03:39:36,772][105620] Updated weights for policy 1, policy_version 1687206 (0.0010) [2023-12-27 03:39:36,830][105620] Updated weights for policy 1, policy_version 1687216 (0.0010) [2023-12-27 03:39:36,935][105692] Updated weights for policy 0, policy_version 1683713 (0.0007) [2023-12-27 03:39:37,002][105692] Updated weights for policy 0, policy_version 1683723 (0.0009) [2023-12-27 03:39:37,047][105692] Updated weights for policy 0, policy_version 1683733 (0.0008) [2023-12-27 03:39:37,106][105692] Updated weights for policy 0, policy_version 1683743 (0.0008) [2023-12-27 03:39:37,567][105620] Updated weights for policy 1, policy_version 1687226 (0.0010) [2023-12-27 03:39:37,619][105620] Updated weights for policy 1, policy_version 1687236 (0.0010) [2023-12-27 03:39:37,667][105620] Updated weights for policy 1, policy_version 1687246 (0.0010) [2023-12-27 03:39:37,833][105692] Updated weights for policy 0, policy_version 1683753 (0.0008) [2023-12-27 03:39:37,884][105692] Updated weights for policy 0, policy_version 1683763 (0.0008) [2023-12-27 03:39:37,941][105692] Updated weights for policy 0, policy_version 1683773 (0.0008) [2023-12-27 03:39:38,408][105620] Updated weights for policy 1, policy_version 1687256 (0.0007) [2023-12-27 03:39:38,469][105620] Updated weights for policy 1, policy_version 1687266 (0.0006) [2023-12-27 03:39:38,517][105620] Updated weights for policy 1, policy_version 1687276 (0.0010) [2023-12-27 03:39:38,635][105692] Updated weights for policy 0, policy_version 1683783 (0.0008) [2023-12-27 03:39:38,694][105692] Updated weights for policy 0, policy_version 1683793 (0.0008) [2023-12-27 03:39:38,751][105692] Updated weights for policy 0, policy_version 1683803 (0.0008) [2023-12-27 03:39:39,246][105620] Updated weights for policy 1, policy_version 1687286 (0.0009) [2023-12-27 03:39:39,306][105620] Updated weights for policy 1, policy_version 1687296 (0.0008) [2023-12-27 03:39:39,377][105620] Updated weights for policy 1, policy_version 1687306 (0.0008) [2023-12-27 03:39:39,480][105692] Updated weights for policy 0, policy_version 1683813 (0.0009) [2023-12-27 03:39:39,548][105692] Updated weights for policy 0, policy_version 1683823 (0.0010) [2023-12-27 03:39:39,601][105692] Updated weights for policy 0, policy_version 1683833 (0.0009) [2023-12-27 03:39:40,076][105620] Updated weights for policy 1, policy_version 1687316 (0.0009) [2023-12-27 03:39:40,132][105620] Updated weights for policy 1, policy_version 1687326 (0.0011) [2023-12-27 03:39:40,185][105620] Updated weights for policy 1, policy_version 1687336 (0.0011) [2023-12-27 03:39:40,418][105692] Updated weights for policy 0, policy_version 1683843 (0.0009) [2023-12-27 03:39:40,474][105692] Updated weights for policy 0, policy_version 1683853 (0.0007) [2023-12-27 03:39:40,527][105692] Updated weights for policy 0, policy_version 1683863 (0.0008) [2023-12-27 03:39:40,959][105620] Updated weights for policy 1, policy_version 1687346 (0.0011) [2023-12-27 03:39:41,022][105620] Updated weights for policy 1, policy_version 1687356 (0.0009) [2023-12-27 03:39:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.5, 300 sec: 19549.7). Total num frames: 863158272. Throughput: 0: 10303.1, 1: 9543.3. Samples: 863171268. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:39:41,063][104569] Avg episode reward: [(0, '8898.541'), (1, '8896.003')] [2023-12-27 03:39:41,090][105620] Updated weights for policy 1, policy_version 1687366 (0.0010) [2023-12-27 03:39:41,157][105620] Updated weights for policy 1, policy_version 1687376 (0.0011) [2023-12-27 03:39:41,313][105692] Updated weights for policy 0, policy_version 1683873 (0.0008) [2023-12-27 03:39:41,374][105692] Updated weights for policy 0, policy_version 1683883 (0.0008) [2023-12-27 03:39:41,438][105692] Updated weights for policy 0, policy_version 1683893 (0.0008) [2023-12-27 03:39:41,498][105692] Updated weights for policy 0, policy_version 1683903 (0.0008) [2023-12-27 03:39:41,993][105620] Updated weights for policy 1, policy_version 1687386 (0.0007) [2023-12-27 03:39:42,044][105620] Updated weights for policy 1, policy_version 1687396 (0.0005) [2023-12-27 03:39:42,113][105620] Updated weights for policy 1, policy_version 1687406 (0.0008) [2023-12-27 03:39:42,273][105692] Updated weights for policy 0, policy_version 1683913 (0.0007) [2023-12-27 03:39:42,336][105692] Updated weights for policy 0, policy_version 1683923 (0.0006) [2023-12-27 03:39:42,395][105692] Updated weights for policy 0, policy_version 1683933 (0.0008) [2023-12-27 03:39:42,865][105620] Updated weights for policy 1, policy_version 1687416 (0.0008) [2023-12-27 03:39:42,926][105620] Updated weights for policy 1, policy_version 1687426 (0.0010) [2023-12-27 03:39:42,993][105620] Updated weights for policy 1, policy_version 1687436 (0.0010) [2023-12-27 03:39:43,039][105692] Updated weights for policy 0, policy_version 1683943 (0.0009) [2023-12-27 03:39:43,112][105692] Updated weights for policy 0, policy_version 1683953 (0.0011) [2023-12-27 03:39:43,181][105692] Updated weights for policy 0, policy_version 1683963 (0.0011) [2023-12-27 03:39:43,676][105620] Updated weights for policy 1, policy_version 1687446 (0.0006) [2023-12-27 03:39:43,737][105620] Updated weights for policy 1, policy_version 1687456 (0.0005) [2023-12-27 03:39:43,789][105620] Updated weights for policy 1, policy_version 1687466 (0.0009) [2023-12-27 03:39:43,881][105692] Updated weights for policy 0, policy_version 1683973 (0.0010) [2023-12-27 03:39:43,930][105692] Updated weights for policy 0, policy_version 1683983 (0.0007) [2023-12-27 03:39:43,994][105692] Updated weights for policy 0, policy_version 1683993 (0.0010) [2023-12-27 03:39:44,460][105620] Updated weights for policy 1, policy_version 1687476 (0.0008) [2023-12-27 03:39:44,523][105620] Updated weights for policy 1, policy_version 1687486 (0.0007) [2023-12-27 03:39:44,585][105620] Updated weights for policy 1, policy_version 1687496 (0.0007) [2023-12-27 03:39:44,860][105692] Updated weights for policy 0, policy_version 1684003 (0.0009) [2023-12-27 03:39:44,921][105692] Updated weights for policy 0, policy_version 1684013 (0.0010) [2023-12-27 03:39:44,984][105692] Updated weights for policy 0, policy_version 1684023 (0.0010) [2023-12-27 03:39:45,185][105620] Updated weights for policy 1, policy_version 1687506 (0.0006) [2023-12-27 03:39:45,249][105620] Updated weights for policy 1, policy_version 1687516 (0.0009) [2023-12-27 03:39:45,322][105620] Updated weights for policy 1, policy_version 1687526 (0.0006) [2023-12-27 03:39:45,389][105620] Updated weights for policy 1, policy_version 1687536 (0.0005) [2023-12-27 03:39:45,716][105692] Updated weights for policy 0, policy_version 1684033 (0.0009) [2023-12-27 03:39:45,781][105692] Updated weights for policy 0, policy_version 1684043 (0.0009) [2023-12-27 03:39:45,841][105692] Updated weights for policy 0, policy_version 1684053 (0.0009) [2023-12-27 03:39:45,888][105692] Updated weights for policy 0, policy_version 1684063 (0.0009) [2023-12-27 03:39:46,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 863256576. Throughput: 0: 10238.0, 1: 9501.1. Samples: 863227684. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:39:46,062][104569] Avg episode reward: [(0, '8806.262'), (1, '9078.646')] [2023-12-27 03:39:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001684064_431185920.pth... [2023-12-27 03:39:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001682848_430874624.pth [2023-12-27 03:39:46,073][105620] Updated weights for policy 1, policy_version 1687546 (0.0008) [2023-12-27 03:39:46,127][105620] Updated weights for policy 1, policy_version 1687556 (0.0008) [2023-12-27 03:39:46,187][105620] Updated weights for policy 1, policy_version 1687566 (0.0009) [2023-12-27 03:39:46,199][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001687568_432078848.pth... [2023-12-27 03:39:46,203][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001686416_431783936.pth [2023-12-27 03:39:46,689][105692] Updated weights for policy 0, policy_version 1684073 (0.0010) [2023-12-27 03:39:46,747][105692] Updated weights for policy 0, policy_version 1684083 (0.0009) [2023-12-27 03:39:46,797][105692] Updated weights for policy 0, policy_version 1684093 (0.0010) [2023-12-27 03:39:46,812][105620] Updated weights for policy 1, policy_version 1687576 (0.0006) [2023-12-27 03:39:46,873][105620] Updated weights for policy 1, policy_version 1687586 (0.0005) [2023-12-27 03:39:46,937][105620] Updated weights for policy 1, policy_version 1687596 (0.0007) [2023-12-27 03:39:47,565][105692] Updated weights for policy 0, policy_version 1684103 (0.0009) [2023-12-27 03:39:47,566][105620] Updated weights for policy 1, policy_version 1687606 (0.0006) [2023-12-27 03:39:47,618][105692] Updated weights for policy 0, policy_version 1684113 (0.0008) [2023-12-27 03:39:47,628][105620] Updated weights for policy 1, policy_version 1687616 (0.0006) [2023-12-27 03:39:47,672][105692] Updated weights for policy 0, policy_version 1684123 (0.0006) [2023-12-27 03:39:47,678][105620] Updated weights for policy 1, policy_version 1687626 (0.0006) [2023-12-27 03:39:48,394][105692] Updated weights for policy 0, policy_version 1684133 (0.0006) [2023-12-27 03:39:48,415][105620] Updated weights for policy 1, policy_version 1687636 (0.0007) [2023-12-27 03:39:48,452][105692] Updated weights for policy 0, policy_version 1684143 (0.0006) [2023-12-27 03:39:48,470][105620] Updated weights for policy 1, policy_version 1687646 (0.0008) [2023-12-27 03:39:48,508][105692] Updated weights for policy 0, policy_version 1684153 (0.0005) [2023-12-27 03:39:48,530][105620] Updated weights for policy 1, policy_version 1687656 (0.0009) [2023-12-27 03:39:49,092][105692] Updated weights for policy 0, policy_version 1684163 (0.0005) [2023-12-27 03:39:49,151][105692] Updated weights for policy 0, policy_version 1684173 (0.0007) [2023-12-27 03:39:49,209][105692] Updated weights for policy 0, policy_version 1684183 (0.0005) [2023-12-27 03:39:49,370][105620] Updated weights for policy 1, policy_version 1687666 (0.0009) [2023-12-27 03:39:49,440][105620] Updated weights for policy 1, policy_version 1687676 (0.0008) [2023-12-27 03:39:49,512][105620] Updated weights for policy 1, policy_version 1687686 (0.0009) [2023-12-27 03:39:49,582][105620] Updated weights for policy 1, policy_version 1687696 (0.0009) [2023-12-27 03:39:49,838][105692] Updated weights for policy 0, policy_version 1684193 (0.0011) [2023-12-27 03:39:49,902][105692] Updated weights for policy 0, policy_version 1684203 (0.0011) [2023-12-27 03:39:49,969][105692] Updated weights for policy 0, policy_version 1684213 (0.0011) [2023-12-27 03:39:50,025][105692] Updated weights for policy 0, policy_version 1684223 (0.0010) [2023-12-27 03:39:50,373][105620] Updated weights for policy 1, policy_version 1687706 (0.0008) [2023-12-27 03:39:50,427][105620] Updated weights for policy 1, policy_version 1687716 (0.0008) [2023-12-27 03:39:50,485][105620] Updated weights for policy 1, policy_version 1687726 (0.0008) [2023-12-27 03:39:50,744][105692] Updated weights for policy 0, policy_version 1684233 (0.0006) [2023-12-27 03:39:50,804][105692] Updated weights for policy 0, policy_version 1684243 (0.0005) [2023-12-27 03:39:50,864][105692] Updated weights for policy 0, policy_version 1684253 (0.0006) [2023-12-27 03:39:51,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 863354880. Throughput: 0: 10221.2, 1: 9555.8. Samples: 863343908. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:39:51,062][104569] Avg episode reward: [(0, '8806.348'), (1, '9171.907')] [2023-12-27 03:39:51,252][105620] Updated weights for policy 1, policy_version 1687736 (0.0007) [2023-12-27 03:39:51,315][105620] Updated weights for policy 1, policy_version 1687746 (0.0008) [2023-12-27 03:39:51,382][105620] Updated weights for policy 1, policy_version 1687756 (0.0008) [2023-12-27 03:39:51,479][105692] Updated weights for policy 0, policy_version 1684263 (0.0006) [2023-12-27 03:39:51,536][105692] Updated weights for policy 0, policy_version 1684273 (0.0006) [2023-12-27 03:39:51,604][105692] Updated weights for policy 0, policy_version 1684283 (0.0006) [2023-12-27 03:39:52,081][105620] Updated weights for policy 1, policy_version 1687766 (0.0008) [2023-12-27 03:39:52,130][105620] Updated weights for policy 1, policy_version 1687776 (0.0005) [2023-12-27 03:39:52,187][105620] Updated weights for policy 1, policy_version 1687786 (0.0007) [2023-12-27 03:39:52,350][105692] Updated weights for policy 0, policy_version 1684293 (0.0008) [2023-12-27 03:39:52,412][105692] Updated weights for policy 0, policy_version 1684303 (0.0006) [2023-12-27 03:39:52,476][105692] Updated weights for policy 0, policy_version 1684313 (0.0006) [2023-12-27 03:39:52,867][105620] Updated weights for policy 1, policy_version 1687796 (0.0007) [2023-12-27 03:39:52,927][105620] Updated weights for policy 1, policy_version 1687806 (0.0009) [2023-12-27 03:39:52,982][105620] Updated weights for policy 1, policy_version 1687816 (0.0009) [2023-12-27 03:39:53,061][105692] Updated weights for policy 0, policy_version 1684323 (0.0006) [2023-12-27 03:39:53,128][105692] Updated weights for policy 0, policy_version 1684333 (0.0006) [2023-12-27 03:39:53,177][105692] Updated weights for policy 0, policy_version 1684343 (0.0008) [2023-12-27 03:39:53,756][105620] Updated weights for policy 1, policy_version 1687826 (0.0008) [2023-12-27 03:39:53,805][105620] Updated weights for policy 1, policy_version 1687836 (0.0008) [2023-12-27 03:39:53,855][105620] Updated weights for policy 1, policy_version 1687846 (0.0009) [2023-12-27 03:39:53,898][105692] Updated weights for policy 0, policy_version 1684353 (0.0009) [2023-12-27 03:39:53,905][105620] Updated weights for policy 1, policy_version 1687856 (0.0008) [2023-12-27 03:39:53,961][105692] Updated weights for policy 0, policy_version 1684363 (0.0009) [2023-12-27 03:39:54,025][105692] Updated weights for policy 0, policy_version 1684373 (0.0009) [2023-12-27 03:39:54,082][105692] Updated weights for policy 0, policy_version 1684383 (0.0009) [2023-12-27 03:39:54,698][105620] Updated weights for policy 1, policy_version 1687866 (0.0006) [2023-12-27 03:39:54,761][105620] Updated weights for policy 1, policy_version 1687876 (0.0005) [2023-12-27 03:39:54,831][105620] Updated weights for policy 1, policy_version 1687886 (0.0005) [2023-12-27 03:39:54,856][105692] Updated weights for policy 0, policy_version 1684393 (0.0009) [2023-12-27 03:39:54,914][105692] Updated weights for policy 0, policy_version 1684404 (0.0010) [2023-12-27 03:39:54,984][105692] Updated weights for policy 0, policy_version 1684415 (0.0010) [2023-12-27 03:39:55,485][105620] Updated weights for policy 1, policy_version 1687896 (0.0008) [2023-12-27 03:39:55,546][105620] Updated weights for policy 1, policy_version 1687906 (0.0009) [2023-12-27 03:39:55,601][105620] Updated weights for policy 1, policy_version 1687916 (0.0008) [2023-12-27 03:39:55,607][105692] Updated weights for policy 0, policy_version 1684425 (0.0010) [2023-12-27 03:39:55,654][105692] Updated weights for policy 0, policy_version 1684435 (0.0007) [2023-12-27 03:39:55,700][105692] Updated weights for policy 0, policy_version 1684445 (0.0008) [2023-12-27 03:39:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 863453184. Throughput: 0: 10142.6, 1: 9588.9. Samples: 863461396. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:39:56,062][104569] Avg episode reward: [(0, '8806.015'), (1, '9172.854')] [2023-12-27 03:39:56,320][105620] Updated weights for policy 1, policy_version 1687926 (0.0009) [2023-12-27 03:39:56,382][105620] Updated weights for policy 1, policy_version 1687936 (0.0009) [2023-12-27 03:39:56,447][105620] Updated weights for policy 1, policy_version 1687946 (0.0009) [2023-12-27 03:39:56,475][105692] Updated weights for policy 0, policy_version 1684455 (0.0008) [2023-12-27 03:39:56,526][105692] Updated weights for policy 0, policy_version 1684465 (0.0009) [2023-12-27 03:39:56,573][105692] Updated weights for policy 0, policy_version 1684475 (0.0008) [2023-12-27 03:39:57,094][105620] Updated weights for policy 1, policy_version 1687956 (0.0008) [2023-12-27 03:39:57,152][105620] Updated weights for policy 1, policy_version 1687966 (0.0010) [2023-12-27 03:39:57,213][105620] Updated weights for policy 1, policy_version 1687976 (0.0010) [2023-12-27 03:39:57,401][105692] Updated weights for policy 0, policy_version 1684485 (0.0008) [2023-12-27 03:39:57,466][105692] Updated weights for policy 0, policy_version 1684495 (0.0008) [2023-12-27 03:39:57,531][105692] Updated weights for policy 0, policy_version 1684505 (0.0008) [2023-12-27 03:39:57,928][105620] Updated weights for policy 1, policy_version 1687986 (0.0009) [2023-12-27 03:39:57,976][105620] Updated weights for policy 1, policy_version 1687996 (0.0010) [2023-12-27 03:39:58,030][105620] Updated weights for policy 1, policy_version 1688006 (0.0010) [2023-12-27 03:39:58,078][105620] Updated weights for policy 1, policy_version 1688016 (0.0010) [2023-12-27 03:39:58,225][105692] Updated weights for policy 0, policy_version 1684515 (0.0008) [2023-12-27 03:39:58,277][105692] Updated weights for policy 0, policy_version 1684525 (0.0008) [2023-12-27 03:39:58,338][105692] Updated weights for policy 0, policy_version 1684535 (0.0008) [2023-12-27 03:39:58,900][105620] Updated weights for policy 1, policy_version 1688026 (0.0008) [2023-12-27 03:39:58,965][105620] Updated weights for policy 1, policy_version 1688036 (0.0008) [2023-12-27 03:39:59,024][105620] Updated weights for policy 1, policy_version 1688046 (0.0008) [2023-12-27 03:39:59,203][105692] Updated weights for policy 0, policy_version 1684545 (0.0007) [2023-12-27 03:39:59,272][105692] Updated weights for policy 0, policy_version 1684555 (0.0007) [2023-12-27 03:39:59,335][105692] Updated weights for policy 0, policy_version 1684565 (0.0011) [2023-12-27 03:39:59,401][105692] Updated weights for policy 0, policy_version 1684575 (0.0008) [2023-12-27 03:39:59,755][105620] Updated weights for policy 1, policy_version 1688056 (0.0006) [2023-12-27 03:39:59,812][105620] Updated weights for policy 1, policy_version 1688066 (0.0006) [2023-12-27 03:39:59,872][105620] Updated weights for policy 1, policy_version 1688076 (0.0008) [2023-12-27 03:40:00,131][105692] Updated weights for policy 0, policy_version 1684585 (0.0010) [2023-12-27 03:40:00,179][105692] Updated weights for policy 0, policy_version 1684595 (0.0010) [2023-12-27 03:40:00,238][105692] Updated weights for policy 0, policy_version 1684605 (0.0010) [2023-12-27 03:40:00,562][105620] Updated weights for policy 1, policy_version 1688086 (0.0007) [2023-12-27 03:40:00,616][105620] Updated weights for policy 1, policy_version 1688096 (0.0008) [2023-12-27 03:40:00,668][105620] Updated weights for policy 1, policy_version 1688106 (0.0008) [2023-12-27 03:40:01,005][105692] Updated weights for policy 0, policy_version 1684615 (0.0010) [2023-12-27 03:40:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 863543296. Throughput: 0: 10088.8, 1: 9583.6. Samples: 863517204. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:40:01,063][104569] Avg episode reward: [(0, '8894.721'), (1, '9173.242')] [2023-12-27 03:40:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001688112_432218112.pth... [2023-12-27 03:40:01,068][105692] Updated weights for policy 0, policy_version 1684625 (0.0010) [2023-12-27 03:40:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001686992_431931392.pth [2023-12-27 03:40:01,117][105692] Updated weights for policy 0, policy_version 1684635 (0.0006) [2023-12-27 03:40:01,146][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001684640_431333376.pth... [2023-12-27 03:40:01,150][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001683488_431038464.pth [2023-12-27 03:40:01,405][105620] Updated weights for policy 1, policy_version 1688116 (0.0008) [2023-12-27 03:40:01,467][105620] Updated weights for policy 1, policy_version 1688126 (0.0009) [2023-12-27 03:40:01,527][105620] Updated weights for policy 1, policy_version 1688136 (0.0009) [2023-12-27 03:40:01,850][105692] Updated weights for policy 0, policy_version 1684645 (0.0009) [2023-12-27 03:40:01,905][105692] Updated weights for policy 0, policy_version 1684655 (0.0010) [2023-12-27 03:40:01,963][105692] Updated weights for policy 0, policy_version 1684665 (0.0010) [2023-12-27 03:40:02,297][105620] Updated weights for policy 1, policy_version 1688146 (0.0008) [2023-12-27 03:40:02,354][105620] Updated weights for policy 1, policy_version 1688156 (0.0008) [2023-12-27 03:40:02,420][105620] Updated weights for policy 1, policy_version 1688166 (0.0009) [2023-12-27 03:40:02,476][105620] Updated weights for policy 1, policy_version 1688176 (0.0008) [2023-12-27 03:40:02,735][105692] Updated weights for policy 0, policy_version 1684675 (0.0010) [2023-12-27 03:40:02,794][105692] Updated weights for policy 0, policy_version 1684685 (0.0010) [2023-12-27 03:40:02,858][105692] Updated weights for policy 0, policy_version 1684695 (0.0010) [2023-12-27 03:40:03,250][105620] Updated weights for policy 1, policy_version 1688186 (0.0008) [2023-12-27 03:40:03,298][105620] Updated weights for policy 1, policy_version 1688196 (0.0008) [2023-12-27 03:40:03,346][105620] Updated weights for policy 1, policy_version 1688206 (0.0007) [2023-12-27 03:40:03,595][105692] Updated weights for policy 0, policy_version 1684705 (0.0010) [2023-12-27 03:40:03,646][105692] Updated weights for policy 0, policy_version 1684715 (0.0010) [2023-12-27 03:40:03,700][105692] Updated weights for policy 0, policy_version 1684725 (0.0010) [2023-12-27 03:40:03,758][105692] Updated weights for policy 0, policy_version 1684735 (0.0010) [2023-12-27 03:40:04,119][105620] Updated weights for policy 1, policy_version 1688216 (0.0009) [2023-12-27 03:40:04,172][105620] Updated weights for policy 1, policy_version 1688226 (0.0008) [2023-12-27 03:40:04,216][105620] Updated weights for policy 1, policy_version 1688236 (0.0008) [2023-12-27 03:40:04,510][105692] Updated weights for policy 0, policy_version 1684745 (0.0010) [2023-12-27 03:40:04,562][105692] Updated weights for policy 0, policy_version 1684755 (0.0010) [2023-12-27 03:40:04,620][105692] Updated weights for policy 0, policy_version 1684765 (0.0010) [2023-12-27 03:40:04,998][105620] Updated weights for policy 1, policy_version 1688246 (0.0009) [2023-12-27 03:40:05,056][105620] Updated weights for policy 1, policy_version 1688256 (0.0008) [2023-12-27 03:40:05,114][105620] Updated weights for policy 1, policy_version 1688266 (0.0008) [2023-12-27 03:40:05,362][105692] Updated weights for policy 0, policy_version 1684775 (0.0010) [2023-12-27 03:40:05,413][105692] Updated weights for policy 0, policy_version 1684785 (0.0010) [2023-12-27 03:40:05,467][105692] Updated weights for policy 0, policy_version 1684795 (0.0010) [2023-12-27 03:40:05,873][105620] Updated weights for policy 1, policy_version 1688276 (0.0009) [2023-12-27 03:40:05,916][105620] Updated weights for policy 1, policy_version 1688286 (0.0010) [2023-12-27 03:40:05,964][105620] Updated weights for policy 1, policy_version 1688296 (0.0010) [2023-12-27 03:40:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 863641600. Throughput: 0: 9952.0, 1: 9608.2. Samples: 863629312. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:40:06,063][104569] Avg episode reward: [(0, '8803.729'), (1, '9080.664')] [2023-12-27 03:40:06,180][105692] Updated weights for policy 0, policy_version 1684805 (0.0010) [2023-12-27 03:40:06,235][105692] Updated weights for policy 0, policy_version 1684815 (0.0010) [2023-12-27 03:40:06,298][105692] Updated weights for policy 0, policy_version 1684825 (0.0011) [2023-12-27 03:40:06,658][105620] Updated weights for policy 1, policy_version 1688306 (0.0010) [2023-12-27 03:40:06,710][105620] Updated weights for policy 1, policy_version 1688316 (0.0011) [2023-12-27 03:40:06,759][105620] Updated weights for policy 1, policy_version 1688326 (0.0011) [2023-12-27 03:40:06,807][105620] Updated weights for policy 1, policy_version 1688336 (0.0011) [2023-12-27 03:40:07,048][105692] Updated weights for policy 0, policy_version 1684835 (0.0010) [2023-12-27 03:40:07,096][105692] Updated weights for policy 0, policy_version 1684845 (0.0010) [2023-12-27 03:40:07,148][105692] Updated weights for policy 0, policy_version 1684855 (0.0010) [2023-12-27 03:40:07,539][105620] Updated weights for policy 1, policy_version 1688346 (0.0009) [2023-12-27 03:40:07,609][105620] Updated weights for policy 1, policy_version 1688356 (0.0006) [2023-12-27 03:40:07,669][105620] Updated weights for policy 1, policy_version 1688366 (0.0008) [2023-12-27 03:40:07,708][105692] Updated weights for policy 0, policy_version 1684865 (0.0005) [2023-12-27 03:40:07,759][105692] Updated weights for policy 0, policy_version 1684875 (0.0005) [2023-12-27 03:40:07,816][105692] Updated weights for policy 0, policy_version 1684885 (0.0006) [2023-12-27 03:40:07,872][105692] Updated weights for policy 0, policy_version 1684895 (0.0005) [2023-12-27 03:40:08,385][105692] Updated weights for policy 0, policy_version 1684905 (0.0009) [2023-12-27 03:40:08,411][105620] Updated weights for policy 1, policy_version 1688376 (0.0006) [2023-12-27 03:40:08,444][105692] Updated weights for policy 0, policy_version 1684915 (0.0010) [2023-12-27 03:40:08,466][105620] Updated weights for policy 1, policy_version 1688386 (0.0007) [2023-12-27 03:40:08,504][105692] Updated weights for policy 0, policy_version 1684925 (0.0010) [2023-12-27 03:40:08,525][105620] Updated weights for policy 1, policy_version 1688396 (0.0005) [2023-12-27 03:40:09,248][105620] Updated weights for policy 1, policy_version 1688406 (0.0007) [2023-12-27 03:40:09,268][105692] Updated weights for policy 0, policy_version 1684935 (0.0009) [2023-12-27 03:40:09,309][105620] Updated weights for policy 1, policy_version 1688416 (0.0009) [2023-12-27 03:40:09,332][105692] Updated weights for policy 0, policy_version 1684945 (0.0008) [2023-12-27 03:40:09,374][105620] Updated weights for policy 1, policy_version 1688426 (0.0008) [2023-12-27 03:40:09,387][105692] Updated weights for policy 0, policy_version 1684955 (0.0008) [2023-12-27 03:40:10,037][105692] Updated weights for policy 0, policy_version 1684965 (0.0009) [2023-12-27 03:40:10,092][105692] Updated weights for policy 0, policy_version 1684975 (0.0009) [2023-12-27 03:40:10,139][105692] Updated weights for policy 0, policy_version 1684985 (0.0009) [2023-12-27 03:40:10,230][105620] Updated weights for policy 1, policy_version 1688436 (0.0008) [2023-12-27 03:40:10,301][105620] Updated weights for policy 1, policy_version 1688446 (0.0010) [2023-12-27 03:40:10,366][105620] Updated weights for policy 1, policy_version 1688456 (0.0009) [2023-12-27 03:40:10,840][105692] Updated weights for policy 0, policy_version 1684995 (0.0009) [2023-12-27 03:40:10,907][105692] Updated weights for policy 0, policy_version 1685005 (0.0009) [2023-12-27 03:40:10,965][105692] Updated weights for policy 0, policy_version 1685015 (0.0008) [2023-12-27 03:40:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 863739904. Throughput: 0: 9922.6, 1: 9611.5. Samples: 863747248. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:40:11,063][104569] Avg episode reward: [(0, '8617.501'), (1, '8987.557')] [2023-12-27 03:40:11,063][105620] Updated weights for policy 1, policy_version 1688466 (0.0009) [2023-12-27 03:40:11,129][105620] Updated weights for policy 1, policy_version 1688476 (0.0009) [2023-12-27 03:40:11,194][105620] Updated weights for policy 1, policy_version 1688486 (0.0009) [2023-12-27 03:40:11,244][105620] Updated weights for policy 1, policy_version 1688496 (0.0010) [2023-12-27 03:40:11,817][105692] Updated weights for policy 0, policy_version 1685025 (0.0010) [2023-12-27 03:40:11,882][105692] Updated weights for policy 0, policy_version 1685035 (0.0006) [2023-12-27 03:40:11,944][105692] Updated weights for policy 0, policy_version 1685045 (0.0008) [2023-12-27 03:40:12,003][105692] Updated weights for policy 0, policy_version 1685055 (0.0010) [2023-12-27 03:40:12,035][105620] Updated weights for policy 1, policy_version 1688506 (0.0009) [2023-12-27 03:40:12,091][105620] Updated weights for policy 1, policy_version 1688516 (0.0008) [2023-12-27 03:40:12,155][105620] Updated weights for policy 1, policy_version 1688526 (0.0008) [2023-12-27 03:40:12,735][105692] Updated weights for policy 0, policy_version 1685065 (0.0009) [2023-12-27 03:40:12,792][105692] Updated weights for policy 0, policy_version 1685075 (0.0008) [2023-12-27 03:40:12,862][105692] Updated weights for policy 0, policy_version 1685085 (0.0006) [2023-12-27 03:40:12,891][105620] Updated weights for policy 1, policy_version 1688536 (0.0006) [2023-12-27 03:40:12,957][105620] Updated weights for policy 1, policy_version 1688546 (0.0009) [2023-12-27 03:40:13,018][105620] Updated weights for policy 1, policy_version 1688556 (0.0009) [2023-12-27 03:40:13,505][105692] Updated weights for policy 0, policy_version 1685095 (0.0006) [2023-12-27 03:40:13,571][105692] Updated weights for policy 0, policy_version 1685105 (0.0005) [2023-12-27 03:40:13,628][105692] Updated weights for policy 0, policy_version 1685115 (0.0009) [2023-12-27 03:40:13,819][105620] Updated weights for policy 1, policy_version 1688567 (0.0009) [2023-12-27 03:40:13,878][105620] Updated weights for policy 1, policy_version 1688578 (0.0009) [2023-12-27 03:40:13,930][105620] Updated weights for policy 1, policy_version 1688588 (0.0009) [2023-12-27 03:40:14,333][105692] Updated weights for policy 0, policy_version 1685125 (0.0007) [2023-12-27 03:40:14,401][105692] Updated weights for policy 0, policy_version 1685135 (0.0005) [2023-12-27 03:40:14,466][105692] Updated weights for policy 0, policy_version 1685145 (0.0006) [2023-12-27 03:40:14,669][105620] Updated weights for policy 1, policy_version 1688598 (0.0008) [2023-12-27 03:40:14,737][105620] Updated weights for policy 1, policy_version 1688608 (0.0009) [2023-12-27 03:40:14,810][105620] Updated weights for policy 1, policy_version 1688618 (0.0009) [2023-12-27 03:40:15,118][105692] Updated weights for policy 0, policy_version 1685155 (0.0008) [2023-12-27 03:40:15,177][105692] Updated weights for policy 0, policy_version 1685165 (0.0010) [2023-12-27 03:40:15,241][105692] Updated weights for policy 0, policy_version 1685175 (0.0009) [2023-12-27 03:40:15,481][105620] Updated weights for policy 1, policy_version 1688628 (0.0009) [2023-12-27 03:40:15,539][105620] Updated weights for policy 1, policy_version 1688639 (0.0010) [2023-12-27 03:40:15,598][105620] Updated weights for policy 1, policy_version 1688650 (0.0010) [2023-12-27 03:40:15,898][105692] Updated weights for policy 0, policy_version 1685185 (0.0009) [2023-12-27 03:40:15,957][105692] Updated weights for policy 0, policy_version 1685195 (0.0005) [2023-12-27 03:40:16,006][105692] Updated weights for policy 0, policy_version 1685205 (0.0010) [2023-12-27 03:40:16,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.4, 300 sec: 19494.2). Total num frames: 863830016. Throughput: 0: 9802.3, 1: 9515.5. Samples: 863802936. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:40:16,062][104569] Avg episode reward: [(0, '8530.088'), (1, '8989.690')] [2023-12-27 03:40:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001688656_432357376.pth... [2023-12-27 03:40:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001687568_432078848.pth [2023-12-27 03:40:16,071][105692] Updated weights for policy 0, policy_version 1685215 (0.0011) [2023-12-27 03:40:16,075][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001685216_431480832.pth... [2023-12-27 03:40:16,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001684064_431185920.pth [2023-12-27 03:40:16,321][105620] Updated weights for policy 1, policy_version 1688660 (0.0007) [2023-12-27 03:40:16,391][105620] Updated weights for policy 1, policy_version 1688670 (0.0006) [2023-12-27 03:40:16,434][105586] KL-divergence is very high: 167.7630 [2023-12-27 03:40:16,445][105586] KL-divergence is very high: 169.4792 [2023-12-27 03:40:16,455][105620] Updated weights for policy 1, policy_version 1688680 (0.0008) [2023-12-27 03:40:16,474][105586] KL-divergence is very high: 189.6541 [2023-12-27 03:40:16,484][105586] KL-divergence is very high: 177.3249 [2023-12-27 03:40:16,695][105692] Updated weights for policy 0, policy_version 1685225 (0.0009) [2023-12-27 03:40:16,755][105692] Updated weights for policy 0, policy_version 1685235 (0.0009) [2023-12-27 03:40:16,805][105692] Updated weights for policy 0, policy_version 1685245 (0.0009) [2023-12-27 03:40:17,029][105620] Updated weights for policy 1, policy_version 1688690 (0.0009) [2023-12-27 03:40:17,095][105620] Updated weights for policy 1, policy_version 1688700 (0.0005) [2023-12-27 03:40:17,155][105620] Updated weights for policy 1, policy_version 1688710 (0.0005) [2023-12-27 03:40:17,216][105620] Updated weights for policy 1, policy_version 1688720 (0.0005) [2023-12-27 03:40:17,522][105692] Updated weights for policy 0, policy_version 1685255 (0.0006) [2023-12-27 03:40:17,592][105692] Updated weights for policy 0, policy_version 1685265 (0.0005) [2023-12-27 03:40:17,650][105692] Updated weights for policy 0, policy_version 1685275 (0.0008) [2023-12-27 03:40:17,709][105620] Updated weights for policy 1, policy_version 1688730 (0.0011) [2023-12-27 03:40:17,768][105620] Updated weights for policy 1, policy_version 1688740 (0.0010) [2023-12-27 03:40:17,836][105620] Updated weights for policy 1, policy_version 1688750 (0.0010) [2023-12-27 03:40:18,363][105692] Updated weights for policy 0, policy_version 1685285 (0.0008) [2023-12-27 03:40:18,421][105692] Updated weights for policy 0, policy_version 1685295 (0.0006) [2023-12-27 03:40:18,483][105692] Updated weights for policy 0, policy_version 1685305 (0.0006) [2023-12-27 03:40:18,586][105620] Updated weights for policy 1, policy_version 1688760 (0.0010) [2023-12-27 03:40:18,642][105620] Updated weights for policy 1, policy_version 1688770 (0.0008) [2023-12-27 03:40:18,702][105620] Updated weights for policy 1, policy_version 1688780 (0.0009) [2023-12-27 03:40:19,159][105692] Updated weights for policy 0, policy_version 1685315 (0.0008) [2023-12-27 03:40:19,229][105692] Updated weights for policy 0, policy_version 1685325 (0.0006) [2023-12-27 03:40:19,293][105692] Updated weights for policy 0, policy_version 1685335 (0.0009) [2023-12-27 03:40:19,483][105620] Updated weights for policy 1, policy_version 1688790 (0.0009) [2023-12-27 03:40:19,546][105620] Updated weights for policy 1, policy_version 1688800 (0.0009) [2023-12-27 03:40:19,605][105620] Updated weights for policy 1, policy_version 1688810 (0.0009) [2023-12-27 03:40:19,990][105692] Updated weights for policy 0, policy_version 1685345 (0.0009) [2023-12-27 03:40:20,053][105692] Updated weights for policy 0, policy_version 1685355 (0.0009) [2023-12-27 03:40:20,104][105692] Updated weights for policy 0, policy_version 1685365 (0.0009) [2023-12-27 03:40:20,160][105692] Updated weights for policy 0, policy_version 1685376 (0.0007) [2023-12-27 03:40:20,380][105620] Updated weights for policy 1, policy_version 1688820 (0.0008) [2023-12-27 03:40:20,439][105620] Updated weights for policy 1, policy_version 1688830 (0.0009) [2023-12-27 03:40:20,490][105620] Updated weights for policy 1, policy_version 1688840 (0.0009) [2023-12-27 03:40:20,945][105692] Updated weights for policy 0, policy_version 1685386 (0.0008) [2023-12-27 03:40:21,012][105692] Updated weights for policy 0, policy_version 1685396 (0.0006) [2023-12-27 03:40:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 863928320. Throughput: 0: 9779.1, 1: 9545.0. Samples: 863923648. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:40:21,062][104569] Avg episode reward: [(0, '8622.312'), (1, '8804.123')] [2023-12-27 03:40:21,081][105692] Updated weights for policy 0, policy_version 1685406 (0.0011) [2023-12-27 03:40:21,239][105620] Updated weights for policy 1, policy_version 1688850 (0.0009) [2023-12-27 03:40:21,309][105620] Updated weights for policy 1, policy_version 1688860 (0.0008) [2023-12-27 03:40:21,379][105620] Updated weights for policy 1, policy_version 1688870 (0.0008) [2023-12-27 03:40:21,451][105620] Updated weights for policy 1, policy_version 1688880 (0.0010) [2023-12-27 03:40:21,891][105692] Updated weights for policy 0, policy_version 1685416 (0.0009) [2023-12-27 03:40:21,958][105692] Updated weights for policy 0, policy_version 1685426 (0.0009) [2023-12-27 03:40:22,023][105692] Updated weights for policy 0, policy_version 1685436 (0.0010) [2023-12-27 03:40:22,162][105620] Updated weights for policy 1, policy_version 1688890 (0.0009) [2023-12-27 03:40:22,218][105620] Updated weights for policy 1, policy_version 1688900 (0.0009) [2023-12-27 03:40:22,267][105620] Updated weights for policy 1, policy_version 1688910 (0.0008) [2023-12-27 03:40:22,802][105692] Updated weights for policy 0, policy_version 1685446 (0.0009) [2023-12-27 03:40:22,853][105692] Updated weights for policy 0, policy_version 1685456 (0.0009) [2023-12-27 03:40:22,904][105692] Updated weights for policy 0, policy_version 1685466 (0.0008) [2023-12-27 03:40:23,081][105620] Updated weights for policy 1, policy_version 1688920 (0.0008) [2023-12-27 03:40:23,140][105620] Updated weights for policy 1, policy_version 1688930 (0.0008) [2023-12-27 03:40:23,200][105620] Updated weights for policy 1, policy_version 1688940 (0.0009) [2023-12-27 03:40:23,741][105692] Updated weights for policy 0, policy_version 1685476 (0.0009) [2023-12-27 03:40:23,789][105692] Updated weights for policy 0, policy_version 1685487 (0.0009) [2023-12-27 03:40:23,805][105620] Updated weights for policy 1, policy_version 1688950 (0.0007) [2023-12-27 03:40:23,836][105692] Updated weights for policy 0, policy_version 1685497 (0.0005) [2023-12-27 03:40:23,872][105620] Updated weights for policy 1, policy_version 1688960 (0.0005) [2023-12-27 03:40:23,934][105620] Updated weights for policy 1, policy_version 1688970 (0.0005) [2023-12-27 03:40:24,476][105620] Updated weights for policy 1, policy_version 1688980 (0.0007) [2023-12-27 03:40:24,523][105620] Updated weights for policy 1, policy_version 1688990 (0.0008) [2023-12-27 03:40:24,586][105620] Updated weights for policy 1, policy_version 1689000 (0.0009) [2023-12-27 03:40:24,601][105692] Updated weights for policy 0, policy_version 1685507 (0.0005) [2023-12-27 03:40:24,650][105692] Updated weights for policy 0, policy_version 1685517 (0.0007) [2023-12-27 03:40:24,698][105692] Updated weights for policy 0, policy_version 1685527 (0.0009) [2023-12-27 03:40:25,221][105620] Updated weights for policy 1, policy_version 1689010 (0.0009) [2023-12-27 03:40:25,281][105620] Updated weights for policy 1, policy_version 1689020 (0.0005) [2023-12-27 03:40:25,341][105620] Updated weights for policy 1, policy_version 1689030 (0.0005) [2023-12-27 03:40:25,396][105620] Updated weights for policy 1, policy_version 1689040 (0.0005) [2023-12-27 03:40:25,529][105692] Updated weights for policy 0, policy_version 1685537 (0.0009) [2023-12-27 03:40:25,578][105692] Updated weights for policy 0, policy_version 1685548 (0.0008) [2023-12-27 03:40:25,624][105692] Updated weights for policy 0, policy_version 1685558 (0.0005) [2023-12-27 03:40:25,675][105692] Updated weights for policy 0, policy_version 1685568 (0.0009) [2023-12-27 03:40:25,900][105620] Updated weights for policy 1, policy_version 1689050 (0.0005) [2023-12-27 03:40:25,952][105620] Updated weights for policy 1, policy_version 1689060 (0.0005) [2023-12-27 03:40:26,007][105620] Updated weights for policy 1, policy_version 1689070 (0.0005) [2023-12-27 03:40:26,062][104569] Fps is (10 sec: 20479.3, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 864034816. Throughput: 0: 9622.5, 1: 9675.3. Samples: 864039676. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:40:26,063][104569] Avg episode reward: [(0, '8711.416'), (1, '8709.254')] [2023-12-27 03:40:26,493][105692] Updated weights for policy 0, policy_version 1685578 (0.0009) [2023-12-27 03:40:26,546][105692] Updated weights for policy 0, policy_version 1685588 (0.0010) [2023-12-27 03:40:26,599][105692] Updated weights for policy 0, policy_version 1685598 (0.0010) [2023-12-27 03:40:26,637][105620] Updated weights for policy 1, policy_version 1689080 (0.0005) [2023-12-27 03:40:26,693][105620] Updated weights for policy 1, policy_version 1689090 (0.0005) [2023-12-27 03:40:26,748][105620] Updated weights for policy 1, policy_version 1689100 (0.0005) [2023-12-27 03:40:27,246][105620] Updated weights for policy 1, policy_version 1689110 (0.0005) [2023-12-27 03:40:27,303][105620] Updated weights for policy 1, policy_version 1689120 (0.0006) [2023-12-27 03:40:27,364][105620] Updated weights for policy 1, policy_version 1689130 (0.0007) [2023-12-27 03:40:27,481][105692] Updated weights for policy 0, policy_version 1685608 (0.0009) [2023-12-27 03:40:27,535][105692] Updated weights for policy 0, policy_version 1685619 (0.0010) [2023-12-27 03:40:27,587][105692] Updated weights for policy 0, policy_version 1685629 (0.0009) [2023-12-27 03:40:27,928][105620] Updated weights for policy 1, policy_version 1689140 (0.0008) [2023-12-27 03:40:27,975][105620] Updated weights for policy 1, policy_version 1689150 (0.0010) [2023-12-27 03:40:28,026][105620] Updated weights for policy 1, policy_version 1689160 (0.0010) [2023-12-27 03:40:28,407][105692] Updated weights for policy 0, policy_version 1685640 (0.0010) [2023-12-27 03:40:28,464][105692] Updated weights for policy 0, policy_version 1685650 (0.0009) [2023-12-27 03:40:28,517][105692] Updated weights for policy 0, policy_version 1685661 (0.0009) [2023-12-27 03:40:28,732][105620] Updated weights for policy 1, policy_version 1689170 (0.0010) [2023-12-27 03:40:28,795][105620] Updated weights for policy 1, policy_version 1689180 (0.0008) [2023-12-27 03:40:28,858][105620] Updated weights for policy 1, policy_version 1689190 (0.0008) [2023-12-27 03:40:28,921][105620] Updated weights for policy 1, policy_version 1689200 (0.0006) [2023-12-27 03:40:29,378][105692] Updated weights for policy 0, policy_version 1685671 (0.0009) [2023-12-27 03:40:29,429][105692] Updated weights for policy 0, policy_version 1685681 (0.0008) [2023-12-27 03:40:29,487][105692] Updated weights for policy 0, policy_version 1685691 (0.0008) [2023-12-27 03:40:29,523][105620] Updated weights for policy 1, policy_version 1689210 (0.0009) [2023-12-27 03:40:29,572][105620] Updated weights for policy 1, policy_version 1689220 (0.0010) [2023-12-27 03:40:29,625][105620] Updated weights for policy 1, policy_version 1689230 (0.0010) [2023-12-27 03:40:30,244][105692] Updated weights for policy 0, policy_version 1685701 (0.0007) [2023-12-27 03:40:30,306][105692] Updated weights for policy 0, policy_version 1685711 (0.0008) [2023-12-27 03:40:30,396][105692] Updated weights for policy 0, policy_version 1685721 (0.0008) [2023-12-27 03:40:30,398][105620] Updated weights for policy 1, policy_version 1689240 (0.0010) [2023-12-27 03:40:30,444][105620] Updated weights for policy 1, policy_version 1689250 (0.0006) [2023-12-27 03:40:30,493][105620] Updated weights for policy 1, policy_version 1689260 (0.0005) [2023-12-27 03:40:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 864124928. Throughput: 0: 9564.0, 1: 9800.3. Samples: 864099076. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:40:31,062][104569] Avg episode reward: [(0, '8715.060'), (1, '8804.024')] [2023-12-27 03:40:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001685728_431611904.pth... [2023-12-27 03:40:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001689264_432513024.pth... [2023-12-27 03:40:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001684640_431333376.pth [2023-12-27 03:40:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001688112_432218112.pth [2023-12-27 03:40:31,130][105620] Updated weights for policy 1, policy_version 1689270 (0.0006) [2023-12-27 03:40:31,146][105692] Updated weights for policy 0, policy_version 1685731 (0.0008) [2023-12-27 03:40:31,186][105620] Updated weights for policy 1, policy_version 1689280 (0.0006) [2023-12-27 03:40:31,206][105692] Updated weights for policy 0, policy_version 1685741 (0.0008) [2023-12-27 03:40:31,233][105620] Updated weights for policy 1, policy_version 1689290 (0.0008) [2023-12-27 03:40:31,272][105692] Updated weights for policy 0, policy_version 1685751 (0.0008) [2023-12-27 03:40:31,895][105620] Updated weights for policy 1, policy_version 1689300 (0.0008) [2023-12-27 03:40:31,946][105620] Updated weights for policy 1, policy_version 1689310 (0.0009) [2023-12-27 03:40:32,000][105620] Updated weights for policy 1, policy_version 1689320 (0.0009) [2023-12-27 03:40:32,015][105692] Updated weights for policy 0, policy_version 1685761 (0.0006) [2023-12-27 03:40:32,066][105692] Updated weights for policy 0, policy_version 1685771 (0.0007) [2023-12-27 03:40:32,116][105692] Updated weights for policy 0, policy_version 1685781 (0.0009) [2023-12-27 03:40:32,163][105692] Updated weights for policy 0, policy_version 1685791 (0.0008) [2023-12-27 03:40:32,729][105620] Updated weights for policy 1, policy_version 1689330 (0.0008) [2023-12-27 03:40:32,776][105620] Updated weights for policy 1, policy_version 1689340 (0.0005) [2023-12-27 03:40:32,825][105620] Updated weights for policy 1, policy_version 1689350 (0.0005) [2023-12-27 03:40:32,877][105620] Updated weights for policy 1, policy_version 1689360 (0.0005) [2023-12-27 03:40:32,915][105692] Updated weights for policy 0, policy_version 1685801 (0.0009) [2023-12-27 03:40:32,972][105692] Updated weights for policy 0, policy_version 1685811 (0.0010) [2023-12-27 03:40:33,031][105692] Updated weights for policy 0, policy_version 1685821 (0.0010) [2023-12-27 03:40:33,449][105620] Updated weights for policy 1, policy_version 1689370 (0.0009) [2023-12-27 03:40:33,513][105620] Updated weights for policy 1, policy_version 1689380 (0.0009) [2023-12-27 03:40:33,576][105620] Updated weights for policy 1, policy_version 1689390 (0.0008) [2023-12-27 03:40:33,825][105692] Updated weights for policy 0, policy_version 1685831 (0.0009) [2023-12-27 03:40:33,878][105692] Updated weights for policy 0, policy_version 1685841 (0.0009) [2023-12-27 03:40:33,930][105692] Updated weights for policy 0, policy_version 1685851 (0.0009) [2023-12-27 03:40:34,221][105620] Updated weights for policy 1, policy_version 1689400 (0.0010) [2023-12-27 03:40:34,281][105620] Updated weights for policy 1, policy_version 1689410 (0.0009) [2023-12-27 03:40:34,340][105620] Updated weights for policy 1, policy_version 1689420 (0.0009) [2023-12-27 03:40:34,731][105692] Updated weights for policy 0, policy_version 1685861 (0.0009) [2023-12-27 03:40:34,792][105692] Updated weights for policy 0, policy_version 1685871 (0.0008) [2023-12-27 03:40:34,846][105692] Updated weights for policy 0, policy_version 1685881 (0.0009) [2023-12-27 03:40:35,064][105620] Updated weights for policy 1, policy_version 1689430 (0.0009) [2023-12-27 03:40:35,124][105620] Updated weights for policy 1, policy_version 1689440 (0.0006) [2023-12-27 03:40:35,182][105620] Updated weights for policy 1, policy_version 1689450 (0.0005) [2023-12-27 03:40:35,628][105692] Updated weights for policy 0, policy_version 1685891 (0.0009) [2023-12-27 03:40:35,686][105692] Updated weights for policy 0, policy_version 1685901 (0.0009) [2023-12-27 03:40:35,748][105692] Updated weights for policy 0, policy_version 1685911 (0.0009) [2023-12-27 03:40:35,873][105620] Updated weights for policy 1, policy_version 1689460 (0.0009) [2023-12-27 03:40:35,925][105620] Updated weights for policy 1, policy_version 1689470 (0.0010) [2023-12-27 03:40:35,987][105620] Updated weights for policy 1, policy_version 1689480 (0.0009) [2023-12-27 03:40:36,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 864231424. Throughput: 0: 9506.2, 1: 9871.6. Samples: 864215908. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:40:36,062][104569] Avg episode reward: [(0, '8532.704'), (1, '8528.364')] [2023-12-27 03:40:36,381][105692] Updated weights for policy 0, policy_version 1685921 (0.0009) [2023-12-27 03:40:36,436][105692] Updated weights for policy 0, policy_version 1685931 (0.0005) [2023-12-27 03:40:36,495][105692] Updated weights for policy 0, policy_version 1685941 (0.0011) [2023-12-27 03:40:36,554][105692] Updated weights for policy 0, policy_version 1685951 (0.0011) [2023-12-27 03:40:36,781][105620] Updated weights for policy 1, policy_version 1689490 (0.0010) [2023-12-27 03:40:36,838][105620] Updated weights for policy 1, policy_version 1689500 (0.0009) [2023-12-27 03:40:36,908][105620] Updated weights for policy 1, policy_version 1689510 (0.0005) [2023-12-27 03:40:36,970][105620] Updated weights for policy 1, policy_version 1689520 (0.0007) [2023-12-27 03:40:37,156][105692] Updated weights for policy 0, policy_version 1685961 (0.0008) [2023-12-27 03:40:37,208][105692] Updated weights for policy 0, policy_version 1685971 (0.0010) [2023-12-27 03:40:37,264][105692] Updated weights for policy 0, policy_version 1685981 (0.0010) [2023-12-27 03:40:37,687][105620] Updated weights for policy 1, policy_version 1689530 (0.0006) [2023-12-27 03:40:37,749][105620] Updated weights for policy 1, policy_version 1689540 (0.0006) [2023-12-27 03:40:37,815][105620] Updated weights for policy 1, policy_version 1689550 (0.0006) [2023-12-27 03:40:38,005][105692] Updated weights for policy 0, policy_version 1685991 (0.0010) [2023-12-27 03:40:38,056][105692] Updated weights for policy 0, policy_version 1686001 (0.0010) [2023-12-27 03:40:38,103][105692] Updated weights for policy 0, policy_version 1686011 (0.0009) [2023-12-27 03:40:38,317][105620] Updated weights for policy 1, policy_version 1689560 (0.0006) [2023-12-27 03:40:38,385][105620] Updated weights for policy 1, policy_version 1689570 (0.0008) [2023-12-27 03:40:38,449][105620] Updated weights for policy 1, policy_version 1689580 (0.0005) [2023-12-27 03:40:38,874][105692] Updated weights for policy 0, policy_version 1686021 (0.0010) [2023-12-27 03:40:38,937][105692] Updated weights for policy 0, policy_version 1686031 (0.0011) [2023-12-27 03:40:38,991][105692] Updated weights for policy 0, policy_version 1686041 (0.0010) [2023-12-27 03:40:39,056][105620] Updated weights for policy 1, policy_version 1689590 (0.0008) [2023-12-27 03:40:39,114][105620] Updated weights for policy 1, policy_version 1689600 (0.0009) [2023-12-27 03:40:39,169][105620] Updated weights for policy 1, policy_version 1689610 (0.0010) [2023-12-27 03:40:39,672][105692] Updated weights for policy 0, policy_version 1686051 (0.0010) [2023-12-27 03:40:39,730][105692] Updated weights for policy 0, policy_version 1686061 (0.0006) [2023-12-27 03:40:39,790][105692] Updated weights for policy 0, policy_version 1686071 (0.0005) [2023-12-27 03:40:40,033][105620] Updated weights for policy 1, policy_version 1689620 (0.0009) [2023-12-27 03:40:40,091][105620] Updated weights for policy 1, policy_version 1689630 (0.0010) [2023-12-27 03:40:40,144][105620] Updated weights for policy 1, policy_version 1689640 (0.0010) [2023-12-27 03:40:40,424][105692] Updated weights for policy 0, policy_version 1686081 (0.0009) [2023-12-27 03:40:40,479][105692] Updated weights for policy 0, policy_version 1686091 (0.0009) [2023-12-27 03:40:40,538][105692] Updated weights for policy 0, policy_version 1686101 (0.0009) [2023-12-27 03:40:40,600][105692] Updated weights for policy 0, policy_version 1686111 (0.0009) [2023-12-27 03:40:40,855][105620] Updated weights for policy 1, policy_version 1689650 (0.0009) [2023-12-27 03:40:40,913][105620] Updated weights for policy 1, policy_version 1689660 (0.0010) [2023-12-27 03:40:40,966][105620] Updated weights for policy 1, policy_version 1689670 (0.0009) [2023-12-27 03:40:41,022][105620] Updated weights for policy 1, policy_version 1689680 (0.0010) [2023-12-27 03:40:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 864329728. Throughput: 0: 9506.3, 1: 9916.5. Samples: 864335424. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:40:41,062][104569] Avg episode reward: [(0, '8624.509'), (1, '8804.521')] [2023-12-27 03:40:41,415][105692] Updated weights for policy 0, policy_version 1686121 (0.0008) [2023-12-27 03:40:41,469][105692] Updated weights for policy 0, policy_version 1686131 (0.0008) [2023-12-27 03:40:41,518][105692] Updated weights for policy 0, policy_version 1686141 (0.0009) [2023-12-27 03:40:41,793][105620] Updated weights for policy 1, policy_version 1689690 (0.0008) [2023-12-27 03:40:41,859][105620] Updated weights for policy 1, policy_version 1689700 (0.0009) [2023-12-27 03:40:41,921][105620] Updated weights for policy 1, policy_version 1689710 (0.0008) [2023-12-27 03:40:42,256][105692] Updated weights for policy 0, policy_version 1686151 (0.0009) [2023-12-27 03:40:42,319][105692] Updated weights for policy 0, policy_version 1686161 (0.0009) [2023-12-27 03:40:42,387][105692] Updated weights for policy 0, policy_version 1686171 (0.0009) [2023-12-27 03:40:42,727][105620] Updated weights for policy 1, policy_version 1689720 (0.0009) [2023-12-27 03:40:42,778][105620] Updated weights for policy 1, policy_version 1689730 (0.0008) [2023-12-27 03:40:42,838][105620] Updated weights for policy 1, policy_version 1689740 (0.0008) [2023-12-27 03:40:43,076][105692] Updated weights for policy 0, policy_version 1686181 (0.0009) [2023-12-27 03:40:43,130][105692] Updated weights for policy 0, policy_version 1686191 (0.0010) [2023-12-27 03:40:43,184][105692] Updated weights for policy 0, policy_version 1686203 (0.0010) [2023-12-27 03:40:43,604][105620] Updated weights for policy 1, policy_version 1689750 (0.0008) [2023-12-27 03:40:43,662][105620] Updated weights for policy 1, policy_version 1689760 (0.0005) [2023-12-27 03:40:43,717][105620] Updated weights for policy 1, policy_version 1689770 (0.0005) [2023-12-27 03:40:43,829][105692] Updated weights for policy 0, policy_version 1686213 (0.0006) [2023-12-27 03:40:43,878][105692] Updated weights for policy 0, policy_version 1686223 (0.0009) [2023-12-27 03:40:43,924][105692] Updated weights for policy 0, policy_version 1686233 (0.0011) [2023-12-27 03:40:44,386][105620] Updated weights for policy 1, policy_version 1689780 (0.0005) [2023-12-27 03:40:44,440][105620] Updated weights for policy 1, policy_version 1689790 (0.0005) [2023-12-27 03:40:44,498][105620] Updated weights for policy 1, policy_version 1689800 (0.0005) [2023-12-27 03:40:44,621][105692] Updated weights for policy 0, policy_version 1686243 (0.0009) [2023-12-27 03:40:44,689][105692] Updated weights for policy 0, policy_version 1686253 (0.0006) [2023-12-27 03:40:44,741][105692] Updated weights for policy 0, policy_version 1686263 (0.0005) [2023-12-27 03:40:45,315][105620] Updated weights for policy 1, policy_version 1689810 (0.0008) [2023-12-27 03:40:45,354][105692] Updated weights for policy 0, policy_version 1686273 (0.0009) [2023-12-27 03:40:45,377][105620] Updated weights for policy 1, policy_version 1689820 (0.0007) [2023-12-27 03:40:45,407][105692] Updated weights for policy 0, policy_version 1686283 (0.0010) [2023-12-27 03:40:45,438][105620] Updated weights for policy 1, policy_version 1689830 (0.0007) [2023-12-27 03:40:45,469][105692] Updated weights for policy 0, policy_version 1686293 (0.0008) [2023-12-27 03:40:45,504][105620] Updated weights for policy 1, policy_version 1689840 (0.0008) [2023-12-27 03:40:45,522][105692] Updated weights for policy 0, policy_version 1686303 (0.0007) [2023-12-27 03:40:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 864419840. Throughput: 0: 9546.7, 1: 9896.9. Samples: 864392164. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:40:46,062][104569] Avg episode reward: [(0, '8623.023'), (1, '8898.180')] [2023-12-27 03:40:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001686304_431759360.pth... [2023-12-27 03:40:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001689840_432660480.pth... [2023-12-27 03:40:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001688656_432357376.pth [2023-12-27 03:40:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001685216_431480832.pth [2023-12-27 03:40:46,217][105692] Updated weights for policy 0, policy_version 1686313 (0.0010) [2023-12-27 03:40:46,235][105620] Updated weights for policy 1, policy_version 1689850 (0.0005) [2023-12-27 03:40:46,272][105692] Updated weights for policy 0, policy_version 1686323 (0.0010) [2023-12-27 03:40:46,295][105620] Updated weights for policy 1, policy_version 1689860 (0.0006) [2023-12-27 03:40:46,320][105692] Updated weights for policy 0, policy_version 1686333 (0.0010) [2023-12-27 03:40:46,351][105620] Updated weights for policy 1, policy_version 1689870 (0.0007) [2023-12-27 03:40:47,018][105620] Updated weights for policy 1, policy_version 1689880 (0.0010) [2023-12-27 03:40:47,069][105620] Updated weights for policy 1, policy_version 1689890 (0.0010) [2023-12-27 03:40:47,072][105692] Updated weights for policy 0, policy_version 1686343 (0.0010) [2023-12-27 03:40:47,117][105620] Updated weights for policy 1, policy_version 1689900 (0.0010) [2023-12-27 03:40:47,120][105692] Updated weights for policy 0, policy_version 1686353 (0.0010) [2023-12-27 03:40:47,164][105692] Updated weights for policy 0, policy_version 1686363 (0.0010) [2023-12-27 03:40:47,884][105620] Updated weights for policy 1, policy_version 1689910 (0.0007) [2023-12-27 03:40:47,931][105692] Updated weights for policy 0, policy_version 1686373 (0.0010) [2023-12-27 03:40:47,944][105620] Updated weights for policy 1, policy_version 1689920 (0.0007) [2023-12-27 03:40:47,995][105692] Updated weights for policy 0, policy_version 1686383 (0.0010) [2023-12-27 03:40:48,005][105620] Updated weights for policy 1, policy_version 1689930 (0.0008) [2023-12-27 03:40:48,057][105692] Updated weights for policy 0, policy_version 1686393 (0.0010) [2023-12-27 03:40:48,630][105620] Updated weights for policy 1, policy_version 1689940 (0.0008) [2023-12-27 03:40:48,693][105620] Updated weights for policy 1, policy_version 1689950 (0.0011) [2023-12-27 03:40:48,763][105620] Updated weights for policy 1, policy_version 1689960 (0.0011) [2023-12-27 03:40:48,770][105692] Updated weights for policy 0, policy_version 1686403 (0.0010) [2023-12-27 03:40:48,822][105692] Updated weights for policy 0, policy_version 1686413 (0.0010) [2023-12-27 03:40:48,878][105692] Updated weights for policy 0, policy_version 1686423 (0.0011) [2023-12-27 03:40:49,394][105620] Updated weights for policy 1, policy_version 1689970 (0.0010) [2023-12-27 03:40:49,459][105620] Updated weights for policy 1, policy_version 1689980 (0.0008) [2023-12-27 03:40:49,510][105620] Updated weights for policy 1, policy_version 1689990 (0.0008) [2023-12-27 03:40:49,568][105620] Updated weights for policy 1, policy_version 1690000 (0.0007) [2023-12-27 03:40:49,621][105692] Updated weights for policy 0, policy_version 1686433 (0.0010) [2023-12-27 03:40:49,681][105692] Updated weights for policy 0, policy_version 1686443 (0.0008) [2023-12-27 03:40:49,732][105692] Updated weights for policy 0, policy_version 1686453 (0.0008) [2023-12-27 03:40:49,791][105692] Updated weights for policy 0, policy_version 1686463 (0.0008) [2023-12-27 03:40:50,320][105620] Updated weights for policy 1, policy_version 1690010 (0.0010) [2023-12-27 03:40:50,375][105620] Updated weights for policy 1, policy_version 1690020 (0.0009) [2023-12-27 03:40:50,435][105620] Updated weights for policy 1, policy_version 1690030 (0.0010) [2023-12-27 03:40:50,554][105692] Updated weights for policy 0, policy_version 1686473 (0.0005) [2023-12-27 03:40:50,614][105692] Updated weights for policy 0, policy_version 1686483 (0.0009) [2023-12-27 03:40:50,670][105692] Updated weights for policy 0, policy_version 1686493 (0.0009) [2023-12-27 03:40:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 864518144. Throughput: 0: 9621.7, 1: 9953.8. Samples: 864510208. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:40:51,063][104569] Avg episode reward: [(0, '8802.377'), (1, '8623.607')] [2023-12-27 03:40:51,114][105620] Updated weights for policy 1, policy_version 1690040 (0.0010) [2023-12-27 03:40:51,175][105620] Updated weights for policy 1, policy_version 1690050 (0.0011) [2023-12-27 03:40:51,235][105620] Updated weights for policy 1, policy_version 1690060 (0.0010) [2023-12-27 03:40:51,385][105692] Updated weights for policy 0, policy_version 1686503 (0.0009) [2023-12-27 03:40:51,445][105692] Updated weights for policy 0, policy_version 1686513 (0.0006) [2023-12-27 03:40:51,503][105692] Updated weights for policy 0, policy_version 1686523 (0.0006) [2023-12-27 03:40:51,987][105620] Updated weights for policy 1, policy_version 1690070 (0.0008) [2023-12-27 03:40:52,045][105620] Updated weights for policy 1, policy_version 1690080 (0.0006) [2023-12-27 03:40:52,104][105620] Updated weights for policy 1, policy_version 1690090 (0.0008) [2023-12-27 03:40:52,256][105692] Updated weights for policy 0, policy_version 1686533 (0.0009) [2023-12-27 03:40:52,326][105692] Updated weights for policy 0, policy_version 1686543 (0.0009) [2023-12-27 03:40:52,390][105692] Updated weights for policy 0, policy_version 1686553 (0.0009) [2023-12-27 03:40:52,792][105620] Updated weights for policy 1, policy_version 1690100 (0.0009) [2023-12-27 03:40:52,856][105620] Updated weights for policy 1, policy_version 1690110 (0.0009) [2023-12-27 03:40:52,921][105620] Updated weights for policy 1, policy_version 1690120 (0.0010) [2023-12-27 03:40:53,198][105692] Updated weights for policy 0, policy_version 1686563 (0.0010) [2023-12-27 03:40:53,257][105692] Updated weights for policy 0, policy_version 1686573 (0.0014) [2023-12-27 03:40:53,315][105692] Updated weights for policy 0, policy_version 1686584 (0.0009) [2023-12-27 03:40:53,527][105620] Updated weights for policy 1, policy_version 1690130 (0.0008) [2023-12-27 03:40:53,586][105620] Updated weights for policy 1, policy_version 1690140 (0.0006) [2023-12-27 03:40:53,658][105620] Updated weights for policy 1, policy_version 1690150 (0.0008) [2023-12-27 03:40:53,715][105620] Updated weights for policy 1, policy_version 1690160 (0.0010) [2023-12-27 03:40:54,185][105692] Updated weights for policy 0, policy_version 1686594 (0.0010) [2023-12-27 03:40:54,251][105692] Updated weights for policy 0, policy_version 1686604 (0.0009) [2023-12-27 03:40:54,294][105620] Updated weights for policy 1, policy_version 1690170 (0.0005) [2023-12-27 03:40:54,300][105692] Updated weights for policy 0, policy_version 1686614 (0.0009) [2023-12-27 03:40:54,345][105620] Updated weights for policy 1, policy_version 1690180 (0.0005) [2023-12-27 03:40:54,361][105692] Updated weights for policy 0, policy_version 1686624 (0.0009) [2023-12-27 03:40:54,407][105620] Updated weights for policy 1, policy_version 1690190 (0.0006) [2023-12-27 03:40:55,098][105620] Updated weights for policy 1, policy_version 1690200 (0.0010) [2023-12-27 03:40:55,135][105692] Updated weights for policy 0, policy_version 1686634 (0.0007) [2023-12-27 03:40:55,156][105620] Updated weights for policy 1, policy_version 1690210 (0.0010) [2023-12-27 03:40:55,188][105692] Updated weights for policy 0, policy_version 1686644 (0.0009) [2023-12-27 03:40:55,223][105620] Updated weights for policy 1, policy_version 1690220 (0.0010) [2023-12-27 03:40:55,250][105692] Updated weights for policy 0, policy_version 1686654 (0.0009) [2023-12-27 03:40:55,830][105620] Updated weights for policy 1, policy_version 1690230 (0.0006) [2023-12-27 03:40:55,888][105620] Updated weights for policy 1, policy_version 1690240 (0.0005) [2023-12-27 03:40:55,946][105620] Updated weights for policy 1, policy_version 1690250 (0.0008) [2023-12-27 03:40:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 864616448. Throughput: 0: 9447.4, 1: 10068.3. Samples: 864625452. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:40:56,063][104569] Avg episode reward: [(0, '8803.491'), (1, '8717.869')] [2023-12-27 03:40:56,116][105692] Updated weights for policy 0, policy_version 1686664 (0.0009) [2023-12-27 03:40:56,188][105692] Updated weights for policy 0, policy_version 1686674 (0.0010) [2023-12-27 03:40:56,259][105692] Updated weights for policy 0, policy_version 1686684 (0.0009) [2023-12-27 03:40:56,482][105620] Updated weights for policy 1, policy_version 1690260 (0.0008) [2023-12-27 03:40:56,538][105620] Updated weights for policy 1, policy_version 1690270 (0.0005) [2023-12-27 03:40:56,595][105620] Updated weights for policy 1, policy_version 1690280 (0.0005) [2023-12-27 03:40:57,131][105692] Updated weights for policy 0, policy_version 1686694 (0.0010) [2023-12-27 03:40:57,141][105620] Updated weights for policy 1, policy_version 1690290 (0.0005) [2023-12-27 03:40:57,185][105620] Updated weights for policy 1, policy_version 1690300 (0.0005) [2023-12-27 03:40:57,185][105692] Updated weights for policy 0, policy_version 1686704 (0.0009) [2023-12-27 03:40:57,235][105620] Updated weights for policy 1, policy_version 1690310 (0.0005) [2023-12-27 03:40:57,237][105692] Updated weights for policy 0, policy_version 1686714 (0.0008) [2023-12-27 03:40:57,279][105620] Updated weights for policy 1, policy_version 1690320 (0.0006) [2023-12-27 03:40:57,887][105620] Updated weights for policy 1, policy_version 1690330 (0.0008) [2023-12-27 03:40:57,947][105620] Updated weights for policy 1, policy_version 1690340 (0.0010) [2023-12-27 03:40:58,002][105620] Updated weights for policy 1, policy_version 1690350 (0.0010) [2023-12-27 03:40:58,086][105692] Updated weights for policy 0, policy_version 1686724 (0.0009) [2023-12-27 03:40:58,140][105692] Updated weights for policy 0, policy_version 1686734 (0.0010) [2023-12-27 03:40:58,204][105692] Updated weights for policy 0, policy_version 1686744 (0.0007) [2023-12-27 03:40:58,742][105620] Updated weights for policy 1, policy_version 1690360 (0.0009) [2023-12-27 03:40:58,812][105620] Updated weights for policy 1, policy_version 1690370 (0.0008) [2023-12-27 03:40:58,875][105620] Updated weights for policy 1, policy_version 1690380 (0.0007) [2023-12-27 03:40:59,054][105692] Updated weights for policy 0, policy_version 1686754 (0.0007) [2023-12-27 03:40:59,114][105692] Updated weights for policy 0, policy_version 1686764 (0.0007) [2023-12-27 03:40:59,174][105692] Updated weights for policy 0, policy_version 1686774 (0.0008) [2023-12-27 03:40:59,238][105692] Updated weights for policy 0, policy_version 1686784 (0.0010) [2023-12-27 03:40:59,674][105620] Updated weights for policy 1, policy_version 1690390 (0.0009) [2023-12-27 03:40:59,732][105620] Updated weights for policy 1, policy_version 1690400 (0.0010) [2023-12-27 03:40:59,800][105620] Updated weights for policy 1, policy_version 1690410 (0.0010) [2023-12-27 03:40:59,970][105692] Updated weights for policy 0, policy_version 1686794 (0.0009) [2023-12-27 03:41:00,032][105692] Updated weights for policy 0, policy_version 1686804 (0.0007) [2023-12-27 03:41:00,084][105692] Updated weights for policy 0, policy_version 1686814 (0.0006) [2023-12-27 03:41:00,512][105620] Updated weights for policy 1, policy_version 1690420 (0.0011) [2023-12-27 03:41:00,566][105620] Updated weights for policy 1, policy_version 1690430 (0.0010) [2023-12-27 03:41:00,618][105620] Updated weights for policy 1, policy_version 1690440 (0.0010) [2023-12-27 03:41:00,716][105692] Updated weights for policy 0, policy_version 1686824 (0.0010) [2023-12-27 03:41:00,767][105692] Updated weights for policy 0, policy_version 1686834 (0.0010) [2023-12-27 03:41:00,817][105692] Updated weights for policy 0, policy_version 1686844 (0.0010) [2023-12-27 03:41:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 864714752. Throughput: 0: 9347.5, 1: 10212.2. Samples: 864683128. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:41:01,063][104569] Avg episode reward: [(0, '8348.568'), (1, '8624.941')] [2023-12-27 03:41:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001686848_431898624.pth... [2023-12-27 03:41:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001690448_432816128.pth... [2023-12-27 03:41:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001685728_431611904.pth [2023-12-27 03:41:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001689264_432513024.pth [2023-12-27 03:41:01,318][105620] Updated weights for policy 1, policy_version 1690450 (0.0010) [2023-12-27 03:41:01,381][105620] Updated weights for policy 1, policy_version 1690460 (0.0009) [2023-12-27 03:41:01,435][105620] Updated weights for policy 1, policy_version 1690470 (0.0008) [2023-12-27 03:41:01,493][105620] Updated weights for policy 1, policy_version 1690480 (0.0009) [2023-12-27 03:41:01,541][105692] Updated weights for policy 0, policy_version 1686854 (0.0010) [2023-12-27 03:41:01,601][105692] Updated weights for policy 0, policy_version 1686864 (0.0009) [2023-12-27 03:41:01,668][105692] Updated weights for policy 0, policy_version 1686874 (0.0009) [2023-12-27 03:41:02,279][105620] Updated weights for policy 1, policy_version 1690490 (0.0011) [2023-12-27 03:41:02,345][105620] Updated weights for policy 1, policy_version 1690500 (0.0011) [2023-12-27 03:41:02,387][105692] Updated weights for policy 0, policy_version 1686884 (0.0009) [2023-12-27 03:41:02,415][105620] Updated weights for policy 1, policy_version 1690510 (0.0010) [2023-12-27 03:41:02,449][105692] Updated weights for policy 0, policy_version 1686894 (0.0010) [2023-12-27 03:41:02,494][105692] Updated weights for policy 0, policy_version 1686904 (0.0008) [2023-12-27 03:41:03,131][105620] Updated weights for policy 1, policy_version 1690520 (0.0009) [2023-12-27 03:41:03,191][105620] Updated weights for policy 1, policy_version 1690530 (0.0006) [2023-12-27 03:41:03,247][105620] Updated weights for policy 1, policy_version 1690540 (0.0005) [2023-12-27 03:41:03,264][105692] Updated weights for policy 0, policy_version 1686914 (0.0008) [2023-12-27 03:41:03,316][105692] Updated weights for policy 0, policy_version 1686924 (0.0009) [2023-12-27 03:41:03,372][105692] Updated weights for policy 0, policy_version 1686934 (0.0010) [2023-12-27 03:41:03,429][105692] Updated weights for policy 0, policy_version 1686944 (0.0010) [2023-12-27 03:41:03,903][105620] Updated weights for policy 1, policy_version 1690550 (0.0008) [2023-12-27 03:41:03,967][105620] Updated weights for policy 1, policy_version 1690560 (0.0010) [2023-12-27 03:41:04,023][105620] Updated weights for policy 1, policy_version 1690570 (0.0009) [2023-12-27 03:41:04,222][105692] Updated weights for policy 0, policy_version 1686954 (0.0008) [2023-12-27 03:41:04,283][105692] Updated weights for policy 0, policy_version 1686964 (0.0009) [2023-12-27 03:41:04,349][105692] Updated weights for policy 0, policy_version 1686974 (0.0010) [2023-12-27 03:41:04,739][105620] Updated weights for policy 1, policy_version 1690580 (0.0009) [2023-12-27 03:41:04,794][105620] Updated weights for policy 1, policy_version 1690590 (0.0010) [2023-12-27 03:41:04,856][105620] Updated weights for policy 1, policy_version 1690600 (0.0010) [2023-12-27 03:41:05,113][105692] Updated weights for policy 0, policy_version 1686984 (0.0009) [2023-12-27 03:41:05,159][105692] Updated weights for policy 0, policy_version 1686994 (0.0008) [2023-12-27 03:41:05,207][105692] Updated weights for policy 0, policy_version 1687004 (0.0007) [2023-12-27 03:41:05,571][105620] Updated weights for policy 1, policy_version 1690610 (0.0010) [2023-12-27 03:41:05,619][105620] Updated weights for policy 1, policy_version 1690620 (0.0010) [2023-12-27 03:41:05,664][105620] Updated weights for policy 1, policy_version 1690630 (0.0010) [2023-12-27 03:41:05,712][105620] Updated weights for policy 1, policy_version 1690640 (0.0010) [2023-12-27 03:41:06,010][105692] Updated weights for policy 0, policy_version 1687014 (0.0009) [2023-12-27 03:41:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 864804864. Throughput: 0: 9277.6, 1: 10148.4. Samples: 864797816. Policy #0 lag: (min: 26.0, avg: 48.1, max: 58.0) [2023-12-27 03:41:06,062][104569] Avg episode reward: [(0, '8167.413'), (1, '8714.633')] [2023-12-27 03:41:06,068][105692] Updated weights for policy 0, policy_version 1687024 (0.0008) [2023-12-27 03:41:06,136][105692] Updated weights for policy 0, policy_version 1687034 (0.0008) [2023-12-27 03:41:06,443][105620] Updated weights for policy 1, policy_version 1690650 (0.0010) [2023-12-27 03:41:06,499][105620] Updated weights for policy 1, policy_version 1690660 (0.0010) [2023-12-27 03:41:06,556][105620] Updated weights for policy 1, policy_version 1690670 (0.0011) [2023-12-27 03:41:06,879][105692] Updated weights for policy 0, policy_version 1687044 (0.0009) [2023-12-27 03:41:06,931][105692] Updated weights for policy 0, policy_version 1687054 (0.0008) [2023-12-27 03:41:06,990][105692] Updated weights for policy 0, policy_version 1687064 (0.0008) [2023-12-27 03:41:07,297][105620] Updated weights for policy 1, policy_version 1690680 (0.0008) [2023-12-27 03:41:07,364][105620] Updated weights for policy 1, policy_version 1690690 (0.0005) [2023-12-27 03:41:07,432][105620] Updated weights for policy 1, policy_version 1690700 (0.0008) [2023-12-27 03:41:07,833][105692] Updated weights for policy 0, policy_version 1687074 (0.0009) [2023-12-27 03:41:07,892][105692] Updated weights for policy 0, policy_version 1687084 (0.0009) [2023-12-27 03:41:07,952][105692] Updated weights for policy 0, policy_version 1687094 (0.0010) [2023-12-27 03:41:07,983][105620] Updated weights for policy 1, policy_version 1690710 (0.0007) [2023-12-27 03:41:08,000][105692] Updated weights for policy 0, policy_version 1687104 (0.0007) [2023-12-27 03:41:08,038][105620] Updated weights for policy 1, policy_version 1690720 (0.0010) [2023-12-27 03:41:08,103][105620] Updated weights for policy 1, policy_version 1690730 (0.0010) [2023-12-27 03:41:08,685][105692] Updated weights for policy 0, policy_version 1687114 (0.0006) [2023-12-27 03:41:08,737][105620] Updated weights for policy 1, policy_version 1690740 (0.0008) [2023-12-27 03:41:08,747][105692] Updated weights for policy 0, policy_version 1687124 (0.0007) [2023-12-27 03:41:08,797][105620] Updated weights for policy 1, policy_version 1690750 (0.0010) [2023-12-27 03:41:08,800][105692] Updated weights for policy 0, policy_version 1687134 (0.0006) [2023-12-27 03:41:08,853][105620] Updated weights for policy 1, policy_version 1690760 (0.0010) [2023-12-27 03:41:09,444][105692] Updated weights for policy 0, policy_version 1687144 (0.0008) [2023-12-27 03:41:09,504][105692] Updated weights for policy 0, policy_version 1687154 (0.0007) [2023-12-27 03:41:09,572][105692] Updated weights for policy 0, policy_version 1687164 (0.0008) [2023-12-27 03:41:09,581][105620] Updated weights for policy 1, policy_version 1690770 (0.0010) [2023-12-27 03:41:09,646][105620] Updated weights for policy 1, policy_version 1690780 (0.0009) [2023-12-27 03:41:09,704][105620] Updated weights for policy 1, policy_version 1690790 (0.0010) [2023-12-27 03:41:09,758][105620] Updated weights for policy 1, policy_version 1690800 (0.0010) [2023-12-27 03:41:10,282][105692] Updated weights for policy 0, policy_version 1687174 (0.0009) [2023-12-27 03:41:10,337][105692] Updated weights for policy 0, policy_version 1687184 (0.0009) [2023-12-27 03:41:10,392][105692] Updated weights for policy 0, policy_version 1687194 (0.0009) [2023-12-27 03:41:10,508][105620] Updated weights for policy 1, policy_version 1690810 (0.0008) [2023-12-27 03:41:10,570][105620] Updated weights for policy 1, policy_version 1690820 (0.0008) [2023-12-27 03:41:10,629][105620] Updated weights for policy 1, policy_version 1690830 (0.0005) [2023-12-27 03:41:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 864903168. Throughput: 0: 9342.6, 1: 10108.2. Samples: 864914960. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:41:11,063][104569] Avg episode reward: [(0, '8257.160'), (1, '8899.990')] [2023-12-27 03:41:11,221][105692] Updated weights for policy 0, policy_version 1687204 (0.0008) [2023-12-27 03:41:11,271][105620] Updated weights for policy 1, policy_version 1690840 (0.0006) [2023-12-27 03:41:11,277][105692] Updated weights for policy 0, policy_version 1687214 (0.0008) [2023-12-27 03:41:11,327][105692] Updated weights for policy 0, policy_version 1687224 (0.0008) [2023-12-27 03:41:11,342][105620] Updated weights for policy 1, policy_version 1690850 (0.0006) [2023-12-27 03:41:11,412][105620] Updated weights for policy 1, policy_version 1690860 (0.0008) [2023-12-27 03:41:12,049][105692] Updated weights for policy 0, policy_version 1687234 (0.0009) [2023-12-27 03:41:12,074][105620] Updated weights for policy 1, policy_version 1690870 (0.0006) [2023-12-27 03:41:12,111][105692] Updated weights for policy 0, policy_version 1687244 (0.0006) [2023-12-27 03:41:12,126][105620] Updated weights for policy 1, policy_version 1690880 (0.0009) [2023-12-27 03:41:12,168][105692] Updated weights for policy 0, policy_version 1687254 (0.0005) [2023-12-27 03:41:12,187][105620] Updated weights for policy 1, policy_version 1690890 (0.0007) [2023-12-27 03:41:12,231][105692] Updated weights for policy 0, policy_version 1687264 (0.0008) [2023-12-27 03:41:12,894][105620] Updated weights for policy 1, policy_version 1690900 (0.0006) [2023-12-27 03:41:12,928][105692] Updated weights for policy 0, policy_version 1687274 (0.0009) [2023-12-27 03:41:12,949][105620] Updated weights for policy 1, policy_version 1690910 (0.0007) [2023-12-27 03:41:12,984][105692] Updated weights for policy 0, policy_version 1687284 (0.0009) [2023-12-27 03:41:13,001][105620] Updated weights for policy 1, policy_version 1690920 (0.0007) [2023-12-27 03:41:13,037][105692] Updated weights for policy 0, policy_version 1687294 (0.0009) [2023-12-27 03:41:13,743][105692] Updated weights for policy 0, policy_version 1687304 (0.0009) [2023-12-27 03:41:13,797][105692] Updated weights for policy 0, policy_version 1687314 (0.0009) [2023-12-27 03:41:13,806][105620] Updated weights for policy 1, policy_version 1690930 (0.0007) [2023-12-27 03:41:13,853][105692] Updated weights for policy 0, policy_version 1687324 (0.0007) [2023-12-27 03:41:13,855][105620] Updated weights for policy 1, policy_version 1690940 (0.0006) [2023-12-27 03:41:13,906][105620] Updated weights for policy 1, policy_version 1690950 (0.0009) [2023-12-27 03:41:13,961][105620] Updated weights for policy 1, policy_version 1690960 (0.0010) [2023-12-27 03:41:14,487][105692] Updated weights for policy 0, policy_version 1687334 (0.0008) [2023-12-27 03:41:14,535][105692] Updated weights for policy 0, policy_version 1687344 (0.0009) [2023-12-27 03:41:14,588][105692] Updated weights for policy 0, policy_version 1687354 (0.0008) [2023-12-27 03:41:14,781][105620] Updated weights for policy 1, policy_version 1690970 (0.0008) [2023-12-27 03:41:14,845][105620] Updated weights for policy 1, policy_version 1690980 (0.0008) [2023-12-27 03:41:14,909][105620] Updated weights for policy 1, policy_version 1690990 (0.0008) [2023-12-27 03:41:15,400][105692] Updated weights for policy 0, policy_version 1687364 (0.0010) [2023-12-27 03:41:15,449][105692] Updated weights for policy 0, policy_version 1687374 (0.0010) [2023-12-27 03:41:15,511][105692] Updated weights for policy 0, policy_version 1687384 (0.0011) [2023-12-27 03:41:15,667][105620] Updated weights for policy 1, policy_version 1691000 (0.0008) [2023-12-27 03:41:15,720][105620] Updated weights for policy 1, policy_version 1691010 (0.0008) [2023-12-27 03:41:15,774][105620] Updated weights for policy 1, policy_version 1691020 (0.0008) [2023-12-27 03:41:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 865001472. Throughput: 0: 9393.6, 1: 10003.1. Samples: 864971936. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:41:16,063][104569] Avg episode reward: [(0, '8349.296'), (1, '9083.631')] [2023-12-27 03:41:16,074][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001687392_432037888.pth... [2023-12-27 03:41:16,075][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001691024_432963584.pth... [2023-12-27 03:41:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001686304_431759360.pth [2023-12-27 03:41:16,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001689840_432660480.pth [2023-12-27 03:41:16,232][105692] Updated weights for policy 0, policy_version 1687394 (0.0010) [2023-12-27 03:41:16,277][105692] Updated weights for policy 0, policy_version 1687404 (0.0005) [2023-12-27 03:41:16,328][105692] Updated weights for policy 0, policy_version 1687414 (0.0005) [2023-12-27 03:41:16,372][105692] Updated weights for policy 0, policy_version 1687424 (0.0005) [2023-12-27 03:41:16,557][105620] Updated weights for policy 1, policy_version 1691030 (0.0009) [2023-12-27 03:41:16,612][105620] Updated weights for policy 1, policy_version 1691040 (0.0009) [2023-12-27 03:41:16,670][105620] Updated weights for policy 1, policy_version 1691050 (0.0009) [2023-12-27 03:41:17,045][105692] Updated weights for policy 0, policy_version 1687434 (0.0009) [2023-12-27 03:41:17,103][105692] Updated weights for policy 0, policy_version 1687444 (0.0009) [2023-12-27 03:41:17,162][105692] Updated weights for policy 0, policy_version 1687454 (0.0009) [2023-12-27 03:41:17,460][105620] Updated weights for policy 1, policy_version 1691060 (0.0009) [2023-12-27 03:41:17,507][105620] Updated weights for policy 1, policy_version 1691070 (0.0008) [2023-12-27 03:41:17,555][105620] Updated weights for policy 1, policy_version 1691080 (0.0009) [2023-12-27 03:41:17,909][105692] Updated weights for policy 0, policy_version 1687464 (0.0009) [2023-12-27 03:41:17,966][105692] Updated weights for policy 0, policy_version 1687474 (0.0009) [2023-12-27 03:41:18,029][105692] Updated weights for policy 0, policy_version 1687484 (0.0008) [2023-12-27 03:41:18,283][105620] Updated weights for policy 1, policy_version 1691090 (0.0008) [2023-12-27 03:41:18,343][105620] Updated weights for policy 1, policy_version 1691100 (0.0007) [2023-12-27 03:41:18,388][105620] Updated weights for policy 1, policy_version 1691110 (0.0007) [2023-12-27 03:41:18,443][105620] Updated weights for policy 1, policy_version 1691120 (0.0008) [2023-12-27 03:41:18,817][105692] Updated weights for policy 0, policy_version 1687494 (0.0009) [2023-12-27 03:41:18,882][105692] Updated weights for policy 0, policy_version 1687504 (0.0009) [2023-12-27 03:41:18,936][105692] Updated weights for policy 0, policy_version 1687514 (0.0008) [2023-12-27 03:41:19,170][105620] Updated weights for policy 1, policy_version 1691130 (0.0006) [2023-12-27 03:41:19,229][105620] Updated weights for policy 1, policy_version 1691140 (0.0009) [2023-12-27 03:41:19,299][105620] Updated weights for policy 1, policy_version 1691150 (0.0006) [2023-12-27 03:41:19,744][105692] Updated weights for policy 0, policy_version 1687524 (0.0009) [2023-12-27 03:41:19,803][105692] Updated weights for policy 0, policy_version 1687534 (0.0009) [2023-12-27 03:41:19,872][105692] Updated weights for policy 0, policy_version 1687544 (0.0008) [2023-12-27 03:41:20,023][105620] Updated weights for policy 1, policy_version 1691160 (0.0009) [2023-12-27 03:41:20,079][105620] Updated weights for policy 1, policy_version 1691170 (0.0009) [2023-12-27 03:41:20,138][105620] Updated weights for policy 1, policy_version 1691180 (0.0009) [2023-12-27 03:41:20,632][105692] Updated weights for policy 0, policy_version 1687554 (0.0008) [2023-12-27 03:41:20,692][105692] Updated weights for policy 0, policy_version 1687564 (0.0008) [2023-12-27 03:41:20,758][105692] Updated weights for policy 0, policy_version 1687574 (0.0009) [2023-12-27 03:41:20,827][105692] Updated weights for policy 0, policy_version 1687584 (0.0009) [2023-12-27 03:41:20,913][105620] Updated weights for policy 1, policy_version 1691190 (0.0009) [2023-12-27 03:41:20,972][105620] Updated weights for policy 1, policy_version 1691200 (0.0009) [2023-12-27 03:41:21,028][105620] Updated weights for policy 1, policy_version 1691210 (0.0009) [2023-12-27 03:41:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 865091584. Throughput: 0: 9461.8, 1: 9853.7. Samples: 865085104. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:41:21,062][104569] Avg episode reward: [(0, '8261.111'), (1, '9264.410')] [2023-12-27 03:41:21,585][105692] Updated weights for policy 0, policy_version 1687594 (0.0008) [2023-12-27 03:41:21,650][105692] Updated weights for policy 0, policy_version 1687604 (0.0008) [2023-12-27 03:41:21,715][105692] Updated weights for policy 0, policy_version 1687614 (0.0009) [2023-12-27 03:41:21,865][105620] Updated weights for policy 1, policy_version 1691220 (0.0009) [2023-12-27 03:41:21,921][105620] Updated weights for policy 1, policy_version 1691230 (0.0010) [2023-12-27 03:41:21,976][105620] Updated weights for policy 1, policy_version 1691240 (0.0009) [2023-12-27 03:41:22,495][105692] Updated weights for policy 0, policy_version 1687624 (0.0008) [2023-12-27 03:41:22,553][105692] Updated weights for policy 0, policy_version 1687634 (0.0009) [2023-12-27 03:41:22,613][105692] Updated weights for policy 0, policy_version 1687644 (0.0008) [2023-12-27 03:41:22,757][105620] Updated weights for policy 1, policy_version 1691250 (0.0010) [2023-12-27 03:41:22,819][105620] Updated weights for policy 1, policy_version 1691260 (0.0011) [2023-12-27 03:41:22,883][105620] Updated weights for policy 1, policy_version 1691270 (0.0010) [2023-12-27 03:41:22,943][105620] Updated weights for policy 1, policy_version 1691280 (0.0011) [2023-12-27 03:41:23,396][105692] Updated weights for policy 0, policy_version 1687654 (0.0009) [2023-12-27 03:41:23,449][105692] Updated weights for policy 0, policy_version 1687665 (0.0010) [2023-12-27 03:41:23,506][105692] Updated weights for policy 0, policy_version 1687676 (0.0010) [2023-12-27 03:41:23,558][105620] Updated weights for policy 1, policy_version 1691290 (0.0005) [2023-12-27 03:41:23,619][105620] Updated weights for policy 1, policy_version 1691300 (0.0005) [2023-12-27 03:41:23,677][105620] Updated weights for policy 1, policy_version 1691310 (0.0005) [2023-12-27 03:41:24,218][105620] Updated weights for policy 1, policy_version 1691320 (0.0009) [2023-12-27 03:41:24,265][105620] Updated weights for policy 1, policy_version 1691330 (0.0008) [2023-12-27 03:41:24,306][105692] Updated weights for policy 0, policy_version 1687686 (0.0009) [2023-12-27 03:41:24,316][105620] Updated weights for policy 1, policy_version 1691340 (0.0009) [2023-12-27 03:41:24,371][105692] Updated weights for policy 0, policy_version 1687696 (0.0008) [2023-12-27 03:41:24,432][105692] Updated weights for policy 0, policy_version 1687706 (0.0009) [2023-12-27 03:41:25,088][105620] Updated weights for policy 1, policy_version 1691350 (0.0008) [2023-12-27 03:41:25,136][105620] Updated weights for policy 1, policy_version 1691360 (0.0009) [2023-12-27 03:41:25,182][105692] Updated weights for policy 0, policy_version 1687716 (0.0009) [2023-12-27 03:41:25,188][105620] Updated weights for policy 1, policy_version 1691370 (0.0009) [2023-12-27 03:41:25,241][105692] Updated weights for policy 0, policy_version 1687726 (0.0007) [2023-12-27 03:41:25,302][105692] Updated weights for policy 0, policy_version 1687736 (0.0009) [2023-12-27 03:41:25,907][105620] Updated weights for policy 1, policy_version 1691380 (0.0007) [2023-12-27 03:41:25,962][105620] Updated weights for policy 1, policy_version 1691390 (0.0007) [2023-12-27 03:41:26,009][105620] Updated weights for policy 1, policy_version 1691400 (0.0008) [2023-12-27 03:41:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.3, 300 sec: 19466.4). Total num frames: 865189888. Throughput: 0: 9320.0, 1: 9827.3. Samples: 865197052. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:41:26,063][104569] Avg episode reward: [(0, '8348.840'), (1, '9264.402')] [2023-12-27 03:41:26,071][105692] Updated weights for policy 0, policy_version 1687746 (0.0009) [2023-12-27 03:41:26,123][105692] Updated weights for policy 0, policy_version 1687756 (0.0008) [2023-12-27 03:41:26,169][105692] Updated weights for policy 0, policy_version 1687766 (0.0008) [2023-12-27 03:41:26,223][105692] Updated weights for policy 0, policy_version 1687776 (0.0009) [2023-12-27 03:41:26,661][105620] Updated weights for policy 1, policy_version 1691410 (0.0009) [2023-12-27 03:41:26,711][105620] Updated weights for policy 1, policy_version 1691420 (0.0009) [2023-12-27 03:41:26,762][105620] Updated weights for policy 1, policy_version 1691430 (0.0009) [2023-12-27 03:41:26,823][105620] Updated weights for policy 1, policy_version 1691440 (0.0006) [2023-12-27 03:41:27,068][105692] Updated weights for policy 0, policy_version 1687786 (0.0010) [2023-12-27 03:41:27,130][105692] Updated weights for policy 0, policy_version 1687796 (0.0010) [2023-12-27 03:41:27,197][105692] Updated weights for policy 0, policy_version 1687806 (0.0010) [2023-12-27 03:41:27,424][105620] Updated weights for policy 1, policy_version 1691450 (0.0005) [2023-12-27 03:41:27,492][105620] Updated weights for policy 1, policy_version 1691460 (0.0005) [2023-12-27 03:41:27,552][105620] Updated weights for policy 1, policy_version 1691470 (0.0008) [2023-12-27 03:41:28,045][105692] Updated weights for policy 0, policy_version 1687816 (0.0009) [2023-12-27 03:41:28,103][105692] Updated weights for policy 0, policy_version 1687826 (0.0007) [2023-12-27 03:41:28,109][105620] Updated weights for policy 1, policy_version 1691480 (0.0007) [2023-12-27 03:41:28,154][105692] Updated weights for policy 0, policy_version 1687836 (0.0006) [2023-12-27 03:41:28,164][105620] Updated weights for policy 1, policy_version 1691490 (0.0007) [2023-12-27 03:41:28,217][105620] Updated weights for policy 1, policy_version 1691500 (0.0006) [2023-12-27 03:41:28,814][105620] Updated weights for policy 1, policy_version 1691510 (0.0007) [2023-12-27 03:41:28,870][105620] Updated weights for policy 1, policy_version 1691520 (0.0008) [2023-12-27 03:41:28,921][105620] Updated weights for policy 1, policy_version 1691530 (0.0008) [2023-12-27 03:41:28,995][105692] Updated weights for policy 0, policy_version 1687846 (0.0008) [2023-12-27 03:41:29,052][105692] Updated weights for policy 0, policy_version 1687856 (0.0009) [2023-12-27 03:41:29,115][105692] Updated weights for policy 0, policy_version 1687866 (0.0009) [2023-12-27 03:41:29,672][105620] Updated weights for policy 1, policy_version 1691540 (0.0009) [2023-12-27 03:41:29,731][105620] Updated weights for policy 1, policy_version 1691550 (0.0009) [2023-12-27 03:41:29,782][105620] Updated weights for policy 1, policy_version 1691560 (0.0009) [2023-12-27 03:41:29,864][105692] Updated weights for policy 0, policy_version 1687876 (0.0008) [2023-12-27 03:41:29,918][105692] Updated weights for policy 0, policy_version 1687886 (0.0010) [2023-12-27 03:41:29,973][105692] Updated weights for policy 0, policy_version 1687896 (0.0009) [2023-12-27 03:41:30,489][105620] Updated weights for policy 1, policy_version 1691570 (0.0008) [2023-12-27 03:41:30,550][105620] Updated weights for policy 1, policy_version 1691580 (0.0006) [2023-12-27 03:41:30,611][105620] Updated weights for policy 1, policy_version 1691590 (0.0006) [2023-12-27 03:41:30,672][105620] Updated weights for policy 1, policy_version 1691600 (0.0006) [2023-12-27 03:41:30,788][105692] Updated weights for policy 0, policy_version 1687906 (0.0010) [2023-12-27 03:41:30,860][105692] Updated weights for policy 0, policy_version 1687916 (0.0009) [2023-12-27 03:41:30,918][105692] Updated weights for policy 0, policy_version 1687926 (0.0008) [2023-12-27 03:41:30,973][105692] Updated weights for policy 0, policy_version 1687936 (0.0005) [2023-12-27 03:41:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 865288192. Throughput: 0: 9250.0, 1: 9958.4. Samples: 865256544. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:41:31,063][104569] Avg episode reward: [(0, '8437.824'), (1, '8989.218')] [2023-12-27 03:41:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001687936_432177152.pth... [2023-12-27 03:41:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001691600_433111040.pth... [2023-12-27 03:41:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001686848_431898624.pth [2023-12-27 03:41:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001690448_432816128.pth [2023-12-27 03:41:31,278][105620] Updated weights for policy 1, policy_version 1691610 (0.0011) [2023-12-27 03:41:31,332][105620] Updated weights for policy 1, policy_version 1691620 (0.0009) [2023-12-27 03:41:31,401][105620] Updated weights for policy 1, policy_version 1691630 (0.0011) [2023-12-27 03:41:31,740][105692] Updated weights for policy 0, policy_version 1687946 (0.0007) [2023-12-27 03:41:31,810][105692] Updated weights for policy 0, policy_version 1687956 (0.0006) [2023-12-27 03:41:31,873][105692] Updated weights for policy 0, policy_version 1687966 (0.0007) [2023-12-27 03:41:32,069][105620] Updated weights for policy 1, policy_version 1691640 (0.0010) [2023-12-27 03:41:32,119][105620] Updated weights for policy 1, policy_version 1691650 (0.0011) [2023-12-27 03:41:32,167][105620] Updated weights for policy 1, policy_version 1691660 (0.0010) [2023-12-27 03:41:32,483][105692] Updated weights for policy 0, policy_version 1687976 (0.0009) [2023-12-27 03:41:32,542][105692] Updated weights for policy 0, policy_version 1687986 (0.0008) [2023-12-27 03:41:32,598][105692] Updated weights for policy 0, policy_version 1687996 (0.0008) [2023-12-27 03:41:32,962][105620] Updated weights for policy 1, policy_version 1691670 (0.0011) [2023-12-27 03:41:33,021][105620] Updated weights for policy 1, policy_version 1691680 (0.0011) [2023-12-27 03:41:33,080][105620] Updated weights for policy 1, policy_version 1691690 (0.0010) [2023-12-27 03:41:33,357][105692] Updated weights for policy 0, policy_version 1688006 (0.0008) [2023-12-27 03:41:33,402][105692] Updated weights for policy 0, policy_version 1688016 (0.0005) [2023-12-27 03:41:33,452][105692] Updated weights for policy 0, policy_version 1688026 (0.0005) [2023-12-27 03:41:33,819][105620] Updated weights for policy 1, policy_version 1691700 (0.0010) [2023-12-27 03:41:33,863][105620] Updated weights for policy 1, policy_version 1691710 (0.0010) [2023-12-27 03:41:33,913][105620] Updated weights for policy 1, policy_version 1691720 (0.0010) [2023-12-27 03:41:33,972][105692] Updated weights for policy 0, policy_version 1688036 (0.0005) [2023-12-27 03:41:34,027][105692] Updated weights for policy 0, policy_version 1688046 (0.0005) [2023-12-27 03:41:34,086][105692] Updated weights for policy 0, policy_version 1688056 (0.0005) [2023-12-27 03:41:34,719][105620] Updated weights for policy 1, policy_version 1691730 (0.0010) [2023-12-27 03:41:34,727][105692] Updated weights for policy 0, policy_version 1688066 (0.0007) [2023-12-27 03:41:34,776][105692] Updated weights for policy 0, policy_version 1688076 (0.0007) [2023-12-27 03:41:34,782][105620] Updated weights for policy 1, policy_version 1691740 (0.0010) [2023-12-27 03:41:34,828][105692] Updated weights for policy 0, policy_version 1688086 (0.0008) [2023-12-27 03:41:34,846][105620] Updated weights for policy 1, policy_version 1691750 (0.0010) [2023-12-27 03:41:34,884][105692] Updated weights for policy 0, policy_version 1688096 (0.0005) [2023-12-27 03:41:34,894][105620] Updated weights for policy 1, policy_version 1691760 (0.0010) [2023-12-27 03:41:35,629][105620] Updated weights for policy 1, policy_version 1691770 (0.0011) [2023-12-27 03:41:35,647][105692] Updated weights for policy 0, policy_version 1688106 (0.0007) [2023-12-27 03:41:35,673][105620] Updated weights for policy 1, policy_version 1691780 (0.0010) [2023-12-27 03:41:35,709][105692] Updated weights for policy 0, policy_version 1688116 (0.0007) [2023-12-27 03:41:35,731][105620] Updated weights for policy 1, policy_version 1691790 (0.0010) [2023-12-27 03:41:35,765][105692] Updated weights for policy 0, policy_version 1688126 (0.0006) [2023-12-27 03:41:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 865386496. Throughput: 0: 9249.1, 1: 9948.4. Samples: 865374092. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:41:36,062][104569] Avg episode reward: [(0, '8804.909'), (1, '8990.348')] [2023-12-27 03:41:36,370][105620] Updated weights for policy 1, policy_version 1691800 (0.0011) [2023-12-27 03:41:36,445][105620] Updated weights for policy 1, policy_version 1691810 (0.0007) [2023-12-27 03:41:36,511][105620] Updated weights for policy 1, policy_version 1691820 (0.0009) [2023-12-27 03:41:36,558][105692] Updated weights for policy 0, policy_version 1688136 (0.0007) [2023-12-27 03:41:36,623][105692] Updated weights for policy 0, policy_version 1688146 (0.0009) [2023-12-27 03:41:36,680][105692] Updated weights for policy 0, policy_version 1688156 (0.0008) [2023-12-27 03:41:37,208][105620] Updated weights for policy 1, policy_version 1691830 (0.0009) [2023-12-27 03:41:37,266][105620] Updated weights for policy 1, policy_version 1691840 (0.0010) [2023-12-27 03:41:37,325][105620] Updated weights for policy 1, policy_version 1691850 (0.0010) [2023-12-27 03:41:37,427][105692] Updated weights for policy 0, policy_version 1688166 (0.0008) [2023-12-27 03:41:37,490][105692] Updated weights for policy 0, policy_version 1688176 (0.0009) [2023-12-27 03:41:37,553][105692] Updated weights for policy 0, policy_version 1688186 (0.0008) [2023-12-27 03:41:38,069][105620] Updated weights for policy 1, policy_version 1691860 (0.0009) [2023-12-27 03:41:38,135][105620] Updated weights for policy 1, policy_version 1691870 (0.0006) [2023-12-27 03:41:38,186][105620] Updated weights for policy 1, policy_version 1691880 (0.0008) [2023-12-27 03:41:38,331][105692] Updated weights for policy 0, policy_version 1688196 (0.0009) [2023-12-27 03:41:38,395][105692] Updated weights for policy 0, policy_version 1688206 (0.0009) [2023-12-27 03:41:38,457][105692] Updated weights for policy 0, policy_version 1688216 (0.0009) [2023-12-27 03:41:38,810][105620] Updated weights for policy 1, policy_version 1691890 (0.0008) [2023-12-27 03:41:38,875][105620] Updated weights for policy 1, policy_version 1691900 (0.0005) [2023-12-27 03:41:38,937][105620] Updated weights for policy 1, policy_version 1691910 (0.0005) [2023-12-27 03:41:38,986][105620] Updated weights for policy 1, policy_version 1691920 (0.0005) [2023-12-27 03:41:39,310][105692] Updated weights for policy 0, policy_version 1688226 (0.0009) [2023-12-27 03:41:39,376][105692] Updated weights for policy 0, policy_version 1688236 (0.0009) [2023-12-27 03:41:39,440][105692] Updated weights for policy 0, policy_version 1688246 (0.0009) [2023-12-27 03:41:39,510][105692] Updated weights for policy 0, policy_version 1688256 (0.0010) [2023-12-27 03:41:39,614][105620] Updated weights for policy 1, policy_version 1691930 (0.0008) [2023-12-27 03:41:39,679][105620] Updated weights for policy 1, policy_version 1691940 (0.0009) [2023-12-27 03:41:39,738][105620] Updated weights for policy 1, policy_version 1691950 (0.0009) [2023-12-27 03:41:40,297][105692] Updated weights for policy 0, policy_version 1688266 (0.0009) [2023-12-27 03:41:40,366][105692] Updated weights for policy 0, policy_version 1688276 (0.0009) [2023-12-27 03:41:40,428][105692] Updated weights for policy 0, policy_version 1688286 (0.0009) [2023-12-27 03:41:40,472][105620] Updated weights for policy 1, policy_version 1691960 (0.0008) [2023-12-27 03:41:40,530][105620] Updated weights for policy 1, policy_version 1691970 (0.0009) [2023-12-27 03:41:40,591][105620] Updated weights for policy 1, policy_version 1691980 (0.0009) [2023-12-27 03:41:41,018][105692] Updated weights for policy 0, policy_version 1688296 (0.0006) [2023-12-27 03:41:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19438.7). Total num frames: 865476608. Throughput: 0: 9246.7, 1: 9902.2. Samples: 865487152. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:41:41,063][104569] Avg episode reward: [(0, '8343.986'), (1, '9172.704')] [2023-12-27 03:41:41,085][105692] Updated weights for policy 0, policy_version 1688306 (0.0008) [2023-12-27 03:41:41,154][105692] Updated weights for policy 0, policy_version 1688316 (0.0009) [2023-12-27 03:41:41,369][105620] Updated weights for policy 1, policy_version 1691990 (0.0009) [2023-12-27 03:41:41,437][105620] Updated weights for policy 1, policy_version 1692000 (0.0009) [2023-12-27 03:41:41,504][105620] Updated weights for policy 1, policy_version 1692010 (0.0009) [2023-12-27 03:41:41,957][105692] Updated weights for policy 0, policy_version 1688326 (0.0006) [2023-12-27 03:41:42,021][105692] Updated weights for policy 0, policy_version 1688336 (0.0008) [2023-12-27 03:41:42,081][105692] Updated weights for policy 0, policy_version 1688346 (0.0008) [2023-12-27 03:41:42,200][105620] Updated weights for policy 1, policy_version 1692020 (0.0009) [2023-12-27 03:41:42,262][105620] Updated weights for policy 1, policy_version 1692030 (0.0010) [2023-12-27 03:41:42,325][105620] Updated weights for policy 1, policy_version 1692040 (0.0010) [2023-12-27 03:41:42,814][105692] Updated weights for policy 0, policy_version 1688356 (0.0009) [2023-12-27 03:41:42,871][105692] Updated weights for policy 0, policy_version 1688366 (0.0007) [2023-12-27 03:41:42,936][105692] Updated weights for policy 0, policy_version 1688376 (0.0007) [2023-12-27 03:41:43,085][105620] Updated weights for policy 1, policy_version 1692050 (0.0009) [2023-12-27 03:41:43,146][105620] Updated weights for policy 1, policy_version 1692060 (0.0011) [2023-12-27 03:41:43,208][105620] Updated weights for policy 1, policy_version 1692070 (0.0010) [2023-12-27 03:41:43,276][105620] Updated weights for policy 1, policy_version 1692080 (0.0010) [2023-12-27 03:41:43,549][105692] Updated weights for policy 0, policy_version 1688386 (0.0006) [2023-12-27 03:41:43,603][105692] Updated weights for policy 0, policy_version 1688396 (0.0009) [2023-12-27 03:41:43,652][105692] Updated weights for policy 0, policy_version 1688406 (0.0005) [2023-12-27 03:41:43,725][105692] Updated weights for policy 0, policy_version 1688416 (0.0006) [2023-12-27 03:41:43,856][105620] Updated weights for policy 1, policy_version 1692090 (0.0005) [2023-12-27 03:41:43,921][105620] Updated weights for policy 1, policy_version 1692100 (0.0008) [2023-12-27 03:41:43,979][105620] Updated weights for policy 1, policy_version 1692110 (0.0010) [2023-12-27 03:41:44,426][105692] Updated weights for policy 0, policy_version 1688426 (0.0011) [2023-12-27 03:41:44,482][105692] Updated weights for policy 0, policy_version 1688436 (0.0011) [2023-12-27 03:41:44,530][105692] Updated weights for policy 0, policy_version 1688446 (0.0010) [2023-12-27 03:41:44,616][105620] Updated weights for policy 1, policy_version 1692120 (0.0006) [2023-12-27 03:41:44,676][105620] Updated weights for policy 1, policy_version 1692130 (0.0010) [2023-12-27 03:41:44,731][105620] Updated weights for policy 1, policy_version 1692140 (0.0010) [2023-12-27 03:41:45,304][105692] Updated weights for policy 0, policy_version 1688456 (0.0011) [2023-12-27 03:41:45,361][105692] Updated weights for policy 0, policy_version 1688466 (0.0011) [2023-12-27 03:41:45,414][105692] Updated weights for policy 0, policy_version 1688476 (0.0011) [2023-12-27 03:41:45,448][105620] Updated weights for policy 1, policy_version 1692150 (0.0010) [2023-12-27 03:41:45,500][105620] Updated weights for policy 1, policy_version 1692160 (0.0010) [2023-12-27 03:41:45,558][105620] Updated weights for policy 1, policy_version 1692170 (0.0010) [2023-12-27 03:41:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 865574912. Throughput: 0: 9381.0, 1: 9837.9. Samples: 865547976. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:41:46,062][104569] Avg episode reward: [(0, '8070.739'), (1, '9263.984')] [2023-12-27 03:41:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001692176_433258496.pth... [2023-12-27 03:41:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001691024_432963584.pth [2023-12-27 03:41:46,073][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_001692176_433258496.pth [2023-12-27 03:41:46,081][105692] Updated weights for policy 0, policy_version 1688486 (0.0008) [2023-12-27 03:41:46,130][105692] Updated weights for policy 0, policy_version 1688496 (0.0005) [2023-12-27 03:41:46,188][105620] Updated weights for policy 1, policy_version 1692180 (0.0008) [2023-12-27 03:41:46,194][105692] Updated weights for policy 0, policy_version 1688506 (0.0005) [2023-12-27 03:41:46,235][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001688512_432324608.pth... [2023-12-27 03:41:46,240][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001687392_432037888.pth [2023-12-27 03:41:46,241][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_001688512_432324608.pth [2023-12-27 03:41:46,252][105620] Updated weights for policy 1, policy_version 1692190 (0.0005) [2023-12-27 03:41:46,309][105620] Updated weights for policy 1, policy_version 1692200 (0.0007) [2023-12-27 03:41:46,797][105620] Updated weights for policy 1, policy_version 1692210 (0.0005) [2023-12-27 03:41:46,856][105620] Updated weights for policy 1, policy_version 1692220 (0.0008) [2023-12-27 03:41:46,907][105620] Updated weights for policy 1, policy_version 1692230 (0.0008) [2023-12-27 03:41:46,926][105692] Updated weights for policy 0, policy_version 1688516 (0.0007) [2023-12-27 03:41:46,954][105620] Updated weights for policy 1, policy_version 1692240 (0.0005) [2023-12-27 03:41:46,981][105692] Updated weights for policy 0, policy_version 1688526 (0.0010) [2023-12-27 03:41:47,035][105692] Updated weights for policy 0, policy_version 1688536 (0.0010) [2023-12-27 03:41:47,496][105620] Updated weights for policy 1, policy_version 1692250 (0.0011) [2023-12-27 03:41:47,548][105620] Updated weights for policy 1, policy_version 1692260 (0.0010) [2023-12-27 03:41:47,602][105620] Updated weights for policy 1, policy_version 1692270 (0.0010) [2023-12-27 03:41:47,907][105692] Updated weights for policy 0, policy_version 1688546 (0.0010) [2023-12-27 03:41:47,960][105692] Updated weights for policy 0, policy_version 1688556 (0.0010) [2023-12-27 03:41:48,015][105692] Updated weights for policy 0, policy_version 1688567 (0.0009) [2023-12-27 03:41:48,295][105620] Updated weights for policy 1, policy_version 1692280 (0.0009) [2023-12-27 03:41:48,358][105620] Updated weights for policy 1, policy_version 1692290 (0.0007) [2023-12-27 03:41:48,409][105620] Updated weights for policy 1, policy_version 1692300 (0.0005) [2023-12-27 03:41:48,828][105692] Updated weights for policy 0, policy_version 1688577 (0.0008) [2023-12-27 03:41:48,891][105692] Updated weights for policy 0, policy_version 1688587 (0.0008) [2023-12-27 03:41:48,957][105692] Updated weights for policy 0, policy_version 1688597 (0.0009) [2023-12-27 03:41:49,013][105692] Updated weights for policy 0, policy_version 1688607 (0.0008) [2023-12-27 03:41:49,116][105620] Updated weights for policy 1, policy_version 1692310 (0.0009) [2023-12-27 03:41:49,175][105620] Updated weights for policy 1, policy_version 1692320 (0.0010) [2023-12-27 03:41:49,247][105620] Updated weights for policy 1, policy_version 1692330 (0.0007) [2023-12-27 03:41:49,782][105692] Updated weights for policy 0, policy_version 1688617 (0.0009) [2023-12-27 03:41:49,840][105692] Updated weights for policy 0, policy_version 1688627 (0.0009) [2023-12-27 03:41:49,900][105692] Updated weights for policy 0, policy_version 1688637 (0.0008) [2023-12-27 03:41:49,918][105620] Updated weights for policy 1, policy_version 1692340 (0.0008) [2023-12-27 03:41:49,982][105620] Updated weights for policy 1, policy_version 1692350 (0.0010) [2023-12-27 03:41:50,043][105620] Updated weights for policy 1, policy_version 1692360 (0.0008) [2023-12-27 03:41:50,674][105692] Updated weights for policy 0, policy_version 1688647 (0.0008) [2023-12-27 03:41:50,723][105692] Updated weights for policy 0, policy_version 1688657 (0.0008) [2023-12-27 03:41:50,777][105692] Updated weights for policy 0, policy_version 1688667 (0.0008) [2023-12-27 03:41:50,808][105620] Updated weights for policy 1, policy_version 1692370 (0.0008) [2023-12-27 03:41:50,875][105620] Updated weights for policy 1, policy_version 1692380 (0.0006) [2023-12-27 03:41:50,938][105620] Updated weights for policy 1, policy_version 1692390 (0.0010) [2023-12-27 03:41:51,004][105620] Updated weights for policy 1, policy_version 1692400 (0.0010) [2023-12-27 03:41:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 865681408. Throughput: 0: 9336.8, 1: 9986.9. Samples: 865667384. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:41:51,062][104569] Avg episode reward: [(0, '8440.251'), (1, '9356.409')] [2023-12-27 03:41:51,579][105692] Updated weights for policy 0, policy_version 1688677 (0.0007) [2023-12-27 03:41:51,636][105620] Updated weights for policy 1, policy_version 1692410 (0.0008) [2023-12-27 03:41:51,645][105692] Updated weights for policy 0, policy_version 1688687 (0.0008) [2023-12-27 03:41:51,688][105620] Updated weights for policy 1, policy_version 1692420 (0.0009) [2023-12-27 03:41:51,705][105692] Updated weights for policy 0, policy_version 1688697 (0.0007) [2023-12-27 03:41:51,748][105620] Updated weights for policy 1, policy_version 1692430 (0.0008) [2023-12-27 03:41:52,396][105620] Updated weights for policy 1, policy_version 1692440 (0.0009) [2023-12-27 03:41:52,407][105692] Updated weights for policy 0, policy_version 1688707 (0.0010) [2023-12-27 03:41:52,453][105620] Updated weights for policy 1, policy_version 1692450 (0.0006) [2023-12-27 03:41:52,459][105692] Updated weights for policy 0, policy_version 1688717 (0.0009) [2023-12-27 03:41:52,503][105620] Updated weights for policy 1, policy_version 1692460 (0.0006) [2023-12-27 03:41:52,509][105692] Updated weights for policy 0, policy_version 1688727 (0.0009) [2023-12-27 03:41:53,211][105620] Updated weights for policy 1, policy_version 1692470 (0.0008) [2023-12-27 03:41:53,278][105620] Updated weights for policy 1, policy_version 1692480 (0.0009) [2023-12-27 03:41:53,293][105692] Updated weights for policy 0, policy_version 1688737 (0.0007) [2023-12-27 03:41:53,330][105620] Updated weights for policy 1, policy_version 1692490 (0.0009) [2023-12-27 03:41:53,351][105692] Updated weights for policy 0, policy_version 1688747 (0.0006) [2023-12-27 03:41:53,405][105692] Updated weights for policy 0, policy_version 1688757 (0.0008) [2023-12-27 03:41:53,452][105692] Updated weights for policy 0, policy_version 1688767 (0.0009) [2023-12-27 03:41:54,090][105692] Updated weights for policy 0, policy_version 1688777 (0.0009) [2023-12-27 03:41:54,125][105620] Updated weights for policy 1, policy_version 1692500 (0.0008) [2023-12-27 03:41:54,147][105692] Updated weights for policy 0, policy_version 1688787 (0.0005) [2023-12-27 03:41:54,180][105620] Updated weights for policy 1, policy_version 1692510 (0.0009) [2023-12-27 03:41:54,208][105692] Updated weights for policy 0, policy_version 1688797 (0.0006) [2023-12-27 03:41:54,228][105620] Updated weights for policy 1, policy_version 1692520 (0.0008) [2023-12-27 03:41:54,849][105692] Updated weights for policy 0, policy_version 1688807 (0.0006) [2023-12-27 03:41:54,907][105692] Updated weights for policy 0, policy_version 1688817 (0.0005) [2023-12-27 03:41:54,967][105692] Updated weights for policy 0, policy_version 1688827 (0.0007) [2023-12-27 03:41:54,999][105620] Updated weights for policy 1, policy_version 1692530 (0.0008) [2023-12-27 03:41:55,057][105620] Updated weights for policy 1, policy_version 1692540 (0.0009) [2023-12-27 03:41:55,115][105620] Updated weights for policy 1, policy_version 1692550 (0.0011) [2023-12-27 03:41:55,168][105620] Updated weights for policy 1, policy_version 1692560 (0.0010) [2023-12-27 03:41:55,581][105692] Updated weights for policy 0, policy_version 1688837 (0.0009) [2023-12-27 03:41:55,626][105692] Updated weights for policy 0, policy_version 1688847 (0.0010) [2023-12-27 03:41:55,670][105692] Updated weights for policy 0, policy_version 1688857 (0.0010) [2023-12-27 03:41:55,950][105620] Updated weights for policy 1, policy_version 1692570 (0.0006) [2023-12-27 03:41:56,017][105620] Updated weights for policy 1, policy_version 1692580 (0.0007) [2023-12-27 03:41:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 865771520. Throughput: 0: 9393.8, 1: 9915.0. Samples: 865783856. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:41:56,062][104569] Avg episode reward: [(0, '8441.580'), (1, '9356.399')] [2023-12-27 03:41:56,073][105620] Updated weights for policy 1, policy_version 1692590 (0.0009) [2023-12-27 03:41:56,310][105692] Updated weights for policy 0, policy_version 1688867 (0.0010) [2023-12-27 03:41:56,361][105692] Updated weights for policy 0, policy_version 1688877 (0.0008) [2023-12-27 03:41:56,409][105692] Updated weights for policy 0, policy_version 1688887 (0.0009) [2023-12-27 03:41:56,833][105620] Updated weights for policy 1, policy_version 1692600 (0.0009) [2023-12-27 03:41:56,887][105620] Updated weights for policy 1, policy_version 1692610 (0.0009) [2023-12-27 03:41:56,944][105620] Updated weights for policy 1, policy_version 1692621 (0.0010) [2023-12-27 03:41:57,053][105692] Updated weights for policy 0, policy_version 1688897 (0.0009) [2023-12-27 03:41:57,118][105692] Updated weights for policy 0, policy_version 1688907 (0.0008) [2023-12-27 03:41:57,180][105692] Updated weights for policy 0, policy_version 1688917 (0.0009) [2023-12-27 03:41:57,240][105692] Updated weights for policy 0, policy_version 1688927 (0.0009) [2023-12-27 03:41:57,771][105692] Updated weights for policy 0, policy_version 1688937 (0.0005) [2023-12-27 03:41:57,798][105620] Updated weights for policy 1, policy_version 1692631 (0.0009) [2023-12-27 03:41:57,826][105692] Updated weights for policy 0, policy_version 1688947 (0.0005) [2023-12-27 03:41:57,853][105620] Updated weights for policy 1, policy_version 1692641 (0.0010) [2023-12-27 03:41:57,878][105692] Updated weights for policy 0, policy_version 1688957 (0.0005) [2023-12-27 03:41:57,911][105620] Updated weights for policy 1, policy_version 1692651 (0.0010) [2023-12-27 03:41:58,508][105692] Updated weights for policy 0, policy_version 1688967 (0.0009) [2023-12-27 03:41:58,574][105692] Updated weights for policy 0, policy_version 1688977 (0.0010) [2023-12-27 03:41:58,641][105692] Updated weights for policy 0, policy_version 1688987 (0.0010) [2023-12-27 03:41:58,660][105620] Updated weights for policy 1, policy_version 1692661 (0.0010) [2023-12-27 03:41:58,734][105620] Updated weights for policy 1, policy_version 1692671 (0.0008) [2023-12-27 03:41:58,803][105620] Updated weights for policy 1, policy_version 1692681 (0.0008) [2023-12-27 03:41:59,583][105692] Updated weights for policy 0, policy_version 1688997 (0.0010) [2023-12-27 03:41:59,642][105620] Updated weights for policy 1, policy_version 1692691 (0.0008) [2023-12-27 03:41:59,648][105692] Updated weights for policy 0, policy_version 1689007 (0.0006) [2023-12-27 03:41:59,694][105620] Updated weights for policy 1, policy_version 1692701 (0.0005) [2023-12-27 03:41:59,707][105692] Updated weights for policy 0, policy_version 1689017 (0.0006) [2023-12-27 03:41:59,754][105620] Updated weights for policy 1, policy_version 1692711 (0.0005) [2023-12-27 03:42:00,386][105692] Updated weights for policy 0, policy_version 1689027 (0.0006) [2023-12-27 03:42:00,446][105692] Updated weights for policy 0, policy_version 1689037 (0.0008) [2023-12-27 03:42:00,463][105620] Updated weights for policy 1, policy_version 1692721 (0.0005) [2023-12-27 03:42:00,505][105692] Updated weights for policy 0, policy_version 1689047 (0.0010) [2023-12-27 03:42:00,530][105620] Updated weights for policy 1, policy_version 1692731 (0.0006) [2023-12-27 03:42:00,593][105620] Updated weights for policy 1, policy_version 1692741 (0.0007) [2023-12-27 03:42:00,648][105620] Updated weights for policy 1, policy_version 1692751 (0.0009) [2023-12-27 03:42:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 865869824. Throughput: 0: 9513.6, 1: 9847.8. Samples: 865843192. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:42:01,062][104569] Avg episode reward: [(0, '8627.859'), (1, '9081.588')] [2023-12-27 03:42:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001689056_432463872.pth... [2023-12-27 03:42:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001692752_433405952.pth... [2023-12-27 03:42:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001687936_432177152.pth [2023-12-27 03:42:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001691600_433111040.pth [2023-12-27 03:42:01,255][105692] Updated weights for policy 0, policy_version 1689057 (0.0008) [2023-12-27 03:42:01,308][105692] Updated weights for policy 0, policy_version 1689067 (0.0007) [2023-12-27 03:42:01,311][105620] Updated weights for policy 1, policy_version 1692761 (0.0009) [2023-12-27 03:42:01,369][105692] Updated weights for policy 0, policy_version 1689077 (0.0009) [2023-12-27 03:42:01,379][105620] Updated weights for policy 1, policy_version 1692771 (0.0008) [2023-12-27 03:42:01,429][105692] Updated weights for policy 0, policy_version 1689087 (0.0007) [2023-12-27 03:42:01,433][105620] Updated weights for policy 1, policy_version 1692781 (0.0006) [2023-12-27 03:42:02,089][105692] Updated weights for policy 0, policy_version 1689097 (0.0006) [2023-12-27 03:42:02,145][105692] Updated weights for policy 0, policy_version 1689107 (0.0005) [2023-12-27 03:42:02,201][105692] Updated weights for policy 0, policy_version 1689117 (0.0006) [2023-12-27 03:42:02,222][105620] Updated weights for policy 1, policy_version 1692791 (0.0008) [2023-12-27 03:42:02,284][105620] Updated weights for policy 1, policy_version 1692801 (0.0007) [2023-12-27 03:42:02,388][105620] Updated weights for policy 1, policy_version 1692811 (0.0007) [2023-12-27 03:42:02,765][105692] Updated weights for policy 0, policy_version 1689127 (0.0008) [2023-12-27 03:42:02,823][105692] Updated weights for policy 0, policy_version 1689137 (0.0009) [2023-12-27 03:42:02,873][105692] Updated weights for policy 0, policy_version 1689147 (0.0009) [2023-12-27 03:42:03,122][105620] Updated weights for policy 1, policy_version 1692821 (0.0009) [2023-12-27 03:42:03,182][105620] Updated weights for policy 1, policy_version 1692831 (0.0009) [2023-12-27 03:42:03,244][105620] Updated weights for policy 1, policy_version 1692841 (0.0010) [2023-12-27 03:42:03,510][105692] Updated weights for policy 0, policy_version 1689157 (0.0009) [2023-12-27 03:42:03,579][105692] Updated weights for policy 0, policy_version 1689167 (0.0006) [2023-12-27 03:42:03,646][105692] Updated weights for policy 0, policy_version 1689177 (0.0006) [2023-12-27 03:42:04,049][105620] Updated weights for policy 1, policy_version 1692852 (0.0010) [2023-12-27 03:42:04,108][105620] Updated weights for policy 1, policy_version 1692862 (0.0010) [2023-12-27 03:42:04,163][105620] Updated weights for policy 1, policy_version 1692872 (0.0010) [2023-12-27 03:42:04,303][105692] Updated weights for policy 0, policy_version 1689187 (0.0005) [2023-12-27 03:42:04,371][105692] Updated weights for policy 0, policy_version 1689197 (0.0005) [2023-12-27 03:42:04,432][105692] Updated weights for policy 0, policy_version 1689207 (0.0006) [2023-12-27 03:42:04,744][105620] Updated weights for policy 1, policy_version 1692882 (0.0009) [2023-12-27 03:42:04,809][105620] Updated weights for policy 1, policy_version 1692892 (0.0007) [2023-12-27 03:42:04,870][105620] Updated weights for policy 1, policy_version 1692902 (0.0010) [2023-12-27 03:42:04,932][105620] Updated weights for policy 1, policy_version 1692912 (0.0010) [2023-12-27 03:42:05,176][105692] Updated weights for policy 0, policy_version 1689217 (0.0007) [2023-12-27 03:42:05,240][105692] Updated weights for policy 0, policy_version 1689227 (0.0005) [2023-12-27 03:42:05,302][105692] Updated weights for policy 0, policy_version 1689237 (0.0005) [2023-12-27 03:42:05,365][105692] Updated weights for policy 0, policy_version 1689247 (0.0005) [2023-12-27 03:42:05,510][105620] Updated weights for policy 1, policy_version 1692922 (0.0010) [2023-12-27 03:42:05,558][105620] Updated weights for policy 1, policy_version 1692932 (0.0010) [2023-12-27 03:42:05,606][105620] Updated weights for policy 1, policy_version 1692942 (0.0010) [2023-12-27 03:42:05,941][105692] Updated weights for policy 0, policy_version 1689257 (0.0005) [2023-12-27 03:42:05,995][105692] Updated weights for policy 0, policy_version 1689267 (0.0005) [2023-12-27 03:42:06,056][105692] Updated weights for policy 0, policy_version 1689277 (0.0005) [2023-12-27 03:42:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 865968128. Throughput: 0: 9555.4, 1: 9889.4. Samples: 865960120. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:42:06,062][104569] Avg episode reward: [(0, '8810.053'), (1, '8714.026')] [2023-12-27 03:42:06,287][105620] Updated weights for policy 1, policy_version 1692952 (0.0007) [2023-12-27 03:42:06,351][105620] Updated weights for policy 1, policy_version 1692962 (0.0009) [2023-12-27 03:42:06,420][105620] Updated weights for policy 1, policy_version 1692972 (0.0006) [2023-12-27 03:42:06,596][105692] Updated weights for policy 0, policy_version 1689287 (0.0005) [2023-12-27 03:42:06,664][105692] Updated weights for policy 0, policy_version 1689297 (0.0005) [2023-12-27 03:42:06,732][105692] Updated weights for policy 0, policy_version 1689307 (0.0006) [2023-12-27 03:42:06,983][105620] Updated weights for policy 1, policy_version 1692982 (0.0005) [2023-12-27 03:42:07,033][105620] Updated weights for policy 1, policy_version 1692992 (0.0005) [2023-12-27 03:42:07,099][105620] Updated weights for policy 1, policy_version 1693002 (0.0008) [2023-12-27 03:42:07,288][105692] Updated weights for policy 0, policy_version 1689317 (0.0008) [2023-12-27 03:42:07,351][105692] Updated weights for policy 0, policy_version 1689327 (0.0006) [2023-12-27 03:42:07,412][105692] Updated weights for policy 0, policy_version 1689337 (0.0006) [2023-12-27 03:42:07,666][105620] Updated weights for policy 1, policy_version 1693012 (0.0007) [2023-12-27 03:42:07,717][105620] Updated weights for policy 1, policy_version 1693022 (0.0009) [2023-12-27 03:42:07,769][105620] Updated weights for policy 1, policy_version 1693032 (0.0008) [2023-12-27 03:42:07,992][105692] Updated weights for policy 0, policy_version 1689347 (0.0006) [2023-12-27 03:42:08,055][105692] Updated weights for policy 0, policy_version 1689357 (0.0006) [2023-12-27 03:42:08,128][105692] Updated weights for policy 0, policy_version 1689367 (0.0005) [2023-12-27 03:42:08,444][105620] Updated weights for policy 1, policy_version 1693042 (0.0008) [2023-12-27 03:42:08,513][105620] Updated weights for policy 1, policy_version 1693052 (0.0008) [2023-12-27 03:42:08,569][105620] Updated weights for policy 1, policy_version 1693062 (0.0006) [2023-12-27 03:42:08,639][105620] Updated weights for policy 1, policy_version 1693072 (0.0007) [2023-12-27 03:42:08,667][105692] Updated weights for policy 0, policy_version 1689377 (0.0005) [2023-12-27 03:42:08,731][105692] Updated weights for policy 0, policy_version 1689387 (0.0007) [2023-12-27 03:42:08,793][105692] Updated weights for policy 0, policy_version 1689397 (0.0008) [2023-12-27 03:42:08,856][105692] Updated weights for policy 0, policy_version 1689407 (0.0011) [2023-12-27 03:42:09,304][105620] Updated weights for policy 1, policy_version 1693082 (0.0007) [2023-12-27 03:42:09,368][105620] Updated weights for policy 1, policy_version 1693092 (0.0008) [2023-12-27 03:42:09,430][105620] Updated weights for policy 1, policy_version 1693102 (0.0010) [2023-12-27 03:42:09,524][105692] Updated weights for policy 0, policy_version 1689417 (0.0010) [2023-12-27 03:42:09,580][105692] Updated weights for policy 0, policy_version 1689427 (0.0011) [2023-12-27 03:42:09,640][105692] Updated weights for policy 0, policy_version 1689437 (0.0010) [2023-12-27 03:42:10,176][105620] Updated weights for policy 1, policy_version 1693112 (0.0008) [2023-12-27 03:42:10,236][105620] Updated weights for policy 1, policy_version 1693122 (0.0008) [2023-12-27 03:42:10,300][105620] Updated weights for policy 1, policy_version 1693132 (0.0008) [2023-12-27 03:42:10,408][105692] Updated weights for policy 0, policy_version 1689447 (0.0011) [2023-12-27 03:42:10,464][105692] Updated weights for policy 0, policy_version 1689457 (0.0010) [2023-12-27 03:42:10,526][105692] Updated weights for policy 0, policy_version 1689467 (0.0011) [2023-12-27 03:42:10,905][105620] Updated weights for policy 1, policy_version 1693142 (0.0008) [2023-12-27 03:42:10,962][105620] Updated weights for policy 1, policy_version 1693152 (0.0010) [2023-12-27 03:42:11,027][105620] Updated weights for policy 1, policy_version 1693162 (0.0009) [2023-12-27 03:42:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 866074624. Throughput: 0: 9785.8, 1: 10022.5. Samples: 866088424. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:42:11,062][104569] Avg episode reward: [(0, '8717.320'), (1, '8893.843')] [2023-12-27 03:42:11,256][105692] Updated weights for policy 0, policy_version 1689477 (0.0008) [2023-12-27 03:42:11,318][105692] Updated weights for policy 0, policy_version 1689487 (0.0010) [2023-12-27 03:42:11,387][105692] Updated weights for policy 0, policy_version 1689497 (0.0011) [2023-12-27 03:42:11,845][105620] Updated weights for policy 1, policy_version 1693172 (0.0009) [2023-12-27 03:42:11,907][105620] Updated weights for policy 1, policy_version 1693182 (0.0008) [2023-12-27 03:42:11,973][105620] Updated weights for policy 1, policy_version 1693192 (0.0008) [2023-12-27 03:42:12,173][105692] Updated weights for policy 0, policy_version 1689507 (0.0011) [2023-12-27 03:42:12,236][105692] Updated weights for policy 0, policy_version 1689517 (0.0010) [2023-12-27 03:42:12,304][105692] Updated weights for policy 0, policy_version 1689527 (0.0010) [2023-12-27 03:42:12,722][105620] Updated weights for policy 1, policy_version 1693202 (0.0008) [2023-12-27 03:42:12,787][105620] Updated weights for policy 1, policy_version 1693212 (0.0007) [2023-12-27 03:42:12,856][105620] Updated weights for policy 1, policy_version 1693222 (0.0006) [2023-12-27 03:42:12,926][105620] Updated weights for policy 1, policy_version 1693232 (0.0006) [2023-12-27 03:42:12,970][105692] Updated weights for policy 0, policy_version 1689537 (0.0011) [2023-12-27 03:42:13,029][105692] Updated weights for policy 0, policy_version 1689547 (0.0010) [2023-12-27 03:42:13,092][105692] Updated weights for policy 0, policy_version 1689557 (0.0010) [2023-12-27 03:42:13,140][105692] Updated weights for policy 0, policy_version 1689567 (0.0010) [2023-12-27 03:42:13,462][105620] Updated weights for policy 1, policy_version 1693242 (0.0006) [2023-12-27 03:42:13,520][105620] Updated weights for policy 1, policy_version 1693252 (0.0006) [2023-12-27 03:42:13,583][105620] Updated weights for policy 1, policy_version 1693262 (0.0006) [2023-12-27 03:42:13,836][105692] Updated weights for policy 0, policy_version 1689577 (0.0009) [2023-12-27 03:42:13,892][105692] Updated weights for policy 0, policy_version 1689587 (0.0012) [2023-12-27 03:42:13,949][105692] Updated weights for policy 0, policy_version 1689598 (0.0010) [2023-12-27 03:42:14,115][105620] Updated weights for policy 1, policy_version 1693272 (0.0006) [2023-12-27 03:42:14,166][105586] KL-divergence is very high: 104.2710 [2023-12-27 03:42:14,169][105620] Updated weights for policy 1, policy_version 1693282 (0.0008) [2023-12-27 03:42:14,203][105586] KL-divergence is very high: 107.8903 [2023-12-27 03:42:14,219][105620] Updated weights for policy 1, policy_version 1693293 (0.0008) [2023-12-27 03:42:14,654][105692] Updated weights for policy 0, policy_version 1689608 (0.0010) [2023-12-27 03:42:14,705][105692] Updated weights for policy 0, policy_version 1689618 (0.0010) [2023-12-27 03:42:14,756][105692] Updated weights for policy 0, policy_version 1689628 (0.0010) [2023-12-27 03:42:14,972][105620] Updated weights for policy 1, policy_version 1693303 (0.0009) [2023-12-27 03:42:15,029][105620] Updated weights for policy 1, policy_version 1693313 (0.0011) [2023-12-27 03:42:15,075][105620] Updated weights for policy 1, policy_version 1693323 (0.0011) [2023-12-27 03:42:15,478][105692] Updated weights for policy 0, policy_version 1689638 (0.0008) [2023-12-27 03:42:15,540][105692] Updated weights for policy 0, policy_version 1689648 (0.0008) [2023-12-27 03:42:15,599][105692] Updated weights for policy 0, policy_version 1689658 (0.0010) [2023-12-27 03:42:15,807][105620] Updated weights for policy 1, policy_version 1693333 (0.0008) [2023-12-27 03:42:15,855][105620] Updated weights for policy 1, policy_version 1693343 (0.0005) [2023-12-27 03:42:15,909][105620] Updated weights for policy 1, policy_version 1693353 (0.0005) [2023-12-27 03:42:16,062][104569] Fps is (10 sec: 21298.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 866181120. Throughput: 0: 9872.4, 1: 9949.1. Samples: 866148508. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:42:16,063][104569] Avg episode reward: [(0, '8715.665'), (1, '8710.070')] [2023-12-27 03:42:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001689664_432619520.pth... [2023-12-27 03:42:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001693360_433561600.pth... [2023-12-27 03:42:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001688512_432324608.pth [2023-12-27 03:42:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001692176_433258496.pth [2023-12-27 03:42:16,347][105692] Updated weights for policy 0, policy_version 1689668 (0.0011) [2023-12-27 03:42:16,416][105692] Updated weights for policy 0, policy_version 1689678 (0.0011) [2023-12-27 03:42:16,471][105692] Updated weights for policy 0, policy_version 1689688 (0.0010) [2023-12-27 03:42:16,492][105620] Updated weights for policy 1, policy_version 1693363 (0.0007) [2023-12-27 03:42:16,547][105620] Updated weights for policy 1, policy_version 1693373 (0.0007) [2023-12-27 03:42:16,599][105620] Updated weights for policy 1, policy_version 1693383 (0.0007) [2023-12-27 03:42:17,124][105692] Updated weights for policy 0, policy_version 1689698 (0.0010) [2023-12-27 03:42:17,171][105620] Updated weights for policy 1, policy_version 1693393 (0.0010) [2023-12-27 03:42:17,183][105692] Updated weights for policy 0, policy_version 1689708 (0.0011) [2023-12-27 03:42:17,231][105620] Updated weights for policy 1, policy_version 1693403 (0.0009) [2023-12-27 03:42:17,241][105692] Updated weights for policy 0, policy_version 1689718 (0.0010) [2023-12-27 03:42:17,290][105620] Updated weights for policy 1, policy_version 1693413 (0.0010) [2023-12-27 03:42:17,296][105692] Updated weights for policy 0, policy_version 1689728 (0.0010) [2023-12-27 03:42:17,351][105620] Updated weights for policy 1, policy_version 1693423 (0.0010) [2023-12-27 03:42:18,027][105692] Updated weights for policy 0, policy_version 1689738 (0.0009) [2023-12-27 03:42:18,067][105620] Updated weights for policy 1, policy_version 1693433 (0.0008) [2023-12-27 03:42:18,092][105692] Updated weights for policy 0, policy_version 1689748 (0.0005) [2023-12-27 03:42:18,127][105620] Updated weights for policy 1, policy_version 1693443 (0.0009) [2023-12-27 03:42:18,151][105692] Updated weights for policy 0, policy_version 1689758 (0.0005) [2023-12-27 03:42:18,176][105620] Updated weights for policy 1, policy_version 1693453 (0.0010) [2023-12-27 03:42:18,718][105692] Updated weights for policy 0, policy_version 1689768 (0.0008) [2023-12-27 03:42:18,776][105692] Updated weights for policy 0, policy_version 1689778 (0.0009) [2023-12-27 03:42:18,839][105692] Updated weights for policy 0, policy_version 1689788 (0.0009) [2023-12-27 03:42:18,958][105620] Updated weights for policy 1, policy_version 1693463 (0.0009) [2023-12-27 03:42:19,017][105620] Updated weights for policy 1, policy_version 1693473 (0.0009) [2023-12-27 03:42:19,074][105620] Updated weights for policy 1, policy_version 1693483 (0.0008) [2023-12-27 03:42:19,625][105692] Updated weights for policy 0, policy_version 1689798 (0.0009) [2023-12-27 03:42:19,682][105692] Updated weights for policy 0, policy_version 1689808 (0.0010) [2023-12-27 03:42:19,746][105692] Updated weights for policy 0, policy_version 1689818 (0.0009) [2023-12-27 03:42:19,849][105620] Updated weights for policy 1, policy_version 1693493 (0.0009) [2023-12-27 03:42:19,907][105620] Updated weights for policy 1, policy_version 1693503 (0.0011) [2023-12-27 03:42:19,974][105620] Updated weights for policy 1, policy_version 1693513 (0.0010) [2023-12-27 03:42:20,531][105692] Updated weights for policy 0, policy_version 1689828 (0.0009) [2023-12-27 03:42:20,595][105692] Updated weights for policy 0, policy_version 1689838 (0.0009) [2023-12-27 03:42:20,657][105692] Updated weights for policy 0, policy_version 1689848 (0.0008) [2023-12-27 03:42:20,740][105620] Updated weights for policy 1, policy_version 1693523 (0.0010) [2023-12-27 03:42:20,796][105620] Updated weights for policy 1, policy_version 1693533 (0.0011) [2023-12-27 03:42:20,855][105620] Updated weights for policy 1, policy_version 1693543 (0.0010) [2023-12-27 03:42:21,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 866279424. Throughput: 0: 9861.0, 1: 9992.0. Samples: 866267476. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:42:21,063][104569] Avg episode reward: [(0, '8350.551'), (1, '8620.461')] [2023-12-27 03:42:21,456][105692] Updated weights for policy 0, policy_version 1689858 (0.0008) [2023-12-27 03:42:21,517][105692] Updated weights for policy 0, policy_version 1689868 (0.0009) [2023-12-27 03:42:21,581][105692] Updated weights for policy 0, policy_version 1689878 (0.0008) [2023-12-27 03:42:21,583][105620] Updated weights for policy 1, policy_version 1693553 (0.0011) [2023-12-27 03:42:21,639][105620] Updated weights for policy 1, policy_version 1693563 (0.0010) [2023-12-27 03:42:21,657][105692] Updated weights for policy 0, policy_version 1689888 (0.0006) [2023-12-27 03:42:21,706][105620] Updated weights for policy 1, policy_version 1693573 (0.0007) [2023-12-27 03:42:21,776][105620] Updated weights for policy 1, policy_version 1693583 (0.0008) [2023-12-27 03:42:22,472][105620] Updated weights for policy 1, policy_version 1693593 (0.0008) [2023-12-27 03:42:22,495][105692] Updated weights for policy 0, policy_version 1689898 (0.0008) [2023-12-27 03:42:22,526][105620] Updated weights for policy 1, policy_version 1693603 (0.0006) [2023-12-27 03:42:22,555][105692] Updated weights for policy 0, policy_version 1689908 (0.0008) [2023-12-27 03:42:22,579][105620] Updated weights for policy 1, policy_version 1693613 (0.0007) [2023-12-27 03:42:22,618][105692] Updated weights for policy 0, policy_version 1689918 (0.0008) [2023-12-27 03:42:23,311][105620] Updated weights for policy 1, policy_version 1693623 (0.0006) [2023-12-27 03:42:23,372][105620] Updated weights for policy 1, policy_version 1693633 (0.0006) [2023-12-27 03:42:23,382][105692] Updated weights for policy 0, policy_version 1689928 (0.0009) [2023-12-27 03:42:23,432][105620] Updated weights for policy 1, policy_version 1693643 (0.0007) [2023-12-27 03:42:23,438][105692] Updated weights for policy 0, policy_version 1689938 (0.0006) [2023-12-27 03:42:23,490][105692] Updated weights for policy 0, policy_version 1689948 (0.0007) [2023-12-27 03:42:24,161][105620] Updated weights for policy 1, policy_version 1693653 (0.0008) [2023-12-27 03:42:24,224][105620] Updated weights for policy 1, policy_version 1693663 (0.0007) [2023-12-27 03:42:24,239][105692] Updated weights for policy 0, policy_version 1689958 (0.0008) [2023-12-27 03:42:24,287][105692] Updated weights for policy 0, policy_version 1689968 (0.0006) [2023-12-27 03:42:24,288][105620] Updated weights for policy 1, policy_version 1693673 (0.0009) [2023-12-27 03:42:24,345][105692] Updated weights for policy 0, policy_version 1689978 (0.0006) [2023-12-27 03:42:24,981][105692] Updated weights for policy 0, policy_version 1689988 (0.0006) [2023-12-27 03:42:25,037][105692] Updated weights for policy 0, policy_version 1689998 (0.0009) [2023-12-27 03:42:25,083][105620] Updated weights for policy 1, policy_version 1693683 (0.0007) [2023-12-27 03:42:25,094][105692] Updated weights for policy 0, policy_version 1690008 (0.0008) [2023-12-27 03:42:25,141][105620] Updated weights for policy 1, policy_version 1693693 (0.0006) [2023-12-27 03:42:25,192][105620] Updated weights for policy 1, policy_version 1693703 (0.0009) [2023-12-27 03:42:25,846][105692] Updated weights for policy 0, policy_version 1690018 (0.0006) [2023-12-27 03:42:25,891][105692] Updated weights for policy 0, policy_version 1690028 (0.0007) [2023-12-27 03:42:25,908][105620] Updated weights for policy 1, policy_version 1693713 (0.0008) [2023-12-27 03:42:25,940][105692] Updated weights for policy 0, policy_version 1690038 (0.0007) [2023-12-27 03:42:25,959][105620] Updated weights for policy 1, policy_version 1693723 (0.0008) [2023-12-27 03:42:25,997][105692] Updated weights for policy 0, policy_version 1690048 (0.0007) [2023-12-27 03:42:26,012][105620] Updated weights for policy 1, policy_version 1693733 (0.0007) [2023-12-27 03:42:26,062][105620] Updated weights for policy 1, policy_version 1693743 (0.0009) [2023-12-27 03:42:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 866369536. Throughput: 0: 9901.7, 1: 9937.2. Samples: 866379904. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:42:26,063][104569] Avg episode reward: [(0, '8438.668'), (1, '9080.299')] [2023-12-27 03:42:26,741][105692] Updated weights for policy 0, policy_version 1690058 (0.0009) [2023-12-27 03:42:26,796][105692] Updated weights for policy 0, policy_version 1690068 (0.0009) [2023-12-27 03:42:26,825][105620] Updated weights for policy 1, policy_version 1693753 (0.0008) [2023-12-27 03:42:26,846][105692] Updated weights for policy 0, policy_version 1690078 (0.0007) [2023-12-27 03:42:26,874][105620] Updated weights for policy 1, policy_version 1693763 (0.0007) [2023-12-27 03:42:26,927][105620] Updated weights for policy 1, policy_version 1693773 (0.0007) [2023-12-27 03:42:27,639][105692] Updated weights for policy 0, policy_version 1690088 (0.0007) [2023-12-27 03:42:27,642][105620] Updated weights for policy 1, policy_version 1693783 (0.0007) [2023-12-27 03:42:27,688][105620] Updated weights for policy 1, policy_version 1693793 (0.0007) [2023-12-27 03:42:27,698][105692] Updated weights for policy 0, policy_version 1690098 (0.0008) [2023-12-27 03:42:27,734][105620] Updated weights for policy 1, policy_version 1693803 (0.0008) [2023-12-27 03:42:27,753][105692] Updated weights for policy 0, policy_version 1690108 (0.0008) [2023-12-27 03:42:28,501][105692] Updated weights for policy 0, policy_version 1690118 (0.0008) [2023-12-27 03:42:28,507][105620] Updated weights for policy 1, policy_version 1693813 (0.0007) [2023-12-27 03:42:28,556][105620] Updated weights for policy 1, policy_version 1693823 (0.0006) [2023-12-27 03:42:28,558][105692] Updated weights for policy 0, policy_version 1690128 (0.0006) [2023-12-27 03:42:28,612][105620] Updated weights for policy 1, policy_version 1693833 (0.0006) [2023-12-27 03:42:28,614][105692] Updated weights for policy 0, policy_version 1690138 (0.0006) [2023-12-27 03:42:29,372][105692] Updated weights for policy 0, policy_version 1690148 (0.0009) [2023-12-27 03:42:29,394][105620] Updated weights for policy 1, policy_version 1693843 (0.0006) [2023-12-27 03:42:29,435][105692] Updated weights for policy 0, policy_version 1690158 (0.0010) [2023-12-27 03:42:29,448][105620] Updated weights for policy 1, policy_version 1693853 (0.0005) [2023-12-27 03:42:29,493][105692] Updated weights for policy 0, policy_version 1690168 (0.0011) [2023-12-27 03:42:29,512][105620] Updated weights for policy 1, policy_version 1693863 (0.0006) [2023-12-27 03:42:30,234][105692] Updated weights for policy 0, policy_version 1690178 (0.0010) [2023-12-27 03:42:30,280][105620] Updated weights for policy 1, policy_version 1693873 (0.0007) [2023-12-27 03:42:30,291][105692] Updated weights for policy 0, policy_version 1690188 (0.0009) [2023-12-27 03:42:30,336][105620] Updated weights for policy 1, policy_version 1693883 (0.0009) [2023-12-27 03:42:30,342][105692] Updated weights for policy 0, policy_version 1690198 (0.0008) [2023-12-27 03:42:30,391][105620] Updated weights for policy 1, policy_version 1693893 (0.0007) [2023-12-27 03:42:30,398][105692] Updated weights for policy 0, policy_version 1690208 (0.0006) [2023-12-27 03:42:30,443][105620] Updated weights for policy 1, policy_version 1693903 (0.0009) [2023-12-27 03:42:31,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 866459648. Throughput: 0: 9843.1, 1: 9892.4. Samples: 866436072. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:42:31,062][104569] Avg episode reward: [(0, '8806.532'), (1, '8989.336')] [2023-12-27 03:42:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001693904_433700864.pth... [2023-12-27 03:42:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001692752_433405952.pth [2023-12-27 03:42:31,089][105692] Updated weights for policy 0, policy_version 1690218 (0.0008) [2023-12-27 03:42:31,152][105692] Updated weights for policy 0, policy_version 1690228 (0.0008) [2023-12-27 03:42:31,194][105620] Updated weights for policy 1, policy_version 1693913 (0.0010) [2023-12-27 03:42:31,202][105692] Updated weights for policy 0, policy_version 1690238 (0.0007) [2023-12-27 03:42:31,211][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001690240_432766976.pth... [2023-12-27 03:42:31,215][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001689056_432463872.pth [2023-12-27 03:42:31,257][105620] Updated weights for policy 1, policy_version 1693923 (0.0010) [2023-12-27 03:42:31,310][105586] KL-divergence is very high: 114.1773 [2023-12-27 03:42:31,322][105620] Updated weights for policy 1, policy_version 1693933 (0.0010) [2023-12-27 03:42:31,950][105692] Updated weights for policy 0, policy_version 1690248 (0.0007) [2023-12-27 03:42:31,989][105620] Updated weights for policy 1, policy_version 1693943 (0.0007) [2023-12-27 03:42:32,007][105692] Updated weights for policy 0, policy_version 1690258 (0.0008) [2023-12-27 03:42:32,052][105620] Updated weights for policy 1, policy_version 1693953 (0.0006) [2023-12-27 03:42:32,064][105692] Updated weights for policy 0, policy_version 1690268 (0.0006) [2023-12-27 03:42:32,113][105620] Updated weights for policy 1, policy_version 1693963 (0.0008) [2023-12-27 03:42:32,686][105692] Updated weights for policy 0, policy_version 1690278 (0.0006) [2023-12-27 03:42:32,754][105692] Updated weights for policy 0, policy_version 1690288 (0.0006) [2023-12-27 03:42:32,818][105692] Updated weights for policy 0, policy_version 1690298 (0.0006) [2023-12-27 03:42:32,821][105620] Updated weights for policy 1, policy_version 1693973 (0.0009) [2023-12-27 03:42:32,876][105620] Updated weights for policy 1, policy_version 1693983 (0.0010) [2023-12-27 03:42:32,934][105620] Updated weights for policy 1, policy_version 1693993 (0.0010) [2023-12-27 03:42:33,480][105692] Updated weights for policy 0, policy_version 1690308 (0.0005) [2023-12-27 03:42:33,549][105692] Updated weights for policy 0, policy_version 1690318 (0.0005) [2023-12-27 03:42:33,563][105620] Updated weights for policy 1, policy_version 1694003 (0.0009) [2023-12-27 03:42:33,610][105692] Updated weights for policy 0, policy_version 1690328 (0.0005) [2023-12-27 03:42:33,631][105620] Updated weights for policy 1, policy_version 1694013 (0.0005) [2023-12-27 03:42:33,687][105620] Updated weights for policy 1, policy_version 1694023 (0.0005) [2023-12-27 03:42:34,149][105692] Updated weights for policy 0, policy_version 1690338 (0.0006) [2023-12-27 03:42:34,216][105692] Updated weights for policy 0, policy_version 1690348 (0.0008) [2023-12-27 03:42:34,279][105692] Updated weights for policy 0, policy_version 1690358 (0.0011) [2023-12-27 03:42:34,297][105620] Updated weights for policy 1, policy_version 1694033 (0.0005) [2023-12-27 03:42:34,343][105692] Updated weights for policy 0, policy_version 1690368 (0.0008) [2023-12-27 03:42:34,362][105620] Updated weights for policy 1, policy_version 1694043 (0.0008) [2023-12-27 03:42:34,430][105620] Updated weights for policy 1, policy_version 1694053 (0.0010) [2023-12-27 03:42:34,491][105620] Updated weights for policy 1, policy_version 1694063 (0.0011) [2023-12-27 03:42:35,000][105692] Updated weights for policy 0, policy_version 1690378 (0.0008) [2023-12-27 03:42:35,055][105692] Updated weights for policy 0, policy_version 1690388 (0.0010) [2023-12-27 03:42:35,107][105692] Updated weights for policy 0, policy_version 1690398 (0.0010) [2023-12-27 03:42:35,117][105620] Updated weights for policy 1, policy_version 1694073 (0.0006) [2023-12-27 03:42:35,175][105620] Updated weights for policy 1, policy_version 1694083 (0.0009) [2023-12-27 03:42:35,236][105620] Updated weights for policy 1, policy_version 1694093 (0.0008) [2023-12-27 03:42:35,839][105692] Updated weights for policy 0, policy_version 1690408 (0.0007) [2023-12-27 03:42:35,905][105692] Updated weights for policy 0, policy_version 1690418 (0.0006) [2023-12-27 03:42:35,931][105620] Updated weights for policy 1, policy_version 1694103 (0.0008) [2023-12-27 03:42:35,975][105692] Updated weights for policy 0, policy_version 1690428 (0.0005) [2023-12-27 03:42:35,984][105620] Updated weights for policy 1, policy_version 1694113 (0.0009) [2023-12-27 03:42:36,041][105620] Updated weights for policy 1, policy_version 1694124 (0.0010) [2023-12-27 03:42:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 866574336. Throughput: 0: 9965.4, 1: 9794.4. Samples: 866556576. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:42:36,063][104569] Avg episode reward: [(0, '8808.238'), (1, '9081.738')] [2023-12-27 03:42:36,530][105692] Updated weights for policy 0, policy_version 1690438 (0.0007) [2023-12-27 03:42:36,601][105692] Updated weights for policy 0, policy_version 1690448 (0.0008) [2023-12-27 03:42:36,663][105692] Updated weights for policy 0, policy_version 1690458 (0.0010) [2023-12-27 03:42:36,843][105620] Updated weights for policy 1, policy_version 1694134 (0.0010) [2023-12-27 03:42:36,909][105620] Updated weights for policy 1, policy_version 1694144 (0.0010) [2023-12-27 03:42:36,976][105620] Updated weights for policy 1, policy_version 1694154 (0.0011) [2023-12-27 03:42:37,263][105692] Updated weights for policy 0, policy_version 1690468 (0.0010) [2023-12-27 03:42:37,321][105692] Updated weights for policy 0, policy_version 1690478 (0.0010) [2023-12-27 03:42:37,383][105692] Updated weights for policy 0, policy_version 1690488 (0.0005) [2023-12-27 03:42:37,679][105620] Updated weights for policy 1, policy_version 1694164 (0.0008) [2023-12-27 03:42:37,742][105620] Updated weights for policy 1, policy_version 1694174 (0.0010) [2023-12-27 03:42:37,805][105620] Updated weights for policy 1, policy_version 1694184 (0.0010) [2023-12-27 03:42:38,058][105692] Updated weights for policy 0, policy_version 1690498 (0.0009) [2023-12-27 03:42:38,106][105692] Updated weights for policy 0, policy_version 1690508 (0.0010) [2023-12-27 03:42:38,154][105692] Updated weights for policy 0, policy_version 1690518 (0.0010) [2023-12-27 03:42:38,212][105692] Updated weights for policy 0, policy_version 1690528 (0.0010) [2023-12-27 03:42:38,517][105620] Updated weights for policy 1, policy_version 1694194 (0.0010) [2023-12-27 03:42:38,561][105620] Updated weights for policy 1, policy_version 1694204 (0.0008) [2023-12-27 03:42:38,614][105620] Updated weights for policy 1, policy_version 1694214 (0.0008) [2023-12-27 03:42:38,661][105620] Updated weights for policy 1, policy_version 1694224 (0.0007) [2023-12-27 03:42:38,985][105692] Updated weights for policy 0, policy_version 1690538 (0.0011) [2023-12-27 03:42:39,046][105692] Updated weights for policy 0, policy_version 1690548 (0.0010) [2023-12-27 03:42:39,098][105692] Updated weights for policy 0, policy_version 1690558 (0.0010) [2023-12-27 03:42:39,446][105620] Updated weights for policy 1, policy_version 1694234 (0.0008) [2023-12-27 03:42:39,509][105620] Updated weights for policy 1, policy_version 1694244 (0.0008) [2023-12-27 03:42:39,571][105620] Updated weights for policy 1, policy_version 1694254 (0.0008) [2023-12-27 03:42:39,795][105692] Updated weights for policy 0, policy_version 1690568 (0.0011) [2023-12-27 03:42:39,853][105692] Updated weights for policy 0, policy_version 1690578 (0.0011) [2023-12-27 03:42:39,911][105692] Updated weights for policy 0, policy_version 1690588 (0.0007) [2023-12-27 03:42:40,275][105620] Updated weights for policy 1, policy_version 1694264 (0.0010) [2023-12-27 03:42:40,327][105620] Updated weights for policy 1, policy_version 1694274 (0.0010) [2023-12-27 03:42:40,390][105620] Updated weights for policy 1, policy_version 1694284 (0.0011) [2023-12-27 03:42:40,626][105692] Updated weights for policy 0, policy_version 1690598 (0.0008) [2023-12-27 03:42:40,693][105692] Updated weights for policy 0, policy_version 1690608 (0.0006) [2023-12-27 03:42:40,759][105692] Updated weights for policy 0, policy_version 1690618 (0.0008) [2023-12-27 03:42:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.4, 300 sec: 19466.4). Total num frames: 866664448. Throughput: 0: 10015.6, 1: 9806.2. Samples: 866675836. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:42:41,062][104569] Avg episode reward: [(0, '8534.879'), (1, '9171.624')] [2023-12-27 03:42:41,101][105620] Updated weights for policy 1, policy_version 1694294 (0.0008) [2023-12-27 03:42:41,168][105620] Updated weights for policy 1, policy_version 1694304 (0.0008) [2023-12-27 03:42:41,234][105620] Updated weights for policy 1, policy_version 1694314 (0.0005) [2023-12-27 03:42:41,485][105692] Updated weights for policy 0, policy_version 1690628 (0.0009) [2023-12-27 03:42:41,538][105692] Updated weights for policy 0, policy_version 1690638 (0.0009) [2023-12-27 03:42:41,590][105692] Updated weights for policy 0, policy_version 1690648 (0.0009) [2023-12-27 03:42:41,977][105620] Updated weights for policy 1, policy_version 1694324 (0.0008) [2023-12-27 03:42:42,039][105620] Updated weights for policy 1, policy_version 1694334 (0.0009) [2023-12-27 03:42:42,090][105620] Updated weights for policy 1, policy_version 1694344 (0.0009) [2023-12-27 03:42:42,353][105692] Updated weights for policy 0, policy_version 1690658 (0.0010) [2023-12-27 03:42:42,414][105692] Updated weights for policy 0, policy_version 1690668 (0.0008) [2023-12-27 03:42:42,470][105692] Updated weights for policy 0, policy_version 1690678 (0.0008) [2023-12-27 03:42:42,529][105692] Updated weights for policy 0, policy_version 1690688 (0.0008) [2023-12-27 03:42:42,876][105620] Updated weights for policy 1, policy_version 1694354 (0.0009) [2023-12-27 03:42:42,941][105620] Updated weights for policy 1, policy_version 1694364 (0.0010) [2023-12-27 03:42:43,010][105620] Updated weights for policy 1, policy_version 1694374 (0.0010) [2023-12-27 03:42:43,072][105620] Updated weights for policy 1, policy_version 1694384 (0.0010) [2023-12-27 03:42:43,310][105692] Updated weights for policy 0, policy_version 1690698 (0.0008) [2023-12-27 03:42:43,358][105692] Updated weights for policy 0, policy_version 1690708 (0.0008) [2023-12-27 03:42:43,407][105692] Updated weights for policy 0, policy_version 1690718 (0.0008) [2023-12-27 03:42:43,783][105620] Updated weights for policy 1, policy_version 1694394 (0.0010) [2023-12-27 03:42:43,846][105620] Updated weights for policy 1, policy_version 1694404 (0.0010) [2023-12-27 03:42:43,911][105620] Updated weights for policy 1, policy_version 1694414 (0.0010) [2023-12-27 03:42:44,129][105692] Updated weights for policy 0, policy_version 1690728 (0.0006) [2023-12-27 03:42:44,180][105692] Updated weights for policy 0, policy_version 1690738 (0.0007) [2023-12-27 03:42:44,245][105692] Updated weights for policy 0, policy_version 1690748 (0.0010) [2023-12-27 03:42:44,548][105620] Updated weights for policy 1, policy_version 1694424 (0.0009) [2023-12-27 03:42:44,596][105620] Updated weights for policy 1, policy_version 1694434 (0.0009) [2023-12-27 03:42:44,644][105620] Updated weights for policy 1, policy_version 1694444 (0.0010) [2023-12-27 03:42:44,979][105692] Updated weights for policy 0, policy_version 1690758 (0.0010) [2023-12-27 03:42:45,032][105692] Updated weights for policy 0, policy_version 1690768 (0.0011) [2023-12-27 03:42:45,082][105692] Updated weights for policy 0, policy_version 1690778 (0.0011) [2023-12-27 03:42:45,439][105620] Updated weights for policy 1, policy_version 1694454 (0.0010) [2023-12-27 03:42:45,508][105620] Updated weights for policy 1, policy_version 1694464 (0.0010) [2023-12-27 03:42:45,562][105620] Updated weights for policy 1, policy_version 1694474 (0.0010) [2023-12-27 03:42:45,818][105692] Updated weights for policy 0, policy_version 1690788 (0.0009) [2023-12-27 03:42:45,881][105692] Updated weights for policy 0, policy_version 1690798 (0.0009) [2023-12-27 03:42:45,947][105692] Updated weights for policy 0, policy_version 1690808 (0.0010) [2023-12-27 03:42:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 866762752. Throughput: 0: 9882.2, 1: 9847.9. Samples: 866731048. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:42:46,062][104569] Avg episode reward: [(0, '8166.839'), (1, '9171.654')] [2023-12-27 03:42:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001690816_432914432.pth... [2023-12-27 03:42:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001694480_433848320.pth... [2023-12-27 03:42:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001689664_432619520.pth [2023-12-27 03:42:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001693360_433561600.pth [2023-12-27 03:42:46,251][105620] Updated weights for policy 1, policy_version 1694484 (0.0010) [2023-12-27 03:42:46,316][105620] Updated weights for policy 1, policy_version 1694494 (0.0009) [2023-12-27 03:42:46,381][105620] Updated weights for policy 1, policy_version 1694504 (0.0009) [2023-12-27 03:42:46,652][105692] Updated weights for policy 0, policy_version 1690818 (0.0007) [2023-12-27 03:42:46,700][105692] Updated weights for policy 0, policy_version 1690828 (0.0009) [2023-12-27 03:42:46,745][105692] Updated weights for policy 0, policy_version 1690838 (0.0010) [2023-12-27 03:42:46,795][105692] Updated weights for policy 0, policy_version 1690848 (0.0007) [2023-12-27 03:42:47,102][105620] Updated weights for policy 1, policy_version 1694514 (0.0011) [2023-12-27 03:42:47,156][105620] Updated weights for policy 1, policy_version 1694524 (0.0010) [2023-12-27 03:42:47,213][105620] Updated weights for policy 1, policy_version 1694534 (0.0010) [2023-12-27 03:42:47,270][105620] Updated weights for policy 1, policy_version 1694544 (0.0010) [2023-12-27 03:42:47,537][105692] Updated weights for policy 0, policy_version 1690858 (0.0010) [2023-12-27 03:42:47,599][105692] Updated weights for policy 0, policy_version 1690868 (0.0010) [2023-12-27 03:42:47,666][105692] Updated weights for policy 0, policy_version 1690878 (0.0007) [2023-12-27 03:42:47,932][105620] Updated weights for policy 1, policy_version 1694554 (0.0010) [2023-12-27 03:42:47,992][105620] Updated weights for policy 1, policy_version 1694564 (0.0009) [2023-12-27 03:42:48,047][105620] Updated weights for policy 1, policy_version 1694574 (0.0009) [2023-12-27 03:42:48,362][105692] Updated weights for policy 0, policy_version 1690888 (0.0008) [2023-12-27 03:42:48,424][105692] Updated weights for policy 0, policy_version 1690898 (0.0005) [2023-12-27 03:42:48,484][105692] Updated weights for policy 0, policy_version 1690908 (0.0006) [2023-12-27 03:42:48,784][105620] Updated weights for policy 1, policy_version 1694584 (0.0007) [2023-12-27 03:42:48,846][105620] Updated weights for policy 1, policy_version 1694594 (0.0011) [2023-12-27 03:42:48,912][105620] Updated weights for policy 1, policy_version 1694604 (0.0011) [2023-12-27 03:42:49,042][105692] Updated weights for policy 0, policy_version 1690918 (0.0008) [2023-12-27 03:42:49,096][105692] Updated weights for policy 0, policy_version 1690928 (0.0010) [2023-12-27 03:42:49,159][105692] Updated weights for policy 0, policy_version 1690938 (0.0009) [2023-12-27 03:42:49,557][105620] Updated weights for policy 1, policy_version 1694614 (0.0007) [2023-12-27 03:42:49,611][105620] Updated weights for policy 1, policy_version 1694624 (0.0005) [2023-12-27 03:42:49,677][105620] Updated weights for policy 1, policy_version 1694634 (0.0006) [2023-12-27 03:42:49,962][105692] Updated weights for policy 0, policy_version 1690948 (0.0010) [2023-12-27 03:42:50,025][105692] Updated weights for policy 0, policy_version 1690958 (0.0008) [2023-12-27 03:42:50,091][105692] Updated weights for policy 0, policy_version 1690968 (0.0008) [2023-12-27 03:42:50,314][105620] Updated weights for policy 1, policy_version 1694644 (0.0006) [2023-12-27 03:42:50,364][105620] Updated weights for policy 1, policy_version 1694654 (0.0008) [2023-12-27 03:42:50,405][105586] KL-divergence is very high: 209.9437 [2023-12-27 03:42:50,411][105586] KL-divergence is very high: 233.9963 [2023-12-27 03:42:50,416][105586] KL-divergence is very high: 133.0595 [2023-12-27 03:42:50,416][105620] Updated weights for policy 1, policy_version 1694664 (0.0009) [2023-12-27 03:42:50,432][105586] KL-divergence is very high: 139.3859 [2023-12-27 03:42:50,448][105586] KL-divergence is very high: 316.3513 [2023-12-27 03:42:50,453][105586] KL-divergence is very high: 324.3431 [2023-12-27 03:42:50,817][105692] Updated weights for policy 0, policy_version 1690978 (0.0008) [2023-12-27 03:42:50,869][105692] Updated weights for policy 0, policy_version 1690988 (0.0008) [2023-12-27 03:42:50,923][105692] Updated weights for policy 0, policy_version 1690998 (0.0007) [2023-12-27 03:42:50,980][105692] Updated weights for policy 0, policy_version 1691008 (0.0006) [2023-12-27 03:42:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 866861056. Throughput: 0: 9892.9, 1: 9913.3. Samples: 866851400. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:42:51,063][104569] Avg episode reward: [(0, '8252.934'), (1, '8988.194')] [2023-12-27 03:42:51,132][105620] Updated weights for policy 1, policy_version 1694674 (0.0008) [2023-12-27 03:42:51,195][105620] Updated weights for policy 1, policy_version 1694684 (0.0008) [2023-12-27 03:42:51,254][105620] Updated weights for policy 1, policy_version 1694694 (0.0008) [2023-12-27 03:42:51,317][105620] Updated weights for policy 1, policy_version 1694704 (0.0008) [2023-12-27 03:42:51,685][105692] Updated weights for policy 0, policy_version 1691018 (0.0009) [2023-12-27 03:42:51,748][105692] Updated weights for policy 0, policy_version 1691028 (0.0009) [2023-12-27 03:42:51,805][105692] Updated weights for policy 0, policy_version 1691038 (0.0009) [2023-12-27 03:42:52,131][105620] Updated weights for policy 1, policy_version 1694714 (0.0009) [2023-12-27 03:42:52,192][105620] Updated weights for policy 1, policy_version 1694724 (0.0009) [2023-12-27 03:42:52,256][105620] Updated weights for policy 1, policy_version 1694734 (0.0009) [2023-12-27 03:42:52,588][105692] Updated weights for policy 0, policy_version 1691048 (0.0010) [2023-12-27 03:42:52,656][105692] Updated weights for policy 0, policy_version 1691058 (0.0010) [2023-12-27 03:42:52,722][105692] Updated weights for policy 0, policy_version 1691068 (0.0010) [2023-12-27 03:42:52,912][105620] Updated weights for policy 1, policy_version 1694744 (0.0009) [2023-12-27 03:42:52,962][105620] Updated weights for policy 1, policy_version 1694755 (0.0009) [2023-12-27 03:42:53,024][105620] Updated weights for policy 1, policy_version 1694765 (0.0007) [2023-12-27 03:42:53,485][105692] Updated weights for policy 0, policy_version 1691078 (0.0009) [2023-12-27 03:42:53,536][105692] Updated weights for policy 0, policy_version 1691088 (0.0009) [2023-12-27 03:42:53,590][105692] Updated weights for policy 0, policy_version 1691098 (0.0009) [2023-12-27 03:42:53,632][105620] Updated weights for policy 1, policy_version 1694775 (0.0006) [2023-12-27 03:42:53,689][105620] Updated weights for policy 1, policy_version 1694785 (0.0009) [2023-12-27 03:42:53,747][105620] Updated weights for policy 1, policy_version 1694795 (0.0009) [2023-12-27 03:42:54,364][105692] Updated weights for policy 0, policy_version 1691108 (0.0006) [2023-12-27 03:42:54,365][105620] Updated weights for policy 1, policy_version 1694805 (0.0010) [2023-12-27 03:42:54,412][105692] Updated weights for policy 0, policy_version 1691118 (0.0005) [2023-12-27 03:42:54,421][105620] Updated weights for policy 1, policy_version 1694815 (0.0010) [2023-12-27 03:42:54,464][105692] Updated weights for policy 0, policy_version 1691128 (0.0006) [2023-12-27 03:42:54,476][105620] Updated weights for policy 1, policy_version 1694825 (0.0008) [2023-12-27 03:42:55,048][105620] Updated weights for policy 1, policy_version 1694835 (0.0008) [2023-12-27 03:42:55,103][105620] Updated weights for policy 1, policy_version 1694845 (0.0009) [2023-12-27 03:42:55,165][105620] Updated weights for policy 1, policy_version 1694855 (0.0007) [2023-12-27 03:42:55,298][105692] Updated weights for policy 0, policy_version 1691138 (0.0009) [2023-12-27 03:42:55,349][105692] Updated weights for policy 0, policy_version 1691148 (0.0009) [2023-12-27 03:42:55,402][105692] Updated weights for policy 0, policy_version 1691158 (0.0008) [2023-12-27 03:42:55,452][105692] Updated weights for policy 0, policy_version 1691168 (0.0009) [2023-12-27 03:42:55,901][105620] Updated weights for policy 1, policy_version 1694865 (0.0008) [2023-12-27 03:42:55,962][105620] Updated weights for policy 1, policy_version 1694875 (0.0009) [2023-12-27 03:42:56,026][105620] Updated weights for policy 1, policy_version 1694885 (0.0008) [2023-12-27 03:42:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 866951168. Throughput: 0: 9675.9, 1: 9842.6. Samples: 866966756. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:42:56,062][104569] Avg episode reward: [(0, '8528.938'), (1, '8528.115')] [2023-12-27 03:42:56,087][105620] Updated weights for policy 1, policy_version 1694895 (0.0009) [2023-12-27 03:42:56,216][105692] Updated weights for policy 0, policy_version 1691178 (0.0010) [2023-12-27 03:42:56,275][105692] Updated weights for policy 0, policy_version 1691189 (0.0009) [2023-12-27 03:42:56,323][105692] Updated weights for policy 0, policy_version 1691199 (0.0009) [2023-12-27 03:42:56,762][105620] Updated weights for policy 1, policy_version 1694905 (0.0009) [2023-12-27 03:42:56,810][105620] Updated weights for policy 1, policy_version 1694915 (0.0008) [2023-12-27 03:42:56,860][105620] Updated weights for policy 1, policy_version 1694925 (0.0008) [2023-12-27 03:42:57,147][105692] Updated weights for policy 0, policy_version 1691209 (0.0009) [2023-12-27 03:42:57,199][105692] Updated weights for policy 0, policy_version 1691219 (0.0009) [2023-12-27 03:42:57,256][105692] Updated weights for policy 0, policy_version 1691230 (0.0010) [2023-12-27 03:42:57,522][105620] Updated weights for policy 1, policy_version 1694935 (0.0009) [2023-12-27 03:42:57,568][105620] Updated weights for policy 1, policy_version 1694945 (0.0008) [2023-12-27 03:42:57,630][105620] Updated weights for policy 1, policy_version 1694955 (0.0008) [2023-12-27 03:42:58,005][105692] Updated weights for policy 0, policy_version 1691240 (0.0009) [2023-12-27 03:42:58,052][105692] Updated weights for policy 0, policy_version 1691250 (0.0008) [2023-12-27 03:42:58,103][105692] Updated weights for policy 0, policy_version 1691260 (0.0008) [2023-12-27 03:42:58,262][105620] Updated weights for policy 1, policy_version 1694965 (0.0008) [2023-12-27 03:42:58,322][105620] Updated weights for policy 1, policy_version 1694975 (0.0011) [2023-12-27 03:42:58,392][105620] Updated weights for policy 1, policy_version 1694985 (0.0008) [2023-12-27 03:42:59,076][105692] Updated weights for policy 0, policy_version 1691270 (0.0008) [2023-12-27 03:42:59,131][105692] Updated weights for policy 0, policy_version 1691280 (0.0005) [2023-12-27 03:42:59,192][105692] Updated weights for policy 0, policy_version 1691290 (0.0006) [2023-12-27 03:42:59,198][105620] Updated weights for policy 1, policy_version 1694995 (0.0008) [2023-12-27 03:42:59,272][105620] Updated weights for policy 1, policy_version 1695005 (0.0009) [2023-12-27 03:42:59,339][105620] Updated weights for policy 1, policy_version 1695015 (0.0008) [2023-12-27 03:42:59,977][105692] Updated weights for policy 0, policy_version 1691300 (0.0007) [2023-12-27 03:43:00,029][105692] Updated weights for policy 0, policy_version 1691310 (0.0010) [2023-12-27 03:43:00,043][105620] Updated weights for policy 1, policy_version 1695025 (0.0007) [2023-12-27 03:43:00,089][105692] Updated weights for policy 0, policy_version 1691320 (0.0008) [2023-12-27 03:43:00,102][105620] Updated weights for policy 1, policy_version 1695035 (0.0006) [2023-12-27 03:43:00,167][105620] Updated weights for policy 1, policy_version 1695045 (0.0008) [2023-12-27 03:43:00,232][105620] Updated weights for policy 1, policy_version 1695055 (0.0007) [2023-12-27 03:43:00,806][105620] Updated weights for policy 1, policy_version 1695065 (0.0008) [2023-12-27 03:43:00,852][105620] Updated weights for policy 1, policy_version 1695075 (0.0008) [2023-12-27 03:43:00,898][105620] Updated weights for policy 1, policy_version 1695085 (0.0008) [2023-12-27 03:43:00,928][105692] Updated weights for policy 0, policy_version 1691330 (0.0008) [2023-12-27 03:43:00,987][105692] Updated weights for policy 0, policy_version 1691340 (0.0009) [2023-12-27 03:43:01,054][105692] Updated weights for policy 0, policy_version 1691350 (0.0007) [2023-12-27 03:43:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 867049472. Throughput: 0: 9617.5, 1: 9838.2. Samples: 867024016. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:43:01,063][104569] Avg episode reward: [(0, '8626.512'), (1, '8895.580')] [2023-12-27 03:43:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001695088_434003968.pth... [2023-12-27 03:43:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001693904_433700864.pth [2023-12-27 03:43:01,105][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001691360_433053696.pth... [2023-12-27 03:43:01,106][105692] Updated weights for policy 0, policy_version 1691360 (0.0008) [2023-12-27 03:43:01,108][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001690240_432766976.pth [2023-12-27 03:43:01,567][105620] Updated weights for policy 1, policy_version 1695095 (0.0006) [2023-12-27 03:43:01,631][105620] Updated weights for policy 1, policy_version 1695105 (0.0009) [2023-12-27 03:43:01,685][105620] Updated weights for policy 1, policy_version 1695115 (0.0009) [2023-12-27 03:43:01,932][105692] Updated weights for policy 0, policy_version 1691370 (0.0009) [2023-12-27 03:43:01,983][105692] Updated weights for policy 0, policy_version 1691380 (0.0009) [2023-12-27 03:43:02,038][105692] Updated weights for policy 0, policy_version 1691390 (0.0009) [2023-12-27 03:43:02,389][105620] Updated weights for policy 1, policy_version 1695125 (0.0010) [2023-12-27 03:43:02,448][105620] Updated weights for policy 1, policy_version 1695135 (0.0010) [2023-12-27 03:43:02,506][105620] Updated weights for policy 1, policy_version 1695145 (0.0010) [2023-12-27 03:43:02,820][105692] Updated weights for policy 0, policy_version 1691400 (0.0009) [2023-12-27 03:43:02,873][105692] Updated weights for policy 0, policy_version 1691410 (0.0008) [2023-12-27 03:43:02,931][105692] Updated weights for policy 0, policy_version 1691420 (0.0008) [2023-12-27 03:43:03,206][105620] Updated weights for policy 1, policy_version 1695155 (0.0010) [2023-12-27 03:43:03,253][105620] Updated weights for policy 1, policy_version 1695165 (0.0010) [2023-12-27 03:43:03,308][105620] Updated weights for policy 1, policy_version 1695175 (0.0010) [2023-12-27 03:43:03,686][105692] Updated weights for policy 0, policy_version 1691430 (0.0008) [2023-12-27 03:43:03,743][105692] Updated weights for policy 0, policy_version 1691440 (0.0008) [2023-12-27 03:43:03,801][105692] Updated weights for policy 0, policy_version 1691450 (0.0008) [2023-12-27 03:43:04,068][105620] Updated weights for policy 1, policy_version 1695185 (0.0010) [2023-12-27 03:43:04,119][105620] Updated weights for policy 1, policy_version 1695195 (0.0010) [2023-12-27 03:43:04,175][105620] Updated weights for policy 1, policy_version 1695205 (0.0010) [2023-12-27 03:43:04,240][105620] Updated weights for policy 1, policy_version 1695215 (0.0010) [2023-12-27 03:43:04,500][105692] Updated weights for policy 0, policy_version 1691460 (0.0007) [2023-12-27 03:43:04,565][105692] Updated weights for policy 0, policy_version 1691470 (0.0005) [2023-12-27 03:43:04,633][105692] Updated weights for policy 0, policy_version 1691480 (0.0006) [2023-12-27 03:43:05,027][105620] Updated weights for policy 1, policy_version 1695225 (0.0006) [2023-12-27 03:43:05,075][105620] Updated weights for policy 1, policy_version 1695235 (0.0005) [2023-12-27 03:43:05,124][105620] Updated weights for policy 1, policy_version 1695245 (0.0006) [2023-12-27 03:43:05,210][105692] Updated weights for policy 0, policy_version 1691490 (0.0007) [2023-12-27 03:43:05,271][105692] Updated weights for policy 0, policy_version 1691500 (0.0005) [2023-12-27 03:43:05,325][105692] Updated weights for policy 0, policy_version 1691510 (0.0006) [2023-12-27 03:43:05,371][105692] Updated weights for policy 0, policy_version 1691520 (0.0005) [2023-12-27 03:43:05,740][105620] Updated weights for policy 1, policy_version 1695255 (0.0009) [2023-12-27 03:43:05,788][105620] Updated weights for policy 1, policy_version 1695265 (0.0010) [2023-12-27 03:43:05,836][105620] Updated weights for policy 1, policy_version 1695275 (0.0010) [2023-12-27 03:43:06,061][105692] Updated weights for policy 0, policy_version 1691530 (0.0009) [2023-12-27 03:43:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 867147776. Throughput: 0: 9500.8, 1: 9824.6. Samples: 867137116. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 03:43:06,062][104569] Avg episode reward: [(0, '8623.223'), (1, '9180.481')] [2023-12-27 03:43:06,123][105692] Updated weights for policy 0, policy_version 1691540 (0.0008) [2023-12-27 03:43:06,189][105692] Updated weights for policy 0, policy_version 1691550 (0.0009) [2023-12-27 03:43:06,598][105620] Updated weights for policy 1, policy_version 1695285 (0.0011) [2023-12-27 03:43:06,661][105620] Updated weights for policy 1, policy_version 1695295 (0.0011) [2023-12-27 03:43:06,727][105620] Updated weights for policy 1, policy_version 1695305 (0.0011) [2023-12-27 03:43:06,971][105692] Updated weights for policy 0, policy_version 1691560 (0.0010) [2023-12-27 03:43:07,024][105692] Updated weights for policy 0, policy_version 1691570 (0.0010) [2023-12-27 03:43:07,082][105692] Updated weights for policy 0, policy_version 1691580 (0.0010) [2023-12-27 03:43:07,323][105620] Updated weights for policy 1, policy_version 1695315 (0.0009) [2023-12-27 03:43:07,386][105620] Updated weights for policy 1, policy_version 1695325 (0.0008) [2023-12-27 03:43:07,442][105620] Updated weights for policy 1, policy_version 1695335 (0.0011) [2023-12-27 03:43:07,919][105692] Updated weights for policy 0, policy_version 1691590 (0.0009) [2023-12-27 03:43:07,970][105692] Updated weights for policy 0, policy_version 1691600 (0.0008) [2023-12-27 03:43:08,038][105692] Updated weights for policy 0, policy_version 1691610 (0.0008) [2023-12-27 03:43:08,147][105620] Updated weights for policy 1, policy_version 1695345 (0.0011) [2023-12-27 03:43:08,202][105620] Updated weights for policy 1, policy_version 1695355 (0.0010) [2023-12-27 03:43:08,253][105620] Updated weights for policy 1, policy_version 1695365 (0.0010) [2023-12-27 03:43:08,301][105620] Updated weights for policy 1, policy_version 1695375 (0.0010) [2023-12-27 03:43:08,737][105692] Updated weights for policy 0, policy_version 1691620 (0.0008) [2023-12-27 03:43:08,788][105692] Updated weights for policy 0, policy_version 1691630 (0.0008) [2023-12-27 03:43:08,838][105692] Updated weights for policy 0, policy_version 1691640 (0.0006) [2023-12-27 03:43:09,088][105620] Updated weights for policy 1, policy_version 1695385 (0.0010) [2023-12-27 03:43:09,150][105620] Updated weights for policy 1, policy_version 1695395 (0.0010) [2023-12-27 03:43:09,214][105620] Updated weights for policy 1, policy_version 1695405 (0.0011) [2023-12-27 03:43:09,522][105692] Updated weights for policy 0, policy_version 1691650 (0.0006) [2023-12-27 03:43:09,596][105692] Updated weights for policy 0, policy_version 1691660 (0.0008) [2023-12-27 03:43:09,650][105692] Updated weights for policy 0, policy_version 1691670 (0.0010) [2023-12-27 03:43:09,704][105692] Updated weights for policy 0, policy_version 1691680 (0.0010) [2023-12-27 03:43:09,887][105620] Updated weights for policy 1, policy_version 1695415 (0.0012) [2023-12-27 03:43:09,949][105620] Updated weights for policy 1, policy_version 1695425 (0.0009) [2023-12-27 03:43:10,014][105620] Updated weights for policy 1, policy_version 1695435 (0.0008) [2023-12-27 03:43:10,418][105692] Updated weights for policy 0, policy_version 1691690 (0.0006) [2023-12-27 03:43:10,481][105692] Updated weights for policy 0, policy_version 1691700 (0.0007) [2023-12-27 03:43:10,539][105692] Updated weights for policy 0, policy_version 1691710 (0.0009) [2023-12-27 03:43:10,768][105620] Updated weights for policy 1, policy_version 1695445 (0.0007) [2023-12-27 03:43:10,845][105620] Updated weights for policy 1, policy_version 1695455 (0.0006) [2023-12-27 03:43:10,902][105620] Updated weights for policy 1, policy_version 1695465 (0.0009) [2023-12-27 03:43:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 867246080. Throughput: 0: 9554.8, 1: 9878.9. Samples: 867254420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:43:11,062][104569] Avg episode reward: [(0, '8433.937'), (1, '8921.985')] [2023-12-27 03:43:11,202][105692] Updated weights for policy 0, policy_version 1691720 (0.0008) [2023-12-27 03:43:11,260][105692] Updated weights for policy 0, policy_version 1691730 (0.0010) [2023-12-27 03:43:11,319][105692] Updated weights for policy 0, policy_version 1691740 (0.0009) [2023-12-27 03:43:11,611][105620] Updated weights for policy 1, policy_version 1695476 (0.0009) [2023-12-27 03:43:11,678][105620] Updated weights for policy 1, policy_version 1695486 (0.0009) [2023-12-27 03:43:11,745][105620] Updated weights for policy 1, policy_version 1695496 (0.0010) [2023-12-27 03:43:12,184][105692] Updated weights for policy 0, policy_version 1691750 (0.0008) [2023-12-27 03:43:12,250][105692] Updated weights for policy 0, policy_version 1691760 (0.0008) [2023-12-27 03:43:12,317][105692] Updated weights for policy 0, policy_version 1691770 (0.0007) [2023-12-27 03:43:12,400][105620] Updated weights for policy 1, policy_version 1695506 (0.0009) [2023-12-27 03:43:12,463][105620] Updated weights for policy 1, policy_version 1695516 (0.0009) [2023-12-27 03:43:12,522][105620] Updated weights for policy 1, policy_version 1695526 (0.0009) [2023-12-27 03:43:12,584][105620] Updated weights for policy 1, policy_version 1695536 (0.0009) [2023-12-27 03:43:13,048][105692] Updated weights for policy 0, policy_version 1691780 (0.0007) [2023-12-27 03:43:13,104][105692] Updated weights for policy 0, policy_version 1691790 (0.0008) [2023-12-27 03:43:13,151][105692] Updated weights for policy 0, policy_version 1691800 (0.0008) [2023-12-27 03:43:13,389][105620] Updated weights for policy 1, policy_version 1695546 (0.0009) [2023-12-27 03:43:13,451][105620] Updated weights for policy 1, policy_version 1695556 (0.0009) [2023-12-27 03:43:13,507][105620] Updated weights for policy 1, policy_version 1695566 (0.0009) [2023-12-27 03:43:13,794][105692] Updated weights for policy 0, policy_version 1691810 (0.0009) [2023-12-27 03:43:13,859][105692] Updated weights for policy 0, policy_version 1691820 (0.0005) [2023-12-27 03:43:13,907][105692] Updated weights for policy 0, policy_version 1691830 (0.0005) [2023-12-27 03:43:13,960][105692] Updated weights for policy 0, policy_version 1691840 (0.0005) [2023-12-27 03:43:14,289][105620] Updated weights for policy 1, policy_version 1695576 (0.0006) [2023-12-27 03:43:14,349][105620] Updated weights for policy 1, policy_version 1695586 (0.0005) [2023-12-27 03:43:14,394][105620] Updated weights for policy 1, policy_version 1695596 (0.0005) [2023-12-27 03:43:14,635][105692] Updated weights for policy 0, policy_version 1691850 (0.0008) [2023-12-27 03:43:14,695][105692] Updated weights for policy 0, policy_version 1691860 (0.0009) [2023-12-27 03:43:14,751][105692] Updated weights for policy 0, policy_version 1691871 (0.0009) [2023-12-27 03:43:15,067][105620] Updated weights for policy 1, policy_version 1695606 (0.0009) [2023-12-27 03:43:15,134][105620] Updated weights for policy 1, policy_version 1695616 (0.0009) [2023-12-27 03:43:15,198][105620] Updated weights for policy 1, policy_version 1695626 (0.0009) [2023-12-27 03:43:15,532][105692] Updated weights for policy 0, policy_version 1691881 (0.0009) [2023-12-27 03:43:15,583][105692] Updated weights for policy 0, policy_version 1691891 (0.0008) [2023-12-27 03:43:15,629][105692] Updated weights for policy 0, policy_version 1691901 (0.0008) [2023-12-27 03:43:15,961][105620] Updated weights for policy 1, policy_version 1695636 (0.0009) [2023-12-27 03:43:16,019][105620] Updated weights for policy 1, policy_version 1695646 (0.0009) [2023-12-27 03:43:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 867336192. Throughput: 0: 9563.4, 1: 9855.4. Samples: 867309920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:43:16,063][104569] Avg episode reward: [(0, '8164.874'), (1, '8738.070')] [2023-12-27 03:43:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001691904_433192960.pth... [2023-12-27 03:43:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001690816_432914432.pth [2023-12-27 03:43:16,086][105620] Updated weights for policy 1, policy_version 1695656 (0.0010) [2023-12-27 03:43:16,128][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001695664_434151424.pth... [2023-12-27 03:43:16,131][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001694480_433848320.pth [2023-12-27 03:43:16,400][105692] Updated weights for policy 0, policy_version 1691911 (0.0009) [2023-12-27 03:43:16,466][105692] Updated weights for policy 0, policy_version 1691921 (0.0009) [2023-12-27 03:43:16,527][105692] Updated weights for policy 0, policy_version 1691931 (0.0008) [2023-12-27 03:43:16,853][105620] Updated weights for policy 1, policy_version 1695666 (0.0009) [2023-12-27 03:43:16,906][105620] Updated weights for policy 1, policy_version 1695676 (0.0008) [2023-12-27 03:43:16,961][105620] Updated weights for policy 1, policy_version 1695686 (0.0008) [2023-12-27 03:43:17,014][105620] Updated weights for policy 1, policy_version 1695696 (0.0009) [2023-12-27 03:43:17,242][105692] Updated weights for policy 0, policy_version 1691941 (0.0009) [2023-12-27 03:43:17,296][105692] Updated weights for policy 0, policy_version 1691951 (0.0009) [2023-12-27 03:43:17,358][105692] Updated weights for policy 0, policy_version 1691961 (0.0009) [2023-12-27 03:43:17,759][105620] Updated weights for policy 1, policy_version 1695706 (0.0009) [2023-12-27 03:43:17,807][105620] Updated weights for policy 1, policy_version 1695716 (0.0009) [2023-12-27 03:43:17,869][105620] Updated weights for policy 1, policy_version 1695726 (0.0009) [2023-12-27 03:43:18,114][105692] Updated weights for policy 0, policy_version 1691971 (0.0009) [2023-12-27 03:43:18,175][105692] Updated weights for policy 0, policy_version 1691981 (0.0009) [2023-12-27 03:43:18,240][105692] Updated weights for policy 0, policy_version 1691991 (0.0009) [2023-12-27 03:43:18,670][105620] Updated weights for policy 1, policy_version 1695736 (0.0009) [2023-12-27 03:43:18,734][105620] Updated weights for policy 1, policy_version 1695746 (0.0009) [2023-12-27 03:43:18,789][105620] Updated weights for policy 1, policy_version 1695756 (0.0009) [2023-12-27 03:43:18,929][105692] Updated weights for policy 0, policy_version 1692001 (0.0010) [2023-12-27 03:43:18,984][105692] Updated weights for policy 0, policy_version 1692011 (0.0009) [2023-12-27 03:43:19,035][105692] Updated weights for policy 0, policy_version 1692021 (0.0009) [2023-12-27 03:43:19,087][105692] Updated weights for policy 0, policy_version 1692031 (0.0009) [2023-12-27 03:43:19,468][105620] Updated weights for policy 1, policy_version 1695766 (0.0008) [2023-12-27 03:43:19,535][105620] Updated weights for policy 1, policy_version 1695776 (0.0008) [2023-12-27 03:43:19,593][105620] Updated weights for policy 1, policy_version 1695786 (0.0008) [2023-12-27 03:43:19,945][105692] Updated weights for policy 0, policy_version 1692041 (0.0010) [2023-12-27 03:43:20,010][105692] Updated weights for policy 0, policy_version 1692051 (0.0010) [2023-12-27 03:43:20,065][105692] Updated weights for policy 0, policy_version 1692062 (0.0011) [2023-12-27 03:43:20,284][105620] Updated weights for policy 1, policy_version 1695796 (0.0007) [2023-12-27 03:43:20,354][105620] Updated weights for policy 1, policy_version 1695806 (0.0006) [2023-12-27 03:43:20,419][105620] Updated weights for policy 1, policy_version 1695816 (0.0008) [2023-12-27 03:43:20,830][105692] Updated weights for policy 0, policy_version 1692072 (0.0010) [2023-12-27 03:43:20,893][105692] Updated weights for policy 0, policy_version 1692082 (0.0008) [2023-12-27 03:43:20,956][105692] Updated weights for policy 0, policy_version 1692092 (0.0006) [2023-12-27 03:43:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 867434496. Throughput: 0: 9484.4, 1: 9788.9. Samples: 867423876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:43:21,063][104569] Avg episode reward: [(0, '8352.083'), (1, '8820.829')] [2023-12-27 03:43:21,125][105620] Updated weights for policy 1, policy_version 1695826 (0.0006) [2023-12-27 03:43:21,195][105620] Updated weights for policy 1, policy_version 1695836 (0.0006) [2023-12-27 03:43:21,266][105620] Updated weights for policy 1, policy_version 1695846 (0.0006) [2023-12-27 03:43:21,340][105620] Updated weights for policy 1, policy_version 1695856 (0.0006) [2023-12-27 03:43:21,646][105692] Updated weights for policy 0, policy_version 1692102 (0.0008) [2023-12-27 03:43:21,705][105692] Updated weights for policy 0, policy_version 1692112 (0.0006) [2023-12-27 03:43:21,774][105692] Updated weights for policy 0, policy_version 1692122 (0.0011) [2023-12-27 03:43:22,105][105620] Updated weights for policy 1, policy_version 1695866 (0.0009) [2023-12-27 03:43:22,166][105620] Updated weights for policy 1, policy_version 1695876 (0.0009) [2023-12-27 03:43:22,233][105620] Updated weights for policy 1, policy_version 1695886 (0.0008) [2023-12-27 03:43:22,481][105692] Updated weights for policy 0, policy_version 1692132 (0.0009) [2023-12-27 03:43:22,546][105692] Updated weights for policy 0, policy_version 1692142 (0.0011) [2023-12-27 03:43:22,613][105692] Updated weights for policy 0, policy_version 1692152 (0.0011) [2023-12-27 03:43:23,059][105620] Updated weights for policy 1, policy_version 1695896 (0.0008) [2023-12-27 03:43:23,127][105620] Updated weights for policy 1, policy_version 1695906 (0.0008) [2023-12-27 03:43:23,175][105620] Updated weights for policy 1, policy_version 1695916 (0.0008) [2023-12-27 03:43:23,296][105692] Updated weights for policy 0, policy_version 1692162 (0.0010) [2023-12-27 03:43:23,344][105692] Updated weights for policy 0, policy_version 1692172 (0.0010) [2023-12-27 03:43:23,398][105692] Updated weights for policy 0, policy_version 1692182 (0.0010) [2023-12-27 03:43:23,449][105692] Updated weights for policy 0, policy_version 1692192 (0.0010) [2023-12-27 03:43:23,943][105620] Updated weights for policy 1, policy_version 1695926 (0.0009) [2023-12-27 03:43:23,998][105620] Updated weights for policy 1, policy_version 1695936 (0.0009) [2023-12-27 03:43:24,028][105692] Updated weights for policy 0, policy_version 1692202 (0.0007) [2023-12-27 03:43:24,047][105620] Updated weights for policy 1, policy_version 1695946 (0.0006) [2023-12-27 03:43:24,080][105692] Updated weights for policy 0, policy_version 1692212 (0.0010) [2023-12-27 03:43:24,140][105692] Updated weights for policy 0, policy_version 1692222 (0.0010) [2023-12-27 03:43:24,701][105620] Updated weights for policy 1, policy_version 1695956 (0.0006) [2023-12-27 03:43:24,754][105620] Updated weights for policy 1, policy_version 1695966 (0.0006) [2023-12-27 03:43:24,803][105620] Updated weights for policy 1, policy_version 1695976 (0.0005) [2023-12-27 03:43:24,913][105692] Updated weights for policy 0, policy_version 1692232 (0.0008) [2023-12-27 03:43:24,984][105692] Updated weights for policy 0, policy_version 1692242 (0.0009) [2023-12-27 03:43:25,047][105692] Updated weights for policy 0, policy_version 1692252 (0.0010) [2023-12-27 03:43:25,345][105620] Updated weights for policy 1, policy_version 1695986 (0.0005) [2023-12-27 03:43:25,403][105620] Updated weights for policy 1, policy_version 1695996 (0.0008) [2023-12-27 03:43:25,460][105620] Updated weights for policy 1, policy_version 1696006 (0.0009) [2023-12-27 03:43:25,512][105620] Updated weights for policy 1, policy_version 1696016 (0.0008) [2023-12-27 03:43:25,726][105692] Updated weights for policy 0, policy_version 1692262 (0.0010) [2023-12-27 03:43:25,774][105692] Updated weights for policy 0, policy_version 1692272 (0.0010) [2023-12-27 03:43:25,832][105692] Updated weights for policy 0, policy_version 1692282 (0.0010) [2023-12-27 03:43:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 867532800. Throughput: 0: 9439.1, 1: 9798.8. Samples: 867541544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:43:26,063][104569] Avg episode reward: [(0, '9081.476'), (1, '9179.634')] [2023-12-27 03:43:26,252][105620] Updated weights for policy 1, policy_version 1696026 (0.0010) [2023-12-27 03:43:26,312][105620] Updated weights for policy 1, policy_version 1696036 (0.0010) [2023-12-27 03:43:26,360][105620] Updated weights for policy 1, policy_version 1696046 (0.0007) [2023-12-27 03:43:26,511][105692] Updated weights for policy 0, policy_version 1692292 (0.0009) [2023-12-27 03:43:26,566][105692] Updated weights for policy 0, policy_version 1692302 (0.0011) [2023-12-27 03:43:26,615][105692] Updated weights for policy 0, policy_version 1692312 (0.0011) [2023-12-27 03:43:26,973][105620] Updated weights for policy 1, policy_version 1696056 (0.0010) [2023-12-27 03:43:27,022][105620] Updated weights for policy 1, policy_version 1696066 (0.0010) [2023-12-27 03:43:27,079][105620] Updated weights for policy 1, policy_version 1696076 (0.0005) [2023-12-27 03:43:27,362][105692] Updated weights for policy 0, policy_version 1692322 (0.0010) [2023-12-27 03:43:27,422][105692] Updated weights for policy 0, policy_version 1692332 (0.0010) [2023-12-27 03:43:27,486][105692] Updated weights for policy 0, policy_version 1692342 (0.0010) [2023-12-27 03:43:27,550][105692] Updated weights for policy 0, policy_version 1692352 (0.0010) [2023-12-27 03:43:27,773][105620] Updated weights for policy 1, policy_version 1696086 (0.0007) [2023-12-27 03:43:27,836][105620] Updated weights for policy 1, policy_version 1696096 (0.0007) [2023-12-27 03:43:27,893][105620] Updated weights for policy 1, policy_version 1696106 (0.0008) [2023-12-27 03:43:28,200][105692] Updated weights for policy 0, policy_version 1692362 (0.0005) [2023-12-27 03:43:28,256][105692] Updated weights for policy 0, policy_version 1692372 (0.0005) [2023-12-27 03:43:28,307][105692] Updated weights for policy 0, policy_version 1692382 (0.0006) [2023-12-27 03:43:28,618][105620] Updated weights for policy 1, policy_version 1696116 (0.0009) [2023-12-27 03:43:28,676][105620] Updated weights for policy 1, policy_version 1696126 (0.0007) [2023-12-27 03:43:28,741][105620] Updated weights for policy 1, policy_version 1696136 (0.0007) [2023-12-27 03:43:28,950][105692] Updated weights for policy 0, policy_version 1692392 (0.0006) [2023-12-27 03:43:29,001][105692] Updated weights for policy 0, policy_version 1692402 (0.0005) [2023-12-27 03:43:29,061][105692] Updated weights for policy 0, policy_version 1692412 (0.0005) [2023-12-27 03:43:29,423][105620] Updated weights for policy 1, policy_version 1696146 (0.0009) [2023-12-27 03:43:29,488][105620] Updated weights for policy 1, policy_version 1696156 (0.0007) [2023-12-27 03:43:29,547][105620] Updated weights for policy 1, policy_version 1696166 (0.0006) [2023-12-27 03:43:29,597][105620] Updated weights for policy 1, policy_version 1696176 (0.0006) [2023-12-27 03:43:29,701][105692] Updated weights for policy 0, policy_version 1692422 (0.0008) [2023-12-27 03:43:29,746][105692] Updated weights for policy 0, policy_version 1692432 (0.0010) [2023-12-27 03:43:29,797][105692] Updated weights for policy 0, policy_version 1692442 (0.0011) [2023-12-27 03:43:30,340][105620] Updated weights for policy 1, policy_version 1696186 (0.0008) [2023-12-27 03:43:30,397][105620] Updated weights for policy 1, policy_version 1696196 (0.0008) [2023-12-27 03:43:30,454][105620] Updated weights for policy 1, policy_version 1696206 (0.0010) [2023-12-27 03:43:30,536][105692] Updated weights for policy 0, policy_version 1692452 (0.0008) [2023-12-27 03:43:30,589][105692] Updated weights for policy 0, policy_version 1692462 (0.0006) [2023-12-27 03:43:30,647][105692] Updated weights for policy 0, policy_version 1692472 (0.0006) [2023-12-27 03:43:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 867631104. Throughput: 0: 9501.9, 1: 9843.8. Samples: 867601604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:43:31,062][104569] Avg episode reward: [(0, '8719.279'), (1, '9079.180')] [2023-12-27 03:43:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001692480_433340416.pth... [2023-12-27 03:43:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001696208_434290688.pth... [2023-12-27 03:43:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001691360_433053696.pth [2023-12-27 03:43:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001695088_434003968.pth [2023-12-27 03:43:31,216][105620] Updated weights for policy 1, policy_version 1696216 (0.0008) [2023-12-27 03:43:31,274][105620] Updated weights for policy 1, policy_version 1696226 (0.0008) [2023-12-27 03:43:31,329][105692] Updated weights for policy 0, policy_version 1692482 (0.0006) [2023-12-27 03:43:31,342][105620] Updated weights for policy 1, policy_version 1696236 (0.0009) [2023-12-27 03:43:31,392][105692] Updated weights for policy 0, policy_version 1692492 (0.0009) [2023-12-27 03:43:31,447][105692] Updated weights for policy 0, policy_version 1692502 (0.0006) [2023-12-27 03:43:31,499][105692] Updated weights for policy 0, policy_version 1692512 (0.0007) [2023-12-27 03:43:32,101][105620] Updated weights for policy 1, policy_version 1696246 (0.0009) [2023-12-27 03:43:32,161][105620] Updated weights for policy 1, policy_version 1696256 (0.0011) [2023-12-27 03:43:32,220][105692] Updated weights for policy 0, policy_version 1692522 (0.0008) [2023-12-27 03:43:32,222][105620] Updated weights for policy 1, policy_version 1696266 (0.0010) [2023-12-27 03:43:32,278][105692] Updated weights for policy 0, policy_version 1692532 (0.0007) [2023-12-27 03:43:32,335][105692] Updated weights for policy 0, policy_version 1692542 (0.0008) [2023-12-27 03:43:32,940][105692] Updated weights for policy 0, policy_version 1692552 (0.0006) [2023-12-27 03:43:32,966][105620] Updated weights for policy 1, policy_version 1696276 (0.0009) [2023-12-27 03:43:33,003][105692] Updated weights for policy 0, policy_version 1692562 (0.0005) [2023-12-27 03:43:33,027][105620] Updated weights for policy 1, policy_version 1696286 (0.0008) [2023-12-27 03:43:33,068][105692] Updated weights for policy 0, policy_version 1692572 (0.0009) [2023-12-27 03:43:33,074][105620] Updated weights for policy 1, policy_version 1696296 (0.0005) [2023-12-27 03:43:33,657][105620] Updated weights for policy 1, policy_version 1696306 (0.0005) [2023-12-27 03:43:33,688][105692] Updated weights for policy 0, policy_version 1692582 (0.0007) [2023-12-27 03:43:33,716][105620] Updated weights for policy 1, policy_version 1696316 (0.0009) [2023-12-27 03:43:33,743][105692] Updated weights for policy 0, policy_version 1692592 (0.0006) [2023-12-27 03:43:33,769][105620] Updated weights for policy 1, policy_version 1696326 (0.0006) [2023-12-27 03:43:33,796][105692] Updated weights for policy 0, policy_version 1692602 (0.0009) [2023-12-27 03:43:33,819][105620] Updated weights for policy 1, policy_version 1696336 (0.0005) [2023-12-27 03:43:34,397][105620] Updated weights for policy 1, policy_version 1696346 (0.0009) [2023-12-27 03:43:34,459][105620] Updated weights for policy 1, policy_version 1696356 (0.0009) [2023-12-27 03:43:34,512][105620] Updated weights for policy 1, policy_version 1696366 (0.0009) [2023-12-27 03:43:34,550][105692] Updated weights for policy 0, policy_version 1692612 (0.0007) [2023-12-27 03:43:34,606][105692] Updated weights for policy 0, policy_version 1692622 (0.0006) [2023-12-27 03:43:34,660][105692] Updated weights for policy 0, policy_version 1692632 (0.0005) [2023-12-27 03:43:35,215][105692] Updated weights for policy 0, policy_version 1692642 (0.0005) [2023-12-27 03:43:35,227][105620] Updated weights for policy 1, policy_version 1696376 (0.0009) [2023-12-27 03:43:35,267][105692] Updated weights for policy 0, policy_version 1692652 (0.0006) [2023-12-27 03:43:35,279][105620] Updated weights for policy 1, policy_version 1696386 (0.0008) [2023-12-27 03:43:35,320][105692] Updated weights for policy 0, policy_version 1692662 (0.0005) [2023-12-27 03:43:35,338][105620] Updated weights for policy 1, policy_version 1696396 (0.0009) [2023-12-27 03:43:35,370][105692] Updated weights for policy 0, policy_version 1692672 (0.0009) [2023-12-27 03:43:36,025][105692] Updated weights for policy 0, policy_version 1692682 (0.0009) [2023-12-27 03:43:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 867729408. Throughput: 0: 9538.5, 1: 9830.0. Samples: 867722980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:43:36,063][104569] Avg episode reward: [(0, '8531.963'), (1, '8804.731')] [2023-12-27 03:43:36,085][105692] Updated weights for policy 0, policy_version 1692692 (0.0009) [2023-12-27 03:43:36,148][105692] Updated weights for policy 0, policy_version 1692702 (0.0011) [2023-12-27 03:43:36,159][105620] Updated weights for policy 1, policy_version 1696406 (0.0007) [2023-12-27 03:43:36,216][105620] Updated weights for policy 1, policy_version 1696416 (0.0008) [2023-12-27 03:43:36,264][105620] Updated weights for policy 1, policy_version 1696426 (0.0008) [2023-12-27 03:43:36,864][105692] Updated weights for policy 0, policy_version 1692712 (0.0011) [2023-12-27 03:43:36,920][105692] Updated weights for policy 0, policy_version 1692722 (0.0011) [2023-12-27 03:43:36,983][105692] Updated weights for policy 0, policy_version 1692732 (0.0009) [2023-12-27 03:43:37,000][105620] Updated weights for policy 1, policy_version 1696436 (0.0008) [2023-12-27 03:43:37,063][105620] Updated weights for policy 1, policy_version 1696446 (0.0008) [2023-12-27 03:43:37,117][105620] Updated weights for policy 1, policy_version 1696456 (0.0008) [2023-12-27 03:43:37,650][105692] Updated weights for policy 0, policy_version 1692742 (0.0010) [2023-12-27 03:43:37,702][105692] Updated weights for policy 0, policy_version 1692752 (0.0010) [2023-12-27 03:43:37,767][105692] Updated weights for policy 0, policy_version 1692762 (0.0010) [2023-12-27 03:43:37,772][105620] Updated weights for policy 1, policy_version 1696466 (0.0008) [2023-12-27 03:43:37,829][105620] Updated weights for policy 1, policy_version 1696476 (0.0005) [2023-12-27 03:43:37,876][105620] Updated weights for policy 1, policy_version 1696486 (0.0009) [2023-12-27 03:43:37,929][105620] Updated weights for policy 1, policy_version 1696496 (0.0010) [2023-12-27 03:43:38,457][105692] Updated weights for policy 0, policy_version 1692772 (0.0010) [2023-12-27 03:43:38,520][105692] Updated weights for policy 0, policy_version 1692782 (0.0011) [2023-12-27 03:43:38,582][105692] Updated weights for policy 0, policy_version 1692792 (0.0011) [2023-12-27 03:43:38,660][105620] Updated weights for policy 1, policy_version 1696506 (0.0010) [2023-12-27 03:43:38,718][105620] Updated weights for policy 1, policy_version 1696516 (0.0009) [2023-12-27 03:43:38,766][105620] Updated weights for policy 1, policy_version 1696526 (0.0010) [2023-12-27 03:43:39,368][105692] Updated weights for policy 0, policy_version 1692802 (0.0011) [2023-12-27 03:43:39,441][105692] Updated weights for policy 0, policy_version 1692812 (0.0014) [2023-12-27 03:43:39,499][105692] Updated weights for policy 0, policy_version 1692822 (0.0009) [2023-12-27 03:43:39,533][105620] Updated weights for policy 1, policy_version 1696536 (0.0009) [2023-12-27 03:43:39,555][105692] Updated weights for policy 0, policy_version 1692832 (0.0008) [2023-12-27 03:43:39,585][105620] Updated weights for policy 1, policy_version 1696546 (0.0010) [2023-12-27 03:43:39,637][105620] Updated weights for policy 1, policy_version 1696556 (0.0010) [2023-12-27 03:43:40,351][105692] Updated weights for policy 0, policy_version 1692842 (0.0008) [2023-12-27 03:43:40,410][105692] Updated weights for policy 0, policy_version 1692852 (0.0008) [2023-12-27 03:43:40,431][105620] Updated weights for policy 1, policy_version 1696566 (0.0010) [2023-12-27 03:43:40,466][105692] Updated weights for policy 0, policy_version 1692862 (0.0007) [2023-12-27 03:43:40,494][105620] Updated weights for policy 1, policy_version 1696576 (0.0011) [2023-12-27 03:43:40,556][105620] Updated weights for policy 1, policy_version 1696586 (0.0010) [2023-12-27 03:43:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 867827712. Throughput: 0: 9642.5, 1: 9737.6. Samples: 867838860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:43:41,063][104569] Avg episode reward: [(0, '8803.149'), (1, '8988.823')] [2023-12-27 03:43:41,277][105620] Updated weights for policy 1, policy_version 1696596 (0.0010) [2023-12-27 03:43:41,279][105692] Updated weights for policy 0, policy_version 1692872 (0.0008) [2023-12-27 03:43:41,325][105620] Updated weights for policy 1, policy_version 1696606 (0.0008) [2023-12-27 03:43:41,344][105692] Updated weights for policy 0, policy_version 1692882 (0.0007) [2023-12-27 03:43:41,387][105620] Updated weights for policy 1, policy_version 1696616 (0.0008) [2023-12-27 03:43:41,409][105692] Updated weights for policy 0, policy_version 1692892 (0.0007) [2023-12-27 03:43:42,154][105620] Updated weights for policy 1, policy_version 1696626 (0.0009) [2023-12-27 03:43:42,208][105620] Updated weights for policy 1, policy_version 1696636 (0.0007) [2023-12-27 03:43:42,223][105692] Updated weights for policy 0, policy_version 1692902 (0.0007) [2023-12-27 03:43:42,267][105620] Updated weights for policy 1, policy_version 1696646 (0.0010) [2023-12-27 03:43:42,274][105692] Updated weights for policy 0, policy_version 1692912 (0.0006) [2023-12-27 03:43:42,330][105620] Updated weights for policy 1, policy_version 1696656 (0.0010) [2023-12-27 03:43:42,337][105692] Updated weights for policy 0, policy_version 1692922 (0.0008) [2023-12-27 03:43:43,071][105620] Updated weights for policy 1, policy_version 1696666 (0.0010) [2023-12-27 03:43:43,093][105692] Updated weights for policy 0, policy_version 1692932 (0.0008) [2023-12-27 03:43:43,129][105620] Updated weights for policy 1, policy_version 1696676 (0.0010) [2023-12-27 03:43:43,151][105692] Updated weights for policy 0, policy_version 1692942 (0.0007) [2023-12-27 03:43:43,183][105620] Updated weights for policy 1, policy_version 1696686 (0.0010) [2023-12-27 03:43:43,206][105692] Updated weights for policy 0, policy_version 1692952 (0.0007) [2023-12-27 03:43:43,920][105620] Updated weights for policy 1, policy_version 1696696 (0.0010) [2023-12-27 03:43:43,967][105692] Updated weights for policy 0, policy_version 1692962 (0.0008) [2023-12-27 03:43:43,972][105620] Updated weights for policy 1, policy_version 1696706 (0.0010) [2023-12-27 03:43:44,021][105692] Updated weights for policy 0, policy_version 1692972 (0.0006) [2023-12-27 03:43:44,030][105620] Updated weights for policy 1, policy_version 1696716 (0.0010) [2023-12-27 03:43:44,079][105692] Updated weights for policy 0, policy_version 1692982 (0.0007) [2023-12-27 03:43:44,143][105692] Updated weights for policy 0, policy_version 1692992 (0.0008) [2023-12-27 03:43:44,785][105620] Updated weights for policy 1, policy_version 1696726 (0.0010) [2023-12-27 03:43:44,851][105620] Updated weights for policy 1, policy_version 1696736 (0.0009) [2023-12-27 03:43:44,898][105692] Updated weights for policy 0, policy_version 1693002 (0.0008) [2023-12-27 03:43:44,911][105620] Updated weights for policy 1, policy_version 1696746 (0.0011) [2023-12-27 03:43:44,958][105692] Updated weights for policy 0, policy_version 1693012 (0.0006) [2023-12-27 03:43:45,022][105692] Updated weights for policy 0, policy_version 1693022 (0.0006) [2023-12-27 03:43:45,663][105692] Updated weights for policy 0, policy_version 1693032 (0.0005) [2023-12-27 03:43:45,664][105620] Updated weights for policy 1, policy_version 1696756 (0.0011) [2023-12-27 03:43:45,712][105620] Updated weights for policy 1, policy_version 1696766 (0.0010) [2023-12-27 03:43:45,719][105692] Updated weights for policy 0, policy_version 1693042 (0.0006) [2023-12-27 03:43:45,764][105620] Updated weights for policy 1, policy_version 1696776 (0.0010) [2023-12-27 03:43:45,786][105692] Updated weights for policy 0, policy_version 1693052 (0.0006) [2023-12-27 03:43:46,062][104569] Fps is (10 sec: 19659.8, 60 sec: 19387.6, 300 sec: 19494.1). Total num frames: 867926016. Throughput: 0: 9638.8, 1: 9697.7. Samples: 867894168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:43:46,064][104569] Avg episode reward: [(0, '8896.702'), (1, '8986.606')] [2023-12-27 03:43:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001693056_433487872.pth... [2023-12-27 03:43:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001696784_434438144.pth... [2023-12-27 03:43:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001695664_434151424.pth [2023-12-27 03:43:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001691904_433192960.pth [2023-12-27 03:43:46,418][105692] Updated weights for policy 0, policy_version 1693062 (0.0007) [2023-12-27 03:43:46,438][105620] Updated weights for policy 1, policy_version 1696786 (0.0010) [2023-12-27 03:43:46,475][105692] Updated weights for policy 0, policy_version 1693072 (0.0006) [2023-12-27 03:43:46,503][105620] Updated weights for policy 1, policy_version 1696796 (0.0010) [2023-12-27 03:43:46,529][105692] Updated weights for policy 0, policy_version 1693082 (0.0005) [2023-12-27 03:43:46,570][105620] Updated weights for policy 1, policy_version 1696806 (0.0008) [2023-12-27 03:43:46,639][105620] Updated weights for policy 1, policy_version 1696816 (0.0011) [2023-12-27 03:43:47,121][105692] Updated weights for policy 0, policy_version 1693092 (0.0007) [2023-12-27 03:43:47,171][105692] Updated weights for policy 0, policy_version 1693102 (0.0006) [2023-12-27 03:43:47,223][105692] Updated weights for policy 0, policy_version 1693112 (0.0008) [2023-12-27 03:43:47,368][105620] Updated weights for policy 1, policy_version 1696826 (0.0010) [2023-12-27 03:43:47,433][105620] Updated weights for policy 1, policy_version 1696836 (0.0010) [2023-12-27 03:43:47,498][105620] Updated weights for policy 1, policy_version 1696846 (0.0010) [2023-12-27 03:43:47,926][105692] Updated weights for policy 0, policy_version 1693122 (0.0008) [2023-12-27 03:43:47,979][105692] Updated weights for policy 0, policy_version 1693132 (0.0009) [2023-12-27 03:43:48,040][105692] Updated weights for policy 0, policy_version 1693142 (0.0009) [2023-12-27 03:43:48,108][105692] Updated weights for policy 0, policy_version 1693152 (0.0007) [2023-12-27 03:43:48,132][105620] Updated weights for policy 1, policy_version 1696856 (0.0006) [2023-12-27 03:43:48,200][105620] Updated weights for policy 1, policy_version 1696866 (0.0008) [2023-12-27 03:43:48,262][105620] Updated weights for policy 1, policy_version 1696876 (0.0010) [2023-12-27 03:43:48,768][105692] Updated weights for policy 0, policy_version 1693162 (0.0010) [2023-12-27 03:43:48,829][105692] Updated weights for policy 0, policy_version 1693172 (0.0008) [2023-12-27 03:43:48,892][105692] Updated weights for policy 0, policy_version 1693182 (0.0008) [2023-12-27 03:43:49,017][105620] Updated weights for policy 1, policy_version 1696886 (0.0009) [2023-12-27 03:43:49,079][105620] Updated weights for policy 1, policy_version 1696896 (0.0008) [2023-12-27 03:43:49,138][105620] Updated weights for policy 1, policy_version 1696906 (0.0008) [2023-12-27 03:43:49,636][105692] Updated weights for policy 0, policy_version 1693192 (0.0009) [2023-12-27 03:43:49,707][105692] Updated weights for policy 0, policy_version 1693202 (0.0006) [2023-12-27 03:43:49,772][105692] Updated weights for policy 0, policy_version 1693212 (0.0006) [2023-12-27 03:43:49,836][105620] Updated weights for policy 1, policy_version 1696916 (0.0008) [2023-12-27 03:43:49,889][105620] Updated weights for policy 1, policy_version 1696926 (0.0008) [2023-12-27 03:43:49,946][105620] Updated weights for policy 1, policy_version 1696936 (0.0009) [2023-12-27 03:43:50,474][105692] Updated weights for policy 0, policy_version 1693222 (0.0008) [2023-12-27 03:43:50,523][105692] Updated weights for policy 0, policy_version 1693232 (0.0008) [2023-12-27 03:43:50,578][105692] Updated weights for policy 0, policy_version 1693242 (0.0008) [2023-12-27 03:43:50,724][105620] Updated weights for policy 1, policy_version 1696946 (0.0009) [2023-12-27 03:43:50,794][105620] Updated weights for policy 1, policy_version 1696956 (0.0011) [2023-12-27 03:43:50,844][105620] Updated weights for policy 1, policy_version 1696966 (0.0011) [2023-12-27 03:43:50,905][105620] Updated weights for policy 1, policy_version 1696976 (0.0010) [2023-12-27 03:43:51,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 868024320. Throughput: 0: 9798.2, 1: 9660.5. Samples: 868012760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:43:51,062][104569] Avg episode reward: [(0, '8804.495'), (1, '8625.802')] [2023-12-27 03:43:51,312][105692] Updated weights for policy 0, policy_version 1693252 (0.0008) [2023-12-27 03:43:51,379][105692] Updated weights for policy 0, policy_version 1693262 (0.0008) [2023-12-27 03:43:51,428][105692] Updated weights for policy 0, policy_version 1693272 (0.0008) [2023-12-27 03:43:51,625][105620] Updated weights for policy 1, policy_version 1696986 (0.0009) [2023-12-27 03:43:51,693][105620] Updated weights for policy 1, policy_version 1696996 (0.0009) [2023-12-27 03:43:51,762][105620] Updated weights for policy 1, policy_version 1697006 (0.0008) [2023-12-27 03:43:52,155][105692] Updated weights for policy 0, policy_version 1693282 (0.0008) [2023-12-27 03:43:52,213][105692] Updated weights for policy 0, policy_version 1693292 (0.0005) [2023-12-27 03:43:52,269][105692] Updated weights for policy 0, policy_version 1693302 (0.0006) [2023-12-27 03:43:52,338][105692] Updated weights for policy 0, policy_version 1693312 (0.0008) [2023-12-27 03:43:52,481][105620] Updated weights for policy 1, policy_version 1697016 (0.0006) [2023-12-27 03:43:52,545][105620] Updated weights for policy 1, policy_version 1697026 (0.0006) [2023-12-27 03:43:52,602][105620] Updated weights for policy 1, policy_version 1697036 (0.0007) [2023-12-27 03:43:53,094][105692] Updated weights for policy 0, policy_version 1693322 (0.0009) [2023-12-27 03:43:53,141][105692] Updated weights for policy 0, policy_version 1693332 (0.0009) [2023-12-27 03:43:53,191][105692] Updated weights for policy 0, policy_version 1693342 (0.0009) [2023-12-27 03:43:53,272][105620] Updated weights for policy 1, policy_version 1697046 (0.0010) [2023-12-27 03:43:53,333][105620] Updated weights for policy 1, policy_version 1697056 (0.0009) [2023-12-27 03:43:53,395][105620] Updated weights for policy 1, policy_version 1697066 (0.0009) [2023-12-27 03:43:53,921][105692] Updated weights for policy 0, policy_version 1693352 (0.0006) [2023-12-27 03:43:53,987][105692] Updated weights for policy 0, policy_version 1693362 (0.0008) [2023-12-27 03:43:54,035][105692] Updated weights for policy 0, policy_version 1693372 (0.0010) [2023-12-27 03:43:54,092][105620] Updated weights for policy 1, policy_version 1697076 (0.0008) [2023-12-27 03:43:54,148][105620] Updated weights for policy 1, policy_version 1697086 (0.0007) [2023-12-27 03:43:54,207][105620] Updated weights for policy 1, policy_version 1697096 (0.0008) [2023-12-27 03:43:54,689][105692] Updated weights for policy 0, policy_version 1693382 (0.0008) [2023-12-27 03:43:54,753][105692] Updated weights for policy 0, policy_version 1693392 (0.0005) [2023-12-27 03:43:54,812][105692] Updated weights for policy 0, policy_version 1693402 (0.0005) [2023-12-27 03:43:55,020][105620] Updated weights for policy 1, policy_version 1697106 (0.0007) [2023-12-27 03:43:55,076][105620] Updated weights for policy 1, policy_version 1697116 (0.0006) [2023-12-27 03:43:55,133][105620] Updated weights for policy 1, policy_version 1697126 (0.0005) [2023-12-27 03:43:55,199][105620] Updated weights for policy 1, policy_version 1697136 (0.0005) [2023-12-27 03:43:55,399][105692] Updated weights for policy 0, policy_version 1693412 (0.0006) [2023-12-27 03:43:55,444][105692] Updated weights for policy 0, policy_version 1693422 (0.0005) [2023-12-27 03:43:55,492][105692] Updated weights for policy 0, policy_version 1693432 (0.0006) [2023-12-27 03:43:55,816][105620] Updated weights for policy 1, policy_version 1697146 (0.0005) [2023-12-27 03:43:55,881][105620] Updated weights for policy 1, policy_version 1697156 (0.0005) [2023-12-27 03:43:55,941][105620] Updated weights for policy 1, policy_version 1697166 (0.0005) [2023-12-27 03:43:56,062][104569] Fps is (10 sec: 19661.8, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 868122624. Throughput: 0: 9839.5, 1: 9631.3. Samples: 868130608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:43:56,063][104569] Avg episode reward: [(0, '8986.958'), (1, '8625.428')] [2023-12-27 03:43:56,134][105692] Updated weights for policy 0, policy_version 1693442 (0.0007) [2023-12-27 03:43:56,192][105692] Updated weights for policy 0, policy_version 1693452 (0.0009) [2023-12-27 03:43:56,241][105692] Updated weights for policy 0, policy_version 1693462 (0.0009) [2023-12-27 03:43:56,290][105692] Updated weights for policy 0, policy_version 1693472 (0.0009) [2023-12-27 03:43:56,497][105620] Updated weights for policy 1, policy_version 1697176 (0.0005) [2023-12-27 03:43:56,560][105620] Updated weights for policy 1, policy_version 1697186 (0.0005) [2023-12-27 03:43:56,618][105620] Updated weights for policy 1, policy_version 1697196 (0.0007) [2023-12-27 03:43:57,043][105692] Updated weights for policy 0, policy_version 1693482 (0.0005) [2023-12-27 03:43:57,088][105692] Updated weights for policy 0, policy_version 1693492 (0.0007) [2023-12-27 03:43:57,142][105692] Updated weights for policy 0, policy_version 1693502 (0.0010) [2023-12-27 03:43:57,313][105620] Updated weights for policy 1, policy_version 1697206 (0.0008) [2023-12-27 03:43:57,363][105620] Updated weights for policy 1, policy_version 1697216 (0.0005) [2023-12-27 03:43:57,415][105620] Updated weights for policy 1, policy_version 1697226 (0.0005) [2023-12-27 03:43:57,861][105692] Updated weights for policy 0, policy_version 1693512 (0.0010) [2023-12-27 03:43:57,923][105692] Updated weights for policy 0, policy_version 1693522 (0.0010) [2023-12-27 03:43:57,975][105692] Updated weights for policy 0, policy_version 1693532 (0.0010) [2023-12-27 03:43:58,042][105620] Updated weights for policy 1, policy_version 1697236 (0.0005) [2023-12-27 03:43:58,102][105620] Updated weights for policy 1, policy_version 1697246 (0.0006) [2023-12-27 03:43:58,152][105620] Updated weights for policy 1, policy_version 1697256 (0.0008) [2023-12-27 03:43:58,760][105692] Updated weights for policy 0, policy_version 1693542 (0.0009) [2023-12-27 03:43:58,836][105692] Updated weights for policy 0, policy_version 1693553 (0.0009) [2023-12-27 03:43:58,881][105620] Updated weights for policy 1, policy_version 1697266 (0.0008) [2023-12-27 03:43:58,900][105692] Updated weights for policy 0, policy_version 1693563 (0.0007) [2023-12-27 03:43:58,950][105620] Updated weights for policy 1, policy_version 1697276 (0.0008) [2023-12-27 03:43:59,009][105620] Updated weights for policy 1, policy_version 1697286 (0.0006) [2023-12-27 03:43:59,075][105620] Updated weights for policy 1, policy_version 1697296 (0.0006) [2023-12-27 03:43:59,570][105692] Updated weights for policy 0, policy_version 1693573 (0.0008) [2023-12-27 03:43:59,625][105692] Updated weights for policy 0, policy_version 1693583 (0.0008) [2023-12-27 03:43:59,688][105692] Updated weights for policy 0, policy_version 1693593 (0.0009) [2023-12-27 03:43:59,769][105620] Updated weights for policy 1, policy_version 1697306 (0.0005) [2023-12-27 03:43:59,837][105620] Updated weights for policy 1, policy_version 1697316 (0.0007) [2023-12-27 03:43:59,904][105620] Updated weights for policy 1, policy_version 1697326 (0.0008) [2023-12-27 03:44:00,399][105692] Updated weights for policy 0, policy_version 1693603 (0.0009) [2023-12-27 03:44:00,459][105692] Updated weights for policy 0, policy_version 1693613 (0.0005) [2023-12-27 03:44:00,519][105692] Updated weights for policy 0, policy_version 1693623 (0.0006) [2023-12-27 03:44:00,630][105620] Updated weights for policy 1, policy_version 1697336 (0.0009) [2023-12-27 03:44:00,682][105620] Updated weights for policy 1, policy_version 1697346 (0.0009) [2023-12-27 03:44:00,750][105620] Updated weights for policy 1, policy_version 1697356 (0.0009) [2023-12-27 03:44:01,027][105692] Updated weights for policy 0, policy_version 1693633 (0.0006) [2023-12-27 03:44:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 868220928. Throughput: 0: 9863.3, 1: 9725.5. Samples: 868191416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:44:01,062][104569] Avg episode reward: [(0, '8988.528'), (1, '8803.575')] [2023-12-27 03:44:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001697360_434585600.pth... [2023-12-27 03:44:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001696208_434290688.pth [2023-12-27 03:44:01,089][105692] Updated weights for policy 0, policy_version 1693643 (0.0008) [2023-12-27 03:44:01,152][105692] Updated weights for policy 0, policy_version 1693653 (0.0009) [2023-12-27 03:44:01,213][105692] Updated weights for policy 0, policy_version 1693663 (0.0009) [2023-12-27 03:44:01,218][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001693664_433643520.pth... [2023-12-27 03:44:01,221][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001692480_433340416.pth [2023-12-27 03:44:01,570][105620] Updated weights for policy 1, policy_version 1697366 (0.0010) [2023-12-27 03:44:01,636][105620] Updated weights for policy 1, policy_version 1697376 (0.0008) [2023-12-27 03:44:01,694][105620] Updated weights for policy 1, policy_version 1697386 (0.0009) [2023-12-27 03:44:02,016][105692] Updated weights for policy 0, policy_version 1693673 (0.0007) [2023-12-27 03:44:02,065][105692] Updated weights for policy 0, policy_version 1693683 (0.0008) [2023-12-27 03:44:02,123][105692] Updated weights for policy 0, policy_version 1693693 (0.0005) [2023-12-27 03:44:02,442][105620] Updated weights for policy 1, policy_version 1697396 (0.0009) [2023-12-27 03:44:02,493][105620] Updated weights for policy 1, policy_version 1697406 (0.0009) [2023-12-27 03:44:02,540][105620] Updated weights for policy 1, policy_version 1697416 (0.0008) [2023-12-27 03:44:02,745][105692] Updated weights for policy 0, policy_version 1693703 (0.0007) [2023-12-27 03:44:02,796][105692] Updated weights for policy 0, policy_version 1693713 (0.0009) [2023-12-27 03:44:02,844][105692] Updated weights for policy 0, policy_version 1693723 (0.0009) [2023-12-27 03:44:03,343][105620] Updated weights for policy 1, policy_version 1697426 (0.0008) [2023-12-27 03:44:03,396][105620] Updated weights for policy 1, policy_version 1697436 (0.0005) [2023-12-27 03:44:03,448][105620] Updated weights for policy 1, policy_version 1697446 (0.0005) [2023-12-27 03:44:03,501][105620] Updated weights for policy 1, policy_version 1697456 (0.0005) [2023-12-27 03:44:03,588][105692] Updated weights for policy 0, policy_version 1693733 (0.0009) [2023-12-27 03:44:03,643][105692] Updated weights for policy 0, policy_version 1693743 (0.0009) [2023-12-27 03:44:03,696][105692] Updated weights for policy 0, policy_version 1693753 (0.0005) [2023-12-27 03:44:04,051][105620] Updated weights for policy 1, policy_version 1697466 (0.0008) [2023-12-27 03:44:04,105][105620] Updated weights for policy 1, policy_version 1697476 (0.0009) [2023-12-27 03:44:04,168][105620] Updated weights for policy 1, policy_version 1697486 (0.0007) [2023-12-27 03:44:04,447][105692] Updated weights for policy 0, policy_version 1693763 (0.0006) [2023-12-27 03:44:04,505][105692] Updated weights for policy 0, policy_version 1693773 (0.0009) [2023-12-27 03:44:04,561][105692] Updated weights for policy 0, policy_version 1693783 (0.0009) [2023-12-27 03:44:04,830][105620] Updated weights for policy 1, policy_version 1697496 (0.0006) [2023-12-27 03:44:04,890][105620] Updated weights for policy 1, policy_version 1697506 (0.0009) [2023-12-27 03:44:04,949][105620] Updated weights for policy 1, policy_version 1697516 (0.0009) [2023-12-27 03:44:05,247][105692] Updated weights for policy 0, policy_version 1693793 (0.0009) [2023-12-27 03:44:05,295][105692] Updated weights for policy 0, policy_version 1693803 (0.0009) [2023-12-27 03:44:05,345][105692] Updated weights for policy 0, policy_version 1693813 (0.0009) [2023-12-27 03:44:05,403][105692] Updated weights for policy 0, policy_version 1693823 (0.0009) [2023-12-27 03:44:05,688][105620] Updated weights for policy 1, policy_version 1697526 (0.0008) [2023-12-27 03:44:05,742][105620] Updated weights for policy 1, policy_version 1697536 (0.0009) [2023-12-27 03:44:05,804][105620] Updated weights for policy 1, policy_version 1697546 (0.0006) [2023-12-27 03:44:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 868319232. Throughput: 0: 9911.7, 1: 9754.5. Samples: 868308856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:44:06,063][104569] Avg episode reward: [(0, '8804.809'), (1, '9078.358')] [2023-12-27 03:44:06,243][105692] Updated weights for policy 0, policy_version 1693833 (0.0008) [2023-12-27 03:44:06,311][105692] Updated weights for policy 0, policy_version 1693843 (0.0009) [2023-12-27 03:44:06,372][105692] Updated weights for policy 0, policy_version 1693853 (0.0008) [2023-12-27 03:44:06,416][105620] Updated weights for policy 1, policy_version 1697556 (0.0006) [2023-12-27 03:44:06,486][105620] Updated weights for policy 1, policy_version 1697566 (0.0008) [2023-12-27 03:44:06,551][105620] Updated weights for policy 1, policy_version 1697576 (0.0008) [2023-12-27 03:44:07,083][105692] Updated weights for policy 0, policy_version 1693863 (0.0006) [2023-12-27 03:44:07,145][105692] Updated weights for policy 0, policy_version 1693873 (0.0006) [2023-12-27 03:44:07,203][105692] Updated weights for policy 0, policy_version 1693883 (0.0006) [2023-12-27 03:44:07,323][105620] Updated weights for policy 1, policy_version 1697586 (0.0009) [2023-12-27 03:44:07,387][105620] Updated weights for policy 1, policy_version 1697596 (0.0008) [2023-12-27 03:44:07,442][105620] Updated weights for policy 1, policy_version 1697606 (0.0009) [2023-12-27 03:44:07,494][105620] Updated weights for policy 1, policy_version 1697616 (0.0009) [2023-12-27 03:44:07,830][105692] Updated weights for policy 0, policy_version 1693893 (0.0005) [2023-12-27 03:44:07,884][105692] Updated weights for policy 0, policy_version 1693903 (0.0005) [2023-12-27 03:44:07,948][105692] Updated weights for policy 0, policy_version 1693913 (0.0005) [2023-12-27 03:44:08,385][105620] Updated weights for policy 1, policy_version 1697626 (0.0007) [2023-12-27 03:44:08,440][105620] Updated weights for policy 1, policy_version 1697636 (0.0009) [2023-12-27 03:44:08,491][105620] Updated weights for policy 1, policy_version 1697646 (0.0006) [2023-12-27 03:44:08,528][105692] Updated weights for policy 0, policy_version 1693923 (0.0007) [2023-12-27 03:44:08,575][105692] Updated weights for policy 0, policy_version 1693933 (0.0009) [2023-12-27 03:44:08,634][105692] Updated weights for policy 0, policy_version 1693943 (0.0009) [2023-12-27 03:44:09,204][105620] Updated weights for policy 1, policy_version 1697656 (0.0008) [2023-12-27 03:44:09,266][105620] Updated weights for policy 1, policy_version 1697666 (0.0009) [2023-12-27 03:44:09,325][105620] Updated weights for policy 1, policy_version 1697676 (0.0008) [2023-12-27 03:44:09,421][105692] Updated weights for policy 0, policy_version 1693953 (0.0009) [2023-12-27 03:44:09,479][105692] Updated weights for policy 0, policy_version 1693963 (0.0007) [2023-12-27 03:44:09,535][105692] Updated weights for policy 0, policy_version 1693973 (0.0009) [2023-12-27 03:44:09,597][105692] Updated weights for policy 0, policy_version 1693983 (0.0009) [2023-12-27 03:44:10,081][105620] Updated weights for policy 1, policy_version 1697686 (0.0008) [2023-12-27 03:44:10,151][105620] Updated weights for policy 1, policy_version 1697696 (0.0010) [2023-12-27 03:44:10,218][105620] Updated weights for policy 1, policy_version 1697706 (0.0010) [2023-12-27 03:44:10,289][105692] Updated weights for policy 0, policy_version 1693993 (0.0007) [2023-12-27 03:44:10,342][105692] Updated weights for policy 0, policy_version 1694003 (0.0008) [2023-12-27 03:44:10,401][105692] Updated weights for policy 0, policy_version 1694013 (0.0006) [2023-12-27 03:44:10,982][105620] Updated weights for policy 1, policy_version 1697716 (0.0011) [2023-12-27 03:44:11,046][105620] Updated weights for policy 1, policy_version 1697726 (0.0011) [2023-12-27 03:44:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 868409344. Throughput: 0: 9918.9, 1: 9698.1. Samples: 868424308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:44:11,062][104569] Avg episode reward: [(0, '8712.044'), (1, '9170.709')] [2023-12-27 03:44:11,064][105692] Updated weights for policy 0, policy_version 1694023 (0.0007) [2023-12-27 03:44:11,108][105620] Updated weights for policy 1, policy_version 1697736 (0.0009) [2023-12-27 03:44:11,126][105692] Updated weights for policy 0, policy_version 1694033 (0.0007) [2023-12-27 03:44:11,191][105692] Updated weights for policy 0, policy_version 1694043 (0.0007) [2023-12-27 03:44:11,821][105620] Updated weights for policy 1, policy_version 1697746 (0.0008) [2023-12-27 03:44:11,885][105620] Updated weights for policy 1, policy_version 1697756 (0.0006) [2023-12-27 03:44:11,944][105620] Updated weights for policy 1, policy_version 1697766 (0.0006) [2023-12-27 03:44:12,006][105620] Updated weights for policy 1, policy_version 1697776 (0.0006) [2023-12-27 03:44:12,012][105692] Updated weights for policy 0, policy_version 1694053 (0.0009) [2023-12-27 03:44:12,070][105692] Updated weights for policy 0, policy_version 1694063 (0.0010) [2023-12-27 03:44:12,128][105692] Updated weights for policy 0, policy_version 1694073 (0.0006) [2023-12-27 03:44:12,656][105620] Updated weights for policy 1, policy_version 1697786 (0.0008) [2023-12-27 03:44:12,706][105620] Updated weights for policy 1, policy_version 1697796 (0.0008) [2023-12-27 03:44:12,762][105620] Updated weights for policy 1, policy_version 1697806 (0.0007) [2023-12-27 03:44:12,801][105692] Updated weights for policy 0, policy_version 1694083 (0.0009) [2023-12-27 03:44:12,867][105692] Updated weights for policy 0, policy_version 1694093 (0.0010) [2023-12-27 03:44:12,930][105692] Updated weights for policy 0, policy_version 1694103 (0.0009) [2023-12-27 03:44:13,492][105620] Updated weights for policy 1, policy_version 1697816 (0.0008) [2023-12-27 03:44:13,552][105620] Updated weights for policy 1, policy_version 1697826 (0.0008) [2023-12-27 03:44:13,614][105620] Updated weights for policy 1, policy_version 1697836 (0.0005) [2023-12-27 03:44:13,617][105692] Updated weights for policy 0, policy_version 1694113 (0.0008) [2023-12-27 03:44:13,680][105692] Updated weights for policy 0, policy_version 1694123 (0.0009) [2023-12-27 03:44:13,737][105692] Updated weights for policy 0, policy_version 1694133 (0.0008) [2023-12-27 03:44:13,798][105692] Updated weights for policy 0, policy_version 1694143 (0.0009) [2023-12-27 03:44:14,273][105620] Updated weights for policy 1, policy_version 1697846 (0.0008) [2023-12-27 03:44:14,324][105620] Updated weights for policy 1, policy_version 1697856 (0.0010) [2023-12-27 03:44:14,369][105620] Updated weights for policy 1, policy_version 1697866 (0.0010) [2023-12-27 03:44:14,464][105692] Updated weights for policy 0, policy_version 1694153 (0.0006) [2023-12-27 03:44:14,519][105692] Updated weights for policy 0, policy_version 1694163 (0.0006) [2023-12-27 03:44:14,576][105692] Updated weights for policy 0, policy_version 1694173 (0.0005) [2023-12-27 03:44:15,119][105692] Updated weights for policy 0, policy_version 1694183 (0.0006) [2023-12-27 03:44:15,128][105620] Updated weights for policy 1, policy_version 1697876 (0.0010) [2023-12-27 03:44:15,178][105692] Updated weights for policy 0, policy_version 1694193 (0.0006) [2023-12-27 03:44:15,192][105620] Updated weights for policy 1, policy_version 1697886 (0.0008) [2023-12-27 03:44:15,232][105692] Updated weights for policy 0, policy_version 1694203 (0.0006) [2023-12-27 03:44:15,250][105620] Updated weights for policy 1, policy_version 1697896 (0.0009) [2023-12-27 03:44:15,876][105620] Updated weights for policy 1, policy_version 1697906 (0.0008) [2023-12-27 03:44:15,930][105620] Updated weights for policy 1, policy_version 1697916 (0.0005) [2023-12-27 03:44:15,987][105620] Updated weights for policy 1, policy_version 1697926 (0.0005) [2023-12-27 03:44:15,994][105692] Updated weights for policy 0, policy_version 1694213 (0.0008) [2023-12-27 03:44:16,040][105620] Updated weights for policy 1, policy_version 1697936 (0.0006) [2023-12-27 03:44:16,059][105692] Updated weights for policy 0, policy_version 1694223 (0.0008) [2023-12-27 03:44:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 868515840. Throughput: 0: 9907.6, 1: 9692.9. Samples: 868483628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:44:16,062][104569] Avg episode reward: [(0, '8803.900'), (1, '8993.335')] [2023-12-27 03:44:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001697936_434733056.pth... [2023-12-27 03:44:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001696784_434438144.pth [2023-12-27 03:44:16,126][105692] Updated weights for policy 0, policy_version 1694233 (0.0008) [2023-12-27 03:44:16,169][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001694240_433790976.pth... [2023-12-27 03:44:16,175][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001693056_433487872.pth [2023-12-27 03:44:16,631][105620] Updated weights for policy 1, policy_version 1697946 (0.0007) [2023-12-27 03:44:16,687][105692] Updated weights for policy 0, policy_version 1694243 (0.0008) [2023-12-27 03:44:16,707][105620] Updated weights for policy 1, policy_version 1697956 (0.0007) [2023-12-27 03:44:16,743][105692] Updated weights for policy 0, policy_version 1694253 (0.0006) [2023-12-27 03:44:16,766][105620] Updated weights for policy 1, policy_version 1697966 (0.0011) [2023-12-27 03:44:16,806][105692] Updated weights for policy 0, policy_version 1694263 (0.0008) [2023-12-27 03:44:17,464][105692] Updated weights for policy 0, policy_version 1694273 (0.0009) [2023-12-27 03:44:17,500][105620] Updated weights for policy 1, policy_version 1697976 (0.0006) [2023-12-27 03:44:17,523][105692] Updated weights for policy 0, policy_version 1694283 (0.0007) [2023-12-27 03:44:17,549][105620] Updated weights for policy 1, policy_version 1697986 (0.0005) [2023-12-27 03:44:17,585][105692] Updated weights for policy 0, policy_version 1694293 (0.0005) [2023-12-27 03:44:17,603][105620] Updated weights for policy 1, policy_version 1697996 (0.0005) [2023-12-27 03:44:17,646][105692] Updated weights for policy 0, policy_version 1694303 (0.0005) [2023-12-27 03:44:18,133][105620] Updated weights for policy 1, policy_version 1698006 (0.0007) [2023-12-27 03:44:18,190][105620] Updated weights for policy 1, policy_version 1698016 (0.0005) [2023-12-27 03:44:18,240][105620] Updated weights for policy 1, policy_version 1698026 (0.0005) [2023-12-27 03:44:18,429][105692] Updated weights for policy 0, policy_version 1694313 (0.0009) [2023-12-27 03:44:18,496][105692] Updated weights for policy 0, policy_version 1694323 (0.0009) [2023-12-27 03:44:18,559][105692] Updated weights for policy 0, policy_version 1694333 (0.0009) [2023-12-27 03:44:18,940][105620] Updated weights for policy 1, policy_version 1698036 (0.0007) [2023-12-27 03:44:18,992][105620] Updated weights for policy 1, policy_version 1698046 (0.0008) [2023-12-27 03:44:19,059][105620] Updated weights for policy 1, policy_version 1698056 (0.0008) [2023-12-27 03:44:19,316][105692] Updated weights for policy 0, policy_version 1694343 (0.0009) [2023-12-27 03:44:19,381][105692] Updated weights for policy 0, policy_version 1694353 (0.0009) [2023-12-27 03:44:19,436][105692] Updated weights for policy 0, policy_version 1694363 (0.0009) [2023-12-27 03:44:19,774][105620] Updated weights for policy 1, policy_version 1698066 (0.0008) [2023-12-27 03:44:19,834][105620] Updated weights for policy 1, policy_version 1698076 (0.0006) [2023-12-27 03:44:19,901][105620] Updated weights for policy 1, policy_version 1698086 (0.0008) [2023-12-27 03:44:19,965][105620] Updated weights for policy 1, policy_version 1698096 (0.0009) [2023-12-27 03:44:20,216][105692] Updated weights for policy 0, policy_version 1694373 (0.0009) [2023-12-27 03:44:20,276][105692] Updated weights for policy 0, policy_version 1694383 (0.0010) [2023-12-27 03:44:20,335][105692] Updated weights for policy 0, policy_version 1694393 (0.0009) [2023-12-27 03:44:20,670][105620] Updated weights for policy 1, policy_version 1698106 (0.0008) [2023-12-27 03:44:20,729][105620] Updated weights for policy 1, policy_version 1698116 (0.0009) [2023-12-27 03:44:20,795][105620] Updated weights for policy 1, policy_version 1698126 (0.0009) [2023-12-27 03:44:21,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 868614144. Throughput: 0: 9872.4, 1: 9749.7. Samples: 868605976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:44:21,063][104569] Avg episode reward: [(0, '8987.007'), (1, '8993.440')] [2023-12-27 03:44:21,104][105692] Updated weights for policy 0, policy_version 1694403 (0.0009) [2023-12-27 03:44:21,168][105692] Updated weights for policy 0, policy_version 1694413 (0.0008) [2023-12-27 03:44:21,221][105692] Updated weights for policy 0, policy_version 1694423 (0.0008) [2023-12-27 03:44:21,597][105620] Updated weights for policy 1, policy_version 1698136 (0.0009) [2023-12-27 03:44:21,653][105620] Updated weights for policy 1, policy_version 1698146 (0.0010) [2023-12-27 03:44:21,713][105620] Updated weights for policy 1, policy_version 1698156 (0.0009) [2023-12-27 03:44:21,963][105692] Updated weights for policy 0, policy_version 1694433 (0.0009) [2023-12-27 03:44:22,035][105692] Updated weights for policy 0, policy_version 1694443 (0.0008) [2023-12-27 03:44:22,091][105692] Updated weights for policy 0, policy_version 1694453 (0.0010) [2023-12-27 03:44:22,148][105692] Updated weights for policy 0, policy_version 1694463 (0.0007) [2023-12-27 03:44:22,488][105620] Updated weights for policy 1, policy_version 1698166 (0.0008) [2023-12-27 03:44:22,553][105620] Updated weights for policy 1, policy_version 1698176 (0.0009) [2023-12-27 03:44:22,620][105620] Updated weights for policy 1, policy_version 1698186 (0.0008) [2023-12-27 03:44:22,875][105692] Updated weights for policy 0, policy_version 1694473 (0.0010) [2023-12-27 03:44:22,936][105692] Updated weights for policy 0, policy_version 1694483 (0.0011) [2023-12-27 03:44:22,989][105692] Updated weights for policy 0, policy_version 1694493 (0.0011) [2023-12-27 03:44:23,326][105620] Updated weights for policy 1, policy_version 1698196 (0.0008) [2023-12-27 03:44:23,383][105620] Updated weights for policy 1, policy_version 1698206 (0.0008) [2023-12-27 03:44:23,447][105620] Updated weights for policy 1, policy_version 1698216 (0.0008) [2023-12-27 03:44:23,718][105692] Updated weights for policy 0, policy_version 1694503 (0.0009) [2023-12-27 03:44:23,769][105692] Updated weights for policy 0, policy_version 1694513 (0.0009) [2023-12-27 03:44:23,828][105692] Updated weights for policy 0, policy_version 1694523 (0.0009) [2023-12-27 03:44:24,216][105620] Updated weights for policy 1, policy_version 1698226 (0.0009) [2023-12-27 03:44:24,270][105620] Updated weights for policy 1, policy_version 1698236 (0.0009) [2023-12-27 03:44:24,323][105620] Updated weights for policy 1, policy_version 1698246 (0.0008) [2023-12-27 03:44:24,377][105620] Updated weights for policy 1, policy_version 1698256 (0.0009) [2023-12-27 03:44:24,545][105692] Updated weights for policy 0, policy_version 1694533 (0.0009) [2023-12-27 03:44:24,598][105692] Updated weights for policy 0, policy_version 1694543 (0.0009) [2023-12-27 03:44:24,649][105692] Updated weights for policy 0, policy_version 1694553 (0.0009) [2023-12-27 03:44:25,179][105620] Updated weights for policy 1, policy_version 1698266 (0.0007) [2023-12-27 03:44:25,234][105620] Updated weights for policy 1, policy_version 1698276 (0.0005) [2023-12-27 03:44:25,291][105620] Updated weights for policy 1, policy_version 1698286 (0.0005) [2023-12-27 03:44:25,325][105692] Updated weights for policy 0, policy_version 1694563 (0.0009) [2023-12-27 03:44:25,395][105692] Updated weights for policy 0, policy_version 1694573 (0.0010) [2023-12-27 03:44:25,461][105692] Updated weights for policy 0, policy_version 1694583 (0.0010) [2023-12-27 03:44:25,846][105620] Updated weights for policy 1, policy_version 1698296 (0.0009) [2023-12-27 03:44:25,912][105620] Updated weights for policy 1, policy_version 1698306 (0.0010) [2023-12-27 03:44:25,970][105620] Updated weights for policy 1, policy_version 1698316 (0.0011) [2023-12-27 03:44:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.7, 300 sec: 19466.4). Total num frames: 868712448. Throughput: 0: 9820.0, 1: 9746.2. Samples: 868719340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:44:26,063][104569] Avg episode reward: [(0, '8896.377'), (1, '9263.101')] [2023-12-27 03:44:26,201][105692] Updated weights for policy 0, policy_version 1694593 (0.0009) [2023-12-27 03:44:26,253][105692] Updated weights for policy 0, policy_version 1694603 (0.0008) [2023-12-27 03:44:26,310][105692] Updated weights for policy 0, policy_version 1694613 (0.0008) [2023-12-27 03:44:26,370][105692] Updated weights for policy 0, policy_version 1694623 (0.0008) [2023-12-27 03:44:26,628][105620] Updated weights for policy 1, policy_version 1698326 (0.0006) [2023-12-27 03:44:26,698][105620] Updated weights for policy 1, policy_version 1698336 (0.0010) [2023-12-27 03:44:26,768][105620] Updated weights for policy 1, policy_version 1698346 (0.0010) [2023-12-27 03:44:26,971][105692] Updated weights for policy 0, policy_version 1694633 (0.0007) [2023-12-27 03:44:27,025][105692] Updated weights for policy 0, policy_version 1694643 (0.0009) [2023-12-27 03:44:27,075][105692] Updated weights for policy 0, policy_version 1694653 (0.0009) [2023-12-27 03:44:27,478][105620] Updated weights for policy 1, policy_version 1698356 (0.0009) [2023-12-27 03:44:27,535][105620] Updated weights for policy 1, policy_version 1698367 (0.0010) [2023-12-27 03:44:27,588][105620] Updated weights for policy 1, policy_version 1698377 (0.0010) [2023-12-27 03:44:27,718][105692] Updated weights for policy 0, policy_version 1694663 (0.0007) [2023-12-27 03:44:27,764][105692] Updated weights for policy 0, policy_version 1694673 (0.0005) [2023-12-27 03:44:27,819][105692] Updated weights for policy 0, policy_version 1694683 (0.0005) [2023-12-27 03:44:28,263][105620] Updated weights for policy 1, policy_version 1698387 (0.0008) [2023-12-27 03:44:28,319][105620] Updated weights for policy 1, policy_version 1698397 (0.0007) [2023-12-27 03:44:28,351][105692] Updated weights for policy 0, policy_version 1694693 (0.0006) [2023-12-27 03:44:28,381][105620] Updated weights for policy 1, policy_version 1698407 (0.0008) [2023-12-27 03:44:28,407][105692] Updated weights for policy 0, policy_version 1694703 (0.0009) [2023-12-27 03:44:28,466][105692] Updated weights for policy 0, policy_version 1694713 (0.0007) [2023-12-27 03:44:29,012][105620] Updated weights for policy 1, policy_version 1698417 (0.0006) [2023-12-27 03:44:29,083][105620] Updated weights for policy 1, policy_version 1698427 (0.0005) [2023-12-27 03:44:29,153][105620] Updated weights for policy 1, policy_version 1698437 (0.0005) [2023-12-27 03:44:29,226][105620] Updated weights for policy 1, policy_version 1698447 (0.0007) [2023-12-27 03:44:29,315][105692] Updated weights for policy 0, policy_version 1694723 (0.0009) [2023-12-27 03:44:29,382][105692] Updated weights for policy 0, policy_version 1694733 (0.0008) [2023-12-27 03:44:29,433][105692] Updated weights for policy 0, policy_version 1694743 (0.0005) [2023-12-27 03:44:29,904][105620] Updated weights for policy 1, policy_version 1698457 (0.0008) [2023-12-27 03:44:29,969][105620] Updated weights for policy 1, policy_version 1698467 (0.0008) [2023-12-27 03:44:30,021][105620] Updated weights for policy 1, policy_version 1698477 (0.0008) [2023-12-27 03:44:30,119][105692] Updated weights for policy 0, policy_version 1694753 (0.0006) [2023-12-27 03:44:30,167][105692] Updated weights for policy 0, policy_version 1694763 (0.0008) [2023-12-27 03:44:30,219][105692] Updated weights for policy 0, policy_version 1694773 (0.0010) [2023-12-27 03:44:30,270][105692] Updated weights for policy 0, policy_version 1694783 (0.0010) [2023-12-27 03:44:30,710][105620] Updated weights for policy 1, policy_version 1698487 (0.0009) [2023-12-27 03:44:30,764][105620] Updated weights for policy 1, policy_version 1698498 (0.0010) [2023-12-27 03:44:30,817][105620] Updated weights for policy 1, policy_version 1698510 (0.0010) [2023-12-27 03:44:30,880][105692] Updated weights for policy 0, policy_version 1694793 (0.0007) [2023-12-27 03:44:30,942][105692] Updated weights for policy 0, policy_version 1694803 (0.0010) [2023-12-27 03:44:31,004][105692] Updated weights for policy 0, policy_version 1694813 (0.0010) [2023-12-27 03:44:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 868818944. Throughput: 0: 9958.3, 1: 9800.3. Samples: 868783296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:44:31,062][104569] Avg episode reward: [(0, '8346.519'), (1, '9355.497')] [2023-12-27 03:44:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001694816_433938432.pth... [2023-12-27 03:44:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001698512_434880512.pth... [2023-12-27 03:44:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001693664_433643520.pth [2023-12-27 03:44:31,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001697360_434585600.pth [2023-12-27 03:44:31,562][105620] Updated weights for policy 1, policy_version 1698520 (0.0009) [2023-12-27 03:44:31,625][105620] Updated weights for policy 1, policy_version 1698530 (0.0009) [2023-12-27 03:44:31,686][105620] Updated weights for policy 1, policy_version 1698540 (0.0009) [2023-12-27 03:44:31,783][105692] Updated weights for policy 0, policy_version 1694823 (0.0008) [2023-12-27 03:44:31,840][105692] Updated weights for policy 0, policy_version 1694833 (0.0005) [2023-12-27 03:44:31,887][105692] Updated weights for policy 0, policy_version 1694843 (0.0005) [2023-12-27 03:44:32,349][105620] Updated weights for policy 1, policy_version 1698550 (0.0009) [2023-12-27 03:44:32,410][105620] Updated weights for policy 1, policy_version 1698560 (0.0008) [2023-12-27 03:44:32,467][105620] Updated weights for policy 1, policy_version 1698570 (0.0008) [2023-12-27 03:44:32,637][105692] Updated weights for policy 0, policy_version 1694853 (0.0007) [2023-12-27 03:44:32,690][105692] Updated weights for policy 0, policy_version 1694864 (0.0010) [2023-12-27 03:44:32,745][105692] Updated weights for policy 0, policy_version 1694874 (0.0009) [2023-12-27 03:44:33,139][105620] Updated weights for policy 1, policy_version 1698580 (0.0009) [2023-12-27 03:44:33,192][105620] Updated weights for policy 1, policy_version 1698592 (0.0010) [2023-12-27 03:44:33,240][105620] Updated weights for policy 1, policy_version 1698603 (0.0007) [2023-12-27 03:44:33,484][105692] Updated weights for policy 0, policy_version 1694884 (0.0009) [2023-12-27 03:44:33,545][105692] Updated weights for policy 0, policy_version 1694894 (0.0009) [2023-12-27 03:44:33,593][105692] Updated weights for policy 0, policy_version 1694904 (0.0009) [2023-12-27 03:44:33,882][105620] Updated weights for policy 1, policy_version 1698613 (0.0007) [2023-12-27 03:44:33,946][105620] Updated weights for policy 1, policy_version 1698623 (0.0008) [2023-12-27 03:44:34,007][105620] Updated weights for policy 1, policy_version 1698633 (0.0005) [2023-12-27 03:44:34,427][105692] Updated weights for policy 0, policy_version 1694914 (0.0010) [2023-12-27 03:44:34,496][105692] Updated weights for policy 0, policy_version 1694924 (0.0010) [2023-12-27 03:44:34,555][105692] Updated weights for policy 0, policy_version 1694934 (0.0010) [2023-12-27 03:44:34,614][105692] Updated weights for policy 0, policy_version 1694944 (0.0008) [2023-12-27 03:44:34,655][105620] Updated weights for policy 1, policy_version 1698643 (0.0007) [2023-12-27 03:44:34,720][105620] Updated weights for policy 1, policy_version 1698653 (0.0008) [2023-12-27 03:44:34,780][105620] Updated weights for policy 1, policy_version 1698663 (0.0010) [2023-12-27 03:44:35,380][105692] Updated weights for policy 0, policy_version 1694954 (0.0007) [2023-12-27 03:44:35,424][105692] Updated weights for policy 0, policy_version 1694964 (0.0008) [2023-12-27 03:44:35,469][105692] Updated weights for policy 0, policy_version 1694974 (0.0008) [2023-12-27 03:44:35,471][105620] Updated weights for policy 1, policy_version 1698673 (0.0010) [2023-12-27 03:44:35,522][105620] Updated weights for policy 1, policy_version 1698683 (0.0010) [2023-12-27 03:44:35,578][105620] Updated weights for policy 1, policy_version 1698693 (0.0011) [2023-12-27 03:44:35,632][105620] Updated weights for policy 1, policy_version 1698703 (0.0007) [2023-12-27 03:44:36,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 868909056. Throughput: 0: 9859.1, 1: 9870.4. Samples: 868900588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:44:36,062][104569] Avg episode reward: [(0, '8436.791'), (1, '9171.781')] [2023-12-27 03:44:36,259][105692] Updated weights for policy 0, policy_version 1694984 (0.0006) [2023-12-27 03:44:36,323][105692] Updated weights for policy 0, policy_version 1694994 (0.0008) [2023-12-27 03:44:36,373][105620] Updated weights for policy 1, policy_version 1698713 (0.0011) [2023-12-27 03:44:36,387][105692] Updated weights for policy 0, policy_version 1695004 (0.0006) [2023-12-27 03:44:36,439][105620] Updated weights for policy 1, policy_version 1698723 (0.0009) [2023-12-27 03:44:36,500][105620] Updated weights for policy 1, policy_version 1698733 (0.0008) [2023-12-27 03:44:37,138][105692] Updated weights for policy 0, policy_version 1695014 (0.0008) [2023-12-27 03:44:37,178][105620] Updated weights for policy 1, policy_version 1698743 (0.0007) [2023-12-27 03:44:37,188][105692] Updated weights for policy 0, policy_version 1695024 (0.0010) [2023-12-27 03:44:37,231][105620] Updated weights for policy 1, policy_version 1698753 (0.0007) [2023-12-27 03:44:37,242][105692] Updated weights for policy 0, policy_version 1695034 (0.0008) [2023-12-27 03:44:37,290][105620] Updated weights for policy 1, policy_version 1698763 (0.0010) [2023-12-27 03:44:37,972][105692] Updated weights for policy 0, policy_version 1695044 (0.0007) [2023-12-27 03:44:38,011][105620] Updated weights for policy 1, policy_version 1698773 (0.0009) [2023-12-27 03:44:38,028][105692] Updated weights for policy 0, policy_version 1695054 (0.0006) [2023-12-27 03:44:38,079][105620] Updated weights for policy 1, policy_version 1698783 (0.0007) [2023-12-27 03:44:38,082][105692] Updated weights for policy 0, policy_version 1695064 (0.0008) [2023-12-27 03:44:38,144][105620] Updated weights for policy 1, policy_version 1698793 (0.0010) [2023-12-27 03:44:38,855][105620] Updated weights for policy 1, policy_version 1698803 (0.0009) [2023-12-27 03:44:38,856][105692] Updated weights for policy 0, policy_version 1695074 (0.0008) [2023-12-27 03:44:38,902][105692] Updated weights for policy 0, policy_version 1695084 (0.0007) [2023-12-27 03:44:38,915][105620] Updated weights for policy 1, policy_version 1698813 (0.0010) [2023-12-27 03:44:38,955][105692] Updated weights for policy 0, policy_version 1695094 (0.0006) [2023-12-27 03:44:38,980][105620] Updated weights for policy 1, policy_version 1698823 (0.0009) [2023-12-27 03:44:39,014][105692] Updated weights for policy 0, policy_version 1695104 (0.0005) [2023-12-27 03:44:39,691][105692] Updated weights for policy 0, policy_version 1695114 (0.0009) [2023-12-27 03:44:39,746][105692] Updated weights for policy 0, policy_version 1695124 (0.0009) [2023-12-27 03:44:39,794][105620] Updated weights for policy 1, policy_version 1698833 (0.0010) [2023-12-27 03:44:39,800][105692] Updated weights for policy 0, policy_version 1695134 (0.0009) [2023-12-27 03:44:39,860][105620] Updated weights for policy 1, policy_version 1698843 (0.0008) [2023-12-27 03:44:39,923][105620] Updated weights for policy 1, policy_version 1698853 (0.0010) [2023-12-27 03:44:39,983][105620] Updated weights for policy 1, policy_version 1698863 (0.0010) [2023-12-27 03:44:40,562][105692] Updated weights for policy 0, policy_version 1695144 (0.0008) [2023-12-27 03:44:40,616][105692] Updated weights for policy 0, policy_version 1695154 (0.0006) [2023-12-27 03:44:40,665][105692] Updated weights for policy 0, policy_version 1695164 (0.0005) [2023-12-27 03:44:40,779][105620] Updated weights for policy 1, policy_version 1698873 (0.0010) [2023-12-27 03:44:40,838][105620] Updated weights for policy 1, policy_version 1698883 (0.0010) [2023-12-27 03:44:40,900][105620] Updated weights for policy 1, policy_version 1698893 (0.0007) [2023-12-27 03:44:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.9, 300 sec: 19494.2). Total num frames: 869007360. Throughput: 0: 9791.2, 1: 9838.9. Samples: 869013960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:44:41,062][104569] Avg episode reward: [(0, '8709.870'), (1, '8714.520')] [2023-12-27 03:44:41,290][105692] Updated weights for policy 0, policy_version 1695174 (0.0007) [2023-12-27 03:44:41,357][105692] Updated weights for policy 0, policy_version 1695184 (0.0009) [2023-12-27 03:44:41,425][105692] Updated weights for policy 0, policy_version 1695194 (0.0008) [2023-12-27 03:44:41,626][105620] Updated weights for policy 1, policy_version 1698903 (0.0009) [2023-12-27 03:44:41,686][105620] Updated weights for policy 1, policy_version 1698913 (0.0008) [2023-12-27 03:44:41,749][105620] Updated weights for policy 1, policy_version 1698923 (0.0008) [2023-12-27 03:44:42,146][105692] Updated weights for policy 0, policy_version 1695204 (0.0009) [2023-12-27 03:44:42,208][105692] Updated weights for policy 0, policy_version 1695214 (0.0008) [2023-12-27 03:44:42,267][105692] Updated weights for policy 0, policy_version 1695224 (0.0008) [2023-12-27 03:44:42,536][105620] Updated weights for policy 1, policy_version 1698933 (0.0009) [2023-12-27 03:44:42,608][105620] Updated weights for policy 1, policy_version 1698943 (0.0010) [2023-12-27 03:44:42,671][105620] Updated weights for policy 1, policy_version 1698953 (0.0009) [2023-12-27 03:44:42,985][105692] Updated weights for policy 0, policy_version 1695234 (0.0009) [2023-12-27 03:44:43,031][105692] Updated weights for policy 0, policy_version 1695244 (0.0005) [2023-12-27 03:44:43,084][105692] Updated weights for policy 0, policy_version 1695254 (0.0008) [2023-12-27 03:44:43,135][105692] Updated weights for policy 0, policy_version 1695264 (0.0010) [2023-12-27 03:44:43,353][105620] Updated weights for policy 1, policy_version 1698963 (0.0008) [2023-12-27 03:44:43,399][105620] Updated weights for policy 1, policy_version 1698973 (0.0005) [2023-12-27 03:44:43,454][105620] Updated weights for policy 1, policy_version 1698983 (0.0005) [2023-12-27 03:44:43,813][105692] Updated weights for policy 0, policy_version 1695274 (0.0010) [2023-12-27 03:44:43,864][105692] Updated weights for policy 0, policy_version 1695284 (0.0010) [2023-12-27 03:44:43,916][105692] Updated weights for policy 0, policy_version 1695294 (0.0007) [2023-12-27 03:44:44,099][105620] Updated weights for policy 1, policy_version 1698993 (0.0006) [2023-12-27 03:44:44,148][105620] Updated weights for policy 1, policy_version 1699003 (0.0005) [2023-12-27 03:44:44,200][105620] Updated weights for policy 1, policy_version 1699013 (0.0005) [2023-12-27 03:44:44,249][105620] Updated weights for policy 1, policy_version 1699023 (0.0008) [2023-12-27 03:44:44,681][105692] Updated weights for policy 0, policy_version 1695304 (0.0009) [2023-12-27 03:44:44,740][105692] Updated weights for policy 0, policy_version 1695315 (0.0011) [2023-12-27 03:44:44,800][105692] Updated weights for policy 0, policy_version 1695325 (0.0008) [2023-12-27 03:44:44,869][105620] Updated weights for policy 1, policy_version 1699033 (0.0006) [2023-12-27 03:44:44,933][105620] Updated weights for policy 1, policy_version 1699043 (0.0006) [2023-12-27 03:44:44,993][105620] Updated weights for policy 1, policy_version 1699053 (0.0009) [2023-12-27 03:44:45,459][105692] Updated weights for policy 0, policy_version 1695335 (0.0009) [2023-12-27 03:44:45,516][105692] Updated weights for policy 0, policy_version 1695346 (0.0009) [2023-12-27 03:44:45,557][105620] Updated weights for policy 1, policy_version 1699063 (0.0007) [2023-12-27 03:44:45,574][105692] Updated weights for policy 0, policy_version 1695356 (0.0009) [2023-12-27 03:44:45,618][105620] Updated weights for policy 1, policy_version 1699073 (0.0009) [2023-12-27 03:44:45,663][105620] Updated weights for policy 1, policy_version 1699083 (0.0010) [2023-12-27 03:44:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19661.0, 300 sec: 19494.2). Total num frames: 869105664. Throughput: 0: 9823.3, 1: 9771.9. Samples: 869073200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:44:46,062][104569] Avg episode reward: [(0, '8526.133'), (1, '8713.716')] [2023-12-27 03:44:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001695360_434077696.pth... [2023-12-27 03:44:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001699088_435027968.pth... [2023-12-27 03:44:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001694240_433790976.pth [2023-12-27 03:44:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001697936_434733056.pth [2023-12-27 03:44:46,344][105692] Updated weights for policy 0, policy_version 1695366 (0.0009) [2023-12-27 03:44:46,381][105620] Updated weights for policy 1, policy_version 1699093 (0.0010) [2023-12-27 03:44:46,390][105692] Updated weights for policy 0, policy_version 1695376 (0.0009) [2023-12-27 03:44:46,439][105692] Updated weights for policy 0, policy_version 1695386 (0.0007) [2023-12-27 03:44:46,442][105620] Updated weights for policy 1, policy_version 1699103 (0.0010) [2023-12-27 03:44:46,504][105620] Updated weights for policy 1, policy_version 1699113 (0.0010) [2023-12-27 03:44:47,092][105692] Updated weights for policy 0, policy_version 1695396 (0.0009) [2023-12-27 03:44:47,153][105692] Updated weights for policy 0, policy_version 1695406 (0.0008) [2023-12-27 03:44:47,217][105692] Updated weights for policy 0, policy_version 1695416 (0.0005) [2023-12-27 03:44:47,254][105620] Updated weights for policy 1, policy_version 1699123 (0.0011) [2023-12-27 03:44:47,306][105620] Updated weights for policy 1, policy_version 1699133 (0.0011) [2023-12-27 03:44:47,370][105620] Updated weights for policy 1, policy_version 1699143 (0.0011) [2023-12-27 03:44:47,826][105692] Updated weights for policy 0, policy_version 1695426 (0.0006) [2023-12-27 03:44:47,890][105692] Updated weights for policy 0, policy_version 1695436 (0.0005) [2023-12-27 03:44:47,943][105692] Updated weights for policy 0, policy_version 1695446 (0.0008) [2023-12-27 03:44:48,005][105692] Updated weights for policy 0, policy_version 1695456 (0.0011) [2023-12-27 03:44:48,037][105620] Updated weights for policy 1, policy_version 1699153 (0.0011) [2023-12-27 03:44:48,099][105620] Updated weights for policy 1, policy_version 1699163 (0.0010) [2023-12-27 03:44:48,157][105620] Updated weights for policy 1, policy_version 1699173 (0.0010) [2023-12-27 03:44:48,205][105620] Updated weights for policy 1, policy_version 1699183 (0.0010) [2023-12-27 03:44:48,705][105692] Updated weights for policy 0, policy_version 1695466 (0.0010) [2023-12-27 03:44:48,767][105692] Updated weights for policy 0, policy_version 1695476 (0.0010) [2023-12-27 03:44:48,824][105692] Updated weights for policy 0, policy_version 1695486 (0.0006) [2023-12-27 03:44:48,913][105620] Updated weights for policy 1, policy_version 1699193 (0.0006) [2023-12-27 03:44:48,968][105620] Updated weights for policy 1, policy_version 1699203 (0.0006) [2023-12-27 03:44:49,018][105620] Updated weights for policy 1, policy_version 1699213 (0.0006) [2023-12-27 03:44:49,430][105692] Updated weights for policy 0, policy_version 1695496 (0.0005) [2023-12-27 03:44:49,475][105692] Updated weights for policy 0, policy_version 1695506 (0.0005) [2023-12-27 03:44:49,529][105692] Updated weights for policy 0, policy_version 1695516 (0.0005) [2023-12-27 03:44:49,683][105620] Updated weights for policy 1, policy_version 1699223 (0.0006) [2023-12-27 03:44:49,731][105620] Updated weights for policy 1, policy_version 1699233 (0.0005) [2023-12-27 03:44:49,793][105620] Updated weights for policy 1, policy_version 1699243 (0.0007) [2023-12-27 03:44:50,347][105692] Updated weights for policy 0, policy_version 1695526 (0.0009) [2023-12-27 03:44:50,402][105692] Updated weights for policy 0, policy_version 1695536 (0.0008) [2023-12-27 03:44:50,410][105620] Updated weights for policy 1, policy_version 1699253 (0.0006) [2023-12-27 03:44:50,449][105692] Updated weights for policy 0, policy_version 1695546 (0.0006) [2023-12-27 03:44:50,467][105620] Updated weights for policy 1, policy_version 1699263 (0.0008) [2023-12-27 03:44:50,522][105620] Updated weights for policy 1, policy_version 1699273 (0.0009) [2023-12-27 03:44:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 869203968. Throughput: 0: 9863.6, 1: 9861.5. Samples: 869196480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:44:51,062][104569] Avg episode reward: [(0, '8433.599'), (1, '9079.926')] [2023-12-27 03:44:51,181][105620] Updated weights for policy 1, policy_version 1699283 (0.0009) [2023-12-27 03:44:51,236][105620] Updated weights for policy 1, policy_version 1699293 (0.0009) [2023-12-27 03:44:51,292][105692] Updated weights for policy 0, policy_version 1695556 (0.0008) [2023-12-27 03:44:51,293][105620] Updated weights for policy 1, policy_version 1699303 (0.0010) [2023-12-27 03:44:51,357][105692] Updated weights for policy 0, policy_version 1695566 (0.0009) [2023-12-27 03:44:51,420][105692] Updated weights for policy 0, policy_version 1695576 (0.0008) [2023-12-27 03:44:52,077][105692] Updated weights for policy 0, policy_version 1695586 (0.0006) [2023-12-27 03:44:52,122][105620] Updated weights for policy 1, policy_version 1699313 (0.0010) [2023-12-27 03:44:52,126][105692] Updated weights for policy 0, policy_version 1695596 (0.0006) [2023-12-27 03:44:52,177][105692] Updated weights for policy 0, policy_version 1695606 (0.0005) [2023-12-27 03:44:52,182][105620] Updated weights for policy 1, policy_version 1699323 (0.0009) [2023-12-27 03:44:52,227][105692] Updated weights for policy 0, policy_version 1695616 (0.0005) [2023-12-27 03:44:52,246][105620] Updated weights for policy 1, policy_version 1699333 (0.0009) [2023-12-27 03:44:52,303][105620] Updated weights for policy 1, policy_version 1699343 (0.0009) [2023-12-27 03:44:52,883][105692] Updated weights for policy 0, policy_version 1695626 (0.0010) [2023-12-27 03:44:52,934][105692] Updated weights for policy 0, policy_version 1695636 (0.0010) [2023-12-27 03:44:53,000][105692] Updated weights for policy 0, policy_version 1695646 (0.0011) [2023-12-27 03:44:53,113][105620] Updated weights for policy 1, policy_version 1699353 (0.0010) [2023-12-27 03:44:53,177][105620] Updated weights for policy 1, policy_version 1699363 (0.0010) [2023-12-27 03:44:53,225][105620] Updated weights for policy 1, policy_version 1699373 (0.0010) [2023-12-27 03:44:53,613][105692] Updated weights for policy 0, policy_version 1695656 (0.0006) [2023-12-27 03:44:53,659][105692] Updated weights for policy 0, policy_version 1695666 (0.0006) [2023-12-27 03:44:53,718][105692] Updated weights for policy 0, policy_version 1695676 (0.0011) [2023-12-27 03:44:53,859][105620] Updated weights for policy 1, policy_version 1699383 (0.0009) [2023-12-27 03:44:53,912][105620] Updated weights for policy 1, policy_version 1699393 (0.0010) [2023-12-27 03:44:53,972][105620] Updated weights for policy 1, policy_version 1699403 (0.0008) [2023-12-27 03:44:54,436][105692] Updated weights for policy 0, policy_version 1695686 (0.0008) [2023-12-27 03:44:54,499][105692] Updated weights for policy 0, policy_version 1695696 (0.0006) [2023-12-27 03:44:54,550][105692] Updated weights for policy 0, policy_version 1695706 (0.0007) [2023-12-27 03:44:54,558][105620] Updated weights for policy 1, policy_version 1699413 (0.0010) [2023-12-27 03:44:54,611][105620] Updated weights for policy 1, policy_version 1699423 (0.0010) [2023-12-27 03:44:54,674][105620] Updated weights for policy 1, policy_version 1699433 (0.0011) [2023-12-27 03:44:55,121][105692] Updated weights for policy 0, policy_version 1695716 (0.0006) [2023-12-27 03:44:55,179][105692] Updated weights for policy 0, policy_version 1695726 (0.0007) [2023-12-27 03:44:55,240][105692] Updated weights for policy 0, policy_version 1695736 (0.0010) [2023-12-27 03:44:55,396][105620] Updated weights for policy 1, policy_version 1699443 (0.0010) [2023-12-27 03:44:55,448][105620] Updated weights for policy 1, policy_version 1699453 (0.0010) [2023-12-27 03:44:55,504][105620] Updated weights for policy 1, policy_version 1699463 (0.0005) [2023-12-27 03:44:55,877][105692] Updated weights for policy 0, policy_version 1695746 (0.0010) [2023-12-27 03:44:55,944][105692] Updated weights for policy 0, policy_version 1695756 (0.0009) [2023-12-27 03:44:56,002][105692] Updated weights for policy 0, policy_version 1695766 (0.0009) [2023-12-27 03:44:56,062][105692] Updated weights for policy 0, policy_version 1695776 (0.0009) [2023-12-27 03:44:56,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 869310464. Throughput: 0: 9871.0, 1: 9952.2. Samples: 869316352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:44:56,062][104569] Avg episode reward: [(0, '8255.213'), (1, '9081.905')] [2023-12-27 03:44:56,219][105620] Updated weights for policy 1, policy_version 1699473 (0.0006) [2023-12-27 03:44:56,277][105620] Updated weights for policy 1, policy_version 1699483 (0.0010) [2023-12-27 03:44:56,330][105620] Updated weights for policy 1, policy_version 1699493 (0.0009) [2023-12-27 03:44:56,388][105620] Updated weights for policy 1, policy_version 1699503 (0.0010) [2023-12-27 03:44:56,644][105692] Updated weights for policy 0, policy_version 1695786 (0.0010) [2023-12-27 03:44:56,689][105692] Updated weights for policy 0, policy_version 1695796 (0.0010) [2023-12-27 03:44:56,733][105692] Updated weights for policy 0, policy_version 1695806 (0.0010) [2023-12-27 03:44:57,138][105620] Updated weights for policy 1, policy_version 1699513 (0.0010) [2023-12-27 03:44:57,193][105620] Updated weights for policy 1, policy_version 1699523 (0.0010) [2023-12-27 03:44:57,242][105620] Updated weights for policy 1, policy_version 1699533 (0.0011) [2023-12-27 03:44:57,404][105692] Updated weights for policy 0, policy_version 1695816 (0.0008) [2023-12-27 03:44:57,463][105692] Updated weights for policy 0, policy_version 1695826 (0.0006) [2023-12-27 03:44:57,525][105692] Updated weights for policy 0, policy_version 1695836 (0.0008) [2023-12-27 03:44:57,880][105620] Updated weights for policy 1, policy_version 1699543 (0.0007) [2023-12-27 03:44:57,939][105620] Updated weights for policy 1, policy_version 1699553 (0.0005) [2023-12-27 03:44:58,004][105620] Updated weights for policy 1, policy_version 1699563 (0.0005) [2023-12-27 03:44:58,193][105692] Updated weights for policy 0, policy_version 1695846 (0.0009) [2023-12-27 03:44:58,252][105692] Updated weights for policy 0, policy_version 1695856 (0.0010) [2023-12-27 03:44:58,314][105692] Updated weights for policy 0, policy_version 1695866 (0.0011) [2023-12-27 03:44:58,604][105620] Updated weights for policy 1, policy_version 1699573 (0.0007) [2023-12-27 03:44:58,654][105620] Updated weights for policy 1, policy_version 1699583 (0.0008) [2023-12-27 03:44:58,700][105620] Updated weights for policy 1, policy_version 1699593 (0.0008) [2023-12-27 03:44:59,095][105692] Updated weights for policy 0, policy_version 1695876 (0.0009) [2023-12-27 03:44:59,156][105692] Updated weights for policy 0, policy_version 1695886 (0.0008) [2023-12-27 03:44:59,217][105692] Updated weights for policy 0, policy_version 1695896 (0.0007) [2023-12-27 03:44:59,570][105620] Updated weights for policy 1, policy_version 1699603 (0.0007) [2023-12-27 03:44:59,631][105620] Updated weights for policy 1, policy_version 1699613 (0.0006) [2023-12-27 03:44:59,701][105620] Updated weights for policy 1, policy_version 1699623 (0.0005) [2023-12-27 03:45:00,034][105692] Updated weights for policy 0, policy_version 1695906 (0.0008) [2023-12-27 03:45:00,091][105692] Updated weights for policy 0, policy_version 1695916 (0.0008) [2023-12-27 03:45:00,153][105692] Updated weights for policy 0, policy_version 1695926 (0.0010) [2023-12-27 03:45:00,214][105692] Updated weights for policy 0, policy_version 1695936 (0.0010) [2023-12-27 03:45:00,293][105620] Updated weights for policy 1, policy_version 1699633 (0.0005) [2023-12-27 03:45:00,360][105620] Updated weights for policy 1, policy_version 1699643 (0.0007) [2023-12-27 03:45:00,419][105620] Updated weights for policy 1, policy_version 1699653 (0.0007) [2023-12-27 03:45:00,482][105620] Updated weights for policy 1, policy_version 1699663 (0.0005) [2023-12-27 03:45:00,902][105692] Updated weights for policy 0, policy_version 1695946 (0.0005) [2023-12-27 03:45:00,950][105692] Updated weights for policy 0, policy_version 1695956 (0.0005) [2023-12-27 03:45:00,997][105692] Updated weights for policy 0, policy_version 1695966 (0.0005) [2023-12-27 03:45:01,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 869408768. Throughput: 0: 9919.3, 1: 9962.8. Samples: 869378324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:45:01,063][104569] Avg episode reward: [(0, '8350.118'), (1, '9081.876')] [2023-12-27 03:45:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001695968_434233344.pth... [2023-12-27 03:45:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001699664_435175424.pth... [2023-12-27 03:45:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001694816_433938432.pth [2023-12-27 03:45:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001698512_434880512.pth [2023-12-27 03:45:01,219][105620] Updated weights for policy 1, policy_version 1699673 (0.0009) [2023-12-27 03:45:01,286][105620] Updated weights for policy 1, policy_version 1699683 (0.0007) [2023-12-27 03:45:01,356][105620] Updated weights for policy 1, policy_version 1699693 (0.0006) [2023-12-27 03:45:01,677][105692] Updated weights for policy 0, policy_version 1695976 (0.0008) [2023-12-27 03:45:01,738][105692] Updated weights for policy 0, policy_version 1695986 (0.0008) [2023-12-27 03:45:01,798][105692] Updated weights for policy 0, policy_version 1695996 (0.0008) [2023-12-27 03:45:02,051][105620] Updated weights for policy 1, policy_version 1699703 (0.0006) [2023-12-27 03:45:02,111][105620] Updated weights for policy 1, policy_version 1699713 (0.0006) [2023-12-27 03:45:02,178][105620] Updated weights for policy 1, policy_version 1699723 (0.0006) [2023-12-27 03:45:02,533][105692] Updated weights for policy 0, policy_version 1696006 (0.0009) [2023-12-27 03:45:02,603][105692] Updated weights for policy 0, policy_version 1696016 (0.0009) [2023-12-27 03:45:02,673][105692] Updated weights for policy 0, policy_version 1696026 (0.0010) [2023-12-27 03:45:02,782][105620] Updated weights for policy 1, policy_version 1699733 (0.0008) [2023-12-27 03:45:02,848][105620] Updated weights for policy 1, policy_version 1699743 (0.0009) [2023-12-27 03:45:02,913][105620] Updated weights for policy 1, policy_version 1699753 (0.0009) [2023-12-27 03:45:03,328][105692] Updated weights for policy 0, policy_version 1696036 (0.0009) [2023-12-27 03:45:03,380][105692] Updated weights for policy 0, policy_version 1696046 (0.0009) [2023-12-27 03:45:03,427][105692] Updated weights for policy 0, policy_version 1696056 (0.0009) [2023-12-27 03:45:03,691][105620] Updated weights for policy 1, policy_version 1699763 (0.0009) [2023-12-27 03:45:03,740][105620] Updated weights for policy 1, policy_version 1699773 (0.0008) [2023-12-27 03:45:03,790][105620] Updated weights for policy 1, policy_version 1699783 (0.0009) [2023-12-27 03:45:04,209][105692] Updated weights for policy 0, policy_version 1696066 (0.0009) [2023-12-27 03:45:04,276][105692] Updated weights for policy 0, policy_version 1696076 (0.0010) [2023-12-27 03:45:04,348][105692] Updated weights for policy 0, policy_version 1696086 (0.0010) [2023-12-27 03:45:04,406][105692] Updated weights for policy 0, policy_version 1696096 (0.0010) [2023-12-27 03:45:04,515][105620] Updated weights for policy 1, policy_version 1699793 (0.0009) [2023-12-27 03:45:04,573][105620] Updated weights for policy 1, policy_version 1699803 (0.0009) [2023-12-27 03:45:04,635][105620] Updated weights for policy 1, policy_version 1699813 (0.0009) [2023-12-27 03:45:04,695][105620] Updated weights for policy 1, policy_version 1699823 (0.0009) [2023-12-27 03:45:05,202][105692] Updated weights for policy 0, policy_version 1696106 (0.0009) [2023-12-27 03:45:05,262][105692] Updated weights for policy 0, policy_version 1696116 (0.0010) [2023-12-27 03:45:05,320][105692] Updated weights for policy 0, policy_version 1696126 (0.0009) [2023-12-27 03:45:05,382][105620] Updated weights for policy 1, policy_version 1699833 (0.0006) [2023-12-27 03:45:05,450][105620] Updated weights for policy 1, policy_version 1699843 (0.0005) [2023-12-27 03:45:05,517][105620] Updated weights for policy 1, policy_version 1699853 (0.0006) [2023-12-27 03:45:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 869498880. Throughput: 0: 9832.5, 1: 9873.5. Samples: 869492740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:45:06,062][104569] Avg episode reward: [(0, '8712.935'), (1, '9081.887')] [2023-12-27 03:45:06,092][105620] Updated weights for policy 1, policy_version 1699863 (0.0008) [2023-12-27 03:45:06,153][105620] Updated weights for policy 1, policy_version 1699873 (0.0008) [2023-12-27 03:45:06,164][105692] Updated weights for policy 0, policy_version 1696136 (0.0009) [2023-12-27 03:45:06,207][105620] Updated weights for policy 1, policy_version 1699883 (0.0008) [2023-12-27 03:45:06,228][105692] Updated weights for policy 0, policy_version 1696146 (0.0008) [2023-12-27 03:45:06,289][105692] Updated weights for policy 0, policy_version 1696156 (0.0009) [2023-12-27 03:45:06,841][105620] Updated weights for policy 1, policy_version 1699893 (0.0007) [2023-12-27 03:45:06,900][105620] Updated weights for policy 1, policy_version 1699903 (0.0011) [2023-12-27 03:45:06,963][105620] Updated weights for policy 1, policy_version 1699913 (0.0010) [2023-12-27 03:45:07,153][105692] Updated weights for policy 0, policy_version 1696166 (0.0009) [2023-12-27 03:45:07,206][105692] Updated weights for policy 0, policy_version 1696176 (0.0008) [2023-12-27 03:45:07,256][105692] Updated weights for policy 0, policy_version 1696186 (0.0008) [2023-12-27 03:45:07,597][105620] Updated weights for policy 1, policy_version 1699923 (0.0008) [2023-12-27 03:45:07,661][105620] Updated weights for policy 1, policy_version 1699933 (0.0007) [2023-12-27 03:45:07,727][105620] Updated weights for policy 1, policy_version 1699943 (0.0008) [2023-12-27 03:45:08,081][105692] Updated weights for policy 0, policy_version 1696196 (0.0008) [2023-12-27 03:45:08,142][105692] Updated weights for policy 0, policy_version 1696206 (0.0008) [2023-12-27 03:45:08,204][105692] Updated weights for policy 0, policy_version 1696216 (0.0009) [2023-12-27 03:45:08,333][105620] Updated weights for policy 1, policy_version 1699953 (0.0005) [2023-12-27 03:45:08,398][105620] Updated weights for policy 1, policy_version 1699963 (0.0006) [2023-12-27 03:45:08,467][105620] Updated weights for policy 1, policy_version 1699973 (0.0007) [2023-12-27 03:45:08,528][105620] Updated weights for policy 1, policy_version 1699983 (0.0006) [2023-12-27 03:45:09,033][105692] Updated weights for policy 0, policy_version 1696226 (0.0009) [2023-12-27 03:45:09,096][105692] Updated weights for policy 0, policy_version 1696236 (0.0009) [2023-12-27 03:45:09,128][105620] Updated weights for policy 1, policy_version 1699993 (0.0006) [2023-12-27 03:45:09,153][105692] Updated weights for policy 0, policy_version 1696246 (0.0008) [2023-12-27 03:45:09,177][105620] Updated weights for policy 1, policy_version 1700003 (0.0006) [2023-12-27 03:45:09,210][105692] Updated weights for policy 0, policy_version 1696256 (0.0010) [2023-12-27 03:45:09,245][105620] Updated weights for policy 1, policy_version 1700013 (0.0011) [2023-12-27 03:45:09,881][105620] Updated weights for policy 1, policy_version 1700023 (0.0008) [2023-12-27 03:45:09,951][105620] Updated weights for policy 1, policy_version 1700033 (0.0007) [2023-12-27 03:45:09,995][105692] Updated weights for policy 0, policy_version 1696266 (0.0007) [2023-12-27 03:45:10,013][105620] Updated weights for policy 1, policy_version 1700043 (0.0010) [2023-12-27 03:45:10,053][105692] Updated weights for policy 0, policy_version 1696276 (0.0007) [2023-12-27 03:45:10,114][105692] Updated weights for policy 0, policy_version 1696286 (0.0008) [2023-12-27 03:45:10,781][105620] Updated weights for policy 1, policy_version 1700053 (0.0010) [2023-12-27 03:45:10,804][105692] Updated weights for policy 0, policy_version 1696296 (0.0006) [2023-12-27 03:45:10,833][105620] Updated weights for policy 1, policy_version 1700063 (0.0010) [2023-12-27 03:45:10,864][105692] Updated weights for policy 0, policy_version 1696306 (0.0006) [2023-12-27 03:45:10,892][105620] Updated weights for policy 1, policy_version 1700073 (0.0010) [2023-12-27 03:45:10,922][105692] Updated weights for policy 0, policy_version 1696316 (0.0009) [2023-12-27 03:45:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19933.8, 300 sec: 19577.5). Total num frames: 869605376. Throughput: 0: 9721.4, 1: 10029.5. Samples: 869608124. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:45:11,063][104569] Avg episode reward: [(0, '8525.863'), (1, '8713.552')] [2023-12-27 03:45:11,619][105620] Updated weights for policy 1, policy_version 1700083 (0.0010) [2023-12-27 03:45:11,684][105620] Updated weights for policy 1, policy_version 1700093 (0.0012) [2023-12-27 03:45:11,729][105692] Updated weights for policy 0, policy_version 1696326 (0.0007) [2023-12-27 03:45:11,744][105620] Updated weights for policy 1, policy_version 1700103 (0.0010) [2023-12-27 03:45:11,796][105692] Updated weights for policy 0, policy_version 1696336 (0.0008) [2023-12-27 03:45:11,850][105692] Updated weights for policy 0, policy_version 1696346 (0.0009) [2023-12-27 03:45:12,491][105620] Updated weights for policy 1, policy_version 1700113 (0.0009) [2023-12-27 03:45:12,540][105620] Updated weights for policy 1, policy_version 1700123 (0.0009) [2023-12-27 03:45:12,601][105620] Updated weights for policy 1, policy_version 1700133 (0.0009) [2023-12-27 03:45:12,624][105692] Updated weights for policy 0, policy_version 1696356 (0.0008) [2023-12-27 03:45:12,656][105620] Updated weights for policy 1, policy_version 1700143 (0.0006) [2023-12-27 03:45:12,688][105692] Updated weights for policy 0, policy_version 1696366 (0.0009) [2023-12-27 03:45:12,743][105692] Updated weights for policy 0, policy_version 1696376 (0.0009) [2023-12-27 03:45:13,428][105620] Updated weights for policy 1, policy_version 1700153 (0.0010) [2023-12-27 03:45:13,479][105692] Updated weights for policy 0, policy_version 1696386 (0.0008) [2023-12-27 03:45:13,487][105620] Updated weights for policy 1, policy_version 1700163 (0.0010) [2023-12-27 03:45:13,537][105692] Updated weights for policy 0, policy_version 1696396 (0.0005) [2023-12-27 03:45:13,542][105620] Updated weights for policy 1, policy_version 1700173 (0.0010) [2023-12-27 03:45:13,591][105692] Updated weights for policy 0, policy_version 1696406 (0.0009) [2023-12-27 03:45:13,638][105692] Updated weights for policy 0, policy_version 1696416 (0.0009) [2023-12-27 03:45:14,198][105620] Updated weights for policy 1, policy_version 1700183 (0.0010) [2023-12-27 03:45:14,250][105620] Updated weights for policy 1, policy_version 1700193 (0.0010) [2023-12-27 03:45:14,302][105620] Updated weights for policy 1, policy_version 1700203 (0.0010) [2023-12-27 03:45:14,339][105692] Updated weights for policy 0, policy_version 1696426 (0.0005) [2023-12-27 03:45:14,392][105692] Updated weights for policy 0, policy_version 1696436 (0.0006) [2023-12-27 03:45:14,442][105692] Updated weights for policy 0, policy_version 1696446 (0.0007) [2023-12-27 03:45:15,001][105620] Updated weights for policy 1, policy_version 1700213 (0.0010) [2023-12-27 03:45:15,056][105620] Updated weights for policy 1, policy_version 1700223 (0.0006) [2023-12-27 03:45:15,111][105620] Updated weights for policy 1, policy_version 1700233 (0.0005) [2023-12-27 03:45:15,226][105692] Updated weights for policy 0, policy_version 1696456 (0.0009) [2023-12-27 03:45:15,289][105692] Updated weights for policy 0, policy_version 1696466 (0.0010) [2023-12-27 03:45:15,347][105692] Updated weights for policy 0, policy_version 1696476 (0.0008) [2023-12-27 03:45:15,703][105620] Updated weights for policy 1, policy_version 1700243 (0.0007) [2023-12-27 03:45:15,750][105620] Updated weights for policy 1, policy_version 1700253 (0.0010) [2023-12-27 03:45:15,794][105620] Updated weights for policy 1, policy_version 1700263 (0.0010) [2023-12-27 03:45:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 869695488. Throughput: 0: 9615.8, 1: 9981.4. Samples: 869665172. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:45:16,062][104569] Avg episode reward: [(0, '8434.644'), (1, '8438.387')] [2023-12-27 03:45:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001696480_434364416.pth... [2023-12-27 03:45:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001700272_435331072.pth... [2023-12-27 03:45:16,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001695360_434077696.pth [2023-12-27 03:45:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001699088_435027968.pth [2023-12-27 03:45:16,197][105692] Updated weights for policy 0, policy_version 1696486 (0.0009) [2023-12-27 03:45:16,263][105692] Updated weights for policy 0, policy_version 1696496 (0.0010) [2023-12-27 03:45:16,330][105692] Updated weights for policy 0, policy_version 1696506 (0.0010) [2023-12-27 03:45:16,390][105620] Updated weights for policy 1, policy_version 1700273 (0.0010) [2023-12-27 03:45:16,448][105620] Updated weights for policy 1, policy_version 1700283 (0.0005) [2023-12-27 03:45:16,494][105620] Updated weights for policy 1, policy_version 1700293 (0.0005) [2023-12-27 03:45:16,540][105620] Updated weights for policy 1, policy_version 1700303 (0.0005) [2023-12-27 03:45:17,109][105620] Updated weights for policy 1, policy_version 1700313 (0.0006) [2023-12-27 03:45:17,174][105620] Updated weights for policy 1, policy_version 1700323 (0.0008) [2023-12-27 03:45:17,214][105692] Updated weights for policy 0, policy_version 1696516 (0.0008) [2023-12-27 03:45:17,237][105620] Updated weights for policy 1, policy_version 1700333 (0.0008) [2023-12-27 03:45:17,270][105692] Updated weights for policy 0, policy_version 1696526 (0.0007) [2023-12-27 03:45:17,318][105692] Updated weights for policy 0, policy_version 1696536 (0.0009) [2023-12-27 03:45:17,907][105620] Updated weights for policy 1, policy_version 1700343 (0.0008) [2023-12-27 03:45:17,966][105620] Updated weights for policy 1, policy_version 1700353 (0.0009) [2023-12-27 03:45:18,048][105620] Updated weights for policy 1, policy_version 1700363 (0.0009) [2023-12-27 03:45:18,058][105692] Updated weights for policy 0, policy_version 1696546 (0.0009) [2023-12-27 03:45:18,111][105692] Updated weights for policy 0, policy_version 1696556 (0.0008) [2023-12-27 03:45:18,173][105692] Updated weights for policy 0, policy_version 1696566 (0.0008) [2023-12-27 03:45:18,224][105692] Updated weights for policy 0, policy_version 1696576 (0.0009) [2023-12-27 03:45:18,712][105620] Updated weights for policy 1, policy_version 1700373 (0.0008) [2023-12-27 03:45:18,768][105620] Updated weights for policy 1, policy_version 1700383 (0.0009) [2023-12-27 03:45:18,837][105620] Updated weights for policy 1, policy_version 1700393 (0.0011) [2023-12-27 03:45:19,048][105692] Updated weights for policy 0, policy_version 1696586 (0.0009) [2023-12-27 03:45:19,105][105692] Updated weights for policy 0, policy_version 1696596 (0.0008) [2023-12-27 03:45:19,161][105692] Updated weights for policy 0, policy_version 1696606 (0.0008) [2023-12-27 03:45:19,542][105620] Updated weights for policy 1, policy_version 1700403 (0.0011) [2023-12-27 03:45:19,597][105620] Updated weights for policy 1, policy_version 1700413 (0.0011) [2023-12-27 03:45:19,646][105620] Updated weights for policy 1, policy_version 1700423 (0.0011) [2023-12-27 03:45:19,930][105692] Updated weights for policy 0, policy_version 1696616 (0.0009) [2023-12-27 03:45:20,000][105692] Updated weights for policy 0, policy_version 1696626 (0.0008) [2023-12-27 03:45:20,072][105692] Updated weights for policy 0, policy_version 1696636 (0.0009) [2023-12-27 03:45:20,324][105620] Updated weights for policy 1, policy_version 1700433 (0.0010) [2023-12-27 03:45:20,389][105620] Updated weights for policy 1, policy_version 1700443 (0.0006) [2023-12-27 03:45:20,448][105620] Updated weights for policy 1, policy_version 1700453 (0.0008) [2023-12-27 03:45:20,506][105620] Updated weights for policy 1, policy_version 1700463 (0.0010) [2023-12-27 03:45:20,896][105692] Updated weights for policy 0, policy_version 1696646 (0.0008) [2023-12-27 03:45:20,954][105692] Updated weights for policy 0, policy_version 1696656 (0.0009) [2023-12-27 03:45:21,011][105692] Updated weights for policy 0, policy_version 1696666 (0.0009) [2023-12-27 03:45:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 869793792. Throughput: 0: 9543.6, 1: 10046.2. Samples: 869782128. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:45:21,062][104569] Avg episode reward: [(0, '8348.767'), (1, '8989.619')] [2023-12-27 03:45:21,130][105620] Updated weights for policy 1, policy_version 1700473 (0.0008) [2023-12-27 03:45:21,178][105620] Updated weights for policy 1, policy_version 1700483 (0.0008) [2023-12-27 03:45:21,231][105620] Updated weights for policy 1, policy_version 1700493 (0.0008) [2023-12-27 03:45:21,859][105692] Updated weights for policy 0, policy_version 1696676 (0.0008) [2023-12-27 03:45:21,926][105692] Updated weights for policy 0, policy_version 1696686 (0.0008) [2023-12-27 03:45:21,991][105692] Updated weights for policy 0, policy_version 1696696 (0.0008) [2023-12-27 03:45:22,028][105620] Updated weights for policy 1, policy_version 1700503 (0.0010) [2023-12-27 03:45:22,091][105620] Updated weights for policy 1, policy_version 1700513 (0.0011) [2023-12-27 03:45:22,149][105620] Updated weights for policy 1, policy_version 1700523 (0.0011) [2023-12-27 03:45:22,749][105692] Updated weights for policy 0, policy_version 1696706 (0.0006) [2023-12-27 03:45:22,799][105692] Updated weights for policy 0, policy_version 1696716 (0.0008) [2023-12-27 03:45:22,859][105692] Updated weights for policy 0, policy_version 1696726 (0.0008) [2023-12-27 03:45:22,912][105620] Updated weights for policy 1, policy_version 1700533 (0.0010) [2023-12-27 03:45:22,919][105692] Updated weights for policy 0, policy_version 1696736 (0.0008) [2023-12-27 03:45:22,974][105620] Updated weights for policy 1, policy_version 1700543 (0.0011) [2023-12-27 03:45:23,041][105620] Updated weights for policy 1, policy_version 1700553 (0.0011) [2023-12-27 03:45:23,696][105620] Updated weights for policy 1, policy_version 1700563 (0.0010) [2023-12-27 03:45:23,718][105692] Updated weights for policy 0, policy_version 1696746 (0.0009) [2023-12-27 03:45:23,748][105620] Updated weights for policy 1, policy_version 1700573 (0.0007) [2023-12-27 03:45:23,774][105692] Updated weights for policy 0, policy_version 1696756 (0.0008) [2023-12-27 03:45:23,796][105620] Updated weights for policy 1, policy_version 1700583 (0.0006) [2023-12-27 03:45:23,826][105692] Updated weights for policy 0, policy_version 1696766 (0.0007) [2023-12-27 03:45:24,451][105620] Updated weights for policy 1, policy_version 1700593 (0.0007) [2023-12-27 03:45:24,512][105620] Updated weights for policy 1, policy_version 1700603 (0.0009) [2023-12-27 03:45:24,572][105620] Updated weights for policy 1, policy_version 1700613 (0.0009) [2023-12-27 03:45:24,629][105692] Updated weights for policy 0, policy_version 1696776 (0.0007) [2023-12-27 03:45:24,632][105620] Updated weights for policy 1, policy_version 1700623 (0.0007) [2023-12-27 03:45:24,678][105692] Updated weights for policy 0, policy_version 1696786 (0.0008) [2023-12-27 03:45:24,728][105692] Updated weights for policy 0, policy_version 1696796 (0.0009) [2023-12-27 03:45:25,345][105620] Updated weights for policy 1, policy_version 1700633 (0.0008) [2023-12-27 03:45:25,393][105620] Updated weights for policy 1, policy_version 1700643 (0.0009) [2023-12-27 03:45:25,457][105620] Updated weights for policy 1, policy_version 1700653 (0.0008) [2023-12-27 03:45:25,507][105692] Updated weights for policy 0, policy_version 1696806 (0.0009) [2023-12-27 03:45:25,562][105692] Updated weights for policy 0, policy_version 1696816 (0.0009) [2023-12-27 03:45:25,623][105692] Updated weights for policy 0, policy_version 1696826 (0.0008) [2023-12-27 03:45:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 869883904. Throughput: 0: 9467.2, 1: 10095.4. Samples: 869894276. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:45:26,062][104569] Avg episode reward: [(0, '8624.148'), (1, '9172.070')] [2023-12-27 03:45:26,233][105620] Updated weights for policy 1, policy_version 1700663 (0.0009) [2023-12-27 03:45:26,287][105620] Updated weights for policy 1, policy_version 1700673 (0.0008) [2023-12-27 03:45:26,348][105620] Updated weights for policy 1, policy_version 1700683 (0.0009) [2023-12-27 03:45:26,389][105692] Updated weights for policy 0, policy_version 1696836 (0.0008) [2023-12-27 03:45:26,436][105692] Updated weights for policy 0, policy_version 1696846 (0.0009) [2023-12-27 03:45:26,488][105692] Updated weights for policy 0, policy_version 1696856 (0.0009) [2023-12-27 03:45:27,113][105620] Updated weights for policy 1, policy_version 1700693 (0.0009) [2023-12-27 03:45:27,114][105692] Updated weights for policy 0, policy_version 1696866 (0.0009) [2023-12-27 03:45:27,158][105692] Updated weights for policy 0, policy_version 1696876 (0.0007) [2023-12-27 03:45:27,171][105620] Updated weights for policy 1, policy_version 1700703 (0.0007) [2023-12-27 03:45:27,208][105692] Updated weights for policy 0, policy_version 1696886 (0.0007) [2023-12-27 03:45:27,229][105620] Updated weights for policy 1, policy_version 1700713 (0.0008) [2023-12-27 03:45:27,260][105692] Updated weights for policy 0, policy_version 1696896 (0.0007) [2023-12-27 03:45:27,918][105620] Updated weights for policy 1, policy_version 1700723 (0.0010) [2023-12-27 03:45:27,964][105620] Updated weights for policy 1, policy_version 1700733 (0.0008) [2023-12-27 03:45:28,011][105620] Updated weights for policy 1, policy_version 1700743 (0.0008) [2023-12-27 03:45:28,045][105692] Updated weights for policy 0, policy_version 1696906 (0.0008) [2023-12-27 03:45:28,097][105692] Updated weights for policy 0, policy_version 1696916 (0.0008) [2023-12-27 03:45:28,158][105692] Updated weights for policy 0, policy_version 1696926 (0.0009) [2023-12-27 03:45:28,690][105620] Updated weights for policy 1, policy_version 1700753 (0.0005) [2023-12-27 03:45:28,747][105620] Updated weights for policy 1, policy_version 1700763 (0.0007) [2023-12-27 03:45:28,809][105620] Updated weights for policy 1, policy_version 1700773 (0.0007) [2023-12-27 03:45:28,862][105620] Updated weights for policy 1, policy_version 1700783 (0.0005) [2023-12-27 03:45:28,934][105692] Updated weights for policy 0, policy_version 1696936 (0.0008) [2023-12-27 03:45:28,993][105692] Updated weights for policy 0, policy_version 1696946 (0.0008) [2023-12-27 03:45:29,052][105692] Updated weights for policy 0, policy_version 1696956 (0.0008) [2023-12-27 03:45:29,574][105620] Updated weights for policy 1, policy_version 1700793 (0.0010) [2023-12-27 03:45:29,637][105620] Updated weights for policy 1, policy_version 1700803 (0.0007) [2023-12-27 03:45:29,707][105620] Updated weights for policy 1, policy_version 1700813 (0.0005) [2023-12-27 03:45:29,827][105692] Updated weights for policy 0, policy_version 1696966 (0.0007) [2023-12-27 03:45:29,887][105692] Updated weights for policy 0, policy_version 1696976 (0.0009) [2023-12-27 03:45:29,950][105692] Updated weights for policy 0, policy_version 1696986 (0.0009) [2023-12-27 03:45:30,368][105620] Updated weights for policy 1, policy_version 1700823 (0.0008) [2023-12-27 03:45:30,419][105620] Updated weights for policy 1, policy_version 1700833 (0.0009) [2023-12-27 03:45:30,474][105620] Updated weights for policy 1, policy_version 1700843 (0.0009) [2023-12-27 03:45:30,696][105692] Updated weights for policy 0, policy_version 1696996 (0.0008) [2023-12-27 03:45:30,742][105692] Updated weights for policy 0, policy_version 1697006 (0.0009) [2023-12-27 03:45:30,800][105692] Updated weights for policy 0, policy_version 1697016 (0.0009) [2023-12-27 03:45:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 869982208. Throughput: 0: 9427.4, 1: 10119.2. Samples: 869952800. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:45:31,063][104569] Avg episode reward: [(0, '8628.120'), (1, '9262.993')] [2023-12-27 03:45:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001697024_434503680.pth... [2023-12-27 03:45:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001700848_435478528.pth... [2023-12-27 03:45:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001699664_435175424.pth [2023-12-27 03:45:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001695968_434233344.pth [2023-12-27 03:45:31,271][105620] Updated weights for policy 1, policy_version 1700853 (0.0009) [2023-12-27 03:45:31,337][105620] Updated weights for policy 1, policy_version 1700863 (0.0009) [2023-12-27 03:45:31,394][105620] Updated weights for policy 1, policy_version 1700873 (0.0009) [2023-12-27 03:45:31,549][105692] Updated weights for policy 0, policy_version 1697026 (0.0008) [2023-12-27 03:45:31,613][105692] Updated weights for policy 0, policy_version 1697036 (0.0009) [2023-12-27 03:45:31,672][105692] Updated weights for policy 0, policy_version 1697046 (0.0008) [2023-12-27 03:45:31,744][105692] Updated weights for policy 0, policy_version 1697056 (0.0008) [2023-12-27 03:45:32,102][105620] Updated weights for policy 1, policy_version 1700883 (0.0009) [2023-12-27 03:45:32,154][105620] Updated weights for policy 1, policy_version 1700893 (0.0009) [2023-12-27 03:45:32,212][105620] Updated weights for policy 1, policy_version 1700903 (0.0009) [2023-12-27 03:45:32,447][105692] Updated weights for policy 0, policy_version 1697066 (0.0009) [2023-12-27 03:45:32,503][105692] Updated weights for policy 0, policy_version 1697076 (0.0009) [2023-12-27 03:45:32,566][105692] Updated weights for policy 0, policy_version 1697086 (0.0009) [2023-12-27 03:45:32,880][105620] Updated weights for policy 1, policy_version 1700913 (0.0008) [2023-12-27 03:45:32,932][105620] Updated weights for policy 1, policy_version 1700923 (0.0005) [2023-12-27 03:45:32,987][105620] Updated weights for policy 1, policy_version 1700933 (0.0008) [2023-12-27 03:45:33,034][105620] Updated weights for policy 1, policy_version 1700943 (0.0008) [2023-12-27 03:45:33,207][105692] Updated weights for policy 0, policy_version 1697096 (0.0010) [2023-12-27 03:45:33,262][105692] Updated weights for policy 0, policy_version 1697106 (0.0010) [2023-12-27 03:45:33,310][105692] Updated weights for policy 0, policy_version 1697116 (0.0010) [2023-12-27 03:45:33,631][105620] Updated weights for policy 1, policy_version 1700953 (0.0005) [2023-12-27 03:45:33,697][105620] Updated weights for policy 1, policy_version 1700963 (0.0005) [2023-12-27 03:45:33,764][105620] Updated weights for policy 1, policy_version 1700973 (0.0005) [2023-12-27 03:45:34,060][105692] Updated weights for policy 0, policy_version 1697126 (0.0010) [2023-12-27 03:45:34,115][105692] Updated weights for policy 0, policy_version 1697136 (0.0010) [2023-12-27 03:45:34,183][105692] Updated weights for policy 0, policy_version 1697146 (0.0011) [2023-12-27 03:45:34,410][105620] Updated weights for policy 1, policy_version 1700983 (0.0007) [2023-12-27 03:45:34,477][105620] Updated weights for policy 1, policy_version 1700993 (0.0008) [2023-12-27 03:45:34,538][105620] Updated weights for policy 1, policy_version 1701003 (0.0009) [2023-12-27 03:45:34,909][105692] Updated weights for policy 0, policy_version 1697156 (0.0008) [2023-12-27 03:45:34,957][105692] Updated weights for policy 0, policy_version 1697166 (0.0011) [2023-12-27 03:45:35,010][105692] Updated weights for policy 0, policy_version 1697176 (0.0011) [2023-12-27 03:45:35,292][105620] Updated weights for policy 1, policy_version 1701013 (0.0009) [2023-12-27 03:45:35,351][105620] Updated weights for policy 1, policy_version 1701023 (0.0008) [2023-12-27 03:45:35,400][105620] Updated weights for policy 1, policy_version 1701033 (0.0008) [2023-12-27 03:45:35,751][105692] Updated weights for policy 0, policy_version 1697186 (0.0011) [2023-12-27 03:45:35,798][105692] Updated weights for policy 0, policy_version 1697196 (0.0010) [2023-12-27 03:45:35,850][105692] Updated weights for policy 0, policy_version 1697206 (0.0010) [2023-12-27 03:45:35,907][105692] Updated weights for policy 0, policy_version 1697216 (0.0010) [2023-12-27 03:45:36,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 870080512. Throughput: 0: 9341.0, 1: 10066.7. Samples: 870069832. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:45:36,063][104569] Avg episode reward: [(0, '8441.446'), (1, '8985.577')] [2023-12-27 03:45:36,167][105620] Updated weights for policy 1, policy_version 1701043 (0.0008) [2023-12-27 03:45:36,228][105620] Updated weights for policy 1, policy_version 1701053 (0.0009) [2023-12-27 03:45:36,282][105620] Updated weights for policy 1, policy_version 1701063 (0.0008) [2023-12-27 03:45:36,690][105692] Updated weights for policy 0, policy_version 1697226 (0.0011) [2023-12-27 03:45:36,752][105692] Updated weights for policy 0, policy_version 1697236 (0.0011) [2023-12-27 03:45:36,814][105692] Updated weights for policy 0, policy_version 1697246 (0.0010) [2023-12-27 03:45:37,022][105620] Updated weights for policy 1, policy_version 1701073 (0.0006) [2023-12-27 03:45:37,082][105620] Updated weights for policy 1, policy_version 1701083 (0.0008) [2023-12-27 03:45:37,143][105620] Updated weights for policy 1, policy_version 1701093 (0.0008) [2023-12-27 03:45:37,196][105620] Updated weights for policy 1, policy_version 1701103 (0.0008) [2023-12-27 03:45:37,559][105692] Updated weights for policy 0, policy_version 1697256 (0.0010) [2023-12-27 03:45:37,614][105692] Updated weights for policy 0, policy_version 1697266 (0.0010) [2023-12-27 03:45:37,676][105692] Updated weights for policy 0, policy_version 1697276 (0.0011) [2023-12-27 03:45:37,965][105620] Updated weights for policy 1, policy_version 1701113 (0.0009) [2023-12-27 03:45:38,012][105620] Updated weights for policy 1, policy_version 1701123 (0.0009) [2023-12-27 03:45:38,059][105620] Updated weights for policy 1, policy_version 1701133 (0.0009) [2023-12-27 03:45:38,344][105692] Updated weights for policy 0, policy_version 1697286 (0.0010) [2023-12-27 03:45:38,407][105692] Updated weights for policy 0, policy_version 1697296 (0.0009) [2023-12-27 03:45:38,463][105692] Updated weights for policy 0, policy_version 1697306 (0.0008) [2023-12-27 03:45:38,939][105620] Updated weights for policy 1, policy_version 1701143 (0.0009) [2023-12-27 03:45:39,001][105620] Updated weights for policy 1, policy_version 1701153 (0.0009) [2023-12-27 03:45:39,051][105692] Updated weights for policy 0, policy_version 1697316 (0.0009) [2023-12-27 03:45:39,061][105620] Updated weights for policy 1, policy_version 1701163 (0.0007) [2023-12-27 03:45:39,099][105692] Updated weights for policy 0, policy_version 1697326 (0.0007) [2023-12-27 03:45:39,145][105692] Updated weights for policy 0, policy_version 1697336 (0.0008) [2023-12-27 03:45:39,816][105620] Updated weights for policy 1, policy_version 1701173 (0.0006) [2023-12-27 03:45:39,882][105620] Updated weights for policy 1, policy_version 1701183 (0.0008) [2023-12-27 03:45:39,948][105620] Updated weights for policy 1, policy_version 1701193 (0.0008) [2023-12-27 03:45:39,973][105692] Updated weights for policy 0, policy_version 1697346 (0.0008) [2023-12-27 03:45:40,032][105692] Updated weights for policy 0, policy_version 1697356 (0.0006) [2023-12-27 03:45:40,085][105692] Updated weights for policy 0, policy_version 1697366 (0.0006) [2023-12-27 03:45:40,152][105692] Updated weights for policy 0, policy_version 1697376 (0.0009) [2023-12-27 03:45:40,732][105620] Updated weights for policy 1, policy_version 1701203 (0.0008) [2023-12-27 03:45:40,796][105620] Updated weights for policy 1, policy_version 1701213 (0.0007) [2023-12-27 03:45:40,810][105692] Updated weights for policy 0, policy_version 1697386 (0.0008) [2023-12-27 03:45:40,862][105620] Updated weights for policy 1, policy_version 1701223 (0.0008) [2023-12-27 03:45:40,877][105692] Updated weights for policy 0, policy_version 1697396 (0.0006) [2023-12-27 03:45:40,944][105692] Updated weights for policy 0, policy_version 1697406 (0.0005) [2023-12-27 03:45:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 870178816. Throughput: 0: 9312.7, 1: 9930.6. Samples: 870182300. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:45:41,062][104569] Avg episode reward: [(0, '8532.060'), (1, '8985.746')] [2023-12-27 03:45:41,657][105692] Updated weights for policy 0, policy_version 1697417 (0.0009) [2023-12-27 03:45:41,668][105620] Updated weights for policy 1, policy_version 1701233 (0.0008) [2023-12-27 03:45:41,722][105692] Updated weights for policy 0, policy_version 1697427 (0.0007) [2023-12-27 03:45:41,729][105620] Updated weights for policy 1, policy_version 1701243 (0.0009) [2023-12-27 03:45:41,787][105692] Updated weights for policy 0, policy_version 1697437 (0.0010) [2023-12-27 03:45:41,795][105620] Updated weights for policy 1, policy_version 1701253 (0.0008) [2023-12-27 03:45:41,848][105620] Updated weights for policy 1, policy_version 1701263 (0.0008) [2023-12-27 03:45:42,487][105620] Updated weights for policy 1, policy_version 1701273 (0.0010) [2023-12-27 03:45:42,495][105692] Updated weights for policy 0, policy_version 1697447 (0.0006) [2023-12-27 03:45:42,547][105620] Updated weights for policy 1, policy_version 1701283 (0.0008) [2023-12-27 03:45:42,551][105692] Updated weights for policy 0, policy_version 1697457 (0.0006) [2023-12-27 03:45:42,603][105620] Updated weights for policy 1, policy_version 1701293 (0.0007) [2023-12-27 03:45:42,616][105692] Updated weights for policy 0, policy_version 1697467 (0.0007) [2023-12-27 03:45:43,325][105620] Updated weights for policy 1, policy_version 1701303 (0.0007) [2023-12-27 03:45:43,355][105692] Updated weights for policy 0, policy_version 1697477 (0.0007) [2023-12-27 03:45:43,385][105620] Updated weights for policy 1, policy_version 1701313 (0.0007) [2023-12-27 03:45:43,411][105692] Updated weights for policy 0, policy_version 1697487 (0.0006) [2023-12-27 03:45:43,444][105620] Updated weights for policy 1, policy_version 1701323 (0.0009) [2023-12-27 03:45:43,469][105692] Updated weights for policy 0, policy_version 1697497 (0.0006) [2023-12-27 03:45:44,108][105692] Updated weights for policy 0, policy_version 1697507 (0.0005) [2023-12-27 03:45:44,163][105692] Updated weights for policy 0, policy_version 1697517 (0.0005) [2023-12-27 03:45:44,174][105620] Updated weights for policy 1, policy_version 1701333 (0.0008) [2023-12-27 03:45:44,213][105692] Updated weights for policy 0, policy_version 1697527 (0.0006) [2023-12-27 03:45:44,226][105620] Updated weights for policy 1, policy_version 1701343 (0.0008) [2023-12-27 03:45:44,277][105620] Updated weights for policy 1, policy_version 1701353 (0.0008) [2023-12-27 03:45:44,881][105692] Updated weights for policy 0, policy_version 1697537 (0.0007) [2023-12-27 03:45:44,934][105692] Updated weights for policy 0, policy_version 1697547 (0.0011) [2023-12-27 03:45:44,986][105692] Updated weights for policy 0, policy_version 1697557 (0.0011) [2023-12-27 03:45:45,012][105620] Updated weights for policy 1, policy_version 1701363 (0.0008) [2023-12-27 03:45:45,040][105692] Updated weights for policy 0, policy_version 1697567 (0.0011) [2023-12-27 03:45:45,076][105620] Updated weights for policy 1, policy_version 1701373 (0.0005) [2023-12-27 03:45:45,135][105620] Updated weights for policy 1, policy_version 1701383 (0.0005) [2023-12-27 03:45:45,767][105620] Updated weights for policy 1, policy_version 1701393 (0.0006) [2023-12-27 03:45:45,819][105620] Updated weights for policy 1, policy_version 1701403 (0.0006) [2023-12-27 03:45:45,833][105692] Updated weights for policy 0, policy_version 1697577 (0.0010) [2023-12-27 03:45:45,871][105620] Updated weights for policy 1, policy_version 1701413 (0.0005) [2023-12-27 03:45:45,889][105692] Updated weights for policy 0, policy_version 1697587 (0.0010) [2023-12-27 03:45:45,926][105620] Updated weights for policy 1, policy_version 1701423 (0.0005) [2023-12-27 03:45:45,951][105692] Updated weights for policy 0, policy_version 1697597 (0.0010) [2023-12-27 03:45:46,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 870277120. Throughput: 0: 9248.7, 1: 9912.6. Samples: 870240576. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:45:46,062][104569] Avg episode reward: [(0, '8443.815'), (1, '9170.747')] [2023-12-27 03:45:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001697600_434651136.pth... [2023-12-27 03:45:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001701424_435625984.pth... [2023-12-27 03:45:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001700272_435331072.pth [2023-12-27 03:45:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001696480_434364416.pth [2023-12-27 03:45:46,675][105692] Updated weights for policy 0, policy_version 1697607 (0.0007) [2023-12-27 03:45:46,684][105620] Updated weights for policy 1, policy_version 1701433 (0.0008) [2023-12-27 03:45:46,731][105692] Updated weights for policy 0, policy_version 1697617 (0.0005) [2023-12-27 03:45:46,731][105620] Updated weights for policy 1, policy_version 1701443 (0.0007) [2023-12-27 03:45:46,779][105620] Updated weights for policy 1, policy_version 1701453 (0.0007) [2023-12-27 03:45:46,812][105692] Updated weights for policy 0, policy_version 1697627 (0.0007) [2023-12-27 03:45:47,459][105692] Updated weights for policy 0, policy_version 1697637 (0.0011) [2023-12-27 03:45:47,522][105692] Updated weights for policy 0, policy_version 1697647 (0.0010) [2023-12-27 03:45:47,532][105620] Updated weights for policy 1, policy_version 1701463 (0.0006) [2023-12-27 03:45:47,585][105692] Updated weights for policy 0, policy_version 1697657 (0.0010) [2023-12-27 03:45:47,592][105620] Updated weights for policy 1, policy_version 1701473 (0.0005) [2023-12-27 03:45:47,645][105620] Updated weights for policy 1, policy_version 1701483 (0.0006) [2023-12-27 03:45:48,294][105692] Updated weights for policy 0, policy_version 1697667 (0.0009) [2023-12-27 03:45:48,353][105692] Updated weights for policy 0, policy_version 1697677 (0.0008) [2023-12-27 03:45:48,387][105620] Updated weights for policy 1, policy_version 1701493 (0.0009) [2023-12-27 03:45:48,411][105692] Updated weights for policy 0, policy_version 1697687 (0.0009) [2023-12-27 03:45:48,439][105620] Updated weights for policy 1, policy_version 1701503 (0.0010) [2023-12-27 03:45:48,497][105620] Updated weights for policy 1, policy_version 1701513 (0.0009) [2023-12-27 03:45:49,186][105620] Updated weights for policy 1, policy_version 1701523 (0.0010) [2023-12-27 03:45:49,215][105692] Updated weights for policy 0, policy_version 1697697 (0.0008) [2023-12-27 03:45:49,246][105620] Updated weights for policy 1, policy_version 1701533 (0.0012) [2023-12-27 03:45:49,273][105692] Updated weights for policy 0, policy_version 1697707 (0.0007) [2023-12-27 03:45:49,306][105620] Updated weights for policy 1, policy_version 1701543 (0.0011) [2023-12-27 03:45:49,339][105692] Updated weights for policy 0, policy_version 1697717 (0.0009) [2023-12-27 03:45:49,400][105692] Updated weights for policy 0, policy_version 1697727 (0.0010) [2023-12-27 03:45:50,000][105620] Updated weights for policy 1, policy_version 1701553 (0.0010) [2023-12-27 03:45:50,062][105620] Updated weights for policy 1, policy_version 1701563 (0.0006) [2023-12-27 03:45:50,114][105620] Updated weights for policy 1, policy_version 1701573 (0.0008) [2023-12-27 03:45:50,167][105620] Updated weights for policy 1, policy_version 1701583 (0.0006) [2023-12-27 03:45:50,169][105692] Updated weights for policy 0, policy_version 1697737 (0.0010) [2023-12-27 03:45:50,231][105692] Updated weights for policy 0, policy_version 1697747 (0.0010) [2023-12-27 03:45:50,290][105692] Updated weights for policy 0, policy_version 1697757 (0.0009) [2023-12-27 03:45:50,909][105620] Updated weights for policy 1, policy_version 1701593 (0.0008) [2023-12-27 03:45:50,914][105692] Updated weights for policy 0, policy_version 1697767 (0.0007) [2023-12-27 03:45:50,968][105620] Updated weights for policy 1, policy_version 1701603 (0.0006) [2023-12-27 03:45:50,974][105692] Updated weights for policy 0, policy_version 1697777 (0.0008) [2023-12-27 03:45:51,026][105620] Updated weights for policy 1, policy_version 1701613 (0.0007) [2023-12-27 03:45:51,042][105692] Updated weights for policy 0, policy_version 1697787 (0.0006) [2023-12-27 03:45:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 870367232. Throughput: 0: 9291.6, 1: 9918.9. Samples: 870357216. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:45:51,063][104569] Avg episode reward: [(0, '8715.584'), (1, '8986.783')] [2023-12-27 03:45:51,748][105620] Updated weights for policy 1, policy_version 1701623 (0.0008) [2023-12-27 03:45:51,787][105692] Updated weights for policy 0, policy_version 1697797 (0.0007) [2023-12-27 03:45:51,812][105620] Updated weights for policy 1, policy_version 1701633 (0.0006) [2023-12-27 03:45:51,851][105692] Updated weights for policy 0, policy_version 1697807 (0.0008) [2023-12-27 03:45:51,864][105620] Updated weights for policy 1, policy_version 1701643 (0.0005) [2023-12-27 03:45:51,909][105692] Updated weights for policy 0, policy_version 1697817 (0.0009) [2023-12-27 03:45:52,619][105692] Updated weights for policy 0, policy_version 1697827 (0.0007) [2023-12-27 03:45:52,624][105620] Updated weights for policy 1, policy_version 1701653 (0.0008) [2023-12-27 03:45:52,678][105620] Updated weights for policy 1, policy_version 1701663 (0.0007) [2023-12-27 03:45:52,678][105692] Updated weights for policy 0, policy_version 1697837 (0.0007) [2023-12-27 03:45:52,732][105692] Updated weights for policy 0, policy_version 1697847 (0.0008) [2023-12-27 03:45:52,737][105620] Updated weights for policy 1, policy_version 1701673 (0.0007) [2023-12-27 03:45:53,309][105620] Updated weights for policy 1, policy_version 1701683 (0.0006) [2023-12-27 03:45:53,380][105692] Updated weights for policy 0, policy_version 1697857 (0.0006) [2023-12-27 03:45:53,381][105620] Updated weights for policy 1, policy_version 1701693 (0.0005) [2023-12-27 03:45:53,443][105692] Updated weights for policy 0, policy_version 1697867 (0.0006) [2023-12-27 03:45:53,451][105620] Updated weights for policy 1, policy_version 1701703 (0.0005) [2023-12-27 03:45:53,494][105692] Updated weights for policy 0, policy_version 1697877 (0.0006) [2023-12-27 03:45:53,557][105692] Updated weights for policy 0, policy_version 1697887 (0.0005) [2023-12-27 03:45:53,965][105620] Updated weights for policy 1, policy_version 1701713 (0.0007) [2023-12-27 03:45:54,024][105620] Updated weights for policy 1, policy_version 1701723 (0.0010) [2023-12-27 03:45:54,076][105620] Updated weights for policy 1, policy_version 1701733 (0.0010) [2023-12-27 03:45:54,129][105692] Updated weights for policy 0, policy_version 1697897 (0.0008) [2023-12-27 03:45:54,139][105620] Updated weights for policy 1, policy_version 1701743 (0.0011) [2023-12-27 03:45:54,186][105692] Updated weights for policy 0, policy_version 1697907 (0.0009) [2023-12-27 03:45:54,240][105692] Updated weights for policy 0, policy_version 1697917 (0.0010) [2023-12-27 03:45:54,874][105620] Updated weights for policy 1, policy_version 1701753 (0.0007) [2023-12-27 03:45:54,910][105692] Updated weights for policy 0, policy_version 1697927 (0.0007) [2023-12-27 03:45:54,925][105620] Updated weights for policy 1, policy_version 1701763 (0.0005) [2023-12-27 03:45:54,977][105692] Updated weights for policy 0, policy_version 1697937 (0.0007) [2023-12-27 03:45:54,994][105620] Updated weights for policy 1, policy_version 1701773 (0.0008) [2023-12-27 03:45:55,032][105692] Updated weights for policy 0, policy_version 1697947 (0.0009) [2023-12-27 03:45:55,582][105620] Updated weights for policy 1, policy_version 1701783 (0.0007) [2023-12-27 03:45:55,654][105620] Updated weights for policy 1, policy_version 1701793 (0.0007) [2023-12-27 03:45:55,687][105692] Updated weights for policy 0, policy_version 1697957 (0.0007) [2023-12-27 03:45:55,707][105620] Updated weights for policy 1, policy_version 1701803 (0.0009) [2023-12-27 03:45:55,739][105692] Updated weights for policy 0, policy_version 1697967 (0.0005) [2023-12-27 03:45:55,786][105692] Updated weights for policy 0, policy_version 1697977 (0.0007) [2023-12-27 03:45:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 870473728. Throughput: 0: 9509.0, 1: 9875.3. Samples: 870480420. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:45:56,063][104569] Avg episode reward: [(0, '9169.497'), (1, '8809.032')] [2023-12-27 03:45:56,457][105620] Updated weights for policy 1, policy_version 1701813 (0.0008) [2023-12-27 03:45:56,500][105692] Updated weights for policy 0, policy_version 1697987 (0.0009) [2023-12-27 03:45:56,506][105620] Updated weights for policy 1, policy_version 1701823 (0.0008) [2023-12-27 03:45:56,552][105620] Updated weights for policy 1, policy_version 1701833 (0.0007) [2023-12-27 03:45:56,558][105692] Updated weights for policy 0, policy_version 1697997 (0.0010) [2023-12-27 03:45:56,610][105692] Updated weights for policy 0, policy_version 1698007 (0.0010) [2023-12-27 03:45:57,327][105620] Updated weights for policy 1, policy_version 1701843 (0.0006) [2023-12-27 03:45:57,365][105692] Updated weights for policy 0, policy_version 1698018 (0.0010) [2023-12-27 03:45:57,392][105620] Updated weights for policy 1, policy_version 1701853 (0.0007) [2023-12-27 03:45:57,418][105692] Updated weights for policy 0, policy_version 1698028 (0.0006) [2023-12-27 03:45:57,448][105620] Updated weights for policy 1, policy_version 1701863 (0.0007) [2023-12-27 03:45:57,464][105692] Updated weights for policy 0, policy_version 1698038 (0.0005) [2023-12-27 03:45:57,515][105692] Updated weights for policy 0, policy_version 1698048 (0.0005) [2023-12-27 03:45:58,060][105620] Updated weights for policy 1, policy_version 1701873 (0.0008) [2023-12-27 03:45:58,113][105620] Updated weights for policy 1, policy_version 1701883 (0.0006) [2023-12-27 03:45:58,173][105620] Updated weights for policy 1, policy_version 1701893 (0.0007) [2023-12-27 03:45:58,235][105620] Updated weights for policy 1, policy_version 1701903 (0.0008) [2023-12-27 03:45:58,240][105692] Updated weights for policy 0, policy_version 1698058 (0.0008) [2023-12-27 03:45:58,296][105692] Updated weights for policy 0, policy_version 1698068 (0.0006) [2023-12-27 03:45:58,361][105692] Updated weights for policy 0, policy_version 1698078 (0.0008) [2023-12-27 03:45:58,977][105620] Updated weights for policy 1, policy_version 1701913 (0.0008) [2023-12-27 03:45:59,042][105620] Updated weights for policy 1, policy_version 1701923 (0.0008) [2023-12-27 03:45:59,097][105620] Updated weights for policy 1, policy_version 1701933 (0.0007) [2023-12-27 03:45:59,136][105692] Updated weights for policy 0, policy_version 1698088 (0.0007) [2023-12-27 03:45:59,192][105692] Updated weights for policy 0, policy_version 1698098 (0.0007) [2023-12-27 03:45:59,260][105692] Updated weights for policy 0, policy_version 1698108 (0.0009) [2023-12-27 03:45:59,902][105620] Updated weights for policy 1, policy_version 1701943 (0.0008) [2023-12-27 03:45:59,967][105620] Updated weights for policy 1, policy_version 1701953 (0.0010) [2023-12-27 03:45:59,989][105692] Updated weights for policy 0, policy_version 1698118 (0.0008) [2023-12-27 03:46:00,020][105620] Updated weights for policy 1, policy_version 1701963 (0.0006) [2023-12-27 03:46:00,046][105692] Updated weights for policy 0, policy_version 1698128 (0.0007) [2023-12-27 03:46:00,107][105692] Updated weights for policy 0, policy_version 1698138 (0.0009) [2023-12-27 03:46:00,786][105692] Updated weights for policy 0, policy_version 1698148 (0.0010) [2023-12-27 03:46:00,809][105620] Updated weights for policy 1, policy_version 1701973 (0.0007) [2023-12-27 03:46:00,840][105692] Updated weights for policy 0, policy_version 1698158 (0.0010) [2023-12-27 03:46:00,858][105620] Updated weights for policy 1, policy_version 1701983 (0.0006) [2023-12-27 03:46:00,891][105692] Updated weights for policy 0, policy_version 1698168 (0.0010) [2023-12-27 03:46:00,910][105620] Updated weights for policy 1, policy_version 1701993 (0.0010) [2023-12-27 03:46:01,062][104569] Fps is (10 sec: 20479.2, 60 sec: 19387.6, 300 sec: 19549.7). Total num frames: 870572032. Throughput: 0: 9523.0, 1: 9887.5. Samples: 870538652. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:46:01,063][104569] Avg episode reward: [(0, '8620.443'), (1, '8810.564')] [2023-12-27 03:46:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001698176_434798592.pth... [2023-12-27 03:46:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001702000_435773440.pth... [2023-12-27 03:46:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001697024_434503680.pth [2023-12-27 03:46:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001700848_435478528.pth [2023-12-27 03:46:01,654][105620] Updated weights for policy 1, policy_version 1702003 (0.0009) [2023-12-27 03:46:01,673][105692] Updated weights for policy 0, policy_version 1698178 (0.0010) [2023-12-27 03:46:01,714][105620] Updated weights for policy 1, policy_version 1702013 (0.0007) [2023-12-27 03:46:01,731][105692] Updated weights for policy 0, policy_version 1698188 (0.0010) [2023-12-27 03:46:01,775][105620] Updated weights for policy 1, policy_version 1702023 (0.0007) [2023-12-27 03:46:01,784][105692] Updated weights for policy 0, policy_version 1698198 (0.0008) [2023-12-27 03:46:01,838][105692] Updated weights for policy 0, policy_version 1698208 (0.0007) [2023-12-27 03:46:02,499][105620] Updated weights for policy 1, policy_version 1702033 (0.0008) [2023-12-27 03:46:02,565][105620] Updated weights for policy 1, policy_version 1702043 (0.0009) [2023-12-27 03:46:02,580][105586] KL-divergence is very high: 108.1433 [2023-12-27 03:46:02,611][105692] Updated weights for policy 0, policy_version 1698218 (0.0010) [2023-12-27 03:46:02,626][105620] Updated weights for policy 1, policy_version 1702053 (0.0009) [2023-12-27 03:46:02,631][105586] KL-divergence is very high: 215.1534 [2023-12-27 03:46:02,661][105692] Updated weights for policy 0, policy_version 1698228 (0.0005) [2023-12-27 03:46:02,680][105586] KL-divergence is very high: 252.5798 [2023-12-27 03:46:02,687][105620] Updated weights for policy 1, policy_version 1702063 (0.0009) [2023-12-27 03:46:02,715][105692] Updated weights for policy 0, policy_version 1698238 (0.0005) [2023-12-27 03:46:03,338][105620] Updated weights for policy 1, policy_version 1702073 (0.0008) [2023-12-27 03:46:03,396][105620] Updated weights for policy 1, policy_version 1702083 (0.0007) [2023-12-27 03:46:03,414][105692] Updated weights for policy 0, policy_version 1698248 (0.0010) [2023-12-27 03:46:03,449][105620] Updated weights for policy 1, policy_version 1702093 (0.0005) [2023-12-27 03:46:03,466][105692] Updated weights for policy 0, policy_version 1698258 (0.0010) [2023-12-27 03:46:03,518][105692] Updated weights for policy 0, policy_version 1698268 (0.0010) [2023-12-27 03:46:04,155][105620] Updated weights for policy 1, policy_version 1702103 (0.0009) [2023-12-27 03:46:04,214][105620] Updated weights for policy 1, policy_version 1702113 (0.0008) [2023-12-27 03:46:04,279][105620] Updated weights for policy 1, policy_version 1702123 (0.0009) [2023-12-27 03:46:04,280][105692] Updated weights for policy 0, policy_version 1698278 (0.0011) [2023-12-27 03:46:04,347][105692] Updated weights for policy 0, policy_version 1698288 (0.0011) [2023-12-27 03:46:04,410][105692] Updated weights for policy 0, policy_version 1698298 (0.0011) [2023-12-27 03:46:05,044][105620] Updated weights for policy 1, policy_version 1702133 (0.0006) [2023-12-27 03:46:05,092][105620] Updated weights for policy 1, policy_version 1702143 (0.0007) [2023-12-27 03:46:05,144][105692] Updated weights for policy 0, policy_version 1698308 (0.0009) [2023-12-27 03:46:05,146][105620] Updated weights for policy 1, policy_version 1702153 (0.0009) [2023-12-27 03:46:05,200][105692] Updated weights for policy 0, policy_version 1698318 (0.0006) [2023-12-27 03:46:05,256][105692] Updated weights for policy 0, policy_version 1698328 (0.0007) [2023-12-27 03:46:05,845][105620] Updated weights for policy 1, policy_version 1702163 (0.0008) [2023-12-27 03:46:05,899][105620] Updated weights for policy 1, policy_version 1702173 (0.0005) [2023-12-27 03:46:05,958][105620] Updated weights for policy 1, policy_version 1702183 (0.0006) [2023-12-27 03:46:05,964][105692] Updated weights for policy 0, policy_version 1698338 (0.0009) [2023-12-27 03:46:06,013][105692] Updated weights for policy 0, policy_version 1698348 (0.0006) [2023-12-27 03:46:06,060][105692] Updated weights for policy 0, policy_version 1698358 (0.0008) [2023-12-27 03:46:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 870662144. Throughput: 0: 9598.6, 1: 9716.6. Samples: 870651312. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:46:06,062][104569] Avg episode reward: [(0, '8620.688'), (1, '8804.466')] [2023-12-27 03:46:06,113][105692] Updated weights for policy 0, policy_version 1698368 (0.0007) [2023-12-27 03:46:06,717][105620] Updated weights for policy 1, policy_version 1702193 (0.0008) [2023-12-27 03:46:06,781][105620] Updated weights for policy 1, policy_version 1702203 (0.0008) [2023-12-27 03:46:06,836][105620] Updated weights for policy 1, policy_version 1702213 (0.0009) [2023-12-27 03:46:06,901][105620] Updated weights for policy 1, policy_version 1702223 (0.0009) [2023-12-27 03:46:06,943][105692] Updated weights for policy 0, policy_version 1698378 (0.0008) [2023-12-27 03:46:07,006][105692] Updated weights for policy 0, policy_version 1698388 (0.0008) [2023-12-27 03:46:07,066][105692] Updated weights for policy 0, policy_version 1698398 (0.0010) [2023-12-27 03:46:07,634][105620] Updated weights for policy 1, policy_version 1702233 (0.0009) [2023-12-27 03:46:07,706][105620] Updated weights for policy 1, policy_version 1702243 (0.0009) [2023-12-27 03:46:07,770][105620] Updated weights for policy 1, policy_version 1702253 (0.0008) [2023-12-27 03:46:07,861][105692] Updated weights for policy 0, policy_version 1698408 (0.0010) [2023-12-27 03:46:07,924][105692] Updated weights for policy 0, policy_version 1698418 (0.0010) [2023-12-27 03:46:07,989][105692] Updated weights for policy 0, policy_version 1698428 (0.0008) [2023-12-27 03:46:08,573][105620] Updated weights for policy 1, policy_version 1702263 (0.0009) [2023-12-27 03:46:08,636][105620] Updated weights for policy 1, policy_version 1702273 (0.0008) [2023-12-27 03:46:08,699][105620] Updated weights for policy 1, policy_version 1702283 (0.0009) [2023-12-27 03:46:08,752][105692] Updated weights for policy 0, policy_version 1698438 (0.0008) [2023-12-27 03:46:08,806][105692] Updated weights for policy 0, policy_version 1698448 (0.0009) [2023-12-27 03:46:08,862][105692] Updated weights for policy 0, policy_version 1698458 (0.0009) [2023-12-27 03:46:09,532][105620] Updated weights for policy 1, policy_version 1702293 (0.0008) [2023-12-27 03:46:09,590][105692] Updated weights for policy 0, policy_version 1698468 (0.0010) [2023-12-27 03:46:09,599][105620] Updated weights for policy 1, policy_version 1702303 (0.0008) [2023-12-27 03:46:09,655][105692] Updated weights for policy 0, policy_version 1698478 (0.0011) [2023-12-27 03:46:09,663][105620] Updated weights for policy 1, policy_version 1702313 (0.0008) [2023-12-27 03:46:09,713][105692] Updated weights for policy 0, policy_version 1698488 (0.0011) [2023-12-27 03:46:10,364][105620] Updated weights for policy 1, policy_version 1702323 (0.0009) [2023-12-27 03:46:10,428][105620] Updated weights for policy 1, policy_version 1702333 (0.0008) [2023-12-27 03:46:10,487][105620] Updated weights for policy 1, policy_version 1702343 (0.0008) [2023-12-27 03:46:10,488][105692] Updated weights for policy 0, policy_version 1698498 (0.0011) [2023-12-27 03:46:10,545][105692] Updated weights for policy 0, policy_version 1698508 (0.0011) [2023-12-27 03:46:10,611][105692] Updated weights for policy 0, policy_version 1698518 (0.0011) [2023-12-27 03:46:10,671][105692] Updated weights for policy 0, policy_version 1698528 (0.0011) [2023-12-27 03:46:11,062][104569] Fps is (10 sec: 18023.2, 60 sec: 19114.7, 300 sec: 19494.2). Total num frames: 870752256. Throughput: 0: 9652.7, 1: 9631.0. Samples: 870762044. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:46:11,062][104569] Avg episode reward: [(0, '8986.365'), (1, '8986.767')] [2023-12-27 03:46:11,329][105620] Updated weights for policy 1, policy_version 1702353 (0.0007) [2023-12-27 03:46:11,342][105692] Updated weights for policy 0, policy_version 1698538 (0.0011) [2023-12-27 03:46:11,396][105620] Updated weights for policy 1, policy_version 1702363 (0.0010) [2023-12-27 03:46:11,407][105692] Updated weights for policy 0, policy_version 1698548 (0.0010) [2023-12-27 03:46:11,454][105620] Updated weights for policy 1, policy_version 1702373 (0.0008) [2023-12-27 03:46:11,468][105692] Updated weights for policy 0, policy_version 1698558 (0.0011) [2023-12-27 03:46:11,505][105620] Updated weights for policy 1, policy_version 1702383 (0.0007) [2023-12-27 03:46:12,262][105620] Updated weights for policy 1, policy_version 1702393 (0.0008) [2023-12-27 03:46:12,329][105620] Updated weights for policy 1, policy_version 1702403 (0.0009) [2023-12-27 03:46:12,336][105692] Updated weights for policy 0, policy_version 1698568 (0.0008) [2023-12-27 03:46:12,397][105620] Updated weights for policy 1, policy_version 1702413 (0.0008) [2023-12-27 03:46:12,403][105692] Updated weights for policy 0, policy_version 1698578 (0.0009) [2023-12-27 03:46:12,463][105692] Updated weights for policy 0, policy_version 1698588 (0.0010) [2023-12-27 03:46:13,104][105692] Updated weights for policy 0, policy_version 1698598 (0.0006) [2023-12-27 03:46:13,156][105692] Updated weights for policy 0, policy_version 1698608 (0.0006) [2023-12-27 03:46:13,209][105692] Updated weights for policy 0, policy_version 1698618 (0.0006) [2023-12-27 03:46:13,278][105620] Updated weights for policy 1, policy_version 1702423 (0.0009) [2023-12-27 03:46:13,345][105620] Updated weights for policy 1, policy_version 1702433 (0.0009) [2023-12-27 03:46:13,407][105620] Updated weights for policy 1, policy_version 1702443 (0.0009) [2023-12-27 03:46:14,004][105692] Updated weights for policy 0, policy_version 1698628 (0.0007) [2023-12-27 03:46:14,024][105620] Updated weights for policy 1, policy_version 1702453 (0.0006) [2023-12-27 03:46:14,052][105692] Updated weights for policy 0, policy_version 1698638 (0.0009) [2023-12-27 03:46:14,074][105620] Updated weights for policy 1, policy_version 1702463 (0.0005) [2023-12-27 03:46:14,104][105692] Updated weights for policy 0, policy_version 1698648 (0.0008) [2023-12-27 03:46:14,128][105620] Updated weights for policy 1, policy_version 1702473 (0.0006) [2023-12-27 03:46:14,703][105620] Updated weights for policy 1, policy_version 1702483 (0.0007) [2023-12-27 03:46:14,751][105620] Updated weights for policy 1, policy_version 1702493 (0.0005) [2023-12-27 03:46:14,816][105620] Updated weights for policy 1, policy_version 1702503 (0.0007) [2023-12-27 03:46:14,898][105692] Updated weights for policy 0, policy_version 1698658 (0.0007) [2023-12-27 03:46:14,962][105692] Updated weights for policy 0, policy_version 1698668 (0.0010) [2023-12-27 03:46:15,029][105692] Updated weights for policy 0, policy_version 1698678 (0.0009) [2023-12-27 03:46:15,377][105620] Updated weights for policy 1, policy_version 1702513 (0.0008) [2023-12-27 03:46:15,445][105620] Updated weights for policy 1, policy_version 1702523 (0.0008) [2023-12-27 03:46:15,500][105620] Updated weights for policy 1, policy_version 1702533 (0.0009) [2023-12-27 03:46:15,558][105620] Updated weights for policy 1, policy_version 1702543 (0.0008) [2023-12-27 03:46:15,848][105692] Updated weights for policy 0, policy_version 1698689 (0.0010) [2023-12-27 03:46:15,906][105692] Updated weights for policy 0, policy_version 1698699 (0.0011) [2023-12-27 03:46:15,971][105692] Updated weights for policy 0, policy_version 1698709 (0.0011) [2023-12-27 03:46:16,030][105692] Updated weights for policy 0, policy_version 1698719 (0.0010) [2023-12-27 03:46:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 870850560. Throughput: 0: 9650.6, 1: 9580.6. Samples: 870818200. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:46:16,062][104569] Avg episode reward: [(0, '9261.494'), (1, '8894.052')] [2023-12-27 03:46:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001698720_434937856.pth... [2023-12-27 03:46:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001702544_435912704.pth... [2023-12-27 03:46:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001697600_434651136.pth [2023-12-27 03:46:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001701424_435625984.pth [2023-12-27 03:46:16,312][105620] Updated weights for policy 1, policy_version 1702553 (0.0009) [2023-12-27 03:46:16,378][105620] Updated weights for policy 1, policy_version 1702563 (0.0008) [2023-12-27 03:46:16,442][105620] Updated weights for policy 1, policy_version 1702573 (0.0008) [2023-12-27 03:46:16,767][105692] Updated weights for policy 0, policy_version 1698729 (0.0010) [2023-12-27 03:46:16,821][105692] Updated weights for policy 0, policy_version 1698739 (0.0010) [2023-12-27 03:46:16,875][105692] Updated weights for policy 0, policy_version 1698749 (0.0010) [2023-12-27 03:46:17,061][105620] Updated weights for policy 1, policy_version 1702583 (0.0010) [2023-12-27 03:46:17,119][105620] Updated weights for policy 1, policy_version 1702593 (0.0010) [2023-12-27 03:46:17,175][105620] Updated weights for policy 1, policy_version 1702603 (0.0010) [2023-12-27 03:46:17,615][105692] Updated weights for policy 0, policy_version 1698759 (0.0009) [2023-12-27 03:46:17,678][105692] Updated weights for policy 0, policy_version 1698769 (0.0011) [2023-12-27 03:46:17,744][105692] Updated weights for policy 0, policy_version 1698779 (0.0011) [2023-12-27 03:46:17,824][105620] Updated weights for policy 1, policy_version 1702613 (0.0009) [2023-12-27 03:46:17,885][105620] Updated weights for policy 1, policy_version 1702623 (0.0008) [2023-12-27 03:46:17,938][105620] Updated weights for policy 1, policy_version 1702633 (0.0008) [2023-12-27 03:46:18,493][105692] Updated weights for policy 0, policy_version 1698789 (0.0010) [2023-12-27 03:46:18,538][105692] Updated weights for policy 0, policy_version 1698799 (0.0010) [2023-12-27 03:46:18,590][105692] Updated weights for policy 0, policy_version 1698809 (0.0011) [2023-12-27 03:46:18,698][105620] Updated weights for policy 1, policy_version 1702643 (0.0007) [2023-12-27 03:46:18,754][105620] Updated weights for policy 1, policy_version 1702653 (0.0008) [2023-12-27 03:46:18,806][105620] Updated weights for policy 1, policy_version 1702663 (0.0008) [2023-12-27 03:46:19,357][105692] Updated weights for policy 0, policy_version 1698819 (0.0010) [2023-12-27 03:46:19,423][105692] Updated weights for policy 0, policy_version 1698829 (0.0006) [2023-12-27 03:46:19,486][105692] Updated weights for policy 0, policy_version 1698839 (0.0007) [2023-12-27 03:46:19,614][105620] Updated weights for policy 1, policy_version 1702673 (0.0008) [2023-12-27 03:46:19,671][105620] Updated weights for policy 1, policy_version 1702683 (0.0006) [2023-12-27 03:46:19,742][105620] Updated weights for policy 1, policy_version 1702693 (0.0006) [2023-12-27 03:46:19,807][105620] Updated weights for policy 1, policy_version 1702703 (0.0008) [2023-12-27 03:46:20,266][105692] Updated weights for policy 0, policy_version 1698849 (0.0008) [2023-12-27 03:46:20,336][105692] Updated weights for policy 0, policy_version 1698859 (0.0011) [2023-12-27 03:46:20,395][105620] Updated weights for policy 1, policy_version 1702713 (0.0006) [2023-12-27 03:46:20,397][105692] Updated weights for policy 0, policy_version 1698869 (0.0011) [2023-12-27 03:46:20,456][105620] Updated weights for policy 1, policy_version 1702723 (0.0005) [2023-12-27 03:46:20,457][105692] Updated weights for policy 0, policy_version 1698879 (0.0011) [2023-12-27 03:46:20,515][105620] Updated weights for policy 1, policy_version 1702733 (0.0008) [2023-12-27 03:46:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19114.6, 300 sec: 19494.2). Total num frames: 870940672. Throughput: 0: 9589.9, 1: 9613.7. Samples: 870933992. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:46:21,063][104569] Avg episode reward: [(0, '8620.813'), (1, '9078.852')] [2023-12-27 03:46:21,224][105692] Updated weights for policy 0, policy_version 1698889 (0.0009) [2023-12-27 03:46:21,270][105620] Updated weights for policy 1, policy_version 1702743 (0.0007) [2023-12-27 03:46:21,292][105692] Updated weights for policy 0, policy_version 1698899 (0.0008) [2023-12-27 03:46:21,332][105620] Updated weights for policy 1, policy_version 1702753 (0.0008) [2023-12-27 03:46:21,353][105692] Updated weights for policy 0, policy_version 1698909 (0.0006) [2023-12-27 03:46:21,398][105620] Updated weights for policy 1, policy_version 1702763 (0.0009) [2023-12-27 03:46:22,083][105692] Updated weights for policy 0, policy_version 1698919 (0.0009) [2023-12-27 03:46:22,143][105692] Updated weights for policy 0, policy_version 1698929 (0.0008) [2023-12-27 03:46:22,194][105620] Updated weights for policy 1, policy_version 1702773 (0.0007) [2023-12-27 03:46:22,209][105692] Updated weights for policy 0, policy_version 1698939 (0.0010) [2023-12-27 03:46:22,261][105620] Updated weights for policy 1, policy_version 1702783 (0.0008) [2023-12-27 03:46:22,324][105620] Updated weights for policy 1, policy_version 1702793 (0.0009) [2023-12-27 03:46:22,996][105692] Updated weights for policy 0, policy_version 1698949 (0.0010) [2023-12-27 03:46:23,052][105692] Updated weights for policy 0, policy_version 1698959 (0.0010) [2023-12-27 03:46:23,059][105620] Updated weights for policy 1, policy_version 1702803 (0.0008) [2023-12-27 03:46:23,105][105692] Updated weights for policy 0, policy_version 1698969 (0.0010) [2023-12-27 03:46:23,115][105620] Updated weights for policy 1, policy_version 1702813 (0.0006) [2023-12-27 03:46:23,173][105620] Updated weights for policy 1, policy_version 1702823 (0.0007) [2023-12-27 03:46:23,855][105692] Updated weights for policy 0, policy_version 1698979 (0.0009) [2023-12-27 03:46:23,905][105692] Updated weights for policy 0, policy_version 1698989 (0.0006) [2023-12-27 03:46:23,943][105620] Updated weights for policy 1, policy_version 1702833 (0.0008) [2023-12-27 03:46:23,956][105692] Updated weights for policy 0, policy_version 1698999 (0.0008) [2023-12-27 03:46:24,000][105620] Updated weights for policy 1, policy_version 1702843 (0.0007) [2023-12-27 03:46:24,065][105620] Updated weights for policy 1, policy_version 1702853 (0.0009) [2023-12-27 03:46:24,116][105620] Updated weights for policy 1, policy_version 1702863 (0.0009) [2023-12-27 03:46:24,661][105692] Updated weights for policy 0, policy_version 1699009 (0.0008) [2023-12-27 03:46:24,708][105692] Updated weights for policy 0, policy_version 1699019 (0.0010) [2023-12-27 03:46:24,769][105692] Updated weights for policy 0, policy_version 1699029 (0.0010) [2023-12-27 03:46:24,828][105692] Updated weights for policy 0, policy_version 1699039 (0.0010) [2023-12-27 03:46:24,892][105620] Updated weights for policy 1, policy_version 1702873 (0.0008) [2023-12-27 03:46:24,943][105620] Updated weights for policy 1, policy_version 1702883 (0.0008) [2023-12-27 03:46:24,991][105620] Updated weights for policy 1, policy_version 1702893 (0.0008) [2023-12-27 03:46:25,452][105692] Updated weights for policy 0, policy_version 1699049 (0.0006) [2023-12-27 03:46:25,509][105692] Updated weights for policy 0, policy_version 1699059 (0.0005) [2023-12-27 03:46:25,566][105692] Updated weights for policy 0, policy_version 1699069 (0.0005) [2023-12-27 03:46:25,777][105620] Updated weights for policy 1, policy_version 1702903 (0.0009) [2023-12-27 03:46:25,842][105620] Updated weights for policy 1, policy_version 1702913 (0.0009) [2023-12-27 03:46:25,904][105620] Updated weights for policy 1, policy_version 1702923 (0.0008) [2023-12-27 03:46:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 871038976. Throughput: 0: 9557.6, 1: 9653.1. Samples: 871046780. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:46:26,062][104569] Avg episode reward: [(0, '8438.569'), (1, '9081.236')] [2023-12-27 03:46:26,091][105692] Updated weights for policy 0, policy_version 1699079 (0.0007) [2023-12-27 03:46:26,142][105692] Updated weights for policy 0, policy_version 1699089 (0.0005) [2023-12-27 03:46:26,203][105692] Updated weights for policy 0, policy_version 1699099 (0.0005) [2023-12-27 03:46:26,659][105620] Updated weights for policy 1, policy_version 1702933 (0.0006) [2023-12-27 03:46:26,713][105620] Updated weights for policy 1, policy_version 1702943 (0.0009) [2023-12-27 03:46:26,765][105620] Updated weights for policy 1, policy_version 1702953 (0.0007) [2023-12-27 03:46:26,844][105692] Updated weights for policy 0, policy_version 1699109 (0.0007) [2023-12-27 03:46:26,904][105692] Updated weights for policy 0, policy_version 1699119 (0.0009) [2023-12-27 03:46:26,971][105692] Updated weights for policy 0, policy_version 1699129 (0.0008) [2023-12-27 03:46:27,486][105620] Updated weights for policy 1, policy_version 1702963 (0.0008) [2023-12-27 03:46:27,547][105620] Updated weights for policy 1, policy_version 1702973 (0.0008) [2023-12-27 03:46:27,598][105620] Updated weights for policy 1, policy_version 1702983 (0.0007) [2023-12-27 03:46:27,704][105692] Updated weights for policy 0, policy_version 1699139 (0.0008) [2023-12-27 03:46:27,759][105692] Updated weights for policy 0, policy_version 1699149 (0.0005) [2023-12-27 03:46:27,823][105692] Updated weights for policy 0, policy_version 1699159 (0.0006) [2023-12-27 03:46:28,371][105692] Updated weights for policy 0, policy_version 1699169 (0.0006) [2023-12-27 03:46:28,397][105620] Updated weights for policy 1, policy_version 1702993 (0.0008) [2023-12-27 03:46:28,430][105692] Updated weights for policy 0, policy_version 1699179 (0.0011) [2023-12-27 03:46:28,452][105620] Updated weights for policy 1, policy_version 1703003 (0.0005) [2023-12-27 03:46:28,489][105692] Updated weights for policy 0, policy_version 1699189 (0.0011) [2023-12-27 03:46:28,502][105620] Updated weights for policy 1, policy_version 1703013 (0.0005) [2023-12-27 03:46:28,552][105692] Updated weights for policy 0, policy_version 1699199 (0.0010) [2023-12-27 03:46:28,556][105620] Updated weights for policy 1, policy_version 1703023 (0.0007) [2023-12-27 03:46:29,164][105692] Updated weights for policy 0, policy_version 1699209 (0.0010) [2023-12-27 03:46:29,219][105692] Updated weights for policy 0, policy_version 1699219 (0.0010) [2023-12-27 03:46:29,277][105692] Updated weights for policy 0, policy_version 1699229 (0.0007) [2023-12-27 03:46:29,286][105620] Updated weights for policy 1, policy_version 1703033 (0.0008) [2023-12-27 03:46:29,354][105620] Updated weights for policy 1, policy_version 1703043 (0.0007) [2023-12-27 03:46:29,409][105620] Updated weights for policy 1, policy_version 1703053 (0.0007) [2023-12-27 03:46:29,975][105692] Updated weights for policy 0, policy_version 1699239 (0.0010) [2023-12-27 03:46:30,024][105692] Updated weights for policy 0, policy_version 1699249 (0.0007) [2023-12-27 03:46:30,053][105620] Updated weights for policy 1, policy_version 1703063 (0.0007) [2023-12-27 03:46:30,073][105692] Updated weights for policy 0, policy_version 1699259 (0.0010) [2023-12-27 03:46:30,107][105620] Updated weights for policy 1, policy_version 1703073 (0.0005) [2023-12-27 03:46:30,171][105620] Updated weights for policy 1, policy_version 1703083 (0.0008) [2023-12-27 03:46:30,825][105692] Updated weights for policy 0, policy_version 1699269 (0.0010) [2023-12-27 03:46:30,847][105620] Updated weights for policy 1, policy_version 1703093 (0.0010) [2023-12-27 03:46:30,879][105692] Updated weights for policy 0, policy_version 1699279 (0.0010) [2023-12-27 03:46:30,891][105620] Updated weights for policy 1, policy_version 1703103 (0.0010) [2023-12-27 03:46:30,933][105692] Updated weights for policy 0, policy_version 1699289 (0.0010) [2023-12-27 03:46:30,949][105620] Updated weights for policy 1, policy_version 1703113 (0.0010) [2023-12-27 03:46:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 871145472. Throughput: 0: 9636.2, 1: 9630.6. Samples: 871107584. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:46:31,063][104569] Avg episode reward: [(0, '8617.934'), (1, '8989.926')] [2023-12-27 03:46:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001699296_435085312.pth... [2023-12-27 03:46:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001703120_436060160.pth... [2023-12-27 03:46:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001702000_435773440.pth [2023-12-27 03:46:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001698176_434798592.pth [2023-12-27 03:46:31,665][105692] Updated weights for policy 0, policy_version 1699299 (0.0007) [2023-12-27 03:46:31,729][105620] Updated weights for policy 1, policy_version 1703123 (0.0009) [2023-12-27 03:46:31,731][105692] Updated weights for policy 0, policy_version 1699309 (0.0009) [2023-12-27 03:46:31,792][105620] Updated weights for policy 1, policy_version 1703133 (0.0007) [2023-12-27 03:46:31,794][105692] Updated weights for policy 0, policy_version 1699319 (0.0006) [2023-12-27 03:46:31,855][105620] Updated weights for policy 1, policy_version 1703143 (0.0008) [2023-12-27 03:46:32,345][105692] Updated weights for policy 0, policy_version 1699329 (0.0006) [2023-12-27 03:46:32,401][105692] Updated weights for policy 0, policy_version 1699339 (0.0010) [2023-12-27 03:46:32,450][105692] Updated weights for policy 0, policy_version 1699349 (0.0010) [2023-12-27 03:46:32,482][105620] Updated weights for policy 1, policy_version 1703153 (0.0010) [2023-12-27 03:46:32,500][105692] Updated weights for policy 0, policy_version 1699359 (0.0010) [2023-12-27 03:46:32,536][105620] Updated weights for policy 1, policy_version 1703163 (0.0007) [2023-12-27 03:46:32,598][105620] Updated weights for policy 1, policy_version 1703173 (0.0007) [2023-12-27 03:46:32,653][105620] Updated weights for policy 1, policy_version 1703183 (0.0007) [2023-12-27 03:46:33,239][105692] Updated weights for policy 0, policy_version 1699369 (0.0006) [2023-12-27 03:46:33,305][105692] Updated weights for policy 0, policy_version 1699379 (0.0005) [2023-12-27 03:46:33,364][105620] Updated weights for policy 1, policy_version 1703193 (0.0009) [2023-12-27 03:46:33,365][105692] Updated weights for policy 0, policy_version 1699389 (0.0005) [2023-12-27 03:46:33,413][105620] Updated weights for policy 1, policy_version 1703203 (0.0009) [2023-12-27 03:46:33,465][105620] Updated weights for policy 1, policy_version 1703213 (0.0009) [2023-12-27 03:46:33,971][105692] Updated weights for policy 0, policy_version 1699399 (0.0006) [2023-12-27 03:46:34,018][105692] Updated weights for policy 0, policy_version 1699409 (0.0005) [2023-12-27 03:46:34,074][105692] Updated weights for policy 0, policy_version 1699419 (0.0006) [2023-12-27 03:46:34,286][105620] Updated weights for policy 1, policy_version 1703223 (0.0008) [2023-12-27 03:46:34,348][105620] Updated weights for policy 1, policy_version 1703233 (0.0008) [2023-12-27 03:46:34,409][105620] Updated weights for policy 1, policy_version 1703243 (0.0009) [2023-12-27 03:46:34,733][105692] Updated weights for policy 0, policy_version 1699429 (0.0009) [2023-12-27 03:46:34,787][105692] Updated weights for policy 0, policy_version 1699439 (0.0009) [2023-12-27 03:46:34,843][105692] Updated weights for policy 0, policy_version 1699449 (0.0009) [2023-12-27 03:46:35,132][105620] Updated weights for policy 1, policy_version 1703253 (0.0007) [2023-12-27 03:46:35,192][105620] Updated weights for policy 1, policy_version 1703263 (0.0005) [2023-12-27 03:46:35,258][105620] Updated weights for policy 1, policy_version 1703273 (0.0011) [2023-12-27 03:46:35,617][105692] Updated weights for policy 0, policy_version 1699459 (0.0007) [2023-12-27 03:46:35,661][105692] Updated weights for policy 0, policy_version 1699469 (0.0007) [2023-12-27 03:46:35,709][105692] Updated weights for policy 0, policy_version 1699479 (0.0008) [2023-12-27 03:46:35,956][105620] Updated weights for policy 1, policy_version 1703283 (0.0011) [2023-12-27 03:46:36,007][105620] Updated weights for policy 1, policy_version 1703293 (0.0010) [2023-12-27 03:46:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.3, 300 sec: 19522.0). Total num frames: 871235584. Throughput: 0: 9728.4, 1: 9633.6. Samples: 871228504. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:46:36,062][104569] Avg episode reward: [(0, '8344.305'), (1, '9079.468')] [2023-12-27 03:46:36,068][105620] Updated weights for policy 1, policy_version 1703303 (0.0010) [2023-12-27 03:46:36,508][105692] Updated weights for policy 0, policy_version 1699489 (0.0008) [2023-12-27 03:46:36,569][105692] Updated weights for policy 0, policy_version 1699499 (0.0009) [2023-12-27 03:46:36,629][105692] Updated weights for policy 0, policy_version 1699509 (0.0008) [2023-12-27 03:46:36,690][105692] Updated weights for policy 0, policy_version 1699519 (0.0008) [2023-12-27 03:46:36,824][105620] Updated weights for policy 1, policy_version 1703313 (0.0011) [2023-12-27 03:46:36,890][105620] Updated weights for policy 1, policy_version 1703323 (0.0011) [2023-12-27 03:46:36,965][105620] Updated weights for policy 1, policy_version 1703333 (0.0011) [2023-12-27 03:46:37,027][105620] Updated weights for policy 1, policy_version 1703343 (0.0010) [2023-12-27 03:46:37,306][105692] Updated weights for policy 0, policy_version 1699529 (0.0005) [2023-12-27 03:46:37,359][105692] Updated weights for policy 0, policy_version 1699539 (0.0005) [2023-12-27 03:46:37,408][105692] Updated weights for policy 0, policy_version 1699549 (0.0005) [2023-12-27 03:46:37,740][105620] Updated weights for policy 1, policy_version 1703353 (0.0011) [2023-12-27 03:46:37,796][105620] Updated weights for policy 1, policy_version 1703363 (0.0011) [2023-12-27 03:46:37,859][105620] Updated weights for policy 1, policy_version 1703373 (0.0011) [2023-12-27 03:46:37,965][105692] Updated weights for policy 0, policy_version 1699559 (0.0006) [2023-12-27 03:46:38,024][105692] Updated weights for policy 0, policy_version 1699569 (0.0007) [2023-12-27 03:46:38,083][105692] Updated weights for policy 0, policy_version 1699579 (0.0008) [2023-12-27 03:46:38,550][105620] Updated weights for policy 1, policy_version 1703383 (0.0011) [2023-12-27 03:46:38,612][105620] Updated weights for policy 1, policy_version 1703393 (0.0010) [2023-12-27 03:46:38,668][105620] Updated weights for policy 1, policy_version 1703403 (0.0010) [2023-12-27 03:46:38,822][105692] Updated weights for policy 0, policy_version 1699589 (0.0007) [2023-12-27 03:46:38,884][105692] Updated weights for policy 0, policy_version 1699599 (0.0007) [2023-12-27 03:46:38,941][105692] Updated weights for policy 0, policy_version 1699609 (0.0008) [2023-12-27 03:46:39,428][105620] Updated weights for policy 1, policy_version 1703413 (0.0010) [2023-12-27 03:46:39,486][105620] Updated weights for policy 1, policy_version 1703423 (0.0009) [2023-12-27 03:46:39,538][105620] Updated weights for policy 1, policy_version 1703433 (0.0009) [2023-12-27 03:46:39,703][105692] Updated weights for policy 0, policy_version 1699619 (0.0009) [2023-12-27 03:46:39,771][105692] Updated weights for policy 0, policy_version 1699629 (0.0007) [2023-12-27 03:46:39,834][105692] Updated weights for policy 0, policy_version 1699639 (0.0008) [2023-12-27 03:46:40,350][105620] Updated weights for policy 1, policy_version 1703443 (0.0009) [2023-12-27 03:46:40,412][105620] Updated weights for policy 1, policy_version 1703453 (0.0008) [2023-12-27 03:46:40,478][105620] Updated weights for policy 1, policy_version 1703463 (0.0008) [2023-12-27 03:46:40,567][105692] Updated weights for policy 0, policy_version 1699649 (0.0008) [2023-12-27 03:46:40,626][105692] Updated weights for policy 0, policy_version 1699659 (0.0006) [2023-12-27 03:46:40,685][105692] Updated weights for policy 0, policy_version 1699669 (0.0009) [2023-12-27 03:46:40,731][105692] Updated weights for policy 0, policy_version 1699679 (0.0008) [2023-12-27 03:46:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 871333888. Throughput: 0: 9667.7, 1: 9511.8. Samples: 871343492. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:46:41,063][104569] Avg episode reward: [(0, '8715.091'), (1, '9263.138')] [2023-12-27 03:46:41,293][105620] Updated weights for policy 1, policy_version 1703473 (0.0009) [2023-12-27 03:46:41,355][105620] Updated weights for policy 1, policy_version 1703483 (0.0008) [2023-12-27 03:46:41,412][105620] Updated weights for policy 1, policy_version 1703493 (0.0010) [2023-12-27 03:46:41,453][105692] Updated weights for policy 0, policy_version 1699689 (0.0006) [2023-12-27 03:46:41,479][105620] Updated weights for policy 1, policy_version 1703503 (0.0009) [2023-12-27 03:46:41,517][105692] Updated weights for policy 0, policy_version 1699699 (0.0008) [2023-12-27 03:46:41,577][105692] Updated weights for policy 0, policy_version 1699709 (0.0010) [2023-12-27 03:46:42,257][105692] Updated weights for policy 0, policy_version 1699719 (0.0008) [2023-12-27 03:46:42,300][105620] Updated weights for policy 1, policy_version 1703513 (0.0007) [2023-12-27 03:46:42,317][105692] Updated weights for policy 0, policy_version 1699729 (0.0009) [2023-12-27 03:46:42,364][105620] Updated weights for policy 1, policy_version 1703523 (0.0007) [2023-12-27 03:46:42,380][105692] Updated weights for policy 0, policy_version 1699739 (0.0011) [2023-12-27 03:46:42,428][105620] Updated weights for policy 1, policy_version 1703533 (0.0008) [2023-12-27 03:46:43,080][105692] Updated weights for policy 0, policy_version 1699749 (0.0008) [2023-12-27 03:46:43,131][105692] Updated weights for policy 0, policy_version 1699759 (0.0007) [2023-12-27 03:46:43,182][105620] Updated weights for policy 1, policy_version 1703543 (0.0007) [2023-12-27 03:46:43,195][105692] Updated weights for policy 0, policy_version 1699769 (0.0010) [2023-12-27 03:46:43,240][105620] Updated weights for policy 1, policy_version 1703553 (0.0005) [2023-12-27 03:46:43,296][105620] Updated weights for policy 1, policy_version 1703563 (0.0008) [2023-12-27 03:46:43,909][105692] Updated weights for policy 0, policy_version 1699779 (0.0011) [2023-12-27 03:46:43,959][105692] Updated weights for policy 0, policy_version 1699789 (0.0008) [2023-12-27 03:46:43,961][105620] Updated weights for policy 1, policy_version 1703573 (0.0007) [2023-12-27 03:46:44,008][105692] Updated weights for policy 0, policy_version 1699799 (0.0005) [2023-12-27 03:46:44,012][105620] Updated weights for policy 1, policy_version 1703583 (0.0006) [2023-12-27 03:46:44,075][105620] Updated weights for policy 1, policy_version 1703593 (0.0005) [2023-12-27 03:46:44,605][105620] Updated weights for policy 1, policy_version 1703603 (0.0006) [2023-12-27 03:46:44,671][105620] Updated weights for policy 1, policy_version 1703613 (0.0011) [2023-12-27 03:46:44,692][105692] Updated weights for policy 0, policy_version 1699809 (0.0005) [2023-12-27 03:46:44,737][105620] Updated weights for policy 1, policy_version 1703623 (0.0011) [2023-12-27 03:46:44,740][105692] Updated weights for policy 0, policy_version 1699819 (0.0007) [2023-12-27 03:46:44,803][105692] Updated weights for policy 0, policy_version 1699829 (0.0009) [2023-12-27 03:46:44,860][105692] Updated weights for policy 0, policy_version 1699839 (0.0011) [2023-12-27 03:46:45,479][105620] Updated weights for policy 1, policy_version 1703633 (0.0011) [2023-12-27 03:46:45,528][105620] Updated weights for policy 1, policy_version 1703643 (0.0011) [2023-12-27 03:46:45,593][105620] Updated weights for policy 1, policy_version 1703653 (0.0009) [2023-12-27 03:46:45,633][105692] Updated weights for policy 0, policy_version 1699849 (0.0011) [2023-12-27 03:46:45,651][105620] Updated weights for policy 1, policy_version 1703663 (0.0005) [2023-12-27 03:46:45,685][105692] Updated weights for policy 0, policy_version 1699859 (0.0010) [2023-12-27 03:46:45,744][105692] Updated weights for policy 0, policy_version 1699869 (0.0011) [2023-12-27 03:46:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 871432192. Throughput: 0: 9677.6, 1: 9472.2. Samples: 871400384. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:46:46,063][104569] Avg episode reward: [(0, '8987.573'), (1, '9080.104')] [2023-12-27 03:46:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001699872_435232768.pth... [2023-12-27 03:46:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001703664_436199424.pth... [2023-12-27 03:46:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001702544_435912704.pth [2023-12-27 03:46:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001698720_434937856.pth [2023-12-27 03:46:46,242][105620] Updated weights for policy 1, policy_version 1703673 (0.0010) [2023-12-27 03:46:46,300][105620] Updated weights for policy 1, policy_version 1703683 (0.0010) [2023-12-27 03:46:46,355][105620] Updated weights for policy 1, policy_version 1703693 (0.0010) [2023-12-27 03:46:46,492][105692] Updated weights for policy 0, policy_version 1699879 (0.0010) [2023-12-27 03:46:46,540][105692] Updated weights for policy 0, policy_version 1699889 (0.0005) [2023-12-27 03:46:46,587][105692] Updated weights for policy 0, policy_version 1699899 (0.0005) [2023-12-27 03:46:47,099][105620] Updated weights for policy 1, policy_version 1703703 (0.0009) [2023-12-27 03:46:47,160][105620] Updated weights for policy 1, policy_version 1703713 (0.0009) [2023-12-27 03:46:47,217][105620] Updated weights for policy 1, policy_version 1703723 (0.0008) [2023-12-27 03:46:47,290][105692] Updated weights for policy 0, policy_version 1699909 (0.0008) [2023-12-27 03:46:47,341][105692] Updated weights for policy 0, policy_version 1699919 (0.0009) [2023-12-27 03:46:47,399][105692] Updated weights for policy 0, policy_version 1699929 (0.0009) [2023-12-27 03:46:48,021][105620] Updated weights for policy 1, policy_version 1703733 (0.0009) [2023-12-27 03:46:48,082][105620] Updated weights for policy 1, policy_version 1703743 (0.0008) [2023-12-27 03:46:48,097][105692] Updated weights for policy 0, policy_version 1699939 (0.0008) [2023-12-27 03:46:48,136][105620] Updated weights for policy 1, policy_version 1703753 (0.0008) [2023-12-27 03:46:48,154][105692] Updated weights for policy 0, policy_version 1699949 (0.0007) [2023-12-27 03:46:48,218][105692] Updated weights for policy 0, policy_version 1699959 (0.0008) [2023-12-27 03:46:48,912][105620] Updated weights for policy 1, policy_version 1703763 (0.0008) [2023-12-27 03:46:48,970][105620] Updated weights for policy 1, policy_version 1703773 (0.0008) [2023-12-27 03:46:48,979][105692] Updated weights for policy 0, policy_version 1699969 (0.0009) [2023-12-27 03:46:49,029][105620] Updated weights for policy 1, policy_version 1703783 (0.0009) [2023-12-27 03:46:49,039][105692] Updated weights for policy 0, policy_version 1699979 (0.0005) [2023-12-27 03:46:49,105][105692] Updated weights for policy 0, policy_version 1699989 (0.0005) [2023-12-27 03:46:49,168][105692] Updated weights for policy 0, policy_version 1699999 (0.0005) [2023-12-27 03:46:49,729][105620] Updated weights for policy 1, policy_version 1703793 (0.0009) [2023-12-27 03:46:49,759][105692] Updated weights for policy 0, policy_version 1700009 (0.0007) [2023-12-27 03:46:49,788][105620] Updated weights for policy 1, policy_version 1703803 (0.0007) [2023-12-27 03:46:49,820][105692] Updated weights for policy 0, policy_version 1700019 (0.0008) [2023-12-27 03:46:49,850][105620] Updated weights for policy 1, policy_version 1703813 (0.0007) [2023-12-27 03:46:49,884][105692] Updated weights for policy 0, policy_version 1700029 (0.0006) [2023-12-27 03:46:49,914][105620] Updated weights for policy 1, policy_version 1703823 (0.0006) [2023-12-27 03:46:50,545][105620] Updated weights for policy 1, policy_version 1703833 (0.0010) [2023-12-27 03:46:50,588][105692] Updated weights for policy 0, policy_version 1700039 (0.0007) [2023-12-27 03:46:50,607][105620] Updated weights for policy 1, policy_version 1703843 (0.0010) [2023-12-27 03:46:50,650][105692] Updated weights for policy 0, policy_version 1700049 (0.0006) [2023-12-27 03:46:50,670][105620] Updated weights for policy 1, policy_version 1703853 (0.0010) [2023-12-27 03:46:50,706][105692] Updated weights for policy 0, policy_version 1700059 (0.0006) [2023-12-27 03:46:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 871530496. Throughput: 0: 9740.9, 1: 9547.1. Samples: 871519272. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:46:51,063][104569] Avg episode reward: [(0, '8159.707'), (1, '9080.169')] [2023-12-27 03:46:51,434][105692] Updated weights for policy 0, policy_version 1700069 (0.0008) [2023-12-27 03:46:51,464][105620] Updated weights for policy 1, policy_version 1703863 (0.0010) [2023-12-27 03:46:51,498][105692] Updated weights for policy 0, policy_version 1700079 (0.0006) [2023-12-27 03:46:51,526][105620] Updated weights for policy 1, policy_version 1703873 (0.0010) [2023-12-27 03:46:51,557][105692] Updated weights for policy 0, policy_version 1700089 (0.0006) [2023-12-27 03:46:51,575][105620] Updated weights for policy 1, policy_version 1703883 (0.0010) [2023-12-27 03:46:52,201][105692] Updated weights for policy 0, policy_version 1700099 (0.0007) [2023-12-27 03:46:52,264][105692] Updated weights for policy 0, policy_version 1700109 (0.0009) [2023-12-27 03:46:52,327][105620] Updated weights for policy 1, policy_version 1703893 (0.0010) [2023-12-27 03:46:52,331][105692] Updated weights for policy 0, policy_version 1700119 (0.0009) [2023-12-27 03:46:52,392][105620] Updated weights for policy 1, policy_version 1703903 (0.0009) [2023-12-27 03:46:52,450][105620] Updated weights for policy 1, policy_version 1703913 (0.0009) [2023-12-27 03:46:53,048][105692] Updated weights for policy 0, policy_version 1700129 (0.0008) [2023-12-27 03:46:53,109][105692] Updated weights for policy 0, policy_version 1700139 (0.0010) [2023-12-27 03:46:53,163][105692] Updated weights for policy 0, policy_version 1700149 (0.0010) [2023-12-27 03:46:53,205][105620] Updated weights for policy 1, policy_version 1703923 (0.0010) [2023-12-27 03:46:53,227][105692] Updated weights for policy 0, policy_version 1700159 (0.0010) [2023-12-27 03:46:53,252][105620] Updated weights for policy 1, policy_version 1703933 (0.0010) [2023-12-27 03:46:53,303][105620] Updated weights for policy 1, policy_version 1703943 (0.0010) [2023-12-27 03:46:53,807][105692] Updated weights for policy 0, policy_version 1700169 (0.0007) [2023-12-27 03:46:53,860][105692] Updated weights for policy 0, policy_version 1700179 (0.0005) [2023-12-27 03:46:53,909][105692] Updated weights for policy 0, policy_version 1700189 (0.0008) [2023-12-27 03:46:54,064][105620] Updated weights for policy 1, policy_version 1703953 (0.0010) [2023-12-27 03:46:54,116][105620] Updated weights for policy 1, policy_version 1703963 (0.0010) [2023-12-27 03:46:54,172][105620] Updated weights for policy 1, policy_version 1703973 (0.0010) [2023-12-27 03:46:54,229][105620] Updated weights for policy 1, policy_version 1703983 (0.0010) [2023-12-27 03:46:54,509][105692] Updated weights for policy 0, policy_version 1700199 (0.0007) [2023-12-27 03:46:54,577][105692] Updated weights for policy 0, policy_version 1700209 (0.0007) [2023-12-27 03:46:54,645][105692] Updated weights for policy 0, policy_version 1700219 (0.0010) [2023-12-27 03:46:54,987][105620] Updated weights for policy 1, policy_version 1703993 (0.0008) [2023-12-27 03:46:55,043][105620] Updated weights for policy 1, policy_version 1704003 (0.0010) [2023-12-27 03:46:55,093][105620] Updated weights for policy 1, policy_version 1704013 (0.0007) [2023-12-27 03:46:55,293][105692] Updated weights for policy 0, policy_version 1700229 (0.0010) [2023-12-27 03:46:55,354][105692] Updated weights for policy 0, policy_version 1700239 (0.0010) [2023-12-27 03:46:55,412][105692] Updated weights for policy 0, policy_version 1700249 (0.0008) [2023-12-27 03:46:55,727][105620] Updated weights for policy 1, policy_version 1704023 (0.0009) [2023-12-27 03:46:55,772][105620] Updated weights for policy 1, policy_version 1704033 (0.0010) [2023-12-27 03:46:55,820][105620] Updated weights for policy 1, policy_version 1704043 (0.0010) [2023-12-27 03:46:55,978][105692] Updated weights for policy 0, policy_version 1700259 (0.0005) [2023-12-27 03:46:56,039][105692] Updated weights for policy 0, policy_version 1700269 (0.0005) [2023-12-27 03:46:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 871628800. Throughput: 0: 9888.5, 1: 9594.5. Samples: 871638780. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:46:56,062][104569] Avg episode reward: [(0, '8343.348'), (1, '9171.862')] [2023-12-27 03:46:56,086][105692] Updated weights for policy 0, policy_version 1700279 (0.0008) [2023-12-27 03:46:56,476][105620] Updated weights for policy 1, policy_version 1704053 (0.0008) [2023-12-27 03:46:56,542][105620] Updated weights for policy 1, policy_version 1704063 (0.0008) [2023-12-27 03:46:56,604][105620] Updated weights for policy 1, policy_version 1704073 (0.0008) [2023-12-27 03:46:56,687][105692] Updated weights for policy 0, policy_version 1700289 (0.0010) [2023-12-27 03:46:56,731][105692] Updated weights for policy 0, policy_version 1700299 (0.0006) [2023-12-27 03:46:56,775][105692] Updated weights for policy 0, policy_version 1700309 (0.0010) [2023-12-27 03:46:56,831][105692] Updated weights for policy 0, policy_version 1700319 (0.0007) [2023-12-27 03:46:57,171][105620] Updated weights for policy 1, policy_version 1704083 (0.0007) [2023-12-27 03:46:57,219][105620] Updated weights for policy 1, policy_version 1704093 (0.0005) [2023-12-27 03:46:57,273][105620] Updated weights for policy 1, policy_version 1704103 (0.0007) [2023-12-27 03:46:57,490][105692] Updated weights for policy 0, policy_version 1700329 (0.0008) [2023-12-27 03:46:57,541][105692] Updated weights for policy 0, policy_version 1700339 (0.0011) [2023-12-27 03:46:57,589][105692] Updated weights for policy 0, policy_version 1700349 (0.0010) [2023-12-27 03:46:57,883][105620] Updated weights for policy 1, policy_version 1704113 (0.0006) [2023-12-27 03:46:57,945][105620] Updated weights for policy 1, policy_version 1704123 (0.0005) [2023-12-27 03:46:57,998][105620] Updated weights for policy 1, policy_version 1704133 (0.0006) [2023-12-27 03:46:58,050][105620] Updated weights for policy 1, policy_version 1704143 (0.0005) [2023-12-27 03:46:58,274][105692] Updated weights for policy 0, policy_version 1700359 (0.0008) [2023-12-27 03:46:58,335][105692] Updated weights for policy 0, policy_version 1700369 (0.0009) [2023-12-27 03:46:58,397][105692] Updated weights for policy 0, policy_version 1700379 (0.0008) [2023-12-27 03:46:58,735][105620] Updated weights for policy 1, policy_version 1704153 (0.0007) [2023-12-27 03:46:58,803][105620] Updated weights for policy 1, policy_version 1704163 (0.0007) [2023-12-27 03:46:58,876][105620] Updated weights for policy 1, policy_version 1704173 (0.0008) [2023-12-27 03:46:59,239][105692] Updated weights for policy 0, policy_version 1700389 (0.0008) [2023-12-27 03:46:59,300][105692] Updated weights for policy 0, policy_version 1700399 (0.0008) [2023-12-27 03:46:59,370][105692] Updated weights for policy 0, policy_version 1700409 (0.0008) [2023-12-27 03:46:59,594][105620] Updated weights for policy 1, policy_version 1704183 (0.0009) [2023-12-27 03:46:59,652][105620] Updated weights for policy 1, policy_version 1704193 (0.0010) [2023-12-27 03:46:59,714][105620] Updated weights for policy 1, policy_version 1704203 (0.0010) [2023-12-27 03:47:00,116][105692] Updated weights for policy 0, policy_version 1700419 (0.0009) [2023-12-27 03:47:00,168][105692] Updated weights for policy 0, policy_version 1700429 (0.0008) [2023-12-27 03:47:00,217][105692] Updated weights for policy 0, policy_version 1700439 (0.0008) [2023-12-27 03:47:00,448][105620] Updated weights for policy 1, policy_version 1704213 (0.0010) [2023-12-27 03:47:00,512][105620] Updated weights for policy 1, policy_version 1704223 (0.0010) [2023-12-27 03:47:00,573][105620] Updated weights for policy 1, policy_version 1704233 (0.0011) [2023-12-27 03:47:00,948][105692] Updated weights for policy 0, policy_version 1700449 (0.0008) [2023-12-27 03:47:00,995][105692] Updated weights for policy 0, policy_version 1700459 (0.0010) [2023-12-27 03:47:01,057][105692] Updated weights for policy 0, policy_version 1700469 (0.0011) [2023-12-27 03:47:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.3, 300 sec: 19521.9). Total num frames: 871727104. Throughput: 0: 9966.0, 1: 9703.5. Samples: 871703328. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:47:01,062][104569] Avg episode reward: [(0, '8712.620'), (1, '9079.217')] [2023-12-27 03:47:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001704240_436346880.pth... [2023-12-27 03:47:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001703120_436060160.pth [2023-12-27 03:47:01,112][105692] Updated weights for policy 0, policy_version 1700479 (0.0010) [2023-12-27 03:47:01,120][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001700480_435388416.pth... [2023-12-27 03:47:01,124][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001699296_435085312.pth [2023-12-27 03:47:01,266][105620] Updated weights for policy 1, policy_version 1704243 (0.0010) [2023-12-27 03:47:01,335][105620] Updated weights for policy 1, policy_version 1704253 (0.0006) [2023-12-27 03:47:01,402][105620] Updated weights for policy 1, policy_version 1704263 (0.0010) [2023-12-27 03:47:01,830][105692] Updated weights for policy 0, policy_version 1700489 (0.0010) [2023-12-27 03:47:01,885][105692] Updated weights for policy 0, policy_version 1700499 (0.0010) [2023-12-27 03:47:01,947][105692] Updated weights for policy 0, policy_version 1700509 (0.0011) [2023-12-27 03:47:02,062][105620] Updated weights for policy 1, policy_version 1704273 (0.0011) [2023-12-27 03:47:02,127][105620] Updated weights for policy 1, policy_version 1704283 (0.0010) [2023-12-27 03:47:02,196][105620] Updated weights for policy 1, policy_version 1704293 (0.0010) [2023-12-27 03:47:02,262][105620] Updated weights for policy 1, policy_version 1704303 (0.0010) [2023-12-27 03:47:02,643][105692] Updated weights for policy 0, policy_version 1700519 (0.0006) [2023-12-27 03:47:02,689][105692] Updated weights for policy 0, policy_version 1700529 (0.0005) [2023-12-27 03:47:02,753][105692] Updated weights for policy 0, policy_version 1700539 (0.0006) [2023-12-27 03:47:03,036][105620] Updated weights for policy 1, policy_version 1704313 (0.0007) [2023-12-27 03:47:03,094][105620] Updated weights for policy 1, policy_version 1704323 (0.0006) [2023-12-27 03:47:03,151][105620] Updated weights for policy 1, policy_version 1704333 (0.0009) [2023-12-27 03:47:03,321][105692] Updated weights for policy 0, policy_version 1700549 (0.0008) [2023-12-27 03:47:03,373][105692] Updated weights for policy 0, policy_version 1700559 (0.0005) [2023-12-27 03:47:03,426][105692] Updated weights for policy 0, policy_version 1700569 (0.0005) [2023-12-27 03:47:03,918][105620] Updated weights for policy 1, policy_version 1704343 (0.0011) [2023-12-27 03:47:03,981][105620] Updated weights for policy 1, policy_version 1704353 (0.0006) [2023-12-27 03:47:04,045][105620] Updated weights for policy 1, policy_version 1704363 (0.0006) [2023-12-27 03:47:04,086][105692] Updated weights for policy 0, policy_version 1700579 (0.0006) [2023-12-27 03:47:04,145][105692] Updated weights for policy 0, policy_version 1700589 (0.0009) [2023-12-27 03:47:04,207][105692] Updated weights for policy 0, policy_version 1700599 (0.0009) [2023-12-27 03:47:04,669][105620] Updated weights for policy 1, policy_version 1704373 (0.0010) [2023-12-27 03:47:04,729][105620] Updated weights for policy 1, policy_version 1704383 (0.0009) [2023-12-27 03:47:04,794][105620] Updated weights for policy 1, policy_version 1704393 (0.0008) [2023-12-27 03:47:04,887][105692] Updated weights for policy 0, policy_version 1700609 (0.0010) [2023-12-27 03:47:04,936][105692] Updated weights for policy 0, policy_version 1700619 (0.0006) [2023-12-27 03:47:04,985][105692] Updated weights for policy 0, policy_version 1700629 (0.0005) [2023-12-27 03:47:05,039][105692] Updated weights for policy 0, policy_version 1700639 (0.0006) [2023-12-27 03:47:05,523][105620] Updated weights for policy 1, policy_version 1704403 (0.0009) [2023-12-27 03:47:05,579][105620] Updated weights for policy 1, policy_version 1704413 (0.0010) [2023-12-27 03:47:05,642][105620] Updated weights for policy 1, policy_version 1704423 (0.0009) [2023-12-27 03:47:05,664][105692] Updated weights for policy 0, policy_version 1700649 (0.0009) [2023-12-27 03:47:05,719][105692] Updated weights for policy 0, policy_version 1700659 (0.0007) [2023-12-27 03:47:05,781][105692] Updated weights for policy 0, policy_version 1700669 (0.0010) [2023-12-27 03:47:06,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 871833600. Throughput: 0: 10069.4, 1: 9614.3. Samples: 871819760. Policy #0 lag: (min: 0.0, avg: 23.1, max: 32.0) [2023-12-27 03:47:06,063][104569] Avg episode reward: [(0, '8899.379'), (1, '8988.416')] [2023-12-27 03:47:06,353][105620] Updated weights for policy 1, policy_version 1704433 (0.0007) [2023-12-27 03:47:06,411][105620] Updated weights for policy 1, policy_version 1704443 (0.0007) [2023-12-27 03:47:06,443][105692] Updated weights for policy 0, policy_version 1700680 (0.0010) [2023-12-27 03:47:06,477][105620] Updated weights for policy 1, policy_version 1704453 (0.0006) [2023-12-27 03:47:06,512][105692] Updated weights for policy 0, policy_version 1700690 (0.0009) [2023-12-27 03:47:06,537][105620] Updated weights for policy 1, policy_version 1704463 (0.0006) [2023-12-27 03:47:06,575][105692] Updated weights for policy 0, policy_version 1700700 (0.0009) [2023-12-27 03:47:07,189][105620] Updated weights for policy 1, policy_version 1704473 (0.0006) [2023-12-27 03:47:07,257][105620] Updated weights for policy 1, policy_version 1704483 (0.0006) [2023-12-27 03:47:07,327][105620] Updated weights for policy 1, policy_version 1704493 (0.0006) [2023-12-27 03:47:07,355][105692] Updated weights for policy 0, policy_version 1700710 (0.0009) [2023-12-27 03:47:07,411][105692] Updated weights for policy 0, policy_version 1700720 (0.0010) [2023-12-27 03:47:07,470][105692] Updated weights for policy 0, policy_version 1700730 (0.0008) [2023-12-27 03:47:07,838][105620] Updated weights for policy 1, policy_version 1704503 (0.0006) [2023-12-27 03:47:07,904][105620] Updated weights for policy 1, policy_version 1704513 (0.0009) [2023-12-27 03:47:07,968][105620] Updated weights for policy 1, policy_version 1704523 (0.0010) [2023-12-27 03:47:08,228][105692] Updated weights for policy 0, policy_version 1700740 (0.0008) [2023-12-27 03:47:08,272][105692] Updated weights for policy 0, policy_version 1700750 (0.0007) [2023-12-27 03:47:08,324][105692] Updated weights for policy 0, policy_version 1700760 (0.0006) [2023-12-27 03:47:08,642][105620] Updated weights for policy 1, policy_version 1704533 (0.0008) [2023-12-27 03:47:08,694][105620] Updated weights for policy 1, policy_version 1704543 (0.0011) [2023-12-27 03:47:08,750][105620] Updated weights for policy 1, policy_version 1704553 (0.0011) [2023-12-27 03:47:08,970][105692] Updated weights for policy 0, policy_version 1700770 (0.0008) [2023-12-27 03:47:09,037][105692] Updated weights for policy 0, policy_version 1700780 (0.0008) [2023-12-27 03:47:09,099][105692] Updated weights for policy 0, policy_version 1700790 (0.0007) [2023-12-27 03:47:09,161][105692] Updated weights for policy 0, policy_version 1700800 (0.0008) [2023-12-27 03:47:09,471][105620] Updated weights for policy 1, policy_version 1704563 (0.0011) [2023-12-27 03:47:09,525][105620] Updated weights for policy 1, policy_version 1704573 (0.0011) [2023-12-27 03:47:09,586][105620] Updated weights for policy 1, policy_version 1704583 (0.0006) [2023-12-27 03:47:09,860][105692] Updated weights for policy 0, policy_version 1700811 (0.0007) [2023-12-27 03:47:09,921][105692] Updated weights for policy 0, policy_version 1700821 (0.0011) [2023-12-27 03:47:10,004][105692] Updated weights for policy 0, policy_version 1700831 (0.0010) [2023-12-27 03:47:10,308][105620] Updated weights for policy 1, policy_version 1704593 (0.0006) [2023-12-27 03:47:10,376][105620] Updated weights for policy 1, policy_version 1704603 (0.0006) [2023-12-27 03:47:10,443][105620] Updated weights for policy 1, policy_version 1704613 (0.0008) [2023-12-27 03:47:10,504][105620] Updated weights for policy 1, policy_version 1704623 (0.0009) [2023-12-27 03:47:10,738][105692] Updated weights for policy 0, policy_version 1700841 (0.0011) [2023-12-27 03:47:10,799][105692] Updated weights for policy 0, policy_version 1700851 (0.0010) [2023-12-27 03:47:10,846][105692] Updated weights for policy 0, policy_version 1700861 (0.0010) [2023-12-27 03:47:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 871931904. Throughput: 0: 10125.8, 1: 9741.5. Samples: 871940808. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:47:11,062][104569] Avg episode reward: [(0, '8715.508'), (1, '8803.672')] [2023-12-27 03:47:11,194][105620] Updated weights for policy 1, policy_version 1704633 (0.0009) [2023-12-27 03:47:11,262][105620] Updated weights for policy 1, policy_version 1704643 (0.0011) [2023-12-27 03:47:11,318][105620] Updated weights for policy 1, policy_version 1704653 (0.0010) [2023-12-27 03:47:11,622][105692] Updated weights for policy 0, policy_version 1700871 (0.0009) [2023-12-27 03:47:11,686][105692] Updated weights for policy 0, policy_version 1700881 (0.0009) [2023-12-27 03:47:11,755][105692] Updated weights for policy 0, policy_version 1700891 (0.0009) [2023-12-27 03:47:11,990][105620] Updated weights for policy 1, policy_version 1704663 (0.0009) [2023-12-27 03:47:12,040][105620] Updated weights for policy 1, policy_version 1704673 (0.0009) [2023-12-27 03:47:12,094][105620] Updated weights for policy 1, policy_version 1704683 (0.0009) [2023-12-27 03:47:12,531][105692] Updated weights for policy 0, policy_version 1700901 (0.0008) [2023-12-27 03:47:12,590][105692] Updated weights for policy 0, policy_version 1700911 (0.0009) [2023-12-27 03:47:12,652][105692] Updated weights for policy 0, policy_version 1700921 (0.0009) [2023-12-27 03:47:12,850][105620] Updated weights for policy 1, policy_version 1704693 (0.0009) [2023-12-27 03:47:12,913][105620] Updated weights for policy 1, policy_version 1704703 (0.0009) [2023-12-27 03:47:12,975][105620] Updated weights for policy 1, policy_version 1704713 (0.0007) [2023-12-27 03:47:13,384][105692] Updated weights for policy 0, policy_version 1700931 (0.0009) [2023-12-27 03:47:13,446][105692] Updated weights for policy 0, policy_version 1700941 (0.0008) [2023-12-27 03:47:13,500][105692] Updated weights for policy 0, policy_version 1700951 (0.0008) [2023-12-27 03:47:13,637][105620] Updated weights for policy 1, policy_version 1704723 (0.0006) [2023-12-27 03:47:13,691][105620] Updated weights for policy 1, policy_version 1704733 (0.0009) [2023-12-27 03:47:13,742][105620] Updated weights for policy 1, policy_version 1704743 (0.0009) [2023-12-27 03:47:14,173][105692] Updated weights for policy 0, policy_version 1700961 (0.0009) [2023-12-27 03:47:14,221][105692] Updated weights for policy 0, policy_version 1700971 (0.0009) [2023-12-27 03:47:14,271][105692] Updated weights for policy 0, policy_version 1700981 (0.0009) [2023-12-27 03:47:14,318][105692] Updated weights for policy 0, policy_version 1700991 (0.0009) [2023-12-27 03:47:14,516][105620] Updated weights for policy 1, policy_version 1704753 (0.0010) [2023-12-27 03:47:14,563][105620] Updated weights for policy 1, policy_version 1704763 (0.0008) [2023-12-27 03:47:14,616][105620] Updated weights for policy 1, policy_version 1704773 (0.0009) [2023-12-27 03:47:14,673][105620] Updated weights for policy 1, policy_version 1704783 (0.0007) [2023-12-27 03:47:15,111][105692] Updated weights for policy 0, policy_version 1701001 (0.0010) [2023-12-27 03:47:15,169][105692] Updated weights for policy 0, policy_version 1701011 (0.0006) [2023-12-27 03:47:15,235][105692] Updated weights for policy 0, policy_version 1701021 (0.0008) [2023-12-27 03:47:15,378][105620] Updated weights for policy 1, policy_version 1704793 (0.0009) [2023-12-27 03:47:15,440][105620] Updated weights for policy 1, policy_version 1704803 (0.0009) [2023-12-27 03:47:15,495][105620] Updated weights for policy 1, policy_version 1704813 (0.0009) [2023-12-27 03:47:15,975][105692] Updated weights for policy 0, policy_version 1701031 (0.0006) [2023-12-27 03:47:16,037][105692] Updated weights for policy 0, policy_version 1701041 (0.0008) [2023-12-27 03:47:16,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 872022016. Throughput: 0: 10011.2, 1: 9762.0. Samples: 871997376. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:47:16,062][104569] Avg episode reward: [(0, '8711.782'), (1, '8893.778')] [2023-12-27 03:47:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001704816_436494336.pth... [2023-12-27 03:47:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001703664_436199424.pth [2023-12-27 03:47:16,103][105692] Updated weights for policy 0, policy_version 1701051 (0.0010) [2023-12-27 03:47:16,138][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001701056_435535872.pth... [2023-12-27 03:47:16,143][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001699872_435232768.pth [2023-12-27 03:47:16,157][105620] Updated weights for policy 1, policy_version 1704823 (0.0007) [2023-12-27 03:47:16,217][105620] Updated weights for policy 1, policy_version 1704833 (0.0005) [2023-12-27 03:47:16,280][105620] Updated weights for policy 1, policy_version 1704844 (0.0008) [2023-12-27 03:47:16,828][105692] Updated weights for policy 0, policy_version 1701061 (0.0008) [2023-12-27 03:47:16,857][105620] Updated weights for policy 1, policy_version 1704854 (0.0006) [2023-12-27 03:47:16,879][105692] Updated weights for policy 0, policy_version 1701071 (0.0008) [2023-12-27 03:47:16,906][105620] Updated weights for policy 1, policy_version 1704864 (0.0007) [2023-12-27 03:47:16,929][105692] Updated weights for policy 0, policy_version 1701081 (0.0007) [2023-12-27 03:47:16,955][105620] Updated weights for policy 1, policy_version 1704874 (0.0008) [2023-12-27 03:47:17,593][105620] Updated weights for policy 1, policy_version 1704884 (0.0008) [2023-12-27 03:47:17,651][105620] Updated weights for policy 1, policy_version 1704894 (0.0009) [2023-12-27 03:47:17,699][105620] Updated weights for policy 1, policy_version 1704904 (0.0009) [2023-12-27 03:47:17,710][105692] Updated weights for policy 0, policy_version 1701091 (0.0005) [2023-12-27 03:47:17,761][105692] Updated weights for policy 0, policy_version 1701101 (0.0007) [2023-12-27 03:47:17,808][105692] Updated weights for policy 0, policy_version 1701111 (0.0009) [2023-12-27 03:47:18,447][105620] Updated weights for policy 1, policy_version 1704914 (0.0008) [2023-12-27 03:47:18,506][105620] Updated weights for policy 1, policy_version 1704924 (0.0009) [2023-12-27 03:47:18,571][105620] Updated weights for policy 1, policy_version 1704935 (0.0010) [2023-12-27 03:47:18,586][105692] Updated weights for policy 0, policy_version 1701121 (0.0008) [2023-12-27 03:47:18,647][105692] Updated weights for policy 0, policy_version 1701131 (0.0006) [2023-12-27 03:47:18,705][105692] Updated weights for policy 0, policy_version 1701141 (0.0006) [2023-12-27 03:47:18,766][105692] Updated weights for policy 0, policy_version 1701151 (0.0010) [2023-12-27 03:47:19,362][105620] Updated weights for policy 1, policy_version 1704945 (0.0007) [2023-12-27 03:47:19,422][105620] Updated weights for policy 1, policy_version 1704955 (0.0008) [2023-12-27 03:47:19,462][105692] Updated weights for policy 0, policy_version 1701161 (0.0008) [2023-12-27 03:47:19,481][105620] Updated weights for policy 1, policy_version 1704965 (0.0007) [2023-12-27 03:47:19,519][105692] Updated weights for policy 0, policy_version 1701171 (0.0008) [2023-12-27 03:47:19,545][105620] Updated weights for policy 1, policy_version 1704975 (0.0008) [2023-12-27 03:47:19,575][105692] Updated weights for policy 0, policy_version 1701181 (0.0008) [2023-12-27 03:47:20,293][105620] Updated weights for policy 1, policy_version 1704985 (0.0008) [2023-12-27 03:47:20,300][105692] Updated weights for policy 0, policy_version 1701191 (0.0007) [2023-12-27 03:47:20,348][105692] Updated weights for policy 0, policy_version 1701201 (0.0007) [2023-12-27 03:47:20,350][105620] Updated weights for policy 1, policy_version 1704995 (0.0007) [2023-12-27 03:47:20,407][105692] Updated weights for policy 0, policy_version 1701211 (0.0007) [2023-12-27 03:47:20,409][105620] Updated weights for policy 1, policy_version 1705005 (0.0006) [2023-12-27 03:47:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 872120320. Throughput: 0: 9896.0, 1: 9791.9. Samples: 872114460. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:47:21,062][104569] Avg episode reward: [(0, '8712.107'), (1, '9079.691')] [2023-12-27 03:47:21,183][105620] Updated weights for policy 1, policy_version 1705015 (0.0008) [2023-12-27 03:47:21,208][105692] Updated weights for policy 0, policy_version 1701221 (0.0007) [2023-12-27 03:47:21,247][105620] Updated weights for policy 1, policy_version 1705025 (0.0006) [2023-12-27 03:47:21,273][105692] Updated weights for policy 0, policy_version 1701231 (0.0008) [2023-12-27 03:47:21,313][105620] Updated weights for policy 1, policy_version 1705035 (0.0008) [2023-12-27 03:47:21,333][105692] Updated weights for policy 0, policy_version 1701241 (0.0007) [2023-12-27 03:47:22,034][105620] Updated weights for policy 1, policy_version 1705045 (0.0008) [2023-12-27 03:47:22,034][105692] Updated weights for policy 0, policy_version 1701251 (0.0008) [2023-12-27 03:47:22,069][105586] KL-divergence is very high: 123.3064 [2023-12-27 03:47:22,096][105692] Updated weights for policy 0, policy_version 1701261 (0.0008) [2023-12-27 03:47:22,101][105620] Updated weights for policy 1, policy_version 1705055 (0.0008) [2023-12-27 03:47:22,121][105586] KL-divergence is very high: 237.6417 [2023-12-27 03:47:22,146][105692] Updated weights for policy 0, policy_version 1701271 (0.0008) [2023-12-27 03:47:22,166][105620] Updated weights for policy 1, policy_version 1705065 (0.0007) [2023-12-27 03:47:22,170][105586] KL-divergence is very high: 288.5091 [2023-12-27 03:47:22,865][105620] Updated weights for policy 1, policy_version 1705075 (0.0009) [2023-12-27 03:47:22,930][105620] Updated weights for policy 1, policy_version 1705085 (0.0008) [2023-12-27 03:47:22,981][105692] Updated weights for policy 0, policy_version 1701281 (0.0010) [2023-12-27 03:47:22,995][105620] Updated weights for policy 1, policy_version 1705095 (0.0006) [2023-12-27 03:47:23,039][105692] Updated weights for policy 0, policy_version 1701291 (0.0009) [2023-12-27 03:47:23,101][105692] Updated weights for policy 0, policy_version 1701301 (0.0008) [2023-12-27 03:47:23,169][105692] Updated weights for policy 0, policy_version 1701311 (0.0007) [2023-12-27 03:47:23,634][105620] Updated weights for policy 1, policy_version 1705105 (0.0007) [2023-12-27 03:47:23,694][105620] Updated weights for policy 1, policy_version 1705115 (0.0009) [2023-12-27 03:47:23,750][105620] Updated weights for policy 1, policy_version 1705125 (0.0009) [2023-12-27 03:47:23,794][105692] Updated weights for policy 0, policy_version 1701321 (0.0005) [2023-12-27 03:47:23,806][105620] Updated weights for policy 1, policy_version 1705135 (0.0008) [2023-12-27 03:47:23,840][105692] Updated weights for policy 0, policy_version 1701331 (0.0006) [2023-12-27 03:47:23,886][105692] Updated weights for policy 0, policy_version 1701341 (0.0005) [2023-12-27 03:47:24,444][105692] Updated weights for policy 0, policy_version 1701351 (0.0008) [2023-12-27 03:47:24,495][105692] Updated weights for policy 0, policy_version 1701361 (0.0009) [2023-12-27 03:47:24,546][105692] Updated weights for policy 0, policy_version 1701371 (0.0009) [2023-12-27 03:47:24,654][105620] Updated weights for policy 1, policy_version 1705145 (0.0006) [2023-12-27 03:47:24,715][105620] Updated weights for policy 1, policy_version 1705155 (0.0007) [2023-12-27 03:47:24,769][105620] Updated weights for policy 1, policy_version 1705165 (0.0005) [2023-12-27 03:47:25,271][105692] Updated weights for policy 0, policy_version 1701381 (0.0010) [2023-12-27 03:47:25,320][105692] Updated weights for policy 0, policy_version 1701391 (0.0010) [2023-12-27 03:47:25,330][105620] Updated weights for policy 1, policy_version 1705175 (0.0005) [2023-12-27 03:47:25,375][105692] Updated weights for policy 0, policy_version 1701401 (0.0010) [2023-12-27 03:47:25,389][105620] Updated weights for policy 1, policy_version 1705185 (0.0005) [2023-12-27 03:47:25,449][105620] Updated weights for policy 1, policy_version 1705195 (0.0007) [2023-12-27 03:47:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 872218624. Throughput: 0: 9912.5, 1: 9829.8. Samples: 872231892. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:47:26,062][104569] Avg episode reward: [(0, '8533.005'), (1, '9081.923')] [2023-12-27 03:47:26,112][105692] Updated weights for policy 0, policy_version 1701411 (0.0010) [2023-12-27 03:47:26,175][105620] Updated weights for policy 1, policy_version 1705205 (0.0007) [2023-12-27 03:47:26,176][105692] Updated weights for policy 0, policy_version 1701421 (0.0006) [2023-12-27 03:47:26,231][105620] Updated weights for policy 1, policy_version 1705215 (0.0005) [2023-12-27 03:47:26,241][105692] Updated weights for policy 0, policy_version 1701431 (0.0006) [2023-12-27 03:47:26,294][105620] Updated weights for policy 1, policy_version 1705225 (0.0005) [2023-12-27 03:47:26,881][105620] Updated weights for policy 1, policy_version 1705235 (0.0005) [2023-12-27 03:47:26,917][105692] Updated weights for policy 0, policy_version 1701441 (0.0008) [2023-12-27 03:47:26,937][105620] Updated weights for policy 1, policy_version 1705245 (0.0005) [2023-12-27 03:47:26,968][105692] Updated weights for policy 0, policy_version 1701451 (0.0006) [2023-12-27 03:47:26,988][105620] Updated weights for policy 1, policy_version 1705255 (0.0005) [2023-12-27 03:47:27,015][105692] Updated weights for policy 0, policy_version 1701461 (0.0005) [2023-12-27 03:47:27,070][105692] Updated weights for policy 0, policy_version 1701471 (0.0005) [2023-12-27 03:47:27,658][105692] Updated weights for policy 0, policy_version 1701481 (0.0005) [2023-12-27 03:47:27,696][105620] Updated weights for policy 1, policy_version 1705265 (0.0005) [2023-12-27 03:47:27,708][105692] Updated weights for policy 0, policy_version 1701491 (0.0005) [2023-12-27 03:47:27,752][105620] Updated weights for policy 1, policy_version 1705275 (0.0008) [2023-12-27 03:47:27,764][105692] Updated weights for policy 0, policy_version 1701501 (0.0005) [2023-12-27 03:47:27,802][105620] Updated weights for policy 1, policy_version 1705285 (0.0007) [2023-12-27 03:47:27,846][105620] Updated weights for policy 1, policy_version 1705295 (0.0008) [2023-12-27 03:47:28,451][105692] Updated weights for policy 0, policy_version 1701511 (0.0009) [2023-12-27 03:47:28,505][105692] Updated weights for policy 0, policy_version 1701521 (0.0010) [2023-12-27 03:47:28,568][105692] Updated weights for policy 0, policy_version 1701531 (0.0010) [2023-12-27 03:47:28,610][105620] Updated weights for policy 1, policy_version 1705305 (0.0006) [2023-12-27 03:47:28,664][105620] Updated weights for policy 1, policy_version 1705315 (0.0008) [2023-12-27 03:47:28,720][105620] Updated weights for policy 1, policy_version 1705325 (0.0008) [2023-12-27 03:47:29,303][105692] Updated weights for policy 0, policy_version 1701541 (0.0010) [2023-12-27 03:47:29,370][105692] Updated weights for policy 0, policy_version 1701551 (0.0009) [2023-12-27 03:47:29,414][105620] Updated weights for policy 1, policy_version 1705335 (0.0010) [2023-12-27 03:47:29,426][105692] Updated weights for policy 0, policy_version 1701561 (0.0011) [2023-12-27 03:47:29,476][105620] Updated weights for policy 1, policy_version 1705345 (0.0007) [2023-12-27 03:47:29,527][105620] Updated weights for policy 1, policy_version 1705355 (0.0008) [2023-12-27 03:47:30,061][105692] Updated weights for policy 0, policy_version 1701571 (0.0010) [2023-12-27 03:47:30,134][105692] Updated weights for policy 0, policy_version 1701581 (0.0011) [2023-12-27 03:47:30,203][105692] Updated weights for policy 0, policy_version 1701591 (0.0011) [2023-12-27 03:47:30,311][105620] Updated weights for policy 1, policy_version 1705365 (0.0008) [2023-12-27 03:47:30,370][105620] Updated weights for policy 1, policy_version 1705375 (0.0008) [2023-12-27 03:47:30,422][105620] Updated weights for policy 1, policy_version 1705385 (0.0008) [2023-12-27 03:47:30,922][105692] Updated weights for policy 0, policy_version 1701601 (0.0011) [2023-12-27 03:47:30,982][105692] Updated weights for policy 0, policy_version 1701611 (0.0010) [2023-12-27 03:47:31,006][105620] Updated weights for policy 1, policy_version 1705395 (0.0007) [2023-12-27 03:47:31,044][105692] Updated weights for policy 0, policy_version 1701621 (0.0012) [2023-12-27 03:47:31,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 872316928. Throughput: 0: 9946.8, 1: 9907.9. Samples: 872293848. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:47:31,063][104569] Avg episode reward: [(0, '8258.572'), (1, '9080.898')] [2023-12-27 03:47:31,066][105620] Updated weights for policy 1, policy_version 1705405 (0.0010) [2023-12-27 03:47:31,108][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001701632_435683328.pth... [2023-12-27 03:47:31,110][105692] Updated weights for policy 0, policy_version 1701632 (0.0007) [2023-12-27 03:47:31,112][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001700480_435388416.pth [2023-12-27 03:47:31,129][105620] Updated weights for policy 1, policy_version 1705415 (0.0011) [2023-12-27 03:47:31,185][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001705424_436649984.pth... [2023-12-27 03:47:31,189][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001704240_436346880.pth [2023-12-27 03:47:31,773][105620] Updated weights for policy 1, policy_version 1705425 (0.0010) [2023-12-27 03:47:31,834][105620] Updated weights for policy 1, policy_version 1705435 (0.0010) [2023-12-27 03:47:31,886][105620] Updated weights for policy 1, policy_version 1705445 (0.0010) [2023-12-27 03:47:31,898][105692] Updated weights for policy 0, policy_version 1701642 (0.0011) [2023-12-27 03:47:31,934][105620] Updated weights for policy 1, policy_version 1705455 (0.0011) [2023-12-27 03:47:31,953][105692] Updated weights for policy 0, policy_version 1701652 (0.0010) [2023-12-27 03:47:32,015][105692] Updated weights for policy 0, policy_version 1701662 (0.0010) [2023-12-27 03:47:32,557][105620] Updated weights for policy 1, policy_version 1705465 (0.0006) [2023-12-27 03:47:32,605][105620] Updated weights for policy 1, policy_version 1705475 (0.0005) [2023-12-27 03:47:32,653][105620] Updated weights for policy 1, policy_version 1705485 (0.0005) [2023-12-27 03:47:32,774][105692] Updated weights for policy 0, policy_version 1701672 (0.0010) [2023-12-27 03:47:32,831][105692] Updated weights for policy 0, policy_version 1701682 (0.0010) [2023-12-27 03:47:32,888][105692] Updated weights for policy 0, policy_version 1701692 (0.0010) [2023-12-27 03:47:33,332][105620] Updated weights for policy 1, policy_version 1705495 (0.0007) [2023-12-27 03:47:33,393][105620] Updated weights for policy 1, policy_version 1705505 (0.0006) [2023-12-27 03:47:33,456][105620] Updated weights for policy 1, policy_version 1705515 (0.0007) [2023-12-27 03:47:33,570][105692] Updated weights for policy 0, policy_version 1701702 (0.0010) [2023-12-27 03:47:33,628][105692] Updated weights for policy 0, policy_version 1701712 (0.0010) [2023-12-27 03:47:33,689][105692] Updated weights for policy 0, policy_version 1701722 (0.0010) [2023-12-27 03:47:34,103][105620] Updated weights for policy 1, policy_version 1705525 (0.0008) [2023-12-27 03:47:34,157][105620] Updated weights for policy 1, policy_version 1705535 (0.0010) [2023-12-27 03:47:34,212][105620] Updated weights for policy 1, policy_version 1705545 (0.0010) [2023-12-27 03:47:34,254][105692] Updated weights for policy 0, policy_version 1701732 (0.0006) [2023-12-27 03:47:34,319][105692] Updated weights for policy 0, policy_version 1701742 (0.0006) [2023-12-27 03:47:34,387][105692] Updated weights for policy 0, policy_version 1701752 (0.0005) [2023-12-27 03:47:34,915][105620] Updated weights for policy 1, policy_version 1705555 (0.0009) [2023-12-27 03:47:34,984][105692] Updated weights for policy 0, policy_version 1701762 (0.0007) [2023-12-27 03:47:34,988][105620] Updated weights for policy 1, policy_version 1705565 (0.0006) [2023-12-27 03:47:35,038][105620] Updated weights for policy 1, policy_version 1705575 (0.0006) [2023-12-27 03:47:35,047][105692] Updated weights for policy 0, policy_version 1701772 (0.0011) [2023-12-27 03:47:35,106][105692] Updated weights for policy 0, policy_version 1701782 (0.0005) [2023-12-27 03:47:35,154][105692] Updated weights for policy 0, policy_version 1701792 (0.0006) [2023-12-27 03:47:35,665][105620] Updated weights for policy 1, policy_version 1705585 (0.0006) [2023-12-27 03:47:35,723][105692] Updated weights for policy 0, policy_version 1701802 (0.0007) [2023-12-27 03:47:35,736][105620] Updated weights for policy 1, policy_version 1705595 (0.0006) [2023-12-27 03:47:35,781][105692] Updated weights for policy 0, policy_version 1701812 (0.0010) [2023-12-27 03:47:35,800][105620] Updated weights for policy 1, policy_version 1705605 (0.0008) [2023-12-27 03:47:35,836][105692] Updated weights for policy 0, policy_version 1701822 (0.0010) [2023-12-27 03:47:35,861][105620] Updated weights for policy 1, policy_version 1705615 (0.0008) [2023-12-27 03:47:36,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19933.9, 300 sec: 19549.7). Total num frames: 872431616. Throughput: 0: 9956.6, 1: 9959.4. Samples: 872415492. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:47:36,062][104569] Avg episode reward: [(0, '8350.244'), (1, '9079.119')] [2023-12-27 03:47:36,491][105620] Updated weights for policy 1, policy_version 1705625 (0.0008) [2023-12-27 03:47:36,562][105620] Updated weights for policy 1, policy_version 1705635 (0.0009) [2023-12-27 03:47:36,596][105692] Updated weights for policy 0, policy_version 1701832 (0.0011) [2023-12-27 03:47:36,620][105620] Updated weights for policy 1, policy_version 1705645 (0.0011) [2023-12-27 03:47:36,656][105692] Updated weights for policy 0, policy_version 1701842 (0.0011) [2023-12-27 03:47:36,708][105692] Updated weights for policy 0, policy_version 1701852 (0.0010) [2023-12-27 03:47:37,264][105620] Updated weights for policy 1, policy_version 1705655 (0.0007) [2023-12-27 03:47:37,321][105620] Updated weights for policy 1, policy_version 1705665 (0.0005) [2023-12-27 03:47:37,325][105692] Updated weights for policy 0, policy_version 1701862 (0.0009) [2023-12-27 03:47:37,373][105620] Updated weights for policy 1, policy_version 1705675 (0.0008) [2023-12-27 03:47:37,381][105692] Updated weights for policy 0, policy_version 1701872 (0.0010) [2023-12-27 03:47:37,450][105692] Updated weights for policy 0, policy_version 1701882 (0.0011) [2023-12-27 03:47:37,931][105620] Updated weights for policy 1, policy_version 1705685 (0.0008) [2023-12-27 03:47:37,982][105620] Updated weights for policy 1, policy_version 1705695 (0.0005) [2023-12-27 03:47:38,037][105620] Updated weights for policy 1, policy_version 1705705 (0.0005) [2023-12-27 03:47:38,193][105692] Updated weights for policy 0, policy_version 1701892 (0.0010) [2023-12-27 03:47:38,252][105692] Updated weights for policy 0, policy_version 1701902 (0.0010) [2023-12-27 03:47:38,304][105692] Updated weights for policy 0, policy_version 1701912 (0.0009) [2023-12-27 03:47:38,767][105620] Updated weights for policy 1, policy_version 1705715 (0.0006) [2023-12-27 03:47:38,824][105620] Updated weights for policy 1, policy_version 1705725 (0.0009) [2023-12-27 03:47:38,889][105620] Updated weights for policy 1, policy_version 1705735 (0.0009) [2023-12-27 03:47:39,148][105692] Updated weights for policy 0, policy_version 1701922 (0.0008) [2023-12-27 03:47:39,213][105692] Updated weights for policy 0, policy_version 1701932 (0.0006) [2023-12-27 03:47:39,272][105692] Updated weights for policy 0, policy_version 1701942 (0.0006) [2023-12-27 03:47:39,333][105692] Updated weights for policy 0, policy_version 1701952 (0.0007) [2023-12-27 03:47:39,649][105620] Updated weights for policy 1, policy_version 1705745 (0.0009) [2023-12-27 03:47:39,702][105620] Updated weights for policy 1, policy_version 1705755 (0.0011) [2023-12-27 03:47:39,769][105620] Updated weights for policy 1, policy_version 1705765 (0.0011) [2023-12-27 03:47:39,836][105620] Updated weights for policy 1, policy_version 1705775 (0.0011) [2023-12-27 03:47:39,956][105692] Updated weights for policy 0, policy_version 1701962 (0.0007) [2023-12-27 03:47:40,020][105692] Updated weights for policy 0, policy_version 1701972 (0.0010) [2023-12-27 03:47:40,078][105692] Updated weights for policy 0, policy_version 1701982 (0.0008) [2023-12-27 03:47:40,624][105620] Updated weights for policy 1, policy_version 1705785 (0.0011) [2023-12-27 03:47:40,687][105620] Updated weights for policy 1, policy_version 1705795 (0.0011) [2023-12-27 03:47:40,736][105620] Updated weights for policy 1, policy_version 1705805 (0.0008) [2023-12-27 03:47:40,854][105692] Updated weights for policy 0, policy_version 1701992 (0.0010) [2023-12-27 03:47:40,915][105692] Updated weights for policy 0, policy_version 1702002 (0.0010) [2023-12-27 03:47:40,975][105692] Updated weights for policy 0, policy_version 1702012 (0.0007) [2023-12-27 03:47:41,062][104569] Fps is (10 sec: 21299.7, 60 sec: 19933.9, 300 sec: 19549.7). Total num frames: 872529920. Throughput: 0: 9891.8, 1: 10035.4. Samples: 872535504. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:47:41,062][104569] Avg episode reward: [(0, '8623.422'), (1, '8987.154')] [2023-12-27 03:47:41,461][105620] Updated weights for policy 1, policy_version 1705815 (0.0009) [2023-12-27 03:47:41,511][105620] Updated weights for policy 1, policy_version 1705825 (0.0011) [2023-12-27 03:47:41,559][105620] Updated weights for policy 1, policy_version 1705835 (0.0008) [2023-12-27 03:47:41,779][105692] Updated weights for policy 0, policy_version 1702022 (0.0009) [2023-12-27 03:47:41,844][105692] Updated weights for policy 0, policy_version 1702032 (0.0008) [2023-12-27 03:47:41,910][105692] Updated weights for policy 0, policy_version 1702042 (0.0008) [2023-12-27 03:47:42,405][105620] Updated weights for policy 1, policy_version 1705845 (0.0009) [2023-12-27 03:47:42,469][105620] Updated weights for policy 1, policy_version 1705855 (0.0009) [2023-12-27 03:47:42,537][105620] Updated weights for policy 1, policy_version 1705865 (0.0007) [2023-12-27 03:47:42,621][105692] Updated weights for policy 0, policy_version 1702052 (0.0009) [2023-12-27 03:47:42,680][105692] Updated weights for policy 0, policy_version 1702062 (0.0010) [2023-12-27 03:47:42,743][105692] Updated weights for policy 0, policy_version 1702072 (0.0011) [2023-12-27 03:47:43,167][105620] Updated weights for policy 1, policy_version 1705875 (0.0010) [2023-12-27 03:47:43,226][105620] Updated weights for policy 1, policy_version 1705885 (0.0011) [2023-12-27 03:47:43,296][105620] Updated weights for policy 1, policy_version 1705895 (0.0011) [2023-12-27 03:47:43,446][105692] Updated weights for policy 0, policy_version 1702082 (0.0010) [2023-12-27 03:47:43,495][105692] Updated weights for policy 0, policy_version 1702092 (0.0005) [2023-12-27 03:47:43,554][105692] Updated weights for policy 0, policy_version 1702102 (0.0006) [2023-12-27 03:47:43,620][105692] Updated weights for policy 0, policy_version 1702112 (0.0011) [2023-12-27 03:47:43,959][105620] Updated weights for policy 1, policy_version 1705905 (0.0011) [2023-12-27 03:47:44,008][105620] Updated weights for policy 1, policy_version 1705915 (0.0011) [2023-12-27 03:47:44,060][105620] Updated weights for policy 1, policy_version 1705925 (0.0010) [2023-12-27 03:47:44,120][105620] Updated weights for policy 1, policy_version 1705935 (0.0010) [2023-12-27 03:47:44,260][105692] Updated weights for policy 0, policy_version 1702122 (0.0011) [2023-12-27 03:47:44,307][105692] Updated weights for policy 0, policy_version 1702132 (0.0010) [2023-12-27 03:47:44,362][105692] Updated weights for policy 0, policy_version 1702142 (0.0010) [2023-12-27 03:47:44,809][105620] Updated weights for policy 1, policy_version 1705945 (0.0009) [2023-12-27 03:47:44,861][105620] Updated weights for policy 1, policy_version 1705955 (0.0008) [2023-12-27 03:47:44,917][105620] Updated weights for policy 1, policy_version 1705965 (0.0008) [2023-12-27 03:47:45,115][105692] Updated weights for policy 0, policy_version 1702152 (0.0011) [2023-12-27 03:47:45,179][105692] Updated weights for policy 0, policy_version 1702162 (0.0011) [2023-12-27 03:47:45,248][105692] Updated weights for policy 0, policy_version 1702172 (0.0009) [2023-12-27 03:47:45,688][105620] Updated weights for policy 1, policy_version 1705975 (0.0007) [2023-12-27 03:47:45,747][105620] Updated weights for policy 1, policy_version 1705985 (0.0007) [2023-12-27 03:47:45,808][105620] Updated weights for policy 1, policy_version 1705995 (0.0009) [2023-12-27 03:47:45,859][105692] Updated weights for policy 0, policy_version 1702182 (0.0008) [2023-12-27 03:47:45,916][105692] Updated weights for policy 0, policy_version 1702192 (0.0010) [2023-12-27 03:47:45,975][105692] Updated weights for policy 0, policy_version 1702202 (0.0010) [2023-12-27 03:47:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19933.9, 300 sec: 19549.7). Total num frames: 872628224. Throughput: 0: 9801.6, 1: 9965.7. Samples: 872592856. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:47:46,063][104569] Avg episode reward: [(0, '8620.354'), (1, '9170.939')] [2023-12-27 03:47:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001702208_435830784.pth... [2023-12-27 03:47:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001706000_436797440.pth... [2023-12-27 03:47:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001701056_435535872.pth [2023-12-27 03:47:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001704816_436494336.pth [2023-12-27 03:47:46,386][105620] Updated weights for policy 1, policy_version 1706005 (0.0007) [2023-12-27 03:47:46,457][105620] Updated weights for policy 1, policy_version 1706015 (0.0008) [2023-12-27 03:47:46,525][105620] Updated weights for policy 1, policy_version 1706025 (0.0006) [2023-12-27 03:47:46,636][105692] Updated weights for policy 0, policy_version 1702212 (0.0009) [2023-12-27 03:47:46,690][105692] Updated weights for policy 0, policy_version 1702222 (0.0010) [2023-12-27 03:47:46,744][105692] Updated weights for policy 0, policy_version 1702234 (0.0011) [2023-12-27 03:47:47,105][105620] Updated weights for policy 1, policy_version 1706035 (0.0007) [2023-12-27 03:47:47,162][105620] Updated weights for policy 1, policy_version 1706045 (0.0005) [2023-12-27 03:47:47,221][105620] Updated weights for policy 1, policy_version 1706055 (0.0010) [2023-12-27 03:47:47,386][105692] Updated weights for policy 0, policy_version 1702245 (0.0009) [2023-12-27 03:47:47,442][105692] Updated weights for policy 0, policy_version 1702256 (0.0012) [2023-12-27 03:47:47,498][105692] Updated weights for policy 0, policy_version 1702266 (0.0010) [2023-12-27 03:47:47,774][105620] Updated weights for policy 1, policy_version 1706065 (0.0010) [2023-12-27 03:47:47,827][105620] Updated weights for policy 1, policy_version 1706075 (0.0006) [2023-12-27 03:47:47,885][105620] Updated weights for policy 1, policy_version 1706085 (0.0008) [2023-12-27 03:47:47,936][105620] Updated weights for policy 1, policy_version 1706095 (0.0005) [2023-12-27 03:47:48,373][105692] Updated weights for policy 0, policy_version 1702276 (0.0009) [2023-12-27 03:47:48,438][105692] Updated weights for policy 0, policy_version 1702286 (0.0007) [2023-12-27 03:47:48,506][105692] Updated weights for policy 0, policy_version 1702296 (0.0008) [2023-12-27 03:47:48,631][105620] Updated weights for policy 1, policy_version 1706105 (0.0006) [2023-12-27 03:47:48,687][105620] Updated weights for policy 1, policy_version 1706115 (0.0011) [2023-12-27 03:47:48,738][105620] Updated weights for policy 1, policy_version 1706125 (0.0010) [2023-12-27 03:47:49,170][105692] Updated weights for policy 0, policy_version 1702306 (0.0007) [2023-12-27 03:47:49,237][105692] Updated weights for policy 0, policy_version 1702316 (0.0009) [2023-12-27 03:47:49,303][105692] Updated weights for policy 0, policy_version 1702326 (0.0010) [2023-12-27 03:47:49,370][105692] Updated weights for policy 0, policy_version 1702336 (0.0011) [2023-12-27 03:47:49,496][105620] Updated weights for policy 1, policy_version 1706135 (0.0011) [2023-12-27 03:47:49,545][105620] Updated weights for policy 1, policy_version 1706145 (0.0010) [2023-12-27 03:47:49,593][105620] Updated weights for policy 1, policy_version 1706155 (0.0010) [2023-12-27 03:47:50,112][105692] Updated weights for policy 0, policy_version 1702346 (0.0011) [2023-12-27 03:47:50,180][105692] Updated weights for policy 0, policy_version 1702356 (0.0011) [2023-12-27 03:47:50,242][105692] Updated weights for policy 0, policy_version 1702366 (0.0010) [2023-12-27 03:47:50,358][105620] Updated weights for policy 1, policy_version 1706165 (0.0008) [2023-12-27 03:47:50,415][105620] Updated weights for policy 1, policy_version 1706175 (0.0009) [2023-12-27 03:47:50,467][105620] Updated weights for policy 1, policy_version 1706185 (0.0010) [2023-12-27 03:47:50,963][105692] Updated weights for policy 0, policy_version 1702376 (0.0011) [2023-12-27 03:47:51,022][105692] Updated weights for policy 0, policy_version 1702386 (0.0011) [2023-12-27 03:47:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 872718336. Throughput: 0: 9812.9, 1: 10067.5. Samples: 872714376. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:47:51,063][104569] Avg episode reward: [(0, '8532.031'), (1, '9170.494')] [2023-12-27 03:47:51,085][105692] Updated weights for policy 0, policy_version 1702396 (0.0011) [2023-12-27 03:47:51,133][105620] Updated weights for policy 1, policy_version 1706195 (0.0010) [2023-12-27 03:47:51,194][105620] Updated weights for policy 1, policy_version 1706205 (0.0011) [2023-12-27 03:47:51,264][105620] Updated weights for policy 1, policy_version 1706215 (0.0011) [2023-12-27 03:47:51,809][105692] Updated weights for policy 0, policy_version 1702406 (0.0011) [2023-12-27 03:47:51,858][105692] Updated weights for policy 0, policy_version 1702416 (0.0011) [2023-12-27 03:47:51,904][105692] Updated weights for policy 0, policy_version 1702426 (0.0010) [2023-12-27 03:47:51,948][105620] Updated weights for policy 1, policy_version 1706225 (0.0010) [2023-12-27 03:47:52,014][105620] Updated weights for policy 1, policy_version 1706235 (0.0009) [2023-12-27 03:47:52,079][105620] Updated weights for policy 1, policy_version 1706245 (0.0011) [2023-12-27 03:47:52,146][105620] Updated weights for policy 1, policy_version 1706255 (0.0009) [2023-12-27 03:47:52,706][105692] Updated weights for policy 0, policy_version 1702436 (0.0010) [2023-12-27 03:47:52,763][105692] Updated weights for policy 0, policy_version 1702446 (0.0011) [2023-12-27 03:47:52,821][105620] Updated weights for policy 1, policy_version 1706265 (0.0010) [2023-12-27 03:47:52,825][105692] Updated weights for policy 0, policy_version 1702456 (0.0009) [2023-12-27 03:47:52,870][105620] Updated weights for policy 1, policy_version 1706275 (0.0011) [2023-12-27 03:47:52,929][105620] Updated weights for policy 1, policy_version 1706286 (0.0010) [2023-12-27 03:47:53,489][105692] Updated weights for policy 0, policy_version 1702466 (0.0006) [2023-12-27 03:47:53,542][105692] Updated weights for policy 0, policy_version 1702476 (0.0010) [2023-12-27 03:47:53,599][105692] Updated weights for policy 0, policy_version 1702486 (0.0006) [2023-12-27 03:47:53,653][105620] Updated weights for policy 1, policy_version 1706296 (0.0009) [2023-12-27 03:47:53,658][105692] Updated weights for policy 0, policy_version 1702496 (0.0005) [2023-12-27 03:47:53,718][105620] Updated weights for policy 1, policy_version 1706306 (0.0010) [2023-12-27 03:47:53,772][105620] Updated weights for policy 1, policy_version 1706316 (0.0010) [2023-12-27 03:47:54,278][105692] Updated weights for policy 0, policy_version 1702506 (0.0010) [2023-12-27 03:47:54,341][105692] Updated weights for policy 0, policy_version 1702516 (0.0011) [2023-12-27 03:47:54,395][105692] Updated weights for policy 0, policy_version 1702526 (0.0010) [2023-12-27 03:47:54,442][105620] Updated weights for policy 1, policy_version 1706326 (0.0010) [2023-12-27 03:47:54,515][105620] Updated weights for policy 1, policy_version 1706336 (0.0010) [2023-12-27 03:47:54,563][105620] Updated weights for policy 1, policy_version 1706346 (0.0010) [2023-12-27 03:47:55,006][105692] Updated weights for policy 0, policy_version 1702536 (0.0007) [2023-12-27 03:47:55,067][105692] Updated weights for policy 0, policy_version 1702546 (0.0008) [2023-12-27 03:47:55,130][105692] Updated weights for policy 0, policy_version 1702557 (0.0006) [2023-12-27 03:47:55,306][105620] Updated weights for policy 1, policy_version 1706356 (0.0010) [2023-12-27 03:47:55,367][105620] Updated weights for policy 1, policy_version 1706366 (0.0010) [2023-12-27 03:47:55,422][105620] Updated weights for policy 1, policy_version 1706376 (0.0010) [2023-12-27 03:47:55,668][105692] Updated weights for policy 0, policy_version 1702567 (0.0005) [2023-12-27 03:47:55,715][105692] Updated weights for policy 0, policy_version 1702577 (0.0005) [2023-12-27 03:47:55,770][105692] Updated weights for policy 0, policy_version 1702587 (0.0005) [2023-12-27 03:47:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 872824832. Throughput: 0: 9844.7, 1: 10014.9. Samples: 872834488. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:47:56,062][104569] Avg episode reward: [(0, '8350.771'), (1, '8988.530')] [2023-12-27 03:47:56,146][105620] Updated weights for policy 1, policy_version 1706386 (0.0009) [2023-12-27 03:47:56,200][105620] Updated weights for policy 1, policy_version 1706396 (0.0005) [2023-12-27 03:47:56,258][105620] Updated weights for policy 1, policy_version 1706406 (0.0005) [2023-12-27 03:47:56,318][105620] Updated weights for policy 1, policy_version 1706416 (0.0010) [2023-12-27 03:47:56,427][105692] Updated weights for policy 0, policy_version 1702597 (0.0007) [2023-12-27 03:47:56,485][105692] Updated weights for policy 0, policy_version 1702607 (0.0010) [2023-12-27 03:47:56,538][105692] Updated weights for policy 0, policy_version 1702617 (0.0009) [2023-12-27 03:47:56,888][105620] Updated weights for policy 1, policy_version 1706426 (0.0010) [2023-12-27 03:47:56,942][105620] Updated weights for policy 1, policy_version 1706436 (0.0010) [2023-12-27 03:47:56,989][105620] Updated weights for policy 1, policy_version 1706446 (0.0010) [2023-12-27 03:47:57,280][105692] Updated weights for policy 0, policy_version 1702627 (0.0008) [2023-12-27 03:47:57,342][105692] Updated weights for policy 0, policy_version 1702637 (0.0006) [2023-12-27 03:47:57,397][105692] Updated weights for policy 0, policy_version 1702647 (0.0005) [2023-12-27 03:47:57,700][105620] Updated weights for policy 1, policy_version 1706456 (0.0010) [2023-12-27 03:47:57,748][105620] Updated weights for policy 1, policy_version 1706466 (0.0010) [2023-12-27 03:47:57,805][105620] Updated weights for policy 1, policy_version 1706476 (0.0009) [2023-12-27 03:47:57,971][105692] Updated weights for policy 0, policy_version 1702657 (0.0006) [2023-12-27 03:47:58,032][105692] Updated weights for policy 0, policy_version 1702667 (0.0010) [2023-12-27 03:47:58,089][105692] Updated weights for policy 0, policy_version 1702678 (0.0010) [2023-12-27 03:47:58,139][105692] Updated weights for policy 0, policy_version 1702688 (0.0009) [2023-12-27 03:47:58,535][105620] Updated weights for policy 1, policy_version 1706486 (0.0009) [2023-12-27 03:47:58,603][105620] Updated weights for policy 1, policy_version 1706496 (0.0009) [2023-12-27 03:47:58,683][105620] Updated weights for policy 1, policy_version 1706506 (0.0008) [2023-12-27 03:47:58,965][105692] Updated weights for policy 0, policy_version 1702698 (0.0008) [2023-12-27 03:47:59,030][105692] Updated weights for policy 0, policy_version 1702708 (0.0008) [2023-12-27 03:47:59,092][105692] Updated weights for policy 0, policy_version 1702718 (0.0008) [2023-12-27 03:47:59,405][105620] Updated weights for policy 1, policy_version 1706516 (0.0009) [2023-12-27 03:47:59,456][105620] Updated weights for policy 1, policy_version 1706527 (0.0008) [2023-12-27 03:47:59,510][105620] Updated weights for policy 1, policy_version 1706537 (0.0008) [2023-12-27 03:47:59,860][105692] Updated weights for policy 0, policy_version 1702728 (0.0008) [2023-12-27 03:47:59,928][105692] Updated weights for policy 0, policy_version 1702738 (0.0008) [2023-12-27 03:47:59,986][105692] Updated weights for policy 0, policy_version 1702748 (0.0008) [2023-12-27 03:48:00,180][105620] Updated weights for policy 1, policy_version 1706547 (0.0008) [2023-12-27 03:48:00,228][105620] Updated weights for policy 1, policy_version 1706557 (0.0008) [2023-12-27 03:48:00,273][105620] Updated weights for policy 1, policy_version 1706567 (0.0009) [2023-12-27 03:48:00,732][105692] Updated weights for policy 0, policy_version 1702758 (0.0008) [2023-12-27 03:48:00,779][105692] Updated weights for policy 0, policy_version 1702768 (0.0009) [2023-12-27 03:48:00,823][105692] Updated weights for policy 0, policy_version 1702778 (0.0008) [2023-12-27 03:48:00,983][105620] Updated weights for policy 1, policy_version 1706577 (0.0009) [2023-12-27 03:48:01,045][105620] Updated weights for policy 1, policy_version 1706587 (0.0010) [2023-12-27 03:48:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 872923136. Throughput: 0: 9913.9, 1: 10033.3. Samples: 872895000. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:48:01,062][104569] Avg episode reward: [(0, '8345.717'), (1, '9083.524')] [2023-12-27 03:48:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001702784_435978240.pth... [2023-12-27 03:48:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001701632_435683328.pth [2023-12-27 03:48:01,097][105620] Updated weights for policy 1, policy_version 1706597 (0.0009) [2023-12-27 03:48:01,157][105620] Updated weights for policy 1, policy_version 1706607 (0.0009) [2023-12-27 03:48:01,164][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001706608_436953088.pth... [2023-12-27 03:48:01,169][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001705424_436649984.pth [2023-12-27 03:48:01,662][105692] Updated weights for policy 0, policy_version 1702788 (0.0009) [2023-12-27 03:48:01,729][105692] Updated weights for policy 0, policy_version 1702798 (0.0009) [2023-12-27 03:48:01,795][105692] Updated weights for policy 0, policy_version 1702808 (0.0009) [2023-12-27 03:48:01,912][105620] Updated weights for policy 1, policy_version 1706617 (0.0006) [2023-12-27 03:48:01,958][105620] Updated weights for policy 1, policy_version 1706627 (0.0005) [2023-12-27 03:48:02,008][105620] Updated weights for policy 1, policy_version 1706637 (0.0005) [2023-12-27 03:48:02,528][105692] Updated weights for policy 0, policy_version 1702818 (0.0010) [2023-12-27 03:48:02,586][105692] Updated weights for policy 0, policy_version 1702828 (0.0010) [2023-12-27 03:48:02,636][105692] Updated weights for policy 0, policy_version 1702838 (0.0009) [2023-12-27 03:48:02,693][105692] Updated weights for policy 0, policy_version 1702848 (0.0007) [2023-12-27 03:48:02,706][105620] Updated weights for policy 1, policy_version 1706647 (0.0009) [2023-12-27 03:48:02,761][105620] Updated weights for policy 1, policy_version 1706657 (0.0010) [2023-12-27 03:48:02,828][105620] Updated weights for policy 1, policy_version 1706667 (0.0011) [2023-12-27 03:48:03,430][105692] Updated weights for policy 0, policy_version 1702858 (0.0008) [2023-12-27 03:48:03,481][105692] Updated weights for policy 0, policy_version 1702868 (0.0009) [2023-12-27 03:48:03,540][105692] Updated weights for policy 0, policy_version 1702878 (0.0009) [2023-12-27 03:48:03,558][105620] Updated weights for policy 1, policy_version 1706677 (0.0008) [2023-12-27 03:48:03,619][105620] Updated weights for policy 1, policy_version 1706687 (0.0009) [2023-12-27 03:48:03,689][105620] Updated weights for policy 1, policy_version 1706697 (0.0006) [2023-12-27 03:48:04,297][105692] Updated weights for policy 0, policy_version 1702888 (0.0006) [2023-12-27 03:48:04,352][105692] Updated weights for policy 0, policy_version 1702898 (0.0007) [2023-12-27 03:48:04,400][105620] Updated weights for policy 1, policy_version 1706707 (0.0008) [2023-12-27 03:48:04,414][105692] Updated weights for policy 0, policy_version 1702908 (0.0008) [2023-12-27 03:48:04,462][105620] Updated weights for policy 1, policy_version 1706717 (0.0008) [2023-12-27 03:48:04,520][105620] Updated weights for policy 1, policy_version 1706727 (0.0009) [2023-12-27 03:48:05,144][105692] Updated weights for policy 0, policy_version 1702918 (0.0008) [2023-12-27 03:48:05,201][105692] Updated weights for policy 0, policy_version 1702928 (0.0009) [2023-12-27 03:48:05,262][105692] Updated weights for policy 0, policy_version 1702938 (0.0009) [2023-12-27 03:48:05,265][105620] Updated weights for policy 1, policy_version 1706737 (0.0009) [2023-12-27 03:48:05,316][105620] Updated weights for policy 1, policy_version 1706747 (0.0007) [2023-12-27 03:48:05,362][105620] Updated weights for policy 1, policy_version 1706757 (0.0009) [2023-12-27 03:48:05,417][105620] Updated weights for policy 1, policy_version 1706767 (0.0009) [2023-12-27 03:48:05,993][105692] Updated weights for policy 0, policy_version 1702948 (0.0008) [2023-12-27 03:48:06,049][105692] Updated weights for policy 0, policy_version 1702958 (0.0009) [2023-12-27 03:48:06,059][105620] Updated weights for policy 1, policy_version 1706777 (0.0006) [2023-12-27 03:48:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 873013248. Throughput: 0: 9870.6, 1: 10002.8. Samples: 873008764. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:48:06,062][104569] Avg episode reward: [(0, '8710.564'), (1, '9083.628')] [2023-12-27 03:48:06,094][105692] Updated weights for policy 0, policy_version 1702968 (0.0008) [2023-12-27 03:48:06,114][105620] Updated weights for policy 1, policy_version 1706787 (0.0009) [2023-12-27 03:48:06,177][105620] Updated weights for policy 1, policy_version 1706797 (0.0009) [2023-12-27 03:48:06,818][105692] Updated weights for policy 0, policy_version 1702978 (0.0009) [2023-12-27 03:48:06,864][105692] Updated weights for policy 0, policy_version 1702988 (0.0008) [2023-12-27 03:48:06,911][105692] Updated weights for policy 0, policy_version 1702998 (0.0009) [2023-12-27 03:48:06,950][105620] Updated weights for policy 1, policy_version 1706807 (0.0008) [2023-12-27 03:48:06,974][105692] Updated weights for policy 0, policy_version 1703008 (0.0006) [2023-12-27 03:48:07,011][105620] Updated weights for policy 1, policy_version 1706817 (0.0008) [2023-12-27 03:48:07,063][105620] Updated weights for policy 1, policy_version 1706827 (0.0009) [2023-12-27 03:48:07,785][105692] Updated weights for policy 0, policy_version 1703018 (0.0008) [2023-12-27 03:48:07,794][105620] Updated weights for policy 1, policy_version 1706837 (0.0009) [2023-12-27 03:48:07,835][105692] Updated weights for policy 0, policy_version 1703028 (0.0007) [2023-12-27 03:48:07,852][105620] Updated weights for policy 1, policy_version 1706847 (0.0009) [2023-12-27 03:48:07,885][105692] Updated weights for policy 0, policy_version 1703038 (0.0008) [2023-12-27 03:48:07,912][105620] Updated weights for policy 1, policy_version 1706857 (0.0008) [2023-12-27 03:48:08,617][105620] Updated weights for policy 1, policy_version 1706867 (0.0009) [2023-12-27 03:48:08,677][105692] Updated weights for policy 0, policy_version 1703048 (0.0006) [2023-12-27 03:48:08,681][105620] Updated weights for policy 1, policy_version 1706877 (0.0008) [2023-12-27 03:48:08,727][105692] Updated weights for policy 0, policy_version 1703058 (0.0006) [2023-12-27 03:48:08,742][105620] Updated weights for policy 1, policy_version 1706887 (0.0008) [2023-12-27 03:48:08,785][105692] Updated weights for policy 0, policy_version 1703068 (0.0006) [2023-12-27 03:48:09,499][105620] Updated weights for policy 1, policy_version 1706897 (0.0009) [2023-12-27 03:48:09,550][105692] Updated weights for policy 0, policy_version 1703078 (0.0010) [2023-12-27 03:48:09,567][105620] Updated weights for policy 1, policy_version 1706907 (0.0005) [2023-12-27 03:48:09,602][105692] Updated weights for policy 0, policy_version 1703088 (0.0007) [2023-12-27 03:48:09,623][105620] Updated weights for policy 1, policy_version 1706917 (0.0007) [2023-12-27 03:48:09,650][105692] Updated weights for policy 0, policy_version 1703098 (0.0008) [2023-12-27 03:48:09,673][105620] Updated weights for policy 1, policy_version 1706927 (0.0007) [2023-12-27 03:48:10,350][105692] Updated weights for policy 0, policy_version 1703108 (0.0007) [2023-12-27 03:48:10,414][105692] Updated weights for policy 0, policy_version 1703118 (0.0009) [2023-12-27 03:48:10,469][105620] Updated weights for policy 1, policy_version 1706937 (0.0009) [2023-12-27 03:48:10,472][105692] Updated weights for policy 0, policy_version 1703128 (0.0007) [2023-12-27 03:48:10,523][105620] Updated weights for policy 1, policy_version 1706947 (0.0006) [2023-12-27 03:48:10,576][105620] Updated weights for policy 1, policy_version 1706957 (0.0007) [2023-12-27 03:48:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 873111552. Throughput: 0: 9794.2, 1: 9990.1. Samples: 873122188. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:48:11,062][104569] Avg episode reward: [(0, '8897.910'), (1, '8991.562')] [2023-12-27 03:48:11,285][105692] Updated weights for policy 0, policy_version 1703138 (0.0008) [2023-12-27 03:48:11,296][105620] Updated weights for policy 1, policy_version 1706967 (0.0009) [2023-12-27 03:48:11,350][105692] Updated weights for policy 0, policy_version 1703148 (0.0007) [2023-12-27 03:48:11,364][105620] Updated weights for policy 1, policy_version 1706977 (0.0008) [2023-12-27 03:48:11,419][105692] Updated weights for policy 0, policy_version 1703158 (0.0008) [2023-12-27 03:48:11,435][105620] Updated weights for policy 1, policy_version 1706987 (0.0006) [2023-12-27 03:48:11,478][105692] Updated weights for policy 0, policy_version 1703168 (0.0009) [2023-12-27 03:48:12,095][105620] Updated weights for policy 1, policy_version 1706997 (0.0006) [2023-12-27 03:48:12,159][105620] Updated weights for policy 1, policy_version 1707007 (0.0006) [2023-12-27 03:48:12,225][105620] Updated weights for policy 1, policy_version 1707017 (0.0006) [2023-12-27 03:48:12,302][105692] Updated weights for policy 0, policy_version 1703178 (0.0011) [2023-12-27 03:48:12,369][105692] Updated weights for policy 0, policy_version 1703188 (0.0010) [2023-12-27 03:48:12,436][105692] Updated weights for policy 0, policy_version 1703198 (0.0011) [2023-12-27 03:48:12,782][105620] Updated weights for policy 1, policy_version 1707027 (0.0007) [2023-12-27 03:48:12,841][105620] Updated weights for policy 1, policy_version 1707037 (0.0009) [2023-12-27 03:48:12,902][105620] Updated weights for policy 1, policy_version 1707047 (0.0008) [2023-12-27 03:48:13,114][105692] Updated weights for policy 0, policy_version 1703208 (0.0010) [2023-12-27 03:48:13,158][105692] Updated weights for policy 0, policy_version 1703218 (0.0010) [2023-12-27 03:48:13,209][105692] Updated weights for policy 0, policy_version 1703228 (0.0010) [2023-12-27 03:48:13,669][105620] Updated weights for policy 1, policy_version 1707057 (0.0008) [2023-12-27 03:48:13,727][105620] Updated weights for policy 1, policy_version 1707067 (0.0008) [2023-12-27 03:48:13,783][105620] Updated weights for policy 1, policy_version 1707077 (0.0008) [2023-12-27 03:48:13,842][105620] Updated weights for policy 1, policy_version 1707087 (0.0008) [2023-12-27 03:48:13,970][105692] Updated weights for policy 0, policy_version 1703238 (0.0010) [2023-12-27 03:48:14,027][105692] Updated weights for policy 0, policy_version 1703248 (0.0010) [2023-12-27 03:48:14,085][105692] Updated weights for policy 0, policy_version 1703258 (0.0010) [2023-12-27 03:48:14,623][105620] Updated weights for policy 1, policy_version 1707097 (0.0010) [2023-12-27 03:48:14,688][105620] Updated weights for policy 1, policy_version 1707107 (0.0011) [2023-12-27 03:48:14,753][105620] Updated weights for policy 1, policy_version 1707117 (0.0011) [2023-12-27 03:48:14,827][105692] Updated weights for policy 0, policy_version 1703268 (0.0011) [2023-12-27 03:48:14,894][105692] Updated weights for policy 0, policy_version 1703278 (0.0009) [2023-12-27 03:48:14,961][105692] Updated weights for policy 0, policy_version 1703288 (0.0010) [2023-12-27 03:48:15,407][105620] Updated weights for policy 1, policy_version 1707127 (0.0007) [2023-12-27 03:48:15,468][105620] Updated weights for policy 1, policy_version 1707137 (0.0010) [2023-12-27 03:48:15,524][105620] Updated weights for policy 1, policy_version 1707147 (0.0006) [2023-12-27 03:48:15,796][105692] Updated weights for policy 0, policy_version 1703298 (0.0010) [2023-12-27 03:48:15,855][105692] Updated weights for policy 0, policy_version 1703308 (0.0010) [2023-12-27 03:48:15,910][105692] Updated weights for policy 0, policy_version 1703318 (0.0010) [2023-12-27 03:48:15,977][105692] Updated weights for policy 0, policy_version 1703328 (0.0008) [2023-12-27 03:48:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 873209856. Throughput: 0: 9715.6, 1: 9969.2. Samples: 873179660. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:48:16,063][104569] Avg episode reward: [(0, '8990.365'), (1, '8898.315')] [2023-12-27 03:48:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001703328_436117504.pth... [2023-12-27 03:48:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001707152_437092352.pth... [2023-12-27 03:48:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001702208_435830784.pth [2023-12-27 03:48:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001706000_436797440.pth [2023-12-27 03:48:16,195][105620] Updated weights for policy 1, policy_version 1707157 (0.0008) [2023-12-27 03:48:16,246][105620] Updated weights for policy 1, policy_version 1707167 (0.0010) [2023-12-27 03:48:16,294][105620] Updated weights for policy 1, policy_version 1707177 (0.0010) [2023-12-27 03:48:16,639][105692] Updated weights for policy 0, policy_version 1703338 (0.0006) [2023-12-27 03:48:16,693][105692] Updated weights for policy 0, policy_version 1703348 (0.0009) [2023-12-27 03:48:16,747][105692] Updated weights for policy 0, policy_version 1703358 (0.0006) [2023-12-27 03:48:17,073][105620] Updated weights for policy 1, policy_version 1707187 (0.0011) [2023-12-27 03:48:17,122][105620] Updated weights for policy 1, policy_version 1707197 (0.0010) [2023-12-27 03:48:17,173][105620] Updated weights for policy 1, policy_version 1707207 (0.0010) [2023-12-27 03:48:17,285][105692] Updated weights for policy 0, policy_version 1703368 (0.0005) [2023-12-27 03:48:17,354][105692] Updated weights for policy 0, policy_version 1703378 (0.0005) [2023-12-27 03:48:17,416][105692] Updated weights for policy 0, policy_version 1703388 (0.0006) [2023-12-27 03:48:17,889][105620] Updated weights for policy 1, policy_version 1707217 (0.0010) [2023-12-27 03:48:17,911][105692] Updated weights for policy 0, policy_version 1703398 (0.0008) [2023-12-27 03:48:17,945][105620] Updated weights for policy 1, policy_version 1707227 (0.0008) [2023-12-27 03:48:17,964][105692] Updated weights for policy 0, policy_version 1703408 (0.0007) [2023-12-27 03:48:18,003][105620] Updated weights for policy 1, policy_version 1707237 (0.0007) [2023-12-27 03:48:18,021][105692] Updated weights for policy 0, policy_version 1703418 (0.0007) [2023-12-27 03:48:18,062][105620] Updated weights for policy 1, policy_version 1707247 (0.0011) [2023-12-27 03:48:18,656][105692] Updated weights for policy 0, policy_version 1703428 (0.0008) [2023-12-27 03:48:18,719][105692] Updated weights for policy 0, policy_version 1703438 (0.0007) [2023-12-27 03:48:18,788][105692] Updated weights for policy 0, policy_version 1703448 (0.0008) [2023-12-27 03:48:18,796][105620] Updated weights for policy 1, policy_version 1707257 (0.0011) [2023-12-27 03:48:18,853][105620] Updated weights for policy 1, policy_version 1707267 (0.0009) [2023-12-27 03:48:18,907][105620] Updated weights for policy 1, policy_version 1707277 (0.0008) [2023-12-27 03:48:19,495][105692] Updated weights for policy 0, policy_version 1703458 (0.0011) [2023-12-27 03:48:19,561][105692] Updated weights for policy 0, policy_version 1703468 (0.0011) [2023-12-27 03:48:19,613][105620] Updated weights for policy 1, policy_version 1707287 (0.0007) [2023-12-27 03:48:19,620][105692] Updated weights for policy 0, policy_version 1703478 (0.0011) [2023-12-27 03:48:19,661][105620] Updated weights for policy 1, policy_version 1707297 (0.0005) [2023-12-27 03:48:19,680][105692] Updated weights for policy 0, policy_version 1703488 (0.0011) [2023-12-27 03:48:19,718][105620] Updated weights for policy 1, policy_version 1707307 (0.0006) [2023-12-27 03:48:20,437][105620] Updated weights for policy 1, policy_version 1707317 (0.0007) [2023-12-27 03:48:20,452][105692] Updated weights for policy 0, policy_version 1703498 (0.0007) [2023-12-27 03:48:20,499][105620] Updated weights for policy 1, policy_version 1707327 (0.0006) [2023-12-27 03:48:20,516][105692] Updated weights for policy 0, policy_version 1703508 (0.0011) [2023-12-27 03:48:20,555][105620] Updated weights for policy 1, policy_version 1707337 (0.0007) [2023-12-27 03:48:20,574][105692] Updated weights for policy 0, policy_version 1703518 (0.0010) [2023-12-27 03:48:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 873308160. Throughput: 0: 9752.3, 1: 9902.3. Samples: 873299948. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:48:21,062][104569] Avg episode reward: [(0, '8894.914'), (1, '9080.128')] [2023-12-27 03:48:21,310][105620] Updated weights for policy 1, policy_version 1707347 (0.0008) [2023-12-27 03:48:21,336][105692] Updated weights for policy 0, policy_version 1703528 (0.0011) [2023-12-27 03:48:21,385][105620] Updated weights for policy 1, policy_version 1707357 (0.0008) [2023-12-27 03:48:21,407][105692] Updated weights for policy 0, policy_version 1703538 (0.0011) [2023-12-27 03:48:21,452][105620] Updated weights for policy 1, policy_version 1707367 (0.0006) [2023-12-27 03:48:21,462][105692] Updated weights for policy 0, policy_version 1703548 (0.0009) [2023-12-27 03:48:22,155][105692] Updated weights for policy 0, policy_version 1703558 (0.0009) [2023-12-27 03:48:22,246][105692] Updated weights for policy 0, policy_version 1703568 (0.0011) [2023-12-27 03:48:22,270][105620] Updated weights for policy 1, policy_version 1707377 (0.0009) [2023-12-27 03:48:22,309][105692] Updated weights for policy 0, policy_version 1703578 (0.0011) [2023-12-27 03:48:22,332][105620] Updated weights for policy 1, policy_version 1707387 (0.0006) [2023-12-27 03:48:22,396][105620] Updated weights for policy 1, policy_version 1707397 (0.0008) [2023-12-27 03:48:22,461][105620] Updated weights for policy 1, policy_version 1707407 (0.0008) [2023-12-27 03:48:23,039][105692] Updated weights for policy 0, policy_version 1703588 (0.0010) [2023-12-27 03:48:23,089][105692] Updated weights for policy 0, policy_version 1703598 (0.0008) [2023-12-27 03:48:23,142][105692] Updated weights for policy 0, policy_version 1703608 (0.0007) [2023-12-27 03:48:23,160][105620] Updated weights for policy 1, policy_version 1707417 (0.0011) [2023-12-27 03:48:23,222][105620] Updated weights for policy 1, policy_version 1707427 (0.0011) [2023-12-27 03:48:23,288][105620] Updated weights for policy 1, policy_version 1707437 (0.0010) [2023-12-27 03:48:23,755][105692] Updated weights for policy 0, policy_version 1703618 (0.0006) [2023-12-27 03:48:23,814][105692] Updated weights for policy 0, policy_version 1703628 (0.0005) [2023-12-27 03:48:23,876][105692] Updated weights for policy 0, policy_version 1703638 (0.0005) [2023-12-27 03:48:23,932][105692] Updated weights for policy 0, policy_version 1703648 (0.0010) [2023-12-27 03:48:23,975][105620] Updated weights for policy 1, policy_version 1707447 (0.0007) [2023-12-27 03:48:24,042][105620] Updated weights for policy 1, policy_version 1707457 (0.0010) [2023-12-27 03:48:24,112][105620] Updated weights for policy 1, policy_version 1707467 (0.0011) [2023-12-27 03:48:24,524][105692] Updated weights for policy 0, policy_version 1703658 (0.0006) [2023-12-27 03:48:24,586][105692] Updated weights for policy 0, policy_version 1703668 (0.0006) [2023-12-27 03:48:24,633][105692] Updated weights for policy 0, policy_version 1703678 (0.0008) [2023-12-27 03:48:24,777][105620] Updated weights for policy 1, policy_version 1707477 (0.0010) [2023-12-27 03:48:24,836][105620] Updated weights for policy 1, policy_version 1707487 (0.0009) [2023-12-27 03:48:24,890][105620] Updated weights for policy 1, policy_version 1707497 (0.0005) [2023-12-27 03:48:25,342][105692] Updated weights for policy 0, policy_version 1703688 (0.0009) [2023-12-27 03:48:25,396][105692] Updated weights for policy 0, policy_version 1703698 (0.0008) [2023-12-27 03:48:25,444][105692] Updated weights for policy 0, policy_version 1703708 (0.0009) [2023-12-27 03:48:25,573][105620] Updated weights for policy 1, policy_version 1707507 (0.0007) [2023-12-27 03:48:25,627][105620] Updated weights for policy 1, policy_version 1707517 (0.0008) [2023-12-27 03:48:25,674][105620] Updated weights for policy 1, policy_version 1707527 (0.0009) [2023-12-27 03:48:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.2, 300 sec: 19577.5). Total num frames: 873406464. Throughput: 0: 9739.2, 1: 9813.8. Samples: 873415396. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:48:26,063][104569] Avg episode reward: [(0, '8714.678'), (1, '9263.988')] [2023-12-27 03:48:26,208][105692] Updated weights for policy 0, policy_version 1703718 (0.0009) [2023-12-27 03:48:26,255][105692] Updated weights for policy 0, policy_version 1703728 (0.0009) [2023-12-27 03:48:26,302][105692] Updated weights for policy 0, policy_version 1703738 (0.0008) [2023-12-27 03:48:26,433][105620] Updated weights for policy 1, policy_version 1707537 (0.0009) [2023-12-27 03:48:26,486][105620] Updated weights for policy 1, policy_version 1707547 (0.0008) [2023-12-27 03:48:26,541][105620] Updated weights for policy 1, policy_version 1707557 (0.0009) [2023-12-27 03:48:26,596][105620] Updated weights for policy 1, policy_version 1707567 (0.0009) [2023-12-27 03:48:27,034][105692] Updated weights for policy 0, policy_version 1703748 (0.0010) [2023-12-27 03:48:27,092][105692] Updated weights for policy 0, policy_version 1703758 (0.0009) [2023-12-27 03:48:27,139][105692] Updated weights for policy 0, policy_version 1703768 (0.0009) [2023-12-27 03:48:27,335][105620] Updated weights for policy 1, policy_version 1707577 (0.0009) [2023-12-27 03:48:27,385][105620] Updated weights for policy 1, policy_version 1707587 (0.0009) [2023-12-27 03:48:27,431][105620] Updated weights for policy 1, policy_version 1707597 (0.0008) [2023-12-27 03:48:27,916][105692] Updated weights for policy 0, policy_version 1703778 (0.0009) [2023-12-27 03:48:27,971][105692] Updated weights for policy 0, policy_version 1703788 (0.0009) [2023-12-27 03:48:28,025][105692] Updated weights for policy 0, policy_version 1703798 (0.0009) [2023-12-27 03:48:28,082][105692] Updated weights for policy 0, policy_version 1703808 (0.0010) [2023-12-27 03:48:28,173][105620] Updated weights for policy 1, policy_version 1707607 (0.0007) [2023-12-27 03:48:28,222][105620] Updated weights for policy 1, policy_version 1707617 (0.0007) [2023-12-27 03:48:28,270][105620] Updated weights for policy 1, policy_version 1707627 (0.0008) [2023-12-27 03:48:28,886][105692] Updated weights for policy 0, policy_version 1703818 (0.0008) [2023-12-27 03:48:28,940][105692] Updated weights for policy 0, policy_version 1703828 (0.0005) [2023-12-27 03:48:28,994][105692] Updated weights for policy 0, policy_version 1703838 (0.0005) [2023-12-27 03:48:29,021][105620] Updated weights for policy 1, policy_version 1707637 (0.0008) [2023-12-27 03:48:29,076][105620] Updated weights for policy 1, policy_version 1707647 (0.0010) [2023-12-27 03:48:29,135][105620] Updated weights for policy 1, policy_version 1707657 (0.0009) [2023-12-27 03:48:29,709][105692] Updated weights for policy 0, policy_version 1703848 (0.0008) [2023-12-27 03:48:29,756][105692] Updated weights for policy 0, policy_version 1703858 (0.0009) [2023-12-27 03:48:29,808][105692] Updated weights for policy 0, policy_version 1703868 (0.0009) [2023-12-27 03:48:29,893][105620] Updated weights for policy 1, policy_version 1707667 (0.0009) [2023-12-27 03:48:29,952][105620] Updated weights for policy 1, policy_version 1707677 (0.0009) [2023-12-27 03:48:30,004][105620] Updated weights for policy 1, policy_version 1707687 (0.0009) [2023-12-27 03:48:30,542][105692] Updated weights for policy 0, policy_version 1703878 (0.0006) [2023-12-27 03:48:30,592][105692] Updated weights for policy 0, policy_version 1703888 (0.0005) [2023-12-27 03:48:30,638][105692] Updated weights for policy 0, policy_version 1703898 (0.0005) [2023-12-27 03:48:30,745][105620] Updated weights for policy 1, policy_version 1707697 (0.0008) [2023-12-27 03:48:30,806][105620] Updated weights for policy 1, policy_version 1707707 (0.0007) [2023-12-27 03:48:30,853][105620] Updated weights for policy 1, policy_version 1707717 (0.0009) [2023-12-27 03:48:30,904][105620] Updated weights for policy 1, policy_version 1707727 (0.0009) [2023-12-27 03:48:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 873504768. Throughput: 0: 9738.3, 1: 9807.2. Samples: 873472404. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:48:31,062][104569] Avg episode reward: [(0, '8628.576'), (1, '9263.900')] [2023-12-27 03:48:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001707728_437239808.pth... [2023-12-27 03:48:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001703904_436264960.pth... [2023-12-27 03:48:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001706608_436953088.pth [2023-12-27 03:48:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001702784_435978240.pth [2023-12-27 03:48:31,299][105692] Updated weights for policy 0, policy_version 1703908 (0.0006) [2023-12-27 03:48:31,366][105692] Updated weights for policy 0, policy_version 1703918 (0.0006) [2023-12-27 03:48:31,426][105692] Updated weights for policy 0, policy_version 1703928 (0.0009) [2023-12-27 03:48:31,709][105620] Updated weights for policy 1, policy_version 1707737 (0.0008) [2023-12-27 03:48:31,774][105620] Updated weights for policy 1, policy_version 1707747 (0.0008) [2023-12-27 03:48:31,843][105620] Updated weights for policy 1, policy_version 1707757 (0.0009) [2023-12-27 03:48:32,105][105692] Updated weights for policy 0, policy_version 1703938 (0.0009) [2023-12-27 03:48:32,169][105692] Updated weights for policy 0, policy_version 1703948 (0.0008) [2023-12-27 03:48:32,225][105692] Updated weights for policy 0, policy_version 1703958 (0.0008) [2023-12-27 03:48:32,285][105692] Updated weights for policy 0, policy_version 1703968 (0.0008) [2023-12-27 03:48:32,581][105620] Updated weights for policy 1, policy_version 1707767 (0.0010) [2023-12-27 03:48:32,640][105620] Updated weights for policy 1, policy_version 1707777 (0.0010) [2023-12-27 03:48:32,688][105620] Updated weights for policy 1, policy_version 1707787 (0.0010) [2023-12-27 03:48:33,028][105692] Updated weights for policy 0, policy_version 1703978 (0.0005) [2023-12-27 03:48:33,089][105692] Updated weights for policy 0, policy_version 1703988 (0.0005) [2023-12-27 03:48:33,142][105692] Updated weights for policy 0, policy_version 1703998 (0.0005) [2023-12-27 03:48:33,431][105620] Updated weights for policy 1, policy_version 1707797 (0.0010) [2023-12-27 03:48:33,485][105620] Updated weights for policy 1, policy_version 1707807 (0.0010) [2023-12-27 03:48:33,532][105620] Updated weights for policy 1, policy_version 1707817 (0.0010) [2023-12-27 03:48:33,646][105692] Updated weights for policy 0, policy_version 1704008 (0.0005) [2023-12-27 03:48:33,707][105692] Updated weights for policy 0, policy_version 1704018 (0.0005) [2023-12-27 03:48:33,758][105692] Updated weights for policy 0, policy_version 1704028 (0.0010) [2023-12-27 03:48:34,215][105620] Updated weights for policy 1, policy_version 1707827 (0.0010) [2023-12-27 03:48:34,273][105620] Updated weights for policy 1, policy_version 1707837 (0.0010) [2023-12-27 03:48:34,339][105620] Updated weights for policy 1, policy_version 1707847 (0.0010) [2023-12-27 03:48:34,490][105692] Updated weights for policy 0, policy_version 1704038 (0.0011) [2023-12-27 03:48:34,545][105692] Updated weights for policy 0, policy_version 1704048 (0.0010) [2023-12-27 03:48:34,595][105692] Updated weights for policy 0, policy_version 1704058 (0.0010) [2023-12-27 03:48:35,084][105620] Updated weights for policy 1, policy_version 1707857 (0.0010) [2023-12-27 03:48:35,141][105620] Updated weights for policy 1, policy_version 1707867 (0.0010) [2023-12-27 03:48:35,185][105620] Updated weights for policy 1, policy_version 1707877 (0.0010) [2023-12-27 03:48:35,243][105620] Updated weights for policy 1, policy_version 1707887 (0.0010) [2023-12-27 03:48:35,365][105692] Updated weights for policy 0, policy_version 1704068 (0.0010) [2023-12-27 03:48:35,424][105692] Updated weights for policy 0, policy_version 1704078 (0.0006) [2023-12-27 03:48:35,486][105692] Updated weights for policy 0, policy_version 1704088 (0.0006) [2023-12-27 03:48:35,999][105620] Updated weights for policy 1, policy_version 1707897 (0.0010) [2023-12-27 03:48:36,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 873594880. Throughput: 0: 9772.7, 1: 9679.0. Samples: 873589704. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:48:36,063][104569] Avg episode reward: [(0, '8354.511'), (1, '9263.688')] [2023-12-27 03:48:36,064][105620] Updated weights for policy 1, policy_version 1707907 (0.0010) [2023-12-27 03:48:36,101][105692] Updated weights for policy 0, policy_version 1704098 (0.0010) [2023-12-27 03:48:36,131][105620] Updated weights for policy 1, policy_version 1707917 (0.0010) [2023-12-27 03:48:36,163][105692] Updated weights for policy 0, policy_version 1704108 (0.0007) [2023-12-27 03:48:36,227][105692] Updated weights for policy 0, policy_version 1704118 (0.0007) [2023-12-27 03:48:36,288][105692] Updated weights for policy 0, policy_version 1704128 (0.0011) [2023-12-27 03:48:36,867][105620] Updated weights for policy 1, policy_version 1707927 (0.0010) [2023-12-27 03:48:36,870][105692] Updated weights for policy 0, policy_version 1704138 (0.0005) [2023-12-27 03:48:36,923][105620] Updated weights for policy 1, policy_version 1707937 (0.0010) [2023-12-27 03:48:36,927][105692] Updated weights for policy 0, policy_version 1704148 (0.0008) [2023-12-27 03:48:36,971][105620] Updated weights for policy 1, policy_version 1707947 (0.0010) [2023-12-27 03:48:36,986][105692] Updated weights for policy 0, policy_version 1704158 (0.0010) [2023-12-27 03:48:37,570][105620] Updated weights for policy 1, policy_version 1707957 (0.0008) [2023-12-27 03:48:37,629][105620] Updated weights for policy 1, policy_version 1707967 (0.0006) [2023-12-27 03:48:37,661][105692] Updated weights for policy 0, policy_version 1704168 (0.0010) [2023-12-27 03:48:37,690][105620] Updated weights for policy 1, policy_version 1707977 (0.0007) [2023-12-27 03:48:37,729][105692] Updated weights for policy 0, policy_version 1704178 (0.0006) [2023-12-27 03:48:37,795][105692] Updated weights for policy 0, policy_version 1704188 (0.0006) [2023-12-27 03:48:38,294][105620] Updated weights for policy 1, policy_version 1707987 (0.0007) [2023-12-27 03:48:38,366][105620] Updated weights for policy 1, policy_version 1707997 (0.0010) [2023-12-27 03:48:38,377][105692] Updated weights for policy 0, policy_version 1704198 (0.0006) [2023-12-27 03:48:38,428][105692] Updated weights for policy 0, policy_version 1704208 (0.0006) [2023-12-27 03:48:38,432][105620] Updated weights for policy 1, policy_version 1708007 (0.0010) [2023-12-27 03:48:38,493][105692] Updated weights for policy 0, policy_version 1704218 (0.0010) [2023-12-27 03:48:39,113][105620] Updated weights for policy 1, policy_version 1708017 (0.0008) [2023-12-27 03:48:39,157][105692] Updated weights for policy 0, policy_version 1704228 (0.0010) [2023-12-27 03:48:39,177][105620] Updated weights for policy 1, policy_version 1708027 (0.0005) [2023-12-27 03:48:39,206][105692] Updated weights for policy 0, policy_version 1704238 (0.0011) [2023-12-27 03:48:39,235][105620] Updated weights for policy 1, policy_version 1708037 (0.0006) [2023-12-27 03:48:39,270][105692] Updated weights for policy 0, policy_version 1704248 (0.0010) [2023-12-27 03:48:39,299][105620] Updated weights for policy 1, policy_version 1708047 (0.0010) [2023-12-27 03:48:40,071][105692] Updated weights for policy 0, policy_version 1704258 (0.0010) [2023-12-27 03:48:40,083][105620] Updated weights for policy 1, policy_version 1708057 (0.0009) [2023-12-27 03:48:40,134][105692] Updated weights for policy 0, policy_version 1704268 (0.0008) [2023-12-27 03:48:40,140][105620] Updated weights for policy 1, policy_version 1708067 (0.0007) [2023-12-27 03:48:40,189][105620] Updated weights for policy 1, policy_version 1708077 (0.0007) [2023-12-27 03:48:40,197][105692] Updated weights for policy 0, policy_version 1704278 (0.0009) [2023-12-27 03:48:40,263][105692] Updated weights for policy 0, policy_version 1704288 (0.0010) [2023-12-27 03:48:40,803][105620] Updated weights for policy 1, policy_version 1708087 (0.0009) [2023-12-27 03:48:40,861][105620] Updated weights for policy 1, policy_version 1708097 (0.0009) [2023-12-27 03:48:40,919][105620] Updated weights for policy 1, policy_version 1708107 (0.0008) [2023-12-27 03:48:41,027][105692] Updated weights for policy 0, policy_version 1704298 (0.0009) [2023-12-27 03:48:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 873701376. Throughput: 0: 9760.1, 1: 9710.0. Samples: 873710644. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:48:41,062][104569] Avg episode reward: [(0, '8534.862'), (1, '9078.812')] [2023-12-27 03:48:41,092][105692] Updated weights for policy 0, policy_version 1704308 (0.0008) [2023-12-27 03:48:41,167][105692] Updated weights for policy 0, policy_version 1704318 (0.0008) [2023-12-27 03:48:41,742][105620] Updated weights for policy 1, policy_version 1708117 (0.0009) [2023-12-27 03:48:41,804][105620] Updated weights for policy 1, policy_version 1708127 (0.0008) [2023-12-27 03:48:41,843][105692] Updated weights for policy 0, policy_version 1704328 (0.0008) [2023-12-27 03:48:41,858][105620] Updated weights for policy 1, policy_version 1708137 (0.0007) [2023-12-27 03:48:41,901][105692] Updated weights for policy 0, policy_version 1704338 (0.0008) [2023-12-27 03:48:41,967][105692] Updated weights for policy 0, policy_version 1704348 (0.0009) [2023-12-27 03:48:42,585][105620] Updated weights for policy 1, policy_version 1708147 (0.0007) [2023-12-27 03:48:42,646][105620] Updated weights for policy 1, policy_version 1708157 (0.0008) [2023-12-27 03:48:42,704][105620] Updated weights for policy 1, policy_version 1708167 (0.0009) [2023-12-27 03:48:42,762][105692] Updated weights for policy 0, policy_version 1704358 (0.0009) [2023-12-27 03:48:42,832][105692] Updated weights for policy 0, policy_version 1704368 (0.0006) [2023-12-27 03:48:42,900][105692] Updated weights for policy 0, policy_version 1704378 (0.0006) [2023-12-27 03:48:43,460][105620] Updated weights for policy 1, policy_version 1708177 (0.0007) [2023-12-27 03:48:43,521][105620] Updated weights for policy 1, policy_version 1708187 (0.0009) [2023-12-27 03:48:43,560][105692] Updated weights for policy 0, policy_version 1704388 (0.0006) [2023-12-27 03:48:43,584][105620] Updated weights for policy 1, policy_version 1708197 (0.0008) [2023-12-27 03:48:43,607][105692] Updated weights for policy 0, policy_version 1704398 (0.0007) [2023-12-27 03:48:43,644][105620] Updated weights for policy 1, policy_version 1708207 (0.0007) [2023-12-27 03:48:43,659][105692] Updated weights for policy 0, policy_version 1704408 (0.0006) [2023-12-27 03:48:44,343][105620] Updated weights for policy 1, policy_version 1708217 (0.0006) [2023-12-27 03:48:44,394][105620] Updated weights for policy 1, policy_version 1708227 (0.0006) [2023-12-27 03:48:44,453][105620] Updated weights for policy 1, policy_version 1708237 (0.0009) [2023-12-27 03:48:44,459][105692] Updated weights for policy 0, policy_version 1704418 (0.0008) [2023-12-27 03:48:44,512][105692] Updated weights for policy 0, policy_version 1704428 (0.0008) [2023-12-27 03:48:44,559][105692] Updated weights for policy 0, policy_version 1704438 (0.0009) [2023-12-27 03:48:44,614][105692] Updated weights for policy 0, policy_version 1704448 (0.0009) [2023-12-27 03:48:45,183][105620] Updated weights for policy 1, policy_version 1708247 (0.0007) [2023-12-27 03:48:45,249][105620] Updated weights for policy 1, policy_version 1708257 (0.0006) [2023-12-27 03:48:45,311][105620] Updated weights for policy 1, policy_version 1708267 (0.0008) [2023-12-27 03:48:45,350][105692] Updated weights for policy 0, policy_version 1704458 (0.0011) [2023-12-27 03:48:45,406][105692] Updated weights for policy 0, policy_version 1704468 (0.0010) [2023-12-27 03:48:45,465][105692] Updated weights for policy 0, policy_version 1704478 (0.0011) [2023-12-27 03:48:45,955][105620] Updated weights for policy 1, policy_version 1708277 (0.0006) [2023-12-27 03:48:46,004][105620] Updated weights for policy 1, policy_version 1708287 (0.0008) [2023-12-27 03:48:46,049][105620] Updated weights for policy 1, policy_version 1708297 (0.0010) [2023-12-27 03:48:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 873791488. Throughput: 0: 9722.8, 1: 9655.0. Samples: 873767004. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:48:46,063][104569] Avg episode reward: [(0, '8623.987'), (1, '8893.160')] [2023-12-27 03:48:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001704480_436412416.pth... [2023-12-27 03:48:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001703328_436117504.pth [2023-12-27 03:48:46,084][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001708304_437387264.pth... [2023-12-27 03:48:46,087][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001707152_437092352.pth [2023-12-27 03:48:46,234][105692] Updated weights for policy 0, policy_version 1704488 (0.0011) [2023-12-27 03:48:46,285][105692] Updated weights for policy 0, policy_version 1704498 (0.0010) [2023-12-27 03:48:46,337][105692] Updated weights for policy 0, policy_version 1704508 (0.0010) [2023-12-27 03:48:46,628][105620] Updated weights for policy 1, policy_version 1708307 (0.0009) [2023-12-27 03:48:46,687][105620] Updated weights for policy 1, policy_version 1708317 (0.0007) [2023-12-27 03:48:46,755][105620] Updated weights for policy 1, policy_version 1708327 (0.0009) [2023-12-27 03:48:47,016][105692] Updated weights for policy 0, policy_version 1704518 (0.0010) [2023-12-27 03:48:47,083][105692] Updated weights for policy 0, policy_version 1704528 (0.0010) [2023-12-27 03:48:47,149][105692] Updated weights for policy 0, policy_version 1704538 (0.0010) [2023-12-27 03:48:47,336][105620] Updated weights for policy 1, policy_version 1708337 (0.0011) [2023-12-27 03:48:47,399][105620] Updated weights for policy 1, policy_version 1708347 (0.0011) [2023-12-27 03:48:47,459][105620] Updated weights for policy 1, policy_version 1708357 (0.0008) [2023-12-27 03:48:47,510][105620] Updated weights for policy 1, policy_version 1708367 (0.0008) [2023-12-27 03:48:47,811][105692] Updated weights for policy 0, policy_version 1704548 (0.0008) [2023-12-27 03:48:47,869][105692] Updated weights for policy 0, policy_version 1704558 (0.0010) [2023-12-27 03:48:47,928][105692] Updated weights for policy 0, policy_version 1704568 (0.0007) [2023-12-27 03:48:48,212][105620] Updated weights for policy 1, policy_version 1708377 (0.0006) [2023-12-27 03:48:48,279][105620] Updated weights for policy 1, policy_version 1708387 (0.0006) [2023-12-27 03:48:48,334][105620] Updated weights for policy 1, policy_version 1708397 (0.0009) [2023-12-27 03:48:48,518][105692] Updated weights for policy 0, policy_version 1704578 (0.0006) [2023-12-27 03:48:48,579][105692] Updated weights for policy 0, policy_version 1704588 (0.0011) [2023-12-27 03:48:48,632][105692] Updated weights for policy 0, policy_version 1704598 (0.0011) [2023-12-27 03:48:48,677][105692] Updated weights for policy 0, policy_version 1704608 (0.0010) [2023-12-27 03:48:48,931][105620] Updated weights for policy 1, policy_version 1708407 (0.0008) [2023-12-27 03:48:48,985][105620] Updated weights for policy 1, policy_version 1708417 (0.0006) [2023-12-27 03:48:49,047][105620] Updated weights for policy 1, policy_version 1708427 (0.0006) [2023-12-27 03:48:49,467][105692] Updated weights for policy 0, policy_version 1704618 (0.0005) [2023-12-27 03:48:49,525][105692] Updated weights for policy 0, policy_version 1704628 (0.0006) [2023-12-27 03:48:49,589][105692] Updated weights for policy 0, policy_version 1704638 (0.0009) [2023-12-27 03:48:49,723][105620] Updated weights for policy 1, policy_version 1708437 (0.0007) [2023-12-27 03:48:49,773][105620] Updated weights for policy 1, policy_version 1708447 (0.0008) [2023-12-27 03:48:49,824][105620] Updated weights for policy 1, policy_version 1708457 (0.0009) [2023-12-27 03:48:50,304][105692] Updated weights for policy 0, policy_version 1704648 (0.0010) [2023-12-27 03:48:50,351][105692] Updated weights for policy 0, policy_version 1704658 (0.0010) [2023-12-27 03:48:50,404][105692] Updated weights for policy 0, policy_version 1704668 (0.0011) [2023-12-27 03:48:50,585][105620] Updated weights for policy 1, policy_version 1708467 (0.0009) [2023-12-27 03:48:50,648][105620] Updated weights for policy 1, policy_version 1708477 (0.0009) [2023-12-27 03:48:50,711][105620] Updated weights for policy 1, policy_version 1708487 (0.0009) [2023-12-27 03:48:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 873897984. Throughput: 0: 9799.4, 1: 9757.4. Samples: 873888820. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:48:51,063][104569] Avg episode reward: [(0, '8624.284'), (1, '8895.633')] [2023-12-27 03:48:51,159][105692] Updated weights for policy 0, policy_version 1704678 (0.0009) [2023-12-27 03:48:51,222][105692] Updated weights for policy 0, policy_version 1704688 (0.0006) [2023-12-27 03:48:51,290][105692] Updated weights for policy 0, policy_version 1704698 (0.0007) [2023-12-27 03:48:51,509][105620] Updated weights for policy 1, policy_version 1708497 (0.0009) [2023-12-27 03:48:51,573][105620] Updated weights for policy 1, policy_version 1708507 (0.0009) [2023-12-27 03:48:51,640][105620] Updated weights for policy 1, policy_version 1708517 (0.0008) [2023-12-27 03:48:51,702][105620] Updated weights for policy 1, policy_version 1708527 (0.0008) [2023-12-27 03:48:51,947][105692] Updated weights for policy 0, policy_version 1704708 (0.0009) [2023-12-27 03:48:52,012][105692] Updated weights for policy 0, policy_version 1704718 (0.0006) [2023-12-27 03:48:52,080][105692] Updated weights for policy 0, policy_version 1704728 (0.0008) [2023-12-27 03:48:52,339][105620] Updated weights for policy 1, policy_version 1708537 (0.0007) [2023-12-27 03:48:52,406][105620] Updated weights for policy 1, policy_version 1708547 (0.0008) [2023-12-27 03:48:52,466][105620] Updated weights for policy 1, policy_version 1708557 (0.0009) [2023-12-27 03:48:52,756][105692] Updated weights for policy 0, policy_version 1704738 (0.0007) [2023-12-27 03:48:52,806][105692] Updated weights for policy 0, policy_version 1704748 (0.0007) [2023-12-27 03:48:52,854][105692] Updated weights for policy 0, policy_version 1704758 (0.0009) [2023-12-27 03:48:52,910][105692] Updated weights for policy 0, policy_version 1704768 (0.0008) [2023-12-27 03:48:53,120][105620] Updated weights for policy 1, policy_version 1708567 (0.0007) [2023-12-27 03:48:53,171][105620] Updated weights for policy 1, policy_version 1708577 (0.0005) [2023-12-27 03:48:53,226][105620] Updated weights for policy 1, policy_version 1708587 (0.0005) [2023-12-27 03:48:53,623][105692] Updated weights for policy 0, policy_version 1704778 (0.0009) [2023-12-27 03:48:53,680][105692] Updated weights for policy 0, policy_version 1704788 (0.0005) [2023-12-27 03:48:53,729][105692] Updated weights for policy 0, policy_version 1704798 (0.0005) [2023-12-27 03:48:53,844][105620] Updated weights for policy 1, policy_version 1708597 (0.0008) [2023-12-27 03:48:53,905][105620] Updated weights for policy 1, policy_version 1708607 (0.0010) [2023-12-27 03:48:53,963][105620] Updated weights for policy 1, policy_version 1708617 (0.0010) [2023-12-27 03:48:54,332][105692] Updated weights for policy 0, policy_version 1704808 (0.0006) [2023-12-27 03:48:54,395][105692] Updated weights for policy 0, policy_version 1704818 (0.0006) [2023-12-27 03:48:54,467][105692] Updated weights for policy 0, policy_version 1704828 (0.0006) [2023-12-27 03:48:54,521][105620] Updated weights for policy 1, policy_version 1708627 (0.0010) [2023-12-27 03:48:54,585][105620] Updated weights for policy 1, policy_version 1708637 (0.0008) [2023-12-27 03:48:54,650][105620] Updated weights for policy 1, policy_version 1708647 (0.0008) [2023-12-27 03:48:55,004][105692] Updated weights for policy 0, policy_version 1704838 (0.0006) [2023-12-27 03:48:55,076][105692] Updated weights for policy 0, policy_version 1704848 (0.0006) [2023-12-27 03:48:55,139][105692] Updated weights for policy 0, policy_version 1704858 (0.0007) [2023-12-27 03:48:55,263][105620] Updated weights for policy 1, policy_version 1708657 (0.0009) [2023-12-27 03:48:55,325][105620] Updated weights for policy 1, policy_version 1708667 (0.0011) [2023-12-27 03:48:55,382][105620] Updated weights for policy 1, policy_version 1708677 (0.0010) [2023-12-27 03:48:55,434][105620] Updated weights for policy 1, policy_version 1708687 (0.0010) [2023-12-27 03:48:55,801][105692] Updated weights for policy 0, policy_version 1704868 (0.0011) [2023-12-27 03:48:55,856][105692] Updated weights for policy 0, policy_version 1704878 (0.0010) [2023-12-27 03:48:55,914][105692] Updated weights for policy 0, policy_version 1704888 (0.0010) [2023-12-27 03:48:56,062][104569] Fps is (10 sec: 21299.4, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 874004480. Throughput: 0: 9923.1, 1: 9858.1. Samples: 874012344. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:48:56,063][104569] Avg episode reward: [(0, '8622.336'), (1, '8896.368')] [2023-12-27 03:48:56,121][105620] Updated weights for policy 1, policy_version 1708697 (0.0009) [2023-12-27 03:48:56,175][105620] Updated weights for policy 1, policy_version 1708707 (0.0010) [2023-12-27 03:48:56,241][105620] Updated weights for policy 1, policy_version 1708717 (0.0011) [2023-12-27 03:48:56,553][105692] Updated weights for policy 0, policy_version 1704898 (0.0009) [2023-12-27 03:48:56,609][105692] Updated weights for policy 0, policy_version 1704908 (0.0005) [2023-12-27 03:48:56,656][105692] Updated weights for policy 0, policy_version 1704918 (0.0007) [2023-12-27 03:48:56,709][105692] Updated weights for policy 0, policy_version 1704928 (0.0008) [2023-12-27 03:48:56,972][105620] Updated weights for policy 1, policy_version 1708727 (0.0010) [2023-12-27 03:48:57,020][105620] Updated weights for policy 1, policy_version 1708737 (0.0010) [2023-12-27 03:48:57,067][105620] Updated weights for policy 1, policy_version 1708747 (0.0010) [2023-12-27 03:48:57,316][105692] Updated weights for policy 0, policy_version 1704938 (0.0006) [2023-12-27 03:48:57,377][105692] Updated weights for policy 0, policy_version 1704948 (0.0005) [2023-12-27 03:48:57,430][105692] Updated weights for policy 0, policy_version 1704958 (0.0005) [2023-12-27 03:48:57,729][105620] Updated weights for policy 1, policy_version 1708757 (0.0010) [2023-12-27 03:48:57,793][105620] Updated weights for policy 1, policy_version 1708767 (0.0010) [2023-12-27 03:48:57,861][105620] Updated weights for policy 1, policy_version 1708777 (0.0010) [2023-12-27 03:48:57,995][105692] Updated weights for policy 0, policy_version 1704968 (0.0008) [2023-12-27 03:48:58,049][105692] Updated weights for policy 0, policy_version 1704978 (0.0007) [2023-12-27 03:48:58,106][105692] Updated weights for policy 0, policy_version 1704988 (0.0006) [2023-12-27 03:48:58,602][105620] Updated weights for policy 1, policy_version 1708787 (0.0010) [2023-12-27 03:48:58,667][105620] Updated weights for policy 1, policy_version 1708797 (0.0008) [2023-12-27 03:48:58,730][105620] Updated weights for policy 1, policy_version 1708807 (0.0008) [2023-12-27 03:48:58,882][105692] Updated weights for policy 0, policy_version 1704998 (0.0006) [2023-12-27 03:48:58,939][105692] Updated weights for policy 0, policy_version 1705008 (0.0006) [2023-12-27 03:48:58,990][105692] Updated weights for policy 0, policy_version 1705018 (0.0006) [2023-12-27 03:48:59,498][105620] Updated weights for policy 1, policy_version 1708817 (0.0008) [2023-12-27 03:48:59,562][105620] Updated weights for policy 1, policy_version 1708827 (0.0008) [2023-12-27 03:48:59,625][105620] Updated weights for policy 1, policy_version 1708837 (0.0010) [2023-12-27 03:48:59,651][105692] Updated weights for policy 0, policy_version 1705028 (0.0006) [2023-12-27 03:48:59,686][105620] Updated weights for policy 1, policy_version 1708847 (0.0007) [2023-12-27 03:48:59,710][105692] Updated weights for policy 0, policy_version 1705038 (0.0008) [2023-12-27 03:48:59,765][105692] Updated weights for policy 0, policy_version 1705048 (0.0009) [2023-12-27 03:49:00,390][105620] Updated weights for policy 1, policy_version 1708857 (0.0008) [2023-12-27 03:49:00,453][105620] Updated weights for policy 1, policy_version 1708867 (0.0009) [2023-12-27 03:49:00,505][105692] Updated weights for policy 0, policy_version 1705058 (0.0008) [2023-12-27 03:49:00,506][105620] Updated weights for policy 1, policy_version 1708877 (0.0010) [2023-12-27 03:49:00,556][105692] Updated weights for policy 0, policy_version 1705068 (0.0007) [2023-12-27 03:49:00,615][105692] Updated weights for policy 0, policy_version 1705078 (0.0010) [2023-12-27 03:49:01,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 874102784. Throughput: 0: 10027.4, 1: 9835.7. Samples: 874073496. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:49:01,062][104569] Avg episode reward: [(0, '8440.279'), (1, '8896.254')] [2023-12-27 03:49:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001705088_436568064.pth... [2023-12-27 03:49:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001708880_437534720.pth... [2023-12-27 03:49:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001703904_436264960.pth [2023-12-27 03:49:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001707728_437239808.pth [2023-12-27 03:49:01,278][105620] Updated weights for policy 1, policy_version 1708887 (0.0008) [2023-12-27 03:49:01,325][105620] Updated weights for policy 1, policy_version 1708897 (0.0008) [2023-12-27 03:49:01,338][105692] Updated weights for policy 0, policy_version 1705089 (0.0009) [2023-12-27 03:49:01,393][105620] Updated weights for policy 1, policy_version 1708907 (0.0008) [2023-12-27 03:49:01,405][105692] Updated weights for policy 0, policy_version 1705099 (0.0009) [2023-12-27 03:49:01,464][105692] Updated weights for policy 0, policy_version 1705109 (0.0008) [2023-12-27 03:49:01,513][105692] Updated weights for policy 0, policy_version 1705119 (0.0006) [2023-12-27 03:49:02,137][105620] Updated weights for policy 1, policy_version 1708917 (0.0009) [2023-12-27 03:49:02,187][105620] Updated weights for policy 1, policy_version 1708927 (0.0009) [2023-12-27 03:49:02,226][105692] Updated weights for policy 0, policy_version 1705129 (0.0007) [2023-12-27 03:49:02,232][105620] Updated weights for policy 1, policy_version 1708937 (0.0005) [2023-12-27 03:49:02,283][105692] Updated weights for policy 0, policy_version 1705139 (0.0008) [2023-12-27 03:49:02,339][105692] Updated weights for policy 0, policy_version 1705149 (0.0009) [2023-12-27 03:49:02,999][105692] Updated weights for policy 0, policy_version 1705159 (0.0008) [2023-12-27 03:49:03,048][105692] Updated weights for policy 0, policy_version 1705169 (0.0009) [2023-12-27 03:49:03,067][105620] Updated weights for policy 1, policy_version 1708947 (0.0006) [2023-12-27 03:49:03,116][105692] Updated weights for policy 0, policy_version 1705179 (0.0007) [2023-12-27 03:49:03,132][105620] Updated weights for policy 1, policy_version 1708957 (0.0007) [2023-12-27 03:49:03,184][105620] Updated weights for policy 1, policy_version 1708967 (0.0008) [2023-12-27 03:49:03,883][105692] Updated weights for policy 0, policy_version 1705189 (0.0009) [2023-12-27 03:49:03,902][105620] Updated weights for policy 1, policy_version 1708977 (0.0009) [2023-12-27 03:49:03,936][105692] Updated weights for policy 0, policy_version 1705199 (0.0007) [2023-12-27 03:49:03,952][105620] Updated weights for policy 1, policy_version 1708987 (0.0008) [2023-12-27 03:49:03,989][105692] Updated weights for policy 0, policy_version 1705209 (0.0008) [2023-12-27 03:49:04,001][105620] Updated weights for policy 1, policy_version 1708997 (0.0005) [2023-12-27 03:49:04,061][105620] Updated weights for policy 1, policy_version 1709007 (0.0009) [2023-12-27 03:49:04,785][105692] Updated weights for policy 0, policy_version 1705219 (0.0006) [2023-12-27 03:49:04,819][105620] Updated weights for policy 1, policy_version 1709017 (0.0010) [2023-12-27 03:49:04,839][105692] Updated weights for policy 0, policy_version 1705229 (0.0005) [2023-12-27 03:49:04,873][105620] Updated weights for policy 1, policy_version 1709027 (0.0010) [2023-12-27 03:49:04,888][105692] Updated weights for policy 0, policy_version 1705239 (0.0006) [2023-12-27 03:49:04,925][105620] Updated weights for policy 1, policy_version 1709037 (0.0010) [2023-12-27 03:49:05,643][105692] Updated weights for policy 0, policy_version 1705249 (0.0005) [2023-12-27 03:49:05,664][105620] Updated weights for policy 1, policy_version 1709047 (0.0010) [2023-12-27 03:49:05,701][105692] Updated weights for policy 0, policy_version 1705259 (0.0005) [2023-12-27 03:49:05,715][105620] Updated weights for policy 1, policy_version 1709057 (0.0010) [2023-12-27 03:49:05,757][105692] Updated weights for policy 0, policy_version 1705269 (0.0005) [2023-12-27 03:49:05,759][105620] Updated weights for policy 1, policy_version 1709067 (0.0010) [2023-12-27 03:49:05,823][105692] Updated weights for policy 0, policy_version 1705279 (0.0007) [2023-12-27 03:49:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 874201088. Throughput: 0: 9952.5, 1: 9761.7. Samples: 874187088. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 03:49:06,063][104569] Avg episode reward: [(0, '8536.240'), (1, '8897.132')] [2023-12-27 03:49:06,507][105620] Updated weights for policy 1, policy_version 1709077 (0.0010) [2023-12-27 03:49:06,567][105620] Updated weights for policy 1, policy_version 1709087 (0.0008) [2023-12-27 03:49:06,598][105692] Updated weights for policy 0, policy_version 1705289 (0.0007) [2023-12-27 03:49:06,620][105620] Updated weights for policy 1, policy_version 1709097 (0.0006) [2023-12-27 03:49:06,664][105692] Updated weights for policy 0, policy_version 1705299 (0.0008) [2023-12-27 03:49:06,728][105692] Updated weights for policy 0, policy_version 1705309 (0.0009) [2023-12-27 03:49:07,397][105620] Updated weights for policy 1, policy_version 1709107 (0.0009) [2023-12-27 03:49:07,437][105692] Updated weights for policy 0, policy_version 1705319 (0.0007) [2023-12-27 03:49:07,443][105620] Updated weights for policy 1, policy_version 1709117 (0.0007) [2023-12-27 03:49:07,490][105692] Updated weights for policy 0, policy_version 1705329 (0.0007) [2023-12-27 03:49:07,492][105620] Updated weights for policy 1, policy_version 1709127 (0.0006) [2023-12-27 03:49:07,548][105692] Updated weights for policy 0, policy_version 1705339 (0.0006) [2023-12-27 03:49:08,114][105620] Updated weights for policy 1, policy_version 1709137 (0.0006) [2023-12-27 03:49:08,182][105620] Updated weights for policy 1, policy_version 1709147 (0.0009) [2023-12-27 03:49:08,242][105620] Updated weights for policy 1, policy_version 1709157 (0.0009) [2023-12-27 03:49:08,307][105620] Updated weights for policy 1, policy_version 1709167 (0.0009) [2023-12-27 03:49:08,401][105692] Updated weights for policy 0, policy_version 1705349 (0.0007) [2023-12-27 03:49:08,462][105692] Updated weights for policy 0, policy_version 1705359 (0.0010) [2023-12-27 03:49:08,516][105692] Updated weights for policy 0, policy_version 1705369 (0.0010) [2023-12-27 03:49:08,944][105620] Updated weights for policy 1, policy_version 1709177 (0.0006) [2023-12-27 03:49:09,004][105620] Updated weights for policy 1, policy_version 1709187 (0.0005) [2023-12-27 03:49:09,070][105620] Updated weights for policy 1, policy_version 1709197 (0.0005) [2023-12-27 03:49:09,364][105692] Updated weights for policy 0, policy_version 1705379 (0.0009) [2023-12-27 03:49:09,433][105692] Updated weights for policy 0, policy_version 1705389 (0.0008) [2023-12-27 03:49:09,485][105692] Updated weights for policy 0, policy_version 1705399 (0.0008) [2023-12-27 03:49:09,757][105620] Updated weights for policy 1, policy_version 1709207 (0.0010) [2023-12-27 03:49:09,821][105620] Updated weights for policy 1, policy_version 1709217 (0.0011) [2023-12-27 03:49:09,887][105620] Updated weights for policy 1, policy_version 1709227 (0.0011) [2023-12-27 03:49:10,247][105692] Updated weights for policy 0, policy_version 1705409 (0.0008) [2023-12-27 03:49:10,298][105692] Updated weights for policy 0, policy_version 1705419 (0.0008) [2023-12-27 03:49:10,358][105692] Updated weights for policy 0, policy_version 1705429 (0.0007) [2023-12-27 03:49:10,407][105692] Updated weights for policy 0, policy_version 1705439 (0.0005) [2023-12-27 03:49:10,670][105620] Updated weights for policy 1, policy_version 1709237 (0.0008) [2023-12-27 03:49:10,727][105620] Updated weights for policy 1, policy_version 1709247 (0.0009) [2023-12-27 03:49:10,782][105620] Updated weights for policy 1, policy_version 1709257 (0.0010) [2023-12-27 03:49:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 874291200. Throughput: 0: 9857.8, 1: 9812.1. Samples: 874300536. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:49:11,062][104569] Avg episode reward: [(0, '8628.854'), (1, '8709.162')] [2023-12-27 03:49:11,136][105692] Updated weights for policy 0, policy_version 1705449 (0.0006) [2023-12-27 03:49:11,202][105692] Updated weights for policy 0, policy_version 1705459 (0.0009) [2023-12-27 03:49:11,264][105692] Updated weights for policy 0, policy_version 1705469 (0.0009) [2023-12-27 03:49:11,487][105620] Updated weights for policy 1, policy_version 1709267 (0.0010) [2023-12-27 03:49:11,549][105620] Updated weights for policy 1, policy_version 1709277 (0.0009) [2023-12-27 03:49:11,608][105620] Updated weights for policy 1, policy_version 1709287 (0.0009) [2023-12-27 03:49:12,023][105692] Updated weights for policy 0, policy_version 1705479 (0.0009) [2023-12-27 03:49:12,082][105692] Updated weights for policy 0, policy_version 1705489 (0.0008) [2023-12-27 03:49:12,146][105692] Updated weights for policy 0, policy_version 1705499 (0.0007) [2023-12-27 03:49:12,399][105620] Updated weights for policy 1, policy_version 1709297 (0.0008) [2023-12-27 03:49:12,453][105620] Updated weights for policy 1, policy_version 1709307 (0.0010) [2023-12-27 03:49:12,508][105620] Updated weights for policy 1, policy_version 1709317 (0.0009) [2023-12-27 03:49:12,562][105620] Updated weights for policy 1, policy_version 1709327 (0.0008) [2023-12-27 03:49:12,800][105692] Updated weights for policy 0, policy_version 1705509 (0.0008) [2023-12-27 03:49:12,858][105692] Updated weights for policy 0, policy_version 1705519 (0.0008) [2023-12-27 03:49:12,920][105692] Updated weights for policy 0, policy_version 1705529 (0.0008) [2023-12-27 03:49:13,323][105620] Updated weights for policy 1, policy_version 1709337 (0.0006) [2023-12-27 03:49:13,379][105620] Updated weights for policy 1, policy_version 1709347 (0.0005) [2023-12-27 03:49:13,442][105620] Updated weights for policy 1, policy_version 1709357 (0.0005) [2023-12-27 03:49:13,543][105692] Updated weights for policy 0, policy_version 1705539 (0.0005) [2023-12-27 03:49:13,613][105692] Updated weights for policy 0, policy_version 1705549 (0.0006) [2023-12-27 03:49:13,678][105692] Updated weights for policy 0, policy_version 1705559 (0.0005) [2023-12-27 03:49:14,046][105620] Updated weights for policy 1, policy_version 1709367 (0.0008) [2023-12-27 03:49:14,094][105620] Updated weights for policy 1, policy_version 1709377 (0.0009) [2023-12-27 03:49:14,140][105620] Updated weights for policy 1, policy_version 1709387 (0.0008) [2023-12-27 03:49:14,264][105692] Updated weights for policy 0, policy_version 1705569 (0.0006) [2023-12-27 03:49:14,316][105692] Updated weights for policy 0, policy_version 1705579 (0.0010) [2023-12-27 03:49:14,360][105692] Updated weights for policy 0, policy_version 1705589 (0.0010) [2023-12-27 03:49:14,405][105692] Updated weights for policy 0, policy_version 1705599 (0.0010) [2023-12-27 03:49:14,888][105620] Updated weights for policy 1, policy_version 1709397 (0.0008) [2023-12-27 03:49:14,959][105620] Updated weights for policy 1, policy_version 1709407 (0.0007) [2023-12-27 03:49:15,016][105620] Updated weights for policy 1, policy_version 1709417 (0.0011) [2023-12-27 03:49:15,052][105692] Updated weights for policy 0, policy_version 1705609 (0.0008) [2023-12-27 03:49:15,104][105692] Updated weights for policy 0, policy_version 1705619 (0.0009) [2023-12-27 03:49:15,160][105692] Updated weights for policy 0, policy_version 1705629 (0.0011) [2023-12-27 03:49:15,739][105620] Updated weights for policy 1, policy_version 1709427 (0.0011) [2023-12-27 03:49:15,800][105620] Updated weights for policy 1, policy_version 1709437 (0.0010) [2023-12-27 03:49:15,862][105620] Updated weights for policy 1, policy_version 1709447 (0.0010) [2023-12-27 03:49:15,881][105692] Updated weights for policy 0, policy_version 1705639 (0.0011) [2023-12-27 03:49:15,946][105692] Updated weights for policy 0, policy_version 1705649 (0.0008) [2023-12-27 03:49:15,989][105692] Updated weights for policy 0, policy_version 1705659 (0.0005) [2023-12-27 03:49:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 874397696. Throughput: 0: 9901.7, 1: 9825.0. Samples: 874360104. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:49:16,062][104569] Avg episode reward: [(0, '8810.425'), (1, '8713.346')] [2023-12-27 03:49:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001705664_436715520.pth... [2023-12-27 03:49:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001709456_437682176.pth... [2023-12-27 03:49:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001704480_436412416.pth [2023-12-27 03:49:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001708304_437387264.pth [2023-12-27 03:49:16,572][105692] Updated weights for policy 0, policy_version 1705669 (0.0008) [2023-12-27 03:49:16,602][105620] Updated weights for policy 1, policy_version 1709457 (0.0010) [2023-12-27 03:49:16,630][105692] Updated weights for policy 0, policy_version 1705679 (0.0010) [2023-12-27 03:49:16,656][105620] Updated weights for policy 1, policy_version 1709467 (0.0010) [2023-12-27 03:49:16,694][105692] Updated weights for policy 0, policy_version 1705689 (0.0008) [2023-12-27 03:49:16,714][105620] Updated weights for policy 1, policy_version 1709477 (0.0010) [2023-12-27 03:49:16,776][105620] Updated weights for policy 1, policy_version 1709487 (0.0010) [2023-12-27 03:49:17,429][105692] Updated weights for policy 0, policy_version 1705699 (0.0008) [2023-12-27 03:49:17,488][105692] Updated weights for policy 0, policy_version 1705709 (0.0011) [2023-12-27 03:49:17,514][105620] Updated weights for policy 1, policy_version 1709497 (0.0010) [2023-12-27 03:49:17,547][105692] Updated weights for policy 0, policy_version 1705719 (0.0010) [2023-12-27 03:49:17,573][105620] Updated weights for policy 1, policy_version 1709507 (0.0010) [2023-12-27 03:49:17,621][105620] Updated weights for policy 1, policy_version 1709517 (0.0010) [2023-12-27 03:49:18,137][105692] Updated weights for policy 0, policy_version 1705729 (0.0011) [2023-12-27 03:49:18,203][105692] Updated weights for policy 0, policy_version 1705739 (0.0011) [2023-12-27 03:49:18,272][105692] Updated weights for policy 0, policy_version 1705749 (0.0006) [2023-12-27 03:49:18,339][105692] Updated weights for policy 0, policy_version 1705759 (0.0006) [2023-12-27 03:49:18,382][105620] Updated weights for policy 1, policy_version 1709527 (0.0011) [2023-12-27 03:49:18,442][105620] Updated weights for policy 1, policy_version 1709537 (0.0011) [2023-12-27 03:49:18,506][105620] Updated weights for policy 1, policy_version 1709547 (0.0011) [2023-12-27 03:49:19,048][105692] Updated weights for policy 0, policy_version 1705769 (0.0008) [2023-12-27 03:49:19,093][105692] Updated weights for policy 0, policy_version 1705779 (0.0008) [2023-12-27 03:49:19,143][105692] Updated weights for policy 0, policy_version 1705789 (0.0008) [2023-12-27 03:49:19,258][105620] Updated weights for policy 1, policy_version 1709557 (0.0011) [2023-12-27 03:49:19,329][105620] Updated weights for policy 1, policy_version 1709567 (0.0011) [2023-12-27 03:49:19,392][105620] Updated weights for policy 1, policy_version 1709577 (0.0011) [2023-12-27 03:49:19,960][105692] Updated weights for policy 0, policy_version 1705799 (0.0007) [2023-12-27 03:49:20,020][105692] Updated weights for policy 0, policy_version 1705809 (0.0007) [2023-12-27 03:49:20,078][105692] Updated weights for policy 0, policy_version 1705819 (0.0008) [2023-12-27 03:49:20,149][105620] Updated weights for policy 1, policy_version 1709587 (0.0011) [2023-12-27 03:49:20,220][105620] Updated weights for policy 1, policy_version 1709597 (0.0011) [2023-12-27 03:49:20,282][105620] Updated weights for policy 1, policy_version 1709607 (0.0011) [2023-12-27 03:49:20,820][105692] Updated weights for policy 0, policy_version 1705829 (0.0008) [2023-12-27 03:49:20,873][105692] Updated weights for policy 0, policy_version 1705839 (0.0007) [2023-12-27 03:49:20,937][105692] Updated weights for policy 0, policy_version 1705849 (0.0009) [2023-12-27 03:49:20,994][105620] Updated weights for policy 1, policy_version 1709617 (0.0010) [2023-12-27 03:49:21,059][105620] Updated weights for policy 1, policy_version 1709627 (0.0007) [2023-12-27 03:49:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 874487808. Throughput: 0: 9914.6, 1: 9819.4. Samples: 874477732. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:49:21,063][104569] Avg episode reward: [(0, '8901.142'), (1, '8897.356')] [2023-12-27 03:49:21,116][105620] Updated weights for policy 1, policy_version 1709637 (0.0006) [2023-12-27 03:49:21,182][105620] Updated weights for policy 1, policy_version 1709647 (0.0011) [2023-12-27 03:49:21,753][105692] Updated weights for policy 0, policy_version 1705859 (0.0008) [2023-12-27 03:49:21,821][105692] Updated weights for policy 0, policy_version 1705869 (0.0006) [2023-12-27 03:49:21,885][105692] Updated weights for policy 0, policy_version 1705879 (0.0009) [2023-12-27 03:49:21,948][105620] Updated weights for policy 1, policy_version 1709657 (0.0006) [2023-12-27 03:49:22,011][105620] Updated weights for policy 1, policy_version 1709667 (0.0006) [2023-12-27 03:49:22,075][105620] Updated weights for policy 1, policy_version 1709677 (0.0006) [2023-12-27 03:49:22,601][105692] Updated weights for policy 0, policy_version 1705889 (0.0008) [2023-12-27 03:49:22,644][105620] Updated weights for policy 1, policy_version 1709687 (0.0006) [2023-12-27 03:49:22,659][105692] Updated weights for policy 0, policy_version 1705899 (0.0008) [2023-12-27 03:49:22,703][105620] Updated weights for policy 1, policy_version 1709697 (0.0008) [2023-12-27 03:49:22,709][105692] Updated weights for policy 0, policy_version 1705909 (0.0006) [2023-12-27 03:49:22,762][105692] Updated weights for policy 0, policy_version 1705919 (0.0006) [2023-12-27 03:49:22,764][105620] Updated weights for policy 1, policy_version 1709707 (0.0009) [2023-12-27 03:49:23,489][105620] Updated weights for policy 1, policy_version 1709717 (0.0008) [2023-12-27 03:49:23,503][105692] Updated weights for policy 0, policy_version 1705929 (0.0010) [2023-12-27 03:49:23,541][105620] Updated weights for policy 1, policy_version 1709727 (0.0005) [2023-12-27 03:49:23,552][105692] Updated weights for policy 0, policy_version 1705939 (0.0010) [2023-12-27 03:49:23,586][105620] Updated weights for policy 1, policy_version 1709737 (0.0007) [2023-12-27 03:49:23,603][105692] Updated weights for policy 0, policy_version 1705949 (0.0010) [2023-12-27 03:49:24,238][105620] Updated weights for policy 1, policy_version 1709747 (0.0009) [2023-12-27 03:49:24,257][105692] Updated weights for policy 0, policy_version 1705959 (0.0008) [2023-12-27 03:49:24,301][105620] Updated weights for policy 1, policy_version 1709757 (0.0008) [2023-12-27 03:49:24,315][105692] Updated weights for policy 0, policy_version 1705969 (0.0006) [2023-12-27 03:49:24,353][105620] Updated weights for policy 1, policy_version 1709767 (0.0010) [2023-12-27 03:49:24,379][105692] Updated weights for policy 0, policy_version 1705979 (0.0005) [2023-12-27 03:49:24,918][105692] Updated weights for policy 0, policy_version 1705989 (0.0008) [2023-12-27 03:49:24,975][105692] Updated weights for policy 0, policy_version 1705999 (0.0011) [2023-12-27 03:49:25,027][105692] Updated weights for policy 0, policy_version 1706009 (0.0011) [2023-12-27 03:49:25,130][105620] Updated weights for policy 1, policy_version 1709777 (0.0010) [2023-12-27 03:49:25,193][105620] Updated weights for policy 1, policy_version 1709787 (0.0005) [2023-12-27 03:49:25,240][105620] Updated weights for policy 1, policy_version 1709797 (0.0010) [2023-12-27 03:49:25,288][105620] Updated weights for policy 1, policy_version 1709807 (0.0010) [2023-12-27 03:49:25,788][105692] Updated weights for policy 0, policy_version 1706019 (0.0011) [2023-12-27 03:49:25,839][105692] Updated weights for policy 0, policy_version 1706029 (0.0010) [2023-12-27 03:49:25,891][105692] Updated weights for policy 0, policy_version 1706039 (0.0010) [2023-12-27 03:49:25,915][105620] Updated weights for policy 1, policy_version 1709817 (0.0006) [2023-12-27 03:49:25,963][105620] Updated weights for policy 1, policy_version 1709827 (0.0005) [2023-12-27 03:49:26,011][105620] Updated weights for policy 1, policy_version 1709837 (0.0009) [2023-12-27 03:49:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 874594304. Throughput: 0: 9874.7, 1: 9812.6. Samples: 874596576. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:49:26,063][104569] Avg episode reward: [(0, '8530.330'), (1, '9261.923')] [2023-12-27 03:49:26,614][105692] Updated weights for policy 0, policy_version 1706049 (0.0010) [2023-12-27 03:49:26,661][105692] Updated weights for policy 0, policy_version 1706059 (0.0010) [2023-12-27 03:49:26,686][105620] Updated weights for policy 1, policy_version 1709847 (0.0010) [2023-12-27 03:49:26,709][105692] Updated weights for policy 0, policy_version 1706069 (0.0010) [2023-12-27 03:49:26,730][105620] Updated weights for policy 1, policy_version 1709857 (0.0010) [2023-12-27 03:49:26,767][105692] Updated weights for policy 0, policy_version 1706079 (0.0010) [2023-12-27 03:49:26,778][105620] Updated weights for policy 1, policy_version 1709867 (0.0010) [2023-12-27 03:49:27,418][105620] Updated weights for policy 1, policy_version 1709877 (0.0008) [2023-12-27 03:49:27,479][105620] Updated weights for policy 1, policy_version 1709887 (0.0009) [2023-12-27 03:49:27,533][105692] Updated weights for policy 0, policy_version 1706089 (0.0011) [2023-12-27 03:49:27,538][105620] Updated weights for policy 1, policy_version 1709897 (0.0010) [2023-12-27 03:49:27,586][105692] Updated weights for policy 0, policy_version 1706099 (0.0011) [2023-12-27 03:49:27,641][105692] Updated weights for policy 0, policy_version 1706109 (0.0010) [2023-12-27 03:49:28,139][105620] Updated weights for policy 1, policy_version 1709907 (0.0009) [2023-12-27 03:49:28,197][105620] Updated weights for policy 1, policy_version 1709917 (0.0005) [2023-12-27 03:49:28,243][105620] Updated weights for policy 1, policy_version 1709927 (0.0005) [2023-12-27 03:49:28,289][105692] Updated weights for policy 0, policy_version 1706119 (0.0009) [2023-12-27 03:49:28,356][105692] Updated weights for policy 0, policy_version 1706129 (0.0011) [2023-12-27 03:49:28,416][105692] Updated weights for policy 0, policy_version 1706139 (0.0007) [2023-12-27 03:49:28,887][105620] Updated weights for policy 1, policy_version 1709937 (0.0006) [2023-12-27 03:49:28,946][105620] Updated weights for policy 1, policy_version 1709947 (0.0010) [2023-12-27 03:49:28,949][105692] Updated weights for policy 0, policy_version 1706149 (0.0006) [2023-12-27 03:49:28,996][105692] Updated weights for policy 0, policy_version 1706159 (0.0005) [2023-12-27 03:49:29,004][105620] Updated weights for policy 1, policy_version 1709957 (0.0010) [2023-12-27 03:49:29,041][105692] Updated weights for policy 0, policy_version 1706169 (0.0005) [2023-12-27 03:49:29,069][105620] Updated weights for policy 1, policy_version 1709967 (0.0005) [2023-12-27 03:49:29,722][105620] Updated weights for policy 1, policy_version 1709977 (0.0005) [2023-12-27 03:49:29,766][105692] Updated weights for policy 0, policy_version 1706179 (0.0008) [2023-12-27 03:49:29,780][105620] Updated weights for policy 1, policy_version 1709987 (0.0006) [2023-12-27 03:49:29,830][105692] Updated weights for policy 0, policy_version 1706189 (0.0011) [2023-12-27 03:49:29,832][105620] Updated weights for policy 1, policy_version 1709997 (0.0008) [2023-12-27 03:49:29,892][105692] Updated weights for policy 0, policy_version 1706199 (0.0011) [2023-12-27 03:49:30,504][105620] Updated weights for policy 1, policy_version 1710007 (0.0006) [2023-12-27 03:49:30,514][105692] Updated weights for policy 0, policy_version 1706209 (0.0010) [2023-12-27 03:49:30,564][105692] Updated weights for policy 0, policy_version 1706219 (0.0006) [2023-12-27 03:49:30,567][105620] Updated weights for policy 1, policy_version 1710017 (0.0007) [2023-12-27 03:49:30,621][105692] Updated weights for policy 0, policy_version 1706229 (0.0005) [2023-12-27 03:49:30,624][105620] Updated weights for policy 1, policy_version 1710027 (0.0006) [2023-12-27 03:49:30,673][105692] Updated weights for policy 0, policy_version 1706239 (0.0005) [2023-12-27 03:49:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 874692608. Throughput: 0: 9900.7, 1: 9923.9. Samples: 874659112. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:49:31,063][104569] Avg episode reward: [(0, '8345.956'), (1, '9078.712')] [2023-12-27 03:49:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001706240_436862976.pth... [2023-12-27 03:49:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001710032_437829632.pth... [2023-12-27 03:49:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001705088_436568064.pth [2023-12-27 03:49:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001708880_437534720.pth [2023-12-27 03:49:31,167][105620] Updated weights for policy 1, policy_version 1710037 (0.0006) [2023-12-27 03:49:31,221][105620] Updated weights for policy 1, policy_version 1710047 (0.0005) [2023-12-27 03:49:31,244][105692] Updated weights for policy 0, policy_version 1706249 (0.0007) [2023-12-27 03:49:31,283][105620] Updated weights for policy 1, policy_version 1710057 (0.0008) [2023-12-27 03:49:31,307][105692] Updated weights for policy 0, policy_version 1706259 (0.0006) [2023-12-27 03:49:31,373][105692] Updated weights for policy 0, policy_version 1706269 (0.0009) [2023-12-27 03:49:32,004][105692] Updated weights for policy 0, policy_version 1706279 (0.0007) [2023-12-27 03:49:32,058][105620] Updated weights for policy 1, policy_version 1710067 (0.0007) [2023-12-27 03:49:32,060][105692] Updated weights for policy 0, policy_version 1706289 (0.0008) [2023-12-27 03:49:32,112][105620] Updated weights for policy 1, policy_version 1710077 (0.0006) [2023-12-27 03:49:32,115][105692] Updated weights for policy 0, policy_version 1706299 (0.0007) [2023-12-27 03:49:32,165][105620] Updated weights for policy 1, policy_version 1710087 (0.0007) [2023-12-27 03:49:32,871][105692] Updated weights for policy 0, policy_version 1706309 (0.0009) [2023-12-27 03:49:32,897][105620] Updated weights for policy 1, policy_version 1710097 (0.0009) [2023-12-27 03:49:32,915][105692] Updated weights for policy 0, policy_version 1706319 (0.0008) [2023-12-27 03:49:32,945][105620] Updated weights for policy 1, policy_version 1710107 (0.0006) [2023-12-27 03:49:32,967][105692] Updated weights for policy 0, policy_version 1706329 (0.0008) [2023-12-27 03:49:32,996][105620] Updated weights for policy 1, policy_version 1710117 (0.0006) [2023-12-27 03:49:33,045][105620] Updated weights for policy 1, policy_version 1710127 (0.0008) [2023-12-27 03:49:33,692][105692] Updated weights for policy 0, policy_version 1706339 (0.0008) [2023-12-27 03:49:33,751][105692] Updated weights for policy 0, policy_version 1706349 (0.0006) [2023-12-27 03:49:33,802][105692] Updated weights for policy 0, policy_version 1706359 (0.0008) [2023-12-27 03:49:33,839][105620] Updated weights for policy 1, policy_version 1710137 (0.0008) [2023-12-27 03:49:33,896][105620] Updated weights for policy 1, policy_version 1710147 (0.0006) [2023-12-27 03:49:33,951][105620] Updated weights for policy 1, policy_version 1710157 (0.0009) [2023-12-27 03:49:34,528][105692] Updated weights for policy 0, policy_version 1706369 (0.0008) [2023-12-27 03:49:34,591][105692] Updated weights for policy 0, policy_version 1706379 (0.0008) [2023-12-27 03:49:34,646][105692] Updated weights for policy 0, policy_version 1706389 (0.0008) [2023-12-27 03:49:34,704][105692] Updated weights for policy 0, policy_version 1706399 (0.0008) [2023-12-27 03:49:34,727][105620] Updated weights for policy 1, policy_version 1710167 (0.0008) [2023-12-27 03:49:34,803][105620] Updated weights for policy 1, policy_version 1710177 (0.0009) [2023-12-27 03:49:34,868][105620] Updated weights for policy 1, policy_version 1710187 (0.0009) [2023-12-27 03:49:35,354][105692] Updated weights for policy 0, policy_version 1706409 (0.0006) [2023-12-27 03:49:35,402][105692] Updated weights for policy 0, policy_version 1706419 (0.0005) [2023-12-27 03:49:35,454][105692] Updated weights for policy 0, policy_version 1706429 (0.0005) [2023-12-27 03:49:35,685][105620] Updated weights for policy 1, policy_version 1710197 (0.0007) [2023-12-27 03:49:35,733][105620] Updated weights for policy 1, policy_version 1710207 (0.0009) [2023-12-27 03:49:35,786][105620] Updated weights for policy 1, policy_version 1710218 (0.0009) [2023-12-27 03:49:36,024][105692] Updated weights for policy 0, policy_version 1706439 (0.0006) [2023-12-27 03:49:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 874790912. Throughput: 0: 9988.6, 1: 9826.2. Samples: 874780488. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:49:36,062][104569] Avg episode reward: [(0, '8628.718'), (1, '8901.642')] [2023-12-27 03:49:36,071][105692] Updated weights for policy 0, policy_version 1706449 (0.0009) [2023-12-27 03:49:36,133][105692] Updated weights for policy 0, policy_version 1706459 (0.0010) [2023-12-27 03:49:36,626][105620] Updated weights for policy 1, policy_version 1710228 (0.0009) [2023-12-27 03:49:36,676][105620] Updated weights for policy 1, policy_version 1710238 (0.0008) [2023-12-27 03:49:36,726][105620] Updated weights for policy 1, policy_version 1710248 (0.0009) [2023-12-27 03:49:36,879][105692] Updated weights for policy 0, policy_version 1706469 (0.0009) [2023-12-27 03:49:36,938][105692] Updated weights for policy 0, policy_version 1706479 (0.0009) [2023-12-27 03:49:36,985][105692] Updated weights for policy 0, policy_version 1706489 (0.0007) [2023-12-27 03:49:37,576][105692] Updated weights for policy 0, policy_version 1706499 (0.0007) [2023-12-27 03:49:37,578][105620] Updated weights for policy 1, policy_version 1710258 (0.0008) [2023-12-27 03:49:37,631][105692] Updated weights for policy 0, policy_version 1706509 (0.0010) [2023-12-27 03:49:37,638][105620] Updated weights for policy 1, policy_version 1710268 (0.0006) [2023-12-27 03:49:37,686][105692] Updated weights for policy 0, policy_version 1706519 (0.0006) [2023-12-27 03:49:37,697][105620] Updated weights for policy 1, policy_version 1710278 (0.0008) [2023-12-27 03:49:37,749][105620] Updated weights for policy 1, policy_version 1710288 (0.0009) [2023-12-27 03:49:38,278][105692] Updated weights for policy 0, policy_version 1706529 (0.0006) [2023-12-27 03:49:38,340][105692] Updated weights for policy 0, policy_version 1706539 (0.0010) [2023-12-27 03:49:38,399][105692] Updated weights for policy 0, policy_version 1706549 (0.0009) [2023-12-27 03:49:38,458][105692] Updated weights for policy 0, policy_version 1706559 (0.0009) [2023-12-27 03:49:38,593][105620] Updated weights for policy 1, policy_version 1710298 (0.0008) [2023-12-27 03:49:38,651][105620] Updated weights for policy 1, policy_version 1710308 (0.0009) [2023-12-27 03:49:38,713][105620] Updated weights for policy 1, policy_version 1710318 (0.0009) [2023-12-27 03:49:39,178][105692] Updated weights for policy 0, policy_version 1706569 (0.0009) [2023-12-27 03:49:39,240][105692] Updated weights for policy 0, policy_version 1706579 (0.0009) [2023-12-27 03:49:39,307][105692] Updated weights for policy 0, policy_version 1706589 (0.0009) [2023-12-27 03:49:39,488][105620] Updated weights for policy 1, policy_version 1710328 (0.0009) [2023-12-27 03:49:39,545][105620] Updated weights for policy 1, policy_version 1710338 (0.0009) [2023-12-27 03:49:39,607][105620] Updated weights for policy 1, policy_version 1710348 (0.0010) [2023-12-27 03:49:40,032][105692] Updated weights for policy 0, policy_version 1706599 (0.0010) [2023-12-27 03:49:40,088][105692] Updated weights for policy 0, policy_version 1706609 (0.0009) [2023-12-27 03:49:40,151][105692] Updated weights for policy 0, policy_version 1706619 (0.0009) [2023-12-27 03:49:40,380][105620] Updated weights for policy 1, policy_version 1710358 (0.0009) [2023-12-27 03:49:40,439][105620] Updated weights for policy 1, policy_version 1710368 (0.0009) [2023-12-27 03:49:40,522][105620] Updated weights for policy 1, policy_version 1710378 (0.0008) [2023-12-27 03:49:40,906][105692] Updated weights for policy 0, policy_version 1706629 (0.0007) [2023-12-27 03:49:40,963][105692] Updated weights for policy 0, policy_version 1706639 (0.0006) [2023-12-27 03:49:41,013][105692] Updated weights for policy 0, policy_version 1706649 (0.0005) [2023-12-27 03:49:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 874889216. Throughput: 0: 9990.6, 1: 9611.5. Samples: 874894436. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:49:41,063][104569] Avg episode reward: [(0, '8452.633'), (1, '8901.622')] [2023-12-27 03:49:41,289][105620] Updated weights for policy 1, policy_version 1710388 (0.0009) [2023-12-27 03:49:41,347][105620] Updated weights for policy 1, policy_version 1710398 (0.0009) [2023-12-27 03:49:41,417][105620] Updated weights for policy 1, policy_version 1710408 (0.0009) [2023-12-27 03:49:41,765][105692] Updated weights for policy 0, policy_version 1706659 (0.0008) [2023-12-27 03:49:41,818][105692] Updated weights for policy 0, policy_version 1706669 (0.0010) [2023-12-27 03:49:41,871][105692] Updated weights for policy 0, policy_version 1706679 (0.0010) [2023-12-27 03:49:42,087][105620] Updated weights for policy 1, policy_version 1710418 (0.0009) [2023-12-27 03:49:42,141][105620] Updated weights for policy 1, policy_version 1710428 (0.0011) [2023-12-27 03:49:42,198][105620] Updated weights for policy 1, policy_version 1710438 (0.0010) [2023-12-27 03:49:42,261][105620] Updated weights for policy 1, policy_version 1710448 (0.0011) [2023-12-27 03:49:42,708][105692] Updated weights for policy 0, policy_version 1706689 (0.0009) [2023-12-27 03:49:42,771][105692] Updated weights for policy 0, policy_version 1706699 (0.0008) [2023-12-27 03:49:42,830][105692] Updated weights for policy 0, policy_version 1706709 (0.0008) [2023-12-27 03:49:42,886][105692] Updated weights for policy 0, policy_version 1706719 (0.0008) [2023-12-27 03:49:43,002][105620] Updated weights for policy 1, policy_version 1710458 (0.0011) [2023-12-27 03:49:43,062][105620] Updated weights for policy 1, policy_version 1710468 (0.0011) [2023-12-27 03:49:43,122][105620] Updated weights for policy 1, policy_version 1710478 (0.0011) [2023-12-27 03:49:43,639][105692] Updated weights for policy 0, policy_version 1706729 (0.0010) [2023-12-27 03:49:43,704][105692] Updated weights for policy 0, policy_version 1706739 (0.0010) [2023-12-27 03:49:43,725][105620] Updated weights for policy 1, policy_version 1710488 (0.0006) [2023-12-27 03:49:43,763][105692] Updated weights for policy 0, policy_version 1706749 (0.0010) [2023-12-27 03:49:43,779][105620] Updated weights for policy 1, policy_version 1710498 (0.0005) [2023-12-27 03:49:43,825][105620] Updated weights for policy 1, policy_version 1710508 (0.0005) [2023-12-27 03:49:44,392][105620] Updated weights for policy 1, policy_version 1710518 (0.0005) [2023-12-27 03:49:44,440][105692] Updated weights for policy 0, policy_version 1706759 (0.0009) [2023-12-27 03:49:44,448][105620] Updated weights for policy 1, policy_version 1710528 (0.0007) [2023-12-27 03:49:44,495][105620] Updated weights for policy 1, policy_version 1710538 (0.0010) [2023-12-27 03:49:44,504][105692] Updated weights for policy 0, policy_version 1706769 (0.0008) [2023-12-27 03:49:44,566][105692] Updated weights for policy 0, policy_version 1706779 (0.0005) [2023-12-27 03:49:45,157][105620] Updated weights for policy 1, policy_version 1710548 (0.0010) [2023-12-27 03:49:45,220][105620] Updated weights for policy 1, policy_version 1710558 (0.0011) [2023-12-27 03:49:45,232][105692] Updated weights for policy 0, policy_version 1706789 (0.0009) [2023-12-27 03:49:45,280][105620] Updated weights for policy 1, policy_version 1710568 (0.0011) [2023-12-27 03:49:45,293][105692] Updated weights for policy 0, policy_version 1706799 (0.0011) [2023-12-27 03:49:45,352][105692] Updated weights for policy 0, policy_version 1706809 (0.0011) [2023-12-27 03:49:46,008][105620] Updated weights for policy 1, policy_version 1710578 (0.0011) [2023-12-27 03:49:46,018][105692] Updated weights for policy 0, policy_version 1706819 (0.0010) [2023-12-27 03:49:46,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 874979328. Throughput: 0: 9889.5, 1: 9652.5. Samples: 874952888. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:49:46,062][104569] Avg episode reward: [(0, '8898.584'), (1, '8616.884')] [2023-12-27 03:49:46,065][105620] Updated weights for policy 1, policy_version 1710588 (0.0007) [2023-12-27 03:49:46,067][105692] Updated weights for policy 0, policy_version 1706829 (0.0007) [2023-12-27 03:49:46,123][105692] Updated weights for policy 0, policy_version 1706839 (0.0007) [2023-12-27 03:49:46,125][105620] Updated weights for policy 1, policy_version 1710598 (0.0006) [2023-12-27 03:49:46,161][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001706848_437018624.pth... [2023-12-27 03:49:46,164][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001705664_436715520.pth [2023-12-27 03:49:46,180][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001710608_437977088.pth... [2023-12-27 03:49:46,184][105620] Updated weights for policy 1, policy_version 1710608 (0.0009) [2023-12-27 03:49:46,184][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001709456_437682176.pth [2023-12-27 03:49:46,842][105692] Updated weights for policy 0, policy_version 1706849 (0.0006) [2023-12-27 03:49:46,896][105692] Updated weights for policy 0, policy_version 1706859 (0.0009) [2023-12-27 03:49:46,941][105692] Updated weights for policy 0, policy_version 1706869 (0.0007) [2023-12-27 03:49:46,956][105620] Updated weights for policy 1, policy_version 1710618 (0.0008) [2023-12-27 03:49:46,990][105692] Updated weights for policy 0, policy_version 1706879 (0.0006) [2023-12-27 03:49:47,020][105620] Updated weights for policy 1, policy_version 1710628 (0.0009) [2023-12-27 03:49:47,068][105620] Updated weights for policy 1, policy_version 1710638 (0.0009) [2023-12-27 03:49:47,647][105620] Updated weights for policy 1, policy_version 1710648 (0.0006) [2023-12-27 03:49:47,706][105620] Updated weights for policy 1, policy_version 1710658 (0.0005) [2023-12-27 03:49:47,771][105620] Updated weights for policy 1, policy_version 1710669 (0.0009) [2023-12-27 03:49:47,836][105692] Updated weights for policy 0, policy_version 1706889 (0.0005) [2023-12-27 03:49:47,881][105692] Updated weights for policy 0, policy_version 1706899 (0.0005) [2023-12-27 03:49:47,925][105692] Updated weights for policy 0, policy_version 1706909 (0.0006) [2023-12-27 03:49:48,486][105620] Updated weights for policy 1, policy_version 1710679 (0.0009) [2023-12-27 03:49:48,546][105620] Updated weights for policy 1, policy_version 1710689 (0.0009) [2023-12-27 03:49:48,604][105620] Updated weights for policy 1, policy_version 1710699 (0.0009) [2023-12-27 03:49:48,640][105692] Updated weights for policy 0, policy_version 1706919 (0.0008) [2023-12-27 03:49:48,699][105692] Updated weights for policy 0, policy_version 1706929 (0.0009) [2023-12-27 03:49:48,758][105692] Updated weights for policy 0, policy_version 1706939 (0.0009) [2023-12-27 03:49:49,336][105620] Updated weights for policy 1, policy_version 1710709 (0.0008) [2023-12-27 03:49:49,407][105620] Updated weights for policy 1, policy_version 1710719 (0.0007) [2023-12-27 03:49:49,479][105620] Updated weights for policy 1, policy_version 1710729 (0.0008) [2023-12-27 03:49:49,556][105692] Updated weights for policy 0, policy_version 1706949 (0.0009) [2023-12-27 03:49:49,618][105692] Updated weights for policy 0, policy_version 1706959 (0.0007) [2023-12-27 03:49:49,685][105692] Updated weights for policy 0, policy_version 1706969 (0.0006) [2023-12-27 03:49:50,300][105620] Updated weights for policy 1, policy_version 1710739 (0.0009) [2023-12-27 03:49:50,359][105620] Updated weights for policy 1, policy_version 1710749 (0.0010) [2023-12-27 03:49:50,421][105620] Updated weights for policy 1, policy_version 1710759 (0.0010) [2023-12-27 03:49:50,459][105692] Updated weights for policy 0, policy_version 1706979 (0.0006) [2023-12-27 03:49:50,513][105692] Updated weights for policy 0, policy_version 1706989 (0.0007) [2023-12-27 03:49:50,570][105692] Updated weights for policy 0, policy_version 1706999 (0.0007) [2023-12-27 03:49:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 875077632. Throughput: 0: 9886.6, 1: 9731.0. Samples: 875069880. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:49:51,062][104569] Avg episode reward: [(0, '8985.409'), (1, '8641.850')] [2023-12-27 03:49:51,115][105620] Updated weights for policy 1, policy_version 1710769 (0.0010) [2023-12-27 03:49:51,179][105620] Updated weights for policy 1, policy_version 1710779 (0.0006) [2023-12-27 03:49:51,240][105620] Updated weights for policy 1, policy_version 1710789 (0.0006) [2023-12-27 03:49:51,306][105620] Updated weights for policy 1, policy_version 1710799 (0.0007) [2023-12-27 03:49:51,439][105692] Updated weights for policy 0, policy_version 1707009 (0.0009) [2023-12-27 03:49:51,497][105692] Updated weights for policy 0, policy_version 1707019 (0.0006) [2023-12-27 03:49:51,557][105692] Updated weights for policy 0, policy_version 1707029 (0.0005) [2023-12-27 03:49:51,620][105692] Updated weights for policy 0, policy_version 1707039 (0.0006) [2023-12-27 03:49:52,035][105620] Updated weights for policy 1, policy_version 1710809 (0.0010) [2023-12-27 03:49:52,090][105620] Updated weights for policy 1, policy_version 1710819 (0.0010) [2023-12-27 03:49:52,155][105620] Updated weights for policy 1, policy_version 1710829 (0.0010) [2023-12-27 03:49:52,303][105692] Updated weights for policy 0, policy_version 1707049 (0.0008) [2023-12-27 03:49:52,366][105692] Updated weights for policy 0, policy_version 1707059 (0.0008) [2023-12-27 03:49:52,431][105692] Updated weights for policy 0, policy_version 1707069 (0.0008) [2023-12-27 03:49:52,880][105620] Updated weights for policy 1, policy_version 1710839 (0.0007) [2023-12-27 03:49:52,936][105620] Updated weights for policy 1, policy_version 1710849 (0.0009) [2023-12-27 03:49:52,986][105620] Updated weights for policy 1, policy_version 1710859 (0.0008) [2023-12-27 03:49:53,203][105692] Updated weights for policy 0, policy_version 1707079 (0.0006) [2023-12-27 03:49:53,258][105692] Updated weights for policy 0, policy_version 1707089 (0.0005) [2023-12-27 03:49:53,324][105692] Updated weights for policy 0, policy_version 1707099 (0.0007) [2023-12-27 03:49:53,703][105620] Updated weights for policy 1, policy_version 1710869 (0.0009) [2023-12-27 03:49:53,748][105620] Updated weights for policy 1, policy_version 1710879 (0.0006) [2023-12-27 03:49:53,807][105620] Updated weights for policy 1, policy_version 1710889 (0.0006) [2023-12-27 03:49:53,894][105692] Updated weights for policy 0, policy_version 1707109 (0.0006) [2023-12-27 03:49:53,946][105692] Updated weights for policy 0, policy_version 1707119 (0.0005) [2023-12-27 03:49:53,998][105692] Updated weights for policy 0, policy_version 1707129 (0.0006) [2023-12-27 03:49:54,462][105620] Updated weights for policy 1, policy_version 1710899 (0.0006) [2023-12-27 03:49:54,513][105620] Updated weights for policy 1, policy_version 1710909 (0.0009) [2023-12-27 03:49:54,568][105620] Updated weights for policy 1, policy_version 1710919 (0.0009) [2023-12-27 03:49:54,693][105692] Updated weights for policy 0, policy_version 1707139 (0.0010) [2023-12-27 03:49:54,751][105692] Updated weights for policy 0, policy_version 1707150 (0.0010) [2023-12-27 03:49:54,814][105692] Updated weights for policy 0, policy_version 1707160 (0.0010) [2023-12-27 03:49:55,237][105620] Updated weights for policy 1, policy_version 1710929 (0.0010) [2023-12-27 03:49:55,300][105620] Updated weights for policy 1, policy_version 1710939 (0.0010) [2023-12-27 03:49:55,354][105620] Updated weights for policy 1, policy_version 1710949 (0.0010) [2023-12-27 03:49:55,412][105620] Updated weights for policy 1, policy_version 1710959 (0.0010) [2023-12-27 03:49:55,552][105692] Updated weights for policy 0, policy_version 1707170 (0.0010) [2023-12-27 03:49:55,610][105692] Updated weights for policy 0, policy_version 1707180 (0.0010) [2023-12-27 03:49:55,661][105692] Updated weights for policy 0, policy_version 1707190 (0.0008) [2023-12-27 03:49:55,959][105620] Updated weights for policy 1, policy_version 1710969 (0.0010) [2023-12-27 03:49:56,007][105620] Updated weights for policy 1, policy_version 1710979 (0.0010) [2023-12-27 03:49:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 875175936. Throughput: 0: 9949.1, 1: 9772.3. Samples: 875187996. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:49:56,062][104569] Avg episode reward: [(0, '8988.591'), (1, '9102.909')] [2023-12-27 03:49:56,066][105620] Updated weights for policy 1, policy_version 1710989 (0.0010) [2023-12-27 03:49:56,431][105692] Updated weights for policy 0, policy_version 1707201 (0.0010) [2023-12-27 03:49:56,482][105692] Updated weights for policy 0, policy_version 1707211 (0.0008) [2023-12-27 03:49:56,534][105692] Updated weights for policy 0, policy_version 1707221 (0.0008) [2023-12-27 03:49:56,582][105692] Updated weights for policy 0, policy_version 1707231 (0.0010) [2023-12-27 03:49:56,825][105620] Updated weights for policy 1, policy_version 1710999 (0.0010) [2023-12-27 03:49:56,876][105620] Updated weights for policy 1, policy_version 1711009 (0.0010) [2023-12-27 03:49:56,931][105620] Updated weights for policy 1, policy_version 1711019 (0.0010) [2023-12-27 03:49:57,315][105692] Updated weights for policy 0, policy_version 1707241 (0.0006) [2023-12-27 03:49:57,370][105692] Updated weights for policy 0, policy_version 1707251 (0.0006) [2023-12-27 03:49:57,424][105692] Updated weights for policy 0, policy_version 1707261 (0.0005) [2023-12-27 03:49:57,585][105620] Updated weights for policy 1, policy_version 1711029 (0.0008) [2023-12-27 03:49:57,630][105620] Updated weights for policy 1, policy_version 1711039 (0.0005) [2023-12-27 03:49:57,686][105620] Updated weights for policy 1, policy_version 1711049 (0.0005) [2023-12-27 03:49:57,946][105692] Updated weights for policy 0, policy_version 1707271 (0.0005) [2023-12-27 03:49:57,998][105692] Updated weights for policy 0, policy_version 1707281 (0.0005) [2023-12-27 03:49:58,047][105692] Updated weights for policy 0, policy_version 1707291 (0.0005) [2023-12-27 03:49:58,363][105620] Updated weights for policy 1, policy_version 1711059 (0.0006) [2023-12-27 03:49:58,426][105620] Updated weights for policy 1, policy_version 1711069 (0.0007) [2023-12-27 03:49:58,489][105620] Updated weights for policy 1, policy_version 1711079 (0.0008) [2023-12-27 03:49:58,765][105692] Updated weights for policy 0, policy_version 1707301 (0.0007) [2023-12-27 03:49:58,833][105692] Updated weights for policy 0, policy_version 1707311 (0.0008) [2023-12-27 03:49:58,912][105692] Updated weights for policy 0, policy_version 1707321 (0.0008) [2023-12-27 03:49:59,320][105620] Updated weights for policy 1, policy_version 1711089 (0.0008) [2023-12-27 03:49:59,382][105620] Updated weights for policy 1, policy_version 1711099 (0.0008) [2023-12-27 03:49:59,447][105620] Updated weights for policy 1, policy_version 1711109 (0.0008) [2023-12-27 03:49:59,506][105620] Updated weights for policy 1, policy_version 1711119 (0.0008) [2023-12-27 03:49:59,619][105692] Updated weights for policy 0, policy_version 1707331 (0.0008) [2023-12-27 03:49:59,682][105692] Updated weights for policy 0, policy_version 1707341 (0.0008) [2023-12-27 03:49:59,738][105692] Updated weights for policy 0, policy_version 1707351 (0.0008) [2023-12-27 03:50:00,234][105620] Updated weights for policy 1, policy_version 1711129 (0.0006) [2023-12-27 03:50:00,297][105620] Updated weights for policy 1, policy_version 1711139 (0.0005) [2023-12-27 03:50:00,363][105620] Updated weights for policy 1, policy_version 1711149 (0.0008) [2023-12-27 03:50:00,433][105692] Updated weights for policy 0, policy_version 1707361 (0.0008) [2023-12-27 03:50:00,485][105692] Updated weights for policy 0, policy_version 1707371 (0.0009) [2023-12-27 03:50:00,542][105692] Updated weights for policy 0, policy_version 1707381 (0.0006) [2023-12-27 03:50:00,601][105692] Updated weights for policy 0, policy_version 1707391 (0.0006) [2023-12-27 03:50:00,977][105620] Updated weights for policy 1, policy_version 1711159 (0.0006) [2023-12-27 03:50:01,039][105620] Updated weights for policy 1, policy_version 1711169 (0.0006) [2023-12-27 03:50:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 875274240. Throughput: 0: 9969.1, 1: 9768.1. Samples: 875248276. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:50:01,062][104569] Avg episode reward: [(0, '9078.078'), (1, '8987.854')] [2023-12-27 03:50:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001707392_437157888.pth... [2023-12-27 03:50:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001706240_436862976.pth [2023-12-27 03:50:01,102][105620] Updated weights for policy 1, policy_version 1711179 (0.0011) [2023-12-27 03:50:01,134][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001711184_438124544.pth... [2023-12-27 03:50:01,138][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001710032_437829632.pth [2023-12-27 03:50:01,192][105692] Updated weights for policy 0, policy_version 1707401 (0.0009) [2023-12-27 03:50:01,251][105692] Updated weights for policy 0, policy_version 1707411 (0.0010) [2023-12-27 03:50:01,311][105692] Updated weights for policy 0, policy_version 1707421 (0.0009) [2023-12-27 03:50:01,777][105620] Updated weights for policy 1, policy_version 1711189 (0.0009) [2023-12-27 03:50:01,830][105620] Updated weights for policy 1, policy_version 1711199 (0.0005) [2023-12-27 03:50:01,887][105620] Updated weights for policy 1, policy_version 1711209 (0.0005) [2023-12-27 03:50:01,973][105692] Updated weights for policy 0, policy_version 1707431 (0.0006) [2023-12-27 03:50:02,029][105692] Updated weights for policy 0, policy_version 1707441 (0.0005) [2023-12-27 03:50:02,088][105692] Updated weights for policy 0, policy_version 1707451 (0.0005) [2023-12-27 03:50:02,506][105620] Updated weights for policy 1, policy_version 1711219 (0.0006) [2023-12-27 03:50:02,564][105620] Updated weights for policy 1, policy_version 1711229 (0.0010) [2023-12-27 03:50:02,628][105620] Updated weights for policy 1, policy_version 1711239 (0.0009) [2023-12-27 03:50:02,659][105692] Updated weights for policy 0, policy_version 1707461 (0.0008) [2023-12-27 03:50:02,713][105692] Updated weights for policy 0, policy_version 1707471 (0.0008) [2023-12-27 03:50:02,774][105692] Updated weights for policy 0, policy_version 1707481 (0.0005) [2023-12-27 03:50:03,374][105620] Updated weights for policy 1, policy_version 1711249 (0.0008) [2023-12-27 03:50:03,417][105692] Updated weights for policy 0, policy_version 1707491 (0.0007) [2023-12-27 03:50:03,422][105620] Updated weights for policy 1, policy_version 1711259 (0.0010) [2023-12-27 03:50:03,470][105620] Updated weights for policy 1, policy_version 1711269 (0.0010) [2023-12-27 03:50:03,471][105692] Updated weights for policy 0, policy_version 1707501 (0.0010) [2023-12-27 03:50:03,521][105692] Updated weights for policy 0, policy_version 1707511 (0.0007) [2023-12-27 03:50:03,521][105620] Updated weights for policy 1, policy_version 1711279 (0.0010) [2023-12-27 03:50:04,243][105620] Updated weights for policy 1, policy_version 1711289 (0.0009) [2023-12-27 03:50:04,287][105692] Updated weights for policy 0, policy_version 1707521 (0.0008) [2023-12-27 03:50:04,300][105620] Updated weights for policy 1, policy_version 1711299 (0.0008) [2023-12-27 03:50:04,348][105692] Updated weights for policy 0, policy_version 1707531 (0.0006) [2023-12-27 03:50:04,355][105620] Updated weights for policy 1, policy_version 1711309 (0.0008) [2023-12-27 03:50:04,403][105692] Updated weights for policy 0, policy_version 1707541 (0.0008) [2023-12-27 03:50:04,460][105692] Updated weights for policy 0, policy_version 1707551 (0.0008) [2023-12-27 03:50:05,075][105692] Updated weights for policy 0, policy_version 1707561 (0.0008) [2023-12-27 03:50:05,124][105692] Updated weights for policy 0, policy_version 1707571 (0.0008) [2023-12-27 03:50:05,130][105620] Updated weights for policy 1, policy_version 1711319 (0.0008) [2023-12-27 03:50:05,176][105692] Updated weights for policy 0, policy_version 1707581 (0.0007) [2023-12-27 03:50:05,194][105620] Updated weights for policy 1, policy_version 1711329 (0.0008) [2023-12-27 03:50:05,251][105620] Updated weights for policy 1, policy_version 1711339 (0.0009) [2023-12-27 03:50:05,847][105692] Updated weights for policy 0, policy_version 1707591 (0.0006) [2023-12-27 03:50:05,909][105692] Updated weights for policy 0, policy_version 1707601 (0.0010) [2023-12-27 03:50:05,946][105620] Updated weights for policy 1, policy_version 1711349 (0.0007) [2023-12-27 03:50:05,970][105692] Updated weights for policy 0, policy_version 1707611 (0.0009) [2023-12-27 03:50:06,004][105620] Updated weights for policy 1, policy_version 1711359 (0.0008) [2023-12-27 03:50:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 875380736. Throughput: 0: 9961.6, 1: 9834.6. Samples: 875368560. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:50:06,062][104569] Avg episode reward: [(0, '8713.215'), (1, '8806.409')] [2023-12-27 03:50:06,064][105620] Updated weights for policy 1, policy_version 1711369 (0.0010) [2023-12-27 03:50:06,583][105692] Updated weights for policy 0, policy_version 1707621 (0.0007) [2023-12-27 03:50:06,650][105692] Updated weights for policy 0, policy_version 1707631 (0.0009) [2023-12-27 03:50:06,708][105692] Updated weights for policy 0, policy_version 1707641 (0.0010) [2023-12-27 03:50:06,762][105620] Updated weights for policy 1, policy_version 1711379 (0.0010) [2023-12-27 03:50:06,816][105620] Updated weights for policy 1, policy_version 1711389 (0.0010) [2023-12-27 03:50:06,865][105620] Updated weights for policy 1, policy_version 1711399 (0.0010) [2023-12-27 03:50:07,307][105692] Updated weights for policy 0, policy_version 1707651 (0.0007) [2023-12-27 03:50:07,367][105692] Updated weights for policy 0, policy_version 1707661 (0.0006) [2023-12-27 03:50:07,420][105692] Updated weights for policy 0, policy_version 1707671 (0.0005) [2023-12-27 03:50:07,555][105620] Updated weights for policy 1, policy_version 1711409 (0.0010) [2023-12-27 03:50:07,625][105620] Updated weights for policy 1, policy_version 1711419 (0.0005) [2023-12-27 03:50:07,688][105620] Updated weights for policy 1, policy_version 1711429 (0.0008) [2023-12-27 03:50:07,747][105620] Updated weights for policy 1, policy_version 1711439 (0.0010) [2023-12-27 03:50:08,109][105692] Updated weights for policy 0, policy_version 1707681 (0.0006) [2023-12-27 03:50:08,166][105692] Updated weights for policy 0, policy_version 1707691 (0.0010) [2023-12-27 03:50:08,221][105692] Updated weights for policy 0, policy_version 1707701 (0.0010) [2023-12-27 03:50:08,281][105692] Updated weights for policy 0, policy_version 1707711 (0.0006) [2023-12-27 03:50:08,329][105620] Updated weights for policy 1, policy_version 1711450 (0.0008) [2023-12-27 03:50:08,393][105620] Updated weights for policy 1, policy_version 1711460 (0.0011) [2023-12-27 03:50:08,461][105620] Updated weights for policy 1, policy_version 1711470 (0.0009) [2023-12-27 03:50:08,958][105692] Updated weights for policy 0, policy_version 1707722 (0.0009) [2023-12-27 03:50:09,021][105692] Updated weights for policy 0, policy_version 1707732 (0.0008) [2023-12-27 03:50:09,089][105692] Updated weights for policy 0, policy_version 1707742 (0.0008) [2023-12-27 03:50:09,201][105620] Updated weights for policy 1, policy_version 1711480 (0.0006) [2023-12-27 03:50:09,271][105620] Updated weights for policy 1, policy_version 1711490 (0.0008) [2023-12-27 03:50:09,333][105620] Updated weights for policy 1, policy_version 1711500 (0.0009) [2023-12-27 03:50:09,896][105692] Updated weights for policy 0, policy_version 1707752 (0.0008) [2023-12-27 03:50:09,960][105692] Updated weights for policy 0, policy_version 1707762 (0.0008) [2023-12-27 03:50:10,012][105620] Updated weights for policy 1, policy_version 1711510 (0.0007) [2023-12-27 03:50:10,026][105692] Updated weights for policy 0, policy_version 1707772 (0.0007) [2023-12-27 03:50:10,076][105620] Updated weights for policy 1, policy_version 1711520 (0.0008) [2023-12-27 03:50:10,141][105620] Updated weights for policy 1, policy_version 1711530 (0.0009) [2023-12-27 03:50:10,692][105692] Updated weights for policy 0, policy_version 1707782 (0.0009) [2023-12-27 03:50:10,750][105692] Updated weights for policy 0, policy_version 1707792 (0.0009) [2023-12-27 03:50:10,812][105692] Updated weights for policy 0, policy_version 1707802 (0.0005) [2023-12-27 03:50:10,935][105620] Updated weights for policy 1, policy_version 1711540 (0.0009) [2023-12-27 03:50:10,992][105620] Updated weights for policy 1, policy_version 1711550 (0.0009) [2023-12-27 03:50:11,059][105620] Updated weights for policy 1, policy_version 1711561 (0.0009) [2023-12-27 03:50:11,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 875479040. Throughput: 0: 10039.8, 1: 9815.2. Samples: 875490052. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:50:11,063][104569] Avg episode reward: [(0, '8712.931'), (1, '8805.365')] [2023-12-27 03:50:11,435][105692] Updated weights for policy 0, policy_version 1707812 (0.0007) [2023-12-27 03:50:11,491][105692] Updated weights for policy 0, policy_version 1707822 (0.0010) [2023-12-27 03:50:11,550][105692] Updated weights for policy 0, policy_version 1707832 (0.0011) [2023-12-27 03:50:11,879][105620] Updated weights for policy 1, policy_version 1711571 (0.0008) [2023-12-27 03:50:11,943][105620] Updated weights for policy 1, policy_version 1711581 (0.0006) [2023-12-27 03:50:12,006][105620] Updated weights for policy 1, policy_version 1711591 (0.0008) [2023-12-27 03:50:12,317][105692] Updated weights for policy 0, policy_version 1707842 (0.0011) [2023-12-27 03:50:12,384][105692] Updated weights for policy 0, policy_version 1707852 (0.0010) [2023-12-27 03:50:12,438][105692] Updated weights for policy 0, policy_version 1707862 (0.0008) [2023-12-27 03:50:12,486][105692] Updated weights for policy 0, policy_version 1707872 (0.0008) [2023-12-27 03:50:12,626][105620] Updated weights for policy 1, policy_version 1711601 (0.0009) [2023-12-27 03:50:12,685][105620] Updated weights for policy 1, policy_version 1711611 (0.0009) [2023-12-27 03:50:12,743][105620] Updated weights for policy 1, policy_version 1711621 (0.0010) [2023-12-27 03:50:12,802][105620] Updated weights for policy 1, policy_version 1711631 (0.0010) [2023-12-27 03:50:13,210][105692] Updated weights for policy 0, policy_version 1707882 (0.0010) [2023-12-27 03:50:13,258][105692] Updated weights for policy 0, policy_version 1707892 (0.0010) [2023-12-27 03:50:13,313][105692] Updated weights for policy 0, policy_version 1707902 (0.0008) [2023-12-27 03:50:13,516][105620] Updated weights for policy 1, policy_version 1711641 (0.0010) [2023-12-27 03:50:13,577][105620] Updated weights for policy 1, policy_version 1711651 (0.0010) [2023-12-27 03:50:13,641][105620] Updated weights for policy 1, policy_version 1711661 (0.0008) [2023-12-27 03:50:14,075][105692] Updated weights for policy 0, policy_version 1707912 (0.0008) [2023-12-27 03:50:14,127][105692] Updated weights for policy 0, policy_version 1707922 (0.0008) [2023-12-27 03:50:14,176][105692] Updated weights for policy 0, policy_version 1707932 (0.0008) [2023-12-27 03:50:14,328][105620] Updated weights for policy 1, policy_version 1711671 (0.0005) [2023-12-27 03:50:14,393][105620] Updated weights for policy 1, policy_version 1711681 (0.0006) [2023-12-27 03:50:14,441][105620] Updated weights for policy 1, policy_version 1711691 (0.0010) [2023-12-27 03:50:14,993][105692] Updated weights for policy 0, policy_version 1707942 (0.0008) [2023-12-27 03:50:15,061][105692] Updated weights for policy 0, policy_version 1707952 (0.0008) [2023-12-27 03:50:15,111][105620] Updated weights for policy 1, policy_version 1711701 (0.0010) [2023-12-27 03:50:15,119][105692] Updated weights for policy 0, policy_version 1707962 (0.0009) [2023-12-27 03:50:15,167][105620] Updated weights for policy 1, policy_version 1711711 (0.0010) [2023-12-27 03:50:15,220][105620] Updated weights for policy 1, policy_version 1711721 (0.0011) [2023-12-27 03:50:15,784][105692] Updated weights for policy 0, policy_version 1707972 (0.0007) [2023-12-27 03:50:15,851][105692] Updated weights for policy 0, policy_version 1707982 (0.0005) [2023-12-27 03:50:15,919][105692] Updated weights for policy 0, policy_version 1707992 (0.0006) [2023-12-27 03:50:15,948][105620] Updated weights for policy 1, policy_version 1711731 (0.0010) [2023-12-27 03:50:15,993][105620] Updated weights for policy 1, policy_version 1711741 (0.0010) [2023-12-27 03:50:16,038][105620] Updated weights for policy 1, policy_version 1711751 (0.0010) [2023-12-27 03:50:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 875577344. Throughput: 0: 10022.8, 1: 9740.1. Samples: 875548440. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:50:16,063][104569] Avg episode reward: [(0, '8893.261'), (1, '9079.506')] [2023-12-27 03:50:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001708000_437313536.pth... [2023-12-27 03:50:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001706848_437018624.pth [2023-12-27 03:50:16,083][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001711760_438272000.pth... [2023-12-27 03:50:16,086][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001710608_437977088.pth [2023-12-27 03:50:16,628][105692] Updated weights for policy 0, policy_version 1708002 (0.0007) [2023-12-27 03:50:16,638][105620] Updated weights for policy 1, policy_version 1711761 (0.0010) [2023-12-27 03:50:16,681][105692] Updated weights for policy 0, policy_version 1708012 (0.0011) [2023-12-27 03:50:16,686][105620] Updated weights for policy 1, policy_version 1711771 (0.0009) [2023-12-27 03:50:16,730][105620] Updated weights for policy 1, policy_version 1711781 (0.0007) [2023-12-27 03:50:16,747][105692] Updated weights for policy 0, policy_version 1708022 (0.0007) [2023-12-27 03:50:16,790][105620] Updated weights for policy 1, policy_version 1711791 (0.0007) [2023-12-27 03:50:16,810][105692] Updated weights for policy 0, policy_version 1708032 (0.0005) [2023-12-27 03:50:17,391][105620] Updated weights for policy 1, policy_version 1711801 (0.0009) [2023-12-27 03:50:17,442][105620] Updated weights for policy 1, policy_version 1711811 (0.0005) [2023-12-27 03:50:17,489][105692] Updated weights for policy 0, policy_version 1708042 (0.0010) [2023-12-27 03:50:17,492][105620] Updated weights for policy 1, policy_version 1711821 (0.0005) [2023-12-27 03:50:17,537][105692] Updated weights for policy 0, policy_version 1708052 (0.0010) [2023-12-27 03:50:17,581][105692] Updated weights for policy 0, policy_version 1708062 (0.0010) [2023-12-27 03:50:18,125][105620] Updated weights for policy 1, policy_version 1711831 (0.0009) [2023-12-27 03:50:18,173][105620] Updated weights for policy 1, policy_version 1711841 (0.0010) [2023-12-27 03:50:18,221][105620] Updated weights for policy 1, policy_version 1711851 (0.0010) [2023-12-27 03:50:18,241][105692] Updated weights for policy 0, policy_version 1708072 (0.0006) [2023-12-27 03:50:18,299][105692] Updated weights for policy 0, policy_version 1708082 (0.0005) [2023-12-27 03:50:18,357][105692] Updated weights for policy 0, policy_version 1708092 (0.0008) [2023-12-27 03:50:18,976][105620] Updated weights for policy 1, policy_version 1711861 (0.0010) [2023-12-27 03:50:18,980][105692] Updated weights for policy 0, policy_version 1708102 (0.0006) [2023-12-27 03:50:19,023][105620] Updated weights for policy 1, policy_version 1711871 (0.0008) [2023-12-27 03:50:19,030][105692] Updated weights for policy 0, policy_version 1708112 (0.0006) [2023-12-27 03:50:19,070][105620] Updated weights for policy 1, policy_version 1711881 (0.0008) [2023-12-27 03:50:19,076][105692] Updated weights for policy 0, policy_version 1708122 (0.0005) [2023-12-27 03:50:19,738][105692] Updated weights for policy 0, policy_version 1708132 (0.0005) [2023-12-27 03:50:19,796][105692] Updated weights for policy 0, policy_version 1708142 (0.0008) [2023-12-27 03:50:19,863][105692] Updated weights for policy 0, policy_version 1708152 (0.0011) [2023-12-27 03:50:19,939][105620] Updated weights for policy 1, policy_version 1711891 (0.0008) [2023-12-27 03:50:20,000][105620] Updated weights for policy 1, policy_version 1711901 (0.0007) [2023-12-27 03:50:20,067][105620] Updated weights for policy 1, policy_version 1711911 (0.0006) [2023-12-27 03:50:20,528][105692] Updated weights for policy 0, policy_version 1708162 (0.0010) [2023-12-27 03:50:20,599][105692] Updated weights for policy 0, policy_version 1708172 (0.0007) [2023-12-27 03:50:20,655][105692] Updated weights for policy 0, policy_version 1708182 (0.0008) [2023-12-27 03:50:20,713][105692] Updated weights for policy 0, policy_version 1708192 (0.0011) [2023-12-27 03:50:20,880][105620] Updated weights for policy 1, policy_version 1711921 (0.0007) [2023-12-27 03:50:20,934][105620] Updated weights for policy 1, policy_version 1711931 (0.0009) [2023-12-27 03:50:20,998][105620] Updated weights for policy 1, policy_version 1711941 (0.0009) [2023-12-27 03:50:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 875675648. Throughput: 0: 9975.9, 1: 9786.0. Samples: 875669772. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:50:21,062][104569] Avg episode reward: [(0, '8711.409'), (1, '9170.385')] [2023-12-27 03:50:21,074][105620] Updated weights for policy 1, policy_version 1711951 (0.0009) [2023-12-27 03:50:21,458][105692] Updated weights for policy 0, policy_version 1708202 (0.0009) [2023-12-27 03:50:21,528][105692] Updated weights for policy 0, policy_version 1708212 (0.0010) [2023-12-27 03:50:21,587][105692] Updated weights for policy 0, policy_version 1708222 (0.0011) [2023-12-27 03:50:21,943][105620] Updated weights for policy 1, policy_version 1711961 (0.0011) [2023-12-27 03:50:22,007][105620] Updated weights for policy 1, policy_version 1711971 (0.0011) [2023-12-27 03:50:22,067][105620] Updated weights for policy 1, policy_version 1711981 (0.0011) [2023-12-27 03:50:22,302][105692] Updated weights for policy 0, policy_version 1708232 (0.0009) [2023-12-27 03:50:22,374][105692] Updated weights for policy 0, policy_version 1708242 (0.0007) [2023-12-27 03:50:22,444][105692] Updated weights for policy 0, policy_version 1708252 (0.0006) [2023-12-27 03:50:22,802][105620] Updated weights for policy 1, policy_version 1711991 (0.0007) [2023-12-27 03:50:22,859][105620] Updated weights for policy 1, policy_version 1712001 (0.0005) [2023-12-27 03:50:22,915][105620] Updated weights for policy 1, policy_version 1712011 (0.0007) [2023-12-27 03:50:23,227][105692] Updated weights for policy 0, policy_version 1708262 (0.0007) [2023-12-27 03:50:23,278][105692] Updated weights for policy 0, policy_version 1708272 (0.0007) [2023-12-27 03:50:23,329][105692] Updated weights for policy 0, policy_version 1708282 (0.0006) [2023-12-27 03:50:23,557][105620] Updated weights for policy 1, policy_version 1712021 (0.0009) [2023-12-27 03:50:23,605][105620] Updated weights for policy 1, policy_version 1712031 (0.0010) [2023-12-27 03:50:23,652][105620] Updated weights for policy 1, policy_version 1712041 (0.0008) [2023-12-27 03:50:23,888][105692] Updated weights for policy 0, policy_version 1708292 (0.0006) [2023-12-27 03:50:23,959][105692] Updated weights for policy 0, policy_version 1708302 (0.0008) [2023-12-27 03:50:24,007][105692] Updated weights for policy 0, policy_version 1708312 (0.0009) [2023-12-27 03:50:24,381][105620] Updated weights for policy 1, policy_version 1712051 (0.0007) [2023-12-27 03:50:24,441][105620] Updated weights for policy 1, policy_version 1712061 (0.0011) [2023-12-27 03:50:24,501][105620] Updated weights for policy 1, policy_version 1712071 (0.0011) [2023-12-27 03:50:24,673][105692] Updated weights for policy 0, policy_version 1708322 (0.0011) [2023-12-27 03:50:24,731][105692] Updated weights for policy 0, policy_version 1708332 (0.0011) [2023-12-27 03:50:24,793][105692] Updated weights for policy 0, policy_version 1708342 (0.0006) [2023-12-27 03:50:24,851][105692] Updated weights for policy 0, policy_version 1708352 (0.0005) [2023-12-27 03:50:25,250][105620] Updated weights for policy 1, policy_version 1712081 (0.0010) [2023-12-27 03:50:25,316][105620] Updated weights for policy 1, policy_version 1712091 (0.0005) [2023-12-27 03:50:25,383][105620] Updated weights for policy 1, policy_version 1712101 (0.0008) [2023-12-27 03:50:25,446][105620] Updated weights for policy 1, policy_version 1712111 (0.0006) [2023-12-27 03:50:25,478][105692] Updated weights for policy 0, policy_version 1708362 (0.0007) [2023-12-27 03:50:25,547][105692] Updated weights for policy 0, policy_version 1708372 (0.0009) [2023-12-27 03:50:25,604][105692] Updated weights for policy 0, policy_version 1708382 (0.0008) [2023-12-27 03:50:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 875773952. Throughput: 0: 9933.5, 1: 9883.9. Samples: 875786228. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:50:26,063][104569] Avg episode reward: [(0, '8714.480'), (1, '9170.382')] [2023-12-27 03:50:26,113][105620] Updated weights for policy 1, policy_version 1712121 (0.0009) [2023-12-27 03:50:26,175][105620] Updated weights for policy 1, policy_version 1712132 (0.0010) [2023-12-27 03:50:26,235][105620] Updated weights for policy 1, policy_version 1712142 (0.0007) [2023-12-27 03:50:26,240][105692] Updated weights for policy 0, policy_version 1708392 (0.0006) [2023-12-27 03:50:26,288][105692] Updated weights for policy 0, policy_version 1708402 (0.0005) [2023-12-27 03:50:26,341][105692] Updated weights for policy 0, policy_version 1708412 (0.0006) [2023-12-27 03:50:26,800][105620] Updated weights for policy 1, policy_version 1712152 (0.0006) [2023-12-27 03:50:26,849][105620] Updated weights for policy 1, policy_version 1712162 (0.0009) [2023-12-27 03:50:26,871][105692] Updated weights for policy 0, policy_version 1708422 (0.0005) [2023-12-27 03:50:26,898][105620] Updated weights for policy 1, policy_version 1712172 (0.0010) [2023-12-27 03:50:26,928][105692] Updated weights for policy 0, policy_version 1708432 (0.0005) [2023-12-27 03:50:26,991][105692] Updated weights for policy 0, policy_version 1708442 (0.0006) [2023-12-27 03:50:27,510][105620] Updated weights for policy 1, policy_version 1712182 (0.0007) [2023-12-27 03:50:27,553][105620] Updated weights for policy 1, policy_version 1712192 (0.0005) [2023-12-27 03:50:27,589][105692] Updated weights for policy 0, policy_version 1708452 (0.0007) [2023-12-27 03:50:27,603][105620] Updated weights for policy 1, policy_version 1712202 (0.0006) [2023-12-27 03:50:27,640][105692] Updated weights for policy 0, policy_version 1708462 (0.0010) [2023-12-27 03:50:27,697][105692] Updated weights for policy 0, policy_version 1708472 (0.0010) [2023-12-27 03:50:28,225][105620] Updated weights for policy 1, policy_version 1712212 (0.0005) [2023-12-27 03:50:28,283][105620] Updated weights for policy 1, policy_version 1712222 (0.0005) [2023-12-27 03:50:28,338][105620] Updated weights for policy 1, policy_version 1712232 (0.0007) [2023-12-27 03:50:28,423][105692] Updated weights for policy 0, policy_version 1708482 (0.0009) [2023-12-27 03:50:28,485][105692] Updated weights for policy 0, policy_version 1708492 (0.0005) [2023-12-27 03:50:28,544][105692] Updated weights for policy 0, policy_version 1708502 (0.0006) [2023-12-27 03:50:28,606][105692] Updated weights for policy 0, policy_version 1708512 (0.0005) [2023-12-27 03:50:29,034][105620] Updated weights for policy 1, policy_version 1712242 (0.0009) [2023-12-27 03:50:29,092][105620] Updated weights for policy 1, policy_version 1712252 (0.0010) [2023-12-27 03:50:29,103][105692] Updated weights for policy 0, policy_version 1708522 (0.0007) [2023-12-27 03:50:29,142][105620] Updated weights for policy 1, policy_version 1712262 (0.0010) [2023-12-27 03:50:29,156][105692] Updated weights for policy 0, policy_version 1708532 (0.0010) [2023-12-27 03:50:29,197][105620] Updated weights for policy 1, policy_version 1712272 (0.0010) [2023-12-27 03:50:29,212][105692] Updated weights for policy 0, policy_version 1708542 (0.0008) [2023-12-27 03:50:29,897][105620] Updated weights for policy 1, policy_version 1712282 (0.0010) [2023-12-27 03:50:29,919][105692] Updated weights for policy 0, policy_version 1708552 (0.0011) [2023-12-27 03:50:29,950][105620] Updated weights for policy 1, policy_version 1712292 (0.0011) [2023-12-27 03:50:29,985][105692] Updated weights for policy 0, policy_version 1708562 (0.0011) [2023-12-27 03:50:30,011][105620] Updated weights for policy 1, policy_version 1712302 (0.0008) [2023-12-27 03:50:30,048][105692] Updated weights for policy 0, policy_version 1708572 (0.0011) [2023-12-27 03:50:30,705][105620] Updated weights for policy 1, policy_version 1712312 (0.0010) [2023-12-27 03:50:30,761][105620] Updated weights for policy 1, policy_version 1712322 (0.0010) [2023-12-27 03:50:30,783][105692] Updated weights for policy 0, policy_version 1708582 (0.0011) [2023-12-27 03:50:30,813][105620] Updated weights for policy 1, policy_version 1712332 (0.0010) [2023-12-27 03:50:30,841][105692] Updated weights for policy 0, policy_version 1708592 (0.0010) [2023-12-27 03:50:30,885][105692] Updated weights for policy 0, policy_version 1708602 (0.0010) [2023-12-27 03:50:31,062][104569] Fps is (10 sec: 21299.3, 60 sec: 19933.9, 300 sec: 19688.6). Total num frames: 875888640. Throughput: 0: 10059.5, 1: 9930.7. Samples: 875852444. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:50:31,062][104569] Avg episode reward: [(0, '8531.751'), (1, '9171.752')] [2023-12-27 03:50:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001708608_437469184.pth... [2023-12-27 03:50:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001712336_438419456.pth... [2023-12-27 03:50:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001711184_438124544.pth [2023-12-27 03:50:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001707392_437157888.pth [2023-12-27 03:50:31,624][105692] Updated weights for policy 0, policy_version 1708612 (0.0011) [2023-12-27 03:50:31,639][105620] Updated weights for policy 1, policy_version 1712342 (0.0012) [2023-12-27 03:50:31,685][105692] Updated weights for policy 0, policy_version 1708622 (0.0009) [2023-12-27 03:50:31,701][105620] Updated weights for policy 1, policy_version 1712352 (0.0011) [2023-12-27 03:50:31,746][105692] Updated weights for policy 0, policy_version 1708632 (0.0011) [2023-12-27 03:50:31,763][105620] Updated weights for policy 1, policy_version 1712362 (0.0011) [2023-12-27 03:50:32,394][105692] Updated weights for policy 0, policy_version 1708642 (0.0009) [2023-12-27 03:50:32,457][105692] Updated weights for policy 0, policy_version 1708652 (0.0005) [2023-12-27 03:50:32,511][105620] Updated weights for policy 1, policy_version 1712372 (0.0008) [2023-12-27 03:50:32,524][105692] Updated weights for policy 0, policy_version 1708662 (0.0006) [2023-12-27 03:50:32,573][105620] Updated weights for policy 1, policy_version 1712382 (0.0008) [2023-12-27 03:50:32,584][105692] Updated weights for policy 0, policy_version 1708672 (0.0006) [2023-12-27 03:50:32,623][105620] Updated weights for policy 1, policy_version 1712392 (0.0007) [2023-12-27 03:50:33,105][105692] Updated weights for policy 0, policy_version 1708682 (0.0011) [2023-12-27 03:50:33,152][105692] Updated weights for policy 0, policy_version 1708692 (0.0010) [2023-12-27 03:50:33,174][105620] Updated weights for policy 1, policy_version 1712402 (0.0006) [2023-12-27 03:50:33,202][105692] Updated weights for policy 0, policy_version 1708702 (0.0005) [2023-12-27 03:50:33,232][105620] Updated weights for policy 1, policy_version 1712412 (0.0010) [2023-12-27 03:50:33,296][105620] Updated weights for policy 1, policy_version 1712422 (0.0007) [2023-12-27 03:50:33,365][105620] Updated weights for policy 1, policy_version 1712432 (0.0009) [2023-12-27 03:50:33,751][105692] Updated weights for policy 0, policy_version 1708712 (0.0005) [2023-12-27 03:50:33,804][105692] Updated weights for policy 0, policy_version 1708722 (0.0005) [2023-12-27 03:50:33,857][105692] Updated weights for policy 0, policy_version 1708732 (0.0005) [2023-12-27 03:50:34,065][105620] Updated weights for policy 1, policy_version 1712442 (0.0010) [2023-12-27 03:50:34,109][105620] Updated weights for policy 1, policy_version 1712452 (0.0010) [2023-12-27 03:50:34,163][105620] Updated weights for policy 1, policy_version 1712462 (0.0010) [2023-12-27 03:50:34,454][105692] Updated weights for policy 0, policy_version 1708742 (0.0007) [2023-12-27 03:50:34,513][105692] Updated weights for policy 0, policy_version 1708752 (0.0008) [2023-12-27 03:50:34,565][105692] Updated weights for policy 0, policy_version 1708762 (0.0007) [2023-12-27 03:50:34,907][105620] Updated weights for policy 1, policy_version 1712472 (0.0010) [2023-12-27 03:50:34,965][105620] Updated weights for policy 1, policy_version 1712482 (0.0011) [2023-12-27 03:50:35,021][105620] Updated weights for policy 1, policy_version 1712492 (0.0010) [2023-12-27 03:50:35,271][105692] Updated weights for policy 0, policy_version 1708772 (0.0007) [2023-12-27 03:50:35,326][105692] Updated weights for policy 0, policy_version 1708782 (0.0010) [2023-12-27 03:50:35,380][105692] Updated weights for policy 0, policy_version 1708792 (0.0010) [2023-12-27 03:50:35,833][105620] Updated weights for policy 1, policy_version 1712502 (0.0010) [2023-12-27 03:50:35,888][105620] Updated weights for policy 1, policy_version 1712512 (0.0010) [2023-12-27 03:50:35,915][105692] Updated weights for policy 0, policy_version 1708802 (0.0006) [2023-12-27 03:50:35,948][105620] Updated weights for policy 1, policy_version 1712522 (0.0008) [2023-12-27 03:50:35,970][105692] Updated weights for policy 0, policy_version 1708812 (0.0009) [2023-12-27 03:50:36,016][105692] Updated weights for policy 0, policy_version 1708822 (0.0006) [2023-12-27 03:50:36,062][104569] Fps is (10 sec: 21299.8, 60 sec: 19933.9, 300 sec: 19688.6). Total num frames: 875986944. Throughput: 0: 10209.8, 1: 9929.1. Samples: 875976128. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:50:36,062][104569] Avg episode reward: [(0, '8624.096'), (1, '8988.160')] [2023-12-27 03:50:36,066][105692] Updated weights for policy 0, policy_version 1708832 (0.0006) [2023-12-27 03:50:36,711][105620] Updated weights for policy 1, policy_version 1712532 (0.0008) [2023-12-27 03:50:36,755][105692] Updated weights for policy 0, policy_version 1708842 (0.0007) [2023-12-27 03:50:36,774][105620] Updated weights for policy 1, policy_version 1712542 (0.0010) [2023-12-27 03:50:36,810][105692] Updated weights for policy 0, policy_version 1708852 (0.0007) [2023-12-27 03:50:36,831][105620] Updated weights for policy 1, policy_version 1712552 (0.0008) [2023-12-27 03:50:36,870][105692] Updated weights for policy 0, policy_version 1708862 (0.0006) [2023-12-27 03:50:37,456][105620] Updated weights for policy 1, policy_version 1712562 (0.0006) [2023-12-27 03:50:37,515][105620] Updated weights for policy 1, policy_version 1712572 (0.0005) [2023-12-27 03:50:37,585][105620] Updated weights for policy 1, policy_version 1712582 (0.0006) [2023-12-27 03:50:37,633][105692] Updated weights for policy 0, policy_version 1708872 (0.0008) [2023-12-27 03:50:37,640][105620] Updated weights for policy 1, policy_version 1712592 (0.0005) [2023-12-27 03:50:37,690][105692] Updated weights for policy 0, policy_version 1708882 (0.0010) [2023-12-27 03:50:37,746][105692] Updated weights for policy 0, policy_version 1708892 (0.0009) [2023-12-27 03:50:38,216][105620] Updated weights for policy 1, policy_version 1712602 (0.0010) [2023-12-27 03:50:38,261][105620] Updated weights for policy 1, policy_version 1712612 (0.0010) [2023-12-27 03:50:38,306][105620] Updated weights for policy 1, policy_version 1712622 (0.0010) [2023-12-27 03:50:38,477][105692] Updated weights for policy 0, policy_version 1708903 (0.0008) [2023-12-27 03:50:38,546][105692] Updated weights for policy 0, policy_version 1708913 (0.0006) [2023-12-27 03:50:38,602][105692] Updated weights for policy 0, policy_version 1708923 (0.0006) [2023-12-27 03:50:39,008][105620] Updated weights for policy 1, policy_version 1712632 (0.0006) [2023-12-27 03:50:39,068][105620] Updated weights for policy 1, policy_version 1712642 (0.0009) [2023-12-27 03:50:39,129][105620] Updated weights for policy 1, policy_version 1712652 (0.0010) [2023-12-27 03:50:39,205][105692] Updated weights for policy 0, policy_version 1708933 (0.0007) [2023-12-27 03:50:39,271][105692] Updated weights for policy 0, policy_version 1708943 (0.0009) [2023-12-27 03:50:39,342][105692] Updated weights for policy 0, policy_version 1708954 (0.0008) [2023-12-27 03:50:39,902][105620] Updated weights for policy 1, policy_version 1712662 (0.0011) [2023-12-27 03:50:39,968][105620] Updated weights for policy 1, policy_version 1712672 (0.0011) [2023-12-27 03:50:40,030][105620] Updated weights for policy 1, policy_version 1712682 (0.0011) [2023-12-27 03:50:40,069][105692] Updated weights for policy 0, policy_version 1708964 (0.0008) [2023-12-27 03:50:40,139][105692] Updated weights for policy 0, policy_version 1708974 (0.0009) [2023-12-27 03:50:40,201][105692] Updated weights for policy 0, policy_version 1708984 (0.0008) [2023-12-27 03:50:40,747][105620] Updated weights for policy 1, policy_version 1712692 (0.0011) [2023-12-27 03:50:40,795][105620] Updated weights for policy 1, policy_version 1712702 (0.0010) [2023-12-27 03:50:40,847][105620] Updated weights for policy 1, policy_version 1712712 (0.0010) [2023-12-27 03:50:40,945][105692] Updated weights for policy 0, policy_version 1708994 (0.0008) [2023-12-27 03:50:41,014][105692] Updated weights for policy 0, policy_version 1709004 (0.0007) [2023-12-27 03:50:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19688.6). Total num frames: 876085248. Throughput: 0: 10286.3, 1: 9867.9. Samples: 876094936. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:50:41,062][104569] Avg episode reward: [(0, '8804.831'), (1, '8807.468')] [2023-12-27 03:50:41,076][105692] Updated weights for policy 0, policy_version 1709014 (0.0009) [2023-12-27 03:50:41,142][105692] Updated weights for policy 0, policy_version 1709024 (0.0008) [2023-12-27 03:50:41,628][105620] Updated weights for policy 1, policy_version 1712722 (0.0010) [2023-12-27 03:50:41,686][105620] Updated weights for policy 1, policy_version 1712732 (0.0008) [2023-12-27 03:50:41,749][105620] Updated weights for policy 1, policy_version 1712742 (0.0007) [2023-12-27 03:50:41,807][105620] Updated weights for policy 1, policy_version 1712752 (0.0010) [2023-12-27 03:50:41,914][105692] Updated weights for policy 0, policy_version 1709034 (0.0010) [2023-12-27 03:50:41,982][105692] Updated weights for policy 0, policy_version 1709044 (0.0009) [2023-12-27 03:50:42,046][105692] Updated weights for policy 0, policy_version 1709054 (0.0009) [2023-12-27 03:50:42,556][105620] Updated weights for policy 1, policy_version 1712762 (0.0008) [2023-12-27 03:50:42,604][105620] Updated weights for policy 1, policy_version 1712772 (0.0009) [2023-12-27 03:50:42,654][105620] Updated weights for policy 1, policy_version 1712782 (0.0006) [2023-12-27 03:50:42,803][105692] Updated weights for policy 0, policy_version 1709064 (0.0010) [2023-12-27 03:50:42,864][105692] Updated weights for policy 0, policy_version 1709074 (0.0009) [2023-12-27 03:50:42,926][105692] Updated weights for policy 0, policy_version 1709084 (0.0009) [2023-12-27 03:50:43,322][105620] Updated weights for policy 1, policy_version 1712792 (0.0008) [2023-12-27 03:50:43,380][105620] Updated weights for policy 1, policy_version 1712802 (0.0009) [2023-12-27 03:50:43,441][105620] Updated weights for policy 1, policy_version 1712812 (0.0009) [2023-12-27 03:50:43,728][105692] Updated weights for policy 0, policy_version 1709094 (0.0007) [2023-12-27 03:50:43,801][105692] Updated weights for policy 0, policy_version 1709104 (0.0010) [2023-12-27 03:50:43,864][105692] Updated weights for policy 0, policy_version 1709115 (0.0010) [2023-12-27 03:50:44,136][105620] Updated weights for policy 1, policy_version 1712822 (0.0009) [2023-12-27 03:50:44,190][105620] Updated weights for policy 1, policy_version 1712834 (0.0010) [2023-12-27 03:50:44,241][105620] Updated weights for policy 1, policy_version 1712844 (0.0009) [2023-12-27 03:50:44,473][105692] Updated weights for policy 0, policy_version 1709125 (0.0008) [2023-12-27 03:50:44,531][105692] Updated weights for policy 0, policy_version 1709135 (0.0009) [2023-12-27 03:50:44,589][105692] Updated weights for policy 0, policy_version 1709145 (0.0009) [2023-12-27 03:50:45,029][105620] Updated weights for policy 1, policy_version 1712854 (0.0009) [2023-12-27 03:50:45,091][105620] Updated weights for policy 1, policy_version 1712864 (0.0009) [2023-12-27 03:50:45,158][105620] Updated weights for policy 1, policy_version 1712874 (0.0007) [2023-12-27 03:50:45,350][105692] Updated weights for policy 0, policy_version 1709155 (0.0009) [2023-12-27 03:50:45,405][105692] Updated weights for policy 0, policy_version 1709165 (0.0009) [2023-12-27 03:50:45,465][105692] Updated weights for policy 0, policy_version 1709175 (0.0006) [2023-12-27 03:50:45,905][105620] Updated weights for policy 1, policy_version 1712884 (0.0006) [2023-12-27 03:50:45,962][105620] Updated weights for policy 1, policy_version 1712894 (0.0005) [2023-12-27 03:50:46,013][105620] Updated weights for policy 1, policy_version 1712904 (0.0005) [2023-12-27 03:50:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 20070.4, 300 sec: 19716.3). Total num frames: 876183552. Throughput: 0: 10202.1, 1: 9873.9. Samples: 876151700. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:50:46,062][104569] Avg episode reward: [(0, '8803.043'), (1, '8900.879')] [2023-12-27 03:50:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001709184_437616640.pth... [2023-12-27 03:50:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001712912_438566912.pth... [2023-12-27 03:50:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001708000_437313536.pth [2023-12-27 03:50:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001711760_438272000.pth [2023-12-27 03:50:46,190][105692] Updated weights for policy 0, policy_version 1709185 (0.0009) [2023-12-27 03:50:46,236][105692] Updated weights for policy 0, policy_version 1709195 (0.0005) [2023-12-27 03:50:46,283][105692] Updated weights for policy 0, policy_version 1709205 (0.0005) [2023-12-27 03:50:46,336][105692] Updated weights for policy 0, policy_version 1709215 (0.0007) [2023-12-27 03:50:46,555][105620] Updated weights for policy 1, policy_version 1712914 (0.0006) [2023-12-27 03:50:46,603][105620] Updated weights for policy 1, policy_version 1712924 (0.0009) [2023-12-27 03:50:46,650][105620] Updated weights for policy 1, policy_version 1712934 (0.0009) [2023-12-27 03:50:46,705][105620] Updated weights for policy 1, policy_version 1712944 (0.0009) [2023-12-27 03:50:46,958][105692] Updated weights for policy 0, policy_version 1709225 (0.0005) [2023-12-27 03:50:47,020][105692] Updated weights for policy 0, policy_version 1709235 (0.0005) [2023-12-27 03:50:47,078][105692] Updated weights for policy 0, policy_version 1709245 (0.0005) [2023-12-27 03:50:47,518][105620] Updated weights for policy 1, policy_version 1712954 (0.0009) [2023-12-27 03:50:47,569][105620] Updated weights for policy 1, policy_version 1712964 (0.0007) [2023-12-27 03:50:47,618][105620] Updated weights for policy 1, policy_version 1712974 (0.0010) [2023-12-27 03:50:47,715][105692] Updated weights for policy 0, policy_version 1709255 (0.0009) [2023-12-27 03:50:47,776][105692] Updated weights for policy 0, policy_version 1709265 (0.0010) [2023-12-27 03:50:47,832][105692] Updated weights for policy 0, policy_version 1709275 (0.0010) [2023-12-27 03:50:48,271][105620] Updated weights for policy 1, policy_version 1712984 (0.0007) [2023-12-27 03:50:48,332][105620] Updated weights for policy 1, policy_version 1712994 (0.0007) [2023-12-27 03:50:48,387][105620] Updated weights for policy 1, policy_version 1713004 (0.0007) [2023-12-27 03:50:48,463][105692] Updated weights for policy 0, policy_version 1709286 (0.0008) [2023-12-27 03:50:48,529][105692] Updated weights for policy 0, policy_version 1709296 (0.0007) [2023-12-27 03:50:48,597][105692] Updated weights for policy 0, policy_version 1709306 (0.0006) [2023-12-27 03:50:49,062][105620] Updated weights for policy 1, policy_version 1713014 (0.0007) [2023-12-27 03:50:49,120][105620] Updated weights for policy 1, policy_version 1713024 (0.0009) [2023-12-27 03:50:49,169][105620] Updated weights for policy 1, policy_version 1713034 (0.0009) [2023-12-27 03:50:49,294][105692] Updated weights for policy 0, policy_version 1709316 (0.0008) [2023-12-27 03:50:49,362][105692] Updated weights for policy 0, policy_version 1709326 (0.0008) [2023-12-27 03:50:49,425][105692] Updated weights for policy 0, policy_version 1709336 (0.0009) [2023-12-27 03:50:49,972][105620] Updated weights for policy 1, policy_version 1713044 (0.0009) [2023-12-27 03:50:50,034][105620] Updated weights for policy 1, policy_version 1713054 (0.0010) [2023-12-27 03:50:50,084][105620] Updated weights for policy 1, policy_version 1713064 (0.0008) [2023-12-27 03:50:50,186][105692] Updated weights for policy 0, policy_version 1709346 (0.0009) [2023-12-27 03:50:50,245][105692] Updated weights for policy 0, policy_version 1709356 (0.0006) [2023-12-27 03:50:50,304][105692] Updated weights for policy 0, policy_version 1709366 (0.0007) [2023-12-27 03:50:50,365][105692] Updated weights for policy 0, policy_version 1709376 (0.0008) [2023-12-27 03:50:50,870][105620] Updated weights for policy 1, policy_version 1713074 (0.0010) [2023-12-27 03:50:50,931][105620] Updated weights for policy 1, policy_version 1713084 (0.0009) [2023-12-27 03:50:50,990][105620] Updated weights for policy 1, policy_version 1713094 (0.0009) [2023-12-27 03:50:51,050][105692] Updated weights for policy 0, policy_version 1709386 (0.0007) [2023-12-27 03:50:51,052][105620] Updated weights for policy 1, policy_version 1713104 (0.0006) [2023-12-27 03:50:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 20070.4, 300 sec: 19688.6). Total num frames: 876281856. Throughput: 0: 10185.3, 1: 9867.7. Samples: 876270948. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:50:51,064][104569] Avg episode reward: [(0, '8987.509'), (1, '8989.650')] [2023-12-27 03:50:51,108][105692] Updated weights for policy 0, policy_version 1709396 (0.0008) [2023-12-27 03:50:51,169][105692] Updated weights for policy 0, policy_version 1709406 (0.0007) [2023-12-27 03:50:51,866][105620] Updated weights for policy 1, policy_version 1713114 (0.0008) [2023-12-27 03:50:51,925][105620] Updated weights for policy 1, policy_version 1713124 (0.0008) [2023-12-27 03:50:51,954][105692] Updated weights for policy 0, policy_version 1709416 (0.0007) [2023-12-27 03:50:51,981][105620] Updated weights for policy 1, policy_version 1713134 (0.0008) [2023-12-27 03:50:52,018][105692] Updated weights for policy 0, policy_version 1709426 (0.0006) [2023-12-27 03:50:52,081][105692] Updated weights for policy 0, policy_version 1709436 (0.0005) [2023-12-27 03:50:52,748][105620] Updated weights for policy 1, policy_version 1713144 (0.0007) [2023-12-27 03:50:52,750][105692] Updated weights for policy 0, policy_version 1709446 (0.0009) [2023-12-27 03:50:52,806][105692] Updated weights for policy 0, policy_version 1709456 (0.0011) [2023-12-27 03:50:52,812][105620] Updated weights for policy 1, policy_version 1713154 (0.0006) [2023-12-27 03:50:52,859][105692] Updated weights for policy 0, policy_version 1709466 (0.0011) [2023-12-27 03:50:52,869][105620] Updated weights for policy 1, policy_version 1713164 (0.0006) [2023-12-27 03:50:53,511][105692] Updated weights for policy 0, policy_version 1709476 (0.0007) [2023-12-27 03:50:53,570][105692] Updated weights for policy 0, policy_version 1709486 (0.0005) [2023-12-27 03:50:53,591][105620] Updated weights for policy 1, policy_version 1713174 (0.0010) [2023-12-27 03:50:53,620][105692] Updated weights for policy 0, policy_version 1709496 (0.0005) [2023-12-27 03:50:53,637][105620] Updated weights for policy 1, policy_version 1713184 (0.0009) [2023-12-27 03:50:53,699][105620] Updated weights for policy 1, policy_version 1713194 (0.0006) [2023-12-27 03:50:54,142][105692] Updated weights for policy 0, policy_version 1709506 (0.0005) [2023-12-27 03:50:54,217][105692] Updated weights for policy 0, policy_version 1709516 (0.0008) [2023-12-27 03:50:54,261][105620] Updated weights for policy 1, policy_version 1713204 (0.0006) [2023-12-27 03:50:54,283][105692] Updated weights for policy 0, policy_version 1709526 (0.0010) [2023-12-27 03:50:54,308][105620] Updated weights for policy 1, policy_version 1713214 (0.0005) [2023-12-27 03:50:54,345][105692] Updated weights for policy 0, policy_version 1709536 (0.0010) [2023-12-27 03:50:54,351][105620] Updated weights for policy 1, policy_version 1713224 (0.0007) [2023-12-27 03:50:55,002][105692] Updated weights for policy 0, policy_version 1709546 (0.0009) [2023-12-27 03:50:55,018][105620] Updated weights for policy 1, policy_version 1713234 (0.0007) [2023-12-27 03:50:55,068][105620] Updated weights for policy 1, policy_version 1713244 (0.0006) [2023-12-27 03:50:55,070][105692] Updated weights for policy 0, policy_version 1709556 (0.0011) [2023-12-27 03:50:55,120][105620] Updated weights for policy 1, policy_version 1713254 (0.0006) [2023-12-27 03:50:55,126][105692] Updated weights for policy 0, policy_version 1709566 (0.0010) [2023-12-27 03:50:55,167][105620] Updated weights for policy 1, policy_version 1713264 (0.0005) [2023-12-27 03:50:55,774][105620] Updated weights for policy 1, policy_version 1713274 (0.0008) [2023-12-27 03:50:55,833][105620] Updated weights for policy 1, policy_version 1713284 (0.0007) [2023-12-27 03:50:55,852][105692] Updated weights for policy 0, policy_version 1709576 (0.0008) [2023-12-27 03:50:55,893][105620] Updated weights for policy 1, policy_version 1713294 (0.0008) [2023-12-27 03:50:55,912][105692] Updated weights for policy 0, policy_version 1709586 (0.0006) [2023-12-27 03:50:55,967][105692] Updated weights for policy 0, policy_version 1709596 (0.0005) [2023-12-27 03:50:56,062][104569] Fps is (10 sec: 20479.6, 60 sec: 20206.9, 300 sec: 19716.4). Total num frames: 876388352. Throughput: 0: 10154.0, 1: 9871.2. Samples: 876391188. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:50:56,063][104569] Avg episode reward: [(0, '8530.751'), (1, '9080.789')] [2023-12-27 03:50:56,601][105692] Updated weights for policy 0, policy_version 1709606 (0.0008) [2023-12-27 03:50:56,662][105692] Updated weights for policy 0, policy_version 1709616 (0.0010) [2023-12-27 03:50:56,680][105620] Updated weights for policy 1, policy_version 1713304 (0.0006) [2023-12-27 03:50:56,710][105692] Updated weights for policy 0, policy_version 1709626 (0.0010) [2023-12-27 03:50:56,732][105620] Updated weights for policy 1, policy_version 1713314 (0.0005) [2023-12-27 03:50:56,789][105620] Updated weights for policy 1, policy_version 1713324 (0.0007) [2023-12-27 03:50:57,311][105692] Updated weights for policy 0, policy_version 1709636 (0.0008) [2023-12-27 03:50:57,369][105692] Updated weights for policy 0, policy_version 1709646 (0.0007) [2023-12-27 03:50:57,423][105692] Updated weights for policy 0, policy_version 1709656 (0.0007) [2023-12-27 03:50:57,614][105620] Updated weights for policy 1, policy_version 1713334 (0.0009) [2023-12-27 03:50:57,666][105620] Updated weights for policy 1, policy_version 1713344 (0.0008) [2023-12-27 03:50:57,723][105620] Updated weights for policy 1, policy_version 1713354 (0.0008) [2023-12-27 03:50:58,021][105692] Updated weights for policy 0, policy_version 1709666 (0.0005) [2023-12-27 03:50:58,081][105692] Updated weights for policy 0, policy_version 1709676 (0.0007) [2023-12-27 03:50:58,129][105692] Updated weights for policy 0, policy_version 1709686 (0.0008) [2023-12-27 03:50:58,186][105692] Updated weights for policy 0, policy_version 1709696 (0.0008) [2023-12-27 03:50:58,398][105620] Updated weights for policy 1, policy_version 1713364 (0.0009) [2023-12-27 03:50:58,468][105620] Updated weights for policy 1, policy_version 1713374 (0.0006) [2023-12-27 03:50:58,527][105620] Updated weights for policy 1, policy_version 1713384 (0.0009) [2023-12-27 03:50:58,942][105692] Updated weights for policy 0, policy_version 1709706 (0.0009) [2023-12-27 03:50:59,007][105692] Updated weights for policy 0, policy_version 1709716 (0.0008) [2023-12-27 03:50:59,064][105692] Updated weights for policy 0, policy_version 1709726 (0.0008) [2023-12-27 03:50:59,352][105620] Updated weights for policy 1, policy_version 1713394 (0.0010) [2023-12-27 03:50:59,420][105620] Updated weights for policy 1, policy_version 1713404 (0.0010) [2023-12-27 03:50:59,475][105620] Updated weights for policy 1, policy_version 1713414 (0.0010) [2023-12-27 03:50:59,530][105620] Updated weights for policy 1, policy_version 1713424 (0.0010) [2023-12-27 03:50:59,810][105692] Updated weights for policy 0, policy_version 1709736 (0.0006) [2023-12-27 03:50:59,878][105692] Updated weights for policy 0, policy_version 1709746 (0.0006) [2023-12-27 03:50:59,944][105692] Updated weights for policy 0, policy_version 1709756 (0.0007) [2023-12-27 03:51:00,284][105620] Updated weights for policy 1, policy_version 1713434 (0.0007) [2023-12-27 03:51:00,348][105620] Updated weights for policy 1, policy_version 1713444 (0.0006) [2023-12-27 03:51:00,413][105620] Updated weights for policy 1, policy_version 1713454 (0.0010) [2023-12-27 03:51:00,539][105692] Updated weights for policy 0, policy_version 1709766 (0.0007) [2023-12-27 03:51:00,583][105692] Updated weights for policy 0, policy_version 1709776 (0.0005) [2023-12-27 03:51:00,634][105692] Updated weights for policy 0, policy_version 1709786 (0.0005) [2023-12-27 03:51:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 20070.3, 300 sec: 19716.3). Total num frames: 876478464. Throughput: 0: 10215.4, 1: 9834.6. Samples: 876450688. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:51:01,063][104569] Avg episode reward: [(0, '8438.309'), (1, '9171.527')] [2023-12-27 03:51:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001709792_437772288.pth... [2023-12-27 03:51:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001708608_437469184.pth [2023-12-27 03:51:01,080][105620] Updated weights for policy 1, policy_version 1713464 (0.0008) [2023-12-27 03:51:01,150][105620] Updated weights for policy 1, policy_version 1713474 (0.0008) [2023-12-27 03:51:01,180][105692] Updated weights for policy 0, policy_version 1709796 (0.0006) [2023-12-27 03:51:01,213][105620] Updated weights for policy 1, policy_version 1713484 (0.0009) [2023-12-27 03:51:01,232][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001713488_438714368.pth... [2023-12-27 03:51:01,236][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001712336_438419456.pth [2023-12-27 03:51:01,243][105692] Updated weights for policy 0, policy_version 1709806 (0.0006) [2023-12-27 03:51:01,299][105692] Updated weights for policy 0, policy_version 1709816 (0.0007) [2023-12-27 03:51:01,962][105620] Updated weights for policy 1, policy_version 1713494 (0.0010) [2023-12-27 03:51:01,979][105692] Updated weights for policy 0, policy_version 1709826 (0.0008) [2023-12-27 03:51:02,020][105620] Updated weights for policy 1, policy_version 1713504 (0.0010) [2023-12-27 03:51:02,026][105692] Updated weights for policy 0, policy_version 1709836 (0.0008) [2023-12-27 03:51:02,078][105620] Updated weights for policy 1, policy_version 1713514 (0.0010) [2023-12-27 03:51:02,088][105692] Updated weights for policy 0, policy_version 1709846 (0.0006) [2023-12-27 03:51:02,138][105692] Updated weights for policy 0, policy_version 1709856 (0.0009) [2023-12-27 03:51:02,756][105620] Updated weights for policy 1, policy_version 1713524 (0.0010) [2023-12-27 03:51:02,808][105620] Updated weights for policy 1, policy_version 1713534 (0.0009) [2023-12-27 03:51:02,866][105620] Updated weights for policy 1, policy_version 1713544 (0.0009) [2023-12-27 03:51:02,885][105692] Updated weights for policy 0, policy_version 1709866 (0.0007) [2023-12-27 03:51:02,941][105692] Updated weights for policy 0, policy_version 1709876 (0.0005) [2023-12-27 03:51:02,987][105692] Updated weights for policy 0, policy_version 1709886 (0.0005) [2023-12-27 03:51:03,546][105620] Updated weights for policy 1, policy_version 1713554 (0.0007) [2023-12-27 03:51:03,607][105620] Updated weights for policy 1, policy_version 1713564 (0.0010) [2023-12-27 03:51:03,672][105620] Updated weights for policy 1, policy_version 1713574 (0.0010) [2023-12-27 03:51:03,723][105692] Updated weights for policy 0, policy_version 1709896 (0.0005) [2023-12-27 03:51:03,727][105620] Updated weights for policy 1, policy_version 1713584 (0.0010) [2023-12-27 03:51:03,779][105692] Updated weights for policy 0, policy_version 1709906 (0.0005) [2023-12-27 03:51:03,826][105692] Updated weights for policy 0, policy_version 1709916 (0.0008) [2023-12-27 03:51:04,490][105620] Updated weights for policy 1, policy_version 1713594 (0.0011) [2023-12-27 03:51:04,502][105586] KL-divergence is very high: 108.7087 [2023-12-27 03:51:04,549][105586] KL-divergence is very high: 158.7619 [2023-12-27 03:51:04,550][105620] Updated weights for policy 1, policy_version 1713604 (0.0011) [2023-12-27 03:51:04,569][105692] Updated weights for policy 0, policy_version 1709926 (0.0007) [2023-12-27 03:51:04,595][105586] KL-divergence is very high: 149.3385 [2023-12-27 03:51:04,607][105620] Updated weights for policy 1, policy_version 1713614 (0.0011) [2023-12-27 03:51:04,629][105692] Updated weights for policy 0, policy_version 1709936 (0.0006) [2023-12-27 03:51:04,685][105692] Updated weights for policy 0, policy_version 1709946 (0.0008) [2023-12-27 03:51:05,362][105620] Updated weights for policy 1, policy_version 1713624 (0.0011) [2023-12-27 03:51:05,424][105620] Updated weights for policy 1, policy_version 1713634 (0.0010) [2023-12-27 03:51:05,429][105692] Updated weights for policy 0, policy_version 1709956 (0.0008) [2023-12-27 03:51:05,476][105620] Updated weights for policy 1, policy_version 1713644 (0.0010) [2023-12-27 03:51:05,483][105692] Updated weights for policy 0, policy_version 1709966 (0.0006) [2023-12-27 03:51:05,544][105692] Updated weights for policy 0, policy_version 1709976 (0.0008) [2023-12-27 03:51:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19933.8, 300 sec: 19744.1). Total num frames: 876576768. Throughput: 0: 10213.1, 1: 9756.2. Samples: 876568388. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 03:51:06,063][104569] Avg episode reward: [(0, '8346.744'), (1, '8716.383')] [2023-12-27 03:51:06,169][105620] Updated weights for policy 1, policy_version 1713654 (0.0011) [2023-12-27 03:51:06,229][105620] Updated weights for policy 1, policy_version 1713664 (0.0011) [2023-12-27 03:51:06,278][105692] Updated weights for policy 0, policy_version 1709986 (0.0009) [2023-12-27 03:51:06,289][105620] Updated weights for policy 1, policy_version 1713674 (0.0010) [2023-12-27 03:51:06,340][105692] Updated weights for policy 0, policy_version 1709996 (0.0007) [2023-12-27 03:51:06,399][105692] Updated weights for policy 0, policy_version 1710006 (0.0008) [2023-12-27 03:51:06,455][105692] Updated weights for policy 0, policy_version 1710016 (0.0008) [2023-12-27 03:51:07,020][105620] Updated weights for policy 1, policy_version 1713684 (0.0011) [2023-12-27 03:51:07,072][105620] Updated weights for policy 1, policy_version 1713694 (0.0011) [2023-12-27 03:51:07,122][105620] Updated weights for policy 1, policy_version 1713704 (0.0010) [2023-12-27 03:51:07,239][105692] Updated weights for policy 0, policy_version 1710026 (0.0009) [2023-12-27 03:51:07,292][105692] Updated weights for policy 0, policy_version 1710036 (0.0008) [2023-12-27 03:51:07,341][105692] Updated weights for policy 0, policy_version 1710046 (0.0008) [2023-12-27 03:51:07,873][105620] Updated weights for policy 1, policy_version 1713714 (0.0010) [2023-12-27 03:51:07,920][105620] Updated weights for policy 1, policy_version 1713724 (0.0005) [2023-12-27 03:51:07,971][105620] Updated weights for policy 1, policy_version 1713734 (0.0005) [2023-12-27 03:51:08,025][105620] Updated weights for policy 1, policy_version 1713744 (0.0005) [2023-12-27 03:51:08,178][105692] Updated weights for policy 0, policy_version 1710056 (0.0009) [2023-12-27 03:51:08,237][105692] Updated weights for policy 0, policy_version 1710066 (0.0009) [2023-12-27 03:51:08,297][105692] Updated weights for policy 0, policy_version 1710076 (0.0009) [2023-12-27 03:51:08,634][105620] Updated weights for policy 1, policy_version 1713754 (0.0009) [2023-12-27 03:51:08,697][105620] Updated weights for policy 1, policy_version 1713764 (0.0009) [2023-12-27 03:51:08,755][105620] Updated weights for policy 1, policy_version 1713774 (0.0009) [2023-12-27 03:51:09,159][105692] Updated weights for policy 0, policy_version 1710086 (0.0009) [2023-12-27 03:51:09,228][105692] Updated weights for policy 0, policy_version 1710096 (0.0009) [2023-12-27 03:51:09,289][105692] Updated weights for policy 0, policy_version 1710106 (0.0009) [2023-12-27 03:51:09,399][105620] Updated weights for policy 1, policy_version 1713784 (0.0008) [2023-12-27 03:51:09,459][105620] Updated weights for policy 1, policy_version 1713794 (0.0007) [2023-12-27 03:51:09,522][105620] Updated weights for policy 1, policy_version 1713804 (0.0009) [2023-12-27 03:51:10,069][105692] Updated weights for policy 0, policy_version 1710116 (0.0009) [2023-12-27 03:51:10,131][105692] Updated weights for policy 0, policy_version 1710126 (0.0006) [2023-12-27 03:51:10,183][105692] Updated weights for policy 0, policy_version 1710136 (0.0009) [2023-12-27 03:51:10,236][105620] Updated weights for policy 1, policy_version 1713814 (0.0009) [2023-12-27 03:51:10,298][105620] Updated weights for policy 1, policy_version 1713824 (0.0009) [2023-12-27 03:51:10,356][105620] Updated weights for policy 1, policy_version 1713834 (0.0009) [2023-12-27 03:51:10,827][105692] Updated weights for policy 0, policy_version 1710146 (0.0008) [2023-12-27 03:51:10,890][105692] Updated weights for policy 0, policy_version 1710156 (0.0009) [2023-12-27 03:51:10,949][105692] Updated weights for policy 0, policy_version 1710166 (0.0009) [2023-12-27 03:51:11,014][105692] Updated weights for policy 0, policy_version 1710176 (0.0009) [2023-12-27 03:51:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19744.1). Total num frames: 876675072. Throughput: 0: 10080.5, 1: 9819.0. Samples: 876681700. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:51:11,063][104569] Avg episode reward: [(0, '8711.586'), (1, '8625.899')] [2023-12-27 03:51:11,108][105620] Updated weights for policy 1, policy_version 1713844 (0.0008) [2023-12-27 03:51:11,175][105620] Updated weights for policy 1, policy_version 1713854 (0.0008) [2023-12-27 03:51:11,241][105620] Updated weights for policy 1, policy_version 1713864 (0.0009) [2023-12-27 03:51:11,809][105692] Updated weights for policy 0, policy_version 1710186 (0.0010) [2023-12-27 03:51:11,869][105692] Updated weights for policy 0, policy_version 1710197 (0.0011) [2023-12-27 03:51:11,923][105692] Updated weights for policy 0, policy_version 1710208 (0.0011) [2023-12-27 03:51:11,956][105620] Updated weights for policy 1, policy_version 1713874 (0.0008) [2023-12-27 03:51:12,014][105620] Updated weights for policy 1, policy_version 1713884 (0.0006) [2023-12-27 03:51:12,075][105620] Updated weights for policy 1, policy_version 1713894 (0.0009) [2023-12-27 03:51:12,137][105620] Updated weights for policy 1, policy_version 1713904 (0.0009) [2023-12-27 03:51:12,751][105620] Updated weights for policy 1, policy_version 1713914 (0.0006) [2023-12-27 03:51:12,825][105620] Updated weights for policy 1, policy_version 1713924 (0.0006) [2023-12-27 03:51:12,829][105692] Updated weights for policy 0, policy_version 1710218 (0.0009) [2023-12-27 03:51:12,888][105620] Updated weights for policy 1, policy_version 1713934 (0.0006) [2023-12-27 03:51:12,890][105692] Updated weights for policy 0, policy_version 1710228 (0.0009) [2023-12-27 03:51:12,947][105692] Updated weights for policy 0, policy_version 1710238 (0.0009) [2023-12-27 03:51:13,414][105620] Updated weights for policy 1, policy_version 1713944 (0.0009) [2023-12-27 03:51:13,478][105620] Updated weights for policy 1, policy_version 1713954 (0.0009) [2023-12-27 03:51:13,536][105620] Updated weights for policy 1, policy_version 1713964 (0.0009) [2023-12-27 03:51:13,714][105692] Updated weights for policy 0, policy_version 1710248 (0.0007) [2023-12-27 03:51:13,774][105692] Updated weights for policy 0, policy_version 1710258 (0.0008) [2023-12-27 03:51:13,819][105692] Updated weights for policy 0, policy_version 1710268 (0.0008) [2023-12-27 03:51:14,184][105620] Updated weights for policy 1, policy_version 1713974 (0.0006) [2023-12-27 03:51:14,240][105620] Updated weights for policy 1, policy_version 1713984 (0.0006) [2023-12-27 03:51:14,298][105620] Updated weights for policy 1, policy_version 1713994 (0.0005) [2023-12-27 03:51:14,524][105692] Updated weights for policy 0, policy_version 1710278 (0.0008) [2023-12-27 03:51:14,574][105692] Updated weights for policy 0, policy_version 1710288 (0.0005) [2023-12-27 03:51:14,617][105692] Updated weights for policy 0, policy_version 1710298 (0.0005) [2023-12-27 03:51:15,043][105620] Updated weights for policy 1, policy_version 1714004 (0.0007) [2023-12-27 03:51:15,110][105620] Updated weights for policy 1, policy_version 1714014 (0.0009) [2023-12-27 03:51:15,172][105620] Updated weights for policy 1, policy_version 1714024 (0.0009) [2023-12-27 03:51:15,271][105692] Updated weights for policy 0, policy_version 1710308 (0.0006) [2023-12-27 03:51:15,334][105692] Updated weights for policy 0, policy_version 1710318 (0.0008) [2023-12-27 03:51:15,394][105692] Updated weights for policy 0, policy_version 1710328 (0.0010) [2023-12-27 03:51:15,877][105620] Updated weights for policy 1, policy_version 1714034 (0.0010) [2023-12-27 03:51:15,934][105620] Updated weights for policy 1, policy_version 1714044 (0.0008) [2023-12-27 03:51:16,004][105620] Updated weights for policy 1, policy_version 1714054 (0.0006) [2023-12-27 03:51:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19797.4, 300 sec: 19744.1). Total num frames: 876765184. Throughput: 0: 9943.3, 1: 9781.7. Samples: 876740068. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:51:16,062][104569] Avg episode reward: [(0, '8807.348'), (1, '8987.857')] [2023-12-27 03:51:16,075][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001714064_438861824.pth... [2023-12-27 03:51:16,075][105620] Updated weights for policy 1, policy_version 1714064 (0.0006) [2023-12-27 03:51:16,077][105692] Updated weights for policy 0, policy_version 1710338 (0.0009) [2023-12-27 03:51:16,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001712912_438566912.pth [2023-12-27 03:51:16,124][105692] Updated weights for policy 0, policy_version 1710348 (0.0005) [2023-12-27 03:51:16,170][105692] Updated weights for policy 0, policy_version 1710358 (0.0005) [2023-12-27 03:51:16,224][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001710368_437919744.pth... [2023-12-27 03:51:16,226][105692] Updated weights for policy 0, policy_version 1710368 (0.0005) [2023-12-27 03:51:16,227][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001709184_437616640.pth [2023-12-27 03:51:16,674][105620] Updated weights for policy 1, policy_version 1714074 (0.0008) [2023-12-27 03:51:16,725][105620] Updated weights for policy 1, policy_version 1714084 (0.0009) [2023-12-27 03:51:16,775][105620] Updated weights for policy 1, policy_version 1714094 (0.0008) [2023-12-27 03:51:16,932][105692] Updated weights for policy 0, policy_version 1710378 (0.0008) [2023-12-27 03:51:16,977][105692] Updated weights for policy 0, policy_version 1710388 (0.0007) [2023-12-27 03:51:17,037][105692] Updated weights for policy 0, policy_version 1710398 (0.0007) [2023-12-27 03:51:17,571][105620] Updated weights for policy 1, policy_version 1714104 (0.0009) [2023-12-27 03:51:17,622][105620] Updated weights for policy 1, policy_version 1714114 (0.0009) [2023-12-27 03:51:17,677][105620] Updated weights for policy 1, policy_version 1714124 (0.0009) [2023-12-27 03:51:17,738][105692] Updated weights for policy 0, policy_version 1710408 (0.0009) [2023-12-27 03:51:17,789][105692] Updated weights for policy 0, policy_version 1710418 (0.0007) [2023-12-27 03:51:17,844][105692] Updated weights for policy 0, policy_version 1710428 (0.0005) [2023-12-27 03:51:18,459][105692] Updated weights for policy 0, policy_version 1710438 (0.0007) [2023-12-27 03:51:18,506][105692] Updated weights for policy 0, policy_version 1710448 (0.0006) [2023-12-27 03:51:18,508][105620] Updated weights for policy 1, policy_version 1714134 (0.0007) [2023-12-27 03:51:18,560][105692] Updated weights for policy 0, policy_version 1710458 (0.0007) [2023-12-27 03:51:18,566][105620] Updated weights for policy 1, policy_version 1714144 (0.0006) [2023-12-27 03:51:18,629][105620] Updated weights for policy 1, policy_version 1714154 (0.0008) [2023-12-27 03:51:19,362][105692] Updated weights for policy 0, policy_version 1710468 (0.0007) [2023-12-27 03:51:19,418][105620] Updated weights for policy 1, policy_version 1714164 (0.0008) [2023-12-27 03:51:19,420][105692] Updated weights for policy 0, policy_version 1710478 (0.0007) [2023-12-27 03:51:19,475][105692] Updated weights for policy 0, policy_version 1710488 (0.0007) [2023-12-27 03:51:19,479][105620] Updated weights for policy 1, policy_version 1714174 (0.0007) [2023-12-27 03:51:19,543][105620] Updated weights for policy 1, policy_version 1714184 (0.0009) [2023-12-27 03:51:20,226][105620] Updated weights for policy 1, policy_version 1714194 (0.0010) [2023-12-27 03:51:20,294][105620] Updated weights for policy 1, policy_version 1714204 (0.0010) [2023-12-27 03:51:20,296][105692] Updated weights for policy 0, policy_version 1710498 (0.0008) [2023-12-27 03:51:20,353][105692] Updated weights for policy 0, policy_version 1710508 (0.0006) [2023-12-27 03:51:20,359][105620] Updated weights for policy 1, policy_version 1714214 (0.0008) [2023-12-27 03:51:20,410][105692] Updated weights for policy 0, policy_version 1710518 (0.0008) [2023-12-27 03:51:20,423][105620] Updated weights for policy 1, policy_version 1714224 (0.0009) [2023-12-27 03:51:20,466][105692] Updated weights for policy 0, policy_version 1710528 (0.0008) [2023-12-27 03:51:21,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19797.2, 300 sec: 19744.1). Total num frames: 876863488. Throughput: 0: 9846.9, 1: 9744.1. Samples: 876857732. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:51:21,063][104569] Avg episode reward: [(0, '8716.197'), (1, '8986.444')] [2023-12-27 03:51:21,107][105620] Updated weights for policy 1, policy_version 1714234 (0.0008) [2023-12-27 03:51:21,171][105620] Updated weights for policy 1, policy_version 1714244 (0.0009) [2023-12-27 03:51:21,235][105620] Updated weights for policy 1, policy_version 1714254 (0.0009) [2023-12-27 03:51:21,293][105692] Updated weights for policy 0, policy_version 1710538 (0.0009) [2023-12-27 03:51:21,357][105692] Updated weights for policy 0, policy_version 1710548 (0.0009) [2023-12-27 03:51:21,417][105692] Updated weights for policy 0, policy_version 1710558 (0.0008) [2023-12-27 03:51:22,007][105620] Updated weights for policy 1, policy_version 1714264 (0.0006) [2023-12-27 03:51:22,062][105620] Updated weights for policy 1, policy_version 1714274 (0.0007) [2023-12-27 03:51:22,114][105620] Updated weights for policy 1, policy_version 1714284 (0.0007) [2023-12-27 03:51:22,201][105692] Updated weights for policy 0, policy_version 1710568 (0.0009) [2023-12-27 03:51:22,259][105692] Updated weights for policy 0, policy_version 1710578 (0.0009) [2023-12-27 03:51:22,324][105692] Updated weights for policy 0, policy_version 1710588 (0.0008) [2023-12-27 03:51:22,826][105620] Updated weights for policy 1, policy_version 1714294 (0.0008) [2023-12-27 03:51:22,890][105620] Updated weights for policy 1, policy_version 1714304 (0.0009) [2023-12-27 03:51:22,953][105620] Updated weights for policy 1, policy_version 1714314 (0.0008) [2023-12-27 03:51:23,119][105692] Updated weights for policy 0, policy_version 1710598 (0.0009) [2023-12-27 03:51:23,174][105692] Updated weights for policy 0, policy_version 1710608 (0.0009) [2023-12-27 03:51:23,221][105692] Updated weights for policy 0, policy_version 1710618 (0.0009) [2023-12-27 03:51:23,687][105620] Updated weights for policy 1, policy_version 1714324 (0.0009) [2023-12-27 03:51:23,754][105620] Updated weights for policy 1, policy_version 1714334 (0.0009) [2023-12-27 03:51:23,804][105620] Updated weights for policy 1, policy_version 1714344 (0.0009) [2023-12-27 03:51:23,969][105692] Updated weights for policy 0, policy_version 1710628 (0.0009) [2023-12-27 03:51:24,023][105692] Updated weights for policy 0, policy_version 1710638 (0.0009) [2023-12-27 03:51:24,084][105692] Updated weights for policy 0, policy_version 1710648 (0.0009) [2023-12-27 03:51:24,571][105620] Updated weights for policy 1, policy_version 1714354 (0.0009) [2023-12-27 03:51:24,622][105620] Updated weights for policy 1, policy_version 1714364 (0.0009) [2023-12-27 03:51:24,670][105620] Updated weights for policy 1, policy_version 1714374 (0.0009) [2023-12-27 03:51:24,728][105620] Updated weights for policy 1, policy_version 1714384 (0.0009) [2023-12-27 03:51:24,826][105692] Updated weights for policy 0, policy_version 1710658 (0.0010) [2023-12-27 03:51:24,890][105692] Updated weights for policy 0, policy_version 1710668 (0.0008) [2023-12-27 03:51:24,937][105692] Updated weights for policy 0, policy_version 1710678 (0.0009) [2023-12-27 03:51:24,983][105692] Updated weights for policy 0, policy_version 1710688 (0.0008) [2023-12-27 03:51:25,510][105620] Updated weights for policy 1, policy_version 1714394 (0.0009) [2023-12-27 03:51:25,566][105620] Updated weights for policy 1, policy_version 1714404 (0.0009) [2023-12-27 03:51:25,620][105620] Updated weights for policy 1, policy_version 1714414 (0.0009) [2023-12-27 03:51:25,732][105692] Updated weights for policy 0, policy_version 1710698 (0.0007) [2023-12-27 03:51:25,786][105692] Updated weights for policy 0, policy_version 1710708 (0.0005) [2023-12-27 03:51:25,842][105692] Updated weights for policy 0, policy_version 1710718 (0.0005) [2023-12-27 03:51:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19716.3). Total num frames: 876961792. Throughput: 0: 9699.1, 1: 9697.1. Samples: 876967768. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:51:26,062][104569] Avg episode reward: [(0, '9077.871'), (1, '8803.940')] [2023-12-27 03:51:26,429][105620] Updated weights for policy 1, policy_version 1714424 (0.0009) [2023-12-27 03:51:26,479][105620] Updated weights for policy 1, policy_version 1714434 (0.0008) [2023-12-27 03:51:26,521][105692] Updated weights for policy 0, policy_version 1710728 (0.0009) [2023-12-27 03:51:26,538][105620] Updated weights for policy 1, policy_version 1714444 (0.0007) [2023-12-27 03:51:26,577][105692] Updated weights for policy 0, policy_version 1710738 (0.0006) [2023-12-27 03:51:26,622][105692] Updated weights for policy 0, policy_version 1710748 (0.0008) [2023-12-27 03:51:27,321][105620] Updated weights for policy 1, policy_version 1714454 (0.0008) [2023-12-27 03:51:27,335][105692] Updated weights for policy 0, policy_version 1710758 (0.0009) [2023-12-27 03:51:27,381][105620] Updated weights for policy 1, policy_version 1714464 (0.0007) [2023-12-27 03:51:27,383][105692] Updated weights for policy 0, policy_version 1710768 (0.0006) [2023-12-27 03:51:27,431][105692] Updated weights for policy 0, policy_version 1710778 (0.0006) [2023-12-27 03:51:27,436][105620] Updated weights for policy 1, policy_version 1714474 (0.0007) [2023-12-27 03:51:28,183][105692] Updated weights for policy 0, policy_version 1710788 (0.0007) [2023-12-27 03:51:28,193][105620] Updated weights for policy 1, policy_version 1714484 (0.0007) [2023-12-27 03:51:28,231][105692] Updated weights for policy 0, policy_version 1710798 (0.0007) [2023-12-27 03:51:28,252][105620] Updated weights for policy 1, policy_version 1714494 (0.0008) [2023-12-27 03:51:28,278][105692] Updated weights for policy 0, policy_version 1710808 (0.0007) [2023-12-27 03:51:28,296][105620] Updated weights for policy 1, policy_version 1714504 (0.0006) [2023-12-27 03:51:29,051][105620] Updated weights for policy 1, policy_version 1714514 (0.0008) [2023-12-27 03:51:29,054][105692] Updated weights for policy 0, policy_version 1710818 (0.0009) [2023-12-27 03:51:29,100][105620] Updated weights for policy 1, policy_version 1714524 (0.0006) [2023-12-27 03:51:29,103][105692] Updated weights for policy 0, policy_version 1710828 (0.0007) [2023-12-27 03:51:29,155][105620] Updated weights for policy 1, policy_version 1714534 (0.0008) [2023-12-27 03:51:29,161][105692] Updated weights for policy 0, policy_version 1710838 (0.0007) [2023-12-27 03:51:29,215][105620] Updated weights for policy 1, policy_version 1714544 (0.0009) [2023-12-27 03:51:29,218][105692] Updated weights for policy 0, policy_version 1710848 (0.0006) [2023-12-27 03:51:29,888][105692] Updated weights for policy 0, policy_version 1710858 (0.0009) [2023-12-27 03:51:29,918][105620] Updated weights for policy 1, policy_version 1714554 (0.0009) [2023-12-27 03:51:29,946][105692] Updated weights for policy 0, policy_version 1710868 (0.0007) [2023-12-27 03:51:29,986][105620] Updated weights for policy 1, policy_version 1714564 (0.0008) [2023-12-27 03:51:30,002][105692] Updated weights for policy 0, policy_version 1710878 (0.0008) [2023-12-27 03:51:30,050][105620] Updated weights for policy 1, policy_version 1714574 (0.0008) [2023-12-27 03:51:30,667][105692] Updated weights for policy 0, policy_version 1710888 (0.0010) [2023-12-27 03:51:30,724][105692] Updated weights for policy 0, policy_version 1710898 (0.0009) [2023-12-27 03:51:30,784][105692] Updated weights for policy 0, policy_version 1710908 (0.0009) [2023-12-27 03:51:30,836][105620] Updated weights for policy 1, policy_version 1714584 (0.0006) [2023-12-27 03:51:30,899][105620] Updated weights for policy 1, policy_version 1714594 (0.0008) [2023-12-27 03:51:30,948][105620] Updated weights for policy 1, policy_version 1714604 (0.0009) [2023-12-27 03:51:31,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19524.2, 300 sec: 19744.1). Total num frames: 877060096. Throughput: 0: 9751.7, 1: 9664.0. Samples: 877025408. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:51:31,063][104569] Avg episode reward: [(0, '9169.751'), (1, '8529.165')] [2023-12-27 03:51:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001710912_438059008.pth... [2023-12-27 03:51:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001714608_439001088.pth... [2023-12-27 03:51:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001709792_437772288.pth [2023-12-27 03:51:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001713488_438714368.pth [2023-12-27 03:51:31,516][105692] Updated weights for policy 0, policy_version 1710918 (0.0007) [2023-12-27 03:51:31,568][105692] Updated weights for policy 0, policy_version 1710928 (0.0005) [2023-12-27 03:51:31,631][105692] Updated weights for policy 0, policy_version 1710938 (0.0007) [2023-12-27 03:51:31,729][105620] Updated weights for policy 1, policy_version 1714614 (0.0009) [2023-12-27 03:51:31,784][105620] Updated weights for policy 1, policy_version 1714624 (0.0006) [2023-12-27 03:51:31,842][105620] Updated weights for policy 1, policy_version 1714634 (0.0006) [2023-12-27 03:51:32,265][105692] Updated weights for policy 0, policy_version 1710948 (0.0011) [2023-12-27 03:51:32,317][105692] Updated weights for policy 0, policy_version 1710958 (0.0010) [2023-12-27 03:51:32,375][105692] Updated weights for policy 0, policy_version 1710968 (0.0009) [2023-12-27 03:51:32,577][105620] Updated weights for policy 1, policy_version 1714644 (0.0007) [2023-12-27 03:51:32,635][105620] Updated weights for policy 1, policy_version 1714654 (0.0009) [2023-12-27 03:51:32,698][105620] Updated weights for policy 1, policy_version 1714664 (0.0009) [2023-12-27 03:51:33,141][105692] Updated weights for policy 0, policy_version 1710978 (0.0009) [2023-12-27 03:51:33,191][105692] Updated weights for policy 0, policy_version 1710988 (0.0009) [2023-12-27 03:51:33,237][105692] Updated weights for policy 0, policy_version 1710998 (0.0009) [2023-12-27 03:51:33,284][105692] Updated weights for policy 0, policy_version 1711008 (0.0009) [2023-12-27 03:51:33,405][105620] Updated weights for policy 1, policy_version 1714674 (0.0007) [2023-12-27 03:51:33,463][105620] Updated weights for policy 1, policy_version 1714684 (0.0009) [2023-12-27 03:51:33,517][105620] Updated weights for policy 1, policy_version 1714694 (0.0009) [2023-12-27 03:51:33,570][105620] Updated weights for policy 1, policy_version 1714704 (0.0009) [2023-12-27 03:51:34,040][105692] Updated weights for policy 0, policy_version 1711018 (0.0009) [2023-12-27 03:51:34,103][105692] Updated weights for policy 0, policy_version 1711028 (0.0009) [2023-12-27 03:51:34,169][105692] Updated weights for policy 0, policy_version 1711038 (0.0009) [2023-12-27 03:51:34,339][105620] Updated weights for policy 1, policy_version 1714714 (0.0009) [2023-12-27 03:51:34,387][105620] Updated weights for policy 1, policy_version 1714724 (0.0009) [2023-12-27 03:51:34,442][105620] Updated weights for policy 1, policy_version 1714734 (0.0009) [2023-12-27 03:51:34,918][105692] Updated weights for policy 0, policy_version 1711048 (0.0009) [2023-12-27 03:51:34,967][105692] Updated weights for policy 0, policy_version 1711058 (0.0008) [2023-12-27 03:51:35,022][105692] Updated weights for policy 0, policy_version 1711068 (0.0009) [2023-12-27 03:51:35,214][105620] Updated weights for policy 1, policy_version 1714744 (0.0009) [2023-12-27 03:51:35,262][105620] Updated weights for policy 1, policy_version 1714754 (0.0009) [2023-12-27 03:51:35,317][105620] Updated weights for policy 1, policy_version 1714764 (0.0009) [2023-12-27 03:51:35,737][105692] Updated weights for policy 0, policy_version 1711078 (0.0006) [2023-12-27 03:51:35,792][105692] Updated weights for policy 0, policy_version 1711088 (0.0006) [2023-12-27 03:51:35,843][105692] Updated weights for policy 0, policy_version 1711098 (0.0005) [2023-12-27 03:51:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19716.3). Total num frames: 877150208. Throughput: 0: 9714.6, 1: 9604.6. Samples: 877140312. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:51:36,063][104569] Avg episode reward: [(0, '9079.027'), (1, '8895.394')] [2023-12-27 03:51:36,206][105620] Updated weights for policy 1, policy_version 1714774 (0.0010) [2023-12-27 03:51:36,266][105620] Updated weights for policy 1, policy_version 1714784 (0.0009) [2023-12-27 03:51:36,328][105620] Updated weights for policy 1, policy_version 1714794 (0.0009) [2023-12-27 03:51:36,468][105692] Updated weights for policy 0, policy_version 1711108 (0.0007) [2023-12-27 03:51:36,530][105692] Updated weights for policy 0, policy_version 1711118 (0.0009) [2023-12-27 03:51:36,593][105692] Updated weights for policy 0, policy_version 1711128 (0.0009) [2023-12-27 03:51:37,110][105620] Updated weights for policy 1, policy_version 1714804 (0.0010) [2023-12-27 03:51:37,173][105620] Updated weights for policy 1, policy_version 1714814 (0.0009) [2023-12-27 03:51:37,234][105620] Updated weights for policy 1, policy_version 1714824 (0.0008) [2023-12-27 03:51:37,320][105692] Updated weights for policy 0, policy_version 1711138 (0.0008) [2023-12-27 03:51:37,368][105692] Updated weights for policy 0, policy_version 1711148 (0.0009) [2023-12-27 03:51:37,416][105692] Updated weights for policy 0, policy_version 1711158 (0.0009) [2023-12-27 03:51:37,468][105692] Updated weights for policy 0, policy_version 1711168 (0.0009) [2023-12-27 03:51:37,972][105620] Updated weights for policy 1, policy_version 1714834 (0.0008) [2023-12-27 03:51:38,020][105620] Updated weights for policy 1, policy_version 1714844 (0.0007) [2023-12-27 03:51:38,075][105620] Updated weights for policy 1, policy_version 1714854 (0.0008) [2023-12-27 03:51:38,121][105620] Updated weights for policy 1, policy_version 1714864 (0.0007) [2023-12-27 03:51:38,271][105692] Updated weights for policy 0, policy_version 1711178 (0.0006) [2023-12-27 03:51:38,327][105692] Updated weights for policy 0, policy_version 1711188 (0.0006) [2023-12-27 03:51:38,388][105692] Updated weights for policy 0, policy_version 1711198 (0.0007) [2023-12-27 03:51:38,889][105620] Updated weights for policy 1, policy_version 1714874 (0.0010) [2023-12-27 03:51:38,953][105620] Updated weights for policy 1, policy_version 1714884 (0.0009) [2023-12-27 03:51:38,987][105692] Updated weights for policy 0, policy_version 1711208 (0.0006) [2023-12-27 03:51:39,010][105620] Updated weights for policy 1, policy_version 1714894 (0.0006) [2023-12-27 03:51:39,046][105692] Updated weights for policy 0, policy_version 1711218 (0.0008) [2023-12-27 03:51:39,106][105692] Updated weights for policy 0, policy_version 1711228 (0.0008) [2023-12-27 03:51:39,743][105620] Updated weights for policy 1, policy_version 1714904 (0.0008) [2023-12-27 03:51:39,806][105620] Updated weights for policy 1, policy_version 1714914 (0.0009) [2023-12-27 03:51:39,870][105620] Updated weights for policy 1, policy_version 1714924 (0.0009) [2023-12-27 03:51:39,921][105692] Updated weights for policy 0, policy_version 1711238 (0.0008) [2023-12-27 03:51:39,986][105692] Updated weights for policy 0, policy_version 1711248 (0.0009) [2023-12-27 03:51:40,046][105692] Updated weights for policy 0, policy_version 1711258 (0.0008) [2023-12-27 03:51:40,577][105620] Updated weights for policy 1, policy_version 1714934 (0.0009) [2023-12-27 03:51:40,631][105620] Updated weights for policy 1, policy_version 1714944 (0.0010) [2023-12-27 03:51:40,691][105620] Updated weights for policy 1, policy_version 1714954 (0.0009) [2023-12-27 03:51:40,801][105692] Updated weights for policy 0, policy_version 1711268 (0.0009) [2023-12-27 03:51:40,853][105692] Updated weights for policy 0, policy_version 1711278 (0.0009) [2023-12-27 03:51:40,907][105692] Updated weights for policy 0, policy_version 1711288 (0.0008) [2023-12-27 03:51:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19716.3). Total num frames: 877248512. Throughput: 0: 9642.9, 1: 9510.7. Samples: 877253096. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:51:41,063][104569] Avg episode reward: [(0, '8624.436'), (1, '9078.030')] [2023-12-27 03:51:41,509][105620] Updated weights for policy 1, policy_version 1714964 (0.0009) [2023-12-27 03:51:41,567][105620] Updated weights for policy 1, policy_version 1714974 (0.0008) [2023-12-27 03:51:41,622][105620] Updated weights for policy 1, policy_version 1714984 (0.0008) [2023-12-27 03:51:41,690][105692] Updated weights for policy 0, policy_version 1711298 (0.0006) [2023-12-27 03:51:41,758][105692] Updated weights for policy 0, policy_version 1711308 (0.0009) [2023-12-27 03:51:41,819][105692] Updated weights for policy 0, policy_version 1711318 (0.0008) [2023-12-27 03:51:41,880][105692] Updated weights for policy 0, policy_version 1711328 (0.0008) [2023-12-27 03:51:42,439][105620] Updated weights for policy 1, policy_version 1714994 (0.0008) [2023-12-27 03:51:42,498][105620] Updated weights for policy 1, policy_version 1715004 (0.0008) [2023-12-27 03:51:42,557][105620] Updated weights for policy 1, policy_version 1715014 (0.0007) [2023-12-27 03:51:42,585][105692] Updated weights for policy 0, policy_version 1711338 (0.0007) [2023-12-27 03:51:42,615][105620] Updated weights for policy 1, policy_version 1715024 (0.0007) [2023-12-27 03:51:42,647][105692] Updated weights for policy 0, policy_version 1711348 (0.0008) [2023-12-27 03:51:42,702][105692] Updated weights for policy 0, policy_version 1711358 (0.0008) [2023-12-27 03:51:43,386][105692] Updated weights for policy 0, policy_version 1711368 (0.0008) [2023-12-27 03:51:43,403][105620] Updated weights for policy 1, policy_version 1715034 (0.0009) [2023-12-27 03:51:43,432][105692] Updated weights for policy 0, policy_version 1711378 (0.0006) [2023-12-27 03:51:43,459][105620] Updated weights for policy 1, policy_version 1715044 (0.0008) [2023-12-27 03:51:43,489][105692] Updated weights for policy 0, policy_version 1711388 (0.0006) [2023-12-27 03:51:43,516][105620] Updated weights for policy 1, policy_version 1715054 (0.0008) [2023-12-27 03:51:44,253][105620] Updated weights for policy 1, policy_version 1715064 (0.0009) [2023-12-27 03:51:44,254][105692] Updated weights for policy 0, policy_version 1711398 (0.0008) [2023-12-27 03:51:44,304][105620] Updated weights for policy 1, policy_version 1715074 (0.0006) [2023-12-27 03:51:44,313][105692] Updated weights for policy 0, policy_version 1711408 (0.0010) [2023-12-27 03:51:44,354][105620] Updated weights for policy 1, policy_version 1715084 (0.0008) [2023-12-27 03:51:44,368][105692] Updated weights for policy 0, policy_version 1711418 (0.0006) [2023-12-27 03:51:45,096][105692] Updated weights for policy 0, policy_version 1711428 (0.0007) [2023-12-27 03:51:45,158][105692] Updated weights for policy 0, policy_version 1711438 (0.0006) [2023-12-27 03:51:45,173][105620] Updated weights for policy 1, policy_version 1715094 (0.0008) [2023-12-27 03:51:45,207][105692] Updated weights for policy 0, policy_version 1711448 (0.0006) [2023-12-27 03:51:45,230][105620] Updated weights for policy 1, policy_version 1715104 (0.0008) [2023-12-27 03:51:45,290][105620] Updated weights for policy 1, policy_version 1715114 (0.0007) [2023-12-27 03:51:45,848][105692] Updated weights for policy 0, policy_version 1711458 (0.0008) [2023-12-27 03:51:45,897][105692] Updated weights for policy 0, policy_version 1711468 (0.0007) [2023-12-27 03:51:45,941][105692] Updated weights for policy 0, policy_version 1711478 (0.0008) [2023-12-27 03:51:45,989][105692] Updated weights for policy 0, policy_version 1711488 (0.0005) [2023-12-27 03:51:46,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19251.1, 300 sec: 19688.6). Total num frames: 877338624. Throughput: 0: 9575.9, 1: 9493.1. Samples: 877308796. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:51:46,063][104569] Avg episode reward: [(0, '8081.322'), (1, '9078.317')] [2023-12-27 03:51:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001715120_439132160.pth... [2023-12-27 03:51:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001711488_438206464.pth... [2023-12-27 03:51:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001714064_438861824.pth [2023-12-27 03:51:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001710368_437919744.pth [2023-12-27 03:51:46,132][105620] Updated weights for policy 1, policy_version 1715124 (0.0009) [2023-12-27 03:51:46,184][105620] Updated weights for policy 1, policy_version 1715134 (0.0008) [2023-12-27 03:51:46,236][105620] Updated weights for policy 1, policy_version 1715144 (0.0008) [2023-12-27 03:51:46,695][105692] Updated weights for policy 0, policy_version 1711498 (0.0010) [2023-12-27 03:51:46,744][105692] Updated weights for policy 0, policy_version 1711508 (0.0010) [2023-12-27 03:51:46,799][105692] Updated weights for policy 0, policy_version 1711518 (0.0010) [2023-12-27 03:51:47,011][105620] Updated weights for policy 1, policy_version 1715154 (0.0008) [2023-12-27 03:51:47,069][105620] Updated weights for policy 1, policy_version 1715164 (0.0007) [2023-12-27 03:51:47,119][105620] Updated weights for policy 1, policy_version 1715174 (0.0007) [2023-12-27 03:51:47,167][105620] Updated weights for policy 1, policy_version 1715184 (0.0008) [2023-12-27 03:51:47,570][105692] Updated weights for policy 0, policy_version 1711528 (0.0010) [2023-12-27 03:51:47,648][105692] Updated weights for policy 0, policy_version 1711538 (0.0010) [2023-12-27 03:51:47,716][105692] Updated weights for policy 0, policy_version 1711548 (0.0010) [2023-12-27 03:51:47,836][105620] Updated weights for policy 1, policy_version 1715194 (0.0005) [2023-12-27 03:51:47,902][105620] Updated weights for policy 1, policy_version 1715204 (0.0005) [2023-12-27 03:51:47,948][105620] Updated weights for policy 1, policy_version 1715214 (0.0005) [2023-12-27 03:51:48,406][105692] Updated weights for policy 0, policy_version 1711558 (0.0010) [2023-12-27 03:51:48,462][105692] Updated weights for policy 0, policy_version 1711568 (0.0010) [2023-12-27 03:51:48,476][105620] Updated weights for policy 1, policy_version 1715224 (0.0006) [2023-12-27 03:51:48,517][105692] Updated weights for policy 0, policy_version 1711578 (0.0010) [2023-12-27 03:51:48,536][105620] Updated weights for policy 1, policy_version 1715234 (0.0007) [2023-12-27 03:51:48,595][105620] Updated weights for policy 1, policy_version 1715244 (0.0007) [2023-12-27 03:51:49,198][105620] Updated weights for policy 1, policy_version 1715254 (0.0008) [2023-12-27 03:51:49,263][105620] Updated weights for policy 1, policy_version 1715264 (0.0008) [2023-12-27 03:51:49,268][105692] Updated weights for policy 0, policy_version 1711588 (0.0010) [2023-12-27 03:51:49,318][105620] Updated weights for policy 1, policy_version 1715274 (0.0006) [2023-12-27 03:51:49,327][105692] Updated weights for policy 0, policy_version 1711598 (0.0010) [2023-12-27 03:51:49,390][105692] Updated weights for policy 0, policy_version 1711608 (0.0009) [2023-12-27 03:51:50,064][105620] Updated weights for policy 1, policy_version 1715284 (0.0008) [2023-12-27 03:51:50,129][105620] Updated weights for policy 1, policy_version 1715294 (0.0009) [2023-12-27 03:51:50,189][105620] Updated weights for policy 1, policy_version 1715304 (0.0008) [2023-12-27 03:51:50,197][105692] Updated weights for policy 0, policy_version 1711618 (0.0010) [2023-12-27 03:51:50,259][105692] Updated weights for policy 0, policy_version 1711628 (0.0010) [2023-12-27 03:51:50,317][105692] Updated weights for policy 0, policy_version 1711638 (0.0010) [2023-12-27 03:51:50,376][105692] Updated weights for policy 0, policy_version 1711648 (0.0010) [2023-12-27 03:51:50,954][105620] Updated weights for policy 1, policy_version 1715314 (0.0007) [2023-12-27 03:51:51,018][105620] Updated weights for policy 1, policy_version 1715324 (0.0008) [2023-12-27 03:51:51,062][104569] Fps is (10 sec: 18022.6, 60 sec: 19114.7, 300 sec: 19660.8). Total num frames: 877428736. Throughput: 0: 9529.9, 1: 9535.6. Samples: 877426332. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:51:51,062][104569] Avg episode reward: [(0, '8258.459'), (1, '8986.311')] [2023-12-27 03:51:51,076][105620] Updated weights for policy 1, policy_version 1715334 (0.0009) [2023-12-27 03:51:51,132][105620] Updated weights for policy 1, policy_version 1715344 (0.0008) [2023-12-27 03:51:51,152][105692] Updated weights for policy 0, policy_version 1711658 (0.0009) [2023-12-27 03:51:51,218][105692] Updated weights for policy 0, policy_version 1711668 (0.0011) [2023-12-27 03:51:51,282][105692] Updated weights for policy 0, policy_version 1711678 (0.0011) [2023-12-27 03:51:51,871][105620] Updated weights for policy 1, policy_version 1715354 (0.0005) [2023-12-27 03:51:51,931][105620] Updated weights for policy 1, policy_version 1715364 (0.0005) [2023-12-27 03:51:51,990][105620] Updated weights for policy 1, policy_version 1715374 (0.0005) [2023-12-27 03:51:52,047][105692] Updated weights for policy 0, policy_version 1711688 (0.0010) [2023-12-27 03:51:52,099][105692] Updated weights for policy 0, policy_version 1711698 (0.0010) [2023-12-27 03:51:52,153][105692] Updated weights for policy 0, policy_version 1711708 (0.0008) [2023-12-27 03:51:52,712][105620] Updated weights for policy 1, policy_version 1715384 (0.0008) [2023-12-27 03:51:52,772][105620] Updated weights for policy 1, policy_version 1715394 (0.0008) [2023-12-27 03:51:52,797][105692] Updated weights for policy 0, policy_version 1711718 (0.0008) [2023-12-27 03:51:52,832][105620] Updated weights for policy 1, policy_version 1715404 (0.0006) [2023-12-27 03:51:52,850][105692] Updated weights for policy 0, policy_version 1711728 (0.0010) [2023-12-27 03:51:52,901][105692] Updated weights for policy 0, policy_version 1711738 (0.0010) [2023-12-27 03:51:53,604][105620] Updated weights for policy 1, policy_version 1715414 (0.0007) [2023-12-27 03:51:53,620][105692] Updated weights for policy 0, policy_version 1711748 (0.0009) [2023-12-27 03:51:53,666][105620] Updated weights for policy 1, policy_version 1715424 (0.0006) [2023-12-27 03:51:53,682][105692] Updated weights for policy 0, policy_version 1711758 (0.0008) [2023-12-27 03:51:53,720][105620] Updated weights for policy 1, policy_version 1715434 (0.0008) [2023-12-27 03:51:53,749][105692] Updated weights for policy 0, policy_version 1711768 (0.0009) [2023-12-27 03:51:54,432][105692] Updated weights for policy 0, policy_version 1711778 (0.0008) [2023-12-27 03:51:54,477][105620] Updated weights for policy 1, policy_version 1715444 (0.0006) [2023-12-27 03:51:54,495][105692] Updated weights for policy 0, policy_version 1711788 (0.0010) [2023-12-27 03:51:54,545][105620] Updated weights for policy 1, policy_version 1715454 (0.0006) [2023-12-27 03:51:54,550][105692] Updated weights for policy 0, policy_version 1711798 (0.0009) [2023-12-27 03:51:54,609][105692] Updated weights for policy 0, policy_version 1711808 (0.0010) [2023-12-27 03:51:54,612][105620] Updated weights for policy 1, policy_version 1715464 (0.0006) [2023-12-27 03:51:55,239][105620] Updated weights for policy 1, policy_version 1715474 (0.0007) [2023-12-27 03:51:55,264][105692] Updated weights for policy 0, policy_version 1711818 (0.0011) [2023-12-27 03:51:55,295][105620] Updated weights for policy 1, policy_version 1715484 (0.0011) [2023-12-27 03:51:55,321][105692] Updated weights for policy 0, policy_version 1711828 (0.0009) [2023-12-27 03:51:55,356][105620] Updated weights for policy 1, policy_version 1715494 (0.0011) [2023-12-27 03:51:55,381][105692] Updated weights for policy 0, policy_version 1711838 (0.0005) [2023-12-27 03:51:55,406][105620] Updated weights for policy 1, policy_version 1715504 (0.0011) [2023-12-27 03:51:55,939][105692] Updated weights for policy 0, policy_version 1711848 (0.0007) [2023-12-27 03:51:55,993][105692] Updated weights for policy 0, policy_version 1711858 (0.0008) [2023-12-27 03:51:56,050][105692] Updated weights for policy 0, policy_version 1711868 (0.0010) [2023-12-27 03:51:56,062][104569] Fps is (10 sec: 18842.1, 60 sec: 18978.2, 300 sec: 19660.8). Total num frames: 877527040. Throughput: 0: 9648.9, 1: 9461.3. Samples: 877541656. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:51:56,063][104569] Avg episode reward: [(0, '8623.312'), (1, '8895.190')] [2023-12-27 03:51:56,228][105620] Updated weights for policy 1, policy_version 1715514 (0.0009) [2023-12-27 03:51:56,288][105620] Updated weights for policy 1, policy_version 1715524 (0.0011) [2023-12-27 03:51:56,337][105620] Updated weights for policy 1, policy_version 1715534 (0.0010) [2023-12-27 03:51:56,738][105692] Updated weights for policy 0, policy_version 1711878 (0.0008) [2023-12-27 03:51:56,793][105692] Updated weights for policy 0, policy_version 1711888 (0.0008) [2023-12-27 03:51:56,843][105692] Updated weights for policy 0, policy_version 1711898 (0.0010) [2023-12-27 03:51:56,984][105620] Updated weights for policy 1, policy_version 1715544 (0.0009) [2023-12-27 03:51:57,049][105620] Updated weights for policy 1, policy_version 1715554 (0.0009) [2023-12-27 03:51:57,106][105620] Updated weights for policy 1, policy_version 1715564 (0.0005) [2023-12-27 03:51:57,455][105692] Updated weights for policy 0, policy_version 1711908 (0.0009) [2023-12-27 03:51:57,506][105692] Updated weights for policy 0, policy_version 1711918 (0.0008) [2023-12-27 03:51:57,572][105692] Updated weights for policy 0, policy_version 1711928 (0.0005) [2023-12-27 03:51:57,758][105620] Updated weights for policy 1, policy_version 1715574 (0.0008) [2023-12-27 03:51:57,812][105620] Updated weights for policy 1, policy_version 1715584 (0.0010) [2023-12-27 03:51:57,860][105620] Updated weights for policy 1, policy_version 1715594 (0.0010) [2023-12-27 03:51:58,179][105692] Updated weights for policy 0, policy_version 1711938 (0.0006) [2023-12-27 03:51:58,231][105692] Updated weights for policy 0, policy_version 1711948 (0.0010) [2023-12-27 03:51:58,286][105692] Updated weights for policy 0, policy_version 1711958 (0.0011) [2023-12-27 03:51:58,342][105692] Updated weights for policy 0, policy_version 1711968 (0.0010) [2023-12-27 03:51:58,693][105620] Updated weights for policy 1, policy_version 1715604 (0.0010) [2023-12-27 03:51:58,753][105620] Updated weights for policy 1, policy_version 1715614 (0.0008) [2023-12-27 03:51:58,829][105620] Updated weights for policy 1, policy_version 1715624 (0.0010) [2023-12-27 03:51:59,097][105692] Updated weights for policy 0, policy_version 1711978 (0.0009) [2023-12-27 03:51:59,146][105692] Updated weights for policy 0, policy_version 1711988 (0.0009) [2023-12-27 03:51:59,204][105692] Updated weights for policy 0, policy_version 1711998 (0.0009) [2023-12-27 03:51:59,652][105620] Updated weights for policy 1, policy_version 1715634 (0.0008) [2023-12-27 03:51:59,709][105620] Updated weights for policy 1, policy_version 1715644 (0.0007) [2023-12-27 03:51:59,785][105620] Updated weights for policy 1, policy_version 1715654 (0.0011) [2023-12-27 03:51:59,851][105620] Updated weights for policy 1, policy_version 1715664 (0.0010) [2023-12-27 03:52:00,030][105692] Updated weights for policy 0, policy_version 1712008 (0.0008) [2023-12-27 03:52:00,096][105692] Updated weights for policy 0, policy_version 1712018 (0.0011) [2023-12-27 03:52:00,165][105692] Updated weights for policy 0, policy_version 1712028 (0.0011) [2023-12-27 03:52:00,510][105620] Updated weights for policy 1, policy_version 1715674 (0.0011) [2023-12-27 03:52:00,573][105620] Updated weights for policy 1, policy_version 1715684 (0.0010) [2023-12-27 03:52:00,640][105620] Updated weights for policy 1, policy_version 1715694 (0.0008) [2023-12-27 03:52:00,785][105692] Updated weights for policy 0, policy_version 1712038 (0.0008) [2023-12-27 03:52:00,843][105692] Updated weights for policy 0, policy_version 1712048 (0.0007) [2023-12-27 03:52:00,895][105692] Updated weights for policy 0, policy_version 1712058 (0.0009) [2023-12-27 03:52:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19251.2, 300 sec: 19660.8). Total num frames: 877633536. Throughput: 0: 9769.8, 1: 9398.9. Samples: 877602660. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:52:01,062][104569] Avg episode reward: [(0, '8447.178'), (1, '8896.775')] [2023-12-27 03:52:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001712064_438353920.pth... [2023-12-27 03:52:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001715696_439279616.pth... [2023-12-27 03:52:01,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001710912_438059008.pth [2023-12-27 03:52:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001714608_439001088.pth [2023-12-27 03:52:01,276][105620] Updated weights for policy 1, policy_version 1715704 (0.0008) [2023-12-27 03:52:01,331][105620] Updated weights for policy 1, policy_version 1715714 (0.0008) [2023-12-27 03:52:01,399][105620] Updated weights for policy 1, policy_version 1715724 (0.0007) [2023-12-27 03:52:01,607][105692] Updated weights for policy 0, policy_version 1712068 (0.0009) [2023-12-27 03:52:01,668][105692] Updated weights for policy 0, policy_version 1712078 (0.0009) [2023-12-27 03:52:01,722][105692] Updated weights for policy 0, policy_version 1712088 (0.0006) [2023-12-27 03:52:02,125][105620] Updated weights for policy 1, policy_version 1715734 (0.0007) [2023-12-27 03:52:02,180][105620] Updated weights for policy 1, policy_version 1715744 (0.0008) [2023-12-27 03:52:02,236][105620] Updated weights for policy 1, policy_version 1715754 (0.0008) [2023-12-27 03:52:02,398][105692] Updated weights for policy 0, policy_version 1712098 (0.0010) [2023-12-27 03:52:02,457][105692] Updated weights for policy 0, policy_version 1712108 (0.0007) [2023-12-27 03:52:02,510][105692] Updated weights for policy 0, policy_version 1712118 (0.0010) [2023-12-27 03:52:02,569][105692] Updated weights for policy 0, policy_version 1712128 (0.0010) [2023-12-27 03:52:02,992][105620] Updated weights for policy 1, policy_version 1715764 (0.0007) [2023-12-27 03:52:03,050][105620] Updated weights for policy 1, policy_version 1715774 (0.0009) [2023-12-27 03:52:03,104][105620] Updated weights for policy 1, policy_version 1715784 (0.0005) [2023-12-27 03:52:03,173][105692] Updated weights for policy 0, policy_version 1712138 (0.0005) [2023-12-27 03:52:03,217][105692] Updated weights for policy 0, policy_version 1712148 (0.0006) [2023-12-27 03:52:03,261][105692] Updated weights for policy 0, policy_version 1712158 (0.0010) [2023-12-27 03:52:03,637][105620] Updated weights for policy 1, policy_version 1715794 (0.0005) [2023-12-27 03:52:03,699][105620] Updated weights for policy 1, policy_version 1715804 (0.0005) [2023-12-27 03:52:03,758][105620] Updated weights for policy 1, policy_version 1715814 (0.0009) [2023-12-27 03:52:03,819][105620] Updated weights for policy 1, policy_version 1715824 (0.0010) [2023-12-27 03:52:03,866][105692] Updated weights for policy 0, policy_version 1712168 (0.0008) [2023-12-27 03:52:03,915][105692] Updated weights for policy 0, policy_version 1712178 (0.0006) [2023-12-27 03:52:03,971][105692] Updated weights for policy 0, policy_version 1712188 (0.0011) [2023-12-27 03:52:04,420][105620] Updated weights for policy 1, policy_version 1715834 (0.0007) [2023-12-27 03:52:04,488][105620] Updated weights for policy 1, policy_version 1715844 (0.0008) [2023-12-27 03:52:04,554][105620] Updated weights for policy 1, policy_version 1715854 (0.0007) [2023-12-27 03:52:04,725][105692] Updated weights for policy 0, policy_version 1712198 (0.0011) [2023-12-27 03:52:04,769][105692] Updated weights for policy 0, policy_version 1712208 (0.0010) [2023-12-27 03:52:04,821][105692] Updated weights for policy 0, policy_version 1712218 (0.0010) [2023-12-27 03:52:05,226][105620] Updated weights for policy 1, policy_version 1715864 (0.0010) [2023-12-27 03:52:05,281][105620] Updated weights for policy 1, policy_version 1715874 (0.0010) [2023-12-27 03:52:05,335][105620] Updated weights for policy 1, policy_version 1715884 (0.0010) [2023-12-27 03:52:05,450][105692] Updated weights for policy 0, policy_version 1712228 (0.0008) [2023-12-27 03:52:05,521][105692] Updated weights for policy 0, policy_version 1712238 (0.0006) [2023-12-27 03:52:05,576][105692] Updated weights for policy 0, policy_version 1712248 (0.0006) [2023-12-27 03:52:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19251.2, 300 sec: 19660.8). Total num frames: 877731840. Throughput: 0: 9766.6, 1: 9493.4. Samples: 877724424. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:52:06,062][104569] Avg episode reward: [(0, '8721.586'), (1, '8894.230')] [2023-12-27 03:52:06,067][105620] Updated weights for policy 1, policy_version 1715894 (0.0010) [2023-12-27 03:52:06,136][105620] Updated weights for policy 1, policy_version 1715904 (0.0007) [2023-12-27 03:52:06,199][105620] Updated weights for policy 1, policy_version 1715914 (0.0006) [2023-12-27 03:52:06,260][105692] Updated weights for policy 0, policy_version 1712258 (0.0007) [2023-12-27 03:52:06,318][105692] Updated weights for policy 0, policy_version 1712268 (0.0007) [2023-12-27 03:52:06,380][105692] Updated weights for policy 0, policy_version 1712278 (0.0006) [2023-12-27 03:52:06,447][105692] Updated weights for policy 0, policy_version 1712288 (0.0006) [2023-12-27 03:52:06,818][105620] Updated weights for policy 1, policy_version 1715924 (0.0007) [2023-12-27 03:52:06,877][105620] Updated weights for policy 1, policy_version 1715934 (0.0007) [2023-12-27 03:52:06,934][105620] Updated weights for policy 1, policy_version 1715944 (0.0008) [2023-12-27 03:52:07,007][105692] Updated weights for policy 0, policy_version 1712298 (0.0007) [2023-12-27 03:52:07,066][105692] Updated weights for policy 0, policy_version 1712308 (0.0005) [2023-12-27 03:52:07,124][105692] Updated weights for policy 0, policy_version 1712318 (0.0005) [2023-12-27 03:52:07,510][105620] Updated weights for policy 1, policy_version 1715954 (0.0007) [2023-12-27 03:52:07,567][105620] Updated weights for policy 1, policy_version 1715964 (0.0005) [2023-12-27 03:52:07,628][105620] Updated weights for policy 1, policy_version 1715974 (0.0005) [2023-12-27 03:52:07,689][105620] Updated weights for policy 1, policy_version 1715984 (0.0006) [2023-12-27 03:52:07,772][105692] Updated weights for policy 0, policy_version 1712328 (0.0007) [2023-12-27 03:52:07,820][105692] Updated weights for policy 0, policy_version 1712338 (0.0005) [2023-12-27 03:52:07,878][105692] Updated weights for policy 0, policy_version 1712348 (0.0005) [2023-12-27 03:52:08,317][105620] Updated weights for policy 1, policy_version 1715994 (0.0005) [2023-12-27 03:52:08,381][105620] Updated weights for policy 1, policy_version 1716004 (0.0007) [2023-12-27 03:52:08,439][105620] Updated weights for policy 1, policy_version 1716014 (0.0008) [2023-12-27 03:52:08,455][105692] Updated weights for policy 0, policy_version 1712358 (0.0009) [2023-12-27 03:52:08,517][105692] Updated weights for policy 0, policy_version 1712368 (0.0011) [2023-12-27 03:52:08,582][105692] Updated weights for policy 0, policy_version 1712378 (0.0011) [2023-12-27 03:52:09,196][105692] Updated weights for policy 0, policy_version 1712388 (0.0011) [2023-12-27 03:52:09,214][105620] Updated weights for policy 1, policy_version 1716024 (0.0009) [2023-12-27 03:52:09,260][105692] Updated weights for policy 0, policy_version 1712398 (0.0010) [2023-12-27 03:52:09,279][105620] Updated weights for policy 1, policy_version 1716034 (0.0007) [2023-12-27 03:52:09,325][105692] Updated weights for policy 0, policy_version 1712408 (0.0007) [2023-12-27 03:52:09,346][105620] Updated weights for policy 1, policy_version 1716044 (0.0007) [2023-12-27 03:52:09,978][105692] Updated weights for policy 0, policy_version 1712418 (0.0009) [2023-12-27 03:52:10,046][105692] Updated weights for policy 0, policy_version 1712428 (0.0009) [2023-12-27 03:52:10,108][105692] Updated weights for policy 0, policy_version 1712438 (0.0008) [2023-12-27 03:52:10,166][105620] Updated weights for policy 1, policy_version 1716054 (0.0007) [2023-12-27 03:52:10,172][105692] Updated weights for policy 0, policy_version 1712448 (0.0009) [2023-12-27 03:52:10,224][105620] Updated weights for policy 1, policy_version 1716064 (0.0009) [2023-12-27 03:52:10,292][105620] Updated weights for policy 1, policy_version 1716074 (0.0008) [2023-12-27 03:52:10,883][105692] Updated weights for policy 0, policy_version 1712458 (0.0008) [2023-12-27 03:52:10,949][105692] Updated weights for policy 0, policy_version 1712468 (0.0009) [2023-12-27 03:52:11,005][105692] Updated weights for policy 0, policy_version 1712478 (0.0009) [2023-12-27 03:52:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.8, 300 sec: 19716.3). Total num frames: 877838336. Throughput: 0: 9985.7, 1: 9555.0. Samples: 877847100. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:52:11,062][104569] Avg episode reward: [(0, '8532.887'), (1, '8984.407')] [2023-12-27 03:52:11,080][105620] Updated weights for policy 1, policy_version 1716084 (0.0008) [2023-12-27 03:52:11,153][105620] Updated weights for policy 1, policy_version 1716094 (0.0009) [2023-12-27 03:52:11,226][105620] Updated weights for policy 1, policy_version 1716104 (0.0009) [2023-12-27 03:52:11,777][105692] Updated weights for policy 0, policy_version 1712488 (0.0008) [2023-12-27 03:52:11,839][105692] Updated weights for policy 0, policy_version 1712498 (0.0009) [2023-12-27 03:52:11,894][105692] Updated weights for policy 0, policy_version 1712508 (0.0009) [2023-12-27 03:52:11,983][105620] Updated weights for policy 1, policy_version 1716114 (0.0009) [2023-12-27 03:52:12,041][105620] Updated weights for policy 1, policy_version 1716124 (0.0009) [2023-12-27 03:52:12,097][105620] Updated weights for policy 1, policy_version 1716134 (0.0009) [2023-12-27 03:52:12,166][105620] Updated weights for policy 1, policy_version 1716144 (0.0006) [2023-12-27 03:52:12,676][105692] Updated weights for policy 0, policy_version 1712518 (0.0009) [2023-12-27 03:52:12,741][105692] Updated weights for policy 0, policy_version 1712528 (0.0008) [2023-12-27 03:52:12,811][105692] Updated weights for policy 0, policy_version 1712538 (0.0009) [2023-12-27 03:52:12,884][105620] Updated weights for policy 1, policy_version 1716154 (0.0008) [2023-12-27 03:52:12,942][105620] Updated weights for policy 1, policy_version 1716164 (0.0009) [2023-12-27 03:52:12,992][105620] Updated weights for policy 1, policy_version 1716174 (0.0009) [2023-12-27 03:52:13,467][105692] Updated weights for policy 0, policy_version 1712548 (0.0008) [2023-12-27 03:52:13,515][105692] Updated weights for policy 0, policy_version 1712558 (0.0009) [2023-12-27 03:52:13,562][105692] Updated weights for policy 0, policy_version 1712568 (0.0008) [2023-12-27 03:52:13,768][105620] Updated weights for policy 1, policy_version 1716184 (0.0009) [2023-12-27 03:52:13,821][105620] Updated weights for policy 1, policy_version 1716194 (0.0010) [2023-12-27 03:52:13,895][105620] Updated weights for policy 1, policy_version 1716204 (0.0009) [2023-12-27 03:52:14,259][105692] Updated weights for policy 0, policy_version 1712578 (0.0009) [2023-12-27 03:52:14,317][105692] Updated weights for policy 0, policy_version 1712588 (0.0009) [2023-12-27 03:52:14,375][105692] Updated weights for policy 0, policy_version 1712598 (0.0009) [2023-12-27 03:52:14,432][105692] Updated weights for policy 0, policy_version 1712608 (0.0009) [2023-12-27 03:52:14,653][105620] Updated weights for policy 1, policy_version 1716214 (0.0009) [2023-12-27 03:52:14,721][105620] Updated weights for policy 1, policy_version 1716224 (0.0009) [2023-12-27 03:52:14,785][105620] Updated weights for policy 1, policy_version 1716234 (0.0008) [2023-12-27 03:52:15,186][105692] Updated weights for policy 0, policy_version 1712618 (0.0008) [2023-12-27 03:52:15,246][105692] Updated weights for policy 0, policy_version 1712628 (0.0007) [2023-12-27 03:52:15,310][105692] Updated weights for policy 0, policy_version 1712638 (0.0007) [2023-12-27 03:52:15,574][105620] Updated weights for policy 1, policy_version 1716244 (0.0008) [2023-12-27 03:52:15,641][105620] Updated weights for policy 1, policy_version 1716254 (0.0010) [2023-12-27 03:52:15,695][105620] Updated weights for policy 1, policy_version 1716264 (0.0010) [2023-12-27 03:52:15,873][105692] Updated weights for policy 0, policy_version 1712648 (0.0009) [2023-12-27 03:52:15,938][105692] Updated weights for policy 0, policy_version 1712658 (0.0009) [2023-12-27 03:52:15,986][105692] Updated weights for policy 0, policy_version 1712668 (0.0009) [2023-12-27 03:52:16,062][104569] Fps is (10 sec: 20479.4, 60 sec: 19524.2, 300 sec: 19716.3). Total num frames: 877936640. Throughput: 0: 9954.8, 1: 9536.2. Samples: 877902504. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:52:16,063][104569] Avg episode reward: [(0, '8256.162'), (1, '9077.774')] [2023-12-27 03:52:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001712672_438509568.pth... [2023-12-27 03:52:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001716272_439427072.pth... [2023-12-27 03:52:16,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001715120_439132160.pth [2023-12-27 03:52:16,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001711488_438206464.pth [2023-12-27 03:52:16,444][105620] Updated weights for policy 1, policy_version 1716274 (0.0008) [2023-12-27 03:52:16,507][105620] Updated weights for policy 1, policy_version 1716284 (0.0007) [2023-12-27 03:52:16,565][105620] Updated weights for policy 1, policy_version 1716294 (0.0009) [2023-12-27 03:52:16,623][105620] Updated weights for policy 1, policy_version 1716304 (0.0009) [2023-12-27 03:52:16,779][105692] Updated weights for policy 0, policy_version 1712678 (0.0009) [2023-12-27 03:52:16,836][105692] Updated weights for policy 0, policy_version 1712688 (0.0009) [2023-12-27 03:52:16,891][105692] Updated weights for policy 0, policy_version 1712698 (0.0009) [2023-12-27 03:52:17,334][105620] Updated weights for policy 1, policy_version 1716314 (0.0009) [2023-12-27 03:52:17,389][105620] Updated weights for policy 1, policy_version 1716324 (0.0008) [2023-12-27 03:52:17,462][105620] Updated weights for policy 1, policy_version 1716334 (0.0005) [2023-12-27 03:52:17,668][105692] Updated weights for policy 0, policy_version 1712708 (0.0009) [2023-12-27 03:52:17,715][105692] Updated weights for policy 0, policy_version 1712718 (0.0009) [2023-12-27 03:52:17,767][105692] Updated weights for policy 0, policy_version 1712728 (0.0009) [2023-12-27 03:52:18,211][105620] Updated weights for policy 1, policy_version 1716344 (0.0009) [2023-12-27 03:52:18,262][105620] Updated weights for policy 1, policy_version 1716354 (0.0006) [2023-12-27 03:52:18,323][105620] Updated weights for policy 1, policy_version 1716364 (0.0007) [2023-12-27 03:52:18,452][105692] Updated weights for policy 0, policy_version 1712738 (0.0009) [2023-12-27 03:52:18,510][105692] Updated weights for policy 0, policy_version 1712748 (0.0008) [2023-12-27 03:52:18,572][105692] Updated weights for policy 0, policy_version 1712758 (0.0008) [2023-12-27 03:52:18,638][105692] Updated weights for policy 0, policy_version 1712768 (0.0009) [2023-12-27 03:52:19,002][105620] Updated weights for policy 1, policy_version 1716374 (0.0006) [2023-12-27 03:52:19,067][105620] Updated weights for policy 1, policy_version 1716384 (0.0005) [2023-12-27 03:52:19,135][105620] Updated weights for policy 1, policy_version 1716394 (0.0006) [2023-12-27 03:52:19,497][105692] Updated weights for policy 0, policy_version 1712778 (0.0008) [2023-12-27 03:52:19,558][105692] Updated weights for policy 0, policy_version 1712788 (0.0008) [2023-12-27 03:52:19,620][105692] Updated weights for policy 0, policy_version 1712798 (0.0008) [2023-12-27 03:52:19,682][105620] Updated weights for policy 1, policy_version 1716404 (0.0007) [2023-12-27 03:52:19,738][105620] Updated weights for policy 1, policy_version 1716414 (0.0007) [2023-12-27 03:52:19,804][105620] Updated weights for policy 1, policy_version 1716424 (0.0009) [2023-12-27 03:52:20,392][105692] Updated weights for policy 0, policy_version 1712808 (0.0010) [2023-12-27 03:52:20,445][105692] Updated weights for policy 0, policy_version 1712818 (0.0010) [2023-12-27 03:52:20,490][105620] Updated weights for policy 1, policy_version 1716434 (0.0008) [2023-12-27 03:52:20,500][105692] Updated weights for policy 0, policy_version 1712828 (0.0010) [2023-12-27 03:52:20,543][105620] Updated weights for policy 1, policy_version 1716444 (0.0007) [2023-12-27 03:52:20,608][105620] Updated weights for policy 1, policy_version 1716455 (0.0009) [2023-12-27 03:52:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.9, 300 sec: 19688.6). Total num frames: 878026752. Throughput: 0: 9925.3, 1: 9580.8. Samples: 878018088. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:52:21,062][104569] Avg episode reward: [(0, '8622.347'), (1, '8709.264')] [2023-12-27 03:52:21,194][105692] Updated weights for policy 0, policy_version 1712838 (0.0010) [2023-12-27 03:52:21,262][105692] Updated weights for policy 0, policy_version 1712848 (0.0011) [2023-12-27 03:52:21,325][105692] Updated weights for policy 0, policy_version 1712858 (0.0010) [2023-12-27 03:52:21,342][105620] Updated weights for policy 1, policy_version 1716465 (0.0007) [2023-12-27 03:52:21,409][105620] Updated weights for policy 1, policy_version 1716475 (0.0008) [2023-12-27 03:52:21,478][105620] Updated weights for policy 1, policy_version 1716485 (0.0010) [2023-12-27 03:52:21,540][105620] Updated weights for policy 1, policy_version 1716495 (0.0009) [2023-12-27 03:52:22,085][105692] Updated weights for policy 0, policy_version 1712868 (0.0009) [2023-12-27 03:52:22,149][105692] Updated weights for policy 0, policy_version 1712878 (0.0009) [2023-12-27 03:52:22,212][105692] Updated weights for policy 0, policy_version 1712888 (0.0006) [2023-12-27 03:52:22,308][105620] Updated weights for policy 1, policy_version 1716505 (0.0009) [2023-12-27 03:52:22,374][105620] Updated weights for policy 1, policy_version 1716515 (0.0009) [2023-12-27 03:52:22,429][105620] Updated weights for policy 1, policy_version 1716525 (0.0009) [2023-12-27 03:52:22,939][105692] Updated weights for policy 0, policy_version 1712898 (0.0008) [2023-12-27 03:52:22,988][105692] Updated weights for policy 0, policy_version 1712908 (0.0009) [2023-12-27 03:52:23,047][105692] Updated weights for policy 0, policy_version 1712918 (0.0010) [2023-12-27 03:52:23,106][105692] Updated weights for policy 0, policy_version 1712928 (0.0010) [2023-12-27 03:52:23,178][105620] Updated weights for policy 1, policy_version 1716535 (0.0009) [2023-12-27 03:52:23,231][105620] Updated weights for policy 1, policy_version 1716545 (0.0008) [2023-12-27 03:52:23,235][105586] KL-divergence is very high: 103.6597 [2023-12-27 03:52:23,277][105586] KL-divergence is very high: 152.2423 [2023-12-27 03:52:23,284][105620] Updated weights for policy 1, policy_version 1716555 (0.0009) [2023-12-27 03:52:23,838][105692] Updated weights for policy 0, policy_version 1712938 (0.0011) [2023-12-27 03:52:23,890][105692] Updated weights for policy 0, policy_version 1712948 (0.0010) [2023-12-27 03:52:23,928][105586] KL-divergence is very high: 163.2585 [2023-12-27 03:52:23,951][105692] Updated weights for policy 0, policy_version 1712958 (0.0005) [2023-12-27 03:52:23,965][105620] Updated weights for policy 1, policy_version 1716565 (0.0009) [2023-12-27 03:52:23,980][105586] KL-divergence is very high: 147.0130 [2023-12-27 03:52:24,017][105620] Updated weights for policy 1, policy_version 1716575 (0.0008) [2023-12-27 03:52:24,020][105586] KL-divergence is very high: 124.7718 [2023-12-27 03:52:24,062][105586] KL-divergence is very high: 101.9636 [2023-12-27 03:52:24,069][105620] Updated weights for policy 1, policy_version 1716585 (0.0008) [2023-12-27 03:52:24,561][105692] Updated weights for policy 0, policy_version 1712968 (0.0006) [2023-12-27 03:52:24,619][105692] Updated weights for policy 0, policy_version 1712978 (0.0010) [2023-12-27 03:52:24,676][105692] Updated weights for policy 0, policy_version 1712988 (0.0010) [2023-12-27 03:52:24,905][105620] Updated weights for policy 1, policy_version 1716595 (0.0008) [2023-12-27 03:52:24,975][105620] Updated weights for policy 1, policy_version 1716605 (0.0009) [2023-12-27 03:52:25,034][105620] Updated weights for policy 1, policy_version 1716615 (0.0009) [2023-12-27 03:52:25,343][105692] Updated weights for policy 0, policy_version 1712998 (0.0008) [2023-12-27 03:52:25,403][105692] Updated weights for policy 0, policy_version 1713008 (0.0008) [2023-12-27 03:52:25,465][105692] Updated weights for policy 0, policy_version 1713018 (0.0009) [2023-12-27 03:52:25,868][105620] Updated weights for policy 1, policy_version 1716625 (0.0009) [2023-12-27 03:52:25,931][105620] Updated weights for policy 1, policy_version 1716635 (0.0010) [2023-12-27 03:52:25,986][105620] Updated weights for policy 1, policy_version 1716646 (0.0010) [2023-12-27 03:52:26,035][105620] Updated weights for policy 1, policy_version 1716656 (0.0007) [2023-12-27 03:52:26,036][105692] Updated weights for policy 0, policy_version 1713028 (0.0008) [2023-12-27 03:52:26,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19387.8, 300 sec: 19688.6). Total num frames: 878125056. Throughput: 0: 9942.2, 1: 9587.4. Samples: 878131924. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:52:26,062][104569] Avg episode reward: [(0, '8714.661'), (1, '8341.110')] [2023-12-27 03:52:26,082][105692] Updated weights for policy 0, policy_version 1713038 (0.0008) [2023-12-27 03:52:26,140][105692] Updated weights for policy 0, policy_version 1713048 (0.0009) [2023-12-27 03:52:26,709][105692] Updated weights for policy 0, policy_version 1713058 (0.0006) [2023-12-27 03:52:26,755][105692] Updated weights for policy 0, policy_version 1713068 (0.0010) [2023-12-27 03:52:26,804][105692] Updated weights for policy 0, policy_version 1713078 (0.0010) [2023-12-27 03:52:26,858][105692] Updated weights for policy 0, policy_version 1713088 (0.0007) [2023-12-27 03:52:26,911][105620] Updated weights for policy 1, policy_version 1716666 (0.0010) [2023-12-27 03:52:26,965][105620] Updated weights for policy 1, policy_version 1716676 (0.0008) [2023-12-27 03:52:27,024][105620] Updated weights for policy 1, policy_version 1716686 (0.0008) [2023-12-27 03:52:27,491][105692] Updated weights for policy 0, policy_version 1713098 (0.0007) [2023-12-27 03:52:27,539][105692] Updated weights for policy 0, policy_version 1713108 (0.0009) [2023-12-27 03:52:27,590][105692] Updated weights for policy 0, policy_version 1713118 (0.0010) [2023-12-27 03:52:27,861][105620] Updated weights for policy 1, policy_version 1716696 (0.0008) [2023-12-27 03:52:27,930][105620] Updated weights for policy 1, policy_version 1716706 (0.0008) [2023-12-27 03:52:27,992][105620] Updated weights for policy 1, policy_version 1716716 (0.0008) [2023-12-27 03:52:28,341][105692] Updated weights for policy 0, policy_version 1713128 (0.0011) [2023-12-27 03:52:28,397][105692] Updated weights for policy 0, policy_version 1713138 (0.0010) [2023-12-27 03:52:28,453][105692] Updated weights for policy 0, policy_version 1713148 (0.0010) [2023-12-27 03:52:28,751][105620] Updated weights for policy 1, policy_version 1716726 (0.0009) [2023-12-27 03:52:28,814][105620] Updated weights for policy 1, policy_version 1716736 (0.0008) [2023-12-27 03:52:28,879][105620] Updated weights for policy 1, policy_version 1716746 (0.0008) [2023-12-27 03:52:29,194][105692] Updated weights for policy 0, policy_version 1713158 (0.0011) [2023-12-27 03:52:29,256][105692] Updated weights for policy 0, policy_version 1713168 (0.0010) [2023-12-27 03:52:29,312][105692] Updated weights for policy 0, policy_version 1713178 (0.0009) [2023-12-27 03:52:29,604][105620] Updated weights for policy 1, policy_version 1716756 (0.0008) [2023-12-27 03:52:29,660][105620] Updated weights for policy 1, policy_version 1716766 (0.0008) [2023-12-27 03:52:29,716][105620] Updated weights for policy 1, policy_version 1716776 (0.0008) [2023-12-27 03:52:29,947][105692] Updated weights for policy 0, policy_version 1713188 (0.0009) [2023-12-27 03:52:30,005][105692] Updated weights for policy 0, policy_version 1713198 (0.0009) [2023-12-27 03:52:30,060][105692] Updated weights for policy 0, policy_version 1713208 (0.0009) [2023-12-27 03:52:30,470][105620] Updated weights for policy 1, policy_version 1716786 (0.0009) [2023-12-27 03:52:30,528][105620] Updated weights for policy 1, policy_version 1716796 (0.0006) [2023-12-27 03:52:30,586][105620] Updated weights for policy 1, policy_version 1716806 (0.0005) [2023-12-27 03:52:30,643][105620] Updated weights for policy 1, policy_version 1716816 (0.0005) [2023-12-27 03:52:30,873][105692] Updated weights for policy 0, policy_version 1713218 (0.0009) [2023-12-27 03:52:30,920][105692] Updated weights for policy 0, policy_version 1713228 (0.0008) [2023-12-27 03:52:30,965][105692] Updated weights for policy 0, policy_version 1713238 (0.0008) [2023-12-27 03:52:31,016][105692] Updated weights for policy 0, policy_version 1713248 (0.0008) [2023-12-27 03:52:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 878223360. Throughput: 0: 10025.2, 1: 9566.8. Samples: 878190432. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:52:31,062][104569] Avg episode reward: [(0, '8716.344'), (1, '8710.740')] [2023-12-27 03:52:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001713248_438657024.pth... [2023-12-27 03:52:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001716816_439566336.pth... [2023-12-27 03:52:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001712064_438353920.pth [2023-12-27 03:52:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001715696_439279616.pth [2023-12-27 03:52:31,286][105620] Updated weights for policy 1, policy_version 1716826 (0.0008) [2023-12-27 03:52:31,351][105620] Updated weights for policy 1, policy_version 1716836 (0.0008) [2023-12-27 03:52:31,414][105620] Updated weights for policy 1, policy_version 1716846 (0.0010) [2023-12-27 03:52:31,814][105692] Updated weights for policy 0, policy_version 1713258 (0.0009) [2023-12-27 03:52:31,868][105692] Updated weights for policy 0, policy_version 1713268 (0.0009) [2023-12-27 03:52:31,920][105692] Updated weights for policy 0, policy_version 1713278 (0.0009) [2023-12-27 03:52:32,036][105620] Updated weights for policy 1, policy_version 1716856 (0.0005) [2023-12-27 03:52:32,087][105620] Updated weights for policy 1, policy_version 1716866 (0.0005) [2023-12-27 03:52:32,141][105620] Updated weights for policy 1, policy_version 1716876 (0.0006) [2023-12-27 03:52:32,690][105692] Updated weights for policy 0, policy_version 1713288 (0.0006) [2023-12-27 03:52:32,744][105620] Updated weights for policy 1, policy_version 1716886 (0.0007) [2023-12-27 03:52:32,751][105692] Updated weights for policy 0, policy_version 1713298 (0.0008) [2023-12-27 03:52:32,808][105692] Updated weights for policy 0, policy_version 1713308 (0.0007) [2023-12-27 03:52:32,810][105620] Updated weights for policy 1, policy_version 1716896 (0.0006) [2023-12-27 03:52:32,879][105620] Updated weights for policy 1, policy_version 1716906 (0.0005) [2023-12-27 03:52:33,414][105620] Updated weights for policy 1, policy_version 1716916 (0.0007) [2023-12-27 03:52:33,461][105620] Updated weights for policy 1, policy_version 1716926 (0.0008) [2023-12-27 03:52:33,488][105692] Updated weights for policy 0, policy_version 1713318 (0.0007) [2023-12-27 03:52:33,507][105620] Updated weights for policy 1, policy_version 1716936 (0.0007) [2023-12-27 03:52:33,537][105692] Updated weights for policy 0, policy_version 1713328 (0.0007) [2023-12-27 03:52:33,582][105692] Updated weights for policy 0, policy_version 1713338 (0.0008) [2023-12-27 03:52:34,196][105620] Updated weights for policy 1, policy_version 1716946 (0.0006) [2023-12-27 03:52:34,265][105620] Updated weights for policy 1, policy_version 1716956 (0.0006) [2023-12-27 03:52:34,326][105620] Updated weights for policy 1, policy_version 1716966 (0.0006) [2023-12-27 03:52:34,386][105620] Updated weights for policy 1, policy_version 1716976 (0.0007) [2023-12-27 03:52:34,405][105692] Updated weights for policy 0, policy_version 1713348 (0.0009) [2023-12-27 03:52:34,473][105692] Updated weights for policy 0, policy_version 1713358 (0.0008) [2023-12-27 03:52:34,530][105692] Updated weights for policy 0, policy_version 1713368 (0.0010) [2023-12-27 03:52:35,041][105620] Updated weights for policy 1, policy_version 1716986 (0.0009) [2023-12-27 03:52:35,091][105620] Updated weights for policy 1, policy_version 1716996 (0.0008) [2023-12-27 03:52:35,142][105620] Updated weights for policy 1, policy_version 1717006 (0.0009) [2023-12-27 03:52:35,279][105692] Updated weights for policy 0, policy_version 1713378 (0.0009) [2023-12-27 03:52:35,336][105692] Updated weights for policy 0, policy_version 1713388 (0.0008) [2023-12-27 03:52:35,390][105692] Updated weights for policy 0, policy_version 1713398 (0.0009) [2023-12-27 03:52:35,445][105692] Updated weights for policy 0, policy_version 1713408 (0.0010) [2023-12-27 03:52:35,914][105620] Updated weights for policy 1, policy_version 1717016 (0.0006) [2023-12-27 03:52:35,979][105620] Updated weights for policy 1, policy_version 1717026 (0.0005) [2023-12-27 03:52:36,047][105620] Updated weights for policy 1, policy_version 1717036 (0.0005) [2023-12-27 03:52:36,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 878313472. Throughput: 0: 9996.0, 1: 9653.9. Samples: 878310580. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:52:36,063][104569] Avg episode reward: [(0, '8899.421'), (1, '9170.395')] [2023-12-27 03:52:36,260][105692] Updated weights for policy 0, policy_version 1713418 (0.0006) [2023-12-27 03:52:36,323][105692] Updated weights for policy 0, policy_version 1713428 (0.0009) [2023-12-27 03:52:36,385][105692] Updated weights for policy 0, policy_version 1713438 (0.0007) [2023-12-27 03:52:36,647][105620] Updated weights for policy 1, policy_version 1717046 (0.0008) [2023-12-27 03:52:36,713][105620] Updated weights for policy 1, policy_version 1717056 (0.0009) [2023-12-27 03:52:36,773][105620] Updated weights for policy 1, policy_version 1717066 (0.0008) [2023-12-27 03:52:37,052][105692] Updated weights for policy 0, policy_version 1713448 (0.0005) [2023-12-27 03:52:37,105][105692] Updated weights for policy 0, policy_version 1713458 (0.0005) [2023-12-27 03:52:37,151][105692] Updated weights for policy 0, policy_version 1713468 (0.0005) [2023-12-27 03:52:37,533][105620] Updated weights for policy 1, policy_version 1717076 (0.0008) [2023-12-27 03:52:37,595][105620] Updated weights for policy 1, policy_version 1717086 (0.0008) [2023-12-27 03:52:37,654][105620] Updated weights for policy 1, policy_version 1717096 (0.0009) [2023-12-27 03:52:37,824][105692] Updated weights for policy 0, policy_version 1713478 (0.0007) [2023-12-27 03:52:37,886][105692] Updated weights for policy 0, policy_version 1713488 (0.0009) [2023-12-27 03:52:37,949][105692] Updated weights for policy 0, policy_version 1713498 (0.0009) [2023-12-27 03:52:38,405][105620] Updated weights for policy 1, policy_version 1717106 (0.0008) [2023-12-27 03:52:38,457][105620] Updated weights for policy 1, policy_version 1717116 (0.0008) [2023-12-27 03:52:38,512][105620] Updated weights for policy 1, policy_version 1717126 (0.0009) [2023-12-27 03:52:38,569][105620] Updated weights for policy 1, policy_version 1717136 (0.0008) [2023-12-27 03:52:38,698][105692] Updated weights for policy 0, policy_version 1713508 (0.0008) [2023-12-27 03:52:38,751][105692] Updated weights for policy 0, policy_version 1713518 (0.0005) [2023-12-27 03:52:38,803][105692] Updated weights for policy 0, policy_version 1713528 (0.0007) [2023-12-27 03:52:39,286][105620] Updated weights for policy 1, policy_version 1717146 (0.0011) [2023-12-27 03:52:39,356][105620] Updated weights for policy 1, policy_version 1717156 (0.0011) [2023-12-27 03:52:39,421][105620] Updated weights for policy 1, policy_version 1717166 (0.0008) [2023-12-27 03:52:39,553][105692] Updated weights for policy 0, policy_version 1713538 (0.0008) [2023-12-27 03:52:39,616][105692] Updated weights for policy 0, policy_version 1713548 (0.0009) [2023-12-27 03:52:39,677][105692] Updated weights for policy 0, policy_version 1713558 (0.0008) [2023-12-27 03:52:39,741][105692] Updated weights for policy 0, policy_version 1713568 (0.0008) [2023-12-27 03:52:40,168][105620] Updated weights for policy 1, policy_version 1717176 (0.0006) [2023-12-27 03:52:40,220][105620] Updated weights for policy 1, policy_version 1717186 (0.0009) [2023-12-27 03:52:40,274][105620] Updated weights for policy 1, policy_version 1717196 (0.0008) [2023-12-27 03:52:40,495][105692] Updated weights for policy 0, policy_version 1713578 (0.0009) [2023-12-27 03:52:40,553][105692] Updated weights for policy 0, policy_version 1713588 (0.0009) [2023-12-27 03:52:40,605][105692] Updated weights for policy 0, policy_version 1713598 (0.0010) [2023-12-27 03:52:41,047][105620] Updated weights for policy 1, policy_version 1717206 (0.0009) [2023-12-27 03:52:41,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 878411776. Throughput: 0: 9939.3, 1: 9675.7. Samples: 878424332. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:52:41,063][104569] Avg episode reward: [(0, '8899.192'), (1, '9078.850')] [2023-12-27 03:52:41,107][105620] Updated weights for policy 1, policy_version 1717216 (0.0008) [2023-12-27 03:52:41,168][105620] Updated weights for policy 1, policy_version 1717226 (0.0008) [2023-12-27 03:52:41,405][105692] Updated weights for policy 0, policy_version 1713608 (0.0009) [2023-12-27 03:52:41,467][105692] Updated weights for policy 0, policy_version 1713618 (0.0008) [2023-12-27 03:52:41,529][105692] Updated weights for policy 0, policy_version 1713628 (0.0008) [2023-12-27 03:52:41,926][105620] Updated weights for policy 1, policy_version 1717236 (0.0009) [2023-12-27 03:52:41,987][105620] Updated weights for policy 1, policy_version 1717246 (0.0010) [2023-12-27 03:52:42,047][105620] Updated weights for policy 1, policy_version 1717256 (0.0010) [2023-12-27 03:52:42,261][105692] Updated weights for policy 0, policy_version 1713638 (0.0007) [2023-12-27 03:52:42,323][105692] Updated weights for policy 0, policy_version 1713648 (0.0008) [2023-12-27 03:52:42,386][105692] Updated weights for policy 0, policy_version 1713658 (0.0008) [2023-12-27 03:52:42,787][105620] Updated weights for policy 1, policy_version 1717266 (0.0011) [2023-12-27 03:52:42,852][105620] Updated weights for policy 1, policy_version 1717276 (0.0010) [2023-12-27 03:52:42,921][105620] Updated weights for policy 1, policy_version 1717286 (0.0005) [2023-12-27 03:52:42,985][105620] Updated weights for policy 1, policy_version 1717296 (0.0005) [2023-12-27 03:52:43,122][105692] Updated weights for policy 0, policy_version 1713668 (0.0007) [2023-12-27 03:52:43,180][105692] Updated weights for policy 0, policy_version 1713678 (0.0008) [2023-12-27 03:52:43,244][105692] Updated weights for policy 0, policy_version 1713688 (0.0008) [2023-12-27 03:52:43,622][105620] Updated weights for policy 1, policy_version 1717306 (0.0009) [2023-12-27 03:52:43,667][105620] Updated weights for policy 1, policy_version 1717316 (0.0005) [2023-12-27 03:52:43,728][105620] Updated weights for policy 1, policy_version 1717326 (0.0010) [2023-12-27 03:52:44,002][105692] Updated weights for policy 0, policy_version 1713698 (0.0008) [2023-12-27 03:52:44,057][105692] Updated weights for policy 0, policy_version 1713708 (0.0008) [2023-12-27 03:52:44,110][105692] Updated weights for policy 0, policy_version 1713718 (0.0008) [2023-12-27 03:52:44,168][105692] Updated weights for policy 0, policy_version 1713728 (0.0010) [2023-12-27 03:52:44,386][105620] Updated weights for policy 1, policy_version 1717336 (0.0007) [2023-12-27 03:52:44,445][105620] Updated weights for policy 1, policy_version 1717346 (0.0011) [2023-12-27 03:52:44,504][105620] Updated weights for policy 1, policy_version 1717356 (0.0010) [2023-12-27 03:52:44,929][105692] Updated weights for policy 0, policy_version 1713738 (0.0009) [2023-12-27 03:52:44,999][105692] Updated weights for policy 0, policy_version 1713748 (0.0011) [2023-12-27 03:52:45,059][105692] Updated weights for policy 0, policy_version 1713758 (0.0009) [2023-12-27 03:52:45,163][105620] Updated weights for policy 1, policy_version 1717366 (0.0007) [2023-12-27 03:52:45,222][105620] Updated weights for policy 1, policy_version 1717376 (0.0008) [2023-12-27 03:52:45,272][105620] Updated weights for policy 1, policy_version 1717386 (0.0011) [2023-12-27 03:52:45,798][105692] Updated weights for policy 0, policy_version 1713768 (0.0010) [2023-12-27 03:52:45,842][105692] Updated weights for policy 0, policy_version 1713778 (0.0010) [2023-12-27 03:52:45,890][105692] Updated weights for policy 0, policy_version 1713788 (0.0010) [2023-12-27 03:52:46,006][105620] Updated weights for policy 1, policy_version 1717396 (0.0011) [2023-12-27 03:52:46,051][105620] Updated weights for policy 1, policy_version 1717406 (0.0010) [2023-12-27 03:52:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.4, 300 sec: 19633.0). Total num frames: 878510080. Throughput: 0: 9828.0, 1: 9684.5. Samples: 878480724. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:52:46,062][104569] Avg episode reward: [(0, '8990.538'), (1, '9078.916')] [2023-12-27 03:52:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001713792_438796288.pth... [2023-12-27 03:52:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001712672_438509568.pth [2023-12-27 03:52:46,115][105620] Updated weights for policy 1, policy_version 1717416 (0.0010) [2023-12-27 03:52:46,163][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001717424_439721984.pth... [2023-12-27 03:52:46,168][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001716272_439427072.pth [2023-12-27 03:52:46,625][105692] Updated weights for policy 0, policy_version 1713798 (0.0007) [2023-12-27 03:52:46,673][105692] Updated weights for policy 0, policy_version 1713808 (0.0005) [2023-12-27 03:52:46,731][105692] Updated weights for policy 0, policy_version 1713818 (0.0006) [2023-12-27 03:52:46,802][105620] Updated weights for policy 1, policy_version 1717426 (0.0010) [2023-12-27 03:52:46,867][105620] Updated weights for policy 1, policy_version 1717436 (0.0011) [2023-12-27 03:52:46,923][105620] Updated weights for policy 1, policy_version 1717446 (0.0010) [2023-12-27 03:52:46,986][105620] Updated weights for policy 1, policy_version 1717456 (0.0011) [2023-12-27 03:52:47,351][105692] Updated weights for policy 0, policy_version 1713828 (0.0008) [2023-12-27 03:52:47,409][105692] Updated weights for policy 0, policy_version 1713838 (0.0005) [2023-12-27 03:52:47,457][105692] Updated weights for policy 0, policy_version 1713848 (0.0007) [2023-12-27 03:52:47,656][105620] Updated weights for policy 1, policy_version 1717466 (0.0010) [2023-12-27 03:52:47,700][105620] Updated weights for policy 1, policy_version 1717476 (0.0007) [2023-12-27 03:52:47,747][105620] Updated weights for policy 1, policy_version 1717486 (0.0005) [2023-12-27 03:52:48,029][105692] Updated weights for policy 0, policy_version 1713858 (0.0009) [2023-12-27 03:52:48,088][105692] Updated weights for policy 0, policy_version 1713868 (0.0006) [2023-12-27 03:52:48,143][105692] Updated weights for policy 0, policy_version 1713878 (0.0008) [2023-12-27 03:52:48,193][105692] Updated weights for policy 0, policy_version 1713888 (0.0011) [2023-12-27 03:52:48,545][105620] Updated weights for policy 1, policy_version 1717496 (0.0007) [2023-12-27 03:52:48,593][105620] Updated weights for policy 1, policy_version 1717506 (0.0005) [2023-12-27 03:52:48,647][105620] Updated weights for policy 1, policy_version 1717516 (0.0007) [2023-12-27 03:52:48,957][105692] Updated weights for policy 0, policy_version 1713898 (0.0010) [2023-12-27 03:52:49,025][105692] Updated weights for policy 0, policy_version 1713908 (0.0010) [2023-12-27 03:52:49,085][105692] Updated weights for policy 0, policy_version 1713918 (0.0009) [2023-12-27 03:52:49,308][105620] Updated weights for policy 1, policy_version 1717526 (0.0010) [2023-12-27 03:52:49,367][105620] Updated weights for policy 1, policy_version 1717536 (0.0008) [2023-12-27 03:52:49,426][105620] Updated weights for policy 1, policy_version 1717546 (0.0008) [2023-12-27 03:52:49,847][105692] Updated weights for policy 0, policy_version 1713928 (0.0009) [2023-12-27 03:52:49,910][105692] Updated weights for policy 0, policy_version 1713938 (0.0010) [2023-12-27 03:52:49,976][105692] Updated weights for policy 0, policy_version 1713948 (0.0008) [2023-12-27 03:52:50,181][105620] Updated weights for policy 1, policy_version 1717556 (0.0008) [2023-12-27 03:52:50,237][105620] Updated weights for policy 1, policy_version 1717566 (0.0006) [2023-12-27 03:52:50,296][105620] Updated weights for policy 1, policy_version 1717576 (0.0008) [2023-12-27 03:52:50,614][105692] Updated weights for policy 0, policy_version 1713958 (0.0009) [2023-12-27 03:52:50,680][105692] Updated weights for policy 0, policy_version 1713968 (0.0011) [2023-12-27 03:52:50,746][105692] Updated weights for policy 0, policy_version 1713978 (0.0011) [2023-12-27 03:52:50,980][105620] Updated weights for policy 1, policy_version 1717586 (0.0008) [2023-12-27 03:52:51,034][105620] Updated weights for policy 1, policy_version 1717596 (0.0008) [2023-12-27 03:52:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 878608384. Throughput: 0: 9798.3, 1: 9641.8. Samples: 878599228. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:52:51,062][104569] Avg episode reward: [(0, '8988.976'), (1, '8836.342')] [2023-12-27 03:52:51,089][105620] Updated weights for policy 1, policy_version 1717606 (0.0008) [2023-12-27 03:52:51,152][105620] Updated weights for policy 1, policy_version 1717616 (0.0008) [2023-12-27 03:52:51,490][105692] Updated weights for policy 0, policy_version 1713988 (0.0010) [2023-12-27 03:52:51,542][105692] Updated weights for policy 0, policy_version 1713998 (0.0008) [2023-12-27 03:52:51,598][105692] Updated weights for policy 0, policy_version 1714008 (0.0008) [2023-12-27 03:52:51,940][105620] Updated weights for policy 1, policy_version 1717626 (0.0008) [2023-12-27 03:52:51,987][105620] Updated weights for policy 1, policy_version 1717636 (0.0009) [2023-12-27 03:52:52,045][105620] Updated weights for policy 1, policy_version 1717646 (0.0008) [2023-12-27 03:52:52,369][105692] Updated weights for policy 0, policy_version 1714018 (0.0008) [2023-12-27 03:52:52,435][105692] Updated weights for policy 0, policy_version 1714028 (0.0009) [2023-12-27 03:52:52,499][105692] Updated weights for policy 0, policy_version 1714038 (0.0009) [2023-12-27 03:52:52,561][105692] Updated weights for policy 0, policy_version 1714048 (0.0009) [2023-12-27 03:52:52,835][105620] Updated weights for policy 1, policy_version 1717656 (0.0009) [2023-12-27 03:52:52,894][105620] Updated weights for policy 1, policy_version 1717666 (0.0006) [2023-12-27 03:52:52,945][105620] Updated weights for policy 1, policy_version 1717676 (0.0005) [2023-12-27 03:52:53,247][105692] Updated weights for policy 0, policy_version 1714058 (0.0005) [2023-12-27 03:52:53,305][105692] Updated weights for policy 0, policy_version 1714068 (0.0005) [2023-12-27 03:52:53,365][105692] Updated weights for policy 0, policy_version 1714078 (0.0005) [2023-12-27 03:52:53,756][105620] Updated weights for policy 1, policy_version 1717686 (0.0008) [2023-12-27 03:52:53,810][105620] Updated weights for policy 1, policy_version 1717696 (0.0009) [2023-12-27 03:52:53,873][105620] Updated weights for policy 1, policy_version 1717706 (0.0009) [2023-12-27 03:52:53,929][105692] Updated weights for policy 0, policy_version 1714088 (0.0005) [2023-12-27 03:52:53,982][105692] Updated weights for policy 0, policy_version 1714098 (0.0005) [2023-12-27 03:52:54,038][105692] Updated weights for policy 0, policy_version 1714108 (0.0009) [2023-12-27 03:52:54,505][105620] Updated weights for policy 1, policy_version 1717716 (0.0006) [2023-12-27 03:52:54,566][105620] Updated weights for policy 1, policy_version 1717726 (0.0009) [2023-12-27 03:52:54,615][105620] Updated weights for policy 1, policy_version 1717736 (0.0008) [2023-12-27 03:52:54,760][105692] Updated weights for policy 0, policy_version 1714119 (0.0007) [2023-12-27 03:52:54,816][105692] Updated weights for policy 0, policy_version 1714129 (0.0006) [2023-12-27 03:52:54,879][105692] Updated weights for policy 0, policy_version 1714139 (0.0005) [2023-12-27 03:52:55,452][105620] Updated weights for policy 1, policy_version 1717746 (0.0009) [2023-12-27 03:52:55,484][105692] Updated weights for policy 0, policy_version 1714149 (0.0007) [2023-12-27 03:52:55,499][105620] Updated weights for policy 1, policy_version 1717756 (0.0007) [2023-12-27 03:52:55,529][105692] Updated weights for policy 0, policy_version 1714159 (0.0007) [2023-12-27 03:52:55,552][105620] Updated weights for policy 1, policy_version 1717766 (0.0006) [2023-12-27 03:52:55,579][105692] Updated weights for policy 0, policy_version 1714169 (0.0008) [2023-12-27 03:52:55,607][105620] Updated weights for policy 1, policy_version 1717776 (0.0008) [2023-12-27 03:52:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 878706688. Throughput: 0: 9728.1, 1: 9574.3. Samples: 878715708. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:52:56,062][104569] Avg episode reward: [(0, '8529.479'), (1, '8836.249')] [2023-12-27 03:52:56,201][105692] Updated weights for policy 0, policy_version 1714179 (0.0009) [2023-12-27 03:52:56,253][105692] Updated weights for policy 0, policy_version 1714189 (0.0010) [2023-12-27 03:52:56,305][105692] Updated weights for policy 0, policy_version 1714199 (0.0008) [2023-12-27 03:52:56,447][105620] Updated weights for policy 1, policy_version 1717786 (0.0010) [2023-12-27 03:52:56,496][105620] Updated weights for policy 1, policy_version 1717796 (0.0009) [2023-12-27 03:52:56,543][105620] Updated weights for policy 1, policy_version 1717806 (0.0009) [2023-12-27 03:52:57,023][105692] Updated weights for policy 0, policy_version 1714209 (0.0005) [2023-12-27 03:52:57,091][105692] Updated weights for policy 0, policy_version 1714219 (0.0005) [2023-12-27 03:52:57,146][105692] Updated weights for policy 0, policy_version 1714229 (0.0005) [2023-12-27 03:52:57,192][105620] Updated weights for policy 1, policy_version 1717816 (0.0007) [2023-12-27 03:52:57,211][105692] Updated weights for policy 0, policy_version 1714239 (0.0005) [2023-12-27 03:52:57,247][105620] Updated weights for policy 1, policy_version 1717827 (0.0009) [2023-12-27 03:52:57,295][105620] Updated weights for policy 1, policy_version 1717837 (0.0009) [2023-12-27 03:52:57,922][105620] Updated weights for policy 1, policy_version 1717847 (0.0007) [2023-12-27 03:52:57,952][105692] Updated weights for policy 0, policy_version 1714249 (0.0007) [2023-12-27 03:52:57,967][105620] Updated weights for policy 1, policy_version 1717857 (0.0006) [2023-12-27 03:52:58,006][105692] Updated weights for policy 0, policy_version 1714259 (0.0008) [2023-12-27 03:52:58,015][105620] Updated weights for policy 1, policy_version 1717867 (0.0009) [2023-12-27 03:52:58,060][105692] Updated weights for policy 0, policy_version 1714269 (0.0007) [2023-12-27 03:52:58,790][105620] Updated weights for policy 1, policy_version 1717877 (0.0008) [2023-12-27 03:52:58,861][105620] Updated weights for policy 1, policy_version 1717887 (0.0007) [2023-12-27 03:52:58,907][105692] Updated weights for policy 0, policy_version 1714279 (0.0008) [2023-12-27 03:52:58,929][105620] Updated weights for policy 1, policy_version 1717897 (0.0008) [2023-12-27 03:52:58,981][105692] Updated weights for policy 0, policy_version 1714289 (0.0009) [2023-12-27 03:52:59,050][105692] Updated weights for policy 0, policy_version 1714299 (0.0005) [2023-12-27 03:52:59,769][105620] Updated weights for policy 1, policy_version 1717907 (0.0007) [2023-12-27 03:52:59,779][105692] Updated weights for policy 0, policy_version 1714309 (0.0008) [2023-12-27 03:52:59,829][105620] Updated weights for policy 1, policy_version 1717917 (0.0007) [2023-12-27 03:52:59,842][105692] Updated weights for policy 0, policy_version 1714319 (0.0009) [2023-12-27 03:52:59,892][105620] Updated weights for policy 1, policy_version 1717927 (0.0008) [2023-12-27 03:52:59,905][105692] Updated weights for policy 0, policy_version 1714329 (0.0008) [2023-12-27 03:53:00,567][105692] Updated weights for policy 0, policy_version 1714339 (0.0009) [2023-12-27 03:53:00,598][105620] Updated weights for policy 1, policy_version 1717937 (0.0008) [2023-12-27 03:53:00,625][105692] Updated weights for policy 0, policy_version 1714349 (0.0010) [2023-12-27 03:53:00,650][105620] Updated weights for policy 1, policy_version 1717947 (0.0008) [2023-12-27 03:53:00,686][105692] Updated weights for policy 0, policy_version 1714359 (0.0010) [2023-12-27 03:53:00,693][105620] Updated weights for policy 1, policy_version 1717957 (0.0007) [2023-12-27 03:53:00,749][105620] Updated weights for policy 1, policy_version 1717967 (0.0009) [2023-12-27 03:53:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 878804992. Throughput: 0: 9757.3, 1: 9630.4. Samples: 878774944. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:53:01,062][104569] Avg episode reward: [(0, '8619.735'), (1, '8988.669')] [2023-12-27 03:53:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001714368_438943744.pth... [2023-12-27 03:53:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001717968_439861248.pth... [2023-12-27 03:53:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001713248_438657024.pth [2023-12-27 03:53:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001716816_439566336.pth [2023-12-27 03:53:01,354][105692] Updated weights for policy 0, policy_version 1714369 (0.0010) [2023-12-27 03:53:01,420][105692] Updated weights for policy 0, policy_version 1714379 (0.0010) [2023-12-27 03:53:01,476][105692] Updated weights for policy 0, policy_version 1714389 (0.0010) [2023-12-27 03:53:01,514][105620] Updated weights for policy 1, policy_version 1717977 (0.0007) [2023-12-27 03:53:01,525][105692] Updated weights for policy 0, policy_version 1714399 (0.0010) [2023-12-27 03:53:01,569][105620] Updated weights for policy 1, policy_version 1717987 (0.0007) [2023-12-27 03:53:01,638][105620] Updated weights for policy 1, policy_version 1717997 (0.0007) [2023-12-27 03:53:02,336][105692] Updated weights for policy 0, policy_version 1714409 (0.0010) [2023-12-27 03:53:02,343][105620] Updated weights for policy 1, policy_version 1718007 (0.0007) [2023-12-27 03:53:02,398][105692] Updated weights for policy 0, policy_version 1714419 (0.0011) [2023-12-27 03:53:02,405][105620] Updated weights for policy 1, policy_version 1718017 (0.0008) [2023-12-27 03:53:02,450][105692] Updated weights for policy 0, policy_version 1714429 (0.0010) [2023-12-27 03:53:02,452][105620] Updated weights for policy 1, policy_version 1718027 (0.0006) [2023-12-27 03:53:03,026][105620] Updated weights for policy 1, policy_version 1718037 (0.0005) [2023-12-27 03:53:03,072][105620] Updated weights for policy 1, policy_version 1718047 (0.0005) [2023-12-27 03:53:03,128][105620] Updated weights for policy 1, policy_version 1718057 (0.0005) [2023-12-27 03:53:03,209][105692] Updated weights for policy 0, policy_version 1714439 (0.0010) [2023-12-27 03:53:03,266][105692] Updated weights for policy 0, policy_version 1714449 (0.0010) [2023-12-27 03:53:03,324][105692] Updated weights for policy 0, policy_version 1714459 (0.0010) [2023-12-27 03:53:03,649][105620] Updated weights for policy 1, policy_version 1718067 (0.0005) [2023-12-27 03:53:03,710][105620] Updated weights for policy 1, policy_version 1718077 (0.0005) [2023-12-27 03:53:03,769][105620] Updated weights for policy 1, policy_version 1718087 (0.0005) [2023-12-27 03:53:04,007][105692] Updated weights for policy 0, policy_version 1714469 (0.0008) [2023-12-27 03:53:04,067][105692] Updated weights for policy 0, policy_version 1714479 (0.0006) [2023-12-27 03:53:04,127][105692] Updated weights for policy 0, policy_version 1714489 (0.0006) [2023-12-27 03:53:04,329][105620] Updated weights for policy 1, policy_version 1718097 (0.0006) [2023-12-27 03:53:04,394][105620] Updated weights for policy 1, policy_version 1718107 (0.0008) [2023-12-27 03:53:04,452][105620] Updated weights for policy 1, policy_version 1718117 (0.0008) [2023-12-27 03:53:04,511][105620] Updated weights for policy 1, policy_version 1718127 (0.0008) [2023-12-27 03:53:04,827][105692] Updated weights for policy 0, policy_version 1714499 (0.0006) [2023-12-27 03:53:04,900][105692] Updated weights for policy 0, policy_version 1714509 (0.0005) [2023-12-27 03:53:04,963][105692] Updated weights for policy 0, policy_version 1714519 (0.0006) [2023-12-27 03:53:05,131][105620] Updated weights for policy 1, policy_version 1718137 (0.0006) [2023-12-27 03:53:05,190][105620] Updated weights for policy 1, policy_version 1718147 (0.0005) [2023-12-27 03:53:05,250][105620] Updated weights for policy 1, policy_version 1718157 (0.0009) [2023-12-27 03:53:05,589][105692] Updated weights for policy 0, policy_version 1714529 (0.0006) [2023-12-27 03:53:05,651][105692] Updated weights for policy 0, policy_version 1714540 (0.0010) [2023-12-27 03:53:05,706][105692] Updated weights for policy 0, policy_version 1714552 (0.0011) [2023-12-27 03:53:05,853][105620] Updated weights for policy 1, policy_version 1718167 (0.0009) [2023-12-27 03:53:05,907][105620] Updated weights for policy 1, policy_version 1718177 (0.0009) [2023-12-27 03:53:05,958][105620] Updated weights for policy 1, policy_version 1718187 (0.0009) [2023-12-27 03:53:06,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 878911488. Throughput: 0: 9757.5, 1: 9734.0. Samples: 878895208. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-12-27 03:53:06,062][104569] Avg episode reward: [(0, '8438.086'), (1, '9081.421')] [2023-12-27 03:53:06,540][105692] Updated weights for policy 0, policy_version 1714562 (0.0009) [2023-12-27 03:53:06,602][105692] Updated weights for policy 0, policy_version 1714572 (0.0010) [2023-12-27 03:53:06,658][105692] Updated weights for policy 0, policy_version 1714582 (0.0007) [2023-12-27 03:53:06,681][105620] Updated weights for policy 1, policy_version 1718197 (0.0008) [2023-12-27 03:53:06,707][105692] Updated weights for policy 0, policy_version 1714592 (0.0006) [2023-12-27 03:53:06,738][105620] Updated weights for policy 1, policy_version 1718207 (0.0009) [2023-12-27 03:53:06,791][105620] Updated weights for policy 1, policy_version 1718217 (0.0010) [2023-12-27 03:53:07,372][105692] Updated weights for policy 0, policy_version 1714602 (0.0009) [2023-12-27 03:53:07,427][105692] Updated weights for policy 0, policy_version 1714612 (0.0009) [2023-12-27 03:53:07,478][105692] Updated weights for policy 0, policy_version 1714622 (0.0009) [2023-12-27 03:53:07,598][105620] Updated weights for policy 1, policy_version 1718227 (0.0009) [2023-12-27 03:53:07,653][105620] Updated weights for policy 1, policy_version 1718237 (0.0009) [2023-12-27 03:53:07,706][105586] KL-divergence is very high: 137.9713 [2023-12-27 03:53:07,708][105620] Updated weights for policy 1, policy_version 1718247 (0.0009) [2023-12-27 03:53:07,750][105586] KL-divergence is very high: 159.5218 [2023-12-27 03:53:08,178][105692] Updated weights for policy 0, policy_version 1714632 (0.0006) [2023-12-27 03:53:08,224][105692] Updated weights for policy 0, policy_version 1714642 (0.0005) [2023-12-27 03:53:08,268][105692] Updated weights for policy 0, policy_version 1714652 (0.0005) [2023-12-27 03:53:08,536][105620] Updated weights for policy 1, policy_version 1718257 (0.0009) [2023-12-27 03:53:08,595][105620] Updated weights for policy 1, policy_version 1718267 (0.0006) [2023-12-27 03:53:08,641][105620] Updated weights for policy 1, policy_version 1718277 (0.0006) [2023-12-27 03:53:08,701][105620] Updated weights for policy 1, policy_version 1718287 (0.0007) [2023-12-27 03:53:08,992][105692] Updated weights for policy 0, policy_version 1714662 (0.0008) [2023-12-27 03:53:09,048][105692] Updated weights for policy 0, policy_version 1714672 (0.0005) [2023-12-27 03:53:09,102][105692] Updated weights for policy 0, policy_version 1714682 (0.0005) [2023-12-27 03:53:09,471][105620] Updated weights for policy 1, policy_version 1718297 (0.0008) [2023-12-27 03:53:09,540][105620] Updated weights for policy 1, policy_version 1718307 (0.0008) [2023-12-27 03:53:09,606][105620] Updated weights for policy 1, policy_version 1718317 (0.0009) [2023-12-27 03:53:09,765][105692] Updated weights for policy 0, policy_version 1714692 (0.0006) [2023-12-27 03:53:09,821][105692] Updated weights for policy 0, policy_version 1714702 (0.0009) [2023-12-27 03:53:09,889][105692] Updated weights for policy 0, policy_version 1714712 (0.0008) [2023-12-27 03:53:10,396][105620] Updated weights for policy 1, policy_version 1718327 (0.0009) [2023-12-27 03:53:10,466][105620] Updated weights for policy 1, policy_version 1718337 (0.0009) [2023-12-27 03:53:10,527][105620] Updated weights for policy 1, policy_version 1718347 (0.0009) [2023-12-27 03:53:10,545][105692] Updated weights for policy 0, policy_version 1714722 (0.0008) [2023-12-27 03:53:10,610][105692] Updated weights for policy 0, policy_version 1714732 (0.0005) [2023-12-27 03:53:10,672][105692] Updated weights for policy 0, policy_version 1714742 (0.0005) [2023-12-27 03:53:10,737][105692] Updated weights for policy 0, policy_version 1714752 (0.0006) [2023-12-27 03:53:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 879001600. Throughput: 0: 9788.8, 1: 9739.7. Samples: 879010708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:53:11,062][104569] Avg episode reward: [(0, '8345.416'), (1, '8995.601')] [2023-12-27 03:53:11,343][105692] Updated weights for policy 0, policy_version 1714762 (0.0007) [2023-12-27 03:53:11,392][105620] Updated weights for policy 1, policy_version 1718357 (0.0009) [2023-12-27 03:53:11,408][105692] Updated weights for policy 0, policy_version 1714772 (0.0007) [2023-12-27 03:53:11,464][105620] Updated weights for policy 1, policy_version 1718367 (0.0008) [2023-12-27 03:53:11,473][105692] Updated weights for policy 0, policy_version 1714782 (0.0008) [2023-12-27 03:53:11,528][105620] Updated weights for policy 1, policy_version 1718377 (0.0009) [2023-12-27 03:53:12,122][105692] Updated weights for policy 0, policy_version 1714792 (0.0009) [2023-12-27 03:53:12,182][105692] Updated weights for policy 0, policy_version 1714802 (0.0009) [2023-12-27 03:53:12,246][105692] Updated weights for policy 0, policy_version 1714812 (0.0009) [2023-12-27 03:53:12,314][105620] Updated weights for policy 1, policy_version 1718387 (0.0010) [2023-12-27 03:53:12,379][105620] Updated weights for policy 1, policy_version 1718397 (0.0009) [2023-12-27 03:53:12,437][105620] Updated weights for policy 1, policy_version 1718407 (0.0009) [2023-12-27 03:53:12,965][105692] Updated weights for policy 0, policy_version 1714822 (0.0009) [2023-12-27 03:53:13,013][105692] Updated weights for policy 0, policy_version 1714832 (0.0008) [2023-12-27 03:53:13,067][105692] Updated weights for policy 0, policy_version 1714842 (0.0008) [2023-12-27 03:53:13,229][105620] Updated weights for policy 1, policy_version 1718417 (0.0009) [2023-12-27 03:53:13,275][105620] Updated weights for policy 1, policy_version 1718427 (0.0005) [2023-12-27 03:53:13,323][105620] Updated weights for policy 1, policy_version 1718437 (0.0005) [2023-12-27 03:53:13,384][105620] Updated weights for policy 1, policy_version 1718447 (0.0008) [2023-12-27 03:53:13,716][105692] Updated weights for policy 0, policy_version 1714852 (0.0008) [2023-12-27 03:53:13,779][105692] Updated weights for policy 0, policy_version 1714862 (0.0010) [2023-12-27 03:53:13,837][105692] Updated weights for policy 0, policy_version 1714872 (0.0012) [2023-12-27 03:53:14,005][105620] Updated weights for policy 1, policy_version 1718457 (0.0009) [2023-12-27 03:53:14,056][105620] Updated weights for policy 1, policy_version 1718467 (0.0008) [2023-12-27 03:53:14,110][105620] Updated weights for policy 1, policy_version 1718477 (0.0009) [2023-12-27 03:53:14,439][105692] Updated weights for policy 0, policy_version 1714882 (0.0010) [2023-12-27 03:53:14,490][105692] Updated weights for policy 0, policy_version 1714892 (0.0005) [2023-12-27 03:53:14,544][105692] Updated weights for policy 0, policy_version 1714902 (0.0005) [2023-12-27 03:53:14,601][105692] Updated weights for policy 0, policy_version 1714912 (0.0005) [2023-12-27 03:53:14,946][105620] Updated weights for policy 1, policy_version 1718487 (0.0009) [2023-12-27 03:53:14,999][105620] Updated weights for policy 1, policy_version 1718497 (0.0010) [2023-12-27 03:53:15,055][105620] Updated weights for policy 1, policy_version 1718507 (0.0008) [2023-12-27 03:53:15,175][105692] Updated weights for policy 0, policy_version 1714922 (0.0010) [2023-12-27 03:53:15,239][105692] Updated weights for policy 0, policy_version 1714932 (0.0011) [2023-12-27 03:53:15,302][105692] Updated weights for policy 0, policy_version 1714942 (0.0011) [2023-12-27 03:53:15,827][105620] Updated weights for policy 1, policy_version 1718517 (0.0007) [2023-12-27 03:53:15,875][105620] Updated weights for policy 1, policy_version 1718527 (0.0005) [2023-12-27 03:53:15,923][105620] Updated weights for policy 1, policy_version 1718537 (0.0005) [2023-12-27 03:53:16,048][105692] Updated weights for policy 0, policy_version 1714952 (0.0010) [2023-12-27 03:53:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 879099904. Throughput: 0: 9754.2, 1: 9781.7. Samples: 879069548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:53:16,062][104569] Avg episode reward: [(0, '8713.018'), (1, '8632.780')] [2023-12-27 03:53:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001718544_440008704.pth... [2023-12-27 03:53:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001717424_439721984.pth [2023-12-27 03:53:16,111][105692] Updated weights for policy 0, policy_version 1714962 (0.0010) [2023-12-27 03:53:16,165][105692] Updated weights for policy 0, policy_version 1714973 (0.0010) [2023-12-27 03:53:16,175][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001714976_439099392.pth... [2023-12-27 03:53:16,178][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001713792_438796288.pth [2023-12-27 03:53:16,519][105620] Updated weights for policy 1, policy_version 1718547 (0.0007) [2023-12-27 03:53:16,575][105620] Updated weights for policy 1, policy_version 1718557 (0.0010) [2023-12-27 03:53:16,633][105620] Updated weights for policy 1, policy_version 1718567 (0.0010) [2023-12-27 03:53:16,772][105692] Updated weights for policy 0, policy_version 1714984 (0.0008) [2023-12-27 03:53:16,828][105692] Updated weights for policy 0, policy_version 1714994 (0.0008) [2023-12-27 03:53:16,880][105692] Updated weights for policy 0, policy_version 1715004 (0.0008) [2023-12-27 03:53:17,358][105620] Updated weights for policy 1, policy_version 1718577 (0.0010) [2023-12-27 03:53:17,422][105620] Updated weights for policy 1, policy_version 1718587 (0.0008) [2023-12-27 03:53:17,477][105620] Updated weights for policy 1, policy_version 1718597 (0.0008) [2023-12-27 03:53:17,537][105620] Updated weights for policy 1, policy_version 1718607 (0.0008) [2023-12-27 03:53:17,670][105692] Updated weights for policy 0, policy_version 1715014 (0.0010) [2023-12-27 03:53:17,727][105692] Updated weights for policy 0, policy_version 1715024 (0.0010) [2023-12-27 03:53:17,786][105692] Updated weights for policy 0, policy_version 1715034 (0.0011) [2023-12-27 03:53:18,243][105620] Updated weights for policy 1, policy_version 1718617 (0.0008) [2023-12-27 03:53:18,294][105620] Updated weights for policy 1, policy_version 1718627 (0.0008) [2023-12-27 03:53:18,353][105620] Updated weights for policy 1, policy_version 1718637 (0.0007) [2023-12-27 03:53:18,499][105692] Updated weights for policy 0, policy_version 1715044 (0.0010) [2023-12-27 03:53:18,557][105692] Updated weights for policy 0, policy_version 1715054 (0.0008) [2023-12-27 03:53:18,607][105692] Updated weights for policy 0, policy_version 1715064 (0.0006) [2023-12-27 03:53:19,051][105620] Updated weights for policy 1, policy_version 1718647 (0.0009) [2023-12-27 03:53:19,116][105620] Updated weights for policy 1, policy_version 1718657 (0.0009) [2023-12-27 03:53:19,180][105620] Updated weights for policy 1, policy_version 1718667 (0.0009) [2023-12-27 03:53:19,322][105692] Updated weights for policy 0, policy_version 1715074 (0.0009) [2023-12-27 03:53:19,395][105692] Updated weights for policy 0, policy_version 1715084 (0.0008) [2023-12-27 03:53:19,458][105692] Updated weights for policy 0, policy_version 1715094 (0.0009) [2023-12-27 03:53:19,523][105692] Updated weights for policy 0, policy_version 1715104 (0.0009) [2023-12-27 03:53:19,998][105620] Updated weights for policy 1, policy_version 1718677 (0.0009) [2023-12-27 03:53:20,060][105620] Updated weights for policy 1, policy_version 1718687 (0.0009) [2023-12-27 03:53:20,119][105620] Updated weights for policy 1, policy_version 1718697 (0.0009) [2023-12-27 03:53:20,204][105692] Updated weights for policy 0, policy_version 1715114 (0.0009) [2023-12-27 03:53:20,267][105692] Updated weights for policy 0, policy_version 1715124 (0.0009) [2023-12-27 03:53:20,323][105692] Updated weights for policy 0, policy_version 1715134 (0.0009) [2023-12-27 03:53:20,855][105620] Updated weights for policy 1, policy_version 1718707 (0.0007) [2023-12-27 03:53:20,915][105620] Updated weights for policy 1, policy_version 1718717 (0.0005) [2023-12-27 03:53:20,973][105620] Updated weights for policy 1, policy_version 1718727 (0.0007) [2023-12-27 03:53:21,043][105692] Updated weights for policy 0, policy_version 1715144 (0.0008) [2023-12-27 03:53:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 879198208. Throughput: 0: 9832.7, 1: 9657.1. Samples: 879187620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:53:21,063][104569] Avg episode reward: [(0, '8622.473'), (1, '8719.943')] [2023-12-27 03:53:21,108][105692] Updated weights for policy 0, policy_version 1715154 (0.0009) [2023-12-27 03:53:21,171][105692] Updated weights for policy 0, policy_version 1715164 (0.0009) [2023-12-27 03:53:21,642][105620] Updated weights for policy 1, policy_version 1718737 (0.0008) [2023-12-27 03:53:21,710][105620] Updated weights for policy 1, policy_version 1718747 (0.0008) [2023-12-27 03:53:21,770][105620] Updated weights for policy 1, policy_version 1718757 (0.0008) [2023-12-27 03:53:21,831][105620] Updated weights for policy 1, policy_version 1718767 (0.0009) [2023-12-27 03:53:21,924][105692] Updated weights for policy 0, policy_version 1715174 (0.0009) [2023-12-27 03:53:21,979][105692] Updated weights for policy 0, policy_version 1715184 (0.0008) [2023-12-27 03:53:22,037][105692] Updated weights for policy 0, policy_version 1715194 (0.0008) [2023-12-27 03:53:22,631][105620] Updated weights for policy 1, policy_version 1718777 (0.0011) [2023-12-27 03:53:22,696][105620] Updated weights for policy 1, policy_version 1718787 (0.0010) [2023-12-27 03:53:22,760][105620] Updated weights for policy 1, policy_version 1718797 (0.0008) [2023-12-27 03:53:22,822][105692] Updated weights for policy 0, policy_version 1715204 (0.0009) [2023-12-27 03:53:22,887][105692] Updated weights for policy 0, policy_version 1715214 (0.0008) [2023-12-27 03:53:22,955][105692] Updated weights for policy 0, policy_version 1715224 (0.0009) [2023-12-27 03:53:23,496][105620] Updated weights for policy 1, policy_version 1718807 (0.0010) [2023-12-27 03:53:23,551][105620] Updated weights for policy 1, policy_version 1718817 (0.0010) [2023-12-27 03:53:23,609][105620] Updated weights for policy 1, policy_version 1718827 (0.0010) [2023-12-27 03:53:23,719][105692] Updated weights for policy 0, policy_version 1715234 (0.0008) [2023-12-27 03:53:23,774][105692] Updated weights for policy 0, policy_version 1715244 (0.0008) [2023-12-27 03:53:23,820][105692] Updated weights for policy 0, policy_version 1715254 (0.0008) [2023-12-27 03:53:23,878][105692] Updated weights for policy 0, policy_version 1715264 (0.0008) [2023-12-27 03:53:24,357][105620] Updated weights for policy 1, policy_version 1718837 (0.0009) [2023-12-27 03:53:24,424][105620] Updated weights for policy 1, policy_version 1718847 (0.0007) [2023-12-27 03:53:24,485][105620] Updated weights for policy 1, policy_version 1718857 (0.0008) [2023-12-27 03:53:24,626][105692] Updated weights for policy 0, policy_version 1715274 (0.0005) [2023-12-27 03:53:24,696][105692] Updated weights for policy 0, policy_version 1715284 (0.0005) [2023-12-27 03:53:24,763][105692] Updated weights for policy 0, policy_version 1715294 (0.0005) [2023-12-27 03:53:25,232][105620] Updated weights for policy 1, policy_version 1718867 (0.0010) [2023-12-27 03:53:25,284][105620] Updated weights for policy 1, policy_version 1718877 (0.0006) [2023-12-27 03:53:25,343][105620] Updated weights for policy 1, policy_version 1718887 (0.0007) [2023-12-27 03:53:25,350][105692] Updated weights for policy 0, policy_version 1715304 (0.0006) [2023-12-27 03:53:25,401][105692] Updated weights for policy 0, policy_version 1715314 (0.0007) [2023-12-27 03:53:25,455][105692] Updated weights for policy 0, policy_version 1715324 (0.0009) [2023-12-27 03:53:25,936][105620] Updated weights for policy 1, policy_version 1718897 (0.0006) [2023-12-27 03:53:25,990][105620] Updated weights for policy 1, policy_version 1718907 (0.0006) [2023-12-27 03:53:26,044][105620] Updated weights for policy 1, policy_version 1718917 (0.0005) [2023-12-27 03:53:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 879288320. Throughput: 0: 9848.2, 1: 9675.6. Samples: 879302900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:53:26,062][104569] Avg episode reward: [(0, '8621.671'), (1, '9081.111')] [2023-12-27 03:53:26,095][105620] Updated weights for policy 1, policy_version 1718927 (0.0006) [2023-12-27 03:53:26,187][105692] Updated weights for policy 0, policy_version 1715334 (0.0010) [2023-12-27 03:53:26,238][105692] Updated weights for policy 0, policy_version 1715344 (0.0010) [2023-12-27 03:53:26,289][105692] Updated weights for policy 0, policy_version 1715354 (0.0010) [2023-12-27 03:53:26,800][105620] Updated weights for policy 1, policy_version 1718937 (0.0009) [2023-12-27 03:53:26,853][105620] Updated weights for policy 1, policy_version 1718947 (0.0008) [2023-12-27 03:53:26,913][105692] Updated weights for policy 0, policy_version 1715364 (0.0009) [2023-12-27 03:53:26,918][105620] Updated weights for policy 1, policy_version 1718957 (0.0008) [2023-12-27 03:53:26,966][105692] Updated weights for policy 0, policy_version 1715374 (0.0007) [2023-12-27 03:53:27,028][105692] Updated weights for policy 0, policy_version 1715384 (0.0006) [2023-12-27 03:53:27,626][105620] Updated weights for policy 1, policy_version 1718967 (0.0009) [2023-12-27 03:53:27,676][105620] Updated weights for policy 1, policy_version 1718977 (0.0008) [2023-12-27 03:53:27,733][105620] Updated weights for policy 1, policy_version 1718987 (0.0009) [2023-12-27 03:53:27,748][105692] Updated weights for policy 0, policy_version 1715394 (0.0005) [2023-12-27 03:53:27,796][105692] Updated weights for policy 0, policy_version 1715404 (0.0005) [2023-12-27 03:53:27,858][105692] Updated weights for policy 0, policy_version 1715414 (0.0005) [2023-12-27 03:53:27,905][105692] Updated weights for policy 0, policy_version 1715424 (0.0008) [2023-12-27 03:53:28,470][105620] Updated weights for policy 1, policy_version 1718997 (0.0009) [2023-12-27 03:53:28,522][105620] Updated weights for policy 1, policy_version 1719007 (0.0008) [2023-12-27 03:53:28,555][105692] Updated weights for policy 0, policy_version 1715434 (0.0010) [2023-12-27 03:53:28,573][105620] Updated weights for policy 1, policy_version 1719017 (0.0007) [2023-12-27 03:53:28,616][105692] Updated weights for policy 0, policy_version 1715444 (0.0006) [2023-12-27 03:53:28,706][105692] Updated weights for policy 0, policy_version 1715454 (0.0010) [2023-12-27 03:53:29,265][105692] Updated weights for policy 0, policy_version 1715464 (0.0009) [2023-12-27 03:53:29,317][105692] Updated weights for policy 0, policy_version 1715474 (0.0010) [2023-12-27 03:53:29,372][105692] Updated weights for policy 0, policy_version 1715484 (0.0012) [2023-12-27 03:53:29,411][105620] Updated weights for policy 1, policy_version 1719027 (0.0007) [2023-12-27 03:53:29,469][105620] Updated weights for policy 1, policy_version 1719037 (0.0005) [2023-12-27 03:53:29,528][105620] Updated weights for policy 1, policy_version 1719047 (0.0005) [2023-12-27 03:53:30,032][105692] Updated weights for policy 0, policy_version 1715494 (0.0011) [2023-12-27 03:53:30,083][105692] Updated weights for policy 0, policy_version 1715504 (0.0010) [2023-12-27 03:53:30,131][105692] Updated weights for policy 0, policy_version 1715514 (0.0010) [2023-12-27 03:53:30,152][105620] Updated weights for policy 1, policy_version 1719057 (0.0006) [2023-12-27 03:53:30,207][105620] Updated weights for policy 1, policy_version 1719067 (0.0005) [2023-12-27 03:53:30,258][105620] Updated weights for policy 1, policy_version 1719077 (0.0006) [2023-12-27 03:53:30,310][105620] Updated weights for policy 1, policy_version 1719087 (0.0011) [2023-12-27 03:53:30,792][105692] Updated weights for policy 0, policy_version 1715524 (0.0008) [2023-12-27 03:53:30,849][105692] Updated weights for policy 0, policy_version 1715534 (0.0006) [2023-12-27 03:53:30,910][105692] Updated weights for policy 0, policy_version 1715544 (0.0010) [2023-12-27 03:53:31,058][105620] Updated weights for policy 1, policy_version 1719097 (0.0010) [2023-12-27 03:53:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 879394816. Throughput: 0: 9937.6, 1: 9676.3. Samples: 879363348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:53:31,062][104569] Avg episode reward: [(0, '9082.347'), (1, '8713.454')] [2023-12-27 03:53:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001715552_439246848.pth... [2023-12-27 03:53:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001714368_438943744.pth [2023-12-27 03:53:31,115][105620] Updated weights for policy 1, policy_version 1719107 (0.0010) [2023-12-27 03:53:31,172][105620] Updated weights for policy 1, policy_version 1719117 (0.0011) [2023-12-27 03:53:31,185][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001719120_440156160.pth... [2023-12-27 03:53:31,188][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001717968_439861248.pth [2023-12-27 03:53:31,696][105692] Updated weights for policy 0, policy_version 1715554 (0.0010) [2023-12-27 03:53:31,766][105692] Updated weights for policy 0, policy_version 1715564 (0.0008) [2023-12-27 03:53:31,823][105692] Updated weights for policy 0, policy_version 1715574 (0.0009) [2023-12-27 03:53:31,850][105620] Updated weights for policy 1, policy_version 1719127 (0.0008) [2023-12-27 03:53:31,878][105692] Updated weights for policy 0, policy_version 1715584 (0.0008) [2023-12-27 03:53:31,914][105620] Updated weights for policy 1, policy_version 1719137 (0.0006) [2023-12-27 03:53:31,977][105620] Updated weights for policy 1, policy_version 1719147 (0.0008) [2023-12-27 03:53:32,645][105692] Updated weights for policy 0, policy_version 1715594 (0.0008) [2023-12-27 03:53:32,698][105692] Updated weights for policy 0, policy_version 1715604 (0.0005) [2023-12-27 03:53:32,736][105620] Updated weights for policy 1, policy_version 1719157 (0.0009) [2023-12-27 03:53:32,758][105692] Updated weights for policy 0, policy_version 1715614 (0.0005) [2023-12-27 03:53:32,795][105620] Updated weights for policy 1, policy_version 1719167 (0.0011) [2023-12-27 03:53:32,851][105620] Updated weights for policy 1, policy_version 1719177 (0.0010) [2023-12-27 03:53:33,348][105692] Updated weights for policy 0, policy_version 1715624 (0.0005) [2023-12-27 03:53:33,397][105692] Updated weights for policy 0, policy_version 1715634 (0.0006) [2023-12-27 03:53:33,440][105692] Updated weights for policy 0, policy_version 1715644 (0.0005) [2023-12-27 03:53:33,574][105620] Updated weights for policy 1, policy_version 1719187 (0.0009) [2023-12-27 03:53:33,642][105620] Updated weights for policy 1, policy_version 1719197 (0.0005) [2023-12-27 03:53:33,705][105620] Updated weights for policy 1, policy_version 1719207 (0.0005) [2023-12-27 03:53:34,021][105692] Updated weights for policy 0, policy_version 1715654 (0.0005) [2023-12-27 03:53:34,085][105692] Updated weights for policy 0, policy_version 1715664 (0.0007) [2023-12-27 03:53:34,145][105692] Updated weights for policy 0, policy_version 1715674 (0.0009) [2023-12-27 03:53:34,209][105620] Updated weights for policy 1, policy_version 1719217 (0.0005) [2023-12-27 03:53:34,269][105620] Updated weights for policy 1, policy_version 1719227 (0.0006) [2023-12-27 03:53:34,327][105620] Updated weights for policy 1, policy_version 1719237 (0.0005) [2023-12-27 03:53:34,395][105620] Updated weights for policy 1, policy_version 1719247 (0.0005) [2023-12-27 03:53:34,788][105692] Updated weights for policy 0, policy_version 1715684 (0.0007) [2023-12-27 03:53:34,852][105692] Updated weights for policy 0, policy_version 1715694 (0.0006) [2023-12-27 03:53:34,912][105692] Updated weights for policy 0, policy_version 1715704 (0.0007) [2023-12-27 03:53:35,009][105620] Updated weights for policy 1, policy_version 1719257 (0.0010) [2023-12-27 03:53:35,082][105620] Updated weights for policy 1, policy_version 1719267 (0.0011) [2023-12-27 03:53:35,148][105620] Updated weights for policy 1, policy_version 1719277 (0.0011) [2023-12-27 03:53:35,662][105692] Updated weights for policy 0, policy_version 1715714 (0.0008) [2023-12-27 03:53:35,723][105692] Updated weights for policy 0, policy_version 1715724 (0.0010) [2023-12-27 03:53:35,759][105620] Updated weights for policy 1, policy_version 1719287 (0.0009) [2023-12-27 03:53:35,771][105692] Updated weights for policy 0, policy_version 1715734 (0.0008) [2023-12-27 03:53:35,821][105620] Updated weights for policy 1, policy_version 1719297 (0.0005) [2023-12-27 03:53:35,833][105692] Updated weights for policy 0, policy_version 1715744 (0.0008) [2023-12-27 03:53:35,885][105620] Updated weights for policy 1, policy_version 1719307 (0.0005) [2023-12-27 03:53:36,062][104569] Fps is (10 sec: 21299.0, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 879501312. Throughput: 0: 10012.4, 1: 9710.4. Samples: 879486756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:53:36,063][104569] Avg episode reward: [(0, '8441.404'), (1, '8346.315')] [2023-12-27 03:53:36,455][105620] Updated weights for policy 1, policy_version 1719317 (0.0005) [2023-12-27 03:53:36,517][105620] Updated weights for policy 1, policy_version 1719327 (0.0010) [2023-12-27 03:53:36,568][105620] Updated weights for policy 1, policy_version 1719337 (0.0009) [2023-12-27 03:53:36,579][105692] Updated weights for policy 0, policy_version 1715754 (0.0008) [2023-12-27 03:53:36,640][105692] Updated weights for policy 0, policy_version 1715764 (0.0007) [2023-12-27 03:53:36,706][105692] Updated weights for policy 0, policy_version 1715774 (0.0007) [2023-12-27 03:53:37,255][105620] Updated weights for policy 1, policy_version 1719347 (0.0009) [2023-12-27 03:53:37,319][105620] Updated weights for policy 1, policy_version 1719357 (0.0006) [2023-12-27 03:53:37,388][105620] Updated weights for policy 1, policy_version 1719367 (0.0006) [2023-12-27 03:53:37,490][105692] Updated weights for policy 0, policy_version 1715784 (0.0009) [2023-12-27 03:53:37,537][105692] Updated weights for policy 0, policy_version 1715794 (0.0009) [2023-12-27 03:53:37,589][105692] Updated weights for policy 0, policy_version 1715805 (0.0009) [2023-12-27 03:53:38,027][105620] Updated weights for policy 1, policy_version 1719377 (0.0009) [2023-12-27 03:53:38,077][105620] Updated weights for policy 1, policy_version 1719387 (0.0009) [2023-12-27 03:53:38,134][105620] Updated weights for policy 1, policy_version 1719397 (0.0008) [2023-12-27 03:53:38,190][105620] Updated weights for policy 1, policy_version 1719407 (0.0007) [2023-12-27 03:53:38,356][105692] Updated weights for policy 0, policy_version 1715815 (0.0010) [2023-12-27 03:53:38,418][105692] Updated weights for policy 0, policy_version 1715825 (0.0011) [2023-12-27 03:53:38,466][105692] Updated weights for policy 0, policy_version 1715835 (0.0011) [2023-12-27 03:53:38,908][105620] Updated weights for policy 1, policy_version 1719417 (0.0007) [2023-12-27 03:53:38,958][105620] Updated weights for policy 1, policy_version 1719427 (0.0008) [2023-12-27 03:53:39,004][105620] Updated weights for policy 1, policy_version 1719437 (0.0008) [2023-12-27 03:53:39,190][105692] Updated weights for policy 0, policy_version 1715845 (0.0009) [2023-12-27 03:53:39,258][105692] Updated weights for policy 0, policy_version 1715855 (0.0008) [2023-12-27 03:53:39,324][105692] Updated weights for policy 0, policy_version 1715865 (0.0009) [2023-12-27 03:53:39,873][105620] Updated weights for policy 1, policy_version 1719447 (0.0010) [2023-12-27 03:53:39,940][105620] Updated weights for policy 1, policy_version 1719457 (0.0010) [2023-12-27 03:53:39,959][105692] Updated weights for policy 0, policy_version 1715875 (0.0009) [2023-12-27 03:53:40,004][105620] Updated weights for policy 1, policy_version 1719467 (0.0007) [2023-12-27 03:53:40,017][105692] Updated weights for policy 0, policy_version 1715885 (0.0008) [2023-12-27 03:53:40,074][105692] Updated weights for policy 0, policy_version 1715895 (0.0010) [2023-12-27 03:53:40,605][105620] Updated weights for policy 1, policy_version 1719477 (0.0006) [2023-12-27 03:53:40,664][105620] Updated weights for policy 1, policy_version 1719487 (0.0008) [2023-12-27 03:53:40,734][105620] Updated weights for policy 1, policy_version 1719497 (0.0009) [2023-12-27 03:53:40,899][105692] Updated weights for policy 0, policy_version 1715905 (0.0008) [2023-12-27 03:53:40,951][105692] Updated weights for policy 0, policy_version 1715915 (0.0008) [2023-12-27 03:53:41,000][105692] Updated weights for policy 0, policy_version 1715925 (0.0008) [2023-12-27 03:53:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 879591424. Throughput: 0: 9918.5, 1: 9820.9. Samples: 879603980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:53:41,063][104569] Avg episode reward: [(0, '7895.340'), (1, '8804.549')] [2023-12-27 03:53:41,067][105692] Updated weights for policy 0, policy_version 1715935 (0.0008) [2023-12-27 03:53:41,487][105620] Updated weights for policy 1, policy_version 1719507 (0.0009) [2023-12-27 03:53:41,545][105620] Updated weights for policy 1, policy_version 1719517 (0.0009) [2023-12-27 03:53:41,600][105620] Updated weights for policy 1, policy_version 1719527 (0.0009) [2023-12-27 03:53:41,899][105692] Updated weights for policy 0, policy_version 1715945 (0.0010) [2023-12-27 03:53:41,964][105692] Updated weights for policy 0, policy_version 1715955 (0.0009) [2023-12-27 03:53:42,027][105692] Updated weights for policy 0, policy_version 1715965 (0.0009) [2023-12-27 03:53:42,356][105620] Updated weights for policy 1, policy_version 1719537 (0.0010) [2023-12-27 03:53:42,413][105620] Updated weights for policy 1, policy_version 1719547 (0.0009) [2023-12-27 03:53:42,467][105620] Updated weights for policy 1, policy_version 1719557 (0.0009) [2023-12-27 03:53:42,516][105620] Updated weights for policy 1, policy_version 1719567 (0.0009) [2023-12-27 03:53:42,850][105692] Updated weights for policy 0, policy_version 1715975 (0.0010) [2023-12-27 03:53:42,920][105692] Updated weights for policy 0, policy_version 1715985 (0.0007) [2023-12-27 03:53:42,985][105692] Updated weights for policy 0, policy_version 1715995 (0.0005) [2023-12-27 03:53:43,256][105620] Updated weights for policy 1, policy_version 1719577 (0.0009) [2023-12-27 03:53:43,318][105620] Updated weights for policy 1, policy_version 1719587 (0.0009) [2023-12-27 03:53:43,378][105620] Updated weights for policy 1, policy_version 1719598 (0.0008) [2023-12-27 03:53:43,538][105692] Updated weights for policy 0, policy_version 1716005 (0.0006) [2023-12-27 03:53:43,591][105692] Updated weights for policy 0, policy_version 1716015 (0.0006) [2023-12-27 03:53:43,642][105692] Updated weights for policy 0, policy_version 1716025 (0.0010) [2023-12-27 03:53:44,039][105620] Updated weights for policy 1, policy_version 1719608 (0.0007) [2023-12-27 03:53:44,087][105620] Updated weights for policy 1, policy_version 1719618 (0.0008) [2023-12-27 03:53:44,139][105620] Updated weights for policy 1, policy_version 1719628 (0.0008) [2023-12-27 03:53:44,310][105692] Updated weights for policy 0, policy_version 1716035 (0.0010) [2023-12-27 03:53:44,370][105692] Updated weights for policy 0, policy_version 1716045 (0.0011) [2023-12-27 03:53:44,432][105692] Updated weights for policy 0, policy_version 1716055 (0.0006) [2023-12-27 03:53:44,902][105620] Updated weights for policy 1, policy_version 1719638 (0.0008) [2023-12-27 03:53:44,966][105620] Updated weights for policy 1, policy_version 1719648 (0.0008) [2023-12-27 03:53:45,030][105620] Updated weights for policy 1, policy_version 1719658 (0.0008) [2023-12-27 03:53:45,108][105692] Updated weights for policy 0, policy_version 1716065 (0.0007) [2023-12-27 03:53:45,163][105692] Updated weights for policy 0, policy_version 1716075 (0.0009) [2023-12-27 03:53:45,223][105692] Updated weights for policy 0, policy_version 1716085 (0.0009) [2023-12-27 03:53:45,288][105692] Updated weights for policy 0, policy_version 1716095 (0.0009) [2023-12-27 03:53:45,675][105620] Updated weights for policy 1, policy_version 1719668 (0.0007) [2023-12-27 03:53:45,744][105620] Updated weights for policy 1, policy_version 1719678 (0.0006) [2023-12-27 03:53:45,797][105620] Updated weights for policy 1, policy_version 1719688 (0.0008) [2023-12-27 03:53:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 879689728. Throughput: 0: 9900.3, 1: 9811.5. Samples: 879661980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:53:46,062][104569] Avg episode reward: [(0, '8172.169'), (1, '8632.890')] [2023-12-27 03:53:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001716096_439386112.pth... [2023-12-27 03:53:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001719696_440303616.pth... [2023-12-27 03:53:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001714976_439099392.pth [2023-12-27 03:53:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001718544_440008704.pth [2023-12-27 03:53:46,151][105692] Updated weights for policy 0, policy_version 1716105 (0.0009) [2023-12-27 03:53:46,219][105692] Updated weights for policy 0, policy_version 1716115 (0.0006) [2023-12-27 03:53:46,286][105692] Updated weights for policy 0, policy_version 1716125 (0.0009) [2023-12-27 03:53:46,446][105620] Updated weights for policy 1, policy_version 1719698 (0.0006) [2023-12-27 03:53:46,506][105620] Updated weights for policy 1, policy_version 1719708 (0.0005) [2023-12-27 03:53:46,562][105620] Updated weights for policy 1, policy_version 1719718 (0.0005) [2023-12-27 03:53:46,615][105620] Updated weights for policy 1, policy_version 1719728 (0.0009) [2023-12-27 03:53:47,027][105692] Updated weights for policy 0, policy_version 1716135 (0.0009) [2023-12-27 03:53:47,087][105692] Updated weights for policy 0, policy_version 1716145 (0.0009) [2023-12-27 03:53:47,145][105692] Updated weights for policy 0, policy_version 1716155 (0.0009) [2023-12-27 03:53:47,319][105620] Updated weights for policy 1, policy_version 1719738 (0.0009) [2023-12-27 03:53:47,366][105620] Updated weights for policy 1, policy_version 1719748 (0.0009) [2023-12-27 03:53:47,426][105620] Updated weights for policy 1, policy_version 1719758 (0.0009) [2023-12-27 03:53:47,817][105692] Updated weights for policy 0, policy_version 1716165 (0.0008) [2023-12-27 03:53:47,880][105692] Updated weights for policy 0, policy_version 1716175 (0.0006) [2023-12-27 03:53:47,939][105692] Updated weights for policy 0, policy_version 1716185 (0.0009) [2023-12-27 03:53:48,140][105620] Updated weights for policy 1, policy_version 1719768 (0.0006) [2023-12-27 03:53:48,188][105620] Updated weights for policy 1, policy_version 1719778 (0.0005) [2023-12-27 03:53:48,241][105620] Updated weights for policy 1, policy_version 1719788 (0.0006) [2023-12-27 03:53:48,582][105692] Updated weights for policy 0, policy_version 1716195 (0.0008) [2023-12-27 03:53:48,648][105692] Updated weights for policy 0, policy_version 1716205 (0.0006) [2023-12-27 03:53:48,706][105692] Updated weights for policy 0, policy_version 1716215 (0.0006) [2023-12-27 03:53:48,811][105620] Updated weights for policy 1, policy_version 1719798 (0.0005) [2023-12-27 03:53:48,878][105620] Updated weights for policy 1, policy_version 1719808 (0.0005) [2023-12-27 03:53:48,940][105620] Updated weights for policy 1, policy_version 1719818 (0.0008) [2023-12-27 03:53:49,275][105692] Updated weights for policy 0, policy_version 1716225 (0.0006) [2023-12-27 03:53:49,339][105692] Updated weights for policy 0, policy_version 1716235 (0.0005) [2023-12-27 03:53:49,408][105692] Updated weights for policy 0, policy_version 1716245 (0.0007) [2023-12-27 03:53:49,476][105692] Updated weights for policy 0, policy_version 1716255 (0.0007) [2023-12-27 03:53:49,648][105620] Updated weights for policy 1, policy_version 1719828 (0.0006) [2023-12-27 03:53:49,705][105620] Updated weights for policy 1, policy_version 1719838 (0.0008) [2023-12-27 03:53:49,767][105620] Updated weights for policy 1, policy_version 1719848 (0.0009) [2023-12-27 03:53:50,113][105692] Updated weights for policy 0, policy_version 1716265 (0.0006) [2023-12-27 03:53:50,174][105692] Updated weights for policy 0, policy_version 1716275 (0.0009) [2023-12-27 03:53:50,223][105692] Updated weights for policy 0, policy_version 1716285 (0.0009) [2023-12-27 03:53:50,607][105620] Updated weights for policy 1, policy_version 1719858 (0.0009) [2023-12-27 03:53:50,670][105620] Updated weights for policy 1, policy_version 1719868 (0.0006) [2023-12-27 03:53:50,730][105620] Updated weights for policy 1, policy_version 1719878 (0.0005) [2023-12-27 03:53:50,777][105620] Updated weights for policy 1, policy_version 1719888 (0.0005) [2023-12-27 03:53:50,933][105692] Updated weights for policy 0, policy_version 1716295 (0.0009) [2023-12-27 03:53:50,988][105692] Updated weights for policy 0, policy_version 1716305 (0.0010) [2023-12-27 03:53:51,054][105692] Updated weights for policy 0, policy_version 1716316 (0.0007) [2023-12-27 03:53:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 879788032. Throughput: 0: 9950.6, 1: 9757.2. Samples: 879782060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:53:51,063][104569] Avg episode reward: [(0, '8626.997'), (1, '8447.181')] [2023-12-27 03:53:51,400][105620] Updated weights for policy 1, policy_version 1719898 (0.0008) [2023-12-27 03:53:51,447][105620] Updated weights for policy 1, policy_version 1719908 (0.0008) [2023-12-27 03:53:51,501][105620] Updated weights for policy 1, policy_version 1719918 (0.0008) [2023-12-27 03:53:51,855][105692] Updated weights for policy 0, policy_version 1716326 (0.0010) [2023-12-27 03:53:51,914][105692] Updated weights for policy 0, policy_version 1716336 (0.0011) [2023-12-27 03:53:51,977][105692] Updated weights for policy 0, policy_version 1716346 (0.0011) [2023-12-27 03:53:52,305][105620] Updated weights for policy 1, policy_version 1719928 (0.0009) [2023-12-27 03:53:52,367][105620] Updated weights for policy 1, policy_version 1719938 (0.0008) [2023-12-27 03:53:52,431][105620] Updated weights for policy 1, policy_version 1719948 (0.0008) [2023-12-27 03:53:52,684][105692] Updated weights for policy 0, policy_version 1716356 (0.0010) [2023-12-27 03:53:52,751][105692] Updated weights for policy 0, policy_version 1716366 (0.0010) [2023-12-27 03:53:52,808][105692] Updated weights for policy 0, policy_version 1716376 (0.0009) [2023-12-27 03:53:53,058][105620] Updated weights for policy 1, policy_version 1719958 (0.0006) [2023-12-27 03:53:53,112][105620] Updated weights for policy 1, policy_version 1719968 (0.0005) [2023-12-27 03:53:53,170][105620] Updated weights for policy 1, policy_version 1719978 (0.0005) [2023-12-27 03:53:53,635][105692] Updated weights for policy 0, policy_version 1716386 (0.0010) [2023-12-27 03:53:53,682][105692] Updated weights for policy 0, policy_version 1716396 (0.0008) [2023-12-27 03:53:53,729][105692] Updated weights for policy 0, policy_version 1716406 (0.0009) [2023-12-27 03:53:53,776][105692] Updated weights for policy 0, policy_version 1716416 (0.0009) [2023-12-27 03:53:53,815][105620] Updated weights for policy 1, policy_version 1719988 (0.0006) [2023-12-27 03:53:53,866][105620] Updated weights for policy 1, policy_version 1719998 (0.0009) [2023-12-27 03:53:53,918][105620] Updated weights for policy 1, policy_version 1720008 (0.0009) [2023-12-27 03:53:54,580][105692] Updated weights for policy 0, policy_version 1716426 (0.0009) [2023-12-27 03:53:54,631][105692] Updated weights for policy 0, policy_version 1716436 (0.0008) [2023-12-27 03:53:54,687][105692] Updated weights for policy 0, policy_version 1716446 (0.0005) [2023-12-27 03:53:54,704][105620] Updated weights for policy 1, policy_version 1720018 (0.0009) [2023-12-27 03:53:54,759][105620] Updated weights for policy 1, policy_version 1720028 (0.0009) [2023-12-27 03:53:54,812][105620] Updated weights for policy 1, policy_version 1720038 (0.0008) [2023-12-27 03:53:54,863][105620] Updated weights for policy 1, policy_version 1720048 (0.0008) [2023-12-27 03:53:55,316][105692] Updated weights for policy 0, policy_version 1716456 (0.0007) [2023-12-27 03:53:55,371][105692] Updated weights for policy 0, policy_version 1716466 (0.0005) [2023-12-27 03:53:55,428][105692] Updated weights for policy 0, policy_version 1716476 (0.0006) [2023-12-27 03:53:55,703][105620] Updated weights for policy 1, policy_version 1720058 (0.0010) [2023-12-27 03:53:55,761][105620] Updated weights for policy 1, policy_version 1720068 (0.0010) [2023-12-27 03:53:55,831][105620] Updated weights for policy 1, policy_version 1720078 (0.0009) [2023-12-27 03:53:56,060][105692] Updated weights for policy 0, policy_version 1716486 (0.0009) [2023-12-27 03:53:56,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.7, 300 sec: 19605.2). Total num frames: 879886336. Throughput: 0: 9889.4, 1: 9780.0. Samples: 879895832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:53:56,063][104569] Avg episode reward: [(0, '8627.342'), (1, '8801.929')] [2023-12-27 03:53:56,113][105692] Updated weights for policy 0, policy_version 1716496 (0.0006) [2023-12-27 03:53:56,166][105692] Updated weights for policy 0, policy_version 1716506 (0.0009) [2023-12-27 03:53:56,549][105620] Updated weights for policy 1, policy_version 1720088 (0.0009) [2023-12-27 03:53:56,597][105620] Updated weights for policy 1, policy_version 1720098 (0.0009) [2023-12-27 03:53:56,647][105620] Updated weights for policy 1, policy_version 1720108 (0.0008) [2023-12-27 03:53:56,911][105692] Updated weights for policy 0, policy_version 1716516 (0.0009) [2023-12-27 03:53:56,963][105692] Updated weights for policy 0, policy_version 1716526 (0.0009) [2023-12-27 03:53:57,014][105692] Updated weights for policy 0, policy_version 1716536 (0.0009) [2023-12-27 03:53:57,455][105620] Updated weights for policy 1, policy_version 1720118 (0.0009) [2023-12-27 03:53:57,517][105620] Updated weights for policy 1, policy_version 1720128 (0.0009) [2023-12-27 03:53:57,575][105620] Updated weights for policy 1, policy_version 1720138 (0.0009) [2023-12-27 03:53:57,708][105692] Updated weights for policy 0, policy_version 1716546 (0.0009) [2023-12-27 03:53:57,755][105692] Updated weights for policy 0, policy_version 1716556 (0.0009) [2023-12-27 03:53:57,801][105692] Updated weights for policy 0, policy_version 1716566 (0.0009) [2023-12-27 03:53:57,848][105692] Updated weights for policy 0, policy_version 1716576 (0.0008) [2023-12-27 03:53:58,324][105620] Updated weights for policy 1, policy_version 1720148 (0.0009) [2023-12-27 03:53:58,401][105620] Updated weights for policy 1, policy_version 1720158 (0.0007) [2023-12-27 03:53:58,462][105620] Updated weights for policy 1, policy_version 1720168 (0.0008) [2023-12-27 03:53:58,658][105692] Updated weights for policy 0, policy_version 1716586 (0.0008) [2023-12-27 03:53:58,722][105692] Updated weights for policy 0, policy_version 1716596 (0.0008) [2023-12-27 03:53:58,786][105692] Updated weights for policy 0, policy_version 1716606 (0.0007) [2023-12-27 03:53:59,293][105620] Updated weights for policy 1, policy_version 1720178 (0.0008) [2023-12-27 03:53:59,360][105620] Updated weights for policy 1, policy_version 1720188 (0.0008) [2023-12-27 03:53:59,429][105620] Updated weights for policy 1, policy_version 1720198 (0.0010) [2023-12-27 03:53:59,496][105620] Updated weights for policy 1, policy_version 1720208 (0.0009) [2023-12-27 03:53:59,562][105692] Updated weights for policy 0, policy_version 1716616 (0.0009) [2023-12-27 03:53:59,620][105692] Updated weights for policy 0, policy_version 1716626 (0.0005) [2023-12-27 03:53:59,673][105692] Updated weights for policy 0, policy_version 1716636 (0.0007) [2023-12-27 03:54:00,274][105620] Updated weights for policy 1, policy_version 1720218 (0.0007) [2023-12-27 03:54:00,312][105692] Updated weights for policy 0, policy_version 1716646 (0.0007) [2023-12-27 03:54:00,329][105620] Updated weights for policy 1, policy_version 1720228 (0.0010) [2023-12-27 03:54:00,359][105692] Updated weights for policy 0, policy_version 1716656 (0.0007) [2023-12-27 03:54:00,389][105620] Updated weights for policy 1, policy_version 1720238 (0.0009) [2023-12-27 03:54:00,408][105692] Updated weights for policy 0, policy_version 1716666 (0.0006) [2023-12-27 03:54:00,983][105620] Updated weights for policy 1, policy_version 1720248 (0.0006) [2023-12-27 03:54:01,051][105620] Updated weights for policy 1, policy_version 1720258 (0.0007) [2023-12-27 03:54:01,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 879976448. Throughput: 0: 9847.6, 1: 9778.2. Samples: 879952712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:54:01,062][104569] Avg episode reward: [(0, '8807.829'), (1, '8804.826')] [2023-12-27 03:54:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001716672_439533568.pth... [2023-12-27 03:54:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001715552_439246848.pth [2023-12-27 03:54:01,114][105620] Updated weights for policy 1, policy_version 1720268 (0.0007) [2023-12-27 03:54:01,140][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001720272_440451072.pth... [2023-12-27 03:54:01,145][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001719120_440156160.pth [2023-12-27 03:54:01,235][105692] Updated weights for policy 0, policy_version 1716676 (0.0009) [2023-12-27 03:54:01,291][105692] Updated weights for policy 0, policy_version 1716686 (0.0008) [2023-12-27 03:54:01,347][105692] Updated weights for policy 0, policy_version 1716696 (0.0008) [2023-12-27 03:54:01,757][105620] Updated weights for policy 1, policy_version 1720278 (0.0008) [2023-12-27 03:54:01,810][105620] Updated weights for policy 1, policy_version 1720288 (0.0006) [2023-12-27 03:54:01,862][105620] Updated weights for policy 1, policy_version 1720298 (0.0005) [2023-12-27 03:54:02,123][105692] Updated weights for policy 0, policy_version 1716706 (0.0008) [2023-12-27 03:54:02,177][105692] Updated weights for policy 0, policy_version 1716716 (0.0010) [2023-12-27 03:54:02,238][105692] Updated weights for policy 0, policy_version 1716726 (0.0006) [2023-12-27 03:54:02,299][105692] Updated weights for policy 0, policy_version 1716736 (0.0009) [2023-12-27 03:54:02,502][105620] Updated weights for policy 1, policy_version 1720308 (0.0008) [2023-12-27 03:54:02,567][105620] Updated weights for policy 1, policy_version 1720318 (0.0007) [2023-12-27 03:54:02,631][105620] Updated weights for policy 1, policy_version 1720328 (0.0008) [2023-12-27 03:54:03,045][105692] Updated weights for policy 0, policy_version 1716746 (0.0010) [2023-12-27 03:54:03,094][105692] Updated weights for policy 0, policy_version 1716756 (0.0009) [2023-12-27 03:54:03,150][105692] Updated weights for policy 0, policy_version 1716766 (0.0009) [2023-12-27 03:54:03,286][105620] Updated weights for policy 1, policy_version 1720338 (0.0008) [2023-12-27 03:54:03,335][105620] Updated weights for policy 1, policy_version 1720348 (0.0005) [2023-12-27 03:54:03,382][105620] Updated weights for policy 1, policy_version 1720358 (0.0006) [2023-12-27 03:54:03,428][105620] Updated weights for policy 1, policy_version 1720368 (0.0008) [2023-12-27 03:54:03,891][105692] Updated weights for policy 0, policy_version 1716776 (0.0010) [2023-12-27 03:54:03,943][105692] Updated weights for policy 0, policy_version 1716786 (0.0009) [2023-12-27 03:54:03,995][105692] Updated weights for policy 0, policy_version 1716796 (0.0009) [2023-12-27 03:54:04,197][105620] Updated weights for policy 1, policy_version 1720378 (0.0008) [2023-12-27 03:54:04,258][105620] Updated weights for policy 1, policy_version 1720388 (0.0009) [2023-12-27 03:54:04,315][105620] Updated weights for policy 1, policy_version 1720398 (0.0010) [2023-12-27 03:54:04,756][105692] Updated weights for policy 0, policy_version 1716806 (0.0009) [2023-12-27 03:54:04,816][105692] Updated weights for policy 0, policy_version 1716816 (0.0009) [2023-12-27 03:54:04,862][105692] Updated weights for policy 0, policy_version 1716826 (0.0007) [2023-12-27 03:54:05,106][105620] Updated weights for policy 1, policy_version 1720408 (0.0009) [2023-12-27 03:54:05,161][105620] Updated weights for policy 1, policy_version 1720418 (0.0012) [2023-12-27 03:54:05,207][105620] Updated weights for policy 1, policy_version 1720428 (0.0009) [2023-12-27 03:54:05,507][105692] Updated weights for policy 0, policy_version 1716836 (0.0007) [2023-12-27 03:54:05,559][105692] Updated weights for policy 0, policy_version 1716846 (0.0009) [2023-12-27 03:54:05,608][105692] Updated weights for policy 0, policy_version 1716856 (0.0009) [2023-12-27 03:54:05,985][105620] Updated weights for policy 1, policy_version 1720438 (0.0009) [2023-12-27 03:54:06,035][105620] Updated weights for policy 1, policy_version 1720448 (0.0009) [2023-12-27 03:54:06,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 880074752. Throughput: 0: 9754.0, 1: 9820.6. Samples: 880068476. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:54:06,062][104569] Avg episode reward: [(0, '8721.281'), (1, '8987.878')] [2023-12-27 03:54:06,083][105620] Updated weights for policy 1, policy_version 1720458 (0.0008) [2023-12-27 03:54:06,335][105692] Updated weights for policy 0, policy_version 1716866 (0.0009) [2023-12-27 03:54:06,400][105692] Updated weights for policy 0, policy_version 1716876 (0.0008) [2023-12-27 03:54:06,461][105692] Updated weights for policy 0, policy_version 1716886 (0.0009) [2023-12-27 03:54:06,524][105692] Updated weights for policy 0, policy_version 1716896 (0.0009) [2023-12-27 03:54:06,927][105620] Updated weights for policy 1, policy_version 1720468 (0.0008) [2023-12-27 03:54:06,991][105620] Updated weights for policy 1, policy_version 1720478 (0.0009) [2023-12-27 03:54:07,046][105620] Updated weights for policy 1, policy_version 1720488 (0.0008) [2023-12-27 03:54:07,267][105692] Updated weights for policy 0, policy_version 1716906 (0.0006) [2023-12-27 03:54:07,319][105692] Updated weights for policy 0, policy_version 1716916 (0.0008) [2023-12-27 03:54:07,374][105692] Updated weights for policy 0, policy_version 1716926 (0.0009) [2023-12-27 03:54:07,823][105620] Updated weights for policy 1, policy_version 1720498 (0.0009) [2023-12-27 03:54:07,881][105620] Updated weights for policy 1, policy_version 1720508 (0.0008) [2023-12-27 03:54:07,928][105620] Updated weights for policy 1, policy_version 1720518 (0.0008) [2023-12-27 03:54:07,967][105692] Updated weights for policy 0, policy_version 1716936 (0.0007) [2023-12-27 03:54:07,977][105620] Updated weights for policy 1, policy_version 1720528 (0.0006) [2023-12-27 03:54:08,016][105692] Updated weights for policy 0, policy_version 1716946 (0.0008) [2023-12-27 03:54:08,068][105692] Updated weights for policy 0, policy_version 1716956 (0.0009) [2023-12-27 03:54:08,765][105620] Updated weights for policy 1, policy_version 1720538 (0.0009) [2023-12-27 03:54:08,824][105620] Updated weights for policy 1, policy_version 1720548 (0.0007) [2023-12-27 03:54:08,848][105692] Updated weights for policy 0, policy_version 1716966 (0.0009) [2023-12-27 03:54:08,880][105620] Updated weights for policy 1, policy_version 1720558 (0.0010) [2023-12-27 03:54:08,903][105692] Updated weights for policy 0, policy_version 1716976 (0.0009) [2023-12-27 03:54:08,965][105692] Updated weights for policy 0, policy_version 1716986 (0.0009) [2023-12-27 03:54:09,621][105620] Updated weights for policy 1, policy_version 1720568 (0.0009) [2023-12-27 03:54:09,675][105620] Updated weights for policy 1, policy_version 1720578 (0.0008) [2023-12-27 03:54:09,734][105692] Updated weights for policy 0, policy_version 1716996 (0.0009) [2023-12-27 03:54:09,736][105620] Updated weights for policy 1, policy_version 1720588 (0.0008) [2023-12-27 03:54:09,796][105692] Updated weights for policy 0, policy_version 1717006 (0.0008) [2023-12-27 03:54:09,866][105692] Updated weights for policy 0, policy_version 1717016 (0.0008) [2023-12-27 03:54:10,518][105620] Updated weights for policy 1, policy_version 1720598 (0.0009) [2023-12-27 03:54:10,566][105620] Updated weights for policy 1, policy_version 1720608 (0.0011) [2023-12-27 03:54:10,623][105620] Updated weights for policy 1, policy_version 1720618 (0.0011) [2023-12-27 03:54:10,633][105692] Updated weights for policy 0, policy_version 1717026 (0.0008) [2023-12-27 03:54:10,686][105692] Updated weights for policy 0, policy_version 1717036 (0.0007) [2023-12-27 03:54:10,735][105692] Updated weights for policy 0, policy_version 1717046 (0.0008) [2023-12-27 03:54:10,795][105692] Updated weights for policy 0, policy_version 1717056 (0.0008) [2023-12-27 03:54:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 880173056. Throughput: 0: 9773.6, 1: 9738.3. Samples: 880180936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:54:11,062][104569] Avg episode reward: [(0, '8629.199'), (1, '8813.018')] [2023-12-27 03:54:11,337][105620] Updated weights for policy 1, policy_version 1720628 (0.0010) [2023-12-27 03:54:11,404][105620] Updated weights for policy 1, policy_version 1720638 (0.0009) [2023-12-27 03:54:11,461][105620] Updated weights for policy 1, policy_version 1720648 (0.0009) [2023-12-27 03:54:11,630][105692] Updated weights for policy 0, policy_version 1717066 (0.0009) [2023-12-27 03:54:11,693][105692] Updated weights for policy 0, policy_version 1717076 (0.0010) [2023-12-27 03:54:11,757][105692] Updated weights for policy 0, policy_version 1717086 (0.0009) [2023-12-27 03:54:12,234][105620] Updated weights for policy 1, policy_version 1720658 (0.0009) [2023-12-27 03:54:12,301][105620] Updated weights for policy 1, policy_version 1720668 (0.0009) [2023-12-27 03:54:12,365][105620] Updated weights for policy 1, policy_version 1720678 (0.0008) [2023-12-27 03:54:12,424][105620] Updated weights for policy 1, policy_version 1720688 (0.0007) [2023-12-27 03:54:12,520][105692] Updated weights for policy 0, policy_version 1717096 (0.0006) [2023-12-27 03:54:12,578][105692] Updated weights for policy 0, policy_version 1717106 (0.0007) [2023-12-27 03:54:12,637][105692] Updated weights for policy 0, policy_version 1717116 (0.0010) [2023-12-27 03:54:13,133][105620] Updated weights for policy 1, policy_version 1720698 (0.0005) [2023-12-27 03:54:13,186][105620] Updated weights for policy 1, policy_version 1720708 (0.0005) [2023-12-27 03:54:13,241][105620] Updated weights for policy 1, policy_version 1720718 (0.0005) [2023-12-27 03:54:13,472][105692] Updated weights for policy 0, policy_version 1717126 (0.0010) [2023-12-27 03:54:13,520][105692] Updated weights for policy 0, policy_version 1717136 (0.0010) [2023-12-27 03:54:13,573][105692] Updated weights for policy 0, policy_version 1717146 (0.0010) [2023-12-27 03:54:13,861][105620] Updated weights for policy 1, policy_version 1720728 (0.0008) [2023-12-27 03:54:13,920][105620] Updated weights for policy 1, policy_version 1720738 (0.0009) [2023-12-27 03:54:13,986][105620] Updated weights for policy 1, policy_version 1720748 (0.0010) [2023-12-27 03:54:14,232][105692] Updated weights for policy 0, policy_version 1717156 (0.0008) [2023-12-27 03:54:14,290][105692] Updated weights for policy 0, policy_version 1717166 (0.0005) [2023-12-27 03:54:14,347][105692] Updated weights for policy 0, policy_version 1717176 (0.0005) [2023-12-27 03:54:14,765][105620] Updated weights for policy 1, policy_version 1720758 (0.0009) [2023-12-27 03:54:14,831][105620] Updated weights for policy 1, policy_version 1720768 (0.0008) [2023-12-27 03:54:14,899][105620] Updated weights for policy 1, policy_version 1720778 (0.0009) [2023-12-27 03:54:15,025][105692] Updated weights for policy 0, policy_version 1717186 (0.0007) [2023-12-27 03:54:15,081][105692] Updated weights for policy 0, policy_version 1717196 (0.0011) [2023-12-27 03:54:15,150][105692] Updated weights for policy 0, policy_version 1717206 (0.0011) [2023-12-27 03:54:15,211][105692] Updated weights for policy 0, policy_version 1717216 (0.0011) [2023-12-27 03:54:15,655][105620] Updated weights for policy 1, policy_version 1720788 (0.0008) [2023-12-27 03:54:15,707][105620] Updated weights for policy 1, policy_version 1720798 (0.0008) [2023-12-27 03:54:15,759][105620] Updated weights for policy 1, policy_version 1720808 (0.0008) [2023-12-27 03:54:15,933][105692] Updated weights for policy 0, policy_version 1717226 (0.0006) [2023-12-27 03:54:15,988][105692] Updated weights for policy 0, policy_version 1717236 (0.0005) [2023-12-27 03:54:16,050][105692] Updated weights for policy 0, policy_version 1717246 (0.0006) [2023-12-27 03:54:16,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19387.6, 300 sec: 19577.5). Total num frames: 880263168. Throughput: 0: 9648.4, 1: 9752.6. Samples: 880236400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:54:16,063][104569] Avg episode reward: [(0, '8257.987'), (1, '8719.716')] [2023-12-27 03:54:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001717248_439681024.pth... [2023-12-27 03:54:16,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001720816_440590336.pth... [2023-12-27 03:54:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001719696_440303616.pth [2023-12-27 03:54:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001716096_439386112.pth [2023-12-27 03:54:16,553][105620] Updated weights for policy 1, policy_version 1720819 (0.0007) [2023-12-27 03:54:16,622][105620] Updated weights for policy 1, policy_version 1720829 (0.0007) [2023-12-27 03:54:16,641][105692] Updated weights for policy 0, policy_version 1717256 (0.0005) [2023-12-27 03:54:16,686][105620] Updated weights for policy 1, policy_version 1720839 (0.0008) [2023-12-27 03:54:16,703][105692] Updated weights for policy 0, policy_version 1717266 (0.0005) [2023-12-27 03:54:16,767][105692] Updated weights for policy 0, policy_version 1717276 (0.0005) [2023-12-27 03:54:17,336][105692] Updated weights for policy 0, policy_version 1717286 (0.0007) [2023-12-27 03:54:17,386][105692] Updated weights for policy 0, policy_version 1717296 (0.0007) [2023-12-27 03:54:17,436][105692] Updated weights for policy 0, policy_version 1717306 (0.0007) [2023-12-27 03:54:17,462][105620] Updated weights for policy 1, policy_version 1720849 (0.0008) [2023-12-27 03:54:17,507][105620] Updated weights for policy 1, policy_version 1720859 (0.0009) [2023-12-27 03:54:17,554][105620] Updated weights for policy 1, policy_version 1720869 (0.0008) [2023-12-27 03:54:17,602][105620] Updated weights for policy 1, policy_version 1720879 (0.0009) [2023-12-27 03:54:18,078][105692] Updated weights for policy 0, policy_version 1717316 (0.0010) [2023-12-27 03:54:18,136][105692] Updated weights for policy 0, policy_version 1717326 (0.0009) [2023-12-27 03:54:18,194][105692] Updated weights for policy 0, policy_version 1717336 (0.0009) [2023-12-27 03:54:18,410][105620] Updated weights for policy 1, policy_version 1720889 (0.0007) [2023-12-27 03:54:18,466][105620] Updated weights for policy 1, policy_version 1720899 (0.0007) [2023-12-27 03:54:18,520][105620] Updated weights for policy 1, policy_version 1720909 (0.0006) [2023-12-27 03:54:19,027][105692] Updated weights for policy 0, policy_version 1717346 (0.0009) [2023-12-27 03:54:19,089][105692] Updated weights for policy 0, policy_version 1717356 (0.0009) [2023-12-27 03:54:19,148][105692] Updated weights for policy 0, policy_version 1717366 (0.0009) [2023-12-27 03:54:19,207][105620] Updated weights for policy 1, policy_version 1720919 (0.0007) [2023-12-27 03:54:19,210][105692] Updated weights for policy 0, policy_version 1717376 (0.0007) [2023-12-27 03:54:19,276][105620] Updated weights for policy 1, policy_version 1720929 (0.0008) [2023-12-27 03:54:19,334][105620] Updated weights for policy 1, policy_version 1720939 (0.0009) [2023-12-27 03:54:19,993][105692] Updated weights for policy 0, policy_version 1717386 (0.0010) [2023-12-27 03:54:20,059][105692] Updated weights for policy 0, policy_version 1717396 (0.0010) [2023-12-27 03:54:20,127][105692] Updated weights for policy 0, policy_version 1717406 (0.0011) [2023-12-27 03:54:20,134][105620] Updated weights for policy 1, policy_version 1720949 (0.0008) [2023-12-27 03:54:20,195][105620] Updated weights for policy 1, policy_version 1720959 (0.0010) [2023-12-27 03:54:20,259][105620] Updated weights for policy 1, policy_version 1720969 (0.0011) [2023-12-27 03:54:20,905][105692] Updated weights for policy 0, policy_version 1717416 (0.0011) [2023-12-27 03:54:20,969][105692] Updated weights for policy 0, policy_version 1717426 (0.0011) [2023-12-27 03:54:20,990][105620] Updated weights for policy 1, policy_version 1720979 (0.0009) [2023-12-27 03:54:21,035][105692] Updated weights for policy 0, policy_version 1717436 (0.0011) [2023-12-27 03:54:21,056][105620] Updated weights for policy 1, policy_version 1720989 (0.0008) [2023-12-27 03:54:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 880361472. Throughput: 0: 9626.9, 1: 9613.6. Samples: 880352580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:54:21,062][104569] Avg episode reward: [(0, '8529.077'), (1, '8985.021')] [2023-12-27 03:54:21,127][105620] Updated weights for policy 1, policy_version 1720999 (0.0008) [2023-12-27 03:54:21,811][105692] Updated weights for policy 0, policy_version 1717446 (0.0010) [2023-12-27 03:54:21,870][105692] Updated weights for policy 0, policy_version 1717456 (0.0006) [2023-12-27 03:54:21,895][105620] Updated weights for policy 1, policy_version 1721009 (0.0010) [2023-12-27 03:54:21,935][105692] Updated weights for policy 0, policy_version 1717466 (0.0006) [2023-12-27 03:54:21,962][105620] Updated weights for policy 1, policy_version 1721019 (0.0011) [2023-12-27 03:54:22,025][105620] Updated weights for policy 1, policy_version 1721029 (0.0010) [2023-12-27 03:54:22,081][105620] Updated weights for policy 1, policy_version 1721039 (0.0010) [2023-12-27 03:54:22,603][105692] Updated weights for policy 0, policy_version 1717476 (0.0008) [2023-12-27 03:54:22,666][105692] Updated weights for policy 0, policy_version 1717486 (0.0011) [2023-12-27 03:54:22,731][105692] Updated weights for policy 0, policy_version 1717496 (0.0010) [2023-12-27 03:54:22,817][105620] Updated weights for policy 1, policy_version 1721049 (0.0010) [2023-12-27 03:54:22,880][105620] Updated weights for policy 1, policy_version 1721059 (0.0010) [2023-12-27 03:54:22,939][105620] Updated weights for policy 1, policy_version 1721069 (0.0010) [2023-12-27 03:54:23,418][105692] Updated weights for policy 0, policy_version 1717506 (0.0010) [2023-12-27 03:54:23,463][105692] Updated weights for policy 0, policy_version 1717516 (0.0010) [2023-12-27 03:54:23,520][105692] Updated weights for policy 0, policy_version 1717526 (0.0011) [2023-12-27 03:54:23,574][105692] Updated weights for policy 0, policy_version 1717536 (0.0010) [2023-12-27 03:54:23,647][105620] Updated weights for policy 1, policy_version 1721079 (0.0010) [2023-12-27 03:54:23,708][105620] Updated weights for policy 1, policy_version 1721089 (0.0010) [2023-12-27 03:54:23,772][105620] Updated weights for policy 1, policy_version 1721099 (0.0010) [2023-12-27 03:54:24,246][105692] Updated weights for policy 0, policy_version 1717546 (0.0005) [2023-12-27 03:54:24,308][105692] Updated weights for policy 0, policy_version 1717556 (0.0005) [2023-12-27 03:54:24,369][105692] Updated weights for policy 0, policy_version 1717566 (0.0005) [2023-12-27 03:54:24,496][105620] Updated weights for policy 1, policy_version 1721109 (0.0010) [2023-12-27 03:54:24,544][105620] Updated weights for policy 1, policy_version 1721119 (0.0010) [2023-12-27 03:54:24,588][105620] Updated weights for policy 1, policy_version 1721129 (0.0010) [2023-12-27 03:54:24,980][105692] Updated weights for policy 0, policy_version 1717576 (0.0008) [2023-12-27 03:54:25,036][105692] Updated weights for policy 0, policy_version 1717586 (0.0011) [2023-12-27 03:54:25,091][105692] Updated weights for policy 0, policy_version 1717596 (0.0010) [2023-12-27 03:54:25,351][105620] Updated weights for policy 1, policy_version 1721139 (0.0010) [2023-12-27 03:54:25,406][105620] Updated weights for policy 1, policy_version 1721149 (0.0010) [2023-12-27 03:54:25,467][105620] Updated weights for policy 1, policy_version 1721159 (0.0010) [2023-12-27 03:54:25,699][105692] Updated weights for policy 0, policy_version 1717606 (0.0007) [2023-12-27 03:54:25,760][105692] Updated weights for policy 0, policy_version 1717616 (0.0005) [2023-12-27 03:54:25,816][105692] Updated weights for policy 0, policy_version 1717626 (0.0005) [2023-12-27 03:54:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 880459776. Throughput: 0: 9687.4, 1: 9521.5. Samples: 880468380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:54:26,063][104569] Avg episode reward: [(0, '8624.297'), (1, '9169.931')] [2023-12-27 03:54:26,189][105620] Updated weights for policy 1, policy_version 1721169 (0.0010) [2023-12-27 03:54:26,253][105620] Updated weights for policy 1, policy_version 1721179 (0.0010) [2023-12-27 03:54:26,324][105620] Updated weights for policy 1, policy_version 1721189 (0.0010) [2023-12-27 03:54:26,382][105620] Updated weights for policy 1, policy_version 1721199 (0.0010) [2023-12-27 03:54:26,401][105692] Updated weights for policy 0, policy_version 1717636 (0.0005) [2023-12-27 03:54:26,459][105692] Updated weights for policy 0, policy_version 1717646 (0.0005) [2023-12-27 03:54:26,518][105692] Updated weights for policy 0, policy_version 1717656 (0.0005) [2023-12-27 03:54:27,055][105692] Updated weights for policy 0, policy_version 1717666 (0.0005) [2023-12-27 03:54:27,102][105620] Updated weights for policy 1, policy_version 1721209 (0.0011) [2023-12-27 03:54:27,113][105692] Updated weights for policy 0, policy_version 1717676 (0.0006) [2023-12-27 03:54:27,150][105620] Updated weights for policy 1, policy_version 1721219 (0.0010) [2023-12-27 03:54:27,173][105692] Updated weights for policy 0, policy_version 1717686 (0.0006) [2023-12-27 03:54:27,198][105620] Updated weights for policy 1, policy_version 1721229 (0.0010) [2023-12-27 03:54:27,221][105692] Updated weights for policy 0, policy_version 1717696 (0.0008) [2023-12-27 03:54:27,895][105692] Updated weights for policy 0, policy_version 1717706 (0.0005) [2023-12-27 03:54:27,947][105692] Updated weights for policy 0, policy_version 1717716 (0.0005) [2023-12-27 03:54:27,960][105620] Updated weights for policy 1, policy_version 1721239 (0.0010) [2023-12-27 03:54:28,004][105692] Updated weights for policy 0, policy_version 1717726 (0.0007) [2023-12-27 03:54:28,018][105620] Updated weights for policy 1, policy_version 1721249 (0.0010) [2023-12-27 03:54:28,065][105620] Updated weights for policy 1, policy_version 1721259 (0.0010) [2023-12-27 03:54:28,672][105692] Updated weights for policy 0, policy_version 1717736 (0.0007) [2023-12-27 03:54:28,729][105692] Updated weights for policy 0, policy_version 1717746 (0.0008) [2023-12-27 03:54:28,782][105692] Updated weights for policy 0, policy_version 1717756 (0.0008) [2023-12-27 03:54:28,815][105620] Updated weights for policy 1, policy_version 1721269 (0.0010) [2023-12-27 03:54:28,874][105620] Updated weights for policy 1, policy_version 1721279 (0.0011) [2023-12-27 03:54:28,941][105620] Updated weights for policy 1, policy_version 1721289 (0.0010) [2023-12-27 03:54:29,479][105692] Updated weights for policy 0, policy_version 1717766 (0.0008) [2023-12-27 03:54:29,536][105692] Updated weights for policy 0, policy_version 1717776 (0.0008) [2023-12-27 03:54:29,598][105692] Updated weights for policy 0, policy_version 1717786 (0.0009) [2023-12-27 03:54:29,655][105620] Updated weights for policy 1, policy_version 1721299 (0.0010) [2023-12-27 03:54:29,717][105620] Updated weights for policy 1, policy_version 1721309 (0.0009) [2023-12-27 03:54:29,778][105620] Updated weights for policy 1, policy_version 1721319 (0.0009) [2023-12-27 03:54:30,373][105692] Updated weights for policy 0, policy_version 1717796 (0.0008) [2023-12-27 03:54:30,425][105692] Updated weights for policy 0, policy_version 1717806 (0.0009) [2023-12-27 03:54:30,472][105692] Updated weights for policy 0, policy_version 1717816 (0.0008) [2023-12-27 03:54:30,548][105620] Updated weights for policy 1, policy_version 1721329 (0.0009) [2023-12-27 03:54:30,605][105620] Updated weights for policy 1, policy_version 1721339 (0.0005) [2023-12-27 03:54:30,651][105620] Updated weights for policy 1, policy_version 1721349 (0.0005) [2023-12-27 03:54:30,695][105620] Updated weights for policy 1, policy_version 1721359 (0.0005) [2023-12-27 03:54:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 880558080. Throughput: 0: 9791.5, 1: 9505.5. Samples: 880530344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:54:31,062][104569] Avg episode reward: [(0, '8531.869'), (1, '9077.990')] [2023-12-27 03:54:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001717824_439828480.pth... [2023-12-27 03:54:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001721360_440729600.pth... [2023-12-27 03:54:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001720272_440451072.pth [2023-12-27 03:54:31,082][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001716672_439533568.pth [2023-12-27 03:54:31,280][105692] Updated weights for policy 0, policy_version 1717826 (0.0009) [2023-12-27 03:54:31,316][105620] Updated weights for policy 1, policy_version 1721369 (0.0007) [2023-12-27 03:54:31,337][105692] Updated weights for policy 0, policy_version 1717836 (0.0008) [2023-12-27 03:54:31,378][105620] Updated weights for policy 1, policy_version 1721379 (0.0010) [2023-12-27 03:54:31,417][105692] Updated weights for policy 0, policy_version 1717846 (0.0008) [2023-12-27 03:54:31,448][105620] Updated weights for policy 1, policy_version 1721389 (0.0007) [2023-12-27 03:54:31,467][105692] Updated weights for policy 0, policy_version 1717856 (0.0006) [2023-12-27 03:54:32,188][105692] Updated weights for policy 0, policy_version 1717866 (0.0010) [2023-12-27 03:54:32,189][105620] Updated weights for policy 1, policy_version 1721399 (0.0008) [2023-12-27 03:54:32,236][105692] Updated weights for policy 0, policy_version 1717876 (0.0010) [2023-12-27 03:54:32,249][105620] Updated weights for policy 1, policy_version 1721409 (0.0009) [2023-12-27 03:54:32,295][105692] Updated weights for policy 0, policy_version 1717886 (0.0010) [2023-12-27 03:54:32,310][105620] Updated weights for policy 1, policy_version 1721419 (0.0005) [2023-12-27 03:54:32,950][105692] Updated weights for policy 0, policy_version 1717896 (0.0006) [2023-12-27 03:54:33,015][105692] Updated weights for policy 0, policy_version 1717906 (0.0005) [2023-12-27 03:54:33,069][105692] Updated weights for policy 0, policy_version 1717916 (0.0008) [2023-12-27 03:54:33,122][105620] Updated weights for policy 1, policy_version 1721429 (0.0008) [2023-12-27 03:54:33,170][105620] Updated weights for policy 1, policy_version 1721439 (0.0005) [2023-12-27 03:54:33,232][105620] Updated weights for policy 1, policy_version 1721449 (0.0006) [2023-12-27 03:54:33,776][105620] Updated weights for policy 1, policy_version 1721459 (0.0005) [2023-12-27 03:54:33,808][105692] Updated weights for policy 0, policy_version 1717926 (0.0008) [2023-12-27 03:54:33,839][105620] Updated weights for policy 1, policy_version 1721469 (0.0005) [2023-12-27 03:54:33,864][105692] Updated weights for policy 0, policy_version 1717936 (0.0007) [2023-12-27 03:54:33,896][105620] Updated weights for policy 1, policy_version 1721479 (0.0005) [2023-12-27 03:54:33,919][105692] Updated weights for policy 0, policy_version 1717946 (0.0006) [2023-12-27 03:54:34,515][105692] Updated weights for policy 0, policy_version 1717956 (0.0006) [2023-12-27 03:54:34,575][105692] Updated weights for policy 0, policy_version 1717966 (0.0010) [2023-12-27 03:54:34,598][105620] Updated weights for policy 1, policy_version 1721489 (0.0008) [2023-12-27 03:54:34,639][105692] Updated weights for policy 0, policy_version 1717976 (0.0011) [2023-12-27 03:54:34,661][105620] Updated weights for policy 1, policy_version 1721499 (0.0008) [2023-12-27 03:54:34,730][105620] Updated weights for policy 1, policy_version 1721509 (0.0007) [2023-12-27 03:54:34,786][105620] Updated weights for policy 1, policy_version 1721519 (0.0007) [2023-12-27 03:54:35,256][105692] Updated weights for policy 0, policy_version 1717986 (0.0010) [2023-12-27 03:54:35,315][105692] Updated weights for policy 0, policy_version 1717996 (0.0005) [2023-12-27 03:54:35,379][105692] Updated weights for policy 0, policy_version 1718006 (0.0005) [2023-12-27 03:54:35,438][105692] Updated weights for policy 0, policy_version 1718016 (0.0005) [2023-12-27 03:54:35,478][105620] Updated weights for policy 1, policy_version 1721529 (0.0010) [2023-12-27 03:54:35,530][105620] Updated weights for policy 1, policy_version 1721539 (0.0011) [2023-12-27 03:54:35,558][105586] KL-divergence is very high: 149.9377 [2023-12-27 03:54:35,578][105620] Updated weights for policy 1, policy_version 1721549 (0.0010) [2023-12-27 03:54:35,945][105692] Updated weights for policy 0, policy_version 1718026 (0.0010) [2023-12-27 03:54:36,000][105692] Updated weights for policy 0, policy_version 1718036 (0.0010) [2023-12-27 03:54:36,058][105692] Updated weights for policy 0, policy_version 1718046 (0.0010) [2023-12-27 03:54:36,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 880656384. Throughput: 0: 9765.7, 1: 9485.8. Samples: 880648376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:54:36,062][104569] Avg episode reward: [(0, '8253.814'), (1, '8988.106')] [2023-12-27 03:54:36,355][105620] Updated weights for policy 1, policy_version 1721559 (0.0009) [2023-12-27 03:54:36,416][105620] Updated weights for policy 1, policy_version 1721569 (0.0008) [2023-12-27 03:54:36,477][105620] Updated weights for policy 1, policy_version 1721579 (0.0009) [2023-12-27 03:54:36,804][105692] Updated weights for policy 0, policy_version 1718056 (0.0010) [2023-12-27 03:54:36,853][105692] Updated weights for policy 0, policy_version 1718066 (0.0010) [2023-12-27 03:54:36,903][105692] Updated weights for policy 0, policy_version 1718076 (0.0009) [2023-12-27 03:54:37,249][105620] Updated weights for policy 1, policy_version 1721589 (0.0010) [2023-12-27 03:54:37,310][105620] Updated weights for policy 1, policy_version 1721599 (0.0008) [2023-12-27 03:54:37,369][105620] Updated weights for policy 1, policy_version 1721609 (0.0005) [2023-12-27 03:54:37,618][105692] Updated weights for policy 0, policy_version 1718086 (0.0009) [2023-12-27 03:54:37,678][105692] Updated weights for policy 0, policy_version 1718096 (0.0011) [2023-12-27 03:54:37,731][105692] Updated weights for policy 0, policy_version 1718106 (0.0011) [2023-12-27 03:54:38,048][105620] Updated weights for policy 1, policy_version 1721619 (0.0009) [2023-12-27 03:54:38,103][105620] Updated weights for policy 1, policy_version 1721629 (0.0010) [2023-12-27 03:54:38,147][105620] Updated weights for policy 1, policy_version 1721639 (0.0010) [2023-12-27 03:54:38,424][105692] Updated weights for policy 0, policy_version 1718116 (0.0011) [2023-12-27 03:54:38,482][105692] Updated weights for policy 0, policy_version 1718126 (0.0011) [2023-12-27 03:54:38,546][105692] Updated weights for policy 0, policy_version 1718136 (0.0011) [2023-12-27 03:54:38,803][105620] Updated weights for policy 1, policy_version 1721649 (0.0010) [2023-12-27 03:54:38,853][105620] Updated weights for policy 1, policy_version 1721659 (0.0010) [2023-12-27 03:54:38,905][105620] Updated weights for policy 1, policy_version 1721669 (0.0010) [2023-12-27 03:54:38,952][105620] Updated weights for policy 1, policy_version 1721679 (0.0007) [2023-12-27 03:54:39,305][105692] Updated weights for policy 0, policy_version 1718146 (0.0011) [2023-12-27 03:54:39,375][105692] Updated weights for policy 0, policy_version 1718156 (0.0009) [2023-12-27 03:54:39,442][105692] Updated weights for policy 0, policy_version 1718166 (0.0009) [2023-12-27 03:54:39,497][105692] Updated weights for policy 0, policy_version 1718176 (0.0008) [2023-12-27 03:54:39,626][105620] Updated weights for policy 1, policy_version 1721689 (0.0008) [2023-12-27 03:54:39,677][105620] Updated weights for policy 1, policy_version 1721699 (0.0009) [2023-12-27 03:54:39,743][105620] Updated weights for policy 1, policy_version 1721709 (0.0006) [2023-12-27 03:54:40,249][105692] Updated weights for policy 0, policy_version 1718186 (0.0010) [2023-12-27 03:54:40,309][105692] Updated weights for policy 0, policy_version 1718196 (0.0009) [2023-12-27 03:54:40,372][105692] Updated weights for policy 0, policy_version 1718206 (0.0009) [2023-12-27 03:54:40,450][105620] Updated weights for policy 1, policy_version 1721719 (0.0007) [2023-12-27 03:54:40,514][105620] Updated weights for policy 1, policy_version 1721729 (0.0005) [2023-12-27 03:54:40,573][105620] Updated weights for policy 1, policy_version 1721739 (0.0006) [2023-12-27 03:54:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 880754688. Throughput: 0: 9810.7, 1: 9543.4. Samples: 880766764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:54:41,063][104569] Avg episode reward: [(0, '8167.549'), (1, '8804.588')] [2023-12-27 03:54:41,202][105692] Updated weights for policy 0, policy_version 1718216 (0.0008) [2023-12-27 03:54:41,231][105620] Updated weights for policy 1, policy_version 1721749 (0.0008) [2023-12-27 03:54:41,266][105692] Updated weights for policy 0, policy_version 1718226 (0.0007) [2023-12-27 03:54:41,293][105620] Updated weights for policy 1, policy_version 1721759 (0.0008) [2023-12-27 03:54:41,324][105692] Updated weights for policy 0, policy_version 1718236 (0.0007) [2023-12-27 03:54:41,360][105620] Updated weights for policy 1, policy_version 1721769 (0.0007) [2023-12-27 03:54:42,010][105692] Updated weights for policy 0, policy_version 1718246 (0.0007) [2023-12-27 03:54:42,065][105692] Updated weights for policy 0, policy_version 1718256 (0.0006) [2023-12-27 03:54:42,127][105692] Updated weights for policy 0, policy_version 1718266 (0.0007) [2023-12-27 03:54:42,189][105620] Updated weights for policy 1, policy_version 1721779 (0.0010) [2023-12-27 03:54:42,246][105620] Updated weights for policy 1, policy_version 1721789 (0.0010) [2023-12-27 03:54:42,309][105620] Updated weights for policy 1, policy_version 1721799 (0.0009) [2023-12-27 03:54:42,805][105692] Updated weights for policy 0, policy_version 1718276 (0.0007) [2023-12-27 03:54:42,863][105692] Updated weights for policy 0, policy_version 1718286 (0.0010) [2023-12-27 03:54:42,914][105692] Updated weights for policy 0, policy_version 1718296 (0.0010) [2023-12-27 03:54:43,035][105620] Updated weights for policy 1, policy_version 1721809 (0.0009) [2023-12-27 03:54:43,083][105620] Updated weights for policy 1, policy_version 1721819 (0.0008) [2023-12-27 03:54:43,131][105620] Updated weights for policy 1, policy_version 1721829 (0.0008) [2023-12-27 03:54:43,186][105620] Updated weights for policy 1, policy_version 1721839 (0.0008) [2023-12-27 03:54:43,660][105692] Updated weights for policy 0, policy_version 1718306 (0.0010) [2023-12-27 03:54:43,718][105692] Updated weights for policy 0, policy_version 1718316 (0.0010) [2023-12-27 03:54:43,775][105692] Updated weights for policy 0, policy_version 1718326 (0.0010) [2023-12-27 03:54:43,834][105692] Updated weights for policy 0, policy_version 1718336 (0.0010) [2023-12-27 03:54:43,965][105620] Updated weights for policy 1, policy_version 1721849 (0.0009) [2023-12-27 03:54:44,024][105620] Updated weights for policy 1, policy_version 1721859 (0.0006) [2023-12-27 03:54:44,084][105620] Updated weights for policy 1, policy_version 1721869 (0.0006) [2023-12-27 03:54:44,463][105692] Updated weights for policy 0, policy_version 1718346 (0.0011) [2023-12-27 03:54:44,530][105692] Updated weights for policy 0, policy_version 1718356 (0.0011) [2023-12-27 03:54:44,594][105692] Updated weights for policy 0, policy_version 1718366 (0.0011) [2023-12-27 03:54:44,758][105620] Updated weights for policy 1, policy_version 1721879 (0.0007) [2023-12-27 03:54:44,822][105620] Updated weights for policy 1, policy_version 1721889 (0.0009) [2023-12-27 03:54:44,883][105620] Updated weights for policy 1, policy_version 1721899 (0.0009) [2023-12-27 03:54:45,354][105692] Updated weights for policy 0, policy_version 1718376 (0.0011) [2023-12-27 03:54:45,408][105692] Updated weights for policy 0, policy_version 1718386 (0.0011) [2023-12-27 03:54:45,473][105692] Updated weights for policy 0, policy_version 1718396 (0.0007) [2023-12-27 03:54:45,701][105620] Updated weights for policy 1, policy_version 1721909 (0.0009) [2023-12-27 03:54:45,761][105620] Updated weights for policy 1, policy_version 1721919 (0.0008) [2023-12-27 03:54:45,818][105620] Updated weights for policy 1, policy_version 1721929 (0.0008) [2023-12-27 03:54:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 880852992. Throughput: 0: 9808.0, 1: 9543.7. Samples: 880823540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:54:46,062][104569] Avg episode reward: [(0, '8625.912'), (1, '8804.291')] [2023-12-27 03:54:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001718400_439975936.pth... [2023-12-27 03:54:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001721936_440877056.pth... [2023-12-27 03:54:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001717248_439681024.pth [2023-12-27 03:54:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001720816_440590336.pth [2023-12-27 03:54:46,194][105692] Updated weights for policy 0, policy_version 1718406 (0.0008) [2023-12-27 03:54:46,238][105692] Updated weights for policy 0, policy_version 1718416 (0.0010) [2023-12-27 03:54:46,286][105692] Updated weights for policy 0, policy_version 1718426 (0.0010) [2023-12-27 03:54:46,598][105620] Updated weights for policy 1, policy_version 1721939 (0.0008) [2023-12-27 03:54:46,653][105620] Updated weights for policy 1, policy_version 1721949 (0.0008) [2023-12-27 03:54:46,720][105620] Updated weights for policy 1, policy_version 1721959 (0.0008) [2023-12-27 03:54:47,064][105692] Updated weights for policy 0, policy_version 1718436 (0.0010) [2023-12-27 03:54:47,119][105692] Updated weights for policy 0, policy_version 1718446 (0.0009) [2023-12-27 03:54:47,176][105692] Updated weights for policy 0, policy_version 1718456 (0.0006) [2023-12-27 03:54:47,375][105620] Updated weights for policy 1, policy_version 1721969 (0.0007) [2023-12-27 03:54:47,439][105620] Updated weights for policy 1, policy_version 1721979 (0.0005) [2023-12-27 03:54:47,491][105620] Updated weights for policy 1, policy_version 1721989 (0.0005) [2023-12-27 03:54:47,542][105620] Updated weights for policy 1, policy_version 1721999 (0.0005) [2023-12-27 03:54:47,721][105692] Updated weights for policy 0, policy_version 1718466 (0.0005) [2023-12-27 03:54:47,780][105692] Updated weights for policy 0, policy_version 1718476 (0.0005) [2023-12-27 03:54:47,842][105692] Updated weights for policy 0, policy_version 1718486 (0.0006) [2023-12-27 03:54:47,901][105692] Updated weights for policy 0, policy_version 1718496 (0.0008) [2023-12-27 03:54:48,203][105620] Updated weights for policy 1, policy_version 1722009 (0.0006) [2023-12-27 03:54:48,254][105620] Updated weights for policy 1, policy_version 1722019 (0.0006) [2023-12-27 03:54:48,304][105620] Updated weights for policy 1, policy_version 1722029 (0.0005) [2023-12-27 03:54:48,592][105692] Updated weights for policy 0, policy_version 1718506 (0.0006) [2023-12-27 03:54:48,646][105692] Updated weights for policy 0, policy_version 1718516 (0.0005) [2023-12-27 03:54:48,701][105692] Updated weights for policy 0, policy_version 1718526 (0.0006) [2023-12-27 03:54:49,011][105620] Updated weights for policy 1, policy_version 1722039 (0.0009) [2023-12-27 03:54:49,066][105620] Updated weights for policy 1, policy_version 1722049 (0.0011) [2023-12-27 03:54:49,131][105620] Updated weights for policy 1, policy_version 1722059 (0.0008) [2023-12-27 03:54:49,450][105692] Updated weights for policy 0, policy_version 1718536 (0.0008) [2023-12-27 03:54:49,519][105692] Updated weights for policy 0, policy_version 1718546 (0.0008) [2023-12-27 03:54:49,580][105692] Updated weights for policy 0, policy_version 1718556 (0.0008) [2023-12-27 03:54:49,769][105620] Updated weights for policy 1, policy_version 1722069 (0.0008) [2023-12-27 03:54:49,834][105620] Updated weights for policy 1, policy_version 1722079 (0.0014) [2023-12-27 03:54:49,890][105620] Updated weights for policy 1, policy_version 1722089 (0.0008) [2023-12-27 03:54:50,193][105692] Updated weights for policy 0, policy_version 1718566 (0.0007) [2023-12-27 03:54:50,253][105692] Updated weights for policy 0, policy_version 1718576 (0.0007) [2023-12-27 03:54:50,318][105692] Updated weights for policy 0, policy_version 1718586 (0.0008) [2023-12-27 03:54:50,531][105620] Updated weights for policy 1, policy_version 1722099 (0.0009) [2023-12-27 03:54:50,592][105620] Updated weights for policy 1, policy_version 1722109 (0.0010) [2023-12-27 03:54:50,659][105620] Updated weights for policy 1, policy_version 1722119 (0.0010) [2023-12-27 03:54:50,992][105692] Updated weights for policy 0, policy_version 1718596 (0.0008) [2023-12-27 03:54:51,052][105692] Updated weights for policy 0, policy_version 1718606 (0.0007) [2023-12-27 03:54:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 880951296. Throughput: 0: 9885.7, 1: 9526.3. Samples: 880942016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:54:51,063][104569] Avg episode reward: [(0, '8806.949'), (1, '8898.168')] [2023-12-27 03:54:51,116][105692] Updated weights for policy 0, policy_version 1718616 (0.0008) [2023-12-27 03:54:51,325][105620] Updated weights for policy 1, policy_version 1722129 (0.0011) [2023-12-27 03:54:51,396][105620] Updated weights for policy 1, policy_version 1722139 (0.0010) [2023-12-27 03:54:51,453][105620] Updated weights for policy 1, policy_version 1722149 (0.0005) [2023-12-27 03:54:51,512][105620] Updated weights for policy 1, policy_version 1722159 (0.0007) [2023-12-27 03:54:51,802][105692] Updated weights for policy 0, policy_version 1718626 (0.0008) [2023-12-27 03:54:51,857][105692] Updated weights for policy 0, policy_version 1718636 (0.0007) [2023-12-27 03:54:51,915][105692] Updated weights for policy 0, policy_version 1718646 (0.0006) [2023-12-27 03:54:51,974][105692] Updated weights for policy 0, policy_version 1718656 (0.0010) [2023-12-27 03:54:52,191][105620] Updated weights for policy 1, policy_version 1722169 (0.0009) [2023-12-27 03:54:52,250][105620] Updated weights for policy 1, policy_version 1722179 (0.0009) [2023-12-27 03:54:52,309][105620] Updated weights for policy 1, policy_version 1722189 (0.0009) [2023-12-27 03:54:52,743][105692] Updated weights for policy 0, policy_version 1718666 (0.0009) [2023-12-27 03:54:52,795][105692] Updated weights for policy 0, policy_version 1718676 (0.0009) [2023-12-27 03:54:52,849][105692] Updated weights for policy 0, policy_version 1718686 (0.0008) [2023-12-27 03:54:53,000][105620] Updated weights for policy 1, policy_version 1722199 (0.0006) [2023-12-27 03:54:53,063][105620] Updated weights for policy 1, policy_version 1722209 (0.0006) [2023-12-27 03:54:53,116][105620] Updated weights for policy 1, policy_version 1722219 (0.0009) [2023-12-27 03:54:53,550][105692] Updated weights for policy 0, policy_version 1718696 (0.0009) [2023-12-27 03:54:53,614][105692] Updated weights for policy 0, policy_version 1718706 (0.0009) [2023-12-27 03:54:53,660][105692] Updated weights for policy 0, policy_version 1718716 (0.0009) [2023-12-27 03:54:53,850][105620] Updated weights for policy 1, policy_version 1722229 (0.0009) [2023-12-27 03:54:53,902][105620] Updated weights for policy 1, policy_version 1722239 (0.0008) [2023-12-27 03:54:53,947][105620] Updated weights for policy 1, policy_version 1722249 (0.0005) [2023-12-27 03:54:54,444][105692] Updated weights for policy 0, policy_version 1718726 (0.0009) [2023-12-27 03:54:54,504][105692] Updated weights for policy 0, policy_version 1718736 (0.0009) [2023-12-27 03:54:54,559][105620] Updated weights for policy 1, policy_version 1722259 (0.0006) [2023-12-27 03:54:54,565][105692] Updated weights for policy 0, policy_version 1718746 (0.0009) [2023-12-27 03:54:54,631][105620] Updated weights for policy 1, policy_version 1722269 (0.0007) [2023-12-27 03:54:54,678][105620] Updated weights for policy 1, policy_version 1722279 (0.0008) [2023-12-27 03:54:55,336][105692] Updated weights for policy 0, policy_version 1718756 (0.0008) [2023-12-27 03:54:55,393][105692] Updated weights for policy 0, policy_version 1718766 (0.0009) [2023-12-27 03:54:55,427][105620] Updated weights for policy 1, policy_version 1722289 (0.0008) [2023-12-27 03:54:55,448][105692] Updated weights for policy 0, policy_version 1718776 (0.0008) [2023-12-27 03:54:55,483][105620] Updated weights for policy 1, policy_version 1722299 (0.0007) [2023-12-27 03:54:55,533][105620] Updated weights for policy 1, policy_version 1722309 (0.0009) [2023-12-27 03:54:55,587][105620] Updated weights for policy 1, policy_version 1722319 (0.0009) [2023-12-27 03:54:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 881049600. Throughput: 0: 9885.1, 1: 9661.7. Samples: 881060540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:54:56,063][104569] Avg episode reward: [(0, '9168.789'), (1, '8901.079')] [2023-12-27 03:54:56,189][105692] Updated weights for policy 0, policy_version 1718786 (0.0008) [2023-12-27 03:54:56,238][105692] Updated weights for policy 0, policy_version 1718796 (0.0011) [2023-12-27 03:54:56,285][105692] Updated weights for policy 0, policy_version 1718806 (0.0007) [2023-12-27 03:54:56,340][105692] Updated weights for policy 0, policy_version 1718816 (0.0006) [2023-12-27 03:54:56,356][105620] Updated weights for policy 1, policy_version 1722329 (0.0007) [2023-12-27 03:54:56,407][105620] Updated weights for policy 1, policy_version 1722339 (0.0008) [2023-12-27 03:54:56,453][105620] Updated weights for policy 1, policy_version 1722349 (0.0008) [2023-12-27 03:54:57,046][105692] Updated weights for policy 0, policy_version 1718826 (0.0006) [2023-12-27 03:54:57,113][105692] Updated weights for policy 0, policy_version 1718836 (0.0008) [2023-12-27 03:54:57,164][105692] Updated weights for policy 0, policy_version 1718846 (0.0010) [2023-12-27 03:54:57,266][105620] Updated weights for policy 1, policy_version 1722359 (0.0008) [2023-12-27 03:54:57,332][105620] Updated weights for policy 1, policy_version 1722369 (0.0008) [2023-12-27 03:54:57,387][105620] Updated weights for policy 1, policy_version 1722379 (0.0009) [2023-12-27 03:54:57,853][105692] Updated weights for policy 0, policy_version 1718856 (0.0007) [2023-12-27 03:54:57,914][105692] Updated weights for policy 0, policy_version 1718866 (0.0005) [2023-12-27 03:54:57,962][105692] Updated weights for policy 0, policy_version 1718876 (0.0005) [2023-12-27 03:54:58,054][105620] Updated weights for policy 1, policy_version 1722389 (0.0009) [2023-12-27 03:54:58,103][105620] Updated weights for policy 1, policy_version 1722399 (0.0006) [2023-12-27 03:54:58,157][105620] Updated weights for policy 1, policy_version 1722409 (0.0006) [2023-12-27 03:54:58,528][105692] Updated weights for policy 0, policy_version 1718886 (0.0008) [2023-12-27 03:54:58,592][105692] Updated weights for policy 0, policy_version 1718896 (0.0007) [2023-12-27 03:54:58,659][105692] Updated weights for policy 0, policy_version 1718906 (0.0009) [2023-12-27 03:54:59,003][105620] Updated weights for policy 1, policy_version 1722419 (0.0008) [2023-12-27 03:54:59,059][105620] Updated weights for policy 1, policy_version 1722429 (0.0008) [2023-12-27 03:54:59,121][105620] Updated weights for policy 1, policy_version 1722439 (0.0008) [2023-12-27 03:54:59,414][105692] Updated weights for policy 0, policy_version 1718916 (0.0008) [2023-12-27 03:54:59,459][105692] Updated weights for policy 0, policy_version 1718926 (0.0010) [2023-12-27 03:54:59,504][105692] Updated weights for policy 0, policy_version 1718936 (0.0010) [2023-12-27 03:54:59,883][105620] Updated weights for policy 1, policy_version 1722449 (0.0007) [2023-12-27 03:54:59,940][105620] Updated weights for policy 1, policy_version 1722459 (0.0008) [2023-12-27 03:55:00,022][105620] Updated weights for policy 1, policy_version 1722469 (0.0008) [2023-12-27 03:55:00,079][105620] Updated weights for policy 1, policy_version 1722479 (0.0010) [2023-12-27 03:55:00,292][105692] Updated weights for policy 0, policy_version 1718946 (0.0010) [2023-12-27 03:55:00,344][105692] Updated weights for policy 0, policy_version 1718956 (0.0009) [2023-12-27 03:55:00,400][105692] Updated weights for policy 0, policy_version 1718966 (0.0009) [2023-12-27 03:55:00,450][105692] Updated weights for policy 0, policy_version 1718976 (0.0008) [2023-12-27 03:55:00,776][105620] Updated weights for policy 1, policy_version 1722489 (0.0009) [2023-12-27 03:55:00,845][105620] Updated weights for policy 1, policy_version 1722499 (0.0005) [2023-12-27 03:55:00,906][105620] Updated weights for policy 1, policy_version 1722509 (0.0005) [2023-12-27 03:55:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 881147904. Throughput: 0: 10001.1, 1: 9617.5. Samples: 881119232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:55:01,063][104569] Avg episode reward: [(0, '8986.019'), (1, '8915.723')] [2023-12-27 03:55:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001718976_440123392.pth... [2023-12-27 03:55:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001722512_441024512.pth... [2023-12-27 03:55:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001717824_439828480.pth [2023-12-27 03:55:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001721360_440729600.pth [2023-12-27 03:55:01,154][105692] Updated weights for policy 0, policy_version 1718986 (0.0007) [2023-12-27 03:55:01,218][105692] Updated weights for policy 0, policy_version 1718996 (0.0011) [2023-12-27 03:55:01,285][105692] Updated weights for policy 0, policy_version 1719006 (0.0011) [2023-12-27 03:55:01,562][105620] Updated weights for policy 1, policy_version 1722519 (0.0005) [2023-12-27 03:55:01,629][105620] Updated weights for policy 1, policy_version 1722529 (0.0007) [2023-12-27 03:55:01,689][105620] Updated weights for policy 1, policy_version 1722539 (0.0009) [2023-12-27 03:55:01,999][105692] Updated weights for policy 0, policy_version 1719016 (0.0010) [2023-12-27 03:55:02,054][105692] Updated weights for policy 0, policy_version 1719026 (0.0010) [2023-12-27 03:55:02,120][105692] Updated weights for policy 0, policy_version 1719036 (0.0011) [2023-12-27 03:55:02,287][105620] Updated weights for policy 1, policy_version 1722549 (0.0008) [2023-12-27 03:55:02,344][105620] Updated weights for policy 1, policy_version 1722559 (0.0006) [2023-12-27 03:55:02,396][105620] Updated weights for policy 1, policy_version 1722569 (0.0006) [2023-12-27 03:55:02,801][105692] Updated weights for policy 0, policy_version 1719046 (0.0010) [2023-12-27 03:55:02,865][105692] Updated weights for policy 0, policy_version 1719056 (0.0006) [2023-12-27 03:55:02,933][105692] Updated weights for policy 0, policy_version 1719066 (0.0005) [2023-12-27 03:55:02,964][105620] Updated weights for policy 1, policy_version 1722579 (0.0007) [2023-12-27 03:55:03,029][105620] Updated weights for policy 1, policy_version 1722589 (0.0006) [2023-12-27 03:55:03,089][105620] Updated weights for policy 1, policy_version 1722599 (0.0006) [2023-12-27 03:55:03,454][105692] Updated weights for policy 0, policy_version 1719076 (0.0005) [2023-12-27 03:55:03,506][105692] Updated weights for policy 0, policy_version 1719086 (0.0005) [2023-12-27 03:55:03,556][105692] Updated weights for policy 0, policy_version 1719096 (0.0005) [2023-12-27 03:55:03,773][105620] Updated weights for policy 1, policy_version 1722609 (0.0006) [2023-12-27 03:55:03,840][105620] Updated weights for policy 1, policy_version 1722619 (0.0010) [2023-12-27 03:55:03,896][105620] Updated weights for policy 1, policy_version 1722629 (0.0007) [2023-12-27 03:55:03,956][105620] Updated weights for policy 1, policy_version 1722639 (0.0007) [2023-12-27 03:55:04,147][105692] Updated weights for policy 0, policy_version 1719106 (0.0006) [2023-12-27 03:55:04,204][105692] Updated weights for policy 0, policy_version 1719116 (0.0011) [2023-12-27 03:55:04,256][105692] Updated weights for policy 0, policy_version 1719126 (0.0010) [2023-12-27 03:55:04,320][105692] Updated weights for policy 0, policy_version 1719136 (0.0010) [2023-12-27 03:55:04,580][105620] Updated weights for policy 1, policy_version 1722649 (0.0007) [2023-12-27 03:55:04,635][105620] Updated weights for policy 1, policy_version 1722659 (0.0010) [2023-12-27 03:55:04,693][105620] Updated weights for policy 1, policy_version 1722669 (0.0010) [2023-12-27 03:55:05,083][105692] Updated weights for policy 0, policy_version 1719146 (0.0010) [2023-12-27 03:55:05,138][105692] Updated weights for policy 0, policy_version 1719156 (0.0010) [2023-12-27 03:55:05,194][105692] Updated weights for policy 0, policy_version 1719166 (0.0010) [2023-12-27 03:55:05,353][105620] Updated weights for policy 1, policy_version 1722679 (0.0009) [2023-12-27 03:55:05,408][105620] Updated weights for policy 1, policy_version 1722689 (0.0005) [2023-12-27 03:55:05,470][105620] Updated weights for policy 1, policy_version 1722699 (0.0008) [2023-12-27 03:55:05,849][105692] Updated weights for policy 0, policy_version 1719176 (0.0010) [2023-12-27 03:55:05,914][105692] Updated weights for policy 0, policy_version 1719186 (0.0010) [2023-12-27 03:55:05,984][105692] Updated weights for policy 0, policy_version 1719196 (0.0011) [2023-12-27 03:55:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 881254400. Throughput: 0: 9983.0, 1: 9776.6. Samples: 881241760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 03:55:06,062][104569] Avg episode reward: [(0, '8436.663'), (1, '8362.655')] [2023-12-27 03:55:06,088][105620] Updated weights for policy 1, policy_version 1722709 (0.0008) [2023-12-27 03:55:06,156][105620] Updated weights for policy 1, policy_version 1722719 (0.0009) [2023-12-27 03:55:06,210][105620] Updated weights for policy 1, policy_version 1722729 (0.0011) [2023-12-27 03:55:06,731][105692] Updated weights for policy 0, policy_version 1719206 (0.0009) [2023-12-27 03:55:06,779][105692] Updated weights for policy 0, policy_version 1719216 (0.0007) [2023-12-27 03:55:06,827][105692] Updated weights for policy 0, policy_version 1719226 (0.0008) [2023-12-27 03:55:06,927][105620] Updated weights for policy 1, policy_version 1722739 (0.0010) [2023-12-27 03:55:07,000][105620] Updated weights for policy 1, policy_version 1722749 (0.0009) [2023-12-27 03:55:07,067][105620] Updated weights for policy 1, policy_version 1722759 (0.0010) [2023-12-27 03:55:07,603][105692] Updated weights for policy 0, policy_version 1719236 (0.0007) [2023-12-27 03:55:07,648][105692] Updated weights for policy 0, policy_version 1719246 (0.0005) [2023-12-27 03:55:07,701][105692] Updated weights for policy 0, policy_version 1719256 (0.0010) [2023-12-27 03:55:07,739][105620] Updated weights for policy 1, policy_version 1722769 (0.0010) [2023-12-27 03:55:07,804][105620] Updated weights for policy 1, policy_version 1722779 (0.0005) [2023-12-27 03:55:07,868][105620] Updated weights for policy 1, policy_version 1722789 (0.0005) [2023-12-27 03:55:07,931][105620] Updated weights for policy 1, policy_version 1722799 (0.0005) [2023-12-27 03:55:08,321][105692] Updated weights for policy 0, policy_version 1719266 (0.0007) [2023-12-27 03:55:08,380][105692] Updated weights for policy 0, policy_version 1719276 (0.0007) [2023-12-27 03:55:08,439][105692] Updated weights for policy 0, policy_version 1719286 (0.0006) [2023-12-27 03:55:08,499][105692] Updated weights for policy 0, policy_version 1719296 (0.0005) [2023-12-27 03:55:08,504][105620] Updated weights for policy 1, policy_version 1722809 (0.0010) [2023-12-27 03:55:08,566][105620] Updated weights for policy 1, policy_version 1722819 (0.0010) [2023-12-27 03:55:08,621][105620] Updated weights for policy 1, policy_version 1722829 (0.0010) [2023-12-27 03:55:09,177][105692] Updated weights for policy 0, policy_version 1719306 (0.0009) [2023-12-27 03:55:09,245][105692] Updated weights for policy 0, policy_version 1719316 (0.0010) [2023-12-27 03:55:09,301][105692] Updated weights for policy 0, policy_version 1719326 (0.0011) [2023-12-27 03:55:09,375][105620] Updated weights for policy 1, policy_version 1722839 (0.0012) [2023-12-27 03:55:09,448][105620] Updated weights for policy 1, policy_version 1722849 (0.0011) [2023-12-27 03:55:09,497][105620] Updated weights for policy 1, policy_version 1722859 (0.0011) [2023-12-27 03:55:10,045][105692] Updated weights for policy 0, policy_version 1719336 (0.0011) [2023-12-27 03:55:10,104][105692] Updated weights for policy 0, policy_version 1719346 (0.0010) [2023-12-27 03:55:10,171][105692] Updated weights for policy 0, policy_version 1719356 (0.0010) [2023-12-27 03:55:10,283][105620] Updated weights for policy 1, policy_version 1722869 (0.0011) [2023-12-27 03:55:10,350][105620] Updated weights for policy 1, policy_version 1722879 (0.0010) [2023-12-27 03:55:10,414][105620] Updated weights for policy 1, policy_version 1722889 (0.0009) [2023-12-27 03:55:10,926][105692] Updated weights for policy 0, policy_version 1719366 (0.0010) [2023-12-27 03:55:10,991][105692] Updated weights for policy 0, policy_version 1719376 (0.0007) [2023-12-27 03:55:11,057][105692] Updated weights for policy 0, policy_version 1719386 (0.0010) [2023-12-27 03:55:11,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 881344512. Throughput: 0: 9971.2, 1: 9846.9. Samples: 881360196. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:55:11,063][104569] Avg episode reward: [(0, '8620.343'), (1, '8439.508')] [2023-12-27 03:55:11,144][105620] Updated weights for policy 1, policy_version 1722899 (0.0011) [2023-12-27 03:55:11,210][105620] Updated weights for policy 1, policy_version 1722909 (0.0009) [2023-12-27 03:55:11,275][105620] Updated weights for policy 1, policy_version 1722919 (0.0009) [2023-12-27 03:55:11,835][105692] Updated weights for policy 0, policy_version 1719396 (0.0007) [2023-12-27 03:55:11,898][105692] Updated weights for policy 0, policy_version 1719406 (0.0006) [2023-12-27 03:55:11,957][105692] Updated weights for policy 0, policy_version 1719416 (0.0006) [2023-12-27 03:55:12,026][105620] Updated weights for policy 1, policy_version 1722929 (0.0011) [2023-12-27 03:55:12,086][105620] Updated weights for policy 1, policy_version 1722939 (0.0009) [2023-12-27 03:55:12,145][105620] Updated weights for policy 1, policy_version 1722949 (0.0009) [2023-12-27 03:55:12,203][105620] Updated weights for policy 1, policy_version 1722959 (0.0010) [2023-12-27 03:55:12,594][105692] Updated weights for policy 0, policy_version 1719426 (0.0007) [2023-12-27 03:55:12,643][105692] Updated weights for policy 0, policy_version 1719436 (0.0009) [2023-12-27 03:55:12,698][105692] Updated weights for policy 0, policy_version 1719446 (0.0009) [2023-12-27 03:55:12,762][105692] Updated weights for policy 0, policy_version 1719456 (0.0009) [2023-12-27 03:55:13,007][105620] Updated weights for policy 1, policy_version 1722969 (0.0007) [2023-12-27 03:55:13,068][105620] Updated weights for policy 1, policy_version 1722979 (0.0006) [2023-12-27 03:55:13,125][105620] Updated weights for policy 1, policy_version 1722989 (0.0006) [2023-12-27 03:55:13,471][105692] Updated weights for policy 0, policy_version 1719466 (0.0008) [2023-12-27 03:55:13,521][105692] Updated weights for policy 0, policy_version 1719476 (0.0008) [2023-12-27 03:55:13,582][105692] Updated weights for policy 0, policy_version 1719487 (0.0011) [2023-12-27 03:55:13,735][105620] Updated weights for policy 1, policy_version 1722999 (0.0007) [2023-12-27 03:55:13,798][105620] Updated weights for policy 1, policy_version 1723009 (0.0008) [2023-12-27 03:55:13,869][105620] Updated weights for policy 1, policy_version 1723019 (0.0009) [2023-12-27 03:55:14,305][105692] Updated weights for policy 0, policy_version 1719497 (0.0006) [2023-12-27 03:55:14,360][105692] Updated weights for policy 0, policy_version 1719507 (0.0005) [2023-12-27 03:55:14,408][105692] Updated weights for policy 0, policy_version 1719517 (0.0007) [2023-12-27 03:55:14,568][105620] Updated weights for policy 1, policy_version 1723029 (0.0010) [2023-12-27 03:55:14,629][105620] Updated weights for policy 1, policy_version 1723039 (0.0008) [2023-12-27 03:55:14,688][105620] Updated weights for policy 1, policy_version 1723049 (0.0005) [2023-12-27 03:55:15,047][105692] Updated weights for policy 0, policy_version 1719527 (0.0006) [2023-12-27 03:55:15,108][105692] Updated weights for policy 0, policy_version 1719537 (0.0010) [2023-12-27 03:55:15,171][105692] Updated weights for policy 0, policy_version 1719547 (0.0006) [2023-12-27 03:55:15,269][105620] Updated weights for policy 1, policy_version 1723059 (0.0005) [2023-12-27 03:55:15,338][105620] Updated weights for policy 1, policy_version 1723069 (0.0006) [2023-12-27 03:55:15,401][105620] Updated weights for policy 1, policy_version 1723079 (0.0006) [2023-12-27 03:55:15,834][105692] Updated weights for policy 0, policy_version 1719557 (0.0007) [2023-12-27 03:55:15,890][105692] Updated weights for policy 0, policy_version 1719567 (0.0008) [2023-12-27 03:55:15,945][105692] Updated weights for policy 0, policy_version 1719577 (0.0008) [2023-12-27 03:55:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 881451008. Throughput: 0: 9869.7, 1: 9846.0. Samples: 881417552. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:55:16,063][104569] Avg episode reward: [(0, '8712.003'), (1, '8527.653')] [2023-12-27 03:55:16,065][105620] Updated weights for policy 1, policy_version 1723089 (0.0008) [2023-12-27 03:55:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001719584_440279040.pth... [2023-12-27 03:55:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001718400_439975936.pth [2023-12-27 03:55:16,127][105620] Updated weights for policy 1, policy_version 1723099 (0.0008) [2023-12-27 03:55:16,191][105620] Updated weights for policy 1, policy_version 1723109 (0.0007) [2023-12-27 03:55:16,251][105620] Updated weights for policy 1, policy_version 1723119 (0.0007) [2023-12-27 03:55:16,256][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001723120_441180160.pth... [2023-12-27 03:55:16,261][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001721936_440877056.pth [2023-12-27 03:55:16,673][105692] Updated weights for policy 0, policy_version 1719587 (0.0007) [2023-12-27 03:55:16,723][105692] Updated weights for policy 0, policy_version 1719597 (0.0006) [2023-12-27 03:55:16,776][105692] Updated weights for policy 0, policy_version 1719607 (0.0006) [2023-12-27 03:55:16,971][105620] Updated weights for policy 1, policy_version 1723129 (0.0010) [2023-12-27 03:55:17,030][105620] Updated weights for policy 1, policy_version 1723139 (0.0010) [2023-12-27 03:55:17,078][105620] Updated weights for policy 1, policy_version 1723149 (0.0010) [2023-12-27 03:55:17,477][105692] Updated weights for policy 0, policy_version 1719617 (0.0006) [2023-12-27 03:55:17,536][105692] Updated weights for policy 0, policy_version 1719627 (0.0007) [2023-12-27 03:55:17,588][105692] Updated weights for policy 0, policy_version 1719637 (0.0009) [2023-12-27 03:55:17,650][105692] Updated weights for policy 0, policy_version 1719647 (0.0008) [2023-12-27 03:55:17,835][105620] Updated weights for policy 1, policy_version 1723159 (0.0007) [2023-12-27 03:55:17,895][105620] Updated weights for policy 1, policy_version 1723169 (0.0005) [2023-12-27 03:55:17,946][105620] Updated weights for policy 1, policy_version 1723179 (0.0005) [2023-12-27 03:55:18,294][105692] Updated weights for policy 0, policy_version 1719657 (0.0006) [2023-12-27 03:55:18,362][105692] Updated weights for policy 0, policy_version 1719667 (0.0007) [2023-12-27 03:55:18,425][105692] Updated weights for policy 0, policy_version 1719677 (0.0011) [2023-12-27 03:55:18,619][105620] Updated weights for policy 1, policy_version 1723189 (0.0007) [2023-12-27 03:55:18,676][105620] Updated weights for policy 1, policy_version 1723199 (0.0006) [2023-12-27 03:55:18,730][105620] Updated weights for policy 1, policy_version 1723209 (0.0006) [2023-12-27 03:55:19,061][105692] Updated weights for policy 0, policy_version 1719687 (0.0011) [2023-12-27 03:55:19,110][105692] Updated weights for policy 0, policy_version 1719697 (0.0008) [2023-12-27 03:55:19,162][105692] Updated weights for policy 0, policy_version 1719707 (0.0010) [2023-12-27 03:55:19,489][105620] Updated weights for policy 1, policy_version 1723219 (0.0006) [2023-12-27 03:55:19,557][105620] Updated weights for policy 1, policy_version 1723229 (0.0007) [2023-12-27 03:55:19,617][105620] Updated weights for policy 1, policy_version 1723239 (0.0006) [2023-12-27 03:55:19,994][105692] Updated weights for policy 0, policy_version 1719717 (0.0010) [2023-12-27 03:55:20,055][105692] Updated weights for policy 0, policy_version 1719727 (0.0008) [2023-12-27 03:55:20,113][105692] Updated weights for policy 0, policy_version 1719737 (0.0008) [2023-12-27 03:55:20,337][105620] Updated weights for policy 1, policy_version 1723249 (0.0008) [2023-12-27 03:55:20,395][105620] Updated weights for policy 1, policy_version 1723259 (0.0010) [2023-12-27 03:55:20,457][105620] Updated weights for policy 1, policy_version 1723269 (0.0010) [2023-12-27 03:55:20,510][105620] Updated weights for policy 1, policy_version 1723279 (0.0009) [2023-12-27 03:55:20,784][105692] Updated weights for policy 0, policy_version 1719747 (0.0009) [2023-12-27 03:55:20,848][105692] Updated weights for policy 0, policy_version 1719757 (0.0009) [2023-12-27 03:55:20,914][105692] Updated weights for policy 0, policy_version 1719767 (0.0010) [2023-12-27 03:55:21,062][104569] Fps is (10 sec: 20480.7, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 881549312. Throughput: 0: 9931.7, 1: 9848.9. Samples: 881538504. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:55:21,062][104569] Avg episode reward: [(0, '8256.934'), (1, '8709.122')] [2023-12-27 03:55:21,341][105620] Updated weights for policy 1, policy_version 1723289 (0.0011) [2023-12-27 03:55:21,414][105620] Updated weights for policy 1, policy_version 1723299 (0.0011) [2023-12-27 03:55:21,475][105620] Updated weights for policy 1, policy_version 1723309 (0.0011) [2023-12-27 03:55:21,708][105692] Updated weights for policy 0, policy_version 1719777 (0.0009) [2023-12-27 03:55:21,783][105692] Updated weights for policy 0, policy_version 1719787 (0.0009) [2023-12-27 03:55:21,847][105692] Updated weights for policy 0, policy_version 1719797 (0.0008) [2023-12-27 03:55:21,911][105692] Updated weights for policy 0, policy_version 1719807 (0.0008) [2023-12-27 03:55:22,244][105620] Updated weights for policy 1, policy_version 1723319 (0.0011) [2023-12-27 03:55:22,303][105620] Updated weights for policy 1, policy_version 1723329 (0.0011) [2023-12-27 03:55:22,371][105620] Updated weights for policy 1, policy_version 1723339 (0.0010) [2023-12-27 03:55:22,683][105692] Updated weights for policy 0, policy_version 1719817 (0.0008) [2023-12-27 03:55:22,747][105692] Updated weights for policy 0, policy_version 1719827 (0.0008) [2023-12-27 03:55:22,803][105692] Updated weights for policy 0, policy_version 1719837 (0.0008) [2023-12-27 03:55:23,128][105620] Updated weights for policy 1, policy_version 1723349 (0.0011) [2023-12-27 03:55:23,187][105620] Updated weights for policy 1, policy_version 1723359 (0.0010) [2023-12-27 03:55:23,239][105620] Updated weights for policy 1, policy_version 1723369 (0.0010) [2023-12-27 03:55:23,556][105692] Updated weights for policy 0, policy_version 1719847 (0.0007) [2023-12-27 03:55:23,601][105692] Updated weights for policy 0, policy_version 1719857 (0.0008) [2023-12-27 03:55:23,645][105692] Updated weights for policy 0, policy_version 1719867 (0.0007) [2023-12-27 03:55:23,989][105620] Updated weights for policy 1, policy_version 1723379 (0.0010) [2023-12-27 03:55:24,047][105620] Updated weights for policy 1, policy_version 1723389 (0.0010) [2023-12-27 03:55:24,109][105620] Updated weights for policy 1, policy_version 1723399 (0.0010) [2023-12-27 03:55:24,445][105692] Updated weights for policy 0, policy_version 1719877 (0.0008) [2023-12-27 03:55:24,498][105692] Updated weights for policy 0, policy_version 1719887 (0.0009) [2023-12-27 03:55:24,546][105692] Updated weights for policy 0, policy_version 1719897 (0.0008) [2023-12-27 03:55:24,778][105620] Updated weights for policy 1, policy_version 1723409 (0.0010) [2023-12-27 03:55:24,826][105620] Updated weights for policy 1, policy_version 1723419 (0.0010) [2023-12-27 03:55:24,869][105620] Updated weights for policy 1, policy_version 1723429 (0.0010) [2023-12-27 03:55:24,923][105620] Updated weights for policy 1, policy_version 1723439 (0.0010) [2023-12-27 03:55:25,413][105692] Updated weights for policy 0, policy_version 1719907 (0.0008) [2023-12-27 03:55:25,471][105692] Updated weights for policy 0, policy_version 1719917 (0.0006) [2023-12-27 03:55:25,514][105620] Updated weights for policy 1, policy_version 1723449 (0.0006) [2023-12-27 03:55:25,529][105692] Updated weights for policy 0, policy_version 1719927 (0.0006) [2023-12-27 03:55:25,578][105620] Updated weights for policy 1, policy_version 1723459 (0.0006) [2023-12-27 03:55:25,628][105620] Updated weights for policy 1, policy_version 1723469 (0.0007) [2023-12-27 03:55:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 881639424. Throughput: 0: 9812.7, 1: 9808.4. Samples: 881649712. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:55:26,063][104569] Avg episode reward: [(0, '8348.888'), (1, '8896.500')] [2023-12-27 03:55:26,207][105692] Updated weights for policy 0, policy_version 1719937 (0.0006) [2023-12-27 03:55:26,258][105692] Updated weights for policy 0, policy_version 1719947 (0.0008) [2023-12-27 03:55:26,306][105692] Updated weights for policy 0, policy_version 1719957 (0.0008) [2023-12-27 03:55:26,356][105620] Updated weights for policy 1, policy_version 1723479 (0.0010) [2023-12-27 03:55:26,358][105692] Updated weights for policy 0, policy_version 1719967 (0.0007) [2023-12-27 03:55:26,407][105620] Updated weights for policy 1, policy_version 1723489 (0.0010) [2023-12-27 03:55:26,469][105620] Updated weights for policy 1, policy_version 1723499 (0.0010) [2023-12-27 03:55:27,160][105620] Updated weights for policy 1, policy_version 1723509 (0.0008) [2023-12-27 03:55:27,166][105692] Updated weights for policy 0, policy_version 1719977 (0.0009) [2023-12-27 03:55:27,215][105620] Updated weights for policy 1, policy_version 1723519 (0.0005) [2023-12-27 03:55:27,218][105692] Updated weights for policy 0, policy_version 1719987 (0.0009) [2023-12-27 03:55:27,269][105620] Updated weights for policy 1, policy_version 1723529 (0.0005) [2023-12-27 03:55:27,277][105692] Updated weights for policy 0, policy_version 1719997 (0.0009) [2023-12-27 03:55:27,839][105620] Updated weights for policy 1, policy_version 1723539 (0.0005) [2023-12-27 03:55:27,887][105620] Updated weights for policy 1, policy_version 1723549 (0.0005) [2023-12-27 03:55:27,932][105620] Updated weights for policy 1, policy_version 1723559 (0.0009) [2023-12-27 03:55:28,133][105692] Updated weights for policy 0, policy_version 1720007 (0.0008) [2023-12-27 03:55:28,192][105692] Updated weights for policy 0, policy_version 1720017 (0.0008) [2023-12-27 03:55:28,236][105692] Updated weights for policy 0, policy_version 1720027 (0.0007) [2023-12-27 03:55:28,656][105620] Updated weights for policy 1, policy_version 1723569 (0.0010) [2023-12-27 03:55:28,717][105620] Updated weights for policy 1, policy_version 1723579 (0.0010) [2023-12-27 03:55:28,775][105620] Updated weights for policy 1, policy_version 1723589 (0.0010) [2023-12-27 03:55:28,836][105620] Updated weights for policy 1, policy_version 1723599 (0.0010) [2023-12-27 03:55:28,997][105692] Updated weights for policy 0, policy_version 1720037 (0.0008) [2023-12-27 03:55:29,045][105692] Updated weights for policy 0, policy_version 1720047 (0.0008) [2023-12-27 03:55:29,094][105692] Updated weights for policy 0, policy_version 1720057 (0.0008) [2023-12-27 03:55:29,577][105620] Updated weights for policy 1, policy_version 1723609 (0.0010) [2023-12-27 03:55:29,645][105620] Updated weights for policy 1, policy_version 1723619 (0.0010) [2023-12-27 03:55:29,696][105620] Updated weights for policy 1, policy_version 1723629 (0.0010) [2023-12-27 03:55:29,843][105692] Updated weights for policy 0, policy_version 1720067 (0.0008) [2023-12-27 03:55:29,916][105692] Updated weights for policy 0, policy_version 1720077 (0.0006) [2023-12-27 03:55:29,988][105692] Updated weights for policy 0, policy_version 1720087 (0.0008) [2023-12-27 03:55:30,348][105620] Updated weights for policy 1, policy_version 1723639 (0.0008) [2023-12-27 03:55:30,412][105620] Updated weights for policy 1, policy_version 1723649 (0.0005) [2023-12-27 03:55:30,477][105620] Updated weights for policy 1, policy_version 1723659 (0.0007) [2023-12-27 03:55:30,600][105692] Updated weights for policy 0, policy_version 1720097 (0.0006) [2023-12-27 03:55:30,653][105692] Updated weights for policy 0, policy_version 1720107 (0.0005) [2023-12-27 03:55:30,711][105692] Updated weights for policy 0, policy_version 1720117 (0.0009) [2023-12-27 03:55:30,758][105692] Updated weights for policy 0, policy_version 1720127 (0.0010) [2023-12-27 03:55:31,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19660.7, 300 sec: 19494.2). Total num frames: 881737728. Throughput: 0: 9770.5, 1: 9877.3. Samples: 881707696. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:55:31,063][104569] Avg episode reward: [(0, '8623.741'), (1, '8713.254')] [2023-12-27 03:55:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001720128_440418304.pth... [2023-12-27 03:55:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001723664_441319424.pth... [2023-12-27 03:55:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001718976_440123392.pth [2023-12-27 03:55:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001722512_441024512.pth [2023-12-27 03:55:31,135][105620] Updated weights for policy 1, policy_version 1723669 (0.0009) [2023-12-27 03:55:31,202][105620] Updated weights for policy 1, policy_version 1723679 (0.0011) [2023-12-27 03:55:31,268][105620] Updated weights for policy 1, policy_version 1723689 (0.0011) [2023-12-27 03:55:31,450][105692] Updated weights for policy 0, policy_version 1720137 (0.0011) [2023-12-27 03:55:31,496][105692] Updated weights for policy 0, policy_version 1720147 (0.0010) [2023-12-27 03:55:31,544][105692] Updated weights for policy 0, policy_version 1720157 (0.0010) [2023-12-27 03:55:32,033][105620] Updated weights for policy 1, policy_version 1723699 (0.0011) [2023-12-27 03:55:32,099][105620] Updated weights for policy 1, policy_version 1723709 (0.0010) [2023-12-27 03:55:32,160][105620] Updated weights for policy 1, policy_version 1723719 (0.0008) [2023-12-27 03:55:32,299][105692] Updated weights for policy 0, policy_version 1720167 (0.0010) [2023-12-27 03:55:32,352][105692] Updated weights for policy 0, policy_version 1720177 (0.0008) [2023-12-27 03:55:32,412][105692] Updated weights for policy 0, policy_version 1720187 (0.0009) [2023-12-27 03:55:32,860][105620] Updated weights for policy 1, policy_version 1723729 (0.0009) [2023-12-27 03:55:32,914][105620] Updated weights for policy 1, policy_version 1723739 (0.0007) [2023-12-27 03:55:32,961][105620] Updated weights for policy 1, policy_version 1723749 (0.0008) [2023-12-27 03:55:33,027][105620] Updated weights for policy 1, policy_version 1723759 (0.0010) [2023-12-27 03:55:33,150][105692] Updated weights for policy 0, policy_version 1720197 (0.0007) [2023-12-27 03:55:33,207][105692] Updated weights for policy 0, policy_version 1720207 (0.0005) [2023-12-27 03:55:33,263][105692] Updated weights for policy 0, policy_version 1720217 (0.0005) [2023-12-27 03:55:33,810][105620] Updated weights for policy 1, policy_version 1723769 (0.0009) [2023-12-27 03:55:33,874][105620] Updated weights for policy 1, policy_version 1723779 (0.0010) [2023-12-27 03:55:33,907][105692] Updated weights for policy 0, policy_version 1720227 (0.0005) [2023-12-27 03:55:33,934][105620] Updated weights for policy 1, policy_version 1723790 (0.0009) [2023-12-27 03:55:33,952][105692] Updated weights for policy 0, policy_version 1720237 (0.0005) [2023-12-27 03:55:34,013][105692] Updated weights for policy 0, policy_version 1720247 (0.0008) [2023-12-27 03:55:34,717][105692] Updated weights for policy 0, policy_version 1720257 (0.0006) [2023-12-27 03:55:34,719][105620] Updated weights for policy 1, policy_version 1723800 (0.0009) [2023-12-27 03:55:34,772][105620] Updated weights for policy 1, policy_version 1723810 (0.0008) [2023-12-27 03:55:34,774][105692] Updated weights for policy 0, policy_version 1720267 (0.0006) [2023-12-27 03:55:34,821][105620] Updated weights for policy 1, policy_version 1723820 (0.0007) [2023-12-27 03:55:34,827][105692] Updated weights for policy 0, policy_version 1720277 (0.0006) [2023-12-27 03:55:34,888][105692] Updated weights for policy 0, policy_version 1720287 (0.0008) [2023-12-27 03:55:35,547][105692] Updated weights for policy 0, policy_version 1720297 (0.0010) [2023-12-27 03:55:35,591][105692] Updated weights for policy 0, policy_version 1720307 (0.0010) [2023-12-27 03:55:35,625][105620] Updated weights for policy 1, policy_version 1723830 (0.0007) [2023-12-27 03:55:35,640][105692] Updated weights for policy 0, policy_version 1720317 (0.0010) [2023-12-27 03:55:35,675][105620] Updated weights for policy 1, policy_version 1723840 (0.0007) [2023-12-27 03:55:35,723][105620] Updated weights for policy 1, policy_version 1723850 (0.0007) [2023-12-27 03:55:36,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.7, 300 sec: 19494.2). Total num frames: 881836032. Throughput: 0: 9768.2, 1: 9837.0. Samples: 881824256. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:55:36,063][104569] Avg episode reward: [(0, '8624.895'), (1, '8897.773')] [2023-12-27 03:55:36,431][105692] Updated weights for policy 0, policy_version 1720327 (0.0010) [2023-12-27 03:55:36,485][105620] Updated weights for policy 1, policy_version 1723860 (0.0007) [2023-12-27 03:55:36,494][105692] Updated weights for policy 0, policy_version 1720337 (0.0010) [2023-12-27 03:55:36,541][105620] Updated weights for policy 1, policy_version 1723870 (0.0006) [2023-12-27 03:55:36,554][105692] Updated weights for policy 0, policy_version 1720347 (0.0011) [2023-12-27 03:55:36,602][105620] Updated weights for policy 1, policy_version 1723880 (0.0009) [2023-12-27 03:55:37,221][105692] Updated weights for policy 0, policy_version 1720357 (0.0008) [2023-12-27 03:55:37,294][105692] Updated weights for policy 0, policy_version 1720367 (0.0008) [2023-12-27 03:55:37,366][105692] Updated weights for policy 0, policy_version 1720377 (0.0006) [2023-12-27 03:55:37,420][105620] Updated weights for policy 1, policy_version 1723890 (0.0008) [2023-12-27 03:55:37,479][105620] Updated weights for policy 1, policy_version 1723900 (0.0009) [2023-12-27 03:55:37,542][105620] Updated weights for policy 1, policy_version 1723910 (0.0009) [2023-12-27 03:55:37,613][105620] Updated weights for policy 1, policy_version 1723920 (0.0006) [2023-12-27 03:55:38,177][105692] Updated weights for policy 0, policy_version 1720387 (0.0009) [2023-12-27 03:55:38,222][105692] Updated weights for policy 0, policy_version 1720397 (0.0006) [2023-12-27 03:55:38,224][105620] Updated weights for policy 1, policy_version 1723930 (0.0008) [2023-12-27 03:55:38,264][105692] Updated weights for policy 0, policy_version 1720407 (0.0006) [2023-12-27 03:55:38,278][105620] Updated weights for policy 1, policy_version 1723940 (0.0007) [2023-12-27 03:55:38,342][105620] Updated weights for policy 1, policy_version 1723950 (0.0009) [2023-12-27 03:55:39,032][105692] Updated weights for policy 0, policy_version 1720417 (0.0006) [2023-12-27 03:55:39,095][105692] Updated weights for policy 0, policy_version 1720427 (0.0008) [2023-12-27 03:55:39,109][105620] Updated weights for policy 1, policy_version 1723960 (0.0007) [2023-12-27 03:55:39,155][105692] Updated weights for policy 0, policy_version 1720437 (0.0007) [2023-12-27 03:55:39,165][105620] Updated weights for policy 1, policy_version 1723970 (0.0006) [2023-12-27 03:55:39,215][105692] Updated weights for policy 0, policy_version 1720447 (0.0007) [2023-12-27 03:55:39,225][105620] Updated weights for policy 1, policy_version 1723980 (0.0006) [2023-12-27 03:55:39,967][105692] Updated weights for policy 0, policy_version 1720457 (0.0009) [2023-12-27 03:55:40,019][105692] Updated weights for policy 0, policy_version 1720467 (0.0008) [2023-12-27 03:55:40,026][105620] Updated weights for policy 1, policy_version 1723990 (0.0008) [2023-12-27 03:55:40,076][105692] Updated weights for policy 0, policy_version 1720477 (0.0010) [2023-12-27 03:55:40,084][105620] Updated weights for policy 1, policy_version 1724000 (0.0008) [2023-12-27 03:55:40,150][105620] Updated weights for policy 1, policy_version 1724010 (0.0008) [2023-12-27 03:55:40,864][105620] Updated weights for policy 1, policy_version 1724020 (0.0007) [2023-12-27 03:55:40,871][105692] Updated weights for policy 0, policy_version 1720487 (0.0009) [2023-12-27 03:55:40,918][105620] Updated weights for policy 1, policy_version 1724030 (0.0008) [2023-12-27 03:55:40,928][105692] Updated weights for policy 0, policy_version 1720497 (0.0008) [2023-12-27 03:55:40,974][105620] Updated weights for policy 1, policy_version 1724040 (0.0005) [2023-12-27 03:55:40,980][105692] Updated weights for policy 0, policy_version 1720507 (0.0009) [2023-12-27 03:55:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 881934336. Throughput: 0: 9729.3, 1: 9737.1. Samples: 881936524. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:55:41,062][104569] Avg episode reward: [(0, '8713.915'), (1, '9082.550')] [2023-12-27 03:55:41,690][105692] Updated weights for policy 0, policy_version 1720517 (0.0008) [2023-12-27 03:55:41,698][105620] Updated weights for policy 1, policy_version 1724050 (0.0008) [2023-12-27 03:55:41,755][105692] Updated weights for policy 0, policy_version 1720527 (0.0010) [2023-12-27 03:55:41,769][105620] Updated weights for policy 1, policy_version 1724060 (0.0007) [2023-12-27 03:55:41,809][105692] Updated weights for policy 0, policy_version 1720537 (0.0010) [2023-12-27 03:55:41,831][105620] Updated weights for policy 1, policy_version 1724070 (0.0006) [2023-12-27 03:55:41,895][105620] Updated weights for policy 1, policy_version 1724080 (0.0007) [2023-12-27 03:55:42,538][105692] Updated weights for policy 0, policy_version 1720547 (0.0011) [2023-12-27 03:55:42,599][105692] Updated weights for policy 0, policy_version 1720557 (0.0011) [2023-12-27 03:55:42,651][105620] Updated weights for policy 1, policy_version 1724090 (0.0007) [2023-12-27 03:55:42,659][105692] Updated weights for policy 0, policy_version 1720567 (0.0011) [2023-12-27 03:55:42,723][105620] Updated weights for policy 1, policy_version 1724100 (0.0008) [2023-12-27 03:55:42,786][105620] Updated weights for policy 1, policy_version 1724110 (0.0008) [2023-12-27 03:55:43,342][105620] Updated weights for policy 1, policy_version 1724120 (0.0006) [2023-12-27 03:55:43,399][105620] Updated weights for policy 1, policy_version 1724130 (0.0006) [2023-12-27 03:55:43,408][105692] Updated weights for policy 0, policy_version 1720577 (0.0011) [2023-12-27 03:55:43,448][105620] Updated weights for policy 1, policy_version 1724140 (0.0006) [2023-12-27 03:55:43,470][105692] Updated weights for policy 0, policy_version 1720587 (0.0011) [2023-12-27 03:55:43,528][105692] Updated weights for policy 0, policy_version 1720597 (0.0010) [2023-12-27 03:55:43,597][105692] Updated weights for policy 0, policy_version 1720607 (0.0011) [2023-12-27 03:55:44,055][105620] Updated weights for policy 1, policy_version 1724150 (0.0008) [2023-12-27 03:55:44,103][105620] Updated weights for policy 1, policy_version 1724160 (0.0010) [2023-12-27 03:55:44,158][105620] Updated weights for policy 1, policy_version 1724170 (0.0011) [2023-12-27 03:55:44,241][105692] Updated weights for policy 0, policy_version 1720617 (0.0006) [2023-12-27 03:55:44,304][105692] Updated weights for policy 0, policy_version 1720627 (0.0005) [2023-12-27 03:55:44,366][105692] Updated weights for policy 0, policy_version 1720637 (0.0005) [2023-12-27 03:55:44,849][105620] Updated weights for policy 1, policy_version 1724180 (0.0011) [2023-12-27 03:55:44,909][105620] Updated weights for policy 1, policy_version 1724190 (0.0011) [2023-12-27 03:55:44,957][105692] Updated weights for policy 0, policy_version 1720647 (0.0006) [2023-12-27 03:55:44,977][105620] Updated weights for policy 1, policy_version 1724200 (0.0011) [2023-12-27 03:55:45,019][105692] Updated weights for policy 0, policy_version 1720657 (0.0006) [2023-12-27 03:55:45,076][105692] Updated weights for policy 0, policy_version 1720667 (0.0007) [2023-12-27 03:55:45,695][105620] Updated weights for policy 1, policy_version 1724210 (0.0010) [2023-12-27 03:55:45,737][105692] Updated weights for policy 0, policy_version 1720677 (0.0008) [2023-12-27 03:55:45,747][105620] Updated weights for policy 1, policy_version 1724220 (0.0005) [2023-12-27 03:55:45,784][105692] Updated weights for policy 0, policy_version 1720687 (0.0005) [2023-12-27 03:55:45,793][105620] Updated weights for policy 1, policy_version 1724230 (0.0005) [2023-12-27 03:55:45,836][105692] Updated weights for policy 0, policy_version 1720697 (0.0006) [2023-12-27 03:55:45,843][105620] Updated weights for policy 1, policy_version 1724240 (0.0010) [2023-12-27 03:55:46,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 882032640. Throughput: 0: 9664.1, 1: 9819.0. Samples: 881995972. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:55:46,062][104569] Avg episode reward: [(0, '8621.771'), (1, '9082.809')] [2023-12-27 03:55:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001724240_441466880.pth... [2023-12-27 03:55:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001720704_440565760.pth... [2023-12-27 03:55:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001723120_441180160.pth [2023-12-27 03:55:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001719584_440279040.pth [2023-12-27 03:55:46,404][105692] Updated weights for policy 0, policy_version 1720707 (0.0006) [2023-12-27 03:55:46,463][105692] Updated weights for policy 0, policy_version 1720717 (0.0005) [2023-12-27 03:55:46,526][105692] Updated weights for policy 0, policy_version 1720727 (0.0006) [2023-12-27 03:55:46,579][105620] Updated weights for policy 1, policy_version 1724250 (0.0010) [2023-12-27 03:55:46,645][105620] Updated weights for policy 1, policy_version 1724260 (0.0011) [2023-12-27 03:55:46,708][105620] Updated weights for policy 1, policy_version 1724270 (0.0011) [2023-12-27 03:55:47,149][105692] Updated weights for policy 0, policy_version 1720737 (0.0007) [2023-12-27 03:55:47,204][105692] Updated weights for policy 0, policy_version 1720747 (0.0008) [2023-12-27 03:55:47,259][105692] Updated weights for policy 0, policy_version 1720757 (0.0008) [2023-12-27 03:55:47,328][105692] Updated weights for policy 0, policy_version 1720767 (0.0008) [2023-12-27 03:55:47,419][105620] Updated weights for policy 1, policy_version 1724280 (0.0010) [2023-12-27 03:55:47,468][105620] Updated weights for policy 1, policy_version 1724290 (0.0010) [2023-12-27 03:55:47,519][105620] Updated weights for policy 1, policy_version 1724300 (0.0010) [2023-12-27 03:55:48,073][105692] Updated weights for policy 0, policy_version 1720777 (0.0010) [2023-12-27 03:55:48,130][105692] Updated weights for policy 0, policy_version 1720787 (0.0009) [2023-12-27 03:55:48,183][105692] Updated weights for policy 0, policy_version 1720797 (0.0008) [2023-12-27 03:55:48,239][105620] Updated weights for policy 1, policy_version 1724310 (0.0010) [2023-12-27 03:55:48,287][105620] Updated weights for policy 1, policy_version 1724320 (0.0009) [2023-12-27 03:55:48,349][105620] Updated weights for policy 1, policy_version 1724330 (0.0010) [2023-12-27 03:55:48,967][105692] Updated weights for policy 0, policy_version 1720807 (0.0010) [2023-12-27 03:55:49,016][105692] Updated weights for policy 0, policy_version 1720817 (0.0010) [2023-12-27 03:55:49,064][105692] Updated weights for policy 0, policy_version 1720827 (0.0010) [2023-12-27 03:55:49,067][105620] Updated weights for policy 1, policy_version 1724340 (0.0010) [2023-12-27 03:55:49,126][105620] Updated weights for policy 1, policy_version 1724350 (0.0010) [2023-12-27 03:55:49,185][105620] Updated weights for policy 1, policy_version 1724360 (0.0011) [2023-12-27 03:55:49,860][105692] Updated weights for policy 0, policy_version 1720837 (0.0009) [2023-12-27 03:55:49,863][105620] Updated weights for policy 1, policy_version 1724370 (0.0008) [2023-12-27 03:55:49,921][105692] Updated weights for policy 0, policy_version 1720847 (0.0008) [2023-12-27 03:55:49,923][105620] Updated weights for policy 1, policy_version 1724380 (0.0006) [2023-12-27 03:55:49,990][105692] Updated weights for policy 0, policy_version 1720857 (0.0007) [2023-12-27 03:55:49,995][105620] Updated weights for policy 1, policy_version 1724390 (0.0008) [2023-12-27 03:55:50,056][105620] Updated weights for policy 1, policy_version 1724400 (0.0008) [2023-12-27 03:55:50,783][105692] Updated weights for policy 0, policy_version 1720867 (0.0009) [2023-12-27 03:55:50,821][105620] Updated weights for policy 1, policy_version 1724410 (0.0008) [2023-12-27 03:55:50,836][105692] Updated weights for policy 0, policy_version 1720877 (0.0007) [2023-12-27 03:55:50,882][105620] Updated weights for policy 1, policy_version 1724420 (0.0009) [2023-12-27 03:55:50,896][105692] Updated weights for policy 0, policy_version 1720887 (0.0006) [2023-12-27 03:55:50,936][105620] Updated weights for policy 1, policy_version 1724430 (0.0007) [2023-12-27 03:55:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 882130944. Throughput: 0: 9695.2, 1: 9744.9. Samples: 882116564. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:55:51,062][104569] Avg episode reward: [(0, '8713.621'), (1, '8713.541')] [2023-12-27 03:55:51,673][105692] Updated weights for policy 0, policy_version 1720897 (0.0007) [2023-12-27 03:55:51,722][105620] Updated weights for policy 1, policy_version 1724440 (0.0007) [2023-12-27 03:55:51,739][105692] Updated weights for policy 0, policy_version 1720907 (0.0008) [2023-12-27 03:55:51,781][105620] Updated weights for policy 1, policy_version 1724450 (0.0008) [2023-12-27 03:55:51,800][105692] Updated weights for policy 0, policy_version 1720917 (0.0006) [2023-12-27 03:55:51,838][105620] Updated weights for policy 1, policy_version 1724460 (0.0005) [2023-12-27 03:55:51,862][105692] Updated weights for policy 0, policy_version 1720927 (0.0009) [2023-12-27 03:55:52,538][105620] Updated weights for policy 1, policy_version 1724470 (0.0007) [2023-12-27 03:55:52,586][105620] Updated weights for policy 1, policy_version 1724480 (0.0005) [2023-12-27 03:55:52,625][105692] Updated weights for policy 0, policy_version 1720937 (0.0008) [2023-12-27 03:55:52,633][105620] Updated weights for policy 1, policy_version 1724490 (0.0007) [2023-12-27 03:55:52,686][105692] Updated weights for policy 0, policy_version 1720947 (0.0005) [2023-12-27 03:55:52,752][105692] Updated weights for policy 0, policy_version 1720957 (0.0007) [2023-12-27 03:55:53,375][105620] Updated weights for policy 1, policy_version 1724500 (0.0007) [2023-12-27 03:55:53,431][105620] Updated weights for policy 1, policy_version 1724510 (0.0005) [2023-12-27 03:55:53,479][105620] Updated weights for policy 1, policy_version 1724520 (0.0005) [2023-12-27 03:55:53,509][105692] Updated weights for policy 0, policy_version 1720967 (0.0008) [2023-12-27 03:55:53,566][105692] Updated weights for policy 0, policy_version 1720977 (0.0011) [2023-12-27 03:55:53,620][105692] Updated weights for policy 0, policy_version 1720988 (0.0009) [2023-12-27 03:55:54,138][105620] Updated weights for policy 1, policy_version 1724530 (0.0006) [2023-12-27 03:55:54,200][105620] Updated weights for policy 1, policy_version 1724540 (0.0010) [2023-12-27 03:55:54,255][105620] Updated weights for policy 1, policy_version 1724550 (0.0010) [2023-12-27 03:55:54,319][105620] Updated weights for policy 1, policy_version 1724560 (0.0010) [2023-12-27 03:55:54,410][105692] Updated weights for policy 0, policy_version 1720998 (0.0008) [2023-12-27 03:55:54,462][105692] Updated weights for policy 0, policy_version 1721008 (0.0008) [2023-12-27 03:55:54,514][105692] Updated weights for policy 0, policy_version 1721018 (0.0008) [2023-12-27 03:55:55,042][105620] Updated weights for policy 1, policy_version 1724570 (0.0010) [2023-12-27 03:55:55,088][105620] Updated weights for policy 1, policy_version 1724580 (0.0009) [2023-12-27 03:55:55,143][105620] Updated weights for policy 1, policy_version 1724590 (0.0010) [2023-12-27 03:55:55,265][105692] Updated weights for policy 0, policy_version 1721028 (0.0007) [2023-12-27 03:55:55,320][105692] Updated weights for policy 0, policy_version 1721038 (0.0005) [2023-12-27 03:55:55,371][105692] Updated weights for policy 0, policy_version 1721048 (0.0005) [2023-12-27 03:55:55,910][105620] Updated weights for policy 1, policy_version 1724600 (0.0010) [2023-12-27 03:55:55,925][105692] Updated weights for policy 0, policy_version 1721058 (0.0006) [2023-12-27 03:55:55,975][105620] Updated weights for policy 1, policy_version 1724610 (0.0010) [2023-12-27 03:55:55,979][105692] Updated weights for policy 0, policy_version 1721068 (0.0007) [2023-12-27 03:55:56,035][105620] Updated weights for policy 1, policy_version 1724620 (0.0011) [2023-12-27 03:55:56,041][105692] Updated weights for policy 0, policy_version 1721078 (0.0007) [2023-12-27 03:55:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 882221056. Throughput: 0: 9612.9, 1: 9689.6. Samples: 882228804. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:55:56,063][104569] Avg episode reward: [(0, '8805.798'), (1, '8896.397')] [2023-12-27 03:55:56,102][105692] Updated weights for policy 0, policy_version 1721088 (0.0007) [2023-12-27 03:55:56,758][105620] Updated weights for policy 1, policy_version 1724630 (0.0010) [2023-12-27 03:55:56,818][105620] Updated weights for policy 1, policy_version 1724640 (0.0010) [2023-12-27 03:55:56,821][105692] Updated weights for policy 0, policy_version 1721098 (0.0006) [2023-12-27 03:55:56,873][105620] Updated weights for policy 1, policy_version 1724650 (0.0010) [2023-12-27 03:55:56,879][105692] Updated weights for policy 0, policy_version 1721108 (0.0005) [2023-12-27 03:55:56,935][105692] Updated weights for policy 0, policy_version 1721118 (0.0006) [2023-12-27 03:55:57,605][105620] Updated weights for policy 1, policy_version 1724660 (0.0008) [2023-12-27 03:55:57,665][105620] Updated weights for policy 1, policy_version 1724670 (0.0007) [2023-12-27 03:55:57,699][105692] Updated weights for policy 0, policy_version 1721128 (0.0006) [2023-12-27 03:55:57,724][105620] Updated weights for policy 1, policy_version 1724680 (0.0009) [2023-12-27 03:55:57,758][105692] Updated weights for policy 0, policy_version 1721138 (0.0008) [2023-12-27 03:55:57,810][105692] Updated weights for policy 0, policy_version 1721148 (0.0008) [2023-12-27 03:55:58,318][105620] Updated weights for policy 1, policy_version 1724690 (0.0010) [2023-12-27 03:55:58,385][105620] Updated weights for policy 1, policy_version 1724700 (0.0008) [2023-12-27 03:55:58,457][105620] Updated weights for policy 1, policy_version 1724710 (0.0008) [2023-12-27 03:55:58,517][105620] Updated weights for policy 1, policy_version 1724720 (0.0009) [2023-12-27 03:55:58,673][105692] Updated weights for policy 0, policy_version 1721158 (0.0009) [2023-12-27 03:55:58,735][105692] Updated weights for policy 0, policy_version 1721168 (0.0010) [2023-12-27 03:55:58,801][105692] Updated weights for policy 0, policy_version 1721178 (0.0008) [2023-12-27 03:55:59,287][105620] Updated weights for policy 1, policy_version 1724730 (0.0009) [2023-12-27 03:55:59,354][105620] Updated weights for policy 1, policy_version 1724740 (0.0008) [2023-12-27 03:55:59,419][105620] Updated weights for policy 1, policy_version 1724750 (0.0008) [2023-12-27 03:55:59,662][105692] Updated weights for policy 0, policy_version 1721188 (0.0010) [2023-12-27 03:55:59,716][105692] Updated weights for policy 0, policy_version 1721198 (0.0010) [2023-12-27 03:55:59,771][105692] Updated weights for policy 0, policy_version 1721208 (0.0010) [2023-12-27 03:56:00,234][105620] Updated weights for policy 1, policy_version 1724760 (0.0010) [2023-12-27 03:56:00,300][105620] Updated weights for policy 1, policy_version 1724770 (0.0010) [2023-12-27 03:56:00,364][105620] Updated weights for policy 1, policy_version 1724780 (0.0010) [2023-12-27 03:56:00,490][105692] Updated weights for policy 0, policy_version 1721218 (0.0009) [2023-12-27 03:56:00,549][105692] Updated weights for policy 0, policy_version 1721228 (0.0006) [2023-12-27 03:56:00,598][105692] Updated weights for policy 0, policy_version 1721238 (0.0010) [2023-12-27 03:56:00,652][105692] Updated weights for policy 0, policy_version 1721248 (0.0010) [2023-12-27 03:56:00,996][105620] Updated weights for policy 1, policy_version 1724790 (0.0009) [2023-12-27 03:56:01,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 882311168. Throughput: 0: 9601.8, 1: 9710.8. Samples: 882286616. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:56:01,063][105620] Updated weights for policy 1, policy_version 1724800 (0.0007) [2023-12-27 03:56:01,063][104569] Avg episode reward: [(0, '8167.087'), (1, '9080.474')] [2023-12-27 03:56:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001721248_440705024.pth... [2023-12-27 03:56:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001720128_440418304.pth [2023-12-27 03:56:01,123][105620] Updated weights for policy 1, policy_version 1724810 (0.0006) [2023-12-27 03:56:01,164][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001724816_441614336.pth... [2023-12-27 03:56:01,168][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001723664_441319424.pth [2023-12-27 03:56:01,386][105692] Updated weights for policy 0, policy_version 1721258 (0.0008) [2023-12-27 03:56:01,439][105692] Updated weights for policy 0, policy_version 1721268 (0.0006) [2023-12-27 03:56:01,496][105692] Updated weights for policy 0, policy_version 1721278 (0.0010) [2023-12-27 03:56:01,754][105620] Updated weights for policy 1, policy_version 1724820 (0.0010) [2023-12-27 03:56:01,812][105620] Updated weights for policy 1, policy_version 1724830 (0.0010) [2023-12-27 03:56:01,872][105620] Updated weights for policy 1, policy_version 1724840 (0.0008) [2023-12-27 03:56:02,336][105692] Updated weights for policy 0, policy_version 1721288 (0.0010) [2023-12-27 03:56:02,394][105692] Updated weights for policy 0, policy_version 1721298 (0.0008) [2023-12-27 03:56:02,453][105692] Updated weights for policy 0, policy_version 1721308 (0.0009) [2023-12-27 03:56:02,485][105620] Updated weights for policy 1, policy_version 1724850 (0.0006) [2023-12-27 03:56:02,547][105620] Updated weights for policy 1, policy_version 1724860 (0.0009) [2023-12-27 03:56:02,605][105620] Updated weights for policy 1, policy_version 1724870 (0.0008) [2023-12-27 03:56:02,662][105620] Updated weights for policy 1, policy_version 1724880 (0.0009) [2023-12-27 03:56:03,146][105692] Updated weights for policy 0, policy_version 1721318 (0.0008) [2023-12-27 03:56:03,209][105692] Updated weights for policy 0, policy_version 1721328 (0.0009) [2023-12-27 03:56:03,259][105692] Updated weights for policy 0, policy_version 1721338 (0.0009) [2023-12-27 03:56:03,379][105620] Updated weights for policy 1, policy_version 1724890 (0.0010) [2023-12-27 03:56:03,421][105620] Updated weights for policy 1, policy_version 1724900 (0.0008) [2023-12-27 03:56:03,469][105620] Updated weights for policy 1, policy_version 1724910 (0.0005) [2023-12-27 03:56:04,075][105620] Updated weights for policy 1, policy_version 1724920 (0.0008) [2023-12-27 03:56:04,111][105692] Updated weights for policy 0, policy_version 1721348 (0.0007) [2023-12-27 03:56:04,131][105620] Updated weights for policy 1, policy_version 1724930 (0.0008) [2023-12-27 03:56:04,169][105692] Updated weights for policy 0, policy_version 1721358 (0.0008) [2023-12-27 03:56:04,190][105620] Updated weights for policy 1, policy_version 1724940 (0.0008) [2023-12-27 03:56:04,234][105692] Updated weights for policy 0, policy_version 1721368 (0.0009) [2023-12-27 03:56:04,876][105620] Updated weights for policy 1, policy_version 1724950 (0.0007) [2023-12-27 03:56:04,922][105620] Updated weights for policy 1, policy_version 1724960 (0.0008) [2023-12-27 03:56:04,970][105620] Updated weights for policy 1, policy_version 1724970 (0.0007) [2023-12-27 03:56:04,990][105692] Updated weights for policy 0, policy_version 1721378 (0.0010) [2023-12-27 03:56:05,049][105692] Updated weights for policy 0, policy_version 1721388 (0.0010) [2023-12-27 03:56:05,111][105692] Updated weights for policy 0, policy_version 1721398 (0.0010) [2023-12-27 03:56:05,180][105692] Updated weights for policy 0, policy_version 1721408 (0.0010) [2023-12-27 03:56:05,633][105620] Updated weights for policy 1, policy_version 1724980 (0.0008) [2023-12-27 03:56:05,679][105620] Updated weights for policy 1, policy_version 1724990 (0.0008) [2023-12-27 03:56:05,733][105620] Updated weights for policy 1, policy_version 1725000 (0.0009) [2023-12-27 03:56:05,871][105692] Updated weights for policy 0, policy_version 1721418 (0.0010) [2023-12-27 03:56:05,926][105692] Updated weights for policy 0, policy_version 1721430 (0.0010) [2023-12-27 03:56:05,983][105692] Updated weights for policy 0, policy_version 1721440 (0.0009) [2023-12-27 03:56:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 882417664. Throughput: 0: 9432.5, 1: 9741.2. Samples: 882401324. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:56:06,062][104569] Avg episode reward: [(0, '8072.256'), (1, '9079.229')] [2023-12-27 03:56:06,465][105620] Updated weights for policy 1, policy_version 1725010 (0.0009) [2023-12-27 03:56:06,525][105620] Updated weights for policy 1, policy_version 1725020 (0.0009) [2023-12-27 03:56:06,576][105620] Updated weights for policy 1, policy_version 1725030 (0.0008) [2023-12-27 03:56:06,623][105620] Updated weights for policy 1, policy_version 1725040 (0.0009) [2023-12-27 03:56:06,812][105692] Updated weights for policy 0, policy_version 1721450 (0.0009) [2023-12-27 03:56:06,864][105692] Updated weights for policy 0, policy_version 1721460 (0.0009) [2023-12-27 03:56:06,920][105692] Updated weights for policy 0, policy_version 1721470 (0.0009) [2023-12-27 03:56:07,289][105620] Updated weights for policy 1, policy_version 1725050 (0.0008) [2023-12-27 03:56:07,345][105620] Updated weights for policy 1, policy_version 1725060 (0.0009) [2023-12-27 03:56:07,393][105620] Updated weights for policy 1, policy_version 1725070 (0.0009) [2023-12-27 03:56:07,647][105692] Updated weights for policy 0, policy_version 1721480 (0.0010) [2023-12-27 03:56:07,716][105692] Updated weights for policy 0, policy_version 1721490 (0.0011) [2023-12-27 03:56:07,776][105692] Updated weights for policy 0, policy_version 1721500 (0.0010) [2023-12-27 03:56:08,139][105620] Updated weights for policy 1, policy_version 1725080 (0.0009) [2023-12-27 03:56:08,200][105620] Updated weights for policy 1, policy_version 1725090 (0.0009) [2023-12-27 03:56:08,255][105620] Updated weights for policy 1, policy_version 1725100 (0.0009) [2023-12-27 03:56:08,462][105692] Updated weights for policy 0, policy_version 1721510 (0.0010) [2023-12-27 03:56:08,525][105692] Updated weights for policy 0, policy_version 1721520 (0.0010) [2023-12-27 03:56:08,586][105692] Updated weights for policy 0, policy_version 1721530 (0.0008) [2023-12-27 03:56:09,065][105620] Updated weights for policy 1, policy_version 1725110 (0.0010) [2023-12-27 03:56:09,119][105620] Updated weights for policy 1, policy_version 1725120 (0.0010) [2023-12-27 03:56:09,178][105620] Updated weights for policy 1, policy_version 1725130 (0.0009) [2023-12-27 03:56:09,219][105692] Updated weights for policy 0, policy_version 1721540 (0.0007) [2023-12-27 03:56:09,285][105692] Updated weights for policy 0, policy_version 1721550 (0.0007) [2023-12-27 03:56:09,355][105692] Updated weights for policy 0, policy_version 1721560 (0.0008) [2023-12-27 03:56:09,940][105620] Updated weights for policy 1, policy_version 1725140 (0.0008) [2023-12-27 03:56:10,004][105620] Updated weights for policy 1, policy_version 1725150 (0.0006) [2023-12-27 03:56:10,064][105620] Updated weights for policy 1, policy_version 1725160 (0.0006) [2023-12-27 03:56:10,175][105692] Updated weights for policy 0, policy_version 1721570 (0.0009) [2023-12-27 03:56:10,244][105692] Updated weights for policy 0, policy_version 1721580 (0.0007) [2023-12-27 03:56:10,310][105692] Updated weights for policy 0, policy_version 1721590 (0.0008) [2023-12-27 03:56:10,367][105692] Updated weights for policy 0, policy_version 1721600 (0.0010) [2023-12-27 03:56:10,666][105620] Updated weights for policy 1, policy_version 1725170 (0.0006) [2023-12-27 03:56:10,726][105620] Updated weights for policy 1, policy_version 1725180 (0.0006) [2023-12-27 03:56:10,791][105620] Updated weights for policy 1, policy_version 1725190 (0.0009) [2023-12-27 03:56:10,853][105620] Updated weights for policy 1, policy_version 1725200 (0.0007) [2023-12-27 03:56:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.9, 300 sec: 19466.4). Total num frames: 882507776. Throughput: 0: 9492.3, 1: 9780.4. Samples: 882516980. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:56:11,062][104569] Avg episode reward: [(0, '8620.572'), (1, '9263.235')] [2023-12-27 03:56:11,112][105692] Updated weights for policy 0, policy_version 1721610 (0.0008) [2023-12-27 03:56:11,181][105692] Updated weights for policy 0, policy_version 1721620 (0.0011) [2023-12-27 03:56:11,242][105692] Updated weights for policy 0, policy_version 1721630 (0.0006) [2023-12-27 03:56:11,529][105620] Updated weights for policy 1, policy_version 1725210 (0.0008) [2023-12-27 03:56:11,581][105620] Updated weights for policy 1, policy_version 1725220 (0.0008) [2023-12-27 03:56:11,642][105620] Updated weights for policy 1, policy_version 1725230 (0.0008) [2023-12-27 03:56:11,962][105692] Updated weights for policy 0, policy_version 1721640 (0.0010) [2023-12-27 03:56:12,012][105692] Updated weights for policy 0, policy_version 1721650 (0.0010) [2023-12-27 03:56:12,066][105692] Updated weights for policy 0, policy_version 1721660 (0.0011) [2023-12-27 03:56:12,362][105620] Updated weights for policy 1, policy_version 1725240 (0.0009) [2023-12-27 03:56:12,419][105620] Updated weights for policy 1, policy_version 1725250 (0.0010) [2023-12-27 03:56:12,468][105620] Updated weights for policy 1, policy_version 1725260 (0.0011) [2023-12-27 03:56:12,733][105692] Updated weights for policy 0, policy_version 1721670 (0.0009) [2023-12-27 03:56:12,794][105692] Updated weights for policy 0, policy_version 1721680 (0.0011) [2023-12-27 03:56:12,854][105692] Updated weights for policy 0, policy_version 1721690 (0.0011) [2023-12-27 03:56:13,186][105620] Updated weights for policy 1, policy_version 1725270 (0.0009) [2023-12-27 03:56:13,249][105620] Updated weights for policy 1, policy_version 1725280 (0.0008) [2023-12-27 03:56:13,297][105620] Updated weights for policy 1, policy_version 1725290 (0.0010) [2023-12-27 03:56:13,505][105692] Updated weights for policy 0, policy_version 1721700 (0.0010) [2023-12-27 03:56:13,562][105692] Updated weights for policy 0, policy_version 1721710 (0.0009) [2023-12-27 03:56:13,609][105692] Updated weights for policy 0, policy_version 1721720 (0.0009) [2023-12-27 03:56:14,117][105620] Updated weights for policy 1, policy_version 1725300 (0.0010) [2023-12-27 03:56:14,179][105620] Updated weights for policy 1, policy_version 1725310 (0.0009) [2023-12-27 03:56:14,226][105692] Updated weights for policy 0, policy_version 1721730 (0.0008) [2023-12-27 03:56:14,246][105620] Updated weights for policy 1, policy_version 1725320 (0.0006) [2023-12-27 03:56:14,288][105692] Updated weights for policy 0, policy_version 1721740 (0.0006) [2023-12-27 03:56:14,355][105692] Updated weights for policy 0, policy_version 1721750 (0.0007) [2023-12-27 03:56:14,412][105692] Updated weights for policy 0, policy_version 1721760 (0.0010) [2023-12-27 03:56:14,921][105620] Updated weights for policy 1, policy_version 1725330 (0.0007) [2023-12-27 03:56:14,982][105620] Updated weights for policy 1, policy_version 1725340 (0.0008) [2023-12-27 03:56:15,042][105620] Updated weights for policy 1, policy_version 1725350 (0.0008) [2023-12-27 03:56:15,095][105620] Updated weights for policy 1, policy_version 1725360 (0.0008) [2023-12-27 03:56:15,148][105692] Updated weights for policy 0, policy_version 1721770 (0.0011) [2023-12-27 03:56:15,215][105692] Updated weights for policy 0, policy_version 1721780 (0.0011) [2023-12-27 03:56:15,275][105692] Updated weights for policy 0, policy_version 1721790 (0.0011) [2023-12-27 03:56:15,799][105620] Updated weights for policy 1, policy_version 1725370 (0.0008) [2023-12-27 03:56:15,858][105620] Updated weights for policy 1, policy_version 1725380 (0.0008) [2023-12-27 03:56:15,923][105620] Updated weights for policy 1, policy_version 1725390 (0.0008) [2023-12-27 03:56:16,007][105692] Updated weights for policy 0, policy_version 1721800 (0.0006) [2023-12-27 03:56:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 882606080. Throughput: 0: 9555.5, 1: 9735.1. Samples: 882575768. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:56:16,062][104569] Avg episode reward: [(0, '8076.474'), (1, '9263.615')] [2023-12-27 03:56:16,063][105692] Updated weights for policy 0, policy_version 1721810 (0.0005) [2023-12-27 03:56:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001725392_441761792.pth... [2023-12-27 03:56:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001724240_441466880.pth [2023-12-27 03:56:16,123][105692] Updated weights for policy 0, policy_version 1721820 (0.0005) [2023-12-27 03:56:16,142][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001721824_440852480.pth... [2023-12-27 03:56:16,145][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001720704_440565760.pth [2023-12-27 03:56:16,681][105620] Updated weights for policy 1, policy_version 1725400 (0.0009) [2023-12-27 03:56:16,707][105692] Updated weights for policy 0, policy_version 1721830 (0.0005) [2023-12-27 03:56:16,736][105620] Updated weights for policy 1, policy_version 1725410 (0.0009) [2023-12-27 03:56:16,759][105692] Updated weights for policy 0, policy_version 1721840 (0.0005) [2023-12-27 03:56:16,786][105620] Updated weights for policy 1, policy_version 1725420 (0.0007) [2023-12-27 03:56:16,812][105692] Updated weights for policy 0, policy_version 1721850 (0.0007) [2023-12-27 03:56:17,510][105620] Updated weights for policy 1, policy_version 1725430 (0.0008) [2023-12-27 03:56:17,551][105692] Updated weights for policy 0, policy_version 1721860 (0.0008) [2023-12-27 03:56:17,561][105620] Updated weights for policy 1, policy_version 1725440 (0.0007) [2023-12-27 03:56:17,611][105692] Updated weights for policy 0, policy_version 1721870 (0.0008) [2023-12-27 03:56:17,617][105620] Updated weights for policy 1, policy_version 1725450 (0.0006) [2023-12-27 03:56:17,668][105692] Updated weights for policy 0, policy_version 1721880 (0.0008) [2023-12-27 03:56:18,206][105620] Updated weights for policy 1, policy_version 1725460 (0.0007) [2023-12-27 03:56:18,265][105620] Updated weights for policy 1, policy_version 1725470 (0.0009) [2023-12-27 03:56:18,329][105620] Updated weights for policy 1, policy_version 1725480 (0.0008) [2023-12-27 03:56:18,505][105692] Updated weights for policy 0, policy_version 1721890 (0.0009) [2023-12-27 03:56:18,571][105692] Updated weights for policy 0, policy_version 1721900 (0.0008) [2023-12-27 03:56:18,632][105692] Updated weights for policy 0, policy_version 1721910 (0.0010) [2023-12-27 03:56:18,694][105692] Updated weights for policy 0, policy_version 1721920 (0.0008) [2023-12-27 03:56:19,081][105620] Updated weights for policy 1, policy_version 1725490 (0.0009) [2023-12-27 03:56:19,132][105620] Updated weights for policy 1, policy_version 1725500 (0.0009) [2023-12-27 03:56:19,183][105620] Updated weights for policy 1, policy_version 1725510 (0.0009) [2023-12-27 03:56:19,236][105620] Updated weights for policy 1, policy_version 1725520 (0.0008) [2023-12-27 03:56:19,437][105692] Updated weights for policy 0, policy_version 1721930 (0.0005) [2023-12-27 03:56:19,506][105692] Updated weights for policy 0, policy_version 1721940 (0.0007) [2023-12-27 03:56:19,577][105692] Updated weights for policy 0, policy_version 1721950 (0.0006) [2023-12-27 03:56:20,039][105620] Updated weights for policy 1, policy_version 1725530 (0.0009) [2023-12-27 03:56:20,100][105620] Updated weights for policy 1, policy_version 1725540 (0.0007) [2023-12-27 03:56:20,172][105620] Updated weights for policy 1, policy_version 1725550 (0.0006) [2023-12-27 03:56:20,328][105692] Updated weights for policy 0, policy_version 1721960 (0.0009) [2023-12-27 03:56:20,388][105692] Updated weights for policy 0, policy_version 1721970 (0.0009) [2023-12-27 03:56:20,452][105692] Updated weights for policy 0, policy_version 1721980 (0.0010) [2023-12-27 03:56:20,832][105620] Updated weights for policy 1, policy_version 1725560 (0.0009) [2023-12-27 03:56:20,895][105620] Updated weights for policy 1, policy_version 1725570 (0.0009) [2023-12-27 03:56:20,954][105620] Updated weights for policy 1, policy_version 1725580 (0.0009) [2023-12-27 03:56:21,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 882704384. Throughput: 0: 9536.2, 1: 9767.8. Samples: 882692932. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:56:21,063][104569] Avg episode reward: [(0, '8164.921'), (1, '9078.760')] [2023-12-27 03:56:21,274][105692] Updated weights for policy 0, policy_version 1721990 (0.0010) [2023-12-27 03:56:21,330][105692] Updated weights for policy 0, policy_version 1722000 (0.0011) [2023-12-27 03:56:21,402][105692] Updated weights for policy 0, policy_version 1722010 (0.0011) [2023-12-27 03:56:21,760][105620] Updated weights for policy 1, policy_version 1725590 (0.0009) [2023-12-27 03:56:21,818][105620] Updated weights for policy 1, policy_version 1725600 (0.0010) [2023-12-27 03:56:21,871][105620] Updated weights for policy 1, policy_version 1725610 (0.0009) [2023-12-27 03:56:22,085][105692] Updated weights for policy 0, policy_version 1722020 (0.0011) [2023-12-27 03:56:22,137][105692] Updated weights for policy 0, policy_version 1722030 (0.0010) [2023-12-27 03:56:22,194][105692] Updated weights for policy 0, policy_version 1722040 (0.0011) [2023-12-27 03:56:22,650][105620] Updated weights for policy 1, policy_version 1725620 (0.0008) [2023-12-27 03:56:22,707][105620] Updated weights for policy 1, policy_version 1725630 (0.0007) [2023-12-27 03:56:22,760][105620] Updated weights for policy 1, policy_version 1725640 (0.0008) [2023-12-27 03:56:22,760][105586] KL-divergence is very high: 120.7177 [2023-12-27 03:56:22,949][105692] Updated weights for policy 0, policy_version 1722050 (0.0011) [2023-12-27 03:56:23,005][105692] Updated weights for policy 0, policy_version 1722060 (0.0011) [2023-12-27 03:56:23,061][105692] Updated weights for policy 0, policy_version 1722070 (0.0011) [2023-12-27 03:56:23,121][105692] Updated weights for policy 0, policy_version 1722080 (0.0011) [2023-12-27 03:56:23,467][105620] Updated weights for policy 1, policy_version 1725650 (0.0008) [2023-12-27 03:56:23,522][105620] Updated weights for policy 1, policy_version 1725660 (0.0009) [2023-12-27 03:56:23,576][105620] Updated weights for policy 1, policy_version 1725670 (0.0010) [2023-12-27 03:56:23,630][105620] Updated weights for policy 1, policy_version 1725680 (0.0010) [2023-12-27 03:56:23,715][105692] Updated weights for policy 0, policy_version 1722090 (0.0008) [2023-12-27 03:56:23,767][105692] Updated weights for policy 0, policy_version 1722100 (0.0010) [2023-12-27 03:56:23,821][105692] Updated weights for policy 0, policy_version 1722110 (0.0010) [2023-12-27 03:56:24,318][105620] Updated weights for policy 1, policy_version 1725690 (0.0008) [2023-12-27 03:56:24,379][105620] Updated weights for policy 1, policy_version 1725700 (0.0005) [2023-12-27 03:56:24,432][105620] Updated weights for policy 1, policy_version 1725710 (0.0005) [2023-12-27 03:56:24,553][105692] Updated weights for policy 0, policy_version 1722120 (0.0009) [2023-12-27 03:56:24,618][105692] Updated weights for policy 0, policy_version 1722130 (0.0009) [2023-12-27 03:56:24,671][105692] Updated weights for policy 0, policy_version 1722140 (0.0008) [2023-12-27 03:56:25,040][105620] Updated weights for policy 1, policy_version 1725720 (0.0008) [2023-12-27 03:56:25,098][105620] Updated weights for policy 1, policy_version 1725730 (0.0009) [2023-12-27 03:56:25,148][105620] Updated weights for policy 1, policy_version 1725740 (0.0008) [2023-12-27 03:56:25,383][105692] Updated weights for policy 0, policy_version 1722150 (0.0009) [2023-12-27 03:56:25,430][105692] Updated weights for policy 0, policy_version 1722160 (0.0009) [2023-12-27 03:56:25,486][105692] Updated weights for policy 0, policy_version 1722171 (0.0010) [2023-12-27 03:56:25,803][105620] Updated weights for policy 1, policy_version 1725750 (0.0009) [2023-12-27 03:56:25,862][105620] Updated weights for policy 1, policy_version 1725760 (0.0010) [2023-12-27 03:56:25,915][105620] Updated weights for policy 1, policy_version 1725770 (0.0009) [2023-12-27 03:56:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 882802688. Throughput: 0: 9550.2, 1: 9836.0. Samples: 882808904. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:56:26,063][104569] Avg episode reward: [(0, '8986.992'), (1, '8896.055')] [2023-12-27 03:56:26,283][105692] Updated weights for policy 0, policy_version 1722181 (0.0010) [2023-12-27 03:56:26,329][105692] Updated weights for policy 0, policy_version 1722191 (0.0008) [2023-12-27 03:56:26,384][105692] Updated weights for policy 0, policy_version 1722201 (0.0009) [2023-12-27 03:56:26,666][105620] Updated weights for policy 1, policy_version 1725781 (0.0008) [2023-12-27 03:56:26,734][105620] Updated weights for policy 1, policy_version 1725791 (0.0008) [2023-12-27 03:56:26,799][105620] Updated weights for policy 1, policy_version 1725801 (0.0009) [2023-12-27 03:56:27,088][105692] Updated weights for policy 0, policy_version 1722211 (0.0008) [2023-12-27 03:56:27,133][105692] Updated weights for policy 0, policy_version 1722221 (0.0005) [2023-12-27 03:56:27,181][105692] Updated weights for policy 0, policy_version 1722231 (0.0009) [2023-12-27 03:56:27,511][105620] Updated weights for policy 1, policy_version 1725811 (0.0008) [2023-12-27 03:56:27,571][105620] Updated weights for policy 1, policy_version 1725821 (0.0008) [2023-12-27 03:56:27,635][105620] Updated weights for policy 1, policy_version 1725831 (0.0009) [2023-12-27 03:56:27,871][105692] Updated weights for policy 0, policy_version 1722241 (0.0008) [2023-12-27 03:56:27,925][105692] Updated weights for policy 0, policy_version 1722251 (0.0006) [2023-12-27 03:56:27,983][105692] Updated weights for policy 0, policy_version 1722261 (0.0006) [2023-12-27 03:56:28,043][105692] Updated weights for policy 0, policy_version 1722271 (0.0006) [2023-12-27 03:56:28,403][105620] Updated weights for policy 1, policy_version 1725841 (0.0009) [2023-12-27 03:56:28,468][105620] Updated weights for policy 1, policy_version 1725851 (0.0009) [2023-12-27 03:56:28,528][105620] Updated weights for policy 1, policy_version 1725861 (0.0008) [2023-12-27 03:56:28,586][105620] Updated weights for policy 1, policy_version 1725871 (0.0009) [2023-12-27 03:56:28,714][105692] Updated weights for policy 0, policy_version 1722281 (0.0009) [2023-12-27 03:56:28,777][105692] Updated weights for policy 0, policy_version 1722291 (0.0010) [2023-12-27 03:56:28,824][105692] Updated weights for policy 0, policy_version 1722301 (0.0009) [2023-12-27 03:56:29,278][105620] Updated weights for policy 1, policy_version 1725881 (0.0008) [2023-12-27 03:56:29,340][105620] Updated weights for policy 1, policy_version 1725891 (0.0009) [2023-12-27 03:56:29,395][105620] Updated weights for policy 1, policy_version 1725901 (0.0006) [2023-12-27 03:56:29,647][105692] Updated weights for policy 0, policy_version 1722311 (0.0008) [2023-12-27 03:56:29,709][105692] Updated weights for policy 0, policy_version 1722321 (0.0009) [2023-12-27 03:56:29,763][105692] Updated weights for policy 0, policy_version 1722331 (0.0008) [2023-12-27 03:56:30,092][105620] Updated weights for policy 1, policy_version 1725911 (0.0006) [2023-12-27 03:56:30,157][105620] Updated weights for policy 1, policy_version 1725921 (0.0006) [2023-12-27 03:56:30,220][105620] Updated weights for policy 1, policy_version 1725931 (0.0009) [2023-12-27 03:56:30,558][105692] Updated weights for policy 0, policy_version 1722341 (0.0009) [2023-12-27 03:56:30,615][105692] Updated weights for policy 0, policy_version 1722351 (0.0009) [2023-12-27 03:56:30,665][105692] Updated weights for policy 0, policy_version 1722361 (0.0009) [2023-12-27 03:56:30,892][105620] Updated weights for policy 1, policy_version 1725941 (0.0007) [2023-12-27 03:56:30,946][105620] Updated weights for policy 1, policy_version 1725951 (0.0005) [2023-12-27 03:56:30,998][105620] Updated weights for policy 1, policy_version 1725961 (0.0006) [2023-12-27 03:56:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 882900992. Throughput: 0: 9594.9, 1: 9770.7. Samples: 882867428. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:56:31,063][104569] Avg episode reward: [(0, '8988.202'), (1, '8989.561')] [2023-12-27 03:56:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001722368_440991744.pth... [2023-12-27 03:56:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001725968_441909248.pth... [2023-12-27 03:56:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001721248_440705024.pth [2023-12-27 03:56:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001724816_441614336.pth [2023-12-27 03:56:31,505][105692] Updated weights for policy 0, policy_version 1722371 (0.0008) [2023-12-27 03:56:31,556][105692] Updated weights for policy 0, policy_version 1722381 (0.0006) [2023-12-27 03:56:31,625][105692] Updated weights for policy 0, policy_version 1722391 (0.0006) [2023-12-27 03:56:31,682][105620] Updated weights for policy 1, policy_version 1725971 (0.0008) [2023-12-27 03:56:31,746][105620] Updated weights for policy 1, policy_version 1725981 (0.0007) [2023-12-27 03:56:31,813][105620] Updated weights for policy 1, policy_version 1725991 (0.0008) [2023-12-27 03:56:32,331][105692] Updated weights for policy 0, policy_version 1722401 (0.0007) [2023-12-27 03:56:32,397][105692] Updated weights for policy 0, policy_version 1722411 (0.0009) [2023-12-27 03:56:32,454][105692] Updated weights for policy 0, policy_version 1722421 (0.0009) [2023-12-27 03:56:32,506][105620] Updated weights for policy 1, policy_version 1726001 (0.0009) [2023-12-27 03:56:32,514][105692] Updated weights for policy 0, policy_version 1722431 (0.0009) [2023-12-27 03:56:32,566][105620] Updated weights for policy 1, policy_version 1726011 (0.0009) [2023-12-27 03:56:32,625][105620] Updated weights for policy 1, policy_version 1726021 (0.0009) [2023-12-27 03:56:32,679][105620] Updated weights for policy 1, policy_version 1726031 (0.0009) [2023-12-27 03:56:33,269][105692] Updated weights for policy 0, policy_version 1722441 (0.0009) [2023-12-27 03:56:33,319][105692] Updated weights for policy 0, policy_version 1722451 (0.0009) [2023-12-27 03:56:33,365][105692] Updated weights for policy 0, policy_version 1722461 (0.0008) [2023-12-27 03:56:33,412][105620] Updated weights for policy 1, policy_version 1726041 (0.0008) [2023-12-27 03:56:33,460][105620] Updated weights for policy 1, policy_version 1726051 (0.0009) [2023-12-27 03:56:33,516][105620] Updated weights for policy 1, policy_version 1726061 (0.0009) [2023-12-27 03:56:34,065][105692] Updated weights for policy 0, policy_version 1722471 (0.0006) [2023-12-27 03:56:34,120][105692] Updated weights for policy 0, policy_version 1722481 (0.0007) [2023-12-27 03:56:34,178][105692] Updated weights for policy 0, policy_version 1722491 (0.0008) [2023-12-27 03:56:34,368][105620] Updated weights for policy 1, policy_version 1726071 (0.0009) [2023-12-27 03:56:34,434][105620] Updated weights for policy 1, policy_version 1726081 (0.0009) [2023-12-27 03:56:34,488][105620] Updated weights for policy 1, policy_version 1726091 (0.0009) [2023-12-27 03:56:34,899][105692] Updated weights for policy 0, policy_version 1722501 (0.0008) [2023-12-27 03:56:34,952][105692] Updated weights for policy 0, policy_version 1722511 (0.0008) [2023-12-27 03:56:35,009][105692] Updated weights for policy 0, policy_version 1722521 (0.0008) [2023-12-27 03:56:35,229][105620] Updated weights for policy 1, policy_version 1726101 (0.0008) [2023-12-27 03:56:35,280][105620] Updated weights for policy 1, policy_version 1726111 (0.0009) [2023-12-27 03:56:35,335][105620] Updated weights for policy 1, policy_version 1726121 (0.0009) [2023-12-27 03:56:35,724][105692] Updated weights for policy 0, policy_version 1722531 (0.0009) [2023-12-27 03:56:35,782][105692] Updated weights for policy 0, policy_version 1722541 (0.0005) [2023-12-27 03:56:35,838][105692] Updated weights for policy 0, policy_version 1722551 (0.0005) [2023-12-27 03:56:36,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 882991104. Throughput: 0: 9456.3, 1: 9756.2. Samples: 882981128. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:56:36,063][104569] Avg episode reward: [(0, '8622.511'), (1, '9079.704')] [2023-12-27 03:56:36,130][105620] Updated weights for policy 1, policy_version 1726131 (0.0009) [2023-12-27 03:56:36,183][105620] Updated weights for policy 1, policy_version 1726141 (0.0009) [2023-12-27 03:56:36,245][105620] Updated weights for policy 1, policy_version 1726151 (0.0005) [2023-12-27 03:56:36,516][105692] Updated weights for policy 0, policy_version 1722561 (0.0006) [2023-12-27 03:56:36,569][105692] Updated weights for policy 0, policy_version 1722571 (0.0009) [2023-12-27 03:56:36,630][105692] Updated weights for policy 0, policy_version 1722581 (0.0006) [2023-12-27 03:56:36,695][105692] Updated weights for policy 0, policy_version 1722591 (0.0008) [2023-12-27 03:56:36,900][105620] Updated weights for policy 1, policy_version 1726161 (0.0006) [2023-12-27 03:56:36,956][105620] Updated weights for policy 1, policy_version 1726171 (0.0008) [2023-12-27 03:56:37,011][105620] Updated weights for policy 1, policy_version 1726181 (0.0009) [2023-12-27 03:56:37,060][105620] Updated weights for policy 1, policy_version 1726191 (0.0008) [2023-12-27 03:56:37,374][105692] Updated weights for policy 0, policy_version 1722601 (0.0010) [2023-12-27 03:56:37,434][105692] Updated weights for policy 0, policy_version 1722611 (0.0009) [2023-12-27 03:56:37,492][105692] Updated weights for policy 0, policy_version 1722621 (0.0008) [2023-12-27 03:56:37,865][105620] Updated weights for policy 1, policy_version 1726201 (0.0008) [2023-12-27 03:56:37,925][105620] Updated weights for policy 1, policy_version 1726211 (0.0009) [2023-12-27 03:56:37,984][105620] Updated weights for policy 1, policy_version 1726221 (0.0009) [2023-12-27 03:56:38,220][105692] Updated weights for policy 0, policy_version 1722631 (0.0009) [2023-12-27 03:56:38,279][105692] Updated weights for policy 0, policy_version 1722641 (0.0009) [2023-12-27 03:56:38,338][105692] Updated weights for policy 0, policy_version 1722651 (0.0009) [2023-12-27 03:56:38,764][105620] Updated weights for policy 1, policy_version 1726231 (0.0009) [2023-12-27 03:56:38,826][105620] Updated weights for policy 1, policy_version 1726241 (0.0008) [2023-12-27 03:56:38,890][105620] Updated weights for policy 1, policy_version 1726251 (0.0008) [2023-12-27 03:56:39,043][105692] Updated weights for policy 0, policy_version 1722661 (0.0009) [2023-12-27 03:56:39,095][105692] Updated weights for policy 0, policy_version 1722671 (0.0010) [2023-12-27 03:56:39,140][105692] Updated weights for policy 0, policy_version 1722681 (0.0009) [2023-12-27 03:56:39,699][105620] Updated weights for policy 1, policy_version 1726261 (0.0009) [2023-12-27 03:56:39,768][105620] Updated weights for policy 1, policy_version 1726271 (0.0008) [2023-12-27 03:56:39,777][105692] Updated weights for policy 0, policy_version 1722691 (0.0006) [2023-12-27 03:56:39,830][105692] Updated weights for policy 0, policy_version 1722701 (0.0007) [2023-12-27 03:56:39,833][105620] Updated weights for policy 1, policy_version 1726281 (0.0007) [2023-12-27 03:56:39,894][105692] Updated weights for policy 0, policy_version 1722711 (0.0010) [2023-12-27 03:56:40,469][105620] Updated weights for policy 1, policy_version 1726291 (0.0008) [2023-12-27 03:56:40,540][105620] Updated weights for policy 1, policy_version 1726301 (0.0008) [2023-12-27 03:56:40,604][105620] Updated weights for policy 1, policy_version 1726311 (0.0008) [2023-12-27 03:56:40,725][105692] Updated weights for policy 0, policy_version 1722721 (0.0009) [2023-12-27 03:56:40,791][105692] Updated weights for policy 0, policy_version 1722731 (0.0008) [2023-12-27 03:56:40,850][105692] Updated weights for policy 0, policy_version 1722741 (0.0005) [2023-12-27 03:56:40,906][105692] Updated weights for policy 0, policy_version 1722751 (0.0006) [2023-12-27 03:56:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 883089408. Throughput: 0: 9539.7, 1: 9732.5. Samples: 883096056. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:56:41,063][104569] Avg episode reward: [(0, '8527.649'), (1, '8805.282')] [2023-12-27 03:56:41,324][105620] Updated weights for policy 1, policy_version 1726321 (0.0007) [2023-12-27 03:56:41,387][105620] Updated weights for policy 1, policy_version 1726331 (0.0009) [2023-12-27 03:56:41,444][105620] Updated weights for policy 1, policy_version 1726341 (0.0008) [2023-12-27 03:56:41,500][105620] Updated weights for policy 1, policy_version 1726351 (0.0009) [2023-12-27 03:56:41,733][105692] Updated weights for policy 0, policy_version 1722761 (0.0009) [2023-12-27 03:56:41,800][105692] Updated weights for policy 0, policy_version 1722771 (0.0009) [2023-12-27 03:56:41,850][105692] Updated weights for policy 0, policy_version 1722781 (0.0008) [2023-12-27 03:56:42,228][105620] Updated weights for policy 1, policy_version 1726361 (0.0009) [2023-12-27 03:56:42,295][105620] Updated weights for policy 1, policy_version 1726371 (0.0008) [2023-12-27 03:56:42,359][105620] Updated weights for policy 1, policy_version 1726381 (0.0009) [2023-12-27 03:56:42,584][105692] Updated weights for policy 0, policy_version 1722791 (0.0006) [2023-12-27 03:56:42,645][105692] Updated weights for policy 0, policy_version 1722801 (0.0005) [2023-12-27 03:56:42,704][105692] Updated weights for policy 0, policy_version 1722811 (0.0006) [2023-12-27 03:56:43,217][105620] Updated weights for policy 1, policy_version 1726391 (0.0007) [2023-12-27 03:56:43,284][105620] Updated weights for policy 1, policy_version 1726401 (0.0005) [2023-12-27 03:56:43,326][105692] Updated weights for policy 0, policy_version 1722821 (0.0007) [2023-12-27 03:56:43,348][105620] Updated weights for policy 1, policy_version 1726411 (0.0005) [2023-12-27 03:56:43,377][105692] Updated weights for policy 0, policy_version 1722831 (0.0005) [2023-12-27 03:56:43,430][105692] Updated weights for policy 0, policy_version 1722842 (0.0009) [2023-12-27 03:56:43,977][105620] Updated weights for policy 1, policy_version 1726421 (0.0007) [2023-12-27 03:56:44,025][105620] Updated weights for policy 1, policy_version 1726431 (0.0009) [2023-12-27 03:56:44,072][105620] Updated weights for policy 1, policy_version 1726441 (0.0009) [2023-12-27 03:56:44,109][105692] Updated weights for policy 0, policy_version 1722853 (0.0008) [2023-12-27 03:56:44,157][105692] Updated weights for policy 0, policy_version 1722863 (0.0008) [2023-12-27 03:56:44,217][105692] Updated weights for policy 0, policy_version 1722873 (0.0010) [2023-12-27 03:56:44,675][105620] Updated weights for policy 1, policy_version 1726451 (0.0008) [2023-12-27 03:56:44,738][105620] Updated weights for policy 1, policy_version 1726461 (0.0007) [2023-12-27 03:56:44,805][105620] Updated weights for policy 1, policy_version 1726471 (0.0007) [2023-12-27 03:56:44,878][105692] Updated weights for policy 0, policy_version 1722883 (0.0008) [2023-12-27 03:56:44,949][105692] Updated weights for policy 0, policy_version 1722893 (0.0006) [2023-12-27 03:56:45,016][105692] Updated weights for policy 0, policy_version 1722903 (0.0007) [2023-12-27 03:56:45,407][105620] Updated weights for policy 1, policy_version 1726481 (0.0007) [2023-12-27 03:56:45,462][105620] Updated weights for policy 1, policy_version 1726491 (0.0009) [2023-12-27 03:56:45,521][105620] Updated weights for policy 1, policy_version 1726501 (0.0009) [2023-12-27 03:56:45,569][105620] Updated weights for policy 1, policy_version 1726511 (0.0009) [2023-12-27 03:56:45,736][105692] Updated weights for policy 0, policy_version 1722913 (0.0010) [2023-12-27 03:56:45,799][105692] Updated weights for policy 0, policy_version 1722924 (0.0010) [2023-12-27 03:56:45,847][105692] Updated weights for policy 0, policy_version 1722934 (0.0009) [2023-12-27 03:56:45,894][105692] Updated weights for policy 0, policy_version 1722944 (0.0009) [2023-12-27 03:56:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 883187712. Throughput: 0: 9554.1, 1: 9715.5. Samples: 883153752. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:56:46,063][104569] Avg episode reward: [(0, '8626.345'), (1, '8807.367')] [2023-12-27 03:56:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001722944_441139200.pth... [2023-12-27 03:56:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001726512_442048512.pth... [2023-12-27 03:56:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001725392_441761792.pth [2023-12-27 03:56:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001721824_440852480.pth [2023-12-27 03:56:46,301][105620] Updated weights for policy 1, policy_version 1726521 (0.0010) [2023-12-27 03:56:46,364][105620] Updated weights for policy 1, policy_version 1726531 (0.0010) [2023-12-27 03:56:46,405][105586] KL-divergence is very high: 124.0691 [2023-12-27 03:56:46,431][105620] Updated weights for policy 1, policy_version 1726541 (0.0010) [2023-12-27 03:56:46,583][105692] Updated weights for policy 0, policy_version 1722954 (0.0009) [2023-12-27 03:56:46,637][105692] Updated weights for policy 0, policy_version 1722965 (0.0010) [2023-12-27 03:56:46,696][105692] Updated weights for policy 0, policy_version 1722975 (0.0008) [2023-12-27 03:56:47,106][105620] Updated weights for policy 1, policy_version 1726551 (0.0007) [2023-12-27 03:56:47,161][105620] Updated weights for policy 1, policy_version 1726561 (0.0006) [2023-12-27 03:56:47,214][105620] Updated weights for policy 1, policy_version 1726571 (0.0005) [2023-12-27 03:56:47,350][105692] Updated weights for policy 0, policy_version 1722985 (0.0009) [2023-12-27 03:56:47,397][105692] Updated weights for policy 0, policy_version 1722995 (0.0010) [2023-12-27 03:56:47,465][105692] Updated weights for policy 0, policy_version 1723005 (0.0010) [2023-12-27 03:56:47,819][105620] Updated weights for policy 1, policy_version 1726581 (0.0009) [2023-12-27 03:56:47,893][105620] Updated weights for policy 1, policy_version 1726591 (0.0009) [2023-12-27 03:56:47,960][105620] Updated weights for policy 1, policy_version 1726601 (0.0010) [2023-12-27 03:56:48,120][105692] Updated weights for policy 0, policy_version 1723015 (0.0007) [2023-12-27 03:56:48,175][105692] Updated weights for policy 0, policy_version 1723025 (0.0009) [2023-12-27 03:56:48,228][105692] Updated weights for policy 0, policy_version 1723035 (0.0008) [2023-12-27 03:56:48,624][105620] Updated weights for policy 1, policy_version 1726611 (0.0010) [2023-12-27 03:56:48,673][105620] Updated weights for policy 1, policy_version 1726621 (0.0010) [2023-12-27 03:56:48,733][105620] Updated weights for policy 1, policy_version 1726631 (0.0010) [2023-12-27 03:56:48,958][105692] Updated weights for policy 0, policy_version 1723045 (0.0008) [2023-12-27 03:56:49,023][105692] Updated weights for policy 0, policy_version 1723055 (0.0008) [2023-12-27 03:56:49,085][105692] Updated weights for policy 0, policy_version 1723065 (0.0009) [2023-12-27 03:56:49,505][105620] Updated weights for policy 1, policy_version 1726641 (0.0010) [2023-12-27 03:56:49,576][105620] Updated weights for policy 1, policy_version 1726651 (0.0009) [2023-12-27 03:56:49,635][105620] Updated weights for policy 1, policy_version 1726661 (0.0007) [2023-12-27 03:56:49,694][105620] Updated weights for policy 1, policy_version 1726671 (0.0009) [2023-12-27 03:56:49,794][105692] Updated weights for policy 0, policy_version 1723075 (0.0008) [2023-12-27 03:56:49,861][105692] Updated weights for policy 0, policy_version 1723085 (0.0007) [2023-12-27 03:56:49,927][105692] Updated weights for policy 0, policy_version 1723095 (0.0009) [2023-12-27 03:56:50,354][105620] Updated weights for policy 1, policy_version 1726681 (0.0008) [2023-12-27 03:56:50,407][105620] Updated weights for policy 1, policy_version 1726691 (0.0008) [2023-12-27 03:56:50,455][105620] Updated weights for policy 1, policy_version 1726701 (0.0009) [2023-12-27 03:56:50,559][105692] Updated weights for policy 0, policy_version 1723105 (0.0010) [2023-12-27 03:56:50,630][105692] Updated weights for policy 0, policy_version 1723115 (0.0009) [2023-12-27 03:56:50,690][105692] Updated weights for policy 0, policy_version 1723125 (0.0008) [2023-12-27 03:56:50,756][105692] Updated weights for policy 0, policy_version 1723135 (0.0006) [2023-12-27 03:56:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 883286016. Throughput: 0: 9688.2, 1: 9723.1. Samples: 883274832. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:56:51,063][104569] Avg episode reward: [(0, '8541.103'), (1, '8989.388')] [2023-12-27 03:56:51,131][105620] Updated weights for policy 1, policy_version 1726711 (0.0009) [2023-12-27 03:56:51,199][105620] Updated weights for policy 1, policy_version 1726721 (0.0009) [2023-12-27 03:56:51,263][105620] Updated weights for policy 1, policy_version 1726731 (0.0008) [2023-12-27 03:56:51,459][105692] Updated weights for policy 0, policy_version 1723145 (0.0008) [2023-12-27 03:56:51,524][105692] Updated weights for policy 0, policy_version 1723155 (0.0009) [2023-12-27 03:56:51,588][105692] Updated weights for policy 0, policy_version 1723165 (0.0009) [2023-12-27 03:56:51,998][105620] Updated weights for policy 1, policy_version 1726741 (0.0008) [2023-12-27 03:56:52,058][105620] Updated weights for policy 1, policy_version 1726751 (0.0008) [2023-12-27 03:56:52,117][105620] Updated weights for policy 1, policy_version 1726761 (0.0009) [2023-12-27 03:56:52,325][105692] Updated weights for policy 0, policy_version 1723175 (0.0008) [2023-12-27 03:56:52,391][105692] Updated weights for policy 0, policy_version 1723185 (0.0009) [2023-12-27 03:56:52,450][105692] Updated weights for policy 0, policy_version 1723195 (0.0009) [2023-12-27 03:56:52,853][105620] Updated weights for policy 1, policy_version 1726771 (0.0009) [2023-12-27 03:56:52,912][105620] Updated weights for policy 1, policy_version 1726781 (0.0006) [2023-12-27 03:56:52,967][105620] Updated weights for policy 1, policy_version 1726791 (0.0006) [2023-12-27 03:56:53,244][105692] Updated weights for policy 0, policy_version 1723205 (0.0009) [2023-12-27 03:56:53,299][105692] Updated weights for policy 0, policy_version 1723216 (0.0010) [2023-12-27 03:56:53,348][105692] Updated weights for policy 0, policy_version 1723226 (0.0009) [2023-12-27 03:56:53,614][105620] Updated weights for policy 1, policy_version 1726801 (0.0006) [2023-12-27 03:56:53,683][105620] Updated weights for policy 1, policy_version 1726811 (0.0007) [2023-12-27 03:56:53,755][105620] Updated weights for policy 1, policy_version 1726821 (0.0006) [2023-12-27 03:56:53,814][105620] Updated weights for policy 1, policy_version 1726831 (0.0008) [2023-12-27 03:56:54,114][105692] Updated weights for policy 0, policy_version 1723236 (0.0009) [2023-12-27 03:56:54,181][105692] Updated weights for policy 0, policy_version 1723246 (0.0011) [2023-12-27 03:56:54,231][105692] Updated weights for policy 0, policy_version 1723256 (0.0010) [2023-12-27 03:56:54,521][105620] Updated weights for policy 1, policy_version 1726841 (0.0009) [2023-12-27 03:56:54,578][105620] Updated weights for policy 1, policy_version 1726851 (0.0010) [2023-12-27 03:56:54,640][105620] Updated weights for policy 1, policy_version 1726862 (0.0009) [2023-12-27 03:56:54,850][105692] Updated weights for policy 0, policy_version 1723266 (0.0010) [2023-12-27 03:56:54,905][105692] Updated weights for policy 0, policy_version 1723276 (0.0009) [2023-12-27 03:56:54,959][105692] Updated weights for policy 0, policy_version 1723286 (0.0009) [2023-12-27 03:56:55,021][105692] Updated weights for policy 0, policy_version 1723296 (0.0008) [2023-12-27 03:56:55,344][105620] Updated weights for policy 1, policy_version 1726872 (0.0007) [2023-12-27 03:56:55,410][105620] Updated weights for policy 1, policy_version 1726882 (0.0008) [2023-12-27 03:56:55,475][105620] Updated weights for policy 1, policy_version 1726892 (0.0007) [2023-12-27 03:56:55,739][105692] Updated weights for policy 0, policy_version 1723306 (0.0006) [2023-12-27 03:56:55,790][105692] Updated weights for policy 0, policy_version 1723316 (0.0007) [2023-12-27 03:56:55,838][105692] Updated weights for policy 0, policy_version 1723326 (0.0009) [2023-12-27 03:56:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 883384320. Throughput: 0: 9727.7, 1: 9724.5. Samples: 883392332. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:56:56,062][104569] Avg episode reward: [(0, '8717.350'), (1, '8988.066')] [2023-12-27 03:56:56,191][105620] Updated weights for policy 1, policy_version 1726902 (0.0008) [2023-12-27 03:56:56,255][105620] Updated weights for policy 1, policy_version 1726912 (0.0009) [2023-12-27 03:56:56,308][105620] Updated weights for policy 1, policy_version 1726922 (0.0009) [2023-12-27 03:56:56,607][105692] Updated weights for policy 0, policy_version 1723336 (0.0009) [2023-12-27 03:56:56,656][105692] Updated weights for policy 0, policy_version 1723346 (0.0009) [2023-12-27 03:56:56,705][105692] Updated weights for policy 0, policy_version 1723356 (0.0009) [2023-12-27 03:56:57,000][105620] Updated weights for policy 1, policy_version 1726932 (0.0008) [2023-12-27 03:56:57,063][105620] Updated weights for policy 1, policy_version 1726942 (0.0005) [2023-12-27 03:56:57,126][105620] Updated weights for policy 1, policy_version 1726952 (0.0006) [2023-12-27 03:56:57,453][105692] Updated weights for policy 0, policy_version 1723366 (0.0009) [2023-12-27 03:56:57,508][105692] Updated weights for policy 0, policy_version 1723377 (0.0011) [2023-12-27 03:56:57,562][105692] Updated weights for policy 0, policy_version 1723388 (0.0010) [2023-12-27 03:56:57,648][105620] Updated weights for policy 1, policy_version 1726962 (0.0008) [2023-12-27 03:56:57,694][105620] Updated weights for policy 1, policy_version 1726972 (0.0008) [2023-12-27 03:56:57,739][105620] Updated weights for policy 1, policy_version 1726982 (0.0008) [2023-12-27 03:56:57,786][105620] Updated weights for policy 1, policy_version 1726992 (0.0009) [2023-12-27 03:56:58,310][105692] Updated weights for policy 0, policy_version 1723398 (0.0008) [2023-12-27 03:56:58,381][105692] Updated weights for policy 0, policy_version 1723408 (0.0007) [2023-12-27 03:56:58,445][105692] Updated weights for policy 0, policy_version 1723418 (0.0008) [2023-12-27 03:56:58,505][105620] Updated weights for policy 1, policy_version 1727002 (0.0011) [2023-12-27 03:56:58,568][105620] Updated weights for policy 1, policy_version 1727012 (0.0011) [2023-12-27 03:56:58,626][105620] Updated weights for policy 1, policy_version 1727022 (0.0010) [2023-12-27 03:56:59,237][105692] Updated weights for policy 0, policy_version 1723428 (0.0008) [2023-12-27 03:56:59,294][105692] Updated weights for policy 0, policy_version 1723438 (0.0009) [2023-12-27 03:56:59,352][105692] Updated weights for policy 0, policy_version 1723448 (0.0008) [2023-12-27 03:56:59,435][105620] Updated weights for policy 1, policy_version 1727032 (0.0006) [2023-12-27 03:56:59,484][105620] Updated weights for policy 1, policy_version 1727042 (0.0005) [2023-12-27 03:56:59,533][105620] Updated weights for policy 1, policy_version 1727052 (0.0005) [2023-12-27 03:57:00,131][105620] Updated weights for policy 1, policy_version 1727062 (0.0007) [2023-12-27 03:57:00,185][105620] Updated weights for policy 1, policy_version 1727072 (0.0008) [2023-12-27 03:57:00,187][105692] Updated weights for policy 0, policy_version 1723458 (0.0009) [2023-12-27 03:57:00,242][105620] Updated weights for policy 1, policy_version 1727082 (0.0007) [2023-12-27 03:57:00,244][105692] Updated weights for policy 0, policy_version 1723468 (0.0007) [2023-12-27 03:57:00,300][105692] Updated weights for policy 0, policy_version 1723478 (0.0009) [2023-12-27 03:57:00,351][105692] Updated weights for policy 0, policy_version 1723488 (0.0009) [2023-12-27 03:57:00,905][105620] Updated weights for policy 1, policy_version 1727092 (0.0007) [2023-12-27 03:57:00,966][105620] Updated weights for policy 1, policy_version 1727102 (0.0009) [2023-12-27 03:57:01,023][105620] Updated weights for policy 1, policy_version 1727112 (0.0009) [2023-12-27 03:57:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 883474432. Throughput: 0: 9691.9, 1: 9771.9. Samples: 883451640. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:57:01,063][104569] Avg episode reward: [(0, '8349.911'), (1, '8897.407')] [2023-12-27 03:57:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001723488_441278464.pth... [2023-12-27 03:57:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001722368_440991744.pth [2023-12-27 03:57:01,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001727120_442204160.pth... [2023-12-27 03:57:01,099][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001725968_441909248.pth [2023-12-27 03:57:01,170][105692] Updated weights for policy 0, policy_version 1723498 (0.0007) [2023-12-27 03:57:01,231][105692] Updated weights for policy 0, policy_version 1723508 (0.0006) [2023-12-27 03:57:01,295][105692] Updated weights for policy 0, policy_version 1723518 (0.0008) [2023-12-27 03:57:01,698][105620] Updated weights for policy 1, policy_version 1727122 (0.0009) [2023-12-27 03:57:01,770][105620] Updated weights for policy 1, policy_version 1727132 (0.0006) [2023-12-27 03:57:01,829][105620] Updated weights for policy 1, policy_version 1727142 (0.0007) [2023-12-27 03:57:01,888][105620] Updated weights for policy 1, policy_version 1727152 (0.0007) [2023-12-27 03:57:02,042][105692] Updated weights for policy 0, policy_version 1723528 (0.0009) [2023-12-27 03:57:02,090][105692] Updated weights for policy 0, policy_version 1723538 (0.0009) [2023-12-27 03:57:02,144][105692] Updated weights for policy 0, policy_version 1723548 (0.0006) [2023-12-27 03:57:02,602][105620] Updated weights for policy 1, policy_version 1727162 (0.0005) [2023-12-27 03:57:02,670][105620] Updated weights for policy 1, policy_version 1727172 (0.0005) [2023-12-27 03:57:02,728][105620] Updated weights for policy 1, policy_version 1727182 (0.0005) [2023-12-27 03:57:02,831][105692] Updated weights for policy 0, policy_version 1723558 (0.0008) [2023-12-27 03:57:02,892][105692] Updated weights for policy 0, policy_version 1723568 (0.0009) [2023-12-27 03:57:02,946][105692] Updated weights for policy 0, policy_version 1723578 (0.0007) [2023-12-27 03:57:03,415][105620] Updated weights for policy 1, policy_version 1727192 (0.0008) [2023-12-27 03:57:03,479][105620] Updated weights for policy 1, policy_version 1727202 (0.0009) [2023-12-27 03:57:03,503][105692] Updated weights for policy 0, policy_version 1723588 (0.0005) [2023-12-27 03:57:03,538][105620] Updated weights for policy 1, policy_version 1727212 (0.0009) [2023-12-27 03:57:03,548][105692] Updated weights for policy 0, policy_version 1723598 (0.0005) [2023-12-27 03:57:03,599][105692] Updated weights for policy 0, policy_version 1723608 (0.0007) [2023-12-27 03:57:04,255][105692] Updated weights for policy 0, policy_version 1723618 (0.0008) [2023-12-27 03:57:04,321][105692] Updated weights for policy 0, policy_version 1723628 (0.0006) [2023-12-27 03:57:04,372][105620] Updated weights for policy 1, policy_version 1727222 (0.0008) [2023-12-27 03:57:04,383][105692] Updated weights for policy 0, policy_version 1723638 (0.0006) [2023-12-27 03:57:04,431][105620] Updated weights for policy 1, policy_version 1727232 (0.0008) [2023-12-27 03:57:04,444][105692] Updated weights for policy 0, policy_version 1723648 (0.0007) [2023-12-27 03:57:04,494][105620] Updated weights for policy 1, policy_version 1727242 (0.0007) [2023-12-27 03:57:05,095][105692] Updated weights for policy 0, policy_version 1723658 (0.0005) [2023-12-27 03:57:05,158][105692] Updated weights for policy 0, policy_version 1723668 (0.0005) [2023-12-27 03:57:05,223][105692] Updated weights for policy 0, policy_version 1723678 (0.0006) [2023-12-27 03:57:05,311][105620] Updated weights for policy 1, policy_version 1727252 (0.0007) [2023-12-27 03:57:05,362][105620] Updated weights for policy 1, policy_version 1727262 (0.0005) [2023-12-27 03:57:05,408][105620] Updated weights for policy 1, policy_version 1727272 (0.0006) [2023-12-27 03:57:05,731][105692] Updated weights for policy 0, policy_version 1723688 (0.0005) [2023-12-27 03:57:05,789][105692] Updated weights for policy 0, policy_version 1723698 (0.0009) [2023-12-27 03:57:05,841][105692] Updated weights for policy 0, policy_version 1723708 (0.0009) [2023-12-27 03:57:06,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 883580928. Throughput: 0: 9683.2, 1: 9769.2. Samples: 883568292. Policy #0 lag: (min: 22.0, avg: 27.9, max: 54.0) [2023-12-27 03:57:06,063][104569] Avg episode reward: [(0, '8344.514'), (1, '8805.412')] [2023-12-27 03:57:06,229][105620] Updated weights for policy 1, policy_version 1727282 (0.0010) [2023-12-27 03:57:06,287][105620] Updated weights for policy 1, policy_version 1727292 (0.0009) [2023-12-27 03:57:06,334][105620] Updated weights for policy 1, policy_version 1727302 (0.0008) [2023-12-27 03:57:06,385][105620] Updated weights for policy 1, policy_version 1727312 (0.0009) [2023-12-27 03:57:06,442][105692] Updated weights for policy 0, policy_version 1723718 (0.0009) [2023-12-27 03:57:06,510][105692] Updated weights for policy 0, policy_version 1723728 (0.0009) [2023-12-27 03:57:06,570][105692] Updated weights for policy 0, policy_version 1723738 (0.0006) [2023-12-27 03:57:07,182][105620] Updated weights for policy 1, policy_version 1727322 (0.0008) [2023-12-27 03:57:07,229][105620] Updated weights for policy 1, policy_version 1727332 (0.0009) [2023-12-27 03:57:07,288][105620] Updated weights for policy 1, policy_version 1727342 (0.0009) [2023-12-27 03:57:07,303][105692] Updated weights for policy 0, policy_version 1723748 (0.0006) [2023-12-27 03:57:07,358][105692] Updated weights for policy 0, policy_version 1723758 (0.0009) [2023-12-27 03:57:07,414][105692] Updated weights for policy 0, policy_version 1723768 (0.0009) [2023-12-27 03:57:08,063][105620] Updated weights for policy 1, policy_version 1727352 (0.0009) [2023-12-27 03:57:08,124][105620] Updated weights for policy 1, policy_version 1727362 (0.0009) [2023-12-27 03:57:08,185][105620] Updated weights for policy 1, policy_version 1727372 (0.0008) [2023-12-27 03:57:08,191][105692] Updated weights for policy 0, policy_version 1723778 (0.0009) [2023-12-27 03:57:08,245][105692] Updated weights for policy 0, policy_version 1723788 (0.0008) [2023-12-27 03:57:08,304][105692] Updated weights for policy 0, policy_version 1723798 (0.0009) [2023-12-27 03:57:08,370][105692] Updated weights for policy 0, policy_version 1723808 (0.0008) [2023-12-27 03:57:08,956][105620] Updated weights for policy 1, policy_version 1727382 (0.0008) [2023-12-27 03:57:09,020][105620] Updated weights for policy 1, policy_version 1727392 (0.0009) [2023-12-27 03:57:09,079][105620] Updated weights for policy 1, policy_version 1727402 (0.0008) [2023-12-27 03:57:09,081][105692] Updated weights for policy 0, policy_version 1723818 (0.0006) [2023-12-27 03:57:09,132][105692] Updated weights for policy 0, policy_version 1723828 (0.0008) [2023-12-27 03:57:09,187][105692] Updated weights for policy 0, policy_version 1723838 (0.0009) [2023-12-27 03:57:09,841][105620] Updated weights for policy 1, policy_version 1727412 (0.0010) [2023-12-27 03:57:09,907][105620] Updated weights for policy 1, policy_version 1727422 (0.0009) [2023-12-27 03:57:09,969][105620] Updated weights for policy 1, policy_version 1727432 (0.0006) [2023-12-27 03:57:09,972][105692] Updated weights for policy 0, policy_version 1723848 (0.0008) [2023-12-27 03:57:10,031][105692] Updated weights for policy 0, policy_version 1723858 (0.0008) [2023-12-27 03:57:10,096][105692] Updated weights for policy 0, policy_version 1723868 (0.0006) [2023-12-27 03:57:10,736][105620] Updated weights for policy 1, policy_version 1727442 (0.0006) [2023-12-27 03:57:10,777][105692] Updated weights for policy 0, policy_version 1723878 (0.0009) [2023-12-27 03:57:10,795][105620] Updated weights for policy 1, policy_version 1727452 (0.0006) [2023-12-27 03:57:10,837][105692] Updated weights for policy 0, policy_version 1723888 (0.0011) [2023-12-27 03:57:10,844][105620] Updated weights for policy 1, policy_version 1727462 (0.0006) [2023-12-27 03:57:10,893][105692] Updated weights for policy 0, policy_version 1723898 (0.0011) [2023-12-27 03:57:10,896][105620] Updated weights for policy 1, policy_version 1727472 (0.0007) [2023-12-27 03:57:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 883679232. Throughput: 0: 9758.9, 1: 9661.6. Samples: 883682828. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:57:11,063][104569] Avg episode reward: [(0, '8526.633'), (1, '8803.712')] [2023-12-27 03:57:11,685][105692] Updated weights for policy 0, policy_version 1723908 (0.0010) [2023-12-27 03:57:11,700][105620] Updated weights for policy 1, policy_version 1727482 (0.0007) [2023-12-27 03:57:11,751][105692] Updated weights for policy 0, policy_version 1723918 (0.0010) [2023-12-27 03:57:11,763][105620] Updated weights for policy 1, policy_version 1727492 (0.0008) [2023-12-27 03:57:11,811][105692] Updated weights for policy 0, policy_version 1723928 (0.0010) [2023-12-27 03:57:11,823][105620] Updated weights for policy 1, policy_version 1727502 (0.0008) [2023-12-27 03:57:12,500][105692] Updated weights for policy 0, policy_version 1723938 (0.0006) [2023-12-27 03:57:12,510][105620] Updated weights for policy 1, policy_version 1727512 (0.0008) [2023-12-27 03:57:12,564][105620] Updated weights for policy 1, policy_version 1727522 (0.0009) [2023-12-27 03:57:12,565][105692] Updated weights for policy 0, policy_version 1723948 (0.0006) [2023-12-27 03:57:12,624][105620] Updated weights for policy 1, policy_version 1727532 (0.0008) [2023-12-27 03:57:12,628][105692] Updated weights for policy 0, policy_version 1723958 (0.0007) [2023-12-27 03:57:12,686][105692] Updated weights for policy 0, policy_version 1723968 (0.0008) [2023-12-27 03:57:13,296][105692] Updated weights for policy 0, policy_version 1723978 (0.0010) [2023-12-27 03:57:13,344][105692] Updated weights for policy 0, policy_version 1723988 (0.0010) [2023-12-27 03:57:13,402][105692] Updated weights for policy 0, policy_version 1723998 (0.0010) [2023-12-27 03:57:13,456][105620] Updated weights for policy 1, policy_version 1727542 (0.0008) [2023-12-27 03:57:13,508][105620] Updated weights for policy 1, policy_version 1727552 (0.0008) [2023-12-27 03:57:13,552][105620] Updated weights for policy 1, policy_version 1727562 (0.0007) [2023-12-27 03:57:14,154][105692] Updated weights for policy 0, policy_version 1724009 (0.0010) [2023-12-27 03:57:14,212][105692] Updated weights for policy 0, policy_version 1724019 (0.0009) [2023-12-27 03:57:14,267][105692] Updated weights for policy 0, policy_version 1724029 (0.0010) [2023-12-27 03:57:14,277][105620] Updated weights for policy 1, policy_version 1727572 (0.0006) [2023-12-27 03:57:14,340][105620] Updated weights for policy 1, policy_version 1727582 (0.0005) [2023-12-27 03:57:14,398][105620] Updated weights for policy 1, policy_version 1727592 (0.0009) [2023-12-27 03:57:15,088][105692] Updated weights for policy 0, policy_version 1724039 (0.0009) [2023-12-27 03:57:15,114][105620] Updated weights for policy 1, policy_version 1727602 (0.0008) [2023-12-27 03:57:15,137][105692] Updated weights for policy 0, policy_version 1724049 (0.0008) [2023-12-27 03:57:15,177][105620] Updated weights for policy 1, policy_version 1727612 (0.0008) [2023-12-27 03:57:15,184][105692] Updated weights for policy 0, policy_version 1724059 (0.0009) [2023-12-27 03:57:15,233][105620] Updated weights for policy 1, policy_version 1727622 (0.0007) [2023-12-27 03:57:15,285][105620] Updated weights for policy 1, policy_version 1727632 (0.0009) [2023-12-27 03:57:15,889][105692] Updated weights for policy 0, policy_version 1724069 (0.0006) [2023-12-27 03:57:15,945][105692] Updated weights for policy 0, policy_version 1724079 (0.0005) [2023-12-27 03:57:15,949][105620] Updated weights for policy 1, policy_version 1727642 (0.0010) [2023-12-27 03:57:15,997][105620] Updated weights for policy 1, policy_version 1727652 (0.0010) [2023-12-27 03:57:15,998][105692] Updated weights for policy 0, policy_version 1724089 (0.0009) [2023-12-27 03:57:16,046][105620] Updated weights for policy 1, policy_version 1727662 (0.0009) [2023-12-27 03:57:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 883777536. Throughput: 0: 9728.8, 1: 9637.9. Samples: 883738932. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:57:16,063][104569] Avg episode reward: [(0, '8072.812'), (1, '8987.346')] [2023-12-27 03:57:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001724096_441434112.pth... [2023-12-27 03:57:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001727664_442343424.pth... [2023-12-27 03:57:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001722944_441139200.pth [2023-12-27 03:57:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001726512_442048512.pth [2023-12-27 03:57:16,584][105692] Updated weights for policy 0, policy_version 1724099 (0.0009) [2023-12-27 03:57:16,636][105692] Updated weights for policy 0, policy_version 1724109 (0.0005) [2023-12-27 03:57:16,659][105620] Updated weights for policy 1, policy_version 1727672 (0.0005) [2023-12-27 03:57:16,690][105692] Updated weights for policy 0, policy_version 1724119 (0.0005) [2023-12-27 03:57:16,715][105620] Updated weights for policy 1, policy_version 1727682 (0.0010) [2023-12-27 03:57:16,763][105620] Updated weights for policy 1, policy_version 1727692 (0.0010) [2023-12-27 03:57:17,391][105692] Updated weights for policy 0, policy_version 1724129 (0.0009) [2023-12-27 03:57:17,402][105620] Updated weights for policy 1, policy_version 1727702 (0.0009) [2023-12-27 03:57:17,456][105692] Updated weights for policy 0, policy_version 1724139 (0.0005) [2023-12-27 03:57:17,458][105620] Updated weights for policy 1, policy_version 1727712 (0.0005) [2023-12-27 03:57:17,516][105692] Updated weights for policy 0, policy_version 1724149 (0.0005) [2023-12-27 03:57:17,517][105620] Updated weights for policy 1, policy_version 1727722 (0.0009) [2023-12-27 03:57:17,570][105692] Updated weights for policy 0, policy_version 1724159 (0.0009) [2023-12-27 03:57:18,158][105620] Updated weights for policy 1, policy_version 1727732 (0.0009) [2023-12-27 03:57:18,221][105620] Updated weights for policy 1, policy_version 1727742 (0.0007) [2023-12-27 03:57:18,238][105692] Updated weights for policy 0, policy_version 1724169 (0.0010) [2023-12-27 03:57:18,280][105620] Updated weights for policy 1, policy_version 1727752 (0.0006) [2023-12-27 03:57:18,293][105692] Updated weights for policy 0, policy_version 1724179 (0.0010) [2023-12-27 03:57:18,348][105692] Updated weights for policy 0, policy_version 1724189 (0.0010) [2023-12-27 03:57:19,052][105620] Updated weights for policy 1, policy_version 1727762 (0.0007) [2023-12-27 03:57:19,086][105692] Updated weights for policy 0, policy_version 1724199 (0.0010) [2023-12-27 03:57:19,112][105620] Updated weights for policy 1, policy_version 1727772 (0.0005) [2023-12-27 03:57:19,144][105692] Updated weights for policy 0, policy_version 1724209 (0.0010) [2023-12-27 03:57:19,174][105620] Updated weights for policy 1, policy_version 1727782 (0.0006) [2023-12-27 03:57:19,204][105692] Updated weights for policy 0, policy_version 1724219 (0.0010) [2023-12-27 03:57:19,226][105620] Updated weights for policy 1, policy_version 1727792 (0.0007) [2023-12-27 03:57:19,981][105692] Updated weights for policy 0, policy_version 1724229 (0.0010) [2023-12-27 03:57:20,037][105692] Updated weights for policy 0, policy_version 1724239 (0.0007) [2023-12-27 03:57:20,044][105620] Updated weights for policy 1, policy_version 1727802 (0.0007) [2023-12-27 03:57:20,087][105692] Updated weights for policy 0, policy_version 1724249 (0.0008) [2023-12-27 03:57:20,095][105620] Updated weights for policy 1, policy_version 1727812 (0.0007) [2023-12-27 03:57:20,141][105620] Updated weights for policy 1, policy_version 1727822 (0.0006) [2023-12-27 03:57:20,888][105620] Updated weights for policy 1, policy_version 1727832 (0.0008) [2023-12-27 03:57:20,890][105692] Updated weights for policy 0, policy_version 1724259 (0.0008) [2023-12-27 03:57:20,946][105620] Updated weights for policy 1, policy_version 1727842 (0.0007) [2023-12-27 03:57:20,954][105692] Updated weights for policy 0, policy_version 1724269 (0.0006) [2023-12-27 03:57:21,003][105620] Updated weights for policy 1, policy_version 1727852 (0.0010) [2023-12-27 03:57:21,018][105692] Updated weights for policy 0, policy_version 1724279 (0.0010) [2023-12-27 03:57:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 883867648. Throughput: 0: 9808.9, 1: 9697.1. Samples: 883858896. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:57:21,062][104569] Avg episode reward: [(0, '7893.221'), (1, '9085.867')] [2023-12-27 03:57:21,694][105620] Updated weights for policy 1, policy_version 1727862 (0.0007) [2023-12-27 03:57:21,760][105620] Updated weights for policy 1, policy_version 1727872 (0.0009) [2023-12-27 03:57:21,823][105620] Updated weights for policy 1, policy_version 1727882 (0.0009) [2023-12-27 03:57:21,838][105692] Updated weights for policy 0, policy_version 1724289 (0.0010) [2023-12-27 03:57:21,905][105692] Updated weights for policy 0, policy_version 1724299 (0.0009) [2023-12-27 03:57:21,965][105692] Updated weights for policy 0, policy_version 1724309 (0.0010) [2023-12-27 03:57:22,034][105692] Updated weights for policy 0, policy_version 1724319 (0.0008) [2023-12-27 03:57:22,536][105620] Updated weights for policy 1, policy_version 1727892 (0.0008) [2023-12-27 03:57:22,596][105620] Updated weights for policy 1, policy_version 1727902 (0.0008) [2023-12-27 03:57:22,655][105620] Updated weights for policy 1, policy_version 1727912 (0.0005) [2023-12-27 03:57:22,745][105692] Updated weights for policy 0, policy_version 1724329 (0.0010) [2023-12-27 03:57:22,797][105692] Updated weights for policy 0, policy_version 1724339 (0.0010) [2023-12-27 03:57:22,856][105692] Updated weights for policy 0, policy_version 1724349 (0.0005) [2023-12-27 03:57:23,416][105620] Updated weights for policy 1, policy_version 1727922 (0.0006) [2023-12-27 03:57:23,476][105620] Updated weights for policy 1, policy_version 1727932 (0.0006) [2023-12-27 03:57:23,536][105620] Updated weights for policy 1, policy_version 1727942 (0.0007) [2023-12-27 03:57:23,588][105620] Updated weights for policy 1, policy_version 1727952 (0.0010) [2023-12-27 03:57:23,635][105692] Updated weights for policy 0, policy_version 1724359 (0.0006) [2023-12-27 03:57:23,689][105692] Updated weights for policy 0, policy_version 1724369 (0.0007) [2023-12-27 03:57:23,740][105692] Updated weights for policy 0, policy_version 1724379 (0.0007) [2023-12-27 03:57:24,319][105620] Updated weights for policy 1, policy_version 1727962 (0.0010) [2023-12-27 03:57:24,376][105620] Updated weights for policy 1, policy_version 1727972 (0.0010) [2023-12-27 03:57:24,426][105620] Updated weights for policy 1, policy_version 1727982 (0.0009) [2023-12-27 03:57:24,448][105692] Updated weights for policy 0, policy_version 1724389 (0.0007) [2023-12-27 03:57:24,506][105692] Updated weights for policy 0, policy_version 1724399 (0.0008) [2023-12-27 03:57:24,562][105692] Updated weights for policy 0, policy_version 1724409 (0.0008) [2023-12-27 03:57:25,159][105620] Updated weights for policy 1, policy_version 1727992 (0.0006) [2023-12-27 03:57:25,216][105620] Updated weights for policy 1, policy_version 1728002 (0.0005) [2023-12-27 03:57:25,269][105620] Updated weights for policy 1, policy_version 1728012 (0.0005) [2023-12-27 03:57:25,377][105692] Updated weights for policy 0, policy_version 1724419 (0.0008) [2023-12-27 03:57:25,424][105692] Updated weights for policy 0, policy_version 1724429 (0.0007) [2023-12-27 03:57:25,479][105692] Updated weights for policy 0, policy_version 1724439 (0.0007) [2023-12-27 03:57:25,803][105620] Updated weights for policy 1, policy_version 1728022 (0.0005) [2023-12-27 03:57:25,854][105620] Updated weights for policy 1, policy_version 1728032 (0.0005) [2023-12-27 03:57:25,906][105620] Updated weights for policy 1, policy_version 1728042 (0.0007) [2023-12-27 03:57:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 883965952. Throughput: 0: 9713.6, 1: 9771.4. Samples: 883972880. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:57:26,063][104569] Avg episode reward: [(0, '8529.460'), (1, '9177.706')] [2023-12-27 03:57:26,155][105692] Updated weights for policy 0, policy_version 1724449 (0.0006) [2023-12-27 03:57:26,211][105692] Updated weights for policy 0, policy_version 1724459 (0.0010) [2023-12-27 03:57:26,266][105692] Updated weights for policy 0, policy_version 1724469 (0.0009) [2023-12-27 03:57:26,320][105692] Updated weights for policy 0, policy_version 1724479 (0.0009) [2023-12-27 03:57:26,522][105620] Updated weights for policy 1, policy_version 1728052 (0.0010) [2023-12-27 03:57:26,570][105620] Updated weights for policy 1, policy_version 1728062 (0.0010) [2023-12-27 03:57:26,617][105620] Updated weights for policy 1, policy_version 1728072 (0.0010) [2023-12-27 03:57:27,097][105692] Updated weights for policy 0, policy_version 1724489 (0.0008) [2023-12-27 03:57:27,145][105692] Updated weights for policy 0, policy_version 1724499 (0.0007) [2023-12-27 03:57:27,188][105692] Updated weights for policy 0, policy_version 1724509 (0.0008) [2023-12-27 03:57:27,384][105620] Updated weights for policy 1, policy_version 1728082 (0.0010) [2023-12-27 03:57:27,438][105620] Updated weights for policy 1, policy_version 1728092 (0.0010) [2023-12-27 03:57:27,495][105620] Updated weights for policy 1, policy_version 1728102 (0.0010) [2023-12-27 03:57:27,539][105620] Updated weights for policy 1, policy_version 1728112 (0.0010) [2023-12-27 03:57:27,954][105692] Updated weights for policy 0, policy_version 1724519 (0.0007) [2023-12-27 03:57:28,012][105692] Updated weights for policy 0, policy_version 1724529 (0.0008) [2023-12-27 03:57:28,059][105692] Updated weights for policy 0, policy_version 1724539 (0.0008) [2023-12-27 03:57:28,294][105620] Updated weights for policy 1, policy_version 1728122 (0.0011) [2023-12-27 03:57:28,359][105620] Updated weights for policy 1, policy_version 1728132 (0.0008) [2023-12-27 03:57:28,421][105620] Updated weights for policy 1, policy_version 1728142 (0.0007) [2023-12-27 03:57:28,780][105692] Updated weights for policy 0, policy_version 1724549 (0.0008) [2023-12-27 03:57:28,828][105692] Updated weights for policy 0, policy_version 1724559 (0.0008) [2023-12-27 03:57:28,876][105692] Updated weights for policy 0, policy_version 1724569 (0.0007) [2023-12-27 03:57:29,135][105620] Updated weights for policy 1, policy_version 1728152 (0.0009) [2023-12-27 03:57:29,180][105620] Updated weights for policy 1, policy_version 1728162 (0.0010) [2023-12-27 03:57:29,232][105620] Updated weights for policy 1, policy_version 1728172 (0.0009) [2023-12-27 03:57:29,671][105692] Updated weights for policy 0, policy_version 1724579 (0.0008) [2023-12-27 03:57:29,725][105692] Updated weights for policy 0, policy_version 1724589 (0.0008) [2023-12-27 03:57:29,779][105692] Updated weights for policy 0, policy_version 1724599 (0.0008) [2023-12-27 03:57:29,998][105620] Updated weights for policy 1, policy_version 1728182 (0.0011) [2023-12-27 03:57:30,050][105620] Updated weights for policy 1, policy_version 1728192 (0.0010) [2023-12-27 03:57:30,106][105620] Updated weights for policy 1, policy_version 1728202 (0.0008) [2023-12-27 03:57:30,622][105692] Updated weights for policy 0, policy_version 1724609 (0.0008) [2023-12-27 03:57:30,672][105692] Updated weights for policy 0, policy_version 1724619 (0.0008) [2023-12-27 03:57:30,728][105692] Updated weights for policy 0, policy_version 1724629 (0.0008) [2023-12-27 03:57:30,733][105620] Updated weights for policy 1, policy_version 1728212 (0.0007) [2023-12-27 03:57:30,778][105692] Updated weights for policy 0, policy_version 1724639 (0.0005) [2023-12-27 03:57:30,788][105620] Updated weights for policy 1, policy_version 1728222 (0.0010) [2023-12-27 03:57:30,842][105620] Updated weights for policy 1, policy_version 1728232 (0.0010) [2023-12-27 03:57:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 884064256. Throughput: 0: 9716.1, 1: 9786.5. Samples: 884031368. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:57:31,062][104569] Avg episode reward: [(0, '8525.818'), (1, '9079.878')] [2023-12-27 03:57:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001724640_441573376.pth... [2023-12-27 03:57:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001728240_442490880.pth... [2023-12-27 03:57:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001723488_441278464.pth [2023-12-27 03:57:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001727120_442204160.pth [2023-12-27 03:57:31,537][105692] Updated weights for policy 0, policy_version 1724649 (0.0008) [2023-12-27 03:57:31,603][105692] Updated weights for policy 0, policy_version 1724659 (0.0008) [2023-12-27 03:57:31,628][105620] Updated weights for policy 1, policy_version 1728242 (0.0010) [2023-12-27 03:57:31,666][105692] Updated weights for policy 0, policy_version 1724669 (0.0009) [2023-12-27 03:57:31,687][105620] Updated weights for policy 1, policy_version 1728252 (0.0007) [2023-12-27 03:57:31,747][105620] Updated weights for policy 1, policy_version 1728262 (0.0007) [2023-12-27 03:57:31,805][105620] Updated weights for policy 1, policy_version 1728272 (0.0008) [2023-12-27 03:57:32,414][105692] Updated weights for policy 0, policy_version 1724679 (0.0009) [2023-12-27 03:57:32,476][105692] Updated weights for policy 0, policy_version 1724689 (0.0009) [2023-12-27 03:57:32,529][105692] Updated weights for policy 0, policy_version 1724699 (0.0009) [2023-12-27 03:57:32,552][105620] Updated weights for policy 1, policy_version 1728282 (0.0006) [2023-12-27 03:57:32,608][105620] Updated weights for policy 1, policy_version 1728292 (0.0008) [2023-12-27 03:57:32,669][105620] Updated weights for policy 1, policy_version 1728302 (0.0009) [2023-12-27 03:57:33,292][105692] Updated weights for policy 0, policy_version 1724709 (0.0008) [2023-12-27 03:57:33,352][105692] Updated weights for policy 0, policy_version 1724719 (0.0008) [2023-12-27 03:57:33,383][105620] Updated weights for policy 1, policy_version 1728312 (0.0007) [2023-12-27 03:57:33,412][105692] Updated weights for policy 0, policy_version 1724729 (0.0007) [2023-12-27 03:57:33,434][105620] Updated weights for policy 1, policy_version 1728322 (0.0010) [2023-12-27 03:57:33,481][105620] Updated weights for policy 1, policy_version 1728332 (0.0010) [2023-12-27 03:57:34,148][105620] Updated weights for policy 1, policy_version 1728342 (0.0008) [2023-12-27 03:57:34,206][105692] Updated weights for policy 0, policy_version 1724739 (0.0006) [2023-12-27 03:57:34,207][105620] Updated weights for policy 1, policy_version 1728352 (0.0007) [2023-12-27 03:57:34,264][105692] Updated weights for policy 0, policy_version 1724749 (0.0010) [2023-12-27 03:57:34,269][105620] Updated weights for policy 1, policy_version 1728362 (0.0006) [2023-12-27 03:57:34,330][105692] Updated weights for policy 0, policy_version 1724759 (0.0009) [2023-12-27 03:57:34,827][105620] Updated weights for policy 1, policy_version 1728372 (0.0007) [2023-12-27 03:57:34,886][105620] Updated weights for policy 1, policy_version 1728382 (0.0010) [2023-12-27 03:57:34,935][105620] Updated weights for policy 1, policy_version 1728392 (0.0010) [2023-12-27 03:57:35,147][105692] Updated weights for policy 0, policy_version 1724769 (0.0010) [2023-12-27 03:57:35,195][105692] Updated weights for policy 0, policy_version 1724779 (0.0006) [2023-12-27 03:57:35,251][105692] Updated weights for policy 0, policy_version 1724789 (0.0006) [2023-12-27 03:57:35,312][105692] Updated weights for policy 0, policy_version 1724799 (0.0008) [2023-12-27 03:57:35,670][105620] Updated weights for policy 1, policy_version 1728402 (0.0009) [2023-12-27 03:57:35,736][105620] Updated weights for policy 1, policy_version 1728412 (0.0009) [2023-12-27 03:57:35,792][105620] Updated weights for policy 1, policy_version 1728422 (0.0010) [2023-12-27 03:57:35,845][105620] Updated weights for policy 1, policy_version 1728432 (0.0011) [2023-12-27 03:57:35,916][105692] Updated weights for policy 0, policy_version 1724809 (0.0006) [2023-12-27 03:57:35,998][105692] Updated weights for policy 0, policy_version 1724819 (0.0006) [2023-12-27 03:57:36,060][105692] Updated weights for policy 0, policy_version 1724829 (0.0008) [2023-12-27 03:57:36,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 884154368. Throughput: 0: 9582.2, 1: 9764.3. Samples: 884145424. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:57:36,063][104569] Avg episode reward: [(0, '8712.056'), (1, '8810.053')] [2023-12-27 03:57:36,547][105620] Updated weights for policy 1, policy_version 1728442 (0.0008) [2023-12-27 03:57:36,608][105620] Updated weights for policy 1, policy_version 1728452 (0.0005) [2023-12-27 03:57:36,671][105620] Updated weights for policy 1, policy_version 1728462 (0.0009) [2023-12-27 03:57:36,781][105692] Updated weights for policy 0, policy_version 1724839 (0.0008) [2023-12-27 03:57:36,836][105692] Updated weights for policy 0, policy_version 1724849 (0.0009) [2023-12-27 03:57:36,886][105692] Updated weights for policy 0, policy_version 1724859 (0.0009) [2023-12-27 03:57:37,310][105620] Updated weights for policy 1, policy_version 1728472 (0.0010) [2023-12-27 03:57:37,376][105620] Updated weights for policy 1, policy_version 1728482 (0.0010) [2023-12-27 03:57:37,441][105620] Updated weights for policy 1, policy_version 1728492 (0.0010) [2023-12-27 03:57:37,685][105692] Updated weights for policy 0, policy_version 1724869 (0.0008) [2023-12-27 03:57:37,736][105692] Updated weights for policy 0, policy_version 1724879 (0.0009) [2023-12-27 03:57:37,792][105692] Updated weights for policy 0, policy_version 1724889 (0.0009) [2023-12-27 03:57:38,057][105620] Updated weights for policy 1, policy_version 1728502 (0.0007) [2023-12-27 03:57:38,111][105620] Updated weights for policy 1, policy_version 1728512 (0.0007) [2023-12-27 03:57:38,160][105620] Updated weights for policy 1, policy_version 1728522 (0.0008) [2023-12-27 03:57:38,663][105692] Updated weights for policy 0, policy_version 1724899 (0.0010) [2023-12-27 03:57:38,716][105692] Updated weights for policy 0, policy_version 1724909 (0.0010) [2023-12-27 03:57:38,768][105692] Updated weights for policy 0, policy_version 1724919 (0.0007) [2023-12-27 03:57:38,800][105620] Updated weights for policy 1, policy_version 1728532 (0.0008) [2023-12-27 03:57:38,865][105620] Updated weights for policy 1, policy_version 1728542 (0.0005) [2023-12-27 03:57:38,928][105620] Updated weights for policy 1, policy_version 1728552 (0.0006) [2023-12-27 03:57:39,569][105692] Updated weights for policy 0, policy_version 1724929 (0.0007) [2023-12-27 03:57:39,596][105620] Updated weights for policy 1, policy_version 1728562 (0.0006) [2023-12-27 03:57:39,638][105692] Updated weights for policy 0, policy_version 1724939 (0.0008) [2023-12-27 03:57:39,666][105620] Updated weights for policy 1, policy_version 1728572 (0.0006) [2023-12-27 03:57:39,700][105692] Updated weights for policy 0, policy_version 1724949 (0.0009) [2023-12-27 03:57:39,724][105620] Updated weights for policy 1, policy_version 1728582 (0.0006) [2023-12-27 03:57:39,761][105692] Updated weights for policy 0, policy_version 1724959 (0.0007) [2023-12-27 03:57:39,775][105620] Updated weights for policy 1, policy_version 1728592 (0.0010) [2023-12-27 03:57:40,436][105692] Updated weights for policy 0, policy_version 1724969 (0.0008) [2023-12-27 03:57:40,486][105692] Updated weights for policy 0, policy_version 1724979 (0.0007) [2023-12-27 03:57:40,509][105620] Updated weights for policy 1, policy_version 1728602 (0.0011) [2023-12-27 03:57:40,543][105692] Updated weights for policy 0, policy_version 1724989 (0.0005) [2023-12-27 03:57:40,569][105620] Updated weights for policy 1, policy_version 1728612 (0.0011) [2023-12-27 03:57:40,626][105620] Updated weights for policy 1, policy_version 1728622 (0.0010) [2023-12-27 03:57:41,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 884252672. Throughput: 0: 9525.7, 1: 9815.8. Samples: 884262700. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:57:41,063][104569] Avg episode reward: [(0, '8713.819'), (1, '9085.376')] [2023-12-27 03:57:41,229][105620] Updated weights for policy 1, policy_version 1728632 (0.0010) [2023-12-27 03:57:41,296][105620] Updated weights for policy 1, policy_version 1728642 (0.0011) [2023-12-27 03:57:41,370][105620] Updated weights for policy 1, policy_version 1728652 (0.0010) [2023-12-27 03:57:41,450][105692] Updated weights for policy 0, policy_version 1724999 (0.0008) [2023-12-27 03:57:41,512][105692] Updated weights for policy 0, policy_version 1725009 (0.0008) [2023-12-27 03:57:41,568][105692] Updated weights for policy 0, policy_version 1725019 (0.0008) [2023-12-27 03:57:42,054][105620] Updated weights for policy 1, policy_version 1728662 (0.0008) [2023-12-27 03:57:42,103][105620] Updated weights for policy 1, policy_version 1728672 (0.0011) [2023-12-27 03:57:42,166][105620] Updated weights for policy 1, policy_version 1728682 (0.0007) [2023-12-27 03:57:42,432][105692] Updated weights for policy 0, policy_version 1725029 (0.0008) [2023-12-27 03:57:42,484][105692] Updated weights for policy 0, policy_version 1725039 (0.0008) [2023-12-27 03:57:42,540][105692] Updated weights for policy 0, policy_version 1725049 (0.0008) [2023-12-27 03:57:42,847][105620] Updated weights for policy 1, policy_version 1728692 (0.0009) [2023-12-27 03:57:42,902][105620] Updated weights for policy 1, policy_version 1728702 (0.0011) [2023-12-27 03:57:42,961][105620] Updated weights for policy 1, policy_version 1728712 (0.0011) [2023-12-27 03:57:43,231][105692] Updated weights for policy 0, policy_version 1725059 (0.0006) [2023-12-27 03:57:43,285][105692] Updated weights for policy 0, policy_version 1725069 (0.0005) [2023-12-27 03:57:43,340][105692] Updated weights for policy 0, policy_version 1725079 (0.0005) [2023-12-27 03:57:43,712][105620] Updated weights for policy 1, policy_version 1728722 (0.0010) [2023-12-27 03:57:43,766][105620] Updated weights for policy 1, policy_version 1728732 (0.0005) [2023-12-27 03:57:43,822][105620] Updated weights for policy 1, policy_version 1728742 (0.0005) [2023-12-27 03:57:43,870][105692] Updated weights for policy 0, policy_version 1725089 (0.0006) [2023-12-27 03:57:43,878][105620] Updated weights for policy 1, policy_version 1728752 (0.0008) [2023-12-27 03:57:43,934][105692] Updated weights for policy 0, policy_version 1725099 (0.0005) [2023-12-27 03:57:43,986][105692] Updated weights for policy 0, policy_version 1725109 (0.0007) [2023-12-27 03:57:44,039][105692] Updated weights for policy 0, policy_version 1725119 (0.0008) [2023-12-27 03:57:44,606][105620] Updated weights for policy 1, policy_version 1728762 (0.0008) [2023-12-27 03:57:44,636][105692] Updated weights for policy 0, policy_version 1725129 (0.0006) [2023-12-27 03:57:44,673][105620] Updated weights for policy 1, policy_version 1728772 (0.0008) [2023-12-27 03:57:44,691][105692] Updated weights for policy 0, policy_version 1725139 (0.0005) [2023-12-27 03:57:44,726][105620] Updated weights for policy 1, policy_version 1728782 (0.0008) [2023-12-27 03:57:44,737][105692] Updated weights for policy 0, policy_version 1725149 (0.0005) [2023-12-27 03:57:45,458][105692] Updated weights for policy 0, policy_version 1725159 (0.0008) [2023-12-27 03:57:45,514][105692] Updated weights for policy 0, policy_version 1725169 (0.0009) [2023-12-27 03:57:45,527][105620] Updated weights for policy 1, policy_version 1728792 (0.0008) [2023-12-27 03:57:45,574][105692] Updated weights for policy 0, policy_version 1725179 (0.0006) [2023-12-27 03:57:45,583][105620] Updated weights for policy 1, policy_version 1728802 (0.0007) [2023-12-27 03:57:45,646][105620] Updated weights for policy 1, policy_version 1728812 (0.0008) [2023-12-27 03:57:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 884350976. Throughput: 0: 9506.5, 1: 9791.8. Samples: 884320068. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:57:46,063][104569] Avg episode reward: [(0, '8625.034'), (1, '9172.810')] [2023-12-27 03:57:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001725184_441712640.pth... [2023-12-27 03:57:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001728816_442638336.pth... [2023-12-27 03:57:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001727664_442343424.pth [2023-12-27 03:57:46,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001724096_441434112.pth [2023-12-27 03:57:46,255][105692] Updated weights for policy 0, policy_version 1725189 (0.0007) [2023-12-27 03:57:46,289][105620] Updated weights for policy 1, policy_version 1728822 (0.0006) [2023-12-27 03:57:46,312][105692] Updated weights for policy 0, policy_version 1725199 (0.0005) [2023-12-27 03:57:46,346][105620] Updated weights for policy 1, policy_version 1728832 (0.0008) [2023-12-27 03:57:46,364][105692] Updated weights for policy 0, policy_version 1725209 (0.0006) [2023-12-27 03:57:46,400][105620] Updated weights for policy 1, policy_version 1728842 (0.0007) [2023-12-27 03:57:47,050][105692] Updated weights for policy 0, policy_version 1725219 (0.0007) [2023-12-27 03:57:47,057][105620] Updated weights for policy 1, policy_version 1728852 (0.0006) [2023-12-27 03:57:47,114][105692] Updated weights for policy 0, policy_version 1725229 (0.0005) [2023-12-27 03:57:47,122][105620] Updated weights for policy 1, policy_version 1728862 (0.0008) [2023-12-27 03:57:47,172][105692] Updated weights for policy 0, policy_version 1725239 (0.0006) [2023-12-27 03:57:47,177][105620] Updated weights for policy 1, policy_version 1728872 (0.0008) [2023-12-27 03:57:47,804][105692] Updated weights for policy 0, policy_version 1725249 (0.0008) [2023-12-27 03:57:47,862][105692] Updated weights for policy 0, policy_version 1725259 (0.0010) [2023-12-27 03:57:47,905][105620] Updated weights for policy 1, policy_version 1728882 (0.0008) [2023-12-27 03:57:47,914][105692] Updated weights for policy 0, policy_version 1725269 (0.0007) [2023-12-27 03:57:47,953][105620] Updated weights for policy 1, policy_version 1728892 (0.0009) [2023-12-27 03:57:47,967][105692] Updated weights for policy 0, policy_version 1725279 (0.0006) [2023-12-27 03:57:48,009][105620] Updated weights for policy 1, policy_version 1728902 (0.0009) [2023-12-27 03:57:48,063][105620] Updated weights for policy 1, policy_version 1728912 (0.0010) [2023-12-27 03:57:48,591][105692] Updated weights for policy 0, policy_version 1725289 (0.0008) [2023-12-27 03:57:48,646][105692] Updated weights for policy 0, policy_version 1725299 (0.0009) [2023-12-27 03:57:48,702][105692] Updated weights for policy 0, policy_version 1725309 (0.0008) [2023-12-27 03:57:48,841][105620] Updated weights for policy 1, policy_version 1728922 (0.0007) [2023-12-27 03:57:48,905][105620] Updated weights for policy 1, policy_version 1728932 (0.0007) [2023-12-27 03:57:48,967][105620] Updated weights for policy 1, policy_version 1728942 (0.0011) [2023-12-27 03:57:49,466][105692] Updated weights for policy 0, policy_version 1725319 (0.0007) [2023-12-27 03:57:49,520][105692] Updated weights for policy 0, policy_version 1725329 (0.0008) [2023-12-27 03:57:49,575][105692] Updated weights for policy 0, policy_version 1725339 (0.0008) [2023-12-27 03:57:49,661][105620] Updated weights for policy 1, policy_version 1728952 (0.0011) [2023-12-27 03:57:49,730][105620] Updated weights for policy 1, policy_version 1728962 (0.0010) [2023-12-27 03:57:49,788][105620] Updated weights for policy 1, policy_version 1728972 (0.0010) [2023-12-27 03:57:50,271][105692] Updated weights for policy 0, policy_version 1725349 (0.0009) [2023-12-27 03:57:50,331][105692] Updated weights for policy 0, policy_version 1725359 (0.0008) [2023-12-27 03:57:50,393][105692] Updated weights for policy 0, policy_version 1725369 (0.0007) [2023-12-27 03:57:50,516][105620] Updated weights for policy 1, policy_version 1728982 (0.0007) [2023-12-27 03:57:50,566][105620] Updated weights for policy 1, policy_version 1728992 (0.0007) [2023-12-27 03:57:50,627][105620] Updated weights for policy 1, policy_version 1729002 (0.0007) [2023-12-27 03:57:51,049][105692] Updated weights for policy 0, policy_version 1725379 (0.0006) [2023-12-27 03:57:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 884449280. Throughput: 0: 9604.8, 1: 9775.7. Samples: 884440408. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:57:51,062][104569] Avg episode reward: [(0, '8713.079'), (1, '8989.093')] [2023-12-27 03:57:51,118][105692] Updated weights for policy 0, policy_version 1725389 (0.0006) [2023-12-27 03:57:51,180][105692] Updated weights for policy 0, policy_version 1725399 (0.0008) [2023-12-27 03:57:51,331][105620] Updated weights for policy 1, policy_version 1729012 (0.0007) [2023-12-27 03:57:51,398][105620] Updated weights for policy 1, policy_version 1729022 (0.0008) [2023-12-27 03:57:51,468][105620] Updated weights for policy 1, policy_version 1729032 (0.0008) [2023-12-27 03:57:51,870][105692] Updated weights for policy 0, policy_version 1725409 (0.0008) [2023-12-27 03:57:51,939][105692] Updated weights for policy 0, policy_version 1725419 (0.0006) [2023-12-27 03:57:52,002][105692] Updated weights for policy 0, policy_version 1725429 (0.0011) [2023-12-27 03:57:52,054][105692] Updated weights for policy 0, policy_version 1725439 (0.0011) [2023-12-27 03:57:52,122][105620] Updated weights for policy 1, policy_version 1729042 (0.0008) [2023-12-27 03:57:52,187][105620] Updated weights for policy 1, policy_version 1729052 (0.0008) [2023-12-27 03:57:52,256][105620] Updated weights for policy 1, policy_version 1729062 (0.0006) [2023-12-27 03:57:52,310][105620] Updated weights for policy 1, policy_version 1729072 (0.0007) [2023-12-27 03:57:52,752][105692] Updated weights for policy 0, policy_version 1725449 (0.0010) [2023-12-27 03:57:52,809][105692] Updated weights for policy 0, policy_version 1725459 (0.0008) [2023-12-27 03:57:52,857][105692] Updated weights for policy 0, policy_version 1725469 (0.0009) [2023-12-27 03:57:53,024][105620] Updated weights for policy 1, policy_version 1729082 (0.0008) [2023-12-27 03:57:53,080][105620] Updated weights for policy 1, policy_version 1729092 (0.0006) [2023-12-27 03:57:53,133][105620] Updated weights for policy 1, policy_version 1729102 (0.0005) [2023-12-27 03:57:53,616][105692] Updated weights for policy 0, policy_version 1725479 (0.0009) [2023-12-27 03:57:53,676][105692] Updated weights for policy 0, policy_version 1725489 (0.0008) [2023-12-27 03:57:53,728][105692] Updated weights for policy 0, policy_version 1725499 (0.0005) [2023-12-27 03:57:53,872][105620] Updated weights for policy 1, policy_version 1729112 (0.0008) [2023-12-27 03:57:53,938][105620] Updated weights for policy 1, policy_version 1729122 (0.0009) [2023-12-27 03:57:53,995][105620] Updated weights for policy 1, policy_version 1729132 (0.0009) [2023-12-27 03:57:54,434][105692] Updated weights for policy 0, policy_version 1725509 (0.0006) [2023-12-27 03:57:54,482][105692] Updated weights for policy 0, policy_version 1725519 (0.0005) [2023-12-27 03:57:54,536][105692] Updated weights for policy 0, policy_version 1725529 (0.0005) [2023-12-27 03:57:54,647][105620] Updated weights for policy 1, policy_version 1729142 (0.0008) [2023-12-27 03:57:54,699][105620] Updated weights for policy 1, policy_version 1729152 (0.0009) [2023-12-27 03:57:54,761][105620] Updated weights for policy 1, policy_version 1729162 (0.0009) [2023-12-27 03:57:55,110][105692] Updated weights for policy 0, policy_version 1725539 (0.0006) [2023-12-27 03:57:55,163][105692] Updated weights for policy 0, policy_version 1725549 (0.0008) [2023-12-27 03:57:55,223][105692] Updated weights for policy 0, policy_version 1725559 (0.0005) [2023-12-27 03:57:55,566][105620] Updated weights for policy 1, policy_version 1729172 (0.0009) [2023-12-27 03:57:55,617][105620] Updated weights for policy 1, policy_version 1729182 (0.0009) [2023-12-27 03:57:55,672][105620] Updated weights for policy 1, policy_version 1729192 (0.0009) [2023-12-27 03:57:55,933][105692] Updated weights for policy 0, policy_version 1725569 (0.0008) [2023-12-27 03:57:55,984][105692] Updated weights for policy 0, policy_version 1725579 (0.0008) [2023-12-27 03:57:56,044][105692] Updated weights for policy 0, policy_version 1725589 (0.0005) [2023-12-27 03:57:56,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 884547584. Throughput: 0: 9607.8, 1: 9861.4. Samples: 884558940. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:57:56,062][104569] Avg episode reward: [(0, '8712.626'), (1, '8991.728')] [2023-12-27 03:57:56,093][105692] Updated weights for policy 0, policy_version 1725599 (0.0005) [2023-12-27 03:57:56,532][105620] Updated weights for policy 1, policy_version 1729202 (0.0009) [2023-12-27 03:57:56,584][105620] Updated weights for policy 1, policy_version 1729212 (0.0007) [2023-12-27 03:57:56,633][105620] Updated weights for policy 1, policy_version 1729222 (0.0005) [2023-12-27 03:57:56,653][105692] Updated weights for policy 0, policy_version 1725609 (0.0008) [2023-12-27 03:57:56,680][105620] Updated weights for policy 1, policy_version 1729232 (0.0006) [2023-12-27 03:57:56,709][105692] Updated weights for policy 0, policy_version 1725619 (0.0006) [2023-12-27 03:57:56,774][105692] Updated weights for policy 0, policy_version 1725629 (0.0006) [2023-12-27 03:57:57,253][105620] Updated weights for policy 1, policy_version 1729242 (0.0005) [2023-12-27 03:57:57,297][105692] Updated weights for policy 0, policy_version 1725639 (0.0006) [2023-12-27 03:57:57,300][105620] Updated weights for policy 1, policy_version 1729252 (0.0008) [2023-12-27 03:57:57,353][105692] Updated weights for policy 0, policy_version 1725649 (0.0006) [2023-12-27 03:57:57,357][105620] Updated weights for policy 1, policy_version 1729262 (0.0008) [2023-12-27 03:57:57,410][105692] Updated weights for policy 0, policy_version 1725659 (0.0008) [2023-12-27 03:57:58,055][105620] Updated weights for policy 1, policy_version 1729272 (0.0008) [2023-12-27 03:57:58,110][105620] Updated weights for policy 1, policy_version 1729282 (0.0005) [2023-12-27 03:57:58,161][105620] Updated weights for policy 1, policy_version 1729292 (0.0007) [2023-12-27 03:57:58,187][105692] Updated weights for policy 0, policy_version 1725669 (0.0009) [2023-12-27 03:57:58,247][105692] Updated weights for policy 0, policy_version 1725679 (0.0007) [2023-12-27 03:57:58,308][105692] Updated weights for policy 0, policy_version 1725689 (0.0007) [2023-12-27 03:57:58,953][105620] Updated weights for policy 1, policy_version 1729302 (0.0008) [2023-12-27 03:57:59,011][105620] Updated weights for policy 1, policy_version 1729312 (0.0011) [2023-12-27 03:57:59,060][105620] Updated weights for policy 1, policy_version 1729322 (0.0010) [2023-12-27 03:57:59,121][105692] Updated weights for policy 0, policy_version 1725699 (0.0009) [2023-12-27 03:57:59,181][105692] Updated weights for policy 0, policy_version 1725709 (0.0011) [2023-12-27 03:57:59,241][105692] Updated weights for policy 0, policy_version 1725719 (0.0010) [2023-12-27 03:57:59,809][105620] Updated weights for policy 1, policy_version 1729332 (0.0010) [2023-12-27 03:57:59,872][105620] Updated weights for policy 1, policy_version 1729342 (0.0011) [2023-12-27 03:57:59,921][105620] Updated weights for policy 1, policy_version 1729352 (0.0010) [2023-12-27 03:58:00,042][105692] Updated weights for policy 0, policy_version 1725729 (0.0009) [2023-12-27 03:58:00,103][105692] Updated weights for policy 0, policy_version 1725739 (0.0008) [2023-12-27 03:58:00,156][105692] Updated weights for policy 0, policy_version 1725749 (0.0008) [2023-12-27 03:58:00,209][105692] Updated weights for policy 0, policy_version 1725759 (0.0008) [2023-12-27 03:58:00,684][105620] Updated weights for policy 1, policy_version 1729362 (0.0011) [2023-12-27 03:58:00,744][105620] Updated weights for policy 1, policy_version 1729372 (0.0010) [2023-12-27 03:58:00,793][105620] Updated weights for policy 1, policy_version 1729382 (0.0009) [2023-12-27 03:58:00,849][105620] Updated weights for policy 1, policy_version 1729392 (0.0006) [2023-12-27 03:58:00,932][105692] Updated weights for policy 0, policy_version 1725769 (0.0006) [2023-12-27 03:58:00,981][105692] Updated weights for policy 0, policy_version 1725779 (0.0005) [2023-12-27 03:58:01,046][105692] Updated weights for policy 0, policy_version 1725789 (0.0007) [2023-12-27 03:58:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 884645888. Throughput: 0: 9669.5, 1: 9915.0. Samples: 884620228. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:58:01,062][104569] Avg episode reward: [(0, '8803.390'), (1, '9083.796')] [2023-12-27 03:58:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001725792_441868288.pth... [2023-12-27 03:58:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001729392_442785792.pth... [2023-12-27 03:58:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001724640_441573376.pth [2023-12-27 03:58:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001728240_442490880.pth [2023-12-27 03:58:01,602][105620] Updated weights for policy 1, policy_version 1729402 (0.0009) [2023-12-27 03:58:01,651][105692] Updated weights for policy 0, policy_version 1725799 (0.0007) [2023-12-27 03:58:01,658][105620] Updated weights for policy 1, policy_version 1729412 (0.0008) [2023-12-27 03:58:01,713][105692] Updated weights for policy 0, policy_version 1725809 (0.0006) [2023-12-27 03:58:01,725][105620] Updated weights for policy 1, policy_version 1729422 (0.0009) [2023-12-27 03:58:01,773][105692] Updated weights for policy 0, policy_version 1725819 (0.0009) [2023-12-27 03:58:02,363][105692] Updated weights for policy 0, policy_version 1725829 (0.0008) [2023-12-27 03:58:02,420][105692] Updated weights for policy 0, policy_version 1725839 (0.0009) [2023-12-27 03:58:02,467][105692] Updated weights for policy 0, policy_version 1725849 (0.0005) [2023-12-27 03:58:02,540][105620] Updated weights for policy 1, policy_version 1729432 (0.0006) [2023-12-27 03:58:02,602][105620] Updated weights for policy 1, policy_version 1729442 (0.0005) [2023-12-27 03:58:02,659][105620] Updated weights for policy 1, policy_version 1729452 (0.0005) [2023-12-27 03:58:03,118][105692] Updated weights for policy 0, policy_version 1725859 (0.0008) [2023-12-27 03:58:03,176][105692] Updated weights for policy 0, policy_version 1725869 (0.0008) [2023-12-27 03:58:03,231][105692] Updated weights for policy 0, policy_version 1725879 (0.0005) [2023-12-27 03:58:03,254][105620] Updated weights for policy 1, policy_version 1729462 (0.0005) [2023-12-27 03:58:03,273][105586] KL-divergence is very high: 153.0991 [2023-12-27 03:58:03,299][105620] Updated weights for policy 1, policy_version 1729472 (0.0006) [2023-12-27 03:58:03,309][105586] KL-divergence is very high: 286.6988 [2023-12-27 03:58:03,346][105620] Updated weights for policy 1, policy_version 1729482 (0.0009) [2023-12-27 03:58:03,348][105586] KL-divergence is very high: 331.4643 [2023-12-27 03:58:03,864][105692] Updated weights for policy 0, policy_version 1725889 (0.0007) [2023-12-27 03:58:03,919][105692] Updated weights for policy 0, policy_version 1725899 (0.0009) [2023-12-27 03:58:03,969][105692] Updated weights for policy 0, policy_version 1725909 (0.0005) [2023-12-27 03:58:03,996][105620] Updated weights for policy 1, policy_version 1729492 (0.0006) [2023-12-27 03:58:04,029][105692] Updated weights for policy 0, policy_version 1725919 (0.0006) [2023-12-27 03:58:04,057][105620] Updated weights for policy 1, policy_version 1729502 (0.0006) [2023-12-27 03:58:04,122][105620] Updated weights for policy 1, policy_version 1729512 (0.0010) [2023-12-27 03:58:04,650][105692] Updated weights for policy 0, policy_version 1725929 (0.0005) [2023-12-27 03:58:04,702][105692] Updated weights for policy 0, policy_version 1725939 (0.0006) [2023-12-27 03:58:04,758][105692] Updated weights for policy 0, policy_version 1725949 (0.0008) [2023-12-27 03:58:04,817][105620] Updated weights for policy 1, policy_version 1729522 (0.0011) [2023-12-27 03:58:04,875][105620] Updated weights for policy 1, policy_version 1729532 (0.0010) [2023-12-27 03:58:04,932][105620] Updated weights for policy 1, policy_version 1729542 (0.0009) [2023-12-27 03:58:04,990][105620] Updated weights for policy 1, policy_version 1729552 (0.0010) [2023-12-27 03:58:05,441][105692] Updated weights for policy 0, policy_version 1725959 (0.0009) [2023-12-27 03:58:05,494][105692] Updated weights for policy 0, policy_version 1725971 (0.0010) [2023-12-27 03:58:05,560][105692] Updated weights for policy 0, policy_version 1725981 (0.0008) [2023-12-27 03:58:05,608][105620] Updated weights for policy 1, policy_version 1729562 (0.0006) [2023-12-27 03:58:05,666][105620] Updated weights for policy 1, policy_version 1729572 (0.0007) [2023-12-27 03:58:05,714][105620] Updated weights for policy 1, policy_version 1729582 (0.0009) [2023-12-27 03:58:06,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 884752384. Throughput: 0: 9726.4, 1: 9864.2. Samples: 884740476. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:58:06,063][104569] Avg episode reward: [(0, '8800.799'), (1, '8807.648')] [2023-12-27 03:58:06,307][105692] Updated weights for policy 0, policy_version 1725991 (0.0009) [2023-12-27 03:58:06,372][105692] Updated weights for policy 0, policy_version 1726001 (0.0009) [2023-12-27 03:58:06,421][105620] Updated weights for policy 1, policy_version 1729592 (0.0007) [2023-12-27 03:58:06,427][105692] Updated weights for policy 0, policy_version 1726011 (0.0007) [2023-12-27 03:58:06,484][105620] Updated weights for policy 1, policy_version 1729602 (0.0008) [2023-12-27 03:58:06,547][105620] Updated weights for policy 1, policy_version 1729612 (0.0009) [2023-12-27 03:58:07,206][105692] Updated weights for policy 0, policy_version 1726021 (0.0007) [2023-12-27 03:58:07,265][105692] Updated weights for policy 0, policy_version 1726031 (0.0007) [2023-12-27 03:58:07,271][105620] Updated weights for policy 1, policy_version 1729622 (0.0007) [2023-12-27 03:58:07,329][105692] Updated weights for policy 0, policy_version 1726041 (0.0007) [2023-12-27 03:58:07,331][105620] Updated weights for policy 1, policy_version 1729632 (0.0006) [2023-12-27 03:58:07,389][105620] Updated weights for policy 1, policy_version 1729642 (0.0007) [2023-12-27 03:58:08,000][105692] Updated weights for policy 0, policy_version 1726051 (0.0006) [2023-12-27 03:58:08,057][105692] Updated weights for policy 0, policy_version 1726061 (0.0009) [2023-12-27 03:58:08,110][105692] Updated weights for policy 0, policy_version 1726071 (0.0008) [2023-12-27 03:58:08,175][105620] Updated weights for policy 1, policy_version 1729652 (0.0008) [2023-12-27 03:58:08,221][105620] Updated weights for policy 1, policy_version 1729662 (0.0009) [2023-12-27 03:58:08,275][105620] Updated weights for policy 1, policy_version 1729672 (0.0009) [2023-12-27 03:58:08,873][105692] Updated weights for policy 0, policy_version 1726081 (0.0009) [2023-12-27 03:58:08,930][105692] Updated weights for policy 0, policy_version 1726091 (0.0009) [2023-12-27 03:58:08,995][105692] Updated weights for policy 0, policy_version 1726101 (0.0009) [2023-12-27 03:58:09,049][105620] Updated weights for policy 1, policy_version 1729682 (0.0008) [2023-12-27 03:58:09,055][105692] Updated weights for policy 0, policy_version 1726111 (0.0009) [2023-12-27 03:58:09,108][105620] Updated weights for policy 1, policy_version 1729692 (0.0005) [2023-12-27 03:58:09,176][105620] Updated weights for policy 1, policy_version 1729702 (0.0006) [2023-12-27 03:58:09,238][105620] Updated weights for policy 1, policy_version 1729712 (0.0008) [2023-12-27 03:58:09,859][105692] Updated weights for policy 0, policy_version 1726121 (0.0009) [2023-12-27 03:58:09,929][105692] Updated weights for policy 0, policy_version 1726131 (0.0009) [2023-12-27 03:58:09,952][105620] Updated weights for policy 1, policy_version 1729722 (0.0007) [2023-12-27 03:58:09,984][105692] Updated weights for policy 0, policy_version 1726141 (0.0008) [2023-12-27 03:58:10,000][105620] Updated weights for policy 1, policy_version 1729732 (0.0007) [2023-12-27 03:58:10,056][105620] Updated weights for policy 1, policy_version 1729742 (0.0005) [2023-12-27 03:58:10,749][105692] Updated weights for policy 0, policy_version 1726151 (0.0007) [2023-12-27 03:58:10,792][105620] Updated weights for policy 1, policy_version 1729752 (0.0008) [2023-12-27 03:58:10,807][105692] Updated weights for policy 0, policy_version 1726161 (0.0006) [2023-12-27 03:58:10,846][105620] Updated weights for policy 1, policy_version 1729762 (0.0007) [2023-12-27 03:58:10,854][105692] Updated weights for policy 0, policy_version 1726171 (0.0005) [2023-12-27 03:58:10,905][105620] Updated weights for policy 1, policy_version 1729772 (0.0009) [2023-12-27 03:58:11,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 884850688. Throughput: 0: 9758.5, 1: 9819.9. Samples: 884853908. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:58:11,062][104569] Avg episode reward: [(0, '8892.018'), (1, '8624.376')] [2023-12-27 03:58:11,574][105692] Updated weights for policy 0, policy_version 1726181 (0.0007) [2023-12-27 03:58:11,634][105692] Updated weights for policy 0, policy_version 1726191 (0.0010) [2023-12-27 03:58:11,686][105692] Updated weights for policy 0, policy_version 1726201 (0.0009) [2023-12-27 03:58:11,785][105620] Updated weights for policy 1, policy_version 1729782 (0.0009) [2023-12-27 03:58:11,844][105620] Updated weights for policy 1, policy_version 1729792 (0.0009) [2023-12-27 03:58:11,892][105620] Updated weights for policy 1, policy_version 1729802 (0.0009) [2023-12-27 03:58:12,495][105692] Updated weights for policy 0, policy_version 1726211 (0.0009) [2023-12-27 03:58:12,557][105692] Updated weights for policy 0, policy_version 1726221 (0.0008) [2023-12-27 03:58:12,618][105692] Updated weights for policy 0, policy_version 1726231 (0.0008) [2023-12-27 03:58:12,680][105620] Updated weights for policy 1, policy_version 1729812 (0.0010) [2023-12-27 03:58:12,741][105620] Updated weights for policy 1, policy_version 1729822 (0.0008) [2023-12-27 03:58:12,810][105620] Updated weights for policy 1, policy_version 1729832 (0.0008) [2023-12-27 03:58:13,293][105692] Updated weights for policy 0, policy_version 1726241 (0.0008) [2023-12-27 03:58:13,345][105692] Updated weights for policy 0, policy_version 1726251 (0.0008) [2023-12-27 03:58:13,397][105692] Updated weights for policy 0, policy_version 1726261 (0.0008) [2023-12-27 03:58:13,456][105692] Updated weights for policy 0, policy_version 1726271 (0.0008) [2023-12-27 03:58:13,529][105620] Updated weights for policy 1, policy_version 1729842 (0.0009) [2023-12-27 03:58:13,587][105620] Updated weights for policy 1, policy_version 1729852 (0.0010) [2023-12-27 03:58:13,640][105620] Updated weights for policy 1, policy_version 1729862 (0.0008) [2023-12-27 03:58:13,703][105620] Updated weights for policy 1, policy_version 1729872 (0.0006) [2023-12-27 03:58:14,267][105692] Updated weights for policy 0, policy_version 1726281 (0.0009) [2023-12-27 03:58:14,306][105620] Updated weights for policy 1, policy_version 1729882 (0.0005) [2023-12-27 03:58:14,328][105692] Updated weights for policy 0, policy_version 1726291 (0.0007) [2023-12-27 03:58:14,362][105620] Updated weights for policy 1, policy_version 1729892 (0.0007) [2023-12-27 03:58:14,385][105692] Updated weights for policy 0, policy_version 1726301 (0.0006) [2023-12-27 03:58:14,412][105620] Updated weights for policy 1, policy_version 1729902 (0.0006) [2023-12-27 03:58:15,174][105692] Updated weights for policy 0, policy_version 1726311 (0.0009) [2023-12-27 03:58:15,184][105620] Updated weights for policy 1, policy_version 1729912 (0.0007) [2023-12-27 03:58:15,223][105692] Updated weights for policy 0, policy_version 1726321 (0.0009) [2023-12-27 03:58:15,245][105620] Updated weights for policy 1, policy_version 1729922 (0.0008) [2023-12-27 03:58:15,285][105692] Updated weights for policy 0, policy_version 1726331 (0.0007) [2023-12-27 03:58:15,300][105620] Updated weights for policy 1, policy_version 1729932 (0.0006) [2023-12-27 03:58:15,959][105620] Updated weights for policy 1, policy_version 1729942 (0.0009) [2023-12-27 03:58:16,010][105620] Updated weights for policy 1, policy_version 1729952 (0.0009) [2023-12-27 03:58:16,062][104569] Fps is (10 sec: 18022.7, 60 sec: 19251.3, 300 sec: 19438.6). Total num frames: 884932608. Throughput: 0: 9754.8, 1: 9786.8. Samples: 884910744. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:58:16,062][104569] Avg episode reward: [(0, '8892.950'), (1, '9080.223')] [2023-12-27 03:58:16,066][105620] Updated weights for policy 1, policy_version 1729962 (0.0008) [2023-12-27 03:58:16,075][105692] Updated weights for policy 0, policy_version 1726341 (0.0007) [2023-12-27 03:58:16,097][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001729968_442933248.pth... [2023-12-27 03:58:16,100][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001728816_442638336.pth [2023-12-27 03:58:16,137][105692] Updated weights for policy 0, policy_version 1726351 (0.0007) [2023-12-27 03:58:16,209][105692] Updated weights for policy 0, policy_version 1726361 (0.0007) [2023-12-27 03:58:16,251][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001726368_442015744.pth... [2023-12-27 03:58:16,256][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001725184_441712640.pth [2023-12-27 03:58:16,732][105692] Updated weights for policy 0, policy_version 1726371 (0.0007) [2023-12-27 03:58:16,786][105692] Updated weights for policy 0, policy_version 1726381 (0.0009) [2023-12-27 03:58:16,841][105692] Updated weights for policy 0, policy_version 1726391 (0.0009) [2023-12-27 03:58:16,914][105620] Updated weights for policy 1, policy_version 1729972 (0.0008) [2023-12-27 03:58:16,974][105620] Updated weights for policy 1, policy_version 1729982 (0.0009) [2023-12-27 03:58:17,024][105620] Updated weights for policy 1, policy_version 1729992 (0.0008) [2023-12-27 03:58:17,592][105692] Updated weights for policy 0, policy_version 1726401 (0.0009) [2023-12-27 03:58:17,624][105620] Updated weights for policy 1, policy_version 1730002 (0.0008) [2023-12-27 03:58:17,650][105692] Updated weights for policy 0, policy_version 1726411 (0.0010) [2023-12-27 03:58:17,681][105620] Updated weights for policy 1, policy_version 1730012 (0.0005) [2023-12-27 03:58:17,713][105692] Updated weights for policy 0, policy_version 1726421 (0.0010) [2023-12-27 03:58:17,733][105620] Updated weights for policy 1, policy_version 1730022 (0.0005) [2023-12-27 03:58:17,758][105692] Updated weights for policy 0, policy_version 1726431 (0.0009) [2023-12-27 03:58:17,793][105620] Updated weights for policy 1, policy_version 1730032 (0.0008) [2023-12-27 03:58:18,355][105620] Updated weights for policy 1, policy_version 1730042 (0.0006) [2023-12-27 03:58:18,415][105620] Updated weights for policy 1, policy_version 1730052 (0.0008) [2023-12-27 03:58:18,480][105620] Updated weights for policy 1, policy_version 1730062 (0.0007) [2023-12-27 03:58:18,552][105692] Updated weights for policy 0, policy_version 1726441 (0.0010) [2023-12-27 03:58:18,611][105692] Updated weights for policy 0, policy_version 1726451 (0.0009) [2023-12-27 03:58:18,666][105692] Updated weights for policy 0, policy_version 1726461 (0.0009) [2023-12-27 03:58:19,152][105620] Updated weights for policy 1, policy_version 1730072 (0.0005) [2023-12-27 03:58:19,210][105620] Updated weights for policy 1, policy_version 1730082 (0.0006) [2023-12-27 03:58:19,271][105620] Updated weights for policy 1, policy_version 1730092 (0.0009) [2023-12-27 03:58:19,463][105692] Updated weights for policy 0, policy_version 1726471 (0.0009) [2023-12-27 03:58:19,529][105692] Updated weights for policy 0, policy_version 1726481 (0.0007) [2023-12-27 03:58:19,595][105692] Updated weights for policy 0, policy_version 1726491 (0.0007) [2023-12-27 03:58:20,026][105620] Updated weights for policy 1, policy_version 1730102 (0.0008) [2023-12-27 03:58:20,081][105620] Updated weights for policy 1, policy_version 1730112 (0.0009) [2023-12-27 03:58:20,134][105620] Updated weights for policy 1, policy_version 1730122 (0.0006) [2023-12-27 03:58:20,351][105692] Updated weights for policy 0, policy_version 1726501 (0.0009) [2023-12-27 03:58:20,410][105692] Updated weights for policy 0, policy_version 1726511 (0.0009) [2023-12-27 03:58:20,468][105692] Updated weights for policy 0, policy_version 1726521 (0.0009) [2023-12-27 03:58:20,892][105620] Updated weights for policy 1, policy_version 1730132 (0.0008) [2023-12-27 03:58:20,952][105620] Updated weights for policy 1, policy_version 1730142 (0.0009) [2023-12-27 03:58:21,004][105620] Updated weights for policy 1, policy_version 1730152 (0.0009) [2023-12-27 03:58:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 885039104. Throughput: 0: 9803.7, 1: 9799.8. Samples: 885027580. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:58:21,062][104569] Avg episode reward: [(0, '8618.178'), (1, '9171.173')] [2023-12-27 03:58:21,265][105692] Updated weights for policy 0, policy_version 1726531 (0.0009) [2023-12-27 03:58:21,332][105692] Updated weights for policy 0, policy_version 1726541 (0.0009) [2023-12-27 03:58:21,402][105692] Updated weights for policy 0, policy_version 1726551 (0.0009) [2023-12-27 03:58:21,793][105620] Updated weights for policy 1, policy_version 1730162 (0.0009) [2023-12-27 03:58:21,854][105620] Updated weights for policy 1, policy_version 1730172 (0.0008) [2023-12-27 03:58:21,912][105620] Updated weights for policy 1, policy_version 1730182 (0.0009) [2023-12-27 03:58:21,971][105620] Updated weights for policy 1, policy_version 1730192 (0.0009) [2023-12-27 03:58:22,144][105692] Updated weights for policy 0, policy_version 1726561 (0.0009) [2023-12-27 03:58:22,207][105692] Updated weights for policy 0, policy_version 1726571 (0.0009) [2023-12-27 03:58:22,276][105692] Updated weights for policy 0, policy_version 1726581 (0.0010) [2023-12-27 03:58:22,330][105692] Updated weights for policy 0, policy_version 1726591 (0.0009) [2023-12-27 03:58:22,718][105620] Updated weights for policy 1, policy_version 1730202 (0.0006) [2023-12-27 03:58:22,784][105620] Updated weights for policy 1, policy_version 1730212 (0.0008) [2023-12-27 03:58:22,849][105620] Updated weights for policy 1, policy_version 1730222 (0.0008) [2023-12-27 03:58:23,089][105692] Updated weights for policy 0, policy_version 1726601 (0.0008) [2023-12-27 03:58:23,150][105692] Updated weights for policy 0, policy_version 1726611 (0.0007) [2023-12-27 03:58:23,201][105692] Updated weights for policy 0, policy_version 1726621 (0.0006) [2023-12-27 03:58:23,538][105620] Updated weights for policy 1, policy_version 1730232 (0.0008) [2023-12-27 03:58:23,589][105620] Updated weights for policy 1, policy_version 1730242 (0.0008) [2023-12-27 03:58:23,633][105620] Updated weights for policy 1, policy_version 1730252 (0.0008) [2023-12-27 03:58:23,919][105692] Updated weights for policy 0, policy_version 1726631 (0.0008) [2023-12-27 03:58:23,978][105692] Updated weights for policy 0, policy_version 1726641 (0.0009) [2023-12-27 03:58:24,038][105692] Updated weights for policy 0, policy_version 1726651 (0.0010) [2023-12-27 03:58:24,391][105620] Updated weights for policy 1, policy_version 1730262 (0.0009) [2023-12-27 03:58:24,442][105620] Updated weights for policy 1, policy_version 1730272 (0.0009) [2023-12-27 03:58:24,494][105620] Updated weights for policy 1, policy_version 1730282 (0.0009) [2023-12-27 03:58:24,754][105692] Updated weights for policy 0, policy_version 1726661 (0.0007) [2023-12-27 03:58:24,804][105692] Updated weights for policy 0, policy_version 1726671 (0.0007) [2023-12-27 03:58:24,852][105692] Updated weights for policy 0, policy_version 1726681 (0.0008) [2023-12-27 03:58:25,201][105620] Updated weights for policy 1, policy_version 1730292 (0.0008) [2023-12-27 03:58:25,266][105620] Updated weights for policy 1, policy_version 1730302 (0.0005) [2023-12-27 03:58:25,333][105620] Updated weights for policy 1, policy_version 1730312 (0.0005) [2023-12-27 03:58:25,655][105692] Updated weights for policy 0, policy_version 1726691 (0.0008) [2023-12-27 03:58:25,703][105692] Updated weights for policy 0, policy_version 1726701 (0.0008) [2023-12-27 03:58:25,752][105692] Updated weights for policy 0, policy_version 1726711 (0.0008) [2023-12-27 03:58:26,006][105620] Updated weights for policy 1, policy_version 1730322 (0.0007) [2023-12-27 03:58:26,057][105620] Updated weights for policy 1, policy_version 1730332 (0.0010) [2023-12-27 03:58:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 885129216. Throughput: 0: 9787.2, 1: 9697.9. Samples: 885139528. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:58:26,063][104569] Avg episode reward: [(0, '8254.769'), (1, '8899.675')] [2023-12-27 03:58:26,115][105620] Updated weights for policy 1, policy_version 1730342 (0.0010) [2023-12-27 03:58:26,179][105620] Updated weights for policy 1, policy_version 1730352 (0.0010) [2023-12-27 03:58:26,526][105692] Updated weights for policy 0, policy_version 1726721 (0.0008) [2023-12-27 03:58:26,575][105692] Updated weights for policy 0, policy_version 1726731 (0.0008) [2023-12-27 03:58:26,624][105692] Updated weights for policy 0, policy_version 1726741 (0.0008) [2023-12-27 03:58:26,688][105692] Updated weights for policy 0, policy_version 1726751 (0.0008) [2023-12-27 03:58:26,898][105620] Updated weights for policy 1, policy_version 1730362 (0.0010) [2023-12-27 03:58:26,960][105620] Updated weights for policy 1, policy_version 1730372 (0.0006) [2023-12-27 03:58:27,028][105620] Updated weights for policy 1, policy_version 1730382 (0.0005) [2023-12-27 03:58:27,316][105692] Updated weights for policy 0, policy_version 1726761 (0.0010) [2023-12-27 03:58:27,386][105692] Updated weights for policy 0, policy_version 1726771 (0.0011) [2023-12-27 03:58:27,437][105692] Updated weights for policy 0, policy_version 1726781 (0.0010) [2023-12-27 03:58:27,539][105620] Updated weights for policy 1, policy_version 1730392 (0.0005) [2023-12-27 03:58:27,596][105620] Updated weights for policy 1, policy_version 1730402 (0.0005) [2023-12-27 03:58:27,650][105620] Updated weights for policy 1, policy_version 1730412 (0.0006) [2023-12-27 03:58:28,054][105692] Updated weights for policy 0, policy_version 1726791 (0.0007) [2023-12-27 03:58:28,109][105692] Updated weights for policy 0, policy_version 1726801 (0.0005) [2023-12-27 03:58:28,155][105692] Updated weights for policy 0, policy_version 1726811 (0.0005) [2023-12-27 03:58:28,327][105620] Updated weights for policy 1, policy_version 1730422 (0.0010) [2023-12-27 03:58:28,384][105620] Updated weights for policy 1, policy_version 1730432 (0.0010) [2023-12-27 03:58:28,439][105620] Updated weights for policy 1, policy_version 1730442 (0.0010) [2023-12-27 03:58:28,769][105692] Updated weights for policy 0, policy_version 1726821 (0.0005) [2023-12-27 03:58:28,816][105692] Updated weights for policy 0, policy_version 1726831 (0.0005) [2023-12-27 03:58:28,869][105692] Updated weights for policy 0, policy_version 1726841 (0.0006) [2023-12-27 03:58:29,182][105620] Updated weights for policy 1, policy_version 1730452 (0.0010) [2023-12-27 03:58:29,255][105620] Updated weights for policy 1, policy_version 1730462 (0.0007) [2023-12-27 03:58:29,321][105620] Updated weights for policy 1, policy_version 1730472 (0.0009) [2023-12-27 03:58:29,451][105692] Updated weights for policy 0, policy_version 1726851 (0.0005) [2023-12-27 03:58:29,505][105692] Updated weights for policy 0, policy_version 1726861 (0.0005) [2023-12-27 03:58:29,554][105692] Updated weights for policy 0, policy_version 1726871 (0.0009) [2023-12-27 03:58:30,031][105620] Updated weights for policy 1, policy_version 1730482 (0.0009) [2023-12-27 03:58:30,095][105620] Updated weights for policy 1, policy_version 1730492 (0.0007) [2023-12-27 03:58:30,150][105620] Updated weights for policy 1, policy_version 1730502 (0.0010) [2023-12-27 03:58:30,222][105620] Updated weights for policy 1, policy_version 1730512 (0.0010) [2023-12-27 03:58:30,246][105692] Updated weights for policy 0, policy_version 1726881 (0.0010) [2023-12-27 03:58:30,314][105692] Updated weights for policy 0, policy_version 1726891 (0.0005) [2023-12-27 03:58:30,385][105692] Updated weights for policy 0, policy_version 1726901 (0.0005) [2023-12-27 03:58:30,452][105692] Updated weights for policy 0, policy_version 1726911 (0.0006) [2023-12-27 03:58:30,812][105620] Updated weights for policy 1, policy_version 1730522 (0.0005) [2023-12-27 03:58:30,882][105620] Updated weights for policy 1, policy_version 1730532 (0.0005) [2023-12-27 03:58:30,948][105620] Updated weights for policy 1, policy_version 1730542 (0.0005) [2023-12-27 03:58:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 885235712. Throughput: 0: 9874.0, 1: 9728.3. Samples: 885202168. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:58:31,062][104569] Avg episode reward: [(0, '8259.291'), (1, '8900.889')] [2023-12-27 03:58:31,066][105692] Updated weights for policy 0, policy_version 1726921 (0.0009) [2023-12-27 03:58:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001730544_443080704.pth... [2023-12-27 03:58:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001729392_442785792.pth [2023-12-27 03:58:31,133][105692] Updated weights for policy 0, policy_version 1726931 (0.0008) [2023-12-27 03:58:31,193][105692] Updated weights for policy 0, policy_version 1726941 (0.0008) [2023-12-27 03:58:31,209][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001726944_442163200.pth... [2023-12-27 03:58:31,212][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001725792_441868288.pth [2023-12-27 03:58:31,534][105620] Updated weights for policy 1, policy_version 1730552 (0.0005) [2023-12-27 03:58:31,592][105620] Updated weights for policy 1, policy_version 1730562 (0.0005) [2023-12-27 03:58:31,658][105620] Updated weights for policy 1, policy_version 1730572 (0.0007) [2023-12-27 03:58:32,005][105692] Updated weights for policy 0, policy_version 1726951 (0.0008) [2023-12-27 03:58:32,054][105692] Updated weights for policy 0, policy_version 1726961 (0.0009) [2023-12-27 03:58:32,108][105692] Updated weights for policy 0, policy_version 1726971 (0.0008) [2023-12-27 03:58:32,280][105620] Updated weights for policy 1, policy_version 1730582 (0.0009) [2023-12-27 03:58:32,327][105620] Updated weights for policy 1, policy_version 1730592 (0.0008) [2023-12-27 03:58:32,390][105620] Updated weights for policy 1, policy_version 1730602 (0.0008) [2023-12-27 03:58:32,961][105692] Updated weights for policy 0, policy_version 1726981 (0.0009) [2023-12-27 03:58:32,998][105620] Updated weights for policy 1, policy_version 1730612 (0.0007) [2023-12-27 03:58:33,017][105692] Updated weights for policy 0, policy_version 1726991 (0.0007) [2023-12-27 03:58:33,051][105620] Updated weights for policy 1, policy_version 1730622 (0.0008) [2023-12-27 03:58:33,062][105692] Updated weights for policy 0, policy_version 1727001 (0.0005) [2023-12-27 03:58:33,108][105620] Updated weights for policy 1, policy_version 1730632 (0.0007) [2023-12-27 03:58:33,799][105692] Updated weights for policy 0, policy_version 1727011 (0.0006) [2023-12-27 03:58:33,859][105692] Updated weights for policy 0, policy_version 1727021 (0.0009) [2023-12-27 03:58:33,865][105620] Updated weights for policy 1, policy_version 1730642 (0.0009) [2023-12-27 03:58:33,915][105692] Updated weights for policy 0, policy_version 1727031 (0.0005) [2023-12-27 03:58:33,917][105620] Updated weights for policy 1, policy_version 1730652 (0.0007) [2023-12-27 03:58:33,971][105620] Updated weights for policy 1, policy_version 1730662 (0.0007) [2023-12-27 03:58:34,029][105620] Updated weights for policy 1, policy_version 1730672 (0.0009) [2023-12-27 03:58:34,597][105692] Updated weights for policy 0, policy_version 1727041 (0.0006) [2023-12-27 03:58:34,652][105692] Updated weights for policy 0, policy_version 1727051 (0.0005) [2023-12-27 03:58:34,723][105692] Updated weights for policy 0, policy_version 1727061 (0.0006) [2023-12-27 03:58:34,767][105620] Updated weights for policy 1, policy_version 1730682 (0.0006) [2023-12-27 03:58:34,792][105692] Updated weights for policy 0, policy_version 1727071 (0.0008) [2023-12-27 03:58:34,828][105620] Updated weights for policy 1, policy_version 1730692 (0.0006) [2023-12-27 03:58:34,887][105620] Updated weights for policy 1, policy_version 1730702 (0.0009) [2023-12-27 03:58:35,462][105692] Updated weights for policy 0, policy_version 1727081 (0.0008) [2023-12-27 03:58:35,512][105620] Updated weights for policy 1, policy_version 1730712 (0.0010) [2023-12-27 03:58:35,514][105692] Updated weights for policy 0, policy_version 1727091 (0.0006) [2023-12-27 03:58:35,563][105620] Updated weights for policy 1, policy_version 1730722 (0.0010) [2023-12-27 03:58:35,570][105692] Updated weights for policy 0, policy_version 1727101 (0.0005) [2023-12-27 03:58:35,625][105620] Updated weights for policy 1, policy_version 1730732 (0.0010) [2023-12-27 03:58:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 885334016. Throughput: 0: 9789.7, 1: 9819.6. Samples: 885322832. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:58:36,063][104569] Avg episode reward: [(0, '8255.754'), (1, '8812.796')] [2023-12-27 03:58:36,215][105620] Updated weights for policy 1, policy_version 1730742 (0.0011) [2023-12-27 03:58:36,275][105620] Updated weights for policy 1, policy_version 1730752 (0.0011) [2023-12-27 03:58:36,335][105620] Updated weights for policy 1, policy_version 1730762 (0.0011) [2023-12-27 03:58:36,396][105692] Updated weights for policy 0, policy_version 1727111 (0.0008) [2023-12-27 03:58:36,445][105692] Updated weights for policy 0, policy_version 1727121 (0.0009) [2023-12-27 03:58:36,504][105692] Updated weights for policy 0, policy_version 1727131 (0.0009) [2023-12-27 03:58:36,989][105620] Updated weights for policy 1, policy_version 1730772 (0.0010) [2023-12-27 03:58:37,044][105620] Updated weights for policy 1, policy_version 1730782 (0.0009) [2023-12-27 03:58:37,107][105620] Updated weights for policy 1, policy_version 1730792 (0.0009) [2023-12-27 03:58:37,317][105692] Updated weights for policy 0, policy_version 1727141 (0.0009) [2023-12-27 03:58:37,368][105692] Updated weights for policy 0, policy_version 1727151 (0.0009) [2023-12-27 03:58:37,419][105692] Updated weights for policy 0, policy_version 1727161 (0.0009) [2023-12-27 03:58:37,862][105620] Updated weights for policy 1, policy_version 1730802 (0.0009) [2023-12-27 03:58:37,924][105620] Updated weights for policy 1, policy_version 1730813 (0.0008) [2023-12-27 03:58:37,972][105620] Updated weights for policy 1, policy_version 1730823 (0.0006) [2023-12-27 03:58:38,205][105692] Updated weights for policy 0, policy_version 1727171 (0.0009) [2023-12-27 03:58:38,263][105692] Updated weights for policy 0, policy_version 1727181 (0.0008) [2023-12-27 03:58:38,314][105692] Updated weights for policy 0, policy_version 1727191 (0.0009) [2023-12-27 03:58:38,651][105620] Updated weights for policy 1, policy_version 1730833 (0.0008) [2023-12-27 03:58:38,712][105620] Updated weights for policy 1, policy_version 1730843 (0.0009) [2023-12-27 03:58:38,770][105620] Updated weights for policy 1, policy_version 1730853 (0.0009) [2023-12-27 03:58:38,832][105620] Updated weights for policy 1, policy_version 1730863 (0.0005) [2023-12-27 03:58:39,202][105692] Updated weights for policy 0, policy_version 1727201 (0.0008) [2023-12-27 03:58:39,264][105692] Updated weights for policy 0, policy_version 1727211 (0.0010) [2023-12-27 03:58:39,311][105692] Updated weights for policy 0, policy_version 1727221 (0.0008) [2023-12-27 03:58:39,384][105692] Updated weights for policy 0, policy_version 1727231 (0.0008) [2023-12-27 03:58:39,438][105620] Updated weights for policy 1, policy_version 1730873 (0.0009) [2023-12-27 03:58:39,507][105620] Updated weights for policy 1, policy_version 1730883 (0.0008) [2023-12-27 03:58:39,577][105620] Updated weights for policy 1, policy_version 1730893 (0.0007) [2023-12-27 03:58:40,216][105692] Updated weights for policy 0, policy_version 1727241 (0.0009) [2023-12-27 03:58:40,279][105620] Updated weights for policy 1, policy_version 1730903 (0.0008) [2023-12-27 03:58:40,283][105692] Updated weights for policy 0, policy_version 1727251 (0.0008) [2023-12-27 03:58:40,340][105620] Updated weights for policy 1, policy_version 1730913 (0.0006) [2023-12-27 03:58:40,342][105692] Updated weights for policy 0, policy_version 1727261 (0.0008) [2023-12-27 03:58:40,402][105620] Updated weights for policy 1, policy_version 1730923 (0.0007) [2023-12-27 03:58:41,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 885424128. Throughput: 0: 9629.0, 1: 9888.2. Samples: 885437212. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:58:41,062][104569] Avg episode reward: [(0, '8529.129'), (1, '8905.722')] [2023-12-27 03:58:41,107][105692] Updated weights for policy 0, policy_version 1727271 (0.0008) [2023-12-27 03:58:41,174][105692] Updated weights for policy 0, policy_version 1727281 (0.0008) [2023-12-27 03:58:41,210][105620] Updated weights for policy 1, policy_version 1730933 (0.0007) [2023-12-27 03:58:41,236][105692] Updated weights for policy 0, policy_version 1727291 (0.0008) [2023-12-27 03:58:41,275][105620] Updated weights for policy 1, policy_version 1730943 (0.0006) [2023-12-27 03:58:41,328][105620] Updated weights for policy 1, policy_version 1730953 (0.0008) [2023-12-27 03:58:42,094][105620] Updated weights for policy 1, policy_version 1730963 (0.0008) [2023-12-27 03:58:42,104][105692] Updated weights for policy 0, policy_version 1727301 (0.0009) [2023-12-27 03:58:42,151][105620] Updated weights for policy 1, policy_version 1730973 (0.0005) [2023-12-27 03:58:42,164][105692] Updated weights for policy 0, policy_version 1727311 (0.0006) [2023-12-27 03:58:42,208][105620] Updated weights for policy 1, policy_version 1730983 (0.0006) [2023-12-27 03:58:42,224][105692] Updated weights for policy 0, policy_version 1727321 (0.0007) [2023-12-27 03:58:42,926][105692] Updated weights for policy 0, policy_version 1727331 (0.0008) [2023-12-27 03:58:42,939][105620] Updated weights for policy 1, policy_version 1730993 (0.0006) [2023-12-27 03:58:42,978][105692] Updated weights for policy 0, policy_version 1727341 (0.0007) [2023-12-27 03:58:42,996][105620] Updated weights for policy 1, policy_version 1731003 (0.0008) [2023-12-27 03:58:43,027][105692] Updated weights for policy 0, policy_version 1727351 (0.0007) [2023-12-27 03:58:43,056][105620] Updated weights for policy 1, policy_version 1731013 (0.0008) [2023-12-27 03:58:43,112][105620] Updated weights for policy 1, policy_version 1731023 (0.0009) [2023-12-27 03:58:43,624][105692] Updated weights for policy 0, policy_version 1727361 (0.0009) [2023-12-27 03:58:43,678][105692] Updated weights for policy 0, policy_version 1727371 (0.0005) [2023-12-27 03:58:43,733][105692] Updated weights for policy 0, policy_version 1727381 (0.0005) [2023-12-27 03:58:43,755][105620] Updated weights for policy 1, policy_version 1731033 (0.0010) [2023-12-27 03:58:43,775][105692] Updated weights for policy 0, policy_version 1727391 (0.0005) [2023-12-27 03:58:43,819][105620] Updated weights for policy 1, policy_version 1731043 (0.0010) [2023-12-27 03:58:43,877][105620] Updated weights for policy 1, policy_version 1731053 (0.0010) [2023-12-27 03:58:44,418][105692] Updated weights for policy 0, policy_version 1727401 (0.0008) [2023-12-27 03:58:44,474][105692] Updated weights for policy 0, policy_version 1727411 (0.0008) [2023-12-27 03:58:44,531][105692] Updated weights for policy 0, policy_version 1727421 (0.0008) [2023-12-27 03:58:44,603][105620] Updated weights for policy 1, policy_version 1731063 (0.0010) [2023-12-27 03:58:44,660][105620] Updated weights for policy 1, policy_version 1731073 (0.0008) [2023-12-27 03:58:44,706][105620] Updated weights for policy 1, policy_version 1731083 (0.0005) [2023-12-27 03:58:45,266][105692] Updated weights for policy 0, policy_version 1727431 (0.0008) [2023-12-27 03:58:45,332][105692] Updated weights for policy 0, policy_version 1727441 (0.0006) [2023-12-27 03:58:45,387][105692] Updated weights for policy 0, policy_version 1727451 (0.0009) [2023-12-27 03:58:45,435][105620] Updated weights for policy 1, policy_version 1731093 (0.0005) [2023-12-27 03:58:45,499][105620] Updated weights for policy 1, policy_version 1731103 (0.0007) [2023-12-27 03:58:45,566][105620] Updated weights for policy 1, policy_version 1731113 (0.0007) [2023-12-27 03:58:45,978][105692] Updated weights for policy 0, policy_version 1727461 (0.0007) [2023-12-27 03:58:46,048][105692] Updated weights for policy 0, policy_version 1727471 (0.0005) [2023-12-27 03:58:46,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.4, 300 sec: 19438.6). Total num frames: 885522432. Throughput: 0: 9566.0, 1: 9880.9. Samples: 885495340. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:58:46,062][104569] Avg episode reward: [(0, '8440.802'), (1, '8992.964')] [2023-12-27 03:58:46,082][105620] Updated weights for policy 1, policy_version 1731123 (0.0005) [2023-12-27 03:58:46,101][105692] Updated weights for policy 0, policy_version 1727481 (0.0006) [2023-12-27 03:58:46,137][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001727488_442302464.pth... [2023-12-27 03:58:46,140][105620] Updated weights for policy 1, policy_version 1731133 (0.0008) [2023-12-27 03:58:46,141][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001726368_442015744.pth [2023-12-27 03:58:46,190][105620] Updated weights for policy 1, policy_version 1731143 (0.0009) [2023-12-27 03:58:46,239][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001731152_443236352.pth... [2023-12-27 03:58:46,243][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001729968_442933248.pth [2023-12-27 03:58:46,636][105692] Updated weights for policy 0, policy_version 1727491 (0.0007) [2023-12-27 03:58:46,691][105692] Updated weights for policy 0, policy_version 1727501 (0.0010) [2023-12-27 03:58:46,747][105692] Updated weights for policy 0, policy_version 1727511 (0.0010) [2023-12-27 03:58:46,903][105620] Updated weights for policy 1, policy_version 1731153 (0.0009) [2023-12-27 03:58:46,976][105620] Updated weights for policy 1, policy_version 1731163 (0.0009) [2023-12-27 03:58:47,038][105620] Updated weights for policy 1, policy_version 1731173 (0.0008) [2023-12-27 03:58:47,101][105620] Updated weights for policy 1, policy_version 1731183 (0.0007) [2023-12-27 03:58:47,426][105692] Updated weights for policy 0, policy_version 1727521 (0.0010) [2023-12-27 03:58:47,480][105692] Updated weights for policy 0, policy_version 1727531 (0.0005) [2023-12-27 03:58:47,528][105692] Updated weights for policy 0, policy_version 1727541 (0.0006) [2023-12-27 03:58:47,577][105692] Updated weights for policy 0, policy_version 1727552 (0.0010) [2023-12-27 03:58:47,868][105620] Updated weights for policy 1, policy_version 1731193 (0.0008) [2023-12-27 03:58:47,927][105620] Updated weights for policy 1, policy_version 1731203 (0.0009) [2023-12-27 03:58:48,001][105620] Updated weights for policy 1, policy_version 1731213 (0.0010) [2023-12-27 03:58:48,272][105692] Updated weights for policy 0, policy_version 1727562 (0.0009) [2023-12-27 03:58:48,325][105692] Updated weights for policy 0, policy_version 1727572 (0.0006) [2023-12-27 03:58:48,384][105692] Updated weights for policy 0, policy_version 1727582 (0.0008) [2023-12-27 03:58:48,776][105620] Updated weights for policy 1, policy_version 1731223 (0.0009) [2023-12-27 03:58:48,839][105620] Updated weights for policy 1, policy_version 1731233 (0.0007) [2023-12-27 03:58:48,902][105620] Updated weights for policy 1, policy_version 1731243 (0.0005) [2023-12-27 03:58:49,099][105692] Updated weights for policy 0, policy_version 1727592 (0.0011) [2023-12-27 03:58:49,162][105692] Updated weights for policy 0, policy_version 1727602 (0.0011) [2023-12-27 03:58:49,218][105692] Updated weights for policy 0, policy_version 1727612 (0.0011) [2023-12-27 03:58:49,588][105620] Updated weights for policy 1, policy_version 1731253 (0.0007) [2023-12-27 03:58:49,636][105620] Updated weights for policy 1, policy_version 1731263 (0.0008) [2023-12-27 03:58:49,696][105620] Updated weights for policy 1, policy_version 1731273 (0.0008) [2023-12-27 03:58:49,966][105692] Updated weights for policy 0, policy_version 1727622 (0.0008) [2023-12-27 03:58:50,034][105692] Updated weights for policy 0, policy_version 1727632 (0.0011) [2023-12-27 03:58:50,103][105692] Updated weights for policy 0, policy_version 1727642 (0.0010) [2023-12-27 03:58:50,496][105620] Updated weights for policy 1, policy_version 1731283 (0.0007) [2023-12-27 03:58:50,558][105620] Updated weights for policy 1, policy_version 1731293 (0.0006) [2023-12-27 03:58:50,626][105620] Updated weights for policy 1, policy_version 1731303 (0.0009) [2023-12-27 03:58:50,827][105692] Updated weights for policy 0, policy_version 1727652 (0.0010) [2023-12-27 03:58:50,886][105692] Updated weights for policy 0, policy_version 1727662 (0.0010) [2023-12-27 03:58:50,951][105692] Updated weights for policy 0, policy_version 1727672 (0.0010) [2023-12-27 03:58:51,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 885628928. Throughput: 0: 9589.9, 1: 9877.6. Samples: 885616512. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:58:51,062][104569] Avg episode reward: [(0, '8351.253'), (1, '8901.023')] [2023-12-27 03:58:51,397][105620] Updated weights for policy 1, policy_version 1731313 (0.0009) [2023-12-27 03:58:51,454][105620] Updated weights for policy 1, policy_version 1731323 (0.0008) [2023-12-27 03:58:51,516][105620] Updated weights for policy 1, policy_version 1731333 (0.0005) [2023-12-27 03:58:51,572][105620] Updated weights for policy 1, policy_version 1731343 (0.0006) [2023-12-27 03:58:51,642][105692] Updated weights for policy 0, policy_version 1727682 (0.0010) [2023-12-27 03:58:51,708][105692] Updated weights for policy 0, policy_version 1727692 (0.0009) [2023-12-27 03:58:51,778][105692] Updated weights for policy 0, policy_version 1727702 (0.0009) [2023-12-27 03:58:51,837][105692] Updated weights for policy 0, policy_version 1727712 (0.0007) [2023-12-27 03:58:52,185][105620] Updated weights for policy 1, policy_version 1731353 (0.0006) [2023-12-27 03:58:52,245][105620] Updated weights for policy 1, policy_version 1731363 (0.0008) [2023-12-27 03:58:52,305][105620] Updated weights for policy 1, policy_version 1731373 (0.0010) [2023-12-27 03:58:52,491][105692] Updated weights for policy 0, policy_version 1727722 (0.0005) [2023-12-27 03:58:52,550][105692] Updated weights for policy 0, policy_version 1727732 (0.0007) [2023-12-27 03:58:52,616][105692] Updated weights for policy 0, policy_version 1727742 (0.0007) [2023-12-27 03:58:53,132][105620] Updated weights for policy 1, policy_version 1731383 (0.0009) [2023-12-27 03:58:53,190][105620] Updated weights for policy 1, policy_version 1731393 (0.0009) [2023-12-27 03:58:53,242][105620] Updated weights for policy 1, policy_version 1731403 (0.0008) [2023-12-27 03:58:53,256][105692] Updated weights for policy 0, policy_version 1727752 (0.0008) [2023-12-27 03:58:53,314][105692] Updated weights for policy 0, policy_version 1727762 (0.0009) [2023-12-27 03:58:53,380][105692] Updated weights for policy 0, policy_version 1727772 (0.0009) [2023-12-27 03:58:53,987][105620] Updated weights for policy 1, policy_version 1731413 (0.0007) [2023-12-27 03:58:54,023][105692] Updated weights for policy 0, policy_version 1727782 (0.0008) [2023-12-27 03:58:54,054][105620] Updated weights for policy 1, policy_version 1731423 (0.0009) [2023-12-27 03:58:54,088][105692] Updated weights for policy 0, policy_version 1727792 (0.0006) [2023-12-27 03:58:54,122][105620] Updated weights for policy 1, policy_version 1731433 (0.0010) [2023-12-27 03:58:54,149][105692] Updated weights for policy 0, policy_version 1727802 (0.0009) [2023-12-27 03:58:54,786][105620] Updated weights for policy 1, policy_version 1731443 (0.0010) [2023-12-27 03:58:54,844][105620] Updated weights for policy 1, policy_version 1731453 (0.0008) [2023-12-27 03:58:54,912][105620] Updated weights for policy 1, policy_version 1731463 (0.0006) [2023-12-27 03:58:54,933][105692] Updated weights for policy 0, policy_version 1727812 (0.0007) [2023-12-27 03:58:55,000][105692] Updated weights for policy 0, policy_version 1727822 (0.0010) [2023-12-27 03:58:55,063][105692] Updated weights for policy 0, policy_version 1727832 (0.0010) [2023-12-27 03:58:55,607][105620] Updated weights for policy 1, policy_version 1731473 (0.0008) [2023-12-27 03:58:55,666][105620] Updated weights for policy 1, policy_version 1731483 (0.0010) [2023-12-27 03:58:55,718][105620] Updated weights for policy 1, policy_version 1731493 (0.0010) [2023-12-27 03:58:55,719][105692] Updated weights for policy 0, policy_version 1727842 (0.0009) [2023-12-27 03:58:55,765][105692] Updated weights for policy 0, policy_version 1727852 (0.0005) [2023-12-27 03:58:55,776][105620] Updated weights for policy 1, policy_version 1731503 (0.0010) [2023-12-27 03:58:55,815][105692] Updated weights for policy 0, policy_version 1727862 (0.0005) [2023-12-27 03:58:55,877][105692] Updated weights for policy 0, policy_version 1727872 (0.0005) [2023-12-27 03:58:56,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 885727232. Throughput: 0: 9650.7, 1: 9869.9. Samples: 885732332. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:58:56,063][104569] Avg episode reward: [(0, '8348.957'), (1, '8898.446')] [2023-12-27 03:58:56,466][105620] Updated weights for policy 1, policy_version 1731513 (0.0010) [2023-12-27 03:58:56,521][105620] Updated weights for policy 1, policy_version 1731523 (0.0010) [2023-12-27 03:58:56,564][105692] Updated weights for policy 0, policy_version 1727882 (0.0005) [2023-12-27 03:58:56,576][105620] Updated weights for policy 1, policy_version 1731533 (0.0010) [2023-12-27 03:58:56,613][105692] Updated weights for policy 0, policy_version 1727892 (0.0005) [2023-12-27 03:58:56,665][105692] Updated weights for policy 0, policy_version 1727902 (0.0005) [2023-12-27 03:58:57,210][105692] Updated weights for policy 0, policy_version 1727912 (0.0005) [2023-12-27 03:58:57,261][105692] Updated weights for policy 0, policy_version 1727922 (0.0005) [2023-12-27 03:58:57,311][105692] Updated weights for policy 0, policy_version 1727932 (0.0006) [2023-12-27 03:58:57,334][105620] Updated weights for policy 1, policy_version 1731543 (0.0009) [2023-12-27 03:58:57,389][105620] Updated weights for policy 1, policy_version 1731553 (0.0010) [2023-12-27 03:58:57,450][105620] Updated weights for policy 1, policy_version 1731563 (0.0010) [2023-12-27 03:58:58,021][105692] Updated weights for policy 0, policy_version 1727942 (0.0008) [2023-12-27 03:58:58,078][105692] Updated weights for policy 0, policy_version 1727954 (0.0010) [2023-12-27 03:58:58,113][105620] Updated weights for policy 1, policy_version 1731573 (0.0008) [2023-12-27 03:58:58,134][105692] Updated weights for policy 0, policy_version 1727964 (0.0008) [2023-12-27 03:58:58,178][105620] Updated weights for policy 1, policy_version 1731583 (0.0009) [2023-12-27 03:58:58,240][105620] Updated weights for policy 1, policy_version 1731593 (0.0008) [2023-12-27 03:58:59,002][105692] Updated weights for policy 0, policy_version 1727974 (0.0009) [2023-12-27 03:58:59,067][105692] Updated weights for policy 0, policy_version 1727984 (0.0009) [2023-12-27 03:58:59,132][105692] Updated weights for policy 0, policy_version 1727994 (0.0009) [2023-12-27 03:58:59,147][105620] Updated weights for policy 1, policy_version 1731603 (0.0008) [2023-12-27 03:58:59,211][105620] Updated weights for policy 1, policy_version 1731613 (0.0008) [2023-12-27 03:58:59,282][105620] Updated weights for policy 1, policy_version 1731623 (0.0008) [2023-12-27 03:58:59,801][105692] Updated weights for policy 0, policy_version 1728004 (0.0007) [2023-12-27 03:58:59,858][105692] Updated weights for policy 0, policy_version 1728014 (0.0009) [2023-12-27 03:58:59,912][105692] Updated weights for policy 0, policy_version 1728024 (0.0010) [2023-12-27 03:58:59,990][105620] Updated weights for policy 1, policy_version 1731633 (0.0010) [2023-12-27 03:59:00,050][105620] Updated weights for policy 1, policy_version 1731643 (0.0007) [2023-12-27 03:59:00,113][105620] Updated weights for policy 1, policy_version 1731653 (0.0005) [2023-12-27 03:59:00,167][105620] Updated weights for policy 1, policy_version 1731663 (0.0005) [2023-12-27 03:59:00,572][105692] Updated weights for policy 0, policy_version 1728034 (0.0008) [2023-12-27 03:59:00,625][105692] Updated weights for policy 0, policy_version 1728044 (0.0006) [2023-12-27 03:59:00,679][105692] Updated weights for policy 0, policy_version 1728054 (0.0008) [2023-12-27 03:59:00,726][105692] Updated weights for policy 0, policy_version 1728064 (0.0007) [2023-12-27 03:59:00,915][105620] Updated weights for policy 1, policy_version 1731673 (0.0008) [2023-12-27 03:59:00,967][105620] Updated weights for policy 1, policy_version 1731683 (0.0009) [2023-12-27 03:59:01,018][105620] Updated weights for policy 1, policy_version 1731693 (0.0009) [2023-12-27 03:59:01,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.7, 300 sec: 19494.2). Total num frames: 885825536. Throughput: 0: 9705.7, 1: 9876.5. Samples: 885791948. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:59:01,063][104569] Avg episode reward: [(0, '8619.803'), (1, '8988.112')] [2023-12-27 03:59:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001728064_442449920.pth... [2023-12-27 03:59:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001731696_443375616.pth... [2023-12-27 03:59:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001726944_442163200.pth [2023-12-27 03:59:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001730544_443080704.pth [2023-12-27 03:59:01,421][105692] Updated weights for policy 0, policy_version 1728074 (0.0008) [2023-12-27 03:59:01,470][105692] Updated weights for policy 0, policy_version 1728084 (0.0008) [2023-12-27 03:59:01,514][105692] Updated weights for policy 0, policy_version 1728094 (0.0007) [2023-12-27 03:59:01,850][105620] Updated weights for policy 1, policy_version 1731703 (0.0009) [2023-12-27 03:59:01,909][105620] Updated weights for policy 1, policy_version 1731713 (0.0008) [2023-12-27 03:59:01,965][105620] Updated weights for policy 1, policy_version 1731724 (0.0010) [2023-12-27 03:59:02,156][105692] Updated weights for policy 0, policy_version 1728104 (0.0005) [2023-12-27 03:59:02,216][105692] Updated weights for policy 0, policy_version 1728114 (0.0005) [2023-12-27 03:59:02,273][105692] Updated weights for policy 0, policy_version 1728124 (0.0008) [2023-12-27 03:59:02,747][105620] Updated weights for policy 1, policy_version 1731734 (0.0007) [2023-12-27 03:59:02,805][105620] Updated weights for policy 1, policy_version 1731744 (0.0008) [2023-12-27 03:59:02,852][105620] Updated weights for policy 1, policy_version 1731754 (0.0009) [2023-12-27 03:59:02,934][105692] Updated weights for policy 0, policy_version 1728134 (0.0008) [2023-12-27 03:59:02,993][105692] Updated weights for policy 0, policy_version 1728144 (0.0008) [2023-12-27 03:59:03,049][105692] Updated weights for policy 0, policy_version 1728154 (0.0006) [2023-12-27 03:59:03,571][105620] Updated weights for policy 1, policy_version 1731764 (0.0006) [2023-12-27 03:59:03,639][105620] Updated weights for policy 1, policy_version 1731774 (0.0008) [2023-12-27 03:59:03,689][105620] Updated weights for policy 1, policy_version 1731784 (0.0008) [2023-12-27 03:59:03,743][105692] Updated weights for policy 0, policy_version 1728164 (0.0006) [2023-12-27 03:59:03,793][105692] Updated weights for policy 0, policy_version 1728174 (0.0009) [2023-12-27 03:59:03,843][105692] Updated weights for policy 0, policy_version 1728184 (0.0009) [2023-12-27 03:59:04,438][105620] Updated weights for policy 1, policy_version 1731794 (0.0009) [2023-12-27 03:59:04,493][105620] Updated weights for policy 1, policy_version 1731804 (0.0009) [2023-12-27 03:59:04,550][105620] Updated weights for policy 1, policy_version 1731814 (0.0009) [2023-12-27 03:59:04,609][105620] Updated weights for policy 1, policy_version 1731824 (0.0008) [2023-12-27 03:59:04,656][105692] Updated weights for policy 0, policy_version 1728194 (0.0010) [2023-12-27 03:59:04,721][105692] Updated weights for policy 0, policy_version 1728204 (0.0011) [2023-12-27 03:59:04,787][105692] Updated weights for policy 0, policy_version 1728214 (0.0007) [2023-12-27 03:59:04,847][105692] Updated weights for policy 0, policy_version 1728224 (0.0006) [2023-12-27 03:59:05,352][105692] Updated weights for policy 0, policy_version 1728234 (0.0005) [2023-12-27 03:59:05,402][105620] Updated weights for policy 1, policy_version 1731834 (0.0008) [2023-12-27 03:59:05,415][105692] Updated weights for policy 0, policy_version 1728244 (0.0005) [2023-12-27 03:59:05,463][105620] Updated weights for policy 1, policy_version 1731844 (0.0006) [2023-12-27 03:59:05,468][105692] Updated weights for policy 0, policy_version 1728254 (0.0009) [2023-12-27 03:59:05,525][105620] Updated weights for policy 1, policy_version 1731854 (0.0007) [2023-12-27 03:59:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 885915648. Throughput: 0: 9799.4, 1: 9750.8. Samples: 885907336. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:59:06,062][104569] Avg episode reward: [(0, '9077.160'), (1, '8900.027')] [2023-12-27 03:59:06,168][105692] Updated weights for policy 0, policy_version 1728264 (0.0009) [2023-12-27 03:59:06,219][105692] Updated weights for policy 0, policy_version 1728274 (0.0009) [2023-12-27 03:59:06,255][105620] Updated weights for policy 1, policy_version 1731864 (0.0007) [2023-12-27 03:59:06,281][105692] Updated weights for policy 0, policy_version 1728284 (0.0008) [2023-12-27 03:59:06,320][105620] Updated weights for policy 1, policy_version 1731874 (0.0008) [2023-12-27 03:59:06,374][105620] Updated weights for policy 1, policy_version 1731884 (0.0009) [2023-12-27 03:59:06,950][105692] Updated weights for policy 0, policy_version 1728294 (0.0008) [2023-12-27 03:59:06,999][105692] Updated weights for policy 0, policy_version 1728304 (0.0009) [2023-12-27 03:59:07,051][105692] Updated weights for policy 0, policy_version 1728314 (0.0009) [2023-12-27 03:59:07,165][105620] Updated weights for policy 1, policy_version 1731894 (0.0009) [2023-12-27 03:59:07,216][105620] Updated weights for policy 1, policy_version 1731904 (0.0009) [2023-12-27 03:59:07,274][105620] Updated weights for policy 1, policy_version 1731914 (0.0009) [2023-12-27 03:59:07,874][105692] Updated weights for policy 0, policy_version 1728324 (0.0009) [2023-12-27 03:59:07,929][105692] Updated weights for policy 0, policy_version 1728334 (0.0009) [2023-12-27 03:59:07,995][105692] Updated weights for policy 0, policy_version 1728344 (0.0008) [2023-12-27 03:59:08,024][105620] Updated weights for policy 1, policy_version 1731924 (0.0010) [2023-12-27 03:59:08,087][105620] Updated weights for policy 1, policy_version 1731934 (0.0009) [2023-12-27 03:59:08,133][105620] Updated weights for policy 1, policy_version 1731944 (0.0008) [2023-12-27 03:59:08,675][105692] Updated weights for policy 0, policy_version 1728354 (0.0007) [2023-12-27 03:59:08,741][105692] Updated weights for policy 0, policy_version 1728364 (0.0009) [2023-12-27 03:59:08,809][105692] Updated weights for policy 0, policy_version 1728374 (0.0009) [2023-12-27 03:59:08,869][105692] Updated weights for policy 0, policy_version 1728384 (0.0009) [2023-12-27 03:59:08,946][105620] Updated weights for policy 1, policy_version 1731954 (0.0009) [2023-12-27 03:59:09,008][105620] Updated weights for policy 1, policy_version 1731964 (0.0008) [2023-12-27 03:59:09,069][105620] Updated weights for policy 1, policy_version 1731974 (0.0008) [2023-12-27 03:59:09,119][105620] Updated weights for policy 1, policy_version 1731984 (0.0008) [2023-12-27 03:59:09,671][105692] Updated weights for policy 0, policy_version 1728394 (0.0008) [2023-12-27 03:59:09,735][105692] Updated weights for policy 0, policy_version 1728404 (0.0010) [2023-12-27 03:59:09,798][105692] Updated weights for policy 0, policy_version 1728414 (0.0011) [2023-12-27 03:59:09,887][105620] Updated weights for policy 1, policy_version 1731994 (0.0008) [2023-12-27 03:59:09,952][105620] Updated weights for policy 1, policy_version 1732004 (0.0008) [2023-12-27 03:59:10,017][105620] Updated weights for policy 1, policy_version 1732014 (0.0009) [2023-12-27 03:59:10,605][105692] Updated weights for policy 0, policy_version 1728424 (0.0009) [2023-12-27 03:59:10,664][105692] Updated weights for policy 0, policy_version 1728434 (0.0010) [2023-12-27 03:59:10,721][105692] Updated weights for policy 0, policy_version 1728444 (0.0009) [2023-12-27 03:59:10,728][105620] Updated weights for policy 1, policy_version 1732024 (0.0007) [2023-12-27 03:59:10,780][105620] Updated weights for policy 1, policy_version 1732034 (0.0010) [2023-12-27 03:59:10,838][105620] Updated weights for policy 1, policy_version 1732044 (0.0010) [2023-12-27 03:59:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 886013952. Throughput: 0: 9874.8, 1: 9705.3. Samples: 886020636. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-12-27 03:59:11,063][104569] Avg episode reward: [(0, '8802.226'), (1, '8901.169')] [2023-12-27 03:59:11,423][105692] Updated weights for policy 0, policy_version 1728454 (0.0008) [2023-12-27 03:59:11,490][105692] Updated weights for policy 0, policy_version 1728464 (0.0009) [2023-12-27 03:59:11,553][105692] Updated weights for policy 0, policy_version 1728474 (0.0009) [2023-12-27 03:59:11,639][105620] Updated weights for policy 1, policy_version 1732054 (0.0009) [2023-12-27 03:59:11,708][105620] Updated weights for policy 1, policy_version 1732064 (0.0009) [2023-12-27 03:59:11,787][105620] Updated weights for policy 1, policy_version 1732074 (0.0008) [2023-12-27 03:59:12,198][105692] Updated weights for policy 0, policy_version 1728484 (0.0008) [2023-12-27 03:59:12,253][105692] Updated weights for policy 0, policy_version 1728494 (0.0006) [2023-12-27 03:59:12,309][105692] Updated weights for policy 0, policy_version 1728504 (0.0009) [2023-12-27 03:59:12,543][105620] Updated weights for policy 1, policy_version 1732084 (0.0009) [2023-12-27 03:59:12,590][105620] Updated weights for policy 1, policy_version 1732094 (0.0009) [2023-12-27 03:59:12,636][105620] Updated weights for policy 1, policy_version 1732104 (0.0008) [2023-12-27 03:59:13,024][105692] Updated weights for policy 0, policy_version 1728514 (0.0009) [2023-12-27 03:59:13,082][105692] Updated weights for policy 0, policy_version 1728524 (0.0009) [2023-12-27 03:59:13,130][105692] Updated weights for policy 0, policy_version 1728534 (0.0009) [2023-12-27 03:59:13,178][105692] Updated weights for policy 0, policy_version 1728544 (0.0009) [2023-12-27 03:59:13,428][105620] Updated weights for policy 1, policy_version 1732114 (0.0008) [2023-12-27 03:59:13,482][105620] Updated weights for policy 1, policy_version 1732124 (0.0005) [2023-12-27 03:59:13,535][105620] Updated weights for policy 1, policy_version 1732134 (0.0005) [2023-12-27 03:59:13,592][105620] Updated weights for policy 1, policy_version 1732144 (0.0007) [2023-12-27 03:59:13,868][105692] Updated weights for policy 0, policy_version 1728554 (0.0005) [2023-12-27 03:59:13,925][105692] Updated weights for policy 0, policy_version 1728564 (0.0005) [2023-12-27 03:59:13,971][105692] Updated weights for policy 0, policy_version 1728574 (0.0008) [2023-12-27 03:59:14,317][105620] Updated weights for policy 1, policy_version 1732154 (0.0010) [2023-12-27 03:59:14,381][105620] Updated weights for policy 1, policy_version 1732164 (0.0010) [2023-12-27 03:59:14,444][105620] Updated weights for policy 1, policy_version 1732174 (0.0009) [2023-12-27 03:59:14,602][105692] Updated weights for policy 0, policy_version 1728584 (0.0007) [2023-12-27 03:59:14,658][105692] Updated weights for policy 0, policy_version 1728594 (0.0005) [2023-12-27 03:59:14,710][105692] Updated weights for policy 0, policy_version 1728604 (0.0005) [2023-12-27 03:59:15,238][105620] Updated weights for policy 1, policy_version 1732184 (0.0009) [2023-12-27 03:59:15,292][105620] Updated weights for policy 1, policy_version 1732194 (0.0009) [2023-12-27 03:59:15,356][105620] Updated weights for policy 1, policy_version 1732204 (0.0009) [2023-12-27 03:59:15,393][105692] Updated weights for policy 0, policy_version 1728614 (0.0008) [2023-12-27 03:59:15,444][105692] Updated weights for policy 0, policy_version 1728624 (0.0009) [2023-12-27 03:59:15,495][105692] Updated weights for policy 0, policy_version 1728634 (0.0009) [2023-12-27 03:59:16,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 886104064. Throughput: 0: 9833.5, 1: 9635.5. Samples: 886078276. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 03:59:16,063][104569] Avg episode reward: [(0, '8527.086'), (1, '8987.877')] [2023-12-27 03:59:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001728640_442597376.pth... [2023-12-27 03:59:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001732208_443506688.pth... [2023-12-27 03:59:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001727488_442302464.pth [2023-12-27 03:59:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001731152_443236352.pth [2023-12-27 03:59:16,120][105620] Updated weights for policy 1, policy_version 1732214 (0.0008) [2023-12-27 03:59:16,171][105620] Updated weights for policy 1, policy_version 1732224 (0.0009) [2023-12-27 03:59:16,219][105692] Updated weights for policy 0, policy_version 1728644 (0.0007) [2023-12-27 03:59:16,229][105620] Updated weights for policy 1, policy_version 1732234 (0.0008) [2023-12-27 03:59:16,268][105692] Updated weights for policy 0, policy_version 1728654 (0.0007) [2023-12-27 03:59:16,314][105692] Updated weights for policy 0, policy_version 1728664 (0.0008) [2023-12-27 03:59:16,992][105620] Updated weights for policy 1, policy_version 1732244 (0.0007) [2023-12-27 03:59:16,995][105692] Updated weights for policy 0, policy_version 1728674 (0.0007) [2023-12-27 03:59:17,044][105620] Updated weights for policy 1, policy_version 1732254 (0.0005) [2023-12-27 03:59:17,058][105692] Updated weights for policy 0, policy_version 1728684 (0.0008) [2023-12-27 03:59:17,106][105620] Updated weights for policy 1, policy_version 1732264 (0.0005) [2023-12-27 03:59:17,122][105692] Updated weights for policy 0, policy_version 1728694 (0.0008) [2023-12-27 03:59:17,195][105692] Updated weights for policy 0, policy_version 1728704 (0.0009) [2023-12-27 03:59:17,726][105620] Updated weights for policy 1, policy_version 1732274 (0.0006) [2023-12-27 03:59:17,775][105620] Updated weights for policy 1, policy_version 1732284 (0.0008) [2023-12-27 03:59:17,832][105620] Updated weights for policy 1, policy_version 1732294 (0.0009) [2023-12-27 03:59:17,888][105620] Updated weights for policy 1, policy_version 1732304 (0.0008) [2023-12-27 03:59:17,922][105692] Updated weights for policy 0, policy_version 1728714 (0.0006) [2023-12-27 03:59:17,990][105692] Updated weights for policy 0, policy_version 1728724 (0.0005) [2023-12-27 03:59:18,053][105692] Updated weights for policy 0, policy_version 1728734 (0.0006) [2023-12-27 03:59:18,670][105692] Updated weights for policy 0, policy_version 1728744 (0.0008) [2023-12-27 03:59:18,712][105620] Updated weights for policy 1, policy_version 1732314 (0.0009) [2023-12-27 03:59:18,734][105692] Updated weights for policy 0, policy_version 1728754 (0.0006) [2023-12-27 03:59:18,773][105620] Updated weights for policy 1, policy_version 1732324 (0.0009) [2023-12-27 03:59:18,801][105692] Updated weights for policy 0, policy_version 1728764 (0.0006) [2023-12-27 03:59:18,829][105620] Updated weights for policy 1, policy_version 1732334 (0.0006) [2023-12-27 03:59:19,362][105692] Updated weights for policy 0, policy_version 1728774 (0.0008) [2023-12-27 03:59:19,425][105692] Updated weights for policy 0, policy_version 1728784 (0.0006) [2023-12-27 03:59:19,496][105692] Updated weights for policy 0, policy_version 1728794 (0.0009) [2023-12-27 03:59:19,692][105620] Updated weights for policy 1, policy_version 1732344 (0.0009) [2023-12-27 03:59:19,754][105620] Updated weights for policy 1, policy_version 1732354 (0.0009) [2023-12-27 03:59:19,812][105620] Updated weights for policy 1, policy_version 1732364 (0.0009) [2023-12-27 03:59:20,145][105692] Updated weights for policy 0, policy_version 1728804 (0.0006) [2023-12-27 03:59:20,221][105692] Updated weights for policy 0, policy_version 1728814 (0.0006) [2023-12-27 03:59:20,287][105692] Updated weights for policy 0, policy_version 1728824 (0.0009) [2023-12-27 03:59:20,540][105620] Updated weights for policy 1, policy_version 1732374 (0.0009) [2023-12-27 03:59:20,604][105620] Updated weights for policy 1, policy_version 1732384 (0.0009) [2023-12-27 03:59:20,670][105620] Updated weights for policy 1, policy_version 1732394 (0.0009) [2023-12-27 03:59:21,026][105692] Updated weights for policy 0, policy_version 1728834 (0.0009) [2023-12-27 03:59:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 886202368. Throughput: 0: 9912.4, 1: 9472.5. Samples: 886195148. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 03:59:21,063][104569] Avg episode reward: [(0, '8532.604'), (1, '8711.725')] [2023-12-27 03:59:21,090][105692] Updated weights for policy 0, policy_version 1728844 (0.0007) [2023-12-27 03:59:21,154][105692] Updated weights for policy 0, policy_version 1728854 (0.0008) [2023-12-27 03:59:21,214][105692] Updated weights for policy 0, policy_version 1728864 (0.0008) [2023-12-27 03:59:21,507][105620] Updated weights for policy 1, policy_version 1732404 (0.0008) [2023-12-27 03:59:21,569][105620] Updated weights for policy 1, policy_version 1732414 (0.0009) [2023-12-27 03:59:21,632][105620] Updated weights for policy 1, policy_version 1732424 (0.0009) [2023-12-27 03:59:21,956][105692] Updated weights for policy 0, policy_version 1728874 (0.0008) [2023-12-27 03:59:22,006][105692] Updated weights for policy 0, policy_version 1728884 (0.0009) [2023-12-27 03:59:22,066][105692] Updated weights for policy 0, policy_version 1728894 (0.0009) [2023-12-27 03:59:22,344][105620] Updated weights for policy 1, policy_version 1732434 (0.0008) [2023-12-27 03:59:22,412][105620] Updated weights for policy 1, policy_version 1732444 (0.0008) [2023-12-27 03:59:22,463][105620] Updated weights for policy 1, policy_version 1732454 (0.0009) [2023-12-27 03:59:22,514][105620] Updated weights for policy 1, policy_version 1732464 (0.0008) [2023-12-27 03:59:22,934][105692] Updated weights for policy 0, policy_version 1728904 (0.0009) [2023-12-27 03:59:22,987][105692] Updated weights for policy 0, policy_version 1728914 (0.0009) [2023-12-27 03:59:23,039][105692] Updated weights for policy 0, policy_version 1728924 (0.0007) [2023-12-27 03:59:23,239][105620] Updated weights for policy 1, policy_version 1732474 (0.0008) [2023-12-27 03:59:23,295][105620] Updated weights for policy 1, policy_version 1732484 (0.0007) [2023-12-27 03:59:23,363][105620] Updated weights for policy 1, policy_version 1732494 (0.0006) [2023-12-27 03:59:23,737][105692] Updated weights for policy 0, policy_version 1728934 (0.0005) [2023-12-27 03:59:23,789][105692] Updated weights for policy 0, policy_version 1728944 (0.0005) [2023-12-27 03:59:23,846][105692] Updated weights for policy 0, policy_version 1728954 (0.0006) [2023-12-27 03:59:24,071][105620] Updated weights for policy 1, policy_version 1732504 (0.0009) [2023-12-27 03:59:24,130][105620] Updated weights for policy 1, policy_version 1732514 (0.0010) [2023-12-27 03:59:24,204][105620] Updated weights for policy 1, policy_version 1732524 (0.0011) [2023-12-27 03:59:24,513][105692] Updated weights for policy 0, policy_version 1728964 (0.0007) [2023-12-27 03:59:24,578][105692] Updated weights for policy 0, policy_version 1728974 (0.0009) [2023-12-27 03:59:24,644][105692] Updated weights for policy 0, policy_version 1728984 (0.0007) [2023-12-27 03:59:24,961][105620] Updated weights for policy 1, policy_version 1732534 (0.0010) [2023-12-27 03:59:25,014][105620] Updated weights for policy 1, policy_version 1732544 (0.0011) [2023-12-27 03:59:25,063][105620] Updated weights for policy 1, policy_version 1732554 (0.0010) [2023-12-27 03:59:25,265][105692] Updated weights for policy 0, policy_version 1728994 (0.0009) [2023-12-27 03:59:25,314][105692] Updated weights for policy 0, policy_version 1729004 (0.0007) [2023-12-27 03:59:25,361][105692] Updated weights for policy 0, policy_version 1729014 (0.0005) [2023-12-27 03:59:25,410][105692] Updated weights for policy 0, policy_version 1729024 (0.0005) [2023-12-27 03:59:25,754][105620] Updated weights for policy 1, policy_version 1732564 (0.0006) [2023-12-27 03:59:25,809][105620] Updated weights for policy 1, policy_version 1732574 (0.0005) [2023-12-27 03:59:25,867][105620] Updated weights for policy 1, policy_version 1732584 (0.0007) [2023-12-27 03:59:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 886300672. Throughput: 0: 10027.0, 1: 9380.8. Samples: 886310564. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 03:59:26,062][104569] Avg episode reward: [(0, '8352.532'), (1, '8896.659')] [2023-12-27 03:59:26,116][105692] Updated weights for policy 0, policy_version 1729034 (0.0010) [2023-12-27 03:59:26,187][105692] Updated weights for policy 0, policy_version 1729044 (0.0010) [2023-12-27 03:59:26,254][105692] Updated weights for policy 0, policy_version 1729054 (0.0010) [2023-12-27 03:59:26,429][105620] Updated weights for policy 1, policy_version 1732594 (0.0009) [2023-12-27 03:59:26,488][105620] Updated weights for policy 1, policy_version 1732604 (0.0007) [2023-12-27 03:59:26,539][105620] Updated weights for policy 1, policy_version 1732614 (0.0009) [2023-12-27 03:59:26,594][105620] Updated weights for policy 1, policy_version 1732624 (0.0009) [2023-12-27 03:59:27,037][105692] Updated weights for policy 0, policy_version 1729064 (0.0009) [2023-12-27 03:59:27,084][105692] Updated weights for policy 0, policy_version 1729074 (0.0008) [2023-12-27 03:59:27,129][105692] Updated weights for policy 0, policy_version 1729084 (0.0008) [2023-12-27 03:59:27,301][105620] Updated weights for policy 1, policy_version 1732634 (0.0009) [2023-12-27 03:59:27,354][105620] Updated weights for policy 1, policy_version 1732644 (0.0010) [2023-12-27 03:59:27,412][105620] Updated weights for policy 1, policy_version 1732654 (0.0009) [2023-12-27 03:59:27,865][105692] Updated weights for policy 0, policy_version 1729094 (0.0007) [2023-12-27 03:59:27,924][105692] Updated weights for policy 0, policy_version 1729104 (0.0005) [2023-12-27 03:59:27,972][105692] Updated weights for policy 0, policy_version 1729114 (0.0005) [2023-12-27 03:59:28,128][105620] Updated weights for policy 1, policy_version 1732664 (0.0008) [2023-12-27 03:59:28,176][105620] Updated weights for policy 1, policy_version 1732674 (0.0005) [2023-12-27 03:59:28,227][105620] Updated weights for policy 1, policy_version 1732684 (0.0005) [2023-12-27 03:59:28,583][105692] Updated weights for policy 0, policy_version 1729124 (0.0007) [2023-12-27 03:59:28,629][105692] Updated weights for policy 0, policy_version 1729134 (0.0008) [2023-12-27 03:59:28,675][105692] Updated weights for policy 0, policy_version 1729144 (0.0009) [2023-12-27 03:59:28,964][105620] Updated weights for policy 1, policy_version 1732694 (0.0008) [2023-12-27 03:59:29,018][105620] Updated weights for policy 1, policy_version 1732704 (0.0009) [2023-12-27 03:59:29,067][105620] Updated weights for policy 1, policy_version 1732714 (0.0008) [2023-12-27 03:59:29,385][105692] Updated weights for policy 0, policy_version 1729154 (0.0008) [2023-12-27 03:59:29,447][105692] Updated weights for policy 0, policy_version 1729164 (0.0005) [2023-12-27 03:59:29,520][105692] Updated weights for policy 0, policy_version 1729174 (0.0007) [2023-12-27 03:59:29,590][105692] Updated weights for policy 0, policy_version 1729184 (0.0007) [2023-12-27 03:59:29,810][105620] Updated weights for policy 1, policy_version 1732724 (0.0009) [2023-12-27 03:59:29,870][105620] Updated weights for policy 1, policy_version 1732734 (0.0008) [2023-12-27 03:59:29,927][105620] Updated weights for policy 1, policy_version 1732744 (0.0008) [2023-12-27 03:59:30,157][105692] Updated weights for policy 0, policy_version 1729194 (0.0010) [2023-12-27 03:59:30,211][105692] Updated weights for policy 0, policy_version 1729204 (0.0008) [2023-12-27 03:59:30,274][105692] Updated weights for policy 0, policy_version 1729214 (0.0010) [2023-12-27 03:59:30,701][105620] Updated weights for policy 1, policy_version 1732754 (0.0008) [2023-12-27 03:59:30,757][105620] Updated weights for policy 1, policy_version 1732764 (0.0009) [2023-12-27 03:59:30,813][105620] Updated weights for policy 1, policy_version 1732774 (0.0009) [2023-12-27 03:59:30,878][105620] Updated weights for policy 1, policy_version 1732784 (0.0008) [2023-12-27 03:59:30,927][105692] Updated weights for policy 0, policy_version 1729224 (0.0006) [2023-12-27 03:59:30,982][105692] Updated weights for policy 0, policy_version 1729234 (0.0007) [2023-12-27 03:59:31,044][105692] Updated weights for policy 0, policy_version 1729244 (0.0011) [2023-12-27 03:59:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 886398976. Throughput: 0: 10037.5, 1: 9418.6. Samples: 886370864. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 03:59:31,063][104569] Avg episode reward: [(0, '8439.745'), (1, '9356.093')] [2023-12-27 03:59:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001729248_442753024.pth... [2023-12-27 03:59:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001732784_443654144.pth... [2023-12-27 03:59:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001728064_442449920.pth [2023-12-27 03:59:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001731696_443375616.pth [2023-12-27 03:59:31,609][105620] Updated weights for policy 1, policy_version 1732794 (0.0008) [2023-12-27 03:59:31,674][105620] Updated weights for policy 1, policy_version 1732804 (0.0008) [2023-12-27 03:59:31,739][105620] Updated weights for policy 1, policy_version 1732814 (0.0008) [2023-12-27 03:59:31,791][105692] Updated weights for policy 0, policy_version 1729254 (0.0011) [2023-12-27 03:59:31,842][105692] Updated weights for policy 0, policy_version 1729264 (0.0010) [2023-12-27 03:59:31,911][105692] Updated weights for policy 0, policy_version 1729274 (0.0011) [2023-12-27 03:59:32,548][105620] Updated weights for policy 1, policy_version 1732824 (0.0010) [2023-12-27 03:59:32,583][105692] Updated weights for policy 0, policy_version 1729284 (0.0009) [2023-12-27 03:59:32,611][105620] Updated weights for policy 1, policy_version 1732834 (0.0010) [2023-12-27 03:59:32,644][105692] Updated weights for policy 0, policy_version 1729294 (0.0007) [2023-12-27 03:59:32,670][105620] Updated weights for policy 1, policy_version 1732844 (0.0011) [2023-12-27 03:59:32,699][105692] Updated weights for policy 0, policy_version 1729304 (0.0005) [2023-12-27 03:59:33,315][105692] Updated weights for policy 0, policy_version 1729314 (0.0007) [2023-12-27 03:59:33,363][105692] Updated weights for policy 0, policy_version 1729324 (0.0008) [2023-12-27 03:59:33,401][105620] Updated weights for policy 1, policy_version 1732854 (0.0011) [2023-12-27 03:59:33,423][105692] Updated weights for policy 0, policy_version 1729334 (0.0008) [2023-12-27 03:59:33,449][105620] Updated weights for policy 1, policy_version 1732864 (0.0010) [2023-12-27 03:59:33,471][105692] Updated weights for policy 0, policy_version 1729344 (0.0005) [2023-12-27 03:59:33,503][105620] Updated weights for policy 1, policy_version 1732874 (0.0010) [2023-12-27 03:59:34,042][105692] Updated weights for policy 0, policy_version 1729354 (0.0005) [2023-12-27 03:59:34,083][105692] Updated weights for policy 0, policy_version 1729364 (0.0005) [2023-12-27 03:59:34,145][105692] Updated weights for policy 0, policy_version 1729374 (0.0006) [2023-12-27 03:59:34,231][105620] Updated weights for policy 1, policy_version 1732884 (0.0008) [2023-12-27 03:59:34,286][105620] Updated weights for policy 1, policy_version 1732894 (0.0005) [2023-12-27 03:59:34,352][105620] Updated weights for policy 1, policy_version 1732904 (0.0006) [2023-12-27 03:59:34,873][105692] Updated weights for policy 0, policy_version 1729384 (0.0008) [2023-12-27 03:59:34,931][105620] Updated weights for policy 1, policy_version 1732914 (0.0006) [2023-12-27 03:59:34,938][105692] Updated weights for policy 0, policy_version 1729394 (0.0005) [2023-12-27 03:59:34,989][105620] Updated weights for policy 1, policy_version 1732924 (0.0005) [2023-12-27 03:59:34,994][105692] Updated weights for policy 0, policy_version 1729404 (0.0008) [2023-12-27 03:59:35,040][105620] Updated weights for policy 1, policy_version 1732934 (0.0008) [2023-12-27 03:59:35,104][105620] Updated weights for policy 1, policy_version 1732944 (0.0007) [2023-12-27 03:59:35,691][105620] Updated weights for policy 1, policy_version 1732954 (0.0010) [2023-12-27 03:59:35,740][105620] Updated weights for policy 1, policy_version 1732964 (0.0010) [2023-12-27 03:59:35,795][105620] Updated weights for policy 1, policy_version 1732974 (0.0009) [2023-12-27 03:59:35,798][105692] Updated weights for policy 0, policy_version 1729414 (0.0008) [2023-12-27 03:59:35,857][105692] Updated weights for policy 0, policy_version 1729424 (0.0008) [2023-12-27 03:59:35,909][105692] Updated weights for policy 0, policy_version 1729434 (0.0008) [2023-12-27 03:59:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 886505472. Throughput: 0: 10042.9, 1: 9399.4. Samples: 886491416. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 03:59:36,062][104569] Avg episode reward: [(0, '8351.684'), (1, '9172.832')] [2023-12-27 03:59:36,524][105620] Updated weights for policy 1, policy_version 1732984 (0.0009) [2023-12-27 03:59:36,588][105620] Updated weights for policy 1, policy_version 1732994 (0.0009) [2023-12-27 03:59:36,651][105620] Updated weights for policy 1, policy_version 1733004 (0.0010) [2023-12-27 03:59:36,709][105692] Updated weights for policy 0, policy_version 1729444 (0.0009) [2023-12-27 03:59:36,772][105692] Updated weights for policy 0, policy_version 1729454 (0.0009) [2023-12-27 03:59:36,821][105692] Updated weights for policy 0, policy_version 1729464 (0.0009) [2023-12-27 03:59:37,328][105620] Updated weights for policy 1, policy_version 1733014 (0.0008) [2023-12-27 03:59:37,378][105620] Updated weights for policy 1, policy_version 1733024 (0.0008) [2023-12-27 03:59:37,443][105620] Updated weights for policy 1, policy_version 1733034 (0.0006) [2023-12-27 03:59:37,521][105692] Updated weights for policy 0, policy_version 1729474 (0.0009) [2023-12-27 03:59:37,571][105692] Updated weights for policy 0, policy_version 1729484 (0.0006) [2023-12-27 03:59:37,618][105692] Updated weights for policy 0, policy_version 1729494 (0.0005) [2023-12-27 03:59:37,665][105692] Updated weights for policy 0, policy_version 1729504 (0.0005) [2023-12-27 03:59:38,126][105620] Updated weights for policy 1, policy_version 1733044 (0.0006) [2023-12-27 03:59:38,187][105620] Updated weights for policy 1, policy_version 1733054 (0.0008) [2023-12-27 03:59:38,250][105620] Updated weights for policy 1, policy_version 1733064 (0.0009) [2023-12-27 03:59:38,346][105692] Updated weights for policy 0, policy_version 1729514 (0.0007) [2023-12-27 03:59:38,407][105692] Updated weights for policy 0, policy_version 1729524 (0.0008) [2023-12-27 03:59:38,463][105692] Updated weights for policy 0, policy_version 1729534 (0.0008) [2023-12-27 03:59:38,846][105620] Updated weights for policy 1, policy_version 1733074 (0.0010) [2023-12-27 03:59:38,904][105620] Updated weights for policy 1, policy_version 1733084 (0.0009) [2023-12-27 03:59:38,956][105620] Updated weights for policy 1, policy_version 1733094 (0.0010) [2023-12-27 03:59:39,005][105620] Updated weights for policy 1, policy_version 1733104 (0.0010) [2023-12-27 03:59:39,281][105692] Updated weights for policy 0, policy_version 1729544 (0.0009) [2023-12-27 03:59:39,340][105692] Updated weights for policy 0, policy_version 1729554 (0.0009) [2023-12-27 03:59:39,403][105692] Updated weights for policy 0, policy_version 1729564 (0.0009) [2023-12-27 03:59:39,691][105620] Updated weights for policy 1, policy_version 1733114 (0.0011) [2023-12-27 03:59:39,750][105620] Updated weights for policy 1, policy_version 1733124 (0.0010) [2023-12-27 03:59:39,814][105620] Updated weights for policy 1, policy_version 1733134 (0.0011) [2023-12-27 03:59:40,121][105692] Updated weights for policy 0, policy_version 1729574 (0.0010) [2023-12-27 03:59:40,181][105692] Updated weights for policy 0, policy_version 1729584 (0.0009) [2023-12-27 03:59:40,245][105692] Updated weights for policy 0, policy_version 1729594 (0.0009) [2023-12-27 03:59:40,586][105620] Updated weights for policy 1, policy_version 1733144 (0.0009) [2023-12-27 03:59:40,649][105620] Updated weights for policy 1, policy_version 1733154 (0.0009) [2023-12-27 03:59:40,711][105620] Updated weights for policy 1, policy_version 1733164 (0.0007) [2023-12-27 03:59:40,966][105692] Updated weights for policy 0, policy_version 1729604 (0.0009) [2023-12-27 03:59:41,021][105692] Updated weights for policy 0, policy_version 1729614 (0.0010) [2023-12-27 03:59:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 886595584. Throughput: 0: 9984.4, 1: 9482.0. Samples: 886608316. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 03:59:41,063][104569] Avg episode reward: [(0, '8534.077'), (1, '9172.892')] [2023-12-27 03:59:41,079][105692] Updated weights for policy 0, policy_version 1729624 (0.0008) [2023-12-27 03:59:41,418][105620] Updated weights for policy 1, policy_version 1733174 (0.0007) [2023-12-27 03:59:41,472][105620] Updated weights for policy 1, policy_version 1733184 (0.0009) [2023-12-27 03:59:41,529][105620] Updated weights for policy 1, policy_version 1733194 (0.0009) [2023-12-27 03:59:41,810][105692] Updated weights for policy 0, policy_version 1729634 (0.0009) [2023-12-27 03:59:41,864][105692] Updated weights for policy 0, policy_version 1729644 (0.0010) [2023-12-27 03:59:41,921][105692] Updated weights for policy 0, policy_version 1729654 (0.0006) [2023-12-27 03:59:41,987][105692] Updated weights for policy 0, policy_version 1729664 (0.0007) [2023-12-27 03:59:42,360][105620] Updated weights for policy 1, policy_version 1733204 (0.0009) [2023-12-27 03:59:42,425][105620] Updated weights for policy 1, policy_version 1733215 (0.0010) [2023-12-27 03:59:42,478][105620] Updated weights for policy 1, policy_version 1733225 (0.0010) [2023-12-27 03:59:42,665][105692] Updated weights for policy 0, policy_version 1729674 (0.0006) [2023-12-27 03:59:42,725][105692] Updated weights for policy 0, policy_version 1729684 (0.0005) [2023-12-27 03:59:42,793][105692] Updated weights for policy 0, policy_version 1729694 (0.0007) [2023-12-27 03:59:43,322][105620] Updated weights for policy 1, policy_version 1733235 (0.0009) [2023-12-27 03:59:43,375][105620] Updated weights for policy 1, policy_version 1733245 (0.0008) [2023-12-27 03:59:43,432][105620] Updated weights for policy 1, policy_version 1733255 (0.0007) [2023-12-27 03:59:43,434][105692] Updated weights for policy 0, policy_version 1729704 (0.0008) [2023-12-27 03:59:43,487][105692] Updated weights for policy 0, policy_version 1729714 (0.0005) [2023-12-27 03:59:43,536][105692] Updated weights for policy 0, policy_version 1729724 (0.0005) [2023-12-27 03:59:44,080][105692] Updated weights for policy 0, policy_version 1729734 (0.0007) [2023-12-27 03:59:44,127][105692] Updated weights for policy 0, policy_version 1729744 (0.0009) [2023-12-27 03:59:44,174][105692] Updated weights for policy 0, policy_version 1729754 (0.0009) [2023-12-27 03:59:44,275][105620] Updated weights for policy 1, policy_version 1733265 (0.0009) [2023-12-27 03:59:44,332][105620] Updated weights for policy 1, policy_version 1733275 (0.0009) [2023-12-27 03:59:44,393][105620] Updated weights for policy 1, policy_version 1733285 (0.0009) [2023-12-27 03:59:44,448][105620] Updated weights for policy 1, policy_version 1733295 (0.0009) [2023-12-27 03:59:44,959][105692] Updated weights for policy 0, policy_version 1729764 (0.0009) [2023-12-27 03:59:45,026][105692] Updated weights for policy 0, policy_version 1729774 (0.0009) [2023-12-27 03:59:45,093][105692] Updated weights for policy 0, policy_version 1729784 (0.0008) [2023-12-27 03:59:45,125][105620] Updated weights for policy 1, policy_version 1733305 (0.0007) [2023-12-27 03:59:45,179][105620] Updated weights for policy 1, policy_version 1733315 (0.0009) [2023-12-27 03:59:45,251][105620] Updated weights for policy 1, policy_version 1733325 (0.0010) [2023-12-27 03:59:45,647][105692] Updated weights for policy 0, policy_version 1729794 (0.0008) [2023-12-27 03:59:45,698][105692] Updated weights for policy 0, policy_version 1729804 (0.0008) [2023-12-27 03:59:45,749][105692] Updated weights for policy 0, policy_version 1729814 (0.0009) [2023-12-27 03:59:45,801][105692] Updated weights for policy 0, policy_version 1729824 (0.0009) [2023-12-27 03:59:46,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 886693888. Throughput: 0: 9949.1, 1: 9425.7. Samples: 886663812. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 03:59:46,063][104569] Avg episode reward: [(0, '8805.093'), (1, '9263.453')] [2023-12-27 03:59:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001729824_442900480.pth... [2023-12-27 03:59:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001728640_442597376.pth [2023-12-27 03:59:46,091][105620] Updated weights for policy 1, policy_version 1733335 (0.0009) [2023-12-27 03:59:46,141][105620] Updated weights for policy 1, policy_version 1733345 (0.0008) [2023-12-27 03:59:46,205][105620] Updated weights for policy 1, policy_version 1733355 (0.0011) [2023-12-27 03:59:46,229][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001733360_443801600.pth... [2023-12-27 03:59:46,232][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001732208_443506688.pth [2023-12-27 03:59:46,437][105692] Updated weights for policy 0, policy_version 1729834 (0.0005) [2023-12-27 03:59:46,484][105692] Updated weights for policy 0, policy_version 1729844 (0.0005) [2023-12-27 03:59:46,535][105692] Updated weights for policy 0, policy_version 1729854 (0.0005) [2023-12-27 03:59:47,065][105692] Updated weights for policy 0, policy_version 1729864 (0.0005) [2023-12-27 03:59:47,110][105620] Updated weights for policy 1, policy_version 1733365 (0.0010) [2023-12-27 03:59:47,113][105692] Updated weights for policy 0, policy_version 1729874 (0.0005) [2023-12-27 03:59:47,159][105620] Updated weights for policy 1, policy_version 1733375 (0.0009) [2023-12-27 03:59:47,161][105692] Updated weights for policy 0, policy_version 1729884 (0.0005) [2023-12-27 03:59:47,209][105620] Updated weights for policy 1, policy_version 1733385 (0.0009) [2023-12-27 03:59:47,708][105692] Updated weights for policy 0, policy_version 1729894 (0.0007) [2023-12-27 03:59:47,766][105692] Updated weights for policy 0, policy_version 1729904 (0.0010) [2023-12-27 03:59:47,826][105692] Updated weights for policy 0, policy_version 1729914 (0.0010) [2023-12-27 03:59:48,085][105620] Updated weights for policy 1, policy_version 1733396 (0.0010) [2023-12-27 03:59:48,139][105620] Updated weights for policy 1, policy_version 1733406 (0.0008) [2023-12-27 03:59:48,187][105620] Updated weights for policy 1, policy_version 1733416 (0.0008) [2023-12-27 03:59:48,552][105692] Updated weights for policy 0, policy_version 1729924 (0.0010) [2023-12-27 03:59:48,615][105692] Updated weights for policy 0, policy_version 1729934 (0.0011) [2023-12-27 03:59:48,678][105692] Updated weights for policy 0, policy_version 1729944 (0.0009) [2023-12-27 03:59:48,982][105620] Updated weights for policy 1, policy_version 1733426 (0.0008) [2023-12-27 03:59:49,042][105620] Updated weights for policy 1, policy_version 1733436 (0.0008) [2023-12-27 03:59:49,097][105620] Updated weights for policy 1, policy_version 1733446 (0.0008) [2023-12-27 03:59:49,142][105620] Updated weights for policy 1, policy_version 1733456 (0.0008) [2023-12-27 03:59:49,410][105692] Updated weights for policy 0, policy_version 1729954 (0.0008) [2023-12-27 03:59:49,469][105692] Updated weights for policy 0, policy_version 1729964 (0.0011) [2023-12-27 03:59:49,535][105692] Updated weights for policy 0, policy_version 1729974 (0.0011) [2023-12-27 03:59:49,585][105692] Updated weights for policy 0, policy_version 1729984 (0.0010) [2023-12-27 03:59:49,984][105620] Updated weights for policy 1, policy_version 1733466 (0.0009) [2023-12-27 03:59:50,043][105620] Updated weights for policy 1, policy_version 1733476 (0.0010) [2023-12-27 03:59:50,095][105620] Updated weights for policy 1, policy_version 1733486 (0.0009) [2023-12-27 03:59:50,273][105692] Updated weights for policy 0, policy_version 1729994 (0.0007) [2023-12-27 03:59:50,329][105692] Updated weights for policy 0, policy_version 1730004 (0.0011) [2023-12-27 03:59:50,385][105692] Updated weights for policy 0, policy_version 1730014 (0.0011) [2023-12-27 03:59:50,942][105620] Updated weights for policy 1, policy_version 1733496 (0.0007) [2023-12-27 03:59:51,006][105620] Updated weights for policy 1, policy_version 1733506 (0.0008) [2023-12-27 03:59:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 886784000. Throughput: 0: 10056.2, 1: 9373.8. Samples: 886781692. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 03:59:51,063][104569] Avg episode reward: [(0, '8714.659'), (1, '8987.072')] [2023-12-27 03:59:51,078][105620] Updated weights for policy 1, policy_version 1733516 (0.0009) [2023-12-27 03:59:51,107][105692] Updated weights for policy 0, policy_version 1730024 (0.0009) [2023-12-27 03:59:51,168][105692] Updated weights for policy 0, policy_version 1730034 (0.0009) [2023-12-27 03:59:51,231][105692] Updated weights for policy 0, policy_version 1730044 (0.0009) [2023-12-27 03:59:51,816][105620] Updated weights for policy 1, policy_version 1733526 (0.0009) [2023-12-27 03:59:51,875][105620] Updated weights for policy 1, policy_version 1733536 (0.0010) [2023-12-27 03:59:51,934][105620] Updated weights for policy 1, policy_version 1733546 (0.0011) [2023-12-27 03:59:52,002][105692] Updated weights for policy 0, policy_version 1730054 (0.0009) [2023-12-27 03:59:52,054][105692] Updated weights for policy 0, policy_version 1730064 (0.0009) [2023-12-27 03:59:52,117][105692] Updated weights for policy 0, policy_version 1730074 (0.0009) [2023-12-27 03:59:52,667][105620] Updated weights for policy 1, policy_version 1733556 (0.0011) [2023-12-27 03:59:52,727][105620] Updated weights for policy 1, policy_version 1733566 (0.0011) [2023-12-27 03:59:52,777][105620] Updated weights for policy 1, policy_version 1733576 (0.0009) [2023-12-27 03:59:52,909][105692] Updated weights for policy 0, policy_version 1730084 (0.0008) [2023-12-27 03:59:52,956][105692] Updated weights for policy 0, policy_version 1730094 (0.0006) [2023-12-27 03:59:53,007][105692] Updated weights for policy 0, policy_version 1730104 (0.0007) [2023-12-27 03:59:53,396][105620] Updated weights for policy 1, policy_version 1733586 (0.0008) [2023-12-27 03:59:53,445][105620] Updated weights for policy 1, policy_version 1733596 (0.0005) [2023-12-27 03:59:53,493][105620] Updated weights for policy 1, policy_version 1733606 (0.0009) [2023-12-27 03:59:53,554][105620] Updated weights for policy 1, policy_version 1733616 (0.0006) [2023-12-27 03:59:53,698][105692] Updated weights for policy 0, policy_version 1730114 (0.0008) [2023-12-27 03:59:53,744][105692] Updated weights for policy 0, policy_version 1730124 (0.0005) [2023-12-27 03:59:53,792][105692] Updated weights for policy 0, policy_version 1730134 (0.0005) [2023-12-27 03:59:53,847][105692] Updated weights for policy 0, policy_version 1730144 (0.0007) [2023-12-27 03:59:54,241][105620] Updated weights for policy 1, policy_version 1733626 (0.0010) [2023-12-27 03:59:54,290][105620] Updated weights for policy 1, policy_version 1733636 (0.0010) [2023-12-27 03:59:54,344][105620] Updated weights for policy 1, policy_version 1733646 (0.0010) [2023-12-27 03:59:54,581][105692] Updated weights for policy 0, policy_version 1730154 (0.0008) [2023-12-27 03:59:54,645][105692] Updated weights for policy 0, policy_version 1730164 (0.0008) [2023-12-27 03:59:54,706][105692] Updated weights for policy 0, policy_version 1730174 (0.0009) [2023-12-27 03:59:55,149][105620] Updated weights for policy 1, policy_version 1733656 (0.0011) [2023-12-27 03:59:55,207][105620] Updated weights for policy 1, policy_version 1733666 (0.0010) [2023-12-27 03:59:55,269][105620] Updated weights for policy 1, policy_version 1733676 (0.0009) [2023-12-27 03:59:55,447][105692] Updated weights for policy 0, policy_version 1730184 (0.0009) [2023-12-27 03:59:55,526][105692] Updated weights for policy 0, policy_version 1730194 (0.0010) [2023-12-27 03:59:55,585][105692] Updated weights for policy 0, policy_version 1730204 (0.0011) [2023-12-27 03:59:55,933][105620] Updated weights for policy 1, policy_version 1733686 (0.0008) [2023-12-27 03:59:55,993][105620] Updated weights for policy 1, policy_version 1733696 (0.0008) [2023-12-27 03:59:56,052][105620] Updated weights for policy 1, policy_version 1733706 (0.0009) [2023-12-27 03:59:56,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 886882304. Throughput: 0: 10026.1, 1: 9435.5. Samples: 886896404. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 03:59:56,062][104569] Avg episode reward: [(0, '8714.013'), (1, '8807.963')] [2023-12-27 03:59:56,377][105692] Updated weights for policy 0, policy_version 1730214 (0.0009) [2023-12-27 03:59:56,440][105692] Updated weights for policy 0, policy_version 1730224 (0.0009) [2023-12-27 03:59:56,495][105692] Updated weights for policy 0, policy_version 1730234 (0.0011) [2023-12-27 03:59:56,642][105620] Updated weights for policy 1, policy_version 1733716 (0.0007) [2023-12-27 03:59:56,688][105620] Updated weights for policy 1, policy_version 1733726 (0.0005) [2023-12-27 03:59:56,736][105620] Updated weights for policy 1, policy_version 1733736 (0.0005) [2023-12-27 03:59:57,298][105620] Updated weights for policy 1, policy_version 1733746 (0.0006) [2023-12-27 03:59:57,340][105692] Updated weights for policy 0, policy_version 1730244 (0.0010) [2023-12-27 03:59:57,346][105620] Updated weights for policy 1, policy_version 1733756 (0.0009) [2023-12-27 03:59:57,392][105692] Updated weights for policy 0, policy_version 1730254 (0.0010) [2023-12-27 03:59:57,396][105620] Updated weights for policy 1, policy_version 1733766 (0.0005) [2023-12-27 03:59:57,433][105692] Updated weights for policy 0, policy_version 1730264 (0.0009) [2023-12-27 03:59:57,451][105620] Updated weights for policy 1, policy_version 1733776 (0.0009) [2023-12-27 03:59:58,012][105692] Updated weights for policy 0, policy_version 1730274 (0.0006) [2023-12-27 03:59:58,074][105692] Updated weights for policy 0, policy_version 1730284 (0.0010) [2023-12-27 03:59:58,132][105692] Updated weights for policy 0, policy_version 1730294 (0.0010) [2023-12-27 03:59:58,159][105620] Updated weights for policy 1, policy_version 1733786 (0.0010) [2023-12-27 03:59:58,193][105692] Updated weights for policy 0, policy_version 1730304 (0.0009) [2023-12-27 03:59:58,217][105620] Updated weights for policy 1, policy_version 1733796 (0.0008) [2023-12-27 03:59:58,273][105620] Updated weights for policy 1, policy_version 1733806 (0.0008) [2023-12-27 03:59:59,027][105620] Updated weights for policy 1, policy_version 1733816 (0.0009) [2023-12-27 03:59:59,037][105692] Updated weights for policy 0, policy_version 1730314 (0.0010) [2023-12-27 03:59:59,087][105692] Updated weights for policy 0, policy_version 1730324 (0.0011) [2023-12-27 03:59:59,087][105620] Updated weights for policy 1, policy_version 1733826 (0.0008) [2023-12-27 03:59:59,138][105620] Updated weights for policy 1, policy_version 1733836 (0.0007) [2023-12-27 03:59:59,142][105692] Updated weights for policy 0, policy_version 1730334 (0.0010) [2023-12-27 03:59:59,840][105692] Updated weights for policy 0, policy_version 1730344 (0.0009) [2023-12-27 03:59:59,894][105620] Updated weights for policy 1, policy_version 1733846 (0.0009) [2023-12-27 03:59:59,895][105692] Updated weights for policy 0, policy_version 1730354 (0.0010) [2023-12-27 03:59:59,955][105692] Updated weights for policy 0, policy_version 1730364 (0.0009) [2023-12-27 03:59:59,957][105620] Updated weights for policy 1, policy_version 1733856 (0.0011) [2023-12-27 04:00:00,017][105620] Updated weights for policy 1, policy_version 1733866 (0.0011) [2023-12-27 04:00:00,597][105620] Updated weights for policy 1, policy_version 1733876 (0.0008) [2023-12-27 04:00:00,658][105620] Updated weights for policy 1, policy_version 1733886 (0.0005) [2023-12-27 04:00:00,679][105692] Updated weights for policy 0, policy_version 1730374 (0.0007) [2023-12-27 04:00:00,716][105620] Updated weights for policy 1, policy_version 1733896 (0.0005) [2023-12-27 04:00:00,736][105692] Updated weights for policy 0, policy_version 1730384 (0.0009) [2023-12-27 04:00:00,793][105692] Updated weights for policy 0, policy_version 1730395 (0.0010) [2023-12-27 04:00:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 886988800. Throughput: 0: 9994.9, 1: 9517.3. Samples: 886956320. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 04:00:01,063][104569] Avg episode reward: [(0, '8804.820'), (1, '8991.178')] [2023-12-27 04:00:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001730400_443047936.pth... [2023-12-27 04:00:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001733904_443940864.pth... [2023-12-27 04:00:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001729248_442753024.pth [2023-12-27 04:00:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001732784_443654144.pth [2023-12-27 04:00:01,362][105620] Updated weights for policy 1, policy_version 1733906 (0.0006) [2023-12-27 04:00:01,427][105620] Updated weights for policy 1, policy_version 1733916 (0.0007) [2023-12-27 04:00:01,490][105620] Updated weights for policy 1, policy_version 1733926 (0.0006) [2023-12-27 04:00:01,508][105692] Updated weights for policy 0, policy_version 1730405 (0.0010) [2023-12-27 04:00:01,545][105620] Updated weights for policy 1, policy_version 1733936 (0.0006) [2023-12-27 04:00:01,565][105692] Updated weights for policy 0, policy_version 1730415 (0.0009) [2023-12-27 04:00:01,624][105692] Updated weights for policy 0, policy_version 1730425 (0.0007) [2023-12-27 04:00:02,185][105620] Updated weights for policy 1, policy_version 1733946 (0.0009) [2023-12-27 04:00:02,231][105620] Updated weights for policy 1, policy_version 1733956 (0.0009) [2023-12-27 04:00:02,293][105620] Updated weights for policy 1, policy_version 1733966 (0.0007) [2023-12-27 04:00:02,317][105692] Updated weights for policy 0, policy_version 1730435 (0.0007) [2023-12-27 04:00:02,382][105692] Updated weights for policy 0, policy_version 1730445 (0.0009) [2023-12-27 04:00:02,440][105692] Updated weights for policy 0, policy_version 1730455 (0.0009) [2023-12-27 04:00:02,992][105620] Updated weights for policy 1, policy_version 1733976 (0.0010) [2023-12-27 04:00:03,040][105620] Updated weights for policy 1, policy_version 1733986 (0.0010) [2023-12-27 04:00:03,101][105620] Updated weights for policy 1, policy_version 1733996 (0.0006) [2023-12-27 04:00:03,226][105692] Updated weights for policy 0, policy_version 1730465 (0.0010) [2023-12-27 04:00:03,273][105692] Updated weights for policy 0, policy_version 1730475 (0.0008) [2023-12-27 04:00:03,325][105692] Updated weights for policy 0, policy_version 1730485 (0.0008) [2023-12-27 04:00:03,378][105692] Updated weights for policy 0, policy_version 1730495 (0.0008) [2023-12-27 04:00:03,747][105620] Updated weights for policy 1, policy_version 1734006 (0.0005) [2023-12-27 04:00:03,801][105620] Updated weights for policy 1, policy_version 1734016 (0.0005) [2023-12-27 04:00:03,861][105620] Updated weights for policy 1, policy_version 1734026 (0.0007) [2023-12-27 04:00:04,080][105692] Updated weights for policy 0, policy_version 1730505 (0.0009) [2023-12-27 04:00:04,137][105692] Updated weights for policy 0, policy_version 1730515 (0.0006) [2023-12-27 04:00:04,196][105692] Updated weights for policy 0, policy_version 1730525 (0.0006) [2023-12-27 04:00:04,546][105620] Updated weights for policy 1, policy_version 1734036 (0.0008) [2023-12-27 04:00:04,614][105620] Updated weights for policy 1, policy_version 1734046 (0.0005) [2023-12-27 04:00:04,680][105620] Updated weights for policy 1, policy_version 1734056 (0.0005) [2023-12-27 04:00:04,990][105692] Updated weights for policy 0, policy_version 1730535 (0.0007) [2023-12-27 04:00:05,042][105692] Updated weights for policy 0, policy_version 1730545 (0.0009) [2023-12-27 04:00:05,090][105692] Updated weights for policy 0, policy_version 1730555 (0.0009) [2023-12-27 04:00:05,198][105620] Updated weights for policy 1, policy_version 1734066 (0.0006) [2023-12-27 04:00:05,252][105620] Updated weights for policy 1, policy_version 1734076 (0.0008) [2023-12-27 04:00:05,317][105620] Updated weights for policy 1, policy_version 1734086 (0.0009) [2023-12-27 04:00:05,380][105620] Updated weights for policy 1, policy_version 1734096 (0.0008) [2023-12-27 04:00:05,889][105692] Updated weights for policy 0, policy_version 1730565 (0.0008) [2023-12-27 04:00:05,950][105692] Updated weights for policy 0, policy_version 1730575 (0.0009) [2023-12-27 04:00:05,998][105692] Updated weights for policy 0, policy_version 1730585 (0.0009) [2023-12-27 04:00:06,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 887087104. Throughput: 0: 9886.3, 1: 9702.8. Samples: 887076656. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 04:00:06,062][104569] Avg episode reward: [(0, '8895.824'), (1, '9171.586')] [2023-12-27 04:00:06,112][105620] Updated weights for policy 1, policy_version 1734106 (0.0009) [2023-12-27 04:00:06,175][105620] Updated weights for policy 1, policy_version 1734116 (0.0009) [2023-12-27 04:00:06,226][105620] Updated weights for policy 1, policy_version 1734126 (0.0009) [2023-12-27 04:00:06,806][105692] Updated weights for policy 0, policy_version 1730595 (0.0007) [2023-12-27 04:00:06,868][105692] Updated weights for policy 0, policy_version 1730605 (0.0009) [2023-12-27 04:00:06,931][105692] Updated weights for policy 0, policy_version 1730615 (0.0007) [2023-12-27 04:00:06,933][105620] Updated weights for policy 1, policy_version 1734136 (0.0008) [2023-12-27 04:00:06,988][105620] Updated weights for policy 1, policy_version 1734146 (0.0007) [2023-12-27 04:00:07,042][105620] Updated weights for policy 1, policy_version 1734156 (0.0009) [2023-12-27 04:00:07,711][105620] Updated weights for policy 1, policy_version 1734166 (0.0007) [2023-12-27 04:00:07,743][105692] Updated weights for policy 0, policy_version 1730625 (0.0007) [2023-12-27 04:00:07,765][105620] Updated weights for policy 1, policy_version 1734176 (0.0008) [2023-12-27 04:00:07,799][105692] Updated weights for policy 0, policy_version 1730635 (0.0008) [2023-12-27 04:00:07,825][105620] Updated weights for policy 1, policy_version 1734186 (0.0007) [2023-12-27 04:00:07,847][105692] Updated weights for policy 0, policy_version 1730645 (0.0006) [2023-12-27 04:00:07,906][105692] Updated weights for policy 0, policy_version 1730655 (0.0008) [2023-12-27 04:00:08,432][105620] Updated weights for policy 1, policy_version 1734196 (0.0008) [2023-12-27 04:00:08,494][105620] Updated weights for policy 1, policy_version 1734206 (0.0009) [2023-12-27 04:00:08,558][105620] Updated weights for policy 1, policy_version 1734216 (0.0009) [2023-12-27 04:00:08,722][105692] Updated weights for policy 0, policy_version 1730665 (0.0009) [2023-12-27 04:00:08,780][105692] Updated weights for policy 0, policy_version 1730675 (0.0006) [2023-12-27 04:00:08,835][105692] Updated weights for policy 0, policy_version 1730685 (0.0006) [2023-12-27 04:00:09,310][105620] Updated weights for policy 1, policy_version 1734226 (0.0009) [2023-12-27 04:00:09,378][105620] Updated weights for policy 1, policy_version 1734236 (0.0009) [2023-12-27 04:00:09,447][105620] Updated weights for policy 1, policy_version 1734246 (0.0009) [2023-12-27 04:00:09,495][105692] Updated weights for policy 0, policy_version 1730695 (0.0006) [2023-12-27 04:00:09,510][105620] Updated weights for policy 1, policy_version 1734256 (0.0008) [2023-12-27 04:00:09,560][105692] Updated weights for policy 0, policy_version 1730705 (0.0007) [2023-12-27 04:00:09,626][105692] Updated weights for policy 0, policy_version 1730715 (0.0009) [2023-12-27 04:00:10,250][105692] Updated weights for policy 0, policy_version 1730725 (0.0008) [2023-12-27 04:00:10,309][105692] Updated weights for policy 0, policy_version 1730735 (0.0006) [2023-12-27 04:00:10,370][105620] Updated weights for policy 1, policy_version 1734266 (0.0009) [2023-12-27 04:00:10,374][105692] Updated weights for policy 0, policy_version 1730745 (0.0005) [2023-12-27 04:00:10,434][105620] Updated weights for policy 1, policy_version 1734276 (0.0009) [2023-12-27 04:00:10,496][105620] Updated weights for policy 1, policy_version 1734286 (0.0009) [2023-12-27 04:00:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 887177216. Throughput: 0: 9822.8, 1: 9721.9. Samples: 887190076. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 04:00:11,062][104569] Avg episode reward: [(0, '8716.279'), (1, '9173.425')] [2023-12-27 04:00:11,101][105692] Updated weights for policy 0, policy_version 1730755 (0.0007) [2023-12-27 04:00:11,177][105692] Updated weights for policy 0, policy_version 1730765 (0.0010) [2023-12-27 04:00:11,244][105692] Updated weights for policy 0, policy_version 1730775 (0.0009) [2023-12-27 04:00:11,296][105620] Updated weights for policy 1, policy_version 1734296 (0.0008) [2023-12-27 04:00:11,358][105620] Updated weights for policy 1, policy_version 1734306 (0.0009) [2023-12-27 04:00:11,433][105620] Updated weights for policy 1, policy_version 1734316 (0.0008) [2023-12-27 04:00:12,076][105692] Updated weights for policy 0, policy_version 1730785 (0.0010) [2023-12-27 04:00:12,080][105620] Updated weights for policy 1, policy_version 1734326 (0.0007) [2023-12-27 04:00:12,138][105692] Updated weights for policy 0, policy_version 1730795 (0.0007) [2023-12-27 04:00:12,139][105620] Updated weights for policy 1, policy_version 1734336 (0.0006) [2023-12-27 04:00:12,205][105620] Updated weights for policy 1, policy_version 1734346 (0.0007) [2023-12-27 04:00:12,208][105692] Updated weights for policy 0, policy_version 1730805 (0.0008) [2023-12-27 04:00:12,272][105692] Updated weights for policy 0, policy_version 1730815 (0.0008) [2023-12-27 04:00:12,922][105620] Updated weights for policy 1, policy_version 1734356 (0.0008) [2023-12-27 04:00:12,980][105620] Updated weights for policy 1, policy_version 1734366 (0.0008) [2023-12-27 04:00:12,983][105692] Updated weights for policy 0, policy_version 1730825 (0.0007) [2023-12-27 04:00:13,037][105692] Updated weights for policy 0, policy_version 1730835 (0.0008) [2023-12-27 04:00:13,039][105620] Updated weights for policy 1, policy_version 1734376 (0.0007) [2023-12-27 04:00:13,090][105692] Updated weights for policy 0, policy_version 1730845 (0.0008) [2023-12-27 04:00:13,783][105620] Updated weights for policy 1, policy_version 1734386 (0.0009) [2023-12-27 04:00:13,829][105620] Updated weights for policy 1, policy_version 1734396 (0.0008) [2023-12-27 04:00:13,869][105692] Updated weights for policy 0, policy_version 1730855 (0.0008) [2023-12-27 04:00:13,878][105620] Updated weights for policy 1, policy_version 1734406 (0.0006) [2023-12-27 04:00:13,916][105692] Updated weights for policy 0, policy_version 1730865 (0.0006) [2023-12-27 04:00:13,941][105620] Updated weights for policy 1, policy_version 1734416 (0.0007) [2023-12-27 04:00:13,966][105692] Updated weights for policy 0, policy_version 1730875 (0.0009) [2023-12-27 04:00:14,622][105620] Updated weights for policy 1, policy_version 1734426 (0.0005) [2023-12-27 04:00:14,677][105620] Updated weights for policy 1, policy_version 1734436 (0.0005) [2023-12-27 04:00:14,742][105620] Updated weights for policy 1, policy_version 1734446 (0.0005) [2023-12-27 04:00:14,791][105692] Updated weights for policy 0, policy_version 1730885 (0.0009) [2023-12-27 04:00:14,847][105692] Updated weights for policy 0, policy_version 1730895 (0.0009) [2023-12-27 04:00:14,895][105692] Updated weights for policy 0, policy_version 1730905 (0.0009) [2023-12-27 04:00:15,438][105620] Updated weights for policy 1, policy_version 1734456 (0.0009) [2023-12-27 04:00:15,494][105620] Updated weights for policy 1, policy_version 1734466 (0.0009) [2023-12-27 04:00:15,551][105620] Updated weights for policy 1, policy_version 1734476 (0.0009) [2023-12-27 04:00:15,569][105692] Updated weights for policy 0, policy_version 1730915 (0.0008) [2023-12-27 04:00:15,623][105692] Updated weights for policy 0, policy_version 1730925 (0.0005) [2023-12-27 04:00:15,670][105692] Updated weights for policy 0, policy_version 1730935 (0.0006) [2023-12-27 04:00:16,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 887275520. Throughput: 0: 9769.3, 1: 9668.7. Samples: 887245576. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 04:00:16,063][104569] Avg episode reward: [(0, '8902.372'), (1, '9173.396')] [2023-12-27 04:00:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001734480_444088320.pth... [2023-12-27 04:00:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001730944_443187200.pth... [2023-12-27 04:00:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001733360_443801600.pth [2023-12-27 04:00:16,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001729824_442900480.pth [2023-12-27 04:00:16,197][105692] Updated weights for policy 0, policy_version 1730945 (0.0005) [2023-12-27 04:00:16,265][105692] Updated weights for policy 0, policy_version 1730955 (0.0006) [2023-12-27 04:00:16,322][105692] Updated weights for policy 0, policy_version 1730965 (0.0005) [2023-12-27 04:00:16,379][105692] Updated weights for policy 0, policy_version 1730975 (0.0006) [2023-12-27 04:00:16,443][105620] Updated weights for policy 1, policy_version 1734486 (0.0009) [2023-12-27 04:00:16,496][105620] Updated weights for policy 1, policy_version 1734497 (0.0010) [2023-12-27 04:00:16,550][105620] Updated weights for policy 1, policy_version 1734508 (0.0010) [2023-12-27 04:00:16,922][105692] Updated weights for policy 0, policy_version 1730985 (0.0010) [2023-12-27 04:00:16,970][105692] Updated weights for policy 0, policy_version 1730995 (0.0010) [2023-12-27 04:00:17,021][105692] Updated weights for policy 0, policy_version 1731005 (0.0010) [2023-12-27 04:00:17,375][105620] Updated weights for policy 1, policy_version 1734518 (0.0009) [2023-12-27 04:00:17,440][105620] Updated weights for policy 1, policy_version 1734528 (0.0009) [2023-12-27 04:00:17,497][105620] Updated weights for policy 1, policy_version 1734538 (0.0009) [2023-12-27 04:00:17,739][105692] Updated weights for policy 0, policy_version 1731015 (0.0009) [2023-12-27 04:00:17,793][105692] Updated weights for policy 0, policy_version 1731025 (0.0009) [2023-12-27 04:00:17,840][105692] Updated weights for policy 0, policy_version 1731035 (0.0009) [2023-12-27 04:00:18,245][105620] Updated weights for policy 1, policy_version 1734548 (0.0009) [2023-12-27 04:00:18,291][105620] Updated weights for policy 1, policy_version 1734558 (0.0008) [2023-12-27 04:00:18,349][105620] Updated weights for policy 1, policy_version 1734568 (0.0009) [2023-12-27 04:00:18,567][105692] Updated weights for policy 0, policy_version 1731045 (0.0007) [2023-12-27 04:00:18,619][105692] Updated weights for policy 0, policy_version 1731055 (0.0005) [2023-12-27 04:00:18,672][105692] Updated weights for policy 0, policy_version 1731065 (0.0005) [2023-12-27 04:00:19,239][105620] Updated weights for policy 1, policy_version 1734578 (0.0009) [2023-12-27 04:00:19,246][105692] Updated weights for policy 0, policy_version 1731075 (0.0007) [2023-12-27 04:00:19,304][105620] Updated weights for policy 1, policy_version 1734588 (0.0008) [2023-12-27 04:00:19,306][105692] Updated weights for policy 0, policy_version 1731085 (0.0006) [2023-12-27 04:00:19,368][105620] Updated weights for policy 1, policy_version 1734598 (0.0009) [2023-12-27 04:00:19,375][105692] Updated weights for policy 0, policy_version 1731095 (0.0006) [2023-12-27 04:00:19,422][105620] Updated weights for policy 1, policy_version 1734608 (0.0009) [2023-12-27 04:00:20,070][105692] Updated weights for policy 0, policy_version 1731105 (0.0007) [2023-12-27 04:00:20,136][105692] Updated weights for policy 0, policy_version 1731115 (0.0009) [2023-12-27 04:00:20,198][105692] Updated weights for policy 0, policy_version 1731125 (0.0009) [2023-12-27 04:00:20,233][105620] Updated weights for policy 1, policy_version 1734618 (0.0006) [2023-12-27 04:00:20,279][105692] Updated weights for policy 0, policy_version 1731135 (0.0008) [2023-12-27 04:00:20,295][105620] Updated weights for policy 1, policy_version 1734628 (0.0007) [2023-12-27 04:00:20,349][105620] Updated weights for policy 1, policy_version 1734638 (0.0009) [2023-12-27 04:00:21,037][105692] Updated weights for policy 0, policy_version 1731145 (0.0009) [2023-12-27 04:00:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 887365632. Throughput: 0: 9768.6, 1: 9583.7. Samples: 887362272. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 04:00:21,063][104569] Avg episode reward: [(0, '8807.517'), (1, '9170.806')] [2023-12-27 04:00:21,109][105692] Updated weights for policy 0, policy_version 1731155 (0.0009) [2023-12-27 04:00:21,168][105692] Updated weights for policy 0, policy_version 1731165 (0.0008) [2023-12-27 04:00:21,175][105620] Updated weights for policy 1, policy_version 1734648 (0.0008) [2023-12-27 04:00:21,241][105620] Updated weights for policy 1, policy_version 1734658 (0.0010) [2023-12-27 04:00:21,309][105620] Updated weights for policy 1, policy_version 1734668 (0.0008) [2023-12-27 04:00:21,974][105692] Updated weights for policy 0, policy_version 1731175 (0.0007) [2023-12-27 04:00:22,035][105692] Updated weights for policy 0, policy_version 1731185 (0.0009) [2023-12-27 04:00:22,093][105692] Updated weights for policy 0, policy_version 1731195 (0.0008) [2023-12-27 04:00:22,112][105620] Updated weights for policy 1, policy_version 1734678 (0.0008) [2023-12-27 04:00:22,161][105620] Updated weights for policy 1, policy_version 1734688 (0.0009) [2023-12-27 04:00:22,208][105620] Updated weights for policy 1, policy_version 1734698 (0.0008) [2023-12-27 04:00:22,891][105692] Updated weights for policy 0, policy_version 1731205 (0.0009) [2023-12-27 04:00:22,941][105620] Updated weights for policy 1, policy_version 1734708 (0.0007) [2023-12-27 04:00:22,959][105692] Updated weights for policy 0, policy_version 1731215 (0.0011) [2023-12-27 04:00:23,009][105620] Updated weights for policy 1, policy_version 1734718 (0.0005) [2023-12-27 04:00:23,013][105692] Updated weights for policy 0, policy_version 1731225 (0.0009) [2023-12-27 04:00:23,074][105620] Updated weights for policy 1, policy_version 1734728 (0.0006) [2023-12-27 04:00:23,688][105692] Updated weights for policy 0, policy_version 1731235 (0.0008) [2023-12-27 04:00:23,691][105620] Updated weights for policy 1, policy_version 1734738 (0.0005) [2023-12-27 04:00:23,736][105620] Updated weights for policy 1, policy_version 1734748 (0.0009) [2023-12-27 04:00:23,743][105692] Updated weights for policy 0, policy_version 1731245 (0.0009) [2023-12-27 04:00:23,788][105620] Updated weights for policy 1, policy_version 1734758 (0.0009) [2023-12-27 04:00:23,795][105692] Updated weights for policy 0, policy_version 1731255 (0.0008) [2023-12-27 04:00:23,837][105620] Updated weights for policy 1, policy_version 1734768 (0.0009) [2023-12-27 04:00:24,453][105692] Updated weights for policy 0, policy_version 1731265 (0.0006) [2023-12-27 04:00:24,502][105692] Updated weights for policy 0, policy_version 1731275 (0.0007) [2023-12-27 04:00:24,552][105692] Updated weights for policy 0, policy_version 1731285 (0.0008) [2023-12-27 04:00:24,567][105620] Updated weights for policy 1, policy_version 1734778 (0.0006) [2023-12-27 04:00:24,600][105692] Updated weights for policy 0, policy_version 1731295 (0.0009) [2023-12-27 04:00:24,624][105620] Updated weights for policy 1, policy_version 1734788 (0.0006) [2023-12-27 04:00:24,684][105620] Updated weights for policy 1, policy_version 1734798 (0.0005) [2023-12-27 04:00:25,347][105620] Updated weights for policy 1, policy_version 1734808 (0.0010) [2023-12-27 04:00:25,353][105692] Updated weights for policy 0, policy_version 1731305 (0.0006) [2023-12-27 04:00:25,403][105620] Updated weights for policy 1, policy_version 1734818 (0.0011) [2023-12-27 04:00:25,409][105692] Updated weights for policy 0, policy_version 1731315 (0.0006) [2023-12-27 04:00:25,456][105620] Updated weights for policy 1, policy_version 1734828 (0.0011) [2023-12-27 04:00:25,466][105692] Updated weights for policy 0, policy_version 1731325 (0.0006) [2023-12-27 04:00:26,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 887463936. Throughput: 0: 9778.0, 1: 9506.9. Samples: 887476140. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 04:00:26,062][104569] Avg episode reward: [(0, '8893.588'), (1, '8987.208')] [2023-12-27 04:00:26,202][105692] Updated weights for policy 0, policy_version 1731335 (0.0005) [2023-12-27 04:00:26,213][105620] Updated weights for policy 1, policy_version 1734838 (0.0010) [2023-12-27 04:00:26,255][105692] Updated weights for policy 0, policy_version 1731345 (0.0005) [2023-12-27 04:00:26,265][105620] Updated weights for policy 1, policy_version 1734848 (0.0010) [2023-12-27 04:00:26,311][105620] Updated weights for policy 1, policy_version 1734858 (0.0010) [2023-12-27 04:00:26,316][105692] Updated weights for policy 0, policy_version 1731355 (0.0006) [2023-12-27 04:00:27,000][105692] Updated weights for policy 0, policy_version 1731365 (0.0007) [2023-12-27 04:00:27,057][105692] Updated weights for policy 0, policy_version 1731375 (0.0008) [2023-12-27 04:00:27,081][105620] Updated weights for policy 1, policy_version 1734868 (0.0011) [2023-12-27 04:00:27,112][105692] Updated weights for policy 0, policy_version 1731385 (0.0009) [2023-12-27 04:00:27,139][105620] Updated weights for policy 1, policy_version 1734878 (0.0010) [2023-12-27 04:00:27,190][105620] Updated weights for policy 1, policy_version 1734888 (0.0010) [2023-12-27 04:00:27,746][105692] Updated weights for policy 0, policy_version 1731395 (0.0005) [2023-12-27 04:00:27,803][105692] Updated weights for policy 0, policy_version 1731405 (0.0005) [2023-12-27 04:00:27,862][105692] Updated weights for policy 0, policy_version 1731415 (0.0005) [2023-12-27 04:00:27,934][105620] Updated weights for policy 1, policy_version 1734898 (0.0010) [2023-12-27 04:00:27,999][105620] Updated weights for policy 1, policy_version 1734908 (0.0010) [2023-12-27 04:00:28,052][105620] Updated weights for policy 1, policy_version 1734918 (0.0011) [2023-12-27 04:00:28,108][105620] Updated weights for policy 1, policy_version 1734928 (0.0010) [2023-12-27 04:00:28,454][105692] Updated weights for policy 0, policy_version 1731425 (0.0005) [2023-12-27 04:00:28,516][105692] Updated weights for policy 0, policy_version 1731435 (0.0006) [2023-12-27 04:00:28,577][105692] Updated weights for policy 0, policy_version 1731445 (0.0007) [2023-12-27 04:00:28,625][105692] Updated weights for policy 0, policy_version 1731455 (0.0005) [2023-12-27 04:00:28,800][105620] Updated weights for policy 1, policy_version 1734938 (0.0011) [2023-12-27 04:00:28,855][105620] Updated weights for policy 1, policy_version 1734948 (0.0010) [2023-12-27 04:00:28,915][105620] Updated weights for policy 1, policy_version 1734958 (0.0007) [2023-12-27 04:00:29,319][105692] Updated weights for policy 0, policy_version 1731465 (0.0008) [2023-12-27 04:00:29,383][105692] Updated weights for policy 0, policy_version 1731475 (0.0008) [2023-12-27 04:00:29,442][105692] Updated weights for policy 0, policy_version 1731485 (0.0008) [2023-12-27 04:00:29,547][105620] Updated weights for policy 1, policy_version 1734968 (0.0009) [2023-12-27 04:00:29,599][105620] Updated weights for policy 1, policy_version 1734978 (0.0010) [2023-12-27 04:00:29,648][105620] Updated weights for policy 1, policy_version 1734988 (0.0010) [2023-12-27 04:00:30,151][105692] Updated weights for policy 0, policy_version 1731495 (0.0007) [2023-12-27 04:00:30,200][105692] Updated weights for policy 0, policy_version 1731505 (0.0008) [2023-12-27 04:00:30,252][105692] Updated weights for policy 0, policy_version 1731515 (0.0008) [2023-12-27 04:00:30,388][105620] Updated weights for policy 1, policy_version 1734998 (0.0010) [2023-12-27 04:00:30,436][105620] Updated weights for policy 1, policy_version 1735008 (0.0010) [2023-12-27 04:00:30,483][105620] Updated weights for policy 1, policy_version 1735018 (0.0010) [2023-12-27 04:00:30,884][105692] Updated weights for policy 0, policy_version 1731525 (0.0007) [2023-12-27 04:00:30,929][105692] Updated weights for policy 0, policy_version 1731535 (0.0008) [2023-12-27 04:00:30,975][105692] Updated weights for policy 0, policy_version 1731545 (0.0007) [2023-12-27 04:00:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 887570432. Throughput: 0: 9816.5, 1: 9571.1. Samples: 887536252. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 04:00:31,063][104569] Avg episode reward: [(0, '8985.713'), (1, '8987.855')] [2023-12-27 04:00:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001731552_443342848.pth... [2023-12-27 04:00:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001735024_444227584.pth... [2023-12-27 04:00:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001733904_443940864.pth [2023-12-27 04:00:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001730400_443047936.pth [2023-12-27 04:00:31,241][105620] Updated weights for policy 1, policy_version 1735028 (0.0010) [2023-12-27 04:00:31,303][105620] Updated weights for policy 1, policy_version 1735038 (0.0011) [2023-12-27 04:00:31,374][105620] Updated weights for policy 1, policy_version 1735048 (0.0012) [2023-12-27 04:00:31,784][105692] Updated weights for policy 0, policy_version 1731555 (0.0008) [2023-12-27 04:00:31,832][105692] Updated weights for policy 0, policy_version 1731565 (0.0008) [2023-12-27 04:00:31,890][105692] Updated weights for policy 0, policy_version 1731575 (0.0009) [2023-12-27 04:00:32,044][105620] Updated weights for policy 1, policy_version 1735058 (0.0009) [2023-12-27 04:00:32,103][105620] Updated weights for policy 1, policy_version 1735068 (0.0009) [2023-12-27 04:00:32,169][105620] Updated weights for policy 1, policy_version 1735078 (0.0011) [2023-12-27 04:00:32,225][105620] Updated weights for policy 1, policy_version 1735088 (0.0008) [2023-12-27 04:00:32,771][105620] Updated weights for policy 1, policy_version 1735098 (0.0008) [2023-12-27 04:00:32,783][105692] Updated weights for policy 0, policy_version 1731585 (0.0009) [2023-12-27 04:00:32,820][105620] Updated weights for policy 1, policy_version 1735108 (0.0010) [2023-12-27 04:00:32,841][105692] Updated weights for policy 0, policy_version 1731595 (0.0007) [2023-12-27 04:00:32,875][105620] Updated weights for policy 1, policy_version 1735118 (0.0010) [2023-12-27 04:00:32,891][105692] Updated weights for policy 0, policy_version 1731605 (0.0006) [2023-12-27 04:00:32,940][105692] Updated weights for policy 0, policy_version 1731615 (0.0005) [2023-12-27 04:00:33,631][105620] Updated weights for policy 1, policy_version 1735128 (0.0010) [2023-12-27 04:00:33,656][105692] Updated weights for policy 0, policy_version 1731625 (0.0006) [2023-12-27 04:00:33,682][105620] Updated weights for policy 1, policy_version 1735138 (0.0010) [2023-12-27 04:00:33,708][105692] Updated weights for policy 0, policy_version 1731635 (0.0005) [2023-12-27 04:00:33,733][105620] Updated weights for policy 1, policy_version 1735148 (0.0010) [2023-12-27 04:00:33,762][105692] Updated weights for policy 0, policy_version 1731645 (0.0005) [2023-12-27 04:00:34,504][105620] Updated weights for policy 1, policy_version 1735158 (0.0010) [2023-12-27 04:00:34,518][105692] Updated weights for policy 0, policy_version 1731655 (0.0006) [2023-12-27 04:00:34,563][105620] Updated weights for policy 1, policy_version 1735168 (0.0009) [2023-12-27 04:00:34,576][105692] Updated weights for policy 0, policy_version 1731665 (0.0007) [2023-12-27 04:00:34,623][105620] Updated weights for policy 1, policy_version 1735178 (0.0011) [2023-12-27 04:00:34,633][105692] Updated weights for policy 0, policy_version 1731675 (0.0005) [2023-12-27 04:00:35,369][105620] Updated weights for policy 1, policy_version 1735188 (0.0011) [2023-12-27 04:00:35,390][105692] Updated weights for policy 0, policy_version 1731685 (0.0007) [2023-12-27 04:00:35,433][105620] Updated weights for policy 1, policy_version 1735198 (0.0008) [2023-12-27 04:00:35,448][105692] Updated weights for policy 0, policy_version 1731695 (0.0008) [2023-12-27 04:00:35,494][105692] Updated weights for policy 0, policy_version 1731705 (0.0008) [2023-12-27 04:00:35,498][105620] Updated weights for policy 1, policy_version 1735208 (0.0008) [2023-12-27 04:00:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 887660544. Throughput: 0: 9629.3, 1: 9727.1. Samples: 887652728. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 04:00:36,062][104569] Avg episode reward: [(0, '8718.108'), (1, '9171.590')] [2023-12-27 04:00:36,149][105620] Updated weights for policy 1, policy_version 1735218 (0.0008) [2023-12-27 04:00:36,209][105620] Updated weights for policy 1, policy_version 1735228 (0.0010) [2023-12-27 04:00:36,236][105692] Updated weights for policy 0, policy_version 1731715 (0.0006) [2023-12-27 04:00:36,263][105620] Updated weights for policy 1, policy_version 1735238 (0.0006) [2023-12-27 04:00:36,297][105692] Updated weights for policy 0, policy_version 1731725 (0.0007) [2023-12-27 04:00:36,316][105620] Updated weights for policy 1, policy_version 1735248 (0.0006) [2023-12-27 04:00:36,360][105692] Updated weights for policy 0, policy_version 1731735 (0.0008) [2023-12-27 04:00:37,045][105620] Updated weights for policy 1, policy_version 1735258 (0.0006) [2023-12-27 04:00:37,094][105620] Updated weights for policy 1, policy_version 1735268 (0.0006) [2023-12-27 04:00:37,111][105692] Updated weights for policy 0, policy_version 1731745 (0.0009) [2023-12-27 04:00:37,141][105620] Updated weights for policy 1, policy_version 1735278 (0.0007) [2023-12-27 04:00:37,178][105692] Updated weights for policy 0, policy_version 1731755 (0.0008) [2023-12-27 04:00:37,247][105692] Updated weights for policy 0, policy_version 1731765 (0.0010) [2023-12-27 04:00:37,308][105692] Updated weights for policy 0, policy_version 1731775 (0.0009) [2023-12-27 04:00:37,835][105620] Updated weights for policy 1, policy_version 1735288 (0.0005) [2023-12-27 04:00:37,897][105620] Updated weights for policy 1, policy_version 1735298 (0.0005) [2023-12-27 04:00:37,943][105620] Updated weights for policy 1, policy_version 1735308 (0.0005) [2023-12-27 04:00:38,051][105692] Updated weights for policy 0, policy_version 1731785 (0.0009) [2023-12-27 04:00:38,117][105692] Updated weights for policy 0, policy_version 1731795 (0.0010) [2023-12-27 04:00:38,174][105692] Updated weights for policy 0, policy_version 1731805 (0.0010) [2023-12-27 04:00:38,505][105620] Updated weights for policy 1, policy_version 1735318 (0.0007) [2023-12-27 04:00:38,558][105620] Updated weights for policy 1, policy_version 1735328 (0.0009) [2023-12-27 04:00:38,614][105620] Updated weights for policy 1, policy_version 1735338 (0.0007) [2023-12-27 04:00:39,018][105692] Updated weights for policy 0, policy_version 1731815 (0.0009) [2023-12-27 04:00:39,072][105692] Updated weights for policy 0, policy_version 1731825 (0.0009) [2023-12-27 04:00:39,122][105692] Updated weights for policy 0, policy_version 1731835 (0.0007) [2023-12-27 04:00:39,344][105620] Updated weights for policy 1, policy_version 1735348 (0.0008) [2023-12-27 04:00:39,415][105620] Updated weights for policy 1, policy_version 1735358 (0.0008) [2023-12-27 04:00:39,473][105620] Updated weights for policy 1, policy_version 1735368 (0.0005) [2023-12-27 04:00:39,922][105692] Updated weights for policy 0, policy_version 1731845 (0.0008) [2023-12-27 04:00:39,986][105692] Updated weights for policy 0, policy_version 1731855 (0.0009) [2023-12-27 04:00:40,055][105692] Updated weights for policy 0, policy_version 1731865 (0.0008) [2023-12-27 04:00:40,084][105620] Updated weights for policy 1, policy_version 1735378 (0.0006) [2023-12-27 04:00:40,146][105620] Updated weights for policy 1, policy_version 1735388 (0.0006) [2023-12-27 04:00:40,148][105586] KL-divergence is very high: 120.0227 [2023-12-27 04:00:40,178][105586] KL-divergence is very high: 156.8463 [2023-12-27 04:00:40,202][105586] KL-divergence is very high: 225.7688 [2023-12-27 04:00:40,214][105620] Updated weights for policy 1, policy_version 1735398 (0.0006) [2023-12-27 04:00:40,227][105586] KL-divergence is very high: 212.9610 [2023-12-27 04:00:40,250][105586] KL-divergence is very high: 260.1977 [2023-12-27 04:00:40,274][105620] Updated weights for policy 1, policy_version 1735408 (0.0006) [2023-12-27 04:00:40,656][105692] Updated weights for policy 0, policy_version 1731875 (0.0007) [2023-12-27 04:00:40,717][105692] Updated weights for policy 0, policy_version 1731885 (0.0005) [2023-12-27 04:00:40,774][105692] Updated weights for policy 0, policy_version 1731895 (0.0005) [2023-12-27 04:00:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 887758848. Throughput: 0: 9599.9, 1: 9795.6. Samples: 887769204. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 04:00:41,062][104569] Avg episode reward: [(0, '8534.100'), (1, '8987.014')] [2023-12-27 04:00:41,079][105620] Updated weights for policy 1, policy_version 1735418 (0.0010) [2023-12-27 04:00:41,141][105620] Updated weights for policy 1, policy_version 1735428 (0.0009) [2023-12-27 04:00:41,214][105620] Updated weights for policy 1, policy_version 1735438 (0.0006) [2023-12-27 04:00:41,487][105692] Updated weights for policy 0, policy_version 1731905 (0.0006) [2023-12-27 04:00:41,541][105692] Updated weights for policy 0, policy_version 1731915 (0.0007) [2023-12-27 04:00:41,592][105692] Updated weights for policy 0, policy_version 1731925 (0.0009) [2023-12-27 04:00:41,655][105692] Updated weights for policy 0, policy_version 1731935 (0.0008) [2023-12-27 04:00:41,921][105620] Updated weights for policy 1, policy_version 1735448 (0.0006) [2023-12-27 04:00:41,983][105620] Updated weights for policy 1, policy_version 1735458 (0.0008) [2023-12-27 04:00:42,047][105620] Updated weights for policy 1, policy_version 1735468 (0.0008) [2023-12-27 04:00:42,401][105692] Updated weights for policy 0, policy_version 1731945 (0.0009) [2023-12-27 04:00:42,465][105692] Updated weights for policy 0, policy_version 1731955 (0.0009) [2023-12-27 04:00:42,527][105692] Updated weights for policy 0, policy_version 1731965 (0.0009) [2023-12-27 04:00:42,795][105620] Updated weights for policy 1, policy_version 1735478 (0.0009) [2023-12-27 04:00:42,842][105620] Updated weights for policy 1, policy_version 1735488 (0.0009) [2023-12-27 04:00:42,900][105620] Updated weights for policy 1, policy_version 1735498 (0.0009) [2023-12-27 04:00:43,303][105692] Updated weights for policy 0, policy_version 1731975 (0.0010) [2023-12-27 04:00:43,362][105692] Updated weights for policy 0, policy_version 1731985 (0.0010) [2023-12-27 04:00:43,420][105692] Updated weights for policy 0, policy_version 1731995 (0.0009) [2023-12-27 04:00:43,580][105620] Updated weights for policy 1, policy_version 1735508 (0.0007) [2023-12-27 04:00:43,627][105620] Updated weights for policy 1, policy_version 1735518 (0.0005) [2023-12-27 04:00:43,673][105620] Updated weights for policy 1, policy_version 1735528 (0.0005) [2023-12-27 04:00:44,237][105620] Updated weights for policy 1, policy_version 1735538 (0.0006) [2023-12-27 04:00:44,298][105620] Updated weights for policy 1, policy_version 1735548 (0.0008) [2023-12-27 04:00:44,311][105692] Updated weights for policy 0, policy_version 1732005 (0.0009) [2023-12-27 04:00:44,354][105620] Updated weights for policy 1, policy_version 1735558 (0.0008) [2023-12-27 04:00:44,374][105692] Updated weights for policy 0, policy_version 1732015 (0.0007) [2023-12-27 04:00:44,407][105620] Updated weights for policy 1, policy_version 1735568 (0.0008) [2023-12-27 04:00:44,435][105692] Updated weights for policy 0, policy_version 1732025 (0.0008) [2023-12-27 04:00:45,136][105620] Updated weights for policy 1, policy_version 1735578 (0.0009) [2023-12-27 04:00:45,191][105692] Updated weights for policy 0, policy_version 1732035 (0.0008) [2023-12-27 04:00:45,199][105620] Updated weights for policy 1, policy_version 1735588 (0.0009) [2023-12-27 04:00:45,249][105692] Updated weights for policy 0, policy_version 1732045 (0.0005) [2023-12-27 04:00:45,265][105620] Updated weights for policy 1, policy_version 1735598 (0.0009) [2023-12-27 04:00:45,304][105692] Updated weights for policy 0, policy_version 1732055 (0.0008) [2023-12-27 04:00:46,006][105620] Updated weights for policy 1, policy_version 1735608 (0.0009) [2023-12-27 04:00:46,050][105692] Updated weights for policy 0, policy_version 1732065 (0.0009) [2023-12-27 04:00:46,052][105620] Updated weights for policy 1, policy_version 1735618 (0.0008) [2023-12-27 04:00:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.3, 300 sec: 19383.1). Total num frames: 887848960. Throughput: 0: 9582.8, 1: 9758.4. Samples: 887826672. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 04:00:46,062][104569] Avg episode reward: [(0, '8807.423'), (1, '8802.191')] [2023-12-27 04:00:46,102][105620] Updated weights for policy 1, policy_version 1735628 (0.0008) [2023-12-27 04:00:46,108][105692] Updated weights for policy 0, policy_version 1732075 (0.0009) [2023-12-27 04:00:46,120][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001735632_444383232.pth... [2023-12-27 04:00:46,124][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001734480_444088320.pth [2023-12-27 04:00:46,169][105692] Updated weights for policy 0, policy_version 1732085 (0.0008) [2023-12-27 04:00:46,224][105692] Updated weights for policy 0, policy_version 1732095 (0.0009) [2023-12-27 04:00:46,227][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001732096_443482112.pth... [2023-12-27 04:00:46,230][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001730944_443187200.pth [2023-12-27 04:00:46,861][105620] Updated weights for policy 1, policy_version 1735638 (0.0008) [2023-12-27 04:00:46,920][105620] Updated weights for policy 1, policy_version 1735648 (0.0009) [2023-12-27 04:00:46,970][105692] Updated weights for policy 0, policy_version 1732105 (0.0007) [2023-12-27 04:00:46,980][105620] Updated weights for policy 1, policy_version 1735658 (0.0007) [2023-12-27 04:00:47,026][105692] Updated weights for policy 0, policy_version 1732115 (0.0008) [2023-12-27 04:00:47,084][105692] Updated weights for policy 0, policy_version 1732125 (0.0009) [2023-12-27 04:00:47,742][105692] Updated weights for policy 0, policy_version 1732135 (0.0007) [2023-12-27 04:00:47,793][105620] Updated weights for policy 1, policy_version 1735668 (0.0008) [2023-12-27 04:00:47,799][105692] Updated weights for policy 0, policy_version 1732145 (0.0006) [2023-12-27 04:00:47,852][105620] Updated weights for policy 1, policy_version 1735678 (0.0008) [2023-12-27 04:00:47,854][105692] Updated weights for policy 0, policy_version 1732155 (0.0007) [2023-12-27 04:00:47,914][105620] Updated weights for policy 1, policy_version 1735688 (0.0009) [2023-12-27 04:00:48,479][105692] Updated weights for policy 0, policy_version 1732165 (0.0008) [2023-12-27 04:00:48,529][105692] Updated weights for policy 0, policy_version 1732175 (0.0009) [2023-12-27 04:00:48,591][105692] Updated weights for policy 0, policy_version 1732185 (0.0009) [2023-12-27 04:00:48,721][105620] Updated weights for policy 1, policy_version 1735698 (0.0009) [2023-12-27 04:00:48,784][105620] Updated weights for policy 1, policy_version 1735708 (0.0009) [2023-12-27 04:00:48,846][105620] Updated weights for policy 1, policy_version 1735718 (0.0008) [2023-12-27 04:00:48,913][105620] Updated weights for policy 1, policy_version 1735728 (0.0006) [2023-12-27 04:00:49,387][105692] Updated weights for policy 0, policy_version 1732195 (0.0009) [2023-12-27 04:00:49,455][105692] Updated weights for policy 0, policy_version 1732205 (0.0006) [2023-12-27 04:00:49,512][105692] Updated weights for policy 0, policy_version 1732215 (0.0005) [2023-12-27 04:00:49,547][105620] Updated weights for policy 1, policy_version 1735738 (0.0009) [2023-12-27 04:00:49,603][105620] Updated weights for policy 1, policy_version 1735748 (0.0009) [2023-12-27 04:00:49,664][105620] Updated weights for policy 1, policy_version 1735758 (0.0009) [2023-12-27 04:00:50,120][105692] Updated weights for policy 0, policy_version 1732225 (0.0005) [2023-12-27 04:00:50,180][105692] Updated weights for policy 0, policy_version 1732235 (0.0006) [2023-12-27 04:00:50,240][105692] Updated weights for policy 0, policy_version 1732245 (0.0006) [2023-12-27 04:00:50,300][105692] Updated weights for policy 0, policy_version 1732255 (0.0007) [2023-12-27 04:00:50,516][105620] Updated weights for policy 1, policy_version 1735768 (0.0009) [2023-12-27 04:00:50,563][105620] Updated weights for policy 1, policy_version 1735778 (0.0008) [2023-12-27 04:00:50,628][105620] Updated weights for policy 1, policy_version 1735788 (0.0007) [2023-12-27 04:00:50,954][105692] Updated weights for policy 0, policy_version 1732265 (0.0007) [2023-12-27 04:00:51,018][105692] Updated weights for policy 0, policy_version 1732275 (0.0007) [2023-12-27 04:00:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 887947264. Throughput: 0: 9579.8, 1: 9613.6. Samples: 887940360. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 04:00:51,062][104569] Avg episode reward: [(0, '8716.988'), (1, '9171.862')] [2023-12-27 04:00:51,080][105692] Updated weights for policy 0, policy_version 1732285 (0.0009) [2023-12-27 04:00:51,326][105620] Updated weights for policy 1, policy_version 1735798 (0.0006) [2023-12-27 04:00:51,397][105620] Updated weights for policy 1, policy_version 1735808 (0.0009) [2023-12-27 04:00:51,456][105620] Updated weights for policy 1, policy_version 1735818 (0.0009) [2023-12-27 04:00:51,866][105692] Updated weights for policy 0, policy_version 1732295 (0.0009) [2023-12-27 04:00:51,920][105692] Updated weights for policy 0, policy_version 1732305 (0.0010) [2023-12-27 04:00:51,972][105692] Updated weights for policy 0, policy_version 1732316 (0.0009) [2023-12-27 04:00:52,156][105620] Updated weights for policy 1, policy_version 1735828 (0.0009) [2023-12-27 04:00:52,223][105620] Updated weights for policy 1, policy_version 1735838 (0.0009) [2023-12-27 04:00:52,289][105620] Updated weights for policy 1, policy_version 1735848 (0.0009) [2023-12-27 04:00:52,771][105692] Updated weights for policy 0, policy_version 1732326 (0.0009) [2023-12-27 04:00:52,834][105692] Updated weights for policy 0, policy_version 1732336 (0.0009) [2023-12-27 04:00:52,900][105692] Updated weights for policy 0, policy_version 1732346 (0.0009) [2023-12-27 04:00:53,056][105620] Updated weights for policy 1, policy_version 1735858 (0.0009) [2023-12-27 04:00:53,118][105620] Updated weights for policy 1, policy_version 1735868 (0.0009) [2023-12-27 04:00:53,164][105620] Updated weights for policy 1, policy_version 1735878 (0.0007) [2023-12-27 04:00:53,211][105620] Updated weights for policy 1, policy_version 1735888 (0.0006) [2023-12-27 04:00:53,628][105692] Updated weights for policy 0, policy_version 1732356 (0.0007) [2023-12-27 04:00:53,692][105692] Updated weights for policy 0, policy_version 1732366 (0.0005) [2023-12-27 04:00:53,749][105692] Updated weights for policy 0, policy_version 1732376 (0.0005) [2023-12-27 04:00:54,054][105620] Updated weights for policy 1, policy_version 1735898 (0.0008) [2023-12-27 04:00:54,117][105620] Updated weights for policy 1, policy_version 1735908 (0.0009) [2023-12-27 04:00:54,173][105620] Updated weights for policy 1, policy_version 1735918 (0.0007) [2023-12-27 04:00:54,332][105692] Updated weights for policy 0, policy_version 1732386 (0.0007) [2023-12-27 04:00:54,381][105692] Updated weights for policy 0, policy_version 1732396 (0.0010) [2023-12-27 04:00:54,433][105692] Updated weights for policy 0, policy_version 1732406 (0.0010) [2023-12-27 04:00:54,495][105692] Updated weights for policy 0, policy_version 1732416 (0.0010) [2023-12-27 04:00:54,825][105620] Updated weights for policy 1, policy_version 1735928 (0.0007) [2023-12-27 04:00:54,885][105620] Updated weights for policy 1, policy_version 1735938 (0.0008) [2023-12-27 04:00:54,938][105620] Updated weights for policy 1, policy_version 1735948 (0.0007) [2023-12-27 04:00:55,227][105692] Updated weights for policy 0, policy_version 1732426 (0.0005) [2023-12-27 04:00:55,296][105692] Updated weights for policy 0, policy_version 1732436 (0.0010) [2023-12-27 04:00:55,365][105692] Updated weights for policy 0, policy_version 1732446 (0.0011) [2023-12-27 04:00:55,720][105620] Updated weights for policy 1, policy_version 1735958 (0.0006) [2023-12-27 04:00:55,787][105620] Updated weights for policy 1, policy_version 1735968 (0.0006) [2023-12-27 04:00:55,842][105620] Updated weights for policy 1, policy_version 1735978 (0.0008) [2023-12-27 04:00:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 888045568. Throughput: 0: 9654.1, 1: 9578.7. Samples: 888055552. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 04:00:56,062][104569] Avg episode reward: [(0, '8528.128'), (1, '8986.414')] [2023-12-27 04:00:56,068][105692] Updated weights for policy 0, policy_version 1732456 (0.0009) [2023-12-27 04:00:56,128][105692] Updated weights for policy 0, policy_version 1732466 (0.0010) [2023-12-27 04:00:56,185][105692] Updated weights for policy 0, policy_version 1732476 (0.0009) [2023-12-27 04:00:56,577][105620] Updated weights for policy 1, policy_version 1735989 (0.0008) [2023-12-27 04:00:56,624][105620] Updated weights for policy 1, policy_version 1735999 (0.0008) [2023-12-27 04:00:56,677][105620] Updated weights for policy 1, policy_version 1736009 (0.0008) [2023-12-27 04:00:56,799][105692] Updated weights for policy 0, policy_version 1732486 (0.0008) [2023-12-27 04:00:56,853][105692] Updated weights for policy 0, policy_version 1732496 (0.0009) [2023-12-27 04:00:56,913][105692] Updated weights for policy 0, policy_version 1732506 (0.0008) [2023-12-27 04:00:57,453][105620] Updated weights for policy 1, policy_version 1736019 (0.0009) [2023-12-27 04:00:57,511][105620] Updated weights for policy 1, policy_version 1736029 (0.0008) [2023-12-27 04:00:57,537][105692] Updated weights for policy 0, policy_version 1732516 (0.0008) [2023-12-27 04:00:57,570][105620] Updated weights for policy 1, policy_version 1736039 (0.0009) [2023-12-27 04:00:57,588][105692] Updated weights for policy 0, policy_version 1732526 (0.0005) [2023-12-27 04:00:57,644][105692] Updated weights for policy 0, policy_version 1732536 (0.0005) [2023-12-27 04:00:58,293][105692] Updated weights for policy 0, policy_version 1732546 (0.0006) [2023-12-27 04:00:58,357][105692] Updated weights for policy 0, policy_version 1732556 (0.0008) [2023-12-27 04:00:58,375][105620] Updated weights for policy 1, policy_version 1736049 (0.0009) [2023-12-27 04:00:58,421][105692] Updated weights for policy 0, policy_version 1732566 (0.0007) [2023-12-27 04:00:58,440][105620] Updated weights for policy 1, policy_version 1736059 (0.0009) [2023-12-27 04:00:58,485][105692] Updated weights for policy 0, policy_version 1732576 (0.0007) [2023-12-27 04:00:58,504][105620] Updated weights for policy 1, policy_version 1736069 (0.0008) [2023-12-27 04:00:58,568][105620] Updated weights for policy 1, policy_version 1736079 (0.0009) [2023-12-27 04:00:59,307][105692] Updated weights for policy 0, policy_version 1732586 (0.0010) [2023-12-27 04:00:59,393][105692] Updated weights for policy 0, policy_version 1732596 (0.0009) [2023-12-27 04:00:59,454][105692] Updated weights for policy 0, policy_version 1732606 (0.0008) [2023-12-27 04:00:59,464][105620] Updated weights for policy 1, policy_version 1736089 (0.0007) [2023-12-27 04:00:59,524][105620] Updated weights for policy 1, policy_version 1736099 (0.0009) [2023-12-27 04:00:59,580][105620] Updated weights for policy 1, policy_version 1736109 (0.0009) [2023-12-27 04:01:00,195][105692] Updated weights for policy 0, policy_version 1732616 (0.0006) [2023-12-27 04:01:00,255][105692] Updated weights for policy 0, policy_version 1732626 (0.0008) [2023-12-27 04:01:00,318][105692] Updated weights for policy 0, policy_version 1732636 (0.0008) [2023-12-27 04:01:00,348][105620] Updated weights for policy 1, policy_version 1736119 (0.0010) [2023-12-27 04:01:00,408][105620] Updated weights for policy 1, policy_version 1736129 (0.0008) [2023-12-27 04:01:00,468][105620] Updated weights for policy 1, policy_version 1736139 (0.0006) [2023-12-27 04:01:01,061][105692] Updated weights for policy 0, policy_version 1732646 (0.0009) [2023-12-27 04:01:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.7, 300 sec: 19383.1). Total num frames: 888135680. Throughput: 0: 9744.9, 1: 9529.8. Samples: 888112936. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 04:01:01,062][104569] Avg episode reward: [(0, '8355.405'), (1, '8986.275')] [2023-12-27 04:01:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001736144_444514304.pth... [2023-12-27 04:01:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001735024_444227584.pth [2023-12-27 04:01:01,125][105692] Updated weights for policy 0, policy_version 1732656 (0.0010) [2023-12-27 04:01:01,145][105620] Updated weights for policy 1, policy_version 1736149 (0.0007) [2023-12-27 04:01:01,183][105692] Updated weights for policy 0, policy_version 1732666 (0.0008) [2023-12-27 04:01:01,201][105620] Updated weights for policy 1, policy_version 1736159 (0.0006) [2023-12-27 04:01:01,211][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001732672_443629568.pth... [2023-12-27 04:01:01,214][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001731552_443342848.pth [2023-12-27 04:01:01,260][105620] Updated weights for policy 1, policy_version 1736169 (0.0005) [2023-12-27 04:01:01,999][105620] Updated weights for policy 1, policy_version 1736179 (0.0006) [2023-12-27 04:01:02,027][105692] Updated weights for policy 0, policy_version 1732676 (0.0007) [2023-12-27 04:01:02,065][105620] Updated weights for policy 1, policy_version 1736189 (0.0008) [2023-12-27 04:01:02,084][105692] Updated weights for policy 0, policy_version 1732686 (0.0006) [2023-12-27 04:01:02,126][105620] Updated weights for policy 1, policy_version 1736199 (0.0008) [2023-12-27 04:01:02,137][105692] Updated weights for policy 0, policy_version 1732696 (0.0006) [2023-12-27 04:01:02,758][105620] Updated weights for policy 1, policy_version 1736209 (0.0008) [2023-12-27 04:01:02,813][105620] Updated weights for policy 1, policy_version 1736219 (0.0005) [2023-12-27 04:01:02,863][105620] Updated weights for policy 1, policy_version 1736229 (0.0005) [2023-12-27 04:01:02,922][105620] Updated weights for policy 1, policy_version 1736239 (0.0008) [2023-12-27 04:01:02,975][105692] Updated weights for policy 0, policy_version 1732706 (0.0008) [2023-12-27 04:01:03,036][105692] Updated weights for policy 0, policy_version 1732716 (0.0009) [2023-12-27 04:01:03,095][105692] Updated weights for policy 0, policy_version 1732726 (0.0009) [2023-12-27 04:01:03,146][105692] Updated weights for policy 0, policy_version 1732736 (0.0009) [2023-12-27 04:01:03,571][105620] Updated weights for policy 1, policy_version 1736249 (0.0009) [2023-12-27 04:01:03,625][105620] Updated weights for policy 1, policy_version 1736259 (0.0009) [2023-12-27 04:01:03,683][105620] Updated weights for policy 1, policy_version 1736269 (0.0009) [2023-12-27 04:01:03,950][105692] Updated weights for policy 0, policy_version 1732746 (0.0009) [2023-12-27 04:01:04,014][105692] Updated weights for policy 0, policy_version 1732756 (0.0009) [2023-12-27 04:01:04,070][105692] Updated weights for policy 0, policy_version 1732766 (0.0009) [2023-12-27 04:01:04,455][105620] Updated weights for policy 1, policy_version 1736279 (0.0009) [2023-12-27 04:01:04,522][105620] Updated weights for policy 1, policy_version 1736289 (0.0008) [2023-12-27 04:01:04,581][105620] Updated weights for policy 1, policy_version 1736299 (0.0009) [2023-12-27 04:01:04,813][105692] Updated weights for policy 0, policy_version 1732776 (0.0009) [2023-12-27 04:01:04,872][105692] Updated weights for policy 0, policy_version 1732786 (0.0007) [2023-12-27 04:01:04,928][105692] Updated weights for policy 0, policy_version 1732796 (0.0005) [2023-12-27 04:01:05,373][105620] Updated weights for policy 1, policy_version 1736309 (0.0007) [2023-12-27 04:01:05,433][105620] Updated weights for policy 1, policy_version 1736319 (0.0005) [2023-12-27 04:01:05,499][105620] Updated weights for policy 1, policy_version 1736329 (0.0005) [2023-12-27 04:01:05,572][105692] Updated weights for policy 0, policy_version 1732806 (0.0005) [2023-12-27 04:01:05,638][105692] Updated weights for policy 0, policy_version 1732816 (0.0006) [2023-12-27 04:01:05,702][105692] Updated weights for policy 0, policy_version 1732826 (0.0009) [2023-12-27 04:01:06,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19114.6, 300 sec: 19410.9). Total num frames: 888233984. Throughput: 0: 9511.2, 1: 9624.0. Samples: 888223364. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 04:01:06,063][104569] Avg episode reward: [(0, '8538.990'), (1, '9262.909')] [2023-12-27 04:01:06,179][105620] Updated weights for policy 1, policy_version 1736339 (0.0007) [2023-12-27 04:01:06,234][105620] Updated weights for policy 1, policy_version 1736349 (0.0008) [2023-12-27 04:01:06,298][105620] Updated weights for policy 1, policy_version 1736359 (0.0011) [2023-12-27 04:01:06,392][105692] Updated weights for policy 0, policy_version 1732836 (0.0011) [2023-12-27 04:01:06,455][105692] Updated weights for policy 0, policy_version 1732846 (0.0009) [2023-12-27 04:01:06,523][105692] Updated weights for policy 0, policy_version 1732856 (0.0007) [2023-12-27 04:01:07,022][105620] Updated weights for policy 1, policy_version 1736369 (0.0009) [2023-12-27 04:01:07,077][105620] Updated weights for policy 1, policy_version 1736379 (0.0010) [2023-12-27 04:01:07,133][105620] Updated weights for policy 1, policy_version 1736389 (0.0010) [2023-12-27 04:01:07,185][105620] Updated weights for policy 1, policy_version 1736399 (0.0010) [2023-12-27 04:01:07,242][105692] Updated weights for policy 0, policy_version 1732866 (0.0006) [2023-12-27 04:01:07,295][105692] Updated weights for policy 0, policy_version 1732876 (0.0008) [2023-12-27 04:01:07,343][105692] Updated weights for policy 0, policy_version 1732886 (0.0008) [2023-12-27 04:01:07,402][105692] Updated weights for policy 0, policy_version 1732896 (0.0008) [2023-12-27 04:01:07,956][105620] Updated weights for policy 1, policy_version 1736409 (0.0010) [2023-12-27 04:01:08,008][105620] Updated weights for policy 1, policy_version 1736419 (0.0011) [2023-12-27 04:01:08,060][105620] Updated weights for policy 1, policy_version 1736429 (0.0011) [2023-12-27 04:01:08,114][105692] Updated weights for policy 0, policy_version 1732906 (0.0006) [2023-12-27 04:01:08,167][105692] Updated weights for policy 0, policy_version 1732916 (0.0006) [2023-12-27 04:01:08,230][105692] Updated weights for policy 0, policy_version 1732926 (0.0011) [2023-12-27 04:01:08,772][105620] Updated weights for policy 1, policy_version 1736439 (0.0009) [2023-12-27 04:01:08,834][105620] Updated weights for policy 1, policy_version 1736449 (0.0010) [2023-12-27 04:01:08,904][105620] Updated weights for policy 1, policy_version 1736459 (0.0011) [2023-12-27 04:01:08,964][105692] Updated weights for policy 0, policy_version 1732936 (0.0007) [2023-12-27 04:01:09,025][105692] Updated weights for policy 0, policy_version 1732946 (0.0006) [2023-12-27 04:01:09,078][105692] Updated weights for policy 0, policy_version 1732956 (0.0005) [2023-12-27 04:01:09,667][105620] Updated weights for policy 1, policy_version 1736469 (0.0011) [2023-12-27 04:01:09,731][105620] Updated weights for policy 1, policy_version 1736479 (0.0011) [2023-12-27 04:01:09,759][105692] Updated weights for policy 0, policy_version 1732966 (0.0009) [2023-12-27 04:01:09,795][105620] Updated weights for policy 1, policy_version 1736489 (0.0011) [2023-12-27 04:01:09,820][105692] Updated weights for policy 0, policy_version 1732976 (0.0010) [2023-12-27 04:01:09,886][105692] Updated weights for policy 0, policy_version 1732986 (0.0011) [2023-12-27 04:01:10,549][105620] Updated weights for policy 1, policy_version 1736499 (0.0010) [2023-12-27 04:01:10,613][105620] Updated weights for policy 1, policy_version 1736509 (0.0007) [2023-12-27 04:01:10,680][105620] Updated weights for policy 1, policy_version 1736519 (0.0009) [2023-12-27 04:01:10,711][105692] Updated weights for policy 0, policy_version 1732996 (0.0010) [2023-12-27 04:01:10,773][105692] Updated weights for policy 0, policy_version 1733006 (0.0008) [2023-12-27 04:01:10,835][105692] Updated weights for policy 0, policy_version 1733016 (0.0009) [2023-12-27 04:01:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 888332288. Throughput: 0: 9558.8, 1: 9606.8. Samples: 888338596. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-12-27 04:01:11,062][104569] Avg episode reward: [(0, '8533.739'), (1, '8804.588')] [2023-12-27 04:01:11,422][105620] Updated weights for policy 1, policy_version 1736529 (0.0006) [2023-12-27 04:01:11,481][105620] Updated weights for policy 1, policy_version 1736539 (0.0009) [2023-12-27 04:01:11,544][105620] Updated weights for policy 1, policy_version 1736549 (0.0009) [2023-12-27 04:01:11,603][105620] Updated weights for policy 1, policy_version 1736559 (0.0009) [2023-12-27 04:01:11,660][105692] Updated weights for policy 0, policy_version 1733026 (0.0010) [2023-12-27 04:01:11,715][105692] Updated weights for policy 0, policy_version 1733036 (0.0010) [2023-12-27 04:01:11,778][105692] Updated weights for policy 0, policy_version 1733046 (0.0009) [2023-12-27 04:01:11,845][105692] Updated weights for policy 0, policy_version 1733056 (0.0009) [2023-12-27 04:01:12,365][105620] Updated weights for policy 1, policy_version 1736569 (0.0009) [2023-12-27 04:01:12,438][105620] Updated weights for policy 1, policy_version 1736579 (0.0009) [2023-12-27 04:01:12,513][105620] Updated weights for policy 1, policy_version 1736589 (0.0008) [2023-12-27 04:01:12,644][105692] Updated weights for policy 0, policy_version 1733066 (0.0007) [2023-12-27 04:01:12,705][105692] Updated weights for policy 0, policy_version 1733076 (0.0009) [2023-12-27 04:01:12,753][105692] Updated weights for policy 0, policy_version 1733086 (0.0009) [2023-12-27 04:01:13,322][105620] Updated weights for policy 1, policy_version 1736599 (0.0009) [2023-12-27 04:01:13,377][105620] Updated weights for policy 1, policy_version 1736609 (0.0009) [2023-12-27 04:01:13,434][105620] Updated weights for policy 1, policy_version 1736619 (0.0009) [2023-12-27 04:01:13,477][105692] Updated weights for policy 0, policy_version 1733096 (0.0008) [2023-12-27 04:01:13,546][105692] Updated weights for policy 0, policy_version 1733106 (0.0005) [2023-12-27 04:01:13,615][105692] Updated weights for policy 0, policy_version 1733116 (0.0005) [2023-12-27 04:01:14,187][105692] Updated weights for policy 0, policy_version 1733126 (0.0008) [2023-12-27 04:01:14,244][105692] Updated weights for policy 0, policy_version 1733136 (0.0009) [2023-12-27 04:01:14,250][105620] Updated weights for policy 1, policy_version 1736629 (0.0007) [2023-12-27 04:01:14,298][105620] Updated weights for policy 1, policy_version 1736639 (0.0009) [2023-12-27 04:01:14,308][105692] Updated weights for policy 0, policy_version 1733146 (0.0008) [2023-12-27 04:01:14,348][105620] Updated weights for policy 1, policy_version 1736649 (0.0006) [2023-12-27 04:01:15,059][105692] Updated weights for policy 0, policy_version 1733156 (0.0006) [2023-12-27 04:01:15,106][105692] Updated weights for policy 0, policy_version 1733166 (0.0005) [2023-12-27 04:01:15,154][105692] Updated weights for policy 0, policy_version 1733176 (0.0005) [2023-12-27 04:01:15,178][105620] Updated weights for policy 1, policy_version 1736659 (0.0009) [2023-12-27 04:01:15,236][105620] Updated weights for policy 1, policy_version 1736669 (0.0007) [2023-12-27 04:01:15,298][105620] Updated weights for policy 1, policy_version 1736679 (0.0010) [2023-12-27 04:01:15,777][105692] Updated weights for policy 0, policy_version 1733186 (0.0007) [2023-12-27 04:01:15,828][105692] Updated weights for policy 0, policy_version 1733196 (0.0009) [2023-12-27 04:01:15,885][105692] Updated weights for policy 0, policy_version 1733206 (0.0009) [2023-12-27 04:01:15,932][105692] Updated weights for policy 0, policy_version 1733216 (0.0009) [2023-12-27 04:01:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19383.1). Total num frames: 888422400. Throughput: 0: 9463.4, 1: 9562.4. Samples: 888392416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:01:16,063][104569] Avg episode reward: [(0, '8166.410'), (1, '8620.696')] [2023-12-27 04:01:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001733216_443768832.pth... [2023-12-27 04:01:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001736688_444653568.pth... [2023-12-27 04:01:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001732096_443482112.pth [2023-12-27 04:01:16,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001735632_444383232.pth [2023-12-27 04:01:16,112][105620] Updated weights for policy 1, policy_version 1736689 (0.0010) [2023-12-27 04:01:16,157][105620] Updated weights for policy 1, policy_version 1736699 (0.0008) [2023-12-27 04:01:16,208][105620] Updated weights for policy 1, policy_version 1736709 (0.0009) [2023-12-27 04:01:16,271][105620] Updated weights for policy 1, policy_version 1736719 (0.0007) [2023-12-27 04:01:16,723][105692] Updated weights for policy 0, policy_version 1733226 (0.0010) [2023-12-27 04:01:16,777][105692] Updated weights for policy 0, policy_version 1733236 (0.0009) [2023-12-27 04:01:16,821][105692] Updated weights for policy 0, policy_version 1733246 (0.0006) [2023-12-27 04:01:16,921][105620] Updated weights for policy 1, policy_version 1736729 (0.0006) [2023-12-27 04:01:16,985][105620] Updated weights for policy 1, policy_version 1736739 (0.0010) [2023-12-27 04:01:17,046][105620] Updated weights for policy 1, policy_version 1736749 (0.0009) [2023-12-27 04:01:17,602][105620] Updated weights for policy 1, policy_version 1736759 (0.0006) [2023-12-27 04:01:17,657][105620] Updated weights for policy 1, policy_version 1736769 (0.0005) [2023-12-27 04:01:17,671][105692] Updated weights for policy 0, policy_version 1733256 (0.0008) [2023-12-27 04:01:17,711][105620] Updated weights for policy 1, policy_version 1736779 (0.0006) [2023-12-27 04:01:17,729][105692] Updated weights for policy 0, policy_version 1733266 (0.0008) [2023-12-27 04:01:17,790][105692] Updated weights for policy 0, policy_version 1733276 (0.0008) [2023-12-27 04:01:18,439][105620] Updated weights for policy 1, policy_version 1736789 (0.0008) [2023-12-27 04:01:18,487][105620] Updated weights for policy 1, policy_version 1736799 (0.0008) [2023-12-27 04:01:18,533][105692] Updated weights for policy 0, policy_version 1733286 (0.0009) [2023-12-27 04:01:18,538][105620] Updated weights for policy 1, policy_version 1736809 (0.0009) [2023-12-27 04:01:18,590][105692] Updated weights for policy 0, policy_version 1733296 (0.0007) [2023-12-27 04:01:18,641][105692] Updated weights for policy 0, policy_version 1733306 (0.0009) [2023-12-27 04:01:19,295][105620] Updated weights for policy 1, policy_version 1736819 (0.0008) [2023-12-27 04:01:19,368][105620] Updated weights for policy 1, policy_version 1736829 (0.0009) [2023-12-27 04:01:19,425][105692] Updated weights for policy 0, policy_version 1733316 (0.0010) [2023-12-27 04:01:19,428][105620] Updated weights for policy 1, policy_version 1736839 (0.0008) [2023-12-27 04:01:19,479][105692] Updated weights for policy 0, policy_version 1733326 (0.0008) [2023-12-27 04:01:19,545][105692] Updated weights for policy 0, policy_version 1733336 (0.0009) [2023-12-27 04:01:20,094][105620] Updated weights for policy 1, policy_version 1736849 (0.0008) [2023-12-27 04:01:20,160][105620] Updated weights for policy 1, policy_version 1736859 (0.0008) [2023-12-27 04:01:20,219][105620] Updated weights for policy 1, policy_version 1736869 (0.0005) [2023-12-27 04:01:20,273][105620] Updated weights for policy 1, policy_version 1736879 (0.0008) [2023-12-27 04:01:20,371][105692] Updated weights for policy 0, policy_version 1733346 (0.0009) [2023-12-27 04:01:20,436][105692] Updated weights for policy 0, policy_version 1733356 (0.0009) [2023-12-27 04:01:20,501][105692] Updated weights for policy 0, policy_version 1733366 (0.0009) [2023-12-27 04:01:20,564][105692] Updated weights for policy 0, policy_version 1733376 (0.0009) [2023-12-27 04:01:21,055][105620] Updated weights for policy 1, policy_version 1736889 (0.0010) [2023-12-27 04:01:21,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19114.7, 300 sec: 19355.3). Total num frames: 888512512. Throughput: 0: 9477.3, 1: 9526.2. Samples: 888507888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:01:21,062][104569] Avg episode reward: [(0, '8168.104'), (1, '8715.732')] [2023-12-27 04:01:21,122][105620] Updated weights for policy 1, policy_version 1736899 (0.0008) [2023-12-27 04:01:21,185][105620] Updated weights for policy 1, policy_version 1736909 (0.0009) [2023-12-27 04:01:21,244][105692] Updated weights for policy 0, policy_version 1733386 (0.0008) [2023-12-27 04:01:21,306][105692] Updated weights for policy 0, policy_version 1733396 (0.0009) [2023-12-27 04:01:21,368][105692] Updated weights for policy 0, policy_version 1733406 (0.0009) [2023-12-27 04:01:21,994][105620] Updated weights for policy 1, policy_version 1736919 (0.0008) [2023-12-27 04:01:22,054][105692] Updated weights for policy 0, policy_version 1733416 (0.0007) [2023-12-27 04:01:22,055][105620] Updated weights for policy 1, policy_version 1736929 (0.0008) [2023-12-27 04:01:22,114][105692] Updated weights for policy 0, policy_version 1733426 (0.0006) [2023-12-27 04:01:22,116][105620] Updated weights for policy 1, policy_version 1736939 (0.0008) [2023-12-27 04:01:22,180][105692] Updated weights for policy 0, policy_version 1733436 (0.0006) [2023-12-27 04:01:22,812][105620] Updated weights for policy 1, policy_version 1736949 (0.0007) [2023-12-27 04:01:22,879][105692] Updated weights for policy 0, policy_version 1733446 (0.0006) [2023-12-27 04:01:22,880][105620] Updated weights for policy 1, policy_version 1736959 (0.0007) [2023-12-27 04:01:22,945][105692] Updated weights for policy 0, policy_version 1733456 (0.0008) [2023-12-27 04:01:22,948][105620] Updated weights for policy 1, policy_version 1736969 (0.0008) [2023-12-27 04:01:23,009][105692] Updated weights for policy 0, policy_version 1733466 (0.0008) [2023-12-27 04:01:23,599][105620] Updated weights for policy 1, policy_version 1736979 (0.0007) [2023-12-27 04:01:23,653][105620] Updated weights for policy 1, policy_version 1736989 (0.0005) [2023-12-27 04:01:23,699][105620] Updated weights for policy 1, policy_version 1736999 (0.0008) [2023-12-27 04:01:23,758][105692] Updated weights for policy 0, policy_version 1733476 (0.0009) [2023-12-27 04:01:23,805][105692] Updated weights for policy 0, policy_version 1733486 (0.0008) [2023-12-27 04:01:23,867][105692] Updated weights for policy 0, policy_version 1733496 (0.0009) [2023-12-27 04:01:24,356][105620] Updated weights for policy 1, policy_version 1737009 (0.0008) [2023-12-27 04:01:24,426][105620] Updated weights for policy 1, policy_version 1737019 (0.0006) [2023-12-27 04:01:24,488][105620] Updated weights for policy 1, policy_version 1737029 (0.0006) [2023-12-27 04:01:24,561][105620] Updated weights for policy 1, policy_version 1737039 (0.0005) [2023-12-27 04:01:24,647][105692] Updated weights for policy 0, policy_version 1733506 (0.0009) [2023-12-27 04:01:24,707][105692] Updated weights for policy 0, policy_version 1733516 (0.0008) [2023-12-27 04:01:24,761][105692] Updated weights for policy 0, policy_version 1733526 (0.0009) [2023-12-27 04:01:24,809][105692] Updated weights for policy 0, policy_version 1733536 (0.0005) [2023-12-27 04:01:25,200][105620] Updated weights for policy 1, policy_version 1737049 (0.0006) [2023-12-27 04:01:25,258][105620] Updated weights for policy 1, policy_version 1737059 (0.0009) [2023-12-27 04:01:25,323][105620] Updated weights for policy 1, policy_version 1737069 (0.0008) [2023-12-27 04:01:25,545][105692] Updated weights for policy 0, policy_version 1733546 (0.0007) [2023-12-27 04:01:25,608][105692] Updated weights for policy 0, policy_version 1733556 (0.0008) [2023-12-27 04:01:25,673][105692] Updated weights for policy 0, policy_version 1733566 (0.0006) [2023-12-27 04:01:26,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19114.7, 300 sec: 19355.3). Total num frames: 888610816. Throughput: 0: 9504.9, 1: 9470.4. Samples: 888623092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:01:26,062][104569] Avg episode reward: [(0, '7895.458'), (1, '9174.788')] [2023-12-27 04:01:26,070][105620] Updated weights for policy 1, policy_version 1737079 (0.0009) [2023-12-27 04:01:26,116][105620] Updated weights for policy 1, policy_version 1737089 (0.0008) [2023-12-27 04:01:26,170][105620] Updated weights for policy 1, policy_version 1737099 (0.0009) [2023-12-27 04:01:26,367][105692] Updated weights for policy 0, policy_version 1733576 (0.0009) [2023-12-27 04:01:26,426][105692] Updated weights for policy 0, policy_version 1733586 (0.0009) [2023-12-27 04:01:26,488][105692] Updated weights for policy 0, policy_version 1733596 (0.0009) [2023-12-27 04:01:26,935][105620] Updated weights for policy 1, policy_version 1737109 (0.0007) [2023-12-27 04:01:26,989][105620] Updated weights for policy 1, policy_version 1737119 (0.0005) [2023-12-27 04:01:27,041][105620] Updated weights for policy 1, policy_version 1737129 (0.0005) [2023-12-27 04:01:27,307][105692] Updated weights for policy 0, policy_version 1733606 (0.0006) [2023-12-27 04:01:27,355][105692] Updated weights for policy 0, policy_version 1733616 (0.0006) [2023-12-27 04:01:27,410][105692] Updated weights for policy 0, policy_version 1733626 (0.0007) [2023-12-27 04:01:27,612][105620] Updated weights for policy 1, policy_version 1737139 (0.0008) [2023-12-27 04:01:27,663][105620] Updated weights for policy 1, policy_version 1737149 (0.0006) [2023-12-27 04:01:27,727][105620] Updated weights for policy 1, policy_version 1737159 (0.0008) [2023-12-27 04:01:28,016][105692] Updated weights for policy 0, policy_version 1733636 (0.0005) [2023-12-27 04:01:28,067][105692] Updated weights for policy 0, policy_version 1733646 (0.0006) [2023-12-27 04:01:28,121][105692] Updated weights for policy 0, policy_version 1733656 (0.0005) [2023-12-27 04:01:28,427][105620] Updated weights for policy 1, policy_version 1737169 (0.0008) [2023-12-27 04:01:28,483][105620] Updated weights for policy 1, policy_version 1737179 (0.0008) [2023-12-27 04:01:28,533][105620] Updated weights for policy 1, policy_version 1737189 (0.0009) [2023-12-27 04:01:28,586][105620] Updated weights for policy 1, policy_version 1737199 (0.0008) [2023-12-27 04:01:28,724][105692] Updated weights for policy 0, policy_version 1733666 (0.0005) [2023-12-27 04:01:28,776][105692] Updated weights for policy 0, policy_version 1733676 (0.0005) [2023-12-27 04:01:28,834][105692] Updated weights for policy 0, policy_version 1733686 (0.0005) [2023-12-27 04:01:28,898][105692] Updated weights for policy 0, policy_version 1733696 (0.0006) [2023-12-27 04:01:29,356][105620] Updated weights for policy 1, policy_version 1737209 (0.0007) [2023-12-27 04:01:29,421][105620] Updated weights for policy 1, policy_version 1737219 (0.0007) [2023-12-27 04:01:29,484][105620] Updated weights for policy 1, policy_version 1737229 (0.0007) [2023-12-27 04:01:29,506][105692] Updated weights for policy 0, policy_version 1733706 (0.0008) [2023-12-27 04:01:29,556][105692] Updated weights for policy 0, policy_version 1733716 (0.0009) [2023-12-27 04:01:29,601][105692] Updated weights for policy 0, policy_version 1733726 (0.0005) [2023-12-27 04:01:30,198][105620] Updated weights for policy 1, policy_version 1737239 (0.0008) [2023-12-27 04:01:30,262][105620] Updated weights for policy 1, policy_version 1737249 (0.0008) [2023-12-27 04:01:30,312][105692] Updated weights for policy 0, policy_version 1733736 (0.0006) [2023-12-27 04:01:30,320][105620] Updated weights for policy 1, policy_version 1737259 (0.0009) [2023-12-27 04:01:30,374][105692] Updated weights for policy 0, policy_version 1733746 (0.0010) [2023-12-27 04:01:30,426][105692] Updated weights for policy 0, policy_version 1733756 (0.0010) [2023-12-27 04:01:31,055][105620] Updated weights for policy 1, policy_version 1737269 (0.0007) [2023-12-27 04:01:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 18978.2, 300 sec: 19383.1). Total num frames: 888709120. Throughput: 0: 9565.1, 1: 9476.5. Samples: 888683544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:01:31,062][104569] Avg episode reward: [(0, '8350.957'), (1, '9262.629')] [2023-12-27 04:01:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001733760_443908096.pth... [2023-12-27 04:01:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001732672_443629568.pth [2023-12-27 04:01:31,123][105620] Updated weights for policy 1, policy_version 1737279 (0.0007) [2023-12-27 04:01:31,131][105692] Updated weights for policy 0, policy_version 1733766 (0.0011) [2023-12-27 04:01:31,184][105620] Updated weights for policy 1, policy_version 1737289 (0.0006) [2023-12-27 04:01:31,190][105692] Updated weights for policy 0, policy_version 1733776 (0.0011) [2023-12-27 04:01:31,222][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001737296_444809216.pth... [2023-12-27 04:01:31,226][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001736144_444514304.pth [2023-12-27 04:01:31,239][105692] Updated weights for policy 0, policy_version 1733786 (0.0010) [2023-12-27 04:01:31,862][105620] Updated weights for policy 1, policy_version 1737299 (0.0005) [2023-12-27 04:01:31,914][105620] Updated weights for policy 1, policy_version 1737309 (0.0005) [2023-12-27 04:01:31,941][105692] Updated weights for policy 0, policy_version 1733796 (0.0009) [2023-12-27 04:01:31,974][105620] Updated weights for policy 1, policy_version 1737319 (0.0007) [2023-12-27 04:01:31,996][105692] Updated weights for policy 0, policy_version 1733806 (0.0010) [2023-12-27 04:01:32,042][105692] Updated weights for policy 0, policy_version 1733816 (0.0010) [2023-12-27 04:01:32,561][105620] Updated weights for policy 1, policy_version 1737329 (0.0007) [2023-12-27 04:01:32,617][105620] Updated weights for policy 1, policy_version 1737339 (0.0005) [2023-12-27 04:01:32,671][105620] Updated weights for policy 1, policy_version 1737349 (0.0008) [2023-12-27 04:01:32,725][105620] Updated weights for policy 1, policy_version 1737359 (0.0010) [2023-12-27 04:01:32,790][105692] Updated weights for policy 0, policy_version 1733826 (0.0010) [2023-12-27 04:01:32,842][105692] Updated weights for policy 0, policy_version 1733836 (0.0010) [2023-12-27 04:01:32,891][105692] Updated weights for policy 0, policy_version 1733846 (0.0009) [2023-12-27 04:01:32,946][105692] Updated weights for policy 0, policy_version 1733856 (0.0010) [2023-12-27 04:01:33,401][105620] Updated weights for policy 1, policy_version 1737369 (0.0009) [2023-12-27 04:01:33,456][105620] Updated weights for policy 1, policy_version 1737380 (0.0009) [2023-12-27 04:01:33,518][105620] Updated weights for policy 1, policy_version 1737390 (0.0008) [2023-12-27 04:01:33,525][105692] Updated weights for policy 0, policy_version 1733866 (0.0006) [2023-12-27 04:01:33,583][105692] Updated weights for policy 0, policy_version 1733876 (0.0007) [2023-12-27 04:01:33,640][105692] Updated weights for policy 0, policy_version 1733886 (0.0005) [2023-12-27 04:01:34,210][105692] Updated weights for policy 0, policy_version 1733896 (0.0006) [2023-12-27 04:01:34,269][105692] Updated weights for policy 0, policy_version 1733906 (0.0006) [2023-12-27 04:01:34,326][105692] Updated weights for policy 0, policy_version 1733916 (0.0007) [2023-12-27 04:01:34,340][105620] Updated weights for policy 1, policy_version 1737400 (0.0007) [2023-12-27 04:01:34,411][105620] Updated weights for policy 1, policy_version 1737410 (0.0009) [2023-12-27 04:01:34,473][105620] Updated weights for policy 1, policy_version 1737420 (0.0009) [2023-12-27 04:01:34,876][105692] Updated weights for policy 0, policy_version 1733926 (0.0006) [2023-12-27 04:01:34,937][105692] Updated weights for policy 0, policy_version 1733936 (0.0010) [2023-12-27 04:01:34,986][105692] Updated weights for policy 0, policy_version 1733946 (0.0009) [2023-12-27 04:01:35,194][105620] Updated weights for policy 1, policy_version 1737430 (0.0007) [2023-12-27 04:01:35,252][105620] Updated weights for policy 1, policy_version 1737440 (0.0007) [2023-12-27 04:01:35,320][105620] Updated weights for policy 1, policy_version 1737450 (0.0006) [2023-12-27 04:01:35,638][105692] Updated weights for policy 0, policy_version 1733956 (0.0007) [2023-12-27 04:01:35,699][105692] Updated weights for policy 0, policy_version 1733966 (0.0005) [2023-12-27 04:01:35,758][105692] Updated weights for policy 0, policy_version 1733976 (0.0005) [2023-12-27 04:01:35,936][105620] Updated weights for policy 1, policy_version 1737460 (0.0010) [2023-12-27 04:01:35,989][105620] Updated weights for policy 1, policy_version 1737470 (0.0009) [2023-12-27 04:01:36,046][105620] Updated weights for policy 1, policy_version 1737480 (0.0010) [2023-12-27 04:01:36,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19251.1, 300 sec: 19410.9). Total num frames: 888815616. Throughput: 0: 9693.7, 1: 9520.5. Samples: 888805004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:01:36,063][104569] Avg episode reward: [(0, '8533.634'), (1, '9170.260')] [2023-12-27 04:01:36,348][105692] Updated weights for policy 0, policy_version 1733986 (0.0006) [2023-12-27 04:01:36,415][105692] Updated weights for policy 0, policy_version 1733996 (0.0009) [2023-12-27 04:01:36,474][105692] Updated weights for policy 0, policy_version 1734006 (0.0009) [2023-12-27 04:01:36,538][105692] Updated weights for policy 0, policy_version 1734016 (0.0009) [2023-12-27 04:01:36,762][105620] Updated weights for policy 1, policy_version 1737490 (0.0009) [2023-12-27 04:01:36,820][105620] Updated weights for policy 1, policy_version 1737500 (0.0009) [2023-12-27 04:01:36,885][105620] Updated weights for policy 1, policy_version 1737510 (0.0009) [2023-12-27 04:01:36,948][105620] Updated weights for policy 1, policy_version 1737520 (0.0009) [2023-12-27 04:01:37,260][105692] Updated weights for policy 0, policy_version 1734026 (0.0010) [2023-12-27 04:01:37,313][105692] Updated weights for policy 0, policy_version 1734036 (0.0010) [2023-12-27 04:01:37,365][105692] Updated weights for policy 0, policy_version 1734046 (0.0010) [2023-12-27 04:01:37,716][105620] Updated weights for policy 1, policy_version 1737530 (0.0006) [2023-12-27 04:01:37,768][105620] Updated weights for policy 1, policy_version 1737540 (0.0005) [2023-12-27 04:01:37,824][105620] Updated weights for policy 1, policy_version 1737550 (0.0006) [2023-12-27 04:01:37,964][105692] Updated weights for policy 0, policy_version 1734056 (0.0006) [2023-12-27 04:01:38,027][105692] Updated weights for policy 0, policy_version 1734066 (0.0006) [2023-12-27 04:01:38,090][105692] Updated weights for policy 0, policy_version 1734076 (0.0005) [2023-12-27 04:01:38,411][105620] Updated weights for policy 1, policy_version 1737560 (0.0008) [2023-12-27 04:01:38,481][105620] Updated weights for policy 1, policy_version 1737570 (0.0011) [2023-12-27 04:01:38,537][105620] Updated weights for policy 1, policy_version 1737580 (0.0010) [2023-12-27 04:01:38,690][105692] Updated weights for policy 0, policy_version 1734086 (0.0007) [2023-12-27 04:01:38,738][105692] Updated weights for policy 0, policy_version 1734096 (0.0008) [2023-12-27 04:01:38,786][105692] Updated weights for policy 0, policy_version 1734106 (0.0008) [2023-12-27 04:01:39,285][105620] Updated weights for policy 1, policy_version 1737590 (0.0011) [2023-12-27 04:01:39,345][105620] Updated weights for policy 1, policy_version 1737600 (0.0010) [2023-12-27 04:01:39,410][105620] Updated weights for policy 1, policy_version 1737610 (0.0009) [2023-12-27 04:01:39,540][105692] Updated weights for policy 0, policy_version 1734116 (0.0005) [2023-12-27 04:01:39,598][105692] Updated weights for policy 0, policy_version 1734126 (0.0009) [2023-12-27 04:01:39,652][105692] Updated weights for policy 0, policy_version 1734136 (0.0010) [2023-12-27 04:01:40,072][105620] Updated weights for policy 1, policy_version 1737620 (0.0011) [2023-12-27 04:01:40,134][105620] Updated weights for policy 1, policy_version 1737630 (0.0008) [2023-12-27 04:01:40,205][105620] Updated weights for policy 1, policy_version 1737640 (0.0011) [2023-12-27 04:01:40,420][105692] Updated weights for policy 0, policy_version 1734146 (0.0009) [2023-12-27 04:01:40,489][105692] Updated weights for policy 0, policy_version 1734156 (0.0009) [2023-12-27 04:01:40,556][105692] Updated weights for policy 0, policy_version 1734166 (0.0008) [2023-12-27 04:01:40,618][105692] Updated weights for policy 0, policy_version 1734176 (0.0009) [2023-12-27 04:01:40,867][105620] Updated weights for policy 1, policy_version 1737650 (0.0011) [2023-12-27 04:01:40,925][105620] Updated weights for policy 1, policy_version 1737660 (0.0010) [2023-12-27 04:01:40,986][105620] Updated weights for policy 1, policy_version 1737670 (0.0010) [2023-12-27 04:01:41,048][105620] Updated weights for policy 1, policy_version 1737680 (0.0010) [2023-12-27 04:01:41,062][104569] Fps is (10 sec: 21299.1, 60 sec: 19387.7, 300 sec: 19438.7). Total num frames: 888922112. Throughput: 0: 9766.8, 1: 9616.6. Samples: 888927808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:01:41,062][104569] Avg episode reward: [(0, '8527.227'), (1, '9169.997')] [2023-12-27 04:01:41,400][105692] Updated weights for policy 0, policy_version 1734186 (0.0008) [2023-12-27 04:01:41,459][105692] Updated weights for policy 0, policy_version 1734196 (0.0009) [2023-12-27 04:01:41,519][105692] Updated weights for policy 0, policy_version 1734206 (0.0008) [2023-12-27 04:01:41,768][105620] Updated weights for policy 1, policy_version 1737690 (0.0009) [2023-12-27 04:01:41,831][105620] Updated weights for policy 1, policy_version 1737700 (0.0006) [2023-12-27 04:01:41,896][105620] Updated weights for policy 1, policy_version 1737710 (0.0011) [2023-12-27 04:01:42,225][105692] Updated weights for policy 0, policy_version 1734216 (0.0010) [2023-12-27 04:01:42,288][105692] Updated weights for policy 0, policy_version 1734226 (0.0011) [2023-12-27 04:01:42,359][105692] Updated weights for policy 0, policy_version 1734236 (0.0010) [2023-12-27 04:01:42,708][105620] Updated weights for policy 1, policy_version 1737720 (0.0009) [2023-12-27 04:01:42,770][105620] Updated weights for policy 1, policy_version 1737730 (0.0008) [2023-12-27 04:01:42,830][105620] Updated weights for policy 1, policy_version 1737740 (0.0009) [2023-12-27 04:01:43,031][105692] Updated weights for policy 0, policy_version 1734246 (0.0010) [2023-12-27 04:01:43,093][105692] Updated weights for policy 0, policy_version 1734256 (0.0010) [2023-12-27 04:01:43,151][105692] Updated weights for policy 0, policy_version 1734266 (0.0010) [2023-12-27 04:01:43,561][105620] Updated weights for policy 1, policy_version 1737750 (0.0009) [2023-12-27 04:01:43,615][105620] Updated weights for policy 1, policy_version 1737760 (0.0009) [2023-12-27 04:01:43,670][105620] Updated weights for policy 1, policy_version 1737770 (0.0011) [2023-12-27 04:01:43,885][105692] Updated weights for policy 0, policy_version 1734276 (0.0010) [2023-12-27 04:01:43,940][105692] Updated weights for policy 0, policy_version 1734286 (0.0010) [2023-12-27 04:01:43,998][105692] Updated weights for policy 0, policy_version 1734296 (0.0010) [2023-12-27 04:01:44,415][105620] Updated weights for policy 1, policy_version 1737780 (0.0010) [2023-12-27 04:01:44,469][105620] Updated weights for policy 1, policy_version 1737790 (0.0011) [2023-12-27 04:01:44,518][105620] Updated weights for policy 1, policy_version 1737800 (0.0010) [2023-12-27 04:01:44,685][105692] Updated weights for policy 0, policy_version 1734306 (0.0010) [2023-12-27 04:01:44,737][105692] Updated weights for policy 0, policy_version 1734316 (0.0011) [2023-12-27 04:01:44,794][105692] Updated weights for policy 0, policy_version 1734326 (0.0009) [2023-12-27 04:01:44,854][105692] Updated weights for policy 0, policy_version 1734336 (0.0005) [2023-12-27 04:01:45,273][105620] Updated weights for policy 1, policy_version 1737810 (0.0010) [2023-12-27 04:01:45,333][105620] Updated weights for policy 1, policy_version 1737820 (0.0008) [2023-12-27 04:01:45,403][105620] Updated weights for policy 1, policy_version 1737830 (0.0007) [2023-12-27 04:01:45,434][105692] Updated weights for policy 0, policy_version 1734346 (0.0011) [2023-12-27 04:01:45,466][105620] Updated weights for policy 1, policy_version 1737840 (0.0005) [2023-12-27 04:01:45,487][105692] Updated weights for policy 0, policy_version 1734356 (0.0011) [2023-12-27 04:01:45,536][105692] Updated weights for policy 0, policy_version 1734366 (0.0010) [2023-12-27 04:01:46,036][105620] Updated weights for policy 1, policy_version 1737850 (0.0005) [2023-12-27 04:01:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 889012224. Throughput: 0: 9710.6, 1: 9646.3. Samples: 888984000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:01:46,063][104569] Avg episode reward: [(0, '9078.000'), (1, '8985.350')] [2023-12-27 04:01:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001734368_444063744.pth... [2023-12-27 04:01:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001733216_443768832.pth [2023-12-27 04:01:46,074][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_001734368_444063744.pth [2023-12-27 04:01:46,092][105620] Updated weights for policy 1, policy_version 1737860 (0.0005) [2023-12-27 04:01:46,159][105620] Updated weights for policy 1, policy_version 1737870 (0.0005) [2023-12-27 04:01:46,169][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001737872_444956672.pth... [2023-12-27 04:01:46,173][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001736688_444653568.pth [2023-12-27 04:01:46,174][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_001737872_444956672.pth [2023-12-27 04:01:46,280][105692] Updated weights for policy 0, policy_version 1734376 (0.0010) [2023-12-27 04:01:46,339][105692] Updated weights for policy 0, policy_version 1734386 (0.0010) [2023-12-27 04:01:46,397][105692] Updated weights for policy 0, policy_version 1734396 (0.0010) [2023-12-27 04:01:46,718][105620] Updated weights for policy 1, policy_version 1737880 (0.0005) [2023-12-27 04:01:46,774][105620] Updated weights for policy 1, policy_version 1737890 (0.0005) [2023-12-27 04:01:46,831][105620] Updated weights for policy 1, policy_version 1737900 (0.0005) [2023-12-27 04:01:47,147][105692] Updated weights for policy 0, policy_version 1734406 (0.0010) [2023-12-27 04:01:47,210][105692] Updated weights for policy 0, policy_version 1734416 (0.0011) [2023-12-27 04:01:47,271][105692] Updated weights for policy 0, policy_version 1734426 (0.0008) [2023-12-27 04:01:47,434][105620] Updated weights for policy 1, policy_version 1737910 (0.0008) [2023-12-27 04:01:47,486][105620] Updated weights for policy 1, policy_version 1737920 (0.0010) [2023-12-27 04:01:47,540][105620] Updated weights for policy 1, policy_version 1737930 (0.0010) [2023-12-27 04:01:47,914][105692] Updated weights for policy 0, policy_version 1734436 (0.0007) [2023-12-27 04:01:47,976][105692] Updated weights for policy 0, policy_version 1734446 (0.0005) [2023-12-27 04:01:48,028][105692] Updated weights for policy 0, policy_version 1734456 (0.0006) [2023-12-27 04:01:48,308][105620] Updated weights for policy 1, policy_version 1737940 (0.0010) [2023-12-27 04:01:48,363][105620] Updated weights for policy 1, policy_version 1737950 (0.0011) [2023-12-27 04:01:48,419][105620] Updated weights for policy 1, policy_version 1737960 (0.0010) [2023-12-27 04:01:48,640][105692] Updated weights for policy 0, policy_version 1734466 (0.0005) [2023-12-27 04:01:48,699][105692] Updated weights for policy 0, policy_version 1734476 (0.0006) [2023-12-27 04:01:48,761][105692] Updated weights for policy 0, policy_version 1734486 (0.0007) [2023-12-27 04:01:48,829][105692] Updated weights for policy 0, policy_version 1734496 (0.0007) [2023-12-27 04:01:49,076][105620] Updated weights for policy 1, policy_version 1737970 (0.0010) [2023-12-27 04:01:49,125][105620] Updated weights for policy 1, policy_version 1737980 (0.0010) [2023-12-27 04:01:49,169][105620] Updated weights for policy 1, policy_version 1737990 (0.0010) [2023-12-27 04:01:49,220][105620] Updated weights for policy 1, policy_version 1738000 (0.0010) [2023-12-27 04:01:49,559][105692] Updated weights for policy 0, policy_version 1734506 (0.0008) [2023-12-27 04:01:49,614][105692] Updated weights for policy 0, policy_version 1734516 (0.0008) [2023-12-27 04:01:49,681][105692] Updated weights for policy 0, policy_version 1734526 (0.0006) [2023-12-27 04:01:50,028][105620] Updated weights for policy 1, policy_version 1738010 (0.0010) [2023-12-27 04:01:50,083][105620] Updated weights for policy 1, policy_version 1738020 (0.0010) [2023-12-27 04:01:50,150][105620] Updated weights for policy 1, policy_version 1738030 (0.0010) [2023-12-27 04:01:50,366][105692] Updated weights for policy 0, policy_version 1734536 (0.0008) [2023-12-27 04:01:50,428][105692] Updated weights for policy 0, policy_version 1734546 (0.0008) [2023-12-27 04:01:50,489][105692] Updated weights for policy 0, policy_version 1734556 (0.0008) [2023-12-27 04:01:50,838][105620] Updated weights for policy 1, policy_version 1738040 (0.0006) [2023-12-27 04:01:50,888][105620] Updated weights for policy 1, policy_version 1738050 (0.0005) [2023-12-27 04:01:50,946][105620] Updated weights for policy 1, policy_version 1738060 (0.0009) [2023-12-27 04:01:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 889118720. Throughput: 0: 9890.6, 1: 9724.1. Samples: 889106024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:01:51,062][104569] Avg episode reward: [(0, '8621.743'), (1, '9170.456')] [2023-12-27 04:01:51,246][105692] Updated weights for policy 0, policy_version 1734566 (0.0009) [2023-12-27 04:01:51,312][105692] Updated weights for policy 0, policy_version 1734576 (0.0011) [2023-12-27 04:01:51,384][105692] Updated weights for policy 0, policy_version 1734586 (0.0008) [2023-12-27 04:01:51,699][105620] Updated weights for policy 1, policy_version 1738070 (0.0009) [2023-12-27 04:01:51,768][105620] Updated weights for policy 1, policy_version 1738080 (0.0007) [2023-12-27 04:01:51,824][105620] Updated weights for policy 1, policy_version 1738090 (0.0005) [2023-12-27 04:01:52,102][105692] Updated weights for policy 0, policy_version 1734596 (0.0008) [2023-12-27 04:01:52,158][105692] Updated weights for policy 0, policy_version 1734606 (0.0005) [2023-12-27 04:01:52,233][105692] Updated weights for policy 0, policy_version 1734616 (0.0006) [2023-12-27 04:01:52,414][105620] Updated weights for policy 1, policy_version 1738100 (0.0007) [2023-12-27 04:01:52,462][105620] Updated weights for policy 1, policy_version 1738110 (0.0008) [2023-12-27 04:01:52,515][105620] Updated weights for policy 1, policy_version 1738120 (0.0009) [2023-12-27 04:01:52,940][105692] Updated weights for policy 0, policy_version 1734626 (0.0010) [2023-12-27 04:01:52,999][105692] Updated weights for policy 0, policy_version 1734636 (0.0010) [2023-12-27 04:01:53,050][105692] Updated weights for policy 0, policy_version 1734646 (0.0009) [2023-12-27 04:01:53,101][105692] Updated weights for policy 0, policy_version 1734656 (0.0010) [2023-12-27 04:01:53,217][105620] Updated weights for policy 1, policy_version 1738130 (0.0007) [2023-12-27 04:01:53,279][105620] Updated weights for policy 1, policy_version 1738140 (0.0010) [2023-12-27 04:01:53,336][105620] Updated weights for policy 1, policy_version 1738150 (0.0010) [2023-12-27 04:01:53,397][105620] Updated weights for policy 1, policy_version 1738160 (0.0010) [2023-12-27 04:01:53,840][105692] Updated weights for policy 0, policy_version 1734666 (0.0008) [2023-12-27 04:01:53,884][105692] Updated weights for policy 0, policy_version 1734676 (0.0010) [2023-12-27 04:01:53,945][105692] Updated weights for policy 0, policy_version 1734686 (0.0010) [2023-12-27 04:01:54,139][105620] Updated weights for policy 1, policy_version 1738170 (0.0010) [2023-12-27 04:01:54,194][105620] Updated weights for policy 1, policy_version 1738180 (0.0010) [2023-12-27 04:01:54,252][105620] Updated weights for policy 1, policy_version 1738190 (0.0010) [2023-12-27 04:01:54,580][105692] Updated weights for policy 0, policy_version 1734696 (0.0009) [2023-12-27 04:01:54,628][105692] Updated weights for policy 0, policy_version 1734706 (0.0010) [2023-12-27 04:01:54,686][105692] Updated weights for policy 0, policy_version 1734716 (0.0010) [2023-12-27 04:01:54,967][105620] Updated weights for policy 1, policy_version 1738200 (0.0010) [2023-12-27 04:01:55,031][105620] Updated weights for policy 1, policy_version 1738210 (0.0010) [2023-12-27 04:01:55,094][105620] Updated weights for policy 1, policy_version 1738220 (0.0011) [2023-12-27 04:01:55,349][105692] Updated weights for policy 0, policy_version 1734726 (0.0011) [2023-12-27 04:01:55,405][105692] Updated weights for policy 0, policy_version 1734736 (0.0008) [2023-12-27 04:01:55,464][105692] Updated weights for policy 0, policy_version 1734746 (0.0005) [2023-12-27 04:01:55,835][105620] Updated weights for policy 1, policy_version 1738230 (0.0011) [2023-12-27 04:01:55,902][105620] Updated weights for policy 1, policy_version 1738240 (0.0011) [2023-12-27 04:01:55,962][105620] Updated weights for policy 1, policy_version 1738250 (0.0011) [2023-12-27 04:01:56,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 889217024. Throughput: 0: 9904.9, 1: 9770.9. Samples: 889224012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:01:56,063][104569] Avg episode reward: [(0, '7982.072'), (1, '9171.110')] [2023-12-27 04:01:56,077][105692] Updated weights for policy 0, policy_version 1734756 (0.0010) [2023-12-27 04:01:56,137][105692] Updated weights for policy 0, policy_version 1734766 (0.0011) [2023-12-27 04:01:56,201][105692] Updated weights for policy 0, policy_version 1734776 (0.0009) [2023-12-27 04:01:56,719][105620] Updated weights for policy 1, policy_version 1738260 (0.0011) [2023-12-27 04:01:56,773][105620] Updated weights for policy 1, policy_version 1738270 (0.0010) [2023-12-27 04:01:56,834][105620] Updated weights for policy 1, policy_version 1738280 (0.0010) [2023-12-27 04:01:56,880][105692] Updated weights for policy 0, policy_version 1734786 (0.0006) [2023-12-27 04:01:56,929][105692] Updated weights for policy 0, policy_version 1734796 (0.0005) [2023-12-27 04:01:56,977][105692] Updated weights for policy 0, policy_version 1734806 (0.0005) [2023-12-27 04:01:57,022][105692] Updated weights for policy 0, policy_version 1734816 (0.0005) [2023-12-27 04:01:57,391][105620] Updated weights for policy 1, policy_version 1738290 (0.0005) [2023-12-27 04:01:57,439][105620] Updated weights for policy 1, policy_version 1738300 (0.0005) [2023-12-27 04:01:57,500][105620] Updated weights for policy 1, policy_version 1738310 (0.0009) [2023-12-27 04:01:57,557][105620] Updated weights for policy 1, policy_version 1738320 (0.0006) [2023-12-27 04:01:57,574][105692] Updated weights for policy 0, policy_version 1734826 (0.0011) [2023-12-27 04:01:57,635][105692] Updated weights for policy 0, policy_version 1734836 (0.0011) [2023-12-27 04:01:57,686][105692] Updated weights for policy 0, policy_version 1734846 (0.0010) [2023-12-27 04:01:58,286][105620] Updated weights for policy 1, policy_version 1738330 (0.0007) [2023-12-27 04:01:58,351][105620] Updated weights for policy 1, policy_version 1738340 (0.0007) [2023-12-27 04:01:58,413][105620] Updated weights for policy 1, policy_version 1738350 (0.0008) [2023-12-27 04:01:58,466][105692] Updated weights for policy 0, policy_version 1734856 (0.0011) [2023-12-27 04:01:58,530][105692] Updated weights for policy 0, policy_version 1734866 (0.0011) [2023-12-27 04:01:58,605][105692] Updated weights for policy 0, policy_version 1734876 (0.0010) [2023-12-27 04:01:59,162][105620] Updated weights for policy 1, policy_version 1738360 (0.0010) [2023-12-27 04:01:59,226][105620] Updated weights for policy 1, policy_version 1738370 (0.0010) [2023-12-27 04:01:59,292][105620] Updated weights for policy 1, policy_version 1738380 (0.0011) [2023-12-27 04:01:59,440][105692] Updated weights for policy 0, policy_version 1734886 (0.0011) [2023-12-27 04:01:59,495][105692] Updated weights for policy 0, policy_version 1734896 (0.0010) [2023-12-27 04:01:59,550][105692] Updated weights for policy 0, policy_version 1734906 (0.0011) [2023-12-27 04:01:59,921][105620] Updated weights for policy 1, policy_version 1738391 (0.0008) [2023-12-27 04:01:59,984][105620] Updated weights for policy 1, policy_version 1738401 (0.0006) [2023-12-27 04:02:00,042][105620] Updated weights for policy 1, policy_version 1738411 (0.0006) [2023-12-27 04:02:00,332][105692] Updated weights for policy 0, policy_version 1734916 (0.0010) [2023-12-27 04:02:00,396][105692] Updated weights for policy 0, policy_version 1734926 (0.0008) [2023-12-27 04:02:00,455][105692] Updated weights for policy 0, policy_version 1734937 (0.0011) [2023-12-27 04:02:00,605][105620] Updated weights for policy 1, policy_version 1738421 (0.0005) [2023-12-27 04:02:00,662][105620] Updated weights for policy 1, policy_version 1738431 (0.0005) [2023-12-27 04:02:00,719][105620] Updated weights for policy 1, policy_version 1738441 (0.0005) [2023-12-27 04:02:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19438.7). Total num frames: 889315328. Throughput: 0: 9995.3, 1: 9843.7. Samples: 889285168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:02:01,062][104569] Avg episode reward: [(0, '8353.676'), (1, '8896.478')] [2023-12-27 04:02:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001738448_445104128.pth... [2023-12-27 04:02:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001734944_444211200.pth... [2023-12-27 04:02:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001737296_444809216.pth [2023-12-27 04:02:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001733760_443908096.pth [2023-12-27 04:02:01,141][105692] Updated weights for policy 0, policy_version 1734948 (0.0011) [2023-12-27 04:02:01,203][105692] Updated weights for policy 0, policy_version 1734958 (0.0008) [2023-12-27 04:02:01,260][105692] Updated weights for policy 0, policy_version 1734968 (0.0008) [2023-12-27 04:02:01,262][105620] Updated weights for policy 1, policy_version 1738451 (0.0007) [2023-12-27 04:02:01,324][105620] Updated weights for policy 1, policy_version 1738461 (0.0010) [2023-12-27 04:02:01,391][105620] Updated weights for policy 1, policy_version 1738471 (0.0011) [2023-12-27 04:02:02,033][105692] Updated weights for policy 0, policy_version 1734978 (0.0009) [2023-12-27 04:02:02,082][105620] Updated weights for policy 1, policy_version 1738481 (0.0010) [2023-12-27 04:02:02,095][105692] Updated weights for policy 0, policy_version 1734988 (0.0010) [2023-12-27 04:02:02,135][105620] Updated weights for policy 1, policy_version 1738491 (0.0010) [2023-12-27 04:02:02,157][105692] Updated weights for policy 0, policy_version 1734998 (0.0011) [2023-12-27 04:02:02,185][105620] Updated weights for policy 1, policy_version 1738501 (0.0010) [2023-12-27 04:02:02,216][105692] Updated weights for policy 0, policy_version 1735008 (0.0010) [2023-12-27 04:02:02,236][105620] Updated weights for policy 1, policy_version 1738511 (0.0007) [2023-12-27 04:02:02,837][105692] Updated weights for policy 0, policy_version 1735018 (0.0010) [2023-12-27 04:02:02,901][105692] Updated weights for policy 0, policy_version 1735028 (0.0010) [2023-12-27 04:02:02,963][105692] Updated weights for policy 0, policy_version 1735038 (0.0010) [2023-12-27 04:02:03,002][105620] Updated weights for policy 1, policy_version 1738521 (0.0007) [2023-12-27 04:02:03,065][105620] Updated weights for policy 1, policy_version 1738531 (0.0006) [2023-12-27 04:02:03,114][105620] Updated weights for policy 1, policy_version 1738541 (0.0009) [2023-12-27 04:02:03,648][105692] Updated weights for policy 0, policy_version 1735048 (0.0010) [2023-12-27 04:02:03,698][105692] Updated weights for policy 0, policy_version 1735058 (0.0010) [2023-12-27 04:02:03,752][105692] Updated weights for policy 0, policy_version 1735068 (0.0010) [2023-12-27 04:02:03,812][105620] Updated weights for policy 1, policy_version 1738551 (0.0010) [2023-12-27 04:02:03,885][105620] Updated weights for policy 1, policy_version 1738561 (0.0010) [2023-12-27 04:02:03,945][105620] Updated weights for policy 1, policy_version 1738571 (0.0007) [2023-12-27 04:02:04,445][105692] Updated weights for policy 0, policy_version 1735078 (0.0008) [2023-12-27 04:02:04,509][105692] Updated weights for policy 0, policy_version 1735088 (0.0006) [2023-12-27 04:02:04,519][105620] Updated weights for policy 1, policy_version 1738581 (0.0008) [2023-12-27 04:02:04,568][105620] Updated weights for policy 1, policy_version 1738591 (0.0010) [2023-12-27 04:02:04,571][105692] Updated weights for policy 0, policy_version 1735098 (0.0005) [2023-12-27 04:02:04,613][105620] Updated weights for policy 1, policy_version 1738601 (0.0010) [2023-12-27 04:02:05,265][105620] Updated weights for policy 1, policy_version 1738611 (0.0009) [2023-12-27 04:02:05,324][105620] Updated weights for policy 1, policy_version 1738621 (0.0006) [2023-12-27 04:02:05,324][105692] Updated weights for policy 0, policy_version 1735108 (0.0006) [2023-12-27 04:02:05,379][105692] Updated weights for policy 0, policy_version 1735118 (0.0009) [2023-12-27 04:02:05,387][105620] Updated weights for policy 1, policy_version 1738631 (0.0007) [2023-12-27 04:02:05,431][105692] Updated weights for policy 0, policy_version 1735128 (0.0007) [2023-12-27 04:02:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 889413632. Throughput: 0: 10011.1, 1: 9951.9. Samples: 889406228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:02:06,063][104569] Avg episode reward: [(0, '8808.606'), (1, '8991.213')] [2023-12-27 04:02:06,067][105620] Updated weights for policy 1, policy_version 1738641 (0.0009) [2023-12-27 04:02:06,133][105620] Updated weights for policy 1, policy_version 1738651 (0.0009) [2023-12-27 04:02:06,180][105692] Updated weights for policy 0, policy_version 1735138 (0.0009) [2023-12-27 04:02:06,198][105620] Updated weights for policy 1, policy_version 1738661 (0.0007) [2023-12-27 04:02:06,246][105692] Updated weights for policy 0, policy_version 1735148 (0.0008) [2023-12-27 04:02:06,255][105620] Updated weights for policy 1, policy_version 1738671 (0.0005) [2023-12-27 04:02:06,302][105692] Updated weights for policy 0, policy_version 1735158 (0.0010) [2023-12-27 04:02:06,368][105692] Updated weights for policy 0, policy_version 1735168 (0.0009) [2023-12-27 04:02:06,845][105620] Updated weights for policy 1, policy_version 1738681 (0.0005) [2023-12-27 04:02:06,899][105620] Updated weights for policy 1, policy_version 1738691 (0.0005) [2023-12-27 04:02:06,946][105620] Updated weights for policy 1, policy_version 1738701 (0.0005) [2023-12-27 04:02:07,182][105692] Updated weights for policy 0, policy_version 1735178 (0.0006) [2023-12-27 04:02:07,234][105692] Updated weights for policy 0, policy_version 1735188 (0.0005) [2023-12-27 04:02:07,285][105692] Updated weights for policy 0, policy_version 1735198 (0.0005) [2023-12-27 04:02:07,558][105620] Updated weights for policy 1, policy_version 1738711 (0.0006) [2023-12-27 04:02:07,613][105620] Updated weights for policy 1, policy_version 1738721 (0.0005) [2023-12-27 04:02:07,675][105620] Updated weights for policy 1, policy_version 1738731 (0.0005) [2023-12-27 04:02:07,930][105692] Updated weights for policy 0, policy_version 1735208 (0.0009) [2023-12-27 04:02:07,983][105692] Updated weights for policy 0, policy_version 1735218 (0.0010) [2023-12-27 04:02:08,042][105692] Updated weights for policy 0, policy_version 1735229 (0.0010) [2023-12-27 04:02:08,200][105620] Updated weights for policy 1, policy_version 1738741 (0.0007) [2023-12-27 04:02:08,261][105620] Updated weights for policy 1, policy_version 1738751 (0.0005) [2023-12-27 04:02:08,322][105620] Updated weights for policy 1, policy_version 1738761 (0.0006) [2023-12-27 04:02:08,927][105692] Updated weights for policy 0, policy_version 1735239 (0.0009) [2023-12-27 04:02:08,952][105620] Updated weights for policy 1, policy_version 1738771 (0.0009) [2023-12-27 04:02:08,986][105692] Updated weights for policy 0, policy_version 1735249 (0.0007) [2023-12-27 04:02:09,004][105620] Updated weights for policy 1, policy_version 1738781 (0.0010) [2023-12-27 04:02:09,039][105692] Updated weights for policy 0, policy_version 1735259 (0.0005) [2023-12-27 04:02:09,065][105620] Updated weights for policy 1, policy_version 1738791 (0.0011) [2023-12-27 04:02:09,797][105620] Updated weights for policy 1, policy_version 1738801 (0.0010) [2023-12-27 04:02:09,859][105692] Updated weights for policy 0, policy_version 1735269 (0.0006) [2023-12-27 04:02:09,865][105620] Updated weights for policy 1, policy_version 1738811 (0.0008) [2023-12-27 04:02:09,921][105692] Updated weights for policy 0, policy_version 1735279 (0.0007) [2023-12-27 04:02:09,930][105620] Updated weights for policy 1, policy_version 1738821 (0.0008) [2023-12-27 04:02:09,984][105692] Updated weights for policy 0, policy_version 1735289 (0.0008) [2023-12-27 04:02:09,992][105620] Updated weights for policy 1, policy_version 1738831 (0.0008) [2023-12-27 04:02:10,663][105620] Updated weights for policy 1, policy_version 1738841 (0.0010) [2023-12-27 04:02:10,729][105620] Updated weights for policy 1, policy_version 1738851 (0.0007) [2023-12-27 04:02:10,791][105620] Updated weights for policy 1, policy_version 1738861 (0.0010) [2023-12-27 04:02:10,818][105692] Updated weights for policy 0, policy_version 1735299 (0.0008) [2023-12-27 04:02:10,881][105692] Updated weights for policy 0, policy_version 1735309 (0.0008) [2023-12-27 04:02:10,948][105692] Updated weights for policy 0, policy_version 1735319 (0.0009) [2023-12-27 04:02:11,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 889520128. Throughput: 0: 9943.6, 1: 10085.4. Samples: 889524400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:02:11,063][104569] Avg episode reward: [(0, '8894.880'), (1, '9081.979')] [2023-12-27 04:02:11,493][105620] Updated weights for policy 1, policy_version 1738871 (0.0010) [2023-12-27 04:02:11,549][105620] Updated weights for policy 1, policy_version 1738881 (0.0009) [2023-12-27 04:02:11,598][105620] Updated weights for policy 1, policy_version 1738891 (0.0009) [2023-12-27 04:02:11,719][105692] Updated weights for policy 0, policy_version 1735329 (0.0009) [2023-12-27 04:02:11,787][105692] Updated weights for policy 0, policy_version 1735339 (0.0008) [2023-12-27 04:02:11,852][105692] Updated weights for policy 0, policy_version 1735349 (0.0009) [2023-12-27 04:02:11,913][105692] Updated weights for policy 0, policy_version 1735359 (0.0009) [2023-12-27 04:02:12,393][105620] Updated weights for policy 1, policy_version 1738901 (0.0008) [2023-12-27 04:02:12,454][105620] Updated weights for policy 1, policy_version 1738911 (0.0008) [2023-12-27 04:02:12,517][105620] Updated weights for policy 1, policy_version 1738921 (0.0005) [2023-12-27 04:02:12,718][105692] Updated weights for policy 0, policy_version 1735369 (0.0007) [2023-12-27 04:02:12,779][105692] Updated weights for policy 0, policy_version 1735379 (0.0008) [2023-12-27 04:02:12,843][105692] Updated weights for policy 0, policy_version 1735389 (0.0008) [2023-12-27 04:02:13,175][105620] Updated weights for policy 1, policy_version 1738931 (0.0007) [2023-12-27 04:02:13,235][105620] Updated weights for policy 1, policy_version 1738941 (0.0007) [2023-12-27 04:02:13,289][105620] Updated weights for policy 1, policy_version 1738951 (0.0006) [2023-12-27 04:02:13,680][105692] Updated weights for policy 0, policy_version 1735399 (0.0010) [2023-12-27 04:02:13,745][105692] Updated weights for policy 0, policy_version 1735409 (0.0010) [2023-12-27 04:02:13,810][105692] Updated weights for policy 0, policy_version 1735419 (0.0009) [2023-12-27 04:02:13,817][105620] Updated weights for policy 1, policy_version 1738961 (0.0005) [2023-12-27 04:02:13,877][105620] Updated weights for policy 1, policy_version 1738971 (0.0005) [2023-12-27 04:02:13,940][105620] Updated weights for policy 1, policy_version 1738981 (0.0005) [2023-12-27 04:02:14,002][105620] Updated weights for policy 1, policy_version 1738991 (0.0009) [2023-12-27 04:02:14,579][105692] Updated weights for policy 0, policy_version 1735429 (0.0010) [2023-12-27 04:02:14,633][105692] Updated weights for policy 0, policy_version 1735439 (0.0010) [2023-12-27 04:02:14,637][105620] Updated weights for policy 1, policy_version 1739001 (0.0006) [2023-12-27 04:02:14,682][105692] Updated weights for policy 0, policy_version 1735449 (0.0009) [2023-12-27 04:02:14,724][105620] Updated weights for policy 1, policy_version 1739011 (0.0005) [2023-12-27 04:02:14,792][105620] Updated weights for policy 1, policy_version 1739021 (0.0007) [2023-12-27 04:02:15,354][105620] Updated weights for policy 1, policy_version 1739031 (0.0009) [2023-12-27 04:02:15,416][105620] Updated weights for policy 1, policy_version 1739041 (0.0011) [2023-12-27 04:02:15,476][105620] Updated weights for policy 1, policy_version 1739051 (0.0011) [2023-12-27 04:02:15,510][105692] Updated weights for policy 0, policy_version 1735459 (0.0010) [2023-12-27 04:02:15,574][105692] Updated weights for policy 0, policy_version 1735469 (0.0007) [2023-12-27 04:02:15,643][105692] Updated weights for policy 0, policy_version 1735479 (0.0006) [2023-12-27 04:02:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 889610240. Throughput: 0: 9851.3, 1: 10104.2. Samples: 889581548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:02:16,063][104569] Avg episode reward: [(0, '8624.981'), (1, '9079.448')] [2023-12-27 04:02:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001735488_444350464.pth... [2023-12-27 04:02:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001739056_445259776.pth... [2023-12-27 04:02:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001734368_444063744.pth [2023-12-27 04:02:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001737872_444956672.pth [2023-12-27 04:02:16,127][105620] Updated weights for policy 1, policy_version 1739061 (0.0008) [2023-12-27 04:02:16,182][105620] Updated weights for policy 1, policy_version 1739071 (0.0007) [2023-12-27 04:02:16,240][105620] Updated weights for policy 1, policy_version 1739081 (0.0011) [2023-12-27 04:02:16,284][105692] Updated weights for policy 0, policy_version 1735489 (0.0006) [2023-12-27 04:02:16,328][105692] Updated weights for policy 0, policy_version 1735499 (0.0008) [2023-12-27 04:02:16,392][105692] Updated weights for policy 0, policy_version 1735509 (0.0009) [2023-12-27 04:02:16,463][105692] Updated weights for policy 0, policy_version 1735519 (0.0006) [2023-12-27 04:02:16,870][105620] Updated weights for policy 1, policy_version 1739091 (0.0009) [2023-12-27 04:02:16,925][105620] Updated weights for policy 1, policy_version 1739101 (0.0005) [2023-12-27 04:02:16,993][105620] Updated weights for policy 1, policy_version 1739111 (0.0006) [2023-12-27 04:02:17,076][105692] Updated weights for policy 0, policy_version 1735529 (0.0010) [2023-12-27 04:02:17,128][105692] Updated weights for policy 0, policy_version 1735539 (0.0010) [2023-12-27 04:02:17,181][105692] Updated weights for policy 0, policy_version 1735549 (0.0010) [2023-12-27 04:02:17,674][105620] Updated weights for policy 1, policy_version 1739121 (0.0008) [2023-12-27 04:02:17,742][105620] Updated weights for policy 1, policy_version 1739131 (0.0008) [2023-12-27 04:02:17,801][105620] Updated weights for policy 1, policy_version 1739141 (0.0008) [2023-12-27 04:02:17,853][105620] Updated weights for policy 1, policy_version 1739151 (0.0008) [2023-12-27 04:02:17,923][105692] Updated weights for policy 0, policy_version 1735559 (0.0010) [2023-12-27 04:02:17,971][105692] Updated weights for policy 0, policy_version 1735569 (0.0010) [2023-12-27 04:02:18,025][105692] Updated weights for policy 0, policy_version 1735579 (0.0010) [2023-12-27 04:02:18,645][105692] Updated weights for policy 0, policy_version 1735589 (0.0008) [2023-12-27 04:02:18,664][105620] Updated weights for policy 1, policy_version 1739161 (0.0006) [2023-12-27 04:02:18,712][105692] Updated weights for policy 0, policy_version 1735599 (0.0009) [2023-12-27 04:02:18,732][105620] Updated weights for policy 1, policy_version 1739171 (0.0005) [2023-12-27 04:02:18,781][105692] Updated weights for policy 0, policy_version 1735609 (0.0010) [2023-12-27 04:02:18,805][105620] Updated weights for policy 1, policy_version 1739181 (0.0006) [2023-12-27 04:02:19,441][105620] Updated weights for policy 1, policy_version 1739191 (0.0007) [2023-12-27 04:02:19,499][105620] Updated weights for policy 1, policy_version 1739201 (0.0009) [2023-12-27 04:02:19,544][105692] Updated weights for policy 0, policy_version 1735619 (0.0011) [2023-12-27 04:02:19,554][105620] Updated weights for policy 1, policy_version 1739211 (0.0007) [2023-12-27 04:02:19,607][105692] Updated weights for policy 0, policy_version 1735629 (0.0010) [2023-12-27 04:02:19,664][105692] Updated weights for policy 0, policy_version 1735639 (0.0009) [2023-12-27 04:02:20,321][105620] Updated weights for policy 1, policy_version 1739221 (0.0007) [2023-12-27 04:02:20,382][105620] Updated weights for policy 1, policy_version 1739231 (0.0009) [2023-12-27 04:02:20,385][105692] Updated weights for policy 0, policy_version 1735649 (0.0009) [2023-12-27 04:02:20,441][105620] Updated weights for policy 1, policy_version 1739241 (0.0008) [2023-12-27 04:02:20,447][105692] Updated weights for policy 0, policy_version 1735659 (0.0007) [2023-12-27 04:02:20,502][105692] Updated weights for policy 0, policy_version 1735669 (0.0005) [2023-12-27 04:02:20,561][105692] Updated weights for policy 0, policy_version 1735679 (0.0009) [2023-12-27 04:02:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19933.8, 300 sec: 19466.4). Total num frames: 889708544. Throughput: 0: 9744.9, 1: 10174.5. Samples: 889701376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:02:21,063][104569] Avg episode reward: [(0, '8535.624'), (1, '9171.654')] [2023-12-27 04:02:21,204][105620] Updated weights for policy 1, policy_version 1739251 (0.0008) [2023-12-27 04:02:21,266][105620] Updated weights for policy 1, policy_version 1739261 (0.0008) [2023-12-27 04:02:21,333][105620] Updated weights for policy 1, policy_version 1739271 (0.0008) [2023-12-27 04:02:21,347][105692] Updated weights for policy 0, policy_version 1735689 (0.0008) [2023-12-27 04:02:21,408][105692] Updated weights for policy 0, policy_version 1735699 (0.0007) [2023-12-27 04:02:21,472][105692] Updated weights for policy 0, policy_version 1735709 (0.0005) [2023-12-27 04:02:22,108][105620] Updated weights for policy 1, policy_version 1739281 (0.0008) [2023-12-27 04:02:22,169][105620] Updated weights for policy 1, policy_version 1739291 (0.0009) [2023-12-27 04:02:22,189][105692] Updated weights for policy 0, policy_version 1735719 (0.0006) [2023-12-27 04:02:22,226][105620] Updated weights for policy 1, policy_version 1739301 (0.0008) [2023-12-27 04:02:22,254][105692] Updated weights for policy 0, policy_version 1735729 (0.0006) [2023-12-27 04:02:22,290][105620] Updated weights for policy 1, policy_version 1739311 (0.0007) [2023-12-27 04:02:22,318][105692] Updated weights for policy 0, policy_version 1735739 (0.0009) [2023-12-27 04:02:23,014][105620] Updated weights for policy 1, policy_version 1739321 (0.0007) [2023-12-27 04:02:23,067][105692] Updated weights for policy 0, policy_version 1735749 (0.0008) [2023-12-27 04:02:23,070][105620] Updated weights for policy 1, policy_version 1739331 (0.0007) [2023-12-27 04:02:23,114][105692] Updated weights for policy 0, policy_version 1735759 (0.0006) [2023-12-27 04:02:23,129][105620] Updated weights for policy 1, policy_version 1739341 (0.0007) [2023-12-27 04:02:23,162][105692] Updated weights for policy 0, policy_version 1735769 (0.0008) [2023-12-27 04:02:23,765][105620] Updated weights for policy 1, policy_version 1739351 (0.0010) [2023-12-27 04:02:23,819][105620] Updated weights for policy 1, policy_version 1739361 (0.0010) [2023-12-27 04:02:23,878][105620] Updated weights for policy 1, policy_version 1739371 (0.0009) [2023-12-27 04:02:23,999][105692] Updated weights for policy 0, policy_version 1735779 (0.0009) [2023-12-27 04:02:24,043][105692] Updated weights for policy 0, policy_version 1735789 (0.0008) [2023-12-27 04:02:24,096][105692] Updated weights for policy 0, policy_version 1735799 (0.0009) [2023-12-27 04:02:24,602][105620] Updated weights for policy 1, policy_version 1739381 (0.0009) [2023-12-27 04:02:24,660][105620] Updated weights for policy 1, policy_version 1739391 (0.0008) [2023-12-27 04:02:24,713][105620] Updated weights for policy 1, policy_version 1739401 (0.0006) [2023-12-27 04:02:24,845][105692] Updated weights for policy 0, policy_version 1735809 (0.0009) [2023-12-27 04:02:24,907][105692] Updated weights for policy 0, policy_version 1735819 (0.0011) [2023-12-27 04:02:24,959][105692] Updated weights for policy 0, policy_version 1735829 (0.0010) [2023-12-27 04:02:25,011][105692] Updated weights for policy 0, policy_version 1735839 (0.0011) [2023-12-27 04:02:25,394][105620] Updated weights for policy 1, policy_version 1739411 (0.0007) [2023-12-27 04:02:25,461][105620] Updated weights for policy 1, policy_version 1739421 (0.0010) [2023-12-27 04:02:25,518][105620] Updated weights for policy 1, policy_version 1739431 (0.0010) [2023-12-27 04:02:25,691][105692] Updated weights for policy 0, policy_version 1735849 (0.0010) [2023-12-27 04:02:25,736][105692] Updated weights for policy 0, policy_version 1735859 (0.0010) [2023-12-27 04:02:25,797][105692] Updated weights for policy 0, policy_version 1735869 (0.0011) [2023-12-27 04:02:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19933.9, 300 sec: 19466.4). Total num frames: 889806848. Throughput: 0: 9609.8, 1: 10113.3. Samples: 889815348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:02:26,062][104569] Avg episode reward: [(0, '8718.983'), (1, '8895.228')] [2023-12-27 04:02:26,259][105620] Updated weights for policy 1, policy_version 1739441 (0.0010) [2023-12-27 04:02:26,321][105620] Updated weights for policy 1, policy_version 1739451 (0.0010) [2023-12-27 04:02:26,383][105620] Updated weights for policy 1, policy_version 1739461 (0.0010) [2023-12-27 04:02:26,431][105692] Updated weights for policy 0, policy_version 1735879 (0.0009) [2023-12-27 04:02:26,445][105620] Updated weights for policy 1, policy_version 1739471 (0.0010) [2023-12-27 04:02:26,480][105692] Updated weights for policy 0, policy_version 1735889 (0.0010) [2023-12-27 04:02:26,528][105692] Updated weights for policy 0, policy_version 1735899 (0.0009) [2023-12-27 04:02:27,075][105620] Updated weights for policy 1, policy_version 1739481 (0.0006) [2023-12-27 04:02:27,134][105620] Updated weights for policy 1, policy_version 1739491 (0.0007) [2023-12-27 04:02:27,201][105620] Updated weights for policy 1, policy_version 1739501 (0.0010) [2023-12-27 04:02:27,265][105692] Updated weights for policy 0, policy_version 1735909 (0.0010) [2023-12-27 04:02:27,324][105692] Updated weights for policy 0, policy_version 1735919 (0.0010) [2023-12-27 04:02:27,372][105692] Updated weights for policy 0, policy_version 1735929 (0.0010) [2023-12-27 04:02:27,813][105620] Updated weights for policy 1, policy_version 1739511 (0.0010) [2023-12-27 04:02:27,868][105620] Updated weights for policy 1, policy_version 1739521 (0.0010) [2023-12-27 04:02:27,920][105620] Updated weights for policy 1, policy_version 1739531 (0.0009) [2023-12-27 04:02:27,996][105692] Updated weights for policy 0, policy_version 1735939 (0.0010) [2023-12-27 04:02:28,053][105692] Updated weights for policy 0, policy_version 1735949 (0.0011) [2023-12-27 04:02:28,109][105692] Updated weights for policy 0, policy_version 1735959 (0.0011) [2023-12-27 04:02:28,631][105620] Updated weights for policy 1, policy_version 1739541 (0.0007) [2023-12-27 04:02:28,697][105620] Updated weights for policy 1, policy_version 1739551 (0.0005) [2023-12-27 04:02:28,758][105620] Updated weights for policy 1, policy_version 1739561 (0.0006) [2023-12-27 04:02:28,888][105692] Updated weights for policy 0, policy_version 1735969 (0.0010) [2023-12-27 04:02:28,943][105692] Updated weights for policy 0, policy_version 1735979 (0.0009) [2023-12-27 04:02:28,997][105692] Updated weights for policy 0, policy_version 1735989 (0.0010) [2023-12-27 04:02:29,050][105692] Updated weights for policy 0, policy_version 1735999 (0.0010) [2023-12-27 04:02:29,405][105620] Updated weights for policy 1, policy_version 1739571 (0.0007) [2023-12-27 04:02:29,464][105620] Updated weights for policy 1, policy_version 1739581 (0.0009) [2023-12-27 04:02:29,533][105620] Updated weights for policy 1, policy_version 1739591 (0.0008) [2023-12-27 04:02:29,778][105692] Updated weights for policy 0, policy_version 1736009 (0.0006) [2023-12-27 04:02:29,845][105692] Updated weights for policy 0, policy_version 1736019 (0.0007) [2023-12-27 04:02:29,898][105692] Updated weights for policy 0, policy_version 1736029 (0.0006) [2023-12-27 04:02:30,279][105620] Updated weights for policy 1, policy_version 1739601 (0.0009) [2023-12-27 04:02:30,334][105620] Updated weights for policy 1, policy_version 1739611 (0.0006) [2023-12-27 04:02:30,389][105620] Updated weights for policy 1, policy_version 1739621 (0.0005) [2023-12-27 04:02:30,447][105620] Updated weights for policy 1, policy_version 1739631 (0.0005) [2023-12-27 04:02:30,699][105692] Updated weights for policy 0, policy_version 1736039 (0.0010) [2023-12-27 04:02:30,743][105692] Updated weights for policy 0, policy_version 1736049 (0.0007) [2023-12-27 04:02:30,787][105692] Updated weights for policy 0, policy_version 1736059 (0.0008) [2023-12-27 04:02:31,021][105620] Updated weights for policy 1, policy_version 1739641 (0.0010) [2023-12-27 04:02:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19933.9, 300 sec: 19494.2). Total num frames: 889905152. Throughput: 0: 9654.1, 1: 10182.7. Samples: 889876652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:02:31,063][104569] Avg episode reward: [(0, '8809.176'), (1, '8893.694')] [2023-12-27 04:02:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001736064_444497920.pth... [2023-12-27 04:02:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001734944_444211200.pth [2023-12-27 04:02:31,083][105620] Updated weights for policy 1, policy_version 1739651 (0.0008) [2023-12-27 04:02:31,155][105620] Updated weights for policy 1, policy_version 1739661 (0.0007) [2023-12-27 04:02:31,173][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001739664_445415424.pth... [2023-12-27 04:02:31,176][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001738448_445104128.pth [2023-12-27 04:02:31,595][105692] Updated weights for policy 0, policy_version 1736069 (0.0008) [2023-12-27 04:02:31,657][105692] Updated weights for policy 0, policy_version 1736079 (0.0008) [2023-12-27 04:02:31,723][105692] Updated weights for policy 0, policy_version 1736089 (0.0008) [2023-12-27 04:02:31,962][105620] Updated weights for policy 1, policy_version 1739671 (0.0007) [2023-12-27 04:02:32,029][105620] Updated weights for policy 1, policy_version 1739681 (0.0005) [2023-12-27 04:02:32,094][105620] Updated weights for policy 1, policy_version 1739691 (0.0005) [2023-12-27 04:02:32,430][105692] Updated weights for policy 0, policy_version 1736099 (0.0009) [2023-12-27 04:02:32,486][105692] Updated weights for policy 0, policy_version 1736109 (0.0009) [2023-12-27 04:02:32,540][105692] Updated weights for policy 0, policy_version 1736119 (0.0009) [2023-12-27 04:02:32,728][105620] Updated weights for policy 1, policy_version 1739701 (0.0007) [2023-12-27 04:02:32,780][105620] Updated weights for policy 1, policy_version 1739711 (0.0005) [2023-12-27 04:02:32,828][105620] Updated weights for policy 1, policy_version 1739721 (0.0007) [2023-12-27 04:02:33,350][105620] Updated weights for policy 1, policy_version 1739731 (0.0005) [2023-12-27 04:02:33,410][105620] Updated weights for policy 1, policy_version 1739741 (0.0005) [2023-12-27 04:02:33,444][105692] Updated weights for policy 0, policy_version 1736129 (0.0009) [2023-12-27 04:02:33,461][105620] Updated weights for policy 1, policy_version 1739751 (0.0005) [2023-12-27 04:02:33,491][105692] Updated weights for policy 0, policy_version 1736139 (0.0009) [2023-12-27 04:02:33,541][105692] Updated weights for policy 0, policy_version 1736149 (0.0009) [2023-12-27 04:02:34,068][105620] Updated weights for policy 1, policy_version 1739761 (0.0006) [2023-12-27 04:02:34,124][105620] Updated weights for policy 1, policy_version 1739771 (0.0005) [2023-12-27 04:02:34,183][105620] Updated weights for policy 1, policy_version 1739781 (0.0008) [2023-12-27 04:02:34,239][105620] Updated weights for policy 1, policy_version 1739791 (0.0007) [2023-12-27 04:02:34,380][105692] Updated weights for policy 0, policy_version 1736161 (0.0010) [2023-12-27 04:02:34,447][105692] Updated weights for policy 0, policy_version 1736171 (0.0007) [2023-12-27 04:02:34,506][105692] Updated weights for policy 0, policy_version 1736181 (0.0009) [2023-12-27 04:02:34,563][105692] Updated weights for policy 0, policy_version 1736191 (0.0009) [2023-12-27 04:02:34,912][105620] Updated weights for policy 1, policy_version 1739801 (0.0006) [2023-12-27 04:02:34,963][105620] Updated weights for policy 1, policy_version 1739811 (0.0005) [2023-12-27 04:02:35,015][105620] Updated weights for policy 1, policy_version 1739821 (0.0005) [2023-12-27 04:02:35,439][105692] Updated weights for policy 0, policy_version 1736201 (0.0009) [2023-12-27 04:02:35,505][105692] Updated weights for policy 0, policy_version 1736211 (0.0009) [2023-12-27 04:02:35,524][105620] Updated weights for policy 1, policy_version 1739831 (0.0009) [2023-12-27 04:02:35,554][105692] Updated weights for policy 0, policy_version 1736221 (0.0005) [2023-12-27 04:02:35,575][105620] Updated weights for policy 1, policy_version 1739841 (0.0010) [2023-12-27 04:02:35,623][105620] Updated weights for policy 1, policy_version 1739851 (0.0010) [2023-12-27 04:02:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.4, 300 sec: 19494.2). Total num frames: 890003456. Throughput: 0: 9497.9, 1: 10227.6. Samples: 889993668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:02:36,062][104569] Avg episode reward: [(0, '8443.752'), (1, '9079.256')] [2023-12-27 04:02:36,330][105692] Updated weights for policy 0, policy_version 1736231 (0.0007) [2023-12-27 04:02:36,380][105620] Updated weights for policy 1, policy_version 1739861 (0.0010) [2023-12-27 04:02:36,387][105692] Updated weights for policy 0, policy_version 1736241 (0.0007) [2023-12-27 04:02:36,425][105620] Updated weights for policy 1, policy_version 1739871 (0.0010) [2023-12-27 04:02:36,444][105692] Updated weights for policy 0, policy_version 1736251 (0.0006) [2023-12-27 04:02:36,475][105620] Updated weights for policy 1, policy_version 1739881 (0.0010) [2023-12-27 04:02:37,185][105692] Updated weights for policy 0, policy_version 1736261 (0.0006) [2023-12-27 04:02:37,242][105692] Updated weights for policy 0, policy_version 1736271 (0.0006) [2023-12-27 04:02:37,248][105620] Updated weights for policy 1, policy_version 1739891 (0.0010) [2023-12-27 04:02:37,301][105692] Updated weights for policy 0, policy_version 1736281 (0.0006) [2023-12-27 04:02:37,303][105620] Updated weights for policy 1, policy_version 1739901 (0.0010) [2023-12-27 04:02:37,361][105620] Updated weights for policy 1, policy_version 1739911 (0.0010) [2023-12-27 04:02:38,050][105620] Updated weights for policy 1, policy_version 1739921 (0.0010) [2023-12-27 04:02:38,080][105692] Updated weights for policy 0, policy_version 1736291 (0.0008) [2023-12-27 04:02:38,114][105620] Updated weights for policy 1, policy_version 1739931 (0.0005) [2023-12-27 04:02:38,143][105692] Updated weights for policy 0, policy_version 1736301 (0.0011) [2023-12-27 04:02:38,177][105620] Updated weights for policy 1, policy_version 1739941 (0.0006) [2023-12-27 04:02:38,199][105692] Updated weights for policy 0, policy_version 1736311 (0.0006) [2023-12-27 04:02:38,240][105620] Updated weights for policy 1, policy_version 1739951 (0.0010) [2023-12-27 04:02:38,899][105692] Updated weights for policy 0, policy_version 1736321 (0.0010) [2023-12-27 04:02:38,933][105620] Updated weights for policy 1, policy_version 1739961 (0.0010) [2023-12-27 04:02:38,954][105692] Updated weights for policy 0, policy_version 1736331 (0.0010) [2023-12-27 04:02:38,991][105620] Updated weights for policy 1, policy_version 1739971 (0.0010) [2023-12-27 04:02:39,016][105692] Updated weights for policy 0, policy_version 1736341 (0.0010) [2023-12-27 04:02:39,039][105620] Updated weights for policy 1, policy_version 1739981 (0.0010) [2023-12-27 04:02:39,072][105692] Updated weights for policy 0, policy_version 1736351 (0.0009) [2023-12-27 04:02:39,741][105692] Updated weights for policy 0, policy_version 1736361 (0.0008) [2023-12-27 04:02:39,801][105692] Updated weights for policy 0, policy_version 1736371 (0.0008) [2023-12-27 04:02:39,825][105620] Updated weights for policy 1, policy_version 1739991 (0.0008) [2023-12-27 04:02:39,866][105692] Updated weights for policy 0, policy_version 1736381 (0.0009) [2023-12-27 04:02:39,890][105620] Updated weights for policy 1, policy_version 1740001 (0.0007) [2023-12-27 04:02:39,960][105620] Updated weights for policy 1, policy_version 1740011 (0.0008) [2023-12-27 04:02:40,634][105620] Updated weights for policy 1, policy_version 1740021 (0.0008) [2023-12-27 04:02:40,655][105692] Updated weights for policy 0, policy_version 1736391 (0.0008) [2023-12-27 04:02:40,691][105620] Updated weights for policy 1, policy_version 1740031 (0.0007) [2023-12-27 04:02:40,710][105692] Updated weights for policy 0, policy_version 1736401 (0.0005) [2023-12-27 04:02:40,744][105620] Updated weights for policy 1, policy_version 1740041 (0.0005) [2023-12-27 04:02:40,775][105692] Updated weights for policy 0, policy_version 1736411 (0.0005) [2023-12-27 04:02:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 890101760. Throughput: 0: 9414.8, 1: 10250.3. Samples: 890108940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:02:41,063][104569] Avg episode reward: [(0, '8353.020'), (1, '9171.235')] [2023-12-27 04:02:41,406][105620] Updated weights for policy 1, policy_version 1740051 (0.0006) [2023-12-27 04:02:41,456][105620] Updated weights for policy 1, policy_version 1740061 (0.0009) [2023-12-27 04:02:41,493][105692] Updated weights for policy 0, policy_version 1736421 (0.0008) [2023-12-27 04:02:41,503][105620] Updated weights for policy 1, policy_version 1740071 (0.0007) [2023-12-27 04:02:41,544][105692] Updated weights for policy 0, policy_version 1736431 (0.0009) [2023-12-27 04:02:41,603][105692] Updated weights for policy 0, policy_version 1736441 (0.0008) [2023-12-27 04:02:42,143][105620] Updated weights for policy 1, policy_version 1740081 (0.0007) [2023-12-27 04:02:42,197][105620] Updated weights for policy 1, policy_version 1740091 (0.0008) [2023-12-27 04:02:42,245][105620] Updated weights for policy 1, policy_version 1740101 (0.0009) [2023-12-27 04:02:42,308][105620] Updated weights for policy 1, policy_version 1740111 (0.0008) [2023-12-27 04:02:42,455][105692] Updated weights for policy 0, policy_version 1736451 (0.0009) [2023-12-27 04:02:42,508][105692] Updated weights for policy 0, policy_version 1736461 (0.0010) [2023-12-27 04:02:42,563][105692] Updated weights for policy 0, policy_version 1736471 (0.0010) [2023-12-27 04:02:43,051][105620] Updated weights for policy 1, policy_version 1740121 (0.0009) [2023-12-27 04:02:43,097][105620] Updated weights for policy 1, policy_version 1740131 (0.0005) [2023-12-27 04:02:43,145][105620] Updated weights for policy 1, policy_version 1740141 (0.0006) [2023-12-27 04:02:43,264][105692] Updated weights for policy 0, policy_version 1736481 (0.0006) [2023-12-27 04:02:43,326][105692] Updated weights for policy 0, policy_version 1736491 (0.0009) [2023-12-27 04:02:43,383][105692] Updated weights for policy 0, policy_version 1736501 (0.0009) [2023-12-27 04:02:43,439][105692] Updated weights for policy 0, policy_version 1736511 (0.0008) [2023-12-27 04:02:43,850][105620] Updated weights for policy 1, policy_version 1740151 (0.0008) [2023-12-27 04:02:43,898][105620] Updated weights for policy 1, policy_version 1740161 (0.0009) [2023-12-27 04:02:43,947][105620] Updated weights for policy 1, policy_version 1740171 (0.0009) [2023-12-27 04:02:44,185][105692] Updated weights for policy 0, policy_version 1736521 (0.0009) [2023-12-27 04:02:44,240][105692] Updated weights for policy 0, policy_version 1736531 (0.0009) [2023-12-27 04:02:44,286][105692] Updated weights for policy 0, policy_version 1736541 (0.0008) [2023-12-27 04:02:44,681][105620] Updated weights for policy 1, policy_version 1740181 (0.0007) [2023-12-27 04:02:44,747][105620] Updated weights for policy 1, policy_version 1740191 (0.0007) [2023-12-27 04:02:44,808][105620] Updated weights for policy 1, policy_version 1740201 (0.0009) [2023-12-27 04:02:45,124][105692] Updated weights for policy 0, policy_version 1736551 (0.0009) [2023-12-27 04:02:45,179][105692] Updated weights for policy 0, policy_version 1736561 (0.0009) [2023-12-27 04:02:45,243][105692] Updated weights for policy 0, policy_version 1736571 (0.0010) [2023-12-27 04:02:45,434][105620] Updated weights for policy 1, policy_version 1740211 (0.0007) [2023-12-27 04:02:45,493][105620] Updated weights for policy 1, policy_version 1740221 (0.0009) [2023-12-27 04:02:45,548][105620] Updated weights for policy 1, policy_version 1740231 (0.0009) [2023-12-27 04:02:45,992][105692] Updated weights for policy 0, policy_version 1736581 (0.0007) [2023-12-27 04:02:46,042][105692] Updated weights for policy 0, policy_version 1736591 (0.0005) [2023-12-27 04:02:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 890191872. Throughput: 0: 9320.8, 1: 10251.8. Samples: 890165936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:02:46,063][104569] Avg episode reward: [(0, '9080.167'), (1, '8895.447')] [2023-12-27 04:02:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001740240_445562880.pth... [2023-12-27 04:02:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001739056_445259776.pth [2023-12-27 04:02:46,090][105692] Updated weights for policy 0, policy_version 1736601 (0.0008) [2023-12-27 04:02:46,122][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001736608_444637184.pth... [2023-12-27 04:02:46,125][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001735488_444350464.pth [2023-12-27 04:02:46,212][105620] Updated weights for policy 1, policy_version 1740241 (0.0008) [2023-12-27 04:02:46,276][105620] Updated weights for policy 1, policy_version 1740251 (0.0008) [2023-12-27 04:02:46,341][105620] Updated weights for policy 1, policy_version 1740261 (0.0009) [2023-12-27 04:02:46,397][105620] Updated weights for policy 1, policy_version 1740271 (0.0008) [2023-12-27 04:02:46,710][105692] Updated weights for policy 0, policy_version 1736611 (0.0009) [2023-12-27 04:02:46,772][105692] Updated weights for policy 0, policy_version 1736621 (0.0008) [2023-12-27 04:02:46,833][105692] Updated weights for policy 0, policy_version 1736631 (0.0006) [2023-12-27 04:02:47,142][105620] Updated weights for policy 1, policy_version 1740281 (0.0009) [2023-12-27 04:02:47,201][105620] Updated weights for policy 1, policy_version 1740291 (0.0010) [2023-12-27 04:02:47,258][105620] Updated weights for policy 1, policy_version 1740301 (0.0009) [2023-12-27 04:02:47,391][105692] Updated weights for policy 0, policy_version 1736641 (0.0009) [2023-12-27 04:02:47,449][105692] Updated weights for policy 0, policy_version 1736651 (0.0005) [2023-12-27 04:02:47,501][105692] Updated weights for policy 0, policy_version 1736661 (0.0005) [2023-12-27 04:02:47,557][105692] Updated weights for policy 0, policy_version 1736671 (0.0005) [2023-12-27 04:02:48,115][105692] Updated weights for policy 0, policy_version 1736681 (0.0006) [2023-12-27 04:02:48,159][105620] Updated weights for policy 1, policy_version 1740311 (0.0009) [2023-12-27 04:02:48,171][105692] Updated weights for policy 0, policy_version 1736691 (0.0005) [2023-12-27 04:02:48,219][105620] Updated weights for policy 1, policy_version 1740321 (0.0008) [2023-12-27 04:02:48,229][105692] Updated weights for policy 0, policy_version 1736701 (0.0005) [2023-12-27 04:02:48,281][105620] Updated weights for policy 1, policy_version 1740331 (0.0009) [2023-12-27 04:02:48,836][105692] Updated weights for policy 0, policy_version 1736711 (0.0008) [2023-12-27 04:02:48,890][105692] Updated weights for policy 0, policy_version 1736721 (0.0009) [2023-12-27 04:02:48,945][105692] Updated weights for policy 0, policy_version 1736731 (0.0009) [2023-12-27 04:02:49,071][105620] Updated weights for policy 1, policy_version 1740342 (0.0010) [2023-12-27 04:02:49,125][105620] Updated weights for policy 1, policy_version 1740353 (0.0010) [2023-12-27 04:02:49,177][105620] Updated weights for policy 1, policy_version 1740364 (0.0010) [2023-12-27 04:02:49,619][105692] Updated weights for policy 0, policy_version 1736741 (0.0010) [2023-12-27 04:02:49,675][105692] Updated weights for policy 0, policy_version 1736751 (0.0008) [2023-12-27 04:02:49,735][105692] Updated weights for policy 0, policy_version 1736761 (0.0008) [2023-12-27 04:02:49,959][105620] Updated weights for policy 1, policy_version 1740375 (0.0009) [2023-12-27 04:02:50,013][105620] Updated weights for policy 1, policy_version 1740385 (0.0008) [2023-12-27 04:02:50,067][105620] Updated weights for policy 1, policy_version 1740395 (0.0010) [2023-12-27 04:02:50,402][105692] Updated weights for policy 0, policy_version 1736771 (0.0009) [2023-12-27 04:02:50,465][105692] Updated weights for policy 0, policy_version 1736781 (0.0006) [2023-12-27 04:02:50,531][105692] Updated weights for policy 0, policy_version 1736791 (0.0008) [2023-12-27 04:02:50,900][105620] Updated weights for policy 1, policy_version 1740405 (0.0009) [2023-12-27 04:02:50,961][105620] Updated weights for policy 1, policy_version 1740415 (0.0008) [2023-12-27 04:02:51,015][105620] Updated weights for policy 1, policy_version 1740425 (0.0006) [2023-12-27 04:02:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 890290176. Throughput: 0: 9422.9, 1: 10109.7. Samples: 890285192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:02:51,063][104569] Avg episode reward: [(0, '8993.478'), (1, '8712.207')] [2023-12-27 04:02:51,265][105692] Updated weights for policy 0, policy_version 1736801 (0.0010) [2023-12-27 04:02:51,329][105692] Updated weights for policy 0, policy_version 1736811 (0.0007) [2023-12-27 04:02:51,393][105692] Updated weights for policy 0, policy_version 1736821 (0.0007) [2023-12-27 04:02:51,451][105692] Updated weights for policy 0, policy_version 1736831 (0.0008) [2023-12-27 04:02:51,649][105620] Updated weights for policy 1, policy_version 1740435 (0.0007) [2023-12-27 04:02:51,709][105620] Updated weights for policy 1, policy_version 1740445 (0.0010) [2023-12-27 04:02:51,773][105620] Updated weights for policy 1, policy_version 1740455 (0.0009) [2023-12-27 04:02:52,152][105692] Updated weights for policy 0, policy_version 1736841 (0.0009) [2023-12-27 04:02:52,206][105692] Updated weights for policy 0, policy_version 1736851 (0.0007) [2023-12-27 04:02:52,260][105692] Updated weights for policy 0, policy_version 1736861 (0.0008) [2023-12-27 04:02:52,598][105620] Updated weights for policy 1, policy_version 1740465 (0.0009) [2023-12-27 04:02:52,661][105620] Updated weights for policy 1, policy_version 1740475 (0.0009) [2023-12-27 04:02:52,715][105620] Updated weights for policy 1, policy_version 1740485 (0.0009) [2023-12-27 04:02:52,768][105620] Updated weights for policy 1, policy_version 1740495 (0.0010) [2023-12-27 04:02:52,892][105692] Updated weights for policy 0, policy_version 1736871 (0.0009) [2023-12-27 04:02:52,950][105692] Updated weights for policy 0, policy_version 1736881 (0.0009) [2023-12-27 04:02:53,009][105692] Updated weights for policy 0, policy_version 1736891 (0.0009) [2023-12-27 04:02:53,608][105620] Updated weights for policy 1, policy_version 1740505 (0.0009) [2023-12-27 04:02:53,640][105692] Updated weights for policy 0, policy_version 1736901 (0.0009) [2023-12-27 04:02:53,658][105620] Updated weights for policy 1, policy_version 1740515 (0.0007) [2023-12-27 04:02:53,699][105692] Updated weights for policy 0, policy_version 1736911 (0.0008) [2023-12-27 04:02:53,710][105620] Updated weights for policy 1, policy_version 1740525 (0.0006) [2023-12-27 04:02:53,752][105692] Updated weights for policy 0, policy_version 1736921 (0.0007) [2023-12-27 04:02:54,387][105620] Updated weights for policy 1, policy_version 1740535 (0.0009) [2023-12-27 04:02:54,451][105620] Updated weights for policy 1, policy_version 1740545 (0.0010) [2023-12-27 04:02:54,480][105692] Updated weights for policy 0, policy_version 1736931 (0.0009) [2023-12-27 04:02:54,497][105620] Updated weights for policy 1, policy_version 1740555 (0.0006) [2023-12-27 04:02:54,540][105692] Updated weights for policy 0, policy_version 1736941 (0.0011) [2023-12-27 04:02:54,592][105692] Updated weights for policy 0, policy_version 1736951 (0.0008) [2023-12-27 04:02:55,215][105620] Updated weights for policy 1, policy_version 1740565 (0.0005) [2023-12-27 04:02:55,255][105692] Updated weights for policy 0, policy_version 1736961 (0.0008) [2023-12-27 04:02:55,264][105620] Updated weights for policy 1, policy_version 1740575 (0.0006) [2023-12-27 04:02:55,311][105692] Updated weights for policy 0, policy_version 1736971 (0.0006) [2023-12-27 04:02:55,330][105620] Updated weights for policy 1, policy_version 1740585 (0.0006) [2023-12-27 04:02:55,376][105692] Updated weights for policy 0, policy_version 1736981 (0.0006) [2023-12-27 04:02:55,443][105692] Updated weights for policy 0, policy_version 1736991 (0.0005) [2023-12-27 04:02:56,011][105692] Updated weights for policy 0, policy_version 1737001 (0.0010) [2023-12-27 04:02:56,034][105620] Updated weights for policy 1, policy_version 1740595 (0.0010) [2023-12-27 04:02:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 890388480. Throughput: 0: 9611.1, 1: 9931.8. Samples: 890403824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:02:56,062][104569] Avg episode reward: [(0, '8993.320'), (1, '8898.216')] [2023-12-27 04:02:56,071][105692] Updated weights for policy 0, policy_version 1737011 (0.0010) [2023-12-27 04:02:56,102][105620] Updated weights for policy 1, policy_version 1740606 (0.0010) [2023-12-27 04:02:56,134][105692] Updated weights for policy 0, policy_version 1737021 (0.0009) [2023-12-27 04:02:56,157][105620] Updated weights for policy 1, policy_version 1740616 (0.0007) [2023-12-27 04:02:56,831][105620] Updated weights for policy 1, policy_version 1740626 (0.0008) [2023-12-27 04:02:56,879][105620] Updated weights for policy 1, policy_version 1740636 (0.0005) [2023-12-27 04:02:56,888][105692] Updated weights for policy 0, policy_version 1737031 (0.0010) [2023-12-27 04:02:56,926][105620] Updated weights for policy 1, policy_version 1740646 (0.0005) [2023-12-27 04:02:56,939][105692] Updated weights for policy 0, policy_version 1737041 (0.0010) [2023-12-27 04:02:56,977][105620] Updated weights for policy 1, policy_version 1740656 (0.0006) [2023-12-27 04:02:56,994][105692] Updated weights for policy 0, policy_version 1737051 (0.0010) [2023-12-27 04:02:57,565][105620] Updated weights for policy 1, policy_version 1740666 (0.0006) [2023-12-27 04:02:57,613][105620] Updated weights for policy 1, policy_version 1740676 (0.0005) [2023-12-27 04:02:57,656][105620] Updated weights for policy 1, policy_version 1740686 (0.0005) [2023-12-27 04:02:57,745][105692] Updated weights for policy 0, policy_version 1737061 (0.0010) [2023-12-27 04:02:57,796][105692] Updated weights for policy 0, policy_version 1737071 (0.0010) [2023-12-27 04:02:57,843][105692] Updated weights for policy 0, policy_version 1737081 (0.0010) [2023-12-27 04:02:58,235][105620] Updated weights for policy 1, policy_version 1740696 (0.0009) [2023-12-27 04:02:58,312][105620] Updated weights for policy 1, policy_version 1740706 (0.0007) [2023-12-27 04:02:58,382][105620] Updated weights for policy 1, policy_version 1740716 (0.0013) [2023-12-27 04:02:58,686][105692] Updated weights for policy 0, policy_version 1737091 (0.0010) [2023-12-27 04:02:58,758][105692] Updated weights for policy 0, policy_version 1737101 (0.0008) [2023-12-27 04:02:58,823][105692] Updated weights for policy 0, policy_version 1737111 (0.0008) [2023-12-27 04:02:59,363][105620] Updated weights for policy 1, policy_version 1740727 (0.0009) [2023-12-27 04:02:59,434][105620] Updated weights for policy 1, policy_version 1740737 (0.0009) [2023-12-27 04:02:59,500][105620] Updated weights for policy 1, policy_version 1740747 (0.0009) [2023-12-27 04:02:59,712][105692] Updated weights for policy 0, policy_version 1737121 (0.0010) [2023-12-27 04:02:59,774][105692] Updated weights for policy 0, policy_version 1737131 (0.0006) [2023-12-27 04:02:59,847][105692] Updated weights for policy 0, policy_version 1737141 (0.0007) [2023-12-27 04:02:59,904][105692] Updated weights for policy 0, policy_version 1737151 (0.0008) [2023-12-27 04:03:00,301][105620] Updated weights for policy 1, policy_version 1740757 (0.0008) [2023-12-27 04:03:00,362][105620] Updated weights for policy 1, policy_version 1740767 (0.0006) [2023-12-27 04:03:00,429][105620] Updated weights for policy 1, policy_version 1740777 (0.0008) [2023-12-27 04:03:00,693][105692] Updated weights for policy 0, policy_version 1737161 (0.0010) [2023-12-27 04:03:00,749][105692] Updated weights for policy 0, policy_version 1737171 (0.0010) [2023-12-27 04:03:00,811][105692] Updated weights for policy 0, policy_version 1737181 (0.0010) [2023-12-27 04:03:01,042][105620] Updated weights for policy 1, policy_version 1740787 (0.0008) [2023-12-27 04:03:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 890486784. Throughput: 0: 9662.2, 1: 9927.6. Samples: 890463084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:03:01,062][104569] Avg episode reward: [(0, '8440.154'), (1, '8990.329')] [2023-12-27 04:03:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001737184_444784640.pth... [2023-12-27 04:03:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001736064_444497920.pth [2023-12-27 04:03:01,106][105620] Updated weights for policy 1, policy_version 1740797 (0.0009) [2023-12-27 04:03:01,187][105620] Updated weights for policy 1, policy_version 1740807 (0.0008) [2023-12-27 04:03:01,245][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001740816_445710336.pth... [2023-12-27 04:03:01,249][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001739664_445415424.pth [2023-12-27 04:03:01,646][105692] Updated weights for policy 0, policy_version 1737191 (0.0009) [2023-12-27 04:03:01,711][105692] Updated weights for policy 0, policy_version 1737201 (0.0009) [2023-12-27 04:03:01,775][105692] Updated weights for policy 0, policy_version 1737211 (0.0008) [2023-12-27 04:03:01,865][105620] Updated weights for policy 1, policy_version 1740817 (0.0009) [2023-12-27 04:03:01,928][105620] Updated weights for policy 1, policy_version 1740827 (0.0009) [2023-12-27 04:03:01,995][105620] Updated weights for policy 1, policy_version 1740837 (0.0006) [2023-12-27 04:03:02,041][105620] Updated weights for policy 1, policy_version 1740847 (0.0008) [2023-12-27 04:03:02,537][105692] Updated weights for policy 0, policy_version 1737221 (0.0008) [2023-12-27 04:03:02,604][105692] Updated weights for policy 0, policy_version 1737231 (0.0011) [2023-12-27 04:03:02,669][105692] Updated weights for policy 0, policy_version 1737241 (0.0011) [2023-12-27 04:03:02,818][105620] Updated weights for policy 1, policy_version 1740857 (0.0009) [2023-12-27 04:03:02,883][105620] Updated weights for policy 1, policy_version 1740867 (0.0005) [2023-12-27 04:03:02,940][105620] Updated weights for policy 1, policy_version 1740877 (0.0006) [2023-12-27 04:03:03,352][105692] Updated weights for policy 0, policy_version 1737251 (0.0010) [2023-12-27 04:03:03,412][105692] Updated weights for policy 0, policy_version 1737261 (0.0007) [2023-12-27 04:03:03,466][105692] Updated weights for policy 0, policy_version 1737271 (0.0010) [2023-12-27 04:03:03,497][105620] Updated weights for policy 1, policy_version 1740887 (0.0006) [2023-12-27 04:03:03,541][105620] Updated weights for policy 1, policy_version 1740897 (0.0007) [2023-12-27 04:03:03,588][105620] Updated weights for policy 1, policy_version 1740907 (0.0007) [2023-12-27 04:03:04,209][105692] Updated weights for policy 0, policy_version 1737281 (0.0010) [2023-12-27 04:03:04,278][105692] Updated weights for policy 0, policy_version 1737291 (0.0010) [2023-12-27 04:03:04,342][105692] Updated weights for policy 0, policy_version 1737301 (0.0009) [2023-12-27 04:03:04,365][105620] Updated weights for policy 1, policy_version 1740917 (0.0008) [2023-12-27 04:03:04,411][105692] Updated weights for policy 0, policy_version 1737311 (0.0009) [2023-12-27 04:03:04,427][105620] Updated weights for policy 1, policy_version 1740927 (0.0006) [2023-12-27 04:03:04,494][105620] Updated weights for policy 1, policy_version 1740937 (0.0008) [2023-12-27 04:03:05,159][105620] Updated weights for policy 1, policy_version 1740947 (0.0009) [2023-12-27 04:03:05,197][105692] Updated weights for policy 0, policy_version 1737321 (0.0007) [2023-12-27 04:03:05,210][105620] Updated weights for policy 1, policy_version 1740957 (0.0010) [2023-12-27 04:03:05,251][105692] Updated weights for policy 0, policy_version 1737331 (0.0006) [2023-12-27 04:03:05,261][105620] Updated weights for policy 1, policy_version 1740967 (0.0010) [2023-12-27 04:03:05,304][105692] Updated weights for policy 0, policy_version 1737341 (0.0005) [2023-12-27 04:03:05,892][105620] Updated weights for policy 1, policy_version 1740977 (0.0010) [2023-12-27 04:03:05,960][105620] Updated weights for policy 1, policy_version 1740987 (0.0005) [2023-12-27 04:03:06,014][105620] Updated weights for policy 1, policy_version 1740997 (0.0006) [2023-12-27 04:03:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 890576896. Throughput: 0: 9541.9, 1: 9843.8. Samples: 890573732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:03:06,062][104569] Avg episode reward: [(0, '8439.105'), (1, '8987.988')] [2023-12-27 04:03:06,065][105620] Updated weights for policy 1, policy_version 1741007 (0.0010) [2023-12-27 04:03:06,145][105692] Updated weights for policy 0, policy_version 1737351 (0.0008) [2023-12-27 04:03:06,197][105692] Updated weights for policy 0, policy_version 1737361 (0.0009) [2023-12-27 04:03:06,253][105692] Updated weights for policy 0, policy_version 1737371 (0.0011) [2023-12-27 04:03:06,735][105620] Updated weights for policy 1, policy_version 1741017 (0.0011) [2023-12-27 04:03:06,790][105620] Updated weights for policy 1, policy_version 1741027 (0.0008) [2023-12-27 04:03:06,849][105620] Updated weights for policy 1, policy_version 1741037 (0.0005) [2023-12-27 04:03:07,059][105692] Updated weights for policy 0, policy_version 1737381 (0.0009) [2023-12-27 04:03:07,126][105692] Updated weights for policy 0, policy_version 1737391 (0.0010) [2023-12-27 04:03:07,195][105692] Updated weights for policy 0, policy_version 1737401 (0.0010) [2023-12-27 04:03:07,498][105620] Updated weights for policy 1, policy_version 1741047 (0.0010) [2023-12-27 04:03:07,545][105620] Updated weights for policy 1, policy_version 1741057 (0.0010) [2023-12-27 04:03:07,602][105620] Updated weights for policy 1, policy_version 1741067 (0.0010) [2023-12-27 04:03:08,006][105692] Updated weights for policy 0, policy_version 1737411 (0.0009) [2023-12-27 04:03:08,074][105692] Updated weights for policy 0, policy_version 1737421 (0.0006) [2023-12-27 04:03:08,131][105692] Updated weights for policy 0, policy_version 1737431 (0.0010) [2023-12-27 04:03:08,196][105620] Updated weights for policy 1, policy_version 1741077 (0.0008) [2023-12-27 04:03:08,256][105620] Updated weights for policy 1, policy_version 1741087 (0.0005) [2023-12-27 04:03:08,307][105620] Updated weights for policy 1, policy_version 1741097 (0.0005) [2023-12-27 04:03:08,903][105692] Updated weights for policy 0, policy_version 1737442 (0.0010) [2023-12-27 04:03:08,960][105692] Updated weights for policy 0, policy_version 1737452 (0.0008) [2023-12-27 04:03:09,011][105620] Updated weights for policy 1, policy_version 1741107 (0.0011) [2023-12-27 04:03:09,021][105692] Updated weights for policy 0, policy_version 1737462 (0.0007) [2023-12-27 04:03:09,067][105620] Updated weights for policy 1, policy_version 1741117 (0.0011) [2023-12-27 04:03:09,071][105692] Updated weights for policy 0, policy_version 1737472 (0.0009) [2023-12-27 04:03:09,125][105620] Updated weights for policy 1, policy_version 1741127 (0.0010) [2023-12-27 04:03:09,842][105692] Updated weights for policy 0, policy_version 1737482 (0.0007) [2023-12-27 04:03:09,897][105620] Updated weights for policy 1, policy_version 1741137 (0.0010) [2023-12-27 04:03:09,899][105692] Updated weights for policy 0, policy_version 1737492 (0.0008) [2023-12-27 04:03:09,959][105692] Updated weights for policy 0, policy_version 1737502 (0.0008) [2023-12-27 04:03:09,966][105620] Updated weights for policy 1, policy_version 1741147 (0.0011) [2023-12-27 04:03:10,026][105620] Updated weights for policy 1, policy_version 1741157 (0.0011) [2023-12-27 04:03:10,075][105620] Updated weights for policy 1, policy_version 1741167 (0.0010) [2023-12-27 04:03:10,596][105692] Updated weights for policy 0, policy_version 1737512 (0.0006) [2023-12-27 04:03:10,654][105692] Updated weights for policy 0, policy_version 1737522 (0.0006) [2023-12-27 04:03:10,712][105692] Updated weights for policy 0, policy_version 1737532 (0.0005) [2023-12-27 04:03:10,766][105620] Updated weights for policy 1, policy_version 1741177 (0.0008) [2023-12-27 04:03:10,830][105620] Updated weights for policy 1, policy_version 1741187 (0.0009) [2023-12-27 04:03:10,895][105620] Updated weights for policy 1, policy_version 1741197 (0.0006) [2023-12-27 04:03:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 890683392. Throughput: 0: 9507.2, 1: 9947.5. Samples: 890690812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:03:11,063][104569] Avg episode reward: [(0, '8899.581'), (1, '9169.506')] [2023-12-27 04:03:11,416][105692] Updated weights for policy 0, policy_version 1737542 (0.0007) [2023-12-27 04:03:11,476][105692] Updated weights for policy 0, policy_version 1737552 (0.0008) [2023-12-27 04:03:11,535][105692] Updated weights for policy 0, policy_version 1737562 (0.0007) [2023-12-27 04:03:11,589][105620] Updated weights for policy 1, policy_version 1741207 (0.0009) [2023-12-27 04:03:11,654][105620] Updated weights for policy 1, policy_version 1741217 (0.0011) [2023-12-27 04:03:11,727][105620] Updated weights for policy 1, policy_version 1741227 (0.0012) [2023-12-27 04:03:12,334][105692] Updated weights for policy 0, policy_version 1737572 (0.0008) [2023-12-27 04:03:12,395][105692] Updated weights for policy 0, policy_version 1737582 (0.0008) [2023-12-27 04:03:12,455][105692] Updated weights for policy 0, policy_version 1737592 (0.0008) [2023-12-27 04:03:12,484][105620] Updated weights for policy 1, policy_version 1741237 (0.0010) [2023-12-27 04:03:12,537][105620] Updated weights for policy 1, policy_version 1741247 (0.0010) [2023-12-27 04:03:12,593][105620] Updated weights for policy 1, policy_version 1741257 (0.0011) [2023-12-27 04:03:13,211][105692] Updated weights for policy 0, policy_version 1737602 (0.0006) [2023-12-27 04:03:13,269][105692] Updated weights for policy 0, policy_version 1737612 (0.0008) [2023-12-27 04:03:13,317][105692] Updated weights for policy 0, policy_version 1737622 (0.0008) [2023-12-27 04:03:13,351][105620] Updated weights for policy 1, policy_version 1741267 (0.0011) [2023-12-27 04:03:13,369][105692] Updated weights for policy 0, policy_version 1737632 (0.0006) [2023-12-27 04:03:13,413][105620] Updated weights for policy 1, policy_version 1741277 (0.0010) [2023-12-27 04:03:13,486][105620] Updated weights for policy 1, policy_version 1741287 (0.0010) [2023-12-27 04:03:13,963][105692] Updated weights for policy 0, policy_version 1737642 (0.0005) [2023-12-27 04:03:14,016][105692] Updated weights for policy 0, policy_version 1737652 (0.0005) [2023-12-27 04:03:14,033][105620] Updated weights for policy 1, policy_version 1741297 (0.0009) [2023-12-27 04:03:14,070][105692] Updated weights for policy 0, policy_version 1737662 (0.0005) [2023-12-27 04:03:14,095][105620] Updated weights for policy 1, policy_version 1741307 (0.0011) [2023-12-27 04:03:14,162][105620] Updated weights for policy 1, policy_version 1741317 (0.0010) [2023-12-27 04:03:14,232][105620] Updated weights for policy 1, policy_version 1741327 (0.0008) [2023-12-27 04:03:14,671][105692] Updated weights for policy 0, policy_version 1737672 (0.0006) [2023-12-27 04:03:14,718][105692] Updated weights for policy 0, policy_version 1737682 (0.0009) [2023-12-27 04:03:14,774][105692] Updated weights for policy 0, policy_version 1737692 (0.0009) [2023-12-27 04:03:15,035][105620] Updated weights for policy 1, policy_version 1741337 (0.0009) [2023-12-27 04:03:15,097][105620] Updated weights for policy 1, policy_version 1741347 (0.0009) [2023-12-27 04:03:15,155][105620] Updated weights for policy 1, policy_version 1741357 (0.0011) [2023-12-27 04:03:15,438][105692] Updated weights for policy 0, policy_version 1737702 (0.0009) [2023-12-27 04:03:15,499][105692] Updated weights for policy 0, policy_version 1737712 (0.0007) [2023-12-27 04:03:15,557][105692] Updated weights for policy 0, policy_version 1737722 (0.0008) [2023-12-27 04:03:15,934][105620] Updated weights for policy 1, policy_version 1741367 (0.0010) [2023-12-27 04:03:15,985][105620] Updated weights for policy 1, policy_version 1741377 (0.0010) [2023-12-27 04:03:16,032][105620] Updated weights for policy 1, policy_version 1741387 (0.0010) [2023-12-27 04:03:16,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 890781696. Throughput: 0: 9448.6, 1: 9905.0. Samples: 890747568. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:03:16,063][104569] Avg episode reward: [(0, '8168.798'), (1, '9169.573')] [2023-12-27 04:03:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001737728_444923904.pth... [2023-12-27 04:03:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001741392_445857792.pth... [2023-12-27 04:03:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001736608_444637184.pth [2023-12-27 04:03:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001740240_445562880.pth [2023-12-27 04:03:16,240][105692] Updated weights for policy 0, policy_version 1737732 (0.0010) [2023-12-27 04:03:16,292][105692] Updated weights for policy 0, policy_version 1737742 (0.0010) [2023-12-27 04:03:16,343][105692] Updated weights for policy 0, policy_version 1737752 (0.0010) [2023-12-27 04:03:16,681][105620] Updated weights for policy 1, policy_version 1741397 (0.0008) [2023-12-27 04:03:16,733][105620] Updated weights for policy 1, policy_version 1741407 (0.0005) [2023-12-27 04:03:16,789][105620] Updated weights for policy 1, policy_version 1741417 (0.0005) [2023-12-27 04:03:16,946][105692] Updated weights for policy 0, policy_version 1737762 (0.0009) [2023-12-27 04:03:16,993][105692] Updated weights for policy 0, policy_version 1737772 (0.0008) [2023-12-27 04:03:17,059][105692] Updated weights for policy 0, policy_version 1737782 (0.0009) [2023-12-27 04:03:17,123][105692] Updated weights for policy 0, policy_version 1737792 (0.0009) [2023-12-27 04:03:17,399][105620] Updated weights for policy 1, policy_version 1741427 (0.0006) [2023-12-27 04:03:17,455][105620] Updated weights for policy 1, policy_version 1741437 (0.0007) [2023-12-27 04:03:17,517][105620] Updated weights for policy 1, policy_version 1741447 (0.0006) [2023-12-27 04:03:17,901][105692] Updated weights for policy 0, policy_version 1737802 (0.0009) [2023-12-27 04:03:17,962][105692] Updated weights for policy 0, policy_version 1737812 (0.0007) [2023-12-27 04:03:18,016][105692] Updated weights for policy 0, policy_version 1737822 (0.0006) [2023-12-27 04:03:18,143][105620] Updated weights for policy 1, policy_version 1741457 (0.0006) [2023-12-27 04:03:18,196][105620] Updated weights for policy 1, policy_version 1741467 (0.0011) [2023-12-27 04:03:18,252][105620] Updated weights for policy 1, policy_version 1741477 (0.0010) [2023-12-27 04:03:18,308][105620] Updated weights for policy 1, policy_version 1741487 (0.0011) [2023-12-27 04:03:18,644][105692] Updated weights for policy 0, policy_version 1737832 (0.0005) [2023-12-27 04:03:18,704][105692] Updated weights for policy 0, policy_version 1737842 (0.0008) [2023-12-27 04:03:18,752][105692] Updated weights for policy 0, policy_version 1737852 (0.0010) [2023-12-27 04:03:18,942][105620] Updated weights for policy 1, policy_version 1741497 (0.0006) [2023-12-27 04:03:18,999][105620] Updated weights for policy 1, policy_version 1741507 (0.0006) [2023-12-27 04:03:19,052][105620] Updated weights for policy 1, policy_version 1741517 (0.0005) [2023-12-27 04:03:19,383][105692] Updated weights for policy 0, policy_version 1737862 (0.0009) [2023-12-27 04:03:19,439][105692] Updated weights for policy 0, policy_version 1737872 (0.0005) [2023-12-27 04:03:19,507][105692] Updated weights for policy 0, policy_version 1737882 (0.0010) [2023-12-27 04:03:19,727][105620] Updated weights for policy 1, policy_version 1741527 (0.0006) [2023-12-27 04:03:19,782][105620] Updated weights for policy 1, policy_version 1741537 (0.0005) [2023-12-27 04:03:19,846][105620] Updated weights for policy 1, policy_version 1741547 (0.0007) [2023-12-27 04:03:20,156][105692] Updated weights for policy 0, policy_version 1737892 (0.0010) [2023-12-27 04:03:20,226][105692] Updated weights for policy 0, policy_version 1737902 (0.0008) [2023-12-27 04:03:20,294][105692] Updated weights for policy 0, policy_version 1737912 (0.0008) [2023-12-27 04:03:20,523][105620] Updated weights for policy 1, policy_version 1741557 (0.0006) [2023-12-27 04:03:20,585][105620] Updated weights for policy 1, policy_version 1741567 (0.0009) [2023-12-27 04:03:20,652][105620] Updated weights for policy 1, policy_version 1741577 (0.0007) [2023-12-27 04:03:21,029][105692] Updated weights for policy 0, policy_version 1737922 (0.0009) [2023-12-27 04:03:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 890880000. Throughput: 0: 9685.9, 1: 9862.0. Samples: 890873320. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:03:21,063][104569] Avg episode reward: [(0, '8440.472'), (1, '8984.699')] [2023-12-27 04:03:21,101][105692] Updated weights for policy 0, policy_version 1737932 (0.0011) [2023-12-27 04:03:21,175][105692] Updated weights for policy 0, policy_version 1737942 (0.0010) [2023-12-27 04:03:21,238][105692] Updated weights for policy 0, policy_version 1737952 (0.0008) [2023-12-27 04:03:21,341][105620] Updated weights for policy 1, policy_version 1741587 (0.0009) [2023-12-27 04:03:21,407][105620] Updated weights for policy 1, policy_version 1741597 (0.0008) [2023-12-27 04:03:21,463][105620] Updated weights for policy 1, policy_version 1741607 (0.0008) [2023-12-27 04:03:22,035][105692] Updated weights for policy 0, policy_version 1737962 (0.0006) [2023-12-27 04:03:22,090][105692] Updated weights for policy 0, policy_version 1737972 (0.0008) [2023-12-27 04:03:22,147][105692] Updated weights for policy 0, policy_version 1737982 (0.0009) [2023-12-27 04:03:22,165][105620] Updated weights for policy 1, policy_version 1741617 (0.0007) [2023-12-27 04:03:22,239][105620] Updated weights for policy 1, policy_version 1741627 (0.0006) [2023-12-27 04:03:22,302][105620] Updated weights for policy 1, policy_version 1741637 (0.0008) [2023-12-27 04:03:22,367][105620] Updated weights for policy 1, policy_version 1741647 (0.0008) [2023-12-27 04:03:22,884][105692] Updated weights for policy 0, policy_version 1737992 (0.0011) [2023-12-27 04:03:22,943][105692] Updated weights for policy 0, policy_version 1738002 (0.0011) [2023-12-27 04:03:23,002][105692] Updated weights for policy 0, policy_version 1738012 (0.0011) [2023-12-27 04:03:23,076][105620] Updated weights for policy 1, policy_version 1741657 (0.0010) [2023-12-27 04:03:23,122][105620] Updated weights for policy 1, policy_version 1741667 (0.0010) [2023-12-27 04:03:23,171][105620] Updated weights for policy 1, policy_version 1741677 (0.0010) [2023-12-27 04:03:23,723][105692] Updated weights for policy 0, policy_version 1738022 (0.0008) [2023-12-27 04:03:23,760][105620] Updated weights for policy 1, policy_version 1741687 (0.0007) [2023-12-27 04:03:23,775][105692] Updated weights for policy 0, policy_version 1738032 (0.0005) [2023-12-27 04:03:23,807][105620] Updated weights for policy 1, policy_version 1741697 (0.0005) [2023-12-27 04:03:23,830][105692] Updated weights for policy 0, policy_version 1738042 (0.0005) [2023-12-27 04:03:23,851][105620] Updated weights for policy 1, policy_version 1741707 (0.0005) [2023-12-27 04:03:24,358][105692] Updated weights for policy 0, policy_version 1738052 (0.0007) [2023-12-27 04:03:24,422][105692] Updated weights for policy 0, policy_version 1738062 (0.0006) [2023-12-27 04:03:24,478][105692] Updated weights for policy 0, policy_version 1738072 (0.0005) [2023-12-27 04:03:24,486][105620] Updated weights for policy 1, policy_version 1741717 (0.0010) [2023-12-27 04:03:24,546][105620] Updated weights for policy 1, policy_version 1741727 (0.0011) [2023-12-27 04:03:24,609][105620] Updated weights for policy 1, policy_version 1741737 (0.0011) [2023-12-27 04:03:25,025][105692] Updated weights for policy 0, policy_version 1738082 (0.0007) [2023-12-27 04:03:25,090][105692] Updated weights for policy 0, policy_version 1738092 (0.0007) [2023-12-27 04:03:25,138][105692] Updated weights for policy 0, policy_version 1738102 (0.0010) [2023-12-27 04:03:25,184][105692] Updated weights for policy 0, policy_version 1738112 (0.0009) [2023-12-27 04:03:25,287][105620] Updated weights for policy 1, policy_version 1741747 (0.0010) [2023-12-27 04:03:25,346][105620] Updated weights for policy 1, policy_version 1741757 (0.0010) [2023-12-27 04:03:25,401][105620] Updated weights for policy 1, policy_version 1741767 (0.0010) [2023-12-27 04:03:25,838][105692] Updated weights for policy 0, policy_version 1738122 (0.0011) [2023-12-27 04:03:25,893][105692] Updated weights for policy 0, policy_version 1738132 (0.0010) [2023-12-27 04:03:25,950][105692] Updated weights for policy 0, policy_version 1738142 (0.0011) [2023-12-27 04:03:26,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 890986496. Throughput: 0: 9812.3, 1: 9898.4. Samples: 890995920. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:03:26,062][104569] Avg episode reward: [(0, '8444.845'), (1, '8985.649')] [2023-12-27 04:03:26,089][105620] Updated weights for policy 1, policy_version 1741777 (0.0010) [2023-12-27 04:03:26,154][105620] Updated weights for policy 1, policy_version 1741787 (0.0010) [2023-12-27 04:03:26,215][105620] Updated weights for policy 1, policy_version 1741797 (0.0010) [2023-12-27 04:03:26,279][105620] Updated weights for policy 1, policy_version 1741807 (0.0010) [2023-12-27 04:03:26,629][105692] Updated weights for policy 0, policy_version 1738152 (0.0008) [2023-12-27 04:03:26,687][105692] Updated weights for policy 0, policy_version 1738162 (0.0005) [2023-12-27 04:03:26,749][105692] Updated weights for policy 0, policy_version 1738172 (0.0011) [2023-12-27 04:03:26,990][105620] Updated weights for policy 1, policy_version 1741817 (0.0006) [2023-12-27 04:03:27,054][105620] Updated weights for policy 1, policy_version 1741827 (0.0008) [2023-12-27 04:03:27,102][105620] Updated weights for policy 1, policy_version 1741837 (0.0010) [2023-12-27 04:03:27,386][105692] Updated weights for policy 0, policy_version 1738182 (0.0011) [2023-12-27 04:03:27,430][105692] Updated weights for policy 0, policy_version 1738192 (0.0010) [2023-12-27 04:03:27,478][105692] Updated weights for policy 0, policy_version 1738202 (0.0008) [2023-12-27 04:03:27,794][105620] Updated weights for policy 1, policy_version 1741847 (0.0007) [2023-12-27 04:03:27,862][105620] Updated weights for policy 1, policy_version 1741857 (0.0005) [2023-12-27 04:03:27,928][105620] Updated weights for policy 1, policy_version 1741867 (0.0005) [2023-12-27 04:03:28,088][105692] Updated weights for policy 0, policy_version 1738212 (0.0005) [2023-12-27 04:03:28,151][105692] Updated weights for policy 0, policy_version 1738222 (0.0009) [2023-12-27 04:03:28,198][105692] Updated weights for policy 0, policy_version 1738232 (0.0010) [2023-12-27 04:03:28,571][105620] Updated weights for policy 1, policy_version 1741877 (0.0008) [2023-12-27 04:03:28,630][105620] Updated weights for policy 1, policy_version 1741887 (0.0010) [2023-12-27 04:03:28,690][105620] Updated weights for policy 1, policy_version 1741897 (0.0010) [2023-12-27 04:03:28,925][105692] Updated weights for policy 0, policy_version 1738242 (0.0010) [2023-12-27 04:03:28,991][105692] Updated weights for policy 0, policy_version 1738252 (0.0011) [2023-12-27 04:03:29,063][105692] Updated weights for policy 0, policy_version 1738262 (0.0011) [2023-12-27 04:03:29,137][105692] Updated weights for policy 0, policy_version 1738272 (0.0006) [2023-12-27 04:03:29,377][105620] Updated weights for policy 1, policy_version 1741907 (0.0010) [2023-12-27 04:03:29,429][105620] Updated weights for policy 1, policy_version 1741917 (0.0008) [2023-12-27 04:03:29,477][105620] Updated weights for policy 1, policy_version 1741927 (0.0008) [2023-12-27 04:03:29,811][105692] Updated weights for policy 0, policy_version 1738282 (0.0009) [2023-12-27 04:03:29,872][105692] Updated weights for policy 0, policy_version 1738292 (0.0008) [2023-12-27 04:03:29,940][105692] Updated weights for policy 0, policy_version 1738302 (0.0007) [2023-12-27 04:03:30,247][105620] Updated weights for policy 1, policy_version 1741937 (0.0008) [2023-12-27 04:03:30,306][105620] Updated weights for policy 1, policy_version 1741947 (0.0010) [2023-12-27 04:03:30,364][105620] Updated weights for policy 1, policy_version 1741957 (0.0011) [2023-12-27 04:03:30,423][105620] Updated weights for policy 1, policy_version 1741967 (0.0010) [2023-12-27 04:03:30,724][105692] Updated weights for policy 0, policy_version 1738312 (0.0008) [2023-12-27 04:03:30,784][105692] Updated weights for policy 0, policy_version 1738322 (0.0006) [2023-12-27 04:03:30,841][105692] Updated weights for policy 0, policy_version 1738332 (0.0005) [2023-12-27 04:03:31,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.7, 300 sec: 19494.2). Total num frames: 891084800. Throughput: 0: 9914.0, 1: 9891.1. Samples: 891057168. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:03:31,063][104569] Avg episode reward: [(0, '8442.415'), (1, '9171.934')] [2023-12-27 04:03:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001738336_445079552.pth... [2023-12-27 04:03:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001741968_446005248.pth... [2023-12-27 04:03:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001737184_444784640.pth [2023-12-27 04:03:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001740816_445710336.pth [2023-12-27 04:03:31,184][105620] Updated weights for policy 1, policy_version 1741977 (0.0009) [2023-12-27 04:03:31,242][105620] Updated weights for policy 1, policy_version 1741987 (0.0009) [2023-12-27 04:03:31,311][105620] Updated weights for policy 1, policy_version 1741997 (0.0009) [2023-12-27 04:03:31,552][105692] Updated weights for policy 0, policy_version 1738342 (0.0008) [2023-12-27 04:03:31,600][105692] Updated weights for policy 0, policy_version 1738352 (0.0008) [2023-12-27 04:03:31,663][105692] Updated weights for policy 0, policy_version 1738362 (0.0007) [2023-12-27 04:03:32,135][105620] Updated weights for policy 1, policy_version 1742007 (0.0009) [2023-12-27 04:03:32,187][105620] Updated weights for policy 1, policy_version 1742017 (0.0009) [2023-12-27 04:03:32,243][105620] Updated weights for policy 1, policy_version 1742027 (0.0008) [2023-12-27 04:03:32,253][105692] Updated weights for policy 0, policy_version 1738372 (0.0007) [2023-12-27 04:03:32,317][105692] Updated weights for policy 0, policy_version 1738382 (0.0009) [2023-12-27 04:03:32,375][105692] Updated weights for policy 0, policy_version 1738392 (0.0009) [2023-12-27 04:03:33,056][105620] Updated weights for policy 1, policy_version 1742037 (0.0008) [2023-12-27 04:03:33,061][105692] Updated weights for policy 0, policy_version 1738402 (0.0008) [2023-12-27 04:03:33,124][105620] Updated weights for policy 1, policy_version 1742047 (0.0008) [2023-12-27 04:03:33,132][105692] Updated weights for policy 0, policy_version 1738412 (0.0007) [2023-12-27 04:03:33,184][105620] Updated weights for policy 1, policy_version 1742057 (0.0007) [2023-12-27 04:03:33,196][105692] Updated weights for policy 0, policy_version 1738422 (0.0006) [2023-12-27 04:03:33,259][105692] Updated weights for policy 0, policy_version 1738432 (0.0007) [2023-12-27 04:03:33,804][105620] Updated weights for policy 1, policy_version 1742067 (0.0007) [2023-12-27 04:03:33,856][105620] Updated weights for policy 1, policy_version 1742077 (0.0005) [2023-12-27 04:03:33,882][105692] Updated weights for policy 0, policy_version 1738442 (0.0007) [2023-12-27 04:03:33,914][105620] Updated weights for policy 1, policy_version 1742087 (0.0005) [2023-12-27 04:03:33,936][105692] Updated weights for policy 0, policy_version 1738452 (0.0006) [2023-12-27 04:03:33,985][105692] Updated weights for policy 0, policy_version 1738462 (0.0005) [2023-12-27 04:03:34,564][105692] Updated weights for policy 0, policy_version 1738472 (0.0007) [2023-12-27 04:03:34,603][105620] Updated weights for policy 1, policy_version 1742097 (0.0010) [2023-12-27 04:03:34,625][105692] Updated weights for policy 0, policy_version 1738482 (0.0008) [2023-12-27 04:03:34,654][105620] Updated weights for policy 1, policy_version 1742107 (0.0008) [2023-12-27 04:03:34,689][105692] Updated weights for policy 0, policy_version 1738492 (0.0007) [2023-12-27 04:03:34,707][105620] Updated weights for policy 1, policy_version 1742117 (0.0007) [2023-12-27 04:03:34,756][105620] Updated weights for policy 1, policy_version 1742127 (0.0008) [2023-12-27 04:03:35,320][105692] Updated weights for policy 0, policy_version 1738502 (0.0006) [2023-12-27 04:03:35,371][105692] Updated weights for policy 0, policy_version 1738512 (0.0005) [2023-12-27 04:03:35,422][105692] Updated weights for policy 0, policy_version 1738522 (0.0005) [2023-12-27 04:03:35,598][105620] Updated weights for policy 1, policy_version 1742138 (0.0008) [2023-12-27 04:03:35,652][105620] Updated weights for policy 1, policy_version 1742148 (0.0005) [2023-12-27 04:03:35,705][105620] Updated weights for policy 1, policy_version 1742158 (0.0005) [2023-12-27 04:03:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 891183104. Throughput: 0: 9882.7, 1: 9899.5. Samples: 891175388. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:03:36,062][104569] Avg episode reward: [(0, '8985.431'), (1, '9079.082')] [2023-12-27 04:03:36,104][105692] Updated weights for policy 0, policy_version 1738532 (0.0007) [2023-12-27 04:03:36,175][105692] Updated weights for policy 0, policy_version 1738542 (0.0009) [2023-12-27 04:03:36,234][105692] Updated weights for policy 0, policy_version 1738552 (0.0009) [2023-12-27 04:03:36,265][105620] Updated weights for policy 1, policy_version 1742168 (0.0007) [2023-12-27 04:03:36,324][105620] Updated weights for policy 1, policy_version 1742178 (0.0008) [2023-12-27 04:03:36,384][105620] Updated weights for policy 1, policy_version 1742188 (0.0008) [2023-12-27 04:03:36,889][105692] Updated weights for policy 0, policy_version 1738562 (0.0009) [2023-12-27 04:03:36,954][105692] Updated weights for policy 0, policy_version 1738572 (0.0006) [2023-12-27 04:03:37,017][105692] Updated weights for policy 0, policy_version 1738582 (0.0006) [2023-12-27 04:03:37,052][105620] Updated weights for policy 1, policy_version 1742198 (0.0006) [2023-12-27 04:03:37,074][105692] Updated weights for policy 0, policy_version 1738592 (0.0007) [2023-12-27 04:03:37,120][105620] Updated weights for policy 1, policy_version 1742208 (0.0006) [2023-12-27 04:03:37,186][105620] Updated weights for policy 1, policy_version 1742218 (0.0009) [2023-12-27 04:03:37,759][105692] Updated weights for policy 0, policy_version 1738602 (0.0009) [2023-12-27 04:03:37,798][105620] Updated weights for policy 1, policy_version 1742228 (0.0006) [2023-12-27 04:03:37,813][105692] Updated weights for policy 0, policy_version 1738612 (0.0010) [2023-12-27 04:03:37,854][105620] Updated weights for policy 1, policy_version 1742238 (0.0005) [2023-12-27 04:03:37,870][105692] Updated weights for policy 0, policy_version 1738622 (0.0009) [2023-12-27 04:03:37,911][105620] Updated weights for policy 1, policy_version 1742248 (0.0005) [2023-12-27 04:03:38,554][105620] Updated weights for policy 1, policy_version 1742258 (0.0007) [2023-12-27 04:03:38,621][105620] Updated weights for policy 1, policy_version 1742268 (0.0009) [2023-12-27 04:03:38,625][105692] Updated weights for policy 0, policy_version 1738632 (0.0007) [2023-12-27 04:03:38,675][105692] Updated weights for policy 0, policy_version 1738642 (0.0006) [2023-12-27 04:03:38,681][105620] Updated weights for policy 1, policy_version 1742278 (0.0007) [2023-12-27 04:03:38,732][105692] Updated weights for policy 0, policy_version 1738652 (0.0006) [2023-12-27 04:03:38,742][105620] Updated weights for policy 1, policy_version 1742288 (0.0007) [2023-12-27 04:03:39,422][105620] Updated weights for policy 1, policy_version 1742298 (0.0009) [2023-12-27 04:03:39,480][105620] Updated weights for policy 1, policy_version 1742308 (0.0008) [2023-12-27 04:03:39,541][105620] Updated weights for policy 1, policy_version 1742318 (0.0009) [2023-12-27 04:03:39,547][105692] Updated weights for policy 0, policy_version 1738662 (0.0007) [2023-12-27 04:03:39,608][105692] Updated weights for policy 0, policy_version 1738672 (0.0005) [2023-12-27 04:03:39,675][105692] Updated weights for policy 0, policy_version 1738682 (0.0006) [2023-12-27 04:03:40,346][105620] Updated weights for policy 1, policy_version 1742328 (0.0008) [2023-12-27 04:03:40,380][105692] Updated weights for policy 0, policy_version 1738692 (0.0007) [2023-12-27 04:03:40,400][105620] Updated weights for policy 1, policy_version 1742338 (0.0008) [2023-12-27 04:03:40,437][105692] Updated weights for policy 0, policy_version 1738702 (0.0008) [2023-12-27 04:03:40,461][105620] Updated weights for policy 1, policy_version 1742348 (0.0007) [2023-12-27 04:03:40,506][105692] Updated weights for policy 0, policy_version 1738712 (0.0006) [2023-12-27 04:03:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 891281408. Throughput: 0: 9805.6, 1: 9987.1. Samples: 891294496. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:03:41,062][104569] Avg episode reward: [(0, '8985.551'), (1, '8986.845')] [2023-12-27 04:03:41,163][105692] Updated weights for policy 0, policy_version 1738722 (0.0006) [2023-12-27 04:03:41,202][105620] Updated weights for policy 1, policy_version 1742358 (0.0009) [2023-12-27 04:03:41,221][105692] Updated weights for policy 0, policy_version 1738732 (0.0008) [2023-12-27 04:03:41,263][105620] Updated weights for policy 1, policy_version 1742368 (0.0009) [2023-12-27 04:03:41,290][105692] Updated weights for policy 0, policy_version 1738742 (0.0008) [2023-12-27 04:03:41,328][105620] Updated weights for policy 1, policy_version 1742378 (0.0009) [2023-12-27 04:03:41,354][105692] Updated weights for policy 0, policy_version 1738752 (0.0008) [2023-12-27 04:03:42,046][105692] Updated weights for policy 0, policy_version 1738762 (0.0010) [2023-12-27 04:03:42,109][105692] Updated weights for policy 0, policy_version 1738772 (0.0009) [2023-12-27 04:03:42,141][105620] Updated weights for policy 1, policy_version 1742388 (0.0008) [2023-12-27 04:03:42,166][105692] Updated weights for policy 0, policy_version 1738782 (0.0008) [2023-12-27 04:03:42,189][105620] Updated weights for policy 1, policy_version 1742398 (0.0008) [2023-12-27 04:03:42,237][105620] Updated weights for policy 1, policy_version 1742408 (0.0009) [2023-12-27 04:03:42,878][105620] Updated weights for policy 1, policy_version 1742418 (0.0009) [2023-12-27 04:03:42,925][105620] Updated weights for policy 1, policy_version 1742428 (0.0009) [2023-12-27 04:03:42,964][105692] Updated weights for policy 0, policy_version 1738792 (0.0008) [2023-12-27 04:03:42,986][105620] Updated weights for policy 1, policy_version 1742438 (0.0006) [2023-12-27 04:03:43,029][105692] Updated weights for policy 0, policy_version 1738802 (0.0007) [2023-12-27 04:03:43,043][105620] Updated weights for policy 1, policy_version 1742448 (0.0006) [2023-12-27 04:03:43,089][105692] Updated weights for policy 0, policy_version 1738812 (0.0008) [2023-12-27 04:03:43,784][105692] Updated weights for policy 0, policy_version 1738822 (0.0008) [2023-12-27 04:03:43,833][105620] Updated weights for policy 1, policy_version 1742458 (0.0008) [2023-12-27 04:03:43,833][105692] Updated weights for policy 0, policy_version 1738832 (0.0008) [2023-12-27 04:03:43,879][105692] Updated weights for policy 0, policy_version 1738842 (0.0006) [2023-12-27 04:03:43,892][105620] Updated weights for policy 1, policy_version 1742468 (0.0007) [2023-12-27 04:03:43,950][105620] Updated weights for policy 1, policy_version 1742478 (0.0008) [2023-12-27 04:03:44,528][105620] Updated weights for policy 1, policy_version 1742488 (0.0008) [2023-12-27 04:03:44,587][105620] Updated weights for policy 1, policy_version 1742498 (0.0007) [2023-12-27 04:03:44,643][105620] Updated weights for policy 1, policy_version 1742508 (0.0008) [2023-12-27 04:03:44,727][105692] Updated weights for policy 0, policy_version 1738852 (0.0008) [2023-12-27 04:03:44,784][105692] Updated weights for policy 0, policy_version 1738862 (0.0009) [2023-12-27 04:03:44,844][105692] Updated weights for policy 0, policy_version 1738872 (0.0009) [2023-12-27 04:03:45,403][105620] Updated weights for policy 1, policy_version 1742518 (0.0009) [2023-12-27 04:03:45,451][105620] Updated weights for policy 1, policy_version 1742528 (0.0009) [2023-12-27 04:03:45,505][105620] Updated weights for policy 1, policy_version 1742538 (0.0009) [2023-12-27 04:03:45,614][105692] Updated weights for policy 0, policy_version 1738882 (0.0009) [2023-12-27 04:03:45,676][105692] Updated weights for policy 0, policy_version 1738892 (0.0009) [2023-12-27 04:03:45,737][105692] Updated weights for policy 0, policy_version 1738902 (0.0009) [2023-12-27 04:03:45,802][105692] Updated weights for policy 0, policy_version 1738912 (0.0009) [2023-12-27 04:03:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 891379712. Throughput: 0: 9800.0, 1: 9923.6. Samples: 891350648. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:03:46,062][104569] Avg episode reward: [(0, '8531.987'), (1, '9078.885')] [2023-12-27 04:03:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001738912_445227008.pth... [2023-12-27 04:03:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001742544_446152704.pth... [2023-12-27 04:03:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001737728_444923904.pth [2023-12-27 04:03:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001741392_445857792.pth [2023-12-27 04:03:46,178][105620] Updated weights for policy 1, policy_version 1742548 (0.0007) [2023-12-27 04:03:46,233][105620] Updated weights for policy 1, policy_version 1742558 (0.0005) [2023-12-27 04:03:46,284][105620] Updated weights for policy 1, policy_version 1742568 (0.0009) [2023-12-27 04:03:46,576][105692] Updated weights for policy 0, policy_version 1738922 (0.0007) [2023-12-27 04:03:46,637][105692] Updated weights for policy 0, policy_version 1738932 (0.0005) [2023-12-27 04:03:46,692][105692] Updated weights for policy 0, policy_version 1738942 (0.0005) [2023-12-27 04:03:46,887][105620] Updated weights for policy 1, policy_version 1742578 (0.0009) [2023-12-27 04:03:46,953][105620] Updated weights for policy 1, policy_version 1742588 (0.0008) [2023-12-27 04:03:47,010][105620] Updated weights for policy 1, policy_version 1742598 (0.0010) [2023-12-27 04:03:47,066][105620] Updated weights for policy 1, policy_version 1742608 (0.0010) [2023-12-27 04:03:47,283][105692] Updated weights for policy 0, policy_version 1738952 (0.0010) [2023-12-27 04:03:47,344][105692] Updated weights for policy 0, policy_version 1738962 (0.0010) [2023-12-27 04:03:47,389][105692] Updated weights for policy 0, policy_version 1738972 (0.0010) [2023-12-27 04:03:47,799][105620] Updated weights for policy 1, policy_version 1742618 (0.0006) [2023-12-27 04:03:47,868][105620] Updated weights for policy 1, policy_version 1742628 (0.0007) [2023-12-27 04:03:47,926][105620] Updated weights for policy 1, policy_version 1742638 (0.0009) [2023-12-27 04:03:48,026][105692] Updated weights for policy 0, policy_version 1738982 (0.0008) [2023-12-27 04:03:48,079][105692] Updated weights for policy 0, policy_version 1738992 (0.0009) [2023-12-27 04:03:48,131][105692] Updated weights for policy 0, policy_version 1739002 (0.0010) [2023-12-27 04:03:48,711][105620] Updated weights for policy 1, policy_version 1742648 (0.0008) [2023-12-27 04:03:48,774][105620] Updated weights for policy 1, policy_version 1742658 (0.0008) [2023-12-27 04:03:48,780][105692] Updated weights for policy 0, policy_version 1739012 (0.0010) [2023-12-27 04:03:48,830][105620] Updated weights for policy 1, policy_version 1742668 (0.0006) [2023-12-27 04:03:48,843][105692] Updated weights for policy 0, policy_version 1739022 (0.0010) [2023-12-27 04:03:48,917][105692] Updated weights for policy 0, policy_version 1739032 (0.0007) [2023-12-27 04:03:49,516][105692] Updated weights for policy 0, policy_version 1739042 (0.0007) [2023-12-27 04:03:49,569][105692] Updated weights for policy 0, policy_version 1739052 (0.0010) [2023-12-27 04:03:49,617][105692] Updated weights for policy 0, policy_version 1739062 (0.0010) [2023-12-27 04:03:49,656][105620] Updated weights for policy 1, policy_version 1742678 (0.0006) [2023-12-27 04:03:49,666][105692] Updated weights for policy 0, policy_version 1739072 (0.0011) [2023-12-27 04:03:49,711][105620] Updated weights for policy 1, policy_version 1742688 (0.0007) [2023-12-27 04:03:49,766][105620] Updated weights for policy 1, policy_version 1742698 (0.0007) [2023-12-27 04:03:50,411][105692] Updated weights for policy 0, policy_version 1739082 (0.0005) [2023-12-27 04:03:50,474][105692] Updated weights for policy 0, policy_version 1739092 (0.0006) [2023-12-27 04:03:50,483][105620] Updated weights for policy 1, policy_version 1742708 (0.0006) [2023-12-27 04:03:50,550][105620] Updated weights for policy 1, policy_version 1742718 (0.0006) [2023-12-27 04:03:50,582][105692] Updated weights for policy 0, policy_version 1739102 (0.0010) [2023-12-27 04:03:50,610][105620] Updated weights for policy 1, policy_version 1742728 (0.0006) [2023-12-27 04:03:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 891478016. Throughput: 0: 9957.5, 1: 9976.0. Samples: 891470740. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:03:51,063][104569] Avg episode reward: [(0, '8436.959'), (1, '9262.470')] [2023-12-27 04:03:51,239][105692] Updated weights for policy 0, policy_version 1739112 (0.0006) [2023-12-27 04:03:51,301][105692] Updated weights for policy 0, policy_version 1739122 (0.0006) [2023-12-27 04:03:51,313][105620] Updated weights for policy 1, policy_version 1742738 (0.0006) [2023-12-27 04:03:51,370][105692] Updated weights for policy 0, policy_version 1739132 (0.0008) [2023-12-27 04:03:51,372][105620] Updated weights for policy 1, policy_version 1742748 (0.0007) [2023-12-27 04:03:51,437][105620] Updated weights for policy 1, policy_version 1742758 (0.0008) [2023-12-27 04:03:51,503][105620] Updated weights for policy 1, policy_version 1742768 (0.0008) [2023-12-27 04:03:52,018][105692] Updated weights for policy 0, policy_version 1739142 (0.0012) [2023-12-27 04:03:52,077][105692] Updated weights for policy 0, policy_version 1739152 (0.0011) [2023-12-27 04:03:52,139][105692] Updated weights for policy 0, policy_version 1739162 (0.0010) [2023-12-27 04:03:52,311][105620] Updated weights for policy 1, policy_version 1742778 (0.0008) [2023-12-27 04:03:52,370][105620] Updated weights for policy 1, policy_version 1742788 (0.0008) [2023-12-27 04:03:52,427][105620] Updated weights for policy 1, policy_version 1742798 (0.0008) [2023-12-27 04:03:52,853][105692] Updated weights for policy 0, policy_version 1739172 (0.0009) [2023-12-27 04:03:52,920][105692] Updated weights for policy 0, policy_version 1739182 (0.0005) [2023-12-27 04:03:52,987][105692] Updated weights for policy 0, policy_version 1739192 (0.0006) [2023-12-27 04:03:53,233][105620] Updated weights for policy 1, policy_version 1742808 (0.0008) [2023-12-27 04:03:53,287][105620] Updated weights for policy 1, policy_version 1742818 (0.0008) [2023-12-27 04:03:53,340][105620] Updated weights for policy 1, policy_version 1742828 (0.0008) [2023-12-27 04:03:53,569][105692] Updated weights for policy 0, policy_version 1739202 (0.0007) [2023-12-27 04:03:53,625][105692] Updated weights for policy 0, policy_version 1739212 (0.0006) [2023-12-27 04:03:53,685][105692] Updated weights for policy 0, policy_version 1739222 (0.0006) [2023-12-27 04:03:53,734][105692] Updated weights for policy 0, policy_version 1739232 (0.0005) [2023-12-27 04:03:53,989][105620] Updated weights for policy 1, policy_version 1742838 (0.0007) [2023-12-27 04:03:54,054][105620] Updated weights for policy 1, policy_version 1742848 (0.0006) [2023-12-27 04:03:54,122][105620] Updated weights for policy 1, policy_version 1742858 (0.0006) [2023-12-27 04:03:54,271][105692] Updated weights for policy 0, policy_version 1739242 (0.0008) [2023-12-27 04:03:54,332][105692] Updated weights for policy 0, policy_version 1739252 (0.0006) [2023-12-27 04:03:54,388][105692] Updated weights for policy 0, policy_version 1739262 (0.0009) [2023-12-27 04:03:54,652][105620] Updated weights for policy 1, policy_version 1742868 (0.0006) [2023-12-27 04:03:54,704][105620] Updated weights for policy 1, policy_version 1742878 (0.0008) [2023-12-27 04:03:54,755][105620] Updated weights for policy 1, policy_version 1742888 (0.0008) [2023-12-27 04:03:55,163][105692] Updated weights for policy 0, policy_version 1739272 (0.0010) [2023-12-27 04:03:55,211][105692] Updated weights for policy 0, policy_version 1739282 (0.0010) [2023-12-27 04:03:55,256][105692] Updated weights for policy 0, policy_version 1739292 (0.0010) [2023-12-27 04:03:55,504][105620] Updated weights for policy 1, policy_version 1742898 (0.0008) [2023-12-27 04:03:55,562][105620] Updated weights for policy 1, policy_version 1742908 (0.0008) [2023-12-27 04:03:55,618][105620] Updated weights for policy 1, policy_version 1742918 (0.0007) [2023-12-27 04:03:55,687][105620] Updated weights for policy 1, policy_version 1742928 (0.0006) [2023-12-27 04:03:55,996][105692] Updated weights for policy 0, policy_version 1739302 (0.0010) [2023-12-27 04:03:56,045][105692] Updated weights for policy 0, policy_version 1739312 (0.0011) [2023-12-27 04:03:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 891576320. Throughput: 0: 10094.2, 1: 9909.7. Samples: 891590984. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:03:56,062][104569] Avg episode reward: [(0, '8620.025'), (1, '8803.917')] [2023-12-27 04:03:56,098][105692] Updated weights for policy 0, policy_version 1739322 (0.0011) [2023-12-27 04:03:56,301][105620] Updated weights for policy 1, policy_version 1742938 (0.0005) [2023-12-27 04:03:56,366][105620] Updated weights for policy 1, policy_version 1742948 (0.0006) [2023-12-27 04:03:56,432][105620] Updated weights for policy 1, policy_version 1742958 (0.0005) [2023-12-27 04:03:56,831][105692] Updated weights for policy 0, policy_version 1739332 (0.0011) [2023-12-27 04:03:56,896][105692] Updated weights for policy 0, policy_version 1739342 (0.0010) [2023-12-27 04:03:56,964][105692] Updated weights for policy 0, policy_version 1739352 (0.0010) [2023-12-27 04:03:57,030][105620] Updated weights for policy 1, policy_version 1742968 (0.0007) [2023-12-27 04:03:57,089][105620] Updated weights for policy 1, policy_version 1742978 (0.0008) [2023-12-27 04:03:57,152][105620] Updated weights for policy 1, policy_version 1742988 (0.0010) [2023-12-27 04:03:57,538][105692] Updated weights for policy 0, policy_version 1739362 (0.0008) [2023-12-27 04:03:57,586][105692] Updated weights for policy 0, policy_version 1739372 (0.0010) [2023-12-27 04:03:57,638][105692] Updated weights for policy 0, policy_version 1739382 (0.0011) [2023-12-27 04:03:57,700][105692] Updated weights for policy 0, policy_version 1739392 (0.0011) [2023-12-27 04:03:57,951][105620] Updated weights for policy 1, policy_version 1742998 (0.0006) [2023-12-27 04:03:58,019][105620] Updated weights for policy 1, policy_version 1743008 (0.0007) [2023-12-27 04:03:58,076][105620] Updated weights for policy 1, policy_version 1743018 (0.0009) [2023-12-27 04:03:58,417][105692] Updated weights for policy 0, policy_version 1739402 (0.0010) [2023-12-27 04:03:58,482][105692] Updated weights for policy 0, policy_version 1739412 (0.0009) [2023-12-27 04:03:58,552][105692] Updated weights for policy 0, policy_version 1739422 (0.0008) [2023-12-27 04:03:58,790][105620] Updated weights for policy 1, policy_version 1743028 (0.0009) [2023-12-27 04:03:58,861][105620] Updated weights for policy 1, policy_version 1743038 (0.0007) [2023-12-27 04:03:58,927][105620] Updated weights for policy 1, policy_version 1743048 (0.0006) [2023-12-27 04:03:59,306][105692] Updated weights for policy 0, policy_version 1739432 (0.0007) [2023-12-27 04:03:59,369][105692] Updated weights for policy 0, policy_version 1739442 (0.0008) [2023-12-27 04:03:59,434][105692] Updated weights for policy 0, policy_version 1739452 (0.0006) [2023-12-27 04:03:59,673][105620] Updated weights for policy 1, policy_version 1743058 (0.0009) [2023-12-27 04:03:59,735][105620] Updated weights for policy 1, policy_version 1743068 (0.0011) [2023-12-27 04:03:59,790][105620] Updated weights for policy 1, policy_version 1743078 (0.0010) [2023-12-27 04:03:59,852][105620] Updated weights for policy 1, policy_version 1743088 (0.0009) [2023-12-27 04:04:00,122][105692] Updated weights for policy 0, policy_version 1739462 (0.0007) [2023-12-27 04:04:00,178][105692] Updated weights for policy 0, policy_version 1739472 (0.0008) [2023-12-27 04:04:00,233][105692] Updated weights for policy 0, policy_version 1739482 (0.0008) [2023-12-27 04:04:00,624][105620] Updated weights for policy 1, policy_version 1743098 (0.0006) [2023-12-27 04:04:00,671][105620] Updated weights for policy 1, policy_version 1743108 (0.0005) [2023-12-27 04:04:00,714][105620] Updated weights for policy 1, policy_version 1743118 (0.0005) [2023-12-27 04:04:00,828][105692] Updated weights for policy 0, policy_version 1739492 (0.0007) [2023-12-27 04:04:00,883][105692] Updated weights for policy 0, policy_version 1739502 (0.0007) [2023-12-27 04:04:00,930][105692] Updated weights for policy 0, policy_version 1739512 (0.0006) [2023-12-27 04:04:01,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19933.8, 300 sec: 19549.7). Total num frames: 891682816. Throughput: 0: 10140.4, 1: 9907.3. Samples: 891649712. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:04:01,063][104569] Avg episode reward: [(0, '8713.281'), (1, '8436.473')] [2023-12-27 04:04:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001739520_445382656.pth... [2023-12-27 04:04:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001743120_446300160.pth... [2023-12-27 04:04:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001738336_445079552.pth [2023-12-27 04:04:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001741968_446005248.pth [2023-12-27 04:04:01,398][105620] Updated weights for policy 1, policy_version 1743128 (0.0008) [2023-12-27 04:04:01,456][105620] Updated weights for policy 1, policy_version 1743138 (0.0008) [2023-12-27 04:04:01,514][105620] Updated weights for policy 1, policy_version 1743148 (0.0008) [2023-12-27 04:04:01,650][105692] Updated weights for policy 0, policy_version 1739522 (0.0006) [2023-12-27 04:04:01,709][105692] Updated weights for policy 0, policy_version 1739532 (0.0010) [2023-12-27 04:04:01,773][105692] Updated weights for policy 0, policy_version 1739542 (0.0009) [2023-12-27 04:04:01,820][105692] Updated weights for policy 0, policy_version 1739552 (0.0008) [2023-12-27 04:04:02,286][105620] Updated weights for policy 1, policy_version 1743158 (0.0009) [2023-12-27 04:04:02,340][105620] Updated weights for policy 1, policy_version 1743168 (0.0009) [2023-12-27 04:04:02,398][105620] Updated weights for policy 1, policy_version 1743178 (0.0009) [2023-12-27 04:04:02,533][105692] Updated weights for policy 0, policy_version 1739562 (0.0005) [2023-12-27 04:04:02,586][105692] Updated weights for policy 0, policy_version 1739572 (0.0008) [2023-12-27 04:04:02,649][105692] Updated weights for policy 0, policy_version 1739582 (0.0010) [2023-12-27 04:04:03,189][105620] Updated weights for policy 1, policy_version 1743188 (0.0009) [2023-12-27 04:04:03,254][105620] Updated weights for policy 1, policy_version 1743198 (0.0008) [2023-12-27 04:04:03,312][105620] Updated weights for policy 1, policy_version 1743208 (0.0007) [2023-12-27 04:04:03,364][105692] Updated weights for policy 0, policy_version 1739592 (0.0010) [2023-12-27 04:04:03,415][105692] Updated weights for policy 0, policy_version 1739602 (0.0010) [2023-12-27 04:04:03,465][105692] Updated weights for policy 0, policy_version 1739612 (0.0010) [2023-12-27 04:04:03,943][105620] Updated weights for policy 1, policy_version 1743218 (0.0007) [2023-12-27 04:04:03,995][105620] Updated weights for policy 1, policy_version 1743228 (0.0010) [2023-12-27 04:04:04,050][105620] Updated weights for policy 1, policy_version 1743238 (0.0010) [2023-12-27 04:04:04,105][105620] Updated weights for policy 1, policy_version 1743248 (0.0010) [2023-12-27 04:04:04,231][105692] Updated weights for policy 0, policy_version 1739622 (0.0010) [2023-12-27 04:04:04,280][105692] Updated weights for policy 0, policy_version 1739632 (0.0010) [2023-12-27 04:04:04,329][105692] Updated weights for policy 0, policy_version 1739642 (0.0011) [2023-12-27 04:04:04,834][105620] Updated weights for policy 1, policy_version 1743259 (0.0008) [2023-12-27 04:04:04,889][105620] Updated weights for policy 1, policy_version 1743269 (0.0005) [2023-12-27 04:04:04,957][105620] Updated weights for policy 1, policy_version 1743279 (0.0005) [2023-12-27 04:04:05,064][105692] Updated weights for policy 0, policy_version 1739652 (0.0009) [2023-12-27 04:04:05,108][105692] Updated weights for policy 0, policy_version 1739662 (0.0005) [2023-12-27 04:04:05,173][105692] Updated weights for policy 0, policy_version 1739672 (0.0006) [2023-12-27 04:04:05,501][105620] Updated weights for policy 1, policy_version 1743289 (0.0006) [2023-12-27 04:04:05,555][105620] Updated weights for policy 1, policy_version 1743299 (0.0005) [2023-12-27 04:04:05,611][105620] Updated weights for policy 1, policy_version 1743309 (0.0005) [2023-12-27 04:04:05,793][105692] Updated weights for policy 0, policy_version 1739682 (0.0006) [2023-12-27 04:04:05,849][105692] Updated weights for policy 0, policy_version 1739692 (0.0009) [2023-12-27 04:04:05,907][105692] Updated weights for policy 0, policy_version 1739702 (0.0010) [2023-12-27 04:04:05,968][105692] Updated weights for policy 0, policy_version 1739712 (0.0007) [2023-12-27 04:04:06,062][104569] Fps is (10 sec: 20479.4, 60 sec: 20070.3, 300 sec: 19549.7). Total num frames: 891781120. Throughput: 0: 10005.0, 1: 9817.9. Samples: 891765356. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:04:06,063][104569] Avg episode reward: [(0, '8713.107'), (1, '8803.480')] [2023-12-27 04:04:06,194][105620] Updated weights for policy 1, policy_version 1743319 (0.0007) [2023-12-27 04:04:06,256][105620] Updated weights for policy 1, policy_version 1743329 (0.0007) [2023-12-27 04:04:06,324][105620] Updated weights for policy 1, policy_version 1743339 (0.0008) [2023-12-27 04:04:06,590][105692] Updated weights for policy 0, policy_version 1739722 (0.0009) [2023-12-27 04:04:06,651][105692] Updated weights for policy 0, policy_version 1739732 (0.0006) [2023-12-27 04:04:06,714][105692] Updated weights for policy 0, policy_version 1739742 (0.0005) [2023-12-27 04:04:07,135][105620] Updated weights for policy 1, policy_version 1743349 (0.0009) [2023-12-27 04:04:07,188][105620] Updated weights for policy 1, policy_version 1743359 (0.0008) [2023-12-27 04:04:07,238][105620] Updated weights for policy 1, policy_version 1743369 (0.0009) [2023-12-27 04:04:07,363][105692] Updated weights for policy 0, policy_version 1739752 (0.0006) [2023-12-27 04:04:07,431][105692] Updated weights for policy 0, policy_version 1739762 (0.0008) [2023-12-27 04:04:07,494][105692] Updated weights for policy 0, policy_version 1739772 (0.0009) [2023-12-27 04:04:08,023][105620] Updated weights for policy 1, policy_version 1743379 (0.0008) [2023-12-27 04:04:08,087][105620] Updated weights for policy 1, policy_version 1743389 (0.0009) [2023-12-27 04:04:08,148][105620] Updated weights for policy 1, policy_version 1743399 (0.0008) [2023-12-27 04:04:08,200][105692] Updated weights for policy 0, policy_version 1739782 (0.0009) [2023-12-27 04:04:08,254][105692] Updated weights for policy 0, policy_version 1739792 (0.0009) [2023-12-27 04:04:08,315][105692] Updated weights for policy 0, policy_version 1739802 (0.0009) [2023-12-27 04:04:08,928][105620] Updated weights for policy 1, policy_version 1743409 (0.0007) [2023-12-27 04:04:08,973][105620] Updated weights for policy 1, policy_version 1743419 (0.0008) [2023-12-27 04:04:09,026][105620] Updated weights for policy 1, policy_version 1743429 (0.0009) [2023-12-27 04:04:09,029][105692] Updated weights for policy 0, policy_version 1739812 (0.0009) [2023-12-27 04:04:09,075][105692] Updated weights for policy 0, policy_version 1739822 (0.0008) [2023-12-27 04:04:09,077][105620] Updated weights for policy 1, policy_version 1743439 (0.0009) [2023-12-27 04:04:09,129][105692] Updated weights for policy 0, policy_version 1739832 (0.0009) [2023-12-27 04:04:09,855][105620] Updated weights for policy 1, policy_version 1743449 (0.0008) [2023-12-27 04:04:09,917][105620] Updated weights for policy 1, policy_version 1743459 (0.0009) [2023-12-27 04:04:09,951][105692] Updated weights for policy 0, policy_version 1739842 (0.0008) [2023-12-27 04:04:09,986][105620] Updated weights for policy 1, policy_version 1743469 (0.0007) [2023-12-27 04:04:10,018][105692] Updated weights for policy 0, policy_version 1739852 (0.0010) [2023-12-27 04:04:10,080][105692] Updated weights for policy 0, policy_version 1739862 (0.0009) [2023-12-27 04:04:10,141][105692] Updated weights for policy 0, policy_version 1739872 (0.0009) [2023-12-27 04:04:10,721][105620] Updated weights for policy 1, policy_version 1743479 (0.0008) [2023-12-27 04:04:10,783][105620] Updated weights for policy 1, policy_version 1743489 (0.0009) [2023-12-27 04:04:10,831][105620] Updated weights for policy 1, policy_version 1743499 (0.0008) [2023-12-27 04:04:10,899][105692] Updated weights for policy 0, policy_version 1739882 (0.0009) [2023-12-27 04:04:10,954][105692] Updated weights for policy 0, policy_version 1739892 (0.0009) [2023-12-27 04:04:11,013][105692] Updated weights for policy 0, policy_version 1739902 (0.0009) [2023-12-27 04:04:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 891879424. Throughput: 0: 9963.9, 1: 9737.8. Samples: 891882500. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:04:11,063][104569] Avg episode reward: [(0, '8802.756'), (1, '9078.273')] [2023-12-27 04:04:11,575][105620] Updated weights for policy 1, policy_version 1743509 (0.0006) [2023-12-27 04:04:11,649][105620] Updated weights for policy 1, policy_version 1743519 (0.0007) [2023-12-27 04:04:11,707][105620] Updated weights for policy 1, policy_version 1743529 (0.0007) [2023-12-27 04:04:11,857][105692] Updated weights for policy 0, policy_version 1739912 (0.0008) [2023-12-27 04:04:11,921][105692] Updated weights for policy 0, policy_version 1739922 (0.0008) [2023-12-27 04:04:11,984][105692] Updated weights for policy 0, policy_version 1739932 (0.0008) [2023-12-27 04:04:12,372][105620] Updated weights for policy 1, policy_version 1743539 (0.0009) [2023-12-27 04:04:12,435][105620] Updated weights for policy 1, policy_version 1743549 (0.0009) [2023-12-27 04:04:12,490][105620] Updated weights for policy 1, policy_version 1743559 (0.0009) [2023-12-27 04:04:12,734][105692] Updated weights for policy 0, policy_version 1739942 (0.0010) [2023-12-27 04:04:12,794][105692] Updated weights for policy 0, policy_version 1739952 (0.0011) [2023-12-27 04:04:12,853][105692] Updated weights for policy 0, policy_version 1739962 (0.0010) [2023-12-27 04:04:13,292][105620] Updated weights for policy 1, policy_version 1743569 (0.0009) [2023-12-27 04:04:13,340][105620] Updated weights for policy 1, policy_version 1743579 (0.0008) [2023-12-27 04:04:13,395][105620] Updated weights for policy 1, policy_version 1743589 (0.0008) [2023-12-27 04:04:13,457][105620] Updated weights for policy 1, policy_version 1743599 (0.0008) [2023-12-27 04:04:13,596][105692] Updated weights for policy 0, policy_version 1739972 (0.0010) [2023-12-27 04:04:13,650][105692] Updated weights for policy 0, policy_version 1739982 (0.0010) [2023-12-27 04:04:13,707][105692] Updated weights for policy 0, policy_version 1739992 (0.0010) [2023-12-27 04:04:14,243][105620] Updated weights for policy 1, policy_version 1743609 (0.0006) [2023-12-27 04:04:14,303][105620] Updated weights for policy 1, policy_version 1743619 (0.0005) [2023-12-27 04:04:14,360][105692] Updated weights for policy 0, policy_version 1740002 (0.0010) [2023-12-27 04:04:14,361][105620] Updated weights for policy 1, policy_version 1743629 (0.0007) [2023-12-27 04:04:14,421][105692] Updated weights for policy 0, policy_version 1740012 (0.0010) [2023-12-27 04:04:14,479][105692] Updated weights for policy 0, policy_version 1740022 (0.0010) [2023-12-27 04:04:14,543][105692] Updated weights for policy 0, policy_version 1740032 (0.0010) [2023-12-27 04:04:15,015][105620] Updated weights for policy 1, policy_version 1743639 (0.0010) [2023-12-27 04:04:15,081][105620] Updated weights for policy 1, policy_version 1743649 (0.0011) [2023-12-27 04:04:15,154][105620] Updated weights for policy 1, policy_version 1743659 (0.0011) [2023-12-27 04:04:15,236][105692] Updated weights for policy 0, policy_version 1740042 (0.0009) [2023-12-27 04:04:15,292][105692] Updated weights for policy 0, policy_version 1740052 (0.0009) [2023-12-27 04:04:15,346][105692] Updated weights for policy 0, policy_version 1740062 (0.0011) [2023-12-27 04:04:15,806][105620] Updated weights for policy 1, policy_version 1743669 (0.0009) [2023-12-27 04:04:15,854][105620] Updated weights for policy 1, policy_version 1743679 (0.0005) [2023-12-27 04:04:15,910][105620] Updated weights for policy 1, policy_version 1743689 (0.0005) [2023-12-27 04:04:16,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 891969536. Throughput: 0: 9871.9, 1: 9716.5. Samples: 891938644. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:04:16,063][104569] Avg episode reward: [(0, '8073.218'), (1, '9262.020')] [2023-12-27 04:04:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001743696_446447616.pth... [2023-12-27 04:04:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001742544_446152704.pth [2023-12-27 04:04:16,090][105692] Updated weights for policy 0, policy_version 1740072 (0.0010) [2023-12-27 04:04:16,135][105692] Updated weights for policy 0, policy_version 1740082 (0.0010) [2023-12-27 04:04:16,187][105692] Updated weights for policy 0, policy_version 1740092 (0.0010) [2023-12-27 04:04:16,202][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001740096_445530112.pth... [2023-12-27 04:04:16,205][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001738912_445227008.pth [2023-12-27 04:04:16,551][105620] Updated weights for policy 1, policy_version 1743699 (0.0008) [2023-12-27 04:04:16,606][105620] Updated weights for policy 1, policy_version 1743710 (0.0009) [2023-12-27 04:04:16,657][105620] Updated weights for policy 1, policy_version 1743720 (0.0009) [2023-12-27 04:04:16,923][105692] Updated weights for policy 0, policy_version 1740102 (0.0009) [2023-12-27 04:04:16,970][105692] Updated weights for policy 0, policy_version 1740112 (0.0009) [2023-12-27 04:04:17,023][105692] Updated weights for policy 0, policy_version 1740123 (0.0009) [2023-12-27 04:04:17,424][105620] Updated weights for policy 1, policy_version 1743730 (0.0008) [2023-12-27 04:04:17,474][105620] Updated weights for policy 1, policy_version 1743740 (0.0009) [2023-12-27 04:04:17,524][105620] Updated weights for policy 1, policy_version 1743750 (0.0009) [2023-12-27 04:04:17,581][105620] Updated weights for policy 1, policy_version 1743760 (0.0009) [2023-12-27 04:04:17,809][105692] Updated weights for policy 0, policy_version 1740133 (0.0007) [2023-12-27 04:04:17,859][105692] Updated weights for policy 0, policy_version 1740143 (0.0005) [2023-12-27 04:04:17,913][105692] Updated weights for policy 0, policy_version 1740153 (0.0005) [2023-12-27 04:04:18,360][105620] Updated weights for policy 1, policy_version 1743770 (0.0010) [2023-12-27 04:04:18,421][105620] Updated weights for policy 1, policy_version 1743780 (0.0010) [2023-12-27 04:04:18,481][105620] Updated weights for policy 1, policy_version 1743790 (0.0009) [2023-12-27 04:04:18,506][105692] Updated weights for policy 0, policy_version 1740163 (0.0005) [2023-12-27 04:04:18,562][105692] Updated weights for policy 0, policy_version 1740173 (0.0006) [2023-12-27 04:04:18,621][105692] Updated weights for policy 0, policy_version 1740183 (0.0006) [2023-12-27 04:04:19,222][105692] Updated weights for policy 0, policy_version 1740193 (0.0006) [2023-12-27 04:04:19,278][105692] Updated weights for policy 0, policy_version 1740203 (0.0007) [2023-12-27 04:04:19,293][105620] Updated weights for policy 1, policy_version 1743800 (0.0009) [2023-12-27 04:04:19,346][105692] Updated weights for policy 0, policy_version 1740213 (0.0008) [2023-12-27 04:04:19,355][105620] Updated weights for policy 1, policy_version 1743810 (0.0007) [2023-12-27 04:04:19,404][105692] Updated weights for policy 0, policy_version 1740223 (0.0007) [2023-12-27 04:04:19,418][105620] Updated weights for policy 1, policy_version 1743820 (0.0009) [2023-12-27 04:04:20,201][105692] Updated weights for policy 0, policy_version 1740233 (0.0007) [2023-12-27 04:04:20,207][105620] Updated weights for policy 1, policy_version 1743830 (0.0010) [2023-12-27 04:04:20,263][105692] Updated weights for policy 0, policy_version 1740243 (0.0006) [2023-12-27 04:04:20,265][105620] Updated weights for policy 1, policy_version 1743840 (0.0008) [2023-12-27 04:04:20,317][105620] Updated weights for policy 1, policy_version 1743850 (0.0008) [2023-12-27 04:04:20,327][105692] Updated weights for policy 0, policy_version 1740253 (0.0009) [2023-12-27 04:04:21,062][104569] Fps is (10 sec: 18022.6, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 892059648. Throughput: 0: 9852.2, 1: 9718.4. Samples: 892056064. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:04:21,063][104569] Avg episode reward: [(0, '8164.815'), (1, '9267.218')] [2023-12-27 04:04:21,100][105692] Updated weights for policy 0, policy_version 1740263 (0.0009) [2023-12-27 04:04:21,111][105620] Updated weights for policy 1, policy_version 1743860 (0.0009) [2023-12-27 04:04:21,169][105692] Updated weights for policy 0, policy_version 1740273 (0.0008) [2023-12-27 04:04:21,180][105620] Updated weights for policy 1, policy_version 1743870 (0.0007) [2023-12-27 04:04:21,225][105692] Updated weights for policy 0, policy_version 1740283 (0.0008) [2023-12-27 04:04:21,245][105620] Updated weights for policy 1, policy_version 1743880 (0.0008) [2023-12-27 04:04:21,971][105620] Updated weights for policy 1, policy_version 1743890 (0.0008) [2023-12-27 04:04:22,034][105620] Updated weights for policy 1, policy_version 1743900 (0.0008) [2023-12-27 04:04:22,058][105692] Updated weights for policy 0, policy_version 1740293 (0.0008) [2023-12-27 04:04:22,094][105620] Updated weights for policy 1, policy_version 1743910 (0.0007) [2023-12-27 04:04:22,120][105692] Updated weights for policy 0, policy_version 1740303 (0.0008) [2023-12-27 04:04:22,148][105620] Updated weights for policy 1, policy_version 1743920 (0.0006) [2023-12-27 04:04:22,182][105692] Updated weights for policy 0, policy_version 1740313 (0.0008) [2023-12-27 04:04:22,863][105620] Updated weights for policy 1, policy_version 1743930 (0.0006) [2023-12-27 04:04:22,911][105620] Updated weights for policy 1, policy_version 1743940 (0.0009) [2023-12-27 04:04:22,952][105692] Updated weights for policy 0, policy_version 1740323 (0.0009) [2023-12-27 04:04:22,965][105620] Updated weights for policy 1, policy_version 1743950 (0.0008) [2023-12-27 04:04:23,016][105692] Updated weights for policy 0, policy_version 1740333 (0.0009) [2023-12-27 04:04:23,082][105692] Updated weights for policy 0, policy_version 1740343 (0.0009) [2023-12-27 04:04:23,613][105620] Updated weights for policy 1, policy_version 1743960 (0.0009) [2023-12-27 04:04:23,665][105620] Updated weights for policy 1, policy_version 1743970 (0.0009) [2023-12-27 04:04:23,719][105620] Updated weights for policy 1, policy_version 1743982 (0.0010) [2023-12-27 04:04:23,807][105692] Updated weights for policy 0, policy_version 1740353 (0.0009) [2023-12-27 04:04:23,856][105692] Updated weights for policy 0, policy_version 1740363 (0.0007) [2023-12-27 04:04:23,904][105692] Updated weights for policy 0, policy_version 1740373 (0.0009) [2023-12-27 04:04:23,969][105692] Updated weights for policy 0, policy_version 1740383 (0.0008) [2023-12-27 04:04:24,504][105620] Updated weights for policy 1, policy_version 1743992 (0.0009) [2023-12-27 04:04:24,563][105620] Updated weights for policy 1, policy_version 1744002 (0.0009) [2023-12-27 04:04:24,622][105620] Updated weights for policy 1, policy_version 1744012 (0.0011) [2023-12-27 04:04:24,683][105692] Updated weights for policy 0, policy_version 1740393 (0.0008) [2023-12-27 04:04:24,727][105692] Updated weights for policy 0, policy_version 1740403 (0.0007) [2023-12-27 04:04:24,780][105692] Updated weights for policy 0, policy_version 1740413 (0.0008) [2023-12-27 04:04:25,333][105620] Updated weights for policy 1, policy_version 1744022 (0.0007) [2023-12-27 04:04:25,382][105620] Updated weights for policy 1, policy_version 1744032 (0.0005) [2023-12-27 04:04:25,437][105620] Updated weights for policy 1, policy_version 1744042 (0.0005) [2023-12-27 04:04:25,595][105692] Updated weights for policy 0, policy_version 1740423 (0.0009) [2023-12-27 04:04:25,648][105692] Updated weights for policy 0, policy_version 1740433 (0.0009) [2023-12-27 04:04:25,707][105692] Updated weights for policy 0, policy_version 1740443 (0.0008) [2023-12-27 04:04:26,056][105620] Updated weights for policy 1, policy_version 1744052 (0.0007) [2023-12-27 04:04:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 892157952. Throughput: 0: 9767.3, 1: 9675.1. Samples: 892169404. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:04:26,062][104569] Avg episode reward: [(0, '8896.411'), (1, '9267.098')] [2023-12-27 04:04:26,111][105620] Updated weights for policy 1, policy_version 1744062 (0.0010) [2023-12-27 04:04:26,166][105620] Updated weights for policy 1, policy_version 1744072 (0.0011) [2023-12-27 04:04:26,498][105692] Updated weights for policy 0, policy_version 1740453 (0.0009) [2023-12-27 04:04:26,557][105692] Updated weights for policy 0, policy_version 1740463 (0.0009) [2023-12-27 04:04:26,620][105692] Updated weights for policy 0, policy_version 1740473 (0.0010) [2023-12-27 04:04:26,802][105620] Updated weights for policy 1, policy_version 1744082 (0.0010) [2023-12-27 04:04:26,860][105620] Updated weights for policy 1, policy_version 1744092 (0.0011) [2023-12-27 04:04:26,920][105620] Updated weights for policy 1, policy_version 1744102 (0.0011) [2023-12-27 04:04:26,986][105620] Updated weights for policy 1, policy_version 1744112 (0.0011) [2023-12-27 04:04:27,448][105692] Updated weights for policy 0, policy_version 1740483 (0.0010) [2023-12-27 04:04:27,506][105692] Updated weights for policy 0, policy_version 1740493 (0.0010) [2023-12-27 04:04:27,564][105692] Updated weights for policy 0, policy_version 1740503 (0.0010) [2023-12-27 04:04:27,638][105620] Updated weights for policy 1, policy_version 1744122 (0.0005) [2023-12-27 04:04:27,691][105620] Updated weights for policy 1, policy_version 1744132 (0.0005) [2023-12-27 04:04:27,736][105620] Updated weights for policy 1, policy_version 1744142 (0.0005) [2023-12-27 04:04:28,152][105692] Updated weights for policy 0, policy_version 1740513 (0.0010) [2023-12-27 04:04:28,202][105692] Updated weights for policy 0, policy_version 1740523 (0.0005) [2023-12-27 04:04:28,250][105692] Updated weights for policy 0, policy_version 1740533 (0.0005) [2023-12-27 04:04:28,295][105692] Updated weights for policy 0, policy_version 1740543 (0.0005) [2023-12-27 04:04:28,337][105620] Updated weights for policy 1, policy_version 1744152 (0.0006) [2023-12-27 04:04:28,391][105620] Updated weights for policy 1, policy_version 1744162 (0.0009) [2023-12-27 04:04:28,447][105620] Updated weights for policy 1, policy_version 1744172 (0.0011) [2023-12-27 04:04:28,997][105692] Updated weights for policy 0, policy_version 1740553 (0.0010) [2023-12-27 04:04:29,042][105692] Updated weights for policy 0, policy_version 1740563 (0.0010) [2023-12-27 04:04:29,086][105692] Updated weights for policy 0, policy_version 1740573 (0.0010) [2023-12-27 04:04:29,193][105620] Updated weights for policy 1, policy_version 1744182 (0.0009) [2023-12-27 04:04:29,258][105620] Updated weights for policy 1, policy_version 1744192 (0.0008) [2023-12-27 04:04:29,313][105620] Updated weights for policy 1, policy_version 1744202 (0.0008) [2023-12-27 04:04:29,834][105692] Updated weights for policy 0, policy_version 1740583 (0.0010) [2023-12-27 04:04:29,889][105692] Updated weights for policy 0, policy_version 1740593 (0.0009) [2023-12-27 04:04:29,948][105692] Updated weights for policy 0, policy_version 1740603 (0.0008) [2023-12-27 04:04:30,059][105620] Updated weights for policy 1, policy_version 1744212 (0.0007) [2023-12-27 04:04:30,116][105620] Updated weights for policy 1, policy_version 1744222 (0.0005) [2023-12-27 04:04:30,176][105620] Updated weights for policy 1, policy_version 1744232 (0.0007) [2023-12-27 04:04:30,628][105692] Updated weights for policy 0, policy_version 1740613 (0.0007) [2023-12-27 04:04:30,686][105692] Updated weights for policy 0, policy_version 1740623 (0.0005) [2023-12-27 04:04:30,737][105692] Updated weights for policy 0, policy_version 1740633 (0.0005) [2023-12-27 04:04:30,908][105620] Updated weights for policy 1, policy_version 1744242 (0.0009) [2023-12-27 04:04:30,960][105620] Updated weights for policy 1, policy_version 1744252 (0.0008) [2023-12-27 04:04:31,017][105620] Updated weights for policy 1, policy_version 1744262 (0.0006) [2023-12-27 04:04:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 892256256. Throughput: 0: 9786.2, 1: 9756.1. Samples: 892230052. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:04:31,063][104569] Avg episode reward: [(0, '8895.260'), (1, '9354.103')] [2023-12-27 04:04:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001740640_445669376.pth... [2023-12-27 04:04:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001739520_445382656.pth [2023-12-27 04:04:31,083][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001744272_446595072.pth... [2023-12-27 04:04:31,085][105620] Updated weights for policy 1, policy_version 1744272 (0.0010) [2023-12-27 04:04:31,086][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001743120_446300160.pth [2023-12-27 04:04:31,423][105692] Updated weights for policy 0, policy_version 1740643 (0.0006) [2023-12-27 04:04:31,483][105692] Updated weights for policy 0, policy_version 1740653 (0.0008) [2023-12-27 04:04:31,551][105692] Updated weights for policy 0, policy_version 1740663 (0.0009) [2023-12-27 04:04:31,803][105620] Updated weights for policy 1, policy_version 1744282 (0.0009) [2023-12-27 04:04:31,865][105620] Updated weights for policy 1, policy_version 1744292 (0.0009) [2023-12-27 04:04:31,915][105620] Updated weights for policy 1, policy_version 1744302 (0.0010) [2023-12-27 04:04:32,158][105692] Updated weights for policy 0, policy_version 1740673 (0.0006) [2023-12-27 04:04:32,225][105692] Updated weights for policy 0, policy_version 1740683 (0.0008) [2023-12-27 04:04:32,282][105692] Updated weights for policy 0, policy_version 1740693 (0.0009) [2023-12-27 04:04:32,334][105692] Updated weights for policy 0, policy_version 1740703 (0.0009) [2023-12-27 04:04:32,574][105620] Updated weights for policy 1, policy_version 1744312 (0.0006) [2023-12-27 04:04:32,629][105620] Updated weights for policy 1, policy_version 1744322 (0.0005) [2023-12-27 04:04:32,690][105620] Updated weights for policy 1, policy_version 1744332 (0.0005) [2023-12-27 04:04:33,098][105692] Updated weights for policy 0, policy_version 1740713 (0.0010) [2023-12-27 04:04:33,168][105692] Updated weights for policy 0, policy_version 1740723 (0.0010) [2023-12-27 04:04:33,239][105692] Updated weights for policy 0, policy_version 1740733 (0.0010) [2023-12-27 04:04:33,295][105620] Updated weights for policy 1, policy_version 1744342 (0.0006) [2023-12-27 04:04:33,349][105620] Updated weights for policy 1, policy_version 1744352 (0.0005) [2023-12-27 04:04:33,408][105620] Updated weights for policy 1, policy_version 1744362 (0.0005) [2023-12-27 04:04:33,940][105620] Updated weights for policy 1, policy_version 1744372 (0.0007) [2023-12-27 04:04:34,004][105620] Updated weights for policy 1, policy_version 1744382 (0.0009) [2023-12-27 04:04:34,031][105692] Updated weights for policy 0, policy_version 1740743 (0.0009) [2023-12-27 04:04:34,062][105620] Updated weights for policy 1, policy_version 1744392 (0.0008) [2023-12-27 04:04:34,076][105692] Updated weights for policy 0, policy_version 1740753 (0.0006) [2023-12-27 04:04:34,134][105692] Updated weights for policy 0, policy_version 1740763 (0.0009) [2023-12-27 04:04:34,748][105620] Updated weights for policy 1, policy_version 1744402 (0.0007) [2023-12-27 04:04:34,809][105620] Updated weights for policy 1, policy_version 1744412 (0.0008) [2023-12-27 04:04:34,871][105620] Updated weights for policy 1, policy_version 1744422 (0.0008) [2023-12-27 04:04:34,931][105620] Updated weights for policy 1, policy_version 1744432 (0.0010) [2023-12-27 04:04:34,999][105692] Updated weights for policy 0, policy_version 1740773 (0.0010) [2023-12-27 04:04:35,056][105692] Updated weights for policy 0, policy_version 1740783 (0.0009) [2023-12-27 04:04:35,104][105692] Updated weights for policy 0, policy_version 1740793 (0.0008) [2023-12-27 04:04:35,624][105620] Updated weights for policy 1, policy_version 1744442 (0.0010) [2023-12-27 04:04:35,676][105620] Updated weights for policy 1, policy_version 1744452 (0.0010) [2023-12-27 04:04:35,724][105620] Updated weights for policy 1, policy_version 1744462 (0.0010) [2023-12-27 04:04:35,903][105692] Updated weights for policy 0, policy_version 1740803 (0.0008) [2023-12-27 04:04:35,963][105692] Updated weights for policy 0, policy_version 1740813 (0.0008) [2023-12-27 04:04:36,016][105692] Updated weights for policy 0, policy_version 1740823 (0.0008) [2023-12-27 04:04:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 892354560. Throughput: 0: 9733.8, 1: 9783.3. Samples: 892349008. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:04:36,063][104569] Avg episode reward: [(0, '8533.843'), (1, '9169.106')] [2023-12-27 04:04:36,495][105620] Updated weights for policy 1, policy_version 1744472 (0.0010) [2023-12-27 04:04:36,570][105620] Updated weights for policy 1, policy_version 1744482 (0.0006) [2023-12-27 04:04:36,639][105620] Updated weights for policy 1, policy_version 1744492 (0.0005) [2023-12-27 04:04:36,706][105692] Updated weights for policy 0, policy_version 1740833 (0.0008) [2023-12-27 04:04:36,767][105692] Updated weights for policy 0, policy_version 1740843 (0.0006) [2023-12-27 04:04:36,832][105692] Updated weights for policy 0, policy_version 1740853 (0.0008) [2023-12-27 04:04:36,899][105692] Updated weights for policy 0, policy_version 1740863 (0.0007) [2023-12-27 04:04:37,196][105620] Updated weights for policy 1, policy_version 1744502 (0.0008) [2023-12-27 04:04:37,241][105620] Updated weights for policy 1, policy_version 1744512 (0.0010) [2023-12-27 04:04:37,296][105620] Updated weights for policy 1, policy_version 1744522 (0.0010) [2023-12-27 04:04:37,419][105692] Updated weights for policy 0, policy_version 1740873 (0.0005) [2023-12-27 04:04:37,480][105692] Updated weights for policy 0, policy_version 1740883 (0.0007) [2023-12-27 04:04:37,543][105692] Updated weights for policy 0, policy_version 1740893 (0.0009) [2023-12-27 04:04:37,994][105620] Updated weights for policy 1, policy_version 1744532 (0.0008) [2023-12-27 04:04:38,051][105620] Updated weights for policy 1, policy_version 1744542 (0.0008) [2023-12-27 04:04:38,114][105620] Updated weights for policy 1, policy_version 1744552 (0.0009) [2023-12-27 04:04:38,233][105692] Updated weights for policy 0, policy_version 1740903 (0.0008) [2023-12-27 04:04:38,295][105692] Updated weights for policy 0, policy_version 1740913 (0.0009) [2023-12-27 04:04:38,359][105692] Updated weights for policy 0, policy_version 1740923 (0.0009) [2023-12-27 04:04:38,860][105620] Updated weights for policy 1, policy_version 1744562 (0.0009) [2023-12-27 04:04:38,916][105620] Updated weights for policy 1, policy_version 1744572 (0.0009) [2023-12-27 04:04:38,969][105620] Updated weights for policy 1, policy_version 1744582 (0.0009) [2023-12-27 04:04:39,012][105692] Updated weights for policy 0, policy_version 1740933 (0.0007) [2023-12-27 04:04:39,018][105620] Updated weights for policy 1, policy_version 1744592 (0.0008) [2023-12-27 04:04:39,064][105692] Updated weights for policy 0, policy_version 1740943 (0.0005) [2023-12-27 04:04:39,126][105692] Updated weights for policy 0, policy_version 1740953 (0.0005) [2023-12-27 04:04:39,802][105620] Updated weights for policy 1, policy_version 1744602 (0.0009) [2023-12-27 04:04:39,836][105692] Updated weights for policy 0, policy_version 1740963 (0.0007) [2023-12-27 04:04:39,869][105620] Updated weights for policy 1, policy_version 1744612 (0.0008) [2023-12-27 04:04:39,893][105692] Updated weights for policy 0, policy_version 1740973 (0.0010) [2023-12-27 04:04:39,929][105620] Updated weights for policy 1, policy_version 1744622 (0.0008) [2023-12-27 04:04:39,960][105692] Updated weights for policy 0, policy_version 1740983 (0.0011) [2023-12-27 04:04:40,629][105692] Updated weights for policy 0, policy_version 1740993 (0.0010) [2023-12-27 04:04:40,661][105620] Updated weights for policy 1, policy_version 1744632 (0.0005) [2023-12-27 04:04:40,693][105692] Updated weights for policy 0, policy_version 1741003 (0.0006) [2023-12-27 04:04:40,708][105620] Updated weights for policy 1, policy_version 1744642 (0.0005) [2023-12-27 04:04:40,757][105692] Updated weights for policy 0, policy_version 1741013 (0.0009) [2023-12-27 04:04:40,772][105620] Updated weights for policy 1, policy_version 1744652 (0.0005) [2023-12-27 04:04:40,823][105692] Updated weights for policy 0, policy_version 1741023 (0.0010) [2023-12-27 04:04:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 892461056. Throughput: 0: 9720.0, 1: 9782.5. Samples: 892468600. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:04:41,063][104569] Avg episode reward: [(0, '8531.329'), (1, '9076.769')] [2023-12-27 04:04:41,401][105620] Updated weights for policy 1, policy_version 1744662 (0.0008) [2023-12-27 04:04:41,469][105620] Updated weights for policy 1, policy_version 1744672 (0.0011) [2023-12-27 04:04:41,534][105620] Updated weights for policy 1, policy_version 1744682 (0.0011) [2023-12-27 04:04:41,534][105692] Updated weights for policy 0, policy_version 1741033 (0.0011) [2023-12-27 04:04:41,592][105692] Updated weights for policy 0, policy_version 1741043 (0.0009) [2023-12-27 04:04:41,661][105692] Updated weights for policy 0, policy_version 1741053 (0.0009) [2023-12-27 04:04:42,307][105620] Updated weights for policy 1, policy_version 1744692 (0.0011) [2023-12-27 04:04:42,368][105620] Updated weights for policy 1, policy_version 1744702 (0.0011) [2023-12-27 04:04:42,421][105620] Updated weights for policy 1, policy_version 1744712 (0.0010) [2023-12-27 04:04:42,437][105692] Updated weights for policy 0, policy_version 1741063 (0.0010) [2023-12-27 04:04:42,499][105692] Updated weights for policy 0, policy_version 1741073 (0.0010) [2023-12-27 04:04:42,561][105692] Updated weights for policy 0, policy_version 1741083 (0.0011) [2023-12-27 04:04:43,170][105620] Updated weights for policy 1, policy_version 1744722 (0.0010) [2023-12-27 04:04:43,221][105620] Updated weights for policy 1, policy_version 1744732 (0.0010) [2023-12-27 04:04:43,269][105620] Updated weights for policy 1, policy_version 1744742 (0.0010) [2023-12-27 04:04:43,308][105692] Updated weights for policy 0, policy_version 1741093 (0.0010) [2023-12-27 04:04:43,321][105620] Updated weights for policy 1, policy_version 1744752 (0.0010) [2023-12-27 04:04:43,360][105692] Updated weights for policy 0, policy_version 1741103 (0.0010) [2023-12-27 04:04:43,408][105692] Updated weights for policy 0, policy_version 1741113 (0.0008) [2023-12-27 04:04:43,965][105692] Updated weights for policy 0, policy_version 1741123 (0.0006) [2023-12-27 04:04:43,969][105620] Updated weights for policy 1, policy_version 1744762 (0.0005) [2023-12-27 04:04:44,014][105692] Updated weights for policy 0, policy_version 1741133 (0.0005) [2023-12-27 04:04:44,019][105620] Updated weights for policy 1, policy_version 1744772 (0.0005) [2023-12-27 04:04:44,070][105620] Updated weights for policy 1, policy_version 1744782 (0.0007) [2023-12-27 04:04:44,075][105692] Updated weights for policy 0, policy_version 1741143 (0.0007) [2023-12-27 04:04:44,628][105620] Updated weights for policy 1, policy_version 1744792 (0.0006) [2023-12-27 04:04:44,659][105692] Updated weights for policy 0, policy_version 1741153 (0.0007) [2023-12-27 04:04:44,691][105620] Updated weights for policy 1, policy_version 1744802 (0.0007) [2023-12-27 04:04:44,729][105692] Updated weights for policy 0, policy_version 1741163 (0.0010) [2023-12-27 04:04:44,753][105620] Updated weights for policy 1, policy_version 1744812 (0.0005) [2023-12-27 04:04:44,791][105692] Updated weights for policy 0, policy_version 1741173 (0.0012) [2023-12-27 04:04:44,850][105692] Updated weights for policy 0, policy_version 1741183 (0.0010) [2023-12-27 04:04:45,458][105620] Updated weights for policy 1, policy_version 1744822 (0.0009) [2023-12-27 04:04:45,516][105620] Updated weights for policy 1, policy_version 1744832 (0.0010) [2023-12-27 04:04:45,570][105692] Updated weights for policy 0, policy_version 1741193 (0.0010) [2023-12-27 04:04:45,578][105620] Updated weights for policy 1, policy_version 1744842 (0.0010) [2023-12-27 04:04:45,616][105692] Updated weights for policy 0, policy_version 1741203 (0.0009) [2023-12-27 04:04:45,664][105692] Updated weights for policy 0, policy_version 1741213 (0.0007) [2023-12-27 04:04:46,062][104569] Fps is (10 sec: 20479.3, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 892559360. Throughput: 0: 9680.0, 1: 9793.9. Samples: 892526044. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:04:46,063][104569] Avg episode reward: [(0, '8707.868'), (1, '9079.009')] [2023-12-27 04:04:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001741216_445816832.pth... [2023-12-27 04:04:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001744848_446742528.pth... [2023-12-27 04:04:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001743696_446447616.pth [2023-12-27 04:04:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001740096_445530112.pth [2023-12-27 04:04:46,271][105620] Updated weights for policy 1, policy_version 1744852 (0.0011) [2023-12-27 04:04:46,322][105620] Updated weights for policy 1, policy_version 1744862 (0.0010) [2023-12-27 04:04:46,377][105620] Updated weights for policy 1, policy_version 1744872 (0.0010) [2023-12-27 04:04:46,427][105692] Updated weights for policy 0, policy_version 1741223 (0.0010) [2023-12-27 04:04:46,475][105692] Updated weights for policy 0, policy_version 1741233 (0.0010) [2023-12-27 04:04:46,523][105692] Updated weights for policy 0, policy_version 1741243 (0.0010) [2023-12-27 04:04:47,023][105620] Updated weights for policy 1, policy_version 1744882 (0.0009) [2023-12-27 04:04:47,089][105620] Updated weights for policy 1, policy_version 1744892 (0.0008) [2023-12-27 04:04:47,143][105620] Updated weights for policy 1, policy_version 1744902 (0.0006) [2023-12-27 04:04:47,204][105620] Updated weights for policy 1, policy_version 1744912 (0.0005) [2023-12-27 04:04:47,289][105692] Updated weights for policy 0, policy_version 1741253 (0.0010) [2023-12-27 04:04:47,353][105692] Updated weights for policy 0, policy_version 1741263 (0.0009) [2023-12-27 04:04:47,412][105692] Updated weights for policy 0, policy_version 1741273 (0.0009) [2023-12-27 04:04:47,871][105620] Updated weights for policy 1, policy_version 1744922 (0.0006) [2023-12-27 04:04:47,939][105620] Updated weights for policy 1, policy_version 1744932 (0.0006) [2023-12-27 04:04:47,967][105692] Updated weights for policy 0, policy_version 1741283 (0.0007) [2023-12-27 04:04:47,996][105620] Updated weights for policy 1, policy_version 1744942 (0.0006) [2023-12-27 04:04:48,019][105692] Updated weights for policy 0, policy_version 1741293 (0.0010) [2023-12-27 04:04:48,079][105692] Updated weights for policy 0, policy_version 1741303 (0.0010) [2023-12-27 04:04:48,593][105620] Updated weights for policy 1, policy_version 1744952 (0.0006) [2023-12-27 04:04:48,663][105620] Updated weights for policy 1, policy_version 1744962 (0.0005) [2023-12-27 04:04:48,728][105620] Updated weights for policy 1, policy_version 1744972 (0.0005) [2023-12-27 04:04:48,778][105692] Updated weights for policy 0, policy_version 1741313 (0.0010) [2023-12-27 04:04:48,840][105692] Updated weights for policy 0, policy_version 1741323 (0.0010) [2023-12-27 04:04:48,902][105692] Updated weights for policy 0, policy_version 1741333 (0.0011) [2023-12-27 04:04:48,966][105692] Updated weights for policy 0, policy_version 1741343 (0.0010) [2023-12-27 04:04:49,268][105620] Updated weights for policy 1, policy_version 1744982 (0.0007) [2023-12-27 04:04:49,336][105620] Updated weights for policy 1, policy_version 1744992 (0.0008) [2023-12-27 04:04:49,398][105620] Updated weights for policy 1, policy_version 1745002 (0.0009) [2023-12-27 04:04:49,737][105692] Updated weights for policy 0, policy_version 1741353 (0.0010) [2023-12-27 04:04:49,797][105692] Updated weights for policy 0, policy_version 1741363 (0.0009) [2023-12-27 04:04:49,860][105692] Updated weights for policy 0, policy_version 1741373 (0.0010) [2023-12-27 04:04:50,025][105620] Updated weights for policy 1, policy_version 1745012 (0.0008) [2023-12-27 04:04:50,083][105620] Updated weights for policy 1, policy_version 1745022 (0.0008) [2023-12-27 04:04:50,145][105620] Updated weights for policy 1, policy_version 1745032 (0.0008) [2023-12-27 04:04:50,526][105692] Updated weights for policy 0, policy_version 1741383 (0.0007) [2023-12-27 04:04:50,589][105692] Updated weights for policy 0, policy_version 1741393 (0.0008) [2023-12-27 04:04:50,650][105692] Updated weights for policy 0, policy_version 1741403 (0.0008) [2023-12-27 04:04:50,846][105620] Updated weights for policy 1, policy_version 1745042 (0.0008) [2023-12-27 04:04:50,907][105620] Updated weights for policy 1, policy_version 1745052 (0.0008) [2023-12-27 04:04:50,959][105620] Updated weights for policy 1, policy_version 1745062 (0.0010) [2023-12-27 04:04:51,011][105620] Updated weights for policy 1, policy_version 1745072 (0.0011) [2023-12-27 04:04:51,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 892665856. Throughput: 0: 9750.0, 1: 9950.6. Samples: 892651880. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:04:51,062][104569] Avg episode reward: [(0, '8617.692'), (1, '9079.145')] [2023-12-27 04:04:51,413][105692] Updated weights for policy 0, policy_version 1741413 (0.0008) [2023-12-27 04:04:51,462][105692] Updated weights for policy 0, policy_version 1741423 (0.0009) [2023-12-27 04:04:51,511][105692] Updated weights for policy 0, policy_version 1741433 (0.0009) [2023-12-27 04:04:51,747][105620] Updated weights for policy 1, policy_version 1745082 (0.0008) [2023-12-27 04:04:51,806][105620] Updated weights for policy 1, policy_version 1745092 (0.0006) [2023-12-27 04:04:51,855][105620] Updated weights for policy 1, policy_version 1745102 (0.0007) [2023-12-27 04:04:52,284][105692] Updated weights for policy 0, policy_version 1741443 (0.0010) [2023-12-27 04:04:52,340][105692] Updated weights for policy 0, policy_version 1741453 (0.0011) [2023-12-27 04:04:52,396][105692] Updated weights for policy 0, policy_version 1741463 (0.0010) [2023-12-27 04:04:52,586][105620] Updated weights for policy 1, policy_version 1745112 (0.0010) [2023-12-27 04:04:52,634][105620] Updated weights for policy 1, policy_version 1745122 (0.0010) [2023-12-27 04:04:52,693][105620] Updated weights for policy 1, policy_version 1745132 (0.0010) [2023-12-27 04:04:53,106][105692] Updated weights for policy 0, policy_version 1741473 (0.0010) [2023-12-27 04:04:53,169][105692] Updated weights for policy 0, policy_version 1741483 (0.0006) [2023-12-27 04:04:53,226][105692] Updated weights for policy 0, policy_version 1741494 (0.0006) [2023-12-27 04:04:53,275][105692] Updated weights for policy 0, policy_version 1741504 (0.0005) [2023-12-27 04:04:53,489][105620] Updated weights for policy 1, policy_version 1745142 (0.0008) [2023-12-27 04:04:53,543][105620] Updated weights for policy 1, policy_version 1745152 (0.0009) [2023-12-27 04:04:53,600][105620] Updated weights for policy 1, policy_version 1745163 (0.0010) [2023-12-27 04:04:53,808][105692] Updated weights for policy 0, policy_version 1741514 (0.0005) [2023-12-27 04:04:53,871][105692] Updated weights for policy 0, policy_version 1741524 (0.0005) [2023-12-27 04:04:53,922][105692] Updated weights for policy 0, policy_version 1741534 (0.0005) [2023-12-27 04:04:54,434][105620] Updated weights for policy 1, policy_version 1745174 (0.0009) [2023-12-27 04:04:54,479][105620] Updated weights for policy 1, policy_version 1745184 (0.0008) [2023-12-27 04:04:54,530][105620] Updated weights for policy 1, policy_version 1745194 (0.0008) [2023-12-27 04:04:54,583][105692] Updated weights for policy 0, policy_version 1741544 (0.0009) [2023-12-27 04:04:54,644][105692] Updated weights for policy 0, policy_version 1741554 (0.0010) [2023-12-27 04:04:54,702][105692] Updated weights for policy 0, policy_version 1741564 (0.0010) [2023-12-27 04:04:55,233][105620] Updated weights for policy 1, policy_version 1745204 (0.0009) [2023-12-27 04:04:55,290][105620] Updated weights for policy 1, policy_version 1745215 (0.0010) [2023-12-27 04:04:55,360][105620] Updated weights for policy 1, policy_version 1745225 (0.0008) [2023-12-27 04:04:55,363][105692] Updated weights for policy 0, policy_version 1741574 (0.0007) [2023-12-27 04:04:55,408][105692] Updated weights for policy 0, policy_version 1741584 (0.0005) [2023-12-27 04:04:55,463][105692] Updated weights for policy 0, policy_version 1741594 (0.0005) [2023-12-27 04:04:56,023][105692] Updated weights for policy 0, policy_version 1741604 (0.0007) [2023-12-27 04:04:56,062][104569] Fps is (10 sec: 19661.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 892755968. Throughput: 0: 9794.1, 1: 9903.5. Samples: 892768888. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:04:56,062][104569] Avg episode reward: [(0, '8711.613'), (1, '9170.635')] [2023-12-27 04:04:56,078][105692] Updated weights for policy 0, policy_version 1741614 (0.0010) [2023-12-27 04:04:56,139][105692] Updated weights for policy 0, policy_version 1741624 (0.0010) [2023-12-27 04:04:56,200][105620] Updated weights for policy 1, policy_version 1745235 (0.0008) [2023-12-27 04:04:56,253][105620] Updated weights for policy 1, policy_version 1745245 (0.0008) [2023-12-27 04:04:56,308][105620] Updated weights for policy 1, policy_version 1745255 (0.0008) [2023-12-27 04:04:56,761][105692] Updated weights for policy 0, policy_version 1741634 (0.0009) [2023-12-27 04:04:56,825][105692] Updated weights for policy 0, policy_version 1741644 (0.0008) [2023-12-27 04:04:56,887][105692] Updated weights for policy 0, policy_version 1741654 (0.0010) [2023-12-27 04:04:56,940][105692] Updated weights for policy 0, policy_version 1741664 (0.0007) [2023-12-27 04:04:57,055][105620] Updated weights for policy 1, policy_version 1745265 (0.0008) [2023-12-27 04:04:57,112][105620] Updated weights for policy 1, policy_version 1745275 (0.0009) [2023-12-27 04:04:57,167][105620] Updated weights for policy 1, policy_version 1745285 (0.0008) [2023-12-27 04:04:57,223][105620] Updated weights for policy 1, policy_version 1745295 (0.0009) [2023-12-27 04:04:57,598][105692] Updated weights for policy 0, policy_version 1741674 (0.0009) [2023-12-27 04:04:57,658][105692] Updated weights for policy 0, policy_version 1741684 (0.0008) [2023-12-27 04:04:57,716][105692] Updated weights for policy 0, policy_version 1741694 (0.0008) [2023-12-27 04:04:57,922][105620] Updated weights for policy 1, policy_version 1745305 (0.0011) [2023-12-27 04:04:57,985][105620] Updated weights for policy 1, policy_version 1745315 (0.0011) [2023-12-27 04:04:58,048][105620] Updated weights for policy 1, policy_version 1745325 (0.0008) [2023-12-27 04:04:58,445][105692] Updated weights for policy 0, policy_version 1741704 (0.0007) [2023-12-27 04:04:58,504][105692] Updated weights for policy 0, policy_version 1741714 (0.0007) [2023-12-27 04:04:58,564][105692] Updated weights for policy 0, policy_version 1741724 (0.0007) [2023-12-27 04:04:58,790][105620] Updated weights for policy 1, policy_version 1745335 (0.0008) [2023-12-27 04:04:58,857][105620] Updated weights for policy 1, policy_version 1745345 (0.0007) [2023-12-27 04:04:58,932][105620] Updated weights for policy 1, policy_version 1745356 (0.0010) [2023-12-27 04:04:59,410][105692] Updated weights for policy 0, policy_version 1741734 (0.0008) [2023-12-27 04:04:59,467][105692] Updated weights for policy 0, policy_version 1741744 (0.0009) [2023-12-27 04:04:59,523][105692] Updated weights for policy 0, policy_version 1741754 (0.0009) [2023-12-27 04:04:59,740][105620] Updated weights for policy 1, policy_version 1745366 (0.0007) [2023-12-27 04:04:59,798][105620] Updated weights for policy 1, policy_version 1745376 (0.0006) [2023-12-27 04:04:59,862][105620] Updated weights for policy 1, policy_version 1745386 (0.0008) [2023-12-27 04:05:00,161][105692] Updated weights for policy 0, policy_version 1741764 (0.0008) [2023-12-27 04:05:00,215][105692] Updated weights for policy 0, policy_version 1741774 (0.0010) [2023-12-27 04:05:00,275][105692] Updated weights for policy 0, policy_version 1741784 (0.0008) [2023-12-27 04:05:00,508][105620] Updated weights for policy 1, policy_version 1745396 (0.0009) [2023-12-27 04:05:00,560][105620] Updated weights for policy 1, policy_version 1745406 (0.0010) [2023-12-27 04:05:00,612][105620] Updated weights for policy 1, policy_version 1745416 (0.0010) [2023-12-27 04:05:00,903][105692] Updated weights for policy 0, policy_version 1741794 (0.0007) [2023-12-27 04:05:00,952][105692] Updated weights for policy 0, policy_version 1741804 (0.0005) [2023-12-27 04:05:00,996][105692] Updated weights for policy 0, policy_version 1741814 (0.0005) [2023-12-27 04:05:01,055][105692] Updated weights for policy 0, policy_version 1741824 (0.0007) [2023-12-27 04:05:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 892862464. Throughput: 0: 9874.5, 1: 9899.3. Samples: 892828464. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:05:01,062][104569] Avg episode reward: [(0, '8626.202'), (1, '8894.032')] [2023-12-27 04:05:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001741824_445972480.pth... [2023-12-27 04:05:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001745424_446889984.pth... [2023-12-27 04:05:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001744272_446595072.pth [2023-12-27 04:05:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001740640_445669376.pth [2023-12-27 04:05:01,400][105620] Updated weights for policy 1, policy_version 1745426 (0.0009) [2023-12-27 04:05:01,462][105620] Updated weights for policy 1, policy_version 1745436 (0.0006) [2023-12-27 04:05:01,527][105620] Updated weights for policy 1, policy_version 1745446 (0.0006) [2023-12-27 04:05:01,598][105620] Updated weights for policy 1, policy_version 1745456 (0.0008) [2023-12-27 04:05:01,660][105692] Updated weights for policy 0, policy_version 1741834 (0.0009) [2023-12-27 04:05:01,728][105692] Updated weights for policy 0, policy_version 1741844 (0.0009) [2023-12-27 04:05:01,785][105692] Updated weights for policy 0, policy_version 1741854 (0.0010) [2023-12-27 04:05:02,161][105620] Updated weights for policy 1, policy_version 1745466 (0.0008) [2023-12-27 04:05:02,228][105620] Updated weights for policy 1, policy_version 1745476 (0.0007) [2023-12-27 04:05:02,300][105620] Updated weights for policy 1, policy_version 1745486 (0.0006) [2023-12-27 04:05:02,585][105692] Updated weights for policy 0, policy_version 1741864 (0.0008) [2023-12-27 04:05:02,645][105692] Updated weights for policy 0, policy_version 1741874 (0.0008) [2023-12-27 04:05:02,712][105692] Updated weights for policy 0, policy_version 1741884 (0.0009) [2023-12-27 04:05:02,934][105620] Updated weights for policy 1, policy_version 1745496 (0.0008) [2023-12-27 04:05:03,002][105620] Updated weights for policy 1, policy_version 1745506 (0.0009) [2023-12-27 04:05:03,060][105620] Updated weights for policy 1, policy_version 1745516 (0.0009) [2023-12-27 04:05:03,391][105692] Updated weights for policy 0, policy_version 1741894 (0.0010) [2023-12-27 04:05:03,445][105692] Updated weights for policy 0, policy_version 1741904 (0.0006) [2023-12-27 04:05:03,493][105692] Updated weights for policy 0, policy_version 1741914 (0.0005) [2023-12-27 04:05:03,779][105620] Updated weights for policy 1, policy_version 1745526 (0.0010) [2023-12-27 04:05:03,823][105620] Updated weights for policy 1, policy_version 1745536 (0.0010) [2023-12-27 04:05:03,884][105620] Updated weights for policy 1, policy_version 1745546 (0.0011) [2023-12-27 04:05:04,104][105692] Updated weights for policy 0, policy_version 1741924 (0.0006) [2023-12-27 04:05:04,158][105692] Updated weights for policy 0, policy_version 1741934 (0.0009) [2023-12-27 04:05:04,216][105692] Updated weights for policy 0, policy_version 1741944 (0.0009) [2023-12-27 04:05:04,663][105620] Updated weights for policy 1, policy_version 1745556 (0.0011) [2023-12-27 04:05:04,723][105620] Updated weights for policy 1, policy_version 1745566 (0.0010) [2023-12-27 04:05:04,786][105620] Updated weights for policy 1, policy_version 1745576 (0.0011) [2023-12-27 04:05:04,973][105692] Updated weights for policy 0, policy_version 1741954 (0.0008) [2023-12-27 04:05:05,037][105692] Updated weights for policy 0, policy_version 1741964 (0.0008) [2023-12-27 04:05:05,088][105692] Updated weights for policy 0, policy_version 1741974 (0.0008) [2023-12-27 04:05:05,140][105692] Updated weights for policy 0, policy_version 1741984 (0.0008) [2023-12-27 04:05:05,532][105620] Updated weights for policy 1, policy_version 1745586 (0.0011) [2023-12-27 04:05:05,594][105620] Updated weights for policy 1, policy_version 1745596 (0.0011) [2023-12-27 04:05:05,650][105620] Updated weights for policy 1, policy_version 1745606 (0.0011) [2023-12-27 04:05:05,699][105620] Updated weights for policy 1, policy_version 1745616 (0.0011) [2023-12-27 04:05:05,799][105692] Updated weights for policy 0, policy_version 1741994 (0.0005) [2023-12-27 04:05:05,845][105692] Updated weights for policy 0, policy_version 1742004 (0.0005) [2023-12-27 04:05:05,907][105692] Updated weights for policy 0, policy_version 1742014 (0.0005) [2023-12-27 04:05:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 892960768. Throughput: 0: 9874.0, 1: 9927.5. Samples: 892947128. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:05:06,062][104569] Avg episode reward: [(0, '8350.786'), (1, '8800.261')] [2023-12-27 04:05:06,448][105620] Updated weights for policy 1, policy_version 1745626 (0.0010) [2023-12-27 04:05:06,504][105620] Updated weights for policy 1, policy_version 1745636 (0.0009) [2023-12-27 04:05:06,522][105692] Updated weights for policy 0, policy_version 1742024 (0.0006) [2023-12-27 04:05:06,567][105620] Updated weights for policy 1, policy_version 1745646 (0.0007) [2023-12-27 04:05:06,588][105692] Updated weights for policy 0, policy_version 1742034 (0.0007) [2023-12-27 04:05:06,660][105692] Updated weights for policy 0, policy_version 1742044 (0.0009) [2023-12-27 04:05:07,221][105620] Updated weights for policy 1, policy_version 1745656 (0.0010) [2023-12-27 04:05:07,286][105620] Updated weights for policy 1, policy_version 1745666 (0.0010) [2023-12-27 04:05:07,325][105692] Updated weights for policy 0, policy_version 1742054 (0.0011) [2023-12-27 04:05:07,346][105620] Updated weights for policy 1, policy_version 1745676 (0.0010) [2023-12-27 04:05:07,376][105692] Updated weights for policy 0, policy_version 1742064 (0.0010) [2023-12-27 04:05:07,439][105692] Updated weights for policy 0, policy_version 1742074 (0.0011) [2023-12-27 04:05:08,092][105620] Updated weights for policy 1, policy_version 1745686 (0.0009) [2023-12-27 04:05:08,117][105692] Updated weights for policy 0, policy_version 1742084 (0.0010) [2023-12-27 04:05:08,148][105620] Updated weights for policy 1, policy_version 1745696 (0.0006) [2023-12-27 04:05:08,176][105692] Updated weights for policy 0, policy_version 1742094 (0.0011) [2023-12-27 04:05:08,197][105620] Updated weights for policy 1, policy_version 1745706 (0.0009) [2023-12-27 04:05:08,228][105692] Updated weights for policy 0, policy_version 1742104 (0.0010) [2023-12-27 04:05:08,817][105620] Updated weights for policy 1, policy_version 1745716 (0.0008) [2023-12-27 04:05:08,875][105620] Updated weights for policy 1, policy_version 1745726 (0.0005) [2023-12-27 04:05:08,935][105620] Updated weights for policy 1, policy_version 1745736 (0.0006) [2023-12-27 04:05:09,072][105692] Updated weights for policy 0, policy_version 1742114 (0.0010) [2023-12-27 04:05:09,125][105692] Updated weights for policy 0, policy_version 1742124 (0.0010) [2023-12-27 04:05:09,183][105692] Updated weights for policy 0, policy_version 1742135 (0.0011) [2023-12-27 04:05:09,550][105620] Updated weights for policy 1, policy_version 1745746 (0.0008) [2023-12-27 04:05:09,617][105620] Updated weights for policy 1, policy_version 1745756 (0.0006) [2023-12-27 04:05:09,683][105620] Updated weights for policy 1, policy_version 1745766 (0.0005) [2023-12-27 04:05:09,752][105620] Updated weights for policy 1, policy_version 1745776 (0.0006) [2023-12-27 04:05:09,849][105692] Updated weights for policy 0, policy_version 1742145 (0.0009) [2023-12-27 04:05:09,904][105692] Updated weights for policy 0, policy_version 1742155 (0.0009) [2023-12-27 04:05:09,964][105692] Updated weights for policy 0, policy_version 1742165 (0.0009) [2023-12-27 04:05:10,026][105692] Updated weights for policy 0, policy_version 1742175 (0.0011) [2023-12-27 04:05:10,401][105620] Updated weights for policy 1, policy_version 1745786 (0.0008) [2023-12-27 04:05:10,457][105620] Updated weights for policy 1, policy_version 1745796 (0.0008) [2023-12-27 04:05:10,516][105620] Updated weights for policy 1, policy_version 1745806 (0.0007) [2023-12-27 04:05:10,809][105692] Updated weights for policy 0, policy_version 1742185 (0.0007) [2023-12-27 04:05:10,871][105692] Updated weights for policy 0, policy_version 1742195 (0.0011) [2023-12-27 04:05:10,933][105692] Updated weights for policy 0, policy_version 1742205 (0.0011) [2023-12-27 04:05:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 893059072. Throughput: 0: 9965.6, 1: 9974.8. Samples: 893066724. Policy #0 lag: (min: 40.0, avg: 47.8, max: 48.0) [2023-12-27 04:05:11,062][104569] Avg episode reward: [(0, '8344.342'), (1, '9076.629')] [2023-12-27 04:05:11,200][105620] Updated weights for policy 1, policy_version 1745816 (0.0008) [2023-12-27 04:05:11,264][105620] Updated weights for policy 1, policy_version 1745826 (0.0008) [2023-12-27 04:05:11,329][105620] Updated weights for policy 1, policy_version 1745836 (0.0009) [2023-12-27 04:05:11,678][105692] Updated weights for policy 0, policy_version 1742215 (0.0011) [2023-12-27 04:05:11,742][105692] Updated weights for policy 0, policy_version 1742225 (0.0010) [2023-12-27 04:05:11,809][105692] Updated weights for policy 0, policy_version 1742235 (0.0011) [2023-12-27 04:05:12,043][105620] Updated weights for policy 1, policy_version 1745846 (0.0008) [2023-12-27 04:05:12,103][105620] Updated weights for policy 1, policy_version 1745856 (0.0008) [2023-12-27 04:05:12,153][105620] Updated weights for policy 1, policy_version 1745866 (0.0008) [2023-12-27 04:05:12,588][105692] Updated weights for policy 0, policy_version 1742245 (0.0010) [2023-12-27 04:05:12,647][105692] Updated weights for policy 0, policy_version 1742255 (0.0009) [2023-12-27 04:05:12,705][105692] Updated weights for policy 0, policy_version 1742265 (0.0008) [2023-12-27 04:05:12,933][105620] Updated weights for policy 1, policy_version 1745876 (0.0008) [2023-12-27 04:05:12,992][105620] Updated weights for policy 1, policy_version 1745886 (0.0010) [2023-12-27 04:05:13,051][105620] Updated weights for policy 1, policy_version 1745896 (0.0010) [2023-12-27 04:05:13,478][105692] Updated weights for policy 0, policy_version 1742275 (0.0009) [2023-12-27 04:05:13,526][105692] Updated weights for policy 0, policy_version 1742285 (0.0009) [2023-12-27 04:05:13,575][105692] Updated weights for policy 0, policy_version 1742295 (0.0008) [2023-12-27 04:05:13,675][105620] Updated weights for policy 1, policy_version 1745906 (0.0010) [2023-12-27 04:05:13,729][105620] Updated weights for policy 1, policy_version 1745916 (0.0009) [2023-12-27 04:05:13,780][105620] Updated weights for policy 1, policy_version 1745926 (0.0009) [2023-12-27 04:05:13,830][105620] Updated weights for policy 1, policy_version 1745936 (0.0009) [2023-12-27 04:05:14,199][105692] Updated weights for policy 0, policy_version 1742305 (0.0007) [2023-12-27 04:05:14,251][105692] Updated weights for policy 0, policy_version 1742315 (0.0008) [2023-12-27 04:05:14,295][105692] Updated weights for policy 0, policy_version 1742325 (0.0010) [2023-12-27 04:05:14,339][105692] Updated weights for policy 0, policy_version 1742335 (0.0010) [2023-12-27 04:05:14,572][105620] Updated weights for policy 1, policy_version 1745946 (0.0008) [2023-12-27 04:05:14,621][105620] Updated weights for policy 1, policy_version 1745956 (0.0008) [2023-12-27 04:05:14,675][105620] Updated weights for policy 1, policy_version 1745966 (0.0008) [2023-12-27 04:05:15,052][105692] Updated weights for policy 0, policy_version 1742345 (0.0006) [2023-12-27 04:05:15,112][105692] Updated weights for policy 0, policy_version 1742355 (0.0006) [2023-12-27 04:05:15,172][105692] Updated weights for policy 0, policy_version 1742365 (0.0009) [2023-12-27 04:05:15,504][105620] Updated weights for policy 1, policy_version 1745976 (0.0008) [2023-12-27 04:05:15,563][105620] Updated weights for policy 1, policy_version 1745986 (0.0008) [2023-12-27 04:05:15,620][105620] Updated weights for policy 1, policy_version 1745996 (0.0007) [2023-12-27 04:05:15,856][105692] Updated weights for policy 0, policy_version 1742375 (0.0005) [2023-12-27 04:05:15,913][105692] Updated weights for policy 0, policy_version 1742385 (0.0006) [2023-12-27 04:05:15,968][105692] Updated weights for policy 0, policy_version 1742395 (0.0010) [2023-12-27 04:05:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 893157376. Throughput: 0: 9923.0, 1: 9913.2. Samples: 893122680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:05:16,063][104569] Avg episode reward: [(0, '8436.603'), (1, '9261.173')] [2023-12-27 04:05:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001742400_446119936.pth... [2023-12-27 04:05:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001746000_447037440.pth... [2023-12-27 04:05:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001741216_445816832.pth [2023-12-27 04:05:16,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001744848_446742528.pth [2023-12-27 04:05:16,346][105620] Updated weights for policy 1, policy_version 1746006 (0.0008) [2023-12-27 04:05:16,399][105620] Updated weights for policy 1, policy_version 1746016 (0.0010) [2023-12-27 04:05:16,447][105620] Updated weights for policy 1, policy_version 1746027 (0.0008) [2023-12-27 04:05:16,601][105692] Updated weights for policy 0, policy_version 1742405 (0.0009) [2023-12-27 04:05:16,656][105692] Updated weights for policy 0, policy_version 1742415 (0.0005) [2023-12-27 04:05:16,708][105692] Updated weights for policy 0, policy_version 1742425 (0.0005) [2023-12-27 04:05:17,232][105692] Updated weights for policy 0, policy_version 1742435 (0.0005) [2023-12-27 04:05:17,288][105692] Updated weights for policy 0, policy_version 1742445 (0.0009) [2023-12-27 04:05:17,335][105620] Updated weights for policy 1, policy_version 1746037 (0.0008) [2023-12-27 04:05:17,344][105692] Updated weights for policy 0, policy_version 1742455 (0.0006) [2023-12-27 04:05:17,386][105620] Updated weights for policy 1, policy_version 1746048 (0.0009) [2023-12-27 04:05:17,448][105620] Updated weights for policy 1, policy_version 1746058 (0.0009) [2023-12-27 04:05:17,944][105692] Updated weights for policy 0, policy_version 1742465 (0.0006) [2023-12-27 04:05:17,999][105692] Updated weights for policy 0, policy_version 1742475 (0.0009) [2023-12-27 04:05:18,048][105692] Updated weights for policy 0, policy_version 1742485 (0.0009) [2023-12-27 04:05:18,104][105692] Updated weights for policy 0, policy_version 1742495 (0.0009) [2023-12-27 04:05:18,263][105620] Updated weights for policy 1, policy_version 1746068 (0.0009) [2023-12-27 04:05:18,330][105620] Updated weights for policy 1, policy_version 1746078 (0.0008) [2023-12-27 04:05:18,389][105620] Updated weights for policy 1, policy_version 1746088 (0.0009) [2023-12-27 04:05:18,886][105692] Updated weights for policy 0, policy_version 1742505 (0.0010) [2023-12-27 04:05:18,946][105692] Updated weights for policy 0, policy_version 1742515 (0.0009) [2023-12-27 04:05:19,008][105692] Updated weights for policy 0, policy_version 1742525 (0.0009) [2023-12-27 04:05:19,096][105620] Updated weights for policy 1, policy_version 1746098 (0.0008) [2023-12-27 04:05:19,149][105620] Updated weights for policy 1, policy_version 1746108 (0.0008) [2023-12-27 04:05:19,195][105620] Updated weights for policy 1, policy_version 1746118 (0.0008) [2023-12-27 04:05:19,257][105620] Updated weights for policy 1, policy_version 1746128 (0.0008) [2023-12-27 04:05:19,800][105692] Updated weights for policy 0, policy_version 1742535 (0.0010) [2023-12-27 04:05:19,861][105692] Updated weights for policy 0, policy_version 1742545 (0.0010) [2023-12-27 04:05:19,926][105692] Updated weights for policy 0, policy_version 1742555 (0.0008) [2023-12-27 04:05:20,042][105620] Updated weights for policy 1, policy_version 1746138 (0.0009) [2023-12-27 04:05:20,104][105620] Updated weights for policy 1, policy_version 1746148 (0.0009) [2023-12-27 04:05:20,161][105620] Updated weights for policy 1, policy_version 1746158 (0.0009) [2023-12-27 04:05:20,736][105692] Updated weights for policy 0, policy_version 1742565 (0.0010) [2023-12-27 04:05:20,801][105692] Updated weights for policy 0, policy_version 1742575 (0.0010) [2023-12-27 04:05:20,853][105692] Updated weights for policy 0, policy_version 1742585 (0.0009) [2023-12-27 04:05:20,871][105620] Updated weights for policy 1, policy_version 1746168 (0.0006) [2023-12-27 04:05:20,934][105620] Updated weights for policy 1, policy_version 1746178 (0.0007) [2023-12-27 04:05:20,984][105620] Updated weights for policy 1, policy_version 1746188 (0.0008) [2023-12-27 04:05:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 893255680. Throughput: 0: 10039.7, 1: 9777.9. Samples: 893240800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:05:21,063][104569] Avg episode reward: [(0, '8260.380'), (1, '9261.265')] [2023-12-27 04:05:21,688][105692] Updated weights for policy 0, policy_version 1742595 (0.0009) [2023-12-27 04:05:21,753][105692] Updated weights for policy 0, policy_version 1742605 (0.0007) [2023-12-27 04:05:21,754][105620] Updated weights for policy 1, policy_version 1746198 (0.0010) [2023-12-27 04:05:21,809][105692] Updated weights for policy 0, policy_version 1742615 (0.0006) [2023-12-27 04:05:21,815][105620] Updated weights for policy 1, policy_version 1746208 (0.0010) [2023-12-27 04:05:21,878][105620] Updated weights for policy 1, policy_version 1746218 (0.0010) [2023-12-27 04:05:22,583][105692] Updated weights for policy 0, policy_version 1742625 (0.0006) [2023-12-27 04:05:22,638][105692] Updated weights for policy 0, policy_version 1742635 (0.0008) [2023-12-27 04:05:22,672][105620] Updated weights for policy 1, policy_version 1746228 (0.0010) [2023-12-27 04:05:22,701][105692] Updated weights for policy 0, policy_version 1742645 (0.0010) [2023-12-27 04:05:22,733][105620] Updated weights for policy 1, policy_version 1746238 (0.0006) [2023-12-27 04:05:22,760][105692] Updated weights for policy 0, policy_version 1742655 (0.0008) [2023-12-27 04:05:22,792][105620] Updated weights for policy 1, policy_version 1746248 (0.0008) [2023-12-27 04:05:23,537][105692] Updated weights for policy 0, policy_version 1742665 (0.0009) [2023-12-27 04:05:23,562][105620] Updated weights for policy 1, policy_version 1746258 (0.0009) [2023-12-27 04:05:23,597][105692] Updated weights for policy 0, policy_version 1742675 (0.0008) [2023-12-27 04:05:23,607][105620] Updated weights for policy 1, policy_version 1746268 (0.0007) [2023-12-27 04:05:23,653][105692] Updated weights for policy 0, policy_version 1742685 (0.0008) [2023-12-27 04:05:23,655][105620] Updated weights for policy 1, policy_version 1746278 (0.0005) [2023-12-27 04:05:23,702][105620] Updated weights for policy 1, policy_version 1746288 (0.0005) [2023-12-27 04:05:24,342][105620] Updated weights for policy 1, policy_version 1746298 (0.0005) [2023-12-27 04:05:24,402][105620] Updated weights for policy 1, policy_version 1746308 (0.0009) [2023-12-27 04:05:24,465][105620] Updated weights for policy 1, policy_version 1746318 (0.0008) [2023-12-27 04:05:24,476][105692] Updated weights for policy 0, policy_version 1742695 (0.0006) [2023-12-27 04:05:24,524][105692] Updated weights for policy 0, policy_version 1742705 (0.0009) [2023-12-27 04:05:24,575][105692] Updated weights for policy 0, policy_version 1742715 (0.0009) [2023-12-27 04:05:25,057][105620] Updated weights for policy 1, policy_version 1746328 (0.0009) [2023-12-27 04:05:25,109][105620] Updated weights for policy 1, policy_version 1746338 (0.0010) [2023-12-27 04:05:25,164][105620] Updated weights for policy 1, policy_version 1746348 (0.0007) [2023-12-27 04:05:25,477][105692] Updated weights for policy 0, policy_version 1742725 (0.0010) [2023-12-27 04:05:25,529][105692] Updated weights for policy 0, policy_version 1742736 (0.0010) [2023-12-27 04:05:25,585][105692] Updated weights for policy 0, policy_version 1742748 (0.0010) [2023-12-27 04:05:25,706][105620] Updated weights for policy 1, policy_version 1746358 (0.0006) [2023-12-27 04:05:25,765][105620] Updated weights for policy 1, policy_version 1746368 (0.0006) [2023-12-27 04:05:25,825][105620] Updated weights for policy 1, policy_version 1746378 (0.0009) [2023-12-27 04:05:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 893345792. Throughput: 0: 9841.7, 1: 9797.6. Samples: 893352368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:05:26,062][104569] Avg episode reward: [(0, '8629.675'), (1, '9171.082')] [2023-12-27 04:05:26,330][105692] Updated weights for policy 0, policy_version 1742758 (0.0009) [2023-12-27 04:05:26,378][105692] Updated weights for policy 0, policy_version 1742768 (0.0009) [2023-12-27 04:05:26,426][105692] Updated weights for policy 0, policy_version 1742778 (0.0009) [2023-12-27 04:05:26,571][105620] Updated weights for policy 1, policy_version 1746388 (0.0009) [2023-12-27 04:05:26,628][105620] Updated weights for policy 1, policy_version 1746398 (0.0009) [2023-12-27 04:05:26,682][105620] Updated weights for policy 1, policy_version 1746408 (0.0008) [2023-12-27 04:05:27,256][105692] Updated weights for policy 0, policy_version 1742788 (0.0009) [2023-12-27 04:05:27,293][105620] Updated weights for policy 1, policy_version 1746418 (0.0008) [2023-12-27 04:05:27,303][105692] Updated weights for policy 0, policy_version 1742798 (0.0009) [2023-12-27 04:05:27,358][105620] Updated weights for policy 1, policy_version 1746428 (0.0008) [2023-12-27 04:05:27,365][105692] Updated weights for policy 0, policy_version 1742808 (0.0008) [2023-12-27 04:05:27,410][105620] Updated weights for policy 1, policy_version 1746438 (0.0008) [2023-12-27 04:05:27,461][105620] Updated weights for policy 1, policy_version 1746448 (0.0008) [2023-12-27 04:05:27,957][105692] Updated weights for policy 0, policy_version 1742818 (0.0006) [2023-12-27 04:05:28,027][105692] Updated weights for policy 0, policy_version 1742828 (0.0007) [2023-12-27 04:05:28,088][105692] Updated weights for policy 0, policy_version 1742838 (0.0005) [2023-12-27 04:05:28,117][105620] Updated weights for policy 1, policy_version 1746458 (0.0008) [2023-12-27 04:05:28,149][105692] Updated weights for policy 0, policy_version 1742848 (0.0006) [2023-12-27 04:05:28,168][105620] Updated weights for policy 1, policy_version 1746468 (0.0006) [2023-12-27 04:05:28,238][105620] Updated weights for policy 1, policy_version 1746478 (0.0008) [2023-12-27 04:05:28,721][105692] Updated weights for policy 0, policy_version 1742858 (0.0006) [2023-12-27 04:05:28,775][105692] Updated weights for policy 0, policy_version 1742868 (0.0005) [2023-12-27 04:05:28,830][105692] Updated weights for policy 0, policy_version 1742878 (0.0005) [2023-12-27 04:05:28,918][105620] Updated weights for policy 1, policy_version 1746488 (0.0008) [2023-12-27 04:05:28,981][105620] Updated weights for policy 1, policy_version 1746499 (0.0009) [2023-12-27 04:05:29,040][105620] Updated weights for policy 1, policy_version 1746509 (0.0009) [2023-12-27 04:05:29,472][105692] Updated weights for policy 0, policy_version 1742888 (0.0009) [2023-12-27 04:05:29,529][105692] Updated weights for policy 0, policy_version 1742898 (0.0009) [2023-12-27 04:05:29,588][105692] Updated weights for policy 0, policy_version 1742908 (0.0009) [2023-12-27 04:05:29,858][105620] Updated weights for policy 1, policy_version 1746519 (0.0010) [2023-12-27 04:05:29,921][105620] Updated weights for policy 1, policy_version 1746529 (0.0009) [2023-12-27 04:05:29,987][105620] Updated weights for policy 1, policy_version 1746539 (0.0009) [2023-12-27 04:05:30,358][105692] Updated weights for policy 0, policy_version 1742918 (0.0008) [2023-12-27 04:05:30,416][105692] Updated weights for policy 0, policy_version 1742928 (0.0009) [2023-12-27 04:05:30,474][105692] Updated weights for policy 0, policy_version 1742938 (0.0009) [2023-12-27 04:05:30,756][105620] Updated weights for policy 1, policy_version 1746549 (0.0009) [2023-12-27 04:05:30,806][105620] Updated weights for policy 1, policy_version 1746559 (0.0009) [2023-12-27 04:05:30,858][105620] Updated weights for policy 1, policy_version 1746569 (0.0008) [2023-12-27 04:05:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19797.3, 300 sec: 19605.2). Total num frames: 893444096. Throughput: 0: 9906.0, 1: 9833.3. Samples: 893414308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:05:31,063][104569] Avg episode reward: [(0, '9082.257'), (1, '8986.075')] [2023-12-27 04:05:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001742944_446259200.pth... [2023-12-27 04:05:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001746576_447184896.pth... [2023-12-27 04:05:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001741824_445972480.pth [2023-12-27 04:05:31,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001745424_446889984.pth [2023-12-27 04:05:31,139][105692] Updated weights for policy 0, policy_version 1742948 (0.0009) [2023-12-27 04:05:31,191][105692] Updated weights for policy 0, policy_version 1742958 (0.0006) [2023-12-27 04:05:31,250][105692] Updated weights for policy 0, policy_version 1742968 (0.0007) [2023-12-27 04:05:31,644][105620] Updated weights for policy 1, policy_version 1746579 (0.0009) [2023-12-27 04:05:31,714][105620] Updated weights for policy 1, policy_version 1746589 (0.0009) [2023-12-27 04:05:31,780][105620] Updated weights for policy 1, policy_version 1746599 (0.0009) [2023-12-27 04:05:32,062][105692] Updated weights for policy 0, policy_version 1742978 (0.0009) [2023-12-27 04:05:32,122][105692] Updated weights for policy 0, policy_version 1742988 (0.0009) [2023-12-27 04:05:32,186][105692] Updated weights for policy 0, policy_version 1742998 (0.0008) [2023-12-27 04:05:32,246][105692] Updated weights for policy 0, policy_version 1743008 (0.0010) [2023-12-27 04:05:32,551][105620] Updated weights for policy 1, policy_version 1746609 (0.0009) [2023-12-27 04:05:32,597][105620] Updated weights for policy 1, policy_version 1746619 (0.0008) [2023-12-27 04:05:32,644][105620] Updated weights for policy 1, policy_version 1746629 (0.0009) [2023-12-27 04:05:32,691][105620] Updated weights for policy 1, policy_version 1746639 (0.0009) [2023-12-27 04:05:32,989][105692] Updated weights for policy 0, policy_version 1743018 (0.0009) [2023-12-27 04:05:33,039][105692] Updated weights for policy 0, policy_version 1743028 (0.0008) [2023-12-27 04:05:33,089][105692] Updated weights for policy 0, policy_version 1743038 (0.0009) [2023-12-27 04:05:33,416][105620] Updated weights for policy 1, policy_version 1746649 (0.0010) [2023-12-27 04:05:33,472][105620] Updated weights for policy 1, policy_version 1746659 (0.0010) [2023-12-27 04:05:33,533][105620] Updated weights for policy 1, policy_version 1746669 (0.0009) [2023-12-27 04:05:33,905][105692] Updated weights for policy 0, policy_version 1743048 (0.0009) [2023-12-27 04:05:33,958][105692] Updated weights for policy 0, policy_version 1743059 (0.0010) [2023-12-27 04:05:34,011][105692] Updated weights for policy 0, policy_version 1743071 (0.0010) [2023-12-27 04:05:34,093][105620] Updated weights for policy 1, policy_version 1746679 (0.0007) [2023-12-27 04:05:34,160][105620] Updated weights for policy 1, policy_version 1746689 (0.0006) [2023-12-27 04:05:34,223][105620] Updated weights for policy 1, policy_version 1746699 (0.0009) [2023-12-27 04:05:34,845][105692] Updated weights for policy 0, policy_version 1743081 (0.0009) [2023-12-27 04:05:34,894][105620] Updated weights for policy 1, policy_version 1746709 (0.0009) [2023-12-27 04:05:34,900][105692] Updated weights for policy 0, policy_version 1743091 (0.0007) [2023-12-27 04:05:34,952][105620] Updated weights for policy 1, policy_version 1746719 (0.0006) [2023-12-27 04:05:34,957][105692] Updated weights for policy 0, policy_version 1743101 (0.0008) [2023-12-27 04:05:35,009][105620] Updated weights for policy 1, policy_version 1746729 (0.0006) [2023-12-27 04:05:35,593][105620] Updated weights for policy 1, policy_version 1746739 (0.0005) [2023-12-27 04:05:35,640][105620] Updated weights for policy 1, policy_version 1746749 (0.0005) [2023-12-27 04:05:35,687][105620] Updated weights for policy 1, policy_version 1746759 (0.0006) [2023-12-27 04:05:35,819][105692] Updated weights for policy 0, policy_version 1743111 (0.0008) [2023-12-27 04:05:35,881][105692] Updated weights for policy 0, policy_version 1743121 (0.0008) [2023-12-27 04:05:35,942][105692] Updated weights for policy 0, policy_version 1743131 (0.0009) [2023-12-27 04:05:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 893542400. Throughput: 0: 9801.5, 1: 9691.8. Samples: 893529080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:05:36,062][104569] Avg episode reward: [(0, '8533.758'), (1, '8986.675')] [2023-12-27 04:05:36,385][105620] Updated weights for policy 1, policy_version 1746769 (0.0009) [2023-12-27 04:05:36,452][105620] Updated weights for policy 1, policy_version 1746779 (0.0009) [2023-12-27 04:05:36,489][105586] KL-divergence is very high: 109.9906 [2023-12-27 04:05:36,524][105620] Updated weights for policy 1, policy_version 1746789 (0.0010) [2023-12-27 04:05:36,547][105586] KL-divergence is very high: 123.2658 [2023-12-27 04:05:36,596][105620] Updated weights for policy 1, policy_version 1746799 (0.0009) [2023-12-27 04:05:36,700][105692] Updated weights for policy 0, policy_version 1743141 (0.0009) [2023-12-27 04:05:36,752][105692] Updated weights for policy 0, policy_version 1743151 (0.0009) [2023-12-27 04:05:36,801][105692] Updated weights for policy 0, policy_version 1743161 (0.0009) [2023-12-27 04:05:37,348][105620] Updated weights for policy 1, policy_version 1746809 (0.0009) [2023-12-27 04:05:37,415][105620] Updated weights for policy 1, policy_version 1746819 (0.0005) [2023-12-27 04:05:37,477][105620] Updated weights for policy 1, policy_version 1746829 (0.0005) [2023-12-27 04:05:37,603][105692] Updated weights for policy 0, policy_version 1743171 (0.0007) [2023-12-27 04:05:37,653][105692] Updated weights for policy 0, policy_version 1743181 (0.0008) [2023-12-27 04:05:37,708][105692] Updated weights for policy 0, policy_version 1743191 (0.0008) [2023-12-27 04:05:38,122][105620] Updated weights for policy 1, policy_version 1746839 (0.0006) [2023-12-27 04:05:38,190][105620] Updated weights for policy 1, policy_version 1746849 (0.0005) [2023-12-27 04:05:38,251][105620] Updated weights for policy 1, policy_version 1746859 (0.0006) [2023-12-27 04:05:38,516][105692] Updated weights for policy 0, policy_version 1743201 (0.0008) [2023-12-27 04:05:38,575][105692] Updated weights for policy 0, policy_version 1743211 (0.0006) [2023-12-27 04:05:38,639][105692] Updated weights for policy 0, policy_version 1743221 (0.0009) [2023-12-27 04:05:38,694][105692] Updated weights for policy 0, policy_version 1743231 (0.0010) [2023-12-27 04:05:38,797][105620] Updated weights for policy 1, policy_version 1746869 (0.0007) [2023-12-27 04:05:38,853][105620] Updated weights for policy 1, policy_version 1746879 (0.0009) [2023-12-27 04:05:38,907][105620] Updated weights for policy 1, policy_version 1746889 (0.0010) [2023-12-27 04:05:39,373][105692] Updated weights for policy 0, policy_version 1743241 (0.0008) [2023-12-27 04:05:39,439][105692] Updated weights for policy 0, policy_version 1743251 (0.0009) [2023-12-27 04:05:39,499][105692] Updated weights for policy 0, policy_version 1743261 (0.0008) [2023-12-27 04:05:39,703][105620] Updated weights for policy 1, policy_version 1746899 (0.0008) [2023-12-27 04:05:39,761][105620] Updated weights for policy 1, policy_version 1746909 (0.0006) [2023-12-27 04:05:39,841][105620] Updated weights for policy 1, policy_version 1746919 (0.0007) [2023-12-27 04:05:40,320][105692] Updated weights for policy 0, policy_version 1743271 (0.0006) [2023-12-27 04:05:40,383][105692] Updated weights for policy 0, policy_version 1743281 (0.0007) [2023-12-27 04:05:40,453][105692] Updated weights for policy 0, policy_version 1743291 (0.0010) [2023-12-27 04:05:40,466][105620] Updated weights for policy 1, policy_version 1746929 (0.0009) [2023-12-27 04:05:40,532][105620] Updated weights for policy 1, policy_version 1746939 (0.0011) [2023-12-27 04:05:40,596][105620] Updated weights for policy 1, policy_version 1746949 (0.0009) [2023-12-27 04:05:40,654][105620] Updated weights for policy 1, policy_version 1746959 (0.0010) [2023-12-27 04:05:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 893632512. Throughput: 0: 9635.8, 1: 9811.0. Samples: 893643996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:05:41,063][104569] Avg episode reward: [(0, '8535.735'), (1, '9076.874')] [2023-12-27 04:05:41,241][105692] Updated weights for policy 0, policy_version 1743301 (0.0008) [2023-12-27 04:05:41,307][105692] Updated weights for policy 0, policy_version 1743311 (0.0009) [2023-12-27 04:05:41,355][105620] Updated weights for policy 1, policy_version 1746969 (0.0008) [2023-12-27 04:05:41,370][105692] Updated weights for policy 0, policy_version 1743321 (0.0009) [2023-12-27 04:05:41,426][105620] Updated weights for policy 1, policy_version 1746979 (0.0007) [2023-12-27 04:05:41,482][105620] Updated weights for policy 1, policy_version 1746989 (0.0007) [2023-12-27 04:05:42,165][105692] Updated weights for policy 0, policy_version 1743331 (0.0010) [2023-12-27 04:05:42,205][105620] Updated weights for policy 1, policy_version 1746999 (0.0010) [2023-12-27 04:05:42,216][105692] Updated weights for policy 0, policy_version 1743341 (0.0006) [2023-12-27 04:05:42,259][105620] Updated weights for policy 1, policy_version 1747009 (0.0011) [2023-12-27 04:05:42,270][105692] Updated weights for policy 0, policy_version 1743351 (0.0006) [2023-12-27 04:05:42,323][105620] Updated weights for policy 1, policy_version 1747019 (0.0011) [2023-12-27 04:05:42,976][105692] Updated weights for policy 0, policy_version 1743361 (0.0008) [2023-12-27 04:05:43,039][105692] Updated weights for policy 0, policy_version 1743371 (0.0005) [2023-12-27 04:05:43,064][105620] Updated weights for policy 1, policy_version 1747029 (0.0008) [2023-12-27 04:05:43,103][105692] Updated weights for policy 0, policy_version 1743381 (0.0006) [2023-12-27 04:05:43,129][105620] Updated weights for policy 1, policy_version 1747039 (0.0006) [2023-12-27 04:05:43,164][105692] Updated weights for policy 0, policy_version 1743391 (0.0007) [2023-12-27 04:05:43,182][105620] Updated weights for policy 1, policy_version 1747049 (0.0005) [2023-12-27 04:05:43,692][105692] Updated weights for policy 0, policy_version 1743401 (0.0006) [2023-12-27 04:05:43,744][105692] Updated weights for policy 0, policy_version 1743411 (0.0005) [2023-12-27 04:05:43,803][105620] Updated weights for policy 1, policy_version 1747059 (0.0006) [2023-12-27 04:05:43,817][105692] Updated weights for policy 0, policy_version 1743421 (0.0006) [2023-12-27 04:05:43,854][105620] Updated weights for policy 1, policy_version 1747069 (0.0005) [2023-12-27 04:05:43,912][105620] Updated weights for policy 1, policy_version 1747079 (0.0008) [2023-12-27 04:05:44,444][105692] Updated weights for policy 0, policy_version 1743431 (0.0009) [2023-12-27 04:05:44,503][105692] Updated weights for policy 0, policy_version 1743441 (0.0010) [2023-12-27 04:05:44,556][105692] Updated weights for policy 0, policy_version 1743451 (0.0009) [2023-12-27 04:05:44,602][105620] Updated weights for policy 1, policy_version 1747089 (0.0010) [2023-12-27 04:05:44,659][105620] Updated weights for policy 1, policy_version 1747099 (0.0005) [2023-12-27 04:05:44,722][105620] Updated weights for policy 1, policy_version 1747109 (0.0009) [2023-12-27 04:05:44,780][105620] Updated weights for policy 1, policy_version 1747119 (0.0009) [2023-12-27 04:05:45,210][105692] Updated weights for policy 0, policy_version 1743461 (0.0005) [2023-12-27 04:05:45,282][105692] Updated weights for policy 0, policy_version 1743471 (0.0006) [2023-12-27 04:05:45,353][105692] Updated weights for policy 0, policy_version 1743481 (0.0007) [2023-12-27 04:05:45,419][105620] Updated weights for policy 1, policy_version 1747129 (0.0008) [2023-12-27 04:05:45,481][105620] Updated weights for policy 1, policy_version 1747139 (0.0009) [2023-12-27 04:05:45,537][105620] Updated weights for policy 1, policy_version 1747149 (0.0012) [2023-12-27 04:05:45,867][105692] Updated weights for policy 0, policy_version 1743491 (0.0011) [2023-12-27 04:05:45,928][105692] Updated weights for policy 0, policy_version 1743501 (0.0011) [2023-12-27 04:05:45,976][105692] Updated weights for policy 0, policy_version 1743511 (0.0010) [2023-12-27 04:05:46,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 893739008. Throughput: 0: 9593.2, 1: 9853.6. Samples: 893703576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:05:46,063][104569] Avg episode reward: [(0, '8715.575'), (1, '9168.591')] [2023-12-27 04:05:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001743520_446406656.pth... [2023-12-27 04:05:46,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001747152_447332352.pth... [2023-12-27 04:05:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001742400_446119936.pth [2023-12-27 04:05:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001746000_447037440.pth [2023-12-27 04:05:46,377][105620] Updated weights for policy 1, policy_version 1747159 (0.0009) [2023-12-27 04:05:46,430][105620] Updated weights for policy 1, policy_version 1747169 (0.0009) [2023-12-27 04:05:46,479][105620] Updated weights for policy 1, policy_version 1747179 (0.0008) [2023-12-27 04:05:46,640][105692] Updated weights for policy 0, policy_version 1743521 (0.0010) [2023-12-27 04:05:46,708][105692] Updated weights for policy 0, policy_version 1743531 (0.0009) [2023-12-27 04:05:46,756][105692] Updated weights for policy 0, policy_version 1743541 (0.0009) [2023-12-27 04:05:46,803][105692] Updated weights for policy 0, policy_version 1743551 (0.0009) [2023-12-27 04:05:47,213][105620] Updated weights for policy 1, policy_version 1747189 (0.0009) [2023-12-27 04:05:47,264][105620] Updated weights for policy 1, policy_version 1747199 (0.0009) [2023-12-27 04:05:47,325][105620] Updated weights for policy 1, policy_version 1747209 (0.0009) [2023-12-27 04:05:47,562][105692] Updated weights for policy 0, policy_version 1743561 (0.0009) [2023-12-27 04:05:47,612][105692] Updated weights for policy 0, policy_version 1743571 (0.0009) [2023-12-27 04:05:47,659][105692] Updated weights for policy 0, policy_version 1743581 (0.0009) [2023-12-27 04:05:48,103][105620] Updated weights for policy 1, policy_version 1747219 (0.0009) [2023-12-27 04:05:48,160][105620] Updated weights for policy 1, policy_version 1747229 (0.0010) [2023-12-27 04:05:48,220][105620] Updated weights for policy 1, policy_version 1747239 (0.0009) [2023-12-27 04:05:48,324][105692] Updated weights for policy 0, policy_version 1743591 (0.0009) [2023-12-27 04:05:48,382][105692] Updated weights for policy 0, policy_version 1743601 (0.0009) [2023-12-27 04:05:48,437][105692] Updated weights for policy 0, policy_version 1743611 (0.0009) [2023-12-27 04:05:49,053][105620] Updated weights for policy 1, policy_version 1747249 (0.0009) [2023-12-27 04:05:49,108][105692] Updated weights for policy 0, policy_version 1743621 (0.0007) [2023-12-27 04:05:49,116][105620] Updated weights for policy 1, policy_version 1747259 (0.0009) [2023-12-27 04:05:49,161][105692] Updated weights for policy 0, policy_version 1743631 (0.0005) [2023-12-27 04:05:49,181][105620] Updated weights for policy 1, policy_version 1747269 (0.0009) [2023-12-27 04:05:49,222][105692] Updated weights for policy 0, policy_version 1743641 (0.0006) [2023-12-27 04:05:49,245][105620] Updated weights for policy 1, policy_version 1747279 (0.0008) [2023-12-27 04:05:49,926][105620] Updated weights for policy 1, policy_version 1747289 (0.0010) [2023-12-27 04:05:49,993][105620] Updated weights for policy 1, policy_version 1747299 (0.0011) [2023-12-27 04:05:50,015][105692] Updated weights for policy 0, policy_version 1743651 (0.0008) [2023-12-27 04:05:50,058][105620] Updated weights for policy 1, policy_version 1747309 (0.0009) [2023-12-27 04:05:50,077][105692] Updated weights for policy 0, policy_version 1743661 (0.0009) [2023-12-27 04:05:50,140][105692] Updated weights for policy 0, policy_version 1743671 (0.0010) [2023-12-27 04:05:50,652][105620] Updated weights for policy 1, policy_version 1747319 (0.0009) [2023-12-27 04:05:50,712][105620] Updated weights for policy 1, policy_version 1747329 (0.0009) [2023-12-27 04:05:50,776][105620] Updated weights for policy 1, policy_version 1747339 (0.0006) [2023-12-27 04:05:50,935][105692] Updated weights for policy 0, policy_version 1743681 (0.0009) [2023-12-27 04:05:50,997][105692] Updated weights for policy 0, policy_version 1743691 (0.0009) [2023-12-27 04:05:51,061][105692] Updated weights for policy 0, policy_version 1743701 (0.0007) [2023-12-27 04:05:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 893829120. Throughput: 0: 9644.9, 1: 9803.3. Samples: 893822296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:05:51,062][104569] Avg episode reward: [(0, '8620.143'), (1, '9263.487')] [2023-12-27 04:05:51,130][105692] Updated weights for policy 0, policy_version 1743711 (0.0006) [2023-12-27 04:05:51,481][105620] Updated weights for policy 1, policy_version 1747349 (0.0009) [2023-12-27 04:05:51,543][105620] Updated weights for policy 1, policy_version 1747359 (0.0011) [2023-12-27 04:05:51,605][105620] Updated weights for policy 1, policy_version 1747369 (0.0008) [2023-12-27 04:05:51,841][105692] Updated weights for policy 0, policy_version 1743721 (0.0006) [2023-12-27 04:05:51,908][105692] Updated weights for policy 0, policy_version 1743731 (0.0007) [2023-12-27 04:05:51,956][105692] Updated weights for policy 0, policy_version 1743741 (0.0008) [2023-12-27 04:05:52,361][105620] Updated weights for policy 1, policy_version 1747379 (0.0008) [2023-12-27 04:05:52,427][105620] Updated weights for policy 1, policy_version 1747389 (0.0009) [2023-12-27 04:05:52,487][105620] Updated weights for policy 1, policy_version 1747399 (0.0008) [2023-12-27 04:05:52,611][105692] Updated weights for policy 0, policy_version 1743751 (0.0007) [2023-12-27 04:05:52,659][105692] Updated weights for policy 0, policy_version 1743761 (0.0005) [2023-12-27 04:05:52,708][105692] Updated weights for policy 0, policy_version 1743771 (0.0010) [2023-12-27 04:05:53,264][105620] Updated weights for policy 1, policy_version 1747409 (0.0010) [2023-12-27 04:05:53,330][105620] Updated weights for policy 1, policy_version 1747419 (0.0010) [2023-12-27 04:05:53,368][105692] Updated weights for policy 0, policy_version 1743781 (0.0008) [2023-12-27 04:05:53,376][105620] Updated weights for policy 1, policy_version 1747429 (0.0008) [2023-12-27 04:05:53,425][105620] Updated weights for policy 1, policy_version 1747439 (0.0009) [2023-12-27 04:05:53,434][105692] Updated weights for policy 0, policy_version 1743791 (0.0006) [2023-12-27 04:05:53,500][105692] Updated weights for policy 0, policy_version 1743801 (0.0005) [2023-12-27 04:05:54,030][105692] Updated weights for policy 0, policy_version 1743811 (0.0009) [2023-12-27 04:05:54,073][105692] Updated weights for policy 0, policy_version 1743821 (0.0005) [2023-12-27 04:05:54,135][105692] Updated weights for policy 0, policy_version 1743831 (0.0008) [2023-12-27 04:05:54,280][105620] Updated weights for policy 1, policy_version 1747449 (0.0005) [2023-12-27 04:05:54,332][105620] Updated weights for policy 1, policy_version 1747459 (0.0008) [2023-12-27 04:05:54,402][105620] Updated weights for policy 1, policy_version 1747469 (0.0008) [2023-12-27 04:05:54,861][105692] Updated weights for policy 0, policy_version 1743841 (0.0010) [2023-12-27 04:05:54,918][105692] Updated weights for policy 0, policy_version 1743851 (0.0010) [2023-12-27 04:05:54,972][105692] Updated weights for policy 0, policy_version 1743861 (0.0011) [2023-12-27 04:05:55,025][105692] Updated weights for policy 0, policy_version 1743871 (0.0010) [2023-12-27 04:05:55,114][105620] Updated weights for policy 1, policy_version 1747479 (0.0008) [2023-12-27 04:05:55,166][105620] Updated weights for policy 1, policy_version 1747489 (0.0008) [2023-12-27 04:05:55,217][105620] Updated weights for policy 1, policy_version 1747499 (0.0007) [2023-12-27 04:05:55,799][105692] Updated weights for policy 0, policy_version 1743881 (0.0010) [2023-12-27 04:05:55,860][105692] Updated weights for policy 0, policy_version 1743891 (0.0010) [2023-12-27 04:05:55,904][105692] Updated weights for policy 0, policy_version 1743901 (0.0010) [2023-12-27 04:05:55,994][105620] Updated weights for policy 1, policy_version 1747509 (0.0008) [2023-12-27 04:05:56,049][105620] Updated weights for policy 1, policy_version 1747519 (0.0006) [2023-12-27 04:05:56,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 893927424. Throughput: 0: 9647.1, 1: 9708.1. Samples: 893937708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:05:56,063][104569] Avg episode reward: [(0, '8712.945'), (1, '8894.624')] [2023-12-27 04:05:56,113][105620] Updated weights for policy 1, policy_version 1747529 (0.0006) [2023-12-27 04:05:56,639][105692] Updated weights for policy 0, policy_version 1743911 (0.0009) [2023-12-27 04:05:56,686][105692] Updated weights for policy 0, policy_version 1743921 (0.0010) [2023-12-27 04:05:56,734][105692] Updated weights for policy 0, policy_version 1743931 (0.0010) [2023-12-27 04:05:56,739][105620] Updated weights for policy 1, policy_version 1747539 (0.0007) [2023-12-27 04:05:56,800][105620] Updated weights for policy 1, policy_version 1747549 (0.0007) [2023-12-27 04:05:56,858][105620] Updated weights for policy 1, policy_version 1747559 (0.0008) [2023-12-27 04:05:57,467][105692] Updated weights for policy 0, policy_version 1743941 (0.0010) [2023-12-27 04:05:57,521][105692] Updated weights for policy 0, policy_version 1743951 (0.0010) [2023-12-27 04:05:57,552][105620] Updated weights for policy 1, policy_version 1747569 (0.0008) [2023-12-27 04:05:57,575][105692] Updated weights for policy 0, policy_version 1743961 (0.0010) [2023-12-27 04:05:57,614][105620] Updated weights for policy 1, policy_version 1747579 (0.0010) [2023-12-27 04:05:57,672][105620] Updated weights for policy 1, policy_version 1747589 (0.0008) [2023-12-27 04:05:57,723][105620] Updated weights for policy 1, policy_version 1747599 (0.0008) [2023-12-27 04:05:58,311][105692] Updated weights for policy 0, policy_version 1743971 (0.0011) [2023-12-27 04:05:58,384][105692] Updated weights for policy 0, policy_version 1743981 (0.0008) [2023-12-27 04:05:58,447][105692] Updated weights for policy 0, policy_version 1743991 (0.0009) [2023-12-27 04:05:58,453][105620] Updated weights for policy 1, policy_version 1747609 (0.0008) [2023-12-27 04:05:58,513][105620] Updated weights for policy 1, policy_version 1747619 (0.0008) [2023-12-27 04:05:58,577][105620] Updated weights for policy 1, policy_version 1747629 (0.0008) [2023-12-27 04:05:59,269][105692] Updated weights for policy 0, policy_version 1744001 (0.0010) [2023-12-27 04:05:59,342][105692] Updated weights for policy 0, policy_version 1744011 (0.0009) [2023-12-27 04:05:59,409][105692] Updated weights for policy 0, policy_version 1744021 (0.0008) [2023-12-27 04:05:59,421][105620] Updated weights for policy 1, policy_version 1747639 (0.0007) [2023-12-27 04:05:59,465][105692] Updated weights for policy 0, policy_version 1744031 (0.0008) [2023-12-27 04:05:59,469][105620] Updated weights for policy 1, policy_version 1747649 (0.0005) [2023-12-27 04:05:59,521][105620] Updated weights for policy 1, policy_version 1747659 (0.0007) [2023-12-27 04:06:00,177][105692] Updated weights for policy 0, policy_version 1744041 (0.0009) [2023-12-27 04:06:00,235][105692] Updated weights for policy 0, policy_version 1744051 (0.0008) [2023-12-27 04:06:00,240][105620] Updated weights for policy 1, policy_version 1747669 (0.0007) [2023-12-27 04:06:00,287][105692] Updated weights for policy 0, policy_version 1744061 (0.0006) [2023-12-27 04:06:00,300][105620] Updated weights for policy 1, policy_version 1747679 (0.0008) [2023-12-27 04:06:00,357][105620] Updated weights for policy 1, policy_version 1747689 (0.0008) [2023-12-27 04:06:00,958][105692] Updated weights for policy 0, policy_version 1744071 (0.0009) [2023-12-27 04:06:01,002][105692] Updated weights for policy 0, policy_version 1744081 (0.0010) [2023-12-27 04:06:01,059][105692] Updated weights for policy 0, policy_version 1744091 (0.0009) [2023-12-27 04:06:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 894017536. Throughput: 0: 9679.1, 1: 9712.3. Samples: 893995292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:06:01,062][104569] Avg episode reward: [(0, '8714.633'), (1, '8984.800')] [2023-12-27 04:06:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001747696_447471616.pth... [2023-12-27 04:06:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001746576_447184896.pth [2023-12-27 04:06:01,086][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001744096_446554112.pth... [2023-12-27 04:06:01,089][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001742944_446259200.pth [2023-12-27 04:06:01,151][105620] Updated weights for policy 1, policy_version 1747699 (0.0009) [2023-12-27 04:06:01,205][105620] Updated weights for policy 1, policy_version 1747709 (0.0008) [2023-12-27 04:06:01,263][105620] Updated weights for policy 1, policy_version 1747719 (0.0008) [2023-12-27 04:06:01,844][105692] Updated weights for policy 0, policy_version 1744101 (0.0008) [2023-12-27 04:06:01,903][105692] Updated weights for policy 0, policy_version 1744111 (0.0009) [2023-12-27 04:06:01,959][105692] Updated weights for policy 0, policy_version 1744121 (0.0008) [2023-12-27 04:06:02,052][105620] Updated weights for policy 1, policy_version 1747729 (0.0009) [2023-12-27 04:06:02,108][105620] Updated weights for policy 1, policy_version 1747739 (0.0009) [2023-12-27 04:06:02,154][105620] Updated weights for policy 1, policy_version 1747749 (0.0008) [2023-12-27 04:06:02,207][105620] Updated weights for policy 1, policy_version 1747759 (0.0009) [2023-12-27 04:06:02,674][105692] Updated weights for policy 0, policy_version 1744131 (0.0009) [2023-12-27 04:06:02,737][105692] Updated weights for policy 0, policy_version 1744141 (0.0009) [2023-12-27 04:06:02,803][105692] Updated weights for policy 0, policy_version 1744151 (0.0010) [2023-12-27 04:06:02,938][105620] Updated weights for policy 1, policy_version 1747769 (0.0009) [2023-12-27 04:06:02,987][105620] Updated weights for policy 1, policy_version 1747779 (0.0009) [2023-12-27 04:06:03,037][105620] Updated weights for policy 1, policy_version 1747789 (0.0007) [2023-12-27 04:06:03,554][105692] Updated weights for policy 0, policy_version 1744161 (0.0009) [2023-12-27 04:06:03,616][105692] Updated weights for policy 0, policy_version 1744171 (0.0010) [2023-12-27 04:06:03,670][105692] Updated weights for policy 0, policy_version 1744181 (0.0010) [2023-12-27 04:06:03,734][105692] Updated weights for policy 0, policy_version 1744191 (0.0011) [2023-12-27 04:06:03,790][105620] Updated weights for policy 1, policy_version 1747799 (0.0007) [2023-12-27 04:06:03,860][105620] Updated weights for policy 1, policy_version 1747809 (0.0007) [2023-12-27 04:06:03,926][105620] Updated weights for policy 1, policy_version 1747819 (0.0008) [2023-12-27 04:06:04,496][105692] Updated weights for policy 0, policy_version 1744201 (0.0009) [2023-12-27 04:06:04,549][105692] Updated weights for policy 0, policy_version 1744211 (0.0008) [2023-12-27 04:06:04,601][105620] Updated weights for policy 1, policy_version 1747829 (0.0007) [2023-12-27 04:06:04,615][105692] Updated weights for policy 0, policy_version 1744221 (0.0006) [2023-12-27 04:06:04,653][105620] Updated weights for policy 1, policy_version 1747839 (0.0005) [2023-12-27 04:06:04,712][105620] Updated weights for policy 1, policy_version 1747849 (0.0006) [2023-12-27 04:06:05,278][105620] Updated weights for policy 1, policy_version 1747859 (0.0006) [2023-12-27 04:06:05,334][105620] Updated weights for policy 1, policy_version 1747869 (0.0005) [2023-12-27 04:06:05,392][105620] Updated weights for policy 1, policy_version 1747879 (0.0005) [2023-12-27 04:06:05,432][105692] Updated weights for policy 0, policy_version 1744231 (0.0010) [2023-12-27 04:06:05,480][105692] Updated weights for policy 0, policy_version 1744241 (0.0010) [2023-12-27 04:06:05,538][105692] Updated weights for policy 0, policy_version 1744251 (0.0010) [2023-12-27 04:06:06,021][105620] Updated weights for policy 1, policy_version 1747889 (0.0006) [2023-12-27 04:06:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 894115840. Throughput: 0: 9527.1, 1: 9748.0. Samples: 894108180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:06:06,062][104569] Avg episode reward: [(0, '8623.539'), (1, '9076.043')] [2023-12-27 04:06:06,076][105620] Updated weights for policy 1, policy_version 1747899 (0.0009) [2023-12-27 04:06:06,138][105620] Updated weights for policy 1, policy_version 1747909 (0.0008) [2023-12-27 04:06:06,142][105692] Updated weights for policy 0, policy_version 1744261 (0.0009) [2023-12-27 04:06:06,192][105620] Updated weights for policy 1, policy_version 1747919 (0.0011) [2023-12-27 04:06:06,202][105692] Updated weights for policy 0, policy_version 1744271 (0.0008) [2023-12-27 04:06:06,268][105692] Updated weights for policy 0, policy_version 1744281 (0.0008) [2023-12-27 04:06:06,990][105692] Updated weights for policy 0, policy_version 1744291 (0.0007) [2023-12-27 04:06:07,004][105620] Updated weights for policy 1, policy_version 1747929 (0.0006) [2023-12-27 04:06:07,052][105692] Updated weights for policy 0, policy_version 1744301 (0.0005) [2023-12-27 04:06:07,074][105620] Updated weights for policy 1, policy_version 1747939 (0.0005) [2023-12-27 04:06:07,114][105692] Updated weights for policy 0, policy_version 1744311 (0.0007) [2023-12-27 04:06:07,136][105620] Updated weights for policy 1, policy_version 1747949 (0.0005) [2023-12-27 04:06:07,731][105692] Updated weights for policy 0, policy_version 1744321 (0.0006) [2023-12-27 04:06:07,789][105620] Updated weights for policy 1, policy_version 1747959 (0.0007) [2023-12-27 04:06:07,793][105692] Updated weights for policy 0, policy_version 1744331 (0.0005) [2023-12-27 04:06:07,846][105692] Updated weights for policy 0, policy_version 1744341 (0.0005) [2023-12-27 04:06:07,856][105620] Updated weights for policy 1, policy_version 1747969 (0.0009) [2023-12-27 04:06:07,895][105692] Updated weights for policy 0, policy_version 1744351 (0.0005) [2023-12-27 04:06:07,913][105620] Updated weights for policy 1, policy_version 1747979 (0.0007) [2023-12-27 04:06:08,483][105692] Updated weights for policy 0, policy_version 1744361 (0.0008) [2023-12-27 04:06:08,536][105692] Updated weights for policy 0, policy_version 1744371 (0.0009) [2023-12-27 04:06:08,597][105692] Updated weights for policy 0, policy_version 1744381 (0.0009) [2023-12-27 04:06:08,700][105620] Updated weights for policy 1, policy_version 1747989 (0.0009) [2023-12-27 04:06:08,763][105620] Updated weights for policy 1, policy_version 1747999 (0.0009) [2023-12-27 04:06:08,817][105620] Updated weights for policy 1, policy_version 1748009 (0.0008) [2023-12-27 04:06:09,400][105692] Updated weights for policy 0, policy_version 1744391 (0.0009) [2023-12-27 04:06:09,464][105692] Updated weights for policy 0, policy_version 1744401 (0.0009) [2023-12-27 04:06:09,523][105692] Updated weights for policy 0, policy_version 1744411 (0.0009) [2023-12-27 04:06:09,579][105620] Updated weights for policy 1, policy_version 1748019 (0.0008) [2023-12-27 04:06:09,638][105620] Updated weights for policy 1, policy_version 1748029 (0.0006) [2023-12-27 04:06:09,698][105620] Updated weights for policy 1, policy_version 1748039 (0.0006) [2023-12-27 04:06:10,361][105692] Updated weights for policy 0, policy_version 1744421 (0.0008) [2023-12-27 04:06:10,363][105620] Updated weights for policy 1, policy_version 1748049 (0.0008) [2023-12-27 04:06:10,418][105692] Updated weights for policy 0, policy_version 1744431 (0.0006) [2023-12-27 04:06:10,424][105620] Updated weights for policy 1, policy_version 1748059 (0.0009) [2023-12-27 04:06:10,476][105692] Updated weights for policy 0, policy_version 1744441 (0.0009) [2023-12-27 04:06:10,483][105620] Updated weights for policy 1, policy_version 1748069 (0.0005) [2023-12-27 04:06:10,555][105620] Updated weights for policy 1, policy_version 1748079 (0.0005) [2023-12-27 04:06:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19633.0). Total num frames: 894214144. Throughput: 0: 9682.5, 1: 9732.7. Samples: 894226052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:06:11,062][104569] Avg episode reward: [(0, '8346.881'), (1, '8988.231')] [2023-12-27 04:06:11,169][105620] Updated weights for policy 1, policy_version 1748089 (0.0008) [2023-12-27 04:06:11,217][105692] Updated weights for policy 0, policy_version 1744451 (0.0009) [2023-12-27 04:06:11,238][105620] Updated weights for policy 1, policy_version 1748099 (0.0006) [2023-12-27 04:06:11,279][105692] Updated weights for policy 0, policy_version 1744461 (0.0009) [2023-12-27 04:06:11,307][105620] Updated weights for policy 1, policy_version 1748109 (0.0007) [2023-12-27 04:06:11,337][105692] Updated weights for policy 0, policy_version 1744471 (0.0009) [2023-12-27 04:06:12,052][105620] Updated weights for policy 1, policy_version 1748119 (0.0009) [2023-12-27 04:06:12,101][105620] Updated weights for policy 1, policy_version 1748129 (0.0009) [2023-12-27 04:06:12,138][105692] Updated weights for policy 0, policy_version 1744481 (0.0009) [2023-12-27 04:06:12,147][105620] Updated weights for policy 1, policy_version 1748139 (0.0009) [2023-12-27 04:06:12,197][105692] Updated weights for policy 0, policy_version 1744491 (0.0009) [2023-12-27 04:06:12,258][105692] Updated weights for policy 0, policy_version 1744501 (0.0009) [2023-12-27 04:06:12,326][105692] Updated weights for policy 0, policy_version 1744511 (0.0009) [2023-12-27 04:06:12,983][105620] Updated weights for policy 1, policy_version 1748149 (0.0007) [2023-12-27 04:06:13,042][105620] Updated weights for policy 1, policy_version 1748159 (0.0007) [2023-12-27 04:06:13,047][105692] Updated weights for policy 0, policy_version 1744521 (0.0008) [2023-12-27 04:06:13,101][105620] Updated weights for policy 1, policy_version 1748169 (0.0008) [2023-12-27 04:06:13,108][105692] Updated weights for policy 0, policy_version 1744531 (0.0009) [2023-12-27 04:06:13,163][105692] Updated weights for policy 0, policy_version 1744541 (0.0010) [2023-12-27 04:06:13,823][105620] Updated weights for policy 1, policy_version 1748179 (0.0007) [2023-12-27 04:06:13,872][105620] Updated weights for policy 1, policy_version 1748189 (0.0008) [2023-12-27 04:06:13,902][105692] Updated weights for policy 0, policy_version 1744551 (0.0008) [2023-12-27 04:06:13,928][105620] Updated weights for policy 1, policy_version 1748199 (0.0007) [2023-12-27 04:06:13,961][105692] Updated weights for policy 0, policy_version 1744561 (0.0008) [2023-12-27 04:06:14,022][105692] Updated weights for policy 0, policy_version 1744571 (0.0008) [2023-12-27 04:06:14,681][105692] Updated weights for policy 0, policy_version 1744581 (0.0007) [2023-12-27 04:06:14,731][105692] Updated weights for policy 0, policy_version 1744591 (0.0006) [2023-12-27 04:06:14,748][105620] Updated weights for policy 1, policy_version 1748209 (0.0008) [2023-12-27 04:06:14,794][105692] Updated weights for policy 0, policy_version 1744601 (0.0006) [2023-12-27 04:06:14,811][105620] Updated weights for policy 1, policy_version 1748219 (0.0009) [2023-12-27 04:06:14,872][105620] Updated weights for policy 1, policy_version 1748229 (0.0009) [2023-12-27 04:06:14,934][105620] Updated weights for policy 1, policy_version 1748239 (0.0009) [2023-12-27 04:06:15,450][105692] Updated weights for policy 0, policy_version 1744611 (0.0006) [2023-12-27 04:06:15,520][105692] Updated weights for policy 0, policy_version 1744621 (0.0005) [2023-12-27 04:06:15,580][105692] Updated weights for policy 0, policy_version 1744631 (0.0010) [2023-12-27 04:06:15,717][105620] Updated weights for policy 1, policy_version 1748249 (0.0008) [2023-12-27 04:06:15,782][105620] Updated weights for policy 1, policy_version 1748259 (0.0009) [2023-12-27 04:06:15,857][105620] Updated weights for policy 1, policy_version 1748269 (0.0010) [2023-12-27 04:06:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19660.8). Total num frames: 894312448. Throughput: 0: 9614.9, 1: 9653.4. Samples: 894281376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:06:16,062][104569] Avg episode reward: [(0, '8348.843'), (1, '9080.723')] [2023-12-27 04:06:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001744640_446693376.pth... [2023-12-27 04:06:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001748272_447619072.pth... [2023-12-27 04:06:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001743520_446406656.pth [2023-12-27 04:06:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001747152_447332352.pth [2023-12-27 04:06:16,103][105692] Updated weights for policy 0, policy_version 1744641 (0.0007) [2023-12-27 04:06:16,176][105692] Updated weights for policy 0, policy_version 1744651 (0.0005) [2023-12-27 04:06:16,244][105692] Updated weights for policy 0, policy_version 1744661 (0.0005) [2023-12-27 04:06:16,291][105692] Updated weights for policy 0, policy_version 1744671 (0.0005) [2023-12-27 04:06:16,677][105620] Updated weights for policy 1, policy_version 1748279 (0.0009) [2023-12-27 04:06:16,743][105620] Updated weights for policy 1, policy_version 1748289 (0.0008) [2023-12-27 04:06:16,802][105620] Updated weights for policy 1, policy_version 1748299 (0.0008) [2023-12-27 04:06:16,856][105692] Updated weights for policy 0, policy_version 1744681 (0.0008) [2023-12-27 04:06:16,916][105692] Updated weights for policy 0, policy_version 1744691 (0.0005) [2023-12-27 04:06:16,984][105692] Updated weights for policy 0, policy_version 1744701 (0.0005) [2023-12-27 04:06:17,478][105692] Updated weights for policy 0, policy_version 1744711 (0.0005) [2023-12-27 04:06:17,538][105692] Updated weights for policy 0, policy_version 1744721 (0.0005) [2023-12-27 04:06:17,601][105692] Updated weights for policy 0, policy_version 1744731 (0.0005) [2023-12-27 04:06:17,681][105620] Updated weights for policy 1, policy_version 1748309 (0.0008) [2023-12-27 04:06:17,744][105620] Updated weights for policy 1, policy_version 1748319 (0.0008) [2023-12-27 04:06:17,802][105620] Updated weights for policy 1, policy_version 1748329 (0.0010) [2023-12-27 04:06:18,125][105692] Updated weights for policy 0, policy_version 1744741 (0.0008) [2023-12-27 04:06:18,183][105692] Updated weights for policy 0, policy_version 1744751 (0.0010) [2023-12-27 04:06:18,241][105692] Updated weights for policy 0, policy_version 1744761 (0.0010) [2023-12-27 04:06:18,643][105620] Updated weights for policy 1, policy_version 1748339 (0.0008) [2023-12-27 04:06:18,702][105620] Updated weights for policy 1, policy_version 1748349 (0.0007) [2023-12-27 04:06:18,755][105620] Updated weights for policy 1, policy_version 1748359 (0.0009) [2023-12-27 04:06:18,925][105692] Updated weights for policy 0, policy_version 1744771 (0.0010) [2023-12-27 04:06:18,990][105692] Updated weights for policy 0, policy_version 1744781 (0.0010) [2023-12-27 04:06:19,048][105692] Updated weights for policy 0, policy_version 1744791 (0.0009) [2023-12-27 04:06:19,471][105620] Updated weights for policy 1, policy_version 1748369 (0.0008) [2023-12-27 04:06:19,531][105620] Updated weights for policy 1, policy_version 1748379 (0.0011) [2023-12-27 04:06:19,594][105620] Updated weights for policy 1, policy_version 1748389 (0.0011) [2023-12-27 04:06:19,661][105620] Updated weights for policy 1, policy_version 1748399 (0.0011) [2023-12-27 04:06:19,786][105692] Updated weights for policy 0, policy_version 1744801 (0.0009) [2023-12-27 04:06:19,852][105692] Updated weights for policy 0, policy_version 1744811 (0.0009) [2023-12-27 04:06:19,912][105692] Updated weights for policy 0, policy_version 1744821 (0.0009) [2023-12-27 04:06:19,980][105692] Updated weights for policy 0, policy_version 1744831 (0.0009) [2023-12-27 04:06:20,411][105620] Updated weights for policy 1, policy_version 1748409 (0.0011) [2023-12-27 04:06:20,468][105620] Updated weights for policy 1, policy_version 1748419 (0.0011) [2023-12-27 04:06:20,524][105620] Updated weights for policy 1, policy_version 1748429 (0.0011) [2023-12-27 04:06:20,798][105692] Updated weights for policy 0, policy_version 1744841 (0.0008) [2023-12-27 04:06:20,854][105692] Updated weights for policy 0, policy_version 1744851 (0.0008) [2023-12-27 04:06:20,907][105692] Updated weights for policy 0, policy_version 1744861 (0.0008) [2023-12-27 04:06:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19660.8). Total num frames: 894410752. Throughput: 0: 9826.0, 1: 9535.7. Samples: 894400360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:06:21,062][104569] Avg episode reward: [(0, '8348.485'), (1, '9082.667')] [2023-12-27 04:06:21,257][105620] Updated weights for policy 1, policy_version 1748439 (0.0011) [2023-12-27 04:06:21,309][105620] Updated weights for policy 1, policy_version 1748449 (0.0011) [2023-12-27 04:06:21,367][105620] Updated weights for policy 1, policy_version 1748459 (0.0010) [2023-12-27 04:06:21,682][105692] Updated weights for policy 0, policy_version 1744871 (0.0006) [2023-12-27 04:06:21,751][105692] Updated weights for policy 0, policy_version 1744881 (0.0009) [2023-12-27 04:06:21,802][105692] Updated weights for policy 0, policy_version 1744891 (0.0009) [2023-12-27 04:06:22,165][105620] Updated weights for policy 1, policy_version 1748469 (0.0009) [2023-12-27 04:06:22,225][105620] Updated weights for policy 1, policy_version 1748479 (0.0009) [2023-12-27 04:06:22,283][105620] Updated weights for policy 1, policy_version 1748489 (0.0009) [2023-12-27 04:06:22,566][105692] Updated weights for policy 0, policy_version 1744901 (0.0007) [2023-12-27 04:06:22,631][105692] Updated weights for policy 0, policy_version 1744911 (0.0009) [2023-12-27 04:06:22,690][105692] Updated weights for policy 0, policy_version 1744921 (0.0009) [2023-12-27 04:06:23,098][105620] Updated weights for policy 1, policy_version 1748499 (0.0010) [2023-12-27 04:06:23,161][105620] Updated weights for policy 1, policy_version 1748509 (0.0010) [2023-12-27 04:06:23,214][105620] Updated weights for policy 1, policy_version 1748520 (0.0010) [2023-12-27 04:06:23,322][105692] Updated weights for policy 0, policy_version 1744931 (0.0007) [2023-12-27 04:06:23,378][105692] Updated weights for policy 0, policy_version 1744941 (0.0005) [2023-12-27 04:06:23,435][105692] Updated weights for policy 0, policy_version 1744951 (0.0005) [2023-12-27 04:06:23,981][105692] Updated weights for policy 0, policy_version 1744961 (0.0010) [2023-12-27 04:06:24,042][105692] Updated weights for policy 0, policy_version 1744971 (0.0008) [2023-12-27 04:06:24,091][105692] Updated weights for policy 0, policy_version 1744981 (0.0008) [2023-12-27 04:06:24,104][105620] Updated weights for policy 1, policy_version 1748531 (0.0010) [2023-12-27 04:06:24,146][105692] Updated weights for policy 0, policy_version 1744991 (0.0009) [2023-12-27 04:06:24,167][105620] Updated weights for policy 1, policy_version 1748541 (0.0008) [2023-12-27 04:06:24,228][105620] Updated weights for policy 1, policy_version 1748551 (0.0010) [2023-12-27 04:06:24,867][105692] Updated weights for policy 0, policy_version 1745001 (0.0009) [2023-12-27 04:06:24,923][105692] Updated weights for policy 0, policy_version 1745011 (0.0009) [2023-12-27 04:06:24,974][105692] Updated weights for policy 0, policy_version 1745021 (0.0009) [2023-12-27 04:06:25,000][105620] Updated weights for policy 1, policy_version 1748561 (0.0009) [2023-12-27 04:06:25,063][105620] Updated weights for policy 1, policy_version 1748571 (0.0009) [2023-12-27 04:06:25,127][105620] Updated weights for policy 1, policy_version 1748581 (0.0008) [2023-12-27 04:06:25,189][105620] Updated weights for policy 1, policy_version 1748591 (0.0008) [2023-12-27 04:06:25,707][105692] Updated weights for policy 0, policy_version 1745031 (0.0010) [2023-12-27 04:06:25,762][105692] Updated weights for policy 0, policy_version 1745041 (0.0010) [2023-12-27 04:06:25,820][105692] Updated weights for policy 0, policy_version 1745051 (0.0010) [2023-12-27 04:06:25,920][105620] Updated weights for policy 1, policy_version 1748601 (0.0008) [2023-12-27 04:06:25,980][105620] Updated weights for policy 1, policy_version 1748611 (0.0007) [2023-12-27 04:06:26,030][105620] Updated weights for policy 1, policy_version 1748621 (0.0006) [2023-12-27 04:06:26,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 894509056. Throughput: 0: 9925.6, 1: 9378.2. Samples: 894512668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:06:26,063][104569] Avg episode reward: [(0, '8531.788'), (1, '8990.043')] [2023-12-27 04:06:26,517][105692] Updated weights for policy 0, policy_version 1745061 (0.0010) [2023-12-27 04:06:26,566][105692] Updated weights for policy 0, policy_version 1745071 (0.0009) [2023-12-27 04:06:26,625][105692] Updated weights for policy 0, policy_version 1745081 (0.0009) [2023-12-27 04:06:26,699][105620] Updated weights for policy 1, policy_version 1748631 (0.0007) [2023-12-27 04:06:26,745][105620] Updated weights for policy 1, policy_version 1748641 (0.0009) [2023-12-27 04:06:26,792][105620] Updated weights for policy 1, policy_version 1748651 (0.0009) [2023-12-27 04:06:27,405][105620] Updated weights for policy 1, policy_version 1748661 (0.0007) [2023-12-27 04:06:27,453][105620] Updated weights for policy 1, policy_version 1748671 (0.0007) [2023-12-27 04:06:27,460][105692] Updated weights for policy 0, policy_version 1745091 (0.0008) [2023-12-27 04:06:27,507][105620] Updated weights for policy 1, policy_version 1748681 (0.0007) [2023-12-27 04:06:27,507][105692] Updated weights for policy 0, policy_version 1745101 (0.0005) [2023-12-27 04:06:27,568][105692] Updated weights for policy 0, policy_version 1745111 (0.0009) [2023-12-27 04:06:28,191][105620] Updated weights for policy 1, policy_version 1748691 (0.0006) [2023-12-27 04:06:28,244][105620] Updated weights for policy 1, policy_version 1748701 (0.0010) [2023-12-27 04:06:28,273][105692] Updated weights for policy 0, policy_version 1745121 (0.0009) [2023-12-27 04:06:28,293][105620] Updated weights for policy 1, policy_version 1748711 (0.0010) [2023-12-27 04:06:28,338][105692] Updated weights for policy 0, policy_version 1745131 (0.0009) [2023-12-27 04:06:28,410][105692] Updated weights for policy 0, policy_version 1745141 (0.0007) [2023-12-27 04:06:28,488][105692] Updated weights for policy 0, policy_version 1745151 (0.0006) [2023-12-27 04:06:29,055][105620] Updated weights for policy 1, policy_version 1748721 (0.0007) [2023-12-27 04:06:29,087][105692] Updated weights for policy 0, policy_version 1745161 (0.0008) [2023-12-27 04:06:29,113][105620] Updated weights for policy 1, policy_version 1748731 (0.0007) [2023-12-27 04:06:29,145][105692] Updated weights for policy 0, policy_version 1745171 (0.0006) [2023-12-27 04:06:29,172][105620] Updated weights for policy 1, policy_version 1748741 (0.0009) [2023-12-27 04:06:29,194][105692] Updated weights for policy 0, policy_version 1745181 (0.0009) [2023-12-27 04:06:29,229][105620] Updated weights for policy 1, policy_version 1748751 (0.0007) [2023-12-27 04:06:29,975][105692] Updated weights for policy 0, policy_version 1745191 (0.0011) [2023-12-27 04:06:29,981][105620] Updated weights for policy 1, policy_version 1748761 (0.0009) [2023-12-27 04:06:30,031][105692] Updated weights for policy 0, policy_version 1745201 (0.0008) [2023-12-27 04:06:30,041][105620] Updated weights for policy 1, policy_version 1748771 (0.0007) [2023-12-27 04:06:30,083][105692] Updated weights for policy 0, policy_version 1745211 (0.0010) [2023-12-27 04:06:30,105][105620] Updated weights for policy 1, policy_version 1748781 (0.0006) [2023-12-27 04:06:30,820][105692] Updated weights for policy 0, policy_version 1745221 (0.0010) [2023-12-27 04:06:30,850][105620] Updated weights for policy 1, policy_version 1748791 (0.0009) [2023-12-27 04:06:30,874][105692] Updated weights for policy 0, policy_version 1745231 (0.0010) [2023-12-27 04:06:30,905][105620] Updated weights for policy 1, policy_version 1748801 (0.0010) [2023-12-27 04:06:30,929][105692] Updated weights for policy 0, policy_version 1745241 (0.0010) [2023-12-27 04:06:30,953][105620] Updated weights for policy 1, policy_version 1748811 (0.0010) [2023-12-27 04:06:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 894607360. Throughput: 0: 9911.9, 1: 9394.4. Samples: 894572356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:06:31,063][104569] Avg episode reward: [(0, '8531.352'), (1, '8987.178')] [2023-12-27 04:06:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001745248_446849024.pth... [2023-12-27 04:06:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001748816_447758336.pth... [2023-12-27 04:06:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001744096_446554112.pth [2023-12-27 04:06:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001747696_447471616.pth [2023-12-27 04:06:31,640][105692] Updated weights for policy 0, policy_version 1745251 (0.0010) [2023-12-27 04:06:31,705][105692] Updated weights for policy 0, policy_version 1745261 (0.0006) [2023-12-27 04:06:31,727][105620] Updated weights for policy 1, policy_version 1748821 (0.0010) [2023-12-27 04:06:31,773][105692] Updated weights for policy 0, policy_version 1745271 (0.0008) [2023-12-27 04:06:31,787][105620] Updated weights for policy 1, policy_version 1748831 (0.0011) [2023-12-27 04:06:31,850][105620] Updated weights for policy 1, policy_version 1748841 (0.0011) [2023-12-27 04:06:32,440][105692] Updated weights for policy 0, policy_version 1745281 (0.0008) [2023-12-27 04:06:32,497][105692] Updated weights for policy 0, policy_version 1745291 (0.0009) [2023-12-27 04:06:32,555][105692] Updated weights for policy 0, policy_version 1745301 (0.0006) [2023-12-27 04:06:32,579][105620] Updated weights for policy 1, policy_version 1748851 (0.0010) [2023-12-27 04:06:32,609][105692] Updated weights for policy 0, policy_version 1745311 (0.0007) [2023-12-27 04:06:32,628][105620] Updated weights for policy 1, policy_version 1748861 (0.0008) [2023-12-27 04:06:32,678][105620] Updated weights for policy 1, policy_version 1748871 (0.0008) [2023-12-27 04:06:33,325][105620] Updated weights for policy 1, policy_version 1748881 (0.0007) [2023-12-27 04:06:33,357][105692] Updated weights for policy 0, policy_version 1745321 (0.0008) [2023-12-27 04:06:33,376][105620] Updated weights for policy 1, policy_version 1748891 (0.0010) [2023-12-27 04:06:33,418][105692] Updated weights for policy 0, policy_version 1745331 (0.0007) [2023-12-27 04:06:33,437][105620] Updated weights for policy 1, policy_version 1748901 (0.0010) [2023-12-27 04:06:33,478][105692] Updated weights for policy 0, policy_version 1745341 (0.0009) [2023-12-27 04:06:33,495][105620] Updated weights for policy 1, policy_version 1748911 (0.0010) [2023-12-27 04:06:34,098][105620] Updated weights for policy 1, policy_version 1748921 (0.0010) [2023-12-27 04:06:34,153][105620] Updated weights for policy 1, policy_version 1748931 (0.0007) [2023-12-27 04:06:34,210][105620] Updated weights for policy 1, policy_version 1748941 (0.0007) [2023-12-27 04:06:34,322][105692] Updated weights for policy 0, policy_version 1745351 (0.0008) [2023-12-27 04:06:34,380][105692] Updated weights for policy 0, policy_version 1745361 (0.0010) [2023-12-27 04:06:34,434][105692] Updated weights for policy 0, policy_version 1745371 (0.0009) [2023-12-27 04:06:34,780][105620] Updated weights for policy 1, policy_version 1748951 (0.0009) [2023-12-27 04:06:34,848][105620] Updated weights for policy 1, policy_version 1748961 (0.0010) [2023-12-27 04:06:34,902][105620] Updated weights for policy 1, policy_version 1748971 (0.0010) [2023-12-27 04:06:35,297][105692] Updated weights for policy 0, policy_version 1745381 (0.0009) [2023-12-27 04:06:35,350][105692] Updated weights for policy 0, policy_version 1745391 (0.0008) [2023-12-27 04:06:35,395][105692] Updated weights for policy 0, policy_version 1745401 (0.0008) [2023-12-27 04:06:35,538][105620] Updated weights for policy 1, policy_version 1748981 (0.0009) [2023-12-27 04:06:35,596][105620] Updated weights for policy 1, policy_version 1748991 (0.0010) [2023-12-27 04:06:35,654][105620] Updated weights for policy 1, policy_version 1749001 (0.0010) [2023-12-27 04:06:36,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 894697472. Throughput: 0: 9766.8, 1: 9476.5. Samples: 894688248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:06:36,063][104569] Avg episode reward: [(0, '8439.599'), (1, '8985.846')] [2023-12-27 04:06:36,174][105692] Updated weights for policy 0, policy_version 1745411 (0.0009) [2023-12-27 04:06:36,232][105692] Updated weights for policy 0, policy_version 1745421 (0.0008) [2023-12-27 04:06:36,295][105692] Updated weights for policy 0, policy_version 1745431 (0.0007) [2023-12-27 04:06:36,458][105620] Updated weights for policy 1, policy_version 1749011 (0.0010) [2023-12-27 04:06:36,528][105620] Updated weights for policy 1, policy_version 1749021 (0.0010) [2023-12-27 04:06:36,594][105620] Updated weights for policy 1, policy_version 1749031 (0.0011) [2023-12-27 04:06:36,991][105692] Updated weights for policy 0, policy_version 1745441 (0.0006) [2023-12-27 04:06:37,052][105692] Updated weights for policy 0, policy_version 1745451 (0.0008) [2023-12-27 04:06:37,098][105692] Updated weights for policy 0, policy_version 1745461 (0.0008) [2023-12-27 04:06:37,147][105692] Updated weights for policy 0, policy_version 1745471 (0.0008) [2023-12-27 04:06:37,286][105620] Updated weights for policy 1, policy_version 1749041 (0.0007) [2023-12-27 04:06:37,345][105620] Updated weights for policy 1, policy_version 1749051 (0.0010) [2023-12-27 04:06:37,397][105620] Updated weights for policy 1, policy_version 1749061 (0.0010) [2023-12-27 04:06:37,454][105620] Updated weights for policy 1, policy_version 1749071 (0.0009) [2023-12-27 04:06:37,946][105692] Updated weights for policy 0, policy_version 1745481 (0.0008) [2023-12-27 04:06:37,998][105692] Updated weights for policy 0, policy_version 1745491 (0.0007) [2023-12-27 04:06:38,048][105692] Updated weights for policy 0, policy_version 1745501 (0.0005) [2023-12-27 04:06:38,244][105620] Updated weights for policy 1, policy_version 1749081 (0.0008) [2023-12-27 04:06:38,313][105620] Updated weights for policy 1, policy_version 1749091 (0.0009) [2023-12-27 04:06:38,381][105620] Updated weights for policy 1, policy_version 1749101 (0.0011) [2023-12-27 04:06:38,702][105692] Updated weights for policy 0, policy_version 1745511 (0.0007) [2023-12-27 04:06:38,754][105692] Updated weights for policy 0, policy_version 1745521 (0.0008) [2023-12-27 04:06:38,809][105692] Updated weights for policy 0, policy_version 1745531 (0.0008) [2023-12-27 04:06:39,027][105620] Updated weights for policy 1, policy_version 1749111 (0.0006) [2023-12-27 04:06:39,083][105620] Updated weights for policy 1, policy_version 1749121 (0.0007) [2023-12-27 04:06:39,142][105620] Updated weights for policy 1, policy_version 1749131 (0.0009) [2023-12-27 04:06:39,602][105692] Updated weights for policy 0, policy_version 1745541 (0.0008) [2023-12-27 04:06:39,654][105692] Updated weights for policy 0, policy_version 1745551 (0.0008) [2023-12-27 04:06:39,710][105692] Updated weights for policy 0, policy_version 1745561 (0.0008) [2023-12-27 04:06:39,786][105620] Updated weights for policy 1, policy_version 1749141 (0.0007) [2023-12-27 04:06:39,852][105620] Updated weights for policy 1, policy_version 1749151 (0.0007) [2023-12-27 04:06:39,915][105620] Updated weights for policy 1, policy_version 1749161 (0.0007) [2023-12-27 04:06:40,483][105692] Updated weights for policy 0, policy_version 1745571 (0.0007) [2023-12-27 04:06:40,546][105692] Updated weights for policy 0, policy_version 1745581 (0.0009) [2023-12-27 04:06:40,612][105692] Updated weights for policy 0, policy_version 1745591 (0.0009) [2023-12-27 04:06:40,658][105620] Updated weights for policy 1, policy_version 1749171 (0.0009) [2023-12-27 04:06:40,714][105620] Updated weights for policy 1, policy_version 1749181 (0.0010) [2023-12-27 04:06:40,773][105620] Updated weights for policy 1, policy_version 1749191 (0.0009) [2023-12-27 04:06:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 894795776. Throughput: 0: 9696.1, 1: 9522.6. Samples: 894802548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:06:41,062][104569] Avg episode reward: [(0, '8894.805'), (1, '8985.179')] [2023-12-27 04:06:41,394][105692] Updated weights for policy 0, policy_version 1745601 (0.0008) [2023-12-27 04:06:41,449][105692] Updated weights for policy 0, policy_version 1745611 (0.0009) [2023-12-27 04:06:41,501][105692] Updated weights for policy 0, policy_version 1745621 (0.0009) [2023-12-27 04:06:41,552][105692] Updated weights for policy 0, policy_version 1745631 (0.0009) [2023-12-27 04:06:41,607][105620] Updated weights for policy 1, policy_version 1749201 (0.0009) [2023-12-27 04:06:41,677][105620] Updated weights for policy 1, policy_version 1749211 (0.0010) [2023-12-27 04:06:41,746][105620] Updated weights for policy 1, policy_version 1749221 (0.0009) [2023-12-27 04:06:41,798][105620] Updated weights for policy 1, policy_version 1749231 (0.0009) [2023-12-27 04:06:42,329][105692] Updated weights for policy 0, policy_version 1745641 (0.0009) [2023-12-27 04:06:42,398][105692] Updated weights for policy 0, policy_version 1745651 (0.0008) [2023-12-27 04:06:42,451][105692] Updated weights for policy 0, policy_version 1745662 (0.0009) [2023-12-27 04:06:42,581][105620] Updated weights for policy 1, policy_version 1749241 (0.0006) [2023-12-27 04:06:42,648][105620] Updated weights for policy 1, policy_version 1749251 (0.0006) [2023-12-27 04:06:42,709][105620] Updated weights for policy 1, policy_version 1749261 (0.0006) [2023-12-27 04:06:43,315][105692] Updated weights for policy 0, policy_version 1745672 (0.0008) [2023-12-27 04:06:43,329][105620] Updated weights for policy 1, policy_version 1749271 (0.0007) [2023-12-27 04:06:43,372][105692] Updated weights for policy 0, policy_version 1745682 (0.0006) [2023-12-27 04:06:43,386][105620] Updated weights for policy 1, policy_version 1749281 (0.0008) [2023-12-27 04:06:43,430][105692] Updated weights for policy 0, policy_version 1745692 (0.0006) [2023-12-27 04:06:43,440][105620] Updated weights for policy 1, policy_version 1749291 (0.0006) [2023-12-27 04:06:44,132][105620] Updated weights for policy 1, policy_version 1749301 (0.0006) [2023-12-27 04:06:44,181][105620] Updated weights for policy 1, policy_version 1749311 (0.0007) [2023-12-27 04:06:44,231][105692] Updated weights for policy 0, policy_version 1745702 (0.0008) [2023-12-27 04:06:44,240][105620] Updated weights for policy 1, policy_version 1749321 (0.0008) [2023-12-27 04:06:44,290][105692] Updated weights for policy 0, policy_version 1745712 (0.0006) [2023-12-27 04:06:44,350][105692] Updated weights for policy 0, policy_version 1745722 (0.0010) [2023-12-27 04:06:44,885][105620] Updated weights for policy 1, policy_version 1749331 (0.0007) [2023-12-27 04:06:44,954][105620] Updated weights for policy 1, policy_version 1749341 (0.0008) [2023-12-27 04:06:45,014][105620] Updated weights for policy 1, policy_version 1749351 (0.0008) [2023-12-27 04:06:45,123][105692] Updated weights for policy 0, policy_version 1745732 (0.0010) [2023-12-27 04:06:45,184][105692] Updated weights for policy 0, policy_version 1745742 (0.0011) [2023-12-27 04:06:45,251][105692] Updated weights for policy 0, policy_version 1745752 (0.0011) [2023-12-27 04:06:45,780][105620] Updated weights for policy 1, policy_version 1749361 (0.0011) [2023-12-27 04:06:45,804][105692] Updated weights for policy 0, policy_version 1745762 (0.0006) [2023-12-27 04:06:45,838][105620] Updated weights for policy 1, policy_version 1749371 (0.0010) [2023-12-27 04:06:45,851][105692] Updated weights for policy 0, policy_version 1745772 (0.0005) [2023-12-27 04:06:45,895][105692] Updated weights for policy 0, policy_version 1745782 (0.0005) [2023-12-27 04:06:45,897][105620] Updated weights for policy 1, policy_version 1749381 (0.0010) [2023-12-27 04:06:45,946][105692] Updated weights for policy 0, policy_version 1745792 (0.0009) [2023-12-27 04:06:45,949][105620] Updated weights for policy 1, policy_version 1749391 (0.0010) [2023-12-27 04:06:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 894894080. Throughput: 0: 9659.6, 1: 9506.1. Samples: 894857752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:06:46,063][104569] Avg episode reward: [(0, '8895.645'), (1, '8985.492')] [2023-12-27 04:06:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001749392_447905792.pth... [2023-12-27 04:06:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001745792_446988288.pth... [2023-12-27 04:06:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001748272_447619072.pth [2023-12-27 04:06:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001744640_446693376.pth [2023-12-27 04:06:46,540][105692] Updated weights for policy 0, policy_version 1745802 (0.0008) [2023-12-27 04:06:46,597][105692] Updated weights for policy 0, policy_version 1745812 (0.0008) [2023-12-27 04:06:46,623][105620] Updated weights for policy 1, policy_version 1749401 (0.0008) [2023-12-27 04:06:46,650][105692] Updated weights for policy 0, policy_version 1745822 (0.0007) [2023-12-27 04:06:46,673][105620] Updated weights for policy 1, policy_version 1749411 (0.0006) [2023-12-27 04:06:46,732][105620] Updated weights for policy 1, policy_version 1749421 (0.0009) [2023-12-27 04:06:47,337][105620] Updated weights for policy 1, policy_version 1749431 (0.0009) [2023-12-27 04:06:47,392][105620] Updated weights for policy 1, policy_version 1749441 (0.0009) [2023-12-27 04:06:47,442][105620] Updated weights for policy 1, policy_version 1749451 (0.0009) [2023-12-27 04:06:47,463][105692] Updated weights for policy 0, policy_version 1745832 (0.0008) [2023-12-27 04:06:47,514][105692] Updated weights for policy 0, policy_version 1745842 (0.0007) [2023-12-27 04:06:47,562][105692] Updated weights for policy 0, policy_version 1745852 (0.0008) [2023-12-27 04:06:48,158][105620] Updated weights for policy 1, policy_version 1749461 (0.0008) [2023-12-27 04:06:48,206][105620] Updated weights for policy 1, policy_version 1749471 (0.0010) [2023-12-27 04:06:48,272][105620] Updated weights for policy 1, policy_version 1749481 (0.0009) [2023-12-27 04:06:48,344][105692] Updated weights for policy 0, policy_version 1745862 (0.0010) [2023-12-27 04:06:48,409][105692] Updated weights for policy 0, policy_version 1745872 (0.0011) [2023-12-27 04:06:48,473][105692] Updated weights for policy 0, policy_version 1745882 (0.0009) [2023-12-27 04:06:48,904][105620] Updated weights for policy 1, policy_version 1749491 (0.0008) [2023-12-27 04:06:48,962][105620] Updated weights for policy 1, policy_version 1749501 (0.0005) [2023-12-27 04:06:49,017][105620] Updated weights for policy 1, policy_version 1749511 (0.0005) [2023-12-27 04:06:49,172][105692] Updated weights for policy 0, policy_version 1745892 (0.0008) [2023-12-27 04:06:49,234][105692] Updated weights for policy 0, policy_version 1745902 (0.0007) [2023-12-27 04:06:49,303][105692] Updated weights for policy 0, policy_version 1745912 (0.0008) [2023-12-27 04:06:49,631][105620] Updated weights for policy 1, policy_version 1749521 (0.0008) [2023-12-27 04:06:49,691][105620] Updated weights for policy 1, policy_version 1749531 (0.0008) [2023-12-27 04:06:49,753][105620] Updated weights for policy 1, policy_version 1749541 (0.0009) [2023-12-27 04:06:49,804][105620] Updated weights for policy 1, policy_version 1749551 (0.0008) [2023-12-27 04:06:50,110][105692] Updated weights for policy 0, policy_version 1745922 (0.0009) [2023-12-27 04:06:50,172][105692] Updated weights for policy 0, policy_version 1745932 (0.0009) [2023-12-27 04:06:50,238][105692] Updated weights for policy 0, policy_version 1745942 (0.0009) [2023-12-27 04:06:50,294][105692] Updated weights for policy 0, policy_version 1745952 (0.0009) [2023-12-27 04:06:50,542][105620] Updated weights for policy 1, policy_version 1749561 (0.0010) [2023-12-27 04:06:50,598][105620] Updated weights for policy 1, policy_version 1749571 (0.0008) [2023-12-27 04:06:50,660][105620] Updated weights for policy 1, policy_version 1749581 (0.0010) [2023-12-27 04:06:50,994][105692] Updated weights for policy 0, policy_version 1745962 (0.0010) [2023-12-27 04:06:51,053][105692] Updated weights for policy 0, policy_version 1745972 (0.0009) [2023-12-27 04:06:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 894984192. Throughput: 0: 9699.2, 1: 9632.4. Samples: 894978100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:06:51,062][104569] Avg episode reward: [(0, '8624.979'), (1, '8988.376')] [2023-12-27 04:06:51,116][105692] Updated weights for policy 0, policy_version 1745982 (0.0007) [2023-12-27 04:06:51,487][105620] Updated weights for policy 1, policy_version 1749591 (0.0009) [2023-12-27 04:06:51,542][105620] Updated weights for policy 1, policy_version 1749601 (0.0010) [2023-12-27 04:06:51,605][105620] Updated weights for policy 1, policy_version 1749611 (0.0009) [2023-12-27 04:06:51,845][105692] Updated weights for policy 0, policy_version 1745992 (0.0009) [2023-12-27 04:06:51,897][105692] Updated weights for policy 0, policy_version 1746002 (0.0009) [2023-12-27 04:06:51,948][105692] Updated weights for policy 0, policy_version 1746012 (0.0009) [2023-12-27 04:06:52,411][105620] Updated weights for policy 1, policy_version 1749621 (0.0009) [2023-12-27 04:06:52,470][105620] Updated weights for policy 1, policy_version 1749631 (0.0009) [2023-12-27 04:06:52,528][105620] Updated weights for policy 1, policy_version 1749641 (0.0009) [2023-12-27 04:06:52,652][105692] Updated weights for policy 0, policy_version 1746022 (0.0010) [2023-12-27 04:06:52,700][105692] Updated weights for policy 0, policy_version 1746032 (0.0009) [2023-12-27 04:06:52,759][105692] Updated weights for policy 0, policy_version 1746042 (0.0008) [2023-12-27 04:06:53,254][105620] Updated weights for policy 1, policy_version 1749651 (0.0009) [2023-12-27 04:06:53,323][105620] Updated weights for policy 1, policy_version 1749661 (0.0011) [2023-12-27 04:06:53,376][105620] Updated weights for policy 1, policy_version 1749671 (0.0009) [2023-12-27 04:06:53,485][105692] Updated weights for policy 0, policy_version 1746052 (0.0008) [2023-12-27 04:06:53,543][105692] Updated weights for policy 0, policy_version 1746062 (0.0007) [2023-12-27 04:06:53,605][105692] Updated weights for policy 0, policy_version 1746072 (0.0005) [2023-12-27 04:06:54,143][105620] Updated weights for policy 1, policy_version 1749681 (0.0009) [2023-12-27 04:06:54,157][105692] Updated weights for policy 0, policy_version 1746082 (0.0005) [2023-12-27 04:06:54,194][105620] Updated weights for policy 1, policy_version 1749691 (0.0009) [2023-12-27 04:06:54,211][105692] Updated weights for policy 0, policy_version 1746092 (0.0006) [2023-12-27 04:06:54,246][105620] Updated weights for policy 1, policy_version 1749701 (0.0008) [2023-12-27 04:06:54,266][105692] Updated weights for policy 0, policy_version 1746102 (0.0005) [2023-12-27 04:06:54,301][105620] Updated weights for policy 1, policy_version 1749711 (0.0008) [2023-12-27 04:06:54,315][105692] Updated weights for policy 0, policy_version 1746112 (0.0005) [2023-12-27 04:06:54,855][105692] Updated weights for policy 0, policy_version 1746122 (0.0006) [2023-12-27 04:06:54,917][105692] Updated weights for policy 0, policy_version 1746132 (0.0005) [2023-12-27 04:06:54,986][105692] Updated weights for policy 0, policy_version 1746142 (0.0009) [2023-12-27 04:06:55,103][105620] Updated weights for policy 1, policy_version 1749721 (0.0007) [2023-12-27 04:06:55,162][105620] Updated weights for policy 1, policy_version 1749731 (0.0011) [2023-12-27 04:06:55,221][105620] Updated weights for policy 1, policy_version 1749741 (0.0006) [2023-12-27 04:06:55,604][105692] Updated weights for policy 0, policy_version 1746152 (0.0009) [2023-12-27 04:06:55,661][105692] Updated weights for policy 0, policy_version 1746162 (0.0006) [2023-12-27 04:06:55,720][105692] Updated weights for policy 0, policy_version 1746172 (0.0007) [2023-12-27 04:06:55,827][105620] Updated weights for policy 1, policy_version 1749751 (0.0006) [2023-12-27 04:06:55,879][105620] Updated weights for policy 1, policy_version 1749761 (0.0010) [2023-12-27 04:06:55,931][105620] Updated weights for policy 1, policy_version 1749771 (0.0005) [2023-12-27 04:06:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 895090688. Throughput: 0: 9787.7, 1: 9553.5. Samples: 895096404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:06:56,062][104569] Avg episode reward: [(0, '8897.436'), (1, '8894.480')] [2023-12-27 04:06:56,465][105692] Updated weights for policy 0, policy_version 1746182 (0.0007) [2023-12-27 04:06:56,512][105692] Updated weights for policy 0, policy_version 1746192 (0.0008) [2023-12-27 04:06:56,546][105620] Updated weights for policy 1, policy_version 1749781 (0.0008) [2023-12-27 04:06:56,565][105692] Updated weights for policy 0, policy_version 1746202 (0.0006) [2023-12-27 04:06:56,597][105620] Updated weights for policy 1, policy_version 1749791 (0.0009) [2023-12-27 04:06:56,648][105620] Updated weights for policy 1, policy_version 1749801 (0.0006) [2023-12-27 04:06:57,279][105620] Updated weights for policy 1, policy_version 1749811 (0.0007) [2023-12-27 04:06:57,335][105620] Updated weights for policy 1, policy_version 1749821 (0.0010) [2023-12-27 04:06:57,354][105692] Updated weights for policy 0, policy_version 1746212 (0.0006) [2023-12-27 04:06:57,383][105620] Updated weights for policy 1, policy_version 1749831 (0.0010) [2023-12-27 04:06:57,397][105692] Updated weights for policy 0, policy_version 1746222 (0.0005) [2023-12-27 04:06:57,441][105692] Updated weights for policy 0, policy_version 1746232 (0.0005) [2023-12-27 04:06:57,955][105620] Updated weights for policy 1, policy_version 1749841 (0.0010) [2023-12-27 04:06:58,013][105620] Updated weights for policy 1, policy_version 1749851 (0.0009) [2023-12-27 04:06:58,079][105620] Updated weights for policy 1, policy_version 1749861 (0.0009) [2023-12-27 04:06:58,143][105620] Updated weights for policy 1, policy_version 1749871 (0.0008) [2023-12-27 04:06:58,150][105692] Updated weights for policy 0, policy_version 1746242 (0.0006) [2023-12-27 04:06:58,223][105692] Updated weights for policy 0, policy_version 1746252 (0.0008) [2023-12-27 04:06:58,283][105692] Updated weights for policy 0, policy_version 1746262 (0.0008) [2023-12-27 04:06:58,356][105692] Updated weights for policy 0, policy_version 1746272 (0.0008) [2023-12-27 04:06:58,968][105620] Updated weights for policy 1, policy_version 1749881 (0.0009) [2023-12-27 04:06:59,014][105620] Updated weights for policy 1, policy_version 1749891 (0.0008) [2023-12-27 04:06:59,061][105620] Updated weights for policy 1, policy_version 1749901 (0.0007) [2023-12-27 04:06:59,067][105692] Updated weights for policy 0, policy_version 1746282 (0.0006) [2023-12-27 04:06:59,132][105692] Updated weights for policy 0, policy_version 1746292 (0.0009) [2023-12-27 04:06:59,198][105692] Updated weights for policy 0, policy_version 1746302 (0.0009) [2023-12-27 04:06:59,672][105620] Updated weights for policy 1, policy_version 1749911 (0.0008) [2023-12-27 04:06:59,719][105620] Updated weights for policy 1, policy_version 1749921 (0.0009) [2023-12-27 04:06:59,772][105620] Updated weights for policy 1, policy_version 1749931 (0.0007) [2023-12-27 04:07:00,060][105692] Updated weights for policy 0, policy_version 1746312 (0.0009) [2023-12-27 04:07:00,107][105692] Updated weights for policy 0, policy_version 1746322 (0.0009) [2023-12-27 04:07:00,157][105692] Updated weights for policy 0, policy_version 1746332 (0.0007) [2023-12-27 04:07:00,514][105620] Updated weights for policy 1, policy_version 1749941 (0.0007) [2023-12-27 04:07:00,565][105620] Updated weights for policy 1, policy_version 1749951 (0.0009) [2023-12-27 04:07:00,621][105620] Updated weights for policy 1, policy_version 1749961 (0.0010) [2023-12-27 04:07:00,870][105692] Updated weights for policy 0, policy_version 1746342 (0.0005) [2023-12-27 04:07:00,930][105692] Updated weights for policy 0, policy_version 1746352 (0.0007) [2023-12-27 04:07:00,983][105692] Updated weights for policy 0, policy_version 1746362 (0.0008) [2023-12-27 04:07:01,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 895188992. Throughput: 0: 9787.9, 1: 9641.2. Samples: 895155688. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:07:01,063][104569] Avg episode reward: [(0, '8991.197'), (1, '9076.557')] [2023-12-27 04:07:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001746368_447135744.pth... [2023-12-27 04:07:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001749968_448053248.pth... [2023-12-27 04:07:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001745248_446849024.pth [2023-12-27 04:07:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001748816_447758336.pth [2023-12-27 04:07:01,398][105620] Updated weights for policy 1, policy_version 1749971 (0.0009) [2023-12-27 04:07:01,460][105620] Updated weights for policy 1, policy_version 1749981 (0.0009) [2023-12-27 04:07:01,517][105620] Updated weights for policy 1, policy_version 1749991 (0.0009) [2023-12-27 04:07:01,702][105692] Updated weights for policy 0, policy_version 1746372 (0.0009) [2023-12-27 04:07:01,762][105692] Updated weights for policy 0, policy_version 1746382 (0.0007) [2023-12-27 04:07:01,813][105692] Updated weights for policy 0, policy_version 1746392 (0.0005) [2023-12-27 04:07:02,207][105620] Updated weights for policy 1, policy_version 1750001 (0.0009) [2023-12-27 04:07:02,269][105620] Updated weights for policy 1, policy_version 1750011 (0.0009) [2023-12-27 04:07:02,331][105620] Updated weights for policy 1, policy_version 1750021 (0.0009) [2023-12-27 04:07:02,395][105620] Updated weights for policy 1, policy_version 1750031 (0.0010) [2023-12-27 04:07:02,547][105692] Updated weights for policy 0, policy_version 1746402 (0.0008) [2023-12-27 04:07:02,617][105692] Updated weights for policy 0, policy_version 1746412 (0.0006) [2023-12-27 04:07:02,675][105692] Updated weights for policy 0, policy_version 1746422 (0.0009) [2023-12-27 04:07:02,744][105692] Updated weights for policy 0, policy_version 1746432 (0.0008) [2023-12-27 04:07:03,106][105620] Updated weights for policy 1, policy_version 1750041 (0.0009) [2023-12-27 04:07:03,164][105620] Updated weights for policy 1, policy_version 1750051 (0.0009) [2023-12-27 04:07:03,214][105620] Updated weights for policy 1, policy_version 1750061 (0.0009) [2023-12-27 04:07:03,464][105692] Updated weights for policy 0, policy_version 1746442 (0.0007) [2023-12-27 04:07:03,521][105692] Updated weights for policy 0, policy_version 1746452 (0.0005) [2023-12-27 04:07:03,571][105692] Updated weights for policy 0, policy_version 1746462 (0.0006) [2023-12-27 04:07:03,960][105620] Updated weights for policy 1, policy_version 1750071 (0.0009) [2023-12-27 04:07:04,022][105620] Updated weights for policy 1, policy_version 1750081 (0.0009) [2023-12-27 04:07:04,086][105620] Updated weights for policy 1, policy_version 1750091 (0.0009) [2023-12-27 04:07:04,225][105692] Updated weights for policy 0, policy_version 1746472 (0.0007) [2023-12-27 04:07:04,285][105692] Updated weights for policy 0, policy_version 1746482 (0.0011) [2023-12-27 04:07:04,352][105692] Updated weights for policy 0, policy_version 1746492 (0.0011) [2023-12-27 04:07:04,918][105620] Updated weights for policy 1, policy_version 1750101 (0.0009) [2023-12-27 04:07:04,981][105620] Updated weights for policy 1, policy_version 1750111 (0.0008) [2023-12-27 04:07:05,003][105692] Updated weights for policy 0, policy_version 1746502 (0.0010) [2023-12-27 04:07:05,040][105620] Updated weights for policy 1, policy_version 1750121 (0.0006) [2023-12-27 04:07:05,065][105692] Updated weights for policy 0, policy_version 1746512 (0.0010) [2023-12-27 04:07:05,121][105692] Updated weights for policy 0, policy_version 1746522 (0.0005) [2023-12-27 04:07:05,741][105620] Updated weights for policy 1, policy_version 1750131 (0.0007) [2023-12-27 04:07:05,805][105620] Updated weights for policy 1, policy_version 1750141 (0.0009) [2023-12-27 04:07:05,815][105692] Updated weights for policy 0, policy_version 1746532 (0.0007) [2023-12-27 04:07:05,866][105620] Updated weights for policy 1, policy_version 1750151 (0.0010) [2023-12-27 04:07:05,880][105692] Updated weights for policy 0, policy_version 1746542 (0.0010) [2023-12-27 04:07:05,938][105692] Updated weights for policy 0, policy_version 1746552 (0.0010) [2023-12-27 04:07:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 895287296. Throughput: 0: 9585.4, 1: 9746.4. Samples: 895270292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:07:06,063][104569] Avg episode reward: [(0, '8717.730'), (1, '9262.746')] [2023-12-27 04:07:06,545][105620] Updated weights for policy 1, policy_version 1750161 (0.0010) [2023-12-27 04:07:06,610][105620] Updated weights for policy 1, policy_version 1750171 (0.0010) [2023-12-27 04:07:06,631][105692] Updated weights for policy 0, policy_version 1746562 (0.0010) [2023-12-27 04:07:06,667][105620] Updated weights for policy 1, policy_version 1750181 (0.0011) [2023-12-27 04:07:06,694][105692] Updated weights for policy 0, policy_version 1746572 (0.0011) [2023-12-27 04:07:06,716][105620] Updated weights for policy 1, policy_version 1750191 (0.0010) [2023-12-27 04:07:06,753][105692] Updated weights for policy 0, policy_version 1746582 (0.0011) [2023-12-27 04:07:06,815][105692] Updated weights for policy 0, policy_version 1746592 (0.0010) [2023-12-27 04:07:07,415][105620] Updated weights for policy 1, policy_version 1750201 (0.0009) [2023-12-27 04:07:07,472][105620] Updated weights for policy 1, policy_version 1750211 (0.0009) [2023-12-27 04:07:07,494][105692] Updated weights for policy 0, policy_version 1746602 (0.0010) [2023-12-27 04:07:07,527][105620] Updated weights for policy 1, policy_version 1750221 (0.0010) [2023-12-27 04:07:07,550][105692] Updated weights for policy 0, policy_version 1746612 (0.0010) [2023-12-27 04:07:07,605][105692] Updated weights for policy 0, policy_version 1746622 (0.0010) [2023-12-27 04:07:08,171][105620] Updated weights for policy 1, policy_version 1750231 (0.0010) [2023-12-27 04:07:08,230][105620] Updated weights for policy 1, policy_version 1750241 (0.0011) [2023-12-27 04:07:08,289][105692] Updated weights for policy 0, policy_version 1746632 (0.0006) [2023-12-27 04:07:08,292][105620] Updated weights for policy 1, policy_version 1750251 (0.0010) [2023-12-27 04:07:08,355][105692] Updated weights for policy 0, policy_version 1746642 (0.0006) [2023-12-27 04:07:08,422][105692] Updated weights for policy 0, policy_version 1746652 (0.0008) [2023-12-27 04:07:09,052][105620] Updated weights for policy 1, policy_version 1750261 (0.0009) [2023-12-27 04:07:09,057][105692] Updated weights for policy 0, policy_version 1746662 (0.0007) [2023-12-27 04:07:09,097][105620] Updated weights for policy 1, policy_version 1750271 (0.0009) [2023-12-27 04:07:09,115][105692] Updated weights for policy 0, policy_version 1746672 (0.0009) [2023-12-27 04:07:09,153][105620] Updated weights for policy 1, policy_version 1750281 (0.0007) [2023-12-27 04:07:09,172][105692] Updated weights for policy 0, policy_version 1746682 (0.0006) [2023-12-27 04:07:09,857][105620] Updated weights for policy 1, policy_version 1750291 (0.0007) [2023-12-27 04:07:09,903][105692] Updated weights for policy 0, policy_version 1746692 (0.0007) [2023-12-27 04:07:09,914][105620] Updated weights for policy 1, policy_version 1750301 (0.0008) [2023-12-27 04:07:09,966][105692] Updated weights for policy 0, policy_version 1746702 (0.0007) [2023-12-27 04:07:09,976][105620] Updated weights for policy 1, policy_version 1750311 (0.0008) [2023-12-27 04:07:10,026][105692] Updated weights for policy 0, policy_version 1746712 (0.0007) [2023-12-27 04:07:10,703][105620] Updated weights for policy 1, policy_version 1750321 (0.0008) [2023-12-27 04:07:10,770][105620] Updated weights for policy 1, policy_version 1750331 (0.0008) [2023-12-27 04:07:10,813][105692] Updated weights for policy 0, policy_version 1746722 (0.0009) [2023-12-27 04:07:10,826][105620] Updated weights for policy 1, policy_version 1750341 (0.0008) [2023-12-27 04:07:10,879][105692] Updated weights for policy 0, policy_version 1746732 (0.0010) [2023-12-27 04:07:10,885][105620] Updated weights for policy 1, policy_version 1750351 (0.0006) [2023-12-27 04:07:10,930][105692] Updated weights for policy 0, policy_version 1746742 (0.0009) [2023-12-27 04:07:10,984][105692] Updated weights for policy 0, policy_version 1746752 (0.0010) [2023-12-27 04:07:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 895385600. Throughput: 0: 9611.5, 1: 9855.2. Samples: 895388664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:07:11,062][104569] Avg episode reward: [(0, '8625.345'), (1, '9170.150')] [2023-12-27 04:07:11,612][105620] Updated weights for policy 1, policy_version 1750361 (0.0006) [2023-12-27 04:07:11,685][105620] Updated weights for policy 1, policy_version 1750371 (0.0008) [2023-12-27 04:07:11,754][105620] Updated weights for policy 1, policy_version 1750381 (0.0008) [2023-12-27 04:07:11,786][105692] Updated weights for policy 0, policy_version 1746762 (0.0008) [2023-12-27 04:07:11,856][105692] Updated weights for policy 0, policy_version 1746772 (0.0008) [2023-12-27 04:07:11,919][105692] Updated weights for policy 0, policy_version 1746782 (0.0008) [2023-12-27 04:07:12,348][105620] Updated weights for policy 1, policy_version 1750391 (0.0009) [2023-12-27 04:07:12,410][105620] Updated weights for policy 1, policy_version 1750401 (0.0009) [2023-12-27 04:07:12,466][105620] Updated weights for policy 1, policy_version 1750411 (0.0008) [2023-12-27 04:07:12,672][105692] Updated weights for policy 0, policy_version 1746792 (0.0007) [2023-12-27 04:07:12,728][105692] Updated weights for policy 0, policy_version 1746802 (0.0006) [2023-12-27 04:07:12,790][105692] Updated weights for policy 0, policy_version 1746812 (0.0006) [2023-12-27 04:07:13,162][105620] Updated weights for policy 1, policy_version 1750421 (0.0007) [2023-12-27 04:07:13,225][105620] Updated weights for policy 1, policy_version 1750431 (0.0005) [2023-12-27 04:07:13,288][105620] Updated weights for policy 1, policy_version 1750441 (0.0005) [2023-12-27 04:07:13,333][105692] Updated weights for policy 0, policy_version 1746822 (0.0007) [2023-12-27 04:07:13,399][105692] Updated weights for policy 0, policy_version 1746832 (0.0006) [2023-12-27 04:07:13,460][105692] Updated weights for policy 0, policy_version 1746842 (0.0006) [2023-12-27 04:07:13,954][105620] Updated weights for policy 1, policy_version 1750451 (0.0009) [2023-12-27 04:07:14,004][105620] Updated weights for policy 1, policy_version 1750461 (0.0010) [2023-12-27 04:07:14,062][105620] Updated weights for policy 1, policy_version 1750471 (0.0010) [2023-12-27 04:07:14,099][105692] Updated weights for policy 0, policy_version 1746852 (0.0008) [2023-12-27 04:07:14,154][105692] Updated weights for policy 0, policy_version 1746862 (0.0010) [2023-12-27 04:07:14,210][105692] Updated weights for policy 0, policy_version 1746872 (0.0005) [2023-12-27 04:07:14,769][105692] Updated weights for policy 0, policy_version 1746882 (0.0006) [2023-12-27 04:07:14,832][105692] Updated weights for policy 0, policy_version 1746892 (0.0011) [2023-12-27 04:07:14,862][105620] Updated weights for policy 1, policy_version 1750481 (0.0010) [2023-12-27 04:07:14,892][105692] Updated weights for policy 0, policy_version 1746902 (0.0008) [2023-12-27 04:07:14,926][105620] Updated weights for policy 1, policy_version 1750491 (0.0008) [2023-12-27 04:07:14,950][105692] Updated weights for policy 0, policy_version 1746912 (0.0010) [2023-12-27 04:07:14,985][105620] Updated weights for policy 1, policy_version 1750501 (0.0007) [2023-12-27 04:07:15,045][105620] Updated weights for policy 1, policy_version 1750511 (0.0008) [2023-12-27 04:07:15,645][105692] Updated weights for policy 0, policy_version 1746922 (0.0011) [2023-12-27 04:07:15,698][105692] Updated weights for policy 0, policy_version 1746932 (0.0011) [2023-12-27 04:07:15,753][105692] Updated weights for policy 0, policy_version 1746942 (0.0010) [2023-12-27 04:07:15,816][105620] Updated weights for policy 1, policy_version 1750521 (0.0011) [2023-12-27 04:07:15,865][105620] Updated weights for policy 1, policy_version 1750531 (0.0010) [2023-12-27 04:07:15,913][105620] Updated weights for policy 1, policy_version 1750541 (0.0010) [2023-12-27 04:07:16,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 895483904. Throughput: 0: 9622.0, 1: 9836.9. Samples: 895448012. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:07:16,063][104569] Avg episode reward: [(0, '8808.329'), (1, '9261.822')] [2023-12-27 04:07:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001746944_447283200.pth... [2023-12-27 04:07:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001750544_448200704.pth... [2023-12-27 04:07:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001745792_446988288.pth [2023-12-27 04:07:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001749392_447905792.pth [2023-12-27 04:07:16,521][105692] Updated weights for policy 0, policy_version 1746952 (0.0010) [2023-12-27 04:07:16,552][105620] Updated weights for policy 1, policy_version 1750551 (0.0010) [2023-12-27 04:07:16,576][105692] Updated weights for policy 0, policy_version 1746962 (0.0010) [2023-12-27 04:07:16,614][105620] Updated weights for policy 1, policy_version 1750561 (0.0005) [2023-12-27 04:07:16,628][105692] Updated weights for policy 0, policy_version 1746972 (0.0011) [2023-12-27 04:07:16,675][105620] Updated weights for policy 1, policy_version 1750571 (0.0005) [2023-12-27 04:07:17,198][105620] Updated weights for policy 1, policy_version 1750581 (0.0006) [2023-12-27 04:07:17,255][105620] Updated weights for policy 1, policy_version 1750591 (0.0005) [2023-12-27 04:07:17,297][105692] Updated weights for policy 0, policy_version 1746982 (0.0007) [2023-12-27 04:07:17,321][105620] Updated weights for policy 1, policy_version 1750601 (0.0005) [2023-12-27 04:07:17,343][105692] Updated weights for policy 0, policy_version 1746992 (0.0005) [2023-12-27 04:07:17,390][105692] Updated weights for policy 0, policy_version 1747002 (0.0005) [2023-12-27 04:07:17,884][105620] Updated weights for policy 1, policy_version 1750611 (0.0007) [2023-12-27 04:07:17,942][105620] Updated weights for policy 1, policy_version 1750621 (0.0009) [2023-12-27 04:07:17,994][105620] Updated weights for policy 1, policy_version 1750631 (0.0008) [2023-12-27 04:07:18,109][105692] Updated weights for policy 0, policy_version 1747012 (0.0008) [2023-12-27 04:07:18,154][105692] Updated weights for policy 0, policy_version 1747022 (0.0011) [2023-12-27 04:07:18,199][105692] Updated weights for policy 0, policy_version 1747032 (0.0010) [2023-12-27 04:07:18,796][105620] Updated weights for policy 1, policy_version 1750641 (0.0008) [2023-12-27 04:07:18,858][105620] Updated weights for policy 1, policy_version 1750651 (0.0005) [2023-12-27 04:07:18,909][105620] Updated weights for policy 1, policy_version 1750661 (0.0005) [2023-12-27 04:07:18,960][105620] Updated weights for policy 1, policy_version 1750671 (0.0005) [2023-12-27 04:07:18,963][105692] Updated weights for policy 0, policy_version 1747042 (0.0011) [2023-12-27 04:07:19,010][105692] Updated weights for policy 0, policy_version 1747052 (0.0010) [2023-12-27 04:07:19,064][105692] Updated weights for policy 0, policy_version 1747062 (0.0010) [2023-12-27 04:07:19,112][105692] Updated weights for policy 0, policy_version 1747072 (0.0010) [2023-12-27 04:07:19,702][105620] Updated weights for policy 1, policy_version 1750681 (0.0008) [2023-12-27 04:07:19,759][105620] Updated weights for policy 1, policy_version 1750691 (0.0009) [2023-12-27 04:07:19,820][105620] Updated weights for policy 1, policy_version 1750701 (0.0009) [2023-12-27 04:07:19,821][105692] Updated weights for policy 0, policy_version 1747082 (0.0006) [2023-12-27 04:07:19,881][105692] Updated weights for policy 0, policy_version 1747092 (0.0009) [2023-12-27 04:07:19,955][105692] Updated weights for policy 0, policy_version 1747102 (0.0010) [2023-12-27 04:07:20,535][105620] Updated weights for policy 1, policy_version 1750711 (0.0009) [2023-12-27 04:07:20,608][105620] Updated weights for policy 1, policy_version 1750721 (0.0008) [2023-12-27 04:07:20,679][105620] Updated weights for policy 1, policy_version 1750731 (0.0006) [2023-12-27 04:07:20,726][105692] Updated weights for policy 0, policy_version 1747112 (0.0009) [2023-12-27 04:07:20,789][105692] Updated weights for policy 0, policy_version 1747122 (0.0010) [2023-12-27 04:07:20,854][105692] Updated weights for policy 0, policy_version 1747132 (0.0009) [2023-12-27 04:07:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 895582208. Throughput: 0: 9728.3, 1: 9840.2. Samples: 895568828. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:07:21,062][104569] Avg episode reward: [(0, '8804.063'), (1, '8893.750')] [2023-12-27 04:07:21,358][105620] Updated weights for policy 1, policy_version 1750741 (0.0008) [2023-12-27 04:07:21,421][105620] Updated weights for policy 1, policy_version 1750751 (0.0007) [2023-12-27 04:07:21,485][105620] Updated weights for policy 1, policy_version 1750761 (0.0007) [2023-12-27 04:07:21,628][105692] Updated weights for policy 0, policy_version 1747142 (0.0009) [2023-12-27 04:07:21,694][105692] Updated weights for policy 0, policy_version 1747152 (0.0007) [2023-12-27 04:07:21,765][105692] Updated weights for policy 0, policy_version 1747162 (0.0008) [2023-12-27 04:07:22,163][105620] Updated weights for policy 1, policy_version 1750771 (0.0009) [2023-12-27 04:07:22,221][105620] Updated weights for policy 1, policy_version 1750781 (0.0009) [2023-12-27 04:07:22,289][105620] Updated weights for policy 1, policy_version 1750791 (0.0008) [2023-12-27 04:07:22,394][105692] Updated weights for policy 0, policy_version 1747172 (0.0008) [2023-12-27 04:07:22,458][105692] Updated weights for policy 0, policy_version 1747182 (0.0008) [2023-12-27 04:07:22,518][105692] Updated weights for policy 0, policy_version 1747192 (0.0008) [2023-12-27 04:07:23,012][105620] Updated weights for policy 1, policy_version 1750801 (0.0008) [2023-12-27 04:07:23,061][105620] Updated weights for policy 1, policy_version 1750811 (0.0009) [2023-12-27 04:07:23,108][105620] Updated weights for policy 1, policy_version 1750821 (0.0006) [2023-12-27 04:07:23,175][105620] Updated weights for policy 1, policy_version 1750831 (0.0006) [2023-12-27 04:07:23,320][105692] Updated weights for policy 0, policy_version 1747202 (0.0009) [2023-12-27 04:07:23,369][105692] Updated weights for policy 0, policy_version 1747212 (0.0010) [2023-12-27 04:07:23,419][105692] Updated weights for policy 0, policy_version 1747222 (0.0007) [2023-12-27 04:07:23,467][105692] Updated weights for policy 0, policy_version 1747232 (0.0007) [2023-12-27 04:07:23,776][105620] Updated weights for policy 1, policy_version 1750841 (0.0010) [2023-12-27 04:07:23,820][105620] Updated weights for policy 1, policy_version 1750851 (0.0010) [2023-12-27 04:07:23,869][105620] Updated weights for policy 1, policy_version 1750861 (0.0010) [2023-12-27 04:07:24,217][105692] Updated weights for policy 0, policy_version 1747242 (0.0011) [2023-12-27 04:07:24,276][105692] Updated weights for policy 0, policy_version 1747252 (0.0010) [2023-12-27 04:07:24,337][105692] Updated weights for policy 0, policy_version 1747262 (0.0005) [2023-12-27 04:07:24,616][105620] Updated weights for policy 1, policy_version 1750871 (0.0009) [2023-12-27 04:07:24,662][105620] Updated weights for policy 1, policy_version 1750881 (0.0007) [2023-12-27 04:07:24,714][105620] Updated weights for policy 1, policy_version 1750891 (0.0008) [2023-12-27 04:07:24,975][105692] Updated weights for policy 0, policy_version 1747272 (0.0009) [2023-12-27 04:07:25,035][105692] Updated weights for policy 0, policy_version 1747282 (0.0011) [2023-12-27 04:07:25,093][105692] Updated weights for policy 0, policy_version 1747292 (0.0011) [2023-12-27 04:07:25,390][105620] Updated weights for policy 1, policy_version 1750901 (0.0009) [2023-12-27 04:07:25,445][105620] Updated weights for policy 1, policy_version 1750911 (0.0010) [2023-12-27 04:07:25,495][105620] Updated weights for policy 1, policy_version 1750921 (0.0010) [2023-12-27 04:07:25,834][105692] Updated weights for policy 0, policy_version 1747302 (0.0010) [2023-12-27 04:07:25,886][105692] Updated weights for policy 0, policy_version 1747312 (0.0011) [2023-12-27 04:07:25,934][105692] Updated weights for policy 0, policy_version 1747322 (0.0010) [2023-12-27 04:07:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 895680512. Throughput: 0: 9748.4, 1: 9895.4. Samples: 895686524. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:07:26,063][104569] Avg episode reward: [(0, '8529.382'), (1, '8713.355')] [2023-12-27 04:07:26,218][105620] Updated weights for policy 1, policy_version 1750931 (0.0009) [2023-12-27 04:07:26,279][105620] Updated weights for policy 1, policy_version 1750941 (0.0005) [2023-12-27 04:07:26,333][105620] Updated weights for policy 1, policy_version 1750951 (0.0006) [2023-12-27 04:07:26,715][105692] Updated weights for policy 0, policy_version 1747332 (0.0010) [2023-12-27 04:07:26,773][105692] Updated weights for policy 0, policy_version 1747342 (0.0010) [2023-12-27 04:07:26,836][105692] Updated weights for policy 0, policy_version 1747352 (0.0011) [2023-12-27 04:07:26,962][105620] Updated weights for policy 1, policy_version 1750961 (0.0011) [2023-12-27 04:07:27,017][105620] Updated weights for policy 1, policy_version 1750971 (0.0008) [2023-12-27 04:07:27,066][105620] Updated weights for policy 1, policy_version 1750981 (0.0008) [2023-12-27 04:07:27,125][105620] Updated weights for policy 1, policy_version 1750991 (0.0008) [2023-12-27 04:07:27,497][105692] Updated weights for policy 0, policy_version 1747362 (0.0010) [2023-12-27 04:07:27,548][105692] Updated weights for policy 0, policy_version 1747372 (0.0010) [2023-12-27 04:07:27,599][105692] Updated weights for policy 0, policy_version 1747382 (0.0010) [2023-12-27 04:07:27,649][105692] Updated weights for policy 0, policy_version 1747392 (0.0010) [2023-12-27 04:07:27,787][105620] Updated weights for policy 1, policy_version 1751001 (0.0009) [2023-12-27 04:07:27,834][105620] Updated weights for policy 1, policy_version 1751011 (0.0009) [2023-12-27 04:07:27,892][105620] Updated weights for policy 1, policy_version 1751021 (0.0008) [2023-12-27 04:07:28,365][105692] Updated weights for policy 0, policy_version 1747402 (0.0010) [2023-12-27 04:07:28,411][105692] Updated weights for policy 0, policy_version 1747412 (0.0010) [2023-12-27 04:07:28,472][105692] Updated weights for policy 0, policy_version 1747422 (0.0010) [2023-12-27 04:07:28,541][105620] Updated weights for policy 1, policy_version 1751031 (0.0008) [2023-12-27 04:07:28,600][105620] Updated weights for policy 1, policy_version 1751041 (0.0008) [2023-12-27 04:07:28,659][105620] Updated weights for policy 1, policy_version 1751051 (0.0008) [2023-12-27 04:07:29,163][105692] Updated weights for policy 0, policy_version 1747432 (0.0007) [2023-12-27 04:07:29,224][105692] Updated weights for policy 0, policy_version 1747442 (0.0006) [2023-12-27 04:07:29,281][105692] Updated weights for policy 0, policy_version 1747452 (0.0009) [2023-12-27 04:07:29,302][105620] Updated weights for policy 1, policy_version 1751061 (0.0008) [2023-12-27 04:07:29,361][105620] Updated weights for policy 1, policy_version 1751071 (0.0009) [2023-12-27 04:07:29,424][105620] Updated weights for policy 1, policy_version 1751081 (0.0008) [2023-12-27 04:07:29,890][105692] Updated weights for policy 0, policy_version 1747462 (0.0006) [2023-12-27 04:07:29,959][105692] Updated weights for policy 0, policy_version 1747472 (0.0009) [2023-12-27 04:07:30,013][105692] Updated weights for policy 0, policy_version 1747482 (0.0008) [2023-12-27 04:07:30,092][105620] Updated weights for policy 1, policy_version 1751091 (0.0008) [2023-12-27 04:07:30,143][105620] Updated weights for policy 1, policy_version 1751101 (0.0008) [2023-12-27 04:07:30,191][105620] Updated weights for policy 1, policy_version 1751111 (0.0007) [2023-12-27 04:07:30,742][105692] Updated weights for policy 0, policy_version 1747492 (0.0009) [2023-12-27 04:07:30,795][105692] Updated weights for policy 0, policy_version 1747502 (0.0010) [2023-12-27 04:07:30,839][105692] Updated weights for policy 0, policy_version 1747512 (0.0005) [2023-12-27 04:07:30,962][105620] Updated weights for policy 1, policy_version 1751121 (0.0008) [2023-12-27 04:07:31,034][105620] Updated weights for policy 1, policy_version 1751131 (0.0007) [2023-12-27 04:07:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 895778816. Throughput: 0: 9802.0, 1: 9954.1. Samples: 895746772. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:07:31,062][104569] Avg episode reward: [(0, '8533.089'), (1, '8897.120')] [2023-12-27 04:07:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001747520_447430656.pth... [2023-12-27 04:07:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001746368_447135744.pth [2023-12-27 04:07:31,097][105620] Updated weights for policy 1, policy_version 1751141 (0.0007) [2023-12-27 04:07:31,165][105620] Updated weights for policy 1, policy_version 1751151 (0.0009) [2023-12-27 04:07:31,169][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001751152_448356352.pth... [2023-12-27 04:07:31,175][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001749968_448053248.pth [2023-12-27 04:07:31,537][105692] Updated weights for policy 0, policy_version 1747522 (0.0006) [2023-12-27 04:07:31,593][105692] Updated weights for policy 0, policy_version 1747532 (0.0009) [2023-12-27 04:07:31,653][105692] Updated weights for policy 0, policy_version 1747542 (0.0010) [2023-12-27 04:07:31,717][105692] Updated weights for policy 0, policy_version 1747552 (0.0010) [2023-12-27 04:07:31,831][105620] Updated weights for policy 1, policy_version 1751161 (0.0011) [2023-12-27 04:07:31,891][105620] Updated weights for policy 1, policy_version 1751171 (0.0011) [2023-12-27 04:07:31,954][105620] Updated weights for policy 1, policy_version 1751181 (0.0011) [2023-12-27 04:07:32,460][105692] Updated weights for policy 0, policy_version 1747562 (0.0010) [2023-12-27 04:07:32,524][105692] Updated weights for policy 0, policy_version 1747572 (0.0010) [2023-12-27 04:07:32,573][105692] Updated weights for policy 0, policy_version 1747582 (0.0010) [2023-12-27 04:07:32,705][105620] Updated weights for policy 1, policy_version 1751191 (0.0011) [2023-12-27 04:07:32,766][105620] Updated weights for policy 1, policy_version 1751201 (0.0010) [2023-12-27 04:07:32,818][105620] Updated weights for policy 1, policy_version 1751211 (0.0010) [2023-12-27 04:07:33,282][105692] Updated weights for policy 0, policy_version 1747592 (0.0010) [2023-12-27 04:07:33,327][105692] Updated weights for policy 0, policy_version 1747602 (0.0010) [2023-12-27 04:07:33,374][105692] Updated weights for policy 0, policy_version 1747612 (0.0010) [2023-12-27 04:07:33,469][105620] Updated weights for policy 1, policy_version 1751221 (0.0010) [2023-12-27 04:07:33,513][105620] Updated weights for policy 1, policy_version 1751231 (0.0010) [2023-12-27 04:07:33,571][105620] Updated weights for policy 1, policy_version 1751241 (0.0010) [2023-12-27 04:07:34,034][105692] Updated weights for policy 0, policy_version 1747622 (0.0007) [2023-12-27 04:07:34,086][105692] Updated weights for policy 0, policy_version 1747632 (0.0005) [2023-12-27 04:07:34,144][105620] Updated weights for policy 1, policy_version 1751251 (0.0009) [2023-12-27 04:07:34,151][105692] Updated weights for policy 0, policy_version 1747642 (0.0007) [2023-12-27 04:07:34,209][105620] Updated weights for policy 1, policy_version 1751261 (0.0007) [2023-12-27 04:07:34,268][105620] Updated weights for policy 1, policy_version 1751271 (0.0006) [2023-12-27 04:07:34,860][105692] Updated weights for policy 0, policy_version 1747652 (0.0010) [2023-12-27 04:07:34,916][105692] Updated weights for policy 0, policy_version 1747662 (0.0009) [2023-12-27 04:07:34,926][105620] Updated weights for policy 1, policy_version 1751281 (0.0008) [2023-12-27 04:07:34,974][105692] Updated weights for policy 0, policy_version 1747672 (0.0010) [2023-12-27 04:07:34,978][105620] Updated weights for policy 1, policy_version 1751291 (0.0010) [2023-12-27 04:07:35,034][105620] Updated weights for policy 1, policy_version 1751301 (0.0011) [2023-12-27 04:07:35,084][105620] Updated weights for policy 1, policy_version 1751311 (0.0007) [2023-12-27 04:07:35,595][105692] Updated weights for policy 0, policy_version 1747682 (0.0009) [2023-12-27 04:07:35,660][105692] Updated weights for policy 0, policy_version 1747692 (0.0010) [2023-12-27 04:07:35,716][105692] Updated weights for policy 0, policy_version 1747702 (0.0010) [2023-12-27 04:07:35,730][105620] Updated weights for policy 1, policy_version 1751321 (0.0006) [2023-12-27 04:07:35,773][105692] Updated weights for policy 0, policy_version 1747712 (0.0009) [2023-12-27 04:07:35,779][105620] Updated weights for policy 1, policy_version 1751331 (0.0006) [2023-12-27 04:07:35,827][105620] Updated weights for policy 1, policy_version 1751341 (0.0005) [2023-12-27 04:07:36,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 895885312. Throughput: 0: 9858.1, 1: 9935.9. Samples: 895868832. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:07:36,063][104569] Avg episode reward: [(0, '8078.283'), (1, '9262.928')] [2023-12-27 04:07:36,458][105692] Updated weights for policy 0, policy_version 1747722 (0.0006) [2023-12-27 04:07:36,477][105620] Updated weights for policy 1, policy_version 1751351 (0.0009) [2023-12-27 04:07:36,518][105692] Updated weights for policy 0, policy_version 1747732 (0.0008) [2023-12-27 04:07:36,534][105620] Updated weights for policy 1, policy_version 1751361 (0.0010) [2023-12-27 04:07:36,570][105692] Updated weights for policy 0, policy_version 1747742 (0.0010) [2023-12-27 04:07:36,593][105620] Updated weights for policy 1, policy_version 1751371 (0.0011) [2023-12-27 04:07:37,176][105692] Updated weights for policy 0, policy_version 1747752 (0.0010) [2023-12-27 04:07:37,241][105692] Updated weights for policy 0, policy_version 1747762 (0.0010) [2023-12-27 04:07:37,298][105620] Updated weights for policy 1, policy_version 1751381 (0.0011) [2023-12-27 04:07:37,304][105692] Updated weights for policy 0, policy_version 1747772 (0.0010) [2023-12-27 04:07:37,350][105620] Updated weights for policy 1, policy_version 1751391 (0.0010) [2023-12-27 04:07:37,402][105620] Updated weights for policy 1, policy_version 1751401 (0.0010) [2023-12-27 04:07:38,043][105692] Updated weights for policy 0, policy_version 1747782 (0.0010) [2023-12-27 04:07:38,081][105620] Updated weights for policy 1, policy_version 1751411 (0.0009) [2023-12-27 04:07:38,099][105692] Updated weights for policy 0, policy_version 1747792 (0.0010) [2023-12-27 04:07:38,136][105620] Updated weights for policy 1, policy_version 1751421 (0.0005) [2023-12-27 04:07:38,154][105692] Updated weights for policy 0, policy_version 1747802 (0.0011) [2023-12-27 04:07:38,204][105620] Updated weights for policy 1, policy_version 1751431 (0.0006) [2023-12-27 04:07:38,905][105692] Updated weights for policy 0, policy_version 1747812 (0.0011) [2023-12-27 04:07:38,963][105692] Updated weights for policy 0, policy_version 1747822 (0.0011) [2023-12-27 04:07:38,971][105620] Updated weights for policy 1, policy_version 1751441 (0.0008) [2023-12-27 04:07:39,021][105620] Updated weights for policy 1, policy_version 1751451 (0.0007) [2023-12-27 04:07:39,023][105692] Updated weights for policy 0, policy_version 1747832 (0.0011) [2023-12-27 04:07:39,070][105620] Updated weights for policy 1, policy_version 1751461 (0.0005) [2023-12-27 04:07:39,122][105620] Updated weights for policy 1, policy_version 1751471 (0.0008) [2023-12-27 04:07:39,769][105692] Updated weights for policy 0, policy_version 1747842 (0.0011) [2023-12-27 04:07:39,836][105692] Updated weights for policy 0, policy_version 1747852 (0.0011) [2023-12-27 04:07:39,904][105692] Updated weights for policy 0, policy_version 1747863 (0.0009) [2023-12-27 04:07:39,940][105620] Updated weights for policy 1, policy_version 1751481 (0.0007) [2023-12-27 04:07:40,007][105620] Updated weights for policy 1, policy_version 1751491 (0.0008) [2023-12-27 04:07:40,081][105620] Updated weights for policy 1, policy_version 1751501 (0.0006) [2023-12-27 04:07:40,663][105692] Updated weights for policy 0, policy_version 1747873 (0.0010) [2023-12-27 04:07:40,726][105692] Updated weights for policy 0, policy_version 1747883 (0.0010) [2023-12-27 04:07:40,741][105620] Updated weights for policy 1, policy_version 1751511 (0.0006) [2023-12-27 04:07:40,786][105692] Updated weights for policy 0, policy_version 1747893 (0.0011) [2023-12-27 04:07:40,792][105620] Updated weights for policy 1, policy_version 1751521 (0.0006) [2023-12-27 04:07:40,837][105620] Updated weights for policy 1, policy_version 1751531 (0.0006) [2023-12-27 04:07:40,838][105692] Updated weights for policy 0, policy_version 1747903 (0.0010) [2023-12-27 04:07:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 895983616. Throughput: 0: 9784.4, 1: 10010.8. Samples: 895987188. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:07:41,062][104569] Avg episode reward: [(0, '7982.857'), (1, '9263.030')] [2023-12-27 04:07:41,598][105692] Updated weights for policy 0, policy_version 1747913 (0.0009) [2023-12-27 04:07:41,614][105620] Updated weights for policy 1, policy_version 1751541 (0.0008) [2023-12-27 04:07:41,664][105692] Updated weights for policy 0, policy_version 1747923 (0.0008) [2023-12-27 04:07:41,678][105620] Updated weights for policy 1, policy_version 1751551 (0.0009) [2023-12-27 04:07:41,729][105692] Updated weights for policy 0, policy_version 1747933 (0.0010) [2023-12-27 04:07:41,744][105620] Updated weights for policy 1, policy_version 1751561 (0.0010) [2023-12-27 04:07:42,377][105620] Updated weights for policy 1, policy_version 1751571 (0.0008) [2023-12-27 04:07:42,445][105620] Updated weights for policy 1, policy_version 1751581 (0.0007) [2023-12-27 04:07:42,515][105620] Updated weights for policy 1, policy_version 1751591 (0.0007) [2023-12-27 04:07:42,537][105692] Updated weights for policy 0, policy_version 1747943 (0.0010) [2023-12-27 04:07:42,596][105692] Updated weights for policy 0, policy_version 1747953 (0.0008) [2023-12-27 04:07:42,661][105692] Updated weights for policy 0, policy_version 1747963 (0.0009) [2023-12-27 04:07:43,079][105620] Updated weights for policy 1, policy_version 1751601 (0.0007) [2023-12-27 04:07:43,135][105620] Updated weights for policy 1, policy_version 1751611 (0.0006) [2023-12-27 04:07:43,203][105620] Updated weights for policy 1, policy_version 1751621 (0.0006) [2023-12-27 04:07:43,256][105620] Updated weights for policy 1, policy_version 1751631 (0.0009) [2023-12-27 04:07:43,407][105692] Updated weights for policy 0, policy_version 1747973 (0.0009) [2023-12-27 04:07:43,472][105692] Updated weights for policy 0, policy_version 1747983 (0.0008) [2023-12-27 04:07:43,536][105692] Updated weights for policy 0, policy_version 1747993 (0.0006) [2023-12-27 04:07:43,945][105620] Updated weights for policy 1, policy_version 1751641 (0.0009) [2023-12-27 04:07:43,994][105620] Updated weights for policy 1, policy_version 1751651 (0.0008) [2023-12-27 04:07:44,054][105620] Updated weights for policy 1, policy_version 1751661 (0.0008) [2023-12-27 04:07:44,230][105692] Updated weights for policy 0, policy_version 1748003 (0.0007) [2023-12-27 04:07:44,288][105692] Updated weights for policy 0, policy_version 1748013 (0.0010) [2023-12-27 04:07:44,342][105692] Updated weights for policy 0, policy_version 1748023 (0.0010) [2023-12-27 04:07:44,813][105620] Updated weights for policy 1, policy_version 1751671 (0.0009) [2023-12-27 04:07:44,870][105620] Updated weights for policy 1, policy_version 1751681 (0.0008) [2023-12-27 04:07:44,932][105620] Updated weights for policy 1, policy_version 1751691 (0.0006) [2023-12-27 04:07:45,095][105692] Updated weights for policy 0, policy_version 1748033 (0.0011) [2023-12-27 04:07:45,161][105692] Updated weights for policy 0, policy_version 1748043 (0.0009) [2023-12-27 04:07:45,210][105692] Updated weights for policy 0, policy_version 1748053 (0.0011) [2023-12-27 04:07:45,256][105692] Updated weights for policy 0, policy_version 1748063 (0.0010) [2023-12-27 04:07:45,494][105620] Updated weights for policy 1, policy_version 1751701 (0.0006) [2023-12-27 04:07:45,550][105620] Updated weights for policy 1, policy_version 1751711 (0.0008) [2023-12-27 04:07:45,614][105620] Updated weights for policy 1, policy_version 1751721 (0.0008) [2023-12-27 04:07:45,946][105692] Updated weights for policy 0, policy_version 1748073 (0.0006) [2023-12-27 04:07:45,997][105692] Updated weights for policy 0, policy_version 1748083 (0.0005) [2023-12-27 04:07:46,055][105692] Updated weights for policy 0, policy_version 1748093 (0.0005) [2023-12-27 04:07:46,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19660.8, 300 sec: 19605.2). Total num frames: 896073728. Throughput: 0: 9787.3, 1: 10007.9. Samples: 896046476. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:07:46,063][104569] Avg episode reward: [(0, '8713.081'), (1, '8801.767')] [2023-12-27 04:07:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001751728_448503808.pth... [2023-12-27 04:07:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001748096_447578112.pth... [2023-12-27 04:07:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001750544_448200704.pth [2023-12-27 04:07:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001746944_447283200.pth [2023-12-27 04:07:46,465][105620] Updated weights for policy 1, policy_version 1751731 (0.0009) [2023-12-27 04:07:46,521][105620] Updated weights for policy 1, policy_version 1751741 (0.0008) [2023-12-27 04:07:46,585][105620] Updated weights for policy 1, policy_version 1751751 (0.0009) [2023-12-27 04:07:46,621][105692] Updated weights for policy 0, policy_version 1748103 (0.0007) [2023-12-27 04:07:46,678][105692] Updated weights for policy 0, policy_version 1748113 (0.0008) [2023-12-27 04:07:46,742][105692] Updated weights for policy 0, policy_version 1748123 (0.0005) [2023-12-27 04:07:47,374][105620] Updated weights for policy 1, policy_version 1751761 (0.0008) [2023-12-27 04:07:47,412][105692] Updated weights for policy 0, policy_version 1748133 (0.0007) [2023-12-27 04:07:47,436][105620] Updated weights for policy 1, policy_version 1751771 (0.0008) [2023-12-27 04:07:47,472][105692] Updated weights for policy 0, policy_version 1748143 (0.0009) [2023-12-27 04:07:47,500][105620] Updated weights for policy 1, policy_version 1751781 (0.0007) [2023-12-27 04:07:47,530][105692] Updated weights for policy 0, policy_version 1748153 (0.0009) [2023-12-27 04:07:47,558][105620] Updated weights for policy 1, policy_version 1751791 (0.0007) [2023-12-27 04:07:48,253][105692] Updated weights for policy 0, policy_version 1748163 (0.0007) [2023-12-27 04:07:48,290][105620] Updated weights for policy 1, policy_version 1751801 (0.0006) [2023-12-27 04:07:48,311][105692] Updated weights for policy 0, policy_version 1748173 (0.0011) [2023-12-27 04:07:48,354][105620] Updated weights for policy 1, policy_version 1751811 (0.0007) [2023-12-27 04:07:48,376][105692] Updated weights for policy 0, policy_version 1748183 (0.0009) [2023-12-27 04:07:48,416][105620] Updated weights for policy 1, policy_version 1751821 (0.0008) [2023-12-27 04:07:48,919][105692] Updated weights for policy 0, policy_version 1748193 (0.0005) [2023-12-27 04:07:48,981][105692] Updated weights for policy 0, policy_version 1748203 (0.0005) [2023-12-27 04:07:49,042][105692] Updated weights for policy 0, policy_version 1748213 (0.0008) [2023-12-27 04:07:49,098][105692] Updated weights for policy 0, policy_version 1748223 (0.0010) [2023-12-27 04:07:49,244][105620] Updated weights for policy 1, policy_version 1751831 (0.0009) [2023-12-27 04:07:49,297][105620] Updated weights for policy 1, policy_version 1751841 (0.0008) [2023-12-27 04:07:49,355][105620] Updated weights for policy 1, policy_version 1751851 (0.0008) [2023-12-27 04:07:49,760][105692] Updated weights for policy 0, policy_version 1748233 (0.0005) [2023-12-27 04:07:49,822][105692] Updated weights for policy 0, policy_version 1748243 (0.0006) [2023-12-27 04:07:49,884][105692] Updated weights for policy 0, policy_version 1748253 (0.0011) [2023-12-27 04:07:50,159][105620] Updated weights for policy 1, policy_version 1751861 (0.0008) [2023-12-27 04:07:50,218][105620] Updated weights for policy 1, policy_version 1751871 (0.0008) [2023-12-27 04:07:50,269][105620] Updated weights for policy 1, policy_version 1751881 (0.0009) [2023-12-27 04:07:50,567][105692] Updated weights for policy 0, policy_version 1748263 (0.0010) [2023-12-27 04:07:50,628][105692] Updated weights for policy 0, policy_version 1748273 (0.0010) [2023-12-27 04:07:50,694][105692] Updated weights for policy 0, policy_version 1748283 (0.0011) [2023-12-27 04:07:51,024][105620] Updated weights for policy 1, policy_version 1751891 (0.0008) [2023-12-27 04:07:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 896172032. Throughput: 0: 9894.8, 1: 9949.2. Samples: 896163268. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:07:51,062][104569] Avg episode reward: [(0, '9078.984'), (1, '8894.180')] [2023-12-27 04:07:51,084][105620] Updated weights for policy 1, policy_version 1751901 (0.0009) [2023-12-27 04:07:51,145][105620] Updated weights for policy 1, policy_version 1751911 (0.0009) [2023-12-27 04:07:51,431][105692] Updated weights for policy 0, policy_version 1748293 (0.0009) [2023-12-27 04:07:51,486][105692] Updated weights for policy 0, policy_version 1748303 (0.0008) [2023-12-27 04:07:51,541][105692] Updated weights for policy 0, policy_version 1748313 (0.0008) [2023-12-27 04:07:51,929][105620] Updated weights for policy 1, policy_version 1751921 (0.0009) [2023-12-27 04:07:51,992][105620] Updated weights for policy 1, policy_version 1751931 (0.0009) [2023-12-27 04:07:52,050][105620] Updated weights for policy 1, policy_version 1751941 (0.0006) [2023-12-27 04:07:52,098][105620] Updated weights for policy 1, policy_version 1751951 (0.0005) [2023-12-27 04:07:52,281][105692] Updated weights for policy 0, policy_version 1748323 (0.0009) [2023-12-27 04:07:52,327][105692] Updated weights for policy 0, policy_version 1748333 (0.0009) [2023-12-27 04:07:52,392][105692] Updated weights for policy 0, policy_version 1748343 (0.0008) [2023-12-27 04:07:52,791][105620] Updated weights for policy 1, policy_version 1751961 (0.0008) [2023-12-27 04:07:52,850][105620] Updated weights for policy 1, policy_version 1751971 (0.0009) [2023-12-27 04:07:52,908][105620] Updated weights for policy 1, policy_version 1751981 (0.0007) [2023-12-27 04:07:53,129][105692] Updated weights for policy 0, policy_version 1748353 (0.0009) [2023-12-27 04:07:53,190][105692] Updated weights for policy 0, policy_version 1748363 (0.0009) [2023-12-27 04:07:53,245][105692] Updated weights for policy 0, policy_version 1748373 (0.0010) [2023-12-27 04:07:53,303][105692] Updated weights for policy 0, policy_version 1748383 (0.0010) [2023-12-27 04:07:53,616][105620] Updated weights for policy 1, policy_version 1751991 (0.0008) [2023-12-27 04:07:53,667][105620] Updated weights for policy 1, policy_version 1752001 (0.0008) [2023-12-27 04:07:53,712][105620] Updated weights for policy 1, policy_version 1752011 (0.0008) [2023-12-27 04:07:54,008][105692] Updated weights for policy 0, policy_version 1748393 (0.0010) [2023-12-27 04:07:54,060][105692] Updated weights for policy 0, policy_version 1748403 (0.0010) [2023-12-27 04:07:54,116][105692] Updated weights for policy 0, policy_version 1748413 (0.0010) [2023-12-27 04:07:54,490][105620] Updated weights for policy 1, policy_version 1752021 (0.0010) [2023-12-27 04:07:54,549][105620] Updated weights for policy 1, policy_version 1752031 (0.0010) [2023-12-27 04:07:54,600][105620] Updated weights for policy 1, policy_version 1752041 (0.0010) [2023-12-27 04:07:54,852][105692] Updated weights for policy 0, policy_version 1748423 (0.0010) [2023-12-27 04:07:54,913][105692] Updated weights for policy 0, policy_version 1748433 (0.0010) [2023-12-27 04:07:54,973][105692] Updated weights for policy 0, policy_version 1748443 (0.0010) [2023-12-27 04:07:55,359][105620] Updated weights for policy 1, policy_version 1752051 (0.0010) [2023-12-27 04:07:55,402][105620] Updated weights for policy 1, policy_version 1752061 (0.0010) [2023-12-27 04:07:55,465][105620] Updated weights for policy 1, policy_version 1752071 (0.0010) [2023-12-27 04:07:55,714][105692] Updated weights for policy 0, policy_version 1748453 (0.0010) [2023-12-27 04:07:55,775][105692] Updated weights for policy 0, policy_version 1748463 (0.0010) [2023-12-27 04:07:55,833][105692] Updated weights for policy 0, policy_version 1748473 (0.0010) [2023-12-27 04:07:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 896270336. Throughput: 0: 9853.7, 1: 9894.3. Samples: 896277324. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:07:56,063][104569] Avg episode reward: [(0, '8713.073'), (1, '9355.481')] [2023-12-27 04:07:56,158][105620] Updated weights for policy 1, policy_version 1752081 (0.0010) [2023-12-27 04:07:56,217][105620] Updated weights for policy 1, policy_version 1752091 (0.0011) [2023-12-27 04:07:56,270][105620] Updated weights for policy 1, policy_version 1752101 (0.0011) [2023-12-27 04:07:56,317][105620] Updated weights for policy 1, policy_version 1752111 (0.0010) [2023-12-27 04:07:56,567][105692] Updated weights for policy 0, policy_version 1748483 (0.0010) [2023-12-27 04:07:56,617][105692] Updated weights for policy 0, policy_version 1748493 (0.0007) [2023-12-27 04:07:56,667][105692] Updated weights for policy 0, policy_version 1748503 (0.0005) [2023-12-27 04:07:57,074][105620] Updated weights for policy 1, policy_version 1752121 (0.0011) [2023-12-27 04:07:57,129][105620] Updated weights for policy 1, policy_version 1752131 (0.0010) [2023-12-27 04:07:57,177][105620] Updated weights for policy 1, policy_version 1752141 (0.0010) [2023-12-27 04:07:57,320][105692] Updated weights for policy 0, policy_version 1748513 (0.0006) [2023-12-27 04:07:57,380][105692] Updated weights for policy 0, policy_version 1748523 (0.0006) [2023-12-27 04:07:57,433][105692] Updated weights for policy 0, policy_version 1748533 (0.0005) [2023-12-27 04:07:57,484][105692] Updated weights for policy 0, policy_version 1748543 (0.0005) [2023-12-27 04:07:57,860][105620] Updated weights for policy 1, policy_version 1752151 (0.0007) [2023-12-27 04:07:57,916][105620] Updated weights for policy 1, policy_version 1752162 (0.0009) [2023-12-27 04:07:57,964][105620] Updated weights for policy 1, policy_version 1752172 (0.0010) [2023-12-27 04:07:58,074][105692] Updated weights for policy 0, policy_version 1748553 (0.0005) [2023-12-27 04:07:58,135][105692] Updated weights for policy 0, policy_version 1748563 (0.0005) [2023-12-27 04:07:58,212][105692] Updated weights for policy 0, policy_version 1748573 (0.0007) [2023-12-27 04:07:58,762][105620] Updated weights for policy 1, policy_version 1752182 (0.0009) [2023-12-27 04:07:58,824][105620] Updated weights for policy 1, policy_version 1752192 (0.0009) [2023-12-27 04:07:58,900][105620] Updated weights for policy 1, policy_version 1752202 (0.0009) [2023-12-27 04:07:58,940][105692] Updated weights for policy 0, policy_version 1748583 (0.0008) [2023-12-27 04:07:59,006][105692] Updated weights for policy 0, policy_version 1748593 (0.0007) [2023-12-27 04:07:59,061][105692] Updated weights for policy 0, policy_version 1748603 (0.0006) [2023-12-27 04:07:59,649][105620] Updated weights for policy 1, policy_version 1752212 (0.0009) [2023-12-27 04:07:59,695][105620] Updated weights for policy 1, policy_version 1752222 (0.0007) [2023-12-27 04:07:59,746][105620] Updated weights for policy 1, policy_version 1752232 (0.0005) [2023-12-27 04:07:59,789][105692] Updated weights for policy 0, policy_version 1748613 (0.0007) [2023-12-27 04:07:59,840][105692] Updated weights for policy 0, policy_version 1748623 (0.0007) [2023-12-27 04:07:59,895][105692] Updated weights for policy 0, policy_version 1748633 (0.0008) [2023-12-27 04:08:00,413][105620] Updated weights for policy 1, policy_version 1752242 (0.0008) [2023-12-27 04:08:00,470][105620] Updated weights for policy 1, policy_version 1752252 (0.0009) [2023-12-27 04:08:00,517][105692] Updated weights for policy 0, policy_version 1748643 (0.0010) [2023-12-27 04:08:00,518][105620] Updated weights for policy 1, policy_version 1752262 (0.0010) [2023-12-27 04:08:00,562][105692] Updated weights for policy 0, policy_version 1748653 (0.0010) [2023-12-27 04:08:00,576][105620] Updated weights for policy 1, policy_version 1752272 (0.0010) [2023-12-27 04:08:00,611][105692] Updated weights for policy 0, policy_version 1748663 (0.0010) [2023-12-27 04:08:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 896368640. Throughput: 0: 9898.0, 1: 9856.6. Samples: 896336964. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:08:01,063][104569] Avg episode reward: [(0, '8713.249'), (1, '9170.398')] [2023-12-27 04:08:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001752272_448643072.pth... [2023-12-27 04:08:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001748672_447725568.pth... [2023-12-27 04:08:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001751152_448356352.pth [2023-12-27 04:08:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001747520_447430656.pth [2023-12-27 04:08:01,299][105692] Updated weights for policy 0, policy_version 1748673 (0.0010) [2023-12-27 04:08:01,325][105620] Updated weights for policy 1, policy_version 1752282 (0.0007) [2023-12-27 04:08:01,357][105692] Updated weights for policy 0, policy_version 1748683 (0.0007) [2023-12-27 04:08:01,381][105620] Updated weights for policy 1, policy_version 1752292 (0.0006) [2023-12-27 04:08:01,420][105692] Updated weights for policy 0, policy_version 1748693 (0.0009) [2023-12-27 04:08:01,435][105620] Updated weights for policy 1, policy_version 1752302 (0.0006) [2023-12-27 04:08:01,467][105692] Updated weights for policy 0, policy_version 1748703 (0.0007) [2023-12-27 04:08:02,164][105620] Updated weights for policy 1, policy_version 1752312 (0.0006) [2023-12-27 04:08:02,171][105692] Updated weights for policy 0, policy_version 1748713 (0.0006) [2023-12-27 04:08:02,220][105620] Updated weights for policy 1, policy_version 1752322 (0.0008) [2023-12-27 04:08:02,224][105692] Updated weights for policy 0, policy_version 1748723 (0.0005) [2023-12-27 04:08:02,283][105692] Updated weights for policy 0, policy_version 1748733 (0.0006) [2023-12-27 04:08:02,284][105620] Updated weights for policy 1, policy_version 1752332 (0.0008) [2023-12-27 04:08:02,967][105692] Updated weights for policy 0, policy_version 1748743 (0.0007) [2023-12-27 04:08:02,976][105620] Updated weights for policy 1, policy_version 1752342 (0.0007) [2023-12-27 04:08:03,028][105692] Updated weights for policy 0, policy_version 1748753 (0.0006) [2023-12-27 04:08:03,046][105620] Updated weights for policy 1, policy_version 1752352 (0.0009) [2023-12-27 04:08:03,084][105692] Updated weights for policy 0, policy_version 1748763 (0.0006) [2023-12-27 04:08:03,108][105620] Updated weights for policy 1, policy_version 1752362 (0.0009) [2023-12-27 04:08:03,707][105620] Updated weights for policy 1, policy_version 1752372 (0.0010) [2023-12-27 04:08:03,741][105692] Updated weights for policy 0, policy_version 1748773 (0.0006) [2023-12-27 04:08:03,765][105620] Updated weights for policy 1, policy_version 1752382 (0.0010) [2023-12-27 04:08:03,786][105692] Updated weights for policy 0, policy_version 1748783 (0.0009) [2023-12-27 04:08:03,820][105620] Updated weights for policy 1, policy_version 1752392 (0.0010) [2023-12-27 04:08:03,834][105692] Updated weights for policy 0, policy_version 1748793 (0.0006) [2023-12-27 04:08:04,566][105620] Updated weights for policy 1, policy_version 1752402 (0.0010) [2023-12-27 04:08:04,626][105620] Updated weights for policy 1, policy_version 1752412 (0.0011) [2023-12-27 04:08:04,635][105692] Updated weights for policy 0, policy_version 1748803 (0.0007) [2023-12-27 04:08:04,680][105620] Updated weights for policy 1, policy_version 1752422 (0.0010) [2023-12-27 04:08:04,690][105692] Updated weights for policy 0, policy_version 1748813 (0.0005) [2023-12-27 04:08:04,738][105620] Updated weights for policy 1, policy_version 1752432 (0.0010) [2023-12-27 04:08:04,742][105692] Updated weights for policy 0, policy_version 1748823 (0.0007) [2023-12-27 04:08:05,482][105620] Updated weights for policy 1, policy_version 1752442 (0.0010) [2023-12-27 04:08:05,507][105692] Updated weights for policy 0, policy_version 1748833 (0.0008) [2023-12-27 04:08:05,541][105620] Updated weights for policy 1, policy_version 1752452 (0.0010) [2023-12-27 04:08:05,559][105692] Updated weights for policy 0, policy_version 1748843 (0.0005) [2023-12-27 04:08:05,596][105620] Updated weights for policy 1, policy_version 1752462 (0.0010) [2023-12-27 04:08:05,614][105692] Updated weights for policy 0, policy_version 1748853 (0.0006) [2023-12-27 04:08:05,673][105692] Updated weights for policy 0, policy_version 1748863 (0.0008) [2023-12-27 04:08:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 896466944. Throughput: 0: 9872.0, 1: 9840.4. Samples: 896455888. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:08:06,063][104569] Avg episode reward: [(0, '8621.169'), (1, '9170.104')] [2023-12-27 04:08:06,374][105620] Updated weights for policy 1, policy_version 1752472 (0.0011) [2023-12-27 04:08:06,434][105620] Updated weights for policy 1, policy_version 1752482 (0.0010) [2023-12-27 04:08:06,444][105692] Updated weights for policy 0, policy_version 1748873 (0.0006) [2023-12-27 04:08:06,492][105620] Updated weights for policy 1, policy_version 1752492 (0.0010) [2023-12-27 04:08:06,494][105692] Updated weights for policy 0, policy_version 1748883 (0.0009) [2023-12-27 04:08:06,549][105692] Updated weights for policy 0, policy_version 1748893 (0.0007) [2023-12-27 04:08:07,232][105620] Updated weights for policy 1, policy_version 1752502 (0.0010) [2023-12-27 04:08:07,287][105620] Updated weights for policy 1, policy_version 1752512 (0.0010) [2023-12-27 04:08:07,332][105692] Updated weights for policy 0, policy_version 1748903 (0.0008) [2023-12-27 04:08:07,338][105620] Updated weights for policy 1, policy_version 1752522 (0.0010) [2023-12-27 04:08:07,388][105692] Updated weights for policy 0, policy_version 1748913 (0.0006) [2023-12-27 04:08:07,457][105692] Updated weights for policy 0, policy_version 1748923 (0.0008) [2023-12-27 04:08:08,112][105620] Updated weights for policy 1, policy_version 1752532 (0.0010) [2023-12-27 04:08:08,177][105620] Updated weights for policy 1, policy_version 1752542 (0.0010) [2023-12-27 04:08:08,225][105692] Updated weights for policy 0, policy_version 1748933 (0.0009) [2023-12-27 04:08:08,238][105620] Updated weights for policy 1, policy_version 1752552 (0.0010) [2023-12-27 04:08:08,286][105692] Updated weights for policy 0, policy_version 1748943 (0.0005) [2023-12-27 04:08:08,338][105692] Updated weights for policy 0, policy_version 1748953 (0.0008) [2023-12-27 04:08:08,907][105620] Updated weights for policy 1, policy_version 1752562 (0.0009) [2023-12-27 04:08:08,973][105620] Updated weights for policy 1, policy_version 1752572 (0.0005) [2023-12-27 04:08:09,040][105620] Updated weights for policy 1, policy_version 1752582 (0.0006) [2023-12-27 04:08:09,088][105692] Updated weights for policy 0, policy_version 1748963 (0.0009) [2023-12-27 04:08:09,102][105620] Updated weights for policy 1, policy_version 1752592 (0.0006) [2023-12-27 04:08:09,143][105692] Updated weights for policy 0, policy_version 1748973 (0.0010) [2023-12-27 04:08:09,206][105692] Updated weights for policy 0, policy_version 1748983 (0.0011) [2023-12-27 04:08:09,686][105620] Updated weights for policy 1, policy_version 1752602 (0.0006) [2023-12-27 04:08:09,741][105620] Updated weights for policy 1, policy_version 1752612 (0.0006) [2023-12-27 04:08:09,808][105620] Updated weights for policy 1, policy_version 1752622 (0.0006) [2023-12-27 04:08:09,977][105692] Updated weights for policy 0, policy_version 1748993 (0.0009) [2023-12-27 04:08:10,037][105692] Updated weights for policy 0, policy_version 1749003 (0.0011) [2023-12-27 04:08:10,093][105692] Updated weights for policy 0, policy_version 1749013 (0.0011) [2023-12-27 04:08:10,153][105692] Updated weights for policy 0, policy_version 1749023 (0.0011) [2023-12-27 04:08:10,426][105620] Updated weights for policy 1, policy_version 1752632 (0.0010) [2023-12-27 04:08:10,478][105620] Updated weights for policy 1, policy_version 1752642 (0.0010) [2023-12-27 04:08:10,530][105620] Updated weights for policy 1, policy_version 1752652 (0.0010) [2023-12-27 04:08:10,910][105692] Updated weights for policy 0, policy_version 1749033 (0.0010) [2023-12-27 04:08:10,973][105692] Updated weights for policy 0, policy_version 1749043 (0.0010) [2023-12-27 04:08:11,048][105692] Updated weights for policy 0, policy_version 1749053 (0.0010) [2023-12-27 04:08:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 896557056. Throughput: 0: 9827.1, 1: 9807.4. Samples: 896570072. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:08:11,062][104569] Avg episode reward: [(0, '8531.791'), (1, '9263.086')] [2023-12-27 04:08:11,298][105620] Updated weights for policy 1, policy_version 1752662 (0.0010) [2023-12-27 04:08:11,365][105620] Updated weights for policy 1, policy_version 1752672 (0.0009) [2023-12-27 04:08:11,423][105620] Updated weights for policy 1, policy_version 1752682 (0.0009) [2023-12-27 04:08:11,793][105692] Updated weights for policy 0, policy_version 1749063 (0.0011) [2023-12-27 04:08:11,858][105692] Updated weights for policy 0, policy_version 1749073 (0.0008) [2023-12-27 04:08:11,911][105692] Updated weights for policy 0, policy_version 1749083 (0.0011) [2023-12-27 04:08:12,217][105620] Updated weights for policy 1, policy_version 1752692 (0.0009) [2023-12-27 04:08:12,270][105620] Updated weights for policy 1, policy_version 1752702 (0.0011) [2023-12-27 04:08:12,327][105620] Updated weights for policy 1, policy_version 1752712 (0.0011) [2023-12-27 04:08:12,677][105692] Updated weights for policy 0, policy_version 1749093 (0.0010) [2023-12-27 04:08:12,731][105692] Updated weights for policy 0, policy_version 1749103 (0.0008) [2023-12-27 04:08:12,790][105692] Updated weights for policy 0, policy_version 1749113 (0.0009) [2023-12-27 04:08:12,961][105620] Updated weights for policy 1, policy_version 1752722 (0.0011) [2023-12-27 04:08:13,011][105620] Updated weights for policy 1, policy_version 1752732 (0.0011) [2023-12-27 04:08:13,060][105620] Updated weights for policy 1, policy_version 1752742 (0.0011) [2023-12-27 04:08:13,114][105620] Updated weights for policy 1, policy_version 1752752 (0.0011) [2023-12-27 04:08:13,438][105692] Updated weights for policy 0, policy_version 1749123 (0.0009) [2023-12-27 04:08:13,486][105692] Updated weights for policy 0, policy_version 1749133 (0.0009) [2023-12-27 04:08:13,535][105692] Updated weights for policy 0, policy_version 1749143 (0.0010) [2023-12-27 04:08:13,897][105620] Updated weights for policy 1, policy_version 1752762 (0.0010) [2023-12-27 04:08:13,965][105620] Updated weights for policy 1, policy_version 1752772 (0.0010) [2023-12-27 04:08:14,026][105620] Updated weights for policy 1, policy_version 1752782 (0.0007) [2023-12-27 04:08:14,158][105692] Updated weights for policy 0, policy_version 1749153 (0.0010) [2023-12-27 04:08:14,219][105692] Updated weights for policy 0, policy_version 1749163 (0.0006) [2023-12-27 04:08:14,284][105692] Updated weights for policy 0, policy_version 1749173 (0.0009) [2023-12-27 04:08:14,346][105692] Updated weights for policy 0, policy_version 1749183 (0.0011) [2023-12-27 04:08:14,740][105620] Updated weights for policy 1, policy_version 1752792 (0.0010) [2023-12-27 04:08:14,800][105620] Updated weights for policy 1, policy_version 1752802 (0.0011) [2023-12-27 04:08:14,861][105620] Updated weights for policy 1, policy_version 1752812 (0.0009) [2023-12-27 04:08:15,084][105692] Updated weights for policy 0, policy_version 1749193 (0.0008) [2023-12-27 04:08:15,146][105692] Updated weights for policy 0, policy_version 1749203 (0.0008) [2023-12-27 04:08:15,206][105692] Updated weights for policy 0, policy_version 1749213 (0.0008) [2023-12-27 04:08:15,605][105620] Updated weights for policy 1, policy_version 1752822 (0.0011) [2023-12-27 04:08:15,663][105620] Updated weights for policy 1, policy_version 1752832 (0.0011) [2023-12-27 04:08:15,722][105620] Updated weights for policy 1, policy_version 1752842 (0.0011) [2023-12-27 04:08:15,854][105692] Updated weights for policy 0, policy_version 1749223 (0.0006) [2023-12-27 04:08:15,918][105692] Updated weights for policy 0, policy_version 1749233 (0.0008) [2023-12-27 04:08:15,972][105692] Updated weights for policy 0, policy_version 1749244 (0.0010) [2023-12-27 04:08:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.9, 300 sec: 19605.3). Total num frames: 896663552. Throughput: 0: 9822.0, 1: 9751.8. Samples: 896627592. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:08:16,062][104569] Avg episode reward: [(0, '8715.325'), (1, '9262.707')] [2023-12-27 04:08:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001749248_447873024.pth... [2023-12-27 04:08:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001752848_448790528.pth... [2023-12-27 04:08:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001751728_448503808.pth [2023-12-27 04:08:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001748096_447578112.pth [2023-12-27 04:08:16,301][105620] Updated weights for policy 1, policy_version 1752852 (0.0011) [2023-12-27 04:08:16,348][105620] Updated weights for policy 1, policy_version 1752862 (0.0010) [2023-12-27 04:08:16,403][105620] Updated weights for policy 1, policy_version 1752872 (0.0010) [2023-12-27 04:08:16,799][105692] Updated weights for policy 0, policy_version 1749255 (0.0008) [2023-12-27 04:08:16,850][105692] Updated weights for policy 0, policy_version 1749265 (0.0009) [2023-12-27 04:08:16,900][105692] Updated weights for policy 0, policy_version 1749275 (0.0006) [2023-12-27 04:08:17,156][105620] Updated weights for policy 1, policy_version 1752882 (0.0010) [2023-12-27 04:08:17,210][105620] Updated weights for policy 1, policy_version 1752892 (0.0006) [2023-12-27 04:08:17,263][105620] Updated weights for policy 1, policy_version 1752902 (0.0005) [2023-12-27 04:08:17,311][105620] Updated weights for policy 1, policy_version 1752912 (0.0005) [2023-12-27 04:08:17,578][105692] Updated weights for policy 0, policy_version 1749285 (0.0008) [2023-12-27 04:08:17,644][105692] Updated weights for policy 0, policy_version 1749295 (0.0010) [2023-12-27 04:08:17,711][105692] Updated weights for policy 0, policy_version 1749305 (0.0010) [2023-12-27 04:08:17,909][105620] Updated weights for policy 1, policy_version 1752922 (0.0008) [2023-12-27 04:08:17,969][105620] Updated weights for policy 1, policy_version 1752932 (0.0007) [2023-12-27 04:08:18,027][105620] Updated weights for policy 1, policy_version 1752942 (0.0006) [2023-12-27 04:08:18,407][105692] Updated weights for policy 0, policy_version 1749315 (0.0010) [2023-12-27 04:08:18,473][105692] Updated weights for policy 0, policy_version 1749325 (0.0008) [2023-12-27 04:08:18,536][105692] Updated weights for policy 0, policy_version 1749335 (0.0008) [2023-12-27 04:08:18,666][105620] Updated weights for policy 1, policy_version 1752952 (0.0008) [2023-12-27 04:08:18,730][105620] Updated weights for policy 1, policy_version 1752962 (0.0005) [2023-12-27 04:08:18,797][105620] Updated weights for policy 1, policy_version 1752972 (0.0009) [2023-12-27 04:08:19,152][105692] Updated weights for policy 0, policy_version 1749345 (0.0005) [2023-12-27 04:08:19,219][105692] Updated weights for policy 0, policy_version 1749355 (0.0009) [2023-12-27 04:08:19,278][105692] Updated weights for policy 0, policy_version 1749365 (0.0009) [2023-12-27 04:08:19,331][105692] Updated weights for policy 0, policy_version 1749375 (0.0011) [2023-12-27 04:08:19,534][105620] Updated weights for policy 1, policy_version 1752982 (0.0011) [2023-12-27 04:08:19,584][105620] Updated weights for policy 1, policy_version 1752992 (0.0009) [2023-12-27 04:08:19,641][105620] Updated weights for policy 1, policy_version 1753002 (0.0008) [2023-12-27 04:08:20,092][105692] Updated weights for policy 0, policy_version 1749385 (0.0011) [2023-12-27 04:08:20,144][105692] Updated weights for policy 0, policy_version 1749395 (0.0011) [2023-12-27 04:08:20,200][105692] Updated weights for policy 0, policy_version 1749405 (0.0011) [2023-12-27 04:08:20,418][105620] Updated weights for policy 1, policy_version 1753012 (0.0009) [2023-12-27 04:08:20,479][105620] Updated weights for policy 1, policy_version 1753022 (0.0009) [2023-12-27 04:08:20,539][105620] Updated weights for policy 1, policy_version 1753032 (0.0008) [2023-12-27 04:08:20,974][105692] Updated weights for policy 0, policy_version 1749415 (0.0011) [2023-12-27 04:08:21,041][105692] Updated weights for policy 0, policy_version 1749425 (0.0010) [2023-12-27 04:08:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 896753664. Throughput: 0: 9804.8, 1: 9725.4. Samples: 896747692. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:08:21,062][104569] Avg episode reward: [(0, '8530.074'), (1, '9262.794')] [2023-12-27 04:08:21,100][105692] Updated weights for policy 0, policy_version 1749435 (0.0009) [2023-12-27 04:08:21,288][105620] Updated weights for policy 1, policy_version 1753042 (0.0008) [2023-12-27 04:08:21,348][105620] Updated weights for policy 1, policy_version 1753052 (0.0011) [2023-12-27 04:08:21,420][105620] Updated weights for policy 1, policy_version 1753062 (0.0009) [2023-12-27 04:08:21,485][105620] Updated weights for policy 1, policy_version 1753072 (0.0008) [2023-12-27 04:08:21,873][105692] Updated weights for policy 0, policy_version 1749445 (0.0008) [2023-12-27 04:08:21,931][105692] Updated weights for policy 0, policy_version 1749455 (0.0009) [2023-12-27 04:08:21,992][105692] Updated weights for policy 0, policy_version 1749465 (0.0008) [2023-12-27 04:08:22,187][105620] Updated weights for policy 1, policy_version 1753082 (0.0011) [2023-12-27 04:08:22,246][105620] Updated weights for policy 1, policy_version 1753092 (0.0011) [2023-12-27 04:08:22,318][105620] Updated weights for policy 1, policy_version 1753102 (0.0010) [2023-12-27 04:08:22,791][105692] Updated weights for policy 0, policy_version 1749475 (0.0009) [2023-12-27 04:08:22,852][105692] Updated weights for policy 0, policy_version 1749485 (0.0006) [2023-12-27 04:08:22,919][105692] Updated weights for policy 0, policy_version 1749495 (0.0007) [2023-12-27 04:08:23,077][105620] Updated weights for policy 1, policy_version 1753112 (0.0010) [2023-12-27 04:08:23,142][105620] Updated weights for policy 1, policy_version 1753122 (0.0010) [2023-12-27 04:08:23,208][105620] Updated weights for policy 1, policy_version 1753132 (0.0010) [2023-12-27 04:08:23,599][105692] Updated weights for policy 0, policy_version 1749505 (0.0008) [2023-12-27 04:08:23,651][105692] Updated weights for policy 0, policy_version 1749515 (0.0005) [2023-12-27 04:08:23,702][105692] Updated weights for policy 0, policy_version 1749525 (0.0006) [2023-12-27 04:08:23,746][105692] Updated weights for policy 0, policy_version 1749535 (0.0005) [2023-12-27 04:08:23,873][105620] Updated weights for policy 1, policy_version 1753142 (0.0010) [2023-12-27 04:08:23,925][105620] Updated weights for policy 1, policy_version 1753152 (0.0010) [2023-12-27 04:08:23,988][105620] Updated weights for policy 1, policy_version 1753162 (0.0010) [2023-12-27 04:08:24,319][105692] Updated weights for policy 0, policy_version 1749545 (0.0008) [2023-12-27 04:08:24,381][105692] Updated weights for policy 0, policy_version 1749555 (0.0009) [2023-12-27 04:08:24,432][105692] Updated weights for policy 0, policy_version 1749565 (0.0006) [2023-12-27 04:08:24,695][105620] Updated weights for policy 1, policy_version 1753172 (0.0007) [2023-12-27 04:08:24,743][105620] Updated weights for policy 1, policy_version 1753182 (0.0005) [2023-12-27 04:08:24,801][105620] Updated weights for policy 1, policy_version 1753192 (0.0005) [2023-12-27 04:08:25,284][105692] Updated weights for policy 0, policy_version 1749575 (0.0009) [2023-12-27 04:08:25,332][105620] Updated weights for policy 1, policy_version 1753202 (0.0005) [2023-12-27 04:08:25,339][105692] Updated weights for policy 0, policy_version 1749585 (0.0007) [2023-12-27 04:08:25,394][105620] Updated weights for policy 1, policy_version 1753212 (0.0007) [2023-12-27 04:08:25,409][105692] Updated weights for policy 0, policy_version 1749595 (0.0007) [2023-12-27 04:08:25,448][105620] Updated weights for policy 1, policy_version 1753222 (0.0007) [2023-12-27 04:08:25,508][105620] Updated weights for policy 1, policy_version 1753232 (0.0005) [2023-12-27 04:08:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 896851968. Throughput: 0: 9745.0, 1: 9749.9. Samples: 896864456. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:08:26,063][104569] Avg episode reward: [(0, '8256.800'), (1, '9176.108')] [2023-12-27 04:08:26,090][105620] Updated weights for policy 1, policy_version 1753242 (0.0010) [2023-12-27 04:08:26,142][105692] Updated weights for policy 0, policy_version 1749605 (0.0008) [2023-12-27 04:08:26,154][105620] Updated weights for policy 1, policy_version 1753252 (0.0010) [2023-12-27 04:08:26,193][105692] Updated weights for policy 0, policy_version 1749615 (0.0005) [2023-12-27 04:08:26,212][105620] Updated weights for policy 1, policy_version 1753262 (0.0010) [2023-12-27 04:08:26,258][105692] Updated weights for policy 0, policy_version 1749625 (0.0006) [2023-12-27 04:08:26,872][105620] Updated weights for policy 1, policy_version 1753272 (0.0010) [2023-12-27 04:08:26,919][105620] Updated weights for policy 1, policy_version 1753282 (0.0010) [2023-12-27 04:08:26,966][105620] Updated weights for policy 1, policy_version 1753292 (0.0010) [2023-12-27 04:08:27,007][105692] Updated weights for policy 0, policy_version 1749635 (0.0008) [2023-12-27 04:08:27,054][105692] Updated weights for policy 0, policy_version 1749645 (0.0008) [2023-12-27 04:08:27,102][105692] Updated weights for policy 0, policy_version 1749655 (0.0008) [2023-12-27 04:08:27,738][105620] Updated weights for policy 1, policy_version 1753302 (0.0010) [2023-12-27 04:08:27,798][105620] Updated weights for policy 1, policy_version 1753312 (0.0008) [2023-12-27 04:08:27,857][105620] Updated weights for policy 1, policy_version 1753322 (0.0008) [2023-12-27 04:08:27,888][105692] Updated weights for policy 0, policy_version 1749665 (0.0008) [2023-12-27 04:08:27,933][105692] Updated weights for policy 0, policy_version 1749675 (0.0007) [2023-12-27 04:08:27,987][105692] Updated weights for policy 0, policy_version 1749685 (0.0008) [2023-12-27 04:08:28,048][105692] Updated weights for policy 0, policy_version 1749695 (0.0005) [2023-12-27 04:08:28,576][105620] Updated weights for policy 1, policy_version 1753332 (0.0009) [2023-12-27 04:08:28,635][105620] Updated weights for policy 1, policy_version 1753342 (0.0010) [2023-12-27 04:08:28,683][105620] Updated weights for policy 1, policy_version 1753352 (0.0010) [2023-12-27 04:08:28,776][105692] Updated weights for policy 0, policy_version 1749705 (0.0008) [2023-12-27 04:08:28,837][105692] Updated weights for policy 0, policy_version 1749715 (0.0008) [2023-12-27 04:08:28,894][105692] Updated weights for policy 0, policy_version 1749725 (0.0008) [2023-12-27 04:08:29,459][105620] Updated weights for policy 1, policy_version 1753362 (0.0009) [2023-12-27 04:08:29,511][105620] Updated weights for policy 1, policy_version 1753372 (0.0010) [2023-12-27 04:08:29,558][105620] Updated weights for policy 1, policy_version 1753382 (0.0010) [2023-12-27 04:08:29,603][105620] Updated weights for policy 1, policy_version 1753392 (0.0010) [2023-12-27 04:08:29,660][105692] Updated weights for policy 0, policy_version 1749735 (0.0008) [2023-12-27 04:08:29,704][105692] Updated weights for policy 0, policy_version 1749745 (0.0008) [2023-12-27 04:08:29,747][105692] Updated weights for policy 0, policy_version 1749755 (0.0008) [2023-12-27 04:08:30,378][105620] Updated weights for policy 1, policy_version 1753402 (0.0010) [2023-12-27 04:08:30,446][105620] Updated weights for policy 1, policy_version 1753412 (0.0010) [2023-12-27 04:08:30,504][105620] Updated weights for policy 1, policy_version 1753422 (0.0010) [2023-12-27 04:08:30,548][105692] Updated weights for policy 0, policy_version 1749765 (0.0008) [2023-12-27 04:08:30,610][105692] Updated weights for policy 0, policy_version 1749775 (0.0008) [2023-12-27 04:08:30,675][105692] Updated weights for policy 0, policy_version 1749785 (0.0005) [2023-12-27 04:08:31,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 896950272. Throughput: 0: 9754.7, 1: 9707.8. Samples: 896922284. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:08:31,063][104569] Avg episode reward: [(0, '8258.595'), (1, '9267.892')] [2023-12-27 04:08:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001749792_448012288.pth... [2023-12-27 04:08:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001748672_447725568.pth [2023-12-27 04:08:31,074][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001753424_448937984.pth... [2023-12-27 04:08:31,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001752272_448643072.pth [2023-12-27 04:08:31,208][105620] Updated weights for policy 1, policy_version 1753432 (0.0007) [2023-12-27 04:08:31,274][105620] Updated weights for policy 1, policy_version 1753442 (0.0006) [2023-12-27 04:08:31,344][105620] Updated weights for policy 1, policy_version 1753452 (0.0005) [2023-12-27 04:08:31,412][105692] Updated weights for policy 0, policy_version 1749795 (0.0007) [2023-12-27 04:08:31,472][105692] Updated weights for policy 0, policy_version 1749805 (0.0008) [2023-12-27 04:08:31,535][105692] Updated weights for policy 0, policy_version 1749815 (0.0007) [2023-12-27 04:08:31,995][105620] Updated weights for policy 1, policy_version 1753462 (0.0008) [2023-12-27 04:08:32,054][105620] Updated weights for policy 1, policy_version 1753472 (0.0009) [2023-12-27 04:08:32,111][105620] Updated weights for policy 1, policy_version 1753482 (0.0007) [2023-12-27 04:08:32,275][105692] Updated weights for policy 0, policy_version 1749825 (0.0006) [2023-12-27 04:08:32,331][105692] Updated weights for policy 0, policy_version 1749835 (0.0010) [2023-12-27 04:08:32,389][105692] Updated weights for policy 0, policy_version 1749845 (0.0008) [2023-12-27 04:08:32,448][105692] Updated weights for policy 0, policy_version 1749855 (0.0010) [2023-12-27 04:08:32,780][105620] Updated weights for policy 1, policy_version 1753492 (0.0007) [2023-12-27 04:08:32,835][105620] Updated weights for policy 1, policy_version 1753502 (0.0009) [2023-12-27 04:08:32,892][105620] Updated weights for policy 1, policy_version 1753512 (0.0009) [2023-12-27 04:08:33,216][105692] Updated weights for policy 0, policy_version 1749865 (0.0006) [2023-12-27 04:08:33,275][105692] Updated weights for policy 0, policy_version 1749875 (0.0005) [2023-12-27 04:08:33,335][105692] Updated weights for policy 0, policy_version 1749885 (0.0005) [2023-12-27 04:08:33,569][105620] Updated weights for policy 1, policy_version 1753522 (0.0006) [2023-12-27 04:08:33,617][105620] Updated weights for policy 1, policy_version 1753532 (0.0010) [2023-12-27 04:08:33,673][105620] Updated weights for policy 1, policy_version 1753542 (0.0010) [2023-12-27 04:08:33,721][105620] Updated weights for policy 1, policy_version 1753552 (0.0010) [2023-12-27 04:08:33,922][105692] Updated weights for policy 0, policy_version 1749895 (0.0006) [2023-12-27 04:08:33,989][105692] Updated weights for policy 0, policy_version 1749905 (0.0009) [2023-12-27 04:08:34,038][105692] Updated weights for policy 0, policy_version 1749915 (0.0008) [2023-12-27 04:08:34,384][105620] Updated weights for policy 1, policy_version 1753562 (0.0007) [2023-12-27 04:08:34,447][105620] Updated weights for policy 1, policy_version 1753572 (0.0008) [2023-12-27 04:08:34,508][105620] Updated weights for policy 1, policy_version 1753582 (0.0006) [2023-12-27 04:08:34,828][105692] Updated weights for policy 0, policy_version 1749925 (0.0008) [2023-12-27 04:08:34,893][105692] Updated weights for policy 0, policy_version 1749935 (0.0009) [2023-12-27 04:08:34,962][105692] Updated weights for policy 0, policy_version 1749945 (0.0009) [2023-12-27 04:08:35,156][105620] Updated weights for policy 1, policy_version 1753592 (0.0008) [2023-12-27 04:08:35,214][105620] Updated weights for policy 1, policy_version 1753602 (0.0009) [2023-12-27 04:08:35,260][105620] Updated weights for policy 1, policy_version 1753612 (0.0008) [2023-12-27 04:08:35,703][105692] Updated weights for policy 0, policy_version 1749955 (0.0009) [2023-12-27 04:08:35,753][105692] Updated weights for policy 0, policy_version 1749965 (0.0009) [2023-12-27 04:08:35,804][105692] Updated weights for policy 0, policy_version 1749976 (0.0009) [2023-12-27 04:08:35,970][105620] Updated weights for policy 1, policy_version 1753622 (0.0007) [2023-12-27 04:08:36,025][105620] Updated weights for policy 1, policy_version 1753632 (0.0005) [2023-12-27 04:08:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 897048576. Throughput: 0: 9634.3, 1: 9824.4. Samples: 897038912. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:08:36,063][104569] Avg episode reward: [(0, '8628.428'), (1, '9262.447')] [2023-12-27 04:08:36,087][105620] Updated weights for policy 1, policy_version 1753642 (0.0005) [2023-12-27 04:08:36,650][105692] Updated weights for policy 0, policy_version 1749986 (0.0009) [2023-12-27 04:08:36,712][105692] Updated weights for policy 0, policy_version 1749996 (0.0008) [2023-12-27 04:08:36,738][105620] Updated weights for policy 1, policy_version 1753652 (0.0007) [2023-12-27 04:08:36,769][105692] Updated weights for policy 0, policy_version 1750006 (0.0008) [2023-12-27 04:08:36,794][105620] Updated weights for policy 1, policy_version 1753662 (0.0006) [2023-12-27 04:08:36,818][105692] Updated weights for policy 0, policy_version 1750016 (0.0008) [2023-12-27 04:08:36,855][105620] Updated weights for policy 1, policy_version 1753672 (0.0006) [2023-12-27 04:08:37,443][105620] Updated weights for policy 1, policy_version 1753682 (0.0006) [2023-12-27 04:08:37,510][105620] Updated weights for policy 1, policy_version 1753692 (0.0005) [2023-12-27 04:08:37,573][105620] Updated weights for policy 1, policy_version 1753702 (0.0007) [2023-12-27 04:08:37,631][105620] Updated weights for policy 1, policy_version 1753712 (0.0009) [2023-12-27 04:08:37,697][105692] Updated weights for policy 0, policy_version 1750026 (0.0009) [2023-12-27 04:08:37,755][105692] Updated weights for policy 0, policy_version 1750036 (0.0009) [2023-12-27 04:08:37,836][105692] Updated weights for policy 0, policy_version 1750046 (0.0008) [2023-12-27 04:08:38,298][105620] Updated weights for policy 1, policy_version 1753722 (0.0009) [2023-12-27 04:08:38,365][105620] Updated weights for policy 1, policy_version 1753732 (0.0007) [2023-12-27 04:08:38,423][105620] Updated weights for policy 1, policy_version 1753742 (0.0005) [2023-12-27 04:08:38,612][105692] Updated weights for policy 0, policy_version 1750056 (0.0010) [2023-12-27 04:08:38,673][105692] Updated weights for policy 0, policy_version 1750066 (0.0009) [2023-12-27 04:08:38,736][105692] Updated weights for policy 0, policy_version 1750076 (0.0009) [2023-12-27 04:08:39,037][105620] Updated weights for policy 1, policy_version 1753752 (0.0007) [2023-12-27 04:08:39,104][105620] Updated weights for policy 1, policy_version 1753762 (0.0005) [2023-12-27 04:08:39,157][105620] Updated weights for policy 1, policy_version 1753772 (0.0008) [2023-12-27 04:08:39,528][105692] Updated weights for policy 0, policy_version 1750086 (0.0010) [2023-12-27 04:08:39,587][105692] Updated weights for policy 0, policy_version 1750096 (0.0008) [2023-12-27 04:08:39,654][105692] Updated weights for policy 0, policy_version 1750106 (0.0008) [2023-12-27 04:08:39,827][105620] Updated weights for policy 1, policy_version 1753782 (0.0009) [2023-12-27 04:08:39,889][105620] Updated weights for policy 1, policy_version 1753792 (0.0008) [2023-12-27 04:08:39,953][105620] Updated weights for policy 1, policy_version 1753802 (0.0008) [2023-12-27 04:08:40,391][105692] Updated weights for policy 0, policy_version 1750116 (0.0008) [2023-12-27 04:08:40,440][105692] Updated weights for policy 0, policy_version 1750126 (0.0008) [2023-12-27 04:08:40,492][105692] Updated weights for policy 0, policy_version 1750136 (0.0008) [2023-12-27 04:08:40,709][105620] Updated weights for policy 1, policy_version 1753812 (0.0008) [2023-12-27 04:08:40,770][105620] Updated weights for policy 1, policy_version 1753822 (0.0010) [2023-12-27 04:08:40,825][105620] Updated weights for policy 1, policy_version 1753832 (0.0006) [2023-12-27 04:08:41,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 897146880. Throughput: 0: 9527.6, 1: 9953.4. Samples: 897153968. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:08:41,062][104569] Avg episode reward: [(0, '8716.754'), (1, '9262.313')] [2023-12-27 04:08:41,261][105692] Updated weights for policy 0, policy_version 1750146 (0.0010) [2023-12-27 04:08:41,312][105692] Updated weights for policy 0, policy_version 1750156 (0.0006) [2023-12-27 04:08:41,377][105692] Updated weights for policy 0, policy_version 1750166 (0.0008) [2023-12-27 04:08:41,443][105692] Updated weights for policy 0, policy_version 1750176 (0.0009) [2023-12-27 04:08:41,561][105620] Updated weights for policy 1, policy_version 1753842 (0.0006) [2023-12-27 04:08:41,619][105620] Updated weights for policy 1, policy_version 1753852 (0.0008) [2023-12-27 04:08:41,690][105620] Updated weights for policy 1, policy_version 1753862 (0.0008) [2023-12-27 04:08:41,758][105620] Updated weights for policy 1, policy_version 1753872 (0.0010) [2023-12-27 04:08:42,198][105692] Updated weights for policy 0, policy_version 1750186 (0.0007) [2023-12-27 04:08:42,255][105692] Updated weights for policy 0, policy_version 1750196 (0.0006) [2023-12-27 04:08:42,321][105692] Updated weights for policy 0, policy_version 1750206 (0.0006) [2023-12-27 04:08:42,504][105620] Updated weights for policy 1, policy_version 1753882 (0.0011) [2023-12-27 04:08:42,554][105620] Updated weights for policy 1, policy_version 1753892 (0.0006) [2023-12-27 04:08:42,626][105620] Updated weights for policy 1, policy_version 1753902 (0.0005) [2023-12-27 04:08:42,910][105692] Updated weights for policy 0, policy_version 1750216 (0.0008) [2023-12-27 04:08:42,961][105692] Updated weights for policy 0, policy_version 1750226 (0.0006) [2023-12-27 04:08:43,016][105692] Updated weights for policy 0, policy_version 1750236 (0.0006) [2023-12-27 04:08:43,281][105620] Updated weights for policy 1, policy_version 1753912 (0.0007) [2023-12-27 04:08:43,339][105620] Updated weights for policy 1, policy_version 1753922 (0.0008) [2023-12-27 04:08:43,383][105620] Updated weights for policy 1, policy_version 1753932 (0.0008) [2023-12-27 04:08:43,789][105692] Updated weights for policy 0, policy_version 1750246 (0.0010) [2023-12-27 04:08:43,844][105692] Updated weights for policy 0, policy_version 1750256 (0.0010) [2023-12-27 04:08:43,902][105692] Updated weights for policy 0, policy_version 1750266 (0.0010) [2023-12-27 04:08:44,128][105620] Updated weights for policy 1, policy_version 1753942 (0.0008) [2023-12-27 04:08:44,192][105620] Updated weights for policy 1, policy_version 1753952 (0.0009) [2023-12-27 04:08:44,252][105620] Updated weights for policy 1, policy_version 1753962 (0.0009) [2023-12-27 04:08:44,643][105692] Updated weights for policy 0, policy_version 1750276 (0.0010) [2023-12-27 04:08:44,700][105692] Updated weights for policy 0, policy_version 1750286 (0.0010) [2023-12-27 04:08:44,764][105692] Updated weights for policy 0, policy_version 1750296 (0.0009) [2023-12-27 04:08:45,057][105620] Updated weights for policy 1, policy_version 1753972 (0.0010) [2023-12-27 04:08:45,113][105620] Updated weights for policy 1, policy_version 1753982 (0.0009) [2023-12-27 04:08:45,168][105620] Updated weights for policy 1, policy_version 1753992 (0.0009) [2023-12-27 04:08:45,532][105692] Updated weights for policy 0, policy_version 1750306 (0.0010) [2023-12-27 04:08:45,583][105692] Updated weights for policy 0, policy_version 1750316 (0.0009) [2023-12-27 04:08:45,630][105692] Updated weights for policy 0, policy_version 1750326 (0.0008) [2023-12-27 04:08:45,679][105692] Updated weights for policy 0, policy_version 1750336 (0.0006) [2023-12-27 04:08:45,953][105620] Updated weights for policy 1, policy_version 1754002 (0.0009) [2023-12-27 04:08:46,006][105620] Updated weights for policy 1, policy_version 1754012 (0.0009) [2023-12-27 04:08:46,060][105620] Updated weights for policy 1, policy_version 1754022 (0.0009) [2023-12-27 04:08:46,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 897236992. Throughput: 0: 9489.7, 1: 9958.3. Samples: 897212120. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:08:46,062][104569] Avg episode reward: [(0, '8536.281'), (1, '9354.617')] [2023-12-27 04:08:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001750336_448151552.pth... [2023-12-27 04:08:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001749248_447873024.pth [2023-12-27 04:08:46,105][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001754032_449093632.pth... [2023-12-27 04:08:46,107][105620] Updated weights for policy 1, policy_version 1754032 (0.0009) [2023-12-27 04:08:46,108][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001752848_448790528.pth [2023-12-27 04:08:46,391][105692] Updated weights for policy 0, policy_version 1750346 (0.0005) [2023-12-27 04:08:46,448][105692] Updated weights for policy 0, policy_version 1750356 (0.0005) [2023-12-27 04:08:46,499][105692] Updated weights for policy 0, policy_version 1750366 (0.0005) [2023-12-27 04:08:46,946][105620] Updated weights for policy 1, policy_version 1754042 (0.0010) [2023-12-27 04:08:47,011][105620] Updated weights for policy 1, policy_version 1754052 (0.0009) [2023-12-27 04:08:47,065][105620] Updated weights for policy 1, policy_version 1754062 (0.0009) [2023-12-27 04:08:47,117][105692] Updated weights for policy 0, policy_version 1750376 (0.0009) [2023-12-27 04:08:47,164][105692] Updated weights for policy 0, policy_version 1750386 (0.0008) [2023-12-27 04:08:47,221][105692] Updated weights for policy 0, policy_version 1750396 (0.0009) [2023-12-27 04:08:47,867][105620] Updated weights for policy 1, policy_version 1754072 (0.0009) [2023-12-27 04:08:47,915][105692] Updated weights for policy 0, policy_version 1750406 (0.0007) [2023-12-27 04:08:47,929][105620] Updated weights for policy 1, policy_version 1754082 (0.0008) [2023-12-27 04:08:47,956][105692] Updated weights for policy 0, policy_version 1750416 (0.0010) [2023-12-27 04:08:47,982][105620] Updated weights for policy 1, policy_version 1754092 (0.0006) [2023-12-27 04:08:48,001][105692] Updated weights for policy 0, policy_version 1750426 (0.0010) [2023-12-27 04:08:48,750][105620] Updated weights for policy 1, policy_version 1754102 (0.0007) [2023-12-27 04:08:48,786][105692] Updated weights for policy 0, policy_version 1750436 (0.0009) [2023-12-27 04:08:48,812][105620] Updated weights for policy 1, policy_version 1754112 (0.0007) [2023-12-27 04:08:48,846][105692] Updated weights for policy 0, policy_version 1750446 (0.0006) [2023-12-27 04:08:48,876][105620] Updated weights for policy 1, policy_version 1754122 (0.0008) [2023-12-27 04:08:48,905][105692] Updated weights for policy 0, policy_version 1750456 (0.0007) [2023-12-27 04:08:49,558][105620] Updated weights for policy 1, policy_version 1754132 (0.0009) [2023-12-27 04:08:49,572][105692] Updated weights for policy 0, policy_version 1750466 (0.0006) [2023-12-27 04:08:49,620][105620] Updated weights for policy 1, policy_version 1754142 (0.0009) [2023-12-27 04:08:49,624][105692] Updated weights for policy 0, policy_version 1750476 (0.0006) [2023-12-27 04:08:49,681][105620] Updated weights for policy 1, policy_version 1754152 (0.0009) [2023-12-27 04:08:49,681][105692] Updated weights for policy 0, policy_version 1750486 (0.0005) [2023-12-27 04:08:49,745][105692] Updated weights for policy 0, policy_version 1750496 (0.0007) [2023-12-27 04:08:50,419][105620] Updated weights for policy 1, policy_version 1754162 (0.0007) [2023-12-27 04:08:50,478][105620] Updated weights for policy 1, policy_version 1754172 (0.0007) [2023-12-27 04:08:50,485][105692] Updated weights for policy 0, policy_version 1750506 (0.0006) [2023-12-27 04:08:50,538][105692] Updated weights for policy 0, policy_version 1750516 (0.0007) [2023-12-27 04:08:50,540][105620] Updated weights for policy 1, policy_version 1754182 (0.0008) [2023-12-27 04:08:50,603][105620] Updated weights for policy 1, policy_version 1754192 (0.0008) [2023-12-27 04:08:50,605][105692] Updated weights for policy 0, policy_version 1750526 (0.0009) [2023-12-27 04:08:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 897335296. Throughput: 0: 9491.6, 1: 9841.9. Samples: 897325896. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:08:51,063][104569] Avg episode reward: [(0, '8531.646'), (1, '9077.434')] [2023-12-27 04:08:51,373][105692] Updated weights for policy 0, policy_version 1750536 (0.0007) [2023-12-27 04:08:51,386][105620] Updated weights for policy 1, policy_version 1754202 (0.0009) [2023-12-27 04:08:51,439][105692] Updated weights for policy 0, policy_version 1750546 (0.0009) [2023-12-27 04:08:51,454][105620] Updated weights for policy 1, policy_version 1754212 (0.0008) [2023-12-27 04:08:51,498][105692] Updated weights for policy 0, policy_version 1750556 (0.0008) [2023-12-27 04:08:51,535][105620] Updated weights for policy 1, policy_version 1754222 (0.0008) [2023-12-27 04:08:52,148][105620] Updated weights for policy 1, policy_version 1754232 (0.0009) [2023-12-27 04:08:52,194][105620] Updated weights for policy 1, policy_version 1754242 (0.0008) [2023-12-27 04:08:52,246][105620] Updated weights for policy 1, policy_version 1754252 (0.0005) [2023-12-27 04:08:52,348][105692] Updated weights for policy 0, policy_version 1750566 (0.0007) [2023-12-27 04:08:52,407][105692] Updated weights for policy 0, policy_version 1750576 (0.0009) [2023-12-27 04:08:52,458][105692] Updated weights for policy 0, policy_version 1750586 (0.0009) [2023-12-27 04:08:53,032][105620] Updated weights for policy 1, policy_version 1754262 (0.0009) [2023-12-27 04:08:53,094][105620] Updated weights for policy 1, policy_version 1754272 (0.0010) [2023-12-27 04:08:53,148][105620] Updated weights for policy 1, policy_version 1754282 (0.0009) [2023-12-27 04:08:53,182][105692] Updated weights for policy 0, policy_version 1750596 (0.0009) [2023-12-27 04:08:53,239][105692] Updated weights for policy 0, policy_version 1750606 (0.0008) [2023-12-27 04:08:53,294][105692] Updated weights for policy 0, policy_version 1750616 (0.0009) [2023-12-27 04:08:53,916][105620] Updated weights for policy 1, policy_version 1754292 (0.0008) [2023-12-27 04:08:53,977][105620] Updated weights for policy 1, policy_version 1754302 (0.0009) [2023-12-27 04:08:54,030][105692] Updated weights for policy 0, policy_version 1750626 (0.0009) [2023-12-27 04:08:54,036][105620] Updated weights for policy 1, policy_version 1754312 (0.0007) [2023-12-27 04:08:54,087][105692] Updated weights for policy 0, policy_version 1750636 (0.0007) [2023-12-27 04:08:54,151][105692] Updated weights for policy 0, policy_version 1750646 (0.0009) [2023-12-27 04:08:54,204][105692] Updated weights for policy 0, policy_version 1750656 (0.0009) [2023-12-27 04:08:54,832][105620] Updated weights for policy 1, policy_version 1754322 (0.0007) [2023-12-27 04:08:54,892][105692] Updated weights for policy 0, policy_version 1750666 (0.0007) [2023-12-27 04:08:54,894][105620] Updated weights for policy 1, policy_version 1754332 (0.0007) [2023-12-27 04:08:54,949][105620] Updated weights for policy 1, policy_version 1754342 (0.0006) [2023-12-27 04:08:54,950][105692] Updated weights for policy 0, policy_version 1750676 (0.0009) [2023-12-27 04:08:55,003][105620] Updated weights for policy 1, policy_version 1754352 (0.0007) [2023-12-27 04:08:55,012][105692] Updated weights for policy 0, policy_version 1750686 (0.0008) [2023-12-27 04:08:55,683][105620] Updated weights for policy 1, policy_version 1754362 (0.0006) [2023-12-27 04:08:55,739][105620] Updated weights for policy 1, policy_version 1754372 (0.0007) [2023-12-27 04:08:55,798][105620] Updated weights for policy 1, policy_version 1754382 (0.0008) [2023-12-27 04:08:55,820][105692] Updated weights for policy 0, policy_version 1750696 (0.0007) [2023-12-27 04:08:55,885][105692] Updated weights for policy 0, policy_version 1750706 (0.0007) [2023-12-27 04:08:55,949][105692] Updated weights for policy 0, policy_version 1750716 (0.0009) [2023-12-27 04:08:56,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 897433600. Throughput: 0: 9503.8, 1: 9794.5. Samples: 897438500. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:08:56,063][104569] Avg episode reward: [(0, '8439.432'), (1, '9077.383')] [2023-12-27 04:08:56,411][105620] Updated weights for policy 1, policy_version 1754392 (0.0006) [2023-12-27 04:08:56,470][105620] Updated weights for policy 1, policy_version 1754402 (0.0005) [2023-12-27 04:08:56,526][105620] Updated weights for policy 1, policy_version 1754412 (0.0005) [2023-12-27 04:08:56,650][105692] Updated weights for policy 0, policy_version 1750726 (0.0009) [2023-12-27 04:08:56,694][105692] Updated weights for policy 0, policy_version 1750736 (0.0008) [2023-12-27 04:08:56,743][105692] Updated weights for policy 0, policy_version 1750746 (0.0008) [2023-12-27 04:08:57,176][105620] Updated weights for policy 1, policy_version 1754422 (0.0008) [2023-12-27 04:08:57,223][105620] Updated weights for policy 1, policy_version 1754432 (0.0010) [2023-12-27 04:08:57,270][105620] Updated weights for policy 1, policy_version 1754442 (0.0010) [2023-12-27 04:08:57,358][105692] Updated weights for policy 0, policy_version 1750756 (0.0007) [2023-12-27 04:08:57,411][105692] Updated weights for policy 0, policy_version 1750766 (0.0008) [2023-12-27 04:08:57,462][105692] Updated weights for policy 0, policy_version 1750776 (0.0010) [2023-12-27 04:08:57,989][105620] Updated weights for policy 1, policy_version 1754452 (0.0010) [2023-12-27 04:08:58,038][105620] Updated weights for policy 1, policy_version 1754462 (0.0010) [2023-12-27 04:08:58,086][105620] Updated weights for policy 1, policy_version 1754472 (0.0007) [2023-12-27 04:08:58,101][105692] Updated weights for policy 0, policy_version 1750786 (0.0008) [2023-12-27 04:08:58,145][105692] Updated weights for policy 0, policy_version 1750796 (0.0008) [2023-12-27 04:08:58,215][105692] Updated weights for policy 0, policy_version 1750806 (0.0008) [2023-12-27 04:08:58,281][105692] Updated weights for policy 0, policy_version 1750816 (0.0010) [2023-12-27 04:08:58,802][105620] Updated weights for policy 1, policy_version 1754482 (0.0006) [2023-12-27 04:08:58,872][105620] Updated weights for policy 1, policy_version 1754492 (0.0008) [2023-12-27 04:08:58,939][105620] Updated weights for policy 1, policy_version 1754502 (0.0009) [2023-12-27 04:08:59,016][105620] Updated weights for policy 1, policy_version 1754512 (0.0007) [2023-12-27 04:08:59,090][105692] Updated weights for policy 0, policy_version 1750826 (0.0008) [2023-12-27 04:08:59,145][105692] Updated weights for policy 0, policy_version 1750836 (0.0009) [2023-12-27 04:08:59,199][105692] Updated weights for policy 0, policy_version 1750846 (0.0009) [2023-12-27 04:08:59,798][105620] Updated weights for policy 1, policy_version 1754522 (0.0010) [2023-12-27 04:08:59,861][105620] Updated weights for policy 1, policy_version 1754532 (0.0009) [2023-12-27 04:08:59,867][105692] Updated weights for policy 0, policy_version 1750856 (0.0008) [2023-12-27 04:08:59,926][105620] Updated weights for policy 1, policy_version 1754542 (0.0008) [2023-12-27 04:08:59,935][105692] Updated weights for policy 0, policy_version 1750866 (0.0007) [2023-12-27 04:08:59,999][105692] Updated weights for policy 0, policy_version 1750876 (0.0009) [2023-12-27 04:09:00,629][105620] Updated weights for policy 1, policy_version 1754552 (0.0009) [2023-12-27 04:09:00,686][105620] Updated weights for policy 1, policy_version 1754562 (0.0009) [2023-12-27 04:09:00,739][105692] Updated weights for policy 0, policy_version 1750886 (0.0007) [2023-12-27 04:09:00,741][105620] Updated weights for policy 1, policy_version 1754572 (0.0006) [2023-12-27 04:09:00,789][105692] Updated weights for policy 0, policy_version 1750896 (0.0006) [2023-12-27 04:09:00,846][105692] Updated weights for policy 0, policy_version 1750906 (0.0009) [2023-12-27 04:09:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 897531904. Throughput: 0: 9540.7, 1: 9849.0. Samples: 897500128. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:09:01,062][104569] Avg episode reward: [(0, '8531.477'), (1, '9171.471')] [2023-12-27 04:09:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001750912_448299008.pth... [2023-12-27 04:09:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001754576_449232896.pth... [2023-12-27 04:09:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001753424_448937984.pth [2023-12-27 04:09:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001749792_448012288.pth [2023-12-27 04:09:01,493][105620] Updated weights for policy 1, policy_version 1754582 (0.0006) [2023-12-27 04:09:01,550][105620] Updated weights for policy 1, policy_version 1754592 (0.0008) [2023-12-27 04:09:01,587][105692] Updated weights for policy 0, policy_version 1750916 (0.0007) [2023-12-27 04:09:01,611][105620] Updated weights for policy 1, policy_version 1754602 (0.0008) [2023-12-27 04:09:01,660][105692] Updated weights for policy 0, policy_version 1750926 (0.0009) [2023-12-27 04:09:01,713][105692] Updated weights for policy 0, policy_version 1750936 (0.0009) [2023-12-27 04:09:02,330][105620] Updated weights for policy 1, policy_version 1754612 (0.0008) [2023-12-27 04:09:02,384][105692] Updated weights for policy 0, policy_version 1750946 (0.0006) [2023-12-27 04:09:02,391][105620] Updated weights for policy 1, policy_version 1754622 (0.0008) [2023-12-27 04:09:02,442][105692] Updated weights for policy 0, policy_version 1750956 (0.0005) [2023-12-27 04:09:02,449][105620] Updated weights for policy 1, policy_version 1754632 (0.0010) [2023-12-27 04:09:02,506][105692] Updated weights for policy 0, policy_version 1750966 (0.0007) [2023-12-27 04:09:02,572][105692] Updated weights for policy 0, policy_version 1750976 (0.0009) [2023-12-27 04:09:03,041][105620] Updated weights for policy 1, policy_version 1754642 (0.0008) [2023-12-27 04:09:03,097][105620] Updated weights for policy 1, policy_version 1754652 (0.0008) [2023-12-27 04:09:03,160][105620] Updated weights for policy 1, policy_version 1754662 (0.0005) [2023-12-27 04:09:03,209][105620] Updated weights for policy 1, policy_version 1754672 (0.0009) [2023-12-27 04:09:03,386][105692] Updated weights for policy 0, policy_version 1750986 (0.0009) [2023-12-27 04:09:03,436][105692] Updated weights for policy 0, policy_version 1750996 (0.0008) [2023-12-27 04:09:03,490][105692] Updated weights for policy 0, policy_version 1751006 (0.0009) [2023-12-27 04:09:03,921][105620] Updated weights for policy 1, policy_version 1754682 (0.0008) [2023-12-27 04:09:03,992][105620] Updated weights for policy 1, policy_version 1754692 (0.0006) [2023-12-27 04:09:04,059][105620] Updated weights for policy 1, policy_version 1754702 (0.0009) [2023-12-27 04:09:04,246][105692] Updated weights for policy 0, policy_version 1751016 (0.0006) [2023-12-27 04:09:04,307][105692] Updated weights for policy 0, policy_version 1751026 (0.0009) [2023-12-27 04:09:04,361][105692] Updated weights for policy 0, policy_version 1751036 (0.0009) [2023-12-27 04:09:04,703][105620] Updated weights for policy 1, policy_version 1754712 (0.0008) [2023-12-27 04:09:04,754][105620] Updated weights for policy 1, policy_version 1754722 (0.0009) [2023-12-27 04:09:04,806][105620] Updated weights for policy 1, policy_version 1754732 (0.0009) [2023-12-27 04:09:05,151][105692] Updated weights for policy 0, policy_version 1751046 (0.0010) [2023-12-27 04:09:05,214][105692] Updated weights for policy 0, policy_version 1751056 (0.0009) [2023-12-27 04:09:05,274][105692] Updated weights for policy 0, policy_version 1751066 (0.0009) [2023-12-27 04:09:05,571][105620] Updated weights for policy 1, policy_version 1754742 (0.0009) [2023-12-27 04:09:05,621][105620] Updated weights for policy 1, policy_version 1754752 (0.0008) [2023-12-27 04:09:05,679][105620] Updated weights for policy 1, policy_version 1754762 (0.0009) [2023-12-27 04:09:06,033][105692] Updated weights for policy 0, policy_version 1751076 (0.0009) [2023-12-27 04:09:06,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 897622016. Throughput: 0: 9458.3, 1: 9806.8. Samples: 897614620. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:09:06,062][104569] Avg episode reward: [(0, '8895.808'), (1, '9171.340')] [2023-12-27 04:09:06,089][105692] Updated weights for policy 0, policy_version 1751086 (0.0009) [2023-12-27 04:09:06,146][105692] Updated weights for policy 0, policy_version 1751096 (0.0008) [2023-12-27 04:09:06,479][105620] Updated weights for policy 1, policy_version 1754772 (0.0009) [2023-12-27 04:09:06,545][105620] Updated weights for policy 1, policy_version 1754782 (0.0009) [2023-12-27 04:09:06,600][105620] Updated weights for policy 1, policy_version 1754792 (0.0010) [2023-12-27 04:09:06,793][105692] Updated weights for policy 0, policy_version 1751106 (0.0009) [2023-12-27 04:09:06,858][105692] Updated weights for policy 0, policy_version 1751116 (0.0008) [2023-12-27 04:09:06,922][105692] Updated weights for policy 0, policy_version 1751126 (0.0009) [2023-12-27 04:09:06,983][105692] Updated weights for policy 0, policy_version 1751136 (0.0009) [2023-12-27 04:09:07,406][105620] Updated weights for policy 1, policy_version 1754802 (0.0009) [2023-12-27 04:09:07,457][105620] Updated weights for policy 1, policy_version 1754812 (0.0009) [2023-12-27 04:09:07,506][105620] Updated weights for policy 1, policy_version 1754822 (0.0009) [2023-12-27 04:09:07,561][105620] Updated weights for policy 1, policy_version 1754832 (0.0009) [2023-12-27 04:09:07,662][105692] Updated weights for policy 0, policy_version 1751146 (0.0010) [2023-12-27 04:09:07,723][105692] Updated weights for policy 0, policy_version 1751156 (0.0010) [2023-12-27 04:09:07,780][105692] Updated weights for policy 0, policy_version 1751166 (0.0008) [2023-12-27 04:09:08,311][105620] Updated weights for policy 1, policy_version 1754842 (0.0009) [2023-12-27 04:09:08,375][105620] Updated weights for policy 1, policy_version 1754852 (0.0010) [2023-12-27 04:09:08,433][105620] Updated weights for policy 1, policy_version 1754862 (0.0009) [2023-12-27 04:09:08,554][105692] Updated weights for policy 0, policy_version 1751176 (0.0008) [2023-12-27 04:09:08,616][105692] Updated weights for policy 0, policy_version 1751186 (0.0009) [2023-12-27 04:09:08,681][105692] Updated weights for policy 0, policy_version 1751196 (0.0009) [2023-12-27 04:09:09,182][105620] Updated weights for policy 1, policy_version 1754872 (0.0010) [2023-12-27 04:09:09,238][105620] Updated weights for policy 1, policy_version 1754882 (0.0008) [2023-12-27 04:09:09,299][105620] Updated weights for policy 1, policy_version 1754892 (0.0009) [2023-12-27 04:09:09,423][105692] Updated weights for policy 0, policy_version 1751206 (0.0009) [2023-12-27 04:09:09,482][105692] Updated weights for policy 0, policy_version 1751216 (0.0009) [2023-12-27 04:09:09,543][105692] Updated weights for policy 0, policy_version 1751226 (0.0008) [2023-12-27 04:09:10,059][105620] Updated weights for policy 1, policy_version 1754902 (0.0009) [2023-12-27 04:09:10,118][105620] Updated weights for policy 1, policy_version 1754912 (0.0006) [2023-12-27 04:09:10,186][105620] Updated weights for policy 1, policy_version 1754922 (0.0006) [2023-12-27 04:09:10,404][105692] Updated weights for policy 0, policy_version 1751236 (0.0009) [2023-12-27 04:09:10,465][105692] Updated weights for policy 0, policy_version 1751246 (0.0010) [2023-12-27 04:09:10,520][105692] Updated weights for policy 0, policy_version 1751256 (0.0008) [2023-12-27 04:09:10,735][105620] Updated weights for policy 1, policy_version 1754932 (0.0007) [2023-12-27 04:09:10,787][105620] Updated weights for policy 1, policy_version 1754942 (0.0007) [2023-12-27 04:09:10,835][105620] Updated weights for policy 1, policy_version 1754952 (0.0010) [2023-12-27 04:09:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 897720320. Throughput: 0: 9444.2, 1: 9731.9. Samples: 897727380. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:09:11,063][104569] Avg episode reward: [(0, '8805.163'), (1, '8894.199')] [2023-12-27 04:09:11,260][105692] Updated weights for policy 0, policy_version 1751266 (0.0009) [2023-12-27 04:09:11,314][105692] Updated weights for policy 0, policy_version 1751276 (0.0009) [2023-12-27 04:09:11,380][105692] Updated weights for policy 0, policy_version 1751286 (0.0009) [2023-12-27 04:09:11,429][105692] Updated weights for policy 0, policy_version 1751296 (0.0010) [2023-12-27 04:09:11,562][105620] Updated weights for policy 1, policy_version 1754962 (0.0007) [2023-12-27 04:09:11,632][105620] Updated weights for policy 1, policy_version 1754972 (0.0009) [2023-12-27 04:09:11,703][105620] Updated weights for policy 1, policy_version 1754982 (0.0009) [2023-12-27 04:09:11,769][105620] Updated weights for policy 1, policy_version 1754992 (0.0009) [2023-12-27 04:09:12,214][105692] Updated weights for policy 0, policy_version 1751306 (0.0009) [2023-12-27 04:09:12,275][105692] Updated weights for policy 0, policy_version 1751316 (0.0008) [2023-12-27 04:09:12,338][105692] Updated weights for policy 0, policy_version 1751326 (0.0008) [2023-12-27 04:09:12,509][105620] Updated weights for policy 1, policy_version 1755002 (0.0010) [2023-12-27 04:09:12,565][105620] Updated weights for policy 1, policy_version 1755012 (0.0010) [2023-12-27 04:09:12,636][105620] Updated weights for policy 1, policy_version 1755022 (0.0006) [2023-12-27 04:09:13,187][105620] Updated weights for policy 1, policy_version 1755032 (0.0006) [2023-12-27 04:09:13,211][105692] Updated weights for policy 0, policy_version 1751336 (0.0009) [2023-12-27 04:09:13,249][105620] Updated weights for policy 1, policy_version 1755042 (0.0010) [2023-12-27 04:09:13,267][105692] Updated weights for policy 0, policy_version 1751346 (0.0005) [2023-12-27 04:09:13,297][105620] Updated weights for policy 1, policy_version 1755052 (0.0010) [2023-12-27 04:09:13,320][105692] Updated weights for policy 0, policy_version 1751356 (0.0006) [2023-12-27 04:09:13,855][105620] Updated weights for policy 1, policy_version 1755062 (0.0007) [2023-12-27 04:09:13,913][105620] Updated weights for policy 1, policy_version 1755072 (0.0010) [2023-12-27 04:09:13,930][105692] Updated weights for policy 0, policy_version 1751366 (0.0007) [2023-12-27 04:09:13,963][105620] Updated weights for policy 1, policy_version 1755082 (0.0010) [2023-12-27 04:09:13,985][105692] Updated weights for policy 0, policy_version 1751376 (0.0006) [2023-12-27 04:09:14,036][105692] Updated weights for policy 0, policy_version 1751386 (0.0007) [2023-12-27 04:09:14,648][105620] Updated weights for policy 1, policy_version 1755092 (0.0010) [2023-12-27 04:09:14,718][105620] Updated weights for policy 1, policy_version 1755102 (0.0011) [2023-12-27 04:09:14,781][105620] Updated weights for policy 1, policy_version 1755112 (0.0012) [2023-12-27 04:09:14,802][105692] Updated weights for policy 0, policy_version 1751396 (0.0007) [2023-12-27 04:09:14,868][105692] Updated weights for policy 0, policy_version 1751406 (0.0007) [2023-12-27 04:09:14,932][105692] Updated weights for policy 0, policy_version 1751416 (0.0009) [2023-12-27 04:09:15,475][105620] Updated weights for policy 1, policy_version 1755122 (0.0013) [2023-12-27 04:09:15,534][105620] Updated weights for policy 1, policy_version 1755132 (0.0009) [2023-12-27 04:09:15,592][105620] Updated weights for policy 1, policy_version 1755142 (0.0007) [2023-12-27 04:09:15,646][105620] Updated weights for policy 1, policy_version 1755152 (0.0008) [2023-12-27 04:09:15,724][105692] Updated weights for policy 0, policy_version 1751426 (0.0009) [2023-12-27 04:09:15,789][105692] Updated weights for policy 0, policy_version 1751436 (0.0009) [2023-12-27 04:09:15,853][105692] Updated weights for policy 0, policy_version 1751446 (0.0009) [2023-12-27 04:09:15,914][105692] Updated weights for policy 0, policy_version 1751456 (0.0008) [2023-12-27 04:09:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 897818624. Throughput: 0: 9417.0, 1: 9790.4. Samples: 897786616. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:09:16,062][104569] Avg episode reward: [(0, '8714.484'), (1, '8894.290')] [2023-12-27 04:09:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001751456_448438272.pth... [2023-12-27 04:09:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001755152_449380352.pth... [2023-12-27 04:09:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001750336_448151552.pth [2023-12-27 04:09:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001754032_449093632.pth [2023-12-27 04:09:16,334][105620] Updated weights for policy 1, policy_version 1755162 (0.0005) [2023-12-27 04:09:16,403][105620] Updated weights for policy 1, policy_version 1755172 (0.0006) [2023-12-27 04:09:16,460][105620] Updated weights for policy 1, policy_version 1755182 (0.0005) [2023-12-27 04:09:16,521][105692] Updated weights for policy 0, policy_version 1751466 (0.0005) [2023-12-27 04:09:16,581][105692] Updated weights for policy 0, policy_version 1751476 (0.0005) [2023-12-27 04:09:16,638][105692] Updated weights for policy 0, policy_version 1751486 (0.0005) [2023-12-27 04:09:17,132][105620] Updated weights for policy 1, policy_version 1755192 (0.0008) [2023-12-27 04:09:17,193][105620] Updated weights for policy 1, policy_version 1755202 (0.0009) [2023-12-27 04:09:17,246][105620] Updated weights for policy 1, policy_version 1755212 (0.0008) [2023-12-27 04:09:17,313][105692] Updated weights for policy 0, policy_version 1751496 (0.0008) [2023-12-27 04:09:17,367][105692] Updated weights for policy 0, policy_version 1751506 (0.0009) [2023-12-27 04:09:17,428][105692] Updated weights for policy 0, policy_version 1751516 (0.0009) [2023-12-27 04:09:17,987][105620] Updated weights for policy 1, policy_version 1755222 (0.0008) [2023-12-27 04:09:18,044][105620] Updated weights for policy 1, policy_version 1755232 (0.0009) [2023-12-27 04:09:18,100][105620] Updated weights for policy 1, policy_version 1755242 (0.0008) [2023-12-27 04:09:18,174][105692] Updated weights for policy 0, policy_version 1751526 (0.0010) [2023-12-27 04:09:18,230][105692] Updated weights for policy 0, policy_version 1751536 (0.0011) [2023-12-27 04:09:18,278][105692] Updated weights for policy 0, policy_version 1751546 (0.0010) [2023-12-27 04:09:18,842][105620] Updated weights for policy 1, policy_version 1755252 (0.0009) [2023-12-27 04:09:18,905][105620] Updated weights for policy 1, policy_version 1755262 (0.0009) [2023-12-27 04:09:18,960][105620] Updated weights for policy 1, policy_version 1755272 (0.0009) [2023-12-27 04:09:19,055][105692] Updated weights for policy 0, policy_version 1751556 (0.0010) [2023-12-27 04:09:19,122][105692] Updated weights for policy 0, policy_version 1751566 (0.0009) [2023-12-27 04:09:19,170][105692] Updated weights for policy 0, policy_version 1751576 (0.0008) [2023-12-27 04:09:19,780][105620] Updated weights for policy 1, policy_version 1755282 (0.0009) [2023-12-27 04:09:19,848][105620] Updated weights for policy 1, policy_version 1755292 (0.0009) [2023-12-27 04:09:19,881][105692] Updated weights for policy 0, policy_version 1751586 (0.0009) [2023-12-27 04:09:19,907][105620] Updated weights for policy 1, policy_version 1755302 (0.0009) [2023-12-27 04:09:19,941][105692] Updated weights for policy 0, policy_version 1751596 (0.0008) [2023-12-27 04:09:19,970][105620] Updated weights for policy 1, policy_version 1755312 (0.0007) [2023-12-27 04:09:20,007][105692] Updated weights for policy 0, policy_version 1751606 (0.0008) [2023-12-27 04:09:20,069][105692] Updated weights for policy 0, policy_version 1751616 (0.0009) [2023-12-27 04:09:20,614][105620] Updated weights for policy 1, policy_version 1755322 (0.0010) [2023-12-27 04:09:20,665][105620] Updated weights for policy 1, policy_version 1755332 (0.0008) [2023-12-27 04:09:20,729][105620] Updated weights for policy 1, policy_version 1755342 (0.0008) [2023-12-27 04:09:20,853][105692] Updated weights for policy 0, policy_version 1751626 (0.0006) [2023-12-27 04:09:20,910][105692] Updated weights for policy 0, policy_version 1751636 (0.0006) [2023-12-27 04:09:20,972][105692] Updated weights for policy 0, policy_version 1751646 (0.0008) [2023-12-27 04:09:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 897916928. Throughput: 0: 9451.9, 1: 9735.3. Samples: 897902336. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:09:21,062][104569] Avg episode reward: [(0, '8621.671'), (1, '9261.988')] [2023-12-27 04:09:21,474][105620] Updated weights for policy 1, policy_version 1755352 (0.0009) [2023-12-27 04:09:21,539][105620] Updated weights for policy 1, policy_version 1755362 (0.0008) [2023-12-27 04:09:21,601][105620] Updated weights for policy 1, policy_version 1755372 (0.0009) [2023-12-27 04:09:21,676][105692] Updated weights for policy 0, policy_version 1751656 (0.0008) [2023-12-27 04:09:21,731][105692] Updated weights for policy 0, policy_version 1751666 (0.0008) [2023-12-27 04:09:21,794][105692] Updated weights for policy 0, policy_version 1751676 (0.0008) [2023-12-27 04:09:22,414][105620] Updated weights for policy 1, policy_version 1755382 (0.0007) [2023-12-27 04:09:22,476][105620] Updated weights for policy 1, policy_version 1755392 (0.0006) [2023-12-27 04:09:22,534][105692] Updated weights for policy 0, policy_version 1751686 (0.0008) [2023-12-27 04:09:22,542][105620] Updated weights for policy 1, policy_version 1755402 (0.0005) [2023-12-27 04:09:22,599][105692] Updated weights for policy 0, policy_version 1751696 (0.0007) [2023-12-27 04:09:22,670][105692] Updated weights for policy 0, policy_version 1751706 (0.0005) [2023-12-27 04:09:23,194][105620] Updated weights for policy 1, policy_version 1755412 (0.0008) [2023-12-27 04:09:23,245][105620] Updated weights for policy 1, policy_version 1755422 (0.0009) [2023-12-27 04:09:23,292][105620] Updated weights for policy 1, policy_version 1755432 (0.0008) [2023-12-27 04:09:23,387][105692] Updated weights for policy 0, policy_version 1751716 (0.0008) [2023-12-27 04:09:23,450][105692] Updated weights for policy 0, policy_version 1751726 (0.0009) [2023-12-27 04:09:23,506][105692] Updated weights for policy 0, policy_version 1751736 (0.0009) [2023-12-27 04:09:24,126][105620] Updated weights for policy 1, policy_version 1755442 (0.0009) [2023-12-27 04:09:24,135][105692] Updated weights for policy 0, policy_version 1751746 (0.0008) [2023-12-27 04:09:24,182][105620] Updated weights for policy 1, policy_version 1755452 (0.0009) [2023-12-27 04:09:24,185][105692] Updated weights for policy 0, policy_version 1751756 (0.0006) [2023-12-27 04:09:24,235][105692] Updated weights for policy 0, policy_version 1751766 (0.0005) [2023-12-27 04:09:24,238][105620] Updated weights for policy 1, policy_version 1755462 (0.0008) [2023-12-27 04:09:24,281][105692] Updated weights for policy 0, policy_version 1751776 (0.0005) [2023-12-27 04:09:24,285][105620] Updated weights for policy 1, policy_version 1755472 (0.0009) [2023-12-27 04:09:24,853][105692] Updated weights for policy 0, policy_version 1751786 (0.0009) [2023-12-27 04:09:24,900][105692] Updated weights for policy 0, policy_version 1751796 (0.0009) [2023-12-27 04:09:24,962][105692] Updated weights for policy 0, policy_version 1751806 (0.0009) [2023-12-27 04:09:25,123][105620] Updated weights for policy 1, policy_version 1755482 (0.0009) [2023-12-27 04:09:25,170][105620] Updated weights for policy 1, policy_version 1755492 (0.0009) [2023-12-27 04:09:25,223][105620] Updated weights for policy 1, policy_version 1755503 (0.0010) [2023-12-27 04:09:25,703][105692] Updated weights for policy 0, policy_version 1751816 (0.0006) [2023-12-27 04:09:25,762][105692] Updated weights for policy 0, policy_version 1751826 (0.0006) [2023-12-27 04:09:25,817][105692] Updated weights for policy 0, policy_version 1751836 (0.0007) [2023-12-27 04:09:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 898007040. Throughput: 0: 9611.6, 1: 9586.9. Samples: 898017904. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:09:26,062][104569] Avg episode reward: [(0, '8527.137'), (1, '9077.031')] [2023-12-27 04:09:26,070][105620] Updated weights for policy 1, policy_version 1755513 (0.0009) [2023-12-27 04:09:26,124][105620] Updated weights for policy 1, policy_version 1755523 (0.0009) [2023-12-27 04:09:26,171][105620] Updated weights for policy 1, policy_version 1755533 (0.0009) [2023-12-27 04:09:26,434][105692] Updated weights for policy 0, policy_version 1751846 (0.0010) [2023-12-27 04:09:26,493][105692] Updated weights for policy 0, policy_version 1751856 (0.0009) [2023-12-27 04:09:26,552][105692] Updated weights for policy 0, policy_version 1751866 (0.0006) [2023-12-27 04:09:26,982][105620] Updated weights for policy 1, policy_version 1755543 (0.0008) [2023-12-27 04:09:27,045][105620] Updated weights for policy 1, policy_version 1755553 (0.0009) [2023-12-27 04:09:27,106][105620] Updated weights for policy 1, policy_version 1755563 (0.0009) [2023-12-27 04:09:27,176][105692] Updated weights for policy 0, policy_version 1751876 (0.0006) [2023-12-27 04:09:27,221][105692] Updated weights for policy 0, policy_version 1751886 (0.0005) [2023-12-27 04:09:27,264][105692] Updated weights for policy 0, policy_version 1751896 (0.0005) [2023-12-27 04:09:27,743][105620] Updated weights for policy 1, policy_version 1755573 (0.0010) [2023-12-27 04:09:27,797][105620] Updated weights for policy 1, policy_version 1755585 (0.0010) [2023-12-27 04:09:27,822][105692] Updated weights for policy 0, policy_version 1751906 (0.0008) [2023-12-27 04:09:27,858][105620] Updated weights for policy 1, policy_version 1755595 (0.0009) [2023-12-27 04:09:27,876][105692] Updated weights for policy 0, policy_version 1751916 (0.0007) [2023-12-27 04:09:27,936][105692] Updated weights for policy 0, policy_version 1751926 (0.0010) [2023-12-27 04:09:28,000][105692] Updated weights for policy 0, policy_version 1751936 (0.0010) [2023-12-27 04:09:28,547][105620] Updated weights for policy 1, policy_version 1755605 (0.0007) [2023-12-27 04:09:28,606][105620] Updated weights for policy 1, policy_version 1755615 (0.0008) [2023-12-27 04:09:28,657][105620] Updated weights for policy 1, policy_version 1755625 (0.0008) [2023-12-27 04:09:28,705][105692] Updated weights for policy 0, policy_version 1751946 (0.0007) [2023-12-27 04:09:28,753][105692] Updated weights for policy 0, policy_version 1751956 (0.0009) [2023-12-27 04:09:28,811][105692] Updated weights for policy 0, policy_version 1751966 (0.0009) [2023-12-27 04:09:29,410][105620] Updated weights for policy 1, policy_version 1755635 (0.0008) [2023-12-27 04:09:29,464][105620] Updated weights for policy 1, policy_version 1755645 (0.0008) [2023-12-27 04:09:29,494][105692] Updated weights for policy 0, policy_version 1751976 (0.0006) [2023-12-27 04:09:29,529][105620] Updated weights for policy 1, policy_version 1755655 (0.0008) [2023-12-27 04:09:29,547][105692] Updated weights for policy 0, policy_version 1751986 (0.0006) [2023-12-27 04:09:29,604][105692] Updated weights for policy 0, policy_version 1751996 (0.0006) [2023-12-27 04:09:30,182][105692] Updated weights for policy 0, policy_version 1752006 (0.0006) [2023-12-27 04:09:30,242][105692] Updated weights for policy 0, policy_version 1752016 (0.0007) [2023-12-27 04:09:30,286][105620] Updated weights for policy 1, policy_version 1755665 (0.0008) [2023-12-27 04:09:30,299][105692] Updated weights for policy 0, policy_version 1752026 (0.0007) [2023-12-27 04:09:30,348][105620] Updated weights for policy 1, policy_version 1755675 (0.0007) [2023-12-27 04:09:30,406][105620] Updated weights for policy 1, policy_version 1755685 (0.0008) [2023-12-27 04:09:30,468][105620] Updated weights for policy 1, policy_version 1755695 (0.0008) [2023-12-27 04:09:31,033][105620] Updated weights for policy 1, policy_version 1755705 (0.0008) [2023-12-27 04:09:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.3, 300 sec: 19494.2). Total num frames: 898105344. Throughput: 0: 9691.0, 1: 9590.0. Samples: 898079764. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:09:31,062][104569] Avg episode reward: [(0, '7884.893'), (1, '9077.104')] [2023-12-27 04:09:31,089][105620] Updated weights for policy 1, policy_version 1755715 (0.0007) [2023-12-27 04:09:31,091][105692] Updated weights for policy 0, policy_version 1752036 (0.0006) [2023-12-27 04:09:31,150][105620] Updated weights for policy 1, policy_version 1755725 (0.0008) [2023-12-27 04:09:31,153][105692] Updated weights for policy 0, policy_version 1752046 (0.0006) [2023-12-27 04:09:31,170][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001755728_449527808.pth... [2023-12-27 04:09:31,174][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001754576_449232896.pth [2023-12-27 04:09:31,211][105692] Updated weights for policy 0, policy_version 1752056 (0.0009) [2023-12-27 04:09:31,253][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001752064_448593920.pth... [2023-12-27 04:09:31,257][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001750912_448299008.pth [2023-12-27 04:09:31,853][105620] Updated weights for policy 1, policy_version 1755735 (0.0009) [2023-12-27 04:09:31,905][105620] Updated weights for policy 1, policy_version 1755746 (0.0010) [2023-12-27 04:09:31,923][105692] Updated weights for policy 0, policy_version 1752066 (0.0008) [2023-12-27 04:09:31,969][105620] Updated weights for policy 1, policy_version 1755756 (0.0008) [2023-12-27 04:09:31,980][105692] Updated weights for policy 0, policy_version 1752076 (0.0006) [2023-12-27 04:09:32,032][105692] Updated weights for policy 0, policy_version 1752086 (0.0006) [2023-12-27 04:09:32,091][105692] Updated weights for policy 0, policy_version 1752096 (0.0007) [2023-12-27 04:09:32,750][105620] Updated weights for policy 1, policy_version 1755766 (0.0008) [2023-12-27 04:09:32,754][105692] Updated weights for policy 0, policy_version 1752106 (0.0005) [2023-12-27 04:09:32,805][105620] Updated weights for policy 1, policy_version 1755776 (0.0008) [2023-12-27 04:09:32,807][105692] Updated weights for policy 0, policy_version 1752116 (0.0007) [2023-12-27 04:09:32,861][105620] Updated weights for policy 1, policy_version 1755786 (0.0007) [2023-12-27 04:09:32,863][105692] Updated weights for policy 0, policy_version 1752126 (0.0006) [2023-12-27 04:09:33,427][105620] Updated weights for policy 1, policy_version 1755796 (0.0007) [2023-12-27 04:09:33,490][105620] Updated weights for policy 1, policy_version 1755806 (0.0008) [2023-12-27 04:09:33,547][105620] Updated weights for policy 1, policy_version 1755816 (0.0006) [2023-12-27 04:09:33,595][105692] Updated weights for policy 0, policy_version 1752136 (0.0009) [2023-12-27 04:09:33,639][105692] Updated weights for policy 0, policy_version 1752146 (0.0010) [2023-12-27 04:09:33,689][105692] Updated weights for policy 0, policy_version 1752156 (0.0010) [2023-12-27 04:09:34,099][105620] Updated weights for policy 1, policy_version 1755826 (0.0006) [2023-12-27 04:09:34,165][105620] Updated weights for policy 1, policy_version 1755836 (0.0008) [2023-12-27 04:09:34,230][105620] Updated weights for policy 1, policy_version 1755846 (0.0007) [2023-12-27 04:09:34,299][105620] Updated weights for policy 1, policy_version 1755856 (0.0009) [2023-12-27 04:09:34,445][105692] Updated weights for policy 0, policy_version 1752166 (0.0008) [2023-12-27 04:09:34,508][105692] Updated weights for policy 0, policy_version 1752176 (0.0009) [2023-12-27 04:09:34,575][105692] Updated weights for policy 0, policy_version 1752186 (0.0010) [2023-12-27 04:09:34,988][105620] Updated weights for policy 1, policy_version 1755866 (0.0009) [2023-12-27 04:09:35,047][105620] Updated weights for policy 1, policy_version 1755876 (0.0009) [2023-12-27 04:09:35,099][105620] Updated weights for policy 1, policy_version 1755886 (0.0010) [2023-12-27 04:09:35,339][105692] Updated weights for policy 0, policy_version 1752196 (0.0009) [2023-12-27 04:09:35,399][105692] Updated weights for policy 0, policy_version 1752206 (0.0009) [2023-12-27 04:09:35,451][105692] Updated weights for policy 0, policy_version 1752216 (0.0009) [2023-12-27 04:09:35,739][105620] Updated weights for policy 1, policy_version 1755896 (0.0007) [2023-12-27 04:09:35,784][105620] Updated weights for policy 1, policy_version 1755906 (0.0006) [2023-12-27 04:09:35,833][105620] Updated weights for policy 1, policy_version 1755916 (0.0005) [2023-12-27 04:09:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 898211840. Throughput: 0: 9682.6, 1: 9763.5. Samples: 898200968. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:09:36,063][104569] Avg episode reward: [(0, '8436.744'), (1, '9261.914')] [2023-12-27 04:09:36,331][105692] Updated weights for policy 0, policy_version 1752226 (0.0009) [2023-12-27 04:09:36,388][105692] Updated weights for policy 0, policy_version 1752236 (0.0008) [2023-12-27 04:09:36,419][105620] Updated weights for policy 1, policy_version 1755926 (0.0008) [2023-12-27 04:09:36,450][105692] Updated weights for policy 0, policy_version 1752246 (0.0007) [2023-12-27 04:09:36,478][105620] Updated weights for policy 1, policy_version 1755936 (0.0008) [2023-12-27 04:09:36,518][105692] Updated weights for policy 0, policy_version 1752256 (0.0006) [2023-12-27 04:09:36,548][105620] Updated weights for policy 1, policy_version 1755946 (0.0006) [2023-12-27 04:09:37,092][105620] Updated weights for policy 1, policy_version 1755956 (0.0005) [2023-12-27 04:09:37,153][105620] Updated weights for policy 1, policy_version 1755966 (0.0006) [2023-12-27 04:09:37,212][105620] Updated weights for policy 1, policy_version 1755976 (0.0009) [2023-12-27 04:09:37,267][105692] Updated weights for policy 0, policy_version 1752266 (0.0007) [2023-12-27 04:09:37,314][105692] Updated weights for policy 0, policy_version 1752276 (0.0009) [2023-12-27 04:09:37,368][105692] Updated weights for policy 0, policy_version 1752286 (0.0010) [2023-12-27 04:09:37,898][105620] Updated weights for policy 1, policy_version 1755986 (0.0007) [2023-12-27 04:09:37,944][105620] Updated weights for policy 1, policy_version 1755996 (0.0010) [2023-12-27 04:09:37,992][105620] Updated weights for policy 1, policy_version 1756006 (0.0010) [2023-12-27 04:09:38,045][105620] Updated weights for policy 1, policy_version 1756016 (0.0010) [2023-12-27 04:09:38,110][105692] Updated weights for policy 0, policy_version 1752296 (0.0007) [2023-12-27 04:09:38,168][105692] Updated weights for policy 0, policy_version 1752306 (0.0006) [2023-12-27 04:09:38,223][105692] Updated weights for policy 0, policy_version 1752316 (0.0007) [2023-12-27 04:09:38,747][105620] Updated weights for policy 1, policy_version 1756026 (0.0007) [2023-12-27 04:09:38,798][105620] Updated weights for policy 1, policy_version 1756036 (0.0009) [2023-12-27 04:09:38,849][105620] Updated weights for policy 1, policy_version 1756046 (0.0009) [2023-12-27 04:09:38,978][105692] Updated weights for policy 0, policy_version 1752326 (0.0008) [2023-12-27 04:09:39,033][105692] Updated weights for policy 0, policy_version 1752336 (0.0009) [2023-12-27 04:09:39,080][105692] Updated weights for policy 0, policy_version 1752346 (0.0009) [2023-12-27 04:09:39,595][105620] Updated weights for policy 1, policy_version 1756056 (0.0009) [2023-12-27 04:09:39,644][105620] Updated weights for policy 1, policy_version 1756066 (0.0009) [2023-12-27 04:09:39,707][105620] Updated weights for policy 1, policy_version 1756076 (0.0009) [2023-12-27 04:09:39,856][105692] Updated weights for policy 0, policy_version 1752356 (0.0008) [2023-12-27 04:09:39,921][105692] Updated weights for policy 0, policy_version 1752366 (0.0006) [2023-12-27 04:09:39,984][105692] Updated weights for policy 0, policy_version 1752376 (0.0009) [2023-12-27 04:09:40,386][105620] Updated weights for policy 1, policy_version 1756086 (0.0008) [2023-12-27 04:09:40,452][105620] Updated weights for policy 1, policy_version 1756096 (0.0007) [2023-12-27 04:09:40,515][105620] Updated weights for policy 1, policy_version 1756106 (0.0008) [2023-12-27 04:09:40,731][105692] Updated weights for policy 0, policy_version 1752386 (0.0009) [2023-12-27 04:09:40,795][105692] Updated weights for policy 0, policy_version 1752396 (0.0005) [2023-12-27 04:09:40,863][105692] Updated weights for policy 0, policy_version 1752406 (0.0006) [2023-12-27 04:09:40,925][105692] Updated weights for policy 0, policy_version 1752416 (0.0007) [2023-12-27 04:09:41,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 898310144. Throughput: 0: 9648.7, 1: 9884.4. Samples: 898317488. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:09:41,062][104569] Avg episode reward: [(0, '8527.595'), (1, '9076.638')] [2023-12-27 04:09:41,255][105620] Updated weights for policy 1, policy_version 1756116 (0.0009) [2023-12-27 04:09:41,307][105620] Updated weights for policy 1, policy_version 1756126 (0.0009) [2023-12-27 04:09:41,373][105620] Updated weights for policy 1, policy_version 1756136 (0.0009) [2023-12-27 04:09:41,574][105692] Updated weights for policy 0, policy_version 1752426 (0.0006) [2023-12-27 04:09:41,638][105692] Updated weights for policy 0, policy_version 1752436 (0.0009) [2023-12-27 04:09:41,711][105692] Updated weights for policy 0, policy_version 1752446 (0.0008) [2023-12-27 04:09:42,184][105620] Updated weights for policy 1, policy_version 1756146 (0.0010) [2023-12-27 04:09:42,237][105620] Updated weights for policy 1, policy_version 1756156 (0.0008) [2023-12-27 04:09:42,298][105620] Updated weights for policy 1, policy_version 1756166 (0.0008) [2023-12-27 04:09:42,366][105620] Updated weights for policy 1, policy_version 1756176 (0.0009) [2023-12-27 04:09:42,444][105692] Updated weights for policy 0, policy_version 1752456 (0.0010) [2023-12-27 04:09:42,507][105692] Updated weights for policy 0, policy_version 1752466 (0.0010) [2023-12-27 04:09:42,575][105692] Updated weights for policy 0, policy_version 1752476 (0.0010) [2023-12-27 04:09:43,132][105620] Updated weights for policy 1, policy_version 1756186 (0.0006) [2023-12-27 04:09:43,187][105620] Updated weights for policy 1, policy_version 1756196 (0.0006) [2023-12-27 04:09:43,249][105620] Updated weights for policy 1, policy_version 1756206 (0.0005) [2023-12-27 04:09:43,335][105692] Updated weights for policy 0, policy_version 1752486 (0.0007) [2023-12-27 04:09:43,388][105692] Updated weights for policy 0, policy_version 1752496 (0.0005) [2023-12-27 04:09:43,445][105692] Updated weights for policy 0, policy_version 1752506 (0.0005) [2023-12-27 04:09:43,771][105620] Updated weights for policy 1, policy_version 1756216 (0.0005) [2023-12-27 04:09:43,834][105620] Updated weights for policy 1, policy_version 1756226 (0.0005) [2023-12-27 04:09:43,885][105620] Updated weights for policy 1, policy_version 1756236 (0.0007) [2023-12-27 04:09:43,977][105692] Updated weights for policy 0, policy_version 1752516 (0.0007) [2023-12-27 04:09:44,035][105692] Updated weights for policy 0, policy_version 1752526 (0.0010) [2023-12-27 04:09:44,089][105692] Updated weights for policy 0, policy_version 1752536 (0.0010) [2023-12-27 04:09:44,585][105620] Updated weights for policy 1, policy_version 1756246 (0.0006) [2023-12-27 04:09:44,643][105620] Updated weights for policy 1, policy_version 1756256 (0.0010) [2023-12-27 04:09:44,694][105692] Updated weights for policy 0, policy_version 1752546 (0.0010) [2023-12-27 04:09:44,707][105620] Updated weights for policy 1, policy_version 1756266 (0.0011) [2023-12-27 04:09:44,759][105692] Updated weights for policy 0, policy_version 1752556 (0.0008) [2023-12-27 04:09:44,828][105692] Updated weights for policy 0, policy_version 1752566 (0.0008) [2023-12-27 04:09:44,892][105692] Updated weights for policy 0, policy_version 1752576 (0.0008) [2023-12-27 04:09:45,412][105620] Updated weights for policy 1, policy_version 1756276 (0.0010) [2023-12-27 04:09:45,471][105620] Updated weights for policy 1, policy_version 1756286 (0.0009) [2023-12-27 04:09:45,524][105620] Updated weights for policy 1, policy_version 1756296 (0.0009) [2023-12-27 04:09:45,571][105692] Updated weights for policy 0, policy_version 1752586 (0.0008) [2023-12-27 04:09:45,621][105692] Updated weights for policy 0, policy_version 1752596 (0.0008) [2023-12-27 04:09:45,670][105692] Updated weights for policy 0, policy_version 1752606 (0.0008) [2023-12-27 04:09:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 898408448. Throughput: 0: 9632.3, 1: 9858.3. Samples: 898377208. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:09:46,063][104569] Avg episode reward: [(0, '8801.997'), (1, '8801.781')] [2023-12-27 04:09:46,073][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001752608_448733184.pth... [2023-12-27 04:09:46,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001756304_449675264.pth... [2023-12-27 04:09:46,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001755152_449380352.pth [2023-12-27 04:09:46,082][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001751456_448438272.pth [2023-12-27 04:09:46,286][105620] Updated weights for policy 1, policy_version 1756306 (0.0008) [2023-12-27 04:09:46,338][105620] Updated weights for policy 1, policy_version 1756316 (0.0008) [2023-12-27 04:09:46,365][105692] Updated weights for policy 0, policy_version 1752616 (0.0009) [2023-12-27 04:09:46,398][105620] Updated weights for policy 1, policy_version 1756326 (0.0009) [2023-12-27 04:09:46,428][105692] Updated weights for policy 0, policy_version 1752626 (0.0006) [2023-12-27 04:09:46,458][105620] Updated weights for policy 1, policy_version 1756336 (0.0009) [2023-12-27 04:09:46,482][105692] Updated weights for policy 0, policy_version 1752636 (0.0005) [2023-12-27 04:09:47,167][105620] Updated weights for policy 1, policy_version 1756346 (0.0010) [2023-12-27 04:09:47,226][105620] Updated weights for policy 1, policy_version 1756356 (0.0010) [2023-12-27 04:09:47,233][105692] Updated weights for policy 0, policy_version 1752646 (0.0007) [2023-12-27 04:09:47,284][105620] Updated weights for policy 1, policy_version 1756366 (0.0010) [2023-12-27 04:09:47,287][105692] Updated weights for policy 0, policy_version 1752656 (0.0005) [2023-12-27 04:09:47,336][105692] Updated weights for policy 0, policy_version 1752666 (0.0005) [2023-12-27 04:09:47,912][105692] Updated weights for policy 0, policy_version 1752676 (0.0007) [2023-12-27 04:09:47,969][105692] Updated weights for policy 0, policy_version 1752686 (0.0010) [2023-12-27 04:09:48,006][105620] Updated weights for policy 1, policy_version 1756376 (0.0007) [2023-12-27 04:09:48,033][105692] Updated weights for policy 0, policy_version 1752696 (0.0009) [2023-12-27 04:09:48,060][105620] Updated weights for policy 1, policy_version 1756386 (0.0006) [2023-12-27 04:09:48,123][105620] Updated weights for policy 1, policy_version 1756396 (0.0007) [2023-12-27 04:09:48,721][105692] Updated weights for policy 0, policy_version 1752706 (0.0006) [2023-12-27 04:09:48,778][105692] Updated weights for policy 0, policy_version 1752716 (0.0005) [2023-12-27 04:09:48,832][105692] Updated weights for policy 0, policy_version 1752726 (0.0005) [2023-12-27 04:09:48,887][105692] Updated weights for policy 0, policy_version 1752736 (0.0006) [2023-12-27 04:09:48,926][105620] Updated weights for policy 1, policy_version 1756406 (0.0009) [2023-12-27 04:09:48,979][105620] Updated weights for policy 1, policy_version 1756416 (0.0010) [2023-12-27 04:09:49,033][105620] Updated weights for policy 1, policy_version 1756426 (0.0009) [2023-12-27 04:09:49,527][105692] Updated weights for policy 0, policy_version 1752746 (0.0010) [2023-12-27 04:09:49,598][105692] Updated weights for policy 0, policy_version 1752756 (0.0008) [2023-12-27 04:09:49,662][105692] Updated weights for policy 0, policy_version 1752766 (0.0009) [2023-12-27 04:09:49,905][105620] Updated weights for policy 1, policy_version 1756436 (0.0009) [2023-12-27 04:09:49,974][105620] Updated weights for policy 1, policy_version 1756446 (0.0008) [2023-12-27 04:09:50,041][105620] Updated weights for policy 1, policy_version 1756456 (0.0009) [2023-12-27 04:09:50,322][105692] Updated weights for policy 0, policy_version 1752776 (0.0006) [2023-12-27 04:09:50,378][105692] Updated weights for policy 0, policy_version 1752786 (0.0005) [2023-12-27 04:09:50,442][105692] Updated weights for policy 0, policy_version 1752796 (0.0005) [2023-12-27 04:09:50,900][105620] Updated weights for policy 1, policy_version 1756466 (0.0010) [2023-12-27 04:09:50,951][105620] Updated weights for policy 1, policy_version 1756476 (0.0008) [2023-12-27 04:09:51,009][105620] Updated weights for policy 1, policy_version 1756486 (0.0009) [2023-12-27 04:09:51,014][105692] Updated weights for policy 0, policy_version 1752806 (0.0005) [2023-12-27 04:09:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 898498560. Throughput: 0: 9791.0, 1: 9791.6. Samples: 898495840. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:09:51,063][104569] Avg episode reward: [(0, '8623.160'), (1, '9079.163')] [2023-12-27 04:09:51,070][105620] Updated weights for policy 1, policy_version 1756496 (0.0007) [2023-12-27 04:09:51,080][105692] Updated weights for policy 0, policy_version 1752816 (0.0011) [2023-12-27 04:09:51,144][105692] Updated weights for policy 0, policy_version 1752826 (0.0006) [2023-12-27 04:09:51,747][105692] Updated weights for policy 0, policy_version 1752836 (0.0007) [2023-12-27 04:09:51,801][105692] Updated weights for policy 0, policy_version 1752846 (0.0006) [2023-12-27 04:09:51,853][105692] Updated weights for policy 0, policy_version 1752856 (0.0006) [2023-12-27 04:09:51,903][105620] Updated weights for policy 1, policy_version 1756506 (0.0007) [2023-12-27 04:09:51,967][105620] Updated weights for policy 1, policy_version 1756516 (0.0007) [2023-12-27 04:09:52,038][105620] Updated weights for policy 1, policy_version 1756526 (0.0008) [2023-12-27 04:09:52,427][105692] Updated weights for policy 0, policy_version 1752866 (0.0005) [2023-12-27 04:09:52,490][105692] Updated weights for policy 0, policy_version 1752876 (0.0009) [2023-12-27 04:09:52,546][105692] Updated weights for policy 0, policy_version 1752886 (0.0008) [2023-12-27 04:09:52,599][105692] Updated weights for policy 0, policy_version 1752896 (0.0010) [2023-12-27 04:09:52,700][105620] Updated weights for policy 1, policy_version 1756536 (0.0008) [2023-12-27 04:09:52,771][105620] Updated weights for policy 1, policy_version 1756546 (0.0009) [2023-12-27 04:09:52,834][105620] Updated weights for policy 1, policy_version 1756556 (0.0008) [2023-12-27 04:09:53,243][105692] Updated weights for policy 0, policy_version 1752906 (0.0010) [2023-12-27 04:09:53,295][105692] Updated weights for policy 0, policy_version 1752916 (0.0010) [2023-12-27 04:09:53,350][105692] Updated weights for policy 0, policy_version 1752926 (0.0009) [2023-12-27 04:09:53,625][105620] Updated weights for policy 1, policy_version 1756566 (0.0007) [2023-12-27 04:09:53,681][105620] Updated weights for policy 1, policy_version 1756576 (0.0008) [2023-12-27 04:09:53,742][105620] Updated weights for policy 1, policy_version 1756586 (0.0009) [2023-12-27 04:09:54,029][105692] Updated weights for policy 0, policy_version 1752936 (0.0007) [2023-12-27 04:09:54,081][105692] Updated weights for policy 0, policy_version 1752946 (0.0006) [2023-12-27 04:09:54,145][105692] Updated weights for policy 0, policy_version 1752956 (0.0005) [2023-12-27 04:09:54,494][105620] Updated weights for policy 1, policy_version 1756596 (0.0010) [2023-12-27 04:09:54,549][105620] Updated weights for policy 1, policy_version 1756606 (0.0009) [2023-12-27 04:09:54,610][105620] Updated weights for policy 1, policy_version 1756616 (0.0009) [2023-12-27 04:09:54,670][105692] Updated weights for policy 0, policy_version 1752966 (0.0005) [2023-12-27 04:09:54,718][105692] Updated weights for policy 0, policy_version 1752976 (0.0005) [2023-12-27 04:09:54,769][105692] Updated weights for policy 0, policy_version 1752986 (0.0005) [2023-12-27 04:09:55,358][105692] Updated weights for policy 0, policy_version 1752996 (0.0006) [2023-12-27 04:09:55,409][105692] Updated weights for policy 0, policy_version 1753006 (0.0009) [2023-12-27 04:09:55,454][105692] Updated weights for policy 0, policy_version 1753016 (0.0008) [2023-12-27 04:09:55,456][105620] Updated weights for policy 1, policy_version 1756626 (0.0010) [2023-12-27 04:09:55,513][105620] Updated weights for policy 1, policy_version 1756636 (0.0008) [2023-12-27 04:09:55,569][105620] Updated weights for policy 1, policy_version 1756646 (0.0009) [2023-12-27 04:09:55,634][105620] Updated weights for policy 1, policy_version 1756656 (0.0009) [2023-12-27 04:09:56,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 898605056. Throughput: 0: 10030.6, 1: 9694.8. Samples: 898615020. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:09:56,062][104569] Avg episode reward: [(0, '8533.598'), (1, '9171.043')] [2023-12-27 04:09:56,215][105692] Updated weights for policy 0, policy_version 1753026 (0.0006) [2023-12-27 04:09:56,269][105692] Updated weights for policy 0, policy_version 1753036 (0.0009) [2023-12-27 04:09:56,323][105692] Updated weights for policy 0, policy_version 1753046 (0.0009) [2023-12-27 04:09:56,377][105620] Updated weights for policy 1, policy_version 1756666 (0.0006) [2023-12-27 04:09:56,386][105692] Updated weights for policy 0, policy_version 1753056 (0.0010) [2023-12-27 04:09:56,442][105620] Updated weights for policy 1, policy_version 1756676 (0.0008) [2023-12-27 04:09:56,497][105620] Updated weights for policy 1, policy_version 1756686 (0.0008) [2023-12-27 04:09:57,017][105692] Updated weights for policy 0, policy_version 1753066 (0.0010) [2023-12-27 04:09:57,068][105692] Updated weights for policy 0, policy_version 1753076 (0.0010) [2023-12-27 04:09:57,112][105692] Updated weights for policy 0, policy_version 1753086 (0.0010) [2023-12-27 04:09:57,321][105620] Updated weights for policy 1, policy_version 1756696 (0.0008) [2023-12-27 04:09:57,373][105620] Updated weights for policy 1, policy_version 1756706 (0.0008) [2023-12-27 04:09:57,420][105620] Updated weights for policy 1, policy_version 1756716 (0.0008) [2023-12-27 04:09:57,852][105692] Updated weights for policy 0, policy_version 1753096 (0.0010) [2023-12-27 04:09:57,896][105692] Updated weights for policy 0, policy_version 1753106 (0.0010) [2023-12-27 04:09:57,946][105692] Updated weights for policy 0, policy_version 1753116 (0.0010) [2023-12-27 04:09:58,184][105620] Updated weights for policy 1, policy_version 1756726 (0.0007) [2023-12-27 04:09:58,250][105620] Updated weights for policy 1, policy_version 1756736 (0.0008) [2023-12-27 04:09:58,307][105620] Updated weights for policy 1, policy_version 1756746 (0.0008) [2023-12-27 04:09:58,762][105692] Updated weights for policy 0, policy_version 1753126 (0.0009) [2023-12-27 04:09:58,826][105692] Updated weights for policy 0, policy_version 1753136 (0.0012) [2023-12-27 04:09:58,894][105692] Updated weights for policy 0, policy_version 1753146 (0.0010) [2023-12-27 04:09:59,169][105620] Updated weights for policy 1, policy_version 1756756 (0.0008) [2023-12-27 04:09:59,238][105620] Updated weights for policy 1, policy_version 1756766 (0.0010) [2023-12-27 04:09:59,299][105620] Updated weights for policy 1, policy_version 1756776 (0.0010) [2023-12-27 04:09:59,711][105692] Updated weights for policy 0, policy_version 1753156 (0.0011) [2023-12-27 04:09:59,769][105692] Updated weights for policy 0, policy_version 1753166 (0.0011) [2023-12-27 04:09:59,830][105692] Updated weights for policy 0, policy_version 1753176 (0.0011) [2023-12-27 04:09:59,928][105620] Updated weights for policy 1, policy_version 1756786 (0.0009) [2023-12-27 04:09:59,980][105620] Updated weights for policy 1, policy_version 1756796 (0.0009) [2023-12-27 04:10:00,041][105620] Updated weights for policy 1, policy_version 1756806 (0.0006) [2023-12-27 04:10:00,106][105620] Updated weights for policy 1, policy_version 1756816 (0.0007) [2023-12-27 04:10:00,536][105692] Updated weights for policy 0, policy_version 1753186 (0.0010) [2023-12-27 04:10:00,583][105692] Updated weights for policy 0, policy_version 1753196 (0.0010) [2023-12-27 04:10:00,635][105692] Updated weights for policy 0, policy_version 1753206 (0.0008) [2023-12-27 04:10:00,682][105692] Updated weights for policy 0, policy_version 1753216 (0.0008) [2023-12-27 04:10:00,718][105620] Updated weights for policy 1, policy_version 1756826 (0.0005) [2023-12-27 04:10:00,776][105620] Updated weights for policy 1, policy_version 1756836 (0.0005) [2023-12-27 04:10:00,834][105620] Updated weights for policy 1, policy_version 1756846 (0.0005) [2023-12-27 04:10:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 898703360. Throughput: 0: 10078.7, 1: 9580.9. Samples: 898671304. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:10:01,063][104569] Avg episode reward: [(0, '8531.629'), (1, '9168.888')] [2023-12-27 04:10:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001756848_449814528.pth... [2023-12-27 04:10:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001753216_448888832.pth... [2023-12-27 04:10:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001752064_448593920.pth [2023-12-27 04:10:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001755728_449527808.pth [2023-12-27 04:10:01,446][105692] Updated weights for policy 0, policy_version 1753226 (0.0010) [2023-12-27 04:10:01,491][105692] Updated weights for policy 0, policy_version 1753236 (0.0010) [2023-12-27 04:10:01,524][105620] Updated weights for policy 1, policy_version 1756856 (0.0007) [2023-12-27 04:10:01,550][105692] Updated weights for policy 0, policy_version 1753246 (0.0006) [2023-12-27 04:10:01,582][105620] Updated weights for policy 1, policy_version 1756866 (0.0009) [2023-12-27 04:10:01,638][105620] Updated weights for policy 1, policy_version 1756876 (0.0009) [2023-12-27 04:10:02,176][105692] Updated weights for policy 0, policy_version 1753256 (0.0010) [2023-12-27 04:10:02,235][105692] Updated weights for policy 0, policy_version 1753266 (0.0011) [2023-12-27 04:10:02,301][105692] Updated weights for policy 0, policy_version 1753276 (0.0007) [2023-12-27 04:10:02,456][105620] Updated weights for policy 1, policy_version 1756886 (0.0008) [2023-12-27 04:10:02,518][105620] Updated weights for policy 1, policy_version 1756896 (0.0008) [2023-12-27 04:10:02,580][105620] Updated weights for policy 1, policy_version 1756906 (0.0008) [2023-12-27 04:10:03,047][105692] Updated weights for policy 0, policy_version 1753286 (0.0010) [2023-12-27 04:10:03,104][105692] Updated weights for policy 0, policy_version 1753296 (0.0010) [2023-12-27 04:10:03,155][105692] Updated weights for policy 0, policy_version 1753306 (0.0010) [2023-12-27 04:10:03,323][105620] Updated weights for policy 1, policy_version 1756916 (0.0008) [2023-12-27 04:10:03,372][105620] Updated weights for policy 1, policy_version 1756926 (0.0006) [2023-12-27 04:10:03,435][105620] Updated weights for policy 1, policy_version 1756936 (0.0008) [2023-12-27 04:10:03,781][105692] Updated weights for policy 0, policy_version 1753316 (0.0008) [2023-12-27 04:10:03,849][105692] Updated weights for policy 0, policy_version 1753326 (0.0006) [2023-12-27 04:10:03,911][105692] Updated weights for policy 0, policy_version 1753336 (0.0006) [2023-12-27 04:10:04,173][105620] Updated weights for policy 1, policy_version 1756946 (0.0010) [2023-12-27 04:10:04,240][105620] Updated weights for policy 1, policy_version 1756956 (0.0006) [2023-12-27 04:10:04,309][105620] Updated weights for policy 1, policy_version 1756966 (0.0006) [2023-12-27 04:10:04,378][105620] Updated weights for policy 1, policy_version 1756976 (0.0008) [2023-12-27 04:10:04,582][105692] Updated weights for policy 0, policy_version 1753346 (0.0008) [2023-12-27 04:10:04,634][105692] Updated weights for policy 0, policy_version 1753356 (0.0011) [2023-12-27 04:10:04,694][105692] Updated weights for policy 0, policy_version 1753366 (0.0009) [2023-12-27 04:10:04,742][105692] Updated weights for policy 0, policy_version 1753376 (0.0007) [2023-12-27 04:10:05,036][105620] Updated weights for policy 1, policy_version 1756986 (0.0008) [2023-12-27 04:10:05,107][105620] Updated weights for policy 1, policy_version 1756996 (0.0009) [2023-12-27 04:10:05,163][105620] Updated weights for policy 1, policy_version 1757006 (0.0008) [2023-12-27 04:10:05,447][105692] Updated weights for policy 0, policy_version 1753386 (0.0011) [2023-12-27 04:10:05,498][105692] Updated weights for policy 0, policy_version 1753396 (0.0010) [2023-12-27 04:10:05,556][105692] Updated weights for policy 0, policy_version 1753406 (0.0010) [2023-12-27 04:10:05,752][105620] Updated weights for policy 1, policy_version 1757016 (0.0005) [2023-12-27 04:10:05,803][105620] Updated weights for policy 1, policy_version 1757026 (0.0006) [2023-12-27 04:10:05,832][105586] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000006 [2023-12-27 04:10:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 898801664. Throughput: 0: 10094.1, 1: 9606.1. Samples: 898788848. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:10:06,062][104569] Avg episode reward: [(0, '8255.553'), (1, '9171.662')] [2023-12-27 04:10:06,301][105692] Updated weights for policy 0, policy_version 1753416 (0.0011) [2023-12-27 04:10:06,360][105692] Updated weights for policy 0, policy_version 1753426 (0.0010) [2023-12-27 04:10:06,419][105692] Updated weights for policy 0, policy_version 1753436 (0.0010) [2023-12-27 04:10:06,465][105620] Updated weights for policy 1, policy_version 1757036 (0.0006) [2023-12-27 04:10:06,526][105620] Updated weights for policy 1, policy_version 1757046 (0.0008) [2023-12-27 04:10:06,562][105586] KL-divergence is very high: 103.4421 [2023-12-27 04:10:06,582][105620] Updated weights for policy 1, policy_version 1757056 (0.0008) [2023-12-27 04:10:06,601][105586] KL-divergence is very high: 108.2453 [2023-12-27 04:10:07,169][105692] Updated weights for policy 0, policy_version 1753446 (0.0010) [2023-12-27 04:10:07,217][105692] Updated weights for policy 0, policy_version 1753456 (0.0010) [2023-12-27 04:10:07,276][105692] Updated weights for policy 0, policy_version 1753466 (0.0010) [2023-12-27 04:10:07,355][105620] Updated weights for policy 1, policy_version 1757066 (0.0008) [2023-12-27 04:10:07,411][105620] Updated weights for policy 1, policy_version 1757076 (0.0008) [2023-12-27 04:10:07,469][105620] Updated weights for policy 1, policy_version 1757086 (0.0009) [2023-12-27 04:10:07,532][105620] Updated weights for policy 1, policy_version 1757096 (0.0008) [2023-12-27 04:10:08,063][105692] Updated weights for policy 0, policy_version 1753476 (0.0008) [2023-12-27 04:10:08,131][105692] Updated weights for policy 0, policy_version 1753486 (0.0006) [2023-12-27 04:10:08,184][105620] Updated weights for policy 1, policy_version 1757106 (0.0006) [2023-12-27 04:10:08,202][105692] Updated weights for policy 0, policy_version 1753496 (0.0006) [2023-12-27 04:10:08,247][105620] Updated weights for policy 1, policy_version 1757116 (0.0007) [2023-12-27 04:10:08,306][105620] Updated weights for policy 1, policy_version 1757126 (0.0006) [2023-12-27 04:10:08,863][105692] Updated weights for policy 0, policy_version 1753506 (0.0009) [2023-12-27 04:10:08,921][105692] Updated weights for policy 0, policy_version 1753516 (0.0009) [2023-12-27 04:10:08,942][105620] Updated weights for policy 1, policy_version 1757136 (0.0010) [2023-12-27 04:10:08,983][105692] Updated weights for policy 0, policy_version 1753526 (0.0008) [2023-12-27 04:10:09,010][105620] Updated weights for policy 1, policy_version 1757146 (0.0009) [2023-12-27 04:10:09,045][105692] Updated weights for policy 0, policy_version 1753536 (0.0008) [2023-12-27 04:10:09,067][105620] Updated weights for policy 1, policy_version 1757156 (0.0008) [2023-12-27 04:10:09,783][105620] Updated weights for policy 1, policy_version 1757166 (0.0007) [2023-12-27 04:10:09,849][105620] Updated weights for policy 1, policy_version 1757176 (0.0007) [2023-12-27 04:10:09,856][105692] Updated weights for policy 0, policy_version 1753546 (0.0007) [2023-12-27 04:10:09,911][105620] Updated weights for policy 1, policy_version 1757186 (0.0007) [2023-12-27 04:10:09,917][105692] Updated weights for policy 0, policy_version 1753556 (0.0007) [2023-12-27 04:10:09,970][105692] Updated weights for policy 0, policy_version 1753566 (0.0009) [2023-12-27 04:10:10,642][105620] Updated weights for policy 1, policy_version 1757196 (0.0009) [2023-12-27 04:10:10,698][105620] Updated weights for policy 1, policy_version 1757206 (0.0011) [2023-12-27 04:10:10,740][105692] Updated weights for policy 0, policy_version 1753576 (0.0007) [2023-12-27 04:10:10,758][105620] Updated weights for policy 1, policy_version 1757216 (0.0011) [2023-12-27 04:10:10,795][105692] Updated weights for policy 0, policy_version 1753586 (0.0006) [2023-12-27 04:10:10,850][105692] Updated weights for policy 0, policy_version 1753596 (0.0008) [2023-12-27 04:10:11,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 898899968. Throughput: 0: 10004.1, 1: 9720.3. Samples: 898905500. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:10:11,063][104569] Avg episode reward: [(0, '8346.398'), (1, '9171.990')] [2023-12-27 04:10:11,547][105620] Updated weights for policy 1, policy_version 1757226 (0.0010) [2023-12-27 04:10:11,611][105620] Updated weights for policy 1, policy_version 1757236 (0.0008) [2023-12-27 04:10:11,640][105692] Updated weights for policy 0, policy_version 1753606 (0.0008) [2023-12-27 04:10:11,677][105620] Updated weights for policy 1, policy_version 1757246 (0.0009) [2023-12-27 04:10:11,697][105692] Updated weights for policy 0, policy_version 1753616 (0.0008) [2023-12-27 04:10:11,748][105620] Updated weights for policy 1, policy_version 1757256 (0.0009) [2023-12-27 04:10:11,769][105692] Updated weights for policy 0, policy_version 1753626 (0.0007) [2023-12-27 04:10:12,339][105620] Updated weights for policy 1, policy_version 1757266 (0.0008) [2023-12-27 04:10:12,366][105692] Updated weights for policy 0, policy_version 1753636 (0.0007) [2023-12-27 04:10:12,405][105620] Updated weights for policy 1, policy_version 1757276 (0.0008) [2023-12-27 04:10:12,431][105692] Updated weights for policy 0, policy_version 1753646 (0.0006) [2023-12-27 04:10:12,473][105620] Updated weights for policy 1, policy_version 1757286 (0.0008) [2023-12-27 04:10:12,486][105692] Updated weights for policy 0, policy_version 1753656 (0.0007) [2023-12-27 04:10:13,100][105692] Updated weights for policy 0, policy_version 1753666 (0.0010) [2023-12-27 04:10:13,150][105692] Updated weights for policy 0, policy_version 1753676 (0.0008) [2023-12-27 04:10:13,195][105692] Updated weights for policy 0, policy_version 1753686 (0.0009) [2023-12-27 04:10:13,236][105620] Updated weights for policy 1, policy_version 1757296 (0.0007) [2023-12-27 04:10:13,246][105692] Updated weights for policy 0, policy_version 1753696 (0.0008) [2023-12-27 04:10:13,281][105620] Updated weights for policy 1, policy_version 1757306 (0.0008) [2023-12-27 04:10:13,336][105620] Updated weights for policy 1, policy_version 1757316 (0.0009) [2023-12-27 04:10:13,943][105692] Updated weights for policy 0, policy_version 1753706 (0.0005) [2023-12-27 04:10:13,999][105692] Updated weights for policy 0, policy_version 1753716 (0.0006) [2023-12-27 04:10:14,059][105692] Updated weights for policy 0, policy_version 1753726 (0.0006) [2023-12-27 04:10:14,174][105620] Updated weights for policy 1, policy_version 1757326 (0.0009) [2023-12-27 04:10:14,228][105620] Updated weights for policy 1, policy_version 1757336 (0.0008) [2023-12-27 04:10:14,280][105620] Updated weights for policy 1, policy_version 1757347 (0.0008) [2023-12-27 04:10:14,689][105692] Updated weights for policy 0, policy_version 1753736 (0.0006) [2023-12-27 04:10:14,742][105692] Updated weights for policy 0, policy_version 1753746 (0.0006) [2023-12-27 04:10:14,804][105692] Updated weights for policy 0, policy_version 1753756 (0.0010) [2023-12-27 04:10:15,041][105620] Updated weights for policy 1, policy_version 1757357 (0.0007) [2023-12-27 04:10:15,100][105620] Updated weights for policy 1, policy_version 1757367 (0.0009) [2023-12-27 04:10:15,161][105620] Updated weights for policy 1, policy_version 1757377 (0.0009) [2023-12-27 04:10:15,426][105692] Updated weights for policy 0, policy_version 1753766 (0.0006) [2023-12-27 04:10:15,477][105692] Updated weights for policy 0, policy_version 1753776 (0.0005) [2023-12-27 04:10:15,533][105692] Updated weights for policy 0, policy_version 1753786 (0.0005) [2023-12-27 04:10:15,996][105620] Updated weights for policy 1, policy_version 1757387 (0.0010) [2023-12-27 04:10:16,044][105620] Updated weights for policy 1, policy_version 1757397 (0.0010) [2023-12-27 04:10:16,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 898990080. Throughput: 0: 9950.6, 1: 9714.9. Samples: 898964716. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:10:16,063][104569] Avg episode reward: [(0, '8352.741'), (1, '9171.956')] [2023-12-27 04:10:16,088][105692] Updated weights for policy 0, policy_version 1753796 (0.0007) [2023-12-27 04:10:16,100][105620] Updated weights for policy 1, policy_version 1757407 (0.0010) [2023-12-27 04:10:16,141][105692] Updated weights for policy 0, policy_version 1753806 (0.0010) [2023-12-27 04:10:16,158][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001757416_449961984.pth... [2023-12-27 04:10:16,162][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001756304_449675264.pth [2023-12-27 04:10:16,192][105692] Updated weights for policy 0, policy_version 1753816 (0.0010) [2023-12-27 04:10:16,228][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001753824_449044480.pth... [2023-12-27 04:10:16,231][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001752608_448733184.pth [2023-12-27 04:10:16,813][105692] Updated weights for policy 0, policy_version 1753826 (0.0009) [2023-12-27 04:10:16,864][105620] Updated weights for policy 1, policy_version 1757417 (0.0010) [2023-12-27 04:10:16,874][105692] Updated weights for policy 0, policy_version 1753836 (0.0009) [2023-12-27 04:10:16,913][105620] Updated weights for policy 1, policy_version 1757427 (0.0006) [2023-12-27 04:10:16,932][105692] Updated weights for policy 0, policy_version 1753846 (0.0010) [2023-12-27 04:10:16,964][105620] Updated weights for policy 1, policy_version 1757437 (0.0006) [2023-12-27 04:10:16,987][105692] Updated weights for policy 0, policy_version 1753856 (0.0009) [2023-12-27 04:10:17,018][105620] Updated weights for policy 1, policy_version 1757447 (0.0005) [2023-12-27 04:10:17,602][105620] Updated weights for policy 1, policy_version 1757457 (0.0007) [2023-12-27 04:10:17,611][105692] Updated weights for policy 0, policy_version 1753866 (0.0007) [2023-12-27 04:10:17,662][105620] Updated weights for policy 1, policy_version 1757467 (0.0008) [2023-12-27 04:10:17,664][105692] Updated weights for policy 0, policy_version 1753876 (0.0005) [2023-12-27 04:10:17,717][105692] Updated weights for policy 0, policy_version 1753886 (0.0006) [2023-12-27 04:10:17,718][105620] Updated weights for policy 1, policy_version 1757477 (0.0009) [2023-12-27 04:10:18,281][105692] Updated weights for policy 0, policy_version 1753896 (0.0009) [2023-12-27 04:10:18,335][105620] Updated weights for policy 1, policy_version 1757487 (0.0008) [2023-12-27 04:10:18,349][105692] Updated weights for policy 0, policy_version 1753906 (0.0009) [2023-12-27 04:10:18,400][105620] Updated weights for policy 1, policy_version 1757497 (0.0006) [2023-12-27 04:10:18,405][105692] Updated weights for policy 0, policy_version 1753916 (0.0011) [2023-12-27 04:10:18,458][105620] Updated weights for policy 1, policy_version 1757507 (0.0007) [2023-12-27 04:10:19,147][105620] Updated weights for policy 1, policy_version 1757517 (0.0009) [2023-12-27 04:10:19,170][105692] Updated weights for policy 0, policy_version 1753926 (0.0008) [2023-12-27 04:10:19,201][105620] Updated weights for policy 1, policy_version 1757527 (0.0007) [2023-12-27 04:10:19,225][105692] Updated weights for policy 0, policy_version 1753936 (0.0009) [2023-12-27 04:10:19,270][105620] Updated weights for policy 1, policy_version 1757537 (0.0008) [2023-12-27 04:10:19,286][105692] Updated weights for policy 0, policy_version 1753946 (0.0007) [2023-12-27 04:10:20,071][105620] Updated weights for policy 1, policy_version 1757547 (0.0009) [2023-12-27 04:10:20,079][105692] Updated weights for policy 0, policy_version 1753956 (0.0007) [2023-12-27 04:10:20,128][105692] Updated weights for policy 0, policy_version 1753966 (0.0008) [2023-12-27 04:10:20,137][105620] Updated weights for policy 1, policy_version 1757557 (0.0011) [2023-12-27 04:10:20,188][105692] Updated weights for policy 0, policy_version 1753976 (0.0006) [2023-12-27 04:10:20,201][105620] Updated weights for policy 1, policy_version 1757567 (0.0011) [2023-12-27 04:10:20,929][105620] Updated weights for policy 1, policy_version 1757577 (0.0011) [2023-12-27 04:10:20,951][105692] Updated weights for policy 0, policy_version 1753986 (0.0010) [2023-12-27 04:10:20,988][105620] Updated weights for policy 1, policy_version 1757587 (0.0011) [2023-12-27 04:10:21,008][105692] Updated weights for policy 0, policy_version 1753996 (0.0011) [2023-12-27 04:10:21,054][105620] Updated weights for policy 1, policy_version 1757597 (0.0010) [2023-12-27 04:10:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 899088384. Throughput: 0: 10067.2, 1: 9632.9. Samples: 899087472. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:10:21,063][104569] Avg episode reward: [(0, '8444.546'), (1, '9261.389')] [2023-12-27 04:10:21,074][105692] Updated weights for policy 0, policy_version 1754006 (0.0011) [2023-12-27 04:10:21,120][105620] Updated weights for policy 1, policy_version 1757607 (0.0009) [2023-12-27 04:10:21,143][105692] Updated weights for policy 0, policy_version 1754016 (0.0011) [2023-12-27 04:10:21,822][105620] Updated weights for policy 1, policy_version 1757617 (0.0010) [2023-12-27 04:10:21,841][105692] Updated weights for policy 0, policy_version 1754026 (0.0011) [2023-12-27 04:10:21,878][105620] Updated weights for policy 1, policy_version 1757627 (0.0011) [2023-12-27 04:10:21,898][105692] Updated weights for policy 0, policy_version 1754036 (0.0011) [2023-12-27 04:10:21,938][105620] Updated weights for policy 1, policy_version 1757637 (0.0011) [2023-12-27 04:10:21,958][105692] Updated weights for policy 0, policy_version 1754046 (0.0011) [2023-12-27 04:10:22,698][105692] Updated weights for policy 0, policy_version 1754056 (0.0008) [2023-12-27 04:10:22,698][105620] Updated weights for policy 1, policy_version 1757647 (0.0010) [2023-12-27 04:10:22,755][105692] Updated weights for policy 0, policy_version 1754066 (0.0008) [2023-12-27 04:10:22,757][105620] Updated weights for policy 1, policy_version 1757657 (0.0008) [2023-12-27 04:10:22,817][105692] Updated weights for policy 0, policy_version 1754076 (0.0007) [2023-12-27 04:10:22,819][105620] Updated weights for policy 1, policy_version 1757667 (0.0006) [2023-12-27 04:10:23,516][105620] Updated weights for policy 1, policy_version 1757677 (0.0008) [2023-12-27 04:10:23,558][105692] Updated weights for policy 0, policy_version 1754086 (0.0008) [2023-12-27 04:10:23,572][105620] Updated weights for policy 1, policy_version 1757687 (0.0006) [2023-12-27 04:10:23,610][105692] Updated weights for policy 0, policy_version 1754096 (0.0008) [2023-12-27 04:10:23,628][105620] Updated weights for policy 1, policy_version 1757697 (0.0007) [2023-12-27 04:10:23,662][105692] Updated weights for policy 0, policy_version 1754106 (0.0007) [2023-12-27 04:10:24,335][105620] Updated weights for policy 1, policy_version 1757707 (0.0006) [2023-12-27 04:10:24,379][105620] Updated weights for policy 1, policy_version 1757717 (0.0007) [2023-12-27 04:10:24,428][105692] Updated weights for policy 0, policy_version 1754116 (0.0008) [2023-12-27 04:10:24,438][105620] Updated weights for policy 1, policy_version 1757727 (0.0008) [2023-12-27 04:10:24,486][105692] Updated weights for policy 0, policy_version 1754126 (0.0007) [2023-12-27 04:10:24,537][105692] Updated weights for policy 0, policy_version 1754136 (0.0009) [2023-12-27 04:10:25,165][105620] Updated weights for policy 1, policy_version 1757737 (0.0009) [2023-12-27 04:10:25,223][105620] Updated weights for policy 1, policy_version 1757747 (0.0009) [2023-12-27 04:10:25,281][105692] Updated weights for policy 0, policy_version 1754146 (0.0008) [2023-12-27 04:10:25,283][105620] Updated weights for policy 1, policy_version 1757757 (0.0008) [2023-12-27 04:10:25,337][105620] Updated weights for policy 1, policy_version 1757767 (0.0007) [2023-12-27 04:10:25,339][105692] Updated weights for policy 0, policy_version 1754156 (0.0008) [2023-12-27 04:10:25,395][105692] Updated weights for policy 0, policy_version 1754166 (0.0009) [2023-12-27 04:10:25,452][105692] Updated weights for policy 0, policy_version 1754176 (0.0009) [2023-12-27 04:10:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 899186688. Throughput: 0: 10100.0, 1: 9522.1. Samples: 899200480. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:10:26,063][104569] Avg episode reward: [(0, '8075.936'), (1, '9169.054')] [2023-12-27 04:10:26,082][105620] Updated weights for policy 1, policy_version 1757777 (0.0008) [2023-12-27 04:10:26,138][105620] Updated weights for policy 1, policy_version 1757787 (0.0008) [2023-12-27 04:10:26,198][105620] Updated weights for policy 1, policy_version 1757797 (0.0007) [2023-12-27 04:10:26,200][105692] Updated weights for policy 0, policy_version 1754186 (0.0008) [2023-12-27 04:10:26,262][105692] Updated weights for policy 0, policy_version 1754196 (0.0008) [2023-12-27 04:10:26,323][105692] Updated weights for policy 0, policy_version 1754206 (0.0009) [2023-12-27 04:10:26,924][105620] Updated weights for policy 1, policy_version 1757807 (0.0008) [2023-12-27 04:10:26,982][105620] Updated weights for policy 1, policy_version 1757817 (0.0006) [2023-12-27 04:10:27,036][105620] Updated weights for policy 1, policy_version 1757827 (0.0008) [2023-12-27 04:10:27,081][105692] Updated weights for policy 0, policy_version 1754216 (0.0008) [2023-12-27 04:10:27,147][105692] Updated weights for policy 0, policy_version 1754226 (0.0006) [2023-12-27 04:10:27,207][105692] Updated weights for policy 0, policy_version 1754236 (0.0005) [2023-12-27 04:10:27,636][105620] Updated weights for policy 1, policy_version 1757837 (0.0007) [2023-12-27 04:10:27,688][105620] Updated weights for policy 1, policy_version 1757847 (0.0009) [2023-12-27 04:10:27,740][105620] Updated weights for policy 1, policy_version 1757857 (0.0008) [2023-12-27 04:10:27,922][105692] Updated weights for policy 0, policy_version 1754246 (0.0007) [2023-12-27 04:10:27,991][105692] Updated weights for policy 0, policy_version 1754256 (0.0010) [2023-12-27 04:10:28,049][105692] Updated weights for policy 0, policy_version 1754266 (0.0009) [2023-12-27 04:10:28,346][105620] Updated weights for policy 1, policy_version 1757867 (0.0010) [2023-12-27 04:10:28,403][105620] Updated weights for policy 1, policy_version 1757877 (0.0005) [2023-12-27 04:10:28,460][105620] Updated weights for policy 1, policy_version 1757887 (0.0005) [2023-12-27 04:10:28,906][105692] Updated weights for policy 0, policy_version 1754276 (0.0009) [2023-12-27 04:10:28,961][105692] Updated weights for policy 0, policy_version 1754286 (0.0008) [2023-12-27 04:10:29,015][105692] Updated weights for policy 0, policy_version 1754296 (0.0008) [2023-12-27 04:10:29,076][105620] Updated weights for policy 1, policy_version 1757897 (0.0006) [2023-12-27 04:10:29,131][105620] Updated weights for policy 1, policy_version 1757907 (0.0011) [2023-12-27 04:10:29,186][105620] Updated weights for policy 1, policy_version 1757917 (0.0011) [2023-12-27 04:10:29,249][105620] Updated weights for policy 1, policy_version 1757927 (0.0008) [2023-12-27 04:10:29,836][105692] Updated weights for policy 0, policy_version 1754306 (0.0009) [2023-12-27 04:10:29,862][105620] Updated weights for policy 1, policy_version 1757937 (0.0007) [2023-12-27 04:10:29,897][105692] Updated weights for policy 0, policy_version 1754316 (0.0007) [2023-12-27 04:10:29,918][105620] Updated weights for policy 1, policy_version 1757947 (0.0009) [2023-12-27 04:10:29,957][105692] Updated weights for policy 0, policy_version 1754326 (0.0008) [2023-12-27 04:10:29,976][105620] Updated weights for policy 1, policy_version 1757957 (0.0009) [2023-12-27 04:10:30,013][105692] Updated weights for policy 0, policy_version 1754336 (0.0009) [2023-12-27 04:10:30,748][105692] Updated weights for policy 0, policy_version 1754346 (0.0007) [2023-12-27 04:10:30,759][105620] Updated weights for policy 1, policy_version 1757967 (0.0009) [2023-12-27 04:10:30,793][105692] Updated weights for policy 0, policy_version 1754356 (0.0007) [2023-12-27 04:10:30,811][105620] Updated weights for policy 1, policy_version 1757977 (0.0006) [2023-12-27 04:10:30,845][105692] Updated weights for policy 0, policy_version 1754366 (0.0007) [2023-12-27 04:10:30,868][105620] Updated weights for policy 1, policy_version 1757987 (0.0007) [2023-12-27 04:10:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 899293184. Throughput: 0: 10046.5, 1: 9578.2. Samples: 899260312. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:10:31,062][104569] Avg episode reward: [(0, '8168.471'), (1, '9169.998')] [2023-12-27 04:10:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001754368_449183744.pth... [2023-12-27 04:10:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001757992_450109440.pth... [2023-12-27 04:10:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001753216_448888832.pth [2023-12-27 04:10:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001756848_449814528.pth [2023-12-27 04:10:31,625][105692] Updated weights for policy 0, policy_version 1754376 (0.0008) [2023-12-27 04:10:31,645][105620] Updated weights for policy 1, policy_version 1757997 (0.0009) [2023-12-27 04:10:31,685][105692] Updated weights for policy 0, policy_version 1754386 (0.0008) [2023-12-27 04:10:31,704][105620] Updated weights for policy 1, policy_version 1758007 (0.0007) [2023-12-27 04:10:31,750][105692] Updated weights for policy 0, policy_version 1754396 (0.0008) [2023-12-27 04:10:31,767][105620] Updated weights for policy 1, policy_version 1758017 (0.0008) [2023-12-27 04:10:32,463][105620] Updated weights for policy 1, policy_version 1758027 (0.0009) [2023-12-27 04:10:32,469][105692] Updated weights for policy 0, policy_version 1754406 (0.0007) [2023-12-27 04:10:32,523][105620] Updated weights for policy 1, policy_version 1758037 (0.0011) [2023-12-27 04:10:32,532][105692] Updated weights for policy 0, policy_version 1754416 (0.0006) [2023-12-27 04:10:32,581][105620] Updated weights for policy 1, policy_version 1758047 (0.0009) [2023-12-27 04:10:32,590][105692] Updated weights for policy 0, policy_version 1754426 (0.0008) [2023-12-27 04:10:33,152][105620] Updated weights for policy 1, policy_version 1758057 (0.0005) [2023-12-27 04:10:33,209][105620] Updated weights for policy 1, policy_version 1758067 (0.0006) [2023-12-27 04:10:33,266][105620] Updated weights for policy 1, policy_version 1758077 (0.0006) [2023-12-27 04:10:33,316][105620] Updated weights for policy 1, policy_version 1758087 (0.0010) [2023-12-27 04:10:33,445][105692] Updated weights for policy 0, policy_version 1754436 (0.0008) [2023-12-27 04:10:33,502][105692] Updated weights for policy 0, policy_version 1754446 (0.0009) [2023-12-27 04:10:33,552][105692] Updated weights for policy 0, policy_version 1754456 (0.0009) [2023-12-27 04:10:33,892][105620] Updated weights for policy 1, policy_version 1758097 (0.0005) [2023-12-27 04:10:33,951][105620] Updated weights for policy 1, policy_version 1758107 (0.0005) [2023-12-27 04:10:34,023][105620] Updated weights for policy 1, policy_version 1758117 (0.0005) [2023-12-27 04:10:34,401][105692] Updated weights for policy 0, policy_version 1754466 (0.0010) [2023-12-27 04:10:34,451][105692] Updated weights for policy 0, policy_version 1754476 (0.0008) [2023-12-27 04:10:34,503][105692] Updated weights for policy 0, policy_version 1754486 (0.0009) [2023-12-27 04:10:34,562][105692] Updated weights for policy 0, policy_version 1754496 (0.0009) [2023-12-27 04:10:34,595][105620] Updated weights for policy 1, policy_version 1758127 (0.0008) [2023-12-27 04:10:34,657][105620] Updated weights for policy 1, policy_version 1758137 (0.0010) [2023-12-27 04:10:34,708][105620] Updated weights for policy 1, policy_version 1758147 (0.0009) [2023-12-27 04:10:35,221][105692] Updated weights for policy 0, policy_version 1754506 (0.0008) [2023-12-27 04:10:35,266][105692] Updated weights for policy 0, policy_version 1754516 (0.0006) [2023-12-27 04:10:35,314][105692] Updated weights for policy 0, policy_version 1754526 (0.0005) [2023-12-27 04:10:35,385][105620] Updated weights for policy 1, policy_version 1758157 (0.0009) [2023-12-27 04:10:35,437][105620] Updated weights for policy 1, policy_version 1758167 (0.0005) [2023-12-27 04:10:35,502][105620] Updated weights for policy 1, policy_version 1758177 (0.0005) [2023-12-27 04:10:35,970][105692] Updated weights for policy 0, policy_version 1754536 (0.0009) [2023-12-27 04:10:36,022][105692] Updated weights for policy 0, policy_version 1754546 (0.0011) [2023-12-27 04:10:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 899383296. Throughput: 0: 9832.4, 1: 9722.3. Samples: 899375800. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:10:36,062][104569] Avg episode reward: [(0, '8351.112'), (1, '9262.246')] [2023-12-27 04:10:36,078][105692] Updated weights for policy 0, policy_version 1754556 (0.0009) [2023-12-27 04:10:36,200][105620] Updated weights for policy 1, policy_version 1758187 (0.0007) [2023-12-27 04:10:36,264][105620] Updated weights for policy 1, policy_version 1758197 (0.0011) [2023-12-27 04:10:36,324][105620] Updated weights for policy 1, policy_version 1758207 (0.0010) [2023-12-27 04:10:36,826][105692] Updated weights for policy 0, policy_version 1754566 (0.0008) [2023-12-27 04:10:36,885][105692] Updated weights for policy 0, policy_version 1754576 (0.0005) [2023-12-27 04:10:36,941][105692] Updated weights for policy 0, policy_version 1754586 (0.0005) [2023-12-27 04:10:37,005][105620] Updated weights for policy 1, policy_version 1758217 (0.0010) [2023-12-27 04:10:37,063][105620] Updated weights for policy 1, policy_version 1758227 (0.0010) [2023-12-27 04:10:37,118][105620] Updated weights for policy 1, policy_version 1758237 (0.0010) [2023-12-27 04:10:37,170][105620] Updated weights for policy 1, policy_version 1758247 (0.0010) [2023-12-27 04:10:37,517][105692] Updated weights for policy 0, policy_version 1754596 (0.0007) [2023-12-27 04:10:37,573][105692] Updated weights for policy 0, policy_version 1754606 (0.0010) [2023-12-27 04:10:37,629][105692] Updated weights for policy 0, policy_version 1754616 (0.0011) [2023-12-27 04:10:37,930][105620] Updated weights for policy 1, policy_version 1758257 (0.0010) [2023-12-27 04:10:37,992][105620] Updated weights for policy 1, policy_version 1758267 (0.0010) [2023-12-27 04:10:38,055][105620] Updated weights for policy 1, policy_version 1758277 (0.0010) [2023-12-27 04:10:38,312][105692] Updated weights for policy 0, policy_version 1754626 (0.0010) [2023-12-27 04:10:38,379][105692] Updated weights for policy 0, policy_version 1754636 (0.0008) [2023-12-27 04:10:38,438][105692] Updated weights for policy 0, policy_version 1754646 (0.0009) [2023-12-27 04:10:38,484][105692] Updated weights for policy 0, policy_version 1754656 (0.0008) [2023-12-27 04:10:38,748][105620] Updated weights for policy 1, policy_version 1758287 (0.0007) [2023-12-27 04:10:38,805][105620] Updated weights for policy 1, policy_version 1758297 (0.0005) [2023-12-27 04:10:38,864][105620] Updated weights for policy 1, policy_version 1758307 (0.0005) [2023-12-27 04:10:39,344][105692] Updated weights for policy 0, policy_version 1754666 (0.0009) [2023-12-27 04:10:39,412][105692] Updated weights for policy 0, policy_version 1754676 (0.0007) [2023-12-27 04:10:39,449][105620] Updated weights for policy 1, policy_version 1758317 (0.0008) [2023-12-27 04:10:39,472][105692] Updated weights for policy 0, policy_version 1754686 (0.0009) [2023-12-27 04:10:39,508][105620] Updated weights for policy 1, policy_version 1758327 (0.0010) [2023-12-27 04:10:39,567][105620] Updated weights for policy 1, policy_version 1758337 (0.0010) [2023-12-27 04:10:40,203][105692] Updated weights for policy 0, policy_version 1754696 (0.0009) [2023-12-27 04:10:40,230][105620] Updated weights for policy 1, policy_version 1758347 (0.0007) [2023-12-27 04:10:40,256][105692] Updated weights for policy 0, policy_version 1754706 (0.0007) [2023-12-27 04:10:40,286][105620] Updated weights for policy 1, policy_version 1758357 (0.0006) [2023-12-27 04:10:40,317][105692] Updated weights for policy 0, policy_version 1754716 (0.0007) [2023-12-27 04:10:40,345][105620] Updated weights for policy 1, policy_version 1758367 (0.0007) [2023-12-27 04:10:40,979][105620] Updated weights for policy 1, policy_version 1758377 (0.0010) [2023-12-27 04:10:41,032][105620] Updated weights for policy 1, policy_version 1758387 (0.0010) [2023-12-27 04:10:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 899481600. Throughput: 0: 9662.3, 1: 9923.5. Samples: 899496384. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:10:41,063][104569] Avg episode reward: [(0, '8079.424'), (1, '9353.796')] [2023-12-27 04:10:41,096][105620] Updated weights for policy 1, policy_version 1758397 (0.0011) [2023-12-27 04:10:41,165][105620] Updated weights for policy 1, policy_version 1758407 (0.0011) [2023-12-27 04:10:41,166][105692] Updated weights for policy 0, policy_version 1754726 (0.0008) [2023-12-27 04:10:41,226][105692] Updated weights for policy 0, policy_version 1754736 (0.0008) [2023-12-27 04:10:41,290][105692] Updated weights for policy 0, policy_version 1754746 (0.0008) [2023-12-27 04:10:41,962][105620] Updated weights for policy 1, policy_version 1758417 (0.0011) [2023-12-27 04:10:42,014][105620] Updated weights for policy 1, policy_version 1758427 (0.0011) [2023-12-27 04:10:42,067][105692] Updated weights for policy 0, policy_version 1754756 (0.0008) [2023-12-27 04:10:42,071][105620] Updated weights for policy 1, policy_version 1758437 (0.0011) [2023-12-27 04:10:42,125][105692] Updated weights for policy 0, policy_version 1754766 (0.0009) [2023-12-27 04:10:42,181][105692] Updated weights for policy 0, policy_version 1754776 (0.0009) [2023-12-27 04:10:42,731][105620] Updated weights for policy 1, policy_version 1758447 (0.0007) [2023-12-27 04:10:42,786][105620] Updated weights for policy 1, policy_version 1758457 (0.0007) [2023-12-27 04:10:42,843][105620] Updated weights for policy 1, policy_version 1758467 (0.0007) [2023-12-27 04:10:42,955][105692] Updated weights for policy 0, policy_version 1754786 (0.0010) [2023-12-27 04:10:43,018][105692] Updated weights for policy 0, policy_version 1754796 (0.0009) [2023-12-27 04:10:43,076][105692] Updated weights for policy 0, policy_version 1754806 (0.0009) [2023-12-27 04:10:43,138][105692] Updated weights for policy 0, policy_version 1754816 (0.0005) [2023-12-27 04:10:43,590][105620] Updated weights for policy 1, policy_version 1758477 (0.0009) [2023-12-27 04:10:43,641][105620] Updated weights for policy 1, policy_version 1758487 (0.0005) [2023-12-27 04:10:43,700][105692] Updated weights for policy 0, policy_version 1754826 (0.0005) [2023-12-27 04:10:43,714][105620] Updated weights for policy 1, policy_version 1758497 (0.0006) [2023-12-27 04:10:43,756][105692] Updated weights for policy 0, policy_version 1754836 (0.0005) [2023-12-27 04:10:43,808][105692] Updated weights for policy 0, policy_version 1754846 (0.0005) [2023-12-27 04:10:44,343][105692] Updated weights for policy 0, policy_version 1754856 (0.0008) [2023-12-27 04:10:44,409][105692] Updated weights for policy 0, policy_version 1754866 (0.0009) [2023-12-27 04:10:44,457][105692] Updated weights for policy 0, policy_version 1754876 (0.0008) [2023-12-27 04:10:44,480][105620] Updated weights for policy 1, policy_version 1758507 (0.0007) [2023-12-27 04:10:44,540][105620] Updated weights for policy 1, policy_version 1758517 (0.0008) [2023-12-27 04:10:44,592][105620] Updated weights for policy 1, policy_version 1758527 (0.0010) [2023-12-27 04:10:45,192][105692] Updated weights for policy 0, policy_version 1754886 (0.0009) [2023-12-27 04:10:45,254][105692] Updated weights for policy 0, policy_version 1754896 (0.0007) [2023-12-27 04:10:45,318][105692] Updated weights for policy 0, policy_version 1754906 (0.0007) [2023-12-27 04:10:45,391][105620] Updated weights for policy 1, policy_version 1758537 (0.0010) [2023-12-27 04:10:45,454][105620] Updated weights for policy 1, policy_version 1758547 (0.0009) [2023-12-27 04:10:45,516][105620] Updated weights for policy 1, policy_version 1758557 (0.0010) [2023-12-27 04:10:45,579][105620] Updated weights for policy 1, policy_version 1758567 (0.0009) [2023-12-27 04:10:45,918][105692] Updated weights for policy 0, policy_version 1754916 (0.0008) [2023-12-27 04:10:45,980][105692] Updated weights for policy 0, policy_version 1754926 (0.0006) [2023-12-27 04:10:46,040][105692] Updated weights for policy 0, policy_version 1754936 (0.0006) [2023-12-27 04:10:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 899579904. Throughput: 0: 9638.3, 1: 9965.3. Samples: 899553468. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:10:46,063][104569] Avg episode reward: [(0, '8627.413'), (1, '9353.775')] [2023-12-27 04:10:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001758568_450256896.pth... [2023-12-27 04:10:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001757416_449961984.pth [2023-12-27 04:10:46,085][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001754944_449331200.pth... [2023-12-27 04:10:46,089][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001753824_449044480.pth [2023-12-27 04:10:46,436][105620] Updated weights for policy 1, policy_version 1758577 (0.0006) [2023-12-27 04:10:46,488][105620] Updated weights for policy 1, policy_version 1758587 (0.0006) [2023-12-27 04:10:46,548][105620] Updated weights for policy 1, policy_version 1758597 (0.0008) [2023-12-27 04:10:46,570][105692] Updated weights for policy 0, policy_version 1754946 (0.0006) [2023-12-27 04:10:46,636][105692] Updated weights for policy 0, policy_version 1754956 (0.0005) [2023-12-27 04:10:46,692][105692] Updated weights for policy 0, policy_version 1754966 (0.0005) [2023-12-27 04:10:46,739][105692] Updated weights for policy 0, policy_version 1754976 (0.0005) [2023-12-27 04:10:47,200][105620] Updated weights for policy 1, policy_version 1758607 (0.0009) [2023-12-27 04:10:47,246][105620] Updated weights for policy 1, policy_version 1758617 (0.0006) [2023-12-27 04:10:47,251][105692] Updated weights for policy 0, policy_version 1754986 (0.0005) [2023-12-27 04:10:47,295][105620] Updated weights for policy 1, policy_version 1758627 (0.0005) [2023-12-27 04:10:47,308][105692] Updated weights for policy 0, policy_version 1754996 (0.0006) [2023-12-27 04:10:47,359][105692] Updated weights for policy 0, policy_version 1755006 (0.0006) [2023-12-27 04:10:47,903][105620] Updated weights for policy 1, policy_version 1758637 (0.0007) [2023-12-27 04:10:47,956][105620] Updated weights for policy 1, policy_version 1758647 (0.0005) [2023-12-27 04:10:48,008][105620] Updated weights for policy 1, policy_version 1758657 (0.0007) [2023-12-27 04:10:48,180][105692] Updated weights for policy 0, policy_version 1755016 (0.0009) [2023-12-27 04:10:48,252][105692] Updated weights for policy 0, policy_version 1755026 (0.0006) [2023-12-27 04:10:48,316][105692] Updated weights for policy 0, policy_version 1755036 (0.0005) [2023-12-27 04:10:48,583][105620] Updated weights for policy 1, policy_version 1758667 (0.0008) [2023-12-27 04:10:48,635][105620] Updated weights for policy 1, policy_version 1758677 (0.0009) [2023-12-27 04:10:48,691][105620] Updated weights for policy 1, policy_version 1758687 (0.0009) [2023-12-27 04:10:49,077][105692] Updated weights for policy 0, policy_version 1755046 (0.0009) [2023-12-27 04:10:49,134][105692] Updated weights for policy 0, policy_version 1755056 (0.0008) [2023-12-27 04:10:49,189][105692] Updated weights for policy 0, policy_version 1755066 (0.0009) [2023-12-27 04:10:49,435][105620] Updated weights for policy 1, policy_version 1758697 (0.0008) [2023-12-27 04:10:49,489][105620] Updated weights for policy 1, policy_version 1758707 (0.0005) [2023-12-27 04:10:49,546][105620] Updated weights for policy 1, policy_version 1758717 (0.0005) [2023-12-27 04:10:49,598][105620] Updated weights for policy 1, policy_version 1758727 (0.0007) [2023-12-27 04:10:49,974][105692] Updated weights for policy 0, policy_version 1755076 (0.0008) [2023-12-27 04:10:50,030][105692] Updated weights for policy 0, policy_version 1755086 (0.0009) [2023-12-27 04:10:50,088][105692] Updated weights for policy 0, policy_version 1755096 (0.0010) [2023-12-27 04:10:50,232][105620] Updated weights for policy 1, policy_version 1758737 (0.0006) [2023-12-27 04:10:50,285][105620] Updated weights for policy 1, policy_version 1758747 (0.0006) [2023-12-27 04:10:50,344][105620] Updated weights for policy 1, policy_version 1758757 (0.0006) [2023-12-27 04:10:50,810][105692] Updated weights for policy 0, policy_version 1755106 (0.0009) [2023-12-27 04:10:50,874][105692] Updated weights for policy 0, policy_version 1755116 (0.0009) [2023-12-27 04:10:50,930][105692] Updated weights for policy 0, policy_version 1755126 (0.0008) [2023-12-27 04:10:50,952][105620] Updated weights for policy 1, policy_version 1758767 (0.0008) [2023-12-27 04:10:50,989][105692] Updated weights for policy 0, policy_version 1755136 (0.0008) [2023-12-27 04:10:51,005][105620] Updated weights for policy 1, policy_version 1758777 (0.0007) [2023-12-27 04:10:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19522.0). Total num frames: 899686400. Throughput: 0: 9738.8, 1: 9967.1. Samples: 899675612. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:10:51,062][104569] Avg episode reward: [(0, '9080.160'), (1, '9261.367')] [2023-12-27 04:10:51,070][105620] Updated weights for policy 1, policy_version 1758787 (0.0009) [2023-12-27 04:10:51,738][105692] Updated weights for policy 0, policy_version 1755146 (0.0009) [2023-12-27 04:10:51,803][105692] Updated weights for policy 0, policy_version 1755156 (0.0006) [2023-12-27 04:10:51,813][105620] Updated weights for policy 1, policy_version 1758797 (0.0008) [2023-12-27 04:10:51,867][105692] Updated weights for policy 0, policy_version 1755166 (0.0007) [2023-12-27 04:10:51,869][105620] Updated weights for policy 1, policy_version 1758807 (0.0007) [2023-12-27 04:10:51,930][105620] Updated weights for policy 1, policy_version 1758817 (0.0009) [2023-12-27 04:10:52,621][105692] Updated weights for policy 0, policy_version 1755176 (0.0008) [2023-12-27 04:10:52,642][105620] Updated weights for policy 1, policy_version 1758827 (0.0008) [2023-12-27 04:10:52,678][105692] Updated weights for policy 0, policy_version 1755186 (0.0007) [2023-12-27 04:10:52,704][105620] Updated weights for policy 1, policy_version 1758837 (0.0007) [2023-12-27 04:10:52,727][105692] Updated weights for policy 0, policy_version 1755196 (0.0007) [2023-12-27 04:10:52,765][105620] Updated weights for policy 1, policy_version 1758847 (0.0009) [2023-12-27 04:10:53,383][105620] Updated weights for policy 1, policy_version 1758857 (0.0008) [2023-12-27 04:10:53,448][105620] Updated weights for policy 1, policy_version 1758867 (0.0005) [2023-12-27 04:10:53,484][105692] Updated weights for policy 0, policy_version 1755206 (0.0006) [2023-12-27 04:10:53,514][105620] Updated weights for policy 1, policy_version 1758877 (0.0005) [2023-12-27 04:10:53,531][105692] Updated weights for policy 0, policy_version 1755216 (0.0005) [2023-12-27 04:10:53,578][105620] Updated weights for policy 1, policy_version 1758887 (0.0005) [2023-12-27 04:10:53,582][105692] Updated weights for policy 0, policy_version 1755226 (0.0006) [2023-12-27 04:10:54,069][105620] Updated weights for policy 1, policy_version 1758897 (0.0005) [2023-12-27 04:10:54,118][105620] Updated weights for policy 1, policy_version 1758907 (0.0005) [2023-12-27 04:10:54,177][105620] Updated weights for policy 1, policy_version 1758917 (0.0005) [2023-12-27 04:10:54,369][105692] Updated weights for policy 0, policy_version 1755236 (0.0008) [2023-12-27 04:10:54,441][105692] Updated weights for policy 0, policy_version 1755246 (0.0005) [2023-12-27 04:10:54,506][105692] Updated weights for policy 0, policy_version 1755256 (0.0005) [2023-12-27 04:10:54,808][105620] Updated weights for policy 1, policy_version 1758927 (0.0008) [2023-12-27 04:10:54,862][105620] Updated weights for policy 1, policy_version 1758937 (0.0009) [2023-12-27 04:10:54,931][105620] Updated weights for policy 1, policy_version 1758947 (0.0006) [2023-12-27 04:10:55,032][105692] Updated weights for policy 0, policy_version 1755266 (0.0005) [2023-12-27 04:10:55,088][105692] Updated weights for policy 0, policy_version 1755276 (0.0005) [2023-12-27 04:10:55,148][105692] Updated weights for policy 0, policy_version 1755286 (0.0006) [2023-12-27 04:10:55,210][105692] Updated weights for policy 0, policy_version 1755296 (0.0006) [2023-12-27 04:10:55,636][105620] Updated weights for policy 1, policy_version 1758957 (0.0008) [2023-12-27 04:10:55,691][105620] Updated weights for policy 1, policy_version 1758967 (0.0008) [2023-12-27 04:10:55,741][105620] Updated weights for policy 1, policy_version 1758977 (0.0009) [2023-12-27 04:10:55,839][105692] Updated weights for policy 0, policy_version 1755306 (0.0005) [2023-12-27 04:10:55,886][105692] Updated weights for policy 0, policy_version 1755316 (0.0006) [2023-12-27 04:10:55,947][105692] Updated weights for policy 0, policy_version 1755326 (0.0006) [2023-12-27 04:10:56,062][104569] Fps is (10 sec: 21299.4, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 899792896. Throughput: 0: 9801.1, 1: 10045.5. Samples: 899798600. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:10:56,062][104569] Avg episode reward: [(0, '8899.275'), (1, '9168.987')] [2023-12-27 04:10:56,377][105620] Updated weights for policy 1, policy_version 1758987 (0.0007) [2023-12-27 04:10:56,435][105620] Updated weights for policy 1, policy_version 1758997 (0.0010) [2023-12-27 04:10:56,493][105620] Updated weights for policy 1, policy_version 1759007 (0.0010) [2023-12-27 04:10:56,589][105692] Updated weights for policy 0, policy_version 1755336 (0.0008) [2023-12-27 04:10:56,646][105692] Updated weights for policy 0, policy_version 1755346 (0.0005) [2023-12-27 04:10:56,711][105692] Updated weights for policy 0, policy_version 1755356 (0.0007) [2023-12-27 04:10:57,096][105620] Updated weights for policy 1, policy_version 1759017 (0.0007) [2023-12-27 04:10:57,148][105620] Updated weights for policy 1, policy_version 1759027 (0.0010) [2023-12-27 04:10:57,192][105620] Updated weights for policy 1, policy_version 1759037 (0.0010) [2023-12-27 04:10:57,240][105620] Updated weights for policy 1, policy_version 1759047 (0.0010) [2023-12-27 04:10:57,259][105692] Updated weights for policy 0, policy_version 1755366 (0.0006) [2023-12-27 04:10:57,319][105692] Updated weights for policy 0, policy_version 1755376 (0.0006) [2023-12-27 04:10:57,379][105692] Updated weights for policy 0, policy_version 1755386 (0.0005) [2023-12-27 04:10:57,911][105620] Updated weights for policy 1, policy_version 1759057 (0.0006) [2023-12-27 04:10:57,964][105620] Updated weights for policy 1, policy_version 1759067 (0.0006) [2023-12-27 04:10:57,981][105692] Updated weights for policy 0, policy_version 1755396 (0.0005) [2023-12-27 04:10:58,015][105620] Updated weights for policy 1, policy_version 1759077 (0.0007) [2023-12-27 04:10:58,049][105692] Updated weights for policy 0, policy_version 1755406 (0.0005) [2023-12-27 04:10:58,112][105692] Updated weights for policy 0, policy_version 1755416 (0.0010) [2023-12-27 04:10:58,692][105620] Updated weights for policy 1, policy_version 1759087 (0.0007) [2023-12-27 04:10:58,763][105620] Updated weights for policy 1, policy_version 1759097 (0.0008) [2023-12-27 04:10:58,827][105620] Updated weights for policy 1, policy_version 1759107 (0.0008) [2023-12-27 04:10:58,829][105692] Updated weights for policy 0, policy_version 1755426 (0.0008) [2023-12-27 04:10:58,896][105692] Updated weights for policy 0, policy_version 1755436 (0.0009) [2023-12-27 04:10:58,959][105692] Updated weights for policy 0, policy_version 1755446 (0.0007) [2023-12-27 04:10:59,015][105692] Updated weights for policy 0, policy_version 1755456 (0.0008) [2023-12-27 04:10:59,578][105620] Updated weights for policy 1, policy_version 1759117 (0.0008) [2023-12-27 04:10:59,615][105692] Updated weights for policy 0, policy_version 1755466 (0.0010) [2023-12-27 04:10:59,629][105620] Updated weights for policy 1, policy_version 1759127 (0.0006) [2023-12-27 04:10:59,659][105692] Updated weights for policy 0, policy_version 1755476 (0.0010) [2023-12-27 04:10:59,681][105620] Updated weights for policy 1, policy_version 1759137 (0.0005) [2023-12-27 04:10:59,710][105692] Updated weights for policy 0, policy_version 1755486 (0.0010) [2023-12-27 04:11:00,363][105692] Updated weights for policy 0, policy_version 1755496 (0.0008) [2023-12-27 04:11:00,427][105692] Updated weights for policy 0, policy_version 1755506 (0.0008) [2023-12-27 04:11:00,486][105692] Updated weights for policy 0, policy_version 1755516 (0.0011) [2023-12-27 04:11:00,488][105620] Updated weights for policy 1, policy_version 1759147 (0.0006) [2023-12-27 04:11:00,547][105620] Updated weights for policy 1, policy_version 1759157 (0.0007) [2023-12-27 04:11:00,606][105620] Updated weights for policy 1, policy_version 1759167 (0.0008) [2023-12-27 04:11:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 899891200. Throughput: 0: 9846.6, 1: 10093.9. Samples: 899862036. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:11:01,062][104569] Avg episode reward: [(0, '8714.745'), (1, '9077.233')] [2023-12-27 04:11:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001755520_449478656.pth... [2023-12-27 04:11:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001759176_450412544.pth... [2023-12-27 04:11:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001754368_449183744.pth [2023-12-27 04:11:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001757992_450109440.pth [2023-12-27 04:11:01,194][105692] Updated weights for policy 0, policy_version 1755526 (0.0010) [2023-12-27 04:11:01,249][105692] Updated weights for policy 0, policy_version 1755536 (0.0010) [2023-12-27 04:11:01,305][105620] Updated weights for policy 1, policy_version 1759177 (0.0007) [2023-12-27 04:11:01,311][105692] Updated weights for policy 0, policy_version 1755546 (0.0009) [2023-12-27 04:11:01,368][105620] Updated weights for policy 1, policy_version 1759187 (0.0007) [2023-12-27 04:11:01,433][105620] Updated weights for policy 1, policy_version 1759197 (0.0006) [2023-12-27 04:11:01,488][105620] Updated weights for policy 1, policy_version 1759207 (0.0006) [2023-12-27 04:11:02,090][105620] Updated weights for policy 1, policy_version 1759217 (0.0009) [2023-12-27 04:11:02,152][105620] Updated weights for policy 1, policy_version 1759227 (0.0009) [2023-12-27 04:11:02,200][105692] Updated weights for policy 0, policy_version 1755556 (0.0007) [2023-12-27 04:11:02,203][105620] Updated weights for policy 1, policy_version 1759237 (0.0007) [2023-12-27 04:11:02,252][105692] Updated weights for policy 0, policy_version 1755566 (0.0009) [2023-12-27 04:11:02,318][105692] Updated weights for policy 0, policy_version 1755577 (0.0011) [2023-12-27 04:11:02,893][105620] Updated weights for policy 1, policy_version 1759247 (0.0008) [2023-12-27 04:11:02,949][105620] Updated weights for policy 1, policy_version 1759257 (0.0009) [2023-12-27 04:11:03,004][105620] Updated weights for policy 1, policy_version 1759267 (0.0009) [2023-12-27 04:11:03,100][105692] Updated weights for policy 0, policy_version 1755587 (0.0009) [2023-12-27 04:11:03,147][105692] Updated weights for policy 0, policy_version 1755597 (0.0009) [2023-12-27 04:11:03,195][105692] Updated weights for policy 0, policy_version 1755607 (0.0009) [2023-12-27 04:11:03,768][105620] Updated weights for policy 1, policy_version 1759277 (0.0009) [2023-12-27 04:11:03,816][105620] Updated weights for policy 1, policy_version 1759287 (0.0009) [2023-12-27 04:11:03,880][105620] Updated weights for policy 1, policy_version 1759297 (0.0009) [2023-12-27 04:11:03,988][105692] Updated weights for policy 0, policy_version 1755617 (0.0009) [2023-12-27 04:11:04,045][105692] Updated weights for policy 0, policy_version 1755627 (0.0009) [2023-12-27 04:11:04,109][105692] Updated weights for policy 0, policy_version 1755637 (0.0010) [2023-12-27 04:11:04,171][105692] Updated weights for policy 0, policy_version 1755647 (0.0009) [2023-12-27 04:11:04,645][105620] Updated weights for policy 1, policy_version 1759307 (0.0008) [2023-12-27 04:11:04,704][105620] Updated weights for policy 1, policy_version 1759317 (0.0009) [2023-12-27 04:11:04,762][105620] Updated weights for policy 1, policy_version 1759327 (0.0007) [2023-12-27 04:11:04,914][105692] Updated weights for policy 0, policy_version 1755657 (0.0008) [2023-12-27 04:11:04,976][105692] Updated weights for policy 0, policy_version 1755667 (0.0006) [2023-12-27 04:11:05,038][105692] Updated weights for policy 0, policy_version 1755677 (0.0006) [2023-12-27 04:11:05,386][105620] Updated weights for policy 1, policy_version 1759337 (0.0007) [2023-12-27 04:11:05,437][105620] Updated weights for policy 1, policy_version 1759347 (0.0007) [2023-12-27 04:11:05,489][105620] Updated weights for policy 1, policy_version 1759357 (0.0005) [2023-12-27 04:11:05,553][105620] Updated weights for policy 1, policy_version 1759367 (0.0005) [2023-12-27 04:11:05,848][105692] Updated weights for policy 0, policy_version 1755687 (0.0010) [2023-12-27 04:11:05,901][105692] Updated weights for policy 0, policy_version 1755697 (0.0010) [2023-12-27 04:11:05,954][105692] Updated weights for policy 0, policy_version 1755709 (0.0009) [2023-12-27 04:11:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 899989504. Throughput: 0: 9672.4, 1: 10092.4. Samples: 899976884. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:11:06,063][104569] Avg episode reward: [(0, '8257.781'), (1, '9169.605')] [2023-12-27 04:11:06,076][105620] Updated weights for policy 1, policy_version 1759377 (0.0008) [2023-12-27 04:11:06,135][105620] Updated weights for policy 1, policy_version 1759387 (0.0012) [2023-12-27 04:11:06,198][105620] Updated weights for policy 1, policy_version 1759397 (0.0011) [2023-12-27 04:11:06,766][105692] Updated weights for policy 0, policy_version 1755719 (0.0007) [2023-12-27 04:11:06,824][105692] Updated weights for policy 0, policy_version 1755729 (0.0006) [2023-12-27 04:11:06,880][105692] Updated weights for policy 0, policy_version 1755739 (0.0008) [2023-12-27 04:11:06,920][105620] Updated weights for policy 1, policy_version 1759407 (0.0011) [2023-12-27 04:11:06,976][105620] Updated weights for policy 1, policy_version 1759417 (0.0010) [2023-12-27 04:11:07,037][105620] Updated weights for policy 1, policy_version 1759427 (0.0011) [2023-12-27 04:11:07,659][105620] Updated weights for policy 1, policy_version 1759437 (0.0008) [2023-12-27 04:11:07,671][105692] Updated weights for policy 0, policy_version 1755749 (0.0009) [2023-12-27 04:11:07,725][105620] Updated weights for policy 1, policy_version 1759447 (0.0006) [2023-12-27 04:11:07,733][105692] Updated weights for policy 0, policy_version 1755759 (0.0008) [2023-12-27 04:11:07,782][105620] Updated weights for policy 1, policy_version 1759457 (0.0007) [2023-12-27 04:11:07,788][105692] Updated weights for policy 0, policy_version 1755769 (0.0010) [2023-12-27 04:11:08,440][105692] Updated weights for policy 0, policy_version 1755779 (0.0009) [2023-12-27 04:11:08,493][105620] Updated weights for policy 1, policy_version 1759467 (0.0006) [2023-12-27 04:11:08,498][105692] Updated weights for policy 0, policy_version 1755789 (0.0006) [2023-12-27 04:11:08,546][105620] Updated weights for policy 1, policy_version 1759477 (0.0009) [2023-12-27 04:11:08,559][105692] Updated weights for policy 0, policy_version 1755799 (0.0005) [2023-12-27 04:11:08,602][105620] Updated weights for policy 1, policy_version 1759487 (0.0007) [2023-12-27 04:11:09,261][105692] Updated weights for policy 0, policy_version 1755809 (0.0009) [2023-12-27 04:11:09,325][105692] Updated weights for policy 0, policy_version 1755819 (0.0011) [2023-12-27 04:11:09,379][105620] Updated weights for policy 1, policy_version 1759497 (0.0007) [2023-12-27 04:11:09,391][105692] Updated weights for policy 0, policy_version 1755829 (0.0010) [2023-12-27 04:11:09,444][105620] Updated weights for policy 1, policy_version 1759507 (0.0008) [2023-12-27 04:11:09,455][105692] Updated weights for policy 0, policy_version 1755839 (0.0011) [2023-12-27 04:11:09,505][105620] Updated weights for policy 1, policy_version 1759517 (0.0008) [2023-12-27 04:11:09,567][105620] Updated weights for policy 1, policy_version 1759527 (0.0008) [2023-12-27 04:11:10,209][105692] Updated weights for policy 0, policy_version 1755849 (0.0009) [2023-12-27 04:11:10,280][105692] Updated weights for policy 0, policy_version 1755859 (0.0009) [2023-12-27 04:11:10,328][105692] Updated weights for policy 0, policy_version 1755869 (0.0009) [2023-12-27 04:11:10,329][105620] Updated weights for policy 1, policy_version 1759537 (0.0007) [2023-12-27 04:11:10,386][105620] Updated weights for policy 1, policy_version 1759547 (0.0007) [2023-12-27 04:11:10,443][105620] Updated weights for policy 1, policy_version 1759557 (0.0008) [2023-12-27 04:11:10,959][105692] Updated weights for policy 0, policy_version 1755879 (0.0008) [2023-12-27 04:11:11,017][105692] Updated weights for policy 0, policy_version 1755889 (0.0009) [2023-12-27 04:11:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 900079616. Throughput: 0: 9680.4, 1: 10147.8. Samples: 900092752. Policy #0 lag: (min: 11.0, avg: 20.5, max: 43.0) [2023-12-27 04:11:11,063][104569] Avg episode reward: [(0, '7984.101'), (1, '9264.092')] [2023-12-27 04:11:11,090][105692] Updated weights for policy 0, policy_version 1755899 (0.0008) [2023-12-27 04:11:11,235][105620] Updated weights for policy 1, policy_version 1759567 (0.0010) [2023-12-27 04:11:11,306][105620] Updated weights for policy 1, policy_version 1759577 (0.0007) [2023-12-27 04:11:11,370][105620] Updated weights for policy 1, policy_version 1759587 (0.0007) [2023-12-27 04:11:11,864][105692] Updated weights for policy 0, policy_version 1755909 (0.0008) [2023-12-27 04:11:11,930][105692] Updated weights for policy 0, policy_version 1755919 (0.0009) [2023-12-27 04:11:11,996][105692] Updated weights for policy 0, policy_version 1755929 (0.0009) [2023-12-27 04:11:12,042][105620] Updated weights for policy 1, policy_version 1759597 (0.0009) [2023-12-27 04:11:12,111][105620] Updated weights for policy 1, policy_version 1759607 (0.0007) [2023-12-27 04:11:12,178][105620] Updated weights for policy 1, policy_version 1759617 (0.0009) [2023-12-27 04:11:12,754][105692] Updated weights for policy 0, policy_version 1755939 (0.0009) [2023-12-27 04:11:12,816][105692] Updated weights for policy 0, policy_version 1755949 (0.0009) [2023-12-27 04:11:12,879][105692] Updated weights for policy 0, policy_version 1755959 (0.0009) [2023-12-27 04:11:12,914][105620] Updated weights for policy 1, policy_version 1759627 (0.0009) [2023-12-27 04:11:12,974][105620] Updated weights for policy 1, policy_version 1759637 (0.0008) [2023-12-27 04:11:13,042][105620] Updated weights for policy 1, policy_version 1759647 (0.0009) [2023-12-27 04:11:13,574][105692] Updated weights for policy 0, policy_version 1755969 (0.0008) [2023-12-27 04:11:13,632][105692] Updated weights for policy 0, policy_version 1755979 (0.0008) [2023-12-27 04:11:13,715][105692] Updated weights for policy 0, policy_version 1755989 (0.0008) [2023-12-27 04:11:13,766][105692] Updated weights for policy 0, policy_version 1755999 (0.0009) [2023-12-27 04:11:13,798][105620] Updated weights for policy 1, policy_version 1759657 (0.0009) [2023-12-27 04:11:13,855][105620] Updated weights for policy 1, policy_version 1759667 (0.0009) [2023-12-27 04:11:13,908][105620] Updated weights for policy 1, policy_version 1759677 (0.0007) [2023-12-27 04:11:13,964][105620] Updated weights for policy 1, policy_version 1759687 (0.0005) [2023-12-27 04:11:14,500][105692] Updated weights for policy 0, policy_version 1756009 (0.0009) [2023-12-27 04:11:14,564][105692] Updated weights for policy 0, policy_version 1756019 (0.0009) [2023-12-27 04:11:14,619][105692] Updated weights for policy 0, policy_version 1756029 (0.0009) [2023-12-27 04:11:14,675][105620] Updated weights for policy 1, policy_version 1759697 (0.0008) [2023-12-27 04:11:14,722][105620] Updated weights for policy 1, policy_version 1759708 (0.0009) [2023-12-27 04:11:14,775][105620] Updated weights for policy 1, policy_version 1759718 (0.0007) [2023-12-27 04:11:15,334][105692] Updated weights for policy 0, policy_version 1756039 (0.0009) [2023-12-27 04:11:15,395][105692] Updated weights for policy 0, policy_version 1756049 (0.0009) [2023-12-27 04:11:15,445][105692] Updated weights for policy 0, policy_version 1756059 (0.0009) [2023-12-27 04:11:15,621][105620] Updated weights for policy 1, policy_version 1759728 (0.0009) [2023-12-27 04:11:15,673][105620] Updated weights for policy 1, policy_version 1759738 (0.0009) [2023-12-27 04:11:15,728][105620] Updated weights for policy 1, policy_version 1759748 (0.0008) [2023-12-27 04:11:16,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 900177920. Throughput: 0: 9717.3, 1: 10062.2. Samples: 900150392. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:11:16,063][104569] Avg episode reward: [(0, '8260.534'), (1, '9263.963')] [2023-12-27 04:11:16,074][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001756064_449617920.pth... [2023-12-27 04:11:16,074][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001759752_450560000.pth... [2023-12-27 04:11:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001754944_449331200.pth [2023-12-27 04:11:16,084][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001758568_450256896.pth [2023-12-27 04:11:16,202][105692] Updated weights for policy 0, policy_version 1756069 (0.0008) [2023-12-27 04:11:16,255][105692] Updated weights for policy 0, policy_version 1756079 (0.0005) [2023-12-27 04:11:16,311][105692] Updated weights for policy 0, policy_version 1756089 (0.0008) [2023-12-27 04:11:16,510][105620] Updated weights for policy 1, policy_version 1759758 (0.0009) [2023-12-27 04:11:16,564][105620] Updated weights for policy 1, policy_version 1759768 (0.0008) [2023-12-27 04:11:16,612][105620] Updated weights for policy 1, policy_version 1759778 (0.0009) [2023-12-27 04:11:17,004][105692] Updated weights for policy 0, policy_version 1756099 (0.0009) [2023-12-27 04:11:17,060][105692] Updated weights for policy 0, policy_version 1756109 (0.0008) [2023-12-27 04:11:17,116][105692] Updated weights for policy 0, policy_version 1756119 (0.0009) [2023-12-27 04:11:17,336][105620] Updated weights for policy 1, policy_version 1759788 (0.0009) [2023-12-27 04:11:17,391][105620] Updated weights for policy 1, policy_version 1759798 (0.0009) [2023-12-27 04:11:17,439][105620] Updated weights for policy 1, policy_version 1759808 (0.0009) [2023-12-27 04:11:17,836][105692] Updated weights for policy 0, policy_version 1756129 (0.0011) [2023-12-27 04:11:17,886][105692] Updated weights for policy 0, policy_version 1756139 (0.0009) [2023-12-27 04:11:17,936][105692] Updated weights for policy 0, policy_version 1756149 (0.0009) [2023-12-27 04:11:17,988][105692] Updated weights for policy 0, policy_version 1756159 (0.0009) [2023-12-27 04:11:18,228][105620] Updated weights for policy 1, policy_version 1759818 (0.0008) [2023-12-27 04:11:18,280][105620] Updated weights for policy 1, policy_version 1759828 (0.0007) [2023-12-27 04:11:18,331][105620] Updated weights for policy 1, policy_version 1759838 (0.0009) [2023-12-27 04:11:18,396][105620] Updated weights for policy 1, policy_version 1759848 (0.0009) [2023-12-27 04:11:18,714][105692] Updated weights for policy 0, policy_version 1756169 (0.0011) [2023-12-27 04:11:18,772][105692] Updated weights for policy 0, policy_version 1756179 (0.0010) [2023-12-27 04:11:18,831][105692] Updated weights for policy 0, policy_version 1756189 (0.0011) [2023-12-27 04:11:19,163][105620] Updated weights for policy 1, policy_version 1759858 (0.0010) [2023-12-27 04:11:19,223][105620] Updated weights for policy 1, policy_version 1759868 (0.0008) [2023-12-27 04:11:19,281][105620] Updated weights for policy 1, policy_version 1759878 (0.0008) [2023-12-27 04:11:19,552][105692] Updated weights for policy 0, policy_version 1756199 (0.0009) [2023-12-27 04:11:19,612][105692] Updated weights for policy 0, policy_version 1756209 (0.0008) [2023-12-27 04:11:19,678][105692] Updated weights for policy 0, policy_version 1756219 (0.0010) [2023-12-27 04:11:20,046][105620] Updated weights for policy 1, policy_version 1759888 (0.0008) [2023-12-27 04:11:20,110][105620] Updated weights for policy 1, policy_version 1759898 (0.0009) [2023-12-27 04:11:20,177][105620] Updated weights for policy 1, policy_version 1759908 (0.0009) [2023-12-27 04:11:20,452][105692] Updated weights for policy 0, policy_version 1756229 (0.0009) [2023-12-27 04:11:20,512][105692] Updated weights for policy 0, policy_version 1756239 (0.0009) [2023-12-27 04:11:20,563][105692] Updated weights for policy 0, policy_version 1756249 (0.0009) [2023-12-27 04:11:20,947][105620] Updated weights for policy 1, policy_version 1759918 (0.0009) [2023-12-27 04:11:21,010][105620] Updated weights for policy 1, policy_version 1759928 (0.0009) [2023-12-27 04:11:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 900268032. Throughput: 0: 9810.2, 1: 9928.3. Samples: 900264036. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:11:21,062][104569] Avg episode reward: [(0, '8989.868'), (1, '9077.684')] [2023-12-27 04:11:21,074][105620] Updated weights for policy 1, policy_version 1759938 (0.0009) [2023-12-27 04:11:21,371][105692] Updated weights for policy 0, policy_version 1756259 (0.0008) [2023-12-27 04:11:21,432][105692] Updated weights for policy 0, policy_version 1756269 (0.0006) [2023-12-27 04:11:21,485][105692] Updated weights for policy 0, policy_version 1756279 (0.0005) [2023-12-27 04:11:21,907][105620] Updated weights for policy 1, policy_version 1759948 (0.0009) [2023-12-27 04:11:21,972][105620] Updated weights for policy 1, policy_version 1759958 (0.0009) [2023-12-27 04:11:22,028][105620] Updated weights for policy 1, policy_version 1759968 (0.0009) [2023-12-27 04:11:22,166][105692] Updated weights for policy 0, policy_version 1756289 (0.0006) [2023-12-27 04:11:22,233][105692] Updated weights for policy 0, policy_version 1756299 (0.0007) [2023-12-27 04:11:22,296][105692] Updated weights for policy 0, policy_version 1756309 (0.0008) [2023-12-27 04:11:22,356][105692] Updated weights for policy 0, policy_version 1756319 (0.0007) [2023-12-27 04:11:22,802][105620] Updated weights for policy 1, policy_version 1759978 (0.0009) [2023-12-27 04:11:22,856][105620] Updated weights for policy 1, policy_version 1759988 (0.0005) [2023-12-27 04:11:22,917][105620] Updated weights for policy 1, policy_version 1759998 (0.0005) [2023-12-27 04:11:22,975][105620] Updated weights for policy 1, policy_version 1760008 (0.0006) [2023-12-27 04:11:23,137][105692] Updated weights for policy 0, policy_version 1756329 (0.0009) [2023-12-27 04:11:23,203][105692] Updated weights for policy 0, policy_version 1756339 (0.0009) [2023-12-27 04:11:23,271][105692] Updated weights for policy 0, policy_version 1756349 (0.0010) [2023-12-27 04:11:23,535][105620] Updated weights for policy 1, policy_version 1760018 (0.0010) [2023-12-27 04:11:23,594][105620] Updated weights for policy 1, policy_version 1760028 (0.0010) [2023-12-27 04:11:23,660][105620] Updated weights for policy 1, policy_version 1760038 (0.0009) [2023-12-27 04:11:23,966][105692] Updated weights for policy 0, policy_version 1756359 (0.0007) [2023-12-27 04:11:24,015][105692] Updated weights for policy 0, policy_version 1756369 (0.0005) [2023-12-27 04:11:24,068][105692] Updated weights for policy 0, policy_version 1756379 (0.0005) [2023-12-27 04:11:24,454][105620] Updated weights for policy 1, policy_version 1760048 (0.0006) [2023-12-27 04:11:24,512][105620] Updated weights for policy 1, policy_version 1760058 (0.0007) [2023-12-27 04:11:24,581][105620] Updated weights for policy 1, policy_version 1760068 (0.0009) [2023-12-27 04:11:24,717][105692] Updated weights for policy 0, policy_version 1756389 (0.0007) [2023-12-27 04:11:24,775][105692] Updated weights for policy 0, policy_version 1756399 (0.0009) [2023-12-27 04:11:24,827][105692] Updated weights for policy 0, policy_version 1756409 (0.0009) [2023-12-27 04:11:25,256][105620] Updated weights for policy 1, policy_version 1760078 (0.0009) [2023-12-27 04:11:25,312][105620] Updated weights for policy 1, policy_version 1760088 (0.0006) [2023-12-27 04:11:25,367][105620] Updated weights for policy 1, policy_version 1760098 (0.0009) [2023-12-27 04:11:25,598][105692] Updated weights for policy 0, policy_version 1756419 (0.0009) [2023-12-27 04:11:25,650][105692] Updated weights for policy 0, policy_version 1756429 (0.0005) [2023-12-27 04:11:25,703][105692] Updated weights for policy 0, policy_version 1756439 (0.0005) [2023-12-27 04:11:26,009][105620] Updated weights for policy 1, policy_version 1760108 (0.0007) [2023-12-27 04:11:26,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 900366336. Throughput: 0: 9763.0, 1: 9829.4. Samples: 900378044. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:11:26,062][104569] Avg episode reward: [(0, '8802.396'), (1, '9077.572')] [2023-12-27 04:11:26,071][105620] Updated weights for policy 1, policy_version 1760118 (0.0006) [2023-12-27 04:11:26,126][105620] Updated weights for policy 1, policy_version 1760128 (0.0005) [2023-12-27 04:11:26,316][105692] Updated weights for policy 0, policy_version 1756449 (0.0006) [2023-12-27 04:11:26,367][105692] Updated weights for policy 0, policy_version 1756459 (0.0010) [2023-12-27 04:11:26,411][105692] Updated weights for policy 0, policy_version 1756469 (0.0010) [2023-12-27 04:11:26,466][105692] Updated weights for policy 0, policy_version 1756479 (0.0008) [2023-12-27 04:11:26,631][105620] Updated weights for policy 1, policy_version 1760138 (0.0006) [2023-12-27 04:11:26,696][105620] Updated weights for policy 1, policy_version 1760148 (0.0008) [2023-12-27 04:11:26,764][105620] Updated weights for policy 1, policy_version 1760158 (0.0009) [2023-12-27 04:11:26,822][105620] Updated weights for policy 1, policy_version 1760168 (0.0007) [2023-12-27 04:11:27,201][105692] Updated weights for policy 0, policy_version 1756489 (0.0008) [2023-12-27 04:11:27,259][105692] Updated weights for policy 0, policy_version 1756499 (0.0010) [2023-12-27 04:11:27,317][105692] Updated weights for policy 0, policy_version 1756509 (0.0010) [2023-12-27 04:11:27,379][105620] Updated weights for policy 1, policy_version 1760178 (0.0010) [2023-12-27 04:11:27,438][105620] Updated weights for policy 1, policy_version 1760188 (0.0009) [2023-12-27 04:11:27,489][105620] Updated weights for policy 1, policy_version 1760198 (0.0005) [2023-12-27 04:11:27,921][105692] Updated weights for policy 0, policy_version 1756519 (0.0007) [2023-12-27 04:11:27,967][105692] Updated weights for policy 0, policy_version 1756529 (0.0005) [2023-12-27 04:11:28,016][105620] Updated weights for policy 1, policy_version 1760208 (0.0005) [2023-12-27 04:11:28,025][105692] Updated weights for policy 0, policy_version 1756539 (0.0008) [2023-12-27 04:11:28,073][105620] Updated weights for policy 1, policy_version 1760218 (0.0005) [2023-12-27 04:11:28,130][105620] Updated weights for policy 1, policy_version 1760228 (0.0005) [2023-12-27 04:11:28,708][105692] Updated weights for policy 0, policy_version 1756549 (0.0009) [2023-12-27 04:11:28,725][105620] Updated weights for policy 1, policy_version 1760238 (0.0008) [2023-12-27 04:11:28,770][105692] Updated weights for policy 0, policy_version 1756559 (0.0006) [2023-12-27 04:11:28,782][105620] Updated weights for policy 1, policy_version 1760248 (0.0010) [2023-12-27 04:11:28,827][105692] Updated weights for policy 0, policy_version 1756569 (0.0009) [2023-12-27 04:11:28,842][105620] Updated weights for policy 1, policy_version 1760258 (0.0005) [2023-12-27 04:11:29,416][105620] Updated weights for policy 1, policy_version 1760268 (0.0006) [2023-12-27 04:11:29,463][105620] Updated weights for policy 1, policy_version 1760278 (0.0005) [2023-12-27 04:11:29,508][105620] Updated weights for policy 1, policy_version 1760288 (0.0009) [2023-12-27 04:11:29,609][105692] Updated weights for policy 0, policy_version 1756579 (0.0011) [2023-12-27 04:11:29,668][105692] Updated weights for policy 0, policy_version 1756589 (0.0011) [2023-12-27 04:11:29,733][105692] Updated weights for policy 0, policy_version 1756599 (0.0010) [2023-12-27 04:11:30,207][105620] Updated weights for policy 1, policy_version 1760298 (0.0007) [2023-12-27 04:11:30,272][105620] Updated weights for policy 1, policy_version 1760308 (0.0006) [2023-12-27 04:11:30,310][105586] KL-divergence is very high: 102.7689 [2023-12-27 04:11:30,340][105620] Updated weights for policy 1, policy_version 1760318 (0.0010) [2023-12-27 04:11:30,359][105586] KL-divergence is very high: 112.8351 [2023-12-27 04:11:30,380][105692] Updated weights for policy 0, policy_version 1756609 (0.0009) [2023-12-27 04:11:30,402][105620] Updated weights for policy 1, policy_version 1760328 (0.0011) [2023-12-27 04:11:30,425][105692] Updated weights for policy 0, policy_version 1756619 (0.0005) [2023-12-27 04:11:30,475][105692] Updated weights for policy 0, policy_version 1756629 (0.0007) [2023-12-27 04:11:30,531][105692] Updated weights for policy 0, policy_version 1756639 (0.0005) [2023-12-27 04:11:30,988][105620] Updated weights for policy 1, policy_version 1760338 (0.0008) [2023-12-27 04:11:31,054][105620] Updated weights for policy 1, policy_version 1760348 (0.0008) [2023-12-27 04:11:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 900472832. Throughput: 0: 9831.6, 1: 9986.3. Samples: 900445272. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:11:31,063][104569] Avg episode reward: [(0, '8437.457'), (1, '9260.766')] [2023-12-27 04:11:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001756640_449765376.pth... [2023-12-27 04:11:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001755520_449478656.pth [2023-12-27 04:11:31,118][105620] Updated weights for policy 1, policy_version 1760358 (0.0006) [2023-12-27 04:11:31,134][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001760360_450715648.pth... [2023-12-27 04:11:31,138][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001759176_450412544.pth [2023-12-27 04:11:31,318][105692] Updated weights for policy 0, policy_version 1756649 (0.0005) [2023-12-27 04:11:31,380][105692] Updated weights for policy 0, policy_version 1756659 (0.0007) [2023-12-27 04:11:31,433][105692] Updated weights for policy 0, policy_version 1756669 (0.0006) [2023-12-27 04:11:31,751][105620] Updated weights for policy 1, policy_version 1760368 (0.0008) [2023-12-27 04:11:31,805][105620] Updated weights for policy 1, policy_version 1760378 (0.0008) [2023-12-27 04:11:31,852][105620] Updated weights for policy 1, policy_version 1760388 (0.0008) [2023-12-27 04:11:32,106][105692] Updated weights for policy 0, policy_version 1756679 (0.0005) [2023-12-27 04:11:32,159][105692] Updated weights for policy 0, policy_version 1756689 (0.0005) [2023-12-27 04:11:32,208][105692] Updated weights for policy 0, policy_version 1756699 (0.0005) [2023-12-27 04:11:32,530][105620] Updated weights for policy 1, policy_version 1760398 (0.0009) [2023-12-27 04:11:32,579][105620] Updated weights for policy 1, policy_version 1760408 (0.0009) [2023-12-27 04:11:32,634][105620] Updated weights for policy 1, policy_version 1760418 (0.0009) [2023-12-27 04:11:32,841][105692] Updated weights for policy 0, policy_version 1756709 (0.0007) [2023-12-27 04:11:32,900][105692] Updated weights for policy 0, policy_version 1756719 (0.0011) [2023-12-27 04:11:32,960][105692] Updated weights for policy 0, policy_version 1756729 (0.0010) [2023-12-27 04:11:33,302][105620] Updated weights for policy 1, policy_version 1760428 (0.0007) [2023-12-27 04:11:33,345][105620] Updated weights for policy 1, policy_version 1760438 (0.0005) [2023-12-27 04:11:33,390][105620] Updated weights for policy 1, policy_version 1760448 (0.0005) [2023-12-27 04:11:33,713][105692] Updated weights for policy 0, policy_version 1756741 (0.0008) [2023-12-27 04:11:33,758][105692] Updated weights for policy 0, policy_version 1756751 (0.0005) [2023-12-27 04:11:33,804][105692] Updated weights for policy 0, policy_version 1756761 (0.0005) [2023-12-27 04:11:33,991][105620] Updated weights for policy 1, policy_version 1760458 (0.0005) [2023-12-27 04:11:34,051][105620] Updated weights for policy 1, policy_version 1760468 (0.0005) [2023-12-27 04:11:34,121][105620] Updated weights for policy 1, policy_version 1760478 (0.0005) [2023-12-27 04:11:34,192][105620] Updated weights for policy 1, policy_version 1760488 (0.0007) [2023-12-27 04:11:34,410][105692] Updated weights for policy 0, policy_version 1756771 (0.0006) [2023-12-27 04:11:34,462][105692] Updated weights for policy 0, policy_version 1756781 (0.0009) [2023-12-27 04:11:34,520][105692] Updated weights for policy 0, policy_version 1756791 (0.0009) [2023-12-27 04:11:34,818][105620] Updated weights for policy 1, policy_version 1760498 (0.0009) [2023-12-27 04:11:34,872][105620] Updated weights for policy 1, policy_version 1760508 (0.0009) [2023-12-27 04:11:34,925][105620] Updated weights for policy 1, policy_version 1760518 (0.0009) [2023-12-27 04:11:35,307][105692] Updated weights for policy 0, policy_version 1756801 (0.0009) [2023-12-27 04:11:35,368][105692] Updated weights for policy 0, policy_version 1756811 (0.0008) [2023-12-27 04:11:35,430][105692] Updated weights for policy 0, policy_version 1756821 (0.0007) [2023-12-27 04:11:35,479][105692] Updated weights for policy 0, policy_version 1756831 (0.0008) [2023-12-27 04:11:35,676][105620] Updated weights for policy 1, policy_version 1760528 (0.0010) [2023-12-27 04:11:35,734][105620] Updated weights for policy 1, policy_version 1760539 (0.0006) [2023-12-27 04:11:35,783][105620] Updated weights for policy 1, policy_version 1760549 (0.0005) [2023-12-27 04:11:36,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 900579328. Throughput: 0: 9758.3, 1: 10107.4. Samples: 900569568. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:11:36,063][104569] Avg episode reward: [(0, '8623.308'), (1, '9260.786')] [2023-12-27 04:11:36,273][105692] Updated weights for policy 0, policy_version 1756841 (0.0009) [2023-12-27 04:11:36,324][105692] Updated weights for policy 0, policy_version 1756851 (0.0009) [2023-12-27 04:11:36,382][105692] Updated weights for policy 0, policy_version 1756861 (0.0008) [2023-12-27 04:11:36,452][105620] Updated weights for policy 1, policy_version 1760559 (0.0008) [2023-12-27 04:11:36,502][105620] Updated weights for policy 1, policy_version 1760569 (0.0006) [2023-12-27 04:11:36,561][105620] Updated weights for policy 1, policy_version 1760579 (0.0006) [2023-12-27 04:11:37,174][105692] Updated weights for policy 0, policy_version 1756871 (0.0008) [2023-12-27 04:11:37,175][105620] Updated weights for policy 1, policy_version 1760589 (0.0007) [2023-12-27 04:11:37,231][105692] Updated weights for policy 0, policy_version 1756881 (0.0007) [2023-12-27 04:11:37,236][105620] Updated weights for policy 1, policy_version 1760599 (0.0010) [2023-12-27 04:11:37,283][105692] Updated weights for policy 0, policy_version 1756891 (0.0008) [2023-12-27 04:11:37,300][105620] Updated weights for policy 1, policy_version 1760609 (0.0010) [2023-12-27 04:11:38,005][105620] Updated weights for policy 1, policy_version 1760619 (0.0008) [2023-12-27 04:11:38,042][105692] Updated weights for policy 0, policy_version 1756901 (0.0006) [2023-12-27 04:11:38,059][105620] Updated weights for policy 1, policy_version 1760629 (0.0008) [2023-12-27 04:11:38,090][105692] Updated weights for policy 0, policy_version 1756911 (0.0005) [2023-12-27 04:11:38,115][105620] Updated weights for policy 1, policy_version 1760639 (0.0008) [2023-12-27 04:11:38,138][105692] Updated weights for policy 0, policy_version 1756921 (0.0005) [2023-12-27 04:11:38,767][105692] Updated weights for policy 0, policy_version 1756931 (0.0007) [2023-12-27 04:11:38,830][105692] Updated weights for policy 0, policy_version 1756941 (0.0010) [2023-12-27 04:11:38,875][105620] Updated weights for policy 1, policy_version 1760649 (0.0008) [2023-12-27 04:11:38,888][105692] Updated weights for policy 0, policy_version 1756951 (0.0009) [2023-12-27 04:11:38,929][105620] Updated weights for policy 1, policy_version 1760659 (0.0008) [2023-12-27 04:11:38,990][105620] Updated weights for policy 1, policy_version 1760669 (0.0009) [2023-12-27 04:11:39,056][105620] Updated weights for policy 1, policy_version 1760679 (0.0008) [2023-12-27 04:11:39,622][105692] Updated weights for policy 0, policy_version 1756961 (0.0008) [2023-12-27 04:11:39,682][105692] Updated weights for policy 0, policy_version 1756971 (0.0006) [2023-12-27 04:11:39,746][105692] Updated weights for policy 0, policy_version 1756981 (0.0006) [2023-12-27 04:11:39,779][105620] Updated weights for policy 1, policy_version 1760689 (0.0007) [2023-12-27 04:11:39,804][105692] Updated weights for policy 0, policy_version 1756991 (0.0007) [2023-12-27 04:11:39,843][105620] Updated weights for policy 1, policy_version 1760699 (0.0007) [2023-12-27 04:11:39,912][105620] Updated weights for policy 1, policy_version 1760709 (0.0006) [2023-12-27 04:11:40,507][105692] Updated weights for policy 0, policy_version 1757001 (0.0009) [2023-12-27 04:11:40,561][105692] Updated weights for policy 0, policy_version 1757011 (0.0008) [2023-12-27 04:11:40,620][105692] Updated weights for policy 0, policy_version 1757021 (0.0007) [2023-12-27 04:11:40,635][105620] Updated weights for policy 1, policy_version 1760719 (0.0008) [2023-12-27 04:11:40,696][105620] Updated weights for policy 1, policy_version 1760729 (0.0008) [2023-12-27 04:11:40,756][105620] Updated weights for policy 1, policy_version 1760739 (0.0009) [2023-12-27 04:11:41,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 900677632. Throughput: 0: 9711.6, 1: 9996.5. Samples: 900685464. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:11:41,062][104569] Avg episode reward: [(0, '8713.454'), (1, '9262.969')] [2023-12-27 04:11:41,434][105692] Updated weights for policy 0, policy_version 1757031 (0.0007) [2023-12-27 04:11:41,500][105692] Updated weights for policy 0, policy_version 1757041 (0.0008) [2023-12-27 04:11:41,549][105620] Updated weights for policy 1, policy_version 1760749 (0.0006) [2023-12-27 04:11:41,565][105692] Updated weights for policy 0, policy_version 1757051 (0.0009) [2023-12-27 04:11:41,605][105620] Updated weights for policy 1, policy_version 1760759 (0.0009) [2023-12-27 04:11:41,676][105620] Updated weights for policy 1, policy_version 1760769 (0.0009) [2023-12-27 04:11:42,197][105692] Updated weights for policy 0, policy_version 1757061 (0.0010) [2023-12-27 04:11:42,255][105692] Updated weights for policy 0, policy_version 1757071 (0.0010) [2023-12-27 04:11:42,321][105692] Updated weights for policy 0, policy_version 1757081 (0.0010) [2023-12-27 04:11:42,497][105620] Updated weights for policy 1, policy_version 1760779 (0.0009) [2023-12-27 04:11:42,559][105620] Updated weights for policy 1, policy_version 1760789 (0.0009) [2023-12-27 04:11:42,620][105620] Updated weights for policy 1, policy_version 1760799 (0.0009) [2023-12-27 04:11:43,050][105692] Updated weights for policy 0, policy_version 1757091 (0.0008) [2023-12-27 04:11:43,117][105692] Updated weights for policy 0, policy_version 1757101 (0.0010) [2023-12-27 04:11:43,174][105692] Updated weights for policy 0, policy_version 1757111 (0.0010) [2023-12-27 04:11:43,268][105620] Updated weights for policy 1, policy_version 1760809 (0.0007) [2023-12-27 04:11:43,316][105620] Updated weights for policy 1, policy_version 1760819 (0.0007) [2023-12-27 04:11:43,378][105620] Updated weights for policy 1, policy_version 1760829 (0.0007) [2023-12-27 04:11:43,443][105620] Updated weights for policy 1, policy_version 1760839 (0.0008) [2023-12-27 04:11:43,782][105692] Updated weights for policy 0, policy_version 1757121 (0.0010) [2023-12-27 04:11:43,843][105692] Updated weights for policy 0, policy_version 1757131 (0.0009) [2023-12-27 04:11:43,904][105692] Updated weights for policy 0, policy_version 1757141 (0.0009) [2023-12-27 04:11:43,962][105692] Updated weights for policy 0, policy_version 1757151 (0.0009) [2023-12-27 04:11:44,139][105620] Updated weights for policy 1, policy_version 1760849 (0.0009) [2023-12-27 04:11:44,209][105620] Updated weights for policy 1, policy_version 1760859 (0.0010) [2023-12-27 04:11:44,267][105620] Updated weights for policy 1, policy_version 1760869 (0.0008) [2023-12-27 04:11:44,693][105692] Updated weights for policy 0, policy_version 1757161 (0.0010) [2023-12-27 04:11:44,745][105692] Updated weights for policy 0, policy_version 1757171 (0.0010) [2023-12-27 04:11:44,802][105692] Updated weights for policy 0, policy_version 1757181 (0.0011) [2023-12-27 04:11:45,019][105620] Updated weights for policy 1, policy_version 1760879 (0.0008) [2023-12-27 04:11:45,071][105620] Updated weights for policy 1, policy_version 1760889 (0.0010) [2023-12-27 04:11:45,122][105620] Updated weights for policy 1, policy_version 1760899 (0.0009) [2023-12-27 04:11:45,604][105692] Updated weights for policy 0, policy_version 1757191 (0.0010) [2023-12-27 04:11:45,669][105692] Updated weights for policy 0, policy_version 1757201 (0.0008) [2023-12-27 04:11:45,732][105692] Updated weights for policy 0, policy_version 1757211 (0.0009) [2023-12-27 04:11:45,860][105620] Updated weights for policy 1, policy_version 1760909 (0.0009) [2023-12-27 04:11:45,914][105620] Updated weights for policy 1, policy_version 1760919 (0.0009) [2023-12-27 04:11:45,979][105620] Updated weights for policy 1, policy_version 1760929 (0.0009) [2023-12-27 04:11:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 900775936. Throughput: 0: 9645.0, 1: 9940.3. Samples: 900743372. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:11:46,063][104569] Avg episode reward: [(0, '8621.693'), (1, '9170.700')] [2023-12-27 04:11:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001757216_449912832.pth... [2023-12-27 04:11:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001760936_450863104.pth... [2023-12-27 04:11:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001759752_450560000.pth [2023-12-27 04:11:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001756064_449617920.pth [2023-12-27 04:11:46,347][105692] Updated weights for policy 0, policy_version 1757221 (0.0007) [2023-12-27 04:11:46,403][105692] Updated weights for policy 0, policy_version 1757231 (0.0008) [2023-12-27 04:11:46,455][105692] Updated weights for policy 0, policy_version 1757241 (0.0010) [2023-12-27 04:11:46,800][105620] Updated weights for policy 1, policy_version 1760939 (0.0009) [2023-12-27 04:11:46,863][105620] Updated weights for policy 1, policy_version 1760949 (0.0008) [2023-12-27 04:11:46,919][105620] Updated weights for policy 1, policy_version 1760959 (0.0006) [2023-12-27 04:11:47,149][105692] Updated weights for policy 0, policy_version 1757251 (0.0009) [2023-12-27 04:11:47,204][105692] Updated weights for policy 0, policy_version 1757261 (0.0010) [2023-12-27 04:11:47,251][105692] Updated weights for policy 0, policy_version 1757271 (0.0010) [2023-12-27 04:11:47,567][105620] Updated weights for policy 1, policy_version 1760969 (0.0006) [2023-12-27 04:11:47,624][105620] Updated weights for policy 1, policy_version 1760979 (0.0005) [2023-12-27 04:11:47,686][105620] Updated weights for policy 1, policy_version 1760989 (0.0008) [2023-12-27 04:11:47,744][105620] Updated weights for policy 1, policy_version 1760999 (0.0010) [2023-12-27 04:11:47,922][105692] Updated weights for policy 0, policy_version 1757281 (0.0010) [2023-12-27 04:11:47,986][105692] Updated weights for policy 0, policy_version 1757291 (0.0005) [2023-12-27 04:11:48,042][105692] Updated weights for policy 0, policy_version 1757301 (0.0005) [2023-12-27 04:11:48,088][105692] Updated weights for policy 0, policy_version 1757311 (0.0005) [2023-12-27 04:11:48,295][105620] Updated weights for policy 1, policy_version 1761009 (0.0006) [2023-12-27 04:11:48,357][105620] Updated weights for policy 1, policy_version 1761019 (0.0007) [2023-12-27 04:11:48,416][105620] Updated weights for policy 1, policy_version 1761029 (0.0009) [2023-12-27 04:11:48,773][105692] Updated weights for policy 0, policy_version 1757321 (0.0008) [2023-12-27 04:11:48,821][105692] Updated weights for policy 0, policy_version 1757331 (0.0007) [2023-12-27 04:11:48,867][105692] Updated weights for policy 0, policy_version 1757341 (0.0005) [2023-12-27 04:11:49,065][105620] Updated weights for policy 1, policy_version 1761039 (0.0009) [2023-12-27 04:11:49,116][105620] Updated weights for policy 1, policy_version 1761049 (0.0010) [2023-12-27 04:11:49,160][105620] Updated weights for policy 1, policy_version 1761059 (0.0010) [2023-12-27 04:11:49,510][105692] Updated weights for policy 0, policy_version 1757351 (0.0007) [2023-12-27 04:11:49,577][105692] Updated weights for policy 0, policy_version 1757361 (0.0006) [2023-12-27 04:11:49,636][105692] Updated weights for policy 0, policy_version 1757371 (0.0005) [2023-12-27 04:11:50,029][105620] Updated weights for policy 1, policy_version 1761069 (0.0010) [2023-12-27 04:11:50,084][105620] Updated weights for policy 1, policy_version 1761079 (0.0009) [2023-12-27 04:11:50,146][105620] Updated weights for policy 1, policy_version 1761089 (0.0009) [2023-12-27 04:11:50,287][105692] Updated weights for policy 0, policy_version 1757381 (0.0007) [2023-12-27 04:11:50,335][105692] Updated weights for policy 0, policy_version 1757391 (0.0009) [2023-12-27 04:11:50,396][105692] Updated weights for policy 0, policy_version 1757401 (0.0008) [2023-12-27 04:11:50,876][105620] Updated weights for policy 1, policy_version 1761099 (0.0008) [2023-12-27 04:11:50,938][105620] Updated weights for policy 1, policy_version 1761109 (0.0007) [2023-12-27 04:11:51,004][105620] Updated weights for policy 1, policy_version 1761119 (0.0006) [2023-12-27 04:11:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 900866048. Throughput: 0: 9723.9, 1: 9938.6. Samples: 900861696. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:11:51,063][104569] Avg episode reward: [(0, '8806.839'), (1, '9170.707')] [2023-12-27 04:11:51,257][105692] Updated weights for policy 0, policy_version 1757411 (0.0009) [2023-12-27 04:11:51,323][105692] Updated weights for policy 0, policy_version 1757421 (0.0009) [2023-12-27 04:11:51,396][105692] Updated weights for policy 0, policy_version 1757431 (0.0012) [2023-12-27 04:11:51,738][105620] Updated weights for policy 1, policy_version 1761129 (0.0008) [2023-12-27 04:11:51,793][105620] Updated weights for policy 1, policy_version 1761139 (0.0006) [2023-12-27 04:11:51,855][105620] Updated weights for policy 1, policy_version 1761149 (0.0005) [2023-12-27 04:11:51,906][105620] Updated weights for policy 1, policy_version 1761159 (0.0005) [2023-12-27 04:11:52,280][105692] Updated weights for policy 0, policy_version 1757441 (0.0008) [2023-12-27 04:11:52,356][105692] Updated weights for policy 0, policy_version 1757451 (0.0009) [2023-12-27 04:11:52,426][105692] Updated weights for policy 0, policy_version 1757461 (0.0010) [2023-12-27 04:11:52,493][105692] Updated weights for policy 0, policy_version 1757471 (0.0009) [2023-12-27 04:11:52,499][105620] Updated weights for policy 1, policy_version 1761169 (0.0007) [2023-12-27 04:11:52,551][105620] Updated weights for policy 1, policy_version 1761179 (0.0009) [2023-12-27 04:11:52,607][105620] Updated weights for policy 1, policy_version 1761189 (0.0008) [2023-12-27 04:11:53,174][105692] Updated weights for policy 0, policy_version 1757481 (0.0008) [2023-12-27 04:11:53,222][105692] Updated weights for policy 0, policy_version 1757491 (0.0008) [2023-12-27 04:11:53,273][105692] Updated weights for policy 0, policy_version 1757501 (0.0008) [2023-12-27 04:11:53,377][105620] Updated weights for policy 1, policy_version 1761199 (0.0007) [2023-12-27 04:11:53,436][105620] Updated weights for policy 1, policy_version 1761209 (0.0005) [2023-12-27 04:11:53,490][105620] Updated weights for policy 1, policy_version 1761219 (0.0006) [2023-12-27 04:11:54,047][105692] Updated weights for policy 0, policy_version 1757511 (0.0007) [2023-12-27 04:11:54,064][105620] Updated weights for policy 1, policy_version 1761229 (0.0008) [2023-12-27 04:11:54,099][105692] Updated weights for policy 0, policy_version 1757521 (0.0005) [2023-12-27 04:11:54,120][105620] Updated weights for policy 1, policy_version 1761239 (0.0010) [2023-12-27 04:11:54,153][105692] Updated weights for policy 0, policy_version 1757531 (0.0006) [2023-12-27 04:11:54,178][105620] Updated weights for policy 1, policy_version 1761249 (0.0011) [2023-12-27 04:11:54,902][105692] Updated weights for policy 0, policy_version 1757541 (0.0007) [2023-12-27 04:11:54,927][105620] Updated weights for policy 1, policy_version 1761259 (0.0011) [2023-12-27 04:11:54,959][105692] Updated weights for policy 0, policy_version 1757551 (0.0006) [2023-12-27 04:11:54,984][105620] Updated weights for policy 1, policy_version 1761269 (0.0008) [2023-12-27 04:11:55,022][105692] Updated weights for policy 0, policy_version 1757561 (0.0009) [2023-12-27 04:11:55,041][105620] Updated weights for policy 1, policy_version 1761279 (0.0006) [2023-12-27 04:11:55,743][105620] Updated weights for policy 1, policy_version 1761289 (0.0009) [2023-12-27 04:11:55,754][105692] Updated weights for policy 0, policy_version 1757571 (0.0009) [2023-12-27 04:11:55,804][105692] Updated weights for policy 0, policy_version 1757581 (0.0008) [2023-12-27 04:11:55,808][105620] Updated weights for policy 1, policy_version 1761299 (0.0005) [2023-12-27 04:11:55,856][105692] Updated weights for policy 0, policy_version 1757591 (0.0008) [2023-12-27 04:11:55,862][105620] Updated weights for policy 1, policy_version 1761309 (0.0005) [2023-12-27 04:11:55,920][105620] Updated weights for policy 1, policy_version 1761319 (0.0007) [2023-12-27 04:11:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 900972544. Throughput: 0: 9709.7, 1: 9938.5. Samples: 900976920. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:11:56,062][104569] Avg episode reward: [(0, '8806.438'), (1, '9260.928')] [2023-12-27 04:11:56,590][105620] Updated weights for policy 1, policy_version 1761329 (0.0007) [2023-12-27 04:11:56,644][105620] Updated weights for policy 1, policy_version 1761339 (0.0005) [2023-12-27 04:11:56,664][105692] Updated weights for policy 0, policy_version 1757601 (0.0008) [2023-12-27 04:11:56,699][105620] Updated weights for policy 1, policy_version 1761349 (0.0005) [2023-12-27 04:11:56,721][105692] Updated weights for policy 0, policy_version 1757611 (0.0009) [2023-12-27 04:11:56,772][105692] Updated weights for policy 0, policy_version 1757621 (0.0009) [2023-12-27 04:11:56,822][105692] Updated weights for policy 0, policy_version 1757631 (0.0009) [2023-12-27 04:11:57,340][105620] Updated weights for policy 1, policy_version 1761359 (0.0008) [2023-12-27 04:11:57,385][105620] Updated weights for policy 1, policy_version 1761369 (0.0008) [2023-12-27 04:11:57,432][105620] Updated weights for policy 1, policy_version 1761379 (0.0009) [2023-12-27 04:11:57,602][105692] Updated weights for policy 0, policy_version 1757641 (0.0009) [2023-12-27 04:11:57,649][105692] Updated weights for policy 0, policy_version 1757651 (0.0009) [2023-12-27 04:11:57,696][105692] Updated weights for policy 0, policy_version 1757661 (0.0009) [2023-12-27 04:11:58,158][105620] Updated weights for policy 1, policy_version 1761389 (0.0009) [2023-12-27 04:11:58,223][105620] Updated weights for policy 1, policy_version 1761399 (0.0009) [2023-12-27 04:11:58,278][105620] Updated weights for policy 1, policy_version 1761409 (0.0009) [2023-12-27 04:11:58,427][105692] Updated weights for policy 0, policy_version 1757671 (0.0009) [2023-12-27 04:11:58,487][105692] Updated weights for policy 0, policy_version 1757681 (0.0009) [2023-12-27 04:11:58,552][105692] Updated weights for policy 0, policy_version 1757691 (0.0009) [2023-12-27 04:11:59,065][105620] Updated weights for policy 1, policy_version 1761419 (0.0009) [2023-12-27 04:11:59,122][105620] Updated weights for policy 1, policy_version 1761429 (0.0009) [2023-12-27 04:11:59,168][105620] Updated weights for policy 1, policy_version 1761439 (0.0008) [2023-12-27 04:11:59,359][105692] Updated weights for policy 0, policy_version 1757701 (0.0009) [2023-12-27 04:11:59,420][105692] Updated weights for policy 0, policy_version 1757711 (0.0008) [2023-12-27 04:11:59,477][105692] Updated weights for policy 0, policy_version 1757721 (0.0009) [2023-12-27 04:11:59,830][105620] Updated weights for policy 1, policy_version 1761449 (0.0008) [2023-12-27 04:11:59,890][105620] Updated weights for policy 1, policy_version 1761459 (0.0007) [2023-12-27 04:11:59,954][105620] Updated weights for policy 1, policy_version 1761469 (0.0007) [2023-12-27 04:12:00,016][105620] Updated weights for policy 1, policy_version 1761479 (0.0008) [2023-12-27 04:12:00,376][105692] Updated weights for policy 0, policy_version 1757731 (0.0009) [2023-12-27 04:12:00,435][105692] Updated weights for policy 0, policy_version 1757741 (0.0007) [2023-12-27 04:12:00,490][105692] Updated weights for policy 0, policy_version 1757751 (0.0007) [2023-12-27 04:12:00,632][105620] Updated weights for policy 1, policy_version 1761489 (0.0010) [2023-12-27 04:12:00,690][105620] Updated weights for policy 1, policy_version 1761499 (0.0010) [2023-12-27 04:12:00,744][105620] Updated weights for policy 1, policy_version 1761509 (0.0010) [2023-12-27 04:12:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 901062656. Throughput: 0: 9668.2, 1: 9961.3. Samples: 901033716. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:12:01,062][104569] Avg episode reward: [(0, '8259.699'), (1, '9078.604')] [2023-12-27 04:12:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001757760_450052096.pth... [2023-12-27 04:12:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001761512_451010560.pth... [2023-12-27 04:12:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001760360_450715648.pth [2023-12-27 04:12:01,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001756640_449765376.pth [2023-12-27 04:12:01,239][105692] Updated weights for policy 0, policy_version 1757761 (0.0007) [2023-12-27 04:12:01,301][105692] Updated weights for policy 0, policy_version 1757771 (0.0008) [2023-12-27 04:12:01,364][105692] Updated weights for policy 0, policy_version 1757781 (0.0008) [2023-12-27 04:12:01,418][105692] Updated weights for policy 0, policy_version 1757791 (0.0009) [2023-12-27 04:12:01,480][105620] Updated weights for policy 1, policy_version 1761519 (0.0007) [2023-12-27 04:12:01,543][105620] Updated weights for policy 1, policy_version 1761529 (0.0006) [2023-12-27 04:12:01,601][105620] Updated weights for policy 1, policy_version 1761539 (0.0007) [2023-12-27 04:12:02,082][105692] Updated weights for policy 0, policy_version 1757801 (0.0006) [2023-12-27 04:12:02,139][105692] Updated weights for policy 0, policy_version 1757811 (0.0005) [2023-12-27 04:12:02,200][105692] Updated weights for policy 0, policy_version 1757821 (0.0006) [2023-12-27 04:12:02,391][105620] Updated weights for policy 1, policy_version 1761549 (0.0007) [2023-12-27 04:12:02,436][105620] Updated weights for policy 1, policy_version 1761559 (0.0008) [2023-12-27 04:12:02,483][105620] Updated weights for policy 1, policy_version 1761569 (0.0005) [2023-12-27 04:12:02,857][105692] Updated weights for policy 0, policy_version 1757831 (0.0006) [2023-12-27 04:12:02,918][105692] Updated weights for policy 0, policy_version 1757841 (0.0005) [2023-12-27 04:12:02,985][105692] Updated weights for policy 0, policy_version 1757851 (0.0006) [2023-12-27 04:12:03,181][105620] Updated weights for policy 1, policy_version 1761579 (0.0005) [2023-12-27 04:12:03,227][105620] Updated weights for policy 1, policy_version 1761589 (0.0005) [2023-12-27 04:12:03,274][105620] Updated weights for policy 1, policy_version 1761599 (0.0006) [2023-12-27 04:12:03,706][105692] Updated weights for policy 0, policy_version 1757861 (0.0009) [2023-12-27 04:12:03,753][105692] Updated weights for policy 0, policy_version 1757871 (0.0009) [2023-12-27 04:12:03,803][105692] Updated weights for policy 0, policy_version 1757881 (0.0009) [2023-12-27 04:12:03,922][105620] Updated weights for policy 1, policy_version 1761609 (0.0009) [2023-12-27 04:12:03,976][105620] Updated weights for policy 1, policy_version 1761619 (0.0008) [2023-12-27 04:12:04,040][105620] Updated weights for policy 1, policy_version 1761629 (0.0009) [2023-12-27 04:12:04,095][105620] Updated weights for policy 1, policy_version 1761639 (0.0009) [2023-12-27 04:12:04,586][105692] Updated weights for policy 0, policy_version 1757891 (0.0008) [2023-12-27 04:12:04,655][105692] Updated weights for policy 0, policy_version 1757901 (0.0009) [2023-12-27 04:12:04,714][105692] Updated weights for policy 0, policy_version 1757911 (0.0011) [2023-12-27 04:12:04,829][105620] Updated weights for policy 1, policy_version 1761649 (0.0008) [2023-12-27 04:12:04,874][105620] Updated weights for policy 1, policy_version 1761659 (0.0008) [2023-12-27 04:12:04,930][105620] Updated weights for policy 1, policy_version 1761669 (0.0009) [2023-12-27 04:12:05,415][105692] Updated weights for policy 0, policy_version 1757921 (0.0011) [2023-12-27 04:12:05,474][105692] Updated weights for policy 0, policy_version 1757931 (0.0011) [2023-12-27 04:12:05,534][105692] Updated weights for policy 0, policy_version 1757941 (0.0009) [2023-12-27 04:12:05,586][105692] Updated weights for policy 0, policy_version 1757951 (0.0011) [2023-12-27 04:12:05,679][105620] Updated weights for policy 1, policy_version 1761679 (0.0007) [2023-12-27 04:12:05,728][105620] Updated weights for policy 1, policy_version 1761689 (0.0005) [2023-12-27 04:12:05,778][105620] Updated weights for policy 1, policy_version 1761699 (0.0007) [2023-12-27 04:12:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 901160960. Throughput: 0: 9642.8, 1: 10039.0. Samples: 901149712. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:12:06,062][104569] Avg episode reward: [(0, '8077.839'), (1, '9171.362')] [2023-12-27 04:12:06,333][105692] Updated weights for policy 0, policy_version 1757961 (0.0011) [2023-12-27 04:12:06,386][105692] Updated weights for policy 0, policy_version 1757971 (0.0011) [2023-12-27 04:12:06,429][105620] Updated weights for policy 1, policy_version 1761709 (0.0008) [2023-12-27 04:12:06,449][105692] Updated weights for policy 0, policy_version 1757981 (0.0011) [2023-12-27 04:12:06,492][105620] Updated weights for policy 1, policy_version 1761719 (0.0009) [2023-12-27 04:12:06,555][105620] Updated weights for policy 1, policy_version 1761729 (0.0009) [2023-12-27 04:12:07,114][105692] Updated weights for policy 0, policy_version 1757991 (0.0007) [2023-12-27 04:12:07,176][105692] Updated weights for policy 0, policy_version 1758001 (0.0005) [2023-12-27 04:12:07,242][105692] Updated weights for policy 0, policy_version 1758011 (0.0005) [2023-12-27 04:12:07,331][105620] Updated weights for policy 1, policy_version 1761739 (0.0008) [2023-12-27 04:12:07,383][105620] Updated weights for policy 1, policy_version 1761749 (0.0006) [2023-12-27 04:12:07,453][105620] Updated weights for policy 1, policy_version 1761759 (0.0005) [2023-12-27 04:12:07,779][105692] Updated weights for policy 0, policy_version 1758021 (0.0007) [2023-12-27 04:12:07,828][105692] Updated weights for policy 0, policy_version 1758031 (0.0010) [2023-12-27 04:12:07,884][105692] Updated weights for policy 0, policy_version 1758041 (0.0010) [2023-12-27 04:12:07,980][105620] Updated weights for policy 1, policy_version 1761769 (0.0005) [2023-12-27 04:12:08,044][105620] Updated weights for policy 1, policy_version 1761779 (0.0008) [2023-12-27 04:12:08,108][105620] Updated weights for policy 1, policy_version 1761789 (0.0008) [2023-12-27 04:12:08,176][105620] Updated weights for policy 1, policy_version 1761799 (0.0008) [2023-12-27 04:12:08,631][105692] Updated weights for policy 0, policy_version 1758051 (0.0009) [2023-12-27 04:12:08,694][105692] Updated weights for policy 0, policy_version 1758061 (0.0008) [2023-12-27 04:12:08,748][105692] Updated weights for policy 0, policy_version 1758071 (0.0009) [2023-12-27 04:12:08,939][105620] Updated weights for policy 1, policy_version 1761809 (0.0009) [2023-12-27 04:12:09,001][105620] Updated weights for policy 1, policy_version 1761819 (0.0009) [2023-12-27 04:12:09,055][105620] Updated weights for policy 1, policy_version 1761829 (0.0008) [2023-12-27 04:12:09,496][105692] Updated weights for policy 0, policy_version 1758081 (0.0008) [2023-12-27 04:12:09,551][105692] Updated weights for policy 0, policy_version 1758091 (0.0009) [2023-12-27 04:12:09,613][105692] Updated weights for policy 0, policy_version 1758101 (0.0009) [2023-12-27 04:12:09,675][105692] Updated weights for policy 0, policy_version 1758111 (0.0009) [2023-12-27 04:12:09,841][105620] Updated weights for policy 1, policy_version 1761839 (0.0010) [2023-12-27 04:12:09,918][105620] Updated weights for policy 1, policy_version 1761849 (0.0010) [2023-12-27 04:12:09,980][105620] Updated weights for policy 1, policy_version 1761859 (0.0009) [2023-12-27 04:12:10,367][105692] Updated weights for policy 0, policy_version 1758121 (0.0009) [2023-12-27 04:12:10,436][105692] Updated weights for policy 0, policy_version 1758131 (0.0009) [2023-12-27 04:12:10,505][105692] Updated weights for policy 0, policy_version 1758141 (0.0008) [2023-12-27 04:12:10,753][105620] Updated weights for policy 1, policy_version 1761869 (0.0009) [2023-12-27 04:12:10,810][105620] Updated weights for policy 1, policy_version 1761879 (0.0010) [2023-12-27 04:12:10,865][105620] Updated weights for policy 1, policy_version 1761889 (0.0010) [2023-12-27 04:12:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 901259264. Throughput: 0: 9701.7, 1: 10041.0. Samples: 901266464. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:12:11,062][104569] Avg episode reward: [(0, '8805.725'), (1, '9262.405')] [2023-12-27 04:12:11,208][105692] Updated weights for policy 0, policy_version 1758151 (0.0007) [2023-12-27 04:12:11,267][105692] Updated weights for policy 0, policy_version 1758161 (0.0007) [2023-12-27 04:12:11,326][105692] Updated weights for policy 0, policy_version 1758171 (0.0008) [2023-12-27 04:12:11,654][105620] Updated weights for policy 1, policy_version 1761899 (0.0009) [2023-12-27 04:12:11,726][105620] Updated weights for policy 1, policy_version 1761909 (0.0007) [2023-12-27 04:12:11,794][105620] Updated weights for policy 1, policy_version 1761919 (0.0009) [2023-12-27 04:12:12,075][105692] Updated weights for policy 0, policy_version 1758181 (0.0008) [2023-12-27 04:12:12,132][105692] Updated weights for policy 0, policy_version 1758191 (0.0008) [2023-12-27 04:12:12,185][105692] Updated weights for policy 0, policy_version 1758201 (0.0008) [2023-12-27 04:12:12,530][105620] Updated weights for policy 1, policy_version 1761929 (0.0009) [2023-12-27 04:12:12,581][105620] Updated weights for policy 1, policy_version 1761939 (0.0010) [2023-12-27 04:12:12,638][105620] Updated weights for policy 1, policy_version 1761949 (0.0010) [2023-12-27 04:12:12,704][105620] Updated weights for policy 1, policy_version 1761959 (0.0010) [2023-12-27 04:12:12,958][105692] Updated weights for policy 0, policy_version 1758211 (0.0008) [2023-12-27 04:12:13,014][105692] Updated weights for policy 0, policy_version 1758221 (0.0008) [2023-12-27 04:12:13,071][105692] Updated weights for policy 0, policy_version 1758231 (0.0007) [2023-12-27 04:12:13,382][105620] Updated weights for policy 1, policy_version 1761969 (0.0010) [2023-12-27 04:12:13,440][105620] Updated weights for policy 1, policy_version 1761979 (0.0010) [2023-12-27 04:12:13,494][105620] Updated weights for policy 1, policy_version 1761989 (0.0010) [2023-12-27 04:12:13,782][105692] Updated weights for policy 0, policy_version 1758241 (0.0009) [2023-12-27 04:12:13,844][105692] Updated weights for policy 0, policy_version 1758251 (0.0008) [2023-12-27 04:12:13,897][105692] Updated weights for policy 0, policy_version 1758261 (0.0008) [2023-12-27 04:12:13,943][105692] Updated weights for policy 0, policy_version 1758271 (0.0007) [2023-12-27 04:12:14,230][105620] Updated weights for policy 1, policy_version 1761999 (0.0010) [2023-12-27 04:12:14,282][105620] Updated weights for policy 1, policy_version 1762009 (0.0006) [2023-12-27 04:12:14,332][105620] Updated weights for policy 1, policy_version 1762019 (0.0010) [2023-12-27 04:12:14,714][105692] Updated weights for policy 0, policy_version 1758281 (0.0008) [2023-12-27 04:12:14,773][105692] Updated weights for policy 0, policy_version 1758291 (0.0008) [2023-12-27 04:12:14,840][105692] Updated weights for policy 0, policy_version 1758301 (0.0009) [2023-12-27 04:12:15,077][105620] Updated weights for policy 1, policy_version 1762029 (0.0010) [2023-12-27 04:12:15,143][105620] Updated weights for policy 1, policy_version 1762039 (0.0010) [2023-12-27 04:12:15,204][105620] Updated weights for policy 1, policy_version 1762049 (0.0010) [2023-12-27 04:12:15,614][105692] Updated weights for policy 0, policy_version 1758311 (0.0009) [2023-12-27 04:12:15,662][105692] Updated weights for policy 0, policy_version 1758321 (0.0008) [2023-12-27 04:12:15,707][105692] Updated weights for policy 0, policy_version 1758331 (0.0008) [2023-12-27 04:12:15,944][105620] Updated weights for policy 1, policy_version 1762059 (0.0010) [2023-12-27 04:12:16,002][105620] Updated weights for policy 1, policy_version 1762069 (0.0009) [2023-12-27 04:12:16,061][105620] Updated weights for policy 1, policy_version 1762079 (0.0007) [2023-12-27 04:12:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.4, 300 sec: 19549.7). Total num frames: 901349376. Throughput: 0: 9636.8, 1: 9880.6. Samples: 901323556. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:12:16,062][104569] Avg episode reward: [(0, '8714.511'), (1, '8987.449')] [2023-12-27 04:12:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001758336_450199552.pth... [2023-12-27 04:12:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001757216_449912832.pth [2023-12-27 04:12:16,110][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001762088_451158016.pth... [2023-12-27 04:12:16,115][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001760936_450863104.pth [2023-12-27 04:12:16,511][105692] Updated weights for policy 0, policy_version 1758341 (0.0008) [2023-12-27 04:12:16,582][105692] Updated weights for policy 0, policy_version 1758351 (0.0006) [2023-12-27 04:12:16,646][105692] Updated weights for policy 0, policy_version 1758361 (0.0006) [2023-12-27 04:12:16,688][105620] Updated weights for policy 1, policy_version 1762089 (0.0010) [2023-12-27 04:12:16,740][105620] Updated weights for policy 1, policy_version 1762099 (0.0010) [2023-12-27 04:12:16,795][105620] Updated weights for policy 1, policy_version 1762109 (0.0010) [2023-12-27 04:12:16,852][105620] Updated weights for policy 1, policy_version 1762119 (0.0009) [2023-12-27 04:12:17,353][105692] Updated weights for policy 0, policy_version 1758371 (0.0008) [2023-12-27 04:12:17,405][105692] Updated weights for policy 0, policy_version 1758381 (0.0007) [2023-12-27 04:12:17,456][105692] Updated weights for policy 0, policy_version 1758391 (0.0008) [2023-12-27 04:12:17,611][105620] Updated weights for policy 1, policy_version 1762129 (0.0010) [2023-12-27 04:12:17,674][105620] Updated weights for policy 1, policy_version 1762139 (0.0008) [2023-12-27 04:12:17,736][105620] Updated weights for policy 1, policy_version 1762149 (0.0005) [2023-12-27 04:12:18,210][105692] Updated weights for policy 0, policy_version 1758401 (0.0007) [2023-12-27 04:12:18,272][105692] Updated weights for policy 0, policy_version 1758411 (0.0009) [2023-12-27 04:12:18,333][105692] Updated weights for policy 0, policy_version 1758421 (0.0008) [2023-12-27 04:12:18,348][105620] Updated weights for policy 1, policy_version 1762159 (0.0006) [2023-12-27 04:12:18,393][105692] Updated weights for policy 0, policy_version 1758431 (0.0006) [2023-12-27 04:12:18,407][105620] Updated weights for policy 1, policy_version 1762169 (0.0008) [2023-12-27 04:12:18,464][105620] Updated weights for policy 1, policy_version 1762179 (0.0009) [2023-12-27 04:12:19,166][105692] Updated weights for policy 0, policy_version 1758441 (0.0006) [2023-12-27 04:12:19,199][105620] Updated weights for policy 1, policy_version 1762189 (0.0009) [2023-12-27 04:12:19,217][105692] Updated weights for policy 0, policy_version 1758451 (0.0006) [2023-12-27 04:12:19,268][105620] Updated weights for policy 1, policy_version 1762199 (0.0008) [2023-12-27 04:12:19,289][105692] Updated weights for policy 0, policy_version 1758461 (0.0006) [2023-12-27 04:12:19,335][105620] Updated weights for policy 1, policy_version 1762209 (0.0007) [2023-12-27 04:12:19,913][105692] Updated weights for policy 0, policy_version 1758471 (0.0009) [2023-12-27 04:12:19,972][105692] Updated weights for policy 0, policy_version 1758481 (0.0010) [2023-12-27 04:12:20,035][105692] Updated weights for policy 0, policy_version 1758491 (0.0009) [2023-12-27 04:12:20,099][105620] Updated weights for policy 1, policy_version 1762219 (0.0008) [2023-12-27 04:12:20,169][105620] Updated weights for policy 1, policy_version 1762229 (0.0007) [2023-12-27 04:12:20,232][105620] Updated weights for policy 1, policy_version 1762239 (0.0010) [2023-12-27 04:12:20,792][105692] Updated weights for policy 0, policy_version 1758501 (0.0008) [2023-12-27 04:12:20,852][105692] Updated weights for policy 0, policy_version 1758511 (0.0008) [2023-12-27 04:12:20,909][105692] Updated weights for policy 0, policy_version 1758521 (0.0006) [2023-12-27 04:12:20,951][105620] Updated weights for policy 1, policy_version 1762249 (0.0010) [2023-12-27 04:12:21,015][105620] Updated weights for policy 1, policy_version 1762259 (0.0011) [2023-12-27 04:12:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 901447680. Throughput: 0: 9543.3, 1: 9751.7. Samples: 901437844. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:12:21,063][104569] Avg episode reward: [(0, '8255.555'), (1, '8986.426')] [2023-12-27 04:12:21,086][105620] Updated weights for policy 1, policy_version 1762269 (0.0011) [2023-12-27 04:12:21,157][105620] Updated weights for policy 1, policy_version 1762279 (0.0009) [2023-12-27 04:12:21,708][105692] Updated weights for policy 0, policy_version 1758531 (0.0007) [2023-12-27 04:12:21,764][105692] Updated weights for policy 0, policy_version 1758541 (0.0009) [2023-12-27 04:12:21,818][105692] Updated weights for policy 0, policy_version 1758551 (0.0008) [2023-12-27 04:12:21,882][105620] Updated weights for policy 1, policy_version 1762289 (0.0010) [2023-12-27 04:12:21,938][105620] Updated weights for policy 1, policy_version 1762299 (0.0010) [2023-12-27 04:12:21,991][105620] Updated weights for policy 1, policy_version 1762309 (0.0010) [2023-12-27 04:12:22,515][105692] Updated weights for policy 0, policy_version 1758561 (0.0007) [2023-12-27 04:12:22,578][105692] Updated weights for policy 0, policy_version 1758571 (0.0009) [2023-12-27 04:12:22,634][105692] Updated weights for policy 0, policy_version 1758581 (0.0009) [2023-12-27 04:12:22,689][105692] Updated weights for policy 0, policy_version 1758591 (0.0010) [2023-12-27 04:12:22,751][105620] Updated weights for policy 1, policy_version 1762319 (0.0007) [2023-12-27 04:12:22,808][105620] Updated weights for policy 1, policy_version 1762329 (0.0007) [2023-12-27 04:12:22,872][105620] Updated weights for policy 1, policy_version 1762339 (0.0007) [2023-12-27 04:12:23,403][105620] Updated weights for policy 1, policy_version 1762349 (0.0007) [2023-12-27 04:12:23,461][105620] Updated weights for policy 1, policy_version 1762359 (0.0009) [2023-12-27 04:12:23,516][105620] Updated weights for policy 1, policy_version 1762369 (0.0008) [2023-12-27 04:12:23,570][105692] Updated weights for policy 0, policy_version 1758601 (0.0007) [2023-12-27 04:12:23,634][105692] Updated weights for policy 0, policy_version 1758611 (0.0010) [2023-12-27 04:12:23,696][105692] Updated weights for policy 0, policy_version 1758621 (0.0009) [2023-12-27 04:12:24,276][105620] Updated weights for policy 1, policy_version 1762379 (0.0008) [2023-12-27 04:12:24,337][105620] Updated weights for policy 1, policy_version 1762389 (0.0009) [2023-12-27 04:12:24,384][105620] Updated weights for policy 1, policy_version 1762399 (0.0008) [2023-12-27 04:12:24,390][105692] Updated weights for policy 0, policy_version 1758631 (0.0007) [2023-12-27 04:12:24,447][105692] Updated weights for policy 0, policy_version 1758641 (0.0008) [2023-12-27 04:12:24,497][105692] Updated weights for policy 0, policy_version 1758651 (0.0009) [2023-12-27 04:12:25,160][105620] Updated weights for policy 1, policy_version 1762409 (0.0008) [2023-12-27 04:12:25,205][105692] Updated weights for policy 0, policy_version 1758661 (0.0007) [2023-12-27 04:12:25,208][105620] Updated weights for policy 1, policy_version 1762419 (0.0009) [2023-12-27 04:12:25,256][105692] Updated weights for policy 0, policy_version 1758671 (0.0005) [2023-12-27 04:12:25,267][105620] Updated weights for policy 1, policy_version 1762429 (0.0008) [2023-12-27 04:12:25,297][105692] Updated weights for policy 0, policy_version 1758681 (0.0005) [2023-12-27 04:12:25,321][105620] Updated weights for policy 1, policy_version 1762439 (0.0008) [2023-12-27 04:12:25,973][105692] Updated weights for policy 0, policy_version 1758691 (0.0006) [2023-12-27 04:12:26,029][105692] Updated weights for policy 0, policy_version 1758701 (0.0005) [2023-12-27 04:12:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 901537792. Throughput: 0: 9547.1, 1: 9714.1. Samples: 901552216. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:12:26,062][104569] Avg episode reward: [(0, '8618.788'), (1, '9168.655')] [2023-12-27 04:12:26,100][105692] Updated weights for policy 0, policy_version 1758711 (0.0005) [2023-12-27 04:12:26,143][105620] Updated weights for policy 1, policy_version 1762449 (0.0007) [2023-12-27 04:12:26,195][105620] Updated weights for policy 1, policy_version 1762459 (0.0008) [2023-12-27 04:12:26,255][105620] Updated weights for policy 1, policy_version 1762469 (0.0008) [2023-12-27 04:12:26,793][105692] Updated weights for policy 0, policy_version 1758721 (0.0007) [2023-12-27 04:12:26,846][105692] Updated weights for policy 0, policy_version 1758731 (0.0006) [2023-12-27 04:12:26,904][105692] Updated weights for policy 0, policy_version 1758741 (0.0007) [2023-12-27 04:12:26,955][105692] Updated weights for policy 0, policy_version 1758751 (0.0009) [2023-12-27 04:12:27,022][105620] Updated weights for policy 1, policy_version 1762479 (0.0009) [2023-12-27 04:12:27,074][105620] Updated weights for policy 1, policy_version 1762489 (0.0008) [2023-12-27 04:12:27,120][105620] Updated weights for policy 1, policy_version 1762499 (0.0008) [2023-12-27 04:12:27,685][105692] Updated weights for policy 0, policy_version 1758761 (0.0009) [2023-12-27 04:12:27,744][105692] Updated weights for policy 0, policy_version 1758771 (0.0009) [2023-12-27 04:12:27,810][105692] Updated weights for policy 0, policy_version 1758781 (0.0009) [2023-12-27 04:12:27,877][105620] Updated weights for policy 1, policy_version 1762509 (0.0010) [2023-12-27 04:12:27,935][105620] Updated weights for policy 1, policy_version 1762519 (0.0005) [2023-12-27 04:12:27,987][105620] Updated weights for policy 1, policy_version 1762529 (0.0007) [2023-12-27 04:12:28,572][105692] Updated weights for policy 0, policy_version 1758791 (0.0008) [2023-12-27 04:12:28,625][105692] Updated weights for policy 0, policy_version 1758801 (0.0008) [2023-12-27 04:12:28,670][105692] Updated weights for policy 0, policy_version 1758811 (0.0008) [2023-12-27 04:12:28,729][105620] Updated weights for policy 1, policy_version 1762539 (0.0010) [2023-12-27 04:12:28,783][105620] Updated weights for policy 1, policy_version 1762549 (0.0010) [2023-12-27 04:12:28,836][105620] Updated weights for policy 1, policy_version 1762559 (0.0011) [2023-12-27 04:12:29,488][105692] Updated weights for policy 0, policy_version 1758821 (0.0008) [2023-12-27 04:12:29,533][105692] Updated weights for policy 0, policy_version 1758831 (0.0007) [2023-12-27 04:12:29,593][105692] Updated weights for policy 0, policy_version 1758841 (0.0008) [2023-12-27 04:12:29,608][105620] Updated weights for policy 1, policy_version 1762569 (0.0010) [2023-12-27 04:12:29,670][105620] Updated weights for policy 1, policy_version 1762579 (0.0011) [2023-12-27 04:12:29,726][105620] Updated weights for policy 1, policy_version 1762589 (0.0011) [2023-12-27 04:12:29,787][105620] Updated weights for policy 1, policy_version 1762599 (0.0010) [2023-12-27 04:12:30,416][105692] Updated weights for policy 0, policy_version 1758851 (0.0007) [2023-12-27 04:12:30,456][105620] Updated weights for policy 1, policy_version 1762609 (0.0008) [2023-12-27 04:12:30,476][105692] Updated weights for policy 0, policy_version 1758861 (0.0007) [2023-12-27 04:12:30,501][105620] Updated weights for policy 1, policy_version 1762619 (0.0006) [2023-12-27 04:12:30,542][105692] Updated weights for policy 0, policy_version 1758871 (0.0008) [2023-12-27 04:12:30,549][105620] Updated weights for policy 1, policy_version 1762629 (0.0007) [2023-12-27 04:12:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 901636096. Throughput: 0: 9527.3, 1: 9702.2. Samples: 901608696. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:12:31,062][104569] Avg episode reward: [(0, '8802.344'), (1, '9168.666')] [2023-12-27 04:12:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001758880_450338816.pth... [2023-12-27 04:12:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001762632_451297280.pth... [2023-12-27 04:12:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001757760_450052096.pth [2023-12-27 04:12:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001761512_451010560.pth [2023-12-27 04:12:31,261][105620] Updated weights for policy 1, policy_version 1762639 (0.0007) [2023-12-27 04:12:31,328][105620] Updated weights for policy 1, policy_version 1762649 (0.0006) [2023-12-27 04:12:31,363][105692] Updated weights for policy 0, policy_version 1758881 (0.0008) [2023-12-27 04:12:31,398][105620] Updated weights for policy 1, policy_version 1762659 (0.0007) [2023-12-27 04:12:31,428][105692] Updated weights for policy 0, policy_version 1758891 (0.0009) [2023-12-27 04:12:31,485][105692] Updated weights for policy 0, policy_version 1758901 (0.0010) [2023-12-27 04:12:31,547][105692] Updated weights for policy 0, policy_version 1758911 (0.0010) [2023-12-27 04:12:32,070][105620] Updated weights for policy 1, policy_version 1762669 (0.0007) [2023-12-27 04:12:32,126][105620] Updated weights for policy 1, policy_version 1762679 (0.0006) [2023-12-27 04:12:32,186][105620] Updated weights for policy 1, policy_version 1762689 (0.0006) [2023-12-27 04:12:32,354][105692] Updated weights for policy 0, policy_version 1758921 (0.0009) [2023-12-27 04:12:32,417][105692] Updated weights for policy 0, policy_version 1758931 (0.0007) [2023-12-27 04:12:32,486][105692] Updated weights for policy 0, policy_version 1758941 (0.0005) [2023-12-27 04:12:32,882][105620] Updated weights for policy 1, policy_version 1762699 (0.0006) [2023-12-27 04:12:32,939][105620] Updated weights for policy 1, policy_version 1762709 (0.0006) [2023-12-27 04:12:32,988][105620] Updated weights for policy 1, policy_version 1762719 (0.0006) [2023-12-27 04:12:33,242][105692] Updated weights for policy 0, policy_version 1758951 (0.0009) [2023-12-27 04:12:33,304][105692] Updated weights for policy 0, policy_version 1758961 (0.0010) [2023-12-27 04:12:33,366][105692] Updated weights for policy 0, policy_version 1758971 (0.0010) [2023-12-27 04:12:33,504][105620] Updated weights for policy 1, policy_version 1762729 (0.0007) [2023-12-27 04:12:33,552][105620] Updated weights for policy 1, policy_version 1762739 (0.0008) [2023-12-27 04:12:33,608][105620] Updated weights for policy 1, policy_version 1762749 (0.0009) [2023-12-27 04:12:33,659][105620] Updated weights for policy 1, policy_version 1762759 (0.0010) [2023-12-27 04:12:34,205][105692] Updated weights for policy 0, policy_version 1758981 (0.0010) [2023-12-27 04:12:34,262][105692] Updated weights for policy 0, policy_version 1758991 (0.0009) [2023-12-27 04:12:34,314][105620] Updated weights for policy 1, policy_version 1762769 (0.0006) [2023-12-27 04:12:34,319][105692] Updated weights for policy 0, policy_version 1759001 (0.0009) [2023-12-27 04:12:34,370][105620] Updated weights for policy 1, policy_version 1762779 (0.0007) [2023-12-27 04:12:34,420][105620] Updated weights for policy 1, policy_version 1762789 (0.0010) [2023-12-27 04:12:35,041][105692] Updated weights for policy 0, policy_version 1759011 (0.0009) [2023-12-27 04:12:35,093][105692] Updated weights for policy 0, policy_version 1759021 (0.0006) [2023-12-27 04:12:35,133][105620] Updated weights for policy 1, policy_version 1762799 (0.0007) [2023-12-27 04:12:35,142][105692] Updated weights for policy 0, policy_version 1759031 (0.0005) [2023-12-27 04:12:35,177][105620] Updated weights for policy 1, policy_version 1762809 (0.0005) [2023-12-27 04:12:35,226][105620] Updated weights for policy 1, policy_version 1762819 (0.0005) [2023-12-27 04:12:35,800][105620] Updated weights for policy 1, policy_version 1762829 (0.0005) [2023-12-27 04:12:35,854][105620] Updated weights for policy 1, policy_version 1762839 (0.0005) [2023-12-27 04:12:35,912][105620] Updated weights for policy 1, policy_version 1762849 (0.0005) [2023-12-27 04:12:35,914][105692] Updated weights for policy 0, policy_version 1759041 (0.0006) [2023-12-27 04:12:35,972][105692] Updated weights for policy 0, policy_version 1759051 (0.0009) [2023-12-27 04:12:36,030][105692] Updated weights for policy 0, policy_version 1759061 (0.0011) [2023-12-27 04:12:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 901734400. Throughput: 0: 9340.1, 1: 9796.2. Samples: 901722828. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:12:36,062][104569] Avg episode reward: [(0, '8623.882'), (1, '9353.571')] [2023-12-27 04:12:36,085][105692] Updated weights for policy 0, policy_version 1759072 (0.0010) [2023-12-27 04:12:36,602][105620] Updated weights for policy 1, policy_version 1762859 (0.0006) [2023-12-27 04:12:36,663][105620] Updated weights for policy 1, policy_version 1762869 (0.0009) [2023-12-27 04:12:36,720][105620] Updated weights for policy 1, policy_version 1762879 (0.0008) [2023-12-27 04:12:36,803][105692] Updated weights for policy 0, policy_version 1759082 (0.0008) [2023-12-27 04:12:36,850][105692] Updated weights for policy 0, policy_version 1759092 (0.0007) [2023-12-27 04:12:36,906][105692] Updated weights for policy 0, policy_version 1759102 (0.0009) [2023-12-27 04:12:37,348][105620] Updated weights for policy 1, policy_version 1762889 (0.0008) [2023-12-27 04:12:37,403][105620] Updated weights for policy 1, policy_version 1762899 (0.0009) [2023-12-27 04:12:37,468][105620] Updated weights for policy 1, policy_version 1762909 (0.0008) [2023-12-27 04:12:37,518][105620] Updated weights for policy 1, policy_version 1762919 (0.0008) [2023-12-27 04:12:37,815][105692] Updated weights for policy 0, policy_version 1759112 (0.0010) [2023-12-27 04:12:37,871][105692] Updated weights for policy 0, policy_version 1759122 (0.0009) [2023-12-27 04:12:37,922][105692] Updated weights for policy 0, policy_version 1759132 (0.0009) [2023-12-27 04:12:38,127][105620] Updated weights for policy 1, policy_version 1762929 (0.0010) [2023-12-27 04:12:38,183][105620] Updated weights for policy 1, policy_version 1762939 (0.0010) [2023-12-27 04:12:38,242][105620] Updated weights for policy 1, policy_version 1762949 (0.0011) [2023-12-27 04:12:38,673][105692] Updated weights for policy 0, policy_version 1759142 (0.0009) [2023-12-27 04:12:38,722][105692] Updated weights for policy 0, policy_version 1759152 (0.0008) [2023-12-27 04:12:38,774][105692] Updated weights for policy 0, policy_version 1759162 (0.0008) [2023-12-27 04:12:39,042][105620] Updated weights for policy 1, policy_version 1762959 (0.0011) [2023-12-27 04:12:39,100][105620] Updated weights for policy 1, policy_version 1762969 (0.0010) [2023-12-27 04:12:39,148][105620] Updated weights for policy 1, policy_version 1762979 (0.0010) [2023-12-27 04:12:39,573][105692] Updated weights for policy 0, policy_version 1759172 (0.0009) [2023-12-27 04:12:39,636][105692] Updated weights for policy 0, policy_version 1759182 (0.0010) [2023-12-27 04:12:39,690][105692] Updated weights for policy 0, policy_version 1759192 (0.0009) [2023-12-27 04:12:39,799][105620] Updated weights for policy 1, policy_version 1762989 (0.0008) [2023-12-27 04:12:39,860][105620] Updated weights for policy 1, policy_version 1762999 (0.0011) [2023-12-27 04:12:39,924][105620] Updated weights for policy 1, policy_version 1763009 (0.0010) [2023-12-27 04:12:40,500][105692] Updated weights for policy 0, policy_version 1759202 (0.0010) [2023-12-27 04:12:40,566][105692] Updated weights for policy 0, policy_version 1759213 (0.0007) [2023-12-27 04:12:40,629][105692] Updated weights for policy 0, policy_version 1759223 (0.0011) [2023-12-27 04:12:40,671][105620] Updated weights for policy 1, policy_version 1763019 (0.0009) [2023-12-27 04:12:40,730][105620] Updated weights for policy 1, policy_version 1763029 (0.0009) [2023-12-27 04:12:40,794][105620] Updated weights for policy 1, policy_version 1763039 (0.0010) [2023-12-27 04:12:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 901832704. Throughput: 0: 9328.4, 1: 9821.3. Samples: 901838660. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:12:41,062][104569] Avg episode reward: [(0, '8627.470'), (1, '9353.444')] [2023-12-27 04:12:41,246][105692] Updated weights for policy 0, policy_version 1759233 (0.0009) [2023-12-27 04:12:41,306][105692] Updated weights for policy 0, policy_version 1759243 (0.0008) [2023-12-27 04:12:41,376][105692] Updated weights for policy 0, policy_version 1759253 (0.0008) [2023-12-27 04:12:41,438][105692] Updated weights for policy 0, policy_version 1759263 (0.0009) [2023-12-27 04:12:41,591][105620] Updated weights for policy 1, policy_version 1763049 (0.0009) [2023-12-27 04:12:41,661][105620] Updated weights for policy 1, policy_version 1763060 (0.0007) [2023-12-27 04:12:41,726][105620] Updated weights for policy 1, policy_version 1763070 (0.0009) [2023-12-27 04:12:41,791][105620] Updated weights for policy 1, policy_version 1763080 (0.0011) [2023-12-27 04:12:42,173][105692] Updated weights for policy 0, policy_version 1759273 (0.0010) [2023-12-27 04:12:42,232][105692] Updated weights for policy 0, policy_version 1759283 (0.0010) [2023-12-27 04:12:42,297][105692] Updated weights for policy 0, policy_version 1759293 (0.0011) [2023-12-27 04:12:42,559][105620] Updated weights for policy 1, policy_version 1763090 (0.0011) [2023-12-27 04:12:42,619][105620] Updated weights for policy 1, policy_version 1763100 (0.0011) [2023-12-27 04:12:42,668][105620] Updated weights for policy 1, policy_version 1763110 (0.0010) [2023-12-27 04:12:42,979][105692] Updated weights for policy 0, policy_version 1759303 (0.0007) [2023-12-27 04:12:43,039][105692] Updated weights for policy 0, policy_version 1759313 (0.0005) [2023-12-27 04:12:43,106][105692] Updated weights for policy 0, policy_version 1759323 (0.0009) [2023-12-27 04:12:43,446][105620] Updated weights for policy 1, policy_version 1763120 (0.0009) [2023-12-27 04:12:43,508][105620] Updated weights for policy 1, policy_version 1763130 (0.0009) [2023-12-27 04:12:43,564][105620] Updated weights for policy 1, policy_version 1763140 (0.0010) [2023-12-27 04:12:43,688][105692] Updated weights for policy 0, policy_version 1759333 (0.0006) [2023-12-27 04:12:43,741][105692] Updated weights for policy 0, policy_version 1759343 (0.0008) [2023-12-27 04:12:43,802][105692] Updated weights for policy 0, policy_version 1759353 (0.0009) [2023-12-27 04:12:44,282][105620] Updated weights for policy 1, policy_version 1763150 (0.0007) [2023-12-27 04:12:44,349][105620] Updated weights for policy 1, policy_version 1763160 (0.0005) [2023-12-27 04:12:44,404][105620] Updated weights for policy 1, policy_version 1763170 (0.0005) [2023-12-27 04:12:44,477][105692] Updated weights for policy 0, policy_version 1759363 (0.0009) [2023-12-27 04:12:44,537][105692] Updated weights for policy 0, policy_version 1759373 (0.0008) [2023-12-27 04:12:44,581][105692] Updated weights for policy 0, policy_version 1759383 (0.0006) [2023-12-27 04:12:45,055][105620] Updated weights for policy 1, policy_version 1763180 (0.0007) [2023-12-27 04:12:45,115][105620] Updated weights for policy 1, policy_version 1763190 (0.0006) [2023-12-27 04:12:45,174][105620] Updated weights for policy 1, policy_version 1763200 (0.0008) [2023-12-27 04:12:45,345][105692] Updated weights for policy 0, policy_version 1759393 (0.0006) [2023-12-27 04:12:45,401][105692] Updated weights for policy 0, policy_version 1759403 (0.0009) [2023-12-27 04:12:45,456][105692] Updated weights for policy 0, policy_version 1759413 (0.0009) [2023-12-27 04:12:45,510][105692] Updated weights for policy 0, policy_version 1759423 (0.0008) [2023-12-27 04:12:45,846][105620] Updated weights for policy 1, policy_version 1763210 (0.0009) [2023-12-27 04:12:45,897][105620] Updated weights for policy 1, policy_version 1763220 (0.0008) [2023-12-27 04:12:45,954][105620] Updated weights for policy 1, policy_version 1763230 (0.0010) [2023-12-27 04:12:46,007][105620] Updated weights for policy 1, policy_version 1763240 (0.0010) [2023-12-27 04:12:46,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19251.1, 300 sec: 19521.9). Total num frames: 901931008. Throughput: 0: 9411.6, 1: 9776.8. Samples: 901897200. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:12:46,063][104569] Avg episode reward: [(0, '8082.945'), (1, '9353.432')] [2023-12-27 04:12:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001763240_451452928.pth... [2023-12-27 04:12:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001759424_450478080.pth... [2023-12-27 04:12:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001762088_451158016.pth [2023-12-27 04:12:46,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001758336_450199552.pth [2023-12-27 04:12:46,205][105692] Updated weights for policy 0, policy_version 1759433 (0.0009) [2023-12-27 04:12:46,265][105692] Updated weights for policy 0, policy_version 1759443 (0.0008) [2023-12-27 04:12:46,325][105692] Updated weights for policy 0, policy_version 1759453 (0.0009) [2023-12-27 04:12:46,697][105620] Updated weights for policy 1, policy_version 1763250 (0.0008) [2023-12-27 04:12:46,742][105620] Updated weights for policy 1, policy_version 1763260 (0.0007) [2023-12-27 04:12:46,791][105620] Updated weights for policy 1, policy_version 1763270 (0.0008) [2023-12-27 04:12:47,056][105692] Updated weights for policy 0, policy_version 1759463 (0.0007) [2023-12-27 04:12:47,114][105692] Updated weights for policy 0, policy_version 1759473 (0.0006) [2023-12-27 04:12:47,174][105692] Updated weights for policy 0, policy_version 1759483 (0.0006) [2023-12-27 04:12:47,586][105620] Updated weights for policy 1, policy_version 1763280 (0.0008) [2023-12-27 04:12:47,648][105620] Updated weights for policy 1, policy_version 1763290 (0.0009) [2023-12-27 04:12:47,700][105620] Updated weights for policy 1, policy_version 1763300 (0.0008) [2023-12-27 04:12:47,707][105692] Updated weights for policy 0, policy_version 1759493 (0.0006) [2023-12-27 04:12:47,765][105692] Updated weights for policy 0, policy_version 1759503 (0.0008) [2023-12-27 04:12:47,818][105692] Updated weights for policy 0, policy_version 1759513 (0.0009) [2023-12-27 04:12:48,461][105620] Updated weights for policy 1, policy_version 1763310 (0.0008) [2023-12-27 04:12:48,519][105620] Updated weights for policy 1, policy_version 1763320 (0.0009) [2023-12-27 04:12:48,576][105620] Updated weights for policy 1, policy_version 1763330 (0.0009) [2023-12-27 04:12:48,591][105692] Updated weights for policy 0, policy_version 1759523 (0.0009) [2023-12-27 04:12:48,646][105692] Updated weights for policy 0, policy_version 1759534 (0.0009) [2023-12-27 04:12:48,701][105692] Updated weights for policy 0, policy_version 1759544 (0.0010) [2023-12-27 04:12:49,308][105620] Updated weights for policy 1, policy_version 1763340 (0.0007) [2023-12-27 04:12:49,373][105620] Updated weights for policy 1, policy_version 1763350 (0.0010) [2023-12-27 04:12:49,419][105692] Updated weights for policy 0, policy_version 1759555 (0.0009) [2023-12-27 04:12:49,442][105620] Updated weights for policy 1, policy_version 1763360 (0.0008) [2023-12-27 04:12:49,470][105692] Updated weights for policy 0, policy_version 1759565 (0.0006) [2023-12-27 04:12:49,517][105692] Updated weights for policy 0, policy_version 1759575 (0.0007) [2023-12-27 04:12:50,138][105620] Updated weights for policy 1, policy_version 1763370 (0.0008) [2023-12-27 04:12:50,188][105620] Updated weights for policy 1, policy_version 1763380 (0.0009) [2023-12-27 04:12:50,234][105620] Updated weights for policy 1, policy_version 1763390 (0.0008) [2023-12-27 04:12:50,280][105620] Updated weights for policy 1, policy_version 1763400 (0.0009) [2023-12-27 04:12:50,325][105692] Updated weights for policy 0, policy_version 1759585 (0.0009) [2023-12-27 04:12:50,385][105692] Updated weights for policy 0, policy_version 1759595 (0.0008) [2023-12-27 04:12:50,440][105692] Updated weights for policy 0, policy_version 1759605 (0.0010) [2023-12-27 04:12:50,495][105692] Updated weights for policy 0, policy_version 1759615 (0.0009) [2023-12-27 04:12:51,013][105620] Updated weights for policy 1, policy_version 1763410 (0.0008) [2023-12-27 04:12:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 902021120. Throughput: 0: 9487.1, 1: 9755.3. Samples: 902015620. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:12:51,063][104569] Avg episode reward: [(0, '8259.327'), (1, '9353.482')] [2023-12-27 04:12:51,083][105620] Updated weights for policy 1, policy_version 1763420 (0.0009) [2023-12-27 04:12:51,150][105620] Updated weights for policy 1, policy_version 1763430 (0.0008) [2023-12-27 04:12:51,302][105692] Updated weights for policy 0, policy_version 1759625 (0.0011) [2023-12-27 04:12:51,362][105692] Updated weights for policy 0, policy_version 1759635 (0.0011) [2023-12-27 04:12:51,424][105692] Updated weights for policy 0, policy_version 1759645 (0.0010) [2023-12-27 04:12:51,838][105620] Updated weights for policy 1, policy_version 1763440 (0.0008) [2023-12-27 04:12:51,901][105620] Updated weights for policy 1, policy_version 1763450 (0.0008) [2023-12-27 04:12:51,956][105620] Updated weights for policy 1, policy_version 1763460 (0.0008) [2023-12-27 04:12:52,182][105692] Updated weights for policy 0, policy_version 1759655 (0.0011) [2023-12-27 04:12:52,231][105692] Updated weights for policy 0, policy_version 1759665 (0.0010) [2023-12-27 04:12:52,289][105692] Updated weights for policy 0, policy_version 1759675 (0.0011) [2023-12-27 04:12:52,743][105620] Updated weights for policy 1, policy_version 1763470 (0.0009) [2023-12-27 04:12:52,807][105620] Updated weights for policy 1, policy_version 1763480 (0.0008) [2023-12-27 04:12:52,864][105620] Updated weights for policy 1, policy_version 1763490 (0.0009) [2023-12-27 04:12:53,065][105692] Updated weights for policy 0, policy_version 1759685 (0.0010) [2023-12-27 04:12:53,125][105692] Updated weights for policy 0, policy_version 1759695 (0.0010) [2023-12-27 04:12:53,181][105692] Updated weights for policy 0, policy_version 1759705 (0.0010) [2023-12-27 04:12:53,594][105620] Updated weights for policy 1, policy_version 1763500 (0.0007) [2023-12-27 04:12:53,655][105620] Updated weights for policy 1, policy_version 1763510 (0.0007) [2023-12-27 04:12:53,718][105620] Updated weights for policy 1, policy_version 1763520 (0.0011) [2023-12-27 04:12:53,826][105692] Updated weights for policy 0, policy_version 1759715 (0.0011) [2023-12-27 04:12:53,882][105692] Updated weights for policy 0, policy_version 1759725 (0.0011) [2023-12-27 04:12:53,940][105692] Updated weights for policy 0, policy_version 1759735 (0.0008) [2023-12-27 04:12:54,438][105620] Updated weights for policy 1, policy_version 1763530 (0.0010) [2023-12-27 04:12:54,500][105620] Updated weights for policy 1, policy_version 1763540 (0.0010) [2023-12-27 04:12:54,568][105620] Updated weights for policy 1, policy_version 1763550 (0.0010) [2023-12-27 04:12:54,629][105620] Updated weights for policy 1, policy_version 1763560 (0.0010) [2023-12-27 04:12:54,643][105692] Updated weights for policy 0, policy_version 1759745 (0.0006) [2023-12-27 04:12:54,692][105692] Updated weights for policy 0, policy_version 1759755 (0.0008) [2023-12-27 04:12:54,745][105692] Updated weights for policy 0, policy_version 1759765 (0.0008) [2023-12-27 04:12:54,790][105692] Updated weights for policy 0, policy_version 1759775 (0.0008) [2023-12-27 04:12:55,356][105620] Updated weights for policy 1, policy_version 1763570 (0.0010) [2023-12-27 04:12:55,413][105620] Updated weights for policy 1, policy_version 1763580 (0.0010) [2023-12-27 04:12:55,468][105620] Updated weights for policy 1, policy_version 1763590 (0.0010) [2023-12-27 04:12:55,573][105692] Updated weights for policy 0, policy_version 1759785 (0.0009) [2023-12-27 04:12:55,632][105692] Updated weights for policy 0, policy_version 1759795 (0.0008) [2023-12-27 04:12:55,681][105692] Updated weights for policy 0, policy_version 1759805 (0.0008) [2023-12-27 04:12:56,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19114.6, 300 sec: 19494.2). Total num frames: 902119424. Throughput: 0: 9418.0, 1: 9738.7. Samples: 902128520. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:12:56,063][104569] Avg episode reward: [(0, '8620.219'), (1, '9169.220')] [2023-12-27 04:12:56,212][105620] Updated weights for policy 1, policy_version 1763600 (0.0010) [2023-12-27 04:12:56,263][105620] Updated weights for policy 1, policy_version 1763610 (0.0006) [2023-12-27 04:12:56,314][105620] Updated weights for policy 1, policy_version 1763620 (0.0005) [2023-12-27 04:12:56,498][105692] Updated weights for policy 0, policy_version 1759815 (0.0010) [2023-12-27 04:12:56,564][105692] Updated weights for policy 0, policy_version 1759825 (0.0010) [2023-12-27 04:12:56,621][105692] Updated weights for policy 0, policy_version 1759835 (0.0009) [2023-12-27 04:12:56,858][105620] Updated weights for policy 1, policy_version 1763630 (0.0005) [2023-12-27 04:12:56,911][105620] Updated weights for policy 1, policy_version 1763640 (0.0005) [2023-12-27 04:12:56,958][105620] Updated weights for policy 1, policy_version 1763650 (0.0005) [2023-12-27 04:12:57,491][105692] Updated weights for policy 0, policy_version 1759845 (0.0009) [2023-12-27 04:12:57,554][105692] Updated weights for policy 0, policy_version 1759855 (0.0009) [2023-12-27 04:12:57,556][105620] Updated weights for policy 1, policy_version 1763660 (0.0007) [2023-12-27 04:12:57,600][105620] Updated weights for policy 1, policy_version 1763670 (0.0010) [2023-12-27 04:12:57,610][105692] Updated weights for policy 0, policy_version 1759865 (0.0006) [2023-12-27 04:12:57,651][105620] Updated weights for policy 1, policy_version 1763680 (0.0010) [2023-12-27 04:12:58,377][105692] Updated weights for policy 0, policy_version 1759875 (0.0006) [2023-12-27 04:12:58,438][105620] Updated weights for policy 1, policy_version 1763690 (0.0010) [2023-12-27 04:12:58,439][105692] Updated weights for policy 0, policy_version 1759885 (0.0008) [2023-12-27 04:12:58,501][105692] Updated weights for policy 0, policy_version 1759895 (0.0007) [2023-12-27 04:12:58,502][105620] Updated weights for policy 1, policy_version 1763700 (0.0007) [2023-12-27 04:12:58,570][105620] Updated weights for policy 1, policy_version 1763710 (0.0010) [2023-12-27 04:12:58,639][105620] Updated weights for policy 1, policy_version 1763720 (0.0010) [2023-12-27 04:12:59,334][105692] Updated weights for policy 0, policy_version 1759905 (0.0007) [2023-12-27 04:12:59,401][105692] Updated weights for policy 0, policy_version 1759915 (0.0011) [2023-12-27 04:12:59,450][105620] Updated weights for policy 1, policy_version 1763730 (0.0007) [2023-12-27 04:12:59,460][105692] Updated weights for policy 0, policy_version 1759925 (0.0010) [2023-12-27 04:12:59,499][105620] Updated weights for policy 1, policy_version 1763740 (0.0006) [2023-12-27 04:12:59,513][105692] Updated weights for policy 0, policy_version 1759935 (0.0010) [2023-12-27 04:12:59,558][105620] Updated weights for policy 1, policy_version 1763750 (0.0006) [2023-12-27 04:13:00,207][105692] Updated weights for policy 0, policy_version 1759945 (0.0010) [2023-12-27 04:13:00,259][105692] Updated weights for policy 0, policy_version 1759955 (0.0010) [2023-12-27 04:13:00,290][105620] Updated weights for policy 1, policy_version 1763760 (0.0006) [2023-12-27 04:13:00,309][105692] Updated weights for policy 0, policy_version 1759965 (0.0010) [2023-12-27 04:13:00,356][105620] Updated weights for policy 1, policy_version 1763770 (0.0008) [2023-12-27 04:13:00,424][105620] Updated weights for policy 1, policy_version 1763780 (0.0009) [2023-12-27 04:13:00,909][105692] Updated weights for policy 0, policy_version 1759975 (0.0007) [2023-12-27 04:13:00,957][105692] Updated weights for policy 0, policy_version 1759985 (0.0006) [2023-12-27 04:13:01,002][105692] Updated weights for policy 0, policy_version 1759995 (0.0005) [2023-12-27 04:13:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 902217728. Throughput: 0: 9355.2, 1: 9800.7. Samples: 902185572. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:13:01,062][104569] Avg episode reward: [(0, '8716.172'), (1, '9076.852')] [2023-12-27 04:13:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001763784_451592192.pth... [2023-12-27 04:13:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001760000_450625536.pth... [2023-12-27 04:13:01,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001762632_451297280.pth [2023-12-27 04:13:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001758880_450338816.pth [2023-12-27 04:13:01,221][105620] Updated weights for policy 1, policy_version 1763790 (0.0008) [2023-12-27 04:13:01,278][105620] Updated weights for policy 1, policy_version 1763800 (0.0008) [2023-12-27 04:13:01,336][105620] Updated weights for policy 1, policy_version 1763810 (0.0008) [2023-12-27 04:13:01,701][105692] Updated weights for policy 0, policy_version 1760005 (0.0009) [2023-12-27 04:13:01,755][105692] Updated weights for policy 0, policy_version 1760015 (0.0012) [2023-12-27 04:13:01,813][105692] Updated weights for policy 0, policy_version 1760025 (0.0010) [2023-12-27 04:13:02,123][105620] Updated weights for policy 1, policy_version 1763820 (0.0009) [2023-12-27 04:13:02,188][105620] Updated weights for policy 1, policy_version 1763830 (0.0009) [2023-12-27 04:13:02,255][105620] Updated weights for policy 1, policy_version 1763840 (0.0009) [2023-12-27 04:13:02,491][105692] Updated weights for policy 0, policy_version 1760035 (0.0009) [2023-12-27 04:13:02,551][105692] Updated weights for policy 0, policy_version 1760045 (0.0008) [2023-12-27 04:13:02,617][105692] Updated weights for policy 0, policy_version 1760055 (0.0006) [2023-12-27 04:13:02,905][105620] Updated weights for policy 1, policy_version 1763850 (0.0009) [2023-12-27 04:13:02,954][105620] Updated weights for policy 1, policy_version 1763860 (0.0005) [2023-12-27 04:13:02,999][105620] Updated weights for policy 1, policy_version 1763870 (0.0005) [2023-12-27 04:13:03,048][105620] Updated weights for policy 1, policy_version 1763880 (0.0008) [2023-12-27 04:13:03,182][105692] Updated weights for policy 0, policy_version 1760065 (0.0005) [2023-12-27 04:13:03,235][105692] Updated weights for policy 0, policy_version 1760075 (0.0005) [2023-12-27 04:13:03,289][105692] Updated weights for policy 0, policy_version 1760085 (0.0005) [2023-12-27 04:13:03,335][105692] Updated weights for policy 0, policy_version 1760095 (0.0005) [2023-12-27 04:13:03,698][105620] Updated weights for policy 1, policy_version 1763890 (0.0009) [2023-12-27 04:13:03,747][105620] Updated weights for policy 1, policy_version 1763900 (0.0008) [2023-12-27 04:13:03,794][105620] Updated weights for policy 1, policy_version 1763910 (0.0007) [2023-12-27 04:13:03,875][105692] Updated weights for policy 0, policy_version 1760105 (0.0008) [2023-12-27 04:13:03,931][105692] Updated weights for policy 0, policy_version 1760115 (0.0008) [2023-12-27 04:13:03,992][105692] Updated weights for policy 0, policy_version 1760125 (0.0009) [2023-12-27 04:13:04,526][105620] Updated weights for policy 1, policy_version 1763920 (0.0006) [2023-12-27 04:13:04,577][105620] Updated weights for policy 1, policy_version 1763930 (0.0005) [2023-12-27 04:13:04,639][105620] Updated weights for policy 1, policy_version 1763940 (0.0007) [2023-12-27 04:13:04,794][105692] Updated weights for policy 0, policy_version 1760135 (0.0008) [2023-12-27 04:13:04,857][105692] Updated weights for policy 0, policy_version 1760145 (0.0008) [2023-12-27 04:13:04,912][105692] Updated weights for policy 0, policy_version 1760155 (0.0009) [2023-12-27 04:13:05,390][105620] Updated weights for policy 1, policy_version 1763950 (0.0009) [2023-12-27 04:13:05,436][105620] Updated weights for policy 1, policy_version 1763960 (0.0008) [2023-12-27 04:13:05,483][105620] Updated weights for policy 1, policy_version 1763970 (0.0008) [2023-12-27 04:13:05,533][105692] Updated weights for policy 0, policy_version 1760165 (0.0009) [2023-12-27 04:13:05,580][105692] Updated weights for policy 0, policy_version 1760175 (0.0009) [2023-12-27 04:13:05,637][105692] Updated weights for policy 0, policy_version 1760185 (0.0009) [2023-12-27 04:13:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.1, 300 sec: 19521.9). Total num frames: 902316032. Throughput: 0: 9489.7, 1: 9783.5. Samples: 902305144. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:13:06,063][104569] Avg episode reward: [(0, '8806.419'), (1, '9261.153')] [2023-12-27 04:13:06,200][105620] Updated weights for policy 1, policy_version 1763980 (0.0008) [2023-12-27 04:13:06,263][105620] Updated weights for policy 1, policy_version 1763990 (0.0010) [2023-12-27 04:13:06,326][105620] Updated weights for policy 1, policy_version 1764000 (0.0011) [2023-12-27 04:13:06,446][105692] Updated weights for policy 0, policy_version 1760195 (0.0009) [2023-12-27 04:13:06,504][105692] Updated weights for policy 0, policy_version 1760205 (0.0008) [2023-12-27 04:13:06,556][105692] Updated weights for policy 0, policy_version 1760215 (0.0009) [2023-12-27 04:13:07,036][105620] Updated weights for policy 1, policy_version 1764010 (0.0010) [2023-12-27 04:13:07,091][105620] Updated weights for policy 1, policy_version 1764020 (0.0010) [2023-12-27 04:13:07,139][105620] Updated weights for policy 1, policy_version 1764030 (0.0010) [2023-12-27 04:13:07,195][105620] Updated weights for policy 1, policy_version 1764040 (0.0010) [2023-12-27 04:13:07,297][105692] Updated weights for policy 0, policy_version 1760225 (0.0008) [2023-12-27 04:13:07,357][105692] Updated weights for policy 0, policy_version 1760235 (0.0008) [2023-12-27 04:13:07,410][105692] Updated weights for policy 0, policy_version 1760245 (0.0008) [2023-12-27 04:13:07,463][105692] Updated weights for policy 0, policy_version 1760255 (0.0008) [2023-12-27 04:13:07,946][105620] Updated weights for policy 1, policy_version 1764050 (0.0005) [2023-12-27 04:13:08,010][105620] Updated weights for policy 1, policy_version 1764060 (0.0010) [2023-12-27 04:13:08,071][105620] Updated weights for policy 1, policy_version 1764070 (0.0010) [2023-12-27 04:13:08,264][105692] Updated weights for policy 0, policy_version 1760265 (0.0008) [2023-12-27 04:13:08,321][105692] Updated weights for policy 0, policy_version 1760275 (0.0008) [2023-12-27 04:13:08,386][105692] Updated weights for policy 0, policy_version 1760285 (0.0008) [2023-12-27 04:13:08,771][105620] Updated weights for policy 1, policy_version 1764080 (0.0011) [2023-12-27 04:13:08,842][105620] Updated weights for policy 1, policy_version 1764090 (0.0011) [2023-12-27 04:13:08,905][105620] Updated weights for policy 1, policy_version 1764100 (0.0011) [2023-12-27 04:13:09,160][105692] Updated weights for policy 0, policy_version 1760295 (0.0007) [2023-12-27 04:13:09,219][105692] Updated weights for policy 0, policy_version 1760305 (0.0008) [2023-12-27 04:13:09,286][105692] Updated weights for policy 0, policy_version 1760315 (0.0007) [2023-12-27 04:13:09,646][105620] Updated weights for policy 1, policy_version 1764110 (0.0009) [2023-12-27 04:13:09,713][105620] Updated weights for policy 1, policy_version 1764120 (0.0011) [2023-12-27 04:13:09,781][105620] Updated weights for policy 1, policy_version 1764130 (0.0011) [2023-12-27 04:13:10,113][105692] Updated weights for policy 0, policy_version 1760325 (0.0008) [2023-12-27 04:13:10,167][105692] Updated weights for policy 0, policy_version 1760335 (0.0008) [2023-12-27 04:13:10,212][105692] Updated weights for policy 0, policy_version 1760345 (0.0009) [2023-12-27 04:13:10,507][105620] Updated weights for policy 1, policy_version 1764140 (0.0011) [2023-12-27 04:13:10,563][105620] Updated weights for policy 1, policy_version 1764150 (0.0010) [2023-12-27 04:13:10,620][105620] Updated weights for policy 1, policy_version 1764160 (0.0010) [2023-12-27 04:13:11,012][105692] Updated weights for policy 0, policy_version 1760355 (0.0007) [2023-12-27 04:13:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.6, 300 sec: 19466.4). Total num frames: 902406144. Throughput: 0: 9453.4, 1: 9770.5. Samples: 902417296. Policy #0 lag: (min: 27.0, avg: 32.2, max: 59.0) [2023-12-27 04:13:11,063][104569] Avg episode reward: [(0, '8437.317'), (1, '9077.904')] [2023-12-27 04:13:11,076][105692] Updated weights for policy 0, policy_version 1760365 (0.0008) [2023-12-27 04:13:11,138][105692] Updated weights for policy 0, policy_version 1760375 (0.0009) [2023-12-27 04:13:11,397][105620] Updated weights for policy 1, policy_version 1764170 (0.0009) [2023-12-27 04:13:11,459][105620] Updated weights for policy 1, policy_version 1764180 (0.0007) [2023-12-27 04:13:11,511][105620] Updated weights for policy 1, policy_version 1764190 (0.0009) [2023-12-27 04:13:11,562][105620] Updated weights for policy 1, policy_version 1764200 (0.0009) [2023-12-27 04:13:11,906][105692] Updated weights for policy 0, policy_version 1760385 (0.0009) [2023-12-27 04:13:11,965][105692] Updated weights for policy 0, policy_version 1760395 (0.0009) [2023-12-27 04:13:12,024][105692] Updated weights for policy 0, policy_version 1760405 (0.0007) [2023-12-27 04:13:12,078][105692] Updated weights for policy 0, policy_version 1760415 (0.0010) [2023-12-27 04:13:12,325][105620] Updated weights for policy 1, policy_version 1764210 (0.0009) [2023-12-27 04:13:12,387][105620] Updated weights for policy 1, policy_version 1764220 (0.0008) [2023-12-27 04:13:12,448][105620] Updated weights for policy 1, policy_version 1764230 (0.0009) [2023-12-27 04:13:12,828][105692] Updated weights for policy 0, policy_version 1760425 (0.0011) [2023-12-27 04:13:12,881][105692] Updated weights for policy 0, policy_version 1760435 (0.0009) [2023-12-27 04:13:12,935][105692] Updated weights for policy 0, policy_version 1760445 (0.0010) [2023-12-27 04:13:13,094][105620] Updated weights for policy 1, policy_version 1764240 (0.0006) [2023-12-27 04:13:13,155][105620] Updated weights for policy 1, policy_version 1764250 (0.0005) [2023-12-27 04:13:13,215][105620] Updated weights for policy 1, policy_version 1764260 (0.0006) [2023-12-27 04:13:13,720][105620] Updated weights for policy 1, policy_version 1764270 (0.0008) [2023-12-27 04:13:13,744][105692] Updated weights for policy 0, policy_version 1760456 (0.0006) [2023-12-27 04:13:13,780][105620] Updated weights for policy 1, policy_version 1764280 (0.0011) [2023-12-27 04:13:13,802][105692] Updated weights for policy 0, policy_version 1760466 (0.0006) [2023-12-27 04:13:13,839][105620] Updated weights for policy 1, policy_version 1764290 (0.0010) [2023-12-27 04:13:13,860][105692] Updated weights for policy 0, policy_version 1760476 (0.0010) [2023-12-27 04:13:14,510][105692] Updated weights for policy 0, policy_version 1760486 (0.0010) [2023-12-27 04:13:14,543][105620] Updated weights for policy 1, policy_version 1764300 (0.0010) [2023-12-27 04:13:14,558][105692] Updated weights for policy 0, policy_version 1760496 (0.0010) [2023-12-27 04:13:14,600][105620] Updated weights for policy 1, policy_version 1764310 (0.0010) [2023-12-27 04:13:14,621][105692] Updated weights for policy 0, policy_version 1760506 (0.0011) [2023-12-27 04:13:14,642][105620] Updated weights for policy 1, policy_version 1764320 (0.0010) [2023-12-27 04:13:15,320][105692] Updated weights for policy 0, policy_version 1760516 (0.0008) [2023-12-27 04:13:15,347][105620] Updated weights for policy 1, policy_version 1764330 (0.0010) [2023-12-27 04:13:15,379][105692] Updated weights for policy 0, policy_version 1760526 (0.0006) [2023-12-27 04:13:15,408][105620] Updated weights for policy 1, policy_version 1764340 (0.0007) [2023-12-27 04:13:15,438][105692] Updated weights for policy 0, policy_version 1760536 (0.0005) [2023-12-27 04:13:15,473][105620] Updated weights for policy 1, policy_version 1764350 (0.0005) [2023-12-27 04:13:15,532][105620] Updated weights for policy 1, policy_version 1764360 (0.0006) [2023-12-27 04:13:16,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 902504448. Throughput: 0: 9414.4, 1: 9838.4. Samples: 902475072. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:13:16,063][104569] Avg episode reward: [(0, '8080.061'), (1, '8985.342')] [2023-12-27 04:13:16,065][105692] Updated weights for policy 0, policy_version 1760546 (0.0006) [2023-12-27 04:13:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001764360_451739648.pth... [2023-12-27 04:13:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001763240_451452928.pth [2023-12-27 04:13:16,123][105692] Updated weights for policy 0, policy_version 1760556 (0.0005) [2023-12-27 04:13:16,169][105692] Updated weights for policy 0, policy_version 1760566 (0.0005) [2023-12-27 04:13:16,185][105620] Updated weights for policy 1, policy_version 1764370 (0.0010) [2023-12-27 04:13:16,220][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001760576_450772992.pth... [2023-12-27 04:13:16,221][105692] Updated weights for policy 0, policy_version 1760576 (0.0005) [2023-12-27 04:13:16,224][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001759424_450478080.pth [2023-12-27 04:13:16,247][105620] Updated weights for policy 1, policy_version 1764380 (0.0010) [2023-12-27 04:13:16,302][105620] Updated weights for policy 1, policy_version 1764390 (0.0011) [2023-12-27 04:13:16,776][105692] Updated weights for policy 0, policy_version 1760586 (0.0005) [2023-12-27 04:13:16,828][105692] Updated weights for policy 0, policy_version 1760596 (0.0006) [2023-12-27 04:13:16,879][105692] Updated weights for policy 0, policy_version 1760606 (0.0006) [2023-12-27 04:13:16,986][105620] Updated weights for policy 1, policy_version 1764400 (0.0011) [2023-12-27 04:13:17,042][105620] Updated weights for policy 1, policy_version 1764410 (0.0011) [2023-12-27 04:13:17,094][105620] Updated weights for policy 1, policy_version 1764420 (0.0010) [2023-12-27 04:13:17,408][105692] Updated weights for policy 0, policy_version 1760616 (0.0005) [2023-12-27 04:13:17,455][105692] Updated weights for policy 0, policy_version 1760626 (0.0007) [2023-12-27 04:13:17,500][105692] Updated weights for policy 0, policy_version 1760636 (0.0010) [2023-12-27 04:13:17,806][105620] Updated weights for policy 1, policy_version 1764430 (0.0010) [2023-12-27 04:13:17,869][105620] Updated weights for policy 1, policy_version 1764440 (0.0010) [2023-12-27 04:13:17,926][105620] Updated weights for policy 1, policy_version 1764450 (0.0010) [2023-12-27 04:13:18,224][105692] Updated weights for policy 0, policy_version 1760646 (0.0009) [2023-12-27 04:13:18,278][105692] Updated weights for policy 0, policy_version 1760656 (0.0010) [2023-12-27 04:13:18,332][105692] Updated weights for policy 0, policy_version 1760666 (0.0010) [2023-12-27 04:13:18,524][105620] Updated weights for policy 1, policy_version 1764460 (0.0010) [2023-12-27 04:13:18,585][105620] Updated weights for policy 1, policy_version 1764470 (0.0009) [2023-12-27 04:13:18,646][105620] Updated weights for policy 1, policy_version 1764480 (0.0011) [2023-12-27 04:13:19,040][105692] Updated weights for policy 0, policy_version 1760676 (0.0007) [2023-12-27 04:13:19,095][105692] Updated weights for policy 0, policy_version 1760686 (0.0009) [2023-12-27 04:13:19,164][105692] Updated weights for policy 0, policy_version 1760696 (0.0009) [2023-12-27 04:13:19,314][105620] Updated weights for policy 1, policy_version 1764490 (0.0011) [2023-12-27 04:13:19,382][105620] Updated weights for policy 1, policy_version 1764500 (0.0010) [2023-12-27 04:13:19,434][105620] Updated weights for policy 1, policy_version 1764510 (0.0008) [2023-12-27 04:13:19,494][105620] Updated weights for policy 1, policy_version 1764520 (0.0007) [2023-12-27 04:13:19,976][105692] Updated weights for policy 0, policy_version 1760706 (0.0008) [2023-12-27 04:13:20,040][105692] Updated weights for policy 0, policy_version 1760716 (0.0011) [2023-12-27 04:13:20,099][105692] Updated weights for policy 0, policy_version 1760726 (0.0006) [2023-12-27 04:13:20,159][105692] Updated weights for policy 0, policy_version 1760736 (0.0006) [2023-12-27 04:13:20,236][105620] Updated weights for policy 1, policy_version 1764530 (0.0008) [2023-12-27 04:13:20,285][105620] Updated weights for policy 1, policy_version 1764540 (0.0006) [2023-12-27 04:13:20,299][105586] KL-divergence is very high: 196.6132 [2023-12-27 04:13:20,347][105620] Updated weights for policy 1, policy_version 1764550 (0.0006) [2023-12-27 04:13:20,347][105586] KL-divergence is very high: 330.8095 [2023-12-27 04:13:20,840][105692] Updated weights for policy 0, policy_version 1760746 (0.0011) [2023-12-27 04:13:20,901][105692] Updated weights for policy 0, policy_version 1760756 (0.0011) [2023-12-27 04:13:20,939][105620] Updated weights for policy 1, policy_version 1764560 (0.0008) [2023-12-27 04:13:20,961][105692] Updated weights for policy 0, policy_version 1760766 (0.0010) [2023-12-27 04:13:20,988][105620] Updated weights for policy 1, policy_version 1764570 (0.0006) [2023-12-27 04:13:21,053][105620] Updated weights for policy 1, policy_version 1764580 (0.0008) [2023-12-27 04:13:21,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 902610944. Throughput: 0: 9676.0, 1: 9802.1. Samples: 902599344. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:13:21,063][104569] Avg episode reward: [(0, '8262.218'), (1, '9075.785')] [2023-12-27 04:13:21,694][105692] Updated weights for policy 0, policy_version 1760776 (0.0007) [2023-12-27 04:13:21,758][105692] Updated weights for policy 0, policy_version 1760786 (0.0009) [2023-12-27 04:13:21,815][105692] Updated weights for policy 0, policy_version 1760796 (0.0006) [2023-12-27 04:13:21,861][105620] Updated weights for policy 1, policy_version 1764590 (0.0006) [2023-12-27 04:13:21,928][105620] Updated weights for policy 1, policy_version 1764600 (0.0006) [2023-12-27 04:13:21,994][105620] Updated weights for policy 1, policy_version 1764610 (0.0006) [2023-12-27 04:13:22,597][105692] Updated weights for policy 0, policy_version 1760806 (0.0009) [2023-12-27 04:13:22,602][105620] Updated weights for policy 1, policy_version 1764620 (0.0006) [2023-12-27 04:13:22,656][105692] Updated weights for policy 0, policy_version 1760816 (0.0007) [2023-12-27 04:13:22,670][105620] Updated weights for policy 1, policy_version 1764630 (0.0006) [2023-12-27 04:13:22,726][105692] Updated weights for policy 0, policy_version 1760826 (0.0009) [2023-12-27 04:13:22,738][105620] Updated weights for policy 1, policy_version 1764640 (0.0006) [2023-12-27 04:13:23,427][105620] Updated weights for policy 1, policy_version 1764650 (0.0007) [2023-12-27 04:13:23,430][105692] Updated weights for policy 0, policy_version 1760836 (0.0007) [2023-12-27 04:13:23,479][105692] Updated weights for policy 0, policy_version 1760846 (0.0006) [2023-12-27 04:13:23,485][105620] Updated weights for policy 1, policy_version 1764660 (0.0006) [2023-12-27 04:13:23,526][105692] Updated weights for policy 0, policy_version 1760856 (0.0007) [2023-12-27 04:13:23,544][105620] Updated weights for policy 1, policy_version 1764670 (0.0008) [2023-12-27 04:13:23,601][105620] Updated weights for policy 1, policy_version 1764680 (0.0006) [2023-12-27 04:13:24,168][105620] Updated weights for policy 1, policy_version 1764690 (0.0010) [2023-12-27 04:13:24,224][105620] Updated weights for policy 1, policy_version 1764700 (0.0009) [2023-12-27 04:13:24,282][105620] Updated weights for policy 1, policy_version 1764710 (0.0009) [2023-12-27 04:13:24,371][105692] Updated weights for policy 0, policy_version 1760866 (0.0008) [2023-12-27 04:13:24,434][105692] Updated weights for policy 0, policy_version 1760876 (0.0011) [2023-12-27 04:13:24,497][105692] Updated weights for policy 0, policy_version 1760886 (0.0011) [2023-12-27 04:13:24,546][105692] Updated weights for policy 0, policy_version 1760896 (0.0011) [2023-12-27 04:13:24,958][105620] Updated weights for policy 1, policy_version 1764720 (0.0006) [2023-12-27 04:13:25,013][105620] Updated weights for policy 1, policy_version 1764730 (0.0006) [2023-12-27 04:13:25,070][105620] Updated weights for policy 1, policy_version 1764740 (0.0005) [2023-12-27 04:13:25,311][105692] Updated weights for policy 0, policy_version 1760906 (0.0008) [2023-12-27 04:13:25,367][105692] Updated weights for policy 0, policy_version 1760916 (0.0008) [2023-12-27 04:13:25,420][105692] Updated weights for policy 0, policy_version 1760926 (0.0010) [2023-12-27 04:13:25,647][105620] Updated weights for policy 1, policy_version 1764750 (0.0006) [2023-12-27 04:13:25,699][105620] Updated weights for policy 1, policy_version 1764760 (0.0006) [2023-12-27 04:13:25,751][105620] Updated weights for policy 1, policy_version 1764770 (0.0007) [2023-12-27 04:13:26,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 902709248. Throughput: 0: 9689.9, 1: 9847.7. Samples: 902717852. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:13:26,063][104569] Avg episode reward: [(0, '8805.315'), (1, '8983.155')] [2023-12-27 04:13:26,179][105692] Updated weights for policy 0, policy_version 1760936 (0.0009) [2023-12-27 04:13:26,241][105692] Updated weights for policy 0, policy_version 1760946 (0.0009) [2023-12-27 04:13:26,298][105692] Updated weights for policy 0, policy_version 1760956 (0.0009) [2023-12-27 04:13:26,470][105620] Updated weights for policy 1, policy_version 1764780 (0.0010) [2023-12-27 04:13:26,540][105620] Updated weights for policy 1, policy_version 1764790 (0.0009) [2023-12-27 04:13:26,603][105620] Updated weights for policy 1, policy_version 1764800 (0.0010) [2023-12-27 04:13:26,879][105692] Updated weights for policy 0, policy_version 1760966 (0.0007) [2023-12-27 04:13:26,924][105692] Updated weights for policy 0, policy_version 1760976 (0.0005) [2023-12-27 04:13:26,968][105692] Updated weights for policy 0, policy_version 1760986 (0.0005) [2023-12-27 04:13:27,181][105620] Updated weights for policy 1, policy_version 1764810 (0.0008) [2023-12-27 04:13:27,239][105620] Updated weights for policy 1, policy_version 1764820 (0.0006) [2023-12-27 04:13:27,295][105620] Updated weights for policy 1, policy_version 1764830 (0.0005) [2023-12-27 04:13:27,355][105620] Updated weights for policy 1, policy_version 1764840 (0.0008) [2023-12-27 04:13:27,770][105692] Updated weights for policy 0, policy_version 1760996 (0.0007) [2023-12-27 04:13:27,825][105692] Updated weights for policy 0, policy_version 1761007 (0.0009) [2023-12-27 04:13:27,860][105620] Updated weights for policy 1, policy_version 1764850 (0.0005) [2023-12-27 04:13:27,884][105692] Updated weights for policy 0, policy_version 1761017 (0.0008) [2023-12-27 04:13:27,911][105620] Updated weights for policy 1, policy_version 1764860 (0.0006) [2023-12-27 04:13:27,959][105620] Updated weights for policy 1, policy_version 1764870 (0.0005) [2023-12-27 04:13:28,533][105620] Updated weights for policy 1, policy_version 1764880 (0.0008) [2023-12-27 04:13:28,591][105620] Updated weights for policy 1, policy_version 1764890 (0.0009) [2023-12-27 04:13:28,648][105620] Updated weights for policy 1, policy_version 1764900 (0.0008) [2023-12-27 04:13:28,715][105692] Updated weights for policy 0, policy_version 1761027 (0.0009) [2023-12-27 04:13:28,766][105692] Updated weights for policy 0, policy_version 1761037 (0.0009) [2023-12-27 04:13:28,820][105692] Updated weights for policy 0, policy_version 1761048 (0.0009) [2023-12-27 04:13:29,239][105620] Updated weights for policy 1, policy_version 1764910 (0.0011) [2023-12-27 04:13:29,296][105620] Updated weights for policy 1, policy_version 1764920 (0.0011) [2023-12-27 04:13:29,359][105620] Updated weights for policy 1, policy_version 1764930 (0.0011) [2023-12-27 04:13:29,490][105692] Updated weights for policy 0, policy_version 1761058 (0.0009) [2023-12-27 04:13:29,555][105692] Updated weights for policy 0, policy_version 1761068 (0.0007) [2023-12-27 04:13:29,602][105692] Updated weights for policy 0, policy_version 1761078 (0.0006) [2023-12-27 04:13:29,650][105692] Updated weights for policy 0, policy_version 1761088 (0.0006) [2023-12-27 04:13:30,096][105620] Updated weights for policy 1, policy_version 1764940 (0.0011) [2023-12-27 04:13:30,151][105620] Updated weights for policy 1, policy_version 1764950 (0.0010) [2023-12-27 04:13:30,212][105620] Updated weights for policy 1, policy_version 1764960 (0.0008) [2023-12-27 04:13:30,257][105692] Updated weights for policy 0, policy_version 1761098 (0.0006) [2023-12-27 04:13:30,323][105692] Updated weights for policy 0, policy_version 1761108 (0.0008) [2023-12-27 04:13:30,387][105692] Updated weights for policy 0, policy_version 1761118 (0.0006) [2023-12-27 04:13:30,938][105620] Updated weights for policy 1, policy_version 1764970 (0.0008) [2023-12-27 04:13:30,988][105620] Updated weights for policy 1, policy_version 1764980 (0.0010) [2023-12-27 04:13:31,039][105620] Updated weights for policy 1, policy_version 1764990 (0.0010) [2023-12-27 04:13:31,049][105692] Updated weights for policy 0, policy_version 1761128 (0.0007) [2023-12-27 04:13:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 902807552. Throughput: 0: 9652.0, 1: 9993.1. Samples: 902781224. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:13:31,062][104569] Avg episode reward: [(0, '8532.571'), (1, '9078.115')] [2023-12-27 04:13:31,099][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001765000_451903488.pth... [2023-12-27 04:13:31,101][105620] Updated weights for policy 1, policy_version 1765000 (0.0010) [2023-12-27 04:13:31,104][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001763784_451592192.pth [2023-12-27 04:13:31,112][105692] Updated weights for policy 0, policy_version 1761138 (0.0005) [2023-12-27 04:13:31,177][105692] Updated weights for policy 0, policy_version 1761148 (0.0008) [2023-12-27 04:13:31,205][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001761152_450920448.pth... [2023-12-27 04:13:31,209][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001760000_450625536.pth [2023-12-27 04:13:31,869][105620] Updated weights for policy 1, policy_version 1765010 (0.0009) [2023-12-27 04:13:31,906][105692] Updated weights for policy 0, policy_version 1761158 (0.0005) [2023-12-27 04:13:31,928][105620] Updated weights for policy 1, policy_version 1765020 (0.0008) [2023-12-27 04:13:31,959][105692] Updated weights for policy 0, policy_version 1761168 (0.0006) [2023-12-27 04:13:31,990][105620] Updated weights for policy 1, policy_version 1765030 (0.0008) [2023-12-27 04:13:32,005][105692] Updated weights for policy 0, policy_version 1761178 (0.0007) [2023-12-27 04:13:32,678][105620] Updated weights for policy 1, policy_version 1765040 (0.0010) [2023-12-27 04:13:32,711][105692] Updated weights for policy 0, policy_version 1761188 (0.0007) [2023-12-27 04:13:32,727][105620] Updated weights for policy 1, policy_version 1765050 (0.0008) [2023-12-27 04:13:32,757][105692] Updated weights for policy 0, policy_version 1761198 (0.0005) [2023-12-27 04:13:32,775][105620] Updated weights for policy 1, policy_version 1765060 (0.0009) [2023-12-27 04:13:32,805][105692] Updated weights for policy 0, policy_version 1761208 (0.0006) [2023-12-27 04:13:33,410][105692] Updated weights for policy 0, policy_version 1761218 (0.0006) [2023-12-27 04:13:33,457][105692] Updated weights for policy 0, policy_version 1761228 (0.0005) [2023-12-27 04:13:33,505][105692] Updated weights for policy 0, policy_version 1761238 (0.0005) [2023-12-27 04:13:33,525][105620] Updated weights for policy 1, policy_version 1765071 (0.0010) [2023-12-27 04:13:33,560][105692] Updated weights for policy 0, policy_version 1761248 (0.0008) [2023-12-27 04:13:33,573][105620] Updated weights for policy 1, policy_version 1765081 (0.0010) [2023-12-27 04:13:33,617][105620] Updated weights for policy 1, policy_version 1765091 (0.0010) [2023-12-27 04:13:34,273][105692] Updated weights for policy 0, policy_version 1761258 (0.0011) [2023-12-27 04:13:34,332][105620] Updated weights for policy 1, policy_version 1765101 (0.0009) [2023-12-27 04:13:34,334][105692] Updated weights for policy 0, policy_version 1761268 (0.0011) [2023-12-27 04:13:34,383][105692] Updated weights for policy 0, policy_version 1761278 (0.0010) [2023-12-27 04:13:34,388][105620] Updated weights for policy 1, policy_version 1765111 (0.0006) [2023-12-27 04:13:34,454][105620] Updated weights for policy 1, policy_version 1765121 (0.0008) [2023-12-27 04:13:35,119][105620] Updated weights for policy 1, policy_version 1765131 (0.0010) [2023-12-27 04:13:35,135][105692] Updated weights for policy 0, policy_version 1761288 (0.0008) [2023-12-27 04:13:35,180][105620] Updated weights for policy 1, policy_version 1765141 (0.0006) [2023-12-27 04:13:35,197][105692] Updated weights for policy 0, policy_version 1761298 (0.0005) [2023-12-27 04:13:35,236][105620] Updated weights for policy 1, policy_version 1765151 (0.0006) [2023-12-27 04:13:35,259][105692] Updated weights for policy 0, policy_version 1761308 (0.0010) [2023-12-27 04:13:35,911][105692] Updated weights for policy 0, policy_version 1761318 (0.0008) [2023-12-27 04:13:35,920][105620] Updated weights for policy 1, policy_version 1765161 (0.0006) [2023-12-27 04:13:35,965][105692] Updated weights for policy 0, policy_version 1761328 (0.0006) [2023-12-27 04:13:35,978][105620] Updated weights for policy 1, policy_version 1765171 (0.0011) [2023-12-27 04:13:36,025][105692] Updated weights for policy 0, policy_version 1761338 (0.0006) [2023-12-27 04:13:36,036][105620] Updated weights for policy 1, policy_version 1765181 (0.0010) [2023-12-27 04:13:36,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 902914048. Throughput: 0: 9681.0, 1: 9995.4. Samples: 902901056. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:13:36,062][104569] Avg episode reward: [(0, '8533.186'), (1, '9263.435')] [2023-12-27 04:13:36,098][105620] Updated weights for policy 1, policy_version 1765191 (0.0010) [2023-12-27 04:13:36,665][105692] Updated weights for policy 0, policy_version 1761348 (0.0006) [2023-12-27 04:13:36,720][105692] Updated weights for policy 0, policy_version 1761358 (0.0008) [2023-12-27 04:13:36,779][105692] Updated weights for policy 0, policy_version 1761368 (0.0008) [2023-12-27 04:13:36,865][105620] Updated weights for policy 1, policy_version 1765201 (0.0011) [2023-12-27 04:13:36,916][105620] Updated weights for policy 1, policy_version 1765211 (0.0010) [2023-12-27 04:13:36,968][105620] Updated weights for policy 1, policy_version 1765221 (0.0010) [2023-12-27 04:13:37,575][105692] Updated weights for policy 0, policy_version 1761378 (0.0009) [2023-12-27 04:13:37,629][105692] Updated weights for policy 0, policy_version 1761388 (0.0009) [2023-12-27 04:13:37,638][105620] Updated weights for policy 1, policy_version 1765231 (0.0008) [2023-12-27 04:13:37,681][105692] Updated weights for policy 0, policy_version 1761399 (0.0009) [2023-12-27 04:13:37,690][105620] Updated weights for policy 1, policy_version 1765241 (0.0010) [2023-12-27 04:13:37,740][105620] Updated weights for policy 1, policy_version 1765251 (0.0011) [2023-12-27 04:13:38,447][105692] Updated weights for policy 0, policy_version 1761409 (0.0007) [2023-12-27 04:13:38,501][105620] Updated weights for policy 1, policy_version 1765261 (0.0011) [2023-12-27 04:13:38,507][105692] Updated weights for policy 0, policy_version 1761419 (0.0007) [2023-12-27 04:13:38,558][105692] Updated weights for policy 0, policy_version 1761429 (0.0005) [2023-12-27 04:13:38,563][105620] Updated weights for policy 1, policy_version 1765271 (0.0010) [2023-12-27 04:13:38,610][105692] Updated weights for policy 0, policy_version 1761439 (0.0006) [2023-12-27 04:13:38,623][105620] Updated weights for policy 1, policy_version 1765281 (0.0011) [2023-12-27 04:13:39,210][105620] Updated weights for policy 1, policy_version 1765291 (0.0009) [2023-12-27 04:13:39,267][105692] Updated weights for policy 0, policy_version 1761449 (0.0009) [2023-12-27 04:13:39,278][105620] Updated weights for policy 1, policy_version 1765301 (0.0010) [2023-12-27 04:13:39,323][105692] Updated weights for policy 0, policy_version 1761459 (0.0011) [2023-12-27 04:13:39,339][105620] Updated weights for policy 1, policy_version 1765311 (0.0011) [2023-12-27 04:13:39,390][105692] Updated weights for policy 0, policy_version 1761469 (0.0010) [2023-12-27 04:13:40,088][105620] Updated weights for policy 1, policy_version 1765321 (0.0009) [2023-12-27 04:13:40,145][105620] Updated weights for policy 1, policy_version 1765331 (0.0011) [2023-12-27 04:13:40,155][105692] Updated weights for policy 0, policy_version 1761479 (0.0007) [2023-12-27 04:13:40,210][105692] Updated weights for policy 0, policy_version 1761489 (0.0006) [2023-12-27 04:13:40,212][105620] Updated weights for policy 1, policy_version 1765341 (0.0011) [2023-12-27 04:13:40,269][105692] Updated weights for policy 0, policy_version 1761499 (0.0006) [2023-12-27 04:13:40,277][105620] Updated weights for policy 1, policy_version 1765351 (0.0011) [2023-12-27 04:13:40,871][105692] Updated weights for policy 0, policy_version 1761509 (0.0005) [2023-12-27 04:13:40,923][105692] Updated weights for policy 0, policy_version 1761519 (0.0006) [2023-12-27 04:13:40,977][105692] Updated weights for policy 0, policy_version 1761529 (0.0006) [2023-12-27 04:13:41,046][105620] Updated weights for policy 1, policy_version 1765361 (0.0009) [2023-12-27 04:13:41,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 903012352. Throughput: 0: 9743.6, 1: 10043.7. Samples: 903018944. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:13:41,063][104569] Avg episode reward: [(0, '9169.550'), (1, '9353.729')] [2023-12-27 04:13:41,102][105620] Updated weights for policy 1, policy_version 1765371 (0.0011) [2023-12-27 04:13:41,165][105620] Updated weights for policy 1, policy_version 1765381 (0.0011) [2023-12-27 04:13:41,711][105692] Updated weights for policy 0, policy_version 1761539 (0.0009) [2023-12-27 04:13:41,782][105692] Updated weights for policy 0, policy_version 1761549 (0.0009) [2023-12-27 04:13:41,850][105692] Updated weights for policy 0, policy_version 1761559 (0.0009) [2023-12-27 04:13:41,975][105620] Updated weights for policy 1, policy_version 1765391 (0.0009) [2023-12-27 04:13:42,036][105620] Updated weights for policy 1, policy_version 1765401 (0.0009) [2023-12-27 04:13:42,088][105620] Updated weights for policy 1, policy_version 1765411 (0.0007) [2023-12-27 04:13:42,572][105692] Updated weights for policy 0, policy_version 1761569 (0.0006) [2023-12-27 04:13:42,639][105692] Updated weights for policy 0, policy_version 1761579 (0.0010) [2023-12-27 04:13:42,689][105692] Updated weights for policy 0, policy_version 1761589 (0.0008) [2023-12-27 04:13:42,748][105692] Updated weights for policy 0, policy_version 1761599 (0.0009) [2023-12-27 04:13:42,812][105620] Updated weights for policy 1, policy_version 1765421 (0.0008) [2023-12-27 04:13:42,871][105620] Updated weights for policy 1, policy_version 1765431 (0.0009) [2023-12-27 04:13:42,926][105620] Updated weights for policy 1, policy_version 1765441 (0.0009) [2023-12-27 04:13:43,463][105692] Updated weights for policy 0, policy_version 1761609 (0.0008) [2023-12-27 04:13:43,522][105692] Updated weights for policy 0, policy_version 1761619 (0.0008) [2023-12-27 04:13:43,582][105692] Updated weights for policy 0, policy_version 1761629 (0.0008) [2023-12-27 04:13:43,713][105620] Updated weights for policy 1, policy_version 1765451 (0.0008) [2023-12-27 04:13:43,762][105620] Updated weights for policy 1, policy_version 1765461 (0.0011) [2023-12-27 04:13:43,807][105620] Updated weights for policy 1, policy_version 1765471 (0.0010) [2023-12-27 04:13:44,319][105692] Updated weights for policy 0, policy_version 1761639 (0.0008) [2023-12-27 04:13:44,367][105692] Updated weights for policy 0, policy_version 1761649 (0.0009) [2023-12-27 04:13:44,414][105692] Updated weights for policy 0, policy_version 1761659 (0.0009) [2023-12-27 04:13:44,548][105620] Updated weights for policy 1, policy_version 1765481 (0.0010) [2023-12-27 04:13:44,610][105620] Updated weights for policy 1, policy_version 1765491 (0.0005) [2023-12-27 04:13:44,660][105620] Updated weights for policy 1, policy_version 1765501 (0.0009) [2023-12-27 04:13:44,721][105620] Updated weights for policy 1, policy_version 1765511 (0.0009) [2023-12-27 04:13:45,186][105692] Updated weights for policy 0, policy_version 1761669 (0.0010) [2023-12-27 04:13:45,241][105692] Updated weights for policy 0, policy_version 1761679 (0.0011) [2023-12-27 04:13:45,301][105692] Updated weights for policy 0, policy_version 1761689 (0.0011) [2023-12-27 04:13:45,488][105620] Updated weights for policy 1, policy_version 1765521 (0.0008) [2023-12-27 04:13:45,547][105620] Updated weights for policy 1, policy_version 1765531 (0.0008) [2023-12-27 04:13:45,602][105620] Updated weights for policy 1, policy_version 1765541 (0.0008) [2023-12-27 04:13:46,012][105692] Updated weights for policy 0, policy_version 1761699 (0.0010) [2023-12-27 04:13:46,059][105692] Updated weights for policy 0, policy_version 1761709 (0.0007) [2023-12-27 04:13:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.4, 300 sec: 19549.7). Total num frames: 903102464. Throughput: 0: 9820.4, 1: 9951.3. Samples: 903075300. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:13:46,063][104569] Avg episode reward: [(0, '8988.900'), (1, '9262.127')] [2023-12-27 04:13:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001765544_452042752.pth... [2023-12-27 04:13:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001764360_451739648.pth [2023-12-27 04:13:46,110][105692] Updated weights for policy 0, policy_version 1761719 (0.0006) [2023-12-27 04:13:46,156][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001761728_451067904.pth... [2023-12-27 04:13:46,159][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001760576_450772992.pth [2023-12-27 04:13:46,427][105620] Updated weights for policy 1, policy_version 1765551 (0.0008) [2023-12-27 04:13:46,478][105620] Updated weights for policy 1, policy_version 1765561 (0.0008) [2023-12-27 04:13:46,546][105620] Updated weights for policy 1, policy_version 1765571 (0.0009) [2023-12-27 04:13:46,728][105692] Updated weights for policy 0, policy_version 1761729 (0.0005) [2023-12-27 04:13:46,789][105692] Updated weights for policy 0, policy_version 1761739 (0.0005) [2023-12-27 04:13:46,849][105692] Updated weights for policy 0, policy_version 1761749 (0.0005) [2023-12-27 04:13:46,917][105692] Updated weights for policy 0, policy_version 1761759 (0.0007) [2023-12-27 04:13:47,368][105620] Updated weights for policy 1, policy_version 1765581 (0.0010) [2023-12-27 04:13:47,413][105620] Updated weights for policy 1, policy_version 1765591 (0.0010) [2023-12-27 04:13:47,469][105620] Updated weights for policy 1, policy_version 1765601 (0.0010) [2023-12-27 04:13:47,476][105692] Updated weights for policy 0, policy_version 1761769 (0.0010) [2023-12-27 04:13:47,527][105692] Updated weights for policy 0, policy_version 1761779 (0.0010) [2023-12-27 04:13:47,572][105692] Updated weights for policy 0, policy_version 1761789 (0.0010) [2023-12-27 04:13:48,166][105620] Updated weights for policy 1, policy_version 1765611 (0.0009) [2023-12-27 04:13:48,226][105620] Updated weights for policy 1, policy_version 1765621 (0.0006) [2023-12-27 04:13:48,290][105620] Updated weights for policy 1, policy_version 1765631 (0.0005) [2023-12-27 04:13:48,339][105692] Updated weights for policy 0, policy_version 1761799 (0.0010) [2023-12-27 04:13:48,404][105692] Updated weights for policy 0, policy_version 1761809 (0.0011) [2023-12-27 04:13:48,460][105692] Updated weights for policy 0, policy_version 1761819 (0.0011) [2023-12-27 04:13:48,977][105620] Updated weights for policy 1, policy_version 1765641 (0.0009) [2023-12-27 04:13:49,043][105620] Updated weights for policy 1, policy_version 1765651 (0.0011) [2023-12-27 04:13:49,090][105692] Updated weights for policy 0, policy_version 1761829 (0.0008) [2023-12-27 04:13:49,098][105620] Updated weights for policy 1, policy_version 1765661 (0.0010) [2023-12-27 04:13:49,136][105692] Updated weights for policy 0, policy_version 1761839 (0.0005) [2023-12-27 04:13:49,154][105620] Updated weights for policy 1, policy_version 1765671 (0.0011) [2023-12-27 04:13:49,180][105692] Updated weights for policy 0, policy_version 1761849 (0.0005) [2023-12-27 04:13:49,856][105620] Updated weights for policy 1, policy_version 1765681 (0.0007) [2023-12-27 04:13:49,922][105620] Updated weights for policy 1, policy_version 1765691 (0.0009) [2023-12-27 04:13:49,938][105692] Updated weights for policy 0, policy_version 1761859 (0.0007) [2023-12-27 04:13:49,985][105620] Updated weights for policy 1, policy_version 1765701 (0.0006) [2023-12-27 04:13:49,990][105692] Updated weights for policy 0, policy_version 1761869 (0.0011) [2023-12-27 04:13:50,047][105692] Updated weights for policy 0, policy_version 1761879 (0.0011) [2023-12-27 04:13:50,693][105620] Updated weights for policy 1, policy_version 1765711 (0.0006) [2023-12-27 04:13:50,744][105692] Updated weights for policy 0, policy_version 1761889 (0.0008) [2023-12-27 04:13:50,755][105620] Updated weights for policy 1, policy_version 1765721 (0.0005) [2023-12-27 04:13:50,807][105692] Updated weights for policy 0, policy_version 1761899 (0.0011) [2023-12-27 04:13:50,817][105620] Updated weights for policy 1, policy_version 1765731 (0.0006) [2023-12-27 04:13:50,867][105692] Updated weights for policy 0, policy_version 1761909 (0.0011) [2023-12-27 04:13:50,922][105692] Updated weights for policy 0, policy_version 1761919 (0.0011) [2023-12-27 04:13:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 903208960. Throughput: 0: 9809.8, 1: 9928.4. Samples: 903193352. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:13:51,062][104569] Avg episode reward: [(0, '8533.796'), (1, '9261.910')] [2023-12-27 04:13:51,563][105620] Updated weights for policy 1, policy_version 1765741 (0.0007) [2023-12-27 04:13:51,600][105692] Updated weights for policy 0, policy_version 1761929 (0.0010) [2023-12-27 04:13:51,630][105620] Updated weights for policy 1, policy_version 1765751 (0.0007) [2023-12-27 04:13:51,685][105692] Updated weights for policy 0, policy_version 1761939 (0.0011) [2023-12-27 04:13:51,694][105620] Updated weights for policy 1, policy_version 1765761 (0.0009) [2023-12-27 04:13:51,746][105692] Updated weights for policy 0, policy_version 1761949 (0.0010) [2023-12-27 04:13:52,396][105692] Updated weights for policy 0, policy_version 1761959 (0.0009) [2023-12-27 04:13:52,449][105620] Updated weights for policy 1, policy_version 1765771 (0.0008) [2023-12-27 04:13:52,459][105692] Updated weights for policy 0, policy_version 1761969 (0.0008) [2023-12-27 04:13:52,511][105620] Updated weights for policy 1, policy_version 1765781 (0.0008) [2023-12-27 04:13:52,517][105692] Updated weights for policy 0, policy_version 1761979 (0.0009) [2023-12-27 04:13:52,570][105620] Updated weights for policy 1, policy_version 1765791 (0.0007) [2023-12-27 04:13:53,251][105692] Updated weights for policy 0, policy_version 1761989 (0.0008) [2023-12-27 04:13:53,292][105620] Updated weights for policy 1, policy_version 1765801 (0.0006) [2023-12-27 04:13:53,307][105692] Updated weights for policy 0, policy_version 1761999 (0.0008) [2023-12-27 04:13:53,345][105620] Updated weights for policy 1, policy_version 1765811 (0.0008) [2023-12-27 04:13:53,355][105692] Updated weights for policy 0, policy_version 1762009 (0.0006) [2023-12-27 04:13:53,399][105620] Updated weights for policy 1, policy_version 1765821 (0.0008) [2023-12-27 04:13:53,453][105620] Updated weights for policy 1, policy_version 1765831 (0.0008) [2023-12-27 04:13:54,084][105620] Updated weights for policy 1, policy_version 1765841 (0.0008) [2023-12-27 04:13:54,104][105692] Updated weights for policy 0, policy_version 1762019 (0.0006) [2023-12-27 04:13:54,133][105620] Updated weights for policy 1, policy_version 1765851 (0.0010) [2023-12-27 04:13:54,152][105692] Updated weights for policy 0, policy_version 1762029 (0.0005) [2023-12-27 04:13:54,189][105620] Updated weights for policy 1, policy_version 1765861 (0.0010) [2023-12-27 04:13:54,198][105692] Updated weights for policy 0, policy_version 1762039 (0.0010) [2023-12-27 04:13:54,919][105620] Updated weights for policy 1, policy_version 1765871 (0.0010) [2023-12-27 04:13:54,929][105692] Updated weights for policy 0, policy_version 1762049 (0.0007) [2023-12-27 04:13:54,979][105620] Updated weights for policy 1, policy_version 1765881 (0.0011) [2023-12-27 04:13:54,989][105692] Updated weights for policy 0, policy_version 1762059 (0.0007) [2023-12-27 04:13:55,039][105620] Updated weights for policy 1, policy_version 1765891 (0.0011) [2023-12-27 04:13:55,042][105692] Updated weights for policy 0, policy_version 1762069 (0.0006) [2023-12-27 04:13:55,097][105692] Updated weights for policy 0, policy_version 1762079 (0.0007) [2023-12-27 04:13:55,699][105620] Updated weights for policy 1, policy_version 1765901 (0.0009) [2023-12-27 04:13:55,755][105620] Updated weights for policy 1, policy_version 1765911 (0.0011) [2023-12-27 04:13:55,797][105692] Updated weights for policy 0, policy_version 1762089 (0.0006) [2023-12-27 04:13:55,817][105620] Updated weights for policy 1, policy_version 1765921 (0.0011) [2023-12-27 04:13:55,867][105692] Updated weights for policy 0, policy_version 1762099 (0.0005) [2023-12-27 04:13:55,937][105692] Updated weights for policy 0, policy_version 1762109 (0.0005) [2023-12-27 04:13:56,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 903307264. Throughput: 0: 9896.6, 1: 9949.9. Samples: 903310388. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:13:56,062][104569] Avg episode reward: [(0, '8532.329'), (1, '9169.880')] [2023-12-27 04:13:56,451][105692] Updated weights for policy 0, policy_version 1762119 (0.0007) [2023-12-27 04:13:56,516][105692] Updated weights for policy 0, policy_version 1762129 (0.0008) [2023-12-27 04:13:56,566][105620] Updated weights for policy 1, policy_version 1765931 (0.0011) [2023-12-27 04:13:56,582][105692] Updated weights for policy 0, policy_version 1762139 (0.0008) [2023-12-27 04:13:56,626][105620] Updated weights for policy 1, policy_version 1765941 (0.0010) [2023-12-27 04:13:56,679][105620] Updated weights for policy 1, policy_version 1765951 (0.0005) [2023-12-27 04:13:57,114][105692] Updated weights for policy 0, policy_version 1762149 (0.0005) [2023-12-27 04:13:57,165][105692] Updated weights for policy 0, policy_version 1762159 (0.0005) [2023-12-27 04:13:57,213][105692] Updated weights for policy 0, policy_version 1762169 (0.0005) [2023-12-27 04:13:57,280][105620] Updated weights for policy 1, policy_version 1765961 (0.0005) [2023-12-27 04:13:57,337][105620] Updated weights for policy 1, policy_version 1765971 (0.0005) [2023-12-27 04:13:57,389][105620] Updated weights for policy 1, policy_version 1765981 (0.0005) [2023-12-27 04:13:57,446][105620] Updated weights for policy 1, policy_version 1765991 (0.0005) [2023-12-27 04:13:57,781][105692] Updated weights for policy 0, policy_version 1762179 (0.0005) [2023-12-27 04:13:57,834][105692] Updated weights for policy 0, policy_version 1762189 (0.0005) [2023-12-27 04:13:57,879][105692] Updated weights for policy 0, policy_version 1762199 (0.0005) [2023-12-27 04:13:58,035][105620] Updated weights for policy 1, policy_version 1766001 (0.0010) [2023-12-27 04:13:58,089][105620] Updated weights for policy 1, policy_version 1766011 (0.0010) [2023-12-27 04:13:58,140][105620] Updated weights for policy 1, policy_version 1766021 (0.0010) [2023-12-27 04:13:58,524][105692] Updated weights for policy 0, policy_version 1762209 (0.0005) [2023-12-27 04:13:58,585][105692] Updated weights for policy 0, policy_version 1762219 (0.0008) [2023-12-27 04:13:58,641][105692] Updated weights for policy 0, policy_version 1762229 (0.0008) [2023-12-27 04:13:58,703][105692] Updated weights for policy 0, policy_version 1762239 (0.0007) [2023-12-27 04:13:59,018][105620] Updated weights for policy 1, policy_version 1766031 (0.0010) [2023-12-27 04:13:59,072][105620] Updated weights for policy 1, policy_version 1766041 (0.0009) [2023-12-27 04:13:59,124][105620] Updated weights for policy 1, policy_version 1766051 (0.0007) [2023-12-27 04:13:59,453][105692] Updated weights for policy 0, policy_version 1762249 (0.0007) [2023-12-27 04:13:59,510][105692] Updated weights for policy 0, policy_version 1762259 (0.0006) [2023-12-27 04:13:59,565][105692] Updated weights for policy 0, policy_version 1762269 (0.0005) [2023-12-27 04:13:59,956][105620] Updated weights for policy 1, policy_version 1766061 (0.0007) [2023-12-27 04:14:00,011][105620] Updated weights for policy 1, policy_version 1766071 (0.0010) [2023-12-27 04:14:00,070][105620] Updated weights for policy 1, policy_version 1766081 (0.0010) [2023-12-27 04:14:00,203][105692] Updated weights for policy 0, policy_version 1762279 (0.0008) [2023-12-27 04:14:00,254][105692] Updated weights for policy 0, policy_version 1762289 (0.0007) [2023-12-27 04:14:00,312][105692] Updated weights for policy 0, policy_version 1762299 (0.0010) [2023-12-27 04:14:00,813][105620] Updated weights for policy 1, policy_version 1766091 (0.0009) [2023-12-27 04:14:00,866][105620] Updated weights for policy 1, policy_version 1766101 (0.0005) [2023-12-27 04:14:00,922][105620] Updated weights for policy 1, policy_version 1766111 (0.0005) [2023-12-27 04:14:01,008][105692] Updated weights for policy 0, policy_version 1762309 (0.0010) [2023-12-27 04:14:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 903405568. Throughput: 0: 10074.2, 1: 9943.5. Samples: 903375864. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:14:01,062][104569] Avg episode reward: [(0, '8535.148'), (1, '9169.750')] [2023-12-27 04:14:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001766120_452190208.pth... [2023-12-27 04:14:01,071][105692] Updated weights for policy 0, policy_version 1762319 (0.0008) [2023-12-27 04:14:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001765000_451903488.pth [2023-12-27 04:14:01,128][105692] Updated weights for policy 0, policy_version 1762329 (0.0006) [2023-12-27 04:14:01,175][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001762336_451223552.pth... [2023-12-27 04:14:01,180][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001761152_450920448.pth [2023-12-27 04:14:01,611][105620] Updated weights for policy 1, policy_version 1766121 (0.0008) [2023-12-27 04:14:01,675][105620] Updated weights for policy 1, policy_version 1766131 (0.0009) [2023-12-27 04:14:01,732][105620] Updated weights for policy 1, policy_version 1766141 (0.0008) [2023-12-27 04:14:01,799][105620] Updated weights for policy 1, policy_version 1766151 (0.0008) [2023-12-27 04:14:01,911][105692] Updated weights for policy 0, policy_version 1762339 (0.0008) [2023-12-27 04:14:01,968][105692] Updated weights for policy 0, policy_version 1762349 (0.0007) [2023-12-27 04:14:02,027][105692] Updated weights for policy 0, policy_version 1762359 (0.0010) [2023-12-27 04:14:02,569][105620] Updated weights for policy 1, policy_version 1766161 (0.0007) [2023-12-27 04:14:02,608][105692] Updated weights for policy 0, policy_version 1762369 (0.0009) [2023-12-27 04:14:02,634][105620] Updated weights for policy 1, policy_version 1766171 (0.0006) [2023-12-27 04:14:02,667][105692] Updated weights for policy 0, policy_version 1762379 (0.0010) [2023-12-27 04:14:02,691][105620] Updated weights for policy 1, policy_version 1766181 (0.0005) [2023-12-27 04:14:02,720][105692] Updated weights for policy 0, policy_version 1762389 (0.0010) [2023-12-27 04:14:02,764][105692] Updated weights for policy 0, policy_version 1762399 (0.0010) [2023-12-27 04:14:03,339][105620] Updated weights for policy 1, policy_version 1766191 (0.0005) [2023-12-27 04:14:03,383][105620] Updated weights for policy 1, policy_version 1766201 (0.0006) [2023-12-27 04:14:03,419][105586] KL-divergence is very high: 103.6595 [2023-12-27 04:14:03,436][105692] Updated weights for policy 0, policy_version 1762409 (0.0009) [2023-12-27 04:14:03,436][105620] Updated weights for policy 1, policy_version 1766211 (0.0005) [2023-12-27 04:14:03,496][105692] Updated weights for policy 0, policy_version 1762419 (0.0005) [2023-12-27 04:14:03,551][105692] Updated weights for policy 0, policy_version 1762429 (0.0006) [2023-12-27 04:14:03,977][105620] Updated weights for policy 1, policy_version 1766221 (0.0008) [2023-12-27 04:14:04,031][105620] Updated weights for policy 1, policy_version 1766231 (0.0010) [2023-12-27 04:14:04,088][105620] Updated weights for policy 1, policy_version 1766241 (0.0009) [2023-12-27 04:14:04,169][105692] Updated weights for policy 0, policy_version 1762439 (0.0011) [2023-12-27 04:14:04,239][105692] Updated weights for policy 0, policy_version 1762449 (0.0011) [2023-12-27 04:14:04,294][105692] Updated weights for policy 0, policy_version 1762459 (0.0006) [2023-12-27 04:14:04,904][105620] Updated weights for policy 1, policy_version 1766251 (0.0009) [2023-12-27 04:14:04,953][105692] Updated weights for policy 0, policy_version 1762469 (0.0008) [2023-12-27 04:14:04,963][105620] Updated weights for policy 1, policy_version 1766261 (0.0007) [2023-12-27 04:14:05,005][105692] Updated weights for policy 0, policy_version 1762479 (0.0010) [2023-12-27 04:14:05,019][105620] Updated weights for policy 1, policy_version 1766271 (0.0005) [2023-12-27 04:14:05,064][105692] Updated weights for policy 0, policy_version 1762489 (0.0010) [2023-12-27 04:14:05,756][105692] Updated weights for policy 0, policy_version 1762499 (0.0010) [2023-12-27 04:14:05,761][105620] Updated weights for policy 1, policy_version 1766281 (0.0006) [2023-12-27 04:14:05,813][105620] Updated weights for policy 1, policy_version 1766291 (0.0005) [2023-12-27 04:14:05,814][105692] Updated weights for policy 0, policy_version 1762509 (0.0010) [2023-12-27 04:14:05,864][105620] Updated weights for policy 1, policy_version 1766301 (0.0005) [2023-12-27 04:14:05,870][105692] Updated weights for policy 0, policy_version 1762519 (0.0011) [2023-12-27 04:14:05,916][105620] Updated weights for policy 1, policy_version 1766311 (0.0006) [2023-12-27 04:14:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19934.0, 300 sec: 19633.0). Total num frames: 903512064. Throughput: 0: 10015.1, 1: 9895.9. Samples: 903495340. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:14:06,062][104569] Avg episode reward: [(0, '8532.834'), (1, '8984.238')] [2023-12-27 04:14:06,574][105692] Updated weights for policy 0, policy_version 1762529 (0.0010) [2023-12-27 04:14:06,637][105692] Updated weights for policy 0, policy_version 1762539 (0.0011) [2023-12-27 04:14:06,699][105620] Updated weights for policy 1, policy_version 1766321 (0.0006) [2023-12-27 04:14:06,701][105692] Updated weights for policy 0, policy_version 1762549 (0.0011) [2023-12-27 04:14:06,751][105620] Updated weights for policy 1, policy_version 1766331 (0.0006) [2023-12-27 04:14:06,763][105692] Updated weights for policy 0, policy_version 1762559 (0.0011) [2023-12-27 04:14:06,802][105620] Updated weights for policy 1, policy_version 1766341 (0.0009) [2023-12-27 04:14:07,471][105692] Updated weights for policy 0, policy_version 1762569 (0.0006) [2023-12-27 04:14:07,526][105692] Updated weights for policy 0, policy_version 1762579 (0.0010) [2023-12-27 04:14:07,562][105620] Updated weights for policy 1, policy_version 1766351 (0.0010) [2023-12-27 04:14:07,593][105692] Updated weights for policy 0, policy_version 1762589 (0.0010) [2023-12-27 04:14:07,619][105620] Updated weights for policy 1, policy_version 1766361 (0.0006) [2023-12-27 04:14:07,677][105620] Updated weights for policy 1, policy_version 1766371 (0.0008) [2023-12-27 04:14:08,292][105692] Updated weights for policy 0, policy_version 1762599 (0.0007) [2023-12-27 04:14:08,359][105692] Updated weights for policy 0, policy_version 1762609 (0.0007) [2023-12-27 04:14:08,364][105620] Updated weights for policy 1, policy_version 1766381 (0.0007) [2023-12-27 04:14:08,422][105692] Updated weights for policy 0, policy_version 1762619 (0.0009) [2023-12-27 04:14:08,427][105620] Updated weights for policy 1, policy_version 1766391 (0.0006) [2023-12-27 04:14:08,488][105620] Updated weights for policy 1, policy_version 1766401 (0.0007) [2023-12-27 04:14:08,999][105692] Updated weights for policy 0, policy_version 1762629 (0.0008) [2023-12-27 04:14:09,070][105692] Updated weights for policy 0, policy_version 1762639 (0.0009) [2023-12-27 04:14:09,129][105692] Updated weights for policy 0, policy_version 1762649 (0.0011) [2023-12-27 04:14:09,131][105620] Updated weights for policy 1, policy_version 1766411 (0.0008) [2023-12-27 04:14:09,181][105620] Updated weights for policy 1, policy_version 1766421 (0.0006) [2023-12-27 04:14:09,241][105620] Updated weights for policy 1, policy_version 1766431 (0.0008) [2023-12-27 04:14:09,891][105692] Updated weights for policy 0, policy_version 1762659 (0.0011) [2023-12-27 04:14:09,955][105620] Updated weights for policy 1, policy_version 1766441 (0.0008) [2023-12-27 04:14:09,963][105692] Updated weights for policy 0, policy_version 1762669 (0.0011) [2023-12-27 04:14:10,018][105620] Updated weights for policy 1, policy_version 1766451 (0.0009) [2023-12-27 04:14:10,020][105692] Updated weights for policy 0, policy_version 1762679 (0.0011) [2023-12-27 04:14:10,083][105620] Updated weights for policy 1, policy_version 1766461 (0.0007) [2023-12-27 04:14:10,144][105620] Updated weights for policy 1, policy_version 1766471 (0.0007) [2023-12-27 04:14:10,768][105620] Updated weights for policy 1, policy_version 1766481 (0.0009) [2023-12-27 04:14:10,783][105692] Updated weights for policy 0, policy_version 1762689 (0.0010) [2023-12-27 04:14:10,831][105620] Updated weights for policy 1, policy_version 1766491 (0.0008) [2023-12-27 04:14:10,846][105692] Updated weights for policy 0, policy_version 1762699 (0.0006) [2023-12-27 04:14:10,892][105620] Updated weights for policy 1, policy_version 1766501 (0.0009) [2023-12-27 04:14:10,910][105692] Updated weights for policy 0, policy_version 1762709 (0.0006) [2023-12-27 04:14:10,967][105692] Updated weights for policy 0, policy_version 1762719 (0.0009) [2023-12-27 04:14:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 20070.4, 300 sec: 19633.0). Total num frames: 903610368. Throughput: 0: 10106.5, 1: 9786.1. Samples: 903613020. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:14:11,062][104569] Avg episode reward: [(0, '8162.545'), (1, '8708.222')] [2023-12-27 04:14:11,707][105620] Updated weights for policy 1, policy_version 1766511 (0.0008) [2023-12-27 04:14:11,778][105620] Updated weights for policy 1, policy_version 1766521 (0.0007) [2023-12-27 04:14:11,792][105692] Updated weights for policy 0, policy_version 1762729 (0.0010) [2023-12-27 04:14:11,828][105620] Updated weights for policy 1, policy_version 1766531 (0.0007) [2023-12-27 04:14:11,848][105692] Updated weights for policy 0, policy_version 1762739 (0.0011) [2023-12-27 04:14:11,907][105692] Updated weights for policy 0, policy_version 1762749 (0.0011) [2023-12-27 04:14:12,598][105620] Updated weights for policy 1, policy_version 1766541 (0.0008) [2023-12-27 04:14:12,659][105620] Updated weights for policy 1, policy_version 1766551 (0.0008) [2023-12-27 04:14:12,709][105692] Updated weights for policy 0, policy_version 1762759 (0.0011) [2023-12-27 04:14:12,720][105620] Updated weights for policy 1, policy_version 1766561 (0.0006) [2023-12-27 04:14:12,768][105692] Updated weights for policy 0, policy_version 1762769 (0.0010) [2023-12-27 04:14:12,830][105692] Updated weights for policy 0, policy_version 1762779 (0.0010) [2023-12-27 04:14:13,478][105620] Updated weights for policy 1, policy_version 1766571 (0.0005) [2023-12-27 04:14:13,526][105620] Updated weights for policy 1, policy_version 1766581 (0.0006) [2023-12-27 04:14:13,541][105692] Updated weights for policy 0, policy_version 1762789 (0.0009) [2023-12-27 04:14:13,578][105620] Updated weights for policy 1, policy_version 1766591 (0.0005) [2023-12-27 04:14:13,590][105692] Updated weights for policy 0, policy_version 1762799 (0.0008) [2023-12-27 04:14:13,640][105692] Updated weights for policy 0, policy_version 1762809 (0.0007) [2023-12-27 04:14:14,237][105620] Updated weights for policy 1, policy_version 1766601 (0.0006) [2023-12-27 04:14:14,305][105620] Updated weights for policy 1, policy_version 1766611 (0.0006) [2023-12-27 04:14:14,368][105620] Updated weights for policy 1, policy_version 1766621 (0.0005) [2023-12-27 04:14:14,410][105692] Updated weights for policy 0, policy_version 1762819 (0.0006) [2023-12-27 04:14:14,435][105620] Updated weights for policy 1, policy_version 1766631 (0.0005) [2023-12-27 04:14:14,470][105692] Updated weights for policy 0, policy_version 1762829 (0.0009) [2023-12-27 04:14:14,530][105692] Updated weights for policy 0, policy_version 1762839 (0.0007) [2023-12-27 04:14:15,066][105620] Updated weights for policy 1, policy_version 1766641 (0.0009) [2023-12-27 04:14:15,133][105620] Updated weights for policy 1, policy_version 1766651 (0.0008) [2023-12-27 04:14:15,193][105620] Updated weights for policy 1, policy_version 1766661 (0.0009) [2023-12-27 04:14:15,325][105692] Updated weights for policy 0, policy_version 1762849 (0.0009) [2023-12-27 04:14:15,373][105692] Updated weights for policy 0, policy_version 1762859 (0.0009) [2023-12-27 04:14:15,421][105692] Updated weights for policy 0, policy_version 1762869 (0.0009) [2023-12-27 04:14:15,472][105692] Updated weights for policy 0, policy_version 1762879 (0.0008) [2023-12-27 04:14:15,922][105620] Updated weights for policy 1, policy_version 1766671 (0.0009) [2023-12-27 04:14:15,982][105620] Updated weights for policy 1, policy_version 1766681 (0.0009) [2023-12-27 04:14:16,043][105620] Updated weights for policy 1, policy_version 1766691 (0.0008) [2023-12-27 04:14:16,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 903692288. Throughput: 0: 10057.0, 1: 9651.0. Samples: 903668080. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:14:16,062][104569] Avg episode reward: [(0, '8171.203'), (1, '9077.457')] [2023-12-27 04:14:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001766696_452337664.pth... [2023-12-27 04:14:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001762880_451362816.pth... [2023-12-27 04:14:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001761728_451067904.pth [2023-12-27 04:14:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001765544_452042752.pth [2023-12-27 04:14:16,285][105692] Updated weights for policy 0, policy_version 1762889 (0.0006) [2023-12-27 04:14:16,339][105692] Updated weights for policy 0, policy_version 1762899 (0.0005) [2023-12-27 04:14:16,402][105692] Updated weights for policy 0, policy_version 1762909 (0.0006) [2023-12-27 04:14:16,777][105620] Updated weights for policy 1, policy_version 1766701 (0.0007) [2023-12-27 04:14:16,841][105620] Updated weights for policy 1, policy_version 1766711 (0.0007) [2023-12-27 04:14:16,892][105620] Updated weights for policy 1, policy_version 1766721 (0.0005) [2023-12-27 04:14:17,122][105692] Updated weights for policy 0, policy_version 1762919 (0.0007) [2023-12-27 04:14:17,175][105692] Updated weights for policy 0, policy_version 1762929 (0.0006) [2023-12-27 04:14:17,233][105692] Updated weights for policy 0, policy_version 1762939 (0.0010) [2023-12-27 04:14:17,567][105620] Updated weights for policy 1, policy_version 1766731 (0.0006) [2023-12-27 04:14:17,617][105620] Updated weights for policy 1, policy_version 1766741 (0.0008) [2023-12-27 04:14:17,663][105620] Updated weights for policy 1, policy_version 1766751 (0.0009) [2023-12-27 04:14:17,931][105692] Updated weights for policy 0, policy_version 1762949 (0.0010) [2023-12-27 04:14:17,975][105692] Updated weights for policy 0, policy_version 1762959 (0.0010) [2023-12-27 04:14:18,031][105692] Updated weights for policy 0, policy_version 1762969 (0.0010) [2023-12-27 04:14:18,472][105620] Updated weights for policy 1, policy_version 1766761 (0.0008) [2023-12-27 04:14:18,531][105620] Updated weights for policy 1, policy_version 1766771 (0.0008) [2023-12-27 04:14:18,593][105620] Updated weights for policy 1, policy_version 1766781 (0.0009) [2023-12-27 04:14:18,650][105620] Updated weights for policy 1, policy_version 1766791 (0.0008) [2023-12-27 04:14:18,784][105692] Updated weights for policy 0, policy_version 1762979 (0.0010) [2023-12-27 04:14:18,846][105692] Updated weights for policy 0, policy_version 1762989 (0.0011) [2023-12-27 04:14:18,904][105692] Updated weights for policy 0, policy_version 1762999 (0.0010) [2023-12-27 04:14:19,424][105620] Updated weights for policy 1, policy_version 1766801 (0.0008) [2023-12-27 04:14:19,474][105620] Updated weights for policy 1, policy_version 1766811 (0.0007) [2023-12-27 04:14:19,535][105620] Updated weights for policy 1, policy_version 1766821 (0.0008) [2023-12-27 04:14:19,662][105692] Updated weights for policy 0, policy_version 1763009 (0.0011) [2023-12-27 04:14:19,722][105692] Updated weights for policy 0, policy_version 1763019 (0.0011) [2023-12-27 04:14:19,784][105692] Updated weights for policy 0, policy_version 1763029 (0.0011) [2023-12-27 04:14:19,850][105692] Updated weights for policy 0, policy_version 1763039 (0.0008) [2023-12-27 04:14:20,310][105620] Updated weights for policy 1, policy_version 1766831 (0.0008) [2023-12-27 04:14:20,370][105620] Updated weights for policy 1, policy_version 1766841 (0.0008) [2023-12-27 04:14:20,423][105620] Updated weights for policy 1, policy_version 1766851 (0.0009) [2023-12-27 04:14:20,563][105692] Updated weights for policy 0, policy_version 1763049 (0.0009) [2023-12-27 04:14:20,626][105692] Updated weights for policy 0, policy_version 1763059 (0.0009) [2023-12-27 04:14:20,692][105692] Updated weights for policy 0, policy_version 1763069 (0.0009) [2023-12-27 04:14:21,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 903790592. Throughput: 0: 9940.9, 1: 9632.0. Samples: 903781836. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:14:21,062][104569] Avg episode reward: [(0, '8352.983'), (1, '9353.756')] [2023-12-27 04:14:21,217][105620] Updated weights for policy 1, policy_version 1766861 (0.0010) [2023-12-27 04:14:21,282][105620] Updated weights for policy 1, policy_version 1766871 (0.0011) [2023-12-27 04:14:21,343][105620] Updated weights for policy 1, policy_version 1766881 (0.0011) [2023-12-27 04:14:21,398][105692] Updated weights for policy 0, policy_version 1763079 (0.0008) [2023-12-27 04:14:21,468][105692] Updated weights for policy 0, policy_version 1763089 (0.0008) [2023-12-27 04:14:21,526][105692] Updated weights for policy 0, policy_version 1763099 (0.0008) [2023-12-27 04:14:22,104][105620] Updated weights for policy 1, policy_version 1766891 (0.0008) [2023-12-27 04:14:22,162][105620] Updated weights for policy 1, policy_version 1766901 (0.0006) [2023-12-27 04:14:22,221][105692] Updated weights for policy 0, policy_version 1763109 (0.0009) [2023-12-27 04:14:22,225][105620] Updated weights for policy 1, policy_version 1766911 (0.0006) [2023-12-27 04:14:22,289][105692] Updated weights for policy 0, policy_version 1763119 (0.0011) [2023-12-27 04:14:22,353][105692] Updated weights for policy 0, policy_version 1763129 (0.0010) [2023-12-27 04:14:22,957][105620] Updated weights for policy 1, policy_version 1766921 (0.0008) [2023-12-27 04:14:23,011][105620] Updated weights for policy 1, policy_version 1766931 (0.0008) [2023-12-27 04:14:23,068][105620] Updated weights for policy 1, policy_version 1766941 (0.0008) [2023-12-27 04:14:23,108][105692] Updated weights for policy 0, policy_version 1763139 (0.0011) [2023-12-27 04:14:23,126][105620] Updated weights for policy 1, policy_version 1766951 (0.0006) [2023-12-27 04:14:23,167][105692] Updated weights for policy 0, policy_version 1763149 (0.0010) [2023-12-27 04:14:23,215][105692] Updated weights for policy 0, policy_version 1763159 (0.0010) [2023-12-27 04:14:23,836][105692] Updated weights for policy 0, policy_version 1763169 (0.0010) [2023-12-27 04:14:23,858][105620] Updated weights for policy 1, policy_version 1766961 (0.0010) [2023-12-27 04:14:23,898][105692] Updated weights for policy 0, policy_version 1763179 (0.0010) [2023-12-27 04:14:23,906][105620] Updated weights for policy 1, policy_version 1766971 (0.0010) [2023-12-27 04:14:23,954][105692] Updated weights for policy 0, policy_version 1763189 (0.0010) [2023-12-27 04:14:23,961][105620] Updated weights for policy 1, policy_version 1766981 (0.0010) [2023-12-27 04:14:24,018][105692] Updated weights for policy 0, policy_version 1763199 (0.0010) [2023-12-27 04:14:24,637][105620] Updated weights for policy 1, policy_version 1766991 (0.0007) [2023-12-27 04:14:24,689][105620] Updated weights for policy 1, policy_version 1767001 (0.0005) [2023-12-27 04:14:24,740][105620] Updated weights for policy 1, policy_version 1767011 (0.0010) [2023-12-27 04:14:24,781][105692] Updated weights for policy 0, policy_version 1763209 (0.0008) [2023-12-27 04:14:24,841][105692] Updated weights for policy 0, policy_version 1763219 (0.0009) [2023-12-27 04:14:24,898][105692] Updated weights for policy 0, policy_version 1763229 (0.0006) [2023-12-27 04:14:25,354][105620] Updated weights for policy 1, policy_version 1767021 (0.0008) [2023-12-27 04:14:25,420][105620] Updated weights for policy 1, policy_version 1767031 (0.0007) [2023-12-27 04:14:25,482][105620] Updated weights for policy 1, policy_version 1767041 (0.0005) [2023-12-27 04:14:25,523][105692] Updated weights for policy 0, policy_version 1763239 (0.0009) [2023-12-27 04:14:25,583][105692] Updated weights for policy 0, policy_version 1763249 (0.0010) [2023-12-27 04:14:25,631][105692] Updated weights for policy 0, policy_version 1763259 (0.0010) [2023-12-27 04:14:26,028][105620] Updated weights for policy 1, policy_version 1767051 (0.0007) [2023-12-27 04:14:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 903888896. Throughput: 0: 9939.1, 1: 9647.8. Samples: 903900356. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:14:26,062][104569] Avg episode reward: [(0, '8713.402'), (1, '9353.608')] [2023-12-27 04:14:26,095][105620] Updated weights for policy 1, policy_version 1767061 (0.0010) [2023-12-27 04:14:26,154][105620] Updated weights for policy 1, policy_version 1767071 (0.0011) [2023-12-27 04:14:26,283][105692] Updated weights for policy 0, policy_version 1763269 (0.0010) [2023-12-27 04:14:26,338][105692] Updated weights for policy 0, policy_version 1763279 (0.0010) [2023-12-27 04:14:26,394][105692] Updated weights for policy 0, policy_version 1763289 (0.0005) [2023-12-27 04:14:26,768][105620] Updated weights for policy 1, policy_version 1767081 (0.0008) [2023-12-27 04:14:26,827][105620] Updated weights for policy 1, policy_version 1767091 (0.0005) [2023-12-27 04:14:26,877][105620] Updated weights for policy 1, policy_version 1767101 (0.0005) [2023-12-27 04:14:26,936][105620] Updated weights for policy 1, policy_version 1767111 (0.0005) [2023-12-27 04:14:27,187][105692] Updated weights for policy 0, policy_version 1763299 (0.0007) [2023-12-27 04:14:27,233][105692] Updated weights for policy 0, policy_version 1763309 (0.0009) [2023-12-27 04:14:27,284][105692] Updated weights for policy 0, policy_version 1763319 (0.0010) [2023-12-27 04:14:27,528][105620] Updated weights for policy 1, policy_version 1767121 (0.0005) [2023-12-27 04:14:27,574][105620] Updated weights for policy 1, policy_version 1767131 (0.0005) [2023-12-27 04:14:27,617][105620] Updated weights for policy 1, policy_version 1767141 (0.0005) [2023-12-27 04:14:27,910][105692] Updated weights for policy 0, policy_version 1763329 (0.0007) [2023-12-27 04:14:27,971][105692] Updated weights for policy 0, policy_version 1763339 (0.0009) [2023-12-27 04:14:28,032][105692] Updated weights for policy 0, policy_version 1763349 (0.0009) [2023-12-27 04:14:28,085][105692] Updated weights for policy 0, policy_version 1763359 (0.0010) [2023-12-27 04:14:28,197][105620] Updated weights for policy 1, policy_version 1767151 (0.0008) [2023-12-27 04:14:28,251][105620] Updated weights for policy 1, policy_version 1767161 (0.0010) [2023-12-27 04:14:28,309][105620] Updated weights for policy 1, policy_version 1767171 (0.0010) [2023-12-27 04:14:28,807][105692] Updated weights for policy 0, policy_version 1763369 (0.0010) [2023-12-27 04:14:28,863][105692] Updated weights for policy 0, policy_version 1763379 (0.0010) [2023-12-27 04:14:28,924][105692] Updated weights for policy 0, policy_version 1763389 (0.0010) [2023-12-27 04:14:28,933][105620] Updated weights for policy 1, policy_version 1767181 (0.0008) [2023-12-27 04:14:28,995][105620] Updated weights for policy 1, policy_version 1767191 (0.0008) [2023-12-27 04:14:29,046][105620] Updated weights for policy 1, policy_version 1767201 (0.0007) [2023-12-27 04:14:29,690][105692] Updated weights for policy 0, policy_version 1763399 (0.0009) [2023-12-27 04:14:29,746][105692] Updated weights for policy 0, policy_version 1763409 (0.0008) [2023-12-27 04:14:29,758][105620] Updated weights for policy 1, policy_version 1767211 (0.0007) [2023-12-27 04:14:29,804][105692] Updated weights for policy 0, policy_version 1763419 (0.0005) [2023-12-27 04:14:29,813][105620] Updated weights for policy 1, policy_version 1767221 (0.0010) [2023-12-27 04:14:29,874][105620] Updated weights for policy 1, policy_version 1767231 (0.0010) [2023-12-27 04:14:30,474][105692] Updated weights for policy 0, policy_version 1763429 (0.0008) [2023-12-27 04:14:30,525][105692] Updated weights for policy 0, policy_version 1763439 (0.0005) [2023-12-27 04:14:30,576][105692] Updated weights for policy 0, policy_version 1763449 (0.0005) [2023-12-27 04:14:30,619][105620] Updated weights for policy 1, policy_version 1767241 (0.0010) [2023-12-27 04:14:30,679][105620] Updated weights for policy 1, policy_version 1767251 (0.0005) [2023-12-27 04:14:30,744][105620] Updated weights for policy 1, policy_version 1767261 (0.0005) [2023-12-27 04:14:30,799][105620] Updated weights for policy 1, policy_version 1767271 (0.0005) [2023-12-27 04:14:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 903995392. Throughput: 0: 9958.6, 1: 9796.5. Samples: 903964276. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:14:31,062][104569] Avg episode reward: [(0, '8712.106'), (1, '9260.971')] [2023-12-27 04:14:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001763456_451510272.pth... [2023-12-27 04:14:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001767272_452485120.pth... [2023-12-27 04:14:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001762336_451223552.pth [2023-12-27 04:14:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001766120_452190208.pth [2023-12-27 04:14:31,298][105692] Updated weights for policy 0, policy_version 1763459 (0.0007) [2023-12-27 04:14:31,358][105692] Updated weights for policy 0, policy_version 1763469 (0.0008) [2023-12-27 04:14:31,387][105620] Updated weights for policy 1, policy_version 1767281 (0.0010) [2023-12-27 04:14:31,417][105692] Updated weights for policy 0, policy_version 1763479 (0.0010) [2023-12-27 04:14:31,435][105620] Updated weights for policy 1, policy_version 1767291 (0.0010) [2023-12-27 04:14:31,490][105620] Updated weights for policy 1, policy_version 1767301 (0.0010) [2023-12-27 04:14:32,215][105692] Updated weights for policy 0, policy_version 1763489 (0.0006) [2023-12-27 04:14:32,244][105620] Updated weights for policy 1, policy_version 1767311 (0.0010) [2023-12-27 04:14:32,269][105692] Updated weights for policy 0, policy_version 1763499 (0.0009) [2023-12-27 04:14:32,304][105620] Updated weights for policy 1, policy_version 1767321 (0.0009) [2023-12-27 04:14:32,325][105692] Updated weights for policy 0, policy_version 1763509 (0.0008) [2023-12-27 04:14:32,360][105620] Updated weights for policy 1, policy_version 1767331 (0.0008) [2023-12-27 04:14:32,382][105692] Updated weights for policy 0, policy_version 1763519 (0.0008) [2023-12-27 04:14:33,004][105692] Updated weights for policy 0, policy_version 1763529 (0.0006) [2023-12-27 04:14:33,060][105692] Updated weights for policy 0, policy_version 1763539 (0.0005) [2023-12-27 04:14:33,115][105692] Updated weights for policy 0, policy_version 1763549 (0.0006) [2023-12-27 04:14:33,115][105620] Updated weights for policy 1, policy_version 1767341 (0.0010) [2023-12-27 04:14:33,172][105620] Updated weights for policy 1, policy_version 1767351 (0.0010) [2023-12-27 04:14:33,236][105620] Updated weights for policy 1, policy_version 1767361 (0.0010) [2023-12-27 04:14:33,722][105692] Updated weights for policy 0, policy_version 1763559 (0.0008) [2023-12-27 04:14:33,777][105692] Updated weights for policy 0, policy_version 1763569 (0.0008) [2023-12-27 04:14:33,837][105692] Updated weights for policy 0, policy_version 1763579 (0.0008) [2023-12-27 04:14:33,973][105620] Updated weights for policy 1, policy_version 1767371 (0.0010) [2023-12-27 04:14:34,034][105620] Updated weights for policy 1, policy_version 1767381 (0.0010) [2023-12-27 04:14:34,091][105620] Updated weights for policy 1, policy_version 1767391 (0.0010) [2023-12-27 04:14:34,509][105692] Updated weights for policy 0, policy_version 1763589 (0.0009) [2023-12-27 04:14:34,569][105692] Updated weights for policy 0, policy_version 1763599 (0.0008) [2023-12-27 04:14:34,635][105692] Updated weights for policy 0, policy_version 1763609 (0.0009) [2023-12-27 04:14:34,728][105620] Updated weights for policy 1, policy_version 1767401 (0.0008) [2023-12-27 04:14:34,776][105620] Updated weights for policy 1, policy_version 1767411 (0.0005) [2023-12-27 04:14:34,823][105620] Updated weights for policy 1, policy_version 1767421 (0.0005) [2023-12-27 04:14:34,879][105620] Updated weights for policy 1, policy_version 1767431 (0.0006) [2023-12-27 04:14:35,361][105692] Updated weights for policy 0, policy_version 1763619 (0.0010) [2023-12-27 04:14:35,421][105692] Updated weights for policy 0, policy_version 1763629 (0.0009) [2023-12-27 04:14:35,479][105692] Updated weights for policy 0, policy_version 1763639 (0.0008) [2023-12-27 04:14:35,564][105620] Updated weights for policy 1, policy_version 1767441 (0.0010) [2023-12-27 04:14:35,627][105620] Updated weights for policy 1, policy_version 1767451 (0.0011) [2023-12-27 04:14:35,685][105620] Updated weights for policy 1, policy_version 1767461 (0.0010) [2023-12-27 04:14:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 904093696. Throughput: 0: 9927.2, 1: 9852.0. Samples: 904083416. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:14:36,062][104569] Avg episode reward: [(0, '8165.479'), (1, '9078.688')] [2023-12-27 04:14:36,219][105692] Updated weights for policy 0, policy_version 1763649 (0.0007) [2023-12-27 04:14:36,282][105692] Updated weights for policy 0, policy_version 1763659 (0.0007) [2023-12-27 04:14:36,348][105692] Updated weights for policy 0, policy_version 1763669 (0.0007) [2023-12-27 04:14:36,394][105620] Updated weights for policy 1, policy_version 1767471 (0.0009) [2023-12-27 04:14:36,405][105692] Updated weights for policy 0, policy_version 1763679 (0.0006) [2023-12-27 04:14:36,468][105620] Updated weights for policy 1, policy_version 1767481 (0.0008) [2023-12-27 04:14:36,537][105620] Updated weights for policy 1, policy_version 1767491 (0.0010) [2023-12-27 04:14:37,064][105692] Updated weights for policy 0, policy_version 1763689 (0.0009) [2023-12-27 04:14:37,126][105692] Updated weights for policy 0, policy_version 1763699 (0.0008) [2023-12-27 04:14:37,184][105692] Updated weights for policy 0, policy_version 1763709 (0.0009) [2023-12-27 04:14:37,276][105620] Updated weights for policy 1, policy_version 1767501 (0.0010) [2023-12-27 04:14:37,328][105620] Updated weights for policy 1, policy_version 1767511 (0.0009) [2023-12-27 04:14:37,375][105620] Updated weights for policy 1, policy_version 1767521 (0.0008) [2023-12-27 04:14:37,948][105620] Updated weights for policy 1, policy_version 1767531 (0.0007) [2023-12-27 04:14:37,995][105620] Updated weights for policy 1, policy_version 1767541 (0.0008) [2023-12-27 04:14:38,039][105692] Updated weights for policy 0, policy_version 1763719 (0.0009) [2023-12-27 04:14:38,049][105620] Updated weights for policy 1, policy_version 1767551 (0.0006) [2023-12-27 04:14:38,098][105692] Updated weights for policy 0, policy_version 1763729 (0.0009) [2023-12-27 04:14:38,152][105692] Updated weights for policy 0, policy_version 1763739 (0.0009) [2023-12-27 04:14:38,739][105620] Updated weights for policy 1, policy_version 1767561 (0.0006) [2023-12-27 04:14:38,807][105620] Updated weights for policy 1, policy_version 1767571 (0.0008) [2023-12-27 04:14:38,869][105620] Updated weights for policy 1, policy_version 1767581 (0.0009) [2023-12-27 04:14:38,926][105620] Updated weights for policy 1, policy_version 1767591 (0.0008) [2023-12-27 04:14:38,940][105692] Updated weights for policy 0, policy_version 1763749 (0.0009) [2023-12-27 04:14:39,003][105692] Updated weights for policy 0, policy_version 1763759 (0.0010) [2023-12-27 04:14:39,062][105692] Updated weights for policy 0, policy_version 1763769 (0.0010) [2023-12-27 04:14:39,609][105620] Updated weights for policy 1, policy_version 1767601 (0.0008) [2023-12-27 04:14:39,672][105620] Updated weights for policy 1, policy_version 1767611 (0.0008) [2023-12-27 04:14:39,732][105620] Updated weights for policy 1, policy_version 1767621 (0.0008) [2023-12-27 04:14:39,875][105692] Updated weights for policy 0, policy_version 1763779 (0.0011) [2023-12-27 04:14:39,925][105692] Updated weights for policy 0, policy_version 1763789 (0.0010) [2023-12-27 04:14:39,984][105692] Updated weights for policy 0, policy_version 1763799 (0.0010) [2023-12-27 04:14:40,515][105620] Updated weights for policy 1, policy_version 1767631 (0.0007) [2023-12-27 04:14:40,567][105620] Updated weights for policy 1, policy_version 1767641 (0.0008) [2023-12-27 04:14:40,620][105620] Updated weights for policy 1, policy_version 1767651 (0.0008) [2023-12-27 04:14:40,719][105692] Updated weights for policy 0, policy_version 1763809 (0.0006) [2023-12-27 04:14:40,771][105692] Updated weights for policy 0, policy_version 1763819 (0.0010) [2023-12-27 04:14:40,823][105692] Updated weights for policy 0, policy_version 1763829 (0.0010) [2023-12-27 04:14:40,889][105692] Updated weights for policy 0, policy_version 1763839 (0.0011) [2023-12-27 04:14:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 904192000. Throughput: 0: 9832.9, 1: 9891.2. Samples: 904197972. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:14:41,063][104569] Avg episode reward: [(0, '8712.278'), (1, '9075.843')] [2023-12-27 04:14:41,348][105620] Updated weights for policy 1, policy_version 1767661 (0.0008) [2023-12-27 04:14:41,420][105620] Updated weights for policy 1, policy_version 1767671 (0.0007) [2023-12-27 04:14:41,496][105620] Updated weights for policy 1, policy_version 1767681 (0.0006) [2023-12-27 04:14:41,674][105692] Updated weights for policy 0, policy_version 1763849 (0.0008) [2023-12-27 04:14:41,741][105692] Updated weights for policy 0, policy_version 1763859 (0.0007) [2023-12-27 04:14:41,804][105692] Updated weights for policy 0, policy_version 1763869 (0.0008) [2023-12-27 04:14:42,198][105620] Updated weights for policy 1, policy_version 1767691 (0.0007) [2023-12-27 04:14:42,266][105620] Updated weights for policy 1, policy_version 1767701 (0.0009) [2023-12-27 04:14:42,330][105620] Updated weights for policy 1, policy_version 1767711 (0.0010) [2023-12-27 04:14:42,487][105692] Updated weights for policy 0, policy_version 1763879 (0.0009) [2023-12-27 04:14:42,543][105692] Updated weights for policy 0, policy_version 1763889 (0.0009) [2023-12-27 04:14:42,605][105692] Updated weights for policy 0, policy_version 1763899 (0.0008) [2023-12-27 04:14:42,995][105620] Updated weights for policy 1, policy_version 1767721 (0.0007) [2023-12-27 04:14:43,053][105620] Updated weights for policy 1, policy_version 1767731 (0.0006) [2023-12-27 04:14:43,110][105620] Updated weights for policy 1, policy_version 1767741 (0.0005) [2023-12-27 04:14:43,176][105620] Updated weights for policy 1, policy_version 1767751 (0.0007) [2023-12-27 04:14:43,369][105692] Updated weights for policy 0, policy_version 1763909 (0.0009) [2023-12-27 04:14:43,415][105692] Updated weights for policy 0, policy_version 1763919 (0.0008) [2023-12-27 04:14:43,461][105692] Updated weights for policy 0, policy_version 1763929 (0.0009) [2023-12-27 04:14:43,826][105620] Updated weights for policy 1, policy_version 1767761 (0.0005) [2023-12-27 04:14:43,883][105620] Updated weights for policy 1, policy_version 1767771 (0.0005) [2023-12-27 04:14:43,937][105620] Updated weights for policy 1, policy_version 1767781 (0.0005) [2023-12-27 04:14:44,292][105692] Updated weights for policy 0, policy_version 1763939 (0.0009) [2023-12-27 04:14:44,351][105692] Updated weights for policy 0, policy_version 1763949 (0.0009) [2023-12-27 04:14:44,398][105692] Updated weights for policy 0, policy_version 1763959 (0.0009) [2023-12-27 04:14:44,611][105620] Updated weights for policy 1, policy_version 1767791 (0.0008) [2023-12-27 04:14:44,669][105620] Updated weights for policy 1, policy_version 1767801 (0.0010) [2023-12-27 04:14:44,722][105620] Updated weights for policy 1, policy_version 1767811 (0.0010) [2023-12-27 04:14:45,017][105692] Updated weights for policy 0, policy_version 1763969 (0.0009) [2023-12-27 04:14:45,081][105692] Updated weights for policy 0, policy_version 1763979 (0.0009) [2023-12-27 04:14:45,143][105692] Updated weights for policy 0, policy_version 1763989 (0.0009) [2023-12-27 04:14:45,202][105692] Updated weights for policy 0, policy_version 1763999 (0.0009) [2023-12-27 04:14:45,560][105620] Updated weights for policy 1, policy_version 1767821 (0.0009) [2023-12-27 04:14:45,609][105620] Updated weights for policy 1, policy_version 1767831 (0.0009) [2023-12-27 04:14:45,662][105620] Updated weights for policy 1, policy_version 1767841 (0.0008) [2023-12-27 04:14:45,897][105692] Updated weights for policy 0, policy_version 1764009 (0.0006) [2023-12-27 04:14:45,965][105692] Updated weights for policy 0, policy_version 1764019 (0.0005) [2023-12-27 04:14:46,028][105692] Updated weights for policy 0, policy_version 1764029 (0.0007) [2023-12-27 04:14:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19633.0). Total num frames: 904290304. Throughput: 0: 9675.7, 1: 9882.8. Samples: 904255996. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:14:46,062][104569] Avg episode reward: [(0, '8713.358'), (1, '8985.249')] [2023-12-27 04:14:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001764032_451657728.pth... [2023-12-27 04:14:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001767848_452632576.pth... [2023-12-27 04:14:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001762880_451362816.pth [2023-12-27 04:14:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001766696_452337664.pth [2023-12-27 04:14:46,469][105620] Updated weights for policy 1, policy_version 1767851 (0.0008) [2023-12-27 04:14:46,525][105620] Updated weights for policy 1, policy_version 1767861 (0.0009) [2023-12-27 04:14:46,583][105620] Updated weights for policy 1, policy_version 1767871 (0.0008) [2023-12-27 04:14:46,687][105692] Updated weights for policy 0, policy_version 1764039 (0.0006) [2023-12-27 04:14:46,745][105692] Updated weights for policy 0, policy_version 1764049 (0.0009) [2023-12-27 04:14:46,802][105692] Updated weights for policy 0, policy_version 1764059 (0.0009) [2023-12-27 04:14:47,327][105620] Updated weights for policy 1, policy_version 1767881 (0.0009) [2023-12-27 04:14:47,379][105620] Updated weights for policy 1, policy_version 1767891 (0.0009) [2023-12-27 04:14:47,432][105620] Updated weights for policy 1, policy_version 1767901 (0.0008) [2023-12-27 04:14:47,487][105620] Updated weights for policy 1, policy_version 1767911 (0.0009) [2023-12-27 04:14:47,538][105692] Updated weights for policy 0, policy_version 1764069 (0.0009) [2023-12-27 04:14:47,600][105692] Updated weights for policy 0, policy_version 1764079 (0.0009) [2023-12-27 04:14:47,666][105692] Updated weights for policy 0, policy_version 1764089 (0.0010) [2023-12-27 04:14:48,130][105620] Updated weights for policy 1, policy_version 1767921 (0.0009) [2023-12-27 04:14:48,191][105620] Updated weights for policy 1, policy_version 1767931 (0.0009) [2023-12-27 04:14:48,246][105620] Updated weights for policy 1, policy_version 1767941 (0.0009) [2023-12-27 04:14:48,466][105692] Updated weights for policy 0, policy_version 1764099 (0.0010) [2023-12-27 04:14:48,513][105692] Updated weights for policy 0, policy_version 1764109 (0.0009) [2023-12-27 04:14:48,561][105692] Updated weights for policy 0, policy_version 1764119 (0.0009) [2023-12-27 04:14:49,007][105620] Updated weights for policy 1, policy_version 1767951 (0.0009) [2023-12-27 04:14:49,064][105620] Updated weights for policy 1, policy_version 1767961 (0.0009) [2023-12-27 04:14:49,118][105620] Updated weights for policy 1, policy_version 1767971 (0.0009) [2023-12-27 04:14:49,334][105692] Updated weights for policy 0, policy_version 1764129 (0.0009) [2023-12-27 04:14:49,400][105692] Updated weights for policy 0, policy_version 1764139 (0.0008) [2023-12-27 04:14:49,466][105692] Updated weights for policy 0, policy_version 1764149 (0.0006) [2023-12-27 04:14:49,529][105692] Updated weights for policy 0, policy_version 1764159 (0.0008) [2023-12-27 04:14:49,977][105620] Updated weights for policy 1, policy_version 1767981 (0.0009) [2023-12-27 04:14:50,046][105620] Updated weights for policy 1, policy_version 1767991 (0.0009) [2023-12-27 04:14:50,123][105620] Updated weights for policy 1, policy_version 1768001 (0.0010) [2023-12-27 04:14:50,125][105692] Updated weights for policy 0, policy_version 1764169 (0.0007) [2023-12-27 04:14:50,183][105692] Updated weights for policy 0, policy_version 1764179 (0.0007) [2023-12-27 04:14:50,232][105692] Updated weights for policy 0, policy_version 1764189 (0.0008) [2023-12-27 04:14:50,862][105620] Updated weights for policy 1, policy_version 1768011 (0.0010) [2023-12-27 04:14:50,910][105620] Updated weights for policy 1, policy_version 1768021 (0.0010) [2023-12-27 04:14:50,966][105620] Updated weights for policy 1, policy_version 1768031 (0.0011) [2023-12-27 04:14:51,052][105692] Updated weights for policy 0, policy_version 1764199 (0.0008) [2023-12-27 04:14:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 904380416. Throughput: 0: 9599.6, 1: 9820.5. Samples: 904369248. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:14:51,063][104569] Avg episode reward: [(0, '8441.780'), (1, '8986.405')] [2023-12-27 04:14:51,106][105692] Updated weights for policy 0, policy_version 1764209 (0.0008) [2023-12-27 04:14:51,163][105692] Updated weights for policy 0, policy_version 1764219 (0.0009) [2023-12-27 04:14:51,750][105620] Updated weights for policy 1, policy_version 1768041 (0.0010) [2023-12-27 04:14:51,818][105620] Updated weights for policy 1, policy_version 1768051 (0.0010) [2023-12-27 04:14:51,880][105620] Updated weights for policy 1, policy_version 1768061 (0.0010) [2023-12-27 04:14:51,946][105620] Updated weights for policy 1, policy_version 1768071 (0.0010) [2023-12-27 04:14:51,956][105692] Updated weights for policy 0, policy_version 1764229 (0.0007) [2023-12-27 04:14:52,008][105692] Updated weights for policy 0, policy_version 1764239 (0.0008) [2023-12-27 04:14:52,072][105692] Updated weights for policy 0, policy_version 1764249 (0.0008) [2023-12-27 04:14:52,676][105620] Updated weights for policy 1, policy_version 1768081 (0.0010) [2023-12-27 04:14:52,724][105620] Updated weights for policy 1, policy_version 1768091 (0.0010) [2023-12-27 04:14:52,780][105620] Updated weights for policy 1, policy_version 1768101 (0.0010) [2023-12-27 04:14:52,861][105692] Updated weights for policy 0, policy_version 1764259 (0.0008) [2023-12-27 04:14:52,920][105692] Updated weights for policy 0, policy_version 1764269 (0.0008) [2023-12-27 04:14:52,979][105692] Updated weights for policy 0, policy_version 1764279 (0.0008) [2023-12-27 04:14:53,550][105620] Updated weights for policy 1, policy_version 1768111 (0.0010) [2023-12-27 04:14:53,605][105620] Updated weights for policy 1, policy_version 1768121 (0.0010) [2023-12-27 04:14:53,657][105620] Updated weights for policy 1, policy_version 1768131 (0.0010) [2023-12-27 04:14:53,738][105692] Updated weights for policy 0, policy_version 1764289 (0.0008) [2023-12-27 04:14:53,791][105692] Updated weights for policy 0, policy_version 1764299 (0.0010) [2023-12-27 04:14:53,848][105692] Updated weights for policy 0, policy_version 1764310 (0.0013) [2023-12-27 04:14:53,908][105692] Updated weights for policy 0, policy_version 1764320 (0.0010) [2023-12-27 04:14:54,220][105620] Updated weights for policy 1, policy_version 1768141 (0.0007) [2023-12-27 04:14:54,288][105620] Updated weights for policy 1, policy_version 1768151 (0.0009) [2023-12-27 04:14:54,350][105620] Updated weights for policy 1, policy_version 1768161 (0.0010) [2023-12-27 04:14:54,700][105692] Updated weights for policy 0, policy_version 1764330 (0.0011) [2023-12-27 04:14:54,752][105692] Updated weights for policy 0, policy_version 1764340 (0.0010) [2023-12-27 04:14:54,804][105692] Updated weights for policy 0, policy_version 1764350 (0.0010) [2023-12-27 04:14:55,059][105620] Updated weights for policy 1, policy_version 1768171 (0.0007) [2023-12-27 04:14:55,106][105620] Updated weights for policy 1, policy_version 1768181 (0.0005) [2023-12-27 04:14:55,161][105620] Updated weights for policy 1, policy_version 1768191 (0.0010) [2023-12-27 04:14:55,428][105692] Updated weights for policy 0, policy_version 1764360 (0.0008) [2023-12-27 04:14:55,484][105692] Updated weights for policy 0, policy_version 1764370 (0.0005) [2023-12-27 04:14:55,544][105692] Updated weights for policy 0, policy_version 1764380 (0.0005) [2023-12-27 04:14:55,886][105620] Updated weights for policy 1, policy_version 1768201 (0.0010) [2023-12-27 04:14:55,938][105620] Updated weights for policy 1, policy_version 1768212 (0.0010) [2023-12-27 04:14:55,982][105620] Updated weights for policy 1, policy_version 1768222 (0.0010) [2023-12-27 04:14:56,028][105620] Updated weights for policy 1, policy_version 1768232 (0.0008) [2023-12-27 04:14:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 904478720. Throughput: 0: 9559.2, 1: 9812.4. Samples: 904484740. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:14:56,062][104569] Avg episode reward: [(0, '8536.916'), (1, '8984.818')] [2023-12-27 04:14:56,191][105692] Updated weights for policy 0, policy_version 1764390 (0.0008) [2023-12-27 04:14:56,236][105692] Updated weights for policy 0, policy_version 1764400 (0.0008) [2023-12-27 04:14:56,286][105692] Updated weights for policy 0, policy_version 1764410 (0.0008) [2023-12-27 04:14:56,763][105620] Updated weights for policy 1, policy_version 1768242 (0.0010) [2023-12-27 04:14:56,811][105620] Updated weights for policy 1, policy_version 1768252 (0.0007) [2023-12-27 04:14:56,864][105620] Updated weights for policy 1, policy_version 1768262 (0.0005) [2023-12-27 04:14:57,046][105692] Updated weights for policy 0, policy_version 1764420 (0.0008) [2023-12-27 04:14:57,105][105692] Updated weights for policy 0, policy_version 1764430 (0.0008) [2023-12-27 04:14:57,157][105692] Updated weights for policy 0, policy_version 1764440 (0.0008) [2023-12-27 04:14:57,540][105620] Updated weights for policy 1, policy_version 1768272 (0.0009) [2023-12-27 04:14:57,587][105620] Updated weights for policy 1, policy_version 1768282 (0.0010) [2023-12-27 04:14:57,645][105620] Updated weights for policy 1, policy_version 1768292 (0.0010) [2023-12-27 04:14:57,800][105692] Updated weights for policy 0, policy_version 1764450 (0.0007) [2023-12-27 04:14:57,853][105692] Updated weights for policy 0, policy_version 1764460 (0.0005) [2023-12-27 04:14:57,901][105692] Updated weights for policy 0, policy_version 1764470 (0.0005) [2023-12-27 04:14:57,951][105692] Updated weights for policy 0, policy_version 1764480 (0.0005) [2023-12-27 04:14:58,410][105620] Updated weights for policy 1, policy_version 1768302 (0.0010) [2023-12-27 04:14:58,478][105620] Updated weights for policy 1, policy_version 1768312 (0.0011) [2023-12-27 04:14:58,544][105620] Updated weights for policy 1, policy_version 1768322 (0.0011) [2023-12-27 04:14:58,619][105692] Updated weights for policy 0, policy_version 1764490 (0.0009) [2023-12-27 04:14:58,689][105692] Updated weights for policy 0, policy_version 1764500 (0.0009) [2023-12-27 04:14:58,752][105692] Updated weights for policy 0, policy_version 1764510 (0.0007) [2023-12-27 04:14:59,466][105620] Updated weights for policy 1, policy_version 1768332 (0.0010) [2023-12-27 04:14:59,514][105692] Updated weights for policy 0, policy_version 1764520 (0.0008) [2023-12-27 04:14:59,523][105620] Updated weights for policy 1, policy_version 1768342 (0.0008) [2023-12-27 04:14:59,574][105692] Updated weights for policy 0, policy_version 1764530 (0.0007) [2023-12-27 04:14:59,576][105620] Updated weights for policy 1, policy_version 1768352 (0.0006) [2023-12-27 04:14:59,631][105692] Updated weights for policy 0, policy_version 1764540 (0.0006) [2023-12-27 04:15:00,194][105620] Updated weights for policy 1, policy_version 1768362 (0.0006) [2023-12-27 04:15:00,247][105620] Updated weights for policy 1, policy_version 1768372 (0.0005) [2023-12-27 04:15:00,296][105620] Updated weights for policy 1, policy_version 1768382 (0.0005) [2023-12-27 04:15:00,358][105620] Updated weights for policy 1, policy_version 1768392 (0.0007) [2023-12-27 04:15:00,431][105692] Updated weights for policy 0, policy_version 1764550 (0.0009) [2023-12-27 04:15:00,489][105692] Updated weights for policy 0, policy_version 1764560 (0.0007) [2023-12-27 04:15:00,555][105692] Updated weights for policy 0, policy_version 1764570 (0.0005) [2023-12-27 04:15:01,056][105620] Updated weights for policy 1, policy_version 1768402 (0.0006) [2023-12-27 04:15:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 904568832. Throughput: 0: 9642.3, 1: 9823.9. Samples: 904544060. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:15:01,063][104569] Avg episode reward: [(0, '8716.784'), (1, '8984.844')] [2023-12-27 04:15:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001764576_451796992.pth... [2023-12-27 04:15:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001763456_451510272.pth [2023-12-27 04:15:01,118][105620] Updated weights for policy 1, policy_version 1768412 (0.0008) [2023-12-27 04:15:01,181][105620] Updated weights for policy 1, policy_version 1768422 (0.0009) [2023-12-27 04:15:01,193][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001768424_452780032.pth... [2023-12-27 04:15:01,198][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001767272_452485120.pth [2023-12-27 04:15:01,261][105692] Updated weights for policy 0, policy_version 1764580 (0.0006) [2023-12-27 04:15:01,325][105692] Updated weights for policy 0, policy_version 1764590 (0.0006) [2023-12-27 04:15:01,396][105692] Updated weights for policy 0, policy_version 1764600 (0.0007) [2023-12-27 04:15:01,794][105620] Updated weights for policy 1, policy_version 1768432 (0.0006) [2023-12-27 04:15:01,858][105620] Updated weights for policy 1, policy_version 1768442 (0.0006) [2023-12-27 04:15:01,916][105620] Updated weights for policy 1, policy_version 1768452 (0.0006) [2023-12-27 04:15:01,994][105692] Updated weights for policy 0, policy_version 1764610 (0.0007) [2023-12-27 04:15:02,066][105692] Updated weights for policy 0, policy_version 1764620 (0.0011) [2023-12-27 04:15:02,127][105692] Updated weights for policy 0, policy_version 1764630 (0.0010) [2023-12-27 04:15:02,182][105692] Updated weights for policy 0, policy_version 1764640 (0.0010) [2023-12-27 04:15:02,561][105620] Updated weights for policy 1, policy_version 1768462 (0.0008) [2023-12-27 04:15:02,625][105620] Updated weights for policy 1, policy_version 1768472 (0.0008) [2023-12-27 04:15:02,686][105620] Updated weights for policy 1, policy_version 1768482 (0.0008) [2023-12-27 04:15:02,845][105692] Updated weights for policy 0, policy_version 1764650 (0.0005) [2023-12-27 04:15:02,901][105692] Updated weights for policy 0, policy_version 1764660 (0.0005) [2023-12-27 04:15:02,964][105692] Updated weights for policy 0, policy_version 1764670 (0.0005) [2023-12-27 04:15:03,400][105620] Updated weights for policy 1, policy_version 1768492 (0.0009) [2023-12-27 04:15:03,456][105620] Updated weights for policy 1, policy_version 1768502 (0.0010) [2023-12-27 04:15:03,514][105620] Updated weights for policy 1, policy_version 1768512 (0.0010) [2023-12-27 04:15:03,603][105692] Updated weights for policy 0, policy_version 1764680 (0.0005) [2023-12-27 04:15:03,650][105692] Updated weights for policy 0, policy_version 1764690 (0.0005) [2023-12-27 04:15:03,703][105692] Updated weights for policy 0, policy_version 1764700 (0.0005) [2023-12-27 04:15:04,263][105620] Updated weights for policy 1, policy_version 1768522 (0.0010) [2023-12-27 04:15:04,311][105620] Updated weights for policy 1, policy_version 1768532 (0.0008) [2023-12-27 04:15:04,354][105692] Updated weights for policy 0, policy_version 1764710 (0.0009) [2023-12-27 04:15:04,358][105620] Updated weights for policy 1, policy_version 1768542 (0.0009) [2023-12-27 04:15:04,412][105692] Updated weights for policy 0, policy_version 1764720 (0.0011) [2023-12-27 04:15:04,415][105620] Updated weights for policy 1, policy_version 1768552 (0.0005) [2023-12-27 04:15:04,467][105692] Updated weights for policy 0, policy_version 1764730 (0.0011) [2023-12-27 04:15:05,054][105620] Updated weights for policy 1, policy_version 1768562 (0.0008) [2023-12-27 04:15:05,110][105620] Updated weights for policy 1, policy_version 1768572 (0.0008) [2023-12-27 04:15:05,172][105692] Updated weights for policy 0, policy_version 1764740 (0.0009) [2023-12-27 04:15:05,177][105620] Updated weights for policy 1, policy_version 1768582 (0.0006) [2023-12-27 04:15:05,225][105692] Updated weights for policy 0, policy_version 1764750 (0.0007) [2023-12-27 04:15:05,284][105692] Updated weights for policy 0, policy_version 1764760 (0.0009) [2023-12-27 04:15:05,883][105620] Updated weights for policy 1, policy_version 1768592 (0.0007) [2023-12-27 04:15:05,949][105620] Updated weights for policy 1, policy_version 1768602 (0.0006) [2023-12-27 04:15:06,010][105692] Updated weights for policy 0, policy_version 1764770 (0.0009) [2023-12-27 04:15:06,013][105620] Updated weights for policy 1, policy_version 1768612 (0.0006) [2023-12-27 04:15:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 904675328. Throughput: 0: 9709.3, 1: 9884.5. Samples: 904663560. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:15:06,062][104569] Avg episode reward: [(0, '8806.776'), (1, '9168.626')] [2023-12-27 04:15:06,070][105692] Updated weights for policy 0, policy_version 1764780 (0.0009) [2023-12-27 04:15:06,137][105692] Updated weights for policy 0, policy_version 1764790 (0.0009) [2023-12-27 04:15:06,216][105692] Updated weights for policy 0, policy_version 1764800 (0.0010) [2023-12-27 04:15:06,606][105620] Updated weights for policy 1, policy_version 1768622 (0.0006) [2023-12-27 04:15:06,667][105620] Updated weights for policy 1, policy_version 1768632 (0.0006) [2023-12-27 04:15:06,735][105620] Updated weights for policy 1, policy_version 1768642 (0.0008) [2023-12-27 04:15:06,894][105692] Updated weights for policy 0, policy_version 1764810 (0.0008) [2023-12-27 04:15:06,953][105692] Updated weights for policy 0, policy_version 1764821 (0.0010) [2023-12-27 04:15:07,012][105692] Updated weights for policy 0, policy_version 1764831 (0.0011) [2023-12-27 04:15:07,268][105620] Updated weights for policy 1, policy_version 1768652 (0.0007) [2023-12-27 04:15:07,335][105620] Updated weights for policy 1, policy_version 1768662 (0.0008) [2023-12-27 04:15:07,395][105620] Updated weights for policy 1, policy_version 1768672 (0.0007) [2023-12-27 04:15:07,842][105692] Updated weights for policy 0, policy_version 1764841 (0.0007) [2023-12-27 04:15:07,900][105692] Updated weights for policy 0, policy_version 1764851 (0.0006) [2023-12-27 04:15:07,957][105692] Updated weights for policy 0, policy_version 1764861 (0.0005) [2023-12-27 04:15:08,056][105620] Updated weights for policy 1, policy_version 1768682 (0.0006) [2023-12-27 04:15:08,113][105620] Updated weights for policy 1, policy_version 1768692 (0.0007) [2023-12-27 04:15:08,170][105620] Updated weights for policy 1, policy_version 1768702 (0.0010) [2023-12-27 04:15:08,224][105620] Updated weights for policy 1, policy_version 1768712 (0.0010) [2023-12-27 04:15:08,521][105692] Updated weights for policy 0, policy_version 1764871 (0.0005) [2023-12-27 04:15:08,587][105692] Updated weights for policy 0, policy_version 1764881 (0.0006) [2023-12-27 04:15:08,643][105692] Updated weights for policy 0, policy_version 1764891 (0.0005) [2023-12-27 04:15:08,986][105620] Updated weights for policy 1, policy_version 1768722 (0.0009) [2023-12-27 04:15:09,047][105620] Updated weights for policy 1, policy_version 1768732 (0.0009) [2023-12-27 04:15:09,107][105620] Updated weights for policy 1, policy_version 1768742 (0.0009) [2023-12-27 04:15:09,265][105692] Updated weights for policy 0, policy_version 1764901 (0.0007) [2023-12-27 04:15:09,324][105692] Updated weights for policy 0, policy_version 1764911 (0.0010) [2023-12-27 04:15:09,391][105692] Updated weights for policy 0, policy_version 1764921 (0.0007) [2023-12-27 04:15:09,881][105620] Updated weights for policy 1, policy_version 1768752 (0.0009) [2023-12-27 04:15:09,951][105620] Updated weights for policy 1, policy_version 1768762 (0.0010) [2023-12-27 04:15:10,012][105620] Updated weights for policy 1, policy_version 1768772 (0.0011) [2023-12-27 04:15:10,188][105692] Updated weights for policy 0, policy_version 1764931 (0.0007) [2023-12-27 04:15:10,246][105692] Updated weights for policy 0, policy_version 1764941 (0.0009) [2023-12-27 04:15:10,304][105692] Updated weights for policy 0, policy_version 1764951 (0.0009) [2023-12-27 04:15:10,661][105620] Updated weights for policy 1, policy_version 1768782 (0.0006) [2023-12-27 04:15:10,723][105620] Updated weights for policy 1, policy_version 1768792 (0.0005) [2023-12-27 04:15:10,778][105620] Updated weights for policy 1, policy_version 1768802 (0.0006) [2023-12-27 04:15:11,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 904773632. Throughput: 0: 9690.1, 1: 9926.5. Samples: 904783104. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-12-27 04:15:11,062][104569] Avg episode reward: [(0, '8718.592'), (1, '9260.921')] [2023-12-27 04:15:11,161][105692] Updated weights for policy 0, policy_version 1764961 (0.0010) [2023-12-27 04:15:11,210][105692] Updated weights for policy 0, policy_version 1764971 (0.0008) [2023-12-27 04:15:11,267][105692] Updated weights for policy 0, policy_version 1764981 (0.0008) [2023-12-27 04:15:11,332][105692] Updated weights for policy 0, policy_version 1764991 (0.0009) [2023-12-27 04:15:11,496][105620] Updated weights for policy 1, policy_version 1768812 (0.0010) [2023-12-27 04:15:11,565][105620] Updated weights for policy 1, policy_version 1768822 (0.0005) [2023-12-27 04:15:11,634][105620] Updated weights for policy 1, policy_version 1768832 (0.0009) [2023-12-27 04:15:12,046][105692] Updated weights for policy 0, policy_version 1765001 (0.0007) [2023-12-27 04:15:12,098][105692] Updated weights for policy 0, policy_version 1765011 (0.0008) [2023-12-27 04:15:12,151][105692] Updated weights for policy 0, policy_version 1765021 (0.0008) [2023-12-27 04:15:12,350][105620] Updated weights for policy 1, policy_version 1768842 (0.0009) [2023-12-27 04:15:12,410][105620] Updated weights for policy 1, policy_version 1768852 (0.0010) [2023-12-27 04:15:12,466][105620] Updated weights for policy 1, policy_version 1768862 (0.0011) [2023-12-27 04:15:12,522][105620] Updated weights for policy 1, policy_version 1768872 (0.0011) [2023-12-27 04:15:12,921][105692] Updated weights for policy 0, policy_version 1765031 (0.0008) [2023-12-27 04:15:12,983][105692] Updated weights for policy 0, policy_version 1765041 (0.0008) [2023-12-27 04:15:13,036][105692] Updated weights for policy 0, policy_version 1765051 (0.0008) [2023-12-27 04:15:13,294][105620] Updated weights for policy 1, policy_version 1768882 (0.0011) [2023-12-27 04:15:13,352][105620] Updated weights for policy 1, policy_version 1768892 (0.0010) [2023-12-27 04:15:13,420][105620] Updated weights for policy 1, policy_version 1768902 (0.0009) [2023-12-27 04:15:13,745][105692] Updated weights for policy 0, policy_version 1765061 (0.0007) [2023-12-27 04:15:13,803][105692] Updated weights for policy 0, policy_version 1765071 (0.0006) [2023-12-27 04:15:13,863][105692] Updated weights for policy 0, policy_version 1765081 (0.0008) [2023-12-27 04:15:14,119][105620] Updated weights for policy 1, policy_version 1768912 (0.0009) [2023-12-27 04:15:14,176][105620] Updated weights for policy 1, policy_version 1768922 (0.0009) [2023-12-27 04:15:14,231][105620] Updated weights for policy 1, policy_version 1768932 (0.0009) [2023-12-27 04:15:14,568][105692] Updated weights for policy 0, policy_version 1765091 (0.0009) [2023-12-27 04:15:14,634][105692] Updated weights for policy 0, policy_version 1765101 (0.0011) [2023-12-27 04:15:14,685][105692] Updated weights for policy 0, policy_version 1765111 (0.0010) [2023-12-27 04:15:14,903][105620] Updated weights for policy 1, policy_version 1768942 (0.0008) [2023-12-27 04:15:14,961][105620] Updated weights for policy 1, policy_version 1768952 (0.0007) [2023-12-27 04:15:15,020][105620] Updated weights for policy 1, policy_version 1768962 (0.0008) [2023-12-27 04:15:15,508][105692] Updated weights for policy 0, policy_version 1765121 (0.0009) [2023-12-27 04:15:15,568][105692] Updated weights for policy 0, policy_version 1765131 (0.0008) [2023-12-27 04:15:15,617][105692] Updated weights for policy 0, policy_version 1765141 (0.0006) [2023-12-27 04:15:15,659][105620] Updated weights for policy 1, policy_version 1768972 (0.0007) [2023-12-27 04:15:15,678][105692] Updated weights for policy 0, policy_version 1765151 (0.0007) [2023-12-27 04:15:15,724][105620] Updated weights for policy 1, policy_version 1768982 (0.0006) [2023-12-27 04:15:15,786][105620] Updated weights for policy 1, policy_version 1768992 (0.0006) [2023-12-27 04:15:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.7, 300 sec: 19605.3). Total num frames: 904871936. Throughput: 0: 9630.4, 1: 9808.8. Samples: 904839040. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:15:16,063][104569] Avg episode reward: [(0, '8540.981'), (1, '9353.421')] [2023-12-27 04:15:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001769000_452927488.pth... [2023-12-27 04:15:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001765152_451944448.pth... [2023-12-27 04:15:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001764032_451657728.pth [2023-12-27 04:15:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001767848_452632576.pth [2023-12-27 04:15:16,396][105692] Updated weights for policy 0, policy_version 1765161 (0.0008) [2023-12-27 04:15:16,454][105692] Updated weights for policy 0, policy_version 1765171 (0.0010) [2023-12-27 04:15:16,487][105620] Updated weights for policy 1, policy_version 1769002 (0.0010) [2023-12-27 04:15:16,513][105692] Updated weights for policy 0, policy_version 1765181 (0.0011) [2023-12-27 04:15:16,542][105620] Updated weights for policy 1, policy_version 1769012 (0.0011) [2023-12-27 04:15:16,596][105620] Updated weights for policy 1, policy_version 1769022 (0.0010) [2023-12-27 04:15:16,643][105620] Updated weights for policy 1, policy_version 1769032 (0.0010) [2023-12-27 04:15:17,158][105692] Updated weights for policy 0, policy_version 1765191 (0.0010) [2023-12-27 04:15:17,213][105692] Updated weights for policy 0, policy_version 1765201 (0.0010) [2023-12-27 04:15:17,274][105692] Updated weights for policy 0, policy_version 1765211 (0.0010) [2023-12-27 04:15:17,393][105620] Updated weights for policy 1, policy_version 1769042 (0.0010) [2023-12-27 04:15:17,457][105620] Updated weights for policy 1, policy_version 1769052 (0.0010) [2023-12-27 04:15:17,513][105620] Updated weights for policy 1, policy_version 1769062 (0.0006) [2023-12-27 04:15:18,035][105692] Updated weights for policy 0, policy_version 1765221 (0.0011) [2023-12-27 04:15:18,096][105692] Updated weights for policy 0, policy_version 1765231 (0.0010) [2023-12-27 04:15:18,155][105620] Updated weights for policy 1, policy_version 1769072 (0.0009) [2023-12-27 04:15:18,159][105692] Updated weights for policy 0, policy_version 1765241 (0.0010) [2023-12-27 04:15:18,208][105620] Updated weights for policy 1, policy_version 1769082 (0.0006) [2023-12-27 04:15:18,266][105620] Updated weights for policy 1, policy_version 1769092 (0.0008) [2023-12-27 04:15:18,849][105692] Updated weights for policy 0, policy_version 1765251 (0.0010) [2023-12-27 04:15:18,901][105692] Updated weights for policy 0, policy_version 1765261 (0.0010) [2023-12-27 04:15:18,957][105620] Updated weights for policy 1, policy_version 1769102 (0.0006) [2023-12-27 04:15:18,961][105692] Updated weights for policy 0, policy_version 1765271 (0.0007) [2023-12-27 04:15:19,020][105620] Updated weights for policy 1, policy_version 1769112 (0.0007) [2023-12-27 04:15:19,080][105620] Updated weights for policy 1, policy_version 1769122 (0.0005) [2023-12-27 04:15:19,607][105692] Updated weights for policy 0, policy_version 1765281 (0.0006) [2023-12-27 04:15:19,663][105620] Updated weights for policy 1, policy_version 1769132 (0.0005) [2023-12-27 04:15:19,678][105692] Updated weights for policy 0, policy_version 1765291 (0.0006) [2023-12-27 04:15:19,730][105620] Updated weights for policy 1, policy_version 1769142 (0.0006) [2023-12-27 04:15:19,740][105692] Updated weights for policy 0, policy_version 1765301 (0.0006) [2023-12-27 04:15:19,790][105620] Updated weights for policy 1, policy_version 1769152 (0.0009) [2023-12-27 04:15:19,793][105692] Updated weights for policy 0, policy_version 1765311 (0.0006) [2023-12-27 04:15:20,389][105692] Updated weights for policy 0, policy_version 1765321 (0.0010) [2023-12-27 04:15:20,444][105692] Updated weights for policy 0, policy_version 1765331 (0.0010) [2023-12-27 04:15:20,509][105692] Updated weights for policy 0, policy_version 1765341 (0.0010) [2023-12-27 04:15:20,558][105620] Updated weights for policy 1, policy_version 1769162 (0.0008) [2023-12-27 04:15:20,623][105620] Updated weights for policy 1, policy_version 1769172 (0.0010) [2023-12-27 04:15:20,683][105620] Updated weights for policy 1, policy_version 1769182 (0.0010) [2023-12-27 04:15:20,750][105620] Updated weights for policy 1, policy_version 1769192 (0.0010) [2023-12-27 04:15:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 904970240. Throughput: 0: 9612.2, 1: 9844.1. Samples: 904958948. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:15:21,062][104569] Avg episode reward: [(0, '8538.211'), (1, '9353.435')] [2023-12-27 04:15:21,255][105692] Updated weights for policy 0, policy_version 1765351 (0.0009) [2023-12-27 04:15:21,314][105692] Updated weights for policy 0, policy_version 1765361 (0.0009) [2023-12-27 04:15:21,380][105692] Updated weights for policy 0, policy_version 1765371 (0.0009) [2023-12-27 04:15:21,570][105620] Updated weights for policy 1, policy_version 1769202 (0.0005) [2023-12-27 04:15:21,639][105620] Updated weights for policy 1, policy_version 1769212 (0.0008) [2023-12-27 04:15:21,687][105620] Updated weights for policy 1, policy_version 1769222 (0.0005) [2023-12-27 04:15:22,266][105692] Updated weights for policy 0, policy_version 1765381 (0.0009) [2023-12-27 04:15:22,332][105692] Updated weights for policy 0, policy_version 1765391 (0.0009) [2023-12-27 04:15:22,396][105692] Updated weights for policy 0, policy_version 1765401 (0.0007) [2023-12-27 04:15:22,406][105620] Updated weights for policy 1, policy_version 1769232 (0.0007) [2023-12-27 04:15:22,458][105620] Updated weights for policy 1, policy_version 1769242 (0.0008) [2023-12-27 04:15:22,525][105620] Updated weights for policy 1, policy_version 1769252 (0.0009) [2023-12-27 04:15:23,126][105692] Updated weights for policy 0, policy_version 1765411 (0.0007) [2023-12-27 04:15:23,194][105692] Updated weights for policy 0, policy_version 1765421 (0.0009) [2023-12-27 04:15:23,246][105692] Updated weights for policy 0, policy_version 1765431 (0.0009) [2023-12-27 04:15:23,297][105620] Updated weights for policy 1, policy_version 1769262 (0.0008) [2023-12-27 04:15:23,347][105620] Updated weights for policy 1, policy_version 1769272 (0.0009) [2023-12-27 04:15:23,398][105620] Updated weights for policy 1, policy_version 1769282 (0.0008) [2023-12-27 04:15:23,990][105692] Updated weights for policy 0, policy_version 1765441 (0.0007) [2023-12-27 04:15:24,048][105692] Updated weights for policy 0, policy_version 1765451 (0.0009) [2023-12-27 04:15:24,102][105692] Updated weights for policy 0, policy_version 1765461 (0.0009) [2023-12-27 04:15:24,152][105620] Updated weights for policy 1, policy_version 1769292 (0.0008) [2023-12-27 04:15:24,158][105692] Updated weights for policy 0, policy_version 1765471 (0.0009) [2023-12-27 04:15:24,204][105620] Updated weights for policy 1, policy_version 1769302 (0.0007) [2023-12-27 04:15:24,253][105620] Updated weights for policy 1, policy_version 1769312 (0.0009) [2023-12-27 04:15:24,924][105620] Updated weights for policy 1, policy_version 1769322 (0.0008) [2023-12-27 04:15:24,969][105692] Updated weights for policy 0, policy_version 1765481 (0.0008) [2023-12-27 04:15:24,972][105620] Updated weights for policy 1, policy_version 1769332 (0.0005) [2023-12-27 04:15:25,019][105620] Updated weights for policy 1, policy_version 1769342 (0.0007) [2023-12-27 04:15:25,029][105692] Updated weights for policy 0, policy_version 1765491 (0.0007) [2023-12-27 04:15:25,067][105620] Updated weights for policy 1, policy_version 1769352 (0.0009) [2023-12-27 04:15:25,081][105692] Updated weights for policy 0, policy_version 1765501 (0.0006) [2023-12-27 04:15:25,706][105692] Updated weights for policy 0, policy_version 1765511 (0.0008) [2023-12-27 04:15:25,739][105620] Updated weights for policy 1, policy_version 1769362 (0.0008) [2023-12-27 04:15:25,762][105692] Updated weights for policy 0, policy_version 1765521 (0.0008) [2023-12-27 04:15:25,785][105620] Updated weights for policy 1, policy_version 1769372 (0.0007) [2023-12-27 04:15:25,821][105692] Updated weights for policy 0, policy_version 1765531 (0.0009) [2023-12-27 04:15:25,845][105620] Updated weights for policy 1, policy_version 1769382 (0.0010) [2023-12-27 04:15:26,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 905068544. Throughput: 0: 9667.7, 1: 9783.8. Samples: 905073284. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:15:26,062][104569] Avg episode reward: [(0, '8714.682'), (1, '9261.965')] [2023-12-27 04:15:26,473][105620] Updated weights for policy 1, policy_version 1769392 (0.0007) [2023-12-27 04:15:26,497][105692] Updated weights for policy 0, policy_version 1765541 (0.0008) [2023-12-27 04:15:26,519][105620] Updated weights for policy 1, policy_version 1769402 (0.0008) [2023-12-27 04:15:26,564][105692] Updated weights for policy 0, policy_version 1765551 (0.0006) [2023-12-27 04:15:26,570][105620] Updated weights for policy 1, policy_version 1769412 (0.0008) [2023-12-27 04:15:26,628][105692] Updated weights for policy 0, policy_version 1765561 (0.0005) [2023-12-27 04:15:27,194][105620] Updated weights for policy 1, policy_version 1769422 (0.0007) [2023-12-27 04:15:27,242][105620] Updated weights for policy 1, policy_version 1769432 (0.0010) [2023-12-27 04:15:27,246][105692] Updated weights for policy 0, policy_version 1765571 (0.0005) [2023-12-27 04:15:27,294][105620] Updated weights for policy 1, policy_version 1769442 (0.0009) [2023-12-27 04:15:27,301][105692] Updated weights for policy 0, policy_version 1765581 (0.0005) [2023-12-27 04:15:27,362][105692] Updated weights for policy 0, policy_version 1765591 (0.0006) [2023-12-27 04:15:27,955][105620] Updated weights for policy 1, policy_version 1769452 (0.0008) [2023-12-27 04:15:27,987][105692] Updated weights for policy 0, policy_version 1765601 (0.0006) [2023-12-27 04:15:27,999][105620] Updated weights for policy 1, policy_version 1769462 (0.0010) [2023-12-27 04:15:28,039][105692] Updated weights for policy 0, policy_version 1765611 (0.0011) [2023-12-27 04:15:28,049][105620] Updated weights for policy 1, policy_version 1769472 (0.0007) [2023-12-27 04:15:28,098][105692] Updated weights for policy 0, policy_version 1765621 (0.0011) [2023-12-27 04:15:28,156][105692] Updated weights for policy 0, policy_version 1765631 (0.0010) [2023-12-27 04:15:28,700][105620] Updated weights for policy 1, policy_version 1769482 (0.0005) [2023-12-27 04:15:28,765][105620] Updated weights for policy 1, policy_version 1769492 (0.0005) [2023-12-27 04:15:28,828][105620] Updated weights for policy 1, policy_version 1769502 (0.0005) [2023-12-27 04:15:28,833][105692] Updated weights for policy 0, policy_version 1765641 (0.0010) [2023-12-27 04:15:28,891][105620] Updated weights for policy 1, policy_version 1769512 (0.0005) [2023-12-27 04:15:28,895][105692] Updated weights for policy 0, policy_version 1765651 (0.0010) [2023-12-27 04:15:28,957][105692] Updated weights for policy 0, policy_version 1765661 (0.0011) [2023-12-27 04:15:29,444][105620] Updated weights for policy 1, policy_version 1769522 (0.0005) [2023-12-27 04:15:29,508][105620] Updated weights for policy 1, policy_version 1769532 (0.0005) [2023-12-27 04:15:29,566][105620] Updated weights for policy 1, policy_version 1769542 (0.0006) [2023-12-27 04:15:29,703][105692] Updated weights for policy 0, policy_version 1765671 (0.0010) [2023-12-27 04:15:29,768][105692] Updated weights for policy 0, policy_version 1765681 (0.0011) [2023-12-27 04:15:29,826][105692] Updated weights for policy 0, policy_version 1765691 (0.0011) [2023-12-27 04:15:30,318][105620] Updated weights for policy 1, policy_version 1769552 (0.0007) [2023-12-27 04:15:30,382][105620] Updated weights for policy 1, policy_version 1769562 (0.0008) [2023-12-27 04:15:30,433][105620] Updated weights for policy 1, policy_version 1769572 (0.0010) [2023-12-27 04:15:30,442][105692] Updated weights for policy 0, policy_version 1765701 (0.0008) [2023-12-27 04:15:30,500][105692] Updated weights for policy 0, policy_version 1765711 (0.0010) [2023-12-27 04:15:30,554][105692] Updated weights for policy 0, policy_version 1765721 (0.0010) [2023-12-27 04:15:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 905166848. Throughput: 0: 9744.0, 1: 9855.4. Samples: 905137968. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:15:31,062][104569] Avg episode reward: [(0, '8259.233'), (1, '9261.834')] [2023-12-27 04:15:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001765728_452091904.pth... [2023-12-27 04:15:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001769576_453074944.pth... [2023-12-27 04:15:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001764576_451796992.pth [2023-12-27 04:15:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001768424_452780032.pth [2023-12-27 04:15:31,126][105620] Updated weights for policy 1, policy_version 1769582 (0.0011) [2023-12-27 04:15:31,192][105620] Updated weights for policy 1, policy_version 1769592 (0.0010) [2023-12-27 04:15:31,251][105620] Updated weights for policy 1, policy_version 1769602 (0.0010) [2023-12-27 04:15:31,256][105692] Updated weights for policy 0, policy_version 1765731 (0.0010) [2023-12-27 04:15:31,316][105692] Updated weights for policy 0, policy_version 1765741 (0.0011) [2023-12-27 04:15:31,376][105692] Updated weights for policy 0, policy_version 1765751 (0.0009) [2023-12-27 04:15:32,011][105620] Updated weights for policy 1, policy_version 1769612 (0.0009) [2023-12-27 04:15:32,070][105620] Updated weights for policy 1, policy_version 1769622 (0.0010) [2023-12-27 04:15:32,126][105692] Updated weights for policy 0, policy_version 1765761 (0.0010) [2023-12-27 04:15:32,133][105620] Updated weights for policy 1, policy_version 1769632 (0.0010) [2023-12-27 04:15:32,175][105692] Updated weights for policy 0, policy_version 1765771 (0.0006) [2023-12-27 04:15:32,233][105692] Updated weights for policy 0, policy_version 1765781 (0.0009) [2023-12-27 04:15:32,294][105692] Updated weights for policy 0, policy_version 1765791 (0.0009) [2023-12-27 04:15:32,855][105620] Updated weights for policy 1, policy_version 1769642 (0.0010) [2023-12-27 04:15:32,888][105692] Updated weights for policy 0, policy_version 1765801 (0.0006) [2023-12-27 04:15:32,915][105620] Updated weights for policy 1, policy_version 1769652 (0.0011) [2023-12-27 04:15:32,945][105692] Updated weights for policy 0, policy_version 1765811 (0.0006) [2023-12-27 04:15:32,970][105620] Updated weights for policy 1, policy_version 1769662 (0.0010) [2023-12-27 04:15:33,001][105692] Updated weights for policy 0, policy_version 1765821 (0.0006) [2023-12-27 04:15:33,029][105620] Updated weights for policy 1, policy_version 1769672 (0.0010) [2023-12-27 04:15:33,709][105692] Updated weights for policy 0, policy_version 1765831 (0.0005) [2023-12-27 04:15:33,724][105620] Updated weights for policy 1, policy_version 1769682 (0.0007) [2023-12-27 04:15:33,754][105692] Updated weights for policy 0, policy_version 1765841 (0.0005) [2023-12-27 04:15:33,778][105620] Updated weights for policy 1, policy_version 1769692 (0.0010) [2023-12-27 04:15:33,804][105692] Updated weights for policy 0, policy_version 1765851 (0.0005) [2023-12-27 04:15:33,822][105620] Updated weights for policy 1, policy_version 1769702 (0.0008) [2023-12-27 04:15:34,372][105620] Updated weights for policy 1, policy_version 1769712 (0.0009) [2023-12-27 04:15:34,417][105692] Updated weights for policy 0, policy_version 1765861 (0.0005) [2023-12-27 04:15:34,431][105620] Updated weights for policy 1, policy_version 1769722 (0.0010) [2023-12-27 04:15:34,474][105692] Updated weights for policy 0, policy_version 1765871 (0.0006) [2023-12-27 04:15:34,480][105620] Updated weights for policy 1, policy_version 1769732 (0.0010) [2023-12-27 04:15:34,529][105692] Updated weights for policy 0, policy_version 1765881 (0.0005) [2023-12-27 04:15:35,114][105692] Updated weights for policy 0, policy_version 1765891 (0.0007) [2023-12-27 04:15:35,165][105620] Updated weights for policy 1, policy_version 1769742 (0.0010) [2023-12-27 04:15:35,167][105692] Updated weights for policy 0, policy_version 1765901 (0.0008) [2023-12-27 04:15:35,223][105692] Updated weights for policy 0, policy_version 1765911 (0.0006) [2023-12-27 04:15:35,223][105620] Updated weights for policy 1, policy_version 1769752 (0.0010) [2023-12-27 04:15:35,278][105620] Updated weights for policy 1, policy_version 1769762 (0.0010) [2023-12-27 04:15:35,882][105692] Updated weights for policy 0, policy_version 1765921 (0.0006) [2023-12-27 04:15:35,949][105692] Updated weights for policy 0, policy_version 1765931 (0.0008) [2023-12-27 04:15:36,004][105692] Updated weights for policy 0, policy_version 1765941 (0.0007) [2023-12-27 04:15:36,010][105620] Updated weights for policy 1, policy_version 1769772 (0.0010) [2023-12-27 04:15:36,056][105692] Updated weights for policy 0, policy_version 1765951 (0.0005) [2023-12-27 04:15:36,058][105620] Updated weights for policy 1, policy_version 1769782 (0.0010) [2023-12-27 04:15:36,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 905273344. Throughput: 0: 9830.4, 1: 9984.3. Samples: 905260908. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:15:36,062][104569] Avg episode reward: [(0, '8257.896'), (1, '9353.250')] [2023-12-27 04:15:36,115][105620] Updated weights for policy 1, policy_version 1769792 (0.0010) [2023-12-27 04:15:36,811][105692] Updated weights for policy 0, policy_version 1765961 (0.0008) [2023-12-27 04:15:36,872][105692] Updated weights for policy 0, policy_version 1765971 (0.0009) [2023-12-27 04:15:36,878][105620] Updated weights for policy 1, policy_version 1769802 (0.0010) [2023-12-27 04:15:36,938][105620] Updated weights for policy 1, policy_version 1769812 (0.0010) [2023-12-27 04:15:36,938][105692] Updated weights for policy 0, policy_version 1765981 (0.0007) [2023-12-27 04:15:37,004][105620] Updated weights for policy 1, policy_version 1769822 (0.0009) [2023-12-27 04:15:37,064][105620] Updated weights for policy 1, policy_version 1769832 (0.0009) [2023-12-27 04:15:37,628][105692] Updated weights for policy 0, policy_version 1765991 (0.0009) [2023-12-27 04:15:37,686][105692] Updated weights for policy 0, policy_version 1766001 (0.0009) [2023-12-27 04:15:37,745][105692] Updated weights for policy 0, policy_version 1766011 (0.0009) [2023-12-27 04:15:37,842][105620] Updated weights for policy 1, policy_version 1769842 (0.0009) [2023-12-27 04:15:37,906][105620] Updated weights for policy 1, policy_version 1769852 (0.0008) [2023-12-27 04:15:37,964][105620] Updated weights for policy 1, policy_version 1769862 (0.0007) [2023-12-27 04:15:38,475][105692] Updated weights for policy 0, policy_version 1766021 (0.0009) [2023-12-27 04:15:38,533][105692] Updated weights for policy 0, policy_version 1766031 (0.0010) [2023-12-27 04:15:38,600][105692] Updated weights for policy 0, policy_version 1766041 (0.0010) [2023-12-27 04:15:38,696][105620] Updated weights for policy 1, policy_version 1769872 (0.0005) [2023-12-27 04:15:38,744][105620] Updated weights for policy 1, policy_version 1769882 (0.0008) [2023-12-27 04:15:38,790][105620] Updated weights for policy 1, policy_version 1769892 (0.0008) [2023-12-27 04:15:39,280][105692] Updated weights for policy 0, policy_version 1766051 (0.0009) [2023-12-27 04:15:39,351][105692] Updated weights for policy 0, policy_version 1766061 (0.0011) [2023-12-27 04:15:39,416][105692] Updated weights for policy 0, policy_version 1766071 (0.0010) [2023-12-27 04:15:39,448][105620] Updated weights for policy 1, policy_version 1769902 (0.0006) [2023-12-27 04:15:39,507][105620] Updated weights for policy 1, policy_version 1769912 (0.0008) [2023-12-27 04:15:39,568][105620] Updated weights for policy 1, policy_version 1769922 (0.0009) [2023-12-27 04:15:40,181][105692] Updated weights for policy 0, policy_version 1766081 (0.0011) [2023-12-27 04:15:40,234][105692] Updated weights for policy 0, policy_version 1766091 (0.0011) [2023-12-27 04:15:40,279][105692] Updated weights for policy 0, policy_version 1766101 (0.0010) [2023-12-27 04:15:40,332][105692] Updated weights for policy 0, policy_version 1766111 (0.0011) [2023-12-27 04:15:40,340][105620] Updated weights for policy 1, policy_version 1769932 (0.0007) [2023-12-27 04:15:40,388][105620] Updated weights for policy 1, policy_version 1769942 (0.0008) [2023-12-27 04:15:40,434][105620] Updated weights for policy 1, policy_version 1769952 (0.0006) [2023-12-27 04:15:41,010][105692] Updated weights for policy 0, policy_version 1766121 (0.0011) [2023-12-27 04:15:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 905363456. Throughput: 0: 9870.5, 1: 9974.9. Samples: 905377784. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:15:41,062][104569] Avg episode reward: [(0, '8622.983'), (1, '9353.318')] [2023-12-27 04:15:41,073][105692] Updated weights for policy 0, policy_version 1766131 (0.0011) [2023-12-27 04:15:41,101][105620] Updated weights for policy 1, policy_version 1769962 (0.0007) [2023-12-27 04:15:41,138][105692] Updated weights for policy 0, policy_version 1766141 (0.0010) [2023-12-27 04:15:41,162][105620] Updated weights for policy 1, policy_version 1769972 (0.0008) [2023-12-27 04:15:41,214][105620] Updated weights for policy 1, policy_version 1769982 (0.0006) [2023-12-27 04:15:41,271][105620] Updated weights for policy 1, policy_version 1769992 (0.0008) [2023-12-27 04:15:41,967][105692] Updated weights for policy 0, policy_version 1766151 (0.0011) [2023-12-27 04:15:42,022][105620] Updated weights for policy 1, policy_version 1770002 (0.0006) [2023-12-27 04:15:42,026][105692] Updated weights for policy 0, policy_version 1766161 (0.0009) [2023-12-27 04:15:42,078][105620] Updated weights for policy 1, policy_version 1770012 (0.0007) [2023-12-27 04:15:42,093][105692] Updated weights for policy 0, policy_version 1766171 (0.0006) [2023-12-27 04:15:42,137][105620] Updated weights for policy 1, policy_version 1770022 (0.0006) [2023-12-27 04:15:42,763][105692] Updated weights for policy 0, policy_version 1766181 (0.0008) [2023-12-27 04:15:42,832][105692] Updated weights for policy 0, policy_version 1766191 (0.0008) [2023-12-27 04:15:42,870][105620] Updated weights for policy 1, policy_version 1770032 (0.0009) [2023-12-27 04:15:42,879][105692] Updated weights for policy 0, policy_version 1766201 (0.0009) [2023-12-27 04:15:42,931][105620] Updated weights for policy 1, policy_version 1770042 (0.0008) [2023-12-27 04:15:42,979][105620] Updated weights for policy 1, policy_version 1770052 (0.0009) [2023-12-27 04:15:43,508][105692] Updated weights for policy 0, policy_version 1766211 (0.0006) [2023-12-27 04:15:43,563][105692] Updated weights for policy 0, policy_version 1766221 (0.0008) [2023-12-27 04:15:43,628][105692] Updated weights for policy 0, policy_version 1766231 (0.0010) [2023-12-27 04:15:43,745][105620] Updated weights for policy 1, policy_version 1770062 (0.0009) [2023-12-27 04:15:43,804][105620] Updated weights for policy 1, policy_version 1770072 (0.0008) [2023-12-27 04:15:43,860][105620] Updated weights for policy 1, policy_version 1770082 (0.0008) [2023-12-27 04:15:44,355][105692] Updated weights for policy 0, policy_version 1766241 (0.0010) [2023-12-27 04:15:44,418][105692] Updated weights for policy 0, policy_version 1766251 (0.0011) [2023-12-27 04:15:44,471][105692] Updated weights for policy 0, policy_version 1766261 (0.0007) [2023-12-27 04:15:44,523][105692] Updated weights for policy 0, policy_version 1766271 (0.0006) [2023-12-27 04:15:44,597][105620] Updated weights for policy 1, policy_version 1770092 (0.0009) [2023-12-27 04:15:44,662][105620] Updated weights for policy 1, policy_version 1770102 (0.0010) [2023-12-27 04:15:44,721][105620] Updated weights for policy 1, policy_version 1770112 (0.0010) [2023-12-27 04:15:45,250][105692] Updated weights for policy 0, policy_version 1766281 (0.0011) [2023-12-27 04:15:45,310][105692] Updated weights for policy 0, policy_version 1766291 (0.0011) [2023-12-27 04:15:45,367][105692] Updated weights for policy 0, policy_version 1766301 (0.0011) [2023-12-27 04:15:45,520][105620] Updated weights for policy 1, policy_version 1770122 (0.0010) [2023-12-27 04:15:45,574][105620] Updated weights for policy 1, policy_version 1770132 (0.0010) [2023-12-27 04:15:45,625][105620] Updated weights for policy 1, policy_version 1770142 (0.0010) [2023-12-27 04:15:45,668][105620] Updated weights for policy 1, policy_version 1770152 (0.0010) [2023-12-27 04:15:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 905461760. Throughput: 0: 9850.6, 1: 9964.8. Samples: 905435752. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:15:46,062][104569] Avg episode reward: [(0, '8804.763'), (1, '9353.350')] [2023-12-27 04:15:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001770152_453222400.pth... [2023-12-27 04:15:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001769000_452927488.pth [2023-12-27 04:15:46,103][105692] Updated weights for policy 0, policy_version 1766311 (0.0011) [2023-12-27 04:15:46,155][105692] Updated weights for policy 0, policy_version 1766321 (0.0010) [2023-12-27 04:15:46,207][105692] Updated weights for policy 0, policy_version 1766331 (0.0010) [2023-12-27 04:15:46,236][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001766336_452247552.pth... [2023-12-27 04:15:46,241][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001765152_451944448.pth [2023-12-27 04:15:46,433][105620] Updated weights for policy 1, policy_version 1770162 (0.0010) [2023-12-27 04:15:46,490][105620] Updated weights for policy 1, policy_version 1770172 (0.0010) [2023-12-27 04:15:46,548][105620] Updated weights for policy 1, policy_version 1770182 (0.0010) [2023-12-27 04:15:46,772][105692] Updated weights for policy 0, policy_version 1766341 (0.0007) [2023-12-27 04:15:46,825][105692] Updated weights for policy 0, policy_version 1766351 (0.0005) [2023-12-27 04:15:46,889][105692] Updated weights for policy 0, policy_version 1766361 (0.0005) [2023-12-27 04:15:47,274][105620] Updated weights for policy 1, policy_version 1770192 (0.0006) [2023-12-27 04:15:47,327][105620] Updated weights for policy 1, policy_version 1770202 (0.0007) [2023-12-27 04:15:47,378][105620] Updated weights for policy 1, policy_version 1770212 (0.0006) [2023-12-27 04:15:47,527][105692] Updated weights for policy 0, policy_version 1766371 (0.0008) [2023-12-27 04:15:47,586][105692] Updated weights for policy 0, policy_version 1766381 (0.0011) [2023-12-27 04:15:47,645][105692] Updated weights for policy 0, policy_version 1766391 (0.0011) [2023-12-27 04:15:48,057][105620] Updated weights for policy 1, policy_version 1770222 (0.0007) [2023-12-27 04:15:48,112][105620] Updated weights for policy 1, policy_version 1770232 (0.0009) [2023-12-27 04:15:48,172][105620] Updated weights for policy 1, policy_version 1770242 (0.0011) [2023-12-27 04:15:48,376][105692] Updated weights for policy 0, policy_version 1766401 (0.0010) [2023-12-27 04:15:48,425][105692] Updated weights for policy 0, policy_version 1766411 (0.0008) [2023-12-27 04:15:48,479][105692] Updated weights for policy 0, policy_version 1766421 (0.0008) [2023-12-27 04:15:48,533][105692] Updated weights for policy 0, policy_version 1766431 (0.0006) [2023-12-27 04:15:48,812][105620] Updated weights for policy 1, policy_version 1770252 (0.0009) [2023-12-27 04:15:48,870][105620] Updated weights for policy 1, policy_version 1770262 (0.0005) [2023-12-27 04:15:48,937][105620] Updated weights for policy 1, policy_version 1770272 (0.0005) [2023-12-27 04:15:49,271][105692] Updated weights for policy 0, policy_version 1766441 (0.0008) [2023-12-27 04:15:49,334][105692] Updated weights for policy 0, policy_version 1766451 (0.0007) [2023-12-27 04:15:49,396][105692] Updated weights for policy 0, policy_version 1766461 (0.0007) [2023-12-27 04:15:49,544][105620] Updated weights for policy 1, policy_version 1770282 (0.0006) [2023-12-27 04:15:49,611][105620] Updated weights for policy 1, policy_version 1770292 (0.0008) [2023-12-27 04:15:49,678][105620] Updated weights for policy 1, policy_version 1770302 (0.0009) [2023-12-27 04:15:49,742][105620] Updated weights for policy 1, policy_version 1770312 (0.0009) [2023-12-27 04:15:50,101][105692] Updated weights for policy 0, policy_version 1766471 (0.0008) [2023-12-27 04:15:50,154][105692] Updated weights for policy 0, policy_version 1766481 (0.0009) [2023-12-27 04:15:50,211][105692] Updated weights for policy 0, policy_version 1766491 (0.0009) [2023-12-27 04:15:50,491][105620] Updated weights for policy 1, policy_version 1770322 (0.0009) [2023-12-27 04:15:50,544][105620] Updated weights for policy 1, policy_version 1770332 (0.0009) [2023-12-27 04:15:50,596][105620] Updated weights for policy 1, policy_version 1770342 (0.0009) [2023-12-27 04:15:51,043][105692] Updated weights for policy 0, policy_version 1766501 (0.0009) [2023-12-27 04:15:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 905560064. Throughput: 0: 9878.5, 1: 9946.6. Samples: 905555692. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:15:51,063][104569] Avg episode reward: [(0, '8352.366'), (1, '9260.875')] [2023-12-27 04:15:51,112][105692] Updated weights for policy 0, policy_version 1766511 (0.0010) [2023-12-27 04:15:51,177][105692] Updated weights for policy 0, policy_version 1766521 (0.0007) [2023-12-27 04:15:51,400][105620] Updated weights for policy 1, policy_version 1770352 (0.0007) [2023-12-27 04:15:51,455][105620] Updated weights for policy 1, policy_version 1770362 (0.0009) [2023-12-27 04:15:51,512][105620] Updated weights for policy 1, policy_version 1770373 (0.0009) [2023-12-27 04:15:51,867][105692] Updated weights for policy 0, policy_version 1766531 (0.0009) [2023-12-27 04:15:51,934][105692] Updated weights for policy 0, policy_version 1766541 (0.0008) [2023-12-27 04:15:51,993][105692] Updated weights for policy 0, policy_version 1766551 (0.0008) [2023-12-27 04:15:52,284][105620] Updated weights for policy 1, policy_version 1770383 (0.0010) [2023-12-27 04:15:52,344][105620] Updated weights for policy 1, policy_version 1770393 (0.0011) [2023-12-27 04:15:52,408][105620] Updated weights for policy 1, policy_version 1770403 (0.0009) [2023-12-27 04:15:52,694][105692] Updated weights for policy 0, policy_version 1766561 (0.0008) [2023-12-27 04:15:52,755][105692] Updated weights for policy 0, policy_version 1766571 (0.0007) [2023-12-27 04:15:52,808][105692] Updated weights for policy 0, policy_version 1766581 (0.0010) [2023-12-27 04:15:52,873][105692] Updated weights for policy 0, policy_version 1766591 (0.0009) [2023-12-27 04:15:53,131][105620] Updated weights for policy 1, policy_version 1770413 (0.0010) [2023-12-27 04:15:53,193][105620] Updated weights for policy 1, policy_version 1770423 (0.0010) [2023-12-27 04:15:53,250][105620] Updated weights for policy 1, policy_version 1770433 (0.0010) [2023-12-27 04:15:53,463][105692] Updated weights for policy 0, policy_version 1766601 (0.0006) [2023-12-27 04:15:53,510][105692] Updated weights for policy 0, policy_version 1766611 (0.0005) [2023-12-27 04:15:53,561][105692] Updated weights for policy 0, policy_version 1766621 (0.0005) [2023-12-27 04:15:54,014][105620] Updated weights for policy 1, policy_version 1770444 (0.0009) [2023-12-27 04:15:54,072][105620] Updated weights for policy 1, policy_version 1770454 (0.0010) [2023-12-27 04:15:54,131][105620] Updated weights for policy 1, policy_version 1770464 (0.0009) [2023-12-27 04:15:54,143][105692] Updated weights for policy 0, policy_version 1766631 (0.0005) [2023-12-27 04:15:54,206][105692] Updated weights for policy 0, policy_version 1766641 (0.0006) [2023-12-27 04:15:54,257][105692] Updated weights for policy 0, policy_version 1766651 (0.0007) [2023-12-27 04:15:54,930][105620] Updated weights for policy 1, policy_version 1770474 (0.0009) [2023-12-27 04:15:54,972][105692] Updated weights for policy 0, policy_version 1766661 (0.0008) [2023-12-27 04:15:54,990][105620] Updated weights for policy 1, policy_version 1770484 (0.0009) [2023-12-27 04:15:55,024][105692] Updated weights for policy 0, policy_version 1766671 (0.0008) [2023-12-27 04:15:55,041][105620] Updated weights for policy 1, policy_version 1770494 (0.0005) [2023-12-27 04:15:55,085][105692] Updated weights for policy 0, policy_version 1766681 (0.0007) [2023-12-27 04:15:55,106][105620] Updated weights for policy 1, policy_version 1770504 (0.0007) [2023-12-27 04:15:55,820][105620] Updated weights for policy 1, policy_version 1770514 (0.0010) [2023-12-27 04:15:55,834][105692] Updated weights for policy 0, policy_version 1766691 (0.0008) [2023-12-27 04:15:55,881][105620] Updated weights for policy 1, policy_version 1770524 (0.0008) [2023-12-27 04:15:55,890][105692] Updated weights for policy 0, policy_version 1766701 (0.0008) [2023-12-27 04:15:55,941][105620] Updated weights for policy 1, policy_version 1770534 (0.0005) [2023-12-27 04:15:55,950][105692] Updated weights for policy 0, policy_version 1766711 (0.0009) [2023-12-27 04:15:56,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 905666560. Throughput: 0: 9912.6, 1: 9801.5. Samples: 905670240. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:15:56,063][104569] Avg episode reward: [(0, '8170.301'), (1, '9260.778')] [2023-12-27 04:15:56,482][105620] Updated weights for policy 1, policy_version 1770544 (0.0007) [2023-12-27 04:15:56,544][105620] Updated weights for policy 1, policy_version 1770554 (0.0010) [2023-12-27 04:15:56,602][105620] Updated weights for policy 1, policy_version 1770564 (0.0010) [2023-12-27 04:15:56,638][105692] Updated weights for policy 0, policy_version 1766721 (0.0009) [2023-12-27 04:15:56,700][105692] Updated weights for policy 0, policy_version 1766731 (0.0005) [2023-12-27 04:15:56,763][105692] Updated weights for policy 0, policy_version 1766741 (0.0009) [2023-12-27 04:15:56,824][105692] Updated weights for policy 0, policy_version 1766751 (0.0009) [2023-12-27 04:15:57,245][105620] Updated weights for policy 1, policy_version 1770574 (0.0007) [2023-12-27 04:15:57,299][105620] Updated weights for policy 1, policy_version 1770584 (0.0006) [2023-12-27 04:15:57,348][105620] Updated weights for policy 1, policy_version 1770594 (0.0008) [2023-12-27 04:15:57,434][105692] Updated weights for policy 0, policy_version 1766761 (0.0010) [2023-12-27 04:15:57,492][105692] Updated weights for policy 0, policy_version 1766771 (0.0011) [2023-12-27 04:15:57,543][105692] Updated weights for policy 0, policy_version 1766781 (0.0010) [2023-12-27 04:15:58,069][105620] Updated weights for policy 1, policy_version 1770604 (0.0008) [2023-12-27 04:15:58,133][105620] Updated weights for policy 1, policy_version 1770614 (0.0008) [2023-12-27 04:15:58,203][105620] Updated weights for policy 1, policy_version 1770624 (0.0008) [2023-12-27 04:15:58,278][105692] Updated weights for policy 0, policy_version 1766791 (0.0009) [2023-12-27 04:15:58,343][105692] Updated weights for policy 0, policy_version 1766801 (0.0008) [2023-12-27 04:15:58,412][105692] Updated weights for policy 0, policy_version 1766811 (0.0008) [2023-12-27 04:15:59,053][105620] Updated weights for policy 1, policy_version 1770634 (0.0008) [2023-12-27 04:15:59,110][105620] Updated weights for policy 1, policy_version 1770644 (0.0007) [2023-12-27 04:15:59,167][105620] Updated weights for policy 1, policy_version 1770654 (0.0007) [2023-12-27 04:15:59,177][105692] Updated weights for policy 0, policy_version 1766821 (0.0008) [2023-12-27 04:15:59,238][105620] Updated weights for policy 1, policy_version 1770664 (0.0006) [2023-12-27 04:15:59,242][105692] Updated weights for policy 0, policy_version 1766831 (0.0009) [2023-12-27 04:15:59,307][105692] Updated weights for policy 0, policy_version 1766841 (0.0009) [2023-12-27 04:16:00,021][105620] Updated weights for policy 1, policy_version 1770674 (0.0008) [2023-12-27 04:16:00,048][105692] Updated weights for policy 0, policy_version 1766851 (0.0008) [2023-12-27 04:16:00,080][105620] Updated weights for policy 1, policy_version 1770684 (0.0008) [2023-12-27 04:16:00,096][105692] Updated weights for policy 0, policy_version 1766861 (0.0008) [2023-12-27 04:16:00,131][105620] Updated weights for policy 1, policy_version 1770694 (0.0008) [2023-12-27 04:16:00,152][105692] Updated weights for policy 0, policy_version 1766871 (0.0008) [2023-12-27 04:16:00,199][105585] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000000 [2023-12-27 04:16:00,825][105692] Updated weights for policy 0, policy_version 1766881 (0.0007) [2023-12-27 04:16:00,872][105692] Updated weights for policy 0, policy_version 1766891 (0.0007) [2023-12-27 04:16:00,888][105620] Updated weights for policy 1, policy_version 1770704 (0.0009) [2023-12-27 04:16:00,928][105692] Updated weights for policy 0, policy_version 1766901 (0.0007) [2023-12-27 04:16:00,946][105620] Updated weights for policy 1, policy_version 1770714 (0.0008) [2023-12-27 04:16:00,981][105692] Updated weights for policy 0, policy_version 1766911 (0.0007) [2023-12-27 04:16:01,008][105620] Updated weights for policy 1, policy_version 1770724 (0.0009) [2023-12-27 04:16:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 905764864. Throughput: 0: 9963.6, 1: 9832.8. Samples: 905729876. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:16:01,063][104569] Avg episode reward: [(0, '8444.265'), (1, '9260.731')] [2023-12-27 04:16:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001770728_453369856.pth... [2023-12-27 04:16:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001766912_452395008.pth... [2023-12-27 04:16:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001769576_453074944.pth [2023-12-27 04:16:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001765728_452091904.pth [2023-12-27 04:16:01,752][105620] Updated weights for policy 1, policy_version 1770734 (0.0008) [2023-12-27 04:16:01,780][105692] Updated weights for policy 0, policy_version 1766921 (0.0009) [2023-12-27 04:16:01,810][105620] Updated weights for policy 1, policy_version 1770744 (0.0008) [2023-12-27 04:16:01,837][105692] Updated weights for policy 0, policy_version 1766931 (0.0008) [2023-12-27 04:16:01,871][105620] Updated weights for policy 1, policy_version 1770754 (0.0008) [2023-12-27 04:16:01,894][105692] Updated weights for policy 0, policy_version 1766941 (0.0011) [2023-12-27 04:16:02,635][105620] Updated weights for policy 1, policy_version 1770764 (0.0008) [2023-12-27 04:16:02,677][105692] Updated weights for policy 0, policy_version 1766951 (0.0010) [2023-12-27 04:16:02,688][105620] Updated weights for policy 1, policy_version 1770774 (0.0006) [2023-12-27 04:16:02,733][105692] Updated weights for policy 0, policy_version 1766961 (0.0007) [2023-12-27 04:16:02,737][105620] Updated weights for policy 1, policy_version 1770784 (0.0007) [2023-12-27 04:16:02,791][105692] Updated weights for policy 0, policy_version 1766971 (0.0007) [2023-12-27 04:16:03,419][105620] Updated weights for policy 1, policy_version 1770794 (0.0006) [2023-12-27 04:16:03,483][105620] Updated weights for policy 1, policy_version 1770804 (0.0008) [2023-12-27 04:16:03,546][105620] Updated weights for policy 1, policy_version 1770814 (0.0009) [2023-12-27 04:16:03,612][105620] Updated weights for policy 1, policy_version 1770824 (0.0009) [2023-12-27 04:16:03,661][105692] Updated weights for policy 0, policy_version 1766981 (0.0009) [2023-12-27 04:16:03,723][105692] Updated weights for policy 0, policy_version 1766991 (0.0009) [2023-12-27 04:16:03,787][105692] Updated weights for policy 0, policy_version 1767001 (0.0006) [2023-12-27 04:16:04,352][105620] Updated weights for policy 1, policy_version 1770834 (0.0006) [2023-12-27 04:16:04,423][105620] Updated weights for policy 1, policy_version 1770844 (0.0009) [2023-12-27 04:16:04,486][105620] Updated weights for policy 1, policy_version 1770854 (0.0009) [2023-12-27 04:16:04,513][105692] Updated weights for policy 0, policy_version 1767011 (0.0007) [2023-12-27 04:16:04,571][105692] Updated weights for policy 0, policy_version 1767021 (0.0009) [2023-12-27 04:16:04,628][105692] Updated weights for policy 0, policy_version 1767031 (0.0009) [2023-12-27 04:16:05,204][105620] Updated weights for policy 1, policy_version 1770864 (0.0009) [2023-12-27 04:16:05,267][105620] Updated weights for policy 1, policy_version 1770874 (0.0008) [2023-12-27 04:16:05,331][105620] Updated weights for policy 1, policy_version 1770884 (0.0008) [2023-12-27 04:16:05,412][105692] Updated weights for policy 0, policy_version 1767041 (0.0010) [2023-12-27 04:16:05,467][105692] Updated weights for policy 0, policy_version 1767051 (0.0009) [2023-12-27 04:16:05,530][105692] Updated weights for policy 0, policy_version 1767061 (0.0009) [2023-12-27 04:16:05,582][105692] Updated weights for policy 0, policy_version 1767071 (0.0009) [2023-12-27 04:16:06,047][105620] Updated weights for policy 1, policy_version 1770894 (0.0009) [2023-12-27 04:16:06,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 905846784. Throughput: 0: 9871.0, 1: 9733.4. Samples: 905841148. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:16:06,063][104569] Avg episode reward: [(0, '8439.294'), (1, '9168.448')] [2023-12-27 04:16:06,110][105620] Updated weights for policy 1, policy_version 1770904 (0.0008) [2023-12-27 04:16:06,182][105620] Updated weights for policy 1, policy_version 1770914 (0.0009) [2023-12-27 04:16:06,382][105692] Updated weights for policy 0, policy_version 1767081 (0.0006) [2023-12-27 04:16:06,446][105692] Updated weights for policy 0, policy_version 1767091 (0.0006) [2023-12-27 04:16:06,503][105692] Updated weights for policy 0, policy_version 1767101 (0.0009) [2023-12-27 04:16:06,962][105620] Updated weights for policy 1, policy_version 1770924 (0.0009) [2023-12-27 04:16:07,028][105620] Updated weights for policy 1, policy_version 1770934 (0.0009) [2023-12-27 04:16:07,095][105620] Updated weights for policy 1, policy_version 1770944 (0.0009) [2023-12-27 04:16:07,214][105692] Updated weights for policy 0, policy_version 1767111 (0.0008) [2023-12-27 04:16:07,283][105692] Updated weights for policy 0, policy_version 1767121 (0.0005) [2023-12-27 04:16:07,341][105692] Updated weights for policy 0, policy_version 1767131 (0.0005) [2023-12-27 04:16:07,825][105620] Updated weights for policy 1, policy_version 1770954 (0.0010) [2023-12-27 04:16:07,891][105620] Updated weights for policy 1, policy_version 1770964 (0.0009) [2023-12-27 04:16:07,951][105620] Updated weights for policy 1, policy_version 1770974 (0.0006) [2023-12-27 04:16:08,007][105620] Updated weights for policy 1, policy_version 1770984 (0.0006) [2023-12-27 04:16:08,058][105692] Updated weights for policy 0, policy_version 1767141 (0.0006) [2023-12-27 04:16:08,126][105692] Updated weights for policy 0, policy_version 1767151 (0.0007) [2023-12-27 04:16:08,194][105692] Updated weights for policy 0, policy_version 1767161 (0.0009) [2023-12-27 04:16:08,694][105620] Updated weights for policy 1, policy_version 1770994 (0.0007) [2023-12-27 04:16:08,758][105620] Updated weights for policy 1, policy_version 1771004 (0.0010) [2023-12-27 04:16:08,826][105620] Updated weights for policy 1, policy_version 1771014 (0.0009) [2023-12-27 04:16:08,956][105692] Updated weights for policy 0, policy_version 1767171 (0.0009) [2023-12-27 04:16:09,021][105692] Updated weights for policy 0, policy_version 1767181 (0.0009) [2023-12-27 04:16:09,073][105692] Updated weights for policy 0, policy_version 1767191 (0.0009) [2023-12-27 04:16:09,615][105620] Updated weights for policy 1, policy_version 1771024 (0.0009) [2023-12-27 04:16:09,674][105620] Updated weights for policy 1, policy_version 1771034 (0.0009) [2023-12-27 04:16:09,735][105620] Updated weights for policy 1, policy_version 1771044 (0.0007) [2023-12-27 04:16:09,899][105692] Updated weights for policy 0, policy_version 1767201 (0.0009) [2023-12-27 04:16:09,963][105692] Updated weights for policy 0, policy_version 1767211 (0.0008) [2023-12-27 04:16:10,031][105692] Updated weights for policy 0, policy_version 1767221 (0.0010) [2023-12-27 04:16:10,095][105692] Updated weights for policy 0, policy_version 1767231 (0.0010) [2023-12-27 04:16:10,485][105620] Updated weights for policy 1, policy_version 1771054 (0.0008) [2023-12-27 04:16:10,536][105620] Updated weights for policy 1, policy_version 1771064 (0.0008) [2023-12-27 04:16:10,598][105620] Updated weights for policy 1, policy_version 1771074 (0.0009) [2023-12-27 04:16:10,855][105692] Updated weights for policy 0, policy_version 1767241 (0.0006) [2023-12-27 04:16:10,918][105692] Updated weights for policy 0, policy_version 1767251 (0.0008) [2023-12-27 04:16:10,977][105692] Updated weights for policy 0, policy_version 1767261 (0.0009) [2023-12-27 04:16:11,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 905945088. Throughput: 0: 9800.2, 1: 9704.7. Samples: 905951004. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:16:11,063][104569] Avg episode reward: [(0, '8350.006'), (1, '9261.072')] [2023-12-27 04:16:11,368][105620] Updated weights for policy 1, policy_version 1771084 (0.0009) [2023-12-27 04:16:11,426][105620] Updated weights for policy 1, policy_version 1771094 (0.0009) [2023-12-27 04:16:11,484][105620] Updated weights for policy 1, policy_version 1771104 (0.0010) [2023-12-27 04:16:11,711][105692] Updated weights for policy 0, policy_version 1767271 (0.0008) [2023-12-27 04:16:11,781][105692] Updated weights for policy 0, policy_version 1767281 (0.0008) [2023-12-27 04:16:11,840][105692] Updated weights for policy 0, policy_version 1767291 (0.0009) [2023-12-27 04:16:12,303][105620] Updated weights for policy 1, policy_version 1771114 (0.0010) [2023-12-27 04:16:12,369][105620] Updated weights for policy 1, policy_version 1771124 (0.0009) [2023-12-27 04:16:12,427][105620] Updated weights for policy 1, policy_version 1771134 (0.0006) [2023-12-27 04:16:12,474][105620] Updated weights for policy 1, policy_version 1771144 (0.0005) [2023-12-27 04:16:12,627][105692] Updated weights for policy 0, policy_version 1767301 (0.0008) [2023-12-27 04:16:12,682][105692] Updated weights for policy 0, policy_version 1767311 (0.0009) [2023-12-27 04:16:12,741][105692] Updated weights for policy 0, policy_version 1767321 (0.0009) [2023-12-27 04:16:13,168][105620] Updated weights for policy 1, policy_version 1771154 (0.0009) [2023-12-27 04:16:13,231][105620] Updated weights for policy 1, policy_version 1771164 (0.0009) [2023-12-27 04:16:13,290][105620] Updated weights for policy 1, policy_version 1771174 (0.0009) [2023-12-27 04:16:13,483][105692] Updated weights for policy 0, policy_version 1767331 (0.0008) [2023-12-27 04:16:13,538][105692] Updated weights for policy 0, policy_version 1767341 (0.0005) [2023-12-27 04:16:13,597][105692] Updated weights for policy 0, policy_version 1767351 (0.0005) [2023-12-27 04:16:14,039][105620] Updated weights for policy 1, policy_version 1771184 (0.0009) [2023-12-27 04:16:14,095][105620] Updated weights for policy 1, policy_version 1771194 (0.0009) [2023-12-27 04:16:14,154][105620] Updated weights for policy 1, policy_version 1771204 (0.0009) [2023-12-27 04:16:14,242][105692] Updated weights for policy 0, policy_version 1767361 (0.0005) [2023-12-27 04:16:14,293][105692] Updated weights for policy 0, policy_version 1767371 (0.0009) [2023-12-27 04:16:14,347][105692] Updated weights for policy 0, policy_version 1767381 (0.0009) [2023-12-27 04:16:14,404][105692] Updated weights for policy 0, policy_version 1767391 (0.0009) [2023-12-27 04:16:14,961][105620] Updated weights for policy 1, policy_version 1771214 (0.0010) [2023-12-27 04:16:15,027][105620] Updated weights for policy 1, policy_version 1771224 (0.0009) [2023-12-27 04:16:15,094][105620] Updated weights for policy 1, policy_version 1771234 (0.0009) [2023-12-27 04:16:15,216][105692] Updated weights for policy 0, policy_version 1767401 (0.0009) [2023-12-27 04:16:15,274][105692] Updated weights for policy 0, policy_version 1767411 (0.0009) [2023-12-27 04:16:15,337][105692] Updated weights for policy 0, policy_version 1767421 (0.0009) [2023-12-27 04:16:15,732][105620] Updated weights for policy 1, policy_version 1771244 (0.0007) [2023-12-27 04:16:15,794][105620] Updated weights for policy 1, policy_version 1771254 (0.0006) [2023-12-27 04:16:15,841][105620] Updated weights for policy 1, policy_version 1771264 (0.0006) [2023-12-27 04:16:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 906035200. Throughput: 0: 9733.4, 1: 9585.9. Samples: 906007340. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:16:16,063][104569] Avg episode reward: [(0, '8532.819'), (1, '9353.484')] [2023-12-27 04:16:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001771272_453509120.pth... [2023-12-27 04:16:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001767424_452526080.pth... [2023-12-27 04:16:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001766336_452247552.pth [2023-12-27 04:16:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001770152_453222400.pth [2023-12-27 04:16:16,226][105692] Updated weights for policy 0, policy_version 1767431 (0.0009) [2023-12-27 04:16:16,290][105692] Updated weights for policy 0, policy_version 1767441 (0.0009) [2023-12-27 04:16:16,348][105692] Updated weights for policy 0, policy_version 1767451 (0.0009) [2023-12-27 04:16:16,422][105620] Updated weights for policy 1, policy_version 1771274 (0.0005) [2023-12-27 04:16:16,482][105620] Updated weights for policy 1, policy_version 1771284 (0.0005) [2023-12-27 04:16:16,545][105620] Updated weights for policy 1, policy_version 1771294 (0.0009) [2023-12-27 04:16:16,590][105620] Updated weights for policy 1, policy_version 1771304 (0.0010) [2023-12-27 04:16:17,159][105692] Updated weights for policy 0, policy_version 1767461 (0.0010) [2023-12-27 04:16:17,215][105692] Updated weights for policy 0, policy_version 1767471 (0.0009) [2023-12-27 04:16:17,243][105620] Updated weights for policy 1, policy_version 1771314 (0.0006) [2023-12-27 04:16:17,268][105692] Updated weights for policy 0, policy_version 1767481 (0.0009) [2023-12-27 04:16:17,299][105620] Updated weights for policy 1, policy_version 1771324 (0.0011) [2023-12-27 04:16:17,357][105620] Updated weights for policy 1, policy_version 1771334 (0.0011) [2023-12-27 04:16:18,027][105692] Updated weights for policy 0, policy_version 1767491 (0.0006) [2023-12-27 04:16:18,080][105620] Updated weights for policy 1, policy_version 1771344 (0.0011) [2023-12-27 04:16:18,088][105692] Updated weights for policy 0, policy_version 1767501 (0.0008) [2023-12-27 04:16:18,136][105620] Updated weights for policy 1, policy_version 1771354 (0.0011) [2023-12-27 04:16:18,145][105692] Updated weights for policy 0, policy_version 1767511 (0.0008) [2023-12-27 04:16:18,197][105620] Updated weights for policy 1, policy_version 1771364 (0.0011) [2023-12-27 04:16:18,908][105620] Updated weights for policy 1, policy_version 1771374 (0.0008) [2023-12-27 04:16:18,947][105692] Updated weights for policy 0, policy_version 1767521 (0.0006) [2023-12-27 04:16:18,955][105620] Updated weights for policy 1, policy_version 1771384 (0.0005) [2023-12-27 04:16:19,011][105692] Updated weights for policy 0, policy_version 1767531 (0.0008) [2023-12-27 04:16:19,011][105620] Updated weights for policy 1, policy_version 1771394 (0.0008) [2023-12-27 04:16:19,067][105692] Updated weights for policy 0, policy_version 1767541 (0.0008) [2023-12-27 04:16:19,129][105692] Updated weights for policy 0, policy_version 1767551 (0.0009) [2023-12-27 04:16:19,694][105620] Updated weights for policy 1, policy_version 1771404 (0.0010) [2023-12-27 04:16:19,758][105620] Updated weights for policy 1, policy_version 1771414 (0.0009) [2023-12-27 04:16:19,825][105620] Updated weights for policy 1, policy_version 1771424 (0.0009) [2023-12-27 04:16:19,916][105692] Updated weights for policy 0, policy_version 1767561 (0.0009) [2023-12-27 04:16:19,982][105692] Updated weights for policy 0, policy_version 1767571 (0.0008) [2023-12-27 04:16:20,045][105692] Updated weights for policy 0, policy_version 1767581 (0.0009) [2023-12-27 04:16:20,626][105620] Updated weights for policy 1, policy_version 1771434 (0.0008) [2023-12-27 04:16:20,684][105620] Updated weights for policy 1, policy_version 1771444 (0.0009) [2023-12-27 04:16:20,748][105620] Updated weights for policy 1, policy_version 1771454 (0.0008) [2023-12-27 04:16:20,819][105620] Updated weights for policy 1, policy_version 1771464 (0.0008) [2023-12-27 04:16:20,821][105692] Updated weights for policy 0, policy_version 1767591 (0.0007) [2023-12-27 04:16:20,868][105692] Updated weights for policy 0, policy_version 1767601 (0.0008) [2023-12-27 04:16:20,916][105692] Updated weights for policy 0, policy_version 1767611 (0.0009) [2023-12-27 04:16:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 906133504. Throughput: 0: 9543.7, 1: 9552.9. Samples: 906120260. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:16:21,063][104569] Avg episode reward: [(0, '8442.201'), (1, '9353.331')] [2023-12-27 04:16:21,540][105620] Updated weights for policy 1, policy_version 1771474 (0.0006) [2023-12-27 04:16:21,601][105620] Updated weights for policy 1, policy_version 1771484 (0.0009) [2023-12-27 04:16:21,669][105620] Updated weights for policy 1, policy_version 1771494 (0.0009) [2023-12-27 04:16:21,724][105692] Updated weights for policy 0, policy_version 1767621 (0.0009) [2023-12-27 04:16:21,786][105692] Updated weights for policy 0, policy_version 1767631 (0.0008) [2023-12-27 04:16:21,856][105692] Updated weights for policy 0, policy_version 1767641 (0.0008) [2023-12-27 04:16:22,311][105620] Updated weights for policy 1, policy_version 1771504 (0.0008) [2023-12-27 04:16:22,381][105620] Updated weights for policy 1, policy_version 1771514 (0.0010) [2023-12-27 04:16:22,440][105620] Updated weights for policy 1, policy_version 1771524 (0.0009) [2023-12-27 04:16:22,572][105692] Updated weights for policy 0, policy_version 1767651 (0.0007) [2023-12-27 04:16:22,645][105692] Updated weights for policy 0, policy_version 1767661 (0.0006) [2023-12-27 04:16:22,706][105692] Updated weights for policy 0, policy_version 1767671 (0.0009) [2023-12-27 04:16:23,113][105620] Updated weights for policy 1, policy_version 1771534 (0.0007) [2023-12-27 04:16:23,171][105620] Updated weights for policy 1, policy_version 1771544 (0.0005) [2023-12-27 04:16:23,232][105620] Updated weights for policy 1, policy_version 1771554 (0.0006) [2023-12-27 04:16:23,472][105692] Updated weights for policy 0, policy_version 1767681 (0.0009) [2023-12-27 04:16:23,518][105692] Updated weights for policy 0, policy_version 1767691 (0.0008) [2023-12-27 04:16:23,569][105692] Updated weights for policy 0, policy_version 1767701 (0.0009) [2023-12-27 04:16:23,627][105692] Updated weights for policy 0, policy_version 1767711 (0.0009) [2023-12-27 04:16:23,819][105620] Updated weights for policy 1, policy_version 1771564 (0.0006) [2023-12-27 04:16:23,872][105620] Updated weights for policy 1, policy_version 1771574 (0.0005) [2023-12-27 04:16:23,922][105620] Updated weights for policy 1, policy_version 1771584 (0.0005) [2023-12-27 04:16:24,490][105692] Updated weights for policy 0, policy_version 1767721 (0.0011) [2023-12-27 04:16:24,524][105620] Updated weights for policy 1, policy_version 1771594 (0.0007) [2023-12-27 04:16:24,539][105692] Updated weights for policy 0, policy_version 1767731 (0.0010) [2023-12-27 04:16:24,569][105620] Updated weights for policy 1, policy_version 1771604 (0.0005) [2023-12-27 04:16:24,591][105692] Updated weights for policy 0, policy_version 1767741 (0.0011) [2023-12-27 04:16:24,630][105620] Updated weights for policy 1, policy_version 1771614 (0.0005) [2023-12-27 04:16:24,692][105620] Updated weights for policy 1, policy_version 1771624 (0.0005) [2023-12-27 04:16:25,205][105692] Updated weights for policy 0, policy_version 1767751 (0.0009) [2023-12-27 04:16:25,256][105692] Updated weights for policy 0, policy_version 1767761 (0.0005) [2023-12-27 04:16:25,303][105692] Updated weights for policy 0, policy_version 1767771 (0.0005) [2023-12-27 04:16:25,345][105620] Updated weights for policy 1, policy_version 1771634 (0.0009) [2023-12-27 04:16:25,405][105620] Updated weights for policy 1, policy_version 1771644 (0.0009) [2023-12-27 04:16:25,461][105620] Updated weights for policy 1, policy_version 1771654 (0.0013) [2023-12-27 04:16:25,841][105692] Updated weights for policy 0, policy_version 1767781 (0.0008) [2023-12-27 04:16:25,889][105692] Updated weights for policy 0, policy_version 1767791 (0.0010) [2023-12-27 04:16:25,947][105692] Updated weights for policy 0, policy_version 1767801 (0.0010) [2023-12-27 04:16:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 906231808. Throughput: 0: 9487.4, 1: 9629.5. Samples: 906238044. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:16:26,062][104569] Avg episode reward: [(0, '8171.080'), (1, '9353.141')] [2023-12-27 04:16:26,208][105620] Updated weights for policy 1, policy_version 1771664 (0.0008) [2023-12-27 04:16:26,271][105620] Updated weights for policy 1, policy_version 1771674 (0.0008) [2023-12-27 04:16:26,328][105620] Updated weights for policy 1, policy_version 1771684 (0.0009) [2023-12-27 04:16:26,730][105692] Updated weights for policy 0, policy_version 1767811 (0.0010) [2023-12-27 04:16:26,782][105692] Updated weights for policy 0, policy_version 1767821 (0.0007) [2023-12-27 04:16:26,834][105692] Updated weights for policy 0, policy_version 1767831 (0.0007) [2023-12-27 04:16:26,951][105620] Updated weights for policy 1, policy_version 1771694 (0.0009) [2023-12-27 04:16:26,999][105620] Updated weights for policy 1, policy_version 1771704 (0.0010) [2023-12-27 04:16:27,057][105620] Updated weights for policy 1, policy_version 1771714 (0.0010) [2023-12-27 04:16:27,488][105692] Updated weights for policy 0, policy_version 1767841 (0.0008) [2023-12-27 04:16:27,535][105692] Updated weights for policy 0, policy_version 1767851 (0.0010) [2023-12-27 04:16:27,586][105692] Updated weights for policy 0, policy_version 1767861 (0.0010) [2023-12-27 04:16:27,646][105692] Updated weights for policy 0, policy_version 1767871 (0.0010) [2023-12-27 04:16:27,731][105620] Updated weights for policy 1, policy_version 1771724 (0.0009) [2023-12-27 04:16:27,774][105620] Updated weights for policy 1, policy_version 1771734 (0.0008) [2023-12-27 04:16:27,827][105620] Updated weights for policy 1, policy_version 1771744 (0.0009) [2023-12-27 04:16:28,332][105692] Updated weights for policy 0, policy_version 1767881 (0.0008) [2023-12-27 04:16:28,394][105692] Updated weights for policy 0, policy_version 1767891 (0.0009) [2023-12-27 04:16:28,452][105692] Updated weights for policy 0, policy_version 1767901 (0.0010) [2023-12-27 04:16:28,474][105620] Updated weights for policy 1, policy_version 1771755 (0.0008) [2023-12-27 04:16:28,530][105620] Updated weights for policy 1, policy_version 1771765 (0.0008) [2023-12-27 04:16:28,588][105620] Updated weights for policy 1, policy_version 1771775 (0.0009) [2023-12-27 04:16:29,139][105692] Updated weights for policy 0, policy_version 1767911 (0.0010) [2023-12-27 04:16:29,200][105692] Updated weights for policy 0, policy_version 1767921 (0.0010) [2023-12-27 04:16:29,243][105620] Updated weights for policy 1, policy_version 1771785 (0.0008) [2023-12-27 04:16:29,265][105692] Updated weights for policy 0, policy_version 1767931 (0.0011) [2023-12-27 04:16:29,306][105620] Updated weights for policy 1, policy_version 1771795 (0.0007) [2023-12-27 04:16:29,371][105620] Updated weights for policy 1, policy_version 1771805 (0.0006) [2023-12-27 04:16:29,432][105620] Updated weights for policy 1, policy_version 1771815 (0.0006) [2023-12-27 04:16:29,978][105692] Updated weights for policy 0, policy_version 1767941 (0.0009) [2023-12-27 04:16:30,041][105692] Updated weights for policy 0, policy_version 1767951 (0.0009) [2023-12-27 04:16:30,092][105692] Updated weights for policy 0, policy_version 1767961 (0.0008) [2023-12-27 04:16:30,100][105620] Updated weights for policy 1, policy_version 1771825 (0.0008) [2023-12-27 04:16:30,158][105620] Updated weights for policy 1, policy_version 1771835 (0.0009) [2023-12-27 04:16:30,216][105620] Updated weights for policy 1, policy_version 1771845 (0.0009) [2023-12-27 04:16:30,880][105692] Updated weights for policy 0, policy_version 1767971 (0.0008) [2023-12-27 04:16:30,882][105620] Updated weights for policy 1, policy_version 1771855 (0.0009) [2023-12-27 04:16:30,931][105692] Updated weights for policy 0, policy_version 1767981 (0.0006) [2023-12-27 04:16:30,938][105620] Updated weights for policy 1, policy_version 1771865 (0.0008) [2023-12-27 04:16:30,980][105692] Updated weights for policy 0, policy_version 1767991 (0.0006) [2023-12-27 04:16:30,993][105620] Updated weights for policy 1, policy_version 1771875 (0.0008) [2023-12-27 04:16:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 906338304. Throughput: 0: 9506.0, 1: 9716.5. Samples: 906300764. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:16:31,062][104569] Avg episode reward: [(0, '8443.621'), (1, '9352.975')] [2023-12-27 04:16:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001768000_452673536.pth... [2023-12-27 04:16:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001771880_453664768.pth... [2023-12-27 04:16:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001766912_452395008.pth [2023-12-27 04:16:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001770728_453369856.pth [2023-12-27 04:16:31,675][105692] Updated weights for policy 0, policy_version 1768001 (0.0006) [2023-12-27 04:16:31,681][105620] Updated weights for policy 1, policy_version 1771885 (0.0007) [2023-12-27 04:16:31,738][105692] Updated weights for policy 0, policy_version 1768011 (0.0008) [2023-12-27 04:16:31,748][105620] Updated weights for policy 1, policy_version 1771895 (0.0007) [2023-12-27 04:16:31,795][105692] Updated weights for policy 0, policy_version 1768021 (0.0007) [2023-12-27 04:16:31,809][105620] Updated weights for policy 1, policy_version 1771905 (0.0008) [2023-12-27 04:16:31,844][105692] Updated weights for policy 0, policy_version 1768031 (0.0006) [2023-12-27 04:16:32,551][105620] Updated weights for policy 1, policy_version 1771915 (0.0007) [2023-12-27 04:16:32,603][105620] Updated weights for policy 1, policy_version 1771925 (0.0008) [2023-12-27 04:16:32,604][105692] Updated weights for policy 0, policy_version 1768041 (0.0007) [2023-12-27 04:16:32,653][105620] Updated weights for policy 1, policy_version 1771935 (0.0007) [2023-12-27 04:16:32,655][105692] Updated weights for policy 0, policy_version 1768051 (0.0006) [2023-12-27 04:16:32,704][105692] Updated weights for policy 0, policy_version 1768061 (0.0008) [2023-12-27 04:16:33,394][105620] Updated weights for policy 1, policy_version 1771945 (0.0007) [2023-12-27 04:16:33,453][105620] Updated weights for policy 1, policy_version 1771955 (0.0008) [2023-12-27 04:16:33,464][105692] Updated weights for policy 0, policy_version 1768071 (0.0009) [2023-12-27 04:16:33,513][105620] Updated weights for policy 1, policy_version 1771965 (0.0007) [2023-12-27 04:16:33,524][105692] Updated weights for policy 0, policy_version 1768081 (0.0006) [2023-12-27 04:16:33,566][105620] Updated weights for policy 1, policy_version 1771975 (0.0008) [2023-12-27 04:16:33,575][105692] Updated weights for policy 0, policy_version 1768091 (0.0006) [2023-12-27 04:16:34,270][105620] Updated weights for policy 1, policy_version 1771985 (0.0009) [2023-12-27 04:16:34,317][105692] Updated weights for policy 0, policy_version 1768101 (0.0008) [2023-12-27 04:16:34,332][105620] Updated weights for policy 1, policy_version 1771995 (0.0007) [2023-12-27 04:16:34,374][105692] Updated weights for policy 0, policy_version 1768111 (0.0005) [2023-12-27 04:16:34,392][105620] Updated weights for policy 1, policy_version 1772005 (0.0008) [2023-12-27 04:16:34,430][105692] Updated weights for policy 0, policy_version 1768121 (0.0007) [2023-12-27 04:16:35,036][105620] Updated weights for policy 1, policy_version 1772015 (0.0007) [2023-12-27 04:16:35,098][105620] Updated weights for policy 1, policy_version 1772025 (0.0009) [2023-12-27 04:16:35,160][105620] Updated weights for policy 1, policy_version 1772035 (0.0009) [2023-12-27 04:16:35,230][105692] Updated weights for policy 0, policy_version 1768131 (0.0009) [2023-12-27 04:16:35,289][105692] Updated weights for policy 0, policy_version 1768141 (0.0009) [2023-12-27 04:16:35,356][105692] Updated weights for policy 0, policy_version 1768151 (0.0008) [2023-12-27 04:16:35,866][105620] Updated weights for policy 1, policy_version 1772045 (0.0010) [2023-12-27 04:16:35,915][105692] Updated weights for policy 0, policy_version 1768161 (0.0005) [2023-12-27 04:16:35,916][105620] Updated weights for policy 1, policy_version 1772055 (0.0009) [2023-12-27 04:16:35,972][105620] Updated weights for policy 1, policy_version 1772065 (0.0009) [2023-12-27 04:16:35,976][105692] Updated weights for policy 0, policy_version 1768171 (0.0005) [2023-12-27 04:16:36,024][105692] Updated weights for policy 0, policy_version 1768181 (0.0005) [2023-12-27 04:16:36,062][104569] Fps is (10 sec: 19660.0, 60 sec: 19251.1, 300 sec: 19494.2). Total num frames: 906428416. Throughput: 0: 9422.7, 1: 9723.3. Samples: 906417268. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:16:36,063][104569] Avg episode reward: [(0, '8627.824'), (1, '9260.549')] [2023-12-27 04:16:36,080][105692] Updated weights for policy 0, policy_version 1768191 (0.0005) [2023-12-27 04:16:36,703][105692] Updated weights for policy 0, policy_version 1768201 (0.0007) [2023-12-27 04:16:36,766][105692] Updated weights for policy 0, policy_version 1768211 (0.0009) [2023-12-27 04:16:36,783][105620] Updated weights for policy 1, policy_version 1772075 (0.0009) [2023-12-27 04:16:36,822][105692] Updated weights for policy 0, policy_version 1768221 (0.0007) [2023-12-27 04:16:36,840][105620] Updated weights for policy 1, policy_version 1772085 (0.0008) [2023-12-27 04:16:36,894][105620] Updated weights for policy 1, policy_version 1772095 (0.0008) [2023-12-27 04:16:37,472][105692] Updated weights for policy 0, policy_version 1768231 (0.0009) [2023-12-27 04:16:37,524][105692] Updated weights for policy 0, policy_version 1768241 (0.0009) [2023-12-27 04:16:37,580][105692] Updated weights for policy 0, policy_version 1768251 (0.0009) [2023-12-27 04:16:37,670][105620] Updated weights for policy 1, policy_version 1772105 (0.0009) [2023-12-27 04:16:37,727][105620] Updated weights for policy 1, policy_version 1772115 (0.0009) [2023-12-27 04:16:37,792][105620] Updated weights for policy 1, policy_version 1772125 (0.0008) [2023-12-27 04:16:37,851][105620] Updated weights for policy 1, policy_version 1772135 (0.0009) [2023-12-27 04:16:38,286][105692] Updated weights for policy 0, policy_version 1768261 (0.0007) [2023-12-27 04:16:38,351][105692] Updated weights for policy 0, policy_version 1768271 (0.0007) [2023-12-27 04:16:38,411][105692] Updated weights for policy 0, policy_version 1768281 (0.0009) [2023-12-27 04:16:38,628][105620] Updated weights for policy 1, policy_version 1772145 (0.0008) [2023-12-27 04:16:38,694][105620] Updated weights for policy 1, policy_version 1772155 (0.0008) [2023-12-27 04:16:38,770][105620] Updated weights for policy 1, policy_version 1772165 (0.0005) [2023-12-27 04:16:39,056][105692] Updated weights for policy 0, policy_version 1768291 (0.0009) [2023-12-27 04:16:39,101][105692] Updated weights for policy 0, policy_version 1768301 (0.0005) [2023-12-27 04:16:39,158][105692] Updated weights for policy 0, policy_version 1768311 (0.0005) [2023-12-27 04:16:39,334][105620] Updated weights for policy 1, policy_version 1772175 (0.0008) [2023-12-27 04:16:39,400][105620] Updated weights for policy 1, policy_version 1772185 (0.0007) [2023-12-27 04:16:39,455][105620] Updated weights for policy 1, policy_version 1772195 (0.0008) [2023-12-27 04:16:39,878][105692] Updated weights for policy 0, policy_version 1768321 (0.0006) [2023-12-27 04:16:39,946][105692] Updated weights for policy 0, policy_version 1768331 (0.0009) [2023-12-27 04:16:40,006][105692] Updated weights for policy 0, policy_version 1768341 (0.0009) [2023-12-27 04:16:40,069][105692] Updated weights for policy 0, policy_version 1768351 (0.0009) [2023-12-27 04:16:40,225][105620] Updated weights for policy 1, policy_version 1772205 (0.0008) [2023-12-27 04:16:40,274][105620] Updated weights for policy 1, policy_version 1772215 (0.0008) [2023-12-27 04:16:40,331][105620] Updated weights for policy 1, policy_version 1772225 (0.0009) [2023-12-27 04:16:40,872][105692] Updated weights for policy 0, policy_version 1768361 (0.0008) [2023-12-27 04:16:40,926][105692] Updated weights for policy 0, policy_version 1768371 (0.0008) [2023-12-27 04:16:40,979][105692] Updated weights for policy 0, policy_version 1768381 (0.0008) [2023-12-27 04:16:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 906526720. Throughput: 0: 9453.6, 1: 9758.1. Samples: 906534764. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:16:41,063][104569] Avg episode reward: [(0, '8533.220'), (1, '9260.729')] [2023-12-27 04:16:41,149][105620] Updated weights for policy 1, policy_version 1772235 (0.0011) [2023-12-27 04:16:41,215][105620] Updated weights for policy 1, policy_version 1772245 (0.0009) [2023-12-27 04:16:41,279][105620] Updated weights for policy 1, policy_version 1772255 (0.0011) [2023-12-27 04:16:41,851][105692] Updated weights for policy 0, policy_version 1768391 (0.0008) [2023-12-27 04:16:41,907][105692] Updated weights for policy 0, policy_version 1768401 (0.0010) [2023-12-27 04:16:41,957][105692] Updated weights for policy 0, policy_version 1768411 (0.0009) [2023-12-27 04:16:41,979][105620] Updated weights for policy 1, policy_version 1772265 (0.0010) [2023-12-27 04:16:42,043][105620] Updated weights for policy 1, policy_version 1772275 (0.0006) [2023-12-27 04:16:42,104][105620] Updated weights for policy 1, policy_version 1772285 (0.0006) [2023-12-27 04:16:42,164][105620] Updated weights for policy 1, policy_version 1772295 (0.0006) [2023-12-27 04:16:42,709][105692] Updated weights for policy 0, policy_version 1768421 (0.0008) [2023-12-27 04:16:42,767][105692] Updated weights for policy 0, policy_version 1768431 (0.0010) [2023-12-27 04:16:42,822][105692] Updated weights for policy 0, policy_version 1768441 (0.0009) [2023-12-27 04:16:42,841][105620] Updated weights for policy 1, policy_version 1772305 (0.0006) [2023-12-27 04:16:42,896][105620] Updated weights for policy 1, policy_version 1772315 (0.0005) [2023-12-27 04:16:42,960][105620] Updated weights for policy 1, policy_version 1772325 (0.0005) [2023-12-27 04:16:43,592][105620] Updated weights for policy 1, policy_version 1772335 (0.0009) [2023-12-27 04:16:43,620][105692] Updated weights for policy 0, policy_version 1768451 (0.0008) [2023-12-27 04:16:43,650][105620] Updated weights for policy 1, policy_version 1772345 (0.0007) [2023-12-27 04:16:43,676][105692] Updated weights for policy 0, policy_version 1768461 (0.0006) [2023-12-27 04:16:43,706][105620] Updated weights for policy 1, policy_version 1772355 (0.0007) [2023-12-27 04:16:43,739][105692] Updated weights for policy 0, policy_version 1768471 (0.0008) [2023-12-27 04:16:44,325][105620] Updated weights for policy 1, policy_version 1772365 (0.0007) [2023-12-27 04:16:44,372][105620] Updated weights for policy 1, policy_version 1772375 (0.0008) [2023-12-27 04:16:44,426][105620] Updated weights for policy 1, policy_version 1772385 (0.0009) [2023-12-27 04:16:44,534][105692] Updated weights for policy 0, policy_version 1768481 (0.0009) [2023-12-27 04:16:44,601][105692] Updated weights for policy 0, policy_version 1768491 (0.0010) [2023-12-27 04:16:44,670][105692] Updated weights for policy 0, policy_version 1768501 (0.0009) [2023-12-27 04:16:44,734][105692] Updated weights for policy 0, policy_version 1768511 (0.0009) [2023-12-27 04:16:45,013][105620] Updated weights for policy 1, policy_version 1772395 (0.0009) [2023-12-27 04:16:45,067][105620] Updated weights for policy 1, policy_version 1772405 (0.0008) [2023-12-27 04:16:45,128][105620] Updated weights for policy 1, policy_version 1772415 (0.0009) [2023-12-27 04:16:45,546][105692] Updated weights for policy 0, policy_version 1768521 (0.0010) [2023-12-27 04:16:45,614][105692] Updated weights for policy 0, policy_version 1768531 (0.0009) [2023-12-27 04:16:45,675][105692] Updated weights for policy 0, policy_version 1768541 (0.0008) [2023-12-27 04:16:45,904][105620] Updated weights for policy 1, policy_version 1772425 (0.0009) [2023-12-27 04:16:45,968][105620] Updated weights for policy 1, policy_version 1772435 (0.0009) [2023-12-27 04:16:46,025][105620] Updated weights for policy 1, policy_version 1772445 (0.0009) [2023-12-27 04:16:46,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 906616832. Throughput: 0: 9385.9, 1: 9770.7. Samples: 906591920. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:16:46,062][104569] Avg episode reward: [(0, '8437.927'), (1, '9260.878')] [2023-12-27 04:16:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001768544_452812800.pth... [2023-12-27 04:16:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001767424_452526080.pth [2023-12-27 04:16:46,084][105620] Updated weights for policy 1, policy_version 1772455 (0.0009) [2023-12-27 04:16:46,087][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001772456_453812224.pth... [2023-12-27 04:16:46,091][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001771272_453509120.pth [2023-12-27 04:16:46,472][105692] Updated weights for policy 0, policy_version 1768551 (0.0006) [2023-12-27 04:16:46,528][105692] Updated weights for policy 0, policy_version 1768561 (0.0005) [2023-12-27 04:16:46,595][105692] Updated weights for policy 0, policy_version 1768571 (0.0005) [2023-12-27 04:16:46,752][105620] Updated weights for policy 1, policy_version 1772465 (0.0006) [2023-12-27 04:16:46,798][105620] Updated weights for policy 1, policy_version 1772475 (0.0005) [2023-12-27 04:16:46,842][105620] Updated weights for policy 1, policy_version 1772485 (0.0005) [2023-12-27 04:16:47,117][105692] Updated weights for policy 0, policy_version 1768581 (0.0007) [2023-12-27 04:16:47,174][105692] Updated weights for policy 0, policy_version 1768591 (0.0009) [2023-12-27 04:16:47,246][105692] Updated weights for policy 0, policy_version 1768601 (0.0008) [2023-12-27 04:16:47,444][105620] Updated weights for policy 1, policy_version 1772495 (0.0008) [2023-12-27 04:16:47,490][105620] Updated weights for policy 1, policy_version 1772505 (0.0006) [2023-12-27 04:16:47,544][105620] Updated weights for policy 1, policy_version 1772515 (0.0005) [2023-12-27 04:16:47,949][105692] Updated weights for policy 0, policy_version 1768611 (0.0008) [2023-12-27 04:16:48,003][105692] Updated weights for policy 0, policy_version 1768621 (0.0006) [2023-12-27 04:16:48,056][105692] Updated weights for policy 0, policy_version 1768631 (0.0005) [2023-12-27 04:16:48,238][105620] Updated weights for policy 1, policy_version 1772525 (0.0007) [2023-12-27 04:16:48,298][105620] Updated weights for policy 1, policy_version 1772535 (0.0008) [2023-12-27 04:16:48,366][105620] Updated weights for policy 1, policy_version 1772545 (0.0009) [2023-12-27 04:16:48,698][105692] Updated weights for policy 0, policy_version 1768641 (0.0005) [2023-12-27 04:16:48,764][105692] Updated weights for policy 0, policy_version 1768651 (0.0006) [2023-12-27 04:16:48,830][105692] Updated weights for policy 0, policy_version 1768661 (0.0006) [2023-12-27 04:16:48,891][105692] Updated weights for policy 0, policy_version 1768671 (0.0009) [2023-12-27 04:16:49,100][105620] Updated weights for policy 1, policy_version 1772555 (0.0007) [2023-12-27 04:16:49,167][105620] Updated weights for policy 1, policy_version 1772565 (0.0008) [2023-12-27 04:16:49,250][105620] Updated weights for policy 1, policy_version 1772575 (0.0009) [2023-12-27 04:16:49,537][105692] Updated weights for policy 0, policy_version 1768681 (0.0006) [2023-12-27 04:16:49,586][105692] Updated weights for policy 0, policy_version 1768691 (0.0005) [2023-12-27 04:16:49,637][105692] Updated weights for policy 0, policy_version 1768701 (0.0006) [2023-12-27 04:16:50,066][105620] Updated weights for policy 1, policy_version 1772585 (0.0009) [2023-12-27 04:16:50,126][105620] Updated weights for policy 1, policy_version 1772595 (0.0008) [2023-12-27 04:16:50,188][105620] Updated weights for policy 1, policy_version 1772605 (0.0010) [2023-12-27 04:16:50,248][105692] Updated weights for policy 0, policy_version 1768711 (0.0006) [2023-12-27 04:16:50,249][105620] Updated weights for policy 1, policy_version 1772615 (0.0010) [2023-12-27 04:16:50,310][105692] Updated weights for policy 0, policy_version 1768721 (0.0006) [2023-12-27 04:16:50,368][105692] Updated weights for policy 0, policy_version 1768731 (0.0005) [2023-12-27 04:16:50,984][105692] Updated weights for policy 0, policy_version 1768741 (0.0007) [2023-12-27 04:16:51,044][105692] Updated weights for policy 0, policy_version 1768751 (0.0009) [2023-12-27 04:16:51,046][105620] Updated weights for policy 1, policy_version 1772625 (0.0007) [2023-12-27 04:16:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 906715136. Throughput: 0: 9464.6, 1: 9863.0. Samples: 906710888. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:16:51,062][104569] Avg episode reward: [(0, '8624.847'), (1, '8986.041')] [2023-12-27 04:16:51,106][105692] Updated weights for policy 0, policy_version 1768761 (0.0009) [2023-12-27 04:16:51,108][105620] Updated weights for policy 1, policy_version 1772635 (0.0008) [2023-12-27 04:16:51,173][105620] Updated weights for policy 1, policy_version 1772645 (0.0008) [2023-12-27 04:16:51,900][105692] Updated weights for policy 0, policy_version 1768771 (0.0010) [2023-12-27 04:16:51,927][105620] Updated weights for policy 1, policy_version 1772655 (0.0009) [2023-12-27 04:16:51,946][105692] Updated weights for policy 0, policy_version 1768781 (0.0005) [2023-12-27 04:16:51,980][105620] Updated weights for policy 1, policy_version 1772665 (0.0008) [2023-12-27 04:16:52,003][105692] Updated weights for policy 0, policy_version 1768791 (0.0007) [2023-12-27 04:16:52,040][105620] Updated weights for policy 1, policy_version 1772675 (0.0010) [2023-12-27 04:16:52,786][105692] Updated weights for policy 0, policy_version 1768801 (0.0009) [2023-12-27 04:16:52,804][105620] Updated weights for policy 1, policy_version 1772685 (0.0008) [2023-12-27 04:16:52,838][105692] Updated weights for policy 0, policy_version 1768811 (0.0008) [2023-12-27 04:16:52,857][105620] Updated weights for policy 1, policy_version 1772695 (0.0007) [2023-12-27 04:16:52,892][105692] Updated weights for policy 0, policy_version 1768821 (0.0007) [2023-12-27 04:16:52,911][105620] Updated weights for policy 1, policy_version 1772705 (0.0007) [2023-12-27 04:16:52,942][105692] Updated weights for policy 0, policy_version 1768831 (0.0007) [2023-12-27 04:16:53,674][105620] Updated weights for policy 1, policy_version 1772715 (0.0008) [2023-12-27 04:16:53,698][105692] Updated weights for policy 0, policy_version 1768841 (0.0008) [2023-12-27 04:16:53,728][105620] Updated weights for policy 1, policy_version 1772725 (0.0009) [2023-12-27 04:16:53,751][105692] Updated weights for policy 0, policy_version 1768851 (0.0007) [2023-12-27 04:16:53,791][105620] Updated weights for policy 1, policy_version 1772735 (0.0008) [2023-12-27 04:16:53,799][105692] Updated weights for policy 0, policy_version 1768861 (0.0006) [2023-12-27 04:16:54,525][105620] Updated weights for policy 1, policy_version 1772745 (0.0010) [2023-12-27 04:16:54,537][105692] Updated weights for policy 0, policy_version 1768871 (0.0005) [2023-12-27 04:16:54,580][105620] Updated weights for policy 1, policy_version 1772755 (0.0009) [2023-12-27 04:16:54,595][105692] Updated weights for policy 0, policy_version 1768881 (0.0007) [2023-12-27 04:16:54,633][105620] Updated weights for policy 1, policy_version 1772765 (0.0008) [2023-12-27 04:16:54,640][105692] Updated weights for policy 0, policy_version 1768891 (0.0006) [2023-12-27 04:16:54,679][105620] Updated weights for policy 1, policy_version 1772775 (0.0008) [2023-12-27 04:16:55,379][105692] Updated weights for policy 0, policy_version 1768901 (0.0007) [2023-12-27 04:16:55,436][105692] Updated weights for policy 0, policy_version 1768911 (0.0009) [2023-12-27 04:16:55,438][105620] Updated weights for policy 1, policy_version 1772785 (0.0006) [2023-12-27 04:16:55,485][105692] Updated weights for policy 0, policy_version 1768921 (0.0007) [2023-12-27 04:16:55,497][105620] Updated weights for policy 1, policy_version 1772795 (0.0008) [2023-12-27 04:16:55,548][105620] Updated weights for policy 1, policy_version 1772805 (0.0008) [2023-12-27 04:16:56,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19114.6, 300 sec: 19494.2). Total num frames: 906813440. Throughput: 0: 9554.4, 1: 9846.2. Samples: 906824032. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:16:56,063][104569] Avg episode reward: [(0, '8723.368'), (1, '8985.827')] [2023-12-27 04:16:56,244][105692] Updated weights for policy 0, policy_version 1768931 (0.0009) [2023-12-27 04:16:56,297][105620] Updated weights for policy 1, policy_version 1772815 (0.0008) [2023-12-27 04:16:56,299][105692] Updated weights for policy 0, policy_version 1768941 (0.0007) [2023-12-27 04:16:56,352][105620] Updated weights for policy 1, policy_version 1772825 (0.0009) [2023-12-27 04:16:56,354][105692] Updated weights for policy 0, policy_version 1768951 (0.0007) [2023-12-27 04:16:56,406][105620] Updated weights for policy 1, policy_version 1772835 (0.0006) [2023-12-27 04:16:57,086][105692] Updated weights for policy 0, policy_version 1768961 (0.0007) [2023-12-27 04:16:57,104][105620] Updated weights for policy 1, policy_version 1772845 (0.0009) [2023-12-27 04:16:57,134][105692] Updated weights for policy 0, policy_version 1768971 (0.0005) [2023-12-27 04:16:57,159][105620] Updated weights for policy 1, policy_version 1772855 (0.0008) [2023-12-27 04:16:57,184][105692] Updated weights for policy 0, policy_version 1768981 (0.0006) [2023-12-27 04:16:57,211][105620] Updated weights for policy 1, policy_version 1772865 (0.0010) [2023-12-27 04:16:57,233][105692] Updated weights for policy 0, policy_version 1768991 (0.0005) [2023-12-27 04:16:57,810][105692] Updated weights for policy 0, policy_version 1769001 (0.0008) [2023-12-27 04:16:57,856][105692] Updated weights for policy 0, policy_version 1769011 (0.0008) [2023-12-27 04:16:57,859][105620] Updated weights for policy 1, policy_version 1772875 (0.0009) [2023-12-27 04:16:57,904][105692] Updated weights for policy 0, policy_version 1769021 (0.0006) [2023-12-27 04:16:57,912][105620] Updated weights for policy 1, policy_version 1772885 (0.0007) [2023-12-27 04:16:57,968][105620] Updated weights for policy 1, policy_version 1772895 (0.0009) [2023-12-27 04:16:58,681][105620] Updated weights for policy 1, policy_version 1772905 (0.0007) [2023-12-27 04:16:58,699][105692] Updated weights for policy 0, policy_version 1769031 (0.0010) [2023-12-27 04:16:58,745][105620] Updated weights for policy 1, policy_version 1772915 (0.0008) [2023-12-27 04:16:58,761][105692] Updated weights for policy 0, policy_version 1769041 (0.0007) [2023-12-27 04:16:58,808][105620] Updated weights for policy 1, policy_version 1772925 (0.0008) [2023-12-27 04:16:58,820][105692] Updated weights for policy 0, policy_version 1769051 (0.0006) [2023-12-27 04:16:58,875][105620] Updated weights for policy 1, policy_version 1772935 (0.0009) [2023-12-27 04:16:59,634][105620] Updated weights for policy 1, policy_version 1772945 (0.0010) [2023-12-27 04:16:59,639][105692] Updated weights for policy 0, policy_version 1769061 (0.0009) [2023-12-27 04:16:59,693][105620] Updated weights for policy 1, policy_version 1772955 (0.0010) [2023-12-27 04:16:59,703][105692] Updated weights for policy 0, policy_version 1769071 (0.0005) [2023-12-27 04:16:59,751][105620] Updated weights for policy 1, policy_version 1772965 (0.0010) [2023-12-27 04:16:59,759][105692] Updated weights for policy 0, policy_version 1769081 (0.0005) [2023-12-27 04:17:00,477][105692] Updated weights for policy 0, policy_version 1769091 (0.0006) [2023-12-27 04:17:00,521][105620] Updated weights for policy 1, policy_version 1772975 (0.0010) [2023-12-27 04:17:00,527][105692] Updated weights for policy 0, policy_version 1769101 (0.0008) [2023-12-27 04:17:00,570][105692] Updated weights for policy 0, policy_version 1769111 (0.0007) [2023-12-27 04:17:00,579][105620] Updated weights for policy 1, policy_version 1772985 (0.0010) [2023-12-27 04:17:00,634][105620] Updated weights for policy 1, policy_version 1772995 (0.0010) [2023-12-27 04:17:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19114.7, 300 sec: 19494.2). Total num frames: 906911744. Throughput: 0: 9598.5, 1: 9884.6. Samples: 906884080. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:17:01,063][104569] Avg episode reward: [(0, '8179.094'), (1, '9076.513')] [2023-12-27 04:17:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001769120_452960256.pth... [2023-12-27 04:17:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001773000_453951488.pth... [2023-12-27 04:17:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001768000_452673536.pth [2023-12-27 04:17:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001771880_453664768.pth [2023-12-27 04:17:01,388][105620] Updated weights for policy 1, policy_version 1773005 (0.0011) [2023-12-27 04:17:01,399][105692] Updated weights for policy 0, policy_version 1769121 (0.0006) [2023-12-27 04:17:01,452][105620] Updated weights for policy 1, policy_version 1773015 (0.0011) [2023-12-27 04:17:01,455][105692] Updated weights for policy 0, policy_version 1769131 (0.0008) [2023-12-27 04:17:01,514][105620] Updated weights for policy 1, policy_version 1773025 (0.0010) [2023-12-27 04:17:01,519][105692] Updated weights for policy 0, policy_version 1769141 (0.0009) [2023-12-27 04:17:01,584][105692] Updated weights for policy 0, policy_version 1769151 (0.0009) [2023-12-27 04:17:02,230][105620] Updated weights for policy 1, policy_version 1773035 (0.0009) [2023-12-27 04:17:02,292][105620] Updated weights for policy 1, policy_version 1773045 (0.0008) [2023-12-27 04:17:02,354][105620] Updated weights for policy 1, policy_version 1773055 (0.0007) [2023-12-27 04:17:02,365][105692] Updated weights for policy 0, policy_version 1769161 (0.0007) [2023-12-27 04:17:02,426][105692] Updated weights for policy 0, policy_version 1769171 (0.0009) [2023-12-27 04:17:02,479][105692] Updated weights for policy 0, policy_version 1769181 (0.0009) [2023-12-27 04:17:02,988][105620] Updated weights for policy 1, policy_version 1773065 (0.0008) [2023-12-27 04:17:03,042][105620] Updated weights for policy 1, policy_version 1773075 (0.0008) [2023-12-27 04:17:03,107][105620] Updated weights for policy 1, policy_version 1773085 (0.0008) [2023-12-27 04:17:03,162][105620] Updated weights for policy 1, policy_version 1773095 (0.0008) [2023-12-27 04:17:03,238][105692] Updated weights for policy 0, policy_version 1769191 (0.0010) [2023-12-27 04:17:03,293][105692] Updated weights for policy 0, policy_version 1769201 (0.0005) [2023-12-27 04:17:03,349][105692] Updated weights for policy 0, policy_version 1769211 (0.0006) [2023-12-27 04:17:03,953][105692] Updated weights for policy 0, policy_version 1769221 (0.0007) [2023-12-27 04:17:03,986][105620] Updated weights for policy 1, policy_version 1773105 (0.0009) [2023-12-27 04:17:04,001][105692] Updated weights for policy 0, policy_version 1769231 (0.0006) [2023-12-27 04:17:04,046][105620] Updated weights for policy 1, policy_version 1773115 (0.0011) [2023-12-27 04:17:04,058][105692] Updated weights for policy 0, policy_version 1769241 (0.0011) [2023-12-27 04:17:04,116][105620] Updated weights for policy 1, policy_version 1773125 (0.0011) [2023-12-27 04:17:04,709][105620] Updated weights for policy 1, policy_version 1773135 (0.0010) [2023-12-27 04:17:04,763][105620] Updated weights for policy 1, policy_version 1773145 (0.0010) [2023-12-27 04:17:04,815][105692] Updated weights for policy 0, policy_version 1769251 (0.0006) [2023-12-27 04:17:04,823][105620] Updated weights for policy 1, policy_version 1773155 (0.0009) [2023-12-27 04:17:04,873][105692] Updated weights for policy 0, policy_version 1769261 (0.0005) [2023-12-27 04:17:04,933][105692] Updated weights for policy 0, policy_version 1769271 (0.0005) [2023-12-27 04:17:05,470][105692] Updated weights for policy 0, policy_version 1769281 (0.0005) [2023-12-27 04:17:05,532][105692] Updated weights for policy 0, policy_version 1769291 (0.0008) [2023-12-27 04:17:05,590][105692] Updated weights for policy 0, policy_version 1769301 (0.0009) [2023-12-27 04:17:05,619][105620] Updated weights for policy 1, policy_version 1773165 (0.0007) [2023-12-27 04:17:05,644][105692] Updated weights for policy 0, policy_version 1769311 (0.0008) [2023-12-27 04:17:05,669][105620] Updated weights for policy 1, policy_version 1773175 (0.0007) [2023-12-27 04:17:05,721][105620] Updated weights for policy 1, policy_version 1773185 (0.0005) [2023-12-27 04:17:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 907010048. Throughput: 0: 9649.8, 1: 9823.9. Samples: 906996580. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:17:06,063][104569] Avg episode reward: [(0, '8262.231'), (1, '9168.767')] [2023-12-27 04:17:06,331][105620] Updated weights for policy 1, policy_version 1773195 (0.0005) [2023-12-27 04:17:06,390][105620] Updated weights for policy 1, policy_version 1773205 (0.0006) [2023-12-27 04:17:06,445][105692] Updated weights for policy 0, policy_version 1769321 (0.0008) [2023-12-27 04:17:06,447][105620] Updated weights for policy 1, policy_version 1773215 (0.0006) [2023-12-27 04:17:06,514][105692] Updated weights for policy 0, policy_version 1769331 (0.0007) [2023-12-27 04:17:06,579][105692] Updated weights for policy 0, policy_version 1769341 (0.0005) [2023-12-27 04:17:07,134][105620] Updated weights for policy 1, policy_version 1773225 (0.0006) [2023-12-27 04:17:07,192][105620] Updated weights for policy 1, policy_version 1773235 (0.0009) [2023-12-27 04:17:07,244][105620] Updated weights for policy 1, policy_version 1773245 (0.0009) [2023-12-27 04:17:07,250][105692] Updated weights for policy 0, policy_version 1769351 (0.0008) [2023-12-27 04:17:07,303][105620] Updated weights for policy 1, policy_version 1773255 (0.0008) [2023-12-27 04:17:07,307][105692] Updated weights for policy 0, policy_version 1769361 (0.0006) [2023-12-27 04:17:07,362][105692] Updated weights for policy 0, policy_version 1769371 (0.0007) [2023-12-27 04:17:07,920][105620] Updated weights for policy 1, policy_version 1773265 (0.0006) [2023-12-27 04:17:07,977][105620] Updated weights for policy 1, policy_version 1773275 (0.0008) [2023-12-27 04:17:08,037][105620] Updated weights for policy 1, policy_version 1773285 (0.0008) [2023-12-27 04:17:08,198][105692] Updated weights for policy 0, policy_version 1769381 (0.0010) [2023-12-27 04:17:08,251][105692] Updated weights for policy 0, policy_version 1769391 (0.0010) [2023-12-27 04:17:08,299][105692] Updated weights for policy 0, policy_version 1769401 (0.0009) [2023-12-27 04:17:08,698][105620] Updated weights for policy 1, policy_version 1773295 (0.0008) [2023-12-27 04:17:08,752][105620] Updated weights for policy 1, policy_version 1773305 (0.0009) [2023-12-27 04:17:08,813][105620] Updated weights for policy 1, policy_version 1773315 (0.0009) [2023-12-27 04:17:09,040][105692] Updated weights for policy 0, policy_version 1769411 (0.0009) [2023-12-27 04:17:09,087][105692] Updated weights for policy 0, policy_version 1769421 (0.0009) [2023-12-27 04:17:09,135][105692] Updated weights for policy 0, policy_version 1769431 (0.0009) [2023-12-27 04:17:09,582][105620] Updated weights for policy 1, policy_version 1773325 (0.0008) [2023-12-27 04:17:09,644][105620] Updated weights for policy 1, policy_version 1773335 (0.0009) [2023-12-27 04:17:09,703][105620] Updated weights for policy 1, policy_version 1773345 (0.0007) [2023-12-27 04:17:10,007][105692] Updated weights for policy 0, policy_version 1769441 (0.0009) [2023-12-27 04:17:10,070][105692] Updated weights for policy 0, policy_version 1769451 (0.0009) [2023-12-27 04:17:10,129][105692] Updated weights for policy 0, policy_version 1769461 (0.0009) [2023-12-27 04:17:10,185][105692] Updated weights for policy 0, policy_version 1769471 (0.0009) [2023-12-27 04:17:10,467][105620] Updated weights for policy 1, policy_version 1773355 (0.0009) [2023-12-27 04:17:10,529][105620] Updated weights for policy 1, policy_version 1773365 (0.0010) [2023-12-27 04:17:10,585][105620] Updated weights for policy 1, policy_version 1773375 (0.0008) [2023-12-27 04:17:10,916][105692] Updated weights for policy 0, policy_version 1769481 (0.0010) [2023-12-27 04:17:10,970][105692] Updated weights for policy 0, policy_version 1769491 (0.0010) [2023-12-27 04:17:11,019][105692] Updated weights for policy 0, policy_version 1769501 (0.0010) [2023-12-27 04:17:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 907108352. Throughput: 0: 9658.3, 1: 9799.6. Samples: 907113648. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:17:11,062][104569] Avg episode reward: [(0, '8625.115'), (1, '9260.164')] [2023-12-27 04:17:11,389][105620] Updated weights for policy 1, policy_version 1773385 (0.0008) [2023-12-27 04:17:11,459][105620] Updated weights for policy 1, policy_version 1773396 (0.0010) [2023-12-27 04:17:11,521][105620] Updated weights for policy 1, policy_version 1773406 (0.0010) [2023-12-27 04:17:11,590][105620] Updated weights for policy 1, policy_version 1773416 (0.0008) [2023-12-27 04:17:11,856][105692] Updated weights for policy 0, policy_version 1769511 (0.0007) [2023-12-27 04:17:11,919][105692] Updated weights for policy 0, policy_version 1769521 (0.0005) [2023-12-27 04:17:11,976][105692] Updated weights for policy 0, policy_version 1769531 (0.0010) [2023-12-27 04:17:12,310][105620] Updated weights for policy 1, policy_version 1773426 (0.0008) [2023-12-27 04:17:12,371][105620] Updated weights for policy 1, policy_version 1773436 (0.0009) [2023-12-27 04:17:12,435][105620] Updated weights for policy 1, policy_version 1773446 (0.0006) [2023-12-27 04:17:12,725][105692] Updated weights for policy 0, policy_version 1769541 (0.0010) [2023-12-27 04:17:12,792][105692] Updated weights for policy 0, policy_version 1769551 (0.0011) [2023-12-27 04:17:12,858][105692] Updated weights for policy 0, policy_version 1769561 (0.0011) [2023-12-27 04:17:13,079][105620] Updated weights for policy 1, policy_version 1773456 (0.0008) [2023-12-27 04:17:13,138][105620] Updated weights for policy 1, policy_version 1773466 (0.0009) [2023-12-27 04:17:13,191][105620] Updated weights for policy 1, policy_version 1773476 (0.0010) [2023-12-27 04:17:13,441][105692] Updated weights for policy 0, policy_version 1769571 (0.0009) [2023-12-27 04:17:13,494][105692] Updated weights for policy 0, policy_version 1769581 (0.0005) [2023-12-27 04:17:13,551][105692] Updated weights for policy 0, policy_version 1769591 (0.0005) [2023-12-27 04:17:13,956][105620] Updated weights for policy 1, policy_version 1773486 (0.0008) [2023-12-27 04:17:14,005][105620] Updated weights for policy 1, policy_version 1773496 (0.0008) [2023-12-27 04:17:14,056][105620] Updated weights for policy 1, policy_version 1773506 (0.0008) [2023-12-27 04:17:14,254][105692] Updated weights for policy 0, policy_version 1769601 (0.0008) [2023-12-27 04:17:14,308][105692] Updated weights for policy 0, policy_version 1769611 (0.0010) [2023-12-27 04:17:14,365][105692] Updated weights for policy 0, policy_version 1769621 (0.0010) [2023-12-27 04:17:14,427][105692] Updated weights for policy 0, policy_version 1769631 (0.0007) [2023-12-27 04:17:14,808][105620] Updated weights for policy 1, policy_version 1773516 (0.0008) [2023-12-27 04:17:14,876][105620] Updated weights for policy 1, policy_version 1773526 (0.0006) [2023-12-27 04:17:14,942][105620] Updated weights for policy 1, policy_version 1773536 (0.0007) [2023-12-27 04:17:15,063][105692] Updated weights for policy 0, policy_version 1769641 (0.0011) [2023-12-27 04:17:15,122][105692] Updated weights for policy 0, policy_version 1769651 (0.0011) [2023-12-27 04:17:15,189][105692] Updated weights for policy 0, policy_version 1769661 (0.0011) [2023-12-27 04:17:15,513][105620] Updated weights for policy 1, policy_version 1773546 (0.0006) [2023-12-27 04:17:15,574][105620] Updated weights for policy 1, policy_version 1773556 (0.0008) [2023-12-27 04:17:15,633][105620] Updated weights for policy 1, policy_version 1773566 (0.0008) [2023-12-27 04:17:15,696][105620] Updated weights for policy 1, policy_version 1773576 (0.0009) [2023-12-27 04:17:15,939][105692] Updated weights for policy 0, policy_version 1769671 (0.0010) [2023-12-27 04:17:15,987][105692] Updated weights for policy 0, policy_version 1769681 (0.0010) [2023-12-27 04:17:16,041][105692] Updated weights for policy 0, policy_version 1769691 (0.0010) [2023-12-27 04:17:16,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19387.6, 300 sec: 19494.2). Total num frames: 907198464. Throughput: 0: 9620.0, 1: 9734.1. Samples: 907171708. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 04:17:16,064][104569] Avg episode reward: [(0, '8901.486'), (1, '8982.602')] [2023-12-27 04:17:16,074][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001769696_453107712.pth... [2023-12-27 04:17:16,074][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001773576_454098944.pth... [2023-12-27 04:17:16,080][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001768544_452812800.pth [2023-12-27 04:17:16,083][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001772456_453812224.pth [2023-12-27 04:17:16,404][105620] Updated weights for policy 1, policy_version 1773586 (0.0008) [2023-12-27 04:17:16,465][105620] Updated weights for policy 1, policy_version 1773596 (0.0009) [2023-12-27 04:17:16,521][105620] Updated weights for policy 1, policy_version 1773606 (0.0009) [2023-12-27 04:17:16,751][105692] Updated weights for policy 0, policy_version 1769701 (0.0008) [2023-12-27 04:17:16,809][105692] Updated weights for policy 0, policy_version 1769711 (0.0005) [2023-12-27 04:17:16,864][105692] Updated weights for policy 0, policy_version 1769721 (0.0006) [2023-12-27 04:17:17,295][105620] Updated weights for policy 1, policy_version 1773616 (0.0006) [2023-12-27 04:17:17,365][105620] Updated weights for policy 1, policy_version 1773626 (0.0009) [2023-12-27 04:17:17,407][105692] Updated weights for policy 0, policy_version 1769731 (0.0006) [2023-12-27 04:17:17,433][105620] Updated weights for policy 1, policy_version 1773636 (0.0009) [2023-12-27 04:17:17,473][105692] Updated weights for policy 0, policy_version 1769741 (0.0006) [2023-12-27 04:17:17,531][105692] Updated weights for policy 0, policy_version 1769751 (0.0006) [2023-12-27 04:17:18,104][105620] Updated weights for policy 1, policy_version 1773646 (0.0007) [2023-12-27 04:17:18,154][105692] Updated weights for policy 0, policy_version 1769761 (0.0006) [2023-12-27 04:17:18,162][105620] Updated weights for policy 1, policy_version 1773656 (0.0005) [2023-12-27 04:17:18,208][105692] Updated weights for policy 0, policy_version 1769771 (0.0006) [2023-12-27 04:17:18,215][105620] Updated weights for policy 1, policy_version 1773666 (0.0010) [2023-12-27 04:17:18,259][105692] Updated weights for policy 0, policy_version 1769781 (0.0005) [2023-12-27 04:17:18,303][105692] Updated weights for policy 0, policy_version 1769791 (0.0007) [2023-12-27 04:17:18,824][105620] Updated weights for policy 1, policy_version 1773676 (0.0010) [2023-12-27 04:17:18,893][105620] Updated weights for policy 1, policy_version 1773686 (0.0011) [2023-12-27 04:17:18,952][105620] Updated weights for policy 1, policy_version 1773696 (0.0010) [2023-12-27 04:17:19,070][105692] Updated weights for policy 0, policy_version 1769801 (0.0009) [2023-12-27 04:17:19,130][105692] Updated weights for policy 0, policy_version 1769811 (0.0008) [2023-12-27 04:17:19,193][105692] Updated weights for policy 0, policy_version 1769821 (0.0008) [2023-12-27 04:17:19,701][105620] Updated weights for policy 1, policy_version 1773706 (0.0010) [2023-12-27 04:17:19,767][105620] Updated weights for policy 1, policy_version 1773716 (0.0009) [2023-12-27 04:17:19,837][105620] Updated weights for policy 1, policy_version 1773726 (0.0011) [2023-12-27 04:17:19,895][105692] Updated weights for policy 0, policy_version 1769831 (0.0010) [2023-12-27 04:17:19,902][105620] Updated weights for policy 1, policy_version 1773736 (0.0011) [2023-12-27 04:17:19,958][105692] Updated weights for policy 0, policy_version 1769841 (0.0007) [2023-12-27 04:17:20,015][105692] Updated weights for policy 0, policy_version 1769851 (0.0007) [2023-12-27 04:17:20,592][105692] Updated weights for policy 0, policy_version 1769861 (0.0007) [2023-12-27 04:17:20,657][105692] Updated weights for policy 0, policy_version 1769871 (0.0006) [2023-12-27 04:17:20,701][105620] Updated weights for policy 1, policy_version 1773746 (0.0009) [2023-12-27 04:17:20,725][105692] Updated weights for policy 0, policy_version 1769881 (0.0006) [2023-12-27 04:17:20,764][105620] Updated weights for policy 1, policy_version 1773756 (0.0007) [2023-12-27 04:17:20,824][105620] Updated weights for policy 1, policy_version 1773766 (0.0009) [2023-12-27 04:17:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 907304960. Throughput: 0: 9709.8, 1: 9722.3. Samples: 907291704. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:17:21,062][104569] Avg episode reward: [(0, '8719.528'), (1, '8983.291')] [2023-12-27 04:17:21,435][105692] Updated weights for policy 0, policy_version 1769891 (0.0007) [2023-12-27 04:17:21,498][105692] Updated weights for policy 0, policy_version 1769901 (0.0009) [2023-12-27 04:17:21,559][105692] Updated weights for policy 0, policy_version 1769911 (0.0008) [2023-12-27 04:17:21,590][105620] Updated weights for policy 1, policy_version 1773776 (0.0008) [2023-12-27 04:17:21,658][105620] Updated weights for policy 1, policy_version 1773786 (0.0007) [2023-12-27 04:17:21,727][105620] Updated weights for policy 1, policy_version 1773796 (0.0009) [2023-12-27 04:17:22,311][105692] Updated weights for policy 0, policy_version 1769921 (0.0009) [2023-12-27 04:17:22,386][105692] Updated weights for policy 0, policy_version 1769931 (0.0008) [2023-12-27 04:17:22,451][105620] Updated weights for policy 1, policy_version 1773806 (0.0007) [2023-12-27 04:17:22,457][105692] Updated weights for policy 0, policy_version 1769941 (0.0008) [2023-12-27 04:17:22,504][105620] Updated weights for policy 1, policy_version 1773816 (0.0006) [2023-12-27 04:17:22,506][105692] Updated weights for policy 0, policy_version 1769951 (0.0008) [2023-12-27 04:17:22,558][105620] Updated weights for policy 1, policy_version 1773826 (0.0007) [2023-12-27 04:17:23,167][105620] Updated weights for policy 1, policy_version 1773836 (0.0008) [2023-12-27 04:17:23,217][105620] Updated weights for policy 1, policy_version 1773846 (0.0009) [2023-12-27 04:17:23,276][105620] Updated weights for policy 1, policy_version 1773856 (0.0005) [2023-12-27 04:17:23,334][105692] Updated weights for policy 0, policy_version 1769961 (0.0008) [2023-12-27 04:17:23,384][105692] Updated weights for policy 0, policy_version 1769971 (0.0008) [2023-12-27 04:17:23,443][105692] Updated weights for policy 0, policy_version 1769981 (0.0008) [2023-12-27 04:17:23,954][105620] Updated weights for policy 1, policy_version 1773866 (0.0006) [2023-12-27 04:17:24,020][105620] Updated weights for policy 1, policy_version 1773876 (0.0010) [2023-12-27 04:17:24,092][105620] Updated weights for policy 1, policy_version 1773886 (0.0010) [2023-12-27 04:17:24,139][105692] Updated weights for policy 0, policy_version 1769991 (0.0007) [2023-12-27 04:17:24,149][105620] Updated weights for policy 1, policy_version 1773896 (0.0008) [2023-12-27 04:17:24,188][105692] Updated weights for policy 0, policy_version 1770001 (0.0008) [2023-12-27 04:17:24,247][105692] Updated weights for policy 0, policy_version 1770011 (0.0009) [2023-12-27 04:17:24,733][105620] Updated weights for policy 1, policy_version 1773906 (0.0005) [2023-12-27 04:17:24,784][105620] Updated weights for policy 1, policy_version 1773916 (0.0009) [2023-12-27 04:17:24,831][105620] Updated weights for policy 1, policy_version 1773926 (0.0009) [2023-12-27 04:17:25,077][105692] Updated weights for policy 0, policy_version 1770021 (0.0007) [2023-12-27 04:17:25,129][105692] Updated weights for policy 0, policy_version 1770031 (0.0005) [2023-12-27 04:17:25,190][105692] Updated weights for policy 0, policy_version 1770041 (0.0005) [2023-12-27 04:17:25,700][105620] Updated weights for policy 1, policy_version 1773936 (0.0009) [2023-12-27 04:17:25,728][105692] Updated weights for policy 0, policy_version 1770051 (0.0006) [2023-12-27 04:17:25,753][105620] Updated weights for policy 1, policy_version 1773946 (0.0008) [2023-12-27 04:17:25,776][105692] Updated weights for policy 0, policy_version 1770061 (0.0005) [2023-12-27 04:17:25,812][105620] Updated weights for policy 1, policy_version 1773956 (0.0007) [2023-12-27 04:17:25,823][105692] Updated weights for policy 0, policy_version 1770071 (0.0007) [2023-12-27 04:17:26,062][104569] Fps is (10 sec: 20481.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 907403264. Throughput: 0: 9669.8, 1: 9757.9. Samples: 907409008. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:17:26,062][104569] Avg episode reward: [(0, '8349.723'), (1, '8983.596')] [2023-12-27 04:17:26,506][105620] Updated weights for policy 1, policy_version 1773966 (0.0006) [2023-12-27 04:17:26,554][105620] Updated weights for policy 1, policy_version 1773976 (0.0005) [2023-12-27 04:17:26,599][105620] Updated weights for policy 1, policy_version 1773986 (0.0005) [2023-12-27 04:17:26,610][105692] Updated weights for policy 0, policy_version 1770081 (0.0008) [2023-12-27 04:17:26,611][105586] KL-divergence is very high: 107.4981 [2023-12-27 04:17:26,668][105692] Updated weights for policy 0, policy_version 1770091 (0.0007) [2023-12-27 04:17:26,714][105692] Updated weights for policy 0, policy_version 1770101 (0.0009) [2023-12-27 04:17:26,765][105692] Updated weights for policy 0, policy_version 1770111 (0.0009) [2023-12-27 04:17:27,304][105620] Updated weights for policy 1, policy_version 1773996 (0.0008) [2023-12-27 04:17:27,356][105620] Updated weights for policy 1, policy_version 1774006 (0.0009) [2023-12-27 04:17:27,405][105620] Updated weights for policy 1, policy_version 1774016 (0.0008) [2023-12-27 04:17:27,496][105692] Updated weights for policy 0, policy_version 1770121 (0.0008) [2023-12-27 04:17:27,543][105692] Updated weights for policy 0, policy_version 1770131 (0.0009) [2023-12-27 04:17:27,593][105692] Updated weights for policy 0, policy_version 1770141 (0.0009) [2023-12-27 04:17:28,110][105620] Updated weights for policy 1, policy_version 1774026 (0.0009) [2023-12-27 04:17:28,166][105620] Updated weights for policy 1, policy_version 1774036 (0.0009) [2023-12-27 04:17:28,216][105620] Updated weights for policy 1, policy_version 1774046 (0.0009) [2023-12-27 04:17:28,269][105620] Updated weights for policy 1, policy_version 1774056 (0.0008) [2023-12-27 04:17:28,324][105692] Updated weights for policy 0, policy_version 1770151 (0.0009) [2023-12-27 04:17:28,387][105692] Updated weights for policy 0, policy_version 1770162 (0.0010) [2023-12-27 04:17:28,439][105692] Updated weights for policy 0, policy_version 1770172 (0.0009) [2023-12-27 04:17:29,041][105620] Updated weights for policy 1, policy_version 1774066 (0.0007) [2023-12-27 04:17:29,042][105692] Updated weights for policy 0, policy_version 1770182 (0.0007) [2023-12-27 04:17:29,090][105692] Updated weights for policy 0, policy_version 1770192 (0.0005) [2023-12-27 04:17:29,100][105620] Updated weights for policy 1, policy_version 1774076 (0.0006) [2023-12-27 04:17:29,145][105692] Updated weights for policy 0, policy_version 1770202 (0.0010) [2023-12-27 04:17:29,154][105620] Updated weights for policy 1, policy_version 1774086 (0.0005) [2023-12-27 04:17:29,802][105620] Updated weights for policy 1, policy_version 1774096 (0.0005) [2023-12-27 04:17:29,859][105692] Updated weights for policy 0, policy_version 1770212 (0.0010) [2023-12-27 04:17:29,863][105620] Updated weights for policy 1, policy_version 1774106 (0.0009) [2023-12-27 04:17:29,916][105692] Updated weights for policy 0, policy_version 1770222 (0.0011) [2023-12-27 04:17:29,923][105620] Updated weights for policy 1, policy_version 1774116 (0.0006) [2023-12-27 04:17:29,972][105692] Updated weights for policy 0, policy_version 1770232 (0.0007) [2023-12-27 04:17:30,498][105620] Updated weights for policy 1, policy_version 1774126 (0.0007) [2023-12-27 04:17:30,556][105620] Updated weights for policy 1, policy_version 1774136 (0.0005) [2023-12-27 04:17:30,620][105620] Updated weights for policy 1, policy_version 1774146 (0.0006) [2023-12-27 04:17:30,786][105692] Updated weights for policy 0, policy_version 1770242 (0.0009) [2023-12-27 04:17:30,838][105692] Updated weights for policy 0, policy_version 1770252 (0.0010) [2023-12-27 04:17:30,895][105692] Updated weights for policy 0, policy_version 1770263 (0.0009) [2023-12-27 04:17:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 907501568. Throughput: 0: 9704.0, 1: 9733.0. Samples: 907466584. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:17:31,062][104569] Avg episode reward: [(0, '8163.858'), (1, '8890.700')] [2023-12-27 04:17:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001770272_453255168.pth... [2023-12-27 04:17:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001774152_454246400.pth... [2023-12-27 04:17:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001769120_452960256.pth [2023-12-27 04:17:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001773000_453951488.pth [2023-12-27 04:17:31,184][105620] Updated weights for policy 1, policy_version 1774156 (0.0007) [2023-12-27 04:17:31,250][105620] Updated weights for policy 1, policy_version 1774166 (0.0010) [2023-12-27 04:17:31,305][105620] Updated weights for policy 1, policy_version 1774176 (0.0010) [2023-12-27 04:17:31,716][105692] Updated weights for policy 0, policy_version 1770273 (0.0011) [2023-12-27 04:17:31,768][105692] Updated weights for policy 0, policy_version 1770283 (0.0008) [2023-12-27 04:17:31,814][105692] Updated weights for policy 0, policy_version 1770293 (0.0006) [2023-12-27 04:17:31,874][105692] Updated weights for policy 0, policy_version 1770303 (0.0008) [2023-12-27 04:17:32,031][105620] Updated weights for policy 1, policy_version 1774186 (0.0009) [2023-12-27 04:17:32,100][105620] Updated weights for policy 1, policy_version 1774196 (0.0011) [2023-12-27 04:17:32,154][105620] Updated weights for policy 1, policy_version 1774206 (0.0010) [2023-12-27 04:17:32,216][105620] Updated weights for policy 1, policy_version 1774216 (0.0010) [2023-12-27 04:17:32,555][105692] Updated weights for policy 0, policy_version 1770313 (0.0010) [2023-12-27 04:17:32,610][105692] Updated weights for policy 0, policy_version 1770323 (0.0010) [2023-12-27 04:17:32,671][105692] Updated weights for policy 0, policy_version 1770333 (0.0010) [2023-12-27 04:17:32,841][105620] Updated weights for policy 1, policy_version 1774226 (0.0011) [2023-12-27 04:17:32,893][105620] Updated weights for policy 1, policy_version 1774236 (0.0008) [2023-12-27 04:17:32,941][105620] Updated weights for policy 1, policy_version 1774246 (0.0005) [2023-12-27 04:17:33,470][105692] Updated weights for policy 0, policy_version 1770343 (0.0007) [2023-12-27 04:17:33,514][105692] Updated weights for policy 0, policy_version 1770353 (0.0005) [2023-12-27 04:17:33,560][105692] Updated weights for policy 0, policy_version 1770363 (0.0005) [2023-12-27 04:17:33,590][105620] Updated weights for policy 1, policy_version 1774256 (0.0009) [2023-12-27 04:17:33,634][105620] Updated weights for policy 1, policy_version 1774266 (0.0010) [2023-12-27 04:17:33,692][105620] Updated weights for policy 1, policy_version 1774276 (0.0009) [2023-12-27 04:17:34,166][105692] Updated weights for policy 0, policy_version 1770373 (0.0007) [2023-12-27 04:17:34,228][105692] Updated weights for policy 0, policy_version 1770383 (0.0008) [2023-12-27 04:17:34,279][105692] Updated weights for policy 0, policy_version 1770393 (0.0006) [2023-12-27 04:17:34,438][105620] Updated weights for policy 1, policy_version 1774286 (0.0009) [2023-12-27 04:17:34,486][105620] Updated weights for policy 1, policy_version 1774296 (0.0011) [2023-12-27 04:17:34,538][105620] Updated weights for policy 1, policy_version 1774306 (0.0011) [2023-12-27 04:17:34,961][105692] Updated weights for policy 0, policy_version 1770403 (0.0009) [2023-12-27 04:17:35,020][105692] Updated weights for policy 0, policy_version 1770413 (0.0011) [2023-12-27 04:17:35,073][105692] Updated weights for policy 0, policy_version 1770423 (0.0011) [2023-12-27 04:17:35,316][105620] Updated weights for policy 1, policy_version 1774316 (0.0010) [2023-12-27 04:17:35,385][105620] Updated weights for policy 1, policy_version 1774326 (0.0010) [2023-12-27 04:17:35,429][105620] Updated weights for policy 1, policy_version 1774336 (0.0010) [2023-12-27 04:17:35,776][105692] Updated weights for policy 0, policy_version 1770433 (0.0010) [2023-12-27 04:17:35,843][105692] Updated weights for policy 0, policy_version 1770443 (0.0006) [2023-12-27 04:17:35,905][105692] Updated weights for policy 0, policy_version 1770453 (0.0005) [2023-12-27 04:17:35,962][105692] Updated weights for policy 0, policy_version 1770463 (0.0005) [2023-12-27 04:17:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.4, 300 sec: 19549.7). Total num frames: 907599872. Throughput: 0: 9713.9, 1: 9783.5. Samples: 907588268. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:17:36,062][104569] Avg episode reward: [(0, '8446.198'), (1, '9167.671')] [2023-12-27 04:17:36,131][105620] Updated weights for policy 1, policy_version 1774346 (0.0010) [2023-12-27 04:17:36,193][105620] Updated weights for policy 1, policy_version 1774356 (0.0006) [2023-12-27 04:17:36,256][105620] Updated weights for policy 1, policy_version 1774366 (0.0007) [2023-12-27 04:17:36,322][105620] Updated weights for policy 1, policy_version 1774376 (0.0011) [2023-12-27 04:17:36,656][105692] Updated weights for policy 0, policy_version 1770473 (0.0009) [2023-12-27 04:17:36,715][105692] Updated weights for policy 0, policy_version 1770483 (0.0008) [2023-12-27 04:17:36,771][105692] Updated weights for policy 0, policy_version 1770493 (0.0010) [2023-12-27 04:17:36,966][105620] Updated weights for policy 1, policy_version 1774386 (0.0010) [2023-12-27 04:17:37,014][105620] Updated weights for policy 1, policy_version 1774396 (0.0011) [2023-12-27 04:17:37,069][105620] Updated weights for policy 1, policy_version 1774406 (0.0010) [2023-12-27 04:17:37,559][105692] Updated weights for policy 0, policy_version 1770503 (0.0007) [2023-12-27 04:17:37,627][105692] Updated weights for policy 0, policy_version 1770513 (0.0006) [2023-12-27 04:17:37,677][105692] Updated weights for policy 0, policy_version 1770523 (0.0007) [2023-12-27 04:17:37,826][105620] Updated weights for policy 1, policy_version 1774416 (0.0011) [2023-12-27 04:17:37,885][105620] Updated weights for policy 1, policy_version 1774426 (0.0010) [2023-12-27 04:17:37,941][105620] Updated weights for policy 1, policy_version 1774436 (0.0010) [2023-12-27 04:17:38,322][105692] Updated weights for policy 0, policy_version 1770533 (0.0009) [2023-12-27 04:17:38,378][105692] Updated weights for policy 0, policy_version 1770543 (0.0011) [2023-12-27 04:17:38,445][105692] Updated weights for policy 0, policy_version 1770553 (0.0011) [2023-12-27 04:17:38,567][105620] Updated weights for policy 1, policy_version 1774447 (0.0007) [2023-12-27 04:17:38,628][105620] Updated weights for policy 1, policy_version 1774457 (0.0005) [2023-12-27 04:17:38,680][105620] Updated weights for policy 1, policy_version 1774467 (0.0005) [2023-12-27 04:17:39,212][105692] Updated weights for policy 0, policy_version 1770563 (0.0013) [2023-12-27 04:17:39,274][105692] Updated weights for policy 0, policy_version 1770573 (0.0010) [2023-12-27 04:17:39,342][105692] Updated weights for policy 0, policy_version 1770583 (0.0011) [2023-12-27 04:17:39,386][105620] Updated weights for policy 1, policy_version 1774477 (0.0008) [2023-12-27 04:17:39,451][105620] Updated weights for policy 1, policy_version 1774487 (0.0011) [2023-12-27 04:17:39,517][105620] Updated weights for policy 1, policy_version 1774497 (0.0010) [2023-12-27 04:17:40,077][105692] Updated weights for policy 0, policy_version 1770593 (0.0010) [2023-12-27 04:17:40,127][105692] Updated weights for policy 0, policy_version 1770603 (0.0006) [2023-12-27 04:17:40,183][105692] Updated weights for policy 0, policy_version 1770613 (0.0011) [2023-12-27 04:17:40,232][105692] Updated weights for policy 0, policy_version 1770623 (0.0011) [2023-12-27 04:17:40,277][105620] Updated weights for policy 1, policy_version 1774507 (0.0010) [2023-12-27 04:17:40,332][105620] Updated weights for policy 1, policy_version 1774517 (0.0010) [2023-12-27 04:17:40,391][105620] Updated weights for policy 1, policy_version 1774527 (0.0010) [2023-12-27 04:17:40,935][105692] Updated weights for policy 0, policy_version 1770633 (0.0006) [2023-12-27 04:17:40,971][105620] Updated weights for policy 1, policy_version 1774537 (0.0008) [2023-12-27 04:17:40,990][105692] Updated weights for policy 0, policy_version 1770643 (0.0005) [2023-12-27 04:17:41,016][105620] Updated weights for policy 1, policy_version 1774547 (0.0010) [2023-12-27 04:17:41,048][105692] Updated weights for policy 0, policy_version 1770653 (0.0007) [2023-12-27 04:17:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 907698176. Throughput: 0: 9716.9, 1: 9910.8. Samples: 907707272. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:17:41,062][104569] Avg episode reward: [(0, '8535.874'), (1, '9352.079')] [2023-12-27 04:17:41,081][105620] Updated weights for policy 1, policy_version 1774557 (0.0009) [2023-12-27 04:17:41,148][105620] Updated weights for policy 1, policy_version 1774567 (0.0010) [2023-12-27 04:17:41,724][105692] Updated weights for policy 0, policy_version 1770663 (0.0007) [2023-12-27 04:17:41,785][105692] Updated weights for policy 0, policy_version 1770673 (0.0011) [2023-12-27 04:17:41,841][105692] Updated weights for policy 0, policy_version 1770683 (0.0011) [2023-12-27 04:17:42,021][105620] Updated weights for policy 1, policy_version 1774577 (0.0010) [2023-12-27 04:17:42,091][105620] Updated weights for policy 1, policy_version 1774587 (0.0011) [2023-12-27 04:17:42,162][105620] Updated weights for policy 1, policy_version 1774597 (0.0011) [2023-12-27 04:17:42,597][105692] Updated weights for policy 0, policy_version 1770693 (0.0011) [2023-12-27 04:17:42,659][105692] Updated weights for policy 0, policy_version 1770703 (0.0011) [2023-12-27 04:17:42,707][105692] Updated weights for policy 0, policy_version 1770713 (0.0010) [2023-12-27 04:17:42,911][105620] Updated weights for policy 1, policy_version 1774607 (0.0011) [2023-12-27 04:17:42,971][105620] Updated weights for policy 1, policy_version 1774617 (0.0010) [2023-12-27 04:17:43,022][105620] Updated weights for policy 1, policy_version 1774627 (0.0010) [2023-12-27 04:17:43,370][105692] Updated weights for policy 0, policy_version 1770723 (0.0009) [2023-12-27 04:17:43,440][105692] Updated weights for policy 0, policy_version 1770733 (0.0005) [2023-12-27 04:17:43,498][105692] Updated weights for policy 0, policy_version 1770743 (0.0008) [2023-12-27 04:17:43,814][105620] Updated weights for policy 1, policy_version 1774637 (0.0008) [2023-12-27 04:17:43,879][105620] Updated weights for policy 1, policy_version 1774647 (0.0005) [2023-12-27 04:17:43,942][105620] Updated weights for policy 1, policy_version 1774657 (0.0008) [2023-12-27 04:17:44,106][105692] Updated weights for policy 0, policy_version 1770753 (0.0008) [2023-12-27 04:17:44,159][105692] Updated weights for policy 0, policy_version 1770763 (0.0006) [2023-12-27 04:17:44,209][105692] Updated weights for policy 0, policy_version 1770773 (0.0007) [2023-12-27 04:17:44,259][105692] Updated weights for policy 0, policy_version 1770783 (0.0008) [2023-12-27 04:17:44,661][105620] Updated weights for policy 1, policy_version 1774668 (0.0009) [2023-12-27 04:17:44,709][105620] Updated weights for policy 1, policy_version 1774678 (0.0009) [2023-12-27 04:17:44,763][105620] Updated weights for policy 1, policy_version 1774688 (0.0008) [2023-12-27 04:17:45,011][105692] Updated weights for policy 0, policy_version 1770793 (0.0010) [2023-12-27 04:17:45,075][105692] Updated weights for policy 0, policy_version 1770803 (0.0007) [2023-12-27 04:17:45,138][105692] Updated weights for policy 0, policy_version 1770813 (0.0009) [2023-12-27 04:17:45,559][105620] Updated weights for policy 1, policy_version 1774698 (0.0009) [2023-12-27 04:17:45,615][105620] Updated weights for policy 1, policy_version 1774708 (0.0010) [2023-12-27 04:17:45,671][105620] Updated weights for policy 1, policy_version 1774718 (0.0011) [2023-12-27 04:17:45,719][105620] Updated weights for policy 1, policy_version 1774728 (0.0010) [2023-12-27 04:17:45,833][105692] Updated weights for policy 0, policy_version 1770823 (0.0008) [2023-12-27 04:17:45,878][105692] Updated weights for policy 0, policy_version 1770833 (0.0008) [2023-12-27 04:17:45,921][105692] Updated weights for policy 0, policy_version 1770843 (0.0007) [2023-12-27 04:17:46,062][104569] Fps is (10 sec: 19660.1, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 907796480. Throughput: 0: 9713.4, 1: 9843.2. Samples: 907764132. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:17:46,063][104569] Avg episode reward: [(0, '8348.082'), (1, '9077.845')] [2023-12-27 04:17:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001774728_454393856.pth... [2023-12-27 04:17:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001770848_453402624.pth... [2023-12-27 04:17:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001773576_454098944.pth [2023-12-27 04:17:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001769696_453107712.pth [2023-12-27 04:17:46,428][105620] Updated weights for policy 1, policy_version 1774738 (0.0010) [2023-12-27 04:17:46,482][105620] Updated weights for policy 1, policy_version 1774748 (0.0010) [2023-12-27 04:17:46,540][105620] Updated weights for policy 1, policy_version 1774758 (0.0010) [2023-12-27 04:17:46,747][105692] Updated weights for policy 0, policy_version 1770853 (0.0009) [2023-12-27 04:17:46,811][105692] Updated weights for policy 0, policy_version 1770863 (0.0010) [2023-12-27 04:17:46,872][105692] Updated weights for policy 0, policy_version 1770873 (0.0009) [2023-12-27 04:17:47,286][105620] Updated weights for policy 1, policy_version 1774768 (0.0010) [2023-12-27 04:17:47,337][105620] Updated weights for policy 1, policy_version 1774778 (0.0010) [2023-12-27 04:17:47,384][105620] Updated weights for policy 1, policy_version 1774788 (0.0010) [2023-12-27 04:17:47,579][105692] Updated weights for policy 0, policy_version 1770883 (0.0009) [2023-12-27 04:17:47,647][105692] Updated weights for policy 0, policy_version 1770893 (0.0010) [2023-12-27 04:17:47,710][105692] Updated weights for policy 0, policy_version 1770903 (0.0008) [2023-12-27 04:17:48,155][105620] Updated weights for policy 1, policy_version 1774798 (0.0010) [2023-12-27 04:17:48,218][105620] Updated weights for policy 1, policy_version 1774808 (0.0010) [2023-12-27 04:17:48,275][105620] Updated weights for policy 1, policy_version 1774818 (0.0010) [2023-12-27 04:17:48,371][105692] Updated weights for policy 0, policy_version 1770913 (0.0011) [2023-12-27 04:17:48,438][105692] Updated weights for policy 0, policy_version 1770923 (0.0011) [2023-12-27 04:17:48,507][105692] Updated weights for policy 0, policy_version 1770933 (0.0010) [2023-12-27 04:17:48,573][105692] Updated weights for policy 0, policy_version 1770943 (0.0011) [2023-12-27 04:17:48,935][105620] Updated weights for policy 1, policy_version 1774828 (0.0006) [2023-12-27 04:17:49,006][105620] Updated weights for policy 1, policy_version 1774838 (0.0007) [2023-12-27 04:17:49,075][105620] Updated weights for policy 1, policy_version 1774848 (0.0009) [2023-12-27 04:17:49,311][105692] Updated weights for policy 0, policy_version 1770953 (0.0006) [2023-12-27 04:17:49,380][105692] Updated weights for policy 0, policy_version 1770963 (0.0009) [2023-12-27 04:17:49,439][105692] Updated weights for policy 0, policy_version 1770973 (0.0009) [2023-12-27 04:17:49,816][105620] Updated weights for policy 1, policy_version 1774858 (0.0009) [2023-12-27 04:17:49,882][105620] Updated weights for policy 1, policy_version 1774868 (0.0008) [2023-12-27 04:17:49,942][105620] Updated weights for policy 1, policy_version 1774878 (0.0009) [2023-12-27 04:17:50,009][105620] Updated weights for policy 1, policy_version 1774888 (0.0006) [2023-12-27 04:17:50,135][105692] Updated weights for policy 0, policy_version 1770983 (0.0008) [2023-12-27 04:17:50,209][105692] Updated weights for policy 0, policy_version 1770993 (0.0007) [2023-12-27 04:17:50,269][105692] Updated weights for policy 0, policy_version 1771003 (0.0008) [2023-12-27 04:17:50,727][105620] Updated weights for policy 1, policy_version 1774898 (0.0008) [2023-12-27 04:17:50,792][105620] Updated weights for policy 1, policy_version 1774908 (0.0009) [2023-12-27 04:17:50,850][105620] Updated weights for policy 1, policy_version 1774918 (0.0007) [2023-12-27 04:17:51,025][105692] Updated weights for policy 0, policy_version 1771013 (0.0009) [2023-12-27 04:17:51,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 907886592. Throughput: 0: 9768.2, 1: 9818.9. Samples: 907877996. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:17:51,062][104569] Avg episode reward: [(0, '8349.095'), (1, '8893.594')] [2023-12-27 04:17:51,087][105692] Updated weights for policy 0, policy_version 1771023 (0.0009) [2023-12-27 04:17:51,153][105692] Updated weights for policy 0, policy_version 1771033 (0.0009) [2023-12-27 04:17:51,610][105620] Updated weights for policy 1, policy_version 1774928 (0.0009) [2023-12-27 04:17:51,675][105620] Updated weights for policy 1, policy_version 1774938 (0.0008) [2023-12-27 04:17:51,744][105620] Updated weights for policy 1, policy_version 1774948 (0.0008) [2023-12-27 04:17:51,776][105692] Updated weights for policy 0, policy_version 1771043 (0.0006) [2023-12-27 04:17:51,838][105692] Updated weights for policy 0, policy_version 1771053 (0.0008) [2023-12-27 04:17:51,897][105692] Updated weights for policy 0, policy_version 1771063 (0.0008) [2023-12-27 04:17:52,424][105620] Updated weights for policy 1, policy_version 1774958 (0.0008) [2023-12-27 04:17:52,496][105620] Updated weights for policy 1, policy_version 1774968 (0.0005) [2023-12-27 04:17:52,560][105620] Updated weights for policy 1, policy_version 1774978 (0.0005) [2023-12-27 04:17:52,679][105692] Updated weights for policy 0, policy_version 1771073 (0.0009) [2023-12-27 04:17:52,738][105692] Updated weights for policy 0, policy_version 1771083 (0.0009) [2023-12-27 04:17:52,796][105692] Updated weights for policy 0, policy_version 1771093 (0.0010) [2023-12-27 04:17:52,851][105692] Updated weights for policy 0, policy_version 1771103 (0.0010) [2023-12-27 04:17:53,101][105620] Updated weights for policy 1, policy_version 1774988 (0.0005) [2023-12-27 04:17:53,157][105620] Updated weights for policy 1, policy_version 1774998 (0.0006) [2023-12-27 04:17:53,219][105620] Updated weights for policy 1, policy_version 1775008 (0.0008) [2023-12-27 04:17:53,605][105692] Updated weights for policy 0, policy_version 1771113 (0.0010) [2023-12-27 04:17:53,650][105692] Updated weights for policy 0, policy_version 1771123 (0.0010) [2023-12-27 04:17:53,704][105692] Updated weights for policy 0, policy_version 1771133 (0.0010) [2023-12-27 04:17:53,850][105620] Updated weights for policy 1, policy_version 1775018 (0.0008) [2023-12-27 04:17:53,913][105620] Updated weights for policy 1, policy_version 1775028 (0.0008) [2023-12-27 04:17:53,972][105620] Updated weights for policy 1, policy_version 1775038 (0.0008) [2023-12-27 04:17:54,017][105620] Updated weights for policy 1, policy_version 1775048 (0.0008) [2023-12-27 04:17:54,456][105692] Updated weights for policy 0, policy_version 1771143 (0.0010) [2023-12-27 04:17:54,508][105692] Updated weights for policy 0, policy_version 1771153 (0.0006) [2023-12-27 04:17:54,555][105692] Updated weights for policy 0, policy_version 1771163 (0.0005) [2023-12-27 04:17:54,697][105620] Updated weights for policy 1, policy_version 1775058 (0.0010) [2023-12-27 04:17:54,751][105620] Updated weights for policy 1, policy_version 1775069 (0.0010) [2023-12-27 04:17:54,805][105620] Updated weights for policy 1, policy_version 1775079 (0.0010) [2023-12-27 04:17:55,114][105692] Updated weights for policy 0, policy_version 1771173 (0.0008) [2023-12-27 04:17:55,171][105692] Updated weights for policy 0, policy_version 1771183 (0.0008) [2023-12-27 04:17:55,227][105692] Updated weights for policy 0, policy_version 1771193 (0.0005) [2023-12-27 04:17:55,714][105620] Updated weights for policy 1, policy_version 1775089 (0.0010) [2023-12-27 04:17:55,778][105620] Updated weights for policy 1, policy_version 1775099 (0.0009) [2023-12-27 04:17:55,779][105692] Updated weights for policy 0, policy_version 1771203 (0.0007) [2023-12-27 04:17:55,835][105692] Updated weights for policy 0, policy_version 1771213 (0.0008) [2023-12-27 04:17:55,837][105620] Updated weights for policy 1, policy_version 1775109 (0.0007) [2023-12-27 04:17:55,881][105692] Updated weights for policy 0, policy_version 1771223 (0.0010) [2023-12-27 04:17:56,062][104569] Fps is (10 sec: 19661.6, 60 sec: 19660.9, 300 sec: 19577.5). Total num frames: 907993088. Throughput: 0: 9834.9, 1: 9792.1. Samples: 907996864. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:17:56,062][104569] Avg episode reward: [(0, '8900.307'), (1, '9167.856')] [2023-12-27 04:17:56,492][105692] Updated weights for policy 0, policy_version 1771233 (0.0010) [2023-12-27 04:17:56,547][105692] Updated weights for policy 0, policy_version 1771243 (0.0005) [2023-12-27 04:17:56,604][105692] Updated weights for policy 0, policy_version 1771253 (0.0005) [2023-12-27 04:17:56,652][105692] Updated weights for policy 0, policy_version 1771263 (0.0006) [2023-12-27 04:17:56,676][105620] Updated weights for policy 1, policy_version 1775119 (0.0008) [2023-12-27 04:17:56,734][105620] Updated weights for policy 1, policy_version 1775130 (0.0008) [2023-12-27 04:17:56,784][105620] Updated weights for policy 1, policy_version 1775140 (0.0009) [2023-12-27 04:17:57,215][105692] Updated weights for policy 0, policy_version 1771273 (0.0005) [2023-12-27 04:17:57,278][105692] Updated weights for policy 0, policy_version 1771283 (0.0005) [2023-12-27 04:17:57,332][105692] Updated weights for policy 0, policy_version 1771293 (0.0008) [2023-12-27 04:17:57,649][105620] Updated weights for policy 1, policy_version 1775150 (0.0008) [2023-12-27 04:17:57,708][105620] Updated weights for policy 1, policy_version 1775160 (0.0008) [2023-12-27 04:17:57,760][105620] Updated weights for policy 1, policy_version 1775170 (0.0007) [2023-12-27 04:17:58,005][105692] Updated weights for policy 0, policy_version 1771303 (0.0010) [2023-12-27 04:17:58,059][105692] Updated weights for policy 0, policy_version 1771313 (0.0010) [2023-12-27 04:17:58,122][105692] Updated weights for policy 0, policy_version 1771323 (0.0006) [2023-12-27 04:17:58,473][105620] Updated weights for policy 1, policy_version 1775180 (0.0008) [2023-12-27 04:17:58,541][105620] Updated weights for policy 1, policy_version 1775190 (0.0008) [2023-12-27 04:17:58,604][105620] Updated weights for policy 1, policy_version 1775200 (0.0008) [2023-12-27 04:17:58,841][105692] Updated weights for policy 0, policy_version 1771333 (0.0008) [2023-12-27 04:17:58,911][105692] Updated weights for policy 0, policy_version 1771343 (0.0009) [2023-12-27 04:17:58,978][105692] Updated weights for policy 0, policy_version 1771353 (0.0009) [2023-12-27 04:17:59,391][105620] Updated weights for policy 1, policy_version 1775210 (0.0008) [2023-12-27 04:17:59,453][105620] Updated weights for policy 1, policy_version 1775220 (0.0009) [2023-12-27 04:17:59,517][105620] Updated weights for policy 1, policy_version 1775230 (0.0010) [2023-12-27 04:17:59,572][105620] Updated weights for policy 1, policy_version 1775240 (0.0008) [2023-12-27 04:17:59,736][105692] Updated weights for policy 0, policy_version 1771363 (0.0007) [2023-12-27 04:17:59,788][105692] Updated weights for policy 0, policy_version 1771373 (0.0010) [2023-12-27 04:17:59,852][105692] Updated weights for policy 0, policy_version 1771384 (0.0009) [2023-12-27 04:18:00,291][105620] Updated weights for policy 1, policy_version 1775250 (0.0009) [2023-12-27 04:18:00,343][105620] Updated weights for policy 1, policy_version 1775260 (0.0009) [2023-12-27 04:18:00,401][105620] Updated weights for policy 1, policy_version 1775270 (0.0009) [2023-12-27 04:18:00,583][105692] Updated weights for policy 0, policy_version 1771394 (0.0008) [2023-12-27 04:18:00,641][105692] Updated weights for policy 0, policy_version 1771404 (0.0009) [2023-12-27 04:18:00,687][105692] Updated weights for policy 0, policy_version 1771414 (0.0008) [2023-12-27 04:18:00,733][105692] Updated weights for policy 0, policy_version 1771424 (0.0008) [2023-12-27 04:18:01,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 908083200. Throughput: 0: 9926.7, 1: 9728.0. Samples: 908056160. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:18:01,062][104569] Avg episode reward: [(0, '8532.965'), (1, '9167.521')] [2023-12-27 04:18:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001771424_453550080.pth... [2023-12-27 04:18:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001775272_454533120.pth... [2023-12-27 04:18:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001774152_454246400.pth [2023-12-27 04:18:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001770272_453255168.pth [2023-12-27 04:18:01,196][105620] Updated weights for policy 1, policy_version 1775280 (0.0009) [2023-12-27 04:18:01,254][105620] Updated weights for policy 1, policy_version 1775290 (0.0009) [2023-12-27 04:18:01,319][105620] Updated weights for policy 1, policy_version 1775300 (0.0009) [2023-12-27 04:18:01,493][105692] Updated weights for policy 0, policy_version 1771434 (0.0009) [2023-12-27 04:18:01,547][105692] Updated weights for policy 0, policy_version 1771445 (0.0008) [2023-12-27 04:18:01,602][105692] Updated weights for policy 0, policy_version 1771455 (0.0009) [2023-12-27 04:18:02,064][105620] Updated weights for policy 1, policy_version 1775310 (0.0009) [2023-12-27 04:18:02,124][105620] Updated weights for policy 1, policy_version 1775320 (0.0008) [2023-12-27 04:18:02,172][105620] Updated weights for policy 1, policy_version 1775330 (0.0009) [2023-12-27 04:18:02,443][105692] Updated weights for policy 0, policy_version 1771465 (0.0008) [2023-12-27 04:18:02,507][105692] Updated weights for policy 0, policy_version 1771475 (0.0010) [2023-12-27 04:18:02,568][105692] Updated weights for policy 0, policy_version 1771485 (0.0009) [2023-12-27 04:18:02,801][105620] Updated weights for policy 1, policy_version 1775340 (0.0006) [2023-12-27 04:18:02,864][105620] Updated weights for policy 1, policy_version 1775350 (0.0005) [2023-12-27 04:18:02,923][105620] Updated weights for policy 1, policy_version 1775360 (0.0007) [2023-12-27 04:18:03,338][105692] Updated weights for policy 0, policy_version 1771495 (0.0008) [2023-12-27 04:18:03,388][105692] Updated weights for policy 0, policy_version 1771505 (0.0007) [2023-12-27 04:18:03,434][105692] Updated weights for policy 0, policy_version 1771515 (0.0005) [2023-12-27 04:18:03,470][105620] Updated weights for policy 1, policy_version 1775370 (0.0006) [2023-12-27 04:18:03,536][105620] Updated weights for policy 1, policy_version 1775380 (0.0005) [2023-12-27 04:18:03,599][105620] Updated weights for policy 1, policy_version 1775390 (0.0006) [2023-12-27 04:18:03,661][105620] Updated weights for policy 1, policy_version 1775400 (0.0006) [2023-12-27 04:18:04,050][105692] Updated weights for policy 0, policy_version 1771525 (0.0007) [2023-12-27 04:18:04,112][105692] Updated weights for policy 0, policy_version 1771535 (0.0009) [2023-12-27 04:18:04,174][105692] Updated weights for policy 0, policy_version 1771545 (0.0010) [2023-12-27 04:18:04,389][105620] Updated weights for policy 1, policy_version 1775410 (0.0008) [2023-12-27 04:18:04,442][105620] Updated weights for policy 1, policy_version 1775420 (0.0008) [2023-12-27 04:18:04,497][105620] Updated weights for policy 1, policy_version 1775430 (0.0008) [2023-12-27 04:18:04,931][105692] Updated weights for policy 0, policy_version 1771555 (0.0011) [2023-12-27 04:18:04,986][105692] Updated weights for policy 0, policy_version 1771565 (0.0009) [2023-12-27 04:18:05,047][105692] Updated weights for policy 0, policy_version 1771575 (0.0010) [2023-12-27 04:18:05,269][105620] Updated weights for policy 1, policy_version 1775440 (0.0010) [2023-12-27 04:18:05,326][105620] Updated weights for policy 1, policy_version 1775450 (0.0008) [2023-12-27 04:18:05,376][105620] Updated weights for policy 1, policy_version 1775460 (0.0005) [2023-12-27 04:18:05,640][105692] Updated weights for policy 0, policy_version 1771585 (0.0010) [2023-12-27 04:18:05,701][105692] Updated weights for policy 0, policy_version 1771595 (0.0007) [2023-12-27 04:18:05,748][105692] Updated weights for policy 0, policy_version 1771605 (0.0009) [2023-12-27 04:18:05,796][105692] Updated weights for policy 0, policy_version 1771615 (0.0010) [2023-12-27 04:18:05,962][105620] Updated weights for policy 1, policy_version 1775470 (0.0007) [2023-12-27 04:18:06,017][105620] Updated weights for policy 1, policy_version 1775480 (0.0008) [2023-12-27 04:18:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 908181504. Throughput: 0: 9824.4, 1: 9706.1. Samples: 908170580. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:18:06,062][104569] Avg episode reward: [(0, '8348.316'), (1, '8982.562')] [2023-12-27 04:18:06,082][105620] Updated weights for policy 1, policy_version 1775490 (0.0009) [2023-12-27 04:18:06,536][105692] Updated weights for policy 0, policy_version 1771625 (0.0011) [2023-12-27 04:18:06,605][105692] Updated weights for policy 0, policy_version 1771635 (0.0010) [2023-12-27 04:18:06,679][105692] Updated weights for policy 0, policy_version 1771645 (0.0011) [2023-12-27 04:18:06,787][105620] Updated weights for policy 1, policy_version 1775500 (0.0007) [2023-12-27 04:18:06,856][105620] Updated weights for policy 1, policy_version 1775510 (0.0006) [2023-12-27 04:18:06,920][105620] Updated weights for policy 1, policy_version 1775520 (0.0006) [2023-12-27 04:18:07,410][105692] Updated weights for policy 0, policy_version 1771655 (0.0010) [2023-12-27 04:18:07,471][105692] Updated weights for policy 0, policy_version 1771665 (0.0010) [2023-12-27 04:18:07,529][105692] Updated weights for policy 0, policy_version 1771675 (0.0010) [2023-12-27 04:18:07,590][105620] Updated weights for policy 1, policy_version 1775530 (0.0007) [2023-12-27 04:18:07,652][105620] Updated weights for policy 1, policy_version 1775540 (0.0008) [2023-12-27 04:18:07,701][105620] Updated weights for policy 1, policy_version 1775550 (0.0008) [2023-12-27 04:18:07,752][105620] Updated weights for policy 1, policy_version 1775560 (0.0008) [2023-12-27 04:18:08,255][105692] Updated weights for policy 0, policy_version 1771685 (0.0009) [2023-12-27 04:18:08,314][105692] Updated weights for policy 0, policy_version 1771695 (0.0009) [2023-12-27 04:18:08,340][105620] Updated weights for policy 1, policy_version 1775570 (0.0007) [2023-12-27 04:18:08,379][105692] Updated weights for policy 0, policy_version 1771705 (0.0008) [2023-12-27 04:18:08,392][105620] Updated weights for policy 1, policy_version 1775580 (0.0007) [2023-12-27 04:18:08,452][105620] Updated weights for policy 1, policy_version 1775590 (0.0006) [2023-12-27 04:18:09,103][105692] Updated weights for policy 0, policy_version 1771715 (0.0008) [2023-12-27 04:18:09,139][105620] Updated weights for policy 1, policy_version 1775600 (0.0008) [2023-12-27 04:18:09,150][105692] Updated weights for policy 0, policy_version 1771725 (0.0005) [2023-12-27 04:18:09,192][105620] Updated weights for policy 1, policy_version 1775610 (0.0008) [2023-12-27 04:18:09,198][105692] Updated weights for policy 0, policy_version 1771735 (0.0007) [2023-12-27 04:18:09,249][105620] Updated weights for policy 1, policy_version 1775620 (0.0008) [2023-12-27 04:18:09,960][105692] Updated weights for policy 0, policy_version 1771745 (0.0008) [2023-12-27 04:18:10,026][105692] Updated weights for policy 0, policy_version 1771755 (0.0008) [2023-12-27 04:18:10,037][105620] Updated weights for policy 1, policy_version 1775630 (0.0007) [2023-12-27 04:18:10,090][105692] Updated weights for policy 0, policy_version 1771765 (0.0007) [2023-12-27 04:18:10,100][105620] Updated weights for policy 1, policy_version 1775640 (0.0008) [2023-12-27 04:18:10,158][105692] Updated weights for policy 0, policy_version 1771775 (0.0009) [2023-12-27 04:18:10,168][105620] Updated weights for policy 1, policy_version 1775650 (0.0007) [2023-12-27 04:18:10,856][105620] Updated weights for policy 1, policy_version 1775660 (0.0009) [2023-12-27 04:18:10,914][105620] Updated weights for policy 1, policy_version 1775670 (0.0009) [2023-12-27 04:18:10,918][105692] Updated weights for policy 0, policy_version 1771785 (0.0007) [2023-12-27 04:18:10,963][105692] Updated weights for policy 0, policy_version 1771795 (0.0007) [2023-12-27 04:18:10,971][105620] Updated weights for policy 1, policy_version 1775680 (0.0008) [2023-12-27 04:18:11,012][105692] Updated weights for policy 0, policy_version 1771805 (0.0009) [2023-12-27 04:18:11,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 908288000. Throughput: 0: 9791.2, 1: 9772.5. Samples: 908289376. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:18:11,062][104569] Avg episode reward: [(0, '8620.271'), (1, '8982.385')] [2023-12-27 04:18:11,768][105620] Updated weights for policy 1, policy_version 1775690 (0.0008) [2023-12-27 04:18:11,823][105620] Updated weights for policy 1, policy_version 1775700 (0.0007) [2023-12-27 04:18:11,854][105692] Updated weights for policy 0, policy_version 1771815 (0.0008) [2023-12-27 04:18:11,882][105620] Updated weights for policy 1, policy_version 1775710 (0.0007) [2023-12-27 04:18:11,911][105692] Updated weights for policy 0, policy_version 1771825 (0.0008) [2023-12-27 04:18:11,946][105620] Updated weights for policy 1, policy_version 1775720 (0.0007) [2023-12-27 04:18:11,973][105692] Updated weights for policy 0, policy_version 1771835 (0.0008) [2023-12-27 04:18:12,653][105692] Updated weights for policy 0, policy_version 1771845 (0.0009) [2023-12-27 04:18:12,712][105692] Updated weights for policy 0, policy_version 1771855 (0.0009) [2023-12-27 04:18:12,773][105692] Updated weights for policy 0, policy_version 1771865 (0.0007) [2023-12-27 04:18:12,783][105620] Updated weights for policy 1, policy_version 1775730 (0.0008) [2023-12-27 04:18:12,847][105620] Updated weights for policy 1, policy_version 1775740 (0.0009) [2023-12-27 04:18:12,910][105620] Updated weights for policy 1, policy_version 1775750 (0.0010) [2023-12-27 04:18:13,428][105692] Updated weights for policy 0, policy_version 1771876 (0.0008) [2023-12-27 04:18:13,487][105692] Updated weights for policy 0, policy_version 1771887 (0.0010) [2023-12-27 04:18:13,535][105692] Updated weights for policy 0, policy_version 1771897 (0.0009) [2023-12-27 04:18:13,609][105620] Updated weights for policy 1, policy_version 1775760 (0.0009) [2023-12-27 04:18:13,665][105620] Updated weights for policy 1, policy_version 1775770 (0.0009) [2023-12-27 04:18:13,723][105620] Updated weights for policy 1, policy_version 1775780 (0.0009) [2023-12-27 04:18:14,316][105692] Updated weights for policy 0, policy_version 1771907 (0.0009) [2023-12-27 04:18:14,362][105692] Updated weights for policy 0, policy_version 1771917 (0.0008) [2023-12-27 04:18:14,409][105692] Updated weights for policy 0, policy_version 1771927 (0.0008) [2023-12-27 04:18:14,470][105620] Updated weights for policy 1, policy_version 1775790 (0.0009) [2023-12-27 04:18:14,528][105620] Updated weights for policy 1, policy_version 1775800 (0.0009) [2023-12-27 04:18:14,586][105620] Updated weights for policy 1, policy_version 1775810 (0.0009) [2023-12-27 04:18:15,197][105692] Updated weights for policy 0, policy_version 1771937 (0.0008) [2023-12-27 04:18:15,257][105692] Updated weights for policy 0, policy_version 1771947 (0.0009) [2023-12-27 04:18:15,306][105620] Updated weights for policy 1, policy_version 1775820 (0.0009) [2023-12-27 04:18:15,321][105692] Updated weights for policy 0, policy_version 1771957 (0.0007) [2023-12-27 04:18:15,373][105620] Updated weights for policy 1, policy_version 1775830 (0.0009) [2023-12-27 04:18:15,385][105692] Updated weights for policy 0, policy_version 1771967 (0.0008) [2023-12-27 04:18:15,443][105620] Updated weights for policy 1, policy_version 1775840 (0.0010) [2023-12-27 04:18:15,943][105692] Updated weights for policy 0, policy_version 1771977 (0.0009) [2023-12-27 04:18:15,991][105692] Updated weights for policy 0, policy_version 1771987 (0.0009) [2023-12-27 04:18:16,045][105692] Updated weights for policy 0, policy_version 1771997 (0.0008) [2023-12-27 04:18:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19661.0, 300 sec: 19549.7). Total num frames: 908378112. Throughput: 0: 9806.1, 1: 9721.9. Samples: 908345344. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:18:16,062][104569] Avg episode reward: [(0, '8350.516'), (1, '9167.457')] [2023-12-27 04:18:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001772000_453697536.pth... [2023-12-27 04:18:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001775848_454680576.pth... [2023-12-27 04:18:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001770848_453402624.pth [2023-12-27 04:18:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001774728_454393856.pth [2023-12-27 04:18:16,270][105620] Updated weights for policy 1, policy_version 1775850 (0.0010) [2023-12-27 04:18:16,318][105620] Updated weights for policy 1, policy_version 1775860 (0.0009) [2023-12-27 04:18:16,368][105620] Updated weights for policy 1, policy_version 1775870 (0.0009) [2023-12-27 04:18:16,427][105620] Updated weights for policy 1, policy_version 1775880 (0.0009) [2023-12-27 04:18:16,832][105692] Updated weights for policy 0, policy_version 1772007 (0.0009) [2023-12-27 04:18:16,884][105692] Updated weights for policy 0, policy_version 1772018 (0.0009) [2023-12-27 04:18:16,942][105692] Updated weights for policy 0, policy_version 1772028 (0.0008) [2023-12-27 04:18:17,140][105620] Updated weights for policy 1, policy_version 1775890 (0.0010) [2023-12-27 04:18:17,205][105620] Updated weights for policy 1, policy_version 1775900 (0.0010) [2023-12-27 04:18:17,269][105620] Updated weights for policy 1, policy_version 1775910 (0.0010) [2023-12-27 04:18:17,590][105692] Updated weights for policy 0, policy_version 1772038 (0.0006) [2023-12-27 04:18:17,644][105692] Updated weights for policy 0, policy_version 1772048 (0.0009) [2023-12-27 04:18:17,699][105692] Updated weights for policy 0, policy_version 1772058 (0.0008) [2023-12-27 04:18:18,003][105620] Updated weights for policy 1, policy_version 1775920 (0.0009) [2023-12-27 04:18:18,057][105620] Updated weights for policy 1, policy_version 1775930 (0.0008) [2023-12-27 04:18:18,116][105620] Updated weights for policy 1, policy_version 1775940 (0.0006) [2023-12-27 04:18:18,338][105692] Updated weights for policy 0, policy_version 1772068 (0.0009) [2023-12-27 04:18:18,399][105692] Updated weights for policy 0, policy_version 1772078 (0.0008) [2023-12-27 04:18:18,456][105692] Updated weights for policy 0, policy_version 1772088 (0.0011) [2023-12-27 04:18:18,857][105620] Updated weights for policy 1, policy_version 1775950 (0.0008) [2023-12-27 04:18:18,916][105620] Updated weights for policy 1, policy_version 1775960 (0.0010) [2023-12-27 04:18:18,978][105620] Updated weights for policy 1, policy_version 1775970 (0.0010) [2023-12-27 04:18:19,173][105692] Updated weights for policy 0, policy_version 1772098 (0.0010) [2023-12-27 04:18:19,229][105692] Updated weights for policy 0, policy_version 1772108 (0.0008) [2023-12-27 04:18:19,285][105692] Updated weights for policy 0, policy_version 1772118 (0.0008) [2023-12-27 04:18:19,354][105692] Updated weights for policy 0, policy_version 1772128 (0.0009) [2023-12-27 04:18:19,770][105620] Updated weights for policy 1, policy_version 1775980 (0.0010) [2023-12-27 04:18:19,823][105620] Updated weights for policy 1, policy_version 1775990 (0.0011) [2023-12-27 04:18:19,881][105620] Updated weights for policy 1, policy_version 1776000 (0.0006) [2023-12-27 04:18:20,147][105692] Updated weights for policy 0, policy_version 1772138 (0.0009) [2023-12-27 04:18:20,212][105692] Updated weights for policy 0, policy_version 1772148 (0.0010) [2023-12-27 04:18:20,274][105692] Updated weights for policy 0, policy_version 1772158 (0.0010) [2023-12-27 04:18:20,490][105620] Updated weights for policy 1, policy_version 1776010 (0.0010) [2023-12-27 04:18:20,546][105620] Updated weights for policy 1, policy_version 1776020 (0.0009) [2023-12-27 04:18:20,608][105620] Updated weights for policy 1, policy_version 1776030 (0.0009) [2023-12-27 04:18:20,665][105620] Updated weights for policy 1, policy_version 1776040 (0.0008) [2023-12-27 04:18:21,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 908468224. Throughput: 0: 9817.5, 1: 9571.6. Samples: 908460780. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:18:21,063][104569] Avg episode reward: [(0, '8352.676'), (1, '9352.417')] [2023-12-27 04:18:21,068][105692] Updated weights for policy 0, policy_version 1772168 (0.0011) [2023-12-27 04:18:21,134][105692] Updated weights for policy 0, policy_version 1772178 (0.0010) [2023-12-27 04:18:21,201][105692] Updated weights for policy 0, policy_version 1772188 (0.0011) [2023-12-27 04:18:21,473][105620] Updated weights for policy 1, policy_version 1776050 (0.0007) [2023-12-27 04:18:21,526][105620] Updated weights for policy 1, policy_version 1776060 (0.0009) [2023-12-27 04:18:21,586][105620] Updated weights for policy 1, policy_version 1776070 (0.0009) [2023-12-27 04:18:22,029][105692] Updated weights for policy 0, policy_version 1772198 (0.0011) [2023-12-27 04:18:22,078][105692] Updated weights for policy 0, policy_version 1772208 (0.0010) [2023-12-27 04:18:22,143][105692] Updated weights for policy 0, policy_version 1772218 (0.0011) [2023-12-27 04:18:22,387][105620] Updated weights for policy 1, policy_version 1776080 (0.0009) [2023-12-27 04:18:22,451][105620] Updated weights for policy 1, policy_version 1776090 (0.0008) [2023-12-27 04:18:22,507][105620] Updated weights for policy 1, policy_version 1776100 (0.0006) [2023-12-27 04:18:22,948][105692] Updated weights for policy 0, policy_version 1772228 (0.0010) [2023-12-27 04:18:23,010][105692] Updated weights for policy 0, policy_version 1772238 (0.0009) [2023-12-27 04:18:23,070][105692] Updated weights for policy 0, policy_version 1772248 (0.0008) [2023-12-27 04:18:23,143][105620] Updated weights for policy 1, policy_version 1776110 (0.0008) [2023-12-27 04:18:23,209][105620] Updated weights for policy 1, policy_version 1776121 (0.0009) [2023-12-27 04:18:23,275][105620] Updated weights for policy 1, policy_version 1776131 (0.0008) [2023-12-27 04:18:23,777][105692] Updated weights for policy 0, policy_version 1772258 (0.0008) [2023-12-27 04:18:23,833][105692] Updated weights for policy 0, policy_version 1772268 (0.0006) [2023-12-27 04:18:23,892][105692] Updated weights for policy 0, policy_version 1772278 (0.0005) [2023-12-27 04:18:23,943][105692] Updated weights for policy 0, policy_version 1772288 (0.0007) [2023-12-27 04:18:24,040][105620] Updated weights for policy 1, policy_version 1776141 (0.0009) [2023-12-27 04:18:24,107][105620] Updated weights for policy 1, policy_version 1776151 (0.0010) [2023-12-27 04:18:24,160][105620] Updated weights for policy 1, policy_version 1776161 (0.0009) [2023-12-27 04:18:24,513][105692] Updated weights for policy 0, policy_version 1772298 (0.0007) [2023-12-27 04:18:24,570][105692] Updated weights for policy 0, policy_version 1772308 (0.0009) [2023-12-27 04:18:24,620][105692] Updated weights for policy 0, policy_version 1772318 (0.0007) [2023-12-27 04:18:24,926][105620] Updated weights for policy 1, policy_version 1776171 (0.0009) [2023-12-27 04:18:24,974][105620] Updated weights for policy 1, policy_version 1776181 (0.0010) [2023-12-27 04:18:25,036][105620] Updated weights for policy 1, policy_version 1776191 (0.0009) [2023-12-27 04:18:25,182][105692] Updated weights for policy 0, policy_version 1772328 (0.0006) [2023-12-27 04:18:25,235][105692] Updated weights for policy 0, policy_version 1772338 (0.0007) [2023-12-27 04:18:25,288][105692] Updated weights for policy 0, policy_version 1772348 (0.0009) [2023-12-27 04:18:25,724][105620] Updated weights for policy 1, policy_version 1776201 (0.0009) [2023-12-27 04:18:25,792][105620] Updated weights for policy 1, policy_version 1776211 (0.0008) [2023-12-27 04:18:25,845][105620] Updated weights for policy 1, policy_version 1776221 (0.0007) [2023-12-27 04:18:25,883][105692] Updated weights for policy 0, policy_version 1772358 (0.0007) [2023-12-27 04:18:25,899][105620] Updated weights for policy 1, policy_version 1776231 (0.0007) [2023-12-27 04:18:25,947][105692] Updated weights for policy 0, policy_version 1772368 (0.0006) [2023-12-27 04:18:26,010][105692] Updated weights for policy 0, policy_version 1772378 (0.0005) [2023-12-27 04:18:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 908574720. Throughput: 0: 9827.9, 1: 9509.0. Samples: 908577436. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:18:26,063][104569] Avg episode reward: [(0, '8528.063'), (1, '9352.414')] [2023-12-27 04:18:26,526][105620] Updated weights for policy 1, policy_version 1776241 (0.0008) [2023-12-27 04:18:26,579][105620] Updated weights for policy 1, policy_version 1776251 (0.0008) [2023-12-27 04:18:26,629][105620] Updated weights for policy 1, policy_version 1776262 (0.0010) [2023-12-27 04:18:26,635][105692] Updated weights for policy 0, policy_version 1772388 (0.0005) [2023-12-27 04:18:26,689][105692] Updated weights for policy 0, policy_version 1772398 (0.0005) [2023-12-27 04:18:26,747][105692] Updated weights for policy 0, policy_version 1772408 (0.0009) [2023-12-27 04:18:27,268][105620] Updated weights for policy 1, policy_version 1776272 (0.0006) [2023-12-27 04:18:27,312][105692] Updated weights for policy 0, policy_version 1772418 (0.0010) [2023-12-27 04:18:27,317][105620] Updated weights for policy 1, policy_version 1776282 (0.0005) [2023-12-27 04:18:27,372][105620] Updated weights for policy 1, policy_version 1776292 (0.0006) [2023-12-27 04:18:27,373][105692] Updated weights for policy 0, policy_version 1772428 (0.0006) [2023-12-27 04:18:27,438][105692] Updated weights for policy 0, policy_version 1772438 (0.0005) [2023-12-27 04:18:27,509][105692] Updated weights for policy 0, policy_version 1772448 (0.0005) [2023-12-27 04:18:27,938][105620] Updated weights for policy 1, policy_version 1776302 (0.0008) [2023-12-27 04:18:27,989][105620] Updated weights for policy 1, policy_version 1776312 (0.0010) [2023-12-27 04:18:28,044][105620] Updated weights for policy 1, policy_version 1776322 (0.0010) [2023-12-27 04:18:28,095][105692] Updated weights for policy 0, policy_version 1772458 (0.0010) [2023-12-27 04:18:28,143][105692] Updated weights for policy 0, policy_version 1772468 (0.0010) [2023-12-27 04:18:28,200][105692] Updated weights for policy 0, policy_version 1772478 (0.0010) [2023-12-27 04:18:28,772][105620] Updated weights for policy 1, policy_version 1776332 (0.0010) [2023-12-27 04:18:28,814][105692] Updated weights for policy 0, policy_version 1772488 (0.0009) [2023-12-27 04:18:28,827][105620] Updated weights for policy 1, policy_version 1776342 (0.0007) [2023-12-27 04:18:28,875][105620] Updated weights for policy 1, policy_version 1776352 (0.0009) [2023-12-27 04:18:28,880][105692] Updated weights for policy 0, policy_version 1772498 (0.0005) [2023-12-27 04:18:28,942][105692] Updated weights for policy 0, policy_version 1772508 (0.0007) [2023-12-27 04:18:29,569][105620] Updated weights for policy 1, policy_version 1776362 (0.0008) [2023-12-27 04:18:29,643][105620] Updated weights for policy 1, policy_version 1776372 (0.0007) [2023-12-27 04:18:29,657][105692] Updated weights for policy 0, policy_version 1772518 (0.0010) [2023-12-27 04:18:29,692][105620] Updated weights for policy 1, policy_version 1776382 (0.0005) [2023-12-27 04:18:29,713][105692] Updated weights for policy 0, policy_version 1772528 (0.0011) [2023-12-27 04:18:29,747][105620] Updated weights for policy 1, policy_version 1776392 (0.0006) [2023-12-27 04:18:29,773][105692] Updated weights for policy 0, policy_version 1772538 (0.0011) [2023-12-27 04:18:30,462][105620] Updated weights for policy 1, policy_version 1776402 (0.0009) [2023-12-27 04:18:30,497][105692] Updated weights for policy 0, policy_version 1772548 (0.0010) [2023-12-27 04:18:30,517][105620] Updated weights for policy 1, policy_version 1776412 (0.0010) [2023-12-27 04:18:30,545][105692] Updated weights for policy 0, policy_version 1772558 (0.0010) [2023-12-27 04:18:30,577][105620] Updated weights for policy 1, policy_version 1776422 (0.0007) [2023-12-27 04:18:30,601][105692] Updated weights for policy 0, policy_version 1772569 (0.0007) [2023-12-27 04:18:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 908673024. Throughput: 0: 9908.2, 1: 9626.5. Samples: 908643184. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:18:31,063][104569] Avg episode reward: [(0, '8526.951'), (1, '9259.626')] [2023-12-27 04:18:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001772576_453844992.pth... [2023-12-27 04:18:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001776424_454828032.pth... [2023-12-27 04:18:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001775272_454533120.pth [2023-12-27 04:18:31,086][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001771424_453550080.pth [2023-12-27 04:18:31,181][105692] Updated weights for policy 0, policy_version 1772579 (0.0006) [2023-12-27 04:18:31,249][105692] Updated weights for policy 0, policy_version 1772589 (0.0008) [2023-12-27 04:18:31,294][105620] Updated weights for policy 1, policy_version 1776432 (0.0008) [2023-12-27 04:18:31,313][105692] Updated weights for policy 0, policy_version 1772599 (0.0008) [2023-12-27 04:18:31,349][105620] Updated weights for policy 1, policy_version 1776442 (0.0008) [2023-12-27 04:18:31,414][105620] Updated weights for policy 1, policy_version 1776452 (0.0009) [2023-12-27 04:18:31,934][105692] Updated weights for policy 0, policy_version 1772609 (0.0009) [2023-12-27 04:18:31,999][105692] Updated weights for policy 0, policy_version 1772619 (0.0009) [2023-12-27 04:18:32,057][105692] Updated weights for policy 0, policy_version 1772629 (0.0009) [2023-12-27 04:18:32,115][105692] Updated weights for policy 0, policy_version 1772639 (0.0009) [2023-12-27 04:18:32,186][105620] Updated weights for policy 1, policy_version 1776462 (0.0009) [2023-12-27 04:18:32,244][105620] Updated weights for policy 1, policy_version 1776472 (0.0009) [2023-12-27 04:18:32,306][105620] Updated weights for policy 1, policy_version 1776482 (0.0009) [2023-12-27 04:18:32,824][105692] Updated weights for policy 0, policy_version 1772649 (0.0006) [2023-12-27 04:18:32,882][105692] Updated weights for policy 0, policy_version 1772659 (0.0010) [2023-12-27 04:18:32,935][105692] Updated weights for policy 0, policy_version 1772669 (0.0010) [2023-12-27 04:18:33,055][105620] Updated weights for policy 1, policy_version 1776492 (0.0009) [2023-12-27 04:18:33,099][105620] Updated weights for policy 1, policy_version 1776502 (0.0008) [2023-12-27 04:18:33,146][105620] Updated weights for policy 1, policy_version 1776512 (0.0008) [2023-12-27 04:18:33,637][105692] Updated weights for policy 0, policy_version 1772679 (0.0010) [2023-12-27 04:18:33,685][105692] Updated weights for policy 0, policy_version 1772689 (0.0009) [2023-12-27 04:18:33,739][105692] Updated weights for policy 0, policy_version 1772699 (0.0005) [2023-12-27 04:18:33,963][105620] Updated weights for policy 1, policy_version 1776522 (0.0007) [2023-12-27 04:18:34,021][105620] Updated weights for policy 1, policy_version 1776532 (0.0007) [2023-12-27 04:18:34,088][105620] Updated weights for policy 1, policy_version 1776542 (0.0005) [2023-12-27 04:18:34,158][105620] Updated weights for policy 1, policy_version 1776552 (0.0008) [2023-12-27 04:18:34,388][105692] Updated weights for policy 0, policy_version 1772709 (0.0006) [2023-12-27 04:18:34,451][105692] Updated weights for policy 0, policy_version 1772719 (0.0010) [2023-12-27 04:18:34,514][105692] Updated weights for policy 0, policy_version 1772729 (0.0011) [2023-12-27 04:18:34,912][105620] Updated weights for policy 1, policy_version 1776562 (0.0008) [2023-12-27 04:18:34,982][105620] Updated weights for policy 1, policy_version 1776572 (0.0009) [2023-12-27 04:18:35,055][105620] Updated weights for policy 1, policy_version 1776582 (0.0009) [2023-12-27 04:18:35,248][105692] Updated weights for policy 0, policy_version 1772739 (0.0009) [2023-12-27 04:18:35,305][105692] Updated weights for policy 0, policy_version 1772749 (0.0010) [2023-12-27 04:18:35,358][105692] Updated weights for policy 0, policy_version 1772759 (0.0009) [2023-12-27 04:18:35,595][105620] Updated weights for policy 1, policy_version 1776592 (0.0008) [2023-12-27 04:18:35,658][105620] Updated weights for policy 1, policy_version 1776602 (0.0008) [2023-12-27 04:18:35,723][105620] Updated weights for policy 1, policy_version 1776612 (0.0006) [2023-12-27 04:18:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 908771328. Throughput: 0: 9996.2, 1: 9652.2. Samples: 908762172. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:18:36,063][104569] Avg episode reward: [(0, '8803.435'), (1, '9074.324')] [2023-12-27 04:18:36,251][105692] Updated weights for policy 0, policy_version 1772769 (0.0008) [2023-12-27 04:18:36,255][105620] Updated weights for policy 1, policy_version 1776622 (0.0008) [2023-12-27 04:18:36,305][105692] Updated weights for policy 0, policy_version 1772779 (0.0007) [2023-12-27 04:18:36,315][105620] Updated weights for policy 1, policy_version 1776632 (0.0011) [2023-12-27 04:18:36,369][105692] Updated weights for policy 0, policy_version 1772789 (0.0007) [2023-12-27 04:18:36,375][105620] Updated weights for policy 1, policy_version 1776642 (0.0007) [2023-12-27 04:18:36,435][105692] Updated weights for policy 0, policy_version 1772799 (0.0008) [2023-12-27 04:18:37,002][105620] Updated weights for policy 1, policy_version 1776652 (0.0008) [2023-12-27 04:18:37,065][105620] Updated weights for policy 1, policy_version 1776662 (0.0010) [2023-12-27 04:18:37,127][105620] Updated weights for policy 1, policy_version 1776672 (0.0008) [2023-12-27 04:18:37,188][105692] Updated weights for policy 0, policy_version 1772809 (0.0008) [2023-12-27 04:18:37,240][105692] Updated weights for policy 0, policy_version 1772819 (0.0008) [2023-12-27 04:18:37,296][105692] Updated weights for policy 0, policy_version 1772829 (0.0008) [2023-12-27 04:18:37,841][105620] Updated weights for policy 1, policy_version 1776682 (0.0009) [2023-12-27 04:18:37,911][105620] Updated weights for policy 1, policy_version 1776692 (0.0005) [2023-12-27 04:18:37,972][105620] Updated weights for policy 1, policy_version 1776702 (0.0005) [2023-12-27 04:18:38,037][105620] Updated weights for policy 1, policy_version 1776712 (0.0005) [2023-12-27 04:18:38,142][105692] Updated weights for policy 0, policy_version 1772839 (0.0010) [2023-12-27 04:18:38,204][105692] Updated weights for policy 0, policy_version 1772850 (0.0009) [2023-12-27 04:18:38,258][105692] Updated weights for policy 0, policy_version 1772860 (0.0010) [2023-12-27 04:18:38,546][105620] Updated weights for policy 1, policy_version 1776722 (0.0010) [2023-12-27 04:18:38,591][105620] Updated weights for policy 1, policy_version 1776732 (0.0010) [2023-12-27 04:18:38,646][105620] Updated weights for policy 1, policy_version 1776742 (0.0010) [2023-12-27 04:18:39,079][105692] Updated weights for policy 0, policy_version 1772870 (0.0010) [2023-12-27 04:18:39,132][105692] Updated weights for policy 0, policy_version 1772880 (0.0009) [2023-12-27 04:18:39,188][105692] Updated weights for policy 0, policy_version 1772890 (0.0009) [2023-12-27 04:18:39,301][105620] Updated weights for policy 1, policy_version 1776752 (0.0007) [2023-12-27 04:18:39,369][105620] Updated weights for policy 1, policy_version 1776762 (0.0008) [2023-12-27 04:18:39,439][105620] Updated weights for policy 1, policy_version 1776772 (0.0009) [2023-12-27 04:18:40,009][105692] Updated weights for policy 0, policy_version 1772900 (0.0008) [2023-12-27 04:18:40,072][105692] Updated weights for policy 0, policy_version 1772910 (0.0010) [2023-12-27 04:18:40,139][105692] Updated weights for policy 0, policy_version 1772920 (0.0009) [2023-12-27 04:18:40,149][105620] Updated weights for policy 1, policy_version 1776782 (0.0007) [2023-12-27 04:18:40,212][105620] Updated weights for policy 1, policy_version 1776792 (0.0005) [2023-12-27 04:18:40,270][105620] Updated weights for policy 1, policy_version 1776802 (0.0009) [2023-12-27 04:18:40,827][105620] Updated weights for policy 1, policy_version 1776812 (0.0009) [2023-12-27 04:18:40,881][105620] Updated weights for policy 1, policy_version 1776822 (0.0008) [2023-12-27 04:18:40,929][105620] Updated weights for policy 1, policy_version 1776832 (0.0008) [2023-12-27 04:18:40,989][105692] Updated weights for policy 0, policy_version 1772930 (0.0009) [2023-12-27 04:18:41,046][105692] Updated weights for policy 0, policy_version 1772940 (0.0008) [2023-12-27 04:18:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 908869632. Throughput: 0: 9808.1, 1: 9808.0. Samples: 908879588. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:18:41,062][104569] Avg episode reward: [(0, '8713.219'), (1, '8982.501')] [2023-12-27 04:18:41,103][105692] Updated weights for policy 0, policy_version 1772950 (0.0008) [2023-12-27 04:18:41,173][105692] Updated weights for policy 0, policy_version 1772960 (0.0008) [2023-12-27 04:18:41,668][105620] Updated weights for policy 1, policy_version 1776842 (0.0010) [2023-12-27 04:18:41,735][105620] Updated weights for policy 1, policy_version 1776852 (0.0008) [2023-12-27 04:18:41,799][105620] Updated weights for policy 1, policy_version 1776862 (0.0006) [2023-12-27 04:18:41,862][105620] Updated weights for policy 1, policy_version 1776872 (0.0009) [2023-12-27 04:18:41,996][105692] Updated weights for policy 0, policy_version 1772970 (0.0009) [2023-12-27 04:18:42,056][105692] Updated weights for policy 0, policy_version 1772980 (0.0009) [2023-12-27 04:18:42,110][105692] Updated weights for policy 0, policy_version 1772990 (0.0009) [2023-12-27 04:18:42,543][105620] Updated weights for policy 1, policy_version 1776882 (0.0005) [2023-12-27 04:18:42,592][105620] Updated weights for policy 1, policy_version 1776892 (0.0006) [2023-12-27 04:18:42,647][105620] Updated weights for policy 1, policy_version 1776902 (0.0007) [2023-12-27 04:18:42,950][105692] Updated weights for policy 0, policy_version 1773000 (0.0009) [2023-12-27 04:18:43,008][105692] Updated weights for policy 0, policy_version 1773010 (0.0006) [2023-12-27 04:18:43,068][105692] Updated weights for policy 0, policy_version 1773020 (0.0007) [2023-12-27 04:18:43,362][105620] Updated weights for policy 1, policy_version 1776912 (0.0010) [2023-12-27 04:18:43,411][105620] Updated weights for policy 1, policy_version 1776922 (0.0010) [2023-12-27 04:18:43,460][105620] Updated weights for policy 1, policy_version 1776932 (0.0010) [2023-12-27 04:18:43,728][105692] Updated weights for policy 0, policy_version 1773030 (0.0007) [2023-12-27 04:18:43,778][105692] Updated weights for policy 0, policy_version 1773040 (0.0007) [2023-12-27 04:18:43,825][105692] Updated weights for policy 0, policy_version 1773050 (0.0009) [2023-12-27 04:18:44,109][105620] Updated weights for policy 1, policy_version 1776942 (0.0007) [2023-12-27 04:18:44,165][105620] Updated weights for policy 1, policy_version 1776952 (0.0005) [2023-12-27 04:18:44,220][105620] Updated weights for policy 1, policy_version 1776962 (0.0005) [2023-12-27 04:18:44,709][105692] Updated weights for policy 0, policy_version 1773061 (0.0010) [2023-12-27 04:18:44,765][105692] Updated weights for policy 0, policy_version 1773071 (0.0007) [2023-12-27 04:18:44,769][105620] Updated weights for policy 1, policy_version 1776972 (0.0008) [2023-12-27 04:18:44,828][105620] Updated weights for policy 1, policy_version 1776982 (0.0007) [2023-12-27 04:18:44,831][105692] Updated weights for policy 0, policy_version 1773081 (0.0009) [2023-12-27 04:18:44,891][105620] Updated weights for policy 1, policy_version 1776992 (0.0010) [2023-12-27 04:18:45,583][105620] Updated weights for policy 1, policy_version 1777002 (0.0009) [2023-12-27 04:18:45,618][105692] Updated weights for policy 0, policy_version 1773091 (0.0006) [2023-12-27 04:18:45,642][105620] Updated weights for policy 1, policy_version 1777012 (0.0008) [2023-12-27 04:18:45,677][105692] Updated weights for policy 0, policy_version 1773101 (0.0006) [2023-12-27 04:18:45,697][105620] Updated weights for policy 1, policy_version 1777022 (0.0008) [2023-12-27 04:18:45,735][105692] Updated weights for policy 0, policy_version 1773111 (0.0007) [2023-12-27 04:18:45,759][105620] Updated weights for policy 1, policy_version 1777032 (0.0008) [2023-12-27 04:18:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 908967936. Throughput: 0: 9667.0, 1: 9889.7. Samples: 908936212. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:18:46,063][104569] Avg episode reward: [(0, '8444.905'), (1, '8982.513')] [2023-12-27 04:18:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001773120_453984256.pth... [2023-12-27 04:18:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001777032_454983680.pth... [2023-12-27 04:18:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001772000_453697536.pth [2023-12-27 04:18:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001775848_454680576.pth [2023-12-27 04:18:46,391][105620] Updated weights for policy 1, policy_version 1777042 (0.0006) [2023-12-27 04:18:46,456][105620] Updated weights for policy 1, policy_version 1777052 (0.0010) [2023-12-27 04:18:46,505][105692] Updated weights for policy 0, policy_version 1773121 (0.0006) [2023-12-27 04:18:46,515][105620] Updated weights for policy 1, policy_version 1777062 (0.0010) [2023-12-27 04:18:46,571][105692] Updated weights for policy 0, policy_version 1773131 (0.0007) [2023-12-27 04:18:46,638][105692] Updated weights for policy 0, policy_version 1773141 (0.0010) [2023-12-27 04:18:46,700][105692] Updated weights for policy 0, policy_version 1773151 (0.0009) [2023-12-27 04:18:47,165][105620] Updated weights for policy 1, policy_version 1777072 (0.0011) [2023-12-27 04:18:47,229][105620] Updated weights for policy 1, policy_version 1777082 (0.0011) [2023-12-27 04:18:47,278][105620] Updated weights for policy 1, policy_version 1777092 (0.0010) [2023-12-27 04:18:47,421][105692] Updated weights for policy 0, policy_version 1773161 (0.0009) [2023-12-27 04:18:47,479][105692] Updated weights for policy 0, policy_version 1773171 (0.0009) [2023-12-27 04:18:47,545][105692] Updated weights for policy 0, policy_version 1773181 (0.0009) [2023-12-27 04:18:47,903][105620] Updated weights for policy 1, policy_version 1777102 (0.0010) [2023-12-27 04:18:47,954][105620] Updated weights for policy 1, policy_version 1777112 (0.0010) [2023-12-27 04:18:48,008][105620] Updated weights for policy 1, policy_version 1777122 (0.0010) [2023-12-27 04:18:48,295][105692] Updated weights for policy 0, policy_version 1773191 (0.0007) [2023-12-27 04:18:48,347][105692] Updated weights for policy 0, policy_version 1773201 (0.0006) [2023-12-27 04:18:48,400][105692] Updated weights for policy 0, policy_version 1773211 (0.0008) [2023-12-27 04:18:48,689][105620] Updated weights for policy 1, policy_version 1777132 (0.0008) [2023-12-27 04:18:48,752][105620] Updated weights for policy 1, policy_version 1777142 (0.0006) [2023-12-27 04:18:48,816][105620] Updated weights for policy 1, policy_version 1777152 (0.0011) [2023-12-27 04:18:49,205][105692] Updated weights for policy 0, policy_version 1773221 (0.0008) [2023-12-27 04:18:49,274][105692] Updated weights for policy 0, policy_version 1773231 (0.0009) [2023-12-27 04:18:49,342][105692] Updated weights for policy 0, policy_version 1773241 (0.0009) [2023-12-27 04:18:49,495][105620] Updated weights for policy 1, policy_version 1777162 (0.0010) [2023-12-27 04:18:49,554][105620] Updated weights for policy 1, policy_version 1777172 (0.0005) [2023-12-27 04:18:49,599][105620] Updated weights for policy 1, policy_version 1777182 (0.0009) [2023-12-27 04:18:49,643][105620] Updated weights for policy 1, policy_version 1777192 (0.0005) [2023-12-27 04:18:50,194][105692] Updated weights for policy 0, policy_version 1773251 (0.0010) [2023-12-27 04:18:50,258][105692] Updated weights for policy 0, policy_version 1773261 (0.0011) [2023-12-27 04:18:50,296][105620] Updated weights for policy 1, policy_version 1777202 (0.0010) [2023-12-27 04:18:50,319][105692] Updated weights for policy 0, policy_version 1773271 (0.0011) [2023-12-27 04:18:50,363][105620] Updated weights for policy 1, policy_version 1777212 (0.0011) [2023-12-27 04:18:50,420][105620] Updated weights for policy 1, policy_version 1777222 (0.0010) [2023-12-27 04:18:50,978][105692] Updated weights for policy 0, policy_version 1773281 (0.0008) [2023-12-27 04:18:51,040][105692] Updated weights for policy 0, policy_version 1773291 (0.0006) [2023-12-27 04:18:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 909058048. Throughput: 0: 9607.5, 1: 10018.9. Samples: 909053768. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:18:51,062][104569] Avg episode reward: [(0, '8262.618'), (1, '9074.770')] [2023-12-27 04:18:51,101][105692] Updated weights for policy 0, policy_version 1773301 (0.0010) [2023-12-27 04:18:51,162][105692] Updated weights for policy 0, policy_version 1773311 (0.0011) [2023-12-27 04:18:51,166][105620] Updated weights for policy 1, policy_version 1777232 (0.0011) [2023-12-27 04:18:51,238][105620] Updated weights for policy 1, policy_version 1777242 (0.0011) [2023-12-27 04:18:51,302][105620] Updated weights for policy 1, policy_version 1777252 (0.0010) [2023-12-27 04:18:51,861][105692] Updated weights for policy 0, policy_version 1773321 (0.0006) [2023-12-27 04:18:51,925][105692] Updated weights for policy 0, policy_version 1773331 (0.0005) [2023-12-27 04:18:51,987][105692] Updated weights for policy 0, policy_version 1773341 (0.0005) [2023-12-27 04:18:52,077][105620] Updated weights for policy 1, policy_version 1777262 (0.0011) [2023-12-27 04:18:52,125][105620] Updated weights for policy 1, policy_version 1777272 (0.0010) [2023-12-27 04:18:52,170][105620] Updated weights for policy 1, policy_version 1777282 (0.0010) [2023-12-27 04:18:52,622][105692] Updated weights for policy 0, policy_version 1773351 (0.0008) [2023-12-27 04:18:52,680][105692] Updated weights for policy 0, policy_version 1773361 (0.0008) [2023-12-27 04:18:52,740][105692] Updated weights for policy 0, policy_version 1773371 (0.0008) [2023-12-27 04:18:52,960][105620] Updated weights for policy 1, policy_version 1777292 (0.0010) [2023-12-27 04:18:53,004][105620] Updated weights for policy 1, policy_version 1777302 (0.0010) [2023-12-27 04:18:53,060][105620] Updated weights for policy 1, policy_version 1777312 (0.0009) [2023-12-27 04:18:53,449][105692] Updated weights for policy 0, policy_version 1773381 (0.0008) [2023-12-27 04:18:53,510][105692] Updated weights for policy 0, policy_version 1773391 (0.0008) [2023-12-27 04:18:53,565][105692] Updated weights for policy 0, policy_version 1773401 (0.0010) [2023-12-27 04:18:53,781][105620] Updated weights for policy 1, policy_version 1777322 (0.0009) [2023-12-27 04:18:53,841][105620] Updated weights for policy 1, policy_version 1777332 (0.0005) [2023-12-27 04:18:53,899][105620] Updated weights for policy 1, policy_version 1777342 (0.0008) [2023-12-27 04:18:53,947][105620] Updated weights for policy 1, policy_version 1777352 (0.0010) [2023-12-27 04:18:54,221][105692] Updated weights for policy 0, policy_version 1773411 (0.0009) [2023-12-27 04:18:54,270][105692] Updated weights for policy 0, policy_version 1773421 (0.0005) [2023-12-27 04:18:54,330][105692] Updated weights for policy 0, policy_version 1773431 (0.0005) [2023-12-27 04:18:54,659][105620] Updated weights for policy 1, policy_version 1777362 (0.0010) [2023-12-27 04:18:54,716][105620] Updated weights for policy 1, policy_version 1777372 (0.0010) [2023-12-27 04:18:54,774][105620] Updated weights for policy 1, policy_version 1777382 (0.0010) [2023-12-27 04:18:54,904][105692] Updated weights for policy 0, policy_version 1773441 (0.0005) [2023-12-27 04:18:54,956][105692] Updated weights for policy 0, policy_version 1773451 (0.0006) [2023-12-27 04:18:55,019][105692] Updated weights for policy 0, policy_version 1773461 (0.0006) [2023-12-27 04:18:55,068][105692] Updated weights for policy 0, policy_version 1773471 (0.0006) [2023-12-27 04:18:55,505][105620] Updated weights for policy 1, policy_version 1777392 (0.0010) [2023-12-27 04:18:55,562][105620] Updated weights for policy 1, policy_version 1777402 (0.0010) [2023-12-27 04:18:55,621][105620] Updated weights for policy 1, policy_version 1777412 (0.0010) [2023-12-27 04:18:55,631][105692] Updated weights for policy 0, policy_version 1773481 (0.0007) [2023-12-27 04:18:55,678][105692] Updated weights for policy 0, policy_version 1773491 (0.0008) [2023-12-27 04:18:55,729][105692] Updated weights for policy 0, policy_version 1773501 (0.0008) [2023-12-27 04:18:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 909164544. Throughput: 0: 9710.6, 1: 9916.1. Samples: 909172576. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:18:56,063][104569] Avg episode reward: [(0, '8084.877'), (1, '9167.336')] [2023-12-27 04:18:56,297][105692] Updated weights for policy 0, policy_version 1773511 (0.0007) [2023-12-27 04:18:56,346][105692] Updated weights for policy 0, policy_version 1773521 (0.0005) [2023-12-27 04:18:56,370][105620] Updated weights for policy 1, policy_version 1777422 (0.0010) [2023-12-27 04:18:56,397][105692] Updated weights for policy 0, policy_version 1773531 (0.0005) [2023-12-27 04:18:56,428][105620] Updated weights for policy 1, policy_version 1777432 (0.0010) [2023-12-27 04:18:56,489][105620] Updated weights for policy 1, policy_version 1777442 (0.0010) [2023-12-27 04:18:57,046][105692] Updated weights for policy 0, policy_version 1773541 (0.0006) [2023-12-27 04:18:57,092][105692] Updated weights for policy 0, policy_version 1773551 (0.0005) [2023-12-27 04:18:57,143][105692] Updated weights for policy 0, policy_version 1773561 (0.0005) [2023-12-27 04:18:57,195][105620] Updated weights for policy 1, policy_version 1777452 (0.0008) [2023-12-27 04:18:57,243][105620] Updated weights for policy 1, policy_version 1777462 (0.0005) [2023-12-27 04:18:57,289][105620] Updated weights for policy 1, policy_version 1777472 (0.0005) [2023-12-27 04:18:57,718][105692] Updated weights for policy 0, policy_version 1773571 (0.0007) [2023-12-27 04:18:57,781][105692] Updated weights for policy 0, policy_version 1773581 (0.0007) [2023-12-27 04:18:57,845][105692] Updated weights for policy 0, policy_version 1773591 (0.0006) [2023-12-27 04:18:57,865][105620] Updated weights for policy 1, policy_version 1777482 (0.0006) [2023-12-27 04:18:57,919][105620] Updated weights for policy 1, policy_version 1777493 (0.0009) [2023-12-27 04:18:57,972][105620] Updated weights for policy 1, policy_version 1777503 (0.0010) [2023-12-27 04:18:58,465][105692] Updated weights for policy 0, policy_version 1773601 (0.0006) [2023-12-27 04:18:58,526][105692] Updated weights for policy 0, policy_version 1773611 (0.0011) [2023-12-27 04:18:58,575][105692] Updated weights for policy 0, policy_version 1773621 (0.0010) [2023-12-27 04:18:58,633][105692] Updated weights for policy 0, policy_version 1773631 (0.0010) [2023-12-27 04:18:58,918][105620] Updated weights for policy 1, policy_version 1777514 (0.0010) [2023-12-27 04:18:58,978][105620] Updated weights for policy 1, policy_version 1777524 (0.0010) [2023-12-27 04:18:59,033][105620] Updated weights for policy 1, policy_version 1777534 (0.0009) [2023-12-27 04:18:59,094][105620] Updated weights for policy 1, policy_version 1777544 (0.0009) [2023-12-27 04:18:59,420][105692] Updated weights for policy 0, policy_version 1773641 (0.0010) [2023-12-27 04:18:59,478][105692] Updated weights for policy 0, policy_version 1773651 (0.0010) [2023-12-27 04:18:59,536][105692] Updated weights for policy 0, policy_version 1773661 (0.0010) [2023-12-27 04:18:59,813][105620] Updated weights for policy 1, policy_version 1777554 (0.0009) [2023-12-27 04:18:59,876][105620] Updated weights for policy 1, policy_version 1777564 (0.0010) [2023-12-27 04:18:59,938][105620] Updated weights for policy 1, policy_version 1777574 (0.0010) [2023-12-27 04:19:00,325][105692] Updated weights for policy 0, policy_version 1773671 (0.0009) [2023-12-27 04:19:00,375][105692] Updated weights for policy 0, policy_version 1773681 (0.0008) [2023-12-27 04:19:00,436][105692] Updated weights for policy 0, policy_version 1773691 (0.0009) [2023-12-27 04:19:00,650][105620] Updated weights for policy 1, policy_version 1777584 (0.0008) [2023-12-27 04:19:00,710][105620] Updated weights for policy 1, policy_version 1777594 (0.0009) [2023-12-27 04:19:00,756][105620] Updated weights for policy 1, policy_version 1777604 (0.0008) [2023-12-27 04:19:01,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 909262848. Throughput: 0: 9815.7, 1: 9959.8. Samples: 909235244. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:19:01,062][104569] Avg episode reward: [(0, '8723.377'), (1, '9260.026')] [2023-12-27 04:19:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001773696_454131712.pth... [2023-12-27 04:19:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001777608_455131136.pth... [2023-12-27 04:19:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001772576_453844992.pth [2023-12-27 04:19:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001776424_454828032.pth [2023-12-27 04:19:01,196][105692] Updated weights for policy 0, policy_version 1773701 (0.0007) [2023-12-27 04:19:01,254][105692] Updated weights for policy 0, policy_version 1773711 (0.0006) [2023-12-27 04:19:01,316][105692] Updated weights for policy 0, policy_version 1773721 (0.0008) [2023-12-27 04:19:01,541][105620] Updated weights for policy 1, policy_version 1777614 (0.0007) [2023-12-27 04:19:01,601][105620] Updated weights for policy 1, policy_version 1777624 (0.0009) [2023-12-27 04:19:01,672][105620] Updated weights for policy 1, policy_version 1777634 (0.0009) [2023-12-27 04:19:02,017][105692] Updated weights for policy 0, policy_version 1773731 (0.0008) [2023-12-27 04:19:02,081][105692] Updated weights for policy 0, policy_version 1773741 (0.0008) [2023-12-27 04:19:02,139][105692] Updated weights for policy 0, policy_version 1773751 (0.0009) [2023-12-27 04:19:02,404][105620] Updated weights for policy 1, policy_version 1777644 (0.0010) [2023-12-27 04:19:02,462][105620] Updated weights for policy 1, policy_version 1777654 (0.0011) [2023-12-27 04:19:02,521][105620] Updated weights for policy 1, policy_version 1777664 (0.0011) [2023-12-27 04:19:02,908][105692] Updated weights for policy 0, policy_version 1773761 (0.0009) [2023-12-27 04:19:02,973][105692] Updated weights for policy 0, policy_version 1773771 (0.0009) [2023-12-27 04:19:03,029][105692] Updated weights for policy 0, policy_version 1773781 (0.0008) [2023-12-27 04:19:03,088][105692] Updated weights for policy 0, policy_version 1773791 (0.0009) [2023-12-27 04:19:03,250][105620] Updated weights for policy 1, policy_version 1777674 (0.0010) [2023-12-27 04:19:03,301][105620] Updated weights for policy 1, policy_version 1777684 (0.0010) [2023-12-27 04:19:03,344][105620] Updated weights for policy 1, policy_version 1777694 (0.0010) [2023-12-27 04:19:03,388][105620] Updated weights for policy 1, policy_version 1777704 (0.0010) [2023-12-27 04:19:03,842][105692] Updated weights for policy 0, policy_version 1773801 (0.0008) [2023-12-27 04:19:03,906][105692] Updated weights for policy 0, policy_version 1773811 (0.0008) [2023-12-27 04:19:03,959][105692] Updated weights for policy 0, policy_version 1773821 (0.0008) [2023-12-27 04:19:04,146][105620] Updated weights for policy 1, policy_version 1777714 (0.0011) [2023-12-27 04:19:04,195][105620] Updated weights for policy 1, policy_version 1777724 (0.0010) [2023-12-27 04:19:04,250][105620] Updated weights for policy 1, policy_version 1777734 (0.0010) [2023-12-27 04:19:04,756][105692] Updated weights for policy 0, policy_version 1773831 (0.0008) [2023-12-27 04:19:04,815][105692] Updated weights for policy 0, policy_version 1773841 (0.0008) [2023-12-27 04:19:04,870][105692] Updated weights for policy 0, policy_version 1773851 (0.0008) [2023-12-27 04:19:05,021][105620] Updated weights for policy 1, policy_version 1777744 (0.0011) [2023-12-27 04:19:05,079][105620] Updated weights for policy 1, policy_version 1777754 (0.0010) [2023-12-27 04:19:05,136][105620] Updated weights for policy 1, policy_version 1777764 (0.0010) [2023-12-27 04:19:05,612][105692] Updated weights for policy 0, policy_version 1773861 (0.0007) [2023-12-27 04:19:05,667][105692] Updated weights for policy 0, policy_version 1773871 (0.0006) [2023-12-27 04:19:05,718][105692] Updated weights for policy 0, policy_version 1773881 (0.0005) [2023-12-27 04:19:05,877][105620] Updated weights for policy 1, policy_version 1777774 (0.0010) [2023-12-27 04:19:05,936][105620] Updated weights for policy 1, policy_version 1777784 (0.0010) [2023-12-27 04:19:06,002][105620] Updated weights for policy 1, policy_version 1777794 (0.0010) [2023-12-27 04:19:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 909361152. Throughput: 0: 9722.7, 1: 9973.4. Samples: 909347104. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:19:06,062][104569] Avg episode reward: [(0, '8897.287'), (1, '9352.694')] [2023-12-27 04:19:06,420][105692] Updated weights for policy 0, policy_version 1773891 (0.0007) [2023-12-27 04:19:06,486][105692] Updated weights for policy 0, policy_version 1773901 (0.0009) [2023-12-27 04:19:06,547][105692] Updated weights for policy 0, policy_version 1773911 (0.0009) [2023-12-27 04:19:06,748][105620] Updated weights for policy 1, policy_version 1777804 (0.0010) [2023-12-27 04:19:06,809][105620] Updated weights for policy 1, policy_version 1777814 (0.0008) [2023-12-27 04:19:06,863][105620] Updated weights for policy 1, policy_version 1777824 (0.0009) [2023-12-27 04:19:07,292][105692] Updated weights for policy 0, policy_version 1773921 (0.0009) [2023-12-27 04:19:07,343][105692] Updated weights for policy 0, policy_version 1773931 (0.0009) [2023-12-27 04:19:07,394][105692] Updated weights for policy 0, policy_version 1773941 (0.0009) [2023-12-27 04:19:07,451][105692] Updated weights for policy 0, policy_version 1773951 (0.0009) [2023-12-27 04:19:07,600][105620] Updated weights for policy 1, policy_version 1777834 (0.0008) [2023-12-27 04:19:07,660][105620] Updated weights for policy 1, policy_version 1777844 (0.0007) [2023-12-27 04:19:07,722][105620] Updated weights for policy 1, policy_version 1777854 (0.0006) [2023-12-27 04:19:07,780][105620] Updated weights for policy 1, policy_version 1777864 (0.0005) [2023-12-27 04:19:08,277][105692] Updated weights for policy 0, policy_version 1773961 (0.0006) [2023-12-27 04:19:08,347][105692] Updated weights for policy 0, policy_version 1773971 (0.0007) [2023-12-27 04:19:08,410][105692] Updated weights for policy 0, policy_version 1773981 (0.0009) [2023-12-27 04:19:08,479][105620] Updated weights for policy 1, policy_version 1777874 (0.0009) [2023-12-27 04:19:08,548][105620] Updated weights for policy 1, policy_version 1777884 (0.0007) [2023-12-27 04:19:08,614][105620] Updated weights for policy 1, policy_version 1777894 (0.0005) [2023-12-27 04:19:09,165][105620] Updated weights for policy 1, policy_version 1777904 (0.0006) [2023-12-27 04:19:09,225][105620] Updated weights for policy 1, policy_version 1777914 (0.0008) [2023-12-27 04:19:09,244][105692] Updated weights for policy 0, policy_version 1773991 (0.0009) [2023-12-27 04:19:09,286][105620] Updated weights for policy 1, policy_version 1777924 (0.0007) [2023-12-27 04:19:09,308][105692] Updated weights for policy 0, policy_version 1774001 (0.0010) [2023-12-27 04:19:09,373][105692] Updated weights for policy 0, policy_version 1774011 (0.0008) [2023-12-27 04:19:09,946][105620] Updated weights for policy 1, policy_version 1777934 (0.0008) [2023-12-27 04:19:10,017][105620] Updated weights for policy 1, policy_version 1777944 (0.0008) [2023-12-27 04:19:10,081][105620] Updated weights for policy 1, policy_version 1777954 (0.0007) [2023-12-27 04:19:10,179][105692] Updated weights for policy 0, policy_version 1774021 (0.0010) [2023-12-27 04:19:10,232][105692] Updated weights for policy 0, policy_version 1774031 (0.0010) [2023-12-27 04:19:10,295][105692] Updated weights for policy 0, policy_version 1774041 (0.0011) [2023-12-27 04:19:10,790][105620] Updated weights for policy 1, policy_version 1777964 (0.0008) [2023-12-27 04:19:10,862][105620] Updated weights for policy 1, policy_version 1777974 (0.0010) [2023-12-27 04:19:10,925][105620] Updated weights for policy 1, policy_version 1777984 (0.0010) [2023-12-27 04:19:10,963][105692] Updated weights for policy 0, policy_version 1774051 (0.0009) [2023-12-27 04:19:11,023][105692] Updated weights for policy 0, policy_version 1774061 (0.0006) [2023-12-27 04:19:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 909451264. Throughput: 0: 9628.1, 1: 9997.3. Samples: 909460576. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:19:11,062][104569] Avg episode reward: [(0, '8446.820'), (1, '9167.644')] [2023-12-27 04:19:11,088][105692] Updated weights for policy 0, policy_version 1774071 (0.0007) [2023-12-27 04:19:11,694][105620] Updated weights for policy 1, policy_version 1777994 (0.0011) [2023-12-27 04:19:11,707][105692] Updated weights for policy 0, policy_version 1774081 (0.0007) [2023-12-27 04:19:11,760][105620] Updated weights for policy 1, policy_version 1778004 (0.0007) [2023-12-27 04:19:11,769][105692] Updated weights for policy 0, policy_version 1774091 (0.0011) [2023-12-27 04:19:11,816][105620] Updated weights for policy 1, policy_version 1778014 (0.0006) [2023-12-27 04:19:11,831][105692] Updated weights for policy 0, policy_version 1774101 (0.0012) [2023-12-27 04:19:11,880][105620] Updated weights for policy 1, policy_version 1778024 (0.0006) [2023-12-27 04:19:11,892][105692] Updated weights for policy 0, policy_version 1774111 (0.0011) [2023-12-27 04:19:12,476][105620] Updated weights for policy 1, policy_version 1778034 (0.0008) [2023-12-27 04:19:12,533][105620] Updated weights for policy 1, policy_version 1778044 (0.0008) [2023-12-27 04:19:12,581][105620] Updated weights for policy 1, policy_version 1778054 (0.0008) [2023-12-27 04:19:12,648][105692] Updated weights for policy 0, policy_version 1774121 (0.0011) [2023-12-27 04:19:12,707][105692] Updated weights for policy 0, policy_version 1774131 (0.0010) [2023-12-27 04:19:12,762][105692] Updated weights for policy 0, policy_version 1774141 (0.0008) [2023-12-27 04:19:13,309][105620] Updated weights for policy 1, policy_version 1778064 (0.0007) [2023-12-27 04:19:13,368][105620] Updated weights for policy 1, policy_version 1778074 (0.0010) [2023-12-27 04:19:13,430][105620] Updated weights for policy 1, policy_version 1778084 (0.0010) [2023-12-27 04:19:13,545][105692] Updated weights for policy 0, policy_version 1774151 (0.0009) [2023-12-27 04:19:13,599][105692] Updated weights for policy 0, policy_version 1774161 (0.0008) [2023-12-27 04:19:13,654][105692] Updated weights for policy 0, policy_version 1774171 (0.0008) [2023-12-27 04:19:14,164][105620] Updated weights for policy 1, policy_version 1778094 (0.0010) [2023-12-27 04:19:14,220][105620] Updated weights for policy 1, policy_version 1778104 (0.0010) [2023-12-27 04:19:14,274][105620] Updated weights for policy 1, policy_version 1778114 (0.0010) [2023-12-27 04:19:14,427][105692] Updated weights for policy 0, policy_version 1774181 (0.0008) [2023-12-27 04:19:14,479][105692] Updated weights for policy 0, policy_version 1774191 (0.0008) [2023-12-27 04:19:14,530][105692] Updated weights for policy 0, policy_version 1774201 (0.0008) [2023-12-27 04:19:15,025][105620] Updated weights for policy 1, policy_version 1778124 (0.0010) [2023-12-27 04:19:15,084][105620] Updated weights for policy 1, policy_version 1778134 (0.0010) [2023-12-27 04:19:15,147][105620] Updated weights for policy 1, policy_version 1778144 (0.0010) [2023-12-27 04:19:15,311][105692] Updated weights for policy 0, policy_version 1774211 (0.0008) [2023-12-27 04:19:15,368][105692] Updated weights for policy 0, policy_version 1774221 (0.0008) [2023-12-27 04:19:15,416][105692] Updated weights for policy 0, policy_version 1774231 (0.0008) [2023-12-27 04:19:15,896][105620] Updated weights for policy 1, policy_version 1778154 (0.0011) [2023-12-27 04:19:15,947][105620] Updated weights for policy 1, policy_version 1778164 (0.0010) [2023-12-27 04:19:15,992][105620] Updated weights for policy 1, policy_version 1778174 (0.0010) [2023-12-27 04:19:16,040][105620] Updated weights for policy 1, policy_version 1778184 (0.0010) [2023-12-27 04:19:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 909549568. Throughput: 0: 9525.3, 1: 9938.2. Samples: 909519044. Policy #0 lag: (min: 18.0, avg: 26.7, max: 50.0) [2023-12-27 04:19:16,062][104569] Avg episode reward: [(0, '8447.568'), (1, '9075.564')] [2023-12-27 04:19:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001774240_454270976.pth... [2023-12-27 04:19:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001778184_455278592.pth... [2023-12-27 04:19:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001773120_453984256.pth [2023-12-27 04:19:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001777032_454983680.pth [2023-12-27 04:19:16,191][105692] Updated weights for policy 0, policy_version 1774241 (0.0008) [2023-12-27 04:19:16,251][105692] Updated weights for policy 0, policy_version 1774251 (0.0008) [2023-12-27 04:19:16,295][105692] Updated weights for policy 0, policy_version 1774261 (0.0008) [2023-12-27 04:19:16,343][105692] Updated weights for policy 0, policy_version 1774271 (0.0008) [2023-12-27 04:19:16,809][105620] Updated weights for policy 1, policy_version 1778194 (0.0010) [2023-12-27 04:19:16,857][105620] Updated weights for policy 1, policy_version 1778204 (0.0010) [2023-12-27 04:19:16,905][105620] Updated weights for policy 1, policy_version 1778214 (0.0010) [2023-12-27 04:19:17,126][105692] Updated weights for policy 0, policy_version 1774281 (0.0010) [2023-12-27 04:19:17,178][105692] Updated weights for policy 0, policy_version 1774291 (0.0011) [2023-12-27 04:19:17,226][105692] Updated weights for policy 0, policy_version 1774301 (0.0010) [2023-12-27 04:19:17,641][105620] Updated weights for policy 1, policy_version 1778224 (0.0009) [2023-12-27 04:19:17,705][105620] Updated weights for policy 1, policy_version 1778234 (0.0005) [2023-12-27 04:19:17,765][105620] Updated weights for policy 1, policy_version 1778244 (0.0005) [2023-12-27 04:19:17,864][105692] Updated weights for policy 0, policy_version 1774311 (0.0007) [2023-12-27 04:19:17,929][105692] Updated weights for policy 0, policy_version 1774321 (0.0007) [2023-12-27 04:19:17,982][105692] Updated weights for policy 0, policy_version 1774331 (0.0006) [2023-12-27 04:19:18,465][105620] Updated weights for policy 1, policy_version 1778254 (0.0006) [2023-12-27 04:19:18,535][105620] Updated weights for policy 1, policy_version 1778264 (0.0009) [2023-12-27 04:19:18,602][105620] Updated weights for policy 1, policy_version 1778274 (0.0009) [2023-12-27 04:19:18,610][105692] Updated weights for policy 0, policy_version 1774341 (0.0008) [2023-12-27 04:19:18,662][105692] Updated weights for policy 0, policy_version 1774351 (0.0011) [2023-12-27 04:19:18,720][105692] Updated weights for policy 0, policy_version 1774361 (0.0011) [2023-12-27 04:19:19,330][105620] Updated weights for policy 1, policy_version 1778284 (0.0007) [2023-12-27 04:19:19,396][105620] Updated weights for policy 1, policy_version 1778294 (0.0007) [2023-12-27 04:19:19,460][105620] Updated weights for policy 1, policy_version 1778304 (0.0007) [2023-12-27 04:19:19,519][105692] Updated weights for policy 0, policy_version 1774371 (0.0009) [2023-12-27 04:19:19,584][105692] Updated weights for policy 0, policy_version 1774381 (0.0009) [2023-12-27 04:19:19,641][105692] Updated weights for policy 0, policy_version 1774391 (0.0008) [2023-12-27 04:19:20,192][105620] Updated weights for policy 1, policy_version 1778314 (0.0008) [2023-12-27 04:19:20,251][105620] Updated weights for policy 1, policy_version 1778324 (0.0009) [2023-12-27 04:19:20,311][105620] Updated weights for policy 1, policy_version 1778334 (0.0008) [2023-12-27 04:19:20,376][105620] Updated weights for policy 1, policy_version 1778344 (0.0008) [2023-12-27 04:19:20,444][105692] Updated weights for policy 0, policy_version 1774401 (0.0008) [2023-12-27 04:19:20,502][105692] Updated weights for policy 0, policy_version 1774411 (0.0009) [2023-12-27 04:19:20,559][105692] Updated weights for policy 0, policy_version 1774421 (0.0010) [2023-12-27 04:19:20,616][105692] Updated weights for policy 0, policy_version 1774431 (0.0009) [2023-12-27 04:19:21,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 909639680. Throughput: 0: 9432.9, 1: 9921.0. Samples: 909633096. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:19:21,063][104569] Avg episode reward: [(0, '8532.837'), (1, '9260.598')] [2023-12-27 04:19:21,173][105620] Updated weights for policy 1, policy_version 1778354 (0.0009) [2023-12-27 04:19:21,238][105620] Updated weights for policy 1, policy_version 1778364 (0.0009) [2023-12-27 04:19:21,302][105620] Updated weights for policy 1, policy_version 1778374 (0.0007) [2023-12-27 04:19:21,313][105692] Updated weights for policy 0, policy_version 1774441 (0.0008) [2023-12-27 04:19:21,379][105692] Updated weights for policy 0, policy_version 1774451 (0.0009) [2023-12-27 04:19:21,445][105692] Updated weights for policy 0, policy_version 1774461 (0.0008) [2023-12-27 04:19:22,084][105620] Updated weights for policy 1, policy_version 1778384 (0.0010) [2023-12-27 04:19:22,127][105692] Updated weights for policy 0, policy_version 1774471 (0.0008) [2023-12-27 04:19:22,144][105620] Updated weights for policy 1, policy_version 1778394 (0.0010) [2023-12-27 04:19:22,185][105692] Updated weights for policy 0, policy_version 1774481 (0.0006) [2023-12-27 04:19:22,207][105620] Updated weights for policy 1, policy_version 1778404 (0.0011) [2023-12-27 04:19:22,244][105692] Updated weights for policy 0, policy_version 1774491 (0.0005) [2023-12-27 04:19:22,950][105692] Updated weights for policy 0, policy_version 1774501 (0.0006) [2023-12-27 04:19:22,967][105620] Updated weights for policy 1, policy_version 1778414 (0.0011) [2023-12-27 04:19:23,000][105692] Updated weights for policy 0, policy_version 1774511 (0.0009) [2023-12-27 04:19:23,026][105620] Updated weights for policy 1, policy_version 1778424 (0.0010) [2023-12-27 04:19:23,048][105692] Updated weights for policy 0, policy_version 1774521 (0.0005) [2023-12-27 04:19:23,089][105620] Updated weights for policy 1, policy_version 1778434 (0.0011) [2023-12-27 04:19:23,717][105692] Updated weights for policy 0, policy_version 1774531 (0.0006) [2023-12-27 04:19:23,773][105692] Updated weights for policy 0, policy_version 1774541 (0.0005) [2023-12-27 04:19:23,804][105620] Updated weights for policy 1, policy_version 1778444 (0.0011) [2023-12-27 04:19:23,824][105692] Updated weights for policy 0, policy_version 1774551 (0.0005) [2023-12-27 04:19:23,860][105620] Updated weights for policy 1, policy_version 1778454 (0.0010) [2023-12-27 04:19:23,919][105620] Updated weights for policy 1, policy_version 1778464 (0.0010) [2023-12-27 04:19:24,383][105692] Updated weights for policy 0, policy_version 1774561 (0.0005) [2023-12-27 04:19:24,442][105692] Updated weights for policy 0, policy_version 1774571 (0.0006) [2023-12-27 04:19:24,497][105692] Updated weights for policy 0, policy_version 1774581 (0.0005) [2023-12-27 04:19:24,554][105692] Updated weights for policy 0, policy_version 1774591 (0.0005) [2023-12-27 04:19:24,680][105620] Updated weights for policy 1, policy_version 1778474 (0.0010) [2023-12-27 04:19:24,738][105620] Updated weights for policy 1, policy_version 1778484 (0.0011) [2023-12-27 04:19:24,797][105620] Updated weights for policy 1, policy_version 1778494 (0.0010) [2023-12-27 04:19:24,855][105620] Updated weights for policy 1, policy_version 1778504 (0.0010) [2023-12-27 04:19:25,083][105692] Updated weights for policy 0, policy_version 1774601 (0.0007) [2023-12-27 04:19:25,149][105692] Updated weights for policy 0, policy_version 1774611 (0.0008) [2023-12-27 04:19:25,212][105692] Updated weights for policy 0, policy_version 1774621 (0.0008) [2023-12-27 04:19:25,532][105620] Updated weights for policy 1, policy_version 1778514 (0.0010) [2023-12-27 04:19:25,587][105620] Updated weights for policy 1, policy_version 1778524 (0.0010) [2023-12-27 04:19:25,649][105620] Updated weights for policy 1, policy_version 1778534 (0.0010) [2023-12-27 04:19:25,936][105692] Updated weights for policy 0, policy_version 1774631 (0.0006) [2023-12-27 04:19:25,986][105692] Updated weights for policy 0, policy_version 1774641 (0.0008) [2023-12-27 04:19:26,038][105692] Updated weights for policy 0, policy_version 1774651 (0.0010) [2023-12-27 04:19:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 909737984. Throughput: 0: 9652.9, 1: 9704.7. Samples: 909750680. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:19:26,062][104569] Avg episode reward: [(0, '8533.367'), (1, '9167.907')] [2023-12-27 04:19:26,259][105620] Updated weights for policy 1, policy_version 1778544 (0.0006) [2023-12-27 04:19:26,308][105620] Updated weights for policy 1, policy_version 1778554 (0.0010) [2023-12-27 04:19:26,363][105620] Updated weights for policy 1, policy_version 1778564 (0.0010) [2023-12-27 04:19:26,876][105692] Updated weights for policy 0, policy_version 1774661 (0.0010) [2023-12-27 04:19:26,929][105692] Updated weights for policy 0, policy_version 1774672 (0.0010) [2023-12-27 04:19:26,979][105692] Updated weights for policy 0, policy_version 1774683 (0.0009) [2023-12-27 04:19:26,992][105620] Updated weights for policy 1, policy_version 1778574 (0.0007) [2023-12-27 04:19:27,047][105620] Updated weights for policy 1, policy_version 1778584 (0.0005) [2023-12-27 04:19:27,097][105620] Updated weights for policy 1, policy_version 1778594 (0.0005) [2023-12-27 04:19:27,631][105620] Updated weights for policy 1, policy_version 1778604 (0.0005) [2023-12-27 04:19:27,697][105620] Updated weights for policy 1, policy_version 1778614 (0.0008) [2023-12-27 04:19:27,715][105692] Updated weights for policy 0, policy_version 1774693 (0.0007) [2023-12-27 04:19:27,755][105620] Updated weights for policy 1, policy_version 1778624 (0.0007) [2023-12-27 04:19:27,770][105692] Updated weights for policy 0, policy_version 1774703 (0.0009) [2023-12-27 04:19:27,823][105692] Updated weights for policy 0, policy_version 1774713 (0.0007) [2023-12-27 04:19:28,438][105620] Updated weights for policy 1, policy_version 1778634 (0.0009) [2023-12-27 04:19:28,450][105692] Updated weights for policy 0, policy_version 1774723 (0.0007) [2023-12-27 04:19:28,499][105692] Updated weights for policy 0, policy_version 1774733 (0.0005) [2023-12-27 04:19:28,500][105620] Updated weights for policy 1, policy_version 1778644 (0.0011) [2023-12-27 04:19:28,560][105692] Updated weights for policy 0, policy_version 1774743 (0.0005) [2023-12-27 04:19:28,562][105620] Updated weights for policy 1, policy_version 1778654 (0.0011) [2023-12-27 04:19:28,621][105620] Updated weights for policy 1, policy_version 1778664 (0.0011) [2023-12-27 04:19:29,183][105692] Updated weights for policy 0, policy_version 1774753 (0.0005) [2023-12-27 04:19:29,248][105692] Updated weights for policy 0, policy_version 1774763 (0.0008) [2023-12-27 04:19:29,315][105692] Updated weights for policy 0, policy_version 1774773 (0.0006) [2023-12-27 04:19:29,378][105692] Updated weights for policy 0, policy_version 1774783 (0.0007) [2023-12-27 04:19:29,385][105620] Updated weights for policy 1, policy_version 1778674 (0.0007) [2023-12-27 04:19:29,445][105620] Updated weights for policy 1, policy_version 1778684 (0.0009) [2023-12-27 04:19:29,504][105620] Updated weights for policy 1, policy_version 1778694 (0.0009) [2023-12-27 04:19:30,057][105692] Updated weights for policy 0, policy_version 1774793 (0.0006) [2023-12-27 04:19:30,120][105692] Updated weights for policy 0, policy_version 1774803 (0.0006) [2023-12-27 04:19:30,176][105692] Updated weights for policy 0, policy_version 1774813 (0.0009) [2023-12-27 04:19:30,253][105620] Updated weights for policy 1, policy_version 1778704 (0.0006) [2023-12-27 04:19:30,305][105620] Updated weights for policy 1, policy_version 1778714 (0.0008) [2023-12-27 04:19:30,353][105620] Updated weights for policy 1, policy_version 1778724 (0.0007) [2023-12-27 04:19:30,762][105692] Updated weights for policy 0, policy_version 1774823 (0.0006) [2023-12-27 04:19:30,828][105692] Updated weights for policy 0, policy_version 1774833 (0.0005) [2023-12-27 04:19:30,879][105692] Updated weights for policy 0, policy_version 1774843 (0.0005) [2023-12-27 04:19:30,995][105620] Updated weights for policy 1, policy_version 1778734 (0.0008) [2023-12-27 04:19:31,061][105620] Updated weights for policy 1, policy_version 1778744 (0.0009) [2023-12-27 04:19:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 909844480. Throughput: 0: 9707.4, 1: 9771.1. Samples: 909812744. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:19:31,063][104569] Avg episode reward: [(0, '8625.361'), (1, '9075.396')] [2023-12-27 04:19:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001774848_454426624.pth... [2023-12-27 04:19:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001773696_454131712.pth [2023-12-27 04:19:31,118][105620] Updated weights for policy 1, policy_version 1778754 (0.0006) [2023-12-27 04:19:31,154][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001778760_455426048.pth... [2023-12-27 04:19:31,159][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001777608_455131136.pth [2023-12-27 04:19:31,558][105692] Updated weights for policy 0, policy_version 1774853 (0.0007) [2023-12-27 04:19:31,610][105692] Updated weights for policy 0, policy_version 1774863 (0.0010) [2023-12-27 04:19:31,673][105692] Updated weights for policy 0, policy_version 1774873 (0.0009) [2023-12-27 04:19:31,775][105620] Updated weights for policy 1, policy_version 1778764 (0.0008) [2023-12-27 04:19:31,836][105620] Updated weights for policy 1, policy_version 1778774 (0.0008) [2023-12-27 04:19:31,896][105620] Updated weights for policy 1, policy_version 1778784 (0.0008) [2023-12-27 04:19:32,311][105692] Updated weights for policy 0, policy_version 1774883 (0.0009) [2023-12-27 04:19:32,375][105692] Updated weights for policy 0, policy_version 1774893 (0.0011) [2023-12-27 04:19:32,430][105692] Updated weights for policy 0, policy_version 1774904 (0.0011) [2023-12-27 04:19:32,598][105620] Updated weights for policy 1, policy_version 1778794 (0.0007) [2023-12-27 04:19:32,663][105620] Updated weights for policy 1, policy_version 1778804 (0.0006) [2023-12-27 04:19:32,712][105620] Updated weights for policy 1, policy_version 1778814 (0.0005) [2023-12-27 04:19:32,765][105620] Updated weights for policy 1, policy_version 1778824 (0.0006) [2023-12-27 04:19:33,158][105692] Updated weights for policy 0, policy_version 1774914 (0.0009) [2023-12-27 04:19:33,213][105692] Updated weights for policy 0, policy_version 1774924 (0.0009) [2023-12-27 04:19:33,260][105692] Updated weights for policy 0, policy_version 1774934 (0.0008) [2023-12-27 04:19:33,308][105692] Updated weights for policy 0, policy_version 1774944 (0.0005) [2023-12-27 04:19:33,427][105620] Updated weights for policy 1, policy_version 1778834 (0.0010) [2023-12-27 04:19:33,480][105620] Updated weights for policy 1, policy_version 1778845 (0.0010) [2023-12-27 04:19:33,534][105620] Updated weights for policy 1, policy_version 1778856 (0.0010) [2023-12-27 04:19:33,842][105692] Updated weights for policy 0, policy_version 1774954 (0.0005) [2023-12-27 04:19:33,895][105692] Updated weights for policy 0, policy_version 1774964 (0.0005) [2023-12-27 04:19:33,944][105692] Updated weights for policy 0, policy_version 1774974 (0.0005) [2023-12-27 04:19:34,490][105692] Updated weights for policy 0, policy_version 1774984 (0.0006) [2023-12-27 04:19:34,495][105620] Updated weights for policy 1, policy_version 1778866 (0.0009) [2023-12-27 04:19:34,554][105692] Updated weights for policy 0, policy_version 1774994 (0.0006) [2023-12-27 04:19:34,556][105620] Updated weights for policy 1, policy_version 1778876 (0.0009) [2023-12-27 04:19:34,620][105692] Updated weights for policy 0, policy_version 1775004 (0.0010) [2023-12-27 04:19:34,623][105620] Updated weights for policy 1, policy_version 1778886 (0.0008) [2023-12-27 04:19:35,171][105692] Updated weights for policy 0, policy_version 1775014 (0.0007) [2023-12-27 04:19:35,227][105692] Updated weights for policy 0, policy_version 1775024 (0.0010) [2023-12-27 04:19:35,292][105692] Updated weights for policy 0, policy_version 1775034 (0.0011) [2023-12-27 04:19:35,481][105620] Updated weights for policy 1, policy_version 1778896 (0.0008) [2023-12-27 04:19:35,525][105620] Updated weights for policy 1, policy_version 1778906 (0.0008) [2023-12-27 04:19:35,573][105620] Updated weights for policy 1, policy_version 1778916 (0.0008) [2023-12-27 04:19:36,026][105692] Updated weights for policy 0, policy_version 1775044 (0.0010) [2023-12-27 04:19:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 909942784. Throughput: 0: 9974.2, 1: 9602.5. Samples: 909934720. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:19:36,062][104569] Avg episode reward: [(0, '8171.693'), (1, '8982.549')] [2023-12-27 04:19:36,078][105692] Updated weights for policy 0, policy_version 1775054 (0.0010) [2023-12-27 04:19:36,136][105692] Updated weights for policy 0, policy_version 1775064 (0.0010) [2023-12-27 04:19:36,218][105620] Updated weights for policy 1, policy_version 1778926 (0.0008) [2023-12-27 04:19:36,282][105620] Updated weights for policy 1, policy_version 1778936 (0.0009) [2023-12-27 04:19:36,346][105620] Updated weights for policy 1, policy_version 1778946 (0.0008) [2023-12-27 04:19:36,918][105692] Updated weights for policy 0, policy_version 1775074 (0.0011) [2023-12-27 04:19:36,974][105692] Updated weights for policy 0, policy_version 1775084 (0.0011) [2023-12-27 04:19:37,034][105620] Updated weights for policy 1, policy_version 1778956 (0.0009) [2023-12-27 04:19:37,034][105692] Updated weights for policy 0, policy_version 1775094 (0.0011) [2023-12-27 04:19:37,102][105692] Updated weights for policy 0, policy_version 1775104 (0.0011) [2023-12-27 04:19:37,102][105620] Updated weights for policy 1, policy_version 1778966 (0.0011) [2023-12-27 04:19:37,162][105620] Updated weights for policy 1, policy_version 1778976 (0.0011) [2023-12-27 04:19:37,804][105692] Updated weights for policy 0, policy_version 1775114 (0.0005) [2023-12-27 04:19:37,859][105692] Updated weights for policy 0, policy_version 1775124 (0.0005) [2023-12-27 04:19:37,913][105620] Updated weights for policy 1, policy_version 1778986 (0.0009) [2023-12-27 04:19:37,918][105692] Updated weights for policy 0, policy_version 1775134 (0.0007) [2023-12-27 04:19:37,976][105620] Updated weights for policy 1, policy_version 1778996 (0.0008) [2023-12-27 04:19:38,029][105620] Updated weights for policy 1, policy_version 1779006 (0.0009) [2023-12-27 04:19:38,083][105620] Updated weights for policy 1, policy_version 1779016 (0.0009) [2023-12-27 04:19:38,569][105692] Updated weights for policy 0, policy_version 1775144 (0.0008) [2023-12-27 04:19:38,635][105692] Updated weights for policy 0, policy_version 1775154 (0.0008) [2023-12-27 04:19:38,699][105692] Updated weights for policy 0, policy_version 1775164 (0.0008) [2023-12-27 04:19:38,889][105620] Updated weights for policy 1, policy_version 1779026 (0.0005) [2023-12-27 04:19:38,952][105620] Updated weights for policy 1, policy_version 1779036 (0.0005) [2023-12-27 04:19:39,013][105620] Updated weights for policy 1, policy_version 1779046 (0.0006) [2023-12-27 04:19:39,483][105692] Updated weights for policy 0, policy_version 1775174 (0.0008) [2023-12-27 04:19:39,540][105692] Updated weights for policy 0, policy_version 1775184 (0.0009) [2023-12-27 04:19:39,595][105620] Updated weights for policy 1, policy_version 1779056 (0.0009) [2023-12-27 04:19:39,602][105692] Updated weights for policy 0, policy_version 1775194 (0.0007) [2023-12-27 04:19:39,648][105620] Updated weights for policy 1, policy_version 1779066 (0.0010) [2023-12-27 04:19:39,707][105620] Updated weights for policy 1, policy_version 1779076 (0.0008) [2023-12-27 04:19:40,410][105620] Updated weights for policy 1, policy_version 1779086 (0.0008) [2023-12-27 04:19:40,411][105692] Updated weights for policy 0, policy_version 1775204 (0.0007) [2023-12-27 04:19:40,468][105692] Updated weights for policy 0, policy_version 1775214 (0.0009) [2023-12-27 04:19:40,475][105620] Updated weights for policy 1, policy_version 1779096 (0.0006) [2023-12-27 04:19:40,527][105692] Updated weights for policy 0, policy_version 1775224 (0.0008) [2023-12-27 04:19:40,545][105620] Updated weights for policy 1, policy_version 1779106 (0.0006) [2023-12-27 04:19:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 910041088. Throughput: 0: 9864.6, 1: 9664.4. Samples: 910051380. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:19:41,063][104569] Avg episode reward: [(0, '8081.377'), (1, '8982.313')] [2023-12-27 04:19:41,251][105620] Updated weights for policy 1, policy_version 1779116 (0.0008) [2023-12-27 04:19:41,314][105620] Updated weights for policy 1, policy_version 1779126 (0.0006) [2023-12-27 04:19:41,371][105692] Updated weights for policy 0, policy_version 1775234 (0.0008) [2023-12-27 04:19:41,386][105620] Updated weights for policy 1, policy_version 1779136 (0.0008) [2023-12-27 04:19:41,434][105692] Updated weights for policy 0, policy_version 1775244 (0.0007) [2023-12-27 04:19:41,494][105692] Updated weights for policy 0, policy_version 1775254 (0.0007) [2023-12-27 04:19:41,552][105692] Updated weights for policy 0, policy_version 1775264 (0.0006) [2023-12-27 04:19:42,149][105692] Updated weights for policy 0, policy_version 1775274 (0.0006) [2023-12-27 04:19:42,196][105620] Updated weights for policy 1, policy_version 1779146 (0.0008) [2023-12-27 04:19:42,213][105692] Updated weights for policy 0, policy_version 1775284 (0.0008) [2023-12-27 04:19:42,259][105620] Updated weights for policy 1, policy_version 1779156 (0.0009) [2023-12-27 04:19:42,275][105692] Updated weights for policy 0, policy_version 1775294 (0.0007) [2023-12-27 04:19:42,328][105620] Updated weights for policy 1, policy_version 1779166 (0.0008) [2023-12-27 04:19:42,393][105620] Updated weights for policy 1, policy_version 1779176 (0.0008) [2023-12-27 04:19:43,048][105620] Updated weights for policy 1, policy_version 1779186 (0.0005) [2023-12-27 04:19:43,095][105692] Updated weights for policy 0, policy_version 1775304 (0.0009) [2023-12-27 04:19:43,115][105620] Updated weights for policy 1, policy_version 1779196 (0.0006) [2023-12-27 04:19:43,147][105692] Updated weights for policy 0, policy_version 1775314 (0.0009) [2023-12-27 04:19:43,171][105620] Updated weights for policy 1, policy_version 1779206 (0.0006) [2023-12-27 04:19:43,206][105692] Updated weights for policy 0, policy_version 1775324 (0.0009) [2023-12-27 04:19:43,749][105620] Updated weights for policy 1, policy_version 1779216 (0.0009) [2023-12-27 04:19:43,811][105620] Updated weights for policy 1, policy_version 1779226 (0.0010) [2023-12-27 04:19:43,872][105620] Updated weights for policy 1, policy_version 1779237 (0.0006) [2023-12-27 04:19:43,999][105692] Updated weights for policy 0, policy_version 1775334 (0.0010) [2023-12-27 04:19:44,053][105692] Updated weights for policy 0, policy_version 1775344 (0.0010) [2023-12-27 04:19:44,103][105692] Updated weights for policy 0, policy_version 1775354 (0.0009) [2023-12-27 04:19:44,435][105620] Updated weights for policy 1, policy_version 1779247 (0.0005) [2023-12-27 04:19:44,495][105620] Updated weights for policy 1, policy_version 1779257 (0.0006) [2023-12-27 04:19:44,550][105620] Updated weights for policy 1, policy_version 1779267 (0.0005) [2023-12-27 04:19:44,801][105692] Updated weights for policy 0, policy_version 1775364 (0.0009) [2023-12-27 04:19:44,865][105692] Updated weights for policy 0, policy_version 1775374 (0.0007) [2023-12-27 04:19:44,925][105692] Updated weights for policy 0, policy_version 1775384 (0.0010) [2023-12-27 04:19:45,145][105620] Updated weights for policy 1, policy_version 1779277 (0.0006) [2023-12-27 04:19:45,209][105620] Updated weights for policy 1, policy_version 1779287 (0.0010) [2023-12-27 04:19:45,276][105620] Updated weights for policy 1, policy_version 1779297 (0.0008) [2023-12-27 04:19:45,567][105692] Updated weights for policy 0, policy_version 1775394 (0.0010) [2023-12-27 04:19:45,625][105692] Updated weights for policy 0, policy_version 1775404 (0.0005) [2023-12-27 04:19:45,681][105692] Updated weights for policy 0, policy_version 1775414 (0.0008) [2023-12-27 04:19:45,736][105692] Updated weights for policy 0, policy_version 1775424 (0.0010) [2023-12-27 04:19:45,906][105620] Updated weights for policy 1, policy_version 1779307 (0.0007) [2023-12-27 04:19:45,968][105620] Updated weights for policy 1, policy_version 1779317 (0.0010) [2023-12-27 04:19:46,026][105620] Updated weights for policy 1, policy_version 1779327 (0.0010) [2023-12-27 04:19:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 910139392. Throughput: 0: 9727.2, 1: 9681.9. Samples: 910108656. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:19:46,063][104569] Avg episode reward: [(0, '8447.540'), (1, '9167.085')] [2023-12-27 04:19:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001779336_455573504.pth... [2023-12-27 04:19:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001775424_454574080.pth... [2023-12-27 04:19:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001778184_455278592.pth [2023-12-27 04:19:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001774240_454270976.pth [2023-12-27 04:19:46,352][105692] Updated weights for policy 0, policy_version 1775434 (0.0007) [2023-12-27 04:19:46,413][105692] Updated weights for policy 0, policy_version 1775444 (0.0010) [2023-12-27 04:19:46,474][105692] Updated weights for policy 0, policy_version 1775454 (0.0010) [2023-12-27 04:19:46,624][105620] Updated weights for policy 1, policy_version 1779337 (0.0010) [2023-12-27 04:19:46,687][105620] Updated weights for policy 1, policy_version 1779347 (0.0005) [2023-12-27 04:19:46,748][105620] Updated weights for policy 1, policy_version 1779357 (0.0005) [2023-12-27 04:19:46,800][105620] Updated weights for policy 1, policy_version 1779367 (0.0005) [2023-12-27 04:19:47,277][105692] Updated weights for policy 0, policy_version 1775464 (0.0007) [2023-12-27 04:19:47,340][105692] Updated weights for policy 0, policy_version 1775474 (0.0005) [2023-12-27 04:19:47,375][105620] Updated weights for policy 1, policy_version 1779377 (0.0007) [2023-12-27 04:19:47,389][105692] Updated weights for policy 0, policy_version 1775484 (0.0005) [2023-12-27 04:19:47,423][105620] Updated weights for policy 1, policy_version 1779387 (0.0010) [2023-12-27 04:19:47,479][105620] Updated weights for policy 1, policy_version 1779397 (0.0010) [2023-12-27 04:19:47,927][105692] Updated weights for policy 0, policy_version 1775494 (0.0008) [2023-12-27 04:19:47,978][105692] Updated weights for policy 0, policy_version 1775504 (0.0009) [2023-12-27 04:19:48,036][105692] Updated weights for policy 0, policy_version 1775514 (0.0008) [2023-12-27 04:19:48,038][105620] Updated weights for policy 1, policy_version 1779407 (0.0010) [2023-12-27 04:19:48,086][105620] Updated weights for policy 1, policy_version 1779417 (0.0010) [2023-12-27 04:19:48,134][105620] Updated weights for policy 1, policy_version 1779427 (0.0010) [2023-12-27 04:19:48,809][105692] Updated weights for policy 0, policy_version 1775524 (0.0005) [2023-12-27 04:19:48,862][105692] Updated weights for policy 0, policy_version 1775534 (0.0007) [2023-12-27 04:19:48,868][105620] Updated weights for policy 1, policy_version 1779437 (0.0008) [2023-12-27 04:19:48,913][105692] Updated weights for policy 0, policy_version 1775544 (0.0007) [2023-12-27 04:19:48,932][105620] Updated weights for policy 1, policy_version 1779447 (0.0005) [2023-12-27 04:19:48,990][105620] Updated weights for policy 1, policy_version 1779457 (0.0008) [2023-12-27 04:19:49,654][105692] Updated weights for policy 0, policy_version 1775554 (0.0007) [2023-12-27 04:19:49,715][105692] Updated weights for policy 0, policy_version 1775564 (0.0009) [2023-12-27 04:19:49,750][105620] Updated weights for policy 1, policy_version 1779467 (0.0008) [2023-12-27 04:19:49,778][105692] Updated weights for policy 0, policy_version 1775574 (0.0009) [2023-12-27 04:19:49,813][105620] Updated weights for policy 1, policy_version 1779477 (0.0008) [2023-12-27 04:19:49,836][105692] Updated weights for policy 0, policy_version 1775584 (0.0007) [2023-12-27 04:19:49,884][105620] Updated weights for policy 1, policy_version 1779487 (0.0008) [2023-12-27 04:19:50,587][105692] Updated weights for policy 0, policy_version 1775594 (0.0010) [2023-12-27 04:19:50,654][105692] Updated weights for policy 0, policy_version 1775604 (0.0009) [2023-12-27 04:19:50,675][105620] Updated weights for policy 1, policy_version 1779497 (0.0009) [2023-12-27 04:19:50,715][105692] Updated weights for policy 0, policy_version 1775614 (0.0008) [2023-12-27 04:19:50,737][105620] Updated weights for policy 1, policy_version 1779507 (0.0008) [2023-12-27 04:19:50,795][105620] Updated weights for policy 1, policy_version 1779517 (0.0010) [2023-12-27 04:19:50,856][105620] Updated weights for policy 1, policy_version 1779527 (0.0009) [2023-12-27 04:19:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 910245888. Throughput: 0: 9845.7, 1: 9857.0. Samples: 910233724. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:19:51,063][104569] Avg episode reward: [(0, '8173.676'), (1, '9167.050')] [2023-12-27 04:19:51,379][105692] Updated weights for policy 0, policy_version 1775624 (0.0007) [2023-12-27 04:19:51,436][105692] Updated weights for policy 0, policy_version 1775634 (0.0008) [2023-12-27 04:19:51,495][105692] Updated weights for policy 0, policy_version 1775644 (0.0009) [2023-12-27 04:19:51,659][105620] Updated weights for policy 1, policy_version 1779537 (0.0010) [2023-12-27 04:19:51,729][105620] Updated weights for policy 1, policy_version 1779547 (0.0008) [2023-12-27 04:19:51,797][105620] Updated weights for policy 1, policy_version 1779557 (0.0009) [2023-12-27 04:19:52,156][105692] Updated weights for policy 0, policy_version 1775654 (0.0009) [2023-12-27 04:19:52,215][105692] Updated weights for policy 0, policy_version 1775664 (0.0009) [2023-12-27 04:19:52,278][105692] Updated weights for policy 0, policy_version 1775674 (0.0009) [2023-12-27 04:19:52,585][105620] Updated weights for policy 1, policy_version 1779567 (0.0007) [2023-12-27 04:19:52,647][105620] Updated weights for policy 1, policy_version 1779577 (0.0007) [2023-12-27 04:19:52,710][105620] Updated weights for policy 1, policy_version 1779587 (0.0008) [2023-12-27 04:19:53,027][105692] Updated weights for policy 0, policy_version 1775684 (0.0009) [2023-12-27 04:19:53,089][105692] Updated weights for policy 0, policy_version 1775694 (0.0010) [2023-12-27 04:19:53,153][105692] Updated weights for policy 0, policy_version 1775704 (0.0009) [2023-12-27 04:19:53,378][105620] Updated weights for policy 1, policy_version 1779597 (0.0008) [2023-12-27 04:19:53,438][105620] Updated weights for policy 1, policy_version 1779607 (0.0006) [2023-12-27 04:19:53,493][105620] Updated weights for policy 1, policy_version 1779617 (0.0005) [2023-12-27 04:19:53,963][105692] Updated weights for policy 0, policy_version 1775714 (0.0008) [2023-12-27 04:19:54,025][105692] Updated weights for policy 0, policy_version 1775724 (0.0006) [2023-12-27 04:19:54,080][105692] Updated weights for policy 0, policy_version 1775734 (0.0005) [2023-12-27 04:19:54,125][105620] Updated weights for policy 1, policy_version 1779627 (0.0006) [2023-12-27 04:19:54,134][105692] Updated weights for policy 0, policy_version 1775744 (0.0005) [2023-12-27 04:19:54,175][105620] Updated weights for policy 1, policy_version 1779637 (0.0010) [2023-12-27 04:19:54,226][105620] Updated weights for policy 1, policy_version 1779647 (0.0010) [2023-12-27 04:19:54,693][105692] Updated weights for policy 0, policy_version 1775754 (0.0005) [2023-12-27 04:19:54,750][105692] Updated weights for policy 0, policy_version 1775764 (0.0005) [2023-12-27 04:19:54,807][105692] Updated weights for policy 0, policy_version 1775774 (0.0005) [2023-12-27 04:19:54,904][105620] Updated weights for policy 1, policy_version 1779657 (0.0010) [2023-12-27 04:19:54,966][105620] Updated weights for policy 1, policy_version 1779667 (0.0009) [2023-12-27 04:19:55,026][105620] Updated weights for policy 1, policy_version 1779677 (0.0008) [2023-12-27 04:19:55,089][105620] Updated weights for policy 1, policy_version 1779687 (0.0011) [2023-12-27 04:19:55,414][105692] Updated weights for policy 0, policy_version 1775784 (0.0009) [2023-12-27 04:19:55,466][105692] Updated weights for policy 0, policy_version 1775794 (0.0010) [2023-12-27 04:19:55,522][105692] Updated weights for policy 0, policy_version 1775804 (0.0010) [2023-12-27 04:19:55,681][105620] Updated weights for policy 1, policy_version 1779697 (0.0007) [2023-12-27 04:19:55,728][105620] Updated weights for policy 1, policy_version 1779707 (0.0006) [2023-12-27 04:19:55,781][105620] Updated weights for policy 1, policy_version 1779717 (0.0006) [2023-12-27 04:19:56,062][104569] Fps is (10 sec: 20479.4, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 910344192. Throughput: 0: 9950.7, 1: 9867.6. Samples: 910352412. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:19:56,064][104569] Avg episode reward: [(0, '8350.980'), (1, '8983.671')] [2023-12-27 04:19:56,345][105620] Updated weights for policy 1, policy_version 1779727 (0.0009) [2023-12-27 04:19:56,399][105620] Updated weights for policy 1, policy_version 1779737 (0.0008) [2023-12-27 04:19:56,399][105692] Updated weights for policy 0, policy_version 1775814 (0.0007) [2023-12-27 04:19:56,459][105692] Updated weights for policy 0, policy_version 1775824 (0.0006) [2023-12-27 04:19:56,461][105620] Updated weights for policy 1, policy_version 1779747 (0.0008) [2023-12-27 04:19:56,511][105692] Updated weights for policy 0, policy_version 1775834 (0.0007) [2023-12-27 04:19:57,021][105620] Updated weights for policy 1, policy_version 1779757 (0.0009) [2023-12-27 04:19:57,069][105620] Updated weights for policy 1, policy_version 1779767 (0.0010) [2023-12-27 04:19:57,122][105620] Updated weights for policy 1, policy_version 1779777 (0.0006) [2023-12-27 04:19:57,158][105692] Updated weights for policy 0, policy_version 1775844 (0.0007) [2023-12-27 04:19:57,216][105692] Updated weights for policy 0, policy_version 1775854 (0.0010) [2023-12-27 04:19:57,267][105692] Updated weights for policy 0, policy_version 1775864 (0.0010) [2023-12-27 04:19:57,677][105620] Updated weights for policy 1, policy_version 1779787 (0.0007) [2023-12-27 04:19:57,725][105620] Updated weights for policy 1, policy_version 1779797 (0.0010) [2023-12-27 04:19:57,772][105620] Updated weights for policy 1, policy_version 1779807 (0.0009) [2023-12-27 04:19:58,017][105692] Updated weights for policy 0, policy_version 1775874 (0.0011) [2023-12-27 04:19:58,071][105692] Updated weights for policy 0, policy_version 1775884 (0.0010) [2023-12-27 04:19:58,126][105692] Updated weights for policy 0, policy_version 1775894 (0.0010) [2023-12-27 04:19:58,191][105692] Updated weights for policy 0, policy_version 1775904 (0.0010) [2023-12-27 04:19:58,400][105620] Updated weights for policy 1, policy_version 1779817 (0.0006) [2023-12-27 04:19:58,467][105620] Updated weights for policy 1, policy_version 1779827 (0.0008) [2023-12-27 04:19:58,529][105620] Updated weights for policy 1, policy_version 1779837 (0.0008) [2023-12-27 04:19:58,586][105620] Updated weights for policy 1, policy_version 1779847 (0.0008) [2023-12-27 04:19:59,080][105692] Updated weights for policy 0, policy_version 1775914 (0.0010) [2023-12-27 04:19:59,124][105692] Updated weights for policy 0, policy_version 1775924 (0.0010) [2023-12-27 04:19:59,179][105692] Updated weights for policy 0, policy_version 1775934 (0.0010) [2023-12-27 04:19:59,377][105620] Updated weights for policy 1, policy_version 1779857 (0.0013) [2023-12-27 04:19:59,443][105620] Updated weights for policy 1, policy_version 1779867 (0.0011) [2023-12-27 04:19:59,514][105620] Updated weights for policy 1, policy_version 1779877 (0.0007) [2023-12-27 04:19:59,885][105692] Updated weights for policy 0, policy_version 1775944 (0.0009) [2023-12-27 04:19:59,956][105692] Updated weights for policy 0, policy_version 1775954 (0.0011) [2023-12-27 04:20:00,009][105692] Updated weights for policy 0, policy_version 1775964 (0.0010) [2023-12-27 04:20:00,140][105620] Updated weights for policy 1, policy_version 1779887 (0.0006) [2023-12-27 04:20:00,194][105620] Updated weights for policy 1, policy_version 1779897 (0.0005) [2023-12-27 04:20:00,246][105620] Updated weights for policy 1, policy_version 1779907 (0.0008) [2023-12-27 04:20:00,670][105692] Updated weights for policy 0, policy_version 1775974 (0.0010) [2023-12-27 04:20:00,723][105692] Updated weights for policy 0, policy_version 1775984 (0.0010) [2023-12-27 04:20:00,787][105692] Updated weights for policy 0, policy_version 1775994 (0.0010) [2023-12-27 04:20:00,937][105620] Updated weights for policy 1, policy_version 1779917 (0.0006) [2023-12-27 04:20:01,000][105620] Updated weights for policy 1, policy_version 1779927 (0.0005) [2023-12-27 04:20:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 910442496. Throughput: 0: 9930.7, 1: 9964.7. Samples: 910414336. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:20:01,062][104569] Avg episode reward: [(0, '8718.665'), (1, '9076.312')] [2023-12-27 04:20:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001776000_454721536.pth... [2023-12-27 04:20:01,068][105620] Updated weights for policy 1, policy_version 1779937 (0.0010) [2023-12-27 04:20:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001774848_454426624.pth [2023-12-27 04:20:01,113][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001779944_455729152.pth... [2023-12-27 04:20:01,118][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001778760_455426048.pth [2023-12-27 04:20:01,564][105692] Updated weights for policy 0, policy_version 1776004 (0.0009) [2023-12-27 04:20:01,623][105692] Updated weights for policy 0, policy_version 1776014 (0.0006) [2023-12-27 04:20:01,679][105692] Updated weights for policy 0, policy_version 1776024 (0.0008) [2023-12-27 04:20:01,744][105620] Updated weights for policy 1, policy_version 1779947 (0.0007) [2023-12-27 04:20:01,798][105620] Updated weights for policy 1, policy_version 1779957 (0.0010) [2023-12-27 04:20:01,860][105620] Updated weights for policy 1, policy_version 1779967 (0.0010) [2023-12-27 04:20:02,330][105692] Updated weights for policy 0, policy_version 1776034 (0.0008) [2023-12-27 04:20:02,396][105692] Updated weights for policy 0, policy_version 1776044 (0.0009) [2023-12-27 04:20:02,463][105692] Updated weights for policy 0, policy_version 1776054 (0.0009) [2023-12-27 04:20:02,518][105692] Updated weights for policy 0, policy_version 1776064 (0.0008) [2023-12-27 04:20:02,671][105620] Updated weights for policy 1, policy_version 1779977 (0.0010) [2023-12-27 04:20:02,726][105620] Updated weights for policy 1, policy_version 1779987 (0.0009) [2023-12-27 04:20:02,784][105620] Updated weights for policy 1, policy_version 1779997 (0.0009) [2023-12-27 04:20:02,845][105620] Updated weights for policy 1, policy_version 1780007 (0.0009) [2023-12-27 04:20:03,229][105692] Updated weights for policy 0, policy_version 1776074 (0.0009) [2023-12-27 04:20:03,283][105692] Updated weights for policy 0, policy_version 1776084 (0.0009) [2023-12-27 04:20:03,340][105692] Updated weights for policy 0, policy_version 1776094 (0.0009) [2023-12-27 04:20:03,529][105620] Updated weights for policy 1, policy_version 1780017 (0.0006) [2023-12-27 04:20:03,575][105620] Updated weights for policy 1, policy_version 1780027 (0.0005) [2023-12-27 04:20:03,638][105620] Updated weights for policy 1, policy_version 1780037 (0.0005) [2023-12-27 04:20:04,214][105620] Updated weights for policy 1, policy_version 1780047 (0.0009) [2023-12-27 04:20:04,250][105692] Updated weights for policy 0, policy_version 1776104 (0.0007) [2023-12-27 04:20:04,276][105620] Updated weights for policy 1, policy_version 1780057 (0.0011) [2023-12-27 04:20:04,312][105692] Updated weights for policy 0, policy_version 1776114 (0.0007) [2023-12-27 04:20:04,340][105620] Updated weights for policy 1, policy_version 1780067 (0.0011) [2023-12-27 04:20:04,372][105692] Updated weights for policy 0, policy_version 1776124 (0.0007) [2023-12-27 04:20:04,942][105620] Updated weights for policy 1, policy_version 1780077 (0.0009) [2023-12-27 04:20:05,012][105620] Updated weights for policy 1, policy_version 1780087 (0.0006) [2023-12-27 04:20:05,071][105620] Updated weights for policy 1, policy_version 1780097 (0.0008) [2023-12-27 04:20:05,183][105692] Updated weights for policy 0, policy_version 1776134 (0.0008) [2023-12-27 04:20:05,242][105692] Updated weights for policy 0, policy_version 1776144 (0.0008) [2023-12-27 04:20:05,296][105692] Updated weights for policy 0, policy_version 1776154 (0.0008) [2023-12-27 04:20:05,679][105620] Updated weights for policy 1, policy_version 1780107 (0.0010) [2023-12-27 04:20:05,733][105620] Updated weights for policy 1, policy_version 1780117 (0.0006) [2023-12-27 04:20:05,792][105620] Updated weights for policy 1, policy_version 1780127 (0.0006) [2023-12-27 04:20:06,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 910540800. Throughput: 0: 9900.5, 1: 10076.0. Samples: 910532040. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:20:06,062][104569] Avg episode reward: [(0, '8719.976'), (1, '9167.885')] [2023-12-27 04:20:06,147][105692] Updated weights for policy 0, policy_version 1776164 (0.0009) [2023-12-27 04:20:06,208][105692] Updated weights for policy 0, policy_version 1776174 (0.0008) [2023-12-27 04:20:06,276][105692] Updated weights for policy 0, policy_version 1776184 (0.0008) [2023-12-27 04:20:06,439][105620] Updated weights for policy 1, policy_version 1780137 (0.0006) [2023-12-27 04:20:06,500][105620] Updated weights for policy 1, policy_version 1780147 (0.0011) [2023-12-27 04:20:06,559][105620] Updated weights for policy 1, policy_version 1780157 (0.0008) [2023-12-27 04:20:06,621][105620] Updated weights for policy 1, policy_version 1780167 (0.0006) [2023-12-27 04:20:07,004][105692] Updated weights for policy 0, policy_version 1776194 (0.0009) [2023-12-27 04:20:07,062][105692] Updated weights for policy 0, policy_version 1776204 (0.0010) [2023-12-27 04:20:07,119][105692] Updated weights for policy 0, policy_version 1776214 (0.0012) [2023-12-27 04:20:07,207][105620] Updated weights for policy 1, policy_version 1780177 (0.0005) [2023-12-27 04:20:07,267][105620] Updated weights for policy 1, policy_version 1780187 (0.0009) [2023-12-27 04:20:07,319][105620] Updated weights for policy 1, policy_version 1780197 (0.0011) [2023-12-27 04:20:07,970][105692] Updated weights for policy 0, policy_version 1776225 (0.0009) [2023-12-27 04:20:07,972][105620] Updated weights for policy 1, policy_version 1780207 (0.0010) [2023-12-27 04:20:08,031][105692] Updated weights for policy 0, policy_version 1776235 (0.0005) [2023-12-27 04:20:08,036][105620] Updated weights for policy 1, policy_version 1780217 (0.0011) [2023-12-27 04:20:08,086][105692] Updated weights for policy 0, policy_version 1776245 (0.0007) [2023-12-27 04:20:08,089][105620] Updated weights for policy 1, policy_version 1780227 (0.0011) [2023-12-27 04:20:08,152][105692] Updated weights for policy 0, policy_version 1776255 (0.0007) [2023-12-27 04:20:08,809][105620] Updated weights for policy 1, policy_version 1780237 (0.0011) [2023-12-27 04:20:08,864][105620] Updated weights for policy 1, policy_version 1780247 (0.0010) [2023-12-27 04:20:08,927][105620] Updated weights for policy 1, policy_version 1780257 (0.0011) [2023-12-27 04:20:08,932][105692] Updated weights for policy 0, policy_version 1776265 (0.0010) [2023-12-27 04:20:08,989][105692] Updated weights for policy 0, policy_version 1776275 (0.0011) [2023-12-27 04:20:09,045][105692] Updated weights for policy 0, policy_version 1776285 (0.0009) [2023-12-27 04:20:09,638][105620] Updated weights for policy 1, policy_version 1780267 (0.0011) [2023-12-27 04:20:09,699][105620] Updated weights for policy 1, policy_version 1780277 (0.0010) [2023-12-27 04:20:09,736][105692] Updated weights for policy 0, policy_version 1776295 (0.0005) [2023-12-27 04:20:09,757][105620] Updated weights for policy 1, policy_version 1780287 (0.0008) [2023-12-27 04:20:09,800][105692] Updated weights for policy 0, policy_version 1776305 (0.0006) [2023-12-27 04:20:09,865][105692] Updated weights for policy 0, policy_version 1776315 (0.0007) [2023-12-27 04:20:10,518][105620] Updated weights for policy 1, policy_version 1780297 (0.0008) [2023-12-27 04:20:10,582][105620] Updated weights for policy 1, policy_version 1780307 (0.0011) [2023-12-27 04:20:10,586][105692] Updated weights for policy 0, policy_version 1776325 (0.0008) [2023-12-27 04:20:10,642][105620] Updated weights for policy 1, policy_version 1780317 (0.0009) [2023-12-27 04:20:10,646][105692] Updated weights for policy 0, policy_version 1776335 (0.0011) [2023-12-27 04:20:10,696][105620] Updated weights for policy 1, policy_version 1780327 (0.0011) [2023-12-27 04:20:10,696][105692] Updated weights for policy 0, policy_version 1776345 (0.0010) [2023-12-27 04:20:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 910639104. Throughput: 0: 9740.3, 1: 10188.3. Samples: 910647472. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:20:11,063][104569] Avg episode reward: [(0, '8717.456'), (1, '9075.367')] [2023-12-27 04:20:11,412][105620] Updated weights for policy 1, policy_version 1780337 (0.0008) [2023-12-27 04:20:11,483][105620] Updated weights for policy 1, policy_version 1780347 (0.0006) [2023-12-27 04:20:11,490][105692] Updated weights for policy 0, policy_version 1776355 (0.0011) [2023-12-27 04:20:11,543][105620] Updated weights for policy 1, policy_version 1780357 (0.0009) [2023-12-27 04:20:11,551][105692] Updated weights for policy 0, policy_version 1776365 (0.0011) [2023-12-27 04:20:11,626][105692] Updated weights for policy 0, policy_version 1776376 (0.0010) [2023-12-27 04:20:12,139][105620] Updated weights for policy 1, policy_version 1780367 (0.0009) [2023-12-27 04:20:12,192][105620] Updated weights for policy 1, policy_version 1780377 (0.0011) [2023-12-27 04:20:12,244][105692] Updated weights for policy 0, policy_version 1776386 (0.0010) [2023-12-27 04:20:12,246][105620] Updated weights for policy 1, policy_version 1780387 (0.0010) [2023-12-27 04:20:12,302][105692] Updated weights for policy 0, policy_version 1776396 (0.0007) [2023-12-27 04:20:12,367][105692] Updated weights for policy 0, policy_version 1776406 (0.0007) [2023-12-27 04:20:12,429][105692] Updated weights for policy 0, policy_version 1776416 (0.0007) [2023-12-27 04:20:12,911][105620] Updated weights for policy 1, policy_version 1780397 (0.0011) [2023-12-27 04:20:12,969][105620] Updated weights for policy 1, policy_version 1780407 (0.0010) [2023-12-27 04:20:13,017][105620] Updated weights for policy 1, policy_version 1780417 (0.0010) [2023-12-27 04:20:13,037][105692] Updated weights for policy 0, policy_version 1776426 (0.0005) [2023-12-27 04:20:13,088][105692] Updated weights for policy 0, policy_version 1776436 (0.0007) [2023-12-27 04:20:13,143][105692] Updated weights for policy 0, policy_version 1776446 (0.0008) [2023-12-27 04:20:13,732][105620] Updated weights for policy 1, policy_version 1780427 (0.0009) [2023-12-27 04:20:13,803][105620] Updated weights for policy 1, policy_version 1780437 (0.0005) [2023-12-27 04:20:13,859][105620] Updated weights for policy 1, policy_version 1780447 (0.0008) [2023-12-27 04:20:13,878][105692] Updated weights for policy 0, policy_version 1776456 (0.0009) [2023-12-27 04:20:13,934][105692] Updated weights for policy 0, policy_version 1776466 (0.0008) [2023-12-27 04:20:13,981][105692] Updated weights for policy 0, policy_version 1776476 (0.0008) [2023-12-27 04:20:14,456][105620] Updated weights for policy 1, policy_version 1780457 (0.0010) [2023-12-27 04:20:14,506][105620] Updated weights for policy 1, policy_version 1780467 (0.0005) [2023-12-27 04:20:14,554][105620] Updated weights for policy 1, policy_version 1780477 (0.0007) [2023-12-27 04:20:14,601][105620] Updated weights for policy 1, policy_version 1780487 (0.0010) [2023-12-27 04:20:14,785][105692] Updated weights for policy 0, policy_version 1776486 (0.0009) [2023-12-27 04:20:14,842][105692] Updated weights for policy 0, policy_version 1776496 (0.0008) [2023-12-27 04:20:14,898][105692] Updated weights for policy 0, policy_version 1776506 (0.0008) [2023-12-27 04:20:15,292][105620] Updated weights for policy 1, policy_version 1780497 (0.0011) [2023-12-27 04:20:15,352][105620] Updated weights for policy 1, policy_version 1780507 (0.0011) [2023-12-27 04:20:15,408][105620] Updated weights for policy 1, policy_version 1780517 (0.0010) [2023-12-27 04:20:15,656][105692] Updated weights for policy 0, policy_version 1776516 (0.0008) [2023-12-27 04:20:15,700][105692] Updated weights for policy 0, policy_version 1776526 (0.0008) [2023-12-27 04:20:15,752][105692] Updated weights for policy 0, policy_version 1776536 (0.0008) [2023-12-27 04:20:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 910737408. Throughput: 0: 9753.5, 1: 10145.2. Samples: 910708184. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:20:16,063][104569] Avg episode reward: [(0, '8530.734'), (1, '9076.300')] [2023-12-27 04:20:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001776544_454860800.pth... [2023-12-27 04:20:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001775424_454574080.pth [2023-12-27 04:20:16,097][105620] Updated weights for policy 1, policy_version 1780527 (0.0006) [2023-12-27 04:20:16,162][105620] Updated weights for policy 1, policy_version 1780537 (0.0005) [2023-12-27 04:20:16,233][105620] Updated weights for policy 1, policy_version 1780547 (0.0005) [2023-12-27 04:20:16,266][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001780552_455884800.pth... [2023-12-27 04:20:16,270][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001779336_455573504.pth [2023-12-27 04:20:16,461][105692] Updated weights for policy 0, policy_version 1776546 (0.0007) [2023-12-27 04:20:16,513][105692] Updated weights for policy 0, policy_version 1776556 (0.0007) [2023-12-27 04:20:16,561][105692] Updated weights for policy 0, policy_version 1776566 (0.0010) [2023-12-27 04:20:16,606][105692] Updated weights for policy 0, policy_version 1776576 (0.0009) [2023-12-27 04:20:16,802][105620] Updated weights for policy 1, policy_version 1780557 (0.0008) [2023-12-27 04:20:16,855][105620] Updated weights for policy 1, policy_version 1780567 (0.0007) [2023-12-27 04:20:16,903][105620] Updated weights for policy 1, policy_version 1780577 (0.0008) [2023-12-27 04:20:17,275][105692] Updated weights for policy 0, policy_version 1776586 (0.0005) [2023-12-27 04:20:17,321][105692] Updated weights for policy 0, policy_version 1776596 (0.0005) [2023-12-27 04:20:17,366][105692] Updated weights for policy 0, policy_version 1776606 (0.0006) [2023-12-27 04:20:17,757][105620] Updated weights for policy 1, policy_version 1780587 (0.0009) [2023-12-27 04:20:17,811][105620] Updated weights for policy 1, policy_version 1780599 (0.0010) [2023-12-27 04:20:17,861][105620] Updated weights for policy 1, policy_version 1780609 (0.0009) [2023-12-27 04:20:17,957][105692] Updated weights for policy 0, policy_version 1776616 (0.0011) [2023-12-27 04:20:18,009][105692] Updated weights for policy 0, policy_version 1776626 (0.0010) [2023-12-27 04:20:18,060][105692] Updated weights for policy 0, policy_version 1776636 (0.0010) [2023-12-27 04:20:18,677][105620] Updated weights for policy 1, policy_version 1780619 (0.0008) [2023-12-27 04:20:18,746][105620] Updated weights for policy 1, policy_version 1780629 (0.0011) [2023-12-27 04:20:18,753][105692] Updated weights for policy 0, policy_version 1776646 (0.0010) [2023-12-27 04:20:18,805][105620] Updated weights for policy 1, policy_version 1780639 (0.0011) [2023-12-27 04:20:18,809][105692] Updated weights for policy 0, policy_version 1776656 (0.0010) [2023-12-27 04:20:18,864][105692] Updated weights for policy 0, policy_version 1776666 (0.0010) [2023-12-27 04:20:19,530][105620] Updated weights for policy 1, policy_version 1780649 (0.0010) [2023-12-27 04:20:19,576][105692] Updated weights for policy 0, policy_version 1776676 (0.0011) [2023-12-27 04:20:19,591][105620] Updated weights for policy 1, policy_version 1780659 (0.0012) [2023-12-27 04:20:19,636][105692] Updated weights for policy 0, policy_version 1776686 (0.0011) [2023-12-27 04:20:19,648][105620] Updated weights for policy 1, policy_version 1780669 (0.0009) [2023-12-27 04:20:19,696][105692] Updated weights for policy 0, policy_version 1776696 (0.0011) [2023-12-27 04:20:19,711][105620] Updated weights for policy 1, policy_version 1780679 (0.0010) [2023-12-27 04:20:20,349][105692] Updated weights for policy 0, policy_version 1776706 (0.0010) [2023-12-27 04:20:20,415][105692] Updated weights for policy 0, policy_version 1776716 (0.0008) [2023-12-27 04:20:20,477][105692] Updated weights for policy 0, policy_version 1776726 (0.0007) [2023-12-27 04:20:20,501][105620] Updated weights for policy 1, policy_version 1780689 (0.0011) [2023-12-27 04:20:20,536][105692] Updated weights for policy 0, policy_version 1776736 (0.0007) [2023-12-27 04:20:20,565][105620] Updated weights for policy 1, policy_version 1780699 (0.0011) [2023-12-27 04:20:20,635][105620] Updated weights for policy 1, policy_version 1780709 (0.0010) [2023-12-27 04:20:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19549.7). Total num frames: 910835712. Throughput: 0: 9636.4, 1: 10197.8. Samples: 910827260. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:20:21,062][104569] Avg episode reward: [(0, '8533.536'), (1, '8800.459')] [2023-12-27 04:20:21,283][105692] Updated weights for policy 0, policy_version 1776746 (0.0008) [2023-12-27 04:20:21,352][105692] Updated weights for policy 0, policy_version 1776756 (0.0009) [2023-12-27 04:20:21,423][105692] Updated weights for policy 0, policy_version 1776766 (0.0008) [2023-12-27 04:20:21,469][105620] Updated weights for policy 1, policy_version 1780719 (0.0009) [2023-12-27 04:20:21,536][105620] Updated weights for policy 1, policy_version 1780729 (0.0008) [2023-12-27 04:20:21,601][105620] Updated weights for policy 1, policy_version 1780739 (0.0008) [2023-12-27 04:20:22,218][105692] Updated weights for policy 0, policy_version 1776776 (0.0009) [2023-12-27 04:20:22,266][105620] Updated weights for policy 1, policy_version 1780749 (0.0007) [2023-12-27 04:20:22,276][105692] Updated weights for policy 0, policy_version 1776786 (0.0009) [2023-12-27 04:20:22,328][105620] Updated weights for policy 1, policy_version 1780759 (0.0008) [2023-12-27 04:20:22,338][105692] Updated weights for policy 0, policy_version 1776796 (0.0007) [2023-12-27 04:20:22,414][105620] Updated weights for policy 1, policy_version 1780769 (0.0008) [2023-12-27 04:20:23,099][105620] Updated weights for policy 1, policy_version 1780779 (0.0008) [2023-12-27 04:20:23,161][105620] Updated weights for policy 1, policy_version 1780789 (0.0009) [2023-12-27 04:20:23,194][105692] Updated weights for policy 0, policy_version 1776806 (0.0008) [2023-12-27 04:20:23,210][105620] Updated weights for policy 1, policy_version 1780799 (0.0009) [2023-12-27 04:20:23,241][105692] Updated weights for policy 0, policy_version 1776816 (0.0009) [2023-12-27 04:20:23,288][105692] Updated weights for policy 0, policy_version 1776826 (0.0008) [2023-12-27 04:20:23,973][105692] Updated weights for policy 0, policy_version 1776836 (0.0005) [2023-12-27 04:20:24,019][105620] Updated weights for policy 1, policy_version 1780809 (0.0007) [2023-12-27 04:20:24,038][105692] Updated weights for policy 0, policy_version 1776846 (0.0006) [2023-12-27 04:20:24,070][105620] Updated weights for policy 1, policy_version 1780819 (0.0006) [2023-12-27 04:20:24,097][105692] Updated weights for policy 0, policy_version 1776856 (0.0007) [2023-12-27 04:20:24,130][105620] Updated weights for policy 1, policy_version 1780829 (0.0008) [2023-12-27 04:20:24,186][105620] Updated weights for policy 1, policy_version 1780839 (0.0008) [2023-12-27 04:20:24,791][105692] Updated weights for policy 0, policy_version 1776866 (0.0007) [2023-12-27 04:20:24,851][105692] Updated weights for policy 0, policy_version 1776876 (0.0007) [2023-12-27 04:20:24,903][105620] Updated weights for policy 1, policy_version 1780849 (0.0008) [2023-12-27 04:20:24,909][105692] Updated weights for policy 0, policy_version 1776886 (0.0010) [2023-12-27 04:20:24,966][105620] Updated weights for policy 1, policy_version 1780859 (0.0007) [2023-12-27 04:20:24,975][105692] Updated weights for policy 0, policy_version 1776896 (0.0007) [2023-12-27 04:20:25,023][105620] Updated weights for policy 1, policy_version 1780869 (0.0009) [2023-12-27 04:20:25,594][105692] Updated weights for policy 0, policy_version 1776906 (0.0010) [2023-12-27 04:20:25,642][105692] Updated weights for policy 0, policy_version 1776916 (0.0010) [2023-12-27 04:20:25,689][105692] Updated weights for policy 0, policy_version 1776926 (0.0010) [2023-12-27 04:20:25,728][105620] Updated weights for policy 1, policy_version 1780879 (0.0009) [2023-12-27 04:20:25,795][105620] Updated weights for policy 1, policy_version 1780889 (0.0008) [2023-12-27 04:20:25,860][105620] Updated weights for policy 1, policy_version 1780899 (0.0008) [2023-12-27 04:20:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19933.9, 300 sec: 19549.7). Total num frames: 910934016. Throughput: 0: 9638.6, 1: 10123.4. Samples: 910940668. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:20:26,063][104569] Avg episode reward: [(0, '8806.803'), (1, '8891.613')] [2023-12-27 04:20:26,431][105692] Updated weights for policy 0, policy_version 1776936 (0.0010) [2023-12-27 04:20:26,487][105692] Updated weights for policy 0, policy_version 1776946 (0.0010) [2023-12-27 04:20:26,502][105620] Updated weights for policy 1, policy_version 1780909 (0.0009) [2023-12-27 04:20:26,542][105692] Updated weights for policy 0, policy_version 1776956 (0.0009) [2023-12-27 04:20:26,559][105620] Updated weights for policy 1, policy_version 1780919 (0.0010) [2023-12-27 04:20:26,622][105620] Updated weights for policy 1, policy_version 1780929 (0.0008) [2023-12-27 04:20:27,195][105620] Updated weights for policy 1, policy_version 1780939 (0.0010) [2023-12-27 04:20:27,229][105692] Updated weights for policy 0, policy_version 1776966 (0.0006) [2023-12-27 04:20:27,254][105620] Updated weights for policy 1, policy_version 1780949 (0.0010) [2023-12-27 04:20:27,287][105692] Updated weights for policy 0, policy_version 1776976 (0.0006) [2023-12-27 04:20:27,315][105620] Updated weights for policy 1, policy_version 1780959 (0.0010) [2023-12-27 04:20:27,342][105692] Updated weights for policy 0, policy_version 1776986 (0.0007) [2023-12-27 04:20:27,917][105620] Updated weights for policy 1, policy_version 1780969 (0.0010) [2023-12-27 04:20:27,939][105692] Updated weights for policy 0, policy_version 1776996 (0.0006) [2023-12-27 04:20:27,966][105620] Updated weights for policy 1, policy_version 1780979 (0.0005) [2023-12-27 04:20:27,998][105692] Updated weights for policy 0, policy_version 1777006 (0.0006) [2023-12-27 04:20:28,016][105620] Updated weights for policy 1, policy_version 1780989 (0.0005) [2023-12-27 04:20:28,046][105692] Updated weights for policy 0, policy_version 1777016 (0.0005) [2023-12-27 04:20:28,067][105620] Updated weights for policy 1, policy_version 1780999 (0.0005) [2023-12-27 04:20:28,665][105620] Updated weights for policy 1, policy_version 1781009 (0.0010) [2023-12-27 04:20:28,713][105620] Updated weights for policy 1, policy_version 1781019 (0.0010) [2023-12-27 04:20:28,714][105692] Updated weights for policy 0, policy_version 1777026 (0.0005) [2023-12-27 04:20:28,765][105692] Updated weights for policy 0, policy_version 1777036 (0.0006) [2023-12-27 04:20:28,766][105620] Updated weights for policy 1, policy_version 1781029 (0.0010) [2023-12-27 04:20:28,819][105692] Updated weights for policy 0, policy_version 1777046 (0.0006) [2023-12-27 04:20:28,877][105692] Updated weights for policy 0, policy_version 1777056 (0.0006) [2023-12-27 04:20:29,484][105620] Updated weights for policy 1, policy_version 1781039 (0.0007) [2023-12-27 04:20:29,533][105620] Updated weights for policy 1, policy_version 1781049 (0.0006) [2023-12-27 04:20:29,581][105620] Updated weights for policy 1, policy_version 1781059 (0.0010) [2023-12-27 04:20:29,644][105692] Updated weights for policy 0, policy_version 1777066 (0.0010) [2023-12-27 04:20:29,701][105692] Updated weights for policy 0, policy_version 1777077 (0.0010) [2023-12-27 04:20:29,756][105692] Updated weights for policy 0, policy_version 1777087 (0.0009) [2023-12-27 04:20:30,221][105620] Updated weights for policy 1, policy_version 1781069 (0.0005) [2023-12-27 04:20:30,286][105620] Updated weights for policy 1, policy_version 1781079 (0.0005) [2023-12-27 04:20:30,353][105620] Updated weights for policy 1, policy_version 1781089 (0.0005) [2023-12-27 04:20:30,623][105692] Updated weights for policy 0, policy_version 1777097 (0.0009) [2023-12-27 04:20:30,667][105692] Updated weights for policy 0, policy_version 1777107 (0.0008) [2023-12-27 04:20:30,716][105692] Updated weights for policy 0, policy_version 1777117 (0.0008) [2023-12-27 04:20:30,945][105620] Updated weights for policy 1, policy_version 1781099 (0.0008) [2023-12-27 04:20:31,002][105620] Updated weights for policy 1, policy_version 1781109 (0.0010) [2023-12-27 04:20:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19521.9). Total num frames: 911032320. Throughput: 0: 9716.1, 1: 10205.1. Samples: 911005108. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:20:31,063][104569] Avg episode reward: [(0, '8717.713'), (1, '9168.421')] [2023-12-27 04:20:31,063][105620] Updated weights for policy 1, policy_version 1781119 (0.0009) [2023-12-27 04:20:31,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001777120_455008256.pth... [2023-12-27 04:20:31,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001776000_454721536.pth [2023-12-27 04:20:31,106][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001781128_456032256.pth... [2023-12-27 04:20:31,110][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001779944_455729152.pth [2023-12-27 04:20:31,511][105692] Updated weights for policy 0, policy_version 1777127 (0.0007) [2023-12-27 04:20:31,576][105692] Updated weights for policy 0, policy_version 1777137 (0.0008) [2023-12-27 04:20:31,632][105692] Updated weights for policy 0, policy_version 1777147 (0.0008) [2023-12-27 04:20:31,789][105620] Updated weights for policy 1, policy_version 1781129 (0.0009) [2023-12-27 04:20:31,847][105620] Updated weights for policy 1, policy_version 1781139 (0.0007) [2023-12-27 04:20:31,908][105620] Updated weights for policy 1, policy_version 1781149 (0.0005) [2023-12-27 04:20:31,965][105620] Updated weights for policy 1, policy_version 1781159 (0.0005) [2023-12-27 04:20:32,481][105692] Updated weights for policy 0, policy_version 1777157 (0.0008) [2023-12-27 04:20:32,529][105692] Updated weights for policy 0, policy_version 1777167 (0.0009) [2023-12-27 04:20:32,539][105620] Updated weights for policy 1, policy_version 1781169 (0.0007) [2023-12-27 04:20:32,578][105692] Updated weights for policy 0, policy_version 1777177 (0.0006) [2023-12-27 04:20:32,598][105620] Updated weights for policy 1, policy_version 1781179 (0.0010) [2023-12-27 04:20:32,661][105620] Updated weights for policy 1, policy_version 1781189 (0.0009) [2023-12-27 04:20:33,365][105692] Updated weights for policy 0, policy_version 1777187 (0.0008) [2023-12-27 04:20:33,376][105620] Updated weights for policy 1, policy_version 1781199 (0.0007) [2023-12-27 04:20:33,417][105692] Updated weights for policy 0, policy_version 1777197 (0.0006) [2023-12-27 04:20:33,435][105620] Updated weights for policy 1, policy_version 1781209 (0.0007) [2023-12-27 04:20:33,473][105692] Updated weights for policy 0, policy_version 1777207 (0.0006) [2023-12-27 04:20:33,490][105620] Updated weights for policy 1, policy_version 1781219 (0.0007) [2023-12-27 04:20:34,037][105692] Updated weights for policy 0, policy_version 1777217 (0.0006) [2023-12-27 04:20:34,089][105692] Updated weights for policy 0, policy_version 1777227 (0.0005) [2023-12-27 04:20:34,148][105692] Updated weights for policy 0, policy_version 1777237 (0.0007) [2023-12-27 04:20:34,201][105692] Updated weights for policy 0, policy_version 1777247 (0.0009) [2023-12-27 04:20:34,224][105620] Updated weights for policy 1, policy_version 1781229 (0.0007) [2023-12-27 04:20:34,286][105620] Updated weights for policy 1, policy_version 1781239 (0.0009) [2023-12-27 04:20:34,350][105620] Updated weights for policy 1, policy_version 1781249 (0.0009) [2023-12-27 04:20:34,919][105692] Updated weights for policy 0, policy_version 1777257 (0.0005) [2023-12-27 04:20:34,990][105692] Updated weights for policy 0, policy_version 1777267 (0.0006) [2023-12-27 04:20:35,052][105692] Updated weights for policy 0, policy_version 1777277 (0.0006) [2023-12-27 04:20:35,112][105620] Updated weights for policy 1, policy_version 1781259 (0.0009) [2023-12-27 04:20:35,167][105620] Updated weights for policy 1, policy_version 1781269 (0.0008) [2023-12-27 04:20:35,236][105620] Updated weights for policy 1, policy_version 1781279 (0.0008) [2023-12-27 04:20:35,712][105692] Updated weights for policy 0, policy_version 1777287 (0.0008) [2023-12-27 04:20:35,774][105692] Updated weights for policy 0, policy_version 1777297 (0.0008) [2023-12-27 04:20:35,835][105692] Updated weights for policy 0, policy_version 1777307 (0.0008) [2023-12-27 04:20:35,978][105620] Updated weights for policy 1, policy_version 1781289 (0.0008) [2023-12-27 04:20:36,031][105620] Updated weights for policy 1, policy_version 1781299 (0.0009) [2023-12-27 04:20:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 911130624. Throughput: 0: 9611.3, 1: 10129.8. Samples: 911122072. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:20:36,062][104569] Avg episode reward: [(0, '8627.242'), (1, '9260.846')] [2023-12-27 04:20:36,083][105620] Updated weights for policy 1, policy_version 1781309 (0.0009) [2023-12-27 04:20:36,145][105620] Updated weights for policy 1, policy_version 1781319 (0.0008) [2023-12-27 04:20:36,532][105692] Updated weights for policy 0, policy_version 1777317 (0.0008) [2023-12-27 04:20:36,598][105692] Updated weights for policy 0, policy_version 1777327 (0.0008) [2023-12-27 04:20:36,660][105692] Updated weights for policy 0, policy_version 1777337 (0.0009) [2023-12-27 04:20:36,939][105620] Updated weights for policy 1, policy_version 1781329 (0.0009) [2023-12-27 04:20:36,994][105620] Updated weights for policy 1, policy_version 1781339 (0.0009) [2023-12-27 04:20:37,053][105620] Updated weights for policy 1, policy_version 1781349 (0.0009) [2023-12-27 04:20:37,358][105692] Updated weights for policy 0, policy_version 1777347 (0.0009) [2023-12-27 04:20:37,408][105692] Updated weights for policy 0, policy_version 1777357 (0.0008) [2023-12-27 04:20:37,463][105692] Updated weights for policy 0, policy_version 1777367 (0.0009) [2023-12-27 04:20:37,733][105620] Updated weights for policy 1, policy_version 1781359 (0.0009) [2023-12-27 04:20:37,790][105620] Updated weights for policy 1, policy_version 1781369 (0.0010) [2023-12-27 04:20:37,842][105620] Updated weights for policy 1, policy_version 1781379 (0.0009) [2023-12-27 04:20:38,171][105692] Updated weights for policy 0, policy_version 1777377 (0.0009) [2023-12-27 04:20:38,240][105692] Updated weights for policy 0, policy_version 1777387 (0.0008) [2023-12-27 04:20:38,303][105692] Updated weights for policy 0, policy_version 1777397 (0.0008) [2023-12-27 04:20:38,371][105692] Updated weights for policy 0, policy_version 1777407 (0.0009) [2023-12-27 04:20:38,609][105620] Updated weights for policy 1, policy_version 1781389 (0.0009) [2023-12-27 04:20:38,667][105620] Updated weights for policy 1, policy_version 1781399 (0.0009) [2023-12-27 04:20:38,734][105620] Updated weights for policy 1, policy_version 1781409 (0.0010) [2023-12-27 04:20:39,020][105692] Updated weights for policy 0, policy_version 1777417 (0.0008) [2023-12-27 04:20:39,079][105692] Updated weights for policy 0, policy_version 1777427 (0.0009) [2023-12-27 04:20:39,136][105692] Updated weights for policy 0, policy_version 1777437 (0.0009) [2023-12-27 04:20:39,624][105620] Updated weights for policy 1, policy_version 1781419 (0.0010) [2023-12-27 04:20:39,683][105620] Updated weights for policy 1, policy_version 1781429 (0.0009) [2023-12-27 04:20:39,745][105620] Updated weights for policy 1, policy_version 1781439 (0.0008) [2023-12-27 04:20:39,875][105692] Updated weights for policy 0, policy_version 1777447 (0.0007) [2023-12-27 04:20:39,941][105692] Updated weights for policy 0, policy_version 1777457 (0.0009) [2023-12-27 04:20:40,006][105692] Updated weights for policy 0, policy_version 1777467 (0.0008) [2023-12-27 04:20:40,514][105620] Updated weights for policy 1, policy_version 1781449 (0.0009) [2023-12-27 04:20:40,565][105620] Updated weights for policy 1, policy_version 1781459 (0.0007) [2023-12-27 04:20:40,614][105620] Updated weights for policy 1, policy_version 1781469 (0.0008) [2023-12-27 04:20:40,666][105620] Updated weights for policy 1, policy_version 1781479 (0.0009) [2023-12-27 04:20:40,709][105692] Updated weights for policy 0, policy_version 1777477 (0.0009) [2023-12-27 04:20:40,770][105692] Updated weights for policy 0, policy_version 1777487 (0.0010) [2023-12-27 04:20:40,834][105692] Updated weights for policy 0, policy_version 1777497 (0.0008) [2023-12-27 04:20:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 911228928. Throughput: 0: 9596.4, 1: 10020.5. Samples: 911235164. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:20:41,062][104569] Avg episode reward: [(0, '8627.436'), (1, '9260.810')] [2023-12-27 04:20:41,464][105620] Updated weights for policy 1, policy_version 1781489 (0.0010) [2023-12-27 04:20:41,506][105692] Updated weights for policy 0, policy_version 1777507 (0.0009) [2023-12-27 04:20:41,521][105620] Updated weights for policy 1, policy_version 1781499 (0.0008) [2023-12-27 04:20:41,565][105692] Updated weights for policy 0, policy_version 1777517 (0.0006) [2023-12-27 04:20:41,588][105620] Updated weights for policy 1, policy_version 1781509 (0.0009) [2023-12-27 04:20:41,634][105692] Updated weights for policy 0, policy_version 1777527 (0.0008) [2023-12-27 04:20:42,370][105692] Updated weights for policy 0, policy_version 1777537 (0.0010) [2023-12-27 04:20:42,398][105620] Updated weights for policy 1, policy_version 1781519 (0.0009) [2023-12-27 04:20:42,439][105692] Updated weights for policy 0, policy_version 1777547 (0.0010) [2023-12-27 04:20:42,458][105620] Updated weights for policy 1, policy_version 1781529 (0.0008) [2023-12-27 04:20:42,497][105692] Updated weights for policy 0, policy_version 1777557 (0.0007) [2023-12-27 04:20:42,514][105620] Updated weights for policy 1, policy_version 1781539 (0.0009) [2023-12-27 04:20:42,552][105692] Updated weights for policy 0, policy_version 1777567 (0.0005) [2023-12-27 04:20:43,214][105692] Updated weights for policy 0, policy_version 1777577 (0.0006) [2023-12-27 04:20:43,269][105692] Updated weights for policy 0, policy_version 1777587 (0.0005) [2023-12-27 04:20:43,315][105620] Updated weights for policy 1, policy_version 1781549 (0.0008) [2023-12-27 04:20:43,329][105692] Updated weights for policy 0, policy_version 1777597 (0.0009) [2023-12-27 04:20:43,367][105620] Updated weights for policy 1, policy_version 1781559 (0.0006) [2023-12-27 04:20:43,432][105620] Updated weights for policy 1, policy_version 1781569 (0.0006) [2023-12-27 04:20:43,896][105692] Updated weights for policy 0, policy_version 1777607 (0.0007) [2023-12-27 04:20:43,954][105692] Updated weights for policy 0, policy_version 1777617 (0.0005) [2023-12-27 04:20:44,011][105692] Updated weights for policy 0, policy_version 1777627 (0.0005) [2023-12-27 04:20:44,274][105620] Updated weights for policy 1, policy_version 1781579 (0.0010) [2023-12-27 04:20:44,345][105620] Updated weights for policy 1, policy_version 1781589 (0.0008) [2023-12-27 04:20:44,398][105620] Updated weights for policy 1, policy_version 1781599 (0.0009) [2023-12-27 04:20:44,602][105692] Updated weights for policy 0, policy_version 1777637 (0.0005) [2023-12-27 04:20:44,666][105692] Updated weights for policy 0, policy_version 1777647 (0.0005) [2023-12-27 04:20:44,737][105692] Updated weights for policy 0, policy_version 1777657 (0.0005) [2023-12-27 04:20:45,176][105620] Updated weights for policy 1, policy_version 1781609 (0.0010) [2023-12-27 04:20:45,239][105620] Updated weights for policy 1, policy_version 1781619 (0.0009) [2023-12-27 04:20:45,304][105620] Updated weights for policy 1, policy_version 1781629 (0.0008) [2023-12-27 04:20:45,370][105620] Updated weights for policy 1, policy_version 1781639 (0.0008) [2023-12-27 04:20:45,373][105692] Updated weights for policy 0, policy_version 1777667 (0.0008) [2023-12-27 04:20:45,421][105692] Updated weights for policy 0, policy_version 1777677 (0.0010) [2023-12-27 04:20:45,477][105692] Updated weights for policy 0, policy_version 1777687 (0.0010) [2023-12-27 04:20:46,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 911319040. Throughput: 0: 9643.3, 1: 9847.8. Samples: 911291444. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:20:46,063][104569] Avg episode reward: [(0, '8447.103'), (1, '9170.180')] [2023-12-27 04:20:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001777696_455155712.pth... [2023-12-27 04:20:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001776544_454860800.pth [2023-12-27 04:20:46,138][105620] Updated weights for policy 1, policy_version 1781649 (0.0009) [2023-12-27 04:20:46,165][105692] Updated weights for policy 0, policy_version 1777697 (0.0010) [2023-12-27 04:20:46,200][105620] Updated weights for policy 1, policy_version 1781659 (0.0009) [2023-12-27 04:20:46,219][105692] Updated weights for policy 0, policy_version 1777707 (0.0005) [2023-12-27 04:20:46,257][105620] Updated weights for policy 1, policy_version 1781669 (0.0008) [2023-12-27 04:20:46,273][105692] Updated weights for policy 0, policy_version 1777717 (0.0005) [2023-12-27 04:20:46,273][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001781672_456171520.pth... [2023-12-27 04:20:46,278][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001780552_455884800.pth [2023-12-27 04:20:46,332][105692] Updated weights for policy 0, policy_version 1777727 (0.0005) [2023-12-27 04:20:46,940][105692] Updated weights for policy 0, policy_version 1777737 (0.0010) [2023-12-27 04:20:46,984][105692] Updated weights for policy 0, policy_version 1777747 (0.0007) [2023-12-27 04:20:47,028][105692] Updated weights for policy 0, policy_version 1777757 (0.0005) [2023-12-27 04:20:47,045][105620] Updated weights for policy 1, policy_version 1781679 (0.0008) [2023-12-27 04:20:47,102][105620] Updated weights for policy 1, policy_version 1781689 (0.0009) [2023-12-27 04:20:47,155][105620] Updated weights for policy 1, policy_version 1781699 (0.0005) [2023-12-27 04:20:47,674][105692] Updated weights for policy 0, policy_version 1777767 (0.0005) [2023-12-27 04:20:47,727][105692] Updated weights for policy 0, policy_version 1777777 (0.0006) [2023-12-27 04:20:47,784][105692] Updated weights for policy 0, policy_version 1777787 (0.0009) [2023-12-27 04:20:47,876][105620] Updated weights for policy 1, policy_version 1781709 (0.0008) [2023-12-27 04:20:47,925][105620] Updated weights for policy 1, policy_version 1781720 (0.0009) [2023-12-27 04:20:47,970][105620] Updated weights for policy 1, policy_version 1781730 (0.0008) [2023-12-27 04:20:48,488][105692] Updated weights for policy 0, policy_version 1777797 (0.0008) [2023-12-27 04:20:48,546][105692] Updated weights for policy 0, policy_version 1777807 (0.0010) [2023-12-27 04:20:48,608][105692] Updated weights for policy 0, policy_version 1777817 (0.0010) [2023-12-27 04:20:48,760][105620] Updated weights for policy 1, policy_version 1781740 (0.0009) [2023-12-27 04:20:48,818][105620] Updated weights for policy 1, policy_version 1781750 (0.0011) [2023-12-27 04:20:48,885][105620] Updated weights for policy 1, policy_version 1781760 (0.0011) [2023-12-27 04:20:49,281][105692] Updated weights for policy 0, policy_version 1777827 (0.0009) [2023-12-27 04:20:49,346][105692] Updated weights for policy 0, policy_version 1777837 (0.0007) [2023-12-27 04:20:49,412][105692] Updated weights for policy 0, policy_version 1777847 (0.0009) [2023-12-27 04:20:49,610][105620] Updated weights for policy 1, policy_version 1781770 (0.0011) [2023-12-27 04:20:49,669][105620] Updated weights for policy 1, policy_version 1781780 (0.0011) [2023-12-27 04:20:49,731][105620] Updated weights for policy 1, policy_version 1781790 (0.0011) [2023-12-27 04:20:49,792][105620] Updated weights for policy 1, policy_version 1781800 (0.0010) [2023-12-27 04:20:50,087][105692] Updated weights for policy 0, policy_version 1777857 (0.0008) [2023-12-27 04:20:50,150][105692] Updated weights for policy 0, policy_version 1777867 (0.0008) [2023-12-27 04:20:50,213][105692] Updated weights for policy 0, policy_version 1777877 (0.0008) [2023-12-27 04:20:50,273][105692] Updated weights for policy 0, policy_version 1777887 (0.0008) [2023-12-27 04:20:50,537][105620] Updated weights for policy 1, policy_version 1781810 (0.0011) [2023-12-27 04:20:50,598][105620] Updated weights for policy 1, policy_version 1781820 (0.0009) [2023-12-27 04:20:50,656][105620] Updated weights for policy 1, policy_version 1781830 (0.0007) [2023-12-27 04:20:51,025][105692] Updated weights for policy 0, policy_version 1777897 (0.0009) [2023-12-27 04:20:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 911417344. Throughput: 0: 9809.4, 1: 9707.4. Samples: 911410292. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:20:51,062][104569] Avg episode reward: [(0, '8627.070'), (1, '9077.937')] [2023-12-27 04:20:51,088][105692] Updated weights for policy 0, policy_version 1777907 (0.0009) [2023-12-27 04:20:51,151][105692] Updated weights for policy 0, policy_version 1777917 (0.0009) [2023-12-27 04:20:51,277][105620] Updated weights for policy 1, policy_version 1781840 (0.0006) [2023-12-27 04:20:51,334][105620] Updated weights for policy 1, policy_version 1781850 (0.0007) [2023-12-27 04:20:51,402][105620] Updated weights for policy 1, policy_version 1781860 (0.0007) [2023-12-27 04:20:51,955][105692] Updated weights for policy 0, policy_version 1777927 (0.0008) [2023-12-27 04:20:52,008][105692] Updated weights for policy 0, policy_version 1777937 (0.0009) [2023-12-27 04:20:52,058][105692] Updated weights for policy 0, policy_version 1777947 (0.0008) [2023-12-27 04:20:52,116][105620] Updated weights for policy 1, policy_version 1781870 (0.0008) [2023-12-27 04:20:52,175][105620] Updated weights for policy 1, policy_version 1781880 (0.0009) [2023-12-27 04:20:52,241][105620] Updated weights for policy 1, policy_version 1781890 (0.0009) [2023-12-27 04:20:52,822][105692] Updated weights for policy 0, policy_version 1777957 (0.0008) [2023-12-27 04:20:52,874][105692] Updated weights for policy 0, policy_version 1777967 (0.0010) [2023-12-27 04:20:52,932][105692] Updated weights for policy 0, policy_version 1777977 (0.0009) [2023-12-27 04:20:53,010][105620] Updated weights for policy 1, policy_version 1781900 (0.0008) [2023-12-27 04:20:53,063][105620] Updated weights for policy 1, policy_version 1781910 (0.0008) [2023-12-27 04:20:53,117][105620] Updated weights for policy 1, policy_version 1781920 (0.0005) [2023-12-27 04:20:53,641][105620] Updated weights for policy 1, policy_version 1781930 (0.0005) [2023-12-27 04:20:53,688][105620] Updated weights for policy 1, policy_version 1781940 (0.0006) [2023-12-27 04:20:53,739][105620] Updated weights for policy 1, policy_version 1781950 (0.0009) [2023-12-27 04:20:53,790][105620] Updated weights for policy 1, policy_version 1781960 (0.0009) [2023-12-27 04:20:53,812][105692] Updated weights for policy 0, policy_version 1777987 (0.0010) [2023-12-27 04:20:53,868][105692] Updated weights for policy 0, policy_version 1777997 (0.0009) [2023-12-27 04:20:53,930][105692] Updated weights for policy 0, policy_version 1778007 (0.0009) [2023-12-27 04:20:54,548][105620] Updated weights for policy 1, policy_version 1781970 (0.0010) [2023-12-27 04:20:54,600][105620] Updated weights for policy 1, policy_version 1781980 (0.0010) [2023-12-27 04:20:54,652][105620] Updated weights for policy 1, policy_version 1781990 (0.0010) [2023-12-27 04:20:54,702][105692] Updated weights for policy 0, policy_version 1778017 (0.0008) [2023-12-27 04:20:54,761][105692] Updated weights for policy 0, policy_version 1778027 (0.0007) [2023-12-27 04:20:54,814][105692] Updated weights for policy 0, policy_version 1778037 (0.0005) [2023-12-27 04:20:54,880][105692] Updated weights for policy 0, policy_version 1778047 (0.0007) [2023-12-27 04:20:55,415][105620] Updated weights for policy 1, policy_version 1782000 (0.0010) [2023-12-27 04:20:55,470][105620] Updated weights for policy 1, policy_version 1782010 (0.0010) [2023-12-27 04:20:55,508][105692] Updated weights for policy 0, policy_version 1778057 (0.0006) [2023-12-27 04:20:55,524][105620] Updated weights for policy 1, policy_version 1782020 (0.0010) [2023-12-27 04:20:55,557][105692] Updated weights for policy 0, policy_version 1778067 (0.0005) [2023-12-27 04:20:55,610][105692] Updated weights for policy 0, policy_version 1778077 (0.0005) [2023-12-27 04:20:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 911515648. Throughput: 0: 9831.6, 1: 9697.2. Samples: 911526272. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:20:56,063][104569] Avg episode reward: [(0, '8078.774'), (1, '9168.303')] [2023-12-27 04:20:56,175][105620] Updated weights for policy 1, policy_version 1782030 (0.0010) [2023-12-27 04:20:56,227][105620] Updated weights for policy 1, policy_version 1782040 (0.0010) [2023-12-27 04:20:56,233][105692] Updated weights for policy 0, policy_version 1778087 (0.0008) [2023-12-27 04:20:56,278][105692] Updated weights for policy 0, policy_version 1778097 (0.0006) [2023-12-27 04:20:56,279][105620] Updated weights for policy 1, policy_version 1782050 (0.0010) [2023-12-27 04:20:56,325][105692] Updated weights for policy 0, policy_version 1778107 (0.0008) [2023-12-27 04:20:56,995][105620] Updated weights for policy 1, policy_version 1782060 (0.0010) [2023-12-27 04:20:57,043][105692] Updated weights for policy 0, policy_version 1778117 (0.0006) [2023-12-27 04:20:57,048][105620] Updated weights for policy 1, policy_version 1782070 (0.0009) [2023-12-27 04:20:57,100][105692] Updated weights for policy 0, policy_version 1778127 (0.0007) [2023-12-27 04:20:57,114][105620] Updated weights for policy 1, policy_version 1782080 (0.0008) [2023-12-27 04:20:57,169][105692] Updated weights for policy 0, policy_version 1778137 (0.0006) [2023-12-27 04:20:57,730][105620] Updated weights for policy 1, policy_version 1782090 (0.0008) [2023-12-27 04:20:57,789][105620] Updated weights for policy 1, policy_version 1782100 (0.0009) [2023-12-27 04:20:57,811][105692] Updated weights for policy 0, policy_version 1778147 (0.0007) [2023-12-27 04:20:57,841][105620] Updated weights for policy 1, policy_version 1782110 (0.0005) [2023-12-27 04:20:57,861][105692] Updated weights for policy 0, policy_version 1778157 (0.0007) [2023-12-27 04:20:57,889][105620] Updated weights for policy 1, policy_version 1782120 (0.0005) [2023-12-27 04:20:57,924][105692] Updated weights for policy 0, policy_version 1778167 (0.0008) [2023-12-27 04:20:58,582][105620] Updated weights for policy 1, policy_version 1782130 (0.0007) [2023-12-27 04:20:58,626][105692] Updated weights for policy 0, policy_version 1778177 (0.0009) [2023-12-27 04:20:58,637][105620] Updated weights for policy 1, policy_version 1782140 (0.0008) [2023-12-27 04:20:58,686][105692] Updated weights for policy 0, policy_version 1778187 (0.0008) [2023-12-27 04:20:58,698][105620] Updated weights for policy 1, policy_version 1782150 (0.0008) [2023-12-27 04:20:58,749][105692] Updated weights for policy 0, policy_version 1778197 (0.0008) [2023-12-27 04:20:58,817][105692] Updated weights for policy 0, policy_version 1778207 (0.0008) [2023-12-27 04:20:59,457][105620] Updated weights for policy 1, policy_version 1782160 (0.0009) [2023-12-27 04:20:59,510][105620] Updated weights for policy 1, policy_version 1782170 (0.0009) [2023-12-27 04:20:59,561][105620] Updated weights for policy 1, policy_version 1782181 (0.0010) [2023-12-27 04:20:59,618][105692] Updated weights for policy 0, policy_version 1778217 (0.0007) [2023-12-27 04:20:59,673][105692] Updated weights for policy 0, policy_version 1778227 (0.0009) [2023-12-27 04:20:59,728][105692] Updated weights for policy 0, policy_version 1778237 (0.0009) [2023-12-27 04:21:00,264][105620] Updated weights for policy 1, policy_version 1782191 (0.0007) [2023-12-27 04:21:00,315][105620] Updated weights for policy 1, policy_version 1782201 (0.0005) [2023-12-27 04:21:00,374][105620] Updated weights for policy 1, policy_version 1782211 (0.0005) [2023-12-27 04:21:00,480][105692] Updated weights for policy 0, policy_version 1778247 (0.0010) [2023-12-27 04:21:00,531][105692] Updated weights for policy 0, policy_version 1778257 (0.0009) [2023-12-27 04:21:00,592][105692] Updated weights for policy 0, policy_version 1778267 (0.0008) [2023-12-27 04:21:00,901][105620] Updated weights for policy 1, policy_version 1782221 (0.0005) [2023-12-27 04:21:00,950][105620] Updated weights for policy 1, policy_version 1782231 (0.0005) [2023-12-27 04:21:01,000][105620] Updated weights for policy 1, policy_version 1782241 (0.0005) [2023-12-27 04:21:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 911622144. Throughput: 0: 9859.1, 1: 9674.0. Samples: 911587172. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:21:01,063][104569] Avg episode reward: [(0, '8083.033'), (1, '9352.076')] [2023-12-27 04:21:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001778272_455303168.pth... [2023-12-27 04:21:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001782248_456318976.pth... [2023-12-27 04:21:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001777120_455008256.pth [2023-12-27 04:21:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001781128_456032256.pth [2023-12-27 04:21:01,422][105692] Updated weights for policy 0, policy_version 1778277 (0.0009) [2023-12-27 04:21:01,475][105692] Updated weights for policy 0, policy_version 1778287 (0.0008) [2023-12-27 04:21:01,527][105692] Updated weights for policy 0, policy_version 1778297 (0.0008) [2023-12-27 04:21:01,714][105620] Updated weights for policy 1, policy_version 1782251 (0.0008) [2023-12-27 04:21:01,780][105620] Updated weights for policy 1, policy_version 1782261 (0.0006) [2023-12-27 04:21:01,843][105620] Updated weights for policy 1, policy_version 1782271 (0.0005) [2023-12-27 04:21:02,261][105692] Updated weights for policy 0, policy_version 1778307 (0.0008) [2023-12-27 04:21:02,325][105692] Updated weights for policy 0, policy_version 1778317 (0.0009) [2023-12-27 04:21:02,388][105692] Updated weights for policy 0, policy_version 1778327 (0.0008) [2023-12-27 04:21:02,474][105620] Updated weights for policy 1, policy_version 1782281 (0.0006) [2023-12-27 04:21:02,530][105620] Updated weights for policy 1, policy_version 1782291 (0.0006) [2023-12-27 04:21:02,590][105620] Updated weights for policy 1, policy_version 1782301 (0.0007) [2023-12-27 04:21:02,648][105620] Updated weights for policy 1, policy_version 1782311 (0.0005) [2023-12-27 04:21:03,109][105692] Updated weights for policy 0, policy_version 1778337 (0.0008) [2023-12-27 04:21:03,176][105692] Updated weights for policy 0, policy_version 1778347 (0.0006) [2023-12-27 04:21:03,232][105692] Updated weights for policy 0, policy_version 1778357 (0.0011) [2023-12-27 04:21:03,258][105620] Updated weights for policy 1, policy_version 1782321 (0.0010) [2023-12-27 04:21:03,287][105692] Updated weights for policy 0, policy_version 1778367 (0.0009) [2023-12-27 04:21:03,307][105620] Updated weights for policy 1, policy_version 1782331 (0.0010) [2023-12-27 04:21:03,358][105620] Updated weights for policy 1, policy_version 1782341 (0.0010) [2023-12-27 04:21:03,996][105692] Updated weights for policy 0, policy_version 1778377 (0.0007) [2023-12-27 04:21:03,999][105620] Updated weights for policy 1, policy_version 1782351 (0.0010) [2023-12-27 04:21:04,054][105692] Updated weights for policy 0, policy_version 1778387 (0.0007) [2023-12-27 04:21:04,061][105620] Updated weights for policy 1, policy_version 1782361 (0.0009) [2023-12-27 04:21:04,114][105620] Updated weights for policy 1, policy_version 1782371 (0.0006) [2023-12-27 04:21:04,118][105692] Updated weights for policy 0, policy_version 1778397 (0.0011) [2023-12-27 04:21:04,826][105620] Updated weights for policy 1, policy_version 1782381 (0.0008) [2023-12-27 04:21:04,870][105692] Updated weights for policy 0, policy_version 1778407 (0.0011) [2023-12-27 04:21:04,874][105620] Updated weights for policy 1, policy_version 1782391 (0.0011) [2023-12-27 04:21:04,921][105692] Updated weights for policy 0, policy_version 1778417 (0.0010) [2023-12-27 04:21:04,923][105620] Updated weights for policy 1, policy_version 1782401 (0.0010) [2023-12-27 04:21:04,969][105692] Updated weights for policy 0, policy_version 1778427 (0.0010) [2023-12-27 04:21:05,697][105620] Updated weights for policy 1, policy_version 1782411 (0.0010) [2023-12-27 04:21:05,747][105692] Updated weights for policy 0, policy_version 1778437 (0.0010) [2023-12-27 04:21:05,760][105620] Updated weights for policy 1, policy_version 1782421 (0.0011) [2023-12-27 04:21:05,803][105692] Updated weights for policy 0, policy_version 1778447 (0.0010) [2023-12-27 04:21:05,827][105620] Updated weights for policy 1, policy_version 1782431 (0.0011) [2023-12-27 04:21:05,859][105692] Updated weights for policy 0, policy_version 1778457 (0.0010) [2023-12-27 04:21:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 911720448. Throughput: 0: 9753.6, 1: 9767.2. Samples: 911705696. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:21:06,063][104569] Avg episode reward: [(0, '8625.504'), (1, '9075.043')] [2023-12-27 04:21:06,571][105620] Updated weights for policy 1, policy_version 1782441 (0.0011) [2023-12-27 04:21:06,596][105692] Updated weights for policy 0, policy_version 1778467 (0.0011) [2023-12-27 04:21:06,627][105620] Updated weights for policy 1, policy_version 1782451 (0.0011) [2023-12-27 04:21:06,652][105692] Updated weights for policy 0, policy_version 1778477 (0.0010) [2023-12-27 04:21:06,685][105620] Updated weights for policy 1, policy_version 1782461 (0.0006) [2023-12-27 04:21:06,715][105692] Updated weights for policy 0, policy_version 1778487 (0.0011) [2023-12-27 04:21:06,745][105620] Updated weights for policy 1, policy_version 1782471 (0.0007) [2023-12-27 04:21:07,460][105692] Updated weights for policy 0, policy_version 1778497 (0.0010) [2023-12-27 04:21:07,470][105620] Updated weights for policy 1, policy_version 1782481 (0.0011) [2023-12-27 04:21:07,523][105692] Updated weights for policy 0, policy_version 1778507 (0.0008) [2023-12-27 04:21:07,526][105620] Updated weights for policy 1, policy_version 1782491 (0.0011) [2023-12-27 04:21:07,574][105620] Updated weights for policy 1, policy_version 1782501 (0.0010) [2023-12-27 04:21:07,578][105692] Updated weights for policy 0, policy_version 1778517 (0.0010) [2023-12-27 04:21:07,653][105692] Updated weights for policy 0, policy_version 1778527 (0.0010) [2023-12-27 04:21:08,288][105620] Updated weights for policy 1, policy_version 1782511 (0.0010) [2023-12-27 04:21:08,354][105620] Updated weights for policy 1, policy_version 1782521 (0.0011) [2023-12-27 04:21:08,359][105692] Updated weights for policy 0, policy_version 1778537 (0.0009) [2023-12-27 04:21:08,414][105620] Updated weights for policy 1, policy_version 1782531 (0.0011) [2023-12-27 04:21:08,422][105692] Updated weights for policy 0, policy_version 1778547 (0.0009) [2023-12-27 04:21:08,478][105692] Updated weights for policy 0, policy_version 1778557 (0.0011) [2023-12-27 04:21:09,069][105620] Updated weights for policy 1, policy_version 1782541 (0.0011) [2023-12-27 04:21:09,121][105620] Updated weights for policy 1, policy_version 1782551 (0.0010) [2023-12-27 04:21:09,169][105620] Updated weights for policy 1, policy_version 1782561 (0.0010) [2023-12-27 04:21:09,250][105692] Updated weights for policy 0, policy_version 1778567 (0.0010) [2023-12-27 04:21:09,309][105692] Updated weights for policy 0, policy_version 1778577 (0.0010) [2023-12-27 04:21:09,376][105692] Updated weights for policy 0, policy_version 1778587 (0.0009) [2023-12-27 04:21:09,956][105620] Updated weights for policy 1, policy_version 1782571 (0.0010) [2023-12-27 04:21:10,028][105620] Updated weights for policy 1, policy_version 1782581 (0.0011) [2023-12-27 04:21:10,082][105620] Updated weights for policy 1, policy_version 1782591 (0.0011) [2023-12-27 04:21:10,141][105692] Updated weights for policy 0, policy_version 1778597 (0.0007) [2023-12-27 04:21:10,206][105692] Updated weights for policy 0, policy_version 1778607 (0.0009) [2023-12-27 04:21:10,270][105692] Updated weights for policy 0, policy_version 1778617 (0.0006) [2023-12-27 04:21:10,793][105620] Updated weights for policy 1, policy_version 1782601 (0.0010) [2023-12-27 04:21:10,851][105620] Updated weights for policy 1, policy_version 1782611 (0.0005) [2023-12-27 04:21:10,899][105620] Updated weights for policy 1, policy_version 1782621 (0.0005) [2023-12-27 04:21:10,946][105620] Updated weights for policy 1, policy_version 1782631 (0.0010) [2023-12-27 04:21:11,003][105692] Updated weights for policy 0, policy_version 1778627 (0.0007) [2023-12-27 04:21:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 911810560. Throughput: 0: 9716.2, 1: 9807.3. Samples: 911819224. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:21:11,062][104569] Avg episode reward: [(0, '8987.900'), (1, '8983.707')] [2023-12-27 04:21:11,073][105692] Updated weights for policy 0, policy_version 1778637 (0.0010) [2023-12-27 04:21:11,129][105692] Updated weights for policy 0, policy_version 1778647 (0.0010) [2023-12-27 04:21:11,703][105620] Updated weights for policy 1, policy_version 1782641 (0.0011) [2023-12-27 04:21:11,771][105620] Updated weights for policy 1, policy_version 1782651 (0.0008) [2023-12-27 04:21:11,818][105620] Updated weights for policy 1, policy_version 1782661 (0.0008) [2023-12-27 04:21:11,926][105692] Updated weights for policy 0, policy_version 1778657 (0.0010) [2023-12-27 04:21:11,990][105692] Updated weights for policy 0, policy_version 1778667 (0.0008) [2023-12-27 04:21:12,049][105692] Updated weights for policy 0, policy_version 1778677 (0.0008) [2023-12-27 04:21:12,113][105692] Updated weights for policy 0, policy_version 1778687 (0.0008) [2023-12-27 04:21:12,558][105620] Updated weights for policy 1, policy_version 1782671 (0.0010) [2023-12-27 04:21:12,621][105620] Updated weights for policy 1, policy_version 1782681 (0.0011) [2023-12-27 04:21:12,689][105620] Updated weights for policy 1, policy_version 1782691 (0.0006) [2023-12-27 04:21:12,852][105692] Updated weights for policy 0, policy_version 1778697 (0.0005) [2023-12-27 04:21:12,909][105692] Updated weights for policy 0, policy_version 1778707 (0.0005) [2023-12-27 04:21:12,969][105692] Updated weights for policy 0, policy_version 1778717 (0.0008) [2023-12-27 04:21:13,414][105620] Updated weights for policy 1, policy_version 1782701 (0.0007) [2023-12-27 04:21:13,471][105620] Updated weights for policy 1, policy_version 1782711 (0.0008) [2023-12-27 04:21:13,519][105620] Updated weights for policy 1, policy_version 1782721 (0.0008) [2023-12-27 04:21:13,654][105692] Updated weights for policy 0, policy_version 1778727 (0.0010) [2023-12-27 04:21:13,703][105692] Updated weights for policy 0, policy_version 1778737 (0.0010) [2023-12-27 04:21:13,761][105692] Updated weights for policy 0, policy_version 1778747 (0.0010) [2023-12-27 04:21:14,155][105620] Updated weights for policy 1, policy_version 1782731 (0.0008) [2023-12-27 04:21:14,202][105620] Updated weights for policy 1, policy_version 1782741 (0.0006) [2023-12-27 04:21:14,252][105620] Updated weights for policy 1, policy_version 1782751 (0.0005) [2023-12-27 04:21:14,516][105692] Updated weights for policy 0, policy_version 1778757 (0.0010) [2023-12-27 04:21:14,567][105692] Updated weights for policy 0, policy_version 1778767 (0.0010) [2023-12-27 04:21:14,622][105692] Updated weights for policy 0, policy_version 1778777 (0.0010) [2023-12-27 04:21:14,938][105620] Updated weights for policy 1, policy_version 1782761 (0.0006) [2023-12-27 04:21:15,003][105620] Updated weights for policy 1, policy_version 1782771 (0.0009) [2023-12-27 04:21:15,071][105620] Updated weights for policy 1, policy_version 1782781 (0.0009) [2023-12-27 04:21:15,139][105620] Updated weights for policy 1, policy_version 1782791 (0.0011) [2023-12-27 04:21:15,362][105692] Updated weights for policy 0, policy_version 1778787 (0.0010) [2023-12-27 04:21:15,429][105692] Updated weights for policy 0, policy_version 1778797 (0.0011) [2023-12-27 04:21:15,493][105692] Updated weights for policy 0, policy_version 1778807 (0.0011) [2023-12-27 04:21:15,900][105620] Updated weights for policy 1, policy_version 1782801 (0.0010) [2023-12-27 04:21:15,948][105620] Updated weights for policy 1, policy_version 1782811 (0.0010) [2023-12-27 04:21:15,992][105620] Updated weights for policy 1, policy_version 1782821 (0.0010) [2023-12-27 04:21:16,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 911908864. Throughput: 0: 9652.5, 1: 9691.6. Samples: 911875588. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 04:21:16,062][104569] Avg episode reward: [(0, '8355.925'), (1, '9261.012')] [2023-12-27 04:21:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001778816_455442432.pth... [2023-12-27 04:21:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001782824_456466432.pth... [2023-12-27 04:21:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001777696_455155712.pth [2023-12-27 04:21:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001781672_456171520.pth [2023-12-27 04:21:16,219][105692] Updated weights for policy 0, policy_version 1778817 (0.0010) [2023-12-27 04:21:16,282][105692] Updated weights for policy 0, policy_version 1778827 (0.0007) [2023-12-27 04:21:16,340][105692] Updated weights for policy 0, policy_version 1778837 (0.0008) [2023-12-27 04:21:16,398][105692] Updated weights for policy 0, policy_version 1778847 (0.0009) [2023-12-27 04:21:16,671][105620] Updated weights for policy 1, policy_version 1782831 (0.0007) [2023-12-27 04:21:16,723][105620] Updated weights for policy 1, policy_version 1782841 (0.0005) [2023-12-27 04:21:16,779][105620] Updated weights for policy 1, policy_version 1782851 (0.0007) [2023-12-27 04:21:17,062][105692] Updated weights for policy 0, policy_version 1778857 (0.0006) [2023-12-27 04:21:17,112][105692] Updated weights for policy 0, policy_version 1778867 (0.0007) [2023-12-27 04:21:17,157][105692] Updated weights for policy 0, policy_version 1778877 (0.0008) [2023-12-27 04:21:17,393][105620] Updated weights for policy 1, policy_version 1782861 (0.0008) [2023-12-27 04:21:17,466][105620] Updated weights for policy 1, policy_version 1782871 (0.0008) [2023-12-27 04:21:17,524][105620] Updated weights for policy 1, policy_version 1782881 (0.0010) [2023-12-27 04:21:17,812][105692] Updated weights for policy 0, policy_version 1778887 (0.0008) [2023-12-27 04:21:17,863][105692] Updated weights for policy 0, policy_version 1778897 (0.0008) [2023-12-27 04:21:17,918][105692] Updated weights for policy 0, policy_version 1778907 (0.0008) [2023-12-27 04:21:18,197][105620] Updated weights for policy 1, policy_version 1782891 (0.0011) [2023-12-27 04:21:18,262][105620] Updated weights for policy 1, policy_version 1782901 (0.0010) [2023-12-27 04:21:18,307][105620] Updated weights for policy 1, policy_version 1782911 (0.0010) [2023-12-27 04:21:18,628][105692] Updated weights for policy 0, policy_version 1778917 (0.0009) [2023-12-27 04:21:18,687][105692] Updated weights for policy 0, policy_version 1778927 (0.0011) [2023-12-27 04:21:18,744][105692] Updated weights for policy 0, policy_version 1778937 (0.0010) [2023-12-27 04:21:19,059][105620] Updated weights for policy 1, policy_version 1782921 (0.0011) [2023-12-27 04:21:19,117][105620] Updated weights for policy 1, policy_version 1782931 (0.0010) [2023-12-27 04:21:19,172][105620] Updated weights for policy 1, policy_version 1782941 (0.0010) [2023-12-27 04:21:19,237][105620] Updated weights for policy 1, policy_version 1782951 (0.0011) [2023-12-27 04:21:19,511][105692] Updated weights for policy 0, policy_version 1778947 (0.0011) [2023-12-27 04:21:19,568][105692] Updated weights for policy 0, policy_version 1778957 (0.0010) [2023-12-27 04:21:19,624][105692] Updated weights for policy 0, policy_version 1778967 (0.0011) [2023-12-27 04:21:20,010][105620] Updated weights for policy 1, policy_version 1782961 (0.0011) [2023-12-27 04:21:20,081][105620] Updated weights for policy 1, policy_version 1782971 (0.0011) [2023-12-27 04:21:20,144][105620] Updated weights for policy 1, policy_version 1782981 (0.0011) [2023-12-27 04:21:20,293][105692] Updated weights for policy 0, policy_version 1778977 (0.0006) [2023-12-27 04:21:20,354][105692] Updated weights for policy 0, policy_version 1778987 (0.0008) [2023-12-27 04:21:20,416][105692] Updated weights for policy 0, policy_version 1778997 (0.0008) [2023-12-27 04:21:20,477][105692] Updated weights for policy 0, policy_version 1779007 (0.0008) [2023-12-27 04:21:20,900][105620] Updated weights for policy 1, policy_version 1782991 (0.0011) [2023-12-27 04:21:20,956][105620] Updated weights for policy 1, policy_version 1783001 (0.0011) [2023-12-27 04:21:21,016][105620] Updated weights for policy 1, policy_version 1783011 (0.0011) [2023-12-27 04:21:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 912007168. Throughput: 0: 9715.0, 1: 9665.2. Samples: 911994184. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:21:21,063][104569] Avg episode reward: [(0, '7992.306'), (1, '9352.372')] [2023-12-27 04:21:21,240][105692] Updated weights for policy 0, policy_version 1779017 (0.0008) [2023-12-27 04:21:21,308][105692] Updated weights for policy 0, policy_version 1779027 (0.0008) [2023-12-27 04:21:21,377][105692] Updated weights for policy 0, policy_version 1779037 (0.0008) [2023-12-27 04:21:21,744][105620] Updated weights for policy 1, policy_version 1783021 (0.0011) [2023-12-27 04:21:21,808][105620] Updated weights for policy 1, policy_version 1783031 (0.0009) [2023-12-27 04:21:21,864][105620] Updated weights for policy 1, policy_version 1783041 (0.0011) [2023-12-27 04:21:22,141][105692] Updated weights for policy 0, policy_version 1779047 (0.0008) [2023-12-27 04:21:22,210][105692] Updated weights for policy 0, policy_version 1779057 (0.0008) [2023-12-27 04:21:22,278][105692] Updated weights for policy 0, policy_version 1779067 (0.0007) [2023-12-27 04:21:22,633][105620] Updated weights for policy 1, policy_version 1783051 (0.0009) [2023-12-27 04:21:22,701][105620] Updated weights for policy 1, policy_version 1783061 (0.0008) [2023-12-27 04:21:22,770][105620] Updated weights for policy 1, policy_version 1783071 (0.0009) [2023-12-27 04:21:22,972][105692] Updated weights for policy 0, policy_version 1779077 (0.0008) [2023-12-27 04:21:23,039][105692] Updated weights for policy 0, policy_version 1779087 (0.0008) [2023-12-27 04:21:23,088][105692] Updated weights for policy 0, policy_version 1779097 (0.0006) [2023-12-27 04:21:23,548][105620] Updated weights for policy 1, policy_version 1783081 (0.0009) [2023-12-27 04:21:23,606][105620] Updated weights for policy 1, policy_version 1783091 (0.0009) [2023-12-27 04:21:23,652][105620] Updated weights for policy 1, policy_version 1783101 (0.0008) [2023-12-27 04:21:23,699][105620] Updated weights for policy 1, policy_version 1783111 (0.0008) [2023-12-27 04:21:23,807][105692] Updated weights for policy 0, policy_version 1779107 (0.0009) [2023-12-27 04:21:23,871][105692] Updated weights for policy 0, policy_version 1779117 (0.0011) [2023-12-27 04:21:23,935][105692] Updated weights for policy 0, policy_version 1779127 (0.0007) [2023-12-27 04:21:24,436][105620] Updated weights for policy 1, policy_version 1783121 (0.0011) [2023-12-27 04:21:24,487][105620] Updated weights for policy 1, policy_version 1783131 (0.0010) [2023-12-27 04:21:24,536][105692] Updated weights for policy 0, policy_version 1779137 (0.0005) [2023-12-27 04:21:24,550][105620] Updated weights for policy 1, policy_version 1783141 (0.0007) [2023-12-27 04:21:24,598][105692] Updated weights for policy 0, policy_version 1779147 (0.0006) [2023-12-27 04:21:24,664][105692] Updated weights for policy 0, policy_version 1779157 (0.0006) [2023-12-27 04:21:24,729][105692] Updated weights for policy 0, policy_version 1779167 (0.0005) [2023-12-27 04:21:25,178][105620] Updated weights for policy 1, policy_version 1783151 (0.0007) [2023-12-27 04:21:25,238][105620] Updated weights for policy 1, policy_version 1783161 (0.0006) [2023-12-27 04:21:25,298][105620] Updated weights for policy 1, policy_version 1783171 (0.0007) [2023-12-27 04:21:25,304][105692] Updated weights for policy 0, policy_version 1779177 (0.0011) [2023-12-27 04:21:25,353][105692] Updated weights for policy 0, policy_version 1779187 (0.0010) [2023-12-27 04:21:25,409][105692] Updated weights for policy 0, policy_version 1779197 (0.0011) [2023-12-27 04:21:25,947][105620] Updated weights for policy 1, policy_version 1783181 (0.0008) [2023-12-27 04:21:26,005][105620] Updated weights for policy 1, policy_version 1783191 (0.0006) [2023-12-27 04:21:26,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 912097280. Throughput: 0: 9736.6, 1: 9740.5. Samples: 912111636. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:21:26,063][104569] Avg episode reward: [(0, '8624.130'), (1, '9259.834')] [2023-12-27 04:21:26,072][105620] Updated weights for policy 1, policy_version 1783201 (0.0005) [2023-12-27 04:21:26,078][105692] Updated weights for policy 0, policy_version 1779207 (0.0011) [2023-12-27 04:21:26,135][105692] Updated weights for policy 0, policy_version 1779217 (0.0011) [2023-12-27 04:21:26,195][105692] Updated weights for policy 0, policy_version 1779227 (0.0011) [2023-12-27 04:21:26,750][105692] Updated weights for policy 0, policy_version 1779237 (0.0007) [2023-12-27 04:21:26,794][105692] Updated weights for policy 0, policy_version 1779247 (0.0005) [2023-12-27 04:21:26,796][105620] Updated weights for policy 1, policy_version 1783211 (0.0006) [2023-12-27 04:21:26,847][105692] Updated weights for policy 0, policy_version 1779257 (0.0007) [2023-12-27 04:21:26,853][105620] Updated weights for policy 1, policy_version 1783221 (0.0007) [2023-12-27 04:21:26,908][105620] Updated weights for policy 1, policy_version 1783231 (0.0006) [2023-12-27 04:21:27,449][105692] Updated weights for policy 0, policy_version 1779267 (0.0009) [2023-12-27 04:21:27,495][105692] Updated weights for policy 0, policy_version 1779277 (0.0005) [2023-12-27 04:21:27,545][105692] Updated weights for policy 0, policy_version 1779287 (0.0005) [2023-12-27 04:21:27,699][105620] Updated weights for policy 1, policy_version 1783241 (0.0007) [2023-12-27 04:21:27,747][105620] Updated weights for policy 1, policy_version 1783251 (0.0008) [2023-12-27 04:21:27,795][105620] Updated weights for policy 1, policy_version 1783261 (0.0008) [2023-12-27 04:21:27,853][105620] Updated weights for policy 1, policy_version 1783271 (0.0008) [2023-12-27 04:21:28,176][105692] Updated weights for policy 0, policy_version 1779297 (0.0005) [2023-12-27 04:21:28,228][105692] Updated weights for policy 0, policy_version 1779307 (0.0010) [2023-12-27 04:21:28,273][105692] Updated weights for policy 0, policy_version 1779317 (0.0010) [2023-12-27 04:21:28,332][105692] Updated weights for policy 0, policy_version 1779327 (0.0010) [2023-12-27 04:21:28,599][105620] Updated weights for policy 1, policy_version 1783281 (0.0008) [2023-12-27 04:21:28,656][105620] Updated weights for policy 1, policy_version 1783291 (0.0008) [2023-12-27 04:21:28,708][105620] Updated weights for policy 1, policy_version 1783301 (0.0011) [2023-12-27 04:21:28,976][105692] Updated weights for policy 0, policy_version 1779337 (0.0011) [2023-12-27 04:21:29,041][105692] Updated weights for policy 0, policy_version 1779347 (0.0011) [2023-12-27 04:21:29,109][105692] Updated weights for policy 0, policy_version 1779357 (0.0010) [2023-12-27 04:21:29,372][105620] Updated weights for policy 1, policy_version 1783311 (0.0010) [2023-12-27 04:21:29,424][105620] Updated weights for policy 1, policy_version 1783321 (0.0008) [2023-12-27 04:21:29,482][105620] Updated weights for policy 1, policy_version 1783331 (0.0007) [2023-12-27 04:21:29,843][105692] Updated weights for policy 0, policy_version 1779367 (0.0010) [2023-12-27 04:21:29,906][105692] Updated weights for policy 0, policy_version 1779377 (0.0011) [2023-12-27 04:21:29,971][105692] Updated weights for policy 0, policy_version 1779387 (0.0008) [2023-12-27 04:21:30,231][105620] Updated weights for policy 1, policy_version 1783341 (0.0011) [2023-12-27 04:21:30,291][105620] Updated weights for policy 1, policy_version 1783351 (0.0011) [2023-12-27 04:21:30,350][105620] Updated weights for policy 1, policy_version 1783361 (0.0010) [2023-12-27 04:21:30,596][105692] Updated weights for policy 0, policy_version 1779397 (0.0007) [2023-12-27 04:21:30,649][105692] Updated weights for policy 0, policy_version 1779407 (0.0005) [2023-12-27 04:21:30,697][105692] Updated weights for policy 0, policy_version 1779417 (0.0005) [2023-12-27 04:21:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 912203776. Throughput: 0: 9815.9, 1: 9777.1. Samples: 912173124. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:21:31,062][104569] Avg episode reward: [(0, '8714.007'), (1, '9167.378')] [2023-12-27 04:21:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001779424_455598080.pth... [2023-12-27 04:21:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001778272_455303168.pth [2023-12-27 04:21:31,075][105620] Updated weights for policy 1, policy_version 1783371 (0.0009) [2023-12-27 04:21:31,150][105620] Updated weights for policy 1, policy_version 1783381 (0.0009) [2023-12-27 04:21:31,212][105620] Updated weights for policy 1, policy_version 1783391 (0.0009) [2023-12-27 04:21:31,267][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001783400_456613888.pth... [2023-12-27 04:21:31,272][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001782248_456318976.pth [2023-12-27 04:21:31,346][105692] Updated weights for policy 0, policy_version 1779427 (0.0008) [2023-12-27 04:21:31,410][105692] Updated weights for policy 0, policy_version 1779437 (0.0007) [2023-12-27 04:21:31,476][105692] Updated weights for policy 0, policy_version 1779447 (0.0006) [2023-12-27 04:21:31,966][105620] Updated weights for policy 1, policy_version 1783401 (0.0008) [2023-12-27 04:21:32,031][105620] Updated weights for policy 1, policy_version 1783411 (0.0010) [2023-12-27 04:21:32,096][105620] Updated weights for policy 1, policy_version 1783421 (0.0006) [2023-12-27 04:21:32,133][105692] Updated weights for policy 0, policy_version 1779457 (0.0006) [2023-12-27 04:21:32,149][105620] Updated weights for policy 1, policy_version 1783431 (0.0010) [2023-12-27 04:21:32,192][105692] Updated weights for policy 0, policy_version 1779467 (0.0011) [2023-12-27 04:21:32,261][105692] Updated weights for policy 0, policy_version 1779477 (0.0011) [2023-12-27 04:21:32,323][105692] Updated weights for policy 0, policy_version 1779487 (0.0010) [2023-12-27 04:21:32,812][105620] Updated weights for policy 1, policy_version 1783441 (0.0010) [2023-12-27 04:21:32,882][105620] Updated weights for policy 1, policy_version 1783451 (0.0010) [2023-12-27 04:21:32,935][105620] Updated weights for policy 1, policy_version 1783461 (0.0010) [2023-12-27 04:21:32,991][105692] Updated weights for policy 0, policy_version 1779497 (0.0006) [2023-12-27 04:21:33,057][105692] Updated weights for policy 0, policy_version 1779507 (0.0007) [2023-12-27 04:21:33,117][105692] Updated weights for policy 0, policy_version 1779517 (0.0008) [2023-12-27 04:21:33,519][105620] Updated weights for policy 1, policy_version 1783471 (0.0005) [2023-12-27 04:21:33,565][105620] Updated weights for policy 1, policy_version 1783481 (0.0005) [2023-12-27 04:21:33,623][105620] Updated weights for policy 1, policy_version 1783491 (0.0006) [2023-12-27 04:21:33,640][105692] Updated weights for policy 0, policy_version 1779527 (0.0008) [2023-12-27 04:21:33,687][105692] Updated weights for policy 0, policy_version 1779537 (0.0010) [2023-12-27 04:21:33,735][105692] Updated weights for policy 0, policy_version 1779547 (0.0007) [2023-12-27 04:21:34,316][105620] Updated weights for policy 1, policy_version 1783501 (0.0009) [2023-12-27 04:21:34,368][105620] Updated weights for policy 1, policy_version 1783511 (0.0010) [2023-12-27 04:21:34,394][105692] Updated weights for policy 0, policy_version 1779557 (0.0007) [2023-12-27 04:21:34,423][105620] Updated weights for policy 1, policy_version 1783521 (0.0010) [2023-12-27 04:21:34,452][105692] Updated weights for policy 0, policy_version 1779567 (0.0011) [2023-12-27 04:21:34,514][105692] Updated weights for policy 0, policy_version 1779577 (0.0010) [2023-12-27 04:21:35,126][105620] Updated weights for policy 1, policy_version 1783531 (0.0007) [2023-12-27 04:21:35,174][105620] Updated weights for policy 1, policy_version 1783541 (0.0010) [2023-12-27 04:21:35,176][105692] Updated weights for policy 0, policy_version 1779587 (0.0011) [2023-12-27 04:21:35,224][105620] Updated weights for policy 1, policy_version 1783551 (0.0010) [2023-12-27 04:21:35,227][105692] Updated weights for policy 0, policy_version 1779597 (0.0010) [2023-12-27 04:21:35,283][105692] Updated weights for policy 0, policy_version 1779607 (0.0011) [2023-12-27 04:21:35,976][105692] Updated weights for policy 0, policy_version 1779617 (0.0010) [2023-12-27 04:21:35,984][105620] Updated weights for policy 1, policy_version 1783561 (0.0010) [2023-12-27 04:21:36,029][105692] Updated weights for policy 0, policy_version 1779627 (0.0011) [2023-12-27 04:21:36,046][105620] Updated weights for policy 1, policy_version 1783571 (0.0008) [2023-12-27 04:21:36,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 912302080. Throughput: 0: 9805.4, 1: 9891.0. Samples: 912296632. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:21:36,062][104569] Avg episode reward: [(0, '8261.120'), (1, '9259.933')] [2023-12-27 04:21:36,093][105692] Updated weights for policy 0, policy_version 1779637 (0.0011) [2023-12-27 04:21:36,114][105620] Updated weights for policy 1, policy_version 1783581 (0.0008) [2023-12-27 04:21:36,156][105692] Updated weights for policy 0, policy_version 1779647 (0.0009) [2023-12-27 04:21:36,175][105620] Updated weights for policy 1, policy_version 1783591 (0.0011) [2023-12-27 04:21:36,908][105620] Updated weights for policy 1, policy_version 1783601 (0.0006) [2023-12-27 04:21:36,933][105692] Updated weights for policy 0, policy_version 1779657 (0.0010) [2023-12-27 04:21:36,977][105692] Updated weights for policy 0, policy_version 1779667 (0.0009) [2023-12-27 04:21:36,980][105620] Updated weights for policy 1, policy_version 1783611 (0.0005) [2023-12-27 04:21:37,038][105692] Updated weights for policy 0, policy_version 1779677 (0.0007) [2023-12-27 04:21:37,046][105620] Updated weights for policy 1, policy_version 1783621 (0.0007) [2023-12-27 04:21:37,696][105620] Updated weights for policy 1, policy_version 1783631 (0.0011) [2023-12-27 04:21:37,725][105692] Updated weights for policy 0, policy_version 1779687 (0.0007) [2023-12-27 04:21:37,746][105620] Updated weights for policy 1, policy_version 1783641 (0.0008) [2023-12-27 04:21:37,788][105692] Updated weights for policy 0, policy_version 1779697 (0.0008) [2023-12-27 04:21:37,802][105620] Updated weights for policy 1, policy_version 1783651 (0.0010) [2023-12-27 04:21:37,848][105692] Updated weights for policy 0, policy_version 1779707 (0.0005) [2023-12-27 04:21:38,431][105620] Updated weights for policy 1, policy_version 1783661 (0.0010) [2023-12-27 04:21:38,491][105620] Updated weights for policy 1, policy_version 1783671 (0.0011) [2023-12-27 04:21:38,494][105692] Updated weights for policy 0, policy_version 1779717 (0.0005) [2023-12-27 04:21:38,545][105692] Updated weights for policy 0, policy_version 1779727 (0.0008) [2023-12-27 04:21:38,557][105620] Updated weights for policy 1, policy_version 1783681 (0.0011) [2023-12-27 04:21:38,599][105692] Updated weights for policy 0, policy_version 1779737 (0.0008) [2023-12-27 04:21:39,283][105692] Updated weights for policy 0, policy_version 1779747 (0.0007) [2023-12-27 04:21:39,315][105620] Updated weights for policy 1, policy_version 1783691 (0.0010) [2023-12-27 04:21:39,346][105692] Updated weights for policy 0, policy_version 1779757 (0.0008) [2023-12-27 04:21:39,383][105620] Updated weights for policy 1, policy_version 1783701 (0.0007) [2023-12-27 04:21:39,409][105692] Updated weights for policy 0, policy_version 1779767 (0.0008) [2023-12-27 04:21:39,447][105620] Updated weights for policy 1, policy_version 1783711 (0.0008) [2023-12-27 04:21:40,145][105692] Updated weights for policy 0, policy_version 1779777 (0.0008) [2023-12-27 04:21:40,204][105620] Updated weights for policy 1, policy_version 1783721 (0.0007) [2023-12-27 04:21:40,209][105692] Updated weights for policy 0, policy_version 1779787 (0.0009) [2023-12-27 04:21:40,261][105620] Updated weights for policy 1, policy_version 1783731 (0.0006) [2023-12-27 04:21:40,267][105692] Updated weights for policy 0, policy_version 1779797 (0.0008) [2023-12-27 04:21:40,326][105620] Updated weights for policy 1, policy_version 1783741 (0.0007) [2023-12-27 04:21:40,328][105692] Updated weights for policy 0, policy_version 1779807 (0.0006) [2023-12-27 04:21:40,386][105620] Updated weights for policy 1, policy_version 1783751 (0.0009) [2023-12-27 04:21:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 912400384. Throughput: 0: 9898.3, 1: 9821.8. Samples: 912413672. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:21:41,062][104569] Avg episode reward: [(0, '7987.161'), (1, '9352.423')] [2023-12-27 04:21:41,065][105692] Updated weights for policy 0, policy_version 1779817 (0.0009) [2023-12-27 04:21:41,128][105692] Updated weights for policy 0, policy_version 1779827 (0.0008) [2023-12-27 04:21:41,156][105620] Updated weights for policy 1, policy_version 1783761 (0.0009) [2023-12-27 04:21:41,189][105692] Updated weights for policy 0, policy_version 1779837 (0.0008) [2023-12-27 04:21:41,216][105620] Updated weights for policy 1, policy_version 1783771 (0.0010) [2023-12-27 04:21:41,282][105620] Updated weights for policy 1, policy_version 1783781 (0.0009) [2023-12-27 04:21:41,967][105692] Updated weights for policy 0, policy_version 1779847 (0.0008) [2023-12-27 04:21:42,030][105692] Updated weights for policy 0, policy_version 1779857 (0.0010) [2023-12-27 04:21:42,085][105692] Updated weights for policy 0, policy_version 1779867 (0.0010) [2023-12-27 04:21:42,092][105620] Updated weights for policy 1, policy_version 1783791 (0.0007) [2023-12-27 04:21:42,143][105620] Updated weights for policy 1, policy_version 1783801 (0.0008) [2023-12-27 04:21:42,192][105620] Updated weights for policy 1, policy_version 1783811 (0.0008) [2023-12-27 04:21:42,794][105692] Updated weights for policy 0, policy_version 1779877 (0.0011) [2023-12-27 04:21:42,856][105692] Updated weights for policy 0, policy_version 1779887 (0.0010) [2023-12-27 04:21:42,914][105692] Updated weights for policy 0, policy_version 1779897 (0.0010) [2023-12-27 04:21:43,022][105620] Updated weights for policy 1, policy_version 1783821 (0.0008) [2023-12-27 04:21:43,090][105620] Updated weights for policy 1, policy_version 1783831 (0.0008) [2023-12-27 04:21:43,143][105620] Updated weights for policy 1, policy_version 1783841 (0.0008) [2023-12-27 04:21:43,641][105692] Updated weights for policy 0, policy_version 1779907 (0.0009) [2023-12-27 04:21:43,686][105692] Updated weights for policy 0, policy_version 1779917 (0.0010) [2023-12-27 04:21:43,740][105692] Updated weights for policy 0, policy_version 1779927 (0.0010) [2023-12-27 04:21:43,901][105620] Updated weights for policy 1, policy_version 1783851 (0.0008) [2023-12-27 04:21:43,958][105620] Updated weights for policy 1, policy_version 1783861 (0.0008) [2023-12-27 04:21:44,021][105620] Updated weights for policy 1, policy_version 1783871 (0.0008) [2023-12-27 04:21:44,481][105692] Updated weights for policy 0, policy_version 1779937 (0.0010) [2023-12-27 04:21:44,532][105692] Updated weights for policy 0, policy_version 1779947 (0.0010) [2023-12-27 04:21:44,586][105692] Updated weights for policy 0, policy_version 1779957 (0.0008) [2023-12-27 04:21:44,640][105692] Updated weights for policy 0, policy_version 1779967 (0.0008) [2023-12-27 04:21:44,758][105620] Updated weights for policy 1, policy_version 1783881 (0.0008) [2023-12-27 04:21:44,818][105620] Updated weights for policy 1, policy_version 1783891 (0.0008) [2023-12-27 04:21:44,867][105620] Updated weights for policy 1, policy_version 1783901 (0.0008) [2023-12-27 04:21:44,930][105620] Updated weights for policy 1, policy_version 1783911 (0.0008) [2023-12-27 04:21:45,358][105692] Updated weights for policy 0, policy_version 1779977 (0.0011) [2023-12-27 04:21:45,417][105692] Updated weights for policy 0, policy_version 1779987 (0.0011) [2023-12-27 04:21:45,476][105692] Updated weights for policy 0, policy_version 1779997 (0.0009) [2023-12-27 04:21:45,717][105620] Updated weights for policy 1, policy_version 1783921 (0.0009) [2023-12-27 04:21:45,764][105620] Updated weights for policy 1, policy_version 1783931 (0.0009) [2023-12-27 04:21:45,811][105620] Updated weights for policy 1, policy_version 1783941 (0.0008) [2023-12-27 04:21:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19660.8, 300 sec: 19605.2). Total num frames: 912498688. Throughput: 0: 9834.9, 1: 9753.0. Samples: 912468632. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:21:46,063][104569] Avg episode reward: [(0, '8161.452'), (1, '8983.778')] [2023-12-27 04:21:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001783944_456753152.pth... [2023-12-27 04:21:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001780000_455745536.pth... [2023-12-27 04:21:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001782824_456466432.pth [2023-12-27 04:21:46,076][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_001783944_456753152.pth [2023-12-27 04:21:46,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001778816_455442432.pth [2023-12-27 04:21:46,081][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_001780000_455745536.pth [2023-12-27 04:21:46,225][105692] Updated weights for policy 0, policy_version 1780008 (0.0010) [2023-12-27 04:21:46,276][105692] Updated weights for policy 0, policy_version 1780018 (0.0009) [2023-12-27 04:21:46,331][105692] Updated weights for policy 0, policy_version 1780028 (0.0009) [2023-12-27 04:21:46,593][105620] Updated weights for policy 1, policy_version 1783951 (0.0009) [2023-12-27 04:21:46,640][105620] Updated weights for policy 1, policy_version 1783961 (0.0009) [2023-12-27 04:21:46,686][105620] Updated weights for policy 1, policy_version 1783971 (0.0008) [2023-12-27 04:21:47,100][105692] Updated weights for policy 0, policy_version 1780038 (0.0009) [2023-12-27 04:21:47,169][105692] Updated weights for policy 0, policy_version 1780048 (0.0009) [2023-12-27 04:21:47,235][105692] Updated weights for policy 0, policy_version 1780058 (0.0009) [2023-12-27 04:21:47,474][105620] Updated weights for policy 1, policy_version 1783981 (0.0009) [2023-12-27 04:21:47,528][105620] Updated weights for policy 1, policy_version 1783991 (0.0009) [2023-12-27 04:21:47,575][105620] Updated weights for policy 1, policy_version 1784001 (0.0009) [2023-12-27 04:21:47,964][105692] Updated weights for policy 0, policy_version 1780068 (0.0009) [2023-12-27 04:21:48,018][105692] Updated weights for policy 0, policy_version 1780078 (0.0008) [2023-12-27 04:21:48,080][105692] Updated weights for policy 0, policy_version 1780088 (0.0009) [2023-12-27 04:21:48,281][105620] Updated weights for policy 1, policy_version 1784011 (0.0008) [2023-12-27 04:21:48,330][105620] Updated weights for policy 1, policy_version 1784021 (0.0005) [2023-12-27 04:21:48,397][105620] Updated weights for policy 1, policy_version 1784031 (0.0008) [2023-12-27 04:21:48,881][105692] Updated weights for policy 0, policy_version 1780098 (0.0009) [2023-12-27 04:21:48,952][105692] Updated weights for policy 0, policy_version 1780108 (0.0009) [2023-12-27 04:21:49,023][105692] Updated weights for policy 0, policy_version 1780118 (0.0008) [2023-12-27 04:21:49,040][105620] Updated weights for policy 1, policy_version 1784041 (0.0006) [2023-12-27 04:21:49,082][105692] Updated weights for policy 0, policy_version 1780128 (0.0006) [2023-12-27 04:21:49,109][105620] Updated weights for policy 1, policy_version 1784051 (0.0006) [2023-12-27 04:21:49,170][105620] Updated weights for policy 1, policy_version 1784061 (0.0008) [2023-12-27 04:21:49,234][105620] Updated weights for policy 1, policy_version 1784071 (0.0008) [2023-12-27 04:21:49,787][105692] Updated weights for policy 0, policy_version 1780138 (0.0010) [2023-12-27 04:21:49,855][105692] Updated weights for policy 0, policy_version 1780148 (0.0009) [2023-12-27 04:21:49,887][105620] Updated weights for policy 1, policy_version 1784081 (0.0008) [2023-12-27 04:21:49,922][105692] Updated weights for policy 0, policy_version 1780158 (0.0006) [2023-12-27 04:21:49,948][105620] Updated weights for policy 1, policy_version 1784091 (0.0008) [2023-12-27 04:21:50,002][105620] Updated weights for policy 1, policy_version 1784101 (0.0009) [2023-12-27 04:21:50,602][105620] Updated weights for policy 1, policy_version 1784111 (0.0009) [2023-12-27 04:21:50,653][105620] Updated weights for policy 1, policy_version 1784121 (0.0007) [2023-12-27 04:21:50,699][105620] Updated weights for policy 1, policy_version 1784131 (0.0008) [2023-12-27 04:21:50,772][105692] Updated weights for policy 0, policy_version 1780168 (0.0008) [2023-12-27 04:21:50,820][105692] Updated weights for policy 0, policy_version 1780178 (0.0009) [2023-12-27 04:21:50,884][105692] Updated weights for policy 0, policy_version 1780188 (0.0009) [2023-12-27 04:21:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 912596992. Throughput: 0: 9856.9, 1: 9630.6. Samples: 912582632. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:21:51,062][104569] Avg episode reward: [(0, '8529.832'), (1, '8983.751')] [2023-12-27 04:21:51,494][105620] Updated weights for policy 1, policy_version 1784141 (0.0010) [2023-12-27 04:21:51,560][105620] Updated weights for policy 1, policy_version 1784151 (0.0007) [2023-12-27 04:21:51,632][105692] Updated weights for policy 0, policy_version 1780198 (0.0009) [2023-12-27 04:21:51,637][105620] Updated weights for policy 1, policy_version 1784161 (0.0006) [2023-12-27 04:21:51,693][105692] Updated weights for policy 0, policy_version 1780208 (0.0008) [2023-12-27 04:21:51,757][105692] Updated weights for policy 0, policy_version 1780218 (0.0008) [2023-12-27 04:21:52,408][105620] Updated weights for policy 1, policy_version 1784171 (0.0008) [2023-12-27 04:21:52,411][105692] Updated weights for policy 0, policy_version 1780228 (0.0007) [2023-12-27 04:21:52,463][105620] Updated weights for policy 1, policy_version 1784181 (0.0006) [2023-12-27 04:21:52,475][105692] Updated weights for policy 0, policy_version 1780238 (0.0008) [2023-12-27 04:21:52,514][105620] Updated weights for policy 1, policy_version 1784191 (0.0007) [2023-12-27 04:21:52,536][105692] Updated weights for policy 0, policy_version 1780248 (0.0008) [2023-12-27 04:21:53,150][105692] Updated weights for policy 0, policy_version 1780258 (0.0008) [2023-12-27 04:21:53,198][105692] Updated weights for policy 0, policy_version 1780268 (0.0007) [2023-12-27 04:21:53,245][105692] Updated weights for policy 0, policy_version 1780278 (0.0009) [2023-12-27 04:21:53,298][105692] Updated weights for policy 0, policy_version 1780288 (0.0006) [2023-12-27 04:21:53,362][105620] Updated weights for policy 1, policy_version 1784201 (0.0007) [2023-12-27 04:21:53,426][105620] Updated weights for policy 1, policy_version 1784211 (0.0009) [2023-12-27 04:21:53,486][105620] Updated weights for policy 1, policy_version 1784221 (0.0009) [2023-12-27 04:21:53,541][105620] Updated weights for policy 1, policy_version 1784231 (0.0009) [2023-12-27 04:21:53,990][105692] Updated weights for policy 0, policy_version 1780298 (0.0005) [2023-12-27 04:21:54,042][105692] Updated weights for policy 0, policy_version 1780308 (0.0005) [2023-12-27 04:21:54,101][105692] Updated weights for policy 0, policy_version 1780318 (0.0007) [2023-12-27 04:21:54,330][105620] Updated weights for policy 1, policy_version 1784241 (0.0008) [2023-12-27 04:21:54,380][105620] Updated weights for policy 1, policy_version 1784251 (0.0008) [2023-12-27 04:21:54,427][105620] Updated weights for policy 1, policy_version 1784261 (0.0009) [2023-12-27 04:21:54,840][105692] Updated weights for policy 0, policy_version 1780328 (0.0008) [2023-12-27 04:21:54,901][105692] Updated weights for policy 0, policy_version 1780338 (0.0009) [2023-12-27 04:21:54,967][105692] Updated weights for policy 0, policy_version 1780348 (0.0009) [2023-12-27 04:21:55,160][105620] Updated weights for policy 1, policy_version 1784271 (0.0009) [2023-12-27 04:21:55,219][105620] Updated weights for policy 1, policy_version 1784281 (0.0009) [2023-12-27 04:21:55,279][105620] Updated weights for policy 1, policy_version 1784291 (0.0009) [2023-12-27 04:21:55,709][105692] Updated weights for policy 0, policy_version 1780358 (0.0009) [2023-12-27 04:21:55,770][105692] Updated weights for policy 0, policy_version 1780368 (0.0009) [2023-12-27 04:21:55,831][105692] Updated weights for policy 0, policy_version 1780378 (0.0008) [2023-12-27 04:21:56,021][105620] Updated weights for policy 1, policy_version 1784301 (0.0009) [2023-12-27 04:21:56,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 912687104. Throughput: 0: 9905.7, 1: 9604.8. Samples: 912697196. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:21:56,063][104569] Avg episode reward: [(0, '8898.963'), (1, '9352.370')] [2023-12-27 04:21:56,079][105620] Updated weights for policy 1, policy_version 1784311 (0.0009) [2023-12-27 04:21:56,126][105620] Updated weights for policy 1, policy_version 1784321 (0.0009) [2023-12-27 04:21:56,588][105692] Updated weights for policy 0, policy_version 1780388 (0.0009) [2023-12-27 04:21:56,653][105692] Updated weights for policy 0, policy_version 1780398 (0.0009) [2023-12-27 04:21:56,718][105692] Updated weights for policy 0, policy_version 1780408 (0.0009) [2023-12-27 04:21:56,869][105620] Updated weights for policy 1, policy_version 1784331 (0.0009) [2023-12-27 04:21:56,925][105620] Updated weights for policy 1, policy_version 1784341 (0.0009) [2023-12-27 04:21:56,980][105620] Updated weights for policy 1, policy_version 1784351 (0.0009) [2023-12-27 04:21:57,477][105692] Updated weights for policy 0, policy_version 1780418 (0.0010) [2023-12-27 04:21:57,523][105692] Updated weights for policy 0, policy_version 1780428 (0.0009) [2023-12-27 04:21:57,576][105692] Updated weights for policy 0, policy_version 1780438 (0.0009) [2023-12-27 04:21:57,628][105692] Updated weights for policy 0, policy_version 1780448 (0.0009) [2023-12-27 04:21:57,743][105620] Updated weights for policy 1, policy_version 1784361 (0.0009) [2023-12-27 04:21:57,801][105620] Updated weights for policy 1, policy_version 1784371 (0.0009) [2023-12-27 04:21:57,859][105620] Updated weights for policy 1, policy_version 1784381 (0.0008) [2023-12-27 04:21:57,906][105620] Updated weights for policy 1, policy_version 1784391 (0.0008) [2023-12-27 04:21:58,420][105692] Updated weights for policy 0, policy_version 1780458 (0.0009) [2023-12-27 04:21:58,481][105692] Updated weights for policy 0, policy_version 1780468 (0.0007) [2023-12-27 04:21:58,540][105692] Updated weights for policy 0, policy_version 1780478 (0.0008) [2023-12-27 04:21:58,690][105620] Updated weights for policy 1, policy_version 1784401 (0.0009) [2023-12-27 04:21:58,743][105620] Updated weights for policy 1, policy_version 1784411 (0.0009) [2023-12-27 04:21:58,812][105620] Updated weights for policy 1, policy_version 1784421 (0.0009) [2023-12-27 04:21:59,259][105692] Updated weights for policy 0, policy_version 1780488 (0.0009) [2023-12-27 04:21:59,322][105692] Updated weights for policy 0, policy_version 1780498 (0.0006) [2023-12-27 04:21:59,387][105692] Updated weights for policy 0, policy_version 1780508 (0.0008) [2023-12-27 04:21:59,677][105620] Updated weights for policy 1, policy_version 1784431 (0.0008) [2023-12-27 04:21:59,726][105620] Updated weights for policy 1, policy_version 1784441 (0.0008) [2023-12-27 04:21:59,782][105620] Updated weights for policy 1, policy_version 1784451 (0.0008) [2023-12-27 04:22:00,040][105692] Updated weights for policy 0, policy_version 1780518 (0.0009) [2023-12-27 04:22:00,091][105692] Updated weights for policy 0, policy_version 1780528 (0.0009) [2023-12-27 04:22:00,139][105692] Updated weights for policy 0, policy_version 1780538 (0.0009) [2023-12-27 04:22:00,625][105620] Updated weights for policy 1, policy_version 1784461 (0.0009) [2023-12-27 04:22:00,675][105620] Updated weights for policy 1, policy_version 1784471 (0.0009) [2023-12-27 04:22:00,728][105620] Updated weights for policy 1, policy_version 1784482 (0.0010) [2023-12-27 04:22:00,786][105692] Updated weights for policy 0, policy_version 1780548 (0.0008) [2023-12-27 04:22:00,844][105692] Updated weights for policy 0, policy_version 1780558 (0.0009) [2023-12-27 04:22:00,891][105692] Updated weights for policy 0, policy_version 1780568 (0.0010) [2023-12-27 04:22:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 912785408. Throughput: 0: 9891.8, 1: 9577.9. Samples: 912751724. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:22:01,062][104569] Avg episode reward: [(0, '8533.586'), (1, '9352.428')] [2023-12-27 04:22:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001780576_455892992.pth... [2023-12-27 04:22:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001784488_456892416.pth... [2023-12-27 04:22:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001779424_455598080.pth [2023-12-27 04:22:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001783400_456613888.pth [2023-12-27 04:22:01,541][105620] Updated weights for policy 1, policy_version 1784493 (0.0008) [2023-12-27 04:22:01,602][105692] Updated weights for policy 0, policy_version 1780578 (0.0010) [2023-12-27 04:22:01,609][105620] Updated weights for policy 1, policy_version 1784503 (0.0006) [2023-12-27 04:22:01,662][105692] Updated weights for policy 0, policy_version 1780588 (0.0008) [2023-12-27 04:22:01,670][105620] Updated weights for policy 1, policy_version 1784513 (0.0008) [2023-12-27 04:22:01,728][105692] Updated weights for policy 0, policy_version 1780598 (0.0009) [2023-12-27 04:22:01,789][105692] Updated weights for policy 0, policy_version 1780608 (0.0008) [2023-12-27 04:22:02,391][105620] Updated weights for policy 1, policy_version 1784523 (0.0009) [2023-12-27 04:22:02,429][105692] Updated weights for policy 0, policy_version 1780618 (0.0007) [2023-12-27 04:22:02,450][105620] Updated weights for policy 1, policy_version 1784534 (0.0007) [2023-12-27 04:22:02,484][105692] Updated weights for policy 0, policy_version 1780628 (0.0007) [2023-12-27 04:22:02,513][105620] Updated weights for policy 1, policy_version 1784544 (0.0005) [2023-12-27 04:22:02,548][105692] Updated weights for policy 0, policy_version 1780638 (0.0005) [2023-12-27 04:22:03,082][105620] Updated weights for policy 1, policy_version 1784554 (0.0007) [2023-12-27 04:22:03,135][105692] Updated weights for policy 0, policy_version 1780648 (0.0005) [2023-12-27 04:22:03,139][105620] Updated weights for policy 1, policy_version 1784564 (0.0009) [2023-12-27 04:22:03,194][105692] Updated weights for policy 0, policy_version 1780658 (0.0007) [2023-12-27 04:22:03,194][105620] Updated weights for policy 1, policy_version 1784574 (0.0005) [2023-12-27 04:22:03,241][105620] Updated weights for policy 1, policy_version 1784584 (0.0005) [2023-12-27 04:22:03,242][105692] Updated weights for policy 0, policy_version 1780668 (0.0009) [2023-12-27 04:22:03,769][105620] Updated weights for policy 1, policy_version 1784594 (0.0005) [2023-12-27 04:22:03,814][105620] Updated weights for policy 1, policy_version 1784604 (0.0007) [2023-12-27 04:22:03,822][105692] Updated weights for policy 0, policy_version 1780678 (0.0007) [2023-12-27 04:22:03,875][105620] Updated weights for policy 1, policy_version 1784614 (0.0007) [2023-12-27 04:22:03,890][105692] Updated weights for policy 0, policy_version 1780688 (0.0008) [2023-12-27 04:22:03,942][105692] Updated weights for policy 0, policy_version 1780698 (0.0009) [2023-12-27 04:22:04,485][105620] Updated weights for policy 1, policy_version 1784624 (0.0009) [2023-12-27 04:22:04,548][105620] Updated weights for policy 1, policy_version 1784634 (0.0009) [2023-12-27 04:22:04,605][105620] Updated weights for policy 1, policy_version 1784644 (0.0009) [2023-12-27 04:22:04,691][105692] Updated weights for policy 0, policy_version 1780708 (0.0010) [2023-12-27 04:22:04,754][105692] Updated weights for policy 0, policy_version 1780718 (0.0009) [2023-12-27 04:22:04,811][105692] Updated weights for policy 0, policy_version 1780728 (0.0009) [2023-12-27 04:22:05,262][105620] Updated weights for policy 1, policy_version 1784654 (0.0007) [2023-12-27 04:22:05,308][105620] Updated weights for policy 1, policy_version 1784664 (0.0005) [2023-12-27 04:22:05,360][105620] Updated weights for policy 1, policy_version 1784674 (0.0007) [2023-12-27 04:22:05,573][105692] Updated weights for policy 0, policy_version 1780738 (0.0009) [2023-12-27 04:22:05,634][105692] Updated weights for policy 0, policy_version 1780748 (0.0009) [2023-12-27 04:22:05,698][105692] Updated weights for policy 0, policy_version 1780758 (0.0009) [2023-12-27 04:22:05,759][105692] Updated weights for policy 0, policy_version 1780768 (0.0009) [2023-12-27 04:22:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 912883712. Throughput: 0: 9979.5, 1: 9577.2. Samples: 912874232. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:22:06,062][104569] Avg episode reward: [(0, '8347.363'), (1, '9259.881')] [2023-12-27 04:22:06,085][105620] Updated weights for policy 1, policy_version 1784684 (0.0009) [2023-12-27 04:22:06,153][105620] Updated weights for policy 1, policy_version 1784694 (0.0008) [2023-12-27 04:22:06,216][105620] Updated weights for policy 1, policy_version 1784704 (0.0009) [2023-12-27 04:22:06,492][105692] Updated weights for policy 0, policy_version 1780778 (0.0008) [2023-12-27 04:22:06,544][105692] Updated weights for policy 0, policy_version 1780788 (0.0009) [2023-12-27 04:22:06,591][105692] Updated weights for policy 0, policy_version 1780798 (0.0009) [2023-12-27 04:22:06,945][105620] Updated weights for policy 1, policy_version 1784714 (0.0009) [2023-12-27 04:22:07,003][105620] Updated weights for policy 1, policy_version 1784724 (0.0009) [2023-12-27 04:22:07,053][105620] Updated weights for policy 1, policy_version 1784734 (0.0009) [2023-12-27 04:22:07,311][105692] Updated weights for policy 0, policy_version 1780808 (0.0009) [2023-12-27 04:22:07,359][105692] Updated weights for policy 0, policy_version 1780818 (0.0005) [2023-12-27 04:22:07,428][105692] Updated weights for policy 0, policy_version 1780828 (0.0005) [2023-12-27 04:22:07,899][105620] Updated weights for policy 1, policy_version 1784745 (0.0010) [2023-12-27 04:22:07,954][105620] Updated weights for policy 1, policy_version 1784755 (0.0009) [2023-12-27 04:22:08,008][105620] Updated weights for policy 1, policy_version 1784765 (0.0009) [2023-12-27 04:22:08,060][105692] Updated weights for policy 0, policy_version 1780838 (0.0006) [2023-12-27 04:22:08,060][105620] Updated weights for policy 1, policy_version 1784775 (0.0011) [2023-12-27 04:22:08,112][105692] Updated weights for policy 0, policy_version 1780848 (0.0009) [2023-12-27 04:22:08,164][105692] Updated weights for policy 0, policy_version 1780858 (0.0009) [2023-12-27 04:22:08,835][105620] Updated weights for policy 1, policy_version 1784785 (0.0009) [2023-12-27 04:22:08,897][105620] Updated weights for policy 1, policy_version 1784795 (0.0009) [2023-12-27 04:22:08,946][105692] Updated weights for policy 0, policy_version 1780868 (0.0008) [2023-12-27 04:22:08,952][105620] Updated weights for policy 1, policy_version 1784805 (0.0007) [2023-12-27 04:22:08,994][105692] Updated weights for policy 0, policy_version 1780878 (0.0007) [2023-12-27 04:22:09,059][105692] Updated weights for policy 0, policy_version 1780888 (0.0010) [2023-12-27 04:22:09,629][105620] Updated weights for policy 1, policy_version 1784815 (0.0009) [2023-12-27 04:22:09,688][105620] Updated weights for policy 1, policy_version 1784825 (0.0010) [2023-12-27 04:22:09,754][105620] Updated weights for policy 1, policy_version 1784835 (0.0011) [2023-12-27 04:22:09,812][105692] Updated weights for policy 0, policy_version 1780899 (0.0010) [2023-12-27 04:22:09,880][105692] Updated weights for policy 0, policy_version 1780909 (0.0009) [2023-12-27 04:22:09,947][105692] Updated weights for policy 0, policy_version 1780919 (0.0008) [2023-12-27 04:22:10,484][105620] Updated weights for policy 1, policy_version 1784845 (0.0010) [2023-12-27 04:22:10,532][105620] Updated weights for policy 1, policy_version 1784855 (0.0010) [2023-12-27 04:22:10,576][105620] Updated weights for policy 1, policy_version 1784865 (0.0010) [2023-12-27 04:22:10,688][105692] Updated weights for policy 0, policy_version 1780929 (0.0007) [2023-12-27 04:22:10,753][105692] Updated weights for policy 0, policy_version 1780939 (0.0009) [2023-12-27 04:22:10,806][105692] Updated weights for policy 0, policy_version 1780950 (0.0010) [2023-12-27 04:22:11,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 912982016. Throughput: 0: 9922.0, 1: 9586.1. Samples: 912989504. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:22:11,063][104569] Avg episode reward: [(0, '8444.390'), (1, '9259.905')] [2023-12-27 04:22:11,241][105620] Updated weights for policy 1, policy_version 1784875 (0.0011) [2023-12-27 04:22:11,307][105620] Updated weights for policy 1, policy_version 1784885 (0.0009) [2023-12-27 04:22:11,377][105620] Updated weights for policy 1, policy_version 1784895 (0.0010) [2023-12-27 04:22:11,604][105692] Updated weights for policy 0, policy_version 1780961 (0.0010) [2023-12-27 04:22:11,667][105692] Updated weights for policy 0, policy_version 1780971 (0.0007) [2023-12-27 04:22:11,718][105692] Updated weights for policy 0, policy_version 1780981 (0.0009) [2023-12-27 04:22:11,786][105692] Updated weights for policy 0, policy_version 1780991 (0.0007) [2023-12-27 04:22:12,156][105620] Updated weights for policy 1, policy_version 1784905 (0.0008) [2023-12-27 04:22:12,209][105620] Updated weights for policy 1, policy_version 1784915 (0.0005) [2023-12-27 04:22:12,275][105620] Updated weights for policy 1, policy_version 1784925 (0.0008) [2023-12-27 04:22:12,334][105620] Updated weights for policy 1, policy_version 1784935 (0.0009) [2023-12-27 04:22:12,598][105692] Updated weights for policy 0, policy_version 1781001 (0.0009) [2023-12-27 04:22:12,661][105692] Updated weights for policy 0, policy_version 1781011 (0.0009) [2023-12-27 04:22:12,713][105692] Updated weights for policy 0, policy_version 1781021 (0.0009) [2023-12-27 04:22:13,055][105620] Updated weights for policy 1, policy_version 1784945 (0.0009) [2023-12-27 04:22:13,113][105620] Updated weights for policy 1, policy_version 1784955 (0.0009) [2023-12-27 04:22:13,170][105620] Updated weights for policy 1, policy_version 1784965 (0.0007) [2023-12-27 04:22:13,428][105692] Updated weights for policy 0, policy_version 1781031 (0.0009) [2023-12-27 04:22:13,484][105692] Updated weights for policy 0, policy_version 1781041 (0.0009) [2023-12-27 04:22:13,547][105692] Updated weights for policy 0, policy_version 1781051 (0.0009) [2023-12-27 04:22:13,840][105620] Updated weights for policy 1, policy_version 1784975 (0.0005) [2023-12-27 04:22:13,910][105620] Updated weights for policy 1, policy_version 1784985 (0.0006) [2023-12-27 04:22:13,976][105620] Updated weights for policy 1, policy_version 1784995 (0.0005) [2023-12-27 04:22:14,385][105692] Updated weights for policy 0, policy_version 1781061 (0.0009) [2023-12-27 04:22:14,432][105692] Updated weights for policy 0, policy_version 1781071 (0.0008) [2023-12-27 04:22:14,480][105692] Updated weights for policy 0, policy_version 1781081 (0.0009) [2023-12-27 04:22:14,527][105620] Updated weights for policy 1, policy_version 1785005 (0.0006) [2023-12-27 04:22:14,585][105620] Updated weights for policy 1, policy_version 1785015 (0.0009) [2023-12-27 04:22:14,644][105620] Updated weights for policy 1, policy_version 1785025 (0.0010) [2023-12-27 04:22:15,291][105620] Updated weights for policy 1, policy_version 1785035 (0.0009) [2023-12-27 04:22:15,327][105692] Updated weights for policy 0, policy_version 1781091 (0.0009) [2023-12-27 04:22:15,356][105620] Updated weights for policy 1, policy_version 1785045 (0.0007) [2023-12-27 04:22:15,385][105692] Updated weights for policy 0, policy_version 1781101 (0.0007) [2023-12-27 04:22:15,420][105620] Updated weights for policy 1, policy_version 1785055 (0.0007) [2023-12-27 04:22:15,443][105692] Updated weights for policy 0, policy_version 1781111 (0.0008) [2023-12-27 04:22:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 913072128. Throughput: 0: 9772.9, 1: 9629.3. Samples: 913046224. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:22:16,062][104569] Avg episode reward: [(0, '8539.521'), (1, '9076.219')] [2023-12-27 04:22:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001781120_456032256.pth... [2023-12-27 04:22:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001785064_457039872.pth... [2023-12-27 04:22:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001780000_455745536.pth [2023-12-27 04:22:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001783944_456753152.pth [2023-12-27 04:22:16,106][105620] Updated weights for policy 1, policy_version 1785065 (0.0009) [2023-12-27 04:22:16,158][105620] Updated weights for policy 1, policy_version 1785075 (0.0009) [2023-12-27 04:22:16,204][105620] Updated weights for policy 1, policy_version 1785085 (0.0009) [2023-12-27 04:22:16,224][105692] Updated weights for policy 0, policy_version 1781121 (0.0007) [2023-12-27 04:22:16,256][105620] Updated weights for policy 1, policy_version 1785095 (0.0009) [2023-12-27 04:22:16,280][105692] Updated weights for policy 0, policy_version 1781131 (0.0008) [2023-12-27 04:22:16,327][105692] Updated weights for policy 0, policy_version 1781141 (0.0008) [2023-12-27 04:22:16,388][105692] Updated weights for policy 0, policy_version 1781151 (0.0009) [2023-12-27 04:22:17,053][105620] Updated weights for policy 1, policy_version 1785105 (0.0007) [2023-12-27 04:22:17,097][105692] Updated weights for policy 0, policy_version 1781161 (0.0006) [2023-12-27 04:22:17,112][105620] Updated weights for policy 1, policy_version 1785115 (0.0009) [2023-12-27 04:22:17,152][105692] Updated weights for policy 0, policy_version 1781171 (0.0006) [2023-12-27 04:22:17,175][105620] Updated weights for policy 1, policy_version 1785125 (0.0011) [2023-12-27 04:22:17,209][105692] Updated weights for policy 0, policy_version 1781181 (0.0005) [2023-12-27 04:22:17,896][105692] Updated weights for policy 0, policy_version 1781191 (0.0005) [2023-12-27 04:22:17,896][105620] Updated weights for policy 1, policy_version 1785135 (0.0010) [2023-12-27 04:22:17,948][105692] Updated weights for policy 0, policy_version 1781201 (0.0008) [2023-12-27 04:22:17,948][105620] Updated weights for policy 1, policy_version 1785145 (0.0010) [2023-12-27 04:22:17,996][105620] Updated weights for policy 1, policy_version 1785155 (0.0010) [2023-12-27 04:22:17,999][105692] Updated weights for policy 0, policy_version 1781211 (0.0007) [2023-12-27 04:22:18,751][105620] Updated weights for policy 1, policy_version 1785165 (0.0009) [2023-12-27 04:22:18,774][105692] Updated weights for policy 0, policy_version 1781221 (0.0007) [2023-12-27 04:22:18,809][105620] Updated weights for policy 1, policy_version 1785175 (0.0008) [2023-12-27 04:22:18,835][105692] Updated weights for policy 0, policy_version 1781231 (0.0006) [2023-12-27 04:22:18,872][105620] Updated weights for policy 1, policy_version 1785185 (0.0009) [2023-12-27 04:22:18,894][105692] Updated weights for policy 0, policy_version 1781241 (0.0007) [2023-12-27 04:22:19,535][105620] Updated weights for policy 1, policy_version 1785195 (0.0007) [2023-12-27 04:22:19,600][105620] Updated weights for policy 1, policy_version 1785205 (0.0008) [2023-12-27 04:22:19,655][105620] Updated weights for policy 1, policy_version 1785215 (0.0009) [2023-12-27 04:22:19,714][105692] Updated weights for policy 0, policy_version 1781251 (0.0009) [2023-12-27 04:22:19,773][105692] Updated weights for policy 0, policy_version 1781261 (0.0010) [2023-12-27 04:22:19,825][105692] Updated weights for policy 0, policy_version 1781271 (0.0009) [2023-12-27 04:22:20,420][105620] Updated weights for policy 1, policy_version 1785225 (0.0009) [2023-12-27 04:22:20,479][105620] Updated weights for policy 1, policy_version 1785235 (0.0008) [2023-12-27 04:22:20,539][105620] Updated weights for policy 1, policy_version 1785245 (0.0008) [2023-12-27 04:22:20,574][105692] Updated weights for policy 0, policy_version 1781281 (0.0009) [2023-12-27 04:22:20,613][105620] Updated weights for policy 1, policy_version 1785255 (0.0008) [2023-12-27 04:22:20,641][105692] Updated weights for policy 0, policy_version 1781291 (0.0011) [2023-12-27 04:22:20,700][105692] Updated weights for policy 0, policy_version 1781301 (0.0010) [2023-12-27 04:22:20,748][105692] Updated weights for policy 0, policy_version 1781311 (0.0009) [2023-12-27 04:22:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 913170432. Throughput: 0: 9580.5, 1: 9612.1. Samples: 913160304. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:22:21,063][104569] Avg episode reward: [(0, '8536.247'), (1, '8892.912')] [2023-12-27 04:22:21,321][105620] Updated weights for policy 1, policy_version 1785265 (0.0008) [2023-12-27 04:22:21,388][105620] Updated weights for policy 1, policy_version 1785275 (0.0007) [2023-12-27 04:22:21,450][105620] Updated weights for policy 1, policy_version 1785285 (0.0006) [2023-12-27 04:22:21,572][105692] Updated weights for policy 0, policy_version 1781321 (0.0008) [2023-12-27 04:22:21,634][105692] Updated weights for policy 0, policy_version 1781331 (0.0008) [2023-12-27 04:22:21,700][105692] Updated weights for policy 0, policy_version 1781341 (0.0009) [2023-12-27 04:22:22,162][105620] Updated weights for policy 1, policy_version 1785295 (0.0007) [2023-12-27 04:22:22,223][105620] Updated weights for policy 1, policy_version 1785305 (0.0008) [2023-12-27 04:22:22,288][105620] Updated weights for policy 1, policy_version 1785315 (0.0008) [2023-12-27 04:22:22,494][105692] Updated weights for policy 0, policy_version 1781351 (0.0010) [2023-12-27 04:22:22,553][105692] Updated weights for policy 0, policy_version 1781361 (0.0008) [2023-12-27 04:22:22,601][105692] Updated weights for policy 0, policy_version 1781371 (0.0009) [2023-12-27 04:22:23,051][105620] Updated weights for policy 1, policy_version 1785325 (0.0009) [2023-12-27 04:22:23,113][105620] Updated weights for policy 1, policy_version 1785335 (0.0009) [2023-12-27 04:22:23,172][105620] Updated weights for policy 1, policy_version 1785345 (0.0009) [2023-12-27 04:22:23,240][105692] Updated weights for policy 0, policy_version 1781381 (0.0008) [2023-12-27 04:22:23,297][105692] Updated weights for policy 0, policy_version 1781391 (0.0009) [2023-12-27 04:22:23,365][105692] Updated weights for policy 0, policy_version 1781401 (0.0009) [2023-12-27 04:22:23,920][105620] Updated weights for policy 1, policy_version 1785355 (0.0009) [2023-12-27 04:22:23,986][105620] Updated weights for policy 1, policy_version 1785365 (0.0009) [2023-12-27 04:22:24,037][105620] Updated weights for policy 1, policy_version 1785375 (0.0008) [2023-12-27 04:22:24,115][105692] Updated weights for policy 0, policy_version 1781411 (0.0009) [2023-12-27 04:22:24,177][105692] Updated weights for policy 0, policy_version 1781421 (0.0009) [2023-12-27 04:22:24,244][105692] Updated weights for policy 0, policy_version 1781431 (0.0009) [2023-12-27 04:22:24,865][105620] Updated weights for policy 1, policy_version 1785385 (0.0009) [2023-12-27 04:22:24,875][105692] Updated weights for policy 0, policy_version 1781441 (0.0010) [2023-12-27 04:22:24,927][105620] Updated weights for policy 1, policy_version 1785395 (0.0007) [2023-12-27 04:22:24,929][105692] Updated weights for policy 0, policy_version 1781451 (0.0006) [2023-12-27 04:22:24,984][105620] Updated weights for policy 1, policy_version 1785405 (0.0008) [2023-12-27 04:22:24,985][105692] Updated weights for policy 0, policy_version 1781461 (0.0006) [2023-12-27 04:22:25,041][105692] Updated weights for policy 0, policy_version 1781471 (0.0006) [2023-12-27 04:22:25,049][105620] Updated weights for policy 1, policy_version 1785415 (0.0009) [2023-12-27 04:22:25,769][105620] Updated weights for policy 1, policy_version 1785425 (0.0006) [2023-12-27 04:22:25,773][105692] Updated weights for policy 0, policy_version 1781481 (0.0009) [2023-12-27 04:22:25,816][105620] Updated weights for policy 1, policy_version 1785435 (0.0005) [2023-12-27 04:22:25,827][105692] Updated weights for policy 0, policy_version 1781491 (0.0009) [2023-12-27 04:22:25,864][105620] Updated weights for policy 1, policy_version 1785445 (0.0005) [2023-12-27 04:22:25,882][105692] Updated weights for policy 0, policy_version 1781501 (0.0009) [2023-12-27 04:22:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 913268736. Throughput: 0: 9525.6, 1: 9580.6. Samples: 913273452. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:22:26,062][104569] Avg episode reward: [(0, '8446.208'), (1, '9169.151')] [2023-12-27 04:22:26,475][105620] Updated weights for policy 1, policy_version 1785455 (0.0008) [2023-12-27 04:22:26,533][105620] Updated weights for policy 1, policy_version 1785465 (0.0009) [2023-12-27 04:22:26,591][105620] Updated weights for policy 1, policy_version 1785475 (0.0009) [2023-12-27 04:22:26,681][105692] Updated weights for policy 0, policy_version 1781511 (0.0009) [2023-12-27 04:22:26,728][105692] Updated weights for policy 0, policy_version 1781521 (0.0009) [2023-12-27 04:22:26,774][105692] Updated weights for policy 0, policy_version 1781531 (0.0008) [2023-12-27 04:22:27,335][105620] Updated weights for policy 1, policy_version 1785485 (0.0010) [2023-12-27 04:22:27,395][105620] Updated weights for policy 1, policy_version 1785495 (0.0009) [2023-12-27 04:22:27,453][105620] Updated weights for policy 1, policy_version 1785505 (0.0006) [2023-12-27 04:22:27,538][105692] Updated weights for policy 0, policy_version 1781541 (0.0009) [2023-12-27 04:22:27,596][105692] Updated weights for policy 0, policy_version 1781551 (0.0010) [2023-12-27 04:22:27,649][105692] Updated weights for policy 0, policy_version 1781561 (0.0010) [2023-12-27 04:22:28,119][105620] Updated weights for policy 1, policy_version 1785515 (0.0006) [2023-12-27 04:22:28,170][105620] Updated weights for policy 1, policy_version 1785525 (0.0007) [2023-12-27 04:22:28,220][105620] Updated weights for policy 1, policy_version 1785535 (0.0008) [2023-12-27 04:22:28,445][105692] Updated weights for policy 0, policy_version 1781571 (0.0009) [2023-12-27 04:22:28,504][105692] Updated weights for policy 0, policy_version 1781581 (0.0011) [2023-12-27 04:22:28,574][105692] Updated weights for policy 0, policy_version 1781591 (0.0009) [2023-12-27 04:22:28,992][105620] Updated weights for policy 1, policy_version 1785545 (0.0009) [2023-12-27 04:22:29,045][105620] Updated weights for policy 1, policy_version 1785555 (0.0010) [2023-12-27 04:22:29,103][105620] Updated weights for policy 1, policy_version 1785565 (0.0010) [2023-12-27 04:22:29,155][105692] Updated weights for policy 0, policy_version 1781601 (0.0010) [2023-12-27 04:22:29,157][105620] Updated weights for policy 1, policy_version 1785575 (0.0009) [2023-12-27 04:22:29,216][105692] Updated weights for policy 0, policy_version 1781611 (0.0009) [2023-12-27 04:22:29,274][105692] Updated weights for policy 0, policy_version 1781621 (0.0007) [2023-12-27 04:22:29,343][105692] Updated weights for policy 0, policy_version 1781631 (0.0010) [2023-12-27 04:22:29,890][105620] Updated weights for policy 1, policy_version 1785585 (0.0007) [2023-12-27 04:22:29,954][105620] Updated weights for policy 1, policy_version 1785595 (0.0007) [2023-12-27 04:22:30,020][105620] Updated weights for policy 1, policy_version 1785605 (0.0008) [2023-12-27 04:22:30,062][105692] Updated weights for policy 0, policy_version 1781641 (0.0010) [2023-12-27 04:22:30,111][105692] Updated weights for policy 0, policy_version 1781651 (0.0010) [2023-12-27 04:22:30,169][105692] Updated weights for policy 0, policy_version 1781661 (0.0011) [2023-12-27 04:22:30,745][105620] Updated weights for policy 1, policy_version 1785615 (0.0008) [2023-12-27 04:22:30,792][105620] Updated weights for policy 1, policy_version 1785625 (0.0008) [2023-12-27 04:22:30,846][105620] Updated weights for policy 1, policy_version 1785635 (0.0008) [2023-12-27 04:22:30,914][105692] Updated weights for policy 0, policy_version 1781671 (0.0011) [2023-12-27 04:22:30,968][105692] Updated weights for policy 0, policy_version 1781681 (0.0011) [2023-12-27 04:22:31,033][105692] Updated weights for policy 0, policy_version 1781691 (0.0011) [2023-12-27 04:22:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.1, 300 sec: 19521.9). Total num frames: 913358848. Throughput: 0: 9497.0, 1: 9649.2. Samples: 913330208. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:22:31,063][104569] Avg episode reward: [(0, '8538.310'), (1, '9167.760')] [2023-12-27 04:22:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001781696_456179712.pth... [2023-12-27 04:22:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001785640_457187328.pth... [2023-12-27 04:22:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001780576_455892992.pth [2023-12-27 04:22:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001784488_456892416.pth [2023-12-27 04:22:31,636][105620] Updated weights for policy 1, policy_version 1785645 (0.0007) [2023-12-27 04:22:31,695][105620] Updated weights for policy 1, policy_version 1785655 (0.0008) [2023-12-27 04:22:31,754][105620] Updated weights for policy 1, policy_version 1785665 (0.0008) [2023-12-27 04:22:31,849][105692] Updated weights for policy 0, policy_version 1781701 (0.0009) [2023-12-27 04:22:31,898][105692] Updated weights for policy 0, policy_version 1781711 (0.0011) [2023-12-27 04:22:31,946][105692] Updated weights for policy 0, policy_version 1781721 (0.0011) [2023-12-27 04:22:32,485][105620] Updated weights for policy 1, policy_version 1785675 (0.0008) [2023-12-27 04:22:32,538][105620] Updated weights for policy 1, policy_version 1785685 (0.0008) [2023-12-27 04:22:32,594][105620] Updated weights for policy 1, policy_version 1785695 (0.0008) [2023-12-27 04:22:32,704][105692] Updated weights for policy 0, policy_version 1781731 (0.0010) [2023-12-27 04:22:32,760][105692] Updated weights for policy 0, policy_version 1781741 (0.0006) [2023-12-27 04:22:32,815][105692] Updated weights for policy 0, policy_version 1781751 (0.0006) [2023-12-27 04:22:33,195][105620] Updated weights for policy 1, policy_version 1785705 (0.0008) [2023-12-27 04:22:33,246][105620] Updated weights for policy 1, policy_version 1785715 (0.0007) [2023-12-27 04:22:33,302][105620] Updated weights for policy 1, policy_version 1785726 (0.0009) [2023-12-27 04:22:33,353][105620] Updated weights for policy 1, policy_version 1785736 (0.0009) [2023-12-27 04:22:33,451][105692] Updated weights for policy 0, policy_version 1781761 (0.0006) [2023-12-27 04:22:33,517][105692] Updated weights for policy 0, policy_version 1781771 (0.0006) [2023-12-27 04:22:33,579][105692] Updated weights for policy 0, policy_version 1781781 (0.0008) [2023-12-27 04:22:33,638][105692] Updated weights for policy 0, policy_version 1781791 (0.0005) [2023-12-27 04:22:34,147][105620] Updated weights for policy 1, policy_version 1785746 (0.0008) [2023-12-27 04:22:34,206][105620] Updated weights for policy 1, policy_version 1785756 (0.0008) [2023-12-27 04:22:34,263][105620] Updated weights for policy 1, policy_version 1785766 (0.0008) [2023-12-27 04:22:34,312][105692] Updated weights for policy 0, policy_version 1781801 (0.0008) [2023-12-27 04:22:34,376][105692] Updated weights for policy 0, policy_version 1781811 (0.0010) [2023-12-27 04:22:34,432][105692] Updated weights for policy 0, policy_version 1781821 (0.0011) [2023-12-27 04:22:34,924][105620] Updated weights for policy 1, policy_version 1785776 (0.0007) [2023-12-27 04:22:34,971][105620] Updated weights for policy 1, policy_version 1785786 (0.0009) [2023-12-27 04:22:35,028][105620] Updated weights for policy 1, policy_version 1785796 (0.0009) [2023-12-27 04:22:35,195][105692] Updated weights for policy 0, policy_version 1781831 (0.0007) [2023-12-27 04:22:35,238][105692] Updated weights for policy 0, policy_version 1781841 (0.0005) [2023-12-27 04:22:35,289][105692] Updated weights for policy 0, policy_version 1781851 (0.0007) [2023-12-27 04:22:35,822][105620] Updated weights for policy 1, policy_version 1785806 (0.0009) [2023-12-27 04:22:35,908][105620] Updated weights for policy 1, policy_version 1785817 (0.0009) [2023-12-27 04:22:35,960][105692] Updated weights for policy 0, policy_version 1781861 (0.0008) [2023-12-27 04:22:35,973][105620] Updated weights for policy 1, policy_version 1785827 (0.0009) [2023-12-27 04:22:36,021][105692] Updated weights for policy 0, policy_version 1781871 (0.0010) [2023-12-27 04:22:36,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 913457152. Throughput: 0: 9559.2, 1: 9669.3. Samples: 913447916. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:22:36,062][104569] Avg episode reward: [(0, '8356.682'), (1, '9167.676')] [2023-12-27 04:22:36,079][105692] Updated weights for policy 0, policy_version 1781881 (0.0010) [2023-12-27 04:22:36,751][105620] Updated weights for policy 1, policy_version 1785837 (0.0009) [2023-12-27 04:22:36,770][105692] Updated weights for policy 0, policy_version 1781891 (0.0011) [2023-12-27 04:22:36,812][105620] Updated weights for policy 1, policy_version 1785847 (0.0009) [2023-12-27 04:22:36,827][105692] Updated weights for policy 0, policy_version 1781901 (0.0011) [2023-12-27 04:22:36,866][105620] Updated weights for policy 1, policy_version 1785857 (0.0009) [2023-12-27 04:22:36,883][105692] Updated weights for policy 0, policy_version 1781911 (0.0011) [2023-12-27 04:22:37,607][105620] Updated weights for policy 1, policy_version 1785867 (0.0010) [2023-12-27 04:22:37,655][105692] Updated weights for policy 0, policy_version 1781921 (0.0009) [2023-12-27 04:22:37,664][105620] Updated weights for policy 1, policy_version 1785877 (0.0007) [2023-12-27 04:22:37,712][105692] Updated weights for policy 0, policy_version 1781931 (0.0011) [2023-12-27 04:22:37,727][105620] Updated weights for policy 1, policy_version 1785887 (0.0007) [2023-12-27 04:22:37,773][105692] Updated weights for policy 0, policy_version 1781941 (0.0011) [2023-12-27 04:22:37,833][105692] Updated weights for policy 0, policy_version 1781951 (0.0009) [2023-12-27 04:22:38,336][105620] Updated weights for policy 1, policy_version 1785897 (0.0011) [2023-12-27 04:22:38,389][105620] Updated weights for policy 1, policy_version 1785907 (0.0011) [2023-12-27 04:22:38,453][105620] Updated weights for policy 1, policy_version 1785917 (0.0009) [2023-12-27 04:22:38,515][105620] Updated weights for policy 1, policy_version 1785927 (0.0008) [2023-12-27 04:22:38,596][105692] Updated weights for policy 0, policy_version 1781961 (0.0011) [2023-12-27 04:22:38,641][105692] Updated weights for policy 0, policy_version 1781971 (0.0010) [2023-12-27 04:22:38,693][105692] Updated weights for policy 0, policy_version 1781981 (0.0011) [2023-12-27 04:22:39,172][105620] Updated weights for policy 1, policy_version 1785937 (0.0005) [2023-12-27 04:22:39,229][105620] Updated weights for policy 1, policy_version 1785947 (0.0006) [2023-12-27 04:22:39,285][105620] Updated weights for policy 1, policy_version 1785957 (0.0008) [2023-12-27 04:22:39,507][105692] Updated weights for policy 0, policy_version 1781991 (0.0009) [2023-12-27 04:22:39,566][105692] Updated weights for policy 0, policy_version 1782001 (0.0008) [2023-12-27 04:22:39,629][105692] Updated weights for policy 0, policy_version 1782011 (0.0009) [2023-12-27 04:22:40,024][105620] Updated weights for policy 1, policy_version 1785967 (0.0010) [2023-12-27 04:22:40,085][105620] Updated weights for policy 1, policy_version 1785977 (0.0011) [2023-12-27 04:22:40,149][105620] Updated weights for policy 1, policy_version 1785987 (0.0011) [2023-12-27 04:22:40,340][105692] Updated weights for policy 0, policy_version 1782021 (0.0008) [2023-12-27 04:22:40,392][105692] Updated weights for policy 0, policy_version 1782031 (0.0011) [2023-12-27 04:22:40,445][105692] Updated weights for policy 0, policy_version 1782041 (0.0011) [2023-12-27 04:22:40,775][105620] Updated weights for policy 1, policy_version 1785997 (0.0008) [2023-12-27 04:22:40,841][105620] Updated weights for policy 1, policy_version 1786007 (0.0005) [2023-12-27 04:22:40,906][105620] Updated weights for policy 1, policy_version 1786017 (0.0009) [2023-12-27 04:22:41,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 913555456. Throughput: 0: 9534.9, 1: 9718.2. Samples: 913563584. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:22:41,062][104569] Avg episode reward: [(0, '8172.283'), (1, '9259.805')] [2023-12-27 04:22:41,234][105692] Updated weights for policy 0, policy_version 1782051 (0.0009) [2023-12-27 04:22:41,299][105692] Updated weights for policy 0, policy_version 1782061 (0.0008) [2023-12-27 04:22:41,373][105692] Updated weights for policy 0, policy_version 1782071 (0.0009) [2023-12-27 04:22:41,607][105620] Updated weights for policy 1, policy_version 1786027 (0.0009) [2023-12-27 04:22:41,675][105620] Updated weights for policy 1, policy_version 1786037 (0.0008) [2023-12-27 04:22:41,737][105620] Updated weights for policy 1, policy_version 1786047 (0.0010) [2023-12-27 04:22:42,063][105692] Updated weights for policy 0, policy_version 1782081 (0.0011) [2023-12-27 04:22:42,130][105692] Updated weights for policy 0, policy_version 1782091 (0.0011) [2023-12-27 04:22:42,196][105692] Updated weights for policy 0, policy_version 1782101 (0.0011) [2023-12-27 04:22:42,269][105692] Updated weights for policy 0, policy_version 1782111 (0.0010) [2023-12-27 04:22:42,539][105620] Updated weights for policy 1, policy_version 1786057 (0.0008) [2023-12-27 04:22:42,586][105620] Updated weights for policy 1, policy_version 1786067 (0.0005) [2023-12-27 04:22:42,647][105620] Updated weights for policy 1, policy_version 1786077 (0.0005) [2023-12-27 04:22:42,704][105620] Updated weights for policy 1, policy_version 1786087 (0.0006) [2023-12-27 04:22:42,993][105692] Updated weights for policy 0, policy_version 1782121 (0.0010) [2023-12-27 04:22:43,047][105692] Updated weights for policy 0, policy_version 1782131 (0.0010) [2023-12-27 04:22:43,107][105692] Updated weights for policy 0, policy_version 1782141 (0.0007) [2023-12-27 04:22:43,310][105620] Updated weights for policy 1, policy_version 1786097 (0.0010) [2023-12-27 04:22:43,374][105620] Updated weights for policy 1, policy_version 1786107 (0.0009) [2023-12-27 04:22:43,437][105620] Updated weights for policy 1, policy_version 1786117 (0.0009) [2023-12-27 04:22:43,743][105692] Updated weights for policy 0, policy_version 1782151 (0.0009) [2023-12-27 04:22:43,808][105692] Updated weights for policy 0, policy_version 1782161 (0.0010) [2023-12-27 04:22:43,874][105692] Updated weights for policy 0, policy_version 1782171 (0.0010) [2023-12-27 04:22:44,124][105620] Updated weights for policy 1, policy_version 1786127 (0.0008) [2023-12-27 04:22:44,172][105620] Updated weights for policy 1, policy_version 1786137 (0.0008) [2023-12-27 04:22:44,217][105620] Updated weights for policy 1, policy_version 1786147 (0.0008) [2023-12-27 04:22:44,593][105692] Updated weights for policy 0, policy_version 1782181 (0.0010) [2023-12-27 04:22:44,644][105692] Updated weights for policy 0, policy_version 1782191 (0.0010) [2023-12-27 04:22:44,695][105692] Updated weights for policy 0, policy_version 1782201 (0.0010) [2023-12-27 04:22:45,002][105620] Updated weights for policy 1, policy_version 1786157 (0.0009) [2023-12-27 04:22:45,051][105620] Updated weights for policy 1, policy_version 1786167 (0.0008) [2023-12-27 04:22:45,101][105620] Updated weights for policy 1, policy_version 1786177 (0.0008) [2023-12-27 04:22:45,467][105692] Updated weights for policy 0, policy_version 1782211 (0.0010) [2023-12-27 04:22:45,532][105692] Updated weights for policy 0, policy_version 1782221 (0.0010) [2023-12-27 04:22:45,598][105692] Updated weights for policy 0, policy_version 1782231 (0.0010) [2023-12-27 04:22:45,905][105620] Updated weights for policy 1, policy_version 1786187 (0.0009) [2023-12-27 04:22:45,969][105620] Updated weights for policy 1, policy_version 1786197 (0.0009) [2023-12-27 04:22:46,033][105620] Updated weights for policy 1, policy_version 1786207 (0.0009) [2023-12-27 04:22:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.7, 300 sec: 19522.0). Total num frames: 913645568. Throughput: 0: 9582.0, 1: 9762.7. Samples: 913622232. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:22:46,062][104569] Avg episode reward: [(0, '8262.350'), (1, '9259.852')] [2023-12-27 04:22:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001782240_456318976.pth... [2023-12-27 04:22:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001781120_456032256.pth [2023-12-27 04:22:46,093][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001786216_457334784.pth... [2023-12-27 04:22:46,098][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001785064_457039872.pth [2023-12-27 04:22:46,271][105692] Updated weights for policy 0, policy_version 1782241 (0.0010) [2023-12-27 04:22:46,323][105692] Updated weights for policy 0, policy_version 1782251 (0.0009) [2023-12-27 04:22:46,370][105692] Updated weights for policy 0, policy_version 1782261 (0.0008) [2023-12-27 04:22:46,427][105692] Updated weights for policy 0, policy_version 1782271 (0.0005) [2023-12-27 04:22:46,640][105620] Updated weights for policy 1, policy_version 1786217 (0.0009) [2023-12-27 04:22:46,686][105620] Updated weights for policy 1, policy_version 1786227 (0.0005) [2023-12-27 04:22:46,734][105620] Updated weights for policy 1, policy_version 1786237 (0.0005) [2023-12-27 04:22:46,788][105620] Updated weights for policy 1, policy_version 1786247 (0.0005) [2023-12-27 04:22:46,993][105692] Updated weights for policy 0, policy_version 1782281 (0.0005) [2023-12-27 04:22:47,040][105692] Updated weights for policy 0, policy_version 1782291 (0.0005) [2023-12-27 04:22:47,085][105692] Updated weights for policy 0, policy_version 1782301 (0.0005) [2023-12-27 04:22:47,352][105620] Updated weights for policy 1, policy_version 1786257 (0.0005) [2023-12-27 04:22:47,416][105620] Updated weights for policy 1, policy_version 1786267 (0.0005) [2023-12-27 04:22:47,478][105620] Updated weights for policy 1, policy_version 1786277 (0.0005) [2023-12-27 04:22:47,649][105692] Updated weights for policy 0, policy_version 1782311 (0.0005) [2023-12-27 04:22:47,707][105692] Updated weights for policy 0, policy_version 1782321 (0.0006) [2023-12-27 04:22:47,766][105692] Updated weights for policy 0, policy_version 1782331 (0.0009) [2023-12-27 04:22:47,987][105620] Updated weights for policy 1, policy_version 1786287 (0.0005) [2023-12-27 04:22:48,049][105620] Updated weights for policy 1, policy_version 1786297 (0.0007) [2023-12-27 04:22:48,102][105620] Updated weights for policy 1, policy_version 1786308 (0.0010) [2023-12-27 04:22:48,404][105692] Updated weights for policy 0, policy_version 1782341 (0.0010) [2023-12-27 04:22:48,457][105692] Updated weights for policy 0, policy_version 1782351 (0.0010) [2023-12-27 04:22:48,516][105692] Updated weights for policy 0, policy_version 1782361 (0.0010) [2023-12-27 04:22:48,868][105620] Updated weights for policy 1, policy_version 1786318 (0.0008) [2023-12-27 04:22:48,935][105620] Updated weights for policy 1, policy_version 1786328 (0.0008) [2023-12-27 04:22:48,994][105620] Updated weights for policy 1, policy_version 1786338 (0.0008) [2023-12-27 04:22:49,281][105692] Updated weights for policy 0, policy_version 1782371 (0.0010) [2023-12-27 04:22:49,361][105692] Updated weights for policy 0, policy_version 1782381 (0.0010) [2023-12-27 04:22:49,420][105692] Updated weights for policy 0, policy_version 1782391 (0.0010) [2023-12-27 04:22:49,729][105620] Updated weights for policy 1, policy_version 1786348 (0.0007) [2023-12-27 04:22:49,786][105620] Updated weights for policy 1, policy_version 1786358 (0.0005) [2023-12-27 04:22:49,855][105620] Updated weights for policy 1, policy_version 1786368 (0.0007) [2023-12-27 04:22:50,199][105692] Updated weights for policy 0, policy_version 1782401 (0.0010) [2023-12-27 04:22:50,258][105692] Updated weights for policy 0, policy_version 1782411 (0.0009) [2023-12-27 04:22:50,318][105692] Updated weights for policy 0, policy_version 1782421 (0.0010) [2023-12-27 04:22:50,375][105692] Updated weights for policy 0, policy_version 1782431 (0.0009) [2023-12-27 04:22:50,422][105620] Updated weights for policy 1, policy_version 1786378 (0.0007) [2023-12-27 04:22:50,486][105620] Updated weights for policy 1, policy_version 1786388 (0.0009) [2023-12-27 04:22:50,549][105620] Updated weights for policy 1, policy_version 1786398 (0.0008) [2023-12-27 04:22:50,617][105620] Updated weights for policy 1, policy_version 1786408 (0.0008) [2023-12-27 04:22:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 913752064. Throughput: 0: 9552.1, 1: 9795.4. Samples: 913744868. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:22:51,062][104569] Avg episode reward: [(0, '8623.602'), (1, '9259.986')] [2023-12-27 04:22:51,231][105692] Updated weights for policy 0, policy_version 1782441 (0.0009) [2023-12-27 04:22:51,286][105620] Updated weights for policy 1, policy_version 1786418 (0.0009) [2023-12-27 04:22:51,296][105692] Updated weights for policy 0, policy_version 1782451 (0.0007) [2023-12-27 04:22:51,350][105620] Updated weights for policy 1, policy_version 1786428 (0.0006) [2023-12-27 04:22:51,364][105692] Updated weights for policy 0, policy_version 1782461 (0.0008) [2023-12-27 04:22:51,412][105620] Updated weights for policy 1, policy_version 1786438 (0.0008) [2023-12-27 04:22:52,065][105692] Updated weights for policy 0, policy_version 1782471 (0.0008) [2023-12-27 04:22:52,127][105692] Updated weights for policy 0, policy_version 1782481 (0.0009) [2023-12-27 04:22:52,150][105620] Updated weights for policy 1, policy_version 1786448 (0.0006) [2023-12-27 04:22:52,187][105692] Updated weights for policy 0, policy_version 1782491 (0.0009) [2023-12-27 04:22:52,204][105620] Updated weights for policy 1, policy_version 1786458 (0.0009) [2023-12-27 04:22:52,263][105620] Updated weights for policy 1, policy_version 1786468 (0.0009) [2023-12-27 04:22:52,926][105692] Updated weights for policy 0, policy_version 1782501 (0.0008) [2023-12-27 04:22:52,974][105692] Updated weights for policy 0, policy_version 1782511 (0.0010) [2023-12-27 04:22:53,032][105692] Updated weights for policy 0, policy_version 1782521 (0.0009) [2023-12-27 04:22:53,065][105620] Updated weights for policy 1, policy_version 1786478 (0.0008) [2023-12-27 04:22:53,120][105620] Updated weights for policy 1, policy_version 1786488 (0.0005) [2023-12-27 04:22:53,179][105620] Updated weights for policy 1, policy_version 1786498 (0.0007) [2023-12-27 04:22:53,743][105692] Updated weights for policy 0, policy_version 1782531 (0.0007) [2023-12-27 04:22:53,797][105692] Updated weights for policy 0, policy_version 1782541 (0.0005) [2023-12-27 04:22:53,847][105692] Updated weights for policy 0, policy_version 1782551 (0.0005) [2023-12-27 04:22:53,908][105620] Updated weights for policy 1, policy_version 1786508 (0.0010) [2023-12-27 04:22:53,956][105620] Updated weights for policy 1, policy_version 1786518 (0.0006) [2023-12-27 04:22:54,017][105620] Updated weights for policy 1, policy_version 1786528 (0.0010) [2023-12-27 04:22:54,494][105692] Updated weights for policy 0, policy_version 1782561 (0.0005) [2023-12-27 04:22:54,550][105692] Updated weights for policy 0, policy_version 1782571 (0.0009) [2023-12-27 04:22:54,599][105692] Updated weights for policy 0, policy_version 1782581 (0.0010) [2023-12-27 04:22:54,645][105620] Updated weights for policy 1, policy_version 1786538 (0.0010) [2023-12-27 04:22:54,651][105692] Updated weights for policy 0, policy_version 1782591 (0.0010) [2023-12-27 04:22:54,704][105620] Updated weights for policy 1, policy_version 1786548 (0.0006) [2023-12-27 04:22:54,771][105620] Updated weights for policy 1, policy_version 1786558 (0.0005) [2023-12-27 04:22:54,840][105620] Updated weights for policy 1, policy_version 1786568 (0.0006) [2023-12-27 04:22:55,296][105692] Updated weights for policy 0, policy_version 1782601 (0.0006) [2023-12-27 04:22:55,357][105692] Updated weights for policy 0, policy_version 1782611 (0.0005) [2023-12-27 04:22:55,416][105692] Updated weights for policy 0, policy_version 1782621 (0.0010) [2023-12-27 04:22:55,468][105620] Updated weights for policy 1, policy_version 1786578 (0.0010) [2023-12-27 04:22:55,520][105620] Updated weights for policy 1, policy_version 1786588 (0.0010) [2023-12-27 04:22:55,569][105620] Updated weights for policy 1, policy_version 1786598 (0.0010) [2023-12-27 04:22:56,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 913850368. Throughput: 0: 9570.2, 1: 9849.1. Samples: 913863368. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:22:56,062][104569] Avg episode reward: [(0, '8260.253'), (1, '9260.057')] [2023-12-27 04:22:56,144][105692] Updated weights for policy 0, policy_version 1782631 (0.0010) [2023-12-27 04:22:56,198][105620] Updated weights for policy 1, policy_version 1786608 (0.0010) [2023-12-27 04:22:56,206][105692] Updated weights for policy 0, policy_version 1782641 (0.0010) [2023-12-27 04:22:56,257][105620] Updated weights for policy 1, policy_version 1786618 (0.0010) [2023-12-27 04:22:56,264][105692] Updated weights for policy 0, policy_version 1782651 (0.0010) [2023-12-27 04:22:56,311][105620] Updated weights for policy 1, policy_version 1786628 (0.0010) [2023-12-27 04:22:57,006][105692] Updated weights for policy 0, policy_version 1782661 (0.0010) [2023-12-27 04:22:57,037][105620] Updated weights for policy 1, policy_version 1786638 (0.0010) [2023-12-27 04:22:57,067][105692] Updated weights for policy 0, policy_version 1782671 (0.0008) [2023-12-27 04:22:57,091][105620] Updated weights for policy 1, policy_version 1786648 (0.0010) [2023-12-27 04:22:57,115][105692] Updated weights for policy 0, policy_version 1782681 (0.0010) [2023-12-27 04:22:57,149][105620] Updated weights for policy 1, policy_version 1786658 (0.0010) [2023-12-27 04:22:57,799][105620] Updated weights for policy 1, policy_version 1786668 (0.0008) [2023-12-27 04:22:57,849][105692] Updated weights for policy 0, policy_version 1782691 (0.0010) [2023-12-27 04:22:57,849][105620] Updated weights for policy 1, policy_version 1786678 (0.0005) [2023-12-27 04:22:57,897][105692] Updated weights for policy 0, policy_version 1782701 (0.0010) [2023-12-27 04:22:57,900][105620] Updated weights for policy 1, policy_version 1786688 (0.0010) [2023-12-27 04:22:57,951][105692] Updated weights for policy 0, policy_version 1782711 (0.0010) [2023-12-27 04:22:58,637][105620] Updated weights for policy 1, policy_version 1786698 (0.0010) [2023-12-27 04:22:58,702][105620] Updated weights for policy 1, policy_version 1786708 (0.0010) [2023-12-27 04:22:58,768][105692] Updated weights for policy 0, policy_version 1782721 (0.0010) [2023-12-27 04:22:58,768][105620] Updated weights for policy 1, policy_version 1786718 (0.0009) [2023-12-27 04:22:58,833][105620] Updated weights for policy 1, policy_version 1786728 (0.0007) [2023-12-27 04:22:58,836][105692] Updated weights for policy 0, policy_version 1782731 (0.0009) [2023-12-27 04:22:58,908][105692] Updated weights for policy 0, policy_version 1782741 (0.0007) [2023-12-27 04:22:58,971][105692] Updated weights for policy 0, policy_version 1782751 (0.0006) [2023-12-27 04:22:59,638][105620] Updated weights for policy 1, policy_version 1786738 (0.0011) [2023-12-27 04:22:59,690][105620] Updated weights for policy 1, policy_version 1786748 (0.0010) [2023-12-27 04:22:59,745][105620] Updated weights for policy 1, policy_version 1786758 (0.0010) [2023-12-27 04:22:59,755][105692] Updated weights for policy 0, policy_version 1782761 (0.0006) [2023-12-27 04:22:59,813][105692] Updated weights for policy 0, policy_version 1782771 (0.0008) [2023-12-27 04:22:59,886][105692] Updated weights for policy 0, policy_version 1782781 (0.0008) [2023-12-27 04:23:00,391][105620] Updated weights for policy 1, policy_version 1786768 (0.0007) [2023-12-27 04:23:00,452][105620] Updated weights for policy 1, policy_version 1786778 (0.0006) [2023-12-27 04:23:00,512][105620] Updated weights for policy 1, policy_version 1786788 (0.0006) [2023-12-27 04:23:00,652][105692] Updated weights for policy 0, policy_version 1782791 (0.0010) [2023-12-27 04:23:00,705][105692] Updated weights for policy 0, policy_version 1782801 (0.0010) [2023-12-27 04:23:00,767][105692] Updated weights for policy 0, policy_version 1782811 (0.0009) [2023-12-27 04:23:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 913948672. Throughput: 0: 9592.7, 1: 9839.6. Samples: 913920676. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:23:01,062][104569] Avg episode reward: [(0, '8535.733'), (1, '9260.290')] [2023-12-27 04:23:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001782816_456466432.pth... [2023-12-27 04:23:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001781696_456179712.pth [2023-12-27 04:23:01,093][105620] Updated weights for policy 1, policy_version 1786798 (0.0008) [2023-12-27 04:23:01,158][105620] Updated weights for policy 1, policy_version 1786808 (0.0009) [2023-12-27 04:23:01,217][105620] Updated weights for policy 1, policy_version 1786818 (0.0010) [2023-12-27 04:23:01,248][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001786824_457490432.pth... [2023-12-27 04:23:01,252][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001785640_457187328.pth [2023-12-27 04:23:01,584][105692] Updated weights for policy 0, policy_version 1782821 (0.0008) [2023-12-27 04:23:01,647][105692] Updated weights for policy 0, policy_version 1782831 (0.0008) [2023-12-27 04:23:01,715][105692] Updated weights for policy 0, policy_version 1782841 (0.0009) [2023-12-27 04:23:01,939][105620] Updated weights for policy 1, policy_version 1786830 (0.0009) [2023-12-27 04:23:01,996][105620] Updated weights for policy 1, policy_version 1786840 (0.0009) [2023-12-27 04:23:02,057][105620] Updated weights for policy 1, policy_version 1786850 (0.0009) [2023-12-27 04:23:02,534][105692] Updated weights for policy 0, policy_version 1782851 (0.0009) [2023-12-27 04:23:02,582][105692] Updated weights for policy 0, policy_version 1782861 (0.0009) [2023-12-27 04:23:02,629][105692] Updated weights for policy 0, policy_version 1782871 (0.0009) [2023-12-27 04:23:02,709][105620] Updated weights for policy 1, policy_version 1786860 (0.0008) [2023-12-27 04:23:02,756][105620] Updated weights for policy 1, policy_version 1786870 (0.0008) [2023-12-27 04:23:02,816][105620] Updated weights for policy 1, policy_version 1786880 (0.0008) [2023-12-27 04:23:03,332][105692] Updated weights for policy 0, policy_version 1782881 (0.0008) [2023-12-27 04:23:03,385][105692] Updated weights for policy 0, policy_version 1782891 (0.0005) [2023-12-27 04:23:03,437][105692] Updated weights for policy 0, policy_version 1782901 (0.0005) [2023-12-27 04:23:03,490][105692] Updated weights for policy 0, policy_version 1782911 (0.0005) [2023-12-27 04:23:03,592][105620] Updated weights for policy 1, policy_version 1786890 (0.0008) [2023-12-27 04:23:03,639][105620] Updated weights for policy 1, policy_version 1786900 (0.0008) [2023-12-27 04:23:03,686][105620] Updated weights for policy 1, policy_version 1786910 (0.0009) [2023-12-27 04:23:04,077][105692] Updated weights for policy 0, policy_version 1782921 (0.0009) [2023-12-27 04:23:04,135][105692] Updated weights for policy 0, policy_version 1782931 (0.0008) [2023-12-27 04:23:04,202][105692] Updated weights for policy 0, policy_version 1782941 (0.0008) [2023-12-27 04:23:04,524][105620] Updated weights for policy 1, policy_version 1786921 (0.0010) [2023-12-27 04:23:04,584][105620] Updated weights for policy 1, policy_version 1786931 (0.0008) [2023-12-27 04:23:04,639][105620] Updated weights for policy 1, policy_version 1786941 (0.0009) [2023-12-27 04:23:04,701][105620] Updated weights for policy 1, policy_version 1786951 (0.0007) [2023-12-27 04:23:04,934][105692] Updated weights for policy 0, policy_version 1782951 (0.0009) [2023-12-27 04:23:04,985][105692] Updated weights for policy 0, policy_version 1782961 (0.0009) [2023-12-27 04:23:05,032][105692] Updated weights for policy 0, policy_version 1782971 (0.0008) [2023-12-27 04:23:05,441][105620] Updated weights for policy 1, policy_version 1786961 (0.0008) [2023-12-27 04:23:05,487][105620] Updated weights for policy 1, policy_version 1786971 (0.0008) [2023-12-27 04:23:05,549][105620] Updated weights for policy 1, policy_version 1786981 (0.0009) [2023-12-27 04:23:05,786][105692] Updated weights for policy 0, policy_version 1782981 (0.0009) [2023-12-27 04:23:05,844][105692] Updated weights for policy 0, policy_version 1782991 (0.0009) [2023-12-27 04:23:05,892][105692] Updated weights for policy 0, policy_version 1783001 (0.0009) [2023-12-27 04:23:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 914046976. Throughput: 0: 9622.5, 1: 9820.6. Samples: 914035240. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:23:06,062][104569] Avg episode reward: [(0, '8806.738'), (1, '9260.302')] [2023-12-27 04:23:06,328][105620] Updated weights for policy 1, policy_version 1786991 (0.0009) [2023-12-27 04:23:06,397][105620] Updated weights for policy 1, policy_version 1787001 (0.0009) [2023-12-27 04:23:06,464][105620] Updated weights for policy 1, policy_version 1787011 (0.0009) [2023-12-27 04:23:06,653][105692] Updated weights for policy 0, policy_version 1783011 (0.0008) [2023-12-27 04:23:06,723][105692] Updated weights for policy 0, policy_version 1783021 (0.0009) [2023-12-27 04:23:06,788][105692] Updated weights for policy 0, policy_version 1783031 (0.0009) [2023-12-27 04:23:07,157][105620] Updated weights for policy 1, policy_version 1787021 (0.0009) [2023-12-27 04:23:07,213][105620] Updated weights for policy 1, policy_version 1787031 (0.0009) [2023-12-27 04:23:07,275][105620] Updated weights for policy 1, policy_version 1787041 (0.0008) [2023-12-27 04:23:07,545][105692] Updated weights for policy 0, policy_version 1783041 (0.0009) [2023-12-27 04:23:07,604][105692] Updated weights for policy 0, policy_version 1783051 (0.0009) [2023-12-27 04:23:07,661][105692] Updated weights for policy 0, policy_version 1783061 (0.0009) [2023-12-27 04:23:07,706][105692] Updated weights for policy 0, policy_version 1783071 (0.0008) [2023-12-27 04:23:07,972][105620] Updated weights for policy 1, policy_version 1787051 (0.0008) [2023-12-27 04:23:08,028][105620] Updated weights for policy 1, policy_version 1787061 (0.0007) [2023-12-27 04:23:08,089][105620] Updated weights for policy 1, policy_version 1787071 (0.0010) [2023-12-27 04:23:08,456][105692] Updated weights for policy 0, policy_version 1783081 (0.0009) [2023-12-27 04:23:08,518][105692] Updated weights for policy 0, policy_version 1783091 (0.0008) [2023-12-27 04:23:08,581][105692] Updated weights for policy 0, policy_version 1783101 (0.0009) [2023-12-27 04:23:08,824][105620] Updated weights for policy 1, policy_version 1787081 (0.0010) [2023-12-27 04:23:08,882][105620] Updated weights for policy 1, policy_version 1787091 (0.0009) [2023-12-27 04:23:08,933][105620] Updated weights for policy 1, policy_version 1787101 (0.0009) [2023-12-27 04:23:08,985][105620] Updated weights for policy 1, policy_version 1787111 (0.0009) [2023-12-27 04:23:09,328][105692] Updated weights for policy 0, policy_version 1783111 (0.0009) [2023-12-27 04:23:09,399][105692] Updated weights for policy 0, policy_version 1783121 (0.0010) [2023-12-27 04:23:09,455][105692] Updated weights for policy 0, policy_version 1783131 (0.0010) [2023-12-27 04:23:09,769][105620] Updated weights for policy 1, policy_version 1787121 (0.0009) [2023-12-27 04:23:09,828][105620] Updated weights for policy 1, policy_version 1787131 (0.0009) [2023-12-27 04:23:09,891][105620] Updated weights for policy 1, policy_version 1787141 (0.0008) [2023-12-27 04:23:10,237][105692] Updated weights for policy 0, policy_version 1783141 (0.0009) [2023-12-27 04:23:10,289][105692] Updated weights for policy 0, policy_version 1783151 (0.0009) [2023-12-27 04:23:10,335][105692] Updated weights for policy 0, policy_version 1783161 (0.0009) [2023-12-27 04:23:10,676][105620] Updated weights for policy 1, policy_version 1787151 (0.0010) [2023-12-27 04:23:10,730][105620] Updated weights for policy 1, policy_version 1787161 (0.0008) [2023-12-27 04:23:10,784][105620] Updated weights for policy 1, policy_version 1787171 (0.0008) [2023-12-27 04:23:11,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.3, 300 sec: 19522.0). Total num frames: 914137088. Throughput: 0: 9578.8, 1: 9822.8. Samples: 914146520. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:23:11,062][104569] Avg episode reward: [(0, '8624.647'), (1, '9260.191')] [2023-12-27 04:23:11,108][105692] Updated weights for policy 0, policy_version 1783171 (0.0007) [2023-12-27 04:23:11,172][105692] Updated weights for policy 0, policy_version 1783181 (0.0008) [2023-12-27 04:23:11,233][105692] Updated weights for policy 0, policy_version 1783191 (0.0008) [2023-12-27 04:23:11,554][105620] Updated weights for policy 1, policy_version 1787181 (0.0009) [2023-12-27 04:23:11,621][105620] Updated weights for policy 1, policy_version 1787191 (0.0009) [2023-12-27 04:23:11,685][105620] Updated weights for policy 1, policy_version 1787201 (0.0008) [2023-12-27 04:23:11,916][105692] Updated weights for policy 0, policy_version 1783201 (0.0008) [2023-12-27 04:23:11,980][105692] Updated weights for policy 0, policy_version 1783211 (0.0007) [2023-12-27 04:23:12,040][105692] Updated weights for policy 0, policy_version 1783221 (0.0008) [2023-12-27 04:23:12,100][105692] Updated weights for policy 0, policy_version 1783231 (0.0007) [2023-12-27 04:23:12,343][105620] Updated weights for policy 1, policy_version 1787211 (0.0007) [2023-12-27 04:23:12,403][105620] Updated weights for policy 1, policy_version 1787221 (0.0006) [2023-12-27 04:23:12,466][105620] Updated weights for policy 1, policy_version 1787231 (0.0006) [2023-12-27 04:23:12,756][105692] Updated weights for policy 0, policy_version 1783241 (0.0009) [2023-12-27 04:23:12,828][105692] Updated weights for policy 0, policy_version 1783251 (0.0009) [2023-12-27 04:23:12,898][105692] Updated weights for policy 0, policy_version 1783261 (0.0010) [2023-12-27 04:23:13,135][105620] Updated weights for policy 1, policy_version 1787241 (0.0008) [2023-12-27 04:23:13,191][105620] Updated weights for policy 1, policy_version 1787251 (0.0008) [2023-12-27 04:23:13,247][105620] Updated weights for policy 1, policy_version 1787261 (0.0005) [2023-12-27 04:23:13,302][105620] Updated weights for policy 1, policy_version 1787271 (0.0005) [2023-12-27 04:23:13,454][105692] Updated weights for policy 0, policy_version 1783272 (0.0006) [2023-12-27 04:23:13,514][105692] Updated weights for policy 0, policy_version 1783282 (0.0005) [2023-12-27 04:23:13,569][105692] Updated weights for policy 0, policy_version 1783292 (0.0005) [2023-12-27 04:23:13,934][105620] Updated weights for policy 1, policy_version 1787281 (0.0005) [2023-12-27 04:23:13,983][105620] Updated weights for policy 1, policy_version 1787291 (0.0005) [2023-12-27 04:23:14,030][105620] Updated weights for policy 1, policy_version 1787301 (0.0005) [2023-12-27 04:23:14,271][105692] Updated weights for policy 0, policy_version 1783302 (0.0005) [2023-12-27 04:23:14,322][105692] Updated weights for policy 0, policy_version 1783312 (0.0005) [2023-12-27 04:23:14,378][105692] Updated weights for policy 0, policy_version 1783322 (0.0005) [2023-12-27 04:23:14,692][105620] Updated weights for policy 1, policy_version 1787311 (0.0009) [2023-12-27 04:23:14,757][105620] Updated weights for policy 1, policy_version 1787321 (0.0006) [2023-12-27 04:23:14,828][105620] Updated weights for policy 1, policy_version 1787331 (0.0009) [2023-12-27 04:23:15,089][105692] Updated weights for policy 0, policy_version 1783332 (0.0009) [2023-12-27 04:23:15,145][105692] Updated weights for policy 0, policy_version 1783342 (0.0010) [2023-12-27 04:23:15,204][105692] Updated weights for policy 0, policy_version 1783352 (0.0010) [2023-12-27 04:23:15,503][105620] Updated weights for policy 1, policy_version 1787341 (0.0011) [2023-12-27 04:23:15,555][105620] Updated weights for policy 1, policy_version 1787351 (0.0011) [2023-12-27 04:23:15,620][105620] Updated weights for policy 1, policy_version 1787361 (0.0010) [2023-12-27 04:23:15,958][105692] Updated weights for policy 0, policy_version 1783362 (0.0011) [2023-12-27 04:23:16,009][105692] Updated weights for policy 0, policy_version 1783372 (0.0010) [2023-12-27 04:23:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 914235392. Throughput: 0: 9674.2, 1: 9840.6. Samples: 914208368. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 04:23:16,062][104569] Avg episode reward: [(0, '8534.160'), (1, '9075.742')] [2023-12-27 04:23:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001787368_457629696.pth... [2023-12-27 04:23:16,068][105692] Updated weights for policy 0, policy_version 1783382 (0.0010) [2023-12-27 04:23:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001786216_457334784.pth [2023-12-27 04:23:16,125][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001783392_456613888.pth... [2023-12-27 04:23:16,125][105692] Updated weights for policy 0, policy_version 1783392 (0.0010) [2023-12-27 04:23:16,130][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001782240_456318976.pth [2023-12-27 04:23:16,334][105620] Updated weights for policy 1, policy_version 1787371 (0.0011) [2023-12-27 04:23:16,393][105620] Updated weights for policy 1, policy_version 1787381 (0.0011) [2023-12-27 04:23:16,448][105620] Updated weights for policy 1, policy_version 1787391 (0.0010) [2023-12-27 04:23:16,720][105692] Updated weights for policy 0, policy_version 1783402 (0.0005) [2023-12-27 04:23:16,779][105692] Updated weights for policy 0, policy_version 1783412 (0.0005) [2023-12-27 04:23:16,842][105692] Updated weights for policy 0, policy_version 1783422 (0.0005) [2023-12-27 04:23:17,210][105620] Updated weights for policy 1, policy_version 1787401 (0.0010) [2023-12-27 04:23:17,275][105620] Updated weights for policy 1, policy_version 1787411 (0.0010) [2023-12-27 04:23:17,343][105620] Updated weights for policy 1, policy_version 1787421 (0.0011) [2023-12-27 04:23:17,393][105692] Updated weights for policy 0, policy_version 1783432 (0.0007) [2023-12-27 04:23:17,411][105620] Updated weights for policy 1, policy_version 1787431 (0.0011) [2023-12-27 04:23:17,451][105692] Updated weights for policy 0, policy_version 1783442 (0.0006) [2023-12-27 04:23:17,507][105692] Updated weights for policy 0, policy_version 1783452 (0.0005) [2023-12-27 04:23:18,015][105692] Updated weights for policy 0, policy_version 1783462 (0.0005) [2023-12-27 04:23:18,066][105692] Updated weights for policy 0, policy_version 1783472 (0.0005) [2023-12-27 04:23:18,123][105692] Updated weights for policy 0, policy_version 1783482 (0.0007) [2023-12-27 04:23:18,138][105620] Updated weights for policy 1, policy_version 1787441 (0.0010) [2023-12-27 04:23:18,194][105620] Updated weights for policy 1, policy_version 1787451 (0.0011) [2023-12-27 04:23:18,248][105620] Updated weights for policy 1, policy_version 1787461 (0.0010) [2023-12-27 04:23:18,751][105692] Updated weights for policy 0, policy_version 1783492 (0.0005) [2023-12-27 04:23:18,813][105692] Updated weights for policy 0, policy_version 1783502 (0.0006) [2023-12-27 04:23:18,872][105692] Updated weights for policy 0, policy_version 1783512 (0.0008) [2023-12-27 04:23:19,006][105620] Updated weights for policy 1, policy_version 1787471 (0.0009) [2023-12-27 04:23:19,052][105620] Updated weights for policy 1, policy_version 1787481 (0.0010) [2023-12-27 04:23:19,105][105620] Updated weights for policy 1, policy_version 1787491 (0.0006) [2023-12-27 04:23:19,595][105692] Updated weights for policy 0, policy_version 1783522 (0.0007) [2023-12-27 04:23:19,668][105692] Updated weights for policy 0, policy_version 1783532 (0.0007) [2023-12-27 04:23:19,736][105692] Updated weights for policy 0, policy_version 1783542 (0.0009) [2023-12-27 04:23:19,796][105620] Updated weights for policy 1, policy_version 1787501 (0.0008) [2023-12-27 04:23:19,796][105692] Updated weights for policy 0, policy_version 1783552 (0.0011) [2023-12-27 04:23:19,855][105620] Updated weights for policy 1, policy_version 1787511 (0.0010) [2023-12-27 04:23:19,915][105620] Updated weights for policy 1, policy_version 1787521 (0.0009) [2023-12-27 04:23:20,534][105692] Updated weights for policy 0, policy_version 1783562 (0.0009) [2023-12-27 04:23:20,598][105692] Updated weights for policy 0, policy_version 1783572 (0.0010) [2023-12-27 04:23:20,660][105692] Updated weights for policy 0, policy_version 1783582 (0.0008) [2023-12-27 04:23:20,685][105620] Updated weights for policy 1, policy_version 1787531 (0.0011) [2023-12-27 04:23:20,738][105620] Updated weights for policy 1, policy_version 1787541 (0.0011) [2023-12-27 04:23:20,801][105620] Updated weights for policy 1, policy_version 1787551 (0.0011) [2023-12-27 04:23:21,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 914341888. Throughput: 0: 9774.6, 1: 9830.0. Samples: 914330128. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:23:21,063][104569] Avg episode reward: [(0, '8263.167'), (1, '9167.961')] [2023-12-27 04:23:21,399][105692] Updated weights for policy 0, policy_version 1783592 (0.0009) [2023-12-27 04:23:21,457][105692] Updated weights for policy 0, policy_version 1783602 (0.0009) [2023-12-27 04:23:21,520][105620] Updated weights for policy 1, policy_version 1787561 (0.0007) [2023-12-27 04:23:21,523][105692] Updated weights for policy 0, policy_version 1783612 (0.0008) [2023-12-27 04:23:21,584][105620] Updated weights for policy 1, policy_version 1787571 (0.0011) [2023-12-27 04:23:21,656][105620] Updated weights for policy 1, policy_version 1787581 (0.0010) [2023-12-27 04:23:21,708][105620] Updated weights for policy 1, policy_version 1787591 (0.0011) [2023-12-27 04:23:22,237][105692] Updated weights for policy 0, policy_version 1783622 (0.0009) [2023-12-27 04:23:22,301][105692] Updated weights for policy 0, policy_version 1783632 (0.0008) [2023-12-27 04:23:22,370][105692] Updated weights for policy 0, policy_version 1783642 (0.0008) [2023-12-27 04:23:22,403][105620] Updated weights for policy 1, policy_version 1787601 (0.0011) [2023-12-27 04:23:22,465][105620] Updated weights for policy 1, policy_version 1787611 (0.0011) [2023-12-27 04:23:22,527][105620] Updated weights for policy 1, policy_version 1787621 (0.0010) [2023-12-27 04:23:23,091][105692] Updated weights for policy 0, policy_version 1783652 (0.0009) [2023-12-27 04:23:23,143][105692] Updated weights for policy 0, policy_version 1783662 (0.0009) [2023-12-27 04:23:23,210][105692] Updated weights for policy 0, policy_version 1783672 (0.0010) [2023-12-27 04:23:23,248][105620] Updated weights for policy 1, policy_version 1787631 (0.0007) [2023-12-27 04:23:23,300][105620] Updated weights for policy 1, policy_version 1787641 (0.0010) [2023-12-27 04:23:23,359][105620] Updated weights for policy 1, policy_version 1787651 (0.0010) [2023-12-27 04:23:23,888][105692] Updated weights for policy 0, policy_version 1783682 (0.0008) [2023-12-27 04:23:23,943][105692] Updated weights for policy 0, policy_version 1783692 (0.0005) [2023-12-27 04:23:23,994][105692] Updated weights for policy 0, policy_version 1783702 (0.0005) [2023-12-27 04:23:24,029][105620] Updated weights for policy 1, policy_version 1787661 (0.0008) [2023-12-27 04:23:24,047][105692] Updated weights for policy 0, policy_version 1783712 (0.0006) [2023-12-27 04:23:24,083][105620] Updated weights for policy 1, policy_version 1787671 (0.0007) [2023-12-27 04:23:24,141][105620] Updated weights for policy 1, policy_version 1787681 (0.0008) [2023-12-27 04:23:24,808][105620] Updated weights for policy 1, policy_version 1787691 (0.0008) [2023-12-27 04:23:24,821][105692] Updated weights for policy 0, policy_version 1783722 (0.0010) [2023-12-27 04:23:24,868][105620] Updated weights for policy 1, policy_version 1787701 (0.0006) [2023-12-27 04:23:24,878][105692] Updated weights for policy 0, policy_version 1783732 (0.0008) [2023-12-27 04:23:24,924][105620] Updated weights for policy 1, policy_version 1787711 (0.0007) [2023-12-27 04:23:24,931][105692] Updated weights for policy 0, policy_version 1783742 (0.0008) [2023-12-27 04:23:25,655][105620] Updated weights for policy 1, policy_version 1787721 (0.0008) [2023-12-27 04:23:25,690][105692] Updated weights for policy 0, policy_version 1783752 (0.0007) [2023-12-27 04:23:25,718][105620] Updated weights for policy 1, policy_version 1787731 (0.0007) [2023-12-27 04:23:25,752][105692] Updated weights for policy 0, policy_version 1783762 (0.0007) [2023-12-27 04:23:25,782][105620] Updated weights for policy 1, policy_version 1787741 (0.0007) [2023-12-27 04:23:25,815][105692] Updated weights for policy 0, policy_version 1783772 (0.0008) [2023-12-27 04:23:25,842][105620] Updated weights for policy 1, policy_version 1787751 (0.0006) [2023-12-27 04:23:26,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 914440192. Throughput: 0: 9776.5, 1: 9830.0. Samples: 914445880. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:23:26,063][104569] Avg episode reward: [(0, '8169.890'), (1, '9259.928')] [2023-12-27 04:23:26,452][105620] Updated weights for policy 1, policy_version 1787761 (0.0005) [2023-12-27 04:23:26,503][105620] Updated weights for policy 1, policy_version 1787771 (0.0008) [2023-12-27 04:23:26,514][105692] Updated weights for policy 0, policy_version 1783782 (0.0007) [2023-12-27 04:23:26,550][105620] Updated weights for policy 1, policy_version 1787781 (0.0005) [2023-12-27 04:23:26,571][105692] Updated weights for policy 0, policy_version 1783792 (0.0009) [2023-12-27 04:23:26,633][105692] Updated weights for policy 0, policy_version 1783802 (0.0009) [2023-12-27 04:23:27,250][105620] Updated weights for policy 1, policy_version 1787791 (0.0009) [2023-12-27 04:23:27,277][105692] Updated weights for policy 0, policy_version 1783812 (0.0006) [2023-12-27 04:23:27,318][105620] Updated weights for policy 1, policy_version 1787801 (0.0008) [2023-12-27 04:23:27,336][105692] Updated weights for policy 0, policy_version 1783822 (0.0009) [2023-12-27 04:23:27,375][105620] Updated weights for policy 1, policy_version 1787811 (0.0009) [2023-12-27 04:23:27,397][105692] Updated weights for policy 0, policy_version 1783832 (0.0010) [2023-12-27 04:23:27,995][105620] Updated weights for policy 1, policy_version 1787821 (0.0010) [2023-12-27 04:23:28,000][105692] Updated weights for policy 0, policy_version 1783842 (0.0009) [2023-12-27 04:23:28,053][105620] Updated weights for policy 1, policy_version 1787831 (0.0010) [2023-12-27 04:23:28,056][105692] Updated weights for policy 0, policy_version 1783852 (0.0010) [2023-12-27 04:23:28,103][105692] Updated weights for policy 0, policy_version 1783862 (0.0010) [2023-12-27 04:23:28,103][105620] Updated weights for policy 1, policy_version 1787841 (0.0010) [2023-12-27 04:23:28,151][105692] Updated weights for policy 0, policy_version 1783872 (0.0010) [2023-12-27 04:23:28,811][105620] Updated weights for policy 1, policy_version 1787851 (0.0010) [2023-12-27 04:23:28,848][105692] Updated weights for policy 0, policy_version 1783882 (0.0011) [2023-12-27 04:23:28,866][105620] Updated weights for policy 1, policy_version 1787861 (0.0010) [2023-12-27 04:23:28,899][105692] Updated weights for policy 0, policy_version 1783892 (0.0010) [2023-12-27 04:23:28,917][105620] Updated weights for policy 1, policy_version 1787871 (0.0010) [2023-12-27 04:23:28,956][105692] Updated weights for policy 0, policy_version 1783902 (0.0010) [2023-12-27 04:23:29,675][105620] Updated weights for policy 1, policy_version 1787881 (0.0010) [2023-12-27 04:23:29,692][105692] Updated weights for policy 0, policy_version 1783912 (0.0010) [2023-12-27 04:23:29,734][105620] Updated weights for policy 1, policy_version 1787891 (0.0011) [2023-12-27 04:23:29,748][105692] Updated weights for policy 0, policy_version 1783922 (0.0008) [2023-12-27 04:23:29,781][105620] Updated weights for policy 1, policy_version 1787901 (0.0009) [2023-12-27 04:23:29,806][105692] Updated weights for policy 0, policy_version 1783932 (0.0010) [2023-12-27 04:23:29,834][105620] Updated weights for policy 1, policy_version 1787911 (0.0008) [2023-12-27 04:23:30,555][105692] Updated weights for policy 0, policy_version 1783942 (0.0011) [2023-12-27 04:23:30,605][105620] Updated weights for policy 1, policy_version 1787921 (0.0010) [2023-12-27 04:23:30,612][105692] Updated weights for policy 0, policy_version 1783952 (0.0010) [2023-12-27 04:23:30,660][105620] Updated weights for policy 1, policy_version 1787931 (0.0006) [2023-12-27 04:23:30,664][105692] Updated weights for policy 0, policy_version 1783962 (0.0010) [2023-12-27 04:23:30,717][105620] Updated weights for policy 1, policy_version 1787941 (0.0006) [2023-12-27 04:23:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 914538496. Throughput: 0: 9811.3, 1: 9879.4. Samples: 914508312. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:23:31,062][104569] Avg episode reward: [(0, '8806.301'), (1, '9259.990')] [2023-12-27 04:23:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001783968_456761344.pth... [2023-12-27 04:23:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001787944_457777152.pth... [2023-12-27 04:23:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001786824_457490432.pth [2023-12-27 04:23:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001782816_456466432.pth [2023-12-27 04:23:31,401][105692] Updated weights for policy 0, policy_version 1783972 (0.0010) [2023-12-27 04:23:31,421][105620] Updated weights for policy 1, policy_version 1787951 (0.0007) [2023-12-27 04:23:31,461][105692] Updated weights for policy 0, policy_version 1783982 (0.0008) [2023-12-27 04:23:31,486][105620] Updated weights for policy 1, policy_version 1787961 (0.0008) [2023-12-27 04:23:31,516][105692] Updated weights for policy 0, policy_version 1783992 (0.0008) [2023-12-27 04:23:31,549][105620] Updated weights for policy 1, policy_version 1787971 (0.0007) [2023-12-27 04:23:32,203][105692] Updated weights for policy 0, policy_version 1784002 (0.0008) [2023-12-27 04:23:32,262][105692] Updated weights for policy 0, policy_version 1784012 (0.0009) [2023-12-27 04:23:32,309][105620] Updated weights for policy 1, policy_version 1787981 (0.0009) [2023-12-27 04:23:32,320][105692] Updated weights for policy 0, policy_version 1784022 (0.0006) [2023-12-27 04:23:32,374][105620] Updated weights for policy 1, policy_version 1787991 (0.0008) [2023-12-27 04:23:32,383][105692] Updated weights for policy 0, policy_version 1784032 (0.0007) [2023-12-27 04:23:32,429][105620] Updated weights for policy 1, policy_version 1788001 (0.0006) [2023-12-27 04:23:33,016][105620] Updated weights for policy 1, policy_version 1788011 (0.0005) [2023-12-27 04:23:33,069][105620] Updated weights for policy 1, policy_version 1788021 (0.0005) [2023-12-27 04:23:33,125][105620] Updated weights for policy 1, policy_version 1788031 (0.0006) [2023-12-27 04:23:33,248][105692] Updated weights for policy 0, policy_version 1784042 (0.0010) [2023-12-27 04:23:33,297][105692] Updated weights for policy 0, policy_version 1784052 (0.0009) [2023-12-27 04:23:33,357][105692] Updated weights for policy 0, policy_version 1784062 (0.0009) [2023-12-27 04:23:33,739][105620] Updated weights for policy 1, policy_version 1788041 (0.0008) [2023-12-27 04:23:33,791][105620] Updated weights for policy 1, policy_version 1788051 (0.0005) [2023-12-27 04:23:33,850][105620] Updated weights for policy 1, policy_version 1788061 (0.0006) [2023-12-27 04:23:33,896][105620] Updated weights for policy 1, policy_version 1788071 (0.0006) [2023-12-27 04:23:34,087][105692] Updated weights for policy 0, policy_version 1784073 (0.0010) [2023-12-27 04:23:34,140][105692] Updated weights for policy 0, policy_version 1784083 (0.0010) [2023-12-27 04:23:34,199][105692] Updated weights for policy 0, policy_version 1784093 (0.0008) [2023-12-27 04:23:34,472][105620] Updated weights for policy 1, policy_version 1788081 (0.0006) [2023-12-27 04:23:34,538][105620] Updated weights for policy 1, policy_version 1788091 (0.0009) [2023-12-27 04:23:34,602][105620] Updated weights for policy 1, policy_version 1788101 (0.0009) [2023-12-27 04:23:35,001][105692] Updated weights for policy 0, policy_version 1784104 (0.0006) [2023-12-27 04:23:35,062][105692] Updated weights for policy 0, policy_version 1784114 (0.0005) [2023-12-27 04:23:35,118][105692] Updated weights for policy 0, policy_version 1784124 (0.0005) [2023-12-27 04:23:35,382][105620] Updated weights for policy 1, policy_version 1788111 (0.0008) [2023-12-27 04:23:35,435][105620] Updated weights for policy 1, policy_version 1788121 (0.0009) [2023-12-27 04:23:35,500][105620] Updated weights for policy 1, policy_version 1788131 (0.0010) [2023-12-27 04:23:35,684][105692] Updated weights for policy 0, policy_version 1784134 (0.0008) [2023-12-27 04:23:35,747][105692] Updated weights for policy 0, policy_version 1784144 (0.0010) [2023-12-27 04:23:35,808][105692] Updated weights for policy 0, policy_version 1784154 (0.0009) [2023-12-27 04:23:36,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 914636800. Throughput: 0: 9680.9, 1: 9869.2. Samples: 914624620. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:23:36,062][104569] Avg episode reward: [(0, '8989.756'), (1, '9259.976')] [2023-12-27 04:23:36,172][105620] Updated weights for policy 1, policy_version 1788141 (0.0009) [2023-12-27 04:23:36,232][105620] Updated weights for policy 1, policy_version 1788151 (0.0007) [2023-12-27 04:23:36,292][105620] Updated weights for policy 1, policy_version 1788161 (0.0006) [2023-12-27 04:23:36,643][105692] Updated weights for policy 0, policy_version 1784164 (0.0007) [2023-12-27 04:23:36,709][105692] Updated weights for policy 0, policy_version 1784174 (0.0007) [2023-12-27 04:23:36,772][105692] Updated weights for policy 0, policy_version 1784184 (0.0009) [2023-12-27 04:23:36,903][105620] Updated weights for policy 1, policy_version 1788171 (0.0005) [2023-12-27 04:23:36,963][105620] Updated weights for policy 1, policy_version 1788181 (0.0008) [2023-12-27 04:23:37,016][105620] Updated weights for policy 1, policy_version 1788191 (0.0009) [2023-12-27 04:23:37,378][105692] Updated weights for policy 0, policy_version 1784194 (0.0006) [2023-12-27 04:23:37,430][105692] Updated weights for policy 0, policy_version 1784204 (0.0006) [2023-12-27 04:23:37,488][105692] Updated weights for policy 0, policy_version 1784214 (0.0009) [2023-12-27 04:23:37,545][105692] Updated weights for policy 0, policy_version 1784224 (0.0009) [2023-12-27 04:23:37,715][105620] Updated weights for policy 1, policy_version 1788201 (0.0008) [2023-12-27 04:23:37,776][105620] Updated weights for policy 1, policy_version 1788211 (0.0009) [2023-12-27 04:23:37,841][105620] Updated weights for policy 1, policy_version 1788221 (0.0009) [2023-12-27 04:23:37,902][105620] Updated weights for policy 1, policy_version 1788231 (0.0009) [2023-12-27 04:23:38,198][105692] Updated weights for policy 0, policy_version 1784234 (0.0010) [2023-12-27 04:23:38,254][105692] Updated weights for policy 0, policy_version 1784244 (0.0009) [2023-12-27 04:23:38,306][105692] Updated weights for policy 0, policy_version 1784254 (0.0009) [2023-12-27 04:23:38,719][105620] Updated weights for policy 1, policy_version 1788241 (0.0009) [2023-12-27 04:23:38,777][105620] Updated weights for policy 1, policy_version 1788251 (0.0009) [2023-12-27 04:23:38,829][105620] Updated weights for policy 1, policy_version 1788261 (0.0009) [2023-12-27 04:23:39,004][105692] Updated weights for policy 0, policy_version 1784264 (0.0009) [2023-12-27 04:23:39,066][105692] Updated weights for policy 0, policy_version 1784274 (0.0009) [2023-12-27 04:23:39,128][105692] Updated weights for policy 0, policy_version 1784284 (0.0009) [2023-12-27 04:23:39,571][105620] Updated weights for policy 1, policy_version 1788271 (0.0007) [2023-12-27 04:23:39,634][105620] Updated weights for policy 1, policy_version 1788281 (0.0005) [2023-12-27 04:23:39,698][105620] Updated weights for policy 1, policy_version 1788291 (0.0006) [2023-12-27 04:23:39,971][105692] Updated weights for policy 0, policy_version 1784294 (0.0009) [2023-12-27 04:23:40,038][105692] Updated weights for policy 0, policy_version 1784304 (0.0008) [2023-12-27 04:23:40,100][105692] Updated weights for policy 0, policy_version 1784314 (0.0008) [2023-12-27 04:23:40,319][105620] Updated weights for policy 1, policy_version 1788301 (0.0009) [2023-12-27 04:23:40,385][105620] Updated weights for policy 1, policy_version 1788311 (0.0011) [2023-12-27 04:23:40,447][105620] Updated weights for policy 1, policy_version 1788321 (0.0010) [2023-12-27 04:23:40,830][105692] Updated weights for policy 0, policy_version 1784324 (0.0009) [2023-12-27 04:23:40,888][105692] Updated weights for policy 0, policy_version 1784334 (0.0010) [2023-12-27 04:23:40,947][105692] Updated weights for policy 0, policy_version 1784344 (0.0010) [2023-12-27 04:23:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 914735104. Throughput: 0: 9705.2, 1: 9816.9. Samples: 914741864. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:23:41,063][104569] Avg episode reward: [(0, '8897.342'), (1, '9074.956')] [2023-12-27 04:23:41,158][105620] Updated weights for policy 1, policy_version 1788331 (0.0011) [2023-12-27 04:23:41,218][105620] Updated weights for policy 1, policy_version 1788341 (0.0011) [2023-12-27 04:23:41,283][105620] Updated weights for policy 1, policy_version 1788351 (0.0011) [2023-12-27 04:23:41,679][105692] Updated weights for policy 0, policy_version 1784354 (0.0011) [2023-12-27 04:23:41,747][105692] Updated weights for policy 0, policy_version 1784364 (0.0010) [2023-12-27 04:23:41,801][105692] Updated weights for policy 0, policy_version 1784374 (0.0010) [2023-12-27 04:23:41,858][105692] Updated weights for policy 0, policy_version 1784384 (0.0008) [2023-12-27 04:23:42,036][105620] Updated weights for policy 1, policy_version 1788361 (0.0010) [2023-12-27 04:23:42,088][105620] Updated weights for policy 1, policy_version 1788371 (0.0010) [2023-12-27 04:23:42,141][105620] Updated weights for policy 1, policy_version 1788381 (0.0010) [2023-12-27 04:23:42,204][105620] Updated weights for policy 1, policy_version 1788391 (0.0011) [2023-12-27 04:23:42,679][105692] Updated weights for policy 0, policy_version 1784394 (0.0011) [2023-12-27 04:23:42,735][105692] Updated weights for policy 0, policy_version 1784404 (0.0008) [2023-12-27 04:23:42,794][105692] Updated weights for policy 0, policy_version 1784414 (0.0008) [2023-12-27 04:23:42,865][105620] Updated weights for policy 1, policy_version 1788401 (0.0010) [2023-12-27 04:23:42,919][105620] Updated weights for policy 1, policy_version 1788411 (0.0010) [2023-12-27 04:23:42,977][105620] Updated weights for policy 1, policy_version 1788421 (0.0010) [2023-12-27 04:23:43,469][105692] Updated weights for policy 0, policy_version 1784424 (0.0009) [2023-12-27 04:23:43,539][105692] Updated weights for policy 0, policy_version 1784434 (0.0009) [2023-12-27 04:23:43,605][105692] Updated weights for policy 0, policy_version 1784444 (0.0010) [2023-12-27 04:23:43,687][105620] Updated weights for policy 1, policy_version 1788431 (0.0010) [2023-12-27 04:23:43,735][105620] Updated weights for policy 1, policy_version 1788441 (0.0010) [2023-12-27 04:23:43,804][105620] Updated weights for policy 1, policy_version 1788451 (0.0010) [2023-12-27 04:23:44,166][105692] Updated weights for policy 0, policy_version 1784454 (0.0008) [2023-12-27 04:23:44,235][105692] Updated weights for policy 0, policy_version 1784464 (0.0005) [2023-12-27 04:23:44,326][105692] Updated weights for policy 0, policy_version 1784474 (0.0008) [2023-12-27 04:23:44,537][105620] Updated weights for policy 1, policy_version 1788461 (0.0010) [2023-12-27 04:23:44,585][105620] Updated weights for policy 1, policy_version 1788471 (0.0010) [2023-12-27 04:23:44,629][105620] Updated weights for policy 1, policy_version 1788481 (0.0010) [2023-12-27 04:23:44,946][105692] Updated weights for policy 0, policy_version 1784484 (0.0009) [2023-12-27 04:23:45,009][105692] Updated weights for policy 0, policy_version 1784494 (0.0010) [2023-12-27 04:23:45,071][105692] Updated weights for policy 0, policy_version 1784504 (0.0007) [2023-12-27 04:23:45,390][105620] Updated weights for policy 1, policy_version 1788491 (0.0010) [2023-12-27 04:23:45,453][105620] Updated weights for policy 1, policy_version 1788501 (0.0011) [2023-12-27 04:23:45,509][105620] Updated weights for policy 1, policy_version 1788511 (0.0010) [2023-12-27 04:23:45,794][105692] Updated weights for policy 0, policy_version 1784514 (0.0006) [2023-12-27 04:23:45,846][105692] Updated weights for policy 0, policy_version 1784524 (0.0010) [2023-12-27 04:23:45,902][105692] Updated weights for policy 0, policy_version 1784534 (0.0008) [2023-12-27 04:23:45,959][105692] Updated weights for policy 0, policy_version 1784544 (0.0007) [2023-12-27 04:23:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 914833408. Throughput: 0: 9705.1, 1: 9815.6. Samples: 914799112. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:23:46,063][104569] Avg episode reward: [(0, '8716.639'), (1, '9167.437')] [2023-12-27 04:23:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001784544_456908800.pth... [2023-12-27 04:23:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001788520_457924608.pth... [2023-12-27 04:23:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001783392_456613888.pth [2023-12-27 04:23:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001787368_457629696.pth [2023-12-27 04:23:46,202][105620] Updated weights for policy 1, policy_version 1788521 (0.0010) [2023-12-27 04:23:46,257][105620] Updated weights for policy 1, policy_version 1788531 (0.0011) [2023-12-27 04:23:46,312][105620] Updated weights for policy 1, policy_version 1788541 (0.0010) [2023-12-27 04:23:46,356][105620] Updated weights for policy 1, policy_version 1788551 (0.0010) [2023-12-27 04:23:46,711][105692] Updated weights for policy 0, policy_version 1784554 (0.0010) [2023-12-27 04:23:46,770][105692] Updated weights for policy 0, policy_version 1784564 (0.0011) [2023-12-27 04:23:46,828][105692] Updated weights for policy 0, policy_version 1784574 (0.0010) [2023-12-27 04:23:47,107][105620] Updated weights for policy 1, policy_version 1788561 (0.0010) [2023-12-27 04:23:47,169][105620] Updated weights for policy 1, policy_version 1788571 (0.0010) [2023-12-27 04:23:47,237][105620] Updated weights for policy 1, policy_version 1788581 (0.0011) [2023-12-27 04:23:47,466][105692] Updated weights for policy 0, policy_version 1784584 (0.0006) [2023-12-27 04:23:47,519][105692] Updated weights for policy 0, policy_version 1784594 (0.0008) [2023-12-27 04:23:47,563][105692] Updated weights for policy 0, policy_version 1784604 (0.0010) [2023-12-27 04:23:47,969][105620] Updated weights for policy 1, policy_version 1788591 (0.0011) [2023-12-27 04:23:48,017][105620] Updated weights for policy 1, policy_version 1788601 (0.0010) [2023-12-27 04:23:48,072][105620] Updated weights for policy 1, policy_version 1788611 (0.0010) [2023-12-27 04:23:48,209][105692] Updated weights for policy 0, policy_version 1784614 (0.0007) [2023-12-27 04:23:48,258][105692] Updated weights for policy 0, policy_version 1784624 (0.0005) [2023-12-27 04:23:48,312][105692] Updated weights for policy 0, policy_version 1784634 (0.0006) [2023-12-27 04:23:48,793][105620] Updated weights for policy 1, policy_version 1788621 (0.0011) [2023-12-27 04:23:48,849][105620] Updated weights for policy 1, policy_version 1788631 (0.0006) [2023-12-27 04:23:48,900][105620] Updated weights for policy 1, policy_version 1788641 (0.0007) [2023-12-27 04:23:49,031][105692] Updated weights for policy 0, policy_version 1784644 (0.0010) [2023-12-27 04:23:49,079][105692] Updated weights for policy 0, policy_version 1784654 (0.0010) [2023-12-27 04:23:49,134][105692] Updated weights for policy 0, policy_version 1784664 (0.0010) [2023-12-27 04:23:49,489][105620] Updated weights for policy 1, policy_version 1788651 (0.0006) [2023-12-27 04:23:49,542][105620] Updated weights for policy 1, policy_version 1788661 (0.0008) [2023-12-27 04:23:49,608][105620] Updated weights for policy 1, policy_version 1788671 (0.0008) [2023-12-27 04:23:49,975][105692] Updated weights for policy 0, policy_version 1784674 (0.0012) [2023-12-27 04:23:50,036][105692] Updated weights for policy 0, policy_version 1784684 (0.0009) [2023-12-27 04:23:50,101][105692] Updated weights for policy 0, policy_version 1784694 (0.0007) [2023-12-27 04:23:50,163][105692] Updated weights for policy 0, policy_version 1784704 (0.0006) [2023-12-27 04:23:50,344][105620] Updated weights for policy 1, policy_version 1788681 (0.0007) [2023-12-27 04:23:50,414][105620] Updated weights for policy 1, policy_version 1788691 (0.0005) [2023-12-27 04:23:50,465][105620] Updated weights for policy 1, policy_version 1788701 (0.0008) [2023-12-27 04:23:50,519][105620] Updated weights for policy 1, policy_version 1788711 (0.0008) [2023-12-27 04:23:50,854][105692] Updated weights for policy 0, policy_version 1784714 (0.0008) [2023-12-27 04:23:50,918][105692] Updated weights for policy 0, policy_version 1784724 (0.0010) [2023-12-27 04:23:50,986][105692] Updated weights for policy 0, policy_version 1784734 (0.0009) [2023-12-27 04:23:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 914931712. Throughput: 0: 9818.0, 1: 9828.5. Samples: 914919336. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:23:51,062][104569] Avg episode reward: [(0, '8626.456'), (1, '9352.557')] [2023-12-27 04:23:51,268][105620] Updated weights for policy 1, policy_version 1788721 (0.0008) [2023-12-27 04:23:51,323][105620] Updated weights for policy 1, policy_version 1788731 (0.0008) [2023-12-27 04:23:51,385][105620] Updated weights for policy 1, policy_version 1788741 (0.0009) [2023-12-27 04:23:51,726][105692] Updated weights for policy 0, policy_version 1784744 (0.0008) [2023-12-27 04:23:51,790][105692] Updated weights for policy 0, policy_version 1784754 (0.0008) [2023-12-27 04:23:51,848][105692] Updated weights for policy 0, policy_version 1784764 (0.0009) [2023-12-27 04:23:52,036][105620] Updated weights for policy 1, policy_version 1788751 (0.0006) [2023-12-27 04:23:52,089][105620] Updated weights for policy 1, policy_version 1788761 (0.0005) [2023-12-27 04:23:52,142][105620] Updated weights for policy 1, policy_version 1788771 (0.0005) [2023-12-27 04:23:52,556][105692] Updated weights for policy 0, policy_version 1784774 (0.0007) [2023-12-27 04:23:52,626][105692] Updated weights for policy 0, policy_version 1784784 (0.0007) [2023-12-27 04:23:52,692][105692] Updated weights for policy 0, policy_version 1784794 (0.0009) [2023-12-27 04:23:52,790][105620] Updated weights for policy 1, policy_version 1788781 (0.0005) [2023-12-27 04:23:52,843][105620] Updated weights for policy 1, policy_version 1788791 (0.0005) [2023-12-27 04:23:52,890][105620] Updated weights for policy 1, policy_version 1788801 (0.0005) [2023-12-27 04:23:53,350][105692] Updated weights for policy 0, policy_version 1784804 (0.0008) [2023-12-27 04:23:53,404][105692] Updated weights for policy 0, policy_version 1784814 (0.0006) [2023-12-27 04:23:53,457][105692] Updated weights for policy 0, policy_version 1784824 (0.0006) [2023-12-27 04:23:53,611][105620] Updated weights for policy 1, policy_version 1788811 (0.0007) [2023-12-27 04:23:53,665][105620] Updated weights for policy 1, policy_version 1788821 (0.0009) [2023-12-27 04:23:53,730][105620] Updated weights for policy 1, policy_version 1788831 (0.0009) [2023-12-27 04:23:54,124][105692] Updated weights for policy 0, policy_version 1784834 (0.0007) [2023-12-27 04:23:54,186][105692] Updated weights for policy 0, policy_version 1784844 (0.0009) [2023-12-27 04:23:54,242][105692] Updated weights for policy 0, policy_version 1784854 (0.0009) [2023-12-27 04:23:54,299][105692] Updated weights for policy 0, policy_version 1784864 (0.0009) [2023-12-27 04:23:54,454][105620] Updated weights for policy 1, policy_version 1788841 (0.0009) [2023-12-27 04:23:54,511][105620] Updated weights for policy 1, policy_version 1788851 (0.0006) [2023-12-27 04:23:54,569][105620] Updated weights for policy 1, policy_version 1788861 (0.0006) [2023-12-27 04:23:54,630][105620] Updated weights for policy 1, policy_version 1788871 (0.0005) [2023-12-27 04:23:55,058][105692] Updated weights for policy 0, policy_version 1784874 (0.0006) [2023-12-27 04:23:55,112][105692] Updated weights for policy 0, policy_version 1784884 (0.0005) [2023-12-27 04:23:55,158][105692] Updated weights for policy 0, policy_version 1784894 (0.0005) [2023-12-27 04:23:55,266][105620] Updated weights for policy 1, policy_version 1788881 (0.0008) [2023-12-27 04:23:55,327][105620] Updated weights for policy 1, policy_version 1788891 (0.0009) [2023-12-27 04:23:55,383][105620] Updated weights for policy 1, policy_version 1788901 (0.0009) [2023-12-27 04:23:55,815][105692] Updated weights for policy 0, policy_version 1784904 (0.0008) [2023-12-27 04:23:55,868][105692] Updated weights for policy 0, policy_version 1784914 (0.0006) [2023-12-27 04:23:55,913][105692] Updated weights for policy 0, policy_version 1784924 (0.0005) [2023-12-27 04:23:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 915030016. Throughput: 0: 9904.0, 1: 9910.4. Samples: 915038168. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:23:56,063][104569] Avg episode reward: [(0, '8713.533'), (1, '9260.355')] [2023-12-27 04:23:56,109][105620] Updated weights for policy 1, policy_version 1788911 (0.0009) [2023-12-27 04:23:56,156][105620] Updated weights for policy 1, policy_version 1788921 (0.0009) [2023-12-27 04:23:56,202][105620] Updated weights for policy 1, policy_version 1788931 (0.0008) [2023-12-27 04:23:56,658][105692] Updated weights for policy 0, policy_version 1784934 (0.0008) [2023-12-27 04:23:56,718][105692] Updated weights for policy 0, policy_version 1784944 (0.0009) [2023-12-27 04:23:56,781][105692] Updated weights for policy 0, policy_version 1784954 (0.0009) [2023-12-27 04:23:56,965][105620] Updated weights for policy 1, policy_version 1788941 (0.0009) [2023-12-27 04:23:57,017][105620] Updated weights for policy 1, policy_version 1788951 (0.0009) [2023-12-27 04:23:57,071][105620] Updated weights for policy 1, policy_version 1788961 (0.0009) [2023-12-27 04:23:57,545][105692] Updated weights for policy 0, policy_version 1784964 (0.0009) [2023-12-27 04:23:57,591][105692] Updated weights for policy 0, policy_version 1784974 (0.0008) [2023-12-27 04:23:57,643][105692] Updated weights for policy 0, policy_version 1784984 (0.0009) [2023-12-27 04:23:57,829][105620] Updated weights for policy 1, policy_version 1788971 (0.0009) [2023-12-27 04:23:57,882][105620] Updated weights for policy 1, policy_version 1788981 (0.0009) [2023-12-27 04:23:57,938][105620] Updated weights for policy 1, policy_version 1788991 (0.0008) [2023-12-27 04:23:58,409][105692] Updated weights for policy 0, policy_version 1784994 (0.0009) [2023-12-27 04:23:58,473][105692] Updated weights for policy 0, policy_version 1785004 (0.0011) [2023-12-27 04:23:58,539][105692] Updated weights for policy 0, policy_version 1785014 (0.0010) [2023-12-27 04:23:58,609][105692] Updated weights for policy 0, policy_version 1785024 (0.0008) [2023-12-27 04:23:58,725][105620] Updated weights for policy 1, policy_version 1789001 (0.0009) [2023-12-27 04:23:58,792][105620] Updated weights for policy 1, policy_version 1789011 (0.0008) [2023-12-27 04:23:58,855][105620] Updated weights for policy 1, policy_version 1789021 (0.0008) [2023-12-27 04:23:58,918][105620] Updated weights for policy 1, policy_version 1789031 (0.0007) [2023-12-27 04:23:59,318][105692] Updated weights for policy 0, policy_version 1785034 (0.0008) [2023-12-27 04:23:59,382][105692] Updated weights for policy 0, policy_version 1785044 (0.0008) [2023-12-27 04:23:59,429][105692] Updated weights for policy 0, policy_version 1785054 (0.0005) [2023-12-27 04:23:59,698][105620] Updated weights for policy 1, policy_version 1789041 (0.0009) [2023-12-27 04:23:59,764][105620] Updated weights for policy 1, policy_version 1789051 (0.0009) [2023-12-27 04:23:59,834][105620] Updated weights for policy 1, policy_version 1789061 (0.0008) [2023-12-27 04:24:00,103][105692] Updated weights for policy 0, policy_version 1785064 (0.0009) [2023-12-27 04:24:00,164][105692] Updated weights for policy 0, policy_version 1785074 (0.0010) [2023-12-27 04:24:00,224][105692] Updated weights for policy 0, policy_version 1785084 (0.0009) [2023-12-27 04:24:00,468][105620] Updated weights for policy 1, policy_version 1789071 (0.0007) [2023-12-27 04:24:00,528][105620] Updated weights for policy 1, policy_version 1789081 (0.0010) [2023-12-27 04:24:00,572][105620] Updated weights for policy 1, policy_version 1789091 (0.0010) [2023-12-27 04:24:00,991][105692] Updated weights for policy 0, policy_version 1785094 (0.0007) [2023-12-27 04:24:01,047][105692] Updated weights for policy 0, policy_version 1785104 (0.0006) [2023-12-27 04:24:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 915120128. Throughput: 0: 9827.1, 1: 9841.1. Samples: 915093440. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:24:01,063][104569] Avg episode reward: [(0, '8534.518'), (1, '9260.475')] [2023-12-27 04:24:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001789096_458072064.pth... [2023-12-27 04:24:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001787944_457777152.pth [2023-12-27 04:24:01,108][105692] Updated weights for policy 0, policy_version 1785114 (0.0006) [2023-12-27 04:24:01,152][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001785120_457056256.pth... [2023-12-27 04:24:01,155][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001783968_456761344.pth [2023-12-27 04:24:01,238][105620] Updated weights for policy 1, policy_version 1789101 (0.0009) [2023-12-27 04:24:01,305][105620] Updated weights for policy 1, policy_version 1789111 (0.0011) [2023-12-27 04:24:01,378][105620] Updated weights for policy 1, policy_version 1789121 (0.0010) [2023-12-27 04:24:01,768][105692] Updated weights for policy 0, policy_version 1785124 (0.0009) [2023-12-27 04:24:01,827][105692] Updated weights for policy 0, policy_version 1785134 (0.0008) [2023-12-27 04:24:01,894][105692] Updated weights for policy 0, policy_version 1785144 (0.0009) [2023-12-27 04:24:02,055][105620] Updated weights for policy 1, policy_version 1789131 (0.0009) [2023-12-27 04:24:02,113][105620] Updated weights for policy 1, policy_version 1789141 (0.0005) [2023-12-27 04:24:02,158][105620] Updated weights for policy 1, policy_version 1789151 (0.0005) [2023-12-27 04:24:02,732][105620] Updated weights for policy 1, policy_version 1789161 (0.0007) [2023-12-27 04:24:02,750][105692] Updated weights for policy 0, policy_version 1785154 (0.0009) [2023-12-27 04:24:02,787][105620] Updated weights for policy 1, policy_version 1789171 (0.0006) [2023-12-27 04:24:02,802][105692] Updated weights for policy 0, policy_version 1785164 (0.0008) [2023-12-27 04:24:02,842][105620] Updated weights for policy 1, policy_version 1789181 (0.0010) [2023-12-27 04:24:02,859][105692] Updated weights for policy 0, policy_version 1785174 (0.0009) [2023-12-27 04:24:02,898][105620] Updated weights for policy 1, policy_version 1789191 (0.0011) [2023-12-27 04:24:02,905][105692] Updated weights for policy 0, policy_version 1785184 (0.0007) [2023-12-27 04:24:03,569][105620] Updated weights for policy 1, policy_version 1789201 (0.0010) [2023-12-27 04:24:03,624][105620] Updated weights for policy 1, policy_version 1789211 (0.0010) [2023-12-27 04:24:03,678][105620] Updated weights for policy 1, policy_version 1789221 (0.0006) [2023-12-27 04:24:03,695][105692] Updated weights for policy 0, policy_version 1785194 (0.0009) [2023-12-27 04:24:03,756][105692] Updated weights for policy 0, policy_version 1785204 (0.0005) [2023-12-27 04:24:03,817][105692] Updated weights for policy 0, policy_version 1785214 (0.0007) [2023-12-27 04:24:04,367][105620] Updated weights for policy 1, policy_version 1789231 (0.0009) [2023-12-27 04:24:04,423][105620] Updated weights for policy 1, policy_version 1789241 (0.0009) [2023-12-27 04:24:04,442][105692] Updated weights for policy 0, policy_version 1785224 (0.0007) [2023-12-27 04:24:04,484][105620] Updated weights for policy 1, policy_version 1789251 (0.0011) [2023-12-27 04:24:04,499][105692] Updated weights for policy 0, policy_version 1785234 (0.0006) [2023-12-27 04:24:04,546][105692] Updated weights for policy 0, policy_version 1785244 (0.0007) [2023-12-27 04:24:05,082][105620] Updated weights for policy 1, policy_version 1789261 (0.0008) [2023-12-27 04:24:05,140][105620] Updated weights for policy 1, policy_version 1789271 (0.0009) [2023-12-27 04:24:05,195][105620] Updated weights for policy 1, policy_version 1789281 (0.0010) [2023-12-27 04:24:05,262][105692] Updated weights for policy 0, policy_version 1785254 (0.0007) [2023-12-27 04:24:05,313][105692] Updated weights for policy 0, policy_version 1785264 (0.0005) [2023-12-27 04:24:05,359][105692] Updated weights for policy 0, policy_version 1785274 (0.0007) [2023-12-27 04:24:05,928][105620] Updated weights for policy 1, policy_version 1789291 (0.0010) [2023-12-27 04:24:05,988][105620] Updated weights for policy 1, policy_version 1789301 (0.0005) [2023-12-27 04:24:06,029][105692] Updated weights for policy 0, policy_version 1785284 (0.0009) [2023-12-27 04:24:06,048][105620] Updated weights for policy 1, policy_version 1789311 (0.0005) [2023-12-27 04:24:06,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 915218432. Throughput: 0: 9675.8, 1: 9939.4. Samples: 915212812. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:24:06,062][104569] Avg episode reward: [(0, '8808.559'), (1, '9167.977')] [2023-12-27 04:24:06,074][105692] Updated weights for policy 0, policy_version 1785294 (0.0011) [2023-12-27 04:24:06,130][105692] Updated weights for policy 0, policy_version 1785304 (0.0011) [2023-12-27 04:24:06,645][105620] Updated weights for policy 1, policy_version 1789321 (0.0005) [2023-12-27 04:24:06,704][105620] Updated weights for policy 1, policy_version 1789331 (0.0006) [2023-12-27 04:24:06,758][105620] Updated weights for policy 1, policy_version 1789341 (0.0007) [2023-12-27 04:24:06,818][105620] Updated weights for policy 1, policy_version 1789351 (0.0005) [2023-12-27 04:24:06,923][105692] Updated weights for policy 0, policy_version 1785314 (0.0011) [2023-12-27 04:24:06,994][105692] Updated weights for policy 0, policy_version 1785324 (0.0011) [2023-12-27 04:24:07,053][105692] Updated weights for policy 0, policy_version 1785334 (0.0011) [2023-12-27 04:24:07,113][105692] Updated weights for policy 0, policy_version 1785344 (0.0011) [2023-12-27 04:24:07,486][105620] Updated weights for policy 1, policy_version 1789361 (0.0007) [2023-12-27 04:24:07,537][105620] Updated weights for policy 1, policy_version 1789371 (0.0006) [2023-12-27 04:24:07,595][105620] Updated weights for policy 1, policy_version 1789381 (0.0007) [2023-12-27 04:24:07,852][105692] Updated weights for policy 0, policy_version 1785354 (0.0008) [2023-12-27 04:24:07,916][105692] Updated weights for policy 0, policy_version 1785364 (0.0005) [2023-12-27 04:24:07,984][105692] Updated weights for policy 0, policy_version 1785374 (0.0005) [2023-12-27 04:24:08,329][105620] Updated weights for policy 1, policy_version 1789391 (0.0006) [2023-12-27 04:24:08,389][105620] Updated weights for policy 1, policy_version 1789401 (0.0009) [2023-12-27 04:24:08,452][105620] Updated weights for policy 1, policy_version 1789411 (0.0010) [2023-12-27 04:24:08,660][105692] Updated weights for policy 0, policy_version 1785384 (0.0007) [2023-12-27 04:24:08,717][105692] Updated weights for policy 0, policy_version 1785394 (0.0009) [2023-12-27 04:24:08,776][105692] Updated weights for policy 0, policy_version 1785404 (0.0010) [2023-12-27 04:24:09,058][105620] Updated weights for policy 1, policy_version 1789421 (0.0008) [2023-12-27 04:24:09,116][105620] Updated weights for policy 1, policy_version 1789431 (0.0005) [2023-12-27 04:24:09,171][105620] Updated weights for policy 1, policy_version 1789441 (0.0005) [2023-12-27 04:24:09,658][105692] Updated weights for policy 0, policy_version 1785414 (0.0009) [2023-12-27 04:24:09,725][105692] Updated weights for policy 0, policy_version 1785424 (0.0009) [2023-12-27 04:24:09,785][105692] Updated weights for policy 0, policy_version 1785434 (0.0009) [2023-12-27 04:24:09,870][105620] Updated weights for policy 1, policy_version 1789451 (0.0006) [2023-12-27 04:24:09,925][105620] Updated weights for policy 1, policy_version 1789461 (0.0008) [2023-12-27 04:24:09,986][105620] Updated weights for policy 1, policy_version 1789471 (0.0009) [2023-12-27 04:24:10,458][105692] Updated weights for policy 0, policy_version 1785444 (0.0009) [2023-12-27 04:24:10,522][105692] Updated weights for policy 0, policy_version 1785454 (0.0009) [2023-12-27 04:24:10,589][105692] Updated weights for policy 0, policy_version 1785464 (0.0010) [2023-12-27 04:24:10,745][105620] Updated weights for policy 1, policy_version 1789481 (0.0009) [2023-12-27 04:24:10,810][105620] Updated weights for policy 1, policy_version 1789491 (0.0009) [2023-12-27 04:24:10,872][105620] Updated weights for policy 1, policy_version 1789501 (0.0009) [2023-12-27 04:24:10,935][105620] Updated weights for policy 1, policy_version 1789511 (0.0008) [2023-12-27 04:24:11,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 915324928. Throughput: 0: 9681.0, 1: 9978.8. Samples: 915330568. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:24:11,062][104569] Avg episode reward: [(0, '8714.572'), (1, '9075.192')] [2023-12-27 04:24:11,353][105692] Updated weights for policy 0, policy_version 1785474 (0.0010) [2023-12-27 04:24:11,420][105692] Updated weights for policy 0, policy_version 1785484 (0.0009) [2023-12-27 04:24:11,483][105692] Updated weights for policy 0, policy_version 1785494 (0.0009) [2023-12-27 04:24:11,546][105692] Updated weights for policy 0, policy_version 1785504 (0.0010) [2023-12-27 04:24:11,727][105620] Updated weights for policy 1, policy_version 1789521 (0.0009) [2023-12-27 04:24:11,793][105620] Updated weights for policy 1, policy_version 1789531 (0.0009) [2023-12-27 04:24:11,856][105620] Updated weights for policy 1, policy_version 1789541 (0.0009) [2023-12-27 04:24:12,339][105692] Updated weights for policy 0, policy_version 1785514 (0.0009) [2023-12-27 04:24:12,399][105692] Updated weights for policy 0, policy_version 1785524 (0.0010) [2023-12-27 04:24:12,455][105692] Updated weights for policy 0, policy_version 1785534 (0.0011) [2023-12-27 04:24:12,603][105620] Updated weights for policy 1, policy_version 1789551 (0.0009) [2023-12-27 04:24:12,668][105620] Updated weights for policy 1, policy_version 1789561 (0.0007) [2023-12-27 04:24:12,733][105620] Updated weights for policy 1, policy_version 1789571 (0.0008) [2023-12-27 04:24:13,191][105692] Updated weights for policy 0, policy_version 1785544 (0.0010) [2023-12-27 04:24:13,246][105692] Updated weights for policy 0, policy_version 1785554 (0.0010) [2023-12-27 04:24:13,297][105692] Updated weights for policy 0, policy_version 1785564 (0.0010) [2023-12-27 04:24:13,448][105620] Updated weights for policy 1, policy_version 1789581 (0.0008) [2023-12-27 04:24:13,507][105620] Updated weights for policy 1, policy_version 1789593 (0.0011) [2023-12-27 04:24:13,558][105620] Updated weights for policy 1, policy_version 1789603 (0.0009) [2023-12-27 04:24:13,944][105692] Updated weights for policy 0, policy_version 1785574 (0.0007) [2023-12-27 04:24:14,002][105692] Updated weights for policy 0, policy_version 1785584 (0.0009) [2023-12-27 04:24:14,051][105692] Updated weights for policy 0, policy_version 1785594 (0.0011) [2023-12-27 04:24:14,361][105620] Updated weights for policy 1, policy_version 1789613 (0.0007) [2023-12-27 04:24:14,416][105620] Updated weights for policy 1, policy_version 1789623 (0.0009) [2023-12-27 04:24:14,480][105620] Updated weights for policy 1, policy_version 1789633 (0.0007) [2023-12-27 04:24:14,775][105692] Updated weights for policy 0, policy_version 1785604 (0.0010) [2023-12-27 04:24:14,842][105692] Updated weights for policy 0, policy_version 1785614 (0.0010) [2023-12-27 04:24:14,897][105692] Updated weights for policy 0, policy_version 1785624 (0.0011) [2023-12-27 04:24:15,148][105620] Updated weights for policy 1, policy_version 1789643 (0.0008) [2023-12-27 04:24:15,206][105620] Updated weights for policy 1, policy_version 1789653 (0.0010) [2023-12-27 04:24:15,263][105620] Updated weights for policy 1, policy_version 1789663 (0.0008) [2023-12-27 04:24:15,638][105692] Updated weights for policy 0, policy_version 1785634 (0.0007) [2023-12-27 04:24:15,689][105692] Updated weights for policy 0, policy_version 1785644 (0.0010) [2023-12-27 04:24:15,740][105692] Updated weights for policy 0, policy_version 1785654 (0.0010) [2023-12-27 04:24:15,791][105692] Updated weights for policy 0, policy_version 1785664 (0.0010) [2023-12-27 04:24:16,029][105620] Updated weights for policy 1, policy_version 1789673 (0.0008) [2023-12-27 04:24:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 915415040. Throughput: 0: 9621.0, 1: 9885.1. Samples: 915386084. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:24:16,062][104569] Avg episode reward: [(0, '7895.669'), (1, '9167.325')] [2023-12-27 04:24:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001785664_457195520.pth... [2023-12-27 04:24:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001784544_456908800.pth [2023-12-27 04:24:16,088][105620] Updated weights for policy 1, policy_version 1789683 (0.0008) [2023-12-27 04:24:16,151][105620] Updated weights for policy 1, policy_version 1789693 (0.0008) [2023-12-27 04:24:16,217][105620] Updated weights for policy 1, policy_version 1789703 (0.0008) [2023-12-27 04:24:16,222][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001789704_458227712.pth... [2023-12-27 04:24:16,227][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001788520_457924608.pth [2023-12-27 04:24:16,534][105692] Updated weights for policy 0, policy_version 1785674 (0.0011) [2023-12-27 04:24:16,594][105692] Updated weights for policy 0, policy_version 1785684 (0.0011) [2023-12-27 04:24:16,655][105692] Updated weights for policy 0, policy_version 1785694 (0.0010) [2023-12-27 04:24:16,992][105620] Updated weights for policy 1, policy_version 1789713 (0.0006) [2023-12-27 04:24:17,058][105620] Updated weights for policy 1, policy_version 1789723 (0.0010) [2023-12-27 04:24:17,129][105620] Updated weights for policy 1, policy_version 1789733 (0.0009) [2023-12-27 04:24:17,314][105692] Updated weights for policy 0, policy_version 1785704 (0.0011) [2023-12-27 04:24:17,371][105692] Updated weights for policy 0, policy_version 1785714 (0.0010) [2023-12-27 04:24:17,439][105692] Updated weights for policy 0, policy_version 1785724 (0.0010) [2023-12-27 04:24:17,861][105620] Updated weights for policy 1, policy_version 1789743 (0.0007) [2023-12-27 04:24:17,916][105620] Updated weights for policy 1, policy_version 1789753 (0.0008) [2023-12-27 04:24:17,975][105620] Updated weights for policy 1, policy_version 1789763 (0.0008) [2023-12-27 04:24:18,095][105692] Updated weights for policy 0, policy_version 1785734 (0.0006) [2023-12-27 04:24:18,146][105692] Updated weights for policy 0, policy_version 1785744 (0.0010) [2023-12-27 04:24:18,194][105692] Updated weights for policy 0, policy_version 1785754 (0.0007) [2023-12-27 04:24:18,746][105620] Updated weights for policy 1, policy_version 1789773 (0.0009) [2023-12-27 04:24:18,806][105620] Updated weights for policy 1, policy_version 1789783 (0.0008) [2023-12-27 04:24:18,866][105620] Updated weights for policy 1, policy_version 1789793 (0.0008) [2023-12-27 04:24:18,893][105692] Updated weights for policy 0, policy_version 1785764 (0.0007) [2023-12-27 04:24:18,952][105692] Updated weights for policy 0, policy_version 1785774 (0.0010) [2023-12-27 04:24:19,011][105692] Updated weights for policy 0, policy_version 1785784 (0.0011) [2023-12-27 04:24:19,538][105620] Updated weights for policy 1, policy_version 1789803 (0.0009) [2023-12-27 04:24:19,604][105620] Updated weights for policy 1, policy_version 1789813 (0.0009) [2023-12-27 04:24:19,666][105620] Updated weights for policy 1, policy_version 1789823 (0.0007) [2023-12-27 04:24:19,799][105692] Updated weights for policy 0, policy_version 1785794 (0.0010) [2023-12-27 04:24:19,861][105692] Updated weights for policy 0, policy_version 1785804 (0.0007) [2023-12-27 04:24:19,925][105692] Updated weights for policy 0, policy_version 1785814 (0.0008) [2023-12-27 04:24:19,994][105692] Updated weights for policy 0, policy_version 1785824 (0.0009) [2023-12-27 04:24:20,446][105620] Updated weights for policy 1, policy_version 1789833 (0.0009) [2023-12-27 04:24:20,498][105620] Updated weights for policy 1, policy_version 1789843 (0.0009) [2023-12-27 04:24:20,550][105620] Updated weights for policy 1, policy_version 1789853 (0.0009) [2023-12-27 04:24:20,614][105620] Updated weights for policy 1, policy_version 1789863 (0.0010) [2023-12-27 04:24:20,729][105692] Updated weights for policy 0, policy_version 1785834 (0.0008) [2023-12-27 04:24:20,800][105692] Updated weights for policy 0, policy_version 1785844 (0.0009) [2023-12-27 04:24:20,859][105692] Updated weights for policy 0, policy_version 1785854 (0.0009) [2023-12-27 04:24:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 915513344. Throughput: 0: 9695.9, 1: 9792.7. Samples: 915501608. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:24:21,063][104569] Avg episode reward: [(0, '8257.014'), (1, '9352.360')] [2023-12-27 04:24:21,410][105620] Updated weights for policy 1, policy_version 1789873 (0.0008) [2023-12-27 04:24:21,481][105620] Updated weights for policy 1, policy_version 1789883 (0.0006) [2023-12-27 04:24:21,550][105620] Updated weights for policy 1, policy_version 1789893 (0.0006) [2023-12-27 04:24:21,692][105692] Updated weights for policy 0, policy_version 1785864 (0.0007) [2023-12-27 04:24:21,765][105692] Updated weights for policy 0, policy_version 1785874 (0.0009) [2023-12-27 04:24:21,831][105692] Updated weights for policy 0, policy_version 1785884 (0.0009) [2023-12-27 04:24:22,238][105620] Updated weights for policy 1, policy_version 1789903 (0.0008) [2023-12-27 04:24:22,298][105620] Updated weights for policy 1, policy_version 1789913 (0.0008) [2023-12-27 04:24:22,365][105620] Updated weights for policy 1, policy_version 1789923 (0.0009) [2023-12-27 04:24:22,449][105692] Updated weights for policy 0, policy_version 1785894 (0.0008) [2023-12-27 04:24:22,506][105692] Updated weights for policy 0, policy_version 1785904 (0.0006) [2023-12-27 04:24:22,558][105692] Updated weights for policy 0, policy_version 1785914 (0.0005) [2023-12-27 04:24:23,063][105620] Updated weights for policy 1, policy_version 1789933 (0.0010) [2023-12-27 04:24:23,120][105620] Updated weights for policy 1, policy_version 1789943 (0.0009) [2023-12-27 04:24:23,189][105620] Updated weights for policy 1, policy_version 1789953 (0.0006) [2023-12-27 04:24:23,199][105692] Updated weights for policy 0, policy_version 1785924 (0.0007) [2023-12-27 04:24:23,263][105692] Updated weights for policy 0, policy_version 1785934 (0.0005) [2023-12-27 04:24:23,326][105692] Updated weights for policy 0, policy_version 1785944 (0.0005) [2023-12-27 04:24:23,906][105620] Updated weights for policy 1, policy_version 1789963 (0.0008) [2023-12-27 04:24:23,953][105620] Updated weights for policy 1, policy_version 1789973 (0.0009) [2023-12-27 04:24:23,955][105692] Updated weights for policy 0, policy_version 1785954 (0.0006) [2023-12-27 04:24:24,011][105620] Updated weights for policy 1, policy_version 1789983 (0.0007) [2023-12-27 04:24:24,026][105692] Updated weights for policy 0, policy_version 1785964 (0.0007) [2023-12-27 04:24:24,080][105692] Updated weights for policy 0, policy_version 1785974 (0.0005) [2023-12-27 04:24:24,133][105692] Updated weights for policy 0, policy_version 1785984 (0.0005) [2023-12-27 04:24:24,660][105692] Updated weights for policy 0, policy_version 1785994 (0.0006) [2023-12-27 04:24:24,708][105692] Updated weights for policy 0, policy_version 1786004 (0.0005) [2023-12-27 04:24:24,761][105692] Updated weights for policy 0, policy_version 1786014 (0.0005) [2023-12-27 04:24:24,891][105620] Updated weights for policy 1, policy_version 1789993 (0.0009) [2023-12-27 04:24:24,938][105620] Updated weights for policy 1, policy_version 1790003 (0.0009) [2023-12-27 04:24:24,989][105620] Updated weights for policy 1, policy_version 1790013 (0.0010) [2023-12-27 04:24:25,041][105620] Updated weights for policy 1, policy_version 1790023 (0.0009) [2023-12-27 04:24:25,356][105692] Updated weights for policy 0, policy_version 1786024 (0.0008) [2023-12-27 04:24:25,414][105692] Updated weights for policy 0, policy_version 1786034 (0.0008) [2023-12-27 04:24:25,467][105692] Updated weights for policy 0, policy_version 1786044 (0.0008) [2023-12-27 04:24:25,869][105620] Updated weights for policy 1, policy_version 1790033 (0.0010) [2023-12-27 04:24:25,921][105620] Updated weights for policy 1, policy_version 1790043 (0.0009) [2023-12-27 04:24:25,978][105620] Updated weights for policy 1, policy_version 1790054 (0.0008) [2023-12-27 04:24:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.4, 300 sec: 19549.7). Total num frames: 915611648. Throughput: 0: 9754.2, 1: 9719.9. Samples: 915618200. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:24:26,062][104569] Avg episode reward: [(0, '8439.213'), (1, '9352.600')] [2023-12-27 04:24:26,071][105692] Updated weights for policy 0, policy_version 1786054 (0.0009) [2023-12-27 04:24:26,136][105692] Updated weights for policy 0, policy_version 1786064 (0.0009) [2023-12-27 04:24:26,184][105692] Updated weights for policy 0, policy_version 1786074 (0.0007) [2023-12-27 04:24:26,740][105620] Updated weights for policy 1, policy_version 1790064 (0.0009) [2023-12-27 04:24:26,795][105620] Updated weights for policy 1, policy_version 1790074 (0.0009) [2023-12-27 04:24:26,851][105620] Updated weights for policy 1, policy_version 1790084 (0.0005) [2023-12-27 04:24:26,966][105692] Updated weights for policy 0, policy_version 1786084 (0.0008) [2023-12-27 04:24:27,024][105692] Updated weights for policy 0, policy_version 1786094 (0.0009) [2023-12-27 04:24:27,070][105692] Updated weights for policy 0, policy_version 1786104 (0.0009) [2023-12-27 04:24:27,498][105620] Updated weights for policy 1, policy_version 1790094 (0.0008) [2023-12-27 04:24:27,555][105620] Updated weights for policy 1, policy_version 1790105 (0.0010) [2023-12-27 04:24:27,607][105620] Updated weights for policy 1, policy_version 1790115 (0.0009) [2023-12-27 04:24:27,652][105692] Updated weights for policy 0, policy_version 1786114 (0.0006) [2023-12-27 04:24:27,702][105692] Updated weights for policy 0, policy_version 1786124 (0.0007) [2023-12-27 04:24:27,765][105692] Updated weights for policy 0, policy_version 1786134 (0.0009) [2023-12-27 04:24:27,827][105692] Updated weights for policy 0, policy_version 1786144 (0.0009) [2023-12-27 04:24:28,362][105620] Updated weights for policy 1, policy_version 1790126 (0.0009) [2023-12-27 04:24:28,420][105620] Updated weights for policy 1, policy_version 1790136 (0.0008) [2023-12-27 04:24:28,487][105620] Updated weights for policy 1, policy_version 1790146 (0.0008) [2023-12-27 04:24:28,577][105692] Updated weights for policy 0, policy_version 1786154 (0.0010) [2023-12-27 04:24:28,635][105692] Updated weights for policy 0, policy_version 1786164 (0.0010) [2023-12-27 04:24:28,684][105692] Updated weights for policy 0, policy_version 1786174 (0.0009) [2023-12-27 04:24:29,154][105620] Updated weights for policy 1, policy_version 1790156 (0.0007) [2023-12-27 04:24:29,220][105620] Updated weights for policy 1, policy_version 1790166 (0.0008) [2023-12-27 04:24:29,285][105620] Updated weights for policy 1, policy_version 1790176 (0.0010) [2023-12-27 04:24:29,392][105692] Updated weights for policy 0, policy_version 1786184 (0.0007) [2023-12-27 04:24:29,458][105692] Updated weights for policy 0, policy_version 1786194 (0.0006) [2023-12-27 04:24:29,515][105692] Updated weights for policy 0, policy_version 1786204 (0.0005) [2023-12-27 04:24:30,090][105620] Updated weights for policy 1, policy_version 1790186 (0.0009) [2023-12-27 04:24:30,153][105620] Updated weights for policy 1, policy_version 1790196 (0.0011) [2023-12-27 04:24:30,167][105692] Updated weights for policy 0, policy_version 1786214 (0.0005) [2023-12-27 04:24:30,212][105620] Updated weights for policy 1, policy_version 1790206 (0.0011) [2023-12-27 04:24:30,223][105692] Updated weights for policy 0, policy_version 1786224 (0.0005) [2023-12-27 04:24:30,268][105620] Updated weights for policy 1, policy_version 1790216 (0.0010) [2023-12-27 04:24:30,282][105692] Updated weights for policy 0, policy_version 1786234 (0.0006) [2023-12-27 04:24:31,013][105620] Updated weights for policy 1, policy_version 1790226 (0.0010) [2023-12-27 04:24:31,036][105692] Updated weights for policy 0, policy_version 1786244 (0.0008) [2023-12-27 04:24:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 915701760. Throughput: 0: 9813.7, 1: 9728.5. Samples: 915678508. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:24:31,062][104569] Avg episode reward: [(0, '8347.461'), (1, '9352.789')] [2023-12-27 04:24:31,076][105620] Updated weights for policy 1, policy_version 1790236 (0.0011) [2023-12-27 04:24:31,091][105692] Updated weights for policy 0, policy_version 1786254 (0.0007) [2023-12-27 04:24:31,133][105620] Updated weights for policy 1, policy_version 1790246 (0.0010) [2023-12-27 04:24:31,146][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001790248_458366976.pth... [2023-12-27 04:24:31,151][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001789096_458072064.pth [2023-12-27 04:24:31,152][105692] Updated weights for policy 0, policy_version 1786264 (0.0007) [2023-12-27 04:24:31,192][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001786272_457351168.pth... [2023-12-27 04:24:31,195][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001785120_457056256.pth [2023-12-27 04:24:31,894][105620] Updated weights for policy 1, policy_version 1790256 (0.0011) [2023-12-27 04:24:31,933][105692] Updated weights for policy 0, policy_version 1786274 (0.0008) [2023-12-27 04:24:31,958][105620] Updated weights for policy 1, policy_version 1790266 (0.0011) [2023-12-27 04:24:31,991][105692] Updated weights for policy 0, policy_version 1786284 (0.0007) [2023-12-27 04:24:32,020][105620] Updated weights for policy 1, policy_version 1790276 (0.0010) [2023-12-27 04:24:32,054][105692] Updated weights for policy 0, policy_version 1786294 (0.0006) [2023-12-27 04:24:32,106][105692] Updated weights for policy 0, policy_version 1786304 (0.0008) [2023-12-27 04:24:32,731][105620] Updated weights for policy 1, policy_version 1790286 (0.0009) [2023-12-27 04:24:32,783][105692] Updated weights for policy 0, policy_version 1786314 (0.0010) [2023-12-27 04:24:32,794][105620] Updated weights for policy 1, policy_version 1790296 (0.0007) [2023-12-27 04:24:32,836][105692] Updated weights for policy 0, policy_version 1786324 (0.0010) [2023-12-27 04:24:32,849][105620] Updated weights for policy 1, policy_version 1790306 (0.0007) [2023-12-27 04:24:32,884][105692] Updated weights for policy 0, policy_version 1786334 (0.0010) [2023-12-27 04:24:33,436][105620] Updated weights for policy 1, policy_version 1790316 (0.0006) [2023-12-27 04:24:33,486][105620] Updated weights for policy 1, policy_version 1790326 (0.0005) [2023-12-27 04:24:33,532][105692] Updated weights for policy 0, policy_version 1786344 (0.0006) [2023-12-27 04:24:33,538][105620] Updated weights for policy 1, policy_version 1790336 (0.0009) [2023-12-27 04:24:33,579][105692] Updated weights for policy 0, policy_version 1786354 (0.0005) [2023-12-27 04:24:33,627][105692] Updated weights for policy 0, policy_version 1786364 (0.0007) [2023-12-27 04:24:34,083][105620] Updated weights for policy 1, policy_version 1790346 (0.0007) [2023-12-27 04:24:34,153][105620] Updated weights for policy 1, policy_version 1790356 (0.0006) [2023-12-27 04:24:34,217][105620] Updated weights for policy 1, policy_version 1790366 (0.0009) [2023-12-27 04:24:34,276][105620] Updated weights for policy 1, policy_version 1790376 (0.0008) [2023-12-27 04:24:34,336][105692] Updated weights for policy 0, policy_version 1786374 (0.0010) [2023-12-27 04:24:34,406][105692] Updated weights for policy 0, policy_version 1786384 (0.0009) [2023-12-27 04:24:34,473][105692] Updated weights for policy 0, policy_version 1786394 (0.0009) [2023-12-27 04:24:34,908][105620] Updated weights for policy 1, policy_version 1790386 (0.0009) [2023-12-27 04:24:34,971][105620] Updated weights for policy 1, policy_version 1790396 (0.0010) [2023-12-27 04:24:35,033][105620] Updated weights for policy 1, policy_version 1790406 (0.0011) [2023-12-27 04:24:35,267][105692] Updated weights for policy 0, policy_version 1786404 (0.0009) [2023-12-27 04:24:35,323][105692] Updated weights for policy 0, policy_version 1786414 (0.0008) [2023-12-27 04:24:35,371][105692] Updated weights for policy 0, policy_version 1786424 (0.0008) [2023-12-27 04:24:35,784][105620] Updated weights for policy 1, policy_version 1790416 (0.0010) [2023-12-27 04:24:35,842][105620] Updated weights for policy 1, policy_version 1790426 (0.0010) [2023-12-27 04:24:35,897][105620] Updated weights for policy 1, policy_version 1790436 (0.0010) [2023-12-27 04:24:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 915808256. Throughput: 0: 9763.1, 1: 9757.5. Samples: 915797764. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:24:36,062][104569] Avg episode reward: [(0, '8536.720'), (1, '9172.129')] [2023-12-27 04:24:36,155][105692] Updated weights for policy 0, policy_version 1786434 (0.0008) [2023-12-27 04:24:36,206][105692] Updated weights for policy 0, policy_version 1786444 (0.0009) [2023-12-27 04:24:36,268][105692] Updated weights for policy 0, policy_version 1786454 (0.0009) [2023-12-27 04:24:36,322][105692] Updated weights for policy 0, policy_version 1786464 (0.0008) [2023-12-27 04:24:36,688][105620] Updated weights for policy 1, policy_version 1790446 (0.0010) [2023-12-27 04:24:36,744][105620] Updated weights for policy 1, policy_version 1790456 (0.0011) [2023-12-27 04:24:36,806][105620] Updated weights for policy 1, policy_version 1790466 (0.0010) [2023-12-27 04:24:37,117][105692] Updated weights for policy 0, policy_version 1786474 (0.0008) [2023-12-27 04:24:37,177][105692] Updated weights for policy 0, policy_version 1786484 (0.0008) [2023-12-27 04:24:37,239][105692] Updated weights for policy 0, policy_version 1786494 (0.0008) [2023-12-27 04:24:37,609][105620] Updated weights for policy 1, policy_version 1790476 (0.0010) [2023-12-27 04:24:37,668][105620] Updated weights for policy 1, policy_version 1790486 (0.0010) [2023-12-27 04:24:37,728][105620] Updated weights for policy 1, policy_version 1790496 (0.0011) [2023-12-27 04:24:38,047][105692] Updated weights for policy 0, policy_version 1786504 (0.0010) [2023-12-27 04:24:38,110][105692] Updated weights for policy 0, policy_version 1786514 (0.0007) [2023-12-27 04:24:38,165][105692] Updated weights for policy 0, policy_version 1786524 (0.0010) [2023-12-27 04:24:38,480][105620] Updated weights for policy 1, policy_version 1790506 (0.0011) [2023-12-27 04:24:38,535][105620] Updated weights for policy 1, policy_version 1790516 (0.0010) [2023-12-27 04:24:38,593][105620] Updated weights for policy 1, policy_version 1790526 (0.0008) [2023-12-27 04:24:38,656][105620] Updated weights for policy 1, policy_version 1790536 (0.0006) [2023-12-27 04:24:38,895][105692] Updated weights for policy 0, policy_version 1786534 (0.0010) [2023-12-27 04:24:38,952][105692] Updated weights for policy 0, policy_version 1786544 (0.0011) [2023-12-27 04:24:39,019][105692] Updated weights for policy 0, policy_version 1786554 (0.0011) [2023-12-27 04:24:39,332][105620] Updated weights for policy 1, policy_version 1790546 (0.0006) [2023-12-27 04:24:39,403][105620] Updated weights for policy 1, policy_version 1790556 (0.0011) [2023-12-27 04:24:39,459][105620] Updated weights for policy 1, policy_version 1790566 (0.0011) [2023-12-27 04:24:39,764][105692] Updated weights for policy 0, policy_version 1786564 (0.0010) [2023-12-27 04:24:39,827][105692] Updated weights for policy 0, policy_version 1786574 (0.0009) [2023-12-27 04:24:39,881][105692] Updated weights for policy 0, policy_version 1786584 (0.0008) [2023-12-27 04:24:40,262][105620] Updated weights for policy 1, policy_version 1790576 (0.0007) [2023-12-27 04:24:40,325][105620] Updated weights for policy 1, policy_version 1790586 (0.0005) [2023-12-27 04:24:40,382][105620] Updated weights for policy 1, policy_version 1790596 (0.0005) [2023-12-27 04:24:40,567][105692] Updated weights for policy 0, policy_version 1786594 (0.0008) [2023-12-27 04:24:40,626][105692] Updated weights for policy 0, policy_version 1786604 (0.0009) [2023-12-27 04:24:40,695][105692] Updated weights for policy 0, policy_version 1786614 (0.0011) [2023-12-27 04:24:40,755][105692] Updated weights for policy 0, policy_version 1786624 (0.0011) [2023-12-27 04:24:40,906][105620] Updated weights for policy 1, policy_version 1790606 (0.0005) [2023-12-27 04:24:40,960][105620] Updated weights for policy 1, policy_version 1790616 (0.0007) [2023-12-27 04:24:41,018][105620] Updated weights for policy 1, policy_version 1790626 (0.0009) [2023-12-27 04:24:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 915906560. Throughput: 0: 9687.6, 1: 9725.4. Samples: 915911748. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:24:41,062][104569] Avg episode reward: [(0, '8721.512'), (1, '9172.044')] [2023-12-27 04:24:41,412][105692] Updated weights for policy 0, policy_version 1786634 (0.0011) [2023-12-27 04:24:41,472][105692] Updated weights for policy 0, policy_version 1786644 (0.0009) [2023-12-27 04:24:41,528][105692] Updated weights for policy 0, policy_version 1786654 (0.0010) [2023-12-27 04:24:41,808][105620] Updated weights for policy 1, policy_version 1790636 (0.0007) [2023-12-27 04:24:41,870][105620] Updated weights for policy 1, policy_version 1790646 (0.0007) [2023-12-27 04:24:41,936][105620] Updated weights for policy 1, policy_version 1790656 (0.0009) [2023-12-27 04:24:42,305][105692] Updated weights for policy 0, policy_version 1786664 (0.0009) [2023-12-27 04:24:42,364][105692] Updated weights for policy 0, policy_version 1786674 (0.0008) [2023-12-27 04:24:42,433][105692] Updated weights for policy 0, policy_version 1786684 (0.0010) [2023-12-27 04:24:42,632][105620] Updated weights for policy 1, policy_version 1790666 (0.0008) [2023-12-27 04:24:42,695][105620] Updated weights for policy 1, policy_version 1790676 (0.0007) [2023-12-27 04:24:42,748][105620] Updated weights for policy 1, policy_version 1790687 (0.0010) [2023-12-27 04:24:43,096][105692] Updated weights for policy 0, policy_version 1786694 (0.0009) [2023-12-27 04:24:43,152][105692] Updated weights for policy 0, policy_version 1786704 (0.0009) [2023-12-27 04:24:43,203][105692] Updated weights for policy 0, policy_version 1786714 (0.0007) [2023-12-27 04:24:43,463][105620] Updated weights for policy 1, policy_version 1790697 (0.0009) [2023-12-27 04:24:43,519][105620] Updated weights for policy 1, policy_version 1790707 (0.0005) [2023-12-27 04:24:43,589][105620] Updated weights for policy 1, policy_version 1790717 (0.0005) [2023-12-27 04:24:43,650][105620] Updated weights for policy 1, policy_version 1790727 (0.0010) [2023-12-27 04:24:44,008][105692] Updated weights for policy 0, policy_version 1786724 (0.0008) [2023-12-27 04:24:44,068][105692] Updated weights for policy 0, policy_version 1786734 (0.0008) [2023-12-27 04:24:44,126][105692] Updated weights for policy 0, policy_version 1786744 (0.0008) [2023-12-27 04:24:44,310][105620] Updated weights for policy 1, policy_version 1790737 (0.0010) [2023-12-27 04:24:44,361][105620] Updated weights for policy 1, policy_version 1790747 (0.0010) [2023-12-27 04:24:44,409][105620] Updated weights for policy 1, policy_version 1790757 (0.0010) [2023-12-27 04:24:44,835][105692] Updated weights for policy 0, policy_version 1786754 (0.0006) [2023-12-27 04:24:44,894][105692] Updated weights for policy 0, policy_version 1786764 (0.0008) [2023-12-27 04:24:44,956][105692] Updated weights for policy 0, policy_version 1786774 (0.0008) [2023-12-27 04:24:45,022][105692] Updated weights for policy 0, policy_version 1786784 (0.0009) [2023-12-27 04:24:45,175][105620] Updated weights for policy 1, policy_version 1790767 (0.0010) [2023-12-27 04:24:45,234][105620] Updated weights for policy 1, policy_version 1790777 (0.0011) [2023-12-27 04:24:45,290][105620] Updated weights for policy 1, policy_version 1790787 (0.0010) [2023-12-27 04:24:45,803][105692] Updated weights for policy 0, policy_version 1786794 (0.0009) [2023-12-27 04:24:45,855][105692] Updated weights for policy 0, policy_version 1786804 (0.0010) [2023-12-27 04:24:45,912][105692] Updated weights for policy 0, policy_version 1786814 (0.0009) [2023-12-27 04:24:45,958][105620] Updated weights for policy 1, policy_version 1790797 (0.0010) [2023-12-27 04:24:46,024][105620] Updated weights for policy 1, policy_version 1790807 (0.0010) [2023-12-27 04:24:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 915996672. Throughput: 0: 9703.9, 1: 9752.3. Samples: 915968964. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:24:46,062][104569] Avg episode reward: [(0, '8810.848'), (1, '9260.369')] [2023-12-27 04:24:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001786816_457490432.pth... [2023-12-27 04:24:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001785664_457195520.pth [2023-12-27 04:24:46,085][105620] Updated weights for policy 1, policy_version 1790817 (0.0010) [2023-12-27 04:24:46,127][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001790824_458514432.pth... [2023-12-27 04:24:46,132][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001789704_458227712.pth [2023-12-27 04:24:46,706][105692] Updated weights for policy 0, policy_version 1786824 (0.0009) [2023-12-27 04:24:46,762][105692] Updated weights for policy 0, policy_version 1786834 (0.0010) [2023-12-27 04:24:46,795][105620] Updated weights for policy 1, policy_version 1790827 (0.0010) [2023-12-27 04:24:46,818][105692] Updated weights for policy 0, policy_version 1786844 (0.0007) [2023-12-27 04:24:46,853][105620] Updated weights for policy 1, policy_version 1790837 (0.0006) [2023-12-27 04:24:46,908][105620] Updated weights for policy 1, policy_version 1790847 (0.0008) [2023-12-27 04:24:47,567][105620] Updated weights for policy 1, policy_version 1790857 (0.0008) [2023-12-27 04:24:47,592][105692] Updated weights for policy 0, policy_version 1786854 (0.0009) [2023-12-27 04:24:47,626][105620] Updated weights for policy 1, policy_version 1790867 (0.0009) [2023-12-27 04:24:47,641][105692] Updated weights for policy 0, policy_version 1786864 (0.0005) [2023-12-27 04:24:47,684][105620] Updated weights for policy 1, policy_version 1790877 (0.0008) [2023-12-27 04:24:47,687][105692] Updated weights for policy 0, policy_version 1786874 (0.0005) [2023-12-27 04:24:47,742][105620] Updated weights for policy 1, policy_version 1790887 (0.0009) [2023-12-27 04:24:48,411][105692] Updated weights for policy 0, policy_version 1786884 (0.0009) [2023-12-27 04:24:48,471][105692] Updated weights for policy 0, policy_version 1786894 (0.0011) [2023-12-27 04:24:48,519][105692] Updated weights for policy 0, policy_version 1786904 (0.0011) [2023-12-27 04:24:48,527][105620] Updated weights for policy 1, policy_version 1790897 (0.0008) [2023-12-27 04:24:48,586][105620] Updated weights for policy 1, policy_version 1790907 (0.0008) [2023-12-27 04:24:48,648][105620] Updated weights for policy 1, policy_version 1790917 (0.0009) [2023-12-27 04:24:49,129][105692] Updated weights for policy 0, policy_version 1786914 (0.0010) [2023-12-27 04:24:49,192][105692] Updated weights for policy 0, policy_version 1786924 (0.0010) [2023-12-27 04:24:49,262][105692] Updated weights for policy 0, policy_version 1786934 (0.0009) [2023-12-27 04:24:49,328][105692] Updated weights for policy 0, policy_version 1786944 (0.0008) [2023-12-27 04:24:49,463][105620] Updated weights for policy 1, policy_version 1790927 (0.0008) [2023-12-27 04:24:49,515][105620] Updated weights for policy 1, policy_version 1790937 (0.0008) [2023-12-27 04:24:49,564][105620] Updated weights for policy 1, policy_version 1790947 (0.0008) [2023-12-27 04:24:49,999][105692] Updated weights for policy 0, policy_version 1786954 (0.0009) [2023-12-27 04:24:50,067][105692] Updated weights for policy 0, policy_version 1786964 (0.0011) [2023-12-27 04:24:50,129][105692] Updated weights for policy 0, policy_version 1786974 (0.0011) [2023-12-27 04:24:50,314][105620] Updated weights for policy 1, policy_version 1790957 (0.0007) [2023-12-27 04:24:50,377][105620] Updated weights for policy 1, policy_version 1790967 (0.0008) [2023-12-27 04:24:50,438][105620] Updated weights for policy 1, policy_version 1790977 (0.0006) [2023-12-27 04:24:50,869][105692] Updated weights for policy 0, policy_version 1786984 (0.0011) [2023-12-27 04:24:50,922][105692] Updated weights for policy 0, policy_version 1786994 (0.0010) [2023-12-27 04:24:50,985][105692] Updated weights for policy 0, policy_version 1787004 (0.0011) [2023-12-27 04:24:51,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 916094976. Throughput: 0: 9712.3, 1: 9620.4. Samples: 916082788. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:24:51,063][104569] Avg episode reward: [(0, '8176.875'), (1, '9260.486')] [2023-12-27 04:24:51,145][105620] Updated weights for policy 1, policy_version 1790987 (0.0006) [2023-12-27 04:24:51,214][105620] Updated weights for policy 1, policy_version 1790997 (0.0007) [2023-12-27 04:24:51,280][105620] Updated weights for policy 1, policy_version 1791007 (0.0007) [2023-12-27 04:24:51,769][105692] Updated weights for policy 0, policy_version 1787014 (0.0008) [2023-12-27 04:24:51,837][105692] Updated weights for policy 0, policy_version 1787024 (0.0006) [2023-12-27 04:24:51,907][105692] Updated weights for policy 0, policy_version 1787034 (0.0007) [2023-12-27 04:24:52,033][105620] Updated weights for policy 1, policy_version 1791017 (0.0007) [2023-12-27 04:24:52,097][105620] Updated weights for policy 1, policy_version 1791027 (0.0007) [2023-12-27 04:24:52,168][105620] Updated weights for policy 1, policy_version 1791037 (0.0006) [2023-12-27 04:24:52,237][105620] Updated weights for policy 1, policy_version 1791047 (0.0009) [2023-12-27 04:24:52,523][105692] Updated weights for policy 0, policy_version 1787044 (0.0007) [2023-12-27 04:24:52,578][105692] Updated weights for policy 0, policy_version 1787054 (0.0009) [2023-12-27 04:24:52,636][105692] Updated weights for policy 0, policy_version 1787064 (0.0010) [2023-12-27 04:24:52,880][105620] Updated weights for policy 1, policy_version 1791057 (0.0007) [2023-12-27 04:24:52,929][105620] Updated weights for policy 1, policy_version 1791067 (0.0009) [2023-12-27 04:24:52,993][105620] Updated weights for policy 1, policy_version 1791077 (0.0009) [2023-12-27 04:24:53,478][105692] Updated weights for policy 0, policy_version 1787074 (0.0009) [2023-12-27 04:24:53,539][105692] Updated weights for policy 0, policy_version 1787084 (0.0009) [2023-12-27 04:24:53,605][105692] Updated weights for policy 0, policy_version 1787094 (0.0008) [2023-12-27 04:24:53,643][105620] Updated weights for policy 1, policy_version 1791087 (0.0008) [2023-12-27 04:24:53,654][105692] Updated weights for policy 0, policy_version 1787104 (0.0007) [2023-12-27 04:24:53,692][105620] Updated weights for policy 1, policy_version 1791097 (0.0008) [2023-12-27 04:24:53,739][105620] Updated weights for policy 1, policy_version 1791107 (0.0008) [2023-12-27 04:24:54,338][105692] Updated weights for policy 0, policy_version 1787114 (0.0005) [2023-12-27 04:24:54,400][105692] Updated weights for policy 0, policy_version 1787124 (0.0007) [2023-12-27 04:24:54,455][105692] Updated weights for policy 0, policy_version 1787134 (0.0005) [2023-12-27 04:24:54,575][105620] Updated weights for policy 1, policy_version 1791117 (0.0010) [2023-12-27 04:24:54,627][105620] Updated weights for policy 1, policy_version 1791127 (0.0010) [2023-12-27 04:24:54,680][105620] Updated weights for policy 1, policy_version 1791137 (0.0010) [2023-12-27 04:24:55,001][105692] Updated weights for policy 0, policy_version 1787144 (0.0007) [2023-12-27 04:24:55,058][105692] Updated weights for policy 0, policy_version 1787154 (0.0009) [2023-12-27 04:24:55,105][105692] Updated weights for policy 0, policy_version 1787164 (0.0007) [2023-12-27 04:24:55,542][105620] Updated weights for policy 1, policy_version 1791148 (0.0009) [2023-12-27 04:24:55,597][105620] Updated weights for policy 1, policy_version 1791158 (0.0008) [2023-12-27 04:24:55,656][105620] Updated weights for policy 1, policy_version 1791168 (0.0008) [2023-12-27 04:24:55,761][105692] Updated weights for policy 0, policy_version 1787174 (0.0007) [2023-12-27 04:24:55,813][105692] Updated weights for policy 0, policy_version 1787184 (0.0008) [2023-12-27 04:24:55,864][105692] Updated weights for policy 0, policy_version 1787194 (0.0008) [2023-12-27 04:24:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 916193280. Throughput: 0: 9765.6, 1: 9542.7. Samples: 916199440. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:24:56,062][104569] Avg episode reward: [(0, '8538.080'), (1, '9167.746')] [2023-12-27 04:24:56,385][105620] Updated weights for policy 1, policy_version 1791178 (0.0008) [2023-12-27 04:24:56,439][105620] Updated weights for policy 1, policy_version 1791188 (0.0010) [2023-12-27 04:24:56,490][105620] Updated weights for policy 1, policy_version 1791198 (0.0009) [2023-12-27 04:24:56,548][105620] Updated weights for policy 1, policy_version 1791208 (0.0009) [2023-12-27 04:24:56,559][105692] Updated weights for policy 0, policy_version 1787204 (0.0008) [2023-12-27 04:24:56,622][105692] Updated weights for policy 0, policy_version 1787214 (0.0009) [2023-12-27 04:24:56,685][105692] Updated weights for policy 0, policy_version 1787224 (0.0010) [2023-12-27 04:24:57,235][105620] Updated weights for policy 1, policy_version 1791218 (0.0009) [2023-12-27 04:24:57,289][105620] Updated weights for policy 1, policy_version 1791228 (0.0008) [2023-12-27 04:24:57,343][105620] Updated weights for policy 1, policy_version 1791238 (0.0008) [2023-12-27 04:24:57,389][105692] Updated weights for policy 0, policy_version 1787234 (0.0009) [2023-12-27 04:24:57,451][105692] Updated weights for policy 0, policy_version 1787244 (0.0005) [2023-12-27 04:24:57,500][105692] Updated weights for policy 0, policy_version 1787254 (0.0005) [2023-12-27 04:24:57,551][105692] Updated weights for policy 0, policy_version 1787264 (0.0005) [2023-12-27 04:24:58,139][105692] Updated weights for policy 0, policy_version 1787274 (0.0009) [2023-12-27 04:24:58,187][105620] Updated weights for policy 1, policy_version 1791248 (0.0008) [2023-12-27 04:24:58,200][105692] Updated weights for policy 0, policy_version 1787284 (0.0008) [2023-12-27 04:24:58,245][105620] Updated weights for policy 1, policy_version 1791258 (0.0006) [2023-12-27 04:24:58,260][105692] Updated weights for policy 0, policy_version 1787294 (0.0009) [2023-12-27 04:24:58,313][105620] Updated weights for policy 1, policy_version 1791268 (0.0009) [2023-12-27 04:24:59,157][105692] Updated weights for policy 0, policy_version 1787304 (0.0009) [2023-12-27 04:24:59,222][105692] Updated weights for policy 0, policy_version 1787314 (0.0009) [2023-12-27 04:24:59,231][105620] Updated weights for policy 1, policy_version 1791278 (0.0009) [2023-12-27 04:24:59,292][105692] Updated weights for policy 0, policy_version 1787324 (0.0007) [2023-12-27 04:24:59,300][105620] Updated weights for policy 1, policy_version 1791288 (0.0008) [2023-12-27 04:24:59,374][105620] Updated weights for policy 1, policy_version 1791298 (0.0012) [2023-12-27 04:25:00,079][105692] Updated weights for policy 0, policy_version 1787334 (0.0008) [2023-12-27 04:25:00,138][105692] Updated weights for policy 0, policy_version 1787344 (0.0008) [2023-12-27 04:25:00,165][105620] Updated weights for policy 1, policy_version 1791308 (0.0010) [2023-12-27 04:25:00,204][105692] Updated weights for policy 0, policy_version 1787354 (0.0009) [2023-12-27 04:25:00,216][105620] Updated weights for policy 1, policy_version 1791318 (0.0010) [2023-12-27 04:25:00,264][105620] Updated weights for policy 1, policy_version 1791328 (0.0010) [2023-12-27 04:25:00,933][105692] Updated weights for policy 0, policy_version 1787364 (0.0008) [2023-12-27 04:25:00,987][105692] Updated weights for policy 0, policy_version 1787374 (0.0008) [2023-12-27 04:25:01,013][105620] Updated weights for policy 1, policy_version 1791338 (0.0010) [2023-12-27 04:25:01,061][105692] Updated weights for policy 0, policy_version 1787384 (0.0006) [2023-12-27 04:25:01,062][104569] Fps is (10 sec: 18022.6, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 916275200. Throughput: 0: 9808.4, 1: 9533.8. Samples: 916256484. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:25:01,063][104569] Avg episode reward: [(0, '8626.104'), (1, '8983.838')] [2023-12-27 04:25:01,078][105620] Updated weights for policy 1, policy_version 1791348 (0.0011) [2023-12-27 04:25:01,110][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001787392_457637888.pth... [2023-12-27 04:25:01,113][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001786272_457351168.pth [2023-12-27 04:25:01,134][105620] Updated weights for policy 1, policy_version 1791358 (0.0011) [2023-12-27 04:25:01,199][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001791368_458653696.pth... [2023-12-27 04:25:01,200][105620] Updated weights for policy 1, policy_version 1791368 (0.0010) [2023-12-27 04:25:01,204][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001790248_458366976.pth [2023-12-27 04:25:01,735][105692] Updated weights for policy 0, policy_version 1787394 (0.0006) [2023-12-27 04:25:01,803][105692] Updated weights for policy 0, policy_version 1787404 (0.0008) [2023-12-27 04:25:01,850][105692] Updated weights for policy 0, policy_version 1787414 (0.0008) [2023-12-27 04:25:01,904][105692] Updated weights for policy 0, policy_version 1787424 (0.0008) [2023-12-27 04:25:01,994][105620] Updated weights for policy 1, policy_version 1791378 (0.0010) [2023-12-27 04:25:02,056][105620] Updated weights for policy 1, policy_version 1791388 (0.0009) [2023-12-27 04:25:02,116][105620] Updated weights for policy 1, policy_version 1791398 (0.0010) [2023-12-27 04:25:02,503][105692] Updated weights for policy 0, policy_version 1787434 (0.0009) [2023-12-27 04:25:02,557][105692] Updated weights for policy 0, policy_version 1787444 (0.0009) [2023-12-27 04:25:02,619][105692] Updated weights for policy 0, policy_version 1787454 (0.0009) [2023-12-27 04:25:02,933][105620] Updated weights for policy 1, policy_version 1791408 (0.0009) [2023-12-27 04:25:02,980][105620] Updated weights for policy 1, policy_version 1791418 (0.0009) [2023-12-27 04:25:03,026][105620] Updated weights for policy 1, policy_version 1791428 (0.0009) [2023-12-27 04:25:03,295][105692] Updated weights for policy 0, policy_version 1787464 (0.0008) [2023-12-27 04:25:03,354][105692] Updated weights for policy 0, policy_version 1787474 (0.0008) [2023-12-27 04:25:03,415][105692] Updated weights for policy 0, policy_version 1787484 (0.0005) [2023-12-27 04:25:03,935][105692] Updated weights for policy 0, policy_version 1787494 (0.0007) [2023-12-27 04:25:03,946][105620] Updated weights for policy 1, policy_version 1791438 (0.0008) [2023-12-27 04:25:03,993][105692] Updated weights for policy 0, policy_version 1787504 (0.0008) [2023-12-27 04:25:04,002][105620] Updated weights for policy 1, policy_version 1791448 (0.0007) [2023-12-27 04:25:04,053][105692] Updated weights for policy 0, policy_version 1787514 (0.0008) [2023-12-27 04:25:04,054][105620] Updated weights for policy 1, policy_version 1791458 (0.0007) [2023-12-27 04:25:04,749][105692] Updated weights for policy 0, policy_version 1787524 (0.0009) [2023-12-27 04:25:04,797][105692] Updated weights for policy 0, policy_version 1787534 (0.0010) [2023-12-27 04:25:04,855][105620] Updated weights for policy 1, policy_version 1791468 (0.0007) [2023-12-27 04:25:04,860][105692] Updated weights for policy 0, policy_version 1787544 (0.0010) [2023-12-27 04:25:04,904][105620] Updated weights for policy 1, policy_version 1791478 (0.0009) [2023-12-27 04:25:04,959][105620] Updated weights for policy 1, policy_version 1791488 (0.0008) [2023-12-27 04:25:05,577][105692] Updated weights for policy 0, policy_version 1787554 (0.0010) [2023-12-27 04:25:05,633][105692] Updated weights for policy 0, policy_version 1787564 (0.0010) [2023-12-27 04:25:05,695][105692] Updated weights for policy 0, policy_version 1787574 (0.0010) [2023-12-27 04:25:05,706][105620] Updated weights for policy 1, policy_version 1791498 (0.0009) [2023-12-27 04:25:05,751][105692] Updated weights for policy 0, policy_version 1787584 (0.0010) [2023-12-27 04:25:05,765][105620] Updated weights for policy 1, policy_version 1791508 (0.0006) [2023-12-27 04:25:05,817][105620] Updated weights for policy 1, policy_version 1791518 (0.0007) [2023-12-27 04:25:05,876][105620] Updated weights for policy 1, policy_version 1791528 (0.0008) [2023-12-27 04:25:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 916381696. Throughput: 0: 9820.8, 1: 9441.5. Samples: 916368412. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:25:06,062][104569] Avg episode reward: [(0, '8806.186'), (1, '9076.082')] [2023-12-27 04:25:06,435][105692] Updated weights for policy 0, policy_version 1787594 (0.0008) [2023-12-27 04:25:06,501][105692] Updated weights for policy 0, policy_version 1787604 (0.0008) [2023-12-27 04:25:06,564][105692] Updated weights for policy 0, policy_version 1787614 (0.0009) [2023-12-27 04:25:06,690][105620] Updated weights for policy 1, policy_version 1791538 (0.0010) [2023-12-27 04:25:06,761][105620] Updated weights for policy 1, policy_version 1791548 (0.0010) [2023-12-27 04:25:06,829][105620] Updated weights for policy 1, policy_version 1791558 (0.0008) [2023-12-27 04:25:07,230][105692] Updated weights for policy 0, policy_version 1787624 (0.0009) [2023-12-27 04:25:07,285][105692] Updated weights for policy 0, policy_version 1787634 (0.0010) [2023-12-27 04:25:07,338][105692] Updated weights for policy 0, policy_version 1787644 (0.0010) [2023-12-27 04:25:07,598][105620] Updated weights for policy 1, policy_version 1791568 (0.0008) [2023-12-27 04:25:07,659][105620] Updated weights for policy 1, policy_version 1791578 (0.0008) [2023-12-27 04:25:07,719][105620] Updated weights for policy 1, policy_version 1791588 (0.0008) [2023-12-27 04:25:08,109][105692] Updated weights for policy 0, policy_version 1787654 (0.0011) [2023-12-27 04:25:08,161][105692] Updated weights for policy 0, policy_version 1787664 (0.0010) [2023-12-27 04:25:08,219][105692] Updated weights for policy 0, policy_version 1787674 (0.0010) [2023-12-27 04:25:08,499][105620] Updated weights for policy 1, policy_version 1791598 (0.0008) [2023-12-27 04:25:08,560][105620] Updated weights for policy 1, policy_version 1791608 (0.0009) [2023-12-27 04:25:08,610][105620] Updated weights for policy 1, policy_version 1791618 (0.0008) [2023-12-27 04:25:08,951][105692] Updated weights for policy 0, policy_version 1787684 (0.0009) [2023-12-27 04:25:09,010][105692] Updated weights for policy 0, policy_version 1787694 (0.0005) [2023-12-27 04:25:09,072][105692] Updated weights for policy 0, policy_version 1787704 (0.0007) [2023-12-27 04:25:09,407][105620] Updated weights for policy 1, policy_version 1791628 (0.0008) [2023-12-27 04:25:09,464][105620] Updated weights for policy 1, policy_version 1791638 (0.0009) [2023-12-27 04:25:09,513][105620] Updated weights for policy 1, policy_version 1791648 (0.0009) [2023-12-27 04:25:09,763][105692] Updated weights for policy 0, policy_version 1787714 (0.0006) [2023-12-27 04:25:09,822][105692] Updated weights for policy 0, policy_version 1787724 (0.0009) [2023-12-27 04:25:09,891][105692] Updated weights for policy 0, policy_version 1787734 (0.0009) [2023-12-27 04:25:09,945][105692] Updated weights for policy 0, policy_version 1787744 (0.0008) [2023-12-27 04:25:10,339][105620] Updated weights for policy 1, policy_version 1791658 (0.0008) [2023-12-27 04:25:10,398][105620] Updated weights for policy 1, policy_version 1791668 (0.0009) [2023-12-27 04:25:10,449][105620] Updated weights for policy 1, policy_version 1791678 (0.0009) [2023-12-27 04:25:10,507][105620] Updated weights for policy 1, policy_version 1791688 (0.0009) [2023-12-27 04:25:10,627][105692] Updated weights for policy 0, policy_version 1787754 (0.0009) [2023-12-27 04:25:10,678][105692] Updated weights for policy 0, policy_version 1787764 (0.0009) [2023-12-27 04:25:10,737][105692] Updated weights for policy 0, policy_version 1787774 (0.0009) [2023-12-27 04:25:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19114.7, 300 sec: 19438.7). Total num frames: 916471808. Throughput: 0: 9770.3, 1: 9414.8. Samples: 916481532. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:25:11,062][104569] Avg episode reward: [(0, '8534.376'), (1, '9167.353')] [2023-12-27 04:25:11,162][105620] Updated weights for policy 1, policy_version 1791698 (0.0009) [2023-12-27 04:25:11,217][105620] Updated weights for policy 1, policy_version 1791708 (0.0006) [2023-12-27 04:25:11,284][105620] Updated weights for policy 1, policy_version 1791718 (0.0009) [2023-12-27 04:25:11,591][105692] Updated weights for policy 0, policy_version 1787784 (0.0009) [2023-12-27 04:25:11,657][105692] Updated weights for policy 0, policy_version 1787794 (0.0008) [2023-12-27 04:25:11,724][105692] Updated weights for policy 0, policy_version 1787804 (0.0007) [2023-12-27 04:25:12,049][105620] Updated weights for policy 1, policy_version 1791728 (0.0010) [2023-12-27 04:25:12,110][105620] Updated weights for policy 1, policy_version 1791738 (0.0008) [2023-12-27 04:25:12,174][105620] Updated weights for policy 1, policy_version 1791748 (0.0009) [2023-12-27 04:25:12,445][105692] Updated weights for policy 0, policy_version 1787814 (0.0010) [2023-12-27 04:25:12,493][105692] Updated weights for policy 0, policy_version 1787824 (0.0009) [2023-12-27 04:25:12,540][105692] Updated weights for policy 0, policy_version 1787834 (0.0008) [2023-12-27 04:25:12,976][105620] Updated weights for policy 1, policy_version 1791758 (0.0009) [2023-12-27 04:25:13,029][105620] Updated weights for policy 1, policy_version 1791768 (0.0009) [2023-12-27 04:25:13,089][105620] Updated weights for policy 1, policy_version 1791778 (0.0009) [2023-12-27 04:25:13,215][105692] Updated weights for policy 0, policy_version 1787844 (0.0008) [2023-12-27 04:25:13,277][105692] Updated weights for policy 0, policy_version 1787854 (0.0006) [2023-12-27 04:25:13,335][105692] Updated weights for policy 0, policy_version 1787864 (0.0008) [2023-12-27 04:25:13,867][105620] Updated weights for policy 1, policy_version 1791788 (0.0010) [2023-12-27 04:25:13,915][105620] Updated weights for policy 1, policy_version 1791798 (0.0008) [2023-12-27 04:25:13,966][105620] Updated weights for policy 1, policy_version 1791808 (0.0009) [2023-12-27 04:25:14,070][105692] Updated weights for policy 0, policy_version 1787874 (0.0009) [2023-12-27 04:25:14,128][105692] Updated weights for policy 0, policy_version 1787884 (0.0009) [2023-12-27 04:25:14,194][105692] Updated weights for policy 0, policy_version 1787894 (0.0009) [2023-12-27 04:25:14,248][105692] Updated weights for policy 0, policy_version 1787904 (0.0009) [2023-12-27 04:25:14,778][105620] Updated weights for policy 1, policy_version 1791818 (0.0009) [2023-12-27 04:25:14,842][105620] Updated weights for policy 1, policy_version 1791828 (0.0007) [2023-12-27 04:25:14,899][105692] Updated weights for policy 0, policy_version 1787914 (0.0007) [2023-12-27 04:25:14,909][105620] Updated weights for policy 1, policy_version 1791838 (0.0006) [2023-12-27 04:25:14,954][105692] Updated weights for policy 0, policy_version 1787924 (0.0008) [2023-12-27 04:25:14,971][105620] Updated weights for policy 1, policy_version 1791848 (0.0010) [2023-12-27 04:25:15,011][105692] Updated weights for policy 0, policy_version 1787934 (0.0009) [2023-12-27 04:25:15,602][105692] Updated weights for policy 0, policy_version 1787944 (0.0006) [2023-12-27 04:25:15,661][105692] Updated weights for policy 0, policy_version 1787954 (0.0007) [2023-12-27 04:25:15,684][105620] Updated weights for policy 1, policy_version 1791858 (0.0006) [2023-12-27 04:25:15,719][105692] Updated weights for policy 0, policy_version 1787964 (0.0008) [2023-12-27 04:25:15,750][105620] Updated weights for policy 1, policy_version 1791868 (0.0006) [2023-12-27 04:25:15,803][105620] Updated weights for policy 1, policy_version 1791878 (0.0006) [2023-12-27 04:25:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 916570112. Throughput: 0: 9711.1, 1: 9362.4. Samples: 916536816. Policy #0 lag: (min: 18.0, avg: 44.8, max: 50.0) [2023-12-27 04:25:16,062][104569] Avg episode reward: [(0, '8354.025'), (1, '9167.468')] [2023-12-27 04:25:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001787968_457785344.pth... [2023-12-27 04:25:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001791880_458784768.pth... [2023-12-27 04:25:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001790824_458514432.pth [2023-12-27 04:25:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001786816_457490432.pth [2023-12-27 04:25:16,384][105692] Updated weights for policy 0, policy_version 1787974 (0.0007) [2023-12-27 04:25:16,446][105692] Updated weights for policy 0, policy_version 1787984 (0.0010) [2023-12-27 04:25:16,509][105692] Updated weights for policy 0, policy_version 1787994 (0.0011) [2023-12-27 04:25:16,523][105620] Updated weights for policy 1, policy_version 1791888 (0.0006) [2023-12-27 04:25:16,569][105620] Updated weights for policy 1, policy_version 1791898 (0.0005) [2023-12-27 04:25:16,617][105620] Updated weights for policy 1, policy_version 1791908 (0.0005) [2023-12-27 04:25:17,169][105692] Updated weights for policy 0, policy_version 1788004 (0.0010) [2023-12-27 04:25:17,189][105620] Updated weights for policy 1, policy_version 1791918 (0.0006) [2023-12-27 04:25:17,225][105692] Updated weights for policy 0, policy_version 1788014 (0.0009) [2023-12-27 04:25:17,235][105620] Updated weights for policy 1, policy_version 1791928 (0.0006) [2023-12-27 04:25:17,279][105692] Updated weights for policy 0, policy_version 1788024 (0.0008) [2023-12-27 04:25:17,282][105620] Updated weights for policy 1, policy_version 1791938 (0.0007) [2023-12-27 04:25:17,913][105692] Updated weights for policy 0, policy_version 1788034 (0.0008) [2023-12-27 04:25:17,927][105620] Updated weights for policy 1, policy_version 1791948 (0.0005) [2023-12-27 04:25:17,973][105692] Updated weights for policy 0, policy_version 1788044 (0.0010) [2023-12-27 04:25:17,993][105620] Updated weights for policy 1, policy_version 1791958 (0.0005) [2023-12-27 04:25:18,027][105692] Updated weights for policy 0, policy_version 1788054 (0.0008) [2023-12-27 04:25:18,053][105620] Updated weights for policy 1, policy_version 1791968 (0.0005) [2023-12-27 04:25:18,091][105692] Updated weights for policy 0, policy_version 1788064 (0.0006) [2023-12-27 04:25:18,637][105620] Updated weights for policy 1, policy_version 1791978 (0.0006) [2023-12-27 04:25:18,694][105620] Updated weights for policy 1, policy_version 1791988 (0.0009) [2023-12-27 04:25:18,762][105620] Updated weights for policy 1, policy_version 1791998 (0.0008) [2023-12-27 04:25:18,794][105692] Updated weights for policy 0, policy_version 1788074 (0.0007) [2023-12-27 04:25:18,825][105620] Updated weights for policy 1, policy_version 1792008 (0.0008) [2023-12-27 04:25:18,847][105692] Updated weights for policy 0, policy_version 1788084 (0.0007) [2023-12-27 04:25:18,901][105692] Updated weights for policy 0, policy_version 1788094 (0.0009) [2023-12-27 04:25:19,624][105692] Updated weights for policy 0, policy_version 1788104 (0.0006) [2023-12-27 04:25:19,659][105620] Updated weights for policy 1, policy_version 1792018 (0.0008) [2023-12-27 04:25:19,682][105692] Updated weights for policy 0, policy_version 1788114 (0.0007) [2023-12-27 04:25:19,720][105620] Updated weights for policy 1, policy_version 1792028 (0.0009) [2023-12-27 04:25:19,735][105692] Updated weights for policy 0, policy_version 1788124 (0.0006) [2023-12-27 04:25:19,780][105620] Updated weights for policy 1, policy_version 1792038 (0.0007) [2023-12-27 04:25:20,400][105620] Updated weights for policy 1, policy_version 1792048 (0.0009) [2023-12-27 04:25:20,449][105620] Updated weights for policy 1, policy_version 1792058 (0.0009) [2023-12-27 04:25:20,497][105620] Updated weights for policy 1, policy_version 1792068 (0.0009) [2023-12-27 04:25:20,541][105692] Updated weights for policy 0, policy_version 1788134 (0.0007) [2023-12-27 04:25:20,604][105692] Updated weights for policy 0, policy_version 1788144 (0.0008) [2023-12-27 04:25:20,652][105692] Updated weights for policy 0, policy_version 1788154 (0.0006) [2023-12-27 04:25:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 916668416. Throughput: 0: 9782.3, 1: 9336.6. Samples: 916658116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:25:21,062][104569] Avg episode reward: [(0, '8259.717'), (1, '9075.065')] [2023-12-27 04:25:21,369][105620] Updated weights for policy 1, policy_version 1792078 (0.0008) [2023-12-27 04:25:21,392][105692] Updated weights for policy 0, policy_version 1788164 (0.0007) [2023-12-27 04:25:21,444][105620] Updated weights for policy 1, policy_version 1792088 (0.0008) [2023-12-27 04:25:21,468][105692] Updated weights for policy 0, policy_version 1788174 (0.0007) [2023-12-27 04:25:21,506][105620] Updated weights for policy 1, policy_version 1792098 (0.0006) [2023-12-27 04:25:21,529][105692] Updated weights for policy 0, policy_version 1788184 (0.0008) [2023-12-27 04:25:22,212][105692] Updated weights for policy 0, policy_version 1788194 (0.0008) [2023-12-27 04:25:22,269][105692] Updated weights for policy 0, policy_version 1788204 (0.0009) [2023-12-27 04:25:22,297][105620] Updated weights for policy 1, policy_version 1792108 (0.0007) [2023-12-27 04:25:22,324][105692] Updated weights for policy 0, policy_version 1788214 (0.0007) [2023-12-27 04:25:22,366][105620] Updated weights for policy 1, policy_version 1792118 (0.0009) [2023-12-27 04:25:22,392][105692] Updated weights for policy 0, policy_version 1788224 (0.0008) [2023-12-27 04:25:22,426][105620] Updated weights for policy 1, policy_version 1792128 (0.0007) [2023-12-27 04:25:23,188][105692] Updated weights for policy 0, policy_version 1788234 (0.0008) [2023-12-27 04:25:23,194][105620] Updated weights for policy 1, policy_version 1792138 (0.0008) [2023-12-27 04:25:23,249][105692] Updated weights for policy 0, policy_version 1788244 (0.0009) [2023-12-27 04:25:23,253][105620] Updated weights for policy 1, policy_version 1792148 (0.0009) [2023-12-27 04:25:23,310][105692] Updated weights for policy 0, policy_version 1788254 (0.0011) [2023-12-27 04:25:23,316][105620] Updated weights for policy 1, policy_version 1792158 (0.0006) [2023-12-27 04:25:23,380][105620] Updated weights for policy 1, policy_version 1792168 (0.0008) [2023-12-27 04:25:23,951][105692] Updated weights for policy 0, policy_version 1788264 (0.0011) [2023-12-27 04:25:24,013][105692] Updated weights for policy 0, policy_version 1788274 (0.0011) [2023-12-27 04:25:24,065][105692] Updated weights for policy 0, policy_version 1788284 (0.0010) [2023-12-27 04:25:24,156][105620] Updated weights for policy 1, policy_version 1792178 (0.0008) [2023-12-27 04:25:24,200][105620] Updated weights for policy 1, policy_version 1792188 (0.0008) [2023-12-27 04:25:24,246][105620] Updated weights for policy 1, policy_version 1792198 (0.0008) [2023-12-27 04:25:24,712][105692] Updated weights for policy 0, policy_version 1788294 (0.0010) [2023-12-27 04:25:24,757][105692] Updated weights for policy 0, policy_version 1788304 (0.0010) [2023-12-27 04:25:24,815][105692] Updated weights for policy 0, policy_version 1788314 (0.0010) [2023-12-27 04:25:25,046][105620] Updated weights for policy 1, policy_version 1792208 (0.0008) [2023-12-27 04:25:25,101][105620] Updated weights for policy 1, policy_version 1792218 (0.0008) [2023-12-27 04:25:25,144][105620] Updated weights for policy 1, policy_version 1792228 (0.0008) [2023-12-27 04:25:25,584][105692] Updated weights for policy 0, policy_version 1788324 (0.0010) [2023-12-27 04:25:25,629][105692] Updated weights for policy 0, policy_version 1788334 (0.0010) [2023-12-27 04:25:25,673][105692] Updated weights for policy 0, policy_version 1788344 (0.0010) [2023-12-27 04:25:25,930][105620] Updated weights for policy 1, policy_version 1792238 (0.0009) [2023-12-27 04:25:25,984][105620] Updated weights for policy 1, policy_version 1792249 (0.0010) [2023-12-27 04:25:26,041][105620] Updated weights for policy 1, policy_version 1792259 (0.0010) [2023-12-27 04:25:26,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19114.6, 300 sec: 19410.9). Total num frames: 916758528. Throughput: 0: 9823.4, 1: 9265.7. Samples: 916770764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:25:26,063][104569] Avg episode reward: [(0, '8258.129'), (1, '9167.688')] [2023-12-27 04:25:26,265][105692] Updated weights for policy 0, policy_version 1788354 (0.0009) [2023-12-27 04:25:26,320][105692] Updated weights for policy 0, policy_version 1788364 (0.0005) [2023-12-27 04:25:26,385][105692] Updated weights for policy 0, policy_version 1788374 (0.0005) [2023-12-27 04:25:26,441][105692] Updated weights for policy 0, policy_version 1788384 (0.0005) [2023-12-27 04:25:26,814][105620] Updated weights for policy 1, policy_version 1792270 (0.0008) [2023-12-27 04:25:26,865][105620] Updated weights for policy 1, policy_version 1792280 (0.0008) [2023-12-27 04:25:26,920][105620] Updated weights for policy 1, policy_version 1792290 (0.0008) [2023-12-27 04:25:27,004][105692] Updated weights for policy 0, policy_version 1788394 (0.0010) [2023-12-27 04:25:27,052][105692] Updated weights for policy 0, policy_version 1788404 (0.0010) [2023-12-27 04:25:27,103][105692] Updated weights for policy 0, policy_version 1788414 (0.0010) [2023-12-27 04:25:27,716][105692] Updated weights for policy 0, policy_version 1788424 (0.0006) [2023-12-27 04:25:27,746][105620] Updated weights for policy 1, policy_version 1792300 (0.0008) [2023-12-27 04:25:27,772][105692] Updated weights for policy 0, policy_version 1788434 (0.0005) [2023-12-27 04:25:27,805][105620] Updated weights for policy 1, policy_version 1792310 (0.0009) [2023-12-27 04:25:27,833][105692] Updated weights for policy 0, policy_version 1788444 (0.0005) [2023-12-27 04:25:27,860][105620] Updated weights for policy 1, policy_version 1792320 (0.0009) [2023-12-27 04:25:28,393][105692] Updated weights for policy 0, policy_version 1788454 (0.0009) [2023-12-27 04:25:28,453][105692] Updated weights for policy 0, policy_version 1788464 (0.0009) [2023-12-27 04:25:28,508][105692] Updated weights for policy 0, policy_version 1788474 (0.0010) [2023-12-27 04:25:28,629][105620] Updated weights for policy 1, policy_version 1792330 (0.0007) [2023-12-27 04:25:28,679][105620] Updated weights for policy 1, policy_version 1792340 (0.0007) [2023-12-27 04:25:28,726][105620] Updated weights for policy 1, policy_version 1792350 (0.0008) [2023-12-27 04:25:28,776][105620] Updated weights for policy 1, policy_version 1792360 (0.0006) [2023-12-27 04:25:29,261][105692] Updated weights for policy 0, policy_version 1788484 (0.0009) [2023-12-27 04:25:29,322][105692] Updated weights for policy 0, policy_version 1788494 (0.0007) [2023-12-27 04:25:29,388][105692] Updated weights for policy 0, policy_version 1788504 (0.0007) [2023-12-27 04:25:29,418][105620] Updated weights for policy 1, policy_version 1792370 (0.0008) [2023-12-27 04:25:29,476][105620] Updated weights for policy 1, policy_version 1792380 (0.0008) [2023-12-27 04:25:29,543][105620] Updated weights for policy 1, policy_version 1792390 (0.0009) [2023-12-27 04:25:30,129][105692] Updated weights for policy 0, policy_version 1788514 (0.0008) [2023-12-27 04:25:30,190][105692] Updated weights for policy 0, policy_version 1788524 (0.0008) [2023-12-27 04:25:30,249][105692] Updated weights for policy 0, policy_version 1788534 (0.0010) [2023-12-27 04:25:30,296][105692] Updated weights for policy 0, policy_version 1788544 (0.0007) [2023-12-27 04:25:30,318][105620] Updated weights for policy 1, policy_version 1792400 (0.0008) [2023-12-27 04:25:30,372][105620] Updated weights for policy 1, policy_version 1792410 (0.0007) [2023-12-27 04:25:30,421][105620] Updated weights for policy 1, policy_version 1792420 (0.0005) [2023-12-27 04:25:30,982][105620] Updated weights for policy 1, policy_version 1792430 (0.0006) [2023-12-27 04:25:31,048][105620] Updated weights for policy 1, policy_version 1792440 (0.0007) [2023-12-27 04:25:31,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 916856832. Throughput: 0: 9957.7, 1: 9235.5. Samples: 916832664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:25:31,063][104569] Avg episode reward: [(0, '8439.566'), (1, '9260.202')] [2023-12-27 04:25:31,095][105692] Updated weights for policy 0, policy_version 1788554 (0.0008) [2023-12-27 04:25:31,102][105620] Updated weights for policy 1, policy_version 1792450 (0.0010) [2023-12-27 04:25:31,137][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001792456_458932224.pth... [2023-12-27 04:25:31,142][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001791368_458653696.pth [2023-12-27 04:25:31,156][105692] Updated weights for policy 0, policy_version 1788564 (0.0007) [2023-12-27 04:25:31,218][105692] Updated weights for policy 0, policy_version 1788574 (0.0009) [2023-12-27 04:25:31,230][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001788576_457940992.pth... [2023-12-27 04:25:31,234][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001787392_457637888.pth [2023-12-27 04:25:31,765][105620] Updated weights for policy 1, policy_version 1792460 (0.0010) [2023-12-27 04:25:31,827][105620] Updated weights for policy 1, policy_version 1792470 (0.0010) [2023-12-27 04:25:31,889][105620] Updated weights for policy 1, policy_version 1792480 (0.0010) [2023-12-27 04:25:31,996][105692] Updated weights for policy 0, policy_version 1788584 (0.0008) [2023-12-27 04:25:32,055][105692] Updated weights for policy 0, policy_version 1788594 (0.0008) [2023-12-27 04:25:32,113][105692] Updated weights for policy 0, policy_version 1788604 (0.0007) [2023-12-27 04:25:32,553][105620] Updated weights for policy 1, policy_version 1792490 (0.0011) [2023-12-27 04:25:32,602][105620] Updated weights for policy 1, policy_version 1792500 (0.0011) [2023-12-27 04:25:32,651][105620] Updated weights for policy 1, policy_version 1792510 (0.0011) [2023-12-27 04:25:32,703][105620] Updated weights for policy 1, policy_version 1792520 (0.0010) [2023-12-27 04:25:32,783][105692] Updated weights for policy 0, policy_version 1788614 (0.0009) [2023-12-27 04:25:32,848][105692] Updated weights for policy 0, policy_version 1788624 (0.0005) [2023-12-27 04:25:32,917][105692] Updated weights for policy 0, policy_version 1788634 (0.0005) [2023-12-27 04:25:33,354][105620] Updated weights for policy 1, policy_version 1792530 (0.0007) [2023-12-27 04:25:33,402][105620] Updated weights for policy 1, policy_version 1792540 (0.0008) [2023-12-27 04:25:33,449][105620] Updated weights for policy 1, policy_version 1792550 (0.0007) [2023-12-27 04:25:33,482][105692] Updated weights for policy 0, policy_version 1788644 (0.0007) [2023-12-27 04:25:33,543][105692] Updated weights for policy 0, policy_version 1788654 (0.0010) [2023-12-27 04:25:33,587][105692] Updated weights for policy 0, policy_version 1788664 (0.0010) [2023-12-27 04:25:34,128][105620] Updated weights for policy 1, policy_version 1792560 (0.0010) [2023-12-27 04:25:34,191][105620] Updated weights for policy 1, policy_version 1792570 (0.0009) [2023-12-27 04:25:34,255][105620] Updated weights for policy 1, policy_version 1792580 (0.0009) [2023-12-27 04:25:34,369][105692] Updated weights for policy 0, policy_version 1788674 (0.0010) [2023-12-27 04:25:34,432][105692] Updated weights for policy 0, policy_version 1788684 (0.0008) [2023-12-27 04:25:34,495][105692] Updated weights for policy 0, policy_version 1788694 (0.0008) [2023-12-27 04:25:34,563][105692] Updated weights for policy 0, policy_version 1788704 (0.0008) [2023-12-27 04:25:35,017][105620] Updated weights for policy 1, policy_version 1792590 (0.0006) [2023-12-27 04:25:35,084][105620] Updated weights for policy 1, policy_version 1792600 (0.0006) [2023-12-27 04:25:35,141][105620] Updated weights for policy 1, policy_version 1792610 (0.0005) [2023-12-27 04:25:35,268][105692] Updated weights for policy 0, policy_version 1788714 (0.0005) [2023-12-27 04:25:35,321][105692] Updated weights for policy 0, policy_version 1788724 (0.0008) [2023-12-27 04:25:35,374][105692] Updated weights for policy 0, policy_version 1788734 (0.0006) [2023-12-27 04:25:35,758][105620] Updated weights for policy 1, policy_version 1792620 (0.0007) [2023-12-27 04:25:35,809][105620] Updated weights for policy 1, policy_version 1792630 (0.0005) [2023-12-27 04:25:35,866][105620] Updated weights for policy 1, policy_version 1792640 (0.0005) [2023-12-27 04:25:35,906][105692] Updated weights for policy 0, policy_version 1788744 (0.0009) [2023-12-27 04:25:35,964][105692] Updated weights for policy 0, policy_version 1788754 (0.0010) [2023-12-27 04:25:36,019][105692] Updated weights for policy 0, policy_version 1788764 (0.0010) [2023-12-27 04:25:36,062][104569] Fps is (10 sec: 21299.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 916971520. Throughput: 0: 9954.3, 1: 9345.5. Samples: 916951276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:25:36,063][104569] Avg episode reward: [(0, '8625.590'), (1, '9262.992')] [2023-12-27 04:25:36,550][105620] Updated weights for policy 1, policy_version 1792650 (0.0006) [2023-12-27 04:25:36,616][105620] Updated weights for policy 1, policy_version 1792660 (0.0010) [2023-12-27 04:25:36,672][105620] Updated weights for policy 1, policy_version 1792670 (0.0010) [2023-12-27 04:25:36,703][105692] Updated weights for policy 0, policy_version 1788774 (0.0007) [2023-12-27 04:25:36,727][105620] Updated weights for policy 1, policy_version 1792680 (0.0010) [2023-12-27 04:25:36,764][105692] Updated weights for policy 0, policy_version 1788784 (0.0005) [2023-12-27 04:25:36,819][105692] Updated weights for policy 0, policy_version 1788794 (0.0007) [2023-12-27 04:25:37,399][105692] Updated weights for policy 0, policy_version 1788804 (0.0007) [2023-12-27 04:25:37,463][105692] Updated weights for policy 0, policy_version 1788814 (0.0011) [2023-12-27 04:25:37,464][105620] Updated weights for policy 1, policy_version 1792690 (0.0006) [2023-12-27 04:25:37,519][105692] Updated weights for policy 0, policy_version 1788824 (0.0011) [2023-12-27 04:25:37,525][105620] Updated weights for policy 1, policy_version 1792700 (0.0007) [2023-12-27 04:25:37,575][105620] Updated weights for policy 1, policy_version 1792710 (0.0007) [2023-12-27 04:25:38,212][105620] Updated weights for policy 1, policy_version 1792720 (0.0008) [2023-12-27 04:25:38,268][105620] Updated weights for policy 1, policy_version 1792730 (0.0007) [2023-12-27 04:25:38,268][105692] Updated weights for policy 0, policy_version 1788834 (0.0009) [2023-12-27 04:25:38,317][105620] Updated weights for policy 1, policy_version 1792740 (0.0009) [2023-12-27 04:25:38,331][105692] Updated weights for policy 0, policy_version 1788844 (0.0007) [2023-12-27 04:25:38,392][105692] Updated weights for policy 0, policy_version 1788854 (0.0009) [2023-12-27 04:25:38,457][105692] Updated weights for policy 0, policy_version 1788864 (0.0007) [2023-12-27 04:25:38,975][105620] Updated weights for policy 1, policy_version 1792750 (0.0006) [2023-12-27 04:25:39,001][105692] Updated weights for policy 0, policy_version 1788874 (0.0009) [2023-12-27 04:25:39,032][105620] Updated weights for policy 1, policy_version 1792760 (0.0006) [2023-12-27 04:25:39,058][105692] Updated weights for policy 0, policy_version 1788884 (0.0011) [2023-12-27 04:25:39,085][105620] Updated weights for policy 1, policy_version 1792770 (0.0006) [2023-12-27 04:25:39,115][105692] Updated weights for policy 0, policy_version 1788894 (0.0011) [2023-12-27 04:25:39,787][105620] Updated weights for policy 1, policy_version 1792780 (0.0006) [2023-12-27 04:25:39,850][105620] Updated weights for policy 1, policy_version 1792790 (0.0008) [2023-12-27 04:25:39,876][105692] Updated weights for policy 0, policy_version 1788904 (0.0008) [2023-12-27 04:25:39,918][105620] Updated weights for policy 1, policy_version 1792800 (0.0008) [2023-12-27 04:25:39,945][105692] Updated weights for policy 0, policy_version 1788914 (0.0009) [2023-12-27 04:25:40,009][105692] Updated weights for policy 0, policy_version 1788924 (0.0010) [2023-12-27 04:25:40,646][105620] Updated weights for policy 1, policy_version 1792810 (0.0008) [2023-12-27 04:25:40,651][105692] Updated weights for policy 0, policy_version 1788934 (0.0007) [2023-12-27 04:25:40,711][105620] Updated weights for policy 1, policy_version 1792820 (0.0009) [2023-12-27 04:25:40,714][105692] Updated weights for policy 0, policy_version 1788944 (0.0006) [2023-12-27 04:25:40,772][105620] Updated weights for policy 1, policy_version 1792830 (0.0009) [2023-12-27 04:25:40,776][105692] Updated weights for policy 0, policy_version 1788954 (0.0005) [2023-12-27 04:25:40,835][105620] Updated weights for policy 1, policy_version 1792840 (0.0010) [2023-12-27 04:25:41,062][104569] Fps is (10 sec: 21299.4, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 917069824. Throughput: 0: 10038.8, 1: 9406.0. Samples: 917074460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:25:41,063][104569] Avg episode reward: [(0, '8621.835'), (1, '9262.869')] [2023-12-27 04:25:41,350][105692] Updated weights for policy 0, policy_version 1788964 (0.0007) [2023-12-27 04:25:41,424][105692] Updated weights for policy 0, policy_version 1788974 (0.0010) [2023-12-27 04:25:41,500][105692] Updated weights for policy 0, policy_version 1788984 (0.0010) [2023-12-27 04:25:41,585][105620] Updated weights for policy 1, policy_version 1792850 (0.0008) [2023-12-27 04:25:41,646][105620] Updated weights for policy 1, policy_version 1792860 (0.0009) [2023-12-27 04:25:41,701][105620] Updated weights for policy 1, policy_version 1792870 (0.0009) [2023-12-27 04:25:42,323][105692] Updated weights for policy 0, policy_version 1788994 (0.0010) [2023-12-27 04:25:42,392][105692] Updated weights for policy 0, policy_version 1789004 (0.0009) [2023-12-27 04:25:42,400][105620] Updated weights for policy 1, policy_version 1792880 (0.0008) [2023-12-27 04:25:42,449][105692] Updated weights for policy 0, policy_version 1789014 (0.0009) [2023-12-27 04:25:42,460][105620] Updated weights for policy 1, policy_version 1792890 (0.0006) [2023-12-27 04:25:42,502][105692] Updated weights for policy 0, policy_version 1789024 (0.0008) [2023-12-27 04:25:42,515][105620] Updated weights for policy 1, policy_version 1792900 (0.0005) [2023-12-27 04:25:43,160][105620] Updated weights for policy 1, policy_version 1792910 (0.0007) [2023-12-27 04:25:43,218][105620] Updated weights for policy 1, policy_version 1792920 (0.0006) [2023-12-27 04:25:43,284][105620] Updated weights for policy 1, policy_version 1792930 (0.0006) [2023-12-27 04:25:43,339][105692] Updated weights for policy 0, policy_version 1789034 (0.0008) [2023-12-27 04:25:43,401][105692] Updated weights for policy 0, policy_version 1789044 (0.0010) [2023-12-27 04:25:43,452][105692] Updated weights for policy 0, policy_version 1789054 (0.0009) [2023-12-27 04:25:43,919][105620] Updated weights for policy 1, policy_version 1792940 (0.0007) [2023-12-27 04:25:43,964][105620] Updated weights for policy 1, policy_version 1792950 (0.0008) [2023-12-27 04:25:44,025][105620] Updated weights for policy 1, policy_version 1792960 (0.0009) [2023-12-27 04:25:44,180][105692] Updated weights for policy 0, policy_version 1789064 (0.0008) [2023-12-27 04:25:44,239][105692] Updated weights for policy 0, policy_version 1789074 (0.0009) [2023-12-27 04:25:44,301][105692] Updated weights for policy 0, policy_version 1789084 (0.0008) [2023-12-27 04:25:44,795][105620] Updated weights for policy 1, policy_version 1792970 (0.0009) [2023-12-27 04:25:44,858][105620] Updated weights for policy 1, policy_version 1792980 (0.0006) [2023-12-27 04:25:44,925][105620] Updated weights for policy 1, policy_version 1792990 (0.0009) [2023-12-27 04:25:44,989][105620] Updated weights for policy 1, policy_version 1793000 (0.0009) [2023-12-27 04:25:45,076][105692] Updated weights for policy 0, policy_version 1789094 (0.0009) [2023-12-27 04:25:45,143][105692] Updated weights for policy 0, policy_version 1789104 (0.0010) [2023-12-27 04:25:45,211][105692] Updated weights for policy 0, policy_version 1789114 (0.0010) [2023-12-27 04:25:45,597][105620] Updated weights for policy 1, policy_version 1793010 (0.0005) [2023-12-27 04:25:45,656][105620] Updated weights for policy 1, policy_version 1793020 (0.0005) [2023-12-27 04:25:45,717][105620] Updated weights for policy 1, policy_version 1793030 (0.0008) [2023-12-27 04:25:45,963][105692] Updated weights for policy 0, policy_version 1789124 (0.0008) [2023-12-27 04:25:46,026][105692] Updated weights for policy 0, policy_version 1789134 (0.0009) [2023-12-27 04:25:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 917159936. Throughput: 0: 9970.9, 1: 9501.5. Samples: 917132744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:25:46,063][104569] Avg episode reward: [(0, '8804.752'), (1, '9259.695')] [2023-12-27 04:25:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001793032_459079680.pth... [2023-12-27 04:25:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001791880_458784768.pth [2023-12-27 04:25:46,087][105692] Updated weights for policy 0, policy_version 1789144 (0.0006) [2023-12-27 04:25:46,126][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001789152_458088448.pth... [2023-12-27 04:25:46,129][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001787968_457785344.pth [2023-12-27 04:25:46,463][105620] Updated weights for policy 1, policy_version 1793040 (0.0006) [2023-12-27 04:25:46,516][105620] Updated weights for policy 1, policy_version 1793050 (0.0005) [2023-12-27 04:25:46,572][105620] Updated weights for policy 1, policy_version 1793060 (0.0005) [2023-12-27 04:25:46,765][105692] Updated weights for policy 0, policy_version 1789154 (0.0007) [2023-12-27 04:25:46,814][105692] Updated weights for policy 0, policy_version 1789164 (0.0011) [2023-12-27 04:25:46,863][105692] Updated weights for policy 0, policy_version 1789174 (0.0011) [2023-12-27 04:25:46,911][105692] Updated weights for policy 0, policy_version 1789184 (0.0010) [2023-12-27 04:25:47,099][105620] Updated weights for policy 1, policy_version 1793070 (0.0008) [2023-12-27 04:25:47,150][105620] Updated weights for policy 1, policy_version 1793080 (0.0010) [2023-12-27 04:25:47,204][105620] Updated weights for policy 1, policy_version 1793090 (0.0010) [2023-12-27 04:25:47,652][105692] Updated weights for policy 0, policy_version 1789194 (0.0008) [2023-12-27 04:25:47,697][105692] Updated weights for policy 0, policy_version 1789204 (0.0008) [2023-12-27 04:25:47,753][105692] Updated weights for policy 0, policy_version 1789214 (0.0008) [2023-12-27 04:25:47,951][105620] Updated weights for policy 1, policy_version 1793100 (0.0008) [2023-12-27 04:25:48,017][105620] Updated weights for policy 1, policy_version 1793110 (0.0007) [2023-12-27 04:25:48,078][105620] Updated weights for policy 1, policy_version 1793120 (0.0010) [2023-12-27 04:25:48,564][105692] Updated weights for policy 0, policy_version 1789224 (0.0007) [2023-12-27 04:25:48,626][105692] Updated weights for policy 0, policy_version 1789234 (0.0005) [2023-12-27 04:25:48,693][105692] Updated weights for policy 0, policy_version 1789244 (0.0005) [2023-12-27 04:25:48,740][105620] Updated weights for policy 1, policy_version 1793130 (0.0010) [2023-12-27 04:25:48,799][105620] Updated weights for policy 1, policy_version 1793140 (0.0007) [2023-12-27 04:25:48,855][105620] Updated weights for policy 1, policy_version 1793150 (0.0008) [2023-12-27 04:25:48,918][105620] Updated weights for policy 1, policy_version 1793160 (0.0008) [2023-12-27 04:25:49,380][105692] Updated weights for policy 0, policy_version 1789254 (0.0009) [2023-12-27 04:25:49,432][105692] Updated weights for policy 0, policy_version 1789264 (0.0011) [2023-12-27 04:25:49,481][105692] Updated weights for policy 0, policy_version 1789274 (0.0010) [2023-12-27 04:25:49,726][105620] Updated weights for policy 1, policy_version 1793170 (0.0008) [2023-12-27 04:25:49,793][105620] Updated weights for policy 1, policy_version 1793180 (0.0005) [2023-12-27 04:25:49,863][105620] Updated weights for policy 1, policy_version 1793190 (0.0008) [2023-12-27 04:25:50,234][105692] Updated weights for policy 0, policy_version 1789284 (0.0011) [2023-12-27 04:25:50,300][105692] Updated weights for policy 0, policy_version 1789294 (0.0011) [2023-12-27 04:25:50,372][105692] Updated weights for policy 0, policy_version 1789304 (0.0011) [2023-12-27 04:25:50,619][105620] Updated weights for policy 1, policy_version 1793200 (0.0007) [2023-12-27 04:25:50,680][105620] Updated weights for policy 1, policy_version 1793210 (0.0006) [2023-12-27 04:25:50,745][105620] Updated weights for policy 1, policy_version 1793220 (0.0006) [2023-12-27 04:25:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 917258240. Throughput: 0: 9913.2, 1: 9653.9. Samples: 917248928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:25:51,062][104569] Avg episode reward: [(0, '8537.021'), (1, '9167.046')] [2023-12-27 04:25:51,104][105692] Updated weights for policy 0, policy_version 1789314 (0.0009) [2023-12-27 04:25:51,166][105692] Updated weights for policy 0, policy_version 1789324 (0.0011) [2023-12-27 04:25:51,225][105692] Updated weights for policy 0, policy_version 1789334 (0.0011) [2023-12-27 04:25:51,282][105692] Updated weights for policy 0, policy_version 1789344 (0.0011) [2023-12-27 04:25:51,369][105620] Updated weights for policy 1, policy_version 1793230 (0.0008) [2023-12-27 04:25:51,431][105620] Updated weights for policy 1, policy_version 1793240 (0.0009) [2023-12-27 04:25:51,494][105620] Updated weights for policy 1, policy_version 1793250 (0.0009) [2023-12-27 04:25:52,062][105692] Updated weights for policy 0, policy_version 1789354 (0.0010) [2023-12-27 04:25:52,139][105692] Updated weights for policy 0, policy_version 1789364 (0.0009) [2023-12-27 04:25:52,196][105620] Updated weights for policy 1, policy_version 1793260 (0.0008) [2023-12-27 04:25:52,196][105692] Updated weights for policy 0, policy_version 1789374 (0.0010) [2023-12-27 04:25:52,248][105620] Updated weights for policy 1, policy_version 1793270 (0.0009) [2023-12-27 04:25:52,313][105620] Updated weights for policy 1, policy_version 1793280 (0.0009) [2023-12-27 04:25:53,002][105620] Updated weights for policy 1, policy_version 1793290 (0.0008) [2023-12-27 04:25:53,030][105692] Updated weights for policy 0, policy_version 1789384 (0.0009) [2023-12-27 04:25:53,069][105620] Updated weights for policy 1, policy_version 1793300 (0.0006) [2023-12-27 04:25:53,080][105692] Updated weights for policy 0, policy_version 1789394 (0.0009) [2023-12-27 04:25:53,137][105620] Updated weights for policy 1, policy_version 1793310 (0.0006) [2023-12-27 04:25:53,138][105692] Updated weights for policy 0, policy_version 1789404 (0.0009) [2023-12-27 04:25:53,201][105620] Updated weights for policy 1, policy_version 1793320 (0.0008) [2023-12-27 04:25:53,854][105620] Updated weights for policy 1, policy_version 1793330 (0.0010) [2023-12-27 04:25:53,902][105620] Updated weights for policy 1, policy_version 1793340 (0.0010) [2023-12-27 04:25:53,928][105692] Updated weights for policy 0, policy_version 1789414 (0.0006) [2023-12-27 04:25:53,946][105620] Updated weights for policy 1, policy_version 1793350 (0.0010) [2023-12-27 04:25:53,982][105692] Updated weights for policy 0, policy_version 1789424 (0.0007) [2023-12-27 04:25:54,041][105692] Updated weights for policy 0, policy_version 1789434 (0.0008) [2023-12-27 04:25:54,707][105620] Updated weights for policy 1, policy_version 1793360 (0.0010) [2023-12-27 04:25:54,751][105620] Updated weights for policy 1, policy_version 1793370 (0.0010) [2023-12-27 04:25:54,799][105620] Updated weights for policy 1, policy_version 1793380 (0.0010) [2023-12-27 04:25:54,808][105692] Updated weights for policy 0, policy_version 1789444 (0.0008) [2023-12-27 04:25:54,852][105692] Updated weights for policy 0, policy_version 1789454 (0.0008) [2023-12-27 04:25:54,897][105692] Updated weights for policy 0, policy_version 1789464 (0.0008) [2023-12-27 04:25:55,548][105620] Updated weights for policy 1, policy_version 1793390 (0.0010) [2023-12-27 04:25:55,593][105620] Updated weights for policy 1, policy_version 1793400 (0.0010) [2023-12-27 04:25:55,641][105620] Updated weights for policy 1, policy_version 1793410 (0.0010) [2023-12-27 04:25:55,692][105692] Updated weights for policy 0, policy_version 1789474 (0.0008) [2023-12-27 04:25:55,751][105692] Updated weights for policy 0, policy_version 1789484 (0.0008) [2023-12-27 04:25:55,806][105692] Updated weights for policy 0, policy_version 1789494 (0.0008) [2023-12-27 04:25:55,872][105692] Updated weights for policy 0, policy_version 1789504 (0.0008) [2023-12-27 04:25:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.7, 300 sec: 19438.7). Total num frames: 917356544. Throughput: 0: 9805.7, 1: 9762.1. Samples: 917362084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:25:56,062][104569] Avg episode reward: [(0, '8263.534'), (1, '9167.167')] [2023-12-27 04:25:56,408][105620] Updated weights for policy 1, policy_version 1793420 (0.0010) [2023-12-27 04:25:56,452][105620] Updated weights for policy 1, policy_version 1793430 (0.0010) [2023-12-27 04:25:56,499][105620] Updated weights for policy 1, policy_version 1793440 (0.0010) [2023-12-27 04:25:56,627][105692] Updated weights for policy 0, policy_version 1789514 (0.0008) [2023-12-27 04:25:56,672][105692] Updated weights for policy 0, policy_version 1789524 (0.0008) [2023-12-27 04:25:56,722][105692] Updated weights for policy 0, policy_version 1789534 (0.0009) [2023-12-27 04:25:57,207][105620] Updated weights for policy 1, policy_version 1793450 (0.0010) [2023-12-27 04:25:57,257][105620] Updated weights for policy 1, policy_version 1793460 (0.0009) [2023-12-27 04:25:57,314][105620] Updated weights for policy 1, policy_version 1793470 (0.0009) [2023-12-27 04:25:57,376][105620] Updated weights for policy 1, policy_version 1793480 (0.0007) [2023-12-27 04:25:57,492][105692] Updated weights for policy 0, policy_version 1789544 (0.0008) [2023-12-27 04:25:57,549][105692] Updated weights for policy 0, policy_version 1789554 (0.0009) [2023-12-27 04:25:57,603][105692] Updated weights for policy 0, policy_version 1789565 (0.0010) [2023-12-27 04:25:57,977][105620] Updated weights for policy 1, policy_version 1793490 (0.0005) [2023-12-27 04:25:58,033][105620] Updated weights for policy 1, policy_version 1793500 (0.0005) [2023-12-27 04:25:58,091][105620] Updated weights for policy 1, policy_version 1793510 (0.0005) [2023-12-27 04:25:58,450][105692] Updated weights for policy 0, policy_version 1789575 (0.0008) [2023-12-27 04:25:58,510][105692] Updated weights for policy 0, policy_version 1789585 (0.0010) [2023-12-27 04:25:58,574][105692] Updated weights for policy 0, policy_version 1789596 (0.0010) [2023-12-27 04:25:58,803][105620] Updated weights for policy 1, policy_version 1793520 (0.0008) [2023-12-27 04:25:58,871][105620] Updated weights for policy 1, policy_version 1793530 (0.0008) [2023-12-27 04:25:58,931][105620] Updated weights for policy 1, policy_version 1793540 (0.0009) [2023-12-27 04:25:59,301][105692] Updated weights for policy 0, policy_version 1789606 (0.0009) [2023-12-27 04:25:59,362][105692] Updated weights for policy 0, policy_version 1789616 (0.0011) [2023-12-27 04:25:59,428][105692] Updated weights for policy 0, policy_version 1789626 (0.0010) [2023-12-27 04:25:59,742][105620] Updated weights for policy 1, policy_version 1793550 (0.0008) [2023-12-27 04:25:59,806][105620] Updated weights for policy 1, policy_version 1793560 (0.0008) [2023-12-27 04:25:59,871][105620] Updated weights for policy 1, policy_version 1793570 (0.0008) [2023-12-27 04:26:00,174][105692] Updated weights for policy 0, policy_version 1789636 (0.0010) [2023-12-27 04:26:00,236][105692] Updated weights for policy 0, policy_version 1789646 (0.0010) [2023-12-27 04:26:00,305][105692] Updated weights for policy 0, policy_version 1789656 (0.0010) [2023-12-27 04:26:00,536][105620] Updated weights for policy 1, policy_version 1793580 (0.0009) [2023-12-27 04:26:00,593][105620] Updated weights for policy 1, policy_version 1793590 (0.0009) [2023-12-27 04:26:00,653][105620] Updated weights for policy 1, policy_version 1793600 (0.0009) [2023-12-27 04:26:00,992][105692] Updated weights for policy 0, policy_version 1789666 (0.0009) [2023-12-27 04:26:01,056][105692] Updated weights for policy 0, policy_version 1789676 (0.0009) [2023-12-27 04:26:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 917446656. Throughput: 0: 9778.0, 1: 9830.3. Samples: 917419188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:26:01,062][104569] Avg episode reward: [(0, '8353.227'), (1, '9259.547')] [2023-12-27 04:26:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001793608_459227136.pth... [2023-12-27 04:26:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001792456_458932224.pth [2023-12-27 04:26:01,119][105692] Updated weights for policy 0, policy_version 1789686 (0.0008) [2023-12-27 04:26:01,184][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001789696_458227712.pth... [2023-12-27 04:26:01,185][105692] Updated weights for policy 0, policy_version 1789696 (0.0008) [2023-12-27 04:26:01,188][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001788576_457940992.pth [2023-12-27 04:26:01,386][105620] Updated weights for policy 1, policy_version 1793610 (0.0008) [2023-12-27 04:26:01,449][105620] Updated weights for policy 1, policy_version 1793620 (0.0008) [2023-12-27 04:26:01,507][105620] Updated weights for policy 1, policy_version 1793630 (0.0008) [2023-12-27 04:26:01,569][105620] Updated weights for policy 1, policy_version 1793640 (0.0008) [2023-12-27 04:26:01,921][105692] Updated weights for policy 0, policy_version 1789706 (0.0008) [2023-12-27 04:26:01,983][105692] Updated weights for policy 0, policy_version 1789716 (0.0009) [2023-12-27 04:26:02,042][105692] Updated weights for policy 0, policy_version 1789726 (0.0009) [2023-12-27 04:26:02,329][105620] Updated weights for policy 1, policy_version 1793650 (0.0005) [2023-12-27 04:26:02,386][105620] Updated weights for policy 1, policy_version 1793660 (0.0007) [2023-12-27 04:26:02,432][105620] Updated weights for policy 1, policy_version 1793670 (0.0005) [2023-12-27 04:26:02,818][105692] Updated weights for policy 0, policy_version 1789736 (0.0008) [2023-12-27 04:26:02,873][105692] Updated weights for policy 0, policy_version 1789746 (0.0009) [2023-12-27 04:26:02,931][105692] Updated weights for policy 0, policy_version 1789756 (0.0009) [2023-12-27 04:26:03,161][105620] Updated weights for policy 1, policy_version 1793680 (0.0006) [2023-12-27 04:26:03,223][105620] Updated weights for policy 1, policy_version 1793690 (0.0006) [2023-12-27 04:26:03,288][105620] Updated weights for policy 1, policy_version 1793700 (0.0005) [2023-12-27 04:26:03,643][105692] Updated weights for policy 0, policy_version 1789766 (0.0009) [2023-12-27 04:26:03,694][105692] Updated weights for policy 0, policy_version 1789776 (0.0009) [2023-12-27 04:26:03,749][105692] Updated weights for policy 0, policy_version 1789786 (0.0011) [2023-12-27 04:26:03,799][105620] Updated weights for policy 1, policy_version 1793710 (0.0005) [2023-12-27 04:26:03,854][105620] Updated weights for policy 1, policy_version 1793720 (0.0006) [2023-12-27 04:26:03,918][105620] Updated weights for policy 1, policy_version 1793730 (0.0009) [2023-12-27 04:26:04,519][105692] Updated weights for policy 0, policy_version 1789797 (0.0008) [2023-12-27 04:26:04,580][105692] Updated weights for policy 0, policy_version 1789807 (0.0008) [2023-12-27 04:26:04,606][105620] Updated weights for policy 1, policy_version 1793740 (0.0010) [2023-12-27 04:26:04,637][105692] Updated weights for policy 0, policy_version 1789817 (0.0009) [2023-12-27 04:26:04,659][105620] Updated weights for policy 1, policy_version 1793750 (0.0011) [2023-12-27 04:26:04,718][105620] Updated weights for policy 1, policy_version 1793760 (0.0011) [2023-12-27 04:26:05,385][105692] Updated weights for policy 0, policy_version 1789827 (0.0006) [2023-12-27 04:26:05,440][105692] Updated weights for policy 0, policy_version 1789837 (0.0007) [2023-12-27 04:26:05,443][105620] Updated weights for policy 1, policy_version 1793770 (0.0011) [2023-12-27 04:26:05,493][105692] Updated weights for policy 0, policy_version 1789847 (0.0006) [2023-12-27 04:26:05,499][105620] Updated weights for policy 1, policy_version 1793780 (0.0011) [2023-12-27 04:26:05,555][105620] Updated weights for policy 1, policy_version 1793790 (0.0011) [2023-12-27 04:26:05,608][105620] Updated weights for policy 1, policy_version 1793800 (0.0011) [2023-12-27 04:26:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 917544960. Throughput: 0: 9658.4, 1: 9831.4. Samples: 917535156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:26:06,062][104569] Avg episode reward: [(0, '8352.062'), (1, '9259.294')] [2023-12-27 04:26:06,270][105692] Updated weights for policy 0, policy_version 1789857 (0.0007) [2023-12-27 04:26:06,319][105692] Updated weights for policy 0, policy_version 1789867 (0.0008) [2023-12-27 04:26:06,380][105692] Updated weights for policy 0, policy_version 1789877 (0.0006) [2023-12-27 04:26:06,385][105620] Updated weights for policy 1, policy_version 1793810 (0.0011) [2023-12-27 04:26:06,437][105692] Updated weights for policy 0, policy_version 1789887 (0.0008) [2023-12-27 04:26:06,451][105620] Updated weights for policy 1, policy_version 1793820 (0.0011) [2023-12-27 04:26:06,520][105620] Updated weights for policy 1, policy_version 1793830 (0.0011) [2023-12-27 04:26:07,207][105692] Updated weights for policy 0, policy_version 1789897 (0.0008) [2023-12-27 04:26:07,251][105620] Updated weights for policy 1, policy_version 1793840 (0.0011) [2023-12-27 04:26:07,258][105692] Updated weights for policy 0, policy_version 1789907 (0.0006) [2023-12-27 04:26:07,299][105620] Updated weights for policy 1, policy_version 1793850 (0.0010) [2023-12-27 04:26:07,303][105692] Updated weights for policy 0, policy_version 1789917 (0.0009) [2023-12-27 04:26:07,358][105620] Updated weights for policy 1, policy_version 1793860 (0.0010) [2023-12-27 04:26:08,099][105692] Updated weights for policy 0, policy_version 1789927 (0.0008) [2023-12-27 04:26:08,104][105620] Updated weights for policy 1, policy_version 1793870 (0.0010) [2023-12-27 04:26:08,154][105620] Updated weights for policy 1, policy_version 1793880 (0.0009) [2023-12-27 04:26:08,161][105692] Updated weights for policy 0, policy_version 1789937 (0.0007) [2023-12-27 04:26:08,207][105620] Updated weights for policy 1, policy_version 1793890 (0.0009) [2023-12-27 04:26:08,221][105692] Updated weights for policy 0, policy_version 1789947 (0.0007) [2023-12-27 04:26:08,979][105692] Updated weights for policy 0, policy_version 1789957 (0.0008) [2023-12-27 04:26:08,979][105620] Updated weights for policy 1, policy_version 1793900 (0.0010) [2023-12-27 04:26:09,024][105692] Updated weights for policy 0, policy_version 1789967 (0.0008) [2023-12-27 04:26:09,038][105620] Updated weights for policy 1, policy_version 1793910 (0.0010) [2023-12-27 04:26:09,073][105692] Updated weights for policy 0, policy_version 1789977 (0.0007) [2023-12-27 04:26:09,103][105620] Updated weights for policy 1, policy_version 1793920 (0.0010) [2023-12-27 04:26:09,862][105620] Updated weights for policy 1, policy_version 1793930 (0.0011) [2023-12-27 04:26:09,881][105692] Updated weights for policy 0, policy_version 1789987 (0.0007) [2023-12-27 04:26:09,928][105620] Updated weights for policy 1, policy_version 1793940 (0.0010) [2023-12-27 04:26:09,947][105692] Updated weights for policy 0, policy_version 1789997 (0.0008) [2023-12-27 04:26:09,987][105620] Updated weights for policy 1, policy_version 1793950 (0.0011) [2023-12-27 04:26:10,009][105692] Updated weights for policy 0, policy_version 1790007 (0.0008) [2023-12-27 04:26:10,044][105620] Updated weights for policy 1, policy_version 1793960 (0.0010) [2023-12-27 04:26:10,782][105692] Updated weights for policy 0, policy_version 1790017 (0.0006) [2023-12-27 04:26:10,797][105620] Updated weights for policy 1, policy_version 1793970 (0.0010) [2023-12-27 04:26:10,835][105692] Updated weights for policy 0, policy_version 1790027 (0.0005) [2023-12-27 04:26:10,848][105620] Updated weights for policy 1, policy_version 1793980 (0.0010) [2023-12-27 04:26:10,884][105692] Updated weights for policy 0, policy_version 1790037 (0.0006) [2023-12-27 04:26:10,907][105620] Updated weights for policy 1, policy_version 1793990 (0.0010) [2023-12-27 04:26:10,942][105692] Updated weights for policy 0, policy_version 1790047 (0.0009) [2023-12-27 04:26:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 917643264. Throughput: 0: 9586.6, 1: 9845.7. Samples: 917645212. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:26:11,062][104569] Avg episode reward: [(0, '8625.301'), (1, '9259.347')] [2023-12-27 04:26:11,726][105620] Updated weights for policy 1, policy_version 1794000 (0.0008) [2023-12-27 04:26:11,786][105620] Updated weights for policy 1, policy_version 1794010 (0.0008) [2023-12-27 04:26:11,819][105692] Updated weights for policy 0, policy_version 1790057 (0.0007) [2023-12-27 04:26:11,836][105620] Updated weights for policy 1, policy_version 1794020 (0.0008) [2023-12-27 04:26:11,872][105692] Updated weights for policy 0, policy_version 1790067 (0.0007) [2023-12-27 04:26:11,932][105692] Updated weights for policy 0, policy_version 1790077 (0.0008) [2023-12-27 04:26:12,609][105620] Updated weights for policy 1, policy_version 1794030 (0.0010) [2023-12-27 04:26:12,662][105620] Updated weights for policy 1, policy_version 1794040 (0.0011) [2023-12-27 04:26:12,719][105692] Updated weights for policy 0, policy_version 1790087 (0.0006) [2023-12-27 04:26:12,724][105620] Updated weights for policy 1, policy_version 1794050 (0.0010) [2023-12-27 04:26:12,777][105692] Updated weights for policy 0, policy_version 1790097 (0.0007) [2023-12-27 04:26:12,830][105692] Updated weights for policy 0, policy_version 1790107 (0.0008) [2023-12-27 04:26:13,423][105692] Updated weights for policy 0, policy_version 1790117 (0.0007) [2023-12-27 04:26:13,463][105620] Updated weights for policy 1, policy_version 1794060 (0.0010) [2023-12-27 04:26:13,481][105692] Updated weights for policy 0, policy_version 1790127 (0.0006) [2023-12-27 04:26:13,528][105620] Updated weights for policy 1, policy_version 1794070 (0.0010) [2023-12-27 04:26:13,538][105692] Updated weights for policy 0, policy_version 1790137 (0.0005) [2023-12-27 04:26:13,585][105620] Updated weights for policy 1, policy_version 1794080 (0.0010) [2023-12-27 04:26:14,236][105692] Updated weights for policy 0, policy_version 1790147 (0.0006) [2023-12-27 04:26:14,292][105692] Updated weights for policy 0, policy_version 1790157 (0.0008) [2023-12-27 04:26:14,322][105620] Updated weights for policy 1, policy_version 1794090 (0.0010) [2023-12-27 04:26:14,347][105692] Updated weights for policy 0, policy_version 1790167 (0.0008) [2023-12-27 04:26:14,383][105620] Updated weights for policy 1, policy_version 1794100 (0.0010) [2023-12-27 04:26:14,444][105620] Updated weights for policy 1, policy_version 1794110 (0.0010) [2023-12-27 04:26:14,508][105620] Updated weights for policy 1, policy_version 1794120 (0.0010) [2023-12-27 04:26:15,138][105692] Updated weights for policy 0, policy_version 1790177 (0.0006) [2023-12-27 04:26:15,190][105692] Updated weights for policy 0, policy_version 1790187 (0.0008) [2023-12-27 04:26:15,231][105620] Updated weights for policy 1, policy_version 1794130 (0.0010) [2023-12-27 04:26:15,254][105692] Updated weights for policy 0, policy_version 1790197 (0.0006) [2023-12-27 04:26:15,290][105620] Updated weights for policy 1, policy_version 1794140 (0.0010) [2023-12-27 04:26:15,312][105692] Updated weights for policy 0, policy_version 1790207 (0.0007) [2023-12-27 04:26:15,355][105620] Updated weights for policy 1, policy_version 1794150 (0.0010) [2023-12-27 04:26:16,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 917725184. Throughput: 0: 9450.7, 1: 9852.6. Samples: 917701308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:26:16,062][104569] Avg episode reward: [(0, '8535.446'), (1, '9259.438')] [2023-12-27 04:26:16,079][105692] Updated weights for policy 0, policy_version 1790217 (0.0007) [2023-12-27 04:26:16,096][105620] Updated weights for policy 1, policy_version 1794160 (0.0010) [2023-12-27 04:26:16,136][105692] Updated weights for policy 0, policy_version 1790227 (0.0010) [2023-12-27 04:26:16,154][105620] Updated weights for policy 1, policy_version 1794170 (0.0010) [2023-12-27 04:26:16,195][105692] Updated weights for policy 0, policy_version 1790237 (0.0005) [2023-12-27 04:26:16,205][105620] Updated weights for policy 1, policy_version 1794180 (0.0010) [2023-12-27 04:26:16,210][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001790240_458366976.pth... [2023-12-27 04:26:16,214][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001789152_458088448.pth [2023-12-27 04:26:16,224][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001794184_459374592.pth... [2023-12-27 04:26:16,227][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001793032_459079680.pth [2023-12-27 04:26:16,941][105620] Updated weights for policy 1, policy_version 1794190 (0.0010) [2023-12-27 04:26:16,942][105692] Updated weights for policy 0, policy_version 1790247 (0.0006) [2023-12-27 04:26:16,993][105620] Updated weights for policy 1, policy_version 1794200 (0.0010) [2023-12-27 04:26:16,998][105692] Updated weights for policy 0, policy_version 1790257 (0.0006) [2023-12-27 04:26:17,050][105692] Updated weights for policy 0, policy_version 1790267 (0.0011) [2023-12-27 04:26:17,051][105620] Updated weights for policy 1, policy_version 1794210 (0.0010) [2023-12-27 04:26:17,738][105692] Updated weights for policy 0, policy_version 1790277 (0.0006) [2023-12-27 04:26:17,767][105620] Updated weights for policy 1, policy_version 1794220 (0.0010) [2023-12-27 04:26:17,799][105692] Updated weights for policy 0, policy_version 1790287 (0.0005) [2023-12-27 04:26:17,819][105620] Updated weights for policy 1, policy_version 1794230 (0.0010) [2023-12-27 04:26:17,843][105692] Updated weights for policy 0, policy_version 1790297 (0.0005) [2023-12-27 04:26:17,874][105620] Updated weights for policy 1, policy_version 1794240 (0.0010) [2023-12-27 04:26:18,499][105692] Updated weights for policy 0, policy_version 1790307 (0.0007) [2023-12-27 04:26:18,558][105692] Updated weights for policy 0, policy_version 1790317 (0.0011) [2023-12-27 04:26:18,565][105620] Updated weights for policy 1, policy_version 1794250 (0.0010) [2023-12-27 04:26:18,617][105692] Updated weights for policy 0, policy_version 1790327 (0.0011) [2023-12-27 04:26:18,623][105620] Updated weights for policy 1, policy_version 1794260 (0.0007) [2023-12-27 04:26:18,678][105620] Updated weights for policy 1, policy_version 1794270 (0.0010) [2023-12-27 04:26:18,727][105620] Updated weights for policy 1, policy_version 1794280 (0.0006) [2023-12-27 04:26:19,381][105692] Updated weights for policy 0, policy_version 1790337 (0.0010) [2023-12-27 04:26:19,398][105620] Updated weights for policy 1, policy_version 1794290 (0.0008) [2023-12-27 04:26:19,437][105692] Updated weights for policy 0, policy_version 1790347 (0.0008) [2023-12-27 04:26:19,447][105620] Updated weights for policy 1, policy_version 1794300 (0.0010) [2023-12-27 04:26:19,491][105692] Updated weights for policy 0, policy_version 1790357 (0.0008) [2023-12-27 04:26:19,506][105620] Updated weights for policy 1, policy_version 1794310 (0.0011) [2023-12-27 04:26:19,549][105692] Updated weights for policy 0, policy_version 1790367 (0.0008) [2023-12-27 04:26:20,264][105620] Updated weights for policy 1, policy_version 1794320 (0.0007) [2023-12-27 04:26:20,331][105620] Updated weights for policy 1, policy_version 1794330 (0.0006) [2023-12-27 04:26:20,352][105692] Updated weights for policy 0, policy_version 1790377 (0.0009) [2023-12-27 04:26:20,394][105620] Updated weights for policy 1, policy_version 1794340 (0.0008) [2023-12-27 04:26:20,408][105692] Updated weights for policy 0, policy_version 1790387 (0.0006) [2023-12-27 04:26:20,458][105692] Updated weights for policy 0, policy_version 1790397 (0.0008) [2023-12-27 04:26:21,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 917823488. Throughput: 0: 9452.5, 1: 9779.6. Samples: 917816716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:26:21,062][104569] Avg episode reward: [(0, '8445.528'), (1, '9074.800')] [2023-12-27 04:26:21,074][105620] Updated weights for policy 1, policy_version 1794350 (0.0008) [2023-12-27 04:26:21,140][105620] Updated weights for policy 1, policy_version 1794360 (0.0009) [2023-12-27 04:26:21,201][105620] Updated weights for policy 1, policy_version 1794370 (0.0009) [2023-12-27 04:26:21,294][105692] Updated weights for policy 0, policy_version 1790407 (0.0010) [2023-12-27 04:26:21,359][105692] Updated weights for policy 0, policy_version 1790417 (0.0010) [2023-12-27 04:26:21,431][105692] Updated weights for policy 0, policy_version 1790427 (0.0007) [2023-12-27 04:26:21,940][105620] Updated weights for policy 1, policy_version 1794380 (0.0008) [2023-12-27 04:26:22,007][105620] Updated weights for policy 1, policy_version 1794390 (0.0009) [2023-12-27 04:26:22,072][105620] Updated weights for policy 1, policy_version 1794400 (0.0010) [2023-12-27 04:26:22,227][105692] Updated weights for policy 0, policy_version 1790437 (0.0010) [2023-12-27 04:26:22,294][105692] Updated weights for policy 0, policy_version 1790447 (0.0009) [2023-12-27 04:26:22,364][105692] Updated weights for policy 0, policy_version 1790457 (0.0009) [2023-12-27 04:26:22,702][105620] Updated weights for policy 1, policy_version 1794410 (0.0009) [2023-12-27 04:26:22,769][105620] Updated weights for policy 1, policy_version 1794420 (0.0005) [2023-12-27 04:26:22,836][105620] Updated weights for policy 1, policy_version 1794430 (0.0005) [2023-12-27 04:26:22,905][105620] Updated weights for policy 1, policy_version 1794440 (0.0005) [2023-12-27 04:26:23,076][105692] Updated weights for policy 0, policy_version 1790467 (0.0007) [2023-12-27 04:26:23,138][105692] Updated weights for policy 0, policy_version 1790477 (0.0011) [2023-12-27 04:26:23,200][105692] Updated weights for policy 0, policy_version 1790487 (0.0009) [2023-12-27 04:26:23,528][105620] Updated weights for policy 1, policy_version 1794450 (0.0008) [2023-12-27 04:26:23,583][105620] Updated weights for policy 1, policy_version 1794460 (0.0009) [2023-12-27 04:26:23,641][105620] Updated weights for policy 1, policy_version 1794470 (0.0007) [2023-12-27 04:26:23,969][105692] Updated weights for policy 0, policy_version 1790497 (0.0009) [2023-12-27 04:26:24,045][105692] Updated weights for policy 0, policy_version 1790507 (0.0011) [2023-12-27 04:26:24,107][105692] Updated weights for policy 0, policy_version 1790517 (0.0010) [2023-12-27 04:26:24,170][105692] Updated weights for policy 0, policy_version 1790527 (0.0010) [2023-12-27 04:26:24,295][105620] Updated weights for policy 1, policy_version 1794480 (0.0006) [2023-12-27 04:26:24,349][105620] Updated weights for policy 1, policy_version 1794490 (0.0007) [2023-12-27 04:26:24,408][105620] Updated weights for policy 1, policy_version 1794500 (0.0008) [2023-12-27 04:26:24,883][105692] Updated weights for policy 0, policy_version 1790537 (0.0009) [2023-12-27 04:26:24,936][105692] Updated weights for policy 0, policy_version 1790547 (0.0010) [2023-12-27 04:26:24,989][105692] Updated weights for policy 0, policy_version 1790558 (0.0009) [2023-12-27 04:26:25,023][105620] Updated weights for policy 1, policy_version 1794510 (0.0007) [2023-12-27 04:26:25,072][105620] Updated weights for policy 1, policy_version 1794520 (0.0008) [2023-12-27 04:26:25,131][105620] Updated weights for policy 1, policy_version 1794530 (0.0009) [2023-12-27 04:26:25,795][105692] Updated weights for policy 0, policy_version 1790568 (0.0008) [2023-12-27 04:26:25,839][105692] Updated weights for policy 0, policy_version 1790578 (0.0008) [2023-12-27 04:26:25,872][105620] Updated weights for policy 1, policy_version 1794540 (0.0010) [2023-12-27 04:26:25,889][105692] Updated weights for policy 0, policy_version 1790588 (0.0008) [2023-12-27 04:26:25,927][105620] Updated weights for policy 1, policy_version 1794550 (0.0010) [2023-12-27 04:26:25,985][105620] Updated weights for policy 1, policy_version 1794560 (0.0010) [2023-12-27 04:26:26,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 917929984. Throughput: 0: 9254.9, 1: 9804.0. Samples: 917932112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:26:26,063][104569] Avg episode reward: [(0, '8539.423'), (1, '9167.504')] [2023-12-27 04:26:26,607][105620] Updated weights for policy 1, policy_version 1794570 (0.0010) [2023-12-27 04:26:26,618][105692] Updated weights for policy 0, policy_version 1790598 (0.0007) [2023-12-27 04:26:26,664][105692] Updated weights for policy 0, policy_version 1790608 (0.0006) [2023-12-27 04:26:26,666][105620] Updated weights for policy 1, policy_version 1794580 (0.0009) [2023-12-27 04:26:26,712][105692] Updated weights for policy 0, policy_version 1790618 (0.0006) [2023-12-27 04:26:26,720][105620] Updated weights for policy 1, policy_version 1794590 (0.0008) [2023-12-27 04:26:26,779][105620] Updated weights for policy 1, policy_version 1794600 (0.0009) [2023-12-27 04:26:27,292][105692] Updated weights for policy 0, policy_version 1790628 (0.0005) [2023-12-27 04:26:27,344][105692] Updated weights for policy 0, policy_version 1790638 (0.0006) [2023-12-27 04:26:27,395][105692] Updated weights for policy 0, policy_version 1790648 (0.0005) [2023-12-27 04:26:27,427][105620] Updated weights for policy 1, policy_version 1794610 (0.0009) [2023-12-27 04:26:27,476][105620] Updated weights for policy 1, policy_version 1794621 (0.0009) [2023-12-27 04:26:27,524][105620] Updated weights for policy 1, policy_version 1794631 (0.0010) [2023-12-27 04:26:27,913][105692] Updated weights for policy 0, policy_version 1790658 (0.0005) [2023-12-27 04:26:27,974][105692] Updated weights for policy 0, policy_version 1790668 (0.0005) [2023-12-27 04:26:28,038][105692] Updated weights for policy 0, policy_version 1790678 (0.0005) [2023-12-27 04:26:28,100][105692] Updated weights for policy 0, policy_version 1790688 (0.0005) [2023-12-27 04:26:28,299][105620] Updated weights for policy 1, policy_version 1794641 (0.0010) [2023-12-27 04:26:28,352][105620] Updated weights for policy 1, policy_version 1794651 (0.0009) [2023-12-27 04:26:28,411][105620] Updated weights for policy 1, policy_version 1794661 (0.0009) [2023-12-27 04:26:28,706][105692] Updated weights for policy 0, policy_version 1790698 (0.0009) [2023-12-27 04:26:28,763][105692] Updated weights for policy 0, policy_version 1790708 (0.0009) [2023-12-27 04:26:28,825][105692] Updated weights for policy 0, policy_version 1790718 (0.0010) [2023-12-27 04:26:29,038][105620] Updated weights for policy 1, policy_version 1794671 (0.0007) [2023-12-27 04:26:29,097][105620] Updated weights for policy 1, policy_version 1794681 (0.0005) [2023-12-27 04:26:29,159][105620] Updated weights for policy 1, policy_version 1794691 (0.0009) [2023-12-27 04:26:29,627][105692] Updated weights for policy 0, policy_version 1790728 (0.0008) [2023-12-27 04:26:29,676][105692] Updated weights for policy 0, policy_version 1790738 (0.0008) [2023-12-27 04:26:29,731][105692] Updated weights for policy 0, policy_version 1790748 (0.0008) [2023-12-27 04:26:29,835][105620] Updated weights for policy 1, policy_version 1794701 (0.0010) [2023-12-27 04:26:29,883][105620] Updated weights for policy 1, policy_version 1794711 (0.0009) [2023-12-27 04:26:29,944][105620] Updated weights for policy 1, policy_version 1794721 (0.0010) [2023-12-27 04:26:30,497][105692] Updated weights for policy 0, policy_version 1790758 (0.0008) [2023-12-27 04:26:30,548][105692] Updated weights for policy 0, policy_version 1790768 (0.0005) [2023-12-27 04:26:30,604][105692] Updated weights for policy 0, policy_version 1790778 (0.0006) [2023-12-27 04:26:30,610][105620] Updated weights for policy 1, policy_version 1794731 (0.0011) [2023-12-27 04:26:30,676][105620] Updated weights for policy 1, policy_version 1794741 (0.0011) [2023-12-27 04:26:30,734][105620] Updated weights for policy 1, policy_version 1794751 (0.0009) [2023-12-27 04:26:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 918028288. Throughput: 0: 9395.9, 1: 9806.9. Samples: 917996864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:26:31,063][104569] Avg episode reward: [(0, '8537.396'), (1, '9259.983')] [2023-12-27 04:26:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001790784_458506240.pth... [2023-12-27 04:26:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001794760_459522048.pth... [2023-12-27 04:26:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001793608_459227136.pth [2023-12-27 04:26:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001789696_458227712.pth [2023-12-27 04:26:31,322][105692] Updated weights for policy 0, policy_version 1790788 (0.0008) [2023-12-27 04:26:31,373][105620] Updated weights for policy 1, policy_version 1794761 (0.0005) [2023-12-27 04:26:31,397][105692] Updated weights for policy 0, policy_version 1790798 (0.0008) [2023-12-27 04:26:31,440][105620] Updated weights for policy 1, policy_version 1794771 (0.0008) [2023-12-27 04:26:31,451][105692] Updated weights for policy 0, policy_version 1790808 (0.0009) [2023-12-27 04:26:31,499][105620] Updated weights for policy 1, policy_version 1794781 (0.0010) [2023-12-27 04:26:31,557][105620] Updated weights for policy 1, policy_version 1794791 (0.0010) [2023-12-27 04:26:32,104][105692] Updated weights for policy 0, policy_version 1790818 (0.0008) [2023-12-27 04:26:32,172][105692] Updated weights for policy 0, policy_version 1790828 (0.0005) [2023-12-27 04:26:32,225][105692] Updated weights for policy 0, policy_version 1790838 (0.0005) [2023-12-27 04:26:32,282][105692] Updated weights for policy 0, policy_version 1790848 (0.0006) [2023-12-27 04:26:32,296][105620] Updated weights for policy 1, policy_version 1794801 (0.0011) [2023-12-27 04:26:32,351][105620] Updated weights for policy 1, policy_version 1794811 (0.0010) [2023-12-27 04:26:32,407][105620] Updated weights for policy 1, policy_version 1794821 (0.0011) [2023-12-27 04:26:32,922][105692] Updated weights for policy 0, policy_version 1790858 (0.0008) [2023-12-27 04:26:32,980][105692] Updated weights for policy 0, policy_version 1790868 (0.0007) [2023-12-27 04:26:33,027][105692] Updated weights for policy 0, policy_version 1790878 (0.0008) [2023-12-27 04:26:33,161][105620] Updated weights for policy 1, policy_version 1794831 (0.0010) [2023-12-27 04:26:33,215][105620] Updated weights for policy 1, policy_version 1794841 (0.0010) [2023-12-27 04:26:33,270][105620] Updated weights for policy 1, policy_version 1794851 (0.0010) [2023-12-27 04:26:33,619][105692] Updated weights for policy 0, policy_version 1790888 (0.0005) [2023-12-27 04:26:33,672][105692] Updated weights for policy 0, policy_version 1790898 (0.0007) [2023-12-27 04:26:33,727][105692] Updated weights for policy 0, policy_version 1790908 (0.0006) [2023-12-27 04:26:34,014][105620] Updated weights for policy 1, policy_version 1794861 (0.0008) [2023-12-27 04:26:34,057][105620] Updated weights for policy 1, policy_version 1794871 (0.0005) [2023-12-27 04:26:34,106][105620] Updated weights for policy 1, policy_version 1794881 (0.0005) [2023-12-27 04:26:34,369][105692] Updated weights for policy 0, policy_version 1790918 (0.0008) [2023-12-27 04:26:34,428][105692] Updated weights for policy 0, policy_version 1790928 (0.0008) [2023-12-27 04:26:34,482][105692] Updated weights for policy 0, policy_version 1790938 (0.0007) [2023-12-27 04:26:34,852][105620] Updated weights for policy 1, policy_version 1794891 (0.0009) [2023-12-27 04:26:34,900][105620] Updated weights for policy 1, policy_version 1794901 (0.0010) [2023-12-27 04:26:34,945][105620] Updated weights for policy 1, policy_version 1794911 (0.0010) [2023-12-27 04:26:35,086][105692] Updated weights for policy 0, policy_version 1790948 (0.0005) [2023-12-27 04:26:35,142][105692] Updated weights for policy 0, policy_version 1790958 (0.0005) [2023-12-27 04:26:35,194][105692] Updated weights for policy 0, policy_version 1790968 (0.0010) [2023-12-27 04:26:35,703][105620] Updated weights for policy 1, policy_version 1794921 (0.0010) [2023-12-27 04:26:35,762][105620] Updated weights for policy 1, policy_version 1794931 (0.0011) [2023-12-27 04:26:35,821][105620] Updated weights for policy 1, policy_version 1794941 (0.0010) [2023-12-27 04:26:35,826][105692] Updated weights for policy 0, policy_version 1790978 (0.0009) [2023-12-27 04:26:35,874][105620] Updated weights for policy 1, policy_version 1794951 (0.0010) [2023-12-27 04:26:35,887][105692] Updated weights for policy 0, policy_version 1790988 (0.0006) [2023-12-27 04:26:35,931][105692] Updated weights for policy 0, policy_version 1790998 (0.0005) [2023-12-27 04:26:35,983][105692] Updated weights for policy 0, policy_version 1791008 (0.0005) [2023-12-27 04:26:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 918134784. Throughput: 0: 9471.5, 1: 9792.2. Samples: 918115796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:26:36,063][104569] Avg episode reward: [(0, '8172.253'), (1, '9259.918')] [2023-12-27 04:26:36,628][105692] Updated weights for policy 0, policy_version 1791018 (0.0008) [2023-12-27 04:26:36,653][105620] Updated weights for policy 1, policy_version 1794961 (0.0011) [2023-12-27 04:26:36,683][105692] Updated weights for policy 0, policy_version 1791028 (0.0008) [2023-12-27 04:26:36,717][105620] Updated weights for policy 1, policy_version 1794971 (0.0011) [2023-12-27 04:26:36,741][105692] Updated weights for policy 0, policy_version 1791038 (0.0007) [2023-12-27 04:26:36,784][105620] Updated weights for policy 1, policy_version 1794981 (0.0011) [2023-12-27 04:26:37,408][105692] Updated weights for policy 0, policy_version 1791048 (0.0008) [2023-12-27 04:26:37,452][105692] Updated weights for policy 0, policy_version 1791058 (0.0008) [2023-12-27 04:26:37,506][105620] Updated weights for policy 1, policy_version 1794991 (0.0011) [2023-12-27 04:26:37,516][105692] Updated weights for policy 0, policy_version 1791068 (0.0007) [2023-12-27 04:26:37,564][105620] Updated weights for policy 1, policy_version 1795001 (0.0010) [2023-12-27 04:26:37,626][105620] Updated weights for policy 1, policy_version 1795011 (0.0011) [2023-12-27 04:26:38,265][105620] Updated weights for policy 1, policy_version 1795021 (0.0009) [2023-12-27 04:26:38,302][105692] Updated weights for policy 0, policy_version 1791078 (0.0007) [2023-12-27 04:26:38,334][105620] Updated weights for policy 1, policy_version 1795031 (0.0009) [2023-12-27 04:26:38,362][105692] Updated weights for policy 0, policy_version 1791088 (0.0008) [2023-12-27 04:26:38,394][105620] Updated weights for policy 1, policy_version 1795041 (0.0007) [2023-12-27 04:26:38,421][105692] Updated weights for policy 0, policy_version 1791098 (0.0008) [2023-12-27 04:26:39,137][105620] Updated weights for policy 1, policy_version 1795051 (0.0008) [2023-12-27 04:26:39,177][105692] Updated weights for policy 0, policy_version 1791108 (0.0008) [2023-12-27 04:26:39,185][105620] Updated weights for policy 1, policy_version 1795061 (0.0009) [2023-12-27 04:26:39,231][105692] Updated weights for policy 0, policy_version 1791118 (0.0008) [2023-12-27 04:26:39,244][105620] Updated weights for policy 1, policy_version 1795071 (0.0007) [2023-12-27 04:26:39,290][105692] Updated weights for policy 0, policy_version 1791128 (0.0007) [2023-12-27 04:26:39,962][105692] Updated weights for policy 0, policy_version 1791138 (0.0010) [2023-12-27 04:26:40,016][105620] Updated weights for policy 1, policy_version 1795081 (0.0006) [2023-12-27 04:26:40,023][105692] Updated weights for policy 0, policy_version 1791148 (0.0009) [2023-12-27 04:26:40,079][105620] Updated weights for policy 1, policy_version 1795091 (0.0007) [2023-12-27 04:26:40,081][105692] Updated weights for policy 0, policy_version 1791158 (0.0006) [2023-12-27 04:26:40,138][105620] Updated weights for policy 1, policy_version 1795101 (0.0007) [2023-12-27 04:26:40,144][105692] Updated weights for policy 0, policy_version 1791168 (0.0008) [2023-12-27 04:26:40,202][105620] Updated weights for policy 1, policy_version 1795111 (0.0007) [2023-12-27 04:26:40,757][105692] Updated weights for policy 0, policy_version 1791178 (0.0006) [2023-12-27 04:26:40,817][105692] Updated weights for policy 0, policy_version 1791188 (0.0006) [2023-12-27 04:26:40,883][105692] Updated weights for policy 0, policy_version 1791198 (0.0009) [2023-12-27 04:26:41,023][105620] Updated weights for policy 1, policy_version 1795121 (0.0010) [2023-12-27 04:26:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 918224896. Throughput: 0: 9619.5, 1: 9737.2. Samples: 918233140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:26:41,063][104569] Avg episode reward: [(0, '8263.862'), (1, '9260.033')] [2023-12-27 04:26:41,095][105620] Updated weights for policy 1, policy_version 1795131 (0.0010) [2023-12-27 04:26:41,160][105620] Updated weights for policy 1, policy_version 1795141 (0.0009) [2023-12-27 04:26:41,593][105692] Updated weights for policy 0, policy_version 1791208 (0.0008) [2023-12-27 04:26:41,656][105692] Updated weights for policy 0, policy_version 1791218 (0.0009) [2023-12-27 04:26:41,721][105692] Updated weights for policy 0, policy_version 1791228 (0.0008) [2023-12-27 04:26:42,021][105620] Updated weights for policy 1, policy_version 1795151 (0.0007) [2023-12-27 04:26:42,085][105620] Updated weights for policy 1, policy_version 1795161 (0.0007) [2023-12-27 04:26:42,147][105620] Updated weights for policy 1, policy_version 1795171 (0.0009) [2023-12-27 04:26:42,414][105692] Updated weights for policy 0, policy_version 1791238 (0.0008) [2023-12-27 04:26:42,468][105692] Updated weights for policy 0, policy_version 1791248 (0.0008) [2023-12-27 04:26:42,525][105692] Updated weights for policy 0, policy_version 1791258 (0.0009) [2023-12-27 04:26:42,801][105620] Updated weights for policy 1, policy_version 1795181 (0.0011) [2023-12-27 04:26:42,859][105620] Updated weights for policy 1, policy_version 1795191 (0.0010) [2023-12-27 04:26:42,916][105620] Updated weights for policy 1, policy_version 1795201 (0.0010) [2023-12-27 04:26:43,346][105692] Updated weights for policy 0, policy_version 1791268 (0.0008) [2023-12-27 04:26:43,394][105692] Updated weights for policy 0, policy_version 1791278 (0.0008) [2023-12-27 04:26:43,442][105692] Updated weights for policy 0, policy_version 1791288 (0.0009) [2023-12-27 04:26:43,576][105620] Updated weights for policy 1, policy_version 1795211 (0.0011) [2023-12-27 04:26:43,628][105620] Updated weights for policy 1, policy_version 1795221 (0.0010) [2023-12-27 04:26:43,680][105620] Updated weights for policy 1, policy_version 1795231 (0.0010) [2023-12-27 04:26:44,150][105692] Updated weights for policy 0, policy_version 1791298 (0.0010) [2023-12-27 04:26:44,215][105692] Updated weights for policy 0, policy_version 1791308 (0.0010) [2023-12-27 04:26:44,273][105692] Updated weights for policy 0, policy_version 1791318 (0.0010) [2023-12-27 04:26:44,300][105620] Updated weights for policy 1, policy_version 1795241 (0.0010) [2023-12-27 04:26:44,328][105692] Updated weights for policy 0, policy_version 1791328 (0.0011) [2023-12-27 04:26:44,354][105620] Updated weights for policy 1, policy_version 1795251 (0.0005) [2023-12-27 04:26:44,408][105620] Updated weights for policy 1, policy_version 1795261 (0.0005) [2023-12-27 04:26:44,456][105620] Updated weights for policy 1, policy_version 1795271 (0.0006) [2023-12-27 04:26:45,030][105692] Updated weights for policy 0, policy_version 1791338 (0.0007) [2023-12-27 04:26:45,065][105620] Updated weights for policy 1, policy_version 1795281 (0.0008) [2023-12-27 04:26:45,084][105692] Updated weights for policy 0, policy_version 1791348 (0.0007) [2023-12-27 04:26:45,123][105620] Updated weights for policy 1, policy_version 1795291 (0.0007) [2023-12-27 04:26:45,142][105692] Updated weights for policy 0, policy_version 1791358 (0.0007) [2023-12-27 04:26:45,180][105620] Updated weights for policy 1, policy_version 1795301 (0.0007) [2023-12-27 04:26:45,813][105692] Updated weights for policy 0, policy_version 1791368 (0.0007) [2023-12-27 04:26:45,863][105692] Updated weights for policy 0, policy_version 1791378 (0.0005) [2023-12-27 04:26:45,916][105692] Updated weights for policy 0, policy_version 1791388 (0.0005) [2023-12-27 04:26:45,974][105620] Updated weights for policy 1, policy_version 1795311 (0.0009) [2023-12-27 04:26:46,032][105620] Updated weights for policy 1, policy_version 1795321 (0.0010) [2023-12-27 04:26:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 918323200. Throughput: 0: 9649.1, 1: 9709.9. Samples: 918290344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:26:46,062][104569] Avg episode reward: [(0, '8170.348'), (1, '9260.075')] [2023-12-27 04:26:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001791392_458661888.pth... [2023-12-27 04:26:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001790240_458366976.pth [2023-12-27 04:26:46,094][105620] Updated weights for policy 1, policy_version 1795331 (0.0010) [2023-12-27 04:26:46,123][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001795336_459669504.pth... [2023-12-27 04:26:46,128][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001794184_459374592.pth [2023-12-27 04:26:46,528][105692] Updated weights for policy 0, policy_version 1791398 (0.0005) [2023-12-27 04:26:46,572][105692] Updated weights for policy 0, policy_version 1791408 (0.0005) [2023-12-27 04:26:46,620][105692] Updated weights for policy 0, policy_version 1791418 (0.0007) [2023-12-27 04:26:46,784][105620] Updated weights for policy 1, policy_version 1795341 (0.0009) [2023-12-27 04:26:46,845][105620] Updated weights for policy 1, policy_version 1795351 (0.0009) [2023-12-27 04:26:46,907][105620] Updated weights for policy 1, policy_version 1795361 (0.0009) [2023-12-27 04:26:47,377][105692] Updated weights for policy 0, policy_version 1791428 (0.0009) [2023-12-27 04:26:47,431][105692] Updated weights for policy 0, policy_version 1791438 (0.0009) [2023-12-27 04:26:47,492][105692] Updated weights for policy 0, policy_version 1791448 (0.0008) [2023-12-27 04:26:47,570][105620] Updated weights for policy 1, policy_version 1795371 (0.0009) [2023-12-27 04:26:47,625][105620] Updated weights for policy 1, policy_version 1795381 (0.0011) [2023-12-27 04:26:47,683][105620] Updated weights for policy 1, policy_version 1795391 (0.0010) [2023-12-27 04:26:48,262][105692] Updated weights for policy 0, policy_version 1791458 (0.0009) [2023-12-27 04:26:48,315][105692] Updated weights for policy 0, policy_version 1791468 (0.0008) [2023-12-27 04:26:48,342][105620] Updated weights for policy 1, policy_version 1795401 (0.0007) [2023-12-27 04:26:48,374][105692] Updated weights for policy 0, policy_version 1791478 (0.0007) [2023-12-27 04:26:48,403][105620] Updated weights for policy 1, policy_version 1795411 (0.0010) [2023-12-27 04:26:48,433][105692] Updated weights for policy 0, policy_version 1791488 (0.0006) [2023-12-27 04:26:48,464][105620] Updated weights for policy 1, policy_version 1795421 (0.0010) [2023-12-27 04:26:48,524][105620] Updated weights for policy 1, policy_version 1795431 (0.0011) [2023-12-27 04:26:49,205][105692] Updated weights for policy 0, policy_version 1791498 (0.0008) [2023-12-27 04:26:49,258][105620] Updated weights for policy 1, policy_version 1795441 (0.0008) [2023-12-27 04:26:49,268][105692] Updated weights for policy 0, policy_version 1791508 (0.0008) [2023-12-27 04:26:49,325][105620] Updated weights for policy 1, policy_version 1795451 (0.0007) [2023-12-27 04:26:49,327][105692] Updated weights for policy 0, policy_version 1791518 (0.0008) [2023-12-27 04:26:49,388][105620] Updated weights for policy 1, policy_version 1795461 (0.0007) [2023-12-27 04:26:50,072][105620] Updated weights for policy 1, policy_version 1795471 (0.0009) [2023-12-27 04:26:50,074][105692] Updated weights for policy 0, policy_version 1791528 (0.0007) [2023-12-27 04:26:50,131][105692] Updated weights for policy 0, policy_version 1791538 (0.0006) [2023-12-27 04:26:50,132][105620] Updated weights for policy 1, policy_version 1795481 (0.0008) [2023-12-27 04:26:50,194][105620] Updated weights for policy 1, policy_version 1795491 (0.0008) [2023-12-27 04:26:50,198][105692] Updated weights for policy 0, policy_version 1791549 (0.0007) [2023-12-27 04:26:50,927][105692] Updated weights for policy 0, policy_version 1791559 (0.0009) [2023-12-27 04:26:50,964][105620] Updated weights for policy 1, policy_version 1795501 (0.0008) [2023-12-27 04:26:50,983][105692] Updated weights for policy 0, policy_version 1791569 (0.0008) [2023-12-27 04:26:51,029][105620] Updated weights for policy 1, policy_version 1795511 (0.0007) [2023-12-27 04:26:51,048][105692] Updated weights for policy 0, policy_version 1791579 (0.0007) [2023-12-27 04:26:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 918413312. Throughput: 0: 9702.1, 1: 9751.0. Samples: 918410548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:26:51,063][104569] Avg episode reward: [(0, '8533.631'), (1, '9262.039')] [2023-12-27 04:26:51,097][105620] Updated weights for policy 1, policy_version 1795521 (0.0007) [2023-12-27 04:26:51,692][105692] Updated weights for policy 0, policy_version 1791589 (0.0008) [2023-12-27 04:26:51,759][105692] Updated weights for policy 0, policy_version 1791599 (0.0008) [2023-12-27 04:26:51,828][105692] Updated weights for policy 0, policy_version 1791609 (0.0009) [2023-12-27 04:26:51,864][105620] Updated weights for policy 1, policy_version 1795531 (0.0008) [2023-12-27 04:26:51,927][105620] Updated weights for policy 1, policy_version 1795541 (0.0008) [2023-12-27 04:26:51,982][105620] Updated weights for policy 1, policy_version 1795551 (0.0009) [2023-12-27 04:26:52,588][105692] Updated weights for policy 0, policy_version 1791619 (0.0009) [2023-12-27 04:26:52,651][105692] Updated weights for policy 0, policy_version 1791629 (0.0009) [2023-12-27 04:26:52,706][105692] Updated weights for policy 0, policy_version 1791639 (0.0009) [2023-12-27 04:26:52,736][105620] Updated weights for policy 1, policy_version 1795561 (0.0009) [2023-12-27 04:26:52,794][105620] Updated weights for policy 1, policy_version 1795571 (0.0007) [2023-12-27 04:26:52,859][105620] Updated weights for policy 1, policy_version 1795581 (0.0009) [2023-12-27 04:26:52,918][105620] Updated weights for policy 1, policy_version 1795591 (0.0008) [2023-12-27 04:26:53,481][105692] Updated weights for policy 0, policy_version 1791649 (0.0009) [2023-12-27 04:26:53,534][105692] Updated weights for policy 0, policy_version 1791659 (0.0010) [2023-12-27 04:26:53,598][105692] Updated weights for policy 0, policy_version 1791669 (0.0007) [2023-12-27 04:26:53,617][105620] Updated weights for policy 1, policy_version 1795601 (0.0008) [2023-12-27 04:26:53,665][105692] Updated weights for policy 0, policy_version 1791679 (0.0010) [2023-12-27 04:26:53,683][105620] Updated weights for policy 1, policy_version 1795611 (0.0008) [2023-12-27 04:26:53,748][105620] Updated weights for policy 1, policy_version 1795621 (0.0008) [2023-12-27 04:26:54,293][105692] Updated weights for policy 0, policy_version 1791689 (0.0006) [2023-12-27 04:26:54,353][105692] Updated weights for policy 0, policy_version 1791699 (0.0007) [2023-12-27 04:26:54,416][105692] Updated weights for policy 0, policy_version 1791709 (0.0005) [2023-12-27 04:26:54,511][105620] Updated weights for policy 1, policy_version 1795631 (0.0006) [2023-12-27 04:26:54,566][105620] Updated weights for policy 1, policy_version 1795641 (0.0005) [2023-12-27 04:26:54,620][105620] Updated weights for policy 1, policy_version 1795651 (0.0008) [2023-12-27 04:26:55,008][105692] Updated weights for policy 0, policy_version 1791719 (0.0010) [2023-12-27 04:26:55,064][105692] Updated weights for policy 0, policy_version 1791729 (0.0010) [2023-12-27 04:26:55,115][105692] Updated weights for policy 0, policy_version 1791739 (0.0010) [2023-12-27 04:26:55,232][105620] Updated weights for policy 1, policy_version 1795661 (0.0010) [2023-12-27 04:26:55,280][105620] Updated weights for policy 1, policy_version 1795671 (0.0010) [2023-12-27 04:26:55,337][105620] Updated weights for policy 1, policy_version 1795681 (0.0010) [2023-12-27 04:26:55,803][105692] Updated weights for policy 0, policy_version 1791749 (0.0010) [2023-12-27 04:26:55,860][105692] Updated weights for policy 0, policy_version 1791759 (0.0010) [2023-12-27 04:26:55,916][105692] Updated weights for policy 0, policy_version 1791769 (0.0005) [2023-12-27 04:26:55,953][105620] Updated weights for policy 1, policy_version 1795691 (0.0009) [2023-12-27 04:26:56,012][105620] Updated weights for policy 1, policy_version 1795701 (0.0008) [2023-12-27 04:26:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 918519808. Throughput: 0: 9829.9, 1: 9810.3. Samples: 918529024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:26:56,063][104569] Avg episode reward: [(0, '8712.184'), (1, '9079.132')] [2023-12-27 04:26:56,074][105620] Updated weights for policy 1, policy_version 1795711 (0.0010) [2023-12-27 04:26:56,641][105692] Updated weights for policy 0, policy_version 1791779 (0.0009) [2023-12-27 04:26:56,659][105620] Updated weights for policy 1, policy_version 1795721 (0.0010) [2023-12-27 04:26:56,692][105692] Updated weights for policy 0, policy_version 1791789 (0.0010) [2023-12-27 04:26:56,715][105620] Updated weights for policy 1, policy_version 1795731 (0.0006) [2023-12-27 04:26:56,739][105692] Updated weights for policy 0, policy_version 1791799 (0.0008) [2023-12-27 04:26:56,774][105620] Updated weights for policy 1, policy_version 1795741 (0.0005) [2023-12-27 04:26:56,826][105620] Updated weights for policy 1, policy_version 1795751 (0.0005) [2023-12-27 04:26:57,301][105692] Updated weights for policy 0, policy_version 1791809 (0.0006) [2023-12-27 04:26:57,356][105692] Updated weights for policy 0, policy_version 1791819 (0.0010) [2023-12-27 04:26:57,363][105620] Updated weights for policy 1, policy_version 1795761 (0.0007) [2023-12-27 04:26:57,412][105692] Updated weights for policy 0, policy_version 1791829 (0.0010) [2023-12-27 04:26:57,414][105620] Updated weights for policy 1, policy_version 1795771 (0.0006) [2023-12-27 04:26:57,463][105620] Updated weights for policy 1, policy_version 1795781 (0.0008) [2023-12-27 04:26:57,472][105692] Updated weights for policy 0, policy_version 1791839 (0.0010) [2023-12-27 04:26:58,089][105620] Updated weights for policy 1, policy_version 1795791 (0.0007) [2023-12-27 04:26:58,145][105620] Updated weights for policy 1, policy_version 1795801 (0.0005) [2023-12-27 04:26:58,154][105692] Updated weights for policy 0, policy_version 1791849 (0.0010) [2023-12-27 04:26:58,209][105620] Updated weights for policy 1, policy_version 1795811 (0.0008) [2023-12-27 04:26:58,216][105692] Updated weights for policy 0, policy_version 1791859 (0.0010) [2023-12-27 04:26:58,272][105692] Updated weights for policy 0, policy_version 1791869 (0.0010) [2023-12-27 04:26:58,957][105620] Updated weights for policy 1, policy_version 1795821 (0.0008) [2023-12-27 04:26:59,029][105620] Updated weights for policy 1, policy_version 1795831 (0.0008) [2023-12-27 04:26:59,095][105620] Updated weights for policy 1, policy_version 1795841 (0.0008) [2023-12-27 04:26:59,100][105692] Updated weights for policy 0, policy_version 1791879 (0.0008) [2023-12-27 04:26:59,160][105692] Updated weights for policy 0, policy_version 1791889 (0.0007) [2023-12-27 04:26:59,225][105692] Updated weights for policy 0, policy_version 1791899 (0.0008) [2023-12-27 04:26:59,782][105620] Updated weights for policy 1, policy_version 1795851 (0.0007) [2023-12-27 04:26:59,843][105620] Updated weights for policy 1, policy_version 1795861 (0.0009) [2023-12-27 04:26:59,908][105620] Updated weights for policy 1, policy_version 1795871 (0.0007) [2023-12-27 04:26:59,930][105692] Updated weights for policy 0, policy_version 1791909 (0.0008) [2023-12-27 04:26:59,988][105692] Updated weights for policy 0, policy_version 1791919 (0.0007) [2023-12-27 04:27:00,033][105692] Updated weights for policy 0, policy_version 1791929 (0.0008) [2023-12-27 04:27:00,553][105620] Updated weights for policy 1, policy_version 1795881 (0.0008) [2023-12-27 04:27:00,607][105620] Updated weights for policy 1, policy_version 1795891 (0.0010) [2023-12-27 04:27:00,658][105620] Updated weights for policy 1, policy_version 1795901 (0.0010) [2023-12-27 04:27:00,716][105620] Updated weights for policy 1, policy_version 1795911 (0.0010) [2023-12-27 04:27:00,831][105692] Updated weights for policy 0, policy_version 1791939 (0.0008) [2023-12-27 04:27:00,887][105692] Updated weights for policy 0, policy_version 1791951 (0.0010) [2023-12-27 04:27:00,933][105692] Updated weights for policy 0, policy_version 1791961 (0.0010) [2023-12-27 04:27:01,062][104569] Fps is (10 sec: 21299.0, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 918626304. Throughput: 0: 9877.5, 1: 9919.9. Samples: 918592192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:27:01,063][104569] Avg episode reward: [(0, '8533.413'), (1, '9168.967')] [2023-12-27 04:27:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001795912_459816960.pth... [2023-12-27 04:27:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001791968_458809344.pth... [2023-12-27 04:27:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001794760_459522048.pth [2023-12-27 04:27:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001790784_458506240.pth [2023-12-27 04:27:01,425][105620] Updated weights for policy 1, policy_version 1795921 (0.0010) [2023-12-27 04:27:01,478][105620] Updated weights for policy 1, policy_version 1795931 (0.0011) [2023-12-27 04:27:01,540][105620] Updated weights for policy 1, policy_version 1795941 (0.0009) [2023-12-27 04:27:01,747][105692] Updated weights for policy 0, policy_version 1791971 (0.0010) [2023-12-27 04:27:01,813][105692] Updated weights for policy 0, policy_version 1791981 (0.0008) [2023-12-27 04:27:01,869][105692] Updated weights for policy 0, policy_version 1791991 (0.0010) [2023-12-27 04:27:02,290][105620] Updated weights for policy 1, policy_version 1795951 (0.0006) [2023-12-27 04:27:02,351][105620] Updated weights for policy 1, policy_version 1795961 (0.0007) [2023-12-27 04:27:02,419][105620] Updated weights for policy 1, policy_version 1795971 (0.0009) [2023-12-27 04:27:02,551][105692] Updated weights for policy 0, policy_version 1792001 (0.0009) [2023-12-27 04:27:02,611][105692] Updated weights for policy 0, policy_version 1792011 (0.0007) [2023-12-27 04:27:02,671][105692] Updated weights for policy 0, policy_version 1792021 (0.0006) [2023-12-27 04:27:02,732][105692] Updated weights for policy 0, policy_version 1792031 (0.0005) [2023-12-27 04:27:03,149][105620] Updated weights for policy 1, policy_version 1795981 (0.0007) [2023-12-27 04:27:03,201][105620] Updated weights for policy 1, policy_version 1795991 (0.0005) [2023-12-27 04:27:03,259][105620] Updated weights for policy 1, policy_version 1796001 (0.0007) [2023-12-27 04:27:03,272][105692] Updated weights for policy 0, policy_version 1792041 (0.0005) [2023-12-27 04:27:03,334][105692] Updated weights for policy 0, policy_version 1792051 (0.0006) [2023-12-27 04:27:03,390][105692] Updated weights for policy 0, policy_version 1792061 (0.0005) [2023-12-27 04:27:03,807][105620] Updated weights for policy 1, policy_version 1796011 (0.0007) [2023-12-27 04:27:03,867][105620] Updated weights for policy 1, policy_version 1796021 (0.0007) [2023-12-27 04:27:03,920][105620] Updated weights for policy 1, policy_version 1796031 (0.0008) [2023-12-27 04:27:03,992][105692] Updated weights for policy 0, policy_version 1792071 (0.0009) [2023-12-27 04:27:04,041][105692] Updated weights for policy 0, policy_version 1792081 (0.0010) [2023-12-27 04:27:04,090][105692] Updated weights for policy 0, policy_version 1792091 (0.0011) [2023-12-27 04:27:04,648][105620] Updated weights for policy 1, policy_version 1796041 (0.0008) [2023-12-27 04:27:04,719][105620] Updated weights for policy 1, policy_version 1796051 (0.0005) [2023-12-27 04:27:04,783][105620] Updated weights for policy 1, policy_version 1796061 (0.0005) [2023-12-27 04:27:04,829][105620] Updated weights for policy 1, policy_version 1796071 (0.0005) [2023-12-27 04:27:04,833][105692] Updated weights for policy 0, policy_version 1792101 (0.0008) [2023-12-27 04:27:04,884][105692] Updated weights for policy 0, policy_version 1792111 (0.0005) [2023-12-27 04:27:04,939][105692] Updated weights for policy 0, policy_version 1792121 (0.0005) [2023-12-27 04:27:05,456][105692] Updated weights for policy 0, policy_version 1792131 (0.0005) [2023-12-27 04:27:05,518][105692] Updated weights for policy 0, policy_version 1792141 (0.0006) [2023-12-27 04:27:05,572][105692] Updated weights for policy 0, policy_version 1792151 (0.0010) [2023-12-27 04:27:05,586][105620] Updated weights for policy 1, policy_version 1796081 (0.0006) [2023-12-27 04:27:05,633][105620] Updated weights for policy 1, policy_version 1796091 (0.0006) [2023-12-27 04:27:05,686][105620] Updated weights for policy 1, policy_version 1796101 (0.0005) [2023-12-27 04:27:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.7, 300 sec: 19466.4). Total num frames: 918724608. Throughput: 0: 9916.9, 1: 9969.1. Samples: 918711588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:27:06,063][104569] Avg episode reward: [(0, '8357.570'), (1, '9074.980')] [2023-12-27 04:27:06,158][105692] Updated weights for policy 0, policy_version 1792161 (0.0010) [2023-12-27 04:27:06,217][105692] Updated weights for policy 0, policy_version 1792171 (0.0011) [2023-12-27 04:27:06,283][105692] Updated weights for policy 0, policy_version 1792181 (0.0010) [2023-12-27 04:27:06,349][105692] Updated weights for policy 0, policy_version 1792191 (0.0011) [2023-12-27 04:27:06,440][105620] Updated weights for policy 1, policy_version 1796111 (0.0009) [2023-12-27 04:27:06,510][105620] Updated weights for policy 1, policy_version 1796121 (0.0010) [2023-12-27 04:27:06,575][105620] Updated weights for policy 1, policy_version 1796131 (0.0008) [2023-12-27 04:27:06,944][105692] Updated weights for policy 0, policy_version 1792201 (0.0006) [2023-12-27 04:27:07,010][105692] Updated weights for policy 0, policy_version 1792211 (0.0006) [2023-12-27 04:27:07,068][105692] Updated weights for policy 0, policy_version 1792221 (0.0006) [2023-12-27 04:27:07,354][105620] Updated weights for policy 1, policy_version 1796141 (0.0008) [2023-12-27 04:27:07,419][105620] Updated weights for policy 1, policy_version 1796151 (0.0005) [2023-12-27 04:27:07,491][105620] Updated weights for policy 1, policy_version 1796161 (0.0005) [2023-12-27 04:27:07,692][105692] Updated weights for policy 0, policy_version 1792231 (0.0009) [2023-12-27 04:27:07,753][105692] Updated weights for policy 0, policy_version 1792241 (0.0011) [2023-12-27 04:27:07,815][105692] Updated weights for policy 0, policy_version 1792251 (0.0006) [2023-12-27 04:27:08,031][105620] Updated weights for policy 1, policy_version 1796171 (0.0005) [2023-12-27 04:27:08,080][105620] Updated weights for policy 1, policy_version 1796181 (0.0008) [2023-12-27 04:27:08,142][105620] Updated weights for policy 1, policy_version 1796191 (0.0009) [2023-12-27 04:27:08,597][105692] Updated weights for policy 0, policy_version 1792261 (0.0008) [2023-12-27 04:27:08,662][105692] Updated weights for policy 0, policy_version 1792271 (0.0009) [2023-12-27 04:27:08,723][105692] Updated weights for policy 0, policy_version 1792281 (0.0009) [2023-12-27 04:27:08,774][105620] Updated weights for policy 1, policy_version 1796201 (0.0006) [2023-12-27 04:27:08,834][105620] Updated weights for policy 1, policy_version 1796211 (0.0009) [2023-12-27 04:27:08,881][105620] Updated weights for policy 1, policy_version 1796221 (0.0008) [2023-12-27 04:27:08,944][105620] Updated weights for policy 1, policy_version 1796231 (0.0009) [2023-12-27 04:27:09,503][105692] Updated weights for policy 0, policy_version 1792291 (0.0008) [2023-12-27 04:27:09,572][105692] Updated weights for policy 0, policy_version 1792301 (0.0009) [2023-12-27 04:27:09,632][105692] Updated weights for policy 0, policy_version 1792311 (0.0007) [2023-12-27 04:27:09,662][105620] Updated weights for policy 1, policy_version 1796241 (0.0010) [2023-12-27 04:27:09,725][105620] Updated weights for policy 1, policy_version 1796251 (0.0007) [2023-12-27 04:27:09,793][105620] Updated weights for policy 1, policy_version 1796261 (0.0010) [2023-12-27 04:27:10,298][105692] Updated weights for policy 0, policy_version 1792321 (0.0007) [2023-12-27 04:27:10,359][105692] Updated weights for policy 0, policy_version 1792331 (0.0010) [2023-12-27 04:27:10,420][105692] Updated weights for policy 0, policy_version 1792341 (0.0008) [2023-12-27 04:27:10,479][105692] Updated weights for policy 0, policy_version 1792351 (0.0008) [2023-12-27 04:27:10,532][105620] Updated weights for policy 1, policy_version 1796271 (0.0010) [2023-12-27 04:27:10,595][105620] Updated weights for policy 1, policy_version 1796281 (0.0011) [2023-12-27 04:27:10,657][105620] Updated weights for policy 1, policy_version 1796291 (0.0010) [2023-12-27 04:27:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 918822912. Throughput: 0: 10091.8, 1: 9919.4. Samples: 918832616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:27:11,062][104569] Avg episode reward: [(0, '8813.525'), (1, '9074.988')] [2023-12-27 04:27:11,307][105692] Updated weights for policy 0, policy_version 1792361 (0.0007) [2023-12-27 04:27:11,335][105620] Updated weights for policy 1, policy_version 1796301 (0.0012) [2023-12-27 04:27:11,372][105692] Updated weights for policy 0, policy_version 1792371 (0.0007) [2023-12-27 04:27:11,403][105620] Updated weights for policy 1, policy_version 1796311 (0.0010) [2023-12-27 04:27:11,436][105692] Updated weights for policy 0, policy_version 1792381 (0.0006) [2023-12-27 04:27:11,468][105620] Updated weights for policy 1, policy_version 1796322 (0.0007) [2023-12-27 04:27:12,160][105620] Updated weights for policy 1, policy_version 1796332 (0.0009) [2023-12-27 04:27:12,225][105620] Updated weights for policy 1, policy_version 1796342 (0.0008) [2023-12-27 04:27:12,239][105692] Updated weights for policy 0, policy_version 1792391 (0.0008) [2023-12-27 04:27:12,292][105620] Updated weights for policy 1, policy_version 1796352 (0.0008) [2023-12-27 04:27:12,302][105692] Updated weights for policy 0, policy_version 1792401 (0.0009) [2023-12-27 04:27:12,372][105692] Updated weights for policy 0, policy_version 1792411 (0.0008) [2023-12-27 04:27:12,994][105620] Updated weights for policy 1, policy_version 1796362 (0.0008) [2023-12-27 04:27:13,048][105620] Updated weights for policy 1, policy_version 1796372 (0.0008) [2023-12-27 04:27:13,102][105620] Updated weights for policy 1, policy_version 1796382 (0.0009) [2023-12-27 04:27:13,156][105620] Updated weights for policy 1, policy_version 1796392 (0.0009) [2023-12-27 04:27:13,159][105692] Updated weights for policy 0, policy_version 1792421 (0.0007) [2023-12-27 04:27:13,220][105692] Updated weights for policy 0, policy_version 1792431 (0.0007) [2023-12-27 04:27:13,278][105692] Updated weights for policy 0, policy_version 1792441 (0.0009) [2023-12-27 04:27:13,771][105620] Updated weights for policy 1, policy_version 1796402 (0.0005) [2023-12-27 04:27:13,829][105620] Updated weights for policy 1, policy_version 1796412 (0.0005) [2023-12-27 04:27:13,895][105620] Updated weights for policy 1, policy_version 1796422 (0.0005) [2023-12-27 04:27:14,011][105692] Updated weights for policy 0, policy_version 1792451 (0.0008) [2023-12-27 04:27:14,084][105692] Updated weights for policy 0, policy_version 1792461 (0.0005) [2023-12-27 04:27:14,155][105692] Updated weights for policy 0, policy_version 1792471 (0.0005) [2023-12-27 04:27:14,473][105620] Updated weights for policy 1, policy_version 1796432 (0.0006) [2023-12-27 04:27:14,542][105620] Updated weights for policy 1, policy_version 1796442 (0.0008) [2023-12-27 04:27:14,597][105620] Updated weights for policy 1, policy_version 1796452 (0.0008) [2023-12-27 04:27:14,710][105692] Updated weights for policy 0, policy_version 1792481 (0.0006) [2023-12-27 04:27:14,786][105692] Updated weights for policy 0, policy_version 1792491 (0.0011) [2023-12-27 04:27:14,846][105692] Updated weights for policy 0, policy_version 1792501 (0.0011) [2023-12-27 04:27:14,918][105692] Updated weights for policy 0, policy_version 1792511 (0.0011) [2023-12-27 04:27:15,187][105620] Updated weights for policy 1, policy_version 1796462 (0.0007) [2023-12-27 04:27:15,245][105620] Updated weights for policy 1, policy_version 1796472 (0.0007) [2023-12-27 04:27:15,306][105620] Updated weights for policy 1, policy_version 1796482 (0.0008) [2023-12-27 04:27:15,666][105692] Updated weights for policy 0, policy_version 1792521 (0.0011) [2023-12-27 04:27:15,721][105692] Updated weights for policy 0, policy_version 1792531 (0.0010) [2023-12-27 04:27:15,769][105692] Updated weights for policy 0, policy_version 1792541 (0.0010) [2023-12-27 04:27:16,005][105620] Updated weights for policy 1, policy_version 1796492 (0.0008) [2023-12-27 04:27:16,052][105620] Updated weights for policy 1, policy_version 1796502 (0.0007) [2023-12-27 04:27:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19933.9, 300 sec: 19494.2). Total num frames: 918921216. Throughput: 0: 9926.8, 1: 9911.9. Samples: 918889604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:27:16,062][104569] Avg episode reward: [(0, '8717.325'), (1, '9351.528')] [2023-12-27 04:27:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001792544_458956800.pth... [2023-12-27 04:27:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001791392_458661888.pth [2023-12-27 04:27:16,100][105620] Updated weights for policy 1, policy_version 1796512 (0.0008) [2023-12-27 04:27:16,135][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001796520_459972608.pth... [2023-12-27 04:27:16,138][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001795336_459669504.pth [2023-12-27 04:27:16,530][105692] Updated weights for policy 0, policy_version 1792551 (0.0010) [2023-12-27 04:27:16,588][105692] Updated weights for policy 0, policy_version 1792561 (0.0010) [2023-12-27 04:27:16,643][105692] Updated weights for policy 0, policy_version 1792571 (0.0010) [2023-12-27 04:27:16,751][105620] Updated weights for policy 1, policy_version 1796522 (0.0007) [2023-12-27 04:27:16,801][105620] Updated weights for policy 1, policy_version 1796532 (0.0007) [2023-12-27 04:27:16,846][105620] Updated weights for policy 1, policy_version 1796542 (0.0008) [2023-12-27 04:27:16,900][105620] Updated weights for policy 1, policy_version 1796552 (0.0008) [2023-12-27 04:27:17,392][105692] Updated weights for policy 0, policy_version 1792581 (0.0010) [2023-12-27 04:27:17,436][105692] Updated weights for policy 0, policy_version 1792591 (0.0010) [2023-12-27 04:27:17,487][105692] Updated weights for policy 0, policy_version 1792601 (0.0010) [2023-12-27 04:27:17,599][105620] Updated weights for policy 1, policy_version 1796562 (0.0008) [2023-12-27 04:27:17,657][105620] Updated weights for policy 1, policy_version 1796572 (0.0007) [2023-12-27 04:27:17,712][105620] Updated weights for policy 1, policy_version 1796582 (0.0007) [2023-12-27 04:27:18,232][105692] Updated weights for policy 0, policy_version 1792611 (0.0010) [2023-12-27 04:27:18,284][105692] Updated weights for policy 0, policy_version 1792621 (0.0010) [2023-12-27 04:27:18,345][105692] Updated weights for policy 0, policy_version 1792631 (0.0009) [2023-12-27 04:27:18,379][105620] Updated weights for policy 1, policy_version 1796592 (0.0007) [2023-12-27 04:27:18,432][105620] Updated weights for policy 1, policy_version 1796602 (0.0005) [2023-12-27 04:27:18,484][105620] Updated weights for policy 1, policy_version 1796612 (0.0005) [2023-12-27 04:27:19,097][105692] Updated weights for policy 0, policy_version 1792641 (0.0010) [2023-12-27 04:27:19,154][105692] Updated weights for policy 0, policy_version 1792651 (0.0010) [2023-12-27 04:27:19,191][105620] Updated weights for policy 1, policy_version 1796622 (0.0005) [2023-12-27 04:27:19,212][105692] Updated weights for policy 0, policy_version 1792661 (0.0010) [2023-12-27 04:27:19,253][105620] Updated weights for policy 1, policy_version 1796632 (0.0008) [2023-12-27 04:27:19,273][105692] Updated weights for policy 0, policy_version 1792671 (0.0008) [2023-12-27 04:27:19,320][105620] Updated weights for policy 1, policy_version 1796642 (0.0008) [2023-12-27 04:27:20,032][105692] Updated weights for policy 0, policy_version 1792681 (0.0009) [2023-12-27 04:27:20,083][105620] Updated weights for policy 1, policy_version 1796652 (0.0008) [2023-12-27 04:27:20,092][105692] Updated weights for policy 0, policy_version 1792691 (0.0010) [2023-12-27 04:27:20,145][105620] Updated weights for policy 1, policy_version 1796662 (0.0011) [2023-12-27 04:27:20,148][105692] Updated weights for policy 0, policy_version 1792701 (0.0011) [2023-12-27 04:27:20,211][105620] Updated weights for policy 1, policy_version 1796672 (0.0009) [2023-12-27 04:27:20,879][105692] Updated weights for policy 0, policy_version 1792711 (0.0008) [2023-12-27 04:27:20,935][105620] Updated weights for policy 1, policy_version 1796682 (0.0009) [2023-12-27 04:27:20,942][105692] Updated weights for policy 0, policy_version 1792721 (0.0011) [2023-12-27 04:27:20,989][105620] Updated weights for policy 1, policy_version 1796692 (0.0009) [2023-12-27 04:27:21,008][105692] Updated weights for policy 0, policy_version 1792731 (0.0009) [2023-12-27 04:27:21,052][105620] Updated weights for policy 1, policy_version 1796702 (0.0008) [2023-12-27 04:27:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19933.8, 300 sec: 19494.2). Total num frames: 919019520. Throughput: 0: 9876.4, 1: 9985.2. Samples: 919009568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:27:21,063][104569] Avg episode reward: [(0, '8898.225'), (1, '9259.398')] [2023-12-27 04:27:21,117][105620] Updated weights for policy 1, policy_version 1796712 (0.0009) [2023-12-27 04:27:21,652][105692] Updated weights for policy 0, policy_version 1792741 (0.0007) [2023-12-27 04:27:21,719][105692] Updated weights for policy 0, policy_version 1792751 (0.0009) [2023-12-27 04:27:21,781][105692] Updated weights for policy 0, policy_version 1792761 (0.0008) [2023-12-27 04:27:21,954][105620] Updated weights for policy 1, policy_version 1796722 (0.0008) [2023-12-27 04:27:22,013][105620] Updated weights for policy 1, policy_version 1796732 (0.0009) [2023-12-27 04:27:22,068][105620] Updated weights for policy 1, policy_version 1796742 (0.0008) [2023-12-27 04:27:22,529][105692] Updated weights for policy 0, policy_version 1792771 (0.0011) [2023-12-27 04:27:22,595][105692] Updated weights for policy 0, policy_version 1792781 (0.0010) [2023-12-27 04:27:22,653][105692] Updated weights for policy 0, policy_version 1792791 (0.0011) [2023-12-27 04:27:22,856][105620] Updated weights for policy 1, policy_version 1796752 (0.0008) [2023-12-27 04:27:22,916][105620] Updated weights for policy 1, policy_version 1796762 (0.0008) [2023-12-27 04:27:22,975][105620] Updated weights for policy 1, policy_version 1796772 (0.0008) [2023-12-27 04:27:23,399][105692] Updated weights for policy 0, policy_version 1792801 (0.0011) [2023-12-27 04:27:23,456][105692] Updated weights for policy 0, policy_version 1792811 (0.0011) [2023-12-27 04:27:23,509][105692] Updated weights for policy 0, policy_version 1792821 (0.0011) [2023-12-27 04:27:23,544][105620] Updated weights for policy 1, policy_version 1796782 (0.0007) [2023-12-27 04:27:23,562][105692] Updated weights for policy 0, policy_version 1792831 (0.0011) [2023-12-27 04:27:23,615][105620] Updated weights for policy 1, policy_version 1796792 (0.0007) [2023-12-27 04:27:23,682][105620] Updated weights for policy 1, policy_version 1796802 (0.0007) [2023-12-27 04:27:24,365][105692] Updated weights for policy 0, policy_version 1792841 (0.0011) [2023-12-27 04:27:24,423][105692] Updated weights for policy 0, policy_version 1792851 (0.0010) [2023-12-27 04:27:24,437][105620] Updated weights for policy 1, policy_version 1796812 (0.0006) [2023-12-27 04:27:24,474][105692] Updated weights for policy 0, policy_version 1792861 (0.0010) [2023-12-27 04:27:24,496][105620] Updated weights for policy 1, policy_version 1796822 (0.0005) [2023-12-27 04:27:24,554][105620] Updated weights for policy 1, policy_version 1796832 (0.0006) [2023-12-27 04:27:25,110][105620] Updated weights for policy 1, policy_version 1796842 (0.0005) [2023-12-27 04:27:25,162][105620] Updated weights for policy 1, policy_version 1796852 (0.0005) [2023-12-27 04:27:25,220][105620] Updated weights for policy 1, policy_version 1796862 (0.0006) [2023-12-27 04:27:25,230][105692] Updated weights for policy 0, policy_version 1792871 (0.0011) [2023-12-27 04:27:25,276][105620] Updated weights for policy 1, policy_version 1796872 (0.0008) [2023-12-27 04:27:25,288][105692] Updated weights for policy 0, policy_version 1792881 (0.0012) [2023-12-27 04:27:25,358][105692] Updated weights for policy 0, policy_version 1792891 (0.0011) [2023-12-27 04:27:25,970][105620] Updated weights for policy 1, policy_version 1796882 (0.0009) [2023-12-27 04:27:26,021][105620] Updated weights for policy 1, policy_version 1796892 (0.0010) [2023-12-27 04:27:26,038][105692] Updated weights for policy 0, policy_version 1792901 (0.0008) [2023-12-27 04:27:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 919109632. Throughput: 0: 9775.2, 1: 10047.4. Samples: 919125156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:27:26,062][104569] Avg episode reward: [(0, '8896.938'), (1, '9170.169')] [2023-12-27 04:27:26,077][105620] Updated weights for policy 1, policy_version 1796902 (0.0008) [2023-12-27 04:27:26,088][105692] Updated weights for policy 0, policy_version 1792911 (0.0005) [2023-12-27 04:27:26,137][105692] Updated weights for policy 0, policy_version 1792921 (0.0005) [2023-12-27 04:27:26,688][105692] Updated weights for policy 0, policy_version 1792931 (0.0006) [2023-12-27 04:27:26,750][105692] Updated weights for policy 0, policy_version 1792941 (0.0007) [2023-12-27 04:27:26,797][105692] Updated weights for policy 0, policy_version 1792951 (0.0009) [2023-12-27 04:27:26,876][105620] Updated weights for policy 1, policy_version 1796912 (0.0008) [2023-12-27 04:27:26,950][105620] Updated weights for policy 1, policy_version 1796922 (0.0009) [2023-12-27 04:27:27,017][105620] Updated weights for policy 1, policy_version 1796932 (0.0010) [2023-12-27 04:27:27,426][105692] Updated weights for policy 0, policy_version 1792961 (0.0006) [2023-12-27 04:27:27,477][105692] Updated weights for policy 0, policy_version 1792971 (0.0010) [2023-12-27 04:27:27,529][105692] Updated weights for policy 0, policy_version 1792982 (0.0010) [2023-12-27 04:27:27,578][105692] Updated weights for policy 0, policy_version 1792992 (0.0009) [2023-12-27 04:27:27,599][105620] Updated weights for policy 1, policy_version 1796942 (0.0010) [2023-12-27 04:27:27,649][105620] Updated weights for policy 1, policy_version 1796952 (0.0008) [2023-12-27 04:27:27,699][105620] Updated weights for policy 1, policy_version 1796962 (0.0009) [2023-12-27 04:27:28,306][105692] Updated weights for policy 0, policy_version 1793002 (0.0005) [2023-12-27 04:27:28,369][105692] Updated weights for policy 0, policy_version 1793012 (0.0007) [2023-12-27 04:27:28,429][105692] Updated weights for policy 0, policy_version 1793022 (0.0008) [2023-12-27 04:27:28,504][105620] Updated weights for policy 1, policy_version 1796972 (0.0009) [2023-12-27 04:27:28,564][105620] Updated weights for policy 1, policy_version 1796982 (0.0008) [2023-12-27 04:27:28,621][105620] Updated weights for policy 1, policy_version 1796992 (0.0008) [2023-12-27 04:27:29,104][105692] Updated weights for policy 0, policy_version 1793032 (0.0010) [2023-12-27 04:27:29,162][105692] Updated weights for policy 0, policy_version 1793042 (0.0010) [2023-12-27 04:27:29,227][105692] Updated weights for policy 0, policy_version 1793052 (0.0009) [2023-12-27 04:27:29,397][105620] Updated weights for policy 1, policy_version 1797002 (0.0009) [2023-12-27 04:27:29,452][105620] Updated weights for policy 1, policy_version 1797012 (0.0008) [2023-12-27 04:27:29,507][105620] Updated weights for policy 1, policy_version 1797022 (0.0007) [2023-12-27 04:27:29,558][105620] Updated weights for policy 1, policy_version 1797032 (0.0008) [2023-12-27 04:27:29,952][105692] Updated weights for policy 0, policy_version 1793062 (0.0009) [2023-12-27 04:27:30,005][105692] Updated weights for policy 0, policy_version 1793072 (0.0010) [2023-12-27 04:27:30,060][105692] Updated weights for policy 0, policy_version 1793082 (0.0010) [2023-12-27 04:27:30,332][105620] Updated weights for policy 1, policy_version 1797042 (0.0007) [2023-12-27 04:27:30,378][105620] Updated weights for policy 1, policy_version 1797052 (0.0007) [2023-12-27 04:27:30,430][105620] Updated weights for policy 1, policy_version 1797062 (0.0007) [2023-12-27 04:27:30,804][105692] Updated weights for policy 0, policy_version 1793092 (0.0011) [2023-12-27 04:27:30,852][105692] Updated weights for policy 0, policy_version 1793102 (0.0010) [2023-12-27 04:27:30,896][105692] Updated weights for policy 0, policy_version 1793112 (0.0010) [2023-12-27 04:27:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 919216128. Throughput: 0: 9872.3, 1: 10030.5. Samples: 919185968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:27:31,062][104569] Avg episode reward: [(0, '8623.399'), (1, '9078.248')] [2023-12-27 04:27:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001793120_459104256.pth... [2023-12-27 04:27:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001797064_460111872.pth... [2023-12-27 04:27:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001795912_459816960.pth [2023-12-27 04:27:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001791968_458809344.pth [2023-12-27 04:27:31,188][105620] Updated weights for policy 1, policy_version 1797072 (0.0008) [2023-12-27 04:27:31,246][105620] Updated weights for policy 1, policy_version 1797082 (0.0007) [2023-12-27 04:27:31,309][105620] Updated weights for policy 1, policy_version 1797092 (0.0006) [2023-12-27 04:27:31,676][105692] Updated weights for policy 0, policy_version 1793122 (0.0009) [2023-12-27 04:27:31,748][105692] Updated weights for policy 0, policy_version 1793132 (0.0006) [2023-12-27 04:27:31,809][105692] Updated weights for policy 0, policy_version 1793142 (0.0008) [2023-12-27 04:27:31,868][105692] Updated weights for policy 0, policy_version 1793152 (0.0010) [2023-12-27 04:27:31,977][105620] Updated weights for policy 1, policy_version 1797102 (0.0006) [2023-12-27 04:27:32,036][105620] Updated weights for policy 1, policy_version 1797112 (0.0005) [2023-12-27 04:27:32,101][105620] Updated weights for policy 1, policy_version 1797122 (0.0005) [2023-12-27 04:27:32,643][105692] Updated weights for policy 0, policy_version 1793162 (0.0010) [2023-12-27 04:27:32,707][105620] Updated weights for policy 1, policy_version 1797132 (0.0006) [2023-12-27 04:27:32,709][105692] Updated weights for policy 0, policy_version 1793172 (0.0011) [2023-12-27 04:27:32,759][105620] Updated weights for policy 1, policy_version 1797142 (0.0005) [2023-12-27 04:27:32,761][105692] Updated weights for policy 0, policy_version 1793182 (0.0010) [2023-12-27 04:27:32,820][105620] Updated weights for policy 1, policy_version 1797152 (0.0008) [2023-12-27 04:27:33,376][105692] Updated weights for policy 0, policy_version 1793192 (0.0010) [2023-12-27 04:27:33,443][105692] Updated weights for policy 0, policy_version 1793202 (0.0010) [2023-12-27 04:27:33,500][105692] Updated weights for policy 0, policy_version 1793212 (0.0010) [2023-12-27 04:27:33,570][105620] Updated weights for policy 1, policy_version 1797162 (0.0010) [2023-12-27 04:27:33,624][105620] Updated weights for policy 1, policy_version 1797172 (0.0010) [2023-12-27 04:27:33,681][105620] Updated weights for policy 1, policy_version 1797183 (0.0010) [2023-12-27 04:27:34,137][105692] Updated weights for policy 0, policy_version 1793222 (0.0008) [2023-12-27 04:27:34,196][105692] Updated weights for policy 0, policy_version 1793232 (0.0009) [2023-12-27 04:27:34,256][105692] Updated weights for policy 0, policy_version 1793242 (0.0009) [2023-12-27 04:27:34,286][105585] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000002 [2023-12-27 04:27:34,482][105620] Updated weights for policy 1, policy_version 1797193 (0.0008) [2023-12-27 04:27:34,540][105620] Updated weights for policy 1, policy_version 1797203 (0.0009) [2023-12-27 04:27:34,601][105620] Updated weights for policy 1, policy_version 1797213 (0.0008) [2023-12-27 04:27:34,659][105620] Updated weights for policy 1, policy_version 1797223 (0.0010) [2023-12-27 04:27:34,924][105692] Updated weights for policy 0, policy_version 1793252 (0.0009) [2023-12-27 04:27:34,973][105692] Updated weights for policy 0, policy_version 1793262 (0.0007) [2023-12-27 04:27:35,024][105692] Updated weights for policy 0, policy_version 1793272 (0.0005) [2023-12-27 04:27:35,412][105620] Updated weights for policy 1, policy_version 1797233 (0.0006) [2023-12-27 04:27:35,461][105620] Updated weights for policy 1, policy_version 1797243 (0.0005) [2023-12-27 04:27:35,522][105620] Updated weights for policy 1, policy_version 1797253 (0.0007) [2023-12-27 04:27:35,626][105692] Updated weights for policy 0, policy_version 1793282 (0.0006) [2023-12-27 04:27:35,675][105692] Updated weights for policy 0, policy_version 1793292 (0.0008) [2023-12-27 04:27:35,735][105692] Updated weights for policy 0, policy_version 1793302 (0.0008) [2023-12-27 04:27:35,792][105692] Updated weights for policy 0, policy_version 1793312 (0.0009) [2023-12-27 04:27:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 919314432. Throughput: 0: 9863.4, 1: 9936.3. Samples: 919301536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:27:36,063][104569] Avg episode reward: [(0, '8257.414'), (1, '9075.068')] [2023-12-27 04:27:36,225][105620] Updated weights for policy 1, policy_version 1797263 (0.0011) [2023-12-27 04:27:36,290][105620] Updated weights for policy 1, policy_version 1797273 (0.0008) [2023-12-27 04:27:36,352][105620] Updated weights for policy 1, policy_version 1797283 (0.0010) [2023-12-27 04:27:36,482][105692] Updated weights for policy 0, policy_version 1793322 (0.0009) [2023-12-27 04:27:36,542][105692] Updated weights for policy 0, policy_version 1793332 (0.0009) [2023-12-27 04:27:36,611][105692] Updated weights for policy 0, policy_version 1793342 (0.0008) [2023-12-27 04:27:37,115][105620] Updated weights for policy 1, policy_version 1797293 (0.0010) [2023-12-27 04:27:37,169][105620] Updated weights for policy 1, policy_version 1797303 (0.0009) [2023-12-27 04:27:37,230][105620] Updated weights for policy 1, policy_version 1797313 (0.0009) [2023-12-27 04:27:37,303][105692] Updated weights for policy 0, policy_version 1793352 (0.0009) [2023-12-27 04:27:37,366][105692] Updated weights for policy 0, policy_version 1793362 (0.0011) [2023-12-27 04:27:37,428][105692] Updated weights for policy 0, policy_version 1793372 (0.0010) [2023-12-27 04:27:37,928][105620] Updated weights for policy 1, policy_version 1797323 (0.0008) [2023-12-27 04:27:38,001][105620] Updated weights for policy 1, policy_version 1797333 (0.0009) [2023-12-27 04:27:38,060][105620] Updated weights for policy 1, policy_version 1797343 (0.0011) [2023-12-27 04:27:38,184][105692] Updated weights for policy 0, policy_version 1793382 (0.0010) [2023-12-27 04:27:38,243][105692] Updated weights for policy 0, policy_version 1793392 (0.0010) [2023-12-27 04:27:38,302][105692] Updated weights for policy 0, policy_version 1793402 (0.0010) [2023-12-27 04:27:38,687][105620] Updated weights for policy 1, policy_version 1797353 (0.0010) [2023-12-27 04:27:38,752][105620] Updated weights for policy 1, policy_version 1797363 (0.0008) [2023-12-27 04:27:38,815][105620] Updated weights for policy 1, policy_version 1797373 (0.0008) [2023-12-27 04:27:38,871][105620] Updated weights for policy 1, policy_version 1797383 (0.0008) [2023-12-27 04:27:39,060][105692] Updated weights for policy 0, policy_version 1793412 (0.0011) [2023-12-27 04:27:39,121][105692] Updated weights for policy 0, policy_version 1793422 (0.0010) [2023-12-27 04:27:39,180][105692] Updated weights for policy 0, policy_version 1793432 (0.0010) [2023-12-27 04:27:39,620][105620] Updated weights for policy 1, policy_version 1797393 (0.0009) [2023-12-27 04:27:39,675][105620] Updated weights for policy 1, policy_version 1797403 (0.0009) [2023-12-27 04:27:39,730][105620] Updated weights for policy 1, policy_version 1797413 (0.0009) [2023-12-27 04:27:39,958][105692] Updated weights for policy 0, policy_version 1793442 (0.0011) [2023-12-27 04:27:40,026][105692] Updated weights for policy 0, policy_version 1793452 (0.0011) [2023-12-27 04:27:40,085][105692] Updated weights for policy 0, policy_version 1793462 (0.0010) [2023-12-27 04:27:40,148][105692] Updated weights for policy 0, policy_version 1793472 (0.0010) [2023-12-27 04:27:40,455][105620] Updated weights for policy 1, policy_version 1797423 (0.0007) [2023-12-27 04:27:40,510][105620] Updated weights for policy 1, policy_version 1797433 (0.0005) [2023-12-27 04:27:40,559][105620] Updated weights for policy 1, policy_version 1797443 (0.0005) [2023-12-27 04:27:40,904][105692] Updated weights for policy 0, policy_version 1793482 (0.0007) [2023-12-27 04:27:40,955][105692] Updated weights for policy 0, policy_version 1793492 (0.0005) [2023-12-27 04:27:41,015][105692] Updated weights for policy 0, policy_version 1793502 (0.0006) [2023-12-27 04:27:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 919412736. Throughput: 0: 9837.7, 1: 9961.3. Samples: 919419976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:27:41,062][104569] Avg episode reward: [(0, '8350.243'), (1, '9166.947')] [2023-12-27 04:27:41,241][105620] Updated weights for policy 1, policy_version 1797453 (0.0007) [2023-12-27 04:27:41,308][105620] Updated weights for policy 1, policy_version 1797463 (0.0009) [2023-12-27 04:27:41,382][105620] Updated weights for policy 1, policy_version 1797473 (0.0009) [2023-12-27 04:27:41,690][105692] Updated weights for policy 0, policy_version 1793512 (0.0008) [2023-12-27 04:27:41,763][105692] Updated weights for policy 0, policy_version 1793522 (0.0009) [2023-12-27 04:27:41,827][105692] Updated weights for policy 0, policy_version 1793532 (0.0010) [2023-12-27 04:27:42,079][105620] Updated weights for policy 1, policy_version 1797483 (0.0009) [2023-12-27 04:27:42,142][105620] Updated weights for policy 1, policy_version 1797493 (0.0007) [2023-12-27 04:27:42,204][105620] Updated weights for policy 1, policy_version 1797503 (0.0005) [2023-12-27 04:27:42,516][105692] Updated weights for policy 0, policy_version 1793542 (0.0010) [2023-12-27 04:27:42,568][105692] Updated weights for policy 0, policy_version 1793552 (0.0010) [2023-12-27 04:27:42,623][105692] Updated weights for policy 0, policy_version 1793562 (0.0010) [2023-12-27 04:27:42,860][105620] Updated weights for policy 1, policy_version 1797513 (0.0006) [2023-12-27 04:27:42,913][105620] Updated weights for policy 1, policy_version 1797523 (0.0005) [2023-12-27 04:27:42,969][105620] Updated weights for policy 1, policy_version 1797533 (0.0006) [2023-12-27 04:27:43,025][105620] Updated weights for policy 1, policy_version 1797543 (0.0008) [2023-12-27 04:27:43,304][105692] Updated weights for policy 0, policy_version 1793572 (0.0009) [2023-12-27 04:27:43,370][105692] Updated weights for policy 0, policy_version 1793582 (0.0010) [2023-12-27 04:27:43,438][105692] Updated weights for policy 0, policy_version 1793592 (0.0010) [2023-12-27 04:27:43,685][105620] Updated weights for policy 1, policy_version 1797553 (0.0005) [2023-12-27 04:27:43,730][105620] Updated weights for policy 1, policy_version 1797563 (0.0005) [2023-12-27 04:27:43,788][105620] Updated weights for policy 1, policy_version 1797573 (0.0005) [2023-12-27 04:27:44,147][105692] Updated weights for policy 0, policy_version 1793602 (0.0010) [2023-12-27 04:27:44,216][105692] Updated weights for policy 0, policy_version 1793612 (0.0011) [2023-12-27 04:27:44,279][105692] Updated weights for policy 0, policy_version 1793622 (0.0011) [2023-12-27 04:27:44,345][105692] Updated weights for policy 0, policy_version 1793632 (0.0011) [2023-12-27 04:27:44,374][105620] Updated weights for policy 1, policy_version 1797583 (0.0006) [2023-12-27 04:27:44,431][105620] Updated weights for policy 1, policy_version 1797593 (0.0009) [2023-12-27 04:27:44,494][105620] Updated weights for policy 1, policy_version 1797603 (0.0005) [2023-12-27 04:27:45,063][105620] Updated weights for policy 1, policy_version 1797613 (0.0006) [2023-12-27 04:27:45,081][105692] Updated weights for policy 0, policy_version 1793642 (0.0011) [2023-12-27 04:27:45,126][105620] Updated weights for policy 1, policy_version 1797623 (0.0008) [2023-12-27 04:27:45,143][105692] Updated weights for policy 0, policy_version 1793652 (0.0009) [2023-12-27 04:27:45,184][105620] Updated weights for policy 1, policy_version 1797633 (0.0010) [2023-12-27 04:27:45,206][105692] Updated weights for policy 0, policy_version 1793662 (0.0011) [2023-12-27 04:27:45,905][105620] Updated weights for policy 1, policy_version 1797643 (0.0006) [2023-12-27 04:27:45,952][105692] Updated weights for policy 0, policy_version 1793672 (0.0010) [2023-12-27 04:27:45,964][105620] Updated weights for policy 1, policy_version 1797653 (0.0007) [2023-12-27 04:27:46,004][105692] Updated weights for policy 0, policy_version 1793682 (0.0010) [2023-12-27 04:27:46,014][105620] Updated weights for policy 1, policy_version 1797663 (0.0008) [2023-12-27 04:27:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 919502848. Throughput: 0: 9822.6, 1: 9912.8. Samples: 919480284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:27:46,062][105692] Updated weights for policy 0, policy_version 1793692 (0.0010) [2023-12-27 04:27:46,062][104569] Avg episode reward: [(0, '8808.729'), (1, '9259.369')] [2023-12-27 04:27:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001797672_460267520.pth... [2023-12-27 04:27:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001796520_459972608.pth [2023-12-27 04:27:46,086][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001793696_459251712.pth... [2023-12-27 04:27:46,090][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001792544_458956800.pth [2023-12-27 04:27:46,796][105620] Updated weights for policy 1, policy_version 1797673 (0.0005) [2023-12-27 04:27:46,813][105692] Updated weights for policy 0, policy_version 1793702 (0.0010) [2023-12-27 04:27:46,858][105620] Updated weights for policy 1, policy_version 1797683 (0.0006) [2023-12-27 04:27:46,867][105692] Updated weights for policy 0, policy_version 1793712 (0.0010) [2023-12-27 04:27:46,916][105620] Updated weights for policy 1, policy_version 1797693 (0.0005) [2023-12-27 04:27:46,923][105692] Updated weights for policy 0, policy_version 1793722 (0.0010) [2023-12-27 04:27:46,981][105620] Updated weights for policy 1, policy_version 1797703 (0.0009) [2023-12-27 04:27:47,475][105692] Updated weights for policy 0, policy_version 1793732 (0.0007) [2023-12-27 04:27:47,529][105692] Updated weights for policy 0, policy_version 1793742 (0.0009) [2023-12-27 04:27:47,586][105692] Updated weights for policy 0, policy_version 1793752 (0.0009) [2023-12-27 04:27:47,816][105620] Updated weights for policy 1, policy_version 1797713 (0.0008) [2023-12-27 04:27:47,882][105620] Updated weights for policy 1, policy_version 1797723 (0.0009) [2023-12-27 04:27:47,940][105620] Updated weights for policy 1, policy_version 1797733 (0.0009) [2023-12-27 04:27:48,318][105692] Updated weights for policy 0, policy_version 1793762 (0.0009) [2023-12-27 04:27:48,375][105692] Updated weights for policy 0, policy_version 1793772 (0.0007) [2023-12-27 04:27:48,426][105692] Updated weights for policy 0, policy_version 1793782 (0.0008) [2023-12-27 04:27:48,486][105692] Updated weights for policy 0, policy_version 1793792 (0.0010) [2023-12-27 04:27:48,670][105620] Updated weights for policy 1, policy_version 1797743 (0.0009) [2023-12-27 04:27:48,732][105620] Updated weights for policy 1, policy_version 1797753 (0.0009) [2023-12-27 04:27:48,805][105620] Updated weights for policy 1, policy_version 1797763 (0.0010) [2023-12-27 04:27:49,206][105692] Updated weights for policy 0, policy_version 1793802 (0.0009) [2023-12-27 04:27:49,269][105692] Updated weights for policy 0, policy_version 1793812 (0.0008) [2023-12-27 04:27:49,331][105692] Updated weights for policy 0, policy_version 1793822 (0.0008) [2023-12-27 04:27:49,559][105620] Updated weights for policy 1, policy_version 1797773 (0.0009) [2023-12-27 04:27:49,618][105620] Updated weights for policy 1, policy_version 1797783 (0.0009) [2023-12-27 04:27:49,673][105620] Updated weights for policy 1, policy_version 1797793 (0.0010) [2023-12-27 04:27:50,049][105692] Updated weights for policy 0, policy_version 1793832 (0.0009) [2023-12-27 04:27:50,117][105692] Updated weights for policy 0, policy_version 1793842 (0.0008) [2023-12-27 04:27:50,175][105692] Updated weights for policy 0, policy_version 1793852 (0.0009) [2023-12-27 04:27:50,443][105620] Updated weights for policy 1, policy_version 1797803 (0.0008) [2023-12-27 04:27:50,505][105620] Updated weights for policy 1, policy_version 1797813 (0.0008) [2023-12-27 04:27:50,570][105620] Updated weights for policy 1, policy_version 1797823 (0.0009) [2023-12-27 04:27:50,895][105692] Updated weights for policy 0, policy_version 1793862 (0.0008) [2023-12-27 04:27:50,947][105692] Updated weights for policy 0, policy_version 1793872 (0.0009) [2023-12-27 04:27:50,994][105692] Updated weights for policy 0, policy_version 1793882 (0.0009) [2023-12-27 04:27:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19522.0). Total num frames: 919609344. Throughput: 0: 9802.6, 1: 9833.5. Samples: 919595212. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:27:51,062][104569] Avg episode reward: [(0, '8902.887'), (1, '9351.449')] [2023-12-27 04:27:51,304][105620] Updated weights for policy 1, policy_version 1797833 (0.0008) [2023-12-27 04:27:51,374][105620] Updated weights for policy 1, policy_version 1797843 (0.0009) [2023-12-27 04:27:51,432][105620] Updated weights for policy 1, policy_version 1797853 (0.0008) [2023-12-27 04:27:51,497][105620] Updated weights for policy 1, policy_version 1797863 (0.0008) [2023-12-27 04:27:51,805][105692] Updated weights for policy 0, policy_version 1793892 (0.0009) [2023-12-27 04:27:51,864][105692] Updated weights for policy 0, policy_version 1793902 (0.0010) [2023-12-27 04:27:51,929][105692] Updated weights for policy 0, policy_version 1793912 (0.0009) [2023-12-27 04:27:52,238][105620] Updated weights for policy 1, policy_version 1797873 (0.0009) [2023-12-27 04:27:52,295][105620] Updated weights for policy 1, policy_version 1797883 (0.0009) [2023-12-27 04:27:52,355][105620] Updated weights for policy 1, policy_version 1797893 (0.0008) [2023-12-27 04:27:52,714][105692] Updated weights for policy 0, policy_version 1793922 (0.0009) [2023-12-27 04:27:52,765][105692] Updated weights for policy 0, policy_version 1793932 (0.0009) [2023-12-27 04:27:52,827][105692] Updated weights for policy 0, policy_version 1793942 (0.0009) [2023-12-27 04:27:52,898][105692] Updated weights for policy 0, policy_version 1793952 (0.0009) [2023-12-27 04:27:53,146][105620] Updated weights for policy 1, policy_version 1797903 (0.0007) [2023-12-27 04:27:53,214][105620] Updated weights for policy 1, policy_version 1797913 (0.0008) [2023-12-27 04:27:53,278][105620] Updated weights for policy 1, policy_version 1797923 (0.0008) [2023-12-27 04:27:53,600][105692] Updated weights for policy 0, policy_version 1793962 (0.0005) [2023-12-27 04:27:53,661][105692] Updated weights for policy 0, policy_version 1793972 (0.0005) [2023-12-27 04:27:53,720][105692] Updated weights for policy 0, policy_version 1793982 (0.0005) [2023-12-27 04:27:54,047][105620] Updated weights for policy 1, policy_version 1797933 (0.0009) [2023-12-27 04:27:54,107][105620] Updated weights for policy 1, policy_version 1797943 (0.0011) [2023-12-27 04:27:54,167][105620] Updated weights for policy 1, policy_version 1797953 (0.0010) [2023-12-27 04:27:54,297][105692] Updated weights for policy 0, policy_version 1793992 (0.0005) [2023-12-27 04:27:54,353][105692] Updated weights for policy 0, policy_version 1794002 (0.0006) [2023-12-27 04:27:54,414][105692] Updated weights for policy 0, policy_version 1794012 (0.0007) [2023-12-27 04:27:54,913][105620] Updated weights for policy 1, policy_version 1797963 (0.0009) [2023-12-27 04:27:54,974][105620] Updated weights for policy 1, policy_version 1797973 (0.0007) [2023-12-27 04:27:55,029][105620] Updated weights for policy 1, policy_version 1797983 (0.0009) [2023-12-27 04:27:55,121][105692] Updated weights for policy 0, policy_version 1794022 (0.0009) [2023-12-27 04:27:55,169][105692] Updated weights for policy 0, policy_version 1794032 (0.0009) [2023-12-27 04:27:55,220][105692] Updated weights for policy 0, policy_version 1794042 (0.0009) [2023-12-27 04:27:55,701][105620] Updated weights for policy 1, policy_version 1797993 (0.0009) [2023-12-27 04:27:55,770][105620] Updated weights for policy 1, policy_version 1798003 (0.0006) [2023-12-27 04:27:55,829][105620] Updated weights for policy 1, policy_version 1798013 (0.0007) [2023-12-27 04:27:55,879][105620] Updated weights for policy 1, policy_version 1798023 (0.0007) [2023-12-27 04:27:56,023][105692] Updated weights for policy 0, policy_version 1794052 (0.0008) [2023-12-27 04:27:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 919699456. Throughput: 0: 9699.1, 1: 9795.4. Samples: 919709868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:27:56,062][104569] Avg episode reward: [(0, '8628.376'), (1, '9351.223')] [2023-12-27 04:27:56,070][105692] Updated weights for policy 0, policy_version 1794062 (0.0005) [2023-12-27 04:27:56,115][105692] Updated weights for policy 0, policy_version 1794072 (0.0008) [2023-12-27 04:27:56,592][105620] Updated weights for policy 1, policy_version 1798033 (0.0008) [2023-12-27 04:27:56,639][105620] Updated weights for policy 1, policy_version 1798043 (0.0008) [2023-12-27 04:27:56,682][105620] Updated weights for policy 1, policy_version 1798053 (0.0008) [2023-12-27 04:27:56,729][105692] Updated weights for policy 0, policy_version 1794082 (0.0008) [2023-12-27 04:27:56,778][105692] Updated weights for policy 0, policy_version 1794092 (0.0007) [2023-12-27 04:27:56,845][105692] Updated weights for policy 0, policy_version 1794102 (0.0005) [2023-12-27 04:27:56,911][105692] Updated weights for policy 0, policy_version 1794112 (0.0005) [2023-12-27 04:27:57,366][105620] Updated weights for policy 1, policy_version 1798063 (0.0009) [2023-12-27 04:27:57,418][105620] Updated weights for policy 1, policy_version 1798073 (0.0009) [2023-12-27 04:27:57,469][105620] Updated weights for policy 1, policy_version 1798083 (0.0009) [2023-12-27 04:27:57,518][105692] Updated weights for policy 0, policy_version 1794122 (0.0006) [2023-12-27 04:27:57,577][105692] Updated weights for policy 0, policy_version 1794132 (0.0006) [2023-12-27 04:27:57,642][105692] Updated weights for policy 0, policy_version 1794142 (0.0005) [2023-12-27 04:27:58,106][105620] Updated weights for policy 1, policy_version 1798093 (0.0009) [2023-12-27 04:27:58,167][105620] Updated weights for policy 1, policy_version 1798103 (0.0006) [2023-12-27 04:27:58,232][105620] Updated weights for policy 1, policy_version 1798113 (0.0008) [2023-12-27 04:27:58,254][105692] Updated weights for policy 0, policy_version 1794152 (0.0007) [2023-12-27 04:27:58,316][105692] Updated weights for policy 0, policy_version 1794162 (0.0008) [2023-12-27 04:27:58,387][105692] Updated weights for policy 0, policy_version 1794172 (0.0007) [2023-12-27 04:27:58,991][105620] Updated weights for policy 1, policy_version 1798123 (0.0009) [2023-12-27 04:27:59,052][105620] Updated weights for policy 1, policy_version 1798133 (0.0011) [2023-12-27 04:27:59,058][105692] Updated weights for policy 0, policy_version 1794182 (0.0007) [2023-12-27 04:27:59,114][105620] Updated weights for policy 1, policy_version 1798143 (0.0010) [2023-12-27 04:27:59,116][105692] Updated weights for policy 0, policy_version 1794192 (0.0006) [2023-12-27 04:27:59,167][105692] Updated weights for policy 0, policy_version 1794202 (0.0008) [2023-12-27 04:27:59,851][105692] Updated weights for policy 0, policy_version 1794212 (0.0009) [2023-12-27 04:27:59,866][105620] Updated weights for policy 1, policy_version 1798153 (0.0009) [2023-12-27 04:27:59,912][105692] Updated weights for policy 0, policy_version 1794222 (0.0007) [2023-12-27 04:27:59,924][105620] Updated weights for policy 1, policy_version 1798163 (0.0010) [2023-12-27 04:27:59,975][105692] Updated weights for policy 0, policy_version 1794232 (0.0011) [2023-12-27 04:27:59,986][105620] Updated weights for policy 1, policy_version 1798173 (0.0011) [2023-12-27 04:28:00,051][105620] Updated weights for policy 1, policy_version 1798183 (0.0008) [2023-12-27 04:28:00,687][105692] Updated weights for policy 0, policy_version 1794242 (0.0011) [2023-12-27 04:28:00,730][105692] Updated weights for policy 0, policy_version 1794252 (0.0010) [2023-12-27 04:28:00,748][105620] Updated weights for policy 1, policy_version 1798193 (0.0006) [2023-12-27 04:28:00,777][105692] Updated weights for policy 0, policy_version 1794262 (0.0010) [2023-12-27 04:28:00,792][105620] Updated weights for policy 1, policy_version 1798203 (0.0007) [2023-12-27 04:28:00,827][105692] Updated weights for policy 0, policy_version 1794272 (0.0010) [2023-12-27 04:28:00,838][105620] Updated weights for policy 1, policy_version 1798213 (0.0007) [2023-12-27 04:28:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 919805952. Throughput: 0: 9838.9, 1: 9770.9. Samples: 919772048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:28:01,062][104569] Avg episode reward: [(0, '8447.209'), (1, '9166.578')] [2023-12-27 04:28:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001794272_459399168.pth... [2023-12-27 04:28:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001798216_460406784.pth... [2023-12-27 04:28:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001793120_459104256.pth [2023-12-27 04:28:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001797064_460111872.pth [2023-12-27 04:28:01,633][105692] Updated weights for policy 0, policy_version 1794282 (0.0009) [2023-12-27 04:28:01,669][105620] Updated weights for policy 1, policy_version 1798223 (0.0007) [2023-12-27 04:28:01,690][105692] Updated weights for policy 0, policy_version 1794292 (0.0007) [2023-12-27 04:28:01,733][105620] Updated weights for policy 1, policy_version 1798233 (0.0009) [2023-12-27 04:28:01,754][105692] Updated weights for policy 0, policy_version 1794302 (0.0006) [2023-12-27 04:28:01,788][105620] Updated weights for policy 1, policy_version 1798243 (0.0006) [2023-12-27 04:28:02,370][105620] Updated weights for policy 1, policy_version 1798253 (0.0008) [2023-12-27 04:28:02,420][105620] Updated weights for policy 1, policy_version 1798263 (0.0011) [2023-12-27 04:28:02,421][105692] Updated weights for policy 0, policy_version 1794312 (0.0006) [2023-12-27 04:28:02,481][105692] Updated weights for policy 0, policy_version 1794322 (0.0006) [2023-12-27 04:28:02,482][105620] Updated weights for policy 1, policy_version 1798273 (0.0011) [2023-12-27 04:28:02,538][105692] Updated weights for policy 0, policy_version 1794332 (0.0008) [2023-12-27 04:28:03,214][105692] Updated weights for policy 0, policy_version 1794342 (0.0007) [2023-12-27 04:28:03,271][105692] Updated weights for policy 0, policy_version 1794352 (0.0006) [2023-12-27 04:28:03,276][105620] Updated weights for policy 1, policy_version 1798283 (0.0008) [2023-12-27 04:28:03,323][105692] Updated weights for policy 0, policy_version 1794362 (0.0005) [2023-12-27 04:28:03,328][105620] Updated weights for policy 1, policy_version 1798293 (0.0010) [2023-12-27 04:28:03,380][105620] Updated weights for policy 1, policy_version 1798303 (0.0011) [2023-12-27 04:28:03,966][105692] Updated weights for policy 0, policy_version 1794372 (0.0005) [2023-12-27 04:28:04,030][105692] Updated weights for policy 0, policy_version 1794382 (0.0007) [2023-12-27 04:28:04,063][105620] Updated weights for policy 1, policy_version 1798313 (0.0010) [2023-12-27 04:28:04,083][105692] Updated weights for policy 0, policy_version 1794392 (0.0011) [2023-12-27 04:28:04,122][105620] Updated weights for policy 1, policy_version 1798323 (0.0011) [2023-12-27 04:28:04,182][105620] Updated weights for policy 1, policy_version 1798333 (0.0011) [2023-12-27 04:28:04,241][105620] Updated weights for policy 1, policy_version 1798343 (0.0011) [2023-12-27 04:28:04,759][105692] Updated weights for policy 0, policy_version 1794402 (0.0009) [2023-12-27 04:28:04,825][105692] Updated weights for policy 0, policy_version 1794412 (0.0009) [2023-12-27 04:28:04,889][105692] Updated weights for policy 0, policy_version 1794422 (0.0006) [2023-12-27 04:28:04,947][105620] Updated weights for policy 1, policy_version 1798353 (0.0006) [2023-12-27 04:28:04,949][105692] Updated weights for policy 0, policy_version 1794432 (0.0005) [2023-12-27 04:28:04,998][105620] Updated weights for policy 1, policy_version 1798363 (0.0005) [2023-12-27 04:28:05,048][105620] Updated weights for policy 1, policy_version 1798373 (0.0005) [2023-12-27 04:28:05,458][105692] Updated weights for policy 0, policy_version 1794442 (0.0008) [2023-12-27 04:28:05,510][105692] Updated weights for policy 0, policy_version 1794452 (0.0010) [2023-12-27 04:28:05,564][105692] Updated weights for policy 0, policy_version 1794462 (0.0010) [2023-12-27 04:28:05,662][105620] Updated weights for policy 1, policy_version 1798383 (0.0005) [2023-12-27 04:28:05,717][105620] Updated weights for policy 1, policy_version 1798393 (0.0005) [2023-12-27 04:28:05,781][105620] Updated weights for policy 1, policy_version 1798403 (0.0005) [2023-12-27 04:28:06,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 919904256. Throughput: 0: 9880.0, 1: 9689.1. Samples: 919890176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:28:06,063][104569] Avg episode reward: [(0, '8358.677'), (1, '8981.909')] [2023-12-27 04:28:06,149][105692] Updated weights for policy 0, policy_version 1794472 (0.0011) [2023-12-27 04:28:06,208][105692] Updated weights for policy 0, policy_version 1794482 (0.0010) [2023-12-27 04:28:06,270][105692] Updated weights for policy 0, policy_version 1794492 (0.0010) [2023-12-27 04:28:06,346][105620] Updated weights for policy 1, policy_version 1798413 (0.0007) [2023-12-27 04:28:06,415][105620] Updated weights for policy 1, policy_version 1798423 (0.0009) [2023-12-27 04:28:06,479][105620] Updated weights for policy 1, policy_version 1798433 (0.0008) [2023-12-27 04:28:06,914][105692] Updated weights for policy 0, policy_version 1794502 (0.0010) [2023-12-27 04:28:06,973][105692] Updated weights for policy 0, policy_version 1794512 (0.0011) [2023-12-27 04:28:07,038][105692] Updated weights for policy 0, policy_version 1794522 (0.0009) [2023-12-27 04:28:07,122][105620] Updated weights for policy 1, policy_version 1798443 (0.0008) [2023-12-27 04:28:07,170][105620] Updated weights for policy 1, policy_version 1798453 (0.0010) [2023-12-27 04:28:07,230][105620] Updated weights for policy 1, policy_version 1798463 (0.0006) [2023-12-27 04:28:07,687][105692] Updated weights for policy 0, policy_version 1794532 (0.0009) [2023-12-27 04:28:07,745][105692] Updated weights for policy 0, policy_version 1794542 (0.0006) [2023-12-27 04:28:07,813][105692] Updated weights for policy 0, policy_version 1794552 (0.0005) [2023-12-27 04:28:07,927][105620] Updated weights for policy 1, policy_version 1798473 (0.0006) [2023-12-27 04:28:07,989][105620] Updated weights for policy 1, policy_version 1798483 (0.0011) [2023-12-27 04:28:08,048][105620] Updated weights for policy 1, policy_version 1798493 (0.0010) [2023-12-27 04:28:08,102][105620] Updated weights for policy 1, policy_version 1798503 (0.0010) [2023-12-27 04:28:08,384][105692] Updated weights for policy 0, policy_version 1794562 (0.0006) [2023-12-27 04:28:08,440][105692] Updated weights for policy 0, policy_version 1794572 (0.0010) [2023-12-27 04:28:08,507][105692] Updated weights for policy 0, policy_version 1794582 (0.0005) [2023-12-27 04:28:08,574][105692] Updated weights for policy 0, policy_version 1794592 (0.0010) [2023-12-27 04:28:08,834][105620] Updated weights for policy 1, policy_version 1798513 (0.0010) [2023-12-27 04:28:08,891][105620] Updated weights for policy 1, policy_version 1798523 (0.0011) [2023-12-27 04:28:08,957][105620] Updated weights for policy 1, policy_version 1798533 (0.0007) [2023-12-27 04:28:09,241][105692] Updated weights for policy 0, policy_version 1794602 (0.0009) [2023-12-27 04:28:09,299][105692] Updated weights for policy 0, policy_version 1794612 (0.0009) [2023-12-27 04:28:09,368][105692] Updated weights for policy 0, policy_version 1794622 (0.0008) [2023-12-27 04:28:09,669][105620] Updated weights for policy 1, policy_version 1798543 (0.0009) [2023-12-27 04:28:09,733][105620] Updated weights for policy 1, policy_version 1798553 (0.0008) [2023-12-27 04:28:09,790][105620] Updated weights for policy 1, policy_version 1798563 (0.0008) [2023-12-27 04:28:10,117][105692] Updated weights for policy 0, policy_version 1794632 (0.0011) [2023-12-27 04:28:10,181][105692] Updated weights for policy 0, policy_version 1794642 (0.0010) [2023-12-27 04:28:10,240][105692] Updated weights for policy 0, policy_version 1794652 (0.0010) [2023-12-27 04:28:10,599][105620] Updated weights for policy 1, policy_version 1798573 (0.0008) [2023-12-27 04:28:10,660][105620] Updated weights for policy 1, policy_version 1798583 (0.0010) [2023-12-27 04:28:10,727][105620] Updated weights for policy 1, policy_version 1798593 (0.0010) [2023-12-27 04:28:10,849][105692] Updated weights for policy 0, policy_version 1794662 (0.0007) [2023-12-27 04:28:10,907][105692] Updated weights for policy 0, policy_version 1794672 (0.0010) [2023-12-27 04:28:10,955][105692] Updated weights for policy 0, policy_version 1794682 (0.0010) [2023-12-27 04:28:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 920010752. Throughput: 0: 10046.7, 1: 9702.6. Samples: 920013872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:28:11,062][104569] Avg episode reward: [(0, '8082.936'), (1, '9166.815')] [2023-12-27 04:28:11,528][105620] Updated weights for policy 1, policy_version 1798603 (0.0008) [2023-12-27 04:28:11,573][105620] Updated weights for policy 1, policy_version 1798613 (0.0006) [2023-12-27 04:28:11,631][105620] Updated weights for policy 1, policy_version 1798623 (0.0007) [2023-12-27 04:28:11,780][105692] Updated weights for policy 0, policy_version 1794692 (0.0009) [2023-12-27 04:28:11,837][105692] Updated weights for policy 0, policy_version 1794702 (0.0008) [2023-12-27 04:28:11,896][105692] Updated weights for policy 0, policy_version 1794712 (0.0008) [2023-12-27 04:28:12,372][105620] Updated weights for policy 1, policy_version 1798633 (0.0008) [2023-12-27 04:28:12,438][105620] Updated weights for policy 1, policy_version 1798643 (0.0012) [2023-12-27 04:28:12,504][105620] Updated weights for policy 1, policy_version 1798653 (0.0011) [2023-12-27 04:28:12,566][105620] Updated weights for policy 1, policy_version 1798663 (0.0010) [2023-12-27 04:28:12,602][105692] Updated weights for policy 0, policy_version 1794722 (0.0008) [2023-12-27 04:28:12,660][105692] Updated weights for policy 0, policy_version 1794732 (0.0008) [2023-12-27 04:28:12,719][105692] Updated weights for policy 0, policy_version 1794742 (0.0009) [2023-12-27 04:28:12,777][105692] Updated weights for policy 0, policy_version 1794752 (0.0009) [2023-12-27 04:28:13,215][105620] Updated weights for policy 1, policy_version 1798673 (0.0008) [2023-12-27 04:28:13,284][105620] Updated weights for policy 1, policy_version 1798683 (0.0008) [2023-12-27 04:28:13,345][105620] Updated weights for policy 1, policy_version 1798693 (0.0008) [2023-12-27 04:28:13,460][105692] Updated weights for policy 0, policy_version 1794762 (0.0005) [2023-12-27 04:28:13,516][105692] Updated weights for policy 0, policy_version 1794772 (0.0005) [2023-12-27 04:28:13,575][105692] Updated weights for policy 0, policy_version 1794782 (0.0005) [2023-12-27 04:28:13,883][105620] Updated weights for policy 1, policy_version 1798703 (0.0009) [2023-12-27 04:28:13,941][105620] Updated weights for policy 1, policy_version 1798713 (0.0010) [2023-12-27 04:28:14,003][105620] Updated weights for policy 1, policy_version 1798723 (0.0010) [2023-12-27 04:28:14,120][105692] Updated weights for policy 0, policy_version 1794792 (0.0006) [2023-12-27 04:28:14,182][105692] Updated weights for policy 0, policy_version 1794802 (0.0007) [2023-12-27 04:28:14,238][105692] Updated weights for policy 0, policy_version 1794812 (0.0006) [2023-12-27 04:28:14,721][105620] Updated weights for policy 1, policy_version 1798733 (0.0009) [2023-12-27 04:28:14,769][105620] Updated weights for policy 1, policy_version 1798743 (0.0007) [2023-12-27 04:28:14,806][105692] Updated weights for policy 0, policy_version 1794822 (0.0007) [2023-12-27 04:28:14,827][105620] Updated weights for policy 1, policy_version 1798753 (0.0007) [2023-12-27 04:28:14,870][105692] Updated weights for policy 0, policy_version 1794832 (0.0009) [2023-12-27 04:28:14,943][105692] Updated weights for policy 0, policy_version 1794842 (0.0006) [2023-12-27 04:28:15,533][105620] Updated weights for policy 1, policy_version 1798763 (0.0010) [2023-12-27 04:28:15,596][105620] Updated weights for policy 1, policy_version 1798773 (0.0011) [2023-12-27 04:28:15,626][105692] Updated weights for policy 0, policy_version 1794852 (0.0007) [2023-12-27 04:28:15,646][105620] Updated weights for policy 1, policy_version 1798783 (0.0011) [2023-12-27 04:28:15,686][105692] Updated weights for policy 0, policy_version 1794862 (0.0006) [2023-12-27 04:28:15,744][105692] Updated weights for policy 0, policy_version 1794872 (0.0008) [2023-12-27 04:28:16,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19797.2, 300 sec: 19549.7). Total num frames: 920109056. Throughput: 0: 9992.7, 1: 9762.9. Samples: 920074976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:28:16,063][104569] Avg episode reward: [(0, '8078.062'), (1, '9351.709')] [2023-12-27 04:28:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001798792_460554240.pth... [2023-12-27 04:28:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001794880_459554816.pth... [2023-12-27 04:28:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001797672_460267520.pth [2023-12-27 04:28:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001793696_459251712.pth [2023-12-27 04:28:16,397][105620] Updated weights for policy 1, policy_version 1798793 (0.0010) [2023-12-27 04:28:16,452][105692] Updated weights for policy 0, policy_version 1794882 (0.0008) [2023-12-27 04:28:16,452][105620] Updated weights for policy 1, policy_version 1798803 (0.0011) [2023-12-27 04:28:16,498][105692] Updated weights for policy 0, policy_version 1794892 (0.0005) [2023-12-27 04:28:16,501][105620] Updated weights for policy 1, policy_version 1798813 (0.0010) [2023-12-27 04:28:16,558][105692] Updated weights for policy 0, policy_version 1794902 (0.0006) [2023-12-27 04:28:16,560][105620] Updated weights for policy 1, policy_version 1798823 (0.0010) [2023-12-27 04:28:16,604][105692] Updated weights for policy 0, policy_version 1794912 (0.0005) [2023-12-27 04:28:17,141][105692] Updated weights for policy 0, policy_version 1794922 (0.0005) [2023-12-27 04:28:17,195][105620] Updated weights for policy 1, policy_version 1798833 (0.0006) [2023-12-27 04:28:17,205][105692] Updated weights for policy 0, policy_version 1794932 (0.0006) [2023-12-27 04:28:17,250][105620] Updated weights for policy 1, policy_version 1798843 (0.0005) [2023-12-27 04:28:17,269][105692] Updated weights for policy 0, policy_version 1794942 (0.0007) [2023-12-27 04:28:17,302][105620] Updated weights for policy 1, policy_version 1798853 (0.0008) [2023-12-27 04:28:17,835][105692] Updated weights for policy 0, policy_version 1794952 (0.0006) [2023-12-27 04:28:17,889][105692] Updated weights for policy 0, policy_version 1794962 (0.0005) [2023-12-27 04:28:17,899][105620] Updated weights for policy 1, policy_version 1798863 (0.0007) [2023-12-27 04:28:17,943][105692] Updated weights for policy 0, policy_version 1794972 (0.0005) [2023-12-27 04:28:17,952][105620] Updated weights for policy 1, policy_version 1798873 (0.0005) [2023-12-27 04:28:18,016][105620] Updated weights for policy 1, policy_version 1798883 (0.0006) [2023-12-27 04:28:18,528][105692] Updated weights for policy 0, policy_version 1794982 (0.0009) [2023-12-27 04:28:18,568][105620] Updated weights for policy 1, policy_version 1798893 (0.0009) [2023-12-27 04:28:18,595][105692] Updated weights for policy 0, policy_version 1794992 (0.0011) [2023-12-27 04:28:18,616][105620] Updated weights for policy 1, policy_version 1798903 (0.0007) [2023-12-27 04:28:18,657][105692] Updated weights for policy 0, policy_version 1795002 (0.0010) [2023-12-27 04:28:18,664][105620] Updated weights for policy 1, policy_version 1798913 (0.0007) [2023-12-27 04:28:19,285][105620] Updated weights for policy 1, policy_version 1798923 (0.0008) [2023-12-27 04:28:19,341][105620] Updated weights for policy 1, policy_version 1798933 (0.0011) [2023-12-27 04:28:19,406][105620] Updated weights for policy 1, policy_version 1798943 (0.0011) [2023-12-27 04:28:19,410][105692] Updated weights for policy 0, policy_version 1795012 (0.0011) [2023-12-27 04:28:19,459][105692] Updated weights for policy 0, policy_version 1795022 (0.0011) [2023-12-27 04:28:19,518][105692] Updated weights for policy 0, policy_version 1795032 (0.0008) [2023-12-27 04:28:20,155][105692] Updated weights for policy 0, policy_version 1795042 (0.0007) [2023-12-27 04:28:20,170][105620] Updated weights for policy 1, policy_version 1798953 (0.0010) [2023-12-27 04:28:20,211][105692] Updated weights for policy 0, policy_version 1795052 (0.0010) [2023-12-27 04:28:20,232][105620] Updated weights for policy 1, policy_version 1798963 (0.0008) [2023-12-27 04:28:20,270][105692] Updated weights for policy 0, policy_version 1795062 (0.0007) [2023-12-27 04:28:20,295][105620] Updated weights for policy 1, policy_version 1798973 (0.0008) [2023-12-27 04:28:20,321][105692] Updated weights for policy 0, policy_version 1795072 (0.0005) [2023-12-27 04:28:20,354][105620] Updated weights for policy 1, policy_version 1798983 (0.0009) [2023-12-27 04:28:20,937][105692] Updated weights for policy 0, policy_version 1795082 (0.0005) [2023-12-27 04:28:20,984][105692] Updated weights for policy 0, policy_version 1795092 (0.0005) [2023-12-27 04:28:21,043][105692] Updated weights for policy 0, policy_version 1795102 (0.0008) [2023-12-27 04:28:21,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 920215552. Throughput: 0: 10135.0, 1: 9885.3. Samples: 920202444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:28:21,062][104569] Avg episode reward: [(0, '8532.980'), (1, '9166.882')] [2023-12-27 04:28:21,212][105620] Updated weights for policy 1, policy_version 1798993 (0.0009) [2023-12-27 04:28:21,277][105620] Updated weights for policy 1, policy_version 1799003 (0.0006) [2023-12-27 04:28:21,338][105620] Updated weights for policy 1, policy_version 1799013 (0.0006) [2023-12-27 04:28:21,687][105692] Updated weights for policy 0, policy_version 1795112 (0.0006) [2023-12-27 04:28:21,757][105692] Updated weights for policy 0, policy_version 1795122 (0.0009) [2023-12-27 04:28:21,828][105692] Updated weights for policy 0, policy_version 1795132 (0.0010) [2023-12-27 04:28:22,074][105620] Updated weights for policy 1, policy_version 1799023 (0.0009) [2023-12-27 04:28:22,133][105620] Updated weights for policy 1, policy_version 1799033 (0.0010) [2023-12-27 04:28:22,196][105620] Updated weights for policy 1, policy_version 1799043 (0.0010) [2023-12-27 04:28:22,517][105692] Updated weights for policy 0, policy_version 1795142 (0.0010) [2023-12-27 04:28:22,570][105692] Updated weights for policy 0, policy_version 1795152 (0.0010) [2023-12-27 04:28:22,634][105692] Updated weights for policy 0, policy_version 1795162 (0.0011) [2023-12-27 04:28:23,070][105620] Updated weights for policy 1, policy_version 1799053 (0.0009) [2023-12-27 04:28:23,125][105620] Updated weights for policy 1, policy_version 1799064 (0.0010) [2023-12-27 04:28:23,181][105620] Updated weights for policy 1, policy_version 1799074 (0.0010) [2023-12-27 04:28:23,222][105692] Updated weights for policy 0, policy_version 1795172 (0.0009) [2023-12-27 04:28:23,269][105692] Updated weights for policy 0, policy_version 1795182 (0.0005) [2023-12-27 04:28:23,320][105692] Updated weights for policy 0, policy_version 1795192 (0.0005) [2023-12-27 04:28:23,871][105620] Updated weights for policy 1, policy_version 1799084 (0.0008) [2023-12-27 04:28:23,933][105620] Updated weights for policy 1, policy_version 1799094 (0.0009) [2023-12-27 04:28:23,992][105620] Updated weights for policy 1, policy_version 1799104 (0.0010) [2023-12-27 04:28:24,048][105692] Updated weights for policy 0, policy_version 1795202 (0.0009) [2023-12-27 04:28:24,116][105692] Updated weights for policy 0, policy_version 1795212 (0.0007) [2023-12-27 04:28:24,178][105692] Updated weights for policy 0, policy_version 1795222 (0.0011) [2023-12-27 04:28:24,241][105692] Updated weights for policy 0, policy_version 1795232 (0.0007) [2023-12-27 04:28:24,711][105620] Updated weights for policy 1, policy_version 1799114 (0.0010) [2023-12-27 04:28:24,766][105620] Updated weights for policy 1, policy_version 1799124 (0.0010) [2023-12-27 04:28:24,839][105620] Updated weights for policy 1, policy_version 1799134 (0.0010) [2023-12-27 04:28:24,882][105692] Updated weights for policy 0, policy_version 1795242 (0.0005) [2023-12-27 04:28:24,902][105620] Updated weights for policy 1, policy_version 1799144 (0.0010) [2023-12-27 04:28:24,940][105692] Updated weights for policy 0, policy_version 1795252 (0.0006) [2023-12-27 04:28:24,992][105692] Updated weights for policy 0, policy_version 1795262 (0.0005) [2023-12-27 04:28:25,627][105620] Updated weights for policy 1, policy_version 1799154 (0.0010) [2023-12-27 04:28:25,633][105692] Updated weights for policy 0, policy_version 1795272 (0.0006) [2023-12-27 04:28:25,683][105620] Updated weights for policy 1, policy_version 1799164 (0.0010) [2023-12-27 04:28:25,694][105692] Updated weights for policy 0, policy_version 1795282 (0.0006) [2023-12-27 04:28:25,743][105620] Updated weights for policy 1, policy_version 1799174 (0.0010) [2023-12-27 04:28:25,753][105692] Updated weights for policy 0, policy_version 1795292 (0.0006) [2023-12-27 04:28:26,062][104569] Fps is (10 sec: 20480.4, 60 sec: 20070.4, 300 sec: 19577.5). Total num frames: 920313856. Throughput: 0: 10240.9, 1: 9770.5. Samples: 920320488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:28:26,063][104569] Avg episode reward: [(0, '8627.579'), (1, '9166.801')] [2023-12-27 04:28:26,426][105692] Updated weights for policy 0, policy_version 1795302 (0.0007) [2023-12-27 04:28:26,485][105692] Updated weights for policy 0, policy_version 1795312 (0.0005) [2023-12-27 04:28:26,485][105620] Updated weights for policy 1, policy_version 1799184 (0.0010) [2023-12-27 04:28:26,534][105692] Updated weights for policy 0, policy_version 1795322 (0.0005) [2023-12-27 04:28:26,535][105620] Updated weights for policy 1, policy_version 1799194 (0.0010) [2023-12-27 04:28:26,584][105620] Updated weights for policy 1, policy_version 1799204 (0.0010) [2023-12-27 04:28:27,147][105692] Updated weights for policy 0, policy_version 1795332 (0.0006) [2023-12-27 04:28:27,198][105692] Updated weights for policy 0, policy_version 1795342 (0.0005) [2023-12-27 04:28:27,261][105692] Updated weights for policy 0, policy_version 1795352 (0.0008) [2023-12-27 04:28:27,347][105620] Updated weights for policy 1, policy_version 1799214 (0.0007) [2023-12-27 04:28:27,392][105620] Updated weights for policy 1, policy_version 1799224 (0.0005) [2023-12-27 04:28:27,448][105620] Updated weights for policy 1, policy_version 1799234 (0.0005) [2023-12-27 04:28:27,958][105620] Updated weights for policy 1, policy_version 1799244 (0.0005) [2023-12-27 04:28:28,007][105620] Updated weights for policy 1, policy_version 1799254 (0.0009) [2023-12-27 04:28:28,057][105620] Updated weights for policy 1, policy_version 1799264 (0.0005) [2023-12-27 04:28:28,082][105692] Updated weights for policy 0, policy_version 1795362 (0.0008) [2023-12-27 04:28:28,142][105692] Updated weights for policy 0, policy_version 1795372 (0.0007) [2023-12-27 04:28:28,212][105692] Updated weights for policy 0, policy_version 1795382 (0.0005) [2023-12-27 04:28:28,278][105692] Updated weights for policy 0, policy_version 1795392 (0.0005) [2023-12-27 04:28:28,685][105620] Updated weights for policy 1, policy_version 1799274 (0.0006) [2023-12-27 04:28:28,741][105620] Updated weights for policy 1, policy_version 1799284 (0.0010) [2023-12-27 04:28:28,795][105692] Updated weights for policy 0, policy_version 1795402 (0.0007) [2023-12-27 04:28:28,804][105620] Updated weights for policy 1, policy_version 1799294 (0.0010) [2023-12-27 04:28:28,853][105692] Updated weights for policy 0, policy_version 1795412 (0.0008) [2023-12-27 04:28:28,863][105620] Updated weights for policy 1, policy_version 1799304 (0.0010) [2023-12-27 04:28:28,907][105692] Updated weights for policy 0, policy_version 1795422 (0.0005) [2023-12-27 04:28:29,518][105620] Updated weights for policy 1, policy_version 1799314 (0.0010) [2023-12-27 04:28:29,565][105620] Updated weights for policy 1, policy_version 1799324 (0.0010) [2023-12-27 04:28:29,626][105620] Updated weights for policy 1, policy_version 1799334 (0.0008) [2023-12-27 04:28:29,657][105692] Updated weights for policy 0, policy_version 1795432 (0.0008) [2023-12-27 04:28:29,708][105692] Updated weights for policy 0, policy_version 1795442 (0.0009) [2023-12-27 04:28:29,757][105692] Updated weights for policy 0, policy_version 1795452 (0.0009) [2023-12-27 04:28:30,368][105620] Updated weights for policy 1, policy_version 1799344 (0.0006) [2023-12-27 04:28:30,381][105692] Updated weights for policy 0, policy_version 1795462 (0.0009) [2023-12-27 04:28:30,422][105620] Updated weights for policy 1, policy_version 1799354 (0.0006) [2023-12-27 04:28:30,443][105692] Updated weights for policy 0, policy_version 1795472 (0.0008) [2023-12-27 04:28:30,479][105620] Updated weights for policy 1, policy_version 1799364 (0.0008) [2023-12-27 04:28:30,496][105692] Updated weights for policy 0, policy_version 1795482 (0.0005) [2023-12-27 04:28:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19933.8, 300 sec: 19577.5). Total num frames: 920412160. Throughput: 0: 10258.7, 1: 9802.9. Samples: 920383060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:28:31,063][104569] Avg episode reward: [(0, '8628.671'), (1, '9351.406')] [2023-12-27 04:28:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001795488_459710464.pth... [2023-12-27 04:28:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001794272_459399168.pth [2023-12-27 04:28:31,104][105620] Updated weights for policy 1, policy_version 1799374 (0.0007) [2023-12-27 04:28:31,170][105620] Updated weights for policy 1, policy_version 1799384 (0.0008) [2023-12-27 04:28:31,228][105620] Updated weights for policy 1, policy_version 1799394 (0.0008) [2023-12-27 04:28:31,261][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001799400_460709888.pth... [2023-12-27 04:28:31,265][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001798216_460406784.pth [2023-12-27 04:28:31,287][105692] Updated weights for policy 0, policy_version 1795492 (0.0006) [2023-12-27 04:28:31,353][105692] Updated weights for policy 0, policy_version 1795502 (0.0008) [2023-12-27 04:28:31,407][105692] Updated weights for policy 0, policy_version 1795512 (0.0007) [2023-12-27 04:28:31,939][105620] Updated weights for policy 1, policy_version 1799404 (0.0008) [2023-12-27 04:28:31,996][105620] Updated weights for policy 1, policy_version 1799414 (0.0008) [2023-12-27 04:28:32,052][105620] Updated weights for policy 1, policy_version 1799424 (0.0008) [2023-12-27 04:28:32,150][105692] Updated weights for policy 0, policy_version 1795522 (0.0006) [2023-12-27 04:28:32,198][105692] Updated weights for policy 0, policy_version 1795532 (0.0010) [2023-12-27 04:28:32,246][105692] Updated weights for policy 0, policy_version 1795542 (0.0010) [2023-12-27 04:28:32,298][105692] Updated weights for policy 0, policy_version 1795552 (0.0010) [2023-12-27 04:28:32,687][105620] Updated weights for policy 1, policy_version 1799434 (0.0006) [2023-12-27 04:28:32,746][105620] Updated weights for policy 1, policy_version 1799444 (0.0006) [2023-12-27 04:28:32,802][105620] Updated weights for policy 1, policy_version 1799454 (0.0007) [2023-12-27 04:28:32,853][105620] Updated weights for policy 1, policy_version 1799464 (0.0007) [2023-12-27 04:28:33,066][105692] Updated weights for policy 0, policy_version 1795562 (0.0010) [2023-12-27 04:28:33,116][105692] Updated weights for policy 0, policy_version 1795572 (0.0010) [2023-12-27 04:28:33,167][105692] Updated weights for policy 0, policy_version 1795582 (0.0010) [2023-12-27 04:28:33,566][105620] Updated weights for policy 1, policy_version 1799474 (0.0008) [2023-12-27 04:28:33,617][105620] Updated weights for policy 1, policy_version 1799484 (0.0008) [2023-12-27 04:28:33,661][105620] Updated weights for policy 1, policy_version 1799494 (0.0008) [2023-12-27 04:28:33,911][105692] Updated weights for policy 0, policy_version 1795592 (0.0006) [2023-12-27 04:28:33,964][105692] Updated weights for policy 0, policy_version 1795602 (0.0005) [2023-12-27 04:28:34,027][105692] Updated weights for policy 0, policy_version 1795612 (0.0008) [2023-12-27 04:28:34,406][105620] Updated weights for policy 1, policy_version 1799504 (0.0009) [2023-12-27 04:28:34,463][105620] Updated weights for policy 1, policy_version 1799514 (0.0006) [2023-12-27 04:28:34,512][105620] Updated weights for policy 1, policy_version 1799524 (0.0005) [2023-12-27 04:28:34,772][105692] Updated weights for policy 0, policy_version 1795622 (0.0010) [2023-12-27 04:28:34,823][105692] Updated weights for policy 0, policy_version 1795632 (0.0009) [2023-12-27 04:28:34,882][105692] Updated weights for policy 0, policy_version 1795642 (0.0009) [2023-12-27 04:28:35,160][105620] Updated weights for policy 1, policy_version 1799534 (0.0005) [2023-12-27 04:28:35,220][105620] Updated weights for policy 1, policy_version 1799544 (0.0007) [2023-12-27 04:28:35,280][105620] Updated weights for policy 1, policy_version 1799554 (0.0009) [2023-12-27 04:28:35,617][105692] Updated weights for policy 0, policy_version 1795652 (0.0007) [2023-12-27 04:28:35,671][105692] Updated weights for policy 0, policy_version 1795662 (0.0005) [2023-12-27 04:28:35,730][105692] Updated weights for policy 0, policy_version 1795672 (0.0005) [2023-12-27 04:28:35,911][105620] Updated weights for policy 1, policy_version 1799564 (0.0009) [2023-12-27 04:28:35,981][105620] Updated weights for policy 1, policy_version 1799574 (0.0009) [2023-12-27 04:28:36,042][105620] Updated weights for policy 1, policy_version 1799584 (0.0010) [2023-12-27 04:28:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19933.8, 300 sec: 19577.5). Total num frames: 920510464. Throughput: 0: 10268.5, 1: 9896.0. Samples: 920502620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:28:36,063][104569] Avg episode reward: [(0, '8988.852'), (1, '9260.913')] [2023-12-27 04:28:36,385][105692] Updated weights for policy 0, policy_version 1795682 (0.0007) [2023-12-27 04:28:36,451][105692] Updated weights for policy 0, policy_version 1795692 (0.0006) [2023-12-27 04:28:36,501][105692] Updated weights for policy 0, policy_version 1795702 (0.0008) [2023-12-27 04:28:36,551][105692] Updated weights for policy 0, policy_version 1795712 (0.0008) [2023-12-27 04:28:36,738][105620] Updated weights for policy 1, policy_version 1799594 (0.0010) [2023-12-27 04:28:36,786][105620] Updated weights for policy 1, policy_version 1799604 (0.0010) [2023-12-27 04:28:36,831][105620] Updated weights for policy 1, policy_version 1799614 (0.0010) [2023-12-27 04:28:36,879][105620] Updated weights for policy 1, policy_version 1799624 (0.0010) [2023-12-27 04:28:37,278][105692] Updated weights for policy 0, policy_version 1795722 (0.0008) [2023-12-27 04:28:37,324][105692] Updated weights for policy 0, policy_version 1795732 (0.0008) [2023-12-27 04:28:37,370][105692] Updated weights for policy 0, policy_version 1795742 (0.0008) [2023-12-27 04:28:37,660][105620] Updated weights for policy 1, policy_version 1799634 (0.0010) [2023-12-27 04:28:37,719][105620] Updated weights for policy 1, policy_version 1799644 (0.0010) [2023-12-27 04:28:37,767][105620] Updated weights for policy 1, policy_version 1799654 (0.0010) [2023-12-27 04:28:38,137][105692] Updated weights for policy 0, policy_version 1795752 (0.0008) [2023-12-27 04:28:38,187][105692] Updated weights for policy 0, policy_version 1795762 (0.0005) [2023-12-27 04:28:38,240][105692] Updated weights for policy 0, policy_version 1795772 (0.0005) [2023-12-27 04:28:38,471][105620] Updated weights for policy 1, policy_version 1799664 (0.0006) [2023-12-27 04:28:38,531][105620] Updated weights for policy 1, policy_version 1799674 (0.0006) [2023-12-27 04:28:38,596][105620] Updated weights for policy 1, policy_version 1799685 (0.0007) [2023-12-27 04:28:38,856][105692] Updated weights for policy 0, policy_version 1795782 (0.0007) [2023-12-27 04:28:38,923][105692] Updated weights for policy 0, policy_version 1795792 (0.0009) [2023-12-27 04:28:38,977][105692] Updated weights for policy 0, policy_version 1795802 (0.0007) [2023-12-27 04:28:39,247][105620] Updated weights for policy 1, policy_version 1799695 (0.0008) [2023-12-27 04:28:39,312][105620] Updated weights for policy 1, policy_version 1799705 (0.0009) [2023-12-27 04:28:39,377][105620] Updated weights for policy 1, policy_version 1799715 (0.0008) [2023-12-27 04:28:39,777][105692] Updated weights for policy 0, policy_version 1795812 (0.0009) [2023-12-27 04:28:39,843][105692] Updated weights for policy 0, policy_version 1795822 (0.0009) [2023-12-27 04:28:39,905][105692] Updated weights for policy 0, policy_version 1795832 (0.0006) [2023-12-27 04:28:40,131][105620] Updated weights for policy 1, policy_version 1799725 (0.0009) [2023-12-27 04:28:40,196][105620] Updated weights for policy 1, policy_version 1799735 (0.0009) [2023-12-27 04:28:40,262][105620] Updated weights for policy 1, policy_version 1799745 (0.0009) [2023-12-27 04:28:40,600][105692] Updated weights for policy 0, policy_version 1795842 (0.0007) [2023-12-27 04:28:40,666][105692] Updated weights for policy 0, policy_version 1795852 (0.0005) [2023-12-27 04:28:40,720][105692] Updated weights for policy 0, policy_version 1795862 (0.0005) [2023-12-27 04:28:40,776][105692] Updated weights for policy 0, policy_version 1795872 (0.0005) [2023-12-27 04:28:41,050][105620] Updated weights for policy 1, policy_version 1799755 (0.0009) [2023-12-27 04:28:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19933.8, 300 sec: 19577.5). Total num frames: 920608768. Throughput: 0: 10284.5, 1: 9928.7. Samples: 920619464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:28:41,063][104569] Avg episode reward: [(0, '8991.186'), (1, '9168.534')] [2023-12-27 04:28:41,109][105620] Updated weights for policy 1, policy_version 1799765 (0.0007) [2023-12-27 04:28:41,180][105620] Updated weights for policy 1, policy_version 1799775 (0.0008) [2023-12-27 04:28:41,476][105692] Updated weights for policy 0, policy_version 1795882 (0.0008) [2023-12-27 04:28:41,525][105692] Updated weights for policy 0, policy_version 1795892 (0.0008) [2023-12-27 04:28:41,587][105692] Updated weights for policy 0, policy_version 1795902 (0.0009) [2023-12-27 04:28:41,933][105620] Updated weights for policy 1, policy_version 1799785 (0.0009) [2023-12-27 04:28:41,997][105620] Updated weights for policy 1, policy_version 1799795 (0.0008) [2023-12-27 04:28:42,058][105620] Updated weights for policy 1, policy_version 1799805 (0.0008) [2023-12-27 04:28:42,116][105620] Updated weights for policy 1, policy_version 1799815 (0.0008) [2023-12-27 04:28:42,444][105692] Updated weights for policy 0, policy_version 1795912 (0.0009) [2023-12-27 04:28:42,493][105692] Updated weights for policy 0, policy_version 1795922 (0.0009) [2023-12-27 04:28:42,552][105692] Updated weights for policy 0, policy_version 1795932 (0.0009) [2023-12-27 04:28:42,790][105620] Updated weights for policy 1, policy_version 1799825 (0.0008) [2023-12-27 04:28:42,851][105620] Updated weights for policy 1, policy_version 1799835 (0.0005) [2023-12-27 04:28:42,914][105620] Updated weights for policy 1, policy_version 1799845 (0.0006) [2023-12-27 04:28:43,390][105692] Updated weights for policy 0, policy_version 1795942 (0.0009) [2023-12-27 04:28:43,448][105692] Updated weights for policy 0, policy_version 1795952 (0.0009) [2023-12-27 04:28:43,498][105692] Updated weights for policy 0, policy_version 1795962 (0.0009) [2023-12-27 04:28:43,564][105620] Updated weights for policy 1, policy_version 1799855 (0.0007) [2023-12-27 04:28:43,625][105620] Updated weights for policy 1, policy_version 1799865 (0.0009) [2023-12-27 04:28:43,683][105620] Updated weights for policy 1, policy_version 1799875 (0.0009) [2023-12-27 04:28:44,191][105692] Updated weights for policy 0, policy_version 1795972 (0.0008) [2023-12-27 04:28:44,248][105692] Updated weights for policy 0, policy_version 1795982 (0.0009) [2023-12-27 04:28:44,299][105692] Updated weights for policy 0, policy_version 1795992 (0.0009) [2023-12-27 04:28:44,374][105620] Updated weights for policy 1, policy_version 1799885 (0.0007) [2023-12-27 04:28:44,425][105620] Updated weights for policy 1, policy_version 1799895 (0.0005) [2023-12-27 04:28:44,483][105620] Updated weights for policy 1, policy_version 1799905 (0.0009) [2023-12-27 04:28:45,102][105692] Updated weights for policy 0, policy_version 1796002 (0.0009) [2023-12-27 04:28:45,165][105692] Updated weights for policy 0, policy_version 1796012 (0.0009) [2023-12-27 04:28:45,204][105620] Updated weights for policy 1, policy_version 1799915 (0.0009) [2023-12-27 04:28:45,226][105692] Updated weights for policy 0, policy_version 1796022 (0.0010) [2023-12-27 04:28:45,261][105620] Updated weights for policy 1, policy_version 1799925 (0.0007) [2023-12-27 04:28:45,284][105692] Updated weights for policy 0, policy_version 1796032 (0.0006) [2023-12-27 04:28:45,321][105620] Updated weights for policy 1, policy_version 1799935 (0.0008) [2023-12-27 04:28:45,951][105620] Updated weights for policy 1, policy_version 1799945 (0.0009) [2023-12-27 04:28:46,009][105620] Updated weights for policy 1, policy_version 1799955 (0.0010) [2023-12-27 04:28:46,058][105620] Updated weights for policy 1, policy_version 1799965 (0.0010) [2023-12-27 04:28:46,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19933.9, 300 sec: 19549.7). Total num frames: 920698880. Throughput: 0: 10149.3, 1: 9922.9. Samples: 920675300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:28:46,063][104569] Avg episode reward: [(0, '8442.033'), (1, '9258.888')] [2023-12-27 04:28:46,106][105692] Updated weights for policy 0, policy_version 1796042 (0.0008) [2023-12-27 04:28:46,117][105620] Updated weights for policy 1, policy_version 1799975 (0.0010) [2023-12-27 04:28:46,122][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001799976_460857344.pth... [2023-12-27 04:28:46,126][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001798792_460554240.pth [2023-12-27 04:28:46,171][105692] Updated weights for policy 0, policy_version 1796052 (0.0008) [2023-12-27 04:28:46,240][105692] Updated weights for policy 0, policy_version 1796062 (0.0007) [2023-12-27 04:28:46,250][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001796064_459857920.pth... [2023-12-27 04:28:46,255][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001794880_459554816.pth [2023-12-27 04:28:46,849][105620] Updated weights for policy 1, policy_version 1799985 (0.0007) [2023-12-27 04:28:46,919][105620] Updated weights for policy 1, policy_version 1799995 (0.0006) [2023-12-27 04:28:46,925][105692] Updated weights for policy 0, policy_version 1796072 (0.0008) [2023-12-27 04:28:46,980][105620] Updated weights for policy 1, policy_version 1800005 (0.0008) [2023-12-27 04:28:46,986][105692] Updated weights for policy 0, policy_version 1796082 (0.0006) [2023-12-27 04:28:47,039][105692] Updated weights for policy 0, policy_version 1796092 (0.0008) [2023-12-27 04:28:47,675][105620] Updated weights for policy 1, policy_version 1800015 (0.0009) [2023-12-27 04:28:47,724][105620] Updated weights for policy 1, policy_version 1800026 (0.0007) [2023-12-27 04:28:47,778][105620] Updated weights for policy 1, policy_version 1800036 (0.0009) [2023-12-27 04:28:47,785][105692] Updated weights for policy 0, policy_version 1796102 (0.0007) [2023-12-27 04:28:47,837][105692] Updated weights for policy 0, policy_version 1796112 (0.0008) [2023-12-27 04:28:47,888][105692] Updated weights for policy 0, policy_version 1796122 (0.0008) [2023-12-27 04:28:48,477][105620] Updated weights for policy 1, policy_version 1800046 (0.0008) [2023-12-27 04:28:48,528][105620] Updated weights for policy 1, policy_version 1800056 (0.0009) [2023-12-27 04:28:48,579][105620] Updated weights for policy 1, policy_version 1800066 (0.0009) [2023-12-27 04:28:48,686][105692] Updated weights for policy 0, policy_version 1796132 (0.0009) [2023-12-27 04:28:48,749][105692] Updated weights for policy 0, policy_version 1796142 (0.0009) [2023-12-27 04:28:48,812][105692] Updated weights for policy 0, policy_version 1796152 (0.0009) [2023-12-27 04:28:49,350][105620] Updated weights for policy 1, policy_version 1800076 (0.0009) [2023-12-27 04:28:49,413][105620] Updated weights for policy 1, policy_version 1800086 (0.0009) [2023-12-27 04:28:49,476][105620] Updated weights for policy 1, policy_version 1800096 (0.0010) [2023-12-27 04:28:49,573][105692] Updated weights for policy 0, policy_version 1796162 (0.0010) [2023-12-27 04:28:49,631][105692] Updated weights for policy 0, policy_version 1796172 (0.0009) [2023-12-27 04:28:49,679][105692] Updated weights for policy 0, policy_version 1796182 (0.0009) [2023-12-27 04:28:49,730][105692] Updated weights for policy 0, policy_version 1796192 (0.0009) [2023-12-27 04:28:50,243][105620] Updated weights for policy 1, policy_version 1800106 (0.0009) [2023-12-27 04:28:50,294][105620] Updated weights for policy 1, policy_version 1800116 (0.0009) [2023-12-27 04:28:50,345][105620] Updated weights for policy 1, policy_version 1800126 (0.0009) [2023-12-27 04:28:50,409][105620] Updated weights for policy 1, policy_version 1800136 (0.0009) [2023-12-27 04:28:50,515][105692] Updated weights for policy 0, policy_version 1796202 (0.0008) [2023-12-27 04:28:50,571][105692] Updated weights for policy 0, policy_version 1796212 (0.0009) [2023-12-27 04:28:50,634][105692] Updated weights for policy 0, policy_version 1796222 (0.0010) [2023-12-27 04:28:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 920797184. Throughput: 0: 10060.8, 1: 9945.4. Samples: 920790456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:28:51,062][104569] Avg episode reward: [(0, '7893.006'), (1, '9350.973')] [2023-12-27 04:28:51,204][105620] Updated weights for policy 1, policy_version 1800146 (0.0008) [2023-12-27 04:28:51,270][105620] Updated weights for policy 1, policy_version 1800156 (0.0009) [2023-12-27 04:28:51,332][105620] Updated weights for policy 1, policy_version 1800166 (0.0009) [2023-12-27 04:28:51,382][105692] Updated weights for policy 0, policy_version 1796232 (0.0008) [2023-12-27 04:28:51,449][105692] Updated weights for policy 0, policy_version 1796242 (0.0009) [2023-12-27 04:28:51,508][105692] Updated weights for policy 0, policy_version 1796252 (0.0009) [2023-12-27 04:28:52,049][105620] Updated weights for policy 1, policy_version 1800176 (0.0009) [2023-12-27 04:28:52,113][105620] Updated weights for policy 1, policy_version 1800186 (0.0009) [2023-12-27 04:28:52,170][105620] Updated weights for policy 1, policy_version 1800196 (0.0008) [2023-12-27 04:28:52,215][105692] Updated weights for policy 0, policy_version 1796262 (0.0009) [2023-12-27 04:28:52,273][105692] Updated weights for policy 0, policy_version 1796272 (0.0007) [2023-12-27 04:28:52,332][105692] Updated weights for policy 0, policy_version 1796282 (0.0006) [2023-12-27 04:28:52,819][105620] Updated weights for policy 1, policy_version 1800206 (0.0008) [2023-12-27 04:28:52,873][105620] Updated weights for policy 1, policy_version 1800216 (0.0008) [2023-12-27 04:28:52,929][105620] Updated weights for policy 1, policy_version 1800226 (0.0009) [2023-12-27 04:28:53,010][105692] Updated weights for policy 0, policy_version 1796292 (0.0008) [2023-12-27 04:28:53,065][105692] Updated weights for policy 0, policy_version 1796302 (0.0009) [2023-12-27 04:28:53,128][105692] Updated weights for policy 0, policy_version 1796312 (0.0009) [2023-12-27 04:28:53,694][105692] Updated weights for policy 0, policy_version 1796322 (0.0008) [2023-12-27 04:28:53,746][105692] Updated weights for policy 0, policy_version 1796332 (0.0005) [2023-12-27 04:28:53,796][105620] Updated weights for policy 1, policy_version 1800237 (0.0008) [2023-12-27 04:28:53,799][105692] Updated weights for policy 0, policy_version 1796342 (0.0005) [2023-12-27 04:28:53,842][105692] Updated weights for policy 0, policy_version 1796352 (0.0005) [2023-12-27 04:28:53,847][105620] Updated weights for policy 1, policy_version 1800247 (0.0009) [2023-12-27 04:28:53,901][105620] Updated weights for policy 1, policy_version 1800257 (0.0009) [2023-12-27 04:28:54,505][105692] Updated weights for policy 0, policy_version 1796362 (0.0009) [2023-12-27 04:28:54,584][105692] Updated weights for policy 0, policy_version 1796372 (0.0009) [2023-12-27 04:28:54,636][105692] Updated weights for policy 0, policy_version 1796382 (0.0009) [2023-12-27 04:28:54,704][105620] Updated weights for policy 1, policy_version 1800267 (0.0010) [2023-12-27 04:28:54,766][105620] Updated weights for policy 1, policy_version 1800277 (0.0009) [2023-12-27 04:28:54,831][105620] Updated weights for policy 1, policy_version 1800287 (0.0009) [2023-12-27 04:28:55,318][105692] Updated weights for policy 0, policy_version 1796392 (0.0006) [2023-12-27 04:28:55,379][105692] Updated weights for policy 0, policy_version 1796402 (0.0008) [2023-12-27 04:28:55,427][105692] Updated weights for policy 0, policy_version 1796412 (0.0010) [2023-12-27 04:28:55,695][105620] Updated weights for policy 1, policy_version 1800297 (0.0009) [2023-12-27 04:28:55,749][105620] Updated weights for policy 1, policy_version 1800307 (0.0010) [2023-12-27 04:28:55,805][105620] Updated weights for policy 1, policy_version 1800318 (0.0008) [2023-12-27 04:28:55,876][105620] Updated weights for policy 1, policy_version 1800328 (0.0009) [2023-12-27 04:28:56,020][105692] Updated weights for policy 0, policy_version 1796422 (0.0008) [2023-12-27 04:28:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 920895488. Throughput: 0: 9980.9, 1: 9804.5. Samples: 920904216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:28:56,062][104569] Avg episode reward: [(0, '8534.996'), (1, '9350.801')] [2023-12-27 04:28:56,075][105692] Updated weights for policy 0, policy_version 1796432 (0.0009) [2023-12-27 04:28:56,129][105692] Updated weights for policy 0, policy_version 1796442 (0.0009) [2023-12-27 04:28:56,643][105620] Updated weights for policy 1, policy_version 1800338 (0.0011) [2023-12-27 04:28:56,703][105620] Updated weights for policy 1, policy_version 1800348 (0.0008) [2023-12-27 04:28:56,764][105692] Updated weights for policy 0, policy_version 1796452 (0.0007) [2023-12-27 04:28:56,769][105620] Updated weights for policy 1, policy_version 1800358 (0.0011) [2023-12-27 04:28:56,811][105692] Updated weights for policy 0, policy_version 1796462 (0.0008) [2023-12-27 04:28:56,870][105692] Updated weights for policy 0, policy_version 1796472 (0.0007) [2023-12-27 04:28:57,402][105620] Updated weights for policy 1, policy_version 1800368 (0.0009) [2023-12-27 04:28:57,449][105620] Updated weights for policy 1, policy_version 1800378 (0.0009) [2023-12-27 04:28:57,506][105620] Updated weights for policy 1, policy_version 1800388 (0.0009) [2023-12-27 04:28:57,655][105692] Updated weights for policy 0, policy_version 1796482 (0.0008) [2023-12-27 04:28:57,702][105692] Updated weights for policy 0, policy_version 1796492 (0.0005) [2023-12-27 04:28:57,755][105692] Updated weights for policy 0, policy_version 1796502 (0.0007) [2023-12-27 04:28:57,811][105692] Updated weights for policy 0, policy_version 1796512 (0.0009) [2023-12-27 04:28:58,248][105620] Updated weights for policy 1, policy_version 1800398 (0.0008) [2023-12-27 04:28:58,309][105620] Updated weights for policy 1, policy_version 1800408 (0.0006) [2023-12-27 04:28:58,380][105620] Updated weights for policy 1, policy_version 1800418 (0.0009) [2023-12-27 04:28:58,498][105692] Updated weights for policy 0, policy_version 1796522 (0.0008) [2023-12-27 04:28:58,555][105692] Updated weights for policy 0, policy_version 1796532 (0.0008) [2023-12-27 04:28:58,620][105692] Updated weights for policy 0, policy_version 1796542 (0.0008) [2023-12-27 04:28:59,151][105620] Updated weights for policy 1, policy_version 1800428 (0.0009) [2023-12-27 04:28:59,204][105620] Updated weights for policy 1, policy_version 1800438 (0.0008) [2023-12-27 04:28:59,267][105620] Updated weights for policy 1, policy_version 1800448 (0.0008) [2023-12-27 04:28:59,456][105692] Updated weights for policy 0, policy_version 1796552 (0.0009) [2023-12-27 04:28:59,524][105692] Updated weights for policy 0, policy_version 1796562 (0.0009) [2023-12-27 04:28:59,585][105692] Updated weights for policy 0, policy_version 1796572 (0.0009) [2023-12-27 04:28:59,940][105620] Updated weights for policy 1, policy_version 1800458 (0.0008) [2023-12-27 04:29:00,000][105620] Updated weights for policy 1, policy_version 1800468 (0.0005) [2023-12-27 04:29:00,060][105620] Updated weights for policy 1, policy_version 1800478 (0.0006) [2023-12-27 04:29:00,120][105620] Updated weights for policy 1, policy_version 1800488 (0.0006) [2023-12-27 04:29:00,477][105692] Updated weights for policy 0, policy_version 1796582 (0.0010) [2023-12-27 04:29:00,528][105692] Updated weights for policy 0, policy_version 1796592 (0.0009) [2023-12-27 04:29:00,579][105692] Updated weights for policy 0, policy_version 1796602 (0.0009) [2023-12-27 04:29:00,708][105620] Updated weights for policy 1, policy_version 1800498 (0.0009) [2023-12-27 04:29:00,762][105620] Updated weights for policy 1, policy_version 1800508 (0.0009) [2023-12-27 04:29:00,808][105620] Updated weights for policy 1, policy_version 1800518 (0.0009) [2023-12-27 04:29:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 920993792. Throughput: 0: 9989.3, 1: 9764.4. Samples: 920963888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:29:01,062][104569] Avg episode reward: [(0, '8353.006'), (1, '9350.813')] [2023-12-27 04:29:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001796608_459997184.pth... [2023-12-27 04:29:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001800520_460996608.pth... [2023-12-27 04:29:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001795488_459710464.pth [2023-12-27 04:29:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001799400_460709888.pth [2023-12-27 04:29:01,283][105692] Updated weights for policy 0, policy_version 1796612 (0.0009) [2023-12-27 04:29:01,349][105692] Updated weights for policy 0, policy_version 1796622 (0.0008) [2023-12-27 04:29:01,419][105692] Updated weights for policy 0, policy_version 1796632 (0.0008) [2023-12-27 04:29:01,664][105620] Updated weights for policy 1, policy_version 1800528 (0.0009) [2023-12-27 04:29:01,730][105620] Updated weights for policy 1, policy_version 1800538 (0.0009) [2023-12-27 04:29:01,784][105620] Updated weights for policy 1, policy_version 1800548 (0.0009) [2023-12-27 04:29:02,099][105692] Updated weights for policy 0, policy_version 1796642 (0.0008) [2023-12-27 04:29:02,154][105692] Updated weights for policy 0, policy_version 1796652 (0.0007) [2023-12-27 04:29:02,218][105692] Updated weights for policy 0, policy_version 1796662 (0.0010) [2023-12-27 04:29:02,270][105692] Updated weights for policy 0, policy_version 1796672 (0.0009) [2023-12-27 04:29:02,427][105620] Updated weights for policy 1, policy_version 1800558 (0.0009) [2023-12-27 04:29:02,474][105620] Updated weights for policy 1, policy_version 1800568 (0.0009) [2023-12-27 04:29:02,527][105620] Updated weights for policy 1, policy_version 1800578 (0.0008) [2023-12-27 04:29:03,009][105692] Updated weights for policy 0, policy_version 1796682 (0.0009) [2023-12-27 04:29:03,076][105692] Updated weights for policy 0, policy_version 1796692 (0.0009) [2023-12-27 04:29:03,137][105692] Updated weights for policy 0, policy_version 1796702 (0.0009) [2023-12-27 04:29:03,299][105620] Updated weights for policy 1, policy_version 1800588 (0.0009) [2023-12-27 04:29:03,345][105620] Updated weights for policy 1, policy_version 1800598 (0.0009) [2023-12-27 04:29:03,391][105620] Updated weights for policy 1, policy_version 1800608 (0.0008) [2023-12-27 04:29:03,810][105692] Updated weights for policy 0, policy_version 1796712 (0.0006) [2023-12-27 04:29:03,868][105692] Updated weights for policy 0, policy_version 1796722 (0.0009) [2023-12-27 04:29:03,929][105692] Updated weights for policy 0, policy_version 1796732 (0.0009) [2023-12-27 04:29:04,073][105620] Updated weights for policy 1, policy_version 1800618 (0.0008) [2023-12-27 04:29:04,136][105620] Updated weights for policy 1, policy_version 1800628 (0.0009) [2023-12-27 04:29:04,191][105620] Updated weights for policy 1, policy_version 1800638 (0.0009) [2023-12-27 04:29:04,250][105620] Updated weights for policy 1, policy_version 1800648 (0.0009) [2023-12-27 04:29:04,682][105692] Updated weights for policy 0, policy_version 1796742 (0.0008) [2023-12-27 04:29:04,739][105692] Updated weights for policy 0, policy_version 1796752 (0.0009) [2023-12-27 04:29:04,802][105692] Updated weights for policy 0, policy_version 1796762 (0.0009) [2023-12-27 04:29:05,019][105620] Updated weights for policy 1, policy_version 1800658 (0.0009) [2023-12-27 04:29:05,083][105620] Updated weights for policy 1, policy_version 1800668 (0.0009) [2023-12-27 04:29:05,136][105620] Updated weights for policy 1, policy_version 1800678 (0.0010) [2023-12-27 04:29:05,444][105692] Updated weights for policy 0, policy_version 1796772 (0.0006) [2023-12-27 04:29:05,505][105692] Updated weights for policy 0, policy_version 1796782 (0.0009) [2023-12-27 04:29:05,566][105692] Updated weights for policy 0, policy_version 1796792 (0.0009) [2023-12-27 04:29:05,968][105620] Updated weights for policy 1, policy_version 1800688 (0.0009) [2023-12-27 04:29:06,026][105620] Updated weights for policy 1, policy_version 1800698 (0.0009) [2023-12-27 04:29:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 921083904. Throughput: 0: 9782.6, 1: 9683.4. Samples: 921078412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:29:06,062][104569] Avg episode reward: [(0, '8445.704'), (1, '9350.852')] [2023-12-27 04:29:06,081][105620] Updated weights for policy 1, policy_version 1800708 (0.0009) [2023-12-27 04:29:06,261][105692] Updated weights for policy 0, policy_version 1796802 (0.0007) [2023-12-27 04:29:06,323][105692] Updated weights for policy 0, policy_version 1796812 (0.0009) [2023-12-27 04:29:06,387][105692] Updated weights for policy 0, policy_version 1796822 (0.0010) [2023-12-27 04:29:06,453][105692] Updated weights for policy 0, policy_version 1796832 (0.0009) [2023-12-27 04:29:06,895][105620] Updated weights for policy 1, policy_version 1800718 (0.0008) [2023-12-27 04:29:06,949][105620] Updated weights for policy 1, policy_version 1800728 (0.0005) [2023-12-27 04:29:07,006][105620] Updated weights for policy 1, policy_version 1800738 (0.0006) [2023-12-27 04:29:07,128][105692] Updated weights for policy 0, policy_version 1796842 (0.0009) [2023-12-27 04:29:07,188][105692] Updated weights for policy 0, policy_version 1796852 (0.0010) [2023-12-27 04:29:07,247][105692] Updated weights for policy 0, policy_version 1796862 (0.0011) [2023-12-27 04:29:07,731][105620] Updated weights for policy 1, policy_version 1800748 (0.0008) [2023-12-27 04:29:07,782][105620] Updated weights for policy 1, policy_version 1800758 (0.0005) [2023-12-27 04:29:07,829][105620] Updated weights for policy 1, policy_version 1800768 (0.0005) [2023-12-27 04:29:08,037][105692] Updated weights for policy 0, policy_version 1796872 (0.0009) [2023-12-27 04:29:08,091][105692] Updated weights for policy 0, policy_version 1796883 (0.0010) [2023-12-27 04:29:08,143][105692] Updated weights for policy 0, policy_version 1796894 (0.0010) [2023-12-27 04:29:08,379][105620] Updated weights for policy 1, policy_version 1800778 (0.0006) [2023-12-27 04:29:08,444][105620] Updated weights for policy 1, policy_version 1800788 (0.0008) [2023-12-27 04:29:08,505][105620] Updated weights for policy 1, policy_version 1800798 (0.0009) [2023-12-27 04:29:08,567][105620] Updated weights for policy 1, policy_version 1800808 (0.0008) [2023-12-27 04:29:08,924][105692] Updated weights for policy 0, policy_version 1796905 (0.0010) [2023-12-27 04:29:08,977][105692] Updated weights for policy 0, policy_version 1796915 (0.0010) [2023-12-27 04:29:09,032][105692] Updated weights for policy 0, policy_version 1796925 (0.0010) [2023-12-27 04:29:09,229][105620] Updated weights for policy 1, policy_version 1800818 (0.0009) [2023-12-27 04:29:09,288][105620] Updated weights for policy 1, policy_version 1800828 (0.0009) [2023-12-27 04:29:09,354][105620] Updated weights for policy 1, policy_version 1800838 (0.0008) [2023-12-27 04:29:09,798][105692] Updated weights for policy 0, policy_version 1796935 (0.0008) [2023-12-27 04:29:09,864][105692] Updated weights for policy 0, policy_version 1796945 (0.0009) [2023-12-27 04:29:09,928][105692] Updated weights for policy 0, policy_version 1796955 (0.0009) [2023-12-27 04:29:10,147][105620] Updated weights for policy 1, policy_version 1800848 (0.0009) [2023-12-27 04:29:10,217][105620] Updated weights for policy 1, policy_version 1800858 (0.0008) [2023-12-27 04:29:10,285][105620] Updated weights for policy 1, policy_version 1800868 (0.0008) [2023-12-27 04:29:10,664][105692] Updated weights for policy 0, policy_version 1796965 (0.0009) [2023-12-27 04:29:10,725][105692] Updated weights for policy 0, policy_version 1796975 (0.0009) [2023-12-27 04:29:10,780][105692] Updated weights for policy 0, policy_version 1796985 (0.0009) [2023-12-27 04:29:10,948][105620] Updated weights for policy 1, policy_version 1800878 (0.0007) [2023-12-27 04:29:10,995][105620] Updated weights for policy 1, policy_version 1800888 (0.0009) [2023-12-27 04:29:11,054][105620] Updated weights for policy 1, policy_version 1800898 (0.0009) [2023-12-27 04:29:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 921182208. Throughput: 0: 9652.7, 1: 9763.6. Samples: 921194216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:29:11,062][104569] Avg episode reward: [(0, '8540.295'), (1, '9167.668')] [2023-12-27 04:29:11,576][105692] Updated weights for policy 0, policy_version 1796995 (0.0009) [2023-12-27 04:29:11,644][105692] Updated weights for policy 0, policy_version 1797005 (0.0009) [2023-12-27 04:29:11,709][105692] Updated weights for policy 0, policy_version 1797015 (0.0007) [2023-12-27 04:29:11,888][105620] Updated weights for policy 1, policy_version 1800908 (0.0009) [2023-12-27 04:29:11,937][105620] Updated weights for policy 1, policy_version 1800918 (0.0009) [2023-12-27 04:29:12,001][105620] Updated weights for policy 1, policy_version 1800928 (0.0009) [2023-12-27 04:29:12,468][105692] Updated weights for policy 0, policy_version 1797025 (0.0010) [2023-12-27 04:29:12,524][105692] Updated weights for policy 0, policy_version 1797035 (0.0008) [2023-12-27 04:29:12,583][105692] Updated weights for policy 0, policy_version 1797045 (0.0009) [2023-12-27 04:29:12,642][105692] Updated weights for policy 0, policy_version 1797055 (0.0009) [2023-12-27 04:29:12,785][105620] Updated weights for policy 1, policy_version 1800938 (0.0009) [2023-12-27 04:29:12,843][105620] Updated weights for policy 1, policy_version 1800948 (0.0009) [2023-12-27 04:29:12,905][105620] Updated weights for policy 1, policy_version 1800958 (0.0009) [2023-12-27 04:29:12,969][105620] Updated weights for policy 1, policy_version 1800968 (0.0005) [2023-12-27 04:29:13,487][105692] Updated weights for policy 0, policy_version 1797065 (0.0008) [2023-12-27 04:29:13,530][105692] Updated weights for policy 0, policy_version 1797075 (0.0007) [2023-12-27 04:29:13,541][105620] Updated weights for policy 1, policy_version 1800978 (0.0007) [2023-12-27 04:29:13,574][105692] Updated weights for policy 0, policy_version 1797085 (0.0007) [2023-12-27 04:29:13,596][105620] Updated weights for policy 1, policy_version 1800988 (0.0008) [2023-12-27 04:29:13,657][105620] Updated weights for policy 1, policy_version 1800998 (0.0008) [2023-12-27 04:29:14,364][105692] Updated weights for policy 0, policy_version 1797095 (0.0007) [2023-12-27 04:29:14,382][105620] Updated weights for policy 1, policy_version 1801008 (0.0008) [2023-12-27 04:29:14,424][105692] Updated weights for policy 0, policy_version 1797105 (0.0007) [2023-12-27 04:29:14,435][105620] Updated weights for policy 1, policy_version 1801018 (0.0006) [2023-12-27 04:29:14,484][105692] Updated weights for policy 0, policy_version 1797115 (0.0008) [2023-12-27 04:29:14,487][105620] Updated weights for policy 1, policy_version 1801028 (0.0006) [2023-12-27 04:29:15,180][105620] Updated weights for policy 1, policy_version 1801038 (0.0008) [2023-12-27 04:29:15,232][105620] Updated weights for policy 1, policy_version 1801048 (0.0009) [2023-12-27 04:29:15,274][105692] Updated weights for policy 0, policy_version 1797125 (0.0006) [2023-12-27 04:29:15,288][105620] Updated weights for policy 1, policy_version 1801058 (0.0009) [2023-12-27 04:29:15,332][105692] Updated weights for policy 0, policy_version 1797135 (0.0007) [2023-12-27 04:29:15,397][105692] Updated weights for policy 0, policy_version 1797145 (0.0008) [2023-12-27 04:29:16,055][105620] Updated weights for policy 1, policy_version 1801068 (0.0006) [2023-12-27 04:29:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 921272320. Throughput: 0: 9549.2, 1: 9685.0. Samples: 921248596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:29:16,062][104569] Avg episode reward: [(0, '8628.944'), (1, '9075.074')] [2023-12-27 04:29:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001797152_460136448.pth... [2023-12-27 04:29:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001796064_459857920.pth [2023-12-27 04:29:16,103][105620] Updated weights for policy 1, policy_version 1801078 (0.0006) [2023-12-27 04:29:16,152][105692] Updated weights for policy 0, policy_version 1797155 (0.0009) [2023-12-27 04:29:16,156][105620] Updated weights for policy 1, policy_version 1801088 (0.0006) [2023-12-27 04:29:16,199][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001801096_461144064.pth... [2023-12-27 04:29:16,202][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001799976_460857344.pth [2023-12-27 04:29:16,209][105692] Updated weights for policy 0, policy_version 1797165 (0.0007) [2023-12-27 04:29:16,275][105692] Updated weights for policy 0, policy_version 1797175 (0.0005) [2023-12-27 04:29:16,834][105620] Updated weights for policy 1, policy_version 1801098 (0.0007) [2023-12-27 04:29:16,892][105620] Updated weights for policy 1, policy_version 1801108 (0.0010) [2023-12-27 04:29:16,954][105620] Updated weights for policy 1, policy_version 1801118 (0.0011) [2023-12-27 04:29:16,975][105692] Updated weights for policy 0, policy_version 1797185 (0.0006) [2023-12-27 04:29:16,999][105620] Updated weights for policy 1, policy_version 1801128 (0.0010) [2023-12-27 04:29:17,038][105692] Updated weights for policy 0, policy_version 1797195 (0.0008) [2023-12-27 04:29:17,104][105692] Updated weights for policy 0, policy_version 1797205 (0.0010) [2023-12-27 04:29:17,169][105692] Updated weights for policy 0, policy_version 1797215 (0.0010) [2023-12-27 04:29:17,589][105620] Updated weights for policy 1, policy_version 1801138 (0.0005) [2023-12-27 04:29:17,649][105620] Updated weights for policy 1, policy_version 1801148 (0.0005) [2023-12-27 04:29:17,707][105620] Updated weights for policy 1, policy_version 1801158 (0.0006) [2023-12-27 04:29:17,855][105692] Updated weights for policy 0, policy_version 1797225 (0.0006) [2023-12-27 04:29:17,908][105692] Updated weights for policy 0, policy_version 1797235 (0.0005) [2023-12-27 04:29:17,959][105692] Updated weights for policy 0, policy_version 1797245 (0.0005) [2023-12-27 04:29:18,319][105620] Updated weights for policy 1, policy_version 1801168 (0.0009) [2023-12-27 04:29:18,384][105620] Updated weights for policy 1, policy_version 1801178 (0.0008) [2023-12-27 04:29:18,453][105620] Updated weights for policy 1, policy_version 1801188 (0.0009) [2023-12-27 04:29:18,551][105692] Updated weights for policy 0, policy_version 1797255 (0.0008) [2023-12-27 04:29:18,609][105692] Updated weights for policy 0, policy_version 1797265 (0.0009) [2023-12-27 04:29:18,676][105692] Updated weights for policy 0, policy_version 1797275 (0.0009) [2023-12-27 04:29:19,130][105620] Updated weights for policy 1, policy_version 1801198 (0.0007) [2023-12-27 04:29:19,176][105620] Updated weights for policy 1, policy_version 1801208 (0.0005) [2023-12-27 04:29:19,226][105620] Updated weights for policy 1, policy_version 1801218 (0.0006) [2023-12-27 04:29:19,522][105692] Updated weights for policy 0, policy_version 1797285 (0.0008) [2023-12-27 04:29:19,572][105692] Updated weights for policy 0, policy_version 1797295 (0.0007) [2023-12-27 04:29:19,626][105692] Updated weights for policy 0, policy_version 1797305 (0.0006) [2023-12-27 04:29:19,835][105620] Updated weights for policy 1, policy_version 1801228 (0.0008) [2023-12-27 04:29:19,905][105620] Updated weights for policy 1, policy_version 1801238 (0.0008) [2023-12-27 04:29:19,974][105620] Updated weights for policy 1, policy_version 1801248 (0.0010) [2023-12-27 04:29:20,283][105692] Updated weights for policy 0, policy_version 1797315 (0.0007) [2023-12-27 04:29:20,345][105692] Updated weights for policy 0, policy_version 1797325 (0.0009) [2023-12-27 04:29:20,392][105692] Updated weights for policy 0, policy_version 1797335 (0.0009) [2023-12-27 04:29:20,680][105620] Updated weights for policy 1, policy_version 1801258 (0.0008) [2023-12-27 04:29:20,747][105620] Updated weights for policy 1, policy_version 1801268 (0.0006) [2023-12-27 04:29:20,816][105620] Updated weights for policy 1, policy_version 1801278 (0.0006) [2023-12-27 04:29:20,885][105620] Updated weights for policy 1, policy_version 1801288 (0.0006) [2023-12-27 04:29:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 921378816. Throughput: 0: 9518.7, 1: 9711.2. Samples: 921367964. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:29:21,062][104569] Avg episode reward: [(0, '8814.409'), (1, '9258.121')] [2023-12-27 04:29:21,229][105692] Updated weights for policy 0, policy_version 1797345 (0.0009) [2023-12-27 04:29:21,295][105692] Updated weights for policy 0, policy_version 1797355 (0.0008) [2023-12-27 04:29:21,365][105692] Updated weights for policy 0, policy_version 1797365 (0.0008) [2023-12-27 04:29:21,424][105692] Updated weights for policy 0, policy_version 1797375 (0.0008) [2023-12-27 04:29:21,540][105620] Updated weights for policy 1, policy_version 1801298 (0.0009) [2023-12-27 04:29:21,592][105620] Updated weights for policy 1, policy_version 1801308 (0.0008) [2023-12-27 04:29:21,651][105620] Updated weights for policy 1, policy_version 1801318 (0.0008) [2023-12-27 04:29:22,164][105692] Updated weights for policy 0, policy_version 1797385 (0.0008) [2023-12-27 04:29:22,226][105692] Updated weights for policy 0, policy_version 1797395 (0.0009) [2023-12-27 04:29:22,293][105692] Updated weights for policy 0, policy_version 1797405 (0.0010) [2023-12-27 04:29:22,471][105620] Updated weights for policy 1, policy_version 1801328 (0.0010) [2023-12-27 04:29:22,538][105620] Updated weights for policy 1, policy_version 1801338 (0.0009) [2023-12-27 04:29:22,600][105620] Updated weights for policy 1, policy_version 1801348 (0.0009) [2023-12-27 04:29:23,047][105692] Updated weights for policy 0, policy_version 1797415 (0.0009) [2023-12-27 04:29:23,105][105692] Updated weights for policy 0, policy_version 1797425 (0.0008) [2023-12-27 04:29:23,167][105692] Updated weights for policy 0, policy_version 1797435 (0.0009) [2023-12-27 04:29:23,348][105620] Updated weights for policy 1, policy_version 1801358 (0.0009) [2023-12-27 04:29:23,399][105620] Updated weights for policy 1, policy_version 1801368 (0.0009) [2023-12-27 04:29:23,456][105620] Updated weights for policy 1, policy_version 1801378 (0.0009) [2023-12-27 04:29:23,911][105692] Updated weights for policy 0, policy_version 1797445 (0.0009) [2023-12-27 04:29:23,973][105692] Updated weights for policy 0, policy_version 1797455 (0.0009) [2023-12-27 04:29:24,031][105692] Updated weights for policy 0, policy_version 1797465 (0.0009) [2023-12-27 04:29:24,220][105620] Updated weights for policy 1, policy_version 1801388 (0.0009) [2023-12-27 04:29:24,270][105620] Updated weights for policy 1, policy_version 1801398 (0.0008) [2023-12-27 04:29:24,325][105620] Updated weights for policy 1, policy_version 1801408 (0.0009) [2023-12-27 04:29:24,776][105692] Updated weights for policy 0, policy_version 1797475 (0.0008) [2023-12-27 04:29:24,827][105692] Updated weights for policy 0, policy_version 1797485 (0.0008) [2023-12-27 04:29:24,891][105692] Updated weights for policy 0, policy_version 1797495 (0.0008) [2023-12-27 04:29:25,109][105620] Updated weights for policy 1, policy_version 1801418 (0.0009) [2023-12-27 04:29:25,161][105620] Updated weights for policy 1, policy_version 1801428 (0.0010) [2023-12-27 04:29:25,209][105620] Updated weights for policy 1, policy_version 1801438 (0.0008) [2023-12-27 04:29:25,260][105620] Updated weights for policy 1, policy_version 1801448 (0.0008) [2023-12-27 04:29:25,661][105692] Updated weights for policy 0, policy_version 1797505 (0.0009) [2023-12-27 04:29:25,718][105692] Updated weights for policy 0, policy_version 1797515 (0.0010) [2023-12-27 04:29:25,771][105692] Updated weights for policy 0, policy_version 1797526 (0.0010) [2023-12-27 04:29:25,826][105692] Updated weights for policy 0, policy_version 1797536 (0.0010) [2023-12-27 04:29:25,951][105620] Updated weights for policy 1, policy_version 1801458 (0.0006) [2023-12-27 04:29:26,009][105620] Updated weights for policy 1, policy_version 1801468 (0.0005) [2023-12-27 04:29:26,062][105620] Updated weights for policy 1, policy_version 1801478 (0.0010) [2023-12-27 04:29:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 921468928. Throughput: 0: 9462.4, 1: 9672.1. Samples: 921480516. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:29:26,062][104569] Avg episode reward: [(0, '8994.926'), (1, '9350.553')] [2023-12-27 04:29:26,658][105620] Updated weights for policy 1, policy_version 1801488 (0.0006) [2023-12-27 04:29:26,693][105692] Updated weights for policy 0, policy_version 1797546 (0.0009) [2023-12-27 04:29:26,703][105620] Updated weights for policy 1, policy_version 1801498 (0.0005) [2023-12-27 04:29:26,742][105692] Updated weights for policy 0, policy_version 1797556 (0.0009) [2023-12-27 04:29:26,749][105620] Updated weights for policy 1, policy_version 1801508 (0.0005) [2023-12-27 04:29:26,792][105692] Updated weights for policy 0, policy_version 1797566 (0.0009) [2023-12-27 04:29:27,402][105620] Updated weights for policy 1, policy_version 1801518 (0.0008) [2023-12-27 04:29:27,462][105620] Updated weights for policy 1, policy_version 1801528 (0.0006) [2023-12-27 04:29:27,517][105620] Updated weights for policy 1, policy_version 1801538 (0.0005) [2023-12-27 04:29:27,644][105692] Updated weights for policy 0, policy_version 1797576 (0.0010) [2023-12-27 04:29:27,702][105692] Updated weights for policy 0, policy_version 1797586 (0.0014) [2023-12-27 04:29:27,767][105692] Updated weights for policy 0, policy_version 1797596 (0.0009) [2023-12-27 04:29:28,117][105620] Updated weights for policy 1, policy_version 1801548 (0.0007) [2023-12-27 04:29:28,167][105620] Updated weights for policy 1, policy_version 1801558 (0.0008) [2023-12-27 04:29:28,225][105620] Updated weights for policy 1, policy_version 1801568 (0.0009) [2023-12-27 04:29:28,546][105692] Updated weights for policy 0, policy_version 1797606 (0.0010) [2023-12-27 04:29:28,609][105692] Updated weights for policy 0, policy_version 1797616 (0.0009) [2023-12-27 04:29:28,669][105692] Updated weights for policy 0, policy_version 1797626 (0.0009) [2023-12-27 04:29:28,963][105620] Updated weights for policy 1, policy_version 1801578 (0.0008) [2023-12-27 04:29:29,025][105620] Updated weights for policy 1, policy_version 1801588 (0.0005) [2023-12-27 04:29:29,083][105620] Updated weights for policy 1, policy_version 1801598 (0.0005) [2023-12-27 04:29:29,141][105620] Updated weights for policy 1, policy_version 1801608 (0.0005) [2023-12-27 04:29:29,370][105692] Updated weights for policy 0, policy_version 1797636 (0.0009) [2023-12-27 04:29:29,434][105692] Updated weights for policy 0, policy_version 1797646 (0.0008) [2023-12-27 04:29:29,501][105692] Updated weights for policy 0, policy_version 1797656 (0.0010) [2023-12-27 04:29:29,725][105620] Updated weights for policy 1, policy_version 1801618 (0.0010) [2023-12-27 04:29:29,786][105620] Updated weights for policy 1, policy_version 1801628 (0.0010) [2023-12-27 04:29:29,847][105620] Updated weights for policy 1, policy_version 1801638 (0.0010) [2023-12-27 04:29:30,278][105692] Updated weights for policy 0, policy_version 1797666 (0.0010) [2023-12-27 04:29:30,340][105692] Updated weights for policy 0, policy_version 1797676 (0.0009) [2023-12-27 04:29:30,397][105692] Updated weights for policy 0, policy_version 1797686 (0.0009) [2023-12-27 04:29:30,455][105692] Updated weights for policy 0, policy_version 1797696 (0.0008) [2023-12-27 04:29:30,587][105620] Updated weights for policy 1, policy_version 1801648 (0.0009) [2023-12-27 04:29:30,640][105620] Updated weights for policy 1, policy_version 1801658 (0.0007) [2023-12-27 04:29:30,695][105620] Updated weights for policy 1, policy_version 1801668 (0.0007) [2023-12-27 04:29:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 921567232. Throughput: 0: 9448.7, 1: 9734.6. Samples: 921538548. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:29:31,062][104569] Avg episode reward: [(0, '8810.139'), (1, '9350.603')] [2023-12-27 04:29:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001797696_460275712.pth... [2023-12-27 04:29:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001801672_461291520.pth... [2023-12-27 04:29:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001796608_459997184.pth [2023-12-27 04:29:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001800520_460996608.pth [2023-12-27 04:29:31,248][105692] Updated weights for policy 0, policy_version 1797706 (0.0006) [2023-12-27 04:29:31,318][105692] Updated weights for policy 0, policy_version 1797716 (0.0008) [2023-12-27 04:29:31,381][105692] Updated weights for policy 0, policy_version 1797726 (0.0009) [2023-12-27 04:29:31,410][105620] Updated weights for policy 1, policy_version 1801678 (0.0007) [2023-12-27 04:29:31,466][105620] Updated weights for policy 1, policy_version 1801688 (0.0005) [2023-12-27 04:29:31,527][105620] Updated weights for policy 1, policy_version 1801698 (0.0008) [2023-12-27 04:29:32,136][105620] Updated weights for policy 1, policy_version 1801708 (0.0008) [2023-12-27 04:29:32,162][105692] Updated weights for policy 0, policy_version 1797736 (0.0007) [2023-12-27 04:29:32,193][105620] Updated weights for policy 1, policy_version 1801718 (0.0006) [2023-12-27 04:29:32,222][105692] Updated weights for policy 0, policy_version 1797746 (0.0008) [2023-12-27 04:29:32,249][105620] Updated weights for policy 1, policy_version 1801728 (0.0006) [2023-12-27 04:29:32,287][105692] Updated weights for policy 0, policy_version 1797756 (0.0008) [2023-12-27 04:29:32,999][105620] Updated weights for policy 1, policy_version 1801738 (0.0007) [2023-12-27 04:29:33,037][105692] Updated weights for policy 0, policy_version 1797766 (0.0006) [2023-12-27 04:29:33,048][105620] Updated weights for policy 1, policy_version 1801748 (0.0007) [2023-12-27 04:29:33,082][105692] Updated weights for policy 0, policy_version 1797776 (0.0005) [2023-12-27 04:29:33,100][105620] Updated weights for policy 1, policy_version 1801758 (0.0007) [2023-12-27 04:29:33,130][105692] Updated weights for policy 0, policy_version 1797786 (0.0006) [2023-12-27 04:29:33,152][105620] Updated weights for policy 1, policy_version 1801768 (0.0007) [2023-12-27 04:29:33,879][105620] Updated weights for policy 1, policy_version 1801778 (0.0009) [2023-12-27 04:29:33,893][105692] Updated weights for policy 0, policy_version 1797796 (0.0008) [2023-12-27 04:29:33,927][105620] Updated weights for policy 1, policy_version 1801788 (0.0006) [2023-12-27 04:29:33,945][105692] Updated weights for policy 0, policy_version 1797806 (0.0007) [2023-12-27 04:29:33,984][105620] Updated weights for policy 1, policy_version 1801798 (0.0008) [2023-12-27 04:29:33,994][105692] Updated weights for policy 0, policy_version 1797816 (0.0007) [2023-12-27 04:29:34,701][105620] Updated weights for policy 1, policy_version 1801808 (0.0009) [2023-12-27 04:29:34,758][105620] Updated weights for policy 1, policy_version 1801818 (0.0005) [2023-12-27 04:29:34,799][105692] Updated weights for policy 0, policy_version 1797826 (0.0008) [2023-12-27 04:29:34,816][105620] Updated weights for policy 1, policy_version 1801828 (0.0008) [2023-12-27 04:29:34,865][105692] Updated weights for policy 0, policy_version 1797836 (0.0007) [2023-12-27 04:29:34,926][105692] Updated weights for policy 0, policy_version 1797846 (0.0008) [2023-12-27 04:29:34,985][105692] Updated weights for policy 0, policy_version 1797856 (0.0009) [2023-12-27 04:29:35,504][105620] Updated weights for policy 1, policy_version 1801838 (0.0007) [2023-12-27 04:29:35,559][105620] Updated weights for policy 1, policy_version 1801848 (0.0009) [2023-12-27 04:29:35,610][105620] Updated weights for policy 1, policy_version 1801858 (0.0009) [2023-12-27 04:29:35,765][105692] Updated weights for policy 0, policy_version 1797866 (0.0010) [2023-12-27 04:29:35,820][105692] Updated weights for policy 0, policy_version 1797876 (0.0010) [2023-12-27 04:29:35,872][105692] Updated weights for policy 0, policy_version 1797886 (0.0009) [2023-12-27 04:29:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 921665536. Throughput: 0: 9425.9, 1: 9758.9. Samples: 921653776. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:29:36,063][104569] Avg episode reward: [(0, '8716.216'), (1, '9350.551')] [2023-12-27 04:29:36,240][105620] Updated weights for policy 1, policy_version 1801868 (0.0008) [2023-12-27 04:29:36,307][105620] Updated weights for policy 1, policy_version 1801878 (0.0009) [2023-12-27 04:29:36,372][105620] Updated weights for policy 1, policy_version 1801888 (0.0009) [2023-12-27 04:29:36,720][105692] Updated weights for policy 0, policy_version 1797897 (0.0009) [2023-12-27 04:29:36,779][105692] Updated weights for policy 0, policy_version 1797907 (0.0009) [2023-12-27 04:29:36,835][105692] Updated weights for policy 0, policy_version 1797917 (0.0009) [2023-12-27 04:29:37,134][105620] Updated weights for policy 1, policy_version 1801898 (0.0009) [2023-12-27 04:29:37,198][105620] Updated weights for policy 1, policy_version 1801908 (0.0008) [2023-12-27 04:29:37,254][105620] Updated weights for policy 1, policy_version 1801918 (0.0006) [2023-12-27 04:29:37,311][105620] Updated weights for policy 1, policy_version 1801928 (0.0005) [2023-12-27 04:29:37,587][105692] Updated weights for policy 0, policy_version 1797927 (0.0008) [2023-12-27 04:29:37,648][105692] Updated weights for policy 0, policy_version 1797937 (0.0008) [2023-12-27 04:29:37,716][105692] Updated weights for policy 0, policy_version 1797947 (0.0009) [2023-12-27 04:29:37,969][105620] Updated weights for policy 1, policy_version 1801938 (0.0005) [2023-12-27 04:29:38,024][105620] Updated weights for policy 1, policy_version 1801948 (0.0009) [2023-12-27 04:29:38,080][105620] Updated weights for policy 1, policy_version 1801958 (0.0009) [2023-12-27 04:29:38,508][105692] Updated weights for policy 0, policy_version 1797957 (0.0009) [2023-12-27 04:29:38,573][105692] Updated weights for policy 0, policy_version 1797967 (0.0009) [2023-12-27 04:29:38,639][105692] Updated weights for policy 0, policy_version 1797977 (0.0009) [2023-12-27 04:29:38,769][105620] Updated weights for policy 1, policy_version 1801968 (0.0009) [2023-12-27 04:29:38,831][105620] Updated weights for policy 1, policy_version 1801978 (0.0009) [2023-12-27 04:29:38,892][105620] Updated weights for policy 1, policy_version 1801988 (0.0009) [2023-12-27 04:29:39,412][105692] Updated weights for policy 0, policy_version 1797987 (0.0009) [2023-12-27 04:29:39,477][105692] Updated weights for policy 0, policy_version 1797997 (0.0008) [2023-12-27 04:29:39,547][105692] Updated weights for policy 0, policy_version 1798007 (0.0008) [2023-12-27 04:29:39,654][105620] Updated weights for policy 1, policy_version 1801998 (0.0007) [2023-12-27 04:29:39,723][105620] Updated weights for policy 1, policy_version 1802008 (0.0007) [2023-12-27 04:29:39,786][105620] Updated weights for policy 1, policy_version 1802018 (0.0008) [2023-12-27 04:29:40,309][105692] Updated weights for policy 0, policy_version 1798017 (0.0009) [2023-12-27 04:29:40,362][105692] Updated weights for policy 0, policy_version 1798027 (0.0009) [2023-12-27 04:29:40,411][105692] Updated weights for policy 0, policy_version 1798037 (0.0009) [2023-12-27 04:29:40,466][105692] Updated weights for policy 0, policy_version 1798047 (0.0009) [2023-12-27 04:29:40,528][105620] Updated weights for policy 1, policy_version 1802028 (0.0009) [2023-12-27 04:29:40,585][105620] Updated weights for policy 1, policy_version 1802038 (0.0009) [2023-12-27 04:29:40,640][105620] Updated weights for policy 1, policy_version 1802048 (0.0008) [2023-12-27 04:29:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19522.0). Total num frames: 921755648. Throughput: 0: 9266.5, 1: 9892.4. Samples: 921766364. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:29:41,062][104569] Avg episode reward: [(0, '8988.359'), (1, '9258.282')] [2023-12-27 04:29:41,306][105692] Updated weights for policy 0, policy_version 1798057 (0.0010) [2023-12-27 04:29:41,341][105620] Updated weights for policy 1, policy_version 1802058 (0.0006) [2023-12-27 04:29:41,368][105692] Updated weights for policy 0, policy_version 1798067 (0.0011) [2023-12-27 04:29:41,416][105620] Updated weights for policy 1, policy_version 1802068 (0.0008) [2023-12-27 04:29:41,435][105692] Updated weights for policy 0, policy_version 1798077 (0.0008) [2023-12-27 04:29:41,476][105620] Updated weights for policy 1, policy_version 1802078 (0.0006) [2023-12-27 04:29:41,535][105620] Updated weights for policy 1, policy_version 1802088 (0.0006) [2023-12-27 04:29:42,179][105692] Updated weights for policy 0, policy_version 1798087 (0.0008) [2023-12-27 04:29:42,230][105692] Updated weights for policy 0, policy_version 1798097 (0.0008) [2023-12-27 04:29:42,295][105692] Updated weights for policy 0, policy_version 1798107 (0.0008) [2023-12-27 04:29:42,298][105620] Updated weights for policy 1, policy_version 1802098 (0.0010) [2023-12-27 04:29:42,359][105620] Updated weights for policy 1, policy_version 1802108 (0.0008) [2023-12-27 04:29:42,424][105620] Updated weights for policy 1, policy_version 1802118 (0.0009) [2023-12-27 04:29:43,080][105692] Updated weights for policy 0, policy_version 1798117 (0.0009) [2023-12-27 04:29:43,144][105692] Updated weights for policy 0, policy_version 1798127 (0.0007) [2023-12-27 04:29:43,174][105620] Updated weights for policy 1, policy_version 1802128 (0.0010) [2023-12-27 04:29:43,203][105692] Updated weights for policy 0, policy_version 1798137 (0.0009) [2023-12-27 04:29:43,234][105620] Updated weights for policy 1, policy_version 1802138 (0.0008) [2023-12-27 04:29:43,296][105620] Updated weights for policy 1, policy_version 1802148 (0.0008) [2023-12-27 04:29:43,838][105692] Updated weights for policy 0, policy_version 1798147 (0.0008) [2023-12-27 04:29:43,887][105692] Updated weights for policy 0, policy_version 1798157 (0.0010) [2023-12-27 04:29:43,934][105692] Updated weights for policy 0, policy_version 1798167 (0.0010) [2023-12-27 04:29:44,024][105620] Updated weights for policy 1, policy_version 1802158 (0.0007) [2023-12-27 04:29:44,072][105620] Updated weights for policy 1, policy_version 1802168 (0.0008) [2023-12-27 04:29:44,121][105620] Updated weights for policy 1, policy_version 1802178 (0.0007) [2023-12-27 04:29:44,617][105692] Updated weights for policy 0, policy_version 1798177 (0.0010) [2023-12-27 04:29:44,685][105692] Updated weights for policy 0, policy_version 1798187 (0.0006) [2023-12-27 04:29:44,750][105692] Updated weights for policy 0, policy_version 1798197 (0.0005) [2023-12-27 04:29:44,826][105620] Updated weights for policy 1, policy_version 1802188 (0.0007) [2023-12-27 04:29:44,887][105620] Updated weights for policy 1, policy_version 1802198 (0.0010) [2023-12-27 04:29:44,940][105620] Updated weights for policy 1, policy_version 1802208 (0.0009) [2023-12-27 04:29:45,425][105692] Updated weights for policy 0, policy_version 1798209 (0.0008) [2023-12-27 04:29:45,493][105692] Updated weights for policy 0, policy_version 1798219 (0.0009) [2023-12-27 04:29:45,545][105692] Updated weights for policy 0, policy_version 1798229 (0.0009) [2023-12-27 04:29:45,593][105692] Updated weights for policy 0, policy_version 1798239 (0.0009) [2023-12-27 04:29:45,728][105620] Updated weights for policy 1, policy_version 1802218 (0.0009) [2023-12-27 04:29:45,777][105620] Updated weights for policy 1, policy_version 1802228 (0.0009) [2023-12-27 04:29:45,823][105620] Updated weights for policy 1, policy_version 1802238 (0.0008) [2023-12-27 04:29:45,869][105620] Updated weights for policy 1, policy_version 1802248 (0.0008) [2023-12-27 04:29:46,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 921853952. Throughput: 0: 9201.8, 1: 9864.1. Samples: 921821852. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:29:46,062][104569] Avg episode reward: [(0, '8902.004'), (1, '9258.400')] [2023-12-27 04:29:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001802248_461438976.pth... [2023-12-27 04:29:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001798240_460414976.pth... [2023-12-27 04:29:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001801096_461144064.pth [2023-12-27 04:29:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001797152_460136448.pth [2023-12-27 04:29:46,371][105692] Updated weights for policy 0, policy_version 1798249 (0.0009) [2023-12-27 04:29:46,426][105692] Updated weights for policy 0, policy_version 1798259 (0.0009) [2023-12-27 04:29:46,482][105692] Updated weights for policy 0, policy_version 1798269 (0.0009) [2023-12-27 04:29:46,641][105620] Updated weights for policy 1, policy_version 1802258 (0.0009) [2023-12-27 04:29:46,689][105620] Updated weights for policy 1, policy_version 1802268 (0.0008) [2023-12-27 04:29:46,737][105620] Updated weights for policy 1, policy_version 1802278 (0.0009) [2023-12-27 04:29:47,268][105692] Updated weights for policy 0, policy_version 1798279 (0.0009) [2023-12-27 04:29:47,315][105692] Updated weights for policy 0, policy_version 1798289 (0.0009) [2023-12-27 04:29:47,363][105692] Updated weights for policy 0, policy_version 1798299 (0.0009) [2023-12-27 04:29:47,442][105620] Updated weights for policy 1, policy_version 1802288 (0.0009) [2023-12-27 04:29:47,496][105620] Updated weights for policy 1, policy_version 1802298 (0.0008) [2023-12-27 04:29:47,543][105620] Updated weights for policy 1, policy_version 1802308 (0.0009) [2023-12-27 04:29:48,076][105692] Updated weights for policy 0, policy_version 1798309 (0.0007) [2023-12-27 04:29:48,122][105692] Updated weights for policy 0, policy_version 1798319 (0.0005) [2023-12-27 04:29:48,175][105692] Updated weights for policy 0, policy_version 1798329 (0.0006) [2023-12-27 04:29:48,343][105620] Updated weights for policy 1, policy_version 1802318 (0.0007) [2023-12-27 04:29:48,402][105620] Updated weights for policy 1, policy_version 1802328 (0.0006) [2023-12-27 04:29:48,458][105620] Updated weights for policy 1, policy_version 1802338 (0.0007) [2023-12-27 04:29:48,907][105692] Updated weights for policy 0, policy_version 1798339 (0.0008) [2023-12-27 04:29:48,973][105692] Updated weights for policy 0, policy_version 1798349 (0.0008) [2023-12-27 04:29:49,032][105692] Updated weights for policy 0, policy_version 1798359 (0.0009) [2023-12-27 04:29:49,185][105620] Updated weights for policy 1, policy_version 1802348 (0.0009) [2023-12-27 04:29:49,249][105620] Updated weights for policy 1, policy_version 1802358 (0.0007) [2023-12-27 04:29:49,327][105620] Updated weights for policy 1, policy_version 1802368 (0.0008) [2023-12-27 04:29:49,766][105692] Updated weights for policy 0, policy_version 1798369 (0.0009) [2023-12-27 04:29:49,825][105692] Updated weights for policy 0, policy_version 1798379 (0.0006) [2023-12-27 04:29:49,885][105692] Updated weights for policy 0, policy_version 1798389 (0.0009) [2023-12-27 04:29:49,956][105692] Updated weights for policy 0, policy_version 1798399 (0.0008) [2023-12-27 04:29:50,110][105620] Updated weights for policy 1, policy_version 1802378 (0.0009) [2023-12-27 04:29:50,181][105620] Updated weights for policy 1, policy_version 1802388 (0.0010) [2023-12-27 04:29:50,247][105620] Updated weights for policy 1, policy_version 1802398 (0.0009) [2023-12-27 04:29:50,321][105620] Updated weights for policy 1, policy_version 1802408 (0.0010) [2023-12-27 04:29:50,596][105692] Updated weights for policy 0, policy_version 1798409 (0.0008) [2023-12-27 04:29:50,660][105692] Updated weights for policy 0, policy_version 1798419 (0.0006) [2023-12-27 04:29:50,725][105692] Updated weights for policy 0, policy_version 1798429 (0.0007) [2023-12-27 04:29:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.7, 300 sec: 19494.2). Total num frames: 921944064. Throughput: 0: 9255.5, 1: 9809.9. Samples: 921936352. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:29:51,063][104569] Avg episode reward: [(0, '8630.446'), (1, '9350.901')] [2023-12-27 04:29:51,208][105620] Updated weights for policy 1, policy_version 1802418 (0.0010) [2023-12-27 04:29:51,266][105620] Updated weights for policy 1, policy_version 1802428 (0.0009) [2023-12-27 04:29:51,331][105620] Updated weights for policy 1, policy_version 1802438 (0.0007) [2023-12-27 04:29:51,340][105692] Updated weights for policy 0, policy_version 1798439 (0.0007) [2023-12-27 04:29:51,404][105692] Updated weights for policy 0, policy_version 1798449 (0.0008) [2023-12-27 04:29:51,460][105692] Updated weights for policy 0, policy_version 1798459 (0.0010) [2023-12-27 04:29:52,045][105620] Updated weights for policy 1, policy_version 1802448 (0.0009) [2023-12-27 04:29:52,099][105620] Updated weights for policy 1, policy_version 1802458 (0.0010) [2023-12-27 04:29:52,166][105620] Updated weights for policy 1, policy_version 1802468 (0.0008) [2023-12-27 04:29:52,211][105692] Updated weights for policy 0, policy_version 1798469 (0.0007) [2023-12-27 04:29:52,273][105692] Updated weights for policy 0, policy_version 1798479 (0.0006) [2023-12-27 04:29:52,334][105692] Updated weights for policy 0, policy_version 1798489 (0.0008) [2023-12-27 04:29:52,814][105620] Updated weights for policy 1, policy_version 1802478 (0.0008) [2023-12-27 04:29:52,877][105620] Updated weights for policy 1, policy_version 1802488 (0.0009) [2023-12-27 04:29:52,926][105620] Updated weights for policy 1, policy_version 1802498 (0.0009) [2023-12-27 04:29:53,118][105692] Updated weights for policy 0, policy_version 1798499 (0.0009) [2023-12-27 04:29:53,173][105692] Updated weights for policy 0, policy_version 1798510 (0.0009) [2023-12-27 04:29:53,235][105692] Updated weights for policy 0, policy_version 1798520 (0.0008) [2023-12-27 04:29:53,658][105620] Updated weights for policy 1, policy_version 1802508 (0.0009) [2023-12-27 04:29:53,717][105620] Updated weights for policy 1, policy_version 1802518 (0.0010) [2023-12-27 04:29:53,770][105620] Updated weights for policy 1, policy_version 1802528 (0.0009) [2023-12-27 04:29:53,788][105692] Updated weights for policy 0, policy_version 1798530 (0.0006) [2023-12-27 04:29:53,832][105692] Updated weights for policy 0, policy_version 1798540 (0.0010) [2023-12-27 04:29:53,876][105692] Updated weights for policy 0, policy_version 1798550 (0.0010) [2023-12-27 04:29:53,921][105692] Updated weights for policy 0, policy_version 1798560 (0.0010) [2023-12-27 04:29:54,527][105692] Updated weights for policy 0, policy_version 1798570 (0.0005) [2023-12-27 04:29:54,575][105692] Updated weights for policy 0, policy_version 1798580 (0.0005) [2023-12-27 04:29:54,627][105692] Updated weights for policy 0, policy_version 1798590 (0.0005) [2023-12-27 04:29:54,655][105620] Updated weights for policy 1, policy_version 1802538 (0.0008) [2023-12-27 04:29:54,721][105620] Updated weights for policy 1, policy_version 1802548 (0.0009) [2023-12-27 04:29:54,783][105620] Updated weights for policy 1, policy_version 1802558 (0.0010) [2023-12-27 04:29:54,841][105620] Updated weights for policy 1, policy_version 1802568 (0.0009) [2023-12-27 04:29:55,210][105692] Updated weights for policy 0, policy_version 1798600 (0.0007) [2023-12-27 04:29:55,260][105692] Updated weights for policy 0, policy_version 1798610 (0.0005) [2023-12-27 04:29:55,313][105692] Updated weights for policy 0, policy_version 1798620 (0.0005) [2023-12-27 04:29:55,700][105620] Updated weights for policy 1, policy_version 1802578 (0.0009) [2023-12-27 04:29:55,758][105620] Updated weights for policy 1, policy_version 1802588 (0.0008) [2023-12-27 04:29:55,816][105620] Updated weights for policy 1, policy_version 1802598 (0.0008) [2023-12-27 04:29:55,936][105692] Updated weights for policy 0, policy_version 1798630 (0.0008) [2023-12-27 04:29:55,995][105692] Updated weights for policy 0, policy_version 1798640 (0.0009) [2023-12-27 04:29:56,055][105692] Updated weights for policy 0, policy_version 1798650 (0.0009) [2023-12-27 04:29:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19549.7). Total num frames: 922042368. Throughput: 0: 9392.5, 1: 9695.2. Samples: 922053164. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:29:56,062][104569] Avg episode reward: [(0, '8443.603'), (1, '9351.055')] [2023-12-27 04:29:56,571][105620] Updated weights for policy 1, policy_version 1802608 (0.0009) [2023-12-27 04:29:56,629][105620] Updated weights for policy 1, policy_version 1802618 (0.0009) [2023-12-27 04:29:56,679][105620] Updated weights for policy 1, policy_version 1802628 (0.0008) [2023-12-27 04:29:56,816][105692] Updated weights for policy 0, policy_version 1798660 (0.0009) [2023-12-27 04:29:56,863][105692] Updated weights for policy 0, policy_version 1798670 (0.0009) [2023-12-27 04:29:56,910][105692] Updated weights for policy 0, policy_version 1798680 (0.0009) [2023-12-27 04:29:57,433][105620] Updated weights for policy 1, policy_version 1802638 (0.0009) [2023-12-27 04:29:57,480][105620] Updated weights for policy 1, policy_version 1802648 (0.0009) [2023-12-27 04:29:57,536][105620] Updated weights for policy 1, policy_version 1802658 (0.0009) [2023-12-27 04:29:57,642][105692] Updated weights for policy 0, policy_version 1798690 (0.0008) [2023-12-27 04:29:57,697][105692] Updated weights for policy 0, policy_version 1798700 (0.0005) [2023-12-27 04:29:57,754][105692] Updated weights for policy 0, policy_version 1798710 (0.0005) [2023-12-27 04:29:57,813][105692] Updated weights for policy 0, policy_version 1798720 (0.0005) [2023-12-27 04:29:58,374][105620] Updated weights for policy 1, policy_version 1802668 (0.0008) [2023-12-27 04:29:58,414][105692] Updated weights for policy 0, policy_version 1798730 (0.0007) [2023-12-27 04:29:58,434][105620] Updated weights for policy 1, policy_version 1802678 (0.0008) [2023-12-27 04:29:58,482][105692] Updated weights for policy 0, policy_version 1798740 (0.0008) [2023-12-27 04:29:58,499][105620] Updated weights for policy 1, policy_version 1802688 (0.0009) [2023-12-27 04:29:58,546][105692] Updated weights for policy 0, policy_version 1798750 (0.0009) [2023-12-27 04:29:59,384][105620] Updated weights for policy 1, policy_version 1802698 (0.0009) [2023-12-27 04:29:59,437][105692] Updated weights for policy 0, policy_version 1798760 (0.0010) [2023-12-27 04:29:59,446][105620] Updated weights for policy 1, policy_version 1802708 (0.0009) [2023-12-27 04:29:59,497][105692] Updated weights for policy 0, policy_version 1798770 (0.0006) [2023-12-27 04:29:59,499][105620] Updated weights for policy 1, policy_version 1802718 (0.0010) [2023-12-27 04:29:59,556][105620] Updated weights for policy 1, policy_version 1802728 (0.0010) [2023-12-27 04:29:59,556][105692] Updated weights for policy 0, policy_version 1798780 (0.0006) [2023-12-27 04:30:00,225][105692] Updated weights for policy 0, policy_version 1798790 (0.0005) [2023-12-27 04:30:00,278][105692] Updated weights for policy 0, policy_version 1798800 (0.0009) [2023-12-27 04:30:00,328][105692] Updated weights for policy 0, policy_version 1798810 (0.0005) [2023-12-27 04:30:00,352][105620] Updated weights for policy 1, policy_version 1802738 (0.0011) [2023-12-27 04:30:00,405][105620] Updated weights for policy 1, policy_version 1802748 (0.0010) [2023-12-27 04:30:00,462][105620] Updated weights for policy 1, policy_version 1802758 (0.0010) [2023-12-27 04:30:00,895][105692] Updated weights for policy 0, policy_version 1798820 (0.0008) [2023-12-27 04:30:00,939][105692] Updated weights for policy 0, policy_version 1798830 (0.0010) [2023-12-27 04:30:00,987][105692] Updated weights for policy 0, policy_version 1798840 (0.0010) [2023-12-27 04:30:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19114.7, 300 sec: 19522.0). Total num frames: 922140672. Throughput: 0: 9473.0, 1: 9661.4. Samples: 922109644. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:30:01,062][104569] Avg episode reward: [(0, '8175.603'), (1, '9351.124')] [2023-12-27 04:30:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001798848_460570624.pth... [2023-12-27 04:30:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001802760_461570048.pth... [2023-12-27 04:30:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001801672_461291520.pth [2023-12-27 04:30:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001797696_460275712.pth [2023-12-27 04:30:01,227][105620] Updated weights for policy 1, policy_version 1802768 (0.0009) [2023-12-27 04:30:01,289][105620] Updated weights for policy 1, policy_version 1802778 (0.0009) [2023-12-27 04:30:01,355][105620] Updated weights for policy 1, policy_version 1802788 (0.0008) [2023-12-27 04:30:01,726][105692] Updated weights for policy 0, policy_version 1798850 (0.0011) [2023-12-27 04:30:01,793][105692] Updated weights for policy 0, policy_version 1798860 (0.0011) [2023-12-27 04:30:01,850][105692] Updated weights for policy 0, policy_version 1798870 (0.0009) [2023-12-27 04:30:01,899][105692] Updated weights for policy 0, policy_version 1798880 (0.0005) [2023-12-27 04:30:02,059][105620] Updated weights for policy 1, policy_version 1802798 (0.0008) [2023-12-27 04:30:02,113][105620] Updated weights for policy 1, policy_version 1802808 (0.0009) [2023-12-27 04:30:02,174][105620] Updated weights for policy 1, policy_version 1802819 (0.0009) [2023-12-27 04:30:02,585][105692] Updated weights for policy 0, policy_version 1798890 (0.0011) [2023-12-27 04:30:02,636][105692] Updated weights for policy 0, policy_version 1798900 (0.0010) [2023-12-27 04:30:02,686][105692] Updated weights for policy 0, policy_version 1798910 (0.0009) [2023-12-27 04:30:02,929][105620] Updated weights for policy 1, policy_version 1802829 (0.0008) [2023-12-27 04:30:02,985][105620] Updated weights for policy 1, policy_version 1802839 (0.0007) [2023-12-27 04:30:03,031][105620] Updated weights for policy 1, policy_version 1802849 (0.0005) [2023-12-27 04:30:03,310][105692] Updated weights for policy 0, policy_version 1798920 (0.0009) [2023-12-27 04:30:03,374][105692] Updated weights for policy 0, policy_version 1798930 (0.0010) [2023-12-27 04:30:03,425][105692] Updated weights for policy 0, policy_version 1798940 (0.0010) [2023-12-27 04:30:03,582][105620] Updated weights for policy 1, policy_version 1802859 (0.0005) [2023-12-27 04:30:03,627][105620] Updated weights for policy 1, policy_version 1802869 (0.0005) [2023-12-27 04:30:03,686][105620] Updated weights for policy 1, policy_version 1802879 (0.0007) [2023-12-27 04:30:04,041][105692] Updated weights for policy 0, policy_version 1798950 (0.0007) [2023-12-27 04:30:04,103][105692] Updated weights for policy 0, policy_version 1798960 (0.0006) [2023-12-27 04:30:04,168][105692] Updated weights for policy 0, policy_version 1798970 (0.0011) [2023-12-27 04:30:04,414][105620] Updated weights for policy 1, policy_version 1802889 (0.0008) [2023-12-27 04:30:04,466][105620] Updated weights for policy 1, policy_version 1802899 (0.0008) [2023-12-27 04:30:04,525][105620] Updated weights for policy 1, policy_version 1802909 (0.0008) [2023-12-27 04:30:04,589][105620] Updated weights for policy 1, policy_version 1802919 (0.0011) [2023-12-27 04:30:04,897][105692] Updated weights for policy 0, policy_version 1798980 (0.0011) [2023-12-27 04:30:04,952][105692] Updated weights for policy 0, policy_version 1798990 (0.0010) [2023-12-27 04:30:04,997][105692] Updated weights for policy 0, policy_version 1799000 (0.0010) [2023-12-27 04:30:05,219][105620] Updated weights for policy 1, policy_version 1802929 (0.0006) [2023-12-27 04:30:05,266][105620] Updated weights for policy 1, policy_version 1802939 (0.0005) [2023-12-27 04:30:05,313][105620] Updated weights for policy 1, policy_version 1802949 (0.0005) [2023-12-27 04:30:05,704][105692] Updated weights for policy 0, policy_version 1799010 (0.0010) [2023-12-27 04:30:05,756][105692] Updated weights for policy 0, policy_version 1799020 (0.0010) [2023-12-27 04:30:05,809][105692] Updated weights for policy 0, policy_version 1799030 (0.0007) [2023-12-27 04:30:05,863][105692] Updated weights for policy 0, policy_version 1799040 (0.0006) [2023-12-27 04:30:06,030][105620] Updated weights for policy 1, policy_version 1802959 (0.0008) [2023-12-27 04:30:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 922238976. Throughput: 0: 9564.2, 1: 9579.8. Samples: 922229448. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:30:06,063][104569] Avg episode reward: [(0, '8445.149'), (1, '9258.696')] [2023-12-27 04:30:06,095][105620] Updated weights for policy 1, policy_version 1802969 (0.0009) [2023-12-27 04:30:06,162][105620] Updated weights for policy 1, policy_version 1802979 (0.0007) [2023-12-27 04:30:06,577][105692] Updated weights for policy 0, policy_version 1799050 (0.0011) [2023-12-27 04:30:06,636][105692] Updated weights for policy 0, policy_version 1799060 (0.0011) [2023-12-27 04:30:06,689][105692] Updated weights for policy 0, policy_version 1799070 (0.0010) [2023-12-27 04:30:06,963][105620] Updated weights for policy 1, policy_version 1802989 (0.0009) [2023-12-27 04:30:07,017][105620] Updated weights for policy 1, policy_version 1802999 (0.0009) [2023-12-27 04:30:07,064][105620] Updated weights for policy 1, policy_version 1803009 (0.0008) [2023-12-27 04:30:07,335][105692] Updated weights for policy 0, policy_version 1799080 (0.0009) [2023-12-27 04:30:07,401][105692] Updated weights for policy 0, policy_version 1799090 (0.0008) [2023-12-27 04:30:07,465][105692] Updated weights for policy 0, policy_version 1799100 (0.0009) [2023-12-27 04:30:07,777][105620] Updated weights for policy 1, policy_version 1803019 (0.0008) [2023-12-27 04:30:07,842][105620] Updated weights for policy 1, policy_version 1803029 (0.0005) [2023-12-27 04:30:07,889][105620] Updated weights for policy 1, policy_version 1803039 (0.0005) [2023-12-27 04:30:08,151][105692] Updated weights for policy 0, policy_version 1799110 (0.0010) [2023-12-27 04:30:08,213][105692] Updated weights for policy 0, policy_version 1799120 (0.0010) [2023-12-27 04:30:08,272][105692] Updated weights for policy 0, policy_version 1799130 (0.0007) [2023-12-27 04:30:08,408][105620] Updated weights for policy 1, policy_version 1803049 (0.0006) [2023-12-27 04:30:08,462][105620] Updated weights for policy 1, policy_version 1803059 (0.0009) [2023-12-27 04:30:08,515][105620] Updated weights for policy 1, policy_version 1803069 (0.0008) [2023-12-27 04:30:08,571][105620] Updated weights for policy 1, policy_version 1803079 (0.0010) [2023-12-27 04:30:09,014][105692] Updated weights for policy 0, policy_version 1799140 (0.0009) [2023-12-27 04:30:09,080][105692] Updated weights for policy 0, policy_version 1799150 (0.0009) [2023-12-27 04:30:09,128][105692] Updated weights for policy 0, policy_version 1799160 (0.0009) [2023-12-27 04:30:09,333][105620] Updated weights for policy 1, policy_version 1803089 (0.0008) [2023-12-27 04:30:09,406][105620] Updated weights for policy 1, policy_version 1803099 (0.0008) [2023-12-27 04:30:09,472][105620] Updated weights for policy 1, policy_version 1803109 (0.0009) [2023-12-27 04:30:09,931][105692] Updated weights for policy 0, policy_version 1799170 (0.0010) [2023-12-27 04:30:09,997][105692] Updated weights for policy 0, policy_version 1799180 (0.0009) [2023-12-27 04:30:10,060][105692] Updated weights for policy 0, policy_version 1799190 (0.0009) [2023-12-27 04:30:10,127][105692] Updated weights for policy 0, policy_version 1799200 (0.0009) [2023-12-27 04:30:10,221][105620] Updated weights for policy 1, policy_version 1803119 (0.0008) [2023-12-27 04:30:10,287][105620] Updated weights for policy 1, policy_version 1803129 (0.0009) [2023-12-27 04:30:10,342][105620] Updated weights for policy 1, policy_version 1803139 (0.0009) [2023-12-27 04:30:10,870][105692] Updated weights for policy 0, policy_version 1799210 (0.0006) [2023-12-27 04:30:10,930][105692] Updated weights for policy 0, policy_version 1799220 (0.0009) [2023-12-27 04:30:10,992][105692] Updated weights for policy 0, policy_version 1799230 (0.0009) [2023-12-27 04:30:11,016][105620] Updated weights for policy 1, policy_version 1803149 (0.0008) [2023-12-27 04:30:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 922337280. Throughput: 0: 9606.8, 1: 9644.8. Samples: 922346836. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:30:11,062][104569] Avg episode reward: [(0, '8257.425'), (1, '9166.110')] [2023-12-27 04:30:11,075][105620] Updated weights for policy 1, policy_version 1803159 (0.0010) [2023-12-27 04:30:11,138][105620] Updated weights for policy 1, policy_version 1803169 (0.0009) [2023-12-27 04:30:11,707][105692] Updated weights for policy 0, policy_version 1799240 (0.0010) [2023-12-27 04:30:11,769][105692] Updated weights for policy 0, policy_version 1799250 (0.0010) [2023-12-27 04:30:11,833][105692] Updated weights for policy 0, policy_version 1799260 (0.0010) [2023-12-27 04:30:11,979][105620] Updated weights for policy 1, policy_version 1803179 (0.0009) [2023-12-27 04:30:12,044][105620] Updated weights for policy 1, policy_version 1803189 (0.0008) [2023-12-27 04:30:12,105][105620] Updated weights for policy 1, policy_version 1803199 (0.0008) [2023-12-27 04:30:12,594][105692] Updated weights for policy 0, policy_version 1799270 (0.0008) [2023-12-27 04:30:12,655][105692] Updated weights for policy 0, policy_version 1799280 (0.0006) [2023-12-27 04:30:12,705][105692] Updated weights for policy 0, policy_version 1799290 (0.0005) [2023-12-27 04:30:12,970][105620] Updated weights for policy 1, policy_version 1803209 (0.0009) [2023-12-27 04:30:13,027][105620] Updated weights for policy 1, policy_version 1803219 (0.0009) [2023-12-27 04:30:13,084][105620] Updated weights for policy 1, policy_version 1803229 (0.0009) [2023-12-27 04:30:13,136][105620] Updated weights for policy 1, policy_version 1803239 (0.0009) [2023-12-27 04:30:13,219][105692] Updated weights for policy 0, policy_version 1799300 (0.0006) [2023-12-27 04:30:13,291][105692] Updated weights for policy 0, policy_version 1799310 (0.0005) [2023-12-27 04:30:13,353][105692] Updated weights for policy 0, policy_version 1799320 (0.0005) [2023-12-27 04:30:13,738][105620] Updated weights for policy 1, policy_version 1803249 (0.0006) [2023-12-27 04:30:13,790][105620] Updated weights for policy 1, policy_version 1803259 (0.0005) [2023-12-27 04:30:13,842][105620] Updated weights for policy 1, policy_version 1803269 (0.0005) [2023-12-27 04:30:13,957][105692] Updated weights for policy 0, policy_version 1799330 (0.0005) [2023-12-27 04:30:14,028][105692] Updated weights for policy 0, policy_version 1799340 (0.0005) [2023-12-27 04:30:14,085][105692] Updated weights for policy 0, policy_version 1799350 (0.0005) [2023-12-27 04:30:14,143][105692] Updated weights for policy 0, policy_version 1799360 (0.0006) [2023-12-27 04:30:14,494][105620] Updated weights for policy 1, policy_version 1803279 (0.0008) [2023-12-27 04:30:14,545][105620] Updated weights for policy 1, policy_version 1803289 (0.0007) [2023-12-27 04:30:14,596][105620] Updated weights for policy 1, policy_version 1803299 (0.0009) [2023-12-27 04:30:14,682][105692] Updated weights for policy 0, policy_version 1799370 (0.0009) [2023-12-27 04:30:14,737][105692] Updated weights for policy 0, policy_version 1799380 (0.0009) [2023-12-27 04:30:14,806][105692] Updated weights for policy 0, policy_version 1799390 (0.0008) [2023-12-27 04:30:15,358][105620] Updated weights for policy 1, policy_version 1803309 (0.0009) [2023-12-27 04:30:15,410][105620] Updated weights for policy 1, policy_version 1803319 (0.0008) [2023-12-27 04:30:15,466][105620] Updated weights for policy 1, policy_version 1803329 (0.0009) [2023-12-27 04:30:15,488][105692] Updated weights for policy 0, policy_version 1799400 (0.0006) [2023-12-27 04:30:15,543][105692] Updated weights for policy 0, policy_version 1799410 (0.0005) [2023-12-27 04:30:15,605][105692] Updated weights for policy 0, policy_version 1799420 (0.0006) [2023-12-27 04:30:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.6, 300 sec: 19549.7). Total num frames: 922435584. Throughput: 0: 9723.0, 1: 9551.2. Samples: 922405896. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:30:16,063][104569] Avg episode reward: [(0, '8080.878'), (1, '9258.383')] [2023-12-27 04:30:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001803336_461717504.pth... [2023-12-27 04:30:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001799424_460718080.pth... [2023-12-27 04:30:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001802248_461438976.pth [2023-12-27 04:30:16,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001798240_460414976.pth [2023-12-27 04:30:16,136][105692] Updated weights for policy 0, policy_version 1799430 (0.0008) [2023-12-27 04:30:16,184][105692] Updated weights for policy 0, policy_version 1799440 (0.0007) [2023-12-27 04:30:16,232][105692] Updated weights for policy 0, policy_version 1799450 (0.0005) [2023-12-27 04:30:16,311][105620] Updated weights for policy 1, policy_version 1803339 (0.0009) [2023-12-27 04:30:16,370][105620] Updated weights for policy 1, policy_version 1803349 (0.0010) [2023-12-27 04:30:16,418][105620] Updated weights for policy 1, policy_version 1803359 (0.0010) [2023-12-27 04:30:16,798][105692] Updated weights for policy 0, policy_version 1799460 (0.0006) [2023-12-27 04:30:16,850][105692] Updated weights for policy 0, policy_version 1799470 (0.0009) [2023-12-27 04:30:16,895][105692] Updated weights for policy 0, policy_version 1799480 (0.0007) [2023-12-27 04:30:17,079][105620] Updated weights for policy 1, policy_version 1803369 (0.0010) [2023-12-27 04:30:17,133][105620] Updated weights for policy 1, policy_version 1803379 (0.0005) [2023-12-27 04:30:17,193][105620] Updated weights for policy 1, policy_version 1803389 (0.0006) [2023-12-27 04:30:17,245][105620] Updated weights for policy 1, policy_version 1803399 (0.0008) [2023-12-27 04:30:17,622][105692] Updated weights for policy 0, policy_version 1799490 (0.0006) [2023-12-27 04:30:17,675][105692] Updated weights for policy 0, policy_version 1799500 (0.0010) [2023-12-27 04:30:17,733][105692] Updated weights for policy 0, policy_version 1799510 (0.0010) [2023-12-27 04:30:17,786][105692] Updated weights for policy 0, policy_version 1799520 (0.0009) [2023-12-27 04:30:17,799][105620] Updated weights for policy 1, policy_version 1803409 (0.0006) [2023-12-27 04:30:17,867][105620] Updated weights for policy 1, policy_version 1803419 (0.0006) [2023-12-27 04:30:17,930][105620] Updated weights for policy 1, policy_version 1803429 (0.0009) [2023-12-27 04:30:18,594][105692] Updated weights for policy 0, policy_version 1799530 (0.0006) [2023-12-27 04:30:18,610][105620] Updated weights for policy 1, policy_version 1803439 (0.0007) [2023-12-27 04:30:18,660][105692] Updated weights for policy 0, policy_version 1799540 (0.0008) [2023-12-27 04:30:18,667][105620] Updated weights for policy 1, policy_version 1803449 (0.0006) [2023-12-27 04:30:18,722][105692] Updated weights for policy 0, policy_version 1799550 (0.0007) [2023-12-27 04:30:18,724][105620] Updated weights for policy 1, policy_version 1803459 (0.0005) [2023-12-27 04:30:19,397][105620] Updated weights for policy 1, policy_version 1803469 (0.0007) [2023-12-27 04:30:19,433][105692] Updated weights for policy 0, policy_version 1799560 (0.0008) [2023-12-27 04:30:19,452][105620] Updated weights for policy 1, policy_version 1803479 (0.0007) [2023-12-27 04:30:19,504][105692] Updated weights for policy 0, policy_version 1799570 (0.0008) [2023-12-27 04:30:19,521][105620] Updated weights for policy 1, policy_version 1803489 (0.0007) [2023-12-27 04:30:19,562][105692] Updated weights for policy 0, policy_version 1799580 (0.0009) [2023-12-27 04:30:20,220][105620] Updated weights for policy 1, policy_version 1803499 (0.0009) [2023-12-27 04:30:20,280][105620] Updated weights for policy 1, policy_version 1803509 (0.0011) [2023-12-27 04:30:20,337][105620] Updated weights for policy 1, policy_version 1803519 (0.0008) [2023-12-27 04:30:20,357][105692] Updated weights for policy 0, policy_version 1799590 (0.0009) [2023-12-27 04:30:20,422][105692] Updated weights for policy 0, policy_version 1799600 (0.0007) [2023-12-27 04:30:20,485][105692] Updated weights for policy 0, policy_version 1799610 (0.0008) [2023-12-27 04:30:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 922533888. Throughput: 0: 9902.7, 1: 9539.4. Samples: 922528668. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:30:21,062][104569] Avg episode reward: [(0, '7996.518'), (1, '9350.818')] [2023-12-27 04:30:21,087][105620] Updated weights for policy 1, policy_version 1803529 (0.0010) [2023-12-27 04:30:21,153][105620] Updated weights for policy 1, policy_version 1803539 (0.0011) [2023-12-27 04:30:21,220][105620] Updated weights for policy 1, policy_version 1803549 (0.0011) [2023-12-27 04:30:21,270][105692] Updated weights for policy 0, policy_version 1799620 (0.0007) [2023-12-27 04:30:21,281][105620] Updated weights for policy 1, policy_version 1803559 (0.0011) [2023-12-27 04:30:21,334][105692] Updated weights for policy 0, policy_version 1799630 (0.0008) [2023-12-27 04:30:21,409][105692] Updated weights for policy 0, policy_version 1799640 (0.0009) [2023-12-27 04:30:22,085][105620] Updated weights for policy 1, policy_version 1803569 (0.0008) [2023-12-27 04:30:22,144][105620] Updated weights for policy 1, policy_version 1803579 (0.0009) [2023-12-27 04:30:22,191][105692] Updated weights for policy 0, policy_version 1799650 (0.0008) [2023-12-27 04:30:22,208][105620] Updated weights for policy 1, policy_version 1803589 (0.0006) [2023-12-27 04:30:22,259][105692] Updated weights for policy 0, policy_version 1799660 (0.0009) [2023-12-27 04:30:22,321][105692] Updated weights for policy 0, policy_version 1799670 (0.0009) [2023-12-27 04:30:22,387][105692] Updated weights for policy 0, policy_version 1799680 (0.0009) [2023-12-27 04:30:22,876][105620] Updated weights for policy 1, policy_version 1803599 (0.0009) [2023-12-27 04:30:22,939][105620] Updated weights for policy 1, policy_version 1803609 (0.0009) [2023-12-27 04:30:22,995][105620] Updated weights for policy 1, policy_version 1803619 (0.0009) [2023-12-27 04:30:23,199][105692] Updated weights for policy 0, policy_version 1799690 (0.0009) [2023-12-27 04:30:23,258][105692] Updated weights for policy 0, policy_version 1799700 (0.0009) [2023-12-27 04:30:23,316][105692] Updated weights for policy 0, policy_version 1799710 (0.0009) [2023-12-27 04:30:23,666][105620] Updated weights for policy 1, policy_version 1803629 (0.0009) [2023-12-27 04:30:23,718][105620] Updated weights for policy 1, policy_version 1803639 (0.0011) [2023-12-27 04:30:23,772][105620] Updated weights for policy 1, policy_version 1803649 (0.0010) [2023-12-27 04:30:24,071][105692] Updated weights for policy 0, policy_version 1799720 (0.0010) [2023-12-27 04:30:24,127][105692] Updated weights for policy 0, policy_version 1799730 (0.0010) [2023-12-27 04:30:24,186][105692] Updated weights for policy 0, policy_version 1799740 (0.0011) [2023-12-27 04:30:24,331][105620] Updated weights for policy 1, policy_version 1803659 (0.0007) [2023-12-27 04:30:24,390][105620] Updated weights for policy 1, policy_version 1803669 (0.0011) [2023-12-27 04:30:24,446][105620] Updated weights for policy 1, policy_version 1803679 (0.0008) [2023-12-27 04:30:24,887][105692] Updated weights for policy 0, policy_version 1799750 (0.0009) [2023-12-27 04:30:24,938][105692] Updated weights for policy 0, policy_version 1799760 (0.0008) [2023-12-27 04:30:24,991][105692] Updated weights for policy 0, policy_version 1799770 (0.0009) [2023-12-27 04:30:25,106][105620] Updated weights for policy 1, policy_version 1803689 (0.0008) [2023-12-27 04:30:25,163][105620] Updated weights for policy 1, policy_version 1803699 (0.0005) [2023-12-27 04:30:25,224][105620] Updated weights for policy 1, policy_version 1803709 (0.0010) [2023-12-27 04:30:25,288][105620] Updated weights for policy 1, policy_version 1803719 (0.0011) [2023-12-27 04:30:25,739][105692] Updated weights for policy 0, policy_version 1799782 (0.0007) [2023-12-27 04:30:25,792][105692] Updated weights for policy 0, policy_version 1799792 (0.0005) [2023-12-27 04:30:25,847][105692] Updated weights for policy 0, policy_version 1799802 (0.0005) [2023-12-27 04:30:25,897][105620] Updated weights for policy 1, policy_version 1803729 (0.0011) [2023-12-27 04:30:25,943][105620] Updated weights for policy 1, policy_version 1803739 (0.0010) [2023-12-27 04:30:25,991][105620] Updated weights for policy 1, policy_version 1803749 (0.0010) [2023-12-27 04:30:26,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 922640384. Throughput: 0: 9909.6, 1: 9573.5. Samples: 922643112. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:30:26,063][104569] Avg episode reward: [(0, '8359.926'), (1, '9258.521')] [2023-12-27 04:30:26,391][105692] Updated weights for policy 0, policy_version 1799812 (0.0005) [2023-12-27 04:30:26,454][105692] Updated weights for policy 0, policy_version 1799822 (0.0005) [2023-12-27 04:30:26,511][105692] Updated weights for policy 0, policy_version 1799832 (0.0006) [2023-12-27 04:30:26,614][105620] Updated weights for policy 1, policy_version 1803759 (0.0007) [2023-12-27 04:30:26,681][105620] Updated weights for policy 1, policy_version 1803769 (0.0005) [2023-12-27 04:30:26,741][105620] Updated weights for policy 1, policy_version 1803779 (0.0009) [2023-12-27 04:30:27,083][105692] Updated weights for policy 0, policy_version 1799842 (0.0008) [2023-12-27 04:30:27,133][105692] Updated weights for policy 0, policy_version 1799852 (0.0007) [2023-12-27 04:30:27,181][105692] Updated weights for policy 0, policy_version 1799862 (0.0008) [2023-12-27 04:30:27,234][105692] Updated weights for policy 0, policy_version 1799872 (0.0008) [2023-12-27 04:30:27,403][105620] Updated weights for policy 1, policy_version 1803789 (0.0010) [2023-12-27 04:30:27,467][105620] Updated weights for policy 1, policy_version 1803799 (0.0010) [2023-12-27 04:30:27,534][105620] Updated weights for policy 1, policy_version 1803809 (0.0010) [2023-12-27 04:30:27,832][105692] Updated weights for policy 0, policy_version 1799882 (0.0008) [2023-12-27 04:30:27,892][105692] Updated weights for policy 0, policy_version 1799892 (0.0010) [2023-12-27 04:30:27,953][105692] Updated weights for policy 0, policy_version 1799902 (0.0008) [2023-12-27 04:30:28,117][105620] Updated weights for policy 1, policy_version 1803819 (0.0009) [2023-12-27 04:30:28,160][105620] Updated weights for policy 1, policy_version 1803829 (0.0005) [2023-12-27 04:30:28,205][105620] Updated weights for policy 1, policy_version 1803839 (0.0005) [2023-12-27 04:30:28,498][105692] Updated weights for policy 0, policy_version 1799912 (0.0005) [2023-12-27 04:30:28,553][105692] Updated weights for policy 0, policy_version 1799922 (0.0005) [2023-12-27 04:30:28,610][105692] Updated weights for policy 0, policy_version 1799932 (0.0005) [2023-12-27 04:30:28,815][105620] Updated weights for policy 1, policy_version 1803849 (0.0005) [2023-12-27 04:30:28,880][105620] Updated weights for policy 1, policy_version 1803859 (0.0006) [2023-12-27 04:30:28,946][105620] Updated weights for policy 1, policy_version 1803869 (0.0005) [2023-12-27 04:30:29,010][105620] Updated weights for policy 1, policy_version 1803879 (0.0008) [2023-12-27 04:30:29,227][105692] Updated weights for policy 0, policy_version 1799942 (0.0009) [2023-12-27 04:30:29,293][105692] Updated weights for policy 0, policy_version 1799952 (0.0010) [2023-12-27 04:30:29,362][105692] Updated weights for policy 0, policy_version 1799962 (0.0010) [2023-12-27 04:30:29,669][105620] Updated weights for policy 1, policy_version 1803889 (0.0009) [2023-12-27 04:30:29,721][105620] Updated weights for policy 1, policy_version 1803899 (0.0009) [2023-12-27 04:30:29,777][105620] Updated weights for policy 1, policy_version 1803909 (0.0010) [2023-12-27 04:30:30,099][105692] Updated weights for policy 0, policy_version 1799972 (0.0009) [2023-12-27 04:30:30,163][105692] Updated weights for policy 0, policy_version 1799982 (0.0008) [2023-12-27 04:30:30,221][105692] Updated weights for policy 0, policy_version 1799992 (0.0010) [2023-12-27 04:30:30,442][105620] Updated weights for policy 1, policy_version 1803919 (0.0009) [2023-12-27 04:30:30,489][105620] Updated weights for policy 1, policy_version 1803929 (0.0008) [2023-12-27 04:30:30,545][105620] Updated weights for policy 1, policy_version 1803939 (0.0009) [2023-12-27 04:30:30,990][105692] Updated weights for policy 0, policy_version 1800002 (0.0009) [2023-12-27 04:30:31,051][105692] Updated weights for policy 0, policy_version 1800012 (0.0009) [2023-12-27 04:30:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 922738688. Throughput: 0: 10096.3, 1: 9698.0. Samples: 922712592. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:30:31,062][104569] Avg episode reward: [(0, '8627.996'), (1, '9074.233')] [2023-12-27 04:30:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001803944_461873152.pth... [2023-12-27 04:30:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001802760_461570048.pth [2023-12-27 04:30:31,116][105692] Updated weights for policy 0, policy_version 1800022 (0.0010) [2023-12-27 04:30:31,180][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001800032_460873728.pth... [2023-12-27 04:30:31,184][105692] Updated weights for policy 0, policy_version 1800032 (0.0009) [2023-12-27 04:30:31,201][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001798848_460570624.pth [2023-12-27 04:30:31,259][105620] Updated weights for policy 1, policy_version 1803949 (0.0008) [2023-12-27 04:30:31,320][105620] Updated weights for policy 1, policy_version 1803959 (0.0008) [2023-12-27 04:30:31,388][105620] Updated weights for policy 1, policy_version 1803969 (0.0008) [2023-12-27 04:30:31,920][105692] Updated weights for policy 0, policy_version 1800042 (0.0009) [2023-12-27 04:30:31,975][105692] Updated weights for policy 0, policy_version 1800052 (0.0008) [2023-12-27 04:30:32,023][105692] Updated weights for policy 0, policy_version 1800062 (0.0006) [2023-12-27 04:30:32,108][105620] Updated weights for policy 1, policy_version 1803979 (0.0007) [2023-12-27 04:30:32,166][105620] Updated weights for policy 1, policy_version 1803989 (0.0010) [2023-12-27 04:30:32,218][105620] Updated weights for policy 1, policy_version 1803999 (0.0010) [2023-12-27 04:30:32,686][105692] Updated weights for policy 0, policy_version 1800072 (0.0008) [2023-12-27 04:30:32,759][105692] Updated weights for policy 0, policy_version 1800082 (0.0005) [2023-12-27 04:30:32,825][105692] Updated weights for policy 0, policy_version 1800092 (0.0005) [2023-12-27 04:30:32,923][105620] Updated weights for policy 1, policy_version 1804009 (0.0008) [2023-12-27 04:30:32,976][105620] Updated weights for policy 1, policy_version 1804019 (0.0005) [2023-12-27 04:30:33,045][105620] Updated weights for policy 1, policy_version 1804029 (0.0005) [2023-12-27 04:30:33,098][105620] Updated weights for policy 1, policy_version 1804039 (0.0007) [2023-12-27 04:30:33,403][105692] Updated weights for policy 0, policy_version 1800102 (0.0008) [2023-12-27 04:30:33,451][105692] Updated weights for policy 0, policy_version 1800112 (0.0008) [2023-12-27 04:30:33,497][105692] Updated weights for policy 0, policy_version 1800122 (0.0005) [2023-12-27 04:30:33,602][105620] Updated weights for policy 1, policy_version 1804049 (0.0007) [2023-12-27 04:30:33,656][105620] Updated weights for policy 1, policy_version 1804059 (0.0006) [2023-12-27 04:30:33,701][105620] Updated weights for policy 1, policy_version 1804069 (0.0005) [2023-12-27 04:30:34,106][105692] Updated weights for policy 0, policy_version 1800132 (0.0005) [2023-12-27 04:30:34,165][105692] Updated weights for policy 0, policy_version 1800142 (0.0007) [2023-12-27 04:30:34,224][105692] Updated weights for policy 0, policy_version 1800152 (0.0006) [2023-12-27 04:30:34,321][105620] Updated weights for policy 1, policy_version 1804079 (0.0005) [2023-12-27 04:30:34,380][105620] Updated weights for policy 1, policy_version 1804089 (0.0006) [2023-12-27 04:30:34,436][105620] Updated weights for policy 1, policy_version 1804099 (0.0005) [2023-12-27 04:30:34,958][105692] Updated weights for policy 0, policy_version 1800162 (0.0008) [2023-12-27 04:30:35,010][105692] Updated weights for policy 0, policy_version 1800172 (0.0011) [2023-12-27 04:30:35,075][105692] Updated weights for policy 0, policy_version 1800182 (0.0007) [2023-12-27 04:30:35,132][105692] Updated weights for policy 0, policy_version 1800192 (0.0006) [2023-12-27 04:30:35,142][105620] Updated weights for policy 1, policy_version 1804109 (0.0007) [2023-12-27 04:30:35,200][105620] Updated weights for policy 1, policy_version 1804119 (0.0005) [2023-12-27 04:30:35,266][105620] Updated weights for policy 1, policy_version 1804129 (0.0006) [2023-12-27 04:30:35,781][105692] Updated weights for policy 0, policy_version 1800202 (0.0008) [2023-12-27 04:30:35,838][105692] Updated weights for policy 0, policy_version 1800212 (0.0008) [2023-12-27 04:30:35,890][105692] Updated weights for policy 0, policy_version 1800222 (0.0008) [2023-12-27 04:30:35,924][105620] Updated weights for policy 1, policy_version 1804139 (0.0008) [2023-12-27 04:30:35,982][105620] Updated weights for policy 1, policy_version 1804149 (0.0007) [2023-12-27 04:30:36,039][105620] Updated weights for policy 1, policy_version 1804159 (0.0011) [2023-12-27 04:30:36,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 922845184. Throughput: 0: 10158.1, 1: 9831.8. Samples: 922835900. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:30:36,063][104569] Avg episode reward: [(0, '8445.218'), (1, '8981.742')] [2023-12-27 04:30:36,671][105620] Updated weights for policy 1, policy_version 1804169 (0.0008) [2023-12-27 04:30:36,691][105692] Updated weights for policy 0, policy_version 1800232 (0.0010) [2023-12-27 04:30:36,732][105620] Updated weights for policy 1, policy_version 1804179 (0.0011) [2023-12-27 04:30:36,751][105692] Updated weights for policy 0, policy_version 1800242 (0.0009) [2023-12-27 04:30:36,787][105620] Updated weights for policy 1, policy_version 1804189 (0.0009) [2023-12-27 04:30:36,809][105692] Updated weights for policy 0, policy_version 1800252 (0.0009) [2023-12-27 04:30:36,850][105620] Updated weights for policy 1, policy_version 1804199 (0.0010) [2023-12-27 04:30:37,506][105620] Updated weights for policy 1, policy_version 1804209 (0.0009) [2023-12-27 04:30:37,555][105620] Updated weights for policy 1, policy_version 1804219 (0.0010) [2023-12-27 04:30:37,603][105620] Updated weights for policy 1, policy_version 1804229 (0.0010) [2023-12-27 04:30:37,658][105692] Updated weights for policy 0, policy_version 1800262 (0.0007) [2023-12-27 04:30:37,718][105692] Updated weights for policy 0, policy_version 1800272 (0.0008) [2023-12-27 04:30:37,770][105692] Updated weights for policy 0, policy_version 1800282 (0.0008) [2023-12-27 04:30:38,358][105620] Updated weights for policy 1, policy_version 1804239 (0.0011) [2023-12-27 04:30:38,407][105620] Updated weights for policy 1, policy_version 1804249 (0.0010) [2023-12-27 04:30:38,456][105620] Updated weights for policy 1, policy_version 1804259 (0.0010) [2023-12-27 04:30:38,484][105692] Updated weights for policy 0, policy_version 1800292 (0.0008) [2023-12-27 04:30:38,539][105692] Updated weights for policy 0, policy_version 1800302 (0.0007) [2023-12-27 04:30:38,596][105692] Updated weights for policy 0, policy_version 1800312 (0.0008) [2023-12-27 04:30:39,191][105620] Updated weights for policy 1, policy_version 1804269 (0.0008) [2023-12-27 04:30:39,256][105620] Updated weights for policy 1, policy_version 1804279 (0.0007) [2023-12-27 04:30:39,263][105692] Updated weights for policy 0, policy_version 1800322 (0.0007) [2023-12-27 04:30:39,319][105620] Updated weights for policy 1, policy_version 1804289 (0.0006) [2023-12-27 04:30:39,321][105692] Updated weights for policy 0, policy_version 1800332 (0.0008) [2023-12-27 04:30:39,386][105692] Updated weights for policy 0, policy_version 1800342 (0.0007) [2023-12-27 04:30:39,452][105692] Updated weights for policy 0, policy_version 1800352 (0.0008) [2023-12-27 04:30:40,043][105620] Updated weights for policy 1, policy_version 1804299 (0.0010) [2023-12-27 04:30:40,102][105620] Updated weights for policy 1, policy_version 1804309 (0.0011) [2023-12-27 04:30:40,155][105620] Updated weights for policy 1, policy_version 1804319 (0.0011) [2023-12-27 04:30:40,178][105692] Updated weights for policy 0, policy_version 1800362 (0.0006) [2023-12-27 04:30:40,233][105692] Updated weights for policy 0, policy_version 1800372 (0.0007) [2023-12-27 04:30:40,294][105692] Updated weights for policy 0, policy_version 1800382 (0.0009) [2023-12-27 04:30:40,929][105620] Updated weights for policy 1, policy_version 1804329 (0.0010) [2023-12-27 04:30:40,967][105692] Updated weights for policy 0, policy_version 1800392 (0.0009) [2023-12-27 04:30:40,978][105620] Updated weights for policy 1, policy_version 1804339 (0.0010) [2023-12-27 04:30:41,024][105692] Updated weights for policy 0, policy_version 1800402 (0.0011) [2023-12-27 04:30:41,031][105620] Updated weights for policy 1, policy_version 1804349 (0.0010) [2023-12-27 04:30:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 922935296. Throughput: 0: 9989.5, 1: 9969.3. Samples: 922951312. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:30:41,063][104569] Avg episode reward: [(0, '8539.390'), (1, '9073.572')] [2023-12-27 04:30:41,094][105692] Updated weights for policy 0, policy_version 1800412 (0.0011) [2023-12-27 04:30:41,098][105620] Updated weights for policy 1, policy_version 1804359 (0.0011) [2023-12-27 04:30:41,860][105692] Updated weights for policy 0, policy_version 1800422 (0.0012) [2023-12-27 04:30:41,913][105620] Updated weights for policy 1, policy_version 1804369 (0.0011) [2023-12-27 04:30:41,923][105692] Updated weights for policy 0, policy_version 1800432 (0.0011) [2023-12-27 04:30:41,963][105620] Updated weights for policy 1, policy_version 1804379 (0.0011) [2023-12-27 04:30:41,980][105692] Updated weights for policy 0, policy_version 1800442 (0.0009) [2023-12-27 04:30:42,024][105620] Updated weights for policy 1, policy_version 1804389 (0.0011) [2023-12-27 04:30:42,684][105692] Updated weights for policy 0, policy_version 1800452 (0.0009) [2023-12-27 04:30:42,744][105692] Updated weights for policy 0, policy_version 1800462 (0.0011) [2023-12-27 04:30:42,748][105620] Updated weights for policy 1, policy_version 1804399 (0.0011) [2023-12-27 04:30:42,800][105692] Updated weights for policy 0, policy_version 1800472 (0.0010) [2023-12-27 04:30:42,801][105620] Updated weights for policy 1, policy_version 1804409 (0.0011) [2023-12-27 04:30:42,853][105620] Updated weights for policy 1, policy_version 1804419 (0.0010) [2023-12-27 04:30:43,421][105692] Updated weights for policy 0, policy_version 1800482 (0.0009) [2023-12-27 04:30:43,486][105692] Updated weights for policy 0, policy_version 1800492 (0.0005) [2023-12-27 04:30:43,543][105692] Updated weights for policy 0, policy_version 1800502 (0.0005) [2023-12-27 04:30:43,581][105620] Updated weights for policy 1, policy_version 1804429 (0.0011) [2023-12-27 04:30:43,604][105692] Updated weights for policy 0, policy_version 1800512 (0.0005) [2023-12-27 04:30:43,636][105620] Updated weights for policy 1, policy_version 1804439 (0.0010) [2023-12-27 04:30:43,698][105620] Updated weights for policy 1, policy_version 1804449 (0.0010) [2023-12-27 04:30:44,100][105692] Updated weights for policy 0, policy_version 1800522 (0.0005) [2023-12-27 04:30:44,146][105692] Updated weights for policy 0, policy_version 1800532 (0.0005) [2023-12-27 04:30:44,192][105692] Updated weights for policy 0, policy_version 1800542 (0.0005) [2023-12-27 04:30:44,454][105620] Updated weights for policy 1, policy_version 1804459 (0.0011) [2023-12-27 04:30:44,500][105620] Updated weights for policy 1, policy_version 1804469 (0.0005) [2023-12-27 04:30:44,549][105620] Updated weights for policy 1, policy_version 1804479 (0.0005) [2023-12-27 04:30:44,849][105692] Updated weights for policy 0, policy_version 1800552 (0.0009) [2023-12-27 04:30:44,914][105692] Updated weights for policy 0, policy_version 1800562 (0.0010) [2023-12-27 04:30:44,984][105692] Updated weights for policy 0, policy_version 1800572 (0.0011) [2023-12-27 04:30:45,224][105620] Updated weights for policy 1, policy_version 1804489 (0.0006) [2023-12-27 04:30:45,288][105620] Updated weights for policy 1, policy_version 1804499 (0.0011) [2023-12-27 04:30:45,351][105620] Updated weights for policy 1, policy_version 1804509 (0.0011) [2023-12-27 04:30:45,419][105620] Updated weights for policy 1, policy_version 1804519 (0.0011) [2023-12-27 04:30:45,728][105692] Updated weights for policy 0, policy_version 1800582 (0.0008) [2023-12-27 04:30:45,780][105692] Updated weights for policy 0, policy_version 1800592 (0.0005) [2023-12-27 04:30:45,832][105692] Updated weights for policy 0, policy_version 1800602 (0.0010) [2023-12-27 04:30:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19797.2, 300 sec: 19605.2). Total num frames: 923041792. Throughput: 0: 10009.5, 1: 9991.9. Samples: 923009716. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:30:46,063][104569] Avg episode reward: [(0, '8536.860'), (1, '9165.928')] [2023-12-27 04:30:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001800608_461021184.pth... [2023-12-27 04:30:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001804520_462020608.pth... [2023-12-27 04:30:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001799424_460718080.pth [2023-12-27 04:30:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001803336_461717504.pth [2023-12-27 04:30:46,175][105620] Updated weights for policy 1, policy_version 1804529 (0.0010) [2023-12-27 04:30:46,234][105620] Updated weights for policy 1, policy_version 1804539 (0.0010) [2023-12-27 04:30:46,281][105620] Updated weights for policy 1, policy_version 1804549 (0.0010) [2023-12-27 04:30:46,482][105692] Updated weights for policy 0, policy_version 1800612 (0.0008) [2023-12-27 04:30:46,528][105692] Updated weights for policy 0, policy_version 1800622 (0.0005) [2023-12-27 04:30:46,573][105692] Updated weights for policy 0, policy_version 1800632 (0.0005) [2023-12-27 04:30:46,965][105620] Updated weights for policy 1, policy_version 1804559 (0.0007) [2023-12-27 04:30:47,018][105620] Updated weights for policy 1, policy_version 1804569 (0.0005) [2023-12-27 04:30:47,065][105620] Updated weights for policy 1, policy_version 1804579 (0.0005) [2023-12-27 04:30:47,151][105692] Updated weights for policy 0, policy_version 1800642 (0.0011) [2023-12-27 04:30:47,199][105692] Updated weights for policy 0, policy_version 1800652 (0.0010) [2023-12-27 04:30:47,243][105692] Updated weights for policy 0, policy_version 1800662 (0.0010) [2023-12-27 04:30:47,291][105692] Updated weights for policy 0, policy_version 1800672 (0.0010) [2023-12-27 04:30:47,718][105620] Updated weights for policy 1, policy_version 1804589 (0.0007) [2023-12-27 04:30:47,774][105620] Updated weights for policy 1, policy_version 1804599 (0.0005) [2023-12-27 04:30:47,834][105620] Updated weights for policy 1, policy_version 1804609 (0.0005) [2023-12-27 04:30:48,042][105692] Updated weights for policy 0, policy_version 1800682 (0.0005) [2023-12-27 04:30:48,107][105692] Updated weights for policy 0, policy_version 1800692 (0.0005) [2023-12-27 04:30:48,172][105692] Updated weights for policy 0, policy_version 1800702 (0.0008) [2023-12-27 04:30:48,448][105620] Updated weights for policy 1, policy_version 1804619 (0.0005) [2023-12-27 04:30:48,510][105620] Updated weights for policy 1, policy_version 1804629 (0.0007) [2023-12-27 04:30:48,561][105620] Updated weights for policy 1, policy_version 1804639 (0.0006) [2023-12-27 04:30:48,838][105692] Updated weights for policy 0, policy_version 1800712 (0.0009) [2023-12-27 04:30:48,896][105692] Updated weights for policy 0, policy_version 1800722 (0.0009) [2023-12-27 04:30:48,957][105692] Updated weights for policy 0, policy_version 1800732 (0.0009) [2023-12-27 04:30:49,206][105620] Updated weights for policy 1, policy_version 1804649 (0.0006) [2023-12-27 04:30:49,274][105620] Updated weights for policy 1, policy_version 1804659 (0.0007) [2023-12-27 04:30:49,343][105620] Updated weights for policy 1, policy_version 1804669 (0.0007) [2023-12-27 04:30:49,406][105620] Updated weights for policy 1, policy_version 1804679 (0.0006) [2023-12-27 04:30:49,679][105692] Updated weights for policy 0, policy_version 1800742 (0.0009) [2023-12-27 04:30:49,740][105692] Updated weights for policy 0, policy_version 1800752 (0.0009) [2023-12-27 04:30:49,803][105692] Updated weights for policy 0, policy_version 1800762 (0.0010) [2023-12-27 04:30:49,965][105620] Updated weights for policy 1, policy_version 1804689 (0.0008) [2023-12-27 04:30:50,028][105620] Updated weights for policy 1, policy_version 1804699 (0.0009) [2023-12-27 04:30:50,088][105620] Updated weights for policy 1, policy_version 1804709 (0.0009) [2023-12-27 04:30:50,517][105692] Updated weights for policy 0, policy_version 1800772 (0.0010) [2023-12-27 04:30:50,577][105692] Updated weights for policy 0, policy_version 1800782 (0.0009) [2023-12-27 04:30:50,637][105692] Updated weights for policy 0, policy_version 1800792 (0.0007) [2023-12-27 04:30:50,843][105620] Updated weights for policy 1, policy_version 1804719 (0.0009) [2023-12-27 04:30:50,897][105620] Updated weights for policy 1, policy_version 1804729 (0.0008) [2023-12-27 04:30:50,948][105620] Updated weights for policy 1, policy_version 1804739 (0.0009) [2023-12-27 04:30:51,062][104569] Fps is (10 sec: 21299.2, 60 sec: 20070.4, 300 sec: 19633.0). Total num frames: 923148288. Throughput: 0: 10047.7, 1: 10076.7. Samples: 923135048. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:30:51,063][104569] Avg episode reward: [(0, '8716.462'), (1, '9166.106')] [2023-12-27 04:30:51,418][105692] Updated weights for policy 0, policy_version 1800802 (0.0008) [2023-12-27 04:30:51,465][105692] Updated weights for policy 0, policy_version 1800812 (0.0008) [2023-12-27 04:30:51,512][105692] Updated weights for policy 0, policy_version 1800822 (0.0009) [2023-12-27 04:30:51,563][105692] Updated weights for policy 0, policy_version 1800832 (0.0009) [2023-12-27 04:30:51,703][105620] Updated weights for policy 1, policy_version 1804749 (0.0008) [2023-12-27 04:30:51,769][105620] Updated weights for policy 1, policy_version 1804759 (0.0008) [2023-12-27 04:30:51,819][105620] Updated weights for policy 1, policy_version 1804769 (0.0009) [2023-12-27 04:30:52,409][105692] Updated weights for policy 0, policy_version 1800842 (0.0009) [2023-12-27 04:30:52,466][105692] Updated weights for policy 0, policy_version 1800852 (0.0010) [2023-12-27 04:30:52,527][105692] Updated weights for policy 0, policy_version 1800862 (0.0009) [2023-12-27 04:30:52,528][105620] Updated weights for policy 1, policy_version 1804779 (0.0008) [2023-12-27 04:30:52,583][105620] Updated weights for policy 1, policy_version 1804789 (0.0008) [2023-12-27 04:30:52,644][105620] Updated weights for policy 1, policy_version 1804799 (0.0008) [2023-12-27 04:30:53,312][105692] Updated weights for policy 0, policy_version 1800872 (0.0007) [2023-12-27 04:30:53,350][105620] Updated weights for policy 1, policy_version 1804809 (0.0008) [2023-12-27 04:30:53,362][105692] Updated weights for policy 0, policy_version 1800882 (0.0009) [2023-12-27 04:30:53,408][105620] Updated weights for policy 1, policy_version 1804819 (0.0007) [2023-12-27 04:30:53,415][105692] Updated weights for policy 0, policy_version 1800892 (0.0006) [2023-12-27 04:30:53,456][105620] Updated weights for policy 1, policy_version 1804829 (0.0007) [2023-12-27 04:30:53,503][105620] Updated weights for policy 1, policy_version 1804839 (0.0008) [2023-12-27 04:30:54,179][105692] Updated weights for policy 0, policy_version 1800902 (0.0008) [2023-12-27 04:30:54,240][105692] Updated weights for policy 0, policy_version 1800912 (0.0007) [2023-12-27 04:30:54,257][105620] Updated weights for policy 1, policy_version 1804849 (0.0010) [2023-12-27 04:30:54,302][105692] Updated weights for policy 0, policy_version 1800922 (0.0005) [2023-12-27 04:30:54,312][105620] Updated weights for policy 1, policy_version 1804859 (0.0010) [2023-12-27 04:30:54,373][105620] Updated weights for policy 1, policy_version 1804869 (0.0010) [2023-12-27 04:30:55,008][105620] Updated weights for policy 1, policy_version 1804879 (0.0007) [2023-12-27 04:30:55,063][105620] Updated weights for policy 1, policy_version 1804889 (0.0005) [2023-12-27 04:30:55,109][105692] Updated weights for policy 0, policy_version 1800932 (0.0007) [2023-12-27 04:30:55,122][105620] Updated weights for policy 1, policy_version 1804899 (0.0008) [2023-12-27 04:30:55,166][105692] Updated weights for policy 0, policy_version 1800942 (0.0006) [2023-12-27 04:30:55,217][105692] Updated weights for policy 0, policy_version 1800952 (0.0008) [2023-12-27 04:30:55,736][105620] Updated weights for policy 1, policy_version 1804909 (0.0010) [2023-12-27 04:30:55,791][105620] Updated weights for policy 1, policy_version 1804919 (0.0007) [2023-12-27 04:30:55,847][105620] Updated weights for policy 1, policy_version 1804929 (0.0006) [2023-12-27 04:30:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.8, 300 sec: 19633.0). Total num frames: 923238400. Throughput: 0: 9959.8, 1: 10075.1. Samples: 923248412. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:30:56,063][104569] Avg episode reward: [(0, '8713.441'), (1, '9258.635')] [2023-12-27 04:30:56,124][105692] Updated weights for policy 0, policy_version 1800962 (0.0008) [2023-12-27 04:30:56,190][105692] Updated weights for policy 0, policy_version 1800972 (0.0008) [2023-12-27 04:30:56,241][105692] Updated weights for policy 0, policy_version 1800982 (0.0008) [2023-12-27 04:30:56,296][105692] Updated weights for policy 0, policy_version 1800992 (0.0008) [2023-12-27 04:30:56,454][105620] Updated weights for policy 1, policy_version 1804939 (0.0007) [2023-12-27 04:30:56,506][105620] Updated weights for policy 1, policy_version 1804949 (0.0010) [2023-12-27 04:30:56,557][105620] Updated weights for policy 1, policy_version 1804959 (0.0010) [2023-12-27 04:30:57,084][105692] Updated weights for policy 0, policy_version 1801002 (0.0009) [2023-12-27 04:30:57,128][105620] Updated weights for policy 1, policy_version 1804969 (0.0010) [2023-12-27 04:30:57,152][105692] Updated weights for policy 0, policy_version 1801012 (0.0010) [2023-12-27 04:30:57,188][105620] Updated weights for policy 1, policy_version 1804979 (0.0005) [2023-12-27 04:30:57,205][105692] Updated weights for policy 0, policy_version 1801022 (0.0009) [2023-12-27 04:30:57,240][105620] Updated weights for policy 1, policy_version 1804989 (0.0007) [2023-12-27 04:30:57,290][105620] Updated weights for policy 1, policy_version 1804999 (0.0009) [2023-12-27 04:30:57,821][105620] Updated weights for policy 1, policy_version 1805009 (0.0005) [2023-12-27 04:30:57,864][105620] Updated weights for policy 1, policy_version 1805019 (0.0005) [2023-12-27 04:30:57,909][105620] Updated weights for policy 1, policy_version 1805029 (0.0005) [2023-12-27 04:30:58,072][105692] Updated weights for policy 0, policy_version 1801032 (0.0008) [2023-12-27 04:30:58,124][105692] Updated weights for policy 0, policy_version 1801042 (0.0008) [2023-12-27 04:30:58,176][105692] Updated weights for policy 0, policy_version 1801052 (0.0008) [2023-12-27 04:30:58,626][105620] Updated weights for policy 1, policy_version 1805039 (0.0009) [2023-12-27 04:30:58,688][105620] Updated weights for policy 1, policy_version 1805049 (0.0011) [2023-12-27 04:30:58,754][105620] Updated weights for policy 1, policy_version 1805059 (0.0009) [2023-12-27 04:30:58,990][105692] Updated weights for policy 0, policy_version 1801062 (0.0011) [2023-12-27 04:30:59,050][105692] Updated weights for policy 0, policy_version 1801072 (0.0009) [2023-12-27 04:30:59,109][105692] Updated weights for policy 0, policy_version 1801082 (0.0010) [2023-12-27 04:30:59,532][105620] Updated weights for policy 1, policy_version 1805069 (0.0006) [2023-12-27 04:30:59,598][105620] Updated weights for policy 1, policy_version 1805079 (0.0006) [2023-12-27 04:30:59,662][105620] Updated weights for policy 1, policy_version 1805089 (0.0009) [2023-12-27 04:30:59,961][105692] Updated weights for policy 0, policy_version 1801092 (0.0008) [2023-12-27 04:31:00,025][105692] Updated weights for policy 0, policy_version 1801102 (0.0007) [2023-12-27 04:31:00,073][105692] Updated weights for policy 0, policy_version 1801112 (0.0008) [2023-12-27 04:31:00,319][105620] Updated weights for policy 1, policy_version 1805099 (0.0008) [2023-12-27 04:31:00,378][105620] Updated weights for policy 1, policy_version 1805109 (0.0011) [2023-12-27 04:31:00,433][105620] Updated weights for policy 1, policy_version 1805119 (0.0010) [2023-12-27 04:31:00,810][105692] Updated weights for policy 0, policy_version 1801122 (0.0007) [2023-12-27 04:31:00,877][105692] Updated weights for policy 0, policy_version 1801132 (0.0005) [2023-12-27 04:31:00,935][105692] Updated weights for policy 0, policy_version 1801142 (0.0005) [2023-12-27 04:31:00,998][105692] Updated weights for policy 0, policy_version 1801152 (0.0008) [2023-12-27 04:31:01,039][105620] Updated weights for policy 1, policy_version 1805129 (0.0010) [2023-12-27 04:31:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19933.8, 300 sec: 19633.0). Total num frames: 923336704. Throughput: 0: 9836.4, 1: 10184.8. Samples: 923306848. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:31:01,062][104569] Avg episode reward: [(0, '8713.467'), (1, '9258.530')] [2023-12-27 04:31:01,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001801152_461160448.pth... [2023-12-27 04:31:01,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001800032_460873728.pth [2023-12-27 04:31:01,104][105620] Updated weights for policy 1, policy_version 1805139 (0.0011) [2023-12-27 04:31:01,171][105620] Updated weights for policy 1, policy_version 1805149 (0.0008) [2023-12-27 04:31:01,223][105620] Updated weights for policy 1, policy_version 1805159 (0.0008) [2023-12-27 04:31:01,229][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001805160_462184448.pth... [2023-12-27 04:31:01,234][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001803944_461873152.pth [2023-12-27 04:31:01,680][105692] Updated weights for policy 0, policy_version 1801162 (0.0010) [2023-12-27 04:31:01,752][105692] Updated weights for policy 0, policy_version 1801172 (0.0008) [2023-12-27 04:31:01,815][105692] Updated weights for policy 0, policy_version 1801182 (0.0008) [2023-12-27 04:31:02,019][105620] Updated weights for policy 1, policy_version 1805169 (0.0010) [2023-12-27 04:31:02,081][105620] Updated weights for policy 1, policy_version 1805179 (0.0007) [2023-12-27 04:31:02,146][105620] Updated weights for policy 1, policy_version 1805189 (0.0008) [2023-12-27 04:31:02,486][105692] Updated weights for policy 0, policy_version 1801192 (0.0008) [2023-12-27 04:31:02,548][105692] Updated weights for policy 0, policy_version 1801202 (0.0007) [2023-12-27 04:31:02,619][105692] Updated weights for policy 0, policy_version 1801212 (0.0005) [2023-12-27 04:31:02,886][105620] Updated weights for policy 1, policy_version 1805199 (0.0010) [2023-12-27 04:31:02,953][105620] Updated weights for policy 1, policy_version 1805209 (0.0011) [2023-12-27 04:31:03,009][105620] Updated weights for policy 1, policy_version 1805219 (0.0011) [2023-12-27 04:31:03,201][105692] Updated weights for policy 0, policy_version 1801222 (0.0007) [2023-12-27 04:31:03,257][105692] Updated weights for policy 0, policy_version 1801232 (0.0008) [2023-12-27 04:31:03,325][105692] Updated weights for policy 0, policy_version 1801242 (0.0008) [2023-12-27 04:31:03,758][105620] Updated weights for policy 1, policy_version 1805229 (0.0008) [2023-12-27 04:31:03,804][105620] Updated weights for policy 1, policy_version 1805239 (0.0006) [2023-12-27 04:31:03,865][105620] Updated weights for policy 1, policy_version 1805249 (0.0009) [2023-12-27 04:31:04,083][105692] Updated weights for policy 0, policy_version 1801252 (0.0010) [2023-12-27 04:31:04,142][105692] Updated weights for policy 0, policy_version 1801262 (0.0011) [2023-12-27 04:31:04,199][105692] Updated weights for policy 0, policy_version 1801272 (0.0011) [2023-12-27 04:31:04,562][105620] Updated weights for policy 1, policy_version 1805259 (0.0008) [2023-12-27 04:31:04,621][105620] Updated weights for policy 1, policy_version 1805269 (0.0009) [2023-12-27 04:31:04,688][105620] Updated weights for policy 1, policy_version 1805279 (0.0008) [2023-12-27 04:31:04,934][105692] Updated weights for policy 0, policy_version 1801282 (0.0008) [2023-12-27 04:31:05,002][105692] Updated weights for policy 0, policy_version 1801292 (0.0010) [2023-12-27 04:31:05,056][105692] Updated weights for policy 0, policy_version 1801303 (0.0009) [2023-12-27 04:31:05,408][105620] Updated weights for policy 1, policy_version 1805289 (0.0009) [2023-12-27 04:31:05,470][105620] Updated weights for policy 1, policy_version 1805299 (0.0006) [2023-12-27 04:31:05,533][105620] Updated weights for policy 1, policy_version 1805309 (0.0009) [2023-12-27 04:31:05,598][105620] Updated weights for policy 1, policy_version 1805319 (0.0009) [2023-12-27 04:31:05,866][105692] Updated weights for policy 0, policy_version 1801313 (0.0009) [2023-12-27 04:31:05,929][105692] Updated weights for policy 0, policy_version 1801323 (0.0009) [2023-12-27 04:31:05,987][105692] Updated weights for policy 0, policy_version 1801333 (0.0009) [2023-12-27 04:31:06,049][105692] Updated weights for policy 0, policy_version 1801343 (0.0009) [2023-12-27 04:31:06,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 923435008. Throughput: 0: 9703.7, 1: 10138.8. Samples: 923421576. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:31:06,062][104569] Avg episode reward: [(0, '8987.911'), (1, '9258.513')] [2023-12-27 04:31:06,312][105620] Updated weights for policy 1, policy_version 1805329 (0.0007) [2023-12-27 04:31:06,388][105620] Updated weights for policy 1, policy_version 1805339 (0.0008) [2023-12-27 04:31:06,455][105620] Updated weights for policy 1, policy_version 1805349 (0.0009) [2023-12-27 04:31:06,755][105692] Updated weights for policy 0, policy_version 1801353 (0.0008) [2023-12-27 04:31:06,817][105692] Updated weights for policy 0, policy_version 1801363 (0.0009) [2023-12-27 04:31:06,871][105692] Updated weights for policy 0, policy_version 1801373 (0.0008) [2023-12-27 04:31:07,159][105620] Updated weights for policy 1, policy_version 1805359 (0.0008) [2023-12-27 04:31:07,225][105620] Updated weights for policy 1, policy_version 1805369 (0.0008) [2023-12-27 04:31:07,289][105620] Updated weights for policy 1, policy_version 1805379 (0.0008) [2023-12-27 04:31:07,711][105692] Updated weights for policy 0, policy_version 1801383 (0.0009) [2023-12-27 04:31:07,764][105692] Updated weights for policy 0, policy_version 1801393 (0.0009) [2023-12-27 04:31:07,819][105692] Updated weights for policy 0, policy_version 1801403 (0.0010) [2023-12-27 04:31:07,989][105620] Updated weights for policy 1, policy_version 1805389 (0.0007) [2023-12-27 04:31:08,043][105620] Updated weights for policy 1, policy_version 1805399 (0.0008) [2023-12-27 04:31:08,094][105620] Updated weights for policy 1, policy_version 1805409 (0.0009) [2023-12-27 04:31:08,598][105692] Updated weights for policy 0, policy_version 1801413 (0.0010) [2023-12-27 04:31:08,668][105692] Updated weights for policy 0, policy_version 1801423 (0.0011) [2023-12-27 04:31:08,730][105692] Updated weights for policy 0, policy_version 1801433 (0.0010) [2023-12-27 04:31:08,884][105620] Updated weights for policy 1, policy_version 1805419 (0.0009) [2023-12-27 04:31:08,950][105620] Updated weights for policy 1, policy_version 1805429 (0.0008) [2023-12-27 04:31:09,012][105620] Updated weights for policy 1, policy_version 1805439 (0.0009) [2023-12-27 04:31:09,451][105692] Updated weights for policy 0, policy_version 1801443 (0.0007) [2023-12-27 04:31:09,512][105692] Updated weights for policy 0, policy_version 1801453 (0.0008) [2023-12-27 04:31:09,575][105692] Updated weights for policy 0, policy_version 1801463 (0.0008) [2023-12-27 04:31:09,782][105620] Updated weights for policy 1, policy_version 1805449 (0.0009) [2023-12-27 04:31:09,853][105620] Updated weights for policy 1, policy_version 1805459 (0.0008) [2023-12-27 04:31:09,922][105620] Updated weights for policy 1, policy_version 1805469 (0.0009) [2023-12-27 04:31:09,995][105620] Updated weights for policy 1, policy_version 1805479 (0.0008) [2023-12-27 04:31:10,323][105692] Updated weights for policy 0, policy_version 1801473 (0.0009) [2023-12-27 04:31:10,388][105692] Updated weights for policy 0, policy_version 1801483 (0.0009) [2023-12-27 04:31:10,458][105692] Updated weights for policy 0, policy_version 1801493 (0.0009) [2023-12-27 04:31:10,521][105692] Updated weights for policy 0, policy_version 1801503 (0.0010) [2023-12-27 04:31:10,674][105620] Updated weights for policy 1, policy_version 1805489 (0.0008) [2023-12-27 04:31:10,726][105620] Updated weights for policy 1, policy_version 1805499 (0.0008) [2023-12-27 04:31:10,782][105620] Updated weights for policy 1, policy_version 1805509 (0.0008) [2023-12-27 04:31:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 923525120. Throughput: 0: 9727.6, 1: 10048.8. Samples: 923533044. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:31:11,063][104569] Avg episode reward: [(0, '8992.233'), (1, '9351.043')] [2023-12-27 04:31:11,256][105692] Updated weights for policy 0, policy_version 1801513 (0.0010) [2023-12-27 04:31:11,315][105692] Updated weights for policy 0, policy_version 1801523 (0.0008) [2023-12-27 04:31:11,386][105692] Updated weights for policy 0, policy_version 1801533 (0.0011) [2023-12-27 04:31:11,584][105620] Updated weights for policy 1, policy_version 1805519 (0.0009) [2023-12-27 04:31:11,652][105620] Updated weights for policy 1, policy_version 1805529 (0.0008) [2023-12-27 04:31:11,715][105620] Updated weights for policy 1, policy_version 1805539 (0.0009) [2023-12-27 04:31:12,179][105692] Updated weights for policy 0, policy_version 1801543 (0.0007) [2023-12-27 04:31:12,245][105692] Updated weights for policy 0, policy_version 1801553 (0.0007) [2023-12-27 04:31:12,312][105692] Updated weights for policy 0, policy_version 1801563 (0.0009) [2023-12-27 04:31:12,544][105620] Updated weights for policy 1, policy_version 1805549 (0.0009) [2023-12-27 04:31:12,612][105620] Updated weights for policy 1, policy_version 1805559 (0.0008) [2023-12-27 04:31:12,678][105620] Updated weights for policy 1, policy_version 1805569 (0.0008) [2023-12-27 04:31:13,015][105692] Updated weights for policy 0, policy_version 1801573 (0.0007) [2023-12-27 04:31:13,066][105692] Updated weights for policy 0, policy_version 1801583 (0.0005) [2023-12-27 04:31:13,070][105585] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000004 [2023-12-27 04:31:13,324][105620] Updated weights for policy 1, policy_version 1805579 (0.0008) [2023-12-27 04:31:13,377][105620] Updated weights for policy 1, policy_version 1805589 (0.0009) [2023-12-27 04:31:13,429][105620] Updated weights for policy 1, policy_version 1805599 (0.0009) [2023-12-27 04:31:13,811][105692] Updated weights for policy 0, policy_version 1801593 (0.0008) [2023-12-27 04:31:13,855][105692] Updated weights for policy 0, policy_version 1801603 (0.0007) [2023-12-27 04:31:13,915][105692] Updated weights for policy 0, policy_version 1801613 (0.0005) [2023-12-27 04:31:14,292][105620] Updated weights for policy 1, policy_version 1805610 (0.0010) [2023-12-27 04:31:14,361][105620] Updated weights for policy 1, policy_version 1805620 (0.0009) [2023-12-27 04:31:14,421][105620] Updated weights for policy 1, policy_version 1805630 (0.0009) [2023-12-27 04:31:14,478][105620] Updated weights for policy 1, policy_version 1805640 (0.0009) [2023-12-27 04:31:14,480][105692] Updated weights for policy 0, policy_version 1801623 (0.0006) [2023-12-27 04:31:14,538][105692] Updated weights for policy 0, policy_version 1801633 (0.0005) [2023-12-27 04:31:14,606][105692] Updated weights for policy 0, policy_version 1801643 (0.0006) [2023-12-27 04:31:15,187][105620] Updated weights for policy 1, policy_version 1805650 (0.0007) [2023-12-27 04:31:15,259][105620] Updated weights for policy 1, policy_version 1805660 (0.0011) [2023-12-27 04:31:15,259][105692] Updated weights for policy 0, policy_version 1801653 (0.0006) [2023-12-27 04:31:15,314][105692] Updated weights for policy 0, policy_version 1801663 (0.0009) [2023-12-27 04:31:15,326][105620] Updated weights for policy 1, policy_version 1805670 (0.0011) [2023-12-27 04:31:15,383][105692] Updated weights for policy 0, policy_version 1801673 (0.0008) [2023-12-27 04:31:16,056][105692] Updated weights for policy 0, policy_version 1801683 (0.0011) [2023-12-27 04:31:16,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19660.9, 300 sec: 19633.0). Total num frames: 923615232. Throughput: 0: 9571.6, 1: 9916.1. Samples: 923589536. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 04:31:16,062][104569] Avg episode reward: [(0, '8449.363'), (1, '9351.054')] [2023-12-27 04:31:16,067][105620] Updated weights for policy 1, policy_version 1805680 (0.0006) [2023-12-27 04:31:16,101][105692] Updated weights for policy 0, policy_version 1801693 (0.0010) [2023-12-27 04:31:16,121][105620] Updated weights for policy 1, policy_version 1805690 (0.0005) [2023-12-27 04:31:16,153][105692] Updated weights for policy 0, policy_version 1801703 (0.0010) [2023-12-27 04:31:16,173][105620] Updated weights for policy 1, policy_version 1805700 (0.0007) [2023-12-27 04:31:16,190][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001805704_462323712.pth... [2023-12-27 04:31:16,193][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001804520_462020608.pth [2023-12-27 04:31:16,204][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001801712_461307904.pth... [2023-12-27 04:31:16,207][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001800608_461021184.pth [2023-12-27 04:31:16,711][105620] Updated weights for policy 1, policy_version 1805710 (0.0006) [2023-12-27 04:31:16,755][105620] Updated weights for policy 1, policy_version 1805720 (0.0006) [2023-12-27 04:31:16,802][105620] Updated weights for policy 1, policy_version 1805730 (0.0005) [2023-12-27 04:31:16,923][105692] Updated weights for policy 0, policy_version 1801713 (0.0010) [2023-12-27 04:31:16,992][105692] Updated weights for policy 0, policy_version 1801723 (0.0011) [2023-12-27 04:31:17,047][105692] Updated weights for policy 0, policy_version 1801733 (0.0010) [2023-12-27 04:31:17,099][105692] Updated weights for policy 0, policy_version 1801743 (0.0010) [2023-12-27 04:31:17,400][105620] Updated weights for policy 1, policy_version 1805740 (0.0006) [2023-12-27 04:31:17,462][105620] Updated weights for policy 1, policy_version 1805750 (0.0005) [2023-12-27 04:31:17,524][105620] Updated weights for policy 1, policy_version 1805760 (0.0007) [2023-12-27 04:31:17,804][105692] Updated weights for policy 0, policy_version 1801753 (0.0006) [2023-12-27 04:31:17,853][105692] Updated weights for policy 0, policy_version 1801763 (0.0005) [2023-12-27 04:31:17,910][105692] Updated weights for policy 0, policy_version 1801773 (0.0005) [2023-12-27 04:31:18,162][105620] Updated weights for policy 1, policy_version 1805770 (0.0007) [2023-12-27 04:31:18,223][105620] Updated weights for policy 1, policy_version 1805780 (0.0008) [2023-12-27 04:31:18,276][105620] Updated weights for policy 1, policy_version 1805790 (0.0008) [2023-12-27 04:31:18,322][105620] Updated weights for policy 1, policy_version 1805800 (0.0010) [2023-12-27 04:31:18,569][105692] Updated weights for policy 0, policy_version 1801783 (0.0006) [2023-12-27 04:31:18,635][105692] Updated weights for policy 0, policy_version 1801793 (0.0005) [2023-12-27 04:31:18,704][105692] Updated weights for policy 0, policy_version 1801803 (0.0007) [2023-12-27 04:31:19,137][105620] Updated weights for policy 1, policy_version 1805810 (0.0010) [2023-12-27 04:31:19,192][105620] Updated weights for policy 1, policy_version 1805820 (0.0010) [2023-12-27 04:31:19,254][105620] Updated weights for policy 1, policy_version 1805830 (0.0011) [2023-12-27 04:31:19,272][105692] Updated weights for policy 0, policy_version 1801813 (0.0010) [2023-12-27 04:31:19,324][105692] Updated weights for policy 0, policy_version 1801823 (0.0011) [2023-12-27 04:31:19,399][105692] Updated weights for policy 0, policy_version 1801833 (0.0009) [2023-12-27 04:31:20,042][105620] Updated weights for policy 1, policy_version 1805840 (0.0011) [2023-12-27 04:31:20,102][105620] Updated weights for policy 1, policy_version 1805850 (0.0008) [2023-12-27 04:31:20,166][105620] Updated weights for policy 1, policy_version 1805860 (0.0011) [2023-12-27 04:31:20,182][105692] Updated weights for policy 0, policy_version 1801843 (0.0011) [2023-12-27 04:31:20,241][105692] Updated weights for policy 0, policy_version 1801853 (0.0011) [2023-12-27 04:31:20,305][105692] Updated weights for policy 0, policy_version 1801863 (0.0011) [2023-12-27 04:31:20,917][105620] Updated weights for policy 1, policy_version 1805870 (0.0011) [2023-12-27 04:31:20,988][105620] Updated weights for policy 1, policy_version 1805880 (0.0011) [2023-12-27 04:31:21,056][105620] Updated weights for policy 1, policy_version 1805890 (0.0011) [2023-12-27 04:31:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 923713536. Throughput: 0: 9599.9, 1: 9850.9. Samples: 923711184. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:31:21,062][104569] Avg episode reward: [(0, '8536.737'), (1, '9258.693')] [2023-12-27 04:31:21,065][105692] Updated weights for policy 0, policy_version 1801873 (0.0011) [2023-12-27 04:31:21,125][105692] Updated weights for policy 0, policy_version 1801883 (0.0011) [2023-12-27 04:31:21,182][105692] Updated weights for policy 0, policy_version 1801893 (0.0011) [2023-12-27 04:31:21,228][105692] Updated weights for policy 0, policy_version 1801903 (0.0010) [2023-12-27 04:31:21,812][105620] Updated weights for policy 1, policy_version 1805900 (0.0009) [2023-12-27 04:31:21,864][105620] Updated weights for policy 1, policy_version 1805910 (0.0007) [2023-12-27 04:31:21,919][105620] Updated weights for policy 1, policy_version 1805920 (0.0008) [2023-12-27 04:31:22,038][105692] Updated weights for policy 0, policy_version 1801913 (0.0011) [2023-12-27 04:31:22,084][105692] Updated weights for policy 0, policy_version 1801923 (0.0009) [2023-12-27 04:31:22,135][105692] Updated weights for policy 0, policy_version 1801933 (0.0006) [2023-12-27 04:31:22,670][105620] Updated weights for policy 1, policy_version 1805930 (0.0009) [2023-12-27 04:31:22,725][105620] Updated weights for policy 1, policy_version 1805940 (0.0009) [2023-12-27 04:31:22,777][105620] Updated weights for policy 1, policy_version 1805950 (0.0009) [2023-12-27 04:31:22,833][105620] Updated weights for policy 1, policy_version 1805960 (0.0009) [2023-12-27 04:31:22,915][105692] Updated weights for policy 0, policy_version 1801943 (0.0010) [2023-12-27 04:31:22,962][105692] Updated weights for policy 0, policy_version 1801953 (0.0008) [2023-12-27 04:31:23,018][105692] Updated weights for policy 0, policy_version 1801963 (0.0008) [2023-12-27 04:31:23,606][105620] Updated weights for policy 1, policy_version 1805970 (0.0009) [2023-12-27 04:31:23,667][105620] Updated weights for policy 1, policy_version 1805980 (0.0008) [2023-12-27 04:31:23,728][105620] Updated weights for policy 1, policy_version 1805990 (0.0009) [2023-12-27 04:31:23,798][105692] Updated weights for policy 0, policy_version 1801973 (0.0008) [2023-12-27 04:31:23,865][105692] Updated weights for policy 0, policy_version 1801983 (0.0009) [2023-12-27 04:31:23,915][105692] Updated weights for policy 0, policy_version 1801993 (0.0008) [2023-12-27 04:31:24,513][105620] Updated weights for policy 1, policy_version 1806000 (0.0010) [2023-12-27 04:31:24,563][105620] Updated weights for policy 1, policy_version 1806010 (0.0008) [2023-12-27 04:31:24,589][105692] Updated weights for policy 0, policy_version 1802003 (0.0008) [2023-12-27 04:31:24,615][105620] Updated weights for policy 1, policy_version 1806020 (0.0009) [2023-12-27 04:31:24,646][105692] Updated weights for policy 0, policy_version 1802013 (0.0007) [2023-12-27 04:31:24,704][105692] Updated weights for policy 0, policy_version 1802023 (0.0009) [2023-12-27 04:31:25,393][105620] Updated weights for policy 1, policy_version 1806030 (0.0010) [2023-12-27 04:31:25,449][105692] Updated weights for policy 0, policy_version 1802033 (0.0009) [2023-12-27 04:31:25,456][105620] Updated weights for policy 1, policy_version 1806040 (0.0008) [2023-12-27 04:31:25,495][105692] Updated weights for policy 0, policy_version 1802043 (0.0007) [2023-12-27 04:31:25,514][105620] Updated weights for policy 1, policy_version 1806050 (0.0007) [2023-12-27 04:31:25,538][105692] Updated weights for policy 0, policy_version 1802053 (0.0007) [2023-12-27 04:31:26,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.3, 300 sec: 19605.2). Total num frames: 923811840. Throughput: 0: 9582.6, 1: 9752.2. Samples: 923821384. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:31:26,063][104569] Avg episode reward: [(0, '8896.387'), (1, '8985.564')] [2023-12-27 04:31:26,280][105620] Updated weights for policy 1, policy_version 1806060 (0.0008) [2023-12-27 04:31:26,330][105692] Updated weights for policy 0, policy_version 1802065 (0.0009) [2023-12-27 04:31:26,331][105620] Updated weights for policy 1, policy_version 1806070 (0.0009) [2023-12-27 04:31:26,389][105692] Updated weights for policy 0, policy_version 1802075 (0.0005) [2023-12-27 04:31:26,398][105620] Updated weights for policy 1, policy_version 1806080 (0.0009) [2023-12-27 04:31:26,447][105692] Updated weights for policy 0, policy_version 1802085 (0.0005) [2023-12-27 04:31:26,504][105692] Updated weights for policy 0, policy_version 1802095 (0.0005) [2023-12-27 04:31:27,117][105692] Updated weights for policy 0, policy_version 1802105 (0.0009) [2023-12-27 04:31:27,166][105692] Updated weights for policy 0, policy_version 1802115 (0.0008) [2023-12-27 04:31:27,178][105620] Updated weights for policy 1, policy_version 1806090 (0.0009) [2023-12-27 04:31:27,214][105692] Updated weights for policy 0, policy_version 1802125 (0.0007) [2023-12-27 04:31:27,228][105620] Updated weights for policy 1, policy_version 1806100 (0.0009) [2023-12-27 04:31:27,273][105620] Updated weights for policy 1, policy_version 1806110 (0.0008) [2023-12-27 04:31:27,329][105620] Updated weights for policy 1, policy_version 1806120 (0.0008) [2023-12-27 04:31:27,803][105692] Updated weights for policy 0, policy_version 1802135 (0.0008) [2023-12-27 04:31:27,860][105692] Updated weights for policy 0, policy_version 1802145 (0.0010) [2023-12-27 04:31:27,924][105692] Updated weights for policy 0, policy_version 1802155 (0.0010) [2023-12-27 04:31:28,217][105620] Updated weights for policy 1, policy_version 1806130 (0.0010) [2023-12-27 04:31:28,271][105620] Updated weights for policy 1, policy_version 1806141 (0.0010) [2023-12-27 04:31:28,328][105620] Updated weights for policy 1, policy_version 1806152 (0.0009) [2023-12-27 04:31:28,494][105692] Updated weights for policy 0, policy_version 1802165 (0.0007) [2023-12-27 04:31:28,547][105692] Updated weights for policy 0, policy_version 1802175 (0.0007) [2023-12-27 04:31:28,614][105692] Updated weights for policy 0, policy_version 1802185 (0.0010) [2023-12-27 04:31:29,085][105620] Updated weights for policy 1, policy_version 1806162 (0.0005) [2023-12-27 04:31:29,130][105620] Updated weights for policy 1, policy_version 1806172 (0.0008) [2023-12-27 04:31:29,185][105620] Updated weights for policy 1, policy_version 1806182 (0.0008) [2023-12-27 04:31:29,330][105692] Updated weights for policy 0, policy_version 1802195 (0.0009) [2023-12-27 04:31:29,397][105692] Updated weights for policy 0, policy_version 1802205 (0.0010) [2023-12-27 04:31:29,454][105692] Updated weights for policy 0, policy_version 1802215 (0.0007) [2023-12-27 04:31:30,011][105620] Updated weights for policy 1, policy_version 1806192 (0.0008) [2023-12-27 04:31:30,070][105620] Updated weights for policy 1, policy_version 1806202 (0.0008) [2023-12-27 04:31:30,094][105692] Updated weights for policy 0, policy_version 1802225 (0.0010) [2023-12-27 04:31:30,135][105620] Updated weights for policy 1, policy_version 1806212 (0.0010) [2023-12-27 04:31:30,148][105692] Updated weights for policy 0, policy_version 1802235 (0.0005) [2023-12-27 04:31:30,206][105692] Updated weights for policy 0, policy_version 1802245 (0.0007) [2023-12-27 04:31:30,268][105692] Updated weights for policy 0, policy_version 1802255 (0.0006) [2023-12-27 04:31:30,790][105692] Updated weights for policy 0, policy_version 1802265 (0.0008) [2023-12-27 04:31:30,847][105692] Updated weights for policy 0, policy_version 1802275 (0.0010) [2023-12-27 04:31:30,910][105692] Updated weights for policy 0, policy_version 1802285 (0.0009) [2023-12-27 04:31:30,985][105620] Updated weights for policy 1, policy_version 1806222 (0.0008) [2023-12-27 04:31:31,048][105620] Updated weights for policy 1, policy_version 1806232 (0.0008) [2023-12-27 04:31:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 923910144. Throughput: 0: 9628.6, 1: 9721.8. Samples: 923880476. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:31:31,062][104569] Avg episode reward: [(0, '8532.194'), (1, '8893.182')] [2023-12-27 04:31:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001802288_461455360.pth... [2023-12-27 04:31:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001801152_461160448.pth [2023-12-27 04:31:31,101][105620] Updated weights for policy 1, policy_version 1806242 (0.0008) [2023-12-27 04:31:31,134][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001806248_462462976.pth... [2023-12-27 04:31:31,139][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001805160_462184448.pth [2023-12-27 04:31:31,653][105692] Updated weights for policy 0, policy_version 1802295 (0.0010) [2023-12-27 04:31:31,711][105692] Updated weights for policy 0, policy_version 1802305 (0.0010) [2023-12-27 04:31:31,769][105692] Updated weights for policy 0, policy_version 1802315 (0.0010) [2023-12-27 04:31:31,914][105620] Updated weights for policy 1, policy_version 1806252 (0.0008) [2023-12-27 04:31:31,959][105620] Updated weights for policy 1, policy_version 1806262 (0.0008) [2023-12-27 04:31:32,019][105620] Updated weights for policy 1, policy_version 1806272 (0.0008) [2023-12-27 04:31:32,518][105692] Updated weights for policy 0, policy_version 1802325 (0.0011) [2023-12-27 04:31:32,570][105692] Updated weights for policy 0, policy_version 1802335 (0.0010) [2023-12-27 04:31:32,625][105692] Updated weights for policy 0, policy_version 1802345 (0.0010) [2023-12-27 04:31:32,794][105620] Updated weights for policy 1, policy_version 1806282 (0.0008) [2023-12-27 04:31:32,845][105620] Updated weights for policy 1, policy_version 1806292 (0.0007) [2023-12-27 04:31:32,896][105620] Updated weights for policy 1, policy_version 1806302 (0.0008) [2023-12-27 04:31:32,948][105620] Updated weights for policy 1, policy_version 1806312 (0.0008) [2023-12-27 04:31:33,381][105692] Updated weights for policy 0, policy_version 1802355 (0.0010) [2023-12-27 04:31:33,432][105692] Updated weights for policy 0, policy_version 1802365 (0.0010) [2023-12-27 04:31:33,482][105692] Updated weights for policy 0, policy_version 1802375 (0.0010) [2023-12-27 04:31:33,722][105620] Updated weights for policy 1, policy_version 1806322 (0.0008) [2023-12-27 04:31:33,776][105620] Updated weights for policy 1, policy_version 1806332 (0.0008) [2023-12-27 04:31:33,832][105620] Updated weights for policy 1, policy_version 1806342 (0.0008) [2023-12-27 04:31:34,221][105692] Updated weights for policy 0, policy_version 1802385 (0.0010) [2023-12-27 04:31:34,286][105692] Updated weights for policy 0, policy_version 1802395 (0.0010) [2023-12-27 04:31:34,346][105692] Updated weights for policy 0, policy_version 1802405 (0.0010) [2023-12-27 04:31:34,398][105692] Updated weights for policy 0, policy_version 1802415 (0.0011) [2023-12-27 04:31:34,636][105620] Updated weights for policy 1, policy_version 1806352 (0.0008) [2023-12-27 04:31:34,697][105620] Updated weights for policy 1, policy_version 1806362 (0.0009) [2023-12-27 04:31:34,758][105620] Updated weights for policy 1, policy_version 1806372 (0.0009) [2023-12-27 04:31:35,146][105692] Updated weights for policy 0, policy_version 1802425 (0.0009) [2023-12-27 04:31:35,207][105692] Updated weights for policy 0, policy_version 1802435 (0.0006) [2023-12-27 04:31:35,255][105692] Updated weights for policy 0, policy_version 1802445 (0.0005) [2023-12-27 04:31:35,563][105620] Updated weights for policy 1, policy_version 1806382 (0.0009) [2023-12-27 04:31:35,617][105620] Updated weights for policy 1, policy_version 1806392 (0.0010) [2023-12-27 04:31:35,669][105620] Updated weights for policy 1, policy_version 1806402 (0.0009) [2023-12-27 04:31:35,803][105692] Updated weights for policy 0, policy_version 1802455 (0.0007) [2023-12-27 04:31:35,865][105692] Updated weights for policy 0, policy_version 1802465 (0.0005) [2023-12-27 04:31:35,922][105692] Updated weights for policy 0, policy_version 1802475 (0.0007) [2023-12-27 04:31:36,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 924008448. Throughput: 0: 9554.4, 1: 9522.0. Samples: 923993488. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:31:36,063][104569] Avg episode reward: [(0, '8534.576'), (1, '8981.435')] [2023-12-27 04:31:36,433][105620] Updated weights for policy 1, policy_version 1806412 (0.0008) [2023-12-27 04:31:36,492][105620] Updated weights for policy 1, policy_version 1806422 (0.0006) [2023-12-27 04:31:36,551][105620] Updated weights for policy 1, policy_version 1806432 (0.0006) [2023-12-27 04:31:36,666][105692] Updated weights for policy 0, policy_version 1802485 (0.0010) [2023-12-27 04:31:36,723][105692] Updated weights for policy 0, policy_version 1802496 (0.0010) [2023-12-27 04:31:36,778][105692] Updated weights for policy 0, policy_version 1802507 (0.0009) [2023-12-27 04:31:37,192][105620] Updated weights for policy 1, policy_version 1806442 (0.0007) [2023-12-27 04:31:37,257][105620] Updated weights for policy 1, policy_version 1806452 (0.0005) [2023-12-27 04:31:37,318][105620] Updated weights for policy 1, policy_version 1806462 (0.0005) [2023-12-27 04:31:37,381][105620] Updated weights for policy 1, policy_version 1806472 (0.0008) [2023-12-27 04:31:37,646][105692] Updated weights for policy 0, policy_version 1802517 (0.0010) [2023-12-27 04:31:37,705][105692] Updated weights for policy 0, policy_version 1802527 (0.0009) [2023-12-27 04:31:37,765][105692] Updated weights for policy 0, policy_version 1802537 (0.0009) [2023-12-27 04:31:38,060][105620] Updated weights for policy 1, policy_version 1806482 (0.0008) [2023-12-27 04:31:38,117][105620] Updated weights for policy 1, policy_version 1806492 (0.0008) [2023-12-27 04:31:38,171][105620] Updated weights for policy 1, policy_version 1806502 (0.0006) [2023-12-27 04:31:38,509][105692] Updated weights for policy 0, policy_version 1802547 (0.0009) [2023-12-27 04:31:38,563][105692] Updated weights for policy 0, policy_version 1802557 (0.0009) [2023-12-27 04:31:38,623][105692] Updated weights for policy 0, policy_version 1802567 (0.0009) [2023-12-27 04:31:38,929][105620] Updated weights for policy 1, policy_version 1806512 (0.0008) [2023-12-27 04:31:38,987][105620] Updated weights for policy 1, policy_version 1806522 (0.0010) [2023-12-27 04:31:39,039][105620] Updated weights for policy 1, policy_version 1806532 (0.0009) [2023-12-27 04:31:39,343][105692] Updated weights for policy 0, policy_version 1802577 (0.0010) [2023-12-27 04:31:39,413][105692] Updated weights for policy 0, policy_version 1802587 (0.0009) [2023-12-27 04:31:39,474][105692] Updated weights for policy 0, policy_version 1802597 (0.0009) [2023-12-27 04:31:39,542][105692] Updated weights for policy 0, policy_version 1802607 (0.0008) [2023-12-27 04:31:39,754][105620] Updated weights for policy 1, policy_version 1806542 (0.0008) [2023-12-27 04:31:39,802][105620] Updated weights for policy 1, policy_version 1806552 (0.0009) [2023-12-27 04:31:39,869][105620] Updated weights for policy 1, policy_version 1806562 (0.0008) [2023-12-27 04:31:40,322][105692] Updated weights for policy 0, policy_version 1802617 (0.0008) [2023-12-27 04:31:40,382][105692] Updated weights for policy 0, policy_version 1802627 (0.0008) [2023-12-27 04:31:40,438][105692] Updated weights for policy 0, policy_version 1802637 (0.0007) [2023-12-27 04:31:40,530][105620] Updated weights for policy 1, policy_version 1806572 (0.0009) [2023-12-27 04:31:40,592][105620] Updated weights for policy 1, policy_version 1806582 (0.0009) [2023-12-27 04:31:40,647][105620] Updated weights for policy 1, policy_version 1806592 (0.0009) [2023-12-27 04:31:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 924098560. Throughput: 0: 9626.5, 1: 9484.7. Samples: 924108412. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:31:41,063][104569] Avg episode reward: [(0, '8809.742'), (1, '9073.624')] [2023-12-27 04:31:41,136][105692] Updated weights for policy 0, policy_version 1802647 (0.0008) [2023-12-27 04:31:41,200][105692] Updated weights for policy 0, policy_version 1802657 (0.0009) [2023-12-27 04:31:41,266][105692] Updated weights for policy 0, policy_version 1802667 (0.0008) [2023-12-27 04:31:41,507][105620] Updated weights for policy 1, policy_version 1806602 (0.0009) [2023-12-27 04:31:41,562][105620] Updated weights for policy 1, policy_version 1806612 (0.0008) [2023-12-27 04:31:41,630][105620] Updated weights for policy 1, policy_version 1806622 (0.0009) [2023-12-27 04:31:41,699][105620] Updated weights for policy 1, policy_version 1806632 (0.0008) [2023-12-27 04:31:41,992][105692] Updated weights for policy 0, policy_version 1802677 (0.0007) [2023-12-27 04:31:42,057][105692] Updated weights for policy 0, policy_version 1802687 (0.0008) [2023-12-27 04:31:42,124][105692] Updated weights for policy 0, policy_version 1802697 (0.0006) [2023-12-27 04:31:42,464][105620] Updated weights for policy 1, policy_version 1806642 (0.0009) [2023-12-27 04:31:42,516][105620] Updated weights for policy 1, policy_version 1806652 (0.0009) [2023-12-27 04:31:42,573][105620] Updated weights for policy 1, policy_version 1806662 (0.0009) [2023-12-27 04:31:42,796][105692] Updated weights for policy 0, policy_version 1802707 (0.0008) [2023-12-27 04:31:42,869][105692] Updated weights for policy 0, policy_version 1802717 (0.0006) [2023-12-27 04:31:42,929][105692] Updated weights for policy 0, policy_version 1802727 (0.0009) [2023-12-27 04:31:43,278][105620] Updated weights for policy 1, policy_version 1806672 (0.0009) [2023-12-27 04:31:43,333][105620] Updated weights for policy 1, policy_version 1806682 (0.0009) [2023-12-27 04:31:43,391][105620] Updated weights for policy 1, policy_version 1806692 (0.0009) [2023-12-27 04:31:43,626][105692] Updated weights for policy 0, policy_version 1802737 (0.0008) [2023-12-27 04:31:43,685][105692] Updated weights for policy 0, policy_version 1802747 (0.0005) [2023-12-27 04:31:43,731][105692] Updated weights for policy 0, policy_version 1802757 (0.0007) [2023-12-27 04:31:43,779][105692] Updated weights for policy 0, policy_version 1802767 (0.0009) [2023-12-27 04:31:44,219][105620] Updated weights for policy 1, policy_version 1806702 (0.0009) [2023-12-27 04:31:44,272][105620] Updated weights for policy 1, policy_version 1806712 (0.0009) [2023-12-27 04:31:44,325][105620] Updated weights for policy 1, policy_version 1806723 (0.0010) [2023-12-27 04:31:44,377][105692] Updated weights for policy 0, policy_version 1802777 (0.0006) [2023-12-27 04:31:44,446][105692] Updated weights for policy 0, policy_version 1802787 (0.0005) [2023-12-27 04:31:44,510][105692] Updated weights for policy 0, policy_version 1802797 (0.0006) [2023-12-27 04:31:45,061][105692] Updated weights for policy 0, policy_version 1802807 (0.0008) [2023-12-27 04:31:45,078][105620] Updated weights for policy 1, policy_version 1806733 (0.0008) [2023-12-27 04:31:45,112][105692] Updated weights for policy 0, policy_version 1802817 (0.0007) [2023-12-27 04:31:45,138][105620] Updated weights for policy 1, policy_version 1806743 (0.0007) [2023-12-27 04:31:45,173][105692] Updated weights for policy 0, policy_version 1802827 (0.0007) [2023-12-27 04:31:45,196][105620] Updated weights for policy 1, policy_version 1806753 (0.0006) [2023-12-27 04:31:45,811][105692] Updated weights for policy 0, policy_version 1802837 (0.0006) [2023-12-27 04:31:45,863][105692] Updated weights for policy 0, policy_version 1802847 (0.0005) [2023-12-27 04:31:45,923][105692] Updated weights for policy 0, policy_version 1802857 (0.0005) [2023-12-27 04:31:45,981][105620] Updated weights for policy 1, policy_version 1806763 (0.0009) [2023-12-27 04:31:46,039][105620] Updated weights for policy 1, policy_version 1806773 (0.0010) [2023-12-27 04:31:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.3, 300 sec: 19605.3). Total num frames: 924196864. Throughput: 0: 9718.2, 1: 9370.0. Samples: 924165816. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:31:46,063][104569] Avg episode reward: [(0, '8807.458'), (1, '9350.943')] [2023-12-27 04:31:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001802864_461602816.pth... [2023-12-27 04:31:46,082][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001801712_461307904.pth [2023-12-27 04:31:46,102][105620] Updated weights for policy 1, policy_version 1806783 (0.0009) [2023-12-27 04:31:46,155][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001806792_462602240.pth... [2023-12-27 04:31:46,159][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001805704_462323712.pth [2023-12-27 04:31:46,546][105692] Updated weights for policy 0, policy_version 1802867 (0.0005) [2023-12-27 04:31:46,609][105692] Updated weights for policy 0, policy_version 1802877 (0.0005) [2023-12-27 04:31:46,662][105692] Updated weights for policy 0, policy_version 1802887 (0.0005) [2023-12-27 04:31:46,904][105620] Updated weights for policy 1, policy_version 1806793 (0.0006) [2023-12-27 04:31:46,957][105620] Updated weights for policy 1, policy_version 1806803 (0.0009) [2023-12-27 04:31:47,011][105620] Updated weights for policy 1, policy_version 1806813 (0.0010) [2023-12-27 04:31:47,079][105620] Updated weights for policy 1, policy_version 1806823 (0.0009) [2023-12-27 04:31:47,204][105692] Updated weights for policy 0, policy_version 1802897 (0.0006) [2023-12-27 04:31:47,251][105692] Updated weights for policy 0, policy_version 1802907 (0.0009) [2023-12-27 04:31:47,304][105692] Updated weights for policy 0, policy_version 1802917 (0.0009) [2023-12-27 04:31:47,355][105692] Updated weights for policy 0, policy_version 1802927 (0.0009) [2023-12-27 04:31:47,892][105620] Updated weights for policy 1, policy_version 1806833 (0.0008) [2023-12-27 04:31:47,949][105620] Updated weights for policy 1, policy_version 1806843 (0.0009) [2023-12-27 04:31:48,003][105620] Updated weights for policy 1, policy_version 1806854 (0.0010) [2023-12-27 04:31:48,016][105692] Updated weights for policy 0, policy_version 1802937 (0.0006) [2023-12-27 04:31:48,085][105692] Updated weights for policy 0, policy_version 1802947 (0.0006) [2023-12-27 04:31:48,139][105692] Updated weights for policy 0, policy_version 1802957 (0.0005) [2023-12-27 04:31:48,759][105692] Updated weights for policy 0, policy_version 1802967 (0.0009) [2023-12-27 04:31:48,805][105620] Updated weights for policy 1, policy_version 1806864 (0.0006) [2023-12-27 04:31:48,815][105692] Updated weights for policy 0, policy_version 1802977 (0.0011) [2023-12-27 04:31:48,861][105620] Updated weights for policy 1, policy_version 1806874 (0.0006) [2023-12-27 04:31:48,874][105692] Updated weights for policy 0, policy_version 1802987 (0.0011) [2023-12-27 04:31:48,917][105620] Updated weights for policy 1, policy_version 1806884 (0.0006) [2023-12-27 04:31:49,640][105692] Updated weights for policy 0, policy_version 1802997 (0.0011) [2023-12-27 04:31:49,688][105620] Updated weights for policy 1, policy_version 1806894 (0.0009) [2023-12-27 04:31:49,702][105692] Updated weights for policy 0, policy_version 1803007 (0.0011) [2023-12-27 04:31:49,754][105620] Updated weights for policy 1, policy_version 1806904 (0.0007) [2023-12-27 04:31:49,764][105692] Updated weights for policy 0, policy_version 1803017 (0.0007) [2023-12-27 04:31:49,801][105620] Updated weights for policy 1, policy_version 1806914 (0.0008) [2023-12-27 04:31:50,399][105692] Updated weights for policy 0, policy_version 1803027 (0.0009) [2023-12-27 04:31:50,456][105692] Updated weights for policy 0, policy_version 1803037 (0.0011) [2023-12-27 04:31:50,504][105692] Updated weights for policy 0, policy_version 1803047 (0.0010) [2023-12-27 04:31:50,631][105620] Updated weights for policy 1, policy_version 1806924 (0.0008) [2023-12-27 04:31:50,693][105620] Updated weights for policy 1, policy_version 1806934 (0.0008) [2023-12-27 04:31:50,752][105620] Updated weights for policy 1, policy_version 1806944 (0.0008) [2023-12-27 04:31:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19114.7, 300 sec: 19577.5). Total num frames: 924295168. Throughput: 0: 9914.3, 1: 9265.9. Samples: 924284684. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:31:51,063][104569] Avg episode reward: [(0, '8718.937'), (1, '9258.548')] [2023-12-27 04:31:51,256][105692] Updated weights for policy 0, policy_version 1803057 (0.0010) [2023-12-27 04:31:51,310][105692] Updated weights for policy 0, policy_version 1803067 (0.0010) [2023-12-27 04:31:51,370][105692] Updated weights for policy 0, policy_version 1803077 (0.0011) [2023-12-27 04:31:51,435][105692] Updated weights for policy 0, policy_version 1803087 (0.0010) [2023-12-27 04:31:51,521][105620] Updated weights for policy 1, policy_version 1806954 (0.0009) [2023-12-27 04:31:51,587][105620] Updated weights for policy 1, policy_version 1806964 (0.0011) [2023-12-27 04:31:51,663][105620] Updated weights for policy 1, policy_version 1806974 (0.0009) [2023-12-27 04:31:51,730][105620] Updated weights for policy 1, policy_version 1806984 (0.0008) [2023-12-27 04:31:52,180][105692] Updated weights for policy 0, policy_version 1803097 (0.0010) [2023-12-27 04:31:52,246][105692] Updated weights for policy 0, policy_version 1803107 (0.0010) [2023-12-27 04:31:52,308][105692] Updated weights for policy 0, policy_version 1803117 (0.0010) [2023-12-27 04:31:52,477][105620] Updated weights for policy 1, policy_version 1806994 (0.0010) [2023-12-27 04:31:52,526][105620] Updated weights for policy 1, policy_version 1807004 (0.0008) [2023-12-27 04:31:52,575][105620] Updated weights for policy 1, policy_version 1807014 (0.0008) [2023-12-27 04:31:52,984][105692] Updated weights for policy 0, policy_version 1803127 (0.0007) [2023-12-27 04:31:53,038][105692] Updated weights for policy 0, policy_version 1803137 (0.0005) [2023-12-27 04:31:53,093][105692] Updated weights for policy 0, policy_version 1803147 (0.0005) [2023-12-27 04:31:53,431][105620] Updated weights for policy 1, policy_version 1807024 (0.0008) [2023-12-27 04:31:53,489][105620] Updated weights for policy 1, policy_version 1807034 (0.0008) [2023-12-27 04:31:53,545][105620] Updated weights for policy 1, policy_version 1807044 (0.0005) [2023-12-27 04:31:53,626][105692] Updated weights for policy 0, policy_version 1803157 (0.0005) [2023-12-27 04:31:53,677][105692] Updated weights for policy 0, policy_version 1803167 (0.0005) [2023-12-27 04:31:53,736][105692] Updated weights for policy 0, policy_version 1803177 (0.0005) [2023-12-27 04:31:54,203][105620] Updated weights for policy 1, policy_version 1807054 (0.0007) [2023-12-27 04:31:54,264][105620] Updated weights for policy 1, policy_version 1807064 (0.0008) [2023-12-27 04:31:54,320][105620] Updated weights for policy 1, policy_version 1807074 (0.0010) [2023-12-27 04:31:54,343][105692] Updated weights for policy 0, policy_version 1803187 (0.0006) [2023-12-27 04:31:54,393][105692] Updated weights for policy 0, policy_version 1803197 (0.0009) [2023-12-27 04:31:54,455][105692] Updated weights for policy 0, policy_version 1803207 (0.0010) [2023-12-27 04:31:55,093][105620] Updated weights for policy 1, policy_version 1807084 (0.0008) [2023-12-27 04:31:55,148][105620] Updated weights for policy 1, policy_version 1807094 (0.0008) [2023-12-27 04:31:55,167][105692] Updated weights for policy 0, policy_version 1803217 (0.0010) [2023-12-27 04:31:55,198][105620] Updated weights for policy 1, policy_version 1807104 (0.0007) [2023-12-27 04:31:55,223][105692] Updated weights for policy 0, policy_version 1803227 (0.0011) [2023-12-27 04:31:55,268][105692] Updated weights for policy 0, policy_version 1803237 (0.0010) [2023-12-27 04:31:55,319][105692] Updated weights for policy 0, policy_version 1803247 (0.0010) [2023-12-27 04:31:55,931][105620] Updated weights for policy 1, policy_version 1807114 (0.0006) [2023-12-27 04:31:55,987][105620] Updated weights for policy 1, policy_version 1807124 (0.0009) [2023-12-27 04:31:56,055][105620] Updated weights for policy 1, policy_version 1807134 (0.0007) [2023-12-27 04:31:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19522.0). Total num frames: 924385280. Throughput: 0: 10064.1, 1: 9219.5. Samples: 924400808. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:31:56,062][104569] Avg episode reward: [(0, '8268.265'), (1, '9258.480')] [2023-12-27 04:31:56,076][105692] Updated weights for policy 0, policy_version 1803257 (0.0011) [2023-12-27 04:31:56,116][105620] Updated weights for policy 1, policy_version 1807144 (0.0008) [2023-12-27 04:31:56,132][105692] Updated weights for policy 0, policy_version 1803267 (0.0011) [2023-12-27 04:31:56,185][105692] Updated weights for policy 0, policy_version 1803277 (0.0011) [2023-12-27 04:31:56,889][105620] Updated weights for policy 1, policy_version 1807154 (0.0010) [2023-12-27 04:31:56,895][105692] Updated weights for policy 0, policy_version 1803287 (0.0007) [2023-12-27 04:31:56,939][105620] Updated weights for policy 1, policy_version 1807164 (0.0008) [2023-12-27 04:31:56,944][105692] Updated weights for policy 0, policy_version 1803297 (0.0005) [2023-12-27 04:31:56,989][105620] Updated weights for policy 1, policy_version 1807174 (0.0008) [2023-12-27 04:31:56,990][105692] Updated weights for policy 0, policy_version 1803307 (0.0006) [2023-12-27 04:31:57,663][105620] Updated weights for policy 1, policy_version 1807184 (0.0008) [2023-12-27 04:31:57,682][105692] Updated weights for policy 0, policy_version 1803317 (0.0010) [2023-12-27 04:31:57,716][105620] Updated weights for policy 1, policy_version 1807194 (0.0007) [2023-12-27 04:31:57,730][105692] Updated weights for policy 0, policy_version 1803327 (0.0010) [2023-12-27 04:31:57,776][105620] Updated weights for policy 1, policy_version 1807204 (0.0007) [2023-12-27 04:31:57,778][105692] Updated weights for policy 0, policy_version 1803337 (0.0008) [2023-12-27 04:31:58,357][105692] Updated weights for policy 0, policy_version 1803347 (0.0007) [2023-12-27 04:31:58,422][105692] Updated weights for policy 0, policy_version 1803357 (0.0010) [2023-12-27 04:31:58,489][105692] Updated weights for policy 0, policy_version 1803367 (0.0011) [2023-12-27 04:31:58,660][105620] Updated weights for policy 1, policy_version 1807214 (0.0008) [2023-12-27 04:31:58,719][105620] Updated weights for policy 1, policy_version 1807224 (0.0008) [2023-12-27 04:31:58,780][105620] Updated weights for policy 1, policy_version 1807234 (0.0009) [2023-12-27 04:31:59,335][105692] Updated weights for policy 0, policy_version 1803377 (0.0010) [2023-12-27 04:31:59,397][105692] Updated weights for policy 0, policy_version 1803387 (0.0010) [2023-12-27 04:31:59,459][105692] Updated weights for policy 0, policy_version 1803397 (0.0010) [2023-12-27 04:31:59,511][105692] Updated weights for policy 0, policy_version 1803407 (0.0010) [2023-12-27 04:31:59,583][105620] Updated weights for policy 1, policy_version 1807244 (0.0008) [2023-12-27 04:31:59,648][105620] Updated weights for policy 1, policy_version 1807254 (0.0008) [2023-12-27 04:31:59,715][105620] Updated weights for policy 1, policy_version 1807264 (0.0009) [2023-12-27 04:32:00,269][105692] Updated weights for policy 0, policy_version 1803417 (0.0006) [2023-12-27 04:32:00,333][105692] Updated weights for policy 0, policy_version 1803427 (0.0007) [2023-12-27 04:32:00,384][105620] Updated weights for policy 1, policy_version 1807274 (0.0009) [2023-12-27 04:32:00,398][105692] Updated weights for policy 0, policy_version 1803437 (0.0007) [2023-12-27 04:32:00,442][105620] Updated weights for policy 1, policy_version 1807284 (0.0007) [2023-12-27 04:32:00,499][105620] Updated weights for policy 1, policy_version 1807294 (0.0008) [2023-12-27 04:32:00,570][105620] Updated weights for policy 1, policy_version 1807304 (0.0010) [2023-12-27 04:32:00,909][105692] Updated weights for policy 0, policy_version 1803447 (0.0006) [2023-12-27 04:32:00,961][105692] Updated weights for policy 0, policy_version 1803457 (0.0009) [2023-12-27 04:32:01,018][105692] Updated weights for policy 0, policy_version 1803467 (0.0010) [2023-12-27 04:32:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 924491776. Throughput: 0: 10104.7, 1: 9208.5. Samples: 924458632. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:32:01,062][104569] Avg episode reward: [(0, '8628.222'), (1, '9166.030')] [2023-12-27 04:32:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001803472_461758464.pth... [2023-12-27 04:32:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001807304_462733312.pth... [2023-12-27 04:32:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001802288_461455360.pth [2023-12-27 04:32:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001806248_462462976.pth [2023-12-27 04:32:01,393][105620] Updated weights for policy 1, policy_version 1807314 (0.0008) [2023-12-27 04:32:01,442][105620] Updated weights for policy 1, policy_version 1807324 (0.0008) [2023-12-27 04:32:01,499][105620] Updated weights for policy 1, policy_version 1807334 (0.0008) [2023-12-27 04:32:01,761][105692] Updated weights for policy 0, policy_version 1803477 (0.0011) [2023-12-27 04:32:01,820][105692] Updated weights for policy 0, policy_version 1803487 (0.0010) [2023-12-27 04:32:01,878][105692] Updated weights for policy 0, policy_version 1803497 (0.0011) [2023-12-27 04:32:02,213][105620] Updated weights for policy 1, policy_version 1807344 (0.0010) [2023-12-27 04:32:02,269][105620] Updated weights for policy 1, policy_version 1807354 (0.0011) [2023-12-27 04:32:02,331][105620] Updated weights for policy 1, policy_version 1807364 (0.0010) [2023-12-27 04:32:02,629][105692] Updated weights for policy 0, policy_version 1803507 (0.0011) [2023-12-27 04:32:02,683][105692] Updated weights for policy 0, policy_version 1803517 (0.0010) [2023-12-27 04:32:02,741][105692] Updated weights for policy 0, policy_version 1803527 (0.0010) [2023-12-27 04:32:03,082][105620] Updated weights for policy 1, policy_version 1807374 (0.0008) [2023-12-27 04:32:03,134][105620] Updated weights for policy 1, policy_version 1807384 (0.0008) [2023-12-27 04:32:03,197][105620] Updated weights for policy 1, policy_version 1807394 (0.0007) [2023-12-27 04:32:03,481][105692] Updated weights for policy 0, policy_version 1803537 (0.0010) [2023-12-27 04:32:03,531][105692] Updated weights for policy 0, policy_version 1803547 (0.0005) [2023-12-27 04:32:03,576][105692] Updated weights for policy 0, policy_version 1803557 (0.0005) [2023-12-27 04:32:03,622][105692] Updated weights for policy 0, policy_version 1803567 (0.0005) [2023-12-27 04:32:03,957][105620] Updated weights for policy 1, policy_version 1807404 (0.0007) [2023-12-27 04:32:04,016][105620] Updated weights for policy 1, policy_version 1807414 (0.0008) [2023-12-27 04:32:04,077][105620] Updated weights for policy 1, policy_version 1807424 (0.0008) [2023-12-27 04:32:04,262][105692] Updated weights for policy 0, policy_version 1803577 (0.0010) [2023-12-27 04:32:04,324][105692] Updated weights for policy 0, policy_version 1803587 (0.0011) [2023-12-27 04:32:04,385][105692] Updated weights for policy 0, policy_version 1803597 (0.0008) [2023-12-27 04:32:04,915][105620] Updated weights for policy 1, policy_version 1807434 (0.0010) [2023-12-27 04:32:04,937][105692] Updated weights for policy 0, policy_version 1803607 (0.0005) [2023-12-27 04:32:04,972][105620] Updated weights for policy 1, policy_version 1807444 (0.0009) [2023-12-27 04:32:04,995][105692] Updated weights for policy 0, policy_version 1803617 (0.0005) [2023-12-27 04:32:05,035][105620] Updated weights for policy 1, policy_version 1807454 (0.0007) [2023-12-27 04:32:05,052][105692] Updated weights for policy 0, policy_version 1803627 (0.0010) [2023-12-27 04:32:05,098][105620] Updated weights for policy 1, policy_version 1807464 (0.0006) [2023-12-27 04:32:05,668][105692] Updated weights for policy 0, policy_version 1803637 (0.0008) [2023-12-27 04:32:05,727][105692] Updated weights for policy 0, policy_version 1803647 (0.0005) [2023-12-27 04:32:05,791][105692] Updated weights for policy 0, policy_version 1803657 (0.0006) [2023-12-27 04:32:05,904][105620] Updated weights for policy 1, policy_version 1807474 (0.0008) [2023-12-27 04:32:05,966][105620] Updated weights for policy 1, policy_version 1807484 (0.0007) [2023-12-27 04:32:06,030][105620] Updated weights for policy 1, policy_version 1807494 (0.0010) [2023-12-27 04:32:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 924590080. Throughput: 0: 10048.4, 1: 9111.8. Samples: 924573392. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:32:06,062][104569] Avg episode reward: [(0, '8623.907'), (1, '9258.357')] [2023-12-27 04:32:06,358][105692] Updated weights for policy 0, policy_version 1803667 (0.0011) [2023-12-27 04:32:06,418][105692] Updated weights for policy 0, policy_version 1803677 (0.0011) [2023-12-27 04:32:06,478][105692] Updated weights for policy 0, policy_version 1803687 (0.0011) [2023-12-27 04:32:06,659][105620] Updated weights for policy 1, policy_version 1807504 (0.0006) [2023-12-27 04:32:06,719][105620] Updated weights for policy 1, policy_version 1807514 (0.0005) [2023-12-27 04:32:06,779][105620] Updated weights for policy 1, policy_version 1807524 (0.0005) [2023-12-27 04:32:07,204][105692] Updated weights for policy 0, policy_version 1803697 (0.0010) [2023-12-27 04:32:07,260][105692] Updated weights for policy 0, policy_version 1803707 (0.0007) [2023-12-27 04:32:07,305][105692] Updated weights for policy 0, policy_version 1803717 (0.0010) [2023-12-27 04:32:07,360][105692] Updated weights for policy 0, policy_version 1803727 (0.0010) [2023-12-27 04:32:07,402][105620] Updated weights for policy 1, policy_version 1807534 (0.0011) [2023-12-27 04:32:07,460][105620] Updated weights for policy 1, policy_version 1807544 (0.0010) [2023-12-27 04:32:07,518][105620] Updated weights for policy 1, policy_version 1807554 (0.0010) [2023-12-27 04:32:08,003][105692] Updated weights for policy 0, policy_version 1803737 (0.0006) [2023-12-27 04:32:08,055][105692] Updated weights for policy 0, policy_version 1803747 (0.0009) [2023-12-27 04:32:08,113][105692] Updated weights for policy 0, policy_version 1803757 (0.0006) [2023-12-27 04:32:08,257][105620] Updated weights for policy 1, policy_version 1807564 (0.0009) [2023-12-27 04:32:08,317][105620] Updated weights for policy 1, policy_version 1807574 (0.0006) [2023-12-27 04:32:08,381][105620] Updated weights for policy 1, policy_version 1807584 (0.0008) [2023-12-27 04:32:08,732][105692] Updated weights for policy 0, policy_version 1803767 (0.0009) [2023-12-27 04:32:08,787][105692] Updated weights for policy 0, policy_version 1803777 (0.0010) [2023-12-27 04:32:08,850][105692] Updated weights for policy 0, policy_version 1803787 (0.0011) [2023-12-27 04:32:09,067][105620] Updated weights for policy 1, policy_version 1807594 (0.0008) [2023-12-27 04:32:09,125][105620] Updated weights for policy 1, policy_version 1807605 (0.0010) [2023-12-27 04:32:09,184][105620] Updated weights for policy 1, policy_version 1807615 (0.0009) [2023-12-27 04:32:09,517][105692] Updated weights for policy 0, policy_version 1803797 (0.0011) [2023-12-27 04:32:09,577][105692] Updated weights for policy 0, policy_version 1803807 (0.0011) [2023-12-27 04:32:09,644][105692] Updated weights for policy 0, policy_version 1803817 (0.0010) [2023-12-27 04:32:10,015][105620] Updated weights for policy 1, policy_version 1807625 (0.0009) [2023-12-27 04:32:10,078][105620] Updated weights for policy 1, policy_version 1807635 (0.0008) [2023-12-27 04:32:10,138][105620] Updated weights for policy 1, policy_version 1807645 (0.0008) [2023-12-27 04:32:10,197][105620] Updated weights for policy 1, policy_version 1807655 (0.0010) [2023-12-27 04:32:10,331][105692] Updated weights for policy 0, policy_version 1803827 (0.0008) [2023-12-27 04:32:10,401][105692] Updated weights for policy 0, policy_version 1803837 (0.0009) [2023-12-27 04:32:10,466][105692] Updated weights for policy 0, policy_version 1803847 (0.0006) [2023-12-27 04:32:10,866][105620] Updated weights for policy 1, policy_version 1807665 (0.0008) [2023-12-27 04:32:10,924][105620] Updated weights for policy 1, policy_version 1807675 (0.0009) [2023-12-27 04:32:10,990][105620] Updated weights for policy 1, policy_version 1807685 (0.0009) [2023-12-27 04:32:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 924688384. Throughput: 0: 10226.0, 1: 9194.4. Samples: 924695300. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:32:11,063][104569] Avg episode reward: [(0, '8443.964'), (1, '9258.369')] [2023-12-27 04:32:11,229][105692] Updated weights for policy 0, policy_version 1803857 (0.0008) [2023-12-27 04:32:11,289][105692] Updated weights for policy 0, policy_version 1803867 (0.0008) [2023-12-27 04:32:11,352][105692] Updated weights for policy 0, policy_version 1803877 (0.0008) [2023-12-27 04:32:11,408][105692] Updated weights for policy 0, policy_version 1803887 (0.0008) [2023-12-27 04:32:11,799][105620] Updated weights for policy 1, policy_version 1807695 (0.0009) [2023-12-27 04:32:11,852][105620] Updated weights for policy 1, policy_version 1807705 (0.0006) [2023-12-27 04:32:11,904][105620] Updated weights for policy 1, policy_version 1807715 (0.0007) [2023-12-27 04:32:12,189][105692] Updated weights for policy 0, policy_version 1803897 (0.0010) [2023-12-27 04:32:12,256][105692] Updated weights for policy 0, policy_version 1803907 (0.0010) [2023-12-27 04:32:12,322][105692] Updated weights for policy 0, policy_version 1803917 (0.0009) [2023-12-27 04:32:12,590][105620] Updated weights for policy 1, policy_version 1807725 (0.0005) [2023-12-27 04:32:12,649][105620] Updated weights for policy 1, policy_version 1807735 (0.0005) [2023-12-27 04:32:12,708][105620] Updated weights for policy 1, policy_version 1807745 (0.0005) [2023-12-27 04:32:13,087][105692] Updated weights for policy 0, policy_version 1803927 (0.0006) [2023-12-27 04:32:13,148][105692] Updated weights for policy 0, policy_version 1803937 (0.0006) [2023-12-27 04:32:13,205][105692] Updated weights for policy 0, policy_version 1803947 (0.0005) [2023-12-27 04:32:13,341][105620] Updated weights for policy 1, policy_version 1807755 (0.0007) [2023-12-27 04:32:13,395][105620] Updated weights for policy 1, policy_version 1807765 (0.0009) [2023-12-27 04:32:13,453][105620] Updated weights for policy 1, policy_version 1807775 (0.0008) [2023-12-27 04:32:13,787][105692] Updated weights for policy 0, policy_version 1803957 (0.0005) [2023-12-27 04:32:13,843][105692] Updated weights for policy 0, policy_version 1803967 (0.0008) [2023-12-27 04:32:13,901][105692] Updated weights for policy 0, policy_version 1803977 (0.0010) [2023-12-27 04:32:14,253][105620] Updated weights for policy 1, policy_version 1807785 (0.0008) [2023-12-27 04:32:14,307][105620] Updated weights for policy 1, policy_version 1807795 (0.0008) [2023-12-27 04:32:14,366][105620] Updated weights for policy 1, policy_version 1807805 (0.0008) [2023-12-27 04:32:14,428][105620] Updated weights for policy 1, policy_version 1807815 (0.0008) [2023-12-27 04:32:14,629][105692] Updated weights for policy 0, policy_version 1803987 (0.0010) [2023-12-27 04:32:14,690][105692] Updated weights for policy 0, policy_version 1803997 (0.0010) [2023-12-27 04:32:14,755][105692] Updated weights for policy 0, policy_version 1804007 (0.0010) [2023-12-27 04:32:15,193][105620] Updated weights for policy 1, policy_version 1807825 (0.0009) [2023-12-27 04:32:15,254][105620] Updated weights for policy 1, policy_version 1807835 (0.0010) [2023-12-27 04:32:15,317][105620] Updated weights for policy 1, policy_version 1807845 (0.0010) [2023-12-27 04:32:15,320][105692] Updated weights for policy 0, policy_version 1804017 (0.0010) [2023-12-27 04:32:15,384][105692] Updated weights for policy 0, policy_version 1804027 (0.0008) [2023-12-27 04:32:15,447][105692] Updated weights for policy 0, policy_version 1804037 (0.0008) [2023-12-27 04:32:15,507][105692] Updated weights for policy 0, policy_version 1804047 (0.0011) [2023-12-27 04:32:16,060][105692] Updated weights for policy 0, policy_version 1804057 (0.0008) [2023-12-27 04:32:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 924778496. Throughput: 0: 10126.9, 1: 9250.9. Samples: 924752476. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:32:16,062][104569] Avg episode reward: [(0, '8896.948'), (1, '9258.438')] [2023-12-27 04:32:16,075][105620] Updated weights for policy 1, policy_version 1807855 (0.0005) [2023-12-27 04:32:16,113][105692] Updated weights for policy 0, policy_version 1804067 (0.0008) [2023-12-27 04:32:16,132][105620] Updated weights for policy 1, policy_version 1807865 (0.0006) [2023-12-27 04:32:16,169][105692] Updated weights for policy 0, policy_version 1804077 (0.0006) [2023-12-27 04:32:16,185][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001804080_461914112.pth... [2023-12-27 04:32:16,189][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001802864_461602816.pth [2023-12-27 04:32:16,191][105620] Updated weights for policy 1, policy_version 1807875 (0.0008) [2023-12-27 04:32:16,220][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001807880_462880768.pth... [2023-12-27 04:32:16,224][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001806792_462602240.pth [2023-12-27 04:32:16,857][105692] Updated weights for policy 0, policy_version 1804087 (0.0005) [2023-12-27 04:32:16,904][105692] Updated weights for policy 0, policy_version 1804097 (0.0005) [2023-12-27 04:32:16,932][105620] Updated weights for policy 1, policy_version 1807885 (0.0010) [2023-12-27 04:32:16,960][105692] Updated weights for policy 0, policy_version 1804107 (0.0006) [2023-12-27 04:32:16,984][105620] Updated weights for policy 1, policy_version 1807895 (0.0010) [2023-12-27 04:32:17,043][105620] Updated weights for policy 1, policy_version 1807905 (0.0010) [2023-12-27 04:32:17,546][105692] Updated weights for policy 0, policy_version 1804117 (0.0007) [2023-12-27 04:32:17,606][105692] Updated weights for policy 0, policy_version 1804127 (0.0010) [2023-12-27 04:32:17,624][105620] Updated weights for policy 1, policy_version 1807915 (0.0010) [2023-12-27 04:32:17,672][105692] Updated weights for policy 0, policy_version 1804137 (0.0011) [2023-12-27 04:32:17,679][105620] Updated weights for policy 1, policy_version 1807925 (0.0010) [2023-12-27 04:32:17,733][105620] Updated weights for policy 1, policy_version 1807935 (0.0010) [2023-12-27 04:32:18,400][105620] Updated weights for policy 1, policy_version 1807945 (0.0010) [2023-12-27 04:32:18,424][105692] Updated weights for policy 0, policy_version 1804147 (0.0010) [2023-12-27 04:32:18,458][105620] Updated weights for policy 1, policy_version 1807955 (0.0006) [2023-12-27 04:32:18,492][105692] Updated weights for policy 0, policy_version 1804157 (0.0008) [2023-12-27 04:32:18,514][105620] Updated weights for policy 1, policy_version 1807965 (0.0006) [2023-12-27 04:32:18,558][105692] Updated weights for policy 0, policy_version 1804167 (0.0009) [2023-12-27 04:32:18,574][105620] Updated weights for policy 1, policy_version 1807975 (0.0008) [2023-12-27 04:32:19,259][105692] Updated weights for policy 0, policy_version 1804177 (0.0008) [2023-12-27 04:32:19,309][105620] Updated weights for policy 1, policy_version 1807985 (0.0006) [2023-12-27 04:32:19,319][105692] Updated weights for policy 0, policy_version 1804187 (0.0011) [2023-12-27 04:32:19,373][105620] Updated weights for policy 1, policy_version 1807995 (0.0009) [2023-12-27 04:32:19,382][105692] Updated weights for policy 0, policy_version 1804197 (0.0010) [2023-12-27 04:32:19,429][105620] Updated weights for policy 1, policy_version 1808005 (0.0005) [2023-12-27 04:32:19,435][105692] Updated weights for policy 0, policy_version 1804207 (0.0011) [2023-12-27 04:32:20,135][105692] Updated weights for policy 0, policy_version 1804217 (0.0006) [2023-12-27 04:32:20,191][105692] Updated weights for policy 0, policy_version 1804227 (0.0006) [2023-12-27 04:32:20,191][105620] Updated weights for policy 1, policy_version 1808015 (0.0009) [2023-12-27 04:32:20,252][105692] Updated weights for policy 0, policy_version 1804237 (0.0006) [2023-12-27 04:32:20,256][105620] Updated weights for policy 1, policy_version 1808025 (0.0009) [2023-12-27 04:32:20,326][105620] Updated weights for policy 1, policy_version 1808035 (0.0011) [2023-12-27 04:32:20,876][105692] Updated weights for policy 0, policy_version 1804247 (0.0009) [2023-12-27 04:32:20,940][105692] Updated weights for policy 0, policy_version 1804257 (0.0011) [2023-12-27 04:32:21,000][105692] Updated weights for policy 0, policy_version 1804267 (0.0011) [2023-12-27 04:32:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 924884992. Throughput: 0: 10195.5, 1: 9357.2. Samples: 924873360. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:32:21,062][104569] Avg episode reward: [(0, '8806.266'), (1, '9350.930')] [2023-12-27 04:32:21,079][105620] Updated weights for policy 1, policy_version 1808045 (0.0012) [2023-12-27 04:32:21,137][105620] Updated weights for policy 1, policy_version 1808055 (0.0008) [2023-12-27 04:32:21,204][105620] Updated weights for policy 1, policy_version 1808065 (0.0008) [2023-12-27 04:32:21,812][105692] Updated weights for policy 0, policy_version 1804277 (0.0008) [2023-12-27 04:32:21,868][105692] Updated weights for policy 0, policy_version 1804287 (0.0010) [2023-12-27 04:32:21,921][105620] Updated weights for policy 1, policy_version 1808075 (0.0009) [2023-12-27 04:32:21,923][105692] Updated weights for policy 0, policy_version 1804297 (0.0010) [2023-12-27 04:32:21,985][105620] Updated weights for policy 1, policy_version 1808085 (0.0007) [2023-12-27 04:32:22,054][105620] Updated weights for policy 1, policy_version 1808095 (0.0006) [2023-12-27 04:32:22,676][105620] Updated weights for policy 1, policy_version 1808105 (0.0008) [2023-12-27 04:32:22,738][105620] Updated weights for policy 1, policy_version 1808115 (0.0010) [2023-12-27 04:32:22,768][105692] Updated weights for policy 0, policy_version 1804307 (0.0008) [2023-12-27 04:32:22,798][105620] Updated weights for policy 1, policy_version 1808125 (0.0006) [2023-12-27 04:32:22,826][105692] Updated weights for policy 0, policy_version 1804317 (0.0009) [2023-12-27 04:32:22,866][105620] Updated weights for policy 1, policy_version 1808135 (0.0009) [2023-12-27 04:32:22,890][105692] Updated weights for policy 0, policy_version 1804327 (0.0007) [2023-12-27 04:32:23,512][105620] Updated weights for policy 1, policy_version 1808145 (0.0010) [2023-12-27 04:32:23,560][105620] Updated weights for policy 1, policy_version 1808155 (0.0010) [2023-12-27 04:32:23,612][105620] Updated weights for policy 1, policy_version 1808165 (0.0010) [2023-12-27 04:32:23,614][105692] Updated weights for policy 0, policy_version 1804337 (0.0009) [2023-12-27 04:32:23,673][105692] Updated weights for policy 0, policy_version 1804347 (0.0007) [2023-12-27 04:32:23,736][105692] Updated weights for policy 0, policy_version 1804357 (0.0008) [2023-12-27 04:32:23,789][105692] Updated weights for policy 0, policy_version 1804367 (0.0008) [2023-12-27 04:32:24,377][105620] Updated weights for policy 1, policy_version 1808175 (0.0009) [2023-12-27 04:32:24,406][105692] Updated weights for policy 0, policy_version 1804377 (0.0007) [2023-12-27 04:32:24,437][105620] Updated weights for policy 1, policy_version 1808185 (0.0009) [2023-12-27 04:32:24,468][105692] Updated weights for policy 0, policy_version 1804387 (0.0006) [2023-12-27 04:32:24,490][105620] Updated weights for policy 1, policy_version 1808195 (0.0009) [2023-12-27 04:32:24,529][105692] Updated weights for policy 0, policy_version 1804397 (0.0007) [2023-12-27 04:32:25,164][105692] Updated weights for policy 0, policy_version 1804407 (0.0009) [2023-12-27 04:32:25,217][105692] Updated weights for policy 0, policy_version 1804417 (0.0008) [2023-12-27 04:32:25,219][105620] Updated weights for policy 1, policy_version 1808205 (0.0007) [2023-12-27 04:32:25,276][105692] Updated weights for policy 0, policy_version 1804427 (0.0006) [2023-12-27 04:32:25,278][105620] Updated weights for policy 1, policy_version 1808215 (0.0008) [2023-12-27 04:32:25,328][105620] Updated weights for policy 1, policy_version 1808225 (0.0008) [2023-12-27 04:32:25,878][105620] Updated weights for policy 1, policy_version 1808235 (0.0008) [2023-12-27 04:32:25,927][105620] Updated weights for policy 1, policy_version 1808245 (0.0005) [2023-12-27 04:32:25,986][105620] Updated weights for policy 1, policy_version 1808255 (0.0005) [2023-12-27 04:32:26,011][105692] Updated weights for policy 0, policy_version 1804437 (0.0007) [2023-12-27 04:32:26,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 924983296. Throughput: 0: 10228.8, 1: 9407.1. Samples: 924992028. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:32:26,062][105692] Updated weights for policy 0, policy_version 1804447 (0.0009) [2023-12-27 04:32:26,063][104569] Avg episode reward: [(0, '8535.986'), (1, '9258.409')] [2023-12-27 04:32:26,115][105692] Updated weights for policy 0, policy_version 1804457 (0.0009) [2023-12-27 04:32:26,596][105620] Updated weights for policy 1, policy_version 1808265 (0.0006) [2023-12-27 04:32:26,649][105620] Updated weights for policy 1, policy_version 1808275 (0.0008) [2023-12-27 04:32:26,697][105620] Updated weights for policy 1, policy_version 1808285 (0.0008) [2023-12-27 04:32:26,756][105620] Updated weights for policy 1, policy_version 1808295 (0.0009) [2023-12-27 04:32:26,896][105692] Updated weights for policy 0, policy_version 1804467 (0.0009) [2023-12-27 04:32:26,955][105692] Updated weights for policy 0, policy_version 1804477 (0.0011) [2023-12-27 04:32:27,007][105692] Updated weights for policy 0, policy_version 1804487 (0.0010) [2023-12-27 04:32:27,478][105620] Updated weights for policy 1, policy_version 1808305 (0.0008) [2023-12-27 04:32:27,543][105620] Updated weights for policy 1, policy_version 1808315 (0.0009) [2023-12-27 04:32:27,594][105620] Updated weights for policy 1, policy_version 1808325 (0.0008) [2023-12-27 04:32:27,738][105692] Updated weights for policy 0, policy_version 1804497 (0.0010) [2023-12-27 04:32:27,803][105692] Updated weights for policy 0, policy_version 1804507 (0.0005) [2023-12-27 04:32:27,859][105692] Updated weights for policy 0, policy_version 1804517 (0.0005) [2023-12-27 04:32:27,911][105692] Updated weights for policy 0, policy_version 1804527 (0.0005) [2023-12-27 04:32:28,169][105620] Updated weights for policy 1, policy_version 1808335 (0.0006) [2023-12-27 04:32:28,215][105620] Updated weights for policy 1, policy_version 1808345 (0.0005) [2023-12-27 04:32:28,267][105620] Updated weights for policy 1, policy_version 1808355 (0.0005) [2023-12-27 04:32:28,571][105692] Updated weights for policy 0, policy_version 1804537 (0.0009) [2023-12-27 04:32:28,623][105692] Updated weights for policy 0, policy_version 1804547 (0.0010) [2023-12-27 04:32:28,678][105692] Updated weights for policy 0, policy_version 1804557 (0.0010) [2023-12-27 04:32:29,001][105620] Updated weights for policy 1, policy_version 1808365 (0.0005) [2023-12-27 04:32:29,064][105620] Updated weights for policy 1, policy_version 1808375 (0.0005) [2023-12-27 04:32:29,124][105620] Updated weights for policy 1, policy_version 1808385 (0.0005) [2023-12-27 04:32:29,392][105692] Updated weights for policy 0, policy_version 1804567 (0.0010) [2023-12-27 04:32:29,459][105692] Updated weights for policy 0, policy_version 1804577 (0.0010) [2023-12-27 04:32:29,521][105692] Updated weights for policy 0, policy_version 1804587 (0.0010) [2023-12-27 04:32:29,672][105620] Updated weights for policy 1, policy_version 1808395 (0.0006) [2023-12-27 04:32:29,726][105620] Updated weights for policy 1, policy_version 1808405 (0.0009) [2023-12-27 04:32:29,788][105620] Updated weights for policy 1, policy_version 1808415 (0.0009) [2023-12-27 04:32:30,331][105692] Updated weights for policy 0, policy_version 1804597 (0.0008) [2023-12-27 04:32:30,390][105692] Updated weights for policy 0, policy_version 1804607 (0.0006) [2023-12-27 04:32:30,451][105692] Updated weights for policy 0, policy_version 1804617 (0.0005) [2023-12-27 04:32:30,484][105620] Updated weights for policy 1, policy_version 1808425 (0.0008) [2023-12-27 04:32:30,542][105620] Updated weights for policy 1, policy_version 1808435 (0.0007) [2023-12-27 04:32:30,587][105620] Updated weights for policy 1, policy_version 1808445 (0.0007) [2023-12-27 04:32:30,654][105620] Updated weights for policy 1, policy_version 1808455 (0.0005) [2023-12-27 04:32:31,020][105692] Updated weights for policy 0, policy_version 1804627 (0.0006) [2023-12-27 04:32:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 925081600. Throughput: 0: 10231.8, 1: 9483.7. Samples: 925053008. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:32:31,062][104569] Avg episode reward: [(0, '8809.693'), (1, '9165.928')] [2023-12-27 04:32:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001808456_463028224.pth... [2023-12-27 04:32:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001807304_462733312.pth [2023-12-27 04:32:31,083][105692] Updated weights for policy 0, policy_version 1804637 (0.0007) [2023-12-27 04:32:31,139][105692] Updated weights for policy 0, policy_version 1804647 (0.0007) [2023-12-27 04:32:31,197][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001804656_462061568.pth... [2023-12-27 04:32:31,201][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001803472_461758464.pth [2023-12-27 04:32:31,277][105620] Updated weights for policy 1, policy_version 1808465 (0.0008) [2023-12-27 04:32:31,330][105620] Updated weights for policy 1, policy_version 1808475 (0.0008) [2023-12-27 04:32:31,399][105620] Updated weights for policy 1, policy_version 1808485 (0.0007) [2023-12-27 04:32:31,871][105692] Updated weights for policy 0, policy_version 1804657 (0.0009) [2023-12-27 04:32:31,938][105692] Updated weights for policy 0, policy_version 1804667 (0.0005) [2023-12-27 04:32:32,009][105692] Updated weights for policy 0, policy_version 1804677 (0.0007) [2023-12-27 04:32:32,024][105620] Updated weights for policy 1, policy_version 1808495 (0.0007) [2023-12-27 04:32:32,076][105692] Updated weights for policy 0, policy_version 1804687 (0.0007) [2023-12-27 04:32:32,076][105620] Updated weights for policy 1, policy_version 1808505 (0.0007) [2023-12-27 04:32:32,127][105620] Updated weights for policy 1, policy_version 1808515 (0.0005) [2023-12-27 04:32:32,748][105692] Updated weights for policy 0, policy_version 1804697 (0.0008) [2023-12-27 04:32:32,806][105692] Updated weights for policy 0, policy_version 1804707 (0.0008) [2023-12-27 04:32:32,837][105620] Updated weights for policy 1, policy_version 1808525 (0.0008) [2023-12-27 04:32:32,857][105692] Updated weights for policy 0, policy_version 1804717 (0.0006) [2023-12-27 04:32:32,889][105620] Updated weights for policy 1, policy_version 1808535 (0.0011) [2023-12-27 04:32:32,942][105620] Updated weights for policy 1, policy_version 1808545 (0.0011) [2023-12-27 04:32:33,442][105692] Updated weights for policy 0, policy_version 1804727 (0.0007) [2023-12-27 04:32:33,492][105692] Updated weights for policy 0, policy_version 1804737 (0.0008) [2023-12-27 04:32:33,536][105692] Updated weights for policy 0, policy_version 1804747 (0.0008) [2023-12-27 04:32:33,754][105620] Updated weights for policy 1, policy_version 1808555 (0.0011) [2023-12-27 04:32:33,805][105620] Updated weights for policy 1, policy_version 1808565 (0.0010) [2023-12-27 04:32:33,849][105620] Updated weights for policy 1, policy_version 1808575 (0.0010) [2023-12-27 04:32:34,278][105692] Updated weights for policy 0, policy_version 1804757 (0.0007) [2023-12-27 04:32:34,347][105692] Updated weights for policy 0, policy_version 1804767 (0.0005) [2023-12-27 04:32:34,411][105692] Updated weights for policy 0, policy_version 1804777 (0.0006) [2023-12-27 04:32:34,638][105620] Updated weights for policy 1, policy_version 1808585 (0.0010) [2023-12-27 04:32:34,707][105620] Updated weights for policy 1, policy_version 1808595 (0.0008) [2023-12-27 04:32:34,765][105620] Updated weights for policy 1, policy_version 1808605 (0.0008) [2023-12-27 04:32:34,820][105620] Updated weights for policy 1, policy_version 1808616 (0.0009) [2023-12-27 04:32:34,989][105692] Updated weights for policy 0, policy_version 1804787 (0.0008) [2023-12-27 04:32:35,048][105692] Updated weights for policy 0, policy_version 1804797 (0.0008) [2023-12-27 04:32:35,103][105692] Updated weights for policy 0, policy_version 1804807 (0.0007) [2023-12-27 04:32:35,507][105620] Updated weights for policy 1, policy_version 1808626 (0.0005) [2023-12-27 04:32:35,567][105620] Updated weights for policy 1, policy_version 1808636 (0.0005) [2023-12-27 04:32:35,632][105620] Updated weights for policy 1, policy_version 1808646 (0.0006) [2023-12-27 04:32:35,972][105692] Updated weights for policy 0, policy_version 1804817 (0.0007) [2023-12-27 04:32:36,036][105692] Updated weights for policy 0, policy_version 1804827 (0.0009) [2023-12-27 04:32:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 925179904. Throughput: 0: 10112.6, 1: 9645.4. Samples: 925173792. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:32:36,062][104569] Avg episode reward: [(0, '9263.703'), (1, '9166.278')] [2023-12-27 04:32:36,096][105692] Updated weights for policy 0, policy_version 1804837 (0.0009) [2023-12-27 04:32:36,158][105620] Updated weights for policy 1, policy_version 1808656 (0.0006) [2023-12-27 04:32:36,159][105692] Updated weights for policy 0, policy_version 1804847 (0.0008) [2023-12-27 04:32:36,208][105620] Updated weights for policy 1, policy_version 1808666 (0.0007) [2023-12-27 04:32:36,258][105620] Updated weights for policy 1, policy_version 1808676 (0.0007) [2023-12-27 04:32:36,858][105692] Updated weights for policy 0, policy_version 1804857 (0.0005) [2023-12-27 04:32:36,908][105692] Updated weights for policy 0, policy_version 1804867 (0.0005) [2023-12-27 04:32:36,966][105692] Updated weights for policy 0, policy_version 1804877 (0.0009) [2023-12-27 04:32:37,008][105620] Updated weights for policy 1, policy_version 1808686 (0.0007) [2023-12-27 04:32:37,068][105620] Updated weights for policy 1, policy_version 1808696 (0.0008) [2023-12-27 04:32:37,128][105620] Updated weights for policy 1, policy_version 1808706 (0.0006) [2023-12-27 04:32:37,654][105692] Updated weights for policy 0, policy_version 1804887 (0.0007) [2023-12-27 04:32:37,705][105692] Updated weights for policy 0, policy_version 1804897 (0.0009) [2023-12-27 04:32:37,761][105692] Updated weights for policy 0, policy_version 1804907 (0.0010) [2023-12-27 04:32:37,822][105620] Updated weights for policy 1, policy_version 1808716 (0.0007) [2023-12-27 04:32:37,887][105620] Updated weights for policy 1, policy_version 1808726 (0.0007) [2023-12-27 04:32:37,946][105620] Updated weights for policy 1, policy_version 1808736 (0.0005) [2023-12-27 04:32:38,529][105692] Updated weights for policy 0, policy_version 1804918 (0.0009) [2023-12-27 04:32:38,586][105692] Updated weights for policy 0, policy_version 1804928 (0.0008) [2023-12-27 04:32:38,643][105692] Updated weights for policy 0, policy_version 1804938 (0.0008) [2023-12-27 04:32:38,652][105620] Updated weights for policy 1, policy_version 1808746 (0.0007) [2023-12-27 04:32:38,714][105620] Updated weights for policy 1, policy_version 1808756 (0.0011) [2023-12-27 04:32:38,773][105620] Updated weights for policy 1, policy_version 1808766 (0.0011) [2023-12-27 04:32:38,838][105620] Updated weights for policy 1, policy_version 1808776 (0.0010) [2023-12-27 04:32:39,404][105692] Updated weights for policy 0, policy_version 1804948 (0.0009) [2023-12-27 04:32:39,469][105692] Updated weights for policy 0, policy_version 1804958 (0.0008) [2023-12-27 04:32:39,534][105692] Updated weights for policy 0, policy_version 1804968 (0.0008) [2023-12-27 04:32:39,594][105620] Updated weights for policy 1, policy_version 1808786 (0.0011) [2023-12-27 04:32:39,655][105620] Updated weights for policy 1, policy_version 1808796 (0.0011) [2023-12-27 04:32:39,719][105620] Updated weights for policy 1, policy_version 1808806 (0.0011) [2023-12-27 04:32:40,223][105692] Updated weights for policy 0, policy_version 1804978 (0.0008) [2023-12-27 04:32:40,283][105692] Updated weights for policy 0, policy_version 1804988 (0.0008) [2023-12-27 04:32:40,347][105692] Updated weights for policy 0, policy_version 1804998 (0.0006) [2023-12-27 04:32:40,410][105692] Updated weights for policy 0, policy_version 1805008 (0.0006) [2023-12-27 04:32:40,533][105620] Updated weights for policy 1, policy_version 1808816 (0.0009) [2023-12-27 04:32:40,595][105620] Updated weights for policy 1, policy_version 1808826 (0.0010) [2023-12-27 04:32:40,648][105620] Updated weights for policy 1, policy_version 1808836 (0.0010) [2023-12-27 04:32:40,961][105692] Updated weights for policy 0, policy_version 1805018 (0.0005) [2023-12-27 04:32:41,027][105692] Updated weights for policy 0, policy_version 1805028 (0.0007) [2023-12-27 04:32:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 925278208. Throughput: 0: 10033.6, 1: 9719.7. Samples: 925289708. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:32:41,063][104569] Avg episode reward: [(0, '8633.933'), (1, '9166.216')] [2023-12-27 04:32:41,092][105692] Updated weights for policy 0, policy_version 1805038 (0.0009) [2023-12-27 04:32:41,534][105620] Updated weights for policy 1, policy_version 1808846 (0.0009) [2023-12-27 04:32:41,590][105620] Updated weights for policy 1, policy_version 1808856 (0.0008) [2023-12-27 04:32:41,655][105620] Updated weights for policy 1, policy_version 1808866 (0.0006) [2023-12-27 04:32:41,862][105692] Updated weights for policy 0, policy_version 1805048 (0.0010) [2023-12-27 04:32:41,913][105692] Updated weights for policy 0, policy_version 1805058 (0.0009) [2023-12-27 04:32:41,963][105692] Updated weights for policy 0, policy_version 1805068 (0.0011) [2023-12-27 04:32:42,397][105620] Updated weights for policy 1, policy_version 1808876 (0.0007) [2023-12-27 04:32:42,458][105620] Updated weights for policy 1, policy_version 1808886 (0.0006) [2023-12-27 04:32:42,527][105620] Updated weights for policy 1, policy_version 1808896 (0.0005) [2023-12-27 04:32:42,732][105692] Updated weights for policy 0, policy_version 1805078 (0.0007) [2023-12-27 04:32:42,792][105692] Updated weights for policy 0, policy_version 1805088 (0.0005) [2023-12-27 04:32:42,853][105692] Updated weights for policy 0, policy_version 1805098 (0.0005) [2023-12-27 04:32:43,219][105620] Updated weights for policy 1, policy_version 1808906 (0.0007) [2023-12-27 04:32:43,274][105620] Updated weights for policy 1, policy_version 1808916 (0.0010) [2023-12-27 04:32:43,337][105620] Updated weights for policy 1, policy_version 1808926 (0.0011) [2023-12-27 04:32:43,396][105620] Updated weights for policy 1, policy_version 1808936 (0.0010) [2023-12-27 04:32:43,411][105692] Updated weights for policy 0, policy_version 1805108 (0.0006) [2023-12-27 04:32:43,473][105692] Updated weights for policy 0, policy_version 1805118 (0.0008) [2023-12-27 04:32:43,533][105692] Updated weights for policy 0, policy_version 1805128 (0.0007) [2023-12-27 04:32:44,114][105692] Updated weights for policy 0, policy_version 1805138 (0.0006) [2023-12-27 04:32:44,143][105620] Updated weights for policy 1, policy_version 1808946 (0.0010) [2023-12-27 04:32:44,173][105692] Updated weights for policy 0, policy_version 1805148 (0.0006) [2023-12-27 04:32:44,198][105620] Updated weights for policy 1, policy_version 1808956 (0.0010) [2023-12-27 04:32:44,230][105692] Updated weights for policy 0, policy_version 1805158 (0.0005) [2023-12-27 04:32:44,263][105620] Updated weights for policy 1, policy_version 1808966 (0.0010) [2023-12-27 04:32:44,285][105692] Updated weights for policy 0, policy_version 1805168 (0.0006) [2023-12-27 04:32:44,935][105620] Updated weights for policy 1, policy_version 1808976 (0.0011) [2023-12-27 04:32:44,995][105620] Updated weights for policy 1, policy_version 1808986 (0.0011) [2023-12-27 04:32:45,051][105620] Updated weights for policy 1, policy_version 1808996 (0.0011) [2023-12-27 04:32:45,059][105692] Updated weights for policy 0, policy_version 1805178 (0.0011) [2023-12-27 04:32:45,120][105692] Updated weights for policy 0, policy_version 1805188 (0.0011) [2023-12-27 04:32:45,183][105692] Updated weights for policy 0, policy_version 1805198 (0.0011) [2023-12-27 04:32:45,790][105692] Updated weights for policy 0, policy_version 1805208 (0.0006) [2023-12-27 04:32:45,819][105620] Updated weights for policy 1, policy_version 1809006 (0.0011) [2023-12-27 04:32:45,847][105692] Updated weights for policy 0, policy_version 1805218 (0.0009) [2023-12-27 04:32:45,873][105620] Updated weights for policy 1, policy_version 1809016 (0.0009) [2023-12-27 04:32:45,898][105692] Updated weights for policy 0, policy_version 1805228 (0.0010) [2023-12-27 04:32:45,923][105620] Updated weights for policy 1, policy_version 1809026 (0.0009) [2023-12-27 04:32:46,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 925384704. Throughput: 0: 10026.4, 1: 9743.8. Samples: 925348292. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:32:46,062][104569] Avg episode reward: [(0, '8269.725'), (1, '9350.843')] [2023-12-27 04:32:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001809032_463175680.pth... [2023-12-27 04:32:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001805232_462209024.pth... [2023-12-27 04:32:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001807880_462880768.pth [2023-12-27 04:32:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001804080_461914112.pth [2023-12-27 04:32:46,558][105692] Updated weights for policy 0, policy_version 1805238 (0.0007) [2023-12-27 04:32:46,601][105692] Updated weights for policy 0, policy_version 1805248 (0.0006) [2023-12-27 04:32:46,613][105620] Updated weights for policy 1, policy_version 1809036 (0.0008) [2023-12-27 04:32:46,656][105692] Updated weights for policy 0, policy_version 1805258 (0.0005) [2023-12-27 04:32:46,676][105620] Updated weights for policy 1, policy_version 1809046 (0.0005) [2023-12-27 04:32:46,736][105620] Updated weights for policy 1, policy_version 1809056 (0.0007) [2023-12-27 04:32:47,328][105692] Updated weights for policy 0, policy_version 1805268 (0.0006) [2023-12-27 04:32:47,397][105692] Updated weights for policy 0, policy_version 1805278 (0.0008) [2023-12-27 04:32:47,415][105620] Updated weights for policy 1, policy_version 1809066 (0.0010) [2023-12-27 04:32:47,446][105692] Updated weights for policy 0, policy_version 1805288 (0.0007) [2023-12-27 04:32:47,473][105620] Updated weights for policy 1, policy_version 1809076 (0.0010) [2023-12-27 04:32:47,531][105620] Updated weights for policy 1, policy_version 1809086 (0.0010) [2023-12-27 04:32:47,586][105620] Updated weights for policy 1, policy_version 1809096 (0.0010) [2023-12-27 04:32:48,101][105692] Updated weights for policy 0, policy_version 1805298 (0.0008) [2023-12-27 04:32:48,157][105692] Updated weights for policy 0, policy_version 1805308 (0.0005) [2023-12-27 04:32:48,215][105692] Updated weights for policy 0, policy_version 1805318 (0.0005) [2023-12-27 04:32:48,273][105692] Updated weights for policy 0, policy_version 1805328 (0.0006) [2023-12-27 04:32:48,287][105620] Updated weights for policy 1, policy_version 1809106 (0.0010) [2023-12-27 04:32:48,352][105620] Updated weights for policy 1, policy_version 1809116 (0.0010) [2023-12-27 04:32:48,414][105620] Updated weights for policy 1, policy_version 1809126 (0.0010) [2023-12-27 04:32:48,892][105692] Updated weights for policy 0, policy_version 1805338 (0.0007) [2023-12-27 04:32:48,957][105692] Updated weights for policy 0, policy_version 1805348 (0.0006) [2023-12-27 04:32:49,017][105692] Updated weights for policy 0, policy_version 1805358 (0.0005) [2023-12-27 04:32:49,164][105620] Updated weights for policy 1, policy_version 1809136 (0.0010) [2023-12-27 04:32:49,229][105620] Updated weights for policy 1, policy_version 1809146 (0.0011) [2023-12-27 04:32:49,290][105620] Updated weights for policy 1, policy_version 1809156 (0.0011) [2023-12-27 04:32:49,667][105692] Updated weights for policy 0, policy_version 1805368 (0.0010) [2023-12-27 04:32:49,716][105692] Updated weights for policy 0, policy_version 1805378 (0.0011) [2023-12-27 04:32:49,766][105692] Updated weights for policy 0, policy_version 1805388 (0.0008) [2023-12-27 04:32:50,000][105620] Updated weights for policy 1, policy_version 1809166 (0.0009) [2023-12-27 04:32:50,070][105620] Updated weights for policy 1, policy_version 1809176 (0.0007) [2023-12-27 04:32:50,138][105620] Updated weights for policy 1, policy_version 1809186 (0.0011) [2023-12-27 04:32:50,457][105692] Updated weights for policy 0, policy_version 1805398 (0.0009) [2023-12-27 04:32:50,516][105692] Updated weights for policy 0, policy_version 1805408 (0.0011) [2023-12-27 04:32:50,577][105692] Updated weights for policy 0, policy_version 1805418 (0.0010) [2023-12-27 04:32:50,792][105620] Updated weights for policy 1, policy_version 1809196 (0.0011) [2023-12-27 04:32:50,854][105620] Updated weights for policy 1, policy_version 1809206 (0.0010) [2023-12-27 04:32:50,915][105620] Updated weights for policy 1, policy_version 1809216 (0.0006) [2023-12-27 04:32:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 925483008. Throughput: 0: 10110.0, 1: 9817.1. Samples: 925470112. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:32:51,063][104569] Avg episode reward: [(0, '8627.523'), (1, '9350.942')] [2023-12-27 04:32:51,372][105692] Updated weights for policy 0, policy_version 1805428 (0.0010) [2023-12-27 04:32:51,427][105692] Updated weights for policy 0, policy_version 1805438 (0.0011) [2023-12-27 04:32:51,480][105692] Updated weights for policy 0, policy_version 1805448 (0.0007) [2023-12-27 04:32:51,587][105620] Updated weights for policy 1, policy_version 1809226 (0.0009) [2023-12-27 04:32:51,653][105620] Updated weights for policy 1, policy_version 1809236 (0.0011) [2023-12-27 04:32:51,723][105620] Updated weights for policy 1, policy_version 1809246 (0.0010) [2023-12-27 04:32:51,789][105620] Updated weights for policy 1, policy_version 1809256 (0.0006) [2023-12-27 04:32:52,175][105692] Updated weights for policy 0, policy_version 1805458 (0.0006) [2023-12-27 04:32:52,224][105692] Updated weights for policy 0, policy_version 1805468 (0.0008) [2023-12-27 04:32:52,286][105692] Updated weights for policy 0, policy_version 1805478 (0.0007) [2023-12-27 04:32:52,356][105692] Updated weights for policy 0, policy_version 1805488 (0.0006) [2023-12-27 04:32:52,408][105620] Updated weights for policy 1, policy_version 1809266 (0.0008) [2023-12-27 04:32:52,463][105620] Updated weights for policy 1, policy_version 1809276 (0.0008) [2023-12-27 04:32:52,529][105620] Updated weights for policy 1, policy_version 1809286 (0.0008) [2023-12-27 04:32:53,062][105692] Updated weights for policy 0, policy_version 1805498 (0.0009) [2023-12-27 04:32:53,125][105692] Updated weights for policy 0, policy_version 1805508 (0.0007) [2023-12-27 04:32:53,200][105692] Updated weights for policy 0, policy_version 1805518 (0.0007) [2023-12-27 04:32:53,267][105620] Updated weights for policy 1, policy_version 1809296 (0.0008) [2023-12-27 04:32:53,320][105620] Updated weights for policy 1, policy_version 1809306 (0.0010) [2023-12-27 04:32:53,376][105620] Updated weights for policy 1, policy_version 1809316 (0.0010) [2023-12-27 04:32:53,731][105692] Updated weights for policy 0, policy_version 1805528 (0.0006) [2023-12-27 04:32:53,787][105692] Updated weights for policy 0, policy_version 1805538 (0.0005) [2023-12-27 04:32:53,842][105692] Updated weights for policy 0, policy_version 1805548 (0.0008) [2023-12-27 04:32:54,225][105620] Updated weights for policy 1, policy_version 1809326 (0.0008) [2023-12-27 04:32:54,284][105620] Updated weights for policy 1, policy_version 1809336 (0.0010) [2023-12-27 04:32:54,334][105620] Updated weights for policy 1, policy_version 1809346 (0.0010) [2023-12-27 04:32:54,441][105692] Updated weights for policy 0, policy_version 1805558 (0.0011) [2023-12-27 04:32:54,489][105692] Updated weights for policy 0, policy_version 1805568 (0.0010) [2023-12-27 04:32:54,550][105692] Updated weights for policy 0, policy_version 1805578 (0.0010) [2023-12-27 04:32:55,072][105620] Updated weights for policy 1, policy_version 1809356 (0.0010) [2023-12-27 04:32:55,130][105620] Updated weights for policy 1, policy_version 1809366 (0.0011) [2023-12-27 04:32:55,188][105620] Updated weights for policy 1, policy_version 1809376 (0.0010) [2023-12-27 04:32:55,221][105692] Updated weights for policy 0, policy_version 1805588 (0.0008) [2023-12-27 04:32:55,278][105692] Updated weights for policy 0, policy_version 1805598 (0.0005) [2023-12-27 04:32:55,329][105692] Updated weights for policy 0, policy_version 1805608 (0.0005) [2023-12-27 04:32:55,735][105620] Updated weights for policy 1, policy_version 1809386 (0.0008) [2023-12-27 04:32:55,793][105620] Updated weights for policy 1, policy_version 1809396 (0.0010) [2023-12-27 04:32:55,844][105620] Updated weights for policy 1, policy_version 1809406 (0.0010) [2023-12-27 04:32:55,891][105620] Updated weights for policy 1, policy_version 1809416 (0.0010) [2023-12-27 04:32:56,023][105692] Updated weights for policy 0, policy_version 1805618 (0.0008) [2023-12-27 04:32:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 925581312. Throughput: 0: 10069.2, 1: 9843.8. Samples: 925591388. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:32:56,063][104569] Avg episode reward: [(0, '8629.209'), (1, '9258.694')] [2023-12-27 04:32:56,082][105692] Updated weights for policy 0, policy_version 1805628 (0.0010) [2023-12-27 04:32:56,133][105692] Updated weights for policy 0, policy_version 1805638 (0.0005) [2023-12-27 04:32:56,182][105692] Updated weights for policy 0, policy_version 1805648 (0.0005) [2023-12-27 04:32:56,669][105620] Updated weights for policy 1, policy_version 1809426 (0.0011) [2023-12-27 04:32:56,721][105620] Updated weights for policy 1, policy_version 1809436 (0.0010) [2023-12-27 04:32:56,730][105692] Updated weights for policy 0, policy_version 1805658 (0.0008) [2023-12-27 04:32:56,778][105620] Updated weights for policy 1, policy_version 1809446 (0.0010) [2023-12-27 04:32:56,790][105692] Updated weights for policy 0, policy_version 1805668 (0.0011) [2023-12-27 04:32:56,846][105692] Updated weights for policy 0, policy_version 1805678 (0.0010) [2023-12-27 04:32:57,541][105620] Updated weights for policy 1, policy_version 1809456 (0.0010) [2023-12-27 04:32:57,570][105692] Updated weights for policy 0, policy_version 1805688 (0.0006) [2023-12-27 04:32:57,592][105620] Updated weights for policy 1, policy_version 1809466 (0.0010) [2023-12-27 04:32:57,625][105692] Updated weights for policy 0, policy_version 1805698 (0.0005) [2023-12-27 04:32:57,640][105620] Updated weights for policy 1, policy_version 1809476 (0.0010) [2023-12-27 04:32:57,684][105692] Updated weights for policy 0, policy_version 1805708 (0.0005) [2023-12-27 04:32:58,195][105620] Updated weights for policy 1, policy_version 1809486 (0.0008) [2023-12-27 04:32:58,254][105620] Updated weights for policy 1, policy_version 1809496 (0.0007) [2023-12-27 04:32:58,272][105692] Updated weights for policy 0, policy_version 1805718 (0.0009) [2023-12-27 04:32:58,324][105620] Updated weights for policy 1, policy_version 1809506 (0.0009) [2023-12-27 04:32:58,349][105692] Updated weights for policy 0, policy_version 1805728 (0.0009) [2023-12-27 04:32:58,423][105692] Updated weights for policy 0, policy_version 1805738 (0.0009) [2023-12-27 04:32:59,146][105692] Updated weights for policy 0, policy_version 1805748 (0.0008) [2023-12-27 04:32:59,213][105692] Updated weights for policy 0, policy_version 1805758 (0.0008) [2023-12-27 04:32:59,219][105620] Updated weights for policy 1, policy_version 1809516 (0.0009) [2023-12-27 04:32:59,276][105692] Updated weights for policy 0, policy_version 1805768 (0.0008) [2023-12-27 04:32:59,287][105620] Updated weights for policy 1, policy_version 1809526 (0.0008) [2023-12-27 04:32:59,350][105620] Updated weights for policy 1, policy_version 1809536 (0.0009) [2023-12-27 04:33:00,013][105692] Updated weights for policy 0, policy_version 1805778 (0.0008) [2023-12-27 04:33:00,061][105692] Updated weights for policy 0, policy_version 1805788 (0.0008) [2023-12-27 04:33:00,115][105692] Updated weights for policy 0, policy_version 1805798 (0.0008) [2023-12-27 04:33:00,121][105620] Updated weights for policy 1, policy_version 1809546 (0.0009) [2023-12-27 04:33:00,173][105692] Updated weights for policy 0, policy_version 1805808 (0.0007) [2023-12-27 04:33:00,177][105620] Updated weights for policy 1, policy_version 1809556 (0.0006) [2023-12-27 04:33:00,226][105620] Updated weights for policy 1, policy_version 1809566 (0.0005) [2023-12-27 04:33:00,282][105620] Updated weights for policy 1, policy_version 1809576 (0.0007) [2023-12-27 04:33:00,829][105692] Updated weights for policy 0, policy_version 1805819 (0.0009) [2023-12-27 04:33:00,876][105692] Updated weights for policy 0, policy_version 1805829 (0.0009) [2023-12-27 04:33:00,921][105692] Updated weights for policy 0, policy_version 1805839 (0.0008) [2023-12-27 04:33:01,043][105620] Updated weights for policy 1, policy_version 1809586 (0.0009) [2023-12-27 04:33:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 925679616. Throughput: 0: 10141.4, 1: 9841.7. Samples: 925651720. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:33:01,063][104569] Avg episode reward: [(0, '8260.885'), (1, '9258.575')] [2023-12-27 04:33:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001805840_462364672.pth... [2023-12-27 04:33:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001804656_462061568.pth [2023-12-27 04:33:01,098][105620] Updated weights for policy 1, policy_version 1809596 (0.0009) [2023-12-27 04:33:01,162][105620] Updated weights for policy 1, policy_version 1809606 (0.0009) [2023-12-27 04:33:01,172][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001809608_463323136.pth... [2023-12-27 04:33:01,176][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001808456_463028224.pth [2023-12-27 04:33:01,652][105692] Updated weights for policy 0, policy_version 1805849 (0.0007) [2023-12-27 04:33:01,713][105692] Updated weights for policy 0, policy_version 1805859 (0.0009) [2023-12-27 04:33:01,770][105692] Updated weights for policy 0, policy_version 1805869 (0.0009) [2023-12-27 04:33:01,888][105620] Updated weights for policy 1, policy_version 1809616 (0.0007) [2023-12-27 04:33:01,943][105620] Updated weights for policy 1, policy_version 1809626 (0.0008) [2023-12-27 04:33:02,001][105620] Updated weights for policy 1, policy_version 1809636 (0.0009) [2023-12-27 04:33:02,531][105692] Updated weights for policy 0, policy_version 1805879 (0.0010) [2023-12-27 04:33:02,590][105692] Updated weights for policy 0, policy_version 1805889 (0.0010) [2023-12-27 04:33:02,649][105692] Updated weights for policy 0, policy_version 1805899 (0.0008) [2023-12-27 04:33:02,671][105620] Updated weights for policy 1, policy_version 1809646 (0.0008) [2023-12-27 04:33:02,725][105620] Updated weights for policy 1, policy_version 1809656 (0.0008) [2023-12-27 04:33:02,775][105620] Updated weights for policy 1, policy_version 1809666 (0.0008) [2023-12-27 04:33:03,373][105620] Updated weights for policy 1, policy_version 1809676 (0.0008) [2023-12-27 04:33:03,381][105692] Updated weights for policy 0, policy_version 1805909 (0.0007) [2023-12-27 04:33:03,417][105620] Updated weights for policy 1, policy_version 1809686 (0.0005) [2023-12-27 04:33:03,438][105692] Updated weights for policy 0, policy_version 1805919 (0.0007) [2023-12-27 04:33:03,461][105620] Updated weights for policy 1, policy_version 1809696 (0.0006) [2023-12-27 04:33:03,485][105692] Updated weights for policy 0, policy_version 1805929 (0.0007) [2023-12-27 04:33:04,203][105620] Updated weights for policy 1, policy_version 1809706 (0.0005) [2023-12-27 04:33:04,250][105692] Updated weights for policy 0, policy_version 1805939 (0.0010) [2023-12-27 04:33:04,256][105620] Updated weights for policy 1, policy_version 1809716 (0.0005) [2023-12-27 04:33:04,310][105692] Updated weights for policy 0, policy_version 1805949 (0.0010) [2023-12-27 04:33:04,320][105620] Updated weights for policy 1, policy_version 1809726 (0.0007) [2023-12-27 04:33:04,369][105692] Updated weights for policy 0, policy_version 1805959 (0.0010) [2023-12-27 04:33:04,379][105620] Updated weights for policy 1, policy_version 1809736 (0.0006) [2023-12-27 04:33:04,980][105620] Updated weights for policy 1, policy_version 1809746 (0.0006) [2023-12-27 04:33:05,031][105620] Updated weights for policy 1, policy_version 1809756 (0.0006) [2023-12-27 04:33:05,041][105692] Updated weights for policy 0, policy_version 1805969 (0.0007) [2023-12-27 04:33:05,075][105620] Updated weights for policy 1, policy_version 1809766 (0.0005) [2023-12-27 04:33:05,110][105692] Updated weights for policy 0, policy_version 1805979 (0.0006) [2023-12-27 04:33:05,180][105692] Updated weights for policy 0, policy_version 1805989 (0.0006) [2023-12-27 04:33:05,248][105692] Updated weights for policy 0, policy_version 1805999 (0.0005) [2023-12-27 04:33:05,781][105620] Updated weights for policy 1, policy_version 1809776 (0.0006) [2023-12-27 04:33:05,820][105692] Updated weights for policy 0, policy_version 1806009 (0.0010) [2023-12-27 04:33:05,835][105620] Updated weights for policy 1, policy_version 1809786 (0.0005) [2023-12-27 04:33:05,869][105692] Updated weights for policy 0, policy_version 1806019 (0.0010) [2023-12-27 04:33:05,890][105620] Updated weights for policy 1, policy_version 1809796 (0.0005) [2023-12-27 04:33:05,927][105692] Updated weights for policy 0, policy_version 1806029 (0.0010) [2023-12-27 04:33:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 925786112. Throughput: 0: 10020.0, 1: 9875.5. Samples: 925768656. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:33:06,062][104569] Avg episode reward: [(0, '8711.289'), (1, '9166.600')] [2023-12-27 04:33:06,441][105620] Updated weights for policy 1, policy_version 1809806 (0.0010) [2023-12-27 04:33:06,498][105620] Updated weights for policy 1, policy_version 1809816 (0.0011) [2023-12-27 04:33:06,554][105620] Updated weights for policy 1, policy_version 1809826 (0.0011) [2023-12-27 04:33:06,566][105692] Updated weights for policy 0, policy_version 1806039 (0.0009) [2023-12-27 04:33:06,636][105692] Updated weights for policy 0, policy_version 1806049 (0.0008) [2023-12-27 04:33:06,696][105692] Updated weights for policy 0, policy_version 1806059 (0.0007) [2023-12-27 04:33:07,247][105620] Updated weights for policy 1, policy_version 1809836 (0.0011) [2023-12-27 04:33:07,302][105620] Updated weights for policy 1, policy_version 1809846 (0.0010) [2023-12-27 04:33:07,346][105692] Updated weights for policy 0, policy_version 1806069 (0.0007) [2023-12-27 04:33:07,362][105620] Updated weights for policy 1, policy_version 1809856 (0.0011) [2023-12-27 04:33:07,405][105692] Updated weights for policy 0, policy_version 1806079 (0.0011) [2023-12-27 04:33:07,462][105692] Updated weights for policy 0, policy_version 1806089 (0.0007) [2023-12-27 04:33:08,000][105692] Updated weights for policy 0, policy_version 1806099 (0.0005) [2023-12-27 04:33:08,015][105620] Updated weights for policy 1, policy_version 1809866 (0.0010) [2023-12-27 04:33:08,052][105692] Updated weights for policy 0, policy_version 1806109 (0.0005) [2023-12-27 04:33:08,068][105620] Updated weights for policy 1, policy_version 1809876 (0.0006) [2023-12-27 04:33:08,097][105692] Updated weights for policy 0, policy_version 1806119 (0.0005) [2023-12-27 04:33:08,120][105620] Updated weights for policy 1, policy_version 1809886 (0.0005) [2023-12-27 04:33:08,169][105620] Updated weights for policy 1, policy_version 1809896 (0.0005) [2023-12-27 04:33:08,659][105692] Updated weights for policy 0, policy_version 1806129 (0.0006) [2023-12-27 04:33:08,723][105692] Updated weights for policy 0, policy_version 1806139 (0.0010) [2023-12-27 04:33:08,785][105692] Updated weights for policy 0, policy_version 1806149 (0.0010) [2023-12-27 04:33:08,844][105692] Updated weights for policy 0, policy_version 1806159 (0.0010) [2023-12-27 04:33:08,891][105620] Updated weights for policy 1, policy_version 1809906 (0.0007) [2023-12-27 04:33:08,936][105620] Updated weights for policy 1, policy_version 1809916 (0.0008) [2023-12-27 04:33:08,984][105620] Updated weights for policy 1, policy_version 1809926 (0.0008) [2023-12-27 04:33:09,589][105692] Updated weights for policy 0, policy_version 1806169 (0.0006) [2023-12-27 04:33:09,659][105692] Updated weights for policy 0, policy_version 1806179 (0.0009) [2023-12-27 04:33:09,725][105692] Updated weights for policy 0, policy_version 1806189 (0.0011) [2023-12-27 04:33:09,751][105620] Updated weights for policy 1, policy_version 1809936 (0.0007) [2023-12-27 04:33:09,809][105620] Updated weights for policy 1, policy_version 1809946 (0.0009) [2023-12-27 04:33:09,877][105620] Updated weights for policy 1, policy_version 1809956 (0.0009) [2023-12-27 04:33:10,489][105692] Updated weights for policy 0, policy_version 1806199 (0.0008) [2023-12-27 04:33:10,554][105692] Updated weights for policy 0, policy_version 1806209 (0.0011) [2023-12-27 04:33:10,611][105692] Updated weights for policy 0, policy_version 1806219 (0.0011) [2023-12-27 04:33:10,647][105620] Updated weights for policy 1, policy_version 1809966 (0.0008) [2023-12-27 04:33:10,710][105620] Updated weights for policy 1, policy_version 1809976 (0.0008) [2023-12-27 04:33:10,778][105620] Updated weights for policy 1, policy_version 1809986 (0.0008) [2023-12-27 04:33:11,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19933.8, 300 sec: 19577.5). Total num frames: 925884416. Throughput: 0: 10128.2, 1: 9872.9. Samples: 925892080. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:33:11,063][104569] Avg episode reward: [(0, '9169.875'), (1, '8981.791')] [2023-12-27 04:33:11,355][105692] Updated weights for policy 0, policy_version 1806229 (0.0010) [2023-12-27 04:33:11,417][105692] Updated weights for policy 0, policy_version 1806239 (0.0008) [2023-12-27 04:33:11,480][105692] Updated weights for policy 0, policy_version 1806249 (0.0009) [2023-12-27 04:33:11,505][105620] Updated weights for policy 1, policy_version 1809996 (0.0009) [2023-12-27 04:33:11,572][105620] Updated weights for policy 1, policy_version 1810006 (0.0009) [2023-12-27 04:33:11,632][105620] Updated weights for policy 1, policy_version 1810016 (0.0009) [2023-12-27 04:33:12,288][105692] Updated weights for policy 0, policy_version 1806259 (0.0011) [2023-12-27 04:33:12,363][105692] Updated weights for policy 0, policy_version 1806269 (0.0011) [2023-12-27 04:33:12,419][105620] Updated weights for policy 1, policy_version 1810026 (0.0009) [2023-12-27 04:33:12,421][105692] Updated weights for policy 0, policy_version 1806279 (0.0008) [2023-12-27 04:33:12,474][105620] Updated weights for policy 1, policy_version 1810036 (0.0008) [2023-12-27 04:33:12,536][105620] Updated weights for policy 1, policy_version 1810046 (0.0010) [2023-12-27 04:33:12,596][105620] Updated weights for policy 1, policy_version 1810056 (0.0009) [2023-12-27 04:33:13,052][105692] Updated weights for policy 0, policy_version 1806289 (0.0006) [2023-12-27 04:33:13,097][105692] Updated weights for policy 0, policy_version 1806299 (0.0006) [2023-12-27 04:33:13,146][105692] Updated weights for policy 0, policy_version 1806309 (0.0005) [2023-12-27 04:33:13,203][105692] Updated weights for policy 0, policy_version 1806319 (0.0005) [2023-12-27 04:33:13,399][105620] Updated weights for policy 1, policy_version 1810066 (0.0005) [2023-12-27 04:33:13,473][105620] Updated weights for policy 1, policy_version 1810076 (0.0005) [2023-12-27 04:33:13,533][105620] Updated weights for policy 1, policy_version 1810087 (0.0009) [2023-12-27 04:33:13,761][105692] Updated weights for policy 0, policy_version 1806329 (0.0006) [2023-12-27 04:33:13,810][105692] Updated weights for policy 0, policy_version 1806339 (0.0008) [2023-12-27 04:33:13,858][105692] Updated weights for policy 0, policy_version 1806349 (0.0008) [2023-12-27 04:33:14,202][105620] Updated weights for policy 1, policy_version 1810097 (0.0010) [2023-12-27 04:33:14,260][105620] Updated weights for policy 1, policy_version 1810107 (0.0010) [2023-12-27 04:33:14,314][105620] Updated weights for policy 1, policy_version 1810117 (0.0010) [2023-12-27 04:33:14,586][105692] Updated weights for policy 0, policy_version 1806359 (0.0009) [2023-12-27 04:33:14,636][105692] Updated weights for policy 0, policy_version 1806371 (0.0010) [2023-12-27 04:33:14,694][105692] Updated weights for policy 0, policy_version 1806381 (0.0008) [2023-12-27 04:33:14,980][105620] Updated weights for policy 1, policy_version 1810127 (0.0009) [2023-12-27 04:33:15,049][105620] Updated weights for policy 1, policy_version 1810137 (0.0008) [2023-12-27 04:33:15,128][105620] Updated weights for policy 1, policy_version 1810147 (0.0009) [2023-12-27 04:33:15,295][105692] Updated weights for policy 0, policy_version 1806391 (0.0009) [2023-12-27 04:33:15,348][105692] Updated weights for policy 0, policy_version 1806401 (0.0010) [2023-12-27 04:33:15,399][105692] Updated weights for policy 0, policy_version 1806411 (0.0010) [2023-12-27 04:33:15,776][105620] Updated weights for policy 1, policy_version 1810157 (0.0009) [2023-12-27 04:33:15,827][105620] Updated weights for policy 1, policy_version 1810167 (0.0008) [2023-12-27 04:33:15,875][105620] Updated weights for policy 1, policy_version 1810177 (0.0008) [2023-12-27 04:33:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 20070.4, 300 sec: 19549.7). Total num frames: 925982720. Throughput: 0: 10130.8, 1: 9800.4. Samples: 925949912. Policy #0 lag: (min: 19.0, avg: 27.9, max: 51.0) [2023-12-27 04:33:16,062][104569] Avg episode reward: [(0, '8349.485'), (1, '8983.092')] [2023-12-27 04:33:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001806416_462512128.pth... [2023-12-27 04:33:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001810184_463470592.pth... [2023-12-27 04:33:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001805232_462209024.pth [2023-12-27 04:33:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001809032_463175680.pth [2023-12-27 04:33:16,144][105692] Updated weights for policy 0, policy_version 1806421 (0.0008) [2023-12-27 04:33:16,205][105692] Updated weights for policy 0, policy_version 1806431 (0.0010) [2023-12-27 04:33:16,262][105692] Updated weights for policy 0, policy_version 1806441 (0.0010) [2023-12-27 04:33:16,549][105620] Updated weights for policy 1, policy_version 1810187 (0.0009) [2023-12-27 04:33:16,611][105620] Updated weights for policy 1, policy_version 1810197 (0.0009) [2023-12-27 04:33:16,677][105620] Updated weights for policy 1, policy_version 1810207 (0.0010) [2023-12-27 04:33:16,896][105692] Updated weights for policy 0, policy_version 1806451 (0.0010) [2023-12-27 04:33:16,964][105692] Updated weights for policy 0, policy_version 1806461 (0.0007) [2023-12-27 04:33:17,027][105692] Updated weights for policy 0, policy_version 1806471 (0.0008) [2023-12-27 04:33:17,517][105620] Updated weights for policy 1, policy_version 1810217 (0.0010) [2023-12-27 04:33:17,570][105620] Updated weights for policy 1, policy_version 1810227 (0.0009) [2023-12-27 04:33:17,618][105620] Updated weights for policy 1, policy_version 1810237 (0.0009) [2023-12-27 04:33:17,660][105692] Updated weights for policy 0, policy_version 1806481 (0.0009) [2023-12-27 04:33:17,670][105620] Updated weights for policy 1, policy_version 1810247 (0.0008) [2023-12-27 04:33:17,708][105692] Updated weights for policy 0, policy_version 1806491 (0.0008) [2023-12-27 04:33:17,756][105692] Updated weights for policy 0, policy_version 1806501 (0.0009) [2023-12-27 04:33:17,803][105692] Updated weights for policy 0, policy_version 1806511 (0.0009) [2023-12-27 04:33:18,450][105620] Updated weights for policy 1, policy_version 1810257 (0.0007) [2023-12-27 04:33:18,510][105620] Updated weights for policy 1, policy_version 1810267 (0.0005) [2023-12-27 04:33:18,551][105692] Updated weights for policy 0, policy_version 1806521 (0.0008) [2023-12-27 04:33:18,576][105620] Updated weights for policy 1, policy_version 1810277 (0.0009) [2023-12-27 04:33:18,600][105692] Updated weights for policy 0, policy_version 1806531 (0.0006) [2023-12-27 04:33:18,651][105692] Updated weights for policy 0, policy_version 1806541 (0.0008) [2023-12-27 04:33:19,242][105620] Updated weights for policy 1, policy_version 1810287 (0.0009) [2023-12-27 04:33:19,308][105620] Updated weights for policy 1, policy_version 1810297 (0.0008) [2023-12-27 04:33:19,373][105620] Updated weights for policy 1, policy_version 1810307 (0.0010) [2023-12-27 04:33:19,421][105692] Updated weights for policy 0, policy_version 1806551 (0.0008) [2023-12-27 04:33:19,469][105692] Updated weights for policy 0, policy_version 1806561 (0.0008) [2023-12-27 04:33:19,535][105692] Updated weights for policy 0, policy_version 1806571 (0.0009) [2023-12-27 04:33:20,086][105620] Updated weights for policy 1, policy_version 1810317 (0.0009) [2023-12-27 04:33:20,138][105620] Updated weights for policy 1, policy_version 1810327 (0.0011) [2023-12-27 04:33:20,200][105620] Updated weights for policy 1, policy_version 1810337 (0.0010) [2023-12-27 04:33:20,333][105692] Updated weights for policy 0, policy_version 1806581 (0.0009) [2023-12-27 04:33:20,388][105692] Updated weights for policy 0, policy_version 1806591 (0.0008) [2023-12-27 04:33:20,445][105692] Updated weights for policy 0, policy_version 1806601 (0.0008) [2023-12-27 04:33:20,956][105620] Updated weights for policy 1, policy_version 1810347 (0.0011) [2023-12-27 04:33:21,013][105620] Updated weights for policy 1, policy_version 1810357 (0.0011) [2023-12-27 04:33:21,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19797.4, 300 sec: 19522.0). Total num frames: 926072832. Throughput: 0: 10130.5, 1: 9743.0. Samples: 926068100. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:33:21,062][104569] Avg episode reward: [(0, '8258.875'), (1, '9167.937')] [2023-12-27 04:33:21,074][105620] Updated weights for policy 1, policy_version 1810367 (0.0011) [2023-12-27 04:33:21,234][105692] Updated weights for policy 0, policy_version 1806611 (0.0009) [2023-12-27 04:33:21,298][105692] Updated weights for policy 0, policy_version 1806621 (0.0008) [2023-12-27 04:33:21,358][105692] Updated weights for policy 0, policy_version 1806631 (0.0008) [2023-12-27 04:33:21,855][105620] Updated weights for policy 1, policy_version 1810377 (0.0011) [2023-12-27 04:33:21,927][105620] Updated weights for policy 1, policy_version 1810387 (0.0010) [2023-12-27 04:33:21,990][105620] Updated weights for policy 1, policy_version 1810397 (0.0010) [2023-12-27 04:33:22,043][105620] Updated weights for policy 1, policy_version 1810407 (0.0011) [2023-12-27 04:33:22,174][105692] Updated weights for policy 0, policy_version 1806641 (0.0009) [2023-12-27 04:33:22,230][105692] Updated weights for policy 0, policy_version 1806651 (0.0008) [2023-12-27 04:33:22,296][105692] Updated weights for policy 0, policy_version 1806661 (0.0009) [2023-12-27 04:33:22,363][105692] Updated weights for policy 0, policy_version 1806671 (0.0008) [2023-12-27 04:33:22,754][105620] Updated weights for policy 1, policy_version 1810417 (0.0008) [2023-12-27 04:33:22,822][105620] Updated weights for policy 1, policy_version 1810427 (0.0008) [2023-12-27 04:33:22,884][105620] Updated weights for policy 1, policy_version 1810437 (0.0009) [2023-12-27 04:33:23,185][105692] Updated weights for policy 0, policy_version 1806681 (0.0009) [2023-12-27 04:33:23,238][105692] Updated weights for policy 0, policy_version 1806691 (0.0010) [2023-12-27 04:33:23,290][105692] Updated weights for policy 0, policy_version 1806701 (0.0011) [2023-12-27 04:33:23,604][105620] Updated weights for policy 1, policy_version 1810447 (0.0006) [2023-12-27 04:33:23,652][105620] Updated weights for policy 1, policy_version 1810457 (0.0005) [2023-12-27 04:33:23,697][105620] Updated weights for policy 1, policy_version 1810467 (0.0005) [2023-12-27 04:33:24,069][105692] Updated weights for policy 0, policy_version 1806711 (0.0006) [2023-12-27 04:33:24,130][105692] Updated weights for policy 0, policy_version 1806721 (0.0007) [2023-12-27 04:33:24,195][105692] Updated weights for policy 0, policy_version 1806731 (0.0006) [2023-12-27 04:33:24,229][105620] Updated weights for policy 1, policy_version 1810477 (0.0007) [2023-12-27 04:33:24,284][105620] Updated weights for policy 1, policy_version 1810487 (0.0009) [2023-12-27 04:33:24,343][105620] Updated weights for policy 1, policy_version 1810497 (0.0008) [2023-12-27 04:33:24,846][105692] Updated weights for policy 0, policy_version 1806741 (0.0006) [2023-12-27 04:33:24,908][105692] Updated weights for policy 0, policy_version 1806751 (0.0009) [2023-12-27 04:33:24,974][105692] Updated weights for policy 0, policy_version 1806761 (0.0009) [2023-12-27 04:33:25,064][105620] Updated weights for policy 1, policy_version 1810507 (0.0007) [2023-12-27 04:33:25,121][105620] Updated weights for policy 1, policy_version 1810517 (0.0008) [2023-12-27 04:33:25,181][105620] Updated weights for policy 1, policy_version 1810527 (0.0009) [2023-12-27 04:33:25,712][105692] Updated weights for policy 0, policy_version 1806771 (0.0009) [2023-12-27 04:33:25,759][105692] Updated weights for policy 0, policy_version 1806781 (0.0009) [2023-12-27 04:33:25,806][105692] Updated weights for policy 0, policy_version 1806791 (0.0009) [2023-12-27 04:33:25,933][105620] Updated weights for policy 1, policy_version 1810537 (0.0009) [2023-12-27 04:33:25,982][105620] Updated weights for policy 1, policy_version 1810547 (0.0008) [2023-12-27 04:33:26,040][105620] Updated weights for policy 1, policy_version 1810557 (0.0009) [2023-12-27 04:33:26,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19797.3, 300 sec: 19521.9). Total num frames: 926171136. Throughput: 0: 10063.6, 1: 9760.6. Samples: 926181800. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:33:26,063][104569] Avg episode reward: [(0, '8350.849'), (1, '9258.542')] [2023-12-27 04:33:26,103][105620] Updated weights for policy 1, policy_version 1810567 (0.0008) [2023-12-27 04:33:26,548][105692] Updated weights for policy 0, policy_version 1806801 (0.0009) [2023-12-27 04:33:26,610][105692] Updated weights for policy 0, policy_version 1806811 (0.0009) [2023-12-27 04:33:26,655][105692] Updated weights for policy 0, policy_version 1806821 (0.0008) [2023-12-27 04:33:26,713][105692] Updated weights for policy 0, policy_version 1806832 (0.0010) [2023-12-27 04:33:26,863][105620] Updated weights for policy 1, policy_version 1810577 (0.0008) [2023-12-27 04:33:26,923][105620] Updated weights for policy 1, policy_version 1810587 (0.0008) [2023-12-27 04:33:26,985][105620] Updated weights for policy 1, policy_version 1810597 (0.0009) [2023-12-27 04:33:27,564][105620] Updated weights for policy 1, policy_version 1810607 (0.0009) [2023-12-27 04:33:27,570][105692] Updated weights for policy 0, policy_version 1806842 (0.0007) [2023-12-27 04:33:27,609][105620] Updated weights for policy 1, policy_version 1810617 (0.0006) [2023-12-27 04:33:27,619][105692] Updated weights for policy 0, policy_version 1806852 (0.0007) [2023-12-27 04:33:27,662][105620] Updated weights for policy 1, policy_version 1810627 (0.0008) [2023-12-27 04:33:27,672][105692] Updated weights for policy 0, policy_version 1806862 (0.0009) [2023-12-27 04:33:28,263][105620] Updated weights for policy 1, policy_version 1810637 (0.0008) [2023-12-27 04:33:28,316][105620] Updated weights for policy 1, policy_version 1810647 (0.0008) [2023-12-27 04:33:28,331][105692] Updated weights for policy 0, policy_version 1806872 (0.0008) [2023-12-27 04:33:28,380][105620] Updated weights for policy 1, policy_version 1810657 (0.0007) [2023-12-27 04:33:28,386][105692] Updated weights for policy 0, policy_version 1806882 (0.0010) [2023-12-27 04:33:28,435][105692] Updated weights for policy 0, policy_version 1806892 (0.0010) [2023-12-27 04:33:28,980][105620] Updated weights for policy 1, policy_version 1810667 (0.0006) [2023-12-27 04:33:29,023][105692] Updated weights for policy 0, policy_version 1806902 (0.0009) [2023-12-27 04:33:29,042][105620] Updated weights for policy 1, policy_version 1810677 (0.0005) [2023-12-27 04:33:29,078][105692] Updated weights for policy 0, policy_version 1806912 (0.0010) [2023-12-27 04:33:29,101][105620] Updated weights for policy 1, policy_version 1810687 (0.0006) [2023-12-27 04:33:29,135][105692] Updated weights for policy 0, policy_version 1806922 (0.0006) [2023-12-27 04:33:29,723][105692] Updated weights for policy 0, policy_version 1806932 (0.0005) [2023-12-27 04:33:29,782][105692] Updated weights for policy 0, policy_version 1806942 (0.0007) [2023-12-27 04:33:29,819][105620] Updated weights for policy 1, policy_version 1810697 (0.0009) [2023-12-27 04:33:29,850][105692] Updated weights for policy 0, policy_version 1806952 (0.0011) [2023-12-27 04:33:29,878][105620] Updated weights for policy 1, policy_version 1810707 (0.0007) [2023-12-27 04:33:29,929][105620] Updated weights for policy 1, policy_version 1810717 (0.0008) [2023-12-27 04:33:29,983][105620] Updated weights for policy 1, policy_version 1810727 (0.0008) [2023-12-27 04:33:30,568][105692] Updated weights for policy 0, policy_version 1806962 (0.0010) [2023-12-27 04:33:30,623][105692] Updated weights for policy 0, policy_version 1806972 (0.0010) [2023-12-27 04:33:30,668][105692] Updated weights for policy 0, policy_version 1806982 (0.0010) [2023-12-27 04:33:30,712][105692] Updated weights for policy 0, policy_version 1806992 (0.0010) [2023-12-27 04:33:30,772][105620] Updated weights for policy 1, policy_version 1810737 (0.0008) [2023-12-27 04:33:30,817][105620] Updated weights for policy 1, policy_version 1810747 (0.0008) [2023-12-27 04:33:30,864][105620] Updated weights for policy 1, policy_version 1810757 (0.0008) [2023-12-27 04:33:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19933.9, 300 sec: 19549.7). Total num frames: 926277632. Throughput: 0: 10016.8, 1: 9850.4. Samples: 926242316. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:33:31,062][104569] Avg episode reward: [(0, '8354.806'), (1, '9258.556')] [2023-12-27 04:33:31,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001806992_462659584.pth... [2023-12-27 04:33:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001810760_463618048.pth... [2023-12-27 04:33:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001805840_462364672.pth [2023-12-27 04:33:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001809608_463323136.pth [2023-12-27 04:33:31,485][105692] Updated weights for policy 0, policy_version 1807002 (0.0009) [2023-12-27 04:33:31,537][105692] Updated weights for policy 0, policy_version 1807012 (0.0010) [2023-12-27 04:33:31,597][105692] Updated weights for policy 0, policy_version 1807022 (0.0010) [2023-12-27 04:33:31,668][105620] Updated weights for policy 1, policy_version 1810767 (0.0006) [2023-12-27 04:33:31,735][105620] Updated weights for policy 1, policy_version 1810777 (0.0007) [2023-12-27 04:33:31,792][105620] Updated weights for policy 1, policy_version 1810787 (0.0008) [2023-12-27 04:33:32,229][105692] Updated weights for policy 0, policy_version 1807032 (0.0007) [2023-12-27 04:33:32,291][105692] Updated weights for policy 0, policy_version 1807042 (0.0010) [2023-12-27 04:33:32,357][105692] Updated weights for policy 0, policy_version 1807052 (0.0010) [2023-12-27 04:33:32,477][105620] Updated weights for policy 1, policy_version 1810797 (0.0007) [2023-12-27 04:33:32,533][105620] Updated weights for policy 1, policy_version 1810807 (0.0008) [2023-12-27 04:33:32,582][105620] Updated weights for policy 1, policy_version 1810817 (0.0008) [2023-12-27 04:33:33,084][105692] Updated weights for policy 0, policy_version 1807062 (0.0010) [2023-12-27 04:33:33,148][105692] Updated weights for policy 0, policy_version 1807072 (0.0010) [2023-12-27 04:33:33,210][105692] Updated weights for policy 0, policy_version 1807082 (0.0010) [2023-12-27 04:33:33,245][105620] Updated weights for policy 1, policy_version 1810827 (0.0008) [2023-12-27 04:33:33,301][105620] Updated weights for policy 1, policy_version 1810837 (0.0009) [2023-12-27 04:33:33,348][105620] Updated weights for policy 1, policy_version 1810847 (0.0009) [2023-12-27 04:33:33,776][105692] Updated weights for policy 0, policy_version 1807092 (0.0008) [2023-12-27 04:33:33,840][105692] Updated weights for policy 0, policy_version 1807102 (0.0006) [2023-12-27 04:33:33,901][105692] Updated weights for policy 0, policy_version 1807112 (0.0007) [2023-12-27 04:33:34,186][105620] Updated weights for policy 1, policy_version 1810857 (0.0009) [2023-12-27 04:33:34,256][105620] Updated weights for policy 1, policy_version 1810867 (0.0006) [2023-12-27 04:33:34,324][105620] Updated weights for policy 1, policy_version 1810877 (0.0008) [2023-12-27 04:33:34,386][105620] Updated weights for policy 1, policy_version 1810887 (0.0010) [2023-12-27 04:33:34,518][105692] Updated weights for policy 0, policy_version 1807122 (0.0005) [2023-12-27 04:33:34,572][105692] Updated weights for policy 0, policy_version 1807132 (0.0006) [2023-12-27 04:33:34,637][105692] Updated weights for policy 0, policy_version 1807142 (0.0008) [2023-12-27 04:33:34,690][105692] Updated weights for policy 0, policy_version 1807152 (0.0011) [2023-12-27 04:33:35,031][105620] Updated weights for policy 1, policy_version 1810897 (0.0011) [2023-12-27 04:33:35,089][105620] Updated weights for policy 1, policy_version 1810907 (0.0010) [2023-12-27 04:33:35,151][105620] Updated weights for policy 1, policy_version 1810917 (0.0010) [2023-12-27 04:33:35,412][105692] Updated weights for policy 0, policy_version 1807162 (0.0011) [2023-12-27 04:33:35,474][105692] Updated weights for policy 0, policy_version 1807172 (0.0011) [2023-12-27 04:33:35,539][105692] Updated weights for policy 0, policy_version 1807182 (0.0010) [2023-12-27 04:33:35,910][105620] Updated weights for policy 1, policy_version 1810927 (0.0009) [2023-12-27 04:33:35,965][105620] Updated weights for policy 1, policy_version 1810937 (0.0005) [2023-12-27 04:33:36,017][105620] Updated weights for policy 1, policy_version 1810947 (0.0005) [2023-12-27 04:33:36,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19933.8, 300 sec: 19549.7). Total num frames: 926375936. Throughput: 0: 10013.0, 1: 9822.0. Samples: 926362684. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:33:36,063][104569] Avg episode reward: [(0, '8450.112'), (1, '9258.520')] [2023-12-27 04:33:36,162][105692] Updated weights for policy 0, policy_version 1807192 (0.0008) [2023-12-27 04:33:36,224][105692] Updated weights for policy 0, policy_version 1807202 (0.0008) [2023-12-27 04:33:36,290][105692] Updated weights for policy 0, policy_version 1807212 (0.0007) [2023-12-27 04:33:36,639][105620] Updated weights for policy 1, policy_version 1810957 (0.0006) [2023-12-27 04:33:36,705][105620] Updated weights for policy 1, policy_version 1810967 (0.0006) [2023-12-27 04:33:36,767][105620] Updated weights for policy 1, policy_version 1810977 (0.0006) [2023-12-27 04:33:37,028][105692] Updated weights for policy 0, policy_version 1807222 (0.0009) [2023-12-27 04:33:37,091][105692] Updated weights for policy 0, policy_version 1807232 (0.0007) [2023-12-27 04:33:37,150][105692] Updated weights for policy 0, policy_version 1807242 (0.0010) [2023-12-27 04:33:37,379][105620] Updated weights for policy 1, policy_version 1810987 (0.0005) [2023-12-27 04:33:37,436][105620] Updated weights for policy 1, policy_version 1810997 (0.0009) [2023-12-27 04:33:37,489][105620] Updated weights for policy 1, policy_version 1811007 (0.0009) [2023-12-27 04:33:37,797][105692] Updated weights for policy 0, policy_version 1807252 (0.0007) [2023-12-27 04:33:37,849][105692] Updated weights for policy 0, policy_version 1807262 (0.0006) [2023-12-27 04:33:37,901][105692] Updated weights for policy 0, policy_version 1807272 (0.0006) [2023-12-27 04:33:38,262][105620] Updated weights for policy 1, policy_version 1811017 (0.0008) [2023-12-27 04:33:38,314][105620] Updated weights for policy 1, policy_version 1811027 (0.0008) [2023-12-27 04:33:38,375][105620] Updated weights for policy 1, policy_version 1811037 (0.0008) [2023-12-27 04:33:38,439][105620] Updated weights for policy 1, policy_version 1811047 (0.0009) [2023-12-27 04:33:38,608][105692] Updated weights for policy 0, policy_version 1807282 (0.0008) [2023-12-27 04:33:38,663][105692] Updated weights for policy 0, policy_version 1807292 (0.0010) [2023-12-27 04:33:38,723][105692] Updated weights for policy 0, policy_version 1807302 (0.0010) [2023-12-27 04:33:38,786][105692] Updated weights for policy 0, policy_version 1807312 (0.0010) [2023-12-27 04:33:39,191][105620] Updated weights for policy 1, policy_version 1811057 (0.0006) [2023-12-27 04:33:39,257][105620] Updated weights for policy 1, policy_version 1811067 (0.0007) [2023-12-27 04:33:39,315][105620] Updated weights for policy 1, policy_version 1811077 (0.0005) [2023-12-27 04:33:39,576][105692] Updated weights for policy 0, policy_version 1807322 (0.0007) [2023-12-27 04:33:39,641][105692] Updated weights for policy 0, policy_version 1807332 (0.0008) [2023-12-27 04:33:39,702][105692] Updated weights for policy 0, policy_version 1807342 (0.0008) [2023-12-27 04:33:40,045][105620] Updated weights for policy 1, policy_version 1811087 (0.0008) [2023-12-27 04:33:40,104][105620] Updated weights for policy 1, policy_version 1811097 (0.0009) [2023-12-27 04:33:40,164][105620] Updated weights for policy 1, policy_version 1811107 (0.0009) [2023-12-27 04:33:40,439][105692] Updated weights for policy 0, policy_version 1807352 (0.0009) [2023-12-27 04:33:40,492][105692] Updated weights for policy 0, policy_version 1807362 (0.0010) [2023-12-27 04:33:40,540][105692] Updated weights for policy 0, policy_version 1807372 (0.0009) [2023-12-27 04:33:40,903][105620] Updated weights for policy 1, policy_version 1811117 (0.0008) [2023-12-27 04:33:40,954][105620] Updated weights for policy 1, policy_version 1811127 (0.0007) [2023-12-27 04:33:41,012][105620] Updated weights for policy 1, policy_version 1811137 (0.0005) [2023-12-27 04:33:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 926474240. Throughput: 0: 9929.0, 1: 9801.0. Samples: 926479236. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:33:41,062][104569] Avg episode reward: [(0, '8356.643'), (1, '9166.128')] [2023-12-27 04:33:41,400][105692] Updated weights for policy 0, policy_version 1807382 (0.0009) [2023-12-27 04:33:41,467][105692] Updated weights for policy 0, policy_version 1807392 (0.0009) [2023-12-27 04:33:41,533][105692] Updated weights for policy 0, policy_version 1807402 (0.0009) [2023-12-27 04:33:41,739][105620] Updated weights for policy 1, policy_version 1811147 (0.0008) [2023-12-27 04:33:41,797][105620] Updated weights for policy 1, policy_version 1811157 (0.0006) [2023-12-27 04:33:41,860][105620] Updated weights for policy 1, policy_version 1811167 (0.0006) [2023-12-27 04:33:42,375][105692] Updated weights for policy 0, policy_version 1807412 (0.0009) [2023-12-27 04:33:42,440][105692] Updated weights for policy 0, policy_version 1807422 (0.0008) [2023-12-27 04:33:42,495][105692] Updated weights for policy 0, policy_version 1807432 (0.0007) [2023-12-27 04:33:42,525][105620] Updated weights for policy 1, policy_version 1811177 (0.0006) [2023-12-27 04:33:42,594][105620] Updated weights for policy 1, policy_version 1811187 (0.0009) [2023-12-27 04:33:42,652][105620] Updated weights for policy 1, policy_version 1811197 (0.0010) [2023-12-27 04:33:42,710][105620] Updated weights for policy 1, policy_version 1811207 (0.0010) [2023-12-27 04:33:43,123][105692] Updated weights for policy 0, policy_version 1807442 (0.0006) [2023-12-27 04:33:43,171][105692] Updated weights for policy 0, policy_version 1807452 (0.0007) [2023-12-27 04:33:43,220][105692] Updated weights for policy 0, policy_version 1807462 (0.0007) [2023-12-27 04:33:43,278][105692] Updated weights for policy 0, policy_version 1807472 (0.0007) [2023-12-27 04:33:43,489][105620] Updated weights for policy 1, policy_version 1811217 (0.0011) [2023-12-27 04:33:43,544][105620] Updated weights for policy 1, policy_version 1811227 (0.0010) [2023-12-27 04:33:43,599][105620] Updated weights for policy 1, policy_version 1811237 (0.0010) [2023-12-27 04:33:43,960][105692] Updated weights for policy 0, policy_version 1807482 (0.0007) [2023-12-27 04:33:44,018][105692] Updated weights for policy 0, policy_version 1807492 (0.0010) [2023-12-27 04:33:44,066][105692] Updated weights for policy 0, policy_version 1807502 (0.0010) [2023-12-27 04:33:44,303][105620] Updated weights for policy 1, policy_version 1811247 (0.0009) [2023-12-27 04:33:44,373][105620] Updated weights for policy 1, policy_version 1811257 (0.0006) [2023-12-27 04:33:44,435][105620] Updated weights for policy 1, policy_version 1811267 (0.0005) [2023-12-27 04:33:44,808][105692] Updated weights for policy 0, policy_version 1807512 (0.0007) [2023-12-27 04:33:44,872][105692] Updated weights for policy 0, policy_version 1807522 (0.0006) [2023-12-27 04:33:44,919][105692] Updated weights for policy 0, policy_version 1807532 (0.0006) [2023-12-27 04:33:45,134][105620] Updated weights for policy 1, policy_version 1811277 (0.0007) [2023-12-27 04:33:45,194][105620] Updated weights for policy 1, policy_version 1811287 (0.0009) [2023-12-27 04:33:45,249][105620] Updated weights for policy 1, policy_version 1811297 (0.0009) [2023-12-27 04:33:45,611][105692] Updated weights for policy 0, policy_version 1807542 (0.0008) [2023-12-27 04:33:45,663][105692] Updated weights for policy 0, policy_version 1807552 (0.0005) [2023-12-27 04:33:45,720][105692] Updated weights for policy 0, policy_version 1807562 (0.0007) [2023-12-27 04:33:46,010][105620] Updated weights for policy 1, policy_version 1811307 (0.0010) [2023-12-27 04:33:46,062][104569] Fps is (10 sec: 18840.9, 60 sec: 19660.7, 300 sec: 19549.7). Total num frames: 926564352. Throughput: 0: 9862.9, 1: 9803.2. Samples: 926536696. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:33:46,063][104569] Avg episode reward: [(0, '8535.838'), (1, '9258.497')] [2023-12-27 04:33:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001807568_462807040.pth... [2023-12-27 04:33:46,071][105620] Updated weights for policy 1, policy_version 1811317 (0.0007) [2023-12-27 04:33:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001806416_462512128.pth [2023-12-27 04:33:46,134][105620] Updated weights for policy 1, policy_version 1811327 (0.0010) [2023-12-27 04:33:46,189][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001811336_463765504.pth... [2023-12-27 04:33:46,193][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001810184_463470592.pth [2023-12-27 04:33:46,465][105692] Updated weights for policy 0, policy_version 1807572 (0.0009) [2023-12-27 04:33:46,525][105692] Updated weights for policy 0, policy_version 1807582 (0.0009) [2023-12-27 04:33:46,580][105692] Updated weights for policy 0, policy_version 1807592 (0.0005) [2023-12-27 04:33:46,899][105620] Updated weights for policy 1, policy_version 1811337 (0.0008) [2023-12-27 04:33:46,954][105620] Updated weights for policy 1, policy_version 1811347 (0.0007) [2023-12-27 04:33:47,009][105620] Updated weights for policy 1, policy_version 1811357 (0.0008) [2023-12-27 04:33:47,063][105620] Updated weights for policy 1, policy_version 1811367 (0.0008) [2023-12-27 04:33:47,254][105692] Updated weights for policy 0, policy_version 1807602 (0.0006) [2023-12-27 04:33:47,304][105692] Updated weights for policy 0, policy_version 1807612 (0.0006) [2023-12-27 04:33:47,370][105692] Updated weights for policy 0, policy_version 1807622 (0.0007) [2023-12-27 04:33:47,434][105692] Updated weights for policy 0, policy_version 1807632 (0.0009) [2023-12-27 04:33:47,903][105620] Updated weights for policy 1, policy_version 1811377 (0.0009) [2023-12-27 04:33:47,970][105620] Updated weights for policy 1, policy_version 1811387 (0.0009) [2023-12-27 04:33:48,003][105692] Updated weights for policy 0, policy_version 1807642 (0.0007) [2023-12-27 04:33:48,031][105620] Updated weights for policy 1, policy_version 1811397 (0.0007) [2023-12-27 04:33:48,064][105692] Updated weights for policy 0, policy_version 1807652 (0.0005) [2023-12-27 04:33:48,133][105692] Updated weights for policy 0, policy_version 1807662 (0.0005) [2023-12-27 04:33:48,790][105692] Updated weights for policy 0, policy_version 1807672 (0.0005) [2023-12-27 04:33:48,851][105692] Updated weights for policy 0, policy_version 1807682 (0.0006) [2023-12-27 04:33:48,866][105620] Updated weights for policy 1, policy_version 1811407 (0.0010) [2023-12-27 04:33:48,911][105692] Updated weights for policy 0, policy_version 1807692 (0.0006) [2023-12-27 04:33:48,922][105620] Updated weights for policy 1, policy_version 1811417 (0.0008) [2023-12-27 04:33:48,977][105620] Updated weights for policy 1, policy_version 1811427 (0.0009) [2023-12-27 04:33:49,511][105692] Updated weights for policy 0, policy_version 1807702 (0.0005) [2023-12-27 04:33:49,569][105692] Updated weights for policy 0, policy_version 1807712 (0.0006) [2023-12-27 04:33:49,630][105692] Updated weights for policy 0, policy_version 1807722 (0.0009) [2023-12-27 04:33:49,795][105620] Updated weights for policy 1, policy_version 1811437 (0.0009) [2023-12-27 04:33:49,867][105620] Updated weights for policy 1, policy_version 1811447 (0.0009) [2023-12-27 04:33:49,934][105620] Updated weights for policy 1, policy_version 1811457 (0.0009) [2023-12-27 04:33:50,310][105692] Updated weights for policy 0, policy_version 1807732 (0.0008) [2023-12-27 04:33:50,365][105692] Updated weights for policy 0, policy_version 1807742 (0.0009) [2023-12-27 04:33:50,412][105692] Updated weights for policy 0, policy_version 1807752 (0.0008) [2023-12-27 04:33:50,709][105620] Updated weights for policy 1, policy_version 1811467 (0.0009) [2023-12-27 04:33:50,775][105620] Updated weights for policy 1, policy_version 1811477 (0.0009) [2023-12-27 04:33:50,834][105620] Updated weights for policy 1, policy_version 1811487 (0.0009) [2023-12-27 04:33:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 926662656. Throughput: 0: 9970.9, 1: 9672.7. Samples: 926652620. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:33:51,062][104569] Avg episode reward: [(0, '8810.612'), (1, '9258.581')] [2023-12-27 04:33:51,180][105692] Updated weights for policy 0, policy_version 1807762 (0.0009) [2023-12-27 04:33:51,240][105692] Updated weights for policy 0, policy_version 1807772 (0.0009) [2023-12-27 04:33:51,304][105692] Updated weights for policy 0, policy_version 1807782 (0.0010) [2023-12-27 04:33:51,368][105692] Updated weights for policy 0, policy_version 1807792 (0.0009) [2023-12-27 04:33:51,591][105620] Updated weights for policy 1, policy_version 1811497 (0.0008) [2023-12-27 04:33:51,663][105620] Updated weights for policy 1, policy_version 1811507 (0.0008) [2023-12-27 04:33:51,737][105620] Updated weights for policy 1, policy_version 1811517 (0.0010) [2023-12-27 04:33:51,795][105620] Updated weights for policy 1, policy_version 1811527 (0.0009) [2023-12-27 04:33:52,157][105692] Updated weights for policy 0, policy_version 1807802 (0.0009) [2023-12-27 04:33:52,212][105692] Updated weights for policy 0, policy_version 1807812 (0.0009) [2023-12-27 04:33:52,269][105692] Updated weights for policy 0, policy_version 1807822 (0.0008) [2023-12-27 04:33:52,536][105620] Updated weights for policy 1, policy_version 1811537 (0.0009) [2023-12-27 04:33:52,584][105620] Updated weights for policy 1, policy_version 1811547 (0.0009) [2023-12-27 04:33:52,638][105620] Updated weights for policy 1, policy_version 1811557 (0.0009) [2023-12-27 04:33:53,034][105692] Updated weights for policy 0, policy_version 1807832 (0.0009) [2023-12-27 04:33:53,091][105692] Updated weights for policy 0, policy_version 1807842 (0.0009) [2023-12-27 04:33:53,144][105692] Updated weights for policy 0, policy_version 1807852 (0.0008) [2023-12-27 04:33:53,400][105620] Updated weights for policy 1, policy_version 1811567 (0.0009) [2023-12-27 04:33:53,455][105620] Updated weights for policy 1, policy_version 1811577 (0.0009) [2023-12-27 04:33:53,510][105620] Updated weights for policy 1, policy_version 1811587 (0.0009) [2023-12-27 04:33:53,879][105692] Updated weights for policy 0, policy_version 1807862 (0.0009) [2023-12-27 04:33:53,940][105692] Updated weights for policy 0, policy_version 1807872 (0.0009) [2023-12-27 04:33:54,002][105692] Updated weights for policy 0, policy_version 1807882 (0.0009) [2023-12-27 04:33:54,264][105620] Updated weights for policy 1, policy_version 1811597 (0.0009) [2023-12-27 04:33:54,313][105620] Updated weights for policy 1, policy_version 1811607 (0.0009) [2023-12-27 04:33:54,363][105620] Updated weights for policy 1, policy_version 1811617 (0.0009) [2023-12-27 04:33:54,740][105692] Updated weights for policy 0, policy_version 1807892 (0.0008) [2023-12-27 04:33:54,786][105692] Updated weights for policy 0, policy_version 1807902 (0.0009) [2023-12-27 04:33:54,851][105692] Updated weights for policy 0, policy_version 1807912 (0.0009) [2023-12-27 04:33:55,137][105620] Updated weights for policy 1, policy_version 1811627 (0.0009) [2023-12-27 04:33:55,198][105620] Updated weights for policy 1, policy_version 1811637 (0.0009) [2023-12-27 04:33:55,252][105620] Updated weights for policy 1, policy_version 1811647 (0.0009) [2023-12-27 04:33:55,616][105692] Updated weights for policy 0, policy_version 1807922 (0.0009) [2023-12-27 04:33:55,673][105692] Updated weights for policy 0, policy_version 1807932 (0.0009) [2023-12-27 04:33:55,734][105692] Updated weights for policy 0, policy_version 1807942 (0.0009) [2023-12-27 04:33:55,795][105692] Updated weights for policy 0, policy_version 1807952 (0.0009) [2023-12-27 04:33:56,003][105620] Updated weights for policy 1, policy_version 1811657 (0.0009) [2023-12-27 04:33:56,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 926752768. Throughput: 0: 9800.4, 1: 9574.3. Samples: 926763940. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:33:56,063][104569] Avg episode reward: [(0, '8267.926'), (1, '9168.818')] [2023-12-27 04:33:56,066][105620] Updated weights for policy 1, policy_version 1811667 (0.0008) [2023-12-27 04:33:56,127][105620] Updated weights for policy 1, policy_version 1811677 (0.0008) [2023-12-27 04:33:56,190][105620] Updated weights for policy 1, policy_version 1811687 (0.0007) [2023-12-27 04:33:56,582][105692] Updated weights for policy 0, policy_version 1807962 (0.0009) [2023-12-27 04:33:56,627][105692] Updated weights for policy 0, policy_version 1807972 (0.0009) [2023-12-27 04:33:56,677][105692] Updated weights for policy 0, policy_version 1807982 (0.0008) [2023-12-27 04:33:56,797][105620] Updated weights for policy 1, policy_version 1811697 (0.0008) [2023-12-27 04:33:56,847][105620] Updated weights for policy 1, policy_version 1811707 (0.0008) [2023-12-27 04:33:56,893][105620] Updated weights for policy 1, policy_version 1811717 (0.0009) [2023-12-27 04:33:57,436][105692] Updated weights for policy 0, policy_version 1807992 (0.0009) [2023-12-27 04:33:57,493][105692] Updated weights for policy 0, policy_version 1808002 (0.0009) [2023-12-27 04:33:57,539][105692] Updated weights for policy 0, policy_version 1808012 (0.0008) [2023-12-27 04:33:57,667][105620] Updated weights for policy 1, policy_version 1811727 (0.0009) [2023-12-27 04:33:57,722][105620] Updated weights for policy 1, policy_version 1811737 (0.0009) [2023-12-27 04:33:57,769][105620] Updated weights for policy 1, policy_version 1811747 (0.0008) [2023-12-27 04:33:58,230][105692] Updated weights for policy 0, policy_version 1808022 (0.0009) [2023-12-27 04:33:58,297][105692] Updated weights for policy 0, policy_version 1808032 (0.0008) [2023-12-27 04:33:58,361][105692] Updated weights for policy 0, policy_version 1808042 (0.0009) [2023-12-27 04:33:58,468][105620] Updated weights for policy 1, policy_version 1811757 (0.0008) [2023-12-27 04:33:58,534][105620] Updated weights for policy 1, policy_version 1811767 (0.0008) [2023-12-27 04:33:58,595][105620] Updated weights for policy 1, policy_version 1811777 (0.0008) [2023-12-27 04:33:59,161][105692] Updated weights for policy 0, policy_version 1808052 (0.0009) [2023-12-27 04:33:59,234][105692] Updated weights for policy 0, policy_version 1808062 (0.0009) [2023-12-27 04:33:59,299][105692] Updated weights for policy 0, policy_version 1808072 (0.0009) [2023-12-27 04:33:59,340][105620] Updated weights for policy 1, policy_version 1811787 (0.0008) [2023-12-27 04:33:59,408][105620] Updated weights for policy 1, policy_version 1811797 (0.0008) [2023-12-27 04:33:59,456][105620] Updated weights for policy 1, policy_version 1811807 (0.0009) [2023-12-27 04:34:00,040][105692] Updated weights for policy 0, policy_version 1808082 (0.0008) [2023-12-27 04:34:00,094][105692] Updated weights for policy 0, policy_version 1808092 (0.0008) [2023-12-27 04:34:00,144][105692] Updated weights for policy 0, policy_version 1808102 (0.0008) [2023-12-27 04:34:00,191][105692] Updated weights for policy 0, policy_version 1808112 (0.0008) [2023-12-27 04:34:00,251][105620] Updated weights for policy 1, policy_version 1811817 (0.0009) [2023-12-27 04:34:00,299][105620] Updated weights for policy 1, policy_version 1811827 (0.0006) [2023-12-27 04:34:00,353][105620] Updated weights for policy 1, policy_version 1811837 (0.0005) [2023-12-27 04:34:00,411][105620] Updated weights for policy 1, policy_version 1811847 (0.0005) [2023-12-27 04:34:00,951][105692] Updated weights for policy 0, policy_version 1808122 (0.0010) [2023-12-27 04:34:00,997][105620] Updated weights for policy 1, policy_version 1811857 (0.0005) [2023-12-27 04:34:01,013][105692] Updated weights for policy 0, policy_version 1808132 (0.0010) [2023-12-27 04:34:01,056][105620] Updated weights for policy 1, policy_version 1811867 (0.0007) [2023-12-27 04:34:01,062][104569] Fps is (10 sec: 18022.1, 60 sec: 19387.8, 300 sec: 19521.9). Total num frames: 926842880. Throughput: 0: 9756.4, 1: 9606.5. Samples: 926821244. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:34:01,063][104569] Avg episode reward: [(0, '8357.164'), (1, '9168.791')] [2023-12-27 04:34:01,074][105692] Updated weights for policy 0, policy_version 1808142 (0.0011) [2023-12-27 04:34:01,083][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001808144_462954496.pth... [2023-12-27 04:34:01,088][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001806992_462659584.pth [2023-12-27 04:34:01,116][105620] Updated weights for policy 1, policy_version 1811877 (0.0006) [2023-12-27 04:34:01,134][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001811880_463904768.pth... [2023-12-27 04:34:01,139][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001810760_463618048.pth [2023-12-27 04:34:01,736][105620] Updated weights for policy 1, policy_version 1811887 (0.0008) [2023-12-27 04:34:01,759][105692] Updated weights for policy 0, policy_version 1808152 (0.0009) [2023-12-27 04:34:01,805][105620] Updated weights for policy 1, policy_version 1811897 (0.0006) [2023-12-27 04:34:01,821][105692] Updated weights for policy 0, policy_version 1808162 (0.0007) [2023-12-27 04:34:01,873][105620] Updated weights for policy 1, policy_version 1811907 (0.0008) [2023-12-27 04:34:01,874][105692] Updated weights for policy 0, policy_version 1808172 (0.0007) [2023-12-27 04:34:02,567][105620] Updated weights for policy 1, policy_version 1811917 (0.0008) [2023-12-27 04:34:02,590][105692] Updated weights for policy 0, policy_version 1808182 (0.0007) [2023-12-27 04:34:02,625][105620] Updated weights for policy 1, policy_version 1811927 (0.0010) [2023-12-27 04:34:02,640][105692] Updated weights for policy 0, policy_version 1808192 (0.0006) [2023-12-27 04:34:02,678][105620] Updated weights for policy 1, policy_version 1811937 (0.0011) [2023-12-27 04:34:02,692][105692] Updated weights for policy 0, policy_version 1808202 (0.0005) [2023-12-27 04:34:03,444][105620] Updated weights for policy 1, policy_version 1811947 (0.0009) [2023-12-27 04:34:03,471][105692] Updated weights for policy 0, policy_version 1808212 (0.0007) [2023-12-27 04:34:03,500][105620] Updated weights for policy 1, policy_version 1811957 (0.0009) [2023-12-27 04:34:03,533][105692] Updated weights for policy 0, policy_version 1808222 (0.0005) [2023-12-27 04:34:03,546][105620] Updated weights for policy 1, policy_version 1811967 (0.0008) [2023-12-27 04:34:03,590][105692] Updated weights for policy 0, policy_version 1808232 (0.0005) [2023-12-27 04:34:04,304][105620] Updated weights for policy 1, policy_version 1811977 (0.0008) [2023-12-27 04:34:04,308][105692] Updated weights for policy 0, policy_version 1808242 (0.0009) [2023-12-27 04:34:04,362][105692] Updated weights for policy 0, policy_version 1808252 (0.0007) [2023-12-27 04:34:04,363][105620] Updated weights for policy 1, policy_version 1811987 (0.0008) [2023-12-27 04:34:04,419][105692] Updated weights for policy 0, policy_version 1808262 (0.0009) [2023-12-27 04:34:04,426][105620] Updated weights for policy 1, policy_version 1811997 (0.0009) [2023-12-27 04:34:04,477][105692] Updated weights for policy 0, policy_version 1808272 (0.0009) [2023-12-27 04:34:04,488][105620] Updated weights for policy 1, policy_version 1812007 (0.0009) [2023-12-27 04:34:05,155][105692] Updated weights for policy 0, policy_version 1808282 (0.0005) [2023-12-27 04:34:05,213][105692] Updated weights for policy 0, policy_version 1808292 (0.0007) [2023-12-27 04:34:05,266][105692] Updated weights for policy 0, policy_version 1808302 (0.0008) [2023-12-27 04:34:05,300][105620] Updated weights for policy 1, policy_version 1812017 (0.0008) [2023-12-27 04:34:05,357][105620] Updated weights for policy 1, policy_version 1812027 (0.0008) [2023-12-27 04:34:05,418][105620] Updated weights for policy 1, policy_version 1812037 (0.0009) [2023-12-27 04:34:05,872][105692] Updated weights for policy 0, policy_version 1808312 (0.0006) [2023-12-27 04:34:05,921][105692] Updated weights for policy 0, policy_version 1808322 (0.0005) [2023-12-27 04:34:05,972][105692] Updated weights for policy 0, policy_version 1808332 (0.0005) [2023-12-27 04:34:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 926949376. Throughput: 0: 9670.9, 1: 9624.1. Samples: 926936376. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:34:06,062][104569] Avg episode reward: [(0, '8713.873'), (1, '9350.945')] [2023-12-27 04:34:06,300][105620] Updated weights for policy 1, policy_version 1812047 (0.0009) [2023-12-27 04:34:06,351][105620] Updated weights for policy 1, policy_version 1812057 (0.0008) [2023-12-27 04:34:06,399][105620] Updated weights for policy 1, policy_version 1812067 (0.0009) [2023-12-27 04:34:06,628][105692] Updated weights for policy 0, policy_version 1808342 (0.0007) [2023-12-27 04:34:06,685][105692] Updated weights for policy 0, policy_version 1808352 (0.0009) [2023-12-27 04:34:06,753][105692] Updated weights for policy 0, policy_version 1808362 (0.0009) [2023-12-27 04:34:07,187][105620] Updated weights for policy 1, policy_version 1812077 (0.0009) [2023-12-27 04:34:07,242][105620] Updated weights for policy 1, policy_version 1812087 (0.0009) [2023-12-27 04:34:07,301][105620] Updated weights for policy 1, policy_version 1812097 (0.0010) [2023-12-27 04:34:07,525][105692] Updated weights for policy 0, policy_version 1808372 (0.0008) [2023-12-27 04:34:07,588][105692] Updated weights for policy 0, policy_version 1808382 (0.0009) [2023-12-27 04:34:07,655][105692] Updated weights for policy 0, policy_version 1808392 (0.0009) [2023-12-27 04:34:07,973][105620] Updated weights for policy 1, policy_version 1812107 (0.0007) [2023-12-27 04:34:08,025][105620] Updated weights for policy 1, policy_version 1812117 (0.0006) [2023-12-27 04:34:08,080][105620] Updated weights for policy 1, policy_version 1812127 (0.0010) [2023-12-27 04:34:08,449][105692] Updated weights for policy 0, policy_version 1808402 (0.0008) [2023-12-27 04:34:08,502][105692] Updated weights for policy 0, policy_version 1808412 (0.0008) [2023-12-27 04:34:08,548][105692] Updated weights for policy 0, policy_version 1808422 (0.0008) [2023-12-27 04:34:08,597][105692] Updated weights for policy 0, policy_version 1808432 (0.0008) [2023-12-27 04:34:08,803][105620] Updated weights for policy 1, policy_version 1812137 (0.0010) [2023-12-27 04:34:08,853][105620] Updated weights for policy 1, policy_version 1812147 (0.0009) [2023-12-27 04:34:08,909][105620] Updated weights for policy 1, policy_version 1812157 (0.0010) [2023-12-27 04:34:08,958][105620] Updated weights for policy 1, policy_version 1812167 (0.0010) [2023-12-27 04:34:09,388][105692] Updated weights for policy 0, policy_version 1808442 (0.0008) [2023-12-27 04:34:09,458][105692] Updated weights for policy 0, policy_version 1808452 (0.0009) [2023-12-27 04:34:09,515][105692] Updated weights for policy 0, policy_version 1808462 (0.0008) [2023-12-27 04:34:09,710][105620] Updated weights for policy 1, policy_version 1812177 (0.0007) [2023-12-27 04:34:09,770][105620] Updated weights for policy 1, policy_version 1812187 (0.0011) [2023-12-27 04:34:09,834][105620] Updated weights for policy 1, policy_version 1812197 (0.0011) [2023-12-27 04:34:10,412][105692] Updated weights for policy 0, policy_version 1808472 (0.0008) [2023-12-27 04:34:10,466][105620] Updated weights for policy 1, policy_version 1812207 (0.0011) [2023-12-27 04:34:10,473][105692] Updated weights for policy 0, policy_version 1808482 (0.0006) [2023-12-27 04:34:10,524][105620] Updated weights for policy 1, policy_version 1812217 (0.0009) [2023-12-27 04:34:10,524][105692] Updated weights for policy 0, policy_version 1808492 (0.0008) [2023-12-27 04:34:10,585][105620] Updated weights for policy 1, policy_version 1812227 (0.0006) [2023-12-27 04:34:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.3, 300 sec: 19549.7). Total num frames: 927039488. Throughput: 0: 9715.9, 1: 9589.1. Samples: 927050524. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:34:11,062][104569] Avg episode reward: [(0, '8804.805'), (1, '9258.660')] [2023-12-27 04:34:11,295][105620] Updated weights for policy 1, policy_version 1812237 (0.0008) [2023-12-27 04:34:11,351][105692] Updated weights for policy 0, policy_version 1808502 (0.0008) [2023-12-27 04:34:11,357][105620] Updated weights for policy 1, policy_version 1812247 (0.0009) [2023-12-27 04:34:11,418][105692] Updated weights for policy 0, policy_version 1808512 (0.0009) [2023-12-27 04:34:11,421][105620] Updated weights for policy 1, policy_version 1812257 (0.0011) [2023-12-27 04:34:11,469][105692] Updated weights for policy 0, policy_version 1808522 (0.0006) [2023-12-27 04:34:12,190][105692] Updated weights for policy 0, policy_version 1808532 (0.0008) [2023-12-27 04:34:12,209][105620] Updated weights for policy 1, policy_version 1812267 (0.0010) [2023-12-27 04:34:12,244][105692] Updated weights for policy 0, policy_version 1808542 (0.0009) [2023-12-27 04:34:12,274][105620] Updated weights for policy 1, policy_version 1812277 (0.0008) [2023-12-27 04:34:12,310][105692] Updated weights for policy 0, policy_version 1808552 (0.0008) [2023-12-27 04:34:12,335][105620] Updated weights for policy 1, policy_version 1812287 (0.0007) [2023-12-27 04:34:12,967][105620] Updated weights for policy 1, policy_version 1812297 (0.0008) [2023-12-27 04:34:13,022][105620] Updated weights for policy 1, policy_version 1812307 (0.0006) [2023-12-27 04:34:13,080][105620] Updated weights for policy 1, policy_version 1812317 (0.0009) [2023-12-27 04:34:13,119][105692] Updated weights for policy 0, policy_version 1808562 (0.0007) [2023-12-27 04:34:13,129][105620] Updated weights for policy 1, policy_version 1812327 (0.0010) [2023-12-27 04:34:13,178][105692] Updated weights for policy 0, policy_version 1808572 (0.0009) [2023-12-27 04:34:13,240][105692] Updated weights for policy 0, policy_version 1808583 (0.0011) [2023-12-27 04:34:13,753][105620] Updated weights for policy 1, policy_version 1812337 (0.0007) [2023-12-27 04:34:13,805][105620] Updated weights for policy 1, policy_version 1812347 (0.0005) [2023-12-27 04:34:13,860][105620] Updated weights for policy 1, policy_version 1812357 (0.0005) [2023-12-27 04:34:13,874][105692] Updated weights for policy 0, policy_version 1808593 (0.0009) [2023-12-27 04:34:13,926][105692] Updated weights for policy 0, policy_version 1808603 (0.0006) [2023-12-27 04:34:13,979][105692] Updated weights for policy 0, policy_version 1808614 (0.0009) [2023-12-27 04:34:14,035][105692] Updated weights for policy 0, policy_version 1808624 (0.0008) [2023-12-27 04:34:14,484][105620] Updated weights for policy 1, policy_version 1812367 (0.0009) [2023-12-27 04:34:14,532][105620] Updated weights for policy 1, policy_version 1812377 (0.0010) [2023-12-27 04:34:14,588][105620] Updated weights for policy 1, policy_version 1812387 (0.0010) [2023-12-27 04:34:14,680][105692] Updated weights for policy 0, policy_version 1808634 (0.0010) [2023-12-27 04:34:14,738][105692] Updated weights for policy 0, policy_version 1808644 (0.0010) [2023-12-27 04:34:14,802][105692] Updated weights for policy 0, policy_version 1808654 (0.0011) [2023-12-27 04:34:15,327][105620] Updated weights for policy 1, policy_version 1812397 (0.0009) [2023-12-27 04:34:15,383][105620] Updated weights for policy 1, policy_version 1812407 (0.0011) [2023-12-27 04:34:15,445][105620] Updated weights for policy 1, policy_version 1812417 (0.0010) [2023-12-27 04:34:15,568][105692] Updated weights for policy 0, policy_version 1808664 (0.0011) [2023-12-27 04:34:15,624][105692] Updated weights for policy 0, policy_version 1808674 (0.0011) [2023-12-27 04:34:15,673][105692] Updated weights for policy 0, policy_version 1808684 (0.0010) [2023-12-27 04:34:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 927137792. Throughput: 0: 9701.4, 1: 9548.3. Samples: 927108552. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:34:16,062][104569] Avg episode reward: [(0, '8898.430'), (1, '9166.760')] [2023-12-27 04:34:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001812424_464044032.pth... [2023-12-27 04:34:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001808688_463093760.pth... [2023-12-27 04:34:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001811336_463765504.pth [2023-12-27 04:34:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001807568_462807040.pth [2023-12-27 04:34:16,159][105620] Updated weights for policy 1, policy_version 1812427 (0.0010) [2023-12-27 04:34:16,216][105620] Updated weights for policy 1, policy_version 1812437 (0.0010) [2023-12-27 04:34:16,271][105620] Updated weights for policy 1, policy_version 1812447 (0.0010) [2023-12-27 04:34:16,438][105692] Updated weights for policy 0, policy_version 1808694 (0.0010) [2023-12-27 04:34:16,496][105692] Updated weights for policy 0, policy_version 1808704 (0.0007) [2023-12-27 04:34:16,546][105692] Updated weights for policy 0, policy_version 1808714 (0.0005) [2023-12-27 04:34:17,006][105620] Updated weights for policy 1, policy_version 1812457 (0.0010) [2023-12-27 04:34:17,063][105620] Updated weights for policy 1, policy_version 1812467 (0.0007) [2023-12-27 04:34:17,112][105620] Updated weights for policy 1, policy_version 1812477 (0.0005) [2023-12-27 04:34:17,159][105620] Updated weights for policy 1, policy_version 1812487 (0.0005) [2023-12-27 04:34:17,198][105692] Updated weights for policy 0, policy_version 1808724 (0.0006) [2023-12-27 04:34:17,263][105692] Updated weights for policy 0, policy_version 1808734 (0.0005) [2023-12-27 04:34:17,329][105692] Updated weights for policy 0, policy_version 1808744 (0.0008) [2023-12-27 04:34:17,852][105620] Updated weights for policy 1, policy_version 1812497 (0.0008) [2023-12-27 04:34:17,897][105620] Updated weights for policy 1, policy_version 1812507 (0.0008) [2023-12-27 04:34:17,941][105620] Updated weights for policy 1, policy_version 1812517 (0.0008) [2023-12-27 04:34:17,959][105692] Updated weights for policy 0, policy_version 1808754 (0.0010) [2023-12-27 04:34:18,008][105692] Updated weights for policy 0, policy_version 1808764 (0.0011) [2023-12-27 04:34:18,060][105692] Updated weights for policy 0, policy_version 1808774 (0.0010) [2023-12-27 04:34:18,116][105692] Updated weights for policy 0, policy_version 1808784 (0.0008) [2023-12-27 04:34:18,686][105620] Updated weights for policy 1, policy_version 1812527 (0.0008) [2023-12-27 04:34:18,701][105692] Updated weights for policy 0, policy_version 1808794 (0.0011) [2023-12-27 04:34:18,744][105620] Updated weights for policy 1, policy_version 1812537 (0.0006) [2023-12-27 04:34:18,761][105692] Updated weights for policy 0, policy_version 1808804 (0.0008) [2023-12-27 04:34:18,807][105620] Updated weights for policy 1, policy_version 1812547 (0.0010) [2023-12-27 04:34:18,856][105692] Updated weights for policy 0, policy_version 1808814 (0.0011) [2023-12-27 04:34:19,557][105620] Updated weights for policy 1, policy_version 1812557 (0.0008) [2023-12-27 04:34:19,588][105692] Updated weights for policy 0, policy_version 1808824 (0.0007) [2023-12-27 04:34:19,620][105620] Updated weights for policy 1, policy_version 1812567 (0.0008) [2023-12-27 04:34:19,647][105692] Updated weights for policy 0, policy_version 1808834 (0.0007) [2023-12-27 04:34:19,678][105620] Updated weights for policy 1, policy_version 1812577 (0.0008) [2023-12-27 04:34:19,700][105692] Updated weights for policy 0, policy_version 1808844 (0.0006) [2023-12-27 04:34:20,456][105620] Updated weights for policy 1, policy_version 1812587 (0.0010) [2023-12-27 04:34:20,461][105692] Updated weights for policy 0, policy_version 1808854 (0.0006) [2023-12-27 04:34:20,511][105692] Updated weights for policy 0, policy_version 1808864 (0.0005) [2023-12-27 04:34:20,520][105620] Updated weights for policy 1, policy_version 1812597 (0.0011) [2023-12-27 04:34:20,564][105692] Updated weights for policy 0, policy_version 1808874 (0.0005) [2023-12-27 04:34:20,582][105620] Updated weights for policy 1, policy_version 1812607 (0.0010) [2023-12-27 04:34:21,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 927236096. Throughput: 0: 9651.4, 1: 9564.9. Samples: 927227420. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:34:21,063][104569] Avg episode reward: [(0, '8714.082'), (1, '9259.050')] [2023-12-27 04:34:21,287][105692] Updated weights for policy 0, policy_version 1808884 (0.0008) [2023-12-27 04:34:21,362][105692] Updated weights for policy 0, policy_version 1808894 (0.0009) [2023-12-27 04:34:21,377][105620] Updated weights for policy 1, policy_version 1812617 (0.0011) [2023-12-27 04:34:21,422][105692] Updated weights for policy 0, policy_version 1808904 (0.0006) [2023-12-27 04:34:21,437][105620] Updated weights for policy 1, policy_version 1812627 (0.0011) [2023-12-27 04:34:21,493][105620] Updated weights for policy 1, policy_version 1812637 (0.0011) [2023-12-27 04:34:21,542][105620] Updated weights for policy 1, policy_version 1812647 (0.0010) [2023-12-27 04:34:22,115][105692] Updated weights for policy 0, policy_version 1808914 (0.0006) [2023-12-27 04:34:22,162][105692] Updated weights for policy 0, policy_version 1808924 (0.0008) [2023-12-27 04:34:22,213][105692] Updated weights for policy 0, policy_version 1808934 (0.0008) [2023-12-27 04:34:22,271][105692] Updated weights for policy 0, policy_version 1808944 (0.0009) [2023-12-27 04:34:22,364][105620] Updated weights for policy 1, policy_version 1812657 (0.0008) [2023-12-27 04:34:22,417][105620] Updated weights for policy 1, policy_version 1812667 (0.0010) [2023-12-27 04:34:22,477][105620] Updated weights for policy 1, policy_version 1812677 (0.0010) [2023-12-27 04:34:23,070][105692] Updated weights for policy 0, policy_version 1808954 (0.0008) [2023-12-27 04:34:23,126][105692] Updated weights for policy 0, policy_version 1808964 (0.0008) [2023-12-27 04:34:23,180][105692] Updated weights for policy 0, policy_version 1808974 (0.0009) [2023-12-27 04:34:23,232][105620] Updated weights for policy 1, policy_version 1812687 (0.0010) [2023-12-27 04:34:23,289][105620] Updated weights for policy 1, policy_version 1812697 (0.0011) [2023-12-27 04:34:23,347][105620] Updated weights for policy 1, policy_version 1812707 (0.0010) [2023-12-27 04:34:23,931][105620] Updated weights for policy 1, policy_version 1812717 (0.0010) [2023-12-27 04:34:23,978][105692] Updated weights for policy 0, policy_version 1808984 (0.0006) [2023-12-27 04:34:23,987][105620] Updated weights for policy 1, policy_version 1812727 (0.0008) [2023-12-27 04:34:24,032][105692] Updated weights for policy 0, policy_version 1808994 (0.0006) [2023-12-27 04:34:24,048][105620] Updated weights for policy 1, policy_version 1812737 (0.0008) [2023-12-27 04:34:24,080][105692] Updated weights for policy 0, policy_version 1809004 (0.0008) [2023-12-27 04:34:24,780][105620] Updated weights for policy 1, policy_version 1812747 (0.0007) [2023-12-27 04:34:24,829][105692] Updated weights for policy 0, policy_version 1809014 (0.0007) [2023-12-27 04:34:24,838][105620] Updated weights for policy 1, policy_version 1812757 (0.0008) [2023-12-27 04:34:24,875][105692] Updated weights for policy 0, policy_version 1809024 (0.0005) [2023-12-27 04:34:24,894][105620] Updated weights for policy 1, policy_version 1812767 (0.0008) [2023-12-27 04:34:24,927][105692] Updated weights for policy 0, policy_version 1809034 (0.0005) [2023-12-27 04:34:25,588][105692] Updated weights for policy 0, policy_version 1809044 (0.0005) [2023-12-27 04:34:25,605][105620] Updated weights for policy 1, policy_version 1812777 (0.0008) [2023-12-27 04:34:25,637][105692] Updated weights for policy 0, policy_version 1809054 (0.0005) [2023-12-27 04:34:25,662][105620] Updated weights for policy 1, policy_version 1812787 (0.0009) [2023-12-27 04:34:25,691][105692] Updated weights for policy 0, policy_version 1809064 (0.0005) [2023-12-27 04:34:25,723][105620] Updated weights for policy 1, policy_version 1812797 (0.0009) [2023-12-27 04:34:25,785][105620] Updated weights for policy 1, policy_version 1812807 (0.0010) [2023-12-27 04:34:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 927334400. Throughput: 0: 9635.5, 1: 9517.5. Samples: 927341124. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:34:26,063][104569] Avg episode reward: [(0, '8168.837'), (1, '9350.997')] [2023-12-27 04:34:26,281][105692] Updated weights for policy 0, policy_version 1809074 (0.0005) [2023-12-27 04:34:26,343][105692] Updated weights for policy 0, policy_version 1809084 (0.0007) [2023-12-27 04:34:26,397][105692] Updated weights for policy 0, policy_version 1809094 (0.0010) [2023-12-27 04:34:26,458][105692] Updated weights for policy 0, policy_version 1809104 (0.0010) [2023-12-27 04:34:26,512][105620] Updated weights for policy 1, policy_version 1812817 (0.0006) [2023-12-27 04:34:26,555][105620] Updated weights for policy 1, policy_version 1812827 (0.0005) [2023-12-27 04:34:26,601][105620] Updated weights for policy 1, policy_version 1812837 (0.0005) [2023-12-27 04:34:27,174][105692] Updated weights for policy 0, policy_version 1809114 (0.0005) [2023-12-27 04:34:27,208][105620] Updated weights for policy 1, policy_version 1812847 (0.0006) [2023-12-27 04:34:27,241][105692] Updated weights for policy 0, policy_version 1809124 (0.0005) [2023-12-27 04:34:27,262][105620] Updated weights for policy 1, policy_version 1812857 (0.0009) [2023-12-27 04:34:27,312][105692] Updated weights for policy 0, policy_version 1809134 (0.0007) [2023-12-27 04:34:27,324][105620] Updated weights for policy 1, policy_version 1812867 (0.0007) [2023-12-27 04:34:27,983][105692] Updated weights for policy 0, policy_version 1809144 (0.0010) [2023-12-27 04:34:28,035][105692] Updated weights for policy 0, policy_version 1809154 (0.0010) [2023-12-27 04:34:28,083][105620] Updated weights for policy 1, policy_version 1812877 (0.0008) [2023-12-27 04:34:28,090][105692] Updated weights for policy 0, policy_version 1809164 (0.0011) [2023-12-27 04:34:28,146][105620] Updated weights for policy 1, policy_version 1812887 (0.0006) [2023-12-27 04:34:28,210][105620] Updated weights for policy 1, policy_version 1812897 (0.0005) [2023-12-27 04:34:28,783][105692] Updated weights for policy 0, policy_version 1809174 (0.0008) [2023-12-27 04:34:28,837][105692] Updated weights for policy 0, policy_version 1809184 (0.0008) [2023-12-27 04:34:28,897][105692] Updated weights for policy 0, policy_version 1809194 (0.0009) [2023-12-27 04:34:28,901][105620] Updated weights for policy 1, policy_version 1812907 (0.0007) [2023-12-27 04:34:28,947][105620] Updated weights for policy 1, policy_version 1812917 (0.0008) [2023-12-27 04:34:28,993][105620] Updated weights for policy 1, policy_version 1812927 (0.0008) [2023-12-27 04:34:29,580][105692] Updated weights for policy 0, policy_version 1809204 (0.0007) [2023-12-27 04:34:29,641][105692] Updated weights for policy 0, policy_version 1809214 (0.0008) [2023-12-27 04:34:29,707][105692] Updated weights for policy 0, policy_version 1809224 (0.0005) [2023-12-27 04:34:29,807][105620] Updated weights for policy 1, policy_version 1812937 (0.0008) [2023-12-27 04:34:29,877][105620] Updated weights for policy 1, policy_version 1812947 (0.0008) [2023-12-27 04:34:29,944][105620] Updated weights for policy 1, policy_version 1812957 (0.0009) [2023-12-27 04:34:30,004][105620] Updated weights for policy 1, policy_version 1812967 (0.0009) [2023-12-27 04:34:30,258][105692] Updated weights for policy 0, policy_version 1809234 (0.0006) [2023-12-27 04:34:30,316][105692] Updated weights for policy 0, policy_version 1809244 (0.0006) [2023-12-27 04:34:30,370][105692] Updated weights for policy 0, policy_version 1809254 (0.0007) [2023-12-27 04:34:30,423][105692] Updated weights for policy 0, policy_version 1809264 (0.0006) [2023-12-27 04:34:30,783][105620] Updated weights for policy 1, policy_version 1812977 (0.0010) [2023-12-27 04:34:30,839][105620] Updated weights for policy 1, policy_version 1812987 (0.0005) [2023-12-27 04:34:30,897][105620] Updated weights for policy 1, policy_version 1812997 (0.0005) [2023-12-27 04:34:31,051][105692] Updated weights for policy 0, policy_version 1809274 (0.0009) [2023-12-27 04:34:31,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 927432704. Throughput: 0: 9673.6, 1: 9560.2. Samples: 927402208. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:34:31,063][104569] Avg episode reward: [(0, '8351.377'), (1, '9350.973')] [2023-12-27 04:34:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001813000_464191488.pth... [2023-12-27 04:34:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001811880_463904768.pth [2023-12-27 04:34:31,107][105692] Updated weights for policy 0, policy_version 1809284 (0.0010) [2023-12-27 04:34:31,169][105692] Updated weights for policy 0, policy_version 1809294 (0.0010) [2023-12-27 04:34:31,179][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001809296_463249408.pth... [2023-12-27 04:34:31,182][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001808144_462954496.pth [2023-12-27 04:34:31,619][105620] Updated weights for policy 1, policy_version 1813007 (0.0009) [2023-12-27 04:34:31,680][105620] Updated weights for policy 1, policy_version 1813017 (0.0006) [2023-12-27 04:34:31,747][105620] Updated weights for policy 1, policy_version 1813027 (0.0007) [2023-12-27 04:34:31,896][105692] Updated weights for policy 0, policy_version 1809304 (0.0007) [2023-12-27 04:34:31,948][105692] Updated weights for policy 0, policy_version 1809314 (0.0005) [2023-12-27 04:34:32,002][105692] Updated weights for policy 0, policy_version 1809324 (0.0005) [2023-12-27 04:34:32,369][105620] Updated weights for policy 1, policy_version 1813037 (0.0008) [2023-12-27 04:34:32,427][105620] Updated weights for policy 1, policy_version 1813047 (0.0005) [2023-12-27 04:34:32,489][105620] Updated weights for policy 1, policy_version 1813057 (0.0006) [2023-12-27 04:34:32,700][105692] Updated weights for policy 0, policy_version 1809334 (0.0005) [2023-12-27 04:34:32,746][105692] Updated weights for policy 0, policy_version 1809344 (0.0005) [2023-12-27 04:34:32,793][105692] Updated weights for policy 0, policy_version 1809354 (0.0005) [2023-12-27 04:34:33,069][105620] Updated weights for policy 1, policy_version 1813067 (0.0007) [2023-12-27 04:34:33,120][105620] Updated weights for policy 1, policy_version 1813077 (0.0005) [2023-12-27 04:34:33,183][105620] Updated weights for policy 1, policy_version 1813087 (0.0005) [2023-12-27 04:34:33,402][105692] Updated weights for policy 0, policy_version 1809364 (0.0005) [2023-12-27 04:34:33,463][105692] Updated weights for policy 0, policy_version 1809374 (0.0005) [2023-12-27 04:34:33,514][105692] Updated weights for policy 0, policy_version 1809384 (0.0005) [2023-12-27 04:34:33,865][105620] Updated weights for policy 1, policy_version 1813097 (0.0006) [2023-12-27 04:34:33,913][105620] Updated weights for policy 1, policy_version 1813107 (0.0008) [2023-12-27 04:34:33,967][105620] Updated weights for policy 1, policy_version 1813117 (0.0008) [2023-12-27 04:34:34,026][105620] Updated weights for policy 1, policy_version 1813127 (0.0008) [2023-12-27 04:34:34,083][105692] Updated weights for policy 0, policy_version 1809394 (0.0006) [2023-12-27 04:34:34,146][105692] Updated weights for policy 0, policy_version 1809404 (0.0010) [2023-12-27 04:34:34,214][105692] Updated weights for policy 0, policy_version 1809414 (0.0010) [2023-12-27 04:34:34,275][105692] Updated weights for policy 0, policy_version 1809424 (0.0011) [2023-12-27 04:34:34,798][105620] Updated weights for policy 1, policy_version 1813137 (0.0006) [2023-12-27 04:34:34,857][105620] Updated weights for policy 1, policy_version 1813147 (0.0008) [2023-12-27 04:34:34,909][105620] Updated weights for policy 1, policy_version 1813157 (0.0008) [2023-12-27 04:34:35,010][105692] Updated weights for policy 0, policy_version 1809434 (0.0010) [2023-12-27 04:34:35,071][105692] Updated weights for policy 0, policy_version 1809444 (0.0010) [2023-12-27 04:34:35,129][105692] Updated weights for policy 0, policy_version 1809454 (0.0010) [2023-12-27 04:34:35,590][105620] Updated weights for policy 1, policy_version 1813167 (0.0008) [2023-12-27 04:34:35,649][105620] Updated weights for policy 1, policy_version 1813177 (0.0008) [2023-12-27 04:34:35,703][105620] Updated weights for policy 1, policy_version 1813187 (0.0008) [2023-12-27 04:34:35,874][105692] Updated weights for policy 0, policy_version 1809464 (0.0010) [2023-12-27 04:34:35,931][105692] Updated weights for policy 0, policy_version 1809474 (0.0010) [2023-12-27 04:34:35,992][105692] Updated weights for policy 0, policy_version 1809484 (0.0010) [2023-12-27 04:34:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 927539200. Throughput: 0: 9724.6, 1: 9669.4. Samples: 927525352. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:34:36,063][104569] Avg episode reward: [(0, '8620.825'), (1, '9350.947')] [2023-12-27 04:34:36,349][105620] Updated weights for policy 1, policy_version 1813197 (0.0007) [2023-12-27 04:34:36,417][105620] Updated weights for policy 1, policy_version 1813207 (0.0006) [2023-12-27 04:34:36,491][105620] Updated weights for policy 1, policy_version 1813217 (0.0006) [2023-12-27 04:34:36,651][105692] Updated weights for policy 0, policy_version 1809494 (0.0011) [2023-12-27 04:34:36,705][105692] Updated weights for policy 0, policy_version 1809504 (0.0011) [2023-12-27 04:34:36,758][105692] Updated weights for policy 0, policy_version 1809514 (0.0011) [2023-12-27 04:34:37,109][105620] Updated weights for policy 1, policy_version 1813227 (0.0007) [2023-12-27 04:34:37,161][105620] Updated weights for policy 1, policy_version 1813237 (0.0006) [2023-12-27 04:34:37,210][105620] Updated weights for policy 1, policy_version 1813247 (0.0007) [2023-12-27 04:34:37,556][105692] Updated weights for policy 0, policy_version 1809524 (0.0010) [2023-12-27 04:34:37,609][105692] Updated weights for policy 0, policy_version 1809534 (0.0010) [2023-12-27 04:34:37,661][105692] Updated weights for policy 0, policy_version 1809544 (0.0009) [2023-12-27 04:34:37,756][105620] Updated weights for policy 1, policy_version 1813257 (0.0006) [2023-12-27 04:34:37,804][105620] Updated weights for policy 1, policy_version 1813267 (0.0009) [2023-12-27 04:34:37,853][105620] Updated weights for policy 1, policy_version 1813277 (0.0009) [2023-12-27 04:34:37,904][105620] Updated weights for policy 1, policy_version 1813287 (0.0009) [2023-12-27 04:34:38,422][105692] Updated weights for policy 0, policy_version 1809554 (0.0009) [2023-12-27 04:34:38,473][105692] Updated weights for policy 0, policy_version 1809564 (0.0008) [2023-12-27 04:34:38,534][105692] Updated weights for policy 0, policy_version 1809574 (0.0009) [2023-12-27 04:34:38,595][105692] Updated weights for policy 0, policy_version 1809584 (0.0005) [2023-12-27 04:34:38,739][105620] Updated weights for policy 1, policy_version 1813297 (0.0009) [2023-12-27 04:34:38,803][105620] Updated weights for policy 1, policy_version 1813307 (0.0008) [2023-12-27 04:34:38,856][105620] Updated weights for policy 1, policy_version 1813317 (0.0008) [2023-12-27 04:34:39,265][105692] Updated weights for policy 0, policy_version 1809594 (0.0008) [2023-12-27 04:34:39,328][105692] Updated weights for policy 0, policy_version 1809604 (0.0008) [2023-12-27 04:34:39,399][105692] Updated weights for policy 0, policy_version 1809614 (0.0010) [2023-12-27 04:34:39,690][105620] Updated weights for policy 1, policy_version 1813327 (0.0010) [2023-12-27 04:34:39,745][105620] Updated weights for policy 1, policy_version 1813337 (0.0009) [2023-12-27 04:34:39,798][105620] Updated weights for policy 1, policy_version 1813347 (0.0008) [2023-12-27 04:34:39,822][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000005 [2023-12-27 04:34:40,026][105692] Updated weights for policy 0, policy_version 1809624 (0.0010) [2023-12-27 04:34:40,092][105692] Updated weights for policy 0, policy_version 1809634 (0.0011) [2023-12-27 04:34:40,156][105692] Updated weights for policy 0, policy_version 1809644 (0.0011) [2023-12-27 04:34:40,526][105620] Updated weights for policy 1, policy_version 1813357 (0.0007) [2023-12-27 04:34:40,591][105620] Updated weights for policy 1, policy_version 1813367 (0.0005) [2023-12-27 04:34:40,651][105620] Updated weights for policy 1, policy_version 1813377 (0.0006) [2023-12-27 04:34:40,840][105692] Updated weights for policy 0, policy_version 1809654 (0.0010) [2023-12-27 04:34:40,888][105692] Updated weights for policy 0, policy_version 1809664 (0.0010) [2023-12-27 04:34:40,940][105692] Updated weights for policy 0, policy_version 1809674 (0.0010) [2023-12-27 04:34:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 927637504. Throughput: 0: 9775.1, 1: 9782.0. Samples: 927644012. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:34:41,062][104569] Avg episode reward: [(0, '8171.272'), (1, '9351.023')] [2023-12-27 04:34:41,289][105620] Updated weights for policy 1, policy_version 1813387 (0.0007) [2023-12-27 04:34:41,343][105620] Updated weights for policy 1, policy_version 1813397 (0.0011) [2023-12-27 04:34:41,419][105620] Updated weights for policy 1, policy_version 1813407 (0.0010) [2023-12-27 04:34:41,690][105692] Updated weights for policy 0, policy_version 1809684 (0.0008) [2023-12-27 04:34:41,755][105692] Updated weights for policy 0, policy_version 1809694 (0.0006) [2023-12-27 04:34:41,824][105692] Updated weights for policy 0, policy_version 1809704 (0.0006) [2023-12-27 04:34:42,252][105620] Updated weights for policy 1, policy_version 1813417 (0.0010) [2023-12-27 04:34:42,316][105620] Updated weights for policy 1, policy_version 1813427 (0.0009) [2023-12-27 04:34:42,385][105620] Updated weights for policy 1, policy_version 1813437 (0.0008) [2023-12-27 04:34:42,451][105620] Updated weights for policy 1, policy_version 1813447 (0.0009) [2023-12-27 04:34:42,501][105692] Updated weights for policy 0, policy_version 1809714 (0.0008) [2023-12-27 04:34:42,560][105692] Updated weights for policy 0, policy_version 1809724 (0.0007) [2023-12-27 04:34:42,618][105692] Updated weights for policy 0, policy_version 1809734 (0.0006) [2023-12-27 04:34:42,679][105692] Updated weights for policy 0, policy_version 1809744 (0.0006) [2023-12-27 04:34:43,293][105620] Updated weights for policy 1, policy_version 1813457 (0.0008) [2023-12-27 04:34:43,297][105692] Updated weights for policy 0, policy_version 1809754 (0.0005) [2023-12-27 04:34:43,346][105620] Updated weights for policy 1, policy_version 1813467 (0.0009) [2023-12-27 04:34:43,348][105692] Updated weights for policy 0, policy_version 1809764 (0.0005) [2023-12-27 04:34:43,411][105692] Updated weights for policy 0, policy_version 1809774 (0.0005) [2023-12-27 04:34:43,412][105620] Updated weights for policy 1, policy_version 1813477 (0.0008) [2023-12-27 04:34:44,016][105692] Updated weights for policy 0, policy_version 1809784 (0.0007) [2023-12-27 04:34:44,077][105692] Updated weights for policy 0, policy_version 1809794 (0.0009) [2023-12-27 04:34:44,127][105692] Updated weights for policy 0, policy_version 1809804 (0.0009) [2023-12-27 04:34:44,194][105620] Updated weights for policy 1, policy_version 1813487 (0.0010) [2023-12-27 04:34:44,259][105620] Updated weights for policy 1, policy_version 1813497 (0.0009) [2023-12-27 04:34:44,321][105620] Updated weights for policy 1, policy_version 1813507 (0.0010) [2023-12-27 04:34:44,891][105692] Updated weights for policy 0, policy_version 1809814 (0.0010) [2023-12-27 04:34:44,917][105620] Updated weights for policy 1, policy_version 1813517 (0.0008) [2023-12-27 04:34:44,943][105692] Updated weights for policy 0, policy_version 1809824 (0.0010) [2023-12-27 04:34:44,977][105620] Updated weights for policy 1, policy_version 1813527 (0.0008) [2023-12-27 04:34:45,008][105692] Updated weights for policy 0, policy_version 1809834 (0.0011) [2023-12-27 04:34:45,039][105620] Updated weights for policy 1, policy_version 1813537 (0.0009) [2023-12-27 04:34:45,660][105620] Updated weights for policy 1, policy_version 1813547 (0.0009) [2023-12-27 04:34:45,702][105692] Updated weights for policy 0, policy_version 1809844 (0.0009) [2023-12-27 04:34:45,726][105620] Updated weights for policy 1, policy_version 1813557 (0.0006) [2023-12-27 04:34:45,749][105692] Updated weights for policy 0, policy_version 1809854 (0.0007) [2023-12-27 04:34:45,789][105620] Updated weights for policy 1, policy_version 1813567 (0.0008) [2023-12-27 04:34:45,800][105692] Updated weights for policy 0, policy_version 1809864 (0.0007) [2023-12-27 04:34:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.4, 300 sec: 19633.0). Total num frames: 927735808. Throughput: 0: 9840.1, 1: 9692.6. Samples: 927700216. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:34:46,063][104569] Avg episode reward: [(0, '8169.295'), (1, '9351.055')] [2023-12-27 04:34:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001809872_463396864.pth... [2023-12-27 04:34:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001813576_464338944.pth... [2023-12-27 04:34:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001808688_463093760.pth [2023-12-27 04:34:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001812424_464044032.pth [2023-12-27 04:34:46,334][105620] Updated weights for policy 1, policy_version 1813577 (0.0008) [2023-12-27 04:34:46,399][105620] Updated weights for policy 1, policy_version 1813587 (0.0010) [2023-12-27 04:34:46,450][105692] Updated weights for policy 0, policy_version 1809874 (0.0007) [2023-12-27 04:34:46,457][105620] Updated weights for policy 1, policy_version 1813597 (0.0010) [2023-12-27 04:34:46,502][105692] Updated weights for policy 0, policy_version 1809884 (0.0005) [2023-12-27 04:34:46,515][105620] Updated weights for policy 1, policy_version 1813607 (0.0010) [2023-12-27 04:34:46,548][105692] Updated weights for policy 0, policy_version 1809894 (0.0009) [2023-12-27 04:34:46,592][105692] Updated weights for policy 0, policy_version 1809904 (0.0009) [2023-12-27 04:34:47,173][105692] Updated weights for policy 0, policy_version 1809914 (0.0005) [2023-12-27 04:34:47,208][105620] Updated weights for policy 1, policy_version 1813617 (0.0008) [2023-12-27 04:34:47,221][105692] Updated weights for policy 0, policy_version 1809924 (0.0005) [2023-12-27 04:34:47,275][105620] Updated weights for policy 1, policy_version 1813627 (0.0007) [2023-12-27 04:34:47,286][105692] Updated weights for policy 0, policy_version 1809934 (0.0005) [2023-12-27 04:34:47,344][105620] Updated weights for policy 1, policy_version 1813637 (0.0009) [2023-12-27 04:34:47,893][105692] Updated weights for policy 0, policy_version 1809944 (0.0007) [2023-12-27 04:34:47,952][105692] Updated weights for policy 0, policy_version 1809954 (0.0005) [2023-12-27 04:34:48,002][105692] Updated weights for policy 0, policy_version 1809964 (0.0005) [2023-12-27 04:34:48,048][105620] Updated weights for policy 1, policy_version 1813647 (0.0009) [2023-12-27 04:34:48,096][105620] Updated weights for policy 1, policy_version 1813657 (0.0010) [2023-12-27 04:34:48,145][105620] Updated weights for policy 1, policy_version 1813667 (0.0009) [2023-12-27 04:34:48,583][105692] Updated weights for policy 0, policy_version 1809974 (0.0005) [2023-12-27 04:34:48,649][105692] Updated weights for policy 0, policy_version 1809984 (0.0008) [2023-12-27 04:34:48,702][105692] Updated weights for policy 0, policy_version 1809994 (0.0008) [2023-12-27 04:34:48,919][105620] Updated weights for policy 1, policy_version 1813677 (0.0010) [2023-12-27 04:34:48,978][105620] Updated weights for policy 1, policy_version 1813687 (0.0010) [2023-12-27 04:34:49,034][105620] Updated weights for policy 1, policy_version 1813697 (0.0011) [2023-12-27 04:34:49,428][105692] Updated weights for policy 0, policy_version 1810004 (0.0008) [2023-12-27 04:34:49,490][105692] Updated weights for policy 0, policy_version 1810014 (0.0009) [2023-12-27 04:34:49,552][105692] Updated weights for policy 0, policy_version 1810024 (0.0009) [2023-12-27 04:34:49,820][105620] Updated weights for policy 1, policy_version 1813707 (0.0011) [2023-12-27 04:34:49,886][105620] Updated weights for policy 1, policy_version 1813717 (0.0011) [2023-12-27 04:34:49,947][105620] Updated weights for policy 1, policy_version 1813727 (0.0011) [2023-12-27 04:34:50,283][105692] Updated weights for policy 0, policy_version 1810034 (0.0008) [2023-12-27 04:34:50,340][105692] Updated weights for policy 0, policy_version 1810044 (0.0008) [2023-12-27 04:34:50,404][105692] Updated weights for policy 0, policy_version 1810054 (0.0008) [2023-12-27 04:34:50,468][105692] Updated weights for policy 0, policy_version 1810064 (0.0008) [2023-12-27 04:34:50,676][105620] Updated weights for policy 1, policy_version 1813737 (0.0010) [2023-12-27 04:34:50,742][105620] Updated weights for policy 1, policy_version 1813747 (0.0011) [2023-12-27 04:34:50,809][105620] Updated weights for policy 1, policy_version 1813757 (0.0011) [2023-12-27 04:34:50,868][105620] Updated weights for policy 1, policy_version 1813767 (0.0010) [2023-12-27 04:34:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 927834112. Throughput: 0: 9987.7, 1: 9737.7. Samples: 927824020. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:34:51,063][104569] Avg episode reward: [(0, '8624.169'), (1, '9350.922')] [2023-12-27 04:34:51,241][105692] Updated weights for policy 0, policy_version 1810074 (0.0008) [2023-12-27 04:34:51,311][105692] Updated weights for policy 0, policy_version 1810084 (0.0008) [2023-12-27 04:34:51,372][105692] Updated weights for policy 0, policy_version 1810094 (0.0008) [2023-12-27 04:34:51,618][105620] Updated weights for policy 1, policy_version 1813777 (0.0011) [2023-12-27 04:34:51,679][105620] Updated weights for policy 1, policy_version 1813787 (0.0011) [2023-12-27 04:34:51,740][105620] Updated weights for policy 1, policy_version 1813797 (0.0011) [2023-12-27 04:34:52,118][105692] Updated weights for policy 0, policy_version 1810104 (0.0008) [2023-12-27 04:34:52,174][105692] Updated weights for policy 0, policy_version 1810114 (0.0008) [2023-12-27 04:34:52,233][105692] Updated weights for policy 0, policy_version 1810124 (0.0008) [2023-12-27 04:34:52,497][105620] Updated weights for policy 1, policy_version 1813807 (0.0010) [2023-12-27 04:34:52,557][105620] Updated weights for policy 1, policy_version 1813817 (0.0011) [2023-12-27 04:34:52,619][105620] Updated weights for policy 1, policy_version 1813827 (0.0011) [2023-12-27 04:34:53,015][105692] Updated weights for policy 0, policy_version 1810134 (0.0008) [2023-12-27 04:34:53,070][105692] Updated weights for policy 0, policy_version 1810144 (0.0008) [2023-12-27 04:34:53,132][105692] Updated weights for policy 0, policy_version 1810154 (0.0008) [2023-12-27 04:34:53,362][105620] Updated weights for policy 1, policy_version 1813837 (0.0011) [2023-12-27 04:34:53,421][105620] Updated weights for policy 1, policy_version 1813847 (0.0010) [2023-12-27 04:34:53,486][105620] Updated weights for policy 1, policy_version 1813857 (0.0011) [2023-12-27 04:34:53,944][105692] Updated weights for policy 0, policy_version 1810164 (0.0008) [2023-12-27 04:34:54,002][105692] Updated weights for policy 0, policy_version 1810174 (0.0010) [2023-12-27 04:34:54,056][105692] Updated weights for policy 0, policy_version 1810185 (0.0009) [2023-12-27 04:34:54,065][105620] Updated weights for policy 1, policy_version 1813867 (0.0010) [2023-12-27 04:34:54,132][105620] Updated weights for policy 1, policy_version 1813877 (0.0008) [2023-12-27 04:34:54,197][105620] Updated weights for policy 1, policy_version 1813887 (0.0007) [2023-12-27 04:34:54,762][105692] Updated weights for policy 0, policy_version 1810195 (0.0006) [2023-12-27 04:34:54,821][105692] Updated weights for policy 0, policy_version 1810205 (0.0005) [2023-12-27 04:34:54,874][105620] Updated weights for policy 1, policy_version 1813897 (0.0009) [2023-12-27 04:34:54,889][105692] Updated weights for policy 0, policy_version 1810215 (0.0007) [2023-12-27 04:34:54,938][105620] Updated weights for policy 1, policy_version 1813907 (0.0006) [2023-12-27 04:34:55,001][105620] Updated weights for policy 1, policy_version 1813917 (0.0006) [2023-12-27 04:34:55,058][105620] Updated weights for policy 1, policy_version 1813927 (0.0007) [2023-12-27 04:34:55,559][105692] Updated weights for policy 0, policy_version 1810225 (0.0009) [2023-12-27 04:34:55,627][105692] Updated weights for policy 0, policy_version 1810235 (0.0007) [2023-12-27 04:34:55,654][105620] Updated weights for policy 1, policy_version 1813937 (0.0009) [2023-12-27 04:34:55,685][105692] Updated weights for policy 0, policy_version 1810245 (0.0007) [2023-12-27 04:34:55,706][105620] Updated weights for policy 1, policy_version 1813947 (0.0010) [2023-12-27 04:34:55,737][105692] Updated weights for policy 0, policy_version 1810255 (0.0007) [2023-12-27 04:34:55,766][105620] Updated weights for policy 1, policy_version 1813957 (0.0009) [2023-12-27 04:34:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.7, 300 sec: 19633.0). Total num frames: 927932416. Throughput: 0: 9967.8, 1: 9771.6. Samples: 927938800. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:34:56,063][104569] Avg episode reward: [(0, '8899.446'), (1, '9350.903')] [2023-12-27 04:34:56,443][105692] Updated weights for policy 0, policy_version 1810265 (0.0008) [2023-12-27 04:34:56,501][105692] Updated weights for policy 0, policy_version 1810275 (0.0007) [2023-12-27 04:34:56,511][105620] Updated weights for policy 1, policy_version 1813967 (0.0010) [2023-12-27 04:34:56,550][105692] Updated weights for policy 0, policy_version 1810285 (0.0006) [2023-12-27 04:34:56,567][105620] Updated weights for policy 1, policy_version 1813977 (0.0010) [2023-12-27 04:34:56,630][105620] Updated weights for policy 1, policy_version 1813987 (0.0010) [2023-12-27 04:34:57,269][105620] Updated weights for policy 1, policy_version 1813997 (0.0008) [2023-12-27 04:34:57,321][105620] Updated weights for policy 1, policy_version 1814007 (0.0007) [2023-12-27 04:34:57,365][105692] Updated weights for policy 0, policy_version 1810295 (0.0007) [2023-12-27 04:34:57,374][105620] Updated weights for policy 1, policy_version 1814017 (0.0006) [2023-12-27 04:34:57,417][105692] Updated weights for policy 0, policy_version 1810305 (0.0009) [2023-12-27 04:34:57,469][105692] Updated weights for policy 0, policy_version 1810315 (0.0009) [2023-12-27 04:34:58,073][105620] Updated weights for policy 1, policy_version 1814027 (0.0005) [2023-12-27 04:34:58,125][105620] Updated weights for policy 1, policy_version 1814037 (0.0005) [2023-12-27 04:34:58,180][105692] Updated weights for policy 0, policy_version 1810325 (0.0009) [2023-12-27 04:34:58,190][105620] Updated weights for policy 1, policy_version 1814047 (0.0008) [2023-12-27 04:34:58,236][105692] Updated weights for policy 0, policy_version 1810335 (0.0007) [2023-12-27 04:34:58,302][105692] Updated weights for policy 0, policy_version 1810345 (0.0008) [2023-12-27 04:34:59,021][105620] Updated weights for policy 1, policy_version 1814057 (0.0010) [2023-12-27 04:34:59,079][105620] Updated weights for policy 1, policy_version 1814067 (0.0009) [2023-12-27 04:34:59,132][105620] Updated weights for policy 1, policy_version 1814077 (0.0009) [2023-12-27 04:34:59,185][105620] Updated weights for policy 1, policy_version 1814087 (0.0009) [2023-12-27 04:34:59,189][105692] Updated weights for policy 0, policy_version 1810355 (0.0008) [2023-12-27 04:34:59,250][105692] Updated weights for policy 0, policy_version 1810365 (0.0009) [2023-12-27 04:34:59,310][105692] Updated weights for policy 0, policy_version 1810375 (0.0008) [2023-12-27 04:35:00,062][105692] Updated weights for policy 0, policy_version 1810385 (0.0009) [2023-12-27 04:35:00,065][105620] Updated weights for policy 1, policy_version 1814097 (0.0008) [2023-12-27 04:35:00,115][105692] Updated weights for policy 0, policy_version 1810395 (0.0007) [2023-12-27 04:35:00,128][105620] Updated weights for policy 1, policy_version 1814107 (0.0008) [2023-12-27 04:35:00,161][105692] Updated weights for policy 0, policy_version 1810405 (0.0007) [2023-12-27 04:35:00,183][105620] Updated weights for policy 1, policy_version 1814117 (0.0006) [2023-12-27 04:35:00,206][105692] Updated weights for policy 0, policy_version 1810415 (0.0008) [2023-12-27 04:35:00,813][105620] Updated weights for policy 1, policy_version 1814127 (0.0005) [2023-12-27 04:35:00,863][105620] Updated weights for policy 1, policy_version 1814137 (0.0005) [2023-12-27 04:35:00,901][105692] Updated weights for policy 0, policy_version 1810425 (0.0008) [2023-12-27 04:35:00,921][105620] Updated weights for policy 1, policy_version 1814147 (0.0007) [2023-12-27 04:35:00,959][105692] Updated weights for policy 0, policy_version 1810435 (0.0008) [2023-12-27 04:35:01,014][105692] Updated weights for policy 0, policy_version 1810445 (0.0008) [2023-12-27 04:35:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 928030720. Throughput: 0: 9974.8, 1: 9748.2. Samples: 927996092. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:35:01,063][104569] Avg episode reward: [(0, '8803.303'), (1, '9258.571')] [2023-12-27 04:35:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001810448_463544320.pth... [2023-12-27 04:35:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001814152_464486400.pth... [2023-12-27 04:35:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001809296_463249408.pth [2023-12-27 04:35:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001813000_464191488.pth [2023-12-27 04:35:01,643][105620] Updated weights for policy 1, policy_version 1814157 (0.0010) [2023-12-27 04:35:01,700][105692] Updated weights for policy 0, policy_version 1810455 (0.0008) [2023-12-27 04:35:01,700][105620] Updated weights for policy 1, policy_version 1814167 (0.0010) [2023-12-27 04:35:01,751][105692] Updated weights for policy 0, policy_version 1810465 (0.0007) [2023-12-27 04:35:01,760][105620] Updated weights for policy 1, policy_version 1814177 (0.0009) [2023-12-27 04:35:01,805][105692] Updated weights for policy 0, policy_version 1810475 (0.0007) [2023-12-27 04:35:02,402][105620] Updated weights for policy 1, policy_version 1814187 (0.0011) [2023-12-27 04:35:02,464][105620] Updated weights for policy 1, policy_version 1814197 (0.0010) [2023-12-27 04:35:02,530][105620] Updated weights for policy 1, policy_version 1814207 (0.0008) [2023-12-27 04:35:02,639][105692] Updated weights for policy 0, policy_version 1810485 (0.0009) [2023-12-27 04:35:02,698][105692] Updated weights for policy 0, policy_version 1810495 (0.0008) [2023-12-27 04:35:02,767][105692] Updated weights for policy 0, policy_version 1810505 (0.0008) [2023-12-27 04:35:03,254][105620] Updated weights for policy 1, policy_version 1814217 (0.0008) [2023-12-27 04:35:03,312][105620] Updated weights for policy 1, policy_version 1814227 (0.0009) [2023-12-27 04:35:03,326][105692] Updated weights for policy 0, policy_version 1810515 (0.0007) [2023-12-27 04:35:03,374][105620] Updated weights for policy 1, policy_version 1814237 (0.0005) [2023-12-27 04:35:03,380][105692] Updated weights for policy 0, policy_version 1810525 (0.0006) [2023-12-27 04:35:03,436][105692] Updated weights for policy 0, policy_version 1810535 (0.0007) [2023-12-27 04:35:03,469][105620] Updated weights for policy 1, policy_version 1814247 (0.0006) [2023-12-27 04:35:04,030][105692] Updated weights for policy 0, policy_version 1810545 (0.0007) [2023-12-27 04:35:04,081][105692] Updated weights for policy 0, policy_version 1810555 (0.0008) [2023-12-27 04:35:04,133][105692] Updated weights for policy 0, policy_version 1810565 (0.0008) [2023-12-27 04:35:04,155][105620] Updated weights for policy 1, policy_version 1814257 (0.0009) [2023-12-27 04:35:04,188][105692] Updated weights for policy 0, policy_version 1810575 (0.0006) [2023-12-27 04:35:04,216][105620] Updated weights for policy 1, policy_version 1814267 (0.0011) [2023-12-27 04:35:04,278][105620] Updated weights for policy 1, policy_version 1814277 (0.0010) [2023-12-27 04:35:04,929][105620] Updated weights for policy 1, policy_version 1814287 (0.0006) [2023-12-27 04:35:04,997][105620] Updated weights for policy 1, policy_version 1814297 (0.0005) [2023-12-27 04:35:05,029][105692] Updated weights for policy 0, policy_version 1810585 (0.0008) [2023-12-27 04:35:05,057][105620] Updated weights for policy 1, policy_version 1814307 (0.0010) [2023-12-27 04:35:05,087][105692] Updated weights for policy 0, policy_version 1810595 (0.0006) [2023-12-27 04:35:05,151][105692] Updated weights for policy 0, policy_version 1810606 (0.0011) [2023-12-27 04:35:05,700][105620] Updated weights for policy 1, policy_version 1814317 (0.0010) [2023-12-27 04:35:05,756][105620] Updated weights for policy 1, policy_version 1814327 (0.0008) [2023-12-27 04:35:05,810][105620] Updated weights for policy 1, policy_version 1814337 (0.0005) [2023-12-27 04:35:05,937][105692] Updated weights for policy 0, policy_version 1810616 (0.0010) [2023-12-27 04:35:06,000][105692] Updated weights for policy 0, policy_version 1810626 (0.0010) [2023-12-27 04:35:06,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 928120832. Throughput: 0: 9927.0, 1: 9767.3. Samples: 928113656. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:35:06,062][104569] Avg episode reward: [(0, '8716.386'), (1, '9167.263')] [2023-12-27 04:35:06,062][105692] Updated weights for policy 0, policy_version 1810636 (0.0009) [2023-12-27 04:35:06,523][105620] Updated weights for policy 1, policy_version 1814347 (0.0007) [2023-12-27 04:35:06,586][105620] Updated weights for policy 1, policy_version 1814357 (0.0008) [2023-12-27 04:35:06,657][105620] Updated weights for policy 1, policy_version 1814367 (0.0006) [2023-12-27 04:35:06,788][105692] Updated weights for policy 0, policy_version 1810646 (0.0010) [2023-12-27 04:35:06,854][105692] Updated weights for policy 0, policy_version 1810656 (0.0010) [2023-12-27 04:35:06,917][105692] Updated weights for policy 0, policy_version 1810666 (0.0010) [2023-12-27 04:35:07,262][105620] Updated weights for policy 1, policy_version 1814377 (0.0008) [2023-12-27 04:35:07,320][105620] Updated weights for policy 1, policy_version 1814387 (0.0009) [2023-12-27 04:35:07,375][105620] Updated weights for policy 1, policy_version 1814397 (0.0009) [2023-12-27 04:35:07,420][105620] Updated weights for policy 1, policy_version 1814407 (0.0010) [2023-12-27 04:35:07,738][105692] Updated weights for policy 0, policy_version 1810676 (0.0009) [2023-12-27 04:35:07,783][105692] Updated weights for policy 0, policy_version 1810686 (0.0009) [2023-12-27 04:35:07,829][105692] Updated weights for policy 0, policy_version 1810697 (0.0009) [2023-12-27 04:35:08,119][105620] Updated weights for policy 1, policy_version 1814417 (0.0010) [2023-12-27 04:35:08,167][105620] Updated weights for policy 1, policy_version 1814427 (0.0010) [2023-12-27 04:35:08,232][105620] Updated weights for policy 1, policy_version 1814437 (0.0010) [2023-12-27 04:35:08,514][105692] Updated weights for policy 0, policy_version 1810707 (0.0007) [2023-12-27 04:35:08,581][105692] Updated weights for policy 0, policy_version 1810717 (0.0007) [2023-12-27 04:35:08,644][105692] Updated weights for policy 0, policy_version 1810727 (0.0009) [2023-12-27 04:35:08,911][105620] Updated weights for policy 1, policy_version 1814447 (0.0007) [2023-12-27 04:35:08,957][105620] Updated weights for policy 1, policy_version 1814457 (0.0007) [2023-12-27 04:35:09,010][105620] Updated weights for policy 1, policy_version 1814467 (0.0010) [2023-12-27 04:35:09,301][105692] Updated weights for policy 0, policy_version 1810737 (0.0005) [2023-12-27 04:35:09,364][105692] Updated weights for policy 0, policy_version 1810747 (0.0006) [2023-12-27 04:35:09,434][105692] Updated weights for policy 0, policy_version 1810757 (0.0008) [2023-12-27 04:35:09,490][105692] Updated weights for policy 0, policy_version 1810767 (0.0006) [2023-12-27 04:35:09,859][105620] Updated weights for policy 1, policy_version 1814477 (0.0010) [2023-12-27 04:35:09,924][105620] Updated weights for policy 1, policy_version 1814487 (0.0009) [2023-12-27 04:35:09,994][105620] Updated weights for policy 1, policy_version 1814497 (0.0009) [2023-12-27 04:35:10,164][105692] Updated weights for policy 0, policy_version 1810777 (0.0009) [2023-12-27 04:35:10,224][105692] Updated weights for policy 0, policy_version 1810787 (0.0009) [2023-12-27 04:35:10,289][105692] Updated weights for policy 0, policy_version 1810797 (0.0009) [2023-12-27 04:35:10,662][105620] Updated weights for policy 1, policy_version 1814507 (0.0008) [2023-12-27 04:35:10,728][105620] Updated weights for policy 1, policy_version 1814517 (0.0007) [2023-12-27 04:35:10,795][105620] Updated weights for policy 1, policy_version 1814527 (0.0009) [2023-12-27 04:35:11,061][105692] Updated weights for policy 0, policy_version 1810807 (0.0009) [2023-12-27 04:35:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 928219136. Throughput: 0: 9891.7, 1: 9829.8. Samples: 928228588. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:35:11,062][104569] Avg episode reward: [(0, '8811.723'), (1, '9259.593')] [2023-12-27 04:35:11,129][105692] Updated weights for policy 0, policy_version 1810817 (0.0010) [2023-12-27 04:35:11,192][105692] Updated weights for policy 0, policy_version 1810827 (0.0011) [2023-12-27 04:35:11,566][105620] Updated weights for policy 1, policy_version 1814537 (0.0009) [2023-12-27 04:35:11,624][105620] Updated weights for policy 1, policy_version 1814547 (0.0008) [2023-12-27 04:35:11,686][105620] Updated weights for policy 1, policy_version 1814557 (0.0008) [2023-12-27 04:35:11,731][105620] Updated weights for policy 1, policy_version 1814567 (0.0008) [2023-12-27 04:35:11,954][105692] Updated weights for policy 0, policy_version 1810837 (0.0011) [2023-12-27 04:35:12,013][105692] Updated weights for policy 0, policy_version 1810847 (0.0011) [2023-12-27 04:35:12,073][105692] Updated weights for policy 0, policy_version 1810857 (0.0011) [2023-12-27 04:35:12,514][105620] Updated weights for policy 1, policy_version 1814577 (0.0006) [2023-12-27 04:35:12,571][105620] Updated weights for policy 1, policy_version 1814587 (0.0005) [2023-12-27 04:35:12,629][105620] Updated weights for policy 1, policy_version 1814597 (0.0006) [2023-12-27 04:35:12,797][105692] Updated weights for policy 0, policy_version 1810867 (0.0009) [2023-12-27 04:35:12,856][105692] Updated weights for policy 0, policy_version 1810877 (0.0010) [2023-12-27 04:35:12,914][105692] Updated weights for policy 0, policy_version 1810887 (0.0010) [2023-12-27 04:35:13,338][105620] Updated weights for policy 1, policy_version 1814607 (0.0008) [2023-12-27 04:35:13,394][105620] Updated weights for policy 1, policy_version 1814617 (0.0008) [2023-12-27 04:35:13,461][105620] Updated weights for policy 1, policy_version 1814627 (0.0008) [2023-12-27 04:35:13,662][105692] Updated weights for policy 0, policy_version 1810897 (0.0011) [2023-12-27 04:35:13,721][105692] Updated weights for policy 0, policy_version 1810907 (0.0011) [2023-12-27 04:35:13,770][105692] Updated weights for policy 0, policy_version 1810917 (0.0011) [2023-12-27 04:35:13,827][105692] Updated weights for policy 0, policy_version 1810927 (0.0010) [2023-12-27 04:35:14,207][105620] Updated weights for policy 1, policy_version 1814637 (0.0009) [2023-12-27 04:35:14,256][105620] Updated weights for policy 1, policy_version 1814647 (0.0010) [2023-12-27 04:35:14,307][105620] Updated weights for policy 1, policy_version 1814657 (0.0009) [2023-12-27 04:35:14,516][105692] Updated weights for policy 0, policy_version 1810937 (0.0011) [2023-12-27 04:35:14,574][105692] Updated weights for policy 0, policy_version 1810947 (0.0010) [2023-12-27 04:35:14,635][105692] Updated weights for policy 0, policy_version 1810957 (0.0011) [2023-12-27 04:35:14,998][105620] Updated weights for policy 1, policy_version 1814667 (0.0005) [2023-12-27 04:35:15,065][105620] Updated weights for policy 1, policy_version 1814677 (0.0005) [2023-12-27 04:35:15,132][105620] Updated weights for policy 1, policy_version 1814687 (0.0006) [2023-12-27 04:35:15,392][105692] Updated weights for policy 0, policy_version 1810967 (0.0010) [2023-12-27 04:35:15,462][105692] Updated weights for policy 0, policy_version 1810977 (0.0009) [2023-12-27 04:35:15,528][105692] Updated weights for policy 0, policy_version 1810987 (0.0008) [2023-12-27 04:35:15,735][105620] Updated weights for policy 1, policy_version 1814697 (0.0007) [2023-12-27 04:35:15,794][105620] Updated weights for policy 1, policy_version 1814707 (0.0011) [2023-12-27 04:35:15,854][105620] Updated weights for policy 1, policy_version 1814717 (0.0011) [2023-12-27 04:35:15,903][105620] Updated weights for policy 1, policy_version 1814727 (0.0010) [2023-12-27 04:35:16,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19660.7, 300 sec: 19605.2). Total num frames: 928317440. Throughput: 0: 9853.1, 1: 9763.6. Samples: 928284964. Policy #0 lag: (min: 31.0, avg: 31.2, max: 44.0) [2023-12-27 04:35:16,063][104569] Avg episode reward: [(0, '8989.966'), (1, '9166.137')] [2023-12-27 04:35:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001810992_463683584.pth... [2023-12-27 04:35:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001814728_464633856.pth... [2023-12-27 04:35:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001809872_463396864.pth [2023-12-27 04:35:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001813576_464338944.pth [2023-12-27 04:35:16,246][105692] Updated weights for policy 0, policy_version 1810997 (0.0009) [2023-12-27 04:35:16,302][105692] Updated weights for policy 0, policy_version 1811007 (0.0008) [2023-12-27 04:35:16,349][105692] Updated weights for policy 0, policy_version 1811017 (0.0008) [2023-12-27 04:35:16,660][105620] Updated weights for policy 1, policy_version 1814737 (0.0010) [2023-12-27 04:35:16,706][105620] Updated weights for policy 1, policy_version 1814747 (0.0010) [2023-12-27 04:35:16,761][105620] Updated weights for policy 1, policy_version 1814757 (0.0010) [2023-12-27 04:35:17,131][105692] Updated weights for policy 0, policy_version 1811027 (0.0008) [2023-12-27 04:35:17,185][105692] Updated weights for policy 0, policy_version 1811037 (0.0008) [2023-12-27 04:35:17,244][105692] Updated weights for policy 0, policy_version 1811047 (0.0008) [2023-12-27 04:35:17,510][105620] Updated weights for policy 1, policy_version 1814767 (0.0010) [2023-12-27 04:35:17,564][105620] Updated weights for policy 1, policy_version 1814777 (0.0010) [2023-12-27 04:35:17,612][105620] Updated weights for policy 1, policy_version 1814787 (0.0010) [2023-12-27 04:35:18,018][105692] Updated weights for policy 0, policy_version 1811057 (0.0008) [2023-12-27 04:35:18,082][105692] Updated weights for policy 0, policy_version 1811067 (0.0008) [2023-12-27 04:35:18,146][105692] Updated weights for policy 0, policy_version 1811077 (0.0008) [2023-12-27 04:35:18,206][105692] Updated weights for policy 0, policy_version 1811087 (0.0011) [2023-12-27 04:35:18,375][105620] Updated weights for policy 1, policy_version 1814797 (0.0010) [2023-12-27 04:35:18,431][105620] Updated weights for policy 1, policy_version 1814807 (0.0011) [2023-12-27 04:35:18,493][105620] Updated weights for policy 1, policy_version 1814817 (0.0011) [2023-12-27 04:35:18,890][105692] Updated weights for policy 0, policy_version 1811097 (0.0010) [2023-12-27 04:35:18,952][105692] Updated weights for policy 0, policy_version 1811107 (0.0011) [2023-12-27 04:35:19,011][105692] Updated weights for policy 0, policy_version 1811117 (0.0010) [2023-12-27 04:35:19,243][105620] Updated weights for policy 1, policy_version 1814827 (0.0011) [2023-12-27 04:35:19,314][105620] Updated weights for policy 1, policy_version 1814837 (0.0011) [2023-12-27 04:35:19,376][105620] Updated weights for policy 1, policy_version 1814847 (0.0011) [2023-12-27 04:35:19,771][105692] Updated weights for policy 0, policy_version 1811127 (0.0008) [2023-12-27 04:35:19,826][105692] Updated weights for policy 0, policy_version 1811137 (0.0008) [2023-12-27 04:35:19,896][105692] Updated weights for policy 0, policy_version 1811147 (0.0006) [2023-12-27 04:35:20,110][105620] Updated weights for policy 1, policy_version 1814857 (0.0010) [2023-12-27 04:35:20,166][105620] Updated weights for policy 1, policy_version 1814867 (0.0011) [2023-12-27 04:35:20,227][105620] Updated weights for policy 1, policy_version 1814877 (0.0011) [2023-12-27 04:35:20,288][105620] Updated weights for policy 1, policy_version 1814887 (0.0011) [2023-12-27 04:35:20,575][105692] Updated weights for policy 0, policy_version 1811157 (0.0007) [2023-12-27 04:35:20,640][105692] Updated weights for policy 0, policy_version 1811167 (0.0009) [2023-12-27 04:35:20,690][105692] Updated weights for policy 0, policy_version 1811177 (0.0009) [2023-12-27 04:35:20,960][105620] Updated weights for policy 1, policy_version 1814897 (0.0011) [2023-12-27 04:35:21,029][105620] Updated weights for policy 1, policy_version 1814907 (0.0010) [2023-12-27 04:35:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 928407552. Throughput: 0: 9679.3, 1: 9739.1. Samples: 928399180. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:35:21,062][104569] Avg episode reward: [(0, '8988.290'), (1, '9166.072')] [2023-12-27 04:35:21,100][105620] Updated weights for policy 1, policy_version 1814917 (0.0008) [2023-12-27 04:35:21,519][105692] Updated weights for policy 0, policy_version 1811187 (0.0009) [2023-12-27 04:35:21,580][105692] Updated weights for policy 0, policy_version 1811197 (0.0008) [2023-12-27 04:35:21,647][105692] Updated weights for policy 0, policy_version 1811207 (0.0008) [2023-12-27 04:35:21,835][105620] Updated weights for policy 1, policy_version 1814927 (0.0007) [2023-12-27 04:35:21,895][105620] Updated weights for policy 1, policy_version 1814937 (0.0005) [2023-12-27 04:35:21,960][105620] Updated weights for policy 1, policy_version 1814947 (0.0006) [2023-12-27 04:35:22,498][105692] Updated weights for policy 0, policy_version 1811217 (0.0007) [2023-12-27 04:35:22,549][105692] Updated weights for policy 0, policy_version 1811227 (0.0008) [2023-12-27 04:35:22,599][105692] Updated weights for policy 0, policy_version 1811237 (0.0008) [2023-12-27 04:35:22,644][105620] Updated weights for policy 1, policy_version 1814957 (0.0008) [2023-12-27 04:35:22,654][105692] Updated weights for policy 0, policy_version 1811247 (0.0007) [2023-12-27 04:35:22,706][105620] Updated weights for policy 1, policy_version 1814967 (0.0007) [2023-12-27 04:35:22,773][105620] Updated weights for policy 1, policy_version 1814977 (0.0005) [2023-12-27 04:35:23,449][105620] Updated weights for policy 1, policy_version 1814987 (0.0009) [2023-12-27 04:35:23,471][105692] Updated weights for policy 0, policy_version 1811257 (0.0007) [2023-12-27 04:35:23,494][105620] Updated weights for policy 1, policy_version 1814997 (0.0010) [2023-12-27 04:35:23,529][105692] Updated weights for policy 0, policy_version 1811267 (0.0009) [2023-12-27 04:35:23,553][105620] Updated weights for policy 1, policy_version 1815007 (0.0010) [2023-12-27 04:35:23,584][105692] Updated weights for policy 0, policy_version 1811277 (0.0010) [2023-12-27 04:35:24,281][105620] Updated weights for policy 1, policy_version 1815017 (0.0010) [2023-12-27 04:35:24,333][105620] Updated weights for policy 1, policy_version 1815027 (0.0010) [2023-12-27 04:35:24,355][105692] Updated weights for policy 0, policy_version 1811287 (0.0007) [2023-12-27 04:35:24,391][105620] Updated weights for policy 1, policy_version 1815037 (0.0010) [2023-12-27 04:35:24,414][105692] Updated weights for policy 0, policy_version 1811297 (0.0008) [2023-12-27 04:35:24,450][105620] Updated weights for policy 1, policy_version 1815047 (0.0011) [2023-12-27 04:35:24,473][105692] Updated weights for policy 0, policy_version 1811307 (0.0007) [2023-12-27 04:35:25,175][105692] Updated weights for policy 0, policy_version 1811317 (0.0007) [2023-12-27 04:35:25,185][105620] Updated weights for policy 1, policy_version 1815057 (0.0010) [2023-12-27 04:35:25,234][105692] Updated weights for policy 0, policy_version 1811327 (0.0005) [2023-12-27 04:35:25,240][105620] Updated weights for policy 1, policy_version 1815067 (0.0010) [2023-12-27 04:35:25,286][105692] Updated weights for policy 0, policy_version 1811337 (0.0005) [2023-12-27 04:35:25,300][105620] Updated weights for policy 1, policy_version 1815077 (0.0010) [2023-12-27 04:35:25,940][105620] Updated weights for policy 1, policy_version 1815087 (0.0007) [2023-12-27 04:35:25,991][105620] Updated weights for policy 1, policy_version 1815097 (0.0005) [2023-12-27 04:35:26,049][105620] Updated weights for policy 1, policy_version 1815107 (0.0009) [2023-12-27 04:35:26,062][104569] Fps is (10 sec: 18022.7, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 928497664. Throughput: 0: 9603.1, 1: 9715.8. Samples: 928513364. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:35:26,063][104569] Avg episode reward: [(0, '8807.706'), (1, '9258.504')] [2023-12-27 04:35:26,078][105692] Updated weights for policy 0, policy_version 1811347 (0.0006) [2023-12-27 04:35:26,130][105692] Updated weights for policy 0, policy_version 1811357 (0.0009) [2023-12-27 04:35:26,182][105692] Updated weights for policy 0, policy_version 1811367 (0.0008) [2023-12-27 04:35:26,753][105620] Updated weights for policy 1, policy_version 1815117 (0.0008) [2023-12-27 04:35:26,805][105620] Updated weights for policy 1, policy_version 1815127 (0.0005) [2023-12-27 04:35:26,851][105620] Updated weights for policy 1, policy_version 1815137 (0.0006) [2023-12-27 04:35:26,948][105692] Updated weights for policy 0, policy_version 1811377 (0.0008) [2023-12-27 04:35:27,002][105692] Updated weights for policy 0, policy_version 1811387 (0.0008) [2023-12-27 04:35:27,049][105692] Updated weights for policy 0, policy_version 1811397 (0.0010) [2023-12-27 04:35:27,103][105692] Updated weights for policy 0, policy_version 1811407 (0.0010) [2023-12-27 04:35:27,424][105620] Updated weights for policy 1, policy_version 1815147 (0.0005) [2023-12-27 04:35:27,496][105620] Updated weights for policy 1, policy_version 1815157 (0.0006) [2023-12-27 04:35:27,559][105620] Updated weights for policy 1, policy_version 1815167 (0.0008) [2023-12-27 04:35:27,958][105692] Updated weights for policy 0, policy_version 1811417 (0.0009) [2023-12-27 04:35:28,019][105692] Updated weights for policy 0, policy_version 1811427 (0.0009) [2023-12-27 04:35:28,079][105692] Updated weights for policy 0, policy_version 1811437 (0.0008) [2023-12-27 04:35:28,260][105620] Updated weights for policy 1, policy_version 1815177 (0.0009) [2023-12-27 04:35:28,325][105620] Updated weights for policy 1, policy_version 1815187 (0.0009) [2023-12-27 04:35:28,384][105620] Updated weights for policy 1, policy_version 1815197 (0.0007) [2023-12-27 04:35:28,450][105620] Updated weights for policy 1, policy_version 1815207 (0.0006) [2023-12-27 04:35:28,919][105692] Updated weights for policy 0, policy_version 1811447 (0.0007) [2023-12-27 04:35:28,980][105692] Updated weights for policy 0, policy_version 1811457 (0.0008) [2023-12-27 04:35:29,007][105620] Updated weights for policy 1, policy_version 1815217 (0.0007) [2023-12-27 04:35:29,037][105692] Updated weights for policy 0, policy_version 1811467 (0.0007) [2023-12-27 04:35:29,065][105620] Updated weights for policy 1, policy_version 1815227 (0.0007) [2023-12-27 04:35:29,125][105620] Updated weights for policy 1, policy_version 1815237 (0.0009) [2023-12-27 04:35:29,748][105692] Updated weights for policy 0, policy_version 1811477 (0.0006) [2023-12-27 04:35:29,756][105620] Updated weights for policy 1, policy_version 1815247 (0.0007) [2023-12-27 04:35:29,807][105620] Updated weights for policy 1, policy_version 1815257 (0.0008) [2023-12-27 04:35:29,809][105692] Updated weights for policy 0, policy_version 1811487 (0.0005) [2023-12-27 04:35:29,866][105620] Updated weights for policy 1, policy_version 1815267 (0.0006) [2023-12-27 04:35:29,870][105692] Updated weights for policy 0, policy_version 1811497 (0.0007) [2023-12-27 04:35:30,578][105692] Updated weights for policy 0, policy_version 1811507 (0.0008) [2023-12-27 04:35:30,592][105620] Updated weights for policy 1, policy_version 1815277 (0.0008) [2023-12-27 04:35:30,635][105692] Updated weights for policy 0, policy_version 1811517 (0.0006) [2023-12-27 04:35:30,637][105620] Updated weights for policy 1, policy_version 1815287 (0.0005) [2023-12-27 04:35:30,688][105620] Updated weights for policy 1, policy_version 1815297 (0.0006) [2023-12-27 04:35:30,691][105692] Updated weights for policy 0, policy_version 1811527 (0.0006) [2023-12-27 04:35:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 928604160. Throughput: 0: 9505.5, 1: 9867.8. Samples: 928572012. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:35:31,062][104569] Avg episode reward: [(0, '8532.021'), (1, '9258.508')] [2023-12-27 04:35:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001811536_463822848.pth... [2023-12-27 04:35:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001815304_464781312.pth... [2023-12-27 04:35:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001814152_464486400.pth [2023-12-27 04:35:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001810448_463544320.pth [2023-12-27 04:35:31,438][105620] Updated weights for policy 1, policy_version 1815307 (0.0008) [2023-12-27 04:35:31,485][105692] Updated weights for policy 0, policy_version 1811537 (0.0006) [2023-12-27 04:35:31,499][105620] Updated weights for policy 1, policy_version 1815317 (0.0009) [2023-12-27 04:35:31,550][105692] Updated weights for policy 0, policy_version 1811547 (0.0007) [2023-12-27 04:35:31,562][105620] Updated weights for policy 1, policy_version 1815327 (0.0007) [2023-12-27 04:35:31,610][105692] Updated weights for policy 0, policy_version 1811557 (0.0007) [2023-12-27 04:35:31,668][105692] Updated weights for policy 0, policy_version 1811567 (0.0009) [2023-12-27 04:35:32,265][105620] Updated weights for policy 1, policy_version 1815337 (0.0007) [2023-12-27 04:35:32,327][105620] Updated weights for policy 1, policy_version 1815347 (0.0007) [2023-12-27 04:35:32,390][105620] Updated weights for policy 1, policy_version 1815357 (0.0010) [2023-12-27 04:35:32,455][105620] Updated weights for policy 1, policy_version 1815367 (0.0008) [2023-12-27 04:35:32,504][105692] Updated weights for policy 0, policy_version 1811577 (0.0007) [2023-12-27 04:35:32,575][105692] Updated weights for policy 0, policy_version 1811587 (0.0006) [2023-12-27 04:35:32,643][105692] Updated weights for policy 0, policy_version 1811597 (0.0006) [2023-12-27 04:35:33,028][105620] Updated weights for policy 1, policy_version 1815377 (0.0005) [2023-12-27 04:35:33,086][105620] Updated weights for policy 1, policy_version 1815387 (0.0005) [2023-12-27 04:35:33,142][105620] Updated weights for policy 1, policy_version 1815397 (0.0007) [2023-12-27 04:35:33,354][105692] Updated weights for policy 0, policy_version 1811608 (0.0009) [2023-12-27 04:35:33,407][105692] Updated weights for policy 0, policy_version 1811618 (0.0009) [2023-12-27 04:35:33,461][105692] Updated weights for policy 0, policy_version 1811628 (0.0008) [2023-12-27 04:35:33,717][105620] Updated weights for policy 1, policy_version 1815407 (0.0010) [2023-12-27 04:35:33,765][105620] Updated weights for policy 1, policy_version 1815417 (0.0008) [2023-12-27 04:35:33,815][105620] Updated weights for policy 1, policy_version 1815427 (0.0005) [2023-12-27 04:35:34,312][105692] Updated weights for policy 0, policy_version 1811638 (0.0009) [2023-12-27 04:35:34,374][105692] Updated weights for policy 0, policy_version 1811648 (0.0009) [2023-12-27 04:35:34,379][105620] Updated weights for policy 1, policy_version 1815437 (0.0008) [2023-12-27 04:35:34,436][105692] Updated weights for policy 0, policy_version 1811658 (0.0006) [2023-12-27 04:35:34,445][105620] Updated weights for policy 1, policy_version 1815447 (0.0010) [2023-12-27 04:35:34,497][105620] Updated weights for policy 1, policy_version 1815457 (0.0005) [2023-12-27 04:35:35,164][105692] Updated weights for policy 0, policy_version 1811668 (0.0007) [2023-12-27 04:35:35,167][105620] Updated weights for policy 1, policy_version 1815467 (0.0007) [2023-12-27 04:35:35,224][105620] Updated weights for policy 1, policy_version 1815477 (0.0005) [2023-12-27 04:35:35,227][105692] Updated weights for policy 0, policy_version 1811678 (0.0005) [2023-12-27 04:35:35,276][105692] Updated weights for policy 0, policy_version 1811688 (0.0005) [2023-12-27 04:35:35,278][105620] Updated weights for policy 1, policy_version 1815487 (0.0005) [2023-12-27 04:35:35,889][105620] Updated weights for policy 1, policy_version 1815497 (0.0008) [2023-12-27 04:35:35,895][105692] Updated weights for policy 0, policy_version 1811698 (0.0005) [2023-12-27 04:35:35,944][105620] Updated weights for policy 1, policy_version 1815507 (0.0008) [2023-12-27 04:35:35,952][105692] Updated weights for policy 0, policy_version 1811708 (0.0005) [2023-12-27 04:35:35,996][105620] Updated weights for policy 1, policy_version 1815517 (0.0008) [2023-12-27 04:35:36,007][105692] Updated weights for policy 0, policy_version 1811718 (0.0005) [2023-12-27 04:35:36,045][105620] Updated weights for policy 1, policy_version 1815527 (0.0009) [2023-12-27 04:35:36,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 928702464. Throughput: 0: 9310.8, 1: 9919.8. Samples: 928689396. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:35:36,062][104569] Avg episode reward: [(0, '8712.221'), (1, '9166.193')] [2023-12-27 04:35:36,080][105692] Updated weights for policy 0, policy_version 1811728 (0.0005) [2023-12-27 04:35:36,681][105692] Updated weights for policy 0, policy_version 1811738 (0.0005) [2023-12-27 04:35:36,745][105692] Updated weights for policy 0, policy_version 1811748 (0.0006) [2023-12-27 04:35:36,812][105692] Updated weights for policy 0, policy_version 1811758 (0.0008) [2023-12-27 04:35:36,814][105620] Updated weights for policy 1, policy_version 1815537 (0.0006) [2023-12-27 04:35:36,875][105620] Updated weights for policy 1, policy_version 1815547 (0.0008) [2023-12-27 04:35:36,934][105620] Updated weights for policy 1, policy_version 1815557 (0.0009) [2023-12-27 04:35:37,501][105692] Updated weights for policy 0, policy_version 1811768 (0.0008) [2023-12-27 04:35:37,556][105692] Updated weights for policy 0, policy_version 1811778 (0.0009) [2023-12-27 04:35:37,609][105692] Updated weights for policy 0, policy_version 1811788 (0.0005) [2023-12-27 04:35:37,706][105620] Updated weights for policy 1, policy_version 1815567 (0.0009) [2023-12-27 04:35:37,774][105620] Updated weights for policy 1, policy_version 1815577 (0.0008) [2023-12-27 04:35:37,841][105620] Updated weights for policy 1, policy_version 1815587 (0.0008) [2023-12-27 04:35:38,335][105692] Updated weights for policy 0, policy_version 1811798 (0.0010) [2023-12-27 04:35:38,396][105692] Updated weights for policy 0, policy_version 1811808 (0.0011) [2023-12-27 04:35:38,448][105692] Updated weights for policy 0, policy_version 1811818 (0.0011) [2023-12-27 04:35:38,640][105620] Updated weights for policy 1, policy_version 1815597 (0.0008) [2023-12-27 04:35:38,699][105620] Updated weights for policy 1, policy_version 1815607 (0.0009) [2023-12-27 04:35:38,763][105620] Updated weights for policy 1, policy_version 1815617 (0.0009) [2023-12-27 04:35:39,182][105692] Updated weights for policy 0, policy_version 1811828 (0.0009) [2023-12-27 04:35:39,242][105692] Updated weights for policy 0, policy_version 1811838 (0.0009) [2023-12-27 04:35:39,308][105692] Updated weights for policy 0, policy_version 1811848 (0.0008) [2023-12-27 04:35:39,552][105620] Updated weights for policy 1, policy_version 1815627 (0.0009) [2023-12-27 04:35:39,604][105620] Updated weights for policy 1, policy_version 1815637 (0.0010) [2023-12-27 04:35:39,656][105620] Updated weights for policy 1, policy_version 1815647 (0.0010) [2023-12-27 04:35:40,016][105692] Updated weights for policy 0, policy_version 1811858 (0.0008) [2023-12-27 04:35:40,075][105692] Updated weights for policy 0, policy_version 1811868 (0.0006) [2023-12-27 04:35:40,136][105692] Updated weights for policy 0, policy_version 1811878 (0.0005) [2023-12-27 04:35:40,203][105692] Updated weights for policy 0, policy_version 1811888 (0.0007) [2023-12-27 04:35:40,405][105620] Updated weights for policy 1, policy_version 1815657 (0.0010) [2023-12-27 04:35:40,457][105620] Updated weights for policy 1, policy_version 1815667 (0.0009) [2023-12-27 04:35:40,519][105620] Updated weights for policy 1, policy_version 1815677 (0.0010) [2023-12-27 04:35:40,584][105620] Updated weights for policy 1, policy_version 1815687 (0.0009) [2023-12-27 04:35:40,769][105692] Updated weights for policy 0, policy_version 1811898 (0.0009) [2023-12-27 04:35:40,832][105692] Updated weights for policy 0, policy_version 1811908 (0.0009) [2023-12-27 04:35:40,897][105692] Updated weights for policy 0, policy_version 1811918 (0.0010) [2023-12-27 04:35:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 928800768. Throughput: 0: 9431.1, 1: 9865.5. Samples: 928807140. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:35:41,063][104569] Avg episode reward: [(0, '8536.371'), (1, '9073.786')] [2023-12-27 04:35:41,393][105620] Updated weights for policy 1, policy_version 1815697 (0.0008) [2023-12-27 04:35:41,462][105620] Updated weights for policy 1, policy_version 1815707 (0.0009) [2023-12-27 04:35:41,525][105620] Updated weights for policy 1, policy_version 1815717 (0.0011) [2023-12-27 04:35:41,707][105692] Updated weights for policy 0, policy_version 1811928 (0.0008) [2023-12-27 04:35:41,773][105692] Updated weights for policy 0, policy_version 1811938 (0.0009) [2023-12-27 04:35:41,827][105692] Updated weights for policy 0, policy_version 1811948 (0.0005) [2023-12-27 04:35:42,169][105620] Updated weights for policy 1, policy_version 1815727 (0.0007) [2023-12-27 04:35:42,243][105620] Updated weights for policy 1, policy_version 1815737 (0.0008) [2023-12-27 04:35:42,308][105620] Updated weights for policy 1, policy_version 1815747 (0.0010) [2023-12-27 04:35:42,597][105692] Updated weights for policy 0, policy_version 1811958 (0.0009) [2023-12-27 04:35:42,664][105692] Updated weights for policy 0, policy_version 1811968 (0.0010) [2023-12-27 04:35:42,727][105692] Updated weights for policy 0, policy_version 1811978 (0.0010) [2023-12-27 04:35:42,930][105620] Updated weights for policy 1, policy_version 1815757 (0.0009) [2023-12-27 04:35:42,991][105620] Updated weights for policy 1, policy_version 1815767 (0.0008) [2023-12-27 04:35:43,054][105620] Updated weights for policy 1, policy_version 1815777 (0.0008) [2023-12-27 04:35:43,527][105692] Updated weights for policy 0, policy_version 1811988 (0.0010) [2023-12-27 04:35:43,574][105692] Updated weights for policy 0, policy_version 1811998 (0.0006) [2023-12-27 04:35:43,627][105692] Updated weights for policy 0, policy_version 1812008 (0.0005) [2023-12-27 04:35:43,752][105620] Updated weights for policy 1, policy_version 1815787 (0.0007) [2023-12-27 04:35:43,802][105620] Updated weights for policy 1, policy_version 1815797 (0.0005) [2023-12-27 04:35:43,858][105620] Updated weights for policy 1, policy_version 1815807 (0.0005) [2023-12-27 04:35:44,166][105692] Updated weights for policy 0, policy_version 1812018 (0.0005) [2023-12-27 04:35:44,222][105692] Updated weights for policy 0, policy_version 1812028 (0.0005) [2023-12-27 04:35:44,275][105692] Updated weights for policy 0, policy_version 1812038 (0.0005) [2023-12-27 04:35:44,330][105692] Updated weights for policy 0, policy_version 1812048 (0.0005) [2023-12-27 04:35:44,439][105620] Updated weights for policy 1, policy_version 1815817 (0.0005) [2023-12-27 04:35:44,504][105620] Updated weights for policy 1, policy_version 1815827 (0.0010) [2023-12-27 04:35:44,559][105620] Updated weights for policy 1, policy_version 1815837 (0.0009) [2023-12-27 04:35:44,625][105620] Updated weights for policy 1, policy_version 1815847 (0.0009) [2023-12-27 04:35:45,022][105692] Updated weights for policy 0, policy_version 1812058 (0.0008) [2023-12-27 04:35:45,088][105692] Updated weights for policy 0, policy_version 1812068 (0.0009) [2023-12-27 04:35:45,158][105692] Updated weights for policy 0, policy_version 1812078 (0.0009) [2023-12-27 04:35:45,372][105620] Updated weights for policy 1, policy_version 1815857 (0.0009) [2023-12-27 04:35:45,427][105620] Updated weights for policy 1, policy_version 1815867 (0.0008) [2023-12-27 04:35:45,478][105620] Updated weights for policy 1, policy_version 1815877 (0.0009) [2023-12-27 04:35:45,866][105692] Updated weights for policy 0, policy_version 1812088 (0.0008) [2023-12-27 04:35:45,917][105692] Updated weights for policy 0, policy_version 1812098 (0.0008) [2023-12-27 04:35:45,975][105692] Updated weights for policy 0, policy_version 1812108 (0.0008) [2023-12-27 04:35:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 928899072. Throughput: 0: 9403.9, 1: 9891.7. Samples: 928864392. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:35:46,062][104569] Avg episode reward: [(0, '8358.504'), (1, '9258.262')] [2023-12-27 04:35:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001815880_464928768.pth... [2023-12-27 04:35:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001812112_463970304.pth... [2023-12-27 04:35:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001810992_463683584.pth [2023-12-27 04:35:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001814728_464633856.pth [2023-12-27 04:35:46,240][105620] Updated weights for policy 1, policy_version 1815887 (0.0009) [2023-12-27 04:35:46,298][105620] Updated weights for policy 1, policy_version 1815897 (0.0010) [2023-12-27 04:35:46,357][105620] Updated weights for policy 1, policy_version 1815907 (0.0010) [2023-12-27 04:35:46,754][105692] Updated weights for policy 0, policy_version 1812118 (0.0008) [2023-12-27 04:35:46,801][105692] Updated weights for policy 0, policy_version 1812128 (0.0005) [2023-12-27 04:35:46,856][105692] Updated weights for policy 0, policy_version 1812138 (0.0005) [2023-12-27 04:35:47,036][105620] Updated weights for policy 1, policy_version 1815917 (0.0008) [2023-12-27 04:35:47,094][105620] Updated weights for policy 1, policy_version 1815927 (0.0006) [2023-12-27 04:35:47,153][105620] Updated weights for policy 1, policy_version 1815937 (0.0006) [2023-12-27 04:35:47,564][105692] Updated weights for policy 0, policy_version 1812148 (0.0005) [2023-12-27 04:35:47,615][105692] Updated weights for policy 0, policy_version 1812158 (0.0005) [2023-12-27 04:35:47,670][105692] Updated weights for policy 0, policy_version 1812168 (0.0006) [2023-12-27 04:35:47,778][105620] Updated weights for policy 1, policy_version 1815947 (0.0007) [2023-12-27 04:35:47,841][105620] Updated weights for policy 1, policy_version 1815957 (0.0009) [2023-12-27 04:35:47,903][105620] Updated weights for policy 1, policy_version 1815967 (0.0009) [2023-12-27 04:35:48,371][105692] Updated weights for policy 0, policy_version 1812178 (0.0009) [2023-12-27 04:35:48,419][105692] Updated weights for policy 0, policy_version 1812188 (0.0008) [2023-12-27 04:35:48,472][105692] Updated weights for policy 0, policy_version 1812198 (0.0008) [2023-12-27 04:35:48,518][105692] Updated weights for policy 0, policy_version 1812208 (0.0008) [2023-12-27 04:35:48,664][105620] Updated weights for policy 1, policy_version 1815977 (0.0009) [2023-12-27 04:35:48,723][105620] Updated weights for policy 1, policy_version 1815987 (0.0009) [2023-12-27 04:35:48,785][105620] Updated weights for policy 1, policy_version 1815997 (0.0009) [2023-12-27 04:35:48,850][105620] Updated weights for policy 1, policy_version 1816007 (0.0010) [2023-12-27 04:35:49,293][105692] Updated weights for policy 0, policy_version 1812218 (0.0008) [2023-12-27 04:35:49,353][105692] Updated weights for policy 0, policy_version 1812228 (0.0008) [2023-12-27 04:35:49,411][105692] Updated weights for policy 0, policy_version 1812238 (0.0008) [2023-12-27 04:35:49,632][105620] Updated weights for policy 1, policy_version 1816017 (0.0009) [2023-12-27 04:35:49,687][105620] Updated weights for policy 1, policy_version 1816027 (0.0009) [2023-12-27 04:35:49,743][105620] Updated weights for policy 1, policy_version 1816037 (0.0008) [2023-12-27 04:35:50,098][105692] Updated weights for policy 0, policy_version 1812248 (0.0008) [2023-12-27 04:35:50,153][105692] Updated weights for policy 0, policy_version 1812258 (0.0010) [2023-12-27 04:35:50,212][105692] Updated weights for policy 0, policy_version 1812268 (0.0009) [2023-12-27 04:35:50,531][105620] Updated weights for policy 1, policy_version 1816047 (0.0009) [2023-12-27 04:35:50,591][105620] Updated weights for policy 1, policy_version 1816057 (0.0009) [2023-12-27 04:35:50,650][105620] Updated weights for policy 1, policy_version 1816067 (0.0010) [2023-12-27 04:35:50,888][105692] Updated weights for policy 0, policy_version 1812278 (0.0007) [2023-12-27 04:35:50,952][105692] Updated weights for policy 0, policy_version 1812288 (0.0006) [2023-12-27 04:35:51,009][105692] Updated weights for policy 0, policy_version 1812298 (0.0007) [2023-12-27 04:35:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 928997376. Throughput: 0: 9427.2, 1: 9868.8. Samples: 928981976. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:35:51,063][104569] Avg episode reward: [(0, '8719.281'), (1, '9258.409')] [2023-12-27 04:35:51,452][105620] Updated weights for policy 1, policy_version 1816077 (0.0010) [2023-12-27 04:35:51,510][105620] Updated weights for policy 1, policy_version 1816087 (0.0010) [2023-12-27 04:35:51,574][105620] Updated weights for policy 1, policy_version 1816097 (0.0008) [2023-12-27 04:35:51,674][105692] Updated weights for policy 0, policy_version 1812308 (0.0009) [2023-12-27 04:35:51,733][105692] Updated weights for policy 0, policy_version 1812318 (0.0010) [2023-12-27 04:35:51,795][105692] Updated weights for policy 0, policy_version 1812328 (0.0009) [2023-12-27 04:35:52,240][105620] Updated weights for policy 1, policy_version 1816107 (0.0006) [2023-12-27 04:35:52,307][105620] Updated weights for policy 1, policy_version 1816117 (0.0007) [2023-12-27 04:35:52,370][105620] Updated weights for policy 1, policy_version 1816127 (0.0009) [2023-12-27 04:35:52,658][105692] Updated weights for policy 0, policy_version 1812338 (0.0008) [2023-12-27 04:35:52,721][105692] Updated weights for policy 0, policy_version 1812348 (0.0009) [2023-12-27 04:35:52,780][105692] Updated weights for policy 0, policy_version 1812358 (0.0009) [2023-12-27 04:35:52,843][105692] Updated weights for policy 0, policy_version 1812368 (0.0009) [2023-12-27 04:35:53,061][105620] Updated weights for policy 1, policy_version 1816137 (0.0008) [2023-12-27 04:35:53,123][105620] Updated weights for policy 1, policy_version 1816147 (0.0009) [2023-12-27 04:35:53,175][105620] Updated weights for policy 1, policy_version 1816157 (0.0009) [2023-12-27 04:35:53,233][105620] Updated weights for policy 1, policy_version 1816167 (0.0009) [2023-12-27 04:35:53,597][105692] Updated weights for policy 0, policy_version 1812378 (0.0006) [2023-12-27 04:35:53,641][105692] Updated weights for policy 0, policy_version 1812388 (0.0005) [2023-12-27 04:35:53,696][105692] Updated weights for policy 0, policy_version 1812398 (0.0005) [2023-12-27 04:35:53,906][105620] Updated weights for policy 1, policy_version 1816177 (0.0006) [2023-12-27 04:35:53,958][105620] Updated weights for policy 1, policy_version 1816187 (0.0005) [2023-12-27 04:35:54,010][105620] Updated weights for policy 1, policy_version 1816197 (0.0005) [2023-12-27 04:35:54,245][105692] Updated weights for policy 0, policy_version 1812408 (0.0008) [2023-12-27 04:35:54,304][105692] Updated weights for policy 0, policy_version 1812418 (0.0008) [2023-12-27 04:35:54,368][105692] Updated weights for policy 0, policy_version 1812428 (0.0008) [2023-12-27 04:35:54,678][105620] Updated weights for policy 1, policy_version 1816207 (0.0009) [2023-12-27 04:35:54,731][105620] Updated weights for policy 1, policy_version 1816217 (0.0010) [2023-12-27 04:35:54,782][105620] Updated weights for policy 1, policy_version 1816227 (0.0009) [2023-12-27 04:35:55,001][105692] Updated weights for policy 0, policy_version 1812438 (0.0008) [2023-12-27 04:35:55,054][105692] Updated weights for policy 0, policy_version 1812448 (0.0009) [2023-12-27 04:35:55,109][105692] Updated weights for policy 0, policy_version 1812458 (0.0009) [2023-12-27 04:35:55,542][105620] Updated weights for policy 1, policy_version 1816237 (0.0008) [2023-12-27 04:35:55,592][105620] Updated weights for policy 1, policy_version 1816247 (0.0009) [2023-12-27 04:35:55,647][105620] Updated weights for policy 1, policy_version 1816258 (0.0011) [2023-12-27 04:35:55,822][105692] Updated weights for policy 0, policy_version 1812468 (0.0009) [2023-12-27 04:35:55,870][105692] Updated weights for policy 0, policy_version 1812478 (0.0009) [2023-12-27 04:35:55,921][105692] Updated weights for policy 0, policy_version 1812488 (0.0009) [2023-12-27 04:35:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 929095680. Throughput: 0: 9513.5, 1: 9845.7. Samples: 929099752. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:35:56,063][104569] Avg episode reward: [(0, '8714.961'), (1, '9258.352')] [2023-12-27 04:35:56,448][105620] Updated weights for policy 1, policy_version 1816268 (0.0010) [2023-12-27 04:35:56,501][105620] Updated weights for policy 1, policy_version 1816278 (0.0008) [2023-12-27 04:35:56,554][105620] Updated weights for policy 1, policy_version 1816288 (0.0006) [2023-12-27 04:35:56,693][105692] Updated weights for policy 0, policy_version 1812498 (0.0009) [2023-12-27 04:35:56,747][105692] Updated weights for policy 0, policy_version 1812508 (0.0009) [2023-12-27 04:35:56,802][105692] Updated weights for policy 0, policy_version 1812518 (0.0009) [2023-12-27 04:35:56,858][105692] Updated weights for policy 0, policy_version 1812528 (0.0009) [2023-12-27 04:35:57,168][105620] Updated weights for policy 1, policy_version 1816298 (0.0007) [2023-12-27 04:35:57,218][105620] Updated weights for policy 1, policy_version 1816308 (0.0005) [2023-12-27 04:35:57,266][105620] Updated weights for policy 1, policy_version 1816318 (0.0005) [2023-12-27 04:35:57,323][105620] Updated weights for policy 1, policy_version 1816328 (0.0006) [2023-12-27 04:35:57,714][105692] Updated weights for policy 0, policy_version 1812538 (0.0009) [2023-12-27 04:35:57,769][105692] Updated weights for policy 0, policy_version 1812548 (0.0005) [2023-12-27 04:35:57,829][105692] Updated weights for policy 0, policy_version 1812558 (0.0006) [2023-12-27 04:35:58,059][105620] Updated weights for policy 1, policy_version 1816338 (0.0009) [2023-12-27 04:35:58,124][105620] Updated weights for policy 1, policy_version 1816348 (0.0008) [2023-12-27 04:35:58,195][105620] Updated weights for policy 1, policy_version 1816358 (0.0010) [2023-12-27 04:35:58,473][105692] Updated weights for policy 0, policy_version 1812568 (0.0008) [2023-12-27 04:35:58,542][105692] Updated weights for policy 0, policy_version 1812578 (0.0009) [2023-12-27 04:35:58,606][105692] Updated weights for policy 0, policy_version 1812588 (0.0008) [2023-12-27 04:35:58,982][105620] Updated weights for policy 1, policy_version 1816368 (0.0010) [2023-12-27 04:35:59,028][105620] Updated weights for policy 1, policy_version 1816378 (0.0008) [2023-12-27 04:35:59,079][105620] Updated weights for policy 1, policy_version 1816388 (0.0005) [2023-12-27 04:35:59,325][105692] Updated weights for policy 0, policy_version 1812598 (0.0007) [2023-12-27 04:35:59,383][105692] Updated weights for policy 0, policy_version 1812608 (0.0009) [2023-12-27 04:35:59,439][105692] Updated weights for policy 0, policy_version 1812618 (0.0008) [2023-12-27 04:35:59,809][105620] Updated weights for policy 1, policy_version 1816398 (0.0005) [2023-12-27 04:35:59,873][105620] Updated weights for policy 1, policy_version 1816408 (0.0006) [2023-12-27 04:35:59,930][105620] Updated weights for policy 1, policy_version 1816418 (0.0007) [2023-12-27 04:36:00,105][105692] Updated weights for policy 0, policy_version 1812628 (0.0008) [2023-12-27 04:36:00,161][105692] Updated weights for policy 0, policy_version 1812638 (0.0006) [2023-12-27 04:36:00,216][105692] Updated weights for policy 0, policy_version 1812648 (0.0008) [2023-12-27 04:36:00,621][105620] Updated weights for policy 1, policy_version 1816428 (0.0009) [2023-12-27 04:36:00,674][105620] Updated weights for policy 1, policy_version 1816438 (0.0009) [2023-12-27 04:36:00,738][105620] Updated weights for policy 1, policy_version 1816448 (0.0009) [2023-12-27 04:36:00,855][105692] Updated weights for policy 0, policy_version 1812658 (0.0010) [2023-12-27 04:36:00,907][105692] Updated weights for policy 0, policy_version 1812668 (0.0006) [2023-12-27 04:36:00,958][105692] Updated weights for policy 0, policy_version 1812678 (0.0008) [2023-12-27 04:36:01,005][105692] Updated weights for policy 0, policy_version 1812688 (0.0010) [2023-12-27 04:36:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 929193984. Throughput: 0: 9522.3, 1: 9872.4. Samples: 929157720. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:36:01,062][104569] Avg episode reward: [(0, '8805.595'), (1, '9259.069')] [2023-12-27 04:36:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001812688_464117760.pth... [2023-12-27 04:36:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001816456_465076224.pth... [2023-12-27 04:36:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001811536_463822848.pth [2023-12-27 04:36:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001815304_464781312.pth [2023-12-27 04:36:01,580][105620] Updated weights for policy 1, policy_version 1816458 (0.0009) [2023-12-27 04:36:01,640][105620] Updated weights for policy 1, policy_version 1816468 (0.0008) [2023-12-27 04:36:01,693][105620] Updated weights for policy 1, policy_version 1816478 (0.0009) [2023-12-27 04:36:01,722][105692] Updated weights for policy 0, policy_version 1812698 (0.0008) [2023-12-27 04:36:01,762][105620] Updated weights for policy 1, policy_version 1816488 (0.0006) [2023-12-27 04:36:01,780][105692] Updated weights for policy 0, policy_version 1812708 (0.0008) [2023-12-27 04:36:01,835][105692] Updated weights for policy 0, policy_version 1812718 (0.0008) [2023-12-27 04:36:02,347][105620] Updated weights for policy 1, policy_version 1816498 (0.0009) [2023-12-27 04:36:02,421][105620] Updated weights for policy 1, policy_version 1816508 (0.0009) [2023-12-27 04:36:02,483][105620] Updated weights for policy 1, policy_version 1816518 (0.0008) [2023-12-27 04:36:02,665][105692] Updated weights for policy 0, policy_version 1812728 (0.0009) [2023-12-27 04:36:02,727][105692] Updated weights for policy 0, policy_version 1812738 (0.0009) [2023-12-27 04:36:02,789][105692] Updated weights for policy 0, policy_version 1812748 (0.0009) [2023-12-27 04:36:03,208][105620] Updated weights for policy 1, policy_version 1816528 (0.0009) [2023-12-27 04:36:03,263][105620] Updated weights for policy 1, policy_version 1816538 (0.0009) [2023-12-27 04:36:03,317][105620] Updated weights for policy 1, policy_version 1816548 (0.0009) [2023-12-27 04:36:03,536][105692] Updated weights for policy 0, policy_version 1812758 (0.0010) [2023-12-27 04:36:03,584][105692] Updated weights for policy 0, policy_version 1812768 (0.0006) [2023-12-27 04:36:03,634][105692] Updated weights for policy 0, policy_version 1812778 (0.0006) [2023-12-27 04:36:04,062][105620] Updated weights for policy 1, policy_version 1816558 (0.0009) [2023-12-27 04:36:04,124][105620] Updated weights for policy 1, policy_version 1816568 (0.0010) [2023-12-27 04:36:04,189][105620] Updated weights for policy 1, policy_version 1816578 (0.0008) [2023-12-27 04:36:04,241][105692] Updated weights for policy 0, policy_version 1812788 (0.0006) [2023-12-27 04:36:04,297][105692] Updated weights for policy 0, policy_version 1812798 (0.0006) [2023-12-27 04:36:04,354][105692] Updated weights for policy 0, policy_version 1812808 (0.0011) [2023-12-27 04:36:04,803][105620] Updated weights for policy 1, policy_version 1816588 (0.0007) [2023-12-27 04:36:04,868][105620] Updated weights for policy 1, policy_version 1816598 (0.0005) [2023-12-27 04:36:04,939][105620] Updated weights for policy 1, policy_version 1816608 (0.0005) [2023-12-27 04:36:04,979][105692] Updated weights for policy 0, policy_version 1812818 (0.0009) [2023-12-27 04:36:05,024][105692] Updated weights for policy 0, policy_version 1812828 (0.0005) [2023-12-27 04:36:05,080][105692] Updated weights for policy 0, policy_version 1812838 (0.0005) [2023-12-27 04:36:05,129][105692] Updated weights for policy 0, policy_version 1812848 (0.0009) [2023-12-27 04:36:05,565][105620] Updated weights for policy 1, policy_version 1816618 (0.0006) [2023-12-27 04:36:05,630][105620] Updated weights for policy 1, policy_version 1816628 (0.0008) [2023-12-27 04:36:05,694][105620] Updated weights for policy 1, policy_version 1816638 (0.0006) [2023-12-27 04:36:05,695][105692] Updated weights for policy 0, policy_version 1812858 (0.0010) [2023-12-27 04:36:05,746][105692] Updated weights for policy 0, policy_version 1812868 (0.0010) [2023-12-27 04:36:05,758][105620] Updated weights for policy 1, policy_version 1816648 (0.0006) [2023-12-27 04:36:05,800][105692] Updated weights for policy 0, policy_version 1812878 (0.0010) [2023-12-27 04:36:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 929292288. Throughput: 0: 9585.6, 1: 9909.3. Samples: 929276452. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:36:06,062][104569] Avg episode reward: [(0, '8807.544'), (1, '9166.747')] [2023-12-27 04:36:06,456][105620] Updated weights for policy 1, policy_version 1816658 (0.0010) [2023-12-27 04:36:06,516][105620] Updated weights for policy 1, policy_version 1816668 (0.0009) [2023-12-27 04:36:06,538][105692] Updated weights for policy 0, policy_version 1812888 (0.0006) [2023-12-27 04:36:06,568][105620] Updated weights for policy 1, policy_version 1816678 (0.0009) [2023-12-27 04:36:06,602][105692] Updated weights for policy 0, policy_version 1812898 (0.0010) [2023-12-27 04:36:06,655][105692] Updated weights for policy 0, policy_version 1812908 (0.0010) [2023-12-27 04:36:07,365][105620] Updated weights for policy 1, policy_version 1816688 (0.0008) [2023-12-27 04:36:07,377][105692] Updated weights for policy 0, policy_version 1812918 (0.0010) [2023-12-27 04:36:07,417][105620] Updated weights for policy 1, policy_version 1816698 (0.0007) [2023-12-27 04:36:07,440][105692] Updated weights for policy 0, policy_version 1812928 (0.0010) [2023-12-27 04:36:07,475][105620] Updated weights for policy 1, policy_version 1816708 (0.0007) [2023-12-27 04:36:07,501][105692] Updated weights for policy 0, policy_version 1812938 (0.0008) [2023-12-27 04:36:08,138][105692] Updated weights for policy 0, policy_version 1812948 (0.0008) [2023-12-27 04:36:08,197][105692] Updated weights for policy 0, policy_version 1812958 (0.0007) [2023-12-27 04:36:08,257][105692] Updated weights for policy 0, policy_version 1812968 (0.0008) [2023-12-27 04:36:08,284][105620] Updated weights for policy 1, policy_version 1816718 (0.0007) [2023-12-27 04:36:08,341][105620] Updated weights for policy 1, policy_version 1816728 (0.0007) [2023-12-27 04:36:08,395][105620] Updated weights for policy 1, policy_version 1816738 (0.0006) [2023-12-27 04:36:08,957][105692] Updated weights for policy 0, policy_version 1812978 (0.0010) [2023-12-27 04:36:09,023][105692] Updated weights for policy 0, policy_version 1812988 (0.0009) [2023-12-27 04:36:09,077][105692] Updated weights for policy 0, policy_version 1812998 (0.0007) [2023-12-27 04:36:09,137][105620] Updated weights for policy 1, policy_version 1816748 (0.0009) [2023-12-27 04:36:09,137][105692] Updated weights for policy 0, policy_version 1813008 (0.0009) [2023-12-27 04:36:09,198][105620] Updated weights for policy 1, policy_version 1816758 (0.0009) [2023-12-27 04:36:09,265][105620] Updated weights for policy 1, policy_version 1816768 (0.0008) [2023-12-27 04:36:09,875][105692] Updated weights for policy 0, policy_version 1813018 (0.0007) [2023-12-27 04:36:09,944][105620] Updated weights for policy 1, policy_version 1816778 (0.0006) [2023-12-27 04:36:09,945][105692] Updated weights for policy 0, policy_version 1813028 (0.0007) [2023-12-27 04:36:10,008][105620] Updated weights for policy 1, policy_version 1816788 (0.0009) [2023-12-27 04:36:10,013][105692] Updated weights for policy 0, policy_version 1813038 (0.0007) [2023-12-27 04:36:10,064][105620] Updated weights for policy 1, policy_version 1816798 (0.0008) [2023-12-27 04:36:10,117][105620] Updated weights for policy 1, policy_version 1816808 (0.0008) [2023-12-27 04:36:10,721][105692] Updated weights for policy 0, policy_version 1813048 (0.0010) [2023-12-27 04:36:10,790][105692] Updated weights for policy 0, policy_version 1813058 (0.0011) [2023-12-27 04:36:10,852][105692] Updated weights for policy 0, policy_version 1813068 (0.0010) [2023-12-27 04:36:10,875][105620] Updated weights for policy 1, policy_version 1816818 (0.0007) [2023-12-27 04:36:10,940][105620] Updated weights for policy 1, policy_version 1816828 (0.0009) [2023-12-27 04:36:11,001][105620] Updated weights for policy 1, policy_version 1816838 (0.0006) [2023-12-27 04:36:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 929390592. Throughput: 0: 9737.8, 1: 9840.7. Samples: 929394396. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:36:11,062][104569] Avg episode reward: [(0, '8626.398'), (1, '8981.492')] [2023-12-27 04:36:11,612][105692] Updated weights for policy 0, policy_version 1813078 (0.0011) [2023-12-27 04:36:11,675][105692] Updated weights for policy 0, policy_version 1813088 (0.0010) [2023-12-27 04:36:11,741][105692] Updated weights for policy 0, policy_version 1813098 (0.0009) [2023-12-27 04:36:11,761][105620] Updated weights for policy 1, policy_version 1816848 (0.0006) [2023-12-27 04:36:11,812][105620] Updated weights for policy 1, policy_version 1816858 (0.0009) [2023-12-27 04:36:11,871][105620] Updated weights for policy 1, policy_version 1816868 (0.0009) [2023-12-27 04:36:12,518][105692] Updated weights for policy 0, policy_version 1813108 (0.0009) [2023-12-27 04:36:12,576][105692] Updated weights for policy 0, policy_version 1813118 (0.0009) [2023-12-27 04:36:12,635][105692] Updated weights for policy 0, policy_version 1813128 (0.0007) [2023-12-27 04:36:12,636][105620] Updated weights for policy 1, policy_version 1816878 (0.0009) [2023-12-27 04:36:12,699][105620] Updated weights for policy 1, policy_version 1816888 (0.0008) [2023-12-27 04:36:12,756][105620] Updated weights for policy 1, policy_version 1816899 (0.0013) [2023-12-27 04:36:13,220][105692] Updated weights for policy 0, policy_version 1813138 (0.0006) [2023-12-27 04:36:13,273][105692] Updated weights for policy 0, policy_version 1813148 (0.0009) [2023-12-27 04:36:13,329][105692] Updated weights for policy 0, policy_version 1813158 (0.0010) [2023-12-27 04:36:13,387][105692] Updated weights for policy 0, policy_version 1813168 (0.0009) [2023-12-27 04:36:13,465][105620] Updated weights for policy 1, policy_version 1816909 (0.0009) [2023-12-27 04:36:13,533][105620] Updated weights for policy 1, policy_version 1816919 (0.0009) [2023-12-27 04:36:13,590][105620] Updated weights for policy 1, policy_version 1816929 (0.0009) [2023-12-27 04:36:14,048][105692] Updated weights for policy 0, policy_version 1813178 (0.0009) [2023-12-27 04:36:14,108][105692] Updated weights for policy 0, policy_version 1813188 (0.0010) [2023-12-27 04:36:14,173][105692] Updated weights for policy 0, policy_version 1813198 (0.0009) [2023-12-27 04:36:14,291][105620] Updated weights for policy 1, policy_version 1816939 (0.0009) [2023-12-27 04:36:14,349][105620] Updated weights for policy 1, policy_version 1816949 (0.0009) [2023-12-27 04:36:14,419][105620] Updated weights for policy 1, policy_version 1816959 (0.0009) [2023-12-27 04:36:14,869][105692] Updated weights for policy 0, policy_version 1813208 (0.0008) [2023-12-27 04:36:14,930][105692] Updated weights for policy 0, policy_version 1813218 (0.0007) [2023-12-27 04:36:14,995][105692] Updated weights for policy 0, policy_version 1813228 (0.0007) [2023-12-27 04:36:15,207][105620] Updated weights for policy 1, policy_version 1816969 (0.0009) [2023-12-27 04:36:15,260][105620] Updated weights for policy 1, policy_version 1816979 (0.0009) [2023-12-27 04:36:15,323][105620] Updated weights for policy 1, policy_version 1816989 (0.0009) [2023-12-27 04:36:15,392][105620] Updated weights for policy 1, policy_version 1816999 (0.0009) [2023-12-27 04:36:15,739][105692] Updated weights for policy 0, policy_version 1813238 (0.0010) [2023-12-27 04:36:15,796][105692] Updated weights for policy 0, policy_version 1813248 (0.0008) [2023-12-27 04:36:15,849][105692] Updated weights for policy 0, policy_version 1813258 (0.0009) [2023-12-27 04:36:16,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 929480704. Throughput: 0: 9795.0, 1: 9747.7. Samples: 929451432. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:36:16,062][104569] Avg episode reward: [(0, '8532.225'), (1, '8798.146')] [2023-12-27 04:36:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001813264_464265216.pth... [2023-12-27 04:36:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001812112_463970304.pth [2023-12-27 04:36:16,126][105620] Updated weights for policy 1, policy_version 1817009 (0.0009) [2023-12-27 04:36:16,187][105620] Updated weights for policy 1, policy_version 1817019 (0.0009) [2023-12-27 04:36:16,239][105620] Updated weights for policy 1, policy_version 1817029 (0.0009) [2023-12-27 04:36:16,253][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001817032_465223680.pth... [2023-12-27 04:36:16,258][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001815880_464928768.pth [2023-12-27 04:36:16,689][105692] Updated weights for policy 0, policy_version 1813268 (0.0009) [2023-12-27 04:36:16,750][105692] Updated weights for policy 0, policy_version 1813278 (0.0009) [2023-12-27 04:36:16,811][105692] Updated weights for policy 0, policy_version 1813288 (0.0009) [2023-12-27 04:36:16,858][105620] Updated weights for policy 1, policy_version 1817039 (0.0008) [2023-12-27 04:36:16,917][105620] Updated weights for policy 1, policy_version 1817049 (0.0009) [2023-12-27 04:36:16,979][105620] Updated weights for policy 1, policy_version 1817059 (0.0007) [2023-12-27 04:36:17,581][105692] Updated weights for policy 0, policy_version 1813298 (0.0007) [2023-12-27 04:36:17,638][105692] Updated weights for policy 0, policy_version 1813308 (0.0009) [2023-12-27 04:36:17,673][105620] Updated weights for policy 1, policy_version 1817069 (0.0006) [2023-12-27 04:36:17,702][105692] Updated weights for policy 0, policy_version 1813318 (0.0008) [2023-12-27 04:36:17,729][105620] Updated weights for policy 1, policy_version 1817079 (0.0006) [2023-12-27 04:36:17,764][105692] Updated weights for policy 0, policy_version 1813328 (0.0008) [2023-12-27 04:36:17,792][105620] Updated weights for policy 1, policy_version 1817089 (0.0007) [2023-12-27 04:36:18,482][105692] Updated weights for policy 0, policy_version 1813338 (0.0009) [2023-12-27 04:36:18,497][105620] Updated weights for policy 1, policy_version 1817099 (0.0008) [2023-12-27 04:36:18,531][105692] Updated weights for policy 0, policy_version 1813348 (0.0005) [2023-12-27 04:36:18,557][105620] Updated weights for policy 1, policy_version 1817109 (0.0008) [2023-12-27 04:36:18,586][105692] Updated weights for policy 0, policy_version 1813358 (0.0006) [2023-12-27 04:36:18,615][105620] Updated weights for policy 1, policy_version 1817119 (0.0006) [2023-12-27 04:36:19,203][105620] Updated weights for policy 1, policy_version 1817129 (0.0006) [2023-12-27 04:36:19,265][105620] Updated weights for policy 1, policy_version 1817139 (0.0009) [2023-12-27 04:36:19,319][105620] Updated weights for policy 1, policy_version 1817149 (0.0007) [2023-12-27 04:36:19,379][105620] Updated weights for policy 1, policy_version 1817159 (0.0009) [2023-12-27 04:36:19,447][105692] Updated weights for policy 0, policy_version 1813368 (0.0006) [2023-12-27 04:36:19,514][105692] Updated weights for policy 0, policy_version 1813378 (0.0007) [2023-12-27 04:36:19,578][105692] Updated weights for policy 0, policy_version 1813388 (0.0006) [2023-12-27 04:36:20,176][105620] Updated weights for policy 1, policy_version 1817169 (0.0010) [2023-12-27 04:36:20,236][105620] Updated weights for policy 1, policy_version 1817179 (0.0009) [2023-12-27 04:36:20,273][105692] Updated weights for policy 0, policy_version 1813398 (0.0008) [2023-12-27 04:36:20,293][105620] Updated weights for policy 1, policy_version 1817189 (0.0007) [2023-12-27 04:36:20,340][105692] Updated weights for policy 0, policy_version 1813408 (0.0008) [2023-12-27 04:36:20,404][105692] Updated weights for policy 0, policy_version 1813418 (0.0008) [2023-12-27 04:36:21,062][104569] Fps is (10 sec: 18022.1, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 929570816. Throughput: 0: 9824.9, 1: 9668.9. Samples: 929566624. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:36:21,063][104569] Avg episode reward: [(0, '8263.512'), (1, '8982.607')] [2023-12-27 04:36:21,078][105620] Updated weights for policy 1, policy_version 1817199 (0.0008) [2023-12-27 04:36:21,142][105620] Updated weights for policy 1, policy_version 1817209 (0.0009) [2023-12-27 04:36:21,173][105692] Updated weights for policy 0, policy_version 1813428 (0.0009) [2023-12-27 04:36:21,204][105620] Updated weights for policy 1, policy_version 1817219 (0.0008) [2023-12-27 04:36:21,235][105692] Updated weights for policy 0, policy_version 1813438 (0.0007) [2023-12-27 04:36:21,298][105692] Updated weights for policy 0, policy_version 1813448 (0.0010) [2023-12-27 04:36:21,933][105620] Updated weights for policy 1, policy_version 1817229 (0.0008) [2023-12-27 04:36:21,999][105620] Updated weights for policy 1, policy_version 1817239 (0.0009) [2023-12-27 04:36:22,051][105620] Updated weights for policy 1, policy_version 1817249 (0.0009) [2023-12-27 04:36:22,096][105692] Updated weights for policy 0, policy_version 1813458 (0.0009) [2023-12-27 04:36:22,148][105692] Updated weights for policy 0, policy_version 1813468 (0.0009) [2023-12-27 04:36:22,200][105692] Updated weights for policy 0, policy_version 1813478 (0.0009) [2023-12-27 04:36:22,251][105692] Updated weights for policy 0, policy_version 1813488 (0.0009) [2023-12-27 04:36:22,818][105620] Updated weights for policy 1, policy_version 1817259 (0.0009) [2023-12-27 04:36:22,875][105620] Updated weights for policy 1, policy_version 1817269 (0.0009) [2023-12-27 04:36:22,936][105620] Updated weights for policy 1, policy_version 1817279 (0.0009) [2023-12-27 04:36:23,041][105692] Updated weights for policy 0, policy_version 1813498 (0.0009) [2023-12-27 04:36:23,089][105692] Updated weights for policy 0, policy_version 1813508 (0.0009) [2023-12-27 04:36:23,137][105692] Updated weights for policy 0, policy_version 1813518 (0.0010) [2023-12-27 04:36:23,692][105620] Updated weights for policy 1, policy_version 1817289 (0.0009) [2023-12-27 04:36:23,748][105620] Updated weights for policy 1, policy_version 1817299 (0.0005) [2023-12-27 04:36:23,804][105620] Updated weights for policy 1, policy_version 1817309 (0.0005) [2023-12-27 04:36:23,862][105620] Updated weights for policy 1, policy_version 1817319 (0.0005) [2023-12-27 04:36:23,923][105692] Updated weights for policy 0, policy_version 1813528 (0.0008) [2023-12-27 04:36:23,976][105692] Updated weights for policy 0, policy_version 1813538 (0.0009) [2023-12-27 04:36:24,040][105692] Updated weights for policy 0, policy_version 1813548 (0.0008) [2023-12-27 04:36:24,475][105620] Updated weights for policy 1, policy_version 1817329 (0.0007) [2023-12-27 04:36:24,540][105620] Updated weights for policy 1, policy_version 1817339 (0.0005) [2023-12-27 04:36:24,599][105620] Updated weights for policy 1, policy_version 1817349 (0.0007) [2023-12-27 04:36:24,815][105692] Updated weights for policy 0, policy_version 1813558 (0.0010) [2023-12-27 04:36:24,870][105692] Updated weights for policy 0, policy_version 1813568 (0.0010) [2023-12-27 04:36:24,935][105692] Updated weights for policy 0, policy_version 1813578 (0.0010) [2023-12-27 04:36:25,209][105620] Updated weights for policy 1, policy_version 1817359 (0.0010) [2023-12-27 04:36:25,260][105620] Updated weights for policy 1, policy_version 1817369 (0.0010) [2023-12-27 04:36:25,305][105620] Updated weights for policy 1, policy_version 1817379 (0.0010) [2023-12-27 04:36:25,634][105692] Updated weights for policy 0, policy_version 1813588 (0.0010) [2023-12-27 04:36:25,691][105692] Updated weights for policy 0, policy_version 1813598 (0.0010) [2023-12-27 04:36:25,749][105692] Updated weights for policy 0, policy_version 1813608 (0.0010) [2023-12-27 04:36:25,957][105620] Updated weights for policy 1, policy_version 1817389 (0.0008) [2023-12-27 04:36:26,015][105620] Updated weights for policy 1, policy_version 1817399 (0.0005) [2023-12-27 04:36:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 929669120. Throughput: 0: 9687.0, 1: 9729.2. Samples: 929680864. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:36:26,062][104569] Avg episode reward: [(0, '8721.113'), (1, '9258.409')] [2023-12-27 04:36:26,079][105620] Updated weights for policy 1, policy_version 1817409 (0.0005) [2023-12-27 04:36:26,469][105692] Updated weights for policy 0, policy_version 1813618 (0.0010) [2023-12-27 04:36:26,537][105692] Updated weights for policy 0, policy_version 1813628 (0.0011) [2023-12-27 04:36:26,598][105692] Updated weights for policy 0, policy_version 1813638 (0.0010) [2023-12-27 04:36:26,663][105692] Updated weights for policy 0, policy_version 1813648 (0.0010) [2023-12-27 04:36:26,760][105620] Updated weights for policy 1, policy_version 1817419 (0.0010) [2023-12-27 04:36:26,815][105620] Updated weights for policy 1, policy_version 1817429 (0.0010) [2023-12-27 04:36:26,863][105620] Updated weights for policy 1, policy_version 1817439 (0.0010) [2023-12-27 04:36:27,225][105692] Updated weights for policy 0, policy_version 1813658 (0.0010) [2023-12-27 04:36:27,280][105692] Updated weights for policy 0, policy_version 1813668 (0.0010) [2023-12-27 04:36:27,350][105692] Updated weights for policy 0, policy_version 1813678 (0.0011) [2023-12-27 04:36:27,539][105620] Updated weights for policy 1, policy_version 1817449 (0.0010) [2023-12-27 04:36:27,599][105620] Updated weights for policy 1, policy_version 1817459 (0.0008) [2023-12-27 04:36:27,660][105620] Updated weights for policy 1, policy_version 1817469 (0.0010) [2023-12-27 04:36:27,708][105620] Updated weights for policy 1, policy_version 1817479 (0.0010) [2023-12-27 04:36:28,080][105692] Updated weights for policy 0, policy_version 1813688 (0.0008) [2023-12-27 04:36:28,130][105692] Updated weights for policy 0, policy_version 1813698 (0.0008) [2023-12-27 04:36:28,192][105692] Updated weights for policy 0, policy_version 1813708 (0.0008) [2023-12-27 04:36:28,385][105620] Updated weights for policy 1, policy_version 1817489 (0.0010) [2023-12-27 04:36:28,452][105620] Updated weights for policy 1, policy_version 1817499 (0.0007) [2023-12-27 04:36:28,513][105620] Updated weights for policy 1, policy_version 1817509 (0.0010) [2023-12-27 04:36:28,842][105692] Updated weights for policy 0, policy_version 1813718 (0.0006) [2023-12-27 04:36:28,900][105692] Updated weights for policy 0, policy_version 1813728 (0.0008) [2023-12-27 04:36:28,951][105692] Updated weights for policy 0, policy_version 1813738 (0.0009) [2023-12-27 04:36:29,204][105620] Updated weights for policy 1, policy_version 1817519 (0.0007) [2023-12-27 04:36:29,270][105620] Updated weights for policy 1, policy_version 1817529 (0.0008) [2023-12-27 04:36:29,331][105620] Updated weights for policy 1, policy_version 1817539 (0.0010) [2023-12-27 04:36:29,698][105692] Updated weights for policy 0, policy_version 1813748 (0.0007) [2023-12-27 04:36:29,760][105692] Updated weights for policy 0, policy_version 1813758 (0.0005) [2023-12-27 04:36:29,821][105692] Updated weights for policy 0, policy_version 1813768 (0.0007) [2023-12-27 04:36:30,103][105620] Updated weights for policy 1, policy_version 1817549 (0.0008) [2023-12-27 04:36:30,154][105620] Updated weights for policy 1, policy_version 1817559 (0.0005) [2023-12-27 04:36:30,203][105620] Updated weights for policy 1, policy_version 1817569 (0.0005) [2023-12-27 04:36:30,563][105692] Updated weights for policy 0, policy_version 1813778 (0.0008) [2023-12-27 04:36:30,615][105692] Updated weights for policy 0, policy_version 1813788 (0.0009) [2023-12-27 04:36:30,676][105692] Updated weights for policy 0, policy_version 1813798 (0.0009) [2023-12-27 04:36:30,738][105692] Updated weights for policy 0, policy_version 1813808 (0.0007) [2023-12-27 04:36:30,800][105620] Updated weights for policy 1, policy_version 1817579 (0.0007) [2023-12-27 04:36:30,848][105620] Updated weights for policy 1, policy_version 1817589 (0.0010) [2023-12-27 04:36:30,896][105620] Updated weights for policy 1, policy_version 1817599 (0.0010) [2023-12-27 04:36:31,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 929775616. Throughput: 0: 9762.7, 1: 9721.1. Samples: 929741160. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:36:31,062][104569] Avg episode reward: [(0, '8530.278'), (1, '9350.830')] [2023-12-27 04:36:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001813808_464404480.pth... [2023-12-27 04:36:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001817608_465371136.pth... [2023-12-27 04:36:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001812688_464117760.pth [2023-12-27 04:36:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001816456_465076224.pth [2023-12-27 04:36:31,452][105692] Updated weights for policy 0, policy_version 1813818 (0.0010) [2023-12-27 04:36:31,507][105692] Updated weights for policy 0, policy_version 1813829 (0.0010) [2023-12-27 04:36:31,563][105692] Updated weights for policy 0, policy_version 1813839 (0.0008) [2023-12-27 04:36:31,611][105620] Updated weights for policy 1, policy_version 1817609 (0.0010) [2023-12-27 04:36:31,677][105620] Updated weights for policy 1, policy_version 1817619 (0.0008) [2023-12-27 04:36:31,741][105620] Updated weights for policy 1, policy_version 1817629 (0.0010) [2023-12-27 04:36:31,803][105620] Updated weights for policy 1, policy_version 1817639 (0.0009) [2023-12-27 04:36:32,242][105692] Updated weights for policy 0, policy_version 1813849 (0.0011) [2023-12-27 04:36:32,304][105692] Updated weights for policy 0, policy_version 1813859 (0.0011) [2023-12-27 04:36:32,363][105692] Updated weights for policy 0, policy_version 1813869 (0.0011) [2023-12-27 04:36:32,533][105620] Updated weights for policy 1, policy_version 1817649 (0.0010) [2023-12-27 04:36:32,585][105620] Updated weights for policy 1, policy_version 1817659 (0.0010) [2023-12-27 04:36:32,647][105620] Updated weights for policy 1, policy_version 1817669 (0.0010) [2023-12-27 04:36:33,097][105692] Updated weights for policy 0, policy_version 1813879 (0.0007) [2023-12-27 04:36:33,142][105692] Updated weights for policy 0, policy_version 1813889 (0.0005) [2023-12-27 04:36:33,187][105692] Updated weights for policy 0, policy_version 1813899 (0.0005) [2023-12-27 04:36:33,342][105620] Updated weights for policy 1, policy_version 1817679 (0.0010) [2023-12-27 04:36:33,390][105620] Updated weights for policy 1, policy_version 1817689 (0.0010) [2023-12-27 04:36:33,442][105620] Updated weights for policy 1, policy_version 1817699 (0.0010) [2023-12-27 04:36:33,728][105692] Updated weights for policy 0, policy_version 1813909 (0.0007) [2023-12-27 04:36:33,787][105692] Updated weights for policy 0, policy_version 1813919 (0.0010) [2023-12-27 04:36:33,831][105692] Updated weights for policy 0, policy_version 1813929 (0.0010) [2023-12-27 04:36:34,110][105620] Updated weights for policy 1, policy_version 1817709 (0.0009) [2023-12-27 04:36:34,171][105620] Updated weights for policy 1, policy_version 1817719 (0.0008) [2023-12-27 04:36:34,237][105620] Updated weights for policy 1, policy_version 1817729 (0.0008) [2023-12-27 04:36:34,569][105692] Updated weights for policy 0, policy_version 1813939 (0.0010) [2023-12-27 04:36:34,626][105692] Updated weights for policy 0, policy_version 1813949 (0.0011) [2023-12-27 04:36:34,679][105692] Updated weights for policy 0, policy_version 1813959 (0.0011) [2023-12-27 04:36:34,846][105620] Updated weights for policy 1, policy_version 1817739 (0.0008) [2023-12-27 04:36:34,902][105620] Updated weights for policy 1, policy_version 1817749 (0.0008) [2023-12-27 04:36:34,963][105620] Updated weights for policy 1, policy_version 1817759 (0.0008) [2023-12-27 04:36:35,396][105692] Updated weights for policy 0, policy_version 1813969 (0.0011) [2023-12-27 04:36:35,455][105692] Updated weights for policy 0, policy_version 1813979 (0.0010) [2023-12-27 04:36:35,514][105692] Updated weights for policy 0, policy_version 1813989 (0.0011) [2023-12-27 04:36:35,572][105692] Updated weights for policy 0, policy_version 1813999 (0.0010) [2023-12-27 04:36:35,748][105620] Updated weights for policy 1, policy_version 1817769 (0.0008) [2023-12-27 04:36:35,803][105620] Updated weights for policy 1, policy_version 1817779 (0.0005) [2023-12-27 04:36:35,856][105620] Updated weights for policy 1, policy_version 1817789 (0.0005) [2023-12-27 04:36:35,911][105620] Updated weights for policy 1, policy_version 1817799 (0.0005) [2023-12-27 04:36:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 929873920. Throughput: 0: 9770.2, 1: 9792.5. Samples: 929862296. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:36:36,062][104569] Avg episode reward: [(0, '8531.048'), (1, '9350.780')] [2023-12-27 04:36:36,341][105692] Updated weights for policy 0, policy_version 1814009 (0.0011) [2023-12-27 04:36:36,409][105692] Updated weights for policy 0, policy_version 1814019 (0.0011) [2023-12-27 04:36:36,479][105692] Updated weights for policy 0, policy_version 1814029 (0.0011) [2023-12-27 04:36:36,515][105620] Updated weights for policy 1, policy_version 1817809 (0.0008) [2023-12-27 04:36:36,575][105620] Updated weights for policy 1, policy_version 1817819 (0.0008) [2023-12-27 04:36:36,634][105620] Updated weights for policy 1, policy_version 1817829 (0.0009) [2023-12-27 04:36:37,070][105692] Updated weights for policy 0, policy_version 1814039 (0.0011) [2023-12-27 04:36:37,125][105692] Updated weights for policy 0, policy_version 1814049 (0.0011) [2023-12-27 04:36:37,177][105692] Updated weights for policy 0, policy_version 1814059 (0.0010) [2023-12-27 04:36:37,277][105620] Updated weights for policy 1, policy_version 1817839 (0.0007) [2023-12-27 04:36:37,326][105620] Updated weights for policy 1, policy_version 1817849 (0.0005) [2023-12-27 04:36:37,384][105620] Updated weights for policy 1, policy_version 1817859 (0.0006) [2023-12-27 04:36:37,883][105692] Updated weights for policy 0, policy_version 1814069 (0.0011) [2023-12-27 04:36:37,938][105692] Updated weights for policy 0, policy_version 1814079 (0.0007) [2023-12-27 04:36:38,003][105692] Updated weights for policy 0, policy_version 1814089 (0.0007) [2023-12-27 04:36:38,022][105620] Updated weights for policy 1, policy_version 1817869 (0.0010) [2023-12-27 04:36:38,082][105620] Updated weights for policy 1, policy_version 1817879 (0.0010) [2023-12-27 04:36:38,143][105620] Updated weights for policy 1, policy_version 1817889 (0.0010) [2023-12-27 04:36:38,600][105692] Updated weights for policy 0, policy_version 1814099 (0.0007) [2023-12-27 04:36:38,656][105692] Updated weights for policy 0, policy_version 1814109 (0.0006) [2023-12-27 04:36:38,715][105692] Updated weights for policy 0, policy_version 1814119 (0.0005) [2023-12-27 04:36:38,886][105620] Updated weights for policy 1, policy_version 1817899 (0.0010) [2023-12-27 04:36:38,939][105620] Updated weights for policy 1, policy_version 1817909 (0.0011) [2023-12-27 04:36:38,999][105620] Updated weights for policy 1, policy_version 1817919 (0.0011) [2023-12-27 04:36:39,411][105692] Updated weights for policy 0, policy_version 1814129 (0.0006) [2023-12-27 04:36:39,474][105692] Updated weights for policy 0, policy_version 1814139 (0.0008) [2023-12-27 04:36:39,539][105692] Updated weights for policy 0, policy_version 1814149 (0.0008) [2023-12-27 04:36:39,601][105692] Updated weights for policy 0, policy_version 1814159 (0.0008) [2023-12-27 04:36:39,758][105620] Updated weights for policy 1, policy_version 1817929 (0.0010) [2023-12-27 04:36:39,820][105620] Updated weights for policy 1, policy_version 1817939 (0.0006) [2023-12-27 04:36:39,887][105620] Updated weights for policy 1, policy_version 1817949 (0.0009) [2023-12-27 04:36:39,953][105620] Updated weights for policy 1, policy_version 1817959 (0.0009) [2023-12-27 04:36:40,414][105692] Updated weights for policy 0, policy_version 1814169 (0.0008) [2023-12-27 04:36:40,473][105692] Updated weights for policy 0, policy_version 1814179 (0.0009) [2023-12-27 04:36:40,534][105692] Updated weights for policy 0, policy_version 1814189 (0.0010) [2023-12-27 04:36:40,654][105620] Updated weights for policy 1, policy_version 1817969 (0.0008) [2023-12-27 04:36:40,712][105620] Updated weights for policy 1, policy_version 1817979 (0.0009) [2023-12-27 04:36:40,770][105620] Updated weights for policy 1, policy_version 1817989 (0.0010) [2023-12-27 04:36:41,063][104569] Fps is (10 sec: 19659.1, 60 sec: 19524.0, 300 sec: 19577.4). Total num frames: 929972224. Throughput: 0: 9754.1, 1: 9808.3. Samples: 929980080. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:36:41,063][104569] Avg episode reward: [(0, '8530.077'), (1, '9258.235')] [2023-12-27 04:36:41,220][105692] Updated weights for policy 0, policy_version 1814199 (0.0009) [2023-12-27 04:36:41,284][105692] Updated weights for policy 0, policy_version 1814209 (0.0011) [2023-12-27 04:36:41,344][105692] Updated weights for policy 0, policy_version 1814219 (0.0011) [2023-12-27 04:36:41,690][105620] Updated weights for policy 1, policy_version 1817999 (0.0009) [2023-12-27 04:36:41,756][105620] Updated weights for policy 1, policy_version 1818009 (0.0008) [2023-12-27 04:36:41,822][105620] Updated weights for policy 1, policy_version 1818019 (0.0008) [2023-12-27 04:36:42,048][105692] Updated weights for policy 0, policy_version 1814229 (0.0008) [2023-12-27 04:36:42,120][105692] Updated weights for policy 0, policy_version 1814239 (0.0006) [2023-12-27 04:36:42,187][105692] Updated weights for policy 0, policy_version 1814249 (0.0009) [2023-12-27 04:36:42,559][105620] Updated weights for policy 1, policy_version 1818029 (0.0007) [2023-12-27 04:36:42,628][105620] Updated weights for policy 1, policy_version 1818039 (0.0008) [2023-12-27 04:36:42,689][105620] Updated weights for policy 1, policy_version 1818049 (0.0009) [2023-12-27 04:36:42,896][105692] Updated weights for policy 0, policy_version 1814259 (0.0008) [2023-12-27 04:36:42,954][105692] Updated weights for policy 0, policy_version 1814269 (0.0005) [2023-12-27 04:36:43,009][105692] Updated weights for policy 0, policy_version 1814279 (0.0008) [2023-12-27 04:36:43,357][105620] Updated weights for policy 1, policy_version 1818059 (0.0007) [2023-12-27 04:36:43,412][105620] Updated weights for policy 1, policy_version 1818069 (0.0006) [2023-12-27 04:36:43,481][105620] Updated weights for policy 1, policy_version 1818079 (0.0008) [2023-12-27 04:36:43,567][105692] Updated weights for policy 0, policy_version 1814289 (0.0008) [2023-12-27 04:36:43,626][105692] Updated weights for policy 0, policy_version 1814299 (0.0007) [2023-12-27 04:36:43,686][105692] Updated weights for policy 0, policy_version 1814309 (0.0005) [2023-12-27 04:36:43,741][105692] Updated weights for policy 0, policy_version 1814319 (0.0007) [2023-12-27 04:36:44,105][105620] Updated weights for policy 1, policy_version 1818089 (0.0008) [2023-12-27 04:36:44,153][105620] Updated weights for policy 1, policy_version 1818099 (0.0006) [2023-12-27 04:36:44,208][105620] Updated weights for policy 1, policy_version 1818109 (0.0009) [2023-12-27 04:36:44,260][105620] Updated weights for policy 1, policy_version 1818119 (0.0010) [2023-12-27 04:36:44,466][105692] Updated weights for policy 0, policy_version 1814329 (0.0010) [2023-12-27 04:36:44,518][105692] Updated weights for policy 0, policy_version 1814339 (0.0010) [2023-12-27 04:36:44,580][105692] Updated weights for policy 0, policy_version 1814349 (0.0011) [2023-12-27 04:36:44,949][105620] Updated weights for policy 1, policy_version 1818129 (0.0006) [2023-12-27 04:36:45,018][105620] Updated weights for policy 1, policy_version 1818139 (0.0007) [2023-12-27 04:36:45,076][105620] Updated weights for policy 1, policy_version 1818149 (0.0009) [2023-12-27 04:36:45,333][105692] Updated weights for policy 0, policy_version 1814359 (0.0010) [2023-12-27 04:36:45,390][105692] Updated weights for policy 0, policy_version 1814369 (0.0008) [2023-12-27 04:36:45,443][105692] Updated weights for policy 0, policy_version 1814379 (0.0011) [2023-12-27 04:36:45,820][105620] Updated weights for policy 1, policy_version 1818159 (0.0008) [2023-12-27 04:36:45,877][105620] Updated weights for policy 1, policy_version 1818169 (0.0008) [2023-12-27 04:36:45,942][105620] Updated weights for policy 1, policy_version 1818179 (0.0009) [2023-12-27 04:36:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 930070528. Throughput: 0: 9816.5, 1: 9793.4. Samples: 930040168. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:36:46,063][104569] Avg episode reward: [(0, '8352.801'), (1, '9165.779')] [2023-12-27 04:36:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001814384_464551936.pth... [2023-12-27 04:36:46,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001818184_465518592.pth... [2023-12-27 04:36:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001813264_464265216.pth [2023-12-27 04:36:46,094][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001817032_465223680.pth [2023-12-27 04:36:46,125][105692] Updated weights for policy 0, policy_version 1814389 (0.0010) [2023-12-27 04:36:46,184][105692] Updated weights for policy 0, policy_version 1814399 (0.0009) [2023-12-27 04:36:46,240][105692] Updated weights for policy 0, policy_version 1814409 (0.0009) [2023-12-27 04:36:46,625][105620] Updated weights for policy 1, policy_version 1818189 (0.0007) [2023-12-27 04:36:46,683][105620] Updated weights for policy 1, policy_version 1818199 (0.0005) [2023-12-27 04:36:46,749][105620] Updated weights for policy 1, policy_version 1818209 (0.0005) [2023-12-27 04:36:46,952][105692] Updated weights for policy 0, policy_version 1814419 (0.0009) [2023-12-27 04:36:47,015][105692] Updated weights for policy 0, policy_version 1814429 (0.0006) [2023-12-27 04:36:47,084][105692] Updated weights for policy 0, policy_version 1814439 (0.0005) [2023-12-27 04:36:47,410][105620] Updated weights for policy 1, policy_version 1818219 (0.0006) [2023-12-27 04:36:47,468][105620] Updated weights for policy 1, policy_version 1818229 (0.0007) [2023-12-27 04:36:47,513][105620] Updated weights for policy 1, policy_version 1818239 (0.0005) [2023-12-27 04:36:47,627][105692] Updated weights for policy 0, policy_version 1814449 (0.0006) [2023-12-27 04:36:47,676][105692] Updated weights for policy 0, policy_version 1814459 (0.0009) [2023-12-27 04:36:47,726][105692] Updated weights for policy 0, policy_version 1814469 (0.0007) [2023-12-27 04:36:47,787][105692] Updated weights for policy 0, policy_version 1814479 (0.0010) [2023-12-27 04:36:48,224][105620] Updated weights for policy 1, policy_version 1818249 (0.0005) [2023-12-27 04:36:48,284][105620] Updated weights for policy 1, policy_version 1818259 (0.0009) [2023-12-27 04:36:48,340][105620] Updated weights for policy 1, policy_version 1818269 (0.0009) [2023-12-27 04:36:48,401][105620] Updated weights for policy 1, policy_version 1818279 (0.0008) [2023-12-27 04:36:48,449][105692] Updated weights for policy 0, policy_version 1814489 (0.0011) [2023-12-27 04:36:48,499][105692] Updated weights for policy 0, policy_version 1814499 (0.0007) [2023-12-27 04:36:48,560][105692] Updated weights for policy 0, policy_version 1814509 (0.0006) [2023-12-27 04:36:49,065][105620] Updated weights for policy 1, policy_version 1818289 (0.0007) [2023-12-27 04:36:49,119][105620] Updated weights for policy 1, policy_version 1818299 (0.0005) [2023-12-27 04:36:49,163][105620] Updated weights for policy 1, policy_version 1818309 (0.0005) [2023-12-27 04:36:49,218][105692] Updated weights for policy 0, policy_version 1814519 (0.0007) [2023-12-27 04:36:49,286][105692] Updated weights for policy 0, policy_version 1814529 (0.0010) [2023-12-27 04:36:49,354][105692] Updated weights for policy 0, policy_version 1814539 (0.0010) [2023-12-27 04:36:49,832][105620] Updated weights for policy 1, policy_version 1818319 (0.0006) [2023-12-27 04:36:49,892][105620] Updated weights for policy 1, policy_version 1818329 (0.0009) [2023-12-27 04:36:49,949][105692] Updated weights for policy 0, policy_version 1814549 (0.0011) [2023-12-27 04:36:49,953][105620] Updated weights for policy 1, policy_version 1818339 (0.0008) [2023-12-27 04:36:50,006][105692] Updated weights for policy 0, policy_version 1814559 (0.0011) [2023-12-27 04:36:50,073][105692] Updated weights for policy 0, policy_version 1814569 (0.0010) [2023-12-27 04:36:50,683][105692] Updated weights for policy 0, policy_version 1814579 (0.0009) [2023-12-27 04:36:50,739][105692] Updated weights for policy 0, policy_version 1814589 (0.0005) [2023-12-27 04:36:50,750][105620] Updated weights for policy 1, policy_version 1818349 (0.0006) [2023-12-27 04:36:50,795][105692] Updated weights for policy 0, policy_version 1814599 (0.0007) [2023-12-27 04:36:50,821][105620] Updated weights for policy 1, policy_version 1818359 (0.0006) [2023-12-27 04:36:50,892][105620] Updated weights for policy 1, policy_version 1818369 (0.0007) [2023-12-27 04:36:51,062][104569] Fps is (10 sec: 20481.7, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 930177024. Throughput: 0: 9856.2, 1: 9824.6. Samples: 930162088. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:36:51,063][104569] Avg episode reward: [(0, '8447.668'), (1, '9258.207')] [2023-12-27 04:36:51,462][105692] Updated weights for policy 0, policy_version 1814609 (0.0007) [2023-12-27 04:36:51,524][105692] Updated weights for policy 0, policy_version 1814619 (0.0010) [2023-12-27 04:36:51,582][105692] Updated weights for policy 0, policy_version 1814629 (0.0010) [2023-12-27 04:36:51,643][105692] Updated weights for policy 0, policy_version 1814639 (0.0011) [2023-12-27 04:36:51,666][105620] Updated weights for policy 1, policy_version 1818379 (0.0009) [2023-12-27 04:36:51,730][105620] Updated weights for policy 1, policy_version 1818389 (0.0011) [2023-12-27 04:36:51,790][105620] Updated weights for policy 1, policy_version 1818399 (0.0010) [2023-12-27 04:36:52,388][105692] Updated weights for policy 0, policy_version 1814649 (0.0008) [2023-12-27 04:36:52,453][105692] Updated weights for policy 0, policy_version 1814659 (0.0005) [2023-12-27 04:36:52,515][105620] Updated weights for policy 1, policy_version 1818409 (0.0010) [2023-12-27 04:36:52,516][105692] Updated weights for policy 0, policy_version 1814669 (0.0008) [2023-12-27 04:36:52,574][105620] Updated weights for policy 1, policy_version 1818419 (0.0006) [2023-12-27 04:36:52,634][105620] Updated weights for policy 1, policy_version 1818429 (0.0007) [2023-12-27 04:36:52,691][105620] Updated weights for policy 1, policy_version 1818439 (0.0010) [2023-12-27 04:36:53,164][105692] Updated weights for policy 0, policy_version 1814679 (0.0007) [2023-12-27 04:36:53,215][105692] Updated weights for policy 0, policy_version 1814689 (0.0005) [2023-12-27 04:36:53,276][105692] Updated weights for policy 0, policy_version 1814699 (0.0005) [2023-12-27 04:36:53,366][105620] Updated weights for policy 1, policy_version 1818449 (0.0010) [2023-12-27 04:36:53,424][105620] Updated weights for policy 1, policy_version 1818459 (0.0010) [2023-12-27 04:36:53,469][105620] Updated weights for policy 1, policy_version 1818469 (0.0010) [2023-12-27 04:36:53,870][105692] Updated weights for policy 0, policy_version 1814709 (0.0007) [2023-12-27 04:36:53,936][105692] Updated weights for policy 0, policy_version 1814719 (0.0008) [2023-12-27 04:36:53,998][105692] Updated weights for policy 0, policy_version 1814729 (0.0008) [2023-12-27 04:36:54,238][105620] Updated weights for policy 1, policy_version 1818479 (0.0011) [2023-12-27 04:36:54,290][105620] Updated weights for policy 1, policy_version 1818489 (0.0011) [2023-12-27 04:36:54,353][105620] Updated weights for policy 1, policy_version 1818499 (0.0011) [2023-12-27 04:36:54,617][105692] Updated weights for policy 0, policy_version 1814739 (0.0009) [2023-12-27 04:36:54,670][105692] Updated weights for policy 0, policy_version 1814749 (0.0005) [2023-12-27 04:36:54,747][105692] Updated weights for policy 0, policy_version 1814759 (0.0010) [2023-12-27 04:36:55,104][105620] Updated weights for policy 1, policy_version 1818509 (0.0011) [2023-12-27 04:36:55,162][105620] Updated weights for policy 1, policy_version 1818519 (0.0006) [2023-12-27 04:36:55,219][105620] Updated weights for policy 1, policy_version 1818529 (0.0005) [2023-12-27 04:36:55,350][105692] Updated weights for policy 0, policy_version 1814769 (0.0010) [2023-12-27 04:36:55,404][105692] Updated weights for policy 0, policy_version 1814779 (0.0005) [2023-12-27 04:36:55,467][105692] Updated weights for policy 0, policy_version 1814789 (0.0006) [2023-12-27 04:36:55,517][105692] Updated weights for policy 0, policy_version 1814799 (0.0010) [2023-12-27 04:36:55,832][105620] Updated weights for policy 1, policy_version 1818539 (0.0005) [2023-12-27 04:36:55,885][105620] Updated weights for policy 1, policy_version 1818549 (0.0008) [2023-12-27 04:36:55,934][105620] Updated weights for policy 1, policy_version 1818559 (0.0009) [2023-12-27 04:36:56,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 930275328. Throughput: 0: 9905.8, 1: 9835.4. Samples: 930282752. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:36:56,063][104569] Avg episode reward: [(0, '8625.699'), (1, '9350.560')] [2023-12-27 04:36:56,218][105692] Updated weights for policy 0, policy_version 1814809 (0.0010) [2023-12-27 04:36:56,263][105692] Updated weights for policy 0, policy_version 1814819 (0.0010) [2023-12-27 04:36:56,310][105692] Updated weights for policy 0, policy_version 1814829 (0.0010) [2023-12-27 04:36:56,605][105620] Updated weights for policy 1, policy_version 1818569 (0.0010) [2023-12-27 04:36:56,665][105620] Updated weights for policy 1, policy_version 1818579 (0.0008) [2023-12-27 04:36:56,713][105620] Updated weights for policy 1, policy_version 1818589 (0.0007) [2023-12-27 04:36:56,762][105620] Updated weights for policy 1, policy_version 1818599 (0.0008) [2023-12-27 04:36:57,059][105692] Updated weights for policy 0, policy_version 1814839 (0.0007) [2023-12-27 04:36:57,117][105692] Updated weights for policy 0, policy_version 1814849 (0.0005) [2023-12-27 04:36:57,182][105692] Updated weights for policy 0, policy_version 1814859 (0.0005) [2023-12-27 04:36:57,555][105620] Updated weights for policy 1, policy_version 1818609 (0.0009) [2023-12-27 04:36:57,609][105620] Updated weights for policy 1, policy_version 1818619 (0.0007) [2023-12-27 04:36:57,678][105620] Updated weights for policy 1, policy_version 1818629 (0.0005) [2023-12-27 04:36:57,708][105692] Updated weights for policy 0, policy_version 1814869 (0.0006) [2023-12-27 04:36:57,770][105692] Updated weights for policy 0, policy_version 1814879 (0.0010) [2023-12-27 04:36:57,828][105692] Updated weights for policy 0, policy_version 1814889 (0.0010) [2023-12-27 04:36:58,275][105620] Updated weights for policy 1, policy_version 1818639 (0.0006) [2023-12-27 04:36:58,338][105620] Updated weights for policy 1, policy_version 1818649 (0.0011) [2023-12-27 04:36:58,410][105620] Updated weights for policy 1, policy_version 1818659 (0.0008) [2023-12-27 04:36:58,601][105692] Updated weights for policy 0, policy_version 1814899 (0.0009) [2023-12-27 04:36:58,669][105692] Updated weights for policy 0, policy_version 1814909 (0.0007) [2023-12-27 04:36:58,721][105692] Updated weights for policy 0, policy_version 1814919 (0.0008) [2023-12-27 04:36:59,235][105620] Updated weights for policy 1, policy_version 1818669 (0.0008) [2023-12-27 04:36:59,306][105620] Updated weights for policy 1, policy_version 1818679 (0.0010) [2023-12-27 04:36:59,377][105620] Updated weights for policy 1, policy_version 1818689 (0.0009) [2023-12-27 04:36:59,544][105692] Updated weights for policy 0, policy_version 1814929 (0.0008) [2023-12-27 04:36:59,597][105692] Updated weights for policy 0, policy_version 1814939 (0.0009) [2023-12-27 04:36:59,654][105692] Updated weights for policy 0, policy_version 1814949 (0.0010) [2023-12-27 04:36:59,710][105692] Updated weights for policy 0, policy_version 1814959 (0.0009) [2023-12-27 04:37:00,120][105620] Updated weights for policy 1, policy_version 1818699 (0.0008) [2023-12-27 04:37:00,180][105620] Updated weights for policy 1, policy_version 1818709 (0.0008) [2023-12-27 04:37:00,231][105620] Updated weights for policy 1, policy_version 1818719 (0.0008) [2023-12-27 04:37:00,515][105692] Updated weights for policy 0, policy_version 1814969 (0.0010) [2023-12-27 04:37:00,576][105692] Updated weights for policy 0, policy_version 1814979 (0.0010) [2023-12-27 04:37:00,640][105692] Updated weights for policy 0, policy_version 1814989 (0.0010) [2023-12-27 04:37:00,959][105620] Updated weights for policy 1, policy_version 1818729 (0.0008) [2023-12-27 04:37:01,016][105620] Updated weights for policy 1, policy_version 1818739 (0.0010) [2023-12-27 04:37:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 930365440. Throughput: 0: 9938.4, 1: 9865.2. Samples: 930342596. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:37:01,062][104569] Avg episode reward: [(0, '8438.777'), (1, '9350.550')] [2023-12-27 04:37:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001814992_464707584.pth... [2023-12-27 04:37:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001813808_464404480.pth [2023-12-27 04:37:01,085][105620] Updated weights for policy 1, policy_version 1818749 (0.0010) [2023-12-27 04:37:01,157][105620] Updated weights for policy 1, policy_version 1818759 (0.0011) [2023-12-27 04:37:01,164][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001818760_465666048.pth... [2023-12-27 04:37:01,170][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001817608_465371136.pth [2023-12-27 04:37:01,379][105692] Updated weights for policy 0, policy_version 1814999 (0.0011) [2023-12-27 04:37:01,433][105692] Updated weights for policy 0, policy_version 1815009 (0.0009) [2023-12-27 04:37:01,486][105692] Updated weights for policy 0, policy_version 1815019 (0.0007) [2023-12-27 04:37:01,885][105620] Updated weights for policy 1, policy_version 1818769 (0.0008) [2023-12-27 04:37:01,940][105620] Updated weights for policy 1, policy_version 1818779 (0.0010) [2023-12-27 04:37:01,995][105620] Updated weights for policy 1, policy_version 1818789 (0.0010) [2023-12-27 04:37:02,196][105692] Updated weights for policy 0, policy_version 1815029 (0.0010) [2023-12-27 04:37:02,252][105692] Updated weights for policy 0, policy_version 1815039 (0.0007) [2023-12-27 04:37:02,321][105692] Updated weights for policy 0, policy_version 1815049 (0.0009) [2023-12-27 04:37:02,682][105620] Updated weights for policy 1, policy_version 1818799 (0.0007) [2023-12-27 04:37:02,748][105620] Updated weights for policy 1, policy_version 1818809 (0.0006) [2023-12-27 04:37:02,814][105620] Updated weights for policy 1, policy_version 1818819 (0.0005) [2023-12-27 04:37:03,077][105692] Updated weights for policy 0, policy_version 1815059 (0.0008) [2023-12-27 04:37:03,135][105692] Updated weights for policy 0, policy_version 1815070 (0.0010) [2023-12-27 04:37:03,192][105692] Updated weights for policy 0, policy_version 1815080 (0.0009) [2023-12-27 04:37:03,390][105620] Updated weights for policy 1, policy_version 1818829 (0.0006) [2023-12-27 04:37:03,448][105620] Updated weights for policy 1, policy_version 1818839 (0.0005) [2023-12-27 04:37:03,508][105620] Updated weights for policy 1, policy_version 1818849 (0.0007) [2023-12-27 04:37:04,027][105692] Updated weights for policy 0, policy_version 1815090 (0.0008) [2023-12-27 04:37:04,082][105692] Updated weights for policy 0, policy_version 1815100 (0.0008) [2023-12-27 04:37:04,099][105620] Updated weights for policy 1, policy_version 1818859 (0.0008) [2023-12-27 04:37:04,149][105692] Updated weights for policy 0, policy_version 1815110 (0.0007) [2023-12-27 04:37:04,158][105620] Updated weights for policy 1, policy_version 1818869 (0.0008) [2023-12-27 04:37:04,212][105692] Updated weights for policy 0, policy_version 1815120 (0.0008) [2023-12-27 04:37:04,214][105620] Updated weights for policy 1, policy_version 1818879 (0.0008) [2023-12-27 04:37:04,897][105620] Updated weights for policy 1, policy_version 1818889 (0.0008) [2023-12-27 04:37:04,933][105692] Updated weights for policy 0, policy_version 1815130 (0.0005) [2023-12-27 04:37:04,950][105620] Updated weights for policy 1, policy_version 1818899 (0.0005) [2023-12-27 04:37:04,988][105692] Updated weights for policy 0, policy_version 1815140 (0.0005) [2023-12-27 04:37:05,007][105620] Updated weights for policy 1, policy_version 1818909 (0.0005) [2023-12-27 04:37:05,046][105692] Updated weights for policy 0, policy_version 1815150 (0.0006) [2023-12-27 04:37:05,063][105620] Updated weights for policy 1, policy_version 1818919 (0.0005) [2023-12-27 04:37:05,594][105620] Updated weights for policy 1, policy_version 1818929 (0.0008) [2023-12-27 04:37:05,644][105620] Updated weights for policy 1, policy_version 1818939 (0.0008) [2023-12-27 04:37:05,704][105620] Updated weights for policy 1, policy_version 1818949 (0.0009) [2023-12-27 04:37:05,813][105692] Updated weights for policy 0, policy_version 1815160 (0.0009) [2023-12-27 04:37:05,871][105692] Updated weights for policy 0, policy_version 1815170 (0.0009) [2023-12-27 04:37:05,925][105692] Updated weights for policy 0, policy_version 1815180 (0.0010) [2023-12-27 04:37:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 930471936. Throughput: 0: 9901.2, 1: 9890.8. Samples: 930457268. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:37:06,063][104569] Avg episode reward: [(0, '8443.068'), (1, '9258.240')] [2023-12-27 04:37:06,421][105620] Updated weights for policy 1, policy_version 1818959 (0.0009) [2023-12-27 04:37:06,491][105620] Updated weights for policy 1, policy_version 1818969 (0.0008) [2023-12-27 04:37:06,559][105620] Updated weights for policy 1, policy_version 1818979 (0.0011) [2023-12-27 04:37:06,632][105692] Updated weights for policy 0, policy_version 1815190 (0.0009) [2023-12-27 04:37:06,689][105692] Updated weights for policy 0, policy_version 1815200 (0.0008) [2023-12-27 04:37:06,749][105692] Updated weights for policy 0, policy_version 1815210 (0.0009) [2023-12-27 04:37:07,304][105620] Updated weights for policy 1, policy_version 1818989 (0.0011) [2023-12-27 04:37:07,356][105620] Updated weights for policy 1, policy_version 1818999 (0.0010) [2023-12-27 04:37:07,387][105692] Updated weights for policy 0, policy_version 1815220 (0.0007) [2023-12-27 04:37:07,416][105620] Updated weights for policy 1, policy_version 1819009 (0.0011) [2023-12-27 04:37:07,445][105692] Updated weights for policy 0, policy_version 1815230 (0.0010) [2023-12-27 04:37:07,503][105692] Updated weights for policy 0, policy_version 1815240 (0.0010) [2023-12-27 04:37:08,021][105620] Updated weights for policy 1, policy_version 1819019 (0.0010) [2023-12-27 04:37:08,072][105620] Updated weights for policy 1, policy_version 1819029 (0.0010) [2023-12-27 04:37:08,120][105620] Updated weights for policy 1, policy_version 1819039 (0.0010) [2023-12-27 04:37:08,264][105692] Updated weights for policy 0, policy_version 1815250 (0.0008) [2023-12-27 04:37:08,321][105692] Updated weights for policy 0, policy_version 1815260 (0.0010) [2023-12-27 04:37:08,386][105692] Updated weights for policy 0, policy_version 1815270 (0.0007) [2023-12-27 04:37:08,444][105692] Updated weights for policy 0, policy_version 1815280 (0.0008) [2023-12-27 04:37:08,922][105620] Updated weights for policy 1, policy_version 1819049 (0.0010) [2023-12-27 04:37:08,974][105620] Updated weights for policy 1, policy_version 1819059 (0.0010) [2023-12-27 04:37:09,028][105620] Updated weights for policy 1, policy_version 1819069 (0.0006) [2023-12-27 04:37:09,038][105692] Updated weights for policy 0, policy_version 1815290 (0.0010) [2023-12-27 04:37:09,080][105620] Updated weights for policy 1, policy_version 1819079 (0.0005) [2023-12-27 04:37:09,086][105692] Updated weights for policy 0, policy_version 1815300 (0.0011) [2023-12-27 04:37:09,144][105692] Updated weights for policy 0, policy_version 1815310 (0.0010) [2023-12-27 04:37:09,857][105620] Updated weights for policy 1, policy_version 1819089 (0.0008) [2023-12-27 04:37:09,919][105620] Updated weights for policy 1, policy_version 1819099 (0.0009) [2023-12-27 04:37:09,955][105692] Updated weights for policy 0, policy_version 1815320 (0.0011) [2023-12-27 04:37:09,976][105620] Updated weights for policy 1, policy_version 1819109 (0.0007) [2023-12-27 04:37:10,014][105692] Updated weights for policy 0, policy_version 1815330 (0.0009) [2023-12-27 04:37:10,074][105692] Updated weights for policy 0, policy_version 1815340 (0.0011) [2023-12-27 04:37:10,746][105620] Updated weights for policy 1, policy_version 1819119 (0.0007) [2023-12-27 04:37:10,807][105620] Updated weights for policy 1, policy_version 1819129 (0.0008) [2023-12-27 04:37:10,831][105692] Updated weights for policy 0, policy_version 1815350 (0.0011) [2023-12-27 04:37:10,877][105620] Updated weights for policy 1, policy_version 1819139 (0.0005) [2023-12-27 04:37:10,890][105692] Updated weights for policy 0, policy_version 1815360 (0.0010) [2023-12-27 04:37:10,949][105692] Updated weights for policy 0, policy_version 1815370 (0.0010) [2023-12-27 04:37:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 930570240. Throughput: 0: 9975.9, 1: 9882.9. Samples: 930574512. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:37:11,063][104569] Avg episode reward: [(0, '8718.623'), (1, '9258.203')] [2023-12-27 04:37:11,590][105620] Updated weights for policy 1, policy_version 1819149 (0.0008) [2023-12-27 04:37:11,663][105620] Updated weights for policy 1, policy_version 1819159 (0.0008) [2023-12-27 04:37:11,703][105692] Updated weights for policy 0, policy_version 1815380 (0.0008) [2023-12-27 04:37:11,720][105620] Updated weights for policy 1, policy_version 1819169 (0.0009) [2023-12-27 04:37:11,781][105692] Updated weights for policy 0, policy_version 1815390 (0.0008) [2023-12-27 04:37:11,847][105692] Updated weights for policy 0, policy_version 1815400 (0.0009) [2023-12-27 04:37:12,524][105692] Updated weights for policy 0, policy_version 1815410 (0.0009) [2023-12-27 04:37:12,556][105620] Updated weights for policy 1, policy_version 1819179 (0.0008) [2023-12-27 04:37:12,583][105692] Updated weights for policy 0, policy_version 1815420 (0.0007) [2023-12-27 04:37:12,610][105620] Updated weights for policy 1, policy_version 1819189 (0.0008) [2023-12-27 04:37:12,641][105692] Updated weights for policy 0, policy_version 1815430 (0.0008) [2023-12-27 04:37:12,664][105620] Updated weights for policy 1, policy_version 1819199 (0.0006) [2023-12-27 04:37:12,701][105692] Updated weights for policy 0, policy_version 1815440 (0.0008) [2023-12-27 04:37:13,395][105620] Updated weights for policy 1, policy_version 1819209 (0.0008) [2023-12-27 04:37:13,459][105620] Updated weights for policy 1, policy_version 1819219 (0.0005) [2023-12-27 04:37:13,471][105692] Updated weights for policy 0, policy_version 1815450 (0.0009) [2023-12-27 04:37:13,515][105692] Updated weights for policy 0, policy_version 1815460 (0.0007) [2023-12-27 04:37:13,522][105620] Updated weights for policy 1, policy_version 1819229 (0.0007) [2023-12-27 04:37:13,562][105692] Updated weights for policy 0, policy_version 1815470 (0.0006) [2023-12-27 04:37:13,584][105620] Updated weights for policy 1, policy_version 1819239 (0.0008) [2023-12-27 04:37:14,317][105620] Updated weights for policy 1, policy_version 1819249 (0.0009) [2023-12-27 04:37:14,367][105692] Updated weights for policy 0, policy_version 1815480 (0.0010) [2023-12-27 04:37:14,381][105620] Updated weights for policy 1, policy_version 1819259 (0.0007) [2023-12-27 04:37:14,428][105692] Updated weights for policy 0, policy_version 1815490 (0.0008) [2023-12-27 04:37:14,442][105620] Updated weights for policy 1, policy_version 1819269 (0.0006) [2023-12-27 04:37:14,482][105692] Updated weights for policy 0, policy_version 1815500 (0.0007) [2023-12-27 04:37:15,135][105692] Updated weights for policy 0, policy_version 1815510 (0.0009) [2023-12-27 04:37:15,205][105692] Updated weights for policy 0, policy_version 1815520 (0.0009) [2023-12-27 04:37:15,259][105620] Updated weights for policy 1, policy_version 1819279 (0.0006) [2023-12-27 04:37:15,276][105692] Updated weights for policy 0, policy_version 1815530 (0.0009) [2023-12-27 04:37:15,319][105620] Updated weights for policy 1, policy_version 1819289 (0.0009) [2023-12-27 04:37:15,382][105620] Updated weights for policy 1, policy_version 1819299 (0.0009) [2023-12-27 04:37:16,008][105692] Updated weights for policy 0, policy_version 1815540 (0.0006) [2023-12-27 04:37:16,062][104569] Fps is (10 sec: 18022.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 930652160. Throughput: 0: 9921.1, 1: 9830.2. Samples: 930629968. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 04:37:16,062][104569] Avg episode reward: [(0, '8623.591'), (1, '9073.392')] [2023-12-27 04:37:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001819304_465805312.pth... [2023-12-27 04:37:16,070][105692] Updated weights for policy 0, policy_version 1815550 (0.0009) [2023-12-27 04:37:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001818184_465518592.pth [2023-12-27 04:37:16,126][105620] Updated weights for policy 1, policy_version 1819309 (0.0006) [2023-12-27 04:37:16,131][105692] Updated weights for policy 0, policy_version 1815560 (0.0007) [2023-12-27 04:37:16,180][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001815568_464855040.pth... [2023-12-27 04:37:16,184][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001814384_464551936.pth [2023-12-27 04:37:16,189][105620] Updated weights for policy 1, policy_version 1819319 (0.0006) [2023-12-27 04:37:16,254][105620] Updated weights for policy 1, policy_version 1819329 (0.0009) [2023-12-27 04:37:16,719][105692] Updated weights for policy 0, policy_version 1815570 (0.0007) [2023-12-27 04:37:16,778][105692] Updated weights for policy 0, policy_version 1815580 (0.0006) [2023-12-27 04:37:16,831][105692] Updated weights for policy 0, policy_version 1815590 (0.0005) [2023-12-27 04:37:16,888][105620] Updated weights for policy 1, policy_version 1819339 (0.0008) [2023-12-27 04:37:16,890][105692] Updated weights for policy 0, policy_version 1815600 (0.0008) [2023-12-27 04:37:16,944][105620] Updated weights for policy 1, policy_version 1819349 (0.0005) [2023-12-27 04:37:17,000][105620] Updated weights for policy 1, policy_version 1819359 (0.0005) [2023-12-27 04:37:17,481][105692] Updated weights for policy 0, policy_version 1815610 (0.0005) [2023-12-27 04:37:17,542][105692] Updated weights for policy 0, policy_version 1815620 (0.0005) [2023-12-27 04:37:17,588][105692] Updated weights for policy 0, policy_version 1815630 (0.0005) [2023-12-27 04:37:17,800][105620] Updated weights for policy 1, policy_version 1819369 (0.0008) [2023-12-27 04:37:17,857][105620] Updated weights for policy 1, policy_version 1819379 (0.0006) [2023-12-27 04:37:17,911][105620] Updated weights for policy 1, policy_version 1819389 (0.0008) [2023-12-27 04:37:17,965][105620] Updated weights for policy 1, policy_version 1819399 (0.0006) [2023-12-27 04:37:18,273][105692] Updated weights for policy 0, policy_version 1815640 (0.0008) [2023-12-27 04:37:18,350][105692] Updated weights for policy 0, policy_version 1815650 (0.0009) [2023-12-27 04:37:18,409][105692] Updated weights for policy 0, policy_version 1815660 (0.0008) [2023-12-27 04:37:18,639][105620] Updated weights for policy 1, policy_version 1819409 (0.0011) [2023-12-27 04:37:18,701][105620] Updated weights for policy 1, policy_version 1819419 (0.0009) [2023-12-27 04:37:18,756][105620] Updated weights for policy 1, policy_version 1819429 (0.0010) [2023-12-27 04:37:19,063][105692] Updated weights for policy 0, policy_version 1815670 (0.0007) [2023-12-27 04:37:19,110][105692] Updated weights for policy 0, policy_version 1815680 (0.0005) [2023-12-27 04:37:19,157][105692] Updated weights for policy 0, policy_version 1815690 (0.0005) [2023-12-27 04:37:19,531][105620] Updated weights for policy 1, policy_version 1819439 (0.0009) [2023-12-27 04:37:19,600][105620] Updated weights for policy 1, policy_version 1819449 (0.0008) [2023-12-27 04:37:19,665][105620] Updated weights for policy 1, policy_version 1819459 (0.0008) [2023-12-27 04:37:19,869][105692] Updated weights for policy 0, policy_version 1815700 (0.0007) [2023-12-27 04:37:19,936][105692] Updated weights for policy 0, policy_version 1815710 (0.0009) [2023-12-27 04:37:19,990][105692] Updated weights for policy 0, policy_version 1815720 (0.0011) [2023-12-27 04:37:20,327][105620] Updated weights for policy 1, policy_version 1819469 (0.0006) [2023-12-27 04:37:20,384][105620] Updated weights for policy 1, policy_version 1819479 (0.0006) [2023-12-27 04:37:20,443][105620] Updated weights for policy 1, policy_version 1819489 (0.0006) [2023-12-27 04:37:20,763][105692] Updated weights for policy 0, policy_version 1815730 (0.0011) [2023-12-27 04:37:20,816][105692] Updated weights for policy 0, policy_version 1815740 (0.0011) [2023-12-27 04:37:20,876][105692] Updated weights for policy 0, policy_version 1815750 (0.0011) [2023-12-27 04:37:20,950][105692] Updated weights for policy 0, policy_version 1815760 (0.0011) [2023-12-27 04:37:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 930758656. Throughput: 0: 9954.1, 1: 9733.5. Samples: 930748240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:37:21,062][104569] Avg episode reward: [(0, '8714.635'), (1, '8981.225')] [2023-12-27 04:37:21,102][105620] Updated weights for policy 1, policy_version 1819499 (0.0006) [2023-12-27 04:37:21,166][105620] Updated weights for policy 1, policy_version 1819509 (0.0010) [2023-12-27 04:37:21,224][105620] Updated weights for policy 1, policy_version 1819519 (0.0006) [2023-12-27 04:37:21,715][105692] Updated weights for policy 0, policy_version 1815770 (0.0010) [2023-12-27 04:37:21,777][105692] Updated weights for policy 0, policy_version 1815780 (0.0008) [2023-12-27 04:37:21,839][105692] Updated weights for policy 0, policy_version 1815790 (0.0009) [2023-12-27 04:37:21,938][105620] Updated weights for policy 1, policy_version 1819529 (0.0007) [2023-12-27 04:37:22,001][105620] Updated weights for policy 1, policy_version 1819539 (0.0006) [2023-12-27 04:37:22,072][105620] Updated weights for policy 1, policy_version 1819549 (0.0006) [2023-12-27 04:37:22,135][105620] Updated weights for policy 1, policy_version 1819559 (0.0010) [2023-12-27 04:37:22,655][105692] Updated weights for policy 0, policy_version 1815800 (0.0009) [2023-12-27 04:37:22,712][105692] Updated weights for policy 0, policy_version 1815810 (0.0009) [2023-12-27 04:37:22,772][105692] Updated weights for policy 0, policy_version 1815820 (0.0009) [2023-12-27 04:37:22,782][105620] Updated weights for policy 1, policy_version 1819569 (0.0007) [2023-12-27 04:37:22,839][105620] Updated weights for policy 1, policy_version 1819579 (0.0008) [2023-12-27 04:37:22,890][105620] Updated weights for policy 1, policy_version 1819589 (0.0009) [2023-12-27 04:37:23,542][105692] Updated weights for policy 0, policy_version 1815830 (0.0008) [2023-12-27 04:37:23,585][105692] Updated weights for policy 0, policy_version 1815840 (0.0008) [2023-12-27 04:37:23,639][105620] Updated weights for policy 1, policy_version 1819599 (0.0009) [2023-12-27 04:37:23,643][105692] Updated weights for policy 0, policy_version 1815850 (0.0008) [2023-12-27 04:37:23,694][105620] Updated weights for policy 1, policy_version 1819609 (0.0006) [2023-12-27 04:37:23,757][105620] Updated weights for policy 1, policy_version 1819619 (0.0006) [2023-12-27 04:37:24,251][105692] Updated weights for policy 0, policy_version 1815860 (0.0010) [2023-12-27 04:37:24,321][105692] Updated weights for policy 0, policy_version 1815870 (0.0007) [2023-12-27 04:37:24,340][105620] Updated weights for policy 1, policy_version 1819629 (0.0005) [2023-12-27 04:37:24,372][105692] Updated weights for policy 0, policy_version 1815880 (0.0006) [2023-12-27 04:37:24,392][105620] Updated weights for policy 1, policy_version 1819639 (0.0005) [2023-12-27 04:37:24,439][105620] Updated weights for policy 1, policy_version 1819649 (0.0007) [2023-12-27 04:37:24,947][105692] Updated weights for policy 0, policy_version 1815890 (0.0008) [2023-12-27 04:37:25,011][105692] Updated weights for policy 0, policy_version 1815900 (0.0006) [2023-12-27 04:37:25,034][105620] Updated weights for policy 1, policy_version 1819659 (0.0008) [2023-12-27 04:37:25,069][105692] Updated weights for policy 0, policy_version 1815910 (0.0010) [2023-12-27 04:37:25,099][105620] Updated weights for policy 1, policy_version 1819669 (0.0006) [2023-12-27 04:37:25,131][105692] Updated weights for policy 0, policy_version 1815920 (0.0010) [2023-12-27 04:37:25,164][105620] Updated weights for policy 1, policy_version 1819679 (0.0005) [2023-12-27 04:37:25,738][105620] Updated weights for policy 1, policy_version 1819689 (0.0005) [2023-12-27 04:37:25,786][105692] Updated weights for policy 0, policy_version 1815930 (0.0006) [2023-12-27 04:37:25,803][105620] Updated weights for policy 1, policy_version 1819699 (0.0006) [2023-12-27 04:37:25,836][105692] Updated weights for policy 0, policy_version 1815940 (0.0005) [2023-12-27 04:37:25,866][105620] Updated weights for policy 1, policy_version 1819709 (0.0005) [2023-12-27 04:37:25,889][105692] Updated weights for policy 0, policy_version 1815950 (0.0005) [2023-12-27 04:37:25,933][105620] Updated weights for policy 1, policy_version 1819719 (0.0006) [2023-12-27 04:37:26,062][104569] Fps is (10 sec: 21299.3, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 930865152. Throughput: 0: 9938.2, 1: 9840.3. Samples: 930870092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:37:26,062][104569] Avg episode reward: [(0, '8810.170'), (1, '9168.832')] [2023-12-27 04:37:26,443][105692] Updated weights for policy 0, policy_version 1815960 (0.0005) [2023-12-27 04:37:26,489][105692] Updated weights for policy 0, policy_version 1815970 (0.0005) [2023-12-27 04:37:26,536][105692] Updated weights for policy 0, policy_version 1815980 (0.0005) [2023-12-27 04:37:26,613][105620] Updated weights for policy 1, policy_version 1819729 (0.0006) [2023-12-27 04:37:26,669][105620] Updated weights for policy 1, policy_version 1819739 (0.0005) [2023-12-27 04:37:26,730][105620] Updated weights for policy 1, policy_version 1819749 (0.0005) [2023-12-27 04:37:27,202][105692] Updated weights for policy 0, policy_version 1815990 (0.0007) [2023-12-27 04:37:27,247][105692] Updated weights for policy 0, policy_version 1816000 (0.0008) [2023-12-27 04:37:27,300][105692] Updated weights for policy 0, policy_version 1816010 (0.0009) [2023-12-27 04:37:27,362][105620] Updated weights for policy 1, policy_version 1819759 (0.0007) [2023-12-27 04:37:27,418][105620] Updated weights for policy 1, policy_version 1819769 (0.0010) [2023-12-27 04:37:27,472][105620] Updated weights for policy 1, policy_version 1819779 (0.0010) [2023-12-27 04:37:28,109][105692] Updated weights for policy 0, policy_version 1816020 (0.0007) [2023-12-27 04:37:28,113][105620] Updated weights for policy 1, policy_version 1819789 (0.0008) [2023-12-27 04:37:28,167][105620] Updated weights for policy 1, policy_version 1819799 (0.0005) [2023-12-27 04:37:28,168][105692] Updated weights for policy 0, policy_version 1816030 (0.0008) [2023-12-27 04:37:28,216][105620] Updated weights for policy 1, policy_version 1819809 (0.0006) [2023-12-27 04:37:28,221][105692] Updated weights for policy 0, policy_version 1816040 (0.0009) [2023-12-27 04:37:28,744][105620] Updated weights for policy 1, policy_version 1819819 (0.0005) [2023-12-27 04:37:28,807][105620] Updated weights for policy 1, policy_version 1819829 (0.0005) [2023-12-27 04:37:28,873][105620] Updated weights for policy 1, policy_version 1819839 (0.0006) [2023-12-27 04:37:29,115][105692] Updated weights for policy 0, policy_version 1816050 (0.0009) [2023-12-27 04:37:29,182][105692] Updated weights for policy 0, policy_version 1816060 (0.0009) [2023-12-27 04:37:29,243][105692] Updated weights for policy 0, policy_version 1816070 (0.0008) [2023-12-27 04:37:29,299][105692] Updated weights for policy 0, policy_version 1816080 (0.0008) [2023-12-27 04:37:29,528][105620] Updated weights for policy 1, policy_version 1819849 (0.0010) [2023-12-27 04:37:29,588][105620] Updated weights for policy 1, policy_version 1819859 (0.0009) [2023-12-27 04:37:29,646][105620] Updated weights for policy 1, policy_version 1819869 (0.0010) [2023-12-27 04:37:29,701][105620] Updated weights for policy 1, policy_version 1819879 (0.0010) [2023-12-27 04:37:29,955][105692] Updated weights for policy 0, policy_version 1816090 (0.0006) [2023-12-27 04:37:30,017][105692] Updated weights for policy 0, policy_version 1816100 (0.0007) [2023-12-27 04:37:30,083][105692] Updated weights for policy 0, policy_version 1816110 (0.0008) [2023-12-27 04:37:30,435][105620] Updated weights for policy 1, policy_version 1819889 (0.0010) [2023-12-27 04:37:30,484][105620] Updated weights for policy 1, policy_version 1819899 (0.0010) [2023-12-27 04:37:30,536][105620] Updated weights for policy 1, policy_version 1819909 (0.0010) [2023-12-27 04:37:30,732][105692] Updated weights for policy 0, policy_version 1816120 (0.0006) [2023-12-27 04:37:30,806][105692] Updated weights for policy 0, policy_version 1816130 (0.0005) [2023-12-27 04:37:30,857][105692] Updated weights for policy 0, policy_version 1816140 (0.0005) [2023-12-27 04:37:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 930963456. Throughput: 0: 9906.3, 1: 9939.1. Samples: 930933204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:37:31,062][104569] Avg episode reward: [(0, '8813.609'), (1, '9168.582')] [2023-12-27 04:37:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001816144_465002496.pth... [2023-12-27 04:37:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001819912_465960960.pth... [2023-12-27 04:37:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001814992_464707584.pth [2023-12-27 04:37:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001818760_465666048.pth [2023-12-27 04:37:31,166][105620] Updated weights for policy 1, policy_version 1819919 (0.0009) [2023-12-27 04:37:31,230][105620] Updated weights for policy 1, policy_version 1819929 (0.0008) [2023-12-27 04:37:31,289][105620] Updated weights for policy 1, policy_version 1819939 (0.0009) [2023-12-27 04:37:31,487][105692] Updated weights for policy 0, policy_version 1816150 (0.0008) [2023-12-27 04:37:31,534][105692] Updated weights for policy 0, policy_version 1816160 (0.0009) [2023-12-27 04:37:31,588][105692] Updated weights for policy 0, policy_version 1816170 (0.0005) [2023-12-27 04:37:32,011][105620] Updated weights for policy 1, policy_version 1819949 (0.0007) [2023-12-27 04:37:32,057][105620] Updated weights for policy 1, policy_version 1819959 (0.0008) [2023-12-27 04:37:32,111][105620] Updated weights for policy 1, policy_version 1819969 (0.0009) [2023-12-27 04:37:32,402][105692] Updated weights for policy 0, policy_version 1816180 (0.0008) [2023-12-27 04:37:32,460][105692] Updated weights for policy 0, policy_version 1816190 (0.0010) [2023-12-27 04:37:32,521][105692] Updated weights for policy 0, policy_version 1816200 (0.0009) [2023-12-27 04:37:32,828][105620] Updated weights for policy 1, policy_version 1819979 (0.0008) [2023-12-27 04:37:32,877][105620] Updated weights for policy 1, policy_version 1819989 (0.0006) [2023-12-27 04:37:32,938][105620] Updated weights for policy 1, policy_version 1819999 (0.0011) [2023-12-27 04:37:33,210][105692] Updated weights for policy 0, policy_version 1816210 (0.0009) [2023-12-27 04:37:33,268][105692] Updated weights for policy 0, policy_version 1816220 (0.0010) [2023-12-27 04:37:33,321][105692] Updated weights for policy 0, policy_version 1816231 (0.0010) [2023-12-27 04:37:33,539][105620] Updated weights for policy 1, policy_version 1820009 (0.0010) [2023-12-27 04:37:33,594][105620] Updated weights for policy 1, policy_version 1820019 (0.0005) [2023-12-27 04:37:33,646][105620] Updated weights for policy 1, policy_version 1820029 (0.0005) [2023-12-27 04:37:33,691][105620] Updated weights for policy 1, policy_version 1820039 (0.0005) [2023-12-27 04:37:34,021][105692] Updated weights for policy 0, policy_version 1816241 (0.0006) [2023-12-27 04:37:34,076][105692] Updated weights for policy 0, policy_version 1816251 (0.0009) [2023-12-27 04:37:34,133][105692] Updated weights for policy 0, policy_version 1816261 (0.0009) [2023-12-27 04:37:34,194][105692] Updated weights for policy 0, policy_version 1816271 (0.0008) [2023-12-27 04:37:34,225][105620] Updated weights for policy 1, policy_version 1820049 (0.0008) [2023-12-27 04:37:34,274][105620] Updated weights for policy 1, policy_version 1820059 (0.0009) [2023-12-27 04:37:34,331][105620] Updated weights for policy 1, policy_version 1820069 (0.0009) [2023-12-27 04:37:34,984][105620] Updated weights for policy 1, policy_version 1820079 (0.0009) [2023-12-27 04:37:35,009][105692] Updated weights for policy 0, policy_version 1816281 (0.0006) [2023-12-27 04:37:35,047][105620] Updated weights for policy 1, policy_version 1820089 (0.0008) [2023-12-27 04:37:35,072][105692] Updated weights for policy 0, policy_version 1816291 (0.0005) [2023-12-27 04:37:35,107][105620] Updated weights for policy 1, policy_version 1820099 (0.0008) [2023-12-27 04:37:35,128][105692] Updated weights for policy 0, policy_version 1816301 (0.0006) [2023-12-27 04:37:35,803][105692] Updated weights for policy 0, policy_version 1816311 (0.0010) [2023-12-27 04:37:35,818][105620] Updated weights for policy 1, policy_version 1820109 (0.0008) [2023-12-27 04:37:35,852][105692] Updated weights for policy 0, policy_version 1816321 (0.0010) [2023-12-27 04:37:35,870][105620] Updated weights for policy 1, policy_version 1820119 (0.0005) [2023-12-27 04:37:35,908][105692] Updated weights for policy 0, policy_version 1816331 (0.0010) [2023-12-27 04:37:35,924][105620] Updated weights for policy 1, policy_version 1820129 (0.0006) [2023-12-27 04:37:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19933.9, 300 sec: 19633.0). Total num frames: 931069952. Throughput: 0: 9842.5, 1: 9984.3. Samples: 931054292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:37:36,062][104569] Avg episode reward: [(0, '8536.863'), (1, '9166.014')] [2023-12-27 04:37:36,488][105692] Updated weights for policy 0, policy_version 1816341 (0.0006) [2023-12-27 04:37:36,527][105620] Updated weights for policy 1, policy_version 1820139 (0.0007) [2023-12-27 04:37:36,558][105692] Updated weights for policy 0, policy_version 1816351 (0.0005) [2023-12-27 04:37:36,587][105620] Updated weights for policy 1, policy_version 1820149 (0.0011) [2023-12-27 04:37:36,619][105692] Updated weights for policy 0, policy_version 1816361 (0.0005) [2023-12-27 04:37:36,646][105620] Updated weights for policy 1, policy_version 1820159 (0.0010) [2023-12-27 04:37:37,238][105692] Updated weights for policy 0, policy_version 1816371 (0.0007) [2023-12-27 04:37:37,275][105620] Updated weights for policy 1, policy_version 1820169 (0.0008) [2023-12-27 04:37:37,300][105692] Updated weights for policy 0, policy_version 1816381 (0.0010) [2023-12-27 04:37:37,333][105620] Updated weights for policy 1, policy_version 1820179 (0.0006) [2023-12-27 04:37:37,353][105692] Updated weights for policy 0, policy_version 1816391 (0.0011) [2023-12-27 04:37:37,380][105620] Updated weights for policy 1, policy_version 1820189 (0.0005) [2023-12-27 04:37:37,434][105620] Updated weights for policy 1, policy_version 1820199 (0.0010) [2023-12-27 04:37:38,035][105692] Updated weights for policy 0, policy_version 1816401 (0.0010) [2023-12-27 04:37:38,064][105620] Updated weights for policy 1, policy_version 1820209 (0.0007) [2023-12-27 04:37:38,085][105692] Updated weights for policy 0, policy_version 1816411 (0.0009) [2023-12-27 04:37:38,114][105620] Updated weights for policy 1, policy_version 1820219 (0.0006) [2023-12-27 04:37:38,131][105692] Updated weights for policy 0, policy_version 1816421 (0.0007) [2023-12-27 04:37:38,166][105620] Updated weights for policy 1, policy_version 1820229 (0.0006) [2023-12-27 04:37:38,183][105692] Updated weights for policy 0, policy_version 1816431 (0.0009) [2023-12-27 04:37:38,797][105620] Updated weights for policy 1, policy_version 1820239 (0.0007) [2023-12-27 04:37:38,846][105692] Updated weights for policy 0, policy_version 1816441 (0.0008) [2023-12-27 04:37:38,867][105620] Updated weights for policy 1, policy_version 1820249 (0.0010) [2023-12-27 04:37:38,897][105692] Updated weights for policy 0, policy_version 1816451 (0.0007) [2023-12-27 04:37:38,934][105620] Updated weights for policy 1, policy_version 1820259 (0.0005) [2023-12-27 04:37:38,953][105692] Updated weights for policy 0, policy_version 1816461 (0.0008) [2023-12-27 04:37:39,597][105620] Updated weights for policy 1, policy_version 1820269 (0.0008) [2023-12-27 04:37:39,621][105692] Updated weights for policy 0, policy_version 1816471 (0.0008) [2023-12-27 04:37:39,657][105620] Updated weights for policy 1, policy_version 1820279 (0.0010) [2023-12-27 04:37:39,679][105692] Updated weights for policy 0, policy_version 1816481 (0.0007) [2023-12-27 04:37:39,721][105620] Updated weights for policy 1, policy_version 1820289 (0.0011) [2023-12-27 04:37:39,740][105692] Updated weights for policy 0, policy_version 1816491 (0.0008) [2023-12-27 04:37:40,465][105692] Updated weights for policy 0, policy_version 1816501 (0.0008) [2023-12-27 04:37:40,489][105620] Updated weights for policy 1, policy_version 1820299 (0.0011) [2023-12-27 04:37:40,519][105692] Updated weights for policy 0, policy_version 1816511 (0.0006) [2023-12-27 04:37:40,549][105620] Updated weights for policy 1, policy_version 1820309 (0.0011) [2023-12-27 04:37:40,584][105692] Updated weights for policy 0, policy_version 1816521 (0.0007) [2023-12-27 04:37:40,608][105620] Updated weights for policy 1, policy_version 1820319 (0.0011) [2023-12-27 04:37:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19934.2, 300 sec: 19605.3). Total num frames: 931168256. Throughput: 0: 9803.7, 1: 10089.3. Samples: 931177932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:37:41,062][104569] Avg episode reward: [(0, '8716.213'), (1, '9166.188')] [2023-12-27 04:37:41,354][105692] Updated weights for policy 0, policy_version 1816531 (0.0006) [2023-12-27 04:37:41,358][105620] Updated weights for policy 1, policy_version 1820329 (0.0011) [2023-12-27 04:37:41,417][105692] Updated weights for policy 0, policy_version 1816541 (0.0007) [2023-12-27 04:37:41,419][105620] Updated weights for policy 1, policy_version 1820339 (0.0010) [2023-12-27 04:37:41,468][105620] Updated weights for policy 1, policy_version 1820349 (0.0009) [2023-12-27 04:37:41,468][105692] Updated weights for policy 0, policy_version 1816551 (0.0005) [2023-12-27 04:37:41,523][105620] Updated weights for policy 1, policy_version 1820359 (0.0007) [2023-12-27 04:37:42,242][105620] Updated weights for policy 1, policy_version 1820369 (0.0005) [2023-12-27 04:37:42,271][105692] Updated weights for policy 0, policy_version 1816561 (0.0007) [2023-12-27 04:37:42,305][105620] Updated weights for policy 1, policy_version 1820379 (0.0006) [2023-12-27 04:37:42,333][105692] Updated weights for policy 0, policy_version 1816571 (0.0008) [2023-12-27 04:37:42,366][105620] Updated weights for policy 1, policy_version 1820389 (0.0007) [2023-12-27 04:37:42,398][105692] Updated weights for policy 0, policy_version 1816581 (0.0008) [2023-12-27 04:37:42,456][105692] Updated weights for policy 0, policy_version 1816591 (0.0008) [2023-12-27 04:37:42,974][105620] Updated weights for policy 1, policy_version 1820399 (0.0005) [2023-12-27 04:37:43,034][105620] Updated weights for policy 1, policy_version 1820409 (0.0007) [2023-12-27 04:37:43,097][105620] Updated weights for policy 1, policy_version 1820419 (0.0007) [2023-12-27 04:37:43,253][105692] Updated weights for policy 0, policy_version 1816601 (0.0009) [2023-12-27 04:37:43,312][105692] Updated weights for policy 0, policy_version 1816611 (0.0010) [2023-12-27 04:37:43,386][105692] Updated weights for policy 0, policy_version 1816621 (0.0009) [2023-12-27 04:37:43,767][105620] Updated weights for policy 1, policy_version 1820429 (0.0005) [2023-12-27 04:37:43,819][105620] Updated weights for policy 1, policy_version 1820439 (0.0009) [2023-12-27 04:37:43,874][105620] Updated weights for policy 1, policy_version 1820449 (0.0011) [2023-12-27 04:37:44,159][105692] Updated weights for policy 0, policy_version 1816631 (0.0009) [2023-12-27 04:37:44,217][105692] Updated weights for policy 0, policy_version 1816641 (0.0007) [2023-12-27 04:37:44,277][105692] Updated weights for policy 0, policy_version 1816651 (0.0005) [2023-12-27 04:37:44,626][105620] Updated weights for policy 1, policy_version 1820459 (0.0010) [2023-12-27 04:37:44,692][105620] Updated weights for policy 1, policy_version 1820469 (0.0010) [2023-12-27 04:37:44,765][105620] Updated weights for policy 1, policy_version 1820479 (0.0008) [2023-12-27 04:37:44,880][105692] Updated weights for policy 0, policy_version 1816661 (0.0006) [2023-12-27 04:37:44,938][105692] Updated weights for policy 0, policy_version 1816671 (0.0006) [2023-12-27 04:37:44,992][105692] Updated weights for policy 0, policy_version 1816681 (0.0006) [2023-12-27 04:37:45,494][105620] Updated weights for policy 1, policy_version 1820489 (0.0011) [2023-12-27 04:37:45,558][105620] Updated weights for policy 1, policy_version 1820499 (0.0011) [2023-12-27 04:37:45,618][105620] Updated weights for policy 1, policy_version 1820509 (0.0011) [2023-12-27 04:37:45,632][105692] Updated weights for policy 0, policy_version 1816691 (0.0007) [2023-12-27 04:37:45,688][105620] Updated weights for policy 1, policy_version 1820519 (0.0011) [2023-12-27 04:37:45,695][105692] Updated weights for policy 0, policy_version 1816701 (0.0008) [2023-12-27 04:37:45,757][105692] Updated weights for policy 0, policy_version 1816711 (0.0006) [2023-12-27 04:37:46,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19933.9, 300 sec: 19605.2). Total num frames: 931266560. Throughput: 0: 9709.6, 1: 10108.3. Samples: 931234408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:37:46,063][104569] Avg episode reward: [(0, '8990.247'), (1, '9165.999')] [2023-12-27 04:37:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001816720_465149952.pth... [2023-12-27 04:37:46,074][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001820520_466116608.pth... [2023-12-27 04:37:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001815568_464855040.pth [2023-12-27 04:37:46,082][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001819304_465805312.pth [2023-12-27 04:37:46,326][105692] Updated weights for policy 0, policy_version 1816721 (0.0005) [2023-12-27 04:37:46,370][105692] Updated weights for policy 0, policy_version 1816731 (0.0005) [2023-12-27 04:37:46,384][105620] Updated weights for policy 1, policy_version 1820529 (0.0009) [2023-12-27 04:37:46,428][105692] Updated weights for policy 0, policy_version 1816741 (0.0006) [2023-12-27 04:37:46,442][105620] Updated weights for policy 1, policy_version 1820539 (0.0007) [2023-12-27 04:37:46,476][105692] Updated weights for policy 0, policy_version 1816751 (0.0006) [2023-12-27 04:37:46,497][105620] Updated weights for policy 1, policy_version 1820549 (0.0010) [2023-12-27 04:37:47,207][105620] Updated weights for policy 1, policy_version 1820559 (0.0009) [2023-12-27 04:37:47,238][105692] Updated weights for policy 0, policy_version 1816761 (0.0008) [2023-12-27 04:37:47,257][105620] Updated weights for policy 1, policy_version 1820569 (0.0008) [2023-12-27 04:37:47,292][105692] Updated weights for policy 0, policy_version 1816771 (0.0006) [2023-12-27 04:37:47,320][105620] Updated weights for policy 1, policy_version 1820579 (0.0009) [2023-12-27 04:37:47,339][105692] Updated weights for policy 0, policy_version 1816781 (0.0007) [2023-12-27 04:37:48,088][105620] Updated weights for policy 1, policy_version 1820589 (0.0007) [2023-12-27 04:37:48,101][105692] Updated weights for policy 0, policy_version 1816791 (0.0009) [2023-12-27 04:37:48,148][105620] Updated weights for policy 1, policy_version 1820599 (0.0007) [2023-12-27 04:37:48,159][105692] Updated weights for policy 0, policy_version 1816801 (0.0007) [2023-12-27 04:37:48,209][105620] Updated weights for policy 1, policy_version 1820609 (0.0006) [2023-12-27 04:37:48,226][105692] Updated weights for policy 0, policy_version 1816811 (0.0007) [2023-12-27 04:37:48,895][105620] Updated weights for policy 1, policy_version 1820619 (0.0007) [2023-12-27 04:37:48,957][105620] Updated weights for policy 1, policy_version 1820629 (0.0006) [2023-12-27 04:37:49,004][105692] Updated weights for policy 0, policy_version 1816821 (0.0008) [2023-12-27 04:37:49,008][105620] Updated weights for policy 1, policy_version 1820639 (0.0005) [2023-12-27 04:37:49,066][105692] Updated weights for policy 0, policy_version 1816831 (0.0008) [2023-12-27 04:37:49,128][105692] Updated weights for policy 0, policy_version 1816841 (0.0009) [2023-12-27 04:37:49,655][105620] Updated weights for policy 1, policy_version 1820649 (0.0007) [2023-12-27 04:37:49,712][105620] Updated weights for policy 1, policy_version 1820659 (0.0007) [2023-12-27 04:37:49,766][105620] Updated weights for policy 1, policy_version 1820669 (0.0007) [2023-12-27 04:37:49,832][105620] Updated weights for policy 1, policy_version 1820679 (0.0006) [2023-12-27 04:37:49,917][105692] Updated weights for policy 0, policy_version 1816851 (0.0009) [2023-12-27 04:37:49,992][105692] Updated weights for policy 0, policy_version 1816861 (0.0010) [2023-12-27 04:37:50,052][105692] Updated weights for policy 0, policy_version 1816871 (0.0006) [2023-12-27 04:37:50,522][105620] Updated weights for policy 1, policy_version 1820689 (0.0009) [2023-12-27 04:37:50,577][105620] Updated weights for policy 1, policy_version 1820699 (0.0009) [2023-12-27 04:37:50,639][105620] Updated weights for policy 1, policy_version 1820709 (0.0009) [2023-12-27 04:37:50,645][105692] Updated weights for policy 0, policy_version 1816881 (0.0006) [2023-12-27 04:37:50,711][105692] Updated weights for policy 0, policy_version 1816891 (0.0009) [2023-12-27 04:37:50,770][105692] Updated weights for policy 0, policy_version 1816901 (0.0009) [2023-12-27 04:37:50,822][105692] Updated weights for policy 0, policy_version 1816911 (0.0009) [2023-12-27 04:37:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 931364864. Throughput: 0: 9820.7, 1: 10061.1. Samples: 931351944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:37:51,062][104569] Avg episode reward: [(0, '8440.846'), (1, '9258.302')] [2023-12-27 04:37:51,377][105620] Updated weights for policy 1, policy_version 1820719 (0.0008) [2023-12-27 04:37:51,438][105620] Updated weights for policy 1, policy_version 1820729 (0.0009) [2023-12-27 04:37:51,503][105620] Updated weights for policy 1, policy_version 1820739 (0.0009) [2023-12-27 04:37:51,581][105692] Updated weights for policy 0, policy_version 1816921 (0.0008) [2023-12-27 04:37:51,645][105692] Updated weights for policy 0, policy_version 1816931 (0.0008) [2023-12-27 04:37:51,705][105692] Updated weights for policy 0, policy_version 1816941 (0.0008) [2023-12-27 04:37:52,285][105620] Updated weights for policy 1, policy_version 1820749 (0.0009) [2023-12-27 04:37:52,342][105620] Updated weights for policy 1, policy_version 1820759 (0.0009) [2023-12-27 04:37:52,404][105620] Updated weights for policy 1, policy_version 1820769 (0.0008) [2023-12-27 04:37:52,434][105692] Updated weights for policy 0, policy_version 1816951 (0.0008) [2023-12-27 04:37:52,498][105692] Updated weights for policy 0, policy_version 1816961 (0.0008) [2023-12-27 04:37:52,562][105692] Updated weights for policy 0, policy_version 1816971 (0.0009) [2023-12-27 04:37:53,093][105620] Updated weights for policy 1, policy_version 1820779 (0.0008) [2023-12-27 04:37:53,142][105620] Updated weights for policy 1, policy_version 1820789 (0.0009) [2023-12-27 04:37:53,197][105620] Updated weights for policy 1, policy_version 1820799 (0.0010) [2023-12-27 04:37:53,270][105692] Updated weights for policy 0, policy_version 1816981 (0.0009) [2023-12-27 04:37:53,332][105692] Updated weights for policy 0, policy_version 1816991 (0.0009) [2023-12-27 04:37:53,380][105692] Updated weights for policy 0, policy_version 1817001 (0.0009) [2023-12-27 04:37:53,957][105620] Updated weights for policy 1, policy_version 1820809 (0.0008) [2023-12-27 04:37:54,013][105620] Updated weights for policy 1, policy_version 1820819 (0.0009) [2023-12-27 04:37:54,071][105620] Updated weights for policy 1, policy_version 1820829 (0.0009) [2023-12-27 04:37:54,073][105692] Updated weights for policy 0, policy_version 1817011 (0.0008) [2023-12-27 04:37:54,132][105692] Updated weights for policy 0, policy_version 1817021 (0.0005) [2023-12-27 04:37:54,134][105620] Updated weights for policy 1, policy_version 1820839 (0.0009) [2023-12-27 04:37:54,194][105692] Updated weights for policy 0, policy_version 1817031 (0.0005) [2023-12-27 04:37:54,712][105692] Updated weights for policy 0, policy_version 1817041 (0.0006) [2023-12-27 04:37:54,759][105692] Updated weights for policy 0, policy_version 1817051 (0.0010) [2023-12-27 04:37:54,800][105692] Updated weights for policy 0, policy_version 1817061 (0.0010) [2023-12-27 04:37:54,855][105692] Updated weights for policy 0, policy_version 1817071 (0.0010) [2023-12-27 04:37:54,948][105620] Updated weights for policy 1, policy_version 1820849 (0.0006) [2023-12-27 04:37:55,000][105620] Updated weights for policy 1, policy_version 1820859 (0.0008) [2023-12-27 04:37:55,051][105620] Updated weights for policy 1, policy_version 1820869 (0.0008) [2023-12-27 04:37:55,516][105692] Updated weights for policy 0, policy_version 1817081 (0.0006) [2023-12-27 04:37:55,563][105692] Updated weights for policy 0, policy_version 1817091 (0.0005) [2023-12-27 04:37:55,618][105692] Updated weights for policy 0, policy_version 1817101 (0.0005) [2023-12-27 04:37:55,842][105620] Updated weights for policy 1, policy_version 1820879 (0.0008) [2023-12-27 04:37:55,894][105620] Updated weights for policy 1, policy_version 1820889 (0.0010) [2023-12-27 04:37:55,952][105620] Updated weights for policy 1, policy_version 1820900 (0.0010) [2023-12-27 04:37:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 931463168. Throughput: 0: 9889.0, 1: 9995.7. Samples: 931469328. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:37:56,063][104569] Avg episode reward: [(0, '8713.515'), (1, '9258.291')] [2023-12-27 04:37:56,175][105692] Updated weights for policy 0, policy_version 1817111 (0.0005) [2023-12-27 04:37:56,230][105692] Updated weights for policy 0, policy_version 1817121 (0.0005) [2023-12-27 04:37:56,291][105692] Updated weights for policy 0, policy_version 1817131 (0.0008) [2023-12-27 04:37:56,845][105620] Updated weights for policy 1, policy_version 1820911 (0.0008) [2023-12-27 04:37:56,859][105692] Updated weights for policy 0, policy_version 1817141 (0.0007) [2023-12-27 04:37:56,902][105620] Updated weights for policy 1, policy_version 1820921 (0.0005) [2023-12-27 04:37:56,917][105692] Updated weights for policy 0, policy_version 1817151 (0.0009) [2023-12-27 04:37:56,954][105620] Updated weights for policy 1, policy_version 1820931 (0.0005) [2023-12-27 04:37:56,965][105692] Updated weights for policy 0, policy_version 1817161 (0.0008) [2023-12-27 04:37:57,489][105620] Updated weights for policy 1, policy_version 1820941 (0.0005) [2023-12-27 04:37:57,535][105620] Updated weights for policy 1, policy_version 1820951 (0.0005) [2023-12-27 04:37:57,589][105620] Updated weights for policy 1, policy_version 1820961 (0.0005) [2023-12-27 04:37:57,815][105692] Updated weights for policy 0, policy_version 1817171 (0.0008) [2023-12-27 04:37:57,881][105692] Updated weights for policy 0, policy_version 1817181 (0.0008) [2023-12-27 04:37:57,945][105692] Updated weights for policy 0, policy_version 1817191 (0.0008) [2023-12-27 04:37:58,103][105620] Updated weights for policy 1, policy_version 1820971 (0.0005) [2023-12-27 04:37:58,150][105620] Updated weights for policy 1, policy_version 1820981 (0.0005) [2023-12-27 04:37:58,211][105620] Updated weights for policy 1, policy_version 1820991 (0.0007) [2023-12-27 04:37:58,643][105692] Updated weights for policy 0, policy_version 1817201 (0.0006) [2023-12-27 04:37:58,710][105692] Updated weights for policy 0, policy_version 1817211 (0.0009) [2023-12-27 04:37:58,778][105692] Updated weights for policy 0, policy_version 1817221 (0.0008) [2023-12-27 04:37:58,845][105692] Updated weights for policy 0, policy_version 1817231 (0.0009) [2023-12-27 04:37:58,937][105620] Updated weights for policy 1, policy_version 1821001 (0.0008) [2023-12-27 04:37:59,000][105620] Updated weights for policy 1, policy_version 1821011 (0.0008) [2023-12-27 04:37:59,066][105620] Updated weights for policy 1, policy_version 1821021 (0.0008) [2023-12-27 04:37:59,124][105620] Updated weights for policy 1, policy_version 1821031 (0.0008) [2023-12-27 04:37:59,627][105692] Updated weights for policy 0, policy_version 1817241 (0.0009) [2023-12-27 04:37:59,686][105692] Updated weights for policy 0, policy_version 1817251 (0.0009) [2023-12-27 04:37:59,739][105692] Updated weights for policy 0, policy_version 1817261 (0.0007) [2023-12-27 04:37:59,936][105620] Updated weights for policy 1, policy_version 1821041 (0.0009) [2023-12-27 04:37:59,988][105620] Updated weights for policy 1, policy_version 1821051 (0.0008) [2023-12-27 04:38:00,046][105620] Updated weights for policy 1, policy_version 1821061 (0.0008) [2023-12-27 04:38:00,481][105692] Updated weights for policy 0, policy_version 1817271 (0.0010) [2023-12-27 04:38:00,536][105692] Updated weights for policy 0, policy_version 1817281 (0.0009) [2023-12-27 04:38:00,583][105692] Updated weights for policy 0, policy_version 1817291 (0.0009) [2023-12-27 04:38:00,768][105620] Updated weights for policy 1, policy_version 1821071 (0.0008) [2023-12-27 04:38:00,834][105620] Updated weights for policy 1, policy_version 1821081 (0.0008) [2023-12-27 04:38:00,895][105620] Updated weights for policy 1, policy_version 1821091 (0.0008) [2023-12-27 04:38:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19933.8, 300 sec: 19577.5). Total num frames: 931561472. Throughput: 0: 9952.5, 1: 10083.3. Samples: 931531584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:38:01,062][104569] Avg episode reward: [(0, '9262.147'), (1, '9258.416')] [2023-12-27 04:38:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001817296_465297408.pth... [2023-12-27 04:38:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001821096_466264064.pth... [2023-12-27 04:38:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001819912_465960960.pth [2023-12-27 04:38:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001816144_465002496.pth [2023-12-27 04:38:01,389][105692] Updated weights for policy 0, policy_version 1817301 (0.0009) [2023-12-27 04:38:01,443][105692] Updated weights for policy 0, policy_version 1817311 (0.0009) [2023-12-27 04:38:01,495][105692] Updated weights for policy 0, policy_version 1817321 (0.0009) [2023-12-27 04:38:01,571][105620] Updated weights for policy 1, policy_version 1821101 (0.0009) [2023-12-27 04:38:01,642][105620] Updated weights for policy 1, policy_version 1821111 (0.0010) [2023-12-27 04:38:01,707][105620] Updated weights for policy 1, policy_version 1821121 (0.0009) [2023-12-27 04:38:02,270][105692] Updated weights for policy 0, policy_version 1817331 (0.0009) [2023-12-27 04:38:02,323][105692] Updated weights for policy 0, policy_version 1817342 (0.0010) [2023-12-27 04:38:02,383][105692] Updated weights for policy 0, policy_version 1817352 (0.0008) [2023-12-27 04:38:02,417][105620] Updated weights for policy 1, policy_version 1821131 (0.0008) [2023-12-27 04:38:02,471][105620] Updated weights for policy 1, policy_version 1821141 (0.0009) [2023-12-27 04:38:02,526][105620] Updated weights for policy 1, policy_version 1821151 (0.0009) [2023-12-27 04:38:03,142][105620] Updated weights for policy 1, policy_version 1821161 (0.0009) [2023-12-27 04:38:03,195][105620] Updated weights for policy 1, policy_version 1821171 (0.0005) [2023-12-27 04:38:03,208][105692] Updated weights for policy 0, policy_version 1817362 (0.0008) [2023-12-27 04:38:03,240][105620] Updated weights for policy 1, policy_version 1821181 (0.0005) [2023-12-27 04:38:03,266][105692] Updated weights for policy 0, policy_version 1817372 (0.0010) [2023-12-27 04:38:03,296][105620] Updated weights for policy 1, policy_version 1821191 (0.0006) [2023-12-27 04:38:03,323][105692] Updated weights for policy 0, policy_version 1817382 (0.0010) [2023-12-27 04:38:03,381][105692] Updated weights for policy 0, policy_version 1817392 (0.0010) [2023-12-27 04:38:03,823][105620] Updated weights for policy 1, policy_version 1821201 (0.0006) [2023-12-27 04:38:03,882][105620] Updated weights for policy 1, policy_version 1821211 (0.0007) [2023-12-27 04:38:03,930][105620] Updated weights for policy 1, policy_version 1821221 (0.0007) [2023-12-27 04:38:04,008][105692] Updated weights for policy 0, policy_version 1817402 (0.0009) [2023-12-27 04:38:04,075][105692] Updated weights for policy 0, policy_version 1817412 (0.0010) [2023-12-27 04:38:04,136][105692] Updated weights for policy 0, policy_version 1817422 (0.0009) [2023-12-27 04:38:04,653][105620] Updated weights for policy 1, policy_version 1821231 (0.0009) [2023-12-27 04:38:04,708][105620] Updated weights for policy 1, policy_version 1821241 (0.0009) [2023-12-27 04:38:04,755][105620] Updated weights for policy 1, policy_version 1821251 (0.0008) [2023-12-27 04:38:04,868][105692] Updated weights for policy 0, policy_version 1817432 (0.0009) [2023-12-27 04:38:04,929][105692] Updated weights for policy 0, policy_version 1817442 (0.0009) [2023-12-27 04:38:04,989][105692] Updated weights for policy 0, policy_version 1817452 (0.0009) [2023-12-27 04:38:05,451][105620] Updated weights for policy 1, policy_version 1821261 (0.0008) [2023-12-27 04:38:05,511][105620] Updated weights for policy 1, policy_version 1821271 (0.0005) [2023-12-27 04:38:05,567][105620] Updated weights for policy 1, policy_version 1821281 (0.0005) [2023-12-27 04:38:05,799][105692] Updated weights for policy 0, policy_version 1817462 (0.0010) [2023-12-27 04:38:05,852][105692] Updated weights for policy 0, policy_version 1817472 (0.0009) [2023-12-27 04:38:05,911][105692] Updated weights for policy 0, policy_version 1817482 (0.0008) [2023-12-27 04:38:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 931659776. Throughput: 0: 9824.7, 1: 10158.4. Samples: 931647480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:38:06,063][104569] Avg episode reward: [(0, '8990.927'), (1, '9260.670')] [2023-12-27 04:38:06,130][105620] Updated weights for policy 1, policy_version 1821291 (0.0006) [2023-12-27 04:38:06,189][105620] Updated weights for policy 1, policy_version 1821301 (0.0008) [2023-12-27 04:38:06,249][105620] Updated weights for policy 1, policy_version 1821311 (0.0008) [2023-12-27 04:38:06,612][105692] Updated weights for policy 0, policy_version 1817492 (0.0007) [2023-12-27 04:38:06,668][105692] Updated weights for policy 0, policy_version 1817502 (0.0010) [2023-12-27 04:38:06,726][105692] Updated weights for policy 0, policy_version 1817512 (0.0005) [2023-12-27 04:38:07,028][105620] Updated weights for policy 1, policy_version 1821321 (0.0008) [2023-12-27 04:38:07,090][105620] Updated weights for policy 1, policy_version 1821331 (0.0011) [2023-12-27 04:38:07,149][105620] Updated weights for policy 1, policy_version 1821341 (0.0011) [2023-12-27 04:38:07,212][105620] Updated weights for policy 1, policy_version 1821351 (0.0011) [2023-12-27 04:38:07,362][105692] Updated weights for policy 0, policy_version 1817522 (0.0006) [2023-12-27 04:38:07,422][105692] Updated weights for policy 0, policy_version 1817532 (0.0011) [2023-12-27 04:38:07,470][105692] Updated weights for policy 0, policy_version 1817542 (0.0010) [2023-12-27 04:38:07,519][105692] Updated weights for policy 0, policy_version 1817552 (0.0010) [2023-12-27 04:38:07,898][105620] Updated weights for policy 1, policy_version 1821361 (0.0007) [2023-12-27 04:38:07,952][105620] Updated weights for policy 1, policy_version 1821371 (0.0010) [2023-12-27 04:38:08,011][105620] Updated weights for policy 1, policy_version 1821382 (0.0011) [2023-12-27 04:38:08,187][105692] Updated weights for policy 0, policy_version 1817562 (0.0005) [2023-12-27 04:38:08,247][105692] Updated weights for policy 0, policy_version 1817572 (0.0005) [2023-12-27 04:38:08,303][105692] Updated weights for policy 0, policy_version 1817582 (0.0010) [2023-12-27 04:38:08,737][105620] Updated weights for policy 1, policy_version 1821392 (0.0010) [2023-12-27 04:38:08,786][105620] Updated weights for policy 1, policy_version 1821402 (0.0011) [2023-12-27 04:38:08,834][105620] Updated weights for policy 1, policy_version 1821412 (0.0011) [2023-12-27 04:38:09,051][105692] Updated weights for policy 0, policy_version 1817592 (0.0009) [2023-12-27 04:38:09,102][105692] Updated weights for policy 0, policy_version 1817602 (0.0009) [2023-12-27 04:38:09,149][105692] Updated weights for policy 0, policy_version 1817612 (0.0009) [2023-12-27 04:38:09,521][105620] Updated weights for policy 1, policy_version 1821422 (0.0008) [2023-12-27 04:38:09,584][105620] Updated weights for policy 1, policy_version 1821432 (0.0009) [2023-12-27 04:38:09,644][105620] Updated weights for policy 1, policy_version 1821442 (0.0009) [2023-12-27 04:38:09,959][105692] Updated weights for policy 0, policy_version 1817622 (0.0009) [2023-12-27 04:38:10,022][105692] Updated weights for policy 0, policy_version 1817632 (0.0009) [2023-12-27 04:38:10,081][105692] Updated weights for policy 0, policy_version 1817642 (0.0009) [2023-12-27 04:38:10,429][105620] Updated weights for policy 1, policy_version 1821452 (0.0009) [2023-12-27 04:38:10,495][105620] Updated weights for policy 1, policy_version 1821462 (0.0010) [2023-12-27 04:38:10,553][105620] Updated weights for policy 1, policy_version 1821472 (0.0007) [2023-12-27 04:38:10,732][105692] Updated weights for policy 0, policy_version 1817652 (0.0008) [2023-12-27 04:38:10,789][105692] Updated weights for policy 0, policy_version 1817662 (0.0009) [2023-12-27 04:38:10,845][105692] Updated weights for policy 0, policy_version 1817672 (0.0008) [2023-12-27 04:38:11,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 931758080. Throughput: 0: 9817.2, 1: 10068.8. Samples: 931764960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:38:11,062][104569] Avg episode reward: [(0, '8442.869'), (1, '9169.972')] [2023-12-27 04:38:11,242][105620] Updated weights for policy 1, policy_version 1821482 (0.0006) [2023-12-27 04:38:11,309][105620] Updated weights for policy 1, policy_version 1821492 (0.0008) [2023-12-27 04:38:11,376][105620] Updated weights for policy 1, policy_version 1821502 (0.0009) [2023-12-27 04:38:11,445][105620] Updated weights for policy 1, policy_version 1821512 (0.0009) [2023-12-27 04:38:11,670][105692] Updated weights for policy 0, policy_version 1817682 (0.0009) [2023-12-27 04:38:11,735][105692] Updated weights for policy 0, policy_version 1817692 (0.0010) [2023-12-27 04:38:11,793][105692] Updated weights for policy 0, policy_version 1817702 (0.0009) [2023-12-27 04:38:11,854][105692] Updated weights for policy 0, policy_version 1817712 (0.0008) [2023-12-27 04:38:12,195][105620] Updated weights for policy 1, policy_version 1821522 (0.0010) [2023-12-27 04:38:12,257][105620] Updated weights for policy 1, policy_version 1821532 (0.0006) [2023-12-27 04:38:12,322][105620] Updated weights for policy 1, policy_version 1821542 (0.0008) [2023-12-27 04:38:12,537][105692] Updated weights for policy 0, policy_version 1817722 (0.0010) [2023-12-27 04:38:12,582][105692] Updated weights for policy 0, policy_version 1817732 (0.0010) [2023-12-27 04:38:12,637][105692] Updated weights for policy 0, policy_version 1817742 (0.0011) [2023-12-27 04:38:13,035][105620] Updated weights for policy 1, policy_version 1821552 (0.0008) [2023-12-27 04:38:13,091][105620] Updated weights for policy 1, policy_version 1821562 (0.0008) [2023-12-27 04:38:13,145][105620] Updated weights for policy 1, policy_version 1821572 (0.0007) [2023-12-27 04:38:13,311][105692] Updated weights for policy 0, policy_version 1817752 (0.0008) [2023-12-27 04:38:13,370][105692] Updated weights for policy 0, policy_version 1817762 (0.0005) [2023-12-27 04:38:13,432][105692] Updated weights for policy 0, policy_version 1817772 (0.0007) [2023-12-27 04:38:13,916][105620] Updated weights for policy 1, policy_version 1821582 (0.0009) [2023-12-27 04:38:13,983][105620] Updated weights for policy 1, policy_version 1821592 (0.0007) [2023-12-27 04:38:14,029][105620] Updated weights for policy 1, policy_version 1821602 (0.0008) [2023-12-27 04:38:14,113][105692] Updated weights for policy 0, policy_version 1817782 (0.0010) [2023-12-27 04:38:14,171][105692] Updated weights for policy 0, policy_version 1817792 (0.0009) [2023-12-27 04:38:14,228][105692] Updated weights for policy 0, policy_version 1817802 (0.0008) [2023-12-27 04:38:14,788][105620] Updated weights for policy 1, policy_version 1821612 (0.0009) [2023-12-27 04:38:14,841][105620] Updated weights for policy 1, policy_version 1821622 (0.0010) [2023-12-27 04:38:14,904][105620] Updated weights for policy 1, policy_version 1821632 (0.0010) [2023-12-27 04:38:15,015][105692] Updated weights for policy 0, policy_version 1817812 (0.0009) [2023-12-27 04:38:15,078][105692] Updated weights for policy 0, policy_version 1817822 (0.0008) [2023-12-27 04:38:15,144][105692] Updated weights for policy 0, policy_version 1817832 (0.0008) [2023-12-27 04:38:15,592][105620] Updated weights for policy 1, policy_version 1821642 (0.0010) [2023-12-27 04:38:15,640][105620] Updated weights for policy 1, policy_version 1821652 (0.0010) [2023-12-27 04:38:15,706][105620] Updated weights for policy 1, policy_version 1821662 (0.0010) [2023-12-27 04:38:15,769][105620] Updated weights for policy 1, policy_version 1821672 (0.0010) [2023-12-27 04:38:15,820][105692] Updated weights for policy 0, policy_version 1817842 (0.0008) [2023-12-27 04:38:15,872][105692] Updated weights for policy 0, policy_version 1817852 (0.0008) [2023-12-27 04:38:15,933][105692] Updated weights for policy 0, policy_version 1817862 (0.0008) [2023-12-27 04:38:15,987][105692] Updated weights for policy 0, policy_version 1817872 (0.0009) [2023-12-27 04:38:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 20070.3, 300 sec: 19605.2). Total num frames: 931856384. Throughput: 0: 9800.3, 1: 9962.7. Samples: 931822544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:38:16,063][104569] Avg episode reward: [(0, '8533.374'), (1, '9260.099')] [2023-12-27 04:38:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001817872_465444864.pth... [2023-12-27 04:38:16,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001821672_466411520.pth... [2023-12-27 04:38:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001816720_465149952.pth [2023-12-27 04:38:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001820520_466116608.pth [2023-12-27 04:38:16,526][105620] Updated weights for policy 1, policy_version 1821682 (0.0011) [2023-12-27 04:38:16,581][105620] Updated weights for policy 1, policy_version 1821692 (0.0010) [2023-12-27 04:38:16,640][105620] Updated weights for policy 1, policy_version 1821702 (0.0010) [2023-12-27 04:38:16,684][105692] Updated weights for policy 0, policy_version 1817882 (0.0007) [2023-12-27 04:38:16,750][105692] Updated weights for policy 0, policy_version 1817892 (0.0008) [2023-12-27 04:38:16,809][105692] Updated weights for policy 0, policy_version 1817902 (0.0009) [2023-12-27 04:38:17,241][105620] Updated weights for policy 1, policy_version 1821712 (0.0010) [2023-12-27 04:38:17,299][105620] Updated weights for policy 1, policy_version 1821722 (0.0009) [2023-12-27 04:38:17,361][105620] Updated weights for policy 1, policy_version 1821732 (0.0006) [2023-12-27 04:38:17,645][105692] Updated weights for policy 0, policy_version 1817912 (0.0010) [2023-12-27 04:38:17,693][105692] Updated weights for policy 0, policy_version 1817922 (0.0009) [2023-12-27 04:38:17,745][105692] Updated weights for policy 0, policy_version 1817932 (0.0008) [2023-12-27 04:38:18,066][105620] Updated weights for policy 1, policy_version 1821742 (0.0010) [2023-12-27 04:38:18,129][105620] Updated weights for policy 1, policy_version 1821752 (0.0010) [2023-12-27 04:38:18,177][105620] Updated weights for policy 1, policy_version 1821762 (0.0009) [2023-12-27 04:38:18,414][105692] Updated weights for policy 0, policy_version 1817942 (0.0009) [2023-12-27 04:38:18,476][105692] Updated weights for policy 0, policy_version 1817952 (0.0009) [2023-12-27 04:38:18,538][105692] Updated weights for policy 0, policy_version 1817962 (0.0009) [2023-12-27 04:38:18,953][105620] Updated weights for policy 1, policy_version 1821772 (0.0009) [2023-12-27 04:38:19,004][105620] Updated weights for policy 1, policy_version 1821782 (0.0009) [2023-12-27 04:38:19,051][105620] Updated weights for policy 1, policy_version 1821792 (0.0008) [2023-12-27 04:38:19,237][105692] Updated weights for policy 0, policy_version 1817972 (0.0009) [2023-12-27 04:38:19,303][105692] Updated weights for policy 0, policy_version 1817982 (0.0008) [2023-12-27 04:38:19,371][105692] Updated weights for policy 0, policy_version 1817992 (0.0008) [2023-12-27 04:38:19,818][105620] Updated weights for policy 1, policy_version 1821802 (0.0009) [2023-12-27 04:38:19,882][105620] Updated weights for policy 1, policy_version 1821812 (0.0009) [2023-12-27 04:38:19,947][105620] Updated weights for policy 1, policy_version 1821822 (0.0010) [2023-12-27 04:38:20,013][105620] Updated weights for policy 1, policy_version 1821832 (0.0009) [2023-12-27 04:38:20,152][105692] Updated weights for policy 0, policy_version 1818002 (0.0008) [2023-12-27 04:38:20,222][105692] Updated weights for policy 0, policy_version 1818012 (0.0008) [2023-12-27 04:38:20,291][105692] Updated weights for policy 0, policy_version 1818022 (0.0008) [2023-12-27 04:38:20,354][105692] Updated weights for policy 0, policy_version 1818032 (0.0008) [2023-12-27 04:38:20,758][105620] Updated weights for policy 1, policy_version 1821842 (0.0009) [2023-12-27 04:38:20,826][105620] Updated weights for policy 1, policy_version 1821852 (0.0011) [2023-12-27 04:38:20,889][105620] Updated weights for policy 1, policy_version 1821862 (0.0011) [2023-12-27 04:38:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 931946496. Throughput: 0: 9771.6, 1: 9856.7. Samples: 931937568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:38:21,063][104569] Avg episode reward: [(0, '8533.110'), (1, '9258.294')] [2023-12-27 04:38:21,137][105692] Updated weights for policy 0, policy_version 1818042 (0.0008) [2023-12-27 04:38:21,200][105692] Updated weights for policy 0, policy_version 1818052 (0.0008) [2023-12-27 04:38:21,261][105692] Updated weights for policy 0, policy_version 1818062 (0.0008) [2023-12-27 04:38:21,670][105620] Updated weights for policy 1, policy_version 1821872 (0.0008) [2023-12-27 04:38:21,735][105620] Updated weights for policy 1, policy_version 1821882 (0.0008) [2023-12-27 04:38:21,780][105620] Updated weights for policy 1, policy_version 1821892 (0.0005) [2023-12-27 04:38:22,075][105692] Updated weights for policy 0, policy_version 1818072 (0.0010) [2023-12-27 04:38:22,128][105692] Updated weights for policy 0, policy_version 1818082 (0.0008) [2023-12-27 04:38:22,186][105692] Updated weights for policy 0, policy_version 1818092 (0.0008) [2023-12-27 04:38:22,414][105620] Updated weights for policy 1, policy_version 1821902 (0.0009) [2023-12-27 04:38:22,475][105620] Updated weights for policy 1, policy_version 1821912 (0.0011) [2023-12-27 04:38:22,521][105620] Updated weights for policy 1, policy_version 1821922 (0.0010) [2023-12-27 04:38:22,966][105692] Updated weights for policy 0, policy_version 1818102 (0.0009) [2023-12-27 04:38:23,024][105692] Updated weights for policy 0, policy_version 1818112 (0.0008) [2023-12-27 04:38:23,072][105692] Updated weights for policy 0, policy_version 1818122 (0.0006) [2023-12-27 04:38:23,330][105620] Updated weights for policy 1, policy_version 1821932 (0.0011) [2023-12-27 04:38:23,398][105620] Updated weights for policy 1, policy_version 1821942 (0.0007) [2023-12-27 04:38:23,466][105620] Updated weights for policy 1, policy_version 1821952 (0.0005) [2023-12-27 04:38:23,669][105692] Updated weights for policy 0, policy_version 1818132 (0.0006) [2023-12-27 04:38:23,726][105692] Updated weights for policy 0, policy_version 1818142 (0.0010) [2023-12-27 04:38:23,780][105692] Updated weights for policy 0, policy_version 1818152 (0.0010) [2023-12-27 04:38:23,981][105620] Updated weights for policy 1, policy_version 1821962 (0.0007) [2023-12-27 04:38:24,027][105620] Updated weights for policy 1, policy_version 1821972 (0.0005) [2023-12-27 04:38:24,078][105620] Updated weights for policy 1, policy_version 1821982 (0.0005) [2023-12-27 04:38:24,148][105620] Updated weights for policy 1, policy_version 1821992 (0.0005) [2023-12-27 04:38:24,619][105692] Updated weights for policy 0, policy_version 1818162 (0.0008) [2023-12-27 04:38:24,665][105692] Updated weights for policy 0, policy_version 1818172 (0.0006) [2023-12-27 04:38:24,714][105692] Updated weights for policy 0, policy_version 1818182 (0.0005) [2023-12-27 04:38:24,716][105620] Updated weights for policy 1, policy_version 1822002 (0.0005) [2023-12-27 04:38:24,765][105620] Updated weights for policy 1, policy_version 1822012 (0.0005) [2023-12-27 04:38:24,769][105692] Updated weights for policy 0, policy_version 1818192 (0.0008) [2023-12-27 04:38:24,812][105620] Updated weights for policy 1, policy_version 1822022 (0.0005) [2023-12-27 04:38:25,336][105620] Updated weights for policy 1, policy_version 1822032 (0.0005) [2023-12-27 04:38:25,387][105620] Updated weights for policy 1, policy_version 1822042 (0.0005) [2023-12-27 04:38:25,439][105620] Updated weights for policy 1, policy_version 1822052 (0.0006) [2023-12-27 04:38:25,570][105692] Updated weights for policy 0, policy_version 1818202 (0.0005) [2023-12-27 04:38:25,620][105692] Updated weights for policy 0, policy_version 1818212 (0.0007) [2023-12-27 04:38:25,675][105692] Updated weights for policy 0, policy_version 1818222 (0.0012) [2023-12-27 04:38:26,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 932044800. Throughput: 0: 9614.2, 1: 9893.6. Samples: 932055784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:38:26,062][104569] Avg episode reward: [(0, '8440.682'), (1, '9165.863')] [2023-12-27 04:38:26,097][105620] Updated weights for policy 1, policy_version 1822062 (0.0009) [2023-12-27 04:38:26,155][105620] Updated weights for policy 1, policy_version 1822072 (0.0010) [2023-12-27 04:38:26,226][105620] Updated weights for policy 1, policy_version 1822082 (0.0009) [2023-12-27 04:38:26,353][105692] Updated weights for policy 0, policy_version 1818232 (0.0006) [2023-12-27 04:38:26,401][105692] Updated weights for policy 0, policy_version 1818242 (0.0007) [2023-12-27 04:38:26,449][105692] Updated weights for policy 0, policy_version 1818252 (0.0010) [2023-12-27 04:38:26,887][105620] Updated weights for policy 1, policy_version 1822092 (0.0006) [2023-12-27 04:38:26,948][105620] Updated weights for policy 1, policy_version 1822102 (0.0005) [2023-12-27 04:38:27,016][105620] Updated weights for policy 1, policy_version 1822112 (0.0010) [2023-12-27 04:38:27,146][105692] Updated weights for policy 0, policy_version 1818262 (0.0007) [2023-12-27 04:38:27,212][105692] Updated weights for policy 0, policy_version 1818272 (0.0005) [2023-12-27 04:38:27,272][105692] Updated weights for policy 0, policy_version 1818282 (0.0005) [2023-12-27 04:38:27,617][105620] Updated weights for policy 1, policy_version 1822122 (0.0009) [2023-12-27 04:38:27,679][105620] Updated weights for policy 1, policy_version 1822132 (0.0005) [2023-12-27 04:38:27,736][105620] Updated weights for policy 1, policy_version 1822142 (0.0005) [2023-12-27 04:38:27,792][105620] Updated weights for policy 1, policy_version 1822152 (0.0005) [2023-12-27 04:38:27,970][105692] Updated weights for policy 0, policy_version 1818292 (0.0008) [2023-12-27 04:38:28,022][105692] Updated weights for policy 0, policy_version 1818303 (0.0008) [2023-12-27 04:38:28,066][105692] Updated weights for policy 0, policy_version 1818313 (0.0008) [2023-12-27 04:38:28,323][105620] Updated weights for policy 1, policy_version 1822162 (0.0005) [2023-12-27 04:38:28,385][105620] Updated weights for policy 1, policy_version 1822172 (0.0011) [2023-12-27 04:38:28,433][105620] Updated weights for policy 1, policy_version 1822182 (0.0010) [2023-12-27 04:38:28,895][105692] Updated weights for policy 0, policy_version 1818323 (0.0008) [2023-12-27 04:38:28,949][105692] Updated weights for policy 0, policy_version 1818333 (0.0008) [2023-12-27 04:38:29,003][105692] Updated weights for policy 0, policy_version 1818343 (0.0009) [2023-12-27 04:38:29,126][105620] Updated weights for policy 1, policy_version 1822192 (0.0009) [2023-12-27 04:38:29,177][105620] Updated weights for policy 1, policy_version 1822202 (0.0009) [2023-12-27 04:38:29,238][105620] Updated weights for policy 1, policy_version 1822212 (0.0009) [2023-12-27 04:38:29,807][105692] Updated weights for policy 0, policy_version 1818353 (0.0009) [2023-12-27 04:38:29,880][105692] Updated weights for policy 0, policy_version 1818363 (0.0006) [2023-12-27 04:38:29,945][105692] Updated weights for policy 0, policy_version 1818373 (0.0007) [2023-12-27 04:38:29,972][105620] Updated weights for policy 1, policy_version 1822222 (0.0008) [2023-12-27 04:38:29,987][105692] Updated weights for policy 0, policy_version 1818383 (0.0006) [2023-12-27 04:38:30,021][105620] Updated weights for policy 1, policy_version 1822232 (0.0009) [2023-12-27 04:38:30,075][105620] Updated weights for policy 1, policy_version 1822242 (0.0008) [2023-12-27 04:38:30,685][105692] Updated weights for policy 0, policy_version 1818393 (0.0005) [2023-12-27 04:38:30,738][105692] Updated weights for policy 0, policy_version 1818403 (0.0006) [2023-12-27 04:38:30,741][105620] Updated weights for policy 1, policy_version 1822252 (0.0008) [2023-12-27 04:38:30,796][105692] Updated weights for policy 0, policy_version 1818413 (0.0007) [2023-12-27 04:38:30,807][105620] Updated weights for policy 1, policy_version 1822262 (0.0007) [2023-12-27 04:38:30,854][105620] Updated weights for policy 1, policy_version 1822272 (0.0009) [2023-12-27 04:38:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 932151296. Throughput: 0: 9684.0, 1: 9941.7. Samples: 932117560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:38:31,062][104569] Avg episode reward: [(0, '8625.475'), (1, '9258.408')] [2023-12-27 04:38:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001818416_465584128.pth... [2023-12-27 04:38:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001822280_466567168.pth... [2023-12-27 04:38:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001821096_466264064.pth [2023-12-27 04:38:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001817296_465297408.pth [2023-12-27 04:38:31,522][105620] Updated weights for policy 1, policy_version 1822282 (0.0008) [2023-12-27 04:38:31,561][105692] Updated weights for policy 0, policy_version 1818423 (0.0007) [2023-12-27 04:38:31,586][105620] Updated weights for policy 1, policy_version 1822292 (0.0006) [2023-12-27 04:38:31,616][105692] Updated weights for policy 0, policy_version 1818433 (0.0009) [2023-12-27 04:38:31,654][105620] Updated weights for policy 1, policy_version 1822302 (0.0008) [2023-12-27 04:38:31,678][105692] Updated weights for policy 0, policy_version 1818443 (0.0006) [2023-12-27 04:38:31,712][105620] Updated weights for policy 1, policy_version 1822312 (0.0008) [2023-12-27 04:38:32,380][105692] Updated weights for policy 0, policy_version 1818453 (0.0007) [2023-12-27 04:38:32,439][105692] Updated weights for policy 0, policy_version 1818463 (0.0007) [2023-12-27 04:38:32,472][105620] Updated weights for policy 1, policy_version 1822322 (0.0008) [2023-12-27 04:38:32,502][105692] Updated weights for policy 0, policy_version 1818473 (0.0007) [2023-12-27 04:38:32,522][105620] Updated weights for policy 1, policy_version 1822332 (0.0008) [2023-12-27 04:38:32,576][105620] Updated weights for policy 1, policy_version 1822342 (0.0010) [2023-12-27 04:38:33,220][105620] Updated weights for policy 1, policy_version 1822352 (0.0010) [2023-12-27 04:38:33,273][105620] Updated weights for policy 1, policy_version 1822362 (0.0009) [2023-12-27 04:38:33,299][105692] Updated weights for policy 0, policy_version 1818483 (0.0008) [2023-12-27 04:38:33,319][105620] Updated weights for policy 1, policy_version 1822372 (0.0005) [2023-12-27 04:38:33,347][105692] Updated weights for policy 0, policy_version 1818493 (0.0008) [2023-12-27 04:38:33,400][105692] Updated weights for policy 0, policy_version 1818505 (0.0010) [2023-12-27 04:38:33,930][105620] Updated weights for policy 1, policy_version 1822382 (0.0008) [2023-12-27 04:38:33,985][105620] Updated weights for policy 1, policy_version 1822392 (0.0010) [2023-12-27 04:38:34,036][105620] Updated weights for policy 1, policy_version 1822402 (0.0010) [2023-12-27 04:38:34,263][105692] Updated weights for policy 0, policy_version 1818516 (0.0010) [2023-12-27 04:38:34,318][105692] Updated weights for policy 0, policy_version 1818526 (0.0009) [2023-12-27 04:38:34,371][105692] Updated weights for policy 0, policy_version 1818536 (0.0008) [2023-12-27 04:38:34,785][105620] Updated weights for policy 1, policy_version 1822412 (0.0008) [2023-12-27 04:38:34,850][105620] Updated weights for policy 1, policy_version 1822422 (0.0006) [2023-12-27 04:38:34,897][105620] Updated weights for policy 1, policy_version 1822432 (0.0005) [2023-12-27 04:38:35,105][105692] Updated weights for policy 0, policy_version 1818546 (0.0008) [2023-12-27 04:38:35,157][105692] Updated weights for policy 0, policy_version 1818556 (0.0008) [2023-12-27 04:38:35,211][105692] Updated weights for policy 0, policy_version 1818566 (0.0008) [2023-12-27 04:38:35,263][105692] Updated weights for policy 0, policy_version 1818576 (0.0008) [2023-12-27 04:38:35,518][105620] Updated weights for policy 1, policy_version 1822442 (0.0010) [2023-12-27 04:38:35,572][105620] Updated weights for policy 1, policy_version 1822452 (0.0010) [2023-12-27 04:38:35,624][105620] Updated weights for policy 1, policy_version 1822462 (0.0010) [2023-12-27 04:38:35,671][105620] Updated weights for policy 1, policy_version 1822472 (0.0010) [2023-12-27 04:38:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 932241408. Throughput: 0: 9584.4, 1: 9988.9. Samples: 932232744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:38:36,062][104569] Avg episode reward: [(0, '8899.833'), (1, '9350.882')] [2023-12-27 04:38:36,066][105692] Updated weights for policy 0, policy_version 1818586 (0.0010) [2023-12-27 04:38:36,127][105692] Updated weights for policy 0, policy_version 1818596 (0.0011) [2023-12-27 04:38:36,186][105692] Updated weights for policy 0, policy_version 1818606 (0.0010) [2023-12-27 04:38:36,412][105620] Updated weights for policy 1, policy_version 1822482 (0.0011) [2023-12-27 04:38:36,475][105620] Updated weights for policy 1, policy_version 1822492 (0.0011) [2023-12-27 04:38:36,545][105620] Updated weights for policy 1, policy_version 1822502 (0.0011) [2023-12-27 04:38:36,956][105692] Updated weights for policy 0, policy_version 1818616 (0.0009) [2023-12-27 04:38:37,015][105692] Updated weights for policy 0, policy_version 1818626 (0.0009) [2023-12-27 04:38:37,079][105692] Updated weights for policy 0, policy_version 1818636 (0.0007) [2023-12-27 04:38:37,231][105620] Updated weights for policy 1, policy_version 1822512 (0.0010) [2023-12-27 04:38:37,277][105620] Updated weights for policy 1, policy_version 1822522 (0.0008) [2023-12-27 04:38:37,341][105620] Updated weights for policy 1, policy_version 1822532 (0.0010) [2023-12-27 04:38:37,804][105692] Updated weights for policy 0, policy_version 1818646 (0.0009) [2023-12-27 04:38:37,847][105692] Updated weights for policy 0, policy_version 1818656 (0.0006) [2023-12-27 04:38:37,906][105692] Updated weights for policy 0, policy_version 1818666 (0.0006) [2023-12-27 04:38:38,131][105620] Updated weights for policy 1, policy_version 1822542 (0.0009) [2023-12-27 04:38:38,193][105620] Updated weights for policy 1, policy_version 1822552 (0.0009) [2023-12-27 04:38:38,244][105620] Updated weights for policy 1, policy_version 1822562 (0.0009) [2023-12-27 04:38:38,626][105692] Updated weights for policy 0, policy_version 1818676 (0.0007) [2023-12-27 04:38:38,693][105692] Updated weights for policy 0, policy_version 1818686 (0.0008) [2023-12-27 04:38:38,747][105692] Updated weights for policy 0, policy_version 1818696 (0.0009) [2023-12-27 04:38:39,037][105620] Updated weights for policy 1, policy_version 1822572 (0.0008) [2023-12-27 04:38:39,086][105620] Updated weights for policy 1, policy_version 1822582 (0.0005) [2023-12-27 04:38:39,136][105620] Updated weights for policy 1, policy_version 1822592 (0.0005) [2023-12-27 04:38:39,413][105692] Updated weights for policy 0, policy_version 1818706 (0.0009) [2023-12-27 04:38:39,471][105692] Updated weights for policy 0, policy_version 1818716 (0.0006) [2023-12-27 04:38:39,538][105692] Updated weights for policy 0, policy_version 1818726 (0.0007) [2023-12-27 04:38:39,597][105692] Updated weights for policy 0, policy_version 1818736 (0.0010) [2023-12-27 04:38:39,832][105620] Updated weights for policy 1, policy_version 1822602 (0.0006) [2023-12-27 04:38:39,895][105620] Updated weights for policy 1, policy_version 1822612 (0.0009) [2023-12-27 04:38:39,960][105620] Updated weights for policy 1, policy_version 1822622 (0.0009) [2023-12-27 04:38:40,023][105620] Updated weights for policy 1, policy_version 1822632 (0.0009) [2023-12-27 04:38:40,349][105692] Updated weights for policy 0, policy_version 1818746 (0.0009) [2023-12-27 04:38:40,411][105692] Updated weights for policy 0, policy_version 1818756 (0.0009) [2023-12-27 04:38:40,469][105692] Updated weights for policy 0, policy_version 1818766 (0.0010) [2023-12-27 04:38:40,730][105620] Updated weights for policy 1, policy_version 1822642 (0.0009) [2023-12-27 04:38:40,800][105620] Updated weights for policy 1, policy_version 1822652 (0.0010) [2023-12-27 04:38:40,864][105620] Updated weights for policy 1, policy_version 1822662 (0.0010) [2023-12-27 04:38:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 932339712. Throughput: 0: 9481.0, 1: 10026.9. Samples: 932347180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:38:41,062][104569] Avg episode reward: [(0, '8987.600'), (1, '9258.809')] [2023-12-27 04:38:41,204][105692] Updated weights for policy 0, policy_version 1818776 (0.0009) [2023-12-27 04:38:41,258][105692] Updated weights for policy 0, policy_version 1818786 (0.0009) [2023-12-27 04:38:41,320][105692] Updated weights for policy 0, policy_version 1818796 (0.0009) [2023-12-27 04:38:41,626][105620] Updated weights for policy 1, policy_version 1822672 (0.0009) [2023-12-27 04:38:41,685][105620] Updated weights for policy 1, policy_version 1822682 (0.0009) [2023-12-27 04:38:41,754][105620] Updated weights for policy 1, policy_version 1822692 (0.0008) [2023-12-27 04:38:42,128][105692] Updated weights for policy 0, policy_version 1818806 (0.0008) [2023-12-27 04:38:42,192][105692] Updated weights for policy 0, policy_version 1818816 (0.0006) [2023-12-27 04:38:42,253][105692] Updated weights for policy 0, policy_version 1818826 (0.0008) [2023-12-27 04:38:42,565][105620] Updated weights for policy 1, policy_version 1822702 (0.0010) [2023-12-27 04:38:42,617][105620] Updated weights for policy 1, policy_version 1822712 (0.0009) [2023-12-27 04:38:42,668][105620] Updated weights for policy 1, policy_version 1822722 (0.0009) [2023-12-27 04:38:42,989][105692] Updated weights for policy 0, policy_version 1818836 (0.0009) [2023-12-27 04:38:43,051][105692] Updated weights for policy 0, policy_version 1818846 (0.0010) [2023-12-27 04:38:43,113][105692] Updated weights for policy 0, policy_version 1818856 (0.0011) [2023-12-27 04:38:43,382][105620] Updated weights for policy 1, policy_version 1822732 (0.0007) [2023-12-27 04:38:43,434][105620] Updated weights for policy 1, policy_version 1822742 (0.0005) [2023-12-27 04:38:43,487][105620] Updated weights for policy 1, policy_version 1822752 (0.0005) [2023-12-27 04:38:43,813][105692] Updated weights for policy 0, policy_version 1818866 (0.0011) [2023-12-27 04:38:43,875][105692] Updated weights for policy 0, policy_version 1818876 (0.0010) [2023-12-27 04:38:43,936][105692] Updated weights for policy 0, policy_version 1818886 (0.0010) [2023-12-27 04:38:43,985][105692] Updated weights for policy 0, policy_version 1818896 (0.0009) [2023-12-27 04:38:44,136][105620] Updated weights for policy 1, policy_version 1822762 (0.0005) [2023-12-27 04:38:44,190][105620] Updated weights for policy 1, policy_version 1822772 (0.0008) [2023-12-27 04:38:44,245][105620] Updated weights for policy 1, policy_version 1822782 (0.0010) [2023-12-27 04:38:44,298][105620] Updated weights for policy 1, policy_version 1822792 (0.0008) [2023-12-27 04:38:44,584][105692] Updated weights for policy 0, policy_version 1818906 (0.0006) [2023-12-27 04:38:44,645][105692] Updated weights for policy 0, policy_version 1818916 (0.0006) [2023-12-27 04:38:44,703][105692] Updated weights for policy 0, policy_version 1818926 (0.0008) [2023-12-27 04:38:45,021][105620] Updated weights for policy 1, policy_version 1822802 (0.0006) [2023-12-27 04:38:45,078][105620] Updated weights for policy 1, policy_version 1822812 (0.0008) [2023-12-27 04:38:45,128][105620] Updated weights for policy 1, policy_version 1822822 (0.0006) [2023-12-27 04:38:45,415][105692] Updated weights for policy 0, policy_version 1818936 (0.0010) [2023-12-27 04:38:45,468][105692] Updated weights for policy 0, policy_version 1818946 (0.0011) [2023-12-27 04:38:45,520][105692] Updated weights for policy 0, policy_version 1818956 (0.0010) [2023-12-27 04:38:45,855][105620] Updated weights for policy 1, policy_version 1822832 (0.0008) [2023-12-27 04:38:45,908][105620] Updated weights for policy 1, policy_version 1822842 (0.0008) [2023-12-27 04:38:45,965][105620] Updated weights for policy 1, policy_version 1822852 (0.0009) [2023-12-27 04:38:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 932438016. Throughput: 0: 9434.8, 1: 9967.8. Samples: 932404696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:38:46,062][104569] Avg episode reward: [(0, '8804.853'), (1, '9166.409')] [2023-12-27 04:38:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001818960_465723392.pth... [2023-12-27 04:38:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001822856_466714624.pth... [2023-12-27 04:38:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001817872_465444864.pth [2023-12-27 04:38:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001821672_466411520.pth [2023-12-27 04:38:46,237][105692] Updated weights for policy 0, policy_version 1818966 (0.0007) [2023-12-27 04:38:46,293][105692] Updated weights for policy 0, policy_version 1818976 (0.0005) [2023-12-27 04:38:46,349][105692] Updated weights for policy 0, policy_version 1818986 (0.0005) [2023-12-27 04:38:46,777][105620] Updated weights for policy 1, policy_version 1822862 (0.0010) [2023-12-27 04:38:46,845][105620] Updated weights for policy 1, policy_version 1822872 (0.0011) [2023-12-27 04:38:46,908][105620] Updated weights for policy 1, policy_version 1822882 (0.0007) [2023-12-27 04:38:46,979][105692] Updated weights for policy 0, policy_version 1818996 (0.0006) [2023-12-27 04:38:47,047][105692] Updated weights for policy 0, policy_version 1819006 (0.0008) [2023-12-27 04:38:47,117][105692] Updated weights for policy 0, policy_version 1819016 (0.0009) [2023-12-27 04:38:47,519][105620] Updated weights for policy 1, policy_version 1822892 (0.0007) [2023-12-27 04:38:47,578][105620] Updated weights for policy 1, policy_version 1822902 (0.0009) [2023-12-27 04:38:47,638][105620] Updated weights for policy 1, policy_version 1822912 (0.0009) [2023-12-27 04:38:47,909][105692] Updated weights for policy 0, policy_version 1819026 (0.0009) [2023-12-27 04:38:47,960][105692] Updated weights for policy 0, policy_version 1819036 (0.0009) [2023-12-27 04:38:48,012][105692] Updated weights for policy 0, policy_version 1819046 (0.0009) [2023-12-27 04:38:48,068][105692] Updated weights for policy 0, policy_version 1819056 (0.0009) [2023-12-27 04:38:48,299][105620] Updated weights for policy 1, policy_version 1822922 (0.0008) [2023-12-27 04:38:48,360][105620] Updated weights for policy 1, policy_version 1822932 (0.0007) [2023-12-27 04:38:48,414][105620] Updated weights for policy 1, policy_version 1822942 (0.0008) [2023-12-27 04:38:48,476][105620] Updated weights for policy 1, policy_version 1822952 (0.0007) [2023-12-27 04:38:48,848][105692] Updated weights for policy 0, policy_version 1819066 (0.0008) [2023-12-27 04:38:48,907][105692] Updated weights for policy 0, policy_version 1819076 (0.0009) [2023-12-27 04:38:48,967][105692] Updated weights for policy 0, policy_version 1819086 (0.0010) [2023-12-27 04:38:49,182][105620] Updated weights for policy 1, policy_version 1822962 (0.0009) [2023-12-27 04:38:49,244][105620] Updated weights for policy 1, policy_version 1822972 (0.0009) [2023-12-27 04:38:49,305][105620] Updated weights for policy 1, policy_version 1822982 (0.0010) [2023-12-27 04:38:49,671][105692] Updated weights for policy 0, policy_version 1819096 (0.0008) [2023-12-27 04:38:49,717][105692] Updated weights for policy 0, policy_version 1819106 (0.0008) [2023-12-27 04:38:49,764][105692] Updated weights for policy 0, policy_version 1819116 (0.0009) [2023-12-27 04:38:50,106][105620] Updated weights for policy 1, policy_version 1822992 (0.0011) [2023-12-27 04:38:50,163][105620] Updated weights for policy 1, policy_version 1823002 (0.0011) [2023-12-27 04:38:50,223][105620] Updated weights for policy 1, policy_version 1823012 (0.0009) [2023-12-27 04:38:50,619][105692] Updated weights for policy 0, policy_version 1819126 (0.0009) [2023-12-27 04:38:50,674][105692] Updated weights for policy 0, policy_version 1819136 (0.0008) [2023-12-27 04:38:50,731][105692] Updated weights for policy 0, policy_version 1819146 (0.0007) [2023-12-27 04:38:50,884][105620] Updated weights for policy 1, policy_version 1823022 (0.0005) [2023-12-27 04:38:50,948][105620] Updated weights for policy 1, policy_version 1823032 (0.0006) [2023-12-27 04:38:51,009][105620] Updated weights for policy 1, policy_version 1823042 (0.0008) [2023-12-27 04:38:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 932536320. Throughput: 0: 9508.6, 1: 9921.4. Samples: 932521824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:38:51,062][104569] Avg episode reward: [(0, '8805.218'), (1, '9166.006')] [2023-12-27 04:38:51,434][105692] Updated weights for policy 0, policy_version 1819156 (0.0007) [2023-12-27 04:38:51,490][105692] Updated weights for policy 0, policy_version 1819166 (0.0008) [2023-12-27 04:38:51,545][105692] Updated weights for policy 0, policy_version 1819176 (0.0008) [2023-12-27 04:38:51,753][105620] Updated weights for policy 1, policy_version 1823052 (0.0009) [2023-12-27 04:38:51,815][105620] Updated weights for policy 1, policy_version 1823062 (0.0009) [2023-12-27 04:38:51,873][105620] Updated weights for policy 1, policy_version 1823072 (0.0009) [2023-12-27 04:38:52,286][105692] Updated weights for policy 0, policy_version 1819186 (0.0008) [2023-12-27 04:38:52,351][105692] Updated weights for policy 0, policy_version 1819196 (0.0009) [2023-12-27 04:38:52,418][105692] Updated weights for policy 0, policy_version 1819206 (0.0008) [2023-12-27 04:38:52,479][105692] Updated weights for policy 0, policy_version 1819216 (0.0008) [2023-12-27 04:38:52,629][105620] Updated weights for policy 1, policy_version 1823082 (0.0008) [2023-12-27 04:38:52,684][105620] Updated weights for policy 1, policy_version 1823092 (0.0010) [2023-12-27 04:38:52,736][105620] Updated weights for policy 1, policy_version 1823102 (0.0009) [2023-12-27 04:38:52,783][105620] Updated weights for policy 1, policy_version 1823112 (0.0008) [2023-12-27 04:38:53,178][105692] Updated weights for policy 0, policy_version 1819226 (0.0011) [2023-12-27 04:38:53,239][105692] Updated weights for policy 0, policy_version 1819236 (0.0010) [2023-12-27 04:38:53,303][105692] Updated weights for policy 0, policy_version 1819246 (0.0010) [2023-12-27 04:38:53,623][105620] Updated weights for policy 1, policy_version 1823122 (0.0010) [2023-12-27 04:38:53,677][105620] Updated weights for policy 1, policy_version 1823132 (0.0010) [2023-12-27 04:38:53,735][105620] Updated weights for policy 1, policy_version 1823143 (0.0010) [2023-12-27 04:38:53,862][105692] Updated weights for policy 0, policy_version 1819256 (0.0006) [2023-12-27 04:38:53,911][105692] Updated weights for policy 0, policy_version 1819266 (0.0005) [2023-12-27 04:38:53,957][105692] Updated weights for policy 0, policy_version 1819276 (0.0008) [2023-12-27 04:38:54,590][105620] Updated weights for policy 1, policy_version 1823153 (0.0008) [2023-12-27 04:38:54,634][105620] Updated weights for policy 1, policy_version 1823163 (0.0008) [2023-12-27 04:38:54,636][105692] Updated weights for policy 0, policy_version 1819286 (0.0010) [2023-12-27 04:38:54,680][105620] Updated weights for policy 1, policy_version 1823173 (0.0007) [2023-12-27 04:38:54,695][105692] Updated weights for policy 0, policy_version 1819296 (0.0010) [2023-12-27 04:38:54,760][105692] Updated weights for policy 0, policy_version 1819306 (0.0010) [2023-12-27 04:38:55,442][105692] Updated weights for policy 0, policy_version 1819316 (0.0008) [2023-12-27 04:38:55,492][105692] Updated weights for policy 0, policy_version 1819326 (0.0005) [2023-12-27 04:38:55,499][105620] Updated weights for policy 1, policy_version 1823183 (0.0008) [2023-12-27 04:38:55,545][105620] Updated weights for policy 1, policy_version 1823193 (0.0008) [2023-12-27 04:38:55,547][105692] Updated weights for policy 0, policy_version 1819336 (0.0006) [2023-12-27 04:38:55,600][105620] Updated weights for policy 1, policy_version 1823203 (0.0008) [2023-12-27 04:38:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 932626432. Throughput: 0: 9539.2, 1: 9810.5. Samples: 932635696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:38:56,062][104569] Avg episode reward: [(0, '8532.494'), (1, '9258.459')] [2023-12-27 04:38:56,247][105692] Updated weights for policy 0, policy_version 1819346 (0.0008) [2023-12-27 04:38:56,305][105692] Updated weights for policy 0, policy_version 1819356 (0.0011) [2023-12-27 04:38:56,359][105692] Updated weights for policy 0, policy_version 1819366 (0.0010) [2023-12-27 04:38:56,378][105620] Updated weights for policy 1, policy_version 1823213 (0.0009) [2023-12-27 04:38:56,421][105692] Updated weights for policy 0, policy_version 1819376 (0.0010) [2023-12-27 04:38:56,438][105620] Updated weights for policy 1, policy_version 1823223 (0.0009) [2023-12-27 04:38:56,494][105620] Updated weights for policy 1, policy_version 1823233 (0.0008) [2023-12-27 04:38:57,156][105692] Updated weights for policy 0, policy_version 1819386 (0.0011) [2023-12-27 04:38:57,205][105692] Updated weights for policy 0, policy_version 1819396 (0.0006) [2023-12-27 04:38:57,251][105692] Updated weights for policy 0, policy_version 1819406 (0.0005) [2023-12-27 04:38:57,251][105620] Updated weights for policy 1, policy_version 1823243 (0.0008) [2023-12-27 04:38:57,302][105620] Updated weights for policy 1, policy_version 1823253 (0.0009) [2023-12-27 04:38:57,355][105620] Updated weights for policy 1, policy_version 1823264 (0.0010) [2023-12-27 04:38:57,942][105692] Updated weights for policy 0, policy_version 1819416 (0.0009) [2023-12-27 04:38:58,006][105692] Updated weights for policy 0, policy_version 1819426 (0.0010) [2023-12-27 04:38:58,064][105692] Updated weights for policy 0, policy_version 1819436 (0.0010) [2023-12-27 04:38:58,122][105620] Updated weights for policy 1, policy_version 1823274 (0.0009) [2023-12-27 04:38:58,187][105620] Updated weights for policy 1, policy_version 1823284 (0.0008) [2023-12-27 04:38:58,240][105620] Updated weights for policy 1, policy_version 1823294 (0.0008) [2023-12-27 04:38:58,298][105620] Updated weights for policy 1, policy_version 1823304 (0.0006) [2023-12-27 04:38:58,815][105692] Updated weights for policy 0, policy_version 1819446 (0.0009) [2023-12-27 04:38:58,884][105692] Updated weights for policy 0, policy_version 1819456 (0.0009) [2023-12-27 04:38:58,957][105692] Updated weights for policy 0, policy_version 1819466 (0.0009) [2023-12-27 04:38:59,091][105620] Updated weights for policy 1, policy_version 1823314 (0.0006) [2023-12-27 04:38:59,138][105620] Updated weights for policy 1, policy_version 1823324 (0.0009) [2023-12-27 04:38:59,191][105620] Updated weights for policy 1, policy_version 1823334 (0.0008) [2023-12-27 04:38:59,690][105692] Updated weights for policy 0, policy_version 1819476 (0.0006) [2023-12-27 04:38:59,745][105692] Updated weights for policy 0, policy_version 1819486 (0.0005) [2023-12-27 04:38:59,808][105692] Updated weights for policy 0, policy_version 1819496 (0.0006) [2023-12-27 04:38:59,966][105620] Updated weights for policy 1, policy_version 1823344 (0.0007) [2023-12-27 04:39:00,032][105620] Updated weights for policy 1, policy_version 1823354 (0.0009) [2023-12-27 04:39:00,094][105620] Updated weights for policy 1, policy_version 1823364 (0.0006) [2023-12-27 04:39:00,506][105692] Updated weights for policy 0, policy_version 1819506 (0.0009) [2023-12-27 04:39:00,563][105692] Updated weights for policy 0, policy_version 1819516 (0.0009) [2023-12-27 04:39:00,610][105692] Updated weights for policy 0, policy_version 1819526 (0.0009) [2023-12-27 04:39:00,656][105692] Updated weights for policy 0, policy_version 1819536 (0.0009) [2023-12-27 04:39:00,783][105620] Updated weights for policy 1, policy_version 1823374 (0.0009) [2023-12-27 04:39:00,839][105620] Updated weights for policy 1, policy_version 1823385 (0.0009) [2023-12-27 04:39:00,893][105620] Updated weights for policy 1, policy_version 1823396 (0.0010) [2023-12-27 04:39:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 932724736. Throughput: 0: 9531.1, 1: 9791.9. Samples: 932692072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:39:01,062][104569] Avg episode reward: [(0, '8625.608'), (1, '9350.883')] [2023-12-27 04:39:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001819536_465870848.pth... [2023-12-27 04:39:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001823400_466853888.pth... [2023-12-27 04:39:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001818416_465584128.pth [2023-12-27 04:39:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001822280_466567168.pth [2023-12-27 04:39:01,356][105692] Updated weights for policy 0, policy_version 1819546 (0.0009) [2023-12-27 04:39:01,415][105692] Updated weights for policy 0, policy_version 1819556 (0.0009) [2023-12-27 04:39:01,466][105692] Updated weights for policy 0, policy_version 1819566 (0.0009) [2023-12-27 04:39:01,698][105620] Updated weights for policy 1, policy_version 1823407 (0.0010) [2023-12-27 04:39:01,763][105620] Updated weights for policy 1, policy_version 1823418 (0.0008) [2023-12-27 04:39:01,814][105620] Updated weights for policy 1, policy_version 1823428 (0.0005) [2023-12-27 04:39:02,249][105692] Updated weights for policy 0, policy_version 1819576 (0.0009) [2023-12-27 04:39:02,309][105692] Updated weights for policy 0, policy_version 1819586 (0.0008) [2023-12-27 04:39:02,373][105692] Updated weights for policy 0, policy_version 1819596 (0.0008) [2023-12-27 04:39:02,580][105620] Updated weights for policy 1, policy_version 1823438 (0.0010) [2023-12-27 04:39:02,644][105620] Updated weights for policy 1, policy_version 1823448 (0.0009) [2023-12-27 04:39:02,707][105620] Updated weights for policy 1, policy_version 1823458 (0.0008) [2023-12-27 04:39:03,084][105692] Updated weights for policy 0, policy_version 1819606 (0.0005) [2023-12-27 04:39:03,141][105692] Updated weights for policy 0, policy_version 1819616 (0.0005) [2023-12-27 04:39:03,195][105692] Updated weights for policy 0, policy_version 1819626 (0.0005) [2023-12-27 04:39:03,433][105620] Updated weights for policy 1, policy_version 1823468 (0.0009) [2023-12-27 04:39:03,495][105620] Updated weights for policy 1, policy_version 1823478 (0.0010) [2023-12-27 04:39:03,552][105620] Updated weights for policy 1, policy_version 1823488 (0.0008) [2023-12-27 04:39:03,781][105692] Updated weights for policy 0, policy_version 1819636 (0.0007) [2023-12-27 04:39:03,838][105692] Updated weights for policy 0, policy_version 1819646 (0.0010) [2023-12-27 04:39:03,900][105692] Updated weights for policy 0, policy_version 1819656 (0.0009) [2023-12-27 04:39:04,168][105620] Updated weights for policy 1, policy_version 1823498 (0.0006) [2023-12-27 04:39:04,241][105620] Updated weights for policy 1, policy_version 1823508 (0.0011) [2023-12-27 04:39:04,311][105620] Updated weights for policy 1, policy_version 1823518 (0.0011) [2023-12-27 04:39:04,374][105620] Updated weights for policy 1, policy_version 1823528 (0.0011) [2023-12-27 04:39:04,620][105692] Updated weights for policy 0, policy_version 1819666 (0.0010) [2023-12-27 04:39:04,688][105692] Updated weights for policy 0, policy_version 1819676 (0.0006) [2023-12-27 04:39:04,747][105692] Updated weights for policy 0, policy_version 1819686 (0.0007) [2023-12-27 04:39:04,811][105692] Updated weights for policy 0, policy_version 1819696 (0.0005) [2023-12-27 04:39:05,065][105620] Updated weights for policy 1, policy_version 1823538 (0.0010) [2023-12-27 04:39:05,117][105620] Updated weights for policy 1, policy_version 1823548 (0.0010) [2023-12-27 04:39:05,162][105620] Updated weights for policy 1, policy_version 1823558 (0.0007) [2023-12-27 04:39:05,341][105692] Updated weights for policy 0, policy_version 1819706 (0.0008) [2023-12-27 04:39:05,396][105692] Updated weights for policy 0, policy_version 1819716 (0.0008) [2023-12-27 04:39:05,444][105692] Updated weights for policy 0, policy_version 1819726 (0.0008) [2023-12-27 04:39:05,867][105620] Updated weights for policy 1, policy_version 1823568 (0.0006) [2023-12-27 04:39:05,920][105620] Updated weights for policy 1, policy_version 1823578 (0.0005) [2023-12-27 04:39:05,966][105620] Updated weights for policy 1, policy_version 1823588 (0.0005) [2023-12-27 04:39:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 932823040. Throughput: 0: 9577.6, 1: 9789.3. Samples: 932809080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:39:06,063][104569] Avg episode reward: [(0, '8805.501'), (1, '9165.898')] [2023-12-27 04:39:06,105][105692] Updated weights for policy 0, policy_version 1819736 (0.0009) [2023-12-27 04:39:06,161][105692] Updated weights for policy 0, policy_version 1819746 (0.0010) [2023-12-27 04:39:06,230][105692] Updated weights for policy 0, policy_version 1819756 (0.0006) [2023-12-27 04:39:06,668][105620] Updated weights for policy 1, policy_version 1823598 (0.0005) [2023-12-27 04:39:06,733][105620] Updated weights for policy 1, policy_version 1823608 (0.0011) [2023-12-27 04:39:06,792][105692] Updated weights for policy 0, policy_version 1819766 (0.0005) [2023-12-27 04:39:06,799][105620] Updated weights for policy 1, policy_version 1823618 (0.0011) [2023-12-27 04:39:06,846][105692] Updated weights for policy 0, policy_version 1819776 (0.0005) [2023-12-27 04:39:06,914][105692] Updated weights for policy 0, policy_version 1819786 (0.0006) [2023-12-27 04:39:07,379][105620] Updated weights for policy 1, policy_version 1823628 (0.0011) [2023-12-27 04:39:07,452][105620] Updated weights for policy 1, policy_version 1823638 (0.0011) [2023-12-27 04:39:07,494][105692] Updated weights for policy 0, policy_version 1819796 (0.0006) [2023-12-27 04:39:07,511][105620] Updated weights for policy 1, policy_version 1823648 (0.0011) [2023-12-27 04:39:07,556][105692] Updated weights for policy 0, policy_version 1819806 (0.0007) [2023-12-27 04:39:07,610][105692] Updated weights for policy 0, policy_version 1819816 (0.0008) [2023-12-27 04:39:08,185][105620] Updated weights for policy 1, policy_version 1823658 (0.0010) [2023-12-27 04:39:08,233][105620] Updated weights for policy 1, policy_version 1823668 (0.0010) [2023-12-27 04:39:08,281][105620] Updated weights for policy 1, policy_version 1823678 (0.0010) [2023-12-27 04:39:08,348][105620] Updated weights for policy 1, policy_version 1823688 (0.0011) [2023-12-27 04:39:08,394][105692] Updated weights for policy 0, policy_version 1819826 (0.0008) [2023-12-27 04:39:08,457][105692] Updated weights for policy 0, policy_version 1819836 (0.0010) [2023-12-27 04:39:08,512][105692] Updated weights for policy 0, policy_version 1819846 (0.0007) [2023-12-27 04:39:08,564][105692] Updated weights for policy 0, policy_version 1819856 (0.0005) [2023-12-27 04:39:09,100][105620] Updated weights for policy 1, policy_version 1823698 (0.0011) [2023-12-27 04:39:09,151][105620] Updated weights for policy 1, policy_version 1823708 (0.0010) [2023-12-27 04:39:09,200][105620] Updated weights for policy 1, policy_version 1823718 (0.0010) [2023-12-27 04:39:09,227][105692] Updated weights for policy 0, policy_version 1819866 (0.0010) [2023-12-27 04:39:09,286][105692] Updated weights for policy 0, policy_version 1819876 (0.0009) [2023-12-27 04:39:09,348][105692] Updated weights for policy 0, policy_version 1819886 (0.0010) [2023-12-27 04:39:09,997][105620] Updated weights for policy 1, policy_version 1823728 (0.0009) [2023-12-27 04:39:10,041][105692] Updated weights for policy 0, policy_version 1819896 (0.0007) [2023-12-27 04:39:10,052][105620] Updated weights for policy 1, policy_version 1823738 (0.0009) [2023-12-27 04:39:10,100][105692] Updated weights for policy 0, policy_version 1819906 (0.0006) [2023-12-27 04:39:10,113][105620] Updated weights for policy 1, policy_version 1823748 (0.0008) [2023-12-27 04:39:10,157][105692] Updated weights for policy 0, policy_version 1819916 (0.0008) [2023-12-27 04:39:10,777][105620] Updated weights for policy 1, policy_version 1823758 (0.0008) [2023-12-27 04:39:10,828][105620] Updated weights for policy 1, policy_version 1823768 (0.0009) [2023-12-27 04:39:10,892][105620] Updated weights for policy 1, policy_version 1823778 (0.0009) [2023-12-27 04:39:10,903][105692] Updated weights for policy 0, policy_version 1819926 (0.0007) [2023-12-27 04:39:10,954][105692] Updated weights for policy 0, policy_version 1819936 (0.0007) [2023-12-27 04:39:11,012][105692] Updated weights for policy 0, policy_version 1819946 (0.0009) [2023-12-27 04:39:11,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 932929536. Throughput: 0: 9756.3, 1: 9714.6. Samples: 932931972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:39:11,062][104569] Avg episode reward: [(0, '8623.692'), (1, '9073.672')] [2023-12-27 04:39:11,692][105620] Updated weights for policy 1, policy_version 1823788 (0.0007) [2023-12-27 04:39:11,760][105620] Updated weights for policy 1, policy_version 1823798 (0.0010) [2023-12-27 04:39:11,761][105692] Updated weights for policy 0, policy_version 1819956 (0.0010) [2023-12-27 04:39:11,817][105620] Updated weights for policy 1, policy_version 1823808 (0.0009) [2023-12-27 04:39:11,825][105692] Updated weights for policy 0, policy_version 1819966 (0.0010) [2023-12-27 04:39:11,885][105692] Updated weights for policy 0, policy_version 1819976 (0.0008) [2023-12-27 04:39:12,526][105620] Updated weights for policy 1, policy_version 1823818 (0.0008) [2023-12-27 04:39:12,577][105620] Updated weights for policy 1, policy_version 1823828 (0.0005) [2023-12-27 04:39:12,630][105620] Updated weights for policy 1, policy_version 1823838 (0.0005) [2023-12-27 04:39:12,688][105620] Updated weights for policy 1, policy_version 1823848 (0.0007) [2023-12-27 04:39:12,709][105692] Updated weights for policy 0, policy_version 1819986 (0.0009) [2023-12-27 04:39:12,774][105692] Updated weights for policy 0, policy_version 1819996 (0.0010) [2023-12-27 04:39:12,833][105692] Updated weights for policy 0, policy_version 1820006 (0.0011) [2023-12-27 04:39:12,896][105692] Updated weights for policy 0, policy_version 1820016 (0.0010) [2023-12-27 04:39:13,386][105620] Updated weights for policy 1, policy_version 1823858 (0.0008) [2023-12-27 04:39:13,440][105620] Updated weights for policy 1, policy_version 1823868 (0.0006) [2023-12-27 04:39:13,491][105620] Updated weights for policy 1, policy_version 1823878 (0.0005) [2023-12-27 04:39:13,646][105692] Updated weights for policy 0, policy_version 1820026 (0.0010) [2023-12-27 04:39:13,693][105692] Updated weights for policy 0, policy_version 1820036 (0.0010) [2023-12-27 04:39:13,755][105692] Updated weights for policy 0, policy_version 1820046 (0.0010) [2023-12-27 04:39:14,183][105620] Updated weights for policy 1, policy_version 1823888 (0.0009) [2023-12-27 04:39:14,243][105620] Updated weights for policy 1, policy_version 1823898 (0.0009) [2023-12-27 04:39:14,298][105620] Updated weights for policy 1, policy_version 1823908 (0.0006) [2023-12-27 04:39:14,450][105692] Updated weights for policy 0, policy_version 1820056 (0.0007) [2023-12-27 04:39:14,503][105692] Updated weights for policy 0, policy_version 1820067 (0.0010) [2023-12-27 04:39:14,552][105692] Updated weights for policy 0, policy_version 1820077 (0.0008) [2023-12-27 04:39:14,909][105620] Updated weights for policy 1, policy_version 1823918 (0.0009) [2023-12-27 04:39:14,975][105620] Updated weights for policy 1, policy_version 1823928 (0.0010) [2023-12-27 04:39:15,038][105620] Updated weights for policy 1, policy_version 1823938 (0.0009) [2023-12-27 04:39:15,356][105692] Updated weights for policy 0, policy_version 1820087 (0.0008) [2023-12-27 04:39:15,403][105692] Updated weights for policy 0, policy_version 1820097 (0.0009) [2023-12-27 04:39:15,454][105692] Updated weights for policy 0, policy_version 1820107 (0.0008) [2023-12-27 04:39:15,793][105620] Updated weights for policy 1, policy_version 1823948 (0.0009) [2023-12-27 04:39:15,851][105620] Updated weights for policy 1, policy_version 1823958 (0.0009) [2023-12-27 04:39:15,905][105620] Updated weights for policy 1, policy_version 1823969 (0.0006) [2023-12-27 04:39:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 933019648. Throughput: 0: 9703.9, 1: 9649.0. Samples: 932988440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:39:16,063][104569] Avg episode reward: [(0, '8444.513'), (1, '9258.472')] [2023-12-27 04:39:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001820112_466018304.pth... [2023-12-27 04:39:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001823976_467001344.pth... [2023-12-27 04:39:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001822856_466714624.pth [2023-12-27 04:39:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001818960_465723392.pth [2023-12-27 04:39:16,297][105692] Updated weights for policy 0, policy_version 1820117 (0.0009) [2023-12-27 04:39:16,355][105692] Updated weights for policy 0, policy_version 1820127 (0.0007) [2023-12-27 04:39:16,411][105692] Updated weights for policy 0, policy_version 1820137 (0.0008) [2023-12-27 04:39:16,523][105620] Updated weights for policy 1, policy_version 1823979 (0.0007) [2023-12-27 04:39:16,582][105620] Updated weights for policy 1, policy_version 1823989 (0.0011) [2023-12-27 04:39:16,640][105620] Updated weights for policy 1, policy_version 1823999 (0.0010) [2023-12-27 04:39:17,174][105692] Updated weights for policy 0, policy_version 1820147 (0.0009) [2023-12-27 04:39:17,222][105692] Updated weights for policy 0, policy_version 1820157 (0.0009) [2023-12-27 04:39:17,278][105692] Updated weights for policy 0, policy_version 1820168 (0.0009) [2023-12-27 04:39:17,333][105620] Updated weights for policy 1, policy_version 1824009 (0.0010) [2023-12-27 04:39:17,382][105620] Updated weights for policy 1, policy_version 1824019 (0.0005) [2023-12-27 04:39:17,427][105620] Updated weights for policy 1, policy_version 1824029 (0.0005) [2023-12-27 04:39:17,480][105620] Updated weights for policy 1, policy_version 1824039 (0.0005) [2023-12-27 04:39:18,034][105620] Updated weights for policy 1, policy_version 1824049 (0.0005) [2023-12-27 04:39:18,102][105620] Updated weights for policy 1, policy_version 1824059 (0.0005) [2023-12-27 04:39:18,120][105692] Updated weights for policy 0, policy_version 1820178 (0.0010) [2023-12-27 04:39:18,169][105620] Updated weights for policy 1, policy_version 1824069 (0.0005) [2023-12-27 04:39:18,187][105692] Updated weights for policy 0, policy_version 1820188 (0.0008) [2023-12-27 04:39:18,255][105692] Updated weights for policy 0, policy_version 1820198 (0.0009) [2023-12-27 04:39:18,331][105692] Updated weights for policy 0, policy_version 1820208 (0.0010) [2023-12-27 04:39:18,751][105620] Updated weights for policy 1, policy_version 1824079 (0.0005) [2023-12-27 04:39:18,812][105620] Updated weights for policy 1, policy_version 1824089 (0.0009) [2023-12-27 04:39:18,877][105620] Updated weights for policy 1, policy_version 1824099 (0.0009) [2023-12-27 04:39:19,102][105692] Updated weights for policy 0, policy_version 1820218 (0.0009) [2023-12-27 04:39:19,153][105692] Updated weights for policy 0, policy_version 1820228 (0.0008) [2023-12-27 04:39:19,213][105692] Updated weights for policy 0, policy_version 1820238 (0.0008) [2023-12-27 04:39:19,621][105620] Updated weights for policy 1, policy_version 1824109 (0.0008) [2023-12-27 04:39:19,681][105620] Updated weights for policy 1, policy_version 1824119 (0.0008) [2023-12-27 04:39:19,745][105620] Updated weights for policy 1, policy_version 1824129 (0.0008) [2023-12-27 04:39:20,060][105692] Updated weights for policy 0, policy_version 1820248 (0.0009) [2023-12-27 04:39:20,118][105692] Updated weights for policy 0, policy_version 1820258 (0.0010) [2023-12-27 04:39:20,179][105692] Updated weights for policy 0, policy_version 1820268 (0.0010) [2023-12-27 04:39:20,505][105620] Updated weights for policy 1, policy_version 1824139 (0.0008) [2023-12-27 04:39:20,571][105620] Updated weights for policy 1, policy_version 1824149 (0.0009) [2023-12-27 04:39:20,641][105620] Updated weights for policy 1, policy_version 1824159 (0.0008) [2023-12-27 04:39:20,946][105692] Updated weights for policy 0, policy_version 1820278 (0.0010) [2023-12-27 04:39:21,011][105692] Updated weights for policy 0, policy_version 1820288 (0.0010) [2023-12-27 04:39:21,062][104569] Fps is (10 sec: 18022.1, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 933109760. Throughput: 0: 9687.3, 1: 9668.0. Samples: 933103732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:39:21,063][104569] Avg episode reward: [(0, '8444.028'), (1, '9168.788')] [2023-12-27 04:39:21,074][105692] Updated weights for policy 0, policy_version 1820298 (0.0008) [2023-12-27 04:39:21,445][105620] Updated weights for policy 1, policy_version 1824169 (0.0008) [2023-12-27 04:39:21,509][105620] Updated weights for policy 1, policy_version 1824179 (0.0005) [2023-12-27 04:39:21,560][105620] Updated weights for policy 1, policy_version 1824189 (0.0006) [2023-12-27 04:39:21,609][105620] Updated weights for policy 1, policy_version 1824199 (0.0008) [2023-12-27 04:39:21,864][105692] Updated weights for policy 0, policy_version 1820308 (0.0009) [2023-12-27 04:39:21,928][105692] Updated weights for policy 0, policy_version 1820318 (0.0011) [2023-12-27 04:39:21,998][105692] Updated weights for policy 0, policy_version 1820328 (0.0011) [2023-12-27 04:39:22,369][105620] Updated weights for policy 1, policy_version 1824209 (0.0008) [2023-12-27 04:39:22,436][105620] Updated weights for policy 1, policy_version 1824219 (0.0008) [2023-12-27 04:39:22,498][105620] Updated weights for policy 1, policy_version 1824229 (0.0008) [2023-12-27 04:39:22,747][105692] Updated weights for policy 0, policy_version 1820338 (0.0011) [2023-12-27 04:39:22,802][105692] Updated weights for policy 0, policy_version 1820348 (0.0011) [2023-12-27 04:39:22,858][105692] Updated weights for policy 0, policy_version 1820358 (0.0010) [2023-12-27 04:39:22,921][105692] Updated weights for policy 0, policy_version 1820368 (0.0011) [2023-12-27 04:39:23,264][105620] Updated weights for policy 1, policy_version 1824239 (0.0009) [2023-12-27 04:39:23,319][105620] Updated weights for policy 1, policy_version 1824249 (0.0008) [2023-12-27 04:39:23,367][105620] Updated weights for policy 1, policy_version 1824259 (0.0008) [2023-12-27 04:39:23,682][105692] Updated weights for policy 0, policy_version 1820378 (0.0010) [2023-12-27 04:39:23,745][105692] Updated weights for policy 0, policy_version 1820388 (0.0011) [2023-12-27 04:39:23,801][105692] Updated weights for policy 0, policy_version 1820398 (0.0011) [2023-12-27 04:39:24,124][105620] Updated weights for policy 1, policy_version 1824269 (0.0007) [2023-12-27 04:39:24,173][105620] Updated weights for policy 1, policy_version 1824279 (0.0005) [2023-12-27 04:39:24,221][105620] Updated weights for policy 1, policy_version 1824289 (0.0006) [2023-12-27 04:39:24,462][105692] Updated weights for policy 0, policy_version 1820408 (0.0011) [2023-12-27 04:39:24,527][105692] Updated weights for policy 0, policy_version 1820418 (0.0010) [2023-12-27 04:39:24,600][105692] Updated weights for policy 0, policy_version 1820428 (0.0011) [2023-12-27 04:39:24,783][105620] Updated weights for policy 1, policy_version 1824299 (0.0009) [2023-12-27 04:39:24,829][105620] Updated weights for policy 1, policy_version 1824309 (0.0005) [2023-12-27 04:39:24,879][105620] Updated weights for policy 1, policy_version 1824319 (0.0008) [2023-12-27 04:39:25,178][105692] Updated weights for policy 0, policy_version 1820438 (0.0009) [2023-12-27 04:39:25,223][105692] Updated weights for policy 0, policy_version 1820448 (0.0010) [2023-12-27 04:39:25,274][105692] Updated weights for policy 0, policy_version 1820458 (0.0010) [2023-12-27 04:39:25,470][105620] Updated weights for policy 1, policy_version 1824329 (0.0010) [2023-12-27 04:39:25,520][105620] Updated weights for policy 1, policy_version 1824339 (0.0010) [2023-12-27 04:39:25,571][105620] Updated weights for policy 1, policy_version 1824349 (0.0010) [2023-12-27 04:39:25,623][105620] Updated weights for policy 1, policy_version 1824359 (0.0010) [2023-12-27 04:39:25,895][105692] Updated weights for policy 0, policy_version 1820468 (0.0008) [2023-12-27 04:39:25,949][105692] Updated weights for policy 0, policy_version 1820478 (0.0006) [2023-12-27 04:39:26,003][105692] Updated weights for policy 0, policy_version 1820488 (0.0005) [2023-12-27 04:39:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 933216256. Throughput: 0: 9696.4, 1: 9704.4. Samples: 933220216. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:39:26,062][104569] Avg episode reward: [(0, '8899.711'), (1, '9076.219')] [2023-12-27 04:39:26,283][105620] Updated weights for policy 1, policy_version 1824369 (0.0010) [2023-12-27 04:39:26,336][105620] Updated weights for policy 1, policy_version 1824379 (0.0010) [2023-12-27 04:39:26,394][105620] Updated weights for policy 1, policy_version 1824389 (0.0010) [2023-12-27 04:39:26,629][105692] Updated weights for policy 0, policy_version 1820498 (0.0005) [2023-12-27 04:39:26,697][105692] Updated weights for policy 0, policy_version 1820508 (0.0006) [2023-12-27 04:39:26,761][105692] Updated weights for policy 0, policy_version 1820518 (0.0010) [2023-12-27 04:39:26,822][105692] Updated weights for policy 0, policy_version 1820528 (0.0010) [2023-12-27 04:39:27,122][105620] Updated weights for policy 1, policy_version 1824399 (0.0011) [2023-12-27 04:39:27,170][105620] Updated weights for policy 1, policy_version 1824409 (0.0010) [2023-12-27 04:39:27,218][105620] Updated weights for policy 1, policy_version 1824419 (0.0010) [2023-12-27 04:39:27,420][105692] Updated weights for policy 0, policy_version 1820538 (0.0010) [2023-12-27 04:39:27,474][105692] Updated weights for policy 0, policy_version 1820548 (0.0007) [2023-12-27 04:39:27,537][105692] Updated weights for policy 0, policy_version 1820558 (0.0007) [2023-12-27 04:39:27,902][105620] Updated weights for policy 1, policy_version 1824429 (0.0010) [2023-12-27 04:39:27,953][105620] Updated weights for policy 1, policy_version 1824439 (0.0010) [2023-12-27 04:39:28,007][105620] Updated weights for policy 1, policy_version 1824449 (0.0010) [2023-12-27 04:39:28,239][105692] Updated weights for policy 0, policy_version 1820568 (0.0010) [2023-12-27 04:39:28,294][105692] Updated weights for policy 0, policy_version 1820578 (0.0010) [2023-12-27 04:39:28,348][105692] Updated weights for policy 0, policy_version 1820588 (0.0010) [2023-12-27 04:39:28,652][105620] Updated weights for policy 1, policy_version 1824459 (0.0007) [2023-12-27 04:39:28,717][105620] Updated weights for policy 1, policy_version 1824469 (0.0007) [2023-12-27 04:39:28,776][105620] Updated weights for policy 1, policy_version 1824479 (0.0010) [2023-12-27 04:39:28,996][105692] Updated weights for policy 0, policy_version 1820598 (0.0010) [2023-12-27 04:39:29,045][105692] Updated weights for policy 0, policy_version 1820608 (0.0010) [2023-12-27 04:39:29,093][105692] Updated weights for policy 0, policy_version 1820618 (0.0010) [2023-12-27 04:39:29,308][105620] Updated weights for policy 1, policy_version 1824489 (0.0005) [2023-12-27 04:39:29,377][105620] Updated weights for policy 1, policy_version 1824499 (0.0009) [2023-12-27 04:39:29,441][105620] Updated weights for policy 1, policy_version 1824509 (0.0010) [2023-12-27 04:39:29,500][105620] Updated weights for policy 1, policy_version 1824519 (0.0010) [2023-12-27 04:39:29,859][105692] Updated weights for policy 0, policy_version 1820628 (0.0009) [2023-12-27 04:39:29,911][105692] Updated weights for policy 0, policy_version 1820638 (0.0008) [2023-12-27 04:39:29,967][105692] Updated weights for policy 0, policy_version 1820648 (0.0008) [2023-12-27 04:39:30,215][105620] Updated weights for policy 1, policy_version 1824529 (0.0010) [2023-12-27 04:39:30,273][105620] Updated weights for policy 1, policy_version 1824539 (0.0010) [2023-12-27 04:39:30,324][105620] Updated weights for policy 1, policy_version 1824549 (0.0010) [2023-12-27 04:39:30,736][105692] Updated weights for policy 0, policy_version 1820658 (0.0008) [2023-12-27 04:39:30,797][105692] Updated weights for policy 0, policy_version 1820668 (0.0005) [2023-12-27 04:39:30,864][105692] Updated weights for policy 0, policy_version 1820678 (0.0005) [2023-12-27 04:39:30,933][105692] Updated weights for policy 0, policy_version 1820688 (0.0005) [2023-12-27 04:39:31,037][105620] Updated weights for policy 1, policy_version 1824559 (0.0011) [2023-12-27 04:39:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 933314560. Throughput: 0: 9772.5, 1: 9765.0. Samples: 933283884. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:39:31,062][104569] Avg episode reward: [(0, '8440.616'), (1, '8980.843')] [2023-12-27 04:39:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001820688_466165760.pth... [2023-12-27 04:39:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001819536_465870848.pth [2023-12-27 04:39:31,100][105620] Updated weights for policy 1, policy_version 1824569 (0.0011) [2023-12-27 04:39:31,167][105620] Updated weights for policy 1, policy_version 1824579 (0.0011) [2023-12-27 04:39:31,190][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001824584_467156992.pth... [2023-12-27 04:39:31,193][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001823400_466853888.pth [2023-12-27 04:39:31,610][105692] Updated weights for policy 0, policy_version 1820698 (0.0008) [2023-12-27 04:39:31,700][105692] Updated weights for policy 0, policy_version 1820708 (0.0009) [2023-12-27 04:39:31,762][105692] Updated weights for policy 0, policy_version 1820718 (0.0008) [2023-12-27 04:39:31,883][105620] Updated weights for policy 1, policy_version 1824589 (0.0010) [2023-12-27 04:39:31,928][105620] Updated weights for policy 1, policy_version 1824599 (0.0010) [2023-12-27 04:39:31,976][105620] Updated weights for policy 1, policy_version 1824609 (0.0010) [2023-12-27 04:39:32,429][105692] Updated weights for policy 0, policy_version 1820728 (0.0010) [2023-12-27 04:39:32,494][105692] Updated weights for policy 0, policy_version 1820738 (0.0010) [2023-12-27 04:39:32,557][105692] Updated weights for policy 0, policy_version 1820748 (0.0008) [2023-12-27 04:39:32,679][105620] Updated weights for policy 1, policy_version 1824619 (0.0009) [2023-12-27 04:39:32,745][105620] Updated weights for policy 1, policy_version 1824629 (0.0010) [2023-12-27 04:39:32,803][105620] Updated weights for policy 1, policy_version 1824639 (0.0005) [2023-12-27 04:39:33,157][105692] Updated weights for policy 0, policy_version 1820758 (0.0005) [2023-12-27 04:39:33,203][105692] Updated weights for policy 0, policy_version 1820768 (0.0005) [2023-12-27 04:39:33,246][105692] Updated weights for policy 0, policy_version 1820778 (0.0005) [2023-12-27 04:39:33,397][105620] Updated weights for policy 1, policy_version 1824649 (0.0006) [2023-12-27 04:39:33,459][105620] Updated weights for policy 1, policy_version 1824659 (0.0010) [2023-12-27 04:39:33,502][105620] Updated weights for policy 1, policy_version 1824669 (0.0010) [2023-12-27 04:39:33,547][105620] Updated weights for policy 1, policy_version 1824679 (0.0010) [2023-12-27 04:39:33,900][105692] Updated weights for policy 0, policy_version 1820788 (0.0007) [2023-12-27 04:39:33,954][105692] Updated weights for policy 0, policy_version 1820798 (0.0008) [2023-12-27 04:39:33,999][105692] Updated weights for policy 0, policy_version 1820808 (0.0005) [2023-12-27 04:39:34,275][105620] Updated weights for policy 1, policy_version 1824689 (0.0010) [2023-12-27 04:39:34,344][105620] Updated weights for policy 1, policy_version 1824699 (0.0009) [2023-12-27 04:39:34,411][105620] Updated weights for policy 1, policy_version 1824709 (0.0010) [2023-12-27 04:39:34,711][105692] Updated weights for policy 0, policy_version 1820818 (0.0007) [2023-12-27 04:39:34,778][105692] Updated weights for policy 0, policy_version 1820828 (0.0011) [2023-12-27 04:39:34,834][105692] Updated weights for policy 0, policy_version 1820838 (0.0010) [2023-12-27 04:39:34,885][105692] Updated weights for policy 0, policy_version 1820848 (0.0010) [2023-12-27 04:39:35,124][105620] Updated weights for policy 1, policy_version 1824719 (0.0006) [2023-12-27 04:39:35,190][105620] Updated weights for policy 1, policy_version 1824729 (0.0009) [2023-12-27 04:39:35,255][105620] Updated weights for policy 1, policy_version 1824739 (0.0007) [2023-12-27 04:39:35,585][105692] Updated weights for policy 0, policy_version 1820858 (0.0010) [2023-12-27 04:39:35,646][105692] Updated weights for policy 0, policy_version 1820868 (0.0010) [2023-12-27 04:39:35,712][105692] Updated weights for policy 0, policy_version 1820878 (0.0008) [2023-12-27 04:39:35,804][105620] Updated weights for policy 1, policy_version 1824749 (0.0008) [2023-12-27 04:39:35,860][105620] Updated weights for policy 1, policy_version 1824759 (0.0007) [2023-12-27 04:39:35,916][105620] Updated weights for policy 1, policy_version 1824769 (0.0008) [2023-12-27 04:39:36,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.7, 300 sec: 19605.3). Total num frames: 933421056. Throughput: 0: 9805.8, 1: 9828.7. Samples: 933405376. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:39:36,063][104569] Avg episode reward: [(0, '8261.617'), (1, '8980.725')] [2023-12-27 04:39:36,424][105692] Updated weights for policy 0, policy_version 1820888 (0.0007) [2023-12-27 04:39:36,492][105692] Updated weights for policy 0, policy_version 1820898 (0.0009) [2023-12-27 04:39:36,556][105692] Updated weights for policy 0, policy_version 1820908 (0.0005) [2023-12-27 04:39:36,611][105620] Updated weights for policy 1, policy_version 1824779 (0.0008) [2023-12-27 04:39:36,677][105620] Updated weights for policy 1, policy_version 1824789 (0.0009) [2023-12-27 04:39:36,755][105620] Updated weights for policy 1, policy_version 1824799 (0.0006) [2023-12-27 04:39:37,124][105692] Updated weights for policy 0, policy_version 1820918 (0.0008) [2023-12-27 04:39:37,188][105692] Updated weights for policy 0, policy_version 1820928 (0.0010) [2023-12-27 04:39:37,250][105692] Updated weights for policy 0, policy_version 1820938 (0.0010) [2023-12-27 04:39:37,429][105620] Updated weights for policy 1, policy_version 1824809 (0.0010) [2023-12-27 04:39:37,484][105620] Updated weights for policy 1, policy_version 1824819 (0.0009) [2023-12-27 04:39:37,542][105620] Updated weights for policy 1, policy_version 1824829 (0.0009) [2023-12-27 04:39:37,596][105620] Updated weights for policy 1, policy_version 1824839 (0.0010) [2023-12-27 04:39:38,049][105692] Updated weights for policy 0, policy_version 1820948 (0.0009) [2023-12-27 04:39:38,104][105692] Updated weights for policy 0, policy_version 1820958 (0.0008) [2023-12-27 04:39:38,167][105692] Updated weights for policy 0, policy_version 1820968 (0.0008) [2023-12-27 04:39:38,246][105620] Updated weights for policy 1, policy_version 1824849 (0.0009) [2023-12-27 04:39:38,305][105620] Updated weights for policy 1, policy_version 1824859 (0.0009) [2023-12-27 04:39:38,372][105620] Updated weights for policy 1, policy_version 1824869 (0.0008) [2023-12-27 04:39:38,925][105692] Updated weights for policy 0, policy_version 1820978 (0.0009) [2023-12-27 04:39:38,977][105692] Updated weights for policy 0, policy_version 1820988 (0.0010) [2023-12-27 04:39:39,031][105692] Updated weights for policy 0, policy_version 1820999 (0.0009) [2023-12-27 04:39:39,096][105620] Updated weights for policy 1, policy_version 1824879 (0.0007) [2023-12-27 04:39:39,152][105620] Updated weights for policy 1, policy_version 1824889 (0.0005) [2023-12-27 04:39:39,208][105620] Updated weights for policy 1, policy_version 1824899 (0.0006) [2023-12-27 04:39:39,861][105620] Updated weights for policy 1, policy_version 1824909 (0.0007) [2023-12-27 04:39:39,895][105692] Updated weights for policy 0, policy_version 1821009 (0.0007) [2023-12-27 04:39:39,931][105620] Updated weights for policy 1, policy_version 1824919 (0.0008) [2023-12-27 04:39:39,960][105692] Updated weights for policy 0, policy_version 1821019 (0.0008) [2023-12-27 04:39:39,992][105620] Updated weights for policy 1, policy_version 1824929 (0.0011) [2023-12-27 04:39:40,024][105692] Updated weights for policy 0, policy_version 1821029 (0.0007) [2023-12-27 04:39:40,091][105692] Updated weights for policy 0, policy_version 1821039 (0.0007) [2023-12-27 04:39:40,630][105620] Updated weights for policy 1, policy_version 1824939 (0.0010) [2023-12-27 04:39:40,692][105620] Updated weights for policy 1, policy_version 1824949 (0.0011) [2023-12-27 04:39:40,761][105620] Updated weights for policy 1, policy_version 1824959 (0.0011) [2023-12-27 04:39:40,886][105692] Updated weights for policy 0, policy_version 1821049 (0.0008) [2023-12-27 04:39:40,944][105692] Updated weights for policy 0, policy_version 1821059 (0.0008) [2023-12-27 04:39:40,989][105692] Updated weights for policy 0, policy_version 1821069 (0.0009) [2023-12-27 04:39:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 933519360. Throughput: 0: 9726.4, 1: 9987.3. Samples: 933522812. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:39:41,063][104569] Avg episode reward: [(0, '8633.166'), (1, '8981.430')] [2023-12-27 04:39:41,500][105620] Updated weights for policy 1, policy_version 1824969 (0.0010) [2023-12-27 04:39:41,569][105620] Updated weights for policy 1, policy_version 1824979 (0.0008) [2023-12-27 04:39:41,636][105620] Updated weights for policy 1, policy_version 1824989 (0.0008) [2023-12-27 04:39:41,696][105620] Updated weights for policy 1, policy_version 1824999 (0.0009) [2023-12-27 04:39:41,808][105692] Updated weights for policy 0, policy_version 1821079 (0.0009) [2023-12-27 04:39:41,872][105692] Updated weights for policy 0, policy_version 1821089 (0.0009) [2023-12-27 04:39:41,922][105692] Updated weights for policy 0, policy_version 1821099 (0.0008) [2023-12-27 04:39:42,458][105620] Updated weights for policy 1, policy_version 1825009 (0.0006) [2023-12-27 04:39:42,517][105620] Updated weights for policy 1, policy_version 1825019 (0.0006) [2023-12-27 04:39:42,573][105620] Updated weights for policy 1, policy_version 1825029 (0.0006) [2023-12-27 04:39:42,715][105692] Updated weights for policy 0, policy_version 1821109 (0.0008) [2023-12-27 04:39:42,777][105692] Updated weights for policy 0, policy_version 1821119 (0.0009) [2023-12-27 04:39:42,839][105692] Updated weights for policy 0, policy_version 1821129 (0.0009) [2023-12-27 04:39:43,245][105620] Updated weights for policy 1, policy_version 1825039 (0.0009) [2023-12-27 04:39:43,302][105620] Updated weights for policy 1, policy_version 1825049 (0.0008) [2023-12-27 04:39:43,359][105620] Updated weights for policy 1, policy_version 1825059 (0.0009) [2023-12-27 04:39:43,537][105692] Updated weights for policy 0, policy_version 1821139 (0.0008) [2023-12-27 04:39:43,595][105692] Updated weights for policy 0, policy_version 1821149 (0.0005) [2023-12-27 04:39:43,654][105692] Updated weights for policy 0, policy_version 1821159 (0.0005) [2023-12-27 04:39:44,131][105620] Updated weights for policy 1, policy_version 1825069 (0.0009) [2023-12-27 04:39:44,196][105620] Updated weights for policy 1, policy_version 1825079 (0.0009) [2023-12-27 04:39:44,255][105692] Updated weights for policy 0, policy_version 1821169 (0.0006) [2023-12-27 04:39:44,257][105620] Updated weights for policy 1, policy_version 1825089 (0.0007) [2023-12-27 04:39:44,316][105692] Updated weights for policy 0, policy_version 1821179 (0.0008) [2023-12-27 04:39:44,379][105692] Updated weights for policy 0, policy_version 1821189 (0.0007) [2023-12-27 04:39:44,443][105692] Updated weights for policy 0, policy_version 1821199 (0.0010) [2023-12-27 04:39:45,024][105620] Updated weights for policy 1, policy_version 1825099 (0.0008) [2023-12-27 04:39:45,081][105620] Updated weights for policy 1, policy_version 1825109 (0.0009) [2023-12-27 04:39:45,142][105620] Updated weights for policy 1, policy_version 1825119 (0.0009) [2023-12-27 04:39:45,167][105692] Updated weights for policy 0, policy_version 1821209 (0.0009) [2023-12-27 04:39:45,233][105692] Updated weights for policy 0, policy_version 1821219 (0.0010) [2023-12-27 04:39:45,283][105692] Updated weights for policy 0, policy_version 1821229 (0.0011) [2023-12-27 04:39:45,798][105620] Updated weights for policy 1, policy_version 1825129 (0.0008) [2023-12-27 04:39:45,868][105620] Updated weights for policy 1, policy_version 1825139 (0.0005) [2023-12-27 04:39:45,938][105620] Updated weights for policy 1, policy_version 1825149 (0.0005) [2023-12-27 04:39:46,007][105620] Updated weights for policy 1, policy_version 1825159 (0.0005) [2023-12-27 04:39:46,023][105692] Updated weights for policy 0, policy_version 1821239 (0.0008) [2023-12-27 04:39:46,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 933609472. Throughput: 0: 9706.7, 1: 10001.1. Samples: 933578920. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:39:46,062][104569] Avg episode reward: [(0, '8167.244'), (1, '9258.604')] [2023-12-27 04:39:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001825160_467304448.pth... [2023-12-27 04:39:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001823976_467001344.pth [2023-12-27 04:39:46,081][105692] Updated weights for policy 0, policy_version 1821249 (0.0010) [2023-12-27 04:39:46,139][105692] Updated weights for policy 0, policy_version 1821259 (0.0010) [2023-12-27 04:39:46,161][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001821264_466313216.pth... [2023-12-27 04:39:46,165][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001820112_466018304.pth [2023-12-27 04:39:46,614][105620] Updated weights for policy 1, policy_version 1825169 (0.0010) [2023-12-27 04:39:46,679][105620] Updated weights for policy 1, policy_version 1825179 (0.0010) [2023-12-27 04:39:46,744][105620] Updated weights for policy 1, policy_version 1825189 (0.0008) [2023-12-27 04:39:46,877][105692] Updated weights for policy 0, policy_version 1821269 (0.0010) [2023-12-27 04:39:46,938][105692] Updated weights for policy 0, policy_version 1821279 (0.0010) [2023-12-27 04:39:46,986][105692] Updated weights for policy 0, policy_version 1821289 (0.0010) [2023-12-27 04:39:47,430][105620] Updated weights for policy 1, policy_version 1825199 (0.0008) [2023-12-27 04:39:47,484][105620] Updated weights for policy 1, policy_version 1825210 (0.0010) [2023-12-27 04:39:47,533][105620] Updated weights for policy 1, policy_version 1825221 (0.0009) [2023-12-27 04:39:47,631][105692] Updated weights for policy 0, policy_version 1821299 (0.0009) [2023-12-27 04:39:47,696][105692] Updated weights for policy 0, policy_version 1821309 (0.0005) [2023-12-27 04:39:47,757][105692] Updated weights for policy 0, policy_version 1821319 (0.0005) [2023-12-27 04:39:48,319][105620] Updated weights for policy 1, policy_version 1825231 (0.0008) [2023-12-27 04:39:48,367][105692] Updated weights for policy 0, policy_version 1821329 (0.0005) [2023-12-27 04:39:48,391][105620] Updated weights for policy 1, policy_version 1825241 (0.0009) [2023-12-27 04:39:48,424][105692] Updated weights for policy 0, policy_version 1821339 (0.0007) [2023-12-27 04:39:48,454][105620] Updated weights for policy 1, policy_version 1825251 (0.0007) [2023-12-27 04:39:48,484][105692] Updated weights for policy 0, policy_version 1821349 (0.0007) [2023-12-27 04:39:48,541][105692] Updated weights for policy 0, policy_version 1821359 (0.0005) [2023-12-27 04:39:49,091][105692] Updated weights for policy 0, policy_version 1821369 (0.0007) [2023-12-27 04:39:49,151][105692] Updated weights for policy 0, policy_version 1821379 (0.0007) [2023-12-27 04:39:49,222][105692] Updated weights for policy 0, policy_version 1821389 (0.0007) [2023-12-27 04:39:49,333][105620] Updated weights for policy 1, policy_version 1825261 (0.0008) [2023-12-27 04:39:49,412][105620] Updated weights for policy 1, policy_version 1825271 (0.0009) [2023-12-27 04:39:49,486][105620] Updated weights for policy 1, policy_version 1825281 (0.0007) [2023-12-27 04:39:49,849][105692] Updated weights for policy 0, policy_version 1821399 (0.0009) [2023-12-27 04:39:49,911][105692] Updated weights for policy 0, policy_version 1821409 (0.0009) [2023-12-27 04:39:49,978][105692] Updated weights for policy 0, policy_version 1821419 (0.0011) [2023-12-27 04:39:50,141][105620] Updated weights for policy 1, policy_version 1825291 (0.0009) [2023-12-27 04:39:50,210][105620] Updated weights for policy 1, policy_version 1825301 (0.0008) [2023-12-27 04:39:50,271][105620] Updated weights for policy 1, policy_version 1825311 (0.0008) [2023-12-27 04:39:50,737][105692] Updated weights for policy 0, policy_version 1821429 (0.0011) [2023-12-27 04:39:50,797][105692] Updated weights for policy 0, policy_version 1821439 (0.0011) [2023-12-27 04:39:50,858][105692] Updated weights for policy 0, policy_version 1821449 (0.0011) [2023-12-27 04:39:51,032][105620] Updated weights for policy 1, policy_version 1825321 (0.0008) [2023-12-27 04:39:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 933707776. Throughput: 0: 9774.0, 1: 9992.5. Samples: 933698576. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:39:51,063][104569] Avg episode reward: [(0, '8351.209'), (1, '9258.073')] [2023-12-27 04:39:51,100][105620] Updated weights for policy 1, policy_version 1825331 (0.0008) [2023-12-27 04:39:51,166][105620] Updated weights for policy 1, policy_version 1825341 (0.0008) [2023-12-27 04:39:51,221][105620] Updated weights for policy 1, policy_version 1825351 (0.0007) [2023-12-27 04:39:51,635][105692] Updated weights for policy 0, policy_version 1821459 (0.0012) [2023-12-27 04:39:51,699][105692] Updated weights for policy 0, policy_version 1821469 (0.0008) [2023-12-27 04:39:51,767][105692] Updated weights for policy 0, policy_version 1821479 (0.0008) [2023-12-27 04:39:52,028][105620] Updated weights for policy 1, policy_version 1825361 (0.0010) [2023-12-27 04:39:52,082][105620] Updated weights for policy 1, policy_version 1825372 (0.0010) [2023-12-27 04:39:52,134][105620] Updated weights for policy 1, policy_version 1825382 (0.0009) [2023-12-27 04:39:52,375][105692] Updated weights for policy 0, policy_version 1821489 (0.0008) [2023-12-27 04:39:52,439][105692] Updated weights for policy 0, policy_version 1821499 (0.0008) [2023-12-27 04:39:52,505][105692] Updated weights for policy 0, policy_version 1821509 (0.0008) [2023-12-27 04:39:52,572][105692] Updated weights for policy 0, policy_version 1821519 (0.0008) [2023-12-27 04:39:52,870][105620] Updated weights for policy 1, policy_version 1825392 (0.0010) [2023-12-27 04:39:52,920][105620] Updated weights for policy 1, policy_version 1825402 (0.0006) [2023-12-27 04:39:52,972][105620] Updated weights for policy 1, policy_version 1825412 (0.0006) [2023-12-27 04:39:53,326][105692] Updated weights for policy 0, policy_version 1821529 (0.0010) [2023-12-27 04:39:53,373][105692] Updated weights for policy 0, policy_version 1821539 (0.0010) [2023-12-27 04:39:53,421][105692] Updated weights for policy 0, policy_version 1821549 (0.0010) [2023-12-27 04:39:53,673][105620] Updated weights for policy 1, policy_version 1825422 (0.0008) [2023-12-27 04:39:53,730][105620] Updated weights for policy 1, policy_version 1825432 (0.0005) [2023-12-27 04:39:53,790][105620] Updated weights for policy 1, policy_version 1825442 (0.0005) [2023-12-27 04:39:54,055][105692] Updated weights for policy 0, policy_version 1821559 (0.0010) [2023-12-27 04:39:54,104][105692] Updated weights for policy 0, policy_version 1821569 (0.0010) [2023-12-27 04:39:54,162][105692] Updated weights for policy 0, policy_version 1821579 (0.0010) [2023-12-27 04:39:54,326][105620] Updated weights for policy 1, policy_version 1825452 (0.0005) [2023-12-27 04:39:54,381][105620] Updated weights for policy 1, policy_version 1825462 (0.0005) [2023-12-27 04:39:54,425][105620] Updated weights for policy 1, policy_version 1825472 (0.0005) [2023-12-27 04:39:54,845][105692] Updated weights for policy 0, policy_version 1821589 (0.0008) [2023-12-27 04:39:54,896][105692] Updated weights for policy 0, policy_version 1821599 (0.0010) [2023-12-27 04:39:54,947][105692] Updated weights for policy 0, policy_version 1821609 (0.0010) [2023-12-27 04:39:55,026][105620] Updated weights for policy 1, policy_version 1825482 (0.0006) [2023-12-27 04:39:55,077][105620] Updated weights for policy 1, policy_version 1825492 (0.0008) [2023-12-27 04:39:55,124][105620] Updated weights for policy 1, policy_version 1825502 (0.0005) [2023-12-27 04:39:55,173][105620] Updated weights for policy 1, policy_version 1825512 (0.0005) [2023-12-27 04:39:55,669][105692] Updated weights for policy 0, policy_version 1821619 (0.0008) [2023-12-27 04:39:55,724][105692] Updated weights for policy 0, policy_version 1821629 (0.0010) [2023-12-27 04:39:55,772][105692] Updated weights for policy 0, policy_version 1821639 (0.0010) [2023-12-27 04:39:55,818][105620] Updated weights for policy 1, policy_version 1825522 (0.0005) [2023-12-27 04:39:55,866][105620] Updated weights for policy 1, policy_version 1825532 (0.0008) [2023-12-27 04:39:55,912][105620] Updated weights for policy 1, policy_version 1825542 (0.0008) [2023-12-27 04:39:56,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 19605.3). Total num frames: 933814272. Throughput: 0: 9690.2, 1: 10003.3. Samples: 933818180. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:39:56,062][104569] Avg episode reward: [(0, '8629.642'), (1, '9074.829')] [2023-12-27 04:39:56,522][105692] Updated weights for policy 0, policy_version 1821649 (0.0010) [2023-12-27 04:39:56,579][105692] Updated weights for policy 0, policy_version 1821659 (0.0010) [2023-12-27 04:39:56,644][105692] Updated weights for policy 0, policy_version 1821669 (0.0010) [2023-12-27 04:39:56,669][105620] Updated weights for policy 1, policy_version 1825552 (0.0007) [2023-12-27 04:39:56,698][105692] Updated weights for policy 0, policy_version 1821679 (0.0010) [2023-12-27 04:39:56,716][105620] Updated weights for policy 1, policy_version 1825562 (0.0007) [2023-12-27 04:39:56,762][105620] Updated weights for policy 1, policy_version 1825572 (0.0008) [2023-12-27 04:39:57,427][105692] Updated weights for policy 0, policy_version 1821689 (0.0010) [2023-12-27 04:39:57,488][105692] Updated weights for policy 0, policy_version 1821699 (0.0010) [2023-12-27 04:39:57,528][105620] Updated weights for policy 1, policy_version 1825582 (0.0007) [2023-12-27 04:39:57,545][105692] Updated weights for policy 0, policy_version 1821709 (0.0010) [2023-12-27 04:39:57,588][105620] Updated weights for policy 1, policy_version 1825592 (0.0007) [2023-12-27 04:39:57,643][105620] Updated weights for policy 1, policy_version 1825602 (0.0008) [2023-12-27 04:39:58,278][105692] Updated weights for policy 0, policy_version 1821719 (0.0010) [2023-12-27 04:39:58,342][105692] Updated weights for policy 0, policy_version 1821729 (0.0010) [2023-12-27 04:39:58,405][105692] Updated weights for policy 0, policy_version 1821739 (0.0008) [2023-12-27 04:39:58,433][105620] Updated weights for policy 1, policy_version 1825612 (0.0008) [2023-12-27 04:39:58,490][105620] Updated weights for policy 1, policy_version 1825622 (0.0008) [2023-12-27 04:39:58,555][105620] Updated weights for policy 1, policy_version 1825632 (0.0008) [2023-12-27 04:39:59,140][105692] Updated weights for policy 0, policy_version 1821749 (0.0008) [2023-12-27 04:39:59,190][105692] Updated weights for policy 0, policy_version 1821759 (0.0010) [2023-12-27 04:39:59,245][105692] Updated weights for policy 0, policy_version 1821769 (0.0012) [2023-12-27 04:39:59,329][105620] Updated weights for policy 1, policy_version 1825642 (0.0009) [2023-12-27 04:39:59,393][105620] Updated weights for policy 1, policy_version 1825652 (0.0007) [2023-12-27 04:39:59,450][105620] Updated weights for policy 1, policy_version 1825662 (0.0007) [2023-12-27 04:39:59,498][105620] Updated weights for policy 1, policy_version 1825672 (0.0009) [2023-12-27 04:40:00,039][105692] Updated weights for policy 0, policy_version 1821779 (0.0009) [2023-12-27 04:40:00,095][105692] Updated weights for policy 0, policy_version 1821789 (0.0006) [2023-12-27 04:40:00,155][105692] Updated weights for policy 0, policy_version 1821799 (0.0007) [2023-12-27 04:40:00,202][105620] Updated weights for policy 1, policy_version 1825682 (0.0011) [2023-12-27 04:40:00,257][105620] Updated weights for policy 1, policy_version 1825692 (0.0010) [2023-12-27 04:40:00,316][105620] Updated weights for policy 1, policy_version 1825702 (0.0010) [2023-12-27 04:40:00,930][105620] Updated weights for policy 1, policy_version 1825712 (0.0010) [2023-12-27 04:40:00,962][105692] Updated weights for policy 0, policy_version 1821809 (0.0006) [2023-12-27 04:40:00,989][105620] Updated weights for policy 1, policy_version 1825722 (0.0010) [2023-12-27 04:40:01,014][105692] Updated weights for policy 0, policy_version 1821819 (0.0006) [2023-12-27 04:40:01,052][105620] Updated weights for policy 1, policy_version 1825732 (0.0009) [2023-12-27 04:40:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 933896192. Throughput: 0: 9715.2, 1: 9965.8. Samples: 933874084. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:40:01,062][104569] Avg episode reward: [(0, '8620.266'), (1, '9167.065')] [2023-12-27 04:40:01,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001825736_467451904.pth... [2023-12-27 04:40:01,075][105692] Updated weights for policy 0, policy_version 1821829 (0.0007) [2023-12-27 04:40:01,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001824584_467156992.pth [2023-12-27 04:40:01,135][105692] Updated weights for policy 0, policy_version 1821839 (0.0008) [2023-12-27 04:40:01,138][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001821840_466460672.pth... [2023-12-27 04:40:01,142][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001820688_466165760.pth [2023-12-27 04:40:01,716][105620] Updated weights for policy 1, policy_version 1825742 (0.0008) [2023-12-27 04:40:01,781][105620] Updated weights for policy 1, policy_version 1825752 (0.0008) [2023-12-27 04:40:01,797][105692] Updated weights for policy 0, policy_version 1821849 (0.0007) [2023-12-27 04:40:01,844][105692] Updated weights for policy 0, policy_version 1821859 (0.0006) [2023-12-27 04:40:01,845][105620] Updated weights for policy 1, policy_version 1825762 (0.0008) [2023-12-27 04:40:01,891][105692] Updated weights for policy 0, policy_version 1821869 (0.0007) [2023-12-27 04:40:02,577][105692] Updated weights for policy 0, policy_version 1821879 (0.0008) [2023-12-27 04:40:02,606][105620] Updated weights for policy 1, policy_version 1825772 (0.0007) [2023-12-27 04:40:02,625][105692] Updated weights for policy 0, policy_version 1821889 (0.0008) [2023-12-27 04:40:02,661][105620] Updated weights for policy 1, policy_version 1825782 (0.0007) [2023-12-27 04:40:02,672][105692] Updated weights for policy 0, policy_version 1821899 (0.0007) [2023-12-27 04:40:02,720][105620] Updated weights for policy 1, policy_version 1825792 (0.0008) [2023-12-27 04:40:03,362][105620] Updated weights for policy 1, policy_version 1825802 (0.0008) [2023-12-27 04:40:03,418][105620] Updated weights for policy 1, policy_version 1825812 (0.0008) [2023-12-27 04:40:03,465][105620] Updated weights for policy 1, policy_version 1825822 (0.0006) [2023-12-27 04:40:03,507][105692] Updated weights for policy 0, policy_version 1821909 (0.0009) [2023-12-27 04:40:03,511][105620] Updated weights for policy 1, policy_version 1825832 (0.0005) [2023-12-27 04:40:03,562][105692] Updated weights for policy 0, policy_version 1821919 (0.0010) [2023-12-27 04:40:03,626][105692] Updated weights for policy 0, policy_version 1821929 (0.0007) [2023-12-27 04:40:04,140][105620] Updated weights for policy 1, policy_version 1825842 (0.0009) [2023-12-27 04:40:04,205][105620] Updated weights for policy 1, policy_version 1825852 (0.0008) [2023-12-27 04:40:04,264][105620] Updated weights for policy 1, policy_version 1825862 (0.0009) [2023-12-27 04:40:04,443][105692] Updated weights for policy 0, policy_version 1821939 (0.0008) [2023-12-27 04:40:04,508][105692] Updated weights for policy 0, policy_version 1821949 (0.0009) [2023-12-27 04:40:04,580][105692] Updated weights for policy 0, policy_version 1821960 (0.0010) [2023-12-27 04:40:04,973][105620] Updated weights for policy 1, policy_version 1825872 (0.0008) [2023-12-27 04:40:05,025][105620] Updated weights for policy 1, policy_version 1825882 (0.0009) [2023-12-27 04:40:05,079][105620] Updated weights for policy 1, policy_version 1825892 (0.0008) [2023-12-27 04:40:05,250][105692] Updated weights for policy 0, policy_version 1821970 (0.0008) [2023-12-27 04:40:05,313][105692] Updated weights for policy 0, policy_version 1821980 (0.0008) [2023-12-27 04:40:05,368][105692] Updated weights for policy 0, policy_version 1821990 (0.0009) [2023-12-27 04:40:05,421][105692] Updated weights for policy 0, policy_version 1822000 (0.0008) [2023-12-27 04:40:05,894][105620] Updated weights for policy 1, policy_version 1825902 (0.0009) [2023-12-27 04:40:05,953][105620] Updated weights for policy 1, policy_version 1825912 (0.0009) [2023-12-27 04:40:06,011][105620] Updated weights for policy 1, policy_version 1825922 (0.0009) [2023-12-27 04:40:06,061][105692] Updated weights for policy 0, policy_version 1822010 (0.0007) [2023-12-27 04:40:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 934002688. Throughput: 0: 9754.0, 1: 9958.3. Samples: 933990784. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:40:06,062][104569] Avg episode reward: [(0, '8530.017'), (1, '9350.311')] [2023-12-27 04:40:06,144][105692] Updated weights for policy 0, policy_version 1822020 (0.0009) [2023-12-27 04:40:06,206][105692] Updated weights for policy 0, policy_version 1822030 (0.0007) [2023-12-27 04:40:06,810][105620] Updated weights for policy 1, policy_version 1825932 (0.0008) [2023-12-27 04:40:06,857][105620] Updated weights for policy 1, policy_version 1825942 (0.0009) [2023-12-27 04:40:06,907][105620] Updated weights for policy 1, policy_version 1825952 (0.0007) [2023-12-27 04:40:06,915][105692] Updated weights for policy 0, policy_version 1822040 (0.0008) [2023-12-27 04:40:06,977][105692] Updated weights for policy 0, policy_version 1822050 (0.0010) [2023-12-27 04:40:07,031][105692] Updated weights for policy 0, policy_version 1822060 (0.0009) [2023-12-27 04:40:07,659][105620] Updated weights for policy 1, policy_version 1825962 (0.0009) [2023-12-27 04:40:07,719][105620] Updated weights for policy 1, policy_version 1825972 (0.0005) [2023-12-27 04:40:07,753][105692] Updated weights for policy 0, policy_version 1822070 (0.0007) [2023-12-27 04:40:07,770][105620] Updated weights for policy 1, policy_version 1825982 (0.0006) [2023-12-27 04:40:07,816][105692] Updated weights for policy 0, policy_version 1822080 (0.0005) [2023-12-27 04:40:07,819][105620] Updated weights for policy 1, policy_version 1825992 (0.0008) [2023-12-27 04:40:07,877][105692] Updated weights for policy 0, policy_version 1822090 (0.0005) [2023-12-27 04:40:08,515][105692] Updated weights for policy 0, policy_version 1822100 (0.0006) [2023-12-27 04:40:08,517][105620] Updated weights for policy 1, policy_version 1826002 (0.0010) [2023-12-27 04:40:08,571][105692] Updated weights for policy 0, policy_version 1822110 (0.0005) [2023-12-27 04:40:08,573][105620] Updated weights for policy 1, policy_version 1826012 (0.0010) [2023-12-27 04:40:08,623][105692] Updated weights for policy 0, policy_version 1822120 (0.0006) [2023-12-27 04:40:08,635][105620] Updated weights for policy 1, policy_version 1826022 (0.0010) [2023-12-27 04:40:09,220][105620] Updated weights for policy 1, policy_version 1826032 (0.0007) [2023-12-27 04:40:09,283][105620] Updated weights for policy 1, policy_version 1826042 (0.0007) [2023-12-27 04:40:09,337][105620] Updated weights for policy 1, policy_version 1826052 (0.0007) [2023-12-27 04:40:09,499][105692] Updated weights for policy 0, policy_version 1822130 (0.0008) [2023-12-27 04:40:09,560][105692] Updated weights for policy 0, policy_version 1822140 (0.0007) [2023-12-27 04:40:09,609][105692] Updated weights for policy 0, policy_version 1822150 (0.0008) [2023-12-27 04:40:09,660][105692] Updated weights for policy 0, policy_version 1822160 (0.0008) [2023-12-27 04:40:10,058][105620] Updated weights for policy 1, policy_version 1826062 (0.0010) [2023-12-27 04:40:10,111][105620] Updated weights for policy 1, policy_version 1826072 (0.0010) [2023-12-27 04:40:10,163][105620] Updated weights for policy 1, policy_version 1826082 (0.0010) [2023-12-27 04:40:10,386][105692] Updated weights for policy 0, policy_version 1822170 (0.0008) [2023-12-27 04:40:10,442][105692] Updated weights for policy 0, policy_version 1822180 (0.0009) [2023-12-27 04:40:10,505][105692] Updated weights for policy 0, policy_version 1822190 (0.0009) [2023-12-27 04:40:10,807][105620] Updated weights for policy 1, policy_version 1826092 (0.0009) [2023-12-27 04:40:10,872][105620] Updated weights for policy 1, policy_version 1826102 (0.0009) [2023-12-27 04:40:10,939][105620] Updated weights for policy 1, policy_version 1826112 (0.0008) [2023-12-27 04:40:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 934100992. Throughput: 0: 9764.9, 1: 9942.3. Samples: 934107040. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:40:11,063][104569] Avg episode reward: [(0, '8719.102'), (1, '9258.011')] [2023-12-27 04:40:11,316][105692] Updated weights for policy 0, policy_version 1822200 (0.0010) [2023-12-27 04:40:11,387][105692] Updated weights for policy 0, policy_version 1822210 (0.0008) [2023-12-27 04:40:11,449][105692] Updated weights for policy 0, policy_version 1822220 (0.0006) [2023-12-27 04:40:11,685][105620] Updated weights for policy 1, policy_version 1826122 (0.0009) [2023-12-27 04:40:11,753][105620] Updated weights for policy 1, policy_version 1826132 (0.0010) [2023-12-27 04:40:11,817][105620] Updated weights for policy 1, policy_version 1826142 (0.0011) [2023-12-27 04:40:11,870][105620] Updated weights for policy 1, policy_version 1826152 (0.0011) [2023-12-27 04:40:12,218][105692] Updated weights for policy 0, policy_version 1822230 (0.0007) [2023-12-27 04:40:12,270][105692] Updated weights for policy 0, policy_version 1822240 (0.0008) [2023-12-27 04:40:12,331][105692] Updated weights for policy 0, policy_version 1822250 (0.0008) [2023-12-27 04:40:12,642][105620] Updated weights for policy 1, policy_version 1826162 (0.0011) [2023-12-27 04:40:12,701][105620] Updated weights for policy 1, policy_version 1826172 (0.0010) [2023-12-27 04:40:12,758][105620] Updated weights for policy 1, policy_version 1826182 (0.0009) [2023-12-27 04:40:13,064][105692] Updated weights for policy 0, policy_version 1822260 (0.0008) [2023-12-27 04:40:13,118][105692] Updated weights for policy 0, policy_version 1822270 (0.0005) [2023-12-27 04:40:13,169][105692] Updated weights for policy 0, policy_version 1822280 (0.0005) [2023-12-27 04:40:13,478][105620] Updated weights for policy 1, policy_version 1826192 (0.0010) [2023-12-27 04:40:13,530][105620] Updated weights for policy 1, policy_version 1826202 (0.0011) [2023-12-27 04:40:13,581][105620] Updated weights for policy 1, policy_version 1826212 (0.0010) [2023-12-27 04:40:13,889][105692] Updated weights for policy 0, policy_version 1822290 (0.0008) [2023-12-27 04:40:13,955][105692] Updated weights for policy 0, policy_version 1822300 (0.0008) [2023-12-27 04:40:14,021][105692] Updated weights for policy 0, policy_version 1822310 (0.0006) [2023-12-27 04:40:14,083][105692] Updated weights for policy 0, policy_version 1822320 (0.0008) [2023-12-27 04:40:14,357][105620] Updated weights for policy 1, policy_version 1826222 (0.0011) [2023-12-27 04:40:14,422][105620] Updated weights for policy 1, policy_version 1826232 (0.0010) [2023-12-27 04:40:14,480][105620] Updated weights for policy 1, policy_version 1826242 (0.0010) [2023-12-27 04:40:14,747][105692] Updated weights for policy 0, policy_version 1822330 (0.0005) [2023-12-27 04:40:14,815][105692] Updated weights for policy 0, policy_version 1822340 (0.0008) [2023-12-27 04:40:14,873][105692] Updated weights for policy 0, policy_version 1822350 (0.0008) [2023-12-27 04:40:15,195][105620] Updated weights for policy 1, policy_version 1826252 (0.0010) [2023-12-27 04:40:15,262][105620] Updated weights for policy 1, policy_version 1826262 (0.0009) [2023-12-27 04:40:15,321][105620] Updated weights for policy 1, policy_version 1826272 (0.0009) [2023-12-27 04:40:15,591][105692] Updated weights for policy 0, policy_version 1822360 (0.0009) [2023-12-27 04:40:15,656][105692] Updated weights for policy 0, policy_version 1822370 (0.0006) [2023-12-27 04:40:15,712][105692] Updated weights for policy 0, policy_version 1822380 (0.0009) [2023-12-27 04:40:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 934191104. Throughput: 0: 9682.6, 1: 9852.4. Samples: 934162956. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:40:16,062][104569] Avg episode reward: [(0, '8901.364'), (1, '9258.066')] [2023-12-27 04:40:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001822384_466599936.pth... [2023-12-27 04:40:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001821264_466313216.pth [2023-12-27 04:40:16,079][105620] Updated weights for policy 1, policy_version 1826282 (0.0009) [2023-12-27 04:40:16,142][105620] Updated weights for policy 1, policy_version 1826292 (0.0009) [2023-12-27 04:40:16,209][105620] Updated weights for policy 1, policy_version 1826302 (0.0010) [2023-12-27 04:40:16,275][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001826312_467599360.pth... [2023-12-27 04:40:16,277][105620] Updated weights for policy 1, policy_version 1826312 (0.0010) [2023-12-27 04:40:16,279][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001825160_467304448.pth [2023-12-27 04:40:16,421][105692] Updated weights for policy 0, policy_version 1822390 (0.0009) [2023-12-27 04:40:16,480][105692] Updated weights for policy 0, policy_version 1822400 (0.0009) [2023-12-27 04:40:16,540][105692] Updated weights for policy 0, policy_version 1822410 (0.0009) [2023-12-27 04:40:16,959][105620] Updated weights for policy 1, policy_version 1826322 (0.0007) [2023-12-27 04:40:17,009][105620] Updated weights for policy 1, policy_version 1826332 (0.0008) [2023-12-27 04:40:17,059][105620] Updated weights for policy 1, policy_version 1826342 (0.0009) [2023-12-27 04:40:17,337][105692] Updated weights for policy 0, policy_version 1822420 (0.0008) [2023-12-27 04:40:17,396][105692] Updated weights for policy 0, policy_version 1822430 (0.0009) [2023-12-27 04:40:17,442][105692] Updated weights for policy 0, policy_version 1822440 (0.0008) [2023-12-27 04:40:17,717][105620] Updated weights for policy 1, policy_version 1826352 (0.0006) [2023-12-27 04:40:17,763][105620] Updated weights for policy 1, policy_version 1826362 (0.0008) [2023-12-27 04:40:17,815][105620] Updated weights for policy 1, policy_version 1826372 (0.0009) [2023-12-27 04:40:18,199][105692] Updated weights for policy 0, policy_version 1822450 (0.0009) [2023-12-27 04:40:18,257][105692] Updated weights for policy 0, policy_version 1822460 (0.0009) [2023-12-27 04:40:18,333][105692] Updated weights for policy 0, policy_version 1822470 (0.0009) [2023-12-27 04:40:18,400][105692] Updated weights for policy 0, policy_version 1822480 (0.0009) [2023-12-27 04:40:18,609][105620] Updated weights for policy 1, policy_version 1826382 (0.0009) [2023-12-27 04:40:18,668][105620] Updated weights for policy 1, policy_version 1826392 (0.0009) [2023-12-27 04:40:18,722][105620] Updated weights for policy 1, policy_version 1826402 (0.0010) [2023-12-27 04:40:19,080][105692] Updated weights for policy 0, policy_version 1822490 (0.0010) [2023-12-27 04:40:19,133][105692] Updated weights for policy 0, policy_version 1822500 (0.0010) [2023-12-27 04:40:19,201][105692] Updated weights for policy 0, policy_version 1822510 (0.0011) [2023-12-27 04:40:19,537][105620] Updated weights for policy 1, policy_version 1826412 (0.0008) [2023-12-27 04:40:19,590][105620] Updated weights for policy 1, policy_version 1826422 (0.0006) [2023-12-27 04:40:19,652][105620] Updated weights for policy 1, policy_version 1826432 (0.0007) [2023-12-27 04:40:19,974][105692] Updated weights for policy 0, policy_version 1822520 (0.0010) [2023-12-27 04:40:20,032][105692] Updated weights for policy 0, policy_version 1822530 (0.0010) [2023-12-27 04:40:20,100][105692] Updated weights for policy 0, policy_version 1822540 (0.0009) [2023-12-27 04:40:20,347][105620] Updated weights for policy 1, policy_version 1826442 (0.0009) [2023-12-27 04:40:20,406][105620] Updated weights for policy 1, policy_version 1826452 (0.0010) [2023-12-27 04:40:20,459][105620] Updated weights for policy 1, policy_version 1826462 (0.0009) [2023-12-27 04:40:20,506][105620] Updated weights for policy 1, policy_version 1826472 (0.0008) [2023-12-27 04:40:20,860][105692] Updated weights for policy 0, policy_version 1822550 (0.0009) [2023-12-27 04:40:20,916][105692] Updated weights for policy 0, policy_version 1822560 (0.0009) [2023-12-27 04:40:20,977][105692] Updated weights for policy 0, policy_version 1822570 (0.0009) [2023-12-27 04:40:21,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 934289408. Throughput: 0: 9603.8, 1: 9770.7. Samples: 934277228. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:40:21,063][104569] Avg episode reward: [(0, '8807.184'), (1, '9258.088')] [2023-12-27 04:40:21,353][105620] Updated weights for policy 1, policy_version 1826482 (0.0009) [2023-12-27 04:40:21,417][105620] Updated weights for policy 1, policy_version 1826492 (0.0008) [2023-12-27 04:40:21,482][105620] Updated weights for policy 1, policy_version 1826502 (0.0008) [2023-12-27 04:40:21,853][105692] Updated weights for policy 0, policy_version 1822580 (0.0010) [2023-12-27 04:40:21,918][105692] Updated weights for policy 0, policy_version 1822590 (0.0011) [2023-12-27 04:40:21,979][105692] Updated weights for policy 0, policy_version 1822600 (0.0011) [2023-12-27 04:40:22,183][105620] Updated weights for policy 1, policy_version 1826512 (0.0006) [2023-12-27 04:40:22,254][105620] Updated weights for policy 1, policy_version 1826522 (0.0007) [2023-12-27 04:40:22,320][105620] Updated weights for policy 1, policy_version 1826532 (0.0008) [2023-12-27 04:40:22,804][105692] Updated weights for policy 0, policy_version 1822610 (0.0010) [2023-12-27 04:40:22,870][105692] Updated weights for policy 0, policy_version 1822620 (0.0008) [2023-12-27 04:40:22,931][105692] Updated weights for policy 0, policy_version 1822630 (0.0008) [2023-12-27 04:40:22,990][105692] Updated weights for policy 0, policy_version 1822640 (0.0008) [2023-12-27 04:40:23,080][105620] Updated weights for policy 1, policy_version 1826542 (0.0010) [2023-12-27 04:40:23,139][105620] Updated weights for policy 1, policy_version 1826552 (0.0005) [2023-12-27 04:40:23,205][105620] Updated weights for policy 1, policy_version 1826562 (0.0007) [2023-12-27 04:40:23,741][105692] Updated weights for policy 0, policy_version 1822650 (0.0005) [2023-12-27 04:40:23,809][105692] Updated weights for policy 0, policy_version 1822660 (0.0005) [2023-12-27 04:40:23,878][105692] Updated weights for policy 0, policy_version 1822670 (0.0005) [2023-12-27 04:40:23,898][105620] Updated weights for policy 1, policy_version 1826572 (0.0008) [2023-12-27 04:40:23,964][105620] Updated weights for policy 1, policy_version 1826582 (0.0007) [2023-12-27 04:40:24,025][105620] Updated weights for policy 1, policy_version 1826592 (0.0005) [2023-12-27 04:40:24,384][105692] Updated weights for policy 0, policy_version 1822680 (0.0006) [2023-12-27 04:40:24,439][105692] Updated weights for policy 0, policy_version 1822690 (0.0005) [2023-12-27 04:40:24,500][105692] Updated weights for policy 0, policy_version 1822700 (0.0006) [2023-12-27 04:40:24,715][105620] Updated weights for policy 1, policy_version 1826602 (0.0006) [2023-12-27 04:40:24,776][105620] Updated weights for policy 1, policy_version 1826612 (0.0010) [2023-12-27 04:40:24,830][105620] Updated weights for policy 1, policy_version 1826622 (0.0010) [2023-12-27 04:40:24,881][105620] Updated weights for policy 1, policy_version 1826632 (0.0010) [2023-12-27 04:40:25,116][105692] Updated weights for policy 0, policy_version 1822710 (0.0008) [2023-12-27 04:40:25,173][105692] Updated weights for policy 0, policy_version 1822720 (0.0010) [2023-12-27 04:40:25,228][105692] Updated weights for policy 0, policy_version 1822730 (0.0005) [2023-12-27 04:40:25,590][105620] Updated weights for policy 1, policy_version 1826642 (0.0005) [2023-12-27 04:40:25,640][105620] Updated weights for policy 1, policy_version 1826652 (0.0005) [2023-12-27 04:40:25,698][105620] Updated weights for policy 1, policy_version 1826662 (0.0005) [2023-12-27 04:40:25,929][105692] Updated weights for policy 0, policy_version 1822740 (0.0008) [2023-12-27 04:40:25,978][105692] Updated weights for policy 0, policy_version 1822750 (0.0009) [2023-12-27 04:40:26,036][105692] Updated weights for policy 0, policy_version 1822760 (0.0005) [2023-12-27 04:40:26,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 934379520. Throughput: 0: 9649.2, 1: 9673.8. Samples: 934392348. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:40:26,063][104569] Avg episode reward: [(0, '8808.075'), (1, '9073.347')] [2023-12-27 04:40:26,385][105620] Updated weights for policy 1, policy_version 1826672 (0.0010) [2023-12-27 04:40:26,433][105620] Updated weights for policy 1, policy_version 1826682 (0.0010) [2023-12-27 04:40:26,488][105620] Updated weights for policy 1, policy_version 1826692 (0.0010) [2023-12-27 04:40:26,714][105692] Updated weights for policy 0, policy_version 1822770 (0.0006) [2023-12-27 04:40:26,764][105692] Updated weights for policy 0, policy_version 1822780 (0.0005) [2023-12-27 04:40:26,810][105692] Updated weights for policy 0, policy_version 1822790 (0.0006) [2023-12-27 04:40:26,859][105692] Updated weights for policy 0, policy_version 1822800 (0.0005) [2023-12-27 04:40:27,203][105620] Updated weights for policy 1, policy_version 1826702 (0.0010) [2023-12-27 04:40:27,265][105620] Updated weights for policy 1, policy_version 1826712 (0.0011) [2023-12-27 04:40:27,324][105620] Updated weights for policy 1, policy_version 1826722 (0.0010) [2023-12-27 04:40:27,501][105692] Updated weights for policy 0, policy_version 1822810 (0.0010) [2023-12-27 04:40:27,560][105692] Updated weights for policy 0, policy_version 1822820 (0.0011) [2023-12-27 04:40:27,622][105692] Updated weights for policy 0, policy_version 1822830 (0.0005) [2023-12-27 04:40:28,053][105620] Updated weights for policy 1, policy_version 1826732 (0.0008) [2023-12-27 04:40:28,114][105620] Updated weights for policy 1, policy_version 1826742 (0.0005) [2023-12-27 04:40:28,185][105692] Updated weights for policy 0, policy_version 1822840 (0.0010) [2023-12-27 04:40:28,197][105620] Updated weights for policy 1, policy_version 1826752 (0.0010) [2023-12-27 04:40:28,232][105692] Updated weights for policy 0, policy_version 1822850 (0.0010) [2023-12-27 04:40:28,286][105692] Updated weights for policy 0, policy_version 1822860 (0.0010) [2023-12-27 04:40:28,873][105620] Updated weights for policy 1, policy_version 1826762 (0.0010) [2023-12-27 04:40:28,942][105620] Updated weights for policy 1, policy_version 1826772 (0.0010) [2023-12-27 04:40:28,977][105692] Updated weights for policy 0, policy_version 1822870 (0.0006) [2023-12-27 04:40:29,004][105620] Updated weights for policy 1, policy_version 1826782 (0.0010) [2023-12-27 04:40:29,031][105692] Updated weights for policy 0, policy_version 1822880 (0.0006) [2023-12-27 04:40:29,053][105620] Updated weights for policy 1, policy_version 1826792 (0.0010) [2023-12-27 04:40:29,085][105692] Updated weights for policy 0, policy_version 1822890 (0.0008) [2023-12-27 04:40:29,649][105620] Updated weights for policy 1, policy_version 1826802 (0.0005) [2023-12-27 04:40:29,713][105620] Updated weights for policy 1, policy_version 1826812 (0.0006) [2023-12-27 04:40:29,770][105620] Updated weights for policy 1, policy_version 1826822 (0.0005) [2023-12-27 04:40:29,908][105692] Updated weights for policy 0, policy_version 1822900 (0.0010) [2023-12-27 04:40:29,974][105692] Updated weights for policy 0, policy_version 1822910 (0.0008) [2023-12-27 04:40:30,037][105692] Updated weights for policy 0, policy_version 1822920 (0.0010) [2023-12-27 04:40:30,334][105620] Updated weights for policy 1, policy_version 1826832 (0.0009) [2023-12-27 04:40:30,381][105620] Updated weights for policy 1, policy_version 1826842 (0.0010) [2023-12-27 04:40:30,442][105620] Updated weights for policy 1, policy_version 1826852 (0.0010) [2023-12-27 04:40:30,823][105692] Updated weights for policy 0, policy_version 1822930 (0.0009) [2023-12-27 04:40:30,875][105692] Updated weights for policy 0, policy_version 1822940 (0.0005) [2023-12-27 04:40:30,929][105692] Updated weights for policy 0, policy_version 1822950 (0.0005) [2023-12-27 04:40:30,981][105692] Updated weights for policy 0, policy_version 1822960 (0.0005) [2023-12-27 04:40:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19605.2). Total num frames: 934486016. Throughput: 0: 9734.5, 1: 9702.3. Samples: 934453576. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:40:31,063][104569] Avg episode reward: [(0, '8901.928'), (1, '9073.424')] [2023-12-27 04:40:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001822960_466747392.pth... [2023-12-27 04:40:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001826856_467738624.pth... [2023-12-27 04:40:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001821840_466460672.pth [2023-12-27 04:40:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001825736_467451904.pth [2023-12-27 04:40:31,111][105620] Updated weights for policy 1, policy_version 1826862 (0.0008) [2023-12-27 04:40:31,172][105620] Updated weights for policy 1, policy_version 1826872 (0.0009) [2023-12-27 04:40:31,236][105620] Updated weights for policy 1, policy_version 1826883 (0.0009) [2023-12-27 04:40:31,654][105692] Updated weights for policy 0, policy_version 1822970 (0.0008) [2023-12-27 04:40:31,712][105692] Updated weights for policy 0, policy_version 1822980 (0.0007) [2023-12-27 04:40:31,780][105692] Updated weights for policy 0, policy_version 1822990 (0.0007) [2023-12-27 04:40:31,984][105620] Updated weights for policy 1, policy_version 1826893 (0.0009) [2023-12-27 04:40:32,029][105620] Updated weights for policy 1, policy_version 1826903 (0.0008) [2023-12-27 04:40:32,084][105620] Updated weights for policy 1, policy_version 1826913 (0.0007) [2023-12-27 04:40:32,409][105692] Updated weights for policy 0, policy_version 1823000 (0.0008) [2023-12-27 04:40:32,460][105692] Updated weights for policy 0, policy_version 1823010 (0.0009) [2023-12-27 04:40:32,515][105692] Updated weights for policy 0, policy_version 1823020 (0.0008) [2023-12-27 04:40:32,791][105620] Updated weights for policy 1, policy_version 1826923 (0.0009) [2023-12-27 04:40:32,851][105620] Updated weights for policy 1, policy_version 1826933 (0.0009) [2023-12-27 04:40:32,909][105620] Updated weights for policy 1, policy_version 1826943 (0.0009) [2023-12-27 04:40:33,196][105692] Updated weights for policy 0, policy_version 1823030 (0.0009) [2023-12-27 04:40:33,246][105692] Updated weights for policy 0, policy_version 1823040 (0.0009) [2023-12-27 04:40:33,297][105692] Updated weights for policy 0, policy_version 1823051 (0.0009) [2023-12-27 04:40:33,540][105620] Updated weights for policy 1, policy_version 1826953 (0.0008) [2023-12-27 04:40:33,590][105620] Updated weights for policy 1, policy_version 1826963 (0.0007) [2023-12-27 04:40:33,639][105620] Updated weights for policy 1, policy_version 1826973 (0.0007) [2023-12-27 04:40:33,687][105620] Updated weights for policy 1, policy_version 1826983 (0.0010) [2023-12-27 04:40:33,888][105692] Updated weights for policy 0, policy_version 1823061 (0.0008) [2023-12-27 04:40:33,935][105692] Updated weights for policy 0, policy_version 1823071 (0.0005) [2023-12-27 04:40:33,984][105692] Updated weights for policy 0, policy_version 1823081 (0.0009) [2023-12-27 04:40:34,419][105620] Updated weights for policy 1, policy_version 1826993 (0.0006) [2023-12-27 04:40:34,486][105620] Updated weights for policy 1, policy_version 1827003 (0.0006) [2023-12-27 04:40:34,549][105620] Updated weights for policy 1, policy_version 1827013 (0.0006) [2023-12-27 04:40:34,668][105692] Updated weights for policy 0, policy_version 1823091 (0.0009) [2023-12-27 04:40:34,730][105692] Updated weights for policy 0, policy_version 1823101 (0.0006) [2023-12-27 04:40:34,788][105692] Updated weights for policy 0, policy_version 1823111 (0.0006) [2023-12-27 04:40:35,098][105620] Updated weights for policy 1, policy_version 1827023 (0.0006) [2023-12-27 04:40:35,153][105620] Updated weights for policy 1, policy_version 1827033 (0.0006) [2023-12-27 04:40:35,199][105620] Updated weights for policy 1, policy_version 1827043 (0.0008) [2023-12-27 04:40:35,310][105692] Updated weights for policy 0, policy_version 1823121 (0.0005) [2023-12-27 04:40:35,374][105692] Updated weights for policy 0, policy_version 1823131 (0.0005) [2023-12-27 04:40:35,425][105692] Updated weights for policy 0, policy_version 1823141 (0.0005) [2023-12-27 04:40:35,488][105692] Updated weights for policy 0, policy_version 1823151 (0.0006) [2023-12-27 04:40:35,879][105620] Updated weights for policy 1, policy_version 1827053 (0.0010) [2023-12-27 04:40:35,927][105620] Updated weights for policy 1, policy_version 1827063 (0.0010) [2023-12-27 04:40:35,972][105620] Updated weights for policy 1, policy_version 1827073 (0.0010) [2023-12-27 04:40:36,062][104569] Fps is (10 sec: 21299.4, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 934592512. Throughput: 0: 9700.6, 1: 9834.3. Samples: 934577644. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:40:36,062][104569] Avg episode reward: [(0, '8992.233'), (1, '9258.173')] [2023-12-27 04:40:36,099][105692] Updated weights for policy 0, policy_version 1823161 (0.0010) [2023-12-27 04:40:36,171][105692] Updated weights for policy 0, policy_version 1823171 (0.0010) [2023-12-27 04:40:36,231][105692] Updated weights for policy 0, policy_version 1823181 (0.0011) [2023-12-27 04:40:36,727][105620] Updated weights for policy 1, policy_version 1827083 (0.0010) [2023-12-27 04:40:36,794][105620] Updated weights for policy 1, policy_version 1827093 (0.0011) [2023-12-27 04:40:36,849][105620] Updated weights for policy 1, policy_version 1827103 (0.0010) [2023-12-27 04:40:36,976][105692] Updated weights for policy 0, policy_version 1823191 (0.0010) [2023-12-27 04:40:37,024][105692] Updated weights for policy 0, policy_version 1823201 (0.0010) [2023-12-27 04:40:37,074][105692] Updated weights for policy 0, policy_version 1823211 (0.0011) [2023-12-27 04:40:37,609][105620] Updated weights for policy 1, policy_version 1827113 (0.0010) [2023-12-27 04:40:37,677][105620] Updated weights for policy 1, policy_version 1827123 (0.0008) [2023-12-27 04:40:37,733][105620] Updated weights for policy 1, policy_version 1827133 (0.0007) [2023-12-27 04:40:37,788][105620] Updated weights for policy 1, policy_version 1827143 (0.0006) [2023-12-27 04:40:37,856][105692] Updated weights for policy 0, policy_version 1823221 (0.0010) [2023-12-27 04:40:37,918][105692] Updated weights for policy 0, policy_version 1823231 (0.0010) [2023-12-27 04:40:37,975][105692] Updated weights for policy 0, policy_version 1823241 (0.0009) [2023-12-27 04:40:38,373][105620] Updated weights for policy 1, policy_version 1827153 (0.0008) [2023-12-27 04:40:38,446][105620] Updated weights for policy 1, policy_version 1827163 (0.0008) [2023-12-27 04:40:38,518][105620] Updated weights for policy 1, policy_version 1827173 (0.0008) [2023-12-27 04:40:38,671][105692] Updated weights for policy 0, policy_version 1823251 (0.0009) [2023-12-27 04:40:38,722][105692] Updated weights for policy 0, policy_version 1823261 (0.0008) [2023-12-27 04:40:38,783][105692] Updated weights for policy 0, policy_version 1823271 (0.0009) [2023-12-27 04:40:39,183][105620] Updated weights for policy 1, policy_version 1827183 (0.0006) [2023-12-27 04:40:39,245][105620] Updated weights for policy 1, policy_version 1827193 (0.0007) [2023-12-27 04:40:39,296][105620] Updated weights for policy 1, policy_version 1827203 (0.0006) [2023-12-27 04:40:39,590][105692] Updated weights for policy 0, policy_version 1823281 (0.0009) [2023-12-27 04:40:39,650][105692] Updated weights for policy 0, policy_version 1823291 (0.0008) [2023-12-27 04:40:39,718][105692] Updated weights for policy 0, policy_version 1823301 (0.0008) [2023-12-27 04:40:39,784][105692] Updated weights for policy 0, policy_version 1823311 (0.0007) [2023-12-27 04:40:40,031][105620] Updated weights for policy 1, policy_version 1827213 (0.0006) [2023-12-27 04:40:40,096][105620] Updated weights for policy 1, policy_version 1827223 (0.0007) [2023-12-27 04:40:40,155][105620] Updated weights for policy 1, policy_version 1827233 (0.0006) [2023-12-27 04:40:40,531][105692] Updated weights for policy 0, policy_version 1823321 (0.0008) [2023-12-27 04:40:40,588][105692] Updated weights for policy 0, policy_version 1823331 (0.0008) [2023-12-27 04:40:40,636][105692] Updated weights for policy 0, policy_version 1823341 (0.0008) [2023-12-27 04:40:40,877][105620] Updated weights for policy 1, policy_version 1827243 (0.0009) [2023-12-27 04:40:40,934][105620] Updated weights for policy 1, policy_version 1827253 (0.0010) [2023-12-27 04:40:40,993][105620] Updated weights for policy 1, policy_version 1827263 (0.0011) [2023-12-27 04:40:41,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 934690816. Throughput: 0: 9694.6, 1: 9813.5. Samples: 934696048. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:40:41,063][104569] Avg episode reward: [(0, '8808.119'), (1, '9258.122')] [2023-12-27 04:40:41,375][105692] Updated weights for policy 0, policy_version 1823351 (0.0008) [2023-12-27 04:40:41,435][105692] Updated weights for policy 0, policy_version 1823361 (0.0009) [2023-12-27 04:40:41,490][105692] Updated weights for policy 0, policy_version 1823371 (0.0009) [2023-12-27 04:40:41,713][105620] Updated weights for policy 1, policy_version 1827273 (0.0010) [2023-12-27 04:40:41,783][105620] Updated weights for policy 1, policy_version 1827283 (0.0009) [2023-12-27 04:40:41,847][105620] Updated weights for policy 1, policy_version 1827293 (0.0005) [2023-12-27 04:40:41,902][105620] Updated weights for policy 1, policy_version 1827303 (0.0008) [2023-12-27 04:40:42,221][105692] Updated weights for policy 0, policy_version 1823381 (0.0009) [2023-12-27 04:40:42,288][105692] Updated weights for policy 0, policy_version 1823391 (0.0010) [2023-12-27 04:40:42,359][105692] Updated weights for policy 0, policy_version 1823401 (0.0009) [2023-12-27 04:40:42,610][105620] Updated weights for policy 1, policy_version 1827313 (0.0008) [2023-12-27 04:40:42,661][105620] Updated weights for policy 1, policy_version 1827323 (0.0009) [2023-12-27 04:40:42,708][105620] Updated weights for policy 1, policy_version 1827333 (0.0009) [2023-12-27 04:40:43,111][105692] Updated weights for policy 0, policy_version 1823411 (0.0009) [2023-12-27 04:40:43,175][105692] Updated weights for policy 0, policy_version 1823421 (0.0008) [2023-12-27 04:40:43,234][105692] Updated weights for policy 0, policy_version 1823431 (0.0009) [2023-12-27 04:40:43,430][105620] Updated weights for policy 1, policy_version 1827343 (0.0009) [2023-12-27 04:40:43,486][105620] Updated weights for policy 1, policy_version 1827353 (0.0009) [2023-12-27 04:40:43,541][105620] Updated weights for policy 1, policy_version 1827363 (0.0010) [2023-12-27 04:40:43,831][105692] Updated weights for policy 0, policy_version 1823441 (0.0008) [2023-12-27 04:40:43,878][105692] Updated weights for policy 0, policy_version 1823451 (0.0005) [2023-12-27 04:40:43,925][105692] Updated weights for policy 0, policy_version 1823461 (0.0005) [2023-12-27 04:40:43,983][105692] Updated weights for policy 0, policy_version 1823471 (0.0005) [2023-12-27 04:40:44,214][105620] Updated weights for policy 1, policy_version 1827373 (0.0007) [2023-12-27 04:40:44,271][105620] Updated weights for policy 1, policy_version 1827383 (0.0005) [2023-12-27 04:40:44,275][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000003 [2023-12-27 04:40:44,593][105692] Updated weights for policy 0, policy_version 1823481 (0.0010) [2023-12-27 04:40:44,652][105692] Updated weights for policy 0, policy_version 1823491 (0.0008) [2023-12-27 04:40:44,708][105692] Updated weights for policy 0, policy_version 1823501 (0.0008) [2023-12-27 04:40:45,047][105620] Updated weights for policy 1, policy_version 1827393 (0.0010) [2023-12-27 04:40:45,111][105620] Updated weights for policy 1, policy_version 1827403 (0.0011) [2023-12-27 04:40:45,175][105620] Updated weights for policy 1, policy_version 1827413 (0.0011) [2023-12-27 04:40:45,427][105692] Updated weights for policy 0, policy_version 1823511 (0.0006) [2023-12-27 04:40:45,498][105692] Updated weights for policy 0, policy_version 1823521 (0.0006) [2023-12-27 04:40:45,567][105692] Updated weights for policy 0, policy_version 1823531 (0.0007) [2023-12-27 04:40:45,801][105620] Updated weights for policy 1, policy_version 1827423 (0.0010) [2023-12-27 04:40:45,864][105620] Updated weights for policy 1, policy_version 1827433 (0.0011) [2023-12-27 04:40:45,920][105620] Updated weights for policy 1, policy_version 1827443 (0.0011) [2023-12-27 04:40:46,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 934789120. Throughput: 0: 9702.7, 1: 9829.9. Samples: 934753056. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:40:46,063][104569] Avg episode reward: [(0, '8447.871'), (1, '9258.091')] [2023-12-27 04:40:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001827448_467894272.pth... [2023-12-27 04:40:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001826312_467599360.pth [2023-12-27 04:40:46,109][105692] Updated weights for policy 0, policy_version 1823541 (0.0005) [2023-12-27 04:40:46,171][105692] Updated weights for policy 0, policy_version 1823551 (0.0005) [2023-12-27 04:40:46,234][105692] Updated weights for policy 0, policy_version 1823561 (0.0007) [2023-12-27 04:40:46,274][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001823568_466903040.pth... [2023-12-27 04:40:46,278][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001822384_466599936.pth [2023-12-27 04:40:46,617][105620] Updated weights for policy 1, policy_version 1827453 (0.0008) [2023-12-27 04:40:46,680][105620] Updated weights for policy 1, policy_version 1827463 (0.0007) [2023-12-27 04:40:46,748][105620] Updated weights for policy 1, policy_version 1827473 (0.0007) [2023-12-27 04:40:46,888][105692] Updated weights for policy 0, policy_version 1823571 (0.0008) [2023-12-27 04:40:46,949][105692] Updated weights for policy 0, policy_version 1823581 (0.0005) [2023-12-27 04:40:47,016][105692] Updated weights for policy 0, policy_version 1823591 (0.0005) [2023-12-27 04:40:47,449][105620] Updated weights for policy 1, policy_version 1827483 (0.0008) [2023-12-27 04:40:47,495][105620] Updated weights for policy 1, policy_version 1827493 (0.0008) [2023-12-27 04:40:47,541][105620] Updated weights for policy 1, policy_version 1827503 (0.0009) [2023-12-27 04:40:47,551][105692] Updated weights for policy 0, policy_version 1823601 (0.0006) [2023-12-27 04:40:47,607][105692] Updated weights for policy 0, policy_version 1823611 (0.0008) [2023-12-27 04:40:47,661][105692] Updated weights for policy 0, policy_version 1823621 (0.0009) [2023-12-27 04:40:47,707][105692] Updated weights for policy 0, policy_version 1823631 (0.0009) [2023-12-27 04:40:48,228][105620] Updated weights for policy 1, policy_version 1827513 (0.0006) [2023-12-27 04:40:48,282][105620] Updated weights for policy 1, policy_version 1827523 (0.0009) [2023-12-27 04:40:48,346][105620] Updated weights for policy 1, policy_version 1827533 (0.0008) [2023-12-27 04:40:48,409][105620] Updated weights for policy 1, policy_version 1827543 (0.0009) [2023-12-27 04:40:48,539][105692] Updated weights for policy 0, policy_version 1823641 (0.0008) [2023-12-27 04:40:48,599][105692] Updated weights for policy 0, policy_version 1823651 (0.0009) [2023-12-27 04:40:48,656][105692] Updated weights for policy 0, policy_version 1823661 (0.0008) [2023-12-27 04:40:49,147][105620] Updated weights for policy 1, policy_version 1827553 (0.0006) [2023-12-27 04:40:49,199][105620] Updated weights for policy 1, policy_version 1827563 (0.0008) [2023-12-27 04:40:49,266][105620] Updated weights for policy 1, policy_version 1827573 (0.0011) [2023-12-27 04:40:49,297][105692] Updated weights for policy 0, policy_version 1823671 (0.0007) [2023-12-27 04:40:49,354][105692] Updated weights for policy 0, policy_version 1823681 (0.0007) [2023-12-27 04:40:49,414][105692] Updated weights for policy 0, policy_version 1823691 (0.0008) [2023-12-27 04:40:50,015][105620] Updated weights for policy 1, policy_version 1827583 (0.0010) [2023-12-27 04:40:50,086][105620] Updated weights for policy 1, policy_version 1827593 (0.0006) [2023-12-27 04:40:50,154][105620] Updated weights for policy 1, policy_version 1827603 (0.0006) [2023-12-27 04:40:50,174][105692] Updated weights for policy 0, policy_version 1823701 (0.0007) [2023-12-27 04:40:50,237][105692] Updated weights for policy 0, policy_version 1823711 (0.0006) [2023-12-27 04:40:50,305][105692] Updated weights for policy 0, policy_version 1823721 (0.0006) [2023-12-27 04:40:50,863][105620] Updated weights for policy 1, policy_version 1827613 (0.0011) [2023-12-27 04:40:50,897][105692] Updated weights for policy 0, policy_version 1823731 (0.0007) [2023-12-27 04:40:50,923][105620] Updated weights for policy 1, policy_version 1827623 (0.0011) [2023-12-27 04:40:50,946][105692] Updated weights for policy 0, policy_version 1823741 (0.0006) [2023-12-27 04:40:50,979][105620] Updated weights for policy 1, policy_version 1827633 (0.0011) [2023-12-27 04:40:51,008][105692] Updated weights for policy 0, policy_version 1823751 (0.0007) [2023-12-27 04:40:51,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 934895616. Throughput: 0: 9877.3, 1: 9804.0. Samples: 934876444. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:40:51,062][104569] Avg episode reward: [(0, '8360.150'), (1, '9350.449')] [2023-12-27 04:40:51,685][105620] Updated weights for policy 1, policy_version 1827643 (0.0010) [2023-12-27 04:40:51,753][105620] Updated weights for policy 1, policy_version 1827653 (0.0011) [2023-12-27 04:40:51,784][105692] Updated weights for policy 0, policy_version 1823761 (0.0007) [2023-12-27 04:40:51,816][105620] Updated weights for policy 1, policy_version 1827663 (0.0011) [2023-12-27 04:40:51,850][105692] Updated weights for policy 0, policy_version 1823771 (0.0006) [2023-12-27 04:40:51,918][105692] Updated weights for policy 0, policy_version 1823781 (0.0006) [2023-12-27 04:40:51,980][105692] Updated weights for policy 0, policy_version 1823791 (0.0006) [2023-12-27 04:40:52,590][105620] Updated weights for policy 1, policy_version 1827673 (0.0010) [2023-12-27 04:40:52,649][105620] Updated weights for policy 1, policy_version 1827683 (0.0009) [2023-12-27 04:40:52,660][105692] Updated weights for policy 0, policy_version 1823801 (0.0007) [2023-12-27 04:40:52,709][105620] Updated weights for policy 1, policy_version 1827693 (0.0006) [2023-12-27 04:40:52,720][105692] Updated weights for policy 0, policy_version 1823811 (0.0009) [2023-12-27 04:40:52,772][105620] Updated weights for policy 1, policy_version 1827703 (0.0008) [2023-12-27 04:40:52,779][105692] Updated weights for policy 0, policy_version 1823821 (0.0008) [2023-12-27 04:40:53,395][105620] Updated weights for policy 1, policy_version 1827713 (0.0007) [2023-12-27 04:40:53,454][105620] Updated weights for policy 1, policy_version 1827723 (0.0005) [2023-12-27 04:40:53,512][105620] Updated weights for policy 1, policy_version 1827733 (0.0006) [2023-12-27 04:40:53,532][105692] Updated weights for policy 0, policy_version 1823831 (0.0007) [2023-12-27 04:40:53,585][105692] Updated weights for policy 0, policy_version 1823841 (0.0010) [2023-12-27 04:40:53,639][105692] Updated weights for policy 0, policy_version 1823851 (0.0010) [2023-12-27 04:40:54,063][105620] Updated weights for policy 1, policy_version 1827743 (0.0009) [2023-12-27 04:40:54,112][105620] Updated weights for policy 1, policy_version 1827753 (0.0011) [2023-12-27 04:40:54,169][105620] Updated weights for policy 1, policy_version 1827763 (0.0011) [2023-12-27 04:40:54,337][105692] Updated weights for policy 0, policy_version 1823861 (0.0009) [2023-12-27 04:40:54,394][105692] Updated weights for policy 0, policy_version 1823872 (0.0009) [2023-12-27 04:40:54,465][105692] Updated weights for policy 0, policy_version 1823882 (0.0010) [2023-12-27 04:40:54,783][105620] Updated weights for policy 1, policy_version 1827773 (0.0008) [2023-12-27 04:40:54,843][105620] Updated weights for policy 1, policy_version 1827783 (0.0006) [2023-12-27 04:40:54,897][105620] Updated weights for policy 1, policy_version 1827793 (0.0005) [2023-12-27 04:40:55,201][105692] Updated weights for policy 0, policy_version 1823892 (0.0009) [2023-12-27 04:40:55,263][105692] Updated weights for policy 0, policy_version 1823902 (0.0011) [2023-12-27 04:40:55,328][105692] Updated weights for policy 0, policy_version 1823912 (0.0011) [2023-12-27 04:40:55,512][105620] Updated weights for policy 1, policy_version 1827803 (0.0005) [2023-12-27 04:40:55,570][105620] Updated weights for policy 1, policy_version 1827813 (0.0005) [2023-12-27 04:40:55,626][105620] Updated weights for policy 1, policy_version 1827823 (0.0008) [2023-12-27 04:40:55,986][105692] Updated weights for policy 0, policy_version 1823922 (0.0007) [2023-12-27 04:40:56,044][105692] Updated weights for policy 0, policy_version 1823932 (0.0011) [2023-12-27 04:40:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 934985728. Throughput: 0: 9883.4, 1: 9888.1. Samples: 934996756. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:40:56,062][104569] Avg episode reward: [(0, '8809.075'), (1, '9165.798')] [2023-12-27 04:40:56,106][105692] Updated weights for policy 0, policy_version 1823942 (0.0011) [2023-12-27 04:40:56,164][105692] Updated weights for policy 0, policy_version 1823952 (0.0011) [2023-12-27 04:40:56,241][105620] Updated weights for policy 1, policy_version 1827833 (0.0009) [2023-12-27 04:40:56,297][105620] Updated weights for policy 1, policy_version 1827843 (0.0005) [2023-12-27 04:40:56,361][105620] Updated weights for policy 1, policy_version 1827853 (0.0005) [2023-12-27 04:40:56,415][105620] Updated weights for policy 1, policy_version 1827863 (0.0005) [2023-12-27 04:40:56,914][105620] Updated weights for policy 1, policy_version 1827873 (0.0005) [2023-12-27 04:40:56,930][105692] Updated weights for policy 0, policy_version 1823962 (0.0009) [2023-12-27 04:40:56,962][105620] Updated weights for policy 1, policy_version 1827883 (0.0005) [2023-12-27 04:40:56,991][105692] Updated weights for policy 0, policy_version 1823972 (0.0009) [2023-12-27 04:40:57,013][105620] Updated weights for policy 1, policy_version 1827893 (0.0005) [2023-12-27 04:40:57,045][105692] Updated weights for policy 0, policy_version 1823982 (0.0008) [2023-12-27 04:40:57,539][105620] Updated weights for policy 1, policy_version 1827903 (0.0009) [2023-12-27 04:40:57,590][105620] Updated weights for policy 1, policy_version 1827913 (0.0010) [2023-12-27 04:40:57,634][105620] Updated weights for policy 1, policy_version 1827923 (0.0010) [2023-12-27 04:40:57,877][105692] Updated weights for policy 0, policy_version 1823992 (0.0010) [2023-12-27 04:40:57,931][105692] Updated weights for policy 0, policy_version 1824002 (0.0010) [2023-12-27 04:40:57,985][105692] Updated weights for policy 0, policy_version 1824012 (0.0010) [2023-12-27 04:40:58,233][105620] Updated weights for policy 1, policy_version 1827933 (0.0009) [2023-12-27 04:40:58,285][105620] Updated weights for policy 1, policy_version 1827943 (0.0010) [2023-12-27 04:40:58,352][105620] Updated weights for policy 1, policy_version 1827953 (0.0010) [2023-12-27 04:40:58,829][105692] Updated weights for policy 0, policy_version 1824022 (0.0009) [2023-12-27 04:40:58,903][105692] Updated weights for policy 0, policy_version 1824032 (0.0008) [2023-12-27 04:40:58,977][105692] Updated weights for policy 0, policy_version 1824042 (0.0008) [2023-12-27 04:40:59,252][105620] Updated weights for policy 1, policy_version 1827963 (0.0012) [2023-12-27 04:40:59,318][105620] Updated weights for policy 1, policy_version 1827973 (0.0008) [2023-12-27 04:40:59,386][105620] Updated weights for policy 1, policy_version 1827983 (0.0008) [2023-12-27 04:40:59,775][105692] Updated weights for policy 0, policy_version 1824052 (0.0009) [2023-12-27 04:40:59,832][105692] Updated weights for policy 0, policy_version 1824062 (0.0006) [2023-12-27 04:40:59,899][105692] Updated weights for policy 0, policy_version 1824072 (0.0009) [2023-12-27 04:41:00,112][105620] Updated weights for policy 1, policy_version 1827993 (0.0008) [2023-12-27 04:41:00,167][105620] Updated weights for policy 1, policy_version 1828003 (0.0009) [2023-12-27 04:41:00,229][105620] Updated weights for policy 1, policy_version 1828013 (0.0009) [2023-12-27 04:41:00,290][105620] Updated weights for policy 1, policy_version 1828023 (0.0009) [2023-12-27 04:41:00,675][105692] Updated weights for policy 0, policy_version 1824082 (0.0009) [2023-12-27 04:41:00,734][105692] Updated weights for policy 0, policy_version 1824092 (0.0009) [2023-12-27 04:41:00,782][105692] Updated weights for policy 0, policy_version 1824102 (0.0009) [2023-12-27 04:41:00,830][105692] Updated weights for policy 0, policy_version 1824112 (0.0009) [2023-12-27 04:41:00,967][105620] Updated weights for policy 1, policy_version 1828033 (0.0008) [2023-12-27 04:41:01,025][105620] Updated weights for policy 1, policy_version 1828043 (0.0009) [2023-12-27 04:41:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19797.3, 300 sec: 19633.0). Total num frames: 935084032. Throughput: 0: 9853.8, 1: 10039.2. Samples: 935058140. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:41:01,063][104569] Avg episode reward: [(0, '8536.254'), (1, '9258.210')] [2023-12-27 04:41:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001824112_467042304.pth... [2023-12-27 04:41:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001822960_466747392.pth [2023-12-27 04:41:01,089][105620] Updated weights for policy 1, policy_version 1828053 (0.0009) [2023-12-27 04:41:01,104][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001828056_468049920.pth... [2023-12-27 04:41:01,107][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001826856_467738624.pth [2023-12-27 04:41:01,545][105692] Updated weights for policy 0, policy_version 1824122 (0.0007) [2023-12-27 04:41:01,613][105692] Updated weights for policy 0, policy_version 1824132 (0.0009) [2023-12-27 04:41:01,670][105692] Updated weights for policy 0, policy_version 1824142 (0.0009) [2023-12-27 04:41:01,744][105620] Updated weights for policy 1, policy_version 1828063 (0.0010) [2023-12-27 04:41:01,803][105620] Updated weights for policy 1, policy_version 1828073 (0.0008) [2023-12-27 04:41:01,869][105620] Updated weights for policy 1, policy_version 1828083 (0.0009) [2023-12-27 04:41:02,283][105692] Updated weights for policy 0, policy_version 1824152 (0.0011) [2023-12-27 04:41:02,342][105692] Updated weights for policy 0, policy_version 1824162 (0.0011) [2023-12-27 04:41:02,412][105692] Updated weights for policy 0, policy_version 1824172 (0.0010) [2023-12-27 04:41:02,573][105620] Updated weights for policy 1, policy_version 1828093 (0.0008) [2023-12-27 04:41:02,623][105620] Updated weights for policy 1, policy_version 1828103 (0.0010) [2023-12-27 04:41:02,681][105620] Updated weights for policy 1, policy_version 1828113 (0.0010) [2023-12-27 04:41:03,106][105692] Updated weights for policy 0, policy_version 1824182 (0.0010) [2023-12-27 04:41:03,163][105692] Updated weights for policy 0, policy_version 1824192 (0.0009) [2023-12-27 04:41:03,231][105692] Updated weights for policy 0, policy_version 1824202 (0.0009) [2023-12-27 04:41:03,319][105620] Updated weights for policy 1, policy_version 1828123 (0.0009) [2023-12-27 04:41:03,376][105620] Updated weights for policy 1, policy_version 1828133 (0.0005) [2023-12-27 04:41:03,430][105620] Updated weights for policy 1, policy_version 1828143 (0.0006) [2023-12-27 04:41:03,945][105692] Updated weights for policy 0, policy_version 1824212 (0.0010) [2023-12-27 04:41:04,010][105692] Updated weights for policy 0, policy_version 1824222 (0.0008) [2023-12-27 04:41:04,026][105620] Updated weights for policy 1, policy_version 1828153 (0.0006) [2023-12-27 04:41:04,076][105692] Updated weights for policy 0, policy_version 1824232 (0.0007) [2023-12-27 04:41:04,087][105620] Updated weights for policy 1, policy_version 1828163 (0.0008) [2023-12-27 04:41:04,154][105620] Updated weights for policy 1, policy_version 1828173 (0.0005) [2023-12-27 04:41:04,222][105620] Updated weights for policy 1, policy_version 1828183 (0.0009) [2023-12-27 04:41:04,823][105620] Updated weights for policy 1, policy_version 1828193 (0.0006) [2023-12-27 04:41:04,878][105620] Updated weights for policy 1, policy_version 1828203 (0.0005) [2023-12-27 04:41:04,896][105692] Updated weights for policy 0, policy_version 1824242 (0.0009) [2023-12-27 04:41:04,939][105620] Updated weights for policy 1, policy_version 1828213 (0.0007) [2023-12-27 04:41:04,961][105692] Updated weights for policy 0, policy_version 1824252 (0.0006) [2023-12-27 04:41:05,030][105692] Updated weights for policy 0, policy_version 1824262 (0.0006) [2023-12-27 04:41:05,086][105692] Updated weights for policy 0, policy_version 1824272 (0.0008) [2023-12-27 04:41:05,615][105620] Updated weights for policy 1, policy_version 1828223 (0.0010) [2023-12-27 04:41:05,676][105620] Updated weights for policy 1, policy_version 1828233 (0.0010) [2023-12-27 04:41:05,738][105620] Updated weights for policy 1, policy_version 1828243 (0.0010) [2023-12-27 04:41:05,790][105692] Updated weights for policy 0, policy_version 1824282 (0.0009) [2023-12-27 04:41:05,841][105692] Updated weights for policy 0, policy_version 1824292 (0.0008) [2023-12-27 04:41:05,893][105692] Updated weights for policy 0, policy_version 1824302 (0.0008) [2023-12-27 04:41:06,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 935190528. Throughput: 0: 9871.4, 1: 10130.4. Samples: 935177308. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:41:06,062][104569] Avg episode reward: [(0, '8536.246'), (1, '9258.046')] [2023-12-27 04:41:06,422][105620] Updated weights for policy 1, policy_version 1828253 (0.0010) [2023-12-27 04:41:06,481][105620] Updated weights for policy 1, policy_version 1828263 (0.0010) [2023-12-27 04:41:06,540][105620] Updated weights for policy 1, policy_version 1828273 (0.0010) [2023-12-27 04:41:06,713][105692] Updated weights for policy 0, policy_version 1824312 (0.0010) [2023-12-27 04:41:06,767][105692] Updated weights for policy 0, policy_version 1824322 (0.0011) [2023-12-27 04:41:06,825][105692] Updated weights for policy 0, policy_version 1824332 (0.0011) [2023-12-27 04:41:07,261][105620] Updated weights for policy 1, policy_version 1828283 (0.0010) [2023-12-27 04:41:07,313][105620] Updated weights for policy 1, policy_version 1828293 (0.0010) [2023-12-27 04:41:07,365][105620] Updated weights for policy 1, policy_version 1828303 (0.0010) [2023-12-27 04:41:07,523][105692] Updated weights for policy 0, policy_version 1824342 (0.0007) [2023-12-27 04:41:07,579][105692] Updated weights for policy 0, policy_version 1824352 (0.0005) [2023-12-27 04:41:07,632][105692] Updated weights for policy 0, policy_version 1824362 (0.0005) [2023-12-27 04:41:08,101][105620] Updated weights for policy 1, policy_version 1828313 (0.0010) [2023-12-27 04:41:08,162][105620] Updated weights for policy 1, policy_version 1828323 (0.0009) [2023-12-27 04:41:08,223][105620] Updated weights for policy 1, policy_version 1828333 (0.0007) [2023-12-27 04:41:08,281][105620] Updated weights for policy 1, policy_version 1828343 (0.0010) [2023-12-27 04:41:08,296][105692] Updated weights for policy 0, policy_version 1824372 (0.0007) [2023-12-27 04:41:08,352][105692] Updated weights for policy 0, policy_version 1824382 (0.0011) [2023-12-27 04:41:08,413][105692] Updated weights for policy 0, policy_version 1824392 (0.0010) [2023-12-27 04:41:08,980][105620] Updated weights for policy 1, policy_version 1828353 (0.0006) [2023-12-27 04:41:09,040][105620] Updated weights for policy 1, policy_version 1828363 (0.0008) [2023-12-27 04:41:09,098][105692] Updated weights for policy 0, policy_version 1824402 (0.0007) [2023-12-27 04:41:09,101][105620] Updated weights for policy 1, policy_version 1828373 (0.0009) [2023-12-27 04:41:09,158][105692] Updated weights for policy 0, policy_version 1824412 (0.0006) [2023-12-27 04:41:09,207][105692] Updated weights for policy 0, policy_version 1824422 (0.0008) [2023-12-27 04:41:09,271][105692] Updated weights for policy 0, policy_version 1824432 (0.0007) [2023-12-27 04:41:09,882][105620] Updated weights for policy 1, policy_version 1828383 (0.0008) [2023-12-27 04:41:09,944][105620] Updated weights for policy 1, policy_version 1828393 (0.0009) [2023-12-27 04:41:10,004][105620] Updated weights for policy 1, policy_version 1828403 (0.0008) [2023-12-27 04:41:10,036][105692] Updated weights for policy 0, policy_version 1824442 (0.0007) [2023-12-27 04:41:10,098][105692] Updated weights for policy 0, policy_version 1824452 (0.0009) [2023-12-27 04:41:10,162][105692] Updated weights for policy 0, policy_version 1824462 (0.0009) [2023-12-27 04:41:10,815][105620] Updated weights for policy 1, policy_version 1828414 (0.0009) [2023-12-27 04:41:10,867][105620] Updated weights for policy 1, policy_version 1828424 (0.0006) [2023-12-27 04:41:10,878][105692] Updated weights for policy 0, policy_version 1824472 (0.0008) [2023-12-27 04:41:10,916][105620] Updated weights for policy 1, policy_version 1828434 (0.0010) [2023-12-27 04:41:10,940][105692] Updated weights for policy 0, policy_version 1824482 (0.0007) [2023-12-27 04:41:10,985][105692] Updated weights for policy 0, policy_version 1824492 (0.0008) [2023-12-27 04:41:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 935288832. Throughput: 0: 9873.9, 1: 10119.7. Samples: 935292060. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:41:11,062][104569] Avg episode reward: [(0, '8723.249'), (1, '9258.018')] [2023-12-27 04:41:11,722][105620] Updated weights for policy 1, policy_version 1828444 (0.0010) [2023-12-27 04:41:11,758][105692] Updated weights for policy 0, policy_version 1824502 (0.0008) [2023-12-27 04:41:11,792][105620] Updated weights for policy 1, policy_version 1828454 (0.0007) [2023-12-27 04:41:11,817][105692] Updated weights for policy 0, policy_version 1824512 (0.0009) [2023-12-27 04:41:11,854][105620] Updated weights for policy 1, policy_version 1828464 (0.0006) [2023-12-27 04:41:11,870][105692] Updated weights for policy 0, policy_version 1824522 (0.0009) [2023-12-27 04:41:12,466][105620] Updated weights for policy 1, policy_version 1828474 (0.0006) [2023-12-27 04:41:12,534][105620] Updated weights for policy 1, policy_version 1828484 (0.0008) [2023-12-27 04:41:12,582][105620] Updated weights for policy 1, policy_version 1828494 (0.0009) [2023-12-27 04:41:12,614][105692] Updated weights for policy 0, policy_version 1824532 (0.0008) [2023-12-27 04:41:12,637][105620] Updated weights for policy 1, policy_version 1828504 (0.0008) [2023-12-27 04:41:12,677][105692] Updated weights for policy 0, policy_version 1824542 (0.0008) [2023-12-27 04:41:12,732][105692] Updated weights for policy 0, policy_version 1824552 (0.0009) [2023-12-27 04:41:13,332][105692] Updated weights for policy 0, policy_version 1824562 (0.0009) [2023-12-27 04:41:13,383][105620] Updated weights for policy 1, policy_version 1828514 (0.0008) [2023-12-27 04:41:13,395][105692] Updated weights for policy 0, policy_version 1824572 (0.0007) [2023-12-27 04:41:13,436][105620] Updated weights for policy 1, policy_version 1828524 (0.0009) [2023-12-27 04:41:13,455][105692] Updated weights for policy 0, policy_version 1824582 (0.0009) [2023-12-27 04:41:13,486][105620] Updated weights for policy 1, policy_version 1828534 (0.0006) [2023-12-27 04:41:13,514][105692] Updated weights for policy 0, policy_version 1824592 (0.0009) [2023-12-27 04:41:14,212][105692] Updated weights for policy 0, policy_version 1824602 (0.0010) [2023-12-27 04:41:14,234][105620] Updated weights for policy 1, policy_version 1828544 (0.0009) [2023-12-27 04:41:14,271][105692] Updated weights for policy 0, policy_version 1824612 (0.0010) [2023-12-27 04:41:14,283][105620] Updated weights for policy 1, policy_version 1828554 (0.0006) [2023-12-27 04:41:14,329][105692] Updated weights for policy 0, policy_version 1824622 (0.0010) [2023-12-27 04:41:14,332][105620] Updated weights for policy 1, policy_version 1828564 (0.0006) [2023-12-27 04:41:15,025][105620] Updated weights for policy 1, policy_version 1828574 (0.0009) [2023-12-27 04:41:15,062][105692] Updated weights for policy 0, policy_version 1824632 (0.0006) [2023-12-27 04:41:15,092][105620] Updated weights for policy 1, policy_version 1828584 (0.0009) [2023-12-27 04:41:15,122][105692] Updated weights for policy 0, policy_version 1824642 (0.0006) [2023-12-27 04:41:15,152][105620] Updated weights for policy 1, policy_version 1828594 (0.0007) [2023-12-27 04:41:15,185][105692] Updated weights for policy 0, policy_version 1824652 (0.0011) [2023-12-27 04:41:15,901][105620] Updated weights for policy 1, policy_version 1828604 (0.0009) [2023-12-27 04:41:15,915][105692] Updated weights for policy 0, policy_version 1824662 (0.0008) [2023-12-27 04:41:15,959][105620] Updated weights for policy 1, policy_version 1828614 (0.0010) [2023-12-27 04:41:15,969][105692] Updated weights for policy 0, policy_version 1824672 (0.0007) [2023-12-27 04:41:16,028][105620] Updated weights for policy 1, policy_version 1828624 (0.0009) [2023-12-27 04:41:16,034][105692] Updated weights for policy 0, policy_version 1824682 (0.0007) [2023-12-27 04:41:16,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 935378944. Throughput: 0: 9811.0, 1: 10119.7. Samples: 935350452. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:41:16,062][104569] Avg episode reward: [(0, '8630.974'), (1, '9258.017')] [2023-12-27 04:41:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001824688_467189760.pth... [2023-12-27 04:41:16,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001823568_466903040.pth [2023-12-27 04:41:16,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001828632_468197376.pth... [2023-12-27 04:41:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001827448_467894272.pth [2023-12-27 04:41:16,725][105620] Updated weights for policy 1, policy_version 1828634 (0.0010) [2023-12-27 04:41:16,772][105692] Updated weights for policy 0, policy_version 1824692 (0.0007) [2023-12-27 04:41:16,774][105620] Updated weights for policy 1, policy_version 1828644 (0.0008) [2023-12-27 04:41:16,815][105692] Updated weights for policy 0, policy_version 1824702 (0.0007) [2023-12-27 04:41:16,825][105620] Updated weights for policy 1, policy_version 1828654 (0.0007) [2023-12-27 04:41:16,861][105692] Updated weights for policy 0, policy_version 1824712 (0.0006) [2023-12-27 04:41:16,877][105620] Updated weights for policy 1, policy_version 1828664 (0.0007) [2023-12-27 04:41:17,507][105692] Updated weights for policy 0, policy_version 1824722 (0.0007) [2023-12-27 04:41:17,558][105692] Updated weights for policy 0, policy_version 1824732 (0.0006) [2023-12-27 04:41:17,586][105620] Updated weights for policy 1, policy_version 1828674 (0.0005) [2023-12-27 04:41:17,606][105692] Updated weights for policy 0, policy_version 1824742 (0.0009) [2023-12-27 04:41:17,637][105620] Updated weights for policy 1, policy_version 1828684 (0.0005) [2023-12-27 04:41:17,654][105692] Updated weights for policy 0, policy_version 1824752 (0.0008) [2023-12-27 04:41:17,690][105620] Updated weights for policy 1, policy_version 1828694 (0.0005) [2023-12-27 04:41:18,235][105620] Updated weights for policy 1, policy_version 1828704 (0.0005) [2023-12-27 04:41:18,286][105620] Updated weights for policy 1, policy_version 1828714 (0.0005) [2023-12-27 04:41:18,343][105620] Updated weights for policy 1, policy_version 1828724 (0.0007) [2023-12-27 04:41:18,511][105692] Updated weights for policy 0, policy_version 1824762 (0.0011) [2023-12-27 04:41:18,561][105692] Updated weights for policy 0, policy_version 1824772 (0.0011) [2023-12-27 04:41:18,631][105692] Updated weights for policy 0, policy_version 1824782 (0.0011) [2023-12-27 04:41:18,910][105620] Updated weights for policy 1, policy_version 1828734 (0.0007) [2023-12-27 04:41:18,962][105620] Updated weights for policy 1, policy_version 1828744 (0.0005) [2023-12-27 04:41:19,019][105620] Updated weights for policy 1, policy_version 1828754 (0.0005) [2023-12-27 04:41:19,318][105692] Updated weights for policy 0, policy_version 1824792 (0.0011) [2023-12-27 04:41:19,382][105692] Updated weights for policy 0, policy_version 1824802 (0.0010) [2023-12-27 04:41:19,441][105692] Updated weights for policy 0, policy_version 1824812 (0.0011) [2023-12-27 04:41:19,686][105620] Updated weights for policy 1, policy_version 1828764 (0.0007) [2023-12-27 04:41:19,746][105620] Updated weights for policy 1, policy_version 1828774 (0.0010) [2023-12-27 04:41:19,802][105620] Updated weights for policy 1, policy_version 1828784 (0.0009) [2023-12-27 04:41:20,207][105692] Updated weights for policy 0, policy_version 1824822 (0.0010) [2023-12-27 04:41:20,265][105692] Updated weights for policy 0, policy_version 1824832 (0.0009) [2023-12-27 04:41:20,325][105692] Updated weights for policy 0, policy_version 1824842 (0.0009) [2023-12-27 04:41:20,542][105620] Updated weights for policy 1, policy_version 1828794 (0.0008) [2023-12-27 04:41:20,612][105620] Updated weights for policy 1, policy_version 1828804 (0.0007) [2023-12-27 04:41:20,682][105620] Updated weights for policy 1, policy_version 1828814 (0.0007) [2023-12-27 04:41:20,745][105620] Updated weights for policy 1, policy_version 1828824 (0.0009) [2023-12-27 04:41:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19797.4, 300 sec: 19688.6). Total num frames: 935477248. Throughput: 0: 9749.7, 1: 10078.3. Samples: 935469904. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-12-27 04:41:21,062][104569] Avg episode reward: [(0, '8349.617'), (1, '9165.785')] [2023-12-27 04:41:21,101][105692] Updated weights for policy 0, policy_version 1824852 (0.0009) [2023-12-27 04:41:21,163][105692] Updated weights for policy 0, policy_version 1824862 (0.0008) [2023-12-27 04:41:21,219][105692] Updated weights for policy 0, policy_version 1824872 (0.0008) [2023-12-27 04:41:21,460][105620] Updated weights for policy 1, policy_version 1828834 (0.0007) [2023-12-27 04:41:21,515][105620] Updated weights for policy 1, policy_version 1828844 (0.0008) [2023-12-27 04:41:21,563][105620] Updated weights for policy 1, policy_version 1828854 (0.0009) [2023-12-27 04:41:22,036][105692] Updated weights for policy 0, policy_version 1824882 (0.0008) [2023-12-27 04:41:22,086][105692] Updated weights for policy 0, policy_version 1824892 (0.0008) [2023-12-27 04:41:22,144][105692] Updated weights for policy 0, policy_version 1824902 (0.0008) [2023-12-27 04:41:22,211][105692] Updated weights for policy 0, policy_version 1824912 (0.0008) [2023-12-27 04:41:22,326][105620] Updated weights for policy 1, policy_version 1828864 (0.0008) [2023-12-27 04:41:22,393][105620] Updated weights for policy 1, policy_version 1828874 (0.0007) [2023-12-27 04:41:22,467][105620] Updated weights for policy 1, policy_version 1828884 (0.0007) [2023-12-27 04:41:23,023][105620] Updated weights for policy 1, policy_version 1828894 (0.0009) [2023-12-27 04:41:23,085][105620] Updated weights for policy 1, policy_version 1828904 (0.0006) [2023-12-27 04:41:23,085][105692] Updated weights for policy 0, policy_version 1824922 (0.0009) [2023-12-27 04:41:23,147][105692] Updated weights for policy 0, policy_version 1824932 (0.0010) [2023-12-27 04:41:23,149][105620] Updated weights for policy 1, policy_version 1828914 (0.0007) [2023-12-27 04:41:23,216][105692] Updated weights for policy 0, policy_version 1824942 (0.0008) [2023-12-27 04:41:23,822][105620] Updated weights for policy 1, policy_version 1828924 (0.0009) [2023-12-27 04:41:23,878][105620] Updated weights for policy 1, policy_version 1828934 (0.0008) [2023-12-27 04:41:23,936][105620] Updated weights for policy 1, policy_version 1828944 (0.0005) [2023-12-27 04:41:24,028][105692] Updated weights for policy 0, policy_version 1824952 (0.0006) [2023-12-27 04:41:24,087][105692] Updated weights for policy 0, policy_version 1824962 (0.0006) [2023-12-27 04:41:24,138][105692] Updated weights for policy 0, policy_version 1824972 (0.0009) [2023-12-27 04:41:24,516][105620] Updated weights for policy 1, policy_version 1828954 (0.0007) [2023-12-27 04:41:24,573][105620] Updated weights for policy 1, policy_version 1828964 (0.0008) [2023-12-27 04:41:24,633][105620] Updated weights for policy 1, policy_version 1828974 (0.0005) [2023-12-27 04:41:24,699][105620] Updated weights for policy 1, policy_version 1828984 (0.0010) [2023-12-27 04:41:24,940][105692] Updated weights for policy 0, policy_version 1824983 (0.0010) [2023-12-27 04:41:25,000][105692] Updated weights for policy 0, policy_version 1824994 (0.0010) [2023-12-27 04:41:25,067][105692] Updated weights for policy 0, policy_version 1825004 (0.0010) [2023-12-27 04:41:25,286][105620] Updated weights for policy 1, policy_version 1828994 (0.0005) [2023-12-27 04:41:25,355][105620] Updated weights for policy 1, policy_version 1829004 (0.0010) [2023-12-27 04:41:25,418][105620] Updated weights for policy 1, policy_version 1829014 (0.0011) [2023-12-27 04:41:25,891][105692] Updated weights for policy 0, policy_version 1825014 (0.0010) [2023-12-27 04:41:25,943][105692] Updated weights for policy 0, policy_version 1825024 (0.0008) [2023-12-27 04:41:25,999][105692] Updated weights for policy 0, policy_version 1825034 (0.0005) [2023-12-27 04:41:26,024][105620] Updated weights for policy 1, policy_version 1829024 (0.0007) [2023-12-27 04:41:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19660.8). Total num frames: 935575552. Throughput: 0: 9590.5, 1: 10150.3. Samples: 935584384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:41:26,066][104569] Avg episode reward: [(0, '7893.775'), (1, '9073.631')] [2023-12-27 04:41:26,080][105620] Updated weights for policy 1, policy_version 1829034 (0.0005) [2023-12-27 04:41:26,131][105620] Updated weights for policy 1, policy_version 1829044 (0.0006) [2023-12-27 04:41:26,730][105692] Updated weights for policy 0, policy_version 1825044 (0.0010) [2023-12-27 04:41:26,768][105620] Updated weights for policy 1, policy_version 1829054 (0.0006) [2023-12-27 04:41:26,788][105692] Updated weights for policy 0, policy_version 1825054 (0.0010) [2023-12-27 04:41:26,823][105620] Updated weights for policy 1, policy_version 1829064 (0.0008) [2023-12-27 04:41:26,846][105692] Updated weights for policy 0, policy_version 1825064 (0.0010) [2023-12-27 04:41:26,876][105620] Updated weights for policy 1, policy_version 1829074 (0.0007) [2023-12-27 04:41:27,494][105620] Updated weights for policy 1, policy_version 1829084 (0.0006) [2023-12-27 04:41:27,519][105692] Updated weights for policy 0, policy_version 1825074 (0.0009) [2023-12-27 04:41:27,561][105620] Updated weights for policy 1, policy_version 1829094 (0.0007) [2023-12-27 04:41:27,566][105692] Updated weights for policy 0, policy_version 1825084 (0.0010) [2023-12-27 04:41:27,612][105620] Updated weights for policy 1, policy_version 1829104 (0.0005) [2023-12-27 04:41:27,614][105692] Updated weights for policy 0, policy_version 1825094 (0.0010) [2023-12-27 04:41:27,661][105692] Updated weights for policy 0, policy_version 1825104 (0.0010) [2023-12-27 04:41:28,313][105620] Updated weights for policy 1, policy_version 1829114 (0.0006) [2023-12-27 04:41:28,357][105692] Updated weights for policy 0, policy_version 1825114 (0.0008) [2023-12-27 04:41:28,378][105620] Updated weights for policy 1, policy_version 1829124 (0.0007) [2023-12-27 04:41:28,411][105692] Updated weights for policy 0, policy_version 1825124 (0.0010) [2023-12-27 04:41:28,440][105620] Updated weights for policy 1, policy_version 1829134 (0.0006) [2023-12-27 04:41:28,465][105692] Updated weights for policy 0, policy_version 1825134 (0.0011) [2023-12-27 04:41:28,498][105620] Updated weights for policy 1, policy_version 1829144 (0.0007) [2023-12-27 04:41:29,136][105620] Updated weights for policy 1, policy_version 1829154 (0.0008) [2023-12-27 04:41:29,181][105692] Updated weights for policy 0, policy_version 1825144 (0.0011) [2023-12-27 04:41:29,188][105620] Updated weights for policy 1, policy_version 1829164 (0.0005) [2023-12-27 04:41:29,239][105692] Updated weights for policy 0, policy_version 1825154 (0.0010) [2023-12-27 04:41:29,247][105620] Updated weights for policy 1, policy_version 1829174 (0.0007) [2023-12-27 04:41:29,302][105692] Updated weights for policy 0, policy_version 1825164 (0.0011) [2023-12-27 04:41:30,016][105620] Updated weights for policy 1, policy_version 1829184 (0.0008) [2023-12-27 04:41:30,062][105692] Updated weights for policy 0, policy_version 1825174 (0.0009) [2023-12-27 04:41:30,063][105620] Updated weights for policy 1, policy_version 1829194 (0.0007) [2023-12-27 04:41:30,111][105620] Updated weights for policy 1, policy_version 1829204 (0.0007) [2023-12-27 04:41:30,114][105692] Updated weights for policy 0, policy_version 1825184 (0.0008) [2023-12-27 04:41:30,172][105692] Updated weights for policy 0, policy_version 1825194 (0.0009) [2023-12-27 04:41:30,784][105620] Updated weights for policy 1, policy_version 1829214 (0.0005) [2023-12-27 04:41:30,834][105620] Updated weights for policy 1, policy_version 1829224 (0.0009) [2023-12-27 04:41:30,858][105692] Updated weights for policy 0, policy_version 1825204 (0.0006) [2023-12-27 04:41:30,887][105620] Updated weights for policy 1, policy_version 1829234 (0.0009) [2023-12-27 04:41:30,906][105692] Updated weights for policy 0, policy_version 1825214 (0.0006) [2023-12-27 04:41:30,955][105692] Updated weights for policy 0, policy_version 1825224 (0.0010) [2023-12-27 04:41:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19933.9, 300 sec: 19688.6). Total num frames: 935682048. Throughput: 0: 9624.3, 1: 10226.6. Samples: 935646344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:41:31,062][104569] Avg episode reward: [(0, '7986.597'), (1, '9166.177')] [2023-12-27 04:41:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001825232_467329024.pth... [2023-12-27 04:41:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001829240_468353024.pth... [2023-12-27 04:41:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001824112_467042304.pth [2023-12-27 04:41:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001828056_468049920.pth [2023-12-27 04:41:31,628][105620] Updated weights for policy 1, policy_version 1829244 (0.0008) [2023-12-27 04:41:31,641][105692] Updated weights for policy 0, policy_version 1825234 (0.0010) [2023-12-27 04:41:31,689][105620] Updated weights for policy 1, policy_version 1829254 (0.0009) [2023-12-27 04:41:31,702][105692] Updated weights for policy 0, policy_version 1825244 (0.0007) [2023-12-27 04:41:31,752][105620] Updated weights for policy 1, policy_version 1829264 (0.0007) [2023-12-27 04:41:31,763][105692] Updated weights for policy 0, policy_version 1825254 (0.0010) [2023-12-27 04:41:31,822][105692] Updated weights for policy 0, policy_version 1825264 (0.0006) [2023-12-27 04:41:32,366][105692] Updated weights for policy 0, policy_version 1825274 (0.0007) [2023-12-27 04:41:32,429][105692] Updated weights for policy 0, policy_version 1825284 (0.0011) [2023-12-27 04:41:32,483][105692] Updated weights for policy 0, policy_version 1825294 (0.0010) [2023-12-27 04:41:32,582][105620] Updated weights for policy 1, policy_version 1829274 (0.0009) [2023-12-27 04:41:32,641][105620] Updated weights for policy 1, policy_version 1829284 (0.0011) [2023-12-27 04:41:32,692][105620] Updated weights for policy 1, policy_version 1829294 (0.0010) [2023-12-27 04:41:32,751][105620] Updated weights for policy 1, policy_version 1829304 (0.0010) [2023-12-27 04:41:33,124][105692] Updated weights for policy 0, policy_version 1825304 (0.0006) [2023-12-27 04:41:33,187][105692] Updated weights for policy 0, policy_version 1825314 (0.0005) [2023-12-27 04:41:33,250][105692] Updated weights for policy 0, policy_version 1825324 (0.0006) [2023-12-27 04:41:33,497][105620] Updated weights for policy 1, policy_version 1829314 (0.0010) [2023-12-27 04:41:33,548][105620] Updated weights for policy 1, policy_version 1829324 (0.0010) [2023-12-27 04:41:33,603][105620] Updated weights for policy 1, policy_version 1829334 (0.0010) [2023-12-27 04:41:33,743][105692] Updated weights for policy 0, policy_version 1825334 (0.0005) [2023-12-27 04:41:33,789][105692] Updated weights for policy 0, policy_version 1825344 (0.0005) [2023-12-27 04:41:33,846][105692] Updated weights for policy 0, policy_version 1825354 (0.0005) [2023-12-27 04:41:34,274][105620] Updated weights for policy 1, policy_version 1829344 (0.0008) [2023-12-27 04:41:34,333][105620] Updated weights for policy 1, policy_version 1829354 (0.0005) [2023-12-27 04:41:34,391][105620] Updated weights for policy 1, policy_version 1829364 (0.0009) [2023-12-27 04:41:34,505][105692] Updated weights for policy 0, policy_version 1825364 (0.0007) [2023-12-27 04:41:34,561][105692] Updated weights for policy 0, policy_version 1825374 (0.0010) [2023-12-27 04:41:34,623][105692] Updated weights for policy 0, policy_version 1825384 (0.0010) [2023-12-27 04:41:35,093][105620] Updated weights for policy 1, policy_version 1829374 (0.0008) [2023-12-27 04:41:35,149][105620] Updated weights for policy 1, policy_version 1829384 (0.0008) [2023-12-27 04:41:35,201][105620] Updated weights for policy 1, policy_version 1829394 (0.0010) [2023-12-27 04:41:35,257][105692] Updated weights for policy 0, policy_version 1825394 (0.0009) [2023-12-27 04:41:35,311][105692] Updated weights for policy 0, policy_version 1825404 (0.0005) [2023-12-27 04:41:35,373][105692] Updated weights for policy 0, policy_version 1825414 (0.0009) [2023-12-27 04:41:35,427][105692] Updated weights for policy 0, policy_version 1825424 (0.0006) [2023-12-27 04:41:35,900][105620] Updated weights for policy 1, policy_version 1829404 (0.0010) [2023-12-27 04:41:35,968][105620] Updated weights for policy 1, policy_version 1829414 (0.0007) [2023-12-27 04:41:36,033][105692] Updated weights for policy 0, policy_version 1825434 (0.0005) [2023-12-27 04:41:36,035][105620] Updated weights for policy 1, policy_version 1829424 (0.0006) [2023-12-27 04:41:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19660.9). Total num frames: 935772160. Throughput: 0: 9639.5, 1: 10170.8. Samples: 935767908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:41:36,062][104569] Avg episode reward: [(0, '8260.876'), (1, '9258.412')] [2023-12-27 04:41:36,090][105692] Updated weights for policy 0, policy_version 1825444 (0.0006) [2023-12-27 04:41:36,158][105692] Updated weights for policy 0, policy_version 1825454 (0.0010) [2023-12-27 04:41:36,607][105620] Updated weights for policy 1, policy_version 1829434 (0.0010) [2023-12-27 04:41:36,675][105620] Updated weights for policy 1, policy_version 1829444 (0.0007) [2023-12-27 04:41:36,744][105620] Updated weights for policy 1, policy_version 1829454 (0.0008) [2023-12-27 04:41:36,798][105620] Updated weights for policy 1, policy_version 1829464 (0.0007) [2023-12-27 04:41:36,861][105692] Updated weights for policy 0, policy_version 1825464 (0.0010) [2023-12-27 04:41:36,926][105692] Updated weights for policy 0, policy_version 1825474 (0.0010) [2023-12-27 04:41:36,974][105692] Updated weights for policy 0, policy_version 1825484 (0.0010) [2023-12-27 04:41:37,350][105620] Updated weights for policy 1, policy_version 1829474 (0.0007) [2023-12-27 04:41:37,415][105620] Updated weights for policy 1, policy_version 1829484 (0.0010) [2023-12-27 04:41:37,482][105620] Updated weights for policy 1, policy_version 1829494 (0.0011) [2023-12-27 04:41:37,687][105692] Updated weights for policy 0, policy_version 1825494 (0.0008) [2023-12-27 04:41:37,747][105692] Updated weights for policy 0, policy_version 1825504 (0.0011) [2023-12-27 04:41:37,797][105692] Updated weights for policy 0, policy_version 1825514 (0.0011) [2023-12-27 04:41:38,094][105620] Updated weights for policy 1, policy_version 1829504 (0.0010) [2023-12-27 04:41:38,147][105620] Updated weights for policy 1, policy_version 1829514 (0.0010) [2023-12-27 04:41:38,200][105620] Updated weights for policy 1, policy_version 1829524 (0.0008) [2023-12-27 04:41:38,484][105692] Updated weights for policy 0, policy_version 1825524 (0.0009) [2023-12-27 04:41:38,546][105692] Updated weights for policy 0, policy_version 1825534 (0.0007) [2023-12-27 04:41:38,610][105692] Updated weights for policy 0, policy_version 1825544 (0.0010) [2023-12-27 04:41:38,828][105620] Updated weights for policy 1, policy_version 1829534 (0.0006) [2023-12-27 04:41:38,897][105620] Updated weights for policy 1, policy_version 1829544 (0.0006) [2023-12-27 04:41:38,961][105620] Updated weights for policy 1, policy_version 1829554 (0.0006) [2023-12-27 04:41:39,185][105692] Updated weights for policy 0, policy_version 1825554 (0.0009) [2023-12-27 04:41:39,257][105692] Updated weights for policy 0, policy_version 1825564 (0.0007) [2023-12-27 04:41:39,315][105692] Updated weights for policy 0, policy_version 1825574 (0.0008) [2023-12-27 04:41:39,384][105692] Updated weights for policy 0, policy_version 1825584 (0.0008) [2023-12-27 04:41:39,553][105620] Updated weights for policy 1, policy_version 1829564 (0.0008) [2023-12-27 04:41:39,605][105620] Updated weights for policy 1, policy_version 1829574 (0.0010) [2023-12-27 04:41:39,662][105620] Updated weights for policy 1, policy_version 1829584 (0.0011) [2023-12-27 04:41:40,163][105692] Updated weights for policy 0, policy_version 1825594 (0.0008) [2023-12-27 04:41:40,231][105692] Updated weights for policy 0, policy_version 1825604 (0.0007) [2023-12-27 04:41:40,299][105692] Updated weights for policy 0, policy_version 1825614 (0.0007) [2023-12-27 04:41:40,345][105620] Updated weights for policy 1, policy_version 1829594 (0.0007) [2023-12-27 04:41:40,410][105620] Updated weights for policy 1, policy_version 1829604 (0.0007) [2023-12-27 04:41:40,477][105620] Updated weights for policy 1, policy_version 1829614 (0.0006) [2023-12-27 04:41:40,547][105620] Updated weights for policy 1, policy_version 1829624 (0.0006) [2023-12-27 04:41:41,042][105692] Updated weights for policy 0, policy_version 1825624 (0.0010) [2023-12-27 04:41:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19688.6). Total num frames: 935878656. Throughput: 0: 9687.1, 1: 10250.5. Samples: 935893948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:41:41,063][104569] Avg episode reward: [(0, '8265.669'), (1, '9166.003')] [2023-12-27 04:41:41,084][105620] Updated weights for policy 1, policy_version 1829634 (0.0008) [2023-12-27 04:41:41,104][105692] Updated weights for policy 0, policy_version 1825634 (0.0012) [2023-12-27 04:41:41,147][105620] Updated weights for policy 1, policy_version 1829644 (0.0006) [2023-12-27 04:41:41,162][105692] Updated weights for policy 0, policy_version 1825644 (0.0010) [2023-12-27 04:41:41,220][105620] Updated weights for policy 1, policy_version 1829654 (0.0008) [2023-12-27 04:41:41,897][105692] Updated weights for policy 0, policy_version 1825654 (0.0009) [2023-12-27 04:41:41,947][105620] Updated weights for policy 1, policy_version 1829664 (0.0007) [2023-12-27 04:41:41,961][105692] Updated weights for policy 0, policy_version 1825664 (0.0011) [2023-12-27 04:41:42,010][105620] Updated weights for policy 1, policy_version 1829674 (0.0007) [2023-12-27 04:41:42,023][105692] Updated weights for policy 0, policy_version 1825674 (0.0011) [2023-12-27 04:41:42,077][105620] Updated weights for policy 1, policy_version 1829684 (0.0006) [2023-12-27 04:41:42,745][105692] Updated weights for policy 0, policy_version 1825684 (0.0010) [2023-12-27 04:41:42,780][105620] Updated weights for policy 1, policy_version 1829694 (0.0007) [2023-12-27 04:41:42,798][105692] Updated weights for policy 0, policy_version 1825694 (0.0011) [2023-12-27 04:41:42,832][105620] Updated weights for policy 1, policy_version 1829704 (0.0006) [2023-12-27 04:41:42,853][105692] Updated weights for policy 0, policy_version 1825704 (0.0010) [2023-12-27 04:41:42,890][105620] Updated weights for policy 1, policy_version 1829714 (0.0008) [2023-12-27 04:41:43,456][105620] Updated weights for policy 1, policy_version 1829724 (0.0006) [2023-12-27 04:41:43,509][105620] Updated weights for policy 1, policy_version 1829734 (0.0007) [2023-12-27 04:41:43,564][105620] Updated weights for policy 1, policy_version 1829744 (0.0010) [2023-12-27 04:41:43,592][105692] Updated weights for policy 0, policy_version 1825714 (0.0009) [2023-12-27 04:41:43,650][105692] Updated weights for policy 0, policy_version 1825724 (0.0010) [2023-12-27 04:41:43,715][105692] Updated weights for policy 0, policy_version 1825734 (0.0007) [2023-12-27 04:41:43,764][105692] Updated weights for policy 0, policy_version 1825744 (0.0008) [2023-12-27 04:41:44,240][105620] Updated weights for policy 1, policy_version 1829754 (0.0009) [2023-12-27 04:41:44,301][105620] Updated weights for policy 1, policy_version 1829764 (0.0006) [2023-12-27 04:41:44,319][105692] Updated weights for policy 0, policy_version 1825754 (0.0005) [2023-12-27 04:41:44,354][105620] Updated weights for policy 1, policy_version 1829774 (0.0005) [2023-12-27 04:41:44,370][105692] Updated weights for policy 0, policy_version 1825764 (0.0005) [2023-12-27 04:41:44,414][105620] Updated weights for policy 1, policy_version 1829784 (0.0006) [2023-12-27 04:41:44,424][105692] Updated weights for policy 0, policy_version 1825774 (0.0007) [2023-12-27 04:41:44,971][105620] Updated weights for policy 1, policy_version 1829794 (0.0006) [2023-12-27 04:41:45,039][105620] Updated weights for policy 1, policy_version 1829804 (0.0009) [2023-12-27 04:41:45,092][105620] Updated weights for policy 1, policy_version 1829814 (0.0011) [2023-12-27 04:41:45,133][105692] Updated weights for policy 0, policy_version 1825784 (0.0010) [2023-12-27 04:41:45,196][105692] Updated weights for policy 0, policy_version 1825794 (0.0010) [2023-12-27 04:41:45,265][105692] Updated weights for policy 0, policy_version 1825804 (0.0006) [2023-12-27 04:41:45,794][105620] Updated weights for policy 1, policy_version 1829824 (0.0006) [2023-12-27 04:41:45,856][105620] Updated weights for policy 1, policy_version 1829834 (0.0005) [2023-12-27 04:41:45,879][105692] Updated weights for policy 0, policy_version 1825814 (0.0010) [2023-12-27 04:41:45,912][105620] Updated weights for policy 1, policy_version 1829844 (0.0005) [2023-12-27 04:41:45,931][105692] Updated weights for policy 0, policy_version 1825824 (0.0010) [2023-12-27 04:41:45,978][105692] Updated weights for policy 0, policy_version 1825834 (0.0010) [2023-12-27 04:41:46,062][104569] Fps is (10 sec: 22117.6, 60 sec: 20070.3, 300 sec: 19716.3). Total num frames: 935993344. Throughput: 0: 9721.9, 1: 10164.0. Samples: 935953012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:41:46,063][104569] Avg episode reward: [(0, '8720.419'), (1, '9165.863')] [2023-12-27 04:41:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001829848_468508672.pth... [2023-12-27 04:41:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001825840_467484672.pth... [2023-12-27 04:41:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001828632_468197376.pth [2023-12-27 04:41:46,076][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_001829848_468508672.pth [2023-12-27 04:41:46,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001824688_467189760.pth [2023-12-27 04:41:46,079][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_001825840_467484672.pth [2023-12-27 04:41:46,514][105620] Updated weights for policy 1, policy_version 1829854 (0.0009) [2023-12-27 04:41:46,558][105692] Updated weights for policy 0, policy_version 1825844 (0.0007) [2023-12-27 04:41:46,579][105620] Updated weights for policy 1, policy_version 1829864 (0.0008) [2023-12-27 04:41:46,613][105692] Updated weights for policy 0, policy_version 1825854 (0.0006) [2023-12-27 04:41:46,640][105620] Updated weights for policy 1, policy_version 1829874 (0.0007) [2023-12-27 04:41:46,679][105692] Updated weights for policy 0, policy_version 1825864 (0.0010) [2023-12-27 04:41:47,313][105620] Updated weights for policy 1, policy_version 1829884 (0.0006) [2023-12-27 04:41:47,369][105620] Updated weights for policy 1, policy_version 1829894 (0.0005) [2023-12-27 04:41:47,375][105692] Updated weights for policy 0, policy_version 1825874 (0.0007) [2023-12-27 04:41:47,424][105620] Updated weights for policy 1, policy_version 1829904 (0.0005) [2023-12-27 04:41:47,437][105692] Updated weights for policy 0, policy_version 1825884 (0.0010) [2023-12-27 04:41:47,492][105692] Updated weights for policy 0, policy_version 1825894 (0.0010) [2023-12-27 04:41:47,548][105692] Updated weights for policy 0, policy_version 1825904 (0.0010) [2023-12-27 04:41:48,091][105620] Updated weights for policy 1, policy_version 1829914 (0.0006) [2023-12-27 04:41:48,140][105620] Updated weights for policy 1, policy_version 1829924 (0.0008) [2023-12-27 04:41:48,186][105620] Updated weights for policy 1, policy_version 1829934 (0.0008) [2023-12-27 04:41:48,237][105620] Updated weights for policy 1, policy_version 1829944 (0.0009) [2023-12-27 04:41:48,259][105692] Updated weights for policy 0, policy_version 1825914 (0.0005) [2023-12-27 04:41:48,318][105692] Updated weights for policy 0, policy_version 1825924 (0.0007) [2023-12-27 04:41:48,381][105692] Updated weights for policy 0, policy_version 1825934 (0.0007) [2023-12-27 04:41:49,007][105692] Updated weights for policy 0, policy_version 1825944 (0.0009) [2023-12-27 04:41:49,072][105692] Updated weights for policy 0, policy_version 1825954 (0.0010) [2023-12-27 04:41:49,078][105620] Updated weights for policy 1, policy_version 1829954 (0.0005) [2023-12-27 04:41:49,129][105620] Updated weights for policy 1, policy_version 1829964 (0.0006) [2023-12-27 04:41:49,130][105692] Updated weights for policy 0, policy_version 1825964 (0.0010) [2023-12-27 04:41:49,176][105620] Updated weights for policy 1, policy_version 1829974 (0.0008) [2023-12-27 04:41:49,847][105620] Updated weights for policy 1, policy_version 1829984 (0.0010) [2023-12-27 04:41:49,900][105620] Updated weights for policy 1, policy_version 1829994 (0.0011) [2023-12-27 04:41:49,927][105692] Updated weights for policy 0, policy_version 1825974 (0.0008) [2023-12-27 04:41:49,966][105620] Updated weights for policy 1, policy_version 1830004 (0.0011) [2023-12-27 04:41:49,993][105692] Updated weights for policy 0, policy_version 1825984 (0.0007) [2023-12-27 04:41:50,052][105692] Updated weights for policy 0, policy_version 1825994 (0.0009) [2023-12-27 04:41:50,720][105620] Updated weights for policy 1, policy_version 1830014 (0.0011) [2023-12-27 04:41:50,775][105620] Updated weights for policy 1, policy_version 1830024 (0.0011) [2023-12-27 04:41:50,819][105692] Updated weights for policy 0, policy_version 1826004 (0.0009) [2023-12-27 04:41:50,831][105620] Updated weights for policy 1, policy_version 1830034 (0.0011) [2023-12-27 04:41:50,871][105692] Updated weights for policy 0, policy_version 1826014 (0.0010) [2023-12-27 04:41:50,923][105692] Updated weights for policy 0, policy_version 1826024 (0.0010) [2023-12-27 04:41:51,062][104569] Fps is (10 sec: 21299.3, 60 sec: 19933.8, 300 sec: 19716.3). Total num frames: 936091648. Throughput: 0: 9824.6, 1: 10176.9. Samples: 936077376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:41:51,062][104569] Avg episode reward: [(0, '8811.378'), (1, '9165.892')] [2023-12-27 04:41:51,657][105692] Updated weights for policy 0, policy_version 1826034 (0.0009) [2023-12-27 04:41:51,701][105620] Updated weights for policy 1, policy_version 1830044 (0.0010) [2023-12-27 04:41:51,726][105692] Updated weights for policy 0, policy_version 1826044 (0.0008) [2023-12-27 04:41:51,767][105620] Updated weights for policy 1, policy_version 1830054 (0.0009) [2023-12-27 04:41:51,789][105692] Updated weights for policy 0, policy_version 1826054 (0.0006) [2023-12-27 04:41:51,831][105620] Updated weights for policy 1, policy_version 1830064 (0.0007) [2023-12-27 04:41:51,853][105692] Updated weights for policy 0, policy_version 1826064 (0.0008) [2023-12-27 04:41:52,543][105620] Updated weights for policy 1, policy_version 1830074 (0.0007) [2023-12-27 04:41:52,574][105692] Updated weights for policy 0, policy_version 1826074 (0.0007) [2023-12-27 04:41:52,605][105620] Updated weights for policy 1, policy_version 1830084 (0.0008) [2023-12-27 04:41:52,636][105692] Updated weights for policy 0, policy_version 1826084 (0.0007) [2023-12-27 04:41:52,658][105620] Updated weights for policy 1, policy_version 1830094 (0.0008) [2023-12-27 04:41:52,693][105692] Updated weights for policy 0, policy_version 1826094 (0.0006) [2023-12-27 04:41:52,721][105620] Updated weights for policy 1, policy_version 1830104 (0.0008) [2023-12-27 04:41:53,346][105620] Updated weights for policy 1, policy_version 1830114 (0.0005) [2023-12-27 04:41:53,410][105620] Updated weights for policy 1, policy_version 1830124 (0.0009) [2023-12-27 04:41:53,456][105692] Updated weights for policy 0, policy_version 1826104 (0.0006) [2023-12-27 04:41:53,468][105620] Updated weights for policy 1, policy_version 1830134 (0.0009) [2023-12-27 04:41:53,505][105692] Updated weights for policy 0, policy_version 1826114 (0.0005) [2023-12-27 04:41:53,565][105692] Updated weights for policy 0, policy_version 1826124 (0.0005) [2023-12-27 04:41:54,028][105620] Updated weights for policy 1, policy_version 1830144 (0.0010) [2023-12-27 04:41:54,094][105620] Updated weights for policy 1, policy_version 1830154 (0.0010) [2023-12-27 04:41:54,124][105692] Updated weights for policy 0, policy_version 1826134 (0.0007) [2023-12-27 04:41:54,143][105620] Updated weights for policy 1, policy_version 1830164 (0.0010) [2023-12-27 04:41:54,183][105692] Updated weights for policy 0, policy_version 1826144 (0.0007) [2023-12-27 04:41:54,241][105692] Updated weights for policy 0, policy_version 1826154 (0.0010) [2023-12-27 04:41:54,762][105620] Updated weights for policy 1, policy_version 1830174 (0.0011) [2023-12-27 04:41:54,810][105620] Updated weights for policy 1, policy_version 1830184 (0.0010) [2023-12-27 04:41:54,863][105620] Updated weights for policy 1, policy_version 1830194 (0.0010) [2023-12-27 04:41:55,088][105692] Updated weights for policy 0, policy_version 1826164 (0.0009) [2023-12-27 04:41:55,148][105692] Updated weights for policy 0, policy_version 1826174 (0.0008) [2023-12-27 04:41:55,200][105692] Updated weights for policy 0, policy_version 1826184 (0.0008) [2023-12-27 04:41:55,623][105620] Updated weights for policy 1, policy_version 1830204 (0.0010) [2023-12-27 04:41:55,685][105620] Updated weights for policy 1, policy_version 1830214 (0.0010) [2023-12-27 04:41:55,740][105620] Updated weights for policy 1, policy_version 1830224 (0.0010) [2023-12-27 04:41:55,977][105692] Updated weights for policy 0, policy_version 1826194 (0.0008) [2023-12-27 04:41:56,041][105692] Updated weights for policy 0, policy_version 1826204 (0.0008) [2023-12-27 04:41:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19933.8, 300 sec: 19716.3). Total num frames: 936181760. Throughput: 0: 9798.9, 1: 10251.3. Samples: 936194324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:41:56,064][104569] Avg episode reward: [(0, '8900.338'), (1, '9258.137')] [2023-12-27 04:41:56,102][105692] Updated weights for policy 0, policy_version 1826214 (0.0007) [2023-12-27 04:41:56,164][105692] Updated weights for policy 0, policy_version 1826224 (0.0005) [2023-12-27 04:41:56,355][105620] Updated weights for policy 1, policy_version 1830234 (0.0008) [2023-12-27 04:41:56,427][105620] Updated weights for policy 1, policy_version 1830244 (0.0011) [2023-12-27 04:41:56,497][105620] Updated weights for policy 1, policy_version 1830254 (0.0010) [2023-12-27 04:41:56,559][105620] Updated weights for policy 1, policy_version 1830264 (0.0010) [2023-12-27 04:41:56,677][105692] Updated weights for policy 0, policy_version 1826234 (0.0009) [2023-12-27 04:41:56,725][105692] Updated weights for policy 0, policy_version 1826244 (0.0008) [2023-12-27 04:41:56,770][105692] Updated weights for policy 0, policy_version 1826254 (0.0008) [2023-12-27 04:41:57,235][105620] Updated weights for policy 1, policy_version 1830274 (0.0009) [2023-12-27 04:41:57,283][105620] Updated weights for policy 1, policy_version 1830284 (0.0009) [2023-12-27 04:41:57,338][105620] Updated weights for policy 1, policy_version 1830294 (0.0009) [2023-12-27 04:41:57,572][105692] Updated weights for policy 0, policy_version 1826264 (0.0008) [2023-12-27 04:41:57,619][105692] Updated weights for policy 0, policy_version 1826274 (0.0008) [2023-12-27 04:41:57,671][105692] Updated weights for policy 0, policy_version 1826284 (0.0005) [2023-12-27 04:41:58,089][105620] Updated weights for policy 1, policy_version 1830304 (0.0006) [2023-12-27 04:41:58,138][105620] Updated weights for policy 1, policy_version 1830314 (0.0006) [2023-12-27 04:41:58,205][105620] Updated weights for policy 1, policy_version 1830324 (0.0007) [2023-12-27 04:41:58,389][105692] Updated weights for policy 0, policy_version 1826294 (0.0007) [2023-12-27 04:41:58,455][105692] Updated weights for policy 0, policy_version 1826304 (0.0009) [2023-12-27 04:41:58,515][105692] Updated weights for policy 0, policy_version 1826314 (0.0010) [2023-12-27 04:41:59,024][105620] Updated weights for policy 1, policy_version 1830334 (0.0008) [2023-12-27 04:41:59,080][105620] Updated weights for policy 1, policy_version 1830344 (0.0009) [2023-12-27 04:41:59,145][105620] Updated weights for policy 1, policy_version 1830354 (0.0008) [2023-12-27 04:41:59,311][105692] Updated weights for policy 0, policy_version 1826324 (0.0009) [2023-12-27 04:41:59,375][105692] Updated weights for policy 0, policy_version 1826334 (0.0009) [2023-12-27 04:41:59,423][105692] Updated weights for policy 0, policy_version 1826344 (0.0010) [2023-12-27 04:41:59,978][105620] Updated weights for policy 1, policy_version 1830364 (0.0008) [2023-12-27 04:42:00,047][105620] Updated weights for policy 1, policy_version 1830374 (0.0006) [2023-12-27 04:42:00,104][105620] Updated weights for policy 1, policy_version 1830384 (0.0005) [2023-12-27 04:42:00,177][105692] Updated weights for policy 0, policy_version 1826354 (0.0010) [2023-12-27 04:42:00,235][105692] Updated weights for policy 0, policy_version 1826364 (0.0010) [2023-12-27 04:42:00,282][105692] Updated weights for policy 0, policy_version 1826374 (0.0010) [2023-12-27 04:42:00,330][105692] Updated weights for policy 0, policy_version 1826384 (0.0010) [2023-12-27 04:42:00,781][105620] Updated weights for policy 1, policy_version 1830394 (0.0006) [2023-12-27 04:42:00,828][105620] Updated weights for policy 1, policy_version 1830404 (0.0007) [2023-12-27 04:42:00,872][105620] Updated weights for policy 1, policy_version 1830414 (0.0008) [2023-12-27 04:42:00,917][105620] Updated weights for policy 1, policy_version 1830424 (0.0008) [2023-12-27 04:42:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19933.8, 300 sec: 19688.6). Total num frames: 936280064. Throughput: 0: 9813.7, 1: 10245.4. Samples: 936253116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:42:01,063][104569] Avg episode reward: [(0, '8807.316'), (1, '9165.857')] [2023-12-27 04:42:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001830424_468656128.pth... [2023-12-27 04:42:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001829240_468353024.pth [2023-12-27 04:42:01,094][105692] Updated weights for policy 0, policy_version 1826394 (0.0011) [2023-12-27 04:42:01,156][105692] Updated weights for policy 0, policy_version 1826404 (0.0010) [2023-12-27 04:42:01,214][105692] Updated weights for policy 0, policy_version 1826414 (0.0009) [2023-12-27 04:42:01,225][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001826416_467632128.pth... [2023-12-27 04:42:01,229][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001825232_467329024.pth [2023-12-27 04:42:01,698][105620] Updated weights for policy 1, policy_version 1830434 (0.0008) [2023-12-27 04:42:01,762][105620] Updated weights for policy 1, policy_version 1830444 (0.0010) [2023-12-27 04:42:01,814][105620] Updated weights for policy 1, policy_version 1830454 (0.0009) [2023-12-27 04:42:01,895][105692] Updated weights for policy 0, policy_version 1826424 (0.0008) [2023-12-27 04:42:01,942][105692] Updated weights for policy 0, policy_version 1826434 (0.0005) [2023-12-27 04:42:02,005][105692] Updated weights for policy 0, policy_version 1826444 (0.0006) [2023-12-27 04:42:02,591][105692] Updated weights for policy 0, policy_version 1826454 (0.0009) [2023-12-27 04:42:02,648][105620] Updated weights for policy 1, policy_version 1830464 (0.0006) [2023-12-27 04:42:02,650][105692] Updated weights for policy 0, policy_version 1826464 (0.0011) [2023-12-27 04:42:02,703][105692] Updated weights for policy 0, policy_version 1826474 (0.0010) [2023-12-27 04:42:02,708][105620] Updated weights for policy 1, policy_version 1830474 (0.0006) [2023-12-27 04:42:02,754][105620] Updated weights for policy 1, policy_version 1830484 (0.0005) [2023-12-27 04:42:03,276][105692] Updated weights for policy 0, policy_version 1826484 (0.0005) [2023-12-27 04:42:03,322][105692] Updated weights for policy 0, policy_version 1826494 (0.0008) [2023-12-27 04:42:03,364][105692] Updated weights for policy 0, policy_version 1826504 (0.0007) [2023-12-27 04:42:03,430][105620] Updated weights for policy 1, policy_version 1830494 (0.0008) [2023-12-27 04:42:03,482][105620] Updated weights for policy 1, policy_version 1830504 (0.0008) [2023-12-27 04:42:03,542][105620] Updated weights for policy 1, policy_version 1830514 (0.0009) [2023-12-27 04:42:04,066][105692] Updated weights for policy 0, policy_version 1826514 (0.0006) [2023-12-27 04:42:04,126][105692] Updated weights for policy 0, policy_version 1826524 (0.0009) [2023-12-27 04:42:04,186][105692] Updated weights for policy 0, policy_version 1826534 (0.0008) [2023-12-27 04:42:04,194][105620] Updated weights for policy 1, policy_version 1830524 (0.0009) [2023-12-27 04:42:04,245][105692] Updated weights for policy 0, policy_version 1826544 (0.0007) [2023-12-27 04:42:04,260][105620] Updated weights for policy 1, policy_version 1830534 (0.0007) [2023-12-27 04:42:04,323][105620] Updated weights for policy 1, policy_version 1830544 (0.0009) [2023-12-27 04:42:04,895][105692] Updated weights for policy 0, policy_version 1826554 (0.0006) [2023-12-27 04:42:04,954][105692] Updated weights for policy 0, policy_version 1826564 (0.0008) [2023-12-27 04:42:05,009][105692] Updated weights for policy 0, policy_version 1826574 (0.0011) [2023-12-27 04:42:05,096][105620] Updated weights for policy 1, policy_version 1830554 (0.0009) [2023-12-27 04:42:05,151][105620] Updated weights for policy 1, policy_version 1830564 (0.0005) [2023-12-27 04:42:05,197][105620] Updated weights for policy 1, policy_version 1830574 (0.0005) [2023-12-27 04:42:05,246][105620] Updated weights for policy 1, policy_version 1830584 (0.0005) [2023-12-27 04:42:05,716][105692] Updated weights for policy 0, policy_version 1826584 (0.0010) [2023-12-27 04:42:05,768][105692] Updated weights for policy 0, policy_version 1826594 (0.0010) [2023-12-27 04:42:05,777][105620] Updated weights for policy 1, policy_version 1830594 (0.0006) [2023-12-27 04:42:05,826][105692] Updated weights for policy 0, policy_version 1826604 (0.0010) [2023-12-27 04:42:05,835][105620] Updated weights for policy 1, policy_version 1830604 (0.0010) [2023-12-27 04:42:05,899][105620] Updated weights for policy 1, policy_version 1830614 (0.0010) [2023-12-27 04:42:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19933.8, 300 sec: 19716.3). Total num frames: 936386560. Throughput: 0: 9870.2, 1: 10132.0. Samples: 936370008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:42:06,063][104569] Avg episode reward: [(0, '8539.350'), (1, '9073.176')] [2023-12-27 04:42:06,580][105692] Updated weights for policy 0, policy_version 1826614 (0.0011) [2023-12-27 04:42:06,610][105620] Updated weights for policy 1, policy_version 1830624 (0.0008) [2023-12-27 04:42:06,629][105692] Updated weights for policy 0, policy_version 1826624 (0.0011) [2023-12-27 04:42:06,664][105620] Updated weights for policy 1, policy_version 1830634 (0.0005) [2023-12-27 04:42:06,679][105692] Updated weights for policy 0, policy_version 1826634 (0.0011) [2023-12-27 04:42:06,713][105620] Updated weights for policy 1, policy_version 1830644 (0.0005) [2023-12-27 04:42:07,370][105692] Updated weights for policy 0, policy_version 1826644 (0.0011) [2023-12-27 04:42:07,454][105692] Updated weights for policy 0, policy_version 1826654 (0.0011) [2023-12-27 04:42:07,492][105620] Updated weights for policy 1, policy_version 1830654 (0.0007) [2023-12-27 04:42:07,507][105692] Updated weights for policy 0, policy_version 1826664 (0.0011) [2023-12-27 04:42:07,544][105620] Updated weights for policy 1, policy_version 1830664 (0.0005) [2023-12-27 04:42:07,609][105620] Updated weights for policy 1, policy_version 1830674 (0.0009) [2023-12-27 04:42:08,065][105692] Updated weights for policy 0, policy_version 1826674 (0.0009) [2023-12-27 04:42:08,124][105692] Updated weights for policy 0, policy_version 1826684 (0.0005) [2023-12-27 04:42:08,186][105692] Updated weights for policy 0, policy_version 1826694 (0.0010) [2023-12-27 04:42:08,246][105692] Updated weights for policy 0, policy_version 1826704 (0.0011) [2023-12-27 04:42:08,317][105620] Updated weights for policy 1, policy_version 1830684 (0.0009) [2023-12-27 04:42:08,377][105620] Updated weights for policy 1, policy_version 1830695 (0.0008) [2023-12-27 04:42:08,426][105620] Updated weights for policy 1, policy_version 1830705 (0.0008) [2023-12-27 04:42:08,913][105692] Updated weights for policy 0, policy_version 1826714 (0.0008) [2023-12-27 04:42:08,966][105692] Updated weights for policy 0, policy_version 1826724 (0.0007) [2023-12-27 04:42:09,025][105692] Updated weights for policy 0, policy_version 1826734 (0.0005) [2023-12-27 04:42:09,105][105620] Updated weights for policy 1, policy_version 1830715 (0.0009) [2023-12-27 04:42:09,166][105620] Updated weights for policy 1, policy_version 1830725 (0.0008) [2023-12-27 04:42:09,231][105620] Updated weights for policy 1, policy_version 1830735 (0.0009) [2023-12-27 04:42:09,752][105692] Updated weights for policy 0, policy_version 1826744 (0.0008) [2023-12-27 04:42:09,825][105692] Updated weights for policy 0, policy_version 1826754 (0.0010) [2023-12-27 04:42:09,881][105692] Updated weights for policy 0, policy_version 1826764 (0.0007) [2023-12-27 04:42:10,006][105620] Updated weights for policy 1, policy_version 1830745 (0.0009) [2023-12-27 04:42:10,065][105620] Updated weights for policy 1, policy_version 1830755 (0.0008) [2023-12-27 04:42:10,125][105620] Updated weights for policy 1, policy_version 1830765 (0.0008) [2023-12-27 04:42:10,184][105620] Updated weights for policy 1, policy_version 1830775 (0.0008) [2023-12-27 04:42:10,594][105692] Updated weights for policy 0, policy_version 1826774 (0.0008) [2023-12-27 04:42:10,646][105692] Updated weights for policy 0, policy_version 1826784 (0.0008) [2023-12-27 04:42:10,700][105692] Updated weights for policy 0, policy_version 1826794 (0.0008) [2023-12-27 04:42:10,940][105620] Updated weights for policy 1, policy_version 1830785 (0.0010) [2023-12-27 04:42:10,995][105620] Updated weights for policy 1, policy_version 1830795 (0.0010) [2023-12-27 04:42:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19744.1). Total num frames: 936476672. Throughput: 0: 10063.7, 1: 10050.7. Samples: 936489536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:42:11,062][105620] Updated weights for policy 1, policy_version 1830805 (0.0011) [2023-12-27 04:42:11,063][104569] Avg episode reward: [(0, '8174.611'), (1, '9165.230')] [2023-12-27 04:42:11,370][105692] Updated weights for policy 0, policy_version 1826804 (0.0008) [2023-12-27 04:42:11,436][105692] Updated weights for policy 0, policy_version 1826814 (0.0009) [2023-12-27 04:42:11,488][105692] Updated weights for policy 0, policy_version 1826824 (0.0011) [2023-12-27 04:42:11,758][105620] Updated weights for policy 1, policy_version 1830815 (0.0008) [2023-12-27 04:42:11,821][105620] Updated weights for policy 1, policy_version 1830825 (0.0010) [2023-12-27 04:42:11,873][105620] Updated weights for policy 1, policy_version 1830835 (0.0010) [2023-12-27 04:42:12,176][105692] Updated weights for policy 0, policy_version 1826834 (0.0010) [2023-12-27 04:42:12,235][105692] Updated weights for policy 0, policy_version 1826844 (0.0008) [2023-12-27 04:42:12,300][105692] Updated weights for policy 0, policy_version 1826854 (0.0009) [2023-12-27 04:42:12,371][105692] Updated weights for policy 0, policy_version 1826864 (0.0008) [2023-12-27 04:42:12,659][105620] Updated weights for policy 1, policy_version 1830845 (0.0009) [2023-12-27 04:42:12,711][105620] Updated weights for policy 1, policy_version 1830855 (0.0009) [2023-12-27 04:42:12,772][105620] Updated weights for policy 1, policy_version 1830865 (0.0008) [2023-12-27 04:42:13,217][105692] Updated weights for policy 0, policy_version 1826874 (0.0009) [2023-12-27 04:42:13,274][105692] Updated weights for policy 0, policy_version 1826884 (0.0009) [2023-12-27 04:42:13,331][105692] Updated weights for policy 0, policy_version 1826894 (0.0009) [2023-12-27 04:42:13,464][105620] Updated weights for policy 1, policy_version 1830875 (0.0006) [2023-12-27 04:42:13,514][105620] Updated weights for policy 1, policy_version 1830885 (0.0009) [2023-12-27 04:42:13,573][105620] Updated weights for policy 1, policy_version 1830895 (0.0008) [2023-12-27 04:42:14,130][105692] Updated weights for policy 0, policy_version 1826904 (0.0009) [2023-12-27 04:42:14,185][105692] Updated weights for policy 0, policy_version 1826914 (0.0007) [2023-12-27 04:42:14,232][105692] Updated weights for policy 0, policy_version 1826924 (0.0008) [2023-12-27 04:42:14,256][105620] Updated weights for policy 1, policy_version 1830905 (0.0007) [2023-12-27 04:42:14,305][105620] Updated weights for policy 1, policy_version 1830915 (0.0005) [2023-12-27 04:42:14,358][105620] Updated weights for policy 1, policy_version 1830925 (0.0005) [2023-12-27 04:42:14,414][105620] Updated weights for policy 1, policy_version 1830935 (0.0007) [2023-12-27 04:42:15,036][105692] Updated weights for policy 0, policy_version 1826934 (0.0008) [2023-12-27 04:42:15,095][105692] Updated weights for policy 0, policy_version 1826944 (0.0009) [2023-12-27 04:42:15,106][105620] Updated weights for policy 1, policy_version 1830945 (0.0006) [2023-12-27 04:42:15,154][105692] Updated weights for policy 0, policy_version 1826954 (0.0009) [2023-12-27 04:42:15,167][105620] Updated weights for policy 1, policy_version 1830955 (0.0006) [2023-12-27 04:42:15,236][105620] Updated weights for policy 1, policy_version 1830965 (0.0005) [2023-12-27 04:42:15,866][105620] Updated weights for policy 1, policy_version 1830975 (0.0008) [2023-12-27 04:42:15,902][105692] Updated weights for policy 0, policy_version 1826964 (0.0009) [2023-12-27 04:42:15,912][105620] Updated weights for policy 1, policy_version 1830985 (0.0006) [2023-12-27 04:42:15,961][105620] Updated weights for policy 1, policy_version 1830995 (0.0005) [2023-12-27 04:42:15,963][105692] Updated weights for policy 0, policy_version 1826974 (0.0008) [2023-12-27 04:42:16,022][105692] Updated weights for policy 0, policy_version 1826984 (0.0007) [2023-12-27 04:42:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19933.8, 300 sec: 19716.3). Total num frames: 936574976. Throughput: 0: 10030.5, 1: 9979.3. Samples: 936546788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:42:16,063][104569] Avg episode reward: [(0, '8624.298'), (1, '9257.770')] [2023-12-27 04:42:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001831000_468803584.pth... [2023-12-27 04:42:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001826992_467779584.pth... [2023-12-27 04:42:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001825840_467484672.pth [2023-12-27 04:42:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001829848_468508672.pth [2023-12-27 04:42:16,708][105692] Updated weights for policy 0, policy_version 1826994 (0.0007) [2023-12-27 04:42:16,723][105620] Updated weights for policy 1, policy_version 1831005 (0.0006) [2023-12-27 04:42:16,766][105692] Updated weights for policy 0, policy_version 1827004 (0.0010) [2023-12-27 04:42:16,793][105620] Updated weights for policy 1, policy_version 1831015 (0.0005) [2023-12-27 04:42:16,820][105692] Updated weights for policy 0, policy_version 1827014 (0.0007) [2023-12-27 04:42:16,858][105620] Updated weights for policy 1, policy_version 1831025 (0.0005) [2023-12-27 04:42:16,873][105692] Updated weights for policy 0, policy_version 1827024 (0.0007) [2023-12-27 04:42:17,460][105620] Updated weights for policy 1, policy_version 1831035 (0.0008) [2023-12-27 04:42:17,522][105620] Updated weights for policy 1, policy_version 1831045 (0.0009) [2023-12-27 04:42:17,584][105620] Updated weights for policy 1, policy_version 1831055 (0.0007) [2023-12-27 04:42:17,617][105692] Updated weights for policy 0, policy_version 1827034 (0.0009) [2023-12-27 04:42:17,668][105692] Updated weights for policy 0, policy_version 1827044 (0.0009) [2023-12-27 04:42:17,724][105692] Updated weights for policy 0, policy_version 1827054 (0.0009) [2023-12-27 04:42:18,203][105620] Updated weights for policy 1, policy_version 1831065 (0.0006) [2023-12-27 04:42:18,254][105620] Updated weights for policy 1, policy_version 1831075 (0.0009) [2023-12-27 04:42:18,306][105620] Updated weights for policy 1, policy_version 1831085 (0.0010) [2023-12-27 04:42:18,369][105620] Updated weights for policy 1, policy_version 1831095 (0.0009) [2023-12-27 04:42:18,592][105692] Updated weights for policy 0, policy_version 1827064 (0.0009) [2023-12-27 04:42:18,660][105692] Updated weights for policy 0, policy_version 1827074 (0.0006) [2023-12-27 04:42:18,722][105692] Updated weights for policy 0, policy_version 1827084 (0.0007) [2023-12-27 04:42:19,087][105620] Updated weights for policy 1, policy_version 1831105 (0.0009) [2023-12-27 04:42:19,144][105620] Updated weights for policy 1, policy_version 1831115 (0.0009) [2023-12-27 04:42:19,198][105620] Updated weights for policy 1, policy_version 1831125 (0.0008) [2023-12-27 04:42:19,447][105692] Updated weights for policy 0, policy_version 1827094 (0.0010) [2023-12-27 04:42:19,504][105692] Updated weights for policy 0, policy_version 1827105 (0.0010) [2023-12-27 04:42:19,563][105692] Updated weights for policy 0, policy_version 1827115 (0.0008) [2023-12-27 04:42:19,886][105620] Updated weights for policy 1, policy_version 1831135 (0.0009) [2023-12-27 04:42:19,949][105620] Updated weights for policy 1, policy_version 1831145 (0.0008) [2023-12-27 04:42:20,015][105620] Updated weights for policy 1, policy_version 1831155 (0.0009) [2023-12-27 04:42:20,387][105692] Updated weights for policy 0, policy_version 1827125 (0.0009) [2023-12-27 04:42:20,451][105692] Updated weights for policy 0, policy_version 1827135 (0.0009) [2023-12-27 04:42:20,507][105692] Updated weights for policy 0, policy_version 1827145 (0.0009) [2023-12-27 04:42:20,765][105620] Updated weights for policy 1, policy_version 1831165 (0.0009) [2023-12-27 04:42:20,812][105620] Updated weights for policy 1, policy_version 1831175 (0.0009) [2023-12-27 04:42:20,859][105620] Updated weights for policy 1, policy_version 1831185 (0.0009) [2023-12-27 04:42:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.8, 300 sec: 19688.6). Total num frames: 936673280. Throughput: 0: 9828.0, 1: 10062.6. Samples: 936662984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:42:21,063][104569] Avg episode reward: [(0, '8990.877'), (1, '9258.074')] [2023-12-27 04:42:21,299][105692] Updated weights for policy 0, policy_version 1827155 (0.0009) [2023-12-27 04:42:21,367][105692] Updated weights for policy 0, policy_version 1827165 (0.0008) [2023-12-27 04:42:21,429][105692] Updated weights for policy 0, policy_version 1827175 (0.0009) [2023-12-27 04:42:21,742][105620] Updated weights for policy 1, policy_version 1831195 (0.0009) [2023-12-27 04:42:21,806][105620] Updated weights for policy 1, policy_version 1831205 (0.0007) [2023-12-27 04:42:21,868][105620] Updated weights for policy 1, policy_version 1831215 (0.0006) [2023-12-27 04:42:22,167][105692] Updated weights for policy 0, policy_version 1827185 (0.0009) [2023-12-27 04:42:22,230][105692] Updated weights for policy 0, policy_version 1827195 (0.0008) [2023-12-27 04:42:22,293][105692] Updated weights for policy 0, policy_version 1827205 (0.0006) [2023-12-27 04:42:22,364][105692] Updated weights for policy 0, policy_version 1827215 (0.0009) [2023-12-27 04:42:22,569][105620] Updated weights for policy 1, policy_version 1831225 (0.0007) [2023-12-27 04:42:22,627][105620] Updated weights for policy 1, policy_version 1831235 (0.0009) [2023-12-27 04:42:22,682][105620] Updated weights for policy 1, policy_version 1831245 (0.0010) [2023-12-27 04:42:22,743][105620] Updated weights for policy 1, policy_version 1831255 (0.0010) [2023-12-27 04:42:23,040][105692] Updated weights for policy 0, policy_version 1827225 (0.0008) [2023-12-27 04:42:23,089][105692] Updated weights for policy 0, policy_version 1827235 (0.0009) [2023-12-27 04:42:23,148][105692] Updated weights for policy 0, policy_version 1827245 (0.0005) [2023-12-27 04:42:23,496][105620] Updated weights for policy 1, policy_version 1831265 (0.0009) [2023-12-27 04:42:23,551][105620] Updated weights for policy 1, policy_version 1831275 (0.0009) [2023-12-27 04:42:23,606][105620] Updated weights for policy 1, policy_version 1831285 (0.0010) [2023-12-27 04:42:23,826][105692] Updated weights for policy 0, policy_version 1827255 (0.0005) [2023-12-27 04:42:23,877][105692] Updated weights for policy 0, policy_version 1827265 (0.0005) [2023-12-27 04:42:23,943][105692] Updated weights for policy 0, policy_version 1827275 (0.0005) [2023-12-27 04:42:24,444][105620] Updated weights for policy 1, policy_version 1831295 (0.0009) [2023-12-27 04:42:24,498][105620] Updated weights for policy 1, policy_version 1831306 (0.0010) [2023-12-27 04:42:24,538][105692] Updated weights for policy 0, policy_version 1827285 (0.0007) [2023-12-27 04:42:24,548][105620] Updated weights for policy 1, policy_version 1831316 (0.0009) [2023-12-27 04:42:24,589][105692] Updated weights for policy 0, policy_version 1827295 (0.0005) [2023-12-27 04:42:24,648][105692] Updated weights for policy 0, policy_version 1827305 (0.0005) [2023-12-27 04:42:25,255][105692] Updated weights for policy 0, policy_version 1827315 (0.0007) [2023-12-27 04:42:25,317][105692] Updated weights for policy 0, policy_version 1827325 (0.0011) [2023-12-27 04:42:25,366][105620] Updated weights for policy 1, policy_version 1831326 (0.0009) [2023-12-27 04:42:25,374][105692] Updated weights for policy 0, policy_version 1827335 (0.0011) [2023-12-27 04:42:25,426][105620] Updated weights for policy 1, policy_version 1831336 (0.0007) [2023-12-27 04:42:25,479][105620] Updated weights for policy 1, policy_version 1831346 (0.0007) [2023-12-27 04:42:26,006][105692] Updated weights for policy 0, policy_version 1827345 (0.0010) [2023-12-27 04:42:26,060][105692] Updated weights for policy 0, policy_version 1827355 (0.0005) [2023-12-27 04:42:26,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 936763392. Throughput: 0: 9798.1, 1: 9841.3. Samples: 936777720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:42:26,062][104569] Avg episode reward: [(0, '8444.247'), (1, '9258.049')] [2023-12-27 04:42:26,101][105620] Updated weights for policy 1, policy_version 1831356 (0.0008) [2023-12-27 04:42:26,108][105692] Updated weights for policy 0, policy_version 1827365 (0.0005) [2023-12-27 04:42:26,150][105620] Updated weights for policy 1, policy_version 1831366 (0.0009) [2023-12-27 04:42:26,159][105692] Updated weights for policy 0, policy_version 1827375 (0.0005) [2023-12-27 04:42:26,201][105620] Updated weights for policy 1, policy_version 1831376 (0.0009) [2023-12-27 04:42:26,721][105692] Updated weights for policy 0, policy_version 1827385 (0.0008) [2023-12-27 04:42:26,765][105692] Updated weights for policy 0, policy_version 1827395 (0.0005) [2023-12-27 04:42:26,816][105692] Updated weights for policy 0, policy_version 1827405 (0.0005) [2023-12-27 04:42:27,069][105620] Updated weights for policy 1, policy_version 1831387 (0.0009) [2023-12-27 04:42:27,124][105620] Updated weights for policy 1, policy_version 1831397 (0.0008) [2023-12-27 04:42:27,168][105620] Updated weights for policy 1, policy_version 1831407 (0.0008) [2023-12-27 04:42:27,511][105692] Updated weights for policy 0, policy_version 1827415 (0.0009) [2023-12-27 04:42:27,559][105692] Updated weights for policy 0, policy_version 1827425 (0.0010) [2023-12-27 04:42:27,607][105692] Updated weights for policy 0, policy_version 1827435 (0.0010) [2023-12-27 04:42:27,933][105620] Updated weights for policy 1, policy_version 1831417 (0.0008) [2023-12-27 04:42:27,998][105620] Updated weights for policy 1, policy_version 1831427 (0.0010) [2023-12-27 04:42:28,066][105620] Updated weights for policy 1, policy_version 1831437 (0.0010) [2023-12-27 04:42:28,118][105620] Updated weights for policy 1, policy_version 1831447 (0.0010) [2023-12-27 04:42:28,274][105692] Updated weights for policy 0, policy_version 1827445 (0.0008) [2023-12-27 04:42:28,337][105692] Updated weights for policy 0, policy_version 1827455 (0.0008) [2023-12-27 04:42:28,398][105692] Updated weights for policy 0, policy_version 1827465 (0.0008) [2023-12-27 04:42:28,847][105620] Updated weights for policy 1, policy_version 1831457 (0.0006) [2023-12-27 04:42:28,906][105620] Updated weights for policy 1, policy_version 1831467 (0.0006) [2023-12-27 04:42:28,969][105620] Updated weights for policy 1, policy_version 1831477 (0.0006) [2023-12-27 04:42:29,035][105692] Updated weights for policy 0, policy_version 1827475 (0.0006) [2023-12-27 04:42:29,103][105692] Updated weights for policy 0, policy_version 1827485 (0.0010) [2023-12-27 04:42:29,151][105692] Updated weights for policy 0, policy_version 1827495 (0.0010) [2023-12-27 04:42:29,657][105620] Updated weights for policy 1, policy_version 1831487 (0.0007) [2023-12-27 04:42:29,717][105620] Updated weights for policy 1, policy_version 1831497 (0.0008) [2023-12-27 04:42:29,775][105620] Updated weights for policy 1, policy_version 1831507 (0.0008) [2023-12-27 04:42:29,878][105692] Updated weights for policy 0, policy_version 1827505 (0.0008) [2023-12-27 04:42:29,943][105692] Updated weights for policy 0, policy_version 1827515 (0.0011) [2023-12-27 04:42:29,994][105692] Updated weights for policy 0, policy_version 1827525 (0.0010) [2023-12-27 04:42:30,053][105692] Updated weights for policy 0, policy_version 1827535 (0.0010) [2023-12-27 04:42:30,522][105620] Updated weights for policy 1, policy_version 1831517 (0.0006) [2023-12-27 04:42:30,583][105620] Updated weights for policy 1, policy_version 1831527 (0.0005) [2023-12-27 04:42:30,642][105620] Updated weights for policy 1, policy_version 1831537 (0.0005) [2023-12-27 04:42:30,725][105692] Updated weights for policy 0, policy_version 1827545 (0.0007) [2023-12-27 04:42:30,780][105692] Updated weights for policy 0, policy_version 1827555 (0.0005) [2023-12-27 04:42:30,836][105692] Updated weights for policy 0, policy_version 1827565 (0.0005) [2023-12-27 04:42:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19660.8). Total num frames: 936869888. Throughput: 0: 9905.9, 1: 9767.0. Samples: 936838284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:42:31,062][104569] Avg episode reward: [(0, '8259.747'), (1, '9350.316')] [2023-12-27 04:42:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001827568_467927040.pth... [2023-12-27 04:42:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001831544_468942848.pth... [2023-12-27 04:42:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001826416_467632128.pth [2023-12-27 04:42:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001830424_468656128.pth [2023-12-27 04:42:31,172][105620] Updated weights for policy 1, policy_version 1831547 (0.0007) [2023-12-27 04:42:31,224][105620] Updated weights for policy 1, policy_version 1831557 (0.0010) [2023-12-27 04:42:31,281][105620] Updated weights for policy 1, policy_version 1831567 (0.0007) [2023-12-27 04:42:31,519][105692] Updated weights for policy 0, policy_version 1827575 (0.0008) [2023-12-27 04:42:31,581][105692] Updated weights for policy 0, policy_version 1827585 (0.0009) [2023-12-27 04:42:31,650][105692] Updated weights for policy 0, policy_version 1827595 (0.0009) [2023-12-27 04:42:32,009][105620] Updated weights for policy 1, policy_version 1831577 (0.0007) [2023-12-27 04:42:32,084][105620] Updated weights for policy 1, policy_version 1831587 (0.0010) [2023-12-27 04:42:32,140][105620] Updated weights for policy 1, policy_version 1831597 (0.0008) [2023-12-27 04:42:32,201][105620] Updated weights for policy 1, policy_version 1831607 (0.0009) [2023-12-27 04:42:32,341][105692] Updated weights for policy 0, policy_version 1827605 (0.0012) [2023-12-27 04:42:32,407][105692] Updated weights for policy 0, policy_version 1827615 (0.0006) [2023-12-27 04:42:32,472][105692] Updated weights for policy 0, policy_version 1827625 (0.0006) [2023-12-27 04:42:32,901][105620] Updated weights for policy 1, policy_version 1831617 (0.0008) [2023-12-27 04:42:32,948][105620] Updated weights for policy 1, policy_version 1831627 (0.0008) [2023-12-27 04:42:33,011][105620] Updated weights for policy 1, policy_version 1831637 (0.0009) [2023-12-27 04:42:33,100][105692] Updated weights for policy 0, policy_version 1827635 (0.0008) [2023-12-27 04:42:33,165][105692] Updated weights for policy 0, policy_version 1827645 (0.0005) [2023-12-27 04:42:33,219][105692] Updated weights for policy 0, policy_version 1827655 (0.0005) [2023-12-27 04:42:33,677][105620] Updated weights for policy 1, policy_version 1831647 (0.0006) [2023-12-27 04:42:33,746][105620] Updated weights for policy 1, policy_version 1831657 (0.0005) [2023-12-27 04:42:33,808][105620] Updated weights for policy 1, policy_version 1831667 (0.0006) [2023-12-27 04:42:33,933][105692] Updated weights for policy 0, policy_version 1827665 (0.0009) [2023-12-27 04:42:33,987][105692] Updated weights for policy 0, policy_version 1827675 (0.0009) [2023-12-27 04:42:34,043][105692] Updated weights for policy 0, policy_version 1827685 (0.0009) [2023-12-27 04:42:34,094][105692] Updated weights for policy 0, policy_version 1827695 (0.0009) [2023-12-27 04:42:34,444][105620] Updated weights for policy 1, policy_version 1831677 (0.0009) [2023-12-27 04:42:34,506][105620] Updated weights for policy 1, policy_version 1831687 (0.0009) [2023-12-27 04:42:34,569][105620] Updated weights for policy 1, policy_version 1831697 (0.0009) [2023-12-27 04:42:34,884][105692] Updated weights for policy 0, policy_version 1827705 (0.0006) [2023-12-27 04:42:34,937][105692] Updated weights for policy 0, policy_version 1827715 (0.0005) [2023-12-27 04:42:34,980][105692] Updated weights for policy 0, policy_version 1827725 (0.0005) [2023-12-27 04:42:35,265][105620] Updated weights for policy 1, policy_version 1831707 (0.0009) [2023-12-27 04:42:35,320][105620] Updated weights for policy 1, policy_version 1831717 (0.0005) [2023-12-27 04:42:35,373][105620] Updated weights for policy 1, policy_version 1831727 (0.0005) [2023-12-27 04:42:35,576][105692] Updated weights for policy 0, policy_version 1827735 (0.0008) [2023-12-27 04:42:35,623][105692] Updated weights for policy 0, policy_version 1827745 (0.0009) [2023-12-27 04:42:35,681][105692] Updated weights for policy 0, policy_version 1827755 (0.0009) [2023-12-27 04:42:36,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19933.8, 300 sec: 19660.8). Total num frames: 936968192. Throughput: 0: 9833.4, 1: 9728.0. Samples: 936957640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:42:36,063][104569] Avg episode reward: [(0, '8172.336'), (1, '9350.397')] [2023-12-27 04:42:36,077][105620] Updated weights for policy 1, policy_version 1831737 (0.0006) [2023-12-27 04:42:36,163][105620] Updated weights for policy 1, policy_version 1831747 (0.0009) [2023-12-27 04:42:36,220][105620] Updated weights for policy 1, policy_version 1831757 (0.0010) [2023-12-27 04:42:36,277][105620] Updated weights for policy 1, policy_version 1831767 (0.0009) [2023-12-27 04:42:36,356][105692] Updated weights for policy 0, policy_version 1827765 (0.0006) [2023-12-27 04:42:36,414][105692] Updated weights for policy 0, policy_version 1827775 (0.0008) [2023-12-27 04:42:36,474][105692] Updated weights for policy 0, policy_version 1827785 (0.0009) [2023-12-27 04:42:37,049][105620] Updated weights for policy 1, policy_version 1831777 (0.0009) [2023-12-27 04:42:37,104][105620] Updated weights for policy 1, policy_version 1831787 (0.0009) [2023-12-27 04:42:37,161][105620] Updated weights for policy 1, policy_version 1831797 (0.0008) [2023-12-27 04:42:37,205][105692] Updated weights for policy 0, policy_version 1827795 (0.0010) [2023-12-27 04:42:37,267][105692] Updated weights for policy 0, policy_version 1827805 (0.0010) [2023-12-27 04:42:37,327][105692] Updated weights for policy 0, policy_version 1827815 (0.0011) [2023-12-27 04:42:37,931][105692] Updated weights for policy 0, policy_version 1827825 (0.0011) [2023-12-27 04:42:37,983][105692] Updated weights for policy 0, policy_version 1827835 (0.0010) [2023-12-27 04:42:37,989][105620] Updated weights for policy 1, policy_version 1831807 (0.0006) [2023-12-27 04:42:38,032][105692] Updated weights for policy 0, policy_version 1827845 (0.0010) [2023-12-27 04:42:38,042][105620] Updated weights for policy 1, policy_version 1831817 (0.0005) [2023-12-27 04:42:38,081][105692] Updated weights for policy 0, policy_version 1827855 (0.0010) [2023-12-27 04:42:38,091][105620] Updated weights for policy 1, policy_version 1831827 (0.0005) [2023-12-27 04:42:38,764][105692] Updated weights for policy 0, policy_version 1827865 (0.0008) [2023-12-27 04:42:38,831][105692] Updated weights for policy 0, policy_version 1827875 (0.0005) [2023-12-27 04:42:38,842][105620] Updated weights for policy 1, policy_version 1831837 (0.0008) [2023-12-27 04:42:38,894][105692] Updated weights for policy 0, policy_version 1827885 (0.0005) [2023-12-27 04:42:38,902][105620] Updated weights for policy 1, policy_version 1831847 (0.0008) [2023-12-27 04:42:38,971][105620] Updated weights for policy 1, policy_version 1831857 (0.0009) [2023-12-27 04:42:39,468][105692] Updated weights for policy 0, policy_version 1827895 (0.0009) [2023-12-27 04:42:39,533][105692] Updated weights for policy 0, policy_version 1827905 (0.0011) [2023-12-27 04:42:39,600][105692] Updated weights for policy 0, policy_version 1827915 (0.0011) [2023-12-27 04:42:39,824][105620] Updated weights for policy 1, policy_version 1831867 (0.0009) [2023-12-27 04:42:39,889][105620] Updated weights for policy 1, policy_version 1831877 (0.0009) [2023-12-27 04:42:39,950][105620] Updated weights for policy 1, policy_version 1831887 (0.0008) [2023-12-27 04:42:40,275][105692] Updated weights for policy 0, policy_version 1827925 (0.0008) [2023-12-27 04:42:40,332][105692] Updated weights for policy 0, policy_version 1827935 (0.0010) [2023-12-27 04:42:40,386][105692] Updated weights for policy 0, policy_version 1827945 (0.0008) [2023-12-27 04:42:40,678][105620] Updated weights for policy 1, policy_version 1831897 (0.0006) [2023-12-27 04:42:40,736][105620] Updated weights for policy 1, policy_version 1831907 (0.0008) [2023-12-27 04:42:40,790][105620] Updated weights for policy 1, policy_version 1831917 (0.0006) [2023-12-27 04:42:40,839][105620] Updated weights for policy 1, policy_version 1831927 (0.0010) [2023-12-27 04:42:40,969][105692] Updated weights for policy 0, policy_version 1827955 (0.0005) [2023-12-27 04:42:41,029][105692] Updated weights for policy 0, policy_version 1827965 (0.0006) [2023-12-27 04:42:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19797.4, 300 sec: 19660.8). Total num frames: 937066496. Throughput: 0: 9981.8, 1: 9623.5. Samples: 937076556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:42:41,062][104569] Avg episode reward: [(0, '8178.178'), (1, '9258.028')] [2023-12-27 04:42:41,100][105692] Updated weights for policy 0, policy_version 1827975 (0.0006) [2023-12-27 04:42:41,559][105620] Updated weights for policy 1, policy_version 1831937 (0.0009) [2023-12-27 04:42:41,625][105620] Updated weights for policy 1, policy_version 1831947 (0.0009) [2023-12-27 04:42:41,690][105620] Updated weights for policy 1, policy_version 1831957 (0.0008) [2023-12-27 04:42:41,817][105692] Updated weights for policy 0, policy_version 1827985 (0.0008) [2023-12-27 04:42:41,883][105692] Updated weights for policy 0, policy_version 1827995 (0.0010) [2023-12-27 04:42:41,943][105692] Updated weights for policy 0, policy_version 1828005 (0.0010) [2023-12-27 04:42:42,006][105692] Updated weights for policy 0, policy_version 1828015 (0.0011) [2023-12-27 04:42:42,393][105620] Updated weights for policy 1, policy_version 1831967 (0.0009) [2023-12-27 04:42:42,461][105620] Updated weights for policy 1, policy_version 1831977 (0.0009) [2023-12-27 04:42:42,522][105620] Updated weights for policy 1, policy_version 1831987 (0.0009) [2023-12-27 04:42:42,669][105692] Updated weights for policy 0, policy_version 1828025 (0.0009) [2023-12-27 04:42:42,727][105692] Updated weights for policy 0, policy_version 1828035 (0.0009) [2023-12-27 04:42:42,785][105692] Updated weights for policy 0, policy_version 1828045 (0.0009) [2023-12-27 04:42:43,136][105620] Updated weights for policy 1, policy_version 1831997 (0.0007) [2023-12-27 04:42:43,197][105620] Updated weights for policy 1, policy_version 1832007 (0.0005) [2023-12-27 04:42:43,248][105620] Updated weights for policy 1, policy_version 1832017 (0.0005) [2023-12-27 04:42:43,705][105692] Updated weights for policy 0, policy_version 1828055 (0.0010) [2023-12-27 04:42:43,756][105692] Updated weights for policy 0, policy_version 1828065 (0.0010) [2023-12-27 04:42:43,759][105620] Updated weights for policy 1, policy_version 1832027 (0.0005) [2023-12-27 04:42:43,816][105692] Updated weights for policy 0, policy_version 1828075 (0.0009) [2023-12-27 04:42:43,826][105620] Updated weights for policy 1, policy_version 1832037 (0.0006) [2023-12-27 04:42:43,884][105620] Updated weights for policy 1, policy_version 1832047 (0.0008) [2023-12-27 04:42:44,503][105692] Updated weights for policy 0, policy_version 1828085 (0.0009) [2023-12-27 04:42:44,558][105692] Updated weights for policy 0, policy_version 1828095 (0.0009) [2023-12-27 04:42:44,564][105620] Updated weights for policy 1, policy_version 1832057 (0.0008) [2023-12-27 04:42:44,605][105692] Updated weights for policy 0, policy_version 1828105 (0.0007) [2023-12-27 04:42:44,624][105620] Updated weights for policy 1, policy_version 1832067 (0.0007) [2023-12-27 04:42:44,680][105620] Updated weights for policy 1, policy_version 1832077 (0.0008) [2023-12-27 04:42:44,738][105620] Updated weights for policy 1, policy_version 1832087 (0.0007) [2023-12-27 04:42:45,297][105692] Updated weights for policy 0, policy_version 1828115 (0.0006) [2023-12-27 04:42:45,350][105692] Updated weights for policy 0, policy_version 1828125 (0.0008) [2023-12-27 04:42:45,411][105692] Updated weights for policy 0, policy_version 1828135 (0.0009) [2023-12-27 04:42:45,434][105620] Updated weights for policy 1, policy_version 1832097 (0.0010) [2023-12-27 04:42:45,500][105620] Updated weights for policy 1, policy_version 1832107 (0.0011) [2023-12-27 04:42:45,549][105620] Updated weights for policy 1, policy_version 1832117 (0.0011) [2023-12-27 04:42:46,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.4, 300 sec: 19660.8). Total num frames: 937164800. Throughput: 0: 9948.7, 1: 9695.1. Samples: 937137088. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:42:46,062][104569] Avg episode reward: [(0, '8356.798'), (1, '9258.147')] [2023-12-27 04:42:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001832120_469090304.pth... [2023-12-27 04:42:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001828144_468074496.pth... [2023-12-27 04:42:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001826992_467779584.pth [2023-12-27 04:42:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001831000_468803584.pth [2023-12-27 04:42:46,131][105692] Updated weights for policy 0, policy_version 1828145 (0.0006) [2023-12-27 04:42:46,204][105692] Updated weights for policy 0, policy_version 1828155 (0.0008) [2023-12-27 04:42:46,263][105620] Updated weights for policy 1, policy_version 1832127 (0.0007) [2023-12-27 04:42:46,268][105692] Updated weights for policy 0, policy_version 1828165 (0.0009) [2023-12-27 04:42:46,325][105620] Updated weights for policy 1, policy_version 1832137 (0.0006) [2023-12-27 04:42:46,327][105692] Updated weights for policy 0, policy_version 1828175 (0.0007) [2023-12-27 04:42:46,377][105620] Updated weights for policy 1, policy_version 1832147 (0.0007) [2023-12-27 04:42:47,074][105692] Updated weights for policy 0, policy_version 1828185 (0.0008) [2023-12-27 04:42:47,108][105620] Updated weights for policy 1, policy_version 1832157 (0.0010) [2023-12-27 04:42:47,131][105692] Updated weights for policy 0, policy_version 1828195 (0.0006) [2023-12-27 04:42:47,165][105620] Updated weights for policy 1, policy_version 1832167 (0.0009) [2023-12-27 04:42:47,192][105692] Updated weights for policy 0, policy_version 1828205 (0.0007) [2023-12-27 04:42:47,228][105620] Updated weights for policy 1, policy_version 1832177 (0.0011) [2023-12-27 04:42:47,885][105692] Updated weights for policy 0, policy_version 1828215 (0.0006) [2023-12-27 04:42:47,919][105620] Updated weights for policy 1, policy_version 1832187 (0.0009) [2023-12-27 04:42:47,955][105692] Updated weights for policy 0, policy_version 1828225 (0.0005) [2023-12-27 04:42:47,980][105620] Updated weights for policy 1, policy_version 1832197 (0.0008) [2023-12-27 04:42:48,008][105692] Updated weights for policy 0, policy_version 1828235 (0.0006) [2023-12-27 04:42:48,036][105620] Updated weights for policy 1, policy_version 1832207 (0.0010) [2023-12-27 04:42:48,656][105692] Updated weights for policy 0, policy_version 1828245 (0.0007) [2023-12-27 04:42:48,711][105692] Updated weights for policy 0, policy_version 1828255 (0.0008) [2023-12-27 04:42:48,727][105620] Updated weights for policy 1, policy_version 1832217 (0.0010) [2023-12-27 04:42:48,768][105692] Updated weights for policy 0, policy_version 1828265 (0.0008) [2023-12-27 04:42:48,782][105620] Updated weights for policy 1, policy_version 1832227 (0.0010) [2023-12-27 04:42:48,844][105620] Updated weights for policy 1, policy_version 1832237 (0.0010) [2023-12-27 04:42:48,900][105620] Updated weights for policy 1, policy_version 1832247 (0.0010) [2023-12-27 04:42:49,535][105692] Updated weights for policy 0, policy_version 1828275 (0.0005) [2023-12-27 04:42:49,601][105692] Updated weights for policy 0, policy_version 1828285 (0.0009) [2023-12-27 04:42:49,663][105692] Updated weights for policy 0, policy_version 1828295 (0.0009) [2023-12-27 04:42:49,672][105620] Updated weights for policy 1, policy_version 1832257 (0.0007) [2023-12-27 04:42:49,731][105620] Updated weights for policy 1, policy_version 1832267 (0.0008) [2023-12-27 04:42:49,794][105620] Updated weights for policy 1, policy_version 1832277 (0.0009) [2023-12-27 04:42:50,434][105692] Updated weights for policy 0, policy_version 1828305 (0.0007) [2023-12-27 04:42:50,464][105620] Updated weights for policy 1, policy_version 1832287 (0.0008) [2023-12-27 04:42:50,493][105692] Updated weights for policy 0, policy_version 1828315 (0.0009) [2023-12-27 04:42:50,511][105620] Updated weights for policy 1, policy_version 1832297 (0.0009) [2023-12-27 04:42:50,547][105692] Updated weights for policy 0, policy_version 1828325 (0.0007) [2023-12-27 04:42:50,558][105620] Updated weights for policy 1, policy_version 1832307 (0.0007) [2023-12-27 04:42:50,613][105692] Updated weights for policy 0, policy_version 1828335 (0.0008) [2023-12-27 04:42:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 937263104. Throughput: 0: 9918.4, 1: 9725.7. Samples: 937253984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:42:51,062][104569] Avg episode reward: [(0, '8263.798'), (1, '9165.753')] [2023-12-27 04:42:51,288][105620] Updated weights for policy 1, policy_version 1832317 (0.0007) [2023-12-27 04:42:51,357][105620] Updated weights for policy 1, policy_version 1832327 (0.0007) [2023-12-27 04:42:51,393][105692] Updated weights for policy 0, policy_version 1828345 (0.0007) [2023-12-27 04:42:51,417][105620] Updated weights for policy 1, policy_version 1832337 (0.0008) [2023-12-27 04:42:51,453][105692] Updated weights for policy 0, policy_version 1828355 (0.0008) [2023-12-27 04:42:51,510][105692] Updated weights for policy 0, policy_version 1828365 (0.0009) [2023-12-27 04:42:52,126][105620] Updated weights for policy 1, policy_version 1832347 (0.0009) [2023-12-27 04:42:52,186][105620] Updated weights for policy 1, policy_version 1832357 (0.0009) [2023-12-27 04:42:52,254][105620] Updated weights for policy 1, policy_version 1832367 (0.0007) [2023-12-27 04:42:52,259][105692] Updated weights for policy 0, policy_version 1828375 (0.0010) [2023-12-27 04:42:52,318][105692] Updated weights for policy 0, policy_version 1828385 (0.0009) [2023-12-27 04:42:52,380][105692] Updated weights for policy 0, policy_version 1828395 (0.0008) [2023-12-27 04:42:52,961][105620] Updated weights for policy 1, policy_version 1832377 (0.0007) [2023-12-27 04:42:53,032][105620] Updated weights for policy 1, policy_version 1832387 (0.0006) [2023-12-27 04:42:53,098][105620] Updated weights for policy 1, policy_version 1832397 (0.0005) [2023-12-27 04:42:53,161][105620] Updated weights for policy 1, policy_version 1832407 (0.0007) [2023-12-27 04:42:53,166][105692] Updated weights for policy 0, policy_version 1828405 (0.0008) [2023-12-27 04:42:53,222][105692] Updated weights for policy 0, policy_version 1828415 (0.0008) [2023-12-27 04:42:53,278][105692] Updated weights for policy 0, policy_version 1828425 (0.0010) [2023-12-27 04:42:53,715][105620] Updated weights for policy 1, policy_version 1832417 (0.0005) [2023-12-27 04:42:53,763][105620] Updated weights for policy 1, policy_version 1832427 (0.0005) [2023-12-27 04:42:53,814][105620] Updated weights for policy 1, policy_version 1832437 (0.0007) [2023-12-27 04:42:54,123][105692] Updated weights for policy 0, policy_version 1828436 (0.0010) [2023-12-27 04:42:54,177][105692] Updated weights for policy 0, policy_version 1828446 (0.0007) [2023-12-27 04:42:54,235][105692] Updated weights for policy 0, policy_version 1828456 (0.0005) [2023-12-27 04:42:54,398][105620] Updated weights for policy 1, policy_version 1832447 (0.0007) [2023-12-27 04:42:54,450][105620] Updated weights for policy 1, policy_version 1832457 (0.0008) [2023-12-27 04:42:54,500][105620] Updated weights for policy 1, policy_version 1832467 (0.0008) [2023-12-27 04:42:54,964][105692] Updated weights for policy 0, policy_version 1828466 (0.0006) [2023-12-27 04:42:55,035][105692] Updated weights for policy 0, policy_version 1828476 (0.0010) [2023-12-27 04:42:55,097][105692] Updated weights for policy 0, policy_version 1828486 (0.0010) [2023-12-27 04:42:55,132][105620] Updated weights for policy 1, policy_version 1832477 (0.0008) [2023-12-27 04:42:55,160][105692] Updated weights for policy 0, policy_version 1828496 (0.0009) [2023-12-27 04:42:55,184][105620] Updated weights for policy 1, policy_version 1832487 (0.0006) [2023-12-27 04:42:55,240][105620] Updated weights for policy 1, policy_version 1832497 (0.0010) [2023-12-27 04:42:55,926][105692] Updated weights for policy 0, policy_version 1828506 (0.0011) [2023-12-27 04:42:55,986][105620] Updated weights for policy 1, policy_version 1832507 (0.0011) [2023-12-27 04:42:55,990][105692] Updated weights for policy 0, policy_version 1828516 (0.0011) [2023-12-27 04:42:56,043][105620] Updated weights for policy 1, policy_version 1832517 (0.0011) [2023-12-27 04:42:56,054][105692] Updated weights for policy 0, policy_version 1828526 (0.0011) [2023-12-27 04:42:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.4, 300 sec: 19633.0). Total num frames: 937353216. Throughput: 0: 9766.4, 1: 9797.6. Samples: 937369916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:42:56,063][104569] Avg episode reward: [(0, '8626.559'), (1, '9166.749')] [2023-12-27 04:42:56,107][105620] Updated weights for policy 1, policy_version 1832527 (0.0011) [2023-12-27 04:42:56,710][105692] Updated weights for policy 0, policy_version 1828536 (0.0007) [2023-12-27 04:42:56,755][105692] Updated weights for policy 0, policy_version 1828546 (0.0005) [2023-12-27 04:42:56,803][105692] Updated weights for policy 0, policy_version 1828556 (0.0005) [2023-12-27 04:42:56,851][105620] Updated weights for policy 1, policy_version 1832537 (0.0011) [2023-12-27 04:42:56,906][105620] Updated weights for policy 1, policy_version 1832547 (0.0010) [2023-12-27 04:42:56,971][105620] Updated weights for policy 1, policy_version 1832557 (0.0010) [2023-12-27 04:42:57,036][105620] Updated weights for policy 1, policy_version 1832567 (0.0010) [2023-12-27 04:42:57,346][105692] Updated weights for policy 0, policy_version 1828566 (0.0005) [2023-12-27 04:42:57,411][105692] Updated weights for policy 0, policy_version 1828576 (0.0005) [2023-12-27 04:42:57,461][105692] Updated weights for policy 0, policy_version 1828586 (0.0005) [2023-12-27 04:42:57,739][105620] Updated weights for policy 1, policy_version 1832577 (0.0010) [2023-12-27 04:42:57,786][105620] Updated weights for policy 1, policy_version 1832587 (0.0010) [2023-12-27 04:42:57,840][105620] Updated weights for policy 1, policy_version 1832597 (0.0010) [2023-12-27 04:42:58,030][105692] Updated weights for policy 0, policy_version 1828596 (0.0007) [2023-12-27 04:42:58,095][105692] Updated weights for policy 0, policy_version 1828606 (0.0009) [2023-12-27 04:42:58,161][105692] Updated weights for policy 0, policy_version 1828616 (0.0009) [2023-12-27 04:42:58,654][105620] Updated weights for policy 1, policy_version 1832607 (0.0008) [2023-12-27 04:42:58,717][105620] Updated weights for policy 1, policy_version 1832617 (0.0008) [2023-12-27 04:42:58,781][105620] Updated weights for policy 1, policy_version 1832627 (0.0007) [2023-12-27 04:42:58,996][105692] Updated weights for policy 0, policy_version 1828626 (0.0008) [2023-12-27 04:42:59,051][105692] Updated weights for policy 0, policy_version 1828636 (0.0008) [2023-12-27 04:42:59,119][105692] Updated weights for policy 0, policy_version 1828646 (0.0008) [2023-12-27 04:42:59,180][105692] Updated weights for policy 0, policy_version 1828656 (0.0008) [2023-12-27 04:42:59,610][105620] Updated weights for policy 1, policy_version 1832637 (0.0008) [2023-12-27 04:42:59,669][105620] Updated weights for policy 1, policy_version 1832647 (0.0008) [2023-12-27 04:42:59,726][105620] Updated weights for policy 1, policy_version 1832657 (0.0008) [2023-12-27 04:43:00,051][105692] Updated weights for policy 0, policy_version 1828666 (0.0008) [2023-12-27 04:43:00,102][105692] Updated weights for policy 0, policy_version 1828676 (0.0008) [2023-12-27 04:43:00,146][105692] Updated weights for policy 0, policy_version 1828686 (0.0008) [2023-12-27 04:43:00,372][105620] Updated weights for policy 1, policy_version 1832667 (0.0009) [2023-12-27 04:43:00,427][105620] Updated weights for policy 1, policy_version 1832677 (0.0009) [2023-12-27 04:43:00,475][105620] Updated weights for policy 1, policy_version 1832687 (0.0008) [2023-12-27 04:43:00,870][105692] Updated weights for policy 0, policy_version 1828696 (0.0006) [2023-12-27 04:43:00,928][105692] Updated weights for policy 0, policy_version 1828706 (0.0007) [2023-12-27 04:43:00,982][105692] Updated weights for policy 0, policy_version 1828716 (0.0009) [2023-12-27 04:43:01,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 937459712. Throughput: 0: 9842.7, 1: 9782.1. Samples: 937429900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:43:01,063][104569] Avg episode reward: [(0, '8446.011'), (1, '9074.603')] [2023-12-27 04:43:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001828720_468221952.pth... [2023-12-27 04:43:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001832696_469237760.pth... [2023-12-27 04:43:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001831544_468942848.pth [2023-12-27 04:43:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001827568_467927040.pth [2023-12-27 04:43:01,274][105620] Updated weights for policy 1, policy_version 1832697 (0.0009) [2023-12-27 04:43:01,326][105620] Updated weights for policy 1, policy_version 1832707 (0.0009) [2023-12-27 04:43:01,394][105620] Updated weights for policy 1, policy_version 1832717 (0.0009) [2023-12-27 04:43:01,449][105620] Updated weights for policy 1, policy_version 1832727 (0.0009) [2023-12-27 04:43:01,717][105692] Updated weights for policy 0, policy_version 1828726 (0.0009) [2023-12-27 04:43:01,780][105692] Updated weights for policy 0, policy_version 1828736 (0.0008) [2023-12-27 04:43:01,839][105692] Updated weights for policy 0, policy_version 1828746 (0.0008) [2023-12-27 04:43:02,112][105620] Updated weights for policy 1, policy_version 1832737 (0.0005) [2023-12-27 04:43:02,163][105620] Updated weights for policy 1, policy_version 1832747 (0.0007) [2023-12-27 04:43:02,212][105620] Updated weights for policy 1, policy_version 1832757 (0.0011) [2023-12-27 04:43:02,652][105692] Updated weights for policy 0, policy_version 1828756 (0.0009) [2023-12-27 04:43:02,714][105692] Updated weights for policy 0, policy_version 1828766 (0.0010) [2023-12-27 04:43:02,771][105692] Updated weights for policy 0, policy_version 1828776 (0.0009) [2023-12-27 04:43:02,886][105620] Updated weights for policy 1, policy_version 1832767 (0.0011) [2023-12-27 04:43:02,930][105620] Updated weights for policy 1, policy_version 1832777 (0.0010) [2023-12-27 04:43:02,975][105620] Updated weights for policy 1, policy_version 1832787 (0.0010) [2023-12-27 04:43:03,512][105692] Updated weights for policy 0, policy_version 1828786 (0.0008) [2023-12-27 04:43:03,572][105692] Updated weights for policy 0, policy_version 1828796 (0.0007) [2023-12-27 04:43:03,634][105692] Updated weights for policy 0, policy_version 1828806 (0.0008) [2023-12-27 04:43:03,695][105692] Updated weights for policy 0, policy_version 1828816 (0.0009) [2023-12-27 04:43:03,736][105620] Updated weights for policy 1, policy_version 1832797 (0.0010) [2023-12-27 04:43:03,790][105620] Updated weights for policy 1, policy_version 1832807 (0.0010) [2023-12-27 04:43:03,838][105620] Updated weights for policy 1, policy_version 1832817 (0.0010) [2023-12-27 04:43:04,306][105692] Updated weights for policy 0, policy_version 1828826 (0.0011) [2023-12-27 04:43:04,372][105692] Updated weights for policy 0, policy_version 1828836 (0.0007) [2023-12-27 04:43:04,443][105692] Updated weights for policy 0, policy_version 1828846 (0.0009) [2023-12-27 04:43:04,555][105620] Updated weights for policy 1, policy_version 1832827 (0.0009) [2023-12-27 04:43:04,616][105620] Updated weights for policy 1, policy_version 1832837 (0.0005) [2023-12-27 04:43:04,671][105620] Updated weights for policy 1, policy_version 1832847 (0.0007) [2023-12-27 04:43:05,130][105692] Updated weights for policy 0, policy_version 1828856 (0.0010) [2023-12-27 04:43:05,195][105692] Updated weights for policy 0, policy_version 1828866 (0.0010) [2023-12-27 04:43:05,240][105620] Updated weights for policy 1, policy_version 1832857 (0.0006) [2023-12-27 04:43:05,247][105692] Updated weights for policy 0, policy_version 1828876 (0.0010) [2023-12-27 04:43:05,295][105620] Updated weights for policy 1, policy_version 1832867 (0.0010) [2023-12-27 04:43:05,349][105620] Updated weights for policy 1, policy_version 1832877 (0.0010) [2023-12-27 04:43:05,402][105620] Updated weights for policy 1, policy_version 1832887 (0.0009) [2023-12-27 04:43:05,857][105692] Updated weights for policy 0, policy_version 1828886 (0.0009) [2023-12-27 04:43:05,909][105692] Updated weights for policy 0, policy_version 1828896 (0.0005) [2023-12-27 04:43:05,963][105692] Updated weights for policy 0, policy_version 1828906 (0.0005) [2023-12-27 04:43:06,040][105620] Updated weights for policy 1, policy_version 1832897 (0.0006) [2023-12-27 04:43:06,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 937558016. Throughput: 0: 9859.7, 1: 9739.5. Samples: 937544948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:43:06,062][104569] Avg episode reward: [(0, '8262.003'), (1, '9165.966')] [2023-12-27 04:43:06,086][105620] Updated weights for policy 1, policy_version 1832907 (0.0005) [2023-12-27 04:43:06,147][105620] Updated weights for policy 1, policy_version 1832917 (0.0008) [2023-12-27 04:43:06,759][105692] Updated weights for policy 0, policy_version 1828916 (0.0009) [2023-12-27 04:43:06,814][105692] Updated weights for policy 0, policy_version 1828926 (0.0009) [2023-12-27 04:43:06,815][105620] Updated weights for policy 1, policy_version 1832927 (0.0006) [2023-12-27 04:43:06,871][105692] Updated weights for policy 0, policy_version 1828936 (0.0009) [2023-12-27 04:43:06,875][105620] Updated weights for policy 1, policy_version 1832937 (0.0006) [2023-12-27 04:43:06,929][105620] Updated weights for policy 1, policy_version 1832947 (0.0005) [2023-12-27 04:43:07,480][105620] Updated weights for policy 1, policy_version 1832957 (0.0007) [2023-12-27 04:43:07,541][105620] Updated weights for policy 1, policy_version 1832967 (0.0009) [2023-12-27 04:43:07,601][105620] Updated weights for policy 1, policy_version 1832977 (0.0009) [2023-12-27 04:43:07,714][105692] Updated weights for policy 0, policy_version 1828946 (0.0009) [2023-12-27 04:43:07,777][105692] Updated weights for policy 0, policy_version 1828956 (0.0010) [2023-12-27 04:43:07,830][105692] Updated weights for policy 0, policy_version 1828966 (0.0010) [2023-12-27 04:43:07,887][105692] Updated weights for policy 0, policy_version 1828976 (0.0010) [2023-12-27 04:43:08,331][105620] Updated weights for policy 1, policy_version 1832987 (0.0009) [2023-12-27 04:43:08,401][105620] Updated weights for policy 1, policy_version 1832997 (0.0006) [2023-12-27 04:43:08,463][105620] Updated weights for policy 1, policy_version 1833007 (0.0006) [2023-12-27 04:43:08,577][105692] Updated weights for policy 0, policy_version 1828986 (0.0011) [2023-12-27 04:43:08,639][105692] Updated weights for policy 0, policy_version 1828996 (0.0011) [2023-12-27 04:43:08,691][105692] Updated weights for policy 0, policy_version 1829006 (0.0009) [2023-12-27 04:43:09,180][105620] Updated weights for policy 1, policy_version 1833017 (0.0008) [2023-12-27 04:43:09,243][105620] Updated weights for policy 1, policy_version 1833027 (0.0008) [2023-12-27 04:43:09,302][105620] Updated weights for policy 1, policy_version 1833037 (0.0008) [2023-12-27 04:43:09,361][105620] Updated weights for policy 1, policy_version 1833047 (0.0008) [2023-12-27 04:43:09,414][105692] Updated weights for policy 0, policy_version 1829016 (0.0008) [2023-12-27 04:43:09,475][105692] Updated weights for policy 0, policy_version 1829026 (0.0008) [2023-12-27 04:43:09,535][105692] Updated weights for policy 0, policy_version 1829036 (0.0005) [2023-12-27 04:43:10,054][105620] Updated weights for policy 1, policy_version 1833057 (0.0006) [2023-12-27 04:43:10,117][105620] Updated weights for policy 1, policy_version 1833067 (0.0006) [2023-12-27 04:43:10,175][105620] Updated weights for policy 1, policy_version 1833077 (0.0009) [2023-12-27 04:43:10,314][105692] Updated weights for policy 0, policy_version 1829046 (0.0006) [2023-12-27 04:43:10,378][105692] Updated weights for policy 0, policy_version 1829056 (0.0008) [2023-12-27 04:43:10,434][105692] Updated weights for policy 0, policy_version 1829066 (0.0009) [2023-12-27 04:43:10,853][105620] Updated weights for policy 1, policy_version 1833087 (0.0007) [2023-12-27 04:43:10,916][105620] Updated weights for policy 1, policy_version 1833097 (0.0005) [2023-12-27 04:43:10,964][105620] Updated weights for policy 1, policy_version 1833107 (0.0006) [2023-12-27 04:43:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 937656320. Throughput: 0: 9810.0, 1: 9879.9. Samples: 937663768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:43:11,063][104569] Avg episode reward: [(0, '8625.727'), (1, '9258.780')] [2023-12-27 04:43:11,080][105692] Updated weights for policy 0, policy_version 1829077 (0.0009) [2023-12-27 04:43:11,157][105692] Updated weights for policy 0, policy_version 1829087 (0.0007) [2023-12-27 04:43:11,227][105692] Updated weights for policy 0, policy_version 1829097 (0.0008) [2023-12-27 04:43:11,614][105620] Updated weights for policy 1, policy_version 1833117 (0.0007) [2023-12-27 04:43:11,684][105620] Updated weights for policy 1, policy_version 1833127 (0.0009) [2023-12-27 04:43:11,755][105620] Updated weights for policy 1, policy_version 1833137 (0.0008) [2023-12-27 04:43:12,011][105692] Updated weights for policy 0, policy_version 1829107 (0.0009) [2023-12-27 04:43:12,068][105692] Updated weights for policy 0, policy_version 1829117 (0.0008) [2023-12-27 04:43:12,128][105692] Updated weights for policy 0, policy_version 1829127 (0.0007) [2023-12-27 04:43:12,540][105620] Updated weights for policy 1, policy_version 1833147 (0.0009) [2023-12-27 04:43:12,600][105620] Updated weights for policy 1, policy_version 1833157 (0.0008) [2023-12-27 04:43:12,659][105620] Updated weights for policy 1, policy_version 1833167 (0.0008) [2023-12-27 04:43:12,850][105692] Updated weights for policy 0, policy_version 1829137 (0.0008) [2023-12-27 04:43:12,912][105692] Updated weights for policy 0, policy_version 1829147 (0.0010) [2023-12-27 04:43:12,964][105692] Updated weights for policy 0, policy_version 1829157 (0.0010) [2023-12-27 04:43:13,012][105692] Updated weights for policy 0, policy_version 1829167 (0.0010) [2023-12-27 04:43:13,393][105620] Updated weights for policy 1, policy_version 1833177 (0.0009) [2023-12-27 04:43:13,446][105620] Updated weights for policy 1, policy_version 1833187 (0.0008) [2023-12-27 04:43:13,495][105620] Updated weights for policy 1, policy_version 1833197 (0.0005) [2023-12-27 04:43:13,546][105620] Updated weights for policy 1, policy_version 1833207 (0.0005) [2023-12-27 04:43:13,641][105692] Updated weights for policy 0, policy_version 1829177 (0.0010) [2023-12-27 04:43:13,704][105692] Updated weights for policy 0, policy_version 1829187 (0.0010) [2023-12-27 04:43:13,756][105692] Updated weights for policy 0, policy_version 1829197 (0.0008) [2023-12-27 04:43:14,207][105620] Updated weights for policy 1, policy_version 1833217 (0.0006) [2023-12-27 04:43:14,261][105620] Updated weights for policy 1, policy_version 1833227 (0.0009) [2023-12-27 04:43:14,322][105620] Updated weights for policy 1, policy_version 1833237 (0.0010) [2023-12-27 04:43:14,499][105692] Updated weights for policy 0, policy_version 1829207 (0.0007) [2023-12-27 04:43:14,562][105692] Updated weights for policy 0, policy_version 1829217 (0.0005) [2023-12-27 04:43:14,620][105692] Updated weights for policy 0, policy_version 1829227 (0.0005) [2023-12-27 04:43:15,007][105620] Updated weights for policy 1, policy_version 1833247 (0.0010) [2023-12-27 04:43:15,070][105620] Updated weights for policy 1, policy_version 1833257 (0.0006) [2023-12-27 04:43:15,138][105620] Updated weights for policy 1, policy_version 1833267 (0.0008) [2023-12-27 04:43:15,265][105692] Updated weights for policy 0, policy_version 1829237 (0.0008) [2023-12-27 04:43:15,332][105692] Updated weights for policy 0, policy_version 1829247 (0.0011) [2023-12-27 04:43:15,397][105692] Updated weights for policy 0, policy_version 1829257 (0.0009) [2023-12-27 04:43:15,815][105620] Updated weights for policy 1, policy_version 1833277 (0.0011) [2023-12-27 04:43:15,873][105620] Updated weights for policy 1, policy_version 1833287 (0.0011) [2023-12-27 04:43:15,935][105620] Updated weights for policy 1, policy_version 1833297 (0.0010) [2023-12-27 04:43:16,052][105692] Updated weights for policy 0, policy_version 1829267 (0.0010) [2023-12-27 04:43:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.9, 300 sec: 19688.6). Total num frames: 937754624. Throughput: 0: 9745.9, 1: 9919.9. Samples: 937723244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:43:16,062][104569] Avg episode reward: [(0, '8537.754'), (1, '9166.625')] [2023-12-27 04:43:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001833304_469393408.pth... [2023-12-27 04:43:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001832120_469090304.pth [2023-12-27 04:43:16,115][105692] Updated weights for policy 0, policy_version 1829277 (0.0011) [2023-12-27 04:43:16,162][105692] Updated weights for policy 0, policy_version 1829287 (0.0010) [2023-12-27 04:43:16,214][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001829296_468369408.pth... [2023-12-27 04:43:16,217][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001828144_468074496.pth [2023-12-27 04:43:16,680][105620] Updated weights for policy 1, policy_version 1833307 (0.0010) [2023-12-27 04:43:16,742][105620] Updated weights for policy 1, policy_version 1833317 (0.0010) [2023-12-27 04:43:16,800][105620] Updated weights for policy 1, policy_version 1833327 (0.0010) [2023-12-27 04:43:16,837][105692] Updated weights for policy 0, policy_version 1829297 (0.0010) [2023-12-27 04:43:16,895][105692] Updated weights for policy 0, policy_version 1829307 (0.0008) [2023-12-27 04:43:16,959][105692] Updated weights for policy 0, policy_version 1829317 (0.0008) [2023-12-27 04:43:17,026][105692] Updated weights for policy 0, policy_version 1829327 (0.0008) [2023-12-27 04:43:17,534][105620] Updated weights for policy 1, policy_version 1833337 (0.0010) [2023-12-27 04:43:17,592][105620] Updated weights for policy 1, policy_version 1833347 (0.0010) [2023-12-27 04:43:17,651][105620] Updated weights for policy 1, policy_version 1833357 (0.0010) [2023-12-27 04:43:17,705][105620] Updated weights for policy 1, policy_version 1833367 (0.0010) [2023-12-27 04:43:17,789][105692] Updated weights for policy 0, policy_version 1829337 (0.0008) [2023-12-27 04:43:17,849][105692] Updated weights for policy 0, policy_version 1829347 (0.0008) [2023-12-27 04:43:17,913][105692] Updated weights for policy 0, policy_version 1829357 (0.0008) [2023-12-27 04:43:18,461][105620] Updated weights for policy 1, policy_version 1833377 (0.0008) [2023-12-27 04:43:18,519][105620] Updated weights for policy 1, policy_version 1833387 (0.0008) [2023-12-27 04:43:18,583][105620] Updated weights for policy 1, policy_version 1833397 (0.0005) [2023-12-27 04:43:18,643][105692] Updated weights for policy 0, policy_version 1829367 (0.0010) [2023-12-27 04:43:18,709][105692] Updated weights for policy 0, policy_version 1829377 (0.0011) [2023-12-27 04:43:18,776][105692] Updated weights for policy 0, policy_version 1829387 (0.0011) [2023-12-27 04:43:19,214][105620] Updated weights for policy 1, policy_version 1833407 (0.0008) [2023-12-27 04:43:19,283][105620] Updated weights for policy 1, policy_version 1833417 (0.0008) [2023-12-27 04:43:19,346][105620] Updated weights for policy 1, policy_version 1833427 (0.0008) [2023-12-27 04:43:19,536][105692] Updated weights for policy 0, policy_version 1829397 (0.0010) [2023-12-27 04:43:19,595][105692] Updated weights for policy 0, policy_version 1829407 (0.0007) [2023-12-27 04:43:19,656][105692] Updated weights for policy 0, policy_version 1829417 (0.0006) [2023-12-27 04:43:20,052][105620] Updated weights for policy 1, policy_version 1833437 (0.0008) [2023-12-27 04:43:20,118][105620] Updated weights for policy 1, policy_version 1833447 (0.0008) [2023-12-27 04:43:20,187][105620] Updated weights for policy 1, policy_version 1833457 (0.0008) [2023-12-27 04:43:20,333][105692] Updated weights for policy 0, policy_version 1829427 (0.0007) [2023-12-27 04:43:20,397][105692] Updated weights for policy 0, policy_version 1829437 (0.0009) [2023-12-27 04:43:20,460][105692] Updated weights for policy 0, policy_version 1829447 (0.0008) [2023-12-27 04:43:20,795][105620] Updated weights for policy 1, policy_version 1833467 (0.0007) [2023-12-27 04:43:20,859][105620] Updated weights for policy 1, policy_version 1833477 (0.0008) [2023-12-27 04:43:20,925][105620] Updated weights for policy 1, policy_version 1833487 (0.0007) [2023-12-27 04:43:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 937852928. Throughput: 0: 9718.0, 1: 9890.5. Samples: 937840020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-12-27 04:43:21,062][104569] Avg episode reward: [(0, '8444.460'), (1, '9258.500')] [2023-12-27 04:43:21,125][105692] Updated weights for policy 0, policy_version 1829457 (0.0006) [2023-12-27 04:43:21,183][105692] Updated weights for policy 0, policy_version 1829467 (0.0008) [2023-12-27 04:43:21,236][105692] Updated weights for policy 0, policy_version 1829477 (0.0006) [2023-12-27 04:43:21,300][105692] Updated weights for policy 0, policy_version 1829487 (0.0009) [2023-12-27 04:43:21,652][105620] Updated weights for policy 1, policy_version 1833497 (0.0006) [2023-12-27 04:43:21,715][105620] Updated weights for policy 1, policy_version 1833507 (0.0009) [2023-12-27 04:43:21,789][105620] Updated weights for policy 1, policy_version 1833517 (0.0009) [2023-12-27 04:43:21,846][105620] Updated weights for policy 1, policy_version 1833527 (0.0010) [2023-12-27 04:43:22,020][105692] Updated weights for policy 0, policy_version 1829497 (0.0006) [2023-12-27 04:43:22,083][105692] Updated weights for policy 0, policy_version 1829507 (0.0009) [2023-12-27 04:43:22,131][105692] Updated weights for policy 0, policy_version 1829517 (0.0007) [2023-12-27 04:43:22,617][105620] Updated weights for policy 1, policy_version 1833537 (0.0006) [2023-12-27 04:43:22,687][105620] Updated weights for policy 1, policy_version 1833547 (0.0005) [2023-12-27 04:43:22,755][105620] Updated weights for policy 1, policy_version 1833557 (0.0007) [2023-12-27 04:43:22,911][105692] Updated weights for policy 0, policy_version 1829527 (0.0008) [2023-12-27 04:43:22,981][105692] Updated weights for policy 0, policy_version 1829537 (0.0009) [2023-12-27 04:43:23,044][105692] Updated weights for policy 0, policy_version 1829547 (0.0009) [2023-12-27 04:43:23,391][105620] Updated weights for policy 1, policy_version 1833567 (0.0009) [2023-12-27 04:43:23,443][105620] Updated weights for policy 1, policy_version 1833577 (0.0009) [2023-12-27 04:43:23,491][105620] Updated weights for policy 1, policy_version 1833587 (0.0009) [2023-12-27 04:43:23,775][105692] Updated weights for policy 0, policy_version 1829557 (0.0009) [2023-12-27 04:43:23,842][105692] Updated weights for policy 0, policy_version 1829567 (0.0009) [2023-12-27 04:43:23,894][105692] Updated weights for policy 0, policy_version 1829577 (0.0007) [2023-12-27 04:43:24,349][105620] Updated weights for policy 1, policy_version 1833597 (0.0009) [2023-12-27 04:43:24,419][105620] Updated weights for policy 1, policy_version 1833607 (0.0007) [2023-12-27 04:43:24,472][105620] Updated weights for policy 1, policy_version 1833617 (0.0008) [2023-12-27 04:43:24,480][105692] Updated weights for policy 0, policy_version 1829587 (0.0006) [2023-12-27 04:43:24,538][105692] Updated weights for policy 0, policy_version 1829597 (0.0006) [2023-12-27 04:43:24,593][105692] Updated weights for policy 0, policy_version 1829607 (0.0008) [2023-12-27 04:43:25,149][105620] Updated weights for policy 1, policy_version 1833627 (0.0009) [2023-12-27 04:43:25,196][105692] Updated weights for policy 0, policy_version 1829617 (0.0006) [2023-12-27 04:43:25,201][105620] Updated weights for policy 1, policy_version 1833637 (0.0011) [2023-12-27 04:43:25,249][105692] Updated weights for policy 0, policy_version 1829627 (0.0007) [2023-12-27 04:43:25,252][105620] Updated weights for policy 1, policy_version 1833647 (0.0006) [2023-12-27 04:43:25,299][105692] Updated weights for policy 0, policy_version 1829637 (0.0010) [2023-12-27 04:43:25,354][105692] Updated weights for policy 0, policy_version 1829647 (0.0010) [2023-12-27 04:43:25,931][105620] Updated weights for policy 1, policy_version 1833657 (0.0006) [2023-12-27 04:43:25,982][105620] Updated weights for policy 1, policy_version 1833667 (0.0010) [2023-12-27 04:43:25,988][105692] Updated weights for policy 0, policy_version 1829657 (0.0006) [2023-12-27 04:43:26,037][105620] Updated weights for policy 1, policy_version 1833677 (0.0010) [2023-12-27 04:43:26,040][105692] Updated weights for policy 0, policy_version 1829667 (0.0006) [2023-12-27 04:43:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19633.0). Total num frames: 937943040. Throughput: 0: 9655.6, 1: 9960.7. Samples: 937959288. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:43:26,062][104569] Avg episode reward: [(0, '8444.586'), (1, '9075.074')] [2023-12-27 04:43:26,088][105620] Updated weights for policy 1, policy_version 1833687 (0.0010) [2023-12-27 04:43:26,098][105692] Updated weights for policy 0, policy_version 1829677 (0.0005) [2023-12-27 04:43:26,696][105692] Updated weights for policy 0, policy_version 1829687 (0.0006) [2023-12-27 04:43:26,750][105692] Updated weights for policy 0, policy_version 1829697 (0.0006) [2023-12-27 04:43:26,798][105620] Updated weights for policy 1, policy_version 1833697 (0.0011) [2023-12-27 04:43:26,804][105692] Updated weights for policy 0, policy_version 1829707 (0.0007) [2023-12-27 04:43:26,860][105620] Updated weights for policy 1, policy_version 1833707 (0.0010) [2023-12-27 04:43:26,926][105620] Updated weights for policy 1, policy_version 1833717 (0.0011) [2023-12-27 04:43:27,383][105692] Updated weights for policy 0, policy_version 1829717 (0.0006) [2023-12-27 04:43:27,449][105692] Updated weights for policy 0, policy_version 1829727 (0.0005) [2023-12-27 04:43:27,509][105692] Updated weights for policy 0, policy_version 1829737 (0.0005) [2023-12-27 04:43:27,553][105620] Updated weights for policy 1, policy_version 1833727 (0.0007) [2023-12-27 04:43:27,600][105620] Updated weights for policy 1, policy_version 1833737 (0.0007) [2023-12-27 04:43:27,644][105620] Updated weights for policy 1, policy_version 1833747 (0.0010) [2023-12-27 04:43:28,050][105692] Updated weights for policy 0, policy_version 1829747 (0.0005) [2023-12-27 04:43:28,107][105692] Updated weights for policy 0, policy_version 1829757 (0.0008) [2023-12-27 04:43:28,163][105692] Updated weights for policy 0, policy_version 1829767 (0.0012) [2023-12-27 04:43:28,307][105620] Updated weights for policy 1, policy_version 1833757 (0.0008) [2023-12-27 04:43:28,373][105620] Updated weights for policy 1, policy_version 1833767 (0.0008) [2023-12-27 04:43:28,432][105620] Updated weights for policy 1, policy_version 1833777 (0.0008) [2023-12-27 04:43:28,913][105692] Updated weights for policy 0, policy_version 1829778 (0.0009) [2023-12-27 04:43:28,980][105692] Updated weights for policy 0, policy_version 1829788 (0.0005) [2023-12-27 04:43:29,038][105692] Updated weights for policy 0, policy_version 1829798 (0.0008) [2023-12-27 04:43:29,088][105692] Updated weights for policy 0, policy_version 1829808 (0.0008) [2023-12-27 04:43:29,132][105620] Updated weights for policy 1, policy_version 1833787 (0.0009) [2023-12-27 04:43:29,196][105620] Updated weights for policy 1, policy_version 1833797 (0.0010) [2023-12-27 04:43:29,261][105620] Updated weights for policy 1, policy_version 1833807 (0.0011) [2023-12-27 04:43:29,825][105692] Updated weights for policy 0, policy_version 1829818 (0.0010) [2023-12-27 04:43:29,882][105692] Updated weights for policy 0, policy_version 1829828 (0.0010) [2023-12-27 04:43:29,941][105692] Updated weights for policy 0, policy_version 1829838 (0.0011) [2023-12-27 04:43:29,967][105620] Updated weights for policy 1, policy_version 1833817 (0.0010) [2023-12-27 04:43:30,026][105620] Updated weights for policy 1, policy_version 1833827 (0.0010) [2023-12-27 04:43:30,073][105620] Updated weights for policy 1, policy_version 1833837 (0.0010) [2023-12-27 04:43:30,121][105620] Updated weights for policy 1, policy_version 1833847 (0.0010) [2023-12-27 04:43:30,643][105692] Updated weights for policy 0, policy_version 1829848 (0.0010) [2023-12-27 04:43:30,689][105692] Updated weights for policy 0, policy_version 1829858 (0.0008) [2023-12-27 04:43:30,735][105692] Updated weights for policy 0, policy_version 1829868 (0.0005) [2023-12-27 04:43:30,899][105620] Updated weights for policy 1, policy_version 1833857 (0.0008) [2023-12-27 04:43:30,962][105620] Updated weights for policy 1, policy_version 1833867 (0.0008) [2023-12-27 04:43:31,014][105620] Updated weights for policy 1, policy_version 1833877 (0.0007) [2023-12-27 04:43:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19716.3). Total num frames: 938057728. Throughput: 0: 9774.7, 1: 9930.5. Samples: 938023820. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:43:31,062][104569] Avg episode reward: [(0, '8448.902'), (1, '9074.902')] [2023-12-27 04:43:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001829872_468516864.pth... [2023-12-27 04:43:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001833880_469540864.pth... [2023-12-27 04:43:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001832696_469237760.pth [2023-12-27 04:43:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001828720_468221952.pth [2023-12-27 04:43:31,498][105692] Updated weights for policy 0, policy_version 1829878 (0.0008) [2023-12-27 04:43:31,565][105692] Updated weights for policy 0, policy_version 1829888 (0.0009) [2023-12-27 04:43:31,631][105692] Updated weights for policy 0, policy_version 1829898 (0.0006) [2023-12-27 04:43:31,774][105620] Updated weights for policy 1, policy_version 1833887 (0.0008) [2023-12-27 04:43:31,835][105620] Updated weights for policy 1, policy_version 1833897 (0.0008) [2023-12-27 04:43:31,894][105620] Updated weights for policy 1, policy_version 1833907 (0.0008) [2023-12-27 04:43:32,287][105692] Updated weights for policy 0, policy_version 1829908 (0.0009) [2023-12-27 04:43:32,356][105692] Updated weights for policy 0, policy_version 1829918 (0.0008) [2023-12-27 04:43:32,423][105692] Updated weights for policy 0, policy_version 1829928 (0.0009) [2023-12-27 04:43:32,688][105620] Updated weights for policy 1, policy_version 1833917 (0.0008) [2023-12-27 04:43:32,749][105620] Updated weights for policy 1, policy_version 1833927 (0.0008) [2023-12-27 04:43:32,813][105620] Updated weights for policy 1, policy_version 1833937 (0.0009) [2023-12-27 04:43:33,156][105692] Updated weights for policy 0, policy_version 1829938 (0.0007) [2023-12-27 04:43:33,204][105692] Updated weights for policy 0, policy_version 1829948 (0.0009) [2023-12-27 04:43:33,253][105692] Updated weights for policy 0, policy_version 1829958 (0.0009) [2023-12-27 04:43:33,306][105692] Updated weights for policy 0, policy_version 1829968 (0.0010) [2023-12-27 04:43:33,530][105620] Updated weights for policy 1, policy_version 1833947 (0.0008) [2023-12-27 04:43:33,583][105620] Updated weights for policy 1, policy_version 1833957 (0.0005) [2023-12-27 04:43:33,636][105620] Updated weights for policy 1, policy_version 1833967 (0.0005) [2023-12-27 04:43:33,989][105692] Updated weights for policy 0, policy_version 1829978 (0.0006) [2023-12-27 04:43:34,039][105692] Updated weights for policy 0, policy_version 1829988 (0.0006) [2023-12-27 04:43:34,100][105692] Updated weights for policy 0, policy_version 1829998 (0.0008) [2023-12-27 04:43:34,339][105620] Updated weights for policy 1, policy_version 1833977 (0.0008) [2023-12-27 04:43:34,409][105620] Updated weights for policy 1, policy_version 1833987 (0.0011) [2023-12-27 04:43:34,468][105620] Updated weights for policy 1, policy_version 1833997 (0.0010) [2023-12-27 04:43:34,531][105620] Updated weights for policy 1, policy_version 1834007 (0.0011) [2023-12-27 04:43:34,741][105692] Updated weights for policy 0, policy_version 1830008 (0.0006) [2023-12-27 04:43:34,794][105692] Updated weights for policy 0, policy_version 1830018 (0.0005) [2023-12-27 04:43:34,857][105692] Updated weights for policy 0, policy_version 1830028 (0.0007) [2023-12-27 04:43:35,238][105620] Updated weights for policy 1, policy_version 1834017 (0.0006) [2023-12-27 04:43:35,298][105620] Updated weights for policy 1, policy_version 1834027 (0.0008) [2023-12-27 04:43:35,353][105620] Updated weights for policy 1, policy_version 1834037 (0.0010) [2023-12-27 04:43:35,413][105692] Updated weights for policy 0, policy_version 1830038 (0.0006) [2023-12-27 04:43:35,481][105692] Updated weights for policy 0, policy_version 1830048 (0.0006) [2023-12-27 04:43:35,548][105692] Updated weights for policy 0, policy_version 1830058 (0.0005) [2023-12-27 04:43:36,047][105692] Updated weights for policy 0, policy_version 1830068 (0.0006) [2023-12-27 04:43:36,058][105620] Updated weights for policy 1, policy_version 1834047 (0.0011) [2023-12-27 04:43:36,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.9, 300 sec: 19688.6). Total num frames: 938147840. Throughput: 0: 9784.9, 1: 9900.5. Samples: 938139828. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:43:36,062][104569] Avg episode reward: [(0, '8622.959'), (1, '9350.230')] [2023-12-27 04:43:36,109][105620] Updated weights for policy 1, policy_version 1834057 (0.0011) [2023-12-27 04:43:36,111][105692] Updated weights for policy 0, policy_version 1830078 (0.0006) [2023-12-27 04:43:36,164][105692] Updated weights for policy 0, policy_version 1830088 (0.0006) [2023-12-27 04:43:36,166][105620] Updated weights for policy 1, policy_version 1834067 (0.0010) [2023-12-27 04:43:36,860][105620] Updated weights for policy 1, policy_version 1834077 (0.0010) [2023-12-27 04:43:36,912][105620] Updated weights for policy 1, policy_version 1834087 (0.0010) [2023-12-27 04:43:36,928][105692] Updated weights for policy 0, policy_version 1830098 (0.0008) [2023-12-27 04:43:36,963][105620] Updated weights for policy 1, policy_version 1834097 (0.0010) [2023-12-27 04:43:36,981][105692] Updated weights for policy 0, policy_version 1830108 (0.0007) [2023-12-27 04:43:37,032][105692] Updated weights for policy 0, policy_version 1830118 (0.0007) [2023-12-27 04:43:37,085][105692] Updated weights for policy 0, policy_version 1830128 (0.0008) [2023-12-27 04:43:37,678][105692] Updated weights for policy 0, policy_version 1830138 (0.0008) [2023-12-27 04:43:37,691][105620] Updated weights for policy 1, policy_version 1834107 (0.0010) [2023-12-27 04:43:37,734][105692] Updated weights for policy 0, policy_version 1830148 (0.0007) [2023-12-27 04:43:37,751][105620] Updated weights for policy 1, policy_version 1834117 (0.0008) [2023-12-27 04:43:37,798][105692] Updated weights for policy 0, policy_version 1830158 (0.0006) [2023-12-27 04:43:37,808][105620] Updated weights for policy 1, policy_version 1834127 (0.0010) [2023-12-27 04:43:38,466][105692] Updated weights for policy 0, policy_version 1830168 (0.0009) [2023-12-27 04:43:38,513][105620] Updated weights for policy 1, policy_version 1834137 (0.0010) [2023-12-27 04:43:38,519][105692] Updated weights for policy 0, policy_version 1830178 (0.0010) [2023-12-27 04:43:38,570][105620] Updated weights for policy 1, policy_version 1834147 (0.0006) [2023-12-27 04:43:38,581][105692] Updated weights for policy 0, policy_version 1830188 (0.0008) [2023-12-27 04:43:38,629][105620] Updated weights for policy 1, policy_version 1834157 (0.0006) [2023-12-27 04:43:38,693][105620] Updated weights for policy 1, policy_version 1834167 (0.0006) [2023-12-27 04:43:39,278][105692] Updated weights for policy 0, policy_version 1830198 (0.0007) [2023-12-27 04:43:39,338][105692] Updated weights for policy 0, policy_version 1830208 (0.0008) [2023-12-27 04:43:39,346][105620] Updated weights for policy 1, policy_version 1834177 (0.0008) [2023-12-27 04:43:39,408][105692] Updated weights for policy 0, policy_version 1830218 (0.0010) [2023-12-27 04:43:39,419][105620] Updated weights for policy 1, policy_version 1834187 (0.0007) [2023-12-27 04:43:39,478][105620] Updated weights for policy 1, policy_version 1834197 (0.0007) [2023-12-27 04:43:40,158][105620] Updated weights for policy 1, policy_version 1834207 (0.0009) [2023-12-27 04:43:40,211][105620] Updated weights for policy 1, policy_version 1834217 (0.0008) [2023-12-27 04:43:40,215][105692] Updated weights for policy 0, policy_version 1830228 (0.0008) [2023-12-27 04:43:40,259][105620] Updated weights for policy 1, policy_version 1834227 (0.0007) [2023-12-27 04:43:40,280][105692] Updated weights for policy 0, policy_version 1830238 (0.0010) [2023-12-27 04:43:40,346][105692] Updated weights for policy 0, policy_version 1830248 (0.0009) [2023-12-27 04:43:40,936][105620] Updated weights for policy 1, policy_version 1834237 (0.0007) [2023-12-27 04:43:40,994][105620] Updated weights for policy 1, policy_version 1834247 (0.0006) [2023-12-27 04:43:41,061][105620] Updated weights for policy 1, policy_version 1834257 (0.0008) [2023-12-27 04:43:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 938246144. Throughput: 0: 9972.0, 1: 9858.2. Samples: 938262276. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:43:41,062][104569] Avg episode reward: [(0, '8353.687'), (1, '9350.101')] [2023-12-27 04:43:41,165][105692] Updated weights for policy 0, policy_version 1830258 (0.0009) [2023-12-27 04:43:41,232][105692] Updated weights for policy 0, policy_version 1830268 (0.0008) [2023-12-27 04:43:41,293][105692] Updated weights for policy 0, policy_version 1830278 (0.0008) [2023-12-27 04:43:41,344][105692] Updated weights for policy 0, policy_version 1830288 (0.0006) [2023-12-27 04:43:41,817][105620] Updated weights for policy 1, policy_version 1834267 (0.0010) [2023-12-27 04:43:41,870][105620] Updated weights for policy 1, policy_version 1834277 (0.0009) [2023-12-27 04:43:41,935][105620] Updated weights for policy 1, policy_version 1834287 (0.0009) [2023-12-27 04:43:42,045][105692] Updated weights for policy 0, policy_version 1830298 (0.0006) [2023-12-27 04:43:42,107][105692] Updated weights for policy 0, policy_version 1830308 (0.0006) [2023-12-27 04:43:42,170][105692] Updated weights for policy 0, policy_version 1830318 (0.0005) [2023-12-27 04:43:42,737][105620] Updated weights for policy 1, policy_version 1834297 (0.0008) [2023-12-27 04:43:42,793][105620] Updated weights for policy 1, policy_version 1834307 (0.0010) [2023-12-27 04:43:42,854][105620] Updated weights for policy 1, policy_version 1834317 (0.0010) [2023-12-27 04:43:42,876][105692] Updated weights for policy 0, policy_version 1830328 (0.0009) [2023-12-27 04:43:42,914][105620] Updated weights for policy 1, policy_version 1834327 (0.0011) [2023-12-27 04:43:42,939][105692] Updated weights for policy 0, policy_version 1830338 (0.0006) [2023-12-27 04:43:43,000][105692] Updated weights for policy 0, policy_version 1830348 (0.0008) [2023-12-27 04:43:43,649][105620] Updated weights for policy 1, policy_version 1834337 (0.0007) [2023-12-27 04:43:43,708][105620] Updated weights for policy 1, policy_version 1834347 (0.0007) [2023-12-27 04:43:43,762][105692] Updated weights for policy 0, policy_version 1830358 (0.0009) [2023-12-27 04:43:43,777][105620] Updated weights for policy 1, policy_version 1834357 (0.0007) [2023-12-27 04:43:43,817][105692] Updated weights for policy 0, policy_version 1830368 (0.0010) [2023-12-27 04:43:43,868][105692] Updated weights for policy 0, policy_version 1830378 (0.0010) [2023-12-27 04:43:44,294][105620] Updated weights for policy 1, policy_version 1834367 (0.0007) [2023-12-27 04:43:44,363][105620] Updated weights for policy 1, policy_version 1834377 (0.0009) [2023-12-27 04:43:44,417][105620] Updated weights for policy 1, policy_version 1834387 (0.0008) [2023-12-27 04:43:44,557][105692] Updated weights for policy 0, policy_version 1830388 (0.0008) [2023-12-27 04:43:44,606][105692] Updated weights for policy 0, policy_version 1830398 (0.0006) [2023-12-27 04:43:44,657][105692] Updated weights for policy 0, policy_version 1830408 (0.0010) [2023-12-27 04:43:45,043][105620] Updated weights for policy 1, policy_version 1834397 (0.0008) [2023-12-27 04:43:45,104][105620] Updated weights for policy 1, policy_version 1834407 (0.0010) [2023-12-27 04:43:45,163][105620] Updated weights for policy 1, policy_version 1834417 (0.0008) [2023-12-27 04:43:45,398][105692] Updated weights for policy 0, policy_version 1830418 (0.0009) [2023-12-27 04:43:45,451][105692] Updated weights for policy 0, policy_version 1830428 (0.0008) [2023-12-27 04:43:45,495][105692] Updated weights for policy 0, policy_version 1830438 (0.0008) [2023-12-27 04:43:45,547][105692] Updated weights for policy 0, policy_version 1830448 (0.0009) [2023-12-27 04:43:45,876][105620] Updated weights for policy 1, policy_version 1834427 (0.0011) [2023-12-27 04:43:45,924][105620] Updated weights for policy 1, policy_version 1834437 (0.0010) [2023-12-27 04:43:45,972][105620] Updated weights for policy 1, policy_version 1834447 (0.0010) [2023-12-27 04:43:46,062][104569] Fps is (10 sec: 20479.3, 60 sec: 19797.2, 300 sec: 19716.3). Total num frames: 938352640. Throughput: 0: 9883.3, 1: 9859.3. Samples: 938318324. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:43:46,063][104569] Avg episode reward: [(0, '8535.882'), (1, '9165.914')] [2023-12-27 04:43:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001834456_469688320.pth... [2023-12-27 04:43:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001830448_468664320.pth... [2023-12-27 04:43:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001833304_469393408.pth [2023-12-27 04:43:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001829296_468369408.pth [2023-12-27 04:43:46,326][105692] Updated weights for policy 0, policy_version 1830458 (0.0010) [2023-12-27 04:43:46,384][105692] Updated weights for policy 0, policy_version 1830468 (0.0010) [2023-12-27 04:43:46,436][105692] Updated weights for policy 0, policy_version 1830478 (0.0008) [2023-12-27 04:43:46,613][105620] Updated weights for policy 1, policy_version 1834457 (0.0010) [2023-12-27 04:43:46,661][105620] Updated weights for policy 1, policy_version 1834467 (0.0010) [2023-12-27 04:43:46,719][105620] Updated weights for policy 1, policy_version 1834477 (0.0010) [2023-12-27 04:43:46,767][105620] Updated weights for policy 1, policy_version 1834487 (0.0010) [2023-12-27 04:43:47,282][105692] Updated weights for policy 0, policy_version 1830488 (0.0009) [2023-12-27 04:43:47,338][105692] Updated weights for policy 0, policy_version 1830498 (0.0010) [2023-12-27 04:43:47,396][105692] Updated weights for policy 0, policy_version 1830508 (0.0008) [2023-12-27 04:43:47,410][105620] Updated weights for policy 1, policy_version 1834497 (0.0007) [2023-12-27 04:43:47,475][105620] Updated weights for policy 1, policy_version 1834507 (0.0005) [2023-12-27 04:43:47,536][105620] Updated weights for policy 1, policy_version 1834517 (0.0009) [2023-12-27 04:43:48,156][105692] Updated weights for policy 0, policy_version 1830518 (0.0008) [2023-12-27 04:43:48,174][105620] Updated weights for policy 1, policy_version 1834527 (0.0007) [2023-12-27 04:43:48,220][105692] Updated weights for policy 0, policy_version 1830528 (0.0008) [2023-12-27 04:43:48,232][105620] Updated weights for policy 1, policy_version 1834537 (0.0007) [2023-12-27 04:43:48,283][105692] Updated weights for policy 0, policy_version 1830538 (0.0009) [2023-12-27 04:43:48,291][105620] Updated weights for policy 1, policy_version 1834547 (0.0006) [2023-12-27 04:43:48,926][105620] Updated weights for policy 1, policy_version 1834557 (0.0006) [2023-12-27 04:43:48,978][105620] Updated weights for policy 1, policy_version 1834567 (0.0006) [2023-12-27 04:43:49,027][105620] Updated weights for policy 1, policy_version 1834577 (0.0007) [2023-12-27 04:43:49,098][105692] Updated weights for policy 0, policy_version 1830548 (0.0009) [2023-12-27 04:43:49,164][105692] Updated weights for policy 0, policy_version 1830558 (0.0010) [2023-12-27 04:43:49,227][105692] Updated weights for policy 0, policy_version 1830568 (0.0010) [2023-12-27 04:43:49,721][105620] Updated weights for policy 1, policy_version 1834587 (0.0007) [2023-12-27 04:43:49,784][105620] Updated weights for policy 1, policy_version 1834597 (0.0005) [2023-12-27 04:43:49,849][105620] Updated weights for policy 1, policy_version 1834607 (0.0007) [2023-12-27 04:43:49,996][105692] Updated weights for policy 0, policy_version 1830578 (0.0009) [2023-12-27 04:43:50,056][105692] Updated weights for policy 0, policy_version 1830588 (0.0009) [2023-12-27 04:43:50,114][105692] Updated weights for policy 0, policy_version 1830598 (0.0007) [2023-12-27 04:43:50,165][105692] Updated weights for policy 0, policy_version 1830608 (0.0008) [2023-12-27 04:43:50,595][105620] Updated weights for policy 1, policy_version 1834617 (0.0010) [2023-12-27 04:43:50,654][105620] Updated weights for policy 1, policy_version 1834627 (0.0009) [2023-12-27 04:43:50,709][105620] Updated weights for policy 1, policy_version 1834637 (0.0008) [2023-12-27 04:43:50,760][105620] Updated weights for policy 1, policy_version 1834647 (0.0009) [2023-12-27 04:43:50,903][105692] Updated weights for policy 0, policy_version 1830618 (0.0008) [2023-12-27 04:43:50,965][105692] Updated weights for policy 0, policy_version 1830628 (0.0009) [2023-12-27 04:43:51,028][105692] Updated weights for policy 0, policy_version 1830638 (0.0008) [2023-12-27 04:43:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19744.1). Total num frames: 938450944. Throughput: 0: 9863.9, 1: 9954.8. Samples: 938436788. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:43:51,062][104569] Avg episode reward: [(0, '8622.838'), (1, '9165.970')] [2023-12-27 04:43:51,466][105620] Updated weights for policy 1, policy_version 1834657 (0.0009) [2023-12-27 04:43:51,524][105620] Updated weights for policy 1, policy_version 1834667 (0.0009) [2023-12-27 04:43:51,578][105620] Updated weights for policy 1, policy_version 1834677 (0.0009) [2023-12-27 04:43:51,843][105692] Updated weights for policy 0, policy_version 1830648 (0.0009) [2023-12-27 04:43:51,900][105692] Updated weights for policy 0, policy_version 1830658 (0.0009) [2023-12-27 04:43:51,964][105692] Updated weights for policy 0, policy_version 1830668 (0.0009) [2023-12-27 04:43:52,253][105620] Updated weights for policy 1, policy_version 1834687 (0.0006) [2023-12-27 04:43:52,310][105620] Updated weights for policy 1, policy_version 1834697 (0.0006) [2023-12-27 04:43:52,372][105620] Updated weights for policy 1, policy_version 1834707 (0.0007) [2023-12-27 04:43:52,812][105692] Updated weights for policy 0, policy_version 1830678 (0.0010) [2023-12-27 04:43:52,871][105692] Updated weights for policy 0, policy_version 1830688 (0.0009) [2023-12-27 04:43:52,930][105692] Updated weights for policy 0, policy_version 1830698 (0.0009) [2023-12-27 04:43:53,029][105620] Updated weights for policy 1, policy_version 1834717 (0.0008) [2023-12-27 04:43:53,084][105620] Updated weights for policy 1, policy_version 1834727 (0.0009) [2023-12-27 04:43:53,139][105620] Updated weights for policy 1, policy_version 1834737 (0.0008) [2023-12-27 04:43:53,732][105692] Updated weights for policy 0, policy_version 1830708 (0.0009) [2023-12-27 04:43:53,791][105692] Updated weights for policy 0, policy_version 1830718 (0.0009) [2023-12-27 04:43:53,826][105620] Updated weights for policy 1, policy_version 1834747 (0.0008) [2023-12-27 04:43:53,839][105692] Updated weights for policy 0, policy_version 1830728 (0.0009) [2023-12-27 04:43:53,877][105620] Updated weights for policy 1, policy_version 1834757 (0.0007) [2023-12-27 04:43:53,930][105620] Updated weights for policy 1, policy_version 1834767 (0.0009) [2023-12-27 04:43:54,582][105620] Updated weights for policy 1, policy_version 1834777 (0.0008) [2023-12-27 04:43:54,647][105620] Updated weights for policy 1, policy_version 1834787 (0.0007) [2023-12-27 04:43:54,661][105692] Updated weights for policy 0, policy_version 1830738 (0.0006) [2023-12-27 04:43:54,719][105692] Updated weights for policy 0, policy_version 1830748 (0.0007) [2023-12-27 04:43:54,728][105620] Updated weights for policy 1, policy_version 1834797 (0.0008) [2023-12-27 04:43:54,778][105692] Updated weights for policy 0, policy_version 1830758 (0.0006) [2023-12-27 04:43:54,789][105620] Updated weights for policy 1, policy_version 1834807 (0.0008) [2023-12-27 04:43:54,838][105692] Updated weights for policy 0, policy_version 1830768 (0.0008) [2023-12-27 04:43:55,441][105620] Updated weights for policy 1, policy_version 1834817 (0.0008) [2023-12-27 04:43:55,493][105620] Updated weights for policy 1, policy_version 1834827 (0.0005) [2023-12-27 04:43:55,551][105620] Updated weights for policy 1, policy_version 1834837 (0.0008) [2023-12-27 04:43:55,596][105692] Updated weights for policy 0, policy_version 1830778 (0.0008) [2023-12-27 04:43:55,648][105692] Updated weights for policy 0, policy_version 1830788 (0.0009) [2023-12-27 04:43:55,706][105692] Updated weights for policy 0, policy_version 1830798 (0.0009) [2023-12-27 04:43:56,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19797.3, 300 sec: 19716.3). Total num frames: 938541056. Throughput: 0: 9788.3, 1: 9920.3. Samples: 938550656. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:43:56,063][104569] Avg episode reward: [(0, '8624.727'), (1, '9257.909')] [2023-12-27 04:43:56,257][105620] Updated weights for policy 1, policy_version 1834847 (0.0008) [2023-12-27 04:43:56,326][105620] Updated weights for policy 1, policy_version 1834857 (0.0005) [2023-12-27 04:43:56,392][105620] Updated weights for policy 1, policy_version 1834867 (0.0006) [2023-12-27 04:43:56,513][105692] Updated weights for policy 0, policy_version 1830808 (0.0010) [2023-12-27 04:43:56,566][105692] Updated weights for policy 0, policy_version 1830818 (0.0010) [2023-12-27 04:43:56,624][105692] Updated weights for policy 0, policy_version 1830828 (0.0010) [2023-12-27 04:43:56,916][105620] Updated weights for policy 1, policy_version 1834877 (0.0007) [2023-12-27 04:43:56,971][105620] Updated weights for policy 1, policy_version 1834887 (0.0008) [2023-12-27 04:43:57,027][105620] Updated weights for policy 1, policy_version 1834897 (0.0006) [2023-12-27 04:43:57,514][105692] Updated weights for policy 0, policy_version 1830838 (0.0009) [2023-12-27 04:43:57,579][105692] Updated weights for policy 0, policy_version 1830848 (0.0009) [2023-12-27 04:43:57,640][105692] Updated weights for policy 0, policy_version 1830858 (0.0008) [2023-12-27 04:43:57,645][105620] Updated weights for policy 1, policy_version 1834907 (0.0006) [2023-12-27 04:43:57,700][105620] Updated weights for policy 1, policy_version 1834917 (0.0008) [2023-12-27 04:43:57,746][105620] Updated weights for policy 1, policy_version 1834927 (0.0008) [2023-12-27 04:43:58,382][105692] Updated weights for policy 0, policy_version 1830868 (0.0007) [2023-12-27 04:43:58,446][105692] Updated weights for policy 0, policy_version 1830878 (0.0009) [2023-12-27 04:43:58,509][105692] Updated weights for policy 0, policy_version 1830888 (0.0010) [2023-12-27 04:43:58,541][105620] Updated weights for policy 1, policy_version 1834937 (0.0008) [2023-12-27 04:43:58,600][105620] Updated weights for policy 1, policy_version 1834947 (0.0008) [2023-12-27 04:43:58,667][105620] Updated weights for policy 1, policy_version 1834957 (0.0009) [2023-12-27 04:43:58,735][105620] Updated weights for policy 1, policy_version 1834967 (0.0009) [2023-12-27 04:43:59,417][105692] Updated weights for policy 0, policy_version 1830898 (0.0007) [2023-12-27 04:43:59,477][105692] Updated weights for policy 0, policy_version 1830908 (0.0009) [2023-12-27 04:43:59,535][105692] Updated weights for policy 0, policy_version 1830918 (0.0009) [2023-12-27 04:43:59,601][105692] Updated weights for policy 0, policy_version 1830928 (0.0008) [2023-12-27 04:43:59,603][105620] Updated weights for policy 1, policy_version 1834977 (0.0007) [2023-12-27 04:43:59,664][105620] Updated weights for policy 1, policy_version 1834987 (0.0009) [2023-12-27 04:43:59,731][105620] Updated weights for policy 1, policy_version 1834997 (0.0009) [2023-12-27 04:44:00,387][105692] Updated weights for policy 0, policy_version 1830938 (0.0006) [2023-12-27 04:44:00,449][105692] Updated weights for policy 0, policy_version 1830948 (0.0007) [2023-12-27 04:44:00,479][105620] Updated weights for policy 1, policy_version 1835007 (0.0010) [2023-12-27 04:44:00,507][105692] Updated weights for policy 0, policy_version 1830958 (0.0007) [2023-12-27 04:44:00,543][105620] Updated weights for policy 1, policy_version 1835017 (0.0010) [2023-12-27 04:44:00,611][105620] Updated weights for policy 1, policy_version 1835028 (0.0011) [2023-12-27 04:44:01,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 938631168. Throughput: 0: 9683.5, 1: 9945.9. Samples: 938606568. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:44:01,062][104569] Avg episode reward: [(0, '8624.447'), (1, '9257.854')] [2023-12-27 04:44:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001835032_469835776.pth... [2023-12-27 04:44:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001833880_469540864.pth [2023-12-27 04:44:01,090][105692] Updated weights for policy 0, policy_version 1830968 (0.0007) [2023-12-27 04:44:01,141][105692] Updated weights for policy 0, policy_version 1830978 (0.0006) [2023-12-27 04:44:01,202][105692] Updated weights for policy 0, policy_version 1830988 (0.0007) [2023-12-27 04:44:01,224][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001830992_468803584.pth... [2023-12-27 04:44:01,227][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001829872_468516864.pth [2023-12-27 04:44:01,285][105620] Updated weights for policy 1, policy_version 1835038 (0.0010) [2023-12-27 04:44:01,348][105620] Updated weights for policy 1, policy_version 1835048 (0.0010) [2023-12-27 04:44:01,417][105620] Updated weights for policy 1, policy_version 1835058 (0.0009) [2023-12-27 04:44:01,895][105692] Updated weights for policy 0, policy_version 1830998 (0.0008) [2023-12-27 04:44:01,950][105692] Updated weights for policy 0, policy_version 1831009 (0.0009) [2023-12-27 04:44:02,005][105692] Updated weights for policy 0, policy_version 1831019 (0.0010) [2023-12-27 04:44:02,074][105620] Updated weights for policy 1, policy_version 1835068 (0.0008) [2023-12-27 04:44:02,130][105620] Updated weights for policy 1, policy_version 1835078 (0.0008) [2023-12-27 04:44:02,188][105620] Updated weights for policy 1, policy_version 1835088 (0.0009) [2023-12-27 04:44:02,806][105692] Updated weights for policy 0, policy_version 1831029 (0.0007) [2023-12-27 04:44:02,860][105692] Updated weights for policy 0, policy_version 1831039 (0.0005) [2023-12-27 04:44:02,926][105692] Updated weights for policy 0, policy_version 1831049 (0.0005) [2023-12-27 04:44:02,930][105620] Updated weights for policy 1, policy_version 1835098 (0.0009) [2023-12-27 04:44:02,984][105620] Updated weights for policy 1, policy_version 1835108 (0.0009) [2023-12-27 04:44:03,046][105620] Updated weights for policy 1, policy_version 1835118 (0.0010) [2023-12-27 04:44:03,106][105620] Updated weights for policy 1, policy_version 1835128 (0.0009) [2023-12-27 04:44:03,596][105692] Updated weights for policy 0, policy_version 1831059 (0.0007) [2023-12-27 04:44:03,650][105692] Updated weights for policy 0, policy_version 1831069 (0.0010) [2023-12-27 04:44:03,705][105692] Updated weights for policy 0, policy_version 1831080 (0.0010) [2023-12-27 04:44:03,733][105620] Updated weights for policy 1, policy_version 1835138 (0.0007) [2023-12-27 04:44:03,785][105620] Updated weights for policy 1, policy_version 1835148 (0.0010) [2023-12-27 04:44:03,847][105620] Updated weights for policy 1, policy_version 1835158 (0.0006) [2023-12-27 04:44:04,495][105692] Updated weights for policy 0, policy_version 1831090 (0.0007) [2023-12-27 04:44:04,546][105692] Updated weights for policy 0, policy_version 1831100 (0.0008) [2023-12-27 04:44:04,549][105620] Updated weights for policy 1, policy_version 1835168 (0.0007) [2023-12-27 04:44:04,596][105692] Updated weights for policy 0, policy_version 1831110 (0.0008) [2023-12-27 04:44:04,603][105620] Updated weights for policy 1, policy_version 1835178 (0.0007) [2023-12-27 04:44:04,646][105692] Updated weights for policy 0, policy_version 1831120 (0.0008) [2023-12-27 04:44:04,649][105620] Updated weights for policy 1, policy_version 1835188 (0.0006) [2023-12-27 04:44:05,226][105620] Updated weights for policy 1, policy_version 1835198 (0.0007) [2023-12-27 04:44:05,283][105620] Updated weights for policy 1, policy_version 1835208 (0.0005) [2023-12-27 04:44:05,347][105620] Updated weights for policy 1, policy_version 1835218 (0.0005) [2023-12-27 04:44:05,541][105692] Updated weights for policy 0, policy_version 1831130 (0.0008) [2023-12-27 04:44:05,597][105692] Updated weights for policy 0, policy_version 1831140 (0.0008) [2023-12-27 04:44:05,656][105692] Updated weights for policy 0, policy_version 1831150 (0.0008) [2023-12-27 04:44:05,930][105620] Updated weights for policy 1, policy_version 1835228 (0.0005) [2023-12-27 04:44:05,990][105620] Updated weights for policy 1, policy_version 1835238 (0.0008) [2023-12-27 04:44:06,044][105620] Updated weights for policy 1, policy_version 1835248 (0.0005) [2023-12-27 04:44:06,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19660.8). Total num frames: 938729472. Throughput: 0: 9644.1, 1: 9956.0. Samples: 938722024. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:44:06,063][104569] Avg episode reward: [(0, '8351.738'), (1, '9258.441')] [2023-12-27 04:44:06,442][105692] Updated weights for policy 0, policy_version 1831160 (0.0008) [2023-12-27 04:44:06,505][105692] Updated weights for policy 0, policy_version 1831170 (0.0006) [2023-12-27 04:44:06,562][105692] Updated weights for policy 0, policy_version 1831180 (0.0006) [2023-12-27 04:44:06,731][105620] Updated weights for policy 1, policy_version 1835258 (0.0006) [2023-12-27 04:44:06,786][105620] Updated weights for policy 1, policy_version 1835268 (0.0009) [2023-12-27 04:44:06,851][105620] Updated weights for policy 1, policy_version 1835278 (0.0009) [2023-12-27 04:44:06,910][105620] Updated weights for policy 1, policy_version 1835288 (0.0007) [2023-12-27 04:44:07,234][105692] Updated weights for policy 0, policy_version 1831190 (0.0006) [2023-12-27 04:44:07,286][105692] Updated weights for policy 0, policy_version 1831200 (0.0005) [2023-12-27 04:44:07,340][105692] Updated weights for policy 0, policy_version 1831210 (0.0006) [2023-12-27 04:44:07,552][105620] Updated weights for policy 1, policy_version 1835298 (0.0005) [2023-12-27 04:44:07,613][105620] Updated weights for policy 1, policy_version 1835308 (0.0007) [2023-12-27 04:44:07,665][105620] Updated weights for policy 1, policy_version 1835318 (0.0008) [2023-12-27 04:44:07,973][105692] Updated weights for policy 0, policy_version 1831220 (0.0009) [2023-12-27 04:44:08,043][105692] Updated weights for policy 0, policy_version 1831230 (0.0006) [2023-12-27 04:44:08,097][105692] Updated weights for policy 0, policy_version 1831240 (0.0009) [2023-12-27 04:44:08,453][105620] Updated weights for policy 1, policy_version 1835328 (0.0009) [2023-12-27 04:44:08,515][105620] Updated weights for policy 1, policy_version 1835338 (0.0009) [2023-12-27 04:44:08,582][105620] Updated weights for policy 1, policy_version 1835348 (0.0010) [2023-12-27 04:44:08,753][105692] Updated weights for policy 0, policy_version 1831250 (0.0009) [2023-12-27 04:44:08,814][105692] Updated weights for policy 0, policy_version 1831260 (0.0009) [2023-12-27 04:44:08,869][105692] Updated weights for policy 0, policy_version 1831270 (0.0009) [2023-12-27 04:44:08,930][105692] Updated weights for policy 0, policy_version 1831280 (0.0009) [2023-12-27 04:44:09,319][105620] Updated weights for policy 1, policy_version 1835358 (0.0008) [2023-12-27 04:44:09,386][105620] Updated weights for policy 1, policy_version 1835368 (0.0008) [2023-12-27 04:44:09,455][105620] Updated weights for policy 1, policy_version 1835378 (0.0008) [2023-12-27 04:44:09,714][105692] Updated weights for policy 0, policy_version 1831290 (0.0007) [2023-12-27 04:44:09,774][105692] Updated weights for policy 0, policy_version 1831300 (0.0008) [2023-12-27 04:44:09,832][105692] Updated weights for policy 0, policy_version 1831310 (0.0009) [2023-12-27 04:44:10,262][105620] Updated weights for policy 1, policy_version 1835388 (0.0008) [2023-12-27 04:44:10,320][105620] Updated weights for policy 1, policy_version 1835398 (0.0008) [2023-12-27 04:44:10,378][105620] Updated weights for policy 1, policy_version 1835408 (0.0008) [2023-12-27 04:44:10,490][105692] Updated weights for policy 0, policy_version 1831320 (0.0010) [2023-12-27 04:44:10,555][105692] Updated weights for policy 0, policy_version 1831330 (0.0010) [2023-12-27 04:44:10,617][105692] Updated weights for policy 0, policy_version 1831340 (0.0009) [2023-12-27 04:44:11,006][105620] Updated weights for policy 1, policy_version 1835418 (0.0007) [2023-12-27 04:44:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 938827776. Throughput: 0: 9587.3, 1: 9981.9. Samples: 938839900. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:44:11,062][104569] Avg episode reward: [(0, '8624.594'), (1, '9258.341')] [2023-12-27 04:44:11,071][105620] Updated weights for policy 1, policy_version 1835428 (0.0008) [2023-12-27 04:44:11,126][105620] Updated weights for policy 1, policy_version 1835438 (0.0009) [2023-12-27 04:44:11,188][105620] Updated weights for policy 1, policy_version 1835448 (0.0009) [2023-12-27 04:44:11,419][105692] Updated weights for policy 0, policy_version 1831350 (0.0010) [2023-12-27 04:44:11,474][105692] Updated weights for policy 0, policy_version 1831360 (0.0010) [2023-12-27 04:44:11,528][105692] Updated weights for policy 0, policy_version 1831371 (0.0010) [2023-12-27 04:44:11,895][105620] Updated weights for policy 1, policy_version 1835458 (0.0006) [2023-12-27 04:44:11,955][105620] Updated weights for policy 1, policy_version 1835468 (0.0006) [2023-12-27 04:44:12,016][105620] Updated weights for policy 1, policy_version 1835478 (0.0006) [2023-12-27 04:44:12,393][105692] Updated weights for policy 0, policy_version 1831381 (0.0009) [2023-12-27 04:44:12,453][105692] Updated weights for policy 0, policy_version 1831391 (0.0009) [2023-12-27 04:44:12,506][105692] Updated weights for policy 0, policy_version 1831401 (0.0008) [2023-12-27 04:44:12,653][105620] Updated weights for policy 1, policy_version 1835488 (0.0008) [2023-12-27 04:44:12,716][105620] Updated weights for policy 1, policy_version 1835498 (0.0009) [2023-12-27 04:44:12,786][105620] Updated weights for policy 1, policy_version 1835508 (0.0010) [2023-12-27 04:44:13,113][105692] Updated weights for policy 0, policy_version 1831411 (0.0009) [2023-12-27 04:44:13,171][105692] Updated weights for policy 0, policy_version 1831421 (0.0009) [2023-12-27 04:44:13,232][105692] Updated weights for policy 0, policy_version 1831431 (0.0009) [2023-12-27 04:44:13,554][105620] Updated weights for policy 1, policy_version 1835518 (0.0010) [2023-12-27 04:44:13,623][105620] Updated weights for policy 1, policy_version 1835528 (0.0010) [2023-12-27 04:44:13,690][105620] Updated weights for policy 1, policy_version 1835538 (0.0010) [2023-12-27 04:44:13,926][105692] Updated weights for policy 0, policy_version 1831441 (0.0009) [2023-12-27 04:44:13,973][105692] Updated weights for policy 0, policy_version 1831451 (0.0007) [2023-12-27 04:44:14,033][105692] Updated weights for policy 0, policy_version 1831461 (0.0005) [2023-12-27 04:44:14,087][105692] Updated weights for policy 0, policy_version 1831471 (0.0006) [2023-12-27 04:44:14,422][105620] Updated weights for policy 1, policy_version 1835548 (0.0008) [2023-12-27 04:44:14,496][105620] Updated weights for policy 1, policy_version 1835558 (0.0006) [2023-12-27 04:44:14,551][105620] Updated weights for policy 1, policy_version 1835568 (0.0008) [2023-12-27 04:44:14,813][105692] Updated weights for policy 0, policy_version 1831481 (0.0008) [2023-12-27 04:44:14,882][105692] Updated weights for policy 0, policy_version 1831491 (0.0008) [2023-12-27 04:44:14,945][105692] Updated weights for policy 0, policy_version 1831501 (0.0008) [2023-12-27 04:44:15,215][105620] Updated weights for policy 1, policy_version 1835578 (0.0008) [2023-12-27 04:44:15,277][105620] Updated weights for policy 1, policy_version 1835588 (0.0005) [2023-12-27 04:44:15,329][105620] Updated weights for policy 1, policy_version 1835598 (0.0009) [2023-12-27 04:44:15,387][105620] Updated weights for policy 1, policy_version 1835608 (0.0009) [2023-12-27 04:44:15,722][105692] Updated weights for policy 0, policy_version 1831511 (0.0006) [2023-12-27 04:44:15,768][105692] Updated weights for policy 0, policy_version 1831521 (0.0005) [2023-12-27 04:44:15,824][105692] Updated weights for policy 0, policy_version 1831531 (0.0005) [2023-12-27 04:44:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19716.3). Total num frames: 938926080. Throughput: 0: 9450.6, 1: 9949.2. Samples: 938896816. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:44:16,063][104569] Avg episode reward: [(0, '8260.945'), (1, '9349.909')] [2023-12-27 04:44:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001831536_468942848.pth... [2023-12-27 04:44:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001830448_468664320.pth [2023-12-27 04:44:16,097][105620] Updated weights for policy 1, policy_version 1835618 (0.0008) [2023-12-27 04:44:16,145][105620] Updated weights for policy 1, policy_version 1835628 (0.0005) [2023-12-27 04:44:16,204][105620] Updated weights for policy 1, policy_version 1835638 (0.0007) [2023-12-27 04:44:16,216][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001835640_469991424.pth... [2023-12-27 04:44:16,221][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001834456_469688320.pth [2023-12-27 04:44:16,582][105692] Updated weights for policy 0, policy_version 1831541 (0.0009) [2023-12-27 04:44:16,646][105692] Updated weights for policy 0, policy_version 1831551 (0.0008) [2023-12-27 04:44:16,709][105692] Updated weights for policy 0, policy_version 1831561 (0.0009) [2023-12-27 04:44:16,754][105620] Updated weights for policy 1, policy_version 1835648 (0.0008) [2023-12-27 04:44:16,802][105620] Updated weights for policy 1, policy_version 1835658 (0.0010) [2023-12-27 04:44:16,851][105620] Updated weights for policy 1, policy_version 1835668 (0.0010) [2023-12-27 04:44:17,397][105692] Updated weights for policy 0, policy_version 1831571 (0.0007) [2023-12-27 04:44:17,467][105692] Updated weights for policy 0, policy_version 1831581 (0.0009) [2023-12-27 04:44:17,535][105692] Updated weights for policy 0, policy_version 1831591 (0.0005) [2023-12-27 04:44:17,660][105620] Updated weights for policy 1, policy_version 1835678 (0.0010) [2023-12-27 04:44:17,731][105620] Updated weights for policy 1, policy_version 1835688 (0.0007) [2023-12-27 04:44:17,788][105620] Updated weights for policy 1, policy_version 1835698 (0.0009) [2023-12-27 04:44:18,252][105692] Updated weights for policy 0, policy_version 1831601 (0.0006) [2023-12-27 04:44:18,310][105692] Updated weights for policy 0, policy_version 1831611 (0.0008) [2023-12-27 04:44:18,365][105692] Updated weights for policy 0, policy_version 1831621 (0.0008) [2023-12-27 04:44:18,393][105620] Updated weights for policy 1, policy_version 1835708 (0.0007) [2023-12-27 04:44:18,407][105692] Updated weights for policy 0, policy_version 1831631 (0.0007) [2023-12-27 04:44:18,458][105620] Updated weights for policy 1, policy_version 1835718 (0.0011) [2023-12-27 04:44:18,524][105620] Updated weights for policy 1, policy_version 1835728 (0.0011) [2023-12-27 04:44:19,181][105692] Updated weights for policy 0, policy_version 1831641 (0.0008) [2023-12-27 04:44:19,244][105692] Updated weights for policy 0, policy_version 1831651 (0.0006) [2023-12-27 04:44:19,253][105620] Updated weights for policy 1, policy_version 1835738 (0.0010) [2023-12-27 04:44:19,302][105692] Updated weights for policy 0, policy_version 1831661 (0.0006) [2023-12-27 04:44:19,312][105620] Updated weights for policy 1, policy_version 1835748 (0.0008) [2023-12-27 04:44:19,391][105620] Updated weights for policy 1, policy_version 1835758 (0.0009) [2023-12-27 04:44:19,455][105620] Updated weights for policy 1, policy_version 1835768 (0.0007) [2023-12-27 04:44:20,056][105692] Updated weights for policy 0, policy_version 1831671 (0.0009) [2023-12-27 04:44:20,117][105692] Updated weights for policy 0, policy_version 1831681 (0.0011) [2023-12-27 04:44:20,130][105620] Updated weights for policy 1, policy_version 1835778 (0.0011) [2023-12-27 04:44:20,178][105692] Updated weights for policy 0, policy_version 1831691 (0.0011) [2023-12-27 04:44:20,187][105620] Updated weights for policy 1, policy_version 1835788 (0.0011) [2023-12-27 04:44:20,247][105620] Updated weights for policy 1, policy_version 1835798 (0.0011) [2023-12-27 04:44:20,925][105692] Updated weights for policy 0, policy_version 1831701 (0.0010) [2023-12-27 04:44:20,976][105620] Updated weights for policy 1, policy_version 1835808 (0.0009) [2023-12-27 04:44:20,986][105692] Updated weights for policy 0, policy_version 1831711 (0.0007) [2023-12-27 04:44:21,036][105620] Updated weights for policy 1, policy_version 1835818 (0.0009) [2023-12-27 04:44:21,054][105692] Updated weights for policy 0, policy_version 1831721 (0.0008) [2023-12-27 04:44:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19660.8). Total num frames: 939016192. Throughput: 0: 9405.7, 1: 10046.0. Samples: 939015152. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:44:21,062][104569] Avg episode reward: [(0, '8075.667'), (1, '9350.064')] [2023-12-27 04:44:21,112][105620] Updated weights for policy 1, policy_version 1835828 (0.0010) [2023-12-27 04:44:21,871][105692] Updated weights for policy 0, policy_version 1831731 (0.0007) [2023-12-27 04:44:21,897][105620] Updated weights for policy 1, policy_version 1835838 (0.0010) [2023-12-27 04:44:21,931][105692] Updated weights for policy 0, policy_version 1831741 (0.0005) [2023-12-27 04:44:21,960][105620] Updated weights for policy 1, policy_version 1835848 (0.0011) [2023-12-27 04:44:21,987][105692] Updated weights for policy 0, policy_version 1831751 (0.0006) [2023-12-27 04:44:22,010][105620] Updated weights for policy 1, policy_version 1835858 (0.0011) [2023-12-27 04:44:22,723][105692] Updated weights for policy 0, policy_version 1831761 (0.0008) [2023-12-27 04:44:22,774][105620] Updated weights for policy 1, policy_version 1835868 (0.0011) [2023-12-27 04:44:22,782][105692] Updated weights for policy 0, policy_version 1831771 (0.0008) [2023-12-27 04:44:22,834][105620] Updated weights for policy 1, policy_version 1835878 (0.0011) [2023-12-27 04:44:22,837][105692] Updated weights for policy 0, policy_version 1831781 (0.0006) [2023-12-27 04:44:22,896][105692] Updated weights for policy 0, policy_version 1831791 (0.0005) [2023-12-27 04:44:22,898][105620] Updated weights for policy 1, policy_version 1835888 (0.0011) [2023-12-27 04:44:23,625][105692] Updated weights for policy 0, policy_version 1831801 (0.0005) [2023-12-27 04:44:23,650][105620] Updated weights for policy 1, policy_version 1835898 (0.0010) [2023-12-27 04:44:23,675][105692] Updated weights for policy 0, policy_version 1831811 (0.0005) [2023-12-27 04:44:23,706][105620] Updated weights for policy 1, policy_version 1835908 (0.0007) [2023-12-27 04:44:23,730][105692] Updated weights for policy 0, policy_version 1831821 (0.0006) [2023-12-27 04:44:23,763][105620] Updated weights for policy 1, policy_version 1835918 (0.0008) [2023-12-27 04:44:23,823][105620] Updated weights for policy 1, policy_version 1835928 (0.0009) [2023-12-27 04:44:24,298][105692] Updated weights for policy 0, policy_version 1831831 (0.0006) [2023-12-27 04:44:24,365][105692] Updated weights for policy 0, policy_version 1831841 (0.0006) [2023-12-27 04:44:24,428][105692] Updated weights for policy 0, policy_version 1831851 (0.0005) [2023-12-27 04:44:24,530][105620] Updated weights for policy 1, policy_version 1835938 (0.0006) [2023-12-27 04:44:24,593][105620] Updated weights for policy 1, policy_version 1835948 (0.0007) [2023-12-27 04:44:24,659][105620] Updated weights for policy 1, policy_version 1835958 (0.0006) [2023-12-27 04:44:24,977][105692] Updated weights for policy 0, policy_version 1831861 (0.0006) [2023-12-27 04:44:25,042][105692] Updated weights for policy 0, policy_version 1831871 (0.0010) [2023-12-27 04:44:25,100][105692] Updated weights for policy 0, policy_version 1831881 (0.0009) [2023-12-27 04:44:25,186][105620] Updated weights for policy 1, policy_version 1835968 (0.0006) [2023-12-27 04:44:25,235][105620] Updated weights for policy 1, policy_version 1835978 (0.0005) [2023-12-27 04:44:25,301][105620] Updated weights for policy 1, policy_version 1835988 (0.0005) [2023-12-27 04:44:25,679][105692] Updated weights for policy 0, policy_version 1831891 (0.0010) [2023-12-27 04:44:25,729][105692] Updated weights for policy 0, policy_version 1831901 (0.0010) [2023-12-27 04:44:25,782][105692] Updated weights for policy 0, policy_version 1831911 (0.0005) [2023-12-27 04:44:25,824][105620] Updated weights for policy 1, policy_version 1835998 (0.0005) [2023-12-27 04:44:25,873][105620] Updated weights for policy 1, policy_version 1836008 (0.0005) [2023-12-27 04:44:25,923][105620] Updated weights for policy 1, policy_version 1836018 (0.0005) [2023-12-27 04:44:26,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19797.3, 300 sec: 19716.3). Total num frames: 939130880. Throughput: 0: 9370.7, 1: 10041.0. Samples: 939135800. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:44:26,062][104569] Avg episode reward: [(0, '8527.682'), (1, '9350.254')] [2023-12-27 04:44:26,356][105692] Updated weights for policy 0, policy_version 1831921 (0.0006) [2023-12-27 04:44:26,415][105692] Updated weights for policy 0, policy_version 1831931 (0.0009) [2023-12-27 04:44:26,442][105620] Updated weights for policy 1, policy_version 1836028 (0.0006) [2023-12-27 04:44:26,469][105692] Updated weights for policy 0, policy_version 1831941 (0.0010) [2023-12-27 04:44:26,498][105620] Updated weights for policy 1, policy_version 1836038 (0.0009) [2023-12-27 04:44:26,524][105692] Updated weights for policy 0, policy_version 1831951 (0.0010) [2023-12-27 04:44:26,544][105620] Updated weights for policy 1, policy_version 1836048 (0.0006) [2023-12-27 04:44:27,126][105692] Updated weights for policy 0, policy_version 1831961 (0.0006) [2023-12-27 04:44:27,174][105692] Updated weights for policy 0, policy_version 1831971 (0.0005) [2023-12-27 04:44:27,224][105692] Updated weights for policy 0, policy_version 1831981 (0.0005) [2023-12-27 04:44:27,366][105620] Updated weights for policy 1, policy_version 1836058 (0.0007) [2023-12-27 04:44:27,419][105620] Updated weights for policy 1, policy_version 1836068 (0.0005) [2023-12-27 04:44:27,475][105620] Updated weights for policy 1, policy_version 1836078 (0.0008) [2023-12-27 04:44:27,782][105692] Updated weights for policy 0, policy_version 1831991 (0.0005) [2023-12-27 04:44:27,854][105692] Updated weights for policy 0, policy_version 1832001 (0.0005) [2023-12-27 04:44:27,909][105692] Updated weights for policy 0, policy_version 1832011 (0.0005) [2023-12-27 04:44:28,073][105620] Updated weights for policy 1, policy_version 1836089 (0.0008) [2023-12-27 04:44:28,128][105620] Updated weights for policy 1, policy_version 1836099 (0.0005) [2023-12-27 04:44:28,185][105620] Updated weights for policy 1, policy_version 1836109 (0.0006) [2023-12-27 04:44:28,248][105620] Updated weights for policy 1, policy_version 1836119 (0.0006) [2023-12-27 04:44:28,436][105692] Updated weights for policy 0, policy_version 1832021 (0.0007) [2023-12-27 04:44:28,497][105692] Updated weights for policy 0, policy_version 1832031 (0.0008) [2023-12-27 04:44:28,567][105692] Updated weights for policy 0, policy_version 1832041 (0.0007) [2023-12-27 04:44:28,972][105620] Updated weights for policy 1, policy_version 1836129 (0.0009) [2023-12-27 04:44:29,030][105620] Updated weights for policy 1, policy_version 1836139 (0.0009) [2023-12-27 04:44:29,090][105620] Updated weights for policy 1, policy_version 1836149 (0.0008) [2023-12-27 04:44:29,156][105692] Updated weights for policy 0, policy_version 1832051 (0.0006) [2023-12-27 04:44:29,216][105692] Updated weights for policy 0, policy_version 1832061 (0.0006) [2023-12-27 04:44:29,272][105692] Updated weights for policy 0, policy_version 1832071 (0.0009) [2023-12-27 04:44:29,803][105620] Updated weights for policy 1, policy_version 1836159 (0.0009) [2023-12-27 04:44:29,859][105620] Updated weights for policy 1, policy_version 1836169 (0.0009) [2023-12-27 04:44:29,914][105620] Updated weights for policy 1, policy_version 1836179 (0.0009) [2023-12-27 04:44:30,068][105692] Updated weights for policy 0, policy_version 1832081 (0.0009) [2023-12-27 04:44:30,121][105692] Updated weights for policy 0, policy_version 1832091 (0.0010) [2023-12-27 04:44:30,170][105692] Updated weights for policy 0, policy_version 1832101 (0.0011) [2023-12-27 04:44:30,219][105692] Updated weights for policy 0, policy_version 1832111 (0.0009) [2023-12-27 04:44:30,647][105620] Updated weights for policy 1, policy_version 1836189 (0.0007) [2023-12-27 04:44:30,714][105620] Updated weights for policy 1, policy_version 1836199 (0.0007) [2023-12-27 04:44:30,779][105620] Updated weights for policy 1, policy_version 1836209 (0.0008) [2023-12-27 04:44:30,978][105692] Updated weights for policy 0, policy_version 1832121 (0.0005) [2023-12-27 04:44:31,027][105692] Updated weights for policy 0, policy_version 1832131 (0.0006) [2023-12-27 04:44:31,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 939229184. Throughput: 0: 9533.2, 1: 10112.3. Samples: 939202368. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:44:31,062][104569] Avg episode reward: [(0, '8806.438'), (1, '9258.010')] [2023-12-27 04:44:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001836216_470138880.pth... [2023-12-27 04:44:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001835032_469835776.pth [2023-12-27 04:44:31,089][105692] Updated weights for policy 0, policy_version 1832141 (0.0007) [2023-12-27 04:44:31,105][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001832144_469098496.pth... [2023-12-27 04:44:31,109][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001830992_468803584.pth [2023-12-27 04:44:31,528][105620] Updated weights for policy 1, policy_version 1836219 (0.0009) [2023-12-27 04:44:31,578][105620] Updated weights for policy 1, policy_version 1836229 (0.0009) [2023-12-27 04:44:31,632][105620] Updated weights for policy 1, policy_version 1836239 (0.0008) [2023-12-27 04:44:31,839][105692] Updated weights for policy 0, policy_version 1832151 (0.0009) [2023-12-27 04:44:31,899][105692] Updated weights for policy 0, policy_version 1832161 (0.0010) [2023-12-27 04:44:31,954][105692] Updated weights for policy 0, policy_version 1832171 (0.0011) [2023-12-27 04:44:32,283][105620] Updated weights for policy 1, policy_version 1836249 (0.0009) [2023-12-27 04:44:32,339][105620] Updated weights for policy 1, policy_version 1836259 (0.0008) [2023-12-27 04:44:32,404][105620] Updated weights for policy 1, policy_version 1836269 (0.0007) [2023-12-27 04:44:32,461][105620] Updated weights for policy 1, policy_version 1836279 (0.0006) [2023-12-27 04:44:32,722][105692] Updated weights for policy 0, policy_version 1832181 (0.0010) [2023-12-27 04:44:32,784][105692] Updated weights for policy 0, policy_version 1832191 (0.0007) [2023-12-27 04:44:32,839][105692] Updated weights for policy 0, policy_version 1832201 (0.0010) [2023-12-27 04:44:33,049][105620] Updated weights for policy 1, policy_version 1836289 (0.0006) [2023-12-27 04:44:33,112][105620] Updated weights for policy 1, policy_version 1836299 (0.0009) [2023-12-27 04:44:33,171][105620] Updated weights for policy 1, policy_version 1836309 (0.0009) [2023-12-27 04:44:33,641][105692] Updated weights for policy 0, policy_version 1832211 (0.0012) [2023-12-27 04:44:33,688][105692] Updated weights for policy 0, policy_version 1832221 (0.0005) [2023-12-27 04:44:33,734][105692] Updated weights for policy 0, policy_version 1832231 (0.0006) [2023-12-27 04:44:33,792][105620] Updated weights for policy 1, policy_version 1836319 (0.0006) [2023-12-27 04:44:33,843][105620] Updated weights for policy 1, policy_version 1836329 (0.0005) [2023-12-27 04:44:33,896][105620] Updated weights for policy 1, policy_version 1836339 (0.0005) [2023-12-27 04:44:34,324][105692] Updated weights for policy 0, policy_version 1832241 (0.0006) [2023-12-27 04:44:34,387][105692] Updated weights for policy 0, policy_version 1832251 (0.0011) [2023-12-27 04:44:34,450][105692] Updated weights for policy 0, policy_version 1832261 (0.0011) [2023-12-27 04:44:34,513][105692] Updated weights for policy 0, policy_version 1832271 (0.0011) [2023-12-27 04:44:34,560][105620] Updated weights for policy 1, policy_version 1836349 (0.0008) [2023-12-27 04:44:34,625][105620] Updated weights for policy 1, policy_version 1836359 (0.0008) [2023-12-27 04:44:34,692][105620] Updated weights for policy 1, policy_version 1836369 (0.0008) [2023-12-27 04:44:35,279][105692] Updated weights for policy 0, policy_version 1832281 (0.0010) [2023-12-27 04:44:35,334][105692] Updated weights for policy 0, policy_version 1832291 (0.0010) [2023-12-27 04:44:35,393][105620] Updated weights for policy 1, policy_version 1836379 (0.0008) [2023-12-27 04:44:35,399][105692] Updated weights for policy 0, policy_version 1832301 (0.0011) [2023-12-27 04:44:35,449][105620] Updated weights for policy 1, policy_version 1836389 (0.0006) [2023-12-27 04:44:35,510][105620] Updated weights for policy 1, policy_version 1836399 (0.0007) [2023-12-27 04:44:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 939327488. Throughput: 0: 9602.7, 1: 10052.0. Samples: 939321252. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:44:36,063][104569] Avg episode reward: [(0, '8442.866'), (1, '9167.757')] [2023-12-27 04:44:36,159][105692] Updated weights for policy 0, policy_version 1832311 (0.0009) [2023-12-27 04:44:36,173][105620] Updated weights for policy 1, policy_version 1836409 (0.0009) [2023-12-27 04:44:36,223][105692] Updated weights for policy 0, policy_version 1832321 (0.0007) [2023-12-27 04:44:36,234][105620] Updated weights for policy 1, policy_version 1836419 (0.0006) [2023-12-27 04:44:36,281][105692] Updated weights for policy 0, policy_version 1832331 (0.0007) [2023-12-27 04:44:36,298][105620] Updated weights for policy 1, policy_version 1836429 (0.0011) [2023-12-27 04:44:36,364][105620] Updated weights for policy 1, policy_version 1836439 (0.0011) [2023-12-27 04:44:36,967][105620] Updated weights for policy 1, policy_version 1836449 (0.0006) [2023-12-27 04:44:37,019][105620] Updated weights for policy 1, policy_version 1836459 (0.0005) [2023-12-27 04:44:37,065][105692] Updated weights for policy 0, policy_version 1832341 (0.0008) [2023-12-27 04:44:37,072][105620] Updated weights for policy 1, policy_version 1836469 (0.0009) [2023-12-27 04:44:37,123][105692] Updated weights for policy 0, policy_version 1832351 (0.0007) [2023-12-27 04:44:37,182][105692] Updated weights for policy 0, policy_version 1832361 (0.0008) [2023-12-27 04:44:37,796][105620] Updated weights for policy 1, policy_version 1836479 (0.0011) [2023-12-27 04:44:37,852][105620] Updated weights for policy 1, policy_version 1836489 (0.0011) [2023-12-27 04:44:37,901][105620] Updated weights for policy 1, policy_version 1836499 (0.0010) [2023-12-27 04:44:37,934][105692] Updated weights for policy 0, policy_version 1832371 (0.0008) [2023-12-27 04:44:37,977][105692] Updated weights for policy 0, policy_version 1832381 (0.0007) [2023-12-27 04:44:38,033][105692] Updated weights for policy 0, policy_version 1832391 (0.0008) [2023-12-27 04:44:38,667][105620] Updated weights for policy 1, policy_version 1836509 (0.0010) [2023-12-27 04:44:38,727][105620] Updated weights for policy 1, policy_version 1836519 (0.0011) [2023-12-27 04:44:38,780][105620] Updated weights for policy 1, policy_version 1836529 (0.0011) [2023-12-27 04:44:38,820][105692] Updated weights for policy 0, policy_version 1832401 (0.0008) [2023-12-27 04:44:38,872][105692] Updated weights for policy 0, policy_version 1832411 (0.0008) [2023-12-27 04:44:38,936][105692] Updated weights for policy 0, policy_version 1832421 (0.0009) [2023-12-27 04:44:38,995][105692] Updated weights for policy 0, policy_version 1832431 (0.0008) [2023-12-27 04:44:39,575][105620] Updated weights for policy 1, policy_version 1836539 (0.0011) [2023-12-27 04:44:39,638][105620] Updated weights for policy 1, policy_version 1836549 (0.0011) [2023-12-27 04:44:39,702][105620] Updated weights for policy 1, policy_version 1836559 (0.0011) [2023-12-27 04:44:39,771][105692] Updated weights for policy 0, policy_version 1832441 (0.0007) [2023-12-27 04:44:39,820][105692] Updated weights for policy 0, policy_version 1832451 (0.0008) [2023-12-27 04:44:39,875][105692] Updated weights for policy 0, policy_version 1832461 (0.0009) [2023-12-27 04:44:40,454][105620] Updated weights for policy 1, policy_version 1836569 (0.0011) [2023-12-27 04:44:40,513][105620] Updated weights for policy 1, policy_version 1836579 (0.0011) [2023-12-27 04:44:40,572][105620] Updated weights for policy 1, policy_version 1836589 (0.0010) [2023-12-27 04:44:40,631][105620] Updated weights for policy 1, policy_version 1836599 (0.0010) [2023-12-27 04:44:40,677][105692] Updated weights for policy 0, policy_version 1832471 (0.0008) [2023-12-27 04:44:40,729][105692] Updated weights for policy 0, policy_version 1832481 (0.0008) [2023-12-27 04:44:40,782][105692] Updated weights for policy 0, policy_version 1832491 (0.0008) [2023-12-27 04:44:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19716.3). Total num frames: 939425792. Throughput: 0: 9633.8, 1: 9998.9. Samples: 939434128. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:44:41,063][104569] Avg episode reward: [(0, '8075.971'), (1, '9167.633')] [2023-12-27 04:44:41,427][105620] Updated weights for policy 1, policy_version 1836609 (0.0011) [2023-12-27 04:44:41,490][105620] Updated weights for policy 1, policy_version 1836619 (0.0011) [2023-12-27 04:44:41,554][105620] Updated weights for policy 1, policy_version 1836629 (0.0011) [2023-12-27 04:44:41,604][105692] Updated weights for policy 0, policy_version 1832501 (0.0008) [2023-12-27 04:44:41,674][105692] Updated weights for policy 0, policy_version 1832511 (0.0009) [2023-12-27 04:44:41,745][105692] Updated weights for policy 0, policy_version 1832521 (0.0008) [2023-12-27 04:44:42,315][105620] Updated weights for policy 1, policy_version 1836639 (0.0009) [2023-12-27 04:44:42,383][105620] Updated weights for policy 1, policy_version 1836649 (0.0008) [2023-12-27 04:44:42,447][105620] Updated weights for policy 1, policy_version 1836659 (0.0008) [2023-12-27 04:44:42,483][105692] Updated weights for policy 0, policy_version 1832531 (0.0008) [2023-12-27 04:44:42,543][105692] Updated weights for policy 0, policy_version 1832541 (0.0007) [2023-12-27 04:44:42,599][105692] Updated weights for policy 0, policy_version 1832551 (0.0008) [2023-12-27 04:44:43,175][105620] Updated weights for policy 1, policy_version 1836669 (0.0009) [2023-12-27 04:44:43,243][105620] Updated weights for policy 1, policy_version 1836679 (0.0010) [2023-12-27 04:44:43,296][105620] Updated weights for policy 1, policy_version 1836690 (0.0010) [2023-12-27 04:44:43,311][105692] Updated weights for policy 0, policy_version 1832561 (0.0009) [2023-12-27 04:44:43,359][105692] Updated weights for policy 0, policy_version 1832571 (0.0005) [2023-12-27 04:44:43,410][105692] Updated weights for policy 0, policy_version 1832581 (0.0009) [2023-12-27 04:44:43,465][105692] Updated weights for policy 0, policy_version 1832591 (0.0009) [2023-12-27 04:44:43,909][105620] Updated weights for policy 1, policy_version 1836700 (0.0009) [2023-12-27 04:44:43,961][105620] Updated weights for policy 1, policy_version 1836710 (0.0006) [2023-12-27 04:44:44,015][105620] Updated weights for policy 1, policy_version 1836720 (0.0005) [2023-12-27 04:44:44,199][105692] Updated weights for policy 0, policy_version 1832601 (0.0011) [2023-12-27 04:44:44,270][105692] Updated weights for policy 0, policy_version 1832611 (0.0008) [2023-12-27 04:44:44,337][105692] Updated weights for policy 0, policy_version 1832621 (0.0007) [2023-12-27 04:44:44,556][105620] Updated weights for policy 1, policy_version 1836730 (0.0006) [2023-12-27 04:44:44,606][105620] Updated weights for policy 1, policy_version 1836740 (0.0008) [2023-12-27 04:44:44,655][105620] Updated weights for policy 1, policy_version 1836750 (0.0008) [2023-12-27 04:44:44,714][105620] Updated weights for policy 1, policy_version 1836760 (0.0010) [2023-12-27 04:44:44,934][105692] Updated weights for policy 0, policy_version 1832631 (0.0010) [2023-12-27 04:44:44,998][105692] Updated weights for policy 0, policy_version 1832641 (0.0011) [2023-12-27 04:44:45,065][105692] Updated weights for policy 0, policy_version 1832651 (0.0008) [2023-12-27 04:44:45,580][105620] Updated weights for policy 1, policy_version 1836770 (0.0010) [2023-12-27 04:44:45,633][105620] Updated weights for policy 1, policy_version 1836781 (0.0010) [2023-12-27 04:44:45,648][105692] Updated weights for policy 0, policy_version 1832661 (0.0007) [2023-12-27 04:44:45,692][105620] Updated weights for policy 1, policy_version 1836791 (0.0008) [2023-12-27 04:44:45,710][105692] Updated weights for policy 0, policy_version 1832671 (0.0006) [2023-12-27 04:44:45,769][105692] Updated weights for policy 0, policy_version 1832681 (0.0006) [2023-12-27 04:44:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19716.3). Total num frames: 939524096. Throughput: 0: 9669.4, 1: 9971.4. Samples: 939490404. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:44:46,062][104569] Avg episode reward: [(0, '8535.734'), (1, '9257.793')] [2023-12-27 04:44:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001836792_470286336.pth... [2023-12-27 04:44:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001832688_469237760.pth... [2023-12-27 04:44:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001835640_469991424.pth [2023-12-27 04:44:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001831536_468942848.pth [2023-12-27 04:44:46,427][105692] Updated weights for policy 0, policy_version 1832691 (0.0007) [2023-12-27 04:44:46,490][105692] Updated weights for policy 0, policy_version 1832701 (0.0007) [2023-12-27 04:44:46,504][105620] Updated weights for policy 1, policy_version 1836801 (0.0009) [2023-12-27 04:44:46,544][105692] Updated weights for policy 0, policy_version 1832711 (0.0007) [2023-12-27 04:44:46,556][105620] Updated weights for policy 1, policy_version 1836811 (0.0007) [2023-12-27 04:44:46,609][105620] Updated weights for policy 1, policy_version 1836821 (0.0008) [2023-12-27 04:44:47,242][105692] Updated weights for policy 0, policy_version 1832721 (0.0006) [2023-12-27 04:44:47,307][105692] Updated weights for policy 0, policy_version 1832731 (0.0009) [2023-12-27 04:44:47,364][105620] Updated weights for policy 1, policy_version 1836831 (0.0007) [2023-12-27 04:44:47,369][105692] Updated weights for policy 0, policy_version 1832741 (0.0009) [2023-12-27 04:44:47,415][105620] Updated weights for policy 1, policy_version 1836841 (0.0005) [2023-12-27 04:44:47,428][105692] Updated weights for policy 0, policy_version 1832751 (0.0008) [2023-12-27 04:44:47,465][105620] Updated weights for policy 1, policy_version 1836851 (0.0008) [2023-12-27 04:44:48,118][105620] Updated weights for policy 1, policy_version 1836861 (0.0009) [2023-12-27 04:44:48,181][105620] Updated weights for policy 1, policy_version 1836871 (0.0008) [2023-12-27 04:44:48,235][105692] Updated weights for policy 0, policy_version 1832761 (0.0010) [2023-12-27 04:44:48,243][105620] Updated weights for policy 1, policy_version 1836881 (0.0006) [2023-12-27 04:44:48,294][105692] Updated weights for policy 0, policy_version 1832771 (0.0011) [2023-12-27 04:44:48,354][105692] Updated weights for policy 0, policy_version 1832781 (0.0010) [2023-12-27 04:44:48,971][105620] Updated weights for policy 1, policy_version 1836891 (0.0008) [2023-12-27 04:44:48,989][105692] Updated weights for policy 0, policy_version 1832791 (0.0008) [2023-12-27 04:44:49,038][105620] Updated weights for policy 1, policy_version 1836901 (0.0008) [2023-12-27 04:44:49,045][105692] Updated weights for policy 0, policy_version 1832801 (0.0006) [2023-12-27 04:44:49,107][105692] Updated weights for policy 0, policy_version 1832811 (0.0005) [2023-12-27 04:44:49,111][105620] Updated weights for policy 1, policy_version 1836911 (0.0008) [2023-12-27 04:44:49,757][105692] Updated weights for policy 0, policy_version 1832821 (0.0009) [2023-12-27 04:44:49,805][105620] Updated weights for policy 1, policy_version 1836921 (0.0008) [2023-12-27 04:44:49,816][105692] Updated weights for policy 0, policy_version 1832831 (0.0010) [2023-12-27 04:44:49,867][105620] Updated weights for policy 1, policy_version 1836931 (0.0010) [2023-12-27 04:44:49,879][105692] Updated weights for policy 0, policy_version 1832841 (0.0011) [2023-12-27 04:44:49,937][105620] Updated weights for policy 1, policy_version 1836941 (0.0007) [2023-12-27 04:44:50,004][105620] Updated weights for policy 1, policy_version 1836951 (0.0006) [2023-12-27 04:44:50,541][105620] Updated weights for policy 1, policy_version 1836961 (0.0005) [2023-12-27 04:44:50,610][105620] Updated weights for policy 1, policy_version 1836971 (0.0009) [2023-12-27 04:44:50,628][105692] Updated weights for policy 0, policy_version 1832851 (0.0010) [2023-12-27 04:44:50,679][105620] Updated weights for policy 1, policy_version 1836981 (0.0006) [2023-12-27 04:44:50,692][105692] Updated weights for policy 0, policy_version 1832861 (0.0008) [2023-12-27 04:44:50,749][105692] Updated weights for policy 0, policy_version 1832871 (0.0011) [2023-12-27 04:44:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19688.6). Total num frames: 939622400. Throughput: 0: 9790.4, 1: 9943.5. Samples: 939610048. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:44:51,062][104569] Avg episode reward: [(0, '8446.511'), (1, '8706.296')] [2023-12-27 04:44:51,295][105620] Updated weights for policy 1, policy_version 1836991 (0.0009) [2023-12-27 04:44:51,374][105620] Updated weights for policy 1, policy_version 1837001 (0.0011) [2023-12-27 04:44:51,441][105620] Updated weights for policy 1, policy_version 1837011 (0.0011) [2023-12-27 04:44:51,491][105692] Updated weights for policy 0, policy_version 1832881 (0.0010) [2023-12-27 04:44:51,555][105692] Updated weights for policy 0, policy_version 1832891 (0.0008) [2023-12-27 04:44:51,612][105692] Updated weights for policy 0, policy_version 1832901 (0.0008) [2023-12-27 04:44:51,680][105692] Updated weights for policy 0, policy_version 1832911 (0.0008) [2023-12-27 04:44:52,118][105620] Updated weights for policy 1, policy_version 1837021 (0.0010) [2023-12-27 04:44:52,182][105620] Updated weights for policy 1, policy_version 1837031 (0.0011) [2023-12-27 04:44:52,234][105620] Updated weights for policy 1, policy_version 1837041 (0.0011) [2023-12-27 04:44:52,367][105692] Updated weights for policy 0, policy_version 1832921 (0.0008) [2023-12-27 04:44:52,431][105692] Updated weights for policy 0, policy_version 1832931 (0.0009) [2023-12-27 04:44:52,490][105692] Updated weights for policy 0, policy_version 1832941 (0.0011) [2023-12-27 04:44:53,036][105620] Updated weights for policy 1, policy_version 1837051 (0.0009) [2023-12-27 04:44:53,098][105620] Updated weights for policy 1, policy_version 1837061 (0.0009) [2023-12-27 04:44:53,161][105620] Updated weights for policy 1, policy_version 1837071 (0.0009) [2023-12-27 04:44:53,174][105692] Updated weights for policy 0, policy_version 1832951 (0.0007) [2023-12-27 04:44:53,221][105692] Updated weights for policy 0, policy_version 1832961 (0.0005) [2023-12-27 04:44:53,270][105692] Updated weights for policy 0, policy_version 1832971 (0.0007) [2023-12-27 04:44:53,832][105692] Updated weights for policy 0, policy_version 1832981 (0.0007) [2023-12-27 04:44:53,886][105692] Updated weights for policy 0, policy_version 1832991 (0.0005) [2023-12-27 04:44:53,907][105620] Updated weights for policy 1, policy_version 1837081 (0.0009) [2023-12-27 04:44:53,936][105692] Updated weights for policy 0, policy_version 1833001 (0.0005) [2023-12-27 04:44:53,964][105620] Updated weights for policy 1, policy_version 1837091 (0.0005) [2023-12-27 04:44:54,011][105620] Updated weights for policy 1, policy_version 1837101 (0.0005) [2023-12-27 04:44:54,072][105620] Updated weights for policy 1, policy_version 1837111 (0.0006) [2023-12-27 04:44:54,520][105692] Updated weights for policy 0, policy_version 1833011 (0.0007) [2023-12-27 04:44:54,583][105692] Updated weights for policy 0, policy_version 1833021 (0.0010) [2023-12-27 04:44:54,641][105692] Updated weights for policy 0, policy_version 1833031 (0.0010) [2023-12-27 04:44:54,747][105620] Updated weights for policy 1, policy_version 1837121 (0.0008) [2023-12-27 04:44:54,801][105620] Updated weights for policy 1, policy_version 1837131 (0.0010) [2023-12-27 04:44:54,854][105620] Updated weights for policy 1, policy_version 1837141 (0.0010) [2023-12-27 04:44:55,297][105692] Updated weights for policy 0, policy_version 1833041 (0.0010) [2023-12-27 04:44:55,356][105692] Updated weights for policy 0, policy_version 1833051 (0.0009) [2023-12-27 04:44:55,411][105692] Updated weights for policy 0, policy_version 1833061 (0.0006) [2023-12-27 04:44:55,463][105692] Updated weights for policy 0, policy_version 1833071 (0.0009) [2023-12-27 04:44:55,678][105620] Updated weights for policy 1, policy_version 1837151 (0.0010) [2023-12-27 04:44:55,744][105620] Updated weights for policy 1, policy_version 1837161 (0.0007) [2023-12-27 04:44:55,815][105620] Updated weights for policy 1, policy_version 1837171 (0.0008) [2023-12-27 04:44:56,032][105692] Updated weights for policy 0, policy_version 1833081 (0.0008) [2023-12-27 04:44:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19744.1). Total num frames: 939720704. Throughput: 0: 9874.2, 1: 9915.8. Samples: 939730452. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:44:56,062][104569] Avg episode reward: [(0, '8447.259'), (1, '8706.382')] [2023-12-27 04:44:56,086][105692] Updated weights for policy 0, policy_version 1833091 (0.0009) [2023-12-27 04:44:56,146][105692] Updated weights for policy 0, policy_version 1833101 (0.0008) [2023-12-27 04:44:56,596][105620] Updated weights for policy 1, policy_version 1837181 (0.0008) [2023-12-27 04:44:56,651][105620] Updated weights for policy 1, policy_version 1837191 (0.0006) [2023-12-27 04:44:56,702][105620] Updated weights for policy 1, policy_version 1837201 (0.0005) [2023-12-27 04:44:56,756][105692] Updated weights for policy 0, policy_version 1833111 (0.0008) [2023-12-27 04:44:56,815][105692] Updated weights for policy 0, policy_version 1833121 (0.0008) [2023-12-27 04:44:56,867][105692] Updated weights for policy 0, policy_version 1833131 (0.0006) [2023-12-27 04:44:57,328][105620] Updated weights for policy 1, policy_version 1837211 (0.0006) [2023-12-27 04:44:57,384][105620] Updated weights for policy 1, policy_version 1837221 (0.0008) [2023-12-27 04:44:57,410][105692] Updated weights for policy 0, policy_version 1833141 (0.0006) [2023-12-27 04:44:57,446][105620] Updated weights for policy 1, policy_version 1837231 (0.0007) [2023-12-27 04:44:57,460][105692] Updated weights for policy 0, policy_version 1833151 (0.0007) [2023-12-27 04:44:57,511][105692] Updated weights for policy 0, policy_version 1833161 (0.0007) [2023-12-27 04:44:58,159][105620] Updated weights for policy 1, policy_version 1837241 (0.0007) [2023-12-27 04:44:58,225][105620] Updated weights for policy 1, policy_version 1837251 (0.0007) [2023-12-27 04:44:58,255][105692] Updated weights for policy 0, policy_version 1833171 (0.0008) [2023-12-27 04:44:58,287][105620] Updated weights for policy 1, policy_version 1837261 (0.0008) [2023-12-27 04:44:58,314][105692] Updated weights for policy 0, policy_version 1833181 (0.0005) [2023-12-27 04:44:58,349][105620] Updated weights for policy 1, policy_version 1837271 (0.0008) [2023-12-27 04:44:58,380][105692] Updated weights for policy 0, policy_version 1833191 (0.0008) [2023-12-27 04:44:59,148][105692] Updated weights for policy 0, policy_version 1833201 (0.0008) [2023-12-27 04:44:59,179][105620] Updated weights for policy 1, policy_version 1837281 (0.0009) [2023-12-27 04:44:59,206][105692] Updated weights for policy 0, policy_version 1833211 (0.0009) [2023-12-27 04:44:59,243][105620] Updated weights for policy 1, policy_version 1837291 (0.0009) [2023-12-27 04:44:59,270][105692] Updated weights for policy 0, policy_version 1833221 (0.0009) [2023-12-27 04:44:59,301][105620] Updated weights for policy 1, policy_version 1837301 (0.0008) [2023-12-27 04:44:59,334][105692] Updated weights for policy 0, policy_version 1833231 (0.0007) [2023-12-27 04:45:00,065][105620] Updated weights for policy 1, policy_version 1837311 (0.0007) [2023-12-27 04:45:00,124][105620] Updated weights for policy 1, policy_version 1837321 (0.0006) [2023-12-27 04:45:00,124][105692] Updated weights for policy 0, policy_version 1833241 (0.0006) [2023-12-27 04:45:00,179][105692] Updated weights for policy 0, policy_version 1833251 (0.0008) [2023-12-27 04:45:00,185][105620] Updated weights for policy 1, policy_version 1837331 (0.0007) [2023-12-27 04:45:00,236][105692] Updated weights for policy 0, policy_version 1833261 (0.0006) [2023-12-27 04:45:00,777][105620] Updated weights for policy 1, policy_version 1837341 (0.0010) [2023-12-27 04:45:00,836][105620] Updated weights for policy 1, policy_version 1837351 (0.0006) [2023-12-27 04:45:00,891][105620] Updated weights for policy 1, policy_version 1837361 (0.0009) [2023-12-27 04:45:00,961][105692] Updated weights for policy 0, policy_version 1833271 (0.0009) [2023-12-27 04:45:01,012][105692] Updated weights for policy 0, policy_version 1833281 (0.0010) [2023-12-27 04:45:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19716.3). Total num frames: 939819008. Throughput: 0: 9989.3, 1: 9900.6. Samples: 939791864. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:45:01,063][104569] Avg episode reward: [(0, '8810.390'), (1, '9165.762')] [2023-12-27 04:45:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001837368_470433792.pth... [2023-12-27 04:45:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001836216_470138880.pth [2023-12-27 04:45:01,077][105692] Updated weights for policy 0, policy_version 1833291 (0.0009) [2023-12-27 04:45:01,105][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001833296_469393408.pth... [2023-12-27 04:45:01,109][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001832144_469098496.pth [2023-12-27 04:45:01,591][105620] Updated weights for policy 1, policy_version 1837371 (0.0010) [2023-12-27 04:45:01,653][105620] Updated weights for policy 1, policy_version 1837381 (0.0008) [2023-12-27 04:45:01,722][105620] Updated weights for policy 1, policy_version 1837391 (0.0011) [2023-12-27 04:45:01,864][105692] Updated weights for policy 0, policy_version 1833301 (0.0009) [2023-12-27 04:45:01,932][105692] Updated weights for policy 0, policy_version 1833311 (0.0009) [2023-12-27 04:45:01,992][105692] Updated weights for policy 0, policy_version 1833321 (0.0009) [2023-12-27 04:45:02,411][105620] Updated weights for policy 1, policy_version 1837401 (0.0007) [2023-12-27 04:45:02,477][105620] Updated weights for policy 1, policy_version 1837411 (0.0008) [2023-12-27 04:45:02,543][105620] Updated weights for policy 1, policy_version 1837421 (0.0008) [2023-12-27 04:45:02,612][105620] Updated weights for policy 1, policy_version 1837431 (0.0008) [2023-12-27 04:45:02,768][105692] Updated weights for policy 0, policy_version 1833331 (0.0009) [2023-12-27 04:45:02,826][105692] Updated weights for policy 0, policy_version 1833341 (0.0011) [2023-12-27 04:45:02,883][105692] Updated weights for policy 0, policy_version 1833351 (0.0010) [2023-12-27 04:45:03,352][105620] Updated weights for policy 1, policy_version 1837441 (0.0010) [2023-12-27 04:45:03,409][105620] Updated weights for policy 1, policy_version 1837451 (0.0009) [2023-12-27 04:45:03,466][105620] Updated weights for policy 1, policy_version 1837461 (0.0008) [2023-12-27 04:45:03,537][105692] Updated weights for policy 0, policy_version 1833361 (0.0010) [2023-12-27 04:45:03,595][105692] Updated weights for policy 0, policy_version 1833371 (0.0010) [2023-12-27 04:45:03,659][105692] Updated weights for policy 0, policy_version 1833381 (0.0010) [2023-12-27 04:45:03,713][105692] Updated weights for policy 0, policy_version 1833391 (0.0009) [2023-12-27 04:45:04,249][105620] Updated weights for policy 1, policy_version 1837471 (0.0007) [2023-12-27 04:45:04,304][105620] Updated weights for policy 1, policy_version 1837481 (0.0010) [2023-12-27 04:45:04,355][105620] Updated weights for policy 1, policy_version 1837491 (0.0009) [2023-12-27 04:45:04,376][105692] Updated weights for policy 0, policy_version 1833401 (0.0007) [2023-12-27 04:45:04,436][105692] Updated weights for policy 0, policy_version 1833411 (0.0009) [2023-12-27 04:45:04,498][105692] Updated weights for policy 0, policy_version 1833421 (0.0008) [2023-12-27 04:45:04,986][105620] Updated weights for policy 1, policy_version 1837501 (0.0006) [2023-12-27 04:45:05,036][105620] Updated weights for policy 1, policy_version 1837511 (0.0007) [2023-12-27 04:45:05,060][105692] Updated weights for policy 0, policy_version 1833431 (0.0009) [2023-12-27 04:45:05,090][105620] Updated weights for policy 1, policy_version 1837521 (0.0008) [2023-12-27 04:45:05,115][105692] Updated weights for policy 0, policy_version 1833441 (0.0010) [2023-12-27 04:45:05,171][105692] Updated weights for policy 0, policy_version 1833451 (0.0008) [2023-12-27 04:45:05,653][105620] Updated weights for policy 1, policy_version 1837531 (0.0008) [2023-12-27 04:45:05,710][105620] Updated weights for policy 1, policy_version 1837541 (0.0010) [2023-12-27 04:45:05,771][105620] Updated weights for policy 1, policy_version 1837551 (0.0010) [2023-12-27 04:45:05,775][105692] Updated weights for policy 0, policy_version 1833461 (0.0008) [2023-12-27 04:45:05,838][105692] Updated weights for policy 0, policy_version 1833471 (0.0010) [2023-12-27 04:45:05,899][105692] Updated weights for policy 0, policy_version 1833481 (0.0010) [2023-12-27 04:45:06,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19933.9, 300 sec: 19744.1). Total num frames: 939925504. Throughput: 0: 9967.8, 1: 9847.4. Samples: 939906836. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:45:06,063][104569] Avg episode reward: [(0, '8537.097'), (1, '9074.235')] [2023-12-27 04:45:06,509][105620] Updated weights for policy 1, policy_version 1837561 (0.0010) [2023-12-27 04:45:06,573][105620] Updated weights for policy 1, policy_version 1837571 (0.0011) [2023-12-27 04:45:06,626][105620] Updated weights for policy 1, policy_version 1837581 (0.0007) [2023-12-27 04:45:06,654][105692] Updated weights for policy 0, policy_version 1833491 (0.0010) [2023-12-27 04:45:06,697][105620] Updated weights for policy 1, policy_version 1837591 (0.0011) [2023-12-27 04:45:06,721][105692] Updated weights for policy 0, policy_version 1833501 (0.0007) [2023-12-27 04:45:06,785][105692] Updated weights for policy 0, policy_version 1833511 (0.0008) [2023-12-27 04:45:07,426][105620] Updated weights for policy 1, policy_version 1837601 (0.0011) [2023-12-27 04:45:07,479][105620] Updated weights for policy 1, policy_version 1837611 (0.0010) [2023-12-27 04:45:07,531][105620] Updated weights for policy 1, policy_version 1837621 (0.0010) [2023-12-27 04:45:07,564][105692] Updated weights for policy 0, policy_version 1833521 (0.0009) [2023-12-27 04:45:07,613][105692] Updated weights for policy 0, policy_version 1833531 (0.0007) [2023-12-27 04:45:07,661][105692] Updated weights for policy 0, policy_version 1833541 (0.0008) [2023-12-27 04:45:07,710][105692] Updated weights for policy 0, policy_version 1833551 (0.0008) [2023-12-27 04:45:08,279][105620] Updated weights for policy 1, policy_version 1837631 (0.0007) [2023-12-27 04:45:08,338][105620] Updated weights for policy 1, policy_version 1837641 (0.0006) [2023-12-27 04:45:08,402][105620] Updated weights for policy 1, policy_version 1837651 (0.0006) [2023-12-27 04:45:08,484][105692] Updated weights for policy 0, policy_version 1833561 (0.0010) [2023-12-27 04:45:08,542][105692] Updated weights for policy 0, policy_version 1833572 (0.0010) [2023-12-27 04:45:08,597][105692] Updated weights for policy 0, policy_version 1833582 (0.0007) [2023-12-27 04:45:09,068][105620] Updated weights for policy 1, policy_version 1837661 (0.0006) [2023-12-27 04:45:09,132][105620] Updated weights for policy 1, policy_version 1837671 (0.0006) [2023-12-27 04:45:09,197][105620] Updated weights for policy 1, policy_version 1837681 (0.0005) [2023-12-27 04:45:09,456][105692] Updated weights for policy 0, policy_version 1833592 (0.0009) [2023-12-27 04:45:09,515][105692] Updated weights for policy 0, policy_version 1833602 (0.0010) [2023-12-27 04:45:09,578][105692] Updated weights for policy 0, policy_version 1833613 (0.0011) [2023-12-27 04:45:09,769][105620] Updated weights for policy 1, policy_version 1837691 (0.0008) [2023-12-27 04:45:09,828][105620] Updated weights for policy 1, policy_version 1837701 (0.0009) [2023-12-27 04:45:09,893][105620] Updated weights for policy 1, policy_version 1837711 (0.0008) [2023-12-27 04:45:10,352][105692] Updated weights for policy 0, policy_version 1833623 (0.0009) [2023-12-27 04:45:10,411][105692] Updated weights for policy 0, policy_version 1833633 (0.0009) [2023-12-27 04:45:10,469][105692] Updated weights for policy 0, policy_version 1833643 (0.0009) [2023-12-27 04:45:10,580][105620] Updated weights for policy 1, policy_version 1837721 (0.0007) [2023-12-27 04:45:10,635][105620] Updated weights for policy 1, policy_version 1837731 (0.0005) [2023-12-27 04:45:10,682][105620] Updated weights for policy 1, policy_version 1837741 (0.0005) [2023-12-27 04:45:10,734][105620] Updated weights for policy 1, policy_version 1837751 (0.0005) [2023-12-27 04:45:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19744.1). Total num frames: 940015616. Throughput: 0: 9888.7, 1: 9880.8. Samples: 940025428. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:45:11,062][104569] Avg episode reward: [(0, '8080.484'), (1, '9166.468')] [2023-12-27 04:45:11,222][105692] Updated weights for policy 0, policy_version 1833653 (0.0008) [2023-12-27 04:45:11,285][105692] Updated weights for policy 0, policy_version 1833663 (0.0010) [2023-12-27 04:45:11,346][105692] Updated weights for policy 0, policy_version 1833673 (0.0010) [2023-12-27 04:45:11,420][105620] Updated weights for policy 1, policy_version 1837761 (0.0007) [2023-12-27 04:45:11,488][105620] Updated weights for policy 1, policy_version 1837771 (0.0006) [2023-12-27 04:45:11,560][105620] Updated weights for policy 1, policy_version 1837781 (0.0006) [2023-12-27 04:45:12,084][105692] Updated weights for policy 0, policy_version 1833683 (0.0008) [2023-12-27 04:45:12,142][105692] Updated weights for policy 0, policy_version 1833693 (0.0007) [2023-12-27 04:45:12,195][105620] Updated weights for policy 1, policy_version 1837791 (0.0009) [2023-12-27 04:45:12,195][105692] Updated weights for policy 0, policy_version 1833703 (0.0009) [2023-12-27 04:45:12,247][105620] Updated weights for policy 1, policy_version 1837801 (0.0010) [2023-12-27 04:45:12,307][105620] Updated weights for policy 1, policy_version 1837811 (0.0011) [2023-12-27 04:45:12,917][105692] Updated weights for policy 0, policy_version 1833713 (0.0009) [2023-12-27 04:45:12,971][105692] Updated weights for policy 0, policy_version 1833723 (0.0010) [2023-12-27 04:45:13,019][105692] Updated weights for policy 0, policy_version 1833733 (0.0007) [2023-12-27 04:45:13,032][105620] Updated weights for policy 1, policy_version 1837821 (0.0011) [2023-12-27 04:45:13,071][105692] Updated weights for policy 0, policy_version 1833743 (0.0008) [2023-12-27 04:45:13,080][105620] Updated weights for policy 1, policy_version 1837831 (0.0010) [2023-12-27 04:45:13,128][105620] Updated weights for policy 1, policy_version 1837841 (0.0010) [2023-12-27 04:45:13,813][105692] Updated weights for policy 0, policy_version 1833753 (0.0009) [2023-12-27 04:45:13,817][105620] Updated weights for policy 1, policy_version 1837851 (0.0009) [2023-12-27 04:45:13,864][105692] Updated weights for policy 0, policy_version 1833763 (0.0006) [2023-12-27 04:45:13,878][105620] Updated weights for policy 1, policy_version 1837861 (0.0011) [2023-12-27 04:45:13,928][105692] Updated weights for policy 0, policy_version 1833773 (0.0005) [2023-12-27 04:45:13,937][105620] Updated weights for policy 1, policy_version 1837871 (0.0010) [2023-12-27 04:45:14,564][105692] Updated weights for policy 0, policy_version 1833783 (0.0006) [2023-12-27 04:45:14,619][105692] Updated weights for policy 0, policy_version 1833793 (0.0006) [2023-12-27 04:45:14,653][105620] Updated weights for policy 1, policy_version 1837881 (0.0010) [2023-12-27 04:45:14,675][105692] Updated weights for policy 0, policy_version 1833803 (0.0007) [2023-12-27 04:45:14,704][105620] Updated weights for policy 1, policy_version 1837891 (0.0006) [2023-12-27 04:45:14,764][105620] Updated weights for policy 1, policy_version 1837901 (0.0009) [2023-12-27 04:45:14,827][105620] Updated weights for policy 1, policy_version 1837911 (0.0009) [2023-12-27 04:45:15,259][105692] Updated weights for policy 0, policy_version 1833813 (0.0008) [2023-12-27 04:45:15,321][105692] Updated weights for policy 0, policy_version 1833823 (0.0007) [2023-12-27 04:45:15,394][105692] Updated weights for policy 0, policy_version 1833833 (0.0006) [2023-12-27 04:45:15,507][105620] Updated weights for policy 1, policy_version 1837921 (0.0008) [2023-12-27 04:45:15,558][105620] Updated weights for policy 1, policy_version 1837932 (0.0010) [2023-12-27 04:45:15,622][105620] Updated weights for policy 1, policy_version 1837942 (0.0006) [2023-12-27 04:45:16,008][105692] Updated weights for policy 0, policy_version 1833843 (0.0007) [2023-12-27 04:45:16,062][104569] Fps is (10 sec: 18840.8, 60 sec: 19797.2, 300 sec: 19744.1). Total num frames: 940113920. Throughput: 0: 9757.4, 1: 9855.7. Samples: 940084968. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:45:16,064][104569] Avg episode reward: [(0, '8443.604'), (1, '9073.372')] [2023-12-27 04:45:16,064][105692] Updated weights for policy 0, policy_version 1833853 (0.0009) [2023-12-27 04:45:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001837944_470581248.pth... [2023-12-27 04:45:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001836792_470286336.pth [2023-12-27 04:45:16,119][105692] Updated weights for policy 0, policy_version 1833863 (0.0011) [2023-12-27 04:45:16,162][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001833872_469540864.pth... [2023-12-27 04:45:16,166][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001832688_469237760.pth [2023-12-27 04:45:16,298][105620] Updated weights for policy 1, policy_version 1837952 (0.0009) [2023-12-27 04:45:16,342][105620] Updated weights for policy 1, policy_version 1837962 (0.0005) [2023-12-27 04:45:16,388][105620] Updated weights for policy 1, policy_version 1837972 (0.0005) [2023-12-27 04:45:16,806][105692] Updated weights for policy 0, policy_version 1833873 (0.0010) [2023-12-27 04:45:16,865][105692] Updated weights for policy 0, policy_version 1833883 (0.0006) [2023-12-27 04:45:16,924][105692] Updated weights for policy 0, policy_version 1833893 (0.0006) [2023-12-27 04:45:16,984][105692] Updated weights for policy 0, policy_version 1833903 (0.0006) [2023-12-27 04:45:17,056][105620] Updated weights for policy 1, policy_version 1837982 (0.0005) [2023-12-27 04:45:17,112][105620] Updated weights for policy 1, policy_version 1837992 (0.0005) [2023-12-27 04:45:17,172][105620] Updated weights for policy 1, policy_version 1838002 (0.0006) [2023-12-27 04:45:17,523][105692] Updated weights for policy 0, policy_version 1833913 (0.0005) [2023-12-27 04:45:17,577][105692] Updated weights for policy 0, policy_version 1833923 (0.0008) [2023-12-27 04:45:17,635][105692] Updated weights for policy 0, policy_version 1833933 (0.0005) [2023-12-27 04:45:17,847][105620] Updated weights for policy 1, policy_version 1838012 (0.0009) [2023-12-27 04:45:17,909][105620] Updated weights for policy 1, policy_version 1838022 (0.0010) [2023-12-27 04:45:17,972][105620] Updated weights for policy 1, policy_version 1838032 (0.0010) [2023-12-27 04:45:18,366][105692] Updated weights for policy 0, policy_version 1833943 (0.0007) [2023-12-27 04:45:18,425][105692] Updated weights for policy 0, policy_version 1833953 (0.0010) [2023-12-27 04:45:18,485][105692] Updated weights for policy 0, policy_version 1833963 (0.0009) [2023-12-27 04:45:18,638][105620] Updated weights for policy 1, policy_version 1838042 (0.0010) [2023-12-27 04:45:18,702][105620] Updated weights for policy 1, policy_version 1838052 (0.0010) [2023-12-27 04:45:18,763][105620] Updated weights for policy 1, policy_version 1838062 (0.0009) [2023-12-27 04:45:18,810][105620] Updated weights for policy 1, policy_version 1838072 (0.0009) [2023-12-27 04:45:19,265][105692] Updated weights for policy 0, policy_version 1833973 (0.0008) [2023-12-27 04:45:19,314][105692] Updated weights for policy 0, policy_version 1833983 (0.0008) [2023-12-27 04:45:19,384][105692] Updated weights for policy 0, policy_version 1833993 (0.0006) [2023-12-27 04:45:19,608][105620] Updated weights for policy 1, policy_version 1838082 (0.0006) [2023-12-27 04:45:19,679][105620] Updated weights for policy 1, policy_version 1838092 (0.0006) [2023-12-27 04:45:19,744][105620] Updated weights for policy 1, policy_version 1838102 (0.0008) [2023-12-27 04:45:20,098][105692] Updated weights for policy 0, policy_version 1834003 (0.0008) [2023-12-27 04:45:20,152][105692] Updated weights for policy 0, policy_version 1834013 (0.0009) [2023-12-27 04:45:20,222][105692] Updated weights for policy 0, policy_version 1834023 (0.0010) [2023-12-27 04:45:20,422][105620] Updated weights for policy 1, policy_version 1838112 (0.0007) [2023-12-27 04:45:20,477][105620] Updated weights for policy 1, policy_version 1838122 (0.0006) [2023-12-27 04:45:20,530][105620] Updated weights for policy 1, policy_version 1838132 (0.0005) [2023-12-27 04:45:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.8, 300 sec: 19771.9). Total num frames: 940212224. Throughput: 0: 9846.7, 1: 9837.1. Samples: 940207028. Policy #0 lag: (min: 5.0, avg: 16.2, max: 37.0) [2023-12-27 04:45:21,063][104569] Avg episode reward: [(0, '8809.628'), (1, '9165.600')] [2023-12-27 04:45:21,083][105692] Updated weights for policy 0, policy_version 1834033 (0.0009) [2023-12-27 04:45:21,147][105692] Updated weights for policy 0, policy_version 1834043 (0.0009) [2023-12-27 04:45:21,187][105620] Updated weights for policy 1, policy_version 1838142 (0.0006) [2023-12-27 04:45:21,211][105692] Updated weights for policy 0, policy_version 1834053 (0.0009) [2023-12-27 04:45:21,250][105620] Updated weights for policy 1, policy_version 1838152 (0.0007) [2023-12-27 04:45:21,280][105692] Updated weights for policy 0, policy_version 1834063 (0.0008) [2023-12-27 04:45:21,313][105620] Updated weights for policy 1, policy_version 1838162 (0.0008) [2023-12-27 04:45:22,031][105620] Updated weights for policy 1, policy_version 1838172 (0.0008) [2023-12-27 04:45:22,071][105692] Updated weights for policy 0, policy_version 1834073 (0.0009) [2023-12-27 04:45:22,092][105620] Updated weights for policy 1, policy_version 1838182 (0.0008) [2023-12-27 04:45:22,132][105692] Updated weights for policy 0, policy_version 1834083 (0.0007) [2023-12-27 04:45:22,150][105620] Updated weights for policy 1, policy_version 1838192 (0.0006) [2023-12-27 04:45:22,199][105692] Updated weights for policy 0, policy_version 1834093 (0.0008) [2023-12-27 04:45:22,893][105620] Updated weights for policy 1, policy_version 1838202 (0.0006) [2023-12-27 04:45:22,904][105692] Updated weights for policy 0, policy_version 1834103 (0.0007) [2023-12-27 04:45:22,956][105620] Updated weights for policy 1, policy_version 1838212 (0.0009) [2023-12-27 04:45:22,969][105692] Updated weights for policy 0, policy_version 1834113 (0.0006) [2023-12-27 04:45:23,020][105620] Updated weights for policy 1, policy_version 1838222 (0.0008) [2023-12-27 04:45:23,036][105692] Updated weights for policy 0, policy_version 1834123 (0.0006) [2023-12-27 04:45:23,080][105620] Updated weights for policy 1, policy_version 1838232 (0.0009) [2023-12-27 04:45:23,632][105692] Updated weights for policy 0, policy_version 1834133 (0.0006) [2023-12-27 04:45:23,680][105692] Updated weights for policy 0, policy_version 1834143 (0.0005) [2023-12-27 04:45:23,739][105692] Updated weights for policy 0, policy_version 1834153 (0.0005) [2023-12-27 04:45:23,923][105620] Updated weights for policy 1, policy_version 1838242 (0.0009) [2023-12-27 04:45:23,988][105620] Updated weights for policy 1, policy_version 1838252 (0.0009) [2023-12-27 04:45:24,058][105620] Updated weights for policy 1, policy_version 1838262 (0.0010) [2023-12-27 04:45:24,287][105692] Updated weights for policy 0, policy_version 1834163 (0.0006) [2023-12-27 04:45:24,343][105692] Updated weights for policy 0, policy_version 1834173 (0.0007) [2023-12-27 04:45:24,398][105692] Updated weights for policy 0, policy_version 1834183 (0.0009) [2023-12-27 04:45:24,848][105620] Updated weights for policy 1, policy_version 1838272 (0.0009) [2023-12-27 04:45:24,893][105620] Updated weights for policy 1, policy_version 1838282 (0.0008) [2023-12-27 04:45:24,948][105620] Updated weights for policy 1, policy_version 1838292 (0.0009) [2023-12-27 04:45:25,127][105692] Updated weights for policy 0, policy_version 1834193 (0.0008) [2023-12-27 04:45:25,190][105692] Updated weights for policy 0, policy_version 1834203 (0.0007) [2023-12-27 04:45:25,250][105692] Updated weights for policy 0, policy_version 1834213 (0.0007) [2023-12-27 04:45:25,301][105692] Updated weights for policy 0, policy_version 1834223 (0.0006) [2023-12-27 04:45:25,672][105620] Updated weights for policy 1, policy_version 1838302 (0.0007) [2023-12-27 04:45:25,727][105620] Updated weights for policy 1, policy_version 1838312 (0.0006) [2023-12-27 04:45:25,781][105620] Updated weights for policy 1, policy_version 1838322 (0.0006) [2023-12-27 04:45:26,022][105692] Updated weights for policy 0, policy_version 1834233 (0.0009) [2023-12-27 04:45:26,062][104569] Fps is (10 sec: 19661.7, 60 sec: 19660.8, 300 sec: 19744.1). Total num frames: 940310528. Throughput: 0: 9934.4, 1: 9810.2. Samples: 940322636. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:45:26,062][104569] Avg episode reward: [(0, '8808.690'), (1, '9257.466')] [2023-12-27 04:45:26,075][105692] Updated weights for policy 0, policy_version 1834243 (0.0010) [2023-12-27 04:45:26,131][105692] Updated weights for policy 0, policy_version 1834253 (0.0013) [2023-12-27 04:45:26,319][105620] Updated weights for policy 1, policy_version 1838332 (0.0006) [2023-12-27 04:45:26,385][105620] Updated weights for policy 1, policy_version 1838342 (0.0005) [2023-12-27 04:45:26,449][105620] Updated weights for policy 1, policy_version 1838352 (0.0005) [2023-12-27 04:45:27,020][105692] Updated weights for policy 0, policy_version 1834263 (0.0010) [2023-12-27 04:45:27,037][105620] Updated weights for policy 1, policy_version 1838362 (0.0006) [2023-12-27 04:45:27,068][105692] Updated weights for policy 0, policy_version 1834273 (0.0009) [2023-12-27 04:45:27,093][105620] Updated weights for policy 1, policy_version 1838372 (0.0005) [2023-12-27 04:45:27,125][105692] Updated weights for policy 0, policy_version 1834283 (0.0008) [2023-12-27 04:45:27,141][105620] Updated weights for policy 1, policy_version 1838382 (0.0005) [2023-12-27 04:45:27,199][105620] Updated weights for policy 1, policy_version 1838392 (0.0006) [2023-12-27 04:45:27,851][105620] Updated weights for policy 1, policy_version 1838402 (0.0009) [2023-12-27 04:45:27,897][105692] Updated weights for policy 0, policy_version 1834293 (0.0008) [2023-12-27 04:45:27,899][105620] Updated weights for policy 1, policy_version 1838412 (0.0007) [2023-12-27 04:45:27,942][105692] Updated weights for policy 0, policy_version 1834303 (0.0005) [2023-12-27 04:45:27,944][105620] Updated weights for policy 1, policy_version 1838422 (0.0006) [2023-12-27 04:45:27,986][105692] Updated weights for policy 0, policy_version 1834313 (0.0007) [2023-12-27 04:45:28,663][105692] Updated weights for policy 0, policy_version 1834324 (0.0009) [2023-12-27 04:45:28,720][105692] Updated weights for policy 0, policy_version 1834334 (0.0007) [2023-12-27 04:45:28,727][105620] Updated weights for policy 1, policy_version 1838432 (0.0007) [2023-12-27 04:45:28,768][105692] Updated weights for policy 0, policy_version 1834344 (0.0005) [2023-12-27 04:45:28,787][105620] Updated weights for policy 1, policy_version 1838442 (0.0008) [2023-12-27 04:45:28,844][105620] Updated weights for policy 1, policy_version 1838452 (0.0008) [2023-12-27 04:45:29,463][105692] Updated weights for policy 0, policy_version 1834354 (0.0006) [2023-12-27 04:45:29,514][105692] Updated weights for policy 0, policy_version 1834364 (0.0005) [2023-12-27 04:45:29,573][105692] Updated weights for policy 0, policy_version 1834374 (0.0005) [2023-12-27 04:45:29,629][105692] Updated weights for policy 0, policy_version 1834384 (0.0005) [2023-12-27 04:45:29,674][105620] Updated weights for policy 1, policy_version 1838462 (0.0009) [2023-12-27 04:45:29,730][105620] Updated weights for policy 1, policy_version 1838472 (0.0009) [2023-12-27 04:45:29,788][105620] Updated weights for policy 1, policy_version 1838482 (0.0009) [2023-12-27 04:45:30,298][105692] Updated weights for policy 0, policy_version 1834394 (0.0008) [2023-12-27 04:45:30,356][105692] Updated weights for policy 0, policy_version 1834404 (0.0008) [2023-12-27 04:45:30,415][105692] Updated weights for policy 0, policy_version 1834414 (0.0006) [2023-12-27 04:45:30,571][105620] Updated weights for policy 1, policy_version 1838492 (0.0009) [2023-12-27 04:45:30,637][105620] Updated weights for policy 1, policy_version 1838502 (0.0009) [2023-12-27 04:45:30,687][105620] Updated weights for policy 1, policy_version 1838512 (0.0009) [2023-12-27 04:45:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19716.3). Total num frames: 940408832. Throughput: 0: 9942.0, 1: 9865.1. Samples: 940381724. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:45:31,063][104569] Avg episode reward: [(0, '8626.591'), (1, '8980.043')] [2023-12-27 04:45:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001834416_469680128.pth... [2023-12-27 04:45:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001838520_470728704.pth... [2023-12-27 04:45:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001837368_470433792.pth [2023-12-27 04:45:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001833296_469393408.pth [2023-12-27 04:45:31,144][105692] Updated weights for policy 0, policy_version 1834424 (0.0009) [2023-12-27 04:45:31,208][105692] Updated weights for policy 0, policy_version 1834434 (0.0008) [2023-12-27 04:45:31,273][105692] Updated weights for policy 0, policy_version 1834444 (0.0009) [2023-12-27 04:45:31,442][105620] Updated weights for policy 1, policy_version 1838522 (0.0009) [2023-12-27 04:45:31,503][105620] Updated weights for policy 1, policy_version 1838532 (0.0010) [2023-12-27 04:45:31,557][105620] Updated weights for policy 1, policy_version 1838542 (0.0010) [2023-12-27 04:45:31,622][105620] Updated weights for policy 1, policy_version 1838552 (0.0009) [2023-12-27 04:45:31,999][105692] Updated weights for policy 0, policy_version 1834454 (0.0008) [2023-12-27 04:45:32,066][105692] Updated weights for policy 0, policy_version 1834464 (0.0009) [2023-12-27 04:45:32,130][105692] Updated weights for policy 0, policy_version 1834474 (0.0011) [2023-12-27 04:45:32,400][105620] Updated weights for policy 1, policy_version 1838562 (0.0008) [2023-12-27 04:45:32,448][105620] Updated weights for policy 1, policy_version 1838572 (0.0008) [2023-12-27 04:45:32,506][105620] Updated weights for policy 1, policy_version 1838582 (0.0009) [2023-12-27 04:45:32,810][105692] Updated weights for policy 0, policy_version 1834484 (0.0009) [2023-12-27 04:45:32,872][105692] Updated weights for policy 0, policy_version 1834494 (0.0005) [2023-12-27 04:45:32,928][105692] Updated weights for policy 0, policy_version 1834504 (0.0005) [2023-12-27 04:45:33,167][105620] Updated weights for policy 1, policy_version 1838592 (0.0008) [2023-12-27 04:45:33,223][105620] Updated weights for policy 1, policy_version 1838602 (0.0009) [2023-12-27 04:45:33,288][105620] Updated weights for policy 1, policy_version 1838612 (0.0005) [2023-12-27 04:45:33,567][105692] Updated weights for policy 0, policy_version 1834514 (0.0006) [2023-12-27 04:45:33,621][105692] Updated weights for policy 0, policy_version 1834524 (0.0009) [2023-12-27 04:45:33,681][105692] Updated weights for policy 0, policy_version 1834534 (0.0008) [2023-12-27 04:45:33,744][105692] Updated weights for policy 0, policy_version 1834544 (0.0008) [2023-12-27 04:45:33,874][105620] Updated weights for policy 1, policy_version 1838622 (0.0008) [2023-12-27 04:45:33,939][105620] Updated weights for policy 1, policy_version 1838632 (0.0010) [2023-12-27 04:45:34,000][105620] Updated weights for policy 1, policy_version 1838642 (0.0010) [2023-12-27 04:45:34,425][105692] Updated weights for policy 0, policy_version 1834554 (0.0006) [2023-12-27 04:45:34,491][105692] Updated weights for policy 0, policy_version 1834564 (0.0008) [2023-12-27 04:45:34,558][105692] Updated weights for policy 0, policy_version 1834574 (0.0007) [2023-12-27 04:45:34,633][105620] Updated weights for policy 1, policy_version 1838652 (0.0010) [2023-12-27 04:45:34,685][105620] Updated weights for policy 1, policy_version 1838662 (0.0011) [2023-12-27 04:45:34,744][105620] Updated weights for policy 1, policy_version 1838672 (0.0011) [2023-12-27 04:45:35,133][105692] Updated weights for policy 0, policy_version 1834584 (0.0010) [2023-12-27 04:45:35,195][105692] Updated weights for policy 0, policy_version 1834594 (0.0011) [2023-12-27 04:45:35,253][105692] Updated weights for policy 0, policy_version 1834604 (0.0010) [2023-12-27 04:45:35,518][105620] Updated weights for policy 1, policy_version 1838682 (0.0011) [2023-12-27 04:45:35,582][105620] Updated weights for policy 1, policy_version 1838692 (0.0010) [2023-12-27 04:45:35,634][105620] Updated weights for policy 1, policy_version 1838702 (0.0011) [2023-12-27 04:45:35,681][105620] Updated weights for policy 1, policy_version 1838712 (0.0010) [2023-12-27 04:45:35,807][105692] Updated weights for policy 0, policy_version 1834614 (0.0006) [2023-12-27 04:45:35,871][105692] Updated weights for policy 0, policy_version 1834624 (0.0005) [2023-12-27 04:45:35,919][105692] Updated weights for policy 0, policy_version 1834634 (0.0005) [2023-12-27 04:45:36,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19797.4, 300 sec: 19744.1). Total num frames: 940515328. Throughput: 0: 9902.2, 1: 9870.3. Samples: 940499812. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:45:36,062][104569] Avg episode reward: [(0, '8267.960'), (1, '9072.564')] [2023-12-27 04:45:36,338][105620] Updated weights for policy 1, policy_version 1838722 (0.0007) [2023-12-27 04:45:36,391][105620] Updated weights for policy 1, policy_version 1838732 (0.0006) [2023-12-27 04:45:36,452][105620] Updated weights for policy 1, policy_version 1838742 (0.0005) [2023-12-27 04:45:36,547][105692] Updated weights for policy 0, policy_version 1834644 (0.0005) [2023-12-27 04:45:36,610][105692] Updated weights for policy 0, policy_version 1834654 (0.0006) [2023-12-27 04:45:36,676][105692] Updated weights for policy 0, policy_version 1834664 (0.0006) [2023-12-27 04:45:37,168][105620] Updated weights for policy 1, policy_version 1838752 (0.0006) [2023-12-27 04:45:37,229][105620] Updated weights for policy 1, policy_version 1838762 (0.0006) [2023-12-27 04:45:37,236][105692] Updated weights for policy 0, policy_version 1834674 (0.0007) [2023-12-27 04:45:37,285][105620] Updated weights for policy 1, policy_version 1838772 (0.0006) [2023-12-27 04:45:37,302][105692] Updated weights for policy 0, policy_version 1834684 (0.0011) [2023-12-27 04:45:37,362][105692] Updated weights for policy 0, policy_version 1834694 (0.0010) [2023-12-27 04:45:37,432][105692] Updated weights for policy 0, policy_version 1834704 (0.0005) [2023-12-27 04:45:37,970][105620] Updated weights for policy 1, policy_version 1838782 (0.0009) [2023-12-27 04:45:37,986][105692] Updated weights for policy 0, policy_version 1834714 (0.0005) [2023-12-27 04:45:38,031][105620] Updated weights for policy 1, policy_version 1838792 (0.0011) [2023-12-27 04:45:38,049][105692] Updated weights for policy 0, policy_version 1834724 (0.0006) [2023-12-27 04:45:38,091][105620] Updated weights for policy 1, policy_version 1838802 (0.0010) [2023-12-27 04:45:38,103][105692] Updated weights for policy 0, policy_version 1834734 (0.0006) [2023-12-27 04:45:38,755][105692] Updated weights for policy 0, policy_version 1834744 (0.0010) [2023-12-27 04:45:38,818][105692] Updated weights for policy 0, policy_version 1834754 (0.0011) [2023-12-27 04:45:38,844][105620] Updated weights for policy 1, policy_version 1838812 (0.0010) [2023-12-27 04:45:38,876][105692] Updated weights for policy 0, policy_version 1834764 (0.0011) [2023-12-27 04:45:38,899][105620] Updated weights for policy 1, policy_version 1838822 (0.0006) [2023-12-27 04:45:38,947][105620] Updated weights for policy 1, policy_version 1838832 (0.0008) [2023-12-27 04:45:39,578][105692] Updated weights for policy 0, policy_version 1834774 (0.0008) [2023-12-27 04:45:39,634][105692] Updated weights for policy 0, policy_version 1834784 (0.0009) [2023-12-27 04:45:39,682][105692] Updated weights for policy 0, policy_version 1834794 (0.0009) [2023-12-27 04:45:39,698][105620] Updated weights for policy 1, policy_version 1838842 (0.0008) [2023-12-27 04:45:39,751][105620] Updated weights for policy 1, policy_version 1838852 (0.0008) [2023-12-27 04:45:39,810][105620] Updated weights for policy 1, policy_version 1838862 (0.0009) [2023-12-27 04:45:39,871][105620] Updated weights for policy 1, policy_version 1838872 (0.0008) [2023-12-27 04:45:40,429][105692] Updated weights for policy 0, policy_version 1834804 (0.0009) [2023-12-27 04:45:40,495][105692] Updated weights for policy 0, policy_version 1834814 (0.0008) [2023-12-27 04:45:40,561][105692] Updated weights for policy 0, policy_version 1834824 (0.0008) [2023-12-27 04:45:40,650][105620] Updated weights for policy 1, policy_version 1838882 (0.0008) [2023-12-27 04:45:40,706][105620] Updated weights for policy 1, policy_version 1838892 (0.0009) [2023-12-27 04:45:40,763][105620] Updated weights for policy 1, policy_version 1838902 (0.0009) [2023-12-27 04:45:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19744.1). Total num frames: 940613632. Throughput: 0: 9972.0, 1: 9836.9. Samples: 940621852. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:45:41,063][104569] Avg episode reward: [(0, '8270.762'), (1, '9349.867')] [2023-12-27 04:45:41,232][105692] Updated weights for policy 0, policy_version 1834834 (0.0009) [2023-12-27 04:45:41,293][105692] Updated weights for policy 0, policy_version 1834844 (0.0009) [2023-12-27 04:45:41,355][105692] Updated weights for policy 0, policy_version 1834854 (0.0009) [2023-12-27 04:45:41,428][105692] Updated weights for policy 0, policy_version 1834864 (0.0010) [2023-12-27 04:45:41,536][105620] Updated weights for policy 1, policy_version 1838912 (0.0009) [2023-12-27 04:45:41,594][105620] Updated weights for policy 1, policy_version 1838922 (0.0009) [2023-12-27 04:45:41,655][105620] Updated weights for policy 1, policy_version 1838932 (0.0009) [2023-12-27 04:45:42,247][105692] Updated weights for policy 0, policy_version 1834874 (0.0009) [2023-12-27 04:45:42,304][105692] Updated weights for policy 0, policy_version 1834884 (0.0009) [2023-12-27 04:45:42,367][105620] Updated weights for policy 1, policy_version 1838942 (0.0008) [2023-12-27 04:45:42,367][105692] Updated weights for policy 0, policy_version 1834894 (0.0008) [2023-12-27 04:45:42,442][105620] Updated weights for policy 1, policy_version 1838952 (0.0008) [2023-12-27 04:45:42,507][105620] Updated weights for policy 1, policy_version 1838962 (0.0008) [2023-12-27 04:45:43,171][105620] Updated weights for policy 1, policy_version 1838972 (0.0008) [2023-12-27 04:45:43,180][105692] Updated weights for policy 0, policy_version 1834904 (0.0009) [2023-12-27 04:45:43,227][105620] Updated weights for policy 1, policy_version 1838982 (0.0007) [2023-12-27 04:45:43,229][105692] Updated weights for policy 0, policy_version 1834914 (0.0006) [2023-12-27 04:45:43,286][105692] Updated weights for policy 0, policy_version 1834924 (0.0009) [2023-12-27 04:45:43,288][105620] Updated weights for policy 1, policy_version 1838992 (0.0007) [2023-12-27 04:45:43,960][105620] Updated weights for policy 1, policy_version 1839002 (0.0008) [2023-12-27 04:45:44,010][105620] Updated weights for policy 1, policy_version 1839012 (0.0005) [2023-12-27 04:45:44,060][105620] Updated weights for policy 1, policy_version 1839022 (0.0009) [2023-12-27 04:45:44,083][105692] Updated weights for policy 0, policy_version 1834934 (0.0009) [2023-12-27 04:45:44,110][105620] Updated weights for policy 1, policy_version 1839032 (0.0007) [2023-12-27 04:45:44,139][105692] Updated weights for policy 0, policy_version 1834944 (0.0008) [2023-12-27 04:45:44,194][105692] Updated weights for policy 0, policy_version 1834954 (0.0009) [2023-12-27 04:45:44,859][105620] Updated weights for policy 1, policy_version 1839042 (0.0006) [2023-12-27 04:45:44,908][105620] Updated weights for policy 1, policy_version 1839052 (0.0009) [2023-12-27 04:45:44,954][105620] Updated weights for policy 1, policy_version 1839062 (0.0008) [2023-12-27 04:45:44,960][105692] Updated weights for policy 0, policy_version 1834964 (0.0007) [2023-12-27 04:45:45,015][105692] Updated weights for policy 0, policy_version 1834974 (0.0009) [2023-12-27 04:45:45,077][105692] Updated weights for policy 0, policy_version 1834984 (0.0009) [2023-12-27 04:45:45,726][105620] Updated weights for policy 1, policy_version 1839072 (0.0009) [2023-12-27 04:45:45,774][105620] Updated weights for policy 1, policy_version 1839082 (0.0009) [2023-12-27 04:45:45,824][105620] Updated weights for policy 1, policy_version 1839092 (0.0009) [2023-12-27 04:45:45,835][105692] Updated weights for policy 0, policy_version 1834994 (0.0009) [2023-12-27 04:45:45,884][105692] Updated weights for policy 0, policy_version 1835004 (0.0008) [2023-12-27 04:45:45,941][105692] Updated weights for policy 0, policy_version 1835014 (0.0009) [2023-12-27 04:45:46,002][105692] Updated weights for policy 0, policy_version 1835024 (0.0009) [2023-12-27 04:45:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19716.3). Total num frames: 940711936. Throughput: 0: 9843.7, 1: 9861.4. Samples: 940678596. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:45:46,062][104569] Avg episode reward: [(0, '8540.358'), (1, '9349.840')] [2023-12-27 04:45:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001835024_469835776.pth... [2023-12-27 04:45:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001839096_470876160.pth... [2023-12-27 04:45:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001833872_469540864.pth [2023-12-27 04:45:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001837944_470581248.pth [2023-12-27 04:45:46,502][105620] Updated weights for policy 1, policy_version 1839102 (0.0009) [2023-12-27 04:45:46,552][105620] Updated weights for policy 1, policy_version 1839112 (0.0009) [2023-12-27 04:45:46,601][105620] Updated weights for policy 1, policy_version 1839122 (0.0008) [2023-12-27 04:45:46,685][105692] Updated weights for policy 0, policy_version 1835034 (0.0009) [2023-12-27 04:45:46,744][105692] Updated weights for policy 0, policy_version 1835044 (0.0009) [2023-12-27 04:45:46,802][105692] Updated weights for policy 0, policy_version 1835055 (0.0010) [2023-12-27 04:45:47,251][105620] Updated weights for policy 1, policy_version 1839132 (0.0009) [2023-12-27 04:45:47,313][105620] Updated weights for policy 1, policy_version 1839142 (0.0010) [2023-12-27 04:45:47,365][105620] Updated weights for policy 1, policy_version 1839152 (0.0010) [2023-12-27 04:45:47,413][105692] Updated weights for policy 0, policy_version 1835065 (0.0007) [2023-12-27 04:45:47,462][105692] Updated weights for policy 0, policy_version 1835075 (0.0005) [2023-12-27 04:45:47,519][105692] Updated weights for policy 0, policy_version 1835085 (0.0005) [2023-12-27 04:45:48,089][105620] Updated weights for policy 1, policy_version 1839162 (0.0008) [2023-12-27 04:45:48,157][105620] Updated weights for policy 1, policy_version 1839172 (0.0005) [2023-12-27 04:45:48,217][105620] Updated weights for policy 1, policy_version 1839182 (0.0007) [2023-12-27 04:45:48,227][105692] Updated weights for policy 0, policy_version 1835095 (0.0007) [2023-12-27 04:45:48,275][105620] Updated weights for policy 1, policy_version 1839192 (0.0006) [2023-12-27 04:45:48,298][105692] Updated weights for policy 0, policy_version 1835105 (0.0009) [2023-12-27 04:45:48,369][105692] Updated weights for policy 0, policy_version 1835115 (0.0008) [2023-12-27 04:45:48,944][105692] Updated weights for policy 0, policy_version 1835125 (0.0009) [2023-12-27 04:45:48,956][105620] Updated weights for policy 1, policy_version 1839202 (0.0011) [2023-12-27 04:45:48,995][105692] Updated weights for policy 0, policy_version 1835135 (0.0010) [2023-12-27 04:45:49,014][105620] Updated weights for policy 1, policy_version 1839212 (0.0010) [2023-12-27 04:45:49,058][105692] Updated weights for policy 0, policy_version 1835145 (0.0008) [2023-12-27 04:45:49,069][105620] Updated weights for policy 1, policy_version 1839222 (0.0010) [2023-12-27 04:45:49,827][105620] Updated weights for policy 1, policy_version 1839232 (0.0009) [2023-12-27 04:45:49,835][105692] Updated weights for policy 0, policy_version 1835155 (0.0010) [2023-12-27 04:45:49,892][105620] Updated weights for policy 1, policy_version 1839242 (0.0007) [2023-12-27 04:45:49,897][105692] Updated weights for policy 0, policy_version 1835165 (0.0008) [2023-12-27 04:45:49,953][105620] Updated weights for policy 1, policy_version 1839252 (0.0008) [2023-12-27 04:45:49,958][105692] Updated weights for policy 0, policy_version 1835175 (0.0008) [2023-12-27 04:45:50,655][105620] Updated weights for policy 1, policy_version 1839262 (0.0010) [2023-12-27 04:45:50,718][105620] Updated weights for policy 1, policy_version 1839272 (0.0010) [2023-12-27 04:45:50,719][105692] Updated weights for policy 0, policy_version 1835185 (0.0008) [2023-12-27 04:45:50,777][105620] Updated weights for policy 1, policy_version 1839282 (0.0011) [2023-12-27 04:45:50,780][105692] Updated weights for policy 0, policy_version 1835195 (0.0006) [2023-12-27 04:45:50,830][105692] Updated weights for policy 0, policy_version 1835205 (0.0006) [2023-12-27 04:45:50,880][105692] Updated weights for policy 0, policy_version 1835215 (0.0005) [2023-12-27 04:45:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19744.1). Total num frames: 940810240. Throughput: 0: 9904.6, 1: 9861.1. Samples: 940796296. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:45:51,063][104569] Avg episode reward: [(0, '8446.358'), (1, '9257.571')] [2023-12-27 04:45:51,558][105620] Updated weights for policy 1, policy_version 1839292 (0.0009) [2023-12-27 04:45:51,609][105692] Updated weights for policy 0, policy_version 1835225 (0.0006) [2023-12-27 04:45:51,621][105620] Updated weights for policy 1, policy_version 1839302 (0.0006) [2023-12-27 04:45:51,676][105692] Updated weights for policy 0, policy_version 1835235 (0.0007) [2023-12-27 04:45:51,691][105620] Updated weights for policy 1, policy_version 1839312 (0.0008) [2023-12-27 04:45:51,741][105692] Updated weights for policy 0, policy_version 1835245 (0.0010) [2023-12-27 04:45:52,364][105692] Updated weights for policy 0, policy_version 1835255 (0.0008) [2023-12-27 04:45:52,419][105620] Updated weights for policy 1, policy_version 1839322 (0.0008) [2023-12-27 04:45:52,430][105692] Updated weights for policy 0, policy_version 1835265 (0.0007) [2023-12-27 04:45:52,486][105620] Updated weights for policy 1, policy_version 1839332 (0.0008) [2023-12-27 04:45:52,495][105692] Updated weights for policy 0, policy_version 1835275 (0.0006) [2023-12-27 04:45:52,550][105620] Updated weights for policy 1, policy_version 1839342 (0.0009) [2023-12-27 04:45:52,617][105620] Updated weights for policy 1, policy_version 1839352 (0.0009) [2023-12-27 04:45:53,243][105620] Updated weights for policy 1, policy_version 1839362 (0.0009) [2023-12-27 04:45:53,253][105692] Updated weights for policy 0, policy_version 1835285 (0.0006) [2023-12-27 04:45:53,301][105620] Updated weights for policy 1, policy_version 1839372 (0.0008) [2023-12-27 04:45:53,302][105692] Updated weights for policy 0, policy_version 1835295 (0.0008) [2023-12-27 04:45:53,350][105692] Updated weights for policy 0, policy_version 1835305 (0.0007) [2023-12-27 04:45:53,356][105620] Updated weights for policy 1, policy_version 1839382 (0.0007) [2023-12-27 04:45:54,051][105692] Updated weights for policy 0, policy_version 1835315 (0.0007) [2023-12-27 04:45:54,108][105692] Updated weights for policy 0, policy_version 1835325 (0.0009) [2023-12-27 04:45:54,144][105620] Updated weights for policy 1, policy_version 1839392 (0.0008) [2023-12-27 04:45:54,158][105692] Updated weights for policy 0, policy_version 1835335 (0.0006) [2023-12-27 04:45:54,196][105620] Updated weights for policy 1, policy_version 1839402 (0.0007) [2023-12-27 04:45:54,256][105620] Updated weights for policy 1, policy_version 1839412 (0.0008) [2023-12-27 04:45:54,873][105692] Updated weights for policy 0, policy_version 1835345 (0.0008) [2023-12-27 04:45:54,924][105692] Updated weights for policy 0, policy_version 1835355 (0.0009) [2023-12-27 04:45:54,986][105692] Updated weights for policy 0, policy_version 1835365 (0.0005) [2023-12-27 04:45:55,047][105620] Updated weights for policy 1, policy_version 1839422 (0.0007) [2023-12-27 04:45:55,049][105692] Updated weights for policy 0, policy_version 1835375 (0.0008) [2023-12-27 04:45:55,106][105620] Updated weights for policy 1, policy_version 1839432 (0.0005) [2023-12-27 04:45:55,157][105620] Updated weights for policy 1, policy_version 1839442 (0.0005) [2023-12-27 04:45:55,785][105692] Updated weights for policy 0, policy_version 1835385 (0.0006) [2023-12-27 04:45:55,810][105620] Updated weights for policy 1, policy_version 1839452 (0.0006) [2023-12-27 04:45:55,838][105692] Updated weights for policy 0, policy_version 1835395 (0.0006) [2023-12-27 04:45:55,868][105620] Updated weights for policy 1, policy_version 1839462 (0.0007) [2023-12-27 04:45:55,896][105692] Updated weights for policy 0, policy_version 1835405 (0.0011) [2023-12-27 04:45:55,924][105620] Updated weights for policy 1, policy_version 1839472 (0.0007) [2023-12-27 04:45:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19744.1). Total num frames: 940908544. Throughput: 0: 9928.6, 1: 9763.6. Samples: 940911572. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:45:56,062][104569] Avg episode reward: [(0, '8170.804'), (1, '9165.223')] [2023-12-27 04:45:56,463][105692] Updated weights for policy 0, policy_version 1835415 (0.0006) [2023-12-27 04:45:56,523][105692] Updated weights for policy 0, policy_version 1835425 (0.0005) [2023-12-27 04:45:56,571][105692] Updated weights for policy 0, policy_version 1835435 (0.0006) [2023-12-27 04:45:56,765][105620] Updated weights for policy 1, policy_version 1839482 (0.0008) [2023-12-27 04:45:56,825][105620] Updated weights for policy 1, policy_version 1839492 (0.0009) [2023-12-27 04:45:56,885][105620] Updated weights for policy 1, policy_version 1839502 (0.0008) [2023-12-27 04:45:56,948][105620] Updated weights for policy 1, policy_version 1839512 (0.0009) [2023-12-27 04:45:57,188][105692] Updated weights for policy 0, policy_version 1835445 (0.0007) [2023-12-27 04:45:57,240][105692] Updated weights for policy 0, policy_version 1835455 (0.0005) [2023-12-27 04:45:57,291][105692] Updated weights for policy 0, policy_version 1835465 (0.0007) [2023-12-27 04:45:57,684][105620] Updated weights for policy 1, policy_version 1839522 (0.0009) [2023-12-27 04:45:57,745][105620] Updated weights for policy 1, policy_version 1839532 (0.0009) [2023-12-27 04:45:57,798][105620] Updated weights for policy 1, policy_version 1839542 (0.0009) [2023-12-27 04:45:57,989][105692] Updated weights for policy 0, policy_version 1835475 (0.0010) [2023-12-27 04:45:58,044][105692] Updated weights for policy 0, policy_version 1835485 (0.0008) [2023-12-27 04:45:58,097][105692] Updated weights for policy 0, policy_version 1835495 (0.0010) [2023-12-27 04:45:58,526][105620] Updated weights for policy 1, policy_version 1839552 (0.0009) [2023-12-27 04:45:58,588][105620] Updated weights for policy 1, policy_version 1839562 (0.0008) [2023-12-27 04:45:58,653][105620] Updated weights for policy 1, policy_version 1839572 (0.0009) [2023-12-27 04:45:59,034][105692] Updated weights for policy 0, policy_version 1835505 (0.0010) [2023-12-27 04:45:59,096][105692] Updated weights for policy 0, policy_version 1835515 (0.0008) [2023-12-27 04:45:59,153][105692] Updated weights for policy 0, policy_version 1835525 (0.0008) [2023-12-27 04:45:59,207][105692] Updated weights for policy 0, policy_version 1835535 (0.0008) [2023-12-27 04:45:59,460][105620] Updated weights for policy 1, policy_version 1839582 (0.0008) [2023-12-27 04:45:59,512][105620] Updated weights for policy 1, policy_version 1839592 (0.0010) [2023-12-27 04:45:59,566][105620] Updated weights for policy 1, policy_version 1839602 (0.0008) [2023-12-27 04:45:59,972][105692] Updated weights for policy 0, policy_version 1835545 (0.0007) [2023-12-27 04:46:00,037][105692] Updated weights for policy 0, policy_version 1835555 (0.0008) [2023-12-27 04:46:00,096][105692] Updated weights for policy 0, policy_version 1835565 (0.0008) [2023-12-27 04:46:00,340][105620] Updated weights for policy 1, policy_version 1839612 (0.0008) [2023-12-27 04:46:00,399][105620] Updated weights for policy 1, policy_version 1839622 (0.0009) [2023-12-27 04:46:00,454][105620] Updated weights for policy 1, policy_version 1839632 (0.0009) [2023-12-27 04:46:00,795][105692] Updated weights for policy 0, policy_version 1835575 (0.0005) [2023-12-27 04:46:00,838][105692] Updated weights for policy 0, policy_version 1835585 (0.0005) [2023-12-27 04:46:00,881][105692] Updated weights for policy 0, policy_version 1835595 (0.0005) [2023-12-27 04:46:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19660.8, 300 sec: 19688.6). Total num frames: 940998656. Throughput: 0: 9969.5, 1: 9692.5. Samples: 940969752. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:46:01,063][104569] Avg episode reward: [(0, '8081.919'), (1, '9257.626')] [2023-12-27 04:46:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001835600_469983232.pth... [2023-12-27 04:46:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001839640_471015424.pth... [2023-12-27 04:46:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001834416_469680128.pth [2023-12-27 04:46:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001838520_470728704.pth [2023-12-27 04:46:01,213][105620] Updated weights for policy 1, policy_version 1839642 (0.0009) [2023-12-27 04:46:01,280][105620] Updated weights for policy 1, policy_version 1839652 (0.0011) [2023-12-27 04:46:01,349][105620] Updated weights for policy 1, policy_version 1839662 (0.0011) [2023-12-27 04:46:01,418][105620] Updated weights for policy 1, policy_version 1839672 (0.0008) [2023-12-27 04:46:01,573][105692] Updated weights for policy 0, policy_version 1835605 (0.0007) [2023-12-27 04:46:01,637][105692] Updated weights for policy 0, policy_version 1835615 (0.0008) [2023-12-27 04:46:01,696][105692] Updated weights for policy 0, policy_version 1835625 (0.0008) [2023-12-27 04:46:02,160][105620] Updated weights for policy 1, policy_version 1839682 (0.0008) [2023-12-27 04:46:02,216][105620] Updated weights for policy 1, policy_version 1839692 (0.0010) [2023-12-27 04:46:02,276][105620] Updated weights for policy 1, policy_version 1839703 (0.0010) [2023-12-27 04:46:02,421][105692] Updated weights for policy 0, policy_version 1835635 (0.0008) [2023-12-27 04:46:02,478][105692] Updated weights for policy 0, policy_version 1835645 (0.0010) [2023-12-27 04:46:02,536][105692] Updated weights for policy 0, policy_version 1835655 (0.0009) [2023-12-27 04:46:02,945][105620] Updated weights for policy 1, policy_version 1839713 (0.0006) [2023-12-27 04:46:03,005][105620] Updated weights for policy 1, policy_version 1839723 (0.0006) [2023-12-27 04:46:03,066][105620] Updated weights for policy 1, policy_version 1839733 (0.0006) [2023-12-27 04:46:03,397][105692] Updated weights for policy 0, policy_version 1835665 (0.0009) [2023-12-27 04:46:03,462][105692] Updated weights for policy 0, policy_version 1835675 (0.0010) [2023-12-27 04:46:03,522][105692] Updated weights for policy 0, policy_version 1835685 (0.0011) [2023-12-27 04:46:03,584][105692] Updated weights for policy 0, policy_version 1835695 (0.0011) [2023-12-27 04:46:03,754][105620] Updated weights for policy 1, policy_version 1839743 (0.0008) [2023-12-27 04:46:03,807][105620] Updated weights for policy 1, policy_version 1839753 (0.0008) [2023-12-27 04:46:03,870][105620] Updated weights for policy 1, policy_version 1839763 (0.0008) [2023-12-27 04:46:04,408][105692] Updated weights for policy 0, policy_version 1835705 (0.0011) [2023-12-27 04:46:04,468][105692] Updated weights for policy 0, policy_version 1835715 (0.0011) [2023-12-27 04:46:04,533][105692] Updated weights for policy 0, policy_version 1835725 (0.0011) [2023-12-27 04:46:04,665][105620] Updated weights for policy 1, policy_version 1839773 (0.0009) [2023-12-27 04:46:04,732][105620] Updated weights for policy 1, policy_version 1839783 (0.0008) [2023-12-27 04:46:04,791][105620] Updated weights for policy 1, policy_version 1839793 (0.0008) [2023-12-27 04:46:05,278][105692] Updated weights for policy 0, policy_version 1835735 (0.0009) [2023-12-27 04:46:05,334][105692] Updated weights for policy 0, policy_version 1835745 (0.0009) [2023-12-27 04:46:05,389][105692] Updated weights for policy 0, policy_version 1835755 (0.0010) [2023-12-27 04:46:05,544][105620] Updated weights for policy 1, policy_version 1839803 (0.0009) [2023-12-27 04:46:05,595][105620] Updated weights for policy 1, policy_version 1839813 (0.0009) [2023-12-27 04:46:05,654][105620] Updated weights for policy 1, policy_version 1839823 (0.0009) [2023-12-27 04:46:06,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 941088768. Throughput: 0: 9804.4, 1: 9608.2. Samples: 941080592. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:46:06,062][104569] Avg episode reward: [(0, '8265.742'), (1, '9257.797')] [2023-12-27 04:46:06,097][105692] Updated weights for policy 0, policy_version 1835765 (0.0009) [2023-12-27 04:46:06,162][105692] Updated weights for policy 0, policy_version 1835775 (0.0007) [2023-12-27 04:46:06,225][105692] Updated weights for policy 0, policy_version 1835785 (0.0009) [2023-12-27 04:46:06,511][105620] Updated weights for policy 1, policy_version 1839833 (0.0009) [2023-12-27 04:46:06,570][105620] Updated weights for policy 1, policy_version 1839843 (0.0009) [2023-12-27 04:46:06,625][105620] Updated weights for policy 1, policy_version 1839853 (0.0009) [2023-12-27 04:46:06,681][105620] Updated weights for policy 1, policy_version 1839863 (0.0009) [2023-12-27 04:46:06,983][105692] Updated weights for policy 0, policy_version 1835795 (0.0009) [2023-12-27 04:46:07,038][105692] Updated weights for policy 0, policy_version 1835805 (0.0009) [2023-12-27 04:46:07,100][105692] Updated weights for policy 0, policy_version 1835815 (0.0009) [2023-12-27 04:46:07,467][105620] Updated weights for policy 1, policy_version 1839873 (0.0008) [2023-12-27 04:46:07,531][105620] Updated weights for policy 1, policy_version 1839883 (0.0009) [2023-12-27 04:46:07,599][105620] Updated weights for policy 1, policy_version 1839893 (0.0006) [2023-12-27 04:46:07,893][105692] Updated weights for policy 0, policy_version 1835825 (0.0009) [2023-12-27 04:46:07,946][105692] Updated weights for policy 0, policy_version 1835835 (0.0008) [2023-12-27 04:46:08,001][105692] Updated weights for policy 0, policy_version 1835845 (0.0005) [2023-12-27 04:46:08,058][105692] Updated weights for policy 0, policy_version 1835855 (0.0005) [2023-12-27 04:46:08,345][105620] Updated weights for policy 1, policy_version 1839903 (0.0007) [2023-12-27 04:46:08,416][105620] Updated weights for policy 1, policy_version 1839913 (0.0007) [2023-12-27 04:46:08,482][105620] Updated weights for policy 1, policy_version 1839923 (0.0009) [2023-12-27 04:46:08,775][105692] Updated weights for policy 0, policy_version 1835865 (0.0008) [2023-12-27 04:46:08,842][105692] Updated weights for policy 0, policy_version 1835875 (0.0009) [2023-12-27 04:46:08,902][105692] Updated weights for policy 0, policy_version 1835885 (0.0009) [2023-12-27 04:46:09,216][105620] Updated weights for policy 1, policy_version 1839933 (0.0009) [2023-12-27 04:46:09,284][105620] Updated weights for policy 1, policy_version 1839943 (0.0008) [2023-12-27 04:46:09,339][105620] Updated weights for policy 1, policy_version 1839953 (0.0011) [2023-12-27 04:46:09,719][105692] Updated weights for policy 0, policy_version 1835895 (0.0008) [2023-12-27 04:46:09,776][105692] Updated weights for policy 0, policy_version 1835905 (0.0008) [2023-12-27 04:46:09,847][105692] Updated weights for policy 0, policy_version 1835915 (0.0008) [2023-12-27 04:46:10,140][105620] Updated weights for policy 1, policy_version 1839963 (0.0010) [2023-12-27 04:46:10,202][105620] Updated weights for policy 1, policy_version 1839973 (0.0008) [2023-12-27 04:46:10,263][105620] Updated weights for policy 1, policy_version 1839983 (0.0011) [2023-12-27 04:46:10,646][105692] Updated weights for policy 0, policy_version 1835925 (0.0009) [2023-12-27 04:46:10,705][105692] Updated weights for policy 0, policy_version 1835935 (0.0008) [2023-12-27 04:46:10,769][105692] Updated weights for policy 0, policy_version 1835945 (0.0008) [2023-12-27 04:46:10,943][105620] Updated weights for policy 1, policy_version 1839993 (0.0007) [2023-12-27 04:46:11,006][105620] Updated weights for policy 1, policy_version 1840003 (0.0009) [2023-12-27 04:46:11,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19387.8, 300 sec: 19660.8). Total num frames: 941178880. Throughput: 0: 9720.5, 1: 9560.8. Samples: 941190296. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:46:11,063][104569] Avg episode reward: [(0, '8451.539'), (1, '9072.968')] [2023-12-27 04:46:11,076][105620] Updated weights for policy 1, policy_version 1840013 (0.0008) [2023-12-27 04:46:11,139][105620] Updated weights for policy 1, policy_version 1840023 (0.0008) [2023-12-27 04:46:11,592][105692] Updated weights for policy 0, policy_version 1835955 (0.0009) [2023-12-27 04:46:11,660][105692] Updated weights for policy 0, policy_version 1835965 (0.0009) [2023-12-27 04:46:11,716][105692] Updated weights for policy 0, policy_version 1835975 (0.0009) [2023-12-27 04:46:11,903][105620] Updated weights for policy 1, policy_version 1840033 (0.0008) [2023-12-27 04:46:11,957][105620] Updated weights for policy 1, policy_version 1840043 (0.0008) [2023-12-27 04:46:12,020][105620] Updated weights for policy 1, policy_version 1840053 (0.0009) [2023-12-27 04:46:12,522][105692] Updated weights for policy 0, policy_version 1835985 (0.0009) [2023-12-27 04:46:12,592][105692] Updated weights for policy 0, policy_version 1835995 (0.0008) [2023-12-27 04:46:12,661][105692] Updated weights for policy 0, policy_version 1836005 (0.0008) [2023-12-27 04:46:12,724][105620] Updated weights for policy 1, policy_version 1840063 (0.0008) [2023-12-27 04:46:12,725][105692] Updated weights for policy 0, policy_version 1836015 (0.0008) [2023-12-27 04:46:12,789][105620] Updated weights for policy 1, policy_version 1840073 (0.0008) [2023-12-27 04:46:12,851][105620] Updated weights for policy 1, policy_version 1840083 (0.0009) [2023-12-27 04:46:13,441][105692] Updated weights for policy 0, policy_version 1836025 (0.0009) [2023-12-27 04:46:13,497][105692] Updated weights for policy 0, policy_version 1836036 (0.0009) [2023-12-27 04:46:13,565][105692] Updated weights for policy 0, policy_version 1836046 (0.0006) [2023-12-27 04:46:13,567][105620] Updated weights for policy 1, policy_version 1840093 (0.0008) [2023-12-27 04:46:13,617][105620] Updated weights for policy 1, policy_version 1840103 (0.0007) [2023-12-27 04:46:13,661][105620] Updated weights for policy 1, policy_version 1840113 (0.0008) [2023-12-27 04:46:14,318][105692] Updated weights for policy 0, policy_version 1836056 (0.0010) [2023-12-27 04:46:14,340][105620] Updated weights for policy 1, policy_version 1840123 (0.0009) [2023-12-27 04:46:14,370][105692] Updated weights for policy 0, policy_version 1836066 (0.0010) [2023-12-27 04:46:14,394][105620] Updated weights for policy 1, policy_version 1840133 (0.0010) [2023-12-27 04:46:14,431][105692] Updated weights for policy 0, policy_version 1836076 (0.0010) [2023-12-27 04:46:14,453][105620] Updated weights for policy 1, policy_version 1840143 (0.0010) [2023-12-27 04:46:15,155][105692] Updated weights for policy 0, policy_version 1836086 (0.0011) [2023-12-27 04:46:15,207][105620] Updated weights for policy 1, policy_version 1840153 (0.0010) [2023-12-27 04:46:15,218][105692] Updated weights for policy 0, policy_version 1836096 (0.0011) [2023-12-27 04:46:15,267][105620] Updated weights for policy 1, policy_version 1840163 (0.0009) [2023-12-27 04:46:15,278][105692] Updated weights for policy 0, policy_version 1836106 (0.0011) [2023-12-27 04:46:15,326][105620] Updated weights for policy 1, policy_version 1840173 (0.0009) [2023-12-27 04:46:15,386][105620] Updated weights for policy 1, policy_version 1840183 (0.0011) [2023-12-27 04:46:15,999][105692] Updated weights for policy 0, policy_version 1836116 (0.0010) [2023-12-27 04:46:16,058][105692] Updated weights for policy 0, policy_version 1836126 (0.0008) [2023-12-27 04:46:16,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19251.3, 300 sec: 19633.0). Total num frames: 941268992. Throughput: 0: 9692.4, 1: 9495.4. Samples: 941245176. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:46:16,063][104569] Avg episode reward: [(0, '8177.861'), (1, '9165.188')] [2023-12-27 04:46:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001840184_471154688.pth... [2023-12-27 04:46:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001839096_470876160.pth [2023-12-27 04:46:16,119][105692] Updated weights for policy 0, policy_version 1836136 (0.0007) [2023-12-27 04:46:16,130][105620] Updated weights for policy 1, policy_version 1840193 (0.0009) [2023-12-27 04:46:16,170][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001836144_470122496.pth... [2023-12-27 04:46:16,175][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001835024_469835776.pth [2023-12-27 04:46:16,179][105620] Updated weights for policy 1, policy_version 1840203 (0.0005) [2023-12-27 04:46:16,228][105620] Updated weights for policy 1, policy_version 1840213 (0.0010) [2023-12-27 04:46:16,750][105692] Updated weights for policy 0, policy_version 1836146 (0.0006) [2023-12-27 04:46:16,801][105692] Updated weights for policy 0, policy_version 1836156 (0.0006) [2023-12-27 04:46:16,847][105620] Updated weights for policy 1, policy_version 1840223 (0.0010) [2023-12-27 04:46:16,852][105692] Updated weights for policy 0, policy_version 1836166 (0.0010) [2023-12-27 04:46:16,895][105620] Updated weights for policy 1, policy_version 1840233 (0.0010) [2023-12-27 04:46:16,897][105692] Updated weights for policy 0, policy_version 1836176 (0.0008) [2023-12-27 04:46:16,954][105620] Updated weights for policy 1, policy_version 1840243 (0.0010) [2023-12-27 04:46:17,563][105692] Updated weights for policy 0, policy_version 1836186 (0.0010) [2023-12-27 04:46:17,635][105692] Updated weights for policy 0, policy_version 1836196 (0.0006) [2023-12-27 04:46:17,686][105620] Updated weights for policy 1, policy_version 1840253 (0.0009) [2023-12-27 04:46:17,695][105692] Updated weights for policy 0, policy_version 1836206 (0.0006) [2023-12-27 04:46:17,738][105620] Updated weights for policy 1, policy_version 1840263 (0.0010) [2023-12-27 04:46:17,799][105620] Updated weights for policy 1, policy_version 1840273 (0.0010) [2023-12-27 04:46:18,281][105692] Updated weights for policy 0, policy_version 1836216 (0.0008) [2023-12-27 04:46:18,333][105692] Updated weights for policy 0, policy_version 1836226 (0.0010) [2023-12-27 04:46:18,400][105692] Updated weights for policy 0, policy_version 1836236 (0.0011) [2023-12-27 04:46:18,590][105620] Updated weights for policy 1, policy_version 1840283 (0.0011) [2023-12-27 04:46:18,658][105620] Updated weights for policy 1, policy_version 1840293 (0.0011) [2023-12-27 04:46:18,720][105620] Updated weights for policy 1, policy_version 1840303 (0.0010) [2023-12-27 04:46:19,171][105692] Updated weights for policy 0, policy_version 1836246 (0.0009) [2023-12-27 04:46:19,232][105692] Updated weights for policy 0, policy_version 1836256 (0.0009) [2023-12-27 04:46:19,287][105692] Updated weights for policy 0, policy_version 1836266 (0.0008) [2023-12-27 04:46:19,496][105620] Updated weights for policy 1, policy_version 1840313 (0.0010) [2023-12-27 04:46:19,556][105620] Updated weights for policy 1, policy_version 1840323 (0.0011) [2023-12-27 04:46:19,616][105620] Updated weights for policy 1, policy_version 1840333 (0.0011) [2023-12-27 04:46:19,672][105620] Updated weights for policy 1, policy_version 1840343 (0.0010) [2023-12-27 04:46:20,072][105692] Updated weights for policy 0, policy_version 1836276 (0.0008) [2023-12-27 04:46:20,128][105692] Updated weights for policy 0, policy_version 1836286 (0.0008) [2023-12-27 04:46:20,178][105692] Updated weights for policy 0, policy_version 1836296 (0.0009) [2023-12-27 04:46:20,427][105620] Updated weights for policy 1, policy_version 1840353 (0.0010) [2023-12-27 04:46:20,486][105620] Updated weights for policy 1, policy_version 1840363 (0.0011) [2023-12-27 04:46:20,531][105620] Updated weights for policy 1, policy_version 1840373 (0.0010) [2023-12-27 04:46:20,952][105692] Updated weights for policy 0, policy_version 1836306 (0.0008) [2023-12-27 04:46:21,009][105692] Updated weights for policy 0, policy_version 1836316 (0.0008) [2023-12-27 04:46:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19633.0). Total num frames: 941367296. Throughput: 0: 9688.8, 1: 9485.6. Samples: 941362660. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:46:21,063][104569] Avg episode reward: [(0, '8536.423'), (1, '8982.456')] [2023-12-27 04:46:21,076][105692] Updated weights for policy 0, policy_version 1836326 (0.0009) [2023-12-27 04:46:21,140][105692] Updated weights for policy 0, policy_version 1836336 (0.0009) [2023-12-27 04:46:21,330][105620] Updated weights for policy 1, policy_version 1840383 (0.0010) [2023-12-27 04:46:21,403][105620] Updated weights for policy 1, policy_version 1840393 (0.0009) [2023-12-27 04:46:21,464][105620] Updated weights for policy 1, policy_version 1840403 (0.0008) [2023-12-27 04:46:21,938][105692] Updated weights for policy 0, policy_version 1836346 (0.0009) [2023-12-27 04:46:21,995][105692] Updated weights for policy 0, policy_version 1836356 (0.0009) [2023-12-27 04:46:22,052][105692] Updated weights for policy 0, policy_version 1836366 (0.0009) [2023-12-27 04:46:22,186][105620] Updated weights for policy 1, policy_version 1840413 (0.0007) [2023-12-27 04:46:22,253][105620] Updated weights for policy 1, policy_version 1840423 (0.0006) [2023-12-27 04:46:22,323][105620] Updated weights for policy 1, policy_version 1840433 (0.0006) [2023-12-27 04:46:22,775][105692] Updated weights for policy 0, policy_version 1836376 (0.0010) [2023-12-27 04:46:22,830][105692] Updated weights for policy 0, policy_version 1836386 (0.0008) [2023-12-27 04:46:22,889][105692] Updated weights for policy 0, policy_version 1836396 (0.0009) [2023-12-27 04:46:22,980][105620] Updated weights for policy 1, policy_version 1840443 (0.0009) [2023-12-27 04:46:23,043][105620] Updated weights for policy 1, policy_version 1840453 (0.0009) [2023-12-27 04:46:23,102][105620] Updated weights for policy 1, policy_version 1840463 (0.0010) [2023-12-27 04:46:23,572][105692] Updated weights for policy 0, policy_version 1836406 (0.0007) [2023-12-27 04:46:23,644][105692] Updated weights for policy 0, policy_version 1836416 (0.0006) [2023-12-27 04:46:23,713][105692] Updated weights for policy 0, policy_version 1836426 (0.0006) [2023-12-27 04:46:23,748][105620] Updated weights for policy 1, policy_version 1840473 (0.0009) [2023-12-27 04:46:23,793][105620] Updated weights for policy 1, policy_version 1840483 (0.0005) [2023-12-27 04:46:23,841][105620] Updated weights for policy 1, policy_version 1840493 (0.0005) [2023-12-27 04:46:23,891][105620] Updated weights for policy 1, policy_version 1840503 (0.0005) [2023-12-27 04:46:24,415][105692] Updated weights for policy 0, policy_version 1836436 (0.0009) [2023-12-27 04:46:24,471][105692] Updated weights for policy 0, policy_version 1836446 (0.0006) [2023-12-27 04:46:24,531][105692] Updated weights for policy 0, policy_version 1836456 (0.0011) [2023-12-27 04:46:24,590][105620] Updated weights for policy 1, policy_version 1840513 (0.0009) [2023-12-27 04:46:24,649][105620] Updated weights for policy 1, policy_version 1840523 (0.0008) [2023-12-27 04:46:24,693][105620] Updated weights for policy 1, policy_version 1840533 (0.0008) [2023-12-27 04:46:25,208][105692] Updated weights for policy 0, policy_version 1836466 (0.0009) [2023-12-27 04:46:25,276][105692] Updated weights for policy 0, policy_version 1836476 (0.0005) [2023-12-27 04:46:25,328][105692] Updated weights for policy 0, policy_version 1836486 (0.0005) [2023-12-27 04:46:25,379][105692] Updated weights for policy 0, policy_version 1836496 (0.0005) [2023-12-27 04:46:25,522][105620] Updated weights for policy 1, policy_version 1840543 (0.0009) [2023-12-27 04:46:25,583][105620] Updated weights for policy 1, policy_version 1840553 (0.0010) [2023-12-27 04:46:25,639][105620] Updated weights for policy 1, policy_version 1840563 (0.0008) [2023-12-27 04:46:25,920][105692] Updated weights for policy 0, policy_version 1836506 (0.0009) [2023-12-27 04:46:25,982][105692] Updated weights for policy 0, policy_version 1836516 (0.0009) [2023-12-27 04:46:26,029][105692] Updated weights for policy 0, policy_version 1836526 (0.0009) [2023-12-27 04:46:26,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 941473792. Throughput: 0: 9538.5, 1: 9477.6. Samples: 941477576. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:46:26,062][104569] Avg episode reward: [(0, '8895.298'), (1, '9075.495')] [2023-12-27 04:46:26,428][105620] Updated weights for policy 1, policy_version 1840573 (0.0009) [2023-12-27 04:46:26,486][105620] Updated weights for policy 1, policy_version 1840583 (0.0009) [2023-12-27 04:46:26,533][105620] Updated weights for policy 1, policy_version 1840593 (0.0009) [2023-12-27 04:46:26,725][105692] Updated weights for policy 0, policy_version 1836536 (0.0009) [2023-12-27 04:46:26,784][105692] Updated weights for policy 0, policy_version 1836546 (0.0008) [2023-12-27 04:46:26,841][105692] Updated weights for policy 0, policy_version 1836556 (0.0009) [2023-12-27 04:46:27,325][105620] Updated weights for policy 1, policy_version 1840603 (0.0008) [2023-12-27 04:46:27,375][105620] Updated weights for policy 1, policy_version 1840613 (0.0005) [2023-12-27 04:46:27,426][105620] Updated weights for policy 1, policy_version 1840623 (0.0005) [2023-12-27 04:46:27,576][105692] Updated weights for policy 0, policy_version 1836566 (0.0008) [2023-12-27 04:46:27,636][105692] Updated weights for policy 0, policy_version 1836576 (0.0008) [2023-12-27 04:46:27,690][105692] Updated weights for policy 0, policy_version 1836586 (0.0008) [2023-12-27 04:46:28,117][105620] Updated weights for policy 1, policy_version 1840633 (0.0007) [2023-12-27 04:46:28,176][105620] Updated weights for policy 1, policy_version 1840643 (0.0005) [2023-12-27 04:46:28,234][105620] Updated weights for policy 1, policy_version 1840653 (0.0005) [2023-12-27 04:46:28,284][105620] Updated weights for policy 1, policy_version 1840663 (0.0005) [2023-12-27 04:46:28,436][105692] Updated weights for policy 0, policy_version 1836596 (0.0007) [2023-12-27 04:46:28,492][105692] Updated weights for policy 0, policy_version 1836606 (0.0008) [2023-12-27 04:46:28,542][105692] Updated weights for policy 0, policy_version 1836616 (0.0009) [2023-12-27 04:46:28,895][105620] Updated weights for policy 1, policy_version 1840673 (0.0008) [2023-12-27 04:46:28,946][105620] Updated weights for policy 1, policy_version 1840683 (0.0009) [2023-12-27 04:46:29,003][105620] Updated weights for policy 1, policy_version 1840693 (0.0009) [2023-12-27 04:46:29,365][105692] Updated weights for policy 0, policy_version 1836627 (0.0009) [2023-12-27 04:46:29,427][105692] Updated weights for policy 0, policy_version 1836637 (0.0009) [2023-12-27 04:46:29,485][105692] Updated weights for policy 0, policy_version 1836647 (0.0010) [2023-12-27 04:46:29,691][105620] Updated weights for policy 1, policy_version 1840703 (0.0006) [2023-12-27 04:46:29,756][105620] Updated weights for policy 1, policy_version 1840713 (0.0005) [2023-12-27 04:46:29,825][105620] Updated weights for policy 1, policy_version 1840723 (0.0006) [2023-12-27 04:46:30,355][105692] Updated weights for policy 0, policy_version 1836658 (0.0009) [2023-12-27 04:46:30,407][105692] Updated weights for policy 0, policy_version 1836668 (0.0008) [2023-12-27 04:46:30,462][105692] Updated weights for policy 0, policy_version 1836678 (0.0008) [2023-12-27 04:46:30,479][105620] Updated weights for policy 1, policy_version 1840733 (0.0009) [2023-12-27 04:46:30,514][105692] Updated weights for policy 0, policy_version 1836688 (0.0007) [2023-12-27 04:46:30,538][105620] Updated weights for policy 1, policy_version 1840743 (0.0010) [2023-12-27 04:46:30,593][105620] Updated weights for policy 1, policy_version 1840753 (0.0010) [2023-12-27 04:46:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19633.0). Total num frames: 941563904. Throughput: 0: 9589.6, 1: 9481.7. Samples: 941536804. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:46:31,062][104569] Avg episode reward: [(0, '8807.638'), (1, '9075.170')] [2023-12-27 04:46:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001836688_470261760.pth... [2023-12-27 04:46:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001840760_471302144.pth... [2023-12-27 04:46:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001835600_469983232.pth [2023-12-27 04:46:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001839640_471015424.pth [2023-12-27 04:46:31,180][105620] Updated weights for policy 1, policy_version 1840763 (0.0010) [2023-12-27 04:46:31,228][105620] Updated weights for policy 1, policy_version 1840773 (0.0010) [2023-12-27 04:46:31,291][105620] Updated weights for policy 1, policy_version 1840783 (0.0007) [2023-12-27 04:46:31,377][105692] Updated weights for policy 0, policy_version 1836698 (0.0008) [2023-12-27 04:46:31,446][105692] Updated weights for policy 0, policy_version 1836708 (0.0009) [2023-12-27 04:46:31,512][105692] Updated weights for policy 0, policy_version 1836718 (0.0009) [2023-12-27 04:46:31,965][105620] Updated weights for policy 1, policy_version 1840793 (0.0009) [2023-12-27 04:46:32,017][105620] Updated weights for policy 1, policy_version 1840803 (0.0010) [2023-12-27 04:46:32,076][105620] Updated weights for policy 1, policy_version 1840813 (0.0011) [2023-12-27 04:46:32,143][105620] Updated weights for policy 1, policy_version 1840823 (0.0011) [2023-12-27 04:46:32,249][105692] Updated weights for policy 0, policy_version 1836728 (0.0006) [2023-12-27 04:46:32,314][105692] Updated weights for policy 0, policy_version 1836738 (0.0008) [2023-12-27 04:46:32,379][105692] Updated weights for policy 0, policy_version 1836748 (0.0008) [2023-12-27 04:46:32,866][105620] Updated weights for policy 1, policy_version 1840833 (0.0007) [2023-12-27 04:46:32,911][105620] Updated weights for policy 1, policy_version 1840843 (0.0005) [2023-12-27 04:46:32,963][105620] Updated weights for policy 1, policy_version 1840853 (0.0005) [2023-12-27 04:46:33,047][105692] Updated weights for policy 0, policy_version 1836758 (0.0009) [2023-12-27 04:46:33,105][105692] Updated weights for policy 0, policy_version 1836768 (0.0010) [2023-12-27 04:46:33,159][105692] Updated weights for policy 0, policy_version 1836778 (0.0010) [2023-12-27 04:46:33,540][105620] Updated weights for policy 1, policy_version 1840863 (0.0009) [2023-12-27 04:46:33,591][105620] Updated weights for policy 1, policy_version 1840873 (0.0010) [2023-12-27 04:46:33,639][105620] Updated weights for policy 1, policy_version 1840883 (0.0010) [2023-12-27 04:46:33,933][105692] Updated weights for policy 0, policy_version 1836788 (0.0009) [2023-12-27 04:46:33,995][105692] Updated weights for policy 0, policy_version 1836798 (0.0008) [2023-12-27 04:46:34,059][105692] Updated weights for policy 0, policy_version 1836808 (0.0009) [2023-12-27 04:46:34,414][105620] Updated weights for policy 1, policy_version 1840893 (0.0010) [2023-12-27 04:46:34,484][105620] Updated weights for policy 1, policy_version 1840903 (0.0011) [2023-12-27 04:46:34,548][105620] Updated weights for policy 1, policy_version 1840913 (0.0011) [2023-12-27 04:46:34,823][105692] Updated weights for policy 0, policy_version 1836818 (0.0008) [2023-12-27 04:46:34,886][105692] Updated weights for policy 0, policy_version 1836828 (0.0007) [2023-12-27 04:46:34,936][105692] Updated weights for policy 0, policy_version 1836838 (0.0007) [2023-12-27 04:46:34,988][105692] Updated weights for policy 0, policy_version 1836848 (0.0008) [2023-12-27 04:46:35,271][105620] Updated weights for policy 1, policy_version 1840923 (0.0011) [2023-12-27 04:46:35,325][105620] Updated weights for policy 1, policy_version 1840933 (0.0010) [2023-12-27 04:46:35,378][105620] Updated weights for policy 1, policy_version 1840943 (0.0009) [2023-12-27 04:46:35,574][105692] Updated weights for policy 0, policy_version 1836858 (0.0005) [2023-12-27 04:46:35,622][105692] Updated weights for policy 0, policy_version 1836868 (0.0005) [2023-12-27 04:46:35,669][105692] Updated weights for policy 0, policy_version 1836878 (0.0008) [2023-12-27 04:46:36,027][105620] Updated weights for policy 1, policy_version 1840953 (0.0005) [2023-12-27 04:46:36,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.6, 300 sec: 19605.3). Total num frames: 941662208. Throughput: 0: 9472.8, 1: 9559.2. Samples: 941652736. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:46:36,062][104569] Avg episode reward: [(0, '8721.060'), (1, '9074.365')] [2023-12-27 04:46:36,080][105620] Updated weights for policy 1, policy_version 1840963 (0.0005) [2023-12-27 04:46:36,143][105620] Updated weights for policy 1, policy_version 1840973 (0.0010) [2023-12-27 04:46:36,202][105620] Updated weights for policy 1, policy_version 1840983 (0.0011) [2023-12-27 04:46:36,391][105692] Updated weights for policy 0, policy_version 1836888 (0.0006) [2023-12-27 04:46:36,455][105692] Updated weights for policy 0, policy_version 1836898 (0.0007) [2023-12-27 04:46:36,517][105692] Updated weights for policy 0, policy_version 1836908 (0.0009) [2023-12-27 04:46:36,941][105620] Updated weights for policy 1, policy_version 1840993 (0.0011) [2023-12-27 04:46:36,994][105620] Updated weights for policy 1, policy_version 1841003 (0.0011) [2023-12-27 04:46:37,052][105620] Updated weights for policy 1, policy_version 1841013 (0.0010) [2023-12-27 04:46:37,219][105692] Updated weights for policy 0, policy_version 1836918 (0.0008) [2023-12-27 04:46:37,277][105692] Updated weights for policy 0, policy_version 1836928 (0.0007) [2023-12-27 04:46:37,342][105692] Updated weights for policy 0, policy_version 1836938 (0.0006) [2023-12-27 04:46:37,749][105620] Updated weights for policy 1, policy_version 1841023 (0.0009) [2023-12-27 04:46:37,808][105620] Updated weights for policy 1, policy_version 1841033 (0.0011) [2023-12-27 04:46:37,866][105620] Updated weights for policy 1, policy_version 1841043 (0.0010) [2023-12-27 04:46:38,105][105692] Updated weights for policy 0, policy_version 1836948 (0.0006) [2023-12-27 04:46:38,160][105692] Updated weights for policy 0, policy_version 1836958 (0.0006) [2023-12-27 04:46:38,209][105692] Updated weights for policy 0, policy_version 1836968 (0.0007) [2023-12-27 04:46:38,531][105620] Updated weights for policy 1, policy_version 1841053 (0.0008) [2023-12-27 04:46:38,591][105620] Updated weights for policy 1, policy_version 1841063 (0.0008) [2023-12-27 04:46:38,646][105620] Updated weights for policy 1, policy_version 1841073 (0.0011) [2023-12-27 04:46:39,013][105692] Updated weights for policy 0, policy_version 1836978 (0.0009) [2023-12-27 04:46:39,077][105692] Updated weights for policy 0, policy_version 1836988 (0.0009) [2023-12-27 04:46:39,138][105692] Updated weights for policy 0, policy_version 1836998 (0.0008) [2023-12-27 04:46:39,204][105692] Updated weights for policy 0, policy_version 1837008 (0.0008) [2023-12-27 04:46:39,343][105620] Updated weights for policy 1, policy_version 1841083 (0.0010) [2023-12-27 04:46:39,413][105620] Updated weights for policy 1, policy_version 1841093 (0.0014) [2023-12-27 04:46:39,470][105620] Updated weights for policy 1, policy_version 1841103 (0.0010) [2023-12-27 04:46:39,985][105692] Updated weights for policy 0, policy_version 1837018 (0.0008) [2023-12-27 04:46:40,049][105692] Updated weights for policy 0, policy_version 1837028 (0.0008) [2023-12-27 04:46:40,120][105692] Updated weights for policy 0, policy_version 1837038 (0.0008) [2023-12-27 04:46:40,252][105620] Updated weights for policy 1, policy_version 1841113 (0.0011) [2023-12-27 04:46:40,309][105620] Updated weights for policy 1, policy_version 1841123 (0.0011) [2023-12-27 04:46:40,369][105620] Updated weights for policy 1, policy_version 1841133 (0.0011) [2023-12-27 04:46:40,429][105620] Updated weights for policy 1, policy_version 1841143 (0.0011) [2023-12-27 04:46:40,873][105692] Updated weights for policy 0, policy_version 1837048 (0.0008) [2023-12-27 04:46:40,936][105692] Updated weights for policy 0, policy_version 1837058 (0.0008) [2023-12-27 04:46:41,003][105692] Updated weights for policy 0, policy_version 1837068 (0.0008) [2023-12-27 04:46:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19114.7, 300 sec: 19549.7). Total num frames: 941760512. Throughput: 0: 9452.6, 1: 9587.9. Samples: 941768396. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:46:41,062][104569] Avg episode reward: [(0, '8811.650'), (1, '9257.164')] [2023-12-27 04:46:41,223][105620] Updated weights for policy 1, policy_version 1841153 (0.0011) [2023-12-27 04:46:41,289][105620] Updated weights for policy 1, policy_version 1841163 (0.0011) [2023-12-27 04:46:41,361][105620] Updated weights for policy 1, policy_version 1841173 (0.0011) [2023-12-27 04:46:41,841][105692] Updated weights for policy 0, policy_version 1837078 (0.0010) [2023-12-27 04:46:41,896][105692] Updated weights for policy 0, policy_version 1837088 (0.0009) [2023-12-27 04:46:41,950][105692] Updated weights for policy 0, policy_version 1837098 (0.0009) [2023-12-27 04:46:42,054][105620] Updated weights for policy 1, policy_version 1841183 (0.0009) [2023-12-27 04:46:42,108][105620] Updated weights for policy 1, policy_version 1841193 (0.0009) [2023-12-27 04:46:42,160][105620] Updated weights for policy 1, policy_version 1841203 (0.0006) [2023-12-27 04:46:42,758][105692] Updated weights for policy 0, policy_version 1837108 (0.0009) [2023-12-27 04:46:42,824][105692] Updated weights for policy 0, policy_version 1837118 (0.0010) [2023-12-27 04:46:42,880][105692] Updated weights for policy 0, policy_version 1837128 (0.0009) [2023-12-27 04:46:42,911][105620] Updated weights for policy 1, policy_version 1841213 (0.0006) [2023-12-27 04:46:42,972][105620] Updated weights for policy 1, policy_version 1841223 (0.0005) [2023-12-27 04:46:43,025][105620] Updated weights for policy 1, policy_version 1841233 (0.0005) [2023-12-27 04:46:43,671][105620] Updated weights for policy 1, policy_version 1841243 (0.0009) [2023-12-27 04:46:43,697][105692] Updated weights for policy 0, policy_version 1837138 (0.0009) [2023-12-27 04:46:43,729][105620] Updated weights for policy 1, policy_version 1841253 (0.0006) [2023-12-27 04:46:43,755][105692] Updated weights for policy 0, policy_version 1837148 (0.0008) [2023-12-27 04:46:43,785][105620] Updated weights for policy 1, policy_version 1841263 (0.0005) [2023-12-27 04:46:43,804][105692] Updated weights for policy 0, policy_version 1837158 (0.0009) [2023-12-27 04:46:43,860][105692] Updated weights for policy 0, policy_version 1837168 (0.0008) [2023-12-27 04:46:44,440][105620] Updated weights for policy 1, policy_version 1841273 (0.0006) [2023-12-27 04:46:44,497][105620] Updated weights for policy 1, policy_version 1841283 (0.0008) [2023-12-27 04:46:44,554][105620] Updated weights for policy 1, policy_version 1841293 (0.0009) [2023-12-27 04:46:44,606][105620] Updated weights for policy 1, policy_version 1841303 (0.0009) [2023-12-27 04:46:44,662][105692] Updated weights for policy 0, policy_version 1837178 (0.0008) [2023-12-27 04:46:44,724][105692] Updated weights for policy 0, policy_version 1837188 (0.0009) [2023-12-27 04:46:44,784][105692] Updated weights for policy 0, policy_version 1837198 (0.0009) [2023-12-27 04:46:45,346][105620] Updated weights for policy 1, policy_version 1841313 (0.0010) [2023-12-27 04:46:45,408][105620] Updated weights for policy 1, policy_version 1841323 (0.0010) [2023-12-27 04:46:45,464][105620] Updated weights for policy 1, policy_version 1841333 (0.0010) [2023-12-27 04:46:45,574][105692] Updated weights for policy 0, policy_version 1837208 (0.0009) [2023-12-27 04:46:45,638][105692] Updated weights for policy 0, policy_version 1837218 (0.0008) [2023-12-27 04:46:45,697][105692] Updated weights for policy 0, policy_version 1837228 (0.0008) [2023-12-27 04:46:46,062][104569] Fps is (10 sec: 18841.0, 60 sec: 18978.0, 300 sec: 19521.9). Total num frames: 941850624. Throughput: 0: 9332.2, 1: 9645.1. Samples: 941823736. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:46:46,063][104569] Avg episode reward: [(0, '8902.926'), (1, '9349.409')] [2023-12-27 04:46:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001837232_470401024.pth... [2023-12-27 04:46:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001841336_471449600.pth... [2023-12-27 04:46:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001836144_470122496.pth [2023-12-27 04:46:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001840184_471154688.pth [2023-12-27 04:46:46,199][105620] Updated weights for policy 1, policy_version 1841343 (0.0010) [2023-12-27 04:46:46,258][105620] Updated weights for policy 1, policy_version 1841353 (0.0011) [2023-12-27 04:46:46,317][105620] Updated weights for policy 1, policy_version 1841363 (0.0011) [2023-12-27 04:46:46,434][105692] Updated weights for policy 0, policy_version 1837238 (0.0008) [2023-12-27 04:46:46,490][105692] Updated weights for policy 0, policy_version 1837248 (0.0008) [2023-12-27 04:46:46,546][105692] Updated weights for policy 0, policy_version 1837258 (0.0008) [2023-12-27 04:46:47,064][105620] Updated weights for policy 1, policy_version 1841373 (0.0010) [2023-12-27 04:46:47,125][105620] Updated weights for policy 1, policy_version 1841383 (0.0010) [2023-12-27 04:46:47,138][105692] Updated weights for policy 0, policy_version 1837268 (0.0006) [2023-12-27 04:46:47,180][105620] Updated weights for policy 1, policy_version 1841393 (0.0010) [2023-12-27 04:46:47,194][105692] Updated weights for policy 0, policy_version 1837278 (0.0007) [2023-12-27 04:46:47,241][105692] Updated weights for policy 0, policy_version 1837288 (0.0007) [2023-12-27 04:46:47,940][105620] Updated weights for policy 1, policy_version 1841403 (0.0010) [2023-12-27 04:46:47,955][105692] Updated weights for policy 0, policy_version 1837298 (0.0008) [2023-12-27 04:46:48,002][105620] Updated weights for policy 1, policy_version 1841413 (0.0010) [2023-12-27 04:46:48,013][105692] Updated weights for policy 0, policy_version 1837308 (0.0005) [2023-12-27 04:46:48,065][105620] Updated weights for policy 1, policy_version 1841423 (0.0010) [2023-12-27 04:46:48,068][105692] Updated weights for policy 0, policy_version 1837318 (0.0005) [2023-12-27 04:46:48,118][105692] Updated weights for policy 0, policy_version 1837328 (0.0005) [2023-12-27 04:46:48,739][105620] Updated weights for policy 1, policy_version 1841433 (0.0010) [2023-12-27 04:46:48,764][105692] Updated weights for policy 0, policy_version 1837338 (0.0009) [2023-12-27 04:46:48,805][105620] Updated weights for policy 1, policy_version 1841443 (0.0010) [2023-12-27 04:46:48,834][105692] Updated weights for policy 0, policy_version 1837348 (0.0007) [2023-12-27 04:46:48,868][105620] Updated weights for policy 1, policy_version 1841453 (0.0008) [2023-12-27 04:46:48,883][105692] Updated weights for policy 0, policy_version 1837358 (0.0006) [2023-12-27 04:46:48,927][105620] Updated weights for policy 1, policy_version 1841463 (0.0009) [2023-12-27 04:46:49,635][105692] Updated weights for policy 0, policy_version 1837368 (0.0009) [2023-12-27 04:46:49,695][105692] Updated weights for policy 0, policy_version 1837378 (0.0007) [2023-12-27 04:46:49,697][105620] Updated weights for policy 1, policy_version 1841473 (0.0008) [2023-12-27 04:46:49,748][105620] Updated weights for policy 1, policy_version 1841483 (0.0006) [2023-12-27 04:46:49,750][105692] Updated weights for policy 0, policy_version 1837388 (0.0008) [2023-12-27 04:46:49,796][105620] Updated weights for policy 1, policy_version 1841493 (0.0007) [2023-12-27 04:46:50,466][105692] Updated weights for policy 0, policy_version 1837398 (0.0007) [2023-12-27 04:46:50,529][105692] Updated weights for policy 0, policy_version 1837408 (0.0008) [2023-12-27 04:46:50,592][105692] Updated weights for policy 0, policy_version 1837418 (0.0009) [2023-12-27 04:46:50,636][105620] Updated weights for policy 1, policy_version 1841503 (0.0008) [2023-12-27 04:46:50,695][105620] Updated weights for policy 1, policy_version 1841513 (0.0009) [2023-12-27 04:46:50,761][105620] Updated weights for policy 1, policy_version 1841523 (0.0009) [2023-12-27 04:46:51,062][104569] Fps is (10 sec: 18841.3, 60 sec: 18978.1, 300 sec: 19549.7). Total num frames: 941948928. Throughput: 0: 9412.9, 1: 9659.6. Samples: 941938856. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:46:51,063][104569] Avg episode reward: [(0, '8536.895'), (1, '9257.216')] [2023-12-27 04:46:51,226][105692] Updated weights for policy 0, policy_version 1837428 (0.0006) [2023-12-27 04:46:51,288][105692] Updated weights for policy 0, policy_version 1837438 (0.0006) [2023-12-27 04:46:51,350][105692] Updated weights for policy 0, policy_version 1837448 (0.0009) [2023-12-27 04:46:51,436][105620] Updated weights for policy 1, policy_version 1841533 (0.0009) [2023-12-27 04:46:51,482][105620] Updated weights for policy 1, policy_version 1841543 (0.0008) [2023-12-27 04:46:51,539][105620] Updated weights for policy 1, policy_version 1841553 (0.0008) [2023-12-27 04:46:52,074][105692] Updated weights for policy 0, policy_version 1837458 (0.0011) [2023-12-27 04:46:52,143][105692] Updated weights for policy 0, policy_version 1837468 (0.0007) [2023-12-27 04:46:52,212][105692] Updated weights for policy 0, policy_version 1837478 (0.0006) [2023-12-27 04:46:52,273][105692] Updated weights for policy 0, policy_version 1837488 (0.0006) [2023-12-27 04:46:52,296][105620] Updated weights for policy 1, policy_version 1841563 (0.0008) [2023-12-27 04:46:52,358][105620] Updated weights for policy 1, policy_version 1841573 (0.0009) [2023-12-27 04:46:52,413][105620] Updated weights for policy 1, policy_version 1841583 (0.0008) [2023-12-27 04:46:52,837][105692] Updated weights for policy 0, policy_version 1837498 (0.0008) [2023-12-27 04:46:52,893][105692] Updated weights for policy 0, policy_version 1837508 (0.0010) [2023-12-27 04:46:52,947][105692] Updated weights for policy 0, policy_version 1837519 (0.0010) [2023-12-27 04:46:53,168][105620] Updated weights for policy 1, policy_version 1841593 (0.0009) [2023-12-27 04:46:53,224][105620] Updated weights for policy 1, policy_version 1841603 (0.0008) [2023-12-27 04:46:53,282][105620] Updated weights for policy 1, policy_version 1841613 (0.0009) [2023-12-27 04:46:53,339][105620] Updated weights for policy 1, policy_version 1841623 (0.0006) [2023-12-27 04:46:53,784][105692] Updated weights for policy 0, policy_version 1837529 (0.0008) [2023-12-27 04:46:53,830][105692] Updated weights for policy 0, policy_version 1837539 (0.0008) [2023-12-27 04:46:53,884][105692] Updated weights for policy 0, policy_version 1837549 (0.0009) [2023-12-27 04:46:53,993][105620] Updated weights for policy 1, policy_version 1841633 (0.0008) [2023-12-27 04:46:54,039][105620] Updated weights for policy 1, policy_version 1841643 (0.0008) [2023-12-27 04:46:54,085][105620] Updated weights for policy 1, policy_version 1841653 (0.0008) [2023-12-27 04:46:54,622][105692] Updated weights for policy 0, policy_version 1837559 (0.0009) [2023-12-27 04:46:54,670][105692] Updated weights for policy 0, policy_version 1837569 (0.0009) [2023-12-27 04:46:54,716][105692] Updated weights for policy 0, policy_version 1837579 (0.0008) [2023-12-27 04:46:54,866][105620] Updated weights for policy 1, policy_version 1841663 (0.0009) [2023-12-27 04:46:54,912][105620] Updated weights for policy 1, policy_version 1841673 (0.0008) [2023-12-27 04:46:54,967][105620] Updated weights for policy 1, policy_version 1841683 (0.0008) [2023-12-27 04:46:55,467][105692] Updated weights for policy 0, policy_version 1837589 (0.0009) [2023-12-27 04:46:55,525][105692] Updated weights for policy 0, policy_version 1837599 (0.0009) [2023-12-27 04:46:55,584][105692] Updated weights for policy 0, policy_version 1837609 (0.0008) [2023-12-27 04:46:55,751][105620] Updated weights for policy 1, policy_version 1841693 (0.0009) [2023-12-27 04:46:55,798][105620] Updated weights for policy 1, policy_version 1841703 (0.0009) [2023-12-27 04:46:55,853][105620] Updated weights for policy 1, policy_version 1841713 (0.0009) [2023-12-27 04:46:56,062][104569] Fps is (10 sec: 19661.5, 60 sec: 18978.1, 300 sec: 19549.7). Total num frames: 942047232. Throughput: 0: 9492.4, 1: 9698.1. Samples: 942053864. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:46:56,062][104569] Avg episode reward: [(0, '8260.895'), (1, '9073.208')] [2023-12-27 04:46:56,342][105692] Updated weights for policy 0, policy_version 1837619 (0.0009) [2023-12-27 04:46:56,394][105692] Updated weights for policy 0, policy_version 1837629 (0.0008) [2023-12-27 04:46:56,446][105692] Updated weights for policy 0, policy_version 1837639 (0.0008) [2023-12-27 04:46:56,640][105620] Updated weights for policy 1, policy_version 1841723 (0.0010) [2023-12-27 04:46:56,688][105620] Updated weights for policy 1, policy_version 1841733 (0.0010) [2023-12-27 04:46:56,735][105620] Updated weights for policy 1, policy_version 1841743 (0.0010) [2023-12-27 04:46:57,105][105692] Updated weights for policy 0, policy_version 1837649 (0.0008) [2023-12-27 04:46:57,172][105692] Updated weights for policy 0, policy_version 1837659 (0.0005) [2023-12-27 04:46:57,230][105692] Updated weights for policy 0, policy_version 1837669 (0.0005) [2023-12-27 04:46:57,291][105692] Updated weights for policy 0, policy_version 1837679 (0.0005) [2023-12-27 04:46:57,482][105620] Updated weights for policy 1, policy_version 1841753 (0.0010) [2023-12-27 04:46:57,541][105620] Updated weights for policy 1, policy_version 1841763 (0.0009) [2023-12-27 04:46:57,594][105620] Updated weights for policy 1, policy_version 1841774 (0.0009) [2023-12-27 04:46:57,642][105620] Updated weights for policy 1, policy_version 1841784 (0.0009) [2023-12-27 04:46:57,789][105692] Updated weights for policy 0, policy_version 1837689 (0.0009) [2023-12-27 04:46:57,839][105692] Updated weights for policy 0, policy_version 1837699 (0.0009) [2023-12-27 04:46:57,892][105692] Updated weights for policy 0, policy_version 1837709 (0.0008) [2023-12-27 04:46:58,357][105620] Updated weights for policy 1, policy_version 1841794 (0.0009) [2023-12-27 04:46:58,413][105620] Updated weights for policy 1, policy_version 1841804 (0.0006) [2023-12-27 04:46:58,476][105620] Updated weights for policy 1, policy_version 1841814 (0.0009) [2023-12-27 04:46:58,756][105692] Updated weights for policy 0, policy_version 1837719 (0.0008) [2023-12-27 04:46:58,823][105692] Updated weights for policy 0, policy_version 1837729 (0.0010) [2023-12-27 04:46:58,901][105692] Updated weights for policy 0, policy_version 1837739 (0.0009) [2023-12-27 04:46:59,224][105620] Updated weights for policy 1, policy_version 1841824 (0.0009) [2023-12-27 04:46:59,289][105620] Updated weights for policy 1, policy_version 1841834 (0.0009) [2023-12-27 04:46:59,354][105620] Updated weights for policy 1, policy_version 1841844 (0.0008) [2023-12-27 04:46:59,739][105692] Updated weights for policy 0, policy_version 1837749 (0.0009) [2023-12-27 04:46:59,787][105692] Updated weights for policy 0, policy_version 1837759 (0.0009) [2023-12-27 04:46:59,845][105692] Updated weights for policy 0, policy_version 1837769 (0.0009) [2023-12-27 04:47:00,102][105620] Updated weights for policy 1, policy_version 1841854 (0.0009) [2023-12-27 04:47:00,149][105620] Updated weights for policy 1, policy_version 1841864 (0.0008) [2023-12-27 04:47:00,199][105620] Updated weights for policy 1, policy_version 1841874 (0.0009) [2023-12-27 04:47:00,599][105692] Updated weights for policy 0, policy_version 1837779 (0.0009) [2023-12-27 04:47:00,661][105692] Updated weights for policy 0, policy_version 1837789 (0.0010) [2023-12-27 04:47:00,717][105692] Updated weights for policy 0, policy_version 1837799 (0.0006) [2023-12-27 04:47:00,941][105620] Updated weights for policy 1, policy_version 1841884 (0.0007) [2023-12-27 04:47:00,998][105620] Updated weights for policy 1, policy_version 1841894 (0.0005) [2023-12-27 04:47:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 18978.1, 300 sec: 19494.2). Total num frames: 942137344. Throughput: 0: 9582.9, 1: 9695.3. Samples: 942112696. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:47:01,063][104569] Avg episode reward: [(0, '8716.669'), (1, '9074.631')] [2023-12-27 04:47:01,065][105620] Updated weights for policy 1, policy_version 1841904 (0.0008) [2023-12-27 04:47:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001837808_470548480.pth... [2023-12-27 04:47:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001836688_470261760.pth [2023-12-27 04:47:01,125][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001841912_471597056.pth... [2023-12-27 04:47:01,130][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001840760_471302144.pth [2023-12-27 04:47:01,472][105692] Updated weights for policy 0, policy_version 1837809 (0.0009) [2023-12-27 04:47:01,538][105692] Updated weights for policy 0, policy_version 1837819 (0.0008) [2023-12-27 04:47:01,603][105692] Updated weights for policy 0, policy_version 1837829 (0.0005) [2023-12-27 04:47:01,672][105692] Updated weights for policy 0, policy_version 1837839 (0.0008) [2023-12-27 04:47:01,875][105620] Updated weights for policy 1, policy_version 1841914 (0.0008) [2023-12-27 04:47:01,929][105620] Updated weights for policy 1, policy_version 1841924 (0.0010) [2023-12-27 04:47:01,981][105620] Updated weights for policy 1, policy_version 1841934 (0.0009) [2023-12-27 04:47:02,038][105620] Updated weights for policy 1, policy_version 1841944 (0.0009) [2023-12-27 04:47:02,278][105692] Updated weights for policy 0, policy_version 1837849 (0.0009) [2023-12-27 04:47:02,330][105692] Updated weights for policy 0, policy_version 1837859 (0.0009) [2023-12-27 04:47:02,389][105692] Updated weights for policy 0, policy_version 1837869 (0.0009) [2023-12-27 04:47:02,794][105620] Updated weights for policy 1, policy_version 1841954 (0.0009) [2023-12-27 04:47:02,858][105620] Updated weights for policy 1, policy_version 1841964 (0.0009) [2023-12-27 04:47:02,912][105620] Updated weights for policy 1, policy_version 1841974 (0.0009) [2023-12-27 04:47:03,153][105692] Updated weights for policy 0, policy_version 1837879 (0.0009) [2023-12-27 04:47:03,212][105692] Updated weights for policy 0, policy_version 1837889 (0.0009) [2023-12-27 04:47:03,270][105692] Updated weights for policy 0, policy_version 1837899 (0.0009) [2023-12-27 04:47:03,668][105620] Updated weights for policy 1, policy_version 1841984 (0.0008) [2023-12-27 04:47:03,718][105620] Updated weights for policy 1, policy_version 1841994 (0.0009) [2023-12-27 04:47:03,771][105620] Updated weights for policy 1, policy_version 1842004 (0.0008) [2023-12-27 04:47:04,017][105692] Updated weights for policy 0, policy_version 1837909 (0.0009) [2023-12-27 04:47:04,075][105692] Updated weights for policy 0, policy_version 1837919 (0.0008) [2023-12-27 04:47:04,138][105692] Updated weights for policy 0, policy_version 1837929 (0.0009) [2023-12-27 04:47:04,549][105620] Updated weights for policy 1, policy_version 1842014 (0.0009) [2023-12-27 04:47:04,608][105620] Updated weights for policy 1, policy_version 1842024 (0.0008) [2023-12-27 04:47:04,661][105620] Updated weights for policy 1, policy_version 1842034 (0.0009) [2023-12-27 04:47:04,920][105692] Updated weights for policy 0, policy_version 1837939 (0.0008) [2023-12-27 04:47:04,978][105692] Updated weights for policy 0, policy_version 1837949 (0.0009) [2023-12-27 04:47:05,028][105692] Updated weights for policy 0, policy_version 1837959 (0.0007) [2023-12-27 04:47:05,432][105620] Updated weights for policy 1, policy_version 1842044 (0.0008) [2023-12-27 04:47:05,482][105620] Updated weights for policy 1, policy_version 1842054 (0.0008) [2023-12-27 04:47:05,544][105620] Updated weights for policy 1, policy_version 1842064 (0.0009) [2023-12-27 04:47:05,756][105692] Updated weights for policy 0, policy_version 1837969 (0.0006) [2023-12-27 04:47:05,805][105692] Updated weights for policy 0, policy_version 1837979 (0.0005) [2023-12-27 04:47:05,860][105692] Updated weights for policy 0, policy_version 1837989 (0.0005) [2023-12-27 04:47:05,918][105692] Updated weights for policy 0, policy_version 1837999 (0.0005) [2023-12-27 04:47:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19522.0). Total num frames: 942235648. Throughput: 0: 9485.1, 1: 9649.2. Samples: 942223700. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:47:06,062][104569] Avg episode reward: [(0, '8987.813'), (1, '9258.584')] [2023-12-27 04:47:06,406][105620] Updated weights for policy 1, policy_version 1842074 (0.0009) [2023-12-27 04:47:06,466][105620] Updated weights for policy 1, policy_version 1842084 (0.0009) [2023-12-27 04:47:06,489][105692] Updated weights for policy 0, policy_version 1838009 (0.0006) [2023-12-27 04:47:06,526][105620] Updated weights for policy 1, policy_version 1842094 (0.0008) [2023-12-27 04:47:06,542][105692] Updated weights for policy 0, policy_version 1838019 (0.0008) [2023-12-27 04:47:06,586][105620] Updated weights for policy 1, policy_version 1842104 (0.0008) [2023-12-27 04:47:06,598][105692] Updated weights for policy 0, policy_version 1838029 (0.0008) [2023-12-27 04:47:07,298][105692] Updated weights for policy 0, policy_version 1838039 (0.0008) [2023-12-27 04:47:07,353][105692] Updated weights for policy 0, policy_version 1838049 (0.0008) [2023-12-27 04:47:07,373][105620] Updated weights for policy 1, policy_version 1842114 (0.0007) [2023-12-27 04:47:07,397][105692] Updated weights for policy 0, policy_version 1838059 (0.0008) [2023-12-27 04:47:07,430][105620] Updated weights for policy 1, policy_version 1842124 (0.0007) [2023-12-27 04:47:07,496][105620] Updated weights for policy 1, policy_version 1842134 (0.0010) [2023-12-27 04:47:08,122][105692] Updated weights for policy 0, policy_version 1838069 (0.0010) [2023-12-27 04:47:08,184][105692] Updated weights for policy 0, policy_version 1838079 (0.0009) [2023-12-27 04:47:08,212][105620] Updated weights for policy 1, policy_version 1842144 (0.0007) [2023-12-27 04:47:08,247][105692] Updated weights for policy 0, policy_version 1838089 (0.0009) [2023-12-27 04:47:08,267][105620] Updated weights for policy 1, policy_version 1842154 (0.0009) [2023-12-27 04:47:08,324][105620] Updated weights for policy 1, policy_version 1842164 (0.0008) [2023-12-27 04:47:09,019][105692] Updated weights for policy 0, policy_version 1838099 (0.0010) [2023-12-27 04:47:09,053][105620] Updated weights for policy 1, policy_version 1842174 (0.0008) [2023-12-27 04:47:09,074][105692] Updated weights for policy 0, policy_version 1838109 (0.0008) [2023-12-27 04:47:09,113][105620] Updated weights for policy 1, policy_version 1842184 (0.0008) [2023-12-27 04:47:09,139][105692] Updated weights for policy 0, policy_version 1838119 (0.0006) [2023-12-27 04:47:09,163][105620] Updated weights for policy 1, policy_version 1842194 (0.0008) [2023-12-27 04:47:09,922][105692] Updated weights for policy 0, policy_version 1838129 (0.0007) [2023-12-27 04:47:09,957][105620] Updated weights for policy 1, policy_version 1842204 (0.0007) [2023-12-27 04:47:09,984][105692] Updated weights for policy 0, policy_version 1838139 (0.0006) [2023-12-27 04:47:10,019][105620] Updated weights for policy 1, policy_version 1842214 (0.0008) [2023-12-27 04:47:10,046][105692] Updated weights for policy 0, policy_version 1838149 (0.0006) [2023-12-27 04:47:10,081][105620] Updated weights for policy 1, policy_version 1842224 (0.0008) [2023-12-27 04:47:10,104][105692] Updated weights for policy 0, policy_version 1838159 (0.0006) [2023-12-27 04:47:10,779][105620] Updated weights for policy 1, policy_version 1842234 (0.0007) [2023-12-27 04:47:10,837][105620] Updated weights for policy 1, policy_version 1842244 (0.0006) [2023-12-27 04:47:10,871][105692] Updated weights for policy 0, policy_version 1838169 (0.0005) [2023-12-27 04:47:10,890][105620] Updated weights for policy 1, policy_version 1842254 (0.0005) [2023-12-27 04:47:10,929][105692] Updated weights for policy 0, policy_version 1838179 (0.0007) [2023-12-27 04:47:10,948][105620] Updated weights for policy 1, policy_version 1842264 (0.0006) [2023-12-27 04:47:10,986][105692] Updated weights for policy 0, policy_version 1838189 (0.0008) [2023-12-27 04:47:11,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 942333952. Throughput: 0: 9485.8, 1: 9618.0. Samples: 942337248. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:47:11,062][104569] Avg episode reward: [(0, '8530.006'), (1, '9349.215')] [2023-12-27 04:47:11,686][105620] Updated weights for policy 1, policy_version 1842274 (0.0010) [2023-12-27 04:47:11,758][105620] Updated weights for policy 1, policy_version 1842284 (0.0008) [2023-12-27 04:47:11,769][105692] Updated weights for policy 0, policy_version 1838199 (0.0008) [2023-12-27 04:47:11,816][105620] Updated weights for policy 1, policy_version 1842294 (0.0007) [2023-12-27 04:47:11,836][105692] Updated weights for policy 0, policy_version 1838209 (0.0007) [2023-12-27 04:47:11,900][105692] Updated weights for policy 0, policy_version 1838219 (0.0009) [2023-12-27 04:47:12,502][105620] Updated weights for policy 1, policy_version 1842304 (0.0006) [2023-12-27 04:47:12,555][105620] Updated weights for policy 1, policy_version 1842314 (0.0011) [2023-12-27 04:47:12,608][105620] Updated weights for policy 1, policy_version 1842324 (0.0009) [2023-12-27 04:47:12,718][105692] Updated weights for policy 0, policy_version 1838229 (0.0009) [2023-12-27 04:47:12,774][105692] Updated weights for policy 0, policy_version 1838239 (0.0008) [2023-12-27 04:47:12,838][105692] Updated weights for policy 0, policy_version 1838249 (0.0008) [2023-12-27 04:47:13,347][105620] Updated weights for policy 1, policy_version 1842334 (0.0011) [2023-12-27 04:47:13,404][105620] Updated weights for policy 1, policy_version 1842344 (0.0010) [2023-12-27 04:47:13,459][105620] Updated weights for policy 1, policy_version 1842354 (0.0010) [2023-12-27 04:47:13,595][105692] Updated weights for policy 0, policy_version 1838259 (0.0008) [2023-12-27 04:47:13,643][105692] Updated weights for policy 0, policy_version 1838269 (0.0008) [2023-12-27 04:47:13,695][105692] Updated weights for policy 0, policy_version 1838279 (0.0008) [2023-12-27 04:47:14,224][105620] Updated weights for policy 1, policy_version 1842364 (0.0010) [2023-12-27 04:47:14,272][105620] Updated weights for policy 1, policy_version 1842374 (0.0010) [2023-12-27 04:47:14,323][105620] Updated weights for policy 1, policy_version 1842384 (0.0010) [2023-12-27 04:47:14,420][105692] Updated weights for policy 0, policy_version 1838289 (0.0009) [2023-12-27 04:47:14,473][105692] Updated weights for policy 0, policy_version 1838299 (0.0005) [2023-12-27 04:47:14,528][105692] Updated weights for policy 0, policy_version 1838309 (0.0006) [2023-12-27 04:47:14,577][105692] Updated weights for policy 0, policy_version 1838319 (0.0005) [2023-12-27 04:47:14,989][105620] Updated weights for policy 1, policy_version 1842394 (0.0010) [2023-12-27 04:47:15,052][105620] Updated weights for policy 1, policy_version 1842404 (0.0011) [2023-12-27 04:47:15,109][105620] Updated weights for policy 1, policy_version 1842414 (0.0011) [2023-12-27 04:47:15,178][105620] Updated weights for policy 1, policy_version 1842424 (0.0011) [2023-12-27 04:47:15,215][105692] Updated weights for policy 0, policy_version 1838329 (0.0006) [2023-12-27 04:47:15,283][105692] Updated weights for policy 0, policy_version 1838339 (0.0006) [2023-12-27 04:47:15,352][105692] Updated weights for policy 0, policy_version 1838349 (0.0006) [2023-12-27 04:47:15,929][105620] Updated weights for policy 1, policy_version 1842434 (0.0011) [2023-12-27 04:47:15,939][105692] Updated weights for policy 0, policy_version 1838359 (0.0006) [2023-12-27 04:47:15,977][105620] Updated weights for policy 1, policy_version 1842444 (0.0010) [2023-12-27 04:47:16,001][105692] Updated weights for policy 0, policy_version 1838369 (0.0005) [2023-12-27 04:47:16,032][105620] Updated weights for policy 1, policy_version 1842454 (0.0010) [2023-12-27 04:47:16,049][105692] Updated weights for policy 0, policy_version 1838379 (0.0005) [2023-12-27 04:47:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 942424064. Throughput: 0: 9429.8, 1: 9589.7. Samples: 942392684. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:47:16,063][104569] Avg episode reward: [(0, '8349.063'), (1, '9256.889')] [2023-12-27 04:47:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001842456_471736320.pth... [2023-12-27 04:47:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001841336_471449600.pth [2023-12-27 04:47:16,075][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001838384_470695936.pth... [2023-12-27 04:47:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001837232_470401024.pth [2023-12-27 04:47:16,648][105692] Updated weights for policy 0, policy_version 1838389 (0.0007) [2023-12-27 04:47:16,711][105692] Updated weights for policy 0, policy_version 1838399 (0.0008) [2023-12-27 04:47:16,771][105692] Updated weights for policy 0, policy_version 1838409 (0.0006) [2023-12-27 04:47:16,776][105620] Updated weights for policy 1, policy_version 1842464 (0.0010) [2023-12-27 04:47:16,834][105620] Updated weights for policy 1, policy_version 1842474 (0.0010) [2023-12-27 04:47:16,892][105620] Updated weights for policy 1, policy_version 1842484 (0.0010) [2023-12-27 04:47:17,434][105692] Updated weights for policy 0, policy_version 1838419 (0.0008) [2023-12-27 04:47:17,490][105692] Updated weights for policy 0, policy_version 1838429 (0.0008) [2023-12-27 04:47:17,549][105692] Updated weights for policy 0, policy_version 1838439 (0.0009) [2023-12-27 04:47:17,621][105620] Updated weights for policy 1, policy_version 1842494 (0.0010) [2023-12-27 04:47:17,677][105620] Updated weights for policy 1, policy_version 1842504 (0.0011) [2023-12-27 04:47:17,740][105620] Updated weights for policy 1, policy_version 1842514 (0.0009) [2023-12-27 04:47:18,357][105620] Updated weights for policy 1, policy_version 1842524 (0.0007) [2023-12-27 04:47:18,362][105692] Updated weights for policy 0, policy_version 1838449 (0.0009) [2023-12-27 04:47:18,412][105620] Updated weights for policy 1, policy_version 1842534 (0.0008) [2023-12-27 04:47:18,414][105692] Updated weights for policy 0, policy_version 1838459 (0.0007) [2023-12-27 04:47:18,472][105692] Updated weights for policy 0, policy_version 1838469 (0.0008) [2023-12-27 04:47:18,477][105620] Updated weights for policy 1, policy_version 1842544 (0.0009) [2023-12-27 04:47:18,526][105692] Updated weights for policy 0, policy_version 1838479 (0.0008) [2023-12-27 04:47:19,122][105620] Updated weights for policy 1, policy_version 1842554 (0.0009) [2023-12-27 04:47:19,184][105620] Updated weights for policy 1, policy_version 1842564 (0.0011) [2023-12-27 04:47:19,255][105620] Updated weights for policy 1, policy_version 1842574 (0.0010) [2023-12-27 04:47:19,320][105620] Updated weights for policy 1, policy_version 1842584 (0.0009) [2023-12-27 04:47:19,374][105692] Updated weights for policy 0, policy_version 1838489 (0.0009) [2023-12-27 04:47:19,426][105692] Updated weights for policy 0, policy_version 1838499 (0.0008) [2023-12-27 04:47:19,484][105692] Updated weights for policy 0, policy_version 1838509 (0.0008) [2023-12-27 04:47:20,091][105620] Updated weights for policy 1, policy_version 1842594 (0.0011) [2023-12-27 04:47:20,148][105620] Updated weights for policy 1, policy_version 1842604 (0.0011) [2023-12-27 04:47:20,205][105620] Updated weights for policy 1, policy_version 1842614 (0.0011) [2023-12-27 04:47:20,303][105692] Updated weights for policy 0, policy_version 1838519 (0.0008) [2023-12-27 04:47:20,360][105692] Updated weights for policy 0, policy_version 1838529 (0.0008) [2023-12-27 04:47:20,421][105692] Updated weights for policy 0, policy_version 1838539 (0.0009) [2023-12-27 04:47:20,977][105620] Updated weights for policy 1, policy_version 1842624 (0.0010) [2023-12-27 04:47:21,048][105620] Updated weights for policy 1, policy_version 1842634 (0.0009) [2023-12-27 04:47:21,062][104569] Fps is (10 sec: 18022.0, 60 sec: 19114.6, 300 sec: 19494.2). Total num frames: 942514176. Throughput: 0: 9555.2, 1: 9523.7. Samples: 942511292. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-12-27 04:47:21,063][104569] Avg episode reward: [(0, '8626.653'), (1, '9256.951')] [2023-12-27 04:47:21,119][105620] Updated weights for policy 1, policy_version 1842644 (0.0009) [2023-12-27 04:47:21,200][105692] Updated weights for policy 0, policy_version 1838549 (0.0007) [2023-12-27 04:47:21,270][105692] Updated weights for policy 0, policy_version 1838559 (0.0007) [2023-12-27 04:47:21,330][105692] Updated weights for policy 0, policy_version 1838569 (0.0007) [2023-12-27 04:47:21,876][105620] Updated weights for policy 1, policy_version 1842654 (0.0009) [2023-12-27 04:47:21,937][105620] Updated weights for policy 1, policy_version 1842664 (0.0009) [2023-12-27 04:47:21,990][105620] Updated weights for policy 1, policy_version 1842674 (0.0008) [2023-12-27 04:47:22,064][105692] Updated weights for policy 0, policy_version 1838579 (0.0009) [2023-12-27 04:47:22,121][105692] Updated weights for policy 0, policy_version 1838589 (0.0010) [2023-12-27 04:47:22,174][105692] Updated weights for policy 0, policy_version 1838599 (0.0010) [2023-12-27 04:47:22,721][105620] Updated weights for policy 1, policy_version 1842684 (0.0007) [2023-12-27 04:47:22,786][105620] Updated weights for policy 1, policy_version 1842694 (0.0008) [2023-12-27 04:47:22,838][105620] Updated weights for policy 1, policy_version 1842704 (0.0009) [2023-12-27 04:47:22,955][105692] Updated weights for policy 0, policy_version 1838609 (0.0009) [2023-12-27 04:47:23,017][105692] Updated weights for policy 0, policy_version 1838619 (0.0009) [2023-12-27 04:47:23,078][105692] Updated weights for policy 0, policy_version 1838629 (0.0009) [2023-12-27 04:47:23,136][105692] Updated weights for policy 0, policy_version 1838639 (0.0010) [2023-12-27 04:47:23,582][105620] Updated weights for policy 1, policy_version 1842714 (0.0006) [2023-12-27 04:47:23,636][105620] Updated weights for policy 1, policy_version 1842724 (0.0008) [2023-12-27 04:47:23,697][105620] Updated weights for policy 1, policy_version 1842734 (0.0009) [2023-12-27 04:47:23,758][105620] Updated weights for policy 1, policy_version 1842744 (0.0009) [2023-12-27 04:47:23,828][105692] Updated weights for policy 0, policy_version 1838649 (0.0009) [2023-12-27 04:47:23,882][105692] Updated weights for policy 0, policy_version 1838659 (0.0009) [2023-12-27 04:47:23,933][105692] Updated weights for policy 0, policy_version 1838669 (0.0009) [2023-12-27 04:47:24,374][105620] Updated weights for policy 1, policy_version 1842754 (0.0009) [2023-12-27 04:47:24,429][105620] Updated weights for policy 1, policy_version 1842764 (0.0009) [2023-12-27 04:47:24,485][105620] Updated weights for policy 1, policy_version 1842774 (0.0009) [2023-12-27 04:47:24,747][105692] Updated weights for policy 0, policy_version 1838679 (0.0009) [2023-12-27 04:47:24,805][105692] Updated weights for policy 0, policy_version 1838689 (0.0009) [2023-12-27 04:47:24,860][105692] Updated weights for policy 0, policy_version 1838699 (0.0009) [2023-12-27 04:47:25,250][105620] Updated weights for policy 1, policy_version 1842784 (0.0009) [2023-12-27 04:47:25,301][105620] Updated weights for policy 1, policy_version 1842794 (0.0009) [2023-12-27 04:47:25,363][105620] Updated weights for policy 1, policy_version 1842804 (0.0009) [2023-12-27 04:47:25,586][105692] Updated weights for policy 0, policy_version 1838709 (0.0009) [2023-12-27 04:47:25,647][105692] Updated weights for policy 0, policy_version 1838719 (0.0009) [2023-12-27 04:47:25,700][105692] Updated weights for policy 0, policy_version 1838729 (0.0008) [2023-12-27 04:47:26,062][104569] Fps is (10 sec: 18841.3, 60 sec: 18978.0, 300 sec: 19466.4). Total num frames: 942612480. Throughput: 0: 9532.4, 1: 9479.6. Samples: 942623940. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:47:26,063][104569] Avg episode reward: [(0, '8444.119'), (1, '9349.227')] [2023-12-27 04:47:26,120][105620] Updated weights for policy 1, policy_version 1842814 (0.0009) [2023-12-27 04:47:26,167][105620] Updated weights for policy 1, policy_version 1842824 (0.0009) [2023-12-27 04:47:26,219][105620] Updated weights for policy 1, policy_version 1842834 (0.0008) [2023-12-27 04:47:26,468][105692] Updated weights for policy 0, policy_version 1838739 (0.0009) [2023-12-27 04:47:26,515][105692] Updated weights for policy 0, policy_version 1838749 (0.0009) [2023-12-27 04:47:26,566][105692] Updated weights for policy 0, policy_version 1838759 (0.0009) [2023-12-27 04:47:26,952][105620] Updated weights for policy 1, policy_version 1842844 (0.0007) [2023-12-27 04:47:27,001][105620] Updated weights for policy 1, policy_version 1842854 (0.0009) [2023-12-27 04:47:27,063][105620] Updated weights for policy 1, policy_version 1842864 (0.0008) [2023-12-27 04:47:27,329][105692] Updated weights for policy 0, policy_version 1838769 (0.0009) [2023-12-27 04:47:27,382][105692] Updated weights for policy 0, policy_version 1838779 (0.0010) [2023-12-27 04:47:27,449][105692] Updated weights for policy 0, policy_version 1838789 (0.0009) [2023-12-27 04:47:27,519][105692] Updated weights for policy 0, policy_version 1838799 (0.0010) [2023-12-27 04:47:27,738][105620] Updated weights for policy 1, policy_version 1842874 (0.0009) [2023-12-27 04:47:27,795][105620] Updated weights for policy 1, policy_version 1842884 (0.0008) [2023-12-27 04:47:27,848][105620] Updated weights for policy 1, policy_version 1842894 (0.0007) [2023-12-27 04:47:27,899][105620] Updated weights for policy 1, policy_version 1842904 (0.0009) [2023-12-27 04:47:28,289][105692] Updated weights for policy 0, policy_version 1838809 (0.0009) [2023-12-27 04:47:28,348][105692] Updated weights for policy 0, policy_version 1838819 (0.0009) [2023-12-27 04:47:28,396][105692] Updated weights for policy 0, policy_version 1838830 (0.0009) [2023-12-27 04:47:28,638][105620] Updated weights for policy 1, policy_version 1842914 (0.0009) [2023-12-27 04:47:28,689][105620] Updated weights for policy 1, policy_version 1842924 (0.0009) [2023-12-27 04:47:28,737][105620] Updated weights for policy 1, policy_version 1842934 (0.0009) [2023-12-27 04:47:29,179][105692] Updated weights for policy 0, policy_version 1838840 (0.0006) [2023-12-27 04:47:29,238][105692] Updated weights for policy 0, policy_version 1838850 (0.0008) [2023-12-27 04:47:29,298][105692] Updated weights for policy 0, policy_version 1838860 (0.0010) [2023-12-27 04:47:29,523][105620] Updated weights for policy 1, policy_version 1842944 (0.0006) [2023-12-27 04:47:29,569][105620] Updated weights for policy 1, policy_version 1842954 (0.0005) [2023-12-27 04:47:29,619][105620] Updated weights for policy 1, policy_version 1842964 (0.0005) [2023-12-27 04:47:30,064][105692] Updated weights for policy 0, policy_version 1838870 (0.0009) [2023-12-27 04:47:30,113][105692] Updated weights for policy 0, policy_version 1838880 (0.0009) [2023-12-27 04:47:30,171][105692] Updated weights for policy 0, policy_version 1838890 (0.0009) [2023-12-27 04:47:30,296][105620] Updated weights for policy 1, policy_version 1842974 (0.0005) [2023-12-27 04:47:30,348][105620] Updated weights for policy 1, policy_version 1842984 (0.0005) [2023-12-27 04:47:30,411][105620] Updated weights for policy 1, policy_version 1842994 (0.0008) [2023-12-27 04:47:30,934][105620] Updated weights for policy 1, policy_version 1843004 (0.0005) [2023-12-27 04:47:30,991][105620] Updated weights for policy 1, policy_version 1843014 (0.0010) [2023-12-27 04:47:31,036][105692] Updated weights for policy 0, policy_version 1838900 (0.0008) [2023-12-27 04:47:31,050][105620] Updated weights for policy 1, policy_version 1843024 (0.0011) [2023-12-27 04:47:31,062][104569] Fps is (10 sec: 18841.9, 60 sec: 18978.1, 300 sec: 19438.7). Total num frames: 942702592. Throughput: 0: 9563.6, 1: 9466.3. Samples: 942680076. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:47:31,062][104569] Avg episode reward: [(0, '8169.901'), (1, '9256.628')] [2023-12-27 04:47:31,093][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001843032_471883776.pth... [2023-12-27 04:47:31,098][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001841912_471597056.pth [2023-12-27 04:47:31,100][105692] Updated weights for policy 0, policy_version 1838910 (0.0006) [2023-12-27 04:47:31,153][105692] Updated weights for policy 0, policy_version 1838920 (0.0008) [2023-12-27 04:47:31,196][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001838928_470835200.pth... [2023-12-27 04:47:31,200][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001837808_470548480.pth [2023-12-27 04:47:31,743][105620] Updated weights for policy 1, policy_version 1843034 (0.0010) [2023-12-27 04:47:31,793][105620] Updated weights for policy 1, policy_version 1843044 (0.0009) [2023-12-27 04:47:31,839][105620] Updated weights for policy 1, policy_version 1843054 (0.0008) [2023-12-27 04:47:31,885][105620] Updated weights for policy 1, policy_version 1843064 (0.0008) [2023-12-27 04:47:31,944][105692] Updated weights for policy 0, policy_version 1838930 (0.0010) [2023-12-27 04:47:31,999][105692] Updated weights for policy 0, policy_version 1838940 (0.0008) [2023-12-27 04:47:32,046][105692] Updated weights for policy 0, policy_version 1838950 (0.0009) [2023-12-27 04:47:32,098][105692] Updated weights for policy 0, policy_version 1838960 (0.0009) [2023-12-27 04:47:32,646][105620] Updated weights for policy 1, policy_version 1843074 (0.0005) [2023-12-27 04:47:32,709][105620] Updated weights for policy 1, policy_version 1843084 (0.0008) [2023-12-27 04:47:32,773][105620] Updated weights for policy 1, policy_version 1843094 (0.0007) [2023-12-27 04:47:32,909][105692] Updated weights for policy 0, policy_version 1838970 (0.0010) [2023-12-27 04:47:32,968][105692] Updated weights for policy 0, policy_version 1838981 (0.0010) [2023-12-27 04:47:33,020][105692] Updated weights for policy 0, policy_version 1838991 (0.0010) [2023-12-27 04:47:33,367][105620] Updated weights for policy 1, policy_version 1843104 (0.0010) [2023-12-27 04:47:33,415][105620] Updated weights for policy 1, policy_version 1843114 (0.0010) [2023-12-27 04:47:33,459][105620] Updated weights for policy 1, policy_version 1843124 (0.0010) [2023-12-27 04:47:33,886][105692] Updated weights for policy 0, policy_version 1839001 (0.0008) [2023-12-27 04:47:33,945][105692] Updated weights for policy 0, policy_version 1839011 (0.0008) [2023-12-27 04:47:33,997][105692] Updated weights for policy 0, policy_version 1839021 (0.0008) [2023-12-27 04:47:34,123][105620] Updated weights for policy 1, policy_version 1843134 (0.0011) [2023-12-27 04:47:34,180][105620] Updated weights for policy 1, policy_version 1843144 (0.0010) [2023-12-27 04:47:34,240][105620] Updated weights for policy 1, policy_version 1843154 (0.0008) [2023-12-27 04:47:34,708][105692] Updated weights for policy 0, policy_version 1839031 (0.0008) [2023-12-27 04:47:34,768][105692] Updated weights for policy 0, policy_version 1839041 (0.0005) [2023-12-27 04:47:34,828][105692] Updated weights for policy 0, policy_version 1839051 (0.0006) [2023-12-27 04:47:35,005][105620] Updated weights for policy 1, policy_version 1843164 (0.0008) [2023-12-27 04:47:35,064][105620] Updated weights for policy 1, policy_version 1843174 (0.0011) [2023-12-27 04:47:35,130][105620] Updated weights for policy 1, policy_version 1843184 (0.0011) [2023-12-27 04:47:35,454][105692] Updated weights for policy 0, policy_version 1839061 (0.0007) [2023-12-27 04:47:35,501][105692] Updated weights for policy 0, policy_version 1839071 (0.0009) [2023-12-27 04:47:35,546][105692] Updated weights for policy 0, policy_version 1839081 (0.0008) [2023-12-27 04:47:35,864][105620] Updated weights for policy 1, policy_version 1843194 (0.0010) [2023-12-27 04:47:35,913][105620] Updated weights for policy 1, policy_version 1843204 (0.0008) [2023-12-27 04:47:35,961][105620] Updated weights for policy 1, policy_version 1843214 (0.0009) [2023-12-27 04:47:36,016][105620] Updated weights for policy 1, policy_version 1843224 (0.0009) [2023-12-27 04:47:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19114.6, 300 sec: 19466.4). Total num frames: 942809088. Throughput: 0: 9453.8, 1: 9582.8. Samples: 942795504. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:47:36,063][104569] Avg episode reward: [(0, '8715.219'), (1, '9164.364')] [2023-12-27 04:47:36,289][105692] Updated weights for policy 0, policy_version 1839091 (0.0009) [2023-12-27 04:47:36,356][105692] Updated weights for policy 0, policy_version 1839101 (0.0009) [2023-12-27 04:47:36,422][105692] Updated weights for policy 0, policy_version 1839111 (0.0010) [2023-12-27 04:47:36,831][105620] Updated weights for policy 1, policy_version 1843234 (0.0008) [2023-12-27 04:47:36,894][105620] Updated weights for policy 1, policy_version 1843244 (0.0008) [2023-12-27 04:47:36,960][105620] Updated weights for policy 1, policy_version 1843254 (0.0009) [2023-12-27 04:47:37,180][105692] Updated weights for policy 0, policy_version 1839121 (0.0009) [2023-12-27 04:47:37,231][105692] Updated weights for policy 0, policy_version 1839131 (0.0009) [2023-12-27 04:47:37,283][105692] Updated weights for policy 0, policy_version 1839141 (0.0009) [2023-12-27 04:47:37,338][105692] Updated weights for policy 0, policy_version 1839151 (0.0007) [2023-12-27 04:47:37,744][105620] Updated weights for policy 1, policy_version 1843264 (0.0010) [2023-12-27 04:47:37,792][105620] Updated weights for policy 1, policy_version 1843274 (0.0010) [2023-12-27 04:47:37,840][105620] Updated weights for policy 1, policy_version 1843284 (0.0010) [2023-12-27 04:47:38,035][105692] Updated weights for policy 0, policy_version 1839161 (0.0006) [2023-12-27 04:47:38,088][105692] Updated weights for policy 0, policy_version 1839171 (0.0005) [2023-12-27 04:47:38,148][105692] Updated weights for policy 0, policy_version 1839181 (0.0005) [2023-12-27 04:47:38,504][105620] Updated weights for policy 1, policy_version 1843294 (0.0010) [2023-12-27 04:47:38,564][105620] Updated weights for policy 1, policy_version 1843304 (0.0011) [2023-12-27 04:47:38,615][105620] Updated weights for policy 1, policy_version 1843314 (0.0006) [2023-12-27 04:47:38,744][105692] Updated weights for policy 0, policy_version 1839191 (0.0008) [2023-12-27 04:47:38,795][105692] Updated weights for policy 0, policy_version 1839201 (0.0010) [2023-12-27 04:47:38,840][105692] Updated weights for policy 0, policy_version 1839211 (0.0010) [2023-12-27 04:47:39,273][105620] Updated weights for policy 1, policy_version 1843324 (0.0006) [2023-12-27 04:47:39,331][105620] Updated weights for policy 1, policy_version 1843334 (0.0009) [2023-12-27 04:47:39,396][105620] Updated weights for policy 1, policy_version 1843344 (0.0009) [2023-12-27 04:47:39,563][105692] Updated weights for policy 0, policy_version 1839221 (0.0009) [2023-12-27 04:47:39,623][105692] Updated weights for policy 0, policy_version 1839231 (0.0008) [2023-12-27 04:47:39,678][105692] Updated weights for policy 0, policy_version 1839241 (0.0006) [2023-12-27 04:47:40,185][105620] Updated weights for policy 1, policy_version 1843354 (0.0008) [2023-12-27 04:47:40,238][105620] Updated weights for policy 1, policy_version 1843364 (0.0009) [2023-12-27 04:47:40,300][105620] Updated weights for policy 1, policy_version 1843374 (0.0009) [2023-12-27 04:47:40,312][105692] Updated weights for policy 0, policy_version 1839251 (0.0005) [2023-12-27 04:47:40,361][105620] Updated weights for policy 1, policy_version 1843384 (0.0011) [2023-12-27 04:47:40,374][105692] Updated weights for policy 0, policy_version 1839261 (0.0010) [2023-12-27 04:47:40,435][105692] Updated weights for policy 0, policy_version 1839271 (0.0008) [2023-12-27 04:47:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 18978.1, 300 sec: 19438.6). Total num frames: 942899200. Throughput: 0: 9497.1, 1: 9578.5. Samples: 942912264. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:47:41,062][104569] Avg episode reward: [(0, '8535.444'), (1, '9073.182')] [2023-12-27 04:47:41,137][105692] Updated weights for policy 0, policy_version 1839281 (0.0009) [2023-12-27 04:47:41,149][105620] Updated weights for policy 1, policy_version 1843394 (0.0009) [2023-12-27 04:47:41,199][105692] Updated weights for policy 0, policy_version 1839291 (0.0007) [2023-12-27 04:47:41,213][105620] Updated weights for policy 1, policy_version 1843404 (0.0007) [2023-12-27 04:47:41,256][105692] Updated weights for policy 0, policy_version 1839301 (0.0010) [2023-12-27 04:47:41,284][105620] Updated weights for policy 1, policy_version 1843414 (0.0007) [2023-12-27 04:47:41,317][105692] Updated weights for policy 0, policy_version 1839311 (0.0011) [2023-12-27 04:47:42,088][105692] Updated weights for policy 0, policy_version 1839321 (0.0011) [2023-12-27 04:47:42,090][105620] Updated weights for policy 1, policy_version 1843424 (0.0006) [2023-12-27 04:47:42,150][105620] Updated weights for policy 1, policy_version 1843434 (0.0007) [2023-12-27 04:47:42,151][105692] Updated weights for policy 0, policy_version 1839331 (0.0011) [2023-12-27 04:47:42,206][105620] Updated weights for policy 1, policy_version 1843444 (0.0006) [2023-12-27 04:47:42,208][105692] Updated weights for policy 0, policy_version 1839341 (0.0011) [2023-12-27 04:47:42,959][105620] Updated weights for policy 1, policy_version 1843454 (0.0006) [2023-12-27 04:47:42,972][105692] Updated weights for policy 0, policy_version 1839351 (0.0009) [2023-12-27 04:47:43,024][105620] Updated weights for policy 1, policy_version 1843464 (0.0007) [2023-12-27 04:47:43,034][105692] Updated weights for policy 0, policy_version 1839361 (0.0010) [2023-12-27 04:47:43,078][105620] Updated weights for policy 1, policy_version 1843474 (0.0006) [2023-12-27 04:47:43,094][105692] Updated weights for policy 0, policy_version 1839371 (0.0008) [2023-12-27 04:47:43,748][105620] Updated weights for policy 1, policy_version 1843484 (0.0007) [2023-12-27 04:47:43,799][105620] Updated weights for policy 1, policy_version 1843494 (0.0009) [2023-12-27 04:47:43,834][105692] Updated weights for policy 0, policy_version 1839381 (0.0007) [2023-12-27 04:47:43,847][105620] Updated weights for policy 1, policy_version 1843504 (0.0009) [2023-12-27 04:47:43,882][105692] Updated weights for policy 0, policy_version 1839391 (0.0010) [2023-12-27 04:47:43,930][105692] Updated weights for policy 0, policy_version 1839401 (0.0010) [2023-12-27 04:47:44,556][105692] Updated weights for policy 0, policy_version 1839411 (0.0008) [2023-12-27 04:47:44,607][105692] Updated weights for policy 0, policy_version 1839421 (0.0010) [2023-12-27 04:47:44,662][105692] Updated weights for policy 0, policy_version 1839431 (0.0009) [2023-12-27 04:47:44,712][105620] Updated weights for policy 1, policy_version 1843514 (0.0006) [2023-12-27 04:47:44,764][105620] Updated weights for policy 1, policy_version 1843524 (0.0008) [2023-12-27 04:47:44,827][105620] Updated weights for policy 1, policy_version 1843534 (0.0009) [2023-12-27 04:47:44,891][105620] Updated weights for policy 1, policy_version 1843544 (0.0008) [2023-12-27 04:47:45,414][105692] Updated weights for policy 0, policy_version 1839441 (0.0010) [2023-12-27 04:47:45,466][105692] Updated weights for policy 0, policy_version 1839451 (0.0009) [2023-12-27 04:47:45,521][105692] Updated weights for policy 0, policy_version 1839461 (0.0009) [2023-12-27 04:47:45,572][105692] Updated weights for policy 0, policy_version 1839471 (0.0009) [2023-12-27 04:47:45,636][105620] Updated weights for policy 1, policy_version 1843554 (0.0009) [2023-12-27 04:47:45,691][105620] Updated weights for policy 1, policy_version 1843564 (0.0009) [2023-12-27 04:47:45,739][105620] Updated weights for policy 1, policy_version 1843574 (0.0009) [2023-12-27 04:47:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.8, 300 sec: 19438.6). Total num frames: 942997504. Throughput: 0: 9437.5, 1: 9574.2. Samples: 942968220. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:47:46,063][104569] Avg episode reward: [(0, '8443.192'), (1, '9165.441')] [2023-12-27 04:47:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001839472_470974464.pth... [2023-12-27 04:47:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001843576_472023040.pth... [2023-12-27 04:47:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001838384_470695936.pth [2023-12-27 04:47:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001842456_471736320.pth [2023-12-27 04:47:46,351][105692] Updated weights for policy 0, policy_version 1839481 (0.0010) [2023-12-27 04:47:46,405][105692] Updated weights for policy 0, policy_version 1839491 (0.0008) [2023-12-27 04:47:46,461][105692] Updated weights for policy 0, policy_version 1839501 (0.0009) [2023-12-27 04:47:46,508][105620] Updated weights for policy 1, policy_version 1843584 (0.0009) [2023-12-27 04:47:46,561][105620] Updated weights for policy 1, policy_version 1843594 (0.0009) [2023-12-27 04:47:46,615][105620] Updated weights for policy 1, policy_version 1843604 (0.0009) [2023-12-27 04:47:47,192][105692] Updated weights for policy 0, policy_version 1839511 (0.0008) [2023-12-27 04:47:47,238][105692] Updated weights for policy 0, policy_version 1839521 (0.0009) [2023-12-27 04:47:47,287][105692] Updated weights for policy 0, policy_version 1839531 (0.0008) [2023-12-27 04:47:47,370][105620] Updated weights for policy 1, policy_version 1843614 (0.0009) [2023-12-27 04:47:47,421][105620] Updated weights for policy 1, policy_version 1843624 (0.0009) [2023-12-27 04:47:47,468][105620] Updated weights for policy 1, policy_version 1843634 (0.0008) [2023-12-27 04:47:48,047][105692] Updated weights for policy 0, policy_version 1839541 (0.0009) [2023-12-27 04:47:48,092][105692] Updated weights for policy 0, policy_version 1839551 (0.0010) [2023-12-27 04:47:48,147][105692] Updated weights for policy 0, policy_version 1839561 (0.0010) [2023-12-27 04:47:48,220][105620] Updated weights for policy 1, policy_version 1843644 (0.0008) [2023-12-27 04:47:48,272][105620] Updated weights for policy 1, policy_version 1843654 (0.0008) [2023-12-27 04:47:48,321][105620] Updated weights for policy 1, policy_version 1843664 (0.0008) [2023-12-27 04:47:48,885][105692] Updated weights for policy 0, policy_version 1839571 (0.0011) [2023-12-27 04:47:48,952][105692] Updated weights for policy 0, policy_version 1839581 (0.0011) [2023-12-27 04:47:49,011][105692] Updated weights for policy 0, policy_version 1839591 (0.0008) [2023-12-27 04:47:49,032][105620] Updated weights for policy 1, policy_version 1843674 (0.0009) [2023-12-27 04:47:49,097][105620] Updated weights for policy 1, policy_version 1843684 (0.0011) [2023-12-27 04:47:49,162][105620] Updated weights for policy 1, policy_version 1843694 (0.0011) [2023-12-27 04:47:49,229][105620] Updated weights for policy 1, policy_version 1843704 (0.0011) [2023-12-27 04:47:49,764][105692] Updated weights for policy 0, policy_version 1839601 (0.0008) [2023-12-27 04:47:49,822][105692] Updated weights for policy 0, policy_version 1839611 (0.0010) [2023-12-27 04:47:49,902][105692] Updated weights for policy 0, policy_version 1839621 (0.0007) [2023-12-27 04:47:49,977][105692] Updated weights for policy 0, policy_version 1839631 (0.0008) [2023-12-27 04:47:49,983][105620] Updated weights for policy 1, policy_version 1843714 (0.0009) [2023-12-27 04:47:50,030][105620] Updated weights for policy 1, policy_version 1843724 (0.0009) [2023-12-27 04:47:50,077][105620] Updated weights for policy 1, policy_version 1843734 (0.0009) [2023-12-27 04:47:50,666][105692] Updated weights for policy 0, policy_version 1839641 (0.0010) [2023-12-27 04:47:50,723][105692] Updated weights for policy 0, policy_version 1839651 (0.0008) [2023-12-27 04:47:50,792][105692] Updated weights for policy 0, policy_version 1839661 (0.0010) [2023-12-27 04:47:50,854][105620] Updated weights for policy 1, policy_version 1843744 (0.0008) [2023-12-27 04:47:50,907][105620] Updated weights for policy 1, policy_version 1843754 (0.0008) [2023-12-27 04:47:50,967][105620] Updated weights for policy 1, policy_version 1843764 (0.0008) [2023-12-27 04:47:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 943095808. Throughput: 0: 9503.2, 1: 9574.3. Samples: 943082188. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:47:51,063][104569] Avg episode reward: [(0, '8531.813'), (1, '9349.244')] [2023-12-27 04:47:51,558][105692] Updated weights for policy 0, policy_version 1839671 (0.0008) [2023-12-27 04:47:51,619][105692] Updated weights for policy 0, policy_version 1839681 (0.0006) [2023-12-27 04:47:51,689][105692] Updated weights for policy 0, policy_version 1839691 (0.0009) [2023-12-27 04:47:51,733][105620] Updated weights for policy 1, policy_version 1843774 (0.0008) [2023-12-27 04:47:51,796][105620] Updated weights for policy 1, policy_version 1843784 (0.0006) [2023-12-27 04:47:51,866][105620] Updated weights for policy 1, policy_version 1843794 (0.0006) [2023-12-27 04:47:52,451][105692] Updated weights for policy 0, policy_version 1839701 (0.0009) [2023-12-27 04:47:52,511][105692] Updated weights for policy 0, policy_version 1839711 (0.0008) [2023-12-27 04:47:52,561][105692] Updated weights for policy 0, policy_version 1839721 (0.0008) [2023-12-27 04:47:52,571][105620] Updated weights for policy 1, policy_version 1843804 (0.0010) [2023-12-27 04:47:52,634][105620] Updated weights for policy 1, policy_version 1843814 (0.0011) [2023-12-27 04:47:52,692][105620] Updated weights for policy 1, policy_version 1843824 (0.0010) [2023-12-27 04:47:53,310][105692] Updated weights for policy 0, policy_version 1839731 (0.0007) [2023-12-27 04:47:53,337][105620] Updated weights for policy 1, policy_version 1843834 (0.0011) [2023-12-27 04:47:53,371][105692] Updated weights for policy 0, policy_version 1839741 (0.0006) [2023-12-27 04:47:53,392][105620] Updated weights for policy 1, policy_version 1843844 (0.0010) [2023-12-27 04:47:53,430][105692] Updated weights for policy 0, policy_version 1839751 (0.0006) [2023-12-27 04:47:53,437][105620] Updated weights for policy 1, policy_version 1843854 (0.0010) [2023-12-27 04:47:53,491][105620] Updated weights for policy 1, policy_version 1843864 (0.0007) [2023-12-27 04:47:54,002][105692] Updated weights for policy 0, policy_version 1839761 (0.0008) [2023-12-27 04:47:54,067][105620] Updated weights for policy 1, policy_version 1843874 (0.0005) [2023-12-27 04:47:54,068][105692] Updated weights for policy 0, policy_version 1839771 (0.0008) [2023-12-27 04:47:54,121][105620] Updated weights for policy 1, policy_version 1843884 (0.0005) [2023-12-27 04:47:54,127][105692] Updated weights for policy 0, policy_version 1839781 (0.0006) [2023-12-27 04:47:54,182][105620] Updated weights for policy 1, policy_version 1843894 (0.0005) [2023-12-27 04:47:54,190][105692] Updated weights for policy 0, policy_version 1839791 (0.0009) [2023-12-27 04:47:54,704][105620] Updated weights for policy 1, policy_version 1843904 (0.0005) [2023-12-27 04:47:54,755][105620] Updated weights for policy 1, policy_version 1843914 (0.0010) [2023-12-27 04:47:54,782][105692] Updated weights for policy 0, policy_version 1839801 (0.0006) [2023-12-27 04:47:54,807][105620] Updated weights for policy 1, policy_version 1843924 (0.0010) [2023-12-27 04:47:54,833][105692] Updated weights for policy 0, policy_version 1839811 (0.0006) [2023-12-27 04:47:54,889][105692] Updated weights for policy 0, policy_version 1839821 (0.0007) [2023-12-27 04:47:55,409][105620] Updated weights for policy 1, policy_version 1843934 (0.0008) [2023-12-27 04:47:55,479][105620] Updated weights for policy 1, policy_version 1843944 (0.0005) [2023-12-27 04:47:55,536][105620] Updated weights for policy 1, policy_version 1843954 (0.0006) [2023-12-27 04:47:55,616][105692] Updated weights for policy 0, policy_version 1839831 (0.0009) [2023-12-27 04:47:55,674][105692] Updated weights for policy 0, policy_version 1839842 (0.0011) [2023-12-27 04:47:55,724][105692] Updated weights for policy 0, policy_version 1839852 (0.0008) [2023-12-27 04:47:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19114.6, 300 sec: 19438.6). Total num frames: 943194112. Throughput: 0: 9523.4, 1: 9748.1. Samples: 943204464. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:47:56,063][104569] Avg episode reward: [(0, '8259.284'), (1, '9164.690')] [2023-12-27 04:47:56,143][105620] Updated weights for policy 1, policy_version 1843964 (0.0008) [2023-12-27 04:47:56,195][105620] Updated weights for policy 1, policy_version 1843974 (0.0006) [2023-12-27 04:47:56,241][105620] Updated weights for policy 1, policy_version 1843984 (0.0005) [2023-12-27 04:47:56,348][105692] Updated weights for policy 0, policy_version 1839862 (0.0009) [2023-12-27 04:47:56,412][105692] Updated weights for policy 0, policy_version 1839872 (0.0006) [2023-12-27 04:47:56,472][105692] Updated weights for policy 0, policy_version 1839882 (0.0006) [2023-12-27 04:47:56,948][105620] Updated weights for policy 1, policy_version 1843994 (0.0005) [2023-12-27 04:47:56,997][105620] Updated weights for policy 1, policy_version 1844004 (0.0005) [2023-12-27 04:47:57,050][105620] Updated weights for policy 1, policy_version 1844014 (0.0005) [2023-12-27 04:47:57,056][105692] Updated weights for policy 0, policy_version 1839892 (0.0007) [2023-12-27 04:47:57,101][105620] Updated weights for policy 1, policy_version 1844024 (0.0006) [2023-12-27 04:47:57,104][105692] Updated weights for policy 0, policy_version 1839902 (0.0010) [2023-12-27 04:47:57,150][105692] Updated weights for policy 0, policy_version 1839912 (0.0006) [2023-12-27 04:47:57,663][105620] Updated weights for policy 1, policy_version 1844034 (0.0005) [2023-12-27 04:47:57,714][105620] Updated weights for policy 1, policy_version 1844044 (0.0005) [2023-12-27 04:47:57,772][105620] Updated weights for policy 1, policy_version 1844054 (0.0007) [2023-12-27 04:47:57,877][105692] Updated weights for policy 0, policy_version 1839922 (0.0005) [2023-12-27 04:47:57,924][105692] Updated weights for policy 0, policy_version 1839932 (0.0009) [2023-12-27 04:47:57,968][105692] Updated weights for policy 0, policy_version 1839942 (0.0006) [2023-12-27 04:47:58,023][105692] Updated weights for policy 0, policy_version 1839952 (0.0005) [2023-12-27 04:47:58,473][105620] Updated weights for policy 1, policy_version 1844064 (0.0008) [2023-12-27 04:47:58,538][105620] Updated weights for policy 1, policy_version 1844074 (0.0008) [2023-12-27 04:47:58,603][105620] Updated weights for policy 1, policy_version 1844084 (0.0008) [2023-12-27 04:47:58,732][105692] Updated weights for policy 0, policy_version 1839962 (0.0007) [2023-12-27 04:47:58,802][105692] Updated weights for policy 0, policy_version 1839972 (0.0008) [2023-12-27 04:47:58,871][105692] Updated weights for policy 0, policy_version 1839982 (0.0009) [2023-12-27 04:47:59,443][105620] Updated weights for policy 1, policy_version 1844094 (0.0007) [2023-12-27 04:47:59,511][105620] Updated weights for policy 1, policy_version 1844104 (0.0007) [2023-12-27 04:47:59,574][105620] Updated weights for policy 1, policy_version 1844114 (0.0009) [2023-12-27 04:47:59,676][105692] Updated weights for policy 0, policy_version 1839992 (0.0009) [2023-12-27 04:47:59,738][105692] Updated weights for policy 0, policy_version 1840002 (0.0009) [2023-12-27 04:47:59,796][105692] Updated weights for policy 0, policy_version 1840012 (0.0008) [2023-12-27 04:48:00,237][105620] Updated weights for policy 1, policy_version 1844124 (0.0009) [2023-12-27 04:48:00,294][105620] Updated weights for policy 1, policy_version 1844134 (0.0009) [2023-12-27 04:48:00,356][105620] Updated weights for policy 1, policy_version 1844144 (0.0010) [2023-12-27 04:48:00,415][105692] Updated weights for policy 0, policy_version 1840022 (0.0007) [2023-12-27 04:48:00,475][105692] Updated weights for policy 0, policy_version 1840032 (0.0005) [2023-12-27 04:48:00,529][105692] Updated weights for policy 0, policy_version 1840042 (0.0008) [2023-12-27 04:48:01,031][105620] Updated weights for policy 1, policy_version 1844154 (0.0008) [2023-12-27 04:48:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 943292416. Throughput: 0: 9637.2, 1: 9790.0. Samples: 943266908. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:48:01,063][104569] Avg episode reward: [(0, '8260.745'), (1, '9164.680')] [2023-12-27 04:48:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001840048_471121920.pth... [2023-12-27 04:48:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001838928_470835200.pth [2023-12-27 04:48:01,088][105620] Updated weights for policy 1, policy_version 1844164 (0.0009) [2023-12-27 04:48:01,156][105620] Updated weights for policy 1, policy_version 1844174 (0.0009) [2023-12-27 04:48:01,208][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001844184_472178688.pth... [2023-12-27 04:48:01,209][105620] Updated weights for policy 1, policy_version 1844184 (0.0008) [2023-12-27 04:48:01,211][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001843032_471883776.pth [2023-12-27 04:48:01,244][105692] Updated weights for policy 0, policy_version 1840052 (0.0009) [2023-12-27 04:48:01,307][105692] Updated weights for policy 0, policy_version 1840062 (0.0009) [2023-12-27 04:48:01,379][105692] Updated weights for policy 0, policy_version 1840072 (0.0009) [2023-12-27 04:48:01,997][105620] Updated weights for policy 1, policy_version 1844194 (0.0009) [2023-12-27 04:48:02,055][105620] Updated weights for policy 1, policy_version 1844204 (0.0005) [2023-12-27 04:48:02,098][105692] Updated weights for policy 0, policy_version 1840082 (0.0006) [2023-12-27 04:48:02,109][105620] Updated weights for policy 1, policy_version 1844214 (0.0006) [2023-12-27 04:48:02,157][105692] Updated weights for policy 0, policy_version 1840092 (0.0010) [2023-12-27 04:48:02,215][105692] Updated weights for policy 0, policy_version 1840102 (0.0009) [2023-12-27 04:48:02,266][105692] Updated weights for policy 0, policy_version 1840112 (0.0009) [2023-12-27 04:48:02,714][105620] Updated weights for policy 1, policy_version 1844224 (0.0006) [2023-12-27 04:48:02,770][105620] Updated weights for policy 1, policy_version 1844234 (0.0006) [2023-12-27 04:48:02,829][105620] Updated weights for policy 1, policy_version 1844244 (0.0006) [2023-12-27 04:48:03,157][105692] Updated weights for policy 0, policy_version 1840122 (0.0010) [2023-12-27 04:48:03,210][105692] Updated weights for policy 0, policy_version 1840132 (0.0010) [2023-12-27 04:48:03,269][105692] Updated weights for policy 0, policy_version 1840143 (0.0010) [2023-12-27 04:48:03,352][105620] Updated weights for policy 1, policy_version 1844254 (0.0005) [2023-12-27 04:48:03,409][105620] Updated weights for policy 1, policy_version 1844264 (0.0005) [2023-12-27 04:48:03,465][105620] Updated weights for policy 1, policy_version 1844274 (0.0005) [2023-12-27 04:48:04,039][105620] Updated weights for policy 1, policy_version 1844284 (0.0007) [2023-12-27 04:48:04,098][105620] Updated weights for policy 1, policy_version 1844294 (0.0010) [2023-12-27 04:48:04,152][105692] Updated weights for policy 0, policy_version 1840153 (0.0008) [2023-12-27 04:48:04,164][105620] Updated weights for policy 1, policy_version 1844304 (0.0011) [2023-12-27 04:48:04,196][105692] Updated weights for policy 0, policy_version 1840163 (0.0007) [2023-12-27 04:48:04,242][105692] Updated weights for policy 0, policy_version 1840173 (0.0006) [2023-12-27 04:48:04,788][105620] Updated weights for policy 1, policy_version 1844314 (0.0010) [2023-12-27 04:48:04,851][105620] Updated weights for policy 1, policy_version 1844324 (0.0009) [2023-12-27 04:48:04,915][105620] Updated weights for policy 1, policy_version 1844334 (0.0010) [2023-12-27 04:48:04,977][105620] Updated weights for policy 1, policy_version 1844344 (0.0009) [2023-12-27 04:48:05,084][105692] Updated weights for policy 0, policy_version 1840183 (0.0008) [2023-12-27 04:48:05,143][105692] Updated weights for policy 0, policy_version 1840193 (0.0009) [2023-12-27 04:48:05,198][105692] Updated weights for policy 0, policy_version 1840203 (0.0009) [2023-12-27 04:48:05,664][105620] Updated weights for policy 1, policy_version 1844354 (0.0006) [2023-12-27 04:48:05,724][105620] Updated weights for policy 1, policy_version 1844364 (0.0006) [2023-12-27 04:48:05,789][105620] Updated weights for policy 1, policy_version 1844374 (0.0009) [2023-12-27 04:48:06,023][105692] Updated weights for policy 0, policy_version 1840213 (0.0009) [2023-12-27 04:48:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 943390720. Throughput: 0: 9520.0, 1: 9877.7. Samples: 943384188. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:48:06,063][104569] Avg episode reward: [(0, '8532.284'), (1, '9165.450')] [2023-12-27 04:48:06,089][105692] Updated weights for policy 0, policy_version 1840223 (0.0010) [2023-12-27 04:48:06,148][105692] Updated weights for policy 0, policy_version 1840233 (0.0009) [2023-12-27 04:48:06,336][105620] Updated weights for policy 1, policy_version 1844384 (0.0007) [2023-12-27 04:48:06,395][105620] Updated weights for policy 1, policy_version 1844394 (0.0005) [2023-12-27 04:48:06,456][105620] Updated weights for policy 1, policy_version 1844404 (0.0010) [2023-12-27 04:48:07,004][105692] Updated weights for policy 0, policy_version 1840243 (0.0009) [2023-12-27 04:48:07,069][105692] Updated weights for policy 0, policy_version 1840253 (0.0006) [2023-12-27 04:48:07,095][105620] Updated weights for policy 1, policy_version 1844414 (0.0010) [2023-12-27 04:48:07,126][105692] Updated weights for policy 0, policy_version 1840263 (0.0007) [2023-12-27 04:48:07,158][105620] Updated weights for policy 1, policy_version 1844424 (0.0010) [2023-12-27 04:48:07,217][105620] Updated weights for policy 1, policy_version 1844434 (0.0009) [2023-12-27 04:48:07,788][105620] Updated weights for policy 1, policy_version 1844444 (0.0006) [2023-12-27 04:48:07,845][105620] Updated weights for policy 1, policy_version 1844454 (0.0005) [2023-12-27 04:48:07,897][105620] Updated weights for policy 1, policy_version 1844464 (0.0005) [2023-12-27 04:48:07,966][105692] Updated weights for policy 0, policy_version 1840273 (0.0009) [2023-12-27 04:48:08,033][105692] Updated weights for policy 0, policy_version 1840283 (0.0008) [2023-12-27 04:48:08,099][105692] Updated weights for policy 0, policy_version 1840293 (0.0010) [2023-12-27 04:48:08,162][105692] Updated weights for policy 0, policy_version 1840303 (0.0010) [2023-12-27 04:48:08,469][105620] Updated weights for policy 1, policy_version 1844474 (0.0007) [2023-12-27 04:48:08,527][105620] Updated weights for policy 1, policy_version 1844484 (0.0010) [2023-12-27 04:48:08,594][105620] Updated weights for policy 1, policy_version 1844494 (0.0006) [2023-12-27 04:48:08,655][105620] Updated weights for policy 1, policy_version 1844504 (0.0009) [2023-12-27 04:48:08,910][105692] Updated weights for policy 0, policy_version 1840313 (0.0008) [2023-12-27 04:48:08,968][105692] Updated weights for policy 0, policy_version 1840323 (0.0008) [2023-12-27 04:48:09,026][105692] Updated weights for policy 0, policy_version 1840333 (0.0008) [2023-12-27 04:48:09,361][105620] Updated weights for policy 1, policy_version 1844514 (0.0009) [2023-12-27 04:48:09,428][105620] Updated weights for policy 1, policy_version 1844524 (0.0012) [2023-12-27 04:48:09,493][105620] Updated weights for policy 1, policy_version 1844534 (0.0011) [2023-12-27 04:48:09,855][105692] Updated weights for policy 0, policy_version 1840343 (0.0008) [2023-12-27 04:48:09,906][105692] Updated weights for policy 0, policy_version 1840353 (0.0008) [2023-12-27 04:48:09,971][105692] Updated weights for policy 0, policy_version 1840363 (0.0009) [2023-12-27 04:48:10,198][105620] Updated weights for policy 1, policy_version 1844544 (0.0011) [2023-12-27 04:48:10,264][105620] Updated weights for policy 1, policy_version 1844554 (0.0011) [2023-12-27 04:48:10,327][105620] Updated weights for policy 1, policy_version 1844564 (0.0011) [2023-12-27 04:48:10,631][105692] Updated weights for policy 0, policy_version 1840373 (0.0008) [2023-12-27 04:48:10,695][105692] Updated weights for policy 0, policy_version 1840383 (0.0006) [2023-12-27 04:48:10,754][105692] Updated weights for policy 0, policy_version 1840393 (0.0005) [2023-12-27 04:48:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 943489024. Throughput: 0: 9451.5, 1: 10022.0. Samples: 943500244. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:48:11,063][104569] Avg episode reward: [(0, '8165.072'), (1, '9165.429')] [2023-12-27 04:48:11,082][105620] Updated weights for policy 1, policy_version 1844574 (0.0011) [2023-12-27 04:48:11,142][105620] Updated weights for policy 1, policy_version 1844584 (0.0010) [2023-12-27 04:48:11,207][105620] Updated weights for policy 1, policy_version 1844594 (0.0011) [2023-12-27 04:48:11,382][105692] Updated weights for policy 0, policy_version 1840403 (0.0007) [2023-12-27 04:48:11,442][105692] Updated weights for policy 0, policy_version 1840413 (0.0009) [2023-12-27 04:48:11,503][105692] Updated weights for policy 0, policy_version 1840423 (0.0008) [2023-12-27 04:48:11,979][105620] Updated weights for policy 1, policy_version 1844604 (0.0009) [2023-12-27 04:48:12,050][105620] Updated weights for policy 1, policy_version 1844614 (0.0008) [2023-12-27 04:48:12,119][105620] Updated weights for policy 1, policy_version 1844624 (0.0010) [2023-12-27 04:48:12,279][105692] Updated weights for policy 0, policy_version 1840433 (0.0008) [2023-12-27 04:48:12,342][105692] Updated weights for policy 0, policy_version 1840443 (0.0009) [2023-12-27 04:48:12,409][105692] Updated weights for policy 0, policy_version 1840453 (0.0009) [2023-12-27 04:48:12,469][105692] Updated weights for policy 0, policy_version 1840463 (0.0008) [2023-12-27 04:48:12,893][105620] Updated weights for policy 1, policy_version 1844634 (0.0009) [2023-12-27 04:48:12,954][105620] Updated weights for policy 1, policy_version 1844644 (0.0008) [2023-12-27 04:48:13,014][105620] Updated weights for policy 1, policy_version 1844654 (0.0008) [2023-12-27 04:48:13,063][105620] Updated weights for policy 1, policy_version 1844664 (0.0008) [2023-12-27 04:48:13,138][105692] Updated weights for policy 0, policy_version 1840473 (0.0010) [2023-12-27 04:48:13,197][105692] Updated weights for policy 0, policy_version 1840483 (0.0010) [2023-12-27 04:48:13,253][105692] Updated weights for policy 0, policy_version 1840493 (0.0008) [2023-12-27 04:48:13,800][105620] Updated weights for policy 1, policy_version 1844674 (0.0006) [2023-12-27 04:48:13,844][105692] Updated weights for policy 0, policy_version 1840503 (0.0007) [2023-12-27 04:48:13,853][105620] Updated weights for policy 1, policy_version 1844684 (0.0007) [2023-12-27 04:48:13,901][105620] Updated weights for policy 1, policy_version 1844694 (0.0007) [2023-12-27 04:48:13,905][105692] Updated weights for policy 0, policy_version 1840513 (0.0008) [2023-12-27 04:48:13,967][105692] Updated weights for policy 0, policy_version 1840523 (0.0010) [2023-12-27 04:48:14,452][105620] Updated weights for policy 1, policy_version 1844704 (0.0007) [2023-12-27 04:48:14,506][105620] Updated weights for policy 1, policy_version 1844714 (0.0005) [2023-12-27 04:48:14,570][105620] Updated weights for policy 1, policy_version 1844724 (0.0005) [2023-12-27 04:48:14,639][105692] Updated weights for policy 0, policy_version 1840533 (0.0010) [2023-12-27 04:48:14,701][105692] Updated weights for policy 0, policy_version 1840543 (0.0008) [2023-12-27 04:48:14,769][105692] Updated weights for policy 0, policy_version 1840553 (0.0011) [2023-12-27 04:48:15,299][105620] Updated weights for policy 1, policy_version 1844734 (0.0007) [2023-12-27 04:48:15,356][105620] Updated weights for policy 1, policy_version 1844744 (0.0006) [2023-12-27 04:48:15,414][105620] Updated weights for policy 1, policy_version 1844754 (0.0006) [2023-12-27 04:48:15,498][105692] Updated weights for policy 0, policy_version 1840563 (0.0010) [2023-12-27 04:48:15,553][105692] Updated weights for policy 0, policy_version 1840573 (0.0005) [2023-12-27 04:48:15,614][105692] Updated weights for policy 0, policy_version 1840583 (0.0006) [2023-12-27 04:48:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 943587328. Throughput: 0: 9522.6, 1: 9983.7. Samples: 943557860. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:48:16,062][104569] Avg episode reward: [(0, '8351.168'), (1, '9349.310')] [2023-12-27 04:48:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001840592_471261184.pth... [2023-12-27 04:48:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001839472_470974464.pth [2023-12-27 04:48:16,096][105620] Updated weights for policy 1, policy_version 1844764 (0.0010) [2023-12-27 04:48:16,148][105620] Updated weights for policy 1, policy_version 1844774 (0.0008) [2023-12-27 04:48:16,201][105620] Updated weights for policy 1, policy_version 1844784 (0.0009) [2023-12-27 04:48:16,240][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001844792_472334336.pth... [2023-12-27 04:48:16,243][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001843576_472023040.pth [2023-12-27 04:48:16,252][105692] Updated weights for policy 0, policy_version 1840593 (0.0008) [2023-12-27 04:48:16,307][105692] Updated weights for policy 0, policy_version 1840603 (0.0009) [2023-12-27 04:48:16,356][105692] Updated weights for policy 0, policy_version 1840613 (0.0008) [2023-12-27 04:48:16,415][105692] Updated weights for policy 0, policy_version 1840623 (0.0009) [2023-12-27 04:48:16,914][105620] Updated weights for policy 1, policy_version 1844794 (0.0008) [2023-12-27 04:48:16,966][105620] Updated weights for policy 1, policy_version 1844804 (0.0009) [2023-12-27 04:48:17,019][105620] Updated weights for policy 1, policy_version 1844814 (0.0009) [2023-12-27 04:48:17,110][105692] Updated weights for policy 0, policy_version 1840633 (0.0008) [2023-12-27 04:48:17,161][105692] Updated weights for policy 0, policy_version 1840643 (0.0008) [2023-12-27 04:48:17,226][105692] Updated weights for policy 0, policy_version 1840653 (0.0007) [2023-12-27 04:48:17,846][105620] Updated weights for policy 1, policy_version 1844825 (0.0008) [2023-12-27 04:48:17,892][105692] Updated weights for policy 0, policy_version 1840663 (0.0007) [2023-12-27 04:48:17,896][105620] Updated weights for policy 1, policy_version 1844835 (0.0008) [2023-12-27 04:48:17,953][105692] Updated weights for policy 0, policy_version 1840673 (0.0005) [2023-12-27 04:48:17,956][105620] Updated weights for policy 1, policy_version 1844845 (0.0010) [2023-12-27 04:48:18,008][105692] Updated weights for policy 0, policy_version 1840683 (0.0005) [2023-12-27 04:48:18,018][105620] Updated weights for policy 1, policy_version 1844855 (0.0011) [2023-12-27 04:48:18,661][105692] Updated weights for policy 0, policy_version 1840693 (0.0006) [2023-12-27 04:48:18,717][105692] Updated weights for policy 0, policy_version 1840703 (0.0006) [2023-12-27 04:48:18,747][105620] Updated weights for policy 1, policy_version 1844865 (0.0008) [2023-12-27 04:48:18,770][105692] Updated weights for policy 0, policy_version 1840713 (0.0007) [2023-12-27 04:48:18,811][105620] Updated weights for policy 1, policy_version 1844875 (0.0009) [2023-12-27 04:48:18,879][105620] Updated weights for policy 1, policy_version 1844885 (0.0010) [2023-12-27 04:48:19,543][105620] Updated weights for policy 1, policy_version 1844895 (0.0007) [2023-12-27 04:48:19,556][105692] Updated weights for policy 0, policy_version 1840723 (0.0006) [2023-12-27 04:48:19,608][105620] Updated weights for policy 1, policy_version 1844905 (0.0007) [2023-12-27 04:48:19,627][105692] Updated weights for policy 0, policy_version 1840733 (0.0009) [2023-12-27 04:48:19,670][105620] Updated weights for policy 1, policy_version 1844915 (0.0008) [2023-12-27 04:48:19,691][105692] Updated weights for policy 0, policy_version 1840743 (0.0007) [2023-12-27 04:48:20,428][105620] Updated weights for policy 1, policy_version 1844925 (0.0008) [2023-12-27 04:48:20,446][105692] Updated weights for policy 0, policy_version 1840753 (0.0009) [2023-12-27 04:48:20,481][105620] Updated weights for policy 1, policy_version 1844935 (0.0007) [2023-12-27 04:48:20,512][105692] Updated weights for policy 0, policy_version 1840763 (0.0010) [2023-12-27 04:48:20,531][105620] Updated weights for policy 1, policy_version 1844945 (0.0006) [2023-12-27 04:48:20,578][105692] Updated weights for policy 0, policy_version 1840773 (0.0010) [2023-12-27 04:48:20,638][105692] Updated weights for policy 0, policy_version 1840783 (0.0011) [2023-12-27 04:48:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 943685632. Throughput: 0: 9681.9, 1: 9922.2. Samples: 943677684. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:48:21,062][104569] Avg episode reward: [(0, '8536.968'), (1, '9256.692')] [2023-12-27 04:48:21,335][105620] Updated weights for policy 1, policy_version 1844955 (0.0009) [2023-12-27 04:48:21,414][105620] Updated weights for policy 1, policy_version 1844965 (0.0007) [2023-12-27 04:48:21,428][105692] Updated weights for policy 0, policy_version 1840793 (0.0009) [2023-12-27 04:48:21,474][105620] Updated weights for policy 1, policy_version 1844975 (0.0006) [2023-12-27 04:48:21,484][105692] Updated weights for policy 0, policy_version 1840803 (0.0008) [2023-12-27 04:48:21,539][105692] Updated weights for policy 0, policy_version 1840813 (0.0007) [2023-12-27 04:48:22,077][105620] Updated weights for policy 1, policy_version 1844985 (0.0007) [2023-12-27 04:48:22,143][105620] Updated weights for policy 1, policy_version 1844995 (0.0009) [2023-12-27 04:48:22,207][105620] Updated weights for policy 1, policy_version 1845005 (0.0009) [2023-12-27 04:48:22,273][105620] Updated weights for policy 1, policy_version 1845015 (0.0009) [2023-12-27 04:48:22,392][105692] Updated weights for policy 0, policy_version 1840823 (0.0009) [2023-12-27 04:48:22,453][105692] Updated weights for policy 0, policy_version 1840833 (0.0010) [2023-12-27 04:48:22,517][105692] Updated weights for policy 0, policy_version 1840843 (0.0009) [2023-12-27 04:48:23,021][105620] Updated weights for policy 1, policy_version 1845025 (0.0006) [2023-12-27 04:48:23,086][105620] Updated weights for policy 1, policy_version 1845035 (0.0006) [2023-12-27 04:48:23,147][105620] Updated weights for policy 1, policy_version 1845045 (0.0008) [2023-12-27 04:48:23,319][105692] Updated weights for policy 0, policy_version 1840853 (0.0009) [2023-12-27 04:48:23,367][105692] Updated weights for policy 0, policy_version 1840863 (0.0009) [2023-12-27 04:48:23,415][105692] Updated weights for policy 0, policy_version 1840873 (0.0009) [2023-12-27 04:48:23,856][105620] Updated weights for policy 1, policy_version 1845055 (0.0009) [2023-12-27 04:48:23,909][105620] Updated weights for policy 1, policy_version 1845065 (0.0009) [2023-12-27 04:48:23,979][105620] Updated weights for policy 1, policy_version 1845075 (0.0009) [2023-12-27 04:48:24,147][105692] Updated weights for policy 0, policy_version 1840883 (0.0010) [2023-12-27 04:48:24,202][105692] Updated weights for policy 0, policy_version 1840893 (0.0010) [2023-12-27 04:48:24,261][105692] Updated weights for policy 0, policy_version 1840903 (0.0011) [2023-12-27 04:48:24,602][105620] Updated weights for policy 1, policy_version 1845085 (0.0008) [2023-12-27 04:48:24,652][105620] Updated weights for policy 1, policy_version 1845095 (0.0009) [2023-12-27 04:48:24,698][105620] Updated weights for policy 1, policy_version 1845105 (0.0007) [2023-12-27 04:48:25,000][105692] Updated weights for policy 0, policy_version 1840913 (0.0010) [2023-12-27 04:48:25,059][105692] Updated weights for policy 0, policy_version 1840923 (0.0009) [2023-12-27 04:48:25,110][105692] Updated weights for policy 0, policy_version 1840933 (0.0009) [2023-12-27 04:48:25,160][105692] Updated weights for policy 0, policy_version 1840943 (0.0007) [2023-12-27 04:48:25,449][105620] Updated weights for policy 1, policy_version 1845115 (0.0006) [2023-12-27 04:48:25,506][105620] Updated weights for policy 1, policy_version 1845125 (0.0006) [2023-12-27 04:48:25,561][105620] Updated weights for policy 1, policy_version 1845135 (0.0006) [2023-12-27 04:48:25,812][105692] Updated weights for policy 0, policy_version 1840953 (0.0008) [2023-12-27 04:48:25,863][105692] Updated weights for policy 0, policy_version 1840963 (0.0008) [2023-12-27 04:48:25,911][105692] Updated weights for policy 0, policy_version 1840973 (0.0008) [2023-12-27 04:48:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 943783936. Throughput: 0: 9554.2, 1: 9967.4. Samples: 943790740. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:48:26,063][104569] Avg episode reward: [(0, '8352.004'), (1, '9256.582')] [2023-12-27 04:48:26,296][105620] Updated weights for policy 1, policy_version 1845145 (0.0008) [2023-12-27 04:48:26,348][105620] Updated weights for policy 1, policy_version 1845155 (0.0010) [2023-12-27 04:48:26,395][105620] Updated weights for policy 1, policy_version 1845165 (0.0010) [2023-12-27 04:48:26,465][105620] Updated weights for policy 1, policy_version 1845175 (0.0007) [2023-12-27 04:48:26,622][105692] Updated weights for policy 0, policy_version 1840983 (0.0009) [2023-12-27 04:48:26,677][105692] Updated weights for policy 0, policy_version 1840993 (0.0010) [2023-12-27 04:48:26,737][105692] Updated weights for policy 0, policy_version 1841003 (0.0010) [2023-12-27 04:48:27,065][105620] Updated weights for policy 1, policy_version 1845185 (0.0010) [2023-12-27 04:48:27,123][105620] Updated weights for policy 1, policy_version 1845195 (0.0010) [2023-12-27 04:48:27,183][105620] Updated weights for policy 1, policy_version 1845205 (0.0010) [2023-12-27 04:48:27,537][105692] Updated weights for policy 0, policy_version 1841013 (0.0009) [2023-12-27 04:48:27,593][105692] Updated weights for policy 0, policy_version 1841023 (0.0010) [2023-12-27 04:48:27,646][105692] Updated weights for policy 0, policy_version 1841034 (0.0010) [2023-12-27 04:48:27,801][105620] Updated weights for policy 1, policy_version 1845215 (0.0009) [2023-12-27 04:48:27,861][105620] Updated weights for policy 1, policy_version 1845225 (0.0011) [2023-12-27 04:48:27,919][105620] Updated weights for policy 1, policy_version 1845235 (0.0011) [2023-12-27 04:48:28,458][105692] Updated weights for policy 0, policy_version 1841044 (0.0008) [2023-12-27 04:48:28,518][105692] Updated weights for policy 0, policy_version 1841054 (0.0005) [2023-12-27 04:48:28,575][105692] Updated weights for policy 0, policy_version 1841064 (0.0007) [2023-12-27 04:48:28,643][105620] Updated weights for policy 1, policy_version 1845245 (0.0009) [2023-12-27 04:48:28,697][105620] Updated weights for policy 1, policy_version 1845255 (0.0009) [2023-12-27 04:48:28,758][105620] Updated weights for policy 1, policy_version 1845265 (0.0009) [2023-12-27 04:48:29,159][105692] Updated weights for policy 0, policy_version 1841074 (0.0009) [2023-12-27 04:48:29,208][105692] Updated weights for policy 0, policy_version 1841084 (0.0010) [2023-12-27 04:48:29,268][105692] Updated weights for policy 0, policy_version 1841094 (0.0011) [2023-12-27 04:48:29,328][105692] Updated weights for policy 0, policy_version 1841104 (0.0009) [2023-12-27 04:48:29,588][105620] Updated weights for policy 1, policy_version 1845275 (0.0009) [2023-12-27 04:48:29,645][105620] Updated weights for policy 1, policy_version 1845285 (0.0009) [2023-12-27 04:48:29,696][105620] Updated weights for policy 1, policy_version 1845295 (0.0009) [2023-12-27 04:48:30,054][105692] Updated weights for policy 0, policy_version 1841114 (0.0009) [2023-12-27 04:48:30,105][105692] Updated weights for policy 0, policy_version 1841124 (0.0009) [2023-12-27 04:48:30,156][105692] Updated weights for policy 0, policy_version 1841134 (0.0009) [2023-12-27 04:48:30,504][105620] Updated weights for policy 1, policy_version 1845305 (0.0009) [2023-12-27 04:48:30,557][105620] Updated weights for policy 1, policy_version 1845316 (0.0009) [2023-12-27 04:48:30,619][105620] Updated weights for policy 1, policy_version 1845326 (0.0009) [2023-12-27 04:48:30,666][105620] Updated weights for policy 1, policy_version 1845336 (0.0009) [2023-12-27 04:48:30,815][105692] Updated weights for policy 0, policy_version 1841144 (0.0008) [2023-12-27 04:48:30,876][105692] Updated weights for policy 0, policy_version 1841154 (0.0009) [2023-12-27 04:48:30,931][105692] Updated weights for policy 0, policy_version 1841164 (0.0009) [2023-12-27 04:48:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 943882240. Throughput: 0: 9567.9, 1: 10018.5. Samples: 943849608. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:48:31,062][104569] Avg episode reward: [(0, '8444.062'), (1, '9349.154')] [2023-12-27 04:48:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001845336_472473600.pth... [2023-12-27 04:48:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001841168_471408640.pth... [2023-12-27 04:48:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001844184_472178688.pth [2023-12-27 04:48:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001840048_471121920.pth [2023-12-27 04:48:31,438][105620] Updated weights for policy 1, policy_version 1845346 (0.0008) [2023-12-27 04:48:31,500][105620] Updated weights for policy 1, policy_version 1845356 (0.0008) [2023-12-27 04:48:31,559][105620] Updated weights for policy 1, policy_version 1845366 (0.0008) [2023-12-27 04:48:31,686][105692] Updated weights for policy 0, policy_version 1841174 (0.0010) [2023-12-27 04:48:31,750][105692] Updated weights for policy 0, policy_version 1841184 (0.0011) [2023-12-27 04:48:31,809][105692] Updated weights for policy 0, policy_version 1841194 (0.0011) [2023-12-27 04:48:32,300][105620] Updated weights for policy 1, policy_version 1845376 (0.0008) [2023-12-27 04:48:32,357][105620] Updated weights for policy 1, policy_version 1845386 (0.0008) [2023-12-27 04:48:32,419][105620] Updated weights for policy 1, policy_version 1845396 (0.0008) [2023-12-27 04:48:32,555][105692] Updated weights for policy 0, policy_version 1841204 (0.0008) [2023-12-27 04:48:32,621][105692] Updated weights for policy 0, policy_version 1841214 (0.0010) [2023-12-27 04:48:32,684][105692] Updated weights for policy 0, policy_version 1841224 (0.0010) [2023-12-27 04:48:33,196][105620] Updated weights for policy 1, policy_version 1845406 (0.0008) [2023-12-27 04:48:33,249][105620] Updated weights for policy 1, policy_version 1845416 (0.0008) [2023-12-27 04:48:33,294][105620] Updated weights for policy 1, policy_version 1845426 (0.0008) [2023-12-27 04:48:33,349][105692] Updated weights for policy 0, policy_version 1841234 (0.0010) [2023-12-27 04:48:33,400][105692] Updated weights for policy 0, policy_version 1841244 (0.0009) [2023-12-27 04:48:33,453][105692] Updated weights for policy 0, policy_version 1841254 (0.0005) [2023-12-27 04:48:33,509][105692] Updated weights for policy 0, policy_version 1841264 (0.0005) [2023-12-27 04:48:34,071][105620] Updated weights for policy 1, policy_version 1845436 (0.0008) [2023-12-27 04:48:34,075][105692] Updated weights for policy 0, policy_version 1841274 (0.0005) [2023-12-27 04:48:34,118][105620] Updated weights for policy 1, policy_version 1845446 (0.0008) [2023-12-27 04:48:34,136][105692] Updated weights for policy 0, policy_version 1841284 (0.0007) [2023-12-27 04:48:34,179][105620] Updated weights for policy 1, policy_version 1845456 (0.0007) [2023-12-27 04:48:34,194][105692] Updated weights for policy 0, policy_version 1841294 (0.0008) [2023-12-27 04:48:34,908][105692] Updated weights for policy 0, policy_version 1841304 (0.0009) [2023-12-27 04:48:34,949][105620] Updated weights for policy 1, policy_version 1845466 (0.0008) [2023-12-27 04:48:34,970][105692] Updated weights for policy 0, policy_version 1841314 (0.0008) [2023-12-27 04:48:35,007][105620] Updated weights for policy 1, policy_version 1845476 (0.0008) [2023-12-27 04:48:35,031][105692] Updated weights for policy 0, policy_version 1841324 (0.0006) [2023-12-27 04:48:35,061][105620] Updated weights for policy 1, policy_version 1845486 (0.0007) [2023-12-27 04:48:35,110][105620] Updated weights for policy 1, policy_version 1845496 (0.0009) [2023-12-27 04:48:35,673][105692] Updated weights for policy 0, policy_version 1841334 (0.0008) [2023-12-27 04:48:35,734][105692] Updated weights for policy 0, policy_version 1841344 (0.0008) [2023-12-27 04:48:35,795][105692] Updated weights for policy 0, policy_version 1841354 (0.0009) [2023-12-27 04:48:35,908][105620] Updated weights for policy 1, policy_version 1845506 (0.0008) [2023-12-27 04:48:35,962][105620] Updated weights for policy 1, policy_version 1845516 (0.0009) [2023-12-27 04:48:36,009][105620] Updated weights for policy 1, policy_version 1845526 (0.0009) [2023-12-27 04:48:36,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 943980544. Throughput: 0: 9621.7, 1: 9993.6. Samples: 943964876. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:48:36,062][104569] Avg episode reward: [(0, '8446.116'), (1, '9349.204')] [2023-12-27 04:48:36,520][105692] Updated weights for policy 0, policy_version 1841364 (0.0010) [2023-12-27 04:48:36,576][105692] Updated weights for policy 0, policy_version 1841374 (0.0009) [2023-12-27 04:48:36,632][105692] Updated weights for policy 0, policy_version 1841384 (0.0009) [2023-12-27 04:48:36,802][105620] Updated weights for policy 1, policy_version 1845536 (0.0009) [2023-12-27 04:48:36,856][105620] Updated weights for policy 1, policy_version 1845546 (0.0009) [2023-12-27 04:48:36,911][105620] Updated weights for policy 1, policy_version 1845556 (0.0009) [2023-12-27 04:48:37,387][105692] Updated weights for policy 0, policy_version 1841394 (0.0009) [2023-12-27 04:48:37,440][105692] Updated weights for policy 0, policy_version 1841404 (0.0007) [2023-12-27 04:48:37,492][105692] Updated weights for policy 0, policy_version 1841414 (0.0009) [2023-12-27 04:48:37,544][105692] Updated weights for policy 0, policy_version 1841424 (0.0009) [2023-12-27 04:48:37,705][105620] Updated weights for policy 1, policy_version 1845566 (0.0009) [2023-12-27 04:48:37,766][105620] Updated weights for policy 1, policy_version 1845576 (0.0010) [2023-12-27 04:48:37,829][105620] Updated weights for policy 1, policy_version 1845586 (0.0009) [2023-12-27 04:48:38,269][105692] Updated weights for policy 0, policy_version 1841434 (0.0009) [2023-12-27 04:48:38,324][105692] Updated weights for policy 0, policy_version 1841444 (0.0009) [2023-12-27 04:48:38,390][105692] Updated weights for policy 0, policy_version 1841454 (0.0009) [2023-12-27 04:48:38,623][105620] Updated weights for policy 1, policy_version 1845596 (0.0009) [2023-12-27 04:48:38,671][105620] Updated weights for policy 1, policy_version 1845606 (0.0009) [2023-12-27 04:48:38,719][105620] Updated weights for policy 1, policy_version 1845616 (0.0008) [2023-12-27 04:48:39,163][105692] Updated weights for policy 0, policy_version 1841464 (0.0006) [2023-12-27 04:48:39,219][105692] Updated weights for policy 0, policy_version 1841474 (0.0006) [2023-12-27 04:48:39,284][105692] Updated weights for policy 0, policy_version 1841484 (0.0007) [2023-12-27 04:48:39,455][105620] Updated weights for policy 1, policy_version 1845626 (0.0009) [2023-12-27 04:48:39,517][105620] Updated weights for policy 1, policy_version 1845636 (0.0010) [2023-12-27 04:48:39,564][105620] Updated weights for policy 1, policy_version 1845646 (0.0009) [2023-12-27 04:48:39,623][105620] Updated weights for policy 1, policy_version 1845656 (0.0009) [2023-12-27 04:48:39,909][105692] Updated weights for policy 0, policy_version 1841494 (0.0009) [2023-12-27 04:48:39,976][105692] Updated weights for policy 0, policy_version 1841504 (0.0009) [2023-12-27 04:48:40,038][105692] Updated weights for policy 0, policy_version 1841514 (0.0009) [2023-12-27 04:48:40,467][105620] Updated weights for policy 1, policy_version 1845666 (0.0009) [2023-12-27 04:48:40,524][105620] Updated weights for policy 1, policy_version 1845676 (0.0008) [2023-12-27 04:48:40,585][105620] Updated weights for policy 1, policy_version 1845686 (0.0010) [2023-12-27 04:48:40,723][105692] Updated weights for policy 0, policy_version 1841524 (0.0010) [2023-12-27 04:48:40,782][105692] Updated weights for policy 0, policy_version 1841534 (0.0009) [2023-12-27 04:48:40,843][105692] Updated weights for policy 0, policy_version 1841544 (0.0008) [2023-12-27 04:48:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 944070656. Throughput: 0: 9604.6, 1: 9788.7. Samples: 944077160. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:48:41,062][104569] Avg episode reward: [(0, '8258.550'), (1, '9349.207')] [2023-12-27 04:48:41,381][105620] Updated weights for policy 1, policy_version 1845696 (0.0010) [2023-12-27 04:48:41,448][105620] Updated weights for policy 1, policy_version 1845706 (0.0009) [2023-12-27 04:48:41,513][105620] Updated weights for policy 1, policy_version 1845716 (0.0008) [2023-12-27 04:48:41,584][105692] Updated weights for policy 0, policy_version 1841554 (0.0009) [2023-12-27 04:48:41,647][105692] Updated weights for policy 0, policy_version 1841564 (0.0009) [2023-12-27 04:48:41,713][105692] Updated weights for policy 0, policy_version 1841574 (0.0009) [2023-12-27 04:48:41,779][105692] Updated weights for policy 0, policy_version 1841584 (0.0009) [2023-12-27 04:48:42,223][105620] Updated weights for policy 1, policy_version 1845726 (0.0008) [2023-12-27 04:48:42,288][105620] Updated weights for policy 1, policy_version 1845736 (0.0009) [2023-12-27 04:48:42,350][105620] Updated weights for policy 1, policy_version 1845746 (0.0009) [2023-12-27 04:48:42,463][105692] Updated weights for policy 0, policy_version 1841594 (0.0008) [2023-12-27 04:48:42,528][105692] Updated weights for policy 0, policy_version 1841604 (0.0009) [2023-12-27 04:48:42,590][105692] Updated weights for policy 0, policy_version 1841614 (0.0009) [2023-12-27 04:48:43,131][105620] Updated weights for policy 1, policy_version 1845756 (0.0008) [2023-12-27 04:48:43,187][105620] Updated weights for policy 1, policy_version 1845766 (0.0009) [2023-12-27 04:48:43,245][105620] Updated weights for policy 1, policy_version 1845776 (0.0009) [2023-12-27 04:48:43,295][105692] Updated weights for policy 0, policy_version 1841624 (0.0006) [2023-12-27 04:48:43,342][105692] Updated weights for policy 0, policy_version 1841634 (0.0006) [2023-12-27 04:48:43,401][105692] Updated weights for policy 0, policy_version 1841644 (0.0009) [2023-12-27 04:48:43,973][105620] Updated weights for policy 1, policy_version 1845786 (0.0009) [2023-12-27 04:48:44,030][105620] Updated weights for policy 1, policy_version 1845796 (0.0009) [2023-12-27 04:48:44,081][105620] Updated weights for policy 1, policy_version 1845806 (0.0009) [2023-12-27 04:48:44,117][105692] Updated weights for policy 0, policy_version 1841654 (0.0008) [2023-12-27 04:48:44,127][105620] Updated weights for policy 1, policy_version 1845816 (0.0007) [2023-12-27 04:48:44,177][105692] Updated weights for policy 0, policy_version 1841664 (0.0008) [2023-12-27 04:48:44,234][105692] Updated weights for policy 0, policy_version 1841674 (0.0009) [2023-12-27 04:48:44,892][105620] Updated weights for policy 1, policy_version 1845826 (0.0010) [2023-12-27 04:48:44,939][105692] Updated weights for policy 0, policy_version 1841684 (0.0008) [2023-12-27 04:48:44,954][105620] Updated weights for policy 1, policy_version 1845836 (0.0008) [2023-12-27 04:48:44,999][105692] Updated weights for policy 0, policy_version 1841694 (0.0006) [2023-12-27 04:48:45,018][105620] Updated weights for policy 1, policy_version 1845846 (0.0008) [2023-12-27 04:48:45,056][105692] Updated weights for policy 0, policy_version 1841704 (0.0011) [2023-12-27 04:48:45,776][105620] Updated weights for policy 1, policy_version 1845856 (0.0007) [2023-12-27 04:48:45,783][105692] Updated weights for policy 0, policy_version 1841714 (0.0010) [2023-12-27 04:48:45,831][105692] Updated weights for policy 0, policy_version 1841724 (0.0006) [2023-12-27 04:48:45,834][105620] Updated weights for policy 1, policy_version 1845866 (0.0007) [2023-12-27 04:48:45,887][105692] Updated weights for policy 0, policy_version 1841734 (0.0007) [2023-12-27 04:48:45,893][105620] Updated weights for policy 1, policy_version 1845876 (0.0007) [2023-12-27 04:48:45,943][105692] Updated weights for policy 0, policy_version 1841744 (0.0008) [2023-12-27 04:48:46,062][104569] Fps is (10 sec: 18840.5, 60 sec: 19524.1, 300 sec: 19383.1). Total num frames: 944168960. Throughput: 0: 9531.3, 1: 9738.8. Samples: 944134068. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:48:46,063][104569] Avg episode reward: [(0, '8258.718'), (1, '9349.190')] [2023-12-27 04:48:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001841744_471556096.pth... [2023-12-27 04:48:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001845880_472612864.pth... [2023-12-27 04:48:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001840592_471261184.pth [2023-12-27 04:48:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001844792_472334336.pth [2023-12-27 04:48:46,641][105692] Updated weights for policy 0, policy_version 1841754 (0.0006) [2023-12-27 04:48:46,687][105692] Updated weights for policy 0, policy_version 1841764 (0.0008) [2023-12-27 04:48:46,705][105620] Updated weights for policy 1, policy_version 1845886 (0.0007) [2023-12-27 04:48:46,736][105692] Updated weights for policy 0, policy_version 1841774 (0.0007) [2023-12-27 04:48:46,764][105620] Updated weights for policy 1, policy_version 1845896 (0.0009) [2023-12-27 04:48:46,810][105620] Updated weights for policy 1, policy_version 1845906 (0.0009) [2023-12-27 04:48:47,494][105620] Updated weights for policy 1, policy_version 1845916 (0.0009) [2023-12-27 04:48:47,509][105692] Updated weights for policy 0, policy_version 1841784 (0.0008) [2023-12-27 04:48:47,553][105620] Updated weights for policy 1, policy_version 1845926 (0.0008) [2023-12-27 04:48:47,559][105692] Updated weights for policy 0, policy_version 1841794 (0.0008) [2023-12-27 04:48:47,610][105620] Updated weights for policy 1, policy_version 1845936 (0.0007) [2023-12-27 04:48:47,620][105692] Updated weights for policy 0, policy_version 1841804 (0.0006) [2023-12-27 04:48:48,326][105620] Updated weights for policy 1, policy_version 1845946 (0.0008) [2023-12-27 04:48:48,386][105620] Updated weights for policy 1, policy_version 1845956 (0.0009) [2023-12-27 04:48:48,418][105692] Updated weights for policy 0, policy_version 1841814 (0.0008) [2023-12-27 04:48:48,460][105620] Updated weights for policy 1, policy_version 1845966 (0.0006) [2023-12-27 04:48:48,470][105692] Updated weights for policy 0, policy_version 1841824 (0.0007) [2023-12-27 04:48:48,525][105692] Updated weights for policy 0, policy_version 1841834 (0.0008) [2023-12-27 04:48:48,525][105620] Updated weights for policy 1, policy_version 1845976 (0.0008) [2023-12-27 04:48:49,240][105620] Updated weights for policy 1, policy_version 1845986 (0.0009) [2023-12-27 04:48:49,299][105620] Updated weights for policy 1, policy_version 1845996 (0.0008) [2023-12-27 04:48:49,313][105692] Updated weights for policy 0, policy_version 1841844 (0.0008) [2023-12-27 04:48:49,367][105620] Updated weights for policy 1, policy_version 1846006 (0.0007) [2023-12-27 04:48:49,385][105692] Updated weights for policy 0, policy_version 1841854 (0.0008) [2023-12-27 04:48:49,456][105692] Updated weights for policy 0, policy_version 1841864 (0.0010) [2023-12-27 04:48:50,043][105620] Updated weights for policy 1, policy_version 1846016 (0.0008) [2023-12-27 04:48:50,098][105620] Updated weights for policy 1, policy_version 1846026 (0.0009) [2023-12-27 04:48:50,165][105620] Updated weights for policy 1, policy_version 1846036 (0.0009) [2023-12-27 04:48:50,262][105692] Updated weights for policy 0, policy_version 1841874 (0.0010) [2023-12-27 04:48:50,327][105692] Updated weights for policy 0, policy_version 1841884 (0.0009) [2023-12-27 04:48:50,389][105692] Updated weights for policy 0, policy_version 1841894 (0.0009) [2023-12-27 04:48:50,433][105692] Updated weights for policy 0, policy_version 1841904 (0.0010) [2023-12-27 04:48:50,977][105620] Updated weights for policy 1, policy_version 1846046 (0.0009) [2023-12-27 04:48:51,040][105620] Updated weights for policy 1, policy_version 1846056 (0.0009) [2023-12-27 04:48:51,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19251.2, 300 sec: 19355.3). Total num frames: 944250880. Throughput: 0: 9584.3, 1: 9609.4. Samples: 944247904. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:48:51,063][104569] Avg episode reward: [(0, '8264.500'), (1, '9164.380')] [2023-12-27 04:48:51,097][105692] Updated weights for policy 0, policy_version 1841914 (0.0011) [2023-12-27 04:48:51,103][105620] Updated weights for policy 1, policy_version 1846066 (0.0006) [2023-12-27 04:48:51,161][105692] Updated weights for policy 0, policy_version 1841924 (0.0009) [2023-12-27 04:48:51,224][105692] Updated weights for policy 0, policy_version 1841934 (0.0010) [2023-12-27 04:48:51,886][105620] Updated weights for policy 1, policy_version 1846076 (0.0007) [2023-12-27 04:48:51,924][105692] Updated weights for policy 0, policy_version 1841944 (0.0009) [2023-12-27 04:48:51,944][105620] Updated weights for policy 1, policy_version 1846086 (0.0009) [2023-12-27 04:48:51,986][105692] Updated weights for policy 0, policy_version 1841954 (0.0010) [2023-12-27 04:48:51,996][105620] Updated weights for policy 1, policy_version 1846096 (0.0006) [2023-12-27 04:48:52,043][105692] Updated weights for policy 0, policy_version 1841964 (0.0010) [2023-12-27 04:48:52,767][105692] Updated weights for policy 0, policy_version 1841974 (0.0010) [2023-12-27 04:48:52,790][105620] Updated weights for policy 1, policy_version 1846106 (0.0007) [2023-12-27 04:48:52,822][105692] Updated weights for policy 0, policy_version 1841984 (0.0005) [2023-12-27 04:48:52,846][105620] Updated weights for policy 1, policy_version 1846116 (0.0009) [2023-12-27 04:48:52,876][105692] Updated weights for policy 0, policy_version 1841994 (0.0005) [2023-12-27 04:48:52,900][105620] Updated weights for policy 1, policy_version 1846126 (0.0008) [2023-12-27 04:48:52,949][105620] Updated weights for policy 1, policy_version 1846136 (0.0008) [2023-12-27 04:48:53,624][105692] Updated weights for policy 0, policy_version 1842004 (0.0007) [2023-12-27 04:48:53,630][105620] Updated weights for policy 1, policy_version 1846146 (0.0008) [2023-12-27 04:48:53,667][105692] Updated weights for policy 0, policy_version 1842014 (0.0007) [2023-12-27 04:48:53,694][105620] Updated weights for policy 1, policy_version 1846156 (0.0009) [2023-12-27 04:48:53,714][105692] Updated weights for policy 0, policy_version 1842024 (0.0006) [2023-12-27 04:48:53,751][105620] Updated weights for policy 1, policy_version 1846166 (0.0008) [2023-12-27 04:48:54,428][105692] Updated weights for policy 0, policy_version 1842034 (0.0008) [2023-12-27 04:48:54,486][105692] Updated weights for policy 0, policy_version 1842044 (0.0009) [2023-12-27 04:48:54,529][105620] Updated weights for policy 1, policy_version 1846176 (0.0007) [2023-12-27 04:48:54,539][105692] Updated weights for policy 0, policy_version 1842054 (0.0007) [2023-12-27 04:48:54,582][105620] Updated weights for policy 1, policy_version 1846186 (0.0006) [2023-12-27 04:48:54,592][105692] Updated weights for policy 0, policy_version 1842064 (0.0006) [2023-12-27 04:48:54,638][105620] Updated weights for policy 1, policy_version 1846196 (0.0008) [2023-12-27 04:48:55,339][105692] Updated weights for policy 0, policy_version 1842074 (0.0009) [2023-12-27 04:48:55,391][105692] Updated weights for policy 0, policy_version 1842084 (0.0010) [2023-12-27 04:48:55,398][105620] Updated weights for policy 1, policy_version 1846206 (0.0009) [2023-12-27 04:48:55,447][105692] Updated weights for policy 0, policy_version 1842094 (0.0006) [2023-12-27 04:48:55,461][105620] Updated weights for policy 1, policy_version 1846216 (0.0008) [2023-12-27 04:48:55,518][105620] Updated weights for policy 1, policy_version 1846226 (0.0008) [2023-12-27 04:48:56,062][104569] Fps is (10 sec: 18023.4, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 944349184. Throughput: 0: 9697.4, 1: 9422.5. Samples: 944360636. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:48:56,062][104569] Avg episode reward: [(0, '8356.883'), (1, '9071.926')] [2023-12-27 04:48:56,209][105692] Updated weights for policy 0, policy_version 1842104 (0.0008) [2023-12-27 04:48:56,258][105692] Updated weights for policy 0, policy_version 1842114 (0.0007) [2023-12-27 04:48:56,260][105620] Updated weights for policy 1, policy_version 1846236 (0.0008) [2023-12-27 04:48:56,308][105692] Updated weights for policy 0, policy_version 1842124 (0.0007) [2023-12-27 04:48:56,324][105620] Updated weights for policy 1, policy_version 1846246 (0.0008) [2023-12-27 04:48:56,380][105620] Updated weights for policy 1, policy_version 1846256 (0.0008) [2023-12-27 04:48:57,080][105692] Updated weights for policy 0, policy_version 1842134 (0.0009) [2023-12-27 04:48:57,090][105620] Updated weights for policy 1, policy_version 1846266 (0.0008) [2023-12-27 04:48:57,133][105692] Updated weights for policy 0, policy_version 1842144 (0.0010) [2023-12-27 04:48:57,153][105620] Updated weights for policy 1, policy_version 1846276 (0.0005) [2023-12-27 04:48:57,186][105692] Updated weights for policy 0, policy_version 1842154 (0.0008) [2023-12-27 04:48:57,212][105620] Updated weights for policy 1, policy_version 1846286 (0.0007) [2023-12-27 04:48:57,268][105620] Updated weights for policy 1, policy_version 1846296 (0.0009) [2023-12-27 04:48:57,955][105692] Updated weights for policy 0, policy_version 1842164 (0.0007) [2023-12-27 04:48:57,972][105620] Updated weights for policy 1, policy_version 1846306 (0.0009) [2023-12-27 04:48:58,009][105692] Updated weights for policy 0, policy_version 1842174 (0.0006) [2023-12-27 04:48:58,024][105620] Updated weights for policy 1, policy_version 1846316 (0.0008) [2023-12-27 04:48:58,057][105692] Updated weights for policy 0, policy_version 1842184 (0.0006) [2023-12-27 04:48:58,080][105620] Updated weights for policy 1, policy_version 1846326 (0.0009) [2023-12-27 04:48:58,755][105692] Updated weights for policy 0, policy_version 1842194 (0.0008) [2023-12-27 04:48:58,827][105692] Updated weights for policy 0, policy_version 1842204 (0.0008) [2023-12-27 04:48:58,897][105692] Updated weights for policy 0, policy_version 1842214 (0.0010) [2023-12-27 04:48:58,966][105692] Updated weights for policy 0, policy_version 1842224 (0.0008) [2023-12-27 04:48:59,007][105620] Updated weights for policy 1, policy_version 1846336 (0.0010) [2023-12-27 04:48:59,074][105620] Updated weights for policy 1, policy_version 1846346 (0.0010) [2023-12-27 04:48:59,144][105620] Updated weights for policy 1, policy_version 1846356 (0.0009) [2023-12-27 04:48:59,781][105692] Updated weights for policy 0, policy_version 1842234 (0.0010) [2023-12-27 04:48:59,837][105692] Updated weights for policy 0, policy_version 1842244 (0.0009) [2023-12-27 04:48:59,904][105692] Updated weights for policy 0, policy_version 1842254 (0.0011) [2023-12-27 04:48:59,951][105620] Updated weights for policy 1, policy_version 1846366 (0.0008) [2023-12-27 04:48:59,998][105620] Updated weights for policy 1, policy_version 1846376 (0.0005) [2023-12-27 04:49:00,054][105620] Updated weights for policy 1, policy_version 1846386 (0.0007) [2023-12-27 04:49:00,626][105692] Updated weights for policy 0, policy_version 1842264 (0.0008) [2023-12-27 04:49:00,689][105692] Updated weights for policy 0, policy_version 1842274 (0.0007) [2023-12-27 04:49:00,738][105692] Updated weights for policy 0, policy_version 1842284 (0.0008) [2023-12-27 04:49:00,775][105620] Updated weights for policy 1, policy_version 1846396 (0.0008) [2023-12-27 04:49:00,833][105620] Updated weights for policy 1, policy_version 1846406 (0.0009) [2023-12-27 04:49:00,888][105620] Updated weights for policy 1, policy_version 1846416 (0.0009) [2023-12-27 04:49:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 944447488. Throughput: 0: 9649.9, 1: 9433.6. Samples: 944416616. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:49:01,062][104569] Avg episode reward: [(0, '8168.715'), (1, '9071.796')] [2023-12-27 04:49:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001842288_471695360.pth... [2023-12-27 04:49:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001846424_472752128.pth... [2023-12-27 04:49:01,069][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001845336_472473600.pth [2023-12-27 04:49:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001841168_471408640.pth [2023-12-27 04:49:01,407][105692] Updated weights for policy 0, policy_version 1842294 (0.0008) [2023-12-27 04:49:01,471][105692] Updated weights for policy 0, policy_version 1842304 (0.0005) [2023-12-27 04:49:01,521][105692] Updated weights for policy 0, policy_version 1842314 (0.0005) [2023-12-27 04:49:01,660][105620] Updated weights for policy 1, policy_version 1846426 (0.0008) [2023-12-27 04:49:01,726][105620] Updated weights for policy 1, policy_version 1846436 (0.0009) [2023-12-27 04:49:01,782][105620] Updated weights for policy 1, policy_version 1846446 (0.0006) [2023-12-27 04:49:01,838][105620] Updated weights for policy 1, policy_version 1846456 (0.0007) [2023-12-27 04:49:02,168][105692] Updated weights for policy 0, policy_version 1842324 (0.0007) [2023-12-27 04:49:02,226][105692] Updated weights for policy 0, policy_version 1842334 (0.0005) [2023-12-27 04:49:02,292][105692] Updated weights for policy 0, policy_version 1842344 (0.0006) [2023-12-27 04:49:02,568][105620] Updated weights for policy 1, policy_version 1846466 (0.0005) [2023-12-27 04:49:02,626][105620] Updated weights for policy 1, policy_version 1846476 (0.0005) [2023-12-27 04:49:02,682][105620] Updated weights for policy 1, policy_version 1846486 (0.0005) [2023-12-27 04:49:03,090][105692] Updated weights for policy 0, policy_version 1842354 (0.0006) [2023-12-27 04:49:03,139][105692] Updated weights for policy 0, policy_version 1842364 (0.0008) [2023-12-27 04:49:03,168][105620] Updated weights for policy 1, policy_version 1846496 (0.0006) [2023-12-27 04:49:03,188][105692] Updated weights for policy 0, policy_version 1842374 (0.0010) [2023-12-27 04:49:03,222][105620] Updated weights for policy 1, policy_version 1846506 (0.0005) [2023-12-27 04:49:03,236][105692] Updated weights for policy 0, policy_version 1842384 (0.0010) [2023-12-27 04:49:03,276][105620] Updated weights for policy 1, policy_version 1846516 (0.0007) [2023-12-27 04:49:03,911][105692] Updated weights for policy 0, policy_version 1842394 (0.0007) [2023-12-27 04:49:03,928][105620] Updated weights for policy 1, policy_version 1846526 (0.0009) [2023-12-27 04:49:03,969][105692] Updated weights for policy 0, policy_version 1842404 (0.0008) [2023-12-27 04:49:03,982][105620] Updated weights for policy 1, policy_version 1846536 (0.0011) [2023-12-27 04:49:04,027][105692] Updated weights for policy 0, policy_version 1842414 (0.0005) [2023-12-27 04:49:04,038][105620] Updated weights for policy 1, policy_version 1846546 (0.0011) [2023-12-27 04:49:04,674][105692] Updated weights for policy 0, policy_version 1842424 (0.0009) [2023-12-27 04:49:04,730][105692] Updated weights for policy 0, policy_version 1842434 (0.0010) [2023-12-27 04:49:04,789][105692] Updated weights for policy 0, policy_version 1842444 (0.0011) [2023-12-27 04:49:04,805][105620] Updated weights for policy 1, policy_version 1846556 (0.0010) [2023-12-27 04:49:04,858][105620] Updated weights for policy 1, policy_version 1846566 (0.0011) [2023-12-27 04:49:04,924][105620] Updated weights for policy 1, policy_version 1846576 (0.0009) [2023-12-27 04:49:05,525][105692] Updated weights for policy 0, policy_version 1842454 (0.0007) [2023-12-27 04:49:05,560][105620] Updated weights for policy 1, policy_version 1846586 (0.0010) [2023-12-27 04:49:05,583][105692] Updated weights for policy 0, policy_version 1842464 (0.0005) [2023-12-27 04:49:05,622][105620] Updated weights for policy 1, policy_version 1846596 (0.0007) [2023-12-27 04:49:05,639][105692] Updated weights for policy 0, policy_version 1842474 (0.0007) [2023-12-27 04:49:05,678][105620] Updated weights for policy 1, policy_version 1846606 (0.0008) [2023-12-27 04:49:05,730][105620] Updated weights for policy 1, policy_version 1846616 (0.0010) [2023-12-27 04:49:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 944545792. Throughput: 0: 9616.7, 1: 9421.2. Samples: 944534392. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:49:06,063][104569] Avg episode reward: [(0, '8167.305'), (1, '9164.505')] [2023-12-27 04:49:06,298][105692] Updated weights for policy 0, policy_version 1842484 (0.0009) [2023-12-27 04:49:06,360][105692] Updated weights for policy 0, policy_version 1842494 (0.0006) [2023-12-27 04:49:06,411][105620] Updated weights for policy 1, policy_version 1846626 (0.0011) [2023-12-27 04:49:06,423][105692] Updated weights for policy 0, policy_version 1842504 (0.0006) [2023-12-27 04:49:06,476][105620] Updated weights for policy 1, policy_version 1846636 (0.0011) [2023-12-27 04:49:06,543][105620] Updated weights for policy 1, policy_version 1846646 (0.0011) [2023-12-27 04:49:07,036][105692] Updated weights for policy 0, policy_version 1842514 (0.0007) [2023-12-27 04:49:07,097][105692] Updated weights for policy 0, policy_version 1842524 (0.0010) [2023-12-27 04:49:07,153][105692] Updated weights for policy 0, policy_version 1842534 (0.0010) [2023-12-27 04:49:07,212][105692] Updated weights for policy 0, policy_version 1842544 (0.0011) [2023-12-27 04:49:07,279][105620] Updated weights for policy 1, policy_version 1846656 (0.0010) [2023-12-27 04:49:07,335][105620] Updated weights for policy 1, policy_version 1846666 (0.0010) [2023-12-27 04:49:07,384][105620] Updated weights for policy 1, policy_version 1846676 (0.0010) [2023-12-27 04:49:07,949][105692] Updated weights for policy 0, policy_version 1842554 (0.0009) [2023-12-27 04:49:08,009][105692] Updated weights for policy 0, policy_version 1842565 (0.0011) [2023-12-27 04:49:08,074][105692] Updated weights for policy 0, policy_version 1842575 (0.0008) [2023-12-27 04:49:08,097][105620] Updated weights for policy 1, policy_version 1846686 (0.0007) [2023-12-27 04:49:08,153][105620] Updated weights for policy 1, policy_version 1846696 (0.0006) [2023-12-27 04:49:08,214][105620] Updated weights for policy 1, policy_version 1846706 (0.0006) [2023-12-27 04:49:08,741][105692] Updated weights for policy 0, policy_version 1842585 (0.0010) [2023-12-27 04:49:08,782][105620] Updated weights for policy 1, policy_version 1846716 (0.0006) [2023-12-27 04:49:08,803][105692] Updated weights for policy 0, policy_version 1842595 (0.0011) [2023-12-27 04:49:08,851][105620] Updated weights for policy 1, policy_version 1846726 (0.0006) [2023-12-27 04:49:08,867][105692] Updated weights for policy 0, policy_version 1842605 (0.0010) [2023-12-27 04:49:08,908][105620] Updated weights for policy 1, policy_version 1846736 (0.0006) [2023-12-27 04:49:09,630][105620] Updated weights for policy 1, policy_version 1846746 (0.0006) [2023-12-27 04:49:09,634][105692] Updated weights for policy 0, policy_version 1842615 (0.0007) [2023-12-27 04:49:09,693][105620] Updated weights for policy 1, policy_version 1846756 (0.0009) [2023-12-27 04:49:09,699][105692] Updated weights for policy 0, policy_version 1842625 (0.0006) [2023-12-27 04:49:09,755][105620] Updated weights for policy 1, policy_version 1846766 (0.0008) [2023-12-27 04:49:09,765][105692] Updated weights for policy 0, policy_version 1842635 (0.0007) [2023-12-27 04:49:09,816][105620] Updated weights for policy 1, policy_version 1846776 (0.0009) [2023-12-27 04:49:10,521][105620] Updated weights for policy 1, policy_version 1846786 (0.0008) [2023-12-27 04:49:10,536][105692] Updated weights for policy 0, policy_version 1842645 (0.0006) [2023-12-27 04:49:10,578][105620] Updated weights for policy 1, policy_version 1846796 (0.0009) [2023-12-27 04:49:10,601][105692] Updated weights for policy 0, policy_version 1842655 (0.0008) [2023-12-27 04:49:10,632][105620] Updated weights for policy 1, policy_version 1846806 (0.0007) [2023-12-27 04:49:10,664][105692] Updated weights for policy 0, policy_version 1842665 (0.0007) [2023-12-27 04:49:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 944644096. Throughput: 0: 9695.8, 1: 9474.2. Samples: 944653384. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:49:11,063][104569] Avg episode reward: [(0, '8630.465'), (1, '9256.956')] [2023-12-27 04:49:11,322][105620] Updated weights for policy 1, policy_version 1846816 (0.0006) [2023-12-27 04:49:11,395][105620] Updated weights for policy 1, policy_version 1846826 (0.0008) [2023-12-27 04:49:11,451][105620] Updated weights for policy 1, policy_version 1846836 (0.0008) [2023-12-27 04:49:11,474][105692] Updated weights for policy 0, policy_version 1842675 (0.0009) [2023-12-27 04:49:11,534][105692] Updated weights for policy 0, policy_version 1842685 (0.0008) [2023-12-27 04:49:11,594][105692] Updated weights for policy 0, policy_version 1842695 (0.0008) [2023-12-27 04:49:12,244][105620] Updated weights for policy 1, policy_version 1846846 (0.0009) [2023-12-27 04:49:12,301][105620] Updated weights for policy 1, policy_version 1846856 (0.0011) [2023-12-27 04:49:12,327][105692] Updated weights for policy 0, policy_version 1842705 (0.0008) [2023-12-27 04:49:12,366][105620] Updated weights for policy 1, policy_version 1846866 (0.0009) [2023-12-27 04:49:12,390][105692] Updated weights for policy 0, policy_version 1842715 (0.0008) [2023-12-27 04:49:12,449][105692] Updated weights for policy 0, policy_version 1842725 (0.0008) [2023-12-27 04:49:12,501][105692] Updated weights for policy 0, policy_version 1842735 (0.0008) [2023-12-27 04:49:13,124][105620] Updated weights for policy 1, policy_version 1846876 (0.0010) [2023-12-27 04:49:13,180][105620] Updated weights for policy 1, policy_version 1846886 (0.0010) [2023-12-27 04:49:13,236][105620] Updated weights for policy 1, policy_version 1846896 (0.0011) [2023-12-27 04:49:13,270][105692] Updated weights for policy 0, policy_version 1842745 (0.0006) [2023-12-27 04:49:13,324][105692] Updated weights for policy 0, policy_version 1842755 (0.0008) [2023-12-27 04:49:13,373][105692] Updated weights for policy 0, policy_version 1842765 (0.0008) [2023-12-27 04:49:13,997][105620] Updated weights for policy 1, policy_version 1846906 (0.0011) [2023-12-27 04:49:14,052][105620] Updated weights for policy 1, policy_version 1846916 (0.0010) [2023-12-27 04:49:14,101][105620] Updated weights for policy 1, policy_version 1846926 (0.0010) [2023-12-27 04:49:14,134][105692] Updated weights for policy 0, policy_version 1842775 (0.0006) [2023-12-27 04:49:14,164][105620] Updated weights for policy 1, policy_version 1846936 (0.0010) [2023-12-27 04:49:14,192][105692] Updated weights for policy 0, policy_version 1842785 (0.0005) [2023-12-27 04:49:14,260][105692] Updated weights for policy 0, policy_version 1842795 (0.0005) [2023-12-27 04:49:14,830][105620] Updated weights for policy 1, policy_version 1846946 (0.0008) [2023-12-27 04:49:14,868][105692] Updated weights for policy 0, policy_version 1842805 (0.0007) [2023-12-27 04:49:14,885][105620] Updated weights for policy 1, policy_version 1846956 (0.0008) [2023-12-27 04:49:14,934][105692] Updated weights for policy 0, policy_version 1842815 (0.0008) [2023-12-27 04:49:14,937][105620] Updated weights for policy 1, policy_version 1846966 (0.0008) [2023-12-27 04:49:14,997][105692] Updated weights for policy 0, policy_version 1842825 (0.0011) [2023-12-27 04:49:15,652][105620] Updated weights for policy 1, policy_version 1846976 (0.0010) [2023-12-27 04:49:15,708][105620] Updated weights for policy 1, policy_version 1846986 (0.0010) [2023-12-27 04:49:15,748][105692] Updated weights for policy 0, policy_version 1842835 (0.0009) [2023-12-27 04:49:15,756][105620] Updated weights for policy 1, policy_version 1846996 (0.0010) [2023-12-27 04:49:15,807][105692] Updated weights for policy 0, policy_version 1842845 (0.0010) [2023-12-27 04:49:15,855][105692] Updated weights for policy 0, policy_version 1842855 (0.0010) [2023-12-27 04:49:16,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 944742400. Throughput: 0: 9673.7, 1: 9411.4. Samples: 944708436. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:49:16,062][104569] Avg episode reward: [(0, '8811.540'), (1, '9257.004')] [2023-12-27 04:49:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001842864_471842816.pth... [2023-12-27 04:49:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001847000_472899584.pth... [2023-12-27 04:49:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001841744_471556096.pth [2023-12-27 04:49:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001845880_472612864.pth [2023-12-27 04:49:16,383][105620] Updated weights for policy 1, policy_version 1847006 (0.0009) [2023-12-27 04:49:16,440][105620] Updated weights for policy 1, policy_version 1847016 (0.0009) [2023-12-27 04:49:16,503][105620] Updated weights for policy 1, policy_version 1847026 (0.0005) [2023-12-27 04:49:16,600][105692] Updated weights for policy 0, policy_version 1842865 (0.0010) [2023-12-27 04:49:16,651][105692] Updated weights for policy 0, policy_version 1842875 (0.0005) [2023-12-27 04:49:16,703][105692] Updated weights for policy 0, policy_version 1842885 (0.0005) [2023-12-27 04:49:16,752][105692] Updated weights for policy 0, policy_version 1842895 (0.0005) [2023-12-27 04:49:17,110][105620] Updated weights for policy 1, policy_version 1847036 (0.0005) [2023-12-27 04:49:17,159][105620] Updated weights for policy 1, policy_version 1847046 (0.0005) [2023-12-27 04:49:17,208][105620] Updated weights for policy 1, policy_version 1847056 (0.0006) [2023-12-27 04:49:17,372][105692] Updated weights for policy 0, policy_version 1842905 (0.0005) [2023-12-27 04:49:17,428][105692] Updated weights for policy 0, policy_version 1842915 (0.0006) [2023-12-27 04:49:17,483][105692] Updated weights for policy 0, policy_version 1842925 (0.0006) [2023-12-27 04:49:18,036][105620] Updated weights for policy 1, policy_version 1847066 (0.0010) [2023-12-27 04:49:18,072][105692] Updated weights for policy 0, policy_version 1842935 (0.0005) [2023-12-27 04:49:18,090][105620] Updated weights for policy 1, policy_version 1847076 (0.0007) [2023-12-27 04:49:18,132][105692] Updated weights for policy 0, policy_version 1842945 (0.0007) [2023-12-27 04:49:18,150][105620] Updated weights for policy 1, policy_version 1847086 (0.0008) [2023-12-27 04:49:18,185][105692] Updated weights for policy 0, policy_version 1842955 (0.0008) [2023-12-27 04:49:18,196][105620] Updated weights for policy 1, policy_version 1847096 (0.0006) [2023-12-27 04:49:18,962][105692] Updated weights for policy 0, policy_version 1842965 (0.0007) [2023-12-27 04:49:18,976][105620] Updated weights for policy 1, policy_version 1847106 (0.0008) [2023-12-27 04:49:19,024][105692] Updated weights for policy 0, policy_version 1842975 (0.0008) [2023-12-27 04:49:19,034][105620] Updated weights for policy 1, policy_version 1847116 (0.0005) [2023-12-27 04:49:19,081][105692] Updated weights for policy 0, policy_version 1842985 (0.0009) [2023-12-27 04:49:19,095][105620] Updated weights for policy 1, policy_version 1847126 (0.0007) [2023-12-27 04:49:19,809][105692] Updated weights for policy 0, policy_version 1842995 (0.0007) [2023-12-27 04:49:19,877][105692] Updated weights for policy 0, policy_version 1843005 (0.0008) [2023-12-27 04:49:19,878][105620] Updated weights for policy 1, policy_version 1847136 (0.0007) [2023-12-27 04:49:19,940][105692] Updated weights for policy 0, policy_version 1843015 (0.0007) [2023-12-27 04:49:19,942][105620] Updated weights for policy 1, policy_version 1847146 (0.0009) [2023-12-27 04:49:20,005][105620] Updated weights for policy 1, policy_version 1847156 (0.0008) [2023-12-27 04:49:20,590][105692] Updated weights for policy 0, policy_version 1843025 (0.0007) [2023-12-27 04:49:20,656][105692] Updated weights for policy 0, policy_version 1843035 (0.0009) [2023-12-27 04:49:20,714][105692] Updated weights for policy 0, policy_version 1843045 (0.0010) [2023-12-27 04:49:20,768][105692] Updated weights for policy 0, policy_version 1843055 (0.0008) [2023-12-27 04:49:20,783][105620] Updated weights for policy 1, policy_version 1847166 (0.0009) [2023-12-27 04:49:20,854][105620] Updated weights for policy 1, policy_version 1847176 (0.0009) [2023-12-27 04:49:20,912][105620] Updated weights for policy 1, policy_version 1847186 (0.0008) [2023-12-27 04:49:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19355.3). Total num frames: 944840704. Throughput: 0: 9658.7, 1: 9499.6. Samples: 944827004. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:49:21,063][104569] Avg episode reward: [(0, '8987.853'), (1, '9072.322')] [2023-12-27 04:49:21,537][105692] Updated weights for policy 0, policy_version 1843065 (0.0009) [2023-12-27 04:49:21,591][105692] Updated weights for policy 0, policy_version 1843075 (0.0008) [2023-12-27 04:49:21,597][105620] Updated weights for policy 1, policy_version 1847196 (0.0007) [2023-12-27 04:49:21,654][105692] Updated weights for policy 0, policy_version 1843085 (0.0006) [2023-12-27 04:49:21,664][105620] Updated weights for policy 1, policy_version 1847206 (0.0009) [2023-12-27 04:49:21,733][105620] Updated weights for policy 1, policy_version 1847216 (0.0009) [2023-12-27 04:49:22,461][105620] Updated weights for policy 1, policy_version 1847226 (0.0009) [2023-12-27 04:49:22,488][105692] Updated weights for policy 0, policy_version 1843095 (0.0006) [2023-12-27 04:49:22,519][105620] Updated weights for policy 1, policy_version 1847236 (0.0008) [2023-12-27 04:49:22,550][105692] Updated weights for policy 0, policy_version 1843105 (0.0007) [2023-12-27 04:49:22,582][105620] Updated weights for policy 1, policy_version 1847246 (0.0007) [2023-12-27 04:49:22,609][105692] Updated weights for policy 0, policy_version 1843115 (0.0008) [2023-12-27 04:49:22,643][105620] Updated weights for policy 1, policy_version 1847256 (0.0009) [2023-12-27 04:49:23,267][105692] Updated weights for policy 0, policy_version 1843125 (0.0007) [2023-12-27 04:49:23,313][105692] Updated weights for policy 0, policy_version 1843135 (0.0005) [2023-12-27 04:49:23,368][105692] Updated weights for policy 0, policy_version 1843145 (0.0005) [2023-12-27 04:49:23,454][105620] Updated weights for policy 1, policy_version 1847266 (0.0009) [2023-12-27 04:49:23,511][105620] Updated weights for policy 1, policy_version 1847276 (0.0010) [2023-12-27 04:49:23,565][105620] Updated weights for policy 1, policy_version 1847286 (0.0010) [2023-12-27 04:49:23,900][105692] Updated weights for policy 0, policy_version 1843155 (0.0008) [2023-12-27 04:49:23,956][105692] Updated weights for policy 0, policy_version 1843165 (0.0009) [2023-12-27 04:49:24,014][105692] Updated weights for policy 0, policy_version 1843175 (0.0010) [2023-12-27 04:49:24,225][105620] Updated weights for policy 1, policy_version 1847296 (0.0008) [2023-12-27 04:49:24,291][105620] Updated weights for policy 1, policy_version 1847306 (0.0011) [2023-12-27 04:49:24,350][105620] Updated weights for policy 1, policy_version 1847316 (0.0011) [2023-12-27 04:49:24,847][105692] Updated weights for policy 0, policy_version 1843185 (0.0010) [2023-12-27 04:49:24,895][105692] Updated weights for policy 0, policy_version 1843195 (0.0009) [2023-12-27 04:49:24,901][105620] Updated weights for policy 1, policy_version 1847326 (0.0007) [2023-12-27 04:49:24,951][105692] Updated weights for policy 0, policy_version 1843205 (0.0009) [2023-12-27 04:49:24,960][105620] Updated weights for policy 1, policy_version 1847336 (0.0005) [2023-12-27 04:49:25,003][105692] Updated weights for policy 0, policy_version 1843215 (0.0009) [2023-12-27 04:49:25,015][105620] Updated weights for policy 1, policy_version 1847346 (0.0005) [2023-12-27 04:49:25,516][105620] Updated weights for policy 1, policy_version 1847356 (0.0006) [2023-12-27 04:49:25,574][105620] Updated weights for policy 1, policy_version 1847366 (0.0005) [2023-12-27 04:49:25,634][105620] Updated weights for policy 1, policy_version 1847376 (0.0005) [2023-12-27 04:49:25,950][105692] Updated weights for policy 0, policy_version 1843225 (0.0009) [2023-12-27 04:49:26,011][105692] Updated weights for policy 0, policy_version 1843235 (0.0010) [2023-12-27 04:49:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19327.6). Total num frames: 944930816. Throughput: 0: 9632.9, 1: 9672.1. Samples: 944945884. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 04:49:26,062][104569] Avg episode reward: [(0, '8446.636'), (1, '9072.091')] [2023-12-27 04:49:26,069][105692] Updated weights for policy 0, policy_version 1843245 (0.0010) [2023-12-27 04:49:26,147][105620] Updated weights for policy 1, policy_version 1847386 (0.0005) [2023-12-27 04:49:26,205][105620] Updated weights for policy 1, policy_version 1847396 (0.0005) [2023-12-27 04:49:26,263][105620] Updated weights for policy 1, policy_version 1847406 (0.0005) [2023-12-27 04:49:26,311][105620] Updated weights for policy 1, policy_version 1847416 (0.0005) [2023-12-27 04:49:26,809][105692] Updated weights for policy 0, policy_version 1843255 (0.0007) [2023-12-27 04:49:26,818][105620] Updated weights for policy 1, policy_version 1847426 (0.0008) [2023-12-27 04:49:26,867][105692] Updated weights for policy 0, policy_version 1843265 (0.0005) [2023-12-27 04:49:26,880][105620] Updated weights for policy 1, policy_version 1847436 (0.0010) [2023-12-27 04:49:26,924][105692] Updated weights for policy 0, policy_version 1843275 (0.0006) [2023-12-27 04:49:26,934][105620] Updated weights for policy 1, policy_version 1847446 (0.0010) [2023-12-27 04:49:27,523][105692] Updated weights for policy 0, policy_version 1843285 (0.0005) [2023-12-27 04:49:27,571][105692] Updated weights for policy 0, policy_version 1843295 (0.0005) [2023-12-27 04:49:27,627][105692] Updated weights for policy 0, policy_version 1843305 (0.0006) [2023-12-27 04:49:27,660][105620] Updated weights for policy 1, policy_version 1847456 (0.0010) [2023-12-27 04:49:27,718][105620] Updated weights for policy 1, policy_version 1847466 (0.0010) [2023-12-27 04:49:27,766][105620] Updated weights for policy 1, policy_version 1847476 (0.0010) [2023-12-27 04:49:28,148][105692] Updated weights for policy 0, policy_version 1843315 (0.0005) [2023-12-27 04:49:28,213][105692] Updated weights for policy 0, policy_version 1843325 (0.0005) [2023-12-27 04:49:28,274][105692] Updated weights for policy 0, policy_version 1843335 (0.0009) [2023-12-27 04:49:28,496][105620] Updated weights for policy 1, policy_version 1847486 (0.0010) [2023-12-27 04:49:28,548][105620] Updated weights for policy 1, policy_version 1847496 (0.0009) [2023-12-27 04:49:28,606][105620] Updated weights for policy 1, policy_version 1847506 (0.0005) [2023-12-27 04:49:29,000][105692] Updated weights for policy 0, policy_version 1843345 (0.0010) [2023-12-27 04:49:29,051][105692] Updated weights for policy 0, policy_version 1843355 (0.0011) [2023-12-27 04:49:29,096][105692] Updated weights for policy 0, policy_version 1843365 (0.0010) [2023-12-27 04:49:29,140][105692] Updated weights for policy 0, policy_version 1843375 (0.0010) [2023-12-27 04:49:29,170][105620] Updated weights for policy 1, policy_version 1847516 (0.0005) [2023-12-27 04:49:29,234][105620] Updated weights for policy 1, policy_version 1847526 (0.0009) [2023-12-27 04:49:29,291][105620] Updated weights for policy 1, policy_version 1847536 (0.0011) [2023-12-27 04:49:29,907][105620] Updated weights for policy 1, policy_version 1847546 (0.0010) [2023-12-27 04:49:29,973][105620] Updated weights for policy 1, policy_version 1847556 (0.0009) [2023-12-27 04:49:29,995][105692] Updated weights for policy 0, policy_version 1843385 (0.0007) [2023-12-27 04:49:30,035][105620] Updated weights for policy 1, policy_version 1847566 (0.0008) [2023-12-27 04:49:30,054][105692] Updated weights for policy 0, policy_version 1843395 (0.0008) [2023-12-27 04:49:30,098][105620] Updated weights for policy 1, policy_version 1847576 (0.0010) [2023-12-27 04:49:30,112][105692] Updated weights for policy 0, policy_version 1843405 (0.0009) [2023-12-27 04:49:30,820][105692] Updated weights for policy 0, policy_version 1843415 (0.0006) [2023-12-27 04:49:30,823][105620] Updated weights for policy 1, policy_version 1847586 (0.0010) [2023-12-27 04:49:30,874][105692] Updated weights for policy 0, policy_version 1843425 (0.0005) [2023-12-27 04:49:30,881][105620] Updated weights for policy 1, policy_version 1847596 (0.0010) [2023-12-27 04:49:30,925][105692] Updated weights for policy 0, policy_version 1843435 (0.0005) [2023-12-27 04:49:30,938][105620] Updated weights for policy 1, policy_version 1847606 (0.0010) [2023-12-27 04:49:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 945045504. Throughput: 0: 9690.7, 1: 9764.6. Samples: 945009548. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:49:31,062][104569] Avg episode reward: [(0, '8351.775'), (1, '9164.217')] [2023-12-27 04:49:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001843440_471990272.pth... [2023-12-27 04:49:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001847608_473055232.pth... [2023-12-27 04:49:31,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001842288_471695360.pth [2023-12-27 04:49:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001846424_472752128.pth [2023-12-27 04:49:31,637][105620] Updated weights for policy 1, policy_version 1847616 (0.0008) [2023-12-27 04:49:31,667][105692] Updated weights for policy 0, policy_version 1843445 (0.0008) [2023-12-27 04:49:31,693][105620] Updated weights for policy 1, policy_version 1847626 (0.0006) [2023-12-27 04:49:31,725][105692] Updated weights for policy 0, policy_version 1843455 (0.0009) [2023-12-27 04:49:31,756][105620] Updated weights for policy 1, policy_version 1847636 (0.0008) [2023-12-27 04:49:31,779][105692] Updated weights for policy 0, policy_version 1843465 (0.0008) [2023-12-27 04:49:32,457][105620] Updated weights for policy 1, policy_version 1847646 (0.0010) [2023-12-27 04:49:32,505][105620] Updated weights for policy 1, policy_version 1847656 (0.0010) [2023-12-27 04:49:32,534][105692] Updated weights for policy 0, policy_version 1843475 (0.0009) [2023-12-27 04:49:32,557][105620] Updated weights for policy 1, policy_version 1847666 (0.0010) [2023-12-27 04:49:32,588][105692] Updated weights for policy 0, policy_version 1843485 (0.0010) [2023-12-27 04:49:32,646][105692] Updated weights for policy 0, policy_version 1843495 (0.0010) [2023-12-27 04:49:33,282][105620] Updated weights for policy 1, policy_version 1847676 (0.0010) [2023-12-27 04:49:33,332][105620] Updated weights for policy 1, policy_version 1847686 (0.0010) [2023-12-27 04:49:33,380][105620] Updated weights for policy 1, policy_version 1847696 (0.0010) [2023-12-27 04:49:33,398][105692] Updated weights for policy 0, policy_version 1843505 (0.0010) [2023-12-27 04:49:33,453][105692] Updated weights for policy 0, policy_version 1843515 (0.0010) [2023-12-27 04:49:33,517][105692] Updated weights for policy 0, policy_version 1843525 (0.0005) [2023-12-27 04:49:33,579][105692] Updated weights for policy 0, policy_version 1843535 (0.0005) [2023-12-27 04:49:34,113][105620] Updated weights for policy 1, policy_version 1847706 (0.0010) [2023-12-27 04:49:34,177][105620] Updated weights for policy 1, policy_version 1847716 (0.0010) [2023-12-27 04:49:34,194][105692] Updated weights for policy 0, policy_version 1843545 (0.0010) [2023-12-27 04:49:34,234][105620] Updated weights for policy 1, policy_version 1847726 (0.0011) [2023-12-27 04:49:34,257][105692] Updated weights for policy 0, policy_version 1843555 (0.0011) [2023-12-27 04:49:34,291][105620] Updated weights for policy 1, policy_version 1847736 (0.0010) [2023-12-27 04:49:34,316][105692] Updated weights for policy 0, policy_version 1843565 (0.0011) [2023-12-27 04:49:35,059][105620] Updated weights for policy 1, policy_version 1847746 (0.0007) [2023-12-27 04:49:35,066][105692] Updated weights for policy 0, policy_version 1843575 (0.0008) [2023-12-27 04:49:35,116][105692] Updated weights for policy 0, policy_version 1843585 (0.0006) [2023-12-27 04:49:35,121][105620] Updated weights for policy 1, policy_version 1847756 (0.0009) [2023-12-27 04:49:35,180][105692] Updated weights for policy 0, policy_version 1843595 (0.0008) [2023-12-27 04:49:35,186][105620] Updated weights for policy 1, policy_version 1847766 (0.0008) [2023-12-27 04:49:35,939][105620] Updated weights for policy 1, policy_version 1847776 (0.0010) [2023-12-27 04:49:35,957][105692] Updated weights for policy 0, policy_version 1843605 (0.0008) [2023-12-27 04:49:35,992][105620] Updated weights for policy 1, policy_version 1847786 (0.0010) [2023-12-27 04:49:36,009][105692] Updated weights for policy 0, policy_version 1843615 (0.0006) [2023-12-27 04:49:36,047][105620] Updated weights for policy 1, policy_version 1847796 (0.0010) [2023-12-27 04:49:36,054][105692] Updated weights for policy 0, policy_version 1843625 (0.0006) [2023-12-27 04:49:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19114.6, 300 sec: 19327.6). Total num frames: 945127424. Throughput: 0: 9691.4, 1: 9818.6. Samples: 945125852. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:49:36,063][104569] Avg episode reward: [(0, '8624.269'), (1, '9164.242')] [2023-12-27 04:49:36,801][105620] Updated weights for policy 1, policy_version 1847806 (0.0010) [2023-12-27 04:49:36,859][105620] Updated weights for policy 1, policy_version 1847816 (0.0010) [2023-12-27 04:49:36,861][105692] Updated weights for policy 0, policy_version 1843635 (0.0008) [2023-12-27 04:49:36,914][105620] Updated weights for policy 1, policy_version 1847826 (0.0010) [2023-12-27 04:49:36,917][105692] Updated weights for policy 0, policy_version 1843645 (0.0006) [2023-12-27 04:49:36,978][105692] Updated weights for policy 0, policy_version 1843655 (0.0007) [2023-12-27 04:49:37,659][105620] Updated weights for policy 1, policy_version 1847836 (0.0010) [2023-12-27 04:49:37,718][105620] Updated weights for policy 1, policy_version 1847846 (0.0007) [2023-12-27 04:49:37,760][105692] Updated weights for policy 0, policy_version 1843665 (0.0008) [2023-12-27 04:49:37,784][105620] Updated weights for policy 1, policy_version 1847856 (0.0010) [2023-12-27 04:49:37,818][105692] Updated weights for policy 0, policy_version 1843675 (0.0009) [2023-12-27 04:49:37,881][105692] Updated weights for policy 0, policy_version 1843685 (0.0008) [2023-12-27 04:49:37,937][105692] Updated weights for policy 0, policy_version 1843695 (0.0008) [2023-12-27 04:49:38,505][105620] Updated weights for policy 1, policy_version 1847866 (0.0009) [2023-12-27 04:49:38,561][105620] Updated weights for policy 1, policy_version 1847876 (0.0007) [2023-12-27 04:49:38,624][105620] Updated weights for policy 1, policy_version 1847886 (0.0008) [2023-12-27 04:49:38,687][105620] Updated weights for policy 1, policy_version 1847896 (0.0011) [2023-12-27 04:49:38,714][105692] Updated weights for policy 0, policy_version 1843705 (0.0008) [2023-12-27 04:49:38,766][105692] Updated weights for policy 0, policy_version 1843715 (0.0008) [2023-12-27 04:49:38,814][105692] Updated weights for policy 0, policy_version 1843725 (0.0008) [2023-12-27 04:49:39,452][105620] Updated weights for policy 1, policy_version 1847906 (0.0011) [2023-12-27 04:49:39,522][105620] Updated weights for policy 1, policy_version 1847916 (0.0010) [2023-12-27 04:49:39,583][105620] Updated weights for policy 1, policy_version 1847926 (0.0011) [2023-12-27 04:49:39,601][105692] Updated weights for policy 0, policy_version 1843735 (0.0009) [2023-12-27 04:49:39,658][105692] Updated weights for policy 0, policy_version 1843745 (0.0008) [2023-12-27 04:49:39,718][105692] Updated weights for policy 0, policy_version 1843755 (0.0008) [2023-12-27 04:49:40,376][105620] Updated weights for policy 1, policy_version 1847936 (0.0009) [2023-12-27 04:49:40,438][105620] Updated weights for policy 1, policy_version 1847946 (0.0009) [2023-12-27 04:49:40,501][105620] Updated weights for policy 1, policy_version 1847956 (0.0008) [2023-12-27 04:49:40,507][105692] Updated weights for policy 0, policy_version 1843765 (0.0008) [2023-12-27 04:49:40,557][105692] Updated weights for policy 0, policy_version 1843775 (0.0008) [2023-12-27 04:49:40,613][105692] Updated weights for policy 0, policy_version 1843785 (0.0009) [2023-12-27 04:49:41,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19251.2, 300 sec: 19327.6). Total num frames: 945225728. Throughput: 0: 9606.0, 1: 9824.6. Samples: 945235016. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:49:41,062][104569] Avg episode reward: [(0, '8532.858'), (1, '9256.629')] [2023-12-27 04:49:41,298][105692] Updated weights for policy 0, policy_version 1843795 (0.0009) [2023-12-27 04:49:41,311][105620] Updated weights for policy 1, policy_version 1847966 (0.0007) [2023-12-27 04:49:41,364][105692] Updated weights for policy 0, policy_version 1843805 (0.0009) [2023-12-27 04:49:41,376][105620] Updated weights for policy 1, policy_version 1847976 (0.0008) [2023-12-27 04:49:41,421][105692] Updated weights for policy 0, policy_version 1843815 (0.0009) [2023-12-27 04:49:41,431][105620] Updated weights for policy 1, policy_version 1847986 (0.0007) [2023-12-27 04:49:42,132][105620] Updated weights for policy 1, policy_version 1847996 (0.0008) [2023-12-27 04:49:42,184][105620] Updated weights for policy 1, policy_version 1848006 (0.0008) [2023-12-27 04:49:42,237][105620] Updated weights for policy 1, policy_version 1848016 (0.0008) [2023-12-27 04:49:42,275][105692] Updated weights for policy 0, policy_version 1843825 (0.0009) [2023-12-27 04:49:42,337][105692] Updated weights for policy 0, policy_version 1843835 (0.0008) [2023-12-27 04:49:42,405][105692] Updated weights for policy 0, policy_version 1843845 (0.0008) [2023-12-27 04:49:42,468][105692] Updated weights for policy 0, policy_version 1843855 (0.0008) [2023-12-27 04:49:42,933][105620] Updated weights for policy 1, policy_version 1848026 (0.0008) [2023-12-27 04:49:42,989][105620] Updated weights for policy 1, policy_version 1848036 (0.0008) [2023-12-27 04:49:43,039][105620] Updated weights for policy 1, policy_version 1848046 (0.0009) [2023-12-27 04:49:43,091][105620] Updated weights for policy 1, policy_version 1848056 (0.0009) [2023-12-27 04:49:43,233][105692] Updated weights for policy 0, policy_version 1843865 (0.0009) [2023-12-27 04:49:43,281][105692] Updated weights for policy 0, policy_version 1843875 (0.0009) [2023-12-27 04:49:43,328][105692] Updated weights for policy 0, policy_version 1843885 (0.0009) [2023-12-27 04:49:43,703][105620] Updated weights for policy 1, policy_version 1848066 (0.0005) [2023-12-27 04:49:43,749][105620] Updated weights for policy 1, policy_version 1848076 (0.0005) [2023-12-27 04:49:43,801][105620] Updated weights for policy 1, policy_version 1848086 (0.0005) [2023-12-27 04:49:44,247][105692] Updated weights for policy 0, policy_version 1843895 (0.0009) [2023-12-27 04:49:44,295][105692] Updated weights for policy 0, policy_version 1843905 (0.0009) [2023-12-27 04:49:44,345][105692] Updated weights for policy 0, policy_version 1843915 (0.0009) [2023-12-27 04:49:44,383][105620] Updated weights for policy 1, policy_version 1848096 (0.0007) [2023-12-27 04:49:44,430][105620] Updated weights for policy 1, policy_version 1848106 (0.0009) [2023-12-27 04:49:44,485][105620] Updated weights for policy 1, policy_version 1848116 (0.0008) [2023-12-27 04:49:45,206][105620] Updated weights for policy 1, policy_version 1848126 (0.0006) [2023-12-27 04:49:45,207][105692] Updated weights for policy 0, policy_version 1843925 (0.0008) [2023-12-27 04:49:45,258][105692] Updated weights for policy 0, policy_version 1843935 (0.0007) [2023-12-27 04:49:45,268][105620] Updated weights for policy 1, policy_version 1848136 (0.0008) [2023-12-27 04:49:45,323][105692] Updated weights for policy 0, policy_version 1843945 (0.0007) [2023-12-27 04:49:45,325][105620] Updated weights for policy 1, policy_version 1848146 (0.0007) [2023-12-27 04:49:46,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19114.8, 300 sec: 19299.8). Total num frames: 945315840. Throughput: 0: 9568.9, 1: 9899.6. Samples: 945292696. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:49:46,062][104569] Avg episode reward: [(0, '8532.595'), (1, '9256.570')] [2023-12-27 04:49:46,063][105620] Updated weights for policy 1, policy_version 1848156 (0.0007) [2023-12-27 04:49:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001843952_472121344.pth... [2023-12-27 04:49:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001842864_471842816.pth [2023-12-27 04:49:46,109][105692] Updated weights for policy 0, policy_version 1843955 (0.0008) [2023-12-27 04:49:46,122][105620] Updated weights for policy 1, policy_version 1848166 (0.0006) [2023-12-27 04:49:46,165][105692] Updated weights for policy 0, policy_version 1843965 (0.0009) [2023-12-27 04:49:46,172][105620] Updated weights for policy 1, policy_version 1848176 (0.0005) [2023-12-27 04:49:46,214][105692] Updated weights for policy 0, policy_version 1843975 (0.0008) [2023-12-27 04:49:46,217][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001848184_473202688.pth... [2023-12-27 04:49:46,221][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001847000_472899584.pth [2023-12-27 04:49:46,812][105620] Updated weights for policy 1, policy_version 1848186 (0.0006) [2023-12-27 04:49:46,874][105620] Updated weights for policy 1, policy_version 1848196 (0.0008) [2023-12-27 04:49:46,924][105620] Updated weights for policy 1, policy_version 1848206 (0.0009) [2023-12-27 04:49:46,986][105692] Updated weights for policy 0, policy_version 1843985 (0.0009) [2023-12-27 04:49:46,987][105620] Updated weights for policy 1, policy_version 1848216 (0.0009) [2023-12-27 04:49:47,039][105692] Updated weights for policy 0, policy_version 1843995 (0.0007) [2023-12-27 04:49:47,092][105692] Updated weights for policy 0, policy_version 1844005 (0.0008) [2023-12-27 04:49:47,144][105692] Updated weights for policy 0, policy_version 1844015 (0.0008) [2023-12-27 04:49:47,722][105620] Updated weights for policy 1, policy_version 1848226 (0.0010) [2023-12-27 04:49:47,781][105620] Updated weights for policy 1, policy_version 1848236 (0.0011) [2023-12-27 04:49:47,846][105620] Updated weights for policy 1, policy_version 1848246 (0.0010) [2023-12-27 04:49:47,902][105692] Updated weights for policy 0, policy_version 1844025 (0.0008) [2023-12-27 04:49:47,959][105692] Updated weights for policy 0, policy_version 1844035 (0.0005) [2023-12-27 04:49:48,013][105692] Updated weights for policy 0, policy_version 1844045 (0.0006) [2023-12-27 04:49:48,577][105620] Updated weights for policy 1, policy_version 1848256 (0.0007) [2023-12-27 04:49:48,639][105620] Updated weights for policy 1, policy_version 1848266 (0.0010) [2023-12-27 04:49:48,670][105692] Updated weights for policy 0, policy_version 1844055 (0.0005) [2023-12-27 04:49:48,688][105620] Updated weights for policy 1, policy_version 1848276 (0.0011) [2023-12-27 04:49:48,743][105692] Updated weights for policy 0, policy_version 1844065 (0.0005) [2023-12-27 04:49:48,801][105692] Updated weights for policy 0, policy_version 1844075 (0.0006) [2023-12-27 04:49:49,276][105620] Updated weights for policy 1, policy_version 1848286 (0.0008) [2023-12-27 04:49:49,342][105620] Updated weights for policy 1, policy_version 1848296 (0.0008) [2023-12-27 04:49:49,412][105620] Updated weights for policy 1, policy_version 1848306 (0.0006) [2023-12-27 04:49:49,611][105692] Updated weights for policy 0, policy_version 1844085 (0.0005) [2023-12-27 04:49:49,671][105692] Updated weights for policy 0, policy_version 1844095 (0.0008) [2023-12-27 04:49:49,740][105692] Updated weights for policy 0, policy_version 1844105 (0.0010) [2023-12-27 04:49:50,031][105620] Updated weights for policy 1, policy_version 1848316 (0.0007) [2023-12-27 04:49:50,083][105620] Updated weights for policy 1, policy_version 1848326 (0.0008) [2023-12-27 04:49:50,141][105620] Updated weights for policy 1, policy_version 1848336 (0.0006) [2023-12-27 04:49:50,541][105692] Updated weights for policy 0, policy_version 1844115 (0.0010) [2023-12-27 04:49:50,606][105692] Updated weights for policy 0, policy_version 1844125 (0.0009) [2023-12-27 04:49:50,669][105692] Updated weights for policy 0, policy_version 1844135 (0.0008) [2023-12-27 04:49:50,793][105620] Updated weights for policy 1, policy_version 1848346 (0.0007) [2023-12-27 04:49:50,852][105620] Updated weights for policy 1, policy_version 1848356 (0.0008) [2023-12-27 04:49:50,915][105620] Updated weights for policy 1, policy_version 1848366 (0.0006) [2023-12-27 04:49:50,975][105620] Updated weights for policy 1, policy_version 1848376 (0.0006) [2023-12-27 04:49:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19327.6). Total num frames: 945422336. Throughput: 0: 9480.8, 1: 9953.4. Samples: 945408928. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:49:51,062][104569] Avg episode reward: [(0, '8442.413'), (1, '9256.443')] [2023-12-27 04:49:51,457][105692] Updated weights for policy 0, policy_version 1844145 (0.0008) [2023-12-27 04:49:51,519][105692] Updated weights for policy 0, policy_version 1844155 (0.0006) [2023-12-27 04:49:51,583][105692] Updated weights for policy 0, policy_version 1844165 (0.0006) [2023-12-27 04:49:51,636][105620] Updated weights for policy 1, policy_version 1848386 (0.0006) [2023-12-27 04:49:51,649][105692] Updated weights for policy 0, policy_version 1844175 (0.0009) [2023-12-27 04:49:51,698][105620] Updated weights for policy 1, policy_version 1848396 (0.0009) [2023-12-27 04:49:51,768][105620] Updated weights for policy 1, policy_version 1848406 (0.0009) [2023-12-27 04:49:52,335][105692] Updated weights for policy 0, policy_version 1844185 (0.0009) [2023-12-27 04:49:52,402][105692] Updated weights for policy 0, policy_version 1844195 (0.0009) [2023-12-27 04:49:52,463][105692] Updated weights for policy 0, policy_version 1844205 (0.0010) [2023-12-27 04:49:52,574][105620] Updated weights for policy 1, policy_version 1848416 (0.0008) [2023-12-27 04:49:52,632][105620] Updated weights for policy 1, policy_version 1848426 (0.0009) [2023-12-27 04:49:52,690][105620] Updated weights for policy 1, policy_version 1848436 (0.0009) [2023-12-27 04:49:53,223][105692] Updated weights for policy 0, policy_version 1844215 (0.0009) [2023-12-27 04:49:53,280][105692] Updated weights for policy 0, policy_version 1844225 (0.0008) [2023-12-27 04:49:53,345][105692] Updated weights for policy 0, policy_version 1844235 (0.0007) [2023-12-27 04:49:53,425][105620] Updated weights for policy 1, policy_version 1848446 (0.0006) [2023-12-27 04:49:53,491][105620] Updated weights for policy 1, policy_version 1848456 (0.0005) [2023-12-27 04:49:53,544][105620] Updated weights for policy 1, policy_version 1848466 (0.0005) [2023-12-27 04:49:54,057][105692] Updated weights for policy 0, policy_version 1844245 (0.0005) [2023-12-27 04:49:54,110][105692] Updated weights for policy 0, policy_version 1844255 (0.0008) [2023-12-27 04:49:54,155][105620] Updated weights for policy 1, policy_version 1848476 (0.0007) [2023-12-27 04:49:54,162][105692] Updated weights for policy 0, policy_version 1844265 (0.0008) [2023-12-27 04:49:54,216][105620] Updated weights for policy 1, policy_version 1848486 (0.0008) [2023-12-27 04:49:54,270][105620] Updated weights for policy 1, policy_version 1848496 (0.0008) [2023-12-27 04:49:54,850][105692] Updated weights for policy 0, policy_version 1844275 (0.0006) [2023-12-27 04:49:54,904][105692] Updated weights for policy 0, policy_version 1844285 (0.0005) [2023-12-27 04:49:54,915][105620] Updated weights for policy 1, policy_version 1848506 (0.0007) [2023-12-27 04:49:54,964][105692] Updated weights for policy 0, policy_version 1844295 (0.0006) [2023-12-27 04:49:54,973][105620] Updated weights for policy 1, policy_version 1848516 (0.0010) [2023-12-27 04:49:55,023][105620] Updated weights for policy 1, policy_version 1848526 (0.0009) [2023-12-27 04:49:55,080][105620] Updated weights for policy 1, policy_version 1848536 (0.0006) [2023-12-27 04:49:55,641][105620] Updated weights for policy 1, policy_version 1848546 (0.0006) [2023-12-27 04:49:55,691][105620] Updated weights for policy 1, policy_version 1848556 (0.0008) [2023-12-27 04:49:55,720][105692] Updated weights for policy 0, policy_version 1844305 (0.0007) [2023-12-27 04:49:55,743][105620] Updated weights for policy 1, policy_version 1848566 (0.0005) [2023-12-27 04:49:55,776][105692] Updated weights for policy 0, policy_version 1844315 (0.0011) [2023-12-27 04:49:55,821][105692] Updated weights for policy 0, policy_version 1844325 (0.0010) [2023-12-27 04:49:55,870][105692] Updated weights for policy 0, policy_version 1844335 (0.0010) [2023-12-27 04:49:56,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.2, 300 sec: 19327.6). Total num frames: 945520640. Throughput: 0: 9427.2, 1: 9996.0. Samples: 945527428. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:49:56,062][104569] Avg episode reward: [(0, '8444.773'), (1, '9164.357')] [2023-12-27 04:49:56,285][105620] Updated weights for policy 1, policy_version 1848576 (0.0009) [2023-12-27 04:49:56,337][105620] Updated weights for policy 1, policy_version 1848586 (0.0010) [2023-12-27 04:49:56,403][105620] Updated weights for policy 1, policy_version 1848596 (0.0011) [2023-12-27 04:49:56,622][105692] Updated weights for policy 0, policy_version 1844345 (0.0010) [2023-12-27 04:49:56,676][105692] Updated weights for policy 0, policy_version 1844355 (0.0010) [2023-12-27 04:49:56,733][105692] Updated weights for policy 0, policy_version 1844365 (0.0010) [2023-12-27 04:49:57,125][105620] Updated weights for policy 1, policy_version 1848606 (0.0010) [2023-12-27 04:49:57,173][105620] Updated weights for policy 1, policy_version 1848616 (0.0010) [2023-12-27 04:49:57,232][105620] Updated weights for policy 1, policy_version 1848626 (0.0010) [2023-12-27 04:49:57,306][105692] Updated weights for policy 0, policy_version 1844375 (0.0009) [2023-12-27 04:49:57,366][105692] Updated weights for policy 0, policy_version 1844385 (0.0010) [2023-12-27 04:49:57,417][105692] Updated weights for policy 0, policy_version 1844395 (0.0009) [2023-12-27 04:49:57,912][105620] Updated weights for policy 1, policy_version 1848636 (0.0010) [2023-12-27 04:49:57,971][105620] Updated weights for policy 1, policy_version 1848646 (0.0010) [2023-12-27 04:49:58,032][105620] Updated weights for policy 1, policy_version 1848656 (0.0010) [2023-12-27 04:49:58,218][105692] Updated weights for policy 0, policy_version 1844405 (0.0009) [2023-12-27 04:49:58,277][105692] Updated weights for policy 0, policy_version 1844415 (0.0006) [2023-12-27 04:49:58,353][105692] Updated weights for policy 0, policy_version 1844425 (0.0006) [2023-12-27 04:49:58,815][105620] Updated weights for policy 1, policy_version 1848666 (0.0010) [2023-12-27 04:49:58,880][105620] Updated weights for policy 1, policy_version 1848676 (0.0009) [2023-12-27 04:49:58,945][105620] Updated weights for policy 1, policy_version 1848686 (0.0008) [2023-12-27 04:49:59,007][105620] Updated weights for policy 1, policy_version 1848696 (0.0006) [2023-12-27 04:49:59,171][105692] Updated weights for policy 0, policy_version 1844435 (0.0008) [2023-12-27 04:49:59,243][105692] Updated weights for policy 0, policy_version 1844445 (0.0009) [2023-12-27 04:49:59,308][105692] Updated weights for policy 0, policy_version 1844455 (0.0011) [2023-12-27 04:49:59,792][105620] Updated weights for policy 1, policy_version 1848706 (0.0010) [2023-12-27 04:49:59,854][105620] Updated weights for policy 1, policy_version 1848716 (0.0008) [2023-12-27 04:49:59,913][105620] Updated weights for policy 1, policy_version 1848726 (0.0008) [2023-12-27 04:50:00,025][105692] Updated weights for policy 0, policy_version 1844465 (0.0009) [2023-12-27 04:50:00,082][105692] Updated weights for policy 0, policy_version 1844475 (0.0009) [2023-12-27 04:50:00,134][105692] Updated weights for policy 0, policy_version 1844485 (0.0009) [2023-12-27 04:50:00,195][105692] Updated weights for policy 0, policy_version 1844495 (0.0009) [2023-12-27 04:50:00,559][105620] Updated weights for policy 1, policy_version 1848736 (0.0008) [2023-12-27 04:50:00,621][105620] Updated weights for policy 1, policy_version 1848746 (0.0007) [2023-12-27 04:50:00,685][105620] Updated weights for policy 1, policy_version 1848756 (0.0006) [2023-12-27 04:50:01,057][105692] Updated weights for policy 0, policy_version 1844505 (0.0010) [2023-12-27 04:50:01,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19387.7, 300 sec: 19272.0). Total num frames: 945610752. Throughput: 0: 9463.3, 1: 10035.9. Samples: 945585904. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:50:01,063][104569] Avg episode reward: [(0, '8718.452'), (1, '9164.059')] [2023-12-27 04:50:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001848760_473350144.pth... [2023-12-27 04:50:01,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001847608_473055232.pth [2023-12-27 04:50:01,108][105692] Updated weights for policy 0, policy_version 1844515 (0.0008) [2023-12-27 04:50:01,169][105692] Updated weights for policy 0, policy_version 1844525 (0.0008) [2023-12-27 04:50:01,185][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001844528_472268800.pth... [2023-12-27 04:50:01,190][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001843440_471990272.pth [2023-12-27 04:50:01,285][105620] Updated weights for policy 1, policy_version 1848766 (0.0009) [2023-12-27 04:50:01,344][105620] Updated weights for policy 1, policy_version 1848776 (0.0009) [2023-12-27 04:50:01,403][105620] Updated weights for policy 1, policy_version 1848786 (0.0009) [2023-12-27 04:50:01,931][105692] Updated weights for policy 0, policy_version 1844535 (0.0006) [2023-12-27 04:50:01,984][105692] Updated weights for policy 0, policy_version 1844545 (0.0006) [2023-12-27 04:50:02,031][105692] Updated weights for policy 0, policy_version 1844555 (0.0009) [2023-12-27 04:50:02,189][105620] Updated weights for policy 1, policy_version 1848796 (0.0009) [2023-12-27 04:50:02,239][105620] Updated weights for policy 1, policy_version 1848806 (0.0009) [2023-12-27 04:50:02,302][105620] Updated weights for policy 1, policy_version 1848816 (0.0009) [2023-12-27 04:50:02,761][105692] Updated weights for policy 0, policy_version 1844565 (0.0008) [2023-12-27 04:50:02,830][105692] Updated weights for policy 0, policy_version 1844575 (0.0007) [2023-12-27 04:50:02,898][105692] Updated weights for policy 0, policy_version 1844585 (0.0010) [2023-12-27 04:50:03,010][105620] Updated weights for policy 1, policy_version 1848826 (0.0009) [2023-12-27 04:50:03,068][105620] Updated weights for policy 1, policy_version 1848836 (0.0006) [2023-12-27 04:50:03,119][105620] Updated weights for policy 1, policy_version 1848846 (0.0005) [2023-12-27 04:50:03,169][105620] Updated weights for policy 1, policy_version 1848856 (0.0008) [2023-12-27 04:50:03,559][105692] Updated weights for policy 0, policy_version 1844595 (0.0007) [2023-12-27 04:50:03,606][105692] Updated weights for policy 0, policy_version 1844605 (0.0009) [2023-12-27 04:50:03,657][105692] Updated weights for policy 0, policy_version 1844616 (0.0009) [2023-12-27 04:50:03,794][105620] Updated weights for policy 1, policy_version 1848866 (0.0008) [2023-12-27 04:50:03,852][105620] Updated weights for policy 1, policy_version 1848877 (0.0009) [2023-12-27 04:50:03,902][105620] Updated weights for policy 1, policy_version 1848887 (0.0008) [2023-12-27 04:50:04,461][105692] Updated weights for policy 0, policy_version 1844626 (0.0010) [2023-12-27 04:50:04,516][105692] Updated weights for policy 0, policy_version 1844636 (0.0010) [2023-12-27 04:50:04,571][105692] Updated weights for policy 0, policy_version 1844646 (0.0010) [2023-12-27 04:50:04,634][105692] Updated weights for policy 0, policy_version 1844656 (0.0011) [2023-12-27 04:50:04,688][105620] Updated weights for policy 1, policy_version 1848897 (0.0010) [2023-12-27 04:50:04,746][105620] Updated weights for policy 1, policy_version 1848907 (0.0010) [2023-12-27 04:50:04,803][105620] Updated weights for policy 1, policy_version 1848917 (0.0010) [2023-12-27 04:50:05,215][105692] Updated weights for policy 0, policy_version 1844666 (0.0006) [2023-12-27 04:50:05,273][105692] Updated weights for policy 0, policy_version 1844676 (0.0010) [2023-12-27 04:50:05,325][105692] Updated weights for policy 0, policy_version 1844686 (0.0010) [2023-12-27 04:50:05,503][105620] Updated weights for policy 1, policy_version 1848927 (0.0007) [2023-12-27 04:50:05,560][105620] Updated weights for policy 1, policy_version 1848937 (0.0005) [2023-12-27 04:50:05,626][105620] Updated weights for policy 1, policy_version 1848947 (0.0005) [2023-12-27 04:50:06,048][105692] Updated weights for policy 0, policy_version 1844696 (0.0011) [2023-12-27 04:50:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19299.8). Total num frames: 945709056. Throughput: 0: 9366.3, 1: 10051.9. Samples: 945700824. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:50:06,062][104569] Avg episode reward: [(0, '8630.623'), (1, '9163.836')] [2023-12-27 04:50:06,109][105692] Updated weights for policy 0, policy_version 1844706 (0.0010) [2023-12-27 04:50:06,171][105692] Updated weights for policy 0, policy_version 1844716 (0.0007) [2023-12-27 04:50:06,225][105620] Updated weights for policy 1, policy_version 1848957 (0.0008) [2023-12-27 04:50:06,289][105620] Updated weights for policy 1, policy_version 1848967 (0.0011) [2023-12-27 04:50:06,351][105620] Updated weights for policy 1, policy_version 1848977 (0.0010) [2023-12-27 04:50:06,816][105692] Updated weights for policy 0, policy_version 1844726 (0.0008) [2023-12-27 04:50:06,881][105692] Updated weights for policy 0, policy_version 1844736 (0.0010) [2023-12-27 04:50:06,947][105692] Updated weights for policy 0, policy_version 1844746 (0.0010) [2023-12-27 04:50:07,094][105620] Updated weights for policy 1, policy_version 1848987 (0.0010) [2023-12-27 04:50:07,145][105620] Updated weights for policy 1, policy_version 1848997 (0.0010) [2023-12-27 04:50:07,204][105620] Updated weights for policy 1, policy_version 1849007 (0.0010) [2023-12-27 04:50:07,608][105692] Updated weights for policy 0, policy_version 1844756 (0.0010) [2023-12-27 04:50:07,659][105692] Updated weights for policy 0, policy_version 1844766 (0.0010) [2023-12-27 04:50:07,713][105692] Updated weights for policy 0, policy_version 1844776 (0.0010) [2023-12-27 04:50:07,807][105620] Updated weights for policy 1, policy_version 1849017 (0.0010) [2023-12-27 04:50:07,870][105620] Updated weights for policy 1, policy_version 1849027 (0.0005) [2023-12-27 04:50:07,927][105620] Updated weights for policy 1, policy_version 1849037 (0.0005) [2023-12-27 04:50:07,977][105620] Updated weights for policy 1, policy_version 1849047 (0.0006) [2023-12-27 04:50:08,478][105692] Updated weights for policy 0, policy_version 1844786 (0.0010) [2023-12-27 04:50:08,546][105692] Updated weights for policy 0, policy_version 1844796 (0.0010) [2023-12-27 04:50:08,606][105692] Updated weights for policy 0, policy_version 1844806 (0.0008) [2023-12-27 04:50:08,613][105620] Updated weights for policy 1, policy_version 1849057 (0.0007) [2023-12-27 04:50:08,665][105692] Updated weights for policy 0, policy_version 1844816 (0.0008) [2023-12-27 04:50:08,671][105620] Updated weights for policy 1, policy_version 1849067 (0.0006) [2023-12-27 04:50:08,733][105620] Updated weights for policy 1, policy_version 1849077 (0.0010) [2023-12-27 04:50:09,428][105692] Updated weights for policy 0, policy_version 1844826 (0.0008) [2023-12-27 04:50:09,445][105620] Updated weights for policy 1, policy_version 1849087 (0.0008) [2023-12-27 04:50:09,481][105692] Updated weights for policy 0, policy_version 1844836 (0.0009) [2023-12-27 04:50:09,495][105620] Updated weights for policy 1, policy_version 1849097 (0.0005) [2023-12-27 04:50:09,544][105692] Updated weights for policy 0, policy_version 1844846 (0.0009) [2023-12-27 04:50:09,548][105620] Updated weights for policy 1, policy_version 1849107 (0.0009) [2023-12-27 04:50:10,234][105620] Updated weights for policy 1, policy_version 1849117 (0.0011) [2023-12-27 04:50:10,288][105620] Updated weights for policy 1, policy_version 1849127 (0.0011) [2023-12-27 04:50:10,294][105692] Updated weights for policy 0, policy_version 1844856 (0.0006) [2023-12-27 04:50:10,350][105620] Updated weights for policy 1, policy_version 1849137 (0.0009) [2023-12-27 04:50:10,358][105692] Updated weights for policy 0, policy_version 1844866 (0.0007) [2023-12-27 04:50:10,419][105692] Updated weights for policy 0, policy_version 1844876 (0.0006) [2023-12-27 04:50:11,034][105692] Updated weights for policy 0, policy_version 1844886 (0.0007) [2023-12-27 04:50:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19299.8). Total num frames: 945807360. Throughput: 0: 9415.9, 1: 10048.5. Samples: 945821784. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:50:11,062][105620] Updated weights for policy 1, policy_version 1849147 (0.0007) [2023-12-27 04:50:11,063][104569] Avg episode reward: [(0, '8357.862'), (1, '9256.195')] [2023-12-27 04:50:11,099][105692] Updated weights for policy 0, policy_version 1844896 (0.0006) [2023-12-27 04:50:11,133][105620] Updated weights for policy 1, policy_version 1849157 (0.0011) [2023-12-27 04:50:11,169][105692] Updated weights for policy 0, policy_version 1844906 (0.0008) [2023-12-27 04:50:11,196][105620] Updated weights for policy 1, policy_version 1849167 (0.0008) [2023-12-27 04:50:11,946][105692] Updated weights for policy 0, policy_version 1844916 (0.0008) [2023-12-27 04:50:11,960][105620] Updated weights for policy 1, policy_version 1849177 (0.0008) [2023-12-27 04:50:12,007][105692] Updated weights for policy 0, policy_version 1844926 (0.0009) [2023-12-27 04:50:12,022][105620] Updated weights for policy 1, policy_version 1849187 (0.0008) [2023-12-27 04:50:12,062][105692] Updated weights for policy 0, policy_version 1844936 (0.0006) [2023-12-27 04:50:12,085][105620] Updated weights for policy 1, policy_version 1849197 (0.0008) [2023-12-27 04:50:12,144][105620] Updated weights for policy 1, policy_version 1849207 (0.0006) [2023-12-27 04:50:12,741][105620] Updated weights for policy 1, policy_version 1849217 (0.0008) [2023-12-27 04:50:12,809][105620] Updated weights for policy 1, policy_version 1849227 (0.0010) [2023-12-27 04:50:12,857][105620] Updated weights for policy 1, policy_version 1849237 (0.0007) [2023-12-27 04:50:12,875][105692] Updated weights for policy 0, policy_version 1844946 (0.0009) [2023-12-27 04:50:12,935][105692] Updated weights for policy 0, policy_version 1844956 (0.0010) [2023-12-27 04:50:12,987][105692] Updated weights for policy 0, policy_version 1844966 (0.0007) [2023-12-27 04:50:13,048][105692] Updated weights for policy 0, policy_version 1844976 (0.0005) [2023-12-27 04:50:13,396][105620] Updated weights for policy 1, policy_version 1849247 (0.0005) [2023-12-27 04:50:13,454][105620] Updated weights for policy 1, policy_version 1849257 (0.0007) [2023-12-27 04:50:13,516][105620] Updated weights for policy 1, policy_version 1849267 (0.0006) [2023-12-27 04:50:13,788][105692] Updated weights for policy 0, policy_version 1844986 (0.0010) [2023-12-27 04:50:13,841][105692] Updated weights for policy 0, policy_version 1844996 (0.0010) [2023-12-27 04:50:13,907][105692] Updated weights for policy 0, policy_version 1845006 (0.0008) [2023-12-27 04:50:14,097][105620] Updated weights for policy 1, policy_version 1849277 (0.0008) [2023-12-27 04:50:14,158][105620] Updated weights for policy 1, policy_version 1849287 (0.0010) [2023-12-27 04:50:14,209][105620] Updated weights for policy 1, policy_version 1849297 (0.0010) [2023-12-27 04:50:14,621][105692] Updated weights for policy 0, policy_version 1845016 (0.0006) [2023-12-27 04:50:14,674][105692] Updated weights for policy 0, policy_version 1845026 (0.0005) [2023-12-27 04:50:14,734][105692] Updated weights for policy 0, policy_version 1845036 (0.0005) [2023-12-27 04:50:14,970][105620] Updated weights for policy 1, policy_version 1849307 (0.0010) [2023-12-27 04:50:15,024][105620] Updated weights for policy 1, policy_version 1849317 (0.0010) [2023-12-27 04:50:15,073][105620] Updated weights for policy 1, policy_version 1849327 (0.0010) [2023-12-27 04:50:15,451][105692] Updated weights for policy 0, policy_version 1845046 (0.0008) [2023-12-27 04:50:15,506][105692] Updated weights for policy 0, policy_version 1845056 (0.0010) [2023-12-27 04:50:15,560][105692] Updated weights for policy 0, policy_version 1845066 (0.0008) [2023-12-27 04:50:15,730][105620] Updated weights for policy 1, policy_version 1849337 (0.0011) [2023-12-27 04:50:15,785][105620] Updated weights for policy 1, policy_version 1849347 (0.0010) [2023-12-27 04:50:15,833][105620] Updated weights for policy 1, policy_version 1849357 (0.0010) [2023-12-27 04:50:15,884][105620] Updated weights for policy 1, policy_version 1849367 (0.0010) [2023-12-27 04:50:16,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19524.2, 300 sec: 19327.6). Total num frames: 945913856. Throughput: 0: 9351.9, 1: 10048.2. Samples: 945882556. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:50:16,063][104569] Avg episode reward: [(0, '7984.199'), (1, '9348.608')] [2023-12-27 04:50:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001845072_472408064.pth... [2023-12-27 04:50:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001849368_473505792.pth... [2023-12-27 04:50:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001843952_472121344.pth [2023-12-27 04:50:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001848184_473202688.pth [2023-12-27 04:50:16,202][105692] Updated weights for policy 0, policy_version 1845076 (0.0006) [2023-12-27 04:50:16,254][105692] Updated weights for policy 0, policy_version 1845086 (0.0008) [2023-12-27 04:50:16,308][105692] Updated weights for policy 0, policy_version 1845096 (0.0008) [2023-12-27 04:50:16,645][105620] Updated weights for policy 1, policy_version 1849377 (0.0010) [2023-12-27 04:50:16,699][105620] Updated weights for policy 1, policy_version 1849387 (0.0010) [2023-12-27 04:50:16,760][105620] Updated weights for policy 1, policy_version 1849397 (0.0010) [2023-12-27 04:50:17,062][105692] Updated weights for policy 0, policy_version 1845106 (0.0008) [2023-12-27 04:50:17,125][105692] Updated weights for policy 0, policy_version 1845116 (0.0008) [2023-12-27 04:50:17,173][105692] Updated weights for policy 0, policy_version 1845126 (0.0009) [2023-12-27 04:50:17,231][105692] Updated weights for policy 0, policy_version 1845136 (0.0008) [2023-12-27 04:50:17,502][105620] Updated weights for policy 1, policy_version 1849407 (0.0010) [2023-12-27 04:50:17,566][105620] Updated weights for policy 1, policy_version 1849417 (0.0010) [2023-12-27 04:50:17,627][105620] Updated weights for policy 1, policy_version 1849427 (0.0010) [2023-12-27 04:50:17,996][105692] Updated weights for policy 0, policy_version 1845146 (0.0008) [2023-12-27 04:50:18,052][105692] Updated weights for policy 0, policy_version 1845156 (0.0009) [2023-12-27 04:50:18,102][105692] Updated weights for policy 0, policy_version 1845166 (0.0009) [2023-12-27 04:50:18,349][105620] Updated weights for policy 1, policy_version 1849437 (0.0010) [2023-12-27 04:50:18,411][105620] Updated weights for policy 1, policy_version 1849447 (0.0008) [2023-12-27 04:50:18,459][105620] Updated weights for policy 1, policy_version 1849457 (0.0010) [2023-12-27 04:50:18,890][105692] Updated weights for policy 0, policy_version 1845176 (0.0009) [2023-12-27 04:50:18,952][105692] Updated weights for policy 0, policy_version 1845186 (0.0009) [2023-12-27 04:50:19,023][105692] Updated weights for policy 0, policy_version 1845196 (0.0009) [2023-12-27 04:50:19,119][105620] Updated weights for policy 1, policy_version 1849467 (0.0009) [2023-12-27 04:50:19,178][105620] Updated weights for policy 1, policy_version 1849477 (0.0008) [2023-12-27 04:50:19,232][105620] Updated weights for policy 1, policy_version 1849487 (0.0005) [2023-12-27 04:50:19,754][105692] Updated weights for policy 0, policy_version 1845206 (0.0010) [2023-12-27 04:50:19,813][105692] Updated weights for policy 0, policy_version 1845216 (0.0009) [2023-12-27 04:50:19,882][105692] Updated weights for policy 0, policy_version 1845226 (0.0009) [2023-12-27 04:50:20,033][105620] Updated weights for policy 1, policy_version 1849497 (0.0009) [2023-12-27 04:50:20,098][105620] Updated weights for policy 1, policy_version 1849507 (0.0009) [2023-12-27 04:50:20,158][105620] Updated weights for policy 1, policy_version 1849517 (0.0009) [2023-12-27 04:50:20,214][105620] Updated weights for policy 1, policy_version 1849527 (0.0005) [2023-12-27 04:50:20,627][105692] Updated weights for policy 0, policy_version 1845236 (0.0009) [2023-12-27 04:50:20,694][105692] Updated weights for policy 0, policy_version 1845246 (0.0009) [2023-12-27 04:50:20,758][105692] Updated weights for policy 0, policy_version 1845256 (0.0007) [2023-12-27 04:50:21,014][105620] Updated weights for policy 1, policy_version 1849537 (0.0009) [2023-12-27 04:50:21,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19299.8). Total num frames: 946003968. Throughput: 0: 9360.0, 1: 10014.1. Samples: 945997684. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:50:21,062][104569] Avg episode reward: [(0, '7888.795'), (1, '9348.648')] [2023-12-27 04:50:21,080][105620] Updated weights for policy 1, policy_version 1849547 (0.0009) [2023-12-27 04:50:21,149][105620] Updated weights for policy 1, policy_version 1849557 (0.0008) [2023-12-27 04:50:21,412][105692] Updated weights for policy 0, policy_version 1845266 (0.0006) [2023-12-27 04:50:21,483][105692] Updated weights for policy 0, policy_version 1845276 (0.0009) [2023-12-27 04:50:21,555][105692] Updated weights for policy 0, policy_version 1845286 (0.0010) [2023-12-27 04:50:21,619][105692] Updated weights for policy 0, policy_version 1845296 (0.0009) [2023-12-27 04:50:21,952][105620] Updated weights for policy 1, policy_version 1849567 (0.0008) [2023-12-27 04:50:22,015][105620] Updated weights for policy 1, policy_version 1849577 (0.0009) [2023-12-27 04:50:22,071][105620] Updated weights for policy 1, policy_version 1849587 (0.0009) [2023-12-27 04:50:22,384][105692] Updated weights for policy 0, policy_version 1845306 (0.0009) [2023-12-27 04:50:22,448][105692] Updated weights for policy 0, policy_version 1845316 (0.0008) [2023-12-27 04:50:22,510][105692] Updated weights for policy 0, policy_version 1845326 (0.0010) [2023-12-27 04:50:22,807][105620] Updated weights for policy 1, policy_version 1849597 (0.0009) [2023-12-27 04:50:22,871][105620] Updated weights for policy 1, policy_version 1849607 (0.0008) [2023-12-27 04:50:22,940][105620] Updated weights for policy 1, policy_version 1849617 (0.0006) [2023-12-27 04:50:23,321][105692] Updated weights for policy 0, policy_version 1845336 (0.0010) [2023-12-27 04:50:23,379][105692] Updated weights for policy 0, policy_version 1845346 (0.0009) [2023-12-27 04:50:23,435][105692] Updated weights for policy 0, policy_version 1845356 (0.0010) [2023-12-27 04:50:23,556][105620] Updated weights for policy 1, policy_version 1849627 (0.0006) [2023-12-27 04:50:23,609][105620] Updated weights for policy 1, policy_version 1849637 (0.0008) [2023-12-27 04:50:23,658][105620] Updated weights for policy 1, policy_version 1849647 (0.0010) [2023-12-27 04:50:24,219][105692] Updated weights for policy 0, policy_version 1845366 (0.0009) [2023-12-27 04:50:24,269][105692] Updated weights for policy 0, policy_version 1845376 (0.0008) [2023-12-27 04:50:24,330][105692] Updated weights for policy 0, policy_version 1845386 (0.0009) [2023-12-27 04:50:24,353][105620] Updated weights for policy 1, policy_version 1849657 (0.0011) [2023-12-27 04:50:24,406][105620] Updated weights for policy 1, policy_version 1849667 (0.0010) [2023-12-27 04:50:24,456][105620] Updated weights for policy 1, policy_version 1849677 (0.0008) [2023-12-27 04:50:24,518][105620] Updated weights for policy 1, policy_version 1849687 (0.0005) [2023-12-27 04:50:25,127][105620] Updated weights for policy 1, policy_version 1849697 (0.0005) [2023-12-27 04:50:25,129][105692] Updated weights for policy 0, policy_version 1845396 (0.0006) [2023-12-27 04:50:25,183][105620] Updated weights for policy 1, policy_version 1849707 (0.0005) [2023-12-27 04:50:25,193][105692] Updated weights for policy 0, policy_version 1845406 (0.0009) [2023-12-27 04:50:25,232][105620] Updated weights for policy 1, policy_version 1849717 (0.0005) [2023-12-27 04:50:25,248][105692] Updated weights for policy 0, policy_version 1845416 (0.0008) [2023-12-27 04:50:25,787][105620] Updated weights for policy 1, policy_version 1849727 (0.0005) [2023-12-27 04:50:25,833][105620] Updated weights for policy 1, policy_version 1849737 (0.0005) [2023-12-27 04:50:25,890][105620] Updated weights for policy 1, policy_version 1849747 (0.0007) [2023-12-27 04:50:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19299.8). Total num frames: 946102272. Throughput: 0: 9360.7, 1: 10127.8. Samples: 946112000. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:50:26,063][104569] Avg episode reward: [(0, '8164.263'), (1, '9256.306')] [2023-12-27 04:50:26,098][105692] Updated weights for policy 0, policy_version 1845426 (0.0009) [2023-12-27 04:50:26,156][105692] Updated weights for policy 0, policy_version 1845436 (0.0009) [2023-12-27 04:50:26,211][105692] Updated weights for policy 0, policy_version 1845446 (0.0009) [2023-12-27 04:50:26,265][105692] Updated weights for policy 0, policy_version 1845456 (0.0009) [2023-12-27 04:50:26,594][105620] Updated weights for policy 1, policy_version 1849757 (0.0009) [2023-12-27 04:50:26,648][105620] Updated weights for policy 1, policy_version 1849767 (0.0008) [2023-12-27 04:50:26,706][105620] Updated weights for policy 1, policy_version 1849777 (0.0010) [2023-12-27 04:50:26,970][105692] Updated weights for policy 0, policy_version 1845466 (0.0005) [2023-12-27 04:50:27,032][105692] Updated weights for policy 0, policy_version 1845476 (0.0008) [2023-12-27 04:50:27,086][105692] Updated weights for policy 0, policy_version 1845486 (0.0009) [2023-12-27 04:50:27,487][105620] Updated weights for policy 1, policy_version 1849787 (0.0010) [2023-12-27 04:50:27,545][105620] Updated weights for policy 1, policy_version 1849797 (0.0009) [2023-12-27 04:50:27,599][105620] Updated weights for policy 1, policy_version 1849807 (0.0009) [2023-12-27 04:50:27,800][105692] Updated weights for policy 0, policy_version 1845496 (0.0009) [2023-12-27 04:50:27,849][105692] Updated weights for policy 0, policy_version 1845506 (0.0008) [2023-12-27 04:50:27,910][105692] Updated weights for policy 0, policy_version 1845516 (0.0009) [2023-12-27 04:50:28,346][105620] Updated weights for policy 1, policy_version 1849817 (0.0008) [2023-12-27 04:50:28,402][105620] Updated weights for policy 1, policy_version 1849827 (0.0006) [2023-12-27 04:50:28,467][105620] Updated weights for policy 1, policy_version 1849837 (0.0005) [2023-12-27 04:50:28,538][105620] Updated weights for policy 1, policy_version 1849847 (0.0009) [2023-12-27 04:50:28,651][105692] Updated weights for policy 0, policy_version 1845526 (0.0007) [2023-12-27 04:50:28,705][105692] Updated weights for policy 0, policy_version 1845536 (0.0005) [2023-12-27 04:50:28,765][105692] Updated weights for policy 0, policy_version 1845546 (0.0008) [2023-12-27 04:50:29,125][105620] Updated weights for policy 1, policy_version 1849857 (0.0009) [2023-12-27 04:50:29,175][105620] Updated weights for policy 1, policy_version 1849867 (0.0006) [2023-12-27 04:50:29,232][105620] Updated weights for policy 1, policy_version 1849877 (0.0006) [2023-12-27 04:50:29,569][105692] Updated weights for policy 0, policy_version 1845556 (0.0010) [2023-12-27 04:50:29,627][105692] Updated weights for policy 0, policy_version 1845566 (0.0009) [2023-12-27 04:50:29,689][105692] Updated weights for policy 0, policy_version 1845576 (0.0009) [2023-12-27 04:50:29,867][105620] Updated weights for policy 1, policy_version 1849887 (0.0009) [2023-12-27 04:50:29,924][105620] Updated weights for policy 1, policy_version 1849897 (0.0009) [2023-12-27 04:50:29,979][105620] Updated weights for policy 1, policy_version 1849907 (0.0009) [2023-12-27 04:50:30,519][105692] Updated weights for policy 0, policy_version 1845586 (0.0009) [2023-12-27 04:50:30,571][105692] Updated weights for policy 0, policy_version 1845596 (0.0008) [2023-12-27 04:50:30,576][105620] Updated weights for policy 1, policy_version 1849917 (0.0007) [2023-12-27 04:50:30,623][105692] Updated weights for policy 0, policy_version 1845606 (0.0006) [2023-12-27 04:50:30,635][105620] Updated weights for policy 1, policy_version 1849927 (0.0008) [2023-12-27 04:50:30,676][105692] Updated weights for policy 0, policy_version 1845616 (0.0008) [2023-12-27 04:50:30,699][105620] Updated weights for policy 1, policy_version 1849937 (0.0008) [2023-12-27 04:50:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19272.0). Total num frames: 946200576. Throughput: 0: 9400.9, 1: 10095.1. Samples: 946170016. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:50:31,062][104569] Avg episode reward: [(0, '8624.044'), (1, '9163.925')] [2023-12-27 04:50:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001845616_472547328.pth... [2023-12-27 04:50:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001849944_473653248.pth... [2023-12-27 04:50:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001848760_473350144.pth [2023-12-27 04:50:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001844528_472268800.pth [2023-12-27 04:50:31,339][105692] Updated weights for policy 0, policy_version 1845626 (0.0009) [2023-12-27 04:50:31,405][105692] Updated weights for policy 0, policy_version 1845636 (0.0008) [2023-12-27 04:50:31,437][105620] Updated weights for policy 1, policy_version 1849947 (0.0007) [2023-12-27 04:50:31,469][105692] Updated weights for policy 0, policy_version 1845646 (0.0008) [2023-12-27 04:50:31,496][105620] Updated weights for policy 1, policy_version 1849957 (0.0009) [2023-12-27 04:50:31,557][105620] Updated weights for policy 1, policy_version 1849967 (0.0009) [2023-12-27 04:50:32,211][105692] Updated weights for policy 0, policy_version 1845656 (0.0005) [2023-12-27 04:50:32,275][105692] Updated weights for policy 0, policy_version 1845666 (0.0007) [2023-12-27 04:50:32,335][105620] Updated weights for policy 1, policy_version 1849977 (0.0008) [2023-12-27 04:50:32,337][105692] Updated weights for policy 0, policy_version 1845676 (0.0009) [2023-12-27 04:50:32,400][105620] Updated weights for policy 1, policy_version 1849987 (0.0008) [2023-12-27 04:50:32,459][105620] Updated weights for policy 1, policy_version 1849997 (0.0008) [2023-12-27 04:50:32,508][105620] Updated weights for policy 1, policy_version 1850007 (0.0008) [2023-12-27 04:50:33,116][105692] Updated weights for policy 0, policy_version 1845686 (0.0008) [2023-12-27 04:50:33,139][105620] Updated weights for policy 1, policy_version 1850017 (0.0006) [2023-12-27 04:50:33,172][105692] Updated weights for policy 0, policy_version 1845696 (0.0008) [2023-12-27 04:50:33,189][105620] Updated weights for policy 1, policy_version 1850027 (0.0005) [2023-12-27 04:50:33,218][105692] Updated weights for policy 0, policy_version 1845706 (0.0008) [2023-12-27 04:50:33,238][105620] Updated weights for policy 1, policy_version 1850037 (0.0006) [2023-12-27 04:50:33,792][105620] Updated weights for policy 1, policy_version 1850047 (0.0009) [2023-12-27 04:50:33,859][105620] Updated weights for policy 1, policy_version 1850057 (0.0010) [2023-12-27 04:50:33,915][105620] Updated weights for policy 1, policy_version 1850067 (0.0010) [2023-12-27 04:50:34,086][105692] Updated weights for policy 0, policy_version 1845716 (0.0009) [2023-12-27 04:50:34,148][105692] Updated weights for policy 0, policy_version 1845726 (0.0009) [2023-12-27 04:50:34,209][105692] Updated weights for policy 0, policy_version 1845736 (0.0007) [2023-12-27 04:50:34,478][105620] Updated weights for policy 1, policy_version 1850077 (0.0008) [2023-12-27 04:50:34,538][105620] Updated weights for policy 1, policy_version 1850087 (0.0010) [2023-12-27 04:50:34,601][105620] Updated weights for policy 1, policy_version 1850097 (0.0008) [2023-12-27 04:50:34,957][105692] Updated weights for policy 0, policy_version 1845746 (0.0006) [2023-12-27 04:50:35,011][105692] Updated weights for policy 0, policy_version 1845756 (0.0009) [2023-12-27 04:50:35,074][105692] Updated weights for policy 0, policy_version 1845766 (0.0009) [2023-12-27 04:50:35,139][105692] Updated weights for policy 0, policy_version 1845776 (0.0010) [2023-12-27 04:50:35,272][105620] Updated weights for policy 1, policy_version 1850107 (0.0007) [2023-12-27 04:50:35,337][105620] Updated weights for policy 1, policy_version 1850117 (0.0005) [2023-12-27 04:50:35,399][105620] Updated weights for policy 1, policy_version 1850127 (0.0005) [2023-12-27 04:50:35,973][105692] Updated weights for policy 0, policy_version 1845786 (0.0005) [2023-12-27 04:50:36,030][105692] Updated weights for policy 0, policy_version 1845796 (0.0006) [2023-12-27 04:50:36,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.8, 300 sec: 19244.3). Total num frames: 946290688. Throughput: 0: 9392.7, 1: 10130.5. Samples: 946287472. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:50:36,062][104569] Avg episode reward: [(0, '8899.530'), (1, '9163.924')] [2023-12-27 04:50:36,068][105620] Updated weights for policy 1, policy_version 1850137 (0.0011) [2023-12-27 04:50:36,096][105692] Updated weights for policy 0, policy_version 1845806 (0.0009) [2023-12-27 04:50:36,129][105620] Updated weights for policy 1, policy_version 1850147 (0.0011) [2023-12-27 04:50:36,189][105620] Updated weights for policy 1, policy_version 1850157 (0.0007) [2023-12-27 04:50:36,259][105620] Updated weights for policy 1, policy_version 1850167 (0.0005) [2023-12-27 04:50:36,810][105692] Updated weights for policy 0, policy_version 1845816 (0.0008) [2023-12-27 04:50:36,876][105692] Updated weights for policy 0, policy_version 1845826 (0.0009) [2023-12-27 04:50:36,933][105620] Updated weights for policy 1, policy_version 1850177 (0.0006) [2023-12-27 04:50:36,935][105692] Updated weights for policy 0, policy_version 1845836 (0.0008) [2023-12-27 04:50:36,982][105620] Updated weights for policy 1, policy_version 1850187 (0.0008) [2023-12-27 04:50:37,042][105620] Updated weights for policy 1, policy_version 1850197 (0.0008) [2023-12-27 04:50:37,684][105692] Updated weights for policy 0, policy_version 1845846 (0.0010) [2023-12-27 04:50:37,735][105692] Updated weights for policy 0, policy_version 1845856 (0.0009) [2023-12-27 04:50:37,785][105692] Updated weights for policy 0, policy_version 1845866 (0.0008) [2023-12-27 04:50:37,837][105620] Updated weights for policy 1, policy_version 1850207 (0.0006) [2023-12-27 04:50:37,893][105620] Updated weights for policy 1, policy_version 1850217 (0.0005) [2023-12-27 04:50:37,963][105620] Updated weights for policy 1, policy_version 1850227 (0.0008) [2023-12-27 04:50:38,453][105692] Updated weights for policy 0, policy_version 1845876 (0.0009) [2023-12-27 04:50:38,519][105692] Updated weights for policy 0, policy_version 1845886 (0.0005) [2023-12-27 04:50:38,586][105692] Updated weights for policy 0, policy_version 1845896 (0.0006) [2023-12-27 04:50:38,718][105620] Updated weights for policy 1, policy_version 1850237 (0.0007) [2023-12-27 04:50:38,769][105620] Updated weights for policy 1, policy_version 1850247 (0.0010) [2023-12-27 04:50:38,823][105620] Updated weights for policy 1, policy_version 1850257 (0.0005) [2023-12-27 04:50:39,282][105692] Updated weights for policy 0, policy_version 1845906 (0.0009) [2023-12-27 04:50:39,352][105692] Updated weights for policy 0, policy_version 1845916 (0.0012) [2023-12-27 04:50:39,420][105692] Updated weights for policy 0, policy_version 1845926 (0.0013) [2023-12-27 04:50:39,469][105620] Updated weights for policy 1, policy_version 1850267 (0.0007) [2023-12-27 04:50:39,480][105692] Updated weights for policy 0, policy_version 1845936 (0.0011) [2023-12-27 04:50:39,532][105620] Updated weights for policy 1, policy_version 1850277 (0.0011) [2023-12-27 04:50:39,592][105620] Updated weights for policy 1, policy_version 1850287 (0.0010) [2023-12-27 04:50:40,261][105692] Updated weights for policy 0, policy_version 1845946 (0.0007) [2023-12-27 04:50:40,324][105620] Updated weights for policy 1, policy_version 1850297 (0.0010) [2023-12-27 04:50:40,329][105692] Updated weights for policy 0, policy_version 1845956 (0.0005) [2023-12-27 04:50:40,387][105620] Updated weights for policy 1, policy_version 1850307 (0.0010) [2023-12-27 04:50:40,420][105692] Updated weights for policy 0, policy_version 1845966 (0.0006) [2023-12-27 04:50:40,457][105620] Updated weights for policy 1, policy_version 1850317 (0.0011) [2023-12-27 04:50:40,513][105620] Updated weights for policy 1, policy_version 1850327 (0.0010) [2023-12-27 04:50:40,973][105692] Updated weights for policy 0, policy_version 1845976 (0.0006) [2023-12-27 04:50:41,036][105692] Updated weights for policy 0, policy_version 1845986 (0.0007) [2023-12-27 04:50:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19244.3). Total num frames: 946388992. Throughput: 0: 9411.8, 1: 10041.3. Samples: 946402816. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:50:41,062][104569] Avg episode reward: [(0, '8446.556'), (1, '9256.303')] [2023-12-27 04:50:41,103][105692] Updated weights for policy 0, policy_version 1845996 (0.0011) [2023-12-27 04:50:41,267][105620] Updated weights for policy 1, policy_version 1850337 (0.0010) [2023-12-27 04:50:41,333][105620] Updated weights for policy 1, policy_version 1850347 (0.0010) [2023-12-27 04:50:41,401][105620] Updated weights for policy 1, policy_version 1850357 (0.0011) [2023-12-27 04:50:41,762][105692] Updated weights for policy 0, policy_version 1846006 (0.0009) [2023-12-27 04:50:41,815][105692] Updated weights for policy 0, policy_version 1846016 (0.0008) [2023-12-27 04:50:41,863][105692] Updated weights for policy 0, policy_version 1846026 (0.0008) [2023-12-27 04:50:42,146][105620] Updated weights for policy 1, policy_version 1850368 (0.0010) [2023-12-27 04:50:42,209][105620] Updated weights for policy 1, policy_version 1850378 (0.0010) [2023-12-27 04:50:42,272][105620] Updated weights for policy 1, policy_version 1850388 (0.0011) [2023-12-27 04:50:42,701][105692] Updated weights for policy 0, policy_version 1846036 (0.0009) [2023-12-27 04:50:42,758][105692] Updated weights for policy 0, policy_version 1846046 (0.0008) [2023-12-27 04:50:42,817][105692] Updated weights for policy 0, policy_version 1846056 (0.0009) [2023-12-27 04:50:42,947][105620] Updated weights for policy 1, policy_version 1850398 (0.0007) [2023-12-27 04:50:43,002][105620] Updated weights for policy 1, policy_version 1850408 (0.0005) [2023-12-27 04:50:43,062][105620] Updated weights for policy 1, policy_version 1850418 (0.0007) [2023-12-27 04:50:43,611][105692] Updated weights for policy 0, policy_version 1846066 (0.0008) [2023-12-27 04:50:43,665][105692] Updated weights for policy 0, policy_version 1846076 (0.0009) [2023-12-27 04:50:43,670][105620] Updated weights for policy 1, policy_version 1850428 (0.0006) [2023-12-27 04:50:43,717][105620] Updated weights for policy 1, policy_version 1850438 (0.0005) [2023-12-27 04:50:43,718][105692] Updated weights for policy 0, policy_version 1846086 (0.0006) [2023-12-27 04:50:43,772][105692] Updated weights for policy 0, policy_version 1846096 (0.0005) [2023-12-27 04:50:43,779][105620] Updated weights for policy 1, policy_version 1850448 (0.0006) [2023-12-27 04:50:44,426][105692] Updated weights for policy 0, policy_version 1846106 (0.0008) [2023-12-27 04:50:44,465][105620] Updated weights for policy 1, policy_version 1850458 (0.0006) [2023-12-27 04:50:44,483][105692] Updated weights for policy 0, policy_version 1846116 (0.0007) [2023-12-27 04:50:44,513][105620] Updated weights for policy 1, policy_version 1850468 (0.0010) [2023-12-27 04:50:44,535][105692] Updated weights for policy 0, policy_version 1846126 (0.0005) [2023-12-27 04:50:44,575][105620] Updated weights for policy 1, policy_version 1850478 (0.0010) [2023-12-27 04:50:44,636][105620] Updated weights for policy 1, policy_version 1850488 (0.0010) [2023-12-27 04:50:45,227][105692] Updated weights for policy 0, policy_version 1846136 (0.0007) [2023-12-27 04:50:45,288][105692] Updated weights for policy 0, policy_version 1846146 (0.0008) [2023-12-27 04:50:45,338][105692] Updated weights for policy 0, policy_version 1846156 (0.0008) [2023-12-27 04:50:45,347][105620] Updated weights for policy 1, policy_version 1850498 (0.0010) [2023-12-27 04:50:45,414][105620] Updated weights for policy 1, policy_version 1850508 (0.0011) [2023-12-27 04:50:45,487][105620] Updated weights for policy 1, policy_version 1850518 (0.0011) [2023-12-27 04:50:46,030][105692] Updated weights for policy 0, policy_version 1846166 (0.0008) [2023-12-27 04:50:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19244.3). Total num frames: 946487296. Throughput: 0: 9390.0, 1: 10056.0. Samples: 946460972. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:50:46,062][104569] Avg episode reward: [(0, '7904.854'), (1, '9256.152')] [2023-12-27 04:50:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001850520_473800704.pth... [2023-12-27 04:50:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001849368_473505792.pth [2023-12-27 04:50:46,090][105692] Updated weights for policy 0, policy_version 1846176 (0.0008) [2023-12-27 04:50:46,158][105692] Updated weights for policy 0, policy_version 1846186 (0.0008) [2023-12-27 04:50:46,191][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001846192_472694784.pth... [2023-12-27 04:50:46,196][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001845072_472408064.pth [2023-12-27 04:50:46,219][105620] Updated weights for policy 1, policy_version 1850528 (0.0010) [2023-12-27 04:50:46,277][105620] Updated weights for policy 1, policy_version 1850538 (0.0010) [2023-12-27 04:50:46,335][105620] Updated weights for policy 1, policy_version 1850548 (0.0010) [2023-12-27 04:50:46,900][105692] Updated weights for policy 0, policy_version 1846196 (0.0008) [2023-12-27 04:50:46,951][105692] Updated weights for policy 0, policy_version 1846206 (0.0008) [2023-12-27 04:50:46,995][105692] Updated weights for policy 0, policy_version 1846216 (0.0008) [2023-12-27 04:50:47,061][105620] Updated weights for policy 1, policy_version 1850558 (0.0010) [2023-12-27 04:50:47,109][105620] Updated weights for policy 1, policy_version 1850568 (0.0010) [2023-12-27 04:50:47,157][105620] Updated weights for policy 1, policy_version 1850578 (0.0010) [2023-12-27 04:50:47,817][105620] Updated weights for policy 1, policy_version 1850588 (0.0008) [2023-12-27 04:50:47,827][105692] Updated weights for policy 0, policy_version 1846226 (0.0008) [2023-12-27 04:50:47,876][105620] Updated weights for policy 1, policy_version 1850598 (0.0005) [2023-12-27 04:50:47,884][105692] Updated weights for policy 0, policy_version 1846236 (0.0009) [2023-12-27 04:50:47,935][105620] Updated weights for policy 1, policy_version 1850608 (0.0005) [2023-12-27 04:50:47,944][105692] Updated weights for policy 0, policy_version 1846246 (0.0009) [2023-12-27 04:50:48,009][105692] Updated weights for policy 0, policy_version 1846256 (0.0009) [2023-12-27 04:50:48,613][105620] Updated weights for policy 1, policy_version 1850618 (0.0006) [2023-12-27 04:50:48,671][105620] Updated weights for policy 1, policy_version 1850628 (0.0009) [2023-12-27 04:50:48,731][105620] Updated weights for policy 1, policy_version 1850638 (0.0009) [2023-12-27 04:50:48,775][105692] Updated weights for policy 0, policy_version 1846266 (0.0010) [2023-12-27 04:50:48,786][105620] Updated weights for policy 1, policy_version 1850648 (0.0008) [2023-12-27 04:50:48,838][105692] Updated weights for policy 0, policy_version 1846276 (0.0010) [2023-12-27 04:50:48,903][105692] Updated weights for policy 0, policy_version 1846286 (0.0009) [2023-12-27 04:50:49,489][105620] Updated weights for policy 1, policy_version 1850658 (0.0008) [2023-12-27 04:50:49,545][105620] Updated weights for policy 1, policy_version 1850668 (0.0010) [2023-12-27 04:50:49,600][105620] Updated weights for policy 1, policy_version 1850678 (0.0010) [2023-12-27 04:50:49,628][105692] Updated weights for policy 0, policy_version 1846296 (0.0006) [2023-12-27 04:50:49,688][105692] Updated weights for policy 0, policy_version 1846306 (0.0009) [2023-12-27 04:50:49,743][105692] Updated weights for policy 0, policy_version 1846316 (0.0010) [2023-12-27 04:50:50,368][105620] Updated weights for policy 1, policy_version 1850688 (0.0008) [2023-12-27 04:50:50,425][105620] Updated weights for policy 1, policy_version 1850698 (0.0007) [2023-12-27 04:50:50,477][105692] Updated weights for policy 0, policy_version 1846326 (0.0011) [2023-12-27 04:50:50,479][105620] Updated weights for policy 1, policy_version 1850708 (0.0006) [2023-12-27 04:50:50,533][105692] Updated weights for policy 0, policy_version 1846336 (0.0010) [2023-12-27 04:50:50,599][105692] Updated weights for policy 0, policy_version 1846346 (0.0008) [2023-12-27 04:50:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19244.3). Total num frames: 946585600. Throughput: 0: 9435.5, 1: 10044.3. Samples: 946577416. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:50:51,062][104569] Avg episode reward: [(0, '8354.349'), (1, '9256.178')] [2023-12-27 04:50:51,197][105620] Updated weights for policy 1, policy_version 1850718 (0.0007) [2023-12-27 04:50:51,249][105620] Updated weights for policy 1, policy_version 1850728 (0.0008) [2023-12-27 04:50:51,308][105620] Updated weights for policy 1, policy_version 1850738 (0.0008) [2023-12-27 04:50:51,314][105692] Updated weights for policy 0, policy_version 1846356 (0.0009) [2023-12-27 04:50:51,376][105692] Updated weights for policy 0, policy_version 1846366 (0.0011) [2023-12-27 04:50:51,433][105692] Updated weights for policy 0, policy_version 1846376 (0.0010) [2023-12-27 04:50:51,996][105620] Updated weights for policy 1, policy_version 1850748 (0.0007) [2023-12-27 04:50:52,058][105620] Updated weights for policy 1, policy_version 1850758 (0.0008) [2023-12-27 04:50:52,120][105620] Updated weights for policy 1, policy_version 1850768 (0.0009) [2023-12-27 04:50:52,181][105692] Updated weights for policy 0, policy_version 1846386 (0.0009) [2023-12-27 04:50:52,244][105692] Updated weights for policy 0, policy_version 1846396 (0.0008) [2023-12-27 04:50:52,303][105692] Updated weights for policy 0, policy_version 1846406 (0.0010) [2023-12-27 04:50:52,364][105692] Updated weights for policy 0, policy_version 1846416 (0.0012) [2023-12-27 04:50:52,895][105620] Updated weights for policy 1, policy_version 1850778 (0.0010) [2023-12-27 04:50:52,960][105620] Updated weights for policy 1, policy_version 1850788 (0.0008) [2023-12-27 04:50:53,031][105620] Updated weights for policy 1, policy_version 1850798 (0.0005) [2023-12-27 04:50:53,098][105620] Updated weights for policy 1, policy_version 1850808 (0.0005) [2023-12-27 04:50:53,100][105692] Updated weights for policy 0, policy_version 1846426 (0.0009) [2023-12-27 04:50:53,159][105692] Updated weights for policy 0, policy_version 1846436 (0.0008) [2023-12-27 04:50:53,213][105692] Updated weights for policy 0, policy_version 1846446 (0.0005) [2023-12-27 04:50:53,737][105620] Updated weights for policy 1, policy_version 1850818 (0.0011) [2023-12-27 04:50:53,796][105620] Updated weights for policy 1, policy_version 1850828 (0.0011) [2023-12-27 04:50:53,862][105620] Updated weights for policy 1, policy_version 1850838 (0.0011) [2023-12-27 04:50:53,930][105692] Updated weights for policy 0, policy_version 1846456 (0.0005) [2023-12-27 04:50:53,983][105692] Updated weights for policy 0, policy_version 1846466 (0.0005) [2023-12-27 04:50:54,028][105692] Updated weights for policy 0, policy_version 1846476 (0.0005) [2023-12-27 04:50:54,590][105620] Updated weights for policy 1, policy_version 1850848 (0.0009) [2023-12-27 04:50:54,643][105620] Updated weights for policy 1, policy_version 1850858 (0.0005) [2023-12-27 04:50:54,696][105692] Updated weights for policy 0, policy_version 1846486 (0.0007) [2023-12-27 04:50:54,699][105620] Updated weights for policy 1, policy_version 1850868 (0.0007) [2023-12-27 04:50:54,754][105692] Updated weights for policy 0, policy_version 1846496 (0.0007) [2023-12-27 04:50:54,816][105692] Updated weights for policy 0, policy_version 1846506 (0.0008) [2023-12-27 04:50:55,367][105620] Updated weights for policy 1, policy_version 1850878 (0.0008) [2023-12-27 04:50:55,419][105620] Updated weights for policy 1, policy_version 1850888 (0.0006) [2023-12-27 04:50:55,478][105620] Updated weights for policy 1, policy_version 1850898 (0.0006) [2023-12-27 04:50:55,612][105692] Updated weights for policy 0, policy_version 1846516 (0.0008) [2023-12-27 04:50:55,663][105692] Updated weights for policy 0, policy_version 1846526 (0.0006) [2023-12-27 04:50:55,715][105692] Updated weights for policy 0, policy_version 1846536 (0.0005) [2023-12-27 04:50:56,022][105620] Updated weights for policy 1, policy_version 1850908 (0.0006) [2023-12-27 04:50:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19272.0). Total num frames: 946683904. Throughput: 0: 9390.7, 1: 10015.8. Samples: 946695072. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:50:56,062][104569] Avg episode reward: [(0, '8444.152'), (1, '9348.865')] [2023-12-27 04:50:56,076][105620] Updated weights for policy 1, policy_version 1850918 (0.0005) [2023-12-27 04:50:56,136][105620] Updated weights for policy 1, policy_version 1850928 (0.0006) [2023-12-27 04:50:56,270][105692] Updated weights for policy 0, policy_version 1846546 (0.0006) [2023-12-27 04:50:56,319][105692] Updated weights for policy 0, policy_version 1846556 (0.0005) [2023-12-27 04:50:56,377][105692] Updated weights for policy 0, policy_version 1846566 (0.0005) [2023-12-27 04:50:56,430][105692] Updated weights for policy 0, policy_version 1846576 (0.0006) [2023-12-27 04:50:56,763][105620] Updated weights for policy 1, policy_version 1850938 (0.0006) [2023-12-27 04:50:56,824][105620] Updated weights for policy 1, policy_version 1850948 (0.0010) [2023-12-27 04:50:56,891][105620] Updated weights for policy 1, policy_version 1850958 (0.0006) [2023-12-27 04:50:56,957][105620] Updated weights for policy 1, policy_version 1850968 (0.0007) [2023-12-27 04:50:57,018][105692] Updated weights for policy 0, policy_version 1846586 (0.0010) [2023-12-27 04:50:57,069][105692] Updated weights for policy 0, policy_version 1846596 (0.0010) [2023-12-27 04:50:57,131][105692] Updated weights for policy 0, policy_version 1846606 (0.0010) [2023-12-27 04:50:57,562][105620] Updated weights for policy 1, policy_version 1850978 (0.0009) [2023-12-27 04:50:57,610][105620] Updated weights for policy 1, policy_version 1850988 (0.0008) [2023-12-27 04:50:57,653][105620] Updated weights for policy 1, policy_version 1850998 (0.0005) [2023-12-27 04:50:57,773][105692] Updated weights for policy 0, policy_version 1846616 (0.0010) [2023-12-27 04:50:57,838][105692] Updated weights for policy 0, policy_version 1846626 (0.0010) [2023-12-27 04:50:57,886][105692] Updated weights for policy 0, policy_version 1846636 (0.0010) [2023-12-27 04:50:58,242][105620] Updated weights for policy 1, policy_version 1851008 (0.0007) [2023-12-27 04:50:58,302][105620] Updated weights for policy 1, policy_version 1851018 (0.0008) [2023-12-27 04:50:58,369][105620] Updated weights for policy 1, policy_version 1851028 (0.0009) [2023-12-27 04:50:58,658][105692] Updated weights for policy 0, policy_version 1846646 (0.0009) [2023-12-27 04:50:58,718][105692] Updated weights for policy 0, policy_version 1846656 (0.0008) [2023-12-27 04:50:58,782][105692] Updated weights for policy 0, policy_version 1846666 (0.0009) [2023-12-27 04:50:59,145][105620] Updated weights for policy 1, policy_version 1851038 (0.0008) [2023-12-27 04:50:59,204][105620] Updated weights for policy 1, policy_version 1851048 (0.0008) [2023-12-27 04:50:59,268][105620] Updated weights for policy 1, policy_version 1851058 (0.0008) [2023-12-27 04:50:59,615][105692] Updated weights for policy 0, policy_version 1846676 (0.0010) [2023-12-27 04:50:59,677][105692] Updated weights for policy 0, policy_version 1846686 (0.0009) [2023-12-27 04:50:59,736][105692] Updated weights for policy 0, policy_version 1846696 (0.0006) [2023-12-27 04:51:00,046][105620] Updated weights for policy 1, policy_version 1851068 (0.0008) [2023-12-27 04:51:00,096][105620] Updated weights for policy 1, policy_version 1851078 (0.0008) [2023-12-27 04:51:00,151][105620] Updated weights for policy 1, policy_version 1851088 (0.0008) [2023-12-27 04:51:00,403][105692] Updated weights for policy 0, policy_version 1846706 (0.0006) [2023-12-27 04:51:00,461][105692] Updated weights for policy 0, policy_version 1846716 (0.0009) [2023-12-27 04:51:00,513][105692] Updated weights for policy 0, policy_version 1846726 (0.0009) [2023-12-27 04:51:00,564][105692] Updated weights for policy 0, policy_version 1846736 (0.0010) [2023-12-27 04:51:00,953][105620] Updated weights for policy 1, policy_version 1851098 (0.0010) [2023-12-27 04:51:01,008][105620] Updated weights for policy 1, policy_version 1851108 (0.0010) [2023-12-27 04:51:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19299.8). Total num frames: 946782208. Throughput: 0: 9482.5, 1: 9994.8. Samples: 946759032. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:51:01,062][104569] Avg episode reward: [(0, '8263.677'), (1, '9164.258')] [2023-12-27 04:51:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001846736_472834048.pth... [2023-12-27 04:51:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001845616_472547328.pth [2023-12-27 04:51:01,076][105620] Updated weights for policy 1, policy_version 1851118 (0.0008) [2023-12-27 04:51:01,136][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001851128_473956352.pth... [2023-12-27 04:51:01,138][105620] Updated weights for policy 1, policy_version 1851128 (0.0007) [2023-12-27 04:51:01,140][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001849944_473653248.pth [2023-12-27 04:51:01,218][105692] Updated weights for policy 0, policy_version 1846746 (0.0009) [2023-12-27 04:51:01,275][105692] Updated weights for policy 0, policy_version 1846756 (0.0009) [2023-12-27 04:51:01,326][105692] Updated weights for policy 0, policy_version 1846766 (0.0009) [2023-12-27 04:51:01,831][105620] Updated weights for policy 1, policy_version 1851138 (0.0008) [2023-12-27 04:51:01,893][105620] Updated weights for policy 1, policy_version 1851148 (0.0010) [2023-12-27 04:51:01,947][105620] Updated weights for policy 1, policy_version 1851158 (0.0010) [2023-12-27 04:51:02,085][105692] Updated weights for policy 0, policy_version 1846776 (0.0008) [2023-12-27 04:51:02,133][105692] Updated weights for policy 0, policy_version 1846786 (0.0008) [2023-12-27 04:51:02,182][105692] Updated weights for policy 0, policy_version 1846796 (0.0008) [2023-12-27 04:51:02,677][105620] Updated weights for policy 1, policy_version 1851168 (0.0006) [2023-12-27 04:51:02,726][105620] Updated weights for policy 1, policy_version 1851178 (0.0006) [2023-12-27 04:51:02,779][105620] Updated weights for policy 1, policy_version 1851188 (0.0008) [2023-12-27 04:51:02,981][105692] Updated weights for policy 0, policy_version 1846806 (0.0009) [2023-12-27 04:51:03,036][105692] Updated weights for policy 0, policy_version 1846816 (0.0010) [2023-12-27 04:51:03,091][105692] Updated weights for policy 0, policy_version 1846826 (0.0010) [2023-12-27 04:51:03,475][105620] Updated weights for policy 1, policy_version 1851198 (0.0009) [2023-12-27 04:51:03,528][105620] Updated weights for policy 1, policy_version 1851208 (0.0010) [2023-12-27 04:51:03,585][105620] Updated weights for policy 1, policy_version 1851218 (0.0010) [2023-12-27 04:51:03,727][105692] Updated weights for policy 0, policy_version 1846836 (0.0008) [2023-12-27 04:51:03,776][105692] Updated weights for policy 0, policy_version 1846846 (0.0008) [2023-12-27 04:51:03,820][105692] Updated weights for policy 0, policy_version 1846856 (0.0010) [2023-12-27 04:51:04,348][105620] Updated weights for policy 1, policy_version 1851228 (0.0008) [2023-12-27 04:51:04,412][105620] Updated weights for policy 1, policy_version 1851238 (0.0006) [2023-12-27 04:51:04,467][105620] Updated weights for policy 1, policy_version 1851248 (0.0008) [2023-12-27 04:51:04,577][105692] Updated weights for policy 0, policy_version 1846866 (0.0011) [2023-12-27 04:51:04,636][105692] Updated weights for policy 0, policy_version 1846876 (0.0010) [2023-12-27 04:51:04,695][105692] Updated weights for policy 0, policy_version 1846886 (0.0010) [2023-12-27 04:51:04,755][105692] Updated weights for policy 0, policy_version 1846896 (0.0010) [2023-12-27 04:51:05,079][105620] Updated weights for policy 1, policy_version 1851258 (0.0008) [2023-12-27 04:51:05,136][105620] Updated weights for policy 1, policy_version 1851268 (0.0009) [2023-12-27 04:51:05,187][105620] Updated weights for policy 1, policy_version 1851278 (0.0010) [2023-12-27 04:51:05,239][105620] Updated weights for policy 1, policy_version 1851288 (0.0010) [2023-12-27 04:51:05,507][105692] Updated weights for policy 0, policy_version 1846906 (0.0011) [2023-12-27 04:51:05,565][105692] Updated weights for policy 0, policy_version 1846916 (0.0010) [2023-12-27 04:51:05,626][105692] Updated weights for policy 0, policy_version 1846926 (0.0009) [2023-12-27 04:51:05,909][105620] Updated weights for policy 1, policy_version 1851298 (0.0005) [2023-12-27 04:51:05,964][105620] Updated weights for policy 1, policy_version 1851308 (0.0006) [2023-12-27 04:51:06,028][105620] Updated weights for policy 1, policy_version 1851318 (0.0008) [2023-12-27 04:51:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19355.3). Total num frames: 946888704. Throughput: 0: 9494.7, 1: 9975.7. Samples: 946873848. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:51:06,062][104569] Avg episode reward: [(0, '8717.000'), (1, '9164.182')] [2023-12-27 04:51:06,333][105692] Updated weights for policy 0, policy_version 1846936 (0.0008) [2023-12-27 04:51:06,390][105692] Updated weights for policy 0, policy_version 1846946 (0.0006) [2023-12-27 04:51:06,445][105692] Updated weights for policy 0, policy_version 1846956 (0.0006) [2023-12-27 04:51:06,713][105620] Updated weights for policy 1, policy_version 1851328 (0.0010) [2023-12-27 04:51:06,765][105620] Updated weights for policy 1, policy_version 1851338 (0.0010) [2023-12-27 04:51:06,814][105620] Updated weights for policy 1, policy_version 1851348 (0.0011) [2023-12-27 04:51:07,140][105692] Updated weights for policy 0, policy_version 1846966 (0.0009) [2023-12-27 04:51:07,192][105692] Updated weights for policy 0, policy_version 1846976 (0.0009) [2023-12-27 04:51:07,240][105692] Updated weights for policy 0, policy_version 1846986 (0.0007) [2023-12-27 04:51:07,564][105620] Updated weights for policy 1, policy_version 1851358 (0.0010) [2023-12-27 04:51:07,609][105620] Updated weights for policy 1, policy_version 1851368 (0.0010) [2023-12-27 04:51:07,660][105620] Updated weights for policy 1, policy_version 1851378 (0.0010) [2023-12-27 04:51:08,019][105692] Updated weights for policy 0, policy_version 1846996 (0.0007) [2023-12-27 04:51:08,082][105692] Updated weights for policy 0, policy_version 1847006 (0.0008) [2023-12-27 04:51:08,135][105692] Updated weights for policy 0, policy_version 1847016 (0.0007) [2023-12-27 04:51:08,352][105620] Updated weights for policy 1, policy_version 1851388 (0.0009) [2023-12-27 04:51:08,414][105620] Updated weights for policy 1, policy_version 1851398 (0.0006) [2023-12-27 04:51:08,470][105620] Updated weights for policy 1, policy_version 1851408 (0.0005) [2023-12-27 04:51:08,873][105692] Updated weights for policy 0, policy_version 1847026 (0.0006) [2023-12-27 04:51:08,926][105692] Updated weights for policy 0, policy_version 1847036 (0.0009) [2023-12-27 04:51:08,981][105692] Updated weights for policy 0, policy_version 1847046 (0.0010) [2023-12-27 04:51:09,038][105692] Updated weights for policy 0, policy_version 1847056 (0.0010) [2023-12-27 04:51:09,105][105620] Updated weights for policy 1, policy_version 1851418 (0.0010) [2023-12-27 04:51:09,164][105620] Updated weights for policy 1, policy_version 1851428 (0.0010) [2023-12-27 04:51:09,227][105620] Updated weights for policy 1, policy_version 1851438 (0.0010) [2023-12-27 04:51:09,287][105620] Updated weights for policy 1, policy_version 1851448 (0.0009) [2023-12-27 04:51:09,856][105692] Updated weights for policy 0, policy_version 1847066 (0.0009) [2023-12-27 04:51:09,921][105692] Updated weights for policy 0, policy_version 1847076 (0.0009) [2023-12-27 04:51:09,988][105692] Updated weights for policy 0, policy_version 1847086 (0.0009) [2023-12-27 04:51:10,102][105620] Updated weights for policy 1, policy_version 1851458 (0.0009) [2023-12-27 04:51:10,168][105620] Updated weights for policy 1, policy_version 1851468 (0.0008) [2023-12-27 04:51:10,234][105620] Updated weights for policy 1, policy_version 1851478 (0.0008) [2023-12-27 04:51:10,750][105692] Updated weights for policy 0, policy_version 1847096 (0.0009) [2023-12-27 04:51:10,815][105692] Updated weights for policy 0, policy_version 1847106 (0.0009) [2023-12-27 04:51:10,875][105692] Updated weights for policy 0, policy_version 1847116 (0.0008) [2023-12-27 04:51:10,917][105620] Updated weights for policy 1, policy_version 1851488 (0.0008) [2023-12-27 04:51:10,965][105620] Updated weights for policy 1, policy_version 1851498 (0.0009) [2023-12-27 04:51:11,018][105620] Updated weights for policy 1, policy_version 1851508 (0.0008) [2023-12-27 04:51:11,062][104569] Fps is (10 sec: 20479.4, 60 sec: 19660.8, 300 sec: 19383.1). Total num frames: 946987008. Throughput: 0: 9543.4, 1: 9969.9. Samples: 946990104. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:51:11,063][104569] Avg episode reward: [(0, '8810.769'), (1, '9348.921')] [2023-12-27 04:51:11,700][105692] Updated weights for policy 0, policy_version 1847126 (0.0008) [2023-12-27 04:51:11,773][105692] Updated weights for policy 0, policy_version 1847136 (0.0008) [2023-12-27 04:51:11,783][105620] Updated weights for policy 1, policy_version 1851518 (0.0007) [2023-12-27 04:51:11,835][105692] Updated weights for policy 0, policy_version 1847146 (0.0007) [2023-12-27 04:51:11,842][105620] Updated weights for policy 1, policy_version 1851528 (0.0009) [2023-12-27 04:51:11,901][105620] Updated weights for policy 1, policy_version 1851538 (0.0011) [2023-12-27 04:51:12,601][105692] Updated weights for policy 0, policy_version 1847156 (0.0006) [2023-12-27 04:51:12,667][105692] Updated weights for policy 0, policy_version 1847166 (0.0008) [2023-12-27 04:51:12,669][105620] Updated weights for policy 1, policy_version 1851548 (0.0011) [2023-12-27 04:51:12,720][105692] Updated weights for policy 0, policy_version 1847176 (0.0011) [2023-12-27 04:51:12,732][105620] Updated weights for policy 1, policy_version 1851558 (0.0010) [2023-12-27 04:51:12,785][105620] Updated weights for policy 1, policy_version 1851568 (0.0010) [2023-12-27 04:51:13,369][105692] Updated weights for policy 0, policy_version 1847186 (0.0010) [2023-12-27 04:51:13,437][105692] Updated weights for policy 0, policy_version 1847196 (0.0006) [2023-12-27 04:51:13,444][105620] Updated weights for policy 1, policy_version 1851578 (0.0009) [2023-12-27 04:51:13,497][105692] Updated weights for policy 0, policy_version 1847206 (0.0005) [2023-12-27 04:51:13,499][105620] Updated weights for policy 1, policy_version 1851588 (0.0006) [2023-12-27 04:51:13,551][105692] Updated weights for policy 0, policy_version 1847216 (0.0009) [2023-12-27 04:51:13,558][105620] Updated weights for policy 1, policy_version 1851598 (0.0006) [2023-12-27 04:51:13,612][105620] Updated weights for policy 1, policy_version 1851608 (0.0005) [2023-12-27 04:51:14,131][105692] Updated weights for policy 0, policy_version 1847226 (0.0011) [2023-12-27 04:51:14,167][105620] Updated weights for policy 1, policy_version 1851618 (0.0008) [2023-12-27 04:51:14,185][105692] Updated weights for policy 0, policy_version 1847236 (0.0009) [2023-12-27 04:51:14,230][105620] Updated weights for policy 1, policy_version 1851628 (0.0008) [2023-12-27 04:51:14,237][105692] Updated weights for policy 0, policy_version 1847246 (0.0011) [2023-12-27 04:51:14,291][105620] Updated weights for policy 1, policy_version 1851638 (0.0007) [2023-12-27 04:51:14,897][105620] Updated weights for policy 1, policy_version 1851648 (0.0006) [2023-12-27 04:51:14,906][105692] Updated weights for policy 0, policy_version 1847256 (0.0011) [2023-12-27 04:51:14,959][105620] Updated weights for policy 1, policy_version 1851658 (0.0007) [2023-12-27 04:51:14,970][105692] Updated weights for policy 0, policy_version 1847266 (0.0011) [2023-12-27 04:51:15,027][105620] Updated weights for policy 1, policy_version 1851668 (0.0006) [2023-12-27 04:51:15,037][105692] Updated weights for policy 0, policy_version 1847276 (0.0011) [2023-12-27 04:51:15,698][105692] Updated weights for policy 0, policy_version 1847286 (0.0008) [2023-12-27 04:51:15,756][105692] Updated weights for policy 0, policy_version 1847296 (0.0005) [2023-12-27 04:51:15,798][105620] Updated weights for policy 1, policy_version 1851678 (0.0006) [2023-12-27 04:51:15,814][105692] Updated weights for policy 0, policy_version 1847306 (0.0006) [2023-12-27 04:51:15,848][105620] Updated weights for policy 1, policy_version 1851688 (0.0006) [2023-12-27 04:51:15,896][105620] Updated weights for policy 1, policy_version 1851698 (0.0008) [2023-12-27 04:51:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19524.2, 300 sec: 19383.1). Total num frames: 947085312. Throughput: 0: 9532.2, 1: 9975.7. Samples: 947047876. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:51:16,063][104569] Avg episode reward: [(0, '8450.622'), (1, '9256.479')] [2023-12-27 04:51:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001847312_472981504.pth... [2023-12-27 04:51:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001851704_474103808.pth... [2023-12-27 04:51:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001846192_472694784.pth [2023-12-27 04:51:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001850520_473800704.pth [2023-12-27 04:51:16,491][105692] Updated weights for policy 0, policy_version 1847316 (0.0010) [2023-12-27 04:51:16,550][105692] Updated weights for policy 0, policy_version 1847326 (0.0010) [2023-12-27 04:51:16,608][105692] Updated weights for policy 0, policy_version 1847336 (0.0010) [2023-12-27 04:51:16,659][105620] Updated weights for policy 1, policy_version 1851708 (0.0007) [2023-12-27 04:51:16,714][105620] Updated weights for policy 1, policy_version 1851718 (0.0007) [2023-12-27 04:51:16,774][105620] Updated weights for policy 1, policy_version 1851728 (0.0008) [2023-12-27 04:51:17,359][105692] Updated weights for policy 0, policy_version 1847346 (0.0010) [2023-12-27 04:51:17,373][105620] Updated weights for policy 1, policy_version 1851738 (0.0009) [2023-12-27 04:51:17,414][105692] Updated weights for policy 0, policy_version 1847356 (0.0008) [2023-12-27 04:51:17,429][105620] Updated weights for policy 1, policy_version 1851748 (0.0008) [2023-12-27 04:51:17,465][105692] Updated weights for policy 0, policy_version 1847366 (0.0007) [2023-12-27 04:51:17,483][105620] Updated weights for policy 1, policy_version 1851758 (0.0007) [2023-12-27 04:51:17,512][105692] Updated weights for policy 0, policy_version 1847376 (0.0010) [2023-12-27 04:51:17,539][105620] Updated weights for policy 1, policy_version 1851768 (0.0007) [2023-12-27 04:51:18,173][105620] Updated weights for policy 1, policy_version 1851778 (0.0007) [2023-12-27 04:51:18,235][105620] Updated weights for policy 1, policy_version 1851788 (0.0009) [2023-12-27 04:51:18,296][105620] Updated weights for policy 1, policy_version 1851798 (0.0007) [2023-12-27 04:51:18,310][105692] Updated weights for policy 0, policy_version 1847386 (0.0007) [2023-12-27 04:51:18,378][105692] Updated weights for policy 0, policy_version 1847396 (0.0009) [2023-12-27 04:51:18,440][105692] Updated weights for policy 0, policy_version 1847406 (0.0010) [2023-12-27 04:51:18,975][105620] Updated weights for policy 1, policy_version 1851808 (0.0008) [2023-12-27 04:51:19,027][105620] Updated weights for policy 1, policy_version 1851818 (0.0009) [2023-12-27 04:51:19,085][105620] Updated weights for policy 1, policy_version 1851828 (0.0009) [2023-12-27 04:51:19,229][105692] Updated weights for policy 0, policy_version 1847416 (0.0009) [2023-12-27 04:51:19,287][105692] Updated weights for policy 0, policy_version 1847426 (0.0009) [2023-12-27 04:51:19,351][105692] Updated weights for policy 0, policy_version 1847436 (0.0009) [2023-12-27 04:51:19,849][105620] Updated weights for policy 1, policy_version 1851838 (0.0008) [2023-12-27 04:51:19,915][105620] Updated weights for policy 1, policy_version 1851848 (0.0010) [2023-12-27 04:51:19,974][105620] Updated weights for policy 1, policy_version 1851858 (0.0009) [2023-12-27 04:51:20,083][105692] Updated weights for policy 0, policy_version 1847446 (0.0010) [2023-12-27 04:51:20,146][105692] Updated weights for policy 0, policy_version 1847456 (0.0011) [2023-12-27 04:51:20,207][105692] Updated weights for policy 0, policy_version 1847466 (0.0011) [2023-12-27 04:51:20,636][105620] Updated weights for policy 1, policy_version 1851868 (0.0008) [2023-12-27 04:51:20,700][105620] Updated weights for policy 1, policy_version 1851878 (0.0010) [2023-12-27 04:51:20,759][105620] Updated weights for policy 1, policy_version 1851888 (0.0011) [2023-12-27 04:51:20,982][105692] Updated weights for policy 0, policy_version 1847476 (0.0011) [2023-12-27 04:51:21,049][105692] Updated weights for policy 0, policy_version 1847486 (0.0011) [2023-12-27 04:51:21,062][104569] Fps is (10 sec: 18842.2, 60 sec: 19524.3, 300 sec: 19327.6). Total num frames: 947175424. Throughput: 0: 9614.7, 1: 9929.8. Samples: 947166976. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:51:21,062][104569] Avg episode reward: [(0, '8539.740'), (1, '9256.369')] [2023-12-27 04:51:21,106][105692] Updated weights for policy 0, policy_version 1847496 (0.0011) [2023-12-27 04:51:21,533][105620] Updated weights for policy 1, policy_version 1851898 (0.0011) [2023-12-27 04:51:21,602][105620] Updated weights for policy 1, policy_version 1851908 (0.0010) [2023-12-27 04:51:21,670][105620] Updated weights for policy 1, policy_version 1851918 (0.0011) [2023-12-27 04:51:21,737][105620] Updated weights for policy 1, policy_version 1851928 (0.0011) [2023-12-27 04:51:21,833][105692] Updated weights for policy 0, policy_version 1847506 (0.0010) [2023-12-27 04:51:21,894][105692] Updated weights for policy 0, policy_version 1847516 (0.0008) [2023-12-27 04:51:21,959][105692] Updated weights for policy 0, policy_version 1847526 (0.0008) [2023-12-27 04:51:22,030][105692] Updated weights for policy 0, policy_version 1847536 (0.0008) [2023-12-27 04:51:22,486][105620] Updated weights for policy 1, policy_version 1851938 (0.0010) [2023-12-27 04:51:22,538][105620] Updated weights for policy 1, policy_version 1851948 (0.0010) [2023-12-27 04:51:22,585][105620] Updated weights for policy 1, policy_version 1851958 (0.0009) [2023-12-27 04:51:22,790][105692] Updated weights for policy 0, policy_version 1847546 (0.0007) [2023-12-27 04:51:22,859][105692] Updated weights for policy 0, policy_version 1847556 (0.0008) [2023-12-27 04:51:22,917][105692] Updated weights for policy 0, policy_version 1847566 (0.0009) [2023-12-27 04:51:23,265][105620] Updated weights for policy 1, policy_version 1851968 (0.0009) [2023-12-27 04:51:23,312][105620] Updated weights for policy 1, policy_version 1851978 (0.0009) [2023-12-27 04:51:23,368][105620] Updated weights for policy 1, policy_version 1851988 (0.0009) [2023-12-27 04:51:23,690][105692] Updated weights for policy 0, policy_version 1847576 (0.0009) [2023-12-27 04:51:23,744][105692] Updated weights for policy 0, policy_version 1847586 (0.0009) [2023-12-27 04:51:23,788][105692] Updated weights for policy 0, policy_version 1847596 (0.0007) [2023-12-27 04:51:24,135][105620] Updated weights for policy 1, policy_version 1851998 (0.0008) [2023-12-27 04:51:24,205][105620] Updated weights for policy 1, policy_version 1852008 (0.0008) [2023-12-27 04:51:24,262][105620] Updated weights for policy 1, policy_version 1852018 (0.0010) [2023-12-27 04:51:24,371][105692] Updated weights for policy 0, policy_version 1847606 (0.0006) [2023-12-27 04:51:24,419][105692] Updated weights for policy 0, policy_version 1847616 (0.0005) [2023-12-27 04:51:24,476][105692] Updated weights for policy 0, policy_version 1847626 (0.0007) [2023-12-27 04:51:25,041][105620] Updated weights for policy 1, policy_version 1852028 (0.0010) [2023-12-27 04:51:25,091][105620] Updated weights for policy 1, policy_version 1852038 (0.0009) [2023-12-27 04:51:25,144][105620] Updated weights for policy 1, policy_version 1852048 (0.0009) [2023-12-27 04:51:25,183][105692] Updated weights for policy 0, policy_version 1847636 (0.0008) [2023-12-27 04:51:25,242][105692] Updated weights for policy 0, policy_version 1847646 (0.0008) [2023-12-27 04:51:25,298][105692] Updated weights for policy 0, policy_version 1847656 (0.0005) [2023-12-27 04:51:25,890][105692] Updated weights for policy 0, policy_version 1847666 (0.0006) [2023-12-27 04:51:25,936][105692] Updated weights for policy 0, policy_version 1847676 (0.0008) [2023-12-27 04:51:25,984][105620] Updated weights for policy 1, policy_version 1852058 (0.0009) [2023-12-27 04:51:25,990][105692] Updated weights for policy 0, policy_version 1847686 (0.0007) [2023-12-27 04:51:26,035][105620] Updated weights for policy 1, policy_version 1852068 (0.0006) [2023-12-27 04:51:26,052][105692] Updated weights for policy 0, policy_version 1847696 (0.0009) [2023-12-27 04:51:26,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19524.3, 300 sec: 19355.3). Total num frames: 947273728. Throughput: 0: 9665.2, 1: 9865.1. Samples: 947281680. Policy #0 lag: (min: 18.0, avg: 32.6, max: 50.0) [2023-12-27 04:51:26,063][104569] Avg episode reward: [(0, '8167.717'), (1, '9256.294')] [2023-12-27 04:51:26,089][105620] Updated weights for policy 1, policy_version 1852078 (0.0009) [2023-12-27 04:51:26,153][105620] Updated weights for policy 1, policy_version 1852088 (0.0009) [2023-12-27 04:51:26,728][105620] Updated weights for policy 1, policy_version 1852098 (0.0005) [2023-12-27 04:51:26,779][105620] Updated weights for policy 1, policy_version 1852108 (0.0005) [2023-12-27 04:51:26,782][105692] Updated weights for policy 0, policy_version 1847706 (0.0010) [2023-12-27 04:51:26,832][105620] Updated weights for policy 1, policy_version 1852118 (0.0006) [2023-12-27 04:51:26,840][105692] Updated weights for policy 0, policy_version 1847716 (0.0010) [2023-12-27 04:51:26,894][105692] Updated weights for policy 0, policy_version 1847726 (0.0010) [2023-12-27 04:51:27,445][105620] Updated weights for policy 1, policy_version 1852128 (0.0008) [2023-12-27 04:51:27,507][105620] Updated weights for policy 1, policy_version 1852138 (0.0005) [2023-12-27 04:51:27,571][105620] Updated weights for policy 1, policy_version 1852148 (0.0005) [2023-12-27 04:51:27,629][105692] Updated weights for policy 0, policy_version 1847736 (0.0010) [2023-12-27 04:51:27,673][105692] Updated weights for policy 0, policy_version 1847746 (0.0010) [2023-12-27 04:51:27,725][105692] Updated weights for policy 0, policy_version 1847756 (0.0010) [2023-12-27 04:51:28,097][105620] Updated weights for policy 1, policy_version 1852158 (0.0007) [2023-12-27 04:51:28,144][105620] Updated weights for policy 1, policy_version 1852168 (0.0008) [2023-12-27 04:51:28,196][105620] Updated weights for policy 1, policy_version 1852178 (0.0007) [2023-12-27 04:51:28,480][105692] Updated weights for policy 0, policy_version 1847766 (0.0009) [2023-12-27 04:51:28,532][105692] Updated weights for policy 0, policy_version 1847776 (0.0008) [2023-12-27 04:51:28,585][105692] Updated weights for policy 0, policy_version 1847786 (0.0010) [2023-12-27 04:51:28,974][105620] Updated weights for policy 1, policy_version 1852188 (0.0008) [2023-12-27 04:51:29,028][105620] Updated weights for policy 1, policy_version 1852198 (0.0008) [2023-12-27 04:51:29,076][105620] Updated weights for policy 1, policy_version 1852208 (0.0008) [2023-12-27 04:51:29,348][105692] Updated weights for policy 0, policy_version 1847796 (0.0010) [2023-12-27 04:51:29,407][105692] Updated weights for policy 0, policy_version 1847806 (0.0009) [2023-12-27 04:51:29,470][105692] Updated weights for policy 0, policy_version 1847816 (0.0010) [2023-12-27 04:51:29,764][105620] Updated weights for policy 1, policy_version 1852218 (0.0008) [2023-12-27 04:51:29,820][105620] Updated weights for policy 1, policy_version 1852228 (0.0006) [2023-12-27 04:51:29,886][105620] Updated weights for policy 1, policy_version 1852238 (0.0007) [2023-12-27 04:51:29,947][105620] Updated weights for policy 1, policy_version 1852248 (0.0011) [2023-12-27 04:51:30,186][105692] Updated weights for policy 0, policy_version 1847826 (0.0010) [2023-12-27 04:51:30,241][105692] Updated weights for policy 0, policy_version 1847836 (0.0010) [2023-12-27 04:51:30,285][105692] Updated weights for policy 0, policy_version 1847846 (0.0010) [2023-12-27 04:51:30,336][105692] Updated weights for policy 0, policy_version 1847856 (0.0010) [2023-12-27 04:51:30,585][105620] Updated weights for policy 1, policy_version 1852258 (0.0005) [2023-12-27 04:51:30,638][105620] Updated weights for policy 1, policy_version 1852268 (0.0005) [2023-12-27 04:51:30,687][105620] Updated weights for policy 1, policy_version 1852278 (0.0005) [2023-12-27 04:51:31,050][105692] Updated weights for policy 0, policy_version 1847866 (0.0011) [2023-12-27 04:51:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19355.3). Total num frames: 947372032. Throughput: 0: 9677.7, 1: 9934.0. Samples: 947343500. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:51:31,062][104569] Avg episode reward: [(0, '7900.952'), (1, '9348.546')] [2023-12-27 04:51:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001852280_474251264.pth... [2023-12-27 04:51:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001851128_473956352.pth [2023-12-27 04:51:31,106][105692] Updated weights for policy 0, policy_version 1847876 (0.0010) [2023-12-27 04:51:31,166][105692] Updated weights for policy 0, policy_version 1847886 (0.0011) [2023-12-27 04:51:31,173][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001847888_473128960.pth... [2023-12-27 04:51:31,176][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001846736_472834048.pth [2023-12-27 04:51:31,352][105620] Updated weights for policy 1, policy_version 1852288 (0.0009) [2023-12-27 04:51:31,408][105620] Updated weights for policy 1, policy_version 1852298 (0.0007) [2023-12-27 04:51:31,466][105620] Updated weights for policy 1, policy_version 1852308 (0.0006) [2023-12-27 04:51:31,859][105692] Updated weights for policy 0, policy_version 1847896 (0.0010) [2023-12-27 04:51:31,914][105692] Updated weights for policy 0, policy_version 1847906 (0.0010) [2023-12-27 04:51:31,979][105692] Updated weights for policy 0, policy_version 1847916 (0.0010) [2023-12-27 04:51:32,107][105620] Updated weights for policy 1, policy_version 1852318 (0.0005) [2023-12-27 04:51:32,180][105620] Updated weights for policy 1, policy_version 1852328 (0.0005) [2023-12-27 04:51:32,229][105620] Updated weights for policy 1, policy_version 1852338 (0.0005) [2023-12-27 04:51:32,697][105692] Updated weights for policy 0, policy_version 1847926 (0.0007) [2023-12-27 04:51:32,758][105692] Updated weights for policy 0, policy_version 1847936 (0.0008) [2023-12-27 04:51:32,827][105692] Updated weights for policy 0, policy_version 1847946 (0.0008) [2023-12-27 04:51:32,902][105620] Updated weights for policy 1, policy_version 1852348 (0.0006) [2023-12-27 04:51:32,965][105620] Updated weights for policy 1, policy_version 1852358 (0.0008) [2023-12-27 04:51:33,027][105620] Updated weights for policy 1, policy_version 1852368 (0.0008) [2023-12-27 04:51:33,509][105692] Updated weights for policy 0, policy_version 1847956 (0.0008) [2023-12-27 04:51:33,563][105692] Updated weights for policy 0, policy_version 1847966 (0.0010) [2023-12-27 04:51:33,621][105692] Updated weights for policy 0, policy_version 1847976 (0.0010) [2023-12-27 04:51:33,740][105620] Updated weights for policy 1, policy_version 1852378 (0.0008) [2023-12-27 04:51:33,795][105620] Updated weights for policy 1, policy_version 1852388 (0.0008) [2023-12-27 04:51:33,838][105620] Updated weights for policy 1, policy_version 1852398 (0.0007) [2023-12-27 04:51:33,882][105620] Updated weights for policy 1, policy_version 1852408 (0.0008) [2023-12-27 04:51:34,294][105692] Updated weights for policy 0, policy_version 1847986 (0.0009) [2023-12-27 04:51:34,354][105692] Updated weights for policy 0, policy_version 1847996 (0.0006) [2023-12-27 04:51:34,416][105692] Updated weights for policy 0, policy_version 1848006 (0.0006) [2023-12-27 04:51:34,476][105692] Updated weights for policy 0, policy_version 1848016 (0.0009) [2023-12-27 04:51:34,686][105620] Updated weights for policy 1, policy_version 1852418 (0.0008) [2023-12-27 04:51:34,751][105620] Updated weights for policy 1, policy_version 1852428 (0.0006) [2023-12-27 04:51:34,810][105620] Updated weights for policy 1, policy_version 1852438 (0.0009) [2023-12-27 04:51:35,149][105692] Updated weights for policy 0, policy_version 1848027 (0.0010) [2023-12-27 04:51:35,206][105692] Updated weights for policy 0, policy_version 1848038 (0.0010) [2023-12-27 04:51:35,263][105692] Updated weights for policy 0, policy_version 1848048 (0.0010) [2023-12-27 04:51:35,397][105620] Updated weights for policy 1, policy_version 1852448 (0.0006) [2023-12-27 04:51:35,451][105620] Updated weights for policy 1, policy_version 1852458 (0.0006) [2023-12-27 04:51:35,497][105620] Updated weights for policy 1, policy_version 1852468 (0.0008) [2023-12-27 04:51:35,975][105692] Updated weights for policy 0, policy_version 1848058 (0.0006) [2023-12-27 04:51:36,023][105692] Updated weights for policy 0, policy_version 1848068 (0.0009) [2023-12-27 04:51:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19355.3). Total num frames: 947470336. Throughput: 0: 9709.0, 1: 9959.5. Samples: 947462504. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:51:36,063][104569] Avg episode reward: [(0, '8540.503'), (1, '9348.454')] [2023-12-27 04:51:36,072][105692] Updated weights for policy 0, policy_version 1848078 (0.0006) [2023-12-27 04:51:36,236][105620] Updated weights for policy 1, policy_version 1852478 (0.0008) [2023-12-27 04:51:36,302][105620] Updated weights for policy 1, policy_version 1852488 (0.0006) [2023-12-27 04:51:36,372][105620] Updated weights for policy 1, policy_version 1852498 (0.0006) [2023-12-27 04:51:36,680][105692] Updated weights for policy 0, policy_version 1848088 (0.0009) [2023-12-27 04:51:36,737][105692] Updated weights for policy 0, policy_version 1848098 (0.0009) [2023-12-27 04:51:36,798][105692] Updated weights for policy 0, policy_version 1848108 (0.0009) [2023-12-27 04:51:36,938][105620] Updated weights for policy 1, policy_version 1852508 (0.0007) [2023-12-27 04:51:36,999][105620] Updated weights for policy 1, policy_version 1852518 (0.0009) [2023-12-27 04:51:37,054][105620] Updated weights for policy 1, policy_version 1852528 (0.0009) [2023-12-27 04:51:37,552][105692] Updated weights for policy 0, policy_version 1848118 (0.0009) [2023-12-27 04:51:37,609][105692] Updated weights for policy 0, policy_version 1848128 (0.0009) [2023-12-27 04:51:37,661][105692] Updated weights for policy 0, policy_version 1848138 (0.0009) [2023-12-27 04:51:37,766][105620] Updated weights for policy 1, policy_version 1852538 (0.0009) [2023-12-27 04:51:37,837][105620] Updated weights for policy 1, policy_version 1852548 (0.0010) [2023-12-27 04:51:37,896][105620] Updated weights for policy 1, policy_version 1852558 (0.0009) [2023-12-27 04:51:37,953][105620] Updated weights for policy 1, policy_version 1852568 (0.0009) [2023-12-27 04:51:38,447][105692] Updated weights for policy 0, policy_version 1848148 (0.0010) [2023-12-27 04:51:38,498][105692] Updated weights for policy 0, policy_version 1848158 (0.0009) [2023-12-27 04:51:38,553][105692] Updated weights for policy 0, policy_version 1848168 (0.0010) [2023-12-27 04:51:38,701][105620] Updated weights for policy 1, policy_version 1852578 (0.0009) [2023-12-27 04:51:38,749][105620] Updated weights for policy 1, policy_version 1852588 (0.0009) [2023-12-27 04:51:38,796][105620] Updated weights for policy 1, policy_version 1852598 (0.0009) [2023-12-27 04:51:39,327][105692] Updated weights for policy 0, policy_version 1848178 (0.0009) [2023-12-27 04:51:39,389][105692] Updated weights for policy 0, policy_version 1848188 (0.0010) [2023-12-27 04:51:39,455][105692] Updated weights for policy 0, policy_version 1848198 (0.0009) [2023-12-27 04:51:39,510][105692] Updated weights for policy 0, policy_version 1848208 (0.0009) [2023-12-27 04:51:39,606][105620] Updated weights for policy 1, policy_version 1852608 (0.0009) [2023-12-27 04:51:39,663][105620] Updated weights for policy 1, policy_version 1852618 (0.0008) [2023-12-27 04:51:39,724][105620] Updated weights for policy 1, policy_version 1852628 (0.0008) [2023-12-27 04:51:40,241][105692] Updated weights for policy 0, policy_version 1848218 (0.0008) [2023-12-27 04:51:40,297][105692] Updated weights for policy 0, policy_version 1848228 (0.0008) [2023-12-27 04:51:40,360][105692] Updated weights for policy 0, policy_version 1848238 (0.0006) [2023-12-27 04:51:40,513][105620] Updated weights for policy 1, policy_version 1852638 (0.0010) [2023-12-27 04:51:40,562][105620] Updated weights for policy 1, policy_version 1852648 (0.0010) [2023-12-27 04:51:40,622][105620] Updated weights for policy 1, policy_version 1852658 (0.0011) [2023-12-27 04:51:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19383.1). Total num frames: 947568640. Throughput: 0: 9715.9, 1: 9931.3. Samples: 947579196. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:51:41,063][104569] Avg episode reward: [(0, '8534.440'), (1, '9071.339')] [2023-12-27 04:51:41,165][105692] Updated weights for policy 0, policy_version 1848248 (0.0008) [2023-12-27 04:51:41,225][105692] Updated weights for policy 0, policy_version 1848258 (0.0007) [2023-12-27 04:51:41,295][105692] Updated weights for policy 0, policy_version 1848268 (0.0008) [2023-12-27 04:51:41,409][105620] Updated weights for policy 1, policy_version 1852668 (0.0009) [2023-12-27 04:51:41,461][105620] Updated weights for policy 1, policy_version 1852678 (0.0007) [2023-12-27 04:51:41,516][105620] Updated weights for policy 1, policy_version 1852688 (0.0008) [2023-12-27 04:51:42,105][105692] Updated weights for policy 0, policy_version 1848278 (0.0010) [2023-12-27 04:51:42,153][105692] Updated weights for policy 0, policy_version 1848288 (0.0011) [2023-12-27 04:51:42,211][105620] Updated weights for policy 1, policy_version 1852698 (0.0006) [2023-12-27 04:51:42,214][105692] Updated weights for policy 0, policy_version 1848299 (0.0009) [2023-12-27 04:51:42,279][105620] Updated weights for policy 1, policy_version 1852708 (0.0009) [2023-12-27 04:51:42,343][105620] Updated weights for policy 1, policy_version 1852718 (0.0008) [2023-12-27 04:51:42,412][105620] Updated weights for policy 1, policy_version 1852728 (0.0008) [2023-12-27 04:51:42,980][105692] Updated weights for policy 0, policy_version 1848309 (0.0006) [2023-12-27 04:51:43,049][105692] Updated weights for policy 0, policy_version 1848319 (0.0006) [2023-12-27 04:51:43,085][105620] Updated weights for policy 1, policy_version 1852738 (0.0007) [2023-12-27 04:51:43,117][105692] Updated weights for policy 0, policy_version 1848329 (0.0010) [2023-12-27 04:51:43,145][105620] Updated weights for policy 1, policy_version 1852748 (0.0008) [2023-12-27 04:51:43,200][105620] Updated weights for policy 1, policy_version 1852758 (0.0008) [2023-12-27 04:51:43,827][105692] Updated weights for policy 0, policy_version 1848339 (0.0011) [2023-12-27 04:51:43,851][105620] Updated weights for policy 1, policy_version 1852768 (0.0006) [2023-12-27 04:51:43,876][105692] Updated weights for policy 0, policy_version 1848349 (0.0010) [2023-12-27 04:51:43,901][105620] Updated weights for policy 1, policy_version 1852778 (0.0005) [2023-12-27 04:51:43,926][105692] Updated weights for policy 0, policy_version 1848359 (0.0011) [2023-12-27 04:51:43,950][105620] Updated weights for policy 1, policy_version 1852788 (0.0006) [2023-12-27 04:51:44,624][105692] Updated weights for policy 0, policy_version 1848369 (0.0010) [2023-12-27 04:51:44,635][105620] Updated weights for policy 1, policy_version 1852798 (0.0011) [2023-12-27 04:51:44,682][105692] Updated weights for policy 0, policy_version 1848379 (0.0010) [2023-12-27 04:51:44,694][105620] Updated weights for policy 1, policy_version 1852808 (0.0010) [2023-12-27 04:51:44,738][105692] Updated weights for policy 0, policy_version 1848389 (0.0011) [2023-12-27 04:51:44,743][105620] Updated weights for policy 1, policy_version 1852818 (0.0010) [2023-12-27 04:51:44,811][105692] Updated weights for policy 0, policy_version 1848399 (0.0010) [2023-12-27 04:51:45,431][105620] Updated weights for policy 1, policy_version 1852828 (0.0011) [2023-12-27 04:51:45,483][105620] Updated weights for policy 1, policy_version 1852838 (0.0010) [2023-12-27 04:51:45,517][105692] Updated weights for policy 0, policy_version 1848409 (0.0006) [2023-12-27 04:51:45,535][105620] Updated weights for policy 1, policy_version 1852848 (0.0010) [2023-12-27 04:51:45,578][105692] Updated weights for policy 0, policy_version 1848419 (0.0006) [2023-12-27 04:51:45,639][105692] Updated weights for policy 0, policy_version 1848429 (0.0008) [2023-12-27 04:51:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.7, 300 sec: 19383.1). Total num frames: 947666944. Throughput: 0: 9601.3, 1: 9900.5. Samples: 947636616. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:51:46,063][104569] Avg episode reward: [(0, '8442.550'), (1, '9071.229')] [2023-12-27 04:51:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001852856_474398720.pth... [2023-12-27 04:51:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001848432_473268224.pth... [2023-12-27 04:51:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001851704_474103808.pth [2023-12-27 04:51:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001847312_472981504.pth [2023-12-27 04:51:46,107][105620] Updated weights for policy 1, policy_version 1852858 (0.0009) [2023-12-27 04:51:46,160][105620] Updated weights for policy 1, policy_version 1852868 (0.0005) [2023-12-27 04:51:46,227][105620] Updated weights for policy 1, policy_version 1852878 (0.0008) [2023-12-27 04:51:46,275][105620] Updated weights for policy 1, policy_version 1852888 (0.0010) [2023-12-27 04:51:46,396][105692] Updated weights for policy 0, policy_version 1848439 (0.0009) [2023-12-27 04:51:46,441][105692] Updated weights for policy 0, policy_version 1848449 (0.0008) [2023-12-27 04:51:46,493][105692] Updated weights for policy 0, policy_version 1848459 (0.0008) [2023-12-27 04:51:46,971][105620] Updated weights for policy 1, policy_version 1852898 (0.0010) [2023-12-27 04:51:47,016][105620] Updated weights for policy 1, policy_version 1852908 (0.0010) [2023-12-27 04:51:47,063][105620] Updated weights for policy 1, policy_version 1852918 (0.0010) [2023-12-27 04:51:47,282][105692] Updated weights for policy 0, policy_version 1848469 (0.0008) [2023-12-27 04:51:47,343][105692] Updated weights for policy 0, policy_version 1848479 (0.0008) [2023-12-27 04:51:47,389][105692] Updated weights for policy 0, policy_version 1848489 (0.0008) [2023-12-27 04:51:47,768][105620] Updated weights for policy 1, policy_version 1852928 (0.0010) [2023-12-27 04:51:47,819][105620] Updated weights for policy 1, policy_version 1852938 (0.0010) [2023-12-27 04:51:47,874][105620] Updated weights for policy 1, policy_version 1852948 (0.0010) [2023-12-27 04:51:47,994][105692] Updated weights for policy 0, policy_version 1848499 (0.0007) [2023-12-27 04:51:48,046][105692] Updated weights for policy 0, policy_version 1848509 (0.0005) [2023-12-27 04:51:48,106][105692] Updated weights for policy 0, policy_version 1848519 (0.0005) [2023-12-27 04:51:48,613][105620] Updated weights for policy 1, policy_version 1852958 (0.0010) [2023-12-27 04:51:48,668][105620] Updated weights for policy 1, policy_version 1852968 (0.0006) [2023-12-27 04:51:48,726][105620] Updated weights for policy 1, policy_version 1852978 (0.0006) [2023-12-27 04:51:48,762][105692] Updated weights for policy 0, policy_version 1848529 (0.0006) [2023-12-27 04:51:48,819][105692] Updated weights for policy 0, policy_version 1848539 (0.0010) [2023-12-27 04:51:48,878][105692] Updated weights for policy 0, policy_version 1848549 (0.0009) [2023-12-27 04:51:48,931][105692] Updated weights for policy 0, policy_version 1848559 (0.0010) [2023-12-27 04:51:49,354][105620] Updated weights for policy 1, policy_version 1852988 (0.0007) [2023-12-27 04:51:49,417][105620] Updated weights for policy 1, policy_version 1852998 (0.0010) [2023-12-27 04:51:49,485][105620] Updated weights for policy 1, policy_version 1853008 (0.0010) [2023-12-27 04:51:49,753][105692] Updated weights for policy 0, policy_version 1848569 (0.0006) [2023-12-27 04:51:49,816][105692] Updated weights for policy 0, policy_version 1848579 (0.0006) [2023-12-27 04:51:49,884][105692] Updated weights for policy 0, policy_version 1848589 (0.0011) [2023-12-27 04:51:50,135][105620] Updated weights for policy 1, policy_version 1853018 (0.0010) [2023-12-27 04:51:50,183][105620] Updated weights for policy 1, policy_version 1853028 (0.0010) [2023-12-27 04:51:50,242][105620] Updated weights for policy 1, policy_version 1853038 (0.0010) [2023-12-27 04:51:50,310][105620] Updated weights for policy 1, policy_version 1853048 (0.0006) [2023-12-27 04:51:50,524][105692] Updated weights for policy 0, policy_version 1848599 (0.0007) [2023-12-27 04:51:50,588][105692] Updated weights for policy 0, policy_version 1848609 (0.0009) [2023-12-27 04:51:50,641][105692] Updated weights for policy 0, policy_version 1848619 (0.0006) [2023-12-27 04:51:50,987][105620] Updated weights for policy 1, policy_version 1853058 (0.0008) [2023-12-27 04:51:51,052][105620] Updated weights for policy 1, policy_version 1853068 (0.0008) [2023-12-27 04:51:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19383.1). Total num frames: 947765248. Throughput: 0: 9612.9, 1: 10004.7. Samples: 947756644. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:51:51,066][104569] Avg episode reward: [(0, '8990.252'), (1, '9348.150')] [2023-12-27 04:51:51,116][105620] Updated weights for policy 1, policy_version 1853078 (0.0008) [2023-12-27 04:51:51,447][105692] Updated weights for policy 0, policy_version 1848629 (0.0009) [2023-12-27 04:51:51,511][105692] Updated weights for policy 0, policy_version 1848639 (0.0010) [2023-12-27 04:51:51,561][105692] Updated weights for policy 0, policy_version 1848649 (0.0008) [2023-12-27 04:51:51,887][105620] Updated weights for policy 1, policy_version 1853088 (0.0009) [2023-12-27 04:51:51,947][105620] Updated weights for policy 1, policy_version 1853098 (0.0008) [2023-12-27 04:51:52,014][105620] Updated weights for policy 1, policy_version 1853108 (0.0009) [2023-12-27 04:51:52,315][105692] Updated weights for policy 0, policy_version 1848659 (0.0008) [2023-12-27 04:51:52,379][105692] Updated weights for policy 0, policy_version 1848669 (0.0009) [2023-12-27 04:51:52,444][105692] Updated weights for policy 0, policy_version 1848679 (0.0009) [2023-12-27 04:51:52,716][105620] Updated weights for policy 1, policy_version 1853118 (0.0009) [2023-12-27 04:51:52,780][105620] Updated weights for policy 1, policy_version 1853128 (0.0007) [2023-12-27 04:51:52,842][105620] Updated weights for policy 1, policy_version 1853138 (0.0008) [2023-12-27 04:51:53,268][105692] Updated weights for policy 0, policy_version 1848689 (0.0010) [2023-12-27 04:51:53,318][105692] Updated weights for policy 0, policy_version 1848699 (0.0009) [2023-12-27 04:51:53,373][105692] Updated weights for policy 0, policy_version 1848709 (0.0008) [2023-12-27 04:51:53,427][105692] Updated weights for policy 0, policy_version 1848719 (0.0009) [2023-12-27 04:51:53,514][105620] Updated weights for policy 1, policy_version 1853148 (0.0009) [2023-12-27 04:51:53,577][105620] Updated weights for policy 1, policy_version 1853158 (0.0008) [2023-12-27 04:51:53,634][105620] Updated weights for policy 1, policy_version 1853168 (0.0009) [2023-12-27 04:51:54,183][105692] Updated weights for policy 0, policy_version 1848729 (0.0008) [2023-12-27 04:51:54,234][105692] Updated weights for policy 0, policy_version 1848739 (0.0009) [2023-12-27 04:51:54,286][105692] Updated weights for policy 0, policy_version 1848749 (0.0008) [2023-12-27 04:51:54,378][105620] Updated weights for policy 1, policy_version 1853178 (0.0009) [2023-12-27 04:51:54,430][105620] Updated weights for policy 1, policy_version 1853188 (0.0010) [2023-12-27 04:51:54,486][105620] Updated weights for policy 1, policy_version 1853198 (0.0010) [2023-12-27 04:51:54,544][105620] Updated weights for policy 1, policy_version 1853208 (0.0010) [2023-12-27 04:51:55,056][105692] Updated weights for policy 0, policy_version 1848759 (0.0009) [2023-12-27 04:51:55,122][105692] Updated weights for policy 0, policy_version 1848769 (0.0009) [2023-12-27 04:51:55,182][105692] Updated weights for policy 0, policy_version 1848780 (0.0009) [2023-12-27 04:51:55,222][105620] Updated weights for policy 1, policy_version 1853218 (0.0005) [2023-12-27 04:51:55,283][105620] Updated weights for policy 1, policy_version 1853228 (0.0009) [2023-12-27 04:51:55,342][105620] Updated weights for policy 1, policy_version 1853238 (0.0010) [2023-12-27 04:51:55,953][105692] Updated weights for policy 0, policy_version 1848790 (0.0007) [2023-12-27 04:51:56,012][105620] Updated weights for policy 1, policy_version 1853248 (0.0009) [2023-12-27 04:51:56,022][105692] Updated weights for policy 0, policy_version 1848800 (0.0008) [2023-12-27 04:51:56,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 947855360. Throughput: 0: 9589.8, 1: 10000.3. Samples: 947871648. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:51:56,062][105620] Updated weights for policy 1, policy_version 1853258 (0.0007) [2023-12-27 04:51:56,062][104569] Avg episode reward: [(0, '8806.708'), (1, '9348.062')] [2023-12-27 04:51:56,085][105692] Updated weights for policy 0, policy_version 1848810 (0.0011) [2023-12-27 04:51:56,123][105620] Updated weights for policy 1, policy_version 1853268 (0.0006) [2023-12-27 04:51:56,765][105692] Updated weights for policy 0, policy_version 1848820 (0.0011) [2023-12-27 04:51:56,830][105692] Updated weights for policy 0, policy_version 1848830 (0.0010) [2023-12-27 04:51:56,892][105692] Updated weights for policy 0, policy_version 1848840 (0.0010) [2023-12-27 04:51:56,893][105620] Updated weights for policy 1, policy_version 1853278 (0.0009) [2023-12-27 04:51:56,952][105620] Updated weights for policy 1, policy_version 1853288 (0.0006) [2023-12-27 04:51:57,007][105620] Updated weights for policy 1, policy_version 1853298 (0.0008) [2023-12-27 04:51:57,524][105692] Updated weights for policy 0, policy_version 1848850 (0.0009) [2023-12-27 04:51:57,574][105692] Updated weights for policy 0, policy_version 1848860 (0.0005) [2023-12-27 04:51:57,625][105692] Updated weights for policy 0, policy_version 1848870 (0.0006) [2023-12-27 04:51:57,685][105692] Updated weights for policy 0, policy_version 1848880 (0.0010) [2023-12-27 04:51:57,784][105620] Updated weights for policy 1, policy_version 1853308 (0.0010) [2023-12-27 04:51:57,834][105620] Updated weights for policy 1, policy_version 1853319 (0.0009) [2023-12-27 04:51:57,885][105620] Updated weights for policy 1, policy_version 1853329 (0.0008) [2023-12-27 04:51:58,398][105692] Updated weights for policy 0, policy_version 1848890 (0.0008) [2023-12-27 04:51:58,464][105692] Updated weights for policy 0, policy_version 1848900 (0.0010) [2023-12-27 04:51:58,527][105692] Updated weights for policy 0, policy_version 1848910 (0.0010) [2023-12-27 04:51:58,711][105620] Updated weights for policy 1, policy_version 1853339 (0.0008) [2023-12-27 04:51:58,773][105620] Updated weights for policy 1, policy_version 1853349 (0.0008) [2023-12-27 04:51:58,837][105620] Updated weights for policy 1, policy_version 1853359 (0.0008) [2023-12-27 04:51:59,265][105692] Updated weights for policy 0, policy_version 1848920 (0.0011) [2023-12-27 04:51:59,317][105692] Updated weights for policy 0, policy_version 1848930 (0.0010) [2023-12-27 04:51:59,379][105692] Updated weights for policy 0, policy_version 1848940 (0.0011) [2023-12-27 04:51:59,608][105620] Updated weights for policy 1, policy_version 1853369 (0.0008) [2023-12-27 04:51:59,669][105620] Updated weights for policy 1, policy_version 1853379 (0.0008) [2023-12-27 04:51:59,727][105620] Updated weights for policy 1, policy_version 1853389 (0.0008) [2023-12-27 04:51:59,786][105620] Updated weights for policy 1, policy_version 1853399 (0.0009) [2023-12-27 04:52:00,140][105692] Updated weights for policy 0, policy_version 1848950 (0.0010) [2023-12-27 04:52:00,195][105692] Updated weights for policy 0, policy_version 1848960 (0.0010) [2023-12-27 04:52:00,240][105692] Updated weights for policy 0, policy_version 1848970 (0.0010) [2023-12-27 04:52:00,539][105620] Updated weights for policy 1, policy_version 1853409 (0.0008) [2023-12-27 04:52:00,591][105620] Updated weights for policy 1, policy_version 1853419 (0.0008) [2023-12-27 04:52:00,643][105620] Updated weights for policy 1, policy_version 1853429 (0.0008) [2023-12-27 04:52:01,004][105692] Updated weights for policy 0, policy_version 1848980 (0.0010) [2023-12-27 04:52:01,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19383.1). Total num frames: 947953664. Throughput: 0: 9634.8, 1: 9929.5. Samples: 947928264. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:52:01,063][104569] Avg episode reward: [(0, '8446.271'), (1, '9255.638')] [2023-12-27 04:52:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001853432_474546176.pth... [2023-12-27 04:52:01,067][105692] Updated weights for policy 0, policy_version 1848990 (0.0010) [2023-12-27 04:52:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001852280_474251264.pth [2023-12-27 04:52:01,125][105692] Updated weights for policy 0, policy_version 1849000 (0.0010) [2023-12-27 04:52:01,177][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001849008_473415680.pth... [2023-12-27 04:52:01,181][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001847888_473128960.pth [2023-12-27 04:52:01,410][105620] Updated weights for policy 1, policy_version 1853439 (0.0008) [2023-12-27 04:52:01,458][105620] Updated weights for policy 1, policy_version 1853449 (0.0008) [2023-12-27 04:52:01,508][105620] Updated weights for policy 1, policy_version 1853459 (0.0007) [2023-12-27 04:52:01,899][105692] Updated weights for policy 0, policy_version 1849010 (0.0011) [2023-12-27 04:52:01,958][105692] Updated weights for policy 0, policy_version 1849020 (0.0011) [2023-12-27 04:52:02,007][105692] Updated weights for policy 0, policy_version 1849030 (0.0010) [2023-12-27 04:52:02,072][105692] Updated weights for policy 0, policy_version 1849040 (0.0011) [2023-12-27 04:52:02,305][105620] Updated weights for policy 1, policy_version 1853469 (0.0009) [2023-12-27 04:52:02,369][105620] Updated weights for policy 1, policy_version 1853479 (0.0011) [2023-12-27 04:52:02,423][105620] Updated weights for policy 1, policy_version 1853489 (0.0011) [2023-12-27 04:52:02,804][105692] Updated weights for policy 0, policy_version 1849050 (0.0010) [2023-12-27 04:52:02,853][105692] Updated weights for policy 0, policy_version 1849060 (0.0010) [2023-12-27 04:52:02,908][105692] Updated weights for policy 0, policy_version 1849070 (0.0010) [2023-12-27 04:52:03,147][105620] Updated weights for policy 1, policy_version 1853499 (0.0010) [2023-12-27 04:52:03,199][105620] Updated weights for policy 1, policy_version 1853509 (0.0006) [2023-12-27 04:52:03,247][105620] Updated weights for policy 1, policy_version 1853519 (0.0005) [2023-12-27 04:52:03,646][105692] Updated weights for policy 0, policy_version 1849080 (0.0010) [2023-12-27 04:52:03,693][105692] Updated weights for policy 0, policy_version 1849090 (0.0009) [2023-12-27 04:52:03,751][105692] Updated weights for policy 0, policy_version 1849100 (0.0005) [2023-12-27 04:52:03,840][105620] Updated weights for policy 1, policy_version 1853529 (0.0010) [2023-12-27 04:52:03,902][105620] Updated weights for policy 1, policy_version 1853539 (0.0007) [2023-12-27 04:52:03,964][105620] Updated weights for policy 1, policy_version 1853549 (0.0007) [2023-12-27 04:52:04,038][105620] Updated weights for policy 1, policy_version 1853559 (0.0006) [2023-12-27 04:52:04,436][105692] Updated weights for policy 0, policy_version 1849110 (0.0007) [2023-12-27 04:52:04,499][105692] Updated weights for policy 0, policy_version 1849120 (0.0009) [2023-12-27 04:52:04,561][105692] Updated weights for policy 0, policy_version 1849130 (0.0009) [2023-12-27 04:52:04,707][105620] Updated weights for policy 1, policy_version 1853569 (0.0006) [2023-12-27 04:52:04,773][105620] Updated weights for policy 1, policy_version 1853579 (0.0008) [2023-12-27 04:52:04,823][105620] Updated weights for policy 1, policy_version 1853589 (0.0009) [2023-12-27 04:52:05,259][105692] Updated weights for policy 0, policy_version 1849140 (0.0008) [2023-12-27 04:52:05,312][105692] Updated weights for policy 0, policy_version 1849150 (0.0005) [2023-12-27 04:52:05,363][105692] Updated weights for policy 0, policy_version 1849160 (0.0006) [2023-12-27 04:52:05,536][105620] Updated weights for policy 1, policy_version 1853599 (0.0009) [2023-12-27 04:52:05,582][105620] Updated weights for policy 1, policy_version 1853609 (0.0009) [2023-12-27 04:52:05,630][105620] Updated weights for policy 1, policy_version 1853619 (0.0009) [2023-12-27 04:52:05,980][105692] Updated weights for policy 0, policy_version 1849170 (0.0008) [2023-12-27 04:52:06,038][105692] Updated weights for policy 0, policy_version 1849180 (0.0008) [2023-12-27 04:52:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 948051968. Throughput: 0: 9594.7, 1: 9869.2. Samples: 948042848. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:52:06,062][104569] Avg episode reward: [(0, '8446.363'), (1, '9163.370')] [2023-12-27 04:52:06,096][105692] Updated weights for policy 0, policy_version 1849190 (0.0009) [2023-12-27 04:52:06,159][105692] Updated weights for policy 0, policy_version 1849200 (0.0009) [2023-12-27 04:52:06,387][105620] Updated weights for policy 1, policy_version 1853629 (0.0008) [2023-12-27 04:52:06,452][105620] Updated weights for policy 1, policy_version 1853639 (0.0008) [2023-12-27 04:52:06,513][105620] Updated weights for policy 1, policy_version 1853649 (0.0007) [2023-12-27 04:52:06,984][105692] Updated weights for policy 0, policy_version 1849210 (0.0010) [2023-12-27 04:52:07,047][105692] Updated weights for policy 0, policy_version 1849220 (0.0010) [2023-12-27 04:52:07,097][105692] Updated weights for policy 0, policy_version 1849230 (0.0009) [2023-12-27 04:52:07,102][105620] Updated weights for policy 1, policy_version 1853659 (0.0007) [2023-12-27 04:52:07,161][105620] Updated weights for policy 1, policy_version 1853669 (0.0011) [2023-12-27 04:52:07,223][105620] Updated weights for policy 1, policy_version 1853679 (0.0011) [2023-12-27 04:52:07,813][105692] Updated weights for policy 0, policy_version 1849240 (0.0006) [2023-12-27 04:52:07,866][105692] Updated weights for policy 0, policy_version 1849250 (0.0005) [2023-12-27 04:52:07,919][105692] Updated weights for policy 0, policy_version 1849260 (0.0005) [2023-12-27 04:52:07,968][105620] Updated weights for policy 1, policy_version 1853689 (0.0010) [2023-12-27 04:52:08,033][105620] Updated weights for policy 1, policy_version 1853699 (0.0006) [2023-12-27 04:52:08,084][105620] Updated weights for policy 1, policy_version 1853709 (0.0005) [2023-12-27 04:52:08,140][105620] Updated weights for policy 1, policy_version 1853719 (0.0009) [2023-12-27 04:52:08,458][105692] Updated weights for policy 0, policy_version 1849270 (0.0006) [2023-12-27 04:52:08,520][105692] Updated weights for policy 0, policy_version 1849280 (0.0010) [2023-12-27 04:52:08,573][105692] Updated weights for policy 0, policy_version 1849290 (0.0009) [2023-12-27 04:52:08,812][105620] Updated weights for policy 1, policy_version 1853729 (0.0009) [2023-12-27 04:52:08,871][105620] Updated weights for policy 1, policy_version 1853739 (0.0009) [2023-12-27 04:52:08,936][105620] Updated weights for policy 1, policy_version 1853749 (0.0009) [2023-12-27 04:52:09,337][105692] Updated weights for policy 0, policy_version 1849300 (0.0009) [2023-12-27 04:52:09,403][105692] Updated weights for policy 0, policy_version 1849310 (0.0008) [2023-12-27 04:52:09,471][105692] Updated weights for policy 0, policy_version 1849320 (0.0008) [2023-12-27 04:52:09,732][105620] Updated weights for policy 1, policy_version 1853759 (0.0008) [2023-12-27 04:52:09,794][105620] Updated weights for policy 1, policy_version 1853769 (0.0007) [2023-12-27 04:52:09,859][105620] Updated weights for policy 1, policy_version 1853779 (0.0008) [2023-12-27 04:52:10,242][105692] Updated weights for policy 0, policy_version 1849330 (0.0008) [2023-12-27 04:52:10,296][105692] Updated weights for policy 0, policy_version 1849340 (0.0005) [2023-12-27 04:52:10,353][105692] Updated weights for policy 0, policy_version 1849350 (0.0006) [2023-12-27 04:52:10,411][105692] Updated weights for policy 0, policy_version 1849360 (0.0006) [2023-12-27 04:52:10,578][105620] Updated weights for policy 1, policy_version 1853789 (0.0006) [2023-12-27 04:52:10,636][105620] Updated weights for policy 1, policy_version 1853799 (0.0005) [2023-12-27 04:52:10,694][105620] Updated weights for policy 1, policy_version 1853809 (0.0005) [2023-12-27 04:52:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 948150272. Throughput: 0: 9611.7, 1: 9955.0. Samples: 948162184. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:52:11,063][104569] Avg episode reward: [(0, '8624.773'), (1, '9255.662')] [2023-12-27 04:52:11,187][105692] Updated weights for policy 0, policy_version 1849370 (0.0008) [2023-12-27 04:52:11,249][105692] Updated weights for policy 0, policy_version 1849380 (0.0007) [2023-12-27 04:52:11,285][105620] Updated weights for policy 1, policy_version 1853819 (0.0007) [2023-12-27 04:52:11,311][105692] Updated weights for policy 0, policy_version 1849390 (0.0007) [2023-12-27 04:52:11,351][105620] Updated weights for policy 1, policy_version 1853829 (0.0011) [2023-12-27 04:52:11,416][105620] Updated weights for policy 1, policy_version 1853839 (0.0010) [2023-12-27 04:52:12,139][105692] Updated weights for policy 0, policy_version 1849400 (0.0009) [2023-12-27 04:52:12,147][105620] Updated weights for policy 1, policy_version 1853849 (0.0010) [2023-12-27 04:52:12,205][105620] Updated weights for policy 1, policy_version 1853859 (0.0006) [2023-12-27 04:52:12,205][105692] Updated weights for policy 0, policy_version 1849410 (0.0008) [2023-12-27 04:52:12,275][105692] Updated weights for policy 0, policy_version 1849420 (0.0008) [2023-12-27 04:52:12,278][105620] Updated weights for policy 1, policy_version 1853869 (0.0007) [2023-12-27 04:52:12,345][105620] Updated weights for policy 1, policy_version 1853879 (0.0008) [2023-12-27 04:52:12,991][105692] Updated weights for policy 0, policy_version 1849430 (0.0009) [2023-12-27 04:52:13,047][105692] Updated weights for policy 0, policy_version 1849440 (0.0009) [2023-12-27 04:52:13,098][105692] Updated weights for policy 0, policy_version 1849450 (0.0008) [2023-12-27 04:52:13,109][105620] Updated weights for policy 1, policy_version 1853889 (0.0007) [2023-12-27 04:52:13,161][105620] Updated weights for policy 1, policy_version 1853899 (0.0008) [2023-12-27 04:52:13,208][105620] Updated weights for policy 1, policy_version 1853909 (0.0008) [2023-12-27 04:52:13,754][105692] Updated weights for policy 0, policy_version 1849460 (0.0007) [2023-12-27 04:52:13,822][105692] Updated weights for policy 0, policy_version 1849470 (0.0006) [2023-12-27 04:52:13,888][105692] Updated weights for policy 0, policy_version 1849480 (0.0009) [2023-12-27 04:52:13,988][105620] Updated weights for policy 1, policy_version 1853919 (0.0009) [2023-12-27 04:52:14,041][105620] Updated weights for policy 1, policy_version 1853929 (0.0008) [2023-12-27 04:52:14,103][105620] Updated weights for policy 1, policy_version 1853939 (0.0009) [2023-12-27 04:52:14,524][105692] Updated weights for policy 0, policy_version 1849490 (0.0009) [2023-12-27 04:52:14,577][105692] Updated weights for policy 0, policy_version 1849500 (0.0005) [2023-12-27 04:52:14,630][105692] Updated weights for policy 0, policy_version 1849510 (0.0007) [2023-12-27 04:52:14,678][105692] Updated weights for policy 0, policy_version 1849520 (0.0009) [2023-12-27 04:52:14,898][105620] Updated weights for policy 1, policy_version 1853949 (0.0008) [2023-12-27 04:52:14,964][105620] Updated weights for policy 1, policy_version 1853959 (0.0009) [2023-12-27 04:52:15,035][105620] Updated weights for policy 1, policy_version 1853969 (0.0010) [2023-12-27 04:52:15,342][105692] Updated weights for policy 0, policy_version 1849530 (0.0009) [2023-12-27 04:52:15,395][105692] Updated weights for policy 0, policy_version 1849540 (0.0009) [2023-12-27 04:52:15,450][105692] Updated weights for policy 0, policy_version 1849550 (0.0009) [2023-12-27 04:52:15,782][105620] Updated weights for policy 1, policy_version 1853979 (0.0008) [2023-12-27 04:52:15,842][105620] Updated weights for policy 1, policy_version 1853989 (0.0007) [2023-12-27 04:52:15,896][105620] Updated weights for policy 1, policy_version 1853999 (0.0010) [2023-12-27 04:52:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.8, 300 sec: 19438.7). Total num frames: 948248576. Throughput: 0: 9589.4, 1: 9848.3. Samples: 948218192. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:52:16,062][104569] Avg episode reward: [(0, '8715.323'), (1, '9347.872')] [2023-12-27 04:52:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001854008_474693632.pth... [2023-12-27 04:52:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001849552_473554944.pth... [2023-12-27 04:52:16,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001852856_474398720.pth [2023-12-27 04:52:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001848432_473268224.pth [2023-12-27 04:52:16,133][105692] Updated weights for policy 0, policy_version 1849560 (0.0006) [2023-12-27 04:52:16,187][105692] Updated weights for policy 0, policy_version 1849570 (0.0005) [2023-12-27 04:52:16,236][105692] Updated weights for policy 0, policy_version 1849580 (0.0005) [2023-12-27 04:52:16,762][105692] Updated weights for policy 0, policy_version 1849590 (0.0005) [2023-12-27 04:52:16,778][105620] Updated weights for policy 1, policy_version 1854009 (0.0010) [2023-12-27 04:52:16,810][105692] Updated weights for policy 0, policy_version 1849600 (0.0006) [2023-12-27 04:52:16,828][105620] Updated weights for policy 1, policy_version 1854019 (0.0007) [2023-12-27 04:52:16,858][105692] Updated weights for policy 0, policy_version 1849610 (0.0005) [2023-12-27 04:52:16,886][105620] Updated weights for policy 1, policy_version 1854029 (0.0009) [2023-12-27 04:52:16,943][105620] Updated weights for policy 1, policy_version 1854039 (0.0009) [2023-12-27 04:52:17,498][105692] Updated weights for policy 0, policy_version 1849620 (0.0008) [2023-12-27 04:52:17,557][105692] Updated weights for policy 0, policy_version 1849630 (0.0010) [2023-12-27 04:52:17,615][105692] Updated weights for policy 0, policy_version 1849640 (0.0010) [2023-12-27 04:52:17,733][105620] Updated weights for policy 1, policy_version 1854049 (0.0008) [2023-12-27 04:52:17,781][105620] Updated weights for policy 1, policy_version 1854059 (0.0008) [2023-12-27 04:52:17,828][105620] Updated weights for policy 1, policy_version 1854069 (0.0007) [2023-12-27 04:52:18,344][105692] Updated weights for policy 0, policy_version 1849650 (0.0010) [2023-12-27 04:52:18,413][105692] Updated weights for policy 0, policy_version 1849660 (0.0010) [2023-12-27 04:52:18,476][105692] Updated weights for policy 0, policy_version 1849670 (0.0007) [2023-12-27 04:52:18,533][105692] Updated weights for policy 0, policy_version 1849680 (0.0006) [2023-12-27 04:52:18,605][105620] Updated weights for policy 1, policy_version 1854079 (0.0007) [2023-12-27 04:52:18,664][105620] Updated weights for policy 1, policy_version 1854089 (0.0010) [2023-12-27 04:52:18,725][105620] Updated weights for policy 1, policy_version 1854099 (0.0008) [2023-12-27 04:52:19,196][105692] Updated weights for policy 0, policy_version 1849690 (0.0011) [2023-12-27 04:52:19,256][105692] Updated weights for policy 0, policy_version 1849700 (0.0011) [2023-12-27 04:52:19,305][105692] Updated weights for policy 0, policy_version 1849710 (0.0010) [2023-12-27 04:52:19,407][105620] Updated weights for policy 1, policy_version 1854109 (0.0008) [2023-12-27 04:52:19,462][105620] Updated weights for policy 1, policy_version 1854119 (0.0010) [2023-12-27 04:52:19,519][105620] Updated weights for policy 1, policy_version 1854129 (0.0010) [2023-12-27 04:52:20,050][105692] Updated weights for policy 0, policy_version 1849720 (0.0010) [2023-12-27 04:52:20,126][105692] Updated weights for policy 0, policy_version 1849730 (0.0010) [2023-12-27 04:52:20,192][105692] Updated weights for policy 0, policy_version 1849740 (0.0008) [2023-12-27 04:52:20,221][105620] Updated weights for policy 1, policy_version 1854139 (0.0010) [2023-12-27 04:52:20,277][105620] Updated weights for policy 1, policy_version 1854149 (0.0010) [2023-12-27 04:52:20,339][105620] Updated weights for policy 1, policy_version 1854159 (0.0010) [2023-12-27 04:52:20,988][105620] Updated weights for policy 1, policy_version 1854169 (0.0011) [2023-12-27 04:52:21,010][105692] Updated weights for policy 0, policy_version 1849750 (0.0007) [2023-12-27 04:52:21,056][105620] Updated weights for policy 1, policy_version 1854179 (0.0011) [2023-12-27 04:52:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 948338688. Throughput: 0: 9669.4, 1: 9721.8. Samples: 948335108. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:52:21,063][104569] Avg episode reward: [(0, '8533.691'), (1, '9347.856')] [2023-12-27 04:52:21,078][105692] Updated weights for policy 0, policy_version 1849760 (0.0007) [2023-12-27 04:52:21,117][105620] Updated weights for policy 1, policy_version 1854189 (0.0010) [2023-12-27 04:52:21,144][105692] Updated weights for policy 0, policy_version 1849770 (0.0006) [2023-12-27 04:52:21,180][105620] Updated weights for policy 1, policy_version 1854199 (0.0011) [2023-12-27 04:52:21,912][105692] Updated weights for policy 0, policy_version 1849780 (0.0006) [2023-12-27 04:52:21,974][105692] Updated weights for policy 0, policy_version 1849790 (0.0007) [2023-12-27 04:52:21,999][105620] Updated weights for policy 1, policy_version 1854209 (0.0010) [2023-12-27 04:52:22,030][105692] Updated weights for policy 0, policy_version 1849800 (0.0007) [2023-12-27 04:52:22,052][105620] Updated weights for policy 1, policy_version 1854219 (0.0010) [2023-12-27 04:52:22,111][105620] Updated weights for policy 1, policy_version 1854229 (0.0010) [2023-12-27 04:52:22,821][105692] Updated weights for policy 0, policy_version 1849810 (0.0006) [2023-12-27 04:52:22,866][105620] Updated weights for policy 1, policy_version 1854239 (0.0011) [2023-12-27 04:52:22,872][105692] Updated weights for policy 0, policy_version 1849820 (0.0009) [2023-12-27 04:52:22,920][105620] Updated weights for policy 1, policy_version 1854249 (0.0010) [2023-12-27 04:52:22,922][105692] Updated weights for policy 0, policy_version 1849830 (0.0006) [2023-12-27 04:52:22,972][105692] Updated weights for policy 0, policy_version 1849840 (0.0007) [2023-12-27 04:52:22,984][105620] Updated weights for policy 1, policy_version 1854259 (0.0010) [2023-12-27 04:52:23,689][105620] Updated weights for policy 1, policy_version 1854269 (0.0008) [2023-12-27 04:52:23,694][105692] Updated weights for policy 0, policy_version 1849850 (0.0007) [2023-12-27 04:52:23,742][105620] Updated weights for policy 1, policy_version 1854279 (0.0005) [2023-12-27 04:52:23,747][105692] Updated weights for policy 0, policy_version 1849860 (0.0009) [2023-12-27 04:52:23,789][105620] Updated weights for policy 1, policy_version 1854289 (0.0005) [2023-12-27 04:52:23,799][105692] Updated weights for policy 0, policy_version 1849870 (0.0009) [2023-12-27 04:52:24,315][105620] Updated weights for policy 1, policy_version 1854299 (0.0005) [2023-12-27 04:52:24,362][105620] Updated weights for policy 1, policy_version 1854309 (0.0005) [2023-12-27 04:52:24,418][105620] Updated weights for policy 1, policy_version 1854319 (0.0008) [2023-12-27 04:52:24,469][105692] Updated weights for policy 0, policy_version 1849880 (0.0009) [2023-12-27 04:52:24,531][105692] Updated weights for policy 0, policy_version 1849890 (0.0010) [2023-12-27 04:52:24,590][105692] Updated weights for policy 0, policy_version 1849900 (0.0010) [2023-12-27 04:52:24,980][105620] Updated weights for policy 1, policy_version 1854329 (0.0005) [2023-12-27 04:52:25,031][105620] Updated weights for policy 1, policy_version 1854339 (0.0005) [2023-12-27 04:52:25,079][105620] Updated weights for policy 1, policy_version 1854349 (0.0007) [2023-12-27 04:52:25,134][105620] Updated weights for policy 1, policy_version 1854359 (0.0010) [2023-12-27 04:52:25,185][105692] Updated weights for policy 0, policy_version 1849910 (0.0007) [2023-12-27 04:52:25,230][105692] Updated weights for policy 0, policy_version 1849920 (0.0005) [2023-12-27 04:52:25,280][105692] Updated weights for policy 0, policy_version 1849930 (0.0008) [2023-12-27 04:52:25,849][105692] Updated weights for policy 0, policy_version 1849940 (0.0007) [2023-12-27 04:52:25,867][105620] Updated weights for policy 1, policy_version 1854369 (0.0010) [2023-12-27 04:52:25,901][105692] Updated weights for policy 0, policy_version 1849950 (0.0005) [2023-12-27 04:52:25,919][105620] Updated weights for policy 1, policy_version 1854379 (0.0010) [2023-12-27 04:52:25,957][105692] Updated weights for policy 0, policy_version 1849960 (0.0007) [2023-12-27 04:52:25,968][105620] Updated weights for policy 1, policy_version 1854389 (0.0010) [2023-12-27 04:52:26,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 948453376. Throughput: 0: 9690.2, 1: 9774.8. Samples: 948455116. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:52:26,062][104569] Avg episode reward: [(0, '8441.802'), (1, '9347.832')] [2023-12-27 04:52:26,590][105692] Updated weights for policy 0, policy_version 1849970 (0.0007) [2023-12-27 04:52:26,637][105692] Updated weights for policy 0, policy_version 1849980 (0.0010) [2023-12-27 04:52:26,696][105692] Updated weights for policy 0, policy_version 1849990 (0.0009) [2023-12-27 04:52:26,742][105620] Updated weights for policy 1, policy_version 1854399 (0.0010) [2023-12-27 04:52:26,762][105692] Updated weights for policy 0, policy_version 1850000 (0.0005) [2023-12-27 04:52:26,801][105620] Updated weights for policy 1, policy_version 1854409 (0.0011) [2023-12-27 04:52:26,866][105620] Updated weights for policy 1, policy_version 1854419 (0.0010) [2023-12-27 04:52:27,359][105692] Updated weights for policy 0, policy_version 1850010 (0.0007) [2023-12-27 04:52:27,427][105692] Updated weights for policy 0, policy_version 1850020 (0.0010) [2023-12-27 04:52:27,484][105692] Updated weights for policy 0, policy_version 1850030 (0.0010) [2023-12-27 04:52:27,604][105620] Updated weights for policy 1, policy_version 1854429 (0.0010) [2023-12-27 04:52:27,655][105620] Updated weights for policy 1, policy_version 1854439 (0.0010) [2023-12-27 04:52:27,717][105620] Updated weights for policy 1, policy_version 1854449 (0.0011) [2023-12-27 04:52:28,099][105692] Updated weights for policy 0, policy_version 1850040 (0.0010) [2023-12-27 04:52:28,157][105692] Updated weights for policy 0, policy_version 1850050 (0.0010) [2023-12-27 04:52:28,215][105692] Updated weights for policy 0, policy_version 1850060 (0.0010) [2023-12-27 04:52:28,497][105620] Updated weights for policy 1, policy_version 1854459 (0.0010) [2023-12-27 04:52:28,563][105620] Updated weights for policy 1, policy_version 1854469 (0.0011) [2023-12-27 04:52:28,619][105620] Updated weights for policy 1, policy_version 1854479 (0.0011) [2023-12-27 04:52:28,922][105692] Updated weights for policy 0, policy_version 1850070 (0.0010) [2023-12-27 04:52:28,973][105692] Updated weights for policy 0, policy_version 1850080 (0.0010) [2023-12-27 04:52:29,026][105692] Updated weights for policy 0, policy_version 1850090 (0.0010) [2023-12-27 04:52:29,365][105620] Updated weights for policy 1, policy_version 1854489 (0.0010) [2023-12-27 04:52:29,427][105620] Updated weights for policy 1, policy_version 1854499 (0.0011) [2023-12-27 04:52:29,486][105620] Updated weights for policy 1, policy_version 1854509 (0.0011) [2023-12-27 04:52:29,545][105620] Updated weights for policy 1, policy_version 1854519 (0.0011) [2023-12-27 04:52:29,695][105692] Updated weights for policy 0, policy_version 1850100 (0.0008) [2023-12-27 04:52:29,754][105692] Updated weights for policy 0, policy_version 1850110 (0.0006) [2023-12-27 04:52:29,805][105692] Updated weights for policy 0, policy_version 1850120 (0.0005) [2023-12-27 04:52:30,297][105620] Updated weights for policy 1, policy_version 1854529 (0.0010) [2023-12-27 04:52:30,351][105620] Updated weights for policy 1, policy_version 1854539 (0.0008) [2023-12-27 04:52:30,412][105620] Updated weights for policy 1, policy_version 1854549 (0.0005) [2023-12-27 04:52:30,472][105692] Updated weights for policy 0, policy_version 1850130 (0.0009) [2023-12-27 04:52:30,533][105692] Updated weights for policy 0, policy_version 1850140 (0.0005) [2023-12-27 04:52:30,592][105692] Updated weights for policy 0, policy_version 1850150 (0.0007) [2023-12-27 04:52:30,644][105692] Updated weights for policy 0, policy_version 1850160 (0.0010) [2023-12-27 04:52:31,007][105620] Updated weights for policy 1, policy_version 1854559 (0.0007) [2023-12-27 04:52:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 948543488. Throughput: 0: 9808.2, 1: 9726.0. Samples: 948515652. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:52:31,063][104569] Avg episode reward: [(0, '8170.362'), (1, '9163.189')] [2023-12-27 04:52:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001850160_473710592.pth... [2023-12-27 04:52:31,070][105620] Updated weights for policy 1, policy_version 1854569 (0.0009) [2023-12-27 04:52:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001849008_473415680.pth [2023-12-27 04:52:31,136][105620] Updated weights for policy 1, policy_version 1854579 (0.0009) [2023-12-27 04:52:31,169][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001854584_474841088.pth... [2023-12-27 04:52:31,175][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001853432_474546176.pth [2023-12-27 04:52:31,360][105692] Updated weights for policy 0, policy_version 1850170 (0.0009) [2023-12-27 04:52:31,424][105692] Updated weights for policy 0, policy_version 1850180 (0.0007) [2023-12-27 04:52:31,480][105692] Updated weights for policy 0, policy_version 1850190 (0.0006) [2023-12-27 04:52:31,906][105620] Updated weights for policy 1, policy_version 1854589 (0.0008) [2023-12-27 04:52:31,960][105620] Updated weights for policy 1, policy_version 1854599 (0.0009) [2023-12-27 04:52:32,016][105620] Updated weights for policy 1, policy_version 1854609 (0.0009) [2023-12-27 04:52:32,124][105692] Updated weights for policy 0, policy_version 1850200 (0.0009) [2023-12-27 04:52:32,182][105692] Updated weights for policy 0, policy_version 1850210 (0.0010) [2023-12-27 04:52:32,236][105692] Updated weights for policy 0, policy_version 1850220 (0.0010) [2023-12-27 04:52:32,701][105620] Updated weights for policy 1, policy_version 1854619 (0.0007) [2023-12-27 04:52:32,755][105620] Updated weights for policy 1, policy_version 1854629 (0.0010) [2023-12-27 04:52:32,803][105620] Updated weights for policy 1, policy_version 1854639 (0.0010) [2023-12-27 04:52:33,004][105692] Updated weights for policy 0, policy_version 1850230 (0.0009) [2023-12-27 04:52:33,063][105692] Updated weights for policy 0, policy_version 1850240 (0.0007) [2023-12-27 04:52:33,122][105692] Updated weights for policy 0, policy_version 1850250 (0.0008) [2023-12-27 04:52:33,567][105620] Updated weights for policy 1, policy_version 1854649 (0.0008) [2023-12-27 04:52:33,616][105620] Updated weights for policy 1, policy_version 1854659 (0.0006) [2023-12-27 04:52:33,667][105620] Updated weights for policy 1, policy_version 1854669 (0.0005) [2023-12-27 04:52:33,671][105692] Updated weights for policy 0, policy_version 1850260 (0.0009) [2023-12-27 04:52:33,713][105620] Updated weights for policy 1, policy_version 1854679 (0.0005) [2023-12-27 04:52:33,715][105692] Updated weights for policy 0, policy_version 1850270 (0.0010) [2023-12-27 04:52:33,762][105692] Updated weights for policy 0, policy_version 1850280 (0.0010) [2023-12-27 04:52:34,263][105620] Updated weights for policy 1, policy_version 1854689 (0.0008) [2023-12-27 04:52:34,320][105620] Updated weights for policy 1, policy_version 1854699 (0.0008) [2023-12-27 04:52:34,387][105620] Updated weights for policy 1, policy_version 1854709 (0.0007) [2023-12-27 04:52:34,433][105692] Updated weights for policy 0, policy_version 1850290 (0.0010) [2023-12-27 04:52:34,504][105692] Updated weights for policy 0, policy_version 1850300 (0.0010) [2023-12-27 04:52:34,577][105692] Updated weights for policy 0, policy_version 1850310 (0.0008) [2023-12-27 04:52:34,635][105692] Updated weights for policy 0, policy_version 1850320 (0.0005) [2023-12-27 04:52:35,012][105620] Updated weights for policy 1, policy_version 1854719 (0.0006) [2023-12-27 04:52:35,077][105620] Updated weights for policy 1, policy_version 1854729 (0.0008) [2023-12-27 04:52:35,144][105620] Updated weights for policy 1, policy_version 1854739 (0.0008) [2023-12-27 04:52:35,327][105692] Updated weights for policy 0, policy_version 1850330 (0.0010) [2023-12-27 04:52:35,388][105692] Updated weights for policy 0, policy_version 1850340 (0.0010) [2023-12-27 04:52:35,433][105692] Updated weights for policy 0, policy_version 1850350 (0.0010) [2023-12-27 04:52:35,711][105620] Updated weights for policy 1, policy_version 1854749 (0.0007) [2023-12-27 04:52:35,770][105620] Updated weights for policy 1, policy_version 1854759 (0.0008) [2023-12-27 04:52:35,825][105620] Updated weights for policy 1, policy_version 1854769 (0.0010) [2023-12-27 04:52:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 948649984. Throughput: 0: 9885.4, 1: 9708.2. Samples: 948638352. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:52:36,063][104569] Avg episode reward: [(0, '7898.614'), (1, '8978.240')] [2023-12-27 04:52:36,151][105692] Updated weights for policy 0, policy_version 1850360 (0.0008) [2023-12-27 04:52:36,209][105692] Updated weights for policy 0, policy_version 1850370 (0.0011) [2023-12-27 04:52:36,269][105692] Updated weights for policy 0, policy_version 1850380 (0.0011) [2023-12-27 04:52:36,515][105620] Updated weights for policy 1, policy_version 1854779 (0.0010) [2023-12-27 04:52:36,581][105620] Updated weights for policy 1, policy_version 1854789 (0.0011) [2023-12-27 04:52:36,649][105620] Updated weights for policy 1, policy_version 1854799 (0.0011) [2023-12-27 04:52:37,017][105692] Updated weights for policy 0, policy_version 1850390 (0.0010) [2023-12-27 04:52:37,067][105692] Updated weights for policy 0, policy_version 1850400 (0.0010) [2023-12-27 04:52:37,124][105692] Updated weights for policy 0, policy_version 1850410 (0.0010) [2023-12-27 04:52:37,341][105620] Updated weights for policy 1, policy_version 1854809 (0.0010) [2023-12-27 04:52:37,391][105620] Updated weights for policy 1, policy_version 1854819 (0.0008) [2023-12-27 04:52:37,444][105620] Updated weights for policy 1, policy_version 1854829 (0.0007) [2023-12-27 04:52:37,503][105620] Updated weights for policy 1, policy_version 1854839 (0.0010) [2023-12-27 04:52:37,879][105692] Updated weights for policy 0, policy_version 1850420 (0.0009) [2023-12-27 04:52:37,938][105692] Updated weights for policy 0, policy_version 1850430 (0.0007) [2023-12-27 04:52:37,994][105692] Updated weights for policy 0, policy_version 1850440 (0.0008) [2023-12-27 04:52:38,166][105620] Updated weights for policy 1, policy_version 1854849 (0.0010) [2023-12-27 04:52:38,214][105620] Updated weights for policy 1, policy_version 1854859 (0.0010) [2023-12-27 04:52:38,259][105620] Updated weights for policy 1, policy_version 1854869 (0.0010) [2023-12-27 04:52:38,573][105692] Updated weights for policy 0, policy_version 1850450 (0.0008) [2023-12-27 04:52:38,628][105692] Updated weights for policy 0, policy_version 1850460 (0.0010) [2023-12-27 04:52:38,677][105692] Updated weights for policy 0, policy_version 1850470 (0.0010) [2023-12-27 04:52:38,729][105692] Updated weights for policy 0, policy_version 1850480 (0.0010) [2023-12-27 04:52:39,022][105620] Updated weights for policy 1, policy_version 1854879 (0.0010) [2023-12-27 04:52:39,080][105620] Updated weights for policy 1, policy_version 1854889 (0.0010) [2023-12-27 04:52:39,151][105620] Updated weights for policy 1, policy_version 1854899 (0.0010) [2023-12-27 04:52:39,496][105692] Updated weights for policy 0, policy_version 1850490 (0.0006) [2023-12-27 04:52:39,556][105692] Updated weights for policy 0, policy_version 1850500 (0.0011) [2023-12-27 04:52:39,618][105692] Updated weights for policy 0, policy_version 1850510 (0.0011) [2023-12-27 04:52:39,897][105620] Updated weights for policy 1, policy_version 1854909 (0.0010) [2023-12-27 04:52:39,956][105620] Updated weights for policy 1, policy_version 1854919 (0.0010) [2023-12-27 04:52:40,006][105620] Updated weights for policy 1, policy_version 1854929 (0.0010) [2023-12-27 04:52:40,373][105692] Updated weights for policy 0, policy_version 1850520 (0.0007) [2023-12-27 04:52:40,442][105692] Updated weights for policy 0, policy_version 1850530 (0.0006) [2023-12-27 04:52:40,507][105692] Updated weights for policy 0, policy_version 1850540 (0.0009) [2023-12-27 04:52:40,760][105620] Updated weights for policy 1, policy_version 1854939 (0.0009) [2023-12-27 04:52:40,825][105620] Updated weights for policy 1, policy_version 1854949 (0.0007) [2023-12-27 04:52:40,888][105620] Updated weights for policy 1, policy_version 1854959 (0.0005) [2023-12-27 04:52:41,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 948748288. Throughput: 0: 9947.4, 1: 9709.3. Samples: 948756200. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:52:41,063][104569] Avg episode reward: [(0, '8532.175'), (1, '8705.150')] [2023-12-27 04:52:41,351][105692] Updated weights for policy 0, policy_version 1850550 (0.0008) [2023-12-27 04:52:41,431][105692] Updated weights for policy 0, policy_version 1850560 (0.0007) [2023-12-27 04:52:41,498][105692] Updated weights for policy 0, policy_version 1850570 (0.0006) [2023-12-27 04:52:41,608][105620] Updated weights for policy 1, policy_version 1854969 (0.0006) [2023-12-27 04:52:41,679][105620] Updated weights for policy 1, policy_version 1854979 (0.0011) [2023-12-27 04:52:41,745][105620] Updated weights for policy 1, policy_version 1854989 (0.0008) [2023-12-27 04:52:41,811][105620] Updated weights for policy 1, policy_version 1854999 (0.0006) [2023-12-27 04:52:42,294][105692] Updated weights for policy 0, policy_version 1850580 (0.0008) [2023-12-27 04:52:42,363][105692] Updated weights for policy 0, policy_version 1850590 (0.0008) [2023-12-27 04:52:42,432][105692] Updated weights for policy 0, policy_version 1850600 (0.0008) [2023-12-27 04:52:42,436][105620] Updated weights for policy 1, policy_version 1855009 (0.0007) [2023-12-27 04:52:42,497][105620] Updated weights for policy 1, policy_version 1855019 (0.0008) [2023-12-27 04:52:42,560][105620] Updated weights for policy 1, policy_version 1855029 (0.0008) [2023-12-27 04:52:43,144][105692] Updated weights for policy 0, policy_version 1850610 (0.0008) [2023-12-27 04:52:43,209][105692] Updated weights for policy 0, policy_version 1850620 (0.0009) [2023-12-27 04:52:43,267][105692] Updated weights for policy 0, policy_version 1850630 (0.0008) [2023-12-27 04:52:43,296][105620] Updated weights for policy 1, policy_version 1855039 (0.0007) [2023-12-27 04:52:43,321][105692] Updated weights for policy 0, policy_version 1850640 (0.0007) [2023-12-27 04:52:43,357][105620] Updated weights for policy 1, policy_version 1855049 (0.0007) [2023-12-27 04:52:43,411][105620] Updated weights for policy 1, policy_version 1855059 (0.0006) [2023-12-27 04:52:43,959][105692] Updated weights for policy 0, policy_version 1850650 (0.0007) [2023-12-27 04:52:43,994][105620] Updated weights for policy 1, policy_version 1855069 (0.0008) [2023-12-27 04:52:44,014][105692] Updated weights for policy 0, policy_version 1850660 (0.0010) [2023-12-27 04:52:44,056][105620] Updated weights for policy 1, policy_version 1855079 (0.0011) [2023-12-27 04:52:44,079][105692] Updated weights for policy 0, policy_version 1850670 (0.0010) [2023-12-27 04:52:44,114][105620] Updated weights for policy 1, policy_version 1855089 (0.0011) [2023-12-27 04:52:44,730][105620] Updated weights for policy 1, policy_version 1855099 (0.0009) [2023-12-27 04:52:44,765][105692] Updated weights for policy 0, policy_version 1850680 (0.0010) [2023-12-27 04:52:44,796][105620] Updated weights for policy 1, policy_version 1855109 (0.0009) [2023-12-27 04:52:44,833][105692] Updated weights for policy 0, policy_version 1850690 (0.0010) [2023-12-27 04:52:44,859][105620] Updated weights for policy 1, policy_version 1855119 (0.0011) [2023-12-27 04:52:44,896][105692] Updated weights for policy 0, policy_version 1850700 (0.0010) [2023-12-27 04:52:45,560][105620] Updated weights for policy 1, policy_version 1855129 (0.0010) [2023-12-27 04:52:45,619][105620] Updated weights for policy 1, policy_version 1855139 (0.0007) [2023-12-27 04:52:45,657][105692] Updated weights for policy 0, policy_version 1850710 (0.0010) [2023-12-27 04:52:45,675][105620] Updated weights for policy 1, policy_version 1855149 (0.0007) [2023-12-27 04:52:45,711][105692] Updated weights for policy 0, policy_version 1850720 (0.0006) [2023-12-27 04:52:45,730][105620] Updated weights for policy 1, policy_version 1855159 (0.0008) [2023-12-27 04:52:45,768][105692] Updated weights for policy 0, policy_version 1850730 (0.0006) [2023-12-27 04:52:46,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 948846592. Throughput: 0: 9880.7, 1: 9793.2. Samples: 948813592. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:52:46,063][104569] Avg episode reward: [(0, '8898.211'), (1, '8889.923')] [2023-12-27 04:52:46,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001850736_473858048.pth... [2023-12-27 04:52:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001855160_474988544.pth... [2023-12-27 04:52:46,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001854008_474693632.pth [2023-12-27 04:52:46,079][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001849552_473554944.pth [2023-12-27 04:52:46,402][105620] Updated weights for policy 1, policy_version 1855169 (0.0006) [2023-12-27 04:52:46,415][105692] Updated weights for policy 0, policy_version 1850740 (0.0007) [2023-12-27 04:52:46,450][105620] Updated weights for policy 1, policy_version 1855179 (0.0007) [2023-12-27 04:52:46,470][105692] Updated weights for policy 0, policy_version 1850750 (0.0006) [2023-12-27 04:52:46,509][105620] Updated weights for policy 1, policy_version 1855189 (0.0008) [2023-12-27 04:52:46,527][105692] Updated weights for policy 0, policy_version 1850760 (0.0006) [2023-12-27 04:52:47,180][105620] Updated weights for policy 1, policy_version 1855199 (0.0009) [2023-12-27 04:52:47,236][105620] Updated weights for policy 1, policy_version 1855209 (0.0009) [2023-12-27 04:52:47,284][105692] Updated weights for policy 0, policy_version 1850770 (0.0007) [2023-12-27 04:52:47,295][105620] Updated weights for policy 1, policy_version 1855219 (0.0008) [2023-12-27 04:52:47,343][105692] Updated weights for policy 0, policy_version 1850780 (0.0007) [2023-12-27 04:52:47,404][105692] Updated weights for policy 0, policy_version 1850790 (0.0005) [2023-12-27 04:52:47,465][105692] Updated weights for policy 0, policy_version 1850800 (0.0009) [2023-12-27 04:52:48,043][105620] Updated weights for policy 1, policy_version 1855229 (0.0009) [2023-12-27 04:52:48,095][105620] Updated weights for policy 1, policy_version 1855239 (0.0008) [2023-12-27 04:52:48,129][105692] Updated weights for policy 0, policy_version 1850810 (0.0008) [2023-12-27 04:52:48,140][105620] Updated weights for policy 1, policy_version 1855249 (0.0007) [2023-12-27 04:52:48,180][105692] Updated weights for policy 0, policy_version 1850820 (0.0007) [2023-12-27 04:52:48,234][105692] Updated weights for policy 0, policy_version 1850830 (0.0009) [2023-12-27 04:52:48,866][105620] Updated weights for policy 1, policy_version 1855259 (0.0007) [2023-12-27 04:52:48,924][105620] Updated weights for policy 1, policy_version 1855269 (0.0011) [2023-12-27 04:52:48,977][105620] Updated weights for policy 1, policy_version 1855279 (0.0011) [2023-12-27 04:52:49,040][105692] Updated weights for policy 0, policy_version 1850840 (0.0009) [2023-12-27 04:52:49,096][105692] Updated weights for policy 0, policy_version 1850850 (0.0008) [2023-12-27 04:52:49,156][105692] Updated weights for policy 0, policy_version 1850860 (0.0008) [2023-12-27 04:52:49,736][105620] Updated weights for policy 1, policy_version 1855289 (0.0010) [2023-12-27 04:52:49,799][105620] Updated weights for policy 1, policy_version 1855299 (0.0006) [2023-12-27 04:52:49,859][105620] Updated weights for policy 1, policy_version 1855309 (0.0011) [2023-12-27 04:52:49,920][105620] Updated weights for policy 1, policy_version 1855319 (0.0010) [2023-12-27 04:52:49,983][105692] Updated weights for policy 0, policy_version 1850870 (0.0009) [2023-12-27 04:52:50,033][105692] Updated weights for policy 0, policy_version 1850880 (0.0011) [2023-12-27 04:52:50,081][105692] Updated weights for policy 0, policy_version 1850890 (0.0010) [2023-12-27 04:52:50,649][105620] Updated weights for policy 1, policy_version 1855329 (0.0006) [2023-12-27 04:52:50,713][105620] Updated weights for policy 1, policy_version 1855339 (0.0007) [2023-12-27 04:52:50,763][105620] Updated weights for policy 1, policy_version 1855349 (0.0008) [2023-12-27 04:52:50,866][105692] Updated weights for policy 0, policy_version 1850900 (0.0010) [2023-12-27 04:52:50,923][105692] Updated weights for policy 0, policy_version 1850910 (0.0008) [2023-12-27 04:52:50,990][105692] Updated weights for policy 0, policy_version 1850920 (0.0009) [2023-12-27 04:52:51,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.9, 300 sec: 19494.2). Total num frames: 948944896. Throughput: 0: 9914.9, 1: 9821.1. Samples: 948930968. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:52:51,062][104569] Avg episode reward: [(0, '8897.085'), (1, '9163.034')] [2023-12-27 04:52:51,540][105620] Updated weights for policy 1, policy_version 1855359 (0.0008) [2023-12-27 04:52:51,592][105620] Updated weights for policy 1, policy_version 1855369 (0.0007) [2023-12-27 04:52:51,658][105620] Updated weights for policy 1, policy_version 1855379 (0.0008) [2023-12-27 04:52:51,767][105692] Updated weights for policy 0, policy_version 1850930 (0.0009) [2023-12-27 04:52:51,823][105692] Updated weights for policy 0, policy_version 1850940 (0.0010) [2023-12-27 04:52:51,883][105692] Updated weights for policy 0, policy_version 1850951 (0.0008) [2023-12-27 04:52:52,375][105620] Updated weights for policy 1, policy_version 1855389 (0.0009) [2023-12-27 04:52:52,436][105620] Updated weights for policy 1, policy_version 1855399 (0.0009) [2023-12-27 04:52:52,493][105620] Updated weights for policy 1, policy_version 1855409 (0.0008) [2023-12-27 04:52:52,667][105692] Updated weights for policy 0, policy_version 1850961 (0.0008) [2023-12-27 04:52:52,727][105692] Updated weights for policy 0, policy_version 1850971 (0.0008) [2023-12-27 04:52:52,781][105692] Updated weights for policy 0, policy_version 1850981 (0.0008) [2023-12-27 04:52:52,841][105692] Updated weights for policy 0, policy_version 1850991 (0.0008) [2023-12-27 04:52:53,174][105620] Updated weights for policy 1, policy_version 1855419 (0.0008) [2023-12-27 04:52:53,235][105620] Updated weights for policy 1, policy_version 1855429 (0.0009) [2023-12-27 04:52:53,293][105620] Updated weights for policy 1, policy_version 1855439 (0.0009) [2023-12-27 04:52:53,663][105692] Updated weights for policy 0, policy_version 1851001 (0.0010) [2023-12-27 04:52:53,719][105692] Updated weights for policy 0, policy_version 1851011 (0.0009) [2023-12-27 04:52:53,778][105692] Updated weights for policy 0, policy_version 1851021 (0.0009) [2023-12-27 04:52:53,932][105620] Updated weights for policy 1, policy_version 1855449 (0.0008) [2023-12-27 04:52:53,993][105620] Updated weights for policy 1, policy_version 1855459 (0.0009) [2023-12-27 04:52:54,047][105620] Updated weights for policy 1, policy_version 1855469 (0.0009) [2023-12-27 04:52:54,094][105620] Updated weights for policy 1, policy_version 1855479 (0.0009) [2023-12-27 04:52:54,656][105692] Updated weights for policy 0, policy_version 1851031 (0.0009) [2023-12-27 04:52:54,706][105692] Updated weights for policy 0, policy_version 1851041 (0.0007) [2023-12-27 04:52:54,708][105620] Updated weights for policy 1, policy_version 1855489 (0.0010) [2023-12-27 04:52:54,759][105692] Updated weights for policy 0, policy_version 1851051 (0.0009) [2023-12-27 04:52:54,762][105620] Updated weights for policy 1, policy_version 1855499 (0.0010) [2023-12-27 04:52:54,823][105620] Updated weights for policy 1, policy_version 1855509 (0.0010) [2023-12-27 04:52:55,542][105692] Updated weights for policy 0, policy_version 1851061 (0.0007) [2023-12-27 04:52:55,564][105620] Updated weights for policy 1, policy_version 1855519 (0.0009) [2023-12-27 04:52:55,588][105692] Updated weights for policy 0, policy_version 1851071 (0.0009) [2023-12-27 04:52:55,626][105620] Updated weights for policy 1, policy_version 1855529 (0.0005) [2023-12-27 04:52:55,637][105692] Updated weights for policy 0, policy_version 1851081 (0.0009) [2023-12-27 04:52:55,680][105620] Updated weights for policy 1, policy_version 1855539 (0.0006) [2023-12-27 04:52:56,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 949035008. Throughput: 0: 9760.3, 1: 9824.6. Samples: 949043504. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:52:56,062][104569] Avg episode reward: [(0, '8807.340'), (1, '8978.506')] [2023-12-27 04:52:56,361][105620] Updated weights for policy 1, policy_version 1855549 (0.0008) [2023-12-27 04:52:56,419][105692] Updated weights for policy 0, policy_version 1851091 (0.0008) [2023-12-27 04:52:56,422][105620] Updated weights for policy 1, policy_version 1855559 (0.0008) [2023-12-27 04:52:56,475][105692] Updated weights for policy 0, policy_version 1851101 (0.0006) [2023-12-27 04:52:56,480][105620] Updated weights for policy 1, policy_version 1855569 (0.0008) [2023-12-27 04:52:56,536][105692] Updated weights for policy 0, policy_version 1851111 (0.0006) [2023-12-27 04:52:57,142][105692] Updated weights for policy 0, policy_version 1851121 (0.0006) [2023-12-27 04:52:57,190][105692] Updated weights for policy 0, policy_version 1851131 (0.0008) [2023-12-27 04:52:57,241][105692] Updated weights for policy 0, policy_version 1851141 (0.0009) [2023-12-27 04:52:57,281][105620] Updated weights for policy 1, policy_version 1855579 (0.0008) [2023-12-27 04:52:57,307][105692] Updated weights for policy 0, policy_version 1851151 (0.0008) [2023-12-27 04:52:57,340][105620] Updated weights for policy 1, policy_version 1855589 (0.0008) [2023-12-27 04:52:57,394][105620] Updated weights for policy 1, policy_version 1855599 (0.0008) [2023-12-27 04:52:58,075][105692] Updated weights for policy 0, policy_version 1851161 (0.0005) [2023-12-27 04:52:58,129][105620] Updated weights for policy 1, policy_version 1855609 (0.0009) [2023-12-27 04:52:58,132][105692] Updated weights for policy 0, policy_version 1851171 (0.0005) [2023-12-27 04:52:58,198][105620] Updated weights for policy 1, policy_version 1855619 (0.0009) [2023-12-27 04:52:58,199][105692] Updated weights for policy 0, policy_version 1851181 (0.0009) [2023-12-27 04:52:58,260][105620] Updated weights for policy 1, policy_version 1855629 (0.0009) [2023-12-27 04:52:58,321][105620] Updated weights for policy 1, policy_version 1855639 (0.0009) [2023-12-27 04:52:58,952][105692] Updated weights for policy 0, policy_version 1851191 (0.0010) [2023-12-27 04:52:59,021][105692] Updated weights for policy 0, policy_version 1851201 (0.0008) [2023-12-27 04:52:59,044][105620] Updated weights for policy 1, policy_version 1855649 (0.0008) [2023-12-27 04:52:59,085][105692] Updated weights for policy 0, policy_version 1851211 (0.0007) [2023-12-27 04:52:59,105][105620] Updated weights for policy 1, policy_version 1855659 (0.0008) [2023-12-27 04:52:59,164][105620] Updated weights for policy 1, policy_version 1855669 (0.0008) [2023-12-27 04:52:59,900][105692] Updated weights for policy 0, policy_version 1851221 (0.0008) [2023-12-27 04:52:59,962][105692] Updated weights for policy 0, policy_version 1851231 (0.0010) [2023-12-27 04:52:59,980][105620] Updated weights for policy 1, policy_version 1855679 (0.0007) [2023-12-27 04:53:00,024][105692] Updated weights for policy 0, policy_version 1851241 (0.0011) [2023-12-27 04:53:00,034][105620] Updated weights for policy 1, policy_version 1855689 (0.0006) [2023-12-27 04:53:00,094][105620] Updated weights for policy 1, policy_version 1855699 (0.0007) [2023-12-27 04:53:00,746][105692] Updated weights for policy 0, policy_version 1851251 (0.0010) [2023-12-27 04:53:00,797][105692] Updated weights for policy 0, policy_version 1851261 (0.0006) [2023-12-27 04:53:00,857][105692] Updated weights for policy 0, policy_version 1851271 (0.0005) [2023-12-27 04:53:00,868][105620] Updated weights for policy 1, policy_version 1855709 (0.0008) [2023-12-27 04:53:00,925][105620] Updated weights for policy 1, policy_version 1855719 (0.0009) [2023-12-27 04:53:00,986][105620] Updated weights for policy 1, policy_version 1855729 (0.0010) [2023-12-27 04:53:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 949133312. Throughput: 0: 9783.9, 1: 9815.5. Samples: 949100164. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:53:01,062][104569] Avg episode reward: [(0, '8715.074'), (1, '9162.913')] [2023-12-27 04:53:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001851280_473997312.pth... [2023-12-27 04:53:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001855736_475136000.pth... [2023-12-27 04:53:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001850160_473710592.pth [2023-12-27 04:53:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001854584_474841088.pth [2023-12-27 04:53:01,535][105692] Updated weights for policy 0, policy_version 1851281 (0.0006) [2023-12-27 04:53:01,591][105692] Updated weights for policy 0, policy_version 1851291 (0.0009) [2023-12-27 04:53:01,650][105692] Updated weights for policy 0, policy_version 1851301 (0.0008) [2023-12-27 04:53:01,709][105692] Updated weights for policy 0, policy_version 1851311 (0.0009) [2023-12-27 04:53:01,780][105620] Updated weights for policy 1, policy_version 1855739 (0.0010) [2023-12-27 04:53:01,839][105620] Updated weights for policy 1, policy_version 1855749 (0.0010) [2023-12-27 04:53:01,887][105620] Updated weights for policy 1, policy_version 1855759 (0.0008) [2023-12-27 04:53:02,395][105692] Updated weights for policy 0, policy_version 1851321 (0.0008) [2023-12-27 04:53:02,450][105692] Updated weights for policy 0, policy_version 1851331 (0.0005) [2023-12-27 04:53:02,508][105692] Updated weights for policy 0, policy_version 1851341 (0.0005) [2023-12-27 04:53:02,743][105620] Updated weights for policy 1, policy_version 1855769 (0.0009) [2023-12-27 04:53:02,804][105620] Updated weights for policy 1, policy_version 1855779 (0.0009) [2023-12-27 04:53:02,861][105620] Updated weights for policy 1, policy_version 1855789 (0.0009) [2023-12-27 04:53:02,920][105620] Updated weights for policy 1, policy_version 1855799 (0.0009) [2023-12-27 04:53:03,163][105692] Updated weights for policy 0, policy_version 1851351 (0.0008) [2023-12-27 04:53:03,219][105692] Updated weights for policy 0, policy_version 1851361 (0.0005) [2023-12-27 04:53:03,273][105692] Updated weights for policy 0, policy_version 1851371 (0.0005) [2023-12-27 04:53:03,765][105620] Updated weights for policy 1, policy_version 1855809 (0.0010) [2023-12-27 04:53:03,821][105620] Updated weights for policy 1, policy_version 1855819 (0.0009) [2023-12-27 04:53:03,844][105692] Updated weights for policy 0, policy_version 1851381 (0.0006) [2023-12-27 04:53:03,879][105620] Updated weights for policy 1, policy_version 1855829 (0.0006) [2023-12-27 04:53:03,901][105692] Updated weights for policy 0, policy_version 1851391 (0.0007) [2023-12-27 04:53:03,962][105692] Updated weights for policy 0, policy_version 1851401 (0.0008) [2023-12-27 04:53:04,612][105620] Updated weights for policy 1, policy_version 1855839 (0.0006) [2023-12-27 04:53:04,666][105620] Updated weights for policy 1, policy_version 1855849 (0.0006) [2023-12-27 04:53:04,677][105692] Updated weights for policy 0, policy_version 1851411 (0.0008) [2023-12-27 04:53:04,725][105620] Updated weights for policy 1, policy_version 1855859 (0.0009) [2023-12-27 04:53:04,730][105692] Updated weights for policy 0, policy_version 1851421 (0.0006) [2023-12-27 04:53:04,789][105692] Updated weights for policy 0, policy_version 1851431 (0.0006) [2023-12-27 04:53:05,374][105692] Updated weights for policy 0, policy_version 1851441 (0.0006) [2023-12-27 04:53:05,435][105692] Updated weights for policy 0, policy_version 1851451 (0.0009) [2023-12-27 04:53:05,449][105620] Updated weights for policy 1, policy_version 1855869 (0.0009) [2023-12-27 04:53:05,490][105692] Updated weights for policy 0, policy_version 1851461 (0.0006) [2023-12-27 04:53:05,508][105620] Updated weights for policy 1, policy_version 1855879 (0.0009) [2023-12-27 04:53:05,547][105692] Updated weights for policy 0, policy_version 1851471 (0.0006) [2023-12-27 04:53:05,568][105620] Updated weights for policy 1, policy_version 1855889 (0.0009) [2023-12-27 04:53:06,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 949223424. Throughput: 0: 9717.2, 1: 9799.1. Samples: 949213340. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:53:06,062][104569] Avg episode reward: [(0, '8531.762'), (1, '9255.307')] [2023-12-27 04:53:06,239][105692] Updated weights for policy 0, policy_version 1851481 (0.0009) [2023-12-27 04:53:06,287][105692] Updated weights for policy 0, policy_version 1851491 (0.0008) [2023-12-27 04:53:06,339][105692] Updated weights for policy 0, policy_version 1851501 (0.0009) [2023-12-27 04:53:06,347][105620] Updated weights for policy 1, policy_version 1855899 (0.0009) [2023-12-27 04:53:06,409][105620] Updated weights for policy 1, policy_version 1855909 (0.0008) [2023-12-27 04:53:06,475][105620] Updated weights for policy 1, policy_version 1855919 (0.0009) [2023-12-27 04:53:07,060][105692] Updated weights for policy 0, policy_version 1851511 (0.0009) [2023-12-27 04:53:07,126][105692] Updated weights for policy 0, policy_version 1851521 (0.0008) [2023-12-27 04:53:07,192][105692] Updated weights for policy 0, policy_version 1851531 (0.0007) [2023-12-27 04:53:07,259][105620] Updated weights for policy 1, policy_version 1855929 (0.0009) [2023-12-27 04:53:07,323][105620] Updated weights for policy 1, policy_version 1855939 (0.0006) [2023-12-27 04:53:07,380][105620] Updated weights for policy 1, policy_version 1855949 (0.0007) [2023-12-27 04:53:07,434][105620] Updated weights for policy 1, policy_version 1855959 (0.0005) [2023-12-27 04:53:07,952][105620] Updated weights for policy 1, policy_version 1855969 (0.0005) [2023-12-27 04:53:08,014][105620] Updated weights for policy 1, policy_version 1855979 (0.0005) [2023-12-27 04:53:08,068][105620] Updated weights for policy 1, policy_version 1855989 (0.0005) [2023-12-27 04:53:08,081][105692] Updated weights for policy 0, policy_version 1851541 (0.0008) [2023-12-27 04:53:08,138][105692] Updated weights for policy 0, policy_version 1851551 (0.0010) [2023-12-27 04:53:08,190][105692] Updated weights for policy 0, policy_version 1851561 (0.0008) [2023-12-27 04:53:08,728][105620] Updated weights for policy 1, policy_version 1855999 (0.0005) [2023-12-27 04:53:08,782][105620] Updated weights for policy 1, policy_version 1856009 (0.0005) [2023-12-27 04:53:08,834][105620] Updated weights for policy 1, policy_version 1856019 (0.0005) [2023-12-27 04:53:09,029][105692] Updated weights for policy 0, policy_version 1851571 (0.0009) [2023-12-27 04:53:09,083][105692] Updated weights for policy 0, policy_version 1851581 (0.0008) [2023-12-27 04:53:09,138][105692] Updated weights for policy 0, policy_version 1851591 (0.0008) [2023-12-27 04:53:09,470][105620] Updated weights for policy 1, policy_version 1856029 (0.0007) [2023-12-27 04:53:09,531][105620] Updated weights for policy 1, policy_version 1856039 (0.0009) [2023-12-27 04:53:09,588][105620] Updated weights for policy 1, policy_version 1856049 (0.0008) [2023-12-27 04:53:09,921][105692] Updated weights for policy 0, policy_version 1851601 (0.0009) [2023-12-27 04:53:09,985][105692] Updated weights for policy 0, policy_version 1851611 (0.0008) [2023-12-27 04:53:10,045][105692] Updated weights for policy 0, policy_version 1851621 (0.0008) [2023-12-27 04:53:10,100][105692] Updated weights for policy 0, policy_version 1851631 (0.0008) [2023-12-27 04:53:10,355][105620] Updated weights for policy 1, policy_version 1856059 (0.0009) [2023-12-27 04:53:10,402][105620] Updated weights for policy 1, policy_version 1856069 (0.0009) [2023-12-27 04:53:10,448][105620] Updated weights for policy 1, policy_version 1856079 (0.0009) [2023-12-27 04:53:10,866][105692] Updated weights for policy 0, policy_version 1851641 (0.0009) [2023-12-27 04:53:10,924][105692] Updated weights for policy 0, policy_version 1851651 (0.0009) [2023-12-27 04:53:10,984][105692] Updated weights for policy 0, policy_version 1851661 (0.0007) [2023-12-27 04:53:11,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 949321728. Throughput: 0: 9667.6, 1: 9739.0. Samples: 949328416. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:53:11,063][104569] Avg episode reward: [(0, '8262.185'), (1, '9347.492')] [2023-12-27 04:53:11,260][105620] Updated weights for policy 1, policy_version 1856089 (0.0008) [2023-12-27 04:53:11,336][105620] Updated weights for policy 1, policy_version 1856099 (0.0010) [2023-12-27 04:53:11,405][105620] Updated weights for policy 1, policy_version 1856109 (0.0006) [2023-12-27 04:53:11,477][105620] Updated weights for policy 1, policy_version 1856119 (0.0009) [2023-12-27 04:53:11,804][105692] Updated weights for policy 0, policy_version 1851671 (0.0008) [2023-12-27 04:53:11,862][105692] Updated weights for policy 0, policy_version 1851681 (0.0009) [2023-12-27 04:53:11,914][105692] Updated weights for policy 0, policy_version 1851691 (0.0009) [2023-12-27 04:53:12,176][105620] Updated weights for policy 1, policy_version 1856129 (0.0008) [2023-12-27 04:53:12,233][105620] Updated weights for policy 1, policy_version 1856139 (0.0010) [2023-12-27 04:53:12,296][105620] Updated weights for policy 1, policy_version 1856149 (0.0010) [2023-12-27 04:53:12,584][105692] Updated weights for policy 0, policy_version 1851701 (0.0009) [2023-12-27 04:53:12,647][105692] Updated weights for policy 0, policy_version 1851711 (0.0009) [2023-12-27 04:53:12,700][105692] Updated weights for policy 0, policy_version 1851721 (0.0009) [2023-12-27 04:53:13,020][105620] Updated weights for policy 1, policy_version 1856159 (0.0009) [2023-12-27 04:53:13,076][105620] Updated weights for policy 1, policy_version 1856169 (0.0009) [2023-12-27 04:53:13,139][105620] Updated weights for policy 1, policy_version 1856179 (0.0010) [2023-12-27 04:53:13,385][105692] Updated weights for policy 0, policy_version 1851731 (0.0009) [2023-12-27 04:53:13,438][105692] Updated weights for policy 0, policy_version 1851741 (0.0010) [2023-12-27 04:53:13,500][105692] Updated weights for policy 0, policy_version 1851751 (0.0010) [2023-12-27 04:53:13,864][105620] Updated weights for policy 1, policy_version 1856189 (0.0008) [2023-12-27 04:53:13,912][105620] Updated weights for policy 1, policy_version 1856199 (0.0006) [2023-12-27 04:53:13,959][105620] Updated weights for policy 1, policy_version 1856210 (0.0009) [2023-12-27 04:53:14,303][105692] Updated weights for policy 0, policy_version 1851761 (0.0009) [2023-12-27 04:53:14,369][105692] Updated weights for policy 0, policy_version 1851771 (0.0005) [2023-12-27 04:53:14,437][105692] Updated weights for policy 0, policy_version 1851781 (0.0009) [2023-12-27 04:53:14,498][105692] Updated weights for policy 0, policy_version 1851791 (0.0007) [2023-12-27 04:53:14,762][105620] Updated weights for policy 1, policy_version 1856220 (0.0008) [2023-12-27 04:53:14,823][105620] Updated weights for policy 1, policy_version 1856230 (0.0010) [2023-12-27 04:53:14,890][105620] Updated weights for policy 1, policy_version 1856240 (0.0011) [2023-12-27 04:53:15,130][105692] Updated weights for policy 0, policy_version 1851801 (0.0010) [2023-12-27 04:53:15,200][105692] Updated weights for policy 0, policy_version 1851811 (0.0011) [2023-12-27 04:53:15,256][105692] Updated weights for policy 0, policy_version 1851821 (0.0010) [2023-12-27 04:53:15,619][105620] Updated weights for policy 1, policy_version 1856250 (0.0011) [2023-12-27 04:53:15,664][105620] Updated weights for policy 1, policy_version 1856260 (0.0010) [2023-12-27 04:53:15,710][105620] Updated weights for policy 1, policy_version 1856270 (0.0010) [2023-12-27 04:53:15,759][105620] Updated weights for policy 1, policy_version 1856280 (0.0010) [2023-12-27 04:53:15,872][105692] Updated weights for policy 0, policy_version 1851831 (0.0007) [2023-12-27 04:53:15,927][105692] Updated weights for policy 0, policy_version 1851841 (0.0007) [2023-12-27 04:53:15,983][105692] Updated weights for policy 0, policy_version 1851851 (0.0010) [2023-12-27 04:53:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 949420032. Throughput: 0: 9572.5, 1: 9750.0. Samples: 949385168. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:53:16,063][104569] Avg episode reward: [(0, '8078.854'), (1, '9347.456')] [2023-12-27 04:53:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001851856_474144768.pth... [2023-12-27 04:53:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001856280_475275264.pth... [2023-12-27 04:53:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001850736_473858048.pth [2023-12-27 04:53:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001855160_474988544.pth [2023-12-27 04:53:16,573][105620] Updated weights for policy 1, policy_version 1856290 (0.0008) [2023-12-27 04:53:16,587][105692] Updated weights for policy 0, policy_version 1851861 (0.0008) [2023-12-27 04:53:16,625][105620] Updated weights for policy 1, policy_version 1856300 (0.0010) [2023-12-27 04:53:16,634][105692] Updated weights for policy 0, policy_version 1851871 (0.0009) [2023-12-27 04:53:16,689][105692] Updated weights for policy 0, policy_version 1851881 (0.0010) [2023-12-27 04:53:16,690][105620] Updated weights for policy 1, policy_version 1856310 (0.0011) [2023-12-27 04:53:17,290][105692] Updated weights for policy 0, policy_version 1851891 (0.0009) [2023-12-27 04:53:17,350][105692] Updated weights for policy 0, policy_version 1851901 (0.0005) [2023-12-27 04:53:17,405][105620] Updated weights for policy 1, policy_version 1856320 (0.0011) [2023-12-27 04:53:17,435][105692] Updated weights for policy 0, policy_version 1851911 (0.0006) [2023-12-27 04:53:17,457][105620] Updated weights for policy 1, policy_version 1856330 (0.0010) [2023-12-27 04:53:17,514][105620] Updated weights for policy 1, policy_version 1856340 (0.0009) [2023-12-27 04:53:18,115][105692] Updated weights for policy 0, policy_version 1851921 (0.0006) [2023-12-27 04:53:18,163][105692] Updated weights for policy 0, policy_version 1851931 (0.0008) [2023-12-27 04:53:18,211][105692] Updated weights for policy 0, policy_version 1851941 (0.0008) [2023-12-27 04:53:18,259][105692] Updated weights for policy 0, policy_version 1851951 (0.0008) [2023-12-27 04:53:18,284][105620] Updated weights for policy 1, policy_version 1856350 (0.0010) [2023-12-27 04:53:18,348][105620] Updated weights for policy 1, policy_version 1856360 (0.0011) [2023-12-27 04:53:18,404][105620] Updated weights for policy 1, policy_version 1856370 (0.0010) [2023-12-27 04:53:19,082][105692] Updated weights for policy 0, policy_version 1851961 (0.0007) [2023-12-27 04:53:19,134][105692] Updated weights for policy 0, policy_version 1851971 (0.0008) [2023-12-27 04:53:19,147][105620] Updated weights for policy 1, policy_version 1856380 (0.0011) [2023-12-27 04:53:19,182][105692] Updated weights for policy 0, policy_version 1851981 (0.0007) [2023-12-27 04:53:19,209][105620] Updated weights for policy 1, policy_version 1856390 (0.0010) [2023-12-27 04:53:19,276][105620] Updated weights for policy 1, policy_version 1856400 (0.0009) [2023-12-27 04:53:19,968][105692] Updated weights for policy 0, policy_version 1851991 (0.0009) [2023-12-27 04:53:20,022][105692] Updated weights for policy 0, policy_version 1852001 (0.0008) [2023-12-27 04:53:20,049][105620] Updated weights for policy 1, policy_version 1856410 (0.0010) [2023-12-27 04:53:20,076][105692] Updated weights for policy 0, policy_version 1852011 (0.0008) [2023-12-27 04:53:20,110][105620] Updated weights for policy 1, policy_version 1856420 (0.0011) [2023-12-27 04:53:20,170][105620] Updated weights for policy 1, policy_version 1856430 (0.0011) [2023-12-27 04:53:20,233][105620] Updated weights for policy 1, policy_version 1856440 (0.0010) [2023-12-27 04:53:20,873][105692] Updated weights for policy 0, policy_version 1852021 (0.0007) [2023-12-27 04:53:20,934][105692] Updated weights for policy 0, policy_version 1852031 (0.0008) [2023-12-27 04:53:20,990][105620] Updated weights for policy 1, policy_version 1856450 (0.0009) [2023-12-27 04:53:21,001][105692] Updated weights for policy 0, policy_version 1852041 (0.0008) [2023-12-27 04:53:21,049][105620] Updated weights for policy 1, policy_version 1856460 (0.0009) [2023-12-27 04:53:21,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 949510144. Throughput: 0: 9551.8, 1: 9625.9. Samples: 949501352. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:53:21,063][104569] Avg episode reward: [(0, '8626.601'), (1, '9347.425')] [2023-12-27 04:53:21,114][105620] Updated weights for policy 1, policy_version 1856470 (0.0009) [2023-12-27 04:53:21,756][105692] Updated weights for policy 0, policy_version 1852051 (0.0010) [2023-12-27 04:53:21,815][105692] Updated weights for policy 0, policy_version 1852061 (0.0009) [2023-12-27 04:53:21,876][105692] Updated weights for policy 0, policy_version 1852071 (0.0008) [2023-12-27 04:53:21,882][105620] Updated weights for policy 1, policy_version 1856480 (0.0007) [2023-12-27 04:53:21,944][105620] Updated weights for policy 1, policy_version 1856490 (0.0007) [2023-12-27 04:53:22,008][105620] Updated weights for policy 1, policy_version 1856500 (0.0005) [2023-12-27 04:53:22,637][105692] Updated weights for policy 0, policy_version 1852081 (0.0008) [2023-12-27 04:53:22,696][105692] Updated weights for policy 0, policy_version 1852091 (0.0011) [2023-12-27 04:53:22,750][105620] Updated weights for policy 1, policy_version 1856510 (0.0008) [2023-12-27 04:53:22,758][105692] Updated weights for policy 0, policy_version 1852101 (0.0009) [2023-12-27 04:53:22,806][105620] Updated weights for policy 1, policy_version 1856520 (0.0008) [2023-12-27 04:53:22,813][105692] Updated weights for policy 0, policy_version 1852111 (0.0010) [2023-12-27 04:53:22,865][105620] Updated weights for policy 1, policy_version 1856530 (0.0005) [2023-12-27 04:53:23,507][105692] Updated weights for policy 0, policy_version 1852121 (0.0009) [2023-12-27 04:53:23,574][105692] Updated weights for policy 0, policy_version 1852131 (0.0008) [2023-12-27 04:53:23,615][105620] Updated weights for policy 1, policy_version 1856540 (0.0007) [2023-12-27 04:53:23,638][105692] Updated weights for policy 0, policy_version 1852141 (0.0007) [2023-12-27 04:53:23,675][105620] Updated weights for policy 1, policy_version 1856550 (0.0007) [2023-12-27 04:53:23,727][105620] Updated weights for policy 1, policy_version 1856560 (0.0009) [2023-12-27 04:53:24,355][105620] Updated weights for policy 1, policy_version 1856570 (0.0009) [2023-12-27 04:53:24,403][105620] Updated weights for policy 1, policy_version 1856580 (0.0008) [2023-12-27 04:53:24,409][105692] Updated weights for policy 0, policy_version 1852151 (0.0008) [2023-12-27 04:53:24,458][105692] Updated weights for policy 0, policy_version 1852161 (0.0006) [2023-12-27 04:53:24,463][105620] Updated weights for policy 1, policy_version 1856590 (0.0008) [2023-12-27 04:53:24,518][105620] Updated weights for policy 1, policy_version 1856600 (0.0007) [2023-12-27 04:53:24,522][105692] Updated weights for policy 0, policy_version 1852171 (0.0010) [2023-12-27 04:53:25,117][105620] Updated weights for policy 1, policy_version 1856610 (0.0009) [2023-12-27 04:53:25,163][105620] Updated weights for policy 1, policy_version 1856620 (0.0008) [2023-12-27 04:53:25,210][105620] Updated weights for policy 1, policy_version 1856630 (0.0009) [2023-12-27 04:53:25,330][105692] Updated weights for policy 0, policy_version 1852181 (0.0010) [2023-12-27 04:53:25,391][105692] Updated weights for policy 0, policy_version 1852191 (0.0009) [2023-12-27 04:53:25,449][105692] Updated weights for policy 0, policy_version 1852201 (0.0009) [2023-12-27 04:53:25,946][105620] Updated weights for policy 1, policy_version 1856640 (0.0009) [2023-12-27 04:53:25,989][105620] Updated weights for policy 1, policy_version 1856650 (0.0006) [2023-12-27 04:53:26,046][105620] Updated weights for policy 1, policy_version 1856660 (0.0006) [2023-12-27 04:53:26,062][104569] Fps is (10 sec: 18022.7, 60 sec: 19114.7, 300 sec: 19383.1). Total num frames: 949600256. Throughput: 0: 9461.3, 1: 9602.2. Samples: 949614056. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 04:53:26,063][104569] Avg episode reward: [(0, '8898.487'), (1, '9254.905')] [2023-12-27 04:53:26,239][105692] Updated weights for policy 0, policy_version 1852211 (0.0008) [2023-12-27 04:53:26,293][105692] Updated weights for policy 0, policy_version 1852221 (0.0009) [2023-12-27 04:53:26,349][105692] Updated weights for policy 0, policy_version 1852231 (0.0008) [2023-12-27 04:53:26,779][105620] Updated weights for policy 1, policy_version 1856670 (0.0009) [2023-12-27 04:53:26,824][105620] Updated weights for policy 1, policy_version 1856680 (0.0008) [2023-12-27 04:53:26,870][105620] Updated weights for policy 1, policy_version 1856690 (0.0009) [2023-12-27 04:53:27,064][105692] Updated weights for policy 0, policy_version 1852241 (0.0009) [2023-12-27 04:53:27,121][105692] Updated weights for policy 0, policy_version 1852252 (0.0011) [2023-12-27 04:53:27,175][105692] Updated weights for policy 0, policy_version 1852263 (0.0010) [2023-12-27 04:53:27,516][105620] Updated weights for policy 1, policy_version 1856700 (0.0009) [2023-12-27 04:53:27,576][105620] Updated weights for policy 1, policy_version 1856710 (0.0009) [2023-12-27 04:53:27,640][105620] Updated weights for policy 1, policy_version 1856720 (0.0009) [2023-12-27 04:53:28,008][105692] Updated weights for policy 0, policy_version 1852274 (0.0010) [2023-12-27 04:53:28,069][105692] Updated weights for policy 0, policy_version 1852284 (0.0009) [2023-12-27 04:53:28,130][105692] Updated weights for policy 0, policy_version 1852294 (0.0009) [2023-12-27 04:53:28,194][105692] Updated weights for policy 0, policy_version 1852304 (0.0009) [2023-12-27 04:53:28,339][105620] Updated weights for policy 1, policy_version 1856730 (0.0009) [2023-12-27 04:53:28,400][105620] Updated weights for policy 1, policy_version 1856740 (0.0007) [2023-12-27 04:53:28,464][105620] Updated weights for policy 1, policy_version 1856750 (0.0008) [2023-12-27 04:53:28,525][105620] Updated weights for policy 1, policy_version 1856760 (0.0008) [2023-12-27 04:53:28,980][105692] Updated weights for policy 0, policy_version 1852314 (0.0007) [2023-12-27 04:53:29,032][105692] Updated weights for policy 0, policy_version 1852324 (0.0009) [2023-12-27 04:53:29,083][105692] Updated weights for policy 0, policy_version 1852334 (0.0009) [2023-12-27 04:53:29,229][105620] Updated weights for policy 1, policy_version 1856770 (0.0009) [2023-12-27 04:53:29,291][105620] Updated weights for policy 1, policy_version 1856780 (0.0008) [2023-12-27 04:53:29,349][105620] Updated weights for policy 1, policy_version 1856790 (0.0009) [2023-12-27 04:53:29,783][105692] Updated weights for policy 0, policy_version 1852344 (0.0009) [2023-12-27 04:53:29,847][105692] Updated weights for policy 0, policy_version 1852354 (0.0007) [2023-12-27 04:53:29,914][105692] Updated weights for policy 0, policy_version 1852364 (0.0006) [2023-12-27 04:53:29,946][105620] Updated weights for policy 1, policy_version 1856800 (0.0008) [2023-12-27 04:53:30,003][105620] Updated weights for policy 1, policy_version 1856810 (0.0005) [2023-12-27 04:53:30,050][105620] Updated weights for policy 1, policy_version 1856820 (0.0005) [2023-12-27 04:53:30,624][105692] Updated weights for policy 0, policy_version 1852374 (0.0010) [2023-12-27 04:53:30,685][105692] Updated weights for policy 0, policy_version 1852384 (0.0010) [2023-12-27 04:53:30,704][105620] Updated weights for policy 1, policy_version 1856830 (0.0006) [2023-12-27 04:53:30,746][105692] Updated weights for policy 0, policy_version 1852394 (0.0010) [2023-12-27 04:53:30,762][105620] Updated weights for policy 1, policy_version 1856840 (0.0007) [2023-12-27 04:53:30,825][105620] Updated weights for policy 1, policy_version 1856850 (0.0009) [2023-12-27 04:53:31,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19387.6, 300 sec: 19410.9). Total num frames: 949706752. Throughput: 0: 9472.5, 1: 9593.0. Samples: 949671544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:53:31,063][104569] Avg episode reward: [(0, '8174.391'), (1, '9254.842')] [2023-12-27 04:53:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001856856_475422720.pth... [2023-12-27 04:53:31,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001852400_474284032.pth... [2023-12-27 04:53:31,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001855736_475136000.pth [2023-12-27 04:53:31,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001851280_473997312.pth [2023-12-27 04:53:31,423][105692] Updated weights for policy 0, policy_version 1852404 (0.0007) [2023-12-27 04:53:31,482][105692] Updated weights for policy 0, policy_version 1852414 (0.0009) [2023-12-27 04:53:31,537][105692] Updated weights for policy 0, policy_version 1852424 (0.0009) [2023-12-27 04:53:31,639][105620] Updated weights for policy 1, policy_version 1856860 (0.0008) [2023-12-27 04:53:31,702][105620] Updated weights for policy 1, policy_version 1856870 (0.0008) [2023-12-27 04:53:31,770][105620] Updated weights for policy 1, policy_version 1856880 (0.0008) [2023-12-27 04:53:32,273][105692] Updated weights for policy 0, policy_version 1852434 (0.0010) [2023-12-27 04:53:32,337][105692] Updated weights for policy 0, policy_version 1852444 (0.0009) [2023-12-27 04:53:32,400][105692] Updated weights for policy 0, policy_version 1852454 (0.0008) [2023-12-27 04:53:32,423][105620] Updated weights for policy 1, policy_version 1856890 (0.0007) [2023-12-27 04:53:32,460][105692] Updated weights for policy 0, policy_version 1852464 (0.0008) [2023-12-27 04:53:32,484][105620] Updated weights for policy 1, policy_version 1856900 (0.0010) [2023-12-27 04:53:32,542][105620] Updated weights for policy 1, policy_version 1856910 (0.0010) [2023-12-27 04:53:32,600][105620] Updated weights for policy 1, policy_version 1856920 (0.0010) [2023-12-27 04:53:33,256][105692] Updated weights for policy 0, policy_version 1852474 (0.0007) [2023-12-27 04:53:33,296][105620] Updated weights for policy 1, policy_version 1856930 (0.0010) [2023-12-27 04:53:33,304][105692] Updated weights for policy 0, policy_version 1852484 (0.0005) [2023-12-27 04:53:33,353][105692] Updated weights for policy 0, policy_version 1852494 (0.0005) [2023-12-27 04:53:33,354][105620] Updated weights for policy 1, policy_version 1856940 (0.0010) [2023-12-27 04:53:33,415][105620] Updated weights for policy 1, policy_version 1856950 (0.0010) [2023-12-27 04:53:33,948][105692] Updated weights for policy 0, policy_version 1852504 (0.0005) [2023-12-27 04:53:33,999][105692] Updated weights for policy 0, policy_version 1852514 (0.0005) [2023-12-27 04:53:34,050][105692] Updated weights for policy 0, policy_version 1852524 (0.0005) [2023-12-27 04:53:34,169][105620] Updated weights for policy 1, policy_version 1856960 (0.0008) [2023-12-27 04:53:34,238][105620] Updated weights for policy 1, policy_version 1856970 (0.0008) [2023-12-27 04:53:34,308][105620] Updated weights for policy 1, policy_version 1856980 (0.0008) [2023-12-27 04:53:34,786][105692] Updated weights for policy 0, policy_version 1852534 (0.0009) [2023-12-27 04:53:34,842][105692] Updated weights for policy 0, policy_version 1852544 (0.0011) [2023-12-27 04:53:34,899][105692] Updated weights for policy 0, policy_version 1852554 (0.0011) [2023-12-27 04:53:35,028][105620] Updated weights for policy 1, policy_version 1856990 (0.0010) [2023-12-27 04:53:35,095][105620] Updated weights for policy 1, policy_version 1857000 (0.0009) [2023-12-27 04:53:35,171][105620] Updated weights for policy 1, policy_version 1857010 (0.0011) [2023-12-27 04:53:35,534][105692] Updated weights for policy 0, policy_version 1852564 (0.0007) [2023-12-27 04:53:35,585][105692] Updated weights for policy 0, policy_version 1852574 (0.0005) [2023-12-27 04:53:35,639][105692] Updated weights for policy 0, policy_version 1852584 (0.0005) [2023-12-27 04:53:35,781][105620] Updated weights for policy 1, policy_version 1857020 (0.0009) [2023-12-27 04:53:35,839][105620] Updated weights for policy 1, policy_version 1857030 (0.0006) [2023-12-27 04:53:35,905][105620] Updated weights for policy 1, policy_version 1857040 (0.0008) [2023-12-27 04:53:36,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19251.1, 300 sec: 19438.6). Total num frames: 949805056. Throughput: 0: 9498.2, 1: 9585.5. Samples: 949789736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:53:36,063][104569] Avg episode reward: [(0, '8355.449'), (1, '9254.910')] [2023-12-27 04:53:36,316][105692] Updated weights for policy 0, policy_version 1852594 (0.0009) [2023-12-27 04:53:36,383][105692] Updated weights for policy 0, policy_version 1852604 (0.0010) [2023-12-27 04:53:36,447][105692] Updated weights for policy 0, policy_version 1852614 (0.0011) [2023-12-27 04:53:36,507][105692] Updated weights for policy 0, policy_version 1852624 (0.0011) [2023-12-27 04:53:36,517][105620] Updated weights for policy 1, policy_version 1857050 (0.0007) [2023-12-27 04:53:36,578][105620] Updated weights for policy 1, policy_version 1857060 (0.0011) [2023-12-27 04:53:36,642][105620] Updated weights for policy 1, policy_version 1857070 (0.0011) [2023-12-27 04:53:36,702][105620] Updated weights for policy 1, policy_version 1857080 (0.0011) [2023-12-27 04:53:37,232][105692] Updated weights for policy 0, policy_version 1852634 (0.0010) [2023-12-27 04:53:37,285][105692] Updated weights for policy 0, policy_version 1852644 (0.0010) [2023-12-27 04:53:37,342][105692] Updated weights for policy 0, policy_version 1852654 (0.0011) [2023-12-27 04:53:37,470][105620] Updated weights for policy 1, policy_version 1857090 (0.0011) [2023-12-27 04:53:37,519][105620] Updated weights for policy 1, policy_version 1857100 (0.0011) [2023-12-27 04:53:37,568][105620] Updated weights for policy 1, policy_version 1857110 (0.0009) [2023-12-27 04:53:38,016][105692] Updated weights for policy 0, policy_version 1852664 (0.0011) [2023-12-27 04:53:38,076][105692] Updated weights for policy 0, policy_version 1852674 (0.0011) [2023-12-27 04:53:38,135][105692] Updated weights for policy 0, policy_version 1852684 (0.0011) [2023-12-27 04:53:38,378][105620] Updated weights for policy 1, policy_version 1857120 (0.0009) [2023-12-27 04:53:38,439][105620] Updated weights for policy 1, policy_version 1857130 (0.0007) [2023-12-27 04:53:38,497][105620] Updated weights for policy 1, policy_version 1857140 (0.0006) [2023-12-27 04:53:38,896][105692] Updated weights for policy 0, policy_version 1852694 (0.0010) [2023-12-27 04:53:38,948][105692] Updated weights for policy 0, policy_version 1852704 (0.0010) [2023-12-27 04:53:38,993][105692] Updated weights for policy 0, policy_version 1852714 (0.0010) [2023-12-27 04:53:39,213][105620] Updated weights for policy 1, policy_version 1857150 (0.0008) [2023-12-27 04:53:39,278][105620] Updated weights for policy 1, policy_version 1857160 (0.0009) [2023-12-27 04:53:39,347][105620] Updated weights for policy 1, policy_version 1857170 (0.0011) [2023-12-27 04:53:39,795][105692] Updated weights for policy 0, policy_version 1852724 (0.0009) [2023-12-27 04:53:39,865][105692] Updated weights for policy 0, policy_version 1852734 (0.0009) [2023-12-27 04:53:39,931][105692] Updated weights for policy 0, policy_version 1852744 (0.0009) [2023-12-27 04:53:40,086][105620] Updated weights for policy 1, policy_version 1857180 (0.0008) [2023-12-27 04:53:40,145][105620] Updated weights for policy 1, policy_version 1857190 (0.0006) [2023-12-27 04:53:40,207][105620] Updated weights for policy 1, policy_version 1857200 (0.0006) [2023-12-27 04:53:40,799][105692] Updated weights for policy 0, policy_version 1852754 (0.0009) [2023-12-27 04:53:40,807][105620] Updated weights for policy 1, policy_version 1857210 (0.0008) [2023-12-27 04:53:40,853][105692] Updated weights for policy 0, policy_version 1852764 (0.0006) [2023-12-27 04:53:40,861][105620] Updated weights for policy 1, policy_version 1857220 (0.0007) [2023-12-27 04:53:40,905][105692] Updated weights for policy 0, policy_version 1852774 (0.0005) [2023-12-27 04:53:40,920][105620] Updated weights for policy 1, policy_version 1857230 (0.0009) [2023-12-27 04:53:40,962][105692] Updated weights for policy 0, policy_version 1852784 (0.0007) [2023-12-27 04:53:40,982][105620] Updated weights for policy 1, policy_version 1857240 (0.0010) [2023-12-27 04:53:41,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19251.2, 300 sec: 19438.7). Total num frames: 949903360. Throughput: 0: 9600.0, 1: 9573.6. Samples: 949906316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:53:41,063][104569] Avg episode reward: [(0, '8717.342'), (1, '9162.621')] [2023-12-27 04:53:41,737][105692] Updated weights for policy 0, policy_version 1852794 (0.0009) [2023-12-27 04:53:41,799][105620] Updated weights for policy 1, policy_version 1857250 (0.0008) [2023-12-27 04:53:41,803][105692] Updated weights for policy 0, policy_version 1852804 (0.0011) [2023-12-27 04:53:41,864][105692] Updated weights for policy 0, policy_version 1852814 (0.0007) [2023-12-27 04:53:41,866][105620] Updated weights for policy 1, policy_version 1857260 (0.0008) [2023-12-27 04:53:41,926][105620] Updated weights for policy 1, policy_version 1857270 (0.0008) [2023-12-27 04:53:42,580][105692] Updated weights for policy 0, policy_version 1852824 (0.0009) [2023-12-27 04:53:42,624][105620] Updated weights for policy 1, policy_version 1857280 (0.0007) [2023-12-27 04:53:42,642][105692] Updated weights for policy 0, policy_version 1852834 (0.0008) [2023-12-27 04:53:42,681][105620] Updated weights for policy 1, policy_version 1857290 (0.0007) [2023-12-27 04:53:42,700][105692] Updated weights for policy 0, policy_version 1852844 (0.0006) [2023-12-27 04:53:42,741][105620] Updated weights for policy 1, policy_version 1857300 (0.0008) [2023-12-27 04:53:43,433][105692] Updated weights for policy 0, policy_version 1852854 (0.0008) [2023-12-27 04:53:43,488][105692] Updated weights for policy 0, policy_version 1852864 (0.0009) [2023-12-27 04:53:43,526][105620] Updated weights for policy 1, policy_version 1857310 (0.0009) [2023-12-27 04:53:43,533][105692] Updated weights for policy 0, policy_version 1852874 (0.0006) [2023-12-27 04:53:43,589][105620] Updated weights for policy 1, policy_version 1857320 (0.0008) [2023-12-27 04:53:43,655][105620] Updated weights for policy 1, policy_version 1857330 (0.0009) [2023-12-27 04:53:44,235][105692] Updated weights for policy 0, policy_version 1852884 (0.0009) [2023-12-27 04:53:44,293][105692] Updated weights for policy 0, policy_version 1852894 (0.0009) [2023-12-27 04:53:44,356][105692] Updated weights for policy 0, policy_version 1852904 (0.0008) [2023-12-27 04:53:44,422][105620] Updated weights for policy 1, policy_version 1857340 (0.0008) [2023-12-27 04:53:44,481][105620] Updated weights for policy 1, policy_version 1857350 (0.0010) [2023-12-27 04:53:44,546][105620] Updated weights for policy 1, policy_version 1857360 (0.0010) [2023-12-27 04:53:45,133][105620] Updated weights for policy 1, policy_version 1857370 (0.0007) [2023-12-27 04:53:45,193][105620] Updated weights for policy 1, policy_version 1857380 (0.0009) [2023-12-27 04:53:45,196][105692] Updated weights for policy 0, policy_version 1852914 (0.0009) [2023-12-27 04:53:45,245][105620] Updated weights for policy 1, policy_version 1857390 (0.0006) [2023-12-27 04:53:45,250][105692] Updated weights for policy 0, policy_version 1852924 (0.0007) [2023-12-27 04:53:45,300][105620] Updated weights for policy 1, policy_version 1857400 (0.0007) [2023-12-27 04:53:45,313][105692] Updated weights for policy 0, policy_version 1852934 (0.0009) [2023-12-27 04:53:45,375][105692] Updated weights for policy 0, policy_version 1852944 (0.0010) [2023-12-27 04:53:45,939][105620] Updated weights for policy 1, policy_version 1857410 (0.0009) [2023-12-27 04:53:45,997][105620] Updated weights for policy 1, policy_version 1857420 (0.0009) [2023-12-27 04:53:46,047][105620] Updated weights for policy 1, policy_version 1857430 (0.0009) [2023-12-27 04:53:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 949993472. Throughput: 0: 9568.9, 1: 9555.7. Samples: 949960772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:53:46,063][104569] Avg episode reward: [(0, '8532.723'), (1, '9070.495')] [2023-12-27 04:53:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001852944_474423296.pth... [2023-12-27 04:53:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001857432_475570176.pth... [2023-12-27 04:53:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001851856_474144768.pth [2023-12-27 04:53:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001856280_475275264.pth [2023-12-27 04:53:46,185][105692] Updated weights for policy 0, policy_version 1852954 (0.0008) [2023-12-27 04:53:46,237][105692] Updated weights for policy 0, policy_version 1852964 (0.0009) [2023-12-27 04:53:46,292][105692] Updated weights for policy 0, policy_version 1852974 (0.0009) [2023-12-27 04:53:46,698][105620] Updated weights for policy 1, policy_version 1857440 (0.0009) [2023-12-27 04:53:46,755][105620] Updated weights for policy 1, policy_version 1857450 (0.0009) [2023-12-27 04:53:46,807][105620] Updated weights for policy 1, policy_version 1857460 (0.0009) [2023-12-27 04:53:47,093][105692] Updated weights for policy 0, policy_version 1852984 (0.0006) [2023-12-27 04:53:47,152][105692] Updated weights for policy 0, policy_version 1852994 (0.0009) [2023-12-27 04:53:47,200][105692] Updated weights for policy 0, policy_version 1853004 (0.0009) [2023-12-27 04:53:47,562][105620] Updated weights for policy 1, policy_version 1857470 (0.0009) [2023-12-27 04:53:47,620][105620] Updated weights for policy 1, policy_version 1857480 (0.0005) [2023-12-27 04:53:47,685][105620] Updated weights for policy 1, policy_version 1857490 (0.0006) [2023-12-27 04:53:47,899][105692] Updated weights for policy 0, policy_version 1853014 (0.0006) [2023-12-27 04:53:47,954][105692] Updated weights for policy 0, policy_version 1853024 (0.0008) [2023-12-27 04:53:48,012][105692] Updated weights for policy 0, policy_version 1853034 (0.0005) [2023-12-27 04:53:48,250][105620] Updated weights for policy 1, policy_version 1857500 (0.0008) [2023-12-27 04:53:48,308][105620] Updated weights for policy 1, policy_version 1857510 (0.0010) [2023-12-27 04:53:48,374][105620] Updated weights for policy 1, policy_version 1857520 (0.0008) [2023-12-27 04:53:48,658][105692] Updated weights for policy 0, policy_version 1853044 (0.0010) [2023-12-27 04:53:48,721][105692] Updated weights for policy 0, policy_version 1853054 (0.0011) [2023-12-27 04:53:48,783][105692] Updated weights for policy 0, policy_version 1853064 (0.0010) [2023-12-27 04:53:48,994][105620] Updated weights for policy 1, policy_version 1857530 (0.0007) [2023-12-27 04:53:49,049][105620] Updated weights for policy 1, policy_version 1857540 (0.0006) [2023-12-27 04:53:49,108][105620] Updated weights for policy 1, policy_version 1857550 (0.0005) [2023-12-27 04:53:49,164][105620] Updated weights for policy 1, policy_version 1857560 (0.0005) [2023-12-27 04:53:49,546][105692] Updated weights for policy 0, policy_version 1853074 (0.0010) [2023-12-27 04:53:49,613][105692] Updated weights for policy 0, policy_version 1853084 (0.0011) [2023-12-27 04:53:49,671][105692] Updated weights for policy 0, policy_version 1853094 (0.0007) [2023-12-27 04:53:49,724][105692] Updated weights for policy 0, policy_version 1853104 (0.0009) [2023-12-27 04:53:49,831][105620] Updated weights for policy 1, policy_version 1857570 (0.0007) [2023-12-27 04:53:49,890][105620] Updated weights for policy 1, policy_version 1857580 (0.0008) [2023-12-27 04:53:49,956][105620] Updated weights for policy 1, policy_version 1857590 (0.0009) [2023-12-27 04:53:50,476][105692] Updated weights for policy 0, policy_version 1853114 (0.0009) [2023-12-27 04:53:50,536][105692] Updated weights for policy 0, policy_version 1853124 (0.0009) [2023-12-27 04:53:50,600][105692] Updated weights for policy 0, policy_version 1853134 (0.0009) [2023-12-27 04:53:50,694][105620] Updated weights for policy 1, policy_version 1857600 (0.0010) [2023-12-27 04:53:50,756][105620] Updated weights for policy 1, policy_version 1857610 (0.0009) [2023-12-27 04:53:50,816][105620] Updated weights for policy 1, policy_version 1857620 (0.0009) [2023-12-27 04:53:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19114.6, 300 sec: 19466.4). Total num frames: 950091776. Throughput: 0: 9494.6, 1: 9778.2. Samples: 950080616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:53:51,063][104569] Avg episode reward: [(0, '8623.903'), (1, '9071.166')] [2023-12-27 04:53:51,252][105692] Updated weights for policy 0, policy_version 1853144 (0.0007) [2023-12-27 04:53:51,302][105692] Updated weights for policy 0, policy_version 1853154 (0.0006) [2023-12-27 04:53:51,365][105692] Updated weights for policy 0, policy_version 1853164 (0.0007) [2023-12-27 04:53:51,669][105620] Updated weights for policy 1, policy_version 1857630 (0.0008) [2023-12-27 04:53:51,735][105620] Updated weights for policy 1, policy_version 1857640 (0.0007) [2023-12-27 04:53:51,804][105620] Updated weights for policy 1, policy_version 1857650 (0.0006) [2023-12-27 04:53:52,007][105692] Updated weights for policy 0, policy_version 1853174 (0.0007) [2023-12-27 04:53:52,075][105692] Updated weights for policy 0, policy_version 1853184 (0.0010) [2023-12-27 04:53:52,141][105692] Updated weights for policy 0, policy_version 1853194 (0.0010) [2023-12-27 04:53:52,500][105620] Updated weights for policy 1, policy_version 1857660 (0.0007) [2023-12-27 04:53:52,554][105620] Updated weights for policy 1, policy_version 1857670 (0.0009) [2023-12-27 04:53:52,605][105620] Updated weights for policy 1, policy_version 1857680 (0.0010) [2023-12-27 04:53:52,893][105692] Updated weights for policy 0, policy_version 1853204 (0.0009) [2023-12-27 04:53:52,946][105692] Updated weights for policy 0, policy_version 1853214 (0.0006) [2023-12-27 04:53:53,005][105692] Updated weights for policy 0, policy_version 1853224 (0.0009) [2023-12-27 04:53:53,365][105620] Updated weights for policy 1, policy_version 1857690 (0.0009) [2023-12-27 04:53:53,422][105620] Updated weights for policy 1, policy_version 1857700 (0.0008) [2023-12-27 04:53:53,477][105620] Updated weights for policy 1, policy_version 1857710 (0.0010) [2023-12-27 04:53:53,529][105620] Updated weights for policy 1, policy_version 1857720 (0.0010) [2023-12-27 04:53:53,679][105692] Updated weights for policy 0, policy_version 1853234 (0.0010) [2023-12-27 04:53:53,740][105692] Updated weights for policy 0, policy_version 1853244 (0.0011) [2023-12-27 04:53:53,808][105692] Updated weights for policy 0, policy_version 1853254 (0.0007) [2023-12-27 04:53:53,875][105692] Updated weights for policy 0, policy_version 1853264 (0.0005) [2023-12-27 04:53:54,136][105620] Updated weights for policy 1, policy_version 1857730 (0.0007) [2023-12-27 04:53:54,196][105620] Updated weights for policy 1, policy_version 1857740 (0.0010) [2023-12-27 04:53:54,257][105620] Updated weights for policy 1, policy_version 1857750 (0.0010) [2023-12-27 04:53:54,497][105692] Updated weights for policy 0, policy_version 1853274 (0.0009) [2023-12-27 04:53:54,549][105692] Updated weights for policy 0, policy_version 1853284 (0.0005) [2023-12-27 04:53:54,606][105692] Updated weights for policy 0, policy_version 1853294 (0.0006) [2023-12-27 04:53:54,858][105620] Updated weights for policy 1, policy_version 1857760 (0.0006) [2023-12-27 04:53:54,909][105620] Updated weights for policy 1, policy_version 1857770 (0.0005) [2023-12-27 04:53:54,967][105620] Updated weights for policy 1, policy_version 1857780 (0.0008) [2023-12-27 04:53:55,255][105692] Updated weights for policy 0, policy_version 1853304 (0.0008) [2023-12-27 04:53:55,307][105692] Updated weights for policy 0, policy_version 1853314 (0.0010) [2023-12-27 04:53:55,356][105692] Updated weights for policy 0, policy_version 1853324 (0.0011) [2023-12-27 04:53:55,642][105620] Updated weights for policy 1, policy_version 1857790 (0.0009) [2023-12-27 04:53:55,703][105620] Updated weights for policy 1, policy_version 1857800 (0.0011) [2023-12-27 04:53:55,758][105620] Updated weights for policy 1, policy_version 1857810 (0.0010) [2023-12-27 04:53:55,993][105692] Updated weights for policy 0, policy_version 1853334 (0.0011) [2023-12-27 04:53:56,042][105692] Updated weights for policy 0, policy_version 1853344 (0.0011) [2023-12-27 04:53:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 950190080. Throughput: 0: 9594.0, 1: 9778.3. Samples: 950200164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:53:56,062][104569] Avg episode reward: [(0, '8717.351'), (1, '9255.612')] [2023-12-27 04:53:56,090][105692] Updated weights for policy 0, policy_version 1853354 (0.0010) [2023-12-27 04:53:56,342][105620] Updated weights for policy 1, policy_version 1857820 (0.0009) [2023-12-27 04:53:56,410][105620] Updated weights for policy 1, policy_version 1857830 (0.0005) [2023-12-27 04:53:56,457][105620] Updated weights for policy 1, policy_version 1857840 (0.0005) [2023-12-27 04:53:56,720][105692] Updated weights for policy 0, policy_version 1853364 (0.0008) [2023-12-27 04:53:56,773][105692] Updated weights for policy 0, policy_version 1853374 (0.0005) [2023-12-27 04:53:56,833][105692] Updated weights for policy 0, policy_version 1853384 (0.0007) [2023-12-27 04:53:57,008][105620] Updated weights for policy 1, policy_version 1857850 (0.0005) [2023-12-27 04:53:57,073][105620] Updated weights for policy 1, policy_version 1857860 (0.0005) [2023-12-27 04:53:57,133][105620] Updated weights for policy 1, policy_version 1857870 (0.0005) [2023-12-27 04:53:57,186][105620] Updated weights for policy 1, policy_version 1857880 (0.0005) [2023-12-27 04:53:57,355][105692] Updated weights for policy 0, policy_version 1853394 (0.0008) [2023-12-27 04:53:57,415][105692] Updated weights for policy 0, policy_version 1853404 (0.0005) [2023-12-27 04:53:57,485][105692] Updated weights for policy 0, policy_version 1853414 (0.0005) [2023-12-27 04:53:57,552][105692] Updated weights for policy 0, policy_version 1853424 (0.0005) [2023-12-27 04:53:57,749][105620] Updated weights for policy 1, policy_version 1857890 (0.0005) [2023-12-27 04:53:57,799][105620] Updated weights for policy 1, policy_version 1857900 (0.0005) [2023-12-27 04:53:57,852][105620] Updated weights for policy 1, policy_version 1857910 (0.0006) [2023-12-27 04:53:58,025][105692] Updated weights for policy 0, policy_version 1853434 (0.0006) [2023-12-27 04:53:58,087][105692] Updated weights for policy 0, policy_version 1853444 (0.0008) [2023-12-27 04:53:58,159][105692] Updated weights for policy 0, policy_version 1853454 (0.0008) [2023-12-27 04:53:58,583][105620] Updated weights for policy 1, policy_version 1857920 (0.0008) [2023-12-27 04:53:58,649][105620] Updated weights for policy 1, policy_version 1857930 (0.0008) [2023-12-27 04:53:58,714][105620] Updated weights for policy 1, policy_version 1857940 (0.0009) [2023-12-27 04:53:58,976][105692] Updated weights for policy 0, policy_version 1853464 (0.0008) [2023-12-27 04:53:59,036][105692] Updated weights for policy 0, policy_version 1853474 (0.0008) [2023-12-27 04:53:59,093][105692] Updated weights for policy 0, policy_version 1853484 (0.0008) [2023-12-27 04:53:59,551][105620] Updated weights for policy 1, policy_version 1857950 (0.0006) [2023-12-27 04:53:59,606][105620] Updated weights for policy 1, policy_version 1857960 (0.0006) [2023-12-27 04:53:59,670][105620] Updated weights for policy 1, policy_version 1857970 (0.0007) [2023-12-27 04:53:59,739][105692] Updated weights for policy 0, policy_version 1853494 (0.0006) [2023-12-27 04:53:59,788][105692] Updated weights for policy 0, policy_version 1853504 (0.0005) [2023-12-27 04:53:59,843][105692] Updated weights for policy 0, policy_version 1853514 (0.0008) [2023-12-27 04:54:00,290][105620] Updated weights for policy 1, policy_version 1857980 (0.0007) [2023-12-27 04:54:00,348][105620] Updated weights for policy 1, policy_version 1857990 (0.0005) [2023-12-27 04:54:00,406][105620] Updated weights for policy 1, policy_version 1858000 (0.0007) [2023-12-27 04:54:00,640][105692] Updated weights for policy 0, policy_version 1853524 (0.0011) [2023-12-27 04:54:00,698][105692] Updated weights for policy 0, policy_version 1853534 (0.0011) [2023-12-27 04:54:00,748][105692] Updated weights for policy 0, policy_version 1853544 (0.0010) [2023-12-27 04:54:01,051][105620] Updated weights for policy 1, policy_version 1858010 (0.0007) [2023-12-27 04:54:01,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 950296576. Throughput: 0: 9730.1, 1: 9868.7. Samples: 950267112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:54:01,062][104569] Avg episode reward: [(0, '8716.421'), (1, '9254.650')] [2023-12-27 04:54:01,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001853552_474578944.pth... [2023-12-27 04:54:01,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001852400_474284032.pth [2023-12-27 04:54:01,110][105620] Updated weights for policy 1, policy_version 1858020 (0.0009) [2023-12-27 04:54:01,173][105620] Updated weights for policy 1, policy_version 1858030 (0.0009) [2023-12-27 04:54:01,234][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001858040_475725824.pth... [2023-12-27 04:54:01,236][105620] Updated weights for policy 1, policy_version 1858040 (0.0009) [2023-12-27 04:54:01,238][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001856856_475422720.pth [2023-12-27 04:54:01,387][105692] Updated weights for policy 0, policy_version 1853554 (0.0006) [2023-12-27 04:54:01,439][105692] Updated weights for policy 0, policy_version 1853564 (0.0009) [2023-12-27 04:54:01,497][105692] Updated weights for policy 0, policy_version 1853574 (0.0009) [2023-12-27 04:54:01,557][105692] Updated weights for policy 0, policy_version 1853584 (0.0009) [2023-12-27 04:54:01,980][105620] Updated weights for policy 1, policy_version 1858050 (0.0007) [2023-12-27 04:54:02,038][105620] Updated weights for policy 1, policy_version 1858060 (0.0009) [2023-12-27 04:54:02,099][105620] Updated weights for policy 1, policy_version 1858070 (0.0008) [2023-12-27 04:54:02,347][105692] Updated weights for policy 0, policy_version 1853594 (0.0009) [2023-12-27 04:54:02,413][105692] Updated weights for policy 0, policy_version 1853604 (0.0006) [2023-12-27 04:54:02,480][105692] Updated weights for policy 0, policy_version 1853614 (0.0007) [2023-12-27 04:54:02,775][105620] Updated weights for policy 1, policy_version 1858080 (0.0007) [2023-12-27 04:54:02,831][105620] Updated weights for policy 1, policy_version 1858090 (0.0007) [2023-12-27 04:54:02,881][105620] Updated weights for policy 1, policy_version 1858100 (0.0005) [2023-12-27 04:54:03,063][105692] Updated weights for policy 0, policy_version 1853624 (0.0006) [2023-12-27 04:54:03,113][105692] Updated weights for policy 0, policy_version 1853634 (0.0005) [2023-12-27 04:54:03,173][105692] Updated weights for policy 0, policy_version 1853644 (0.0005) [2023-12-27 04:54:03,570][105620] Updated weights for policy 1, policy_version 1858110 (0.0005) [2023-12-27 04:54:03,620][105620] Updated weights for policy 1, policy_version 1858120 (0.0007) [2023-12-27 04:54:03,678][105620] Updated weights for policy 1, policy_version 1858130 (0.0013) [2023-12-27 04:54:03,764][105692] Updated weights for policy 0, policy_version 1853654 (0.0006) [2023-12-27 04:54:03,809][105692] Updated weights for policy 0, policy_version 1853664 (0.0006) [2023-12-27 04:54:03,867][105692] Updated weights for policy 0, policy_version 1853674 (0.0007) [2023-12-27 04:54:04,453][105620] Updated weights for policy 1, policy_version 1858141 (0.0009) [2023-12-27 04:54:04,512][105620] Updated weights for policy 1, policy_version 1858151 (0.0009) [2023-12-27 04:54:04,575][105620] Updated weights for policy 1, policy_version 1858161 (0.0009) [2023-12-27 04:54:04,610][105692] Updated weights for policy 0, policy_version 1853684 (0.0007) [2023-12-27 04:54:04,661][105692] Updated weights for policy 0, policy_version 1853694 (0.0009) [2023-12-27 04:54:04,715][105692] Updated weights for policy 0, policy_version 1853704 (0.0009) [2023-12-27 04:54:05,180][105620] Updated weights for policy 1, policy_version 1858171 (0.0007) [2023-12-27 04:54:05,248][105620] Updated weights for policy 1, policy_version 1858181 (0.0007) [2023-12-27 04:54:05,305][105620] Updated weights for policy 1, policy_version 1858191 (0.0008) [2023-12-27 04:54:05,310][105692] Updated weights for policy 0, policy_version 1853714 (0.0007) [2023-12-27 04:54:05,362][105692] Updated weights for policy 0, policy_version 1853724 (0.0010) [2023-12-27 04:54:05,408][105692] Updated weights for policy 0, policy_version 1853734 (0.0007) [2023-12-27 04:54:05,455][105692] Updated weights for policy 0, policy_version 1853744 (0.0008) [2023-12-27 04:54:05,886][105620] Updated weights for policy 1, policy_version 1858201 (0.0007) [2023-12-27 04:54:05,942][105620] Updated weights for policy 1, policy_version 1858211 (0.0011) [2023-12-27 04:54:06,000][105620] Updated weights for policy 1, policy_version 1858221 (0.0010) [2023-12-27 04:54:06,056][105620] Updated weights for policy 1, policy_version 1858231 (0.0010) [2023-12-27 04:54:06,062][104569] Fps is (10 sec: 21299.3, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 950403072. Throughput: 0: 9713.5, 1: 9965.4. Samples: 950386900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:54:06,062][104569] Avg episode reward: [(0, '8444.059'), (1, '9254.574')] [2023-12-27 04:54:06,114][105692] Updated weights for policy 0, policy_version 1853754 (0.0011) [2023-12-27 04:54:06,170][105692] Updated weights for policy 0, policy_version 1853764 (0.0011) [2023-12-27 04:54:06,227][105692] Updated weights for policy 0, policy_version 1853774 (0.0010) [2023-12-27 04:54:06,713][105620] Updated weights for policy 1, policy_version 1858241 (0.0011) [2023-12-27 04:54:06,783][105620] Updated weights for policy 1, policy_version 1858251 (0.0010) [2023-12-27 04:54:06,846][105620] Updated weights for policy 1, policy_version 1858261 (0.0011) [2023-12-27 04:54:06,856][105692] Updated weights for policy 0, policy_version 1853784 (0.0006) [2023-12-27 04:54:06,919][105692] Updated weights for policy 0, policy_version 1853794 (0.0008) [2023-12-27 04:54:06,981][105692] Updated weights for policy 0, policy_version 1853804 (0.0010) [2023-12-27 04:54:07,483][105620] Updated weights for policy 1, policy_version 1858271 (0.0009) [2023-12-27 04:54:07,544][105620] Updated weights for policy 1, policy_version 1858281 (0.0006) [2023-12-27 04:54:07,603][105620] Updated weights for policy 1, policy_version 1858291 (0.0006) [2023-12-27 04:54:07,707][105692] Updated weights for policy 0, policy_version 1853814 (0.0010) [2023-12-27 04:54:07,765][105692] Updated weights for policy 0, policy_version 1853824 (0.0010) [2023-12-27 04:54:07,834][105692] Updated weights for policy 0, policy_version 1853834 (0.0008) [2023-12-27 04:54:08,199][105620] Updated weights for policy 1, policy_version 1858301 (0.0005) [2023-12-27 04:54:08,245][105620] Updated weights for policy 1, policy_version 1858311 (0.0005) [2023-12-27 04:54:08,303][105620] Updated weights for policy 1, policy_version 1858321 (0.0008) [2023-12-27 04:54:08,465][105692] Updated weights for policy 0, policy_version 1853844 (0.0008) [2023-12-27 04:54:08,526][105692] Updated weights for policy 0, policy_version 1853854 (0.0008) [2023-12-27 04:54:08,587][105692] Updated weights for policy 0, policy_version 1853864 (0.0008) [2023-12-27 04:54:08,963][105620] Updated weights for policy 1, policy_version 1858331 (0.0009) [2023-12-27 04:54:09,036][105620] Updated weights for policy 1, policy_version 1858341 (0.0008) [2023-12-27 04:54:09,104][105620] Updated weights for policy 1, policy_version 1858351 (0.0006) [2023-12-27 04:54:09,254][105692] Updated weights for policy 0, policy_version 1853874 (0.0008) [2023-12-27 04:54:09,310][105692] Updated weights for policy 0, policy_version 1853884 (0.0008) [2023-12-27 04:54:09,375][105692] Updated weights for policy 0, policy_version 1853894 (0.0008) [2023-12-27 04:54:09,433][105692] Updated weights for policy 0, policy_version 1853904 (0.0008) [2023-12-27 04:54:09,814][105620] Updated weights for policy 1, policy_version 1858361 (0.0011) [2023-12-27 04:54:09,875][105620] Updated weights for policy 1, policy_version 1858371 (0.0011) [2023-12-27 04:54:09,933][105620] Updated weights for policy 1, policy_version 1858381 (0.0011) [2023-12-27 04:54:09,999][105620] Updated weights for policy 1, policy_version 1858391 (0.0010) [2023-12-27 04:54:10,236][105692] Updated weights for policy 0, policy_version 1853914 (0.0008) [2023-12-27 04:54:10,293][105692] Updated weights for policy 0, policy_version 1853924 (0.0008) [2023-12-27 04:54:10,350][105692] Updated weights for policy 0, policy_version 1853934 (0.0006) [2023-12-27 04:54:10,688][105620] Updated weights for policy 1, policy_version 1858401 (0.0006) [2023-12-27 04:54:10,748][105620] Updated weights for policy 1, policy_version 1858411 (0.0005) [2023-12-27 04:54:10,814][105620] Updated weights for policy 1, policy_version 1858421 (0.0007) [2023-12-27 04:54:10,991][105692] Updated weights for policy 0, policy_version 1853944 (0.0009) [2023-12-27 04:54:11,055][105692] Updated weights for policy 0, policy_version 1853955 (0.0008) [2023-12-27 04:54:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 950501376. Throughput: 0: 9873.1, 1: 10085.7. Samples: 950512204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:54:11,062][104569] Avg episode reward: [(0, '8537.369'), (1, '9346.992')] [2023-12-27 04:54:11,107][105692] Updated weights for policy 0, policy_version 1853965 (0.0008) [2023-12-27 04:54:11,479][105620] Updated weights for policy 1, policy_version 1858431 (0.0008) [2023-12-27 04:54:11,527][105620] Updated weights for policy 1, policy_version 1858441 (0.0009) [2023-12-27 04:54:11,589][105620] Updated weights for policy 1, policy_version 1858451 (0.0008) [2023-12-27 04:54:11,943][105692] Updated weights for policy 0, policy_version 1853975 (0.0006) [2023-12-27 04:54:12,008][105692] Updated weights for policy 0, policy_version 1853985 (0.0005) [2023-12-27 04:54:12,063][105692] Updated weights for policy 0, policy_version 1853995 (0.0005) [2023-12-27 04:54:12,281][105620] Updated weights for policy 1, policy_version 1858461 (0.0008) [2023-12-27 04:54:12,346][105620] Updated weights for policy 1, policy_version 1858471 (0.0009) [2023-12-27 04:54:12,410][105620] Updated weights for policy 1, policy_version 1858481 (0.0009) [2023-12-27 04:54:12,650][105692] Updated weights for policy 0, policy_version 1854005 (0.0006) [2023-12-27 04:54:12,698][105692] Updated weights for policy 0, policy_version 1854015 (0.0006) [2023-12-27 04:54:12,758][105692] Updated weights for policy 0, policy_version 1854025 (0.0005) [2023-12-27 04:54:13,180][105620] Updated weights for policy 1, policy_version 1858491 (0.0009) [2023-12-27 04:54:13,250][105620] Updated weights for policy 1, policy_version 1858501 (0.0005) [2023-12-27 04:54:13,314][105620] Updated weights for policy 1, policy_version 1858511 (0.0007) [2023-12-27 04:54:13,427][105692] Updated weights for policy 0, policy_version 1854035 (0.0009) [2023-12-27 04:54:13,481][105692] Updated weights for policy 0, policy_version 1854045 (0.0010) [2023-12-27 04:54:13,546][105692] Updated weights for policy 0, policy_version 1854055 (0.0010) [2023-12-27 04:54:13,859][105620] Updated weights for policy 1, policy_version 1858521 (0.0007) [2023-12-27 04:54:13,916][105620] Updated weights for policy 1, policy_version 1858531 (0.0005) [2023-12-27 04:54:13,966][105620] Updated weights for policy 1, policy_version 1858541 (0.0005) [2023-12-27 04:54:14,020][105620] Updated weights for policy 1, policy_version 1858551 (0.0005) [2023-12-27 04:54:14,146][105692] Updated weights for policy 0, policy_version 1854065 (0.0009) [2023-12-27 04:54:14,216][105692] Updated weights for policy 0, policy_version 1854075 (0.0006) [2023-12-27 04:54:14,280][105692] Updated weights for policy 0, policy_version 1854085 (0.0006) [2023-12-27 04:54:14,344][105692] Updated weights for policy 0, policy_version 1854095 (0.0009) [2023-12-27 04:54:14,632][105620] Updated weights for policy 1, policy_version 1858561 (0.0008) [2023-12-27 04:54:14,688][105620] Updated weights for policy 1, policy_version 1858571 (0.0008) [2023-12-27 04:54:14,736][105620] Updated weights for policy 1, policy_version 1858581 (0.0009) [2023-12-27 04:54:14,900][105692] Updated weights for policy 0, policy_version 1854105 (0.0008) [2023-12-27 04:54:14,959][105692] Updated weights for policy 0, policy_version 1854115 (0.0008) [2023-12-27 04:54:15,018][105692] Updated weights for policy 0, policy_version 1854125 (0.0008) [2023-12-27 04:54:15,555][105620] Updated weights for policy 1, policy_version 1858591 (0.0009) [2023-12-27 04:54:15,615][105620] Updated weights for policy 1, policy_version 1858601 (0.0006) [2023-12-27 04:54:15,668][105620] Updated weights for policy 1, policy_version 1858611 (0.0010) [2023-12-27 04:54:15,735][105692] Updated weights for policy 0, policy_version 1854135 (0.0006) [2023-12-27 04:54:15,792][105692] Updated weights for policy 0, policy_version 1854145 (0.0010) [2023-12-27 04:54:15,836][105692] Updated weights for policy 0, policy_version 1854155 (0.0010) [2023-12-27 04:54:16,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 950607872. Throughput: 0: 9953.9, 1: 10089.6. Samples: 950573492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:54:16,062][104569] Avg episode reward: [(0, '8269.920'), (1, '9346.935')] [2023-12-27 04:54:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001854160_474734592.pth... [2023-12-27 04:54:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001858616_475873280.pth... [2023-12-27 04:54:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001857432_475570176.pth [2023-12-27 04:54:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001852944_474423296.pth [2023-12-27 04:54:16,390][105620] Updated weights for policy 1, policy_version 1858622 (0.0008) [2023-12-27 04:54:16,441][105620] Updated weights for policy 1, policy_version 1858632 (0.0006) [2023-12-27 04:54:16,462][105692] Updated weights for policy 0, policy_version 1854165 (0.0010) [2023-12-27 04:54:16,508][105620] Updated weights for policy 1, policy_version 1858642 (0.0005) [2023-12-27 04:54:16,518][105692] Updated weights for policy 0, policy_version 1854175 (0.0010) [2023-12-27 04:54:16,579][105692] Updated weights for policy 0, policy_version 1854185 (0.0009) [2023-12-27 04:54:17,035][105620] Updated weights for policy 1, policy_version 1858652 (0.0005) [2023-12-27 04:54:17,096][105620] Updated weights for policy 1, policy_version 1858662 (0.0008) [2023-12-27 04:54:17,154][105620] Updated weights for policy 1, policy_version 1858672 (0.0010) [2023-12-27 04:54:17,300][105692] Updated weights for policy 0, policy_version 1854195 (0.0007) [2023-12-27 04:54:17,358][105692] Updated weights for policy 0, policy_version 1854205 (0.0010) [2023-12-27 04:54:17,414][105692] Updated weights for policy 0, policy_version 1854215 (0.0011) [2023-12-27 04:54:17,752][105620] Updated weights for policy 1, policy_version 1858682 (0.0009) [2023-12-27 04:54:17,800][105620] Updated weights for policy 1, policy_version 1858692 (0.0005) [2023-12-27 04:54:17,849][105620] Updated weights for policy 1, policy_version 1858702 (0.0005) [2023-12-27 04:54:17,900][105620] Updated weights for policy 1, policy_version 1858712 (0.0005) [2023-12-27 04:54:18,028][105692] Updated weights for policy 0, policy_version 1854225 (0.0010) [2023-12-27 04:54:18,090][105692] Updated weights for policy 0, policy_version 1854235 (0.0005) [2023-12-27 04:54:18,149][105692] Updated weights for policy 0, policy_version 1854245 (0.0006) [2023-12-27 04:54:18,207][105692] Updated weights for policy 0, policy_version 1854255 (0.0007) [2023-12-27 04:54:18,540][105620] Updated weights for policy 1, policy_version 1858722 (0.0010) [2023-12-27 04:54:18,608][105620] Updated weights for policy 1, policy_version 1858732 (0.0008) [2023-12-27 04:54:18,673][105620] Updated weights for policy 1, policy_version 1858742 (0.0011) [2023-12-27 04:54:18,883][105692] Updated weights for policy 0, policy_version 1854265 (0.0011) [2023-12-27 04:54:18,935][105692] Updated weights for policy 0, policy_version 1854275 (0.0010) [2023-12-27 04:54:18,983][105692] Updated weights for policy 0, policy_version 1854285 (0.0010) [2023-12-27 04:54:19,382][105620] Updated weights for policy 1, policy_version 1858752 (0.0010) [2023-12-27 04:54:19,444][105620] Updated weights for policy 1, policy_version 1858762 (0.0010) [2023-12-27 04:54:19,517][105620] Updated weights for policy 1, policy_version 1858772 (0.0010) [2023-12-27 04:54:19,757][105692] Updated weights for policy 0, policy_version 1854295 (0.0009) [2023-12-27 04:54:19,819][105692] Updated weights for policy 0, policy_version 1854305 (0.0009) [2023-12-27 04:54:19,891][105692] Updated weights for policy 0, policy_version 1854315 (0.0008) [2023-12-27 04:54:20,288][105620] Updated weights for policy 1, policy_version 1858782 (0.0009) [2023-12-27 04:54:20,343][105620] Updated weights for policy 1, policy_version 1858792 (0.0008) [2023-12-27 04:54:20,399][105620] Updated weights for policy 1, policy_version 1858802 (0.0008) [2023-12-27 04:54:20,668][105692] Updated weights for policy 0, policy_version 1854325 (0.0009) [2023-12-27 04:54:20,731][105692] Updated weights for policy 0, policy_version 1854335 (0.0009) [2023-12-27 04:54:20,782][105692] Updated weights for policy 0, policy_version 1854345 (0.0009) [2023-12-27 04:54:21,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 950706176. Throughput: 0: 10031.0, 1: 10147.1. Samples: 950697744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:54:21,062][104569] Avg episode reward: [(0, '7719.928'), (1, '9254.697')] [2023-12-27 04:54:21,077][105620] Updated weights for policy 1, policy_version 1858812 (0.0010) [2023-12-27 04:54:21,137][105620] Updated weights for policy 1, policy_version 1858822 (0.0009) [2023-12-27 04:54:21,194][105620] Updated weights for policy 1, policy_version 1858832 (0.0011) [2023-12-27 04:54:21,691][105692] Updated weights for policy 0, policy_version 1854356 (0.0009) [2023-12-27 04:54:21,759][105692] Updated weights for policy 0, policy_version 1854366 (0.0008) [2023-12-27 04:54:21,826][105692] Updated weights for policy 0, policy_version 1854376 (0.0009) [2023-12-27 04:54:21,986][105620] Updated weights for policy 1, policy_version 1858842 (0.0010) [2023-12-27 04:54:22,051][105620] Updated weights for policy 1, policy_version 1858852 (0.0006) [2023-12-27 04:54:22,116][105620] Updated weights for policy 1, policy_version 1858862 (0.0009) [2023-12-27 04:54:22,185][105620] Updated weights for policy 1, policy_version 1858872 (0.0007) [2023-12-27 04:54:22,591][105692] Updated weights for policy 0, policy_version 1854386 (0.0008) [2023-12-27 04:54:22,651][105692] Updated weights for policy 0, policy_version 1854396 (0.0008) [2023-12-27 04:54:22,718][105692] Updated weights for policy 0, policy_version 1854406 (0.0008) [2023-12-27 04:54:22,786][105692] Updated weights for policy 0, policy_version 1854416 (0.0008) [2023-12-27 04:54:22,915][105620] Updated weights for policy 1, policy_version 1858882 (0.0011) [2023-12-27 04:54:22,978][105620] Updated weights for policy 1, policy_version 1858892 (0.0011) [2023-12-27 04:54:23,022][105620] Updated weights for policy 1, policy_version 1858902 (0.0010) [2023-12-27 04:54:23,531][105692] Updated weights for policy 0, policy_version 1854426 (0.0009) [2023-12-27 04:54:23,583][105692] Updated weights for policy 0, policy_version 1854436 (0.0007) [2023-12-27 04:54:23,641][105692] Updated weights for policy 0, policy_version 1854446 (0.0006) [2023-12-27 04:54:23,731][105620] Updated weights for policy 1, policy_version 1858912 (0.0006) [2023-12-27 04:54:23,791][105620] Updated weights for policy 1, policy_version 1858922 (0.0005) [2023-12-27 04:54:23,848][105620] Updated weights for policy 1, policy_version 1858932 (0.0007) [2023-12-27 04:54:24,338][105692] Updated weights for policy 0, policy_version 1854456 (0.0010) [2023-12-27 04:54:24,399][105692] Updated weights for policy 0, policy_version 1854466 (0.0008) [2023-12-27 04:54:24,450][105692] Updated weights for policy 0, policy_version 1854476 (0.0010) [2023-12-27 04:54:24,499][105620] Updated weights for policy 1, policy_version 1858942 (0.0008) [2023-12-27 04:54:24,543][105620] Updated weights for policy 1, policy_version 1858952 (0.0010) [2023-12-27 04:54:24,606][105620] Updated weights for policy 1, policy_version 1858962 (0.0006) [2023-12-27 04:54:25,090][105692] Updated weights for policy 0, policy_version 1854486 (0.0011) [2023-12-27 04:54:25,142][105692] Updated weights for policy 0, policy_version 1854496 (0.0010) [2023-12-27 04:54:25,198][105692] Updated weights for policy 0, policy_version 1854506 (0.0007) [2023-12-27 04:54:25,328][105620] Updated weights for policy 1, policy_version 1858972 (0.0007) [2023-12-27 04:54:25,386][105620] Updated weights for policy 1, policy_version 1858982 (0.0010) [2023-12-27 04:54:25,434][105620] Updated weights for policy 1, policy_version 1858992 (0.0010) [2023-12-27 04:54:25,775][105692] Updated weights for policy 0, policy_version 1854516 (0.0005) [2023-12-27 04:54:25,828][105692] Updated weights for policy 0, policy_version 1854526 (0.0005) [2023-12-27 04:54:25,891][105692] Updated weights for policy 0, policy_version 1854536 (0.0005) [2023-12-27 04:54:26,062][104569] Fps is (10 sec: 19660.5, 60 sec: 20070.3, 300 sec: 19521.9). Total num frames: 950804480. Throughput: 0: 10009.6, 1: 10126.9. Samples: 950812460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:54:26,063][104569] Avg episode reward: [(0, '7893.962'), (1, '9254.690')] [2023-12-27 04:54:26,155][105620] Updated weights for policy 1, policy_version 1859002 (0.0010) [2023-12-27 04:54:26,203][105620] Updated weights for policy 1, policy_version 1859012 (0.0010) [2023-12-27 04:54:26,261][105620] Updated weights for policy 1, policy_version 1859022 (0.0008) [2023-12-27 04:54:26,308][105620] Updated weights for policy 1, policy_version 1859032 (0.0008) [2023-12-27 04:54:26,569][105692] Updated weights for policy 0, policy_version 1854546 (0.0007) [2023-12-27 04:54:26,623][105692] Updated weights for policy 0, policy_version 1854556 (0.0010) [2023-12-27 04:54:26,674][105692] Updated weights for policy 0, policy_version 1854566 (0.0010) [2023-12-27 04:54:26,721][105692] Updated weights for policy 0, policy_version 1854576 (0.0010) [2023-12-27 04:54:27,082][105620] Updated weights for policy 1, policy_version 1859042 (0.0006) [2023-12-27 04:54:27,132][105620] Updated weights for policy 1, policy_version 1859052 (0.0006) [2023-12-27 04:54:27,195][105620] Updated weights for policy 1, policy_version 1859062 (0.0006) [2023-12-27 04:54:27,477][105692] Updated weights for policy 0, policy_version 1854586 (0.0010) [2023-12-27 04:54:27,535][105692] Updated weights for policy 0, policy_version 1854596 (0.0010) [2023-12-27 04:54:27,593][105692] Updated weights for policy 0, policy_version 1854606 (0.0010) [2023-12-27 04:54:27,709][105620] Updated weights for policy 1, policy_version 1859072 (0.0007) [2023-12-27 04:54:27,767][105620] Updated weights for policy 1, policy_version 1859082 (0.0010) [2023-12-27 04:54:27,825][105620] Updated weights for policy 1, policy_version 1859092 (0.0010) [2023-12-27 04:54:28,324][105692] Updated weights for policy 0, policy_version 1854616 (0.0011) [2023-12-27 04:54:28,382][105620] Updated weights for policy 1, policy_version 1859102 (0.0011) [2023-12-27 04:54:28,385][105692] Updated weights for policy 0, policy_version 1854626 (0.0011) [2023-12-27 04:54:28,445][105692] Updated weights for policy 0, policy_version 1854636 (0.0011) [2023-12-27 04:54:28,445][105620] Updated weights for policy 1, policy_version 1859112 (0.0011) [2023-12-27 04:54:28,497][105620] Updated weights for policy 1, policy_version 1859122 (0.0010) [2023-12-27 04:54:29,122][105692] Updated weights for policy 0, policy_version 1854646 (0.0007) [2023-12-27 04:54:29,165][105692] Updated weights for policy 0, policy_version 1854656 (0.0005) [2023-12-27 04:54:29,217][105692] Updated weights for policy 0, policy_version 1854666 (0.0009) [2023-12-27 04:54:29,247][105620] Updated weights for policy 1, policy_version 1859132 (0.0010) [2023-12-27 04:54:29,308][105620] Updated weights for policy 1, policy_version 1859142 (0.0010) [2023-12-27 04:54:29,375][105620] Updated weights for policy 1, policy_version 1859152 (0.0008) [2023-12-27 04:54:29,923][105692] Updated weights for policy 0, policy_version 1854676 (0.0009) [2023-12-27 04:54:29,987][105692] Updated weights for policy 0, policy_version 1854686 (0.0009) [2023-12-27 04:54:29,990][105620] Updated weights for policy 1, policy_version 1859162 (0.0006) [2023-12-27 04:54:30,039][105692] Updated weights for policy 0, policy_version 1854696 (0.0010) [2023-12-27 04:54:30,045][105620] Updated weights for policy 1, policy_version 1859172 (0.0005) [2023-12-27 04:54:30,109][105620] Updated weights for policy 1, policy_version 1859182 (0.0006) [2023-12-27 04:54:30,163][105620] Updated weights for policy 1, policy_version 1859192 (0.0009) [2023-12-27 04:54:30,718][105692] Updated weights for policy 0, policy_version 1854706 (0.0010) [2023-12-27 04:54:30,776][105692] Updated weights for policy 0, policy_version 1854716 (0.0010) [2023-12-27 04:54:30,830][105692] Updated weights for policy 0, policy_version 1854726 (0.0010) [2023-12-27 04:54:30,878][105692] Updated weights for policy 0, policy_version 1854736 (0.0010) [2023-12-27 04:54:30,905][105620] Updated weights for policy 1, policy_version 1859202 (0.0010) [2023-12-27 04:54:30,961][105620] Updated weights for policy 1, policy_version 1859212 (0.0008) [2023-12-27 04:54:31,022][105620] Updated weights for policy 1, policy_version 1859222 (0.0005) [2023-12-27 04:54:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 20070.5, 300 sec: 19605.3). Total num frames: 950910976. Throughput: 0: 10053.5, 1: 10234.2. Samples: 950873716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:54:31,062][104569] Avg episode reward: [(0, '8074.108'), (1, '9346.858')] [2023-12-27 04:54:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001854736_474882048.pth... [2023-12-27 04:54:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001859224_476028928.pth... [2023-12-27 04:54:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001853552_474578944.pth [2023-12-27 04:54:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001858040_475725824.pth [2023-12-27 04:54:31,660][105692] Updated weights for policy 0, policy_version 1854746 (0.0008) [2023-12-27 04:54:31,712][105692] Updated weights for policy 0, policy_version 1854756 (0.0006) [2023-12-27 04:54:31,734][105620] Updated weights for policy 1, policy_version 1859232 (0.0007) [2023-12-27 04:54:31,778][105692] Updated weights for policy 0, policy_version 1854767 (0.0007) [2023-12-27 04:54:31,798][105620] Updated weights for policy 1, policy_version 1859242 (0.0008) [2023-12-27 04:54:31,858][105620] Updated weights for policy 1, policy_version 1859252 (0.0009) [2023-12-27 04:54:32,334][105692] Updated weights for policy 0, policy_version 1854777 (0.0008) [2023-12-27 04:54:32,394][105692] Updated weights for policy 0, policy_version 1854787 (0.0011) [2023-12-27 04:54:32,450][105692] Updated weights for policy 0, policy_version 1854797 (0.0011) [2023-12-27 04:54:32,667][105620] Updated weights for policy 1, policy_version 1859262 (0.0007) [2023-12-27 04:54:32,720][105620] Updated weights for policy 1, policy_version 1859272 (0.0005) [2023-12-27 04:54:32,771][105620] Updated weights for policy 1, policy_version 1859282 (0.0005) [2023-12-27 04:54:33,197][105692] Updated weights for policy 0, policy_version 1854807 (0.0011) [2023-12-27 04:54:33,252][105692] Updated weights for policy 0, policy_version 1854817 (0.0010) [2023-12-27 04:54:33,299][105692] Updated weights for policy 0, policy_version 1854827 (0.0010) [2023-12-27 04:54:33,460][105620] Updated weights for policy 1, policy_version 1859292 (0.0007) [2023-12-27 04:54:33,521][105620] Updated weights for policy 1, policy_version 1859302 (0.0008) [2023-12-27 04:54:33,571][105620] Updated weights for policy 1, policy_version 1859312 (0.0008) [2023-12-27 04:54:34,030][105692] Updated weights for policy 0, policy_version 1854837 (0.0008) [2023-12-27 04:54:34,085][105692] Updated weights for policy 0, policy_version 1854847 (0.0005) [2023-12-27 04:54:34,154][105692] Updated weights for policy 0, policy_version 1854857 (0.0008) [2023-12-27 04:54:34,394][105620] Updated weights for policy 1, policy_version 1859322 (0.0008) [2023-12-27 04:54:34,454][105620] Updated weights for policy 1, policy_version 1859332 (0.0007) [2023-12-27 04:54:34,513][105620] Updated weights for policy 1, policy_version 1859342 (0.0009) [2023-12-27 04:54:34,576][105620] Updated weights for policy 1, policy_version 1859352 (0.0008) [2023-12-27 04:54:34,847][105692] Updated weights for policy 0, policy_version 1854867 (0.0009) [2023-12-27 04:54:34,903][105692] Updated weights for policy 0, policy_version 1854877 (0.0005) [2023-12-27 04:54:34,951][105692] Updated weights for policy 0, policy_version 1854887 (0.0006) [2023-12-27 04:54:35,357][105620] Updated weights for policy 1, policy_version 1859362 (0.0008) [2023-12-27 04:54:35,405][105620] Updated weights for policy 1, policy_version 1859372 (0.0008) [2023-12-27 04:54:35,457][105620] Updated weights for policy 1, policy_version 1859382 (0.0008) [2023-12-27 04:54:35,664][105692] Updated weights for policy 0, policy_version 1854897 (0.0010) [2023-12-27 04:54:35,725][105692] Updated weights for policy 0, policy_version 1854907 (0.0010) [2023-12-27 04:54:35,780][105692] Updated weights for policy 0, policy_version 1854917 (0.0010) [2023-12-27 04:54:35,842][105692] Updated weights for policy 0, policy_version 1854927 (0.0010) [2023-12-27 04:54:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 951001088. Throughput: 0: 10140.9, 1: 10081.2. Samples: 950990612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:54:36,063][104569] Avg episode reward: [(0, '8533.621'), (1, '9346.847')] [2023-12-27 04:54:36,252][105620] Updated weights for policy 1, policy_version 1859392 (0.0010) [2023-12-27 04:54:36,322][105620] Updated weights for policy 1, policy_version 1859402 (0.0010) [2023-12-27 04:54:36,387][105620] Updated weights for policy 1, policy_version 1859412 (0.0008) [2023-12-27 04:54:36,520][105692] Updated weights for policy 0, policy_version 1854937 (0.0011) [2023-12-27 04:54:36,582][105692] Updated weights for policy 0, policy_version 1854947 (0.0011) [2023-12-27 04:54:36,635][105692] Updated weights for policy 0, policy_version 1854957 (0.0011) [2023-12-27 04:54:37,169][105620] Updated weights for policy 1, policy_version 1859422 (0.0008) [2023-12-27 04:54:37,226][105620] Updated weights for policy 1, policy_version 1859432 (0.0008) [2023-12-27 04:54:37,275][105620] Updated weights for policy 1, policy_version 1859442 (0.0008) [2023-12-27 04:54:37,376][105692] Updated weights for policy 0, policy_version 1854967 (0.0010) [2023-12-27 04:54:37,425][105692] Updated weights for policy 0, policy_version 1854977 (0.0010) [2023-12-27 04:54:37,477][105692] Updated weights for policy 0, policy_version 1854987 (0.0010) [2023-12-27 04:54:37,931][105620] Updated weights for policy 1, policy_version 1859452 (0.0007) [2023-12-27 04:54:37,993][105620] Updated weights for policy 1, policy_version 1859462 (0.0005) [2023-12-27 04:54:38,047][105620] Updated weights for policy 1, policy_version 1859472 (0.0005) [2023-12-27 04:54:38,297][105692] Updated weights for policy 0, policy_version 1854997 (0.0011) [2023-12-27 04:54:38,364][105692] Updated weights for policy 0, policy_version 1855007 (0.0011) [2023-12-27 04:54:38,424][105692] Updated weights for policy 0, policy_version 1855017 (0.0010) [2023-12-27 04:54:38,712][105620] Updated weights for policy 1, policy_version 1859482 (0.0006) [2023-12-27 04:54:38,778][105620] Updated weights for policy 1, policy_version 1859492 (0.0006) [2023-12-27 04:54:38,845][105620] Updated weights for policy 1, policy_version 1859502 (0.0006) [2023-12-27 04:54:38,909][105620] Updated weights for policy 1, policy_version 1859512 (0.0010) [2023-12-27 04:54:39,086][105692] Updated weights for policy 0, policy_version 1855027 (0.0009) [2023-12-27 04:54:39,131][105692] Updated weights for policy 0, policy_version 1855037 (0.0005) [2023-12-27 04:54:39,186][105692] Updated weights for policy 0, policy_version 1855047 (0.0006) [2023-12-27 04:54:39,486][105620] Updated weights for policy 1, policy_version 1859522 (0.0011) [2023-12-27 04:54:39,550][105620] Updated weights for policy 1, policy_version 1859532 (0.0011) [2023-12-27 04:54:39,607][105620] Updated weights for policy 1, policy_version 1859542 (0.0010) [2023-12-27 04:54:39,923][105692] Updated weights for policy 0, policy_version 1855057 (0.0009) [2023-12-27 04:54:39,985][105692] Updated weights for policy 0, policy_version 1855067 (0.0009) [2023-12-27 04:54:40,054][105692] Updated weights for policy 0, policy_version 1855077 (0.0009) [2023-12-27 04:54:40,119][105692] Updated weights for policy 0, policy_version 1855087 (0.0009) [2023-12-27 04:54:40,333][105620] Updated weights for policy 1, policy_version 1859552 (0.0007) [2023-12-27 04:54:40,398][105620] Updated weights for policy 1, policy_version 1859562 (0.0007) [2023-12-27 04:54:40,459][105620] Updated weights for policy 1, policy_version 1859572 (0.0009) [2023-12-27 04:54:40,856][105692] Updated weights for policy 0, policy_version 1855097 (0.0010) [2023-12-27 04:54:40,918][105692] Updated weights for policy 0, policy_version 1855107 (0.0010) [2023-12-27 04:54:40,981][105692] Updated weights for policy 0, policy_version 1855117 (0.0008) [2023-12-27 04:54:41,002][105620] Updated weights for policy 1, policy_version 1859582 (0.0008) [2023-12-27 04:54:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 951099392. Throughput: 0: 10055.7, 1: 10117.6. Samples: 951107964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:54:41,062][104569] Avg episode reward: [(0, '8989.021'), (1, '9254.329')] [2023-12-27 04:54:41,080][105620] Updated weights for policy 1, policy_version 1859592 (0.0012) [2023-12-27 04:54:41,146][105620] Updated weights for policy 1, policy_version 1859602 (0.0010) [2023-12-27 04:54:41,809][105692] Updated weights for policy 0, policy_version 1855127 (0.0009) [2023-12-27 04:54:41,866][105692] Updated weights for policy 0, policy_version 1855137 (0.0009) [2023-12-27 04:54:41,866][105620] Updated weights for policy 1, policy_version 1859612 (0.0010) [2023-12-27 04:54:41,917][105692] Updated weights for policy 0, policy_version 1855147 (0.0008) [2023-12-27 04:54:41,928][105620] Updated weights for policy 1, policy_version 1859622 (0.0008) [2023-12-27 04:54:41,988][105620] Updated weights for policy 1, policy_version 1859632 (0.0008) [2023-12-27 04:54:42,695][105692] Updated weights for policy 0, policy_version 1855157 (0.0008) [2023-12-27 04:54:42,700][105620] Updated weights for policy 1, policy_version 1859642 (0.0008) [2023-12-27 04:54:42,761][105692] Updated weights for policy 0, policy_version 1855167 (0.0008) [2023-12-27 04:54:42,764][105620] Updated weights for policy 1, policy_version 1859652 (0.0006) [2023-12-27 04:54:42,819][105692] Updated weights for policy 0, policy_version 1855177 (0.0008) [2023-12-27 04:54:42,826][105620] Updated weights for policy 1, policy_version 1859662 (0.0011) [2023-12-27 04:54:42,885][105620] Updated weights for policy 1, policy_version 1859672 (0.0011) [2023-12-27 04:54:43,533][105692] Updated weights for policy 0, policy_version 1855187 (0.0009) [2023-12-27 04:54:43,589][105692] Updated weights for policy 0, policy_version 1855197 (0.0006) [2023-12-27 04:54:43,602][105620] Updated weights for policy 1, policy_version 1859682 (0.0011) [2023-12-27 04:54:43,634][105692] Updated weights for policy 0, policy_version 1855207 (0.0005) [2023-12-27 04:54:43,648][105620] Updated weights for policy 1, policy_version 1859692 (0.0010) [2023-12-27 04:54:43,700][105620] Updated weights for policy 1, policy_version 1859702 (0.0010) [2023-12-27 04:54:44,322][105692] Updated weights for policy 0, policy_version 1855217 (0.0007) [2023-12-27 04:54:44,384][105692] Updated weights for policy 0, policy_version 1855227 (0.0006) [2023-12-27 04:54:44,413][105620] Updated weights for policy 1, policy_version 1859712 (0.0006) [2023-12-27 04:54:44,439][105692] Updated weights for policy 0, policy_version 1855237 (0.0005) [2023-12-27 04:54:44,471][105620] Updated weights for policy 1, policy_version 1859722 (0.0005) [2023-12-27 04:54:44,495][105692] Updated weights for policy 0, policy_version 1855247 (0.0005) [2023-12-27 04:54:44,530][105620] Updated weights for policy 1, policy_version 1859732 (0.0005) [2023-12-27 04:54:45,065][105692] Updated weights for policy 0, policy_version 1855257 (0.0006) [2023-12-27 04:54:45,106][105620] Updated weights for policy 1, policy_version 1859742 (0.0008) [2023-12-27 04:54:45,128][105692] Updated weights for policy 0, policy_version 1855267 (0.0006) [2023-12-27 04:54:45,173][105620] Updated weights for policy 1, policy_version 1859752 (0.0009) [2023-12-27 04:54:45,190][105692] Updated weights for policy 0, policy_version 1855277 (0.0006) [2023-12-27 04:54:45,240][105620] Updated weights for policy 1, policy_version 1859762 (0.0009) [2023-12-27 04:54:45,796][105692] Updated weights for policy 0, policy_version 1855287 (0.0008) [2023-12-27 04:54:45,844][105692] Updated weights for policy 0, policy_version 1855297 (0.0009) [2023-12-27 04:54:45,902][105692] Updated weights for policy 0, policy_version 1855307 (0.0009) [2023-12-27 04:54:46,023][105620] Updated weights for policy 1, policy_version 1859772 (0.0010) [2023-12-27 04:54:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 20070.4, 300 sec: 19577.5). Total num frames: 951197696. Throughput: 0: 9901.7, 1: 10022.8. Samples: 951163716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:54:46,063][104569] Avg episode reward: [(0, '8626.616'), (1, '9254.290')] [2023-12-27 04:54:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001855312_475029504.pth... [2023-12-27 04:54:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001854160_474734592.pth [2023-12-27 04:54:46,080][105620] Updated weights for policy 1, policy_version 1859782 (0.0009) [2023-12-27 04:54:46,136][105620] Updated weights for policy 1, policy_version 1859792 (0.0009) [2023-12-27 04:54:46,172][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001859800_476176384.pth... [2023-12-27 04:54:46,175][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001858616_475873280.pth [2023-12-27 04:54:46,574][105692] Updated weights for policy 0, policy_version 1855317 (0.0010) [2023-12-27 04:54:46,622][105692] Updated weights for policy 0, policy_version 1855327 (0.0010) [2023-12-27 04:54:46,670][105692] Updated weights for policy 0, policy_version 1855337 (0.0010) [2023-12-27 04:54:46,788][105620] Updated weights for policy 1, policy_version 1859802 (0.0008) [2023-12-27 04:54:46,840][105620] Updated weights for policy 1, policy_version 1859812 (0.0005) [2023-12-27 04:54:46,898][105620] Updated weights for policy 1, policy_version 1859822 (0.0008) [2023-12-27 04:54:46,948][105620] Updated weights for policy 1, policy_version 1859832 (0.0008) [2023-12-27 04:54:47,337][105692] Updated weights for policy 0, policy_version 1855347 (0.0010) [2023-12-27 04:54:47,398][105692] Updated weights for policy 0, policy_version 1855357 (0.0010) [2023-12-27 04:54:47,456][105692] Updated weights for policy 0, policy_version 1855367 (0.0010) [2023-12-27 04:54:47,721][105620] Updated weights for policy 1, policy_version 1859842 (0.0005) [2023-12-27 04:54:47,784][105620] Updated weights for policy 1, policy_version 1859852 (0.0005) [2023-12-27 04:54:47,850][105620] Updated weights for policy 1, policy_version 1859862 (0.0005) [2023-12-27 04:54:48,104][105692] Updated weights for policy 0, policy_version 1855377 (0.0010) [2023-12-27 04:54:48,163][105692] Updated weights for policy 0, policy_version 1855387 (0.0006) [2023-12-27 04:54:48,233][105692] Updated weights for policy 0, policy_version 1855397 (0.0009) [2023-12-27 04:54:48,292][105692] Updated weights for policy 0, policy_version 1855407 (0.0011) [2023-12-27 04:54:48,369][105620] Updated weights for policy 1, policy_version 1859872 (0.0007) [2023-12-27 04:54:48,436][105620] Updated weights for policy 1, policy_version 1859882 (0.0006) [2023-12-27 04:54:48,498][105620] Updated weights for policy 1, policy_version 1859892 (0.0008) [2023-12-27 04:54:49,004][105692] Updated weights for policy 0, policy_version 1855417 (0.0010) [2023-12-27 04:54:49,050][105692] Updated weights for policy 0, policy_version 1855427 (0.0006) [2023-12-27 04:54:49,099][105692] Updated weights for policy 0, policy_version 1855437 (0.0009) [2023-12-27 04:54:49,152][105620] Updated weights for policy 1, policy_version 1859902 (0.0006) [2023-12-27 04:54:49,213][105620] Updated weights for policy 1, policy_version 1859912 (0.0005) [2023-12-27 04:54:49,283][105620] Updated weights for policy 1, policy_version 1859922 (0.0009) [2023-12-27 04:54:49,859][105692] Updated weights for policy 0, policy_version 1855447 (0.0009) [2023-12-27 04:54:49,912][105692] Updated weights for policy 0, policy_version 1855457 (0.0010) [2023-12-27 04:54:49,975][105692] Updated weights for policy 0, policy_version 1855467 (0.0010) [2023-12-27 04:54:50,003][105620] Updated weights for policy 1, policy_version 1859932 (0.0009) [2023-12-27 04:54:50,061][105620] Updated weights for policy 1, policy_version 1859942 (0.0008) [2023-12-27 04:54:50,123][105620] Updated weights for policy 1, policy_version 1859952 (0.0008) [2023-12-27 04:54:50,703][105692] Updated weights for policy 0, policy_version 1855477 (0.0007) [2023-12-27 04:54:50,770][105692] Updated weights for policy 0, policy_version 1855487 (0.0008) [2023-12-27 04:54:50,832][105692] Updated weights for policy 0, policy_version 1855497 (0.0008) [2023-12-27 04:54:50,925][105620] Updated weights for policy 1, policy_version 1859962 (0.0008) [2023-12-27 04:54:50,979][105620] Updated weights for policy 1, policy_version 1859972 (0.0009) [2023-12-27 04:54:51,041][105620] Updated weights for policy 1, policy_version 1859982 (0.0009) [2023-12-27 04:54:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 20070.4, 300 sec: 19577.5). Total num frames: 951296000. Throughput: 0: 9956.9, 1: 10069.5. Samples: 951288092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:54:51,063][104569] Avg episode reward: [(0, '8539.731'), (1, '9346.729')] [2023-12-27 04:54:51,100][105620] Updated weights for policy 1, policy_version 1859992 (0.0009) [2023-12-27 04:54:51,532][105692] Updated weights for policy 0, policy_version 1855507 (0.0009) [2023-12-27 04:54:51,595][105692] Updated weights for policy 0, policy_version 1855517 (0.0009) [2023-12-27 04:54:51,663][105692] Updated weights for policy 0, policy_version 1855527 (0.0009) [2023-12-27 04:54:51,862][105620] Updated weights for policy 1, policy_version 1860002 (0.0008) [2023-12-27 04:54:51,913][105620] Updated weights for policy 1, policy_version 1860012 (0.0007) [2023-12-27 04:54:51,968][105620] Updated weights for policy 1, policy_version 1860022 (0.0005) [2023-12-27 04:54:52,502][105692] Updated weights for policy 0, policy_version 1855537 (0.0009) [2023-12-27 04:54:52,564][105692] Updated weights for policy 0, policy_version 1855548 (0.0009) [2023-12-27 04:54:52,608][105620] Updated weights for policy 1, policy_version 1860032 (0.0006) [2023-12-27 04:54:52,618][105692] Updated weights for policy 0, policy_version 1855558 (0.0008) [2023-12-27 04:54:52,669][105692] Updated weights for policy 0, policy_version 1855568 (0.0007) [2023-12-27 04:54:52,671][105620] Updated weights for policy 1, policy_version 1860042 (0.0007) [2023-12-27 04:54:52,736][105620] Updated weights for policy 1, policy_version 1860052 (0.0010) [2023-12-27 04:54:53,426][105692] Updated weights for policy 0, policy_version 1855578 (0.0006) [2023-12-27 04:54:53,472][105692] Updated weights for policy 0, policy_version 1855588 (0.0009) [2023-12-27 04:54:53,495][105620] Updated weights for policy 1, policy_version 1860062 (0.0008) [2023-12-27 04:54:53,533][105692] Updated weights for policy 0, policy_version 1855598 (0.0008) [2023-12-27 04:54:53,553][105620] Updated weights for policy 1, policy_version 1860072 (0.0007) [2023-12-27 04:54:53,614][105620] Updated weights for policy 1, policy_version 1860082 (0.0009) [2023-12-27 04:54:54,263][105692] Updated weights for policy 0, policy_version 1855608 (0.0006) [2023-12-27 04:54:54,330][105692] Updated weights for policy 0, policy_version 1855618 (0.0006) [2023-12-27 04:54:54,382][105692] Updated weights for policy 0, policy_version 1855628 (0.0007) [2023-12-27 04:54:54,386][105620] Updated weights for policy 1, policy_version 1860092 (0.0009) [2023-12-27 04:54:54,439][105620] Updated weights for policy 1, policy_version 1860102 (0.0008) [2023-12-27 04:54:54,496][105620] Updated weights for policy 1, policy_version 1860113 (0.0010) [2023-12-27 04:54:54,912][105692] Updated weights for policy 0, policy_version 1855638 (0.0006) [2023-12-27 04:54:54,969][105692] Updated weights for policy 0, policy_version 1855648 (0.0006) [2023-12-27 04:54:55,025][105692] Updated weights for policy 0, policy_version 1855658 (0.0006) [2023-12-27 04:54:55,371][105620] Updated weights for policy 1, policy_version 1860124 (0.0009) [2023-12-27 04:54:55,425][105620] Updated weights for policy 1, policy_version 1860134 (0.0009) [2023-12-27 04:54:55,472][105620] Updated weights for policy 1, policy_version 1860144 (0.0009) [2023-12-27 04:54:55,621][105692] Updated weights for policy 0, policy_version 1855668 (0.0005) [2023-12-27 04:54:55,686][105692] Updated weights for policy 0, policy_version 1855678 (0.0005) [2023-12-27 04:54:55,748][105692] Updated weights for policy 0, policy_version 1855688 (0.0008) [2023-12-27 04:54:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 20070.4, 300 sec: 19605.3). Total num frames: 951394304. Throughput: 0: 9911.9, 1: 9866.6. Samples: 951402236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:54:56,063][104569] Avg episode reward: [(0, '8081.284'), (1, '9346.723')] [2023-12-27 04:54:56,199][105620] Updated weights for policy 1, policy_version 1860154 (0.0009) [2023-12-27 04:54:56,247][105620] Updated weights for policy 1, policy_version 1860164 (0.0010) [2023-12-27 04:54:56,294][105620] Updated weights for policy 1, policy_version 1860174 (0.0010) [2023-12-27 04:54:56,341][105620] Updated weights for policy 1, policy_version 1860184 (0.0010) [2023-12-27 04:54:56,395][105692] Updated weights for policy 0, policy_version 1855698 (0.0009) [2023-12-27 04:54:56,446][105692] Updated weights for policy 0, policy_version 1855708 (0.0008) [2023-12-27 04:54:56,490][105692] Updated weights for policy 0, policy_version 1855718 (0.0008) [2023-12-27 04:54:56,544][105692] Updated weights for policy 0, policy_version 1855728 (0.0008) [2023-12-27 04:54:57,105][105620] Updated weights for policy 1, policy_version 1860194 (0.0010) [2023-12-27 04:54:57,152][105620] Updated weights for policy 1, policy_version 1860204 (0.0010) [2023-12-27 04:54:57,203][105620] Updated weights for policy 1, policy_version 1860214 (0.0010) [2023-12-27 04:54:57,291][105692] Updated weights for policy 0, policy_version 1855738 (0.0006) [2023-12-27 04:54:57,346][105692] Updated weights for policy 0, policy_version 1855748 (0.0011) [2023-12-27 04:54:57,398][105692] Updated weights for policy 0, policy_version 1855758 (0.0005) [2023-12-27 04:54:57,950][105620] Updated weights for policy 1, policy_version 1860224 (0.0010) [2023-12-27 04:54:58,011][105620] Updated weights for policy 1, policy_version 1860234 (0.0010) [2023-12-27 04:54:58,022][105692] Updated weights for policy 0, policy_version 1855768 (0.0006) [2023-12-27 04:54:58,062][105620] Updated weights for policy 1, policy_version 1860244 (0.0010) [2023-12-27 04:54:58,081][105692] Updated weights for policy 0, policy_version 1855778 (0.0006) [2023-12-27 04:54:58,130][105692] Updated weights for policy 0, policy_version 1855788 (0.0005) [2023-12-27 04:54:58,850][105620] Updated weights for policy 1, policy_version 1860254 (0.0009) [2023-12-27 04:54:58,866][105692] Updated weights for policy 0, policy_version 1855798 (0.0007) [2023-12-27 04:54:58,913][105620] Updated weights for policy 1, policy_version 1860264 (0.0008) [2023-12-27 04:54:58,934][105692] Updated weights for policy 0, policy_version 1855808 (0.0008) [2023-12-27 04:54:58,972][105620] Updated weights for policy 1, policy_version 1860274 (0.0007) [2023-12-27 04:54:58,996][105692] Updated weights for policy 0, policy_version 1855818 (0.0010) [2023-12-27 04:54:59,619][105620] Updated weights for policy 1, policy_version 1860284 (0.0008) [2023-12-27 04:54:59,677][105620] Updated weights for policy 1, policy_version 1860294 (0.0008) [2023-12-27 04:54:59,687][105692] Updated weights for policy 0, policy_version 1855828 (0.0008) [2023-12-27 04:54:59,737][105692] Updated weights for policy 0, policy_version 1855838 (0.0007) [2023-12-27 04:54:59,746][105620] Updated weights for policy 1, policy_version 1860304 (0.0006) [2023-12-27 04:54:59,793][105692] Updated weights for policy 0, policy_version 1855848 (0.0007) [2023-12-27 04:55:00,427][105620] Updated weights for policy 1, policy_version 1860314 (0.0008) [2023-12-27 04:55:00,482][105620] Updated weights for policy 1, policy_version 1860324 (0.0005) [2023-12-27 04:55:00,540][105620] Updated weights for policy 1, policy_version 1860334 (0.0005) [2023-12-27 04:55:00,575][105692] Updated weights for policy 0, policy_version 1855858 (0.0009) [2023-12-27 04:55:00,600][105620] Updated weights for policy 1, policy_version 1860344 (0.0006) [2023-12-27 04:55:00,631][105692] Updated weights for policy 0, policy_version 1855868 (0.0007) [2023-12-27 04:55:00,696][105692] Updated weights for policy 0, policy_version 1855878 (0.0008) [2023-12-27 04:55:00,750][105692] Updated weights for policy 0, policy_version 1855888 (0.0009) [2023-12-27 04:55:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19933.9, 300 sec: 19605.3). Total num frames: 951492608. Throughput: 0: 9909.3, 1: 9829.1. Samples: 951461720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:55:01,062][104569] Avg episode reward: [(0, '8352.735'), (1, '9346.678')] [2023-12-27 04:55:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001855888_475176960.pth... [2023-12-27 04:55:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001860344_476315648.pth... [2023-12-27 04:55:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001854736_474882048.pth [2023-12-27 04:55:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001859224_476028928.pth [2023-12-27 04:55:01,226][105620] Updated weights for policy 1, policy_version 1860354 (0.0005) [2023-12-27 04:55:01,290][105620] Updated weights for policy 1, policy_version 1860364 (0.0006) [2023-12-27 04:55:01,354][105620] Updated weights for policy 1, policy_version 1860374 (0.0008) [2023-12-27 04:55:01,501][105692] Updated weights for policy 0, policy_version 1855898 (0.0006) [2023-12-27 04:55:01,556][105692] Updated weights for policy 0, policy_version 1855908 (0.0005) [2023-12-27 04:55:01,621][105692] Updated weights for policy 0, policy_version 1855918 (0.0007) [2023-12-27 04:55:02,100][105620] Updated weights for policy 1, policy_version 1860385 (0.0009) [2023-12-27 04:55:02,154][105620] Updated weights for policy 1, policy_version 1860395 (0.0008) [2023-12-27 04:55:02,209][105620] Updated weights for policy 1, policy_version 1860405 (0.0006) [2023-12-27 04:55:02,332][105692] Updated weights for policy 0, policy_version 1855928 (0.0010) [2023-12-27 04:55:02,398][105692] Updated weights for policy 0, policy_version 1855938 (0.0011) [2023-12-27 04:55:02,455][105692] Updated weights for policy 0, policy_version 1855948 (0.0010) [2023-12-27 04:55:02,917][105620] Updated weights for policy 1, policy_version 1860415 (0.0006) [2023-12-27 04:55:02,974][105620] Updated weights for policy 1, policy_version 1860425 (0.0007) [2023-12-27 04:55:03,026][105620] Updated weights for policy 1, policy_version 1860435 (0.0005) [2023-12-27 04:55:03,215][105692] Updated weights for policy 0, policy_version 1855958 (0.0009) [2023-12-27 04:55:03,273][105692] Updated weights for policy 0, policy_version 1855968 (0.0009) [2023-12-27 04:55:03,334][105692] Updated weights for policy 0, policy_version 1855978 (0.0005) [2023-12-27 04:55:03,638][105620] Updated weights for policy 1, policy_version 1860445 (0.0008) [2023-12-27 04:55:03,692][105620] Updated weights for policy 1, policy_version 1860456 (0.0009) [2023-12-27 04:55:03,745][105620] Updated weights for policy 1, policy_version 1860468 (0.0010) [2023-12-27 04:55:03,944][105692] Updated weights for policy 0, policy_version 1855988 (0.0005) [2023-12-27 04:55:03,998][105692] Updated weights for policy 0, policy_version 1855998 (0.0005) [2023-12-27 04:55:04,049][105692] Updated weights for policy 0, policy_version 1856008 (0.0005) [2023-12-27 04:55:04,580][105620] Updated weights for policy 1, policy_version 1860478 (0.0008) [2023-12-27 04:55:04,625][105620] Updated weights for policy 1, policy_version 1860488 (0.0008) [2023-12-27 04:55:04,670][105620] Updated weights for policy 1, policy_version 1860498 (0.0007) [2023-12-27 04:55:04,791][105692] Updated weights for policy 0, policy_version 1856018 (0.0009) [2023-12-27 04:55:04,839][105692] Updated weights for policy 0, policy_version 1856028 (0.0010) [2023-12-27 04:55:04,887][105692] Updated weights for policy 0, policy_version 1856038 (0.0010) [2023-12-27 04:55:04,936][105692] Updated weights for policy 0, policy_version 1856048 (0.0008) [2023-12-27 04:55:05,410][105620] Updated weights for policy 1, policy_version 1860508 (0.0007) [2023-12-27 04:55:05,462][105620] Updated weights for policy 1, policy_version 1860518 (0.0005) [2023-12-27 04:55:05,508][105620] Updated weights for policy 1, policy_version 1860528 (0.0005) [2023-12-27 04:55:05,715][105692] Updated weights for policy 0, policy_version 1856058 (0.0011) [2023-12-27 04:55:05,774][105692] Updated weights for policy 0, policy_version 1856068 (0.0011) [2023-12-27 04:55:05,823][105692] Updated weights for policy 0, policy_version 1856078 (0.0010) [2023-12-27 04:55:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19605.3). Total num frames: 951590912. Throughput: 0: 9801.1, 1: 9786.5. Samples: 951579184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:55:06,062][104569] Avg episode reward: [(0, '8261.854'), (1, '9254.470')] [2023-12-27 04:55:06,108][105620] Updated weights for policy 1, policy_version 1860538 (0.0006) [2023-12-27 04:55:06,174][105620] Updated weights for policy 1, policy_version 1860548 (0.0008) [2023-12-27 04:55:06,230][105620] Updated weights for policy 1, policy_version 1860558 (0.0009) [2023-12-27 04:55:06,292][105620] Updated weights for policy 1, policy_version 1860568 (0.0010) [2023-12-27 04:55:06,571][105692] Updated weights for policy 0, policy_version 1856088 (0.0007) [2023-12-27 04:55:06,623][105692] Updated weights for policy 0, policy_version 1856098 (0.0006) [2023-12-27 04:55:06,688][105692] Updated weights for policy 0, policy_version 1856108 (0.0006) [2023-12-27 04:55:07,137][105620] Updated weights for policy 1, policy_version 1860578 (0.0010) [2023-12-27 04:55:07,200][105620] Updated weights for policy 1, policy_version 1860588 (0.0009) [2023-12-27 04:55:07,270][105620] Updated weights for policy 1, policy_version 1860598 (0.0009) [2023-12-27 04:55:07,285][105692] Updated weights for policy 0, policy_version 1856118 (0.0007) [2023-12-27 04:55:07,344][105692] Updated weights for policy 0, policy_version 1856128 (0.0009) [2023-12-27 04:55:07,403][105692] Updated weights for policy 0, policy_version 1856138 (0.0009) [2023-12-27 04:55:08,026][105620] Updated weights for policy 1, policy_version 1860608 (0.0009) [2023-12-27 04:55:08,085][105620] Updated weights for policy 1, policy_version 1860618 (0.0010) [2023-12-27 04:55:08,143][105620] Updated weights for policy 1, policy_version 1860628 (0.0010) [2023-12-27 04:55:08,177][105692] Updated weights for policy 0, policy_version 1856148 (0.0009) [2023-12-27 04:55:08,234][105692] Updated weights for policy 0, policy_version 1856158 (0.0008) [2023-12-27 04:55:08,282][105692] Updated weights for policy 0, policy_version 1856168 (0.0006) [2023-12-27 04:55:08,895][105620] Updated weights for policy 1, policy_version 1860638 (0.0010) [2023-12-27 04:55:08,946][105620] Updated weights for policy 1, policy_version 1860648 (0.0010) [2023-12-27 04:55:08,980][105692] Updated weights for policy 0, policy_version 1856178 (0.0008) [2023-12-27 04:55:09,005][105620] Updated weights for policy 1, policy_version 1860658 (0.0010) [2023-12-27 04:55:09,036][105692] Updated weights for policy 0, policy_version 1856188 (0.0007) [2023-12-27 04:55:09,091][105692] Updated weights for policy 0, policy_version 1856198 (0.0009) [2023-12-27 04:55:09,148][105692] Updated weights for policy 0, policy_version 1856208 (0.0010) [2023-12-27 04:55:09,804][105620] Updated weights for policy 1, policy_version 1860668 (0.0009) [2023-12-27 04:55:09,874][105620] Updated weights for policy 1, policy_version 1860678 (0.0011) [2023-12-27 04:55:09,946][105620] Updated weights for policy 1, policy_version 1860688 (0.0011) [2023-12-27 04:55:09,993][105692] Updated weights for policy 0, policy_version 1856218 (0.0006) [2023-12-27 04:55:10,050][105692] Updated weights for policy 0, policy_version 1856228 (0.0008) [2023-12-27 04:55:10,106][105692] Updated weights for policy 0, policy_version 1856238 (0.0008) [2023-12-27 04:55:10,685][105620] Updated weights for policy 1, policy_version 1860698 (0.0010) [2023-12-27 04:55:10,739][105620] Updated weights for policy 1, policy_version 1860708 (0.0009) [2023-12-27 04:55:10,791][105620] Updated weights for policy 1, policy_version 1860718 (0.0010) [2023-12-27 04:55:10,853][105620] Updated weights for policy 1, policy_version 1860728 (0.0010) [2023-12-27 04:55:10,885][105692] Updated weights for policy 0, policy_version 1856248 (0.0007) [2023-12-27 04:55:10,938][105692] Updated weights for policy 0, policy_version 1856258 (0.0009) [2023-12-27 04:55:10,987][105692] Updated weights for policy 0, policy_version 1856268 (0.0007) [2023-12-27 04:55:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 951689216. Throughput: 0: 9814.0, 1: 9735.6. Samples: 951692188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:55:11,062][104569] Avg episode reward: [(0, '7988.796'), (1, '9254.497')] [2023-12-27 04:55:11,601][105620] Updated weights for policy 1, policy_version 1860738 (0.0006) [2023-12-27 04:55:11,666][105620] Updated weights for policy 1, policy_version 1860748 (0.0009) [2023-12-27 04:55:11,721][105620] Updated weights for policy 1, policy_version 1860758 (0.0009) [2023-12-27 04:55:11,796][105692] Updated weights for policy 0, policy_version 1856278 (0.0008) [2023-12-27 04:55:11,845][105692] Updated weights for policy 0, policy_version 1856288 (0.0009) [2023-12-27 04:55:11,894][105692] Updated weights for policy 0, policy_version 1856298 (0.0009) [2023-12-27 04:55:12,424][105620] Updated weights for policy 1, policy_version 1860768 (0.0010) [2023-12-27 04:55:12,486][105620] Updated weights for policy 1, policy_version 1860778 (0.0009) [2023-12-27 04:55:12,551][105620] Updated weights for policy 1, policy_version 1860788 (0.0008) [2023-12-27 04:55:12,635][105692] Updated weights for policy 0, policy_version 1856308 (0.0009) [2023-12-27 04:55:12,697][105692] Updated weights for policy 0, policy_version 1856318 (0.0010) [2023-12-27 04:55:12,752][105692] Updated weights for policy 0, policy_version 1856328 (0.0010) [2023-12-27 04:55:13,235][105620] Updated weights for policy 1, policy_version 1860798 (0.0009) [2023-12-27 04:55:13,284][105620] Updated weights for policy 1, policy_version 1860808 (0.0008) [2023-12-27 04:55:13,328][105620] Updated weights for policy 1, policy_version 1860818 (0.0008) [2023-12-27 04:55:13,499][105692] Updated weights for policy 0, policy_version 1856338 (0.0010) [2023-12-27 04:55:13,561][105692] Updated weights for policy 0, policy_version 1856348 (0.0009) [2023-12-27 04:55:13,628][105692] Updated weights for policy 0, policy_version 1856358 (0.0010) [2023-12-27 04:55:13,692][105692] Updated weights for policy 0, policy_version 1856368 (0.0010) [2023-12-27 04:55:14,117][105620] Updated weights for policy 1, policy_version 1860828 (0.0009) [2023-12-27 04:55:14,164][105620] Updated weights for policy 1, policy_version 1860838 (0.0010) [2023-12-27 04:55:14,222][105620] Updated weights for policy 1, policy_version 1860848 (0.0010) [2023-12-27 04:55:14,387][105692] Updated weights for policy 0, policy_version 1856378 (0.0005) [2023-12-27 04:55:14,436][105692] Updated weights for policy 0, policy_version 1856388 (0.0007) [2023-12-27 04:55:14,491][105692] Updated weights for policy 0, policy_version 1856398 (0.0010) [2023-12-27 04:55:14,973][105620] Updated weights for policy 1, policy_version 1860858 (0.0011) [2023-12-27 04:55:15,026][105620] Updated weights for policy 1, policy_version 1860868 (0.0011) [2023-12-27 04:55:15,079][105620] Updated weights for policy 1, policy_version 1860878 (0.0011) [2023-12-27 04:55:15,131][105620] Updated weights for policy 1, policy_version 1860888 (0.0010) [2023-12-27 04:55:15,217][105692] Updated weights for policy 0, policy_version 1856408 (0.0010) [2023-12-27 04:55:15,275][105692] Updated weights for policy 0, policy_version 1856418 (0.0010) [2023-12-27 04:55:15,334][105692] Updated weights for policy 0, policy_version 1856428 (0.0010) [2023-12-27 04:55:15,854][105620] Updated weights for policy 1, policy_version 1860898 (0.0007) [2023-12-27 04:55:15,911][105620] Updated weights for policy 1, policy_version 1860908 (0.0009) [2023-12-27 04:55:15,962][105620] Updated weights for policy 1, policy_version 1860918 (0.0005) [2023-12-27 04:55:16,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 951779328. Throughput: 0: 9786.0, 1: 9669.4. Samples: 951749212. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:55:16,063][104569] Avg episode reward: [(0, '8442.701'), (1, '9254.307')] [2023-12-27 04:55:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001860920_476463104.pth... [2023-12-27 04:55:16,069][105692] Updated weights for policy 0, policy_version 1856438 (0.0010) [2023-12-27 04:55:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001859800_476176384.pth [2023-12-27 04:55:16,118][105692] Updated weights for policy 0, policy_version 1856448 (0.0010) [2023-12-27 04:55:16,167][105692] Updated weights for policy 0, policy_version 1856458 (0.0010) [2023-12-27 04:55:16,202][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001856464_475324416.pth... [2023-12-27 04:55:16,207][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001855312_475029504.pth [2023-12-27 04:55:16,511][105620] Updated weights for policy 1, policy_version 1860928 (0.0005) [2023-12-27 04:55:16,560][105620] Updated weights for policy 1, policy_version 1860938 (0.0005) [2023-12-27 04:55:16,609][105620] Updated weights for policy 1, policy_version 1860948 (0.0005) [2023-12-27 04:55:16,949][105692] Updated weights for policy 0, policy_version 1856468 (0.0010) [2023-12-27 04:55:17,001][105692] Updated weights for policy 0, policy_version 1856478 (0.0010) [2023-12-27 04:55:17,056][105692] Updated weights for policy 0, policy_version 1856488 (0.0008) [2023-12-27 04:55:17,295][105620] Updated weights for policy 1, policy_version 1860958 (0.0007) [2023-12-27 04:55:17,357][105620] Updated weights for policy 1, policy_version 1860968 (0.0008) [2023-12-27 04:55:17,416][105620] Updated weights for policy 1, policy_version 1860978 (0.0008) [2023-12-27 04:55:17,793][105692] Updated weights for policy 0, policy_version 1856498 (0.0010) [2023-12-27 04:55:17,861][105692] Updated weights for policy 0, policy_version 1856508 (0.0010) [2023-12-27 04:55:17,923][105692] Updated weights for policy 0, policy_version 1856518 (0.0010) [2023-12-27 04:55:17,991][105692] Updated weights for policy 0, policy_version 1856528 (0.0010) [2023-12-27 04:55:18,008][105620] Updated weights for policy 1, policy_version 1860988 (0.0007) [2023-12-27 04:55:18,060][105620] Updated weights for policy 1, policy_version 1860998 (0.0008) [2023-12-27 04:55:18,116][105620] Updated weights for policy 1, policy_version 1861008 (0.0008) [2023-12-27 04:55:18,731][105620] Updated weights for policy 1, policy_version 1861018 (0.0006) [2023-12-27 04:55:18,733][105692] Updated weights for policy 0, policy_version 1856538 (0.0010) [2023-12-27 04:55:18,794][105620] Updated weights for policy 1, policy_version 1861028 (0.0005) [2023-12-27 04:55:18,795][105692] Updated weights for policy 0, policy_version 1856548 (0.0010) [2023-12-27 04:55:18,853][105620] Updated weights for policy 1, policy_version 1861038 (0.0008) [2023-12-27 04:55:18,858][105692] Updated weights for policy 0, policy_version 1856558 (0.0008) [2023-12-27 04:55:18,918][105620] Updated weights for policy 1, policy_version 1861048 (0.0009) [2023-12-27 04:55:19,535][105692] Updated weights for policy 0, policy_version 1856568 (0.0008) [2023-12-27 04:55:19,595][105692] Updated weights for policy 0, policy_version 1856578 (0.0008) [2023-12-27 04:55:19,654][105620] Updated weights for policy 1, policy_version 1861058 (0.0006) [2023-12-27 04:55:19,655][105692] Updated weights for policy 0, policy_version 1856588 (0.0008) [2023-12-27 04:55:19,714][105620] Updated weights for policy 1, policy_version 1861068 (0.0006) [2023-12-27 04:55:19,781][105620] Updated weights for policy 1, policy_version 1861078 (0.0006) [2023-12-27 04:55:20,419][105692] Updated weights for policy 0, policy_version 1856598 (0.0008) [2023-12-27 04:55:20,449][105620] Updated weights for policy 1, policy_version 1861088 (0.0009) [2023-12-27 04:55:20,470][105692] Updated weights for policy 0, policy_version 1856608 (0.0008) [2023-12-27 04:55:20,504][105620] Updated weights for policy 1, policy_version 1861098 (0.0006) [2023-12-27 04:55:20,522][105692] Updated weights for policy 0, policy_version 1856618 (0.0007) [2023-12-27 04:55:20,565][105620] Updated weights for policy 1, policy_version 1861108 (0.0007) [2023-12-27 04:55:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 951877632. Throughput: 0: 9722.2, 1: 9800.3. Samples: 951869124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:55:21,063][104569] Avg episode reward: [(0, '8530.284'), (1, '9254.177')] [2023-12-27 04:55:21,286][105620] Updated weights for policy 1, policy_version 1861118 (0.0009) [2023-12-27 04:55:21,286][105692] Updated weights for policy 0, policy_version 1856628 (0.0009) [2023-12-27 04:55:21,352][105620] Updated weights for policy 1, policy_version 1861128 (0.0008) [2023-12-27 04:55:21,353][105692] Updated weights for policy 0, policy_version 1856638 (0.0009) [2023-12-27 04:55:21,419][105620] Updated weights for policy 1, policy_version 1861138 (0.0009) [2023-12-27 04:55:21,419][105692] Updated weights for policy 0, policy_version 1856648 (0.0010) [2023-12-27 04:55:22,184][105620] Updated weights for policy 1, policy_version 1861148 (0.0008) [2023-12-27 04:55:22,186][105692] Updated weights for policy 0, policy_version 1856658 (0.0009) [2023-12-27 04:55:22,248][105620] Updated weights for policy 1, policy_version 1861158 (0.0011) [2023-12-27 04:55:22,254][105692] Updated weights for policy 0, policy_version 1856668 (0.0006) [2023-12-27 04:55:22,312][105620] Updated weights for policy 1, policy_version 1861168 (0.0011) [2023-12-27 04:55:22,321][105692] Updated weights for policy 0, policy_version 1856678 (0.0006) [2023-12-27 04:55:22,388][105692] Updated weights for policy 0, policy_version 1856688 (0.0008) [2023-12-27 04:55:22,994][105692] Updated weights for policy 0, policy_version 1856698 (0.0008) [2023-12-27 04:55:23,056][105692] Updated weights for policy 0, policy_version 1856708 (0.0008) [2023-12-27 04:55:23,092][105620] Updated weights for policy 1, policy_version 1861178 (0.0010) [2023-12-27 04:55:23,122][105692] Updated weights for policy 0, policy_version 1856718 (0.0008) [2023-12-27 04:55:23,154][105620] Updated weights for policy 1, policy_version 1861188 (0.0007) [2023-12-27 04:55:23,226][105620] Updated weights for policy 1, policy_version 1861198 (0.0010) [2023-12-27 04:55:23,295][105620] Updated weights for policy 1, policy_version 1861208 (0.0011) [2023-12-27 04:55:23,874][105692] Updated weights for policy 0, policy_version 1856728 (0.0009) [2023-12-27 04:55:23,937][105692] Updated weights for policy 0, policy_version 1856738 (0.0008) [2023-12-27 04:55:23,973][105620] Updated weights for policy 1, policy_version 1861218 (0.0005) [2023-12-27 04:55:23,989][105692] Updated weights for policy 0, policy_version 1856748 (0.0008) [2023-12-27 04:55:24,028][105620] Updated weights for policy 1, policy_version 1861228 (0.0005) [2023-12-27 04:55:24,086][105620] Updated weights for policy 1, policy_version 1861238 (0.0005) [2023-12-27 04:55:24,626][105620] Updated weights for policy 1, policy_version 1861248 (0.0006) [2023-12-27 04:55:24,675][105620] Updated weights for policy 1, policy_version 1861258 (0.0005) [2023-12-27 04:55:24,735][105620] Updated weights for policy 1, policy_version 1861268 (0.0005) [2023-12-27 04:55:24,821][105692] Updated weights for policy 0, policy_version 1856758 (0.0007) [2023-12-27 04:55:24,876][105692] Updated weights for policy 0, policy_version 1856768 (0.0007) [2023-12-27 04:55:24,938][105692] Updated weights for policy 0, policy_version 1856778 (0.0008) [2023-12-27 04:55:25,435][105620] Updated weights for policy 1, policy_version 1861278 (0.0011) [2023-12-27 04:55:25,497][105620] Updated weights for policy 1, policy_version 1861288 (0.0011) [2023-12-27 04:55:25,559][105620] Updated weights for policy 1, policy_version 1861298 (0.0010) [2023-12-27 04:55:25,684][105692] Updated weights for policy 0, policy_version 1856788 (0.0008) [2023-12-27 04:55:25,732][105692] Updated weights for policy 0, policy_version 1856798 (0.0008) [2023-12-27 04:55:25,775][105692] Updated weights for policy 0, policy_version 1856808 (0.0008) [2023-12-27 04:55:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 951975936. Throughput: 0: 9696.8, 1: 9768.7. Samples: 951983912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:55:26,062][104569] Avg episode reward: [(0, '8166.413'), (1, '9346.392')] [2023-12-27 04:55:26,216][105620] Updated weights for policy 1, policy_version 1861308 (0.0010) [2023-12-27 04:55:26,284][105620] Updated weights for policy 1, policy_version 1861318 (0.0010) [2023-12-27 04:55:26,342][105620] Updated weights for policy 1, policy_version 1861328 (0.0010) [2023-12-27 04:55:26,489][105692] Updated weights for policy 0, policy_version 1856818 (0.0008) [2023-12-27 04:55:26,548][105692] Updated weights for policy 0, policy_version 1856828 (0.0007) [2023-12-27 04:55:26,607][105692] Updated weights for policy 0, policy_version 1856838 (0.0008) [2023-12-27 04:55:26,663][105692] Updated weights for policy 0, policy_version 1856848 (0.0008) [2023-12-27 04:55:27,069][105620] Updated weights for policy 1, policy_version 1861338 (0.0011) [2023-12-27 04:55:27,120][105620] Updated weights for policy 1, policy_version 1861348 (0.0010) [2023-12-27 04:55:27,190][105620] Updated weights for policy 1, policy_version 1861358 (0.0010) [2023-12-27 04:55:27,253][105620] Updated weights for policy 1, policy_version 1861368 (0.0010) [2023-12-27 04:55:27,255][105692] Updated weights for policy 0, policy_version 1856858 (0.0007) [2023-12-27 04:55:27,314][105692] Updated weights for policy 0, policy_version 1856868 (0.0008) [2023-12-27 04:55:27,372][105692] Updated weights for policy 0, policy_version 1856878 (0.0008) [2023-12-27 04:55:27,978][105620] Updated weights for policy 1, policy_version 1861378 (0.0011) [2023-12-27 04:55:28,035][105620] Updated weights for policy 1, policy_version 1861388 (0.0010) [2023-12-27 04:55:28,091][105620] Updated weights for policy 1, policy_version 1861398 (0.0010) [2023-12-27 04:55:28,096][105692] Updated weights for policy 0, policy_version 1856888 (0.0008) [2023-12-27 04:55:28,148][105692] Updated weights for policy 0, policy_version 1856898 (0.0006) [2023-12-27 04:55:28,200][105692] Updated weights for policy 0, policy_version 1856908 (0.0007) [2023-12-27 04:55:28,820][105620] Updated weights for policy 1, policy_version 1861408 (0.0010) [2023-12-27 04:55:28,871][105620] Updated weights for policy 1, policy_version 1861418 (0.0010) [2023-12-27 04:55:28,918][105692] Updated weights for policy 0, policy_version 1856918 (0.0006) [2023-12-27 04:55:28,921][105620] Updated weights for policy 1, policy_version 1861428 (0.0010) [2023-12-27 04:55:28,971][105692] Updated weights for policy 0, policy_version 1856928 (0.0007) [2023-12-27 04:55:29,026][105692] Updated weights for policy 0, policy_version 1856938 (0.0010) [2023-12-27 04:55:29,712][105620] Updated weights for policy 1, policy_version 1861438 (0.0010) [2023-12-27 04:55:29,760][105620] Updated weights for policy 1, policy_version 1861448 (0.0010) [2023-12-27 04:55:29,783][105692] Updated weights for policy 0, policy_version 1856948 (0.0009) [2023-12-27 04:55:29,812][105620] Updated weights for policy 1, policy_version 1861458 (0.0010) [2023-12-27 04:55:29,842][105692] Updated weights for policy 0, policy_version 1856958 (0.0006) [2023-12-27 04:55:29,898][105692] Updated weights for policy 0, policy_version 1856968 (0.0008) [2023-12-27 04:55:30,579][105620] Updated weights for policy 1, policy_version 1861468 (0.0009) [2023-12-27 04:55:30,635][105620] Updated weights for policy 1, policy_version 1861478 (0.0011) [2023-12-27 04:55:30,649][105692] Updated weights for policy 0, policy_version 1856978 (0.0008) [2023-12-27 04:55:30,697][105620] Updated weights for policy 1, policy_version 1861488 (0.0009) [2023-12-27 04:55:30,704][105692] Updated weights for policy 0, policy_version 1856988 (0.0008) [2023-12-27 04:55:30,763][105692] Updated weights for policy 0, policy_version 1856998 (0.0009) [2023-12-27 04:55:30,825][105692] Updated weights for policy 0, policy_version 1857008 (0.0009) [2023-12-27 04:55:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19605.3). Total num frames: 952074240. Throughput: 0: 9761.2, 1: 9780.5. Samples: 952043092. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:55:31,063][104569] Avg episode reward: [(0, '7985.391'), (1, '9254.518')] [2023-12-27 04:55:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001857008_475463680.pth... [2023-12-27 04:55:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001861496_476610560.pth... [2023-12-27 04:55:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001860344_476315648.pth [2023-12-27 04:55:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001855888_475176960.pth [2023-12-27 04:55:31,428][105620] Updated weights for policy 1, policy_version 1861498 (0.0006) [2023-12-27 04:55:31,490][105620] Updated weights for policy 1, policy_version 1861508 (0.0010) [2023-12-27 04:55:31,546][105620] Updated weights for policy 1, policy_version 1861518 (0.0007) [2023-12-27 04:55:31,559][105692] Updated weights for policy 0, policy_version 1857018 (0.0008) [2023-12-27 04:55:31,608][105620] Updated weights for policy 1, policy_version 1861528 (0.0009) [2023-12-27 04:55:31,620][105692] Updated weights for policy 0, policy_version 1857028 (0.0007) [2023-12-27 04:55:31,682][105692] Updated weights for policy 0, policy_version 1857038 (0.0008) [2023-12-27 04:55:32,389][105620] Updated weights for policy 1, policy_version 1861538 (0.0008) [2023-12-27 04:55:32,423][105692] Updated weights for policy 0, policy_version 1857048 (0.0007) [2023-12-27 04:55:32,449][105620] Updated weights for policy 1, policy_version 1861548 (0.0008) [2023-12-27 04:55:32,480][105692] Updated weights for policy 0, policy_version 1857058 (0.0005) [2023-12-27 04:55:32,512][105620] Updated weights for policy 1, policy_version 1861558 (0.0009) [2023-12-27 04:55:32,544][105692] Updated weights for policy 0, policy_version 1857068 (0.0005) [2023-12-27 04:55:33,178][105692] Updated weights for policy 0, policy_version 1857078 (0.0008) [2023-12-27 04:55:33,236][105692] Updated weights for policy 0, policy_version 1857088 (0.0009) [2023-12-27 04:55:33,286][105692] Updated weights for policy 0, policy_version 1857098 (0.0007) [2023-12-27 04:55:33,300][105620] Updated weights for policy 1, policy_version 1861568 (0.0008) [2023-12-27 04:55:33,347][105620] Updated weights for policy 1, policy_version 1861578 (0.0007) [2023-12-27 04:55:33,397][105620] Updated weights for policy 1, policy_version 1861588 (0.0009) [2023-12-27 04:55:34,016][105692] Updated weights for policy 0, policy_version 1857108 (0.0008) [2023-12-27 04:55:34,066][105692] Updated weights for policy 0, policy_version 1857118 (0.0009) [2023-12-27 04:55:34,110][105620] Updated weights for policy 1, policy_version 1861598 (0.0008) [2023-12-27 04:55:34,112][105692] Updated weights for policy 0, policy_version 1857128 (0.0007) [2023-12-27 04:55:34,172][105620] Updated weights for policy 1, policy_version 1861608 (0.0007) [2023-12-27 04:55:34,239][105620] Updated weights for policy 1, policy_version 1861618 (0.0006) [2023-12-27 04:55:34,849][105620] Updated weights for policy 1, policy_version 1861628 (0.0006) [2023-12-27 04:55:34,895][105620] Updated weights for policy 1, policy_version 1861638 (0.0008) [2023-12-27 04:55:34,935][105692] Updated weights for policy 0, policy_version 1857138 (0.0007) [2023-12-27 04:55:34,946][105620] Updated weights for policy 1, policy_version 1861648 (0.0007) [2023-12-27 04:55:34,992][105692] Updated weights for policy 0, policy_version 1857148 (0.0006) [2023-12-27 04:55:35,053][105692] Updated weights for policy 0, policy_version 1857158 (0.0007) [2023-12-27 04:55:35,117][105692] Updated weights for policy 0, policy_version 1857168 (0.0007) [2023-12-27 04:55:35,556][105620] Updated weights for policy 1, policy_version 1861658 (0.0007) [2023-12-27 04:55:35,607][105620] Updated weights for policy 1, policy_version 1861668 (0.0009) [2023-12-27 04:55:35,660][105620] Updated weights for policy 1, policy_version 1861678 (0.0008) [2023-12-27 04:55:35,714][105620] Updated weights for policy 1, policy_version 1861688 (0.0007) [2023-12-27 04:55:35,918][105692] Updated weights for policy 0, policy_version 1857178 (0.0009) [2023-12-27 04:55:35,981][105692] Updated weights for policy 0, policy_version 1857188 (0.0009) [2023-12-27 04:55:36,040][105692] Updated weights for policy 0, policy_version 1857198 (0.0009) [2023-12-27 04:55:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 952172544. Throughput: 0: 9645.0, 1: 9673.3. Samples: 952157412. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:55:36,062][104569] Avg episode reward: [(0, '8349.441'), (1, '9254.526')] [2023-12-27 04:55:36,377][105620] Updated weights for policy 1, policy_version 1861698 (0.0005) [2023-12-27 04:55:36,447][105620] Updated weights for policy 1, policy_version 1861708 (0.0006) [2023-12-27 04:55:36,509][105620] Updated weights for policy 1, policy_version 1861718 (0.0009) [2023-12-27 04:55:36,863][105692] Updated weights for policy 0, policy_version 1857208 (0.0010) [2023-12-27 04:55:36,918][105692] Updated weights for policy 0, policy_version 1857218 (0.0009) [2023-12-27 04:55:36,983][105692] Updated weights for policy 0, policy_version 1857228 (0.0008) [2023-12-27 04:55:37,163][105620] Updated weights for policy 1, policy_version 1861728 (0.0009) [2023-12-27 04:55:37,217][105620] Updated weights for policy 1, policy_version 1861738 (0.0010) [2023-12-27 04:55:37,272][105620] Updated weights for policy 1, policy_version 1861748 (0.0008) [2023-12-27 04:55:37,754][105692] Updated weights for policy 0, policy_version 1857238 (0.0009) [2023-12-27 04:55:37,814][105692] Updated weights for policy 0, policy_version 1857248 (0.0009) [2023-12-27 04:55:37,871][105692] Updated weights for policy 0, policy_version 1857258 (0.0009) [2023-12-27 04:55:38,040][105620] Updated weights for policy 1, policy_version 1861758 (0.0009) [2023-12-27 04:55:38,101][105620] Updated weights for policy 1, policy_version 1861768 (0.0009) [2023-12-27 04:55:38,160][105620] Updated weights for policy 1, policy_version 1861778 (0.0009) [2023-12-27 04:55:38,735][105692] Updated weights for policy 0, policy_version 1857268 (0.0009) [2023-12-27 04:55:38,786][105692] Updated weights for policy 0, policy_version 1857278 (0.0009) [2023-12-27 04:55:38,830][105620] Updated weights for policy 1, policy_version 1861788 (0.0007) [2023-12-27 04:55:38,840][105692] Updated weights for policy 0, policy_version 1857288 (0.0008) [2023-12-27 04:55:38,879][105620] Updated weights for policy 1, policy_version 1861798 (0.0007) [2023-12-27 04:55:38,935][105620] Updated weights for policy 1, policy_version 1861808 (0.0009) [2023-12-27 04:55:39,586][105692] Updated weights for policy 0, policy_version 1857298 (0.0006) [2023-12-27 04:55:39,648][105692] Updated weights for policy 0, policy_version 1857308 (0.0008) [2023-12-27 04:55:39,723][105692] Updated weights for policy 0, policy_version 1857318 (0.0007) [2023-12-27 04:55:39,729][105620] Updated weights for policy 1, policy_version 1861818 (0.0009) [2023-12-27 04:55:39,786][105692] Updated weights for policy 0, policy_version 1857328 (0.0010) [2023-12-27 04:55:39,797][105620] Updated weights for policy 1, policy_version 1861828 (0.0006) [2023-12-27 04:55:39,862][105620] Updated weights for policy 1, policy_version 1861838 (0.0008) [2023-12-27 04:55:39,922][105620] Updated weights for policy 1, policy_version 1861848 (0.0008) [2023-12-27 04:55:40,481][105692] Updated weights for policy 0, policy_version 1857338 (0.0011) [2023-12-27 04:55:40,551][105692] Updated weights for policy 0, policy_version 1857348 (0.0010) [2023-12-27 04:55:40,585][105620] Updated weights for policy 1, policy_version 1861858 (0.0006) [2023-12-27 04:55:40,602][105692] Updated weights for policy 0, policy_version 1857358 (0.0010) [2023-12-27 04:55:40,640][105620] Updated weights for policy 1, policy_version 1861868 (0.0008) [2023-12-27 04:55:40,692][105620] Updated weights for policy 1, policy_version 1861878 (0.0008) [2023-12-27 04:55:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 952262656. Throughput: 0: 9520.4, 1: 9801.7. Samples: 952271728. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:55:41,063][104569] Avg episode reward: [(0, '8170.636'), (1, '9346.400')] [2023-12-27 04:55:41,367][105692] Updated weights for policy 0, policy_version 1857368 (0.0007) [2023-12-27 04:55:41,434][105692] Updated weights for policy 0, policy_version 1857378 (0.0011) [2023-12-27 04:55:41,500][105692] Updated weights for policy 0, policy_version 1857388 (0.0011) [2023-12-27 04:55:41,539][105620] Updated weights for policy 1, policy_version 1861888 (0.0007) [2023-12-27 04:55:41,595][105620] Updated weights for policy 1, policy_version 1861898 (0.0008) [2023-12-27 04:55:41,659][105620] Updated weights for policy 1, policy_version 1861908 (0.0008) [2023-12-27 04:55:42,258][105692] Updated weights for policy 0, policy_version 1857398 (0.0010) [2023-12-27 04:55:42,318][105692] Updated weights for policy 0, policy_version 1857408 (0.0008) [2023-12-27 04:55:42,377][105620] Updated weights for policy 1, policy_version 1861918 (0.0009) [2023-12-27 04:55:42,387][105692] Updated weights for policy 0, policy_version 1857418 (0.0008) [2023-12-27 04:55:42,434][105620] Updated weights for policy 1, policy_version 1861928 (0.0009) [2023-12-27 04:55:42,498][105620] Updated weights for policy 1, policy_version 1861938 (0.0008) [2023-12-27 04:55:43,107][105692] Updated weights for policy 0, policy_version 1857428 (0.0008) [2023-12-27 04:55:43,162][105692] Updated weights for policy 0, policy_version 1857438 (0.0009) [2023-12-27 04:55:43,210][105692] Updated weights for policy 0, policy_version 1857448 (0.0009) [2023-12-27 04:55:43,229][105620] Updated weights for policy 1, policy_version 1861948 (0.0008) [2023-12-27 04:55:43,287][105620] Updated weights for policy 1, policy_version 1861958 (0.0006) [2023-12-27 04:55:43,354][105620] Updated weights for policy 1, policy_version 1861968 (0.0005) [2023-12-27 04:55:43,961][105620] Updated weights for policy 1, policy_version 1861978 (0.0006) [2023-12-27 04:55:43,999][105692] Updated weights for policy 0, policy_version 1857458 (0.0008) [2023-12-27 04:55:44,021][105620] Updated weights for policy 1, policy_version 1861988 (0.0009) [2023-12-27 04:55:44,048][105692] Updated weights for policy 0, policy_version 1857468 (0.0007) [2023-12-27 04:55:44,068][105620] Updated weights for policy 1, policy_version 1861998 (0.0007) [2023-12-27 04:55:44,102][105692] Updated weights for policy 0, policy_version 1857478 (0.0006) [2023-12-27 04:55:44,127][105620] Updated weights for policy 1, policy_version 1862008 (0.0008) [2023-12-27 04:55:44,160][105692] Updated weights for policy 0, policy_version 1857488 (0.0006) [2023-12-27 04:55:44,738][105692] Updated weights for policy 0, policy_version 1857498 (0.0005) [2023-12-27 04:55:44,803][105692] Updated weights for policy 0, policy_version 1857510 (0.0007) [2023-12-27 04:55:44,832][105620] Updated weights for policy 1, policy_version 1862018 (0.0008) [2023-12-27 04:55:44,869][105692] Updated weights for policy 0, policy_version 1857520 (0.0007) [2023-12-27 04:55:44,893][105620] Updated weights for policy 1, policy_version 1862028 (0.0008) [2023-12-27 04:55:44,958][105620] Updated weights for policy 1, policy_version 1862038 (0.0007) [2023-12-27 04:55:45,574][105692] Updated weights for policy 0, policy_version 1857530 (0.0009) [2023-12-27 04:55:45,639][105692] Updated weights for policy 0, policy_version 1857540 (0.0008) [2023-12-27 04:55:45,661][105620] Updated weights for policy 1, policy_version 1862048 (0.0007) [2023-12-27 04:55:45,695][105692] Updated weights for policy 0, policy_version 1857550 (0.0008) [2023-12-27 04:55:45,718][105620] Updated weights for policy 1, policy_version 1862058 (0.0008) [2023-12-27 04:55:45,771][105620] Updated weights for policy 1, policy_version 1862068 (0.0008) [2023-12-27 04:55:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 952360960. Throughput: 0: 9449.2, 1: 9814.0. Samples: 952328568. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:55:46,062][104569] Avg episode reward: [(0, '8266.347'), (1, '9346.385')] [2023-12-27 04:55:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001857552_475602944.pth... [2023-12-27 04:55:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001862072_476758016.pth... [2023-12-27 04:55:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001856464_475324416.pth [2023-12-27 04:55:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001860920_476463104.pth [2023-12-27 04:55:46,438][105692] Updated weights for policy 0, policy_version 1857560 (0.0005) [2023-12-27 04:55:46,478][105620] Updated weights for policy 1, policy_version 1862078 (0.0009) [2023-12-27 04:55:46,501][105692] Updated weights for policy 0, policy_version 1857570 (0.0006) [2023-12-27 04:55:46,530][105620] Updated weights for policy 1, policy_version 1862088 (0.0009) [2023-12-27 04:55:46,556][105692] Updated weights for policy 0, policy_version 1857580 (0.0005) [2023-12-27 04:55:46,587][105620] Updated weights for policy 1, policy_version 1862098 (0.0010) [2023-12-27 04:55:47,098][105692] Updated weights for policy 0, policy_version 1857590 (0.0005) [2023-12-27 04:55:47,162][105692] Updated weights for policy 0, policy_version 1857600 (0.0009) [2023-12-27 04:55:47,228][105692] Updated weights for policy 0, policy_version 1857610 (0.0010) [2023-12-27 04:55:47,303][105620] Updated weights for policy 1, policy_version 1862108 (0.0010) [2023-12-27 04:55:47,362][105620] Updated weights for policy 1, policy_version 1862118 (0.0009) [2023-12-27 04:55:47,427][105620] Updated weights for policy 1, policy_version 1862128 (0.0009) [2023-12-27 04:55:47,997][105692] Updated weights for policy 0, policy_version 1857620 (0.0009) [2023-12-27 04:55:48,054][105692] Updated weights for policy 0, policy_version 1857630 (0.0009) [2023-12-27 04:55:48,103][105692] Updated weights for policy 0, policy_version 1857640 (0.0008) [2023-12-27 04:55:48,119][105620] Updated weights for policy 1, policy_version 1862138 (0.0009) [2023-12-27 04:55:48,186][105620] Updated weights for policy 1, policy_version 1862148 (0.0009) [2023-12-27 04:55:48,247][105620] Updated weights for policy 1, policy_version 1862158 (0.0009) [2023-12-27 04:55:48,304][105620] Updated weights for policy 1, policy_version 1862168 (0.0008) [2023-12-27 04:55:48,878][105692] Updated weights for policy 0, policy_version 1857650 (0.0007) [2023-12-27 04:55:48,937][105692] Updated weights for policy 0, policy_version 1857660 (0.0010) [2023-12-27 04:55:48,994][105620] Updated weights for policy 1, policy_version 1862178 (0.0007) [2023-12-27 04:55:48,995][105692] Updated weights for policy 0, policy_version 1857670 (0.0007) [2023-12-27 04:55:49,047][105620] Updated weights for policy 1, policy_version 1862188 (0.0006) [2023-12-27 04:55:49,049][105692] Updated weights for policy 0, policy_version 1857680 (0.0008) [2023-12-27 04:55:49,100][105620] Updated weights for policy 1, policy_version 1862198 (0.0008) [2023-12-27 04:55:49,783][105692] Updated weights for policy 0, policy_version 1857690 (0.0008) [2023-12-27 04:55:49,846][105692] Updated weights for policy 0, policy_version 1857700 (0.0008) [2023-12-27 04:55:49,879][105620] Updated weights for policy 1, policy_version 1862208 (0.0010) [2023-12-27 04:55:49,913][105692] Updated weights for policy 0, policy_version 1857710 (0.0006) [2023-12-27 04:55:49,951][105620] Updated weights for policy 1, policy_version 1862218 (0.0011) [2023-12-27 04:55:50,005][105620] Updated weights for policy 1, policy_version 1862228 (0.0011) [2023-12-27 04:55:50,703][105692] Updated weights for policy 0, policy_version 1857720 (0.0009) [2023-12-27 04:55:50,736][105620] Updated weights for policy 1, policy_version 1862238 (0.0011) [2023-12-27 04:55:50,759][105692] Updated weights for policy 0, policy_version 1857730 (0.0006) [2023-12-27 04:55:50,800][105620] Updated weights for policy 1, policy_version 1862248 (0.0011) [2023-12-27 04:55:50,810][105692] Updated weights for policy 0, policy_version 1857740 (0.0008) [2023-12-27 04:55:50,863][105620] Updated weights for policy 1, policy_version 1862258 (0.0011) [2023-12-27 04:55:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 952459264. Throughput: 0: 9496.8, 1: 9783.9. Samples: 952446816. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:55:51,062][104569] Avg episode reward: [(0, '8630.122'), (1, '9254.052')] [2023-12-27 04:55:51,491][105692] Updated weights for policy 0, policy_version 1857750 (0.0005) [2023-12-27 04:55:51,551][105692] Updated weights for policy 0, policy_version 1857760 (0.0005) [2023-12-27 04:55:51,604][105620] Updated weights for policy 1, policy_version 1862268 (0.0011) [2023-12-27 04:55:51,613][105692] Updated weights for policy 0, policy_version 1857770 (0.0007) [2023-12-27 04:55:51,662][105620] Updated weights for policy 1, policy_version 1862278 (0.0011) [2023-12-27 04:55:51,733][105620] Updated weights for policy 1, policy_version 1862288 (0.0009) [2023-12-27 04:55:52,276][105692] Updated weights for policy 0, policy_version 1857780 (0.0009) [2023-12-27 04:55:52,338][105692] Updated weights for policy 0, policy_version 1857790 (0.0009) [2023-12-27 04:55:52,402][105692] Updated weights for policy 0, policy_version 1857800 (0.0008) [2023-12-27 04:55:52,457][105620] Updated weights for policy 1, policy_version 1862298 (0.0011) [2023-12-27 04:55:52,515][105620] Updated weights for policy 1, policy_version 1862308 (0.0011) [2023-12-27 04:55:52,577][105620] Updated weights for policy 1, policy_version 1862318 (0.0011) [2023-12-27 04:55:52,635][105620] Updated weights for policy 1, policy_version 1862328 (0.0010) [2023-12-27 04:55:53,185][105692] Updated weights for policy 0, policy_version 1857810 (0.0008) [2023-12-27 04:55:53,251][105692] Updated weights for policy 0, policy_version 1857820 (0.0007) [2023-12-27 04:55:53,303][105692] Updated weights for policy 0, policy_version 1857830 (0.0010) [2023-12-27 04:55:53,360][105692] Updated weights for policy 0, policy_version 1857840 (0.0011) [2023-12-27 04:55:53,377][105620] Updated weights for policy 1, policy_version 1862338 (0.0008) [2023-12-27 04:55:53,429][105620] Updated weights for policy 1, policy_version 1862348 (0.0008) [2023-12-27 04:55:53,482][105620] Updated weights for policy 1, policy_version 1862358 (0.0008) [2023-12-27 04:55:54,121][105692] Updated weights for policy 0, policy_version 1857850 (0.0007) [2023-12-27 04:55:54,141][105620] Updated weights for policy 1, policy_version 1862368 (0.0008) [2023-12-27 04:55:54,193][105620] Updated weights for policy 1, policy_version 1862378 (0.0007) [2023-12-27 04:55:54,199][105692] Updated weights for policy 0, policy_version 1857860 (0.0006) [2023-12-27 04:55:54,250][105620] Updated weights for policy 1, policy_version 1862388 (0.0006) [2023-12-27 04:55:54,258][105692] Updated weights for policy 0, policy_version 1857870 (0.0008) [2023-12-27 04:55:54,966][105620] Updated weights for policy 1, policy_version 1862398 (0.0008) [2023-12-27 04:55:55,018][105692] Updated weights for policy 0, policy_version 1857880 (0.0007) [2023-12-27 04:55:55,021][105620] Updated weights for policy 1, policy_version 1862408 (0.0007) [2023-12-27 04:55:55,070][105620] Updated weights for policy 1, policy_version 1862418 (0.0008) [2023-12-27 04:55:55,081][105692] Updated weights for policy 0, policy_version 1857890 (0.0008) [2023-12-27 04:55:55,145][105692] Updated weights for policy 0, policy_version 1857900 (0.0009) [2023-12-27 04:55:55,760][105692] Updated weights for policy 0, policy_version 1857910 (0.0005) [2023-12-27 04:55:55,811][105692] Updated weights for policy 0, policy_version 1857920 (0.0005) [2023-12-27 04:55:55,864][105692] Updated weights for policy 0, policy_version 1857930 (0.0005) [2023-12-27 04:55:55,866][105620] Updated weights for policy 1, policy_version 1862428 (0.0008) [2023-12-27 04:55:55,912][105620] Updated weights for policy 1, policy_version 1862438 (0.0010) [2023-12-27 04:55:55,966][105620] Updated weights for policy 1, policy_version 1862448 (0.0010) [2023-12-27 04:55:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19577.5). Total num frames: 952557568. Throughput: 0: 9509.9, 1: 9805.5. Samples: 952561380. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:55:56,063][104569] Avg episode reward: [(0, '8624.885'), (1, '8981.187')] [2023-12-27 04:55:56,464][105692] Updated weights for policy 0, policy_version 1857941 (0.0008) [2023-12-27 04:55:56,520][105692] Updated weights for policy 0, policy_version 1857951 (0.0010) [2023-12-27 04:55:56,573][105692] Updated weights for policy 0, policy_version 1857961 (0.0010) [2023-12-27 04:55:56,584][105620] Updated weights for policy 1, policy_version 1862458 (0.0010) [2023-12-27 04:55:56,646][105620] Updated weights for policy 1, policy_version 1862468 (0.0006) [2023-12-27 04:55:56,701][105620] Updated weights for policy 1, policy_version 1862478 (0.0007) [2023-12-27 04:55:56,763][105620] Updated weights for policy 1, policy_version 1862488 (0.0009) [2023-12-27 04:55:57,186][105692] Updated weights for policy 0, policy_version 1857972 (0.0008) [2023-12-27 04:55:57,249][105692] Updated weights for policy 0, policy_version 1857982 (0.0005) [2023-12-27 04:55:57,309][105692] Updated weights for policy 0, policy_version 1857992 (0.0006) [2023-12-27 04:55:57,410][105620] Updated weights for policy 1, policy_version 1862498 (0.0005) [2023-12-27 04:55:57,474][105620] Updated weights for policy 1, policy_version 1862508 (0.0006) [2023-12-27 04:55:57,537][105620] Updated weights for policy 1, policy_version 1862518 (0.0010) [2023-12-27 04:55:57,841][105692] Updated weights for policy 0, policy_version 1858002 (0.0005) [2023-12-27 04:55:57,893][105692] Updated weights for policy 0, policy_version 1858012 (0.0005) [2023-12-27 04:55:57,937][105692] Updated weights for policy 0, policy_version 1858022 (0.0005) [2023-12-27 04:55:57,987][105692] Updated weights for policy 0, policy_version 1858032 (0.0008) [2023-12-27 04:55:58,148][105620] Updated weights for policy 1, policy_version 1862528 (0.0006) [2023-12-27 04:55:58,213][105620] Updated weights for policy 1, policy_version 1862538 (0.0009) [2023-12-27 04:55:58,277][105620] Updated weights for policy 1, policy_version 1862548 (0.0011) [2023-12-27 04:55:58,773][105692] Updated weights for policy 0, policy_version 1858042 (0.0008) [2023-12-27 04:55:58,828][105692] Updated weights for policy 0, policy_version 1858052 (0.0008) [2023-12-27 04:55:58,892][105692] Updated weights for policy 0, policy_version 1858062 (0.0008) [2023-12-27 04:55:59,040][105620] Updated weights for policy 1, policy_version 1862558 (0.0010) [2023-12-27 04:55:59,091][105620] Updated weights for policy 1, policy_version 1862568 (0.0010) [2023-12-27 04:55:59,144][105620] Updated weights for policy 1, policy_version 1862578 (0.0010) [2023-12-27 04:55:59,607][105692] Updated weights for policy 0, policy_version 1858072 (0.0010) [2023-12-27 04:55:59,662][105692] Updated weights for policy 0, policy_version 1858082 (0.0010) [2023-12-27 04:55:59,720][105692] Updated weights for policy 0, policy_version 1858092 (0.0010) [2023-12-27 04:55:59,919][105620] Updated weights for policy 1, policy_version 1862588 (0.0010) [2023-12-27 04:55:59,977][105620] Updated weights for policy 1, policy_version 1862598 (0.0011) [2023-12-27 04:56:00,037][105620] Updated weights for policy 1, policy_version 1862608 (0.0011) [2023-12-27 04:56:00,418][105692] Updated weights for policy 0, policy_version 1858102 (0.0009) [2023-12-27 04:56:00,479][105692] Updated weights for policy 0, policy_version 1858112 (0.0009) [2023-12-27 04:56:00,530][105692] Updated weights for policy 0, policy_version 1858122 (0.0008) [2023-12-27 04:56:00,663][105620] Updated weights for policy 1, policy_version 1862618 (0.0009) [2023-12-27 04:56:00,714][105620] Updated weights for policy 1, policy_version 1862628 (0.0009) [2023-12-27 04:56:00,761][105620] Updated weights for policy 1, policy_version 1862638 (0.0010) [2023-12-27 04:56:00,805][105620] Updated weights for policy 1, policy_version 1862648 (0.0010) [2023-12-27 04:56:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 952655872. Throughput: 0: 9617.3, 1: 9856.5. Samples: 952625528. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:56:01,063][104569] Avg episode reward: [(0, '8261.490'), (1, '9073.448')] [2023-12-27 04:56:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001858128_475750400.pth... [2023-12-27 04:56:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001862648_476905472.pth... [2023-12-27 04:56:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001857008_475463680.pth [2023-12-27 04:56:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001861496_476610560.pth [2023-12-27 04:56:01,204][105692] Updated weights for policy 0, policy_version 1858132 (0.0007) [2023-12-27 04:56:01,263][105692] Updated weights for policy 0, policy_version 1858142 (0.0008) [2023-12-27 04:56:01,325][105692] Updated weights for policy 0, policy_version 1858152 (0.0008) [2023-12-27 04:56:01,541][105620] Updated weights for policy 1, policy_version 1862658 (0.0007) [2023-12-27 04:56:01,588][105620] Updated weights for policy 1, policy_version 1862668 (0.0005) [2023-12-27 04:56:01,649][105620] Updated weights for policy 1, policy_version 1862678 (0.0008) [2023-12-27 04:56:02,032][105692] Updated weights for policy 0, policy_version 1858162 (0.0008) [2023-12-27 04:56:02,086][105692] Updated weights for policy 0, policy_version 1858172 (0.0009) [2023-12-27 04:56:02,137][105692] Updated weights for policy 0, policy_version 1858182 (0.0005) [2023-12-27 04:56:02,186][105692] Updated weights for policy 0, policy_version 1858192 (0.0005) [2023-12-27 04:56:02,458][105620] Updated weights for policy 1, policy_version 1862688 (0.0009) [2023-12-27 04:56:02,518][105620] Updated weights for policy 1, policy_version 1862698 (0.0009) [2023-12-27 04:56:02,579][105620] Updated weights for policy 1, policy_version 1862708 (0.0009) [2023-12-27 04:56:02,894][105692] Updated weights for policy 0, policy_version 1858202 (0.0009) [2023-12-27 04:56:02,946][105692] Updated weights for policy 0, policy_version 1858212 (0.0010) [2023-12-27 04:56:03,003][105692] Updated weights for policy 0, policy_version 1858222 (0.0010) [2023-12-27 04:56:03,271][105620] Updated weights for policy 1, policy_version 1862718 (0.0010) [2023-12-27 04:56:03,320][105620] Updated weights for policy 1, policy_version 1862728 (0.0010) [2023-12-27 04:56:03,373][105620] Updated weights for policy 1, policy_version 1862738 (0.0005) [2023-12-27 04:56:03,730][105692] Updated weights for policy 0, policy_version 1858232 (0.0008) [2023-12-27 04:56:03,773][105692] Updated weights for policy 0, policy_version 1858242 (0.0007) [2023-12-27 04:56:03,823][105692] Updated weights for policy 0, policy_version 1858252 (0.0005) [2023-12-27 04:56:04,102][105620] Updated weights for policy 1, policy_version 1862748 (0.0006) [2023-12-27 04:56:04,152][105620] Updated weights for policy 1, policy_version 1862758 (0.0011) [2023-12-27 04:56:04,201][105620] Updated weights for policy 1, policy_version 1862768 (0.0011) [2023-12-27 04:56:04,614][105692] Updated weights for policy 0, policy_version 1858262 (0.0008) [2023-12-27 04:56:04,668][105692] Updated weights for policy 0, policy_version 1858272 (0.0008) [2023-12-27 04:56:04,717][105692] Updated weights for policy 0, policy_version 1858282 (0.0007) [2023-12-27 04:56:04,893][105620] Updated weights for policy 1, policy_version 1862778 (0.0010) [2023-12-27 04:56:04,947][105620] Updated weights for policy 1, policy_version 1862788 (0.0009) [2023-12-27 04:56:05,008][105620] Updated weights for policy 1, policy_version 1862798 (0.0007) [2023-12-27 04:56:05,053][105620] Updated weights for policy 1, policy_version 1862808 (0.0006) [2023-12-27 04:56:05,357][105692] Updated weights for policy 0, policy_version 1858292 (0.0005) [2023-12-27 04:56:05,406][105692] Updated weights for policy 0, policy_version 1858302 (0.0005) [2023-12-27 04:56:05,450][105692] Updated weights for policy 0, policy_version 1858312 (0.0005) [2023-12-27 04:56:05,630][105620] Updated weights for policy 1, policy_version 1862818 (0.0005) [2023-12-27 04:56:05,697][105620] Updated weights for policy 1, policy_version 1862828 (0.0005) [2023-12-27 04:56:05,753][105620] Updated weights for policy 1, policy_version 1862838 (0.0005) [2023-12-27 04:56:05,980][105692] Updated weights for policy 0, policy_version 1858322 (0.0005) [2023-12-27 04:56:06,026][105692] Updated weights for policy 0, policy_version 1858332 (0.0005) [2023-12-27 04:56:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 952754176. Throughput: 0: 9632.7, 1: 9769.9. Samples: 952742240. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:56:06,063][104569] Avg episode reward: [(0, '8808.370'), (1, '8886.572')] [2023-12-27 04:56:06,077][105692] Updated weights for policy 0, policy_version 1858342 (0.0005) [2023-12-27 04:56:06,137][105692] Updated weights for policy 0, policy_version 1858352 (0.0008) [2023-12-27 04:56:06,281][105620] Updated weights for policy 1, policy_version 1862848 (0.0008) [2023-12-27 04:56:06,344][105620] Updated weights for policy 1, policy_version 1862858 (0.0009) [2023-12-27 04:56:06,394][105620] Updated weights for policy 1, policy_version 1862868 (0.0009) [2023-12-27 04:56:06,883][105692] Updated weights for policy 0, policy_version 1858362 (0.0009) [2023-12-27 04:56:06,944][105692] Updated weights for policy 0, policy_version 1858372 (0.0009) [2023-12-27 04:56:06,995][105692] Updated weights for policy 0, policy_version 1858382 (0.0009) [2023-12-27 04:56:07,096][105620] Updated weights for policy 1, policy_version 1862878 (0.0007) [2023-12-27 04:56:07,161][105620] Updated weights for policy 1, policy_version 1862888 (0.0008) [2023-12-27 04:56:07,224][105620] Updated weights for policy 1, policy_version 1862898 (0.0006) [2023-12-27 04:56:07,779][105692] Updated weights for policy 0, policy_version 1858392 (0.0010) [2023-12-27 04:56:07,827][105692] Updated weights for policy 0, policy_version 1858402 (0.0010) [2023-12-27 04:56:07,878][105692] Updated weights for policy 0, policy_version 1858412 (0.0010) [2023-12-27 04:56:07,909][105620] Updated weights for policy 1, policy_version 1862908 (0.0007) [2023-12-27 04:56:07,969][105620] Updated weights for policy 1, policy_version 1862918 (0.0008) [2023-12-27 04:56:08,028][105620] Updated weights for policy 1, policy_version 1862928 (0.0008) [2023-12-27 04:56:08,647][105692] Updated weights for policy 0, policy_version 1858422 (0.0010) [2023-12-27 04:56:08,699][105692] Updated weights for policy 0, policy_version 1858432 (0.0010) [2023-12-27 04:56:08,752][105692] Updated weights for policy 0, policy_version 1858442 (0.0010) [2023-12-27 04:56:08,792][105620] Updated weights for policy 1, policy_version 1862938 (0.0008) [2023-12-27 04:56:08,849][105620] Updated weights for policy 1, policy_version 1862948 (0.0008) [2023-12-27 04:56:08,905][105620] Updated weights for policy 1, policy_version 1862958 (0.0008) [2023-12-27 04:56:08,955][105620] Updated weights for policy 1, policy_version 1862968 (0.0008) [2023-12-27 04:56:09,563][105692] Updated weights for policy 0, policy_version 1858452 (0.0011) [2023-12-27 04:56:09,622][105692] Updated weights for policy 0, policy_version 1858462 (0.0011) [2023-12-27 04:56:09,685][105692] Updated weights for policy 0, policy_version 1858472 (0.0011) [2023-12-27 04:56:09,777][105620] Updated weights for policy 1, policy_version 1862978 (0.0009) [2023-12-27 04:56:09,842][105620] Updated weights for policy 1, policy_version 1862988 (0.0008) [2023-12-27 04:56:09,902][105620] Updated weights for policy 1, policy_version 1862998 (0.0008) [2023-12-27 04:56:10,445][105692] Updated weights for policy 0, policy_version 1858482 (0.0011) [2023-12-27 04:56:10,491][105692] Updated weights for policy 0, policy_version 1858492 (0.0010) [2023-12-27 04:56:10,543][105692] Updated weights for policy 0, policy_version 1858502 (0.0010) [2023-12-27 04:56:10,610][105692] Updated weights for policy 0, policy_version 1858512 (0.0010) [2023-12-27 04:56:10,673][105620] Updated weights for policy 1, policy_version 1863008 (0.0008) [2023-12-27 04:56:10,741][105620] Updated weights for policy 1, policy_version 1863018 (0.0008) [2023-12-27 04:56:10,798][105620] Updated weights for policy 1, policy_version 1863028 (0.0008) [2023-12-27 04:56:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 952852480. Throughput: 0: 9691.5, 1: 9775.6. Samples: 952859928. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:56:11,062][104569] Avg episode reward: [(0, '8990.167'), (1, '8886.466')] [2023-12-27 04:56:11,394][105692] Updated weights for policy 0, policy_version 1858522 (0.0010) [2023-12-27 04:56:11,460][105692] Updated weights for policy 0, policy_version 1858532 (0.0010) [2023-12-27 04:56:11,523][105692] Updated weights for policy 0, policy_version 1858542 (0.0010) [2023-12-27 04:56:11,624][105620] Updated weights for policy 1, policy_version 1863038 (0.0008) [2023-12-27 04:56:11,687][105620] Updated weights for policy 1, policy_version 1863048 (0.0009) [2023-12-27 04:56:11,759][105620] Updated weights for policy 1, policy_version 1863058 (0.0008) [2023-12-27 04:56:12,308][105692] Updated weights for policy 0, policy_version 1858552 (0.0011) [2023-12-27 04:56:12,377][105692] Updated weights for policy 0, policy_version 1858562 (0.0011) [2023-12-27 04:56:12,436][105692] Updated weights for policy 0, policy_version 1858572 (0.0010) [2023-12-27 04:56:12,554][105620] Updated weights for policy 1, policy_version 1863068 (0.0008) [2023-12-27 04:56:12,604][105620] Updated weights for policy 1, policy_version 1863078 (0.0008) [2023-12-27 04:56:12,657][105620] Updated weights for policy 1, policy_version 1863088 (0.0008) [2023-12-27 04:56:13,188][105692] Updated weights for policy 0, policy_version 1858582 (0.0010) [2023-12-27 04:56:13,246][105692] Updated weights for policy 0, policy_version 1858592 (0.0010) [2023-12-27 04:56:13,303][105692] Updated weights for policy 0, policy_version 1858602 (0.0010) [2023-12-27 04:56:13,429][105620] Updated weights for policy 1, policy_version 1863098 (0.0006) [2023-12-27 04:56:13,487][105620] Updated weights for policy 1, policy_version 1863108 (0.0007) [2023-12-27 04:56:13,546][105620] Updated weights for policy 1, policy_version 1863118 (0.0008) [2023-12-27 04:56:13,600][105620] Updated weights for policy 1, policy_version 1863128 (0.0008) [2023-12-27 04:56:14,037][105692] Updated weights for policy 0, policy_version 1858612 (0.0010) [2023-12-27 04:56:14,088][105692] Updated weights for policy 0, policy_version 1858622 (0.0010) [2023-12-27 04:56:14,146][105692] Updated weights for policy 0, policy_version 1858632 (0.0010) [2023-12-27 04:56:14,369][105620] Updated weights for policy 1, policy_version 1863138 (0.0008) [2023-12-27 04:56:14,424][105620] Updated weights for policy 1, policy_version 1863148 (0.0008) [2023-12-27 04:56:14,471][105620] Updated weights for policy 1, policy_version 1863158 (0.0007) [2023-12-27 04:56:14,897][105692] Updated weights for policy 0, policy_version 1858642 (0.0010) [2023-12-27 04:56:14,949][105692] Updated weights for policy 0, policy_version 1858652 (0.0010) [2023-12-27 04:56:15,014][105692] Updated weights for policy 0, policy_version 1858662 (0.0009) [2023-12-27 04:56:15,073][105692] Updated weights for policy 0, policy_version 1858672 (0.0009) [2023-12-27 04:56:15,222][105620] Updated weights for policy 1, policy_version 1863168 (0.0008) [2023-12-27 04:56:15,274][105620] Updated weights for policy 1, policy_version 1863178 (0.0009) [2023-12-27 04:56:15,330][105620] Updated weights for policy 1, policy_version 1863188 (0.0009) [2023-12-27 04:56:15,760][105692] Updated weights for policy 0, policy_version 1858682 (0.0006) [2023-12-27 04:56:15,816][105692] Updated weights for policy 0, policy_version 1858692 (0.0005) [2023-12-27 04:56:15,868][105692] Updated weights for policy 0, policy_version 1858702 (0.0005) [2023-12-27 04:56:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 952942592. Throughput: 0: 9620.1, 1: 9732.4. Samples: 952913952. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:56:16,062][104569] Avg episode reward: [(0, '8536.900'), (1, '9253.607')] [2023-12-27 04:56:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001858704_475897856.pth... [2023-12-27 04:56:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001863192_477044736.pth... [2023-12-27 04:56:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001862072_476758016.pth [2023-12-27 04:56:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001857552_475602944.pth [2023-12-27 04:56:16,237][105620] Updated weights for policy 1, policy_version 1863198 (0.0009) [2023-12-27 04:56:16,302][105620] Updated weights for policy 1, policy_version 1863208 (0.0009) [2023-12-27 04:56:16,353][105620] Updated weights for policy 1, policy_version 1863218 (0.0008) [2023-12-27 04:56:16,426][105692] Updated weights for policy 0, policy_version 1858712 (0.0006) [2023-12-27 04:56:16,480][105692] Updated weights for policy 0, policy_version 1858722 (0.0006) [2023-12-27 04:56:16,534][105692] Updated weights for policy 0, policy_version 1858732 (0.0008) [2023-12-27 04:56:17,099][105692] Updated weights for policy 0, policy_version 1858742 (0.0005) [2023-12-27 04:56:17,159][105692] Updated weights for policy 0, policy_version 1858752 (0.0007) [2023-12-27 04:56:17,205][105620] Updated weights for policy 1, policy_version 1863228 (0.0008) [2023-12-27 04:56:17,212][105692] Updated weights for policy 0, policy_version 1858762 (0.0007) [2023-12-27 04:56:17,258][105620] Updated weights for policy 1, policy_version 1863238 (0.0008) [2023-12-27 04:56:17,319][105620] Updated weights for policy 1, policy_version 1863249 (0.0009) [2023-12-27 04:56:17,766][105692] Updated weights for policy 0, policy_version 1858772 (0.0005) [2023-12-27 04:56:17,821][105692] Updated weights for policy 0, policy_version 1858782 (0.0009) [2023-12-27 04:56:17,872][105692] Updated weights for policy 0, policy_version 1858792 (0.0010) [2023-12-27 04:56:18,199][105620] Updated weights for policy 1, policy_version 1863259 (0.0009) [2023-12-27 04:56:18,269][105620] Updated weights for policy 1, policy_version 1863269 (0.0008) [2023-12-27 04:56:18,318][105620] Updated weights for policy 1, policy_version 1863279 (0.0007) [2023-12-27 04:56:18,597][105692] Updated weights for policy 0, policy_version 1858802 (0.0010) [2023-12-27 04:56:18,656][105692] Updated weights for policy 0, policy_version 1858812 (0.0010) [2023-12-27 04:56:18,715][105692] Updated weights for policy 0, policy_version 1858822 (0.0010) [2023-12-27 04:56:18,771][105692] Updated weights for policy 0, policy_version 1858832 (0.0010) [2023-12-27 04:56:19,088][105620] Updated weights for policy 1, policy_version 1863289 (0.0008) [2023-12-27 04:56:19,148][105620] Updated weights for policy 1, policy_version 1863299 (0.0008) [2023-12-27 04:56:19,207][105620] Updated weights for policy 1, policy_version 1863309 (0.0008) [2023-12-27 04:56:19,270][105620] Updated weights for policy 1, policy_version 1863319 (0.0008) [2023-12-27 04:56:19,550][105692] Updated weights for policy 0, policy_version 1858842 (0.0008) [2023-12-27 04:56:19,610][105692] Updated weights for policy 0, policy_version 1858852 (0.0007) [2023-12-27 04:56:19,682][105692] Updated weights for policy 0, policy_version 1858862 (0.0007) [2023-12-27 04:56:20,059][105620] Updated weights for policy 1, policy_version 1863329 (0.0009) [2023-12-27 04:56:20,115][105620] Updated weights for policy 1, policy_version 1863339 (0.0009) [2023-12-27 04:56:20,178][105620] Updated weights for policy 1, policy_version 1863349 (0.0009) [2023-12-27 04:56:20,364][105692] Updated weights for policy 0, policy_version 1858872 (0.0007) [2023-12-27 04:56:20,417][105692] Updated weights for policy 0, policy_version 1858882 (0.0006) [2023-12-27 04:56:20,465][105692] Updated weights for policy 0, policy_version 1858892 (0.0006) [2023-12-27 04:56:21,001][105620] Updated weights for policy 1, policy_version 1863359 (0.0010) [2023-12-27 04:56:21,062][104569] Fps is (10 sec: 18022.2, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 953032704. Throughput: 0: 9733.6, 1: 9622.7. Samples: 953028448. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:56:21,063][104569] Avg episode reward: [(0, '8352.206'), (1, '9253.762')] [2023-12-27 04:56:21,074][105620] Updated weights for policy 1, policy_version 1863369 (0.0010) [2023-12-27 04:56:21,140][105620] Updated weights for policy 1, policy_version 1863379 (0.0009) [2023-12-27 04:56:21,142][105692] Updated weights for policy 0, policy_version 1858902 (0.0007) [2023-12-27 04:56:21,199][105692] Updated weights for policy 0, policy_version 1858912 (0.0008) [2023-12-27 04:56:21,268][105692] Updated weights for policy 0, policy_version 1858922 (0.0007) [2023-12-27 04:56:21,932][105692] Updated weights for policy 0, policy_version 1858932 (0.0010) [2023-12-27 04:56:21,975][105620] Updated weights for policy 1, policy_version 1863389 (0.0007) [2023-12-27 04:56:21,991][105692] Updated weights for policy 0, policy_version 1858942 (0.0009) [2023-12-27 04:56:22,038][105620] Updated weights for policy 1, policy_version 1863399 (0.0006) [2023-12-27 04:56:22,048][105692] Updated weights for policy 0, policy_version 1858952 (0.0007) [2023-12-27 04:56:22,100][105620] Updated weights for policy 1, policy_version 1863409 (0.0009) [2023-12-27 04:56:22,776][105692] Updated weights for policy 0, policy_version 1858962 (0.0007) [2023-12-27 04:56:22,824][105692] Updated weights for policy 0, policy_version 1858972 (0.0009) [2023-12-27 04:56:22,875][105620] Updated weights for policy 1, policy_version 1863419 (0.0008) [2023-12-27 04:56:22,877][105692] Updated weights for policy 0, policy_version 1858982 (0.0008) [2023-12-27 04:56:22,928][105692] Updated weights for policy 0, policy_version 1858992 (0.0007) [2023-12-27 04:56:22,935][105620] Updated weights for policy 1, policy_version 1863429 (0.0007) [2023-12-27 04:56:23,006][105620] Updated weights for policy 1, policy_version 1863439 (0.0006) [2023-12-27 04:56:23,581][105620] Updated weights for policy 1, policy_version 1863449 (0.0008) [2023-12-27 04:56:23,643][105620] Updated weights for policy 1, policy_version 1863459 (0.0007) [2023-12-27 04:56:23,682][105692] Updated weights for policy 0, policy_version 1859002 (0.0008) [2023-12-27 04:56:23,710][105620] Updated weights for policy 1, policy_version 1863469 (0.0008) [2023-12-27 04:56:23,737][105692] Updated weights for policy 0, policy_version 1859012 (0.0008) [2023-12-27 04:56:23,767][105620] Updated weights for policy 1, policy_version 1863479 (0.0007) [2023-12-27 04:56:23,796][105692] Updated weights for policy 0, policy_version 1859022 (0.0007) [2023-12-27 04:56:24,469][105620] Updated weights for policy 1, policy_version 1863489 (0.0007) [2023-12-27 04:56:24,521][105620] Updated weights for policy 1, policy_version 1863499 (0.0007) [2023-12-27 04:56:24,543][105692] Updated weights for policy 0, policy_version 1859032 (0.0009) [2023-12-27 04:56:24,581][105620] Updated weights for policy 1, policy_version 1863509 (0.0008) [2023-12-27 04:56:24,603][105692] Updated weights for policy 0, policy_version 1859042 (0.0007) [2023-12-27 04:56:24,655][105692] Updated weights for policy 0, policy_version 1859052 (0.0009) [2023-12-27 04:56:25,356][105620] Updated weights for policy 1, policy_version 1863519 (0.0008) [2023-12-27 04:56:25,419][105620] Updated weights for policy 1, policy_version 1863529 (0.0008) [2023-12-27 04:56:25,437][105692] Updated weights for policy 0, policy_version 1859062 (0.0009) [2023-12-27 04:56:25,488][105620] Updated weights for policy 1, policy_version 1863539 (0.0007) [2023-12-27 04:56:25,494][105692] Updated weights for policy 0, policy_version 1859072 (0.0008) [2023-12-27 04:56:25,549][105692] Updated weights for policy 0, policy_version 1859082 (0.0008) [2023-12-27 04:56:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 953131008. Throughput: 0: 9830.5, 1: 9521.3. Samples: 953142556. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:56:26,062][104569] Avg episode reward: [(0, '8166.764'), (1, '9253.760')] [2023-12-27 04:56:26,093][105620] Updated weights for policy 1, policy_version 1863549 (0.0006) [2023-12-27 04:56:26,154][105620] Updated weights for policy 1, policy_version 1863559 (0.0010) [2023-12-27 04:56:26,203][105620] Updated weights for policy 1, policy_version 1863569 (0.0010) [2023-12-27 04:56:26,305][105692] Updated weights for policy 0, policy_version 1859092 (0.0007) [2023-12-27 04:56:26,354][105692] Updated weights for policy 0, policy_version 1859102 (0.0005) [2023-12-27 04:56:26,415][105692] Updated weights for policy 0, policy_version 1859112 (0.0007) [2023-12-27 04:56:26,876][105620] Updated weights for policy 1, policy_version 1863579 (0.0010) [2023-12-27 04:56:26,924][105620] Updated weights for policy 1, policy_version 1863589 (0.0010) [2023-12-27 04:56:26,983][105620] Updated weights for policy 1, policy_version 1863599 (0.0006) [2023-12-27 04:56:27,180][105692] Updated weights for policy 0, policy_version 1859122 (0.0007) [2023-12-27 04:56:27,241][105692] Updated weights for policy 0, policy_version 1859132 (0.0005) [2023-12-27 04:56:27,304][105692] Updated weights for policy 0, policy_version 1859142 (0.0005) [2023-12-27 04:56:27,359][105692] Updated weights for policy 0, policy_version 1859152 (0.0007) [2023-12-27 04:56:27,528][105620] Updated weights for policy 1, policy_version 1863609 (0.0005) [2023-12-27 04:56:27,577][105620] Updated weights for policy 1, policy_version 1863619 (0.0005) [2023-12-27 04:56:27,627][105620] Updated weights for policy 1, policy_version 1863629 (0.0005) [2023-12-27 04:56:27,680][105620] Updated weights for policy 1, policy_version 1863639 (0.0005) [2023-12-27 04:56:28,109][105692] Updated weights for policy 0, policy_version 1859162 (0.0005) [2023-12-27 04:56:28,161][105692] Updated weights for policy 0, policy_version 1859172 (0.0009) [2023-12-27 04:56:28,201][105620] Updated weights for policy 1, policy_version 1863649 (0.0005) [2023-12-27 04:56:28,213][105692] Updated weights for policy 0, policy_version 1859182 (0.0010) [2023-12-27 04:56:28,251][105620] Updated weights for policy 1, policy_version 1863659 (0.0005) [2023-12-27 04:56:28,300][105620] Updated weights for policy 1, policy_version 1863669 (0.0005) [2023-12-27 04:56:28,821][105692] Updated weights for policy 0, policy_version 1859192 (0.0009) [2023-12-27 04:56:28,879][105692] Updated weights for policy 0, policy_version 1859202 (0.0006) [2023-12-27 04:56:28,930][105692] Updated weights for policy 0, policy_version 1859212 (0.0005) [2023-12-27 04:56:28,944][105620] Updated weights for policy 1, policy_version 1863679 (0.0009) [2023-12-27 04:56:29,004][105620] Updated weights for policy 1, policy_version 1863689 (0.0011) [2023-12-27 04:56:29,051][105620] Updated weights for policy 1, policy_version 1863699 (0.0010) [2023-12-27 04:56:29,636][105692] Updated weights for policy 0, policy_version 1859222 (0.0007) [2023-12-27 04:56:29,688][105692] Updated weights for policy 0, policy_version 1859232 (0.0006) [2023-12-27 04:56:29,745][105692] Updated weights for policy 0, policy_version 1859242 (0.0005) [2023-12-27 04:56:29,753][105620] Updated weights for policy 1, policy_version 1863709 (0.0010) [2023-12-27 04:56:29,820][105620] Updated weights for policy 1, policy_version 1863719 (0.0008) [2023-12-27 04:56:29,895][105620] Updated weights for policy 1, policy_version 1863729 (0.0011) [2023-12-27 04:56:30,418][105692] Updated weights for policy 0, policy_version 1859252 (0.0006) [2023-12-27 04:56:30,479][105692] Updated weights for policy 0, policy_version 1859262 (0.0005) [2023-12-27 04:56:30,535][105692] Updated weights for policy 0, policy_version 1859272 (0.0005) [2023-12-27 04:56:30,635][105620] Updated weights for policy 1, policy_version 1863739 (0.0011) [2023-12-27 04:56:30,689][105620] Updated weights for policy 1, policy_version 1863749 (0.0010) [2023-12-27 04:56:30,747][105620] Updated weights for policy 1, policy_version 1863759 (0.0010) [2023-12-27 04:56:31,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 953237504. Throughput: 0: 9859.6, 1: 9650.9. Samples: 953206540. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:56:31,063][104569] Avg episode reward: [(0, '8440.489'), (1, '9253.812')] [2023-12-27 04:56:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001859280_476045312.pth... [2023-12-27 04:56:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001863768_477192192.pth... [2023-12-27 04:56:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001858128_475750400.pth [2023-12-27 04:56:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001862648_476905472.pth [2023-12-27 04:56:31,158][105692] Updated weights for policy 0, policy_version 1859282 (0.0006) [2023-12-27 04:56:31,221][105692] Updated weights for policy 0, policy_version 1859292 (0.0007) [2023-12-27 04:56:31,279][105692] Updated weights for policy 0, policy_version 1859302 (0.0010) [2023-12-27 04:56:31,327][105692] Updated weights for policy 0, policy_version 1859312 (0.0010) [2023-12-27 04:56:31,488][105620] Updated weights for policy 1, policy_version 1863769 (0.0010) [2023-12-27 04:56:31,547][105620] Updated weights for policy 1, policy_version 1863779 (0.0011) [2023-12-27 04:56:31,634][105620] Updated weights for policy 1, policy_version 1863789 (0.0011) [2023-12-27 04:56:31,701][105620] Updated weights for policy 1, policy_version 1863799 (0.0011) [2023-12-27 04:56:32,100][105692] Updated weights for policy 0, policy_version 1859322 (0.0010) [2023-12-27 04:56:32,165][105692] Updated weights for policy 0, policy_version 1859332 (0.0011) [2023-12-27 04:56:32,217][105692] Updated weights for policy 0, policy_version 1859342 (0.0010) [2023-12-27 04:56:32,286][105620] Updated weights for policy 1, policy_version 1863809 (0.0008) [2023-12-27 04:56:32,354][105620] Updated weights for policy 1, policy_version 1863819 (0.0008) [2023-12-27 04:56:32,425][105620] Updated weights for policy 1, policy_version 1863829 (0.0009) [2023-12-27 04:56:32,857][105692] Updated weights for policy 0, policy_version 1859352 (0.0011) [2023-12-27 04:56:32,908][105692] Updated weights for policy 0, policy_version 1859362 (0.0010) [2023-12-27 04:56:32,973][105692] Updated weights for policy 0, policy_version 1859372 (0.0011) [2023-12-27 04:56:33,033][105620] Updated weights for policy 1, policy_version 1863839 (0.0007) [2023-12-27 04:56:33,093][105620] Updated weights for policy 1, policy_version 1863849 (0.0008) [2023-12-27 04:56:33,153][105620] Updated weights for policy 1, policy_version 1863859 (0.0008) [2023-12-27 04:56:33,562][105692] Updated weights for policy 0, policy_version 1859382 (0.0009) [2023-12-27 04:56:33,619][105692] Updated weights for policy 0, policy_version 1859392 (0.0009) [2023-12-27 04:56:33,671][105692] Updated weights for policy 0, policy_version 1859402 (0.0005) [2023-12-27 04:56:33,841][105620] Updated weights for policy 1, policy_version 1863869 (0.0008) [2023-12-27 04:56:33,897][105620] Updated weights for policy 1, policy_version 1863879 (0.0008) [2023-12-27 04:56:33,953][105620] Updated weights for policy 1, policy_version 1863889 (0.0008) [2023-12-27 04:56:34,385][105692] Updated weights for policy 0, policy_version 1859412 (0.0009) [2023-12-27 04:56:34,458][105692] Updated weights for policy 0, policy_version 1859422 (0.0011) [2023-12-27 04:56:34,524][105692] Updated weights for policy 0, policy_version 1859432 (0.0011) [2023-12-27 04:56:34,666][105620] Updated weights for policy 1, policy_version 1863899 (0.0009) [2023-12-27 04:56:34,730][105620] Updated weights for policy 1, policy_version 1863909 (0.0010) [2023-12-27 04:56:34,783][105620] Updated weights for policy 1, policy_version 1863919 (0.0008) [2023-12-27 04:56:35,248][105692] Updated weights for policy 0, policy_version 1859442 (0.0010) [2023-12-27 04:56:35,306][105692] Updated weights for policy 0, policy_version 1859452 (0.0011) [2023-12-27 04:56:35,361][105692] Updated weights for policy 0, policy_version 1859462 (0.0010) [2023-12-27 04:56:35,365][105620] Updated weights for policy 1, policy_version 1863929 (0.0008) [2023-12-27 04:56:35,416][105620] Updated weights for policy 1, policy_version 1863939 (0.0008) [2023-12-27 04:56:35,419][105692] Updated weights for policy 0, policy_version 1859472 (0.0010) [2023-12-27 04:56:35,473][105620] Updated weights for policy 1, policy_version 1863949 (0.0006) [2023-12-27 04:56:35,523][105620] Updated weights for policy 1, policy_version 1863959 (0.0005) [2023-12-27 04:56:36,016][105692] Updated weights for policy 0, policy_version 1859482 (0.0005) [2023-12-27 04:56:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 953335808. Throughput: 0: 9892.2, 1: 9674.1. Samples: 953327304. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:56:36,062][104569] Avg episode reward: [(0, '8627.237'), (1, '9069.338')] [2023-12-27 04:56:36,078][105692] Updated weights for policy 0, policy_version 1859492 (0.0005) [2023-12-27 04:56:36,134][105692] Updated weights for policy 0, policy_version 1859502 (0.0006) [2023-12-27 04:56:36,134][105620] Updated weights for policy 1, policy_version 1863969 (0.0010) [2023-12-27 04:56:36,188][105620] Updated weights for policy 1, policy_version 1863979 (0.0011) [2023-12-27 04:56:36,244][105620] Updated weights for policy 1, policy_version 1863989 (0.0010) [2023-12-27 04:56:36,775][105692] Updated weights for policy 0, policy_version 1859512 (0.0006) [2023-12-27 04:56:36,840][105692] Updated weights for policy 0, policy_version 1859522 (0.0007) [2023-12-27 04:56:36,891][105692] Updated weights for policy 0, policy_version 1859532 (0.0010) [2023-12-27 04:56:36,991][105620] Updated weights for policy 1, policy_version 1863999 (0.0008) [2023-12-27 04:56:37,040][105620] Updated weights for policy 1, policy_version 1864009 (0.0006) [2023-12-27 04:56:37,088][105620] Updated weights for policy 1, policy_version 1864019 (0.0010) [2023-12-27 04:56:37,605][105692] Updated weights for policy 0, policy_version 1859542 (0.0010) [2023-12-27 04:56:37,660][105692] Updated weights for policy 0, policy_version 1859552 (0.0010) [2023-12-27 04:56:37,716][105692] Updated weights for policy 0, policy_version 1859562 (0.0010) [2023-12-27 04:56:37,735][105620] Updated weights for policy 1, policy_version 1864029 (0.0008) [2023-12-27 04:56:37,791][105620] Updated weights for policy 1, policy_version 1864039 (0.0007) [2023-12-27 04:56:37,840][105620] Updated weights for policy 1, policy_version 1864049 (0.0010) [2023-12-27 04:56:38,441][105692] Updated weights for policy 0, policy_version 1859572 (0.0007) [2023-12-27 04:56:38,458][105620] Updated weights for policy 1, policy_version 1864059 (0.0009) [2023-12-27 04:56:38,502][105692] Updated weights for policy 0, policy_version 1859582 (0.0007) [2023-12-27 04:56:38,528][105620] Updated weights for policy 1, policy_version 1864069 (0.0005) [2023-12-27 04:56:38,562][105692] Updated weights for policy 0, policy_version 1859592 (0.0007) [2023-12-27 04:56:38,597][105620] Updated weights for policy 1, policy_version 1864079 (0.0005) [2023-12-27 04:56:39,142][105692] Updated weights for policy 0, policy_version 1859602 (0.0006) [2023-12-27 04:56:39,181][105620] Updated weights for policy 1, policy_version 1864089 (0.0009) [2023-12-27 04:56:39,193][105692] Updated weights for policy 0, policy_version 1859612 (0.0007) [2023-12-27 04:56:39,251][105692] Updated weights for policy 0, policy_version 1859622 (0.0009) [2023-12-27 04:56:39,252][105620] Updated weights for policy 1, policy_version 1864099 (0.0012) [2023-12-27 04:56:39,308][105620] Updated weights for policy 1, policy_version 1864109 (0.0010) [2023-12-27 04:56:39,311][105692] Updated weights for policy 0, policy_version 1859632 (0.0011) [2023-12-27 04:56:39,376][105620] Updated weights for policy 1, policy_version 1864119 (0.0008) [2023-12-27 04:56:40,060][105620] Updated weights for policy 1, policy_version 1864129 (0.0006) [2023-12-27 04:56:40,105][105692] Updated weights for policy 0, policy_version 1859642 (0.0007) [2023-12-27 04:56:40,125][105620] Updated weights for policy 1, policy_version 1864139 (0.0008) [2023-12-27 04:56:40,173][105692] Updated weights for policy 0, policy_version 1859652 (0.0007) [2023-12-27 04:56:40,193][105620] Updated weights for policy 1, policy_version 1864149 (0.0008) [2023-12-27 04:56:40,239][105692] Updated weights for policy 0, policy_version 1859662 (0.0007) [2023-12-27 04:56:40,784][105620] Updated weights for policy 1, policy_version 1864159 (0.0006) [2023-12-27 04:56:40,850][105620] Updated weights for policy 1, policy_version 1864169 (0.0005) [2023-12-27 04:56:40,916][105620] Updated weights for policy 1, policy_version 1864179 (0.0006) [2023-12-27 04:56:40,929][105692] Updated weights for policy 0, policy_version 1859672 (0.0007) [2023-12-27 04:56:40,987][105692] Updated weights for policy 0, policy_version 1859682 (0.0005) [2023-12-27 04:56:41,048][105692] Updated weights for policy 0, policy_version 1859692 (0.0008) [2023-12-27 04:56:41,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 953442304. Throughput: 0: 9946.7, 1: 9859.3. Samples: 953452648. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:56:41,063][104569] Avg episode reward: [(0, '8631.304'), (1, '8979.254')] [2023-12-27 04:56:41,641][105620] Updated weights for policy 1, policy_version 1864189 (0.0010) [2023-12-27 04:56:41,706][105620] Updated weights for policy 1, policy_version 1864199 (0.0008) [2023-12-27 04:56:41,708][105692] Updated weights for policy 0, policy_version 1859702 (0.0008) [2023-12-27 04:56:41,770][105620] Updated weights for policy 1, policy_version 1864209 (0.0009) [2023-12-27 04:56:41,778][105692] Updated weights for policy 0, policy_version 1859712 (0.0007) [2023-12-27 04:56:41,830][105692] Updated weights for policy 0, policy_version 1859722 (0.0007) [2023-12-27 04:56:42,458][105620] Updated weights for policy 1, policy_version 1864219 (0.0009) [2023-12-27 04:56:42,520][105620] Updated weights for policy 1, policy_version 1864229 (0.0007) [2023-12-27 04:56:42,584][105620] Updated weights for policy 1, policy_version 1864239 (0.0007) [2023-12-27 04:56:42,698][105692] Updated weights for policy 0, policy_version 1859732 (0.0009) [2023-12-27 04:56:42,750][105692] Updated weights for policy 0, policy_version 1859742 (0.0009) [2023-12-27 04:56:42,797][105692] Updated weights for policy 0, policy_version 1859752 (0.0009) [2023-12-27 04:56:43,322][105620] Updated weights for policy 1, policy_version 1864249 (0.0008) [2023-12-27 04:56:43,381][105620] Updated weights for policy 1, policy_version 1864259 (0.0005) [2023-12-27 04:56:43,440][105620] Updated weights for policy 1, policy_version 1864269 (0.0005) [2023-12-27 04:56:43,494][105620] Updated weights for policy 1, policy_version 1864279 (0.0007) [2023-12-27 04:56:43,535][105692] Updated weights for policy 0, policy_version 1859762 (0.0008) [2023-12-27 04:56:43,593][105692] Updated weights for policy 0, policy_version 1859772 (0.0005) [2023-12-27 04:56:43,655][105692] Updated weights for policy 0, policy_version 1859782 (0.0006) [2023-12-27 04:56:43,723][105692] Updated weights for policy 0, policy_version 1859792 (0.0008) [2023-12-27 04:56:44,150][105620] Updated weights for policy 1, policy_version 1864289 (0.0008) [2023-12-27 04:56:44,214][105620] Updated weights for policy 1, policy_version 1864299 (0.0008) [2023-12-27 04:56:44,263][105620] Updated weights for policy 1, policy_version 1864309 (0.0006) [2023-12-27 04:56:44,279][105692] Updated weights for policy 0, policy_version 1859802 (0.0008) [2023-12-27 04:56:44,338][105692] Updated weights for policy 0, policy_version 1859812 (0.0009) [2023-12-27 04:56:44,386][105692] Updated weights for policy 0, policy_version 1859822 (0.0008) [2023-12-27 04:56:45,030][105620] Updated weights for policy 1, policy_version 1864319 (0.0008) [2023-12-27 04:56:45,089][105620] Updated weights for policy 1, policy_version 1864329 (0.0009) [2023-12-27 04:56:45,128][105692] Updated weights for policy 0, policy_version 1859832 (0.0007) [2023-12-27 04:56:45,152][105620] Updated weights for policy 1, policy_version 1864339 (0.0007) [2023-12-27 04:56:45,191][105692] Updated weights for policy 0, policy_version 1859842 (0.0006) [2023-12-27 04:56:45,244][105692] Updated weights for policy 0, policy_version 1859852 (0.0006) [2023-12-27 04:56:45,851][105692] Updated weights for policy 0, policy_version 1859862 (0.0005) [2023-12-27 04:56:45,919][105692] Updated weights for policy 0, policy_version 1859872 (0.0005) [2023-12-27 04:56:45,972][105692] Updated weights for policy 0, policy_version 1859882 (0.0005) [2023-12-27 04:56:46,017][105620] Updated weights for policy 1, policy_version 1864349 (0.0009) [2023-12-27 04:56:46,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.7, 300 sec: 19577.5). Total num frames: 953540608. Throughput: 0: 9862.8, 1: 9816.0. Samples: 953511076. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:56:46,063][105620] Updated weights for policy 1, policy_version 1864359 (0.0008) [2023-12-27 04:56:46,063][104569] Avg episode reward: [(0, '7991.739'), (1, '9163.638')] [2023-12-27 04:56:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001859888_476200960.pth... [2023-12-27 04:56:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001858704_475897856.pth [2023-12-27 04:56:46,114][105620] Updated weights for policy 1, policy_version 1864369 (0.0008) [2023-12-27 04:56:46,149][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001864376_477347840.pth... [2023-12-27 04:56:46,154][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001863192_477044736.pth [2023-12-27 04:56:46,524][105692] Updated weights for policy 0, policy_version 1859892 (0.0007) [2023-12-27 04:56:46,583][105692] Updated weights for policy 0, policy_version 1859902 (0.0007) [2023-12-27 04:56:46,637][105692] Updated weights for policy 0, policy_version 1859912 (0.0010) [2023-12-27 04:56:46,987][105620] Updated weights for policy 1, policy_version 1864379 (0.0008) [2023-12-27 04:56:47,044][105620] Updated weights for policy 1, policy_version 1864389 (0.0005) [2023-12-27 04:56:47,111][105620] Updated weights for policy 1, policy_version 1864399 (0.0007) [2023-12-27 04:56:47,168][105692] Updated weights for policy 0, policy_version 1859922 (0.0009) [2023-12-27 04:56:47,226][105692] Updated weights for policy 0, policy_version 1859932 (0.0006) [2023-12-27 04:56:47,276][105692] Updated weights for policy 0, policy_version 1859942 (0.0005) [2023-12-27 04:56:47,323][105692] Updated weights for policy 0, policy_version 1859952 (0.0005) [2023-12-27 04:56:47,777][105620] Updated weights for policy 1, policy_version 1864409 (0.0008) [2023-12-27 04:56:47,846][105620] Updated weights for policy 1, policy_version 1864419 (0.0006) [2023-12-27 04:56:47,886][105692] Updated weights for policy 0, policy_version 1859962 (0.0008) [2023-12-27 04:56:47,906][105620] Updated weights for policy 1, policy_version 1864429 (0.0007) [2023-12-27 04:56:47,941][105692] Updated weights for policy 0, policy_version 1859972 (0.0008) [2023-12-27 04:56:47,965][105620] Updated weights for policy 1, policy_version 1864439 (0.0010) [2023-12-27 04:56:48,000][105692] Updated weights for policy 0, policy_version 1859982 (0.0010) [2023-12-27 04:56:48,664][105692] Updated weights for policy 0, policy_version 1859992 (0.0010) [2023-12-27 04:56:48,721][105692] Updated weights for policy 0, policy_version 1860002 (0.0008) [2023-12-27 04:56:48,752][105620] Updated weights for policy 1, policy_version 1864449 (0.0008) [2023-12-27 04:56:48,771][105692] Updated weights for policy 0, policy_version 1860012 (0.0007) [2023-12-27 04:56:48,805][105620] Updated weights for policy 1, policy_version 1864459 (0.0008) [2023-12-27 04:56:48,862][105620] Updated weights for policy 1, policy_version 1864469 (0.0008) [2023-12-27 04:56:49,494][105692] Updated weights for policy 0, policy_version 1860022 (0.0008) [2023-12-27 04:56:49,544][105692] Updated weights for policy 0, policy_version 1860032 (0.0008) [2023-12-27 04:56:49,600][105692] Updated weights for policy 0, policy_version 1860043 (0.0009) [2023-12-27 04:56:49,643][105620] Updated weights for policy 1, policy_version 1864479 (0.0008) [2023-12-27 04:56:49,708][105620] Updated weights for policy 1, policy_version 1864489 (0.0009) [2023-12-27 04:56:49,769][105620] Updated weights for policy 1, policy_version 1864499 (0.0009) [2023-12-27 04:56:50,446][105692] Updated weights for policy 0, policy_version 1860053 (0.0009) [2023-12-27 04:56:50,465][105620] Updated weights for policy 1, policy_version 1864509 (0.0007) [2023-12-27 04:56:50,504][105692] Updated weights for policy 0, policy_version 1860063 (0.0008) [2023-12-27 04:56:50,522][105620] Updated weights for policy 1, policy_version 1864519 (0.0007) [2023-12-27 04:56:50,572][105692] Updated weights for policy 0, policy_version 1860073 (0.0006) [2023-12-27 04:56:50,586][105620] Updated weights for policy 1, policy_version 1864529 (0.0008) [2023-12-27 04:56:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 953638912. Throughput: 0: 10042.3, 1: 9705.3. Samples: 953630884. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:56:51,062][104569] Avg episode reward: [(0, '8444.123'), (1, '9254.154')] [2023-12-27 04:56:51,296][105620] Updated weights for policy 1, policy_version 1864539 (0.0007) [2023-12-27 04:56:51,310][105692] Updated weights for policy 0, policy_version 1860083 (0.0008) [2023-12-27 04:56:51,354][105620] Updated weights for policy 1, policy_version 1864549 (0.0007) [2023-12-27 04:56:51,370][105692] Updated weights for policy 0, policy_version 1860093 (0.0011) [2023-12-27 04:56:51,425][105620] Updated weights for policy 1, policy_version 1864559 (0.0007) [2023-12-27 04:56:51,434][105692] Updated weights for policy 0, policy_version 1860103 (0.0010) [2023-12-27 04:56:52,175][105620] Updated weights for policy 1, policy_version 1864569 (0.0007) [2023-12-27 04:56:52,214][105692] Updated weights for policy 0, policy_version 1860113 (0.0010) [2023-12-27 04:56:52,227][105620] Updated weights for policy 1, policy_version 1864579 (0.0008) [2023-12-27 04:56:52,282][105692] Updated weights for policy 0, policy_version 1860123 (0.0007) [2023-12-27 04:56:52,289][105620] Updated weights for policy 1, policy_version 1864589 (0.0007) [2023-12-27 04:56:52,344][105692] Updated weights for policy 0, policy_version 1860133 (0.0007) [2023-12-27 04:56:52,353][105620] Updated weights for policy 1, policy_version 1864599 (0.0008) [2023-12-27 04:56:52,407][105692] Updated weights for policy 0, policy_version 1860143 (0.0010) [2023-12-27 04:56:52,950][105620] Updated weights for policy 1, policy_version 1864609 (0.0009) [2023-12-27 04:56:52,995][105620] Updated weights for policy 1, policy_version 1864619 (0.0006) [2023-12-27 04:56:53,061][105620] Updated weights for policy 1, policy_version 1864629 (0.0007) [2023-12-27 04:56:53,213][105692] Updated weights for policy 0, policy_version 1860153 (0.0008) [2023-12-27 04:56:53,264][105692] Updated weights for policy 0, policy_version 1860163 (0.0008) [2023-12-27 04:56:53,319][105692] Updated weights for policy 0, policy_version 1860173 (0.0007) [2023-12-27 04:56:53,797][105620] Updated weights for policy 1, policy_version 1864639 (0.0009) [2023-12-27 04:56:53,858][105620] Updated weights for policy 1, policy_version 1864649 (0.0009) [2023-12-27 04:56:53,925][105620] Updated weights for policy 1, policy_version 1864659 (0.0008) [2023-12-27 04:56:54,082][105692] Updated weights for policy 0, policy_version 1860183 (0.0010) [2023-12-27 04:56:54,141][105692] Updated weights for policy 0, policy_version 1860193 (0.0010) [2023-12-27 04:56:54,200][105692] Updated weights for policy 0, policy_version 1860203 (0.0010) [2023-12-27 04:56:54,584][105620] Updated weights for policy 1, policy_version 1864669 (0.0009) [2023-12-27 04:56:54,630][105620] Updated weights for policy 1, policy_version 1864679 (0.0009) [2023-12-27 04:56:54,676][105620] Updated weights for policy 1, policy_version 1864689 (0.0008) [2023-12-27 04:56:54,970][105692] Updated weights for policy 0, policy_version 1860213 (0.0009) [2023-12-27 04:56:55,030][105692] Updated weights for policy 0, policy_version 1860223 (0.0008) [2023-12-27 04:56:55,093][105692] Updated weights for policy 0, policy_version 1860233 (0.0008) [2023-12-27 04:56:55,366][105620] Updated weights for policy 1, policy_version 1864699 (0.0009) [2023-12-27 04:56:55,424][105620] Updated weights for policy 1, policy_version 1864709 (0.0008) [2023-12-27 04:56:55,493][105620] Updated weights for policy 1, policy_version 1864719 (0.0009) [2023-12-27 04:56:55,715][105692] Updated weights for policy 0, policy_version 1860243 (0.0008) [2023-12-27 04:56:55,776][105692] Updated weights for policy 0, policy_version 1860253 (0.0010) [2023-12-27 04:56:55,838][105692] Updated weights for policy 0, policy_version 1860264 (0.0012) [2023-12-27 04:56:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 953737216. Throughput: 0: 9960.0, 1: 9726.6. Samples: 953745828. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:56:56,063][104569] Avg episode reward: [(0, '8715.740'), (1, '9069.464')] [2023-12-27 04:56:56,183][105620] Updated weights for policy 1, policy_version 1864729 (0.0009) [2023-12-27 04:56:56,240][105620] Updated weights for policy 1, policy_version 1864739 (0.0008) [2023-12-27 04:56:56,298][105620] Updated weights for policy 1, policy_version 1864749 (0.0008) [2023-12-27 04:56:56,363][105620] Updated weights for policy 1, policy_version 1864759 (0.0008) [2023-12-27 04:56:56,650][105692] Updated weights for policy 0, policy_version 1860274 (0.0009) [2023-12-27 04:56:56,697][105692] Updated weights for policy 0, policy_version 1860284 (0.0009) [2023-12-27 04:56:56,747][105692] Updated weights for policy 0, policy_version 1860294 (0.0008) [2023-12-27 04:56:56,811][105692] Updated weights for policy 0, policy_version 1860304 (0.0005) [2023-12-27 04:56:57,050][105620] Updated weights for policy 1, policy_version 1864769 (0.0010) [2023-12-27 04:56:57,099][105620] Updated weights for policy 1, policy_version 1864779 (0.0009) [2023-12-27 04:56:57,156][105620] Updated weights for policy 1, policy_version 1864789 (0.0010) [2023-12-27 04:56:57,363][105692] Updated weights for policy 0, policy_version 1860314 (0.0010) [2023-12-27 04:56:57,409][105692] Updated weights for policy 0, policy_version 1860324 (0.0010) [2023-12-27 04:56:57,467][105692] Updated weights for policy 0, policy_version 1860334 (0.0006) [2023-12-27 04:56:57,882][105620] Updated weights for policy 1, policy_version 1864799 (0.0011) [2023-12-27 04:56:57,940][105620] Updated weights for policy 1, policy_version 1864809 (0.0011) [2023-12-27 04:56:57,994][105620] Updated weights for policy 1, policy_version 1864819 (0.0010) [2023-12-27 04:56:58,066][105692] Updated weights for policy 0, policy_version 1860344 (0.0005) [2023-12-27 04:56:58,122][105692] Updated weights for policy 0, policy_version 1860354 (0.0005) [2023-12-27 04:56:58,182][105692] Updated weights for policy 0, policy_version 1860364 (0.0007) [2023-12-27 04:56:58,752][105620] Updated weights for policy 1, policy_version 1864829 (0.0009) [2023-12-27 04:56:58,819][105620] Updated weights for policy 1, policy_version 1864839 (0.0008) [2023-12-27 04:56:58,853][105692] Updated weights for policy 0, policy_version 1860374 (0.0008) [2023-12-27 04:56:58,894][105620] Updated weights for policy 1, policy_version 1864849 (0.0008) [2023-12-27 04:56:58,924][105692] Updated weights for policy 0, policy_version 1860384 (0.0008) [2023-12-27 04:56:58,993][105692] Updated weights for policy 0, policy_version 1860394 (0.0007) [2023-12-27 04:56:59,661][105620] Updated weights for policy 1, policy_version 1864859 (0.0009) [2023-12-27 04:56:59,694][105692] Updated weights for policy 0, policy_version 1860404 (0.0006) [2023-12-27 04:56:59,720][105620] Updated weights for policy 1, policy_version 1864869 (0.0011) [2023-12-27 04:56:59,737][105692] Updated weights for policy 0, policy_version 1860414 (0.0008) [2023-12-27 04:56:59,778][105620] Updated weights for policy 1, policy_version 1864879 (0.0009) [2023-12-27 04:56:59,781][105586] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000001 [2023-12-27 04:56:59,783][105692] Updated weights for policy 0, policy_version 1860424 (0.0007) [2023-12-27 04:57:00,558][105692] Updated weights for policy 0, policy_version 1860435 (0.0008) [2023-12-27 04:57:00,567][105620] Updated weights for policy 1, policy_version 1864889 (0.0009) [2023-12-27 04:57:00,605][105692] Updated weights for policy 0, policy_version 1860445 (0.0005) [2023-12-27 04:57:00,627][105620] Updated weights for policy 1, policy_version 1864899 (0.0011) [2023-12-27 04:57:00,668][105692] Updated weights for policy 0, policy_version 1860455 (0.0005) [2023-12-27 04:57:00,694][105620] Updated weights for policy 1, policy_version 1864909 (0.0011) [2023-12-27 04:57:01,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19660.7, 300 sec: 19605.2). Total num frames: 953835520. Throughput: 0: 10067.2, 1: 9771.1. Samples: 953806684. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:57:01,063][104569] Avg episode reward: [(0, '8533.662'), (1, '9161.053')] [2023-12-27 04:57:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001860464_476348416.pth... [2023-12-27 04:57:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001864912_477487104.pth... [2023-12-27 04:57:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001859280_476045312.pth [2023-12-27 04:57:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001863768_477192192.pth [2023-12-27 04:57:01,331][105692] Updated weights for policy 0, policy_version 1860465 (0.0006) [2023-12-27 04:57:01,386][105620] Updated weights for policy 1, policy_version 1864919 (0.0009) [2023-12-27 04:57:01,405][105692] Updated weights for policy 0, policy_version 1860475 (0.0009) [2023-12-27 04:57:01,451][105620] Updated weights for policy 1, policy_version 1864929 (0.0007) [2023-12-27 04:57:01,462][105692] Updated weights for policy 0, policy_version 1860485 (0.0011) [2023-12-27 04:57:01,508][105620] Updated weights for policy 1, policy_version 1864939 (0.0011) [2023-12-27 04:57:01,521][105692] Updated weights for policy 0, policy_version 1860495 (0.0011) [2023-12-27 04:57:02,147][105692] Updated weights for policy 0, policy_version 1860505 (0.0008) [2023-12-27 04:57:02,195][105692] Updated weights for policy 0, policy_version 1860515 (0.0010) [2023-12-27 04:57:02,209][105620] Updated weights for policy 1, policy_version 1864949 (0.0010) [2023-12-27 04:57:02,251][105692] Updated weights for policy 0, policy_version 1860525 (0.0011) [2023-12-27 04:57:02,268][105620] Updated weights for policy 1, policy_version 1864959 (0.0011) [2023-12-27 04:57:02,327][105620] Updated weights for policy 1, policy_version 1864969 (0.0011) [2023-12-27 04:57:02,983][105692] Updated weights for policy 0, policy_version 1860535 (0.0008) [2023-12-27 04:57:03,038][105692] Updated weights for policy 0, policy_version 1860545 (0.0008) [2023-12-27 04:57:03,067][105620] Updated weights for policy 1, policy_version 1864979 (0.0011) [2023-12-27 04:57:03,101][105692] Updated weights for policy 0, policy_version 1860555 (0.0006) [2023-12-27 04:57:03,115][105620] Updated weights for policy 1, policy_version 1864989 (0.0010) [2023-12-27 04:57:03,166][105620] Updated weights for policy 1, policy_version 1864999 (0.0009) [2023-12-27 04:57:03,777][105620] Updated weights for policy 1, policy_version 1865009 (0.0005) [2023-12-27 04:57:03,833][105620] Updated weights for policy 1, policy_version 1865019 (0.0005) [2023-12-27 04:57:03,895][105620] Updated weights for policy 1, policy_version 1865029 (0.0008) [2023-12-27 04:57:03,913][105692] Updated weights for policy 0, policy_version 1860565 (0.0007) [2023-12-27 04:57:03,954][105620] Updated weights for policy 1, policy_version 1865039 (0.0008) [2023-12-27 04:57:03,973][105692] Updated weights for policy 0, policy_version 1860575 (0.0009) [2023-12-27 04:57:04,026][105692] Updated weights for policy 0, policy_version 1860585 (0.0008) [2023-12-27 04:57:04,644][105620] Updated weights for policy 1, policy_version 1865049 (0.0008) [2023-12-27 04:57:04,699][105620] Updated weights for policy 1, policy_version 1865059 (0.0010) [2023-12-27 04:57:04,754][105692] Updated weights for policy 0, policy_version 1860595 (0.0007) [2023-12-27 04:57:04,755][105620] Updated weights for policy 1, policy_version 1865069 (0.0010) [2023-12-27 04:57:04,816][105692] Updated weights for policy 0, policy_version 1860605 (0.0005) [2023-12-27 04:57:04,880][105692] Updated weights for policy 0, policy_version 1860615 (0.0005) [2023-12-27 04:57:05,401][105692] Updated weights for policy 0, policy_version 1860625 (0.0007) [2023-12-27 04:57:05,452][105692] Updated weights for policy 0, policy_version 1860635 (0.0009) [2023-12-27 04:57:05,518][105692] Updated weights for policy 0, policy_version 1860645 (0.0006) [2023-12-27 04:57:05,566][105620] Updated weights for policy 1, policy_version 1865079 (0.0009) [2023-12-27 04:57:05,585][105692] Updated weights for policy 0, policy_version 1860655 (0.0006) [2023-12-27 04:57:05,624][105620] Updated weights for policy 1, policy_version 1865089 (0.0010) [2023-12-27 04:57:05,690][105620] Updated weights for policy 1, policy_version 1865099 (0.0008) [2023-12-27 04:57:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 953933824. Throughput: 0: 9954.5, 1: 9913.8. Samples: 953922520. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:57:06,063][104569] Avg episode reward: [(0, '8532.275'), (1, '9161.390')] [2023-12-27 04:57:06,290][105692] Updated weights for policy 0, policy_version 1860665 (0.0010) [2023-12-27 04:57:06,348][105620] Updated weights for policy 1, policy_version 1865109 (0.0006) [2023-12-27 04:57:06,354][105692] Updated weights for policy 0, policy_version 1860675 (0.0011) [2023-12-27 04:57:06,409][105620] Updated weights for policy 1, policy_version 1865119 (0.0006) [2023-12-27 04:57:06,414][105692] Updated weights for policy 0, policy_version 1860685 (0.0011) [2023-12-27 04:57:06,474][105620] Updated weights for policy 1, policy_version 1865129 (0.0007) [2023-12-27 04:57:07,135][105692] Updated weights for policy 0, policy_version 1860695 (0.0011) [2023-12-27 04:57:07,186][105620] Updated weights for policy 1, policy_version 1865139 (0.0009) [2023-12-27 04:57:07,197][105692] Updated weights for policy 0, policy_version 1860705 (0.0006) [2023-12-27 04:57:07,247][105620] Updated weights for policy 1, policy_version 1865149 (0.0009) [2023-12-27 04:57:07,262][105692] Updated weights for policy 0, policy_version 1860715 (0.0005) [2023-12-27 04:57:07,304][105620] Updated weights for policy 1, policy_version 1865159 (0.0009) [2023-12-27 04:57:07,942][105692] Updated weights for policy 0, policy_version 1860725 (0.0008) [2023-12-27 04:57:07,990][105692] Updated weights for policy 0, policy_version 1860735 (0.0010) [2023-12-27 04:57:08,012][105620] Updated weights for policy 1, policy_version 1865169 (0.0008) [2023-12-27 04:57:08,039][105692] Updated weights for policy 0, policy_version 1860745 (0.0010) [2023-12-27 04:57:08,057][105620] Updated weights for policy 1, policy_version 1865179 (0.0006) [2023-12-27 04:57:08,115][105620] Updated weights for policy 1, policy_version 1865189 (0.0007) [2023-12-27 04:57:08,164][105620] Updated weights for policy 1, policy_version 1865199 (0.0008) [2023-12-27 04:57:08,801][105692] Updated weights for policy 0, policy_version 1860755 (0.0011) [2023-12-27 04:57:08,851][105692] Updated weights for policy 0, policy_version 1860765 (0.0010) [2023-12-27 04:57:08,910][105692] Updated weights for policy 0, policy_version 1860775 (0.0011) [2023-12-27 04:57:08,968][105620] Updated weights for policy 1, policy_version 1865209 (0.0008) [2023-12-27 04:57:09,033][105620] Updated weights for policy 1, policy_version 1865219 (0.0007) [2023-12-27 04:57:09,093][105620] Updated weights for policy 1, policy_version 1865229 (0.0007) [2023-12-27 04:57:09,673][105692] Updated weights for policy 0, policy_version 1860785 (0.0010) [2023-12-27 04:57:09,740][105692] Updated weights for policy 0, policy_version 1860795 (0.0011) [2023-12-27 04:57:09,798][105620] Updated weights for policy 1, policy_version 1865239 (0.0007) [2023-12-27 04:57:09,802][105692] Updated weights for policy 0, policy_version 1860805 (0.0010) [2023-12-27 04:57:09,868][105620] Updated weights for policy 1, policy_version 1865249 (0.0007) [2023-12-27 04:57:09,869][105692] Updated weights for policy 0, policy_version 1860815 (0.0009) [2023-12-27 04:57:09,932][105620] Updated weights for policy 1, policy_version 1865259 (0.0006) [2023-12-27 04:57:10,579][105692] Updated weights for policy 0, policy_version 1860825 (0.0007) [2023-12-27 04:57:10,648][105692] Updated weights for policy 0, policy_version 1860835 (0.0011) [2023-12-27 04:57:10,671][105620] Updated weights for policy 1, policy_version 1865269 (0.0007) [2023-12-27 04:57:10,704][105692] Updated weights for policy 0, policy_version 1860845 (0.0010) [2023-12-27 04:57:10,737][105620] Updated weights for policy 1, policy_version 1865279 (0.0006) [2023-12-27 04:57:10,796][105620] Updated weights for policy 1, policy_version 1865289 (0.0005) [2023-12-27 04:57:11,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 954032128. Throughput: 0: 9988.4, 1: 9950.6. Samples: 954039808. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:57:11,062][104569] Avg episode reward: [(0, '8532.169'), (1, '9068.975')] [2023-12-27 04:57:11,450][105620] Updated weights for policy 1, policy_version 1865299 (0.0007) [2023-12-27 04:57:11,455][105692] Updated weights for policy 0, policy_version 1860855 (0.0010) [2023-12-27 04:57:11,506][105620] Updated weights for policy 1, policy_version 1865309 (0.0009) [2023-12-27 04:57:11,515][105692] Updated weights for policy 0, policy_version 1860865 (0.0011) [2023-12-27 04:57:11,570][105620] Updated weights for policy 1, policy_version 1865319 (0.0006) [2023-12-27 04:57:11,580][105692] Updated weights for policy 0, policy_version 1860875 (0.0011) [2023-12-27 04:57:12,287][105692] Updated weights for policy 0, policy_version 1860885 (0.0009) [2023-12-27 04:57:12,318][105620] Updated weights for policy 1, policy_version 1865329 (0.0007) [2023-12-27 04:57:12,355][105692] Updated weights for policy 0, policy_version 1860895 (0.0008) [2023-12-27 04:57:12,384][105620] Updated weights for policy 1, policy_version 1865339 (0.0007) [2023-12-27 04:57:12,415][105692] Updated weights for policy 0, policy_version 1860905 (0.0008) [2023-12-27 04:57:12,446][105620] Updated weights for policy 1, policy_version 1865349 (0.0008) [2023-12-27 04:57:12,510][105620] Updated weights for policy 1, policy_version 1865359 (0.0008) [2023-12-27 04:57:13,154][105692] Updated weights for policy 0, policy_version 1860915 (0.0007) [2023-12-27 04:57:13,209][105692] Updated weights for policy 0, policy_version 1860925 (0.0009) [2023-12-27 04:57:13,256][105620] Updated weights for policy 1, policy_version 1865369 (0.0007) [2023-12-27 04:57:13,262][105692] Updated weights for policy 0, policy_version 1860935 (0.0007) [2023-12-27 04:57:13,316][105620] Updated weights for policy 1, policy_version 1865379 (0.0008) [2023-12-27 04:57:13,372][105620] Updated weights for policy 1, policy_version 1865389 (0.0009) [2023-12-27 04:57:14,047][105692] Updated weights for policy 0, policy_version 1860945 (0.0008) [2023-12-27 04:57:14,066][105620] Updated weights for policy 1, policy_version 1865399 (0.0008) [2023-12-27 04:57:14,105][105692] Updated weights for policy 0, policy_version 1860955 (0.0007) [2023-12-27 04:57:14,127][105620] Updated weights for policy 1, policy_version 1865409 (0.0008) [2023-12-27 04:57:14,159][105692] Updated weights for policy 0, policy_version 1860965 (0.0007) [2023-12-27 04:57:14,178][105620] Updated weights for policy 1, policy_version 1865419 (0.0005) [2023-12-27 04:57:14,208][105692] Updated weights for policy 0, policy_version 1860975 (0.0008) [2023-12-27 04:57:14,830][105620] Updated weights for policy 1, policy_version 1865429 (0.0008) [2023-12-27 04:57:14,897][105620] Updated weights for policy 1, policy_version 1865439 (0.0011) [2023-12-27 04:57:14,960][105620] Updated weights for policy 1, policy_version 1865449 (0.0010) [2023-12-27 04:57:14,979][105692] Updated weights for policy 0, policy_version 1860985 (0.0009) [2023-12-27 04:57:15,038][105692] Updated weights for policy 0, policy_version 1860995 (0.0009) [2023-12-27 04:57:15,103][105692] Updated weights for policy 0, policy_version 1861005 (0.0008) [2023-12-27 04:57:15,699][105620] Updated weights for policy 1, policy_version 1865459 (0.0010) [2023-12-27 04:57:15,764][105620] Updated weights for policy 1, policy_version 1865469 (0.0010) [2023-12-27 04:57:15,789][105692] Updated weights for policy 0, policy_version 1861015 (0.0006) [2023-12-27 04:57:15,826][105620] Updated weights for policy 1, policy_version 1865479 (0.0010) [2023-12-27 04:57:15,846][105692] Updated weights for policy 0, policy_version 1861025 (0.0005) [2023-12-27 04:57:15,905][105692] Updated weights for policy 0, policy_version 1861035 (0.0010) [2023-12-27 04:57:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.2, 300 sec: 19633.0). Total num frames: 954130432. Throughput: 0: 9966.8, 1: 9809.8. Samples: 954096492. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:57:16,063][104569] Avg episode reward: [(0, '8443.089'), (1, '9161.049')] [2023-12-27 04:57:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001861040_476495872.pth... [2023-12-27 04:57:16,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001865488_477634560.pth... [2023-12-27 04:57:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001859888_476200960.pth [2023-12-27 04:57:16,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001864376_477347840.pth [2023-12-27 04:57:16,423][105620] Updated weights for policy 1, policy_version 1865489 (0.0010) [2023-12-27 04:57:16,477][105620] Updated weights for policy 1, policy_version 1865499 (0.0006) [2023-12-27 04:57:16,530][105620] Updated weights for policy 1, policy_version 1865509 (0.0006) [2023-12-27 04:57:16,588][105620] Updated weights for policy 1, policy_version 1865519 (0.0006) [2023-12-27 04:57:16,711][105692] Updated weights for policy 0, policy_version 1861045 (0.0010) [2023-12-27 04:57:16,760][105692] Updated weights for policy 0, policy_version 1861055 (0.0009) [2023-12-27 04:57:16,810][105692] Updated weights for policy 0, policy_version 1861065 (0.0009) [2023-12-27 04:57:17,161][105620] Updated weights for policy 1, policy_version 1865529 (0.0005) [2023-12-27 04:57:17,217][105620] Updated weights for policy 1, policy_version 1865539 (0.0006) [2023-12-27 04:57:17,270][105620] Updated weights for policy 1, policy_version 1865549 (0.0005) [2023-12-27 04:57:17,501][105692] Updated weights for policy 0, policy_version 1861075 (0.0009) [2023-12-27 04:57:17,557][105692] Updated weights for policy 0, policy_version 1861085 (0.0005) [2023-12-27 04:57:17,606][105692] Updated weights for policy 0, policy_version 1861095 (0.0005) [2023-12-27 04:57:17,798][105620] Updated weights for policy 1, policy_version 1865559 (0.0005) [2023-12-27 04:57:17,852][105620] Updated weights for policy 1, policy_version 1865569 (0.0009) [2023-12-27 04:57:17,911][105620] Updated weights for policy 1, policy_version 1865581 (0.0011) [2023-12-27 04:57:18,121][105692] Updated weights for policy 0, policy_version 1861105 (0.0005) [2023-12-27 04:57:18,179][105692] Updated weights for policy 0, policy_version 1861115 (0.0005) [2023-12-27 04:57:18,242][105692] Updated weights for policy 0, policy_version 1861125 (0.0006) [2023-12-27 04:57:18,301][105692] Updated weights for policy 0, policy_version 1861135 (0.0008) [2023-12-27 04:57:18,710][105620] Updated weights for policy 1, policy_version 1865591 (0.0007) [2023-12-27 04:57:18,769][105620] Updated weights for policy 1, policy_version 1865601 (0.0009) [2023-12-27 04:57:18,821][105620] Updated weights for policy 1, policy_version 1865611 (0.0009) [2023-12-27 04:57:18,984][105692] Updated weights for policy 0, policy_version 1861145 (0.0008) [2023-12-27 04:57:19,035][105692] Updated weights for policy 0, policy_version 1861155 (0.0009) [2023-12-27 04:57:19,103][105692] Updated weights for policy 0, policy_version 1861165 (0.0009) [2023-12-27 04:57:19,489][105620] Updated weights for policy 1, policy_version 1865621 (0.0008) [2023-12-27 04:57:19,545][105620] Updated weights for policy 1, policy_version 1865631 (0.0006) [2023-12-27 04:57:19,600][105620] Updated weights for policy 1, policy_version 1865641 (0.0009) [2023-12-27 04:57:19,894][105692] Updated weights for policy 0, policy_version 1861175 (0.0007) [2023-12-27 04:57:19,964][105692] Updated weights for policy 0, policy_version 1861185 (0.0007) [2023-12-27 04:57:20,021][105692] Updated weights for policy 0, policy_version 1861195 (0.0008) [2023-12-27 04:57:20,248][105620] Updated weights for policy 1, policy_version 1865651 (0.0008) [2023-12-27 04:57:20,304][105620] Updated weights for policy 1, policy_version 1865661 (0.0006) [2023-12-27 04:57:20,367][105620] Updated weights for policy 1, policy_version 1865671 (0.0007) [2023-12-27 04:57:20,816][105692] Updated weights for policy 0, policy_version 1861205 (0.0009) [2023-12-27 04:57:20,877][105692] Updated weights for policy 0, policy_version 1861215 (0.0010) [2023-12-27 04:57:20,922][105692] Updated weights for policy 0, policy_version 1861225 (0.0010) [2023-12-27 04:57:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19933.9, 300 sec: 19577.5). Total num frames: 954228736. Throughput: 0: 9904.8, 1: 9888.7. Samples: 954218012. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:57:21,063][104569] Avg episode reward: [(0, '8806.783'), (1, '9253.526')] [2023-12-27 04:57:21,092][105620] Updated weights for policy 1, policy_version 1865681 (0.0009) [2023-12-27 04:57:21,153][105620] Updated weights for policy 1, policy_version 1865691 (0.0008) [2023-12-27 04:57:21,210][105620] Updated weights for policy 1, policy_version 1865701 (0.0008) [2023-12-27 04:57:21,271][105620] Updated weights for policy 1, policy_version 1865711 (0.0006) [2023-12-27 04:57:21,796][105692] Updated weights for policy 0, policy_version 1861235 (0.0010) [2023-12-27 04:57:21,859][105692] Updated weights for policy 0, policy_version 1861245 (0.0009) [2023-12-27 04:57:21,926][105692] Updated weights for policy 0, policy_version 1861255 (0.0008) [2023-12-27 04:57:22,032][105620] Updated weights for policy 1, policy_version 1865721 (0.0009) [2023-12-27 04:57:22,087][105620] Updated weights for policy 1, policy_version 1865731 (0.0010) [2023-12-27 04:57:22,149][105620] Updated weights for policy 1, policy_version 1865741 (0.0009) [2023-12-27 04:57:22,646][105692] Updated weights for policy 0, policy_version 1861265 (0.0007) [2023-12-27 04:57:22,706][105692] Updated weights for policy 0, policy_version 1861275 (0.0009) [2023-12-27 04:57:22,763][105692] Updated weights for policy 0, policy_version 1861285 (0.0009) [2023-12-27 04:57:22,814][105692] Updated weights for policy 0, policy_version 1861295 (0.0009) [2023-12-27 04:57:22,942][105620] Updated weights for policy 1, policy_version 1865751 (0.0009) [2023-12-27 04:57:23,000][105620] Updated weights for policy 1, policy_version 1865761 (0.0009) [2023-12-27 04:57:23,058][105620] Updated weights for policy 1, policy_version 1865771 (0.0009) [2023-12-27 04:57:23,607][105692] Updated weights for policy 0, policy_version 1861305 (0.0009) [2023-12-27 04:57:23,665][105692] Updated weights for policy 0, policy_version 1861315 (0.0009) [2023-12-27 04:57:23,727][105692] Updated weights for policy 0, policy_version 1861325 (0.0009) [2023-12-27 04:57:23,789][105620] Updated weights for policy 1, policy_version 1865781 (0.0009) [2023-12-27 04:57:23,851][105620] Updated weights for policy 1, policy_version 1865791 (0.0009) [2023-12-27 04:57:23,912][105620] Updated weights for policy 1, policy_version 1865801 (0.0008) [2023-12-27 04:57:24,478][105692] Updated weights for policy 0, policy_version 1861335 (0.0008) [2023-12-27 04:57:24,531][105692] Updated weights for policy 0, policy_version 1861345 (0.0009) [2023-12-27 04:57:24,591][105692] Updated weights for policy 0, policy_version 1861355 (0.0009) [2023-12-27 04:57:24,665][105620] Updated weights for policy 1, policy_version 1865811 (0.0009) [2023-12-27 04:57:24,720][105620] Updated weights for policy 1, policy_version 1865821 (0.0009) [2023-12-27 04:57:24,778][105620] Updated weights for policy 1, policy_version 1865831 (0.0008) [2023-12-27 04:57:25,372][105692] Updated weights for policy 0, policy_version 1861365 (0.0009) [2023-12-27 04:57:25,427][105692] Updated weights for policy 0, policy_version 1861375 (0.0009) [2023-12-27 04:57:25,485][105692] Updated weights for policy 0, policy_version 1861385 (0.0009) [2023-12-27 04:57:25,533][105620] Updated weights for policy 1, policy_version 1865841 (0.0009) [2023-12-27 04:57:25,596][105620] Updated weights for policy 1, policy_version 1865851 (0.0009) [2023-12-27 04:57:25,654][105620] Updated weights for policy 1, policy_version 1865861 (0.0008) [2023-12-27 04:57:25,714][105620] Updated weights for policy 1, policy_version 1865871 (0.0009) [2023-12-27 04:57:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19797.2, 300 sec: 19577.5). Total num frames: 954318848. Throughput: 0: 9774.9, 1: 9679.2. Samples: 954328084. Policy #0 lag: (min: 27.0, avg: 43.9, max: 59.0) [2023-12-27 04:57:26,063][104569] Avg episode reward: [(0, '8897.637'), (1, '9253.403')] [2023-12-27 04:57:26,247][105692] Updated weights for policy 0, policy_version 1861395 (0.0008) [2023-12-27 04:57:26,302][105692] Updated weights for policy 0, policy_version 1861405 (0.0009) [2023-12-27 04:57:26,354][105692] Updated weights for policy 0, policy_version 1861415 (0.0009) [2023-12-27 04:57:26,456][105620] Updated weights for policy 1, policy_version 1865881 (0.0009) [2023-12-27 04:57:26,513][105620] Updated weights for policy 1, policy_version 1865891 (0.0008) [2023-12-27 04:57:26,570][105620] Updated weights for policy 1, policy_version 1865901 (0.0009) [2023-12-27 04:57:27,137][105692] Updated weights for policy 0, policy_version 1861425 (0.0009) [2023-12-27 04:57:27,202][105692] Updated weights for policy 0, policy_version 1861435 (0.0009) [2023-12-27 04:57:27,250][105692] Updated weights for policy 0, policy_version 1861445 (0.0009) [2023-12-27 04:57:27,295][105692] Updated weights for policy 0, policy_version 1861455 (0.0008) [2023-12-27 04:57:27,315][105620] Updated weights for policy 1, policy_version 1865911 (0.0008) [2023-12-27 04:57:27,368][105620] Updated weights for policy 1, policy_version 1865921 (0.0008) [2023-12-27 04:57:27,415][105620] Updated weights for policy 1, policy_version 1865931 (0.0008) [2023-12-27 04:57:28,068][105692] Updated weights for policy 0, policy_version 1861465 (0.0009) [2023-12-27 04:57:28,115][105692] Updated weights for policy 0, policy_version 1861475 (0.0008) [2023-12-27 04:57:28,171][105692] Updated weights for policy 0, policy_version 1861485 (0.0009) [2023-12-27 04:57:28,182][105620] Updated weights for policy 1, policy_version 1865941 (0.0008) [2023-12-27 04:57:28,234][105620] Updated weights for policy 1, policy_version 1865951 (0.0008) [2023-12-27 04:57:28,286][105620] Updated weights for policy 1, policy_version 1865961 (0.0009) [2023-12-27 04:57:28,979][105692] Updated weights for policy 0, policy_version 1861495 (0.0009) [2023-12-27 04:57:28,986][105620] Updated weights for policy 1, policy_version 1865971 (0.0008) [2023-12-27 04:57:29,031][105692] Updated weights for policy 0, policy_version 1861505 (0.0008) [2023-12-27 04:57:29,038][105620] Updated weights for policy 1, policy_version 1865981 (0.0008) [2023-12-27 04:57:29,080][105692] Updated weights for policy 0, policy_version 1861515 (0.0006) [2023-12-27 04:57:29,091][105620] Updated weights for policy 1, policy_version 1865991 (0.0008) [2023-12-27 04:57:29,851][105620] Updated weights for policy 1, policy_version 1866001 (0.0008) [2023-12-27 04:57:29,855][105692] Updated weights for policy 0, policy_version 1861525 (0.0006) [2023-12-27 04:57:29,907][105620] Updated weights for policy 1, policy_version 1866011 (0.0007) [2023-12-27 04:57:29,911][105692] Updated weights for policy 0, policy_version 1861535 (0.0006) [2023-12-27 04:57:29,973][105692] Updated weights for policy 0, policy_version 1861545 (0.0007) [2023-12-27 04:57:29,975][105620] Updated weights for policy 1, policy_version 1866021 (0.0009) [2023-12-27 04:57:30,033][105620] Updated weights for policy 1, policy_version 1866031 (0.0008) [2023-12-27 04:57:30,625][105692] Updated weights for policy 0, policy_version 1861555 (0.0008) [2023-12-27 04:57:30,678][105692] Updated weights for policy 0, policy_version 1861565 (0.0010) [2023-12-27 04:57:30,736][105692] Updated weights for policy 0, policy_version 1861575 (0.0010) [2023-12-27 04:57:30,830][105620] Updated weights for policy 1, policy_version 1866041 (0.0008) [2023-12-27 04:57:30,890][105620] Updated weights for policy 1, policy_version 1866051 (0.0008) [2023-12-27 04:57:30,945][105620] Updated weights for policy 1, policy_version 1866061 (0.0006) [2023-12-27 04:57:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 954417152. Throughput: 0: 9747.9, 1: 9668.4. Samples: 954384804. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:57:31,062][104569] Avg episode reward: [(0, '8535.563'), (1, '9253.432')] [2023-12-27 04:57:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001861584_476635136.pth... [2023-12-27 04:57:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001866064_477782016.pth... [2023-12-27 04:57:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001860464_476348416.pth [2023-12-27 04:57:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001864912_477487104.pth [2023-12-27 04:57:31,465][105692] Updated weights for policy 0, policy_version 1861585 (0.0010) [2023-12-27 04:57:31,524][105692] Updated weights for policy 0, policy_version 1861595 (0.0005) [2023-12-27 04:57:31,578][105692] Updated weights for policy 0, policy_version 1861605 (0.0005) [2023-12-27 04:57:31,647][105692] Updated weights for policy 0, policy_version 1861615 (0.0006) [2023-12-27 04:57:31,737][105620] Updated weights for policy 1, policy_version 1866071 (0.0009) [2023-12-27 04:57:31,793][105620] Updated weights for policy 1, policy_version 1866081 (0.0010) [2023-12-27 04:57:31,848][105620] Updated weights for policy 1, policy_version 1866091 (0.0010) [2023-12-27 04:57:32,213][105692] Updated weights for policy 0, policy_version 1861625 (0.0006) [2023-12-27 04:57:32,274][105692] Updated weights for policy 0, policy_version 1861635 (0.0007) [2023-12-27 04:57:32,325][105692] Updated weights for policy 0, policy_version 1861645 (0.0008) [2023-12-27 04:57:32,615][105620] Updated weights for policy 1, policy_version 1866101 (0.0010) [2023-12-27 04:57:32,671][105620] Updated weights for policy 1, policy_version 1866111 (0.0010) [2023-12-27 04:57:32,732][105620] Updated weights for policy 1, policy_version 1866121 (0.0010) [2023-12-27 04:57:33,074][105692] Updated weights for policy 0, policy_version 1861655 (0.0008) [2023-12-27 04:57:33,126][105692] Updated weights for policy 0, policy_version 1861665 (0.0008) [2023-12-27 04:57:33,180][105692] Updated weights for policy 0, policy_version 1861675 (0.0008) [2023-12-27 04:57:33,442][105620] Updated weights for policy 1, policy_version 1866131 (0.0010) [2023-12-27 04:57:33,488][105620] Updated weights for policy 1, policy_version 1866141 (0.0009) [2023-12-27 04:57:33,532][105620] Updated weights for policy 1, policy_version 1866151 (0.0010) [2023-12-27 04:57:33,892][105692] Updated weights for policy 0, policy_version 1861685 (0.0007) [2023-12-27 04:57:33,938][105692] Updated weights for policy 0, policy_version 1861695 (0.0006) [2023-12-27 04:57:33,992][105692] Updated weights for policy 0, policy_version 1861705 (0.0005) [2023-12-27 04:57:34,220][105620] Updated weights for policy 1, policy_version 1866161 (0.0007) [2023-12-27 04:57:34,272][105620] Updated weights for policy 1, policy_version 1866171 (0.0011) [2023-12-27 04:57:34,325][105620] Updated weights for policy 1, policy_version 1866181 (0.0010) [2023-12-27 04:57:34,374][105620] Updated weights for policy 1, policy_version 1866191 (0.0010) [2023-12-27 04:57:34,674][105692] Updated weights for policy 0, policy_version 1861715 (0.0007) [2023-12-27 04:57:34,727][105692] Updated weights for policy 0, policy_version 1861725 (0.0011) [2023-12-27 04:57:34,784][105692] Updated weights for policy 0, policy_version 1861735 (0.0009) [2023-12-27 04:57:35,185][105620] Updated weights for policy 1, policy_version 1866201 (0.0008) [2023-12-27 04:57:35,242][105620] Updated weights for policy 1, policy_version 1866211 (0.0010) [2023-12-27 04:57:35,306][105620] Updated weights for policy 1, policy_version 1866221 (0.0009) [2023-12-27 04:57:35,374][105692] Updated weights for policy 0, policy_version 1861745 (0.0006) [2023-12-27 04:57:35,420][105692] Updated weights for policy 0, policy_version 1861755 (0.0005) [2023-12-27 04:57:35,486][105692] Updated weights for policy 0, policy_version 1861765 (0.0005) [2023-12-27 04:57:35,535][105692] Updated weights for policy 0, policy_version 1861775 (0.0006) [2023-12-27 04:57:36,012][105620] Updated weights for policy 1, policy_version 1866231 (0.0008) [2023-12-27 04:57:36,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 954507264. Throughput: 0: 9591.4, 1: 9724.6. Samples: 954500104. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:57:36,062][104569] Avg episode reward: [(0, '8445.354'), (1, '9345.863')] [2023-12-27 04:57:36,072][105620] Updated weights for policy 1, policy_version 1866241 (0.0008) [2023-12-27 04:57:36,099][105692] Updated weights for policy 0, policy_version 1861785 (0.0006) [2023-12-27 04:57:36,133][105620] Updated weights for policy 1, policy_version 1866251 (0.0007) [2023-12-27 04:57:36,164][105692] Updated weights for policy 0, policy_version 1861795 (0.0008) [2023-12-27 04:57:36,228][105692] Updated weights for policy 0, policy_version 1861805 (0.0009) [2023-12-27 04:57:36,748][105620] Updated weights for policy 1, policy_version 1866261 (0.0006) [2023-12-27 04:57:36,795][105620] Updated weights for policy 1, policy_version 1866271 (0.0006) [2023-12-27 04:57:36,844][105620] Updated weights for policy 1, policy_version 1866281 (0.0005) [2023-12-27 04:57:36,913][105692] Updated weights for policy 0, policy_version 1861815 (0.0009) [2023-12-27 04:57:36,976][105692] Updated weights for policy 0, policy_version 1861825 (0.0010) [2023-12-27 04:57:37,041][105692] Updated weights for policy 0, policy_version 1861835 (0.0011) [2023-12-27 04:57:37,475][105620] Updated weights for policy 1, policy_version 1866291 (0.0006) [2023-12-27 04:57:37,527][105620] Updated weights for policy 1, policy_version 1866301 (0.0008) [2023-12-27 04:57:37,586][105620] Updated weights for policy 1, policy_version 1866311 (0.0008) [2023-12-27 04:57:37,779][105692] Updated weights for policy 0, policy_version 1861845 (0.0010) [2023-12-27 04:57:37,827][105692] Updated weights for policy 0, policy_version 1861855 (0.0009) [2023-12-27 04:57:37,881][105692] Updated weights for policy 0, policy_version 1861865 (0.0008) [2023-12-27 04:57:38,395][105620] Updated weights for policy 1, policy_version 1866321 (0.0008) [2023-12-27 04:57:38,458][105620] Updated weights for policy 1, policy_version 1866331 (0.0009) [2023-12-27 04:57:38,520][105620] Updated weights for policy 1, policy_version 1866341 (0.0009) [2023-12-27 04:57:38,548][105692] Updated weights for policy 0, policy_version 1861875 (0.0008) [2023-12-27 04:57:38,573][105620] Updated weights for policy 1, policy_version 1866351 (0.0008) [2023-12-27 04:57:38,602][105692] Updated weights for policy 0, policy_version 1861885 (0.0007) [2023-12-27 04:57:38,665][105692] Updated weights for policy 0, policy_version 1861895 (0.0009) [2023-12-27 04:57:39,267][105620] Updated weights for policy 1, policy_version 1866361 (0.0008) [2023-12-27 04:57:39,330][105620] Updated weights for policy 1, policy_version 1866371 (0.0007) [2023-12-27 04:57:39,396][105620] Updated weights for policy 1, policy_version 1866381 (0.0009) [2023-12-27 04:57:39,523][105692] Updated weights for policy 0, policy_version 1861905 (0.0009) [2023-12-27 04:57:39,578][105692] Updated weights for policy 0, policy_version 1861915 (0.0010) [2023-12-27 04:57:39,635][105692] Updated weights for policy 0, policy_version 1861925 (0.0010) [2023-12-27 04:57:39,689][105692] Updated weights for policy 0, policy_version 1861935 (0.0010) [2023-12-27 04:57:40,031][105620] Updated weights for policy 1, policy_version 1866391 (0.0009) [2023-12-27 04:57:40,096][105620] Updated weights for policy 1, policy_version 1866401 (0.0008) [2023-12-27 04:57:40,158][105620] Updated weights for policy 1, policy_version 1866411 (0.0009) [2023-12-27 04:57:40,522][105692] Updated weights for policy 0, policy_version 1861945 (0.0009) [2023-12-27 04:57:40,572][105692] Updated weights for policy 0, policy_version 1861955 (0.0009) [2023-12-27 04:57:40,632][105692] Updated weights for policy 0, policy_version 1861965 (0.0009) [2023-12-27 04:57:40,881][105620] Updated weights for policy 1, policy_version 1866421 (0.0008) [2023-12-27 04:57:40,936][105620] Updated weights for policy 1, policy_version 1866432 (0.0010) [2023-12-27 04:57:40,990][105620] Updated weights for policy 1, policy_version 1866442 (0.0010) [2023-12-27 04:57:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 954613760. Throughput: 0: 9687.4, 1: 9703.9. Samples: 954618436. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:57:41,062][104569] Avg episode reward: [(0, '8628.348'), (1, '9345.875')] [2023-12-27 04:57:41,376][105692] Updated weights for policy 0, policy_version 1861975 (0.0008) [2023-12-27 04:57:41,437][105692] Updated weights for policy 0, policy_version 1861985 (0.0006) [2023-12-27 04:57:41,493][105692] Updated weights for policy 0, policy_version 1861995 (0.0006) [2023-12-27 04:57:41,776][105620] Updated weights for policy 1, policy_version 1866452 (0.0009) [2023-12-27 04:57:41,841][105620] Updated weights for policy 1, policy_version 1866462 (0.0006) [2023-12-27 04:57:41,896][105620] Updated weights for policy 1, policy_version 1866472 (0.0009) [2023-12-27 04:57:42,193][105692] Updated weights for policy 0, policy_version 1862005 (0.0008) [2023-12-27 04:57:42,250][105692] Updated weights for policy 0, policy_version 1862015 (0.0009) [2023-12-27 04:57:42,306][105692] Updated weights for policy 0, policy_version 1862025 (0.0009) [2023-12-27 04:57:42,586][105620] Updated weights for policy 1, policy_version 1866482 (0.0009) [2023-12-27 04:57:42,652][105620] Updated weights for policy 1, policy_version 1866492 (0.0007) [2023-12-27 04:57:42,714][105620] Updated weights for policy 1, policy_version 1866502 (0.0007) [2023-12-27 04:57:42,777][105620] Updated weights for policy 1, policy_version 1866512 (0.0005) [2023-12-27 04:57:43,185][105692] Updated weights for policy 0, policy_version 1862035 (0.0010) [2023-12-27 04:57:43,240][105692] Updated weights for policy 0, policy_version 1862047 (0.0011) [2023-12-27 04:57:43,304][105692] Updated weights for policy 0, policy_version 1862058 (0.0010) [2023-12-27 04:57:43,374][105620] Updated weights for policy 1, policy_version 1866522 (0.0006) [2023-12-27 04:57:43,429][105620] Updated weights for policy 1, policy_version 1866532 (0.0009) [2023-12-27 04:57:43,485][105620] Updated weights for policy 1, policy_version 1866542 (0.0010) [2023-12-27 04:57:44,076][105692] Updated weights for policy 0, policy_version 1862068 (0.0008) [2023-12-27 04:57:44,131][105692] Updated weights for policy 0, policy_version 1862078 (0.0007) [2023-12-27 04:57:44,131][105620] Updated weights for policy 1, policy_version 1866552 (0.0009) [2023-12-27 04:57:44,194][105620] Updated weights for policy 1, policy_version 1866562 (0.0007) [2023-12-27 04:57:44,195][105692] Updated weights for policy 0, policy_version 1862088 (0.0009) [2023-12-27 04:57:44,257][105620] Updated weights for policy 1, policy_version 1866572 (0.0005) [2023-12-27 04:57:44,923][105620] Updated weights for policy 1, policy_version 1866582 (0.0007) [2023-12-27 04:57:44,980][105692] Updated weights for policy 0, policy_version 1862098 (0.0008) [2023-12-27 04:57:44,988][105620] Updated weights for policy 1, policy_version 1866592 (0.0007) [2023-12-27 04:57:45,039][105692] Updated weights for policy 0, policy_version 1862108 (0.0007) [2023-12-27 04:57:45,053][105620] Updated weights for policy 1, policy_version 1866602 (0.0008) [2023-12-27 04:57:45,091][105692] Updated weights for policy 0, policy_version 1862118 (0.0010) [2023-12-27 04:57:45,165][105692] Updated weights for policy 0, policy_version 1862128 (0.0011) [2023-12-27 04:57:45,771][105620] Updated weights for policy 1, policy_version 1866612 (0.0008) [2023-12-27 04:57:45,815][105620] Updated weights for policy 1, policy_version 1866622 (0.0007) [2023-12-27 04:57:45,864][105620] Updated weights for policy 1, policy_version 1866632 (0.0006) [2023-12-27 04:57:45,893][105692] Updated weights for policy 0, policy_version 1862138 (0.0011) [2023-12-27 04:57:45,947][105692] Updated weights for policy 0, policy_version 1862148 (0.0010) [2023-12-27 04:57:45,996][105692] Updated weights for policy 0, policy_version 1862158 (0.0006) [2023-12-27 04:57:46,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 954712064. Throughput: 0: 9589.9, 1: 9732.5. Samples: 954676188. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:57:46,062][104569] Avg episode reward: [(0, '8448.144'), (1, '9345.797')] [2023-12-27 04:57:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001862160_476782592.pth... [2023-12-27 04:57:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001866640_477929472.pth... [2023-12-27 04:57:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001861040_476495872.pth [2023-12-27 04:57:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001865488_477634560.pth [2023-12-27 04:57:46,542][105620] Updated weights for policy 1, policy_version 1866642 (0.0009) [2023-12-27 04:57:46,598][105620] Updated weights for policy 1, policy_version 1866652 (0.0005) [2023-12-27 04:57:46,658][105620] Updated weights for policy 1, policy_version 1866662 (0.0011) [2023-12-27 04:57:46,670][105692] Updated weights for policy 0, policy_version 1862168 (0.0009) [2023-12-27 04:57:46,716][105620] Updated weights for policy 1, policy_version 1866672 (0.0010) [2023-12-27 04:57:46,728][105692] Updated weights for policy 0, policy_version 1862178 (0.0010) [2023-12-27 04:57:46,786][105692] Updated weights for policy 0, policy_version 1862188 (0.0010) [2023-12-27 04:57:47,298][105620] Updated weights for policy 1, policy_version 1866682 (0.0008) [2023-12-27 04:57:47,353][105620] Updated weights for policy 1, policy_version 1866692 (0.0010) [2023-12-27 04:57:47,405][105620] Updated weights for policy 1, policy_version 1866702 (0.0009) [2023-12-27 04:57:47,520][105692] Updated weights for policy 0, policy_version 1862198 (0.0010) [2023-12-27 04:57:47,581][105692] Updated weights for policy 0, policy_version 1862208 (0.0010) [2023-12-27 04:57:47,639][105692] Updated weights for policy 0, policy_version 1862218 (0.0010) [2023-12-27 04:57:48,115][105620] Updated weights for policy 1, policy_version 1866712 (0.0010) [2023-12-27 04:57:48,175][105620] Updated weights for policy 1, policy_version 1866722 (0.0010) [2023-12-27 04:57:48,231][105620] Updated weights for policy 1, policy_version 1866732 (0.0010) [2023-12-27 04:57:48,364][105692] Updated weights for policy 0, policy_version 1862228 (0.0011) [2023-12-27 04:57:48,422][105692] Updated weights for policy 0, policy_version 1862238 (0.0009) [2023-12-27 04:57:48,489][105692] Updated weights for policy 0, policy_version 1862248 (0.0011) [2023-12-27 04:57:48,988][105620] Updated weights for policy 1, policy_version 1866742 (0.0010) [2023-12-27 04:57:49,047][105620] Updated weights for policy 1, policy_version 1866752 (0.0010) [2023-12-27 04:57:49,107][105620] Updated weights for policy 1, policy_version 1866762 (0.0009) [2023-12-27 04:57:49,218][105692] Updated weights for policy 0, policy_version 1862258 (0.0010) [2023-12-27 04:57:49,276][105692] Updated weights for policy 0, policy_version 1862268 (0.0009) [2023-12-27 04:57:49,330][105692] Updated weights for policy 0, policy_version 1862278 (0.0007) [2023-12-27 04:57:49,393][105692] Updated weights for policy 0, policy_version 1862288 (0.0006) [2023-12-27 04:57:49,841][105620] Updated weights for policy 1, policy_version 1866772 (0.0008) [2023-12-27 04:57:49,909][105620] Updated weights for policy 1, policy_version 1866782 (0.0011) [2023-12-27 04:57:49,976][105620] Updated weights for policy 1, policy_version 1866792 (0.0010) [2023-12-27 04:57:49,976][105692] Updated weights for policy 0, policy_version 1862298 (0.0007) [2023-12-27 04:57:50,036][105692] Updated weights for policy 0, policy_version 1862308 (0.0009) [2023-12-27 04:57:50,088][105692] Updated weights for policy 0, policy_version 1862318 (0.0008) [2023-12-27 04:57:50,714][105620] Updated weights for policy 1, policy_version 1866802 (0.0011) [2023-12-27 04:57:50,771][105620] Updated weights for policy 1, policy_version 1866812 (0.0011) [2023-12-27 04:57:50,817][105692] Updated weights for policy 0, policy_version 1862328 (0.0006) [2023-12-27 04:57:50,824][105620] Updated weights for policy 1, policy_version 1866822 (0.0011) [2023-12-27 04:57:50,878][105692] Updated weights for policy 0, policy_version 1862338 (0.0006) [2023-12-27 04:57:50,887][105620] Updated weights for policy 1, policy_version 1866832 (0.0011) [2023-12-27 04:57:50,933][105692] Updated weights for policy 0, policy_version 1862348 (0.0008) [2023-12-27 04:57:51,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 954810368. Throughput: 0: 9591.3, 1: 9769.1. Samples: 954793736. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:57:51,063][104569] Avg episode reward: [(0, '7992.674'), (1, '9345.738')] [2023-12-27 04:57:51,647][105620] Updated weights for policy 1, policy_version 1866842 (0.0007) [2023-12-27 04:57:51,684][105692] Updated weights for policy 0, policy_version 1862358 (0.0008) [2023-12-27 04:57:51,710][105620] Updated weights for policy 1, policy_version 1866852 (0.0008) [2023-12-27 04:57:51,750][105692] Updated weights for policy 0, policy_version 1862368 (0.0008) [2023-12-27 04:57:51,774][105620] Updated weights for policy 1, policy_version 1866862 (0.0008) [2023-12-27 04:57:51,809][105692] Updated weights for policy 0, policy_version 1862378 (0.0007) [2023-12-27 04:57:52,514][105620] Updated weights for policy 1, policy_version 1866872 (0.0010) [2023-12-27 04:57:52,540][105692] Updated weights for policy 0, policy_version 1862388 (0.0006) [2023-12-27 04:57:52,566][105620] Updated weights for policy 1, policy_version 1866882 (0.0010) [2023-12-27 04:57:52,595][105692] Updated weights for policy 0, policy_version 1862398 (0.0009) [2023-12-27 04:57:52,618][105620] Updated weights for policy 1, policy_version 1866892 (0.0010) [2023-12-27 04:57:52,648][105692] Updated weights for policy 0, policy_version 1862408 (0.0006) [2023-12-27 04:57:53,388][105620] Updated weights for policy 1, policy_version 1866902 (0.0010) [2023-12-27 04:57:53,420][105692] Updated weights for policy 0, policy_version 1862418 (0.0008) [2023-12-27 04:57:53,432][105620] Updated weights for policy 1, policy_version 1866912 (0.0010) [2023-12-27 04:57:53,474][105692] Updated weights for policy 0, policy_version 1862428 (0.0008) [2023-12-27 04:57:53,492][105620] Updated weights for policy 1, policy_version 1866922 (0.0011) [2023-12-27 04:57:53,531][105692] Updated weights for policy 0, policy_version 1862438 (0.0006) [2023-12-27 04:57:53,590][105692] Updated weights for policy 0, policy_version 1862448 (0.0007) [2023-12-27 04:57:54,148][105620] Updated weights for policy 1, policy_version 1866932 (0.0008) [2023-12-27 04:57:54,208][105620] Updated weights for policy 1, policy_version 1866942 (0.0006) [2023-12-27 04:57:54,260][105620] Updated weights for policy 1, policy_version 1866952 (0.0010) [2023-12-27 04:57:54,270][105692] Updated weights for policy 0, policy_version 1862458 (0.0007) [2023-12-27 04:57:54,320][105692] Updated weights for policy 0, policy_version 1862468 (0.0007) [2023-12-27 04:57:54,380][105692] Updated weights for policy 0, policy_version 1862478 (0.0008) [2023-12-27 04:57:54,984][105620] Updated weights for policy 1, policy_version 1866962 (0.0011) [2023-12-27 04:57:55,051][105620] Updated weights for policy 1, policy_version 1866972 (0.0011) [2023-12-27 04:57:55,110][105620] Updated weights for policy 1, policy_version 1866982 (0.0010) [2023-12-27 04:57:55,148][105692] Updated weights for policy 0, policy_version 1862488 (0.0009) [2023-12-27 04:57:55,162][105620] Updated weights for policy 1, policy_version 1866992 (0.0010) [2023-12-27 04:57:55,209][105692] Updated weights for policy 0, policy_version 1862498 (0.0008) [2023-12-27 04:57:55,268][105692] Updated weights for policy 0, policy_version 1862508 (0.0008) [2023-12-27 04:57:55,844][105620] Updated weights for policy 1, policy_version 1867002 (0.0005) [2023-12-27 04:57:55,899][105620] Updated weights for policy 1, policy_version 1867012 (0.0005) [2023-12-27 04:57:55,922][105692] Updated weights for policy 0, policy_version 1862518 (0.0006) [2023-12-27 04:57:55,955][105620] Updated weights for policy 1, policy_version 1867022 (0.0006) [2023-12-27 04:57:55,976][105692] Updated weights for policy 0, policy_version 1862528 (0.0005) [2023-12-27 04:57:56,032][105692] Updated weights for policy 0, policy_version 1862538 (0.0005) [2023-12-27 04:57:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 954900480. Throughput: 0: 9536.9, 1: 9772.2. Samples: 954908720. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:57:56,063][104569] Avg episode reward: [(0, '8265.121'), (1, '9254.356')] [2023-12-27 04:57:56,628][105692] Updated weights for policy 0, policy_version 1862548 (0.0007) [2023-12-27 04:57:56,677][105620] Updated weights for policy 1, policy_version 1867032 (0.0006) [2023-12-27 04:57:56,688][105692] Updated weights for policy 0, policy_version 1862558 (0.0005) [2023-12-27 04:57:56,731][105620] Updated weights for policy 1, policy_version 1867042 (0.0005) [2023-12-27 04:57:56,737][105692] Updated weights for policy 0, policy_version 1862568 (0.0005) [2023-12-27 04:57:56,786][105620] Updated weights for policy 1, policy_version 1867052 (0.0005) [2023-12-27 04:57:57,238][105692] Updated weights for policy 0, policy_version 1862578 (0.0005) [2023-12-27 04:57:57,286][105620] Updated weights for policy 1, policy_version 1867062 (0.0006) [2023-12-27 04:57:57,303][105692] Updated weights for policy 0, policy_version 1862588 (0.0006) [2023-12-27 04:57:57,355][105620] Updated weights for policy 1, policy_version 1867072 (0.0007) [2023-12-27 04:57:57,366][105692] Updated weights for policy 0, policy_version 1862598 (0.0007) [2023-12-27 04:57:57,411][105620] Updated weights for policy 1, policy_version 1867082 (0.0009) [2023-12-27 04:57:57,418][105692] Updated weights for policy 0, policy_version 1862608 (0.0006) [2023-12-27 04:57:57,970][105620] Updated weights for policy 1, policy_version 1867092 (0.0006) [2023-12-27 04:57:58,026][105620] Updated weights for policy 1, policy_version 1867102 (0.0009) [2023-12-27 04:57:58,062][105692] Updated weights for policy 0, policy_version 1862618 (0.0006) [2023-12-27 04:57:58,083][105620] Updated weights for policy 1, policy_version 1867112 (0.0009) [2023-12-27 04:57:58,112][105692] Updated weights for policy 0, policy_version 1862628 (0.0007) [2023-12-27 04:57:58,165][105692] Updated weights for policy 0, policy_version 1862638 (0.0008) [2023-12-27 04:57:58,909][105620] Updated weights for policy 1, policy_version 1867122 (0.0008) [2023-12-27 04:57:58,982][105620] Updated weights for policy 1, policy_version 1867132 (0.0010) [2023-12-27 04:57:59,030][105692] Updated weights for policy 0, policy_version 1862648 (0.0007) [2023-12-27 04:57:59,036][105620] Updated weights for policy 1, policy_version 1867142 (0.0011) [2023-12-27 04:57:59,090][105692] Updated weights for policy 0, policy_version 1862658 (0.0007) [2023-12-27 04:57:59,098][105620] Updated weights for policy 1, policy_version 1867152 (0.0010) [2023-12-27 04:57:59,158][105692] Updated weights for policy 0, policy_version 1862668 (0.0009) [2023-12-27 04:57:59,801][105692] Updated weights for policy 0, policy_version 1862678 (0.0006) [2023-12-27 04:57:59,842][105620] Updated weights for policy 1, policy_version 1867162 (0.0011) [2023-12-27 04:57:59,861][105692] Updated weights for policy 0, policy_version 1862688 (0.0008) [2023-12-27 04:57:59,901][105620] Updated weights for policy 1, policy_version 1867172 (0.0011) [2023-12-27 04:57:59,922][105692] Updated weights for policy 0, policy_version 1862698 (0.0007) [2023-12-27 04:57:59,958][105620] Updated weights for policy 1, policy_version 1867182 (0.0008) [2023-12-27 04:58:00,551][105620] Updated weights for policy 1, policy_version 1867192 (0.0009) [2023-12-27 04:58:00,610][105620] Updated weights for policy 1, policy_version 1867202 (0.0008) [2023-12-27 04:58:00,643][105692] Updated weights for policy 0, policy_version 1862708 (0.0008) [2023-12-27 04:58:00,671][105620] Updated weights for policy 1, policy_version 1867212 (0.0005) [2023-12-27 04:58:00,699][105692] Updated weights for policy 0, policy_version 1862718 (0.0005) [2023-12-27 04:58:00,754][105692] Updated weights for policy 0, policy_version 1862728 (0.0006) [2023-12-27 04:58:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19605.2). Total num frames: 955006976. Throughput: 0: 9659.0, 1: 9841.3. Samples: 954974004. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:58:01,063][104569] Avg episode reward: [(0, '8078.443'), (1, '8700.919')] [2023-12-27 04:58:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001862736_476930048.pth... [2023-12-27 04:58:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001867216_478076928.pth... [2023-12-27 04:58:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001861584_476635136.pth [2023-12-27 04:58:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001866064_477782016.pth [2023-12-27 04:58:01,302][105620] Updated weights for policy 1, policy_version 1867222 (0.0008) [2023-12-27 04:58:01,353][105620] Updated weights for policy 1, policy_version 1867232 (0.0009) [2023-12-27 04:58:01,414][105692] Updated weights for policy 0, policy_version 1862738 (0.0009) [2023-12-27 04:58:01,418][105620] Updated weights for policy 1, policy_version 1867242 (0.0008) [2023-12-27 04:58:01,478][105692] Updated weights for policy 0, policy_version 1862748 (0.0007) [2023-12-27 04:58:01,539][105692] Updated weights for policy 0, policy_version 1862758 (0.0009) [2023-12-27 04:58:01,596][105692] Updated weights for policy 0, policy_version 1862768 (0.0010) [2023-12-27 04:58:02,160][105620] Updated weights for policy 1, policy_version 1867252 (0.0009) [2023-12-27 04:58:02,218][105620] Updated weights for policy 1, policy_version 1867262 (0.0010) [2023-12-27 04:58:02,273][105620] Updated weights for policy 1, policy_version 1867273 (0.0009) [2023-12-27 04:58:02,357][105692] Updated weights for policy 0, policy_version 1862778 (0.0008) [2023-12-27 04:58:02,423][105692] Updated weights for policy 0, policy_version 1862788 (0.0006) [2023-12-27 04:58:02,486][105692] Updated weights for policy 0, policy_version 1862798 (0.0005) [2023-12-27 04:58:03,094][105692] Updated weights for policy 0, policy_version 1862808 (0.0006) [2023-12-27 04:58:03,108][105620] Updated weights for policy 1, policy_version 1867283 (0.0008) [2023-12-27 04:58:03,157][105692] Updated weights for policy 0, policy_version 1862818 (0.0005) [2023-12-27 04:58:03,161][105620] Updated weights for policy 1, policy_version 1867293 (0.0009) [2023-12-27 04:58:03,210][105620] Updated weights for policy 1, policy_version 1867303 (0.0009) [2023-12-27 04:58:03,218][105692] Updated weights for policy 0, policy_version 1862828 (0.0005) [2023-12-27 04:58:03,902][105692] Updated weights for policy 0, policy_version 1862838 (0.0007) [2023-12-27 04:58:03,917][105620] Updated weights for policy 1, policy_version 1867313 (0.0009) [2023-12-27 04:58:03,961][105692] Updated weights for policy 0, policy_version 1862848 (0.0006) [2023-12-27 04:58:03,973][105620] Updated weights for policy 1, policy_version 1867323 (0.0009) [2023-12-27 04:58:04,024][105692] Updated weights for policy 0, policy_version 1862858 (0.0007) [2023-12-27 04:58:04,034][105620] Updated weights for policy 1, policy_version 1867333 (0.0008) [2023-12-27 04:58:04,094][105620] Updated weights for policy 1, policy_version 1867343 (0.0007) [2023-12-27 04:58:04,802][105620] Updated weights for policy 1, policy_version 1867353 (0.0008) [2023-12-27 04:58:04,804][105692] Updated weights for policy 0, policy_version 1862868 (0.0009) [2023-12-27 04:58:04,848][105620] Updated weights for policy 1, policy_version 1867363 (0.0005) [2023-12-27 04:58:04,853][105692] Updated weights for policy 0, policy_version 1862878 (0.0007) [2023-12-27 04:58:04,900][105620] Updated weights for policy 1, policy_version 1867373 (0.0007) [2023-12-27 04:58:04,902][105692] Updated weights for policy 0, policy_version 1862888 (0.0006) [2023-12-27 04:58:05,529][105692] Updated weights for policy 0, policy_version 1862898 (0.0007) [2023-12-27 04:58:05,579][105692] Updated weights for policy 0, policy_version 1862908 (0.0006) [2023-12-27 04:58:05,638][105692] Updated weights for policy 0, policy_version 1862918 (0.0005) [2023-12-27 04:58:05,701][105692] Updated weights for policy 0, policy_version 1862928 (0.0006) [2023-12-27 04:58:05,717][105620] Updated weights for policy 1, policy_version 1867383 (0.0008) [2023-12-27 04:58:05,768][105620] Updated weights for policy 1, policy_version 1867393 (0.0008) [2023-12-27 04:58:05,828][105620] Updated weights for policy 1, policy_version 1867403 (0.0006) [2023-12-27 04:58:06,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 955105280. Throughput: 0: 9660.4, 1: 9740.7. Samples: 955091060. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:58:06,062][104569] Avg episode reward: [(0, '8079.492'), (1, '8884.405')] [2023-12-27 04:58:06,449][105692] Updated weights for policy 0, policy_version 1862938 (0.0008) [2023-12-27 04:58:06,496][105620] Updated weights for policy 1, policy_version 1867413 (0.0007) [2023-12-27 04:58:06,502][105692] Updated weights for policy 0, policy_version 1862948 (0.0007) [2023-12-27 04:58:06,547][105620] Updated weights for policy 1, policy_version 1867423 (0.0009) [2023-12-27 04:58:06,553][105692] Updated weights for policy 0, policy_version 1862958 (0.0008) [2023-12-27 04:58:06,604][105620] Updated weights for policy 1, policy_version 1867433 (0.0008) [2023-12-27 04:58:07,267][105620] Updated weights for policy 1, policy_version 1867443 (0.0009) [2023-12-27 04:58:07,335][105620] Updated weights for policy 1, policy_version 1867453 (0.0009) [2023-12-27 04:58:07,386][105692] Updated weights for policy 0, policy_version 1862968 (0.0008) [2023-12-27 04:58:07,387][105620] Updated weights for policy 1, policy_version 1867463 (0.0005) [2023-12-27 04:58:07,442][105692] Updated weights for policy 0, policy_version 1862978 (0.0009) [2023-12-27 04:58:07,500][105692] Updated weights for policy 0, policy_version 1862988 (0.0006) [2023-12-27 04:58:08,010][105620] Updated weights for policy 1, policy_version 1867473 (0.0006) [2023-12-27 04:58:08,068][105620] Updated weights for policy 1, policy_version 1867483 (0.0010) [2023-12-27 04:58:08,092][105692] Updated weights for policy 0, policy_version 1862998 (0.0005) [2023-12-27 04:58:08,127][105620] Updated weights for policy 1, policy_version 1867493 (0.0007) [2023-12-27 04:58:08,142][105692] Updated weights for policy 0, policy_version 1863008 (0.0005) [2023-12-27 04:58:08,185][105620] Updated weights for policy 1, policy_version 1867503 (0.0005) [2023-12-27 04:58:08,194][105692] Updated weights for policy 0, policy_version 1863018 (0.0005) [2023-12-27 04:58:08,787][105620] Updated weights for policy 1, policy_version 1867513 (0.0010) [2023-12-27 04:58:08,841][105692] Updated weights for policy 0, policy_version 1863028 (0.0005) [2023-12-27 04:58:08,843][105620] Updated weights for policy 1, policy_version 1867523 (0.0010) [2023-12-27 04:58:08,891][105620] Updated weights for policy 1, policy_version 1867533 (0.0010) [2023-12-27 04:58:08,898][105692] Updated weights for policy 0, policy_version 1863038 (0.0006) [2023-12-27 04:58:08,956][105692] Updated weights for policy 0, policy_version 1863048 (0.0008) [2023-12-27 04:58:09,633][105620] Updated weights for policy 1, policy_version 1867543 (0.0011) [2023-12-27 04:58:09,682][105620] Updated weights for policy 1, policy_version 1867553 (0.0011) [2023-12-27 04:58:09,745][105620] Updated weights for policy 1, policy_version 1867563 (0.0011) [2023-12-27 04:58:09,762][105692] Updated weights for policy 0, policy_version 1863058 (0.0009) [2023-12-27 04:58:09,825][105692] Updated weights for policy 0, policy_version 1863068 (0.0006) [2023-12-27 04:58:09,893][105692] Updated weights for policy 0, policy_version 1863078 (0.0007) [2023-12-27 04:58:09,957][105692] Updated weights for policy 0, policy_version 1863088 (0.0007) [2023-12-27 04:58:10,497][105620] Updated weights for policy 1, policy_version 1867573 (0.0008) [2023-12-27 04:58:10,561][105620] Updated weights for policy 1, policy_version 1867583 (0.0005) [2023-12-27 04:58:10,627][105620] Updated weights for policy 1, policy_version 1867593 (0.0008) [2023-12-27 04:58:10,684][105692] Updated weights for policy 0, policy_version 1863098 (0.0007) [2023-12-27 04:58:10,738][105692] Updated weights for policy 0, policy_version 1863108 (0.0007) [2023-12-27 04:58:10,794][105692] Updated weights for policy 0, policy_version 1863118 (0.0008) [2023-12-27 04:58:11,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19605.3). Total num frames: 955203584. Throughput: 0: 9763.1, 1: 9851.5. Samples: 955210736. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:58:11,062][104569] Avg episode reward: [(0, '8173.710'), (1, '9253.265')] [2023-12-27 04:58:11,317][105620] Updated weights for policy 1, policy_version 1867603 (0.0008) [2023-12-27 04:58:11,389][105620] Updated weights for policy 1, policy_version 1867613 (0.0009) [2023-12-27 04:58:11,443][105620] Updated weights for policy 1, policy_version 1867623 (0.0010) [2023-12-27 04:58:11,549][105692] Updated weights for policy 0, policy_version 1863128 (0.0005) [2023-12-27 04:58:11,617][105692] Updated weights for policy 0, policy_version 1863138 (0.0006) [2023-12-27 04:58:11,686][105692] Updated weights for policy 0, policy_version 1863148 (0.0008) [2023-12-27 04:58:12,212][105620] Updated weights for policy 1, policy_version 1867633 (0.0009) [2023-12-27 04:58:12,280][105620] Updated weights for policy 1, policy_version 1867643 (0.0009) [2023-12-27 04:58:12,342][105620] Updated weights for policy 1, policy_version 1867653 (0.0008) [2023-12-27 04:58:12,373][105692] Updated weights for policy 0, policy_version 1863158 (0.0008) [2023-12-27 04:58:12,409][105620] Updated weights for policy 1, policy_version 1867663 (0.0008) [2023-12-27 04:58:12,438][105692] Updated weights for policy 0, policy_version 1863168 (0.0008) [2023-12-27 04:58:12,499][105692] Updated weights for policy 0, policy_version 1863178 (0.0009) [2023-12-27 04:58:13,143][105620] Updated weights for policy 1, policy_version 1867673 (0.0008) [2023-12-27 04:58:13,203][105620] Updated weights for policy 1, policy_version 1867683 (0.0009) [2023-12-27 04:58:13,240][105692] Updated weights for policy 0, policy_version 1863188 (0.0009) [2023-12-27 04:58:13,257][105620] Updated weights for policy 1, policy_version 1867693 (0.0009) [2023-12-27 04:58:13,304][105692] Updated weights for policy 0, policy_version 1863198 (0.0009) [2023-12-27 04:58:13,362][105692] Updated weights for policy 0, policy_version 1863208 (0.0009) [2023-12-27 04:58:13,909][105620] Updated weights for policy 1, policy_version 1867703 (0.0009) [2023-12-27 04:58:13,968][105620] Updated weights for policy 1, policy_version 1867713 (0.0010) [2023-12-27 04:58:14,027][105620] Updated weights for policy 1, policy_version 1867723 (0.0010) [2023-12-27 04:58:14,159][105692] Updated weights for policy 0, policy_version 1863218 (0.0008) [2023-12-27 04:58:14,209][105692] Updated weights for policy 0, policy_version 1863228 (0.0008) [2023-12-27 04:58:14,258][105692] Updated weights for policy 0, policy_version 1863238 (0.0008) [2023-12-27 04:58:14,309][105692] Updated weights for policy 0, policy_version 1863248 (0.0008) [2023-12-27 04:58:14,741][105620] Updated weights for policy 1, policy_version 1867733 (0.0010) [2023-12-27 04:58:14,805][105620] Updated weights for policy 1, policy_version 1867743 (0.0010) [2023-12-27 04:58:14,870][105620] Updated weights for policy 1, policy_version 1867753 (0.0007) [2023-12-27 04:58:15,081][105692] Updated weights for policy 0, policy_version 1863258 (0.0009) [2023-12-27 04:58:15,144][105692] Updated weights for policy 0, policy_version 1863268 (0.0009) [2023-12-27 04:58:15,202][105692] Updated weights for policy 0, policy_version 1863278 (0.0009) [2023-12-27 04:58:15,560][105620] Updated weights for policy 1, policy_version 1867763 (0.0009) [2023-12-27 04:58:15,626][105620] Updated weights for policy 1, policy_version 1867773 (0.0008) [2023-12-27 04:58:15,692][105620] Updated weights for policy 1, policy_version 1867783 (0.0009) [2023-12-27 04:58:15,980][105692] Updated weights for policy 0, policy_version 1863288 (0.0010) [2023-12-27 04:58:16,028][105692] Updated weights for policy 0, policy_version 1863298 (0.0009) [2023-12-27 04:58:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19605.3). Total num frames: 955293696. Throughput: 0: 9772.4, 1: 9834.8. Samples: 955267128. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:58:16,062][104569] Avg episode reward: [(0, '8173.163'), (1, '9160.992')] [2023-12-27 04:58:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001867792_478224384.pth... [2023-12-27 04:58:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001866640_477929472.pth [2023-12-27 04:58:16,082][105692] Updated weights for policy 0, policy_version 1863308 (0.0008) [2023-12-27 04:58:16,099][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001863312_477077504.pth... [2023-12-27 04:58:16,102][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001862160_476782592.pth [2023-12-27 04:58:16,360][105620] Updated weights for policy 1, policy_version 1867793 (0.0009) [2023-12-27 04:58:16,419][105620] Updated weights for policy 1, policy_version 1867803 (0.0008) [2023-12-27 04:58:16,475][105620] Updated weights for policy 1, policy_version 1867813 (0.0006) [2023-12-27 04:58:16,539][105620] Updated weights for policy 1, policy_version 1867823 (0.0009) [2023-12-27 04:58:16,929][105692] Updated weights for policy 0, policy_version 1863318 (0.0008) [2023-12-27 04:58:16,994][105692] Updated weights for policy 0, policy_version 1863328 (0.0007) [2023-12-27 04:58:17,046][105692] Updated weights for policy 0, policy_version 1863339 (0.0009) [2023-12-27 04:58:17,156][105620] Updated weights for policy 1, policy_version 1867833 (0.0009) [2023-12-27 04:58:17,206][105620] Updated weights for policy 1, policy_version 1867843 (0.0008) [2023-12-27 04:58:17,256][105620] Updated weights for policy 1, policy_version 1867853 (0.0008) [2023-12-27 04:58:17,785][105692] Updated weights for policy 0, policy_version 1863349 (0.0009) [2023-12-27 04:58:17,833][105692] Updated weights for policy 0, policy_version 1863359 (0.0009) [2023-12-27 04:58:17,883][105692] Updated weights for policy 0, policy_version 1863369 (0.0008) [2023-12-27 04:58:18,019][105620] Updated weights for policy 1, policy_version 1867863 (0.0009) [2023-12-27 04:58:18,077][105620] Updated weights for policy 1, policy_version 1867873 (0.0008) [2023-12-27 04:58:18,144][105620] Updated weights for policy 1, policy_version 1867883 (0.0006) [2023-12-27 04:58:18,747][105692] Updated weights for policy 0, policy_version 1863379 (0.0009) [2023-12-27 04:58:18,754][105620] Updated weights for policy 1, policy_version 1867893 (0.0006) [2023-12-27 04:58:18,816][105620] Updated weights for policy 1, policy_version 1867903 (0.0005) [2023-12-27 04:58:18,817][105692] Updated weights for policy 0, policy_version 1863389 (0.0009) [2023-12-27 04:58:18,884][105692] Updated weights for policy 0, policy_version 1863399 (0.0010) [2023-12-27 04:58:18,884][105620] Updated weights for policy 1, policy_version 1867913 (0.0005) [2023-12-27 04:58:19,453][105620] Updated weights for policy 1, policy_version 1867923 (0.0006) [2023-12-27 04:58:19,515][105620] Updated weights for policy 1, policy_version 1867933 (0.0007) [2023-12-27 04:58:19,577][105620] Updated weights for policy 1, policy_version 1867943 (0.0007) [2023-12-27 04:58:19,723][105692] Updated weights for policy 0, policy_version 1863409 (0.0009) [2023-12-27 04:58:19,792][105692] Updated weights for policy 0, policy_version 1863419 (0.0008) [2023-12-27 04:58:19,858][105692] Updated weights for policy 0, policy_version 1863429 (0.0008) [2023-12-27 04:58:19,911][105692] Updated weights for policy 0, policy_version 1863439 (0.0006) [2023-12-27 04:58:20,232][105620] Updated weights for policy 1, policy_version 1867953 (0.0007) [2023-12-27 04:58:20,303][105620] Updated weights for policy 1, policy_version 1867963 (0.0007) [2023-12-27 04:58:20,368][105620] Updated weights for policy 1, policy_version 1867973 (0.0006) [2023-12-27 04:58:20,426][105620] Updated weights for policy 1, policy_version 1867983 (0.0006) [2023-12-27 04:58:20,575][105692] Updated weights for policy 0, policy_version 1863449 (0.0012) [2023-12-27 04:58:20,650][105692] Updated weights for policy 0, policy_version 1863459 (0.0011) [2023-12-27 04:58:20,723][105692] Updated weights for policy 0, policy_version 1863469 (0.0009) [2023-12-27 04:58:20,975][105620] Updated weights for policy 1, policy_version 1867993 (0.0006) [2023-12-27 04:58:21,031][105620] Updated weights for policy 1, policy_version 1868003 (0.0009) [2023-12-27 04:58:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 955392000. Throughput: 0: 9632.5, 1: 9960.3. Samples: 955381780. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:58:21,063][104569] Avg episode reward: [(0, '8172.415'), (1, '9253.316')] [2023-12-27 04:58:21,087][105620] Updated weights for policy 1, policy_version 1868013 (0.0010) [2023-12-27 04:58:21,462][105692] Updated weights for policy 0, policy_version 1863479 (0.0007) [2023-12-27 04:58:21,516][105692] Updated weights for policy 0, policy_version 1863489 (0.0006) [2023-12-27 04:58:21,581][105692] Updated weights for policy 0, policy_version 1863499 (0.0005) [2023-12-27 04:58:21,920][105620] Updated weights for policy 1, policy_version 1868023 (0.0007) [2023-12-27 04:58:21,979][105620] Updated weights for policy 1, policy_version 1868033 (0.0010) [2023-12-27 04:58:22,039][105620] Updated weights for policy 1, policy_version 1868043 (0.0008) [2023-12-27 04:58:22,206][105692] Updated weights for policy 0, policy_version 1863509 (0.0006) [2023-12-27 04:58:22,266][105692] Updated weights for policy 0, policy_version 1863519 (0.0007) [2023-12-27 04:58:22,334][105692] Updated weights for policy 0, policy_version 1863529 (0.0008) [2023-12-27 04:58:22,832][105620] Updated weights for policy 1, policy_version 1868053 (0.0009) [2023-12-27 04:58:22,900][105620] Updated weights for policy 1, policy_version 1868063 (0.0008) [2023-12-27 04:58:22,958][105620] Updated weights for policy 1, policy_version 1868073 (0.0008) [2023-12-27 04:58:23,036][105692] Updated weights for policy 0, policy_version 1863539 (0.0011) [2023-12-27 04:58:23,095][105692] Updated weights for policy 0, policy_version 1863549 (0.0011) [2023-12-27 04:58:23,161][105692] Updated weights for policy 0, policy_version 1863559 (0.0009) [2023-12-27 04:58:23,737][105620] Updated weights for policy 1, policy_version 1868083 (0.0007) [2023-12-27 04:58:23,747][105692] Updated weights for policy 0, policy_version 1863569 (0.0008) [2023-12-27 04:58:23,791][105620] Updated weights for policy 1, policy_version 1868093 (0.0005) [2023-12-27 04:58:23,810][105692] Updated weights for policy 0, policy_version 1863579 (0.0006) [2023-12-27 04:58:23,845][105620] Updated weights for policy 1, policy_version 1868103 (0.0005) [2023-12-27 04:58:23,860][105692] Updated weights for policy 0, policy_version 1863589 (0.0007) [2023-12-27 04:58:23,919][105692] Updated weights for policy 0, policy_version 1863599 (0.0006) [2023-12-27 04:58:24,573][105620] Updated weights for policy 1, policy_version 1868113 (0.0007) [2023-12-27 04:58:24,628][105620] Updated weights for policy 1, policy_version 1868123 (0.0008) [2023-12-27 04:58:24,656][105692] Updated weights for policy 0, policy_version 1863609 (0.0009) [2023-12-27 04:58:24,691][105620] Updated weights for policy 1, policy_version 1868133 (0.0007) [2023-12-27 04:58:24,716][105692] Updated weights for policy 0, policy_version 1863619 (0.0011) [2023-12-27 04:58:24,751][105620] Updated weights for policy 1, policy_version 1868143 (0.0007) [2023-12-27 04:58:24,776][105692] Updated weights for policy 0, policy_version 1863629 (0.0010) [2023-12-27 04:58:25,491][105620] Updated weights for policy 1, policy_version 1868153 (0.0008) [2023-12-27 04:58:25,544][105620] Updated weights for policy 1, policy_version 1868163 (0.0006) [2023-12-27 04:58:25,552][105692] Updated weights for policy 0, policy_version 1863639 (0.0010) [2023-12-27 04:58:25,601][105620] Updated weights for policy 1, policy_version 1868173 (0.0008) [2023-12-27 04:58:25,609][105692] Updated weights for policy 0, policy_version 1863649 (0.0010) [2023-12-27 04:58:25,666][105692] Updated weights for policy 0, policy_version 1863659 (0.0010) [2023-12-27 04:58:26,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.4, 300 sec: 19605.3). Total num frames: 955490304. Throughput: 0: 9643.7, 1: 9905.3. Samples: 955498144. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:58:26,062][104569] Avg episode reward: [(0, '8626.537'), (1, '9345.456')] [2023-12-27 04:58:26,378][105620] Updated weights for policy 1, policy_version 1868183 (0.0008) [2023-12-27 04:58:26,409][105692] Updated weights for policy 0, policy_version 1863669 (0.0010) [2023-12-27 04:58:26,442][105620] Updated weights for policy 1, policy_version 1868193 (0.0007) [2023-12-27 04:58:26,457][105692] Updated weights for policy 0, policy_version 1863679 (0.0010) [2023-12-27 04:58:26,504][105620] Updated weights for policy 1, policy_version 1868203 (0.0006) [2023-12-27 04:58:26,504][105692] Updated weights for policy 0, policy_version 1863689 (0.0010) [2023-12-27 04:58:27,038][105620] Updated weights for policy 1, policy_version 1868213 (0.0010) [2023-12-27 04:58:27,096][105620] Updated weights for policy 1, policy_version 1868223 (0.0008) [2023-12-27 04:58:27,151][105620] Updated weights for policy 1, policy_version 1868233 (0.0005) [2023-12-27 04:58:27,248][105692] Updated weights for policy 0, policy_version 1863699 (0.0010) [2023-12-27 04:58:27,313][105692] Updated weights for policy 0, policy_version 1863709 (0.0010) [2023-12-27 04:58:27,366][105692] Updated weights for policy 0, policy_version 1863719 (0.0010) [2023-12-27 04:58:27,777][105620] Updated weights for policy 1, policy_version 1868243 (0.0007) [2023-12-27 04:58:27,834][105620] Updated weights for policy 1, policy_version 1868253 (0.0009) [2023-12-27 04:58:27,887][105620] Updated weights for policy 1, policy_version 1868263 (0.0009) [2023-12-27 04:58:27,988][105692] Updated weights for policy 0, policy_version 1863729 (0.0010) [2023-12-27 04:58:28,053][105692] Updated weights for policy 0, policy_version 1863739 (0.0006) [2023-12-27 04:58:28,107][105692] Updated weights for policy 0, policy_version 1863749 (0.0010) [2023-12-27 04:58:28,159][105692] Updated weights for policy 0, policy_version 1863759 (0.0010) [2023-12-27 04:58:28,697][105620] Updated weights for policy 1, policy_version 1868273 (0.0009) [2023-12-27 04:58:28,763][105620] Updated weights for policy 1, policy_version 1868283 (0.0009) [2023-12-27 04:58:28,774][105692] Updated weights for policy 0, policy_version 1863769 (0.0006) [2023-12-27 04:58:28,815][105620] Updated weights for policy 1, policy_version 1868293 (0.0008) [2023-12-27 04:58:28,839][105692] Updated weights for policy 0, policy_version 1863779 (0.0005) [2023-12-27 04:58:28,872][105620] Updated weights for policy 1, policy_version 1868303 (0.0009) [2023-12-27 04:58:28,897][105692] Updated weights for policy 0, policy_version 1863789 (0.0006) [2023-12-27 04:58:29,611][105620] Updated weights for policy 1, policy_version 1868313 (0.0008) [2023-12-27 04:58:29,616][105692] Updated weights for policy 0, policy_version 1863799 (0.0010) [2023-12-27 04:58:29,671][105692] Updated weights for policy 0, policy_version 1863809 (0.0010) [2023-12-27 04:58:29,673][105620] Updated weights for policy 1, policy_version 1868323 (0.0005) [2023-12-27 04:58:29,719][105692] Updated weights for policy 0, policy_version 1863819 (0.0010) [2023-12-27 04:58:29,725][105620] Updated weights for policy 1, policy_version 1868333 (0.0005) [2023-12-27 04:58:30,334][105692] Updated weights for policy 0, policy_version 1863829 (0.0008) [2023-12-27 04:58:30,395][105692] Updated weights for policy 0, policy_version 1863839 (0.0007) [2023-12-27 04:58:30,426][105620] Updated weights for policy 1, policy_version 1868343 (0.0007) [2023-12-27 04:58:30,461][105692] Updated weights for policy 0, policy_version 1863849 (0.0009) [2023-12-27 04:58:30,488][105620] Updated weights for policy 1, policy_version 1868353 (0.0006) [2023-12-27 04:58:30,535][105620] Updated weights for policy 1, policy_version 1868363 (0.0008) [2023-12-27 04:58:31,056][105692] Updated weights for policy 0, policy_version 1863859 (0.0007) [2023-12-27 04:58:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19605.3). Total num frames: 955588608. Throughput: 0: 9696.3, 1: 9910.2. Samples: 955558484. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:58:31,063][104569] Avg episode reward: [(0, '8990.407'), (1, '9345.438')] [2023-12-27 04:58:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001868368_478371840.pth... [2023-12-27 04:58:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001867216_478076928.pth [2023-12-27 04:58:31,117][105692] Updated weights for policy 0, policy_version 1863869 (0.0006) [2023-12-27 04:58:31,176][105692] Updated weights for policy 0, policy_version 1863879 (0.0007) [2023-12-27 04:58:31,230][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001863888_477224960.pth... [2023-12-27 04:58:31,235][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001862736_476930048.pth [2023-12-27 04:58:31,357][105620] Updated weights for policy 1, policy_version 1868373 (0.0008) [2023-12-27 04:58:31,428][105620] Updated weights for policy 1, policy_version 1868383 (0.0008) [2023-12-27 04:58:31,482][105620] Updated weights for policy 1, policy_version 1868393 (0.0008) [2023-12-27 04:58:31,848][105692] Updated weights for policy 0, policy_version 1863889 (0.0005) [2023-12-27 04:58:31,910][105692] Updated weights for policy 0, policy_version 1863899 (0.0009) [2023-12-27 04:58:31,972][105692] Updated weights for policy 0, policy_version 1863909 (0.0009) [2023-12-27 04:58:32,034][105692] Updated weights for policy 0, policy_version 1863919 (0.0008) [2023-12-27 04:58:32,265][105620] Updated weights for policy 1, policy_version 1868403 (0.0009) [2023-12-27 04:58:32,325][105620] Updated weights for policy 1, policy_version 1868413 (0.0009) [2023-12-27 04:58:32,388][105620] Updated weights for policy 1, policy_version 1868423 (0.0009) [2023-12-27 04:58:32,838][105692] Updated weights for policy 0, policy_version 1863929 (0.0009) [2023-12-27 04:58:32,887][105692] Updated weights for policy 0, policy_version 1863939 (0.0007) [2023-12-27 04:58:32,942][105692] Updated weights for policy 0, policy_version 1863949 (0.0010) [2023-12-27 04:58:32,958][105620] Updated weights for policy 1, policy_version 1868433 (0.0005) [2023-12-27 04:58:33,003][105620] Updated weights for policy 1, policy_version 1868443 (0.0005) [2023-12-27 04:58:33,057][105620] Updated weights for policy 1, policy_version 1868453 (0.0010) [2023-12-27 04:58:33,104][105620] Updated weights for policy 1, policy_version 1868463 (0.0010) [2023-12-27 04:58:33,635][105692] Updated weights for policy 0, policy_version 1863960 (0.0010) [2023-12-27 04:58:33,698][105692] Updated weights for policy 0, policy_version 1863970 (0.0007) [2023-12-27 04:58:33,756][105692] Updated weights for policy 0, policy_version 1863980 (0.0009) [2023-12-27 04:58:33,790][105620] Updated weights for policy 1, policy_version 1868473 (0.0006) [2023-12-27 04:58:33,839][105620] Updated weights for policy 1, policy_version 1868483 (0.0009) [2023-12-27 04:58:33,887][105620] Updated weights for policy 1, policy_version 1868493 (0.0010) [2023-12-27 04:58:34,473][105692] Updated weights for policy 0, policy_version 1863990 (0.0008) [2023-12-27 04:58:34,535][105692] Updated weights for policy 0, policy_version 1864000 (0.0007) [2023-12-27 04:58:34,566][105620] Updated weights for policy 1, policy_version 1868503 (0.0009) [2023-12-27 04:58:34,600][105692] Updated weights for policy 0, policy_version 1864010 (0.0006) [2023-12-27 04:58:34,622][105620] Updated weights for policy 1, policy_version 1868513 (0.0011) [2023-12-27 04:58:34,682][105620] Updated weights for policy 1, policy_version 1868523 (0.0011) [2023-12-27 04:58:35,220][105692] Updated weights for policy 0, policy_version 1864020 (0.0005) [2023-12-27 04:58:35,279][105692] Updated weights for policy 0, policy_version 1864030 (0.0006) [2023-12-27 04:58:35,342][105692] Updated weights for policy 0, policy_version 1864040 (0.0006) [2023-12-27 04:58:35,386][105620] Updated weights for policy 1, policy_version 1868533 (0.0011) [2023-12-27 04:58:35,431][105620] Updated weights for policy 1, policy_version 1868543 (0.0010) [2023-12-27 04:58:35,482][105620] Updated weights for policy 1, policy_version 1868553 (0.0010) [2023-12-27 04:58:35,869][105692] Updated weights for policy 0, policy_version 1864050 (0.0006) [2023-12-27 04:58:35,934][105692] Updated weights for policy 0, policy_version 1864060 (0.0010) [2023-12-27 04:58:36,004][105692] Updated weights for policy 0, policy_version 1864070 (0.0010) [2023-12-27 04:58:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 955686912. Throughput: 0: 9772.1, 1: 9898.8. Samples: 955678928. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:58:36,063][104569] Avg episode reward: [(0, '8540.670'), (1, '9161.606')] [2023-12-27 04:58:36,074][105692] Updated weights for policy 0, policy_version 1864080 (0.0010) [2023-12-27 04:58:36,099][105620] Updated weights for policy 1, policy_version 1868563 (0.0009) [2023-12-27 04:58:36,163][105620] Updated weights for policy 1, policy_version 1868573 (0.0008) [2023-12-27 04:58:36,216][105620] Updated weights for policy 1, policy_version 1868583 (0.0011) [2023-12-27 04:58:36,786][105692] Updated weights for policy 0, policy_version 1864090 (0.0011) [2023-12-27 04:58:36,854][105692] Updated weights for policy 0, policy_version 1864100 (0.0011) [2023-12-27 04:58:36,920][105692] Updated weights for policy 0, policy_version 1864110 (0.0011) [2023-12-27 04:58:36,968][105620] Updated weights for policy 1, policy_version 1868593 (0.0010) [2023-12-27 04:58:37,028][105620] Updated weights for policy 1, policy_version 1868603 (0.0005) [2023-12-27 04:58:37,077][105620] Updated weights for policy 1, policy_version 1868613 (0.0006) [2023-12-27 04:58:37,123][105620] Updated weights for policy 1, policy_version 1868623 (0.0005) [2023-12-27 04:58:37,616][105692] Updated weights for policy 0, policy_version 1864120 (0.0006) [2023-12-27 04:58:37,667][105692] Updated weights for policy 0, policy_version 1864130 (0.0006) [2023-12-27 04:58:37,723][105692] Updated weights for policy 0, policy_version 1864140 (0.0011) [2023-12-27 04:58:37,780][105620] Updated weights for policy 1, policy_version 1868633 (0.0008) [2023-12-27 04:58:37,839][105620] Updated weights for policy 1, policy_version 1868643 (0.0009) [2023-12-27 04:58:37,898][105620] Updated weights for policy 1, policy_version 1868653 (0.0008) [2023-12-27 04:58:38,489][105692] Updated weights for policy 0, policy_version 1864150 (0.0011) [2023-12-27 04:58:38,535][105692] Updated weights for policy 0, policy_version 1864160 (0.0008) [2023-12-27 04:58:38,592][105692] Updated weights for policy 0, policy_version 1864170 (0.0010) [2023-12-27 04:58:38,612][105620] Updated weights for policy 1, policy_version 1868663 (0.0006) [2023-12-27 04:58:38,664][105620] Updated weights for policy 1, policy_version 1868673 (0.0005) [2023-12-27 04:58:38,731][105620] Updated weights for policy 1, policy_version 1868683 (0.0006) [2023-12-27 04:58:39,376][105620] Updated weights for policy 1, policy_version 1868693 (0.0009) [2023-12-27 04:58:39,409][105692] Updated weights for policy 0, policy_version 1864180 (0.0009) [2023-12-27 04:58:39,442][105620] Updated weights for policy 1, policy_version 1868703 (0.0010) [2023-12-27 04:58:39,472][105692] Updated weights for policy 0, policy_version 1864190 (0.0007) [2023-12-27 04:58:39,506][105620] Updated weights for policy 1, policy_version 1868713 (0.0011) [2023-12-27 04:58:39,532][105692] Updated weights for policy 0, policy_version 1864200 (0.0006) [2023-12-27 04:58:40,214][105620] Updated weights for policy 1, policy_version 1868723 (0.0010) [2023-12-27 04:58:40,276][105620] Updated weights for policy 1, policy_version 1868733 (0.0008) [2023-12-27 04:58:40,320][105692] Updated weights for policy 0, policy_version 1864210 (0.0008) [2023-12-27 04:58:40,334][105620] Updated weights for policy 1, policy_version 1868743 (0.0008) [2023-12-27 04:58:40,378][105692] Updated weights for policy 0, policy_version 1864220 (0.0007) [2023-12-27 04:58:40,438][105692] Updated weights for policy 0, policy_version 1864230 (0.0008) [2023-12-27 04:58:40,494][105692] Updated weights for policy 0, policy_version 1864240 (0.0008) [2023-12-27 04:58:41,055][105620] Updated weights for policy 1, policy_version 1868753 (0.0008) [2023-12-27 04:58:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 955785216. Throughput: 0: 9801.4, 1: 9947.0. Samples: 955797400. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:58:41,062][104569] Avg episode reward: [(0, '8449.176'), (1, '9161.660')] [2023-12-27 04:58:41,110][105620] Updated weights for policy 1, policy_version 1868763 (0.0009) [2023-12-27 04:58:41,176][105620] Updated weights for policy 1, policy_version 1868773 (0.0009) [2023-12-27 04:58:41,243][105620] Updated weights for policy 1, policy_version 1868783 (0.0007) [2023-12-27 04:58:41,275][105692] Updated weights for policy 0, policy_version 1864251 (0.0008) [2023-12-27 04:58:41,326][105692] Updated weights for policy 0, policy_version 1864261 (0.0008) [2023-12-27 04:58:41,395][105692] Updated weights for policy 0, policy_version 1864271 (0.0008) [2023-12-27 04:58:42,003][105620] Updated weights for policy 1, policy_version 1868793 (0.0009) [2023-12-27 04:58:42,052][105620] Updated weights for policy 1, policy_version 1868803 (0.0011) [2023-12-27 04:58:42,109][105692] Updated weights for policy 0, policy_version 1864281 (0.0007) [2023-12-27 04:58:42,114][105620] Updated weights for policy 1, policy_version 1868813 (0.0010) [2023-12-27 04:58:42,170][105692] Updated weights for policy 0, policy_version 1864291 (0.0008) [2023-12-27 04:58:42,222][105692] Updated weights for policy 0, policy_version 1864301 (0.0008) [2023-12-27 04:58:42,806][105620] Updated weights for policy 1, policy_version 1868823 (0.0007) [2023-12-27 04:58:42,875][105620] Updated weights for policy 1, policy_version 1868833 (0.0005) [2023-12-27 04:58:42,939][105620] Updated weights for policy 1, policy_version 1868843 (0.0007) [2023-12-27 04:58:43,040][105692] Updated weights for policy 0, policy_version 1864311 (0.0009) [2023-12-27 04:58:43,104][105692] Updated weights for policy 0, policy_version 1864321 (0.0010) [2023-12-27 04:58:43,169][105692] Updated weights for policy 0, policy_version 1864331 (0.0011) [2023-12-27 04:58:43,572][105620] Updated weights for policy 1, policy_version 1868853 (0.0008) [2023-12-27 04:58:43,630][105620] Updated weights for policy 1, policy_version 1868863 (0.0008) [2023-12-27 04:58:43,696][105620] Updated weights for policy 1, policy_version 1868873 (0.0008) [2023-12-27 04:58:43,826][105692] Updated weights for policy 0, policy_version 1864341 (0.0010) [2023-12-27 04:58:43,893][105692] Updated weights for policy 0, policy_version 1864351 (0.0005) [2023-12-27 04:58:43,944][105692] Updated weights for policy 0, policy_version 1864361 (0.0005) [2023-12-27 04:58:44,447][105620] Updated weights for policy 1, policy_version 1868883 (0.0008) [2023-12-27 04:58:44,503][105620] Updated weights for policy 1, policy_version 1868893 (0.0008) [2023-12-27 04:58:44,562][105620] Updated weights for policy 1, policy_version 1868903 (0.0008) [2023-12-27 04:58:44,643][105692] Updated weights for policy 0, policy_version 1864371 (0.0008) [2023-12-27 04:58:44,712][105692] Updated weights for policy 0, policy_version 1864381 (0.0011) [2023-12-27 04:58:44,775][105692] Updated weights for policy 0, policy_version 1864391 (0.0010) [2023-12-27 04:58:45,360][105620] Updated weights for policy 1, policy_version 1868913 (0.0008) [2023-12-27 04:58:45,415][105620] Updated weights for policy 1, policy_version 1868923 (0.0008) [2023-12-27 04:58:45,471][105620] Updated weights for policy 1, policy_version 1868933 (0.0008) [2023-12-27 04:58:45,527][105620] Updated weights for policy 1, policy_version 1868943 (0.0008) [2023-12-27 04:58:45,529][105692] Updated weights for policy 0, policy_version 1864401 (0.0011) [2023-12-27 04:58:45,591][105692] Updated weights for policy 0, policy_version 1864411 (0.0010) [2023-12-27 04:58:45,650][105692] Updated weights for policy 0, policy_version 1864421 (0.0011) [2023-12-27 04:58:45,709][105692] Updated weights for policy 0, policy_version 1864431 (0.0011) [2023-12-27 04:58:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 955883520. Throughput: 0: 9684.0, 1: 9890.8. Samples: 955854868. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:58:46,062][104569] Avg episode reward: [(0, '8636.388'), (1, '9345.521')] [2023-12-27 04:58:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001864432_477364224.pth... [2023-12-27 04:58:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001868944_478519296.pth... [2023-12-27 04:58:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001863312_477077504.pth [2023-12-27 04:58:46,091][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001867792_478224384.pth [2023-12-27 04:58:46,322][105620] Updated weights for policy 1, policy_version 1868953 (0.0008) [2023-12-27 04:58:46,389][105620] Updated weights for policy 1, policy_version 1868963 (0.0008) [2023-12-27 04:58:46,419][105692] Updated weights for policy 0, policy_version 1864441 (0.0011) [2023-12-27 04:58:46,459][105620] Updated weights for policy 1, policy_version 1868973 (0.0007) [2023-12-27 04:58:46,469][105692] Updated weights for policy 0, policy_version 1864451 (0.0008) [2023-12-27 04:58:46,538][105692] Updated weights for policy 0, policy_version 1864461 (0.0005) [2023-12-27 04:58:47,097][105692] Updated weights for policy 0, policy_version 1864471 (0.0009) [2023-12-27 04:58:47,156][105692] Updated weights for policy 0, policy_version 1864481 (0.0010) [2023-12-27 04:58:47,221][105692] Updated weights for policy 0, policy_version 1864491 (0.0005) [2023-12-27 04:58:47,272][105620] Updated weights for policy 1, policy_version 1868983 (0.0007) [2023-12-27 04:58:47,337][105620] Updated weights for policy 1, policy_version 1868993 (0.0005) [2023-12-27 04:58:47,409][105620] Updated weights for policy 1, policy_version 1869003 (0.0008) [2023-12-27 04:58:47,919][105692] Updated weights for policy 0, policy_version 1864501 (0.0011) [2023-12-27 04:58:47,973][105692] Updated weights for policy 0, policy_version 1864511 (0.0010) [2023-12-27 04:58:48,029][105692] Updated weights for policy 0, policy_version 1864521 (0.0008) [2023-12-27 04:58:48,105][105620] Updated weights for policy 1, policy_version 1869013 (0.0009) [2023-12-27 04:58:48,171][105620] Updated weights for policy 1, policy_version 1869023 (0.0006) [2023-12-27 04:58:48,233][105620] Updated weights for policy 1, policy_version 1869033 (0.0010) [2023-12-27 04:58:48,685][105692] Updated weights for policy 0, policy_version 1864531 (0.0010) [2023-12-27 04:58:48,733][105692] Updated weights for policy 0, policy_version 1864541 (0.0011) [2023-12-27 04:58:48,785][105692] Updated weights for policy 0, policy_version 1864551 (0.0011) [2023-12-27 04:58:48,923][105620] Updated weights for policy 1, policy_version 1869043 (0.0007) [2023-12-27 04:58:48,976][105620] Updated weights for policy 1, policy_version 1869053 (0.0005) [2023-12-27 04:58:49,025][105620] Updated weights for policy 1, policy_version 1869063 (0.0006) [2023-12-27 04:58:49,493][105692] Updated weights for policy 0, policy_version 1864561 (0.0011) [2023-12-27 04:58:49,553][105692] Updated weights for policy 0, policy_version 1864571 (0.0007) [2023-12-27 04:58:49,619][105692] Updated weights for policy 0, policy_version 1864581 (0.0006) [2023-12-27 04:58:49,682][105692] Updated weights for policy 0, policy_version 1864591 (0.0009) [2023-12-27 04:58:49,734][105620] Updated weights for policy 1, policy_version 1869073 (0.0008) [2023-12-27 04:58:49,789][105620] Updated weights for policy 1, policy_version 1869083 (0.0007) [2023-12-27 04:58:49,845][105620] Updated weights for policy 1, policy_version 1869093 (0.0008) [2023-12-27 04:58:49,907][105620] Updated weights for policy 1, policy_version 1869103 (0.0008) [2023-12-27 04:58:50,390][105692] Updated weights for policy 0, policy_version 1864601 (0.0011) [2023-12-27 04:58:50,443][105692] Updated weights for policy 0, policy_version 1864611 (0.0011) [2023-12-27 04:58:50,492][105692] Updated weights for policy 0, policy_version 1864621 (0.0011) [2023-12-27 04:58:50,645][105620] Updated weights for policy 1, policy_version 1869113 (0.0009) [2023-12-27 04:58:50,703][105620] Updated weights for policy 1, policy_version 1869123 (0.0008) [2023-12-27 04:58:50,767][105620] Updated weights for policy 1, policy_version 1869133 (0.0008) [2023-12-27 04:58:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 955981824. Throughput: 0: 9717.0, 1: 9828.2. Samples: 955970592. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:58:51,063][104569] Avg episode reward: [(0, '8815.390'), (1, '9253.277')] [2023-12-27 04:58:51,302][105692] Updated weights for policy 0, policy_version 1864631 (0.0011) [2023-12-27 04:58:51,363][105692] Updated weights for policy 0, policy_version 1864641 (0.0013) [2023-12-27 04:58:51,424][105692] Updated weights for policy 0, policy_version 1864651 (0.0011) [2023-12-27 04:58:51,577][105620] Updated weights for policy 1, policy_version 1869143 (0.0008) [2023-12-27 04:58:51,635][105620] Updated weights for policy 1, policy_version 1869153 (0.0008) [2023-12-27 04:58:51,698][105620] Updated weights for policy 1, policy_version 1869163 (0.0008) [2023-12-27 04:58:52,187][105692] Updated weights for policy 0, policy_version 1864661 (0.0010) [2023-12-27 04:58:52,247][105692] Updated weights for policy 0, policy_version 1864671 (0.0011) [2023-12-27 04:58:52,309][105692] Updated weights for policy 0, policy_version 1864681 (0.0011) [2023-12-27 04:58:52,484][105620] Updated weights for policy 1, policy_version 1869173 (0.0008) [2023-12-27 04:58:52,550][105620] Updated weights for policy 1, policy_version 1869183 (0.0008) [2023-12-27 04:58:52,609][105620] Updated weights for policy 1, policy_version 1869193 (0.0008) [2023-12-27 04:58:52,988][105692] Updated weights for policy 0, policy_version 1864691 (0.0009) [2023-12-27 04:58:53,053][105692] Updated weights for policy 0, policy_version 1864701 (0.0008) [2023-12-27 04:58:53,104][105692] Updated weights for policy 0, policy_version 1864711 (0.0010) [2023-12-27 04:58:53,244][105620] Updated weights for policy 1, policy_version 1869203 (0.0008) [2023-12-27 04:58:53,312][105620] Updated weights for policy 1, policy_version 1869213 (0.0005) [2023-12-27 04:58:53,381][105620] Updated weights for policy 1, policy_version 1869223 (0.0006) [2023-12-27 04:58:53,784][105692] Updated weights for policy 0, policy_version 1864721 (0.0010) [2023-12-27 04:58:53,853][105692] Updated weights for policy 0, policy_version 1864731 (0.0010) [2023-12-27 04:58:53,913][105620] Updated weights for policy 1, policy_version 1869233 (0.0008) [2023-12-27 04:58:53,918][105692] Updated weights for policy 0, policy_version 1864741 (0.0005) [2023-12-27 04:58:53,975][105620] Updated weights for policy 1, policy_version 1869243 (0.0007) [2023-12-27 04:58:53,984][105692] Updated weights for policy 0, policy_version 1864751 (0.0010) [2023-12-27 04:58:54,041][105620] Updated weights for policy 1, policy_version 1869253 (0.0005) [2023-12-27 04:58:54,095][105620] Updated weights for policy 1, policy_version 1869263 (0.0008) [2023-12-27 04:58:54,529][105692] Updated weights for policy 0, policy_version 1864761 (0.0006) [2023-12-27 04:58:54,584][105692] Updated weights for policy 0, policy_version 1864771 (0.0005) [2023-12-27 04:58:54,639][105692] Updated weights for policy 0, policy_version 1864781 (0.0005) [2023-12-27 04:58:54,776][105620] Updated weights for policy 1, policy_version 1869273 (0.0010) [2023-12-27 04:58:54,818][105620] Updated weights for policy 1, policy_version 1869283 (0.0010) [2023-12-27 04:58:54,863][105620] Updated weights for policy 1, policy_version 1869293 (0.0007) [2023-12-27 04:58:55,311][105692] Updated weights for policy 0, policy_version 1864791 (0.0005) [2023-12-27 04:58:55,375][105692] Updated weights for policy 0, policy_version 1864801 (0.0009) [2023-12-27 04:58:55,449][105692] Updated weights for policy 0, policy_version 1864811 (0.0008) [2023-12-27 04:58:55,589][105620] Updated weights for policy 1, policy_version 1869303 (0.0005) [2023-12-27 04:58:55,644][105620] Updated weights for policy 1, policy_version 1869313 (0.0005) [2023-12-27 04:58:55,704][105620] Updated weights for policy 1, policy_version 1869323 (0.0009) [2023-12-27 04:58:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19605.3). Total num frames: 956080128. Throughput: 0: 9744.1, 1: 9819.1. Samples: 956091080. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:58:56,062][104569] Avg episode reward: [(0, '8806.625'), (1, '9253.303')] [2023-12-27 04:58:56,107][105692] Updated weights for policy 0, policy_version 1864821 (0.0009) [2023-12-27 04:58:56,177][105692] Updated weights for policy 0, policy_version 1864831 (0.0011) [2023-12-27 04:58:56,243][105692] Updated weights for policy 0, policy_version 1864841 (0.0010) [2023-12-27 04:58:56,348][105620] Updated weights for policy 1, policy_version 1869333 (0.0011) [2023-12-27 04:58:56,400][105620] Updated weights for policy 1, policy_version 1869343 (0.0010) [2023-12-27 04:58:56,458][105620] Updated weights for policy 1, policy_version 1869353 (0.0010) [2023-12-27 04:58:56,895][105692] Updated weights for policy 0, policy_version 1864851 (0.0010) [2023-12-27 04:58:56,940][105692] Updated weights for policy 0, policy_version 1864861 (0.0009) [2023-12-27 04:58:56,986][105692] Updated weights for policy 0, policy_version 1864871 (0.0005) [2023-12-27 04:58:57,118][105620] Updated weights for policy 1, policy_version 1869363 (0.0009) [2023-12-27 04:58:57,161][105620] Updated weights for policy 1, policy_version 1869373 (0.0005) [2023-12-27 04:58:57,224][105620] Updated weights for policy 1, policy_version 1869383 (0.0009) [2023-12-27 04:58:57,719][105692] Updated weights for policy 0, policy_version 1864881 (0.0008) [2023-12-27 04:58:57,772][105692] Updated weights for policy 0, policy_version 1864891 (0.0010) [2023-12-27 04:58:57,800][105620] Updated weights for policy 1, policy_version 1869393 (0.0010) [2023-12-27 04:58:57,830][105692] Updated weights for policy 0, policy_version 1864901 (0.0009) [2023-12-27 04:58:57,859][105620] Updated weights for policy 1, policy_version 1869403 (0.0006) [2023-12-27 04:58:57,889][105692] Updated weights for policy 0, policy_version 1864911 (0.0006) [2023-12-27 04:58:57,913][105620] Updated weights for policy 1, policy_version 1869413 (0.0008) [2023-12-27 04:58:57,972][105620] Updated weights for policy 1, policy_version 1869423 (0.0007) [2023-12-27 04:58:58,614][105620] Updated weights for policy 1, policy_version 1869433 (0.0006) [2023-12-27 04:58:58,670][105620] Updated weights for policy 1, policy_version 1869443 (0.0008) [2023-12-27 04:58:58,730][105620] Updated weights for policy 1, policy_version 1869453 (0.0007) [2023-12-27 04:58:58,762][105692] Updated weights for policy 0, policy_version 1864921 (0.0009) [2023-12-27 04:58:58,835][105692] Updated weights for policy 0, policy_version 1864931 (0.0008) [2023-12-27 04:58:58,911][105692] Updated weights for policy 0, policy_version 1864941 (0.0008) [2023-12-27 04:58:59,573][105620] Updated weights for policy 1, policy_version 1869463 (0.0007) [2023-12-27 04:58:59,624][105620] Updated weights for policy 1, policy_version 1869473 (0.0008) [2023-12-27 04:58:59,680][105620] Updated weights for policy 1, policy_version 1869483 (0.0010) [2023-12-27 04:58:59,683][105692] Updated weights for policy 0, policy_version 1864951 (0.0007) [2023-12-27 04:58:59,740][105692] Updated weights for policy 0, policy_version 1864961 (0.0008) [2023-12-27 04:58:59,794][105692] Updated weights for policy 0, policy_version 1864971 (0.0010) [2023-12-27 04:59:00,351][105620] Updated weights for policy 1, policy_version 1869493 (0.0009) [2023-12-27 04:59:00,415][105620] Updated weights for policy 1, policy_version 1869503 (0.0010) [2023-12-27 04:59:00,475][105620] Updated weights for policy 1, policy_version 1869513 (0.0007) [2023-12-27 04:59:00,588][105692] Updated weights for policy 0, policy_version 1864981 (0.0010) [2023-12-27 04:59:00,642][105692] Updated weights for policy 0, policy_version 1864991 (0.0008) [2023-12-27 04:59:00,689][105692] Updated weights for policy 0, policy_version 1865001 (0.0008) [2023-12-27 04:59:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 956178432. Throughput: 0: 9748.2, 1: 9887.3. Samples: 956150728. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:59:01,063][104569] Avg episode reward: [(0, '8716.157'), (1, '9160.925')] [2023-12-27 04:59:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001865008_477511680.pth... [2023-12-27 04:59:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001869520_478666752.pth... [2023-12-27 04:59:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001868368_478371840.pth [2023-12-27 04:59:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001863888_477224960.pth [2023-12-27 04:59:01,140][105620] Updated weights for policy 1, policy_version 1869523 (0.0009) [2023-12-27 04:59:01,197][105620] Updated weights for policy 1, policy_version 1869533 (0.0009) [2023-12-27 04:59:01,255][105620] Updated weights for policy 1, policy_version 1869544 (0.0010) [2023-12-27 04:59:01,358][105692] Updated weights for policy 0, policy_version 1865011 (0.0009) [2023-12-27 04:59:01,414][105692] Updated weights for policy 0, policy_version 1865021 (0.0010) [2023-12-27 04:59:01,467][105692] Updated weights for policy 0, policy_version 1865031 (0.0010) [2023-12-27 04:59:02,003][105620] Updated weights for policy 1, policy_version 1869554 (0.0010) [2023-12-27 04:59:02,072][105620] Updated weights for policy 1, policy_version 1869564 (0.0011) [2023-12-27 04:59:02,132][105620] Updated weights for policy 1, policy_version 1869574 (0.0011) [2023-12-27 04:59:02,188][105620] Updated weights for policy 1, policy_version 1869584 (0.0011) [2023-12-27 04:59:02,210][105692] Updated weights for policy 0, policy_version 1865041 (0.0009) [2023-12-27 04:59:02,273][105692] Updated weights for policy 0, policy_version 1865051 (0.0008) [2023-12-27 04:59:02,340][105692] Updated weights for policy 0, policy_version 1865061 (0.0008) [2023-12-27 04:59:02,404][105692] Updated weights for policy 0, policy_version 1865071 (0.0008) [2023-12-27 04:59:02,933][105620] Updated weights for policy 1, policy_version 1869594 (0.0010) [2023-12-27 04:59:02,991][105620] Updated weights for policy 1, policy_version 1869604 (0.0010) [2023-12-27 04:59:03,045][105620] Updated weights for policy 1, policy_version 1869614 (0.0010) [2023-12-27 04:59:03,087][105692] Updated weights for policy 0, policy_version 1865081 (0.0007) [2023-12-27 04:59:03,145][105692] Updated weights for policy 0, policy_version 1865091 (0.0008) [2023-12-27 04:59:03,211][105692] Updated weights for policy 0, policy_version 1865101 (0.0008) [2023-12-27 04:59:03,770][105620] Updated weights for policy 1, policy_version 1869624 (0.0010) [2023-12-27 04:59:03,815][105620] Updated weights for policy 1, policy_version 1869634 (0.0010) [2023-12-27 04:59:03,875][105620] Updated weights for policy 1, policy_version 1869644 (0.0012) [2023-12-27 04:59:03,886][105692] Updated weights for policy 0, policy_version 1865111 (0.0008) [2023-12-27 04:59:03,941][105692] Updated weights for policy 0, policy_version 1865121 (0.0007) [2023-12-27 04:59:03,989][105692] Updated weights for policy 0, policy_version 1865131 (0.0008) [2023-12-27 04:59:04,621][105620] Updated weights for policy 1, policy_version 1869654 (0.0011) [2023-12-27 04:59:04,678][105620] Updated weights for policy 1, policy_version 1869664 (0.0009) [2023-12-27 04:59:04,724][105620] Updated weights for policy 1, policy_version 1869674 (0.0006) [2023-12-27 04:59:04,789][105692] Updated weights for policy 0, policy_version 1865141 (0.0008) [2023-12-27 04:59:04,854][105692] Updated weights for policy 0, policy_version 1865151 (0.0010) [2023-12-27 04:59:04,908][105692] Updated weights for policy 0, policy_version 1865161 (0.0010) [2023-12-27 04:59:05,289][105620] Updated weights for policy 1, policy_version 1869684 (0.0008) [2023-12-27 04:59:05,347][105620] Updated weights for policy 1, policy_version 1869694 (0.0006) [2023-12-27 04:59:05,402][105620] Updated weights for policy 1, policy_version 1869704 (0.0010) [2023-12-27 04:59:05,522][105692] Updated weights for policy 0, policy_version 1865171 (0.0007) [2023-12-27 04:59:05,572][105692] Updated weights for policy 0, policy_version 1865181 (0.0009) [2023-12-27 04:59:05,621][105692] Updated weights for policy 0, policy_version 1865191 (0.0008) [2023-12-27 04:59:06,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 956276736. Throughput: 0: 9830.4, 1: 9807.7. Samples: 956265496. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:59:06,062][104569] Avg episode reward: [(0, '8811.512'), (1, '9068.483')] [2023-12-27 04:59:06,069][105620] Updated weights for policy 1, policy_version 1869714 (0.0008) [2023-12-27 04:59:06,139][105620] Updated weights for policy 1, policy_version 1869724 (0.0008) [2023-12-27 04:59:06,199][105620] Updated weights for policy 1, policy_version 1869734 (0.0009) [2023-12-27 04:59:06,259][105620] Updated weights for policy 1, policy_version 1869744 (0.0010) [2023-12-27 04:59:06,405][105692] Updated weights for policy 0, policy_version 1865201 (0.0009) [2023-12-27 04:59:06,462][105692] Updated weights for policy 0, policy_version 1865211 (0.0009) [2023-12-27 04:59:06,532][105692] Updated weights for policy 0, policy_version 1865221 (0.0005) [2023-12-27 04:59:06,602][105692] Updated weights for policy 0, policy_version 1865231 (0.0005) [2023-12-27 04:59:07,110][105620] Updated weights for policy 1, policy_version 1869754 (0.0008) [2023-12-27 04:59:07,122][105692] Updated weights for policy 0, policy_version 1865241 (0.0008) [2023-12-27 04:59:07,164][105620] Updated weights for policy 1, policy_version 1869764 (0.0007) [2023-12-27 04:59:07,181][105692] Updated weights for policy 0, policy_version 1865251 (0.0009) [2023-12-27 04:59:07,214][105620] Updated weights for policy 1, policy_version 1869774 (0.0008) [2023-12-27 04:59:07,240][105692] Updated weights for policy 0, policy_version 1865261 (0.0009) [2023-12-27 04:59:07,886][105620] Updated weights for policy 1, policy_version 1869784 (0.0008) [2023-12-27 04:59:07,936][105620] Updated weights for policy 1, policy_version 1869794 (0.0008) [2023-12-27 04:59:07,993][105620] Updated weights for policy 1, policy_version 1869804 (0.0008) [2023-12-27 04:59:08,014][105692] Updated weights for policy 0, policy_version 1865271 (0.0008) [2023-12-27 04:59:08,065][105692] Updated weights for policy 0, policy_version 1865281 (0.0006) [2023-12-27 04:59:08,113][105692] Updated weights for policy 0, policy_version 1865291 (0.0008) [2023-12-27 04:59:08,800][105620] Updated weights for policy 1, policy_version 1869814 (0.0008) [2023-12-27 04:59:08,858][105620] Updated weights for policy 1, policy_version 1869824 (0.0008) [2023-12-27 04:59:08,874][105692] Updated weights for policy 0, policy_version 1865301 (0.0009) [2023-12-27 04:59:08,913][105620] Updated weights for policy 1, policy_version 1869834 (0.0006) [2023-12-27 04:59:08,934][105692] Updated weights for policy 0, policy_version 1865311 (0.0009) [2023-12-27 04:59:08,991][105692] Updated weights for policy 0, policy_version 1865321 (0.0009) [2023-12-27 04:59:09,715][105620] Updated weights for policy 1, policy_version 1869844 (0.0007) [2023-12-27 04:59:09,777][105620] Updated weights for policy 1, policy_version 1869854 (0.0006) [2023-12-27 04:59:09,779][105692] Updated weights for policy 0, policy_version 1865331 (0.0008) [2023-12-27 04:59:09,839][105620] Updated weights for policy 1, policy_version 1869864 (0.0008) [2023-12-27 04:59:09,853][105692] Updated weights for policy 0, policy_version 1865341 (0.0006) [2023-12-27 04:59:09,914][105692] Updated weights for policy 0, policy_version 1865351 (0.0006) [2023-12-27 04:59:10,598][105620] Updated weights for policy 1, policy_version 1869874 (0.0008) [2023-12-27 04:59:10,653][105620] Updated weights for policy 1, policy_version 1869884 (0.0009) [2023-12-27 04:59:10,695][105692] Updated weights for policy 0, policy_version 1865361 (0.0008) [2023-12-27 04:59:10,704][105620] Updated weights for policy 1, policy_version 1869894 (0.0008) [2023-12-27 04:59:10,753][105620] Updated weights for policy 1, policy_version 1869904 (0.0005) [2023-12-27 04:59:10,763][105692] Updated weights for policy 0, policy_version 1865371 (0.0009) [2023-12-27 04:59:10,816][105692] Updated weights for policy 0, policy_version 1865381 (0.0010) [2023-12-27 04:59:10,878][105692] Updated weights for policy 0, policy_version 1865392 (0.0014) [2023-12-27 04:59:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 956375040. Throughput: 0: 9811.8, 1: 9821.0. Samples: 956381620. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:59:11,062][104569] Avg episode reward: [(0, '8811.709'), (1, '9253.135')] [2023-12-27 04:59:11,493][105620] Updated weights for policy 1, policy_version 1869915 (0.0010) [2023-12-27 04:59:11,562][105620] Updated weights for policy 1, policy_version 1869925 (0.0009) [2023-12-27 04:59:11,629][105692] Updated weights for policy 0, policy_version 1865402 (0.0009) [2023-12-27 04:59:11,629][105620] Updated weights for policy 1, policy_version 1869935 (0.0009) [2023-12-27 04:59:11,700][105692] Updated weights for policy 0, policy_version 1865412 (0.0007) [2023-12-27 04:59:11,769][105692] Updated weights for policy 0, policy_version 1865422 (0.0007) [2023-12-27 04:59:12,383][105620] Updated weights for policy 1, policy_version 1869945 (0.0009) [2023-12-27 04:59:12,448][105620] Updated weights for policy 1, policy_version 1869955 (0.0008) [2023-12-27 04:59:12,483][105692] Updated weights for policy 0, policy_version 1865432 (0.0008) [2023-12-27 04:59:12,508][105620] Updated weights for policy 1, policy_version 1869965 (0.0006) [2023-12-27 04:59:12,540][105692] Updated weights for policy 0, policy_version 1865442 (0.0008) [2023-12-27 04:59:12,594][105692] Updated weights for policy 0, policy_version 1865452 (0.0010) [2023-12-27 04:59:13,137][105620] Updated weights for policy 1, policy_version 1869975 (0.0008) [2023-12-27 04:59:13,185][105620] Updated weights for policy 1, policy_version 1869985 (0.0009) [2023-12-27 04:59:13,247][105620] Updated weights for policy 1, policy_version 1869995 (0.0007) [2023-12-27 04:59:13,393][105692] Updated weights for policy 0, policy_version 1865463 (0.0008) [2023-12-27 04:59:13,453][105692] Updated weights for policy 0, policy_version 1865473 (0.0005) [2023-12-27 04:59:13,521][105692] Updated weights for policy 0, policy_version 1865483 (0.0005) [2023-12-27 04:59:13,854][105620] Updated weights for policy 1, policy_version 1870005 (0.0007) [2023-12-27 04:59:13,919][105620] Updated weights for policy 1, policy_version 1870015 (0.0005) [2023-12-27 04:59:13,980][105620] Updated weights for policy 1, policy_version 1870025 (0.0005) [2023-12-27 04:59:14,035][105692] Updated weights for policy 0, policy_version 1865493 (0.0008) [2023-12-27 04:59:14,090][105692] Updated weights for policy 0, policy_version 1865503 (0.0010) [2023-12-27 04:59:14,148][105692] Updated weights for policy 0, policy_version 1865513 (0.0010) [2023-12-27 04:59:14,649][105620] Updated weights for policy 1, policy_version 1870035 (0.0007) [2023-12-27 04:59:14,707][105620] Updated weights for policy 1, policy_version 1870045 (0.0005) [2023-12-27 04:59:14,777][105620] Updated weights for policy 1, policy_version 1870055 (0.0007) [2023-12-27 04:59:14,870][105692] Updated weights for policy 0, policy_version 1865523 (0.0010) [2023-12-27 04:59:14,933][105692] Updated weights for policy 0, policy_version 1865533 (0.0011) [2023-12-27 04:59:15,002][105692] Updated weights for policy 0, policy_version 1865543 (0.0011) [2023-12-27 04:59:15,499][105620] Updated weights for policy 1, policy_version 1870065 (0.0007) [2023-12-27 04:59:15,561][105620] Updated weights for policy 1, policy_version 1870075 (0.0006) [2023-12-27 04:59:15,620][105620] Updated weights for policy 1, policy_version 1870085 (0.0005) [2023-12-27 04:59:15,675][105620] Updated weights for policy 1, policy_version 1870095 (0.0006) [2023-12-27 04:59:15,733][105692] Updated weights for policy 0, policy_version 1865553 (0.0011) [2023-12-27 04:59:15,792][105692] Updated weights for policy 0, policy_version 1865563 (0.0010) [2023-12-27 04:59:15,856][105692] Updated weights for policy 0, policy_version 1865573 (0.0011) [2023-12-27 04:59:15,912][105692] Updated weights for policy 0, policy_version 1865583 (0.0010) [2023-12-27 04:59:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 956473344. Throughput: 0: 9765.2, 1: 9819.9. Samples: 956439812. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:59:16,063][104569] Avg episode reward: [(0, '8626.394'), (1, '9253.312')] [2023-12-27 04:59:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001870096_478814208.pth... [2023-12-27 04:59:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001865584_477659136.pth... [2023-12-27 04:59:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001864432_477364224.pth [2023-12-27 04:59:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001868944_478519296.pth [2023-12-27 04:59:16,323][105620] Updated weights for policy 1, policy_version 1870105 (0.0005) [2023-12-27 04:59:16,369][105620] Updated weights for policy 1, policy_version 1870115 (0.0005) [2023-12-27 04:59:16,417][105620] Updated weights for policy 1, policy_version 1870125 (0.0005) [2023-12-27 04:59:16,640][105692] Updated weights for policy 0, policy_version 1865593 (0.0006) [2023-12-27 04:59:16,695][105692] Updated weights for policy 0, policy_version 1865603 (0.0005) [2023-12-27 04:59:16,754][105692] Updated weights for policy 0, policy_version 1865613 (0.0005) [2023-12-27 04:59:17,020][105620] Updated weights for policy 1, policy_version 1870135 (0.0007) [2023-12-27 04:59:17,067][105620] Updated weights for policy 1, policy_version 1870145 (0.0009) [2023-12-27 04:59:17,115][105620] Updated weights for policy 1, policy_version 1870155 (0.0009) [2023-12-27 04:59:17,312][105692] Updated weights for policy 0, policy_version 1865623 (0.0005) [2023-12-27 04:59:17,371][105692] Updated weights for policy 0, policy_version 1865633 (0.0006) [2023-12-27 04:59:17,417][105692] Updated weights for policy 0, policy_version 1865643 (0.0005) [2023-12-27 04:59:17,732][105620] Updated weights for policy 1, policy_version 1870165 (0.0007) [2023-12-27 04:59:17,789][105620] Updated weights for policy 1, policy_version 1870175 (0.0006) [2023-12-27 04:59:17,842][105620] Updated weights for policy 1, policy_version 1870185 (0.0005) [2023-12-27 04:59:18,062][105692] Updated weights for policy 0, policy_version 1865653 (0.0009) [2023-12-27 04:59:18,120][105692] Updated weights for policy 0, policy_version 1865663 (0.0009) [2023-12-27 04:59:18,174][105692] Updated weights for policy 0, policy_version 1865673 (0.0009) [2023-12-27 04:59:18,463][105620] Updated weights for policy 1, policy_version 1870195 (0.0007) [2023-12-27 04:59:18,523][105620] Updated weights for policy 1, policy_version 1870205 (0.0010) [2023-12-27 04:59:18,589][105620] Updated weights for policy 1, policy_version 1870215 (0.0010) [2023-12-27 04:59:18,905][105692] Updated weights for policy 0, policy_version 1865683 (0.0007) [2023-12-27 04:59:18,968][105692] Updated weights for policy 0, policy_version 1865693 (0.0011) [2023-12-27 04:59:19,027][105692] Updated weights for policy 0, policy_version 1865703 (0.0011) [2023-12-27 04:59:19,340][105620] Updated weights for policy 1, policy_version 1870225 (0.0010) [2023-12-27 04:59:19,404][105620] Updated weights for policy 1, policy_version 1870235 (0.0010) [2023-12-27 04:59:19,466][105620] Updated weights for policy 1, policy_version 1870245 (0.0010) [2023-12-27 04:59:19,531][105620] Updated weights for policy 1, policy_version 1870255 (0.0010) [2023-12-27 04:59:19,727][105692] Updated weights for policy 0, policy_version 1865713 (0.0010) [2023-12-27 04:59:19,779][105692] Updated weights for policy 0, policy_version 1865723 (0.0009) [2023-12-27 04:59:19,838][105692] Updated weights for policy 0, policy_version 1865733 (0.0008) [2023-12-27 04:59:19,898][105692] Updated weights for policy 0, policy_version 1865743 (0.0008) [2023-12-27 04:59:20,272][105620] Updated weights for policy 1, policy_version 1870265 (0.0007) [2023-12-27 04:59:20,334][105620] Updated weights for policy 1, policy_version 1870275 (0.0008) [2023-12-27 04:59:20,400][105620] Updated weights for policy 1, policy_version 1870285 (0.0007) [2023-12-27 04:59:20,738][105692] Updated weights for policy 0, policy_version 1865753 (0.0008) [2023-12-27 04:59:20,791][105692] Updated weights for policy 0, policy_version 1865763 (0.0010) [2023-12-27 04:59:20,842][105692] Updated weights for policy 0, policy_version 1865773 (0.0009) [2023-12-27 04:59:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 956571648. Throughput: 0: 9772.7, 1: 9854.9. Samples: 956562172. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:59:21,063][104569] Avg episode reward: [(0, '8536.852'), (1, '9253.331')] [2023-12-27 04:59:21,092][105620] Updated weights for policy 1, policy_version 1870295 (0.0010) [2023-12-27 04:59:21,162][105620] Updated weights for policy 1, policy_version 1870305 (0.0011) [2023-12-27 04:59:21,215][105620] Updated weights for policy 1, policy_version 1870315 (0.0011) [2023-12-27 04:59:21,660][105692] Updated weights for policy 0, policy_version 1865783 (0.0008) [2023-12-27 04:59:21,727][105692] Updated weights for policy 0, policy_version 1865793 (0.0009) [2023-12-27 04:59:21,794][105692] Updated weights for policy 0, policy_version 1865803 (0.0009) [2023-12-27 04:59:22,040][105620] Updated weights for policy 1, policy_version 1870325 (0.0010) [2023-12-27 04:59:22,099][105620] Updated weights for policy 1, policy_version 1870335 (0.0009) [2023-12-27 04:59:22,161][105620] Updated weights for policy 1, policy_version 1870345 (0.0008) [2023-12-27 04:59:22,454][105692] Updated weights for policy 0, policy_version 1865813 (0.0008) [2023-12-27 04:59:22,506][105692] Updated weights for policy 0, policy_version 1865823 (0.0009) [2023-12-27 04:59:22,556][105692] Updated weights for policy 0, policy_version 1865833 (0.0008) [2023-12-27 04:59:22,990][105620] Updated weights for policy 1, policy_version 1870355 (0.0009) [2023-12-27 04:59:23,038][105620] Updated weights for policy 1, policy_version 1870365 (0.0009) [2023-12-27 04:59:23,096][105620] Updated weights for policy 1, policy_version 1870375 (0.0009) [2023-12-27 04:59:23,183][105692] Updated weights for policy 0, policy_version 1865843 (0.0007) [2023-12-27 04:59:23,228][105692] Updated weights for policy 0, policy_version 1865853 (0.0005) [2023-12-27 04:59:23,286][105692] Updated weights for policy 0, policy_version 1865863 (0.0009) [2023-12-27 04:59:23,737][105620] Updated weights for policy 1, policy_version 1870385 (0.0009) [2023-12-27 04:59:23,794][105620] Updated weights for policy 1, policy_version 1870395 (0.0010) [2023-12-27 04:59:23,846][105620] Updated weights for policy 1, policy_version 1870405 (0.0010) [2023-12-27 04:59:23,890][105620] Updated weights for policy 1, policy_version 1870415 (0.0010) [2023-12-27 04:59:24,107][105692] Updated weights for policy 0, policy_version 1865873 (0.0009) [2023-12-27 04:59:24,164][105692] Updated weights for policy 0, policy_version 1865883 (0.0009) [2023-12-27 04:59:24,221][105692] Updated weights for policy 0, policy_version 1865893 (0.0008) [2023-12-27 04:59:24,287][105692] Updated weights for policy 0, policy_version 1865903 (0.0006) [2023-12-27 04:59:24,553][105620] Updated weights for policy 1, policy_version 1870425 (0.0010) [2023-12-27 04:59:24,601][105620] Updated weights for policy 1, policy_version 1870435 (0.0010) [2023-12-27 04:59:24,662][105620] Updated weights for policy 1, policy_version 1870445 (0.0010) [2023-12-27 04:59:24,922][105692] Updated weights for policy 0, policy_version 1865913 (0.0007) [2023-12-27 04:59:24,984][105692] Updated weights for policy 0, policy_version 1865923 (0.0005) [2023-12-27 04:59:25,050][105692] Updated weights for policy 0, policy_version 1865933 (0.0006) [2023-12-27 04:59:25,433][105620] Updated weights for policy 1, policy_version 1870455 (0.0011) [2023-12-27 04:59:25,489][105620] Updated weights for policy 1, policy_version 1870465 (0.0010) [2023-12-27 04:59:25,545][105620] Updated weights for policy 1, policy_version 1870475 (0.0010) [2023-12-27 04:59:25,768][105692] Updated weights for policy 0, policy_version 1865943 (0.0006) [2023-12-27 04:59:25,830][105692] Updated weights for policy 0, policy_version 1865953 (0.0007) [2023-12-27 04:59:25,888][105692] Updated weights for policy 0, policy_version 1865963 (0.0010) [2023-12-27 04:59:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 956669952. Throughput: 0: 9758.1, 1: 9801.2. Samples: 956677568. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 04:59:26,062][104569] Avg episode reward: [(0, '8718.831'), (1, '9345.696')] [2023-12-27 04:59:26,181][105620] Updated weights for policy 1, policy_version 1870485 (0.0008) [2023-12-27 04:59:26,230][105620] Updated weights for policy 1, policy_version 1870495 (0.0008) [2023-12-27 04:59:26,286][105620] Updated weights for policy 1, policy_version 1870505 (0.0011) [2023-12-27 04:59:26,633][105692] Updated weights for policy 0, policy_version 1865973 (0.0008) [2023-12-27 04:59:26,677][105692] Updated weights for policy 0, policy_version 1865983 (0.0005) [2023-12-27 04:59:26,732][105692] Updated weights for policy 0, policy_version 1865993 (0.0005) [2023-12-27 04:59:26,984][105620] Updated weights for policy 1, policy_version 1870515 (0.0010) [2023-12-27 04:59:27,031][105620] Updated weights for policy 1, policy_version 1870525 (0.0010) [2023-12-27 04:59:27,082][105620] Updated weights for policy 1, policy_version 1870535 (0.0010) [2023-12-27 04:59:27,307][105692] Updated weights for policy 0, policy_version 1866003 (0.0007) [2023-12-27 04:59:27,365][105692] Updated weights for policy 0, policy_version 1866013 (0.0010) [2023-12-27 04:59:27,413][105692] Updated weights for policy 0, policy_version 1866023 (0.0010) [2023-12-27 04:59:27,840][105620] Updated weights for policy 1, policy_version 1870545 (0.0010) [2023-12-27 04:59:27,894][105620] Updated weights for policy 1, policy_version 1870555 (0.0010) [2023-12-27 04:59:27,949][105620] Updated weights for policy 1, policy_version 1870565 (0.0010) [2023-12-27 04:59:27,993][105692] Updated weights for policy 0, policy_version 1866033 (0.0008) [2023-12-27 04:59:28,000][105620] Updated weights for policy 1, policy_version 1870575 (0.0010) [2023-12-27 04:59:28,040][105692] Updated weights for policy 0, policy_version 1866043 (0.0005) [2023-12-27 04:59:28,089][105692] Updated weights for policy 0, policy_version 1866053 (0.0005) [2023-12-27 04:59:28,134][105692] Updated weights for policy 0, policy_version 1866063 (0.0005) [2023-12-27 04:59:28,694][105620] Updated weights for policy 1, policy_version 1870585 (0.0006) [2023-12-27 04:59:28,713][105692] Updated weights for policy 0, policy_version 1866073 (0.0009) [2023-12-27 04:59:28,754][105620] Updated weights for policy 1, policy_version 1870595 (0.0006) [2023-12-27 04:59:28,766][105692] Updated weights for policy 0, policy_version 1866083 (0.0011) [2023-12-27 04:59:28,813][105620] Updated weights for policy 1, policy_version 1870605 (0.0005) [2023-12-27 04:59:28,818][105692] Updated weights for policy 0, policy_version 1866093 (0.0010) [2023-12-27 04:59:29,510][105620] Updated weights for policy 1, policy_version 1870615 (0.0005) [2023-12-27 04:59:29,555][105692] Updated weights for policy 0, policy_version 1866103 (0.0011) [2023-12-27 04:59:29,569][105620] Updated weights for policy 1, policy_version 1870625 (0.0005) [2023-12-27 04:59:29,618][105692] Updated weights for policy 0, policy_version 1866113 (0.0011) [2023-12-27 04:59:29,631][105620] Updated weights for policy 1, policy_version 1870635 (0.0005) [2023-12-27 04:59:29,685][105692] Updated weights for policy 0, policy_version 1866123 (0.0011) [2023-12-27 04:59:30,234][105620] Updated weights for policy 1, policy_version 1870645 (0.0008) [2023-12-27 04:59:30,290][105620] Updated weights for policy 1, policy_version 1870655 (0.0010) [2023-12-27 04:59:30,345][105620] Updated weights for policy 1, policy_version 1870665 (0.0010) [2023-12-27 04:59:30,433][105692] Updated weights for policy 0, policy_version 1866133 (0.0010) [2023-12-27 04:59:30,492][105692] Updated weights for policy 0, policy_version 1866143 (0.0011) [2023-12-27 04:59:30,553][105692] Updated weights for policy 0, policy_version 1866153 (0.0011) [2023-12-27 04:59:31,013][105620] Updated weights for policy 1, policy_version 1870675 (0.0009) [2023-12-27 04:59:31,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 956768256. Throughput: 0: 9861.5, 1: 9830.6. Samples: 956741012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:59:31,063][104569] Avg episode reward: [(0, '8357.094'), (1, '9070.910')] [2023-12-27 04:59:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001866160_477806592.pth... [2023-12-27 04:59:31,072][105620] Updated weights for policy 1, policy_version 1870685 (0.0006) [2023-12-27 04:59:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001865008_477511680.pth [2023-12-27 04:59:31,143][105620] Updated weights for policy 1, policy_version 1870695 (0.0008) [2023-12-27 04:59:31,196][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001870704_478969856.pth... [2023-12-27 04:59:31,200][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001869520_478666752.pth [2023-12-27 04:59:31,312][105692] Updated weights for policy 0, policy_version 1866163 (0.0010) [2023-12-27 04:59:31,373][105692] Updated weights for policy 0, policy_version 1866173 (0.0008) [2023-12-27 04:59:31,437][105692] Updated weights for policy 0, policy_version 1866183 (0.0009) [2023-12-27 04:59:31,843][105620] Updated weights for policy 1, policy_version 1870705 (0.0008) [2023-12-27 04:59:31,900][105620] Updated weights for policy 1, policy_version 1870715 (0.0005) [2023-12-27 04:59:31,965][105620] Updated weights for policy 1, policy_version 1870725 (0.0007) [2023-12-27 04:59:32,013][105620] Updated weights for policy 1, policy_version 1870735 (0.0009) [2023-12-27 04:59:32,214][105692] Updated weights for policy 0, policy_version 1866193 (0.0008) [2023-12-27 04:59:32,275][105692] Updated weights for policy 0, policy_version 1866203 (0.0009) [2023-12-27 04:59:32,343][105692] Updated weights for policy 0, policy_version 1866213 (0.0009) [2023-12-27 04:59:32,403][105692] Updated weights for policy 0, policy_version 1866223 (0.0008) [2023-12-27 04:59:32,759][105620] Updated weights for policy 1, policy_version 1870745 (0.0010) [2023-12-27 04:59:32,807][105620] Updated weights for policy 1, policy_version 1870755 (0.0009) [2023-12-27 04:59:32,853][105620] Updated weights for policy 1, policy_version 1870765 (0.0008) [2023-12-27 04:59:33,021][105692] Updated weights for policy 0, policy_version 1866233 (0.0006) [2023-12-27 04:59:33,080][105692] Updated weights for policy 0, policy_version 1866243 (0.0008) [2023-12-27 04:59:33,137][105692] Updated weights for policy 0, policy_version 1866254 (0.0010) [2023-12-27 04:59:33,534][105620] Updated weights for policy 1, policy_version 1870775 (0.0008) [2023-12-27 04:59:33,580][105620] Updated weights for policy 1, policy_version 1870785 (0.0009) [2023-12-27 04:59:33,641][105620] Updated weights for policy 1, policy_version 1870795 (0.0009) [2023-12-27 04:59:33,825][105692] Updated weights for policy 0, policy_version 1866264 (0.0007) [2023-12-27 04:59:33,894][105692] Updated weights for policy 0, policy_version 1866274 (0.0006) [2023-12-27 04:59:33,958][105692] Updated weights for policy 0, policy_version 1866284 (0.0005) [2023-12-27 04:59:34,521][105692] Updated weights for policy 0, policy_version 1866294 (0.0005) [2023-12-27 04:59:34,521][105620] Updated weights for policy 1, policy_version 1870805 (0.0008) [2023-12-27 04:59:34,582][105692] Updated weights for policy 0, policy_version 1866304 (0.0008) [2023-12-27 04:59:34,583][105620] Updated weights for policy 1, policy_version 1870815 (0.0008) [2023-12-27 04:59:34,644][105620] Updated weights for policy 1, policy_version 1870825 (0.0007) [2023-12-27 04:59:34,645][105692] Updated weights for policy 0, policy_version 1866314 (0.0009) [2023-12-27 04:59:35,319][105692] Updated weights for policy 0, policy_version 1866324 (0.0010) [2023-12-27 04:59:35,374][105692] Updated weights for policy 0, policy_version 1866334 (0.0010) [2023-12-27 04:59:35,413][105620] Updated weights for policy 1, policy_version 1870835 (0.0006) [2023-12-27 04:59:35,426][105692] Updated weights for policy 0, policy_version 1866344 (0.0010) [2023-12-27 04:59:35,478][105620] Updated weights for policy 1, policy_version 1870845 (0.0008) [2023-12-27 04:59:35,536][105620] Updated weights for policy 1, policy_version 1870855 (0.0009) [2023-12-27 04:59:36,007][105692] Updated weights for policy 0, policy_version 1866354 (0.0008) [2023-12-27 04:59:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 956866560. Throughput: 0: 9838.9, 1: 9879.2. Samples: 956857908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:59:36,063][104569] Avg episode reward: [(0, '8356.832'), (1, '8978.482')] [2023-12-27 04:59:36,066][105692] Updated weights for policy 0, policy_version 1866364 (0.0006) [2023-12-27 04:59:36,128][105692] Updated weights for policy 0, policy_version 1866374 (0.0008) [2023-12-27 04:59:36,187][105692] Updated weights for policy 0, policy_version 1866384 (0.0010) [2023-12-27 04:59:36,405][105620] Updated weights for policy 1, policy_version 1870865 (0.0009) [2023-12-27 04:59:36,465][105620] Updated weights for policy 1, policy_version 1870875 (0.0008) [2023-12-27 04:59:36,526][105620] Updated weights for policy 1, policy_version 1870885 (0.0009) [2023-12-27 04:59:36,588][105620] Updated weights for policy 1, policy_version 1870895 (0.0009) [2023-12-27 04:59:36,909][105692] Updated weights for policy 0, policy_version 1866394 (0.0009) [2023-12-27 04:59:36,970][105692] Updated weights for policy 0, policy_version 1866404 (0.0009) [2023-12-27 04:59:37,024][105692] Updated weights for policy 0, policy_version 1866414 (0.0010) [2023-12-27 04:59:37,371][105620] Updated weights for policy 1, policy_version 1870905 (0.0008) [2023-12-27 04:59:37,431][105620] Updated weights for policy 1, policy_version 1870915 (0.0008) [2023-12-27 04:59:37,482][105620] Updated weights for policy 1, policy_version 1870925 (0.0009) [2023-12-27 04:59:37,712][105692] Updated weights for policy 0, policy_version 1866424 (0.0006) [2023-12-27 04:59:37,770][105692] Updated weights for policy 0, policy_version 1866434 (0.0006) [2023-12-27 04:59:37,830][105692] Updated weights for policy 0, policy_version 1866444 (0.0008) [2023-12-27 04:59:38,283][105620] Updated weights for policy 1, policy_version 1870935 (0.0009) [2023-12-27 04:59:38,348][105620] Updated weights for policy 1, policy_version 1870945 (0.0008) [2023-12-27 04:59:38,404][105620] Updated weights for policy 1, policy_version 1870955 (0.0009) [2023-12-27 04:59:38,543][105692] Updated weights for policy 0, policy_version 1866454 (0.0010) [2023-12-27 04:59:38,595][105692] Updated weights for policy 0, policy_version 1866464 (0.0009) [2023-12-27 04:59:38,651][105692] Updated weights for policy 0, policy_version 1866474 (0.0009) [2023-12-27 04:59:39,087][105620] Updated weights for policy 1, policy_version 1870965 (0.0008) [2023-12-27 04:59:39,142][105620] Updated weights for policy 1, policy_version 1870975 (0.0006) [2023-12-27 04:59:39,193][105620] Updated weights for policy 1, policy_version 1870985 (0.0006) [2023-12-27 04:59:39,481][105692] Updated weights for policy 0, policy_version 1866484 (0.0008) [2023-12-27 04:59:39,538][105692] Updated weights for policy 0, policy_version 1866494 (0.0009) [2023-12-27 04:59:39,597][105692] Updated weights for policy 0, policy_version 1866504 (0.0007) [2023-12-27 04:59:39,941][105620] Updated weights for policy 1, policy_version 1870995 (0.0008) [2023-12-27 04:59:40,005][105620] Updated weights for policy 1, policy_version 1871005 (0.0008) [2023-12-27 04:59:40,059][105620] Updated weights for policy 1, policy_version 1871015 (0.0008) [2023-12-27 04:59:40,304][105692] Updated weights for policy 0, policy_version 1866514 (0.0008) [2023-12-27 04:59:40,363][105692] Updated weights for policy 0, policy_version 1866524 (0.0010) [2023-12-27 04:59:40,423][105692] Updated weights for policy 0, policy_version 1866534 (0.0009) [2023-12-27 04:59:40,486][105692] Updated weights for policy 0, policy_version 1866544 (0.0009) [2023-12-27 04:59:40,784][105620] Updated weights for policy 1, policy_version 1871025 (0.0009) [2023-12-27 04:59:40,840][105620] Updated weights for policy 1, policy_version 1871035 (0.0009) [2023-12-27 04:59:40,892][105620] Updated weights for policy 1, policy_version 1871045 (0.0009) [2023-12-27 04:59:40,945][105620] Updated weights for policy 1, policy_version 1871055 (0.0009) [2023-12-27 04:59:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 956964864. Throughput: 0: 9809.3, 1: 9756.4. Samples: 956971540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:59:41,062][104569] Avg episode reward: [(0, '8627.423'), (1, '9161.018')] [2023-12-27 04:59:41,188][105692] Updated weights for policy 0, policy_version 1866554 (0.0008) [2023-12-27 04:59:41,260][105692] Updated weights for policy 0, policy_version 1866564 (0.0006) [2023-12-27 04:59:41,323][105692] Updated weights for policy 0, policy_version 1866574 (0.0008) [2023-12-27 04:59:41,786][105620] Updated weights for policy 1, policy_version 1871065 (0.0008) [2023-12-27 04:59:41,856][105620] Updated weights for policy 1, policy_version 1871075 (0.0006) [2023-12-27 04:59:41,926][105620] Updated weights for policy 1, policy_version 1871085 (0.0006) [2023-12-27 04:59:42,030][105692] Updated weights for policy 0, policy_version 1866584 (0.0006) [2023-12-27 04:59:42,088][105692] Updated weights for policy 0, policy_version 1866594 (0.0009) [2023-12-27 04:59:42,141][105692] Updated weights for policy 0, policy_version 1866604 (0.0010) [2023-12-27 04:59:42,496][105620] Updated weights for policy 1, policy_version 1871095 (0.0005) [2023-12-27 04:59:42,551][105620] Updated weights for policy 1, policy_version 1871105 (0.0007) [2023-12-27 04:59:42,617][105620] Updated weights for policy 1, policy_version 1871115 (0.0008) [2023-12-27 04:59:43,051][105692] Updated weights for policy 0, policy_version 1866614 (0.0009) [2023-12-27 04:59:43,108][105692] Updated weights for policy 0, policy_version 1866624 (0.0010) [2023-12-27 04:59:43,172][105692] Updated weights for policy 0, policy_version 1866634 (0.0009) [2023-12-27 04:59:43,270][105620] Updated weights for policy 1, policy_version 1871125 (0.0010) [2023-12-27 04:59:43,337][105620] Updated weights for policy 1, policy_version 1871135 (0.0009) [2023-12-27 04:59:43,402][105620] Updated weights for policy 1, policy_version 1871145 (0.0009) [2023-12-27 04:59:43,972][105620] Updated weights for policy 1, policy_version 1871155 (0.0007) [2023-12-27 04:59:44,002][105692] Updated weights for policy 0, policy_version 1866644 (0.0009) [2023-12-27 04:59:44,026][105620] Updated weights for policy 1, policy_version 1871165 (0.0005) [2023-12-27 04:59:44,050][105692] Updated weights for policy 0, policy_version 1866654 (0.0009) [2023-12-27 04:59:44,077][105620] Updated weights for policy 1, policy_version 1871175 (0.0006) [2023-12-27 04:59:44,105][105692] Updated weights for policy 0, policy_version 1866664 (0.0008) [2023-12-27 04:59:44,623][105620] Updated weights for policy 1, policy_version 1871185 (0.0006) [2023-12-27 04:59:44,680][105620] Updated weights for policy 1, policy_version 1871195 (0.0005) [2023-12-27 04:59:44,736][105620] Updated weights for policy 1, policy_version 1871205 (0.0005) [2023-12-27 04:59:44,791][105620] Updated weights for policy 1, policy_version 1871215 (0.0008) [2023-12-27 04:59:44,942][105692] Updated weights for policy 0, policy_version 1866674 (0.0009) [2023-12-27 04:59:44,999][105692] Updated weights for policy 0, policy_version 1866684 (0.0006) [2023-12-27 04:59:45,057][105692] Updated weights for policy 0, policy_version 1866694 (0.0006) [2023-12-27 04:59:45,108][105692] Updated weights for policy 0, policy_version 1866704 (0.0006) [2023-12-27 04:59:45,570][105620] Updated weights for policy 1, policy_version 1871225 (0.0009) [2023-12-27 04:59:45,626][105620] Updated weights for policy 1, policy_version 1871235 (0.0009) [2023-12-27 04:59:45,673][105620] Updated weights for policy 1, policy_version 1871245 (0.0008) [2023-12-27 04:59:45,722][105692] Updated weights for policy 0, policy_version 1866714 (0.0009) [2023-12-27 04:59:45,775][105692] Updated weights for policy 0, policy_version 1866724 (0.0008) [2023-12-27 04:59:45,834][105692] Updated weights for policy 0, policy_version 1866734 (0.0005) [2023-12-27 04:59:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 957063168. Throughput: 0: 9795.2, 1: 9761.0. Samples: 957030756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:59:46,062][104569] Avg episode reward: [(0, '8715.487'), (1, '9254.655')] [2023-12-27 04:59:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001866736_477954048.pth... [2023-12-27 04:59:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001871248_479109120.pth... [2023-12-27 04:59:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001865584_477659136.pth [2023-12-27 04:59:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001870096_478814208.pth [2023-12-27 04:59:46,440][105692] Updated weights for policy 0, policy_version 1866744 (0.0009) [2023-12-27 04:59:46,492][105692] Updated weights for policy 0, policy_version 1866754 (0.0010) [2023-12-27 04:59:46,530][105620] Updated weights for policy 1, policy_version 1871255 (0.0007) [2023-12-27 04:59:46,540][105692] Updated weights for policy 0, policy_version 1866764 (0.0010) [2023-12-27 04:59:46,586][105620] Updated weights for policy 1, policy_version 1871265 (0.0008) [2023-12-27 04:59:46,634][105620] Updated weights for policy 1, policy_version 1871275 (0.0008) [2023-12-27 04:59:47,226][105692] Updated weights for policy 0, policy_version 1866774 (0.0005) [2023-12-27 04:59:47,286][105692] Updated weights for policy 0, policy_version 1866784 (0.0005) [2023-12-27 04:59:47,337][105692] Updated weights for policy 0, policy_version 1866794 (0.0005) [2023-12-27 04:59:47,442][105620] Updated weights for policy 1, policy_version 1871285 (0.0008) [2023-12-27 04:59:47,493][105620] Updated weights for policy 1, policy_version 1871295 (0.0008) [2023-12-27 04:59:47,545][105620] Updated weights for policy 1, policy_version 1871305 (0.0008) [2023-12-27 04:59:48,013][105692] Updated weights for policy 0, policy_version 1866804 (0.0007) [2023-12-27 04:59:48,071][105692] Updated weights for policy 0, policy_version 1866814 (0.0010) [2023-12-27 04:59:48,123][105692] Updated weights for policy 0, policy_version 1866824 (0.0010) [2023-12-27 04:59:48,258][105620] Updated weights for policy 1, policy_version 1871315 (0.0009) [2023-12-27 04:59:48,324][105620] Updated weights for policy 1, policy_version 1871325 (0.0009) [2023-12-27 04:59:48,388][105620] Updated weights for policy 1, policy_version 1871335 (0.0008) [2023-12-27 04:59:48,929][105692] Updated weights for policy 0, policy_version 1866834 (0.0010) [2023-12-27 04:59:48,992][105692] Updated weights for policy 0, policy_version 1866844 (0.0009) [2023-12-27 04:59:48,993][105620] Updated weights for policy 1, policy_version 1871345 (0.0008) [2023-12-27 04:59:49,046][105620] Updated weights for policy 1, policy_version 1871355 (0.0007) [2023-12-27 04:59:49,048][105692] Updated weights for policy 0, policy_version 1866854 (0.0006) [2023-12-27 04:59:49,094][105620] Updated weights for policy 1, policy_version 1871365 (0.0006) [2023-12-27 04:59:49,103][105692] Updated weights for policy 0, policy_version 1866864 (0.0008) [2023-12-27 04:59:49,143][105620] Updated weights for policy 1, policy_version 1871375 (0.0009) [2023-12-27 04:59:49,852][105620] Updated weights for policy 1, policy_version 1871385 (0.0009) [2023-12-27 04:59:49,912][105620] Updated weights for policy 1, policy_version 1871395 (0.0006) [2023-12-27 04:59:49,940][105692] Updated weights for policy 0, policy_version 1866874 (0.0008) [2023-12-27 04:59:49,986][105620] Updated weights for policy 1, policy_version 1871405 (0.0008) [2023-12-27 04:59:50,010][105692] Updated weights for policy 0, policy_version 1866884 (0.0008) [2023-12-27 04:59:50,063][105692] Updated weights for policy 0, policy_version 1866894 (0.0009) [2023-12-27 04:59:50,700][105620] Updated weights for policy 1, policy_version 1871415 (0.0007) [2023-12-27 04:59:50,768][105620] Updated weights for policy 1, policy_version 1871425 (0.0008) [2023-12-27 04:59:50,830][105620] Updated weights for policy 1, policy_version 1871435 (0.0009) [2023-12-27 04:59:50,836][105692] Updated weights for policy 0, policy_version 1866904 (0.0008) [2023-12-27 04:59:50,895][105692] Updated weights for policy 0, policy_version 1866914 (0.0008) [2023-12-27 04:59:50,955][105692] Updated weights for policy 0, policy_version 1866924 (0.0009) [2023-12-27 04:59:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 957161472. Throughput: 0: 9817.2, 1: 9779.0. Samples: 957147328. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:59:51,063][104569] Avg episode reward: [(0, '8894.774'), (1, '9162.327')] [2023-12-27 04:59:51,567][105620] Updated weights for policy 1, policy_version 1871445 (0.0008) [2023-12-27 04:59:51,630][105620] Updated weights for policy 1, policy_version 1871455 (0.0008) [2023-12-27 04:59:51,677][105620] Updated weights for policy 1, policy_version 1871465 (0.0009) [2023-12-27 04:59:51,686][105692] Updated weights for policy 0, policy_version 1866934 (0.0008) [2023-12-27 04:59:51,754][105692] Updated weights for policy 0, policy_version 1866944 (0.0008) [2023-12-27 04:59:51,816][105692] Updated weights for policy 0, policy_version 1866954 (0.0007) [2023-12-27 04:59:52,468][105692] Updated weights for policy 0, policy_version 1866964 (0.0007) [2023-12-27 04:59:52,503][105620] Updated weights for policy 1, policy_version 1871475 (0.0009) [2023-12-27 04:59:52,529][105692] Updated weights for policy 0, policy_version 1866974 (0.0008) [2023-12-27 04:59:52,569][105620] Updated weights for policy 1, policy_version 1871485 (0.0011) [2023-12-27 04:59:52,592][105692] Updated weights for policy 0, policy_version 1866984 (0.0008) [2023-12-27 04:59:52,640][105620] Updated weights for policy 1, policy_version 1871495 (0.0010) [2023-12-27 04:59:53,205][105692] Updated weights for policy 0, policy_version 1866994 (0.0009) [2023-12-27 04:59:53,253][105692] Updated weights for policy 0, policy_version 1867004 (0.0008) [2023-12-27 04:59:53,304][105692] Updated weights for policy 0, policy_version 1867014 (0.0008) [2023-12-27 04:59:53,355][105692] Updated weights for policy 0, policy_version 1867024 (0.0007) [2023-12-27 04:59:53,370][105620] Updated weights for policy 1, policy_version 1871505 (0.0011) [2023-12-27 04:59:53,414][105620] Updated weights for policy 1, policy_version 1871515 (0.0010) [2023-12-27 04:59:53,471][105620] Updated weights for policy 1, policy_version 1871525 (0.0006) [2023-12-27 04:59:53,540][105620] Updated weights for policy 1, policy_version 1871535 (0.0005) [2023-12-27 04:59:54,097][105620] Updated weights for policy 1, policy_version 1871545 (0.0009) [2023-12-27 04:59:54,150][105620] Updated weights for policy 1, policy_version 1871555 (0.0007) [2023-12-27 04:59:54,177][105692] Updated weights for policy 0, policy_version 1867034 (0.0007) [2023-12-27 04:59:54,219][105620] Updated weights for policy 1, policy_version 1871565 (0.0010) [2023-12-27 04:59:54,227][105692] Updated weights for policy 0, policy_version 1867044 (0.0009) [2023-12-27 04:59:54,284][105692] Updated weights for policy 0, policy_version 1867055 (0.0010) [2023-12-27 04:59:54,784][105620] Updated weights for policy 1, policy_version 1871575 (0.0005) [2023-12-27 04:59:54,830][105620] Updated weights for policy 1, policy_version 1871585 (0.0005) [2023-12-27 04:59:54,878][105620] Updated weights for policy 1, policy_version 1871595 (0.0005) [2023-12-27 04:59:55,164][105692] Updated weights for policy 0, policy_version 1867065 (0.0009) [2023-12-27 04:59:55,220][105692] Updated weights for policy 0, policy_version 1867075 (0.0008) [2023-12-27 04:59:55,281][105692] Updated weights for policy 0, policy_version 1867085 (0.0008) [2023-12-27 04:59:55,583][105620] Updated weights for policy 1, policy_version 1871605 (0.0009) [2023-12-27 04:59:55,645][105620] Updated weights for policy 1, policy_version 1871615 (0.0006) [2023-12-27 04:59:55,707][105620] Updated weights for policy 1, policy_version 1871625 (0.0006) [2023-12-27 04:59:56,058][105692] Updated weights for policy 0, policy_version 1867095 (0.0008) [2023-12-27 04:59:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 957251584. Throughput: 0: 9749.0, 1: 9833.0. Samples: 957262808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 04:59:56,063][104569] Avg episode reward: [(0, '8532.481'), (1, '9070.608')] [2023-12-27 04:59:56,114][105692] Updated weights for policy 0, policy_version 1867105 (0.0007) [2023-12-27 04:59:56,176][105692] Updated weights for policy 0, policy_version 1867115 (0.0008) [2023-12-27 04:59:56,415][105620] Updated weights for policy 1, policy_version 1871635 (0.0008) [2023-12-27 04:59:56,465][105620] Updated weights for policy 1, policy_version 1871645 (0.0005) [2023-12-27 04:59:56,516][105620] Updated weights for policy 1, policy_version 1871655 (0.0005) [2023-12-27 04:59:56,995][105692] Updated weights for policy 0, policy_version 1867125 (0.0008) [2023-12-27 04:59:57,038][105692] Updated weights for policy 0, policy_version 1867135 (0.0007) [2023-12-27 04:59:57,087][105692] Updated weights for policy 0, policy_version 1867145 (0.0006) [2023-12-27 04:59:57,123][105620] Updated weights for policy 1, policy_version 1871665 (0.0006) [2023-12-27 04:59:57,194][105620] Updated weights for policy 1, policy_version 1871675 (0.0008) [2023-12-27 04:59:57,261][105620] Updated weights for policy 1, policy_version 1871685 (0.0007) [2023-12-27 04:59:57,324][105620] Updated weights for policy 1, policy_version 1871695 (0.0008) [2023-12-27 04:59:57,658][105692] Updated weights for policy 0, policy_version 1867155 (0.0005) [2023-12-27 04:59:57,714][105692] Updated weights for policy 0, policy_version 1867165 (0.0005) [2023-12-27 04:59:57,768][105692] Updated weights for policy 0, policy_version 1867175 (0.0005) [2023-12-27 04:59:57,989][105620] Updated weights for policy 1, policy_version 1871705 (0.0010) [2023-12-27 04:59:58,048][105620] Updated weights for policy 1, policy_version 1871715 (0.0010) [2023-12-27 04:59:58,104][105620] Updated weights for policy 1, policy_version 1871725 (0.0010) [2023-12-27 04:59:58,423][105692] Updated weights for policy 0, policy_version 1867185 (0.0006) [2023-12-27 04:59:58,486][105692] Updated weights for policy 0, policy_version 1867195 (0.0009) [2023-12-27 04:59:58,552][105692] Updated weights for policy 0, policy_version 1867205 (0.0008) [2023-12-27 04:59:58,611][105692] Updated weights for policy 0, policy_version 1867215 (0.0009) [2023-12-27 04:59:58,853][105620] Updated weights for policy 1, policy_version 1871736 (0.0012) [2023-12-27 04:59:58,924][105620] Updated weights for policy 1, policy_version 1871746 (0.0008) [2023-12-27 04:59:58,990][105620] Updated weights for policy 1, policy_version 1871756 (0.0008) [2023-12-27 04:59:59,410][105692] Updated weights for policy 0, policy_version 1867225 (0.0009) [2023-12-27 04:59:59,466][105692] Updated weights for policy 0, policy_version 1867235 (0.0008) [2023-12-27 04:59:59,523][105692] Updated weights for policy 0, policy_version 1867245 (0.0008) [2023-12-27 04:59:59,738][105620] Updated weights for policy 1, policy_version 1871766 (0.0008) [2023-12-27 04:59:59,794][105620] Updated weights for policy 1, policy_version 1871776 (0.0005) [2023-12-27 04:59:59,849][105620] Updated weights for policy 1, policy_version 1871786 (0.0007) [2023-12-27 05:00:00,224][105692] Updated weights for policy 0, policy_version 1867255 (0.0009) [2023-12-27 05:00:00,285][105692] Updated weights for policy 0, policy_version 1867265 (0.0009) [2023-12-27 05:00:00,346][105692] Updated weights for policy 0, policy_version 1867275 (0.0011) [2023-12-27 05:00:00,609][105620] Updated weights for policy 1, policy_version 1871796 (0.0007) [2023-12-27 05:00:00,667][105620] Updated weights for policy 1, policy_version 1871806 (0.0006) [2023-12-27 05:00:00,724][105620] Updated weights for policy 1, policy_version 1871816 (0.0005) [2023-12-27 05:00:01,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 957349888. Throughput: 0: 9792.1, 1: 9827.3. Samples: 957322684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:00:01,062][104569] Avg episode reward: [(0, '8079.722'), (1, '8978.368')] [2023-12-27 05:00:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001871824_479256576.pth... [2023-12-27 05:00:01,072][105692] Updated weights for policy 0, policy_version 1867285 (0.0009) [2023-12-27 05:00:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001870704_478969856.pth [2023-12-27 05:00:01,140][105692] Updated weights for policy 0, policy_version 1867295 (0.0007) [2023-12-27 05:00:01,200][105692] Updated weights for policy 0, policy_version 1867305 (0.0006) [2023-12-27 05:00:01,245][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001867312_478101504.pth... [2023-12-27 05:00:01,249][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001866160_477806592.pth [2023-12-27 05:00:01,385][105620] Updated weights for policy 1, policy_version 1871826 (0.0008) [2023-12-27 05:00:01,453][105620] Updated weights for policy 1, policy_version 1871836 (0.0008) [2023-12-27 05:00:01,512][105620] Updated weights for policy 1, policy_version 1871846 (0.0010) [2023-12-27 05:00:01,569][105620] Updated weights for policy 1, policy_version 1871856 (0.0010) [2023-12-27 05:00:01,833][105692] Updated weights for policy 0, policy_version 1867315 (0.0007) [2023-12-27 05:00:01,899][105692] Updated weights for policy 0, policy_version 1867325 (0.0009) [2023-12-27 05:00:01,956][105692] Updated weights for policy 0, policy_version 1867335 (0.0009) [2023-12-27 05:00:02,422][105620] Updated weights for policy 1, policy_version 1871866 (0.0008) [2023-12-27 05:00:02,488][105620] Updated weights for policy 1, policy_version 1871876 (0.0007) [2023-12-27 05:00:02,557][105620] Updated weights for policy 1, policy_version 1871886 (0.0007) [2023-12-27 05:00:02,620][105692] Updated weights for policy 0, policy_version 1867345 (0.0009) [2023-12-27 05:00:02,683][105692] Updated weights for policy 0, policy_version 1867355 (0.0009) [2023-12-27 05:00:02,729][105692] Updated weights for policy 0, policy_version 1867365 (0.0009) [2023-12-27 05:00:02,776][105692] Updated weights for policy 0, policy_version 1867375 (0.0009) [2023-12-27 05:00:03,240][105620] Updated weights for policy 1, policy_version 1871896 (0.0008) [2023-12-27 05:00:03,286][105620] Updated weights for policy 1, policy_version 1871906 (0.0009) [2023-12-27 05:00:03,339][105620] Updated weights for policy 1, policy_version 1871916 (0.0008) [2023-12-27 05:00:03,547][105692] Updated weights for policy 0, policy_version 1867385 (0.0008) [2023-12-27 05:00:03,605][105692] Updated weights for policy 0, policy_version 1867395 (0.0009) [2023-12-27 05:00:03,658][105692] Updated weights for policy 0, policy_version 1867405 (0.0006) [2023-12-27 05:00:04,034][105620] Updated weights for policy 1, policy_version 1871926 (0.0009) [2023-12-27 05:00:04,086][105620] Updated weights for policy 1, policy_version 1871936 (0.0011) [2023-12-27 05:00:04,139][105620] Updated weights for policy 1, policy_version 1871946 (0.0011) [2023-12-27 05:00:04,363][105692] Updated weights for policy 0, policy_version 1867415 (0.0009) [2023-12-27 05:00:04,426][105692] Updated weights for policy 0, policy_version 1867425 (0.0011) [2023-12-27 05:00:04,485][105692] Updated weights for policy 0, policy_version 1867435 (0.0011) [2023-12-27 05:00:04,872][105620] Updated weights for policy 1, policy_version 1871956 (0.0008) [2023-12-27 05:00:04,931][105620] Updated weights for policy 1, policy_version 1871966 (0.0005) [2023-12-27 05:00:04,989][105620] Updated weights for policy 1, policy_version 1871976 (0.0005) [2023-12-27 05:00:05,238][105692] Updated weights for policy 0, policy_version 1867445 (0.0009) [2023-12-27 05:00:05,293][105692] Updated weights for policy 0, policy_version 1867455 (0.0008) [2023-12-27 05:00:05,356][105692] Updated weights for policy 0, policy_version 1867465 (0.0008) [2023-12-27 05:00:05,649][105620] Updated weights for policy 1, policy_version 1871986 (0.0007) [2023-12-27 05:00:05,711][105620] Updated weights for policy 1, policy_version 1871996 (0.0010) [2023-12-27 05:00:05,766][105620] Updated weights for policy 1, policy_version 1872006 (0.0010) [2023-12-27 05:00:05,831][105620] Updated weights for policy 1, policy_version 1872016 (0.0010) [2023-12-27 05:00:06,007][105692] Updated weights for policy 0, policy_version 1867475 (0.0007) [2023-12-27 05:00:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 957448192. Throughput: 0: 9720.9, 1: 9741.5. Samples: 957437980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:00:06,062][104569] Avg episode reward: [(0, '7808.766'), (1, '9161.113')] [2023-12-27 05:00:06,069][105692] Updated weights for policy 0, policy_version 1867485 (0.0005) [2023-12-27 05:00:06,132][105692] Updated weights for policy 0, policy_version 1867495 (0.0007) [2023-12-27 05:00:06,503][105620] Updated weights for policy 1, policy_version 1872026 (0.0009) [2023-12-27 05:00:06,570][105620] Updated weights for policy 1, policy_version 1872036 (0.0006) [2023-12-27 05:00:06,633][105620] Updated weights for policy 1, policy_version 1872046 (0.0006) [2023-12-27 05:00:06,685][105692] Updated weights for policy 0, policy_version 1867505 (0.0007) [2023-12-27 05:00:06,754][105692] Updated weights for policy 0, policy_version 1867515 (0.0005) [2023-12-27 05:00:06,815][105692] Updated weights for policy 0, policy_version 1867525 (0.0005) [2023-12-27 05:00:06,874][105692] Updated weights for policy 0, policy_version 1867535 (0.0006) [2023-12-27 05:00:07,226][105620] Updated weights for policy 1, policy_version 1872056 (0.0010) [2023-12-27 05:00:07,285][105620] Updated weights for policy 1, policy_version 1872066 (0.0008) [2023-12-27 05:00:07,345][105620] Updated weights for policy 1, policy_version 1872076 (0.0005) [2023-12-27 05:00:07,468][105692] Updated weights for policy 0, policy_version 1867545 (0.0005) [2023-12-27 05:00:07,526][105692] Updated weights for policy 0, policy_version 1867555 (0.0005) [2023-12-27 05:00:07,584][105692] Updated weights for policy 0, policy_version 1867565 (0.0005) [2023-12-27 05:00:07,888][105620] Updated weights for policy 1, policy_version 1872086 (0.0008) [2023-12-27 05:00:07,937][105620] Updated weights for policy 1, policy_version 1872096 (0.0005) [2023-12-27 05:00:07,986][105620] Updated weights for policy 1, policy_version 1872106 (0.0005) [2023-12-27 05:00:08,183][105692] Updated weights for policy 0, policy_version 1867575 (0.0008) [2023-12-27 05:00:08,241][105692] Updated weights for policy 0, policy_version 1867585 (0.0010) [2023-12-27 05:00:08,291][105692] Updated weights for policy 0, policy_version 1867595 (0.0010) [2023-12-27 05:00:08,720][105620] Updated weights for policy 1, policy_version 1872116 (0.0010) [2023-12-27 05:00:08,778][105620] Updated weights for policy 1, policy_version 1872126 (0.0010) [2023-12-27 05:00:08,844][105620] Updated weights for policy 1, policy_version 1872136 (0.0010) [2023-12-27 05:00:08,993][105692] Updated weights for policy 0, policy_version 1867605 (0.0008) [2023-12-27 05:00:09,053][105692] Updated weights for policy 0, policy_version 1867615 (0.0006) [2023-12-27 05:00:09,116][105692] Updated weights for policy 0, policy_version 1867625 (0.0005) [2023-12-27 05:00:09,597][105620] Updated weights for policy 1, policy_version 1872146 (0.0010) [2023-12-27 05:00:09,661][105620] Updated weights for policy 1, policy_version 1872156 (0.0011) [2023-12-27 05:00:09,720][105620] Updated weights for policy 1, policy_version 1872166 (0.0011) [2023-12-27 05:00:09,784][105620] Updated weights for policy 1, policy_version 1872176 (0.0011) [2023-12-27 05:00:09,861][105692] Updated weights for policy 0, policy_version 1867635 (0.0006) [2023-12-27 05:00:09,915][105692] Updated weights for policy 0, policy_version 1867645 (0.0009) [2023-12-27 05:00:09,975][105692] Updated weights for policy 0, policy_version 1867655 (0.0008) [2023-12-27 05:00:10,558][105620] Updated weights for policy 1, policy_version 1872186 (0.0011) [2023-12-27 05:00:10,618][105620] Updated weights for policy 1, policy_version 1872196 (0.0010) [2023-12-27 05:00:10,630][105692] Updated weights for policy 0, policy_version 1867665 (0.0007) [2023-12-27 05:00:10,674][105620] Updated weights for policy 1, policy_version 1872206 (0.0011) [2023-12-27 05:00:10,688][105692] Updated weights for policy 0, policy_version 1867675 (0.0006) [2023-12-27 05:00:10,737][105692] Updated weights for policy 0, policy_version 1867685 (0.0008) [2023-12-27 05:00:10,801][105692] Updated weights for policy 0, policy_version 1867695 (0.0008) [2023-12-27 05:00:11,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 957554688. Throughput: 0: 9830.4, 1: 9803.8. Samples: 957561108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:00:11,063][104569] Avg episode reward: [(0, '8168.627'), (1, '9253.199')] [2023-12-27 05:00:11,426][105620] Updated weights for policy 1, policy_version 1872216 (0.0009) [2023-12-27 05:00:11,475][105620] Updated weights for policy 1, policy_version 1872226 (0.0008) [2023-12-27 05:00:11,525][105620] Updated weights for policy 1, policy_version 1872236 (0.0008) [2023-12-27 05:00:11,597][105692] Updated weights for policy 0, policy_version 1867705 (0.0008) [2023-12-27 05:00:11,662][105692] Updated weights for policy 0, policy_version 1867715 (0.0007) [2023-12-27 05:00:11,729][105692] Updated weights for policy 0, policy_version 1867725 (0.0009) [2023-12-27 05:00:12,286][105620] Updated weights for policy 1, policy_version 1872246 (0.0009) [2023-12-27 05:00:12,344][105620] Updated weights for policy 1, policy_version 1872256 (0.0008) [2023-12-27 05:00:12,404][105620] Updated weights for policy 1, policy_version 1872266 (0.0009) [2023-12-27 05:00:12,563][105692] Updated weights for policy 0, policy_version 1867735 (0.0010) [2023-12-27 05:00:12,622][105692] Updated weights for policy 0, policy_version 1867745 (0.0010) [2023-12-27 05:00:12,677][105692] Updated weights for policy 0, policy_version 1867755 (0.0008) [2023-12-27 05:00:13,119][105620] Updated weights for policy 1, policy_version 1872276 (0.0008) [2023-12-27 05:00:13,173][105620] Updated weights for policy 1, policy_version 1872286 (0.0009) [2023-12-27 05:00:13,230][105620] Updated weights for policy 1, policy_version 1872296 (0.0008) [2023-12-27 05:00:13,352][105692] Updated weights for policy 0, policy_version 1867765 (0.0009) [2023-12-27 05:00:13,410][105692] Updated weights for policy 0, policy_version 1867775 (0.0009) [2023-12-27 05:00:13,469][105692] Updated weights for policy 0, policy_version 1867785 (0.0006) [2023-12-27 05:00:13,901][105620] Updated weights for policy 1, policy_version 1872306 (0.0008) [2023-12-27 05:00:13,953][105620] Updated weights for policy 1, policy_version 1872316 (0.0006) [2023-12-27 05:00:14,002][105620] Updated weights for policy 1, policy_version 1872326 (0.0006) [2023-12-27 05:00:14,068][105620] Updated weights for policy 1, policy_version 1872336 (0.0007) [2023-12-27 05:00:14,226][105692] Updated weights for policy 0, policy_version 1867795 (0.0006) [2023-12-27 05:00:14,274][105692] Updated weights for policy 0, policy_version 1867805 (0.0009) [2023-12-27 05:00:14,322][105692] Updated weights for policy 0, policy_version 1867815 (0.0009) [2023-12-27 05:00:14,645][105620] Updated weights for policy 1, policy_version 1872346 (0.0009) [2023-12-27 05:00:14,690][105620] Updated weights for policy 1, policy_version 1872356 (0.0006) [2023-12-27 05:00:14,745][105620] Updated weights for policy 1, policy_version 1872366 (0.0005) [2023-12-27 05:00:15,177][105692] Updated weights for policy 0, policy_version 1867825 (0.0009) [2023-12-27 05:00:15,247][105692] Updated weights for policy 0, policy_version 1867835 (0.0007) [2023-12-27 05:00:15,313][105692] Updated weights for policy 0, policy_version 1867845 (0.0009) [2023-12-27 05:00:15,382][105692] Updated weights for policy 0, policy_version 1867855 (0.0007) [2023-12-27 05:00:15,426][105620] Updated weights for policy 1, policy_version 1872376 (0.0006) [2023-12-27 05:00:15,491][105620] Updated weights for policy 1, policy_version 1872386 (0.0009) [2023-12-27 05:00:15,537][105620] Updated weights for policy 1, policy_version 1872396 (0.0010) [2023-12-27 05:00:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 957644800. Throughput: 0: 9721.7, 1: 9771.4. Samples: 957618208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:00:16,063][104569] Avg episode reward: [(0, '8624.551'), (1, '9253.650')] [2023-12-27 05:00:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001872400_479404032.pth... [2023-12-27 05:00:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001867856_478240768.pth... [2023-12-27 05:00:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001871248_479109120.pth [2023-12-27 05:00:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001866736_477954048.pth [2023-12-27 05:00:16,150][105692] Updated weights for policy 0, policy_version 1867865 (0.0009) [2023-12-27 05:00:16,184][105620] Updated weights for policy 1, policy_version 1872406 (0.0008) [2023-12-27 05:00:16,209][105692] Updated weights for policy 0, policy_version 1867875 (0.0005) [2023-12-27 05:00:16,250][105620] Updated weights for policy 1, policy_version 1872416 (0.0005) [2023-12-27 05:00:16,266][105692] Updated weights for policy 0, policy_version 1867885 (0.0005) [2023-12-27 05:00:16,315][105620] Updated weights for policy 1, policy_version 1872426 (0.0007) [2023-12-27 05:00:16,871][105692] Updated weights for policy 0, policy_version 1867895 (0.0005) [2023-12-27 05:00:16,940][105692] Updated weights for policy 0, policy_version 1867905 (0.0005) [2023-12-27 05:00:16,985][105620] Updated weights for policy 1, policy_version 1872436 (0.0008) [2023-12-27 05:00:16,999][105692] Updated weights for policy 0, policy_version 1867915 (0.0006) [2023-12-27 05:00:17,044][105620] Updated weights for policy 1, policy_version 1872446 (0.0006) [2023-12-27 05:00:17,101][105620] Updated weights for policy 1, policy_version 1872456 (0.0005) [2023-12-27 05:00:17,626][105692] Updated weights for policy 0, policy_version 1867925 (0.0008) [2023-12-27 05:00:17,684][105692] Updated weights for policy 0, policy_version 1867935 (0.0011) [2023-12-27 05:00:17,739][105620] Updated weights for policy 1, policy_version 1872466 (0.0006) [2023-12-27 05:00:17,744][105692] Updated weights for policy 0, policy_version 1867945 (0.0011) [2023-12-27 05:00:17,796][105620] Updated weights for policy 1, policy_version 1872476 (0.0010) [2023-12-27 05:00:17,848][105620] Updated weights for policy 1, policy_version 1872486 (0.0007) [2023-12-27 05:00:17,897][105620] Updated weights for policy 1, policy_version 1872496 (0.0010) [2023-12-27 05:00:18,469][105692] Updated weights for policy 0, policy_version 1867955 (0.0011) [2023-12-27 05:00:18,501][105620] Updated weights for policy 1, policy_version 1872506 (0.0007) [2023-12-27 05:00:18,534][105692] Updated weights for policy 0, policy_version 1867965 (0.0011) [2023-12-27 05:00:18,555][105620] Updated weights for policy 1, policy_version 1872516 (0.0007) [2023-12-27 05:00:18,600][105692] Updated weights for policy 0, policy_version 1867975 (0.0010) [2023-12-27 05:00:18,606][105620] Updated weights for policy 1, policy_version 1872526 (0.0007) [2023-12-27 05:00:19,327][105692] Updated weights for policy 0, policy_version 1867985 (0.0011) [2023-12-27 05:00:19,370][105620] Updated weights for policy 1, policy_version 1872536 (0.0010) [2023-12-27 05:00:19,392][105692] Updated weights for policy 0, policy_version 1867995 (0.0010) [2023-12-27 05:00:19,436][105620] Updated weights for policy 1, policy_version 1872546 (0.0011) [2023-12-27 05:00:19,452][105692] Updated weights for policy 0, policy_version 1868005 (0.0010) [2023-12-27 05:00:19,502][105620] Updated weights for policy 1, policy_version 1872556 (0.0011) [2023-12-27 05:00:19,515][105692] Updated weights for policy 0, policy_version 1868015 (0.0011) [2023-12-27 05:00:20,239][105620] Updated weights for policy 1, policy_version 1872566 (0.0010) [2023-12-27 05:00:20,244][105692] Updated weights for policy 0, policy_version 1868025 (0.0007) [2023-12-27 05:00:20,302][105620] Updated weights for policy 1, policy_version 1872576 (0.0010) [2023-12-27 05:00:20,303][105692] Updated weights for policy 0, policy_version 1868035 (0.0010) [2023-12-27 05:00:20,361][105620] Updated weights for policy 1, policy_version 1872586 (0.0011) [2023-12-27 05:00:20,363][105692] Updated weights for policy 0, policy_version 1868045 (0.0010) [2023-12-27 05:00:21,031][105620] Updated weights for policy 1, policy_version 1872596 (0.0009) [2023-12-27 05:00:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 957743104. Throughput: 0: 9670.6, 1: 9876.4. Samples: 957737524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:00:21,063][104569] Avg episode reward: [(0, '8727.043'), (1, '9253.619')] [2023-12-27 05:00:21,099][105620] Updated weights for policy 1, policy_version 1872606 (0.0008) [2023-12-27 05:00:21,115][105692] Updated weights for policy 0, policy_version 1868055 (0.0007) [2023-12-27 05:00:21,162][105620] Updated weights for policy 1, policy_version 1872616 (0.0010) [2023-12-27 05:00:21,179][105692] Updated weights for policy 0, policy_version 1868065 (0.0008) [2023-12-27 05:00:21,243][105692] Updated weights for policy 0, policy_version 1868075 (0.0008) [2023-12-27 05:00:22,012][105620] Updated weights for policy 1, policy_version 1872626 (0.0009) [2023-12-27 05:00:22,038][105692] Updated weights for policy 0, policy_version 1868085 (0.0009) [2023-12-27 05:00:22,062][105620] Updated weights for policy 1, policy_version 1872636 (0.0011) [2023-12-27 05:00:22,095][105692] Updated weights for policy 0, policy_version 1868095 (0.0009) [2023-12-27 05:00:22,126][105620] Updated weights for policy 1, policy_version 1872646 (0.0010) [2023-12-27 05:00:22,159][105692] Updated weights for policy 0, policy_version 1868105 (0.0011) [2023-12-27 05:00:22,188][105620] Updated weights for policy 1, policy_version 1872656 (0.0008) [2023-12-27 05:00:22,860][105620] Updated weights for policy 1, policy_version 1872666 (0.0006) [2023-12-27 05:00:22,912][105692] Updated weights for policy 0, policy_version 1868115 (0.0011) [2023-12-27 05:00:22,913][105620] Updated weights for policy 1, policy_version 1872676 (0.0005) [2023-12-27 05:00:22,966][105620] Updated weights for policy 1, policy_version 1872686 (0.0006) [2023-12-27 05:00:22,975][105692] Updated weights for policy 0, policy_version 1868125 (0.0009) [2023-12-27 05:00:23,029][105692] Updated weights for policy 0, policy_version 1868135 (0.0010) [2023-12-27 05:00:23,593][105620] Updated weights for policy 1, policy_version 1872696 (0.0009) [2023-12-27 05:00:23,628][105692] Updated weights for policy 0, policy_version 1868145 (0.0008) [2023-12-27 05:00:23,652][105620] Updated weights for policy 1, policy_version 1872706 (0.0010) [2023-12-27 05:00:23,692][105692] Updated weights for policy 0, policy_version 1868155 (0.0006) [2023-12-27 05:00:23,707][105620] Updated weights for policy 1, policy_version 1872716 (0.0010) [2023-12-27 05:00:23,743][105692] Updated weights for policy 0, policy_version 1868165 (0.0006) [2023-12-27 05:00:23,799][105692] Updated weights for policy 0, policy_version 1868175 (0.0005) [2023-12-27 05:00:24,448][105692] Updated weights for policy 0, policy_version 1868185 (0.0007) [2023-12-27 05:00:24,449][105620] Updated weights for policy 1, policy_version 1872726 (0.0010) [2023-12-27 05:00:24,502][105692] Updated weights for policy 0, policy_version 1868195 (0.0005) [2023-12-27 05:00:24,515][105620] Updated weights for policy 1, policy_version 1872736 (0.0010) [2023-12-27 05:00:24,561][105692] Updated weights for policy 0, policy_version 1868205 (0.0005) [2023-12-27 05:00:24,570][105620] Updated weights for policy 1, policy_version 1872746 (0.0011) [2023-12-27 05:00:25,097][105692] Updated weights for policy 0, policy_version 1868215 (0.0007) [2023-12-27 05:00:25,156][105692] Updated weights for policy 0, policy_version 1868225 (0.0009) [2023-12-27 05:00:25,214][105692] Updated weights for policy 0, policy_version 1868235 (0.0009) [2023-12-27 05:00:25,387][105620] Updated weights for policy 1, policy_version 1872756 (0.0010) [2023-12-27 05:00:25,437][105620] Updated weights for policy 1, policy_version 1872766 (0.0008) [2023-12-27 05:00:25,485][105620] Updated weights for policy 1, policy_version 1872776 (0.0009) [2023-12-27 05:00:25,953][105692] Updated weights for policy 0, policy_version 1868245 (0.0009) [2023-12-27 05:00:26,015][105692] Updated weights for policy 0, policy_version 1868255 (0.0008) [2023-12-27 05:00:26,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 957841408. Throughput: 0: 9709.8, 1: 9922.0. Samples: 957854968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:00:26,062][104569] Avg episode reward: [(0, '7905.165'), (1, '9345.519')] [2023-12-27 05:00:26,067][105692] Updated weights for policy 0, policy_version 1868265 (0.0009) [2023-12-27 05:00:26,261][105620] Updated weights for policy 1, policy_version 1872786 (0.0009) [2023-12-27 05:00:26,312][105620] Updated weights for policy 1, policy_version 1872796 (0.0009) [2023-12-27 05:00:26,370][105620] Updated weights for policy 1, policy_version 1872806 (0.0008) [2023-12-27 05:00:26,425][105620] Updated weights for policy 1, policy_version 1872816 (0.0008) [2023-12-27 05:00:26,797][105692] Updated weights for policy 0, policy_version 1868275 (0.0009) [2023-12-27 05:00:26,864][105692] Updated weights for policy 0, policy_version 1868285 (0.0010) [2023-12-27 05:00:26,915][105692] Updated weights for policy 0, policy_version 1868295 (0.0010) [2023-12-27 05:00:27,152][105620] Updated weights for policy 1, policy_version 1872826 (0.0005) [2023-12-27 05:00:27,214][105620] Updated weights for policy 1, policy_version 1872836 (0.0005) [2023-12-27 05:00:27,273][105620] Updated weights for policy 1, policy_version 1872846 (0.0010) [2023-12-27 05:00:27,733][105692] Updated weights for policy 0, policy_version 1868305 (0.0009) [2023-12-27 05:00:27,780][105692] Updated weights for policy 0, policy_version 1868315 (0.0010) [2023-12-27 05:00:27,825][105692] Updated weights for policy 0, policy_version 1868325 (0.0010) [2023-12-27 05:00:27,871][105692] Updated weights for policy 0, policy_version 1868335 (0.0009) [2023-12-27 05:00:27,888][105620] Updated weights for policy 1, policy_version 1872856 (0.0009) [2023-12-27 05:00:27,946][105620] Updated weights for policy 1, policy_version 1872866 (0.0008) [2023-12-27 05:00:28,011][105620] Updated weights for policy 1, policy_version 1872876 (0.0008) [2023-12-27 05:00:28,569][105692] Updated weights for policy 0, policy_version 1868345 (0.0009) [2023-12-27 05:00:28,631][105692] Updated weights for policy 0, policy_version 1868355 (0.0010) [2023-12-27 05:00:28,686][105692] Updated weights for policy 0, policy_version 1868365 (0.0010) [2023-12-27 05:00:28,823][105620] Updated weights for policy 1, policy_version 1872886 (0.0008) [2023-12-27 05:00:28,879][105620] Updated weights for policy 1, policy_version 1872896 (0.0008) [2023-12-27 05:00:28,939][105620] Updated weights for policy 1, policy_version 1872906 (0.0009) [2023-12-27 05:00:29,277][105692] Updated weights for policy 0, policy_version 1868375 (0.0008) [2023-12-27 05:00:29,363][105692] Updated weights for policy 0, policy_version 1868385 (0.0009) [2023-12-27 05:00:29,425][105692] Updated weights for policy 0, policy_version 1868395 (0.0010) [2023-12-27 05:00:29,736][105620] Updated weights for policy 1, policy_version 1872916 (0.0008) [2023-12-27 05:00:29,789][105620] Updated weights for policy 1, policy_version 1872926 (0.0005) [2023-12-27 05:00:29,865][105620] Updated weights for policy 1, policy_version 1872936 (0.0007) [2023-12-27 05:00:30,053][105692] Updated weights for policy 0, policy_version 1868405 (0.0009) [2023-12-27 05:00:30,110][105692] Updated weights for policy 0, policy_version 1868415 (0.0008) [2023-12-27 05:00:30,170][105692] Updated weights for policy 0, policy_version 1868425 (0.0006) [2023-12-27 05:00:30,613][105620] Updated weights for policy 1, policy_version 1872946 (0.0007) [2023-12-27 05:00:30,659][105620] Updated weights for policy 1, policy_version 1872956 (0.0008) [2023-12-27 05:00:30,710][105620] Updated weights for policy 1, policy_version 1872966 (0.0007) [2023-12-27 05:00:30,757][105620] Updated weights for policy 1, policy_version 1872976 (0.0008) [2023-12-27 05:00:30,821][105692] Updated weights for policy 0, policy_version 1868435 (0.0007) [2023-12-27 05:00:30,871][105692] Updated weights for policy 0, policy_version 1868445 (0.0010) [2023-12-27 05:00:30,915][105692] Updated weights for policy 0, policy_version 1868455 (0.0010) [2023-12-27 05:00:31,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 957947904. Throughput: 0: 9730.7, 1: 9876.7. Samples: 957913092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:00:31,063][104569] Avg episode reward: [(0, '7990.146'), (1, '9253.073')] [2023-12-27 05:00:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001868464_478396416.pth... [2023-12-27 05:00:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001872976_479551488.pth... [2023-12-27 05:00:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001871824_479256576.pth [2023-12-27 05:00:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001867312_478101504.pth [2023-12-27 05:00:31,573][105620] Updated weights for policy 1, policy_version 1872986 (0.0010) [2023-12-27 05:00:31,593][105692] Updated weights for policy 0, policy_version 1868465 (0.0010) [2023-12-27 05:00:31,636][105620] Updated weights for policy 1, policy_version 1872996 (0.0008) [2023-12-27 05:00:31,669][105692] Updated weights for policy 0, policy_version 1868475 (0.0011) [2023-12-27 05:00:31,699][105620] Updated weights for policy 1, policy_version 1873006 (0.0006) [2023-12-27 05:00:31,722][105692] Updated weights for policy 0, policy_version 1868485 (0.0010) [2023-12-27 05:00:31,787][105692] Updated weights for policy 0, policy_version 1868495 (0.0010) [2023-12-27 05:00:32,417][105692] Updated weights for policy 0, policy_version 1868505 (0.0006) [2023-12-27 05:00:32,481][105692] Updated weights for policy 0, policy_version 1868515 (0.0005) [2023-12-27 05:00:32,539][105620] Updated weights for policy 1, policy_version 1873016 (0.0007) [2023-12-27 05:00:32,540][105692] Updated weights for policy 0, policy_version 1868525 (0.0011) [2023-12-27 05:00:32,596][105620] Updated weights for policy 1, policy_version 1873026 (0.0007) [2023-12-27 05:00:32,662][105620] Updated weights for policy 1, policy_version 1873036 (0.0007) [2023-12-27 05:00:33,171][105692] Updated weights for policy 0, policy_version 1868535 (0.0007) [2023-12-27 05:00:33,226][105692] Updated weights for policy 0, policy_version 1868545 (0.0007) [2023-12-27 05:00:33,287][105692] Updated weights for policy 0, policy_version 1868555 (0.0010) [2023-12-27 05:00:33,461][105620] Updated weights for policy 1, policy_version 1873046 (0.0009) [2023-12-27 05:00:33,527][105620] Updated weights for policy 1, policy_version 1873056 (0.0010) [2023-12-27 05:00:33,578][105620] Updated weights for policy 1, policy_version 1873067 (0.0009) [2023-12-27 05:00:33,893][105692] Updated weights for policy 0, policy_version 1868565 (0.0009) [2023-12-27 05:00:33,946][105692] Updated weights for policy 0, policy_version 1868575 (0.0006) [2023-12-27 05:00:33,997][105692] Updated weights for policy 0, policy_version 1868585 (0.0005) [2023-12-27 05:00:34,405][105620] Updated weights for policy 1, policy_version 1873077 (0.0007) [2023-12-27 05:00:34,475][105620] Updated weights for policy 1, policy_version 1873087 (0.0010) [2023-12-27 05:00:34,535][105620] Updated weights for policy 1, policy_version 1873097 (0.0005) [2023-12-27 05:00:34,540][105692] Updated weights for policy 0, policy_version 1868595 (0.0007) [2023-12-27 05:00:34,597][105692] Updated weights for policy 0, policy_version 1868605 (0.0010) [2023-12-27 05:00:34,652][105692] Updated weights for policy 0, policy_version 1868615 (0.0010) [2023-12-27 05:00:35,231][105620] Updated weights for policy 1, policy_version 1873107 (0.0010) [2023-12-27 05:00:35,290][105620] Updated weights for policy 1, policy_version 1873117 (0.0008) [2023-12-27 05:00:35,338][105620] Updated weights for policy 1, policy_version 1873127 (0.0008) [2023-12-27 05:00:35,355][105692] Updated weights for policy 0, policy_version 1868625 (0.0009) [2023-12-27 05:00:35,409][105692] Updated weights for policy 0, policy_version 1868635 (0.0010) [2023-12-27 05:00:35,470][105692] Updated weights for policy 0, policy_version 1868645 (0.0010) [2023-12-27 05:00:35,523][105692] Updated weights for policy 0, policy_version 1868655 (0.0010) [2023-12-27 05:00:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19577.5). Total num frames: 958038016. Throughput: 0: 9893.7, 1: 9751.2. Samples: 958031348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:00:36,063][104569] Avg episode reward: [(0, '8534.734'), (1, '9253.030')] [2023-12-27 05:00:36,114][105620] Updated weights for policy 1, policy_version 1873137 (0.0009) [2023-12-27 05:00:36,177][105620] Updated weights for policy 1, policy_version 1873147 (0.0008) [2023-12-27 05:00:36,232][105620] Updated weights for policy 1, policy_version 1873157 (0.0008) [2023-12-27 05:00:36,263][105692] Updated weights for policy 0, policy_version 1868665 (0.0010) [2023-12-27 05:00:36,285][105620] Updated weights for policy 1, policy_version 1873167 (0.0007) [2023-12-27 05:00:36,323][105692] Updated weights for policy 0, policy_version 1868675 (0.0010) [2023-12-27 05:00:36,385][105692] Updated weights for policy 0, policy_version 1868685 (0.0010) [2023-12-27 05:00:37,074][105620] Updated weights for policy 1, policy_version 1873177 (0.0008) [2023-12-27 05:00:37,128][105692] Updated weights for policy 0, policy_version 1868695 (0.0010) [2023-12-27 05:00:37,129][105620] Updated weights for policy 1, policy_version 1873187 (0.0009) [2023-12-27 05:00:37,186][105620] Updated weights for policy 1, policy_version 1873197 (0.0006) [2023-12-27 05:00:37,187][105692] Updated weights for policy 0, policy_version 1868705 (0.0010) [2023-12-27 05:00:37,248][105692] Updated weights for policy 0, policy_version 1868715 (0.0010) [2023-12-27 05:00:37,965][105620] Updated weights for policy 1, policy_version 1873207 (0.0007) [2023-12-27 05:00:37,997][105692] Updated weights for policy 0, policy_version 1868725 (0.0010) [2023-12-27 05:00:38,019][105620] Updated weights for policy 1, policy_version 1873217 (0.0008) [2023-12-27 05:00:38,058][105692] Updated weights for policy 0, policy_version 1868735 (0.0010) [2023-12-27 05:00:38,065][105620] Updated weights for policy 1, policy_version 1873227 (0.0007) [2023-12-27 05:00:38,128][105692] Updated weights for policy 0, policy_version 1868745 (0.0010) [2023-12-27 05:00:38,852][105692] Updated weights for policy 0, policy_version 1868755 (0.0010) [2023-12-27 05:00:38,863][105620] Updated weights for policy 1, policy_version 1873237 (0.0008) [2023-12-27 05:00:38,905][105692] Updated weights for policy 0, policy_version 1868765 (0.0007) [2023-12-27 05:00:38,922][105620] Updated weights for policy 1, policy_version 1873247 (0.0008) [2023-12-27 05:00:38,960][105692] Updated weights for policy 0, policy_version 1868775 (0.0009) [2023-12-27 05:00:38,973][105620] Updated weights for policy 1, policy_version 1873257 (0.0008) [2023-12-27 05:00:39,592][105692] Updated weights for policy 0, policy_version 1868785 (0.0010) [2023-12-27 05:00:39,647][105692] Updated weights for policy 0, policy_version 1868795 (0.0006) [2023-12-27 05:00:39,688][105620] Updated weights for policy 1, policy_version 1873267 (0.0007) [2023-12-27 05:00:39,703][105692] Updated weights for policy 0, policy_version 1868805 (0.0010) [2023-12-27 05:00:39,748][105620] Updated weights for policy 1, policy_version 1873277 (0.0008) [2023-12-27 05:00:39,760][105692] Updated weights for policy 0, policy_version 1868815 (0.0010) [2023-12-27 05:00:39,807][105620] Updated weights for policy 1, policy_version 1873287 (0.0008) [2023-12-27 05:00:40,492][105692] Updated weights for policy 0, policy_version 1868825 (0.0010) [2023-12-27 05:00:40,552][105692] Updated weights for policy 0, policy_version 1868835 (0.0010) [2023-12-27 05:00:40,569][105620] Updated weights for policy 1, policy_version 1873297 (0.0009) [2023-12-27 05:00:40,607][105692] Updated weights for policy 0, policy_version 1868845 (0.0010) [2023-12-27 05:00:40,632][105620] Updated weights for policy 1, policy_version 1873307 (0.0011) [2023-12-27 05:00:40,695][105620] Updated weights for policy 1, policy_version 1873317 (0.0009) [2023-12-27 05:00:40,757][105620] Updated weights for policy 1, policy_version 1873327 (0.0010) [2023-12-27 05:00:41,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 958136320. Throughput: 0: 9939.5, 1: 9641.2. Samples: 958143936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:00:41,062][104569] Avg episode reward: [(0, '8078.123'), (1, '9253.321')] [2023-12-27 05:00:41,358][105692] Updated weights for policy 0, policy_version 1868855 (0.0009) [2023-12-27 05:00:41,420][105692] Updated weights for policy 0, policy_version 1868865 (0.0006) [2023-12-27 05:00:41,473][105692] Updated weights for policy 0, policy_version 1868875 (0.0009) [2023-12-27 05:00:41,509][105620] Updated weights for policy 1, policy_version 1873337 (0.0006) [2023-12-27 05:00:41,579][105620] Updated weights for policy 1, policy_version 1873347 (0.0007) [2023-12-27 05:00:41,648][105620] Updated weights for policy 1, policy_version 1873357 (0.0009) [2023-12-27 05:00:42,244][105692] Updated weights for policy 0, policy_version 1868885 (0.0009) [2023-12-27 05:00:42,305][105692] Updated weights for policy 0, policy_version 1868895 (0.0008) [2023-12-27 05:00:42,317][105620] Updated weights for policy 1, policy_version 1873367 (0.0011) [2023-12-27 05:00:42,375][105692] Updated weights for policy 0, policy_version 1868905 (0.0009) [2023-12-27 05:00:42,384][105620] Updated weights for policy 1, policy_version 1873377 (0.0011) [2023-12-27 05:00:42,436][105620] Updated weights for policy 1, policy_version 1873387 (0.0010) [2023-12-27 05:00:43,012][105692] Updated weights for policy 0, policy_version 1868915 (0.0008) [2023-12-27 05:00:43,061][105692] Updated weights for policy 0, policy_version 1868925 (0.0005) [2023-12-27 05:00:43,116][105692] Updated weights for policy 0, policy_version 1868935 (0.0006) [2023-12-27 05:00:43,161][105620] Updated weights for policy 1, policy_version 1873397 (0.0007) [2023-12-27 05:00:43,221][105620] Updated weights for policy 1, policy_version 1873407 (0.0006) [2023-12-27 05:00:43,276][105620] Updated weights for policy 1, policy_version 1873417 (0.0009) [2023-12-27 05:00:43,680][105692] Updated weights for policy 0, policy_version 1868945 (0.0006) [2023-12-27 05:00:43,741][105692] Updated weights for policy 0, policy_version 1868955 (0.0011) [2023-12-27 05:00:43,807][105692] Updated weights for policy 0, policy_version 1868965 (0.0010) [2023-12-27 05:00:43,852][105692] Updated weights for policy 0, policy_version 1868975 (0.0010) [2023-12-27 05:00:43,977][105620] Updated weights for policy 1, policy_version 1873427 (0.0008) [2023-12-27 05:00:44,025][105620] Updated weights for policy 1, policy_version 1873437 (0.0009) [2023-12-27 05:00:44,081][105620] Updated weights for policy 1, policy_version 1873447 (0.0012) [2023-12-27 05:00:44,452][105692] Updated weights for policy 0, policy_version 1868985 (0.0006) [2023-12-27 05:00:44,506][105692] Updated weights for policy 0, policy_version 1868995 (0.0005) [2023-12-27 05:00:44,562][105692] Updated weights for policy 0, policy_version 1869005 (0.0008) [2023-12-27 05:00:44,993][105620] Updated weights for policy 1, policy_version 1873457 (0.0009) [2023-12-27 05:00:45,061][105620] Updated weights for policy 1, policy_version 1873467 (0.0009) [2023-12-27 05:00:45,124][105620] Updated weights for policy 1, policy_version 1873477 (0.0008) [2023-12-27 05:00:45,164][105692] Updated weights for policy 0, policy_version 1869015 (0.0009) [2023-12-27 05:00:45,183][105620] Updated weights for policy 1, policy_version 1873487 (0.0006) [2023-12-27 05:00:45,229][105692] Updated weights for policy 0, policy_version 1869025 (0.0010) [2023-12-27 05:00:45,298][105692] Updated weights for policy 0, policy_version 1869035 (0.0010) [2023-12-27 05:00:45,937][105620] Updated weights for policy 1, policy_version 1873497 (0.0009) [2023-12-27 05:00:45,958][105692] Updated weights for policy 0, policy_version 1869045 (0.0008) [2023-12-27 05:00:45,985][105620] Updated weights for policy 1, policy_version 1873507 (0.0007) [2023-12-27 05:00:46,024][105692] Updated weights for policy 0, policy_version 1869055 (0.0010) [2023-12-27 05:00:46,035][105620] Updated weights for policy 1, policy_version 1873517 (0.0006) [2023-12-27 05:00:46,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 958234624. Throughput: 0: 9949.1, 1: 9620.1. Samples: 958203300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:00:46,063][104569] Avg episode reward: [(0, '7987.807'), (1, '9161.487')] [2023-12-27 05:00:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001873520_479690752.pth... [2023-12-27 05:00:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001872400_479404032.pth [2023-12-27 05:00:46,079][105692] Updated weights for policy 0, policy_version 1869065 (0.0010) [2023-12-27 05:00:46,113][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001869072_478552064.pth... [2023-12-27 05:00:46,117][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001867856_478240768.pth [2023-12-27 05:00:46,732][105692] Updated weights for policy 0, policy_version 1869075 (0.0010) [2023-12-27 05:00:46,794][105692] Updated weights for policy 0, policy_version 1869085 (0.0011) [2023-12-27 05:00:46,825][105620] Updated weights for policy 1, policy_version 1873527 (0.0007) [2023-12-27 05:00:46,856][105692] Updated weights for policy 0, policy_version 1869095 (0.0010) [2023-12-27 05:00:46,874][105620] Updated weights for policy 1, policy_version 1873537 (0.0007) [2023-12-27 05:00:46,932][105620] Updated weights for policy 1, policy_version 1873547 (0.0008) [2023-12-27 05:00:47,559][105692] Updated weights for policy 0, policy_version 1869105 (0.0010) [2023-12-27 05:00:47,617][105692] Updated weights for policy 0, policy_version 1869115 (0.0009) [2023-12-27 05:00:47,672][105692] Updated weights for policy 0, policy_version 1869125 (0.0007) [2023-12-27 05:00:47,698][105620] Updated weights for policy 1, policy_version 1873557 (0.0008) [2023-12-27 05:00:47,735][105692] Updated weights for policy 0, policy_version 1869135 (0.0006) [2023-12-27 05:00:47,749][105620] Updated weights for policy 1, policy_version 1873567 (0.0008) [2023-12-27 05:00:47,814][105620] Updated weights for policy 1, policy_version 1873577 (0.0009) [2023-12-27 05:00:48,435][105692] Updated weights for policy 0, policy_version 1869145 (0.0006) [2023-12-27 05:00:48,502][105692] Updated weights for policy 0, policy_version 1869155 (0.0007) [2023-12-27 05:00:48,551][105620] Updated weights for policy 1, policy_version 1873587 (0.0007) [2023-12-27 05:00:48,569][105692] Updated weights for policy 0, policy_version 1869165 (0.0007) [2023-12-27 05:00:48,615][105620] Updated weights for policy 1, policy_version 1873597 (0.0009) [2023-12-27 05:00:48,683][105620] Updated weights for policy 1, policy_version 1873607 (0.0007) [2023-12-27 05:00:49,182][105692] Updated weights for policy 0, policy_version 1869175 (0.0009) [2023-12-27 05:00:49,245][105692] Updated weights for policy 0, policy_version 1869185 (0.0008) [2023-12-27 05:00:49,301][105692] Updated weights for policy 0, policy_version 1869195 (0.0008) [2023-12-27 05:00:49,470][105620] Updated weights for policy 1, policy_version 1873617 (0.0006) [2023-12-27 05:00:49,518][105620] Updated weights for policy 1, policy_version 1873627 (0.0009) [2023-12-27 05:00:49,567][105620] Updated weights for policy 1, policy_version 1873637 (0.0009) [2023-12-27 05:00:49,614][105620] Updated weights for policy 1, policy_version 1873647 (0.0008) [2023-12-27 05:00:49,988][105692] Updated weights for policy 0, policy_version 1869205 (0.0009) [2023-12-27 05:00:50,049][105692] Updated weights for policy 0, policy_version 1869215 (0.0009) [2023-12-27 05:00:50,106][105692] Updated weights for policy 0, policy_version 1869225 (0.0009) [2023-12-27 05:00:50,445][105620] Updated weights for policy 1, policy_version 1873657 (0.0010) [2023-12-27 05:00:50,495][105620] Updated weights for policy 1, policy_version 1873667 (0.0009) [2023-12-27 05:00:50,543][105620] Updated weights for policy 1, policy_version 1873677 (0.0009) [2023-12-27 05:00:50,847][105692] Updated weights for policy 0, policy_version 1869235 (0.0010) [2023-12-27 05:00:50,901][105692] Updated weights for policy 0, policy_version 1869245 (0.0010) [2023-12-27 05:00:50,950][105692] Updated weights for policy 0, policy_version 1869255 (0.0010) [2023-12-27 05:00:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 958332928. Throughput: 0: 10065.6, 1: 9522.8. Samples: 958319456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:00:51,062][104569] Avg episode reward: [(0, '8350.966'), (1, '9253.564')] [2023-12-27 05:00:51,373][105620] Updated weights for policy 1, policy_version 1873687 (0.0008) [2023-12-27 05:00:51,439][105620] Updated weights for policy 1, policy_version 1873697 (0.0008) [2023-12-27 05:00:51,500][105620] Updated weights for policy 1, policy_version 1873707 (0.0008) [2023-12-27 05:00:51,724][105692] Updated weights for policy 0, policy_version 1869265 (0.0010) [2023-12-27 05:00:51,792][105692] Updated weights for policy 0, policy_version 1869275 (0.0009) [2023-12-27 05:00:51,846][105692] Updated weights for policy 0, policy_version 1869285 (0.0009) [2023-12-27 05:00:51,898][105692] Updated weights for policy 0, policy_version 1869295 (0.0009) [2023-12-27 05:00:52,158][105620] Updated weights for policy 1, policy_version 1873717 (0.0005) [2023-12-27 05:00:52,227][105620] Updated weights for policy 1, policy_version 1873727 (0.0008) [2023-12-27 05:00:52,276][105620] Updated weights for policy 1, policy_version 1873737 (0.0009) [2023-12-27 05:00:52,563][105692] Updated weights for policy 0, policy_version 1869305 (0.0009) [2023-12-27 05:00:52,619][105692] Updated weights for policy 0, policy_version 1869315 (0.0008) [2023-12-27 05:00:52,671][105692] Updated weights for policy 0, policy_version 1869325 (0.0008) [2023-12-27 05:00:53,024][105620] Updated weights for policy 1, policy_version 1873747 (0.0009) [2023-12-27 05:00:53,072][105620] Updated weights for policy 1, policy_version 1873757 (0.0010) [2023-12-27 05:00:53,128][105620] Updated weights for policy 1, policy_version 1873767 (0.0010) [2023-12-27 05:00:53,442][105692] Updated weights for policy 0, policy_version 1869335 (0.0009) [2023-12-27 05:00:53,501][105692] Updated weights for policy 0, policy_version 1869346 (0.0011) [2023-12-27 05:00:53,556][105692] Updated weights for policy 0, policy_version 1869356 (0.0010) [2023-12-27 05:00:53,784][105620] Updated weights for policy 1, policy_version 1873777 (0.0011) [2023-12-27 05:00:53,846][105620] Updated weights for policy 1, policy_version 1873787 (0.0009) [2023-12-27 05:00:53,911][105620] Updated weights for policy 1, policy_version 1873797 (0.0007) [2023-12-27 05:00:53,977][105620] Updated weights for policy 1, policy_version 1873807 (0.0006) [2023-12-27 05:00:54,294][105692] Updated weights for policy 0, policy_version 1869366 (0.0007) [2023-12-27 05:00:54,340][105692] Updated weights for policy 0, policy_version 1869376 (0.0005) [2023-12-27 05:00:54,398][105692] Updated weights for policy 0, policy_version 1869386 (0.0005) [2023-12-27 05:00:54,596][105620] Updated weights for policy 1, policy_version 1873817 (0.0009) [2023-12-27 05:00:54,651][105620] Updated weights for policy 1, policy_version 1873827 (0.0010) [2023-12-27 05:00:54,699][105620] Updated weights for policy 1, policy_version 1873837 (0.0010) [2023-12-27 05:00:55,007][105692] Updated weights for policy 0, policy_version 1869396 (0.0007) [2023-12-27 05:00:55,060][105692] Updated weights for policy 0, policy_version 1869406 (0.0010) [2023-12-27 05:00:55,113][105692] Updated weights for policy 0, policy_version 1869416 (0.0009) [2023-12-27 05:00:55,371][105620] Updated weights for policy 1, policy_version 1873847 (0.0007) [2023-12-27 05:00:55,417][105620] Updated weights for policy 1, policy_version 1873857 (0.0005) [2023-12-27 05:00:55,463][105620] Updated weights for policy 1, policy_version 1873867 (0.0005) [2023-12-27 05:00:55,951][105692] Updated weights for policy 0, policy_version 1869426 (0.0009) [2023-12-27 05:00:56,010][105692] Updated weights for policy 0, policy_version 1869436 (0.0006) [2023-12-27 05:00:56,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 958423040. Throughput: 0: 9957.3, 1: 9529.3. Samples: 958438000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:00:56,062][104569] Avg episode reward: [(0, '8350.317'), (1, '9160.776')] [2023-12-27 05:00:56,076][105692] Updated weights for policy 0, policy_version 1869446 (0.0007) [2023-12-27 05:00:56,120][105620] Updated weights for policy 1, policy_version 1873877 (0.0007) [2023-12-27 05:00:56,137][105692] Updated weights for policy 0, policy_version 1869456 (0.0006) [2023-12-27 05:00:56,182][105620] Updated weights for policy 1, policy_version 1873887 (0.0009) [2023-12-27 05:00:56,240][105620] Updated weights for policy 1, policy_version 1873897 (0.0008) [2023-12-27 05:00:56,786][105692] Updated weights for policy 0, policy_version 1869466 (0.0009) [2023-12-27 05:00:56,833][105692] Updated weights for policy 0, policy_version 1869476 (0.0009) [2023-12-27 05:00:56,880][105692] Updated weights for policy 0, policy_version 1869486 (0.0009) [2023-12-27 05:00:56,991][105620] Updated weights for policy 1, policy_version 1873907 (0.0009) [2023-12-27 05:00:57,037][105620] Updated weights for policy 1, policy_version 1873917 (0.0008) [2023-12-27 05:00:57,091][105620] Updated weights for policy 1, policy_version 1873927 (0.0009) [2023-12-27 05:00:57,671][105692] Updated weights for policy 0, policy_version 1869496 (0.0009) [2023-12-27 05:00:57,723][105692] Updated weights for policy 0, policy_version 1869506 (0.0008) [2023-12-27 05:00:57,773][105692] Updated weights for policy 0, policy_version 1869516 (0.0009) [2023-12-27 05:00:57,798][105620] Updated weights for policy 1, policy_version 1873937 (0.0009) [2023-12-27 05:00:57,861][105620] Updated weights for policy 1, policy_version 1873947 (0.0009) [2023-12-27 05:00:57,923][105620] Updated weights for policy 1, policy_version 1873957 (0.0009) [2023-12-27 05:00:57,977][105620] Updated weights for policy 1, policy_version 1873967 (0.0009) [2023-12-27 05:00:58,539][105692] Updated weights for policy 0, policy_version 1869526 (0.0008) [2023-12-27 05:00:58,604][105692] Updated weights for policy 0, policy_version 1869536 (0.0007) [2023-12-27 05:00:58,676][105692] Updated weights for policy 0, policy_version 1869546 (0.0009) [2023-12-27 05:00:58,815][105620] Updated weights for policy 1, policy_version 1873977 (0.0008) [2023-12-27 05:00:58,881][105620] Updated weights for policy 1, policy_version 1873987 (0.0009) [2023-12-27 05:00:58,976][105620] Updated weights for policy 1, policy_version 1874000 (0.0009) [2023-12-27 05:00:59,494][105692] Updated weights for policy 0, policy_version 1869556 (0.0009) [2023-12-27 05:00:59,550][105692] Updated weights for policy 0, policy_version 1869566 (0.0009) [2023-12-27 05:00:59,608][105692] Updated weights for policy 0, policy_version 1869576 (0.0008) [2023-12-27 05:00:59,741][105620] Updated weights for policy 1, policy_version 1874010 (0.0009) [2023-12-27 05:00:59,793][105620] Updated weights for policy 1, policy_version 1874020 (0.0010) [2023-12-27 05:00:59,856][105620] Updated weights for policy 1, policy_version 1874030 (0.0010) [2023-12-27 05:01:00,422][105692] Updated weights for policy 0, policy_version 1869586 (0.0008) [2023-12-27 05:01:00,475][105692] Updated weights for policy 0, policy_version 1869596 (0.0010) [2023-12-27 05:01:00,515][105620] Updated weights for policy 1, policy_version 1874040 (0.0006) [2023-12-27 05:01:00,523][105692] Updated weights for policy 0, policy_version 1869606 (0.0008) [2023-12-27 05:01:00,562][105620] Updated weights for policy 1, policy_version 1874050 (0.0009) [2023-12-27 05:01:00,580][105692] Updated weights for policy 0, policy_version 1869616 (0.0005) [2023-12-27 05:01:00,611][105620] Updated weights for policy 1, policy_version 1874060 (0.0010) [2023-12-27 05:01:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 958521344. Throughput: 0: 9972.2, 1: 9480.6. Samples: 958493584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:01:01,063][104569] Avg episode reward: [(0, '8625.215'), (1, '9160.786')] [2023-12-27 05:01:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001869616_478691328.pth... [2023-12-27 05:01:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001874064_479830016.pth... [2023-12-27 05:01:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001872976_479551488.pth [2023-12-27 05:01:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001868464_478396416.pth [2023-12-27 05:01:01,383][105620] Updated weights for policy 1, policy_version 1874070 (0.0009) [2023-12-27 05:01:01,408][105692] Updated weights for policy 0, policy_version 1869626 (0.0008) [2023-12-27 05:01:01,438][105620] Updated weights for policy 1, policy_version 1874080 (0.0009) [2023-12-27 05:01:01,465][105692] Updated weights for policy 0, policy_version 1869636 (0.0007) [2023-12-27 05:01:01,495][105620] Updated weights for policy 1, policy_version 1874090 (0.0007) [2023-12-27 05:01:01,522][105692] Updated weights for policy 0, policy_version 1869646 (0.0006) [2023-12-27 05:01:02,244][105620] Updated weights for policy 1, policy_version 1874100 (0.0008) [2023-12-27 05:01:02,309][105620] Updated weights for policy 1, policy_version 1874110 (0.0008) [2023-12-27 05:01:02,319][105692] Updated weights for policy 0, policy_version 1869656 (0.0008) [2023-12-27 05:01:02,378][105620] Updated weights for policy 1, policy_version 1874120 (0.0008) [2023-12-27 05:01:02,389][105692] Updated weights for policy 0, policy_version 1869666 (0.0006) [2023-12-27 05:01:02,453][105692] Updated weights for policy 0, policy_version 1869676 (0.0007) [2023-12-27 05:01:03,163][105692] Updated weights for policy 0, policy_version 1869686 (0.0009) [2023-12-27 05:01:03,210][105620] Updated weights for policy 1, policy_version 1874130 (0.0009) [2023-12-27 05:01:03,215][105692] Updated weights for policy 0, policy_version 1869696 (0.0006) [2023-12-27 05:01:03,265][105692] Updated weights for policy 0, policy_version 1869706 (0.0005) [2023-12-27 05:01:03,266][105620] Updated weights for policy 1, policy_version 1874140 (0.0009) [2023-12-27 05:01:03,314][105620] Updated weights for policy 1, policy_version 1874150 (0.0008) [2023-12-27 05:01:03,363][105620] Updated weights for policy 1, policy_version 1874160 (0.0010) [2023-12-27 05:01:03,967][105692] Updated weights for policy 0, policy_version 1869716 (0.0008) [2023-12-27 05:01:04,029][105692] Updated weights for policy 0, policy_version 1869726 (0.0007) [2023-12-27 05:01:04,067][105620] Updated weights for policy 1, policy_version 1874170 (0.0010) [2023-12-27 05:01:04,085][105692] Updated weights for policy 0, policy_version 1869736 (0.0006) [2023-12-27 05:01:04,123][105620] Updated weights for policy 1, policy_version 1874180 (0.0010) [2023-12-27 05:01:04,184][105620] Updated weights for policy 1, policy_version 1874190 (0.0011) [2023-12-27 05:01:04,886][105692] Updated weights for policy 0, policy_version 1869746 (0.0007) [2023-12-27 05:01:04,948][105692] Updated weights for policy 0, policy_version 1869756 (0.0008) [2023-12-27 05:01:04,994][105620] Updated weights for policy 1, policy_version 1874200 (0.0011) [2023-12-27 05:01:05,005][105692] Updated weights for policy 0, policy_version 1869766 (0.0007) [2023-12-27 05:01:05,050][105620] Updated weights for policy 1, policy_version 1874210 (0.0011) [2023-12-27 05:01:05,064][105692] Updated weights for policy 0, policy_version 1869776 (0.0006) [2023-12-27 05:01:05,109][105620] Updated weights for policy 1, policy_version 1874220 (0.0010) [2023-12-27 05:01:05,705][105692] Updated weights for policy 0, policy_version 1869786 (0.0008) [2023-12-27 05:01:05,752][105692] Updated weights for policy 0, policy_version 1869796 (0.0008) [2023-12-27 05:01:05,820][105692] Updated weights for policy 0, policy_version 1869806 (0.0007) [2023-12-27 05:01:05,837][105620] Updated weights for policy 1, policy_version 1874230 (0.0008) [2023-12-27 05:01:05,894][105620] Updated weights for policy 1, policy_version 1874240 (0.0005) [2023-12-27 05:01:05,953][105620] Updated weights for policy 1, policy_version 1874250 (0.0005) [2023-12-27 05:01:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 958619648. Throughput: 0: 9892.6, 1: 9342.4. Samples: 958603100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:01:06,062][104569] Avg episode reward: [(0, '8080.244'), (1, '9345.418')] [2023-12-27 05:01:06,586][105620] Updated weights for policy 1, policy_version 1874260 (0.0008) [2023-12-27 05:01:06,592][105692] Updated weights for policy 0, policy_version 1869816 (0.0008) [2023-12-27 05:01:06,638][105620] Updated weights for policy 1, policy_version 1874270 (0.0006) [2023-12-27 05:01:06,652][105692] Updated weights for policy 0, policy_version 1869826 (0.0011) [2023-12-27 05:01:06,689][105620] Updated weights for policy 1, policy_version 1874280 (0.0006) [2023-12-27 05:01:06,715][105692] Updated weights for policy 0, policy_version 1869836 (0.0011) [2023-12-27 05:01:07,354][105692] Updated weights for policy 0, policy_version 1869846 (0.0011) [2023-12-27 05:01:07,416][105692] Updated weights for policy 0, policy_version 1869856 (0.0009) [2023-12-27 05:01:07,446][105620] Updated weights for policy 1, policy_version 1874290 (0.0006) [2023-12-27 05:01:07,481][105692] Updated weights for policy 0, policy_version 1869866 (0.0009) [2023-12-27 05:01:07,510][105620] Updated weights for policy 1, policy_version 1874300 (0.0008) [2023-12-27 05:01:07,570][105620] Updated weights for policy 1, policy_version 1874310 (0.0008) [2023-12-27 05:01:07,625][105620] Updated weights for policy 1, policy_version 1874320 (0.0006) [2023-12-27 05:01:08,122][105692] Updated weights for policy 0, policy_version 1869876 (0.0007) [2023-12-27 05:01:08,182][105692] Updated weights for policy 0, policy_version 1869886 (0.0006) [2023-12-27 05:01:08,245][105692] Updated weights for policy 0, policy_version 1869896 (0.0005) [2023-12-27 05:01:08,437][105620] Updated weights for policy 1, policy_version 1874330 (0.0009) [2023-12-27 05:01:08,506][105620] Updated weights for policy 1, policy_version 1874340 (0.0009) [2023-12-27 05:01:08,569][105620] Updated weights for policy 1, policy_version 1874350 (0.0009) [2023-12-27 05:01:08,855][105692] Updated weights for policy 0, policy_version 1869906 (0.0005) [2023-12-27 05:01:08,917][105692] Updated weights for policy 0, policy_version 1869916 (0.0008) [2023-12-27 05:01:08,983][105692] Updated weights for policy 0, policy_version 1869926 (0.0010) [2023-12-27 05:01:09,043][105692] Updated weights for policy 0, policy_version 1869936 (0.0009) [2023-12-27 05:01:09,388][105620] Updated weights for policy 1, policy_version 1874360 (0.0008) [2023-12-27 05:01:09,456][105620] Updated weights for policy 1, policy_version 1874370 (0.0009) [2023-12-27 05:01:09,515][105620] Updated weights for policy 1, policy_version 1874380 (0.0009) [2023-12-27 05:01:09,865][105692] Updated weights for policy 0, policy_version 1869946 (0.0009) [2023-12-27 05:01:09,932][105692] Updated weights for policy 0, policy_version 1869956 (0.0008) [2023-12-27 05:01:09,997][105692] Updated weights for policy 0, policy_version 1869966 (0.0008) [2023-12-27 05:01:10,391][105620] Updated weights for policy 1, policy_version 1874390 (0.0009) [2023-12-27 05:01:10,454][105620] Updated weights for policy 1, policy_version 1874400 (0.0008) [2023-12-27 05:01:10,513][105620] Updated weights for policy 1, policy_version 1874410 (0.0008) [2023-12-27 05:01:10,684][105692] Updated weights for policy 0, policy_version 1869976 (0.0008) [2023-12-27 05:01:10,734][105692] Updated weights for policy 0, policy_version 1869986 (0.0007) [2023-12-27 05:01:10,793][105692] Updated weights for policy 0, policy_version 1869996 (0.0008) [2023-12-27 05:01:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 958709760. Throughput: 0: 9870.8, 1: 9296.8. Samples: 958717512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:01:11,062][104569] Avg episode reward: [(0, '8264.474'), (1, '9345.407')] [2023-12-27 05:01:11,300][105620] Updated weights for policy 1, policy_version 1874420 (0.0008) [2023-12-27 05:01:11,368][105620] Updated weights for policy 1, policy_version 1874430 (0.0009) [2023-12-27 05:01:11,449][105620] Updated weights for policy 1, policy_version 1874440 (0.0009) [2023-12-27 05:01:11,623][105692] Updated weights for policy 0, policy_version 1870006 (0.0010) [2023-12-27 05:01:11,692][105692] Updated weights for policy 0, policy_version 1870016 (0.0011) [2023-12-27 05:01:11,766][105692] Updated weights for policy 0, policy_version 1870026 (0.0006) [2023-12-27 05:01:12,177][105620] Updated weights for policy 1, policy_version 1874450 (0.0008) [2023-12-27 05:01:12,239][105620] Updated weights for policy 1, policy_version 1874460 (0.0008) [2023-12-27 05:01:12,303][105620] Updated weights for policy 1, policy_version 1874470 (0.0010) [2023-12-27 05:01:12,364][105620] Updated weights for policy 1, policy_version 1874480 (0.0009) [2023-12-27 05:01:12,574][105692] Updated weights for policy 0, policy_version 1870036 (0.0007) [2023-12-27 05:01:12,624][105692] Updated weights for policy 0, policy_version 1870046 (0.0005) [2023-12-27 05:01:12,690][105692] Updated weights for policy 0, policy_version 1870056 (0.0009) [2023-12-27 05:01:13,181][105620] Updated weights for policy 1, policy_version 1874490 (0.0009) [2023-12-27 05:01:13,242][105620] Updated weights for policy 1, policy_version 1874500 (0.0009) [2023-12-27 05:01:13,304][105620] Updated weights for policy 1, policy_version 1874510 (0.0009) [2023-12-27 05:01:13,411][105692] Updated weights for policy 0, policy_version 1870066 (0.0008) [2023-12-27 05:01:13,461][105692] Updated weights for policy 0, policy_version 1870076 (0.0008) [2023-12-27 05:01:13,513][105692] Updated weights for policy 0, policy_version 1870086 (0.0009) [2023-12-27 05:01:13,566][105692] Updated weights for policy 0, policy_version 1870096 (0.0008) [2023-12-27 05:01:14,067][105620] Updated weights for policy 1, policy_version 1874520 (0.0010) [2023-12-27 05:01:14,117][105620] Updated weights for policy 1, policy_version 1874530 (0.0010) [2023-12-27 05:01:14,173][105620] Updated weights for policy 1, policy_version 1874540 (0.0010) [2023-12-27 05:01:14,305][105692] Updated weights for policy 0, policy_version 1870106 (0.0008) [2023-12-27 05:01:14,362][105692] Updated weights for policy 0, policy_version 1870116 (0.0008) [2023-12-27 05:01:14,415][105692] Updated weights for policy 0, policy_version 1870126 (0.0008) [2023-12-27 05:01:14,955][105620] Updated weights for policy 1, policy_version 1874550 (0.0010) [2023-12-27 05:01:15,018][105620] Updated weights for policy 1, policy_version 1874560 (0.0011) [2023-12-27 05:01:15,087][105620] Updated weights for policy 1, policy_version 1874570 (0.0011) [2023-12-27 05:01:15,199][105692] Updated weights for policy 0, policy_version 1870136 (0.0009) [2023-12-27 05:01:15,248][105692] Updated weights for policy 0, policy_version 1870146 (0.0008) [2023-12-27 05:01:15,304][105692] Updated weights for policy 0, policy_version 1870156 (0.0008) [2023-12-27 05:01:15,836][105620] Updated weights for policy 1, policy_version 1874580 (0.0011) [2023-12-27 05:01:15,884][105620] Updated weights for policy 1, policy_version 1874590 (0.0010) [2023-12-27 05:01:15,937][105620] Updated weights for policy 1, policy_version 1874600 (0.0010) [2023-12-27 05:01:16,062][104569] Fps is (10 sec: 18021.9, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 958799872. Throughput: 0: 9827.4, 1: 9239.0. Samples: 958771080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:01:16,063][104569] Avg episode reward: [(0, '8539.295'), (1, '9345.454')] [2023-12-27 05:01:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001874608_479969280.pth... [2023-12-27 05:01:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001873520_479690752.pth [2023-12-27 05:01:16,076][105692] Updated weights for policy 0, policy_version 1870166 (0.0008) [2023-12-27 05:01:16,127][105692] Updated weights for policy 0, policy_version 1870176 (0.0008) [2023-12-27 05:01:16,176][105692] Updated weights for policy 0, policy_version 1870186 (0.0008) [2023-12-27 05:01:16,202][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001870192_478838784.pth... [2023-12-27 05:01:16,205][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001869072_478552064.pth [2023-12-27 05:01:16,645][105620] Updated weights for policy 1, policy_version 1874610 (0.0009) [2023-12-27 05:01:16,708][105620] Updated weights for policy 1, policy_version 1874620 (0.0005) [2023-12-27 05:01:16,765][105620] Updated weights for policy 1, policy_version 1874630 (0.0005) [2023-12-27 05:01:16,821][105620] Updated weights for policy 1, policy_version 1874640 (0.0005) [2023-12-27 05:01:16,948][105692] Updated weights for policy 0, policy_version 1870196 (0.0009) [2023-12-27 05:01:16,996][105692] Updated weights for policy 0, policy_version 1870206 (0.0008) [2023-12-27 05:01:17,063][105692] Updated weights for policy 0, policy_version 1870216 (0.0009) [2023-12-27 05:01:17,369][105620] Updated weights for policy 1, policy_version 1874650 (0.0005) [2023-12-27 05:01:17,420][105620] Updated weights for policy 1, policy_version 1874660 (0.0005) [2023-12-27 05:01:17,476][105620] Updated weights for policy 1, policy_version 1874670 (0.0005) [2023-12-27 05:01:17,731][105692] Updated weights for policy 0, policy_version 1870226 (0.0010) [2023-12-27 05:01:17,800][105692] Updated weights for policy 0, policy_version 1870236 (0.0011) [2023-12-27 05:01:17,855][105692] Updated weights for policy 0, policy_version 1870246 (0.0010) [2023-12-27 05:01:17,903][105692] Updated weights for policy 0, policy_version 1870256 (0.0010) [2023-12-27 05:01:18,006][105620] Updated weights for policy 1, policy_version 1874680 (0.0005) [2023-12-27 05:01:18,062][105620] Updated weights for policy 1, policy_version 1874690 (0.0006) [2023-12-27 05:01:18,114][105620] Updated weights for policy 1, policy_version 1874700 (0.0010) [2023-12-27 05:01:18,670][105692] Updated weights for policy 0, policy_version 1870266 (0.0011) [2023-12-27 05:01:18,744][105692] Updated weights for policy 0, policy_version 1870276 (0.0011) [2023-12-27 05:01:18,811][105692] Updated weights for policy 0, policy_version 1870286 (0.0011) [2023-12-27 05:01:18,811][105620] Updated weights for policy 1, policy_version 1874710 (0.0008) [2023-12-27 05:01:18,873][105620] Updated weights for policy 1, policy_version 1874720 (0.0008) [2023-12-27 05:01:18,927][105620] Updated weights for policy 1, policy_version 1874730 (0.0008) [2023-12-27 05:01:19,465][105692] Updated weights for policy 0, policy_version 1870296 (0.0011) [2023-12-27 05:01:19,534][105692] Updated weights for policy 0, policy_version 1870306 (0.0011) [2023-12-27 05:01:19,599][105692] Updated weights for policy 0, policy_version 1870316 (0.0011) [2023-12-27 05:01:19,685][105620] Updated weights for policy 1, policy_version 1874740 (0.0009) [2023-12-27 05:01:19,752][105620] Updated weights for policy 1, policy_version 1874750 (0.0011) [2023-12-27 05:01:19,817][105620] Updated weights for policy 1, policy_version 1874760 (0.0011) [2023-12-27 05:01:20,362][105692] Updated weights for policy 0, policy_version 1870326 (0.0011) [2023-12-27 05:01:20,421][105692] Updated weights for policy 0, policy_version 1870336 (0.0007) [2023-12-27 05:01:20,483][105692] Updated weights for policy 0, policy_version 1870346 (0.0006) [2023-12-27 05:01:20,581][105620] Updated weights for policy 1, policy_version 1874770 (0.0009) [2023-12-27 05:01:20,643][105620] Updated weights for policy 1, policy_version 1874780 (0.0011) [2023-12-27 05:01:20,700][105620] Updated weights for policy 1, policy_version 1874790 (0.0011) [2023-12-27 05:01:20,745][105620] Updated weights for policy 1, policy_version 1874800 (0.0010) [2023-12-27 05:01:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 958898176. Throughput: 0: 9652.8, 1: 9397.7. Samples: 958888620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:01:21,063][104569] Avg episode reward: [(0, '8447.319'), (1, '9164.495')] [2023-12-27 05:01:21,229][105692] Updated weights for policy 0, policy_version 1870356 (0.0007) [2023-12-27 05:01:21,296][105692] Updated weights for policy 0, policy_version 1870366 (0.0008) [2023-12-27 05:01:21,369][105692] Updated weights for policy 0, policy_version 1870376 (0.0008) [2023-12-27 05:01:21,429][105620] Updated weights for policy 1, policy_version 1874810 (0.0007) [2023-12-27 05:01:21,500][105620] Updated weights for policy 1, policy_version 1874820 (0.0006) [2023-12-27 05:01:21,559][105620] Updated weights for policy 1, policy_version 1874830 (0.0006) [2023-12-27 05:01:22,085][105692] Updated weights for policy 0, policy_version 1870386 (0.0008) [2023-12-27 05:01:22,147][105692] Updated weights for policy 0, policy_version 1870396 (0.0009) [2023-12-27 05:01:22,208][105692] Updated weights for policy 0, policy_version 1870406 (0.0009) [2023-12-27 05:01:22,232][105620] Updated weights for policy 1, policy_version 1874840 (0.0007) [2023-12-27 05:01:22,277][105692] Updated weights for policy 0, policy_version 1870416 (0.0008) [2023-12-27 05:01:22,296][105620] Updated weights for policy 1, policy_version 1874850 (0.0007) [2023-12-27 05:01:22,352][105620] Updated weights for policy 1, policy_version 1874860 (0.0009) [2023-12-27 05:01:22,979][105620] Updated weights for policy 1, policy_version 1874870 (0.0009) [2023-12-27 05:01:23,040][105620] Updated weights for policy 1, policy_version 1874880 (0.0009) [2023-12-27 05:01:23,115][105692] Updated weights for policy 0, policy_version 1870426 (0.0008) [2023-12-27 05:01:23,119][105620] Updated weights for policy 1, policy_version 1874890 (0.0007) [2023-12-27 05:01:23,178][105692] Updated weights for policy 0, policy_version 1870436 (0.0009) [2023-12-27 05:01:23,238][105692] Updated weights for policy 0, policy_version 1870446 (0.0010) [2023-12-27 05:01:23,753][105620] Updated weights for policy 1, policy_version 1874900 (0.0007) [2023-12-27 05:01:23,804][105620] Updated weights for policy 1, policy_version 1874910 (0.0009) [2023-12-27 05:01:23,857][105620] Updated weights for policy 1, policy_version 1874920 (0.0008) [2023-12-27 05:01:23,897][105692] Updated weights for policy 0, policy_version 1870456 (0.0007) [2023-12-27 05:01:23,967][105692] Updated weights for policy 0, policy_version 1870466 (0.0009) [2023-12-27 05:01:24,038][105692] Updated weights for policy 0, policy_version 1870476 (0.0010) [2023-12-27 05:01:24,441][105620] Updated weights for policy 1, policy_version 1874930 (0.0007) [2023-12-27 05:01:24,495][105620] Updated weights for policy 1, policy_version 1874940 (0.0005) [2023-12-27 05:01:24,543][105620] Updated weights for policy 1, policy_version 1874950 (0.0005) [2023-12-27 05:01:24,607][105620] Updated weights for policy 1, policy_version 1874960 (0.0007) [2023-12-27 05:01:24,833][105692] Updated weights for policy 0, policy_version 1870486 (0.0009) [2023-12-27 05:01:24,892][105692] Updated weights for policy 0, policy_version 1870496 (0.0008) [2023-12-27 05:01:24,941][105692] Updated weights for policy 0, policy_version 1870506 (0.0008) [2023-12-27 05:01:25,343][105620] Updated weights for policy 1, policy_version 1874970 (0.0010) [2023-12-27 05:01:25,402][105620] Updated weights for policy 1, policy_version 1874980 (0.0010) [2023-12-27 05:01:25,458][105620] Updated weights for policy 1, policy_version 1874990 (0.0010) [2023-12-27 05:01:25,710][105692] Updated weights for policy 0, policy_version 1870516 (0.0007) [2023-12-27 05:01:25,779][105692] Updated weights for policy 0, policy_version 1870526 (0.0005) [2023-12-27 05:01:25,829][105692] Updated weights for policy 0, policy_version 1870536 (0.0005) [2023-12-27 05:01:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.1, 300 sec: 19521.9). Total num frames: 958996480. Throughput: 0: 9583.0, 1: 9533.8. Samples: 959004196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:01:26,063][104569] Avg episode reward: [(0, '8625.034'), (1, '9072.235')] [2023-12-27 05:01:26,226][105620] Updated weights for policy 1, policy_version 1875000 (0.0008) [2023-12-27 05:01:26,282][105620] Updated weights for policy 1, policy_version 1875010 (0.0010) [2023-12-27 05:01:26,338][105620] Updated weights for policy 1, policy_version 1875020 (0.0011) [2023-12-27 05:01:26,410][105692] Updated weights for policy 0, policy_version 1870546 (0.0006) [2023-12-27 05:01:26,466][105692] Updated weights for policy 0, policy_version 1870556 (0.0009) [2023-12-27 05:01:26,525][105692] Updated weights for policy 0, policy_version 1870566 (0.0009) [2023-12-27 05:01:26,580][105692] Updated weights for policy 0, policy_version 1870576 (0.0006) [2023-12-27 05:01:27,004][105620] Updated weights for policy 1, policy_version 1875030 (0.0010) [2023-12-27 05:01:27,055][105620] Updated weights for policy 1, policy_version 1875040 (0.0010) [2023-12-27 05:01:27,099][105620] Updated weights for policy 1, policy_version 1875050 (0.0010) [2023-12-27 05:01:27,183][105692] Updated weights for policy 0, policy_version 1870586 (0.0008) [2023-12-27 05:01:27,242][105692] Updated weights for policy 0, policy_version 1870596 (0.0009) [2023-12-27 05:01:27,295][105692] Updated weights for policy 0, policy_version 1870606 (0.0009) [2023-12-27 05:01:27,819][105620] Updated weights for policy 1, policy_version 1875060 (0.0010) [2023-12-27 05:01:27,872][105620] Updated weights for policy 1, policy_version 1875070 (0.0007) [2023-12-27 05:01:27,918][105620] Updated weights for policy 1, policy_version 1875080 (0.0005) [2023-12-27 05:01:28,102][105692] Updated weights for policy 0, policy_version 1870616 (0.0009) [2023-12-27 05:01:28,154][105692] Updated weights for policy 0, policy_version 1870627 (0.0009) [2023-12-27 05:01:28,211][105692] Updated weights for policy 0, policy_version 1870637 (0.0009) [2023-12-27 05:01:28,521][105620] Updated weights for policy 1, policy_version 1875090 (0.0006) [2023-12-27 05:01:28,573][105620] Updated weights for policy 1, policy_version 1875100 (0.0011) [2023-12-27 05:01:28,633][105620] Updated weights for policy 1, policy_version 1875110 (0.0011) [2023-12-27 05:01:28,691][105620] Updated weights for policy 1, policy_version 1875120 (0.0010) [2023-12-27 05:01:29,023][105692] Updated weights for policy 0, policy_version 1870647 (0.0010) [2023-12-27 05:01:29,075][105692] Updated weights for policy 0, policy_version 1870657 (0.0010) [2023-12-27 05:01:29,128][105692] Updated weights for policy 0, policy_version 1870667 (0.0009) [2023-12-27 05:01:29,313][105620] Updated weights for policy 1, policy_version 1875130 (0.0006) [2023-12-27 05:01:29,382][105620] Updated weights for policy 1, policy_version 1875140 (0.0007) [2023-12-27 05:01:29,452][105620] Updated weights for policy 1, policy_version 1875151 (0.0010) [2023-12-27 05:01:29,926][105692] Updated weights for policy 0, policy_version 1870677 (0.0009) [2023-12-27 05:01:30,001][105692] Updated weights for policy 0, policy_version 1870687 (0.0006) [2023-12-27 05:01:30,063][105692] Updated weights for policy 0, policy_version 1870697 (0.0008) [2023-12-27 05:01:30,087][105620] Updated weights for policy 1, policy_version 1875161 (0.0006) [2023-12-27 05:01:30,148][105620] Updated weights for policy 1, policy_version 1875171 (0.0005) [2023-12-27 05:01:30,202][105620] Updated weights for policy 1, policy_version 1875181 (0.0006) [2023-12-27 05:01:30,787][105620] Updated weights for policy 1, policy_version 1875191 (0.0007) [2023-12-27 05:01:30,841][105620] Updated weights for policy 1, policy_version 1875201 (0.0006) [2023-12-27 05:01:30,880][105692] Updated weights for policy 0, policy_version 1870707 (0.0009) [2023-12-27 05:01:30,891][105620] Updated weights for policy 1, policy_version 1875211 (0.0006) [2023-12-27 05:01:30,931][105692] Updated weights for policy 0, policy_version 1870717 (0.0007) [2023-12-27 05:01:30,982][105692] Updated weights for policy 0, policy_version 1870727 (0.0009) [2023-12-27 05:01:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 959102976. Throughput: 0: 9582.7, 1: 9582.0. Samples: 959065708. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:01:31,062][104569] Avg episode reward: [(0, '8899.597'), (1, '9253.193')] [2023-12-27 05:01:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001875216_480124928.pth... [2023-12-27 05:01:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001870736_478978048.pth... [2023-12-27 05:01:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001874064_479830016.pth [2023-12-27 05:01:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001869616_478691328.pth [2023-12-27 05:01:31,581][105620] Updated weights for policy 1, policy_version 1875221 (0.0008) [2023-12-27 05:01:31,648][105620] Updated weights for policy 1, policy_version 1875231 (0.0009) [2023-12-27 05:01:31,721][105620] Updated weights for policy 1, policy_version 1875241 (0.0009) [2023-12-27 05:01:31,773][105692] Updated weights for policy 0, policy_version 1870737 (0.0009) [2023-12-27 05:01:31,826][105692] Updated weights for policy 0, policy_version 1870747 (0.0009) [2023-12-27 05:01:31,878][105692] Updated weights for policy 0, policy_version 1870757 (0.0007) [2023-12-27 05:01:31,931][105692] Updated weights for policy 0, policy_version 1870767 (0.0007) [2023-12-27 05:01:32,417][105620] Updated weights for policy 1, policy_version 1875251 (0.0009) [2023-12-27 05:01:32,474][105620] Updated weights for policy 1, policy_version 1875261 (0.0009) [2023-12-27 05:01:32,538][105620] Updated weights for policy 1, policy_version 1875271 (0.0008) [2023-12-27 05:01:32,665][105692] Updated weights for policy 0, policy_version 1870777 (0.0009) [2023-12-27 05:01:32,724][105692] Updated weights for policy 0, policy_version 1870787 (0.0010) [2023-12-27 05:01:32,772][105692] Updated weights for policy 0, policy_version 1870797 (0.0009) [2023-12-27 05:01:33,242][105620] Updated weights for policy 1, policy_version 1875281 (0.0006) [2023-12-27 05:01:33,299][105620] Updated weights for policy 1, policy_version 1875291 (0.0008) [2023-12-27 05:01:33,349][105620] Updated weights for policy 1, policy_version 1875301 (0.0009) [2023-12-27 05:01:33,400][105620] Updated weights for policy 1, policy_version 1875311 (0.0009) [2023-12-27 05:01:33,530][105692] Updated weights for policy 0, policy_version 1870807 (0.0008) [2023-12-27 05:01:33,577][105692] Updated weights for policy 0, policy_version 1870817 (0.0009) [2023-12-27 05:01:33,621][105692] Updated weights for policy 0, policy_version 1870827 (0.0010) [2023-12-27 05:01:34,039][105620] Updated weights for policy 1, policy_version 1875321 (0.0007) [2023-12-27 05:01:34,096][105620] Updated weights for policy 1, policy_version 1875331 (0.0006) [2023-12-27 05:01:34,169][105620] Updated weights for policy 1, policy_version 1875341 (0.0006) [2023-12-27 05:01:34,349][105692] Updated weights for policy 0, policy_version 1870837 (0.0010) [2023-12-27 05:01:34,405][105692] Updated weights for policy 0, policy_version 1870847 (0.0009) [2023-12-27 05:01:34,463][105692] Updated weights for policy 0, policy_version 1870857 (0.0008) [2023-12-27 05:01:34,879][105620] Updated weights for policy 1, policy_version 1875351 (0.0008) [2023-12-27 05:01:34,943][105620] Updated weights for policy 1, policy_version 1875361 (0.0008) [2023-12-27 05:01:35,004][105620] Updated weights for policy 1, policy_version 1875371 (0.0006) [2023-12-27 05:01:35,272][105692] Updated weights for policy 0, policy_version 1870867 (0.0010) [2023-12-27 05:01:35,321][105692] Updated weights for policy 0, policy_version 1870878 (0.0008) [2023-12-27 05:01:35,368][105692] Updated weights for policy 0, policy_version 1870888 (0.0005) [2023-12-27 05:01:35,569][105620] Updated weights for policy 1, policy_version 1875381 (0.0006) [2023-12-27 05:01:35,638][105620] Updated weights for policy 1, policy_version 1875391 (0.0008) [2023-12-27 05:01:35,703][105620] Updated weights for policy 1, policy_version 1875401 (0.0008) [2023-12-27 05:01:36,045][105692] Updated weights for policy 0, policy_version 1870898 (0.0005) [2023-12-27 05:01:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 959193088. Throughput: 0: 9395.7, 1: 9786.6. Samples: 959182664. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:01:36,063][104569] Avg episode reward: [(0, '8899.403'), (1, '9161.305')] [2023-12-27 05:01:36,094][105692] Updated weights for policy 0, policy_version 1870908 (0.0005) [2023-12-27 05:01:36,151][105692] Updated weights for policy 0, policy_version 1870918 (0.0008) [2023-12-27 05:01:36,210][105692] Updated weights for policy 0, policy_version 1870928 (0.0008) [2023-12-27 05:01:36,514][105620] Updated weights for policy 1, policy_version 1875411 (0.0009) [2023-12-27 05:01:36,575][105620] Updated weights for policy 1, policy_version 1875421 (0.0008) [2023-12-27 05:01:36,636][105620] Updated weights for policy 1, policy_version 1875431 (0.0008) [2023-12-27 05:01:36,907][105692] Updated weights for policy 0, policy_version 1870938 (0.0010) [2023-12-27 05:01:36,956][105692] Updated weights for policy 0, policy_version 1870948 (0.0011) [2023-12-27 05:01:37,011][105692] Updated weights for policy 0, policy_version 1870958 (0.0010) [2023-12-27 05:01:37,342][105620] Updated weights for policy 1, policy_version 1875441 (0.0008) [2023-12-27 05:01:37,400][105620] Updated weights for policy 1, policy_version 1875451 (0.0005) [2023-12-27 05:01:37,463][105620] Updated weights for policy 1, policy_version 1875461 (0.0005) [2023-12-27 05:01:37,531][105620] Updated weights for policy 1, policy_version 1875471 (0.0007) [2023-12-27 05:01:37,771][105692] Updated weights for policy 0, policy_version 1870968 (0.0011) [2023-12-27 05:01:37,835][105692] Updated weights for policy 0, policy_version 1870978 (0.0010) [2023-12-27 05:01:37,892][105692] Updated weights for policy 0, policy_version 1870988 (0.0009) [2023-12-27 05:01:38,126][105620] Updated weights for policy 1, policy_version 1875481 (0.0006) [2023-12-27 05:01:38,191][105620] Updated weights for policy 1, policy_version 1875491 (0.0005) [2023-12-27 05:01:38,258][105620] Updated weights for policy 1, policy_version 1875501 (0.0009) [2023-12-27 05:01:38,590][105692] Updated weights for policy 0, policy_version 1870998 (0.0009) [2023-12-27 05:01:38,654][105692] Updated weights for policy 0, policy_version 1871008 (0.0008) [2023-12-27 05:01:38,713][105692] Updated weights for policy 0, policy_version 1871018 (0.0011) [2023-12-27 05:01:38,841][105620] Updated weights for policy 1, policy_version 1875511 (0.0009) [2023-12-27 05:01:38,892][105620] Updated weights for policy 1, policy_version 1875521 (0.0008) [2023-12-27 05:01:38,939][105620] Updated weights for policy 1, policy_version 1875531 (0.0008) [2023-12-27 05:01:39,455][105692] Updated weights for policy 0, policy_version 1871028 (0.0010) [2023-12-27 05:01:39,518][105692] Updated weights for policy 0, policy_version 1871038 (0.0011) [2023-12-27 05:01:39,585][105692] Updated weights for policy 0, policy_version 1871048 (0.0008) [2023-12-27 05:01:39,693][105620] Updated weights for policy 1, policy_version 1875541 (0.0009) [2023-12-27 05:01:39,754][105620] Updated weights for policy 1, policy_version 1875551 (0.0009) [2023-12-27 05:01:39,817][105620] Updated weights for policy 1, policy_version 1875561 (0.0008) [2023-12-27 05:01:40,323][105692] Updated weights for policy 0, policy_version 1871058 (0.0011) [2023-12-27 05:01:40,379][105692] Updated weights for policy 0, policy_version 1871068 (0.0011) [2023-12-27 05:01:40,438][105692] Updated weights for policy 0, policy_version 1871078 (0.0010) [2023-12-27 05:01:40,502][105692] Updated weights for policy 0, policy_version 1871088 (0.0010) [2023-12-27 05:01:40,526][105620] Updated weights for policy 1, policy_version 1875571 (0.0008) [2023-12-27 05:01:40,588][105620] Updated weights for policy 1, policy_version 1875581 (0.0007) [2023-12-27 05:01:40,651][105620] Updated weights for policy 1, policy_version 1875591 (0.0008) [2023-12-27 05:01:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 959291392. Throughput: 0: 9394.2, 1: 9787.4. Samples: 959301172. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:01:41,063][104569] Avg episode reward: [(0, '8624.493'), (1, '8977.239')] [2023-12-27 05:01:41,262][105692] Updated weights for policy 0, policy_version 1871098 (0.0010) [2023-12-27 05:01:41,322][105692] Updated weights for policy 0, policy_version 1871108 (0.0010) [2023-12-27 05:01:41,378][105620] Updated weights for policy 1, policy_version 1875601 (0.0007) [2023-12-27 05:01:41,392][105692] Updated weights for policy 0, policy_version 1871118 (0.0008) [2023-12-27 05:01:41,441][105620] Updated weights for policy 1, policy_version 1875611 (0.0007) [2023-12-27 05:01:41,503][105620] Updated weights for policy 1, policy_version 1875621 (0.0010) [2023-12-27 05:01:41,565][105620] Updated weights for policy 1, policy_version 1875631 (0.0009) [2023-12-27 05:01:42,167][105692] Updated weights for policy 0, policy_version 1871128 (0.0009) [2023-12-27 05:01:42,228][105692] Updated weights for policy 0, policy_version 1871138 (0.0009) [2023-12-27 05:01:42,295][105692] Updated weights for policy 0, policy_version 1871148 (0.0009) [2023-12-27 05:01:42,335][105620] Updated weights for policy 1, policy_version 1875641 (0.0008) [2023-12-27 05:01:42,403][105620] Updated weights for policy 1, policy_version 1875651 (0.0008) [2023-12-27 05:01:42,468][105620] Updated weights for policy 1, policy_version 1875661 (0.0009) [2023-12-27 05:01:42,990][105692] Updated weights for policy 0, policy_version 1871158 (0.0006) [2023-12-27 05:01:43,041][105692] Updated weights for policy 0, policy_version 1871168 (0.0006) [2023-12-27 05:01:43,090][105692] Updated weights for policy 0, policy_version 1871178 (0.0008) [2023-12-27 05:01:43,214][105620] Updated weights for policy 1, policy_version 1875671 (0.0010) [2023-12-27 05:01:43,269][105620] Updated weights for policy 1, policy_version 1875681 (0.0010) [2023-12-27 05:01:43,322][105620] Updated weights for policy 1, policy_version 1875691 (0.0010) [2023-12-27 05:01:43,709][105692] Updated weights for policy 0, policy_version 1871188 (0.0007) [2023-12-27 05:01:43,759][105692] Updated weights for policy 0, policy_version 1871198 (0.0005) [2023-12-27 05:01:43,802][105692] Updated weights for policy 0, policy_version 1871208 (0.0005) [2023-12-27 05:01:44,014][105620] Updated weights for policy 1, policy_version 1875701 (0.0010) [2023-12-27 05:01:44,070][105620] Updated weights for policy 1, policy_version 1875711 (0.0011) [2023-12-27 05:01:44,122][105620] Updated weights for policy 1, policy_version 1875721 (0.0010) [2023-12-27 05:01:44,401][105692] Updated weights for policy 0, policy_version 1871218 (0.0005) [2023-12-27 05:01:44,453][105692] Updated weights for policy 0, policy_version 1871228 (0.0006) [2023-12-27 05:01:44,513][105692] Updated weights for policy 0, policy_version 1871238 (0.0008) [2023-12-27 05:01:44,567][105692] Updated weights for policy 0, policy_version 1871248 (0.0008) [2023-12-27 05:01:44,807][105620] Updated weights for policy 1, policy_version 1875731 (0.0009) [2023-12-27 05:01:44,867][105620] Updated weights for policy 1, policy_version 1875741 (0.0006) [2023-12-27 05:01:44,920][105620] Updated weights for policy 1, policy_version 1875751 (0.0005) [2023-12-27 05:01:45,334][105692] Updated weights for policy 0, policy_version 1871258 (0.0011) [2023-12-27 05:01:45,395][105692] Updated weights for policy 0, policy_version 1871268 (0.0009) [2023-12-27 05:01:45,451][105692] Updated weights for policy 0, policy_version 1871278 (0.0010) [2023-12-27 05:01:45,618][105620] Updated weights for policy 1, policy_version 1875761 (0.0006) [2023-12-27 05:01:45,667][105620] Updated weights for policy 1, policy_version 1875771 (0.0010) [2023-12-27 05:01:45,712][105620] Updated weights for policy 1, policy_version 1875781 (0.0010) [2023-12-27 05:01:45,760][105620] Updated weights for policy 1, policy_version 1875791 (0.0010) [2023-12-27 05:01:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19251.1, 300 sec: 19494.2). Total num frames: 959389696. Throughput: 0: 9402.4, 1: 9817.7. Samples: 959358496. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:01:46,063][104569] Avg episode reward: [(0, '7997.203'), (1, '8979.896')] [2023-12-27 05:01:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001871280_479117312.pth... [2023-12-27 05:01:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001875792_480272384.pth... [2023-12-27 05:01:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001870192_478838784.pth [2023-12-27 05:01:46,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001874608_479969280.pth [2023-12-27 05:01:46,077][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_001875792_480272384.pth [2023-12-27 05:01:46,077][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_001871280_479117312.pth [2023-12-27 05:01:46,234][105692] Updated weights for policy 0, policy_version 1871288 (0.0010) [2023-12-27 05:01:46,289][105692] Updated weights for policy 0, policy_version 1871298 (0.0009) [2023-12-27 05:01:46,340][105692] Updated weights for policy 0, policy_version 1871308 (0.0009) [2023-12-27 05:01:46,406][105620] Updated weights for policy 1, policy_version 1875801 (0.0010) [2023-12-27 05:01:46,457][105620] Updated weights for policy 1, policy_version 1875811 (0.0010) [2023-12-27 05:01:46,502][105620] Updated weights for policy 1, policy_version 1875821 (0.0007) [2023-12-27 05:01:47,074][105692] Updated weights for policy 0, policy_version 1871318 (0.0009) [2023-12-27 05:01:47,119][105692] Updated weights for policy 0, policy_version 1871328 (0.0010) [2023-12-27 05:01:47,130][105620] Updated weights for policy 1, policy_version 1875831 (0.0009) [2023-12-27 05:01:47,164][105692] Updated weights for policy 0, policy_version 1871338 (0.0010) [2023-12-27 05:01:47,185][105620] Updated weights for policy 1, policy_version 1875841 (0.0010) [2023-12-27 05:01:47,230][105620] Updated weights for policy 1, policy_version 1875851 (0.0008) [2023-12-27 05:01:47,847][105620] Updated weights for policy 1, policy_version 1875861 (0.0007) [2023-12-27 05:01:47,894][105620] Updated weights for policy 1, policy_version 1875871 (0.0008) [2023-12-27 05:01:47,925][105692] Updated weights for policy 0, policy_version 1871348 (0.0010) [2023-12-27 05:01:47,950][105620] Updated weights for policy 1, policy_version 1875881 (0.0006) [2023-12-27 05:01:47,981][105692] Updated weights for policy 0, policy_version 1871358 (0.0010) [2023-12-27 05:01:48,040][105692] Updated weights for policy 0, policy_version 1871368 (0.0005) [2023-12-27 05:01:48,685][105692] Updated weights for policy 0, policy_version 1871378 (0.0005) [2023-12-27 05:01:48,689][105620] Updated weights for policy 1, policy_version 1875891 (0.0009) [2023-12-27 05:01:48,748][105692] Updated weights for policy 0, policy_version 1871388 (0.0006) [2023-12-27 05:01:48,748][105620] Updated weights for policy 1, policy_version 1875901 (0.0011) [2023-12-27 05:01:48,807][105692] Updated weights for policy 0, policy_version 1871398 (0.0006) [2023-12-27 05:01:48,812][105620] Updated weights for policy 1, policy_version 1875911 (0.0011) [2023-12-27 05:01:48,866][105692] Updated weights for policy 0, policy_version 1871408 (0.0005) [2023-12-27 05:01:49,508][105620] Updated weights for policy 1, policy_version 1875921 (0.0011) [2023-12-27 05:01:49,519][105692] Updated weights for policy 0, policy_version 1871418 (0.0010) [2023-12-27 05:01:49,557][105620] Updated weights for policy 1, policy_version 1875931 (0.0006) [2023-12-27 05:01:49,575][105692] Updated weights for policy 0, policy_version 1871428 (0.0010) [2023-12-27 05:01:49,609][105620] Updated weights for policy 1, policy_version 1875941 (0.0006) [2023-12-27 05:01:49,634][105692] Updated weights for policy 0, policy_version 1871438 (0.0010) [2023-12-27 05:01:49,664][105620] Updated weights for policy 1, policy_version 1875951 (0.0009) [2023-12-27 05:01:50,426][105692] Updated weights for policy 0, policy_version 1871448 (0.0011) [2023-12-27 05:01:50,453][105620] Updated weights for policy 1, policy_version 1875961 (0.0007) [2023-12-27 05:01:50,479][105692] Updated weights for policy 0, policy_version 1871458 (0.0010) [2023-12-27 05:01:50,509][105620] Updated weights for policy 1, policy_version 1875971 (0.0006) [2023-12-27 05:01:50,539][105692] Updated weights for policy 0, policy_version 1871468 (0.0011) [2023-12-27 05:01:50,571][105620] Updated weights for policy 1, policy_version 1875981 (0.0006) [2023-12-27 05:01:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 959488000. Throughput: 0: 9548.5, 1: 9931.7. Samples: 959479712. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:01:51,062][104569] Avg episode reward: [(0, '8267.713'), (1, '9163.916')] [2023-12-27 05:01:51,335][105692] Updated weights for policy 0, policy_version 1871478 (0.0010) [2023-12-27 05:01:51,348][105620] Updated weights for policy 1, policy_version 1875991 (0.0008) [2023-12-27 05:01:51,408][105692] Updated weights for policy 0, policy_version 1871488 (0.0008) [2023-12-27 05:01:51,416][105620] Updated weights for policy 1, policy_version 1876001 (0.0008) [2023-12-27 05:01:51,472][105692] Updated weights for policy 0, policy_version 1871498 (0.0011) [2023-12-27 05:01:51,479][105620] Updated weights for policy 1, policy_version 1876011 (0.0008) [2023-12-27 05:01:52,166][105620] Updated weights for policy 1, policy_version 1876021 (0.0008) [2023-12-27 05:01:52,172][105692] Updated weights for policy 0, policy_version 1871508 (0.0011) [2023-12-27 05:01:52,230][105692] Updated weights for policy 0, policy_version 1871518 (0.0010) [2023-12-27 05:01:52,232][105620] Updated weights for policy 1, policy_version 1876031 (0.0011) [2023-12-27 05:01:52,292][105620] Updated weights for policy 1, policy_version 1876041 (0.0008) [2023-12-27 05:01:52,295][105692] Updated weights for policy 0, policy_version 1871528 (0.0009) [2023-12-27 05:01:52,939][105620] Updated weights for policy 1, policy_version 1876051 (0.0007) [2023-12-27 05:01:52,999][105620] Updated weights for policy 1, policy_version 1876061 (0.0010) [2023-12-27 05:01:53,059][105620] Updated weights for policy 1, policy_version 1876071 (0.0009) [2023-12-27 05:01:53,081][105692] Updated weights for policy 0, policy_version 1871538 (0.0008) [2023-12-27 05:01:53,137][105692] Updated weights for policy 0, policy_version 1871548 (0.0007) [2023-12-27 05:01:53,191][105692] Updated weights for policy 0, policy_version 1871558 (0.0009) [2023-12-27 05:01:53,248][105692] Updated weights for policy 0, policy_version 1871568 (0.0010) [2023-12-27 05:01:53,809][105620] Updated weights for policy 1, policy_version 1876081 (0.0006) [2023-12-27 05:01:53,873][105620] Updated weights for policy 1, policy_version 1876091 (0.0006) [2023-12-27 05:01:53,887][105692] Updated weights for policy 0, policy_version 1871578 (0.0009) [2023-12-27 05:01:53,919][105620] Updated weights for policy 1, policy_version 1876101 (0.0005) [2023-12-27 05:01:53,943][105692] Updated weights for policy 0, policy_version 1871588 (0.0009) [2023-12-27 05:01:53,973][105620] Updated weights for policy 1, policy_version 1876111 (0.0005) [2023-12-27 05:01:54,000][105692] Updated weights for policy 0, policy_version 1871598 (0.0010) [2023-12-27 05:01:54,626][105692] Updated weights for policy 0, policy_version 1871608 (0.0009) [2023-12-27 05:01:54,682][105692] Updated weights for policy 0, policy_version 1871618 (0.0008) [2023-12-27 05:01:54,703][105620] Updated weights for policy 1, policy_version 1876121 (0.0007) [2023-12-27 05:01:54,741][105692] Updated weights for policy 0, policy_version 1871628 (0.0006) [2023-12-27 05:01:54,762][105620] Updated weights for policy 1, policy_version 1876131 (0.0007) [2023-12-27 05:01:54,826][105620] Updated weights for policy 1, policy_version 1876141 (0.0009) [2023-12-27 05:01:55,354][105692] Updated weights for policy 0, policy_version 1871638 (0.0005) [2023-12-27 05:01:55,405][105692] Updated weights for policy 0, policy_version 1871648 (0.0005) [2023-12-27 05:01:55,430][105620] Updated weights for policy 1, policy_version 1876151 (0.0009) [2023-12-27 05:01:55,453][105692] Updated weights for policy 0, policy_version 1871658 (0.0005) [2023-12-27 05:01:55,488][105620] Updated weights for policy 1, policy_version 1876161 (0.0009) [2023-12-27 05:01:55,544][105620] Updated weights for policy 1, policy_version 1876171 (0.0007) [2023-12-27 05:01:56,062][104569] Fps is (10 sec: 19661.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 959586304. Throughput: 0: 9552.5, 1: 10034.1. Samples: 959598912. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:01:56,063][104569] Avg episode reward: [(0, '8624.977'), (1, '9253.275')] [2023-12-27 05:01:56,104][105620] Updated weights for policy 1, policy_version 1876181 (0.0006) [2023-12-27 05:01:56,163][105620] Updated weights for policy 1, policy_version 1876191 (0.0006) [2023-12-27 05:01:56,217][105620] Updated weights for policy 1, policy_version 1876201 (0.0006) [2023-12-27 05:01:56,221][105692] Updated weights for policy 0, policy_version 1871668 (0.0007) [2023-12-27 05:01:56,281][105692] Updated weights for policy 0, policy_version 1871678 (0.0009) [2023-12-27 05:01:56,344][105692] Updated weights for policy 0, policy_version 1871688 (0.0009) [2023-12-27 05:01:56,908][105620] Updated weights for policy 1, policy_version 1876211 (0.0008) [2023-12-27 05:01:56,968][105620] Updated weights for policy 1, policy_version 1876221 (0.0008) [2023-12-27 05:01:57,038][105620] Updated weights for policy 1, policy_version 1876231 (0.0009) [2023-12-27 05:01:57,117][105692] Updated weights for policy 0, policy_version 1871698 (0.0009) [2023-12-27 05:01:57,175][105692] Updated weights for policy 0, policy_version 1871708 (0.0005) [2023-12-27 05:01:57,221][105692] Updated weights for policy 0, policy_version 1871718 (0.0005) [2023-12-27 05:01:57,265][105692] Updated weights for policy 0, policy_version 1871728 (0.0006) [2023-12-27 05:01:57,625][105620] Updated weights for policy 1, policy_version 1876241 (0.0006) [2023-12-27 05:01:57,673][105620] Updated weights for policy 1, policy_version 1876251 (0.0010) [2023-12-27 05:01:57,732][105620] Updated weights for policy 1, policy_version 1876261 (0.0005) [2023-12-27 05:01:57,789][105620] Updated weights for policy 1, policy_version 1876271 (0.0005) [2023-12-27 05:01:58,025][105692] Updated weights for policy 0, policy_version 1871738 (0.0009) [2023-12-27 05:01:58,084][105692] Updated weights for policy 0, policy_version 1871748 (0.0008) [2023-12-27 05:01:58,131][105692] Updated weights for policy 0, policy_version 1871758 (0.0008) [2023-12-27 05:01:58,505][105620] Updated weights for policy 1, policy_version 1876281 (0.0010) [2023-12-27 05:01:58,568][105620] Updated weights for policy 1, policy_version 1876291 (0.0010) [2023-12-27 05:01:58,639][105620] Updated weights for policy 1, policy_version 1876301 (0.0009) [2023-12-27 05:01:59,014][105692] Updated weights for policy 0, policy_version 1871768 (0.0008) [2023-12-27 05:01:59,066][105692] Updated weights for policy 0, policy_version 1871778 (0.0006) [2023-12-27 05:01:59,129][105692] Updated weights for policy 0, policy_version 1871788 (0.0007) [2023-12-27 05:01:59,442][105620] Updated weights for policy 1, policy_version 1876311 (0.0009) [2023-12-27 05:01:59,504][105620] Updated weights for policy 1, policy_version 1876321 (0.0008) [2023-12-27 05:01:59,562][105620] Updated weights for policy 1, policy_version 1876331 (0.0008) [2023-12-27 05:01:59,789][105692] Updated weights for policy 0, policy_version 1871798 (0.0008) [2023-12-27 05:01:59,852][105692] Updated weights for policy 0, policy_version 1871808 (0.0008) [2023-12-27 05:01:59,913][105692] Updated weights for policy 0, policy_version 1871818 (0.0008) [2023-12-27 05:02:00,243][105620] Updated weights for policy 1, policy_version 1876341 (0.0007) [2023-12-27 05:02:00,295][105620] Updated weights for policy 1, policy_version 1876351 (0.0005) [2023-12-27 05:02:00,348][105620] Updated weights for policy 1, policy_version 1876361 (0.0005) [2023-12-27 05:02:00,645][105692] Updated weights for policy 0, policy_version 1871828 (0.0008) [2023-12-27 05:02:00,706][105692] Updated weights for policy 0, policy_version 1871838 (0.0009) [2023-12-27 05:02:00,768][105692] Updated weights for policy 0, policy_version 1871848 (0.0008) [2023-12-27 05:02:00,984][105620] Updated weights for policy 1, policy_version 1876371 (0.0007) [2023-12-27 05:02:01,040][105620] Updated weights for policy 1, policy_version 1876381 (0.0008) [2023-12-27 05:02:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 959684608. Throughput: 0: 9552.4, 1: 10114.8. Samples: 959656100. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:02:01,062][104569] Avg episode reward: [(0, '8626.101'), (1, '9253.206')] [2023-12-27 05:02:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001871856_479264768.pth... [2023-12-27 05:02:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001870736_478978048.pth [2023-12-27 05:02:01,097][105620] Updated weights for policy 1, policy_version 1876391 (0.0007) [2023-12-27 05:02:01,154][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001876400_480428032.pth... [2023-12-27 05:02:01,158][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001875216_480124928.pth [2023-12-27 05:02:01,538][105692] Updated weights for policy 0, policy_version 1871858 (0.0005) [2023-12-27 05:02:01,598][105692] Updated weights for policy 0, policy_version 1871868 (0.0008) [2023-12-27 05:02:01,665][105692] Updated weights for policy 0, policy_version 1871878 (0.0007) [2023-12-27 05:02:01,731][105692] Updated weights for policy 0, policy_version 1871888 (0.0009) [2023-12-27 05:02:01,826][105620] Updated weights for policy 1, policy_version 1876401 (0.0007) [2023-12-27 05:02:01,888][105620] Updated weights for policy 1, policy_version 1876411 (0.0009) [2023-12-27 05:02:01,944][105620] Updated weights for policy 1, policy_version 1876421 (0.0009) [2023-12-27 05:02:01,995][105620] Updated weights for policy 1, policy_version 1876431 (0.0009) [2023-12-27 05:02:02,355][105692] Updated weights for policy 0, policy_version 1871898 (0.0008) [2023-12-27 05:02:02,417][105692] Updated weights for policy 0, policy_version 1871908 (0.0009) [2023-12-27 05:02:02,475][105692] Updated weights for policy 0, policy_version 1871918 (0.0009) [2023-12-27 05:02:02,826][105620] Updated weights for policy 1, policy_version 1876441 (0.0009) [2023-12-27 05:02:02,886][105620] Updated weights for policy 1, policy_version 1876451 (0.0008) [2023-12-27 05:02:02,932][105620] Updated weights for policy 1, policy_version 1876461 (0.0008) [2023-12-27 05:02:03,256][105692] Updated weights for policy 0, policy_version 1871928 (0.0010) [2023-12-27 05:02:03,309][105692] Updated weights for policy 0, policy_version 1871939 (0.0009) [2023-12-27 05:02:03,362][105692] Updated weights for policy 0, policy_version 1871949 (0.0010) [2023-12-27 05:02:03,498][105620] Updated weights for policy 1, policy_version 1876471 (0.0006) [2023-12-27 05:02:03,550][105620] Updated weights for policy 1, policy_version 1876481 (0.0006) [2023-12-27 05:02:03,602][105620] Updated weights for policy 1, policy_version 1876491 (0.0009) [2023-12-27 05:02:04,176][105692] Updated weights for policy 0, policy_version 1871959 (0.0009) [2023-12-27 05:02:04,238][105692] Updated weights for policy 0, policy_version 1871969 (0.0009) [2023-12-27 05:02:04,293][105620] Updated weights for policy 1, policy_version 1876501 (0.0008) [2023-12-27 05:02:04,298][105692] Updated weights for policy 0, policy_version 1871979 (0.0009) [2023-12-27 05:02:04,353][105620] Updated weights for policy 1, policy_version 1876511 (0.0006) [2023-12-27 05:02:04,424][105620] Updated weights for policy 1, policy_version 1876521 (0.0005) [2023-12-27 05:02:05,038][105620] Updated weights for policy 1, policy_version 1876531 (0.0009) [2023-12-27 05:02:05,094][105620] Updated weights for policy 1, policy_version 1876541 (0.0007) [2023-12-27 05:02:05,138][105692] Updated weights for policy 0, policy_version 1871989 (0.0008) [2023-12-27 05:02:05,157][105620] Updated weights for policy 1, policy_version 1876551 (0.0006) [2023-12-27 05:02:05,199][105692] Updated weights for policy 0, policy_version 1871999 (0.0009) [2023-12-27 05:02:05,263][105692] Updated weights for policy 0, policy_version 1872009 (0.0009) [2023-12-27 05:02:05,712][105620] Updated weights for policy 1, policy_version 1876561 (0.0005) [2023-12-27 05:02:05,763][105620] Updated weights for policy 1, policy_version 1876571 (0.0005) [2023-12-27 05:02:05,822][105620] Updated weights for policy 1, policy_version 1876581 (0.0005) [2023-12-27 05:02:05,886][105620] Updated weights for policy 1, policy_version 1876591 (0.0007) [2023-12-27 05:02:06,039][105692] Updated weights for policy 0, policy_version 1872019 (0.0010) [2023-12-27 05:02:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 959782912. Throughput: 0: 9527.4, 1: 10130.3. Samples: 959773216. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:02:06,062][104569] Avg episode reward: [(0, '8444.085'), (1, '9253.699')] [2023-12-27 05:02:06,086][105692] Updated weights for policy 0, policy_version 1872029 (0.0009) [2023-12-27 05:02:06,138][105692] Updated weights for policy 0, policy_version 1872039 (0.0009) [2023-12-27 05:02:06,538][105620] Updated weights for policy 1, policy_version 1876601 (0.0009) [2023-12-27 05:02:06,600][105620] Updated weights for policy 1, policy_version 1876611 (0.0009) [2023-12-27 05:02:06,653][105620] Updated weights for policy 1, policy_version 1876621 (0.0010) [2023-12-27 05:02:06,858][105692] Updated weights for policy 0, policy_version 1872049 (0.0010) [2023-12-27 05:02:06,907][105692] Updated weights for policy 0, policy_version 1872059 (0.0009) [2023-12-27 05:02:06,955][105692] Updated weights for policy 0, policy_version 1872069 (0.0009) [2023-12-27 05:02:07,014][105692] Updated weights for policy 0, policy_version 1872079 (0.0010) [2023-12-27 05:02:07,440][105620] Updated weights for policy 1, policy_version 1876631 (0.0008) [2023-12-27 05:02:07,505][105620] Updated weights for policy 1, policy_version 1876641 (0.0009) [2023-12-27 05:02:07,565][105620] Updated weights for policy 1, policy_version 1876651 (0.0008) [2023-12-27 05:02:07,751][105692] Updated weights for policy 0, policy_version 1872089 (0.0007) [2023-12-27 05:02:07,805][105692] Updated weights for policy 0, policy_version 1872099 (0.0008) [2023-12-27 05:02:07,855][105692] Updated weights for policy 0, policy_version 1872109 (0.0006) [2023-12-27 05:02:08,227][105620] Updated weights for policy 1, policy_version 1876661 (0.0008) [2023-12-27 05:02:08,284][105620] Updated weights for policy 1, policy_version 1876671 (0.0009) [2023-12-27 05:02:08,340][105620] Updated weights for policy 1, policy_version 1876681 (0.0008) [2023-12-27 05:02:08,603][105692] Updated weights for policy 0, policy_version 1872119 (0.0008) [2023-12-27 05:02:08,650][105692] Updated weights for policy 0, policy_version 1872129 (0.0009) [2023-12-27 05:02:08,697][105692] Updated weights for policy 0, policy_version 1872139 (0.0009) [2023-12-27 05:02:09,067][105620] Updated weights for policy 1, policy_version 1876691 (0.0007) [2023-12-27 05:02:09,121][105620] Updated weights for policy 1, policy_version 1876701 (0.0006) [2023-12-27 05:02:09,182][105620] Updated weights for policy 1, policy_version 1876711 (0.0009) [2023-12-27 05:02:09,533][105692] Updated weights for policy 0, policy_version 1872149 (0.0008) [2023-12-27 05:02:09,592][105692] Updated weights for policy 0, policy_version 1872159 (0.0009) [2023-12-27 05:02:09,657][105692] Updated weights for policy 0, policy_version 1872169 (0.0009) [2023-12-27 05:02:09,858][105620] Updated weights for policy 1, policy_version 1876721 (0.0008) [2023-12-27 05:02:09,924][105620] Updated weights for policy 1, policy_version 1876731 (0.0007) [2023-12-27 05:02:09,992][105620] Updated weights for policy 1, policy_version 1876741 (0.0007) [2023-12-27 05:02:10,061][105620] Updated weights for policy 1, policy_version 1876751 (0.0009) [2023-12-27 05:02:10,484][105692] Updated weights for policy 0, policy_version 1872179 (0.0008) [2023-12-27 05:02:10,541][105692] Updated weights for policy 0, policy_version 1872189 (0.0009) [2023-12-27 05:02:10,599][105692] Updated weights for policy 0, policy_version 1872200 (0.0009) [2023-12-27 05:02:10,637][105620] Updated weights for policy 1, policy_version 1876761 (0.0006) [2023-12-27 05:02:10,694][105620] Updated weights for policy 1, policy_version 1876771 (0.0006) [2023-12-27 05:02:10,761][105620] Updated weights for policy 1, policy_version 1876781 (0.0011) [2023-12-27 05:02:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 959881216. Throughput: 0: 9528.0, 1: 10147.3. Samples: 959889584. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:02:11,063][104569] Avg episode reward: [(0, '8267.065'), (1, '9068.263')] [2023-12-27 05:02:11,251][105692] Updated weights for policy 0, policy_version 1872210 (0.0007) [2023-12-27 05:02:11,310][105692] Updated weights for policy 0, policy_version 1872220 (0.0007) [2023-12-27 05:02:11,372][105692] Updated weights for policy 0, policy_version 1872231 (0.0009) [2023-12-27 05:02:11,396][105620] Updated weights for policy 1, policy_version 1876791 (0.0011) [2023-12-27 05:02:11,457][105620] Updated weights for policy 1, policy_version 1876801 (0.0008) [2023-12-27 05:02:11,516][105620] Updated weights for policy 1, policy_version 1876811 (0.0008) [2023-12-27 05:02:12,111][105692] Updated weights for policy 0, policy_version 1872241 (0.0008) [2023-12-27 05:02:12,172][105692] Updated weights for policy 0, policy_version 1872251 (0.0008) [2023-12-27 05:02:12,232][105692] Updated weights for policy 0, policy_version 1872261 (0.0009) [2023-12-27 05:02:12,290][105620] Updated weights for policy 1, policy_version 1876821 (0.0009) [2023-12-27 05:02:12,292][105692] Updated weights for policy 0, policy_version 1872271 (0.0008) [2023-12-27 05:02:12,352][105620] Updated weights for policy 1, policy_version 1876831 (0.0008) [2023-12-27 05:02:12,416][105620] Updated weights for policy 1, policy_version 1876841 (0.0007) [2023-12-27 05:02:13,099][105620] Updated weights for policy 1, policy_version 1876851 (0.0007) [2023-12-27 05:02:13,101][105692] Updated weights for policy 0, policy_version 1872281 (0.0008) [2023-12-27 05:02:13,157][105620] Updated weights for policy 1, policy_version 1876861 (0.0007) [2023-12-27 05:02:13,159][105692] Updated weights for policy 0, policy_version 1872291 (0.0007) [2023-12-27 05:02:13,211][105620] Updated weights for policy 1, policy_version 1876871 (0.0006) [2023-12-27 05:02:13,217][105692] Updated weights for policy 0, policy_version 1872301 (0.0007) [2023-12-27 05:02:13,814][105620] Updated weights for policy 1, policy_version 1876881 (0.0006) [2023-12-27 05:02:13,872][105620] Updated weights for policy 1, policy_version 1876891 (0.0005) [2023-12-27 05:02:13,920][105620] Updated weights for policy 1, policy_version 1876901 (0.0007) [2023-12-27 05:02:13,970][105620] Updated weights for policy 1, policy_version 1876911 (0.0008) [2023-12-27 05:02:14,051][105692] Updated weights for policy 0, policy_version 1872311 (0.0008) [2023-12-27 05:02:14,106][105692] Updated weights for policy 0, policy_version 1872321 (0.0009) [2023-12-27 05:02:14,161][105692] Updated weights for policy 0, policy_version 1872331 (0.0009) [2023-12-27 05:02:14,619][105620] Updated weights for policy 1, policy_version 1876921 (0.0006) [2023-12-27 05:02:14,684][105620] Updated weights for policy 1, policy_version 1876931 (0.0007) [2023-12-27 05:02:14,744][105620] Updated weights for policy 1, policy_version 1876941 (0.0008) [2023-12-27 05:02:15,008][105692] Updated weights for policy 0, policy_version 1872341 (0.0009) [2023-12-27 05:02:15,056][105692] Updated weights for policy 0, policy_version 1872351 (0.0008) [2023-12-27 05:02:15,115][105692] Updated weights for policy 0, policy_version 1872361 (0.0009) [2023-12-27 05:02:15,388][105620] Updated weights for policy 1, policy_version 1876951 (0.0009) [2023-12-27 05:02:15,441][105620] Updated weights for policy 1, policy_version 1876961 (0.0009) [2023-12-27 05:02:15,500][105620] Updated weights for policy 1, policy_version 1876971 (0.0009) [2023-12-27 05:02:15,894][105692] Updated weights for policy 0, policy_version 1872371 (0.0009) [2023-12-27 05:02:15,960][105692] Updated weights for policy 0, policy_version 1872381 (0.0010) [2023-12-27 05:02:16,019][105692] Updated weights for policy 0, policy_version 1872391 (0.0010) [2023-12-27 05:02:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 959971328. Throughput: 0: 9473.1, 1: 10129.9. Samples: 959947848. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:02:16,063][104569] Avg episode reward: [(0, '8631.044'), (1, '8976.024')] [2023-12-27 05:02:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001876976_480575488.pth... [2023-12-27 05:02:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001872400_479404032.pth... [2023-12-27 05:02:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001871280_479117312.pth [2023-12-27 05:02:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001875792_480272384.pth [2023-12-27 05:02:16,174][105620] Updated weights for policy 1, policy_version 1876981 (0.0009) [2023-12-27 05:02:16,237][105620] Updated weights for policy 1, policy_version 1876991 (0.0010) [2023-12-27 05:02:16,292][105620] Updated weights for policy 1, policy_version 1877001 (0.0010) [2023-12-27 05:02:16,728][105692] Updated weights for policy 0, policy_version 1872401 (0.0009) [2023-12-27 05:02:16,778][105692] Updated weights for policy 0, policy_version 1872411 (0.0008) [2023-12-27 05:02:16,829][105692] Updated weights for policy 0, policy_version 1872421 (0.0007) [2023-12-27 05:02:16,887][105692] Updated weights for policy 0, policy_version 1872431 (0.0005) [2023-12-27 05:02:16,909][105620] Updated weights for policy 1, policy_version 1877011 (0.0009) [2023-12-27 05:02:16,958][105620] Updated weights for policy 1, policy_version 1877021 (0.0005) [2023-12-27 05:02:17,012][105620] Updated weights for policy 1, policy_version 1877031 (0.0006) [2023-12-27 05:02:17,463][105692] Updated weights for policy 0, policy_version 1872441 (0.0005) [2023-12-27 05:02:17,513][105692] Updated weights for policy 0, policy_version 1872451 (0.0008) [2023-12-27 05:02:17,565][105692] Updated weights for policy 0, policy_version 1872461 (0.0008) [2023-12-27 05:02:17,628][105620] Updated weights for policy 1, policy_version 1877041 (0.0006) [2023-12-27 05:02:17,688][105620] Updated weights for policy 1, policy_version 1877051 (0.0011) [2023-12-27 05:02:17,747][105620] Updated weights for policy 1, policy_version 1877061 (0.0011) [2023-12-27 05:02:17,806][105620] Updated weights for policy 1, policy_version 1877071 (0.0009) [2023-12-27 05:02:18,171][105692] Updated weights for policy 0, policy_version 1872471 (0.0009) [2023-12-27 05:02:18,220][105692] Updated weights for policy 0, policy_version 1872481 (0.0010) [2023-12-27 05:02:18,271][105692] Updated weights for policy 0, policy_version 1872491 (0.0010) [2023-12-27 05:02:18,385][105620] Updated weights for policy 1, policy_version 1877081 (0.0007) [2023-12-27 05:02:18,445][105620] Updated weights for policy 1, policy_version 1877091 (0.0010) [2023-12-27 05:02:18,503][105620] Updated weights for policy 1, policy_version 1877101 (0.0010) [2023-12-27 05:02:19,017][105692] Updated weights for policy 0, policy_version 1872501 (0.0008) [2023-12-27 05:02:19,073][105692] Updated weights for policy 0, policy_version 1872511 (0.0007) [2023-12-27 05:02:19,090][105620] Updated weights for policy 1, policy_version 1877111 (0.0007) [2023-12-27 05:02:19,125][105692] Updated weights for policy 0, policy_version 1872521 (0.0009) [2023-12-27 05:02:19,144][105620] Updated weights for policy 1, policy_version 1877121 (0.0005) [2023-12-27 05:02:19,206][105620] Updated weights for policy 1, policy_version 1877131 (0.0005) [2023-12-27 05:02:19,768][105692] Updated weights for policy 0, policy_version 1872531 (0.0008) [2023-12-27 05:02:19,818][105692] Updated weights for policy 0, policy_version 1872541 (0.0006) [2023-12-27 05:02:19,855][105620] Updated weights for policy 1, policy_version 1877141 (0.0008) [2023-12-27 05:02:19,877][105692] Updated weights for policy 0, policy_version 1872551 (0.0009) [2023-12-27 05:02:19,914][105620] Updated weights for policy 1, policy_version 1877151 (0.0009) [2023-12-27 05:02:19,977][105620] Updated weights for policy 1, policy_version 1877161 (0.0007) [2023-12-27 05:02:20,558][105692] Updated weights for policy 0, policy_version 1872561 (0.0009) [2023-12-27 05:02:20,570][105620] Updated weights for policy 1, policy_version 1877171 (0.0007) [2023-12-27 05:02:20,631][105692] Updated weights for policy 0, policy_version 1872571 (0.0007) [2023-12-27 05:02:20,639][105620] Updated weights for policy 1, policy_version 1877181 (0.0007) [2023-12-27 05:02:20,696][105692] Updated weights for policy 0, policy_version 1872581 (0.0008) [2023-12-27 05:02:20,698][105620] Updated weights for policy 1, policy_version 1877191 (0.0006) [2023-12-27 05:02:20,762][105692] Updated weights for policy 0, policy_version 1872591 (0.0008) [2023-12-27 05:02:21,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 960086016. Throughput: 0: 9568.6, 1: 10182.3. Samples: 960071448. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:02:21,063][104569] Avg episode reward: [(0, '8537.671'), (1, '9253.044')] [2023-12-27 05:02:21,454][105620] Updated weights for policy 1, policy_version 1877201 (0.0007) [2023-12-27 05:02:21,466][105692] Updated weights for policy 0, policy_version 1872601 (0.0010) [2023-12-27 05:02:21,516][105620] Updated weights for policy 1, policy_version 1877211 (0.0009) [2023-12-27 05:02:21,525][105692] Updated weights for policy 0, policy_version 1872611 (0.0009) [2023-12-27 05:02:21,574][105620] Updated weights for policy 1, policy_version 1877221 (0.0008) [2023-12-27 05:02:21,587][105692] Updated weights for policy 0, policy_version 1872621 (0.0006) [2023-12-27 05:02:21,641][105620] Updated weights for policy 1, policy_version 1877231 (0.0008) [2023-12-27 05:02:22,333][105692] Updated weights for policy 0, policy_version 1872631 (0.0010) [2023-12-27 05:02:22,382][105620] Updated weights for policy 1, policy_version 1877241 (0.0007) [2023-12-27 05:02:22,402][105692] Updated weights for policy 0, policy_version 1872641 (0.0011) [2023-12-27 05:02:22,449][105620] Updated weights for policy 1, policy_version 1877251 (0.0007) [2023-12-27 05:02:22,463][105692] Updated weights for policy 0, policy_version 1872651 (0.0011) [2023-12-27 05:02:22,514][105620] Updated weights for policy 1, policy_version 1877261 (0.0007) [2023-12-27 05:02:23,094][105692] Updated weights for policy 0, policy_version 1872661 (0.0009) [2023-12-27 05:02:23,157][105692] Updated weights for policy 0, policy_version 1872671 (0.0005) [2023-12-27 05:02:23,212][105692] Updated weights for policy 0, policy_version 1872681 (0.0005) [2023-12-27 05:02:23,372][105620] Updated weights for policy 1, policy_version 1877271 (0.0009) [2023-12-27 05:02:23,429][105620] Updated weights for policy 1, policy_version 1877281 (0.0010) [2023-12-27 05:02:23,488][105620] Updated weights for policy 1, policy_version 1877291 (0.0010) [2023-12-27 05:02:23,743][105692] Updated weights for policy 0, policy_version 1872691 (0.0009) [2023-12-27 05:02:23,797][105692] Updated weights for policy 0, policy_version 1872701 (0.0005) [2023-12-27 05:02:23,850][105692] Updated weights for policy 0, policy_version 1872711 (0.0005) [2023-12-27 05:02:24,391][105620] Updated weights for policy 1, policy_version 1877301 (0.0008) [2023-12-27 05:02:24,399][105692] Updated weights for policy 0, policy_version 1872721 (0.0005) [2023-12-27 05:02:24,453][105620] Updated weights for policy 1, policy_version 1877311 (0.0007) [2023-12-27 05:02:24,463][105692] Updated weights for policy 0, policy_version 1872731 (0.0007) [2023-12-27 05:02:24,512][105620] Updated weights for policy 1, policy_version 1877321 (0.0008) [2023-12-27 05:02:24,523][105692] Updated weights for policy 0, policy_version 1872741 (0.0006) [2023-12-27 05:02:24,585][105692] Updated weights for policy 0, policy_version 1872751 (0.0007) [2023-12-27 05:02:25,215][105692] Updated weights for policy 0, policy_version 1872761 (0.0010) [2023-12-27 05:02:25,273][105692] Updated weights for policy 0, policy_version 1872771 (0.0010) [2023-12-27 05:02:25,317][105620] Updated weights for policy 1, policy_version 1877331 (0.0009) [2023-12-27 05:02:25,334][105692] Updated weights for policy 0, policy_version 1872781 (0.0010) [2023-12-27 05:02:25,369][105620] Updated weights for policy 1, policy_version 1877341 (0.0006) [2023-12-27 05:02:25,417][105620] Updated weights for policy 1, policy_version 1877351 (0.0008) [2023-12-27 05:02:25,950][105692] Updated weights for policy 0, policy_version 1872791 (0.0007) [2023-12-27 05:02:26,004][105692] Updated weights for policy 0, policy_version 1872801 (0.0005) [2023-12-27 05:02:26,055][105692] Updated weights for policy 0, policy_version 1872811 (0.0005) [2023-12-27 05:02:26,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.9, 300 sec: 19522.0). Total num frames: 960176128. Throughput: 0: 9687.8, 1: 10034.2. Samples: 960188656. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:02:26,062][104569] Avg episode reward: [(0, '8448.773'), (1, '9069.416')] [2023-12-27 05:02:26,218][105620] Updated weights for policy 1, policy_version 1877361 (0.0007) [2023-12-27 05:02:26,270][105620] Updated weights for policy 1, policy_version 1877371 (0.0008) [2023-12-27 05:02:26,319][105620] Updated weights for policy 1, policy_version 1877381 (0.0009) [2023-12-27 05:02:26,377][105620] Updated weights for policy 1, policy_version 1877391 (0.0010) [2023-12-27 05:02:26,621][105692] Updated weights for policy 0, policy_version 1872821 (0.0005) [2023-12-27 05:02:26,667][105692] Updated weights for policy 0, policy_version 1872831 (0.0006) [2023-12-27 05:02:26,729][105692] Updated weights for policy 0, policy_version 1872841 (0.0011) [2023-12-27 05:02:27,254][105620] Updated weights for policy 1, policy_version 1877401 (0.0008) [2023-12-27 05:02:27,323][105620] Updated weights for policy 1, policy_version 1877411 (0.0008) [2023-12-27 05:02:27,372][105692] Updated weights for policy 0, policy_version 1872851 (0.0009) [2023-12-27 05:02:27,385][105620] Updated weights for policy 1, policy_version 1877421 (0.0007) [2023-12-27 05:02:27,427][105692] Updated weights for policy 0, policy_version 1872861 (0.0005) [2023-12-27 05:02:27,481][105692] Updated weights for policy 0, policy_version 1872871 (0.0005) [2023-12-27 05:02:28,092][105692] Updated weights for policy 0, policy_version 1872881 (0.0008) [2023-12-27 05:02:28,144][105692] Updated weights for policy 0, policy_version 1872891 (0.0011) [2023-12-27 05:02:28,163][105620] Updated weights for policy 1, policy_version 1877431 (0.0006) [2023-12-27 05:02:28,194][105692] Updated weights for policy 0, policy_version 1872901 (0.0011) [2023-12-27 05:02:28,226][105620] Updated weights for policy 1, policy_version 1877441 (0.0008) [2023-12-27 05:02:28,250][105692] Updated weights for policy 0, policy_version 1872911 (0.0006) [2023-12-27 05:02:28,281][105620] Updated weights for policy 1, policy_version 1877451 (0.0009) [2023-12-27 05:02:28,919][105692] Updated weights for policy 0, policy_version 1872921 (0.0010) [2023-12-27 05:02:28,970][105692] Updated weights for policy 0, policy_version 1872931 (0.0010) [2023-12-27 05:02:29,018][105692] Updated weights for policy 0, policy_version 1872941 (0.0010) [2023-12-27 05:02:29,092][105620] Updated weights for policy 1, policy_version 1877461 (0.0008) [2023-12-27 05:02:29,140][105620] Updated weights for policy 1, policy_version 1877471 (0.0008) [2023-12-27 05:02:29,188][105620] Updated weights for policy 1, policy_version 1877481 (0.0008) [2023-12-27 05:02:29,726][105692] Updated weights for policy 0, policy_version 1872951 (0.0010) [2023-12-27 05:02:29,774][105692] Updated weights for policy 0, policy_version 1872961 (0.0010) [2023-12-27 05:02:29,830][105692] Updated weights for policy 0, policy_version 1872971 (0.0010) [2023-12-27 05:02:29,970][105620] Updated weights for policy 1, policy_version 1877491 (0.0008) [2023-12-27 05:02:30,033][105620] Updated weights for policy 1, policy_version 1877501 (0.0008) [2023-12-27 05:02:30,090][105620] Updated weights for policy 1, policy_version 1877511 (0.0009) [2023-12-27 05:02:30,519][105692] Updated weights for policy 0, policy_version 1872981 (0.0009) [2023-12-27 05:02:30,584][105692] Updated weights for policy 0, policy_version 1872991 (0.0005) [2023-12-27 05:02:30,639][105692] Updated weights for policy 0, policy_version 1873001 (0.0007) [2023-12-27 05:02:30,940][105620] Updated weights for policy 1, policy_version 1877522 (0.0010) [2023-12-27 05:02:31,000][105620] Updated weights for policy 1, policy_version 1877532 (0.0009) [2023-12-27 05:02:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 960274432. Throughput: 0: 9792.3, 1: 9975.6. Samples: 960248044. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:02:31,063][104569] Avg episode reward: [(0, '8720.405'), (1, '8976.958')] [2023-12-27 05:02:31,064][105620] Updated weights for policy 1, policy_version 1877542 (0.0009) [2023-12-27 05:02:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001873008_479559680.pth... [2023-12-27 05:02:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001871856_479264768.pth [2023-12-27 05:02:31,126][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001877552_480722944.pth... [2023-12-27 05:02:31,127][105620] Updated weights for policy 1, policy_version 1877552 (0.0008) [2023-12-27 05:02:31,130][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001876400_480428032.pth [2023-12-27 05:02:31,256][105692] Updated weights for policy 0, policy_version 1873011 (0.0007) [2023-12-27 05:02:31,323][105692] Updated weights for policy 0, policy_version 1873021 (0.0011) [2023-12-27 05:02:31,398][105692] Updated weights for policy 0, policy_version 1873031 (0.0009) [2023-12-27 05:02:31,885][105620] Updated weights for policy 1, policy_version 1877562 (0.0005) [2023-12-27 05:02:31,935][105620] Updated weights for policy 1, policy_version 1877572 (0.0008) [2023-12-27 05:02:31,983][105620] Updated weights for policy 1, policy_version 1877582 (0.0007) [2023-12-27 05:02:32,109][105692] Updated weights for policy 0, policy_version 1873041 (0.0009) [2023-12-27 05:02:32,165][105692] Updated weights for policy 0, policy_version 1873051 (0.0011) [2023-12-27 05:02:32,227][105692] Updated weights for policy 0, policy_version 1873061 (0.0011) [2023-12-27 05:02:32,289][105692] Updated weights for policy 0, policy_version 1873071 (0.0011) [2023-12-27 05:02:32,752][105620] Updated weights for policy 1, policy_version 1877592 (0.0008) [2023-12-27 05:02:32,815][105620] Updated weights for policy 1, policy_version 1877602 (0.0008) [2023-12-27 05:02:32,873][105620] Updated weights for policy 1, policy_version 1877612 (0.0008) [2023-12-27 05:02:33,058][105692] Updated weights for policy 0, policy_version 1873081 (0.0010) [2023-12-27 05:02:33,113][105692] Updated weights for policy 0, policy_version 1873091 (0.0010) [2023-12-27 05:02:33,160][105692] Updated weights for policy 0, policy_version 1873101 (0.0010) [2023-12-27 05:02:33,626][105620] Updated weights for policy 1, policy_version 1877622 (0.0008) [2023-12-27 05:02:33,682][105620] Updated weights for policy 1, policy_version 1877632 (0.0008) [2023-12-27 05:02:33,734][105620] Updated weights for policy 1, policy_version 1877642 (0.0008) [2023-12-27 05:02:33,900][105692] Updated weights for policy 0, policy_version 1873111 (0.0010) [2023-12-27 05:02:33,951][105692] Updated weights for policy 0, policy_version 1873121 (0.0010) [2023-12-27 05:02:33,998][105692] Updated weights for policy 0, policy_version 1873131 (0.0010) [2023-12-27 05:02:34,524][105620] Updated weights for policy 1, policy_version 1877652 (0.0008) [2023-12-27 05:02:34,591][105620] Updated weights for policy 1, policy_version 1877662 (0.0005) [2023-12-27 05:02:34,661][105620] Updated weights for policy 1, policy_version 1877672 (0.0006) [2023-12-27 05:02:34,725][105692] Updated weights for policy 0, policy_version 1873141 (0.0010) [2023-12-27 05:02:34,779][105692] Updated weights for policy 0, policy_version 1873151 (0.0010) [2023-12-27 05:02:34,823][105692] Updated weights for policy 0, policy_version 1873161 (0.0010) [2023-12-27 05:02:35,367][105620] Updated weights for policy 1, policy_version 1877682 (0.0008) [2023-12-27 05:02:35,430][105620] Updated weights for policy 1, policy_version 1877692 (0.0008) [2023-12-27 05:02:35,494][105620] Updated weights for policy 1, policy_version 1877702 (0.0008) [2023-12-27 05:02:35,543][105620] Updated weights for policy 1, policy_version 1877712 (0.0006) [2023-12-27 05:02:35,572][105692] Updated weights for policy 0, policy_version 1873171 (0.0011) [2023-12-27 05:02:35,631][105692] Updated weights for policy 0, policy_version 1873181 (0.0010) [2023-12-27 05:02:35,689][105692] Updated weights for policy 0, policy_version 1873191 (0.0010) [2023-12-27 05:02:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.9, 300 sec: 19522.0). Total num frames: 960372736. Throughput: 0: 9791.8, 1: 9835.6. Samples: 960362944. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:02:36,062][104569] Avg episode reward: [(0, '8538.123'), (1, '9252.883')] [2023-12-27 05:02:36,103][105620] Updated weights for policy 1, policy_version 1877722 (0.0006) [2023-12-27 05:02:36,164][105620] Updated weights for policy 1, policy_version 1877732 (0.0009) [2023-12-27 05:02:36,226][105620] Updated weights for policy 1, policy_version 1877742 (0.0009) [2023-12-27 05:02:36,376][105692] Updated weights for policy 0, policy_version 1873201 (0.0010) [2023-12-27 05:02:36,439][105692] Updated weights for policy 0, policy_version 1873211 (0.0010) [2023-12-27 05:02:36,498][105692] Updated weights for policy 0, policy_version 1873221 (0.0011) [2023-12-27 05:02:36,554][105692] Updated weights for policy 0, policy_version 1873231 (0.0010) [2023-12-27 05:02:36,988][105620] Updated weights for policy 1, policy_version 1877752 (0.0010) [2023-12-27 05:02:37,050][105620] Updated weights for policy 1, policy_version 1877762 (0.0009) [2023-12-27 05:02:37,105][105620] Updated weights for policy 1, policy_version 1877772 (0.0009) [2023-12-27 05:02:37,272][105692] Updated weights for policy 0, policy_version 1873241 (0.0009) [2023-12-27 05:02:37,320][105692] Updated weights for policy 0, policy_version 1873251 (0.0008) [2023-12-27 05:02:37,368][105692] Updated weights for policy 0, policy_version 1873261 (0.0009) [2023-12-27 05:02:37,865][105620] Updated weights for policy 1, policy_version 1877782 (0.0009) [2023-12-27 05:02:37,920][105620] Updated weights for policy 1, policy_version 1877792 (0.0009) [2023-12-27 05:02:37,978][105620] Updated weights for policy 1, policy_version 1877802 (0.0008) [2023-12-27 05:02:38,147][105692] Updated weights for policy 0, policy_version 1873271 (0.0009) [2023-12-27 05:02:38,201][105692] Updated weights for policy 0, policy_version 1873281 (0.0009) [2023-12-27 05:02:38,248][105692] Updated weights for policy 0, policy_version 1873291 (0.0009) [2023-12-27 05:02:38,759][105620] Updated weights for policy 1, policy_version 1877812 (0.0008) [2023-12-27 05:02:38,816][105620] Updated weights for policy 1, policy_version 1877822 (0.0007) [2023-12-27 05:02:38,866][105620] Updated weights for policy 1, policy_version 1877832 (0.0008) [2023-12-27 05:02:39,032][105692] Updated weights for policy 0, policy_version 1873301 (0.0007) [2023-12-27 05:02:39,087][105692] Updated weights for policy 0, policy_version 1873311 (0.0005) [2023-12-27 05:02:39,145][105692] Updated weights for policy 0, policy_version 1873321 (0.0005) [2023-12-27 05:02:39,705][105620] Updated weights for policy 1, policy_version 1877842 (0.0007) [2023-12-27 05:02:39,771][105620] Updated weights for policy 1, policy_version 1877852 (0.0009) [2023-12-27 05:02:39,789][105692] Updated weights for policy 0, policy_version 1873331 (0.0007) [2023-12-27 05:02:39,838][105620] Updated weights for policy 1, policy_version 1877862 (0.0008) [2023-12-27 05:02:39,850][105692] Updated weights for policy 0, policy_version 1873341 (0.0008) [2023-12-27 05:02:39,894][105620] Updated weights for policy 1, policy_version 1877872 (0.0007) [2023-12-27 05:02:39,913][105692] Updated weights for policy 0, policy_version 1873351 (0.0008) [2023-12-27 05:02:40,655][105620] Updated weights for policy 1, policy_version 1877882 (0.0009) [2023-12-27 05:02:40,689][105692] Updated weights for policy 0, policy_version 1873361 (0.0008) [2023-12-27 05:02:40,708][105620] Updated weights for policy 1, policy_version 1877892 (0.0008) [2023-12-27 05:02:40,742][105692] Updated weights for policy 0, policy_version 1873371 (0.0007) [2023-12-27 05:02:40,764][105620] Updated weights for policy 1, policy_version 1877902 (0.0007) [2023-12-27 05:02:40,804][105692] Updated weights for policy 0, policy_version 1873381 (0.0007) [2023-12-27 05:02:40,872][105692] Updated weights for policy 0, policy_version 1873391 (0.0009) [2023-12-27 05:02:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 960471040. Throughput: 0: 9743.6, 1: 9754.8. Samples: 960476336. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:02:41,063][104569] Avg episode reward: [(0, '8172.284'), (1, '9252.895')] [2023-12-27 05:02:41,566][105620] Updated weights for policy 1, policy_version 1877912 (0.0008) [2023-12-27 05:02:41,585][105692] Updated weights for policy 0, policy_version 1873401 (0.0007) [2023-12-27 05:02:41,632][105620] Updated weights for policy 1, policy_version 1877922 (0.0009) [2023-12-27 05:02:41,653][105692] Updated weights for policy 0, policy_version 1873411 (0.0008) [2023-12-27 05:02:41,687][105620] Updated weights for policy 1, policy_version 1877932 (0.0006) [2023-12-27 05:02:41,722][105692] Updated weights for policy 0, policy_version 1873421 (0.0007) [2023-12-27 05:02:42,377][105692] Updated weights for policy 0, policy_version 1873431 (0.0010) [2023-12-27 05:02:42,437][105692] Updated weights for policy 0, policy_version 1873441 (0.0011) [2023-12-27 05:02:42,487][105620] Updated weights for policy 1, policy_version 1877942 (0.0007) [2023-12-27 05:02:42,497][105692] Updated weights for policy 0, policy_version 1873451 (0.0011) [2023-12-27 05:02:42,544][105620] Updated weights for policy 1, policy_version 1877952 (0.0006) [2023-12-27 05:02:42,600][105620] Updated weights for policy 1, policy_version 1877962 (0.0008) [2023-12-27 05:02:43,130][105692] Updated weights for policy 0, policy_version 1873461 (0.0008) [2023-12-27 05:02:43,193][105692] Updated weights for policy 0, policy_version 1873471 (0.0005) [2023-12-27 05:02:43,256][105692] Updated weights for policy 0, policy_version 1873481 (0.0005) [2023-12-27 05:02:43,418][105620] Updated weights for policy 1, policy_version 1877972 (0.0009) [2023-12-27 05:02:43,476][105620] Updated weights for policy 1, policy_version 1877982 (0.0010) [2023-12-27 05:02:43,531][105620] Updated weights for policy 1, policy_version 1877992 (0.0010) [2023-12-27 05:02:43,890][105692] Updated weights for policy 0, policy_version 1873491 (0.0007) [2023-12-27 05:02:43,952][105692] Updated weights for policy 0, policy_version 1873501 (0.0011) [2023-12-27 05:02:44,001][105692] Updated weights for policy 0, policy_version 1873511 (0.0010) [2023-12-27 05:02:44,219][105620] Updated weights for policy 1, policy_version 1878002 (0.0008) [2023-12-27 05:02:44,288][105620] Updated weights for policy 1, policy_version 1878012 (0.0009) [2023-12-27 05:02:44,357][105620] Updated weights for policy 1, policy_version 1878022 (0.0010) [2023-12-27 05:02:44,426][105620] Updated weights for policy 1, policy_version 1878032 (0.0010) [2023-12-27 05:02:44,650][105692] Updated weights for policy 0, policy_version 1873521 (0.0011) [2023-12-27 05:02:44,705][105692] Updated weights for policy 0, policy_version 1873531 (0.0010) [2023-12-27 05:02:44,763][105692] Updated weights for policy 0, policy_version 1873541 (0.0010) [2023-12-27 05:02:44,827][105692] Updated weights for policy 0, policy_version 1873551 (0.0011) [2023-12-27 05:02:45,108][105620] Updated weights for policy 1, policy_version 1878042 (0.0008) [2023-12-27 05:02:45,161][105620] Updated weights for policy 1, policy_version 1878052 (0.0006) [2023-12-27 05:02:45,218][105620] Updated weights for policy 1, policy_version 1878062 (0.0008) [2023-12-27 05:02:45,533][105692] Updated weights for policy 0, policy_version 1873561 (0.0011) [2023-12-27 05:02:45,581][105692] Updated weights for policy 0, policy_version 1873571 (0.0010) [2023-12-27 05:02:45,640][105692] Updated weights for policy 0, policy_version 1873581 (0.0010) [2023-12-27 05:02:45,978][105620] Updated weights for policy 1, policy_version 1878072 (0.0006) [2023-12-27 05:02:46,042][105620] Updated weights for policy 1, policy_version 1878082 (0.0006) [2023-12-27 05:02:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.4, 300 sec: 19494.2). Total num frames: 960561152. Throughput: 0: 9839.8, 1: 9686.6. Samples: 960534792. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:02:46,063][104569] Avg episode reward: [(0, '8260.377'), (1, '9253.103')] [2023-12-27 05:02:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001873584_479707136.pth... [2023-12-27 05:02:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001872400_479404032.pth [2023-12-27 05:02:46,104][105620] Updated weights for policy 1, policy_version 1878092 (0.0010) [2023-12-27 05:02:46,127][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001878096_480862208.pth... [2023-12-27 05:02:46,131][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001876976_480575488.pth [2023-12-27 05:02:46,317][105692] Updated weights for policy 0, policy_version 1873591 (0.0010) [2023-12-27 05:02:46,373][105692] Updated weights for policy 0, policy_version 1873601 (0.0010) [2023-12-27 05:02:46,427][105692] Updated weights for policy 0, policy_version 1873611 (0.0010) [2023-12-27 05:02:46,791][105620] Updated weights for policy 1, policy_version 1878102 (0.0010) [2023-12-27 05:02:46,843][105620] Updated weights for policy 1, policy_version 1878112 (0.0010) [2023-12-27 05:02:46,902][105620] Updated weights for policy 1, policy_version 1878122 (0.0010) [2023-12-27 05:02:47,163][105692] Updated weights for policy 0, policy_version 1873621 (0.0010) [2023-12-27 05:02:47,214][105692] Updated weights for policy 0, policy_version 1873631 (0.0010) [2023-12-27 05:02:47,278][105692] Updated weights for policy 0, policy_version 1873641 (0.0007) [2023-12-27 05:02:47,508][105620] Updated weights for policy 1, policy_version 1878132 (0.0008) [2023-12-27 05:02:47,552][105620] Updated weights for policy 1, policy_version 1878142 (0.0005) [2023-12-27 05:02:47,598][105620] Updated weights for policy 1, policy_version 1878152 (0.0005) [2023-12-27 05:02:47,951][105692] Updated weights for policy 0, policy_version 1873651 (0.0008) [2023-12-27 05:02:48,003][105692] Updated weights for policy 0, policy_version 1873661 (0.0010) [2023-12-27 05:02:48,060][105692] Updated weights for policy 0, policy_version 1873671 (0.0008) [2023-12-27 05:02:48,344][105620] Updated weights for policy 1, policy_version 1878162 (0.0008) [2023-12-27 05:02:48,399][105620] Updated weights for policy 1, policy_version 1878172 (0.0008) [2023-12-27 05:02:48,451][105620] Updated weights for policy 1, policy_version 1878182 (0.0008) [2023-12-27 05:02:48,512][105620] Updated weights for policy 1, policy_version 1878192 (0.0007) [2023-12-27 05:02:48,742][105692] Updated weights for policy 0, policy_version 1873681 (0.0005) [2023-12-27 05:02:48,793][105692] Updated weights for policy 0, policy_version 1873691 (0.0005) [2023-12-27 05:02:48,856][105692] Updated weights for policy 0, policy_version 1873701 (0.0009) [2023-12-27 05:02:48,908][105692] Updated weights for policy 0, policy_version 1873711 (0.0010) [2023-12-27 05:02:49,240][105620] Updated weights for policy 1, policy_version 1878202 (0.0005) [2023-12-27 05:02:49,307][105620] Updated weights for policy 1, policy_version 1878212 (0.0008) [2023-12-27 05:02:49,373][105620] Updated weights for policy 1, policy_version 1878222 (0.0008) [2023-12-27 05:02:49,640][105692] Updated weights for policy 0, policy_version 1873721 (0.0006) [2023-12-27 05:02:49,699][105692] Updated weights for policy 0, policy_version 1873731 (0.0005) [2023-12-27 05:02:49,764][105692] Updated weights for policy 0, policy_version 1873741 (0.0005) [2023-12-27 05:02:50,081][105620] Updated weights for policy 1, policy_version 1878232 (0.0006) [2023-12-27 05:02:50,132][105620] Updated weights for policy 1, policy_version 1878242 (0.0006) [2023-12-27 05:02:50,180][105620] Updated weights for policy 1, policy_version 1878252 (0.0005) [2023-12-27 05:02:50,392][105692] Updated weights for policy 0, policy_version 1873751 (0.0007) [2023-12-27 05:02:50,457][105692] Updated weights for policy 0, policy_version 1873761 (0.0008) [2023-12-27 05:02:50,524][105692] Updated weights for policy 0, policy_version 1873771 (0.0008) [2023-12-27 05:02:50,831][105620] Updated weights for policy 1, policy_version 1878262 (0.0006) [2023-12-27 05:02:50,886][105620] Updated weights for policy 1, policy_version 1878272 (0.0005) [2023-12-27 05:02:50,940][105620] Updated weights for policy 1, policy_version 1878282 (0.0005) [2023-12-27 05:02:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 960667648. Throughput: 0: 9923.2, 1: 9635.6. Samples: 960653360. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:02:51,062][104569] Avg episode reward: [(0, '8265.352'), (1, '9070.032')] [2023-12-27 05:02:51,278][105692] Updated weights for policy 0, policy_version 1873781 (0.0008) [2023-12-27 05:02:51,339][105692] Updated weights for policy 0, policy_version 1873791 (0.0008) [2023-12-27 05:02:51,410][105692] Updated weights for policy 0, policy_version 1873801 (0.0009) [2023-12-27 05:02:51,524][105620] Updated weights for policy 1, policy_version 1878292 (0.0008) [2023-12-27 05:02:51,586][105620] Updated weights for policy 1, policy_version 1878302 (0.0009) [2023-12-27 05:02:51,655][105620] Updated weights for policy 1, policy_version 1878312 (0.0009) [2023-12-27 05:02:52,107][105692] Updated weights for policy 0, policy_version 1873811 (0.0010) [2023-12-27 05:02:52,168][105692] Updated weights for policy 0, policy_version 1873821 (0.0009) [2023-12-27 05:02:52,231][105692] Updated weights for policy 0, policy_version 1873831 (0.0008) [2023-12-27 05:02:52,446][105620] Updated weights for policy 1, policy_version 1878322 (0.0009) [2023-12-27 05:02:52,501][105620] Updated weights for policy 1, policy_version 1878332 (0.0009) [2023-12-27 05:02:52,560][105620] Updated weights for policy 1, policy_version 1878342 (0.0009) [2023-12-27 05:02:52,623][105620] Updated weights for policy 1, policy_version 1878352 (0.0009) [2023-12-27 05:02:52,966][105692] Updated weights for policy 0, policy_version 1873841 (0.0008) [2023-12-27 05:02:53,028][105692] Updated weights for policy 0, policy_version 1873851 (0.0009) [2023-12-27 05:02:53,086][105692] Updated weights for policy 0, policy_version 1873861 (0.0008) [2023-12-27 05:02:53,133][105692] Updated weights for policy 0, policy_version 1873871 (0.0009) [2023-12-27 05:02:53,361][105620] Updated weights for policy 1, policy_version 1878362 (0.0009) [2023-12-27 05:02:53,428][105620] Updated weights for policy 1, policy_version 1878372 (0.0009) [2023-12-27 05:02:53,489][105620] Updated weights for policy 1, policy_version 1878382 (0.0009) [2023-12-27 05:02:53,947][105692] Updated weights for policy 0, policy_version 1873881 (0.0010) [2023-12-27 05:02:54,000][105692] Updated weights for policy 0, policy_version 1873891 (0.0009) [2023-12-27 05:02:54,054][105692] Updated weights for policy 0, policy_version 1873901 (0.0010) [2023-12-27 05:02:54,077][105620] Updated weights for policy 1, policy_version 1878392 (0.0006) [2023-12-27 05:02:54,127][105620] Updated weights for policy 1, policy_version 1878402 (0.0009) [2023-12-27 05:02:54,190][105620] Updated weights for policy 1, policy_version 1878412 (0.0009) [2023-12-27 05:02:54,820][105692] Updated weights for policy 0, policy_version 1873911 (0.0009) [2023-12-27 05:02:54,870][105692] Updated weights for policy 0, policy_version 1873921 (0.0009) [2023-12-27 05:02:54,899][105620] Updated weights for policy 1, policy_version 1878422 (0.0008) [2023-12-27 05:02:54,931][105692] Updated weights for policy 0, policy_version 1873931 (0.0008) [2023-12-27 05:02:54,954][105620] Updated weights for policy 1, policy_version 1878432 (0.0007) [2023-12-27 05:02:55,025][105620] Updated weights for policy 1, policy_version 1878442 (0.0009) [2023-12-27 05:02:55,708][105692] Updated weights for policy 0, policy_version 1873941 (0.0008) [2023-12-27 05:02:55,745][105620] Updated weights for policy 1, policy_version 1878452 (0.0010) [2023-12-27 05:02:55,755][105692] Updated weights for policy 0, policy_version 1873951 (0.0007) [2023-12-27 05:02:55,794][105620] Updated weights for policy 1, policy_version 1878462 (0.0010) [2023-12-27 05:02:55,808][105692] Updated weights for policy 0, policy_version 1873961 (0.0006) [2023-12-27 05:02:55,842][105620] Updated weights for policy 1, policy_version 1878472 (0.0010) [2023-12-27 05:02:56,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 960765952. Throughput: 0: 9978.5, 1: 9597.8. Samples: 960770524. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:02:56,064][104569] Avg episode reward: [(0, '7716.118'), (1, '9070.020')] [2023-12-27 05:02:56,578][105692] Updated weights for policy 0, policy_version 1873971 (0.0007) [2023-12-27 05:02:56,615][105620] Updated weights for policy 1, policy_version 1878482 (0.0010) [2023-12-27 05:02:56,631][105692] Updated weights for policy 0, policy_version 1873981 (0.0008) [2023-12-27 05:02:56,668][105620] Updated weights for policy 1, policy_version 1878492 (0.0010) [2023-12-27 05:02:56,691][105692] Updated weights for policy 0, policy_version 1873991 (0.0005) [2023-12-27 05:02:56,720][105620] Updated weights for policy 1, policy_version 1878502 (0.0010) [2023-12-27 05:02:56,778][105620] Updated weights for policy 1, policy_version 1878512 (0.0010) [2023-12-27 05:02:57,446][105692] Updated weights for policy 0, policy_version 1874001 (0.0006) [2023-12-27 05:02:57,492][105620] Updated weights for policy 1, policy_version 1878522 (0.0005) [2023-12-27 05:02:57,499][105692] Updated weights for policy 0, policy_version 1874011 (0.0008) [2023-12-27 05:02:57,543][105620] Updated weights for policy 1, policy_version 1878532 (0.0007) [2023-12-27 05:02:57,553][105692] Updated weights for policy 0, policy_version 1874021 (0.0007) [2023-12-27 05:02:57,590][105620] Updated weights for policy 1, policy_version 1878542 (0.0006) [2023-12-27 05:02:57,600][105692] Updated weights for policy 0, policy_version 1874031 (0.0008) [2023-12-27 05:02:58,153][105620] Updated weights for policy 1, policy_version 1878552 (0.0006) [2023-12-27 05:02:58,222][105620] Updated weights for policy 1, policy_version 1878562 (0.0008) [2023-12-27 05:02:58,290][105620] Updated weights for policy 1, policy_version 1878572 (0.0008) [2023-12-27 05:02:58,478][105692] Updated weights for policy 0, policy_version 1874041 (0.0009) [2023-12-27 05:02:58,540][105692] Updated weights for policy 0, policy_version 1874051 (0.0009) [2023-12-27 05:02:58,603][105692] Updated weights for policy 0, policy_version 1874061 (0.0008) [2023-12-27 05:02:59,105][105620] Updated weights for policy 1, policy_version 1878582 (0.0009) [2023-12-27 05:02:59,171][105620] Updated weights for policy 1, policy_version 1878592 (0.0009) [2023-12-27 05:02:59,231][105620] Updated weights for policy 1, policy_version 1878602 (0.0009) [2023-12-27 05:02:59,426][105692] Updated weights for policy 0, policy_version 1874071 (0.0009) [2023-12-27 05:02:59,485][105692] Updated weights for policy 0, policy_version 1874081 (0.0009) [2023-12-27 05:02:59,542][105692] Updated weights for policy 0, policy_version 1874091 (0.0009) [2023-12-27 05:02:59,948][105620] Updated weights for policy 1, policy_version 1878612 (0.0009) [2023-12-27 05:03:00,006][105620] Updated weights for policy 1, policy_version 1878622 (0.0008) [2023-12-27 05:03:00,063][105620] Updated weights for policy 1, policy_version 1878632 (0.0009) [2023-12-27 05:03:00,296][105692] Updated weights for policy 0, policy_version 1874101 (0.0007) [2023-12-27 05:03:00,362][105692] Updated weights for policy 0, policy_version 1874111 (0.0005) [2023-12-27 05:03:00,414][105692] Updated weights for policy 0, policy_version 1874121 (0.0005) [2023-12-27 05:03:00,915][105620] Updated weights for policy 1, policy_version 1878642 (0.0009) [2023-12-27 05:03:00,965][105692] Updated weights for policy 0, policy_version 1874131 (0.0005) [2023-12-27 05:03:00,968][105620] Updated weights for policy 1, policy_version 1878652 (0.0009) [2023-12-27 05:03:01,014][105692] Updated weights for policy 0, policy_version 1874141 (0.0008) [2023-12-27 05:03:01,015][105620] Updated weights for policy 1, policy_version 1878662 (0.0007) [2023-12-27 05:03:01,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 960847872. Throughput: 0: 9949.5, 1: 9581.7. Samples: 960826752. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:03:01,062][104569] Avg episode reward: [(0, '8443.678'), (1, '9160.827')] [2023-12-27 05:03:01,075][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001878672_481009664.pth... [2023-12-27 05:03:01,077][105620] Updated weights for policy 1, policy_version 1878672 (0.0006) [2023-12-27 05:03:01,079][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001877552_480722944.pth [2023-12-27 05:03:01,080][105692] Updated weights for policy 0, policy_version 1874151 (0.0009) [2023-12-27 05:03:01,142][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001874160_479854592.pth... [2023-12-27 05:03:01,147][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001873008_479559680.pth [2023-12-27 05:03:01,812][105692] Updated weights for policy 0, policy_version 1874161 (0.0007) [2023-12-27 05:03:01,868][105620] Updated weights for policy 1, policy_version 1878682 (0.0010) [2023-12-27 05:03:01,875][105692] Updated weights for policy 0, policy_version 1874171 (0.0006) [2023-12-27 05:03:01,925][105620] Updated weights for policy 1, policy_version 1878692 (0.0009) [2023-12-27 05:03:01,930][105692] Updated weights for policy 0, policy_version 1874181 (0.0005) [2023-12-27 05:03:01,974][105620] Updated weights for policy 1, policy_version 1878702 (0.0008) [2023-12-27 05:03:01,988][105692] Updated weights for policy 0, policy_version 1874191 (0.0007) [2023-12-27 05:03:02,651][105692] Updated weights for policy 0, policy_version 1874201 (0.0008) [2023-12-27 05:03:02,716][105692] Updated weights for policy 0, policy_version 1874211 (0.0007) [2023-12-27 05:03:02,719][105620] Updated weights for policy 1, policy_version 1878712 (0.0007) [2023-12-27 05:03:02,773][105620] Updated weights for policy 1, policy_version 1878722 (0.0007) [2023-12-27 05:03:02,775][105692] Updated weights for policy 0, policy_version 1874221 (0.0007) [2023-12-27 05:03:02,826][105620] Updated weights for policy 1, policy_version 1878732 (0.0005) [2023-12-27 05:03:03,472][105692] Updated weights for policy 0, policy_version 1874231 (0.0008) [2023-12-27 05:03:03,535][105692] Updated weights for policy 0, policy_version 1874241 (0.0008) [2023-12-27 05:03:03,549][105620] Updated weights for policy 1, policy_version 1878742 (0.0007) [2023-12-27 05:03:03,593][105692] Updated weights for policy 0, policy_version 1874251 (0.0008) [2023-12-27 05:03:03,597][105620] Updated weights for policy 1, policy_version 1878752 (0.0008) [2023-12-27 05:03:03,650][105620] Updated weights for policy 1, policy_version 1878762 (0.0007) [2023-12-27 05:03:04,334][105692] Updated weights for policy 0, policy_version 1874261 (0.0010) [2023-12-27 05:03:04,390][105692] Updated weights for policy 0, policy_version 1874271 (0.0009) [2023-12-27 05:03:04,400][105620] Updated weights for policy 1, policy_version 1878772 (0.0008) [2023-12-27 05:03:04,447][105692] Updated weights for policy 0, policy_version 1874281 (0.0007) [2023-12-27 05:03:04,461][105620] Updated weights for policy 1, policy_version 1878782 (0.0008) [2023-12-27 05:03:04,522][105620] Updated weights for policy 1, policy_version 1878792 (0.0009) [2023-12-27 05:03:05,219][105692] Updated weights for policy 0, policy_version 1874291 (0.0008) [2023-12-27 05:03:05,274][105692] Updated weights for policy 0, policy_version 1874301 (0.0010) [2023-12-27 05:03:05,302][105620] Updated weights for policy 1, policy_version 1878802 (0.0008) [2023-12-27 05:03:05,335][105692] Updated weights for policy 0, policy_version 1874311 (0.0010) [2023-12-27 05:03:05,353][105620] Updated weights for policy 1, policy_version 1878812 (0.0007) [2023-12-27 05:03:05,402][105620] Updated weights for policy 1, policy_version 1878822 (0.0008) [2023-12-27 05:03:05,455][105620] Updated weights for policy 1, policy_version 1878832 (0.0005) [2023-12-27 05:03:05,978][105692] Updated weights for policy 0, policy_version 1874321 (0.0005) [2023-12-27 05:03:06,022][105620] Updated weights for policy 1, policy_version 1878842 (0.0007) [2023-12-27 05:03:06,062][104569] Fps is (10 sec: 18022.9, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 960946176. Throughput: 0: 9930.1, 1: 9374.7. Samples: 960940164. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:03:06,063][104569] Avg episode reward: [(0, '8625.186'), (1, '9161.426')] [2023-12-27 05:03:06,069][105692] Updated weights for policy 0, policy_version 1874331 (0.0009) [2023-12-27 05:03:06,072][105620] Updated weights for policy 1, policy_version 1878852 (0.0006) [2023-12-27 05:03:06,126][105692] Updated weights for policy 0, policy_version 1874341 (0.0011) [2023-12-27 05:03:06,133][105620] Updated weights for policy 1, policy_version 1878862 (0.0007) [2023-12-27 05:03:06,186][105692] Updated weights for policy 0, policy_version 1874351 (0.0011) [2023-12-27 05:03:06,791][105620] Updated weights for policy 1, policy_version 1878872 (0.0008) [2023-12-27 05:03:06,859][105620] Updated weights for policy 1, policy_version 1878882 (0.0008) [2023-12-27 05:03:06,915][105692] Updated weights for policy 0, policy_version 1874361 (0.0011) [2023-12-27 05:03:06,923][105620] Updated weights for policy 1, policy_version 1878892 (0.0010) [2023-12-27 05:03:06,978][105692] Updated weights for policy 0, policy_version 1874371 (0.0011) [2023-12-27 05:03:07,043][105692] Updated weights for policy 0, policy_version 1874381 (0.0010) [2023-12-27 05:03:07,664][105620] Updated weights for policy 1, policy_version 1878902 (0.0007) [2023-12-27 05:03:07,665][105692] Updated weights for policy 0, policy_version 1874391 (0.0007) [2023-12-27 05:03:07,719][105620] Updated weights for policy 1, policy_version 1878912 (0.0005) [2023-12-27 05:03:07,724][105692] Updated weights for policy 0, policy_version 1874401 (0.0008) [2023-12-27 05:03:07,773][105620] Updated weights for policy 1, policy_version 1878922 (0.0005) [2023-12-27 05:03:07,780][105692] Updated weights for policy 0, policy_version 1874411 (0.0009) [2023-12-27 05:03:08,311][105620] Updated weights for policy 1, policy_version 1878932 (0.0005) [2023-12-27 05:03:08,381][105620] Updated weights for policy 1, policy_version 1878942 (0.0008) [2023-12-27 05:03:08,443][105620] Updated weights for policy 1, policy_version 1878952 (0.0008) [2023-12-27 05:03:08,582][105692] Updated weights for policy 0, policy_version 1874421 (0.0009) [2023-12-27 05:03:08,649][105692] Updated weights for policy 0, policy_version 1874431 (0.0008) [2023-12-27 05:03:08,716][105692] Updated weights for policy 0, policy_version 1874441 (0.0009) [2023-12-27 05:03:09,115][105620] Updated weights for policy 1, policy_version 1878962 (0.0007) [2023-12-27 05:03:09,179][105620] Updated weights for policy 1, policy_version 1878972 (0.0010) [2023-12-27 05:03:09,246][105620] Updated weights for policy 1, policy_version 1878983 (0.0009) [2023-12-27 05:03:09,481][105692] Updated weights for policy 0, policy_version 1874451 (0.0008) [2023-12-27 05:03:09,543][105692] Updated weights for policy 0, policy_version 1874461 (0.0007) [2023-12-27 05:03:09,596][105692] Updated weights for policy 0, policy_version 1874471 (0.0009) [2023-12-27 05:03:09,942][105620] Updated weights for policy 1, policy_version 1878993 (0.0009) [2023-12-27 05:03:10,002][105620] Updated weights for policy 1, policy_version 1879003 (0.0006) [2023-12-27 05:03:10,056][105620] Updated weights for policy 1, policy_version 1879013 (0.0006) [2023-12-27 05:03:10,114][105620] Updated weights for policy 1, policy_version 1879023 (0.0008) [2023-12-27 05:03:10,336][105692] Updated weights for policy 0, policy_version 1874482 (0.0009) [2023-12-27 05:03:10,394][105692] Updated weights for policy 0, policy_version 1874492 (0.0006) [2023-12-27 05:03:10,461][105692] Updated weights for policy 0, policy_version 1874502 (0.0007) [2023-12-27 05:03:10,521][105692] Updated weights for policy 0, policy_version 1874512 (0.0008) [2023-12-27 05:03:10,801][105620] Updated weights for policy 1, policy_version 1879033 (0.0010) [2023-12-27 05:03:10,852][105620] Updated weights for policy 1, policy_version 1879043 (0.0006) [2023-12-27 05:03:10,902][105620] Updated weights for policy 1, policy_version 1879053 (0.0005) [2023-12-27 05:03:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 961052672. Throughput: 0: 9813.0, 1: 9560.5. Samples: 961060464. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:03:11,062][104569] Avg episode reward: [(0, '8168.857'), (1, '9164.267')] [2023-12-27 05:03:11,195][105692] Updated weights for policy 0, policy_version 1874522 (0.0009) [2023-12-27 05:03:11,261][105692] Updated weights for policy 0, policy_version 1874532 (0.0009) [2023-12-27 05:03:11,325][105692] Updated weights for policy 0, policy_version 1874542 (0.0009) [2023-12-27 05:03:11,675][105620] Updated weights for policy 1, policy_version 1879063 (0.0008) [2023-12-27 05:03:11,749][105620] Updated weights for policy 1, policy_version 1879073 (0.0008) [2023-12-27 05:03:11,811][105620] Updated weights for policy 1, policy_version 1879083 (0.0010) [2023-12-27 05:03:12,135][105692] Updated weights for policy 0, policy_version 1874552 (0.0008) [2023-12-27 05:03:12,198][105692] Updated weights for policy 0, policy_version 1874562 (0.0007) [2023-12-27 05:03:12,262][105692] Updated weights for policy 0, policy_version 1874572 (0.0007) [2023-12-27 05:03:12,544][105620] Updated weights for policy 1, policy_version 1879093 (0.0011) [2023-12-27 05:03:12,596][105620] Updated weights for policy 1, policy_version 1879103 (0.0010) [2023-12-27 05:03:12,660][105620] Updated weights for policy 1, policy_version 1879113 (0.0010) [2023-12-27 05:03:12,919][105692] Updated weights for policy 0, policy_version 1874582 (0.0007) [2023-12-27 05:03:12,982][105692] Updated weights for policy 0, policy_version 1874592 (0.0005) [2023-12-27 05:03:13,033][105692] Updated weights for policy 0, policy_version 1874602 (0.0010) [2023-12-27 05:03:13,394][105620] Updated weights for policy 1, policy_version 1879123 (0.0009) [2023-12-27 05:03:13,442][105620] Updated weights for policy 1, policy_version 1879133 (0.0010) [2023-12-27 05:03:13,486][105620] Updated weights for policy 1, policy_version 1879143 (0.0010) [2023-12-27 05:03:13,607][105692] Updated weights for policy 0, policy_version 1874612 (0.0009) [2023-12-27 05:03:13,676][105692] Updated weights for policy 0, policy_version 1874622 (0.0008) [2023-12-27 05:03:13,729][105692] Updated weights for policy 0, policy_version 1874632 (0.0006) [2023-12-27 05:03:14,097][105620] Updated weights for policy 1, policy_version 1879153 (0.0010) [2023-12-27 05:03:14,164][105620] Updated weights for policy 1, policy_version 1879163 (0.0010) [2023-12-27 05:03:14,218][105620] Updated weights for policy 1, policy_version 1879173 (0.0009) [2023-12-27 05:03:14,277][105620] Updated weights for policy 1, policy_version 1879183 (0.0009) [2023-12-27 05:03:14,408][105692] Updated weights for policy 0, policy_version 1874642 (0.0006) [2023-12-27 05:03:14,466][105692] Updated weights for policy 0, policy_version 1874652 (0.0009) [2023-12-27 05:03:14,524][105692] Updated weights for policy 0, policy_version 1874662 (0.0008) [2023-12-27 05:03:14,576][105692] Updated weights for policy 0, policy_version 1874672 (0.0010) [2023-12-27 05:03:14,966][105620] Updated weights for policy 1, policy_version 1879193 (0.0008) [2023-12-27 05:03:15,032][105620] Updated weights for policy 1, policy_version 1879203 (0.0007) [2023-12-27 05:03:15,092][105620] Updated weights for policy 1, policy_version 1879213 (0.0009) [2023-12-27 05:03:15,394][105692] Updated weights for policy 0, policy_version 1874682 (0.0008) [2023-12-27 05:03:15,452][105692] Updated weights for policy 0, policy_version 1874692 (0.0005) [2023-12-27 05:03:15,512][105692] Updated weights for policy 0, policy_version 1874702 (0.0007) [2023-12-27 05:03:15,820][105620] Updated weights for policy 1, policy_version 1879223 (0.0009) [2023-12-27 05:03:15,876][105620] Updated weights for policy 1, policy_version 1879233 (0.0009) [2023-12-27 05:03:15,934][105620] Updated weights for policy 1, policy_version 1879243 (0.0009) [2023-12-27 05:03:16,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 961150976. Throughput: 0: 9736.6, 1: 9639.7. Samples: 961119980. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:03:16,063][104569] Avg episode reward: [(0, '8170.837'), (1, '9255.965')] [2023-12-27 05:03:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001874704_479993856.pth... [2023-12-27 05:03:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001879248_481157120.pth... [2023-12-27 05:03:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001873584_479707136.pth [2023-12-27 05:03:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001878096_480862208.pth [2023-12-27 05:03:16,191][105692] Updated weights for policy 0, policy_version 1874712 (0.0007) [2023-12-27 05:03:16,259][105692] Updated weights for policy 0, policy_version 1874722 (0.0005) [2023-12-27 05:03:16,327][105692] Updated weights for policy 0, policy_version 1874732 (0.0005) [2023-12-27 05:03:16,657][105620] Updated weights for policy 1, policy_version 1879253 (0.0007) [2023-12-27 05:03:16,728][105620] Updated weights for policy 1, policy_version 1879263 (0.0005) [2023-12-27 05:03:16,792][105620] Updated weights for policy 1, policy_version 1879273 (0.0005) [2023-12-27 05:03:16,971][105692] Updated weights for policy 0, policy_version 1874742 (0.0008) [2023-12-27 05:03:17,032][105692] Updated weights for policy 0, policy_version 1874752 (0.0010) [2023-12-27 05:03:17,093][105692] Updated weights for policy 0, policy_version 1874762 (0.0008) [2023-12-27 05:03:17,321][105620] Updated weights for policy 1, policy_version 1879283 (0.0006) [2023-12-27 05:03:17,378][105620] Updated weights for policy 1, policy_version 1879293 (0.0008) [2023-12-27 05:03:17,437][105620] Updated weights for policy 1, policy_version 1879304 (0.0009) [2023-12-27 05:03:17,688][105692] Updated weights for policy 0, policy_version 1874772 (0.0007) [2023-12-27 05:03:17,746][105692] Updated weights for policy 0, policy_version 1874782 (0.0010) [2023-12-27 05:03:17,807][105692] Updated weights for policy 0, policy_version 1874792 (0.0006) [2023-12-27 05:03:18,175][105620] Updated weights for policy 1, policy_version 1879314 (0.0006) [2023-12-27 05:03:18,233][105620] Updated weights for policy 1, policy_version 1879324 (0.0008) [2023-12-27 05:03:18,291][105620] Updated weights for policy 1, policy_version 1879334 (0.0005) [2023-12-27 05:03:18,357][105620] Updated weights for policy 1, policy_version 1879344 (0.0006) [2023-12-27 05:03:18,441][105692] Updated weights for policy 0, policy_version 1874802 (0.0006) [2023-12-27 05:03:18,499][105692] Updated weights for policy 0, policy_version 1874812 (0.0008) [2023-12-27 05:03:18,554][105692] Updated weights for policy 0, policy_version 1874822 (0.0008) [2023-12-27 05:03:18,605][105692] Updated weights for policy 0, policy_version 1874832 (0.0005) [2023-12-27 05:03:19,132][105620] Updated weights for policy 1, policy_version 1879354 (0.0006) [2023-12-27 05:03:19,186][105620] Updated weights for policy 1, policy_version 1879364 (0.0005) [2023-12-27 05:03:19,211][105692] Updated weights for policy 0, policy_version 1874842 (0.0010) [2023-12-27 05:03:19,241][105620] Updated weights for policy 1, policy_version 1879374 (0.0008) [2023-12-27 05:03:19,279][105692] Updated weights for policy 0, policy_version 1874852 (0.0007) [2023-12-27 05:03:19,343][105692] Updated weights for policy 0, policy_version 1874862 (0.0008) [2023-12-27 05:03:19,976][105620] Updated weights for policy 1, policy_version 1879384 (0.0008) [2023-12-27 05:03:20,009][105692] Updated weights for policy 0, policy_version 1874872 (0.0010) [2023-12-27 05:03:20,040][105620] Updated weights for policy 1, policy_version 1879394 (0.0006) [2023-12-27 05:03:20,072][105692] Updated weights for policy 0, policy_version 1874882 (0.0010) [2023-12-27 05:03:20,101][105620] Updated weights for policy 1, policy_version 1879404 (0.0007) [2023-12-27 05:03:20,129][105692] Updated weights for policy 0, policy_version 1874892 (0.0011) [2023-12-27 05:03:20,798][105692] Updated weights for policy 0, policy_version 1874902 (0.0008) [2023-12-27 05:03:20,803][105620] Updated weights for policy 1, policy_version 1879414 (0.0006) [2023-12-27 05:03:20,856][105692] Updated weights for policy 0, policy_version 1874912 (0.0007) [2023-12-27 05:03:20,871][105620] Updated weights for policy 1, policy_version 1879424 (0.0008) [2023-12-27 05:03:20,915][105692] Updated weights for policy 0, policy_version 1874922 (0.0008) [2023-12-27 05:03:20,934][105620] Updated weights for policy 1, policy_version 1879434 (0.0006) [2023-12-27 05:03:21,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 961257472. Throughput: 0: 9784.3, 1: 9730.7. Samples: 961241124. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:03:21,062][104569] Avg episode reward: [(0, '8084.576'), (1, '9252.998')] [2023-12-27 05:03:21,668][105692] Updated weights for policy 0, policy_version 1874932 (0.0008) [2023-12-27 05:03:21,679][105620] Updated weights for policy 1, policy_version 1879444 (0.0007) [2023-12-27 05:03:21,729][105692] Updated weights for policy 0, policy_version 1874942 (0.0009) [2023-12-27 05:03:21,746][105620] Updated weights for policy 1, policy_version 1879454 (0.0007) [2023-12-27 05:03:21,790][105692] Updated weights for policy 0, policy_version 1874952 (0.0008) [2023-12-27 05:03:21,807][105620] Updated weights for policy 1, policy_version 1879464 (0.0007) [2023-12-27 05:03:22,560][105692] Updated weights for policy 0, policy_version 1874962 (0.0008) [2023-12-27 05:03:22,572][105620] Updated weights for policy 1, policy_version 1879474 (0.0008) [2023-12-27 05:03:22,621][105692] Updated weights for policy 0, policy_version 1874972 (0.0007) [2023-12-27 05:03:22,638][105620] Updated weights for policy 1, policy_version 1879484 (0.0008) [2023-12-27 05:03:22,689][105692] Updated weights for policy 0, policy_version 1874982 (0.0007) [2023-12-27 05:03:22,691][105620] Updated weights for policy 1, policy_version 1879494 (0.0007) [2023-12-27 05:03:22,748][105692] Updated weights for policy 0, policy_version 1874992 (0.0011) [2023-12-27 05:03:22,754][105620] Updated weights for policy 1, policy_version 1879504 (0.0005) [2023-12-27 05:03:23,347][105692] Updated weights for policy 0, policy_version 1875002 (0.0009) [2023-12-27 05:03:23,412][105692] Updated weights for policy 0, policy_version 1875012 (0.0007) [2023-12-27 05:03:23,461][105692] Updated weights for policy 0, policy_version 1875022 (0.0005) [2023-12-27 05:03:23,577][105620] Updated weights for policy 1, policy_version 1879514 (0.0010) [2023-12-27 05:03:23,639][105620] Updated weights for policy 1, policy_version 1879524 (0.0010) [2023-12-27 05:03:23,701][105620] Updated weights for policy 1, policy_version 1879534 (0.0011) [2023-12-27 05:03:24,170][105692] Updated weights for policy 0, policy_version 1875032 (0.0005) [2023-12-27 05:03:24,224][105692] Updated weights for policy 0, policy_version 1875042 (0.0005) [2023-12-27 05:03:24,275][105692] Updated weights for policy 0, policy_version 1875052 (0.0005) [2023-12-27 05:03:24,511][105620] Updated weights for policy 1, policy_version 1879544 (0.0010) [2023-12-27 05:03:24,580][105620] Updated weights for policy 1, policy_version 1879554 (0.0010) [2023-12-27 05:03:24,635][105620] Updated weights for policy 1, policy_version 1879564 (0.0009) [2023-12-27 05:03:24,903][105692] Updated weights for policy 0, policy_version 1875062 (0.0005) [2023-12-27 05:03:24,954][105692] Updated weights for policy 0, policy_version 1875072 (0.0005) [2023-12-27 05:03:25,006][105692] Updated weights for policy 0, policy_version 1875082 (0.0005) [2023-12-27 05:03:25,469][105620] Updated weights for policy 1, policy_version 1879574 (0.0009) [2023-12-27 05:03:25,527][105620] Updated weights for policy 1, policy_version 1879584 (0.0009) [2023-12-27 05:03:25,540][105692] Updated weights for policy 0, policy_version 1875092 (0.0007) [2023-12-27 05:03:25,571][105620] Updated weights for policy 1, policy_version 1879594 (0.0007) [2023-12-27 05:03:25,589][105692] Updated weights for policy 0, policy_version 1875102 (0.0007) [2023-12-27 05:03:25,642][105692] Updated weights for policy 0, policy_version 1875112 (0.0009) [2023-12-27 05:03:26,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 961347584. Throughput: 0: 9888.5, 1: 9674.0. Samples: 961356644. Policy #0 lag: (min: 2.0, avg: 11.6, max: 34.0) [2023-12-27 05:03:26,062][104569] Avg episode reward: [(0, '8537.777'), (1, '9160.574')] [2023-12-27 05:03:26,342][105620] Updated weights for policy 1, policy_version 1879604 (0.0007) [2023-12-27 05:03:26,392][105620] Updated weights for policy 1, policy_version 1879614 (0.0008) [2023-12-27 05:03:26,399][105692] Updated weights for policy 0, policy_version 1875122 (0.0008) [2023-12-27 05:03:26,444][105620] Updated weights for policy 1, policy_version 1879624 (0.0006) [2023-12-27 05:03:26,458][105692] Updated weights for policy 0, policy_version 1875132 (0.0011) [2023-12-27 05:03:26,510][105692] Updated weights for policy 0, policy_version 1875142 (0.0007) [2023-12-27 05:03:26,556][105692] Updated weights for policy 0, policy_version 1875152 (0.0006) [2023-12-27 05:03:27,142][105620] Updated weights for policy 1, policy_version 1879634 (0.0006) [2023-12-27 05:03:27,167][105692] Updated weights for policy 0, policy_version 1875162 (0.0006) [2023-12-27 05:03:27,205][105620] Updated weights for policy 1, policy_version 1879644 (0.0008) [2023-12-27 05:03:27,232][105692] Updated weights for policy 0, policy_version 1875172 (0.0007) [2023-12-27 05:03:27,267][105620] Updated weights for policy 1, policy_version 1879654 (0.0007) [2023-12-27 05:03:27,285][105692] Updated weights for policy 0, policy_version 1875182 (0.0009) [2023-12-27 05:03:27,328][105620] Updated weights for policy 1, policy_version 1879664 (0.0008) [2023-12-27 05:03:27,888][105620] Updated weights for policy 1, policy_version 1879674 (0.0008) [2023-12-27 05:03:27,936][105620] Updated weights for policy 1, policy_version 1879684 (0.0009) [2023-12-27 05:03:27,994][105620] Updated weights for policy 1, policy_version 1879694 (0.0009) [2023-12-27 05:03:28,053][105692] Updated weights for policy 0, policy_version 1875192 (0.0009) [2023-12-27 05:03:28,114][105692] Updated weights for policy 0, policy_version 1875202 (0.0009) [2023-12-27 05:03:28,174][105692] Updated weights for policy 0, policy_version 1875212 (0.0009) [2023-12-27 05:03:28,682][105620] Updated weights for policy 1, policy_version 1879704 (0.0006) [2023-12-27 05:03:28,740][105620] Updated weights for policy 1, policy_version 1879714 (0.0009) [2023-12-27 05:03:28,794][105620] Updated weights for policy 1, policy_version 1879724 (0.0010) [2023-12-27 05:03:28,886][105692] Updated weights for policy 0, policy_version 1875222 (0.0007) [2023-12-27 05:03:28,933][105692] Updated weights for policy 0, policy_version 1875232 (0.0005) [2023-12-27 05:03:28,982][105692] Updated weights for policy 0, policy_version 1875242 (0.0006) [2023-12-27 05:03:29,551][105620] Updated weights for policy 1, policy_version 1879735 (0.0011) [2023-12-27 05:03:29,609][105620] Updated weights for policy 1, policy_version 1879745 (0.0010) [2023-12-27 05:03:29,638][105692] Updated weights for policy 0, policy_version 1875252 (0.0005) [2023-12-27 05:03:29,667][105620] Updated weights for policy 1, policy_version 1879755 (0.0010) [2023-12-27 05:03:29,697][105692] Updated weights for policy 0, policy_version 1875262 (0.0006) [2023-12-27 05:03:29,757][105692] Updated weights for policy 0, policy_version 1875272 (0.0008) [2023-12-27 05:03:30,319][105620] Updated weights for policy 1, policy_version 1879765 (0.0010) [2023-12-27 05:03:30,371][105620] Updated weights for policy 1, policy_version 1879775 (0.0009) [2023-12-27 05:03:30,403][105692] Updated weights for policy 0, policy_version 1875282 (0.0008) [2023-12-27 05:03:30,422][105620] Updated weights for policy 1, policy_version 1879785 (0.0009) [2023-12-27 05:03:30,457][105692] Updated weights for policy 0, policy_version 1875292 (0.0005) [2023-12-27 05:03:30,502][105692] Updated weights for policy 0, policy_version 1875302 (0.0008) [2023-12-27 05:03:30,554][105692] Updated weights for policy 0, policy_version 1875312 (0.0008) [2023-12-27 05:03:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 961445888. Throughput: 0: 9833.7, 1: 9759.0. Samples: 961416464. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:03:31,062][104569] Avg episode reward: [(0, '8441.098'), (1, '9252.938')] [2023-12-27 05:03:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001875312_480149504.pth... [2023-12-27 05:03:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001879792_481296384.pth... [2023-12-27 05:03:31,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001874160_479854592.pth [2023-12-27 05:03:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001878672_481009664.pth [2023-12-27 05:03:31,114][105620] Updated weights for policy 1, policy_version 1879795 (0.0010) [2023-12-27 05:03:31,175][105620] Updated weights for policy 1, policy_version 1879805 (0.0010) [2023-12-27 05:03:31,237][105620] Updated weights for policy 1, policy_version 1879815 (0.0010) [2023-12-27 05:03:31,354][105692] Updated weights for policy 0, policy_version 1875322 (0.0009) [2023-12-27 05:03:31,421][105692] Updated weights for policy 0, policy_version 1875332 (0.0007) [2023-12-27 05:03:31,482][105692] Updated weights for policy 0, policy_version 1875342 (0.0008) [2023-12-27 05:03:32,010][105620] Updated weights for policy 1, policy_version 1879825 (0.0009) [2023-12-27 05:03:32,068][105620] Updated weights for policy 1, policy_version 1879835 (0.0006) [2023-12-27 05:03:32,129][105620] Updated weights for policy 1, policy_version 1879845 (0.0008) [2023-12-27 05:03:32,154][105692] Updated weights for policy 0, policy_version 1875352 (0.0010) [2023-12-27 05:03:32,188][105620] Updated weights for policy 1, policy_version 1879855 (0.0007) [2023-12-27 05:03:32,199][105692] Updated weights for policy 0, policy_version 1875362 (0.0010) [2023-12-27 05:03:32,252][105692] Updated weights for policy 0, policy_version 1875372 (0.0010) [2023-12-27 05:03:32,892][105620] Updated weights for policy 1, policy_version 1879865 (0.0008) [2023-12-27 05:03:32,958][105620] Updated weights for policy 1, policy_version 1879875 (0.0009) [2023-12-27 05:03:32,974][105692] Updated weights for policy 0, policy_version 1875382 (0.0007) [2023-12-27 05:03:33,016][105620] Updated weights for policy 1, policy_version 1879885 (0.0006) [2023-12-27 05:03:33,036][105692] Updated weights for policy 0, policy_version 1875392 (0.0008) [2023-12-27 05:03:33,092][105692] Updated weights for policy 0, policy_version 1875402 (0.0007) [2023-12-27 05:03:33,751][105620] Updated weights for policy 1, policy_version 1879895 (0.0005) [2023-12-27 05:03:33,796][105692] Updated weights for policy 0, policy_version 1875412 (0.0007) [2023-12-27 05:03:33,804][105620] Updated weights for policy 1, policy_version 1879905 (0.0005) [2023-12-27 05:03:33,854][105620] Updated weights for policy 1, policy_version 1879915 (0.0007) [2023-12-27 05:03:33,856][105692] Updated weights for policy 0, policy_version 1875422 (0.0007) [2023-12-27 05:03:33,914][105692] Updated weights for policy 0, policy_version 1875432 (0.0009) [2023-12-27 05:03:34,530][105620] Updated weights for policy 1, policy_version 1879925 (0.0006) [2023-12-27 05:03:34,586][105620] Updated weights for policy 1, policy_version 1879935 (0.0005) [2023-12-27 05:03:34,648][105620] Updated weights for policy 1, policy_version 1879945 (0.0007) [2023-12-27 05:03:34,728][105692] Updated weights for policy 0, policy_version 1875442 (0.0009) [2023-12-27 05:03:34,780][105692] Updated weights for policy 0, policy_version 1875452 (0.0009) [2023-12-27 05:03:34,833][105692] Updated weights for policy 0, policy_version 1875462 (0.0008) [2023-12-27 05:03:34,882][105692] Updated weights for policy 0, policy_version 1875472 (0.0008) [2023-12-27 05:03:35,316][105620] Updated weights for policy 1, policy_version 1879955 (0.0005) [2023-12-27 05:03:35,370][105620] Updated weights for policy 1, policy_version 1879965 (0.0009) [2023-12-27 05:03:35,427][105620] Updated weights for policy 1, policy_version 1879975 (0.0008) [2023-12-27 05:03:35,716][105692] Updated weights for policy 0, policy_version 1875482 (0.0009) [2023-12-27 05:03:35,785][105692] Updated weights for policy 0, policy_version 1875492 (0.0009) [2023-12-27 05:03:35,850][105692] Updated weights for policy 0, policy_version 1875502 (0.0008) [2023-12-27 05:03:36,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 961544192. Throughput: 0: 9825.4, 1: 9763.0. Samples: 961534840. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:03:36,063][104569] Avg episode reward: [(0, '8349.753'), (1, '9345.250')] [2023-12-27 05:03:36,079][105620] Updated weights for policy 1, policy_version 1879985 (0.0008) [2023-12-27 05:03:36,153][105620] Updated weights for policy 1, policy_version 1879995 (0.0009) [2023-12-27 05:03:36,214][105620] Updated weights for policy 1, policy_version 1880005 (0.0009) [2023-12-27 05:03:36,280][105620] Updated weights for policy 1, policy_version 1880015 (0.0007) [2023-12-27 05:03:36,638][105692] Updated weights for policy 0, policy_version 1875512 (0.0008) [2023-12-27 05:03:36,701][105692] Updated weights for policy 0, policy_version 1875522 (0.0009) [2023-12-27 05:03:36,771][105692] Updated weights for policy 0, policy_version 1875532 (0.0008) [2023-12-27 05:03:36,995][105620] Updated weights for policy 1, policy_version 1880025 (0.0006) [2023-12-27 05:03:37,061][105620] Updated weights for policy 1, policy_version 1880035 (0.0011) [2023-12-27 05:03:37,120][105620] Updated weights for policy 1, policy_version 1880045 (0.0011) [2023-12-27 05:03:37,386][105692] Updated weights for policy 0, policy_version 1875542 (0.0007) [2023-12-27 05:03:37,443][105692] Updated weights for policy 0, policy_version 1875552 (0.0008) [2023-12-27 05:03:37,499][105692] Updated weights for policy 0, policy_version 1875562 (0.0008) [2023-12-27 05:03:37,815][105620] Updated weights for policy 1, policy_version 1880055 (0.0010) [2023-12-27 05:03:37,871][105620] Updated weights for policy 1, policy_version 1880065 (0.0010) [2023-12-27 05:03:37,917][105620] Updated weights for policy 1, policy_version 1880075 (0.0010) [2023-12-27 05:03:38,151][105692] Updated weights for policy 0, policy_version 1875572 (0.0007) [2023-12-27 05:03:38,218][105692] Updated weights for policy 0, policy_version 1875582 (0.0006) [2023-12-27 05:03:38,281][105692] Updated weights for policy 0, policy_version 1875592 (0.0005) [2023-12-27 05:03:38,609][105620] Updated weights for policy 1, policy_version 1880085 (0.0008) [2023-12-27 05:03:38,676][105620] Updated weights for policy 1, policy_version 1880095 (0.0006) [2023-12-27 05:03:38,744][105620] Updated weights for policy 1, policy_version 1880105 (0.0011) [2023-12-27 05:03:38,919][105692] Updated weights for policy 0, policy_version 1875602 (0.0011) [2023-12-27 05:03:38,972][105692] Updated weights for policy 0, policy_version 1875612 (0.0010) [2023-12-27 05:03:39,027][105692] Updated weights for policy 0, policy_version 1875622 (0.0010) [2023-12-27 05:03:39,076][105692] Updated weights for policy 0, policy_version 1875632 (0.0010) [2023-12-27 05:03:39,478][105620] Updated weights for policy 1, policy_version 1880115 (0.0008) [2023-12-27 05:03:39,538][105620] Updated weights for policy 1, policy_version 1880125 (0.0008) [2023-12-27 05:03:39,594][105620] Updated weights for policy 1, policy_version 1880135 (0.0008) [2023-12-27 05:03:39,842][105692] Updated weights for policy 0, policy_version 1875642 (0.0009) [2023-12-27 05:03:39,900][105692] Updated weights for policy 0, policy_version 1875652 (0.0010) [2023-12-27 05:03:39,967][105692] Updated weights for policy 0, policy_version 1875662 (0.0009) [2023-12-27 05:03:40,377][105620] Updated weights for policy 1, policy_version 1880145 (0.0008) [2023-12-27 05:03:40,437][105620] Updated weights for policy 1, policy_version 1880155 (0.0007) [2023-12-27 05:03:40,488][105620] Updated weights for policy 1, policy_version 1880165 (0.0005) [2023-12-27 05:03:40,537][105620] Updated weights for policy 1, policy_version 1880175 (0.0005) [2023-12-27 05:03:40,778][105692] Updated weights for policy 0, policy_version 1875672 (0.0009) [2023-12-27 05:03:40,834][105692] Updated weights for policy 0, policy_version 1875682 (0.0010) [2023-12-27 05:03:40,892][105692] Updated weights for policy 0, policy_version 1875692 (0.0007) [2023-12-27 05:03:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 961642496. Throughput: 0: 9825.6, 1: 9756.9. Samples: 961651728. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:03:41,062][104569] Avg episode reward: [(0, '8168.611'), (1, '9345.278')] [2023-12-27 05:03:41,143][105620] Updated weights for policy 1, policy_version 1880185 (0.0009) [2023-12-27 05:03:41,212][105620] Updated weights for policy 1, policy_version 1880195 (0.0006) [2023-12-27 05:03:41,281][105620] Updated weights for policy 1, policy_version 1880205 (0.0006) [2023-12-27 05:03:41,596][105692] Updated weights for policy 0, policy_version 1875702 (0.0006) [2023-12-27 05:03:41,660][105692] Updated weights for policy 0, policy_version 1875712 (0.0009) [2023-12-27 05:03:41,728][105692] Updated weights for policy 0, policy_version 1875722 (0.0010) [2023-12-27 05:03:42,010][105620] Updated weights for policy 1, policy_version 1880215 (0.0007) [2023-12-27 05:03:42,070][105620] Updated weights for policy 1, policy_version 1880225 (0.0008) [2023-12-27 05:03:42,130][105620] Updated weights for policy 1, policy_version 1880235 (0.0008) [2023-12-27 05:03:42,512][105692] Updated weights for policy 0, policy_version 1875732 (0.0010) [2023-12-27 05:03:42,576][105692] Updated weights for policy 0, policy_version 1875742 (0.0010) [2023-12-27 05:03:42,643][105692] Updated weights for policy 0, policy_version 1875752 (0.0009) [2023-12-27 05:03:42,845][105620] Updated weights for policy 1, policy_version 1880245 (0.0007) [2023-12-27 05:03:42,906][105620] Updated weights for policy 1, policy_version 1880255 (0.0006) [2023-12-27 05:03:42,973][105620] Updated weights for policy 1, policy_version 1880265 (0.0008) [2023-12-27 05:03:43,301][105692] Updated weights for policy 0, policy_version 1875762 (0.0005) [2023-12-27 05:03:43,363][105692] Updated weights for policy 0, policy_version 1875772 (0.0005) [2023-12-27 05:03:43,421][105692] Updated weights for policy 0, policy_version 1875782 (0.0005) [2023-12-27 05:03:43,475][105692] Updated weights for policy 0, policy_version 1875792 (0.0005) [2023-12-27 05:03:43,680][105620] Updated weights for policy 1, policy_version 1880275 (0.0006) [2023-12-27 05:03:43,740][105620] Updated weights for policy 1, policy_version 1880285 (0.0006) [2023-12-27 05:03:43,800][105620] Updated weights for policy 1, policy_version 1880296 (0.0006) [2023-12-27 05:03:43,987][105692] Updated weights for policy 0, policy_version 1875802 (0.0005) [2023-12-27 05:03:44,051][105692] Updated weights for policy 0, policy_version 1875812 (0.0005) [2023-12-27 05:03:44,123][105692] Updated weights for policy 0, policy_version 1875822 (0.0005) [2023-12-27 05:03:44,414][105620] Updated weights for policy 1, policy_version 1880306 (0.0006) [2023-12-27 05:03:44,470][105620] Updated weights for policy 1, policy_version 1880316 (0.0009) [2023-12-27 05:03:44,518][105620] Updated weights for policy 1, policy_version 1880326 (0.0009) [2023-12-27 05:03:44,574][105620] Updated weights for policy 1, policy_version 1880336 (0.0009) [2023-12-27 05:03:44,674][105692] Updated weights for policy 0, policy_version 1875832 (0.0006) [2023-12-27 05:03:44,731][105692] Updated weights for policy 0, policy_version 1875842 (0.0007) [2023-12-27 05:03:44,793][105692] Updated weights for policy 0, policy_version 1875852 (0.0010) [2023-12-27 05:03:45,380][105620] Updated weights for policy 1, policy_version 1880346 (0.0009) [2023-12-27 05:03:45,447][105620] Updated weights for policy 1, policy_version 1880356 (0.0009) [2023-12-27 05:03:45,511][105620] Updated weights for policy 1, policy_version 1880366 (0.0009) [2023-12-27 05:03:45,525][105692] Updated weights for policy 0, policy_version 1875862 (0.0010) [2023-12-27 05:03:45,572][105692] Updated weights for policy 0, policy_version 1875872 (0.0009) [2023-12-27 05:03:45,625][105692] Updated weights for policy 0, policy_version 1875883 (0.0010) [2023-12-27 05:03:46,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 961740800. Throughput: 0: 9894.0, 1: 9740.2. Samples: 961710292. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:03:46,062][104569] Avg episode reward: [(0, '8259.001'), (1, '9253.351')] [2023-12-27 05:03:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001875888_480296960.pth... [2023-12-27 05:03:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001880368_481443840.pth... [2023-12-27 05:03:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001879248_481157120.pth [2023-12-27 05:03:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001874704_479993856.pth [2023-12-27 05:03:46,178][105620] Updated weights for policy 1, policy_version 1880376 (0.0010) [2023-12-27 05:03:46,226][105620] Updated weights for policy 1, policy_version 1880386 (0.0010) [2023-12-27 05:03:46,285][105620] Updated weights for policy 1, policy_version 1880396 (0.0010) [2023-12-27 05:03:46,419][105692] Updated weights for policy 0, policy_version 1875893 (0.0009) [2023-12-27 05:03:46,475][105692] Updated weights for policy 0, policy_version 1875903 (0.0008) [2023-12-27 05:03:46,537][105692] Updated weights for policy 0, policy_version 1875913 (0.0009) [2023-12-27 05:03:46,993][105620] Updated weights for policy 1, policy_version 1880406 (0.0009) [2023-12-27 05:03:47,052][105620] Updated weights for policy 1, policy_version 1880416 (0.0008) [2023-12-27 05:03:47,106][105620] Updated weights for policy 1, policy_version 1880426 (0.0007) [2023-12-27 05:03:47,296][105692] Updated weights for policy 0, policy_version 1875923 (0.0009) [2023-12-27 05:03:47,343][105692] Updated weights for policy 0, policy_version 1875933 (0.0010) [2023-12-27 05:03:47,405][105692] Updated weights for policy 0, policy_version 1875943 (0.0010) [2023-12-27 05:03:47,877][105620] Updated weights for policy 1, policy_version 1880436 (0.0008) [2023-12-27 05:03:47,930][105620] Updated weights for policy 1, policy_version 1880446 (0.0009) [2023-12-27 05:03:47,985][105620] Updated weights for policy 1, policy_version 1880457 (0.0010) [2023-12-27 05:03:48,073][105692] Updated weights for policy 0, policy_version 1875953 (0.0011) [2023-12-27 05:03:48,130][105692] Updated weights for policy 0, policy_version 1875963 (0.0007) [2023-12-27 05:03:48,190][105692] Updated weights for policy 0, policy_version 1875973 (0.0005) [2023-12-27 05:03:48,249][105692] Updated weights for policy 0, policy_version 1875983 (0.0007) [2023-12-27 05:03:48,727][105620] Updated weights for policy 1, policy_version 1880467 (0.0009) [2023-12-27 05:03:48,778][105620] Updated weights for policy 1, policy_version 1880477 (0.0009) [2023-12-27 05:03:48,829][105620] Updated weights for policy 1, policy_version 1880487 (0.0009) [2023-12-27 05:03:48,984][105692] Updated weights for policy 0, policy_version 1875993 (0.0006) [2023-12-27 05:03:49,047][105692] Updated weights for policy 0, policy_version 1876003 (0.0007) [2023-12-27 05:03:49,111][105692] Updated weights for policy 0, policy_version 1876013 (0.0009) [2023-12-27 05:03:49,637][105620] Updated weights for policy 1, policy_version 1880497 (0.0009) [2023-12-27 05:03:49,691][105620] Updated weights for policy 1, policy_version 1880507 (0.0009) [2023-12-27 05:03:49,757][105620] Updated weights for policy 1, policy_version 1880517 (0.0007) [2023-12-27 05:03:49,762][105692] Updated weights for policy 0, policy_version 1876023 (0.0007) [2023-12-27 05:03:49,814][105620] Updated weights for policy 1, policy_version 1880527 (0.0007) [2023-12-27 05:03:49,816][105692] Updated weights for policy 0, policy_version 1876033 (0.0006) [2023-12-27 05:03:49,880][105692] Updated weights for policy 0, policy_version 1876043 (0.0008) [2023-12-27 05:03:50,602][105620] Updated weights for policy 1, policy_version 1880537 (0.0008) [2023-12-27 05:03:50,627][105692] Updated weights for policy 0, policy_version 1876053 (0.0009) [2023-12-27 05:03:50,663][105620] Updated weights for policy 1, policy_version 1880547 (0.0009) [2023-12-27 05:03:50,677][105692] Updated weights for policy 0, policy_version 1876063 (0.0008) [2023-12-27 05:03:50,726][105620] Updated weights for policy 1, policy_version 1880557 (0.0007) [2023-12-27 05:03:50,732][105692] Updated weights for policy 0, policy_version 1876073 (0.0008) [2023-12-27 05:03:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 961839104. Throughput: 0: 9941.4, 1: 9772.5. Samples: 961827288. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:03:51,062][104569] Avg episode reward: [(0, '8441.912'), (1, '9161.464')] [2023-12-27 05:03:51,495][105692] Updated weights for policy 0, policy_version 1876083 (0.0008) [2023-12-27 05:03:51,518][105620] Updated weights for policy 1, policy_version 1880567 (0.0008) [2023-12-27 05:03:51,542][105692] Updated weights for policy 0, policy_version 1876093 (0.0008) [2023-12-27 05:03:51,591][105620] Updated weights for policy 1, policy_version 1880577 (0.0007) [2023-12-27 05:03:51,598][105692] Updated weights for policy 0, policy_version 1876103 (0.0009) [2023-12-27 05:03:51,658][105620] Updated weights for policy 1, policy_version 1880587 (0.0008) [2023-12-27 05:03:52,380][105620] Updated weights for policy 1, policy_version 1880597 (0.0008) [2023-12-27 05:03:52,385][105692] Updated weights for policy 0, policy_version 1876113 (0.0007) [2023-12-27 05:03:52,437][105620] Updated weights for policy 1, policy_version 1880607 (0.0007) [2023-12-27 05:03:52,442][105692] Updated weights for policy 0, policy_version 1876123 (0.0008) [2023-12-27 05:03:52,486][105620] Updated weights for policy 1, policy_version 1880617 (0.0007) [2023-12-27 05:03:52,495][105692] Updated weights for policy 0, policy_version 1876133 (0.0008) [2023-12-27 05:03:52,551][105692] Updated weights for policy 0, policy_version 1876143 (0.0006) [2023-12-27 05:03:53,253][105620] Updated weights for policy 1, policy_version 1880627 (0.0009) [2023-12-27 05:03:53,309][105620] Updated weights for policy 1, policy_version 1880637 (0.0007) [2023-12-27 05:03:53,314][105692] Updated weights for policy 0, policy_version 1876153 (0.0008) [2023-12-27 05:03:53,370][105692] Updated weights for policy 0, policy_version 1876163 (0.0005) [2023-12-27 05:03:53,376][105620] Updated weights for policy 1, policy_version 1880647 (0.0009) [2023-12-27 05:03:53,429][105692] Updated weights for policy 0, policy_version 1876173 (0.0006) [2023-12-27 05:03:54,072][105620] Updated weights for policy 1, policy_version 1880657 (0.0007) [2023-12-27 05:03:54,139][105620] Updated weights for policy 1, policy_version 1880667 (0.0009) [2023-12-27 05:03:54,193][105692] Updated weights for policy 0, policy_version 1876183 (0.0008) [2023-12-27 05:03:54,205][105620] Updated weights for policy 1, policy_version 1880677 (0.0007) [2023-12-27 05:03:54,255][105692] Updated weights for policy 0, policy_version 1876193 (0.0008) [2023-12-27 05:03:54,267][105620] Updated weights for policy 1, policy_version 1880687 (0.0006) [2023-12-27 05:03:54,309][105692] Updated weights for policy 0, policy_version 1876203 (0.0009) [2023-12-27 05:03:54,903][105620] Updated weights for policy 1, policy_version 1880697 (0.0007) [2023-12-27 05:03:54,955][105620] Updated weights for policy 1, policy_version 1880707 (0.0009) [2023-12-27 05:03:55,013][105620] Updated weights for policy 1, policy_version 1880717 (0.0010) [2023-12-27 05:03:55,036][105692] Updated weights for policy 0, policy_version 1876213 (0.0008) [2023-12-27 05:03:55,097][105692] Updated weights for policy 0, policy_version 1876223 (0.0006) [2023-12-27 05:03:55,159][105692] Updated weights for policy 0, policy_version 1876233 (0.0009) [2023-12-27 05:03:55,821][105620] Updated weights for policy 1, policy_version 1880727 (0.0009) [2023-12-27 05:03:55,822][105692] Updated weights for policy 0, policy_version 1876243 (0.0008) [2023-12-27 05:03:55,870][105692] Updated weights for policy 0, policy_version 1876253 (0.0005) [2023-12-27 05:03:55,884][105620] Updated weights for policy 1, policy_version 1880737 (0.0009) [2023-12-27 05:03:55,936][105692] Updated weights for policy 0, policy_version 1876263 (0.0005) [2023-12-27 05:03:55,942][105620] Updated weights for policy 1, policy_version 1880747 (0.0008) [2023-12-27 05:03:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.4, 300 sec: 19522.0). Total num frames: 961937408. Throughput: 0: 9901.7, 1: 9633.4. Samples: 961939544. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:03:56,063][104569] Avg episode reward: [(0, '8356.407'), (1, '9253.423')] [2023-12-27 05:03:56,596][105692] Updated weights for policy 0, policy_version 1876273 (0.0005) [2023-12-27 05:03:56,643][105692] Updated weights for policy 0, policy_version 1876283 (0.0009) [2023-12-27 05:03:56,692][105692] Updated weights for policy 0, policy_version 1876293 (0.0008) [2023-12-27 05:03:56,702][105620] Updated weights for policy 1, policy_version 1880757 (0.0009) [2023-12-27 05:03:56,740][105692] Updated weights for policy 0, policy_version 1876303 (0.0006) [2023-12-27 05:03:56,750][105620] Updated weights for policy 1, policy_version 1880767 (0.0007) [2023-12-27 05:03:56,811][105620] Updated weights for policy 1, policy_version 1880778 (0.0009) [2023-12-27 05:03:57,461][105692] Updated weights for policy 0, policy_version 1876313 (0.0008) [2023-12-27 05:03:57,515][105692] Updated weights for policy 0, policy_version 1876323 (0.0009) [2023-12-27 05:03:57,572][105692] Updated weights for policy 0, policy_version 1876333 (0.0009) [2023-12-27 05:03:57,579][105620] Updated weights for policy 1, policy_version 1880788 (0.0008) [2023-12-27 05:03:57,638][105620] Updated weights for policy 1, policy_version 1880798 (0.0008) [2023-12-27 05:03:57,700][105620] Updated weights for policy 1, policy_version 1880808 (0.0009) [2023-12-27 05:03:58,330][105692] Updated weights for policy 0, policy_version 1876343 (0.0008) [2023-12-27 05:03:58,394][105692] Updated weights for policy 0, policy_version 1876353 (0.0010) [2023-12-27 05:03:58,458][105692] Updated weights for policy 0, policy_version 1876363 (0.0008) [2023-12-27 05:03:58,479][105620] Updated weights for policy 1, policy_version 1880818 (0.0008) [2023-12-27 05:03:58,541][105620] Updated weights for policy 1, policy_version 1880828 (0.0008) [2023-12-27 05:03:58,606][105620] Updated weights for policy 1, policy_version 1880838 (0.0007) [2023-12-27 05:03:58,666][105620] Updated weights for policy 1, policy_version 1880848 (0.0008) [2023-12-27 05:03:59,315][105692] Updated weights for policy 0, policy_version 1876373 (0.0009) [2023-12-27 05:03:59,378][105692] Updated weights for policy 0, policy_version 1876383 (0.0009) [2023-12-27 05:03:59,433][105692] Updated weights for policy 0, policy_version 1876393 (0.0007) [2023-12-27 05:03:59,446][105620] Updated weights for policy 1, policy_version 1880858 (0.0009) [2023-12-27 05:03:59,502][105620] Updated weights for policy 1, policy_version 1880868 (0.0008) [2023-12-27 05:03:59,561][105620] Updated weights for policy 1, policy_version 1880878 (0.0009) [2023-12-27 05:04:00,140][105692] Updated weights for policy 0, policy_version 1876403 (0.0008) [2023-12-27 05:04:00,199][105692] Updated weights for policy 0, policy_version 1876413 (0.0009) [2023-12-27 05:04:00,246][105692] Updated weights for policy 0, policy_version 1876423 (0.0008) [2023-12-27 05:04:00,312][105620] Updated weights for policy 1, policy_version 1880888 (0.0005) [2023-12-27 05:04:00,365][105620] Updated weights for policy 1, policy_version 1880898 (0.0005) [2023-12-27 05:04:00,422][105620] Updated weights for policy 1, policy_version 1880908 (0.0005) [2023-12-27 05:04:00,991][105620] Updated weights for policy 1, policy_version 1880918 (0.0006) [2023-12-27 05:04:01,056][105620] Updated weights for policy 1, policy_version 1880928 (0.0008) [2023-12-27 05:04:01,062][104569] Fps is (10 sec: 18022.4, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 962019328. Throughput: 0: 9879.9, 1: 9591.4. Samples: 961996188. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:04:01,063][104569] Avg episode reward: [(0, '8266.852'), (1, '9345.314')] [2023-12-27 05:04:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001876432_480436224.pth... [2023-12-27 05:04:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001875312_480149504.pth [2023-12-27 05:04:01,099][105692] Updated weights for policy 0, policy_version 1876433 (0.0010) [2023-12-27 05:04:01,120][105620] Updated weights for policy 1, policy_version 1880938 (0.0008) [2023-12-27 05:04:01,160][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001880944_481591296.pth... [2023-12-27 05:04:01,164][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001879792_481296384.pth [2023-12-27 05:04:01,166][105692] Updated weights for policy 0, policy_version 1876443 (0.0008) [2023-12-27 05:04:01,221][105692] Updated weights for policy 0, policy_version 1876453 (0.0008) [2023-12-27 05:04:01,276][105692] Updated weights for policy 0, policy_version 1876463 (0.0009) [2023-12-27 05:04:01,844][105620] Updated weights for policy 1, policy_version 1880948 (0.0008) [2023-12-27 05:04:01,895][105620] Updated weights for policy 1, policy_version 1880958 (0.0008) [2023-12-27 05:04:01,917][105692] Updated weights for policy 0, policy_version 1876473 (0.0007) [2023-12-27 05:04:01,959][105620] Updated weights for policy 1, policy_version 1880968 (0.0007) [2023-12-27 05:04:01,985][105692] Updated weights for policy 0, policy_version 1876483 (0.0009) [2023-12-27 05:04:02,051][105692] Updated weights for policy 0, policy_version 1876493 (0.0009) [2023-12-27 05:04:02,595][105620] Updated weights for policy 1, policy_version 1880978 (0.0008) [2023-12-27 05:04:02,654][105620] Updated weights for policy 1, policy_version 1880988 (0.0009) [2023-12-27 05:04:02,702][105620] Updated weights for policy 1, policy_version 1880998 (0.0009) [2023-12-27 05:04:02,748][105620] Updated weights for policy 1, policy_version 1881008 (0.0007) [2023-12-27 05:04:02,815][105692] Updated weights for policy 0, policy_version 1876503 (0.0010) [2023-12-27 05:04:02,866][105692] Updated weights for policy 0, policy_version 1876513 (0.0009) [2023-12-27 05:04:02,930][105692] Updated weights for policy 0, policy_version 1876523 (0.0009) [2023-12-27 05:04:03,471][105620] Updated weights for policy 1, policy_version 1881018 (0.0007) [2023-12-27 05:04:03,525][105620] Updated weights for policy 1, policy_version 1881028 (0.0005) [2023-12-27 05:04:03,579][105620] Updated weights for policy 1, policy_version 1881038 (0.0009) [2023-12-27 05:04:03,754][105692] Updated weights for policy 0, policy_version 1876533 (0.0009) [2023-12-27 05:04:03,801][105692] Updated weights for policy 0, policy_version 1876543 (0.0009) [2023-12-27 05:04:03,853][105692] Updated weights for policy 0, policy_version 1876553 (0.0008) [2023-12-27 05:04:04,266][105620] Updated weights for policy 1, policy_version 1881048 (0.0010) [2023-12-27 05:04:04,328][105620] Updated weights for policy 1, policy_version 1881058 (0.0008) [2023-12-27 05:04:04,393][105620] Updated weights for policy 1, policy_version 1881068 (0.0006) [2023-12-27 05:04:04,669][105692] Updated weights for policy 0, policy_version 1876563 (0.0008) [2023-12-27 05:04:04,734][105692] Updated weights for policy 0, policy_version 1876573 (0.0009) [2023-12-27 05:04:04,798][105692] Updated weights for policy 0, policy_version 1876583 (0.0010) [2023-12-27 05:04:05,067][105620] Updated weights for policy 1, policy_version 1881078 (0.0010) [2023-12-27 05:04:05,121][105620] Updated weights for policy 1, policy_version 1881088 (0.0010) [2023-12-27 05:04:05,175][105620] Updated weights for policy 1, policy_version 1881098 (0.0010) [2023-12-27 05:04:05,435][105692] Updated weights for policy 0, policy_version 1876593 (0.0008) [2023-12-27 05:04:05,492][105692] Updated weights for policy 0, policy_version 1876603 (0.0006) [2023-12-27 05:04:05,552][105692] Updated weights for policy 0, policy_version 1876613 (0.0010) [2023-12-27 05:04:05,616][105692] Updated weights for policy 0, policy_version 1876624 (0.0009) [2023-12-27 05:04:06,005][105620] Updated weights for policy 1, policy_version 1881108 (0.0008) [2023-12-27 05:04:06,057][105620] Updated weights for policy 1, policy_version 1881118 (0.0005) [2023-12-27 05:04:06,062][104569] Fps is (10 sec: 18022.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 962117632. Throughput: 0: 9712.1, 1: 9615.5. Samples: 962110864. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:04:06,063][104569] Avg episode reward: [(0, '8629.331'), (1, '9345.263')] [2023-12-27 05:04:06,113][105620] Updated weights for policy 1, policy_version 1881128 (0.0006) [2023-12-27 05:04:06,183][105692] Updated weights for policy 0, policy_version 1876634 (0.0008) [2023-12-27 05:04:06,238][105692] Updated weights for policy 0, policy_version 1876644 (0.0008) [2023-12-27 05:04:06,301][105692] Updated weights for policy 0, policy_version 1876654 (0.0008) [2023-12-27 05:04:06,815][105620] Updated weights for policy 1, policy_version 1881138 (0.0008) [2023-12-27 05:04:06,865][105620] Updated weights for policy 1, policy_version 1881148 (0.0008) [2023-12-27 05:04:06,921][105620] Updated weights for policy 1, policy_version 1881158 (0.0005) [2023-12-27 05:04:06,977][105620] Updated weights for policy 1, policy_version 1881168 (0.0008) [2023-12-27 05:04:07,070][105692] Updated weights for policy 0, policy_version 1876664 (0.0007) [2023-12-27 05:04:07,126][105692] Updated weights for policy 0, policy_version 1876674 (0.0006) [2023-12-27 05:04:07,187][105692] Updated weights for policy 0, policy_version 1876684 (0.0008) [2023-12-27 05:04:07,747][105692] Updated weights for policy 0, policy_version 1876694 (0.0007) [2023-12-27 05:04:07,786][105620] Updated weights for policy 1, policy_version 1881178 (0.0006) [2023-12-27 05:04:07,792][105692] Updated weights for policy 0, policy_version 1876704 (0.0005) [2023-12-27 05:04:07,844][105620] Updated weights for policy 1, policy_version 1881188 (0.0010) [2023-12-27 05:04:07,850][105692] Updated weights for policy 0, policy_version 1876714 (0.0006) [2023-12-27 05:04:07,893][105620] Updated weights for policy 1, policy_version 1881198 (0.0005) [2023-12-27 05:04:08,528][105692] Updated weights for policy 0, policy_version 1876724 (0.0008) [2023-12-27 05:04:08,533][105620] Updated weights for policy 1, policy_version 1881208 (0.0006) [2023-12-27 05:04:08,589][105692] Updated weights for policy 0, policy_version 1876734 (0.0011) [2023-12-27 05:04:08,598][105620] Updated weights for policy 1, policy_version 1881218 (0.0006) [2023-12-27 05:04:08,650][105692] Updated weights for policy 0, policy_version 1876744 (0.0011) [2023-12-27 05:04:08,662][105620] Updated weights for policy 1, policy_version 1881228 (0.0006) [2023-12-27 05:04:09,275][105620] Updated weights for policy 1, policy_version 1881238 (0.0009) [2023-12-27 05:04:09,342][105620] Updated weights for policy 1, policy_version 1881248 (0.0011) [2023-12-27 05:04:09,353][105692] Updated weights for policy 0, policy_version 1876754 (0.0011) [2023-12-27 05:04:09,407][105620] Updated weights for policy 1, policy_version 1881258 (0.0011) [2023-12-27 05:04:09,421][105692] Updated weights for policy 0, policy_version 1876764 (0.0011) [2023-12-27 05:04:09,484][105692] Updated weights for policy 0, policy_version 1876774 (0.0010) [2023-12-27 05:04:09,544][105692] Updated weights for policy 0, policy_version 1876784 (0.0010) [2023-12-27 05:04:10,172][105620] Updated weights for policy 1, policy_version 1881268 (0.0009) [2023-12-27 05:04:10,230][105620] Updated weights for policy 1, policy_version 1881278 (0.0005) [2023-12-27 05:04:10,288][105620] Updated weights for policy 1, policy_version 1881288 (0.0005) [2023-12-27 05:04:10,341][105692] Updated weights for policy 0, policy_version 1876794 (0.0011) [2023-12-27 05:04:10,400][105692] Updated weights for policy 0, policy_version 1876804 (0.0010) [2023-12-27 05:04:10,463][105692] Updated weights for policy 0, policy_version 1876814 (0.0011) [2023-12-27 05:04:10,902][105620] Updated weights for policy 1, policy_version 1881298 (0.0006) [2023-12-27 05:04:10,951][105620] Updated weights for policy 1, policy_version 1881308 (0.0006) [2023-12-27 05:04:11,008][105620] Updated weights for policy 1, policy_version 1881318 (0.0007) [2023-12-27 05:04:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 962215936. Throughput: 0: 9680.7, 1: 9742.4. Samples: 962230684. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:04:11,063][104569] Avg episode reward: [(0, '8440.305'), (1, '9253.999')] [2023-12-27 05:04:11,078][105620] Updated weights for policy 1, policy_version 1881328 (0.0007) [2023-12-27 05:04:11,086][105692] Updated weights for policy 0, policy_version 1876824 (0.0009) [2023-12-27 05:04:11,156][105692] Updated weights for policy 0, policy_version 1876834 (0.0010) [2023-12-27 05:04:11,220][105692] Updated weights for policy 0, policy_version 1876844 (0.0010) [2023-12-27 05:04:11,801][105620] Updated weights for policy 1, policy_version 1881338 (0.0009) [2023-12-27 05:04:11,851][105620] Updated weights for policy 1, policy_version 1881348 (0.0009) [2023-12-27 05:04:11,900][105620] Updated weights for policy 1, policy_version 1881358 (0.0009) [2023-12-27 05:04:11,988][105692] Updated weights for policy 0, policy_version 1876854 (0.0009) [2023-12-27 05:04:12,037][105692] Updated weights for policy 0, policy_version 1876864 (0.0009) [2023-12-27 05:04:12,087][105692] Updated weights for policy 0, policy_version 1876874 (0.0009) [2023-12-27 05:04:12,697][105620] Updated weights for policy 1, policy_version 1881368 (0.0007) [2023-12-27 05:04:12,755][105620] Updated weights for policy 1, policy_version 1881378 (0.0006) [2023-12-27 05:04:12,821][105620] Updated weights for policy 1, policy_version 1881388 (0.0006) [2023-12-27 05:04:12,868][105692] Updated weights for policy 0, policy_version 1876884 (0.0008) [2023-12-27 05:04:12,936][105692] Updated weights for policy 0, policy_version 1876894 (0.0011) [2023-12-27 05:04:13,012][105692] Updated weights for policy 0, policy_version 1876904 (0.0011) [2023-12-27 05:04:13,423][105620] Updated weights for policy 1, policy_version 1881398 (0.0009) [2023-12-27 05:04:13,481][105620] Updated weights for policy 1, policy_version 1881408 (0.0010) [2023-12-27 05:04:13,543][105620] Updated weights for policy 1, policy_version 1881418 (0.0010) [2023-12-27 05:04:13,598][105692] Updated weights for policy 0, policy_version 1876914 (0.0011) [2023-12-27 05:04:13,646][105692] Updated weights for policy 0, policy_version 1876924 (0.0010) [2023-12-27 05:04:13,697][105692] Updated weights for policy 0, policy_version 1876934 (0.0010) [2023-12-27 05:04:13,761][105692] Updated weights for policy 0, policy_version 1876944 (0.0010) [2023-12-27 05:04:14,222][105620] Updated weights for policy 1, policy_version 1881428 (0.0009) [2023-12-27 05:04:14,268][105620] Updated weights for policy 1, policy_version 1881438 (0.0005) [2023-12-27 05:04:14,321][105620] Updated weights for policy 1, policy_version 1881448 (0.0006) [2023-12-27 05:04:14,414][105692] Updated weights for policy 0, policy_version 1876954 (0.0011) [2023-12-27 05:04:14,462][105692] Updated weights for policy 0, policy_version 1876964 (0.0010) [2023-12-27 05:04:14,517][105692] Updated weights for policy 0, policy_version 1876974 (0.0010) [2023-12-27 05:04:14,882][105620] Updated weights for policy 1, policy_version 1881458 (0.0009) [2023-12-27 05:04:14,941][105620] Updated weights for policy 1, policy_version 1881468 (0.0010) [2023-12-27 05:04:15,007][105620] Updated weights for policy 1, policy_version 1881478 (0.0010) [2023-12-27 05:04:15,070][105620] Updated weights for policy 1, policy_version 1881488 (0.0010) [2023-12-27 05:04:15,268][105692] Updated weights for policy 0, policy_version 1876984 (0.0007) [2023-12-27 05:04:15,331][105692] Updated weights for policy 0, policy_version 1876994 (0.0006) [2023-12-27 05:04:15,394][105692] Updated weights for policy 0, policy_version 1877004 (0.0006) [2023-12-27 05:04:15,785][105620] Updated weights for policy 1, policy_version 1881498 (0.0010) [2023-12-27 05:04:15,839][105620] Updated weights for policy 1, policy_version 1881508 (0.0010) [2023-12-27 05:04:15,890][105620] Updated weights for policy 1, policy_version 1881518 (0.0010) [2023-12-27 05:04:15,991][105692] Updated weights for policy 0, policy_version 1877014 (0.0007) [2023-12-27 05:04:16,051][105692] Updated weights for policy 0, policy_version 1877024 (0.0008) [2023-12-27 05:04:16,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 962322432. Throughput: 0: 9716.9, 1: 9696.8. Samples: 962290080. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:04:16,062][104569] Avg episode reward: [(0, '7896.458'), (1, '9161.513')] [2023-12-27 05:04:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001881520_481738752.pth... [2023-12-27 05:04:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001880368_481443840.pth [2023-12-27 05:04:16,107][105692] Updated weights for policy 0, policy_version 1877034 (0.0008) [2023-12-27 05:04:16,136][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001877040_480591872.pth... [2023-12-27 05:04:16,139][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001875888_480296960.pth [2023-12-27 05:04:16,654][105620] Updated weights for policy 1, policy_version 1881528 (0.0010) [2023-12-27 05:04:16,713][105620] Updated weights for policy 1, policy_version 1881538 (0.0010) [2023-12-27 05:04:16,739][105692] Updated weights for policy 0, policy_version 1877044 (0.0007) [2023-12-27 05:04:16,773][105620] Updated weights for policy 1, policy_version 1881548 (0.0005) [2023-12-27 05:04:16,796][105692] Updated weights for policy 0, policy_version 1877054 (0.0009) [2023-12-27 05:04:16,866][105692] Updated weights for policy 0, policy_version 1877064 (0.0010) [2023-12-27 05:04:17,427][105620] Updated weights for policy 1, policy_version 1881558 (0.0008) [2023-12-27 05:04:17,485][105620] Updated weights for policy 1, policy_version 1881568 (0.0010) [2023-12-27 05:04:17,509][105692] Updated weights for policy 0, policy_version 1877074 (0.0010) [2023-12-27 05:04:17,543][105620] Updated weights for policy 1, policy_version 1881578 (0.0010) [2023-12-27 05:04:17,560][105692] Updated weights for policy 0, policy_version 1877084 (0.0010) [2023-12-27 05:04:17,626][105692] Updated weights for policy 0, policy_version 1877094 (0.0011) [2023-12-27 05:04:17,694][105692] Updated weights for policy 0, policy_version 1877104 (0.0010) [2023-12-27 05:04:18,122][105620] Updated weights for policy 1, policy_version 1881588 (0.0006) [2023-12-27 05:04:18,185][105620] Updated weights for policy 1, policy_version 1881598 (0.0006) [2023-12-27 05:04:18,244][105620] Updated weights for policy 1, policy_version 1881608 (0.0005) [2023-12-27 05:04:18,256][105692] Updated weights for policy 0, policy_version 1877114 (0.0005) [2023-12-27 05:04:18,301][105692] Updated weights for policy 0, policy_version 1877124 (0.0005) [2023-12-27 05:04:18,389][105692] Updated weights for policy 0, policy_version 1877134 (0.0006) [2023-12-27 05:04:18,940][105620] Updated weights for policy 1, policy_version 1881618 (0.0006) [2023-12-27 05:04:18,992][105692] Updated weights for policy 0, policy_version 1877144 (0.0007) [2023-12-27 05:04:18,998][105620] Updated weights for policy 1, policy_version 1881628 (0.0008) [2023-12-27 05:04:19,051][105692] Updated weights for policy 0, policy_version 1877154 (0.0006) [2023-12-27 05:04:19,060][105620] Updated weights for policy 1, policy_version 1881638 (0.0009) [2023-12-27 05:04:19,109][105692] Updated weights for policy 0, policy_version 1877164 (0.0006) [2023-12-27 05:04:19,115][105620] Updated weights for policy 1, policy_version 1881648 (0.0008) [2023-12-27 05:04:19,777][105692] Updated weights for policy 0, policy_version 1877174 (0.0008) [2023-12-27 05:04:19,804][105620] Updated weights for policy 1, policy_version 1881658 (0.0007) [2023-12-27 05:04:19,841][105692] Updated weights for policy 0, policy_version 1877184 (0.0008) [2023-12-27 05:04:19,864][105620] Updated weights for policy 1, policy_version 1881668 (0.0007) [2023-12-27 05:04:19,903][105692] Updated weights for policy 0, policy_version 1877194 (0.0007) [2023-12-27 05:04:19,926][105620] Updated weights for policy 1, policy_version 1881678 (0.0007) [2023-12-27 05:04:20,560][105620] Updated weights for policy 1, policy_version 1881688 (0.0006) [2023-12-27 05:04:20,625][105620] Updated weights for policy 1, policy_version 1881698 (0.0008) [2023-12-27 05:04:20,685][105692] Updated weights for policy 0, policy_version 1877204 (0.0009) [2023-12-27 05:04:20,687][105620] Updated weights for policy 1, policy_version 1881708 (0.0007) [2023-12-27 05:04:20,741][105692] Updated weights for policy 0, policy_version 1877214 (0.0008) [2023-12-27 05:04:20,807][105692] Updated weights for policy 0, policy_version 1877224 (0.0009) [2023-12-27 05:04:21,062][104569] Fps is (10 sec: 21299.2, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 962428928. Throughput: 0: 9808.3, 1: 9764.7. Samples: 962415624. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:04:21,062][104569] Avg episode reward: [(0, '8719.287'), (1, '9252.729')] [2023-12-27 05:04:21,414][105620] Updated weights for policy 1, policy_version 1881718 (0.0009) [2023-12-27 05:04:21,473][105620] Updated weights for policy 1, policy_version 1881728 (0.0009) [2023-12-27 05:04:21,527][105620] Updated weights for policy 1, policy_version 1881738 (0.0009) [2023-12-27 05:04:21,631][105692] Updated weights for policy 0, policy_version 1877234 (0.0009) [2023-12-27 05:04:21,699][105692] Updated weights for policy 0, policy_version 1877244 (0.0007) [2023-12-27 05:04:21,771][105692] Updated weights for policy 0, policy_version 1877254 (0.0008) [2023-12-27 05:04:21,823][105692] Updated weights for policy 0, policy_version 1877264 (0.0010) [2023-12-27 05:04:22,307][105620] Updated weights for policy 1, policy_version 1881748 (0.0009) [2023-12-27 05:04:22,366][105620] Updated weights for policy 1, policy_version 1881758 (0.0009) [2023-12-27 05:04:22,429][105620] Updated weights for policy 1, policy_version 1881768 (0.0009) [2023-12-27 05:04:22,550][105692] Updated weights for policy 0, policy_version 1877274 (0.0009) [2023-12-27 05:04:22,611][105692] Updated weights for policy 0, policy_version 1877284 (0.0008) [2023-12-27 05:04:22,671][105692] Updated weights for policy 0, policy_version 1877294 (0.0011) [2023-12-27 05:04:23,206][105620] Updated weights for policy 1, policy_version 1881778 (0.0009) [2023-12-27 05:04:23,249][105620] Updated weights for policy 1, policy_version 1881788 (0.0008) [2023-12-27 05:04:23,306][105620] Updated weights for policy 1, policy_version 1881798 (0.0007) [2023-12-27 05:04:23,350][105620] Updated weights for policy 1, policy_version 1881808 (0.0005) [2023-12-27 05:04:23,431][105692] Updated weights for policy 0, policy_version 1877304 (0.0010) [2023-12-27 05:04:23,480][105692] Updated weights for policy 0, policy_version 1877314 (0.0010) [2023-12-27 05:04:23,539][105692] Updated weights for policy 0, policy_version 1877324 (0.0010) [2023-12-27 05:04:23,982][105620] Updated weights for policy 1, policy_version 1881818 (0.0005) [2023-12-27 05:04:24,038][105620] Updated weights for policy 1, policy_version 1881828 (0.0005) [2023-12-27 05:04:24,084][105620] Updated weights for policy 1, policy_version 1881838 (0.0007) [2023-12-27 05:04:24,306][105692] Updated weights for policy 0, policy_version 1877334 (0.0010) [2023-12-27 05:04:24,368][105692] Updated weights for policy 0, policy_version 1877344 (0.0010) [2023-12-27 05:04:24,435][105692] Updated weights for policy 0, policy_version 1877354 (0.0010) [2023-12-27 05:04:24,682][105620] Updated weights for policy 1, policy_version 1881848 (0.0005) [2023-12-27 05:04:24,731][105620] Updated weights for policy 1, policy_version 1881858 (0.0005) [2023-12-27 05:04:24,789][105620] Updated weights for policy 1, policy_version 1881868 (0.0005) [2023-12-27 05:04:25,174][105692] Updated weights for policy 0, policy_version 1877364 (0.0011) [2023-12-27 05:04:25,232][105692] Updated weights for policy 0, policy_version 1877374 (0.0010) [2023-12-27 05:04:25,292][105692] Updated weights for policy 0, policy_version 1877384 (0.0011) [2023-12-27 05:04:25,393][105620] Updated weights for policy 1, policy_version 1881878 (0.0005) [2023-12-27 05:04:25,459][105620] Updated weights for policy 1, policy_version 1881888 (0.0006) [2023-12-27 05:04:25,522][105620] Updated weights for policy 1, policy_version 1881898 (0.0006) [2023-12-27 05:04:25,906][105692] Updated weights for policy 0, policy_version 1877394 (0.0009) [2023-12-27 05:04:25,957][105692] Updated weights for policy 0, policy_version 1877404 (0.0005) [2023-12-27 05:04:26,016][105692] Updated weights for policy 0, policy_version 1877414 (0.0009) [2023-12-27 05:04:26,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 962519040. Throughput: 0: 9769.6, 1: 9801.2. Samples: 962532420. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:04:26,063][104569] Avg episode reward: [(0, '8627.738'), (1, '9252.905')] [2023-12-27 05:04:26,066][105692] Updated weights for policy 0, policy_version 1877424 (0.0009) [2023-12-27 05:04:26,214][105620] Updated weights for policy 1, policy_version 1881908 (0.0010) [2023-12-27 05:04:26,266][105620] Updated weights for policy 1, policy_version 1881918 (0.0006) [2023-12-27 05:04:26,313][105620] Updated weights for policy 1, policy_version 1881928 (0.0005) [2023-12-27 05:04:26,732][105692] Updated weights for policy 0, policy_version 1877434 (0.0008) [2023-12-27 05:04:26,788][105692] Updated weights for policy 0, policy_version 1877444 (0.0008) [2023-12-27 05:04:26,846][105692] Updated weights for policy 0, policy_version 1877454 (0.0009) [2023-12-27 05:04:26,909][105620] Updated weights for policy 1, policy_version 1881938 (0.0005) [2023-12-27 05:04:26,960][105620] Updated weights for policy 1, policy_version 1881948 (0.0007) [2023-12-27 05:04:27,006][105620] Updated weights for policy 1, policy_version 1881958 (0.0008) [2023-12-27 05:04:27,056][105620] Updated weights for policy 1, policy_version 1881968 (0.0006) [2023-12-27 05:04:27,629][105692] Updated weights for policy 0, policy_version 1877464 (0.0009) [2023-12-27 05:04:27,688][105692] Updated weights for policy 0, policy_version 1877474 (0.0008) [2023-12-27 05:04:27,744][105620] Updated weights for policy 1, policy_version 1881978 (0.0008) [2023-12-27 05:04:27,754][105692] Updated weights for policy 0, policy_version 1877484 (0.0009) [2023-12-27 05:04:27,811][105620] Updated weights for policy 1, policy_version 1881988 (0.0006) [2023-12-27 05:04:27,873][105620] Updated weights for policy 1, policy_version 1881998 (0.0005) [2023-12-27 05:04:28,489][105620] Updated weights for policy 1, policy_version 1882008 (0.0008) [2023-12-27 05:04:28,536][105692] Updated weights for policy 0, policy_version 1877494 (0.0009) [2023-12-27 05:04:28,543][105620] Updated weights for policy 1, policy_version 1882018 (0.0007) [2023-12-27 05:04:28,590][105692] Updated weights for policy 0, policy_version 1877504 (0.0006) [2023-12-27 05:04:28,600][105620] Updated weights for policy 1, policy_version 1882028 (0.0007) [2023-12-27 05:04:28,652][105692] Updated weights for policy 0, policy_version 1877514 (0.0008) [2023-12-27 05:04:29,268][105692] Updated weights for policy 0, policy_version 1877524 (0.0008) [2023-12-27 05:04:29,323][105692] Updated weights for policy 0, policy_version 1877534 (0.0008) [2023-12-27 05:04:29,340][105620] Updated weights for policy 1, policy_version 1882038 (0.0007) [2023-12-27 05:04:29,389][105692] Updated weights for policy 0, policy_version 1877544 (0.0008) [2023-12-27 05:04:29,399][105620] Updated weights for policy 1, policy_version 1882048 (0.0006) [2023-12-27 05:04:29,454][105620] Updated weights for policy 1, policy_version 1882058 (0.0007) [2023-12-27 05:04:30,062][105692] Updated weights for policy 0, policy_version 1877554 (0.0007) [2023-12-27 05:04:30,113][105692] Updated weights for policy 0, policy_version 1877564 (0.0009) [2023-12-27 05:04:30,163][105692] Updated weights for policy 0, policy_version 1877574 (0.0009) [2023-12-27 05:04:30,223][105692] Updated weights for policy 0, policy_version 1877584 (0.0009) [2023-12-27 05:04:30,250][105620] Updated weights for policy 1, policy_version 1882068 (0.0009) [2023-12-27 05:04:30,307][105620] Updated weights for policy 1, policy_version 1882078 (0.0008) [2023-12-27 05:04:30,354][105620] Updated weights for policy 1, policy_version 1882088 (0.0009) [2023-12-27 05:04:30,847][105692] Updated weights for policy 0, policy_version 1877594 (0.0006) [2023-12-27 05:04:30,891][105692] Updated weights for policy 0, policy_version 1877604 (0.0010) [2023-12-27 05:04:30,936][105692] Updated weights for policy 0, policy_version 1877614 (0.0010) [2023-12-27 05:04:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19521.9). Total num frames: 962625536. Throughput: 0: 9769.6, 1: 9873.6. Samples: 962594236. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:04:31,062][104569] Avg episode reward: [(0, '8624.847'), (1, '9161.090')] [2023-12-27 05:04:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001877616_480739328.pth... [2023-12-27 05:04:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001882096_481886208.pth... [2023-12-27 05:04:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001876432_480436224.pth [2023-12-27 05:04:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001880944_481591296.pth [2023-12-27 05:04:31,191][105620] Updated weights for policy 1, policy_version 1882098 (0.0009) [2023-12-27 05:04:31,244][105620] Updated weights for policy 1, policy_version 1882108 (0.0009) [2023-12-27 05:04:31,311][105620] Updated weights for policy 1, policy_version 1882118 (0.0008) [2023-12-27 05:04:31,371][105620] Updated weights for policy 1, policy_version 1882128 (0.0008) [2023-12-27 05:04:31,734][105692] Updated weights for policy 0, policy_version 1877624 (0.0010) [2023-12-27 05:04:31,806][105692] Updated weights for policy 0, policy_version 1877634 (0.0008) [2023-12-27 05:04:31,860][105692] Updated weights for policy 0, policy_version 1877644 (0.0009) [2023-12-27 05:04:32,130][105620] Updated weights for policy 1, policy_version 1882138 (0.0006) [2023-12-27 05:04:32,192][105620] Updated weights for policy 1, policy_version 1882148 (0.0005) [2023-12-27 05:04:32,259][105620] Updated weights for policy 1, policy_version 1882158 (0.0009) [2023-12-27 05:04:32,524][105692] Updated weights for policy 0, policy_version 1877654 (0.0008) [2023-12-27 05:04:32,587][105692] Updated weights for policy 0, policy_version 1877664 (0.0005) [2023-12-27 05:04:32,662][105692] Updated weights for policy 0, policy_version 1877674 (0.0006) [2023-12-27 05:04:32,913][105620] Updated weights for policy 1, policy_version 1882168 (0.0009) [2023-12-27 05:04:32,964][105620] Updated weights for policy 1, policy_version 1882178 (0.0009) [2023-12-27 05:04:33,015][105620] Updated weights for policy 1, policy_version 1882188 (0.0009) [2023-12-27 05:04:33,315][105692] Updated weights for policy 0, policy_version 1877684 (0.0010) [2023-12-27 05:04:33,362][105692] Updated weights for policy 0, policy_version 1877694 (0.0010) [2023-12-27 05:04:33,410][105692] Updated weights for policy 0, policy_version 1877704 (0.0010) [2023-12-27 05:04:33,449][105585] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000001 [2023-12-27 05:04:33,764][105620] Updated weights for policy 1, policy_version 1882198 (0.0008) [2023-12-27 05:04:33,808][105620] Updated weights for policy 1, policy_version 1882208 (0.0008) [2023-12-27 05:04:33,853][105620] Updated weights for policy 1, policy_version 1882218 (0.0008) [2023-12-27 05:04:34,111][105692] Updated weights for policy 0, policy_version 1877714 (0.0008) [2023-12-27 05:04:34,169][105692] Updated weights for policy 0, policy_version 1877724 (0.0009) [2023-12-27 05:04:34,239][105692] Updated weights for policy 0, policy_version 1877734 (0.0008) [2023-12-27 05:04:34,306][105692] Updated weights for policy 0, policy_version 1877744 (0.0008) [2023-12-27 05:04:34,649][105620] Updated weights for policy 1, policy_version 1882228 (0.0008) [2023-12-27 05:04:34,706][105620] Updated weights for policy 1, policy_version 1882238 (0.0008) [2023-12-27 05:04:34,758][105620] Updated weights for policy 1, policy_version 1882248 (0.0008) [2023-12-27 05:04:35,074][105692] Updated weights for policy 0, policy_version 1877754 (0.0011) [2023-12-27 05:04:35,130][105692] Updated weights for policy 0, policy_version 1877764 (0.0011) [2023-12-27 05:04:35,190][105692] Updated weights for policy 0, policy_version 1877774 (0.0011) [2023-12-27 05:04:35,459][105620] Updated weights for policy 1, policy_version 1882258 (0.0008) [2023-12-27 05:04:35,508][105620] Updated weights for policy 1, policy_version 1882268 (0.0008) [2023-12-27 05:04:35,559][105620] Updated weights for policy 1, policy_version 1882278 (0.0007) [2023-12-27 05:04:35,604][105620] Updated weights for policy 1, policy_version 1882288 (0.0008) [2023-12-27 05:04:35,937][105692] Updated weights for policy 0, policy_version 1877784 (0.0010) [2023-12-27 05:04:35,989][105692] Updated weights for policy 0, policy_version 1877794 (0.0010) [2023-12-27 05:04:36,054][105692] Updated weights for policy 0, policy_version 1877804 (0.0011) [2023-12-27 05:04:36,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 962715648. Throughput: 0: 9780.2, 1: 9834.1. Samples: 962709932. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:04:36,063][104569] Avg episode reward: [(0, '7898.554'), (1, '9161.021')] [2023-12-27 05:04:36,373][105620] Updated weights for policy 1, policy_version 1882298 (0.0008) [2023-12-27 05:04:36,438][105620] Updated weights for policy 1, policy_version 1882308 (0.0009) [2023-12-27 05:04:36,491][105620] Updated weights for policy 1, policy_version 1882318 (0.0008) [2023-12-27 05:04:36,814][105692] Updated weights for policy 0, policy_version 1877814 (0.0011) [2023-12-27 05:04:36,880][105692] Updated weights for policy 0, policy_version 1877824 (0.0011) [2023-12-27 05:04:36,932][105692] Updated weights for policy 0, policy_version 1877834 (0.0010) [2023-12-27 05:04:37,255][105620] Updated weights for policy 1, policy_version 1882328 (0.0008) [2023-12-27 05:04:37,311][105620] Updated weights for policy 1, policy_version 1882338 (0.0009) [2023-12-27 05:04:37,368][105620] Updated weights for policy 1, policy_version 1882348 (0.0010) [2023-12-27 05:04:37,650][105692] Updated weights for policy 0, policy_version 1877844 (0.0010) [2023-12-27 05:04:37,698][105692] Updated weights for policy 0, policy_version 1877854 (0.0010) [2023-12-27 05:04:37,754][105692] Updated weights for policy 0, policy_version 1877864 (0.0010) [2023-12-27 05:04:38,129][105620] Updated weights for policy 1, policy_version 1882358 (0.0008) [2023-12-27 05:04:38,184][105620] Updated weights for policy 1, policy_version 1882368 (0.0008) [2023-12-27 05:04:38,235][105620] Updated weights for policy 1, policy_version 1882378 (0.0008) [2023-12-27 05:04:38,489][105692] Updated weights for policy 0, policy_version 1877874 (0.0010) [2023-12-27 05:04:38,560][105692] Updated weights for policy 0, policy_version 1877884 (0.0008) [2023-12-27 05:04:38,608][105692] Updated weights for policy 0, policy_version 1877894 (0.0008) [2023-12-27 05:04:38,656][105692] Updated weights for policy 0, policy_version 1877904 (0.0008) [2023-12-27 05:04:38,932][105620] Updated weights for policy 1, policy_version 1882388 (0.0010) [2023-12-27 05:04:38,994][105620] Updated weights for policy 1, policy_version 1882398 (0.0008) [2023-12-27 05:04:39,060][105620] Updated weights for policy 1, policy_version 1882408 (0.0008) [2023-12-27 05:04:39,417][105692] Updated weights for policy 0, policy_version 1877914 (0.0010) [2023-12-27 05:04:39,474][105692] Updated weights for policy 0, policy_version 1877924 (0.0010) [2023-12-27 05:04:39,526][105692] Updated weights for policy 0, policy_version 1877934 (0.0009) [2023-12-27 05:04:39,854][105620] Updated weights for policy 1, policy_version 1882418 (0.0009) [2023-12-27 05:04:39,917][105620] Updated weights for policy 1, policy_version 1882428 (0.0008) [2023-12-27 05:04:39,980][105620] Updated weights for policy 1, policy_version 1882438 (0.0008) [2023-12-27 05:04:40,034][105620] Updated weights for policy 1, policy_version 1882448 (0.0008) [2023-12-27 05:04:40,275][105692] Updated weights for policy 0, policy_version 1877944 (0.0008) [2023-12-27 05:04:40,331][105692] Updated weights for policy 0, policy_version 1877954 (0.0009) [2023-12-27 05:04:40,390][105692] Updated weights for policy 0, policy_version 1877964 (0.0009) [2023-12-27 05:04:40,763][105620] Updated weights for policy 1, policy_version 1882458 (0.0007) [2023-12-27 05:04:40,818][105620] Updated weights for policy 1, policy_version 1882468 (0.0006) [2023-12-27 05:04:40,877][105620] Updated weights for policy 1, policy_version 1882478 (0.0007) [2023-12-27 05:04:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 962813952. Throughput: 0: 9778.5, 1: 9850.7. Samples: 962822856. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:04:41,063][104569] Avg episode reward: [(0, '8081.214'), (1, '9068.034')] [2023-12-27 05:04:41,205][105692] Updated weights for policy 0, policy_version 1877974 (0.0010) [2023-12-27 05:04:41,270][105692] Updated weights for policy 0, policy_version 1877984 (0.0009) [2023-12-27 05:04:41,322][105692] Updated weights for policy 0, policy_version 1877994 (0.0008) [2023-12-27 05:04:41,543][105620] Updated weights for policy 1, policy_version 1882488 (0.0009) [2023-12-27 05:04:41,605][105620] Updated weights for policy 1, policy_version 1882498 (0.0009) [2023-12-27 05:04:41,668][105620] Updated weights for policy 1, policy_version 1882508 (0.0008) [2023-12-27 05:04:42,078][105692] Updated weights for policy 0, policy_version 1878004 (0.0009) [2023-12-27 05:04:42,147][105692] Updated weights for policy 0, policy_version 1878014 (0.0009) [2023-12-27 05:04:42,202][105692] Updated weights for policy 0, policy_version 1878024 (0.0010) [2023-12-27 05:04:42,364][105620] Updated weights for policy 1, policy_version 1882518 (0.0009) [2023-12-27 05:04:42,419][105620] Updated weights for policy 1, policy_version 1882528 (0.0008) [2023-12-27 05:04:42,466][105620] Updated weights for policy 1, policy_version 1882538 (0.0008) [2023-12-27 05:04:42,991][105692] Updated weights for policy 0, policy_version 1878034 (0.0009) [2023-12-27 05:04:43,049][105692] Updated weights for policy 0, policy_version 1878044 (0.0010) [2023-12-27 05:04:43,111][105692] Updated weights for policy 0, policy_version 1878054 (0.0010) [2023-12-27 05:04:43,154][105620] Updated weights for policy 1, policy_version 1882548 (0.0010) [2023-12-27 05:04:43,170][105692] Updated weights for policy 0, policy_version 1878064 (0.0010) [2023-12-27 05:04:43,220][105620] Updated weights for policy 1, policy_version 1882558 (0.0008) [2023-12-27 05:04:43,277][105620] Updated weights for policy 1, policy_version 1882568 (0.0008) [2023-12-27 05:04:43,933][105692] Updated weights for policy 0, policy_version 1878074 (0.0009) [2023-12-27 05:04:43,985][105692] Updated weights for policy 0, policy_version 1878084 (0.0009) [2023-12-27 05:04:44,029][105620] Updated weights for policy 1, policy_version 1882578 (0.0008) [2023-12-27 05:04:44,031][105692] Updated weights for policy 0, policy_version 1878094 (0.0008) [2023-12-27 05:04:44,078][105620] Updated weights for policy 1, policy_version 1882588 (0.0008) [2023-12-27 05:04:44,134][105620] Updated weights for policy 1, policy_version 1882598 (0.0009) [2023-12-27 05:04:44,189][105620] Updated weights for policy 1, policy_version 1882608 (0.0009) [2023-12-27 05:04:44,777][105692] Updated weights for policy 0, policy_version 1878104 (0.0008) [2023-12-27 05:04:44,843][105692] Updated weights for policy 0, policy_version 1878114 (0.0009) [2023-12-27 05:04:44,910][105692] Updated weights for policy 0, policy_version 1878124 (0.0008) [2023-12-27 05:04:45,021][105620] Updated weights for policy 1, policy_version 1882618 (0.0009) [2023-12-27 05:04:45,079][105620] Updated weights for policy 1, policy_version 1882628 (0.0010) [2023-12-27 05:04:45,151][105620] Updated weights for policy 1, policy_version 1882638 (0.0009) [2023-12-27 05:04:45,478][105692] Updated weights for policy 0, policy_version 1878134 (0.0010) [2023-12-27 05:04:45,531][105692] Updated weights for policy 0, policy_version 1878144 (0.0011) [2023-12-27 05:04:45,577][105692] Updated weights for policy 0, policy_version 1878154 (0.0010) [2023-12-27 05:04:45,983][105620] Updated weights for policy 1, policy_version 1882648 (0.0009) [2023-12-27 05:04:46,036][105620] Updated weights for policy 1, policy_version 1882658 (0.0009) [2023-12-27 05:04:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 962904064. Throughput: 0: 9719.8, 1: 9894.3. Samples: 962878824. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:04:46,062][104569] Avg episode reward: [(0, '8444.629'), (1, '9160.368')] [2023-12-27 05:04:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001878160_480878592.pth... [2023-12-27 05:04:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001877040_480591872.pth [2023-12-27 05:04:46,087][105620] Updated weights for policy 1, policy_version 1882668 (0.0008) [2023-12-27 05:04:46,109][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001882672_482033664.pth... [2023-12-27 05:04:46,113][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001881520_481738752.pth [2023-12-27 05:04:46,265][105692] Updated weights for policy 0, policy_version 1878164 (0.0008) [2023-12-27 05:04:46,310][105692] Updated weights for policy 0, policy_version 1878174 (0.0005) [2023-12-27 05:04:46,357][105692] Updated weights for policy 0, policy_version 1878184 (0.0008) [2023-12-27 05:04:46,894][105620] Updated weights for policy 1, policy_version 1882678 (0.0009) [2023-12-27 05:04:46,949][105620] Updated weights for policy 1, policy_version 1882688 (0.0010) [2023-12-27 05:04:47,003][105620] Updated weights for policy 1, policy_version 1882698 (0.0009) [2023-12-27 05:04:47,064][105692] Updated weights for policy 0, policy_version 1878194 (0.0008) [2023-12-27 05:04:47,121][105692] Updated weights for policy 0, policy_version 1878204 (0.0005) [2023-12-27 05:04:47,180][105692] Updated weights for policy 0, policy_version 1878214 (0.0005) [2023-12-27 05:04:47,239][105692] Updated weights for policy 0, policy_version 1878224 (0.0005) [2023-12-27 05:04:47,803][105692] Updated weights for policy 0, policy_version 1878234 (0.0005) [2023-12-27 05:04:47,859][105692] Updated weights for policy 0, policy_version 1878244 (0.0005) [2023-12-27 05:04:47,876][105620] Updated weights for policy 1, policy_version 1882708 (0.0009) [2023-12-27 05:04:47,907][105692] Updated weights for policy 0, policy_version 1878254 (0.0006) [2023-12-27 05:04:47,929][105620] Updated weights for policy 1, policy_version 1882718 (0.0008) [2023-12-27 05:04:47,986][105620] Updated weights for policy 1, policy_version 1882728 (0.0008) [2023-12-27 05:04:48,524][105692] Updated weights for policy 0, policy_version 1878264 (0.0008) [2023-12-27 05:04:48,572][105692] Updated weights for policy 0, policy_version 1878274 (0.0009) [2023-12-27 05:04:48,634][105692] Updated weights for policy 0, policy_version 1878284 (0.0009) [2023-12-27 05:04:48,792][105620] Updated weights for policy 1, policy_version 1882738 (0.0009) [2023-12-27 05:04:48,848][105620] Updated weights for policy 1, policy_version 1882748 (0.0009) [2023-12-27 05:04:48,912][105620] Updated weights for policy 1, policy_version 1882758 (0.0005) [2023-12-27 05:04:48,973][105620] Updated weights for policy 1, policy_version 1882768 (0.0005) [2023-12-27 05:04:49,477][105692] Updated weights for policy 0, policy_version 1878294 (0.0007) [2023-12-27 05:04:49,529][105692] Updated weights for policy 0, policy_version 1878304 (0.0005) [2023-12-27 05:04:49,547][105620] Updated weights for policy 1, policy_version 1882778 (0.0009) [2023-12-27 05:04:49,591][105692] Updated weights for policy 0, policy_version 1878314 (0.0006) [2023-12-27 05:04:49,608][105620] Updated weights for policy 1, policy_version 1882788 (0.0005) [2023-12-27 05:04:49,676][105620] Updated weights for policy 1, policy_version 1882798 (0.0006) [2023-12-27 05:04:50,311][105692] Updated weights for policy 0, policy_version 1878324 (0.0007) [2023-12-27 05:04:50,370][105692] Updated weights for policy 0, policy_version 1878334 (0.0007) [2023-12-27 05:04:50,382][105620] Updated weights for policy 1, policy_version 1882808 (0.0006) [2023-12-27 05:04:50,431][105692] Updated weights for policy 0, policy_version 1878344 (0.0009) [2023-12-27 05:04:50,442][105620] Updated weights for policy 1, policy_version 1882818 (0.0005) [2023-12-27 05:04:50,497][105620] Updated weights for policy 1, policy_version 1882828 (0.0006) [2023-12-27 05:04:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 963002368. Throughput: 0: 9879.9, 1: 9769.4. Samples: 962995084. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:04:51,062][104569] Avg episode reward: [(0, '8084.481'), (1, '9345.126')] [2023-12-27 05:04:51,205][105620] Updated weights for policy 1, policy_version 1882838 (0.0008) [2023-12-27 05:04:51,235][105692] Updated weights for policy 0, policy_version 1878354 (0.0008) [2023-12-27 05:04:51,264][105620] Updated weights for policy 1, policy_version 1882848 (0.0011) [2023-12-27 05:04:51,295][105692] Updated weights for policy 0, policy_version 1878364 (0.0007) [2023-12-27 05:04:51,323][105620] Updated weights for policy 1, policy_version 1882858 (0.0010) [2023-12-27 05:04:51,358][105692] Updated weights for policy 0, policy_version 1878374 (0.0008) [2023-12-27 05:04:51,420][105692] Updated weights for policy 0, policy_version 1878384 (0.0007) [2023-12-27 05:04:52,101][105620] Updated weights for policy 1, policy_version 1882868 (0.0011) [2023-12-27 05:04:52,134][105692] Updated weights for policy 0, policy_version 1878394 (0.0006) [2023-12-27 05:04:52,163][105620] Updated weights for policy 1, policy_version 1882878 (0.0011) [2023-12-27 05:04:52,186][105692] Updated weights for policy 0, policy_version 1878404 (0.0006) [2023-12-27 05:04:52,223][105620] Updated weights for policy 1, policy_version 1882888 (0.0009) [2023-12-27 05:04:52,230][105692] Updated weights for policy 0, policy_version 1878414 (0.0008) [2023-12-27 05:04:52,893][105620] Updated weights for policy 1, policy_version 1882898 (0.0010) [2023-12-27 05:04:52,949][105620] Updated weights for policy 1, policy_version 1882908 (0.0006) [2023-12-27 05:04:53,015][105620] Updated weights for policy 1, policy_version 1882918 (0.0005) [2023-12-27 05:04:53,031][105692] Updated weights for policy 0, policy_version 1878424 (0.0008) [2023-12-27 05:04:53,075][105620] Updated weights for policy 1, policy_version 1882928 (0.0005) [2023-12-27 05:04:53,088][105692] Updated weights for policy 0, policy_version 1878434 (0.0009) [2023-12-27 05:04:53,147][105692] Updated weights for policy 0, policy_version 1878444 (0.0008) [2023-12-27 05:04:53,701][105620] Updated weights for policy 1, policy_version 1882938 (0.0010) [2023-12-27 05:04:53,759][105620] Updated weights for policy 1, policy_version 1882948 (0.0010) [2023-12-27 05:04:53,815][105620] Updated weights for policy 1, policy_version 1882958 (0.0010) [2023-12-27 05:04:53,878][105692] Updated weights for policy 0, policy_version 1878454 (0.0007) [2023-12-27 05:04:53,934][105692] Updated weights for policy 0, policy_version 1878464 (0.0009) [2023-12-27 05:04:53,985][105692] Updated weights for policy 0, policy_version 1878475 (0.0009) [2023-12-27 05:04:54,470][105620] Updated weights for policy 1, policy_version 1882968 (0.0009) [2023-12-27 05:04:54,518][105620] Updated weights for policy 1, policy_version 1882978 (0.0010) [2023-12-27 05:04:54,576][105620] Updated weights for policy 1, policy_version 1882988 (0.0010) [2023-12-27 05:04:54,820][105692] Updated weights for policy 0, policy_version 1878485 (0.0009) [2023-12-27 05:04:54,875][105692] Updated weights for policy 0, policy_version 1878495 (0.0008) [2023-12-27 05:04:54,923][105692] Updated weights for policy 0, policy_version 1878505 (0.0008) [2023-12-27 05:04:55,329][105620] Updated weights for policy 1, policy_version 1882998 (0.0010) [2023-12-27 05:04:55,389][105620] Updated weights for policy 1, policy_version 1883008 (0.0010) [2023-12-27 05:04:55,444][105620] Updated weights for policy 1, policy_version 1883018 (0.0010) [2023-12-27 05:04:55,696][105692] Updated weights for policy 0, policy_version 1878515 (0.0008) [2023-12-27 05:04:55,740][105692] Updated weights for policy 0, policy_version 1878525 (0.0008) [2023-12-27 05:04:55,785][105692] Updated weights for policy 0, policy_version 1878535 (0.0008) [2023-12-27 05:04:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 963100672. Throughput: 0: 9749.7, 1: 9773.8. Samples: 963109240. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:04:56,062][104569] Avg episode reward: [(0, '7898.796'), (1, '9252.888')] [2023-12-27 05:04:56,183][105620] Updated weights for policy 1, policy_version 1883028 (0.0010) [2023-12-27 05:04:56,246][105620] Updated weights for policy 1, policy_version 1883038 (0.0011) [2023-12-27 05:04:56,307][105620] Updated weights for policy 1, policy_version 1883048 (0.0010) [2023-12-27 05:04:56,572][105692] Updated weights for policy 0, policy_version 1878545 (0.0007) [2023-12-27 05:04:56,624][105692] Updated weights for policy 0, policy_version 1878555 (0.0008) [2023-12-27 05:04:56,686][105692] Updated weights for policy 0, policy_version 1878565 (0.0008) [2023-12-27 05:04:56,739][105692] Updated weights for policy 0, policy_version 1878575 (0.0008) [2023-12-27 05:04:57,046][105620] Updated weights for policy 1, policy_version 1883058 (0.0010) [2023-12-27 05:04:57,107][105620] Updated weights for policy 1, policy_version 1883068 (0.0010) [2023-12-27 05:04:57,166][105620] Updated weights for policy 1, policy_version 1883078 (0.0010) [2023-12-27 05:04:57,227][105620] Updated weights for policy 1, policy_version 1883088 (0.0010) [2023-12-27 05:04:57,515][105692] Updated weights for policy 0, policy_version 1878585 (0.0008) [2023-12-27 05:04:57,563][105692] Updated weights for policy 0, policy_version 1878595 (0.0008) [2023-12-27 05:04:57,615][105692] Updated weights for policy 0, policy_version 1878605 (0.0008) [2023-12-27 05:04:57,959][105620] Updated weights for policy 1, policy_version 1883098 (0.0008) [2023-12-27 05:04:58,026][105620] Updated weights for policy 1, policy_version 1883108 (0.0009) [2023-12-27 05:04:58,076][105620] Updated weights for policy 1, policy_version 1883118 (0.0009) [2023-12-27 05:04:58,366][105692] Updated weights for policy 0, policy_version 1878615 (0.0008) [2023-12-27 05:04:58,425][105692] Updated weights for policy 0, policy_version 1878625 (0.0009) [2023-12-27 05:04:58,485][105692] Updated weights for policy 0, policy_version 1878635 (0.0009) [2023-12-27 05:04:58,885][105620] Updated weights for policy 1, policy_version 1883128 (0.0009) [2023-12-27 05:04:58,938][105620] Updated weights for policy 1, policy_version 1883138 (0.0007) [2023-12-27 05:04:58,987][105620] Updated weights for policy 1, policy_version 1883148 (0.0005) [2023-12-27 05:04:59,329][105692] Updated weights for policy 0, policy_version 1878645 (0.0008) [2023-12-27 05:04:59,392][105692] Updated weights for policy 0, policy_version 1878655 (0.0009) [2023-12-27 05:04:59,450][105692] Updated weights for policy 0, policy_version 1878665 (0.0009) [2023-12-27 05:04:59,661][105620] Updated weights for policy 1, policy_version 1883158 (0.0007) [2023-12-27 05:04:59,719][105620] Updated weights for policy 1, policy_version 1883168 (0.0009) [2023-12-27 05:04:59,777][105620] Updated weights for policy 1, policy_version 1883178 (0.0008) [2023-12-27 05:05:00,198][105692] Updated weights for policy 0, policy_version 1878675 (0.0009) [2023-12-27 05:05:00,260][105692] Updated weights for policy 0, policy_version 1878685 (0.0009) [2023-12-27 05:05:00,324][105692] Updated weights for policy 0, policy_version 1878695 (0.0009) [2023-12-27 05:05:00,526][105620] Updated weights for policy 1, policy_version 1883188 (0.0006) [2023-12-27 05:05:00,578][105620] Updated weights for policy 1, policy_version 1883198 (0.0005) [2023-12-27 05:05:00,635][105620] Updated weights for policy 1, policy_version 1883208 (0.0005) [2023-12-27 05:05:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 963190784. Throughput: 0: 9683.9, 1: 9740.3. Samples: 963164168. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:05:01,062][104569] Avg episode reward: [(0, '8351.668'), (1, '8975.958')] [2023-12-27 05:05:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001878704_481017856.pth... [2023-12-27 05:05:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001883216_482172928.pth... [2023-12-27 05:05:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001877616_480739328.pth [2023-12-27 05:05:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001882096_481886208.pth [2023-12-27 05:05:01,107][105692] Updated weights for policy 0, policy_version 1878705 (0.0008) [2023-12-27 05:05:01,178][105692] Updated weights for policy 0, policy_version 1878715 (0.0009) [2023-12-27 05:05:01,240][105692] Updated weights for policy 0, policy_version 1878725 (0.0009) [2023-12-27 05:05:01,307][105692] Updated weights for policy 0, policy_version 1878735 (0.0008) [2023-12-27 05:05:01,335][105620] Updated weights for policy 1, policy_version 1883218 (0.0006) [2023-12-27 05:05:01,410][105620] Updated weights for policy 1, policy_version 1883228 (0.0009) [2023-12-27 05:05:01,468][105620] Updated weights for policy 1, policy_version 1883238 (0.0006) [2023-12-27 05:05:01,526][105620] Updated weights for policy 1, policy_version 1883248 (0.0005) [2023-12-27 05:05:02,025][105692] Updated weights for policy 0, policy_version 1878745 (0.0006) [2023-12-27 05:05:02,086][105692] Updated weights for policy 0, policy_version 1878755 (0.0006) [2023-12-27 05:05:02,148][105692] Updated weights for policy 0, policy_version 1878765 (0.0005) [2023-12-27 05:05:02,171][105620] Updated weights for policy 1, policy_version 1883258 (0.0008) [2023-12-27 05:05:02,225][105620] Updated weights for policy 1, policy_version 1883268 (0.0010) [2023-12-27 05:05:02,277][105620] Updated weights for policy 1, policy_version 1883278 (0.0009) [2023-12-27 05:05:02,755][105692] Updated weights for policy 0, policy_version 1878775 (0.0008) [2023-12-27 05:05:02,809][105692] Updated weights for policy 0, policy_version 1878785 (0.0006) [2023-12-27 05:05:02,858][105692] Updated weights for policy 0, policy_version 1878795 (0.0005) [2023-12-27 05:05:03,153][105620] Updated weights for policy 1, policy_version 1883288 (0.0010) [2023-12-27 05:05:03,215][105620] Updated weights for policy 1, policy_version 1883298 (0.0010) [2023-12-27 05:05:03,269][105620] Updated weights for policy 1, policy_version 1883308 (0.0010) [2023-12-27 05:05:03,397][105692] Updated weights for policy 0, policy_version 1878805 (0.0005) [2023-12-27 05:05:03,448][105692] Updated weights for policy 0, policy_version 1878815 (0.0005) [2023-12-27 05:05:03,493][105692] Updated weights for policy 0, policy_version 1878825 (0.0005) [2023-12-27 05:05:03,977][105620] Updated weights for policy 1, policy_version 1883318 (0.0009) [2023-12-27 05:05:04,026][105692] Updated weights for policy 0, policy_version 1878835 (0.0005) [2023-12-27 05:05:04,029][105620] Updated weights for policy 1, policy_version 1883328 (0.0009) [2023-12-27 05:05:04,092][105692] Updated weights for policy 0, policy_version 1878845 (0.0005) [2023-12-27 05:05:04,094][105620] Updated weights for policy 1, policy_version 1883338 (0.0009) [2023-12-27 05:05:04,153][105692] Updated weights for policy 0, policy_version 1878855 (0.0007) [2023-12-27 05:05:04,773][105692] Updated weights for policy 0, policy_version 1878865 (0.0007) [2023-12-27 05:05:04,830][105692] Updated weights for policy 0, policy_version 1878875 (0.0008) [2023-12-27 05:05:04,885][105692] Updated weights for policy 0, policy_version 1878885 (0.0009) [2023-12-27 05:05:04,914][105620] Updated weights for policy 1, policy_version 1883348 (0.0007) [2023-12-27 05:05:04,938][105692] Updated weights for policy 0, policy_version 1878895 (0.0008) [2023-12-27 05:05:04,965][105620] Updated weights for policy 1, policy_version 1883358 (0.0008) [2023-12-27 05:05:05,026][105620] Updated weights for policy 1, policy_version 1883368 (0.0008) [2023-12-27 05:05:05,619][105692] Updated weights for policy 0, policy_version 1878905 (0.0009) [2023-12-27 05:05:05,676][105692] Updated weights for policy 0, policy_version 1878915 (0.0009) [2023-12-27 05:05:05,738][105692] Updated weights for policy 0, policy_version 1878925 (0.0009) [2023-12-27 05:05:05,774][105620] Updated weights for policy 1, policy_version 1883378 (0.0009) [2023-12-27 05:05:05,840][105620] Updated weights for policy 1, policy_version 1883388 (0.0009) [2023-12-27 05:05:05,898][105620] Updated weights for policy 1, policy_version 1883398 (0.0009) [2023-12-27 05:05:05,960][105620] Updated weights for policy 1, policy_version 1883408 (0.0009) [2023-12-27 05:05:06,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19660.7, 300 sec: 19466.4). Total num frames: 963297280. Throughput: 0: 9650.2, 1: 9627.3. Samples: 963283116. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:05:06,063][104569] Avg episode reward: [(0, '8257.445'), (1, '8883.848')] [2023-12-27 05:05:06,518][105692] Updated weights for policy 0, policy_version 1878935 (0.0009) [2023-12-27 05:05:06,581][105692] Updated weights for policy 0, policy_version 1878945 (0.0009) [2023-12-27 05:05:06,645][105692] Updated weights for policy 0, policy_version 1878955 (0.0009) [2023-12-27 05:05:06,714][105620] Updated weights for policy 1, policy_version 1883418 (0.0008) [2023-12-27 05:05:06,779][105620] Updated weights for policy 1, policy_version 1883428 (0.0009) [2023-12-27 05:05:06,850][105620] Updated weights for policy 1, policy_version 1883438 (0.0010) [2023-12-27 05:05:07,439][105620] Updated weights for policy 1, policy_version 1883448 (0.0006) [2023-12-27 05:05:07,488][105692] Updated weights for policy 0, policy_version 1878965 (0.0008) [2023-12-27 05:05:07,492][105620] Updated weights for policy 1, policy_version 1883458 (0.0005) [2023-12-27 05:05:07,543][105692] Updated weights for policy 0, policy_version 1878975 (0.0009) [2023-12-27 05:05:07,559][105620] Updated weights for policy 1, policy_version 1883468 (0.0007) [2023-12-27 05:05:07,599][105692] Updated weights for policy 0, policy_version 1878985 (0.0008) [2023-12-27 05:05:08,186][105620] Updated weights for policy 1, policy_version 1883478 (0.0005) [2023-12-27 05:05:08,248][105692] Updated weights for policy 0, policy_version 1878995 (0.0005) [2023-12-27 05:05:08,256][105620] Updated weights for policy 1, policy_version 1883488 (0.0007) [2023-12-27 05:05:08,299][105692] Updated weights for policy 0, policy_version 1879005 (0.0005) [2023-12-27 05:05:08,315][105620] Updated weights for policy 1, policy_version 1883498 (0.0008) [2023-12-27 05:05:08,353][105692] Updated weights for policy 0, policy_version 1879015 (0.0008) [2023-12-27 05:05:09,001][105692] Updated weights for policy 0, policy_version 1879025 (0.0007) [2023-12-27 05:05:09,060][105692] Updated weights for policy 0, policy_version 1879035 (0.0010) [2023-12-27 05:05:09,077][105620] Updated weights for policy 1, policy_version 1883508 (0.0007) [2023-12-27 05:05:09,119][105692] Updated weights for policy 0, policy_version 1879045 (0.0011) [2023-12-27 05:05:09,130][105620] Updated weights for policy 1, policy_version 1883518 (0.0006) [2023-12-27 05:05:09,178][105692] Updated weights for policy 0, policy_version 1879055 (0.0011) [2023-12-27 05:05:09,180][105620] Updated weights for policy 1, policy_version 1883528 (0.0008) [2023-12-27 05:05:09,927][105692] Updated weights for policy 0, policy_version 1879065 (0.0009) [2023-12-27 05:05:09,932][105620] Updated weights for policy 1, policy_version 1883538 (0.0008) [2023-12-27 05:05:09,988][105692] Updated weights for policy 0, policy_version 1879075 (0.0007) [2023-12-27 05:05:09,992][105620] Updated weights for policy 1, policy_version 1883548 (0.0008) [2023-12-27 05:05:10,044][105692] Updated weights for policy 0, policy_version 1879085 (0.0008) [2023-12-27 05:05:10,061][105620] Updated weights for policy 1, policy_version 1883558 (0.0009) [2023-12-27 05:05:10,130][105620] Updated weights for policy 1, policy_version 1883568 (0.0006) [2023-12-27 05:05:10,748][105620] Updated weights for policy 1, policy_version 1883578 (0.0008) [2023-12-27 05:05:10,808][105620] Updated weights for policy 1, policy_version 1883588 (0.0005) [2023-12-27 05:05:10,870][105620] Updated weights for policy 1, policy_version 1883598 (0.0005) [2023-12-27 05:05:10,880][105692] Updated weights for policy 0, policy_version 1879095 (0.0008) [2023-12-27 05:05:10,938][105692] Updated weights for policy 0, policy_version 1879105 (0.0010) [2023-12-27 05:05:10,989][105692] Updated weights for policy 0, policy_version 1879115 (0.0008) [2023-12-27 05:05:11,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 963395584. Throughput: 0: 9684.5, 1: 9581.3. Samples: 963399376. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:05:11,063][104569] Avg episode reward: [(0, '8259.188'), (1, '8976.185')] [2023-12-27 05:05:11,588][105620] Updated weights for policy 1, policy_version 1883608 (0.0010) [2023-12-27 05:05:11,656][105620] Updated weights for policy 1, policy_version 1883618 (0.0010) [2023-12-27 05:05:11,725][105620] Updated weights for policy 1, policy_version 1883628 (0.0011) [2023-12-27 05:05:11,769][105692] Updated weights for policy 0, policy_version 1879125 (0.0009) [2023-12-27 05:05:11,828][105692] Updated weights for policy 0, policy_version 1879135 (0.0008) [2023-12-27 05:05:11,895][105692] Updated weights for policy 0, policy_version 1879145 (0.0009) [2023-12-27 05:05:12,493][105620] Updated weights for policy 1, policy_version 1883638 (0.0011) [2023-12-27 05:05:12,547][105620] Updated weights for policy 1, policy_version 1883648 (0.0010) [2023-12-27 05:05:12,603][105620] Updated weights for policy 1, policy_version 1883658 (0.0010) [2023-12-27 05:05:12,703][105692] Updated weights for policy 0, policy_version 1879155 (0.0009) [2023-12-27 05:05:12,760][105692] Updated weights for policy 0, policy_version 1879165 (0.0007) [2023-12-27 05:05:12,824][105692] Updated weights for policy 0, policy_version 1879175 (0.0008) [2023-12-27 05:05:13,371][105620] Updated weights for policy 1, policy_version 1883668 (0.0010) [2023-12-27 05:05:13,431][105620] Updated weights for policy 1, policy_version 1883678 (0.0011) [2023-12-27 05:05:13,483][105620] Updated weights for policy 1, policy_version 1883688 (0.0010) [2023-12-27 05:05:13,508][105692] Updated weights for policy 0, policy_version 1879185 (0.0008) [2023-12-27 05:05:13,573][105692] Updated weights for policy 0, policy_version 1879195 (0.0007) [2023-12-27 05:05:13,630][105692] Updated weights for policy 0, policy_version 1879205 (0.0008) [2023-12-27 05:05:13,684][105692] Updated weights for policy 0, policy_version 1879215 (0.0008) [2023-12-27 05:05:14,179][105620] Updated weights for policy 1, policy_version 1883698 (0.0010) [2023-12-27 05:05:14,223][105620] Updated weights for policy 1, policy_version 1883708 (0.0010) [2023-12-27 05:05:14,267][105620] Updated weights for policy 1, policy_version 1883718 (0.0010) [2023-12-27 05:05:14,314][105620] Updated weights for policy 1, policy_version 1883728 (0.0010) [2023-12-27 05:05:14,457][105692] Updated weights for policy 0, policy_version 1879226 (0.0010) [2023-12-27 05:05:14,505][105692] Updated weights for policy 0, policy_version 1879237 (0.0009) [2023-12-27 05:05:14,555][105692] Updated weights for policy 0, policy_version 1879247 (0.0008) [2023-12-27 05:05:14,995][105620] Updated weights for policy 1, policy_version 1883738 (0.0007) [2023-12-27 05:05:15,052][105620] Updated weights for policy 1, policy_version 1883748 (0.0006) [2023-12-27 05:05:15,112][105620] Updated weights for policy 1, policy_version 1883758 (0.0006) [2023-12-27 05:05:15,304][105692] Updated weights for policy 0, policy_version 1879257 (0.0007) [2023-12-27 05:05:15,377][105692] Updated weights for policy 0, policy_version 1879267 (0.0006) [2023-12-27 05:05:15,448][105692] Updated weights for policy 0, policy_version 1879277 (0.0006) [2023-12-27 05:05:15,672][105620] Updated weights for policy 1, policy_version 1883768 (0.0006) [2023-12-27 05:05:15,739][105620] Updated weights for policy 1, policy_version 1883778 (0.0006) [2023-12-27 05:05:15,792][105620] Updated weights for policy 1, policy_version 1883788 (0.0006) [2023-12-27 05:05:16,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 963485696. Throughput: 0: 9631.9, 1: 9486.5. Samples: 963454564. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:05:16,063][104569] Avg episode reward: [(0, '8447.122'), (1, '9252.936')] [2023-12-27 05:05:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001879280_481165312.pth... [2023-12-27 05:05:16,071][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001883792_482320384.pth... [2023-12-27 05:05:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001878160_480878592.pth [2023-12-27 05:05:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001882672_482033664.pth [2023-12-27 05:05:16,132][105692] Updated weights for policy 0, policy_version 1879287 (0.0009) [2023-12-27 05:05:16,187][105692] Updated weights for policy 0, policy_version 1879297 (0.0009) [2023-12-27 05:05:16,238][105692] Updated weights for policy 0, policy_version 1879307 (0.0009) [2023-12-27 05:05:16,410][105620] Updated weights for policy 1, policy_version 1883798 (0.0008) [2023-12-27 05:05:16,469][105620] Updated weights for policy 1, policy_version 1883809 (0.0010) [2023-12-27 05:05:16,522][105620] Updated weights for policy 1, policy_version 1883819 (0.0010) [2023-12-27 05:05:16,821][105692] Updated weights for policy 0, policy_version 1879317 (0.0008) [2023-12-27 05:05:16,881][105692] Updated weights for policy 0, policy_version 1879327 (0.0006) [2023-12-27 05:05:16,930][105692] Updated weights for policy 0, policy_version 1879337 (0.0005) [2023-12-27 05:05:17,138][105620] Updated weights for policy 1, policy_version 1883829 (0.0010) [2023-12-27 05:05:17,192][105620] Updated weights for policy 1, policy_version 1883840 (0.0010) [2023-12-27 05:05:17,245][105620] Updated weights for policy 1, policy_version 1883851 (0.0010) [2023-12-27 05:05:17,448][105692] Updated weights for policy 0, policy_version 1879347 (0.0005) [2023-12-27 05:05:17,514][105692] Updated weights for policy 0, policy_version 1879357 (0.0005) [2023-12-27 05:05:17,562][105692] Updated weights for policy 0, policy_version 1879367 (0.0005) [2023-12-27 05:05:18,075][105692] Updated weights for policy 0, policy_version 1879377 (0.0006) [2023-12-27 05:05:18,124][105692] Updated weights for policy 0, policy_version 1879387 (0.0005) [2023-12-27 05:05:18,180][105620] Updated weights for policy 1, policy_version 1883862 (0.0010) [2023-12-27 05:05:18,182][105692] Updated weights for policy 0, policy_version 1879397 (0.0005) [2023-12-27 05:05:18,236][105620] Updated weights for policy 1, policy_version 1883872 (0.0008) [2023-12-27 05:05:18,242][105692] Updated weights for policy 0, policy_version 1879407 (0.0006) [2023-12-27 05:05:18,288][105620] Updated weights for policy 1, policy_version 1883882 (0.0009) [2023-12-27 05:05:18,950][105620] Updated weights for policy 1, policy_version 1883892 (0.0009) [2023-12-27 05:05:18,994][105692] Updated weights for policy 0, policy_version 1879417 (0.0008) [2023-12-27 05:05:19,011][105620] Updated weights for policy 1, policy_version 1883902 (0.0009) [2023-12-27 05:05:19,055][105692] Updated weights for policy 0, policy_version 1879427 (0.0009) [2023-12-27 05:05:19,064][105620] Updated weights for policy 1, policy_version 1883912 (0.0009) [2023-12-27 05:05:19,103][105692] Updated weights for policy 0, policy_version 1879437 (0.0009) [2023-12-27 05:05:19,830][105620] Updated weights for policy 1, policy_version 1883922 (0.0009) [2023-12-27 05:05:19,854][105692] Updated weights for policy 0, policy_version 1879447 (0.0009) [2023-12-27 05:05:19,890][105620] Updated weights for policy 1, policy_version 1883932 (0.0008) [2023-12-27 05:05:19,916][105692] Updated weights for policy 0, policy_version 1879457 (0.0011) [2023-12-27 05:05:19,959][105620] Updated weights for policy 1, policy_version 1883942 (0.0007) [2023-12-27 05:05:19,980][105692] Updated weights for policy 0, policy_version 1879467 (0.0011) [2023-12-27 05:05:20,023][105620] Updated weights for policy 1, policy_version 1883952 (0.0006) [2023-12-27 05:05:20,741][105692] Updated weights for policy 0, policy_version 1879477 (0.0011) [2023-12-27 05:05:20,796][105620] Updated weights for policy 1, policy_version 1883962 (0.0007) [2023-12-27 05:05:20,802][105692] Updated weights for policy 0, policy_version 1879487 (0.0011) [2023-12-27 05:05:20,859][105620] Updated weights for policy 1, policy_version 1883972 (0.0006) [2023-12-27 05:05:20,867][105692] Updated weights for policy 0, policy_version 1879497 (0.0010) [2023-12-27 05:05:20,921][105620] Updated weights for policy 1, policy_version 1883982 (0.0006) [2023-12-27 05:05:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 963592192. Throughput: 0: 9674.1, 1: 9607.4. Samples: 963577600. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:05:21,063][104569] Avg episode reward: [(0, '8451.795'), (1, '9252.992')] [2023-12-27 05:05:21,649][105692] Updated weights for policy 0, policy_version 1879507 (0.0011) [2023-12-27 05:05:21,671][105620] Updated weights for policy 1, policy_version 1883992 (0.0006) [2023-12-27 05:05:21,715][105692] Updated weights for policy 0, policy_version 1879517 (0.0010) [2023-12-27 05:05:21,739][105620] Updated weights for policy 1, policy_version 1884002 (0.0007) [2023-12-27 05:05:21,778][105692] Updated weights for policy 0, policy_version 1879527 (0.0009) [2023-12-27 05:05:21,804][105620] Updated weights for policy 1, policy_version 1884012 (0.0006) [2023-12-27 05:05:22,545][105620] Updated weights for policy 1, policy_version 1884022 (0.0007) [2023-12-27 05:05:22,596][105692] Updated weights for policy 0, policy_version 1879537 (0.0008) [2023-12-27 05:05:22,612][105620] Updated weights for policy 1, policy_version 1884032 (0.0006) [2023-12-27 05:05:22,657][105692] Updated weights for policy 0, policy_version 1879547 (0.0008) [2023-12-27 05:05:22,676][105620] Updated weights for policy 1, policy_version 1884042 (0.0007) [2023-12-27 05:05:22,727][105692] Updated weights for policy 0, policy_version 1879557 (0.0008) [2023-12-27 05:05:22,790][105692] Updated weights for policy 0, policy_version 1879567 (0.0009) [2023-12-27 05:05:23,307][105620] Updated weights for policy 1, policy_version 1884052 (0.0008) [2023-12-27 05:05:23,373][105620] Updated weights for policy 1, policy_version 1884062 (0.0009) [2023-12-27 05:05:23,445][105620] Updated weights for policy 1, policy_version 1884072 (0.0008) [2023-12-27 05:05:23,564][105692] Updated weights for policy 0, policy_version 1879577 (0.0011) [2023-12-27 05:05:23,623][105692] Updated weights for policy 0, policy_version 1879587 (0.0010) [2023-12-27 05:05:23,677][105692] Updated weights for policy 0, policy_version 1879597 (0.0010) [2023-12-27 05:05:24,192][105620] Updated weights for policy 1, policy_version 1884082 (0.0008) [2023-12-27 05:05:24,240][105620] Updated weights for policy 1, policy_version 1884092 (0.0008) [2023-12-27 05:05:24,306][105620] Updated weights for policy 1, policy_version 1884102 (0.0008) [2023-12-27 05:05:24,371][105620] Updated weights for policy 1, policy_version 1884112 (0.0008) [2023-12-27 05:05:24,422][105692] Updated weights for policy 0, policy_version 1879607 (0.0010) [2023-12-27 05:05:24,483][105692] Updated weights for policy 0, policy_version 1879617 (0.0010) [2023-12-27 05:05:24,538][105692] Updated weights for policy 0, policy_version 1879627 (0.0010) [2023-12-27 05:05:25,125][105620] Updated weights for policy 1, policy_version 1884122 (0.0008) [2023-12-27 05:05:25,181][105620] Updated weights for policy 1, policy_version 1884132 (0.0008) [2023-12-27 05:05:25,224][105620] Updated weights for policy 1, policy_version 1884142 (0.0007) [2023-12-27 05:05:25,292][105692] Updated weights for policy 0, policy_version 1879637 (0.0011) [2023-12-27 05:05:25,347][105692] Updated weights for policy 0, policy_version 1879647 (0.0010) [2023-12-27 05:05:25,405][105692] Updated weights for policy 0, policy_version 1879657 (0.0010) [2023-12-27 05:05:26,035][105692] Updated weights for policy 0, policy_version 1879667 (0.0008) [2023-12-27 05:05:26,044][105620] Updated weights for policy 1, policy_version 1884152 (0.0009) [2023-12-27 05:05:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.3, 300 sec: 19410.9). Total num frames: 963674112. Throughput: 0: 9631.8, 1: 9587.7. Samples: 963687728. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:05:26,062][104569] Avg episode reward: [(0, '8540.223'), (1, '9253.136')] [2023-12-27 05:05:26,097][105692] Updated weights for policy 0, policy_version 1879677 (0.0008) [2023-12-27 05:05:26,099][105620] Updated weights for policy 1, policy_version 1884162 (0.0007) [2023-12-27 05:05:26,157][105692] Updated weights for policy 0, policy_version 1879687 (0.0007) [2023-12-27 05:05:26,159][105620] Updated weights for policy 1, policy_version 1884172 (0.0007) [2023-12-27 05:05:26,685][105692] Updated weights for policy 0, policy_version 1879697 (0.0005) [2023-12-27 05:05:26,736][105692] Updated weights for policy 0, policy_version 1879707 (0.0006) [2023-12-27 05:05:26,798][105692] Updated weights for policy 0, policy_version 1879717 (0.0010) [2023-12-27 05:05:26,862][105692] Updated weights for policy 0, policy_version 1879727 (0.0010) [2023-12-27 05:05:27,025][105620] Updated weights for policy 1, policy_version 1884182 (0.0009) [2023-12-27 05:05:27,079][105620] Updated weights for policy 1, policy_version 1884194 (0.0010) [2023-12-27 05:05:27,143][105620] Updated weights for policy 1, policy_version 1884204 (0.0005) [2023-12-27 05:05:27,456][105692] Updated weights for policy 0, policy_version 1879737 (0.0006) [2023-12-27 05:05:27,503][105692] Updated weights for policy 0, policy_version 1879747 (0.0010) [2023-12-27 05:05:27,547][105692] Updated weights for policy 0, policy_version 1879757 (0.0010) [2023-12-27 05:05:27,728][105620] Updated weights for policy 1, policy_version 1884214 (0.0007) [2023-12-27 05:05:27,780][105620] Updated weights for policy 1, policy_version 1884224 (0.0009) [2023-12-27 05:05:27,841][105620] Updated weights for policy 1, policy_version 1884234 (0.0008) [2023-12-27 05:05:28,178][105692] Updated weights for policy 0, policy_version 1879767 (0.0010) [2023-12-27 05:05:28,233][105692] Updated weights for policy 0, policy_version 1879777 (0.0010) [2023-12-27 05:05:28,307][105692] Updated weights for policy 0, policy_version 1879787 (0.0009) [2023-12-27 05:05:28,630][105620] Updated weights for policy 1, policy_version 1884244 (0.0009) [2023-12-27 05:05:28,688][105620] Updated weights for policy 1, policy_version 1884254 (0.0010) [2023-12-27 05:05:28,741][105620] Updated weights for policy 1, policy_version 1884264 (0.0010) [2023-12-27 05:05:29,030][105692] Updated weights for policy 0, policy_version 1879797 (0.0007) [2023-12-27 05:05:29,083][105692] Updated weights for policy 0, policy_version 1879807 (0.0006) [2023-12-27 05:05:29,142][105692] Updated weights for policy 0, policy_version 1879817 (0.0010) [2023-12-27 05:05:29,511][105620] Updated weights for policy 1, policy_version 1884274 (0.0010) [2023-12-27 05:05:29,563][105620] Updated weights for policy 1, policy_version 1884284 (0.0009) [2023-12-27 05:05:29,622][105620] Updated weights for policy 1, policy_version 1884294 (0.0009) [2023-12-27 05:05:29,683][105620] Updated weights for policy 1, policy_version 1884304 (0.0008) [2023-12-27 05:05:29,745][105692] Updated weights for policy 0, policy_version 1879828 (0.0007) [2023-12-27 05:05:29,800][105692] Updated weights for policy 0, policy_version 1879838 (0.0005) [2023-12-27 05:05:29,870][105692] Updated weights for policy 0, policy_version 1879848 (0.0006) [2023-12-27 05:05:30,448][105692] Updated weights for policy 0, policy_version 1879858 (0.0006) [2023-12-27 05:05:30,494][105620] Updated weights for policy 1, policy_version 1884314 (0.0005) [2023-12-27 05:05:30,509][105692] Updated weights for policy 0, policy_version 1879868 (0.0005) [2023-12-27 05:05:30,558][105620] Updated weights for policy 1, policy_version 1884324 (0.0010) [2023-12-27 05:05:30,564][105692] Updated weights for policy 0, policy_version 1879878 (0.0005) [2023-12-27 05:05:30,609][105620] Updated weights for policy 1, policy_version 1884334 (0.0010) [2023-12-27 05:05:30,611][105692] Updated weights for policy 0, policy_version 1879888 (0.0006) [2023-12-27 05:05:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 963780608. Throughput: 0: 9784.9, 1: 9561.5. Samples: 963749412. Policy #0 lag: (min: 10.0, avg: 24.4, max: 42.0) [2023-12-27 05:05:31,063][104569] Avg episode reward: [(0, '8715.046'), (1, '9345.463')] [2023-12-27 05:05:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001879888_481320960.pth... [2023-12-27 05:05:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001884336_482459648.pth... [2023-12-27 05:05:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001878704_481017856.pth [2023-12-27 05:05:31,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001883216_482172928.pth [2023-12-27 05:05:31,315][105620] Updated weights for policy 1, policy_version 1884344 (0.0010) [2023-12-27 05:05:31,325][105692] Updated weights for policy 0, policy_version 1879898 (0.0005) [2023-12-27 05:05:31,374][105620] Updated weights for policy 1, policy_version 1884354 (0.0010) [2023-12-27 05:05:31,389][105692] Updated weights for policy 0, policy_version 1879908 (0.0007) [2023-12-27 05:05:31,438][105620] Updated weights for policy 1, policy_version 1884364 (0.0007) [2023-12-27 05:05:31,456][105692] Updated weights for policy 0, policy_version 1879918 (0.0008) [2023-12-27 05:05:32,109][105620] Updated weights for policy 1, policy_version 1884374 (0.0006) [2023-12-27 05:05:32,168][105620] Updated weights for policy 1, policy_version 1884384 (0.0008) [2023-12-27 05:05:32,230][105620] Updated weights for policy 1, policy_version 1884394 (0.0008) [2023-12-27 05:05:32,252][105692] Updated weights for policy 0, policy_version 1879928 (0.0010) [2023-12-27 05:05:32,317][105692] Updated weights for policy 0, policy_version 1879938 (0.0010) [2023-12-27 05:05:32,386][105692] Updated weights for policy 0, policy_version 1879948 (0.0011) [2023-12-27 05:05:32,906][105620] Updated weights for policy 1, policy_version 1884404 (0.0007) [2023-12-27 05:05:32,966][105620] Updated weights for policy 1, policy_version 1884414 (0.0008) [2023-12-27 05:05:33,029][105620] Updated weights for policy 1, policy_version 1884424 (0.0008) [2023-12-27 05:05:33,139][105692] Updated weights for policy 0, policy_version 1879958 (0.0011) [2023-12-27 05:05:33,204][105692] Updated weights for policy 0, policy_version 1879968 (0.0010) [2023-12-27 05:05:33,266][105692] Updated weights for policy 0, policy_version 1879978 (0.0010) [2023-12-27 05:05:33,780][105620] Updated weights for policy 1, policy_version 1884434 (0.0008) [2023-12-27 05:05:33,828][105620] Updated weights for policy 1, policy_version 1884444 (0.0008) [2023-12-27 05:05:33,879][105620] Updated weights for policy 1, policy_version 1884454 (0.0008) [2023-12-27 05:05:33,935][105620] Updated weights for policy 1, policy_version 1884464 (0.0008) [2023-12-27 05:05:33,992][105692] Updated weights for policy 0, policy_version 1879988 (0.0010) [2023-12-27 05:05:34,049][105692] Updated weights for policy 0, policy_version 1879998 (0.0010) [2023-12-27 05:05:34,103][105692] Updated weights for policy 0, policy_version 1880008 (0.0010) [2023-12-27 05:05:34,644][105620] Updated weights for policy 1, policy_version 1884474 (0.0008) [2023-12-27 05:05:34,707][105620] Updated weights for policy 1, policy_version 1884484 (0.0008) [2023-12-27 05:05:34,773][105620] Updated weights for policy 1, policy_version 1884494 (0.0008) [2023-12-27 05:05:34,871][105692] Updated weights for policy 0, policy_version 1880018 (0.0011) [2023-12-27 05:05:34,926][105692] Updated weights for policy 0, policy_version 1880028 (0.0010) [2023-12-27 05:05:34,971][105692] Updated weights for policy 0, policy_version 1880038 (0.0010) [2023-12-27 05:05:35,019][105692] Updated weights for policy 0, policy_version 1880048 (0.0010) [2023-12-27 05:05:35,488][105620] Updated weights for policy 1, policy_version 1884504 (0.0008) [2023-12-27 05:05:35,546][105620] Updated weights for policy 1, policy_version 1884514 (0.0008) [2023-12-27 05:05:35,608][105620] Updated weights for policy 1, policy_version 1884524 (0.0008) [2023-12-27 05:05:35,781][105692] Updated weights for policy 0, policy_version 1880058 (0.0010) [2023-12-27 05:05:35,829][105692] Updated weights for policy 0, policy_version 1880068 (0.0010) [2023-12-27 05:05:35,881][105692] Updated weights for policy 0, policy_version 1880078 (0.0010) [2023-12-27 05:05:36,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 963878912. Throughput: 0: 9730.6, 1: 9623.3. Samples: 963866016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:05:36,063][104569] Avg episode reward: [(0, '8626.199'), (1, '9253.050')] [2023-12-27 05:05:36,355][105620] Updated weights for policy 1, policy_version 1884534 (0.0008) [2023-12-27 05:05:36,422][105620] Updated weights for policy 1, policy_version 1884544 (0.0006) [2023-12-27 05:05:36,488][105620] Updated weights for policy 1, policy_version 1884554 (0.0006) [2023-12-27 05:05:36,679][105692] Updated weights for policy 0, policy_version 1880088 (0.0010) [2023-12-27 05:05:36,750][105692] Updated weights for policy 0, policy_version 1880098 (0.0011) [2023-12-27 05:05:36,813][105692] Updated weights for policy 0, policy_version 1880108 (0.0010) [2023-12-27 05:05:37,175][105620] Updated weights for policy 1, policy_version 1884564 (0.0007) [2023-12-27 05:05:37,236][105620] Updated weights for policy 1, policy_version 1884574 (0.0009) [2023-12-27 05:05:37,292][105620] Updated weights for policy 1, policy_version 1884584 (0.0009) [2023-12-27 05:05:37,495][105692] Updated weights for policy 0, policy_version 1880118 (0.0007) [2023-12-27 05:05:37,543][105692] Updated weights for policy 0, policy_version 1880128 (0.0005) [2023-12-27 05:05:37,596][105692] Updated weights for policy 0, policy_version 1880138 (0.0005) [2023-12-27 05:05:38,168][105692] Updated weights for policy 0, policy_version 1880148 (0.0005) [2023-12-27 05:05:38,172][105620] Updated weights for policy 1, policy_version 1884594 (0.0009) [2023-12-27 05:05:38,227][105692] Updated weights for policy 0, policy_version 1880158 (0.0007) [2023-12-27 05:05:38,239][105620] Updated weights for policy 1, policy_version 1884604 (0.0007) [2023-12-27 05:05:38,285][105692] Updated weights for policy 0, policy_version 1880168 (0.0006) [2023-12-27 05:05:38,294][105620] Updated weights for policy 1, policy_version 1884614 (0.0009) [2023-12-27 05:05:38,355][105620] Updated weights for policy 1, policy_version 1884624 (0.0007) [2023-12-27 05:05:38,962][105692] Updated weights for policy 0, policy_version 1880178 (0.0007) [2023-12-27 05:05:39,012][105620] Updated weights for policy 1, policy_version 1884634 (0.0009) [2023-12-27 05:05:39,018][105692] Updated weights for policy 0, policy_version 1880188 (0.0006) [2023-12-27 05:05:39,065][105620] Updated weights for policy 1, policy_version 1884644 (0.0011) [2023-12-27 05:05:39,075][105692] Updated weights for policy 0, policy_version 1880198 (0.0005) [2023-12-27 05:05:39,119][105620] Updated weights for policy 1, policy_version 1884654 (0.0010) [2023-12-27 05:05:39,129][105692] Updated weights for policy 0, policy_version 1880208 (0.0007) [2023-12-27 05:05:39,742][105620] Updated weights for policy 1, policy_version 1884664 (0.0008) [2023-12-27 05:05:39,794][105620] Updated weights for policy 1, policy_version 1884675 (0.0010) [2023-12-27 05:05:39,859][105620] Updated weights for policy 1, policy_version 1884685 (0.0007) [2023-12-27 05:05:39,879][105692] Updated weights for policy 0, policy_version 1880218 (0.0009) [2023-12-27 05:05:39,936][105692] Updated weights for policy 0, policy_version 1880228 (0.0008) [2023-12-27 05:05:40,004][105692] Updated weights for policy 0, policy_version 1880238 (0.0009) [2023-12-27 05:05:40,596][105620] Updated weights for policy 1, policy_version 1884695 (0.0007) [2023-12-27 05:05:40,654][105620] Updated weights for policy 1, policy_version 1884705 (0.0006) [2023-12-27 05:05:40,722][105620] Updated weights for policy 1, policy_version 1884715 (0.0005) [2023-12-27 05:05:40,755][105692] Updated weights for policy 0, policy_version 1880248 (0.0006) [2023-12-27 05:05:40,823][105692] Updated weights for policy 0, policy_version 1880258 (0.0007) [2023-12-27 05:05:40,912][105692] Updated weights for policy 0, policy_version 1880268 (0.0006) [2023-12-27 05:05:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 963977216. Throughput: 0: 9815.6, 1: 9601.4. Samples: 963983008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:05:41,063][104569] Avg episode reward: [(0, '8080.761'), (1, '9253.074')] [2023-12-27 05:05:41,463][105620] Updated weights for policy 1, policy_version 1884725 (0.0008) [2023-12-27 05:05:41,492][105692] Updated weights for policy 0, policy_version 1880278 (0.0007) [2023-12-27 05:05:41,524][105620] Updated weights for policy 1, policy_version 1884735 (0.0011) [2023-12-27 05:05:41,553][105692] Updated weights for policy 0, policy_version 1880288 (0.0007) [2023-12-27 05:05:41,585][105620] Updated weights for policy 1, policy_version 1884745 (0.0007) [2023-12-27 05:05:41,615][105692] Updated weights for policy 0, policy_version 1880298 (0.0011) [2023-12-27 05:05:42,318][105692] Updated weights for policy 0, policy_version 1880308 (0.0009) [2023-12-27 05:05:42,335][105620] Updated weights for policy 1, policy_version 1884755 (0.0009) [2023-12-27 05:05:42,384][105692] Updated weights for policy 0, policy_version 1880318 (0.0008) [2023-12-27 05:05:42,396][105620] Updated weights for policy 1, policy_version 1884765 (0.0008) [2023-12-27 05:05:42,440][105692] Updated weights for policy 0, policy_version 1880328 (0.0007) [2023-12-27 05:05:42,455][105620] Updated weights for policy 1, policy_version 1884775 (0.0008) [2023-12-27 05:05:43,072][105692] Updated weights for policy 0, policy_version 1880338 (0.0005) [2023-12-27 05:05:43,137][105692] Updated weights for policy 0, policy_version 1880348 (0.0006) [2023-12-27 05:05:43,202][105692] Updated weights for policy 0, policy_version 1880358 (0.0008) [2023-12-27 05:05:43,226][105620] Updated weights for policy 1, policy_version 1884785 (0.0008) [2023-12-27 05:05:43,263][105692] Updated weights for policy 0, policy_version 1880368 (0.0009) [2023-12-27 05:05:43,288][105620] Updated weights for policy 1, policy_version 1884795 (0.0007) [2023-12-27 05:05:43,341][105620] Updated weights for policy 1, policy_version 1884805 (0.0011) [2023-12-27 05:05:43,390][105620] Updated weights for policy 1, policy_version 1884815 (0.0010) [2023-12-27 05:05:43,961][105692] Updated weights for policy 0, policy_version 1880378 (0.0005) [2023-12-27 05:05:44,021][105692] Updated weights for policy 0, policy_version 1880388 (0.0005) [2023-12-27 05:05:44,049][105620] Updated weights for policy 1, policy_version 1884825 (0.0011) [2023-12-27 05:05:44,084][105692] Updated weights for policy 0, policy_version 1880398 (0.0006) [2023-12-27 05:05:44,108][105620] Updated weights for policy 1, policy_version 1884835 (0.0010) [2023-12-27 05:05:44,167][105620] Updated weights for policy 1, policy_version 1884845 (0.0010) [2023-12-27 05:05:44,745][105692] Updated weights for policy 0, policy_version 1880408 (0.0005) [2023-12-27 05:05:44,802][105692] Updated weights for policy 0, policy_version 1880418 (0.0007) [2023-12-27 05:05:44,859][105692] Updated weights for policy 0, policy_version 1880428 (0.0006) [2023-12-27 05:05:44,895][105620] Updated weights for policy 1, policy_version 1884855 (0.0008) [2023-12-27 05:05:44,950][105620] Updated weights for policy 1, policy_version 1884865 (0.0006) [2023-12-27 05:05:45,016][105620] Updated weights for policy 1, policy_version 1884875 (0.0007) [2023-12-27 05:05:45,453][105692] Updated weights for policy 0, policy_version 1880438 (0.0007) [2023-12-27 05:05:45,507][105692] Updated weights for policy 0, policy_version 1880448 (0.0007) [2023-12-27 05:05:45,559][105692] Updated weights for policy 0, policy_version 1880458 (0.0008) [2023-12-27 05:05:45,722][105620] Updated weights for policy 1, policy_version 1884885 (0.0010) [2023-12-27 05:05:45,776][105620] Updated weights for policy 1, policy_version 1884895 (0.0010) [2023-12-27 05:05:45,835][105620] Updated weights for policy 1, policy_version 1884905 (0.0010) [2023-12-27 05:05:46,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 964075520. Throughput: 0: 9886.5, 1: 9622.8. Samples: 964042084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:05:46,063][104569] Avg episode reward: [(0, '8357.954'), (1, '9345.389')] [2023-12-27 05:05:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001880464_481468416.pth... [2023-12-27 05:05:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001884912_482607104.pth... [2023-12-27 05:05:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001879280_481165312.pth [2023-12-27 05:05:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001883792_482320384.pth [2023-12-27 05:05:46,250][105692] Updated weights for policy 0, policy_version 1880468 (0.0009) [2023-12-27 05:05:46,312][105692] Updated weights for policy 0, policy_version 1880478 (0.0008) [2023-12-27 05:05:46,377][105692] Updated weights for policy 0, policy_version 1880488 (0.0009) [2023-12-27 05:05:46,571][105620] Updated weights for policy 1, policy_version 1884915 (0.0010) [2023-12-27 05:05:46,630][105620] Updated weights for policy 1, policy_version 1884925 (0.0011) [2023-12-27 05:05:46,685][105620] Updated weights for policy 1, policy_version 1884935 (0.0010) [2023-12-27 05:05:47,043][105692] Updated weights for policy 0, policy_version 1880498 (0.0006) [2023-12-27 05:05:47,093][105692] Updated weights for policy 0, policy_version 1880508 (0.0006) [2023-12-27 05:05:47,139][105692] Updated weights for policy 0, policy_version 1880518 (0.0006) [2023-12-27 05:05:47,186][105692] Updated weights for policy 0, policy_version 1880528 (0.0009) [2023-12-27 05:05:47,418][105620] Updated weights for policy 1, policy_version 1884945 (0.0010) [2023-12-27 05:05:47,469][105620] Updated weights for policy 1, policy_version 1884955 (0.0010) [2023-12-27 05:05:47,521][105620] Updated weights for policy 1, policy_version 1884965 (0.0010) [2023-12-27 05:05:47,576][105620] Updated weights for policy 1, policy_version 1884975 (0.0010) [2023-12-27 05:05:47,906][105692] Updated weights for policy 0, policy_version 1880538 (0.0005) [2023-12-27 05:05:47,972][105692] Updated weights for policy 0, policy_version 1880548 (0.0010) [2023-12-27 05:05:48,031][105692] Updated weights for policy 0, policy_version 1880558 (0.0008) [2023-12-27 05:05:48,290][105620] Updated weights for policy 1, policy_version 1884985 (0.0006) [2023-12-27 05:05:48,350][105620] Updated weights for policy 1, policy_version 1884995 (0.0007) [2023-12-27 05:05:48,407][105620] Updated weights for policy 1, policy_version 1885005 (0.0009) [2023-12-27 05:05:48,659][105692] Updated weights for policy 0, policy_version 1880568 (0.0007) [2023-12-27 05:05:48,712][105692] Updated weights for policy 0, policy_version 1880578 (0.0008) [2023-12-27 05:05:48,772][105692] Updated weights for policy 0, policy_version 1880588 (0.0008) [2023-12-27 05:05:49,090][105620] Updated weights for policy 1, policy_version 1885015 (0.0007) [2023-12-27 05:05:49,153][105620] Updated weights for policy 1, policy_version 1885025 (0.0007) [2023-12-27 05:05:49,222][105620] Updated weights for policy 1, policy_version 1885035 (0.0006) [2023-12-27 05:05:49,550][105692] Updated weights for policy 0, policy_version 1880598 (0.0006) [2023-12-27 05:05:49,613][105692] Updated weights for policy 0, policy_version 1880609 (0.0009) [2023-12-27 05:05:49,676][105692] Updated weights for policy 0, policy_version 1880619 (0.0009) [2023-12-27 05:05:49,867][105620] Updated weights for policy 1, policy_version 1885045 (0.0009) [2023-12-27 05:05:49,926][105620] Updated weights for policy 1, policy_version 1885055 (0.0007) [2023-12-27 05:05:49,985][105620] Updated weights for policy 1, policy_version 1885065 (0.0007) [2023-12-27 05:05:50,535][105620] Updated weights for policy 1, policy_version 1885075 (0.0006) [2023-12-27 05:05:50,552][105692] Updated weights for policy 0, policy_version 1880629 (0.0009) [2023-12-27 05:05:50,592][105620] Updated weights for policy 1, policy_version 1885085 (0.0006) [2023-12-27 05:05:50,610][105692] Updated weights for policy 0, policy_version 1880639 (0.0008) [2023-12-27 05:05:50,642][105620] Updated weights for policy 1, policy_version 1885095 (0.0006) [2023-12-27 05:05:50,666][105692] Updated weights for policy 0, policy_version 1880649 (0.0007) [2023-12-27 05:05:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 964173824. Throughput: 0: 9872.1, 1: 9664.6. Samples: 964162264. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:05:51,062][104569] Avg episode reward: [(0, '8266.027'), (1, '9255.012')] [2023-12-27 05:05:51,344][105620] Updated weights for policy 1, policy_version 1885105 (0.0007) [2023-12-27 05:05:51,412][105620] Updated weights for policy 1, policy_version 1885115 (0.0008) [2023-12-27 05:05:51,490][105620] Updated weights for policy 1, policy_version 1885125 (0.0009) [2023-12-27 05:05:51,526][105692] Updated weights for policy 0, policy_version 1880659 (0.0008) [2023-12-27 05:05:51,551][105620] Updated weights for policy 1, policy_version 1885135 (0.0008) [2023-12-27 05:05:51,577][105692] Updated weights for policy 0, policy_version 1880669 (0.0007) [2023-12-27 05:05:51,638][105692] Updated weights for policy 0, policy_version 1880679 (0.0007) [2023-12-27 05:05:52,318][105620] Updated weights for policy 1, policy_version 1885145 (0.0009) [2023-12-27 05:05:52,356][105692] Updated weights for policy 0, policy_version 1880689 (0.0008) [2023-12-27 05:05:52,384][105620] Updated weights for policy 1, policy_version 1885155 (0.0008) [2023-12-27 05:05:52,422][105692] Updated weights for policy 0, policy_version 1880699 (0.0007) [2023-12-27 05:05:52,448][105620] Updated weights for policy 1, policy_version 1885165 (0.0009) [2023-12-27 05:05:52,482][105692] Updated weights for policy 0, policy_version 1880709 (0.0006) [2023-12-27 05:05:52,539][105692] Updated weights for policy 0, policy_version 1880719 (0.0005) [2023-12-27 05:05:53,199][105620] Updated weights for policy 1, policy_version 1885175 (0.0007) [2023-12-27 05:05:53,222][105692] Updated weights for policy 0, policy_version 1880729 (0.0006) [2023-12-27 05:05:53,247][105620] Updated weights for policy 1, policy_version 1885185 (0.0008) [2023-12-27 05:05:53,273][105692] Updated weights for policy 0, policy_version 1880739 (0.0010) [2023-12-27 05:05:53,291][105620] Updated weights for policy 1, policy_version 1885195 (0.0006) [2023-12-27 05:05:53,326][105692] Updated weights for policy 0, policy_version 1880749 (0.0009) [2023-12-27 05:05:53,909][105620] Updated weights for policy 1, policy_version 1885205 (0.0005) [2023-12-27 05:05:53,963][105620] Updated weights for policy 1, policy_version 1885215 (0.0006) [2023-12-27 05:05:54,024][105620] Updated weights for policy 1, policy_version 1885225 (0.0007) [2023-12-27 05:05:54,116][105692] Updated weights for policy 0, policy_version 1880759 (0.0009) [2023-12-27 05:05:54,180][105692] Updated weights for policy 0, policy_version 1880769 (0.0008) [2023-12-27 05:05:54,242][105692] Updated weights for policy 0, policy_version 1880779 (0.0006) [2023-12-27 05:05:54,614][105620] Updated weights for policy 1, policy_version 1885235 (0.0006) [2023-12-27 05:05:54,681][105620] Updated weights for policy 1, policy_version 1885245 (0.0006) [2023-12-27 05:05:54,747][105620] Updated weights for policy 1, policy_version 1885255 (0.0005) [2023-12-27 05:05:54,882][105692] Updated weights for policy 0, policy_version 1880789 (0.0005) [2023-12-27 05:05:54,926][105692] Updated weights for policy 0, policy_version 1880799 (0.0005) [2023-12-27 05:05:54,984][105692] Updated weights for policy 0, policy_version 1880809 (0.0008) [2023-12-27 05:05:55,375][105620] Updated weights for policy 1, policy_version 1885265 (0.0007) [2023-12-27 05:05:55,443][105620] Updated weights for policy 1, policy_version 1885275 (0.0009) [2023-12-27 05:05:55,517][105620] Updated weights for policy 1, policy_version 1885285 (0.0009) [2023-12-27 05:05:55,576][105620] Updated weights for policy 1, policy_version 1885295 (0.0009) [2023-12-27 05:05:55,616][105692] Updated weights for policy 0, policy_version 1880819 (0.0010) [2023-12-27 05:05:55,661][105692] Updated weights for policy 0, policy_version 1880829 (0.0010) [2023-12-27 05:05:55,725][105692] Updated weights for policy 0, policy_version 1880839 (0.0009) [2023-12-27 05:05:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 964272128. Throughput: 0: 9870.6, 1: 9735.2. Samples: 964281636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:05:56,062][104569] Avg episode reward: [(0, '7801.994'), (1, '9254.974')] [2023-12-27 05:05:56,247][105620] Updated weights for policy 1, policy_version 1885305 (0.0009) [2023-12-27 05:05:56,298][105692] Updated weights for policy 0, policy_version 1880849 (0.0010) [2023-12-27 05:05:56,309][105620] Updated weights for policy 1, policy_version 1885315 (0.0009) [2023-12-27 05:05:56,354][105692] Updated weights for policy 0, policy_version 1880859 (0.0007) [2023-12-27 05:05:56,366][105620] Updated weights for policy 1, policy_version 1885325 (0.0009) [2023-12-27 05:05:56,412][105692] Updated weights for policy 0, policy_version 1880869 (0.0006) [2023-12-27 05:05:56,458][105692] Updated weights for policy 0, policy_version 1880879 (0.0006) [2023-12-27 05:05:56,989][105620] Updated weights for policy 1, policy_version 1885335 (0.0009) [2023-12-27 05:05:57,007][105692] Updated weights for policy 0, policy_version 1880889 (0.0006) [2023-12-27 05:05:57,047][105620] Updated weights for policy 1, policy_version 1885345 (0.0007) [2023-12-27 05:05:57,061][105692] Updated weights for policy 0, policy_version 1880899 (0.0006) [2023-12-27 05:05:57,114][105692] Updated weights for policy 0, policy_version 1880909 (0.0006) [2023-12-27 05:05:57,115][105620] Updated weights for policy 1, policy_version 1885355 (0.0006) [2023-12-27 05:05:57,653][105620] Updated weights for policy 1, policy_version 1885365 (0.0007) [2023-12-27 05:05:57,705][105620] Updated weights for policy 1, policy_version 1885375 (0.0008) [2023-12-27 05:05:57,716][105692] Updated weights for policy 0, policy_version 1880919 (0.0005) [2023-12-27 05:05:57,764][105620] Updated weights for policy 1, policy_version 1885385 (0.0006) [2023-12-27 05:05:57,768][105692] Updated weights for policy 0, policy_version 1880929 (0.0006) [2023-12-27 05:05:57,815][105692] Updated weights for policy 0, policy_version 1880939 (0.0005) [2023-12-27 05:05:58,408][105692] Updated weights for policy 0, policy_version 1880949 (0.0007) [2023-12-27 05:05:58,448][105620] Updated weights for policy 1, policy_version 1885395 (0.0006) [2023-12-27 05:05:58,471][105692] Updated weights for policy 0, policy_version 1880959 (0.0011) [2023-12-27 05:05:58,514][105620] Updated weights for policy 1, policy_version 1885405 (0.0008) [2023-12-27 05:05:58,535][105692] Updated weights for policy 0, policy_version 1880969 (0.0011) [2023-12-27 05:05:58,584][105620] Updated weights for policy 1, policy_version 1885415 (0.0008) [2023-12-27 05:05:59,315][105692] Updated weights for policy 0, policy_version 1880979 (0.0009) [2023-12-27 05:05:59,355][105620] Updated weights for policy 1, policy_version 1885425 (0.0008) [2023-12-27 05:05:59,393][105692] Updated weights for policy 0, policy_version 1880989 (0.0009) [2023-12-27 05:05:59,421][105620] Updated weights for policy 1, policy_version 1885435 (0.0008) [2023-12-27 05:05:59,454][105692] Updated weights for policy 0, policy_version 1880999 (0.0006) [2023-12-27 05:05:59,482][105620] Updated weights for policy 1, policy_version 1885445 (0.0009) [2023-12-27 05:05:59,546][105620] Updated weights for policy 1, policy_version 1885455 (0.0009) [2023-12-27 05:06:00,191][105620] Updated weights for policy 1, policy_version 1885465 (0.0008) [2023-12-27 05:06:00,211][105692] Updated weights for policy 0, policy_version 1881009 (0.0006) [2023-12-27 05:06:00,241][105620] Updated weights for policy 1, policy_version 1885475 (0.0007) [2023-12-27 05:06:00,268][105692] Updated weights for policy 0, policy_version 1881019 (0.0009) [2023-12-27 05:06:00,320][105620] Updated weights for policy 1, policy_version 1885485 (0.0008) [2023-12-27 05:06:00,326][105692] Updated weights for policy 0, policy_version 1881029 (0.0007) [2023-12-27 05:06:00,383][105692] Updated weights for policy 0, policy_version 1881039 (0.0008) [2023-12-27 05:06:01,022][105692] Updated weights for policy 0, policy_version 1881049 (0.0006) [2023-12-27 05:06:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 964370432. Throughput: 0: 10027.7, 1: 9799.8. Samples: 964346796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:06:01,062][104569] Avg episode reward: [(0, '8441.385'), (1, '9253.008')] [2023-12-27 05:06:01,083][105620] Updated weights for policy 1, policy_version 1885495 (0.0010) [2023-12-27 05:06:01,089][105692] Updated weights for policy 0, policy_version 1881059 (0.0008) [2023-12-27 05:06:01,160][105620] Updated weights for policy 1, policy_version 1885505 (0.0011) [2023-12-27 05:06:01,165][105692] Updated weights for policy 0, policy_version 1881069 (0.0008) [2023-12-27 05:06:01,180][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001881072_481624064.pth... [2023-12-27 05:06:01,184][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001879888_481320960.pth [2023-12-27 05:06:01,213][105620] Updated weights for policy 1, policy_version 1885515 (0.0010) [2023-12-27 05:06:01,233][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001885520_482762752.pth... [2023-12-27 05:06:01,236][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001884336_482459648.pth [2023-12-27 05:06:01,877][105692] Updated weights for policy 0, policy_version 1881079 (0.0008) [2023-12-27 05:06:01,932][105692] Updated weights for policy 0, policy_version 1881089 (0.0008) [2023-12-27 05:06:01,969][105620] Updated weights for policy 1, policy_version 1885525 (0.0010) [2023-12-27 05:06:01,985][105692] Updated weights for policy 0, policy_version 1881099 (0.0009) [2023-12-27 05:06:02,024][105620] Updated weights for policy 1, policy_version 1885535 (0.0010) [2023-12-27 05:06:02,072][105620] Updated weights for policy 1, policy_version 1885545 (0.0010) [2023-12-27 05:06:02,764][105692] Updated weights for policy 0, policy_version 1881109 (0.0007) [2023-12-27 05:06:02,832][105692] Updated weights for policy 0, policy_version 1881119 (0.0008) [2023-12-27 05:06:02,835][105620] Updated weights for policy 1, policy_version 1885555 (0.0010) [2023-12-27 05:06:02,885][105692] Updated weights for policy 0, policy_version 1881129 (0.0009) [2023-12-27 05:06:02,887][105620] Updated weights for policy 1, policy_version 1885565 (0.0010) [2023-12-27 05:06:02,939][105620] Updated weights for policy 1, policy_version 1885575 (0.0010) [2023-12-27 05:06:03,625][105620] Updated weights for policy 1, policy_version 1885585 (0.0011) [2023-12-27 05:06:03,643][105692] Updated weights for policy 0, policy_version 1881139 (0.0005) [2023-12-27 05:06:03,686][105620] Updated weights for policy 1, policy_version 1885595 (0.0010) [2023-12-27 05:06:03,690][105692] Updated weights for policy 0, policy_version 1881149 (0.0008) [2023-12-27 05:06:03,743][105692] Updated weights for policy 0, policy_version 1881159 (0.0006) [2023-12-27 05:06:03,744][105620] Updated weights for policy 1, policy_version 1885605 (0.0010) [2023-12-27 05:06:03,809][105620] Updated weights for policy 1, policy_version 1885615 (0.0010) [2023-12-27 05:06:04,523][105692] Updated weights for policy 0, policy_version 1881169 (0.0008) [2023-12-27 05:06:04,550][105620] Updated weights for policy 1, policy_version 1885625 (0.0007) [2023-12-27 05:06:04,590][105692] Updated weights for policy 0, policy_version 1881179 (0.0010) [2023-12-27 05:06:04,608][105620] Updated weights for policy 1, policy_version 1885635 (0.0009) [2023-12-27 05:06:04,648][105692] Updated weights for policy 0, policy_version 1881189 (0.0011) [2023-12-27 05:06:04,674][105620] Updated weights for policy 1, policy_version 1885645 (0.0011) [2023-12-27 05:06:04,706][105692] Updated weights for policy 0, policy_version 1881199 (0.0011) [2023-12-27 05:06:05,317][105620] Updated weights for policy 1, policy_version 1885655 (0.0010) [2023-12-27 05:06:05,373][105620] Updated weights for policy 1, policy_version 1885665 (0.0010) [2023-12-27 05:06:05,433][105620] Updated weights for policy 1, policy_version 1885675 (0.0010) [2023-12-27 05:06:05,467][105692] Updated weights for policy 0, policy_version 1881209 (0.0007) [2023-12-27 05:06:05,519][105692] Updated weights for policy 0, policy_version 1881219 (0.0005) [2023-12-27 05:06:05,574][105692] Updated weights for policy 0, policy_version 1881230 (0.0008) [2023-12-27 05:06:06,022][105620] Updated weights for policy 1, policy_version 1885685 (0.0005) [2023-12-27 05:06:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 964468736. Throughput: 0: 9892.9, 1: 9721.7. Samples: 964460256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:06:06,062][104569] Avg episode reward: [(0, '8896.609'), (1, '9253.006')] [2023-12-27 05:06:06,082][105620] Updated weights for policy 1, policy_version 1885695 (0.0006) [2023-12-27 05:06:06,144][105620] Updated weights for policy 1, policy_version 1885705 (0.0009) [2023-12-27 05:06:06,305][105692] Updated weights for policy 0, policy_version 1881240 (0.0007) [2023-12-27 05:06:06,365][105692] Updated weights for policy 0, policy_version 1881250 (0.0005) [2023-12-27 05:06:06,422][105692] Updated weights for policy 0, policy_version 1881260 (0.0005) [2023-12-27 05:06:06,787][105620] Updated weights for policy 1, policy_version 1885715 (0.0011) [2023-12-27 05:06:06,846][105620] Updated weights for policy 1, policy_version 1885725 (0.0011) [2023-12-27 05:06:06,898][105620] Updated weights for policy 1, policy_version 1885735 (0.0011) [2023-12-27 05:06:06,959][105692] Updated weights for policy 0, policy_version 1881270 (0.0006) [2023-12-27 05:06:07,017][105692] Updated weights for policy 0, policy_version 1881280 (0.0005) [2023-12-27 05:06:07,075][105692] Updated weights for policy 0, policy_version 1881290 (0.0006) [2023-12-27 05:06:07,591][105620] Updated weights for policy 1, policy_version 1885745 (0.0010) [2023-12-27 05:06:07,648][105620] Updated weights for policy 1, policy_version 1885755 (0.0008) [2023-12-27 05:06:07,686][105692] Updated weights for policy 0, policy_version 1881300 (0.0008) [2023-12-27 05:06:07,701][105620] Updated weights for policy 1, policy_version 1885765 (0.0010) [2023-12-27 05:06:07,735][105692] Updated weights for policy 0, policy_version 1881310 (0.0009) [2023-12-27 05:06:07,757][105620] Updated weights for policy 1, policy_version 1885775 (0.0010) [2023-12-27 05:06:07,790][105692] Updated weights for policy 0, policy_version 1881320 (0.0010) [2023-12-27 05:06:08,463][105620] Updated weights for policy 1, policy_version 1885785 (0.0010) [2023-12-27 05:06:08,514][105692] Updated weights for policy 0, policy_version 1881330 (0.0009) [2023-12-27 05:06:08,530][105620] Updated weights for policy 1, policy_version 1885795 (0.0008) [2023-12-27 05:06:08,566][105692] Updated weights for policy 0, policy_version 1881340 (0.0010) [2023-12-27 05:06:08,595][105620] Updated weights for policy 1, policy_version 1885805 (0.0008) [2023-12-27 05:06:08,626][105692] Updated weights for policy 0, policy_version 1881350 (0.0010) [2023-12-27 05:06:08,699][105692] Updated weights for policy 0, policy_version 1881360 (0.0011) [2023-12-27 05:06:09,207][105620] Updated weights for policy 1, policy_version 1885815 (0.0006) [2023-12-27 05:06:09,266][105620] Updated weights for policy 1, policy_version 1885825 (0.0009) [2023-12-27 05:06:09,325][105620] Updated weights for policy 1, policy_version 1885835 (0.0012) [2023-12-27 05:06:09,445][105692] Updated weights for policy 0, policy_version 1881370 (0.0009) [2023-12-27 05:06:09,508][105692] Updated weights for policy 0, policy_version 1881380 (0.0008) [2023-12-27 05:06:09,573][105692] Updated weights for policy 0, policy_version 1881390 (0.0008) [2023-12-27 05:06:10,092][105620] Updated weights for policy 1, policy_version 1885845 (0.0007) [2023-12-27 05:06:10,156][105620] Updated weights for policy 1, policy_version 1885855 (0.0007) [2023-12-27 05:06:10,224][105620] Updated weights for policy 1, policy_version 1885865 (0.0009) [2023-12-27 05:06:10,289][105692] Updated weights for policy 0, policy_version 1881400 (0.0006) [2023-12-27 05:06:10,350][105692] Updated weights for policy 0, policy_version 1881410 (0.0009) [2023-12-27 05:06:10,412][105692] Updated weights for policy 0, policy_version 1881420 (0.0009) [2023-12-27 05:06:10,849][105620] Updated weights for policy 1, policy_version 1885875 (0.0008) [2023-12-27 05:06:10,900][105620] Updated weights for policy 1, policy_version 1885885 (0.0007) [2023-12-27 05:06:10,948][105620] Updated weights for policy 1, policy_version 1885895 (0.0009) [2023-12-27 05:06:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 964575232. Throughput: 0: 10003.3, 1: 9871.0. Samples: 964582072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:06:11,062][104569] Avg episode reward: [(0, '8255.694'), (1, '9252.916')] [2023-12-27 05:06:11,229][105692] Updated weights for policy 0, policy_version 1881430 (0.0009) [2023-12-27 05:06:11,299][105692] Updated weights for policy 0, policy_version 1881440 (0.0009) [2023-12-27 05:06:11,366][105692] Updated weights for policy 0, policy_version 1881450 (0.0009) [2023-12-27 05:06:11,719][105620] Updated weights for policy 1, policy_version 1885905 (0.0009) [2023-12-27 05:06:11,792][105620] Updated weights for policy 1, policy_version 1885915 (0.0007) [2023-12-27 05:06:11,855][105620] Updated weights for policy 1, policy_version 1885925 (0.0009) [2023-12-27 05:06:11,914][105620] Updated weights for policy 1, policy_version 1885935 (0.0009) [2023-12-27 05:06:12,097][105692] Updated weights for policy 0, policy_version 1881460 (0.0007) [2023-12-27 05:06:12,169][105692] Updated weights for policy 0, policy_version 1881470 (0.0010) [2023-12-27 05:06:12,232][105692] Updated weights for policy 0, policy_version 1881480 (0.0009) [2023-12-27 05:06:12,645][105620] Updated weights for policy 1, policy_version 1885945 (0.0008) [2023-12-27 05:06:12,707][105620] Updated weights for policy 1, policy_version 1885955 (0.0010) [2023-12-27 05:06:12,766][105620] Updated weights for policy 1, policy_version 1885965 (0.0010) [2023-12-27 05:06:12,930][105692] Updated weights for policy 0, policy_version 1881490 (0.0008) [2023-12-27 05:06:12,978][105692] Updated weights for policy 0, policy_version 1881500 (0.0008) [2023-12-27 05:06:13,027][105692] Updated weights for policy 0, policy_version 1881511 (0.0008) [2023-12-27 05:06:13,480][105620] Updated weights for policy 1, policy_version 1885975 (0.0011) [2023-12-27 05:06:13,530][105620] Updated weights for policy 1, policy_version 1885985 (0.0010) [2023-12-27 05:06:13,577][105620] Updated weights for policy 1, policy_version 1885995 (0.0010) [2023-12-27 05:06:13,783][105692] Updated weights for policy 0, policy_version 1881521 (0.0008) [2023-12-27 05:06:13,840][105692] Updated weights for policy 0, policy_version 1881531 (0.0007) [2023-12-27 05:06:13,893][105692] Updated weights for policy 0, policy_version 1881541 (0.0010) [2023-12-27 05:06:13,951][105692] Updated weights for policy 0, policy_version 1881551 (0.0010) [2023-12-27 05:06:14,284][105620] Updated weights for policy 1, policy_version 1886005 (0.0010) [2023-12-27 05:06:14,340][105620] Updated weights for policy 1, policy_version 1886015 (0.0008) [2023-12-27 05:06:14,400][105620] Updated weights for policy 1, policy_version 1886025 (0.0008) [2023-12-27 05:06:14,705][105692] Updated weights for policy 0, policy_version 1881561 (0.0008) [2023-12-27 05:06:14,774][105692] Updated weights for policy 0, policy_version 1881571 (0.0008) [2023-12-27 05:06:14,835][105692] Updated weights for policy 0, policy_version 1881581 (0.0006) [2023-12-27 05:06:15,256][105620] Updated weights for policy 1, policy_version 1886035 (0.0008) [2023-12-27 05:06:15,316][105620] Updated weights for policy 1, policy_version 1886045 (0.0009) [2023-12-27 05:06:15,383][105620] Updated weights for policy 1, policy_version 1886055 (0.0008) [2023-12-27 05:06:15,413][105692] Updated weights for policy 0, policy_version 1881591 (0.0009) [2023-12-27 05:06:15,472][105692] Updated weights for policy 0, policy_version 1881601 (0.0010) [2023-12-27 05:06:15,538][105692] Updated weights for policy 0, policy_version 1881611 (0.0011) [2023-12-27 05:06:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 964665344. Throughput: 0: 9877.8, 1: 9875.4. Samples: 964638304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:06:16,063][104569] Avg episode reward: [(0, '8263.394'), (1, '9252.972')] [2023-12-27 05:06:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001886064_482902016.pth... [2023-12-27 05:06:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001884912_482607104.pth [2023-12-27 05:06:16,104][105692] Updated weights for policy 0, policy_version 1881621 (0.0011) [2023-12-27 05:06:16,158][105692] Updated weights for policy 0, policy_version 1881631 (0.0010) [2023-12-27 05:06:16,216][105692] Updated weights for policy 0, policy_version 1881641 (0.0010) [2023-12-27 05:06:16,225][105620] Updated weights for policy 1, policy_version 1886065 (0.0006) [2023-12-27 05:06:16,260][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001881648_481771520.pth... [2023-12-27 05:06:16,264][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001880464_481468416.pth [2023-12-27 05:06:16,286][105620] Updated weights for policy 1, policy_version 1886075 (0.0007) [2023-12-27 05:06:16,350][105620] Updated weights for policy 1, policy_version 1886085 (0.0008) [2023-12-27 05:06:16,401][105620] Updated weights for policy 1, policy_version 1886095 (0.0008) [2023-12-27 05:06:16,954][105692] Updated weights for policy 0, policy_version 1881651 (0.0011) [2023-12-27 05:06:17,008][105692] Updated weights for policy 0, policy_version 1881661 (0.0010) [2023-12-27 05:06:17,066][105692] Updated weights for policy 0, policy_version 1881671 (0.0010) [2023-12-27 05:06:17,153][105620] Updated weights for policy 1, policy_version 1886105 (0.0008) [2023-12-27 05:06:17,209][105620] Updated weights for policy 1, policy_version 1886115 (0.0008) [2023-12-27 05:06:17,264][105620] Updated weights for policy 1, policy_version 1886125 (0.0007) [2023-12-27 05:06:17,805][105692] Updated weights for policy 0, policy_version 1881681 (0.0010) [2023-12-27 05:06:17,854][105692] Updated weights for policy 0, policy_version 1881691 (0.0008) [2023-12-27 05:06:17,886][105620] Updated weights for policy 1, policy_version 1886135 (0.0009) [2023-12-27 05:06:17,909][105692] Updated weights for policy 0, policy_version 1881701 (0.0005) [2023-12-27 05:06:17,938][105620] Updated weights for policy 1, policy_version 1886145 (0.0011) [2023-12-27 05:06:17,954][105692] Updated weights for policy 0, policy_version 1881711 (0.0009) [2023-12-27 05:06:17,993][105620] Updated weights for policy 1, policy_version 1886155 (0.0010) [2023-12-27 05:06:18,721][105620] Updated weights for policy 1, policy_version 1886165 (0.0010) [2023-12-27 05:06:18,723][105692] Updated weights for policy 0, policy_version 1881721 (0.0011) [2023-12-27 05:06:18,778][105692] Updated weights for policy 0, policy_version 1881731 (0.0010) [2023-12-27 05:06:18,780][105620] Updated weights for policy 1, policy_version 1886175 (0.0011) [2023-12-27 05:06:18,834][105692] Updated weights for policy 0, policy_version 1881741 (0.0011) [2023-12-27 05:06:18,840][105620] Updated weights for policy 1, policy_version 1886185 (0.0011) [2023-12-27 05:06:19,557][105620] Updated weights for policy 1, policy_version 1886195 (0.0010) [2023-12-27 05:06:19,557][105692] Updated weights for policy 0, policy_version 1881751 (0.0008) [2023-12-27 05:06:19,613][105620] Updated weights for policy 1, policy_version 1886205 (0.0011) [2023-12-27 05:06:19,617][105692] Updated weights for policy 0, policy_version 1881761 (0.0006) [2023-12-27 05:06:19,669][105620] Updated weights for policy 1, policy_version 1886215 (0.0011) [2023-12-27 05:06:19,675][105692] Updated weights for policy 0, policy_version 1881771 (0.0006) [2023-12-27 05:06:20,346][105692] Updated weights for policy 0, policy_version 1881781 (0.0008) [2023-12-27 05:06:20,351][105620] Updated weights for policy 1, policy_version 1886225 (0.0010) [2023-12-27 05:06:20,413][105692] Updated weights for policy 0, policy_version 1881791 (0.0009) [2023-12-27 05:06:20,418][105620] Updated weights for policy 1, policy_version 1886235 (0.0007) [2023-12-27 05:06:20,477][105692] Updated weights for policy 0, policy_version 1881801 (0.0008) [2023-12-27 05:06:20,490][105620] Updated weights for policy 1, policy_version 1886245 (0.0009) [2023-12-27 05:06:20,559][105620] Updated weights for policy 1, policy_version 1886255 (0.0010) [2023-12-27 05:06:21,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 964763648. Throughput: 0: 9894.5, 1: 9838.7. Samples: 964754008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:06:21,063][104569] Avg episode reward: [(0, '8082.344'), (1, '9252.935')] [2023-12-27 05:06:21,249][105692] Updated weights for policy 0, policy_version 1881811 (0.0010) [2023-12-27 05:06:21,259][105620] Updated weights for policy 1, policy_version 1886265 (0.0008) [2023-12-27 05:06:21,313][105692] Updated weights for policy 0, policy_version 1881821 (0.0011) [2023-12-27 05:06:21,315][105620] Updated weights for policy 1, policy_version 1886275 (0.0007) [2023-12-27 05:06:21,377][105692] Updated weights for policy 0, policy_version 1881831 (0.0010) [2023-12-27 05:06:21,378][105620] Updated weights for policy 1, policy_version 1886285 (0.0009) [2023-12-27 05:06:22,052][105692] Updated weights for policy 0, policy_version 1881841 (0.0009) [2023-12-27 05:06:22,104][105692] Updated weights for policy 0, policy_version 1881851 (0.0007) [2023-12-27 05:06:22,138][105620] Updated weights for policy 1, policy_version 1886295 (0.0008) [2023-12-27 05:06:22,163][105692] Updated weights for policy 0, policy_version 1881861 (0.0010) [2023-12-27 05:06:22,198][105620] Updated weights for policy 1, policy_version 1886305 (0.0006) [2023-12-27 05:06:22,223][105692] Updated weights for policy 0, policy_version 1881871 (0.0011) [2023-12-27 05:06:22,254][105620] Updated weights for policy 1, policy_version 1886315 (0.0007) [2023-12-27 05:06:22,958][105692] Updated weights for policy 0, policy_version 1881881 (0.0011) [2023-12-27 05:06:23,017][105692] Updated weights for policy 0, policy_version 1881891 (0.0010) [2023-12-27 05:06:23,055][105620] Updated weights for policy 1, policy_version 1886325 (0.0008) [2023-12-27 05:06:23,075][105692] Updated weights for policy 0, policy_version 1881901 (0.0010) [2023-12-27 05:06:23,102][105620] Updated weights for policy 1, policy_version 1886335 (0.0008) [2023-12-27 05:06:23,161][105620] Updated weights for policy 1, policy_version 1886345 (0.0009) [2023-12-27 05:06:23,801][105692] Updated weights for policy 0, policy_version 1881911 (0.0007) [2023-12-27 05:06:23,825][105620] Updated weights for policy 1, policy_version 1886355 (0.0009) [2023-12-27 05:06:23,864][105692] Updated weights for policy 0, policy_version 1881921 (0.0006) [2023-12-27 05:06:23,887][105620] Updated weights for policy 1, policy_version 1886365 (0.0011) [2023-12-27 05:06:23,924][105692] Updated weights for policy 0, policy_version 1881931 (0.0008) [2023-12-27 05:06:23,945][105620] Updated weights for policy 1, policy_version 1886375 (0.0010) [2023-12-27 05:06:24,645][105692] Updated weights for policy 0, policy_version 1881941 (0.0008) [2023-12-27 05:06:24,663][105620] Updated weights for policy 1, policy_version 1886385 (0.0010) [2023-12-27 05:06:24,704][105692] Updated weights for policy 0, policy_version 1881951 (0.0007) [2023-12-27 05:06:24,720][105620] Updated weights for policy 1, policy_version 1886395 (0.0006) [2023-12-27 05:06:24,772][105692] Updated weights for policy 0, policy_version 1881961 (0.0007) [2023-12-27 05:06:24,779][105620] Updated weights for policy 1, policy_version 1886405 (0.0005) [2023-12-27 05:06:24,836][105620] Updated weights for policy 1, policy_version 1886415 (0.0005) [2023-12-27 05:06:25,448][105620] Updated weights for policy 1, policy_version 1886425 (0.0005) [2023-12-27 05:06:25,513][105620] Updated weights for policy 1, policy_version 1886435 (0.0006) [2023-12-27 05:06:25,565][105692] Updated weights for policy 0, policy_version 1881971 (0.0008) [2023-12-27 05:06:25,574][105620] Updated weights for policy 1, policy_version 1886445 (0.0006) [2023-12-27 05:06:25,613][105692] Updated weights for policy 0, policy_version 1881981 (0.0007) [2023-12-27 05:06:25,661][105692] Updated weights for policy 0, policy_version 1881991 (0.0005) [2023-12-27 05:06:26,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 964861952. Throughput: 0: 9855.0, 1: 9874.3. Samples: 964870828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:06:26,062][104569] Avg episode reward: [(0, '8256.520'), (1, '9161.029')] [2023-12-27 05:06:26,200][105620] Updated weights for policy 1, policy_version 1886455 (0.0009) [2023-12-27 05:06:26,244][105620] Updated weights for policy 1, policy_version 1886465 (0.0010) [2023-12-27 05:06:26,260][105692] Updated weights for policy 0, policy_version 1882001 (0.0005) [2023-12-27 05:06:26,289][105620] Updated weights for policy 1, policy_version 1886475 (0.0010) [2023-12-27 05:06:26,322][105692] Updated weights for policy 0, policy_version 1882011 (0.0006) [2023-12-27 05:06:26,377][105692] Updated weights for policy 0, policy_version 1882021 (0.0005) [2023-12-27 05:06:26,431][105692] Updated weights for policy 0, policy_version 1882031 (0.0006) [2023-12-27 05:06:27,055][105620] Updated weights for policy 1, policy_version 1886485 (0.0009) [2023-12-27 05:06:27,089][105692] Updated weights for policy 0, policy_version 1882041 (0.0006) [2023-12-27 05:06:27,106][105620] Updated weights for policy 1, policy_version 1886495 (0.0007) [2023-12-27 05:06:27,146][105692] Updated weights for policy 0, policy_version 1882051 (0.0005) [2023-12-27 05:06:27,167][105620] Updated weights for policy 1, policy_version 1886505 (0.0005) [2023-12-27 05:06:27,203][105692] Updated weights for policy 0, policy_version 1882061 (0.0009) [2023-12-27 05:06:27,736][105620] Updated weights for policy 1, policy_version 1886515 (0.0005) [2023-12-27 05:06:27,756][105692] Updated weights for policy 0, policy_version 1882071 (0.0007) [2023-12-27 05:06:27,792][105620] Updated weights for policy 1, policy_version 1886525 (0.0007) [2023-12-27 05:06:27,813][105692] Updated weights for policy 0, policy_version 1882081 (0.0006) [2023-12-27 05:06:27,848][105620] Updated weights for policy 1, policy_version 1886535 (0.0007) [2023-12-27 05:06:27,879][105692] Updated weights for policy 0, policy_version 1882091 (0.0005) [2023-12-27 05:06:28,398][105692] Updated weights for policy 0, policy_version 1882101 (0.0007) [2023-12-27 05:06:28,458][105692] Updated weights for policy 0, policy_version 1882111 (0.0008) [2023-12-27 05:06:28,458][105620] Updated weights for policy 1, policy_version 1886545 (0.0005) [2023-12-27 05:06:28,516][105620] Updated weights for policy 1, policy_version 1886555 (0.0008) [2023-12-27 05:06:28,520][105692] Updated weights for policy 0, policy_version 1882121 (0.0005) [2023-12-27 05:06:28,569][105620] Updated weights for policy 1, policy_version 1886565 (0.0005) [2023-12-27 05:06:28,620][105620] Updated weights for policy 1, policy_version 1886575 (0.0009) [2023-12-27 05:06:29,085][105692] Updated weights for policy 0, policy_version 1882131 (0.0005) [2023-12-27 05:06:29,140][105692] Updated weights for policy 0, policy_version 1882141 (0.0005) [2023-12-27 05:06:29,204][105692] Updated weights for policy 0, policy_version 1882151 (0.0005) [2023-12-27 05:06:29,462][105620] Updated weights for policy 1, policy_version 1886585 (0.0009) [2023-12-27 05:06:29,528][105620] Updated weights for policy 1, policy_version 1886595 (0.0009) [2023-12-27 05:06:29,588][105620] Updated weights for policy 1, policy_version 1886605 (0.0008) [2023-12-27 05:06:29,808][105692] Updated weights for policy 0, policy_version 1882161 (0.0007) [2023-12-27 05:06:29,872][105692] Updated weights for policy 0, policy_version 1882171 (0.0010) [2023-12-27 05:06:29,935][105692] Updated weights for policy 0, policy_version 1882181 (0.0009) [2023-12-27 05:06:29,994][105692] Updated weights for policy 0, policy_version 1882191 (0.0008) [2023-12-27 05:06:30,358][105620] Updated weights for policy 1, policy_version 1886615 (0.0009) [2023-12-27 05:06:30,415][105620] Updated weights for policy 1, policy_version 1886625 (0.0009) [2023-12-27 05:06:30,480][105620] Updated weights for policy 1, policy_version 1886635 (0.0010) [2023-12-27 05:06:30,676][105692] Updated weights for policy 0, policy_version 1882201 (0.0009) [2023-12-27 05:06:30,724][105692] Updated weights for policy 0, policy_version 1882211 (0.0010) [2023-12-27 05:06:30,769][105692] Updated weights for policy 0, policy_version 1882221 (0.0010) [2023-12-27 05:06:31,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19577.5). Total num frames: 964968448. Throughput: 0: 9939.5, 1: 9930.2. Samples: 964936224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:06:31,063][104569] Avg episode reward: [(0, '8086.568'), (1, '8976.476')] [2023-12-27 05:06:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001882224_481918976.pth... [2023-12-27 05:06:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001886640_483049472.pth... [2023-12-27 05:06:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001881072_481624064.pth [2023-12-27 05:06:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001885520_482762752.pth [2023-12-27 05:06:31,299][105620] Updated weights for policy 1, policy_version 1886645 (0.0009) [2023-12-27 05:06:31,356][105620] Updated weights for policy 1, policy_version 1886655 (0.0008) [2023-12-27 05:06:31,411][105620] Updated weights for policy 1, policy_version 1886665 (0.0007) [2023-12-27 05:06:31,555][105692] Updated weights for policy 0, policy_version 1882231 (0.0010) [2023-12-27 05:06:31,607][105692] Updated weights for policy 0, policy_version 1882241 (0.0010) [2023-12-27 05:06:31,667][105692] Updated weights for policy 0, policy_version 1882251 (0.0010) [2023-12-27 05:06:32,126][105620] Updated weights for policy 1, policy_version 1886675 (0.0010) [2023-12-27 05:06:32,183][105620] Updated weights for policy 1, policy_version 1886685 (0.0010) [2023-12-27 05:06:32,238][105620] Updated weights for policy 1, policy_version 1886695 (0.0010) [2023-12-27 05:06:32,417][105692] Updated weights for policy 0, policy_version 1882261 (0.0010) [2023-12-27 05:06:32,468][105692] Updated weights for policy 0, policy_version 1882271 (0.0010) [2023-12-27 05:06:32,519][105692] Updated weights for policy 0, policy_version 1882281 (0.0010) [2023-12-27 05:06:32,945][105620] Updated weights for policy 1, policy_version 1886705 (0.0010) [2023-12-27 05:06:32,995][105620] Updated weights for policy 1, policy_version 1886715 (0.0005) [2023-12-27 05:06:33,060][105620] Updated weights for policy 1, policy_version 1886725 (0.0008) [2023-12-27 05:06:33,121][105620] Updated weights for policy 1, policy_version 1886735 (0.0009) [2023-12-27 05:06:33,229][105692] Updated weights for policy 0, policy_version 1882291 (0.0006) [2023-12-27 05:06:33,276][105692] Updated weights for policy 0, policy_version 1882301 (0.0008) [2023-12-27 05:06:33,322][105692] Updated weights for policy 0, policy_version 1882311 (0.0009) [2023-12-27 05:06:33,715][105620] Updated weights for policy 1, policy_version 1886745 (0.0008) [2023-12-27 05:06:33,758][105620] Updated weights for policy 1, policy_version 1886755 (0.0007) [2023-12-27 05:06:33,802][105620] Updated weights for policy 1, policy_version 1886765 (0.0008) [2023-12-27 05:06:33,963][105692] Updated weights for policy 0, policy_version 1882321 (0.0007) [2023-12-27 05:06:34,010][105692] Updated weights for policy 0, policy_version 1882331 (0.0010) [2023-12-27 05:06:34,061][105692] Updated weights for policy 0, policy_version 1882341 (0.0008) [2023-12-27 05:06:34,120][105692] Updated weights for policy 0, policy_version 1882351 (0.0006) [2023-12-27 05:06:34,646][105620] Updated weights for policy 1, policy_version 1886775 (0.0009) [2023-12-27 05:06:34,719][105620] Updated weights for policy 1, policy_version 1886785 (0.0009) [2023-12-27 05:06:34,764][105692] Updated weights for policy 0, policy_version 1882361 (0.0006) [2023-12-27 05:06:34,777][105620] Updated weights for policy 1, policy_version 1886795 (0.0008) [2023-12-27 05:06:34,814][105692] Updated weights for policy 0, policy_version 1882371 (0.0006) [2023-12-27 05:06:34,875][105692] Updated weights for policy 0, policy_version 1882381 (0.0008) [2023-12-27 05:06:35,530][105620] Updated weights for policy 1, policy_version 1886805 (0.0008) [2023-12-27 05:06:35,578][105620] Updated weights for policy 1, policy_version 1886815 (0.0007) [2023-12-27 05:06:35,587][105692] Updated weights for policy 0, policy_version 1882391 (0.0007) [2023-12-27 05:06:35,638][105620] Updated weights for policy 1, policy_version 1886825 (0.0007) [2023-12-27 05:06:35,640][105692] Updated weights for policy 0, policy_version 1882401 (0.0006) [2023-12-27 05:06:35,694][105692] Updated weights for policy 0, policy_version 1882411 (0.0006) [2023-12-27 05:06:36,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 965066752. Throughput: 0: 9958.5, 1: 9867.0. Samples: 965054412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:06:36,063][104569] Avg episode reward: [(0, '8364.892'), (1, '9068.555')] [2023-12-27 05:06:36,377][105620] Updated weights for policy 1, policy_version 1886835 (0.0008) [2023-12-27 05:06:36,444][105620] Updated weights for policy 1, policy_version 1886845 (0.0009) [2023-12-27 05:06:36,455][105692] Updated weights for policy 0, policy_version 1882421 (0.0007) [2023-12-27 05:06:36,506][105620] Updated weights for policy 1, policy_version 1886855 (0.0009) [2023-12-27 05:06:36,516][105692] Updated weights for policy 0, policy_version 1882431 (0.0007) [2023-12-27 05:06:36,579][105692] Updated weights for policy 0, policy_version 1882441 (0.0007) [2023-12-27 05:06:37,197][105620] Updated weights for policy 1, policy_version 1886865 (0.0007) [2023-12-27 05:06:37,249][105620] Updated weights for policy 1, policy_version 1886875 (0.0009) [2023-12-27 05:06:37,297][105620] Updated weights for policy 1, policy_version 1886885 (0.0009) [2023-12-27 05:06:37,355][105620] Updated weights for policy 1, policy_version 1886895 (0.0009) [2023-12-27 05:06:37,356][105692] Updated weights for policy 0, policy_version 1882451 (0.0009) [2023-12-27 05:06:37,407][105692] Updated weights for policy 0, policy_version 1882461 (0.0009) [2023-12-27 05:06:37,458][105692] Updated weights for policy 0, policy_version 1882471 (0.0009) [2023-12-27 05:06:38,129][105620] Updated weights for policy 1, policy_version 1886905 (0.0009) [2023-12-27 05:06:38,180][105620] Updated weights for policy 1, policy_version 1886915 (0.0008) [2023-12-27 05:06:38,226][105620] Updated weights for policy 1, policy_version 1886925 (0.0009) [2023-12-27 05:06:38,230][105692] Updated weights for policy 0, policy_version 1882481 (0.0009) [2023-12-27 05:06:38,288][105692] Updated weights for policy 0, policy_version 1882491 (0.0008) [2023-12-27 05:06:38,351][105692] Updated weights for policy 0, policy_version 1882501 (0.0009) [2023-12-27 05:06:38,414][105692] Updated weights for policy 0, policy_version 1882511 (0.0009) [2023-12-27 05:06:39,006][105620] Updated weights for policy 1, policy_version 1886935 (0.0009) [2023-12-27 05:06:39,067][105620] Updated weights for policy 1, policy_version 1886945 (0.0009) [2023-12-27 05:06:39,128][105620] Updated weights for policy 1, policy_version 1886955 (0.0009) [2023-12-27 05:06:39,164][105692] Updated weights for policy 0, policy_version 1882521 (0.0008) [2023-12-27 05:06:39,215][105692] Updated weights for policy 0, policy_version 1882531 (0.0009) [2023-12-27 05:06:39,280][105692] Updated weights for policy 0, policy_version 1882541 (0.0009) [2023-12-27 05:06:39,840][105620] Updated weights for policy 1, policy_version 1886965 (0.0008) [2023-12-27 05:06:39,902][105620] Updated weights for policy 1, policy_version 1886975 (0.0008) [2023-12-27 05:06:39,968][105620] Updated weights for policy 1, policy_version 1886985 (0.0008) [2023-12-27 05:06:40,091][105692] Updated weights for policy 0, policy_version 1882551 (0.0006) [2023-12-27 05:06:40,147][105692] Updated weights for policy 0, policy_version 1882561 (0.0006) [2023-12-27 05:06:40,217][105692] Updated weights for policy 0, policy_version 1882571 (0.0006) [2023-12-27 05:06:40,754][105620] Updated weights for policy 1, policy_version 1886995 (0.0009) [2023-12-27 05:06:40,826][105620] Updated weights for policy 1, policy_version 1887005 (0.0010) [2023-12-27 05:06:40,845][105692] Updated weights for policy 0, policy_version 1882581 (0.0008) [2023-12-27 05:06:40,889][105620] Updated weights for policy 1, policy_version 1887015 (0.0011) [2023-12-27 05:06:40,908][105692] Updated weights for policy 0, policy_version 1882591 (0.0006) [2023-12-27 05:06:40,957][105692] Updated weights for policy 0, policy_version 1882601 (0.0008) [2023-12-27 05:06:41,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 965165056. Throughput: 0: 9952.1, 1: 9730.3. Samples: 965167344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:06:41,062][104569] Avg episode reward: [(0, '8996.269'), (1, '9345.336')] [2023-12-27 05:06:41,594][105620] Updated weights for policy 1, policy_version 1887025 (0.0011) [2023-12-27 05:06:41,642][105692] Updated weights for policy 0, policy_version 1882611 (0.0008) [2023-12-27 05:06:41,664][105620] Updated weights for policy 1, policy_version 1887035 (0.0008) [2023-12-27 05:06:41,714][105692] Updated weights for policy 0, policy_version 1882621 (0.0010) [2023-12-27 05:06:41,733][105620] Updated weights for policy 1, policy_version 1887045 (0.0008) [2023-12-27 05:06:41,788][105692] Updated weights for policy 0, policy_version 1882631 (0.0009) [2023-12-27 05:06:41,802][105620] Updated weights for policy 1, policy_version 1887055 (0.0006) [2023-12-27 05:06:42,552][105620] Updated weights for policy 1, policy_version 1887065 (0.0009) [2023-12-27 05:06:42,591][105692] Updated weights for policy 0, policy_version 1882641 (0.0009) [2023-12-27 05:06:42,607][105620] Updated weights for policy 1, policy_version 1887075 (0.0008) [2023-12-27 05:06:42,638][105692] Updated weights for policy 0, policy_version 1882651 (0.0008) [2023-12-27 05:06:42,666][105620] Updated weights for policy 1, policy_version 1887085 (0.0007) [2023-12-27 05:06:42,694][105692] Updated weights for policy 0, policy_version 1882661 (0.0007) [2023-12-27 05:06:42,753][105692] Updated weights for policy 0, policy_version 1882671 (0.0009) [2023-12-27 05:06:43,448][105620] Updated weights for policy 1, policy_version 1887095 (0.0008) [2023-12-27 05:06:43,472][105692] Updated weights for policy 0, policy_version 1882681 (0.0008) [2023-12-27 05:06:43,503][105620] Updated weights for policy 1, policy_version 1887105 (0.0008) [2023-12-27 05:06:43,534][105692] Updated weights for policy 0, policy_version 1882691 (0.0008) [2023-12-27 05:06:43,546][105620] Updated weights for policy 1, policy_version 1887115 (0.0005) [2023-12-27 05:06:43,584][105692] Updated weights for policy 0, policy_version 1882701 (0.0008) [2023-12-27 05:06:44,298][105620] Updated weights for policy 1, policy_version 1887125 (0.0007) [2023-12-27 05:06:44,361][105620] Updated weights for policy 1, policy_version 1887135 (0.0007) [2023-12-27 05:06:44,363][105692] Updated weights for policy 0, policy_version 1882711 (0.0007) [2023-12-27 05:06:44,416][105692] Updated weights for policy 0, policy_version 1882721 (0.0006) [2023-12-27 05:06:44,422][105620] Updated weights for policy 1, policy_version 1887145 (0.0009) [2023-12-27 05:06:44,468][105692] Updated weights for policy 0, policy_version 1882731 (0.0007) [2023-12-27 05:06:45,084][105620] Updated weights for policy 1, policy_version 1887155 (0.0008) [2023-12-27 05:06:45,138][105620] Updated weights for policy 1, policy_version 1887165 (0.0008) [2023-12-27 05:06:45,194][105620] Updated weights for policy 1, policy_version 1887175 (0.0009) [2023-12-27 05:06:45,296][105692] Updated weights for policy 0, policy_version 1882741 (0.0009) [2023-12-27 05:06:45,360][105692] Updated weights for policy 0, policy_version 1882751 (0.0010) [2023-12-27 05:06:45,420][105692] Updated weights for policy 0, policy_version 1882761 (0.0010) [2023-12-27 05:06:45,838][105620] Updated weights for policy 1, policy_version 1887185 (0.0008) [2023-12-27 05:06:45,904][105620] Updated weights for policy 1, policy_version 1887195 (0.0006) [2023-12-27 05:06:45,974][105620] Updated weights for policy 1, policy_version 1887205 (0.0006) [2023-12-27 05:06:46,034][105620] Updated weights for policy 1, policy_version 1887215 (0.0005) [2023-12-27 05:06:46,050][105692] Updated weights for policy 0, policy_version 1882771 (0.0008) [2023-12-27 05:06:46,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 965255168. Throughput: 0: 9813.1, 1: 9677.7. Samples: 965223880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:06:46,063][104569] Avg episode reward: [(0, '8442.235'), (1, '9345.405')] [2023-12-27 05:06:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001887216_483196928.pth... [2023-12-27 05:06:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001886064_482902016.pth [2023-12-27 05:06:46,106][105692] Updated weights for policy 0, policy_version 1882781 (0.0005) [2023-12-27 05:06:46,172][105692] Updated weights for policy 0, policy_version 1882791 (0.0005) [2023-12-27 05:06:46,229][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001882800_482066432.pth... [2023-12-27 05:06:46,233][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001881648_481771520.pth [2023-12-27 05:06:46,719][105620] Updated weights for policy 1, policy_version 1887225 (0.0010) [2023-12-27 05:06:46,720][105692] Updated weights for policy 0, policy_version 1882801 (0.0007) [2023-12-27 05:06:46,772][105620] Updated weights for policy 1, policy_version 1887235 (0.0009) [2023-12-27 05:06:46,780][105692] Updated weights for policy 0, policy_version 1882811 (0.0005) [2023-12-27 05:06:46,831][105692] Updated weights for policy 0, policy_version 1882821 (0.0005) [2023-12-27 05:06:46,835][105620] Updated weights for policy 1, policy_version 1887245 (0.0008) [2023-12-27 05:06:46,888][105692] Updated weights for policy 0, policy_version 1882831 (0.0005) [2023-12-27 05:06:47,521][105620] Updated weights for policy 1, policy_version 1887255 (0.0007) [2023-12-27 05:06:47,535][105692] Updated weights for policy 0, policy_version 1882841 (0.0008) [2023-12-27 05:06:47,570][105620] Updated weights for policy 1, policy_version 1887265 (0.0006) [2023-12-27 05:06:47,581][105692] Updated weights for policy 0, policy_version 1882851 (0.0008) [2023-12-27 05:06:47,625][105620] Updated weights for policy 1, policy_version 1887275 (0.0010) [2023-12-27 05:06:47,629][105692] Updated weights for policy 0, policy_version 1882861 (0.0009) [2023-12-27 05:06:48,220][105620] Updated weights for policy 1, policy_version 1887285 (0.0006) [2023-12-27 05:06:48,276][105620] Updated weights for policy 1, policy_version 1887295 (0.0005) [2023-12-27 05:06:48,324][105620] Updated weights for policy 1, policy_version 1887305 (0.0010) [2023-12-27 05:06:48,503][105692] Updated weights for policy 0, policy_version 1882871 (0.0008) [2023-12-27 05:06:48,566][105692] Updated weights for policy 0, policy_version 1882881 (0.0010) [2023-12-27 05:06:48,621][105692] Updated weights for policy 0, policy_version 1882891 (0.0008) [2023-12-27 05:06:49,052][105620] Updated weights for policy 1, policy_version 1887315 (0.0010) [2023-12-27 05:06:49,121][105620] Updated weights for policy 1, policy_version 1887325 (0.0010) [2023-12-27 05:06:49,186][105620] Updated weights for policy 1, policy_version 1887335 (0.0010) [2023-12-27 05:06:49,377][105692] Updated weights for policy 0, policy_version 1882901 (0.0009) [2023-12-27 05:06:49,445][105692] Updated weights for policy 0, policy_version 1882911 (0.0006) [2023-12-27 05:06:49,502][105692] Updated weights for policy 0, policy_version 1882921 (0.0005) [2023-12-27 05:06:49,926][105620] Updated weights for policy 1, policy_version 1887345 (0.0010) [2023-12-27 05:06:50,002][105620] Updated weights for policy 1, policy_version 1887355 (0.0010) [2023-12-27 05:06:50,058][105620] Updated weights for policy 1, policy_version 1887365 (0.0011) [2023-12-27 05:06:50,121][105620] Updated weights for policy 1, policy_version 1887375 (0.0011) [2023-12-27 05:06:50,207][105692] Updated weights for policy 0, policy_version 1882931 (0.0006) [2023-12-27 05:06:50,259][105692] Updated weights for policy 0, policy_version 1882941 (0.0007) [2023-12-27 05:06:50,311][105692] Updated weights for policy 0, policy_version 1882951 (0.0008) [2023-12-27 05:06:50,858][105620] Updated weights for policy 1, policy_version 1887385 (0.0010) [2023-12-27 05:06:50,924][105620] Updated weights for policy 1, policy_version 1887395 (0.0011) [2023-12-27 05:06:50,978][105692] Updated weights for policy 0, policy_version 1882961 (0.0006) [2023-12-27 05:06:50,987][105620] Updated weights for policy 1, policy_version 1887405 (0.0011) [2023-12-27 05:06:51,032][105692] Updated weights for policy 0, policy_version 1882971 (0.0006) [2023-12-27 05:06:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 965353472. Throughput: 0: 9852.4, 1: 9750.6. Samples: 965342392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:06:51,063][104569] Avg episode reward: [(0, '7898.994'), (1, '9253.129')] [2023-12-27 05:06:51,100][105692] Updated weights for policy 0, policy_version 1882981 (0.0007) [2023-12-27 05:06:51,165][105692] Updated weights for policy 0, policy_version 1882991 (0.0008) [2023-12-27 05:06:51,777][105620] Updated weights for policy 1, policy_version 1887415 (0.0009) [2023-12-27 05:06:51,843][105620] Updated weights for policy 1, policy_version 1887425 (0.0010) [2023-12-27 05:06:51,877][105692] Updated weights for policy 0, policy_version 1883001 (0.0006) [2023-12-27 05:06:51,901][105620] Updated weights for policy 1, policy_version 1887435 (0.0010) [2023-12-27 05:06:51,926][105692] Updated weights for policy 0, policy_version 1883011 (0.0009) [2023-12-27 05:06:51,974][105692] Updated weights for policy 0, policy_version 1883021 (0.0008) [2023-12-27 05:06:52,659][105620] Updated weights for policy 1, policy_version 1887445 (0.0010) [2023-12-27 05:06:52,722][105620] Updated weights for policy 1, policy_version 1887455 (0.0011) [2023-12-27 05:06:52,780][105620] Updated weights for policy 1, policy_version 1887465 (0.0008) [2023-12-27 05:06:52,802][105692] Updated weights for policy 0, policy_version 1883031 (0.0009) [2023-12-27 05:06:52,858][105692] Updated weights for policy 0, policy_version 1883041 (0.0009) [2023-12-27 05:06:52,911][105692] Updated weights for policy 0, policy_version 1883051 (0.0010) [2023-12-27 05:06:53,336][105620] Updated weights for policy 1, policy_version 1887475 (0.0006) [2023-12-27 05:06:53,385][105620] Updated weights for policy 1, policy_version 1887485 (0.0005) [2023-12-27 05:06:53,437][105620] Updated weights for policy 1, policy_version 1887495 (0.0005) [2023-12-27 05:06:53,606][105692] Updated weights for policy 0, policy_version 1883061 (0.0008) [2023-12-27 05:06:53,673][105692] Updated weights for policy 0, policy_version 1883071 (0.0007) [2023-12-27 05:06:53,731][105692] Updated weights for policy 0, policy_version 1883081 (0.0010) [2023-12-27 05:06:53,998][105620] Updated weights for policy 1, policy_version 1887505 (0.0008) [2023-12-27 05:06:54,062][105620] Updated weights for policy 1, policy_version 1887515 (0.0005) [2023-12-27 05:06:54,128][105620] Updated weights for policy 1, policy_version 1887525 (0.0008) [2023-12-27 05:06:54,192][105620] Updated weights for policy 1, policy_version 1887535 (0.0007) [2023-12-27 05:06:54,438][105692] Updated weights for policy 0, policy_version 1883091 (0.0009) [2023-12-27 05:06:54,486][105692] Updated weights for policy 0, policy_version 1883101 (0.0008) [2023-12-27 05:06:54,539][105692] Updated weights for policy 0, policy_version 1883111 (0.0008) [2023-12-27 05:06:54,852][105620] Updated weights for policy 1, policy_version 1887545 (0.0010) [2023-12-27 05:06:54,900][105620] Updated weights for policy 1, policy_version 1887555 (0.0010) [2023-12-27 05:06:54,944][105620] Updated weights for policy 1, policy_version 1887565 (0.0010) [2023-12-27 05:06:55,330][105692] Updated weights for policy 0, policy_version 1883121 (0.0008) [2023-12-27 05:06:55,389][105692] Updated weights for policy 0, policy_version 1883131 (0.0008) [2023-12-27 05:06:55,445][105692] Updated weights for policy 0, policy_version 1883141 (0.0008) [2023-12-27 05:06:55,500][105692] Updated weights for policy 0, policy_version 1883151 (0.0008) [2023-12-27 05:06:55,717][105620] Updated weights for policy 1, policy_version 1887575 (0.0010) [2023-12-27 05:06:55,767][105620] Updated weights for policy 1, policy_version 1887585 (0.0010) [2023-12-27 05:06:55,815][105620] Updated weights for policy 1, policy_version 1887595 (0.0010) [2023-12-27 05:06:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 965451776. Throughput: 0: 9819.3, 1: 9681.7. Samples: 965459616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:06:56,063][104569] Avg episode reward: [(0, '8080.839'), (1, '9253.083')] [2023-12-27 05:06:56,247][105692] Updated weights for policy 0, policy_version 1883161 (0.0008) [2023-12-27 05:06:56,291][105692] Updated weights for policy 0, policy_version 1883171 (0.0007) [2023-12-27 05:06:56,342][105692] Updated weights for policy 0, policy_version 1883181 (0.0009) [2023-12-27 05:06:56,564][105620] Updated weights for policy 1, policy_version 1887605 (0.0008) [2023-12-27 05:06:56,619][105620] Updated weights for policy 1, policy_version 1887615 (0.0005) [2023-12-27 05:06:56,669][105620] Updated weights for policy 1, policy_version 1887625 (0.0005) [2023-12-27 05:06:57,165][105620] Updated weights for policy 1, policy_version 1887635 (0.0005) [2023-12-27 05:06:57,217][105620] Updated weights for policy 1, policy_version 1887645 (0.0005) [2023-12-27 05:06:57,227][105692] Updated weights for policy 0, policy_version 1883191 (0.0009) [2023-12-27 05:06:57,267][105620] Updated weights for policy 1, policy_version 1887655 (0.0008) [2023-12-27 05:06:57,285][105692] Updated weights for policy 0, policy_version 1883201 (0.0007) [2023-12-27 05:06:57,340][105692] Updated weights for policy 0, policy_version 1883211 (0.0009) [2023-12-27 05:06:57,888][105620] Updated weights for policy 1, policy_version 1887665 (0.0006) [2023-12-27 05:06:57,952][105620] Updated weights for policy 1, policy_version 1887675 (0.0005) [2023-12-27 05:06:58,012][105620] Updated weights for policy 1, policy_version 1887685 (0.0005) [2023-12-27 05:06:58,072][105620] Updated weights for policy 1, policy_version 1887695 (0.0005) [2023-12-27 05:06:58,203][105692] Updated weights for policy 0, policy_version 1883221 (0.0009) [2023-12-27 05:06:58,271][105692] Updated weights for policy 0, policy_version 1883231 (0.0007) [2023-12-27 05:06:58,339][105692] Updated weights for policy 0, policy_version 1883241 (0.0009) [2023-12-27 05:06:58,684][105620] Updated weights for policy 1, policy_version 1887705 (0.0010) [2023-12-27 05:06:58,753][105620] Updated weights for policy 1, policy_version 1887715 (0.0009) [2023-12-27 05:06:58,821][105620] Updated weights for policy 1, policy_version 1887725 (0.0007) [2023-12-27 05:06:59,114][105692] Updated weights for policy 0, policy_version 1883251 (0.0007) [2023-12-27 05:06:59,167][105692] Updated weights for policy 0, policy_version 1883261 (0.0008) [2023-12-27 05:06:59,225][105692] Updated weights for policy 0, policy_version 1883271 (0.0008) [2023-12-27 05:06:59,585][105620] Updated weights for policy 1, policy_version 1887735 (0.0007) [2023-12-27 05:06:59,643][105620] Updated weights for policy 1, policy_version 1887745 (0.0005) [2023-12-27 05:06:59,700][105620] Updated weights for policy 1, policy_version 1887755 (0.0005) [2023-12-27 05:07:00,102][105692] Updated weights for policy 0, policy_version 1883281 (0.0009) [2023-12-27 05:07:00,154][105692] Updated weights for policy 0, policy_version 1883291 (0.0008) [2023-12-27 05:07:00,210][105692] Updated weights for policy 0, policy_version 1883301 (0.0008) [2023-12-27 05:07:00,265][105692] Updated weights for policy 0, policy_version 1883311 (0.0008) [2023-12-27 05:07:00,333][105620] Updated weights for policy 1, policy_version 1887765 (0.0007) [2023-12-27 05:07:00,394][105620] Updated weights for policy 1, policy_version 1887775 (0.0009) [2023-12-27 05:07:00,457][105620] Updated weights for policy 1, policy_version 1887785 (0.0009) [2023-12-27 05:07:01,040][105692] Updated weights for policy 0, policy_version 1883321 (0.0008) [2023-12-27 05:07:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 965541888. Throughput: 0: 9776.0, 1: 9779.5. Samples: 965518300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:07:01,062][104569] Avg episode reward: [(0, '8081.607'), (1, '9345.334')] [2023-12-27 05:07:01,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001887792_483344384.pth... [2023-12-27 05:07:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001886640_483049472.pth [2023-12-27 05:07:01,102][105692] Updated weights for policy 0, policy_version 1883331 (0.0009) [2023-12-27 05:07:01,167][105692] Updated weights for policy 0, policy_version 1883341 (0.0007) [2023-12-27 05:07:01,186][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001883344_482205696.pth... [2023-12-27 05:07:01,191][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001882224_481918976.pth [2023-12-27 05:07:01,211][105620] Updated weights for policy 1, policy_version 1887795 (0.0008) [2023-12-27 05:07:01,269][105620] Updated weights for policy 1, policy_version 1887805 (0.0009) [2023-12-27 05:07:01,327][105620] Updated weights for policy 1, policy_version 1887815 (0.0008) [2023-12-27 05:07:01,945][105692] Updated weights for policy 0, policy_version 1883351 (0.0009) [2023-12-27 05:07:02,004][105692] Updated weights for policy 0, policy_version 1883361 (0.0009) [2023-12-27 05:07:02,055][105692] Updated weights for policy 0, policy_version 1883371 (0.0009) [2023-12-27 05:07:02,087][105620] Updated weights for policy 1, policy_version 1887825 (0.0009) [2023-12-27 05:07:02,138][105620] Updated weights for policy 1, policy_version 1887835 (0.0007) [2023-12-27 05:07:02,192][105620] Updated weights for policy 1, policy_version 1887845 (0.0007) [2023-12-27 05:07:02,252][105620] Updated weights for policy 1, policy_version 1887855 (0.0006) [2023-12-27 05:07:02,804][105692] Updated weights for policy 0, policy_version 1883381 (0.0010) [2023-12-27 05:07:02,866][105692] Updated weights for policy 0, policy_version 1883391 (0.0009) [2023-12-27 05:07:02,924][105620] Updated weights for policy 1, policy_version 1887865 (0.0008) [2023-12-27 05:07:02,926][105692] Updated weights for policy 0, policy_version 1883401 (0.0007) [2023-12-27 05:07:02,993][105620] Updated weights for policy 1, policy_version 1887875 (0.0005) [2023-12-27 05:07:03,064][105620] Updated weights for policy 1, policy_version 1887885 (0.0005) [2023-12-27 05:07:03,529][105692] Updated weights for policy 0, policy_version 1883411 (0.0008) [2023-12-27 05:07:03,578][105692] Updated weights for policy 0, policy_version 1883421 (0.0006) [2023-12-27 05:07:03,628][105692] Updated weights for policy 0, policy_version 1883431 (0.0005) [2023-12-27 05:07:03,662][105620] Updated weights for policy 1, policy_version 1887895 (0.0007) [2023-12-27 05:07:03,708][105620] Updated weights for policy 1, policy_version 1887905 (0.0005) [2023-12-27 05:07:03,758][105620] Updated weights for policy 1, policy_version 1887915 (0.0005) [2023-12-27 05:07:04,274][105692] Updated weights for policy 0, policy_version 1883441 (0.0006) [2023-12-27 05:07:04,338][105692] Updated weights for policy 0, policy_version 1883451 (0.0011) [2023-12-27 05:07:04,396][105692] Updated weights for policy 0, policy_version 1883461 (0.0010) [2023-12-27 05:07:04,447][105692] Updated weights for policy 0, policy_version 1883471 (0.0010) [2023-12-27 05:07:04,461][105620] Updated weights for policy 1, policy_version 1887925 (0.0006) [2023-12-27 05:07:04,520][105620] Updated weights for policy 1, policy_version 1887935 (0.0008) [2023-12-27 05:07:04,569][105620] Updated weights for policy 1, policy_version 1887945 (0.0008) [2023-12-27 05:07:05,193][105692] Updated weights for policy 0, policy_version 1883481 (0.0008) [2023-12-27 05:07:05,236][105692] Updated weights for policy 0, policy_version 1883491 (0.0005) [2023-12-27 05:07:05,293][105692] Updated weights for policy 0, policy_version 1883501 (0.0005) [2023-12-27 05:07:05,356][105620] Updated weights for policy 1, policy_version 1887955 (0.0007) [2023-12-27 05:07:05,423][105620] Updated weights for policy 1, policy_version 1887965 (0.0007) [2023-12-27 05:07:05,476][105620] Updated weights for policy 1, policy_version 1887975 (0.0010) [2023-12-27 05:07:05,943][105692] Updated weights for policy 0, policy_version 1883511 (0.0009) [2023-12-27 05:07:05,994][105692] Updated weights for policy 0, policy_version 1883521 (0.0009) [2023-12-27 05:07:06,042][105692] Updated weights for policy 0, policy_version 1883531 (0.0009) [2023-12-27 05:07:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 965640192. Throughput: 0: 9697.5, 1: 9855.3. Samples: 965633884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:07:06,063][104569] Avg episode reward: [(0, '7989.824'), (1, '9253.095')] [2023-12-27 05:07:06,208][105620] Updated weights for policy 1, policy_version 1887985 (0.0009) [2023-12-27 05:07:06,270][105620] Updated weights for policy 1, policy_version 1887995 (0.0009) [2023-12-27 05:07:06,335][105620] Updated weights for policy 1, policy_version 1888005 (0.0008) [2023-12-27 05:07:06,390][105620] Updated weights for policy 1, policy_version 1888015 (0.0008) [2023-12-27 05:07:06,795][105692] Updated weights for policy 0, policy_version 1883541 (0.0009) [2023-12-27 05:07:06,856][105692] Updated weights for policy 0, policy_version 1883551 (0.0009) [2023-12-27 05:07:06,919][105692] Updated weights for policy 0, policy_version 1883561 (0.0009) [2023-12-27 05:07:07,045][105620] Updated weights for policy 1, policy_version 1888025 (0.0006) [2023-12-27 05:07:07,100][105620] Updated weights for policy 1, policy_version 1888035 (0.0005) [2023-12-27 05:07:07,150][105620] Updated weights for policy 1, policy_version 1888045 (0.0007) [2023-12-27 05:07:07,681][105692] Updated weights for policy 0, policy_version 1883571 (0.0010) [2023-12-27 05:07:07,751][105692] Updated weights for policy 0, policy_version 1883581 (0.0009) [2023-12-27 05:07:07,753][105620] Updated weights for policy 1, policy_version 1888055 (0.0005) [2023-12-27 05:07:07,811][105620] Updated weights for policy 1, policy_version 1888065 (0.0005) [2023-12-27 05:07:07,811][105692] Updated weights for policy 0, policy_version 1883591 (0.0008) [2023-12-27 05:07:07,876][105620] Updated weights for policy 1, policy_version 1888075 (0.0006) [2023-12-27 05:07:08,425][105620] Updated weights for policy 1, policy_version 1888085 (0.0007) [2023-12-27 05:07:08,486][105620] Updated weights for policy 1, policy_version 1888095 (0.0009) [2023-12-27 05:07:08,507][105692] Updated weights for policy 0, policy_version 1883601 (0.0008) [2023-12-27 05:07:08,552][105620] Updated weights for policy 1, policy_version 1888105 (0.0007) [2023-12-27 05:07:08,564][105692] Updated weights for policy 0, policy_version 1883611 (0.0009) [2023-12-27 05:07:08,621][105692] Updated weights for policy 0, policy_version 1883621 (0.0008) [2023-12-27 05:07:08,676][105692] Updated weights for policy 0, policy_version 1883631 (0.0009) [2023-12-27 05:07:09,217][105620] Updated weights for policy 1, policy_version 1888115 (0.0008) [2023-12-27 05:07:09,280][105620] Updated weights for policy 1, policy_version 1888125 (0.0009) [2023-12-27 05:07:09,347][105620] Updated weights for policy 1, policy_version 1888135 (0.0009) [2023-12-27 05:07:09,447][105692] Updated weights for policy 0, policy_version 1883641 (0.0007) [2023-12-27 05:07:09,505][105692] Updated weights for policy 0, policy_version 1883651 (0.0008) [2023-12-27 05:07:09,557][105692] Updated weights for policy 0, policy_version 1883661 (0.0009) [2023-12-27 05:07:10,135][105620] Updated weights for policy 1, policy_version 1888145 (0.0008) [2023-12-27 05:07:10,206][105620] Updated weights for policy 1, policy_version 1888155 (0.0008) [2023-12-27 05:07:10,258][105620] Updated weights for policy 1, policy_version 1888165 (0.0009) [2023-12-27 05:07:10,312][105692] Updated weights for policy 0, policy_version 1883671 (0.0007) [2023-12-27 05:07:10,321][105620] Updated weights for policy 1, policy_version 1888175 (0.0007) [2023-12-27 05:07:10,373][105692] Updated weights for policy 0, policy_version 1883681 (0.0009) [2023-12-27 05:07:10,426][105692] Updated weights for policy 0, policy_version 1883691 (0.0011) [2023-12-27 05:07:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 965738496. Throughput: 0: 9707.1, 1: 9858.4. Samples: 965751276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:07:11,062][104569] Avg episode reward: [(0, '7806.016'), (1, '9253.085')] [2023-12-27 05:07:11,124][105620] Updated weights for policy 1, policy_version 1888185 (0.0008) [2023-12-27 05:07:11,178][105692] Updated weights for policy 0, policy_version 1883701 (0.0010) [2023-12-27 05:07:11,193][105620] Updated weights for policy 1, policy_version 1888195 (0.0008) [2023-12-27 05:07:11,240][105692] Updated weights for policy 0, policy_version 1883711 (0.0009) [2023-12-27 05:07:11,254][105620] Updated weights for policy 1, policy_version 1888205 (0.0006) [2023-12-27 05:07:11,303][105692] Updated weights for policy 0, policy_version 1883721 (0.0007) [2023-12-27 05:07:12,052][105620] Updated weights for policy 1, policy_version 1888215 (0.0008) [2023-12-27 05:07:12,057][105692] Updated weights for policy 0, policy_version 1883731 (0.0008) [2023-12-27 05:07:12,106][105620] Updated weights for policy 1, policy_version 1888225 (0.0006) [2023-12-27 05:07:12,112][105692] Updated weights for policy 0, policy_version 1883741 (0.0007) [2023-12-27 05:07:12,156][105620] Updated weights for policy 1, policy_version 1888235 (0.0007) [2023-12-27 05:07:12,170][105692] Updated weights for policy 0, policy_version 1883751 (0.0006) [2023-12-27 05:07:12,864][105620] Updated weights for policy 1, policy_version 1888245 (0.0007) [2023-12-27 05:07:12,922][105620] Updated weights for policy 1, policy_version 1888255 (0.0009) [2023-12-27 05:07:12,987][105692] Updated weights for policy 0, policy_version 1883761 (0.0009) [2023-12-27 05:07:12,991][105620] Updated weights for policy 1, policy_version 1888265 (0.0010) [2023-12-27 05:07:13,041][105692] Updated weights for policy 0, policy_version 1883771 (0.0005) [2023-12-27 05:07:13,101][105692] Updated weights for policy 0, policy_version 1883781 (0.0005) [2023-12-27 05:07:13,154][105692] Updated weights for policy 0, policy_version 1883791 (0.0005) [2023-12-27 05:07:13,615][105620] Updated weights for policy 1, policy_version 1888275 (0.0008) [2023-12-27 05:07:13,676][105620] Updated weights for policy 1, policy_version 1888285 (0.0008) [2023-12-27 05:07:13,735][105620] Updated weights for policy 1, policy_version 1888295 (0.0009) [2023-12-27 05:07:13,774][105692] Updated weights for policy 0, policy_version 1883801 (0.0005) [2023-12-27 05:07:13,824][105692] Updated weights for policy 0, policy_version 1883811 (0.0005) [2023-12-27 05:07:13,872][105692] Updated weights for policy 0, policy_version 1883821 (0.0005) [2023-12-27 05:07:14,471][105692] Updated weights for policy 0, policy_version 1883831 (0.0007) [2023-12-27 05:07:14,531][105692] Updated weights for policy 0, policy_version 1883841 (0.0009) [2023-12-27 05:07:14,560][105620] Updated weights for policy 1, policy_version 1888305 (0.0008) [2023-12-27 05:07:14,590][105692] Updated weights for policy 0, policy_version 1883851 (0.0008) [2023-12-27 05:07:14,617][105620] Updated weights for policy 1, policy_version 1888315 (0.0007) [2023-12-27 05:07:14,668][105620] Updated weights for policy 1, policy_version 1888325 (0.0009) [2023-12-27 05:07:14,731][105620] Updated weights for policy 1, policy_version 1888335 (0.0010) [2023-12-27 05:07:15,323][105692] Updated weights for policy 0, policy_version 1883861 (0.0007) [2023-12-27 05:07:15,371][105692] Updated weights for policy 0, policy_version 1883871 (0.0008) [2023-12-27 05:07:15,434][105692] Updated weights for policy 0, policy_version 1883881 (0.0006) [2023-12-27 05:07:15,453][105620] Updated weights for policy 1, policy_version 1888345 (0.0009) [2023-12-27 05:07:15,505][105620] Updated weights for policy 1, policy_version 1888355 (0.0009) [2023-12-27 05:07:15,558][105620] Updated weights for policy 1, policy_version 1888365 (0.0010) [2023-12-27 05:07:16,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 965836800. Throughput: 0: 9572.5, 1: 9795.6. Samples: 965807784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:07:16,062][104569] Avg episode reward: [(0, '7991.049'), (1, '9345.259')] [2023-12-27 05:07:16,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001883888_482344960.pth... [2023-12-27 05:07:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001888368_483491840.pth... [2023-12-27 05:07:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001887216_483196928.pth [2023-12-27 05:07:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001882800_482066432.pth [2023-12-27 05:07:16,150][105692] Updated weights for policy 0, policy_version 1883891 (0.0007) [2023-12-27 05:07:16,213][105692] Updated weights for policy 0, policy_version 1883901 (0.0009) [2023-12-27 05:07:16,273][105692] Updated weights for policy 0, policy_version 1883911 (0.0009) [2023-12-27 05:07:16,279][105620] Updated weights for policy 1, policy_version 1888375 (0.0008) [2023-12-27 05:07:16,324][105620] Updated weights for policy 1, policy_version 1888385 (0.0006) [2023-12-27 05:07:16,385][105620] Updated weights for policy 1, policy_version 1888395 (0.0009) [2023-12-27 05:07:16,937][105692] Updated weights for policy 0, policy_version 1883921 (0.0008) [2023-12-27 05:07:16,989][105692] Updated weights for policy 0, policy_version 1883931 (0.0007) [2023-12-27 05:07:17,056][105692] Updated weights for policy 0, policy_version 1883941 (0.0009) [2023-12-27 05:07:17,116][105692] Updated weights for policy 0, policy_version 1883951 (0.0009) [2023-12-27 05:07:17,118][105620] Updated weights for policy 1, policy_version 1888405 (0.0008) [2023-12-27 05:07:17,178][105620] Updated weights for policy 1, policy_version 1888415 (0.0005) [2023-12-27 05:07:17,239][105620] Updated weights for policy 1, policy_version 1888425 (0.0005) [2023-12-27 05:07:17,781][105620] Updated weights for policy 1, policy_version 1888435 (0.0006) [2023-12-27 05:07:17,827][105620] Updated weights for policy 1, policy_version 1888445 (0.0009) [2023-12-27 05:07:17,875][105620] Updated weights for policy 1, policy_version 1888455 (0.0010) [2023-12-27 05:07:17,939][105692] Updated weights for policy 0, policy_version 1883961 (0.0007) [2023-12-27 05:07:17,991][105692] Updated weights for policy 0, policy_version 1883971 (0.0008) [2023-12-27 05:07:18,047][105692] Updated weights for policy 0, policy_version 1883981 (0.0008) [2023-12-27 05:07:18,671][105620] Updated weights for policy 1, policy_version 1888466 (0.0011) [2023-12-27 05:07:18,738][105620] Updated weights for policy 1, policy_version 1888476 (0.0009) [2023-12-27 05:07:18,803][105620] Updated weights for policy 1, policy_version 1888486 (0.0008) [2023-12-27 05:07:18,826][105692] Updated weights for policy 0, policy_version 1883991 (0.0008) [2023-12-27 05:07:18,863][105620] Updated weights for policy 1, policy_version 1888496 (0.0008) [2023-12-27 05:07:18,881][105692] Updated weights for policy 0, policy_version 1884001 (0.0008) [2023-12-27 05:07:18,937][105692] Updated weights for policy 0, policy_version 1884011 (0.0009) [2023-12-27 05:07:19,601][105692] Updated weights for policy 0, policy_version 1884021 (0.0010) [2023-12-27 05:07:19,641][105620] Updated weights for policy 1, policy_version 1888506 (0.0006) [2023-12-27 05:07:19,661][105692] Updated weights for policy 0, policy_version 1884031 (0.0010) [2023-12-27 05:07:19,701][105620] Updated weights for policy 1, policy_version 1888516 (0.0006) [2023-12-27 05:07:19,719][105692] Updated weights for policy 0, policy_version 1884041 (0.0011) [2023-12-27 05:07:19,761][105620] Updated weights for policy 1, policy_version 1888526 (0.0006) [2023-12-27 05:07:20,486][105692] Updated weights for policy 0, policy_version 1884051 (0.0010) [2023-12-27 05:07:20,504][105620] Updated weights for policy 1, policy_version 1888536 (0.0007) [2023-12-27 05:07:20,540][105692] Updated weights for policy 0, policy_version 1884061 (0.0006) [2023-12-27 05:07:20,572][105620] Updated weights for policy 1, policy_version 1888546 (0.0008) [2023-12-27 05:07:20,609][105692] Updated weights for policy 0, policy_version 1884071 (0.0008) [2023-12-27 05:07:20,645][105620] Updated weights for policy 1, policy_version 1888556 (0.0009) [2023-12-27 05:07:21,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 965935104. Throughput: 0: 9497.4, 1: 9840.0. Samples: 965924596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:07:21,063][104569] Avg episode reward: [(0, '8081.975'), (1, '9345.245')] [2023-12-27 05:07:21,343][105692] Updated weights for policy 0, policy_version 1884081 (0.0009) [2023-12-27 05:07:21,416][105692] Updated weights for policy 0, policy_version 1884091 (0.0009) [2023-12-27 05:07:21,427][105620] Updated weights for policy 1, policy_version 1888566 (0.0007) [2023-12-27 05:07:21,472][105692] Updated weights for policy 0, policy_version 1884101 (0.0008) [2023-12-27 05:07:21,477][105620] Updated weights for policy 1, policy_version 1888576 (0.0006) [2023-12-27 05:07:21,524][105692] Updated weights for policy 0, policy_version 1884111 (0.0007) [2023-12-27 05:07:21,528][105620] Updated weights for policy 1, policy_version 1888586 (0.0007) [2023-12-27 05:07:22,265][105692] Updated weights for policy 0, policy_version 1884121 (0.0006) [2023-12-27 05:07:22,307][105620] Updated weights for policy 1, policy_version 1888596 (0.0006) [2023-12-27 05:07:22,330][105692] Updated weights for policy 0, policy_version 1884131 (0.0009) [2023-12-27 05:07:22,373][105620] Updated weights for policy 1, policy_version 1888606 (0.0006) [2023-12-27 05:07:22,395][105692] Updated weights for policy 0, policy_version 1884141 (0.0009) [2023-12-27 05:07:22,439][105620] Updated weights for policy 1, policy_version 1888616 (0.0008) [2023-12-27 05:07:23,177][105692] Updated weights for policy 0, policy_version 1884151 (0.0009) [2023-12-27 05:07:23,186][105620] Updated weights for policy 1, policy_version 1888626 (0.0009) [2023-12-27 05:07:23,235][105692] Updated weights for policy 0, policy_version 1884161 (0.0007) [2023-12-27 05:07:23,245][105620] Updated weights for policy 1, policy_version 1888636 (0.0008) [2023-12-27 05:07:23,294][105692] Updated weights for policy 0, policy_version 1884171 (0.0005) [2023-12-27 05:07:23,309][105620] Updated weights for policy 1, policy_version 1888646 (0.0009) [2023-12-27 05:07:23,375][105620] Updated weights for policy 1, policy_version 1888656 (0.0010) [2023-12-27 05:07:23,975][105692] Updated weights for policy 0, policy_version 1884181 (0.0007) [2023-12-27 05:07:24,034][105692] Updated weights for policy 0, policy_version 1884191 (0.0007) [2023-12-27 05:07:24,098][105692] Updated weights for policy 0, policy_version 1884201 (0.0008) [2023-12-27 05:07:24,140][105620] Updated weights for policy 1, policy_version 1888666 (0.0006) [2023-12-27 05:07:24,197][105620] Updated weights for policy 1, policy_version 1888676 (0.0009) [2023-12-27 05:07:24,244][105620] Updated weights for policy 1, policy_version 1888686 (0.0009) [2023-12-27 05:07:24,838][105692] Updated weights for policy 0, policy_version 1884211 (0.0009) [2023-12-27 05:07:24,889][105692] Updated weights for policy 0, policy_version 1884221 (0.0009) [2023-12-27 05:07:24,936][105692] Updated weights for policy 0, policy_version 1884231 (0.0009) [2023-12-27 05:07:25,001][105620] Updated weights for policy 1, policy_version 1888696 (0.0008) [2023-12-27 05:07:25,048][105620] Updated weights for policy 1, policy_version 1888706 (0.0008) [2023-12-27 05:07:25,101][105620] Updated weights for policy 1, policy_version 1888716 (0.0008) [2023-12-27 05:07:25,713][105692] Updated weights for policy 0, policy_version 1884241 (0.0007) [2023-12-27 05:07:25,781][105692] Updated weights for policy 0, policy_version 1884251 (0.0009) [2023-12-27 05:07:25,841][105692] Updated weights for policy 0, policy_version 1884261 (0.0008) [2023-12-27 05:07:25,854][105620] Updated weights for policy 1, policy_version 1888726 (0.0008) [2023-12-27 05:07:25,895][105692] Updated weights for policy 0, policy_version 1884271 (0.0008) [2023-12-27 05:07:25,913][105620] Updated weights for policy 1, policy_version 1888736 (0.0007) [2023-12-27 05:07:25,972][105620] Updated weights for policy 1, policy_version 1888746 (0.0009) [2023-12-27 05:07:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 966033408. Throughput: 0: 9495.2, 1: 9805.4. Samples: 966035872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:07:26,062][104569] Avg episode reward: [(0, '8076.332'), (1, '9253.019')] [2023-12-27 05:07:26,647][105692] Updated weights for policy 0, policy_version 1884281 (0.0008) [2023-12-27 05:07:26,682][105620] Updated weights for policy 1, policy_version 1888756 (0.0008) [2023-12-27 05:07:26,693][105692] Updated weights for policy 0, policy_version 1884291 (0.0006) [2023-12-27 05:07:26,731][105620] Updated weights for policy 1, policy_version 1888766 (0.0007) [2023-12-27 05:07:26,737][105692] Updated weights for policy 0, policy_version 1884301 (0.0006) [2023-12-27 05:07:26,785][105620] Updated weights for policy 1, policy_version 1888776 (0.0008) [2023-12-27 05:07:27,440][105620] Updated weights for policy 1, policy_version 1888786 (0.0009) [2023-12-27 05:07:27,496][105620] Updated weights for policy 1, policy_version 1888796 (0.0008) [2023-12-27 05:07:27,535][105692] Updated weights for policy 0, policy_version 1884311 (0.0005) [2023-12-27 05:07:27,552][105620] Updated weights for policy 1, policy_version 1888806 (0.0008) [2023-12-27 05:07:27,598][105692] Updated weights for policy 0, policy_version 1884321 (0.0005) [2023-12-27 05:07:27,607][105620] Updated weights for policy 1, policy_version 1888816 (0.0009) [2023-12-27 05:07:27,657][105692] Updated weights for policy 0, policy_version 1884331 (0.0005) [2023-12-27 05:07:28,342][105692] Updated weights for policy 0, policy_version 1884341 (0.0007) [2023-12-27 05:07:28,372][105620] Updated weights for policy 1, policy_version 1888826 (0.0007) [2023-12-27 05:07:28,402][105692] Updated weights for policy 0, policy_version 1884351 (0.0007) [2023-12-27 05:07:28,428][105620] Updated weights for policy 1, policy_version 1888836 (0.0006) [2023-12-27 05:07:28,455][105692] Updated weights for policy 0, policy_version 1884361 (0.0007) [2023-12-27 05:07:28,474][105620] Updated weights for policy 1, policy_version 1888846 (0.0006) [2023-12-27 05:07:29,215][105692] Updated weights for policy 0, policy_version 1884371 (0.0007) [2023-12-27 05:07:29,252][105620] Updated weights for policy 1, policy_version 1888856 (0.0008) [2023-12-27 05:07:29,275][105692] Updated weights for policy 0, policy_version 1884381 (0.0007) [2023-12-27 05:07:29,307][105620] Updated weights for policy 1, policy_version 1888866 (0.0007) [2023-12-27 05:07:29,331][105692] Updated weights for policy 0, policy_version 1884391 (0.0008) [2023-12-27 05:07:29,382][105620] Updated weights for policy 1, policy_version 1888876 (0.0007) [2023-12-27 05:07:30,035][105692] Updated weights for policy 0, policy_version 1884401 (0.0008) [2023-12-27 05:07:30,084][105692] Updated weights for policy 0, policy_version 1884411 (0.0010) [2023-12-27 05:07:30,142][105692] Updated weights for policy 0, policy_version 1884421 (0.0009) [2023-12-27 05:07:30,146][105620] Updated weights for policy 1, policy_version 1888886 (0.0009) [2023-12-27 05:07:30,194][105692] Updated weights for policy 0, policy_version 1884431 (0.0011) [2023-12-27 05:07:30,204][105620] Updated weights for policy 1, policy_version 1888896 (0.0006) [2023-12-27 05:07:30,254][105620] Updated weights for policy 1, policy_version 1888906 (0.0008) [2023-12-27 05:07:30,930][105692] Updated weights for policy 0, policy_version 1884441 (0.0010) [2023-12-27 05:07:30,991][105692] Updated weights for policy 0, policy_version 1884451 (0.0010) [2023-12-27 05:07:31,013][105620] Updated weights for policy 1, policy_version 1888916 (0.0007) [2023-12-27 05:07:31,047][105692] Updated weights for policy 0, policy_version 1884461 (0.0010) [2023-12-27 05:07:31,062][104569] Fps is (10 sec: 18022.6, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 966115328. Throughput: 0: 9494.2, 1: 9822.1. Samples: 966093116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:07:31,063][104569] Avg episode reward: [(0, '8350.699'), (1, '9069.218')] [2023-12-27 05:07:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001884464_482492416.pth... [2023-12-27 05:07:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001883344_482205696.pth [2023-12-27 05:07:31,077][105620] Updated weights for policy 1, policy_version 1888926 (0.0007) [2023-12-27 05:07:31,140][105620] Updated weights for policy 1, policy_version 1888936 (0.0008) [2023-12-27 05:07:31,191][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001888944_483639296.pth... [2023-12-27 05:07:31,196][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001887792_483344384.pth [2023-12-27 05:07:31,826][105692] Updated weights for policy 0, policy_version 1884471 (0.0009) [2023-12-27 05:07:31,882][105692] Updated weights for policy 0, policy_version 1884481 (0.0009) [2023-12-27 05:07:31,890][105620] Updated weights for policy 1, policy_version 1888946 (0.0009) [2023-12-27 05:07:31,946][105692] Updated weights for policy 0, policy_version 1884491 (0.0009) [2023-12-27 05:07:31,951][105620] Updated weights for policy 1, policy_version 1888956 (0.0006) [2023-12-27 05:07:32,012][105620] Updated weights for policy 1, policy_version 1888966 (0.0005) [2023-12-27 05:07:32,074][105620] Updated weights for policy 1, policy_version 1888976 (0.0006) [2023-12-27 05:07:32,621][105620] Updated weights for policy 1, policy_version 1888986 (0.0005) [2023-12-27 05:07:32,667][105620] Updated weights for policy 1, policy_version 1888996 (0.0005) [2023-12-27 05:07:32,720][105620] Updated weights for policy 1, policy_version 1889006 (0.0006) [2023-12-27 05:07:32,807][105692] Updated weights for policy 0, policy_version 1884501 (0.0007) [2023-12-27 05:07:32,864][105692] Updated weights for policy 0, policy_version 1884511 (0.0006) [2023-12-27 05:07:32,928][105692] Updated weights for policy 0, policy_version 1884521 (0.0008) [2023-12-27 05:07:33,359][105620] Updated weights for policy 1, policy_version 1889016 (0.0008) [2023-12-27 05:07:33,407][105620] Updated weights for policy 1, policy_version 1889027 (0.0009) [2023-12-27 05:07:33,451][105620] Updated weights for policy 1, policy_version 1889037 (0.0007) [2023-12-27 05:07:33,574][105692] Updated weights for policy 0, policy_version 1884531 (0.0009) [2023-12-27 05:07:33,625][105692] Updated weights for policy 0, policy_version 1884541 (0.0009) [2023-12-27 05:07:33,680][105692] Updated weights for policy 0, policy_version 1884552 (0.0010) [2023-12-27 05:07:34,042][105620] Updated weights for policy 1, policy_version 1889047 (0.0005) [2023-12-27 05:07:34,101][105620] Updated weights for policy 1, policy_version 1889057 (0.0008) [2023-12-27 05:07:34,149][105620] Updated weights for policy 1, policy_version 1889067 (0.0010) [2023-12-27 05:07:34,573][105692] Updated weights for policy 0, policy_version 1884563 (0.0009) [2023-12-27 05:07:34,638][105692] Updated weights for policy 0, policy_version 1884573 (0.0009) [2023-12-27 05:07:34,705][105692] Updated weights for policy 0, policy_version 1884583 (0.0008) [2023-12-27 05:07:34,759][105620] Updated weights for policy 1, policy_version 1889077 (0.0010) [2023-12-27 05:07:34,806][105620] Updated weights for policy 1, policy_version 1889087 (0.0009) [2023-12-27 05:07:34,863][105620] Updated weights for policy 1, policy_version 1889097 (0.0008) [2023-12-27 05:07:35,410][105692] Updated weights for policy 0, policy_version 1884593 (0.0008) [2023-12-27 05:07:35,461][105692] Updated weights for policy 0, policy_version 1884603 (0.0005) [2023-12-27 05:07:35,517][105692] Updated weights for policy 0, policy_version 1884613 (0.0005) [2023-12-27 05:07:35,561][105620] Updated weights for policy 1, policy_version 1889107 (0.0008) [2023-12-27 05:07:35,566][105692] Updated weights for policy 0, policy_version 1884623 (0.0008) [2023-12-27 05:07:35,619][105620] Updated weights for policy 1, policy_version 1889117 (0.0010) [2023-12-27 05:07:35,677][105620] Updated weights for policy 1, policy_version 1889127 (0.0010) [2023-12-27 05:07:36,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 966221824. Throughput: 0: 9427.1, 1: 9860.0. Samples: 966210312. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:07:36,063][104569] Avg episode reward: [(0, '8531.291'), (1, '9069.307')] [2023-12-27 05:07:36,167][105692] Updated weights for policy 0, policy_version 1884633 (0.0007) [2023-12-27 05:07:36,237][105692] Updated weights for policy 0, policy_version 1884643 (0.0006) [2023-12-27 05:07:36,306][105692] Updated weights for policy 0, policy_version 1884653 (0.0008) [2023-12-27 05:07:36,325][105620] Updated weights for policy 1, policy_version 1889137 (0.0009) [2023-12-27 05:07:36,396][105620] Updated weights for policy 1, policy_version 1889147 (0.0007) [2023-12-27 05:07:36,462][105620] Updated weights for policy 1, policy_version 1889157 (0.0009) [2023-12-27 05:07:36,521][105620] Updated weights for policy 1, policy_version 1889167 (0.0011) [2023-12-27 05:07:37,010][105692] Updated weights for policy 0, policy_version 1884663 (0.0009) [2023-12-27 05:07:37,069][105692] Updated weights for policy 0, policy_version 1884673 (0.0009) [2023-12-27 05:07:37,130][105620] Updated weights for policy 1, policy_version 1889177 (0.0011) [2023-12-27 05:07:37,132][105692] Updated weights for policy 0, policy_version 1884683 (0.0006) [2023-12-27 05:07:37,190][105620] Updated weights for policy 1, policy_version 1889187 (0.0011) [2023-12-27 05:07:37,239][105620] Updated weights for policy 1, policy_version 1889197 (0.0011) [2023-12-27 05:07:37,781][105692] Updated weights for policy 0, policy_version 1884693 (0.0007) [2023-12-27 05:07:37,843][105692] Updated weights for policy 0, policy_version 1884703 (0.0009) [2023-12-27 05:07:37,897][105692] Updated weights for policy 0, policy_version 1884713 (0.0008) [2023-12-27 05:07:37,986][105620] Updated weights for policy 1, policy_version 1889207 (0.0009) [2023-12-27 05:07:38,043][105620] Updated weights for policy 1, policy_version 1889217 (0.0009) [2023-12-27 05:07:38,101][105620] Updated weights for policy 1, policy_version 1889227 (0.0009) [2023-12-27 05:07:38,623][105692] Updated weights for policy 0, policy_version 1884723 (0.0008) [2023-12-27 05:07:38,677][105692] Updated weights for policy 0, policy_version 1884733 (0.0009) [2023-12-27 05:07:38,727][105692] Updated weights for policy 0, policy_version 1884743 (0.0008) [2023-12-27 05:07:38,880][105620] Updated weights for policy 1, policy_version 1889237 (0.0010) [2023-12-27 05:07:38,925][105620] Updated weights for policy 1, policy_version 1889247 (0.0010) [2023-12-27 05:07:38,973][105620] Updated weights for policy 1, policy_version 1889257 (0.0010) [2023-12-27 05:07:39,504][105692] Updated weights for policy 0, policy_version 1884753 (0.0009) [2023-12-27 05:07:39,572][105692] Updated weights for policy 0, policy_version 1884763 (0.0009) [2023-12-27 05:07:39,631][105692] Updated weights for policy 0, policy_version 1884773 (0.0005) [2023-12-27 05:07:39,696][105692] Updated weights for policy 0, policy_version 1884783 (0.0007) [2023-12-27 05:07:39,763][105620] Updated weights for policy 1, policy_version 1889267 (0.0009) [2023-12-27 05:07:39,835][105620] Updated weights for policy 1, policy_version 1889277 (0.0008) [2023-12-27 05:07:39,899][105620] Updated weights for policy 1, policy_version 1889287 (0.0011) [2023-12-27 05:07:40,447][105692] Updated weights for policy 0, policy_version 1884793 (0.0009) [2023-12-27 05:07:40,497][105692] Updated weights for policy 0, policy_version 1884803 (0.0009) [2023-12-27 05:07:40,556][105692] Updated weights for policy 0, policy_version 1884813 (0.0007) [2023-12-27 05:07:40,616][105620] Updated weights for policy 1, policy_version 1889297 (0.0010) [2023-12-27 05:07:40,679][105620] Updated weights for policy 1, policy_version 1889307 (0.0009) [2023-12-27 05:07:40,723][105620] Updated weights for policy 1, policy_version 1889317 (0.0009) [2023-12-27 05:07:40,772][105620] Updated weights for policy 1, policy_version 1889327 (0.0010) [2023-12-27 05:07:41,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 966320128. Throughput: 0: 9453.3, 1: 9845.3. Samples: 966328048. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:07:41,062][104569] Avg episode reward: [(0, '8440.883'), (1, '9068.505')] [2023-12-27 05:07:41,364][105692] Updated weights for policy 0, policy_version 1884823 (0.0007) [2023-12-27 05:07:41,428][105692] Updated weights for policy 0, policy_version 1884833 (0.0009) [2023-12-27 05:07:41,483][105692] Updated weights for policy 0, policy_version 1884843 (0.0008) [2023-12-27 05:07:41,503][105620] Updated weights for policy 1, policy_version 1889337 (0.0006) [2023-12-27 05:07:41,555][105620] Updated weights for policy 1, policy_version 1889347 (0.0006) [2023-12-27 05:07:41,608][105620] Updated weights for policy 1, policy_version 1889357 (0.0007) [2023-12-27 05:07:42,261][105692] Updated weights for policy 0, policy_version 1884853 (0.0009) [2023-12-27 05:07:42,325][105692] Updated weights for policy 0, policy_version 1884863 (0.0009) [2023-12-27 05:07:42,363][105620] Updated weights for policy 1, policy_version 1889367 (0.0011) [2023-12-27 05:07:42,391][105692] Updated weights for policy 0, policy_version 1884873 (0.0009) [2023-12-27 05:07:42,437][105620] Updated weights for policy 1, policy_version 1889377 (0.0009) [2023-12-27 05:07:42,494][105620] Updated weights for policy 1, policy_version 1889387 (0.0006) [2023-12-27 05:07:43,045][105620] Updated weights for policy 1, policy_version 1889397 (0.0006) [2023-12-27 05:07:43,111][105620] Updated weights for policy 1, policy_version 1889407 (0.0008) [2023-12-27 05:07:43,171][105620] Updated weights for policy 1, policy_version 1889417 (0.0008) [2023-12-27 05:07:43,292][105692] Updated weights for policy 0, policy_version 1884883 (0.0008) [2023-12-27 05:07:43,341][105692] Updated weights for policy 0, policy_version 1884893 (0.0009) [2023-12-27 05:07:43,393][105692] Updated weights for policy 0, policy_version 1884903 (0.0005) [2023-12-27 05:07:43,971][105692] Updated weights for policy 0, policy_version 1884913 (0.0005) [2023-12-27 05:07:43,971][105620] Updated weights for policy 1, policy_version 1889427 (0.0009) [2023-12-27 05:07:44,020][105620] Updated weights for policy 1, policy_version 1889437 (0.0009) [2023-12-27 05:07:44,024][105692] Updated weights for policy 0, policy_version 1884923 (0.0005) [2023-12-27 05:07:44,074][105620] Updated weights for policy 1, policy_version 1889447 (0.0008) [2023-12-27 05:07:44,076][105692] Updated weights for policy 0, policy_version 1884933 (0.0006) [2023-12-27 05:07:44,127][105692] Updated weights for policy 0, policy_version 1884943 (0.0009) [2023-12-27 05:07:44,800][105620] Updated weights for policy 1, policy_version 1889457 (0.0007) [2023-12-27 05:07:44,803][105692] Updated weights for policy 0, policy_version 1884953 (0.0008) [2023-12-27 05:07:44,856][105620] Updated weights for policy 1, policy_version 1889467 (0.0007) [2023-12-27 05:07:44,866][105692] Updated weights for policy 0, policy_version 1884963 (0.0007) [2023-12-27 05:07:44,917][105620] Updated weights for policy 1, policy_version 1889477 (0.0006) [2023-12-27 05:07:44,927][105692] Updated weights for policy 0, policy_version 1884973 (0.0006) [2023-12-27 05:07:44,977][105620] Updated weights for policy 1, policy_version 1889487 (0.0008) [2023-12-27 05:07:45,677][105692] Updated weights for policy 0, policy_version 1884983 (0.0008) [2023-12-27 05:07:45,726][105692] Updated weights for policy 0, policy_version 1884993 (0.0009) [2023-12-27 05:07:45,730][105620] Updated weights for policy 1, policy_version 1889497 (0.0006) [2023-12-27 05:07:45,783][105692] Updated weights for policy 0, policy_version 1885003 (0.0009) [2023-12-27 05:07:45,789][105620] Updated weights for policy 1, policy_version 1889507 (0.0005) [2023-12-27 05:07:45,839][105620] Updated weights for policy 1, policy_version 1889517 (0.0006) [2023-12-27 05:07:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 966418432. Throughput: 0: 9471.8, 1: 9772.2. Samples: 966384280. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:07:46,063][104569] Avg episode reward: [(0, '7993.866'), (1, '9070.720')] [2023-12-27 05:07:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001885008_482631680.pth... [2023-12-27 05:07:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001889520_483786752.pth... [2023-12-27 05:07:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001883888_482344960.pth [2023-12-27 05:07:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001888368_483491840.pth [2023-12-27 05:07:46,454][105620] Updated weights for policy 1, policy_version 1889527 (0.0008) [2023-12-27 05:07:46,507][105620] Updated weights for policy 1, policy_version 1889537 (0.0008) [2023-12-27 05:07:46,549][105692] Updated weights for policy 0, policy_version 1885013 (0.0009) [2023-12-27 05:07:46,560][105620] Updated weights for policy 1, policy_version 1889547 (0.0007) [2023-12-27 05:07:46,604][105692] Updated weights for policy 0, policy_version 1885023 (0.0009) [2023-12-27 05:07:46,657][105692] Updated weights for policy 0, policy_version 1885033 (0.0006) [2023-12-27 05:07:47,169][105620] Updated weights for policy 1, policy_version 1889557 (0.0008) [2023-12-27 05:07:47,195][105692] Updated weights for policy 0, policy_version 1885043 (0.0005) [2023-12-27 05:07:47,225][105620] Updated weights for policy 1, policy_version 1889567 (0.0008) [2023-12-27 05:07:47,245][105692] Updated weights for policy 0, policy_version 1885053 (0.0006) [2023-12-27 05:07:47,277][105620] Updated weights for policy 1, policy_version 1889577 (0.0007) [2023-12-27 05:07:47,297][105692] Updated weights for policy 0, policy_version 1885063 (0.0006) [2023-12-27 05:07:47,987][105692] Updated weights for policy 0, policy_version 1885073 (0.0005) [2023-12-27 05:07:48,043][105692] Updated weights for policy 0, policy_version 1885083 (0.0008) [2023-12-27 05:07:48,044][105620] Updated weights for policy 1, policy_version 1889587 (0.0009) [2023-12-27 05:07:48,097][105692] Updated weights for policy 0, policy_version 1885093 (0.0008) [2023-12-27 05:07:48,103][105620] Updated weights for policy 1, policy_version 1889597 (0.0006) [2023-12-27 05:07:48,149][105692] Updated weights for policy 0, policy_version 1885103 (0.0007) [2023-12-27 05:07:48,163][105620] Updated weights for policy 1, policy_version 1889607 (0.0007) [2023-12-27 05:07:48,844][105620] Updated weights for policy 1, policy_version 1889617 (0.0007) [2023-12-27 05:07:48,905][105620] Updated weights for policy 1, policy_version 1889627 (0.0009) [2023-12-27 05:07:48,948][105692] Updated weights for policy 0, policy_version 1885113 (0.0007) [2023-12-27 05:07:48,951][105620] Updated weights for policy 1, policy_version 1889637 (0.0006) [2023-12-27 05:07:49,005][105620] Updated weights for policy 1, policy_version 1889647 (0.0007) [2023-12-27 05:07:49,012][105692] Updated weights for policy 0, policy_version 1885123 (0.0010) [2023-12-27 05:07:49,073][105692] Updated weights for policy 0, policy_version 1885133 (0.0008) [2023-12-27 05:07:49,775][105692] Updated weights for policy 0, policy_version 1885143 (0.0006) [2023-12-27 05:07:49,789][105620] Updated weights for policy 1, policy_version 1889657 (0.0007) [2023-12-27 05:07:49,828][105692] Updated weights for policy 0, policy_version 1885153 (0.0006) [2023-12-27 05:07:49,852][105620] Updated weights for policy 1, policy_version 1889667 (0.0008) [2023-12-27 05:07:49,894][105692] Updated weights for policy 0, policy_version 1885163 (0.0009) [2023-12-27 05:07:49,914][105620] Updated weights for policy 1, policy_version 1889677 (0.0008) [2023-12-27 05:07:50,656][105692] Updated weights for policy 0, policy_version 1885173 (0.0009) [2023-12-27 05:07:50,697][105620] Updated weights for policy 1, policy_version 1889687 (0.0006) [2023-12-27 05:07:50,726][105692] Updated weights for policy 0, policy_version 1885183 (0.0010) [2023-12-27 05:07:50,763][105620] Updated weights for policy 1, policy_version 1889697 (0.0006) [2023-12-27 05:07:50,794][105692] Updated weights for policy 0, policy_version 1885193 (0.0011) [2023-12-27 05:07:50,830][105620] Updated weights for policy 1, policy_version 1889707 (0.0010) [2023-12-27 05:07:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 966516736. Throughput: 0: 9568.3, 1: 9762.5. Samples: 966503768. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:07:51,063][104569] Avg episode reward: [(0, '7907.084'), (1, '9255.299')] [2023-12-27 05:07:51,526][105692] Updated weights for policy 0, policy_version 1885203 (0.0011) [2023-12-27 05:07:51,559][105620] Updated weights for policy 1, policy_version 1889717 (0.0008) [2023-12-27 05:07:51,578][105692] Updated weights for policy 0, policy_version 1885213 (0.0011) [2023-12-27 05:07:51,625][105620] Updated weights for policy 1, policy_version 1889727 (0.0010) [2023-12-27 05:07:51,641][105692] Updated weights for policy 0, policy_version 1885223 (0.0009) [2023-12-27 05:07:51,683][105620] Updated weights for policy 1, policy_version 1889737 (0.0011) [2023-12-27 05:07:52,358][105692] Updated weights for policy 0, policy_version 1885233 (0.0010) [2023-12-27 05:07:52,371][105620] Updated weights for policy 1, policy_version 1889747 (0.0010) [2023-12-27 05:07:52,418][105692] Updated weights for policy 0, policy_version 1885243 (0.0011) [2023-12-27 05:07:52,427][105620] Updated weights for policy 1, policy_version 1889757 (0.0010) [2023-12-27 05:07:52,476][105692] Updated weights for policy 0, policy_version 1885253 (0.0009) [2023-12-27 05:07:52,490][105620] Updated weights for policy 1, policy_version 1889767 (0.0011) [2023-12-27 05:07:52,536][105692] Updated weights for policy 0, policy_version 1885263 (0.0008) [2023-12-27 05:07:53,083][105692] Updated weights for policy 0, policy_version 1885273 (0.0010) [2023-12-27 05:07:53,144][105692] Updated weights for policy 0, policy_version 1885283 (0.0008) [2023-12-27 05:07:53,178][105620] Updated weights for policy 1, policy_version 1889777 (0.0010) [2023-12-27 05:07:53,208][105692] Updated weights for policy 0, policy_version 1885293 (0.0008) [2023-12-27 05:07:53,237][105620] Updated weights for policy 1, policy_version 1889787 (0.0010) [2023-12-27 05:07:53,289][105620] Updated weights for policy 1, policy_version 1889797 (0.0009) [2023-12-27 05:07:53,338][105620] Updated weights for policy 1, policy_version 1889807 (0.0008) [2023-12-27 05:07:53,922][105692] Updated weights for policy 0, policy_version 1885303 (0.0010) [2023-12-27 05:07:53,981][105692] Updated weights for policy 0, policy_version 1885313 (0.0010) [2023-12-27 05:07:54,008][105620] Updated weights for policy 1, policy_version 1889817 (0.0006) [2023-12-27 05:07:54,043][105692] Updated weights for policy 0, policy_version 1885323 (0.0010) [2023-12-27 05:07:54,060][105620] Updated weights for policy 1, policy_version 1889827 (0.0005) [2023-12-27 05:07:54,108][105620] Updated weights for policy 1, policy_version 1889837 (0.0006) [2023-12-27 05:07:54,650][105620] Updated weights for policy 1, policy_version 1889847 (0.0006) [2023-12-27 05:07:54,700][105620] Updated weights for policy 1, policy_version 1889857 (0.0005) [2023-12-27 05:07:54,732][105692] Updated weights for policy 0, policy_version 1885333 (0.0008) [2023-12-27 05:07:54,747][105620] Updated weights for policy 1, policy_version 1889867 (0.0005) [2023-12-27 05:07:54,781][105692] Updated weights for policy 0, policy_version 1885343 (0.0010) [2023-12-27 05:07:54,829][105692] Updated weights for policy 0, policy_version 1885353 (0.0010) [2023-12-27 05:07:55,371][105620] Updated weights for policy 1, policy_version 1889877 (0.0008) [2023-12-27 05:07:55,423][105620] Updated weights for policy 1, policy_version 1889887 (0.0010) [2023-12-27 05:07:55,440][105692] Updated weights for policy 0, policy_version 1885363 (0.0009) [2023-12-27 05:07:55,474][105620] Updated weights for policy 1, policy_version 1889897 (0.0010) [2023-12-27 05:07:55,497][105692] Updated weights for policy 0, policy_version 1885373 (0.0005) [2023-12-27 05:07:55,549][105692] Updated weights for policy 0, policy_version 1885383 (0.0005) [2023-12-27 05:07:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 966615040. Throughput: 0: 9635.8, 1: 9812.2. Samples: 966626436. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:07:56,062][104569] Avg episode reward: [(0, '7989.223'), (1, '9345.211')] [2023-12-27 05:07:56,119][105692] Updated weights for policy 0, policy_version 1885393 (0.0005) [2023-12-27 05:07:56,160][105620] Updated weights for policy 1, policy_version 1889907 (0.0011) [2023-12-27 05:07:56,169][105692] Updated weights for policy 0, policy_version 1885403 (0.0010) [2023-12-27 05:07:56,218][105620] Updated weights for policy 1, policy_version 1889917 (0.0010) [2023-12-27 05:07:56,228][105692] Updated weights for policy 0, policy_version 1885413 (0.0010) [2023-12-27 05:07:56,270][105620] Updated weights for policy 1, policy_version 1889927 (0.0010) [2023-12-27 05:07:56,289][105692] Updated weights for policy 0, policy_version 1885423 (0.0005) [2023-12-27 05:07:56,894][105692] Updated weights for policy 0, policy_version 1885433 (0.0007) [2023-12-27 05:07:56,909][105620] Updated weights for policy 1, policy_version 1889937 (0.0010) [2023-12-27 05:07:56,951][105692] Updated weights for policy 0, policy_version 1885443 (0.0006) [2023-12-27 05:07:56,957][105620] Updated weights for policy 1, policy_version 1889947 (0.0005) [2023-12-27 05:07:57,006][105692] Updated weights for policy 0, policy_version 1885453 (0.0008) [2023-12-27 05:07:57,017][105620] Updated weights for policy 1, policy_version 1889957 (0.0005) [2023-12-27 05:07:57,088][105620] Updated weights for policy 1, policy_version 1889967 (0.0005) [2023-12-27 05:07:57,556][105692] Updated weights for policy 0, policy_version 1885463 (0.0006) [2023-12-27 05:07:57,602][105692] Updated weights for policy 0, policy_version 1885473 (0.0007) [2023-12-27 05:07:57,649][105692] Updated weights for policy 0, policy_version 1885483 (0.0008) [2023-12-27 05:07:57,682][105620] Updated weights for policy 1, policy_version 1889977 (0.0010) [2023-12-27 05:07:57,739][105620] Updated weights for policy 1, policy_version 1889987 (0.0010) [2023-12-27 05:07:57,782][105620] Updated weights for policy 1, policy_version 1889997 (0.0010) [2023-12-27 05:07:58,406][105692] Updated weights for policy 0, policy_version 1885493 (0.0008) [2023-12-27 05:07:58,462][105692] Updated weights for policy 0, policy_version 1885503 (0.0008) [2023-12-27 05:07:58,527][105692] Updated weights for policy 0, policy_version 1885513 (0.0009) [2023-12-27 05:07:58,582][105620] Updated weights for policy 1, policy_version 1890008 (0.0010) [2023-12-27 05:07:58,645][105620] Updated weights for policy 1, policy_version 1890018 (0.0007) [2023-12-27 05:07:58,698][105620] Updated weights for policy 1, policy_version 1890028 (0.0006) [2023-12-27 05:07:59,271][105692] Updated weights for policy 0, policy_version 1885523 (0.0007) [2023-12-27 05:07:59,338][105692] Updated weights for policy 0, policy_version 1885533 (0.0008) [2023-12-27 05:07:59,398][105692] Updated weights for policy 0, policy_version 1885543 (0.0010) [2023-12-27 05:07:59,450][105620] Updated weights for policy 1, policy_version 1890038 (0.0006) [2023-12-27 05:07:59,519][105620] Updated weights for policy 1, policy_version 1890048 (0.0008) [2023-12-27 05:07:59,583][105620] Updated weights for policy 1, policy_version 1890058 (0.0008) [2023-12-27 05:08:00,132][105692] Updated weights for policy 0, policy_version 1885553 (0.0008) [2023-12-27 05:08:00,196][105692] Updated weights for policy 0, policy_version 1885563 (0.0005) [2023-12-27 05:08:00,259][105692] Updated weights for policy 0, policy_version 1885573 (0.0006) [2023-12-27 05:08:00,278][105620] Updated weights for policy 1, policy_version 1890068 (0.0006) [2023-12-27 05:08:00,316][105692] Updated weights for policy 0, policy_version 1885583 (0.0009) [2023-12-27 05:08:00,339][105620] Updated weights for policy 1, policy_version 1890078 (0.0007) [2023-12-27 05:08:00,395][105620] Updated weights for policy 1, policy_version 1890088 (0.0009) [2023-12-27 05:08:00,878][105692] Updated weights for policy 0, policy_version 1885593 (0.0005) [2023-12-27 05:08:00,944][105692] Updated weights for policy 0, policy_version 1885603 (0.0006) [2023-12-27 05:08:00,996][105692] Updated weights for policy 0, policy_version 1885613 (0.0005) [2023-12-27 05:08:01,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19577.5). Total num frames: 966721536. Throughput: 0: 9723.5, 1: 9864.2. Samples: 966689228. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:08:01,063][104569] Avg episode reward: [(0, '8078.507'), (1, '9345.189')] [2023-12-27 05:08:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001885616_482787328.pth... [2023-12-27 05:08:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001890096_483934208.pth... [2023-12-27 05:08:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001884464_482492416.pth [2023-12-27 05:08:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001888944_483639296.pth [2023-12-27 05:08:01,181][105620] Updated weights for policy 1, policy_version 1890098 (0.0009) [2023-12-27 05:08:01,247][105620] Updated weights for policy 1, policy_version 1890108 (0.0006) [2023-12-27 05:08:01,307][105620] Updated weights for policy 1, policy_version 1890118 (0.0009) [2023-12-27 05:08:01,366][105620] Updated weights for policy 1, policy_version 1890128 (0.0010) [2023-12-27 05:08:01,590][105692] Updated weights for policy 0, policy_version 1885623 (0.0008) [2023-12-27 05:08:01,652][105692] Updated weights for policy 0, policy_version 1885633 (0.0008) [2023-12-27 05:08:01,703][105692] Updated weights for policy 0, policy_version 1885643 (0.0006) [2023-12-27 05:08:02,069][105620] Updated weights for policy 1, policy_version 1890138 (0.0006) [2023-12-27 05:08:02,133][105620] Updated weights for policy 1, policy_version 1890148 (0.0006) [2023-12-27 05:08:02,202][105620] Updated weights for policy 1, policy_version 1890158 (0.0005) [2023-12-27 05:08:02,334][105692] Updated weights for policy 0, policy_version 1885653 (0.0008) [2023-12-27 05:08:02,390][105692] Updated weights for policy 0, policy_version 1885663 (0.0010) [2023-12-27 05:08:02,444][105692] Updated weights for policy 0, policy_version 1885673 (0.0010) [2023-12-27 05:08:02,747][105620] Updated weights for policy 1, policy_version 1890168 (0.0009) [2023-12-27 05:08:02,806][105620] Updated weights for policy 1, policy_version 1890178 (0.0010) [2023-12-27 05:08:02,855][105620] Updated weights for policy 1, policy_version 1890188 (0.0010) [2023-12-27 05:08:03,314][105692] Updated weights for policy 0, policy_version 1885683 (0.0009) [2023-12-27 05:08:03,361][105692] Updated weights for policy 0, policy_version 1885693 (0.0008) [2023-12-27 05:08:03,406][105692] Updated weights for policy 0, policy_version 1885703 (0.0007) [2023-12-27 05:08:03,594][105620] Updated weights for policy 1, policy_version 1890198 (0.0010) [2023-12-27 05:08:03,652][105620] Updated weights for policy 1, policy_version 1890208 (0.0010) [2023-12-27 05:08:03,703][105620] Updated weights for policy 1, policy_version 1890218 (0.0010) [2023-12-27 05:08:04,211][105692] Updated weights for policy 0, policy_version 1885713 (0.0008) [2023-12-27 05:08:04,268][105692] Updated weights for policy 0, policy_version 1885723 (0.0008) [2023-12-27 05:08:04,324][105692] Updated weights for policy 0, policy_version 1885733 (0.0008) [2023-12-27 05:08:04,372][105692] Updated weights for policy 0, policy_version 1885743 (0.0008) [2023-12-27 05:08:04,471][105620] Updated weights for policy 1, policy_version 1890228 (0.0010) [2023-12-27 05:08:04,530][105620] Updated weights for policy 1, policy_version 1890238 (0.0010) [2023-12-27 05:08:04,579][105620] Updated weights for policy 1, policy_version 1890248 (0.0010) [2023-12-27 05:08:05,152][105692] Updated weights for policy 0, policy_version 1885753 (0.0008) [2023-12-27 05:08:05,218][105692] Updated weights for policy 0, policy_version 1885763 (0.0009) [2023-12-27 05:08:05,280][105620] Updated weights for policy 1, policy_version 1890258 (0.0010) [2023-12-27 05:08:05,283][105692] Updated weights for policy 0, policy_version 1885773 (0.0008) [2023-12-27 05:08:05,335][105620] Updated weights for policy 1, policy_version 1890268 (0.0009) [2023-12-27 05:08:05,388][105620] Updated weights for policy 1, policy_version 1890278 (0.0005) [2023-12-27 05:08:05,442][105620] Updated weights for policy 1, policy_version 1890288 (0.0010) [2023-12-27 05:08:06,024][105620] Updated weights for policy 1, policy_version 1890298 (0.0005) [2023-12-27 05:08:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 966811648. Throughput: 0: 9729.7, 1: 9861.6. Samples: 966806204. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:08:06,062][104569] Avg episode reward: [(0, '8259.008'), (1, '9160.671')] [2023-12-27 05:08:06,082][105620] Updated weights for policy 1, policy_version 1890308 (0.0005) [2023-12-27 05:08:06,120][105692] Updated weights for policy 0, policy_version 1885783 (0.0007) [2023-12-27 05:08:06,143][105620] Updated weights for policy 1, policy_version 1890318 (0.0008) [2023-12-27 05:08:06,189][105692] Updated weights for policy 0, policy_version 1885793 (0.0008) [2023-12-27 05:08:06,247][105692] Updated weights for policy 0, policy_version 1885803 (0.0008) [2023-12-27 05:08:06,825][105620] Updated weights for policy 1, policy_version 1890328 (0.0006) [2023-12-27 05:08:06,883][105620] Updated weights for policy 1, policy_version 1890338 (0.0005) [2023-12-27 05:08:06,944][105620] Updated weights for policy 1, policy_version 1890348 (0.0005) [2023-12-27 05:08:07,078][105692] Updated weights for policy 0, policy_version 1885813 (0.0009) [2023-12-27 05:08:07,145][105692] Updated weights for policy 0, policy_version 1885823 (0.0010) [2023-12-27 05:08:07,203][105692] Updated weights for policy 0, policy_version 1885833 (0.0010) [2023-12-27 05:08:07,448][105620] Updated weights for policy 1, policy_version 1890358 (0.0007) [2023-12-27 05:08:07,514][105620] Updated weights for policy 1, policy_version 1890368 (0.0011) [2023-12-27 05:08:07,577][105620] Updated weights for policy 1, policy_version 1890378 (0.0007) [2023-12-27 05:08:08,042][105692] Updated weights for policy 0, policy_version 1885843 (0.0010) [2023-12-27 05:08:08,097][105692] Updated weights for policy 0, policy_version 1885853 (0.0008) [2023-12-27 05:08:08,149][105692] Updated weights for policy 0, policy_version 1885863 (0.0008) [2023-12-27 05:08:08,211][105620] Updated weights for policy 1, policy_version 1890388 (0.0010) [2023-12-27 05:08:08,278][105620] Updated weights for policy 1, policy_version 1890398 (0.0010) [2023-12-27 05:08:08,343][105620] Updated weights for policy 1, policy_version 1890408 (0.0010) [2023-12-27 05:08:08,897][105692] Updated weights for policy 0, policy_version 1885873 (0.0006) [2023-12-27 05:08:08,944][105692] Updated weights for policy 0, policy_version 1885883 (0.0008) [2023-12-27 05:08:09,004][105692] Updated weights for policy 0, policy_version 1885893 (0.0008) [2023-12-27 05:08:09,059][105692] Updated weights for policy 0, policy_version 1885903 (0.0008) [2023-12-27 05:08:09,085][105620] Updated weights for policy 1, policy_version 1890418 (0.0010) [2023-12-27 05:08:09,133][105620] Updated weights for policy 1, policy_version 1890428 (0.0010) [2023-12-27 05:08:09,181][105620] Updated weights for policy 1, policy_version 1890438 (0.0010) [2023-12-27 05:08:09,244][105620] Updated weights for policy 1, policy_version 1890448 (0.0009) [2023-12-27 05:08:09,803][105692] Updated weights for policy 0, policy_version 1885913 (0.0009) [2023-12-27 05:08:09,867][105692] Updated weights for policy 0, policy_version 1885923 (0.0009) [2023-12-27 05:08:09,930][105692] Updated weights for policy 0, policy_version 1885933 (0.0009) [2023-12-27 05:08:09,983][105620] Updated weights for policy 1, policy_version 1890458 (0.0010) [2023-12-27 05:08:10,036][105620] Updated weights for policy 1, policy_version 1890468 (0.0010) [2023-12-27 05:08:10,096][105620] Updated weights for policy 1, policy_version 1890478 (0.0010) [2023-12-27 05:08:10,727][105692] Updated weights for policy 0, policy_version 1885943 (0.0010) [2023-12-27 05:08:10,776][105692] Updated weights for policy 0, policy_version 1885953 (0.0010) [2023-12-27 05:08:10,825][105692] Updated weights for policy 0, policy_version 1885963 (0.0010) [2023-12-27 05:08:10,871][105620] Updated weights for policy 1, policy_version 1890488 (0.0010) [2023-12-27 05:08:10,928][105620] Updated weights for policy 1, policy_version 1890498 (0.0007) [2023-12-27 05:08:10,983][105620] Updated weights for policy 1, policy_version 1890508 (0.0005) [2023-12-27 05:08:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 966918144. Throughput: 0: 9646.8, 1: 10012.6. Samples: 966920548. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:08:11,062][104569] Avg episode reward: [(0, '8165.982'), (1, '9160.645')] [2023-12-27 05:08:11,628][105692] Updated weights for policy 0, policy_version 1885973 (0.0011) [2023-12-27 05:08:11,695][105692] Updated weights for policy 0, policy_version 1885983 (0.0010) [2023-12-27 05:08:11,697][105620] Updated weights for policy 1, policy_version 1890518 (0.0006) [2023-12-27 05:08:11,764][105692] Updated weights for policy 0, policy_version 1885993 (0.0011) [2023-12-27 05:08:11,775][105620] Updated weights for policy 1, policy_version 1890528 (0.0006) [2023-12-27 05:08:11,837][105620] Updated weights for policy 1, policy_version 1890538 (0.0007) [2023-12-27 05:08:12,486][105692] Updated weights for policy 0, policy_version 1886003 (0.0009) [2023-12-27 05:08:12,545][105692] Updated weights for policy 0, policy_version 1886013 (0.0006) [2023-12-27 05:08:12,607][105692] Updated weights for policy 0, policy_version 1886023 (0.0009) [2023-12-27 05:08:12,622][105620] Updated weights for policy 1, policy_version 1890548 (0.0007) [2023-12-27 05:08:12,683][105620] Updated weights for policy 1, policy_version 1890558 (0.0007) [2023-12-27 05:08:12,742][105620] Updated weights for policy 1, policy_version 1890568 (0.0008) [2023-12-27 05:08:13,319][105620] Updated weights for policy 1, policy_version 1890578 (0.0008) [2023-12-27 05:08:13,332][105692] Updated weights for policy 0, policy_version 1886033 (0.0011) [2023-12-27 05:08:13,382][105620] Updated weights for policy 1, policy_version 1890588 (0.0006) [2023-12-27 05:08:13,392][105692] Updated weights for policy 0, policy_version 1886043 (0.0011) [2023-12-27 05:08:13,438][105620] Updated weights for policy 1, policy_version 1890598 (0.0006) [2023-12-27 05:08:13,451][105692] Updated weights for policy 0, policy_version 1886053 (0.0008) [2023-12-27 05:08:13,488][105620] Updated weights for policy 1, policy_version 1890608 (0.0005) [2023-12-27 05:08:13,515][105692] Updated weights for policy 0, policy_version 1886063 (0.0006) [2023-12-27 05:08:14,199][105620] Updated weights for policy 1, policy_version 1890618 (0.0006) [2023-12-27 05:08:14,205][105692] Updated weights for policy 0, policy_version 1886073 (0.0010) [2023-12-27 05:08:14,253][105620] Updated weights for policy 1, policy_version 1890628 (0.0007) [2023-12-27 05:08:14,264][105692] Updated weights for policy 0, policy_version 1886083 (0.0010) [2023-12-27 05:08:14,311][105620] Updated weights for policy 1, policy_version 1890638 (0.0006) [2023-12-27 05:08:14,313][105692] Updated weights for policy 0, policy_version 1886093 (0.0011) [2023-12-27 05:08:15,028][105692] Updated weights for policy 0, policy_version 1886103 (0.0011) [2023-12-27 05:08:15,054][105620] Updated weights for policy 1, policy_version 1890648 (0.0007) [2023-12-27 05:08:15,084][105692] Updated weights for policy 0, policy_version 1886113 (0.0010) [2023-12-27 05:08:15,115][105620] Updated weights for policy 1, policy_version 1890658 (0.0006) [2023-12-27 05:08:15,148][105692] Updated weights for policy 0, policy_version 1886123 (0.0011) [2023-12-27 05:08:15,179][105620] Updated weights for policy 1, policy_version 1890668 (0.0006) [2023-12-27 05:08:15,844][105692] Updated weights for policy 0, policy_version 1886133 (0.0008) [2023-12-27 05:08:15,883][105620] Updated weights for policy 1, policy_version 1890678 (0.0008) [2023-12-27 05:08:15,897][105692] Updated weights for policy 0, policy_version 1886143 (0.0007) [2023-12-27 05:08:15,935][105620] Updated weights for policy 1, policy_version 1890688 (0.0008) [2023-12-27 05:08:15,953][105692] Updated weights for policy 0, policy_version 1886153 (0.0008) [2023-12-27 05:08:15,984][105620] Updated weights for policy 1, policy_version 1890698 (0.0006) [2023-12-27 05:08:16,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 967016448. Throughput: 0: 9644.5, 1: 10028.3. Samples: 966978392. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:08:16,062][104569] Avg episode reward: [(0, '7988.112'), (1, '9253.924')] [2023-12-27 05:08:16,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001890704_484089856.pth... [2023-12-27 05:08:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001886160_482926592.pth... [2023-12-27 05:08:16,069][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001889520_483786752.pth [2023-12-27 05:08:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001885008_482631680.pth [2023-12-27 05:08:16,657][105692] Updated weights for policy 0, policy_version 1886163 (0.0008) [2023-12-27 05:08:16,702][105692] Updated weights for policy 0, policy_version 1886173 (0.0007) [2023-12-27 05:08:16,707][105620] Updated weights for policy 1, policy_version 1890708 (0.0009) [2023-12-27 05:08:16,747][105692] Updated weights for policy 0, policy_version 1886183 (0.0006) [2023-12-27 05:08:16,753][105620] Updated weights for policy 1, policy_version 1890718 (0.0007) [2023-12-27 05:08:16,795][105620] Updated weights for policy 1, policy_version 1890728 (0.0006) [2023-12-27 05:08:17,343][105692] Updated weights for policy 0, policy_version 1886193 (0.0006) [2023-12-27 05:08:17,391][105692] Updated weights for policy 0, policy_version 1886203 (0.0009) [2023-12-27 05:08:17,469][105692] Updated weights for policy 0, policy_version 1886213 (0.0009) [2023-12-27 05:08:17,522][105692] Updated weights for policy 0, policy_version 1886223 (0.0009) [2023-12-27 05:08:17,597][105620] Updated weights for policy 1, policy_version 1890738 (0.0008) [2023-12-27 05:08:17,662][105620] Updated weights for policy 1, policy_version 1890748 (0.0006) [2023-12-27 05:08:17,720][105620] Updated weights for policy 1, policy_version 1890758 (0.0005) [2023-12-27 05:08:17,780][105620] Updated weights for policy 1, policy_version 1890768 (0.0007) [2023-12-27 05:08:18,345][105692] Updated weights for policy 0, policy_version 1886233 (0.0008) [2023-12-27 05:08:18,399][105692] Updated weights for policy 0, policy_version 1886243 (0.0006) [2023-12-27 05:08:18,410][105620] Updated weights for policy 1, policy_version 1890778 (0.0009) [2023-12-27 05:08:18,455][105692] Updated weights for policy 0, policy_version 1886253 (0.0006) [2023-12-27 05:08:18,474][105620] Updated weights for policy 1, policy_version 1890788 (0.0007) [2023-12-27 05:08:18,536][105620] Updated weights for policy 1, policy_version 1890798 (0.0010) [2023-12-27 05:08:19,020][105692] Updated weights for policy 0, policy_version 1886263 (0.0005) [2023-12-27 05:08:19,085][105692] Updated weights for policy 0, policy_version 1886273 (0.0009) [2023-12-27 05:08:19,141][105692] Updated weights for policy 0, policy_version 1886283 (0.0010) [2023-12-27 05:08:19,402][105620] Updated weights for policy 1, policy_version 1890808 (0.0009) [2023-12-27 05:08:19,462][105620] Updated weights for policy 1, policy_version 1890818 (0.0008) [2023-12-27 05:08:19,528][105620] Updated weights for policy 1, policy_version 1890828 (0.0008) [2023-12-27 05:08:19,849][105692] Updated weights for policy 0, policy_version 1886293 (0.0008) [2023-12-27 05:08:19,914][105692] Updated weights for policy 0, policy_version 1886303 (0.0010) [2023-12-27 05:08:19,978][105692] Updated weights for policy 0, policy_version 1886313 (0.0011) [2023-12-27 05:08:20,322][105620] Updated weights for policy 1, policy_version 1890838 (0.0008) [2023-12-27 05:08:20,384][105620] Updated weights for policy 1, policy_version 1890848 (0.0005) [2023-12-27 05:08:20,438][105620] Updated weights for policy 1, policy_version 1890858 (0.0007) [2023-12-27 05:08:20,717][105692] Updated weights for policy 0, policy_version 1886323 (0.0011) [2023-12-27 05:08:20,785][105692] Updated weights for policy 0, policy_version 1886333 (0.0011) [2023-12-27 05:08:20,851][105692] Updated weights for policy 0, policy_version 1886343 (0.0011) [2023-12-27 05:08:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 967106560. Throughput: 0: 9761.0, 1: 9895.9. Samples: 967094872. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:08:21,062][104569] Avg episode reward: [(0, '8262.271'), (1, '9253.945')] [2023-12-27 05:08:21,242][105620] Updated weights for policy 1, policy_version 1890868 (0.0009) [2023-12-27 05:08:21,310][105620] Updated weights for policy 1, policy_version 1890878 (0.0008) [2023-12-27 05:08:21,377][105620] Updated weights for policy 1, policy_version 1890888 (0.0008) [2023-12-27 05:08:21,536][105692] Updated weights for policy 0, policy_version 1886353 (0.0011) [2023-12-27 05:08:21,599][105692] Updated weights for policy 0, policy_version 1886363 (0.0009) [2023-12-27 05:08:21,665][105692] Updated weights for policy 0, policy_version 1886373 (0.0007) [2023-12-27 05:08:21,732][105692] Updated weights for policy 0, policy_version 1886383 (0.0007) [2023-12-27 05:08:22,129][105620] Updated weights for policy 1, policy_version 1890898 (0.0009) [2023-12-27 05:08:22,193][105620] Updated weights for policy 1, policy_version 1890908 (0.0009) [2023-12-27 05:08:22,260][105620] Updated weights for policy 1, policy_version 1890918 (0.0009) [2023-12-27 05:08:22,317][105620] Updated weights for policy 1, policy_version 1890928 (0.0009) [2023-12-27 05:08:22,435][105692] Updated weights for policy 0, policy_version 1886393 (0.0008) [2023-12-27 05:08:22,496][105692] Updated weights for policy 0, policy_version 1886403 (0.0007) [2023-12-27 05:08:22,563][105692] Updated weights for policy 0, policy_version 1886413 (0.0007) [2023-12-27 05:08:23,164][105620] Updated weights for policy 1, policy_version 1890938 (0.0009) [2023-12-27 05:08:23,187][105692] Updated weights for policy 0, policy_version 1886423 (0.0006) [2023-12-27 05:08:23,217][105620] Updated weights for policy 1, policy_version 1890948 (0.0008) [2023-12-27 05:08:23,239][105692] Updated weights for policy 0, policy_version 1886433 (0.0006) [2023-12-27 05:08:23,274][105620] Updated weights for policy 1, policy_version 1890958 (0.0009) [2023-12-27 05:08:23,289][105692] Updated weights for policy 0, policy_version 1886443 (0.0006) [2023-12-27 05:08:23,843][105620] Updated weights for policy 1, policy_version 1890968 (0.0005) [2023-12-27 05:08:23,899][105620] Updated weights for policy 1, policy_version 1890978 (0.0005) [2023-12-27 05:08:23,953][105620] Updated weights for policy 1, policy_version 1890988 (0.0005) [2023-12-27 05:08:24,092][105692] Updated weights for policy 0, policy_version 1886453 (0.0009) [2023-12-27 05:08:24,147][105692] Updated weights for policy 0, policy_version 1886463 (0.0007) [2023-12-27 05:08:24,204][105692] Updated weights for policy 0, policy_version 1886473 (0.0007) [2023-12-27 05:08:24,652][105620] Updated weights for policy 1, policy_version 1890998 (0.0007) [2023-12-27 05:08:24,710][105620] Updated weights for policy 1, policy_version 1891008 (0.0009) [2023-12-27 05:08:24,771][105620] Updated weights for policy 1, policy_version 1891018 (0.0009) [2023-12-27 05:08:24,868][105692] Updated weights for policy 0, policy_version 1886483 (0.0009) [2023-12-27 05:08:24,938][105692] Updated weights for policy 0, policy_version 1886493 (0.0006) [2023-12-27 05:08:25,013][105692] Updated weights for policy 0, policy_version 1886503 (0.0006) [2023-12-27 05:08:25,424][105620] Updated weights for policy 1, policy_version 1891028 (0.0007) [2023-12-27 05:08:25,484][105620] Updated weights for policy 1, policy_version 1891038 (0.0009) [2023-12-27 05:08:25,546][105620] Updated weights for policy 1, policy_version 1891048 (0.0009) [2023-12-27 05:08:25,707][105692] Updated weights for policy 0, policy_version 1886513 (0.0009) [2023-12-27 05:08:25,754][105692] Updated weights for policy 0, policy_version 1886523 (0.0009) [2023-12-27 05:08:25,812][105692] Updated weights for policy 0, policy_version 1886533 (0.0009) [2023-12-27 05:08:25,862][105692] Updated weights for policy 0, policy_version 1886543 (0.0008) [2023-12-27 05:08:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 967204864. Throughput: 0: 9756.5, 1: 9864.9. Samples: 967211012. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:08:26,062][104569] Avg episode reward: [(0, '8266.935'), (1, '9253.982')] [2023-12-27 05:08:26,286][105620] Updated weights for policy 1, policy_version 1891058 (0.0009) [2023-12-27 05:08:26,346][105620] Updated weights for policy 1, policy_version 1891068 (0.0009) [2023-12-27 05:08:26,404][105620] Updated weights for policy 1, policy_version 1891078 (0.0009) [2023-12-27 05:08:26,465][105620] Updated weights for policy 1, policy_version 1891088 (0.0009) [2023-12-27 05:08:26,617][105692] Updated weights for policy 0, policy_version 1886553 (0.0009) [2023-12-27 05:08:26,674][105692] Updated weights for policy 0, policy_version 1886563 (0.0010) [2023-12-27 05:08:26,728][105692] Updated weights for policy 0, policy_version 1886574 (0.0010) [2023-12-27 05:08:27,096][105620] Updated weights for policy 1, policy_version 1891098 (0.0009) [2023-12-27 05:08:27,143][105620] Updated weights for policy 1, policy_version 1891108 (0.0008) [2023-12-27 05:08:27,194][105620] Updated weights for policy 1, policy_version 1891118 (0.0007) [2023-12-27 05:08:27,562][105692] Updated weights for policy 0, policy_version 1886585 (0.0009) [2023-12-27 05:08:27,610][105692] Updated weights for policy 0, policy_version 1886595 (0.0009) [2023-12-27 05:08:27,655][105692] Updated weights for policy 0, policy_version 1886605 (0.0008) [2023-12-27 05:08:27,848][105620] Updated weights for policy 1, policy_version 1891128 (0.0005) [2023-12-27 05:08:27,897][105620] Updated weights for policy 1, policy_version 1891138 (0.0005) [2023-12-27 05:08:27,942][105620] Updated weights for policy 1, policy_version 1891148 (0.0005) [2023-12-27 05:08:28,524][105620] Updated weights for policy 1, policy_version 1891158 (0.0006) [2023-12-27 05:08:28,529][105692] Updated weights for policy 0, policy_version 1886615 (0.0009) [2023-12-27 05:08:28,586][105620] Updated weights for policy 1, policy_version 1891168 (0.0007) [2023-12-27 05:08:28,587][105692] Updated weights for policy 0, policy_version 1886625 (0.0009) [2023-12-27 05:08:28,637][105692] Updated weights for policy 0, policy_version 1886635 (0.0008) [2023-12-27 05:08:28,646][105620] Updated weights for policy 1, policy_version 1891178 (0.0010) [2023-12-27 05:08:29,259][105620] Updated weights for policy 1, policy_version 1891188 (0.0008) [2023-12-27 05:08:29,319][105620] Updated weights for policy 1, policy_version 1891198 (0.0006) [2023-12-27 05:08:29,387][105620] Updated weights for policy 1, policy_version 1891208 (0.0007) [2023-12-27 05:08:29,483][105692] Updated weights for policy 0, policy_version 1886645 (0.0006) [2023-12-27 05:08:29,530][105692] Updated weights for policy 0, policy_version 1886655 (0.0009) [2023-12-27 05:08:29,578][105692] Updated weights for policy 0, policy_version 1886665 (0.0006) [2023-12-27 05:08:30,037][105620] Updated weights for policy 1, policy_version 1891218 (0.0006) [2023-12-27 05:08:30,097][105620] Updated weights for policy 1, policy_version 1891228 (0.0007) [2023-12-27 05:08:30,158][105620] Updated weights for policy 1, policy_version 1891238 (0.0007) [2023-12-27 05:08:30,219][105620] Updated weights for policy 1, policy_version 1891248 (0.0009) [2023-12-27 05:08:30,333][105692] Updated weights for policy 0, policy_version 1886675 (0.0009) [2023-12-27 05:08:30,395][105692] Updated weights for policy 0, policy_version 1886685 (0.0009) [2023-12-27 05:08:30,452][105692] Updated weights for policy 0, policy_version 1886695 (0.0009) [2023-12-27 05:08:30,935][105620] Updated weights for policy 1, policy_version 1891258 (0.0005) [2023-12-27 05:08:30,982][105620] Updated weights for policy 1, policy_version 1891268 (0.0008) [2023-12-27 05:08:31,032][105620] Updated weights for policy 1, policy_version 1891278 (0.0009) [2023-12-27 05:08:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 967303168. Throughput: 0: 9750.4, 1: 9922.8. Samples: 967269572. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:08:31,063][104569] Avg episode reward: [(0, '8089.163'), (1, '9252.924')] [2023-12-27 05:08:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001886704_483065856.pth... [2023-12-27 05:08:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001891280_484237312.pth... [2023-12-27 05:08:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001885616_482787328.pth [2023-12-27 05:08:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001890096_483934208.pth [2023-12-27 05:08:31,142][105692] Updated weights for policy 0, policy_version 1886705 (0.0009) [2023-12-27 05:08:31,205][105692] Updated weights for policy 0, policy_version 1886715 (0.0009) [2023-12-27 05:08:31,261][105692] Updated weights for policy 0, policy_version 1886725 (0.0008) [2023-12-27 05:08:31,322][105692] Updated weights for policy 0, policy_version 1886735 (0.0009) [2023-12-27 05:08:31,715][105620] Updated weights for policy 1, policy_version 1891288 (0.0008) [2023-12-27 05:08:31,777][105620] Updated weights for policy 1, policy_version 1891298 (0.0008) [2023-12-27 05:08:31,825][105620] Updated weights for policy 1, policy_version 1891308 (0.0008) [2023-12-27 05:08:32,020][105692] Updated weights for policy 0, policy_version 1886745 (0.0006) [2023-12-27 05:08:32,077][105692] Updated weights for policy 0, policy_version 1886755 (0.0010) [2023-12-27 05:08:32,134][105692] Updated weights for policy 0, policy_version 1886765 (0.0008) [2023-12-27 05:08:32,531][105620] Updated weights for policy 1, policy_version 1891318 (0.0008) [2023-12-27 05:08:32,592][105620] Updated weights for policy 1, policy_version 1891328 (0.0009) [2023-12-27 05:08:32,651][105620] Updated weights for policy 1, policy_version 1891338 (0.0009) [2023-12-27 05:08:32,909][105692] Updated weights for policy 0, policy_version 1886775 (0.0009) [2023-12-27 05:08:32,966][105692] Updated weights for policy 0, policy_version 1886785 (0.0010) [2023-12-27 05:08:33,020][105692] Updated weights for policy 0, policy_version 1886796 (0.0010) [2023-12-27 05:08:33,276][105620] Updated weights for policy 1, policy_version 1891348 (0.0008) [2023-12-27 05:08:33,337][105620] Updated weights for policy 1, policy_version 1891358 (0.0008) [2023-12-27 05:08:33,396][105620] Updated weights for policy 1, policy_version 1891368 (0.0005) [2023-12-27 05:08:33,857][105692] Updated weights for policy 0, policy_version 1886807 (0.0009) [2023-12-27 05:08:33,907][105692] Updated weights for policy 0, policy_version 1886817 (0.0009) [2023-12-27 05:08:33,958][105692] Updated weights for policy 0, policy_version 1886827 (0.0009) [2023-12-27 05:08:34,079][105620] Updated weights for policy 1, policy_version 1891378 (0.0006) [2023-12-27 05:08:34,137][105620] Updated weights for policy 1, policy_version 1891388 (0.0009) [2023-12-27 05:08:34,201][105620] Updated weights for policy 1, policy_version 1891398 (0.0009) [2023-12-27 05:08:34,263][105620] Updated weights for policy 1, policy_version 1891408 (0.0009) [2023-12-27 05:08:34,729][105692] Updated weights for policy 0, policy_version 1886837 (0.0009) [2023-12-27 05:08:34,787][105692] Updated weights for policy 0, policy_version 1886847 (0.0009) [2023-12-27 05:08:34,834][105692] Updated weights for policy 0, policy_version 1886857 (0.0009) [2023-12-27 05:08:35,033][105620] Updated weights for policy 1, policy_version 1891418 (0.0009) [2023-12-27 05:08:35,080][105620] Updated weights for policy 1, policy_version 1891428 (0.0008) [2023-12-27 05:08:35,131][105620] Updated weights for policy 1, policy_version 1891438 (0.0009) [2023-12-27 05:08:35,591][105692] Updated weights for policy 0, policy_version 1886867 (0.0009) [2023-12-27 05:08:35,643][105692] Updated weights for policy 0, policy_version 1886877 (0.0009) [2023-12-27 05:08:35,701][105692] Updated weights for policy 0, policy_version 1886887 (0.0009) [2023-12-27 05:08:35,899][105620] Updated weights for policy 1, policy_version 1891448 (0.0009) [2023-12-27 05:08:35,946][105620] Updated weights for policy 1, policy_version 1891458 (0.0008) [2023-12-27 05:08:35,994][105620] Updated weights for policy 1, policy_version 1891468 (0.0005) [2023-12-27 05:08:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 967401472. Throughput: 0: 9638.1, 1: 9976.4. Samples: 967386416. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:08:36,062][104569] Avg episode reward: [(0, '8270.520'), (1, '9162.411')] [2023-12-27 05:08:36,497][105692] Updated weights for policy 0, policy_version 1886897 (0.0009) [2023-12-27 05:08:36,553][105692] Updated weights for policy 0, policy_version 1886907 (0.0009) [2023-12-27 05:08:36,615][105692] Updated weights for policy 0, policy_version 1886917 (0.0009) [2023-12-27 05:08:36,670][105692] Updated weights for policy 0, policy_version 1886927 (0.0009) [2023-12-27 05:08:36,718][105620] Updated weights for policy 1, policy_version 1891478 (0.0008) [2023-12-27 05:08:36,765][105620] Updated weights for policy 1, policy_version 1891488 (0.0008) [2023-12-27 05:08:36,813][105620] Updated weights for policy 1, policy_version 1891498 (0.0009) [2023-12-27 05:08:37,483][105692] Updated weights for policy 0, policy_version 1886937 (0.0009) [2023-12-27 05:08:37,492][105620] Updated weights for policy 1, policy_version 1891508 (0.0010) [2023-12-27 05:08:37,546][105692] Updated weights for policy 0, policy_version 1886947 (0.0006) [2023-12-27 05:08:37,548][105620] Updated weights for policy 1, policy_version 1891518 (0.0010) [2023-12-27 05:08:37,602][105692] Updated weights for policy 0, policy_version 1886957 (0.0006) [2023-12-27 05:08:37,603][105620] Updated weights for policy 1, policy_version 1891528 (0.0010) [2023-12-27 05:08:38,191][105692] Updated weights for policy 0, policy_version 1886967 (0.0005) [2023-12-27 05:08:38,248][105620] Updated weights for policy 1, policy_version 1891538 (0.0010) [2023-12-27 05:08:38,253][105692] Updated weights for policy 0, policy_version 1886977 (0.0005) [2023-12-27 05:08:38,297][105620] Updated weights for policy 1, policy_version 1891548 (0.0009) [2023-12-27 05:08:38,316][105692] Updated weights for policy 0, policy_version 1886987 (0.0006) [2023-12-27 05:08:38,358][105620] Updated weights for policy 1, policy_version 1891558 (0.0008) [2023-12-27 05:08:38,425][105620] Updated weights for policy 1, policy_version 1891568 (0.0005) [2023-12-27 05:08:38,854][105692] Updated weights for policy 0, policy_version 1886997 (0.0009) [2023-12-27 05:08:38,909][105692] Updated weights for policy 0, policy_version 1887007 (0.0010) [2023-12-27 05:08:38,967][105692] Updated weights for policy 0, policy_version 1887017 (0.0009) [2023-12-27 05:08:39,138][105620] Updated weights for policy 1, policy_version 1891578 (0.0006) [2023-12-27 05:08:39,193][105620] Updated weights for policy 1, policy_version 1891588 (0.0005) [2023-12-27 05:08:39,263][105620] Updated weights for policy 1, policy_version 1891598 (0.0008) [2023-12-27 05:08:39,706][105692] Updated weights for policy 0, policy_version 1887027 (0.0008) [2023-12-27 05:08:39,759][105692] Updated weights for policy 0, policy_version 1887037 (0.0005) [2023-12-27 05:08:39,813][105692] Updated weights for policy 0, policy_version 1887047 (0.0007) [2023-12-27 05:08:40,017][105620] Updated weights for policy 1, policy_version 1891608 (0.0009) [2023-12-27 05:08:40,083][105620] Updated weights for policy 1, policy_version 1891618 (0.0010) [2023-12-27 05:08:40,143][105620] Updated weights for policy 1, policy_version 1891628 (0.0007) [2023-12-27 05:08:40,447][105692] Updated weights for policy 0, policy_version 1887057 (0.0008) [2023-12-27 05:08:40,518][105692] Updated weights for policy 0, policy_version 1887067 (0.0006) [2023-12-27 05:08:40,574][105692] Updated weights for policy 0, policy_version 1887077 (0.0005) [2023-12-27 05:08:40,629][105692] Updated weights for policy 0, policy_version 1887087 (0.0007) [2023-12-27 05:08:40,890][105620] Updated weights for policy 1, policy_version 1891638 (0.0009) [2023-12-27 05:08:40,943][105620] Updated weights for policy 1, policy_version 1891648 (0.0009) [2023-12-27 05:08:41,009][105620] Updated weights for policy 1, policy_version 1891658 (0.0009) [2023-12-27 05:08:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19522.0). Total num frames: 967499776. Throughput: 0: 9635.1, 1: 9881.9. Samples: 967504700. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:08:41,063][104569] Avg episode reward: [(0, '8358.379'), (1, '9254.705')] [2023-12-27 05:08:41,292][105692] Updated weights for policy 0, policy_version 1887097 (0.0008) [2023-12-27 05:08:41,347][105692] Updated weights for policy 0, policy_version 1887107 (0.0009) [2023-12-27 05:08:41,420][105692] Updated weights for policy 0, policy_version 1887117 (0.0007) [2023-12-27 05:08:41,902][105620] Updated weights for policy 1, policy_version 1891668 (0.0009) [2023-12-27 05:08:41,965][105620] Updated weights for policy 1, policy_version 1891678 (0.0011) [2023-12-27 05:08:42,018][105620] Updated weights for policy 1, policy_version 1891688 (0.0011) [2023-12-27 05:08:42,089][105692] Updated weights for policy 0, policy_version 1887127 (0.0006) [2023-12-27 05:08:42,147][105692] Updated weights for policy 0, policy_version 1887137 (0.0008) [2023-12-27 05:08:42,207][105692] Updated weights for policy 0, policy_version 1887147 (0.0008) [2023-12-27 05:08:42,729][105620] Updated weights for policy 1, policy_version 1891698 (0.0011) [2023-12-27 05:08:42,781][105620] Updated weights for policy 1, policy_version 1891708 (0.0010) [2023-12-27 05:08:42,843][105620] Updated weights for policy 1, policy_version 1891718 (0.0011) [2023-12-27 05:08:42,849][105692] Updated weights for policy 0, policy_version 1887157 (0.0007) [2023-12-27 05:08:42,904][105692] Updated weights for policy 0, policy_version 1887167 (0.0006) [2023-12-27 05:08:42,908][105620] Updated weights for policy 1, policy_version 1891728 (0.0011) [2023-12-27 05:08:42,951][105692] Updated weights for policy 0, policy_version 1887177 (0.0008) [2023-12-27 05:08:43,589][105620] Updated weights for policy 1, policy_version 1891738 (0.0010) [2023-12-27 05:08:43,657][105620] Updated weights for policy 1, policy_version 1891748 (0.0010) [2023-12-27 05:08:43,705][105620] Updated weights for policy 1, policy_version 1891758 (0.0010) [2023-12-27 05:08:43,722][105692] Updated weights for policy 0, policy_version 1887187 (0.0009) [2023-12-27 05:08:43,782][105692] Updated weights for policy 0, policy_version 1887197 (0.0009) [2023-12-27 05:08:43,838][105692] Updated weights for policy 0, policy_version 1887207 (0.0008) [2023-12-27 05:08:44,461][105620] Updated weights for policy 1, policy_version 1891768 (0.0010) [2023-12-27 05:08:44,483][105692] Updated weights for policy 0, policy_version 1887217 (0.0008) [2023-12-27 05:08:44,517][105620] Updated weights for policy 1, policy_version 1891778 (0.0010) [2023-12-27 05:08:44,547][105692] Updated weights for policy 0, policy_version 1887227 (0.0006) [2023-12-27 05:08:44,568][105620] Updated weights for policy 1, policy_version 1891788 (0.0010) [2023-12-27 05:08:44,607][105692] Updated weights for policy 0, policy_version 1887237 (0.0006) [2023-12-27 05:08:44,662][105692] Updated weights for policy 0, policy_version 1887247 (0.0008) [2023-12-27 05:08:45,341][105620] Updated weights for policy 1, policy_version 1891798 (0.0009) [2023-12-27 05:08:45,387][105692] Updated weights for policy 0, policy_version 1887257 (0.0011) [2023-12-27 05:08:45,405][105620] Updated weights for policy 1, policy_version 1891808 (0.0006) [2023-12-27 05:08:45,447][105692] Updated weights for policy 0, policy_version 1887267 (0.0009) [2023-12-27 05:08:45,466][105620] Updated weights for policy 1, policy_version 1891818 (0.0009) [2023-12-27 05:08:45,508][105692] Updated weights for policy 0, policy_version 1887277 (0.0011) [2023-12-27 05:08:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 967589888. Throughput: 0: 9582.0, 1: 9820.0. Samples: 967562320. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:08:46,062][104569] Avg episode reward: [(0, '8352.441'), (1, '9163.932')] [2023-12-27 05:08:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001891824_484376576.pth... [2023-12-27 05:08:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001887280_483213312.pth... [2023-12-27 05:08:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001890704_484089856.pth [2023-12-27 05:08:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001886160_482926592.pth [2023-12-27 05:08:46,237][105620] Updated weights for policy 1, policy_version 1891828 (0.0007) [2023-12-27 05:08:46,292][105620] Updated weights for policy 1, policy_version 1891838 (0.0008) [2023-12-27 05:08:46,329][105692] Updated weights for policy 0, policy_version 1887287 (0.0008) [2023-12-27 05:08:46,350][105620] Updated weights for policy 1, policy_version 1891848 (0.0007) [2023-12-27 05:08:46,376][105692] Updated weights for policy 0, policy_version 1887297 (0.0009) [2023-12-27 05:08:46,437][105692] Updated weights for policy 0, policy_version 1887307 (0.0008) [2023-12-27 05:08:47,120][105620] Updated weights for policy 1, policy_version 1891858 (0.0007) [2023-12-27 05:08:47,121][105692] Updated weights for policy 0, policy_version 1887317 (0.0008) [2023-12-27 05:08:47,180][105692] Updated weights for policy 0, policy_version 1887327 (0.0011) [2023-12-27 05:08:47,182][105620] Updated weights for policy 1, policy_version 1891868 (0.0006) [2023-12-27 05:08:47,236][105692] Updated weights for policy 0, policy_version 1887337 (0.0010) [2023-12-27 05:08:47,241][105620] Updated weights for policy 1, policy_version 1891878 (0.0006) [2023-12-27 05:08:47,307][105620] Updated weights for policy 1, policy_version 1891888 (0.0006) [2023-12-27 05:08:47,833][105620] Updated weights for policy 1, policy_version 1891898 (0.0010) [2023-12-27 05:08:47,891][105620] Updated weights for policy 1, policy_version 1891908 (0.0010) [2023-12-27 05:08:47,939][105620] Updated weights for policy 1, policy_version 1891918 (0.0010) [2023-12-27 05:08:47,943][105692] Updated weights for policy 0, policy_version 1887347 (0.0010) [2023-12-27 05:08:47,997][105692] Updated weights for policy 0, policy_version 1887357 (0.0010) [2023-12-27 05:08:48,056][105692] Updated weights for policy 0, policy_version 1887367 (0.0008) [2023-12-27 05:08:48,565][105620] Updated weights for policy 1, policy_version 1891928 (0.0006) [2023-12-27 05:08:48,613][105620] Updated weights for policy 1, policy_version 1891938 (0.0005) [2023-12-27 05:08:48,661][105620] Updated weights for policy 1, policy_version 1891948 (0.0005) [2023-12-27 05:08:48,793][105692] Updated weights for policy 0, policy_version 1887377 (0.0010) [2023-12-27 05:08:48,848][105692] Updated weights for policy 0, policy_version 1887387 (0.0010) [2023-12-27 05:08:48,896][105692] Updated weights for policy 0, policy_version 1887397 (0.0010) [2023-12-27 05:08:48,948][105692] Updated weights for policy 0, policy_version 1887407 (0.0010) [2023-12-27 05:08:49,383][105620] Updated weights for policy 1, policy_version 1891958 (0.0009) [2023-12-27 05:08:49,446][105620] Updated weights for policy 1, policy_version 1891968 (0.0011) [2023-12-27 05:08:49,505][105620] Updated weights for policy 1, policy_version 1891978 (0.0011) [2023-12-27 05:08:49,659][105692] Updated weights for policy 0, policy_version 1887417 (0.0008) [2023-12-27 05:08:49,711][105692] Updated weights for policy 0, policy_version 1887427 (0.0008) [2023-12-27 05:08:49,763][105692] Updated weights for policy 0, policy_version 1887437 (0.0008) [2023-12-27 05:08:50,237][105620] Updated weights for policy 1, policy_version 1891988 (0.0010) [2023-12-27 05:08:50,289][105620] Updated weights for policy 1, policy_version 1891998 (0.0010) [2023-12-27 05:08:50,338][105620] Updated weights for policy 1, policy_version 1892008 (0.0010) [2023-12-27 05:08:50,519][105692] Updated weights for policy 0, policy_version 1887447 (0.0006) [2023-12-27 05:08:50,592][105692] Updated weights for policy 0, policy_version 1887457 (0.0008) [2023-12-27 05:08:50,648][105692] Updated weights for policy 0, policy_version 1887467 (0.0006) [2023-12-27 05:08:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 967688192. Throughput: 0: 9568.0, 1: 9843.7. Samples: 967679732. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:08:51,063][104569] Avg episode reward: [(0, '8531.164'), (1, '9163.966')] [2023-12-27 05:08:51,079][105620] Updated weights for policy 1, policy_version 1892018 (0.0010) [2023-12-27 05:08:51,144][105620] Updated weights for policy 1, policy_version 1892028 (0.0006) [2023-12-27 05:08:51,211][105620] Updated weights for policy 1, policy_version 1892038 (0.0011) [2023-12-27 05:08:51,281][105692] Updated weights for policy 0, policy_version 1887477 (0.0007) [2023-12-27 05:08:51,283][105620] Updated weights for policy 1, policy_version 1892048 (0.0011) [2023-12-27 05:08:51,334][105692] Updated weights for policy 0, policy_version 1887487 (0.0008) [2023-12-27 05:08:51,398][105692] Updated weights for policy 0, policy_version 1887497 (0.0009) [2023-12-27 05:08:51,974][105620] Updated weights for policy 1, policy_version 1892058 (0.0005) [2023-12-27 05:08:52,036][105620] Updated weights for policy 1, policy_version 1892068 (0.0005) [2023-12-27 05:08:52,098][105620] Updated weights for policy 1, policy_version 1892078 (0.0011) [2023-12-27 05:08:52,198][105692] Updated weights for policy 0, policy_version 1887507 (0.0008) [2023-12-27 05:08:52,261][105692] Updated weights for policy 0, policy_version 1887517 (0.0008) [2023-12-27 05:08:52,324][105692] Updated weights for policy 0, policy_version 1887527 (0.0008) [2023-12-27 05:08:52,784][105620] Updated weights for policy 1, policy_version 1892088 (0.0011) [2023-12-27 05:08:52,842][105620] Updated weights for policy 1, policy_version 1892098 (0.0010) [2023-12-27 05:08:52,897][105620] Updated weights for policy 1, policy_version 1892108 (0.0011) [2023-12-27 05:08:53,084][105692] Updated weights for policy 0, policy_version 1887537 (0.0008) [2023-12-27 05:08:53,143][105692] Updated weights for policy 0, policy_version 1887547 (0.0009) [2023-12-27 05:08:53,195][105692] Updated weights for policy 0, policy_version 1887557 (0.0008) [2023-12-27 05:08:53,253][105692] Updated weights for policy 0, policy_version 1887567 (0.0008) [2023-12-27 05:08:53,587][105620] Updated weights for policy 1, policy_version 1892118 (0.0010) [2023-12-27 05:08:53,644][105620] Updated weights for policy 1, policy_version 1892128 (0.0010) [2023-12-27 05:08:53,701][105620] Updated weights for policy 1, policy_version 1892138 (0.0010) [2023-12-27 05:08:54,033][105692] Updated weights for policy 0, policy_version 1887577 (0.0008) [2023-12-27 05:08:54,088][105692] Updated weights for policy 0, policy_version 1887587 (0.0009) [2023-12-27 05:08:54,150][105692] Updated weights for policy 0, policy_version 1887597 (0.0008) [2023-12-27 05:08:54,440][105620] Updated weights for policy 1, policy_version 1892148 (0.0010) [2023-12-27 05:08:54,498][105620] Updated weights for policy 1, policy_version 1892158 (0.0011) [2023-12-27 05:08:54,550][105620] Updated weights for policy 1, policy_version 1892168 (0.0010) [2023-12-27 05:08:54,928][105692] Updated weights for policy 0, policy_version 1887607 (0.0009) [2023-12-27 05:08:54,993][105692] Updated weights for policy 0, policy_version 1887617 (0.0008) [2023-12-27 05:08:55,051][105692] Updated weights for policy 0, policy_version 1887627 (0.0009) [2023-12-27 05:08:55,261][105620] Updated weights for policy 1, policy_version 1892178 (0.0009) [2023-12-27 05:08:55,311][105620] Updated weights for policy 1, policy_version 1892188 (0.0009) [2023-12-27 05:08:55,362][105620] Updated weights for policy 1, policy_version 1892198 (0.0009) [2023-12-27 05:08:55,411][105620] Updated weights for policy 1, policy_version 1892208 (0.0009) [2023-12-27 05:08:55,793][105692] Updated weights for policy 0, policy_version 1887637 (0.0009) [2023-12-27 05:08:55,848][105692] Updated weights for policy 0, policy_version 1887647 (0.0009) [2023-12-27 05:08:55,899][105692] Updated weights for policy 0, policy_version 1887657 (0.0008) [2023-12-27 05:08:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 967786496. Throughput: 0: 9645.9, 1: 9763.2. Samples: 967793956. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:08:56,063][104569] Avg episode reward: [(0, '8353.856'), (1, '9345.233')] [2023-12-27 05:08:56,199][105620] Updated weights for policy 1, policy_version 1892218 (0.0010) [2023-12-27 05:08:56,257][105620] Updated weights for policy 1, policy_version 1892228 (0.0010) [2023-12-27 05:08:56,312][105620] Updated weights for policy 1, policy_version 1892238 (0.0010) [2023-12-27 05:08:56,530][105692] Updated weights for policy 0, policy_version 1887667 (0.0008) [2023-12-27 05:08:56,589][105692] Updated weights for policy 0, policy_version 1887677 (0.0008) [2023-12-27 05:08:56,649][105692] Updated weights for policy 0, policy_version 1887687 (0.0008) [2023-12-27 05:08:57,170][105620] Updated weights for policy 1, policy_version 1892248 (0.0010) [2023-12-27 05:08:57,218][105620] Updated weights for policy 1, policy_version 1892258 (0.0010) [2023-12-27 05:08:57,295][105692] Updated weights for policy 0, policy_version 1887697 (0.0008) [2023-12-27 05:08:57,298][105620] Updated weights for policy 1, policy_version 1892268 (0.0010) [2023-12-27 05:08:57,356][105692] Updated weights for policy 0, policy_version 1887707 (0.0007) [2023-12-27 05:08:57,410][105692] Updated weights for policy 0, policy_version 1887717 (0.0008) [2023-12-27 05:08:57,457][105692] Updated weights for policy 0, policy_version 1887727 (0.0008) [2023-12-27 05:08:57,930][105620] Updated weights for policy 1, policy_version 1892278 (0.0009) [2023-12-27 05:08:57,974][105620] Updated weights for policy 1, policy_version 1892288 (0.0007) [2023-12-27 05:08:58,017][105620] Updated weights for policy 1, policy_version 1892298 (0.0007) [2023-12-27 05:08:58,274][105692] Updated weights for policy 0, policy_version 1887737 (0.0009) [2023-12-27 05:08:58,331][105692] Updated weights for policy 0, policy_version 1887747 (0.0009) [2023-12-27 05:08:58,396][105692] Updated weights for policy 0, policy_version 1887757 (0.0009) [2023-12-27 05:08:58,950][105620] Updated weights for policy 1, policy_version 1892308 (0.0008) [2023-12-27 05:08:59,005][105620] Updated weights for policy 1, policy_version 1892318 (0.0010) [2023-12-27 05:08:59,072][105620] Updated weights for policy 1, policy_version 1892328 (0.0010) [2023-12-27 05:08:59,279][105692] Updated weights for policy 0, policy_version 1887767 (0.0009) [2023-12-27 05:08:59,346][105692] Updated weights for policy 0, policy_version 1887777 (0.0008) [2023-12-27 05:08:59,410][105692] Updated weights for policy 0, policy_version 1887787 (0.0008) [2023-12-27 05:08:59,909][105620] Updated weights for policy 1, policy_version 1892338 (0.0008) [2023-12-27 05:08:59,969][105620] Updated weights for policy 1, policy_version 1892348 (0.0008) [2023-12-27 05:09:00,020][105620] Updated weights for policy 1, policy_version 1892358 (0.0009) [2023-12-27 05:09:00,074][105620] Updated weights for policy 1, policy_version 1892368 (0.0010) [2023-12-27 05:09:00,145][105692] Updated weights for policy 0, policy_version 1887797 (0.0009) [2023-12-27 05:09:00,195][105692] Updated weights for policy 0, policy_version 1887807 (0.0006) [2023-12-27 05:09:00,259][105692] Updated weights for policy 0, policy_version 1887817 (0.0006) [2023-12-27 05:09:00,788][105620] Updated weights for policy 1, policy_version 1892378 (0.0008) [2023-12-27 05:09:00,841][105620] Updated weights for policy 1, policy_version 1892388 (0.0008) [2023-12-27 05:09:00,895][105620] Updated weights for policy 1, policy_version 1892398 (0.0009) [2023-12-27 05:09:00,925][105692] Updated weights for policy 0, policy_version 1887827 (0.0009) [2023-12-27 05:09:00,982][105692] Updated weights for policy 0, policy_version 1887837 (0.0009) [2023-12-27 05:09:01,056][105692] Updated weights for policy 0, policy_version 1887847 (0.0009) [2023-12-27 05:09:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 967876608. Throughput: 0: 9669.5, 1: 9692.9. Samples: 967849700. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:09:01,063][104569] Avg episode reward: [(0, '7991.094'), (1, '9345.168')] [2023-12-27 05:09:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001892400_484524032.pth... [2023-12-27 05:09:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001891280_484237312.pth [2023-12-27 05:09:01,110][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001887856_483360768.pth... [2023-12-27 05:09:01,115][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001886704_483065856.pth [2023-12-27 05:09:01,682][105620] Updated weights for policy 1, policy_version 1892408 (0.0008) [2023-12-27 05:09:01,745][105620] Updated weights for policy 1, policy_version 1892418 (0.0010) [2023-12-27 05:09:01,782][105692] Updated weights for policy 0, policy_version 1887857 (0.0009) [2023-12-27 05:09:01,794][105620] Updated weights for policy 1, policy_version 1892428 (0.0008) [2023-12-27 05:09:01,834][105692] Updated weights for policy 0, policy_version 1887867 (0.0006) [2023-12-27 05:09:01,887][105692] Updated weights for policy 0, policy_version 1887877 (0.0009) [2023-12-27 05:09:01,949][105692] Updated weights for policy 0, policy_version 1887887 (0.0009) [2023-12-27 05:09:02,616][105620] Updated weights for policy 1, policy_version 1892438 (0.0007) [2023-12-27 05:09:02,639][105692] Updated weights for policy 0, policy_version 1887897 (0.0006) [2023-12-27 05:09:02,676][105620] Updated weights for policy 1, policy_version 1892448 (0.0006) [2023-12-27 05:09:02,697][105692] Updated weights for policy 0, policy_version 1887907 (0.0006) [2023-12-27 05:09:02,735][105620] Updated weights for policy 1, policy_version 1892458 (0.0007) [2023-12-27 05:09:02,758][105692] Updated weights for policy 0, policy_version 1887917 (0.0005) [2023-12-27 05:09:03,323][105620] Updated weights for policy 1, policy_version 1892468 (0.0006) [2023-12-27 05:09:03,381][105620] Updated weights for policy 1, policy_version 1892478 (0.0006) [2023-12-27 05:09:03,431][105620] Updated weights for policy 1, policy_version 1892488 (0.0005) [2023-12-27 05:09:03,433][105692] Updated weights for policy 0, policy_version 1887927 (0.0008) [2023-12-27 05:09:03,490][105692] Updated weights for policy 0, policy_version 1887937 (0.0009) [2023-12-27 05:09:03,540][105692] Updated weights for policy 0, policy_version 1887947 (0.0009) [2023-12-27 05:09:04,061][105620] Updated weights for policy 1, policy_version 1892498 (0.0006) [2023-12-27 05:09:04,123][105620] Updated weights for policy 1, policy_version 1892508 (0.0007) [2023-12-27 05:09:04,185][105620] Updated weights for policy 1, policy_version 1892518 (0.0009) [2023-12-27 05:09:04,247][105620] Updated weights for policy 1, policy_version 1892528 (0.0009) [2023-12-27 05:09:04,310][105692] Updated weights for policy 0, policy_version 1887957 (0.0007) [2023-12-27 05:09:04,374][105692] Updated weights for policy 0, policy_version 1887967 (0.0009) [2023-12-27 05:09:04,431][105692] Updated weights for policy 0, policy_version 1887977 (0.0009) [2023-12-27 05:09:05,008][105620] Updated weights for policy 1, policy_version 1892538 (0.0008) [2023-12-27 05:09:05,069][105620] Updated weights for policy 1, policy_version 1892548 (0.0007) [2023-12-27 05:09:05,115][105620] Updated weights for policy 1, policy_version 1892558 (0.0008) [2023-12-27 05:09:05,189][105692] Updated weights for policy 0, policy_version 1887987 (0.0009) [2023-12-27 05:09:05,248][105692] Updated weights for policy 0, policy_version 1887997 (0.0009) [2023-12-27 05:09:05,311][105692] Updated weights for policy 0, policy_version 1888007 (0.0009) [2023-12-27 05:09:05,863][105620] Updated weights for policy 1, policy_version 1892568 (0.0009) [2023-12-27 05:09:05,920][105620] Updated weights for policy 1, policy_version 1892578 (0.0009) [2023-12-27 05:09:05,978][105620] Updated weights for policy 1, policy_version 1892588 (0.0009) [2023-12-27 05:09:06,053][105692] Updated weights for policy 0, policy_version 1888017 (0.0009) [2023-12-27 05:09:06,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 967974912. Throughput: 0: 9590.3, 1: 9721.5. Samples: 967963904. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:09:06,062][104569] Avg episode reward: [(0, '7898.084'), (1, '9345.253')] [2023-12-27 05:09:06,111][105692] Updated weights for policy 0, policy_version 1888027 (0.0009) [2023-12-27 05:09:06,178][105692] Updated weights for policy 0, policy_version 1888037 (0.0007) [2023-12-27 05:09:06,228][105692] Updated weights for policy 0, policy_version 1888047 (0.0007) [2023-12-27 05:09:06,739][105620] Updated weights for policy 1, policy_version 1892598 (0.0010) [2023-12-27 05:09:06,788][105620] Updated weights for policy 1, policy_version 1892608 (0.0010) [2023-12-27 05:09:06,836][105620] Updated weights for policy 1, policy_version 1892618 (0.0010) [2023-12-27 05:09:06,989][105692] Updated weights for policy 0, policy_version 1888057 (0.0009) [2023-12-27 05:09:07,038][105692] Updated weights for policy 0, policy_version 1888067 (0.0007) [2023-12-27 05:09:07,097][105692] Updated weights for policy 0, policy_version 1888077 (0.0005) [2023-12-27 05:09:07,595][105620] Updated weights for policy 1, policy_version 1892628 (0.0008) [2023-12-27 05:09:07,641][105620] Updated weights for policy 1, policy_version 1892638 (0.0005) [2023-12-27 05:09:07,689][105620] Updated weights for policy 1, policy_version 1892648 (0.0005) [2023-12-27 05:09:07,807][105692] Updated weights for policy 0, policy_version 1888087 (0.0007) [2023-12-27 05:09:07,867][105692] Updated weights for policy 0, policy_version 1888097 (0.0005) [2023-12-27 05:09:07,928][105692] Updated weights for policy 0, policy_version 1888107 (0.0007) [2023-12-27 05:09:08,374][105620] Updated weights for policy 1, policy_version 1892658 (0.0006) [2023-12-27 05:09:08,437][105620] Updated weights for policy 1, policy_version 1892668 (0.0011) [2023-12-27 05:09:08,505][105620] Updated weights for policy 1, policy_version 1892678 (0.0010) [2023-12-27 05:09:08,569][105620] Updated weights for policy 1, policy_version 1892688 (0.0010) [2023-12-27 05:09:08,615][105692] Updated weights for policy 0, policy_version 1888117 (0.0010) [2023-12-27 05:09:08,673][105692] Updated weights for policy 0, policy_version 1888127 (0.0011) [2023-12-27 05:09:08,732][105692] Updated weights for policy 0, policy_version 1888137 (0.0010) [2023-12-27 05:09:09,305][105620] Updated weights for policy 1, policy_version 1892698 (0.0010) [2023-12-27 05:09:09,374][105620] Updated weights for policy 1, policy_version 1892708 (0.0010) [2023-12-27 05:09:09,445][105620] Updated weights for policy 1, policy_version 1892718 (0.0009) [2023-12-27 05:09:09,465][105692] Updated weights for policy 0, policy_version 1888147 (0.0010) [2023-12-27 05:09:09,521][105692] Updated weights for policy 0, policy_version 1888157 (0.0009) [2023-12-27 05:09:09,579][105692] Updated weights for policy 0, policy_version 1888167 (0.0008) [2023-12-27 05:09:10,222][105620] Updated weights for policy 1, policy_version 1892728 (0.0011) [2023-12-27 05:09:10,291][105620] Updated weights for policy 1, policy_version 1892738 (0.0010) [2023-12-27 05:09:10,357][105620] Updated weights for policy 1, policy_version 1892748 (0.0010) [2023-12-27 05:09:10,367][105692] Updated weights for policy 0, policy_version 1888177 (0.0009) [2023-12-27 05:09:10,435][105692] Updated weights for policy 0, policy_version 1888187 (0.0009) [2023-12-27 05:09:10,507][105692] Updated weights for policy 0, policy_version 1888197 (0.0009) [2023-12-27 05:09:10,575][105692] Updated weights for policy 0, policy_version 1888207 (0.0008) [2023-12-27 05:09:11,038][105620] Updated weights for policy 1, policy_version 1892758 (0.0011) [2023-12-27 05:09:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19114.6, 300 sec: 19466.4). Total num frames: 968065024. Throughput: 0: 9543.8, 1: 9705.1. Samples: 968077216. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:09:11,063][104569] Avg episode reward: [(0, '8263.933'), (1, '9345.361')] [2023-12-27 05:09:11,104][105620] Updated weights for policy 1, policy_version 1892768 (0.0010) [2023-12-27 05:09:11,166][105620] Updated weights for policy 1, policy_version 1892778 (0.0009) [2023-12-27 05:09:11,342][105692] Updated weights for policy 0, policy_version 1888217 (0.0009) [2023-12-27 05:09:11,418][105692] Updated weights for policy 0, policy_version 1888227 (0.0009) [2023-12-27 05:09:11,487][105692] Updated weights for policy 0, policy_version 1888237 (0.0010) [2023-12-27 05:09:11,982][105620] Updated weights for policy 1, policy_version 1892788 (0.0009) [2023-12-27 05:09:12,037][105620] Updated weights for policy 1, policy_version 1892798 (0.0008) [2023-12-27 05:09:12,110][105620] Updated weights for policy 1, policy_version 1892808 (0.0009) [2023-12-27 05:09:12,234][105692] Updated weights for policy 0, policy_version 1888247 (0.0008) [2023-12-27 05:09:12,298][105692] Updated weights for policy 0, policy_version 1888257 (0.0008) [2023-12-27 05:09:12,363][105692] Updated weights for policy 0, policy_version 1888267 (0.0008) [2023-12-27 05:09:12,861][105620] Updated weights for policy 1, policy_version 1892818 (0.0008) [2023-12-27 05:09:12,927][105620] Updated weights for policy 1, policy_version 1892828 (0.0005) [2023-12-27 05:09:12,989][105620] Updated weights for policy 1, policy_version 1892838 (0.0009) [2023-12-27 05:09:13,046][105620] Updated weights for policy 1, policy_version 1892848 (0.0005) [2023-12-27 05:09:13,155][105692] Updated weights for policy 0, policy_version 1888277 (0.0007) [2023-12-27 05:09:13,215][105692] Updated weights for policy 0, policy_version 1888287 (0.0009) [2023-12-27 05:09:13,268][105692] Updated weights for policy 0, policy_version 1888297 (0.0010) [2023-12-27 05:09:13,601][105620] Updated weights for policy 1, policy_version 1892858 (0.0005) [2023-12-27 05:09:13,652][105620] Updated weights for policy 1, policy_version 1892868 (0.0005) [2023-12-27 05:09:13,705][105620] Updated weights for policy 1, policy_version 1892878 (0.0007) [2023-12-27 05:09:13,828][105692] Updated weights for policy 0, policy_version 1888307 (0.0006) [2023-12-27 05:09:13,876][105692] Updated weights for policy 0, policy_version 1888317 (0.0010) [2023-12-27 05:09:13,929][105692] Updated weights for policy 0, policy_version 1888327 (0.0010) [2023-12-27 05:09:14,305][105620] Updated weights for policy 1, policy_version 1892888 (0.0008) [2023-12-27 05:09:14,349][105620] Updated weights for policy 1, policy_version 1892898 (0.0005) [2023-12-27 05:09:14,410][105620] Updated weights for policy 1, policy_version 1892908 (0.0008) [2023-12-27 05:09:14,506][105692] Updated weights for policy 0, policy_version 1888337 (0.0005) [2023-12-27 05:09:14,572][105692] Updated weights for policy 0, policy_version 1888347 (0.0007) [2023-12-27 05:09:14,633][105692] Updated weights for policy 0, policy_version 1888357 (0.0010) [2023-12-27 05:09:14,697][105692] Updated weights for policy 0, policy_version 1888367 (0.0010) [2023-12-27 05:09:15,111][105620] Updated weights for policy 1, policy_version 1892918 (0.0011) [2023-12-27 05:09:15,176][105620] Updated weights for policy 1, policy_version 1892928 (0.0011) [2023-12-27 05:09:15,239][105620] Updated weights for policy 1, policy_version 1892938 (0.0011) [2023-12-27 05:09:15,427][105692] Updated weights for policy 0, policy_version 1888377 (0.0010) [2023-12-27 05:09:15,523][105692] Updated weights for policy 0, policy_version 1888387 (0.0014) [2023-12-27 05:09:15,575][105692] Updated weights for policy 0, policy_version 1888397 (0.0010) [2023-12-27 05:09:15,956][105620] Updated weights for policy 1, policy_version 1892948 (0.0009) [2023-12-27 05:09:16,026][105620] Updated weights for policy 1, policy_version 1892958 (0.0006) [2023-12-27 05:09:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19114.6, 300 sec: 19438.6). Total num frames: 968163328. Throughput: 0: 9575.1, 1: 9654.7. Samples: 968134916. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:09:16,063][104569] Avg episode reward: [(0, '8447.633'), (1, '9345.396')] [2023-12-27 05:09:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001888400_483500032.pth... [2023-12-27 05:09:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001887280_483213312.pth [2023-12-27 05:09:16,097][105620] Updated weights for policy 1, policy_version 1892968 (0.0006) [2023-12-27 05:09:16,137][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001892976_484671488.pth... [2023-12-27 05:09:16,140][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001891824_484376576.pth [2023-12-27 05:09:16,233][105692] Updated weights for policy 0, policy_version 1888407 (0.0007) [2023-12-27 05:09:16,289][105692] Updated weights for policy 0, policy_version 1888417 (0.0005) [2023-12-27 05:09:16,340][105692] Updated weights for policy 0, policy_version 1888427 (0.0006) [2023-12-27 05:09:16,732][105620] Updated weights for policy 1, policy_version 1892978 (0.0006) [2023-12-27 05:09:16,789][105620] Updated weights for policy 1, policy_version 1892988 (0.0006) [2023-12-27 05:09:16,858][105620] Updated weights for policy 1, policy_version 1892998 (0.0009) [2023-12-27 05:09:16,914][105620] Updated weights for policy 1, policy_version 1893008 (0.0010) [2023-12-27 05:09:16,915][105692] Updated weights for policy 0, policy_version 1888437 (0.0006) [2023-12-27 05:09:16,977][105692] Updated weights for policy 0, policy_version 1888447 (0.0007) [2023-12-27 05:09:17,042][105692] Updated weights for policy 0, policy_version 1888457 (0.0008) [2023-12-27 05:09:17,472][105620] Updated weights for policy 1, policy_version 1893018 (0.0006) [2023-12-27 05:09:17,519][105620] Updated weights for policy 1, policy_version 1893028 (0.0005) [2023-12-27 05:09:17,574][105620] Updated weights for policy 1, policy_version 1893038 (0.0007) [2023-12-27 05:09:17,754][105692] Updated weights for policy 0, policy_version 1888467 (0.0010) [2023-12-27 05:09:17,819][105692] Updated weights for policy 0, policy_version 1888477 (0.0010) [2023-12-27 05:09:17,888][105692] Updated weights for policy 0, policy_version 1888487 (0.0010) [2023-12-27 05:09:18,162][105620] Updated weights for policy 1, policy_version 1893048 (0.0006) [2023-12-27 05:09:18,216][105620] Updated weights for policy 1, policy_version 1893058 (0.0005) [2023-12-27 05:09:18,269][105620] Updated weights for policy 1, policy_version 1893068 (0.0006) [2023-12-27 05:09:18,679][105692] Updated weights for policy 0, policy_version 1888497 (0.0010) [2023-12-27 05:09:18,742][105692] Updated weights for policy 0, policy_version 1888507 (0.0009) [2023-12-27 05:09:18,805][105692] Updated weights for policy 0, policy_version 1888517 (0.0008) [2023-12-27 05:09:18,863][105620] Updated weights for policy 1, policy_version 1893078 (0.0006) [2023-12-27 05:09:18,867][105692] Updated weights for policy 0, policy_version 1888527 (0.0008) [2023-12-27 05:09:18,933][105620] Updated weights for policy 1, policy_version 1893088 (0.0006) [2023-12-27 05:09:18,985][105620] Updated weights for policy 1, policy_version 1893098 (0.0009) [2023-12-27 05:09:19,603][105692] Updated weights for policy 0, policy_version 1888537 (0.0008) [2023-12-27 05:09:19,651][105692] Updated weights for policy 0, policy_version 1888547 (0.0008) [2023-12-27 05:09:19,697][105692] Updated weights for policy 0, policy_version 1888557 (0.0009) [2023-12-27 05:09:19,748][105620] Updated weights for policy 1, policy_version 1893108 (0.0008) [2023-12-27 05:09:19,811][105620] Updated weights for policy 1, policy_version 1893118 (0.0009) [2023-12-27 05:09:19,882][105620] Updated weights for policy 1, policy_version 1893128 (0.0010) [2023-12-27 05:09:20,470][105692] Updated weights for policy 0, policy_version 1888567 (0.0010) [2023-12-27 05:09:20,522][105692] Updated weights for policy 0, policy_version 1888577 (0.0009) [2023-12-27 05:09:20,588][105692] Updated weights for policy 0, policy_version 1888587 (0.0009) [2023-12-27 05:09:20,635][105620] Updated weights for policy 1, policy_version 1893138 (0.0008) [2023-12-27 05:09:20,697][105620] Updated weights for policy 1, policy_version 1893148 (0.0009) [2023-12-27 05:09:20,764][105620] Updated weights for policy 1, policy_version 1893158 (0.0010) [2023-12-27 05:09:20,823][105620] Updated weights for policy 1, policy_version 1893168 (0.0010) [2023-12-27 05:09:21,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 968269824. Throughput: 0: 9699.6, 1: 9690.1. Samples: 968258956. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:09:21,062][104569] Avg episode reward: [(0, '8447.851'), (1, '9345.401')] [2023-12-27 05:09:21,364][105692] Updated weights for policy 0, policy_version 1888597 (0.0009) [2023-12-27 05:09:21,439][105692] Updated weights for policy 0, policy_version 1888607 (0.0007) [2023-12-27 05:09:21,510][105692] Updated weights for policy 0, policy_version 1888617 (0.0007) [2023-12-27 05:09:21,601][105620] Updated weights for policy 1, policy_version 1893178 (0.0007) [2023-12-27 05:09:21,673][105620] Updated weights for policy 1, policy_version 1893188 (0.0007) [2023-12-27 05:09:21,730][105620] Updated weights for policy 1, policy_version 1893198 (0.0010) [2023-12-27 05:09:22,215][105692] Updated weights for policy 0, policy_version 1888627 (0.0009) [2023-12-27 05:09:22,277][105692] Updated weights for policy 0, policy_version 1888637 (0.0009) [2023-12-27 05:09:22,329][105692] Updated weights for policy 0, policy_version 1888647 (0.0009) [2023-12-27 05:09:22,496][105620] Updated weights for policy 1, policy_version 1893208 (0.0007) [2023-12-27 05:09:22,559][105620] Updated weights for policy 1, policy_version 1893218 (0.0008) [2023-12-27 05:09:22,627][105620] Updated weights for policy 1, policy_version 1893228 (0.0007) [2023-12-27 05:09:23,157][105692] Updated weights for policy 0, policy_version 1888657 (0.0009) [2023-12-27 05:09:23,215][105692] Updated weights for policy 0, policy_version 1888667 (0.0009) [2023-12-27 05:09:23,280][105692] Updated weights for policy 0, policy_version 1888677 (0.0009) [2023-12-27 05:09:23,309][105620] Updated weights for policy 1, policy_version 1893238 (0.0009) [2023-12-27 05:09:23,335][105692] Updated weights for policy 0, policy_version 1888687 (0.0007) [2023-12-27 05:09:23,369][105620] Updated weights for policy 1, policy_version 1893248 (0.0006) [2023-12-27 05:09:23,431][105620] Updated weights for policy 1, policy_version 1893258 (0.0010) [2023-12-27 05:09:23,992][105620] Updated weights for policy 1, policy_version 1893268 (0.0005) [2023-12-27 05:09:24,065][105620] Updated weights for policy 1, policy_version 1893278 (0.0005) [2023-12-27 05:09:24,130][105620] Updated weights for policy 1, policy_version 1893288 (0.0005) [2023-12-27 05:09:24,188][105692] Updated weights for policy 0, policy_version 1888697 (0.0007) [2023-12-27 05:09:24,247][105692] Updated weights for policy 0, policy_version 1888707 (0.0008) [2023-12-27 05:09:24,309][105692] Updated weights for policy 0, policy_version 1888717 (0.0008) [2023-12-27 05:09:24,707][105620] Updated weights for policy 1, policy_version 1893298 (0.0007) [2023-12-27 05:09:24,773][105620] Updated weights for policy 1, policy_version 1893308 (0.0005) [2023-12-27 05:09:24,832][105620] Updated weights for policy 1, policy_version 1893318 (0.0007) [2023-12-27 05:09:25,067][105692] Updated weights for policy 0, policy_version 1888727 (0.0008) [2023-12-27 05:09:25,137][105692] Updated weights for policy 0, policy_version 1888737 (0.0009) [2023-12-27 05:09:25,191][105692] Updated weights for policy 0, policy_version 1888747 (0.0010) [2023-12-27 05:09:25,481][105620] Updated weights for policy 1, policy_version 1893329 (0.0008) [2023-12-27 05:09:25,534][105620] Updated weights for policy 1, policy_version 1893339 (0.0008) [2023-12-27 05:09:25,588][105620] Updated weights for policy 1, policy_version 1893349 (0.0007) [2023-12-27 05:09:25,650][105620] Updated weights for policy 1, policy_version 1893359 (0.0008) [2023-12-27 05:09:25,811][105692] Updated weights for policy 0, policy_version 1888757 (0.0008) [2023-12-27 05:09:25,876][105692] Updated weights for policy 0, policy_version 1888767 (0.0005) [2023-12-27 05:09:25,933][105692] Updated weights for policy 0, policy_version 1888777 (0.0006) [2023-12-27 05:09:26,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 968368128. Throughput: 0: 9583.9, 1: 9729.8. Samples: 968373816. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:09:26,062][104569] Avg episode reward: [(0, '8808.682'), (1, '9345.425')] [2023-12-27 05:09:26,392][105620] Updated weights for policy 1, policy_version 1893369 (0.0006) [2023-12-27 05:09:26,453][105620] Updated weights for policy 1, policy_version 1893379 (0.0010) [2023-12-27 05:09:26,501][105620] Updated weights for policy 1, policy_version 1893389 (0.0010) [2023-12-27 05:09:26,511][105692] Updated weights for policy 0, policy_version 1888787 (0.0007) [2023-12-27 05:09:26,572][105692] Updated weights for policy 0, policy_version 1888797 (0.0010) [2023-12-27 05:09:26,616][105692] Updated weights for policy 0, policy_version 1888807 (0.0010) [2023-12-27 05:09:27,115][105620] Updated weights for policy 1, policy_version 1893399 (0.0011) [2023-12-27 05:09:27,174][105620] Updated weights for policy 1, policy_version 1893409 (0.0011) [2023-12-27 05:09:27,222][105620] Updated weights for policy 1, policy_version 1893419 (0.0011) [2023-12-27 05:09:27,237][105692] Updated weights for policy 0, policy_version 1888817 (0.0010) [2023-12-27 05:09:27,298][105692] Updated weights for policy 0, policy_version 1888827 (0.0007) [2023-12-27 05:09:27,345][105692] Updated weights for policy 0, policy_version 1888837 (0.0006) [2023-12-27 05:09:27,391][105692] Updated weights for policy 0, policy_version 1888847 (0.0005) [2023-12-27 05:09:27,889][105620] Updated weights for policy 1, policy_version 1893429 (0.0008) [2023-12-27 05:09:27,937][105620] Updated weights for policy 1, policy_version 1893439 (0.0005) [2023-12-27 05:09:27,985][105692] Updated weights for policy 0, policy_version 1888857 (0.0010) [2023-12-27 05:09:27,986][105620] Updated weights for policy 1, policy_version 1893449 (0.0005) [2023-12-27 05:09:28,037][105692] Updated weights for policy 0, policy_version 1888867 (0.0010) [2023-12-27 05:09:28,094][105692] Updated weights for policy 0, policy_version 1888877 (0.0010) [2023-12-27 05:09:28,722][105692] Updated weights for policy 0, policy_version 1888887 (0.0011) [2023-12-27 05:09:28,724][105620] Updated weights for policy 1, policy_version 1893459 (0.0009) [2023-12-27 05:09:28,777][105620] Updated weights for policy 1, policy_version 1893469 (0.0006) [2023-12-27 05:09:28,782][105692] Updated weights for policy 0, policy_version 1888897 (0.0011) [2023-12-27 05:09:28,838][105692] Updated weights for policy 0, policy_version 1888907 (0.0011) [2023-12-27 05:09:28,841][105620] Updated weights for policy 1, policy_version 1893479 (0.0006) [2023-12-27 05:09:29,472][105620] Updated weights for policy 1, policy_version 1893489 (0.0007) [2023-12-27 05:09:29,527][105620] Updated weights for policy 1, policy_version 1893499 (0.0008) [2023-12-27 05:09:29,575][105620] Updated weights for policy 1, policy_version 1893509 (0.0008) [2023-12-27 05:09:29,592][105692] Updated weights for policy 0, policy_version 1888917 (0.0010) [2023-12-27 05:09:29,626][105620] Updated weights for policy 1, policy_version 1893519 (0.0005) [2023-12-27 05:09:29,647][105692] Updated weights for policy 0, policy_version 1888927 (0.0010) [2023-12-27 05:09:29,706][105692] Updated weights for policy 0, policy_version 1888937 (0.0011) [2023-12-27 05:09:30,382][105620] Updated weights for policy 1, policy_version 1893529 (0.0008) [2023-12-27 05:09:30,416][105692] Updated weights for policy 0, policy_version 1888947 (0.0011) [2023-12-27 05:09:30,438][105620] Updated weights for policy 1, policy_version 1893539 (0.0006) [2023-12-27 05:09:30,477][105692] Updated weights for policy 0, policy_version 1888957 (0.0011) [2023-12-27 05:09:30,495][105620] Updated weights for policy 1, policy_version 1893549 (0.0005) [2023-12-27 05:09:30,534][105692] Updated weights for policy 0, policy_version 1888967 (0.0008) [2023-12-27 05:09:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 968466432. Throughput: 0: 9672.0, 1: 9789.7. Samples: 968438096. Policy #0 lag: (min: 5.0, avg: 16.9, max: 37.0) [2023-12-27 05:09:31,062][104569] Avg episode reward: [(0, '8900.116'), (1, '9253.441')] [2023-12-27 05:09:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001888976_483647488.pth... [2023-12-27 05:09:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001887856_483360768.pth [2023-12-27 05:09:31,092][105620] Updated weights for policy 1, policy_version 1893559 (0.0008) [2023-12-27 05:09:31,147][105620] Updated weights for policy 1, policy_version 1893569 (0.0008) [2023-12-27 05:09:31,198][105620] Updated weights for policy 1, policy_version 1893579 (0.0009) [2023-12-27 05:09:31,218][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001893584_484827136.pth... [2023-12-27 05:09:31,222][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001892400_484524032.pth [2023-12-27 05:09:31,298][105692] Updated weights for policy 0, policy_version 1888977 (0.0011) [2023-12-27 05:09:31,363][105692] Updated weights for policy 0, policy_version 1888987 (0.0009) [2023-12-27 05:09:31,426][105692] Updated weights for policy 0, policy_version 1888997 (0.0010) [2023-12-27 05:09:31,474][105692] Updated weights for policy 0, policy_version 1889007 (0.0009) [2023-12-27 05:09:31,974][105620] Updated weights for policy 1, policy_version 1893589 (0.0009) [2023-12-27 05:09:32,021][105620] Updated weights for policy 1, policy_version 1893599 (0.0009) [2023-12-27 05:09:32,074][105620] Updated weights for policy 1, policy_version 1893609 (0.0008) [2023-12-27 05:09:32,256][105692] Updated weights for policy 0, policy_version 1889017 (0.0009) [2023-12-27 05:09:32,318][105692] Updated weights for policy 0, policy_version 1889027 (0.0009) [2023-12-27 05:09:32,382][105692] Updated weights for policy 0, policy_version 1889037 (0.0009) [2023-12-27 05:09:32,913][105620] Updated weights for policy 1, policy_version 1893619 (0.0009) [2023-12-27 05:09:32,959][105692] Updated weights for policy 0, policy_version 1889047 (0.0007) [2023-12-27 05:09:32,965][105620] Updated weights for policy 1, policy_version 1893629 (0.0008) [2023-12-27 05:09:33,007][105692] Updated weights for policy 0, policy_version 1889057 (0.0005) [2023-12-27 05:09:33,012][105620] Updated weights for policy 1, policy_version 1893639 (0.0009) [2023-12-27 05:09:33,056][105692] Updated weights for policy 0, policy_version 1889067 (0.0005) [2023-12-27 05:09:33,590][105692] Updated weights for policy 0, policy_version 1889077 (0.0007) [2023-12-27 05:09:33,645][105692] Updated weights for policy 0, policy_version 1889087 (0.0010) [2023-12-27 05:09:33,695][105692] Updated weights for policy 0, policy_version 1889097 (0.0010) [2023-12-27 05:09:33,831][105620] Updated weights for policy 1, policy_version 1893649 (0.0009) [2023-12-27 05:09:33,881][105620] Updated weights for policy 1, policy_version 1893659 (0.0009) [2023-12-27 05:09:33,935][105620] Updated weights for policy 1, policy_version 1893671 (0.0010) [2023-12-27 05:09:34,290][105692] Updated weights for policy 0, policy_version 1889107 (0.0010) [2023-12-27 05:09:34,347][105692] Updated weights for policy 0, policy_version 1889117 (0.0008) [2023-12-27 05:09:34,412][105692] Updated weights for policy 0, policy_version 1889127 (0.0008) [2023-12-27 05:09:34,755][105620] Updated weights for policy 1, policy_version 1893682 (0.0009) [2023-12-27 05:09:34,818][105620] Updated weights for policy 1, policy_version 1893692 (0.0009) [2023-12-27 05:09:34,876][105620] Updated weights for policy 1, policy_version 1893702 (0.0009) [2023-12-27 05:09:34,928][105620] Updated weights for policy 1, policy_version 1893712 (0.0006) [2023-12-27 05:09:35,110][105692] Updated weights for policy 0, policy_version 1889137 (0.0010) [2023-12-27 05:09:35,156][105692] Updated weights for policy 0, policy_version 1889147 (0.0005) [2023-12-27 05:09:35,203][105692] Updated weights for policy 0, policy_version 1889157 (0.0005) [2023-12-27 05:09:35,252][105692] Updated weights for policy 0, policy_version 1889167 (0.0005) [2023-12-27 05:09:35,550][105620] Updated weights for policy 1, policy_version 1893722 (0.0005) [2023-12-27 05:09:35,601][105620] Updated weights for policy 1, policy_version 1893732 (0.0006) [2023-12-27 05:09:35,665][105620] Updated weights for policy 1, policy_version 1893742 (0.0009) [2023-12-27 05:09:35,820][105692] Updated weights for policy 0, policy_version 1889177 (0.0005) [2023-12-27 05:09:35,889][105692] Updated weights for policy 0, policy_version 1889187 (0.0005) [2023-12-27 05:09:35,957][105692] Updated weights for policy 0, policy_version 1889197 (0.0006) [2023-12-27 05:09:36,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 968572928. Throughput: 0: 9717.7, 1: 9739.3. Samples: 968555296. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:09:36,063][104569] Avg episode reward: [(0, '8537.331'), (1, '9253.447')] [2023-12-27 05:09:36,404][105620] Updated weights for policy 1, policy_version 1893752 (0.0008) [2023-12-27 05:09:36,470][105620] Updated weights for policy 1, policy_version 1893762 (0.0009) [2023-12-27 05:09:36,530][105620] Updated weights for policy 1, policy_version 1893772 (0.0008) [2023-12-27 05:09:36,551][105692] Updated weights for policy 0, policy_version 1889207 (0.0009) [2023-12-27 05:09:36,608][105692] Updated weights for policy 0, policy_version 1889217 (0.0006) [2023-12-27 05:09:36,677][105692] Updated weights for policy 0, policy_version 1889227 (0.0006) [2023-12-27 05:09:37,213][105620] Updated weights for policy 1, policy_version 1893782 (0.0007) [2023-12-27 05:09:37,269][105620] Updated weights for policy 1, policy_version 1893792 (0.0006) [2023-12-27 05:09:37,324][105620] Updated weights for policy 1, policy_version 1893802 (0.0006) [2023-12-27 05:09:37,342][105692] Updated weights for policy 0, policy_version 1889237 (0.0008) [2023-12-27 05:09:37,401][105692] Updated weights for policy 0, policy_version 1889247 (0.0009) [2023-12-27 05:09:37,457][105692] Updated weights for policy 0, policy_version 1889257 (0.0005) [2023-12-27 05:09:37,997][105620] Updated weights for policy 1, policy_version 1893812 (0.0007) [2023-12-27 05:09:38,067][105620] Updated weights for policy 1, policy_version 1893822 (0.0006) [2023-12-27 05:09:38,131][105620] Updated weights for policy 1, policy_version 1893832 (0.0007) [2023-12-27 05:09:38,143][105692] Updated weights for policy 0, policy_version 1889267 (0.0007) [2023-12-27 05:09:38,197][105692] Updated weights for policy 0, policy_version 1889277 (0.0010) [2023-12-27 05:09:38,252][105692] Updated weights for policy 0, policy_version 1889287 (0.0010) [2023-12-27 05:09:38,682][105620] Updated weights for policy 1, policy_version 1893842 (0.0008) [2023-12-27 05:09:38,741][105620] Updated weights for policy 1, policy_version 1893852 (0.0008) [2023-12-27 05:09:38,789][105620] Updated weights for policy 1, policy_version 1893862 (0.0008) [2023-12-27 05:09:38,845][105620] Updated weights for policy 1, policy_version 1893872 (0.0005) [2023-12-27 05:09:39,027][105692] Updated weights for policy 0, policy_version 1889297 (0.0010) [2023-12-27 05:09:39,082][105692] Updated weights for policy 0, policy_version 1889307 (0.0010) [2023-12-27 05:09:39,143][105692] Updated weights for policy 0, policy_version 1889317 (0.0010) [2023-12-27 05:09:39,208][105692] Updated weights for policy 0, policy_version 1889327 (0.0010) [2023-12-27 05:09:39,619][105620] Updated weights for policy 1, policy_version 1893882 (0.0008) [2023-12-27 05:09:39,679][105620] Updated weights for policy 1, policy_version 1893892 (0.0007) [2023-12-27 05:09:39,731][105620] Updated weights for policy 1, policy_version 1893902 (0.0009) [2023-12-27 05:09:39,933][105692] Updated weights for policy 0, policy_version 1889337 (0.0009) [2023-12-27 05:09:39,995][105692] Updated weights for policy 0, policy_version 1889347 (0.0009) [2023-12-27 05:09:40,054][105692] Updated weights for policy 0, policy_version 1889357 (0.0009) [2023-12-27 05:09:40,424][105620] Updated weights for policy 1, policy_version 1893912 (0.0007) [2023-12-27 05:09:40,483][105620] Updated weights for policy 1, policy_version 1893922 (0.0006) [2023-12-27 05:09:40,546][105620] Updated weights for policy 1, policy_version 1893932 (0.0006) [2023-12-27 05:09:40,950][105692] Updated weights for policy 0, policy_version 1889367 (0.0010) [2023-12-27 05:09:41,007][105692] Updated weights for policy 0, policy_version 1889377 (0.0006) [2023-12-27 05:09:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 968663040. Throughput: 0: 9824.4, 1: 9842.0. Samples: 968678940. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:09:41,062][104569] Avg episode reward: [(0, '8262.435'), (1, '9345.521')] [2023-12-27 05:09:41,080][105692] Updated weights for policy 0, policy_version 1889387 (0.0009) [2023-12-27 05:09:41,175][105620] Updated weights for policy 1, policy_version 1893942 (0.0007) [2023-12-27 05:09:41,234][105620] Updated weights for policy 1, policy_version 1893952 (0.0008) [2023-12-27 05:09:41,295][105620] Updated weights for policy 1, policy_version 1893962 (0.0008) [2023-12-27 05:09:41,884][105692] Updated weights for policy 0, policy_version 1889397 (0.0009) [2023-12-27 05:09:41,936][105692] Updated weights for policy 0, policy_version 1889407 (0.0009) [2023-12-27 05:09:41,999][105692] Updated weights for policy 0, policy_version 1889417 (0.0010) [2023-12-27 05:09:42,050][105620] Updated weights for policy 1, policy_version 1893972 (0.0007) [2023-12-27 05:09:42,103][105620] Updated weights for policy 1, policy_version 1893982 (0.0008) [2023-12-27 05:09:42,163][105620] Updated weights for policy 1, policy_version 1893992 (0.0009) [2023-12-27 05:09:42,759][105692] Updated weights for policy 0, policy_version 1889427 (0.0010) [2023-12-27 05:09:42,826][105692] Updated weights for policy 0, policy_version 1889437 (0.0008) [2023-12-27 05:09:42,847][105620] Updated weights for policy 1, policy_version 1894002 (0.0008) [2023-12-27 05:09:42,889][105692] Updated weights for policy 0, policy_version 1889447 (0.0011) [2023-12-27 05:09:42,903][105620] Updated weights for policy 1, policy_version 1894012 (0.0007) [2023-12-27 05:09:42,960][105620] Updated weights for policy 1, policy_version 1894022 (0.0006) [2023-12-27 05:09:43,016][105620] Updated weights for policy 1, policy_version 1894032 (0.0008) [2023-12-27 05:09:43,557][105692] Updated weights for policy 0, policy_version 1889457 (0.0010) [2023-12-27 05:09:43,619][105692] Updated weights for policy 0, policy_version 1889467 (0.0005) [2023-12-27 05:09:43,677][105692] Updated weights for policy 0, policy_version 1889477 (0.0006) [2023-12-27 05:09:43,720][105692] Updated weights for policy 0, policy_version 1889487 (0.0005) [2023-12-27 05:09:43,749][105620] Updated weights for policy 1, policy_version 1894042 (0.0009) [2023-12-27 05:09:43,803][105620] Updated weights for policy 1, policy_version 1894052 (0.0009) [2023-12-27 05:09:43,857][105620] Updated weights for policy 1, policy_version 1894062 (0.0009) [2023-12-27 05:09:44,315][105692] Updated weights for policy 0, policy_version 1889497 (0.0009) [2023-12-27 05:09:44,366][105692] Updated weights for policy 0, policy_version 1889507 (0.0009) [2023-12-27 05:09:44,415][105692] Updated weights for policy 0, policy_version 1889517 (0.0008) [2023-12-27 05:09:44,666][105620] Updated weights for policy 1, policy_version 1894072 (0.0008) [2023-12-27 05:09:44,718][105620] Updated weights for policy 1, policy_version 1894082 (0.0009) [2023-12-27 05:09:44,775][105620] Updated weights for policy 1, policy_version 1894092 (0.0007) [2023-12-27 05:09:45,194][105692] Updated weights for policy 0, policy_version 1889527 (0.0009) [2023-12-27 05:09:45,242][105692] Updated weights for policy 0, policy_version 1889537 (0.0009) [2023-12-27 05:09:45,290][105692] Updated weights for policy 0, policy_version 1889547 (0.0009) [2023-12-27 05:09:45,482][105620] Updated weights for policy 1, policy_version 1894102 (0.0009) [2023-12-27 05:09:45,542][105620] Updated weights for policy 1, policy_version 1894112 (0.0009) [2023-12-27 05:09:45,612][105620] Updated weights for policy 1, policy_version 1894122 (0.0009) [2023-12-27 05:09:46,058][105692] Updated weights for policy 0, policy_version 1889557 (0.0010) [2023-12-27 05:09:46,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 968761344. Throughput: 0: 9788.4, 1: 9890.0. Samples: 968735228. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:09:46,062][104569] Avg episode reward: [(0, '8261.421'), (1, '9345.575')] [2023-12-27 05:09:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001894128_484966400.pth... [2023-12-27 05:09:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001892976_484671488.pth [2023-12-27 05:09:46,114][105692] Updated weights for policy 0, policy_version 1889567 (0.0009) [2023-12-27 05:09:46,173][105692] Updated weights for policy 0, policy_version 1889577 (0.0009) [2023-12-27 05:09:46,206][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001889584_483803136.pth... [2023-12-27 05:09:46,209][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001888400_483500032.pth [2023-12-27 05:09:46,309][105620] Updated weights for policy 1, policy_version 1894132 (0.0009) [2023-12-27 05:09:46,368][105620] Updated weights for policy 1, policy_version 1894142 (0.0009) [2023-12-27 05:09:46,419][105620] Updated weights for policy 1, policy_version 1894152 (0.0009) [2023-12-27 05:09:46,877][105692] Updated weights for policy 0, policy_version 1889587 (0.0008) [2023-12-27 05:09:46,930][105692] Updated weights for policy 0, policy_version 1889597 (0.0007) [2023-12-27 05:09:46,986][105692] Updated weights for policy 0, policy_version 1889607 (0.0009) [2023-12-27 05:09:47,156][105620] Updated weights for policy 1, policy_version 1894162 (0.0009) [2023-12-27 05:09:47,220][105620] Updated weights for policy 1, policy_version 1894172 (0.0006) [2023-12-27 05:09:47,272][105620] Updated weights for policy 1, policy_version 1894182 (0.0006) [2023-12-27 05:09:47,323][105620] Updated weights for policy 1, policy_version 1894192 (0.0008) [2023-12-27 05:09:47,701][105692] Updated weights for policy 0, policy_version 1889617 (0.0009) [2023-12-27 05:09:47,748][105692] Updated weights for policy 0, policy_version 1889627 (0.0008) [2023-12-27 05:09:47,794][105692] Updated weights for policy 0, policy_version 1889637 (0.0009) [2023-12-27 05:09:47,841][105692] Updated weights for policy 0, policy_version 1889647 (0.0009) [2023-12-27 05:09:48,014][105620] Updated weights for policy 1, policy_version 1894202 (0.0009) [2023-12-27 05:09:48,064][105620] Updated weights for policy 1, policy_version 1894212 (0.0007) [2023-12-27 05:09:48,115][105620] Updated weights for policy 1, policy_version 1894222 (0.0008) [2023-12-27 05:09:48,679][105692] Updated weights for policy 0, policy_version 1889657 (0.0008) [2023-12-27 05:09:48,743][105692] Updated weights for policy 0, policy_version 1889667 (0.0009) [2023-12-27 05:09:48,800][105692] Updated weights for policy 0, policy_version 1889677 (0.0009) [2023-12-27 05:09:48,827][105620] Updated weights for policy 1, policy_version 1894232 (0.0007) [2023-12-27 05:09:48,883][105620] Updated weights for policy 1, policy_version 1894242 (0.0009) [2023-12-27 05:09:48,933][105620] Updated weights for policy 1, policy_version 1894252 (0.0008) [2023-12-27 05:09:49,541][105692] Updated weights for policy 0, policy_version 1889687 (0.0007) [2023-12-27 05:09:49,597][105692] Updated weights for policy 0, policy_version 1889697 (0.0008) [2023-12-27 05:09:49,652][105692] Updated weights for policy 0, policy_version 1889707 (0.0006) [2023-12-27 05:09:49,739][105620] Updated weights for policy 1, policy_version 1894262 (0.0009) [2023-12-27 05:09:49,800][105620] Updated weights for policy 1, policy_version 1894272 (0.0009) [2023-12-27 05:09:49,867][105620] Updated weights for policy 1, policy_version 1894282 (0.0009) [2023-12-27 05:09:50,286][105692] Updated weights for policy 0, policy_version 1889717 (0.0006) [2023-12-27 05:09:50,347][105692] Updated weights for policy 0, policy_version 1889727 (0.0005) [2023-12-27 05:09:50,410][105692] Updated weights for policy 0, policy_version 1889737 (0.0006) [2023-12-27 05:09:50,715][105620] Updated weights for policy 1, policy_version 1894292 (0.0010) [2023-12-27 05:09:50,776][105620] Updated weights for policy 1, policy_version 1894302 (0.0009) [2023-12-27 05:09:50,836][105620] Updated weights for policy 1, policy_version 1894312 (0.0009) [2023-12-27 05:09:51,061][105692] Updated weights for policy 0, policy_version 1889747 (0.0007) [2023-12-27 05:09:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 968859648. Throughput: 0: 9803.4, 1: 9882.1. Samples: 968849748. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:09:51,062][104569] Avg episode reward: [(0, '8260.945'), (1, '9253.248')] [2023-12-27 05:09:51,131][105692] Updated weights for policy 0, policy_version 1889757 (0.0008) [2023-12-27 05:09:51,212][105692] Updated weights for policy 0, policy_version 1889767 (0.0008) [2023-12-27 05:09:51,624][105620] Updated weights for policy 1, policy_version 1894322 (0.0009) [2023-12-27 05:09:51,683][105620] Updated weights for policy 1, policy_version 1894332 (0.0007) [2023-12-27 05:09:51,748][105620] Updated weights for policy 1, policy_version 1894342 (0.0008) [2023-12-27 05:09:51,795][105620] Updated weights for policy 1, policy_version 1894352 (0.0010) [2023-12-27 05:09:51,959][105692] Updated weights for policy 0, policy_version 1889777 (0.0009) [2023-12-27 05:09:52,021][105692] Updated weights for policy 0, policy_version 1889787 (0.0009) [2023-12-27 05:09:52,082][105692] Updated weights for policy 0, policy_version 1889797 (0.0007) [2023-12-27 05:09:52,144][105692] Updated weights for policy 0, policy_version 1889807 (0.0008) [2023-12-27 05:09:52,568][105620] Updated weights for policy 1, policy_version 1894362 (0.0011) [2023-12-27 05:09:52,618][105620] Updated weights for policy 1, policy_version 1894372 (0.0010) [2023-12-27 05:09:52,671][105620] Updated weights for policy 1, policy_version 1894382 (0.0011) [2023-12-27 05:09:52,886][105692] Updated weights for policy 0, policy_version 1889817 (0.0007) [2023-12-27 05:09:52,941][105692] Updated weights for policy 0, policy_version 1889827 (0.0008) [2023-12-27 05:09:52,991][105692] Updated weights for policy 0, policy_version 1889837 (0.0008) [2023-12-27 05:09:53,454][105620] Updated weights for policy 1, policy_version 1894392 (0.0010) [2023-12-27 05:09:53,514][105620] Updated weights for policy 1, policy_version 1894402 (0.0010) [2023-12-27 05:09:53,582][105620] Updated weights for policy 1, policy_version 1894412 (0.0010) [2023-12-27 05:09:53,749][105692] Updated weights for policy 0, policy_version 1889847 (0.0008) [2023-12-27 05:09:53,800][105692] Updated weights for policy 0, policy_version 1889857 (0.0008) [2023-12-27 05:09:53,847][105692] Updated weights for policy 0, policy_version 1889867 (0.0007) [2023-12-27 05:09:54,297][105620] Updated weights for policy 1, policy_version 1894422 (0.0007) [2023-12-27 05:09:54,345][105620] Updated weights for policy 1, policy_version 1894432 (0.0005) [2023-12-27 05:09:54,396][105620] Updated weights for policy 1, policy_version 1894442 (0.0005) [2023-12-27 05:09:54,642][105692] Updated weights for policy 0, policy_version 1889877 (0.0008) [2023-12-27 05:09:54,694][105692] Updated weights for policy 0, policy_version 1889887 (0.0008) [2023-12-27 05:09:54,746][105692] Updated weights for policy 0, policy_version 1889897 (0.0008) [2023-12-27 05:09:55,134][105620] Updated weights for policy 1, policy_version 1894452 (0.0008) [2023-12-27 05:09:55,186][105620] Updated weights for policy 1, policy_version 1894462 (0.0011) [2023-12-27 05:09:55,243][105620] Updated weights for policy 1, policy_version 1894472 (0.0010) [2023-12-27 05:09:55,516][105692] Updated weights for policy 0, policy_version 1889907 (0.0008) [2023-12-27 05:09:55,572][105692] Updated weights for policy 0, policy_version 1889917 (0.0008) [2023-12-27 05:09:55,633][105692] Updated weights for policy 0, policy_version 1889927 (0.0008) [2023-12-27 05:09:55,973][105620] Updated weights for policy 1, policy_version 1894482 (0.0010) [2023-12-27 05:09:56,031][105620] Updated weights for policy 1, policy_version 1894492 (0.0010) [2023-12-27 05:09:56,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 968949760. Throughput: 0: 9823.9, 1: 9853.8. Samples: 968962712. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:09:56,062][104569] Avg episode reward: [(0, '8261.602'), (1, '9162.163')] [2023-12-27 05:09:56,083][105620] Updated weights for policy 1, policy_version 1894502 (0.0010) [2023-12-27 05:09:56,131][105620] Updated weights for policy 1, policy_version 1894512 (0.0011) [2023-12-27 05:09:56,347][105692] Updated weights for policy 0, policy_version 1889937 (0.0008) [2023-12-27 05:09:56,412][105692] Updated weights for policy 0, policy_version 1889947 (0.0008) [2023-12-27 05:09:56,471][105692] Updated weights for policy 0, policy_version 1889957 (0.0008) [2023-12-27 05:09:56,523][105692] Updated weights for policy 0, policy_version 1889967 (0.0008) [2023-12-27 05:09:56,877][105620] Updated weights for policy 1, policy_version 1894522 (0.0010) [2023-12-27 05:09:56,932][105620] Updated weights for policy 1, policy_version 1894532 (0.0010) [2023-12-27 05:09:56,976][105620] Updated weights for policy 1, policy_version 1894542 (0.0010) [2023-12-27 05:09:57,254][105692] Updated weights for policy 0, policy_version 1889977 (0.0008) [2023-12-27 05:09:57,311][105692] Updated weights for policy 0, policy_version 1889987 (0.0009) [2023-12-27 05:09:57,367][105692] Updated weights for policy 0, policy_version 1889997 (0.0008) [2023-12-27 05:09:57,709][105620] Updated weights for policy 1, policy_version 1894552 (0.0009) [2023-12-27 05:09:57,765][105620] Updated weights for policy 1, policy_version 1894562 (0.0009) [2023-12-27 05:09:57,817][105620] Updated weights for policy 1, policy_version 1894573 (0.0009) [2023-12-27 05:09:58,137][105692] Updated weights for policy 0, policy_version 1890007 (0.0009) [2023-12-27 05:09:58,201][105692] Updated weights for policy 0, policy_version 1890017 (0.0008) [2023-12-27 05:09:58,262][105692] Updated weights for policy 0, policy_version 1890027 (0.0007) [2023-12-27 05:09:58,647][105620] Updated weights for policy 1, policy_version 1894583 (0.0008) [2023-12-27 05:09:58,704][105620] Updated weights for policy 1, policy_version 1894593 (0.0008) [2023-12-27 05:09:58,770][105620] Updated weights for policy 1, policy_version 1894603 (0.0006) [2023-12-27 05:09:59,063][105692] Updated weights for policy 0, policy_version 1890037 (0.0008) [2023-12-27 05:09:59,131][105692] Updated weights for policy 0, policy_version 1890047 (0.0009) [2023-12-27 05:09:59,192][105692] Updated weights for policy 0, policy_version 1890057 (0.0009) [2023-12-27 05:09:59,556][105620] Updated weights for policy 1, policy_version 1894613 (0.0008) [2023-12-27 05:09:59,608][105620] Updated weights for policy 1, policy_version 1894623 (0.0009) [2023-12-27 05:09:59,657][105620] Updated weights for policy 1, policy_version 1894633 (0.0009) [2023-12-27 05:10:00,014][105692] Updated weights for policy 0, policy_version 1890067 (0.0008) [2023-12-27 05:10:00,072][105692] Updated weights for policy 0, policy_version 1890077 (0.0009) [2023-12-27 05:10:00,131][105692] Updated weights for policy 0, policy_version 1890087 (0.0009) [2023-12-27 05:10:00,478][105620] Updated weights for policy 1, policy_version 1894643 (0.0009) [2023-12-27 05:10:00,539][105620] Updated weights for policy 1, policy_version 1894653 (0.0009) [2023-12-27 05:10:00,596][105620] Updated weights for policy 1, policy_version 1894663 (0.0009) [2023-12-27 05:10:00,739][105692] Updated weights for policy 0, policy_version 1890097 (0.0009) [2023-12-27 05:10:00,793][105692] Updated weights for policy 0, policy_version 1890107 (0.0005) [2023-12-27 05:10:00,853][105692] Updated weights for policy 0, policy_version 1890117 (0.0010) [2023-12-27 05:10:00,914][105692] Updated weights for policy 0, policy_version 1890127 (0.0009) [2023-12-27 05:10:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 969048064. Throughput: 0: 9821.2, 1: 9808.6. Samples: 969018256. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:10:01,062][104569] Avg episode reward: [(0, '8536.148'), (1, '9163.231')] [2023-12-27 05:10:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001890128_483942400.pth... [2023-12-27 05:10:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001894672_485105664.pth... [2023-12-27 05:10:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001893584_484827136.pth [2023-12-27 05:10:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001888976_483647488.pth [2023-12-27 05:10:01,428][105620] Updated weights for policy 1, policy_version 1894674 (0.0010) [2023-12-27 05:10:01,479][105620] Updated weights for policy 1, policy_version 1894684 (0.0009) [2023-12-27 05:10:01,530][105620] Updated weights for policy 1, policy_version 1894694 (0.0009) [2023-12-27 05:10:01,585][105620] Updated weights for policy 1, policy_version 1894704 (0.0009) [2023-12-27 05:10:01,600][105692] Updated weights for policy 0, policy_version 1890137 (0.0008) [2023-12-27 05:10:01,664][105692] Updated weights for policy 0, policy_version 1890147 (0.0010) [2023-12-27 05:10:01,725][105692] Updated weights for policy 0, policy_version 1890157 (0.0009) [2023-12-27 05:10:02,319][105620] Updated weights for policy 1, policy_version 1894714 (0.0009) [2023-12-27 05:10:02,388][105620] Updated weights for policy 1, policy_version 1894724 (0.0009) [2023-12-27 05:10:02,442][105620] Updated weights for policy 1, policy_version 1894734 (0.0009) [2023-12-27 05:10:02,479][105692] Updated weights for policy 0, policy_version 1890167 (0.0007) [2023-12-27 05:10:02,535][105692] Updated weights for policy 0, policy_version 1890177 (0.0005) [2023-12-27 05:10:02,604][105692] Updated weights for policy 0, policy_version 1890187 (0.0010) [2023-12-27 05:10:03,095][105620] Updated weights for policy 1, policy_version 1894744 (0.0010) [2023-12-27 05:10:03,153][105620] Updated weights for policy 1, policy_version 1894754 (0.0010) [2023-12-27 05:10:03,210][105620] Updated weights for policy 1, policy_version 1894764 (0.0009) [2023-12-27 05:10:03,328][105692] Updated weights for policy 0, policy_version 1890197 (0.0010) [2023-12-27 05:10:03,380][105692] Updated weights for policy 0, policy_version 1890208 (0.0010) [2023-12-27 05:10:03,450][105692] Updated weights for policy 0, policy_version 1890218 (0.0009) [2023-12-27 05:10:03,761][105620] Updated weights for policy 1, policy_version 1894774 (0.0008) [2023-12-27 05:10:03,807][105620] Updated weights for policy 1, policy_version 1894784 (0.0009) [2023-12-27 05:10:03,867][105620] Updated weights for policy 1, policy_version 1894794 (0.0009) [2023-12-27 05:10:04,162][105692] Updated weights for policy 0, policy_version 1890228 (0.0007) [2023-12-27 05:10:04,220][105692] Updated weights for policy 0, policy_version 1890238 (0.0008) [2023-12-27 05:10:04,275][105692] Updated weights for policy 0, policy_version 1890248 (0.0009) [2023-12-27 05:10:04,569][105620] Updated weights for policy 1, policy_version 1894804 (0.0006) [2023-12-27 05:10:04,631][105620] Updated weights for policy 1, policy_version 1894814 (0.0006) [2023-12-27 05:10:04,691][105620] Updated weights for policy 1, policy_version 1894824 (0.0008) [2023-12-27 05:10:05,068][105692] Updated weights for policy 0, policy_version 1890258 (0.0009) [2023-12-27 05:10:05,126][105692] Updated weights for policy 0, policy_version 1890268 (0.0010) [2023-12-27 05:10:05,185][105692] Updated weights for policy 0, policy_version 1890278 (0.0009) [2023-12-27 05:10:05,245][105692] Updated weights for policy 0, policy_version 1890288 (0.0008) [2023-12-27 05:10:05,358][105620] Updated weights for policy 1, policy_version 1894834 (0.0009) [2023-12-27 05:10:05,404][105620] Updated weights for policy 1, policy_version 1894844 (0.0008) [2023-12-27 05:10:05,451][105620] Updated weights for policy 1, policy_version 1894854 (0.0010) [2023-12-27 05:10:05,499][105620] Updated weights for policy 1, policy_version 1894864 (0.0010) [2023-12-27 05:10:06,056][105692] Updated weights for policy 0, policy_version 1890298 (0.0010) [2023-12-27 05:10:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 969138176. Throughput: 0: 9728.2, 1: 9706.7. Samples: 969133528. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:10:06,062][104569] Avg episode reward: [(0, '8176.926'), (1, '9254.293')] [2023-12-27 05:10:06,122][105692] Updated weights for policy 0, policy_version 1890308 (0.0009) [2023-12-27 05:10:06,140][105620] Updated weights for policy 1, policy_version 1894874 (0.0006) [2023-12-27 05:10:06,190][105692] Updated weights for policy 0, policy_version 1890318 (0.0008) [2023-12-27 05:10:06,202][105620] Updated weights for policy 1, policy_version 1894884 (0.0006) [2023-12-27 05:10:06,264][105620] Updated weights for policy 1, policy_version 1894894 (0.0006) [2023-12-27 05:10:06,848][105620] Updated weights for policy 1, policy_version 1894904 (0.0006) [2023-12-27 05:10:06,901][105620] Updated weights for policy 1, policy_version 1894914 (0.0005) [2023-12-27 05:10:06,952][105620] Updated weights for policy 1, policy_version 1894924 (0.0005) [2023-12-27 05:10:07,088][105692] Updated weights for policy 0, policy_version 1890328 (0.0008) [2023-12-27 05:10:07,151][105692] Updated weights for policy 0, policy_version 1890338 (0.0010) [2023-12-27 05:10:07,218][105692] Updated weights for policy 0, policy_version 1890348 (0.0009) [2023-12-27 05:10:07,565][105620] Updated weights for policy 1, policy_version 1894934 (0.0006) [2023-12-27 05:10:07,632][105620] Updated weights for policy 1, policy_version 1894944 (0.0010) [2023-12-27 05:10:07,700][105620] Updated weights for policy 1, policy_version 1894954 (0.0010) [2023-12-27 05:10:07,954][105692] Updated weights for policy 0, policy_version 1890358 (0.0008) [2023-12-27 05:10:08,011][105692] Updated weights for policy 0, policy_version 1890368 (0.0007) [2023-12-27 05:10:08,083][105692] Updated weights for policy 0, policy_version 1890378 (0.0010) [2023-12-27 05:10:08,345][105620] Updated weights for policy 1, policy_version 1894964 (0.0010) [2023-12-27 05:10:08,401][105620] Updated weights for policy 1, policy_version 1894974 (0.0009) [2023-12-27 05:10:08,464][105620] Updated weights for policy 1, policy_version 1894984 (0.0009) [2023-12-27 05:10:08,835][105692] Updated weights for policy 0, policy_version 1890388 (0.0009) [2023-12-27 05:10:08,890][105692] Updated weights for policy 0, policy_version 1890398 (0.0008) [2023-12-27 05:10:08,947][105692] Updated weights for policy 0, policy_version 1890408 (0.0008) [2023-12-27 05:10:09,161][105620] Updated weights for policy 1, policy_version 1894994 (0.0009) [2023-12-27 05:10:09,217][105620] Updated weights for policy 1, policy_version 1895004 (0.0009) [2023-12-27 05:10:09,285][105620] Updated weights for policy 1, policy_version 1895014 (0.0009) [2023-12-27 05:10:09,350][105620] Updated weights for policy 1, policy_version 1895024 (0.0009) [2023-12-27 05:10:09,737][105692] Updated weights for policy 0, policy_version 1890418 (0.0009) [2023-12-27 05:10:09,795][105692] Updated weights for policy 0, policy_version 1890428 (0.0008) [2023-12-27 05:10:09,860][105692] Updated weights for policy 0, policy_version 1890438 (0.0010) [2023-12-27 05:10:09,924][105692] Updated weights for policy 0, policy_version 1890448 (0.0010) [2023-12-27 05:10:10,126][105620] Updated weights for policy 1, policy_version 1895034 (0.0009) [2023-12-27 05:10:10,188][105620] Updated weights for policy 1, policy_version 1895044 (0.0009) [2023-12-27 05:10:10,248][105620] Updated weights for policy 1, policy_version 1895054 (0.0010) [2023-12-27 05:10:10,648][105692] Updated weights for policy 0, policy_version 1890458 (0.0005) [2023-12-27 05:10:10,707][105692] Updated weights for policy 0, policy_version 1890468 (0.0006) [2023-12-27 05:10:10,763][105692] Updated weights for policy 0, policy_version 1890478 (0.0006) [2023-12-27 05:10:11,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 969236480. Throughput: 0: 9683.2, 1: 9736.6. Samples: 969247708. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:10:11,063][104569] Avg episode reward: [(0, '7900.435'), (1, '9161.956')] [2023-12-27 05:10:11,068][105620] Updated weights for policy 1, policy_version 1895064 (0.0008) [2023-12-27 05:10:11,122][105620] Updated weights for policy 1, policy_version 1895074 (0.0007) [2023-12-27 05:10:11,189][105620] Updated weights for policy 1, policy_version 1895084 (0.0009) [2023-12-27 05:10:11,425][105692] Updated weights for policy 0, policy_version 1890488 (0.0007) [2023-12-27 05:10:11,484][105692] Updated weights for policy 0, policy_version 1890498 (0.0009) [2023-12-27 05:10:11,540][105692] Updated weights for policy 0, policy_version 1890508 (0.0009) [2023-12-27 05:10:11,961][105620] Updated weights for policy 1, policy_version 1895094 (0.0007) [2023-12-27 05:10:12,021][105620] Updated weights for policy 1, policy_version 1895104 (0.0008) [2023-12-27 05:10:12,076][105620] Updated weights for policy 1, policy_version 1895114 (0.0009) [2023-12-27 05:10:12,296][105692] Updated weights for policy 0, policy_version 1890518 (0.0008) [2023-12-27 05:10:12,359][105692] Updated weights for policy 0, policy_version 1890528 (0.0008) [2023-12-27 05:10:12,423][105692] Updated weights for policy 0, policy_version 1890538 (0.0009) [2023-12-27 05:10:12,796][105620] Updated weights for policy 1, policy_version 1895124 (0.0008) [2023-12-27 05:10:12,855][105620] Updated weights for policy 1, policy_version 1895134 (0.0010) [2023-12-27 05:10:12,912][105620] Updated weights for policy 1, policy_version 1895144 (0.0010) [2023-12-27 05:10:13,151][105692] Updated weights for policy 0, policy_version 1890548 (0.0008) [2023-12-27 05:10:13,210][105692] Updated weights for policy 0, policy_version 1890558 (0.0006) [2023-12-27 05:10:13,267][105692] Updated weights for policy 0, policy_version 1890568 (0.0005) [2023-12-27 05:10:13,596][105620] Updated weights for policy 1, policy_version 1895154 (0.0010) [2023-12-27 05:10:13,654][105620] Updated weights for policy 1, policy_version 1895164 (0.0010) [2023-12-27 05:10:13,709][105620] Updated weights for policy 1, policy_version 1895174 (0.0010) [2023-12-27 05:10:13,767][105620] Updated weights for policy 1, policy_version 1895184 (0.0010) [2023-12-27 05:10:13,962][105692] Updated weights for policy 0, policy_version 1890578 (0.0006) [2023-12-27 05:10:14,028][105692] Updated weights for policy 0, policy_version 1890588 (0.0008) [2023-12-27 05:10:14,091][105692] Updated weights for policy 0, policy_version 1890598 (0.0008) [2023-12-27 05:10:14,151][105692] Updated weights for policy 0, policy_version 1890608 (0.0008) [2023-12-27 05:10:14,504][105620] Updated weights for policy 1, policy_version 1895194 (0.0010) [2023-12-27 05:10:14,552][105620] Updated weights for policy 1, policy_version 1895204 (0.0010) [2023-12-27 05:10:14,600][105620] Updated weights for policy 1, policy_version 1895214 (0.0010) [2023-12-27 05:10:14,889][105692] Updated weights for policy 0, policy_version 1890618 (0.0009) [2023-12-27 05:10:14,949][105692] Updated weights for policy 0, policy_version 1890628 (0.0008) [2023-12-27 05:10:15,015][105692] Updated weights for policy 0, policy_version 1890638 (0.0009) [2023-12-27 05:10:15,387][105620] Updated weights for policy 1, policy_version 1895224 (0.0010) [2023-12-27 05:10:15,439][105620] Updated weights for policy 1, policy_version 1895234 (0.0010) [2023-12-27 05:10:15,494][105620] Updated weights for policy 1, policy_version 1895244 (0.0010) [2023-12-27 05:10:15,774][105692] Updated weights for policy 0, policy_version 1890648 (0.0010) [2023-12-27 05:10:15,832][105692] Updated weights for policy 0, policy_version 1890658 (0.0010) [2023-12-27 05:10:15,886][105692] Updated weights for policy 0, policy_version 1890668 (0.0010) [2023-12-27 05:10:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 969334784. Throughput: 0: 9580.7, 1: 9689.2. Samples: 969305244. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:10:16,062][104569] Avg episode reward: [(0, '8261.530'), (1, '8884.675')] [2023-12-27 05:10:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001890672_484081664.pth... [2023-12-27 05:10:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001895248_485253120.pth... [2023-12-27 05:10:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001889584_483803136.pth [2023-12-27 05:10:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001894128_484966400.pth [2023-12-27 05:10:16,242][105620] Updated weights for policy 1, policy_version 1895254 (0.0011) [2023-12-27 05:10:16,304][105620] Updated weights for policy 1, policy_version 1895264 (0.0008) [2023-12-27 05:10:16,363][105620] Updated weights for policy 1, policy_version 1895274 (0.0011) [2023-12-27 05:10:16,645][105692] Updated weights for policy 0, policy_version 1890678 (0.0010) [2023-12-27 05:10:16,700][105692] Updated weights for policy 0, policy_version 1890688 (0.0010) [2023-12-27 05:10:16,767][105692] Updated weights for policy 0, policy_version 1890698 (0.0010) [2023-12-27 05:10:17,116][105620] Updated weights for policy 1, policy_version 1895284 (0.0009) [2023-12-27 05:10:17,181][105620] Updated weights for policy 1, policy_version 1895294 (0.0007) [2023-12-27 05:10:17,250][105620] Updated weights for policy 1, policy_version 1895304 (0.0006) [2023-12-27 05:10:17,482][105692] Updated weights for policy 0, policy_version 1890708 (0.0009) [2023-12-27 05:10:17,558][105692] Updated weights for policy 0, policy_version 1890718 (0.0006) [2023-12-27 05:10:17,622][105692] Updated weights for policy 0, policy_version 1890728 (0.0010) [2023-12-27 05:10:17,829][105620] Updated weights for policy 1, policy_version 1895314 (0.0006) [2023-12-27 05:10:17,892][105620] Updated weights for policy 1, policy_version 1895324 (0.0008) [2023-12-27 05:10:17,954][105620] Updated weights for policy 1, policy_version 1895334 (0.0008) [2023-12-27 05:10:18,013][105620] Updated weights for policy 1, policy_version 1895344 (0.0008) [2023-12-27 05:10:18,172][105692] Updated weights for policy 0, policy_version 1890738 (0.0011) [2023-12-27 05:10:18,231][105692] Updated weights for policy 0, policy_version 1890748 (0.0010) [2023-12-27 05:10:18,295][105692] Updated weights for policy 0, policy_version 1890758 (0.0011) [2023-12-27 05:10:18,362][105692] Updated weights for policy 0, policy_version 1890768 (0.0010) [2023-12-27 05:10:18,667][105620] Updated weights for policy 1, policy_version 1895354 (0.0008) [2023-12-27 05:10:18,726][105620] Updated weights for policy 1, policy_version 1895364 (0.0008) [2023-12-27 05:10:18,785][105620] Updated weights for policy 1, policy_version 1895374 (0.0008) [2023-12-27 05:10:19,093][105692] Updated weights for policy 0, policy_version 1890778 (0.0010) [2023-12-27 05:10:19,151][105692] Updated weights for policy 0, policy_version 1890788 (0.0010) [2023-12-27 05:10:19,203][105692] Updated weights for policy 0, policy_version 1890798 (0.0010) [2023-12-27 05:10:19,558][105620] Updated weights for policy 1, policy_version 1895384 (0.0008) [2023-12-27 05:10:19,613][105620] Updated weights for policy 1, policy_version 1895394 (0.0010) [2023-12-27 05:10:19,667][105620] Updated weights for policy 1, policy_version 1895404 (0.0010) [2023-12-27 05:10:19,834][105692] Updated weights for policy 0, policy_version 1890808 (0.0008) [2023-12-27 05:10:19,901][105692] Updated weights for policy 0, policy_version 1890818 (0.0008) [2023-12-27 05:10:19,962][105692] Updated weights for policy 0, policy_version 1890828 (0.0009) [2023-12-27 05:10:20,432][105620] Updated weights for policy 1, policy_version 1895414 (0.0009) [2023-12-27 05:10:20,495][105620] Updated weights for policy 1, policy_version 1895424 (0.0009) [2023-12-27 05:10:20,557][105620] Updated weights for policy 1, policy_version 1895434 (0.0010) [2023-12-27 05:10:20,657][105692] Updated weights for policy 0, policy_version 1890838 (0.0007) [2023-12-27 05:10:20,724][105692] Updated weights for policy 0, policy_version 1890848 (0.0007) [2023-12-27 05:10:20,786][105692] Updated weights for policy 0, policy_version 1890858 (0.0008) [2023-12-27 05:10:21,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 969433088. Throughput: 0: 9528.3, 1: 9700.7. Samples: 969420596. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:10:21,062][104569] Avg episode reward: [(0, '8352.557'), (1, '9068.237')] [2023-12-27 05:10:21,440][105620] Updated weights for policy 1, policy_version 1895444 (0.0008) [2023-12-27 05:10:21,469][105692] Updated weights for policy 0, policy_version 1890868 (0.0006) [2023-12-27 05:10:21,499][105620] Updated weights for policy 1, policy_version 1895454 (0.0008) [2023-12-27 05:10:21,526][105692] Updated weights for policy 0, policy_version 1890878 (0.0006) [2023-12-27 05:10:21,560][105620] Updated weights for policy 1, policy_version 1895464 (0.0009) [2023-12-27 05:10:21,584][105692] Updated weights for policy 0, policy_version 1890888 (0.0007) [2023-12-27 05:10:22,311][105692] Updated weights for policy 0, policy_version 1890898 (0.0008) [2023-12-27 05:10:22,378][105692] Updated weights for policy 0, policy_version 1890908 (0.0008) [2023-12-27 05:10:22,393][105620] Updated weights for policy 1, policy_version 1895474 (0.0008) [2023-12-27 05:10:22,440][105692] Updated weights for policy 0, policy_version 1890918 (0.0009) [2023-12-27 05:10:22,455][105620] Updated weights for policy 1, policy_version 1895484 (0.0007) [2023-12-27 05:10:22,502][105692] Updated weights for policy 0, policy_version 1890928 (0.0007) [2023-12-27 05:10:22,517][105620] Updated weights for policy 1, policy_version 1895494 (0.0006) [2023-12-27 05:10:22,574][105620] Updated weights for policy 1, policy_version 1895504 (0.0008) [2023-12-27 05:10:23,287][105692] Updated weights for policy 0, policy_version 1890938 (0.0008) [2023-12-27 05:10:23,330][105620] Updated weights for policy 1, policy_version 1895514 (0.0008) [2023-12-27 05:10:23,344][105692] Updated weights for policy 0, policy_version 1890948 (0.0009) [2023-12-27 05:10:23,387][105620] Updated weights for policy 1, policy_version 1895524 (0.0008) [2023-12-27 05:10:23,397][105692] Updated weights for policy 0, policy_version 1890958 (0.0007) [2023-12-27 05:10:23,436][105620] Updated weights for policy 1, policy_version 1895534 (0.0008) [2023-12-27 05:10:24,099][105692] Updated weights for policy 0, policy_version 1890968 (0.0008) [2023-12-27 05:10:24,162][105692] Updated weights for policy 0, policy_version 1890978 (0.0009) [2023-12-27 05:10:24,220][105692] Updated weights for policy 0, policy_version 1890988 (0.0007) [2023-12-27 05:10:24,233][105620] Updated weights for policy 1, policy_version 1895544 (0.0008) [2023-12-27 05:10:24,289][105620] Updated weights for policy 1, policy_version 1895554 (0.0009) [2023-12-27 05:10:24,336][105620] Updated weights for policy 1, policy_version 1895564 (0.0009) [2023-12-27 05:10:24,898][105692] Updated weights for policy 0, policy_version 1890998 (0.0006) [2023-12-27 05:10:24,952][105692] Updated weights for policy 0, policy_version 1891008 (0.0006) [2023-12-27 05:10:24,999][105692] Updated weights for policy 0, policy_version 1891018 (0.0005) [2023-12-27 05:10:25,073][105620] Updated weights for policy 1, policy_version 1895574 (0.0010) [2023-12-27 05:10:25,127][105620] Updated weights for policy 1, policy_version 1895584 (0.0010) [2023-12-27 05:10:25,175][105620] Updated weights for policy 1, policy_version 1895594 (0.0010) [2023-12-27 05:10:25,523][105692] Updated weights for policy 0, policy_version 1891028 (0.0006) [2023-12-27 05:10:25,568][105692] Updated weights for policy 0, policy_version 1891038 (0.0006) [2023-12-27 05:10:25,617][105692] Updated weights for policy 0, policy_version 1891048 (0.0008) [2023-12-27 05:10:25,961][105620] Updated weights for policy 1, policy_version 1895604 (0.0010) [2023-12-27 05:10:26,007][105620] Updated weights for policy 1, policy_version 1895614 (0.0008) [2023-12-27 05:10:26,054][105620] Updated weights for policy 1, policy_version 1895624 (0.0009) [2023-12-27 05:10:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 969523200. Throughput: 0: 9532.0, 1: 9508.5. Samples: 969535764. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:10:26,063][104569] Avg episode reward: [(0, '8349.188'), (1, '9345.458')] [2023-12-27 05:10:26,335][105692] Updated weights for policy 0, policy_version 1891058 (0.0006) [2023-12-27 05:10:26,387][105692] Updated weights for policy 0, policy_version 1891068 (0.0007) [2023-12-27 05:10:26,436][105692] Updated weights for policy 0, policy_version 1891078 (0.0006) [2023-12-27 05:10:26,496][105692] Updated weights for policy 0, policy_version 1891088 (0.0008) [2023-12-27 05:10:26,781][105620] Updated weights for policy 1, policy_version 1895634 (0.0008) [2023-12-27 05:10:26,848][105620] Updated weights for policy 1, policy_version 1895644 (0.0008) [2023-12-27 05:10:26,896][105620] Updated weights for policy 1, policy_version 1895654 (0.0006) [2023-12-27 05:10:26,939][105620] Updated weights for policy 1, policy_version 1895664 (0.0005) [2023-12-27 05:10:27,146][105692] Updated weights for policy 0, policy_version 1891098 (0.0008) [2023-12-27 05:10:27,204][105692] Updated weights for policy 0, policy_version 1891108 (0.0008) [2023-12-27 05:10:27,258][105692] Updated weights for policy 0, policy_version 1891118 (0.0007) [2023-12-27 05:10:27,590][105620] Updated weights for policy 1, policy_version 1895674 (0.0010) [2023-12-27 05:10:27,637][105620] Updated weights for policy 1, policy_version 1895684 (0.0010) [2023-12-27 05:10:27,693][105620] Updated weights for policy 1, policy_version 1895694 (0.0010) [2023-12-27 05:10:27,879][105692] Updated weights for policy 0, policy_version 1891128 (0.0007) [2023-12-27 05:10:27,926][105692] Updated weights for policy 0, policy_version 1891138 (0.0010) [2023-12-27 05:10:27,974][105692] Updated weights for policy 0, policy_version 1891148 (0.0010) [2023-12-27 05:10:28,452][105620] Updated weights for policy 1, policy_version 1895704 (0.0010) [2023-12-27 05:10:28,524][105620] Updated weights for policy 1, policy_version 1895714 (0.0010) [2023-12-27 05:10:28,580][105620] Updated weights for policy 1, policy_version 1895724 (0.0011) [2023-12-27 05:10:28,610][105692] Updated weights for policy 0, policy_version 1891158 (0.0007) [2023-12-27 05:10:28,676][105692] Updated weights for policy 0, policy_version 1891168 (0.0008) [2023-12-27 05:10:28,743][105692] Updated weights for policy 0, policy_version 1891178 (0.0008) [2023-12-27 05:10:29,225][105620] Updated weights for policy 1, policy_version 1895734 (0.0008) [2023-12-27 05:10:29,294][105620] Updated weights for policy 1, policy_version 1895744 (0.0009) [2023-12-27 05:10:29,371][105620] Updated weights for policy 1, policy_version 1895754 (0.0008) [2023-12-27 05:10:29,532][105692] Updated weights for policy 0, policy_version 1891188 (0.0008) [2023-12-27 05:10:29,598][105692] Updated weights for policy 0, policy_version 1891198 (0.0008) [2023-12-27 05:10:29,658][105692] Updated weights for policy 0, policy_version 1891208 (0.0009) [2023-12-27 05:10:30,072][105620] Updated weights for policy 1, policy_version 1895764 (0.0009) [2023-12-27 05:10:30,127][105620] Updated weights for policy 1, policy_version 1895774 (0.0008) [2023-12-27 05:10:30,182][105620] Updated weights for policy 1, policy_version 1895784 (0.0008) [2023-12-27 05:10:30,385][105692] Updated weights for policy 0, policy_version 1891218 (0.0007) [2023-12-27 05:10:30,444][105692] Updated weights for policy 0, policy_version 1891228 (0.0006) [2023-12-27 05:10:30,508][105692] Updated weights for policy 0, policy_version 1891238 (0.0008) [2023-12-27 05:10:30,561][105692] Updated weights for policy 0, policy_version 1891248 (0.0005) [2023-12-27 05:10:31,033][105620] Updated weights for policy 1, policy_version 1895794 (0.0009) [2023-12-27 05:10:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 969621504. Throughput: 0: 9632.0, 1: 9520.3. Samples: 969597080. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:10:31,063][104569] Avg episode reward: [(0, '8444.622'), (1, '9345.437')] [2023-12-27 05:10:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001891248_484229120.pth... [2023-12-27 05:10:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001890128_483942400.pth [2023-12-27 05:10:31,089][105620] Updated weights for policy 1, policy_version 1895804 (0.0009) [2023-12-27 05:10:31,155][105620] Updated weights for policy 1, policy_version 1895814 (0.0009) [2023-12-27 05:10:31,197][105692] Updated weights for policy 0, policy_version 1891258 (0.0006) [2023-12-27 05:10:31,210][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001895824_485400576.pth... [2023-12-27 05:10:31,211][105620] Updated weights for policy 1, policy_version 1895824 (0.0007) [2023-12-27 05:10:31,213][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001894672_485105664.pth [2023-12-27 05:10:31,262][105692] Updated weights for policy 0, policy_version 1891268 (0.0006) [2023-12-27 05:10:31,327][105692] Updated weights for policy 0, policy_version 1891278 (0.0008) [2023-12-27 05:10:31,965][105620] Updated weights for policy 1, policy_version 1895834 (0.0008) [2023-12-27 05:10:32,021][105620] Updated weights for policy 1, policy_version 1895844 (0.0006) [2023-12-27 05:10:32,059][105692] Updated weights for policy 0, policy_version 1891288 (0.0008) [2023-12-27 05:10:32,084][105620] Updated weights for policy 1, policy_version 1895854 (0.0007) [2023-12-27 05:10:32,127][105692] Updated weights for policy 0, policy_version 1891298 (0.0007) [2023-12-27 05:10:32,189][105692] Updated weights for policy 0, policy_version 1891308 (0.0008) [2023-12-27 05:10:32,722][105620] Updated weights for policy 1, policy_version 1895864 (0.0009) [2023-12-27 05:10:32,775][105620] Updated weights for policy 1, policy_version 1895874 (0.0009) [2023-12-27 05:10:32,832][105620] Updated weights for policy 1, policy_version 1895884 (0.0009) [2023-12-27 05:10:32,898][105692] Updated weights for policy 0, policy_version 1891318 (0.0008) [2023-12-27 05:10:32,953][105692] Updated weights for policy 0, policy_version 1891328 (0.0007) [2023-12-27 05:10:33,004][105692] Updated weights for policy 0, policy_version 1891338 (0.0009) [2023-12-27 05:10:33,642][105620] Updated weights for policy 1, policy_version 1895894 (0.0009) [2023-12-27 05:10:33,645][105692] Updated weights for policy 0, policy_version 1891348 (0.0008) [2023-12-27 05:10:33,694][105692] Updated weights for policy 0, policy_version 1891358 (0.0007) [2023-12-27 05:10:33,700][105620] Updated weights for policy 1, policy_version 1895904 (0.0008) [2023-12-27 05:10:33,743][105692] Updated weights for policy 0, policy_version 1891368 (0.0005) [2023-12-27 05:10:33,759][105620] Updated weights for policy 1, policy_version 1895914 (0.0008) [2023-12-27 05:10:34,490][105620] Updated weights for policy 1, policy_version 1895924 (0.0008) [2023-12-27 05:10:34,537][105692] Updated weights for policy 0, policy_version 1891378 (0.0006) [2023-12-27 05:10:34,550][105620] Updated weights for policy 1, policy_version 1895934 (0.0009) [2023-12-27 05:10:34,597][105692] Updated weights for policy 0, policy_version 1891388 (0.0006) [2023-12-27 05:10:34,611][105620] Updated weights for policy 1, policy_version 1895944 (0.0009) [2023-12-27 05:10:34,658][105692] Updated weights for policy 0, policy_version 1891398 (0.0007) [2023-12-27 05:10:34,713][105692] Updated weights for policy 0, policy_version 1891408 (0.0008) [2023-12-27 05:10:35,334][105692] Updated weights for policy 0, policy_version 1891418 (0.0005) [2023-12-27 05:10:35,387][105692] Updated weights for policy 0, policy_version 1891428 (0.0005) [2023-12-27 05:10:35,389][105620] Updated weights for policy 1, policy_version 1895954 (0.0007) [2023-12-27 05:10:35,441][105620] Updated weights for policy 1, policy_version 1895964 (0.0007) [2023-12-27 05:10:35,446][105692] Updated weights for policy 0, policy_version 1891438 (0.0007) [2023-12-27 05:10:35,493][105620] Updated weights for policy 1, policy_version 1895974 (0.0008) [2023-12-27 05:10:35,541][105620] Updated weights for policy 1, policy_version 1895984 (0.0008) [2023-12-27 05:10:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 969719808. Throughput: 0: 9635.6, 1: 9508.9. Samples: 969711256. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:10:36,063][104569] Avg episode reward: [(0, '8716.784'), (1, '9254.121')] [2023-12-27 05:10:36,155][105692] Updated weights for policy 0, policy_version 1891448 (0.0010) [2023-12-27 05:10:36,221][105692] Updated weights for policy 0, policy_version 1891458 (0.0011) [2023-12-27 05:10:36,287][105692] Updated weights for policy 0, policy_version 1891468 (0.0011) [2023-12-27 05:10:36,316][105620] Updated weights for policy 1, policy_version 1895994 (0.0008) [2023-12-27 05:10:36,386][105620] Updated weights for policy 1, policy_version 1896004 (0.0008) [2023-12-27 05:10:36,451][105620] Updated weights for policy 1, policy_version 1896014 (0.0008) [2023-12-27 05:10:36,947][105692] Updated weights for policy 0, policy_version 1891478 (0.0009) [2023-12-27 05:10:37,007][105692] Updated weights for policy 0, policy_version 1891488 (0.0005) [2023-12-27 05:10:37,069][105692] Updated weights for policy 0, policy_version 1891498 (0.0010) [2023-12-27 05:10:37,176][105620] Updated weights for policy 1, policy_version 1896024 (0.0006) [2023-12-27 05:10:37,241][105620] Updated weights for policy 1, policy_version 1896034 (0.0006) [2023-12-27 05:10:37,310][105620] Updated weights for policy 1, policy_version 1896044 (0.0005) [2023-12-27 05:10:37,728][105692] Updated weights for policy 0, policy_version 1891508 (0.0009) [2023-12-27 05:10:37,791][105692] Updated weights for policy 0, policy_version 1891518 (0.0010) [2023-12-27 05:10:37,851][105692] Updated weights for policy 0, policy_version 1891528 (0.0011) [2023-12-27 05:10:37,880][105620] Updated weights for policy 1, policy_version 1896054 (0.0007) [2023-12-27 05:10:37,946][105620] Updated weights for policy 1, policy_version 1896064 (0.0007) [2023-12-27 05:10:38,018][105620] Updated weights for policy 1, policy_version 1896074 (0.0006) [2023-12-27 05:10:38,605][105692] Updated weights for policy 0, policy_version 1891538 (0.0011) [2023-12-27 05:10:38,647][105620] Updated weights for policy 1, policy_version 1896084 (0.0005) [2023-12-27 05:10:38,657][105692] Updated weights for policy 0, policy_version 1891548 (0.0010) [2023-12-27 05:10:38,710][105620] Updated weights for policy 1, policy_version 1896094 (0.0007) [2023-12-27 05:10:38,713][105692] Updated weights for policy 0, policy_version 1891558 (0.0010) [2023-12-27 05:10:38,766][105620] Updated weights for policy 1, policy_version 1896104 (0.0007) [2023-12-27 05:10:38,772][105692] Updated weights for policy 0, policy_version 1891568 (0.0010) [2023-12-27 05:10:39,544][105692] Updated weights for policy 0, policy_version 1891578 (0.0008) [2023-12-27 05:10:39,563][105620] Updated weights for policy 1, policy_version 1896114 (0.0008) [2023-12-27 05:10:39,607][105692] Updated weights for policy 0, policy_version 1891588 (0.0007) [2023-12-27 05:10:39,624][105620] Updated weights for policy 1, policy_version 1896124 (0.0007) [2023-12-27 05:10:39,665][105692] Updated weights for policy 0, policy_version 1891598 (0.0006) [2023-12-27 05:10:39,691][105620] Updated weights for policy 1, policy_version 1896134 (0.0009) [2023-12-27 05:10:39,748][105620] Updated weights for policy 1, policy_version 1896144 (0.0010) [2023-12-27 05:10:40,333][105692] Updated weights for policy 0, policy_version 1891608 (0.0008) [2023-12-27 05:10:40,382][105692] Updated weights for policy 0, policy_version 1891618 (0.0009) [2023-12-27 05:10:40,431][105692] Updated weights for policy 0, policy_version 1891628 (0.0006) [2023-12-27 05:10:40,567][105620] Updated weights for policy 1, policy_version 1896154 (0.0009) [2023-12-27 05:10:40,625][105620] Updated weights for policy 1, policy_version 1896164 (0.0009) [2023-12-27 05:10:40,687][105620] Updated weights for policy 1, policy_version 1896174 (0.0010) [2023-12-27 05:10:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 969818112. Throughput: 0: 9690.8, 1: 9541.7. Samples: 969828172. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:10:41,062][104569] Avg episode reward: [(0, '8805.610'), (1, '9161.824')] [2023-12-27 05:10:41,181][105692] Updated weights for policy 0, policy_version 1891638 (0.0006) [2023-12-27 05:10:41,241][105692] Updated weights for policy 0, policy_version 1891648 (0.0007) [2023-12-27 05:10:41,309][105692] Updated weights for policy 0, policy_version 1891658 (0.0007) [2023-12-27 05:10:41,463][105620] Updated weights for policy 1, policy_version 1896184 (0.0010) [2023-12-27 05:10:41,521][105620] Updated weights for policy 1, policy_version 1896194 (0.0009) [2023-12-27 05:10:41,582][105620] Updated weights for policy 1, policy_version 1896204 (0.0009) [2023-12-27 05:10:41,996][105692] Updated weights for policy 0, policy_version 1891668 (0.0007) [2023-12-27 05:10:42,063][105692] Updated weights for policy 0, policy_version 1891678 (0.0009) [2023-12-27 05:10:42,117][105692] Updated weights for policy 0, policy_version 1891688 (0.0008) [2023-12-27 05:10:42,385][105620] Updated weights for policy 1, policy_version 1896214 (0.0008) [2023-12-27 05:10:42,445][105620] Updated weights for policy 1, policy_version 1896224 (0.0008) [2023-12-27 05:10:42,508][105620] Updated weights for policy 1, policy_version 1896234 (0.0008) [2023-12-27 05:10:42,840][105692] Updated weights for policy 0, policy_version 1891698 (0.0008) [2023-12-27 05:10:42,899][105692] Updated weights for policy 0, policy_version 1891708 (0.0005) [2023-12-27 05:10:42,958][105692] Updated weights for policy 0, policy_version 1891718 (0.0005) [2023-12-27 05:10:43,017][105692] Updated weights for policy 0, policy_version 1891728 (0.0005) [2023-12-27 05:10:43,208][105620] Updated weights for policy 1, policy_version 1896244 (0.0008) [2023-12-27 05:10:43,275][105620] Updated weights for policy 1, policy_version 1896254 (0.0008) [2023-12-27 05:10:43,332][105620] Updated weights for policy 1, policy_version 1896265 (0.0010) [2023-12-27 05:10:43,568][105692] Updated weights for policy 0, policy_version 1891738 (0.0010) [2023-12-27 05:10:43,628][105692] Updated weights for policy 0, policy_version 1891748 (0.0008) [2023-12-27 05:10:43,698][105692] Updated weights for policy 0, policy_version 1891758 (0.0005) [2023-12-27 05:10:44,037][105620] Updated weights for policy 1, policy_version 1896276 (0.0008) [2023-12-27 05:10:44,102][105620] Updated weights for policy 1, policy_version 1896286 (0.0006) [2023-12-27 05:10:44,159][105620] Updated weights for policy 1, policy_version 1896296 (0.0005) [2023-12-27 05:10:44,234][105692] Updated weights for policy 0, policy_version 1891768 (0.0010) [2023-12-27 05:10:44,299][105692] Updated weights for policy 0, policy_version 1891778 (0.0010) [2023-12-27 05:10:44,355][105692] Updated weights for policy 0, policy_version 1891788 (0.0009) [2023-12-27 05:10:44,783][105620] Updated weights for policy 1, policy_version 1896306 (0.0008) [2023-12-27 05:10:44,845][105620] Updated weights for policy 1, policy_version 1896316 (0.0008) [2023-12-27 05:10:44,900][105620] Updated weights for policy 1, policy_version 1896326 (0.0006) [2023-12-27 05:10:44,959][105620] Updated weights for policy 1, policy_version 1896336 (0.0007) [2023-12-27 05:10:44,978][105692] Updated weights for policy 0, policy_version 1891798 (0.0007) [2023-12-27 05:10:45,046][105692] Updated weights for policy 0, policy_version 1891808 (0.0008) [2023-12-27 05:10:45,117][105692] Updated weights for policy 0, policy_version 1891818 (0.0011) [2023-12-27 05:10:45,708][105620] Updated weights for policy 1, policy_version 1896346 (0.0005) [2023-12-27 05:10:45,759][105620] Updated weights for policy 1, policy_version 1896356 (0.0005) [2023-12-27 05:10:45,814][105620] Updated weights for policy 1, policy_version 1896366 (0.0007) [2023-12-27 05:10:45,854][105692] Updated weights for policy 0, policy_version 1891828 (0.0011) [2023-12-27 05:10:45,916][105692] Updated weights for policy 0, policy_version 1891838 (0.0010) [2023-12-27 05:10:45,971][105692] Updated weights for policy 0, policy_version 1891848 (0.0010) [2023-12-27 05:10:46,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 969924608. Throughput: 0: 9754.8, 1: 9551.6. Samples: 969887040. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:10:46,062][104569] Avg episode reward: [(0, '8357.456'), (1, '9253.249')] [2023-12-27 05:10:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001896368_485539840.pth... [2023-12-27 05:10:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001891856_484384768.pth... [2023-12-27 05:10:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001890672_484081664.pth [2023-12-27 05:10:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001895248_485253120.pth [2023-12-27 05:10:46,524][105620] Updated weights for policy 1, policy_version 1896376 (0.0008) [2023-12-27 05:10:46,571][105620] Updated weights for policy 1, policy_version 1896386 (0.0007) [2023-12-27 05:10:46,630][105620] Updated weights for policy 1, policy_version 1896396 (0.0008) [2023-12-27 05:10:46,670][105692] Updated weights for policy 0, policy_version 1891858 (0.0010) [2023-12-27 05:10:46,730][105692] Updated weights for policy 0, policy_version 1891868 (0.0011) [2023-12-27 05:10:46,793][105692] Updated weights for policy 0, policy_version 1891878 (0.0011) [2023-12-27 05:10:46,857][105692] Updated weights for policy 0, policy_version 1891888 (0.0011) [2023-12-27 05:10:47,451][105692] Updated weights for policy 0, policy_version 1891898 (0.0006) [2023-12-27 05:10:47,469][105620] Updated weights for policy 1, policy_version 1896406 (0.0009) [2023-12-27 05:10:47,513][105692] Updated weights for policy 0, policy_version 1891908 (0.0006) [2023-12-27 05:10:47,520][105620] Updated weights for policy 1, policy_version 1896416 (0.0008) [2023-12-27 05:10:47,573][105692] Updated weights for policy 0, policy_version 1891918 (0.0005) [2023-12-27 05:10:47,581][105620] Updated weights for policy 1, policy_version 1896426 (0.0009) [2023-12-27 05:10:48,155][105692] Updated weights for policy 0, policy_version 1891928 (0.0006) [2023-12-27 05:10:48,214][105692] Updated weights for policy 0, policy_version 1891938 (0.0005) [2023-12-27 05:10:48,274][105692] Updated weights for policy 0, policy_version 1891948 (0.0006) [2023-12-27 05:10:48,430][105620] Updated weights for policy 1, policy_version 1896436 (0.0010) [2023-12-27 05:10:48,488][105620] Updated weights for policy 1, policy_version 1896446 (0.0009) [2023-12-27 05:10:48,559][105620] Updated weights for policy 1, policy_version 1896456 (0.0007) [2023-12-27 05:10:48,853][105692] Updated weights for policy 0, policy_version 1891958 (0.0008) [2023-12-27 05:10:48,903][105692] Updated weights for policy 0, policy_version 1891968 (0.0009) [2023-12-27 05:10:48,951][105692] Updated weights for policy 0, policy_version 1891978 (0.0009) [2023-12-27 05:10:49,331][105620] Updated weights for policy 1, policy_version 1896466 (0.0009) [2023-12-27 05:10:49,393][105620] Updated weights for policy 1, policy_version 1896476 (0.0009) [2023-12-27 05:10:49,451][105620] Updated weights for policy 1, policy_version 1896486 (0.0008) [2023-12-27 05:10:49,513][105620] Updated weights for policy 1, policy_version 1896496 (0.0009) [2023-12-27 05:10:49,734][105692] Updated weights for policy 0, policy_version 1891988 (0.0010) [2023-12-27 05:10:49,780][105692] Updated weights for policy 0, policy_version 1891998 (0.0008) [2023-12-27 05:10:49,837][105692] Updated weights for policy 0, policy_version 1892008 (0.0007) [2023-12-27 05:10:50,248][105620] Updated weights for policy 1, policy_version 1896506 (0.0009) [2023-12-27 05:10:50,302][105620] Updated weights for policy 1, policy_version 1896516 (0.0009) [2023-12-27 05:10:50,362][105620] Updated weights for policy 1, policy_version 1896526 (0.0009) [2023-12-27 05:10:50,586][105692] Updated weights for policy 0, policy_version 1892018 (0.0007) [2023-12-27 05:10:50,647][105692] Updated weights for policy 0, policy_version 1892028 (0.0009) [2023-12-27 05:10:50,700][105692] Updated weights for policy 0, policy_version 1892038 (0.0009) [2023-12-27 05:10:50,760][105692] Updated weights for policy 0, policy_version 1892048 (0.0009) [2023-12-27 05:10:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 970014720. Throughput: 0: 9896.0, 1: 9494.3. Samples: 970006092. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:10:51,062][104569] Avg episode reward: [(0, '7810.848'), (1, '9254.748')] [2023-12-27 05:10:51,138][105620] Updated weights for policy 1, policy_version 1896536 (0.0009) [2023-12-27 05:10:51,192][105620] Updated weights for policy 1, policy_version 1896546 (0.0009) [2023-12-27 05:10:51,244][105620] Updated weights for policy 1, policy_version 1896556 (0.0009) [2023-12-27 05:10:51,499][105692] Updated weights for policy 0, policy_version 1892058 (0.0005) [2023-12-27 05:10:51,552][105692] Updated weights for policy 0, policy_version 1892068 (0.0008) [2023-12-27 05:10:51,613][105692] Updated weights for policy 0, policy_version 1892078 (0.0009) [2023-12-27 05:10:52,081][105620] Updated weights for policy 1, policy_version 1896566 (0.0008) [2023-12-27 05:10:52,136][105620] Updated weights for policy 1, policy_version 1896576 (0.0009) [2023-12-27 05:10:52,183][105620] Updated weights for policy 1, policy_version 1896586 (0.0009) [2023-12-27 05:10:52,350][105692] Updated weights for policy 0, policy_version 1892088 (0.0008) [2023-12-27 05:10:52,419][105692] Updated weights for policy 0, policy_version 1892098 (0.0007) [2023-12-27 05:10:52,483][105692] Updated weights for policy 0, policy_version 1892108 (0.0006) [2023-12-27 05:10:53,007][105620] Updated weights for policy 1, policy_version 1896596 (0.0009) [2023-12-27 05:10:53,058][105620] Updated weights for policy 1, policy_version 1896606 (0.0009) [2023-12-27 05:10:53,109][105620] Updated weights for policy 1, policy_version 1896616 (0.0009) [2023-12-27 05:10:53,160][105692] Updated weights for policy 0, policy_version 1892118 (0.0006) [2023-12-27 05:10:53,216][105692] Updated weights for policy 0, policy_version 1892128 (0.0008) [2023-12-27 05:10:53,283][105692] Updated weights for policy 0, policy_version 1892138 (0.0006) [2023-12-27 05:10:53,906][105692] Updated weights for policy 0, policy_version 1892148 (0.0010) [2023-12-27 05:10:53,945][105620] Updated weights for policy 1, policy_version 1896626 (0.0008) [2023-12-27 05:10:53,964][105692] Updated weights for policy 0, policy_version 1892158 (0.0009) [2023-12-27 05:10:53,993][105620] Updated weights for policy 1, policy_version 1896636 (0.0007) [2023-12-27 05:10:54,020][105692] Updated weights for policy 0, policy_version 1892168 (0.0008) [2023-12-27 05:10:54,040][105620] Updated weights for policy 1, policy_version 1896646 (0.0005) [2023-12-27 05:10:54,090][105620] Updated weights for policy 1, policy_version 1896656 (0.0005) [2023-12-27 05:10:54,729][105620] Updated weights for policy 1, policy_version 1896666 (0.0008) [2023-12-27 05:10:54,776][105620] Updated weights for policy 1, policy_version 1896676 (0.0008) [2023-12-27 05:10:54,795][105692] Updated weights for policy 0, policy_version 1892178 (0.0010) [2023-12-27 05:10:54,836][105620] Updated weights for policy 1, policy_version 1896686 (0.0010) [2023-12-27 05:10:54,850][105692] Updated weights for policy 0, policy_version 1892188 (0.0010) [2023-12-27 05:10:54,912][105692] Updated weights for policy 0, policy_version 1892198 (0.0010) [2023-12-27 05:10:54,972][105692] Updated weights for policy 0, policy_version 1892208 (0.0010) [2023-12-27 05:10:55,543][105620] Updated weights for policy 1, policy_version 1896696 (0.0009) [2023-12-27 05:10:55,601][105620] Updated weights for policy 1, policy_version 1896706 (0.0010) [2023-12-27 05:10:55,648][105692] Updated weights for policy 0, policy_version 1892218 (0.0006) [2023-12-27 05:10:55,650][105620] Updated weights for policy 1, policy_version 1896716 (0.0011) [2023-12-27 05:10:55,698][105692] Updated weights for policy 0, policy_version 1892228 (0.0007) [2023-12-27 05:10:55,742][105692] Updated weights for policy 0, policy_version 1892238 (0.0007) [2023-12-27 05:10:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 970113024. Throughput: 0: 10011.9, 1: 9372.5. Samples: 970120004. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:10:56,062][104569] Avg episode reward: [(0, '7899.942'), (1, '9162.395')] [2023-12-27 05:10:56,417][105620] Updated weights for policy 1, policy_version 1896726 (0.0010) [2023-12-27 05:10:56,451][105692] Updated weights for policy 0, policy_version 1892248 (0.0007) [2023-12-27 05:10:56,476][105620] Updated weights for policy 1, policy_version 1896736 (0.0011) [2023-12-27 05:10:56,502][105692] Updated weights for policy 0, policy_version 1892258 (0.0006) [2023-12-27 05:10:56,531][105620] Updated weights for policy 1, policy_version 1896746 (0.0010) [2023-12-27 05:10:56,557][105692] Updated weights for policy 0, policy_version 1892268 (0.0006) [2023-12-27 05:10:57,268][105620] Updated weights for policy 1, policy_version 1896756 (0.0008) [2023-12-27 05:10:57,319][105692] Updated weights for policy 0, policy_version 1892278 (0.0008) [2023-12-27 05:10:57,323][105620] Updated weights for policy 1, policy_version 1896766 (0.0006) [2023-12-27 05:10:57,372][105692] Updated weights for policy 0, policy_version 1892288 (0.0009) [2023-12-27 05:10:57,377][105620] Updated weights for policy 1, policy_version 1896776 (0.0005) [2023-12-27 05:10:57,419][105692] Updated weights for policy 0, policy_version 1892298 (0.0007) [2023-12-27 05:10:57,911][105620] Updated weights for policy 1, policy_version 1896786 (0.0007) [2023-12-27 05:10:57,983][105620] Updated weights for policy 1, policy_version 1896796 (0.0006) [2023-12-27 05:10:58,032][105620] Updated weights for policy 1, policy_version 1896806 (0.0005) [2023-12-27 05:10:58,081][105620] Updated weights for policy 1, policy_version 1896816 (0.0005) [2023-12-27 05:10:58,289][105692] Updated weights for policy 0, policy_version 1892308 (0.0008) [2023-12-27 05:10:58,356][105692] Updated weights for policy 0, policy_version 1892318 (0.0008) [2023-12-27 05:10:58,417][105692] Updated weights for policy 0, policy_version 1892328 (0.0008) [2023-12-27 05:10:58,784][105620] Updated weights for policy 1, policy_version 1896826 (0.0008) [2023-12-27 05:10:58,850][105620] Updated weights for policy 1, policy_version 1896836 (0.0009) [2023-12-27 05:10:58,921][105620] Updated weights for policy 1, policy_version 1896846 (0.0009) [2023-12-27 05:10:59,171][105692] Updated weights for policy 0, policy_version 1892338 (0.0008) [2023-12-27 05:10:59,227][105692] Updated weights for policy 0, policy_version 1892348 (0.0006) [2023-12-27 05:10:59,293][105692] Updated weights for policy 0, policy_version 1892358 (0.0008) [2023-12-27 05:10:59,360][105692] Updated weights for policy 0, policy_version 1892368 (0.0009) [2023-12-27 05:10:59,675][105620] Updated weights for policy 1, policy_version 1896856 (0.0009) [2023-12-27 05:10:59,740][105620] Updated weights for policy 1, policy_version 1896866 (0.0009) [2023-12-27 05:10:59,796][105620] Updated weights for policy 1, policy_version 1896876 (0.0009) [2023-12-27 05:11:00,028][105692] Updated weights for policy 0, policy_version 1892378 (0.0009) [2023-12-27 05:11:00,087][105692] Updated weights for policy 0, policy_version 1892388 (0.0009) [2023-12-27 05:11:00,152][105692] Updated weights for policy 0, policy_version 1892398 (0.0010) [2023-12-27 05:11:00,524][105620] Updated weights for policy 1, policy_version 1896886 (0.0009) [2023-12-27 05:11:00,581][105620] Updated weights for policy 1, policy_version 1896896 (0.0009) [2023-12-27 05:11:00,637][105620] Updated weights for policy 1, policy_version 1896906 (0.0007) [2023-12-27 05:11:00,867][105692] Updated weights for policy 0, policy_version 1892408 (0.0006) [2023-12-27 05:11:00,920][105692] Updated weights for policy 0, policy_version 1892418 (0.0009) [2023-12-27 05:11:00,974][105692] Updated weights for policy 0, policy_version 1892428 (0.0010) [2023-12-27 05:11:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 970211328. Throughput: 0: 9969.0, 1: 9419.2. Samples: 970177712. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:11:01,062][104569] Avg episode reward: [(0, '8170.568'), (1, '9253.293')] [2023-12-27 05:11:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001892432_484532224.pth... [2023-12-27 05:11:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001896912_485679104.pth... [2023-12-27 05:11:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001891248_484229120.pth [2023-12-27 05:11:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001895824_485400576.pth [2023-12-27 05:11:01,270][105620] Updated weights for policy 1, policy_version 1896916 (0.0008) [2023-12-27 05:11:01,325][105620] Updated weights for policy 1, policy_version 1896926 (0.0010) [2023-12-27 05:11:01,392][105620] Updated weights for policy 1, policy_version 1896936 (0.0011) [2023-12-27 05:11:01,801][105692] Updated weights for policy 0, policy_version 1892439 (0.0009) [2023-12-27 05:11:01,860][105692] Updated weights for policy 0, policy_version 1892449 (0.0009) [2023-12-27 05:11:01,914][105692] Updated weights for policy 0, policy_version 1892459 (0.0008) [2023-12-27 05:11:02,152][105620] Updated weights for policy 1, policy_version 1896946 (0.0008) [2023-12-27 05:11:02,205][105620] Updated weights for policy 1, policy_version 1896956 (0.0008) [2023-12-27 05:11:02,268][105620] Updated weights for policy 1, policy_version 1896966 (0.0009) [2023-12-27 05:11:02,322][105620] Updated weights for policy 1, policy_version 1896976 (0.0010) [2023-12-27 05:11:02,696][105692] Updated weights for policy 0, policy_version 1892469 (0.0008) [2023-12-27 05:11:02,747][105692] Updated weights for policy 0, policy_version 1892479 (0.0010) [2023-12-27 05:11:02,801][105692] Updated weights for policy 0, policy_version 1892489 (0.0010) [2023-12-27 05:11:02,956][105620] Updated weights for policy 1, policy_version 1896986 (0.0006) [2023-12-27 05:11:03,008][105620] Updated weights for policy 1, policy_version 1896996 (0.0005) [2023-12-27 05:11:03,066][105620] Updated weights for policy 1, policy_version 1897006 (0.0008) [2023-12-27 05:11:03,585][105692] Updated weights for policy 0, policy_version 1892500 (0.0010) [2023-12-27 05:11:03,646][105692] Updated weights for policy 0, policy_version 1892510 (0.0009) [2023-12-27 05:11:03,709][105692] Updated weights for policy 0, policy_version 1892520 (0.0009) [2023-12-27 05:11:03,754][105620] Updated weights for policy 1, policy_version 1897016 (0.0007) [2023-12-27 05:11:03,816][105620] Updated weights for policy 1, policy_version 1897026 (0.0009) [2023-12-27 05:11:03,883][105620] Updated weights for policy 1, policy_version 1897036 (0.0010) [2023-12-27 05:11:04,440][105692] Updated weights for policy 0, policy_version 1892530 (0.0009) [2023-12-27 05:11:04,489][105692] Updated weights for policy 0, policy_version 1892540 (0.0010) [2023-12-27 05:11:04,539][105692] Updated weights for policy 0, policy_version 1892550 (0.0008) [2023-12-27 05:11:04,580][105620] Updated weights for policy 1, policy_version 1897046 (0.0009) [2023-12-27 05:11:04,587][105692] Updated weights for policy 0, policy_version 1892560 (0.0007) [2023-12-27 05:11:04,630][105620] Updated weights for policy 1, policy_version 1897056 (0.0008) [2023-12-27 05:11:04,680][105620] Updated weights for policy 1, policy_version 1897066 (0.0009) [2023-12-27 05:11:05,279][105692] Updated weights for policy 0, policy_version 1892570 (0.0005) [2023-12-27 05:11:05,334][105692] Updated weights for policy 0, policy_version 1892580 (0.0005) [2023-12-27 05:11:05,373][105620] Updated weights for policy 1, policy_version 1897076 (0.0008) [2023-12-27 05:11:05,385][105692] Updated weights for policy 0, policy_version 1892590 (0.0007) [2023-12-27 05:11:05,430][105620] Updated weights for policy 1, policy_version 1897086 (0.0010) [2023-12-27 05:11:05,478][105620] Updated weights for policy 1, policy_version 1897096 (0.0010) [2023-12-27 05:11:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 970301440. Throughput: 0: 9930.3, 1: 9469.5. Samples: 970293588. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:11:06,062][104569] Avg episode reward: [(0, '8085.763'), (1, '9345.637')] [2023-12-27 05:11:06,089][105692] Updated weights for policy 0, policy_version 1892600 (0.0008) [2023-12-27 05:11:06,150][105692] Updated weights for policy 0, policy_version 1892610 (0.0008) [2023-12-27 05:11:06,208][105692] Updated weights for policy 0, policy_version 1892620 (0.0006) [2023-12-27 05:11:06,215][105620] Updated weights for policy 1, policy_version 1897106 (0.0010) [2023-12-27 05:11:06,272][105620] Updated weights for policy 1, policy_version 1897116 (0.0011) [2023-12-27 05:11:06,332][105620] Updated weights for policy 1, policy_version 1897126 (0.0011) [2023-12-27 05:11:06,398][105620] Updated weights for policy 1, policy_version 1897136 (0.0011) [2023-12-27 05:11:06,890][105692] Updated weights for policy 0, policy_version 1892630 (0.0009) [2023-12-27 05:11:06,948][105692] Updated weights for policy 0, policy_version 1892640 (0.0010) [2023-12-27 05:11:07,010][105692] Updated weights for policy 0, policy_version 1892650 (0.0011) [2023-12-27 05:11:07,143][105620] Updated weights for policy 1, policy_version 1897146 (0.0010) [2023-12-27 05:11:07,192][105620] Updated weights for policy 1, policy_version 1897156 (0.0010) [2023-12-27 05:11:07,245][105620] Updated weights for policy 1, policy_version 1897166 (0.0011) [2023-12-27 05:11:07,662][105692] Updated weights for policy 0, policy_version 1892660 (0.0008) [2023-12-27 05:11:07,719][105692] Updated weights for policy 0, policy_version 1892670 (0.0008) [2023-12-27 05:11:07,781][105692] Updated weights for policy 0, policy_version 1892680 (0.0011) [2023-12-27 05:11:07,928][105620] Updated weights for policy 1, policy_version 1897176 (0.0006) [2023-12-27 05:11:07,993][105620] Updated weights for policy 1, policy_version 1897186 (0.0005) [2023-12-27 05:11:08,053][105620] Updated weights for policy 1, policy_version 1897196 (0.0008) [2023-12-27 05:11:08,445][105692] Updated weights for policy 0, policy_version 1892690 (0.0010) [2023-12-27 05:11:08,495][105692] Updated weights for policy 0, policy_version 1892700 (0.0008) [2023-12-27 05:11:08,548][105692] Updated weights for policy 0, policy_version 1892710 (0.0008) [2023-12-27 05:11:08,600][105692] Updated weights for policy 0, policy_version 1892720 (0.0008) [2023-12-27 05:11:08,754][105620] Updated weights for policy 1, policy_version 1897206 (0.0011) [2023-12-27 05:11:08,804][105620] Updated weights for policy 1, policy_version 1897216 (0.0011) [2023-12-27 05:11:08,850][105620] Updated weights for policy 1, policy_version 1897226 (0.0011) [2023-12-27 05:11:09,268][105692] Updated weights for policy 0, policy_version 1892730 (0.0008) [2023-12-27 05:11:09,322][105692] Updated weights for policy 0, policy_version 1892740 (0.0008) [2023-12-27 05:11:09,389][105692] Updated weights for policy 0, policy_version 1892750 (0.0009) [2023-12-27 05:11:09,652][105620] Updated weights for policy 1, policy_version 1897236 (0.0010) [2023-12-27 05:11:09,701][105620] Updated weights for policy 1, policy_version 1897246 (0.0009) [2023-12-27 05:11:09,753][105620] Updated weights for policy 1, policy_version 1897256 (0.0009) [2023-12-27 05:11:10,099][105692] Updated weights for policy 0, policy_version 1892760 (0.0009) [2023-12-27 05:11:10,159][105692] Updated weights for policy 0, policy_version 1892770 (0.0009) [2023-12-27 05:11:10,219][105692] Updated weights for policy 0, policy_version 1892780 (0.0009) [2023-12-27 05:11:10,562][105620] Updated weights for policy 1, policy_version 1897266 (0.0009) [2023-12-27 05:11:10,626][105620] Updated weights for policy 1, policy_version 1897276 (0.0009) [2023-12-27 05:11:10,679][105620] Updated weights for policy 1, policy_version 1897286 (0.0010) [2023-12-27 05:11:10,731][105620] Updated weights for policy 1, policy_version 1897296 (0.0009) [2023-12-27 05:11:10,891][105692] Updated weights for policy 0, policy_version 1892790 (0.0009) [2023-12-27 05:11:10,953][105692] Updated weights for policy 0, policy_version 1892800 (0.0008) [2023-12-27 05:11:11,021][105692] Updated weights for policy 0, policy_version 1892810 (0.0010) [2023-12-27 05:11:11,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.8, 300 sec: 19438.7). Total num frames: 970399744. Throughput: 0: 9919.6, 1: 9534.1. Samples: 970411184. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:11:11,063][104569] Avg episode reward: [(0, '8543.164'), (1, '9345.546')] [2023-12-27 05:11:11,578][105620] Updated weights for policy 1, policy_version 1897306 (0.0009) [2023-12-27 05:11:11,638][105620] Updated weights for policy 1, policy_version 1897316 (0.0009) [2023-12-27 05:11:11,711][105620] Updated weights for policy 1, policy_version 1897326 (0.0007) [2023-12-27 05:11:11,812][105692] Updated weights for policy 0, policy_version 1892820 (0.0009) [2023-12-27 05:11:11,861][105692] Updated weights for policy 0, policy_version 1892830 (0.0009) [2023-12-27 05:11:11,909][105692] Updated weights for policy 0, policy_version 1892840 (0.0009) [2023-12-27 05:11:12,440][105620] Updated weights for policy 1, policy_version 1897336 (0.0008) [2023-12-27 05:11:12,488][105620] Updated weights for policy 1, policy_version 1897346 (0.0009) [2023-12-27 05:11:12,535][105620] Updated weights for policy 1, policy_version 1897356 (0.0009) [2023-12-27 05:11:12,677][105692] Updated weights for policy 0, policy_version 1892850 (0.0009) [2023-12-27 05:11:12,738][105692] Updated weights for policy 0, policy_version 1892860 (0.0009) [2023-12-27 05:11:12,796][105692] Updated weights for policy 0, policy_version 1892870 (0.0009) [2023-12-27 05:11:12,846][105692] Updated weights for policy 0, policy_version 1892880 (0.0008) [2023-12-27 05:11:13,329][105620] Updated weights for policy 1, policy_version 1897366 (0.0010) [2023-12-27 05:11:13,373][105620] Updated weights for policy 1, policy_version 1897376 (0.0010) [2023-12-27 05:11:13,420][105620] Updated weights for policy 1, policy_version 1897386 (0.0010) [2023-12-27 05:11:13,573][105692] Updated weights for policy 0, policy_version 1892890 (0.0009) [2023-12-27 05:11:13,621][105692] Updated weights for policy 0, policy_version 1892900 (0.0009) [2023-12-27 05:11:13,669][105692] Updated weights for policy 0, policy_version 1892910 (0.0009) [2023-12-27 05:11:14,158][105620] Updated weights for policy 1, policy_version 1897396 (0.0009) [2023-12-27 05:11:14,213][105620] Updated weights for policy 1, policy_version 1897406 (0.0006) [2023-12-27 05:11:14,262][105620] Updated weights for policy 1, policy_version 1897416 (0.0008) [2023-12-27 05:11:14,463][105692] Updated weights for policy 0, policy_version 1892920 (0.0009) [2023-12-27 05:11:14,522][105692] Updated weights for policy 0, policy_version 1892930 (0.0009) [2023-12-27 05:11:14,570][105692] Updated weights for policy 0, policy_version 1892940 (0.0009) [2023-12-27 05:11:14,997][105620] Updated weights for policy 1, policy_version 1897426 (0.0008) [2023-12-27 05:11:15,063][105620] Updated weights for policy 1, policy_version 1897436 (0.0006) [2023-12-27 05:11:15,134][105620] Updated weights for policy 1, policy_version 1897446 (0.0006) [2023-12-27 05:11:15,203][105620] Updated weights for policy 1, policy_version 1897456 (0.0008) [2023-12-27 05:11:15,370][105692] Updated weights for policy 0, policy_version 1892950 (0.0009) [2023-12-27 05:11:15,425][105692] Updated weights for policy 0, policy_version 1892960 (0.0009) [2023-12-27 05:11:15,490][105692] Updated weights for policy 0, policy_version 1892970 (0.0010) [2023-12-27 05:11:15,865][105620] Updated weights for policy 1, policy_version 1897466 (0.0008) [2023-12-27 05:11:15,928][105620] Updated weights for policy 1, policy_version 1897476 (0.0005) [2023-12-27 05:11:15,992][105620] Updated weights for policy 1, policy_version 1897486 (0.0005) [2023-12-27 05:11:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19438.7). Total num frames: 970498048. Throughput: 0: 9842.1, 1: 9499.0. Samples: 970467424. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:11:16,062][104569] Avg episode reward: [(0, '8544.301'), (1, '9254.486')] [2023-12-27 05:11:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001892976_484671488.pth... [2023-12-27 05:11:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001897488_485826560.pth... [2023-12-27 05:11:16,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001891856_484384768.pth [2023-12-27 05:11:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001896368_485539840.pth [2023-12-27 05:11:16,289][105692] Updated weights for policy 0, policy_version 1892980 (0.0007) [2023-12-27 05:11:16,349][105692] Updated weights for policy 0, policy_version 1892990 (0.0007) [2023-12-27 05:11:16,409][105692] Updated weights for policy 0, policy_version 1893000 (0.0008) [2023-12-27 05:11:16,689][105620] Updated weights for policy 1, policy_version 1897496 (0.0005) [2023-12-27 05:11:16,754][105620] Updated weights for policy 1, policy_version 1897506 (0.0005) [2023-12-27 05:11:16,805][105620] Updated weights for policy 1, policy_version 1897516 (0.0007) [2023-12-27 05:11:17,207][105692] Updated weights for policy 0, policy_version 1893010 (0.0008) [2023-12-27 05:11:17,267][105692] Updated weights for policy 0, policy_version 1893020 (0.0008) [2023-12-27 05:11:17,319][105692] Updated weights for policy 0, policy_version 1893030 (0.0008) [2023-12-27 05:11:17,368][105692] Updated weights for policy 0, policy_version 1893040 (0.0008) [2023-12-27 05:11:17,462][105620] Updated weights for policy 1, policy_version 1897526 (0.0010) [2023-12-27 05:11:17,510][105620] Updated weights for policy 1, policy_version 1897536 (0.0010) [2023-12-27 05:11:17,561][105620] Updated weights for policy 1, policy_version 1897546 (0.0010) [2023-12-27 05:11:18,113][105692] Updated weights for policy 0, policy_version 1893050 (0.0009) [2023-12-27 05:11:18,172][105692] Updated weights for policy 0, policy_version 1893060 (0.0009) [2023-12-27 05:11:18,226][105692] Updated weights for policy 0, policy_version 1893070 (0.0010) [2023-12-27 05:11:18,312][105620] Updated weights for policy 1, policy_version 1897556 (0.0009) [2023-12-27 05:11:18,374][105620] Updated weights for policy 1, policy_version 1897566 (0.0008) [2023-12-27 05:11:18,433][105620] Updated weights for policy 1, policy_version 1897576 (0.0009) [2023-12-27 05:11:19,075][105692] Updated weights for policy 0, policy_version 1893080 (0.0010) [2023-12-27 05:11:19,078][105620] Updated weights for policy 1, policy_version 1897586 (0.0009) [2023-12-27 05:11:19,129][105692] Updated weights for policy 0, policy_version 1893090 (0.0008) [2023-12-27 05:11:19,130][105620] Updated weights for policy 1, policy_version 1897596 (0.0007) [2023-12-27 05:11:19,179][105620] Updated weights for policy 1, policy_version 1897606 (0.0006) [2023-12-27 05:11:19,183][105692] Updated weights for policy 0, policy_version 1893100 (0.0008) [2023-12-27 05:11:19,246][105620] Updated weights for policy 1, policy_version 1897616 (0.0007) [2023-12-27 05:11:19,852][105620] Updated weights for policy 1, policy_version 1897626 (0.0009) [2023-12-27 05:11:19,920][105620] Updated weights for policy 1, policy_version 1897636 (0.0009) [2023-12-27 05:11:19,978][105620] Updated weights for policy 1, policy_version 1897646 (0.0008) [2023-12-27 05:11:20,022][105692] Updated weights for policy 0, policy_version 1893110 (0.0010) [2023-12-27 05:11:20,080][105692] Updated weights for policy 0, policy_version 1893120 (0.0008) [2023-12-27 05:11:20,144][105692] Updated weights for policy 0, policy_version 1893130 (0.0008) [2023-12-27 05:11:20,592][105620] Updated weights for policy 1, policy_version 1897656 (0.0009) [2023-12-27 05:11:20,649][105620] Updated weights for policy 1, policy_version 1897666 (0.0010) [2023-12-27 05:11:20,701][105620] Updated weights for policy 1, policy_version 1897676 (0.0010) [2023-12-27 05:11:20,990][105692] Updated weights for policy 0, policy_version 1893140 (0.0009) [2023-12-27 05:11:21,054][105692] Updated weights for policy 0, policy_version 1893150 (0.0008) [2023-12-27 05:11:21,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 970588160. Throughput: 0: 9749.6, 1: 9592.1. Samples: 970581628. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:11:21,062][104569] Avg episode reward: [(0, '8271.443'), (1, '9069.738')] [2023-12-27 05:11:21,124][105692] Updated weights for policy 0, policy_version 1893160 (0.0009) [2023-12-27 05:11:21,397][105620] Updated weights for policy 1, policy_version 1897686 (0.0009) [2023-12-27 05:11:21,457][105620] Updated weights for policy 1, policy_version 1897696 (0.0008) [2023-12-27 05:11:21,508][105620] Updated weights for policy 1, policy_version 1897706 (0.0005) [2023-12-27 05:11:21,896][105692] Updated weights for policy 0, policy_version 1893170 (0.0008) [2023-12-27 05:11:21,960][105692] Updated weights for policy 0, policy_version 1893180 (0.0008) [2023-12-27 05:11:22,023][105692] Updated weights for policy 0, policy_version 1893190 (0.0008) [2023-12-27 05:11:22,082][105692] Updated weights for policy 0, policy_version 1893200 (0.0008) [2023-12-27 05:11:22,138][105620] Updated weights for policy 1, policy_version 1897716 (0.0007) [2023-12-27 05:11:22,202][105620] Updated weights for policy 1, policy_version 1897726 (0.0008) [2023-12-27 05:11:22,253][105620] Updated weights for policy 1, policy_version 1897736 (0.0008) [2023-12-27 05:11:22,852][105692] Updated weights for policy 0, policy_version 1893210 (0.0006) [2023-12-27 05:11:22,922][105692] Updated weights for policy 0, policy_version 1893220 (0.0007) [2023-12-27 05:11:22,981][105620] Updated weights for policy 1, policy_version 1897746 (0.0007) [2023-12-27 05:11:22,991][105692] Updated weights for policy 0, policy_version 1893230 (0.0008) [2023-12-27 05:11:23,044][105620] Updated weights for policy 1, policy_version 1897756 (0.0006) [2023-12-27 05:11:23,095][105620] Updated weights for policy 1, policy_version 1897766 (0.0006) [2023-12-27 05:11:23,152][105620] Updated weights for policy 1, policy_version 1897776 (0.0008) [2023-12-27 05:11:23,680][105692] Updated weights for policy 0, policy_version 1893240 (0.0008) [2023-12-27 05:11:23,744][105692] Updated weights for policy 0, policy_version 1893250 (0.0010) [2023-12-27 05:11:23,809][105692] Updated weights for policy 0, policy_version 1893260 (0.0010) [2023-12-27 05:11:23,864][105620] Updated weights for policy 1, policy_version 1897786 (0.0005) [2023-12-27 05:11:23,918][105620] Updated weights for policy 1, policy_version 1897796 (0.0006) [2023-12-27 05:11:23,971][105620] Updated weights for policy 1, policy_version 1897806 (0.0005) [2023-12-27 05:11:24,620][105620] Updated weights for policy 1, policy_version 1897816 (0.0005) [2023-12-27 05:11:24,633][105692] Updated weights for policy 0, policy_version 1893270 (0.0010) [2023-12-27 05:11:24,679][105692] Updated weights for policy 0, policy_version 1893280 (0.0008) [2023-12-27 05:11:24,687][105620] Updated weights for policy 1, policy_version 1897826 (0.0006) [2023-12-27 05:11:24,738][105692] Updated weights for policy 0, policy_version 1893290 (0.0008) [2023-12-27 05:11:24,748][105620] Updated weights for policy 1, policy_version 1897836 (0.0007) [2023-12-27 05:11:25,466][105692] Updated weights for policy 0, policy_version 1893300 (0.0007) [2023-12-27 05:11:25,468][105620] Updated weights for policy 1, policy_version 1897846 (0.0007) [2023-12-27 05:11:25,525][105692] Updated weights for policy 0, policy_version 1893310 (0.0006) [2023-12-27 05:11:25,530][105620] Updated weights for policy 1, policy_version 1897856 (0.0010) [2023-12-27 05:11:25,579][105692] Updated weights for policy 0, policy_version 1893321 (0.0006) [2023-12-27 05:11:25,591][105620] Updated weights for policy 1, policy_version 1897866 (0.0010) [2023-12-27 05:11:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 970686464. Throughput: 0: 9614.6, 1: 9685.4. Samples: 970696672. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:11:26,062][104569] Avg episode reward: [(0, '8447.816'), (1, '8976.487')] [2023-12-27 05:11:26,140][105692] Updated weights for policy 0, policy_version 1893331 (0.0007) [2023-12-27 05:11:26,197][105692] Updated weights for policy 0, policy_version 1893341 (0.0008) [2023-12-27 05:11:26,259][105692] Updated weights for policy 0, policy_version 1893351 (0.0007) [2023-12-27 05:11:26,278][105620] Updated weights for policy 1, policy_version 1897876 (0.0008) [2023-12-27 05:11:26,343][105620] Updated weights for policy 1, policy_version 1897886 (0.0009) [2023-12-27 05:11:26,408][105620] Updated weights for policy 1, policy_version 1897896 (0.0010) [2023-12-27 05:11:26,864][105692] Updated weights for policy 0, policy_version 1893361 (0.0006) [2023-12-27 05:11:26,917][105692] Updated weights for policy 0, policy_version 1893371 (0.0009) [2023-12-27 05:11:26,967][105692] Updated weights for policy 0, policy_version 1893381 (0.0008) [2023-12-27 05:11:27,024][105692] Updated weights for policy 0, policy_version 1893391 (0.0007) [2023-12-27 05:11:27,149][105620] Updated weights for policy 1, policy_version 1897906 (0.0010) [2023-12-27 05:11:27,206][105620] Updated weights for policy 1, policy_version 1897916 (0.0010) [2023-12-27 05:11:27,258][105620] Updated weights for policy 1, policy_version 1897926 (0.0009) [2023-12-27 05:11:27,310][105620] Updated weights for policy 1, policy_version 1897936 (0.0008) [2023-12-27 05:11:27,750][105692] Updated weights for policy 0, policy_version 1893401 (0.0009) [2023-12-27 05:11:27,819][105692] Updated weights for policy 0, policy_version 1893411 (0.0010) [2023-12-27 05:11:27,881][105692] Updated weights for policy 0, policy_version 1893421 (0.0007) [2023-12-27 05:11:27,896][105620] Updated weights for policy 1, policy_version 1897946 (0.0007) [2023-12-27 05:11:27,958][105620] Updated weights for policy 1, policy_version 1897956 (0.0006) [2023-12-27 05:11:28,028][105620] Updated weights for policy 1, policy_version 1897966 (0.0005) [2023-12-27 05:11:28,617][105692] Updated weights for policy 0, policy_version 1893431 (0.0008) [2023-12-27 05:11:28,673][105692] Updated weights for policy 0, policy_version 1893441 (0.0009) [2023-12-27 05:11:28,694][105620] Updated weights for policy 1, policy_version 1897976 (0.0005) [2023-12-27 05:11:28,729][105692] Updated weights for policy 0, policy_version 1893451 (0.0009) [2023-12-27 05:11:28,742][105620] Updated weights for policy 1, policy_version 1897986 (0.0005) [2023-12-27 05:11:28,799][105620] Updated weights for policy 1, policy_version 1897996 (0.0005) [2023-12-27 05:11:29,496][105620] Updated weights for policy 1, policy_version 1898006 (0.0008) [2023-12-27 05:11:29,530][105692] Updated weights for policy 0, policy_version 1893461 (0.0008) [2023-12-27 05:11:29,552][105620] Updated weights for policy 1, policy_version 1898016 (0.0007) [2023-12-27 05:11:29,590][105692] Updated weights for policy 0, policy_version 1893471 (0.0008) [2023-12-27 05:11:29,614][105620] Updated weights for policy 1, policy_version 1898026 (0.0006) [2023-12-27 05:11:29,648][105692] Updated weights for policy 0, policy_version 1893481 (0.0007) [2023-12-27 05:11:30,369][105620] Updated weights for policy 1, policy_version 1898036 (0.0007) [2023-12-27 05:11:30,380][105692] Updated weights for policy 0, policy_version 1893491 (0.0008) [2023-12-27 05:11:30,414][105620] Updated weights for policy 1, policy_version 1898046 (0.0007) [2023-12-27 05:11:30,429][105692] Updated weights for policy 0, policy_version 1893501 (0.0007) [2023-12-27 05:11:30,471][105620] Updated weights for policy 1, policy_version 1898056 (0.0007) [2023-12-27 05:11:30,478][105692] Updated weights for policy 0, policy_version 1893511 (0.0006) [2023-12-27 05:11:31,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 970784768. Throughput: 0: 9632.0, 1: 9735.0. Samples: 970758556. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 05:11:31,063][104569] Avg episode reward: [(0, '8538.845'), (1, '8976.456')] [2023-12-27 05:11:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001893520_484810752.pth... [2023-12-27 05:11:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001898064_485974016.pth... [2023-12-27 05:11:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001896912_485679104.pth [2023-12-27 05:11:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001892432_484532224.pth [2023-12-27 05:11:31,113][105692] Updated weights for policy 0, policy_version 1893521 (0.0008) [2023-12-27 05:11:31,182][105692] Updated weights for policy 0, policy_version 1893531 (0.0007) [2023-12-27 05:11:31,240][105692] Updated weights for policy 0, policy_version 1893541 (0.0006) [2023-12-27 05:11:31,304][105620] Updated weights for policy 1, policy_version 1898066 (0.0008) [2023-12-27 05:11:31,306][105692] Updated weights for policy 0, policy_version 1893551 (0.0009) [2023-12-27 05:11:31,371][105620] Updated weights for policy 1, policy_version 1898076 (0.0008) [2023-12-27 05:11:31,424][105620] Updated weights for policy 1, policy_version 1898086 (0.0008) [2023-12-27 05:11:31,485][105620] Updated weights for policy 1, policy_version 1898096 (0.0008) [2023-12-27 05:11:32,000][105692] Updated weights for policy 0, policy_version 1893561 (0.0007) [2023-12-27 05:11:32,058][105692] Updated weights for policy 0, policy_version 1893571 (0.0009) [2023-12-27 05:11:32,124][105692] Updated weights for policy 0, policy_version 1893581 (0.0009) [2023-12-27 05:11:32,259][105620] Updated weights for policy 1, policy_version 1898106 (0.0006) [2023-12-27 05:11:32,315][105620] Updated weights for policy 1, policy_version 1898116 (0.0009) [2023-12-27 05:11:32,384][105620] Updated weights for policy 1, policy_version 1898126 (0.0008) [2023-12-27 05:11:32,781][105692] Updated weights for policy 0, policy_version 1893591 (0.0010) [2023-12-27 05:11:32,830][105692] Updated weights for policy 0, policy_version 1893601 (0.0008) [2023-12-27 05:11:32,880][105692] Updated weights for policy 0, policy_version 1893611 (0.0007) [2023-12-27 05:11:32,976][105620] Updated weights for policy 1, policy_version 1898136 (0.0008) [2023-12-27 05:11:33,022][105620] Updated weights for policy 1, policy_version 1898146 (0.0008) [2023-12-27 05:11:33,074][105620] Updated weights for policy 1, policy_version 1898156 (0.0010) [2023-12-27 05:11:33,519][105692] Updated weights for policy 0, policy_version 1893622 (0.0008) [2023-12-27 05:11:33,569][105692] Updated weights for policy 0, policy_version 1893632 (0.0005) [2023-12-27 05:11:33,626][105692] Updated weights for policy 0, policy_version 1893642 (0.0005) [2023-12-27 05:11:33,658][105620] Updated weights for policy 1, policy_version 1898166 (0.0007) [2023-12-27 05:11:33,704][105620] Updated weights for policy 1, policy_version 1898176 (0.0005) [2023-12-27 05:11:33,758][105620] Updated weights for policy 1, policy_version 1898186 (0.0009) [2023-12-27 05:11:34,186][105692] Updated weights for policy 0, policy_version 1893652 (0.0007) [2023-12-27 05:11:34,236][105692] Updated weights for policy 0, policy_version 1893662 (0.0006) [2023-12-27 05:11:34,301][105692] Updated weights for policy 0, policy_version 1893672 (0.0009) [2023-12-27 05:11:34,360][105620] Updated weights for policy 1, policy_version 1898196 (0.0008) [2023-12-27 05:11:34,439][105620] Updated weights for policy 1, policy_version 1898206 (0.0009) [2023-12-27 05:11:34,494][105620] Updated weights for policy 1, policy_version 1898216 (0.0009) [2023-12-27 05:11:34,999][105692] Updated weights for policy 0, policy_version 1893682 (0.0008) [2023-12-27 05:11:35,048][105692] Updated weights for policy 0, policy_version 1893692 (0.0005) [2023-12-27 05:11:35,098][105692] Updated weights for policy 0, policy_version 1893702 (0.0005) [2023-12-27 05:11:35,151][105692] Updated weights for policy 0, policy_version 1893712 (0.0005) [2023-12-27 05:11:35,217][105620] Updated weights for policy 1, policy_version 1898226 (0.0009) [2023-12-27 05:11:35,288][105620] Updated weights for policy 1, policy_version 1898236 (0.0006) [2023-12-27 05:11:35,351][105620] Updated weights for policy 1, policy_version 1898246 (0.0009) [2023-12-27 05:11:35,412][105620] Updated weights for policy 1, policy_version 1898256 (0.0009) [2023-12-27 05:11:35,689][105692] Updated weights for policy 0, policy_version 1893722 (0.0010) [2023-12-27 05:11:35,743][105692] Updated weights for policy 0, policy_version 1893732 (0.0010) [2023-12-27 05:11:35,796][105692] Updated weights for policy 0, policy_version 1893742 (0.0010) [2023-12-27 05:11:36,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 970891264. Throughput: 0: 9583.3, 1: 9828.4. Samples: 970879620. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:11:36,063][104569] Avg episode reward: [(0, '8533.972'), (1, '9160.563')] [2023-12-27 05:11:36,087][105620] Updated weights for policy 1, policy_version 1898266 (0.0006) [2023-12-27 05:11:36,148][105620] Updated weights for policy 1, policy_version 1898276 (0.0008) [2023-12-27 05:11:36,211][105620] Updated weights for policy 1, policy_version 1898286 (0.0006) [2023-12-27 05:11:36,655][105692] Updated weights for policy 0, policy_version 1893752 (0.0010) [2023-12-27 05:11:36,708][105692] Updated weights for policy 0, policy_version 1893762 (0.0009) [2023-12-27 05:11:36,759][105692] Updated weights for policy 0, policy_version 1893772 (0.0009) [2023-12-27 05:11:36,810][105620] Updated weights for policy 1, policy_version 1898296 (0.0008) [2023-12-27 05:11:36,868][105620] Updated weights for policy 1, policy_version 1898306 (0.0009) [2023-12-27 05:11:36,915][105620] Updated weights for policy 1, policy_version 1898316 (0.0009) [2023-12-27 05:11:37,458][105692] Updated weights for policy 0, policy_version 1893782 (0.0009) [2023-12-27 05:11:37,513][105692] Updated weights for policy 0, policy_version 1893792 (0.0009) [2023-12-27 05:11:37,573][105692] Updated weights for policy 0, policy_version 1893802 (0.0009) [2023-12-27 05:11:37,708][105620] Updated weights for policy 1, policy_version 1898326 (0.0007) [2023-12-27 05:11:37,765][105620] Updated weights for policy 1, policy_version 1898336 (0.0009) [2023-12-27 05:11:37,814][105620] Updated weights for policy 1, policy_version 1898346 (0.0011) [2023-12-27 05:11:38,360][105692] Updated weights for policy 0, policy_version 1893812 (0.0009) [2023-12-27 05:11:38,423][105692] Updated weights for policy 0, policy_version 1893822 (0.0007) [2023-12-27 05:11:38,475][105692] Updated weights for policy 0, policy_version 1893832 (0.0008) [2023-12-27 05:11:38,492][105620] Updated weights for policy 1, policy_version 1898356 (0.0010) [2023-12-27 05:11:38,557][105620] Updated weights for policy 1, policy_version 1898366 (0.0010) [2023-12-27 05:11:38,619][105620] Updated weights for policy 1, policy_version 1898376 (0.0010) [2023-12-27 05:11:39,260][105692] Updated weights for policy 0, policy_version 1893842 (0.0010) [2023-12-27 05:11:39,321][105692] Updated weights for policy 0, policy_version 1893852 (0.0009) [2023-12-27 05:11:39,349][105620] Updated weights for policy 1, policy_version 1898386 (0.0010) [2023-12-27 05:11:39,391][105692] Updated weights for policy 0, policy_version 1893862 (0.0009) [2023-12-27 05:11:39,417][105620] Updated weights for policy 1, policy_version 1898396 (0.0009) [2023-12-27 05:11:39,453][105692] Updated weights for policy 0, policy_version 1893872 (0.0009) [2023-12-27 05:11:39,481][105620] Updated weights for policy 1, policy_version 1898406 (0.0009) [2023-12-27 05:11:39,539][105620] Updated weights for policy 1, policy_version 1898416 (0.0006) [2023-12-27 05:11:40,230][105620] Updated weights for policy 1, policy_version 1898426 (0.0008) [2023-12-27 05:11:40,238][105692] Updated weights for policy 0, policy_version 1893882 (0.0006) [2023-12-27 05:11:40,290][105620] Updated weights for policy 1, policy_version 1898436 (0.0007) [2023-12-27 05:11:40,306][105692] Updated weights for policy 0, policy_version 1893892 (0.0006) [2023-12-27 05:11:40,355][105620] Updated weights for policy 1, policy_version 1898446 (0.0007) [2023-12-27 05:11:40,366][105692] Updated weights for policy 0, policy_version 1893902 (0.0006) [2023-12-27 05:11:40,967][105620] Updated weights for policy 1, policy_version 1898457 (0.0007) [2023-12-27 05:11:41,019][105620] Updated weights for policy 1, policy_version 1898467 (0.0008) [2023-12-27 05:11:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 970981376. Throughput: 0: 9555.9, 1: 9920.5. Samples: 970996444. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:11:41,063][104569] Avg episode reward: [(0, '8626.305'), (1, '9255.140')] [2023-12-27 05:11:41,091][105620] Updated weights for policy 1, policy_version 1898477 (0.0007) [2023-12-27 05:11:41,107][105692] Updated weights for policy 0, policy_version 1893912 (0.0007) [2023-12-27 05:11:41,172][105692] Updated weights for policy 0, policy_version 1893922 (0.0009) [2023-12-27 05:11:41,237][105692] Updated weights for policy 0, policy_version 1893932 (0.0010) [2023-12-27 05:11:41,880][105620] Updated weights for policy 1, policy_version 1898487 (0.0009) [2023-12-27 05:11:41,940][105620] Updated weights for policy 1, policy_version 1898497 (0.0008) [2023-12-27 05:11:41,993][105620] Updated weights for policy 1, policy_version 1898507 (0.0008) [2023-12-27 05:11:42,019][105692] Updated weights for policy 0, policy_version 1893942 (0.0010) [2023-12-27 05:11:42,068][105692] Updated weights for policy 0, policy_version 1893952 (0.0010) [2023-12-27 05:11:42,123][105692] Updated weights for policy 0, policy_version 1893962 (0.0010) [2023-12-27 05:11:42,782][105620] Updated weights for policy 1, policy_version 1898517 (0.0007) [2023-12-27 05:11:42,843][105620] Updated weights for policy 1, policy_version 1898527 (0.0008) [2023-12-27 05:11:42,888][105692] Updated weights for policy 0, policy_version 1893972 (0.0011) [2023-12-27 05:11:42,902][105620] Updated weights for policy 1, policy_version 1898537 (0.0007) [2023-12-27 05:11:42,944][105692] Updated weights for policy 0, policy_version 1893982 (0.0010) [2023-12-27 05:11:43,006][105692] Updated weights for policy 0, policy_version 1893992 (0.0010) [2023-12-27 05:11:43,508][105620] Updated weights for policy 1, policy_version 1898547 (0.0006) [2023-12-27 05:11:43,576][105620] Updated weights for policy 1, policy_version 1898557 (0.0005) [2023-12-27 05:11:43,632][105620] Updated weights for policy 1, policy_version 1898567 (0.0005) [2023-12-27 05:11:43,707][105692] Updated weights for policy 0, policy_version 1894002 (0.0009) [2023-12-27 05:11:43,770][105692] Updated weights for policy 0, policy_version 1894012 (0.0009) [2023-12-27 05:11:43,831][105692] Updated weights for policy 0, policy_version 1894022 (0.0010) [2023-12-27 05:11:43,885][105692] Updated weights for policy 0, policy_version 1894032 (0.0010) [2023-12-27 05:11:44,276][105620] Updated weights for policy 1, policy_version 1898577 (0.0005) [2023-12-27 05:11:44,321][105620] Updated weights for policy 1, policy_version 1898587 (0.0008) [2023-12-27 05:11:44,384][105620] Updated weights for policy 1, policy_version 1898597 (0.0008) [2023-12-27 05:11:44,442][105620] Updated weights for policy 1, policy_version 1898607 (0.0008) [2023-12-27 05:11:44,614][105692] Updated weights for policy 0, policy_version 1894042 (0.0010) [2023-12-27 05:11:44,672][105692] Updated weights for policy 0, policy_version 1894052 (0.0010) [2023-12-27 05:11:44,726][105692] Updated weights for policy 0, policy_version 1894062 (0.0009) [2023-12-27 05:11:45,131][105620] Updated weights for policy 1, policy_version 1898617 (0.0008) [2023-12-27 05:11:45,193][105620] Updated weights for policy 1, policy_version 1898627 (0.0010) [2023-12-27 05:11:45,241][105620] Updated weights for policy 1, policy_version 1898637 (0.0009) [2023-12-27 05:11:45,450][105692] Updated weights for policy 0, policy_version 1894072 (0.0006) [2023-12-27 05:11:45,520][105692] Updated weights for policy 0, policy_version 1894082 (0.0006) [2023-12-27 05:11:45,593][105692] Updated weights for policy 0, policy_version 1894092 (0.0006) [2023-12-27 05:11:46,024][105620] Updated weights for policy 1, policy_version 1898647 (0.0009) [2023-12-27 05:11:46,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.1, 300 sec: 19410.9). Total num frames: 971079680. Throughput: 0: 9568.3, 1: 9900.2. Samples: 971053800. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:11:46,063][104569] Avg episode reward: [(0, '8540.769'), (1, '9255.275')] [2023-12-27 05:11:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001894096_484958208.pth... [2023-12-27 05:11:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001892976_484671488.pth [2023-12-27 05:11:46,076][105620] Updated weights for policy 1, policy_version 1898657 (0.0009) [2023-12-27 05:11:46,125][105692] Updated weights for policy 0, policy_version 1894102 (0.0006) [2023-12-27 05:11:46,131][105620] Updated weights for policy 1, policy_version 1898667 (0.0008) [2023-12-27 05:11:46,156][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001898672_486129664.pth... [2023-12-27 05:11:46,161][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001897488_485826560.pth [2023-12-27 05:11:46,177][105692] Updated weights for policy 0, policy_version 1894112 (0.0007) [2023-12-27 05:11:46,228][105692] Updated weights for policy 0, policy_version 1894122 (0.0010) [2023-12-27 05:11:46,875][105620] Updated weights for policy 1, policy_version 1898677 (0.0006) [2023-12-27 05:11:46,943][105620] Updated weights for policy 1, policy_version 1898687 (0.0006) [2023-12-27 05:11:46,957][105692] Updated weights for policy 0, policy_version 1894133 (0.0009) [2023-12-27 05:11:47,004][105620] Updated weights for policy 1, policy_version 1898697 (0.0006) [2023-12-27 05:11:47,013][105692] Updated weights for policy 0, policy_version 1894143 (0.0010) [2023-12-27 05:11:47,070][105692] Updated weights for policy 0, policy_version 1894153 (0.0010) [2023-12-27 05:11:47,627][105620] Updated weights for policy 1, policy_version 1898707 (0.0006) [2023-12-27 05:11:47,692][105620] Updated weights for policy 1, policy_version 1898717 (0.0007) [2023-12-27 05:11:47,748][105620] Updated weights for policy 1, policy_version 1898727 (0.0010) [2023-12-27 05:11:47,814][105692] Updated weights for policy 0, policy_version 1894163 (0.0011) [2023-12-27 05:11:47,872][105692] Updated weights for policy 0, policy_version 1894173 (0.0010) [2023-12-27 05:11:47,929][105692] Updated weights for policy 0, policy_version 1894183 (0.0010) [2023-12-27 05:11:48,470][105620] Updated weights for policy 1, policy_version 1898737 (0.0011) [2023-12-27 05:11:48,529][105620] Updated weights for policy 1, policy_version 1898747 (0.0011) [2023-12-27 05:11:48,595][105620] Updated weights for policy 1, policy_version 1898757 (0.0011) [2023-12-27 05:11:48,656][105692] Updated weights for policy 0, policy_version 1894193 (0.0010) [2023-12-27 05:11:48,661][105620] Updated weights for policy 1, policy_version 1898767 (0.0011) [2023-12-27 05:11:48,715][105692] Updated weights for policy 0, policy_version 1894203 (0.0009) [2023-12-27 05:11:48,776][105692] Updated weights for policy 0, policy_version 1894213 (0.0008) [2023-12-27 05:11:48,833][105692] Updated weights for policy 0, policy_version 1894223 (0.0008) [2023-12-27 05:11:49,470][105620] Updated weights for policy 1, policy_version 1898777 (0.0009) [2023-12-27 05:11:49,529][105620] Updated weights for policy 1, policy_version 1898787 (0.0009) [2023-12-27 05:11:49,586][105620] Updated weights for policy 1, policy_version 1898797 (0.0008) [2023-12-27 05:11:49,605][105692] Updated weights for policy 0, policy_version 1894233 (0.0006) [2023-12-27 05:11:49,664][105692] Updated weights for policy 0, policy_version 1894243 (0.0009) [2023-12-27 05:11:49,726][105692] Updated weights for policy 0, policy_version 1894253 (0.0009) [2023-12-27 05:11:50,331][105620] Updated weights for policy 1, policy_version 1898807 (0.0006) [2023-12-27 05:11:50,404][105620] Updated weights for policy 1, policy_version 1898817 (0.0007) [2023-12-27 05:11:50,448][105692] Updated weights for policy 0, policy_version 1894263 (0.0008) [2023-12-27 05:11:50,465][105620] Updated weights for policy 1, policy_version 1898827 (0.0007) [2023-12-27 05:11:50,504][105692] Updated weights for policy 0, policy_version 1894273 (0.0008) [2023-12-27 05:11:50,567][105692] Updated weights for policy 0, policy_version 1894283 (0.0007) [2023-12-27 05:11:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 971177984. Throughput: 0: 9621.2, 1: 9851.8. Samples: 971169872. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:11:51,062][104569] Avg episode reward: [(0, '8448.046'), (1, '9161.307')] [2023-12-27 05:11:51,138][105620] Updated weights for policy 1, policy_version 1898837 (0.0007) [2023-12-27 05:11:51,190][105620] Updated weights for policy 1, policy_version 1898847 (0.0009) [2023-12-27 05:11:51,243][105620] Updated weights for policy 1, policy_version 1898857 (0.0007) [2023-12-27 05:11:51,305][105692] Updated weights for policy 0, policy_version 1894293 (0.0008) [2023-12-27 05:11:51,373][105692] Updated weights for policy 0, policy_version 1894303 (0.0009) [2023-12-27 05:11:51,432][105692] Updated weights for policy 0, policy_version 1894313 (0.0006) [2023-12-27 05:11:51,878][105620] Updated weights for policy 1, policy_version 1898867 (0.0009) [2023-12-27 05:11:51,924][105620] Updated weights for policy 1, policy_version 1898877 (0.0007) [2023-12-27 05:11:51,981][105620] Updated weights for policy 1, policy_version 1898887 (0.0008) [2023-12-27 05:11:52,242][105692] Updated weights for policy 0, policy_version 1894323 (0.0009) [2023-12-27 05:11:52,305][105692] Updated weights for policy 0, policy_version 1894333 (0.0011) [2023-12-27 05:11:52,364][105692] Updated weights for policy 0, policy_version 1894343 (0.0011) [2023-12-27 05:11:52,765][105620] Updated weights for policy 1, policy_version 1898897 (0.0008) [2023-12-27 05:11:52,827][105620] Updated weights for policy 1, policy_version 1898907 (0.0008) [2023-12-27 05:11:52,885][105620] Updated weights for policy 1, policy_version 1898917 (0.0005) [2023-12-27 05:11:52,942][105620] Updated weights for policy 1, policy_version 1898927 (0.0006) [2023-12-27 05:11:53,066][105692] Updated weights for policy 0, policy_version 1894353 (0.0010) [2023-12-27 05:11:53,122][105692] Updated weights for policy 0, policy_version 1894363 (0.0006) [2023-12-27 05:11:53,177][105692] Updated weights for policy 0, policy_version 1894373 (0.0005) [2023-12-27 05:11:53,233][105692] Updated weights for policy 0, policy_version 1894383 (0.0005) [2023-12-27 05:11:53,754][105692] Updated weights for policy 0, policy_version 1894393 (0.0005) [2023-12-27 05:11:53,767][105620] Updated weights for policy 1, policy_version 1898937 (0.0008) [2023-12-27 05:11:53,814][105692] Updated weights for policy 0, policy_version 1894403 (0.0006) [2023-12-27 05:11:53,824][105620] Updated weights for policy 1, policy_version 1898947 (0.0006) [2023-12-27 05:11:53,867][105692] Updated weights for policy 0, policy_version 1894413 (0.0009) [2023-12-27 05:11:53,881][105620] Updated weights for policy 1, policy_version 1898957 (0.0006) [2023-12-27 05:11:54,467][105692] Updated weights for policy 0, policy_version 1894423 (0.0008) [2023-12-27 05:11:54,525][105692] Updated weights for policy 0, policy_version 1894433 (0.0005) [2023-12-27 05:11:54,577][105692] Updated weights for policy 0, policy_version 1894443 (0.0006) [2023-12-27 05:11:54,692][105620] Updated weights for policy 1, policy_version 1898967 (0.0009) [2023-12-27 05:11:54,746][105620] Updated weights for policy 1, policy_version 1898977 (0.0010) [2023-12-27 05:11:54,798][105620] Updated weights for policy 1, policy_version 1898987 (0.0009) [2023-12-27 05:11:55,225][105692] Updated weights for policy 0, policy_version 1894453 (0.0009) [2023-12-27 05:11:55,287][105692] Updated weights for policy 0, policy_version 1894463 (0.0010) [2023-12-27 05:11:55,339][105692] Updated weights for policy 0, policy_version 1894473 (0.0011) [2023-12-27 05:11:55,558][105620] Updated weights for policy 1, policy_version 1898997 (0.0008) [2023-12-27 05:11:55,614][105620] Updated weights for policy 1, policy_version 1899007 (0.0008) [2023-12-27 05:11:55,670][105620] Updated weights for policy 1, policy_version 1899017 (0.0008) [2023-12-27 05:11:56,051][105692] Updated weights for policy 0, policy_version 1894483 (0.0010) [2023-12-27 05:11:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 971276288. Throughput: 0: 9622.1, 1: 9831.1. Samples: 971286580. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:11:56,063][104569] Avg episode reward: [(0, '8088.909'), (1, '9069.369')] [2023-12-27 05:11:56,109][105692] Updated weights for policy 0, policy_version 1894493 (0.0010) [2023-12-27 05:11:56,157][105692] Updated weights for policy 0, policy_version 1894503 (0.0011) [2023-12-27 05:11:56,485][105620] Updated weights for policy 1, policy_version 1899027 (0.0008) [2023-12-27 05:11:56,542][105620] Updated weights for policy 1, policy_version 1899037 (0.0008) [2023-12-27 05:11:56,594][105620] Updated weights for policy 1, policy_version 1899047 (0.0008) [2023-12-27 05:11:56,854][105692] Updated weights for policy 0, policy_version 1894513 (0.0010) [2023-12-27 05:11:56,902][105692] Updated weights for policy 0, policy_version 1894523 (0.0006) [2023-12-27 05:11:56,949][105692] Updated weights for policy 0, policy_version 1894533 (0.0008) [2023-12-27 05:11:57,001][105692] Updated weights for policy 0, policy_version 1894543 (0.0010) [2023-12-27 05:11:57,213][105620] Updated weights for policy 1, policy_version 1899058 (0.0009) [2023-12-27 05:11:57,286][105620] Updated weights for policy 1, policy_version 1899068 (0.0005) [2023-12-27 05:11:57,355][105620] Updated weights for policy 1, policy_version 1899078 (0.0008) [2023-12-27 05:11:57,425][105620] Updated weights for policy 1, policy_version 1899088 (0.0005) [2023-12-27 05:11:57,587][105692] Updated weights for policy 0, policy_version 1894553 (0.0010) [2023-12-27 05:11:57,641][105692] Updated weights for policy 0, policy_version 1894563 (0.0010) [2023-12-27 05:11:57,691][105692] Updated weights for policy 0, policy_version 1894573 (0.0010) [2023-12-27 05:11:57,895][105620] Updated weights for policy 1, policy_version 1899098 (0.0005) [2023-12-27 05:11:57,964][105620] Updated weights for policy 1, policy_version 1899108 (0.0005) [2023-12-27 05:11:58,018][105620] Updated weights for policy 1, policy_version 1899118 (0.0005) [2023-12-27 05:11:58,347][105692] Updated weights for policy 0, policy_version 1894583 (0.0011) [2023-12-27 05:11:58,404][105692] Updated weights for policy 0, policy_version 1894593 (0.0010) [2023-12-27 05:11:58,467][105692] Updated weights for policy 0, policy_version 1894603 (0.0011) [2023-12-27 05:11:58,687][105620] Updated weights for policy 1, policy_version 1899128 (0.0007) [2023-12-27 05:11:58,753][105620] Updated weights for policy 1, policy_version 1899138 (0.0008) [2023-12-27 05:11:58,826][105620] Updated weights for policy 1, policy_version 1899148 (0.0008) [2023-12-27 05:11:59,293][105692] Updated weights for policy 0, policy_version 1894613 (0.0010) [2023-12-27 05:11:59,356][105692] Updated weights for policy 0, policy_version 1894623 (0.0010) [2023-12-27 05:11:59,416][105692] Updated weights for policy 0, policy_version 1894633 (0.0007) [2023-12-27 05:11:59,547][105620] Updated weights for policy 1, policy_version 1899158 (0.0008) [2023-12-27 05:11:59,604][105620] Updated weights for policy 1, policy_version 1899168 (0.0009) [2023-12-27 05:11:59,657][105620] Updated weights for policy 1, policy_version 1899178 (0.0010) [2023-12-27 05:12:00,029][105692] Updated weights for policy 0, policy_version 1894643 (0.0006) [2023-12-27 05:12:00,087][105692] Updated weights for policy 0, policy_version 1894653 (0.0009) [2023-12-27 05:12:00,145][105692] Updated weights for policy 0, policy_version 1894663 (0.0009) [2023-12-27 05:12:00,460][105620] Updated weights for policy 1, policy_version 1899188 (0.0008) [2023-12-27 05:12:00,517][105620] Updated weights for policy 1, policy_version 1899198 (0.0009) [2023-12-27 05:12:00,571][105620] Updated weights for policy 1, policy_version 1899209 (0.0010) [2023-12-27 05:12:00,834][105692] Updated weights for policy 0, policy_version 1894673 (0.0009) [2023-12-27 05:12:00,885][105692] Updated weights for policy 0, policy_version 1894683 (0.0005) [2023-12-27 05:12:00,946][105692] Updated weights for policy 0, policy_version 1894693 (0.0005) [2023-12-27 05:12:01,005][105692] Updated weights for policy 0, policy_version 1894703 (0.0006) [2023-12-27 05:12:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 971382784. Throughput: 0: 9695.6, 1: 9914.1. Samples: 971349860. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:12:01,062][104569] Avg episode reward: [(0, '8272.025'), (1, '9161.575')] [2023-12-27 05:12:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001894704_485113856.pth... [2023-12-27 05:12:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001899216_486268928.pth... [2023-12-27 05:12:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001898064_485974016.pth [2023-12-27 05:12:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001893520_484810752.pth [2023-12-27 05:12:01,248][105620] Updated weights for policy 1, policy_version 1899219 (0.0009) [2023-12-27 05:12:01,312][105620] Updated weights for policy 1, policy_version 1899229 (0.0010) [2023-12-27 05:12:01,401][105620] Updated weights for policy 1, policy_version 1899239 (0.0009) [2023-12-27 05:12:01,606][105692] Updated weights for policy 0, policy_version 1894713 (0.0007) [2023-12-27 05:12:01,667][105692] Updated weights for policy 0, policy_version 1894723 (0.0008) [2023-12-27 05:12:01,726][105692] Updated weights for policy 0, policy_version 1894733 (0.0008) [2023-12-27 05:12:02,175][105620] Updated weights for policy 1, policy_version 1899249 (0.0009) [2023-12-27 05:12:02,234][105620] Updated weights for policy 1, policy_version 1899259 (0.0009) [2023-12-27 05:12:02,297][105620] Updated weights for policy 1, policy_version 1899269 (0.0009) [2023-12-27 05:12:02,362][105620] Updated weights for policy 1, policy_version 1899279 (0.0009) [2023-12-27 05:12:02,477][105692] Updated weights for policy 0, policy_version 1894743 (0.0008) [2023-12-27 05:12:02,525][105692] Updated weights for policy 0, policy_version 1894753 (0.0009) [2023-12-27 05:12:02,572][105692] Updated weights for policy 0, policy_version 1894764 (0.0009) [2023-12-27 05:12:03,059][105620] Updated weights for policy 1, policy_version 1899289 (0.0009) [2023-12-27 05:12:03,108][105620] Updated weights for policy 1, policy_version 1899299 (0.0008) [2023-12-27 05:12:03,166][105620] Updated weights for policy 1, policy_version 1899309 (0.0009) [2023-12-27 05:12:03,354][105692] Updated weights for policy 0, policy_version 1894774 (0.0009) [2023-12-27 05:12:03,414][105692] Updated weights for policy 0, policy_version 1894784 (0.0009) [2023-12-27 05:12:03,470][105692] Updated weights for policy 0, policy_version 1894794 (0.0008) [2023-12-27 05:12:03,992][105620] Updated weights for policy 1, policy_version 1899319 (0.0009) [2023-12-27 05:12:04,055][105620] Updated weights for policy 1, policy_version 1899329 (0.0009) [2023-12-27 05:12:04,109][105620] Updated weights for policy 1, policy_version 1899339 (0.0009) [2023-12-27 05:12:04,141][105692] Updated weights for policy 0, policy_version 1894804 (0.0009) [2023-12-27 05:12:04,207][105692] Updated weights for policy 0, policy_version 1894814 (0.0009) [2023-12-27 05:12:04,272][105692] Updated weights for policy 0, policy_version 1894824 (0.0009) [2023-12-27 05:12:04,829][105620] Updated weights for policy 1, policy_version 1899349 (0.0008) [2023-12-27 05:12:04,884][105620] Updated weights for policy 1, policy_version 1899359 (0.0008) [2023-12-27 05:12:04,939][105620] Updated weights for policy 1, policy_version 1899369 (0.0006) [2023-12-27 05:12:05,052][105692] Updated weights for policy 0, policy_version 1894834 (0.0010) [2023-12-27 05:12:05,099][105692] Updated weights for policy 0, policy_version 1894844 (0.0007) [2023-12-27 05:12:05,159][105692] Updated weights for policy 0, policy_version 1894854 (0.0005) [2023-12-27 05:12:05,228][105692] Updated weights for policy 0, policy_version 1894864 (0.0006) [2023-12-27 05:12:05,562][105620] Updated weights for policy 1, policy_version 1899379 (0.0007) [2023-12-27 05:12:05,609][105620] Updated weights for policy 1, policy_version 1899389 (0.0010) [2023-12-27 05:12:05,660][105620] Updated weights for policy 1, policy_version 1899399 (0.0009) [2023-12-27 05:12:05,859][105692] Updated weights for policy 0, policy_version 1894874 (0.0010) [2023-12-27 05:12:05,931][105692] Updated weights for policy 0, policy_version 1894884 (0.0007) [2023-12-27 05:12:05,994][105692] Updated weights for policy 0, policy_version 1894894 (0.0008) [2023-12-27 05:12:06,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 971481088. Throughput: 0: 9805.7, 1: 9816.4. Samples: 971464624. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:12:06,063][104569] Avg episode reward: [(0, '8536.176'), (1, '9253.492')] [2023-12-27 05:12:06,325][105620] Updated weights for policy 1, policy_version 1899409 (0.0008) [2023-12-27 05:12:06,391][105620] Updated weights for policy 1, policy_version 1899419 (0.0009) [2023-12-27 05:12:06,459][105620] Updated weights for policy 1, policy_version 1899429 (0.0008) [2023-12-27 05:12:06,524][105620] Updated weights for policy 1, policy_version 1899439 (0.0008) [2023-12-27 05:12:06,608][105692] Updated weights for policy 0, policy_version 1894904 (0.0010) [2023-12-27 05:12:06,667][105692] Updated weights for policy 0, policy_version 1894914 (0.0010) [2023-12-27 05:12:06,730][105692] Updated weights for policy 0, policy_version 1894924 (0.0009) [2023-12-27 05:12:07,186][105620] Updated weights for policy 1, policy_version 1899449 (0.0009) [2023-12-27 05:12:07,234][105620] Updated weights for policy 1, policy_version 1899459 (0.0009) [2023-12-27 05:12:07,285][105620] Updated weights for policy 1, policy_version 1899469 (0.0009) [2023-12-27 05:12:07,509][105692] Updated weights for policy 0, policy_version 1894934 (0.0009) [2023-12-27 05:12:07,568][105692] Updated weights for policy 0, policy_version 1894944 (0.0009) [2023-12-27 05:12:07,618][105692] Updated weights for policy 0, policy_version 1894954 (0.0006) [2023-12-27 05:12:08,013][105620] Updated weights for policy 1, policy_version 1899479 (0.0009) [2023-12-27 05:12:08,067][105620] Updated weights for policy 1, policy_version 1899489 (0.0009) [2023-12-27 05:12:08,112][105620] Updated weights for policy 1, policy_version 1899499 (0.0009) [2023-12-27 05:12:08,338][105692] Updated weights for policy 0, policy_version 1894964 (0.0007) [2023-12-27 05:12:08,401][105692] Updated weights for policy 0, policy_version 1894974 (0.0009) [2023-12-27 05:12:08,453][105692] Updated weights for policy 0, policy_version 1894984 (0.0009) [2023-12-27 05:12:08,923][105620] Updated weights for policy 1, policy_version 1899509 (0.0009) [2023-12-27 05:12:08,977][105620] Updated weights for policy 1, policy_version 1899519 (0.0009) [2023-12-27 05:12:09,032][105620] Updated weights for policy 1, policy_version 1899529 (0.0009) [2023-12-27 05:12:09,170][105692] Updated weights for policy 0, policy_version 1894994 (0.0009) [2023-12-27 05:12:09,232][105692] Updated weights for policy 0, policy_version 1895004 (0.0008) [2023-12-27 05:12:09,304][105692] Updated weights for policy 0, policy_version 1895014 (0.0009) [2023-12-27 05:12:09,382][105692] Updated weights for policy 0, policy_version 1895024 (0.0009) [2023-12-27 05:12:09,807][105620] Updated weights for policy 1, policy_version 1899539 (0.0010) [2023-12-27 05:12:09,877][105620] Updated weights for policy 1, policy_version 1899549 (0.0009) [2023-12-27 05:12:09,946][105620] Updated weights for policy 1, policy_version 1899559 (0.0008) [2023-12-27 05:12:10,178][105692] Updated weights for policy 0, policy_version 1895034 (0.0006) [2023-12-27 05:12:10,235][105692] Updated weights for policy 0, policy_version 1895044 (0.0005) [2023-12-27 05:12:10,292][105692] Updated weights for policy 0, policy_version 1895054 (0.0005) [2023-12-27 05:12:10,765][105620] Updated weights for policy 1, policy_version 1899569 (0.0009) [2023-12-27 05:12:10,830][105620] Updated weights for policy 1, policy_version 1899579 (0.0009) [2023-12-27 05:12:10,891][105620] Updated weights for policy 1, policy_version 1899589 (0.0008) [2023-12-27 05:12:10,939][105620] Updated weights for policy 1, policy_version 1899599 (0.0007) [2023-12-27 05:12:10,957][105692] Updated weights for policy 0, policy_version 1895064 (0.0008) [2023-12-27 05:12:11,009][105692] Updated weights for policy 0, policy_version 1895074 (0.0009) [2023-12-27 05:12:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 971571200. Throughput: 0: 9909.3, 1: 9741.4. Samples: 971580956. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:12:11,062][104569] Avg episode reward: [(0, '8352.870'), (1, '9345.460')] [2023-12-27 05:12:11,081][105692] Updated weights for policy 0, policy_version 1895084 (0.0007) [2023-12-27 05:12:11,740][105620] Updated weights for policy 1, policy_version 1899609 (0.0009) [2023-12-27 05:12:11,804][105620] Updated weights for policy 1, policy_version 1899619 (0.0008) [2023-12-27 05:12:11,831][105692] Updated weights for policy 0, policy_version 1895094 (0.0009) [2023-12-27 05:12:11,865][105620] Updated weights for policy 1, policy_version 1899629 (0.0006) [2023-12-27 05:12:11,894][105692] Updated weights for policy 0, policy_version 1895104 (0.0011) [2023-12-27 05:12:11,939][105692] Updated weights for policy 0, policy_version 1895114 (0.0011) [2023-12-27 05:12:12,648][105620] Updated weights for policy 1, policy_version 1899639 (0.0009) [2023-12-27 05:12:12,670][105692] Updated weights for policy 0, policy_version 1895124 (0.0008) [2023-12-27 05:12:12,709][105620] Updated weights for policy 1, policy_version 1899649 (0.0007) [2023-12-27 05:12:12,728][105692] Updated weights for policy 0, policy_version 1895134 (0.0007) [2023-12-27 05:12:12,763][105620] Updated weights for policy 1, policy_version 1899659 (0.0008) [2023-12-27 05:12:12,785][105692] Updated weights for policy 0, policy_version 1895144 (0.0007) [2023-12-27 05:12:13,421][105620] Updated weights for policy 1, policy_version 1899669 (0.0007) [2023-12-27 05:12:13,475][105620] Updated weights for policy 1, policy_version 1899679 (0.0007) [2023-12-27 05:12:13,531][105620] Updated weights for policy 1, policy_version 1899689 (0.0009) [2023-12-27 05:12:13,590][105692] Updated weights for policy 0, policy_version 1895154 (0.0009) [2023-12-27 05:12:13,636][105692] Updated weights for policy 0, policy_version 1895164 (0.0007) [2023-12-27 05:12:13,682][105692] Updated weights for policy 0, policy_version 1895174 (0.0005) [2023-12-27 05:12:13,729][105692] Updated weights for policy 0, policy_version 1895184 (0.0008) [2023-12-27 05:12:14,134][105620] Updated weights for policy 1, policy_version 1899699 (0.0007) [2023-12-27 05:12:14,189][105620] Updated weights for policy 1, policy_version 1899709 (0.0007) [2023-12-27 05:12:14,247][105620] Updated weights for policy 1, policy_version 1899719 (0.0005) [2023-12-27 05:12:14,613][105692] Updated weights for policy 0, policy_version 1895194 (0.0008) [2023-12-27 05:12:14,670][105692] Updated weights for policy 0, policy_version 1895204 (0.0009) [2023-12-27 05:12:14,717][105692] Updated weights for policy 0, policy_version 1895214 (0.0009) [2023-12-27 05:12:14,849][105620] Updated weights for policy 1, policy_version 1899729 (0.0006) [2023-12-27 05:12:14,916][105620] Updated weights for policy 1, policy_version 1899739 (0.0009) [2023-12-27 05:12:14,975][105620] Updated weights for policy 1, policy_version 1899749 (0.0010) [2023-12-27 05:12:15,032][105620] Updated weights for policy 1, policy_version 1899759 (0.0009) [2023-12-27 05:12:15,465][105692] Updated weights for policy 0, policy_version 1895224 (0.0009) [2023-12-27 05:12:15,520][105692] Updated weights for policy 0, policy_version 1895234 (0.0009) [2023-12-27 05:12:15,580][105692] Updated weights for policy 0, policy_version 1895244 (0.0009) [2023-12-27 05:12:15,759][105620] Updated weights for policy 1, policy_version 1899769 (0.0008) [2023-12-27 05:12:15,815][105620] Updated weights for policy 1, policy_version 1899779 (0.0005) [2023-12-27 05:12:15,873][105620] Updated weights for policy 1, policy_version 1899789 (0.0005) [2023-12-27 05:12:16,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 971669504. Throughput: 0: 9823.0, 1: 9702.7. Samples: 971637212. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:12:16,063][104569] Avg episode reward: [(0, '8445.387'), (1, '9253.729')] [2023-12-27 05:12:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001895248_485253120.pth... [2023-12-27 05:12:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001899792_486416384.pth... [2023-12-27 05:12:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001894096_484958208.pth [2023-12-27 05:12:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001898672_486129664.pth [2023-12-27 05:12:16,430][105692] Updated weights for policy 0, policy_version 1895255 (0.0009) [2023-12-27 05:12:16,471][105620] Updated weights for policy 1, policy_version 1899799 (0.0006) [2023-12-27 05:12:16,477][105692] Updated weights for policy 0, policy_version 1895265 (0.0007) [2023-12-27 05:12:16,520][105620] Updated weights for policy 1, policy_version 1899809 (0.0006) [2023-12-27 05:12:16,537][105692] Updated weights for policy 0, policy_version 1895275 (0.0008) [2023-12-27 05:12:16,569][105620] Updated weights for policy 1, policy_version 1899819 (0.0007) [2023-12-27 05:12:17,289][105620] Updated weights for policy 1, policy_version 1899829 (0.0010) [2023-12-27 05:12:17,339][105692] Updated weights for policy 0, policy_version 1895285 (0.0008) [2023-12-27 05:12:17,342][105620] Updated weights for policy 1, policy_version 1899839 (0.0009) [2023-12-27 05:12:17,391][105692] Updated weights for policy 0, policy_version 1895295 (0.0007) [2023-12-27 05:12:17,397][105620] Updated weights for policy 1, policy_version 1899849 (0.0008) [2023-12-27 05:12:17,444][105692] Updated weights for policy 0, policy_version 1895305 (0.0006) [2023-12-27 05:12:17,997][105620] Updated weights for policy 1, policy_version 1899859 (0.0009) [2023-12-27 05:12:18,068][105620] Updated weights for policy 1, policy_version 1899869 (0.0005) [2023-12-27 05:12:18,136][105620] Updated weights for policy 1, policy_version 1899879 (0.0007) [2023-12-27 05:12:18,311][105692] Updated weights for policy 0, policy_version 1895315 (0.0006) [2023-12-27 05:12:18,389][105692] Updated weights for policy 0, policy_version 1895325 (0.0009) [2023-12-27 05:12:18,453][105692] Updated weights for policy 0, policy_version 1895335 (0.0009) [2023-12-27 05:12:18,760][105620] Updated weights for policy 1, policy_version 1899889 (0.0009) [2023-12-27 05:12:18,824][105620] Updated weights for policy 1, policy_version 1899899 (0.0009) [2023-12-27 05:12:18,883][105620] Updated weights for policy 1, policy_version 1899909 (0.0010) [2023-12-27 05:12:18,944][105620] Updated weights for policy 1, policy_version 1899919 (0.0010) [2023-12-27 05:12:19,121][105692] Updated weights for policy 0, policy_version 1895345 (0.0008) [2023-12-27 05:12:19,179][105692] Updated weights for policy 0, policy_version 1895355 (0.0006) [2023-12-27 05:12:19,235][105692] Updated weights for policy 0, policy_version 1895365 (0.0007) [2023-12-27 05:12:19,295][105692] Updated weights for policy 0, policy_version 1895375 (0.0007) [2023-12-27 05:12:19,746][105620] Updated weights for policy 1, policy_version 1899929 (0.0010) [2023-12-27 05:12:19,808][105620] Updated weights for policy 1, policy_version 1899939 (0.0008) [2023-12-27 05:12:19,867][105620] Updated weights for policy 1, policy_version 1899949 (0.0008) [2023-12-27 05:12:20,010][105692] Updated weights for policy 0, policy_version 1895385 (0.0009) [2023-12-27 05:12:20,075][105692] Updated weights for policy 0, policy_version 1895395 (0.0008) [2023-12-27 05:12:20,139][105692] Updated weights for policy 0, policy_version 1895405 (0.0008) [2023-12-27 05:12:20,563][105620] Updated weights for policy 1, policy_version 1899959 (0.0007) [2023-12-27 05:12:20,626][105620] Updated weights for policy 1, policy_version 1899969 (0.0009) [2023-12-27 05:12:20,692][105620] Updated weights for policy 1, policy_version 1899979 (0.0009) [2023-12-27 05:12:20,926][105692] Updated weights for policy 0, policy_version 1895415 (0.0007) [2023-12-27 05:12:20,996][105692] Updated weights for policy 0, policy_version 1895425 (0.0006) [2023-12-27 05:12:21,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 971759616. Throughput: 0: 9656.8, 1: 9744.7. Samples: 971752688. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:12:21,063][104569] Avg episode reward: [(0, '8445.373'), (1, '9069.122')] [2023-12-27 05:12:21,065][105692] Updated weights for policy 0, policy_version 1895435 (0.0007) [2023-12-27 05:12:21,445][105620] Updated weights for policy 1, policy_version 1899989 (0.0007) [2023-12-27 05:12:21,515][105620] Updated weights for policy 1, policy_version 1899999 (0.0006) [2023-12-27 05:12:21,570][105620] Updated weights for policy 1, policy_version 1900009 (0.0006) [2023-12-27 05:12:21,737][105692] Updated weights for policy 0, policy_version 1895445 (0.0008) [2023-12-27 05:12:21,802][105692] Updated weights for policy 0, policy_version 1895455 (0.0005) [2023-12-27 05:12:21,859][105692] Updated weights for policy 0, policy_version 1895465 (0.0006) [2023-12-27 05:12:22,252][105620] Updated weights for policy 1, policy_version 1900019 (0.0007) [2023-12-27 05:12:22,313][105620] Updated weights for policy 1, policy_version 1900029 (0.0011) [2023-12-27 05:12:22,373][105620] Updated weights for policy 1, policy_version 1900039 (0.0011) [2023-12-27 05:12:22,554][105692] Updated weights for policy 0, policy_version 1895475 (0.0005) [2023-12-27 05:12:22,613][105692] Updated weights for policy 0, policy_version 1895485 (0.0007) [2023-12-27 05:12:22,678][105692] Updated weights for policy 0, policy_version 1895495 (0.0005) [2023-12-27 05:12:23,133][105620] Updated weights for policy 1, policy_version 1900049 (0.0011) [2023-12-27 05:12:23,182][105620] Updated weights for policy 1, policy_version 1900059 (0.0011) [2023-12-27 05:12:23,242][105620] Updated weights for policy 1, policy_version 1900069 (0.0011) [2023-12-27 05:12:23,296][105620] Updated weights for policy 1, policy_version 1900079 (0.0005) [2023-12-27 05:12:23,398][105692] Updated weights for policy 0, policy_version 1895505 (0.0006) [2023-12-27 05:12:23,446][105692] Updated weights for policy 0, policy_version 1895515 (0.0009) [2023-12-27 05:12:23,493][105692] Updated weights for policy 0, policy_version 1895525 (0.0009) [2023-12-27 05:12:23,544][105692] Updated weights for policy 0, policy_version 1895535 (0.0009) [2023-12-27 05:12:23,899][105620] Updated weights for policy 1, policy_version 1900089 (0.0005) [2023-12-27 05:12:23,944][105620] Updated weights for policy 1, policy_version 1900099 (0.0005) [2023-12-27 05:12:24,002][105620] Updated weights for policy 1, policy_version 1900109 (0.0005) [2023-12-27 05:12:24,223][105692] Updated weights for policy 0, policy_version 1895545 (0.0006) [2023-12-27 05:12:24,275][105692] Updated weights for policy 0, policy_version 1895555 (0.0005) [2023-12-27 05:12:24,326][105692] Updated weights for policy 0, policy_version 1895565 (0.0005) [2023-12-27 05:12:24,709][105620] Updated weights for policy 1, policy_version 1900119 (0.0008) [2023-12-27 05:12:24,771][105620] Updated weights for policy 1, policy_version 1900129 (0.0009) [2023-12-27 05:12:24,822][105620] Updated weights for policy 1, policy_version 1900139 (0.0009) [2023-12-27 05:12:24,992][105692] Updated weights for policy 0, policy_version 1895575 (0.0006) [2023-12-27 05:12:25,060][105692] Updated weights for policy 0, policy_version 1895585 (0.0008) [2023-12-27 05:12:25,125][105692] Updated weights for policy 0, policy_version 1895595 (0.0006) [2023-12-27 05:12:25,657][105620] Updated weights for policy 1, policy_version 1900149 (0.0010) [2023-12-27 05:12:25,681][105692] Updated weights for policy 0, policy_version 1895605 (0.0006) [2023-12-27 05:12:25,711][105620] Updated weights for policy 1, policy_version 1900159 (0.0008) [2023-12-27 05:12:25,741][105692] Updated weights for policy 0, policy_version 1895615 (0.0006) [2023-12-27 05:12:25,773][105620] Updated weights for policy 1, policy_version 1900169 (0.0007) [2023-12-27 05:12:25,806][105692] Updated weights for policy 0, policy_version 1895625 (0.0008) [2023-12-27 05:12:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 971866112. Throughput: 0: 9717.1, 1: 9704.0. Samples: 971870392. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:12:26,063][104569] Avg episode reward: [(0, '8627.942'), (1, '9160.876')] [2023-12-27 05:12:26,417][105692] Updated weights for policy 0, policy_version 1895635 (0.0008) [2023-12-27 05:12:26,468][105692] Updated weights for policy 0, policy_version 1895645 (0.0009) [2023-12-27 05:12:26,487][105620] Updated weights for policy 1, policy_version 1900179 (0.0007) [2023-12-27 05:12:26,527][105692] Updated weights for policy 0, policy_version 1895655 (0.0006) [2023-12-27 05:12:26,544][105620] Updated weights for policy 1, policy_version 1900189 (0.0008) [2023-12-27 05:12:26,593][105620] Updated weights for policy 1, policy_version 1900199 (0.0006) [2023-12-27 05:12:27,230][105620] Updated weights for policy 1, policy_version 1900209 (0.0009) [2023-12-27 05:12:27,290][105620] Updated weights for policy 1, policy_version 1900219 (0.0010) [2023-12-27 05:12:27,329][105692] Updated weights for policy 0, policy_version 1895665 (0.0006) [2023-12-27 05:12:27,347][105620] Updated weights for policy 1, policy_version 1900229 (0.0011) [2023-12-27 05:12:27,385][105692] Updated weights for policy 0, policy_version 1895675 (0.0005) [2023-12-27 05:12:27,403][105620] Updated weights for policy 1, policy_version 1900239 (0.0010) [2023-12-27 05:12:27,446][105692] Updated weights for policy 0, policy_version 1895685 (0.0007) [2023-12-27 05:12:27,501][105692] Updated weights for policy 0, policy_version 1895695 (0.0008) [2023-12-27 05:12:28,061][105620] Updated weights for policy 1, policy_version 1900249 (0.0009) [2023-12-27 05:12:28,108][105620] Updated weights for policy 1, policy_version 1900259 (0.0009) [2023-12-27 05:12:28,153][105620] Updated weights for policy 1, policy_version 1900269 (0.0008) [2023-12-27 05:12:28,282][105692] Updated weights for policy 0, policy_version 1895705 (0.0009) [2023-12-27 05:12:28,342][105692] Updated weights for policy 0, policy_version 1895715 (0.0008) [2023-12-27 05:12:28,437][105692] Updated weights for policy 0, policy_version 1895725 (0.0008) [2023-12-27 05:12:28,927][105620] Updated weights for policy 1, policy_version 1900279 (0.0006) [2023-12-27 05:12:28,984][105620] Updated weights for policy 1, policy_version 1900289 (0.0010) [2023-12-27 05:12:29,042][105620] Updated weights for policy 1, policy_version 1900299 (0.0010) [2023-12-27 05:12:29,152][105692] Updated weights for policy 0, policy_version 1895735 (0.0009) [2023-12-27 05:12:29,195][105692] Updated weights for policy 0, policy_version 1895745 (0.0007) [2023-12-27 05:12:29,252][105692] Updated weights for policy 0, policy_version 1895755 (0.0008) [2023-12-27 05:12:29,772][105620] Updated weights for policy 1, policy_version 1900309 (0.0010) [2023-12-27 05:12:29,832][105620] Updated weights for policy 1, policy_version 1900319 (0.0010) [2023-12-27 05:12:29,901][105620] Updated weights for policy 1, policy_version 1900329 (0.0011) [2023-12-27 05:12:29,913][105692] Updated weights for policy 0, policy_version 1895765 (0.0007) [2023-12-27 05:12:29,976][105692] Updated weights for policy 0, policy_version 1895775 (0.0009) [2023-12-27 05:12:30,037][105692] Updated weights for policy 0, policy_version 1895785 (0.0009) [2023-12-27 05:12:30,650][105620] Updated weights for policy 1, policy_version 1900339 (0.0010) [2023-12-27 05:12:30,725][105620] Updated weights for policy 1, policy_version 1900349 (0.0010) [2023-12-27 05:12:30,749][105692] Updated weights for policy 0, policy_version 1895795 (0.0007) [2023-12-27 05:12:30,782][105620] Updated weights for policy 1, policy_version 1900359 (0.0008) [2023-12-27 05:12:30,797][105692] Updated weights for policy 0, policy_version 1895805 (0.0006) [2023-12-27 05:12:30,852][105692] Updated weights for policy 0, policy_version 1895815 (0.0009) [2023-12-27 05:12:31,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 971964416. Throughput: 0: 9730.7, 1: 9733.2. Samples: 971929672. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:12:31,062][104569] Avg episode reward: [(0, '8357.331'), (1, '9253.213')] [2023-12-27 05:12:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001895824_485400576.pth... [2023-12-27 05:12:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001900368_486563840.pth... [2023-12-27 05:12:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001894704_485113856.pth [2023-12-27 05:12:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001899216_486268928.pth [2023-12-27 05:12:31,424][105620] Updated weights for policy 1, policy_version 1900369 (0.0006) [2023-12-27 05:12:31,472][105620] Updated weights for policy 1, policy_version 1900379 (0.0008) [2023-12-27 05:12:31,527][105620] Updated weights for policy 1, policy_version 1900389 (0.0009) [2023-12-27 05:12:31,557][105692] Updated weights for policy 0, policy_version 1895825 (0.0009) [2023-12-27 05:12:31,580][105620] Updated weights for policy 1, policy_version 1900400 (0.0010) [2023-12-27 05:12:31,617][105692] Updated weights for policy 0, policy_version 1895835 (0.0006) [2023-12-27 05:12:31,674][105692] Updated weights for policy 0, policy_version 1895845 (0.0008) [2023-12-27 05:12:31,741][105692] Updated weights for policy 0, policy_version 1895855 (0.0006) [2023-12-27 05:12:32,341][105692] Updated weights for policy 0, policy_version 1895865 (0.0007) [2023-12-27 05:12:32,385][105620] Updated weights for policy 1, policy_version 1900410 (0.0008) [2023-12-27 05:12:32,404][105692] Updated weights for policy 0, policy_version 1895875 (0.0006) [2023-12-27 05:12:32,438][105620] Updated weights for policy 1, policy_version 1900420 (0.0008) [2023-12-27 05:12:32,466][105692] Updated weights for policy 0, policy_version 1895885 (0.0008) [2023-12-27 05:12:32,493][105620] Updated weights for policy 1, policy_version 1900430 (0.0007) [2023-12-27 05:12:33,106][105692] Updated weights for policy 0, policy_version 1895895 (0.0006) [2023-12-27 05:12:33,144][105620] Updated weights for policy 1, policy_version 1900440 (0.0009) [2023-12-27 05:12:33,163][105692] Updated weights for policy 0, policy_version 1895905 (0.0005) [2023-12-27 05:12:33,198][105620] Updated weights for policy 1, policy_version 1900450 (0.0008) [2023-12-27 05:12:33,213][105692] Updated weights for policy 0, policy_version 1895915 (0.0006) [2023-12-27 05:12:33,251][105620] Updated weights for policy 1, policy_version 1900460 (0.0008) [2023-12-27 05:12:33,765][105692] Updated weights for policy 0, policy_version 1895925 (0.0005) [2023-12-27 05:12:33,815][105692] Updated weights for policy 0, policy_version 1895935 (0.0005) [2023-12-27 05:12:33,858][105692] Updated weights for policy 0, policy_version 1895945 (0.0005) [2023-12-27 05:12:34,163][105620] Updated weights for policy 1, policy_version 1900470 (0.0009) [2023-12-27 05:12:34,214][105620] Updated weights for policy 1, policy_version 1900480 (0.0009) [2023-12-27 05:12:34,281][105620] Updated weights for policy 1, policy_version 1900490 (0.0009) [2023-12-27 05:12:34,451][105692] Updated weights for policy 0, policy_version 1895955 (0.0007) [2023-12-27 05:12:34,511][105692] Updated weights for policy 0, policy_version 1895965 (0.0009) [2023-12-27 05:12:34,568][105692] Updated weights for policy 0, policy_version 1895975 (0.0009) [2023-12-27 05:12:34,935][105620] Updated weights for policy 1, policy_version 1900500 (0.0007) [2023-12-27 05:12:34,992][105620] Updated weights for policy 1, policy_version 1900510 (0.0009) [2023-12-27 05:12:35,042][105620] Updated weights for policy 1, policy_version 1900520 (0.0008) [2023-12-27 05:12:35,380][105692] Updated weights for policy 0, policy_version 1895985 (0.0009) [2023-12-27 05:12:35,435][105692] Updated weights for policy 0, policy_version 1895995 (0.0009) [2023-12-27 05:12:35,489][105692] Updated weights for policy 0, policy_version 1896005 (0.0009) [2023-12-27 05:12:35,548][105692] Updated weights for policy 0, policy_version 1896015 (0.0009) [2023-12-27 05:12:35,796][105620] Updated weights for policy 1, policy_version 1900530 (0.0009) [2023-12-27 05:12:35,854][105620] Updated weights for policy 1, policy_version 1900540 (0.0009) [2023-12-27 05:12:35,907][105620] Updated weights for policy 1, policy_version 1900550 (0.0008) [2023-12-27 05:12:35,960][105620] Updated weights for policy 1, policy_version 1900560 (0.0008) [2023-12-27 05:12:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 972062720. Throughput: 0: 9840.7, 1: 9726.1. Samples: 972050380. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:12:36,063][104569] Avg episode reward: [(0, '8083.682'), (1, '9253.123')] [2023-12-27 05:12:36,309][105692] Updated weights for policy 0, policy_version 1896025 (0.0007) [2023-12-27 05:12:36,374][105692] Updated weights for policy 0, policy_version 1896035 (0.0005) [2023-12-27 05:12:36,426][105692] Updated weights for policy 0, policy_version 1896045 (0.0005) [2023-12-27 05:12:36,762][105620] Updated weights for policy 1, policy_version 1900570 (0.0010) [2023-12-27 05:12:36,825][105620] Updated weights for policy 1, policy_version 1900580 (0.0008) [2023-12-27 05:12:36,879][105620] Updated weights for policy 1, policy_version 1900590 (0.0009) [2023-12-27 05:12:37,020][105692] Updated weights for policy 0, policy_version 1896055 (0.0009) [2023-12-27 05:12:37,074][105692] Updated weights for policy 0, policy_version 1896065 (0.0010) [2023-12-27 05:12:37,127][105692] Updated weights for policy 0, policy_version 1896075 (0.0009) [2023-12-27 05:12:37,624][105620] Updated weights for policy 1, policy_version 1900600 (0.0010) [2023-12-27 05:12:37,690][105620] Updated weights for policy 1, policy_version 1900610 (0.0009) [2023-12-27 05:12:37,765][105620] Updated weights for policy 1, policy_version 1900620 (0.0010) [2023-12-27 05:12:37,898][105692] Updated weights for policy 0, policy_version 1896086 (0.0010) [2023-12-27 05:12:37,951][105692] Updated weights for policy 0, policy_version 1896096 (0.0009) [2023-12-27 05:12:37,998][105692] Updated weights for policy 0, policy_version 1896106 (0.0009) [2023-12-27 05:12:38,520][105620] Updated weights for policy 1, policy_version 1900630 (0.0010) [2023-12-27 05:12:38,578][105620] Updated weights for policy 1, policy_version 1900640 (0.0009) [2023-12-27 05:12:38,635][105620] Updated weights for policy 1, policy_version 1900650 (0.0008) [2023-12-27 05:12:38,814][105692] Updated weights for policy 0, policy_version 1896116 (0.0008) [2023-12-27 05:12:38,874][105692] Updated weights for policy 0, policy_version 1896126 (0.0010) [2023-12-27 05:12:38,933][105692] Updated weights for policy 0, policy_version 1896136 (0.0010) [2023-12-27 05:12:39,266][105620] Updated weights for policy 1, policy_version 1900660 (0.0008) [2023-12-27 05:12:39,330][105620] Updated weights for policy 1, policy_version 1900670 (0.0006) [2023-12-27 05:12:39,395][105620] Updated weights for policy 1, policy_version 1900680 (0.0009) [2023-12-27 05:12:39,696][105692] Updated weights for policy 0, policy_version 1896146 (0.0011) [2023-12-27 05:12:39,762][105692] Updated weights for policy 0, policy_version 1896156 (0.0011) [2023-12-27 05:12:39,827][105692] Updated weights for policy 0, policy_version 1896166 (0.0011) [2023-12-27 05:12:39,886][105692] Updated weights for policy 0, policy_version 1896176 (0.0011) [2023-12-27 05:12:40,101][105620] Updated weights for policy 1, policy_version 1900690 (0.0009) [2023-12-27 05:12:40,161][105620] Updated weights for policy 1, policy_version 1900700 (0.0009) [2023-12-27 05:12:40,221][105620] Updated weights for policy 1, policy_version 1900710 (0.0008) [2023-12-27 05:12:40,270][105620] Updated weights for policy 1, policy_version 1900720 (0.0008) [2023-12-27 05:12:40,642][105692] Updated weights for policy 0, policy_version 1896186 (0.0011) [2023-12-27 05:12:40,692][105692] Updated weights for policy 0, policy_version 1896196 (0.0011) [2023-12-27 05:12:40,745][105692] Updated weights for policy 0, policy_version 1896206 (0.0009) [2023-12-27 05:12:40,948][105620] Updated weights for policy 1, policy_version 1900730 (0.0007) [2023-12-27 05:12:41,003][105620] Updated weights for policy 1, policy_version 1900740 (0.0009) [2023-12-27 05:12:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 972152832. Throughput: 0: 9741.6, 1: 9763.4. Samples: 972164304. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:12:41,063][104569] Avg episode reward: [(0, '7990.193'), (1, '9345.375')] [2023-12-27 05:12:41,072][105620] Updated weights for policy 1, policy_version 1900750 (0.0008) [2023-12-27 05:12:41,534][105692] Updated weights for policy 0, policy_version 1896216 (0.0009) [2023-12-27 05:12:41,605][105692] Updated weights for policy 0, policy_version 1896226 (0.0010) [2023-12-27 05:12:41,669][105692] Updated weights for policy 0, policy_version 1896236 (0.0010) [2023-12-27 05:12:41,788][105620] Updated weights for policy 1, policy_version 1900760 (0.0009) [2023-12-27 05:12:41,842][105620] Updated weights for policy 1, policy_version 1900770 (0.0009) [2023-12-27 05:12:41,899][105620] Updated weights for policy 1, policy_version 1900780 (0.0009) [2023-12-27 05:12:42,504][105692] Updated weights for policy 0, policy_version 1896246 (0.0009) [2023-12-27 05:12:42,570][105692] Updated weights for policy 0, policy_version 1896256 (0.0009) [2023-12-27 05:12:42,614][105620] Updated weights for policy 1, policy_version 1900790 (0.0007) [2023-12-27 05:12:42,629][105692] Updated weights for policy 0, policy_version 1896266 (0.0008) [2023-12-27 05:12:42,672][105620] Updated weights for policy 1, policy_version 1900800 (0.0007) [2023-12-27 05:12:42,729][105620] Updated weights for policy 1, policy_version 1900810 (0.0009) [2023-12-27 05:12:43,411][105692] Updated weights for policy 0, policy_version 1896276 (0.0007) [2023-12-27 05:12:43,424][105620] Updated weights for policy 1, policy_version 1900820 (0.0010) [2023-12-27 05:12:43,470][105692] Updated weights for policy 0, policy_version 1896286 (0.0009) [2023-12-27 05:12:43,479][105620] Updated weights for policy 1, policy_version 1900830 (0.0010) [2023-12-27 05:12:43,523][105692] Updated weights for policy 0, policy_version 1896296 (0.0006) [2023-12-27 05:12:43,541][105620] Updated weights for policy 1, policy_version 1900840 (0.0008) [2023-12-27 05:12:44,268][105692] Updated weights for policy 0, policy_version 1896306 (0.0007) [2023-12-27 05:12:44,291][105620] Updated weights for policy 1, policy_version 1900850 (0.0008) [2023-12-27 05:12:44,315][105692] Updated weights for policy 0, policy_version 1896316 (0.0007) [2023-12-27 05:12:44,341][105620] Updated weights for policy 1, policy_version 1900860 (0.0008) [2023-12-27 05:12:44,364][105692] Updated weights for policy 0, policy_version 1896326 (0.0006) [2023-12-27 05:12:44,395][105620] Updated weights for policy 1, policy_version 1900870 (0.0007) [2023-12-27 05:12:44,412][105692] Updated weights for policy 0, policy_version 1896336 (0.0008) [2023-12-27 05:12:44,445][105620] Updated weights for policy 1, policy_version 1900880 (0.0008) [2023-12-27 05:12:45,217][105692] Updated weights for policy 0, policy_version 1896346 (0.0007) [2023-12-27 05:12:45,223][105620] Updated weights for policy 1, policy_version 1900890 (0.0007) [2023-12-27 05:12:45,275][105692] Updated weights for policy 0, policy_version 1896356 (0.0006) [2023-12-27 05:12:45,287][105620] Updated weights for policy 1, policy_version 1900900 (0.0007) [2023-12-27 05:12:45,335][105692] Updated weights for policy 0, policy_version 1896366 (0.0007) [2023-12-27 05:12:45,342][105620] Updated weights for policy 1, policy_version 1900910 (0.0008) [2023-12-27 05:12:46,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 972242944. Throughput: 0: 9625.6, 1: 9708.3. Samples: 972219888. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:12:46,063][104569] Avg episode reward: [(0, '8534.893'), (1, '9345.374')] [2023-12-27 05:12:46,076][105620] Updated weights for policy 1, policy_version 1900920 (0.0010) [2023-12-27 05:12:46,080][105692] Updated weights for policy 0, policy_version 1896376 (0.0006) [2023-12-27 05:12:46,126][105692] Updated weights for policy 0, policy_version 1896386 (0.0006) [2023-12-27 05:12:46,133][105620] Updated weights for policy 1, policy_version 1900930 (0.0009) [2023-12-27 05:12:46,177][105620] Updated weights for policy 1, policy_version 1900940 (0.0006) [2023-12-27 05:12:46,182][105692] Updated weights for policy 0, policy_version 1896396 (0.0009) [2023-12-27 05:12:46,195][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001900944_486711296.pth... [2023-12-27 05:12:46,198][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001899792_486416384.pth [2023-12-27 05:12:46,205][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001896400_485548032.pth... [2023-12-27 05:12:46,209][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001895248_485253120.pth [2023-12-27 05:12:46,925][105692] Updated weights for policy 0, policy_version 1896406 (0.0008) [2023-12-27 05:12:46,927][105620] Updated weights for policy 1, policy_version 1900950 (0.0007) [2023-12-27 05:12:46,976][105620] Updated weights for policy 1, policy_version 1900960 (0.0006) [2023-12-27 05:12:46,985][105692] Updated weights for policy 0, policy_version 1896416 (0.0010) [2023-12-27 05:12:47,031][105620] Updated weights for policy 1, policy_version 1900970 (0.0007) [2023-12-27 05:12:47,044][105692] Updated weights for policy 0, policy_version 1896426 (0.0007) [2023-12-27 05:12:47,675][105620] Updated weights for policy 1, policy_version 1900980 (0.0006) [2023-12-27 05:12:47,722][105620] Updated weights for policy 1, policy_version 1900990 (0.0010) [2023-12-27 05:12:47,773][105620] Updated weights for policy 1, policy_version 1901000 (0.0009) [2023-12-27 05:12:47,850][105692] Updated weights for policy 0, policy_version 1896436 (0.0008) [2023-12-27 05:12:47,897][105692] Updated weights for policy 0, policy_version 1896446 (0.0009) [2023-12-27 05:12:47,962][105692] Updated weights for policy 0, policy_version 1896456 (0.0009) [2023-12-27 05:12:48,545][105620] Updated weights for policy 1, policy_version 1901010 (0.0009) [2023-12-27 05:12:48,604][105620] Updated weights for policy 1, policy_version 1901020 (0.0010) [2023-12-27 05:12:48,667][105620] Updated weights for policy 1, policy_version 1901030 (0.0010) [2023-12-27 05:12:48,730][105620] Updated weights for policy 1, policy_version 1901040 (0.0010) [2023-12-27 05:12:48,744][105692] Updated weights for policy 0, policy_version 1896466 (0.0009) [2023-12-27 05:12:48,804][105692] Updated weights for policy 0, policy_version 1896476 (0.0009) [2023-12-27 05:12:48,868][105692] Updated weights for policy 0, policy_version 1896486 (0.0008) [2023-12-27 05:12:48,925][105692] Updated weights for policy 0, policy_version 1896496 (0.0008) [2023-12-27 05:12:49,502][105620] Updated weights for policy 1, policy_version 1901050 (0.0010) [2023-12-27 05:12:49,564][105620] Updated weights for policy 1, policy_version 1901060 (0.0010) [2023-12-27 05:12:49,630][105620] Updated weights for policy 1, policy_version 1901070 (0.0010) [2023-12-27 05:12:49,662][105692] Updated weights for policy 0, policy_version 1896506 (0.0006) [2023-12-27 05:12:49,725][105692] Updated weights for policy 0, policy_version 1896516 (0.0008) [2023-12-27 05:12:49,788][105692] Updated weights for policy 0, policy_version 1896526 (0.0008) [2023-12-27 05:12:50,421][105620] Updated weights for policy 1, policy_version 1901080 (0.0011) [2023-12-27 05:12:50,486][105620] Updated weights for policy 1, policy_version 1901090 (0.0010) [2023-12-27 05:12:50,501][105692] Updated weights for policy 0, policy_version 1896536 (0.0008) [2023-12-27 05:12:50,552][105620] Updated weights for policy 1, policy_version 1901100 (0.0009) [2023-12-27 05:12:50,561][105692] Updated weights for policy 0, policy_version 1896546 (0.0007) [2023-12-27 05:12:50,621][105692] Updated weights for policy 0, policy_version 1896556 (0.0008) [2023-12-27 05:12:51,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 972341248. Throughput: 0: 9557.4, 1: 9723.2. Samples: 972332252. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:12:51,063][104569] Avg episode reward: [(0, '8529.953'), (1, '9345.341')] [2023-12-27 05:12:51,333][105620] Updated weights for policy 1, policy_version 1901110 (0.0007) [2023-12-27 05:12:51,393][105692] Updated weights for policy 0, policy_version 1896566 (0.0008) [2023-12-27 05:12:51,404][105620] Updated weights for policy 1, policy_version 1901120 (0.0008) [2023-12-27 05:12:51,456][105692] Updated weights for policy 0, policy_version 1896576 (0.0007) [2023-12-27 05:12:51,467][105620] Updated weights for policy 1, policy_version 1901130 (0.0008) [2023-12-27 05:12:51,517][105692] Updated weights for policy 0, policy_version 1896586 (0.0007) [2023-12-27 05:12:52,203][105620] Updated weights for policy 1, policy_version 1901140 (0.0008) [2023-12-27 05:12:52,260][105620] Updated weights for policy 1, policy_version 1901150 (0.0008) [2023-12-27 05:12:52,289][105692] Updated weights for policy 0, policy_version 1896596 (0.0009) [2023-12-27 05:12:52,324][105620] Updated weights for policy 1, policy_version 1901160 (0.0007) [2023-12-27 05:12:52,355][105692] Updated weights for policy 0, policy_version 1896606 (0.0010) [2023-12-27 05:12:52,415][105692] Updated weights for policy 0, policy_version 1896616 (0.0011) [2023-12-27 05:12:52,966][105620] Updated weights for policy 1, policy_version 1901170 (0.0007) [2023-12-27 05:12:53,014][105620] Updated weights for policy 1, policy_version 1901180 (0.0008) [2023-12-27 05:12:53,067][105620] Updated weights for policy 1, policy_version 1901190 (0.0008) [2023-12-27 05:12:53,121][105620] Updated weights for policy 1, policy_version 1901200 (0.0010) [2023-12-27 05:12:53,150][105692] Updated weights for policy 0, policy_version 1896626 (0.0010) [2023-12-27 05:12:53,205][105692] Updated weights for policy 0, policy_version 1896636 (0.0005) [2023-12-27 05:12:53,256][105692] Updated weights for policy 0, policy_version 1896646 (0.0005) [2023-12-27 05:12:53,318][105692] Updated weights for policy 0, policy_version 1896656 (0.0010) [2023-12-27 05:12:53,930][105692] Updated weights for policy 0, policy_version 1896666 (0.0005) [2023-12-27 05:12:53,984][105620] Updated weights for policy 1, policy_version 1901210 (0.0009) [2023-12-27 05:12:53,987][105692] Updated weights for policy 0, policy_version 1896676 (0.0005) [2023-12-27 05:12:54,033][105620] Updated weights for policy 1, policy_version 1901220 (0.0009) [2023-12-27 05:12:54,040][105692] Updated weights for policy 0, policy_version 1896686 (0.0005) [2023-12-27 05:12:54,083][105620] Updated weights for policy 1, policy_version 1901230 (0.0009) [2023-12-27 05:12:54,545][105692] Updated weights for policy 0, policy_version 1896696 (0.0006) [2023-12-27 05:12:54,603][105692] Updated weights for policy 0, policy_version 1896706 (0.0005) [2023-12-27 05:12:54,660][105692] Updated weights for policy 0, policy_version 1896716 (0.0005) [2023-12-27 05:12:55,026][105620] Updated weights for policy 1, policy_version 1901240 (0.0010) [2023-12-27 05:12:55,094][105620] Updated weights for policy 1, policy_version 1901250 (0.0010) [2023-12-27 05:12:55,148][105620] Updated weights for policy 1, policy_version 1901260 (0.0010) [2023-12-27 05:12:55,196][105692] Updated weights for policy 0, policy_version 1896726 (0.0005) [2023-12-27 05:12:55,249][105692] Updated weights for policy 0, policy_version 1896736 (0.0005) [2023-12-27 05:12:55,297][105692] Updated weights for policy 0, policy_version 1896746 (0.0005) [2023-12-27 05:12:55,855][105692] Updated weights for policy 0, policy_version 1896756 (0.0007) [2023-12-27 05:12:55,912][105692] Updated weights for policy 0, policy_version 1896766 (0.0009) [2023-12-27 05:12:55,960][105692] Updated weights for policy 0, policy_version 1896776 (0.0008) [2023-12-27 05:12:55,999][105620] Updated weights for policy 1, policy_version 1901270 (0.0010) [2023-12-27 05:12:56,046][105620] Updated weights for policy 1, policy_version 1901280 (0.0008) [2023-12-27 05:12:56,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 972439552. Throughput: 0: 9638.6, 1: 9612.4. Samples: 972447248. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:12:56,062][104569] Avg episode reward: [(0, '8348.900'), (1, '9255.604')] [2023-12-27 05:12:56,097][105620] Updated weights for policy 1, policy_version 1901290 (0.0009) [2023-12-27 05:12:56,696][105692] Updated weights for policy 0, policy_version 1896786 (0.0009) [2023-12-27 05:12:56,744][105620] Updated weights for policy 1, policy_version 1901300 (0.0008) [2023-12-27 05:12:56,745][105692] Updated weights for policy 0, policy_version 1896796 (0.0007) [2023-12-27 05:12:56,799][105692] Updated weights for policy 0, policy_version 1896806 (0.0007) [2023-12-27 05:12:56,804][105620] Updated weights for policy 1, policy_version 1901310 (0.0006) [2023-12-27 05:12:56,855][105620] Updated weights for policy 1, policy_version 1901320 (0.0005) [2023-12-27 05:12:56,856][105692] Updated weights for policy 0, policy_version 1896816 (0.0009) [2023-12-27 05:12:57,488][105620] Updated weights for policy 1, policy_version 1901330 (0.0005) [2023-12-27 05:12:57,543][105620] Updated weights for policy 1, policy_version 1901340 (0.0005) [2023-12-27 05:12:57,593][105620] Updated weights for policy 1, policy_version 1901350 (0.0005) [2023-12-27 05:12:57,629][105692] Updated weights for policy 0, policy_version 1896826 (0.0010) [2023-12-27 05:12:57,654][105620] Updated weights for policy 1, policy_version 1901360 (0.0005) [2023-12-27 05:12:57,688][105692] Updated weights for policy 0, policy_version 1896836 (0.0009) [2023-12-27 05:12:57,741][105692] Updated weights for policy 0, policy_version 1896847 (0.0010) [2023-12-27 05:12:58,235][105620] Updated weights for policy 1, policy_version 1901370 (0.0009) [2023-12-27 05:12:58,306][105620] Updated weights for policy 1, policy_version 1901380 (0.0006) [2023-12-27 05:12:58,389][105620] Updated weights for policy 1, policy_version 1901390 (0.0009) [2023-12-27 05:12:58,510][105692] Updated weights for policy 0, policy_version 1896857 (0.0011) [2023-12-27 05:12:58,575][105692] Updated weights for policy 0, policy_version 1896867 (0.0010) [2023-12-27 05:12:58,642][105692] Updated weights for policy 0, policy_version 1896877 (0.0008) [2023-12-27 05:12:59,151][105620] Updated weights for policy 1, policy_version 1901400 (0.0009) [2023-12-27 05:12:59,214][105620] Updated weights for policy 1, policy_version 1901410 (0.0010) [2023-12-27 05:12:59,287][105620] Updated weights for policy 1, policy_version 1901420 (0.0008) [2023-12-27 05:12:59,489][105692] Updated weights for policy 0, policy_version 1896887 (0.0009) [2023-12-27 05:12:59,543][105692] Updated weights for policy 0, policy_version 1896897 (0.0009) [2023-12-27 05:12:59,609][105692] Updated weights for policy 0, policy_version 1896907 (0.0009) [2023-12-27 05:13:00,038][105620] Updated weights for policy 1, policy_version 1901430 (0.0007) [2023-12-27 05:13:00,089][105620] Updated weights for policy 1, policy_version 1901440 (0.0008) [2023-12-27 05:13:00,148][105620] Updated weights for policy 1, policy_version 1901450 (0.0008) [2023-12-27 05:13:00,378][105692] Updated weights for policy 0, policy_version 1896917 (0.0009) [2023-12-27 05:13:00,441][105692] Updated weights for policy 0, policy_version 1896927 (0.0009) [2023-12-27 05:13:00,506][105692] Updated weights for policy 0, policy_version 1896937 (0.0009) [2023-12-27 05:13:00,888][105620] Updated weights for policy 1, policy_version 1901460 (0.0009) [2023-12-27 05:13:00,945][105620] Updated weights for policy 1, policy_version 1901470 (0.0008) [2023-12-27 05:13:00,999][105620] Updated weights for policy 1, policy_version 1901480 (0.0009) [2023-12-27 05:13:01,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 972537856. Throughput: 0: 9670.9, 1: 9675.7. Samples: 972507808. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:13:01,063][104569] Avg episode reward: [(0, '8541.136'), (1, '9255.658')] [2023-12-27 05:13:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001896944_485687296.pth... [2023-12-27 05:13:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001901488_486850560.pth... [2023-12-27 05:13:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001895824_485400576.pth [2023-12-27 05:13:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001900368_486563840.pth [2023-12-27 05:13:01,235][105692] Updated weights for policy 0, policy_version 1896947 (0.0009) [2023-12-27 05:13:01,298][105692] Updated weights for policy 0, policy_version 1896957 (0.0009) [2023-12-27 05:13:01,358][105692] Updated weights for policy 0, policy_version 1896967 (0.0008) [2023-12-27 05:13:01,775][105620] Updated weights for policy 1, policy_version 1901490 (0.0008) [2023-12-27 05:13:01,824][105620] Updated weights for policy 1, policy_version 1901500 (0.0009) [2023-12-27 05:13:01,883][105620] Updated weights for policy 1, policy_version 1901510 (0.0009) [2023-12-27 05:13:01,946][105620] Updated weights for policy 1, policy_version 1901520 (0.0009) [2023-12-27 05:13:02,069][105692] Updated weights for policy 0, policy_version 1896977 (0.0007) [2023-12-27 05:13:02,131][105692] Updated weights for policy 0, policy_version 1896987 (0.0009) [2023-12-27 05:13:02,190][105692] Updated weights for policy 0, policy_version 1896997 (0.0009) [2023-12-27 05:13:02,248][105692] Updated weights for policy 0, policy_version 1897007 (0.0009) [2023-12-27 05:13:02,718][105620] Updated weights for policy 1, policy_version 1901530 (0.0010) [2023-12-27 05:13:02,770][105620] Updated weights for policy 1, policy_version 1901540 (0.0008) [2023-12-27 05:13:02,824][105620] Updated weights for policy 1, policy_version 1901550 (0.0009) [2023-12-27 05:13:02,952][105692] Updated weights for policy 0, policy_version 1897017 (0.0007) [2023-12-27 05:13:03,020][105692] Updated weights for policy 0, policy_version 1897027 (0.0005) [2023-12-27 05:13:03,075][105692] Updated weights for policy 0, policy_version 1897037 (0.0005) [2023-12-27 05:13:03,598][105620] Updated weights for policy 1, policy_version 1901560 (0.0008) [2023-12-27 05:13:03,612][105692] Updated weights for policy 0, policy_version 1897047 (0.0005) [2023-12-27 05:13:03,658][105620] Updated weights for policy 1, policy_version 1901570 (0.0009) [2023-12-27 05:13:03,661][105692] Updated weights for policy 0, policy_version 1897057 (0.0005) [2023-12-27 05:13:03,708][105620] Updated weights for policy 1, policy_version 1901580 (0.0009) [2023-12-27 05:13:03,710][105692] Updated weights for policy 0, policy_version 1897067 (0.0005) [2023-12-27 05:13:04,437][105692] Updated weights for policy 0, policy_version 1897077 (0.0005) [2023-12-27 05:13:04,442][105620] Updated weights for policy 1, policy_version 1901590 (0.0007) [2023-12-27 05:13:04,501][105692] Updated weights for policy 0, policy_version 1897087 (0.0008) [2023-12-27 05:13:04,510][105620] Updated weights for policy 1, policy_version 1901600 (0.0006) [2023-12-27 05:13:04,559][105692] Updated weights for policy 0, policy_version 1897097 (0.0009) [2023-12-27 05:13:04,574][105620] Updated weights for policy 1, policy_version 1901610 (0.0006) [2023-12-27 05:13:05,153][105692] Updated weights for policy 0, policy_version 1897107 (0.0009) [2023-12-27 05:13:05,176][105620] Updated weights for policy 1, policy_version 1901620 (0.0009) [2023-12-27 05:13:05,212][105692] Updated weights for policy 0, policy_version 1897117 (0.0005) [2023-12-27 05:13:05,228][105620] Updated weights for policy 1, policy_version 1901630 (0.0010) [2023-12-27 05:13:05,261][105692] Updated weights for policy 0, policy_version 1897127 (0.0010) [2023-12-27 05:13:05,283][105620] Updated weights for policy 1, policy_version 1901640 (0.0010) [2023-12-27 05:13:05,880][105692] Updated weights for policy 0, policy_version 1897137 (0.0010) [2023-12-27 05:13:05,932][105692] Updated weights for policy 0, policy_version 1897147 (0.0009) [2023-12-27 05:13:05,961][105620] Updated weights for policy 1, policy_version 1901650 (0.0009) [2023-12-27 05:13:05,987][105692] Updated weights for policy 0, policy_version 1897157 (0.0009) [2023-12-27 05:13:06,017][105620] Updated weights for policy 1, policy_version 1901660 (0.0005) [2023-12-27 05:13:06,036][105692] Updated weights for policy 0, policy_version 1897167 (0.0009) [2023-12-27 05:13:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 972636160. Throughput: 0: 9758.4, 1: 9565.2. Samples: 972622244. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:13:06,062][104569] Avg episode reward: [(0, '8452.504'), (1, '9345.470')] [2023-12-27 05:13:06,074][105620] Updated weights for policy 1, policy_version 1901670 (0.0008) [2023-12-27 05:13:06,137][105620] Updated weights for policy 1, policy_version 1901680 (0.0010) [2023-12-27 05:13:06,851][105620] Updated weights for policy 1, policy_version 1901690 (0.0010) [2023-12-27 05:13:06,885][105692] Updated weights for policy 0, policy_version 1897177 (0.0007) [2023-12-27 05:13:06,910][105620] Updated weights for policy 1, policy_version 1901700 (0.0010) [2023-12-27 05:13:06,941][105692] Updated weights for policy 0, policy_version 1897187 (0.0009) [2023-12-27 05:13:06,959][105620] Updated weights for policy 1, policy_version 1901710 (0.0010) [2023-12-27 05:13:06,995][105692] Updated weights for policy 0, policy_version 1897197 (0.0007) [2023-12-27 05:13:07,644][105620] Updated weights for policy 1, policy_version 1901720 (0.0006) [2023-12-27 05:13:07,694][105620] Updated weights for policy 1, policy_version 1901730 (0.0005) [2023-12-27 05:13:07,747][105620] Updated weights for policy 1, policy_version 1901740 (0.0007) [2023-12-27 05:13:07,763][105692] Updated weights for policy 0, policy_version 1897207 (0.0010) [2023-12-27 05:13:07,814][105692] Updated weights for policy 0, policy_version 1897217 (0.0010) [2023-12-27 05:13:07,866][105692] Updated weights for policy 0, policy_version 1897227 (0.0010) [2023-12-27 05:13:08,478][105620] Updated weights for policy 1, policy_version 1901750 (0.0007) [2023-12-27 05:13:08,542][105620] Updated weights for policy 1, policy_version 1901760 (0.0008) [2023-12-27 05:13:08,584][105692] Updated weights for policy 0, policy_version 1897237 (0.0008) [2023-12-27 05:13:08,604][105620] Updated weights for policy 1, policy_version 1901770 (0.0011) [2023-12-27 05:13:08,637][105692] Updated weights for policy 0, policy_version 1897247 (0.0009) [2023-12-27 05:13:08,688][105692] Updated weights for policy 0, policy_version 1897257 (0.0010) [2023-12-27 05:13:09,224][105620] Updated weights for policy 1, policy_version 1901780 (0.0009) [2023-12-27 05:13:09,283][105620] Updated weights for policy 1, policy_version 1901790 (0.0007) [2023-12-27 05:13:09,338][105620] Updated weights for policy 1, policy_version 1901800 (0.0005) [2023-12-27 05:13:09,461][105692] Updated weights for policy 0, policy_version 1897267 (0.0010) [2023-12-27 05:13:09,518][105692] Updated weights for policy 0, policy_version 1897277 (0.0009) [2023-12-27 05:13:09,580][105692] Updated weights for policy 0, policy_version 1897287 (0.0008) [2023-12-27 05:13:10,083][105620] Updated weights for policy 1, policy_version 1901810 (0.0007) [2023-12-27 05:13:10,148][105620] Updated weights for policy 1, policy_version 1901820 (0.0008) [2023-12-27 05:13:10,212][105620] Updated weights for policy 1, policy_version 1901830 (0.0009) [2023-12-27 05:13:10,272][105620] Updated weights for policy 1, policy_version 1901840 (0.0005) [2023-12-27 05:13:10,328][105692] Updated weights for policy 0, policy_version 1897297 (0.0009) [2023-12-27 05:13:10,393][105692] Updated weights for policy 0, policy_version 1897307 (0.0007) [2023-12-27 05:13:10,451][105692] Updated weights for policy 0, policy_version 1897317 (0.0005) [2023-12-27 05:13:10,501][105692] Updated weights for policy 0, policy_version 1897327 (0.0005) [2023-12-27 05:13:11,004][105620] Updated weights for policy 1, policy_version 1901850 (0.0009) [2023-12-27 05:13:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19355.3). Total num frames: 972726272. Throughput: 0: 9733.5, 1: 9604.7. Samples: 972740608. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:13:11,062][104569] Avg episode reward: [(0, '8538.053'), (1, '9345.528')] [2023-12-27 05:13:11,066][105620] Updated weights for policy 1, policy_version 1901860 (0.0009) [2023-12-27 05:13:11,079][105692] Updated weights for policy 0, policy_version 1897337 (0.0007) [2023-12-27 05:13:11,132][105620] Updated weights for policy 1, policy_version 1901870 (0.0009) [2023-12-27 05:13:11,142][105692] Updated weights for policy 0, policy_version 1897347 (0.0008) [2023-12-27 05:13:11,203][105692] Updated weights for policy 0, policy_version 1897357 (0.0009) [2023-12-27 05:13:11,941][105620] Updated weights for policy 1, policy_version 1901880 (0.0010) [2023-12-27 05:13:12,004][105692] Updated weights for policy 0, policy_version 1897367 (0.0007) [2023-12-27 05:13:12,006][105620] Updated weights for policy 1, policy_version 1901890 (0.0011) [2023-12-27 05:13:12,064][105620] Updated weights for policy 1, policy_version 1901900 (0.0009) [2023-12-27 05:13:12,068][105692] Updated weights for policy 0, policy_version 1897377 (0.0009) [2023-12-27 05:13:12,135][105692] Updated weights for policy 0, policy_version 1897387 (0.0008) [2023-12-27 05:13:12,798][105692] Updated weights for policy 0, policy_version 1897397 (0.0008) [2023-12-27 05:13:12,811][105620] Updated weights for policy 1, policy_version 1901910 (0.0009) [2023-12-27 05:13:12,870][105692] Updated weights for policy 0, policy_version 1897407 (0.0008) [2023-12-27 05:13:12,873][105620] Updated weights for policy 1, policy_version 1901920 (0.0008) [2023-12-27 05:13:12,921][105692] Updated weights for policy 0, policy_version 1897417 (0.0006) [2023-12-27 05:13:12,927][105620] Updated weights for policy 1, policy_version 1901930 (0.0007) [2023-12-27 05:13:13,490][105620] Updated weights for policy 1, policy_version 1901940 (0.0009) [2023-12-27 05:13:13,540][105620] Updated weights for policy 1, policy_version 1901950 (0.0009) [2023-12-27 05:13:13,601][105620] Updated weights for policy 1, policy_version 1901960 (0.0010) [2023-12-27 05:13:13,725][105692] Updated weights for policy 0, policy_version 1897427 (0.0006) [2023-12-27 05:13:13,771][105692] Updated weights for policy 0, policy_version 1897437 (0.0005) [2023-12-27 05:13:13,817][105692] Updated weights for policy 0, policy_version 1897447 (0.0005) [2023-12-27 05:13:14,204][105620] Updated weights for policy 1, policy_version 1901970 (0.0008) [2023-12-27 05:13:14,251][105620] Updated weights for policy 1, policy_version 1901980 (0.0005) [2023-12-27 05:13:14,310][105620] Updated weights for policy 1, policy_version 1901990 (0.0007) [2023-12-27 05:13:14,359][105620] Updated weights for policy 1, policy_version 1902000 (0.0005) [2023-12-27 05:13:14,387][105692] Updated weights for policy 0, policy_version 1897457 (0.0005) [2023-12-27 05:13:14,443][105692] Updated weights for policy 0, policy_version 1897467 (0.0005) [2023-12-27 05:13:14,494][105692] Updated weights for policy 0, policy_version 1897477 (0.0010) [2023-12-27 05:13:14,554][105692] Updated weights for policy 0, policy_version 1897487 (0.0008) [2023-12-27 05:13:14,989][105620] Updated weights for policy 1, policy_version 1902010 (0.0009) [2023-12-27 05:13:15,046][105620] Updated weights for policy 1, policy_version 1902020 (0.0009) [2023-12-27 05:13:15,103][105620] Updated weights for policy 1, policy_version 1902030 (0.0009) [2023-12-27 05:13:15,252][105692] Updated weights for policy 0, policy_version 1897497 (0.0006) [2023-12-27 05:13:15,322][105692] Updated weights for policy 0, policy_version 1897507 (0.0009) [2023-12-27 05:13:15,392][105692] Updated weights for policy 0, policy_version 1897517 (0.0011) [2023-12-27 05:13:15,765][105620] Updated weights for policy 1, policy_version 1902040 (0.0010) [2023-12-27 05:13:15,816][105620] Updated weights for policy 1, policy_version 1902050 (0.0010) [2023-12-27 05:13:15,874][105620] Updated weights for policy 1, policy_version 1902060 (0.0010) [2023-12-27 05:13:16,049][105692] Updated weights for policy 0, policy_version 1897527 (0.0008) [2023-12-27 05:13:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 972832768. Throughput: 0: 9743.3, 1: 9581.1. Samples: 972799276. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:13:16,063][104569] Avg episode reward: [(0, '8991.204'), (1, '9345.462')] [2023-12-27 05:13:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001902064_486998016.pth... [2023-12-27 05:13:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001900944_486711296.pth [2023-12-27 05:13:16,107][105692] Updated weights for policy 0, policy_version 1897537 (0.0007) [2023-12-27 05:13:16,169][105692] Updated weights for policy 0, policy_version 1897547 (0.0005) [2023-12-27 05:13:16,200][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001897552_485842944.pth... [2023-12-27 05:13:16,206][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001896400_485548032.pth [2023-12-27 05:13:16,590][105620] Updated weights for policy 1, policy_version 1902070 (0.0010) [2023-12-27 05:13:16,649][105620] Updated weights for policy 1, policy_version 1902080 (0.0009) [2023-12-27 05:13:16,705][105620] Updated weights for policy 1, policy_version 1902090 (0.0009) [2023-12-27 05:13:16,827][105692] Updated weights for policy 0, policy_version 1897557 (0.0008) [2023-12-27 05:13:16,891][105692] Updated weights for policy 0, policy_version 1897567 (0.0009) [2023-12-27 05:13:16,944][105692] Updated weights for policy 0, policy_version 1897577 (0.0010) [2023-12-27 05:13:17,320][105620] Updated weights for policy 1, policy_version 1902100 (0.0007) [2023-12-27 05:13:17,380][105620] Updated weights for policy 1, policy_version 1902110 (0.0006) [2023-12-27 05:13:17,443][105620] Updated weights for policy 1, policy_version 1902120 (0.0007) [2023-12-27 05:13:17,608][105692] Updated weights for policy 0, policy_version 1897587 (0.0008) [2023-12-27 05:13:17,660][105692] Updated weights for policy 0, policy_version 1897597 (0.0006) [2023-12-27 05:13:17,717][105692] Updated weights for policy 0, policy_version 1897607 (0.0008) [2023-12-27 05:13:18,151][105620] Updated weights for policy 1, policy_version 1902130 (0.0009) [2023-12-27 05:13:18,205][105620] Updated weights for policy 1, policy_version 1902140 (0.0008) [2023-12-27 05:13:18,250][105620] Updated weights for policy 1, policy_version 1902150 (0.0008) [2023-12-27 05:13:18,298][105620] Updated weights for policy 1, policy_version 1902160 (0.0010) [2023-12-27 05:13:18,512][105692] Updated weights for policy 0, policy_version 1897617 (0.0010) [2023-12-27 05:13:18,577][105692] Updated weights for policy 0, policy_version 1897627 (0.0008) [2023-12-27 05:13:18,635][105692] Updated weights for policy 0, policy_version 1897637 (0.0008) [2023-12-27 05:13:18,700][105692] Updated weights for policy 0, policy_version 1897647 (0.0007) [2023-12-27 05:13:18,996][105620] Updated weights for policy 1, policy_version 1902170 (0.0006) [2023-12-27 05:13:19,066][105620] Updated weights for policy 1, policy_version 1902180 (0.0006) [2023-12-27 05:13:19,130][105620] Updated weights for policy 1, policy_version 1902190 (0.0006) [2023-12-27 05:13:19,336][105692] Updated weights for policy 0, policy_version 1897657 (0.0007) [2023-12-27 05:13:19,401][105692] Updated weights for policy 0, policy_version 1897667 (0.0009) [2023-12-27 05:13:19,467][105692] Updated weights for policy 0, policy_version 1897677 (0.0008) [2023-12-27 05:13:19,831][105620] Updated weights for policy 1, policy_version 1902200 (0.0007) [2023-12-27 05:13:19,902][105620] Updated weights for policy 1, policy_version 1902210 (0.0009) [2023-12-27 05:13:19,970][105620] Updated weights for policy 1, policy_version 1902220 (0.0009) [2023-12-27 05:13:20,227][105692] Updated weights for policy 0, policy_version 1897687 (0.0009) [2023-12-27 05:13:20,290][105692] Updated weights for policy 0, policy_version 1897697 (0.0009) [2023-12-27 05:13:20,360][105692] Updated weights for policy 0, policy_version 1897707 (0.0010) [2023-12-27 05:13:20,594][105620] Updated weights for policy 1, policy_version 1902230 (0.0009) [2023-12-27 05:13:20,661][105620] Updated weights for policy 1, policy_version 1902240 (0.0010) [2023-12-27 05:13:20,724][105620] Updated weights for policy 1, policy_version 1902250 (0.0008) [2023-12-27 05:13:21,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 972931072. Throughput: 0: 9698.0, 1: 9683.9. Samples: 972922568. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:13:21,063][104569] Avg episode reward: [(0, '8722.648'), (1, '9252.999')] [2023-12-27 05:13:21,095][105692] Updated weights for policy 0, policy_version 1897717 (0.0010) [2023-12-27 05:13:21,149][105692] Updated weights for policy 0, policy_version 1897727 (0.0009) [2023-12-27 05:13:21,219][105692] Updated weights for policy 0, policy_version 1897737 (0.0010) [2023-12-27 05:13:21,472][105620] Updated weights for policy 1, policy_version 1902260 (0.0009) [2023-12-27 05:13:21,519][105620] Updated weights for policy 1, policy_version 1902270 (0.0009) [2023-12-27 05:13:21,566][105620] Updated weights for policy 1, policy_version 1902280 (0.0008) [2023-12-27 05:13:22,001][105692] Updated weights for policy 0, policy_version 1897747 (0.0008) [2023-12-27 05:13:22,052][105692] Updated weights for policy 0, policy_version 1897757 (0.0008) [2023-12-27 05:13:22,104][105692] Updated weights for policy 0, policy_version 1897767 (0.0006) [2023-12-27 05:13:22,432][105620] Updated weights for policy 1, policy_version 1902290 (0.0009) [2023-12-27 05:13:22,490][105620] Updated weights for policy 1, policy_version 1902300 (0.0009) [2023-12-27 05:13:22,542][105620] Updated weights for policy 1, policy_version 1902310 (0.0009) [2023-12-27 05:13:22,599][105620] Updated weights for policy 1, policy_version 1902320 (0.0009) [2023-12-27 05:13:22,822][105692] Updated weights for policy 0, policy_version 1897777 (0.0006) [2023-12-27 05:13:22,892][105692] Updated weights for policy 0, policy_version 1897787 (0.0009) [2023-12-27 05:13:22,961][105692] Updated weights for policy 0, policy_version 1897797 (0.0010) [2023-12-27 05:13:23,034][105692] Updated weights for policy 0, policy_version 1897807 (0.0010) [2023-12-27 05:13:23,333][105620] Updated weights for policy 1, policy_version 1902330 (0.0006) [2023-12-27 05:13:23,395][105620] Updated weights for policy 1, policy_version 1902340 (0.0006) [2023-12-27 05:13:23,448][105620] Updated weights for policy 1, policy_version 1902350 (0.0009) [2023-12-27 05:13:23,724][105692] Updated weights for policy 0, policy_version 1897817 (0.0009) [2023-12-27 05:13:23,772][105692] Updated weights for policy 0, policy_version 1897827 (0.0008) [2023-12-27 05:13:23,826][105692] Updated weights for policy 0, policy_version 1897837 (0.0008) [2023-12-27 05:13:24,098][105620] Updated weights for policy 1, policy_version 1902360 (0.0010) [2023-12-27 05:13:24,150][105620] Updated weights for policy 1, policy_version 1902370 (0.0011) [2023-12-27 05:13:24,199][105620] Updated weights for policy 1, policy_version 1902380 (0.0010) [2023-12-27 05:13:24,581][105692] Updated weights for policy 0, policy_version 1897847 (0.0008) [2023-12-27 05:13:24,636][105692] Updated weights for policy 0, policy_version 1897857 (0.0008) [2023-12-27 05:13:24,691][105692] Updated weights for policy 0, policy_version 1897867 (0.0008) [2023-12-27 05:13:24,974][105620] Updated weights for policy 1, policy_version 1902390 (0.0009) [2023-12-27 05:13:25,035][105620] Updated weights for policy 1, policy_version 1902400 (0.0009) [2023-12-27 05:13:25,083][105620] Updated weights for policy 1, policy_version 1902410 (0.0010) [2023-12-27 05:13:25,497][105692] Updated weights for policy 0, policy_version 1897877 (0.0007) [2023-12-27 05:13:25,546][105692] Updated weights for policy 0, policy_version 1897887 (0.0006) [2023-12-27 05:13:25,616][105692] Updated weights for policy 0, policy_version 1897897 (0.0005) [2023-12-27 05:13:25,693][105620] Updated weights for policy 1, policy_version 1902420 (0.0009) [2023-12-27 05:13:25,741][105620] Updated weights for policy 1, policy_version 1902430 (0.0010) [2023-12-27 05:13:25,785][105620] Updated weights for policy 1, policy_version 1902440 (0.0010) [2023-12-27 05:13:26,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 973029376. Throughput: 0: 9689.1, 1: 9701.6. Samples: 973036888. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:13:26,063][104569] Avg episode reward: [(0, '8092.138'), (1, '9160.657')] [2023-12-27 05:13:26,145][105692] Updated weights for policy 0, policy_version 1897907 (0.0006) [2023-12-27 05:13:26,213][105692] Updated weights for policy 0, policy_version 1897917 (0.0005) [2023-12-27 05:13:26,280][105692] Updated weights for policy 0, policy_version 1897927 (0.0006) [2023-12-27 05:13:26,434][105620] Updated weights for policy 1, policy_version 1902450 (0.0010) [2023-12-27 05:13:26,495][105620] Updated weights for policy 1, policy_version 1902460 (0.0008) [2023-12-27 05:13:26,555][105620] Updated weights for policy 1, policy_version 1902470 (0.0009) [2023-12-27 05:13:26,608][105620] Updated weights for policy 1, policy_version 1902480 (0.0005) [2023-12-27 05:13:27,047][105692] Updated weights for policy 0, policy_version 1897937 (0.0010) [2023-12-27 05:13:27,103][105692] Updated weights for policy 0, policy_version 1897947 (0.0010) [2023-12-27 05:13:27,161][105692] Updated weights for policy 0, policy_version 1897957 (0.0011) [2023-12-27 05:13:27,181][105620] Updated weights for policy 1, policy_version 1902490 (0.0006) [2023-12-27 05:13:27,215][105692] Updated weights for policy 0, policy_version 1897967 (0.0008) [2023-12-27 05:13:27,234][105620] Updated weights for policy 1, policy_version 1902500 (0.0006) [2023-12-27 05:13:27,293][105620] Updated weights for policy 1, policy_version 1902510 (0.0005) [2023-12-27 05:13:27,941][105620] Updated weights for policy 1, policy_version 1902520 (0.0009) [2023-12-27 05:13:27,987][105692] Updated weights for policy 0, policy_version 1897977 (0.0010) [2023-12-27 05:13:27,988][105620] Updated weights for policy 1, policy_version 1902530 (0.0010) [2023-12-27 05:13:28,037][105620] Updated weights for policy 1, policy_version 1902540 (0.0006) [2023-12-27 05:13:28,045][105692] Updated weights for policy 0, policy_version 1897987 (0.0010) [2023-12-27 05:13:28,099][105692] Updated weights for policy 0, policy_version 1897997 (0.0010) [2023-12-27 05:13:28,635][105620] Updated weights for policy 1, policy_version 1902550 (0.0006) [2023-12-27 05:13:28,685][105620] Updated weights for policy 1, policy_version 1902560 (0.0005) [2023-12-27 05:13:28,736][105620] Updated weights for policy 1, policy_version 1902570 (0.0006) [2023-12-27 05:13:28,830][105692] Updated weights for policy 0, policy_version 1898007 (0.0011) [2023-12-27 05:13:28,901][105692] Updated weights for policy 0, policy_version 1898017 (0.0006) [2023-12-27 05:13:28,961][105692] Updated weights for policy 0, policy_version 1898027 (0.0006) [2023-12-27 05:13:29,430][105620] Updated weights for policy 1, policy_version 1902580 (0.0007) [2023-12-27 05:13:29,483][105620] Updated weights for policy 1, policy_version 1902590 (0.0007) [2023-12-27 05:13:29,541][105620] Updated weights for policy 1, policy_version 1902600 (0.0008) [2023-12-27 05:13:29,561][105692] Updated weights for policy 0, policy_version 1898037 (0.0008) [2023-12-27 05:13:29,623][105692] Updated weights for policy 0, policy_version 1898047 (0.0010) [2023-12-27 05:13:29,684][105692] Updated weights for policy 0, policy_version 1898057 (0.0009) [2023-12-27 05:13:30,354][105692] Updated weights for policy 0, policy_version 1898067 (0.0008) [2023-12-27 05:13:30,357][105620] Updated weights for policy 1, policy_version 1902610 (0.0006) [2023-12-27 05:13:30,412][105620] Updated weights for policy 1, policy_version 1902620 (0.0008) [2023-12-27 05:13:30,415][105692] Updated weights for policy 0, policy_version 1898077 (0.0008) [2023-12-27 05:13:30,470][105620] Updated weights for policy 1, policy_version 1902630 (0.0009) [2023-12-27 05:13:30,479][105692] Updated weights for policy 0, policy_version 1898087 (0.0009) [2023-12-27 05:13:30,533][105620] Updated weights for policy 1, policy_version 1902640 (0.0008) [2023-12-27 05:13:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 973127680. Throughput: 0: 9762.3, 1: 9808.2. Samples: 973100560. Policy #0 lag: (min: 6.0, avg: 6.9, max: 32.0) [2023-12-27 05:13:31,063][104569] Avg episode reward: [(0, '7999.686'), (1, '9160.794')] [2023-12-27 05:13:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001898096_485982208.pth... [2023-12-27 05:13:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001902640_487145472.pth... [2023-12-27 05:13:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001896944_485687296.pth [2023-12-27 05:13:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001901488_486850560.pth [2023-12-27 05:13:31,248][105692] Updated weights for policy 0, policy_version 1898097 (0.0007) [2023-12-27 05:13:31,263][105620] Updated weights for policy 1, policy_version 1902650 (0.0007) [2023-12-27 05:13:31,306][105692] Updated weights for policy 0, policy_version 1898107 (0.0009) [2023-12-27 05:13:31,330][105620] Updated weights for policy 1, policy_version 1902660 (0.0007) [2023-12-27 05:13:31,365][105692] Updated weights for policy 0, policy_version 1898117 (0.0009) [2023-12-27 05:13:31,396][105620] Updated weights for policy 1, policy_version 1902670 (0.0009) [2023-12-27 05:13:31,424][105692] Updated weights for policy 0, policy_version 1898127 (0.0007) [2023-12-27 05:13:32,088][105692] Updated weights for policy 0, policy_version 1898137 (0.0007) [2023-12-27 05:13:32,126][105620] Updated weights for policy 1, policy_version 1902680 (0.0009) [2023-12-27 05:13:32,141][105692] Updated weights for policy 0, policy_version 1898147 (0.0008) [2023-12-27 05:13:32,190][105620] Updated weights for policy 1, policy_version 1902690 (0.0010) [2023-12-27 05:13:32,196][105692] Updated weights for policy 0, policy_version 1898157 (0.0006) [2023-12-27 05:13:32,249][105620] Updated weights for policy 1, policy_version 1902700 (0.0009) [2023-12-27 05:13:32,895][105692] Updated weights for policy 0, policy_version 1898167 (0.0008) [2023-12-27 05:13:32,947][105692] Updated weights for policy 0, policy_version 1898177 (0.0009) [2023-12-27 05:13:32,999][105692] Updated weights for policy 0, policy_version 1898187 (0.0008) [2023-12-27 05:13:33,012][105620] Updated weights for policy 1, policy_version 1902710 (0.0008) [2023-12-27 05:13:33,065][105620] Updated weights for policy 1, policy_version 1902720 (0.0008) [2023-12-27 05:13:33,117][105620] Updated weights for policy 1, policy_version 1902730 (0.0008) [2023-12-27 05:13:33,789][105692] Updated weights for policy 0, policy_version 1898197 (0.0007) [2023-12-27 05:13:33,828][105620] Updated weights for policy 1, policy_version 1902740 (0.0007) [2023-12-27 05:13:33,843][105692] Updated weights for policy 0, policy_version 1898207 (0.0007) [2023-12-27 05:13:33,873][105620] Updated weights for policy 1, policy_version 1902750 (0.0005) [2023-12-27 05:13:33,898][105692] Updated weights for policy 0, policy_version 1898217 (0.0009) [2023-12-27 05:13:33,918][105620] Updated weights for policy 1, policy_version 1902760 (0.0007) [2023-12-27 05:13:34,665][105692] Updated weights for policy 0, policy_version 1898227 (0.0009) [2023-12-27 05:13:34,695][105620] Updated weights for policy 1, policy_version 1902770 (0.0008) [2023-12-27 05:13:34,736][105692] Updated weights for policy 0, policy_version 1898237 (0.0008) [2023-12-27 05:13:34,757][105620] Updated weights for policy 1, policy_version 1902780 (0.0006) [2023-12-27 05:13:34,804][105692] Updated weights for policy 0, policy_version 1898247 (0.0008) [2023-12-27 05:13:34,821][105620] Updated weights for policy 1, policy_version 1902790 (0.0007) [2023-12-27 05:13:34,878][105620] Updated weights for policy 1, policy_version 1902800 (0.0008) [2023-12-27 05:13:35,361][105692] Updated weights for policy 0, policy_version 1898257 (0.0008) [2023-12-27 05:13:35,411][105692] Updated weights for policy 0, policy_version 1898267 (0.0009) [2023-12-27 05:13:35,461][105692] Updated weights for policy 0, policy_version 1898277 (0.0009) [2023-12-27 05:13:35,514][105692] Updated weights for policy 0, policy_version 1898287 (0.0010) [2023-12-27 05:13:35,550][105620] Updated weights for policy 1, policy_version 1902810 (0.0008) [2023-12-27 05:13:35,600][105620] Updated weights for policy 1, policy_version 1902820 (0.0009) [2023-12-27 05:13:35,657][105620] Updated weights for policy 1, policy_version 1902830 (0.0007) [2023-12-27 05:13:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 973225984. Throughput: 0: 9825.3, 1: 9793.6. Samples: 973215104. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:13:36,063][104569] Avg episode reward: [(0, '8446.444'), (1, '9160.876')] [2023-12-27 05:13:36,190][105692] Updated weights for policy 0, policy_version 1898297 (0.0010) [2023-12-27 05:13:36,213][105620] Updated weights for policy 1, policy_version 1902840 (0.0007) [2023-12-27 05:13:36,254][105692] Updated weights for policy 0, policy_version 1898307 (0.0009) [2023-12-27 05:13:36,264][105620] Updated weights for policy 1, policy_version 1902850 (0.0007) [2023-12-27 05:13:36,315][105620] Updated weights for policy 1, policy_version 1902860 (0.0008) [2023-12-27 05:13:36,320][105692] Updated weights for policy 0, policy_version 1898317 (0.0008) [2023-12-27 05:13:36,988][105692] Updated weights for policy 0, policy_version 1898327 (0.0008) [2023-12-27 05:13:37,045][105692] Updated weights for policy 0, policy_version 1898337 (0.0009) [2023-12-27 05:13:37,098][105620] Updated weights for policy 1, policy_version 1902870 (0.0008) [2023-12-27 05:13:37,100][105692] Updated weights for policy 0, policy_version 1898347 (0.0007) [2023-12-27 05:13:37,148][105620] Updated weights for policy 1, policy_version 1902880 (0.0006) [2023-12-27 05:13:37,207][105620] Updated weights for policy 1, policy_version 1902890 (0.0005) [2023-12-27 05:13:37,870][105692] Updated weights for policy 0, policy_version 1898357 (0.0006) [2023-12-27 05:13:37,883][105620] Updated weights for policy 1, policy_version 1902900 (0.0007) [2023-12-27 05:13:37,916][105692] Updated weights for policy 0, policy_version 1898367 (0.0009) [2023-12-27 05:13:37,938][105620] Updated weights for policy 1, policy_version 1902910 (0.0010) [2023-12-27 05:13:37,965][105692] Updated weights for policy 0, policy_version 1898377 (0.0006) [2023-12-27 05:13:37,997][105620] Updated weights for policy 1, policy_version 1902920 (0.0010) [2023-12-27 05:13:38,744][105692] Updated weights for policy 0, policy_version 1898387 (0.0007) [2023-12-27 05:13:38,749][105620] Updated weights for policy 1, policy_version 1902930 (0.0010) [2023-12-27 05:13:38,806][105692] Updated weights for policy 0, policy_version 1898397 (0.0006) [2023-12-27 05:13:38,807][105620] Updated weights for policy 1, policy_version 1902940 (0.0010) [2023-12-27 05:13:38,859][105620] Updated weights for policy 1, policy_version 1902950 (0.0010) [2023-12-27 05:13:38,861][105692] Updated weights for policy 0, policy_version 1898407 (0.0005) [2023-12-27 05:13:38,916][105620] Updated weights for policy 1, policy_version 1902960 (0.0010) [2023-12-27 05:13:39,648][105692] Updated weights for policy 0, policy_version 1898417 (0.0006) [2023-12-27 05:13:39,711][105692] Updated weights for policy 0, policy_version 1898427 (0.0008) [2023-12-27 05:13:39,735][105620] Updated weights for policy 1, policy_version 1902970 (0.0007) [2023-12-27 05:13:39,771][105692] Updated weights for policy 0, policy_version 1898437 (0.0008) [2023-12-27 05:13:39,797][105620] Updated weights for policy 1, policy_version 1902980 (0.0007) [2023-12-27 05:13:39,837][105692] Updated weights for policy 0, policy_version 1898447 (0.0008) [2023-12-27 05:13:39,859][105620] Updated weights for policy 1, policy_version 1902990 (0.0009) [2023-12-27 05:13:40,523][105692] Updated weights for policy 0, policy_version 1898457 (0.0008) [2023-12-27 05:13:40,583][105692] Updated weights for policy 0, policy_version 1898467 (0.0007) [2023-12-27 05:13:40,619][105620] Updated weights for policy 1, policy_version 1903000 (0.0008) [2023-12-27 05:13:40,638][105692] Updated weights for policy 0, policy_version 1898477 (0.0006) [2023-12-27 05:13:40,677][105620] Updated weights for policy 1, policy_version 1903010 (0.0009) [2023-12-27 05:13:40,738][105620] Updated weights for policy 1, policy_version 1903020 (0.0006) [2023-12-27 05:13:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 973324288. Throughput: 0: 9761.2, 1: 9919.9. Samples: 973332900. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:13:41,062][104569] Avg episode reward: [(0, '8629.479'), (1, '9253.168')] [2023-12-27 05:13:41,392][105692] Updated weights for policy 0, policy_version 1898487 (0.0011) [2023-12-27 05:13:41,442][105620] Updated weights for policy 1, policy_version 1903030 (0.0009) [2023-12-27 05:13:41,456][105692] Updated weights for policy 0, policy_version 1898497 (0.0011) [2023-12-27 05:13:41,496][105620] Updated weights for policy 1, policy_version 1903040 (0.0011) [2023-12-27 05:13:41,519][105692] Updated weights for policy 0, policy_version 1898507 (0.0010) [2023-12-27 05:13:41,549][105620] Updated weights for policy 1, policy_version 1903050 (0.0011) [2023-12-27 05:13:42,225][105620] Updated weights for policy 1, policy_version 1903060 (0.0010) [2023-12-27 05:13:42,280][105620] Updated weights for policy 1, policy_version 1903070 (0.0010) [2023-12-27 05:13:42,339][105692] Updated weights for policy 0, policy_version 1898517 (0.0009) [2023-12-27 05:13:42,348][105620] Updated weights for policy 1, policy_version 1903080 (0.0008) [2023-12-27 05:13:42,406][105692] Updated weights for policy 0, policy_version 1898527 (0.0007) [2023-12-27 05:13:42,472][105692] Updated weights for policy 0, policy_version 1898537 (0.0008) [2023-12-27 05:13:43,083][105620] Updated weights for policy 1, policy_version 1903090 (0.0008) [2023-12-27 05:13:43,141][105620] Updated weights for policy 1, policy_version 1903100 (0.0010) [2023-12-27 05:13:43,147][105692] Updated weights for policy 0, policy_version 1898547 (0.0006) [2023-12-27 05:13:43,203][105620] Updated weights for policy 1, policy_version 1903110 (0.0010) [2023-12-27 05:13:43,205][105692] Updated weights for policy 0, policy_version 1898557 (0.0005) [2023-12-27 05:13:43,251][105620] Updated weights for policy 1, policy_version 1903120 (0.0008) [2023-12-27 05:13:43,259][105692] Updated weights for policy 0, policy_version 1898567 (0.0008) [2023-12-27 05:13:43,911][105620] Updated weights for policy 1, policy_version 1903130 (0.0005) [2023-12-27 05:13:43,976][105620] Updated weights for policy 1, policy_version 1903140 (0.0005) [2023-12-27 05:13:44,036][105620] Updated weights for policy 1, policy_version 1903150 (0.0007) [2023-12-27 05:13:44,039][105692] Updated weights for policy 0, policy_version 1898577 (0.0009) [2023-12-27 05:13:44,095][105692] Updated weights for policy 0, policy_version 1898587 (0.0008) [2023-12-27 05:13:44,156][105692] Updated weights for policy 0, policy_version 1898597 (0.0009) [2023-12-27 05:13:44,223][105692] Updated weights for policy 0, policy_version 1898607 (0.0005) [2023-12-27 05:13:44,775][105620] Updated weights for policy 1, policy_version 1903160 (0.0009) [2023-12-27 05:13:44,837][105620] Updated weights for policy 1, policy_version 1903170 (0.0009) [2023-12-27 05:13:44,863][105692] Updated weights for policy 0, policy_version 1898617 (0.0007) [2023-12-27 05:13:44,898][105620] Updated weights for policy 1, policy_version 1903180 (0.0006) [2023-12-27 05:13:44,922][105692] Updated weights for policy 0, policy_version 1898627 (0.0006) [2023-12-27 05:13:44,985][105692] Updated weights for policy 0, policy_version 1898637 (0.0009) [2023-12-27 05:13:45,515][105620] Updated weights for policy 1, policy_version 1903190 (0.0009) [2023-12-27 05:13:45,577][105620] Updated weights for policy 1, policy_version 1903200 (0.0008) [2023-12-27 05:13:45,646][105620] Updated weights for policy 1, policy_version 1903210 (0.0010) [2023-12-27 05:13:45,777][105692] Updated weights for policy 0, policy_version 1898647 (0.0009) [2023-12-27 05:13:45,835][105692] Updated weights for policy 0, policy_version 1898657 (0.0008) [2023-12-27 05:13:45,894][105692] Updated weights for policy 0, policy_version 1898667 (0.0008) [2023-12-27 05:13:46,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 973422592. Throughput: 0: 9736.5, 1: 9893.3. Samples: 973391152. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:13:46,063][104569] Avg episode reward: [(0, '8715.822'), (1, '9345.410')] [2023-12-27 05:13:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001898672_486129664.pth... [2023-12-27 05:13:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001903216_487292928.pth... [2023-12-27 05:13:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001902064_486998016.pth [2023-12-27 05:13:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001897552_485842944.pth [2023-12-27 05:13:46,389][105620] Updated weights for policy 1, policy_version 1903220 (0.0011) [2023-12-27 05:13:46,451][105620] Updated weights for policy 1, policy_version 1903230 (0.0011) [2023-12-27 05:13:46,507][105620] Updated weights for policy 1, policy_version 1903240 (0.0011) [2023-12-27 05:13:46,591][105692] Updated weights for policy 0, policy_version 1898677 (0.0007) [2023-12-27 05:13:46,637][105692] Updated weights for policy 0, policy_version 1898687 (0.0005) [2023-12-27 05:13:46,689][105692] Updated weights for policy 0, policy_version 1898697 (0.0006) [2023-12-27 05:13:47,241][105692] Updated weights for policy 0, policy_version 1898707 (0.0005) [2023-12-27 05:13:47,288][105620] Updated weights for policy 1, policy_version 1903250 (0.0010) [2023-12-27 05:13:47,299][105692] Updated weights for policy 0, policy_version 1898717 (0.0009) [2023-12-27 05:13:47,351][105692] Updated weights for policy 0, policy_version 1898727 (0.0007) [2023-12-27 05:13:47,352][105620] Updated weights for policy 1, policy_version 1903260 (0.0005) [2023-12-27 05:13:47,406][105620] Updated weights for policy 1, policy_version 1903270 (0.0005) [2023-12-27 05:13:47,457][105620] Updated weights for policy 1, policy_version 1903280 (0.0005) [2023-12-27 05:13:48,037][105692] Updated weights for policy 0, policy_version 1898737 (0.0008) [2023-12-27 05:13:48,090][105692] Updated weights for policy 0, policy_version 1898747 (0.0007) [2023-12-27 05:13:48,148][105692] Updated weights for policy 0, policy_version 1898757 (0.0007) [2023-12-27 05:13:48,165][105620] Updated weights for policy 1, policy_version 1903290 (0.0010) [2023-12-27 05:13:48,209][105692] Updated weights for policy 0, policy_version 1898767 (0.0006) [2023-12-27 05:13:48,231][105620] Updated weights for policy 1, policy_version 1903300 (0.0011) [2023-12-27 05:13:48,295][105620] Updated weights for policy 1, policy_version 1903310 (0.0009) [2023-12-27 05:13:48,941][105692] Updated weights for policy 0, policy_version 1898777 (0.0006) [2023-12-27 05:13:48,989][105620] Updated weights for policy 1, policy_version 1903320 (0.0010) [2023-12-27 05:13:48,995][105692] Updated weights for policy 0, policy_version 1898787 (0.0008) [2023-12-27 05:13:49,038][105620] Updated weights for policy 1, policy_version 1903330 (0.0010) [2023-12-27 05:13:49,044][105692] Updated weights for policy 0, policy_version 1898797 (0.0006) [2023-12-27 05:13:49,093][105620] Updated weights for policy 1, policy_version 1903340 (0.0010) [2023-12-27 05:13:49,718][105692] Updated weights for policy 0, policy_version 1898807 (0.0006) [2023-12-27 05:13:49,755][105620] Updated weights for policy 1, policy_version 1903350 (0.0007) [2023-12-27 05:13:49,780][105692] Updated weights for policy 0, policy_version 1898817 (0.0007) [2023-12-27 05:13:49,826][105620] Updated weights for policy 1, policy_version 1903360 (0.0007) [2023-12-27 05:13:49,841][105692] Updated weights for policy 0, policy_version 1898827 (0.0007) [2023-12-27 05:13:49,889][105620] Updated weights for policy 1, policy_version 1903370 (0.0007) [2023-12-27 05:13:50,488][105692] Updated weights for policy 0, policy_version 1898837 (0.0006) [2023-12-27 05:13:50,549][105692] Updated weights for policy 0, policy_version 1898847 (0.0008) [2023-12-27 05:13:50,563][105620] Updated weights for policy 1, policy_version 1903380 (0.0006) [2023-12-27 05:13:50,612][105692] Updated weights for policy 0, policy_version 1898857 (0.0008) [2023-12-27 05:13:50,633][105620] Updated weights for policy 1, policy_version 1903390 (0.0008) [2023-12-27 05:13:50,695][105620] Updated weights for policy 1, policy_version 1903400 (0.0008) [2023-12-27 05:13:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 973520896. Throughput: 0: 9782.8, 1: 9933.2. Samples: 973509468. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:13:51,063][104569] Avg episode reward: [(0, '8624.855'), (1, '9253.320')] [2023-12-27 05:13:51,354][105692] Updated weights for policy 0, policy_version 1898867 (0.0009) [2023-12-27 05:13:51,385][105620] Updated weights for policy 1, policy_version 1903410 (0.0009) [2023-12-27 05:13:51,424][105692] Updated weights for policy 0, policy_version 1898877 (0.0008) [2023-12-27 05:13:51,446][105620] Updated weights for policy 1, policy_version 1903420 (0.0007) [2023-12-27 05:13:51,480][105692] Updated weights for policy 0, policy_version 1898887 (0.0007) [2023-12-27 05:13:51,495][105620] Updated weights for policy 1, policy_version 1903430 (0.0007) [2023-12-27 05:13:51,554][105620] Updated weights for policy 1, policy_version 1903440 (0.0007) [2023-12-27 05:13:52,191][105620] Updated weights for policy 1, policy_version 1903450 (0.0009) [2023-12-27 05:13:52,256][105620] Updated weights for policy 1, policy_version 1903460 (0.0006) [2023-12-27 05:13:52,295][105692] Updated weights for policy 0, policy_version 1898897 (0.0007) [2023-12-27 05:13:52,315][105620] Updated weights for policy 1, policy_version 1903470 (0.0008) [2023-12-27 05:13:52,363][105692] Updated weights for policy 0, policy_version 1898907 (0.0009) [2023-12-27 05:13:52,426][105692] Updated weights for policy 0, policy_version 1898917 (0.0009) [2023-12-27 05:13:52,485][105692] Updated weights for policy 0, policy_version 1898927 (0.0008) [2023-12-27 05:13:53,098][105620] Updated weights for policy 1, policy_version 1903480 (0.0008) [2023-12-27 05:13:53,143][105692] Updated weights for policy 0, policy_version 1898937 (0.0010) [2023-12-27 05:13:53,146][105620] Updated weights for policy 1, policy_version 1903490 (0.0005) [2023-12-27 05:13:53,199][105692] Updated weights for policy 0, policy_version 1898947 (0.0011) [2023-12-27 05:13:53,201][105620] Updated weights for policy 1, policy_version 1903500 (0.0005) [2023-12-27 05:13:53,257][105692] Updated weights for policy 0, policy_version 1898957 (0.0011) [2023-12-27 05:13:53,913][105620] Updated weights for policy 1, policy_version 1903510 (0.0009) [2023-12-27 05:13:53,975][105620] Updated weights for policy 1, policy_version 1903520 (0.0010) [2023-12-27 05:13:54,014][105692] Updated weights for policy 0, policy_version 1898967 (0.0009) [2023-12-27 05:13:54,030][105620] Updated weights for policy 1, policy_version 1903530 (0.0007) [2023-12-27 05:13:54,070][105692] Updated weights for policy 0, policy_version 1898977 (0.0009) [2023-12-27 05:13:54,128][105692] Updated weights for policy 0, policy_version 1898987 (0.0010) [2023-12-27 05:13:54,680][105620] Updated weights for policy 1, policy_version 1903540 (0.0007) [2023-12-27 05:13:54,730][105620] Updated weights for policy 1, policy_version 1903550 (0.0009) [2023-12-27 05:13:54,783][105620] Updated weights for policy 1, policy_version 1903560 (0.0010) [2023-12-27 05:13:54,942][105692] Updated weights for policy 0, policy_version 1898997 (0.0009) [2023-12-27 05:13:54,999][105692] Updated weights for policy 0, policy_version 1899007 (0.0008) [2023-12-27 05:13:55,055][105692] Updated weights for policy 0, policy_version 1899017 (0.0008) [2023-12-27 05:13:55,545][105620] Updated weights for policy 1, policy_version 1903570 (0.0011) [2023-12-27 05:13:55,606][105620] Updated weights for policy 1, policy_version 1903580 (0.0010) [2023-12-27 05:13:55,662][105620] Updated weights for policy 1, policy_version 1903590 (0.0010) [2023-12-27 05:13:55,713][105620] Updated weights for policy 1, policy_version 1903600 (0.0010) [2023-12-27 05:13:55,811][105692] Updated weights for policy 0, policy_version 1899027 (0.0008) [2023-12-27 05:13:55,866][105692] Updated weights for policy 0, policy_version 1899037 (0.0008) [2023-12-27 05:13:55,921][105692] Updated weights for policy 0, policy_version 1899047 (0.0008) [2023-12-27 05:13:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 973619200. Throughput: 0: 9737.8, 1: 9905.2. Samples: 973624548. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:13:56,063][104569] Avg episode reward: [(0, '8351.838'), (1, '9253.341')] [2023-12-27 05:13:56,469][105620] Updated weights for policy 1, policy_version 1903610 (0.0010) [2023-12-27 05:13:56,520][105620] Updated weights for policy 1, policy_version 1903620 (0.0010) [2023-12-27 05:13:56,570][105620] Updated weights for policy 1, policy_version 1903630 (0.0010) [2023-12-27 05:13:56,674][105692] Updated weights for policy 0, policy_version 1899057 (0.0008) [2023-12-27 05:13:56,735][105692] Updated weights for policy 0, policy_version 1899067 (0.0008) [2023-12-27 05:13:56,796][105692] Updated weights for policy 0, policy_version 1899077 (0.0008) [2023-12-27 05:13:56,855][105692] Updated weights for policy 0, policy_version 1899087 (0.0009) [2023-12-27 05:13:57,208][105620] Updated weights for policy 1, policy_version 1903640 (0.0006) [2023-12-27 05:13:57,260][105620] Updated weights for policy 1, policy_version 1903650 (0.0007) [2023-12-27 05:13:57,318][105620] Updated weights for policy 1, policy_version 1903660 (0.0009) [2023-12-27 05:13:57,616][105692] Updated weights for policy 0, policy_version 1899097 (0.0009) [2023-12-27 05:13:57,675][105692] Updated weights for policy 0, policy_version 1899107 (0.0008) [2023-12-27 05:13:57,739][105692] Updated weights for policy 0, policy_version 1899117 (0.0009) [2023-12-27 05:13:57,946][105620] Updated weights for policy 1, policy_version 1903670 (0.0008) [2023-12-27 05:13:58,007][105620] Updated weights for policy 1, policy_version 1903680 (0.0008) [2023-12-27 05:13:58,070][105620] Updated weights for policy 1, policy_version 1903690 (0.0005) [2023-12-27 05:13:58,560][105692] Updated weights for policy 0, policy_version 1899127 (0.0008) [2023-12-27 05:13:58,628][105692] Updated weights for policy 0, policy_version 1899137 (0.0008) [2023-12-27 05:13:58,695][105692] Updated weights for policy 0, policy_version 1899147 (0.0009) [2023-12-27 05:13:58,779][105620] Updated weights for policy 1, policy_version 1903700 (0.0006) [2023-12-27 05:13:58,853][105620] Updated weights for policy 1, policy_version 1903710 (0.0009) [2023-12-27 05:13:58,916][105620] Updated weights for policy 1, policy_version 1903720 (0.0009) [2023-12-27 05:13:59,478][105692] Updated weights for policy 0, policy_version 1899157 (0.0006) [2023-12-27 05:13:59,536][105692] Updated weights for policy 0, policy_version 1899167 (0.0005) [2023-12-27 05:13:59,591][105692] Updated weights for policy 0, policy_version 1899177 (0.0010) [2023-12-27 05:13:59,750][105620] Updated weights for policy 1, policy_version 1903730 (0.0008) [2023-12-27 05:13:59,803][105620] Updated weights for policy 1, policy_version 1903740 (0.0008) [2023-12-27 05:13:59,867][105620] Updated weights for policy 1, policy_version 1903750 (0.0008) [2023-12-27 05:13:59,918][105620] Updated weights for policy 1, policy_version 1903760 (0.0008) [2023-12-27 05:14:00,344][105692] Updated weights for policy 0, policy_version 1899187 (0.0010) [2023-12-27 05:14:00,398][105692] Updated weights for policy 0, policy_version 1899197 (0.0009) [2023-12-27 05:14:00,456][105692] Updated weights for policy 0, policy_version 1899207 (0.0009) [2023-12-27 05:14:00,708][105620] Updated weights for policy 1, policy_version 1903770 (0.0009) [2023-12-27 05:14:00,756][105620] Updated weights for policy 1, policy_version 1903780 (0.0009) [2023-12-27 05:14:00,810][105620] Updated weights for policy 1, policy_version 1903790 (0.0008) [2023-12-27 05:14:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 973709312. Throughput: 0: 9699.7, 1: 9916.0. Samples: 973681984. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:14:01,063][104569] Avg episode reward: [(0, '8175.435'), (1, '9345.465')] [2023-12-27 05:14:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001899216_486268928.pth... [2023-12-27 05:14:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001903792_487440384.pth... [2023-12-27 05:14:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001902640_487145472.pth [2023-12-27 05:14:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001898096_485982208.pth [2023-12-27 05:14:01,208][105692] Updated weights for policy 0, policy_version 1899217 (0.0009) [2023-12-27 05:14:01,268][105692] Updated weights for policy 0, policy_version 1899227 (0.0009) [2023-12-27 05:14:01,323][105692] Updated weights for policy 0, policy_version 1899237 (0.0009) [2023-12-27 05:14:01,386][105692] Updated weights for policy 0, policy_version 1899247 (0.0009) [2023-12-27 05:14:01,527][105620] Updated weights for policy 1, policy_version 1903800 (0.0007) [2023-12-27 05:14:01,577][105620] Updated weights for policy 1, policy_version 1903810 (0.0009) [2023-12-27 05:14:01,640][105620] Updated weights for policy 1, policy_version 1903820 (0.0009) [2023-12-27 05:14:02,180][105692] Updated weights for policy 0, policy_version 1899257 (0.0009) [2023-12-27 05:14:02,231][105692] Updated weights for policy 0, policy_version 1899267 (0.0009) [2023-12-27 05:14:02,289][105692] Updated weights for policy 0, policy_version 1899277 (0.0008) [2023-12-27 05:14:02,406][105620] Updated weights for policy 1, policy_version 1903830 (0.0006) [2023-12-27 05:14:02,477][105620] Updated weights for policy 1, policy_version 1903840 (0.0005) [2023-12-27 05:14:02,538][105620] Updated weights for policy 1, policy_version 1903850 (0.0005) [2023-12-27 05:14:03,087][105692] Updated weights for policy 0, policy_version 1899287 (0.0008) [2023-12-27 05:14:03,124][105620] Updated weights for policy 1, policy_version 1903860 (0.0005) [2023-12-27 05:14:03,133][105692] Updated weights for policy 0, policy_version 1899297 (0.0009) [2023-12-27 05:14:03,178][105620] Updated weights for policy 1, policy_version 1903870 (0.0005) [2023-12-27 05:14:03,186][105692] Updated weights for policy 0, policy_version 1899307 (0.0008) [2023-12-27 05:14:03,241][105620] Updated weights for policy 1, policy_version 1903880 (0.0005) [2023-12-27 05:14:03,923][105620] Updated weights for policy 1, policy_version 1903890 (0.0006) [2023-12-27 05:14:03,960][105692] Updated weights for policy 0, policy_version 1899317 (0.0008) [2023-12-27 05:14:03,987][105620] Updated weights for policy 1, policy_version 1903900 (0.0008) [2023-12-27 05:14:04,013][105692] Updated weights for policy 0, policy_version 1899327 (0.0008) [2023-12-27 05:14:04,048][105620] Updated weights for policy 1, policy_version 1903910 (0.0007) [2023-12-27 05:14:04,070][105692] Updated weights for policy 0, policy_version 1899337 (0.0005) [2023-12-27 05:14:04,104][105620] Updated weights for policy 1, policy_version 1903920 (0.0008) [2023-12-27 05:14:04,810][105692] Updated weights for policy 0, policy_version 1899347 (0.0007) [2023-12-27 05:14:04,855][105620] Updated weights for policy 1, policy_version 1903930 (0.0009) [2023-12-27 05:14:04,859][105692] Updated weights for policy 0, policy_version 1899357 (0.0005) [2023-12-27 05:14:04,907][105620] Updated weights for policy 1, policy_version 1903940 (0.0008) [2023-12-27 05:14:04,909][105692] Updated weights for policy 0, policy_version 1899367 (0.0006) [2023-12-27 05:14:04,965][105620] Updated weights for policy 1, policy_version 1903950 (0.0007) [2023-12-27 05:14:05,624][105692] Updated weights for policy 0, policy_version 1899377 (0.0006) [2023-12-27 05:14:05,678][105692] Updated weights for policy 0, policy_version 1899387 (0.0009) [2023-12-27 05:14:05,697][105620] Updated weights for policy 1, policy_version 1903960 (0.0007) [2023-12-27 05:14:05,735][105692] Updated weights for policy 0, policy_version 1899397 (0.0006) [2023-12-27 05:14:05,764][105620] Updated weights for policy 1, policy_version 1903970 (0.0007) [2023-12-27 05:14:05,783][105692] Updated weights for policy 0, policy_version 1899407 (0.0007) [2023-12-27 05:14:05,828][105620] Updated weights for policy 1, policy_version 1903980 (0.0009) [2023-12-27 05:14:06,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 973807616. Throughput: 0: 9539.7, 1: 9817.8. Samples: 973793660. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:14:06,064][104569] Avg episode reward: [(0, '8539.307'), (1, '9345.492')] [2023-12-27 05:14:06,446][105692] Updated weights for policy 0, policy_version 1899417 (0.0006) [2023-12-27 05:14:06,501][105692] Updated weights for policy 0, policy_version 1899427 (0.0006) [2023-12-27 05:14:06,558][105692] Updated weights for policy 0, policy_version 1899437 (0.0005) [2023-12-27 05:14:06,641][105620] Updated weights for policy 1, policy_version 1903990 (0.0009) [2023-12-27 05:14:06,704][105620] Updated weights for policy 1, policy_version 1904000 (0.0009) [2023-12-27 05:14:06,766][105620] Updated weights for policy 1, policy_version 1904010 (0.0009) [2023-12-27 05:14:07,180][105692] Updated weights for policy 0, policy_version 1899447 (0.0006) [2023-12-27 05:14:07,239][105692] Updated weights for policy 0, policy_version 1899457 (0.0006) [2023-12-27 05:14:07,292][105692] Updated weights for policy 0, policy_version 1899467 (0.0005) [2023-12-27 05:14:07,589][105620] Updated weights for policy 1, policy_version 1904020 (0.0009) [2023-12-27 05:14:07,636][105620] Updated weights for policy 1, policy_version 1904030 (0.0009) [2023-12-27 05:14:07,684][105620] Updated weights for policy 1, policy_version 1904040 (0.0009) [2023-12-27 05:14:07,922][105692] Updated weights for policy 0, policy_version 1899477 (0.0007) [2023-12-27 05:14:07,977][105692] Updated weights for policy 0, policy_version 1899487 (0.0005) [2023-12-27 05:14:08,037][105692] Updated weights for policy 0, policy_version 1899497 (0.0005) [2023-12-27 05:14:08,510][105620] Updated weights for policy 1, policy_version 1904050 (0.0010) [2023-12-27 05:14:08,569][105620] Updated weights for policy 1, policy_version 1904060 (0.0009) [2023-12-27 05:14:08,623][105620] Updated weights for policy 1, policy_version 1904070 (0.0009) [2023-12-27 05:14:08,682][105620] Updated weights for policy 1, policy_version 1904080 (0.0009) [2023-12-27 05:14:08,708][105692] Updated weights for policy 0, policy_version 1899507 (0.0006) [2023-12-27 05:14:08,758][105692] Updated weights for policy 0, policy_version 1899517 (0.0008) [2023-12-27 05:14:08,813][105692] Updated weights for policy 0, policy_version 1899527 (0.0009) [2023-12-27 05:14:09,452][105620] Updated weights for policy 1, policy_version 1904090 (0.0009) [2023-12-27 05:14:09,517][105620] Updated weights for policy 1, policy_version 1904100 (0.0010) [2023-12-27 05:14:09,572][105620] Updated weights for policy 1, policy_version 1904110 (0.0010) [2023-12-27 05:14:09,598][105692] Updated weights for policy 0, policy_version 1899537 (0.0009) [2023-12-27 05:14:09,654][105692] Updated weights for policy 0, policy_version 1899547 (0.0010) [2023-12-27 05:14:09,719][105692] Updated weights for policy 0, policy_version 1899557 (0.0009) [2023-12-27 05:14:09,785][105692] Updated weights for policy 0, policy_version 1899567 (0.0007) [2023-12-27 05:14:10,320][105620] Updated weights for policy 1, policy_version 1904120 (0.0011) [2023-12-27 05:14:10,379][105620] Updated weights for policy 1, policy_version 1904130 (0.0010) [2023-12-27 05:14:10,449][105620] Updated weights for policy 1, policy_version 1904140 (0.0011) [2023-12-27 05:14:10,529][105692] Updated weights for policy 0, policy_version 1899577 (0.0010) [2023-12-27 05:14:10,580][105692] Updated weights for policy 0, policy_version 1899587 (0.0010) [2023-12-27 05:14:10,629][105692] Updated weights for policy 0, policy_version 1899597 (0.0010) [2023-12-27 05:14:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.2, 300 sec: 19438.7). Total num frames: 973897728. Throughput: 0: 9639.1, 1: 9733.6. Samples: 973908656. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:14:11,063][104569] Avg episode reward: [(0, '8356.480'), (1, '9345.524')] [2023-12-27 05:14:11,146][105620] Updated weights for policy 1, policy_version 1904150 (0.0009) [2023-12-27 05:14:11,208][105620] Updated weights for policy 1, policy_version 1904160 (0.0011) [2023-12-27 05:14:11,273][105620] Updated weights for policy 1, policy_version 1904170 (0.0011) [2023-12-27 05:14:11,371][105692] Updated weights for policy 0, policy_version 1899607 (0.0008) [2023-12-27 05:14:11,437][105692] Updated weights for policy 0, policy_version 1899617 (0.0007) [2023-12-27 05:14:11,490][105692] Updated weights for policy 0, policy_version 1899627 (0.0006) [2023-12-27 05:14:12,107][105620] Updated weights for policy 1, policy_version 1904180 (0.0010) [2023-12-27 05:14:12,165][105620] Updated weights for policy 1, policy_version 1904190 (0.0008) [2023-12-27 05:14:12,220][105620] Updated weights for policy 1, policy_version 1904200 (0.0007) [2023-12-27 05:14:12,236][105692] Updated weights for policy 0, policy_version 1899637 (0.0008) [2023-12-27 05:14:12,289][105692] Updated weights for policy 0, policy_version 1899647 (0.0008) [2023-12-27 05:14:12,356][105692] Updated weights for policy 0, policy_version 1899657 (0.0008) [2023-12-27 05:14:12,849][105620] Updated weights for policy 1, policy_version 1904210 (0.0008) [2023-12-27 05:14:12,900][105620] Updated weights for policy 1, policy_version 1904220 (0.0009) [2023-12-27 05:14:12,967][105620] Updated weights for policy 1, policy_version 1904230 (0.0009) [2023-12-27 05:14:13,030][105620] Updated weights for policy 1, policy_version 1904240 (0.0008) [2023-12-27 05:14:13,166][105692] Updated weights for policy 0, policy_version 1899667 (0.0009) [2023-12-27 05:14:13,222][105692] Updated weights for policy 0, policy_version 1899677 (0.0010) [2023-12-27 05:14:13,284][105692] Updated weights for policy 0, policy_version 1899687 (0.0010) [2023-12-27 05:14:13,674][105620] Updated weights for policy 1, policy_version 1904250 (0.0009) [2023-12-27 05:14:13,719][105620] Updated weights for policy 1, policy_version 1904260 (0.0005) [2023-12-27 05:14:13,775][105620] Updated weights for policy 1, policy_version 1904270 (0.0005) [2023-12-27 05:14:14,092][105692] Updated weights for policy 0, policy_version 1899697 (0.0010) [2023-12-27 05:14:14,149][105692] Updated weights for policy 0, policy_version 1899708 (0.0010) [2023-12-27 05:14:14,206][105692] Updated weights for policy 0, policy_version 1899718 (0.0010) [2023-12-27 05:14:14,272][105692] Updated weights for policy 0, policy_version 1899728 (0.0009) [2023-12-27 05:14:14,388][105620] Updated weights for policy 1, policy_version 1904280 (0.0005) [2023-12-27 05:14:14,446][105620] Updated weights for policy 1, policy_version 1904290 (0.0005) [2023-12-27 05:14:14,502][105620] Updated weights for policy 1, policy_version 1904300 (0.0007) [2023-12-27 05:14:15,105][105692] Updated weights for policy 0, policy_version 1899738 (0.0008) [2023-12-27 05:14:15,138][105620] Updated weights for policy 1, policy_version 1904310 (0.0010) [2023-12-27 05:14:15,166][105692] Updated weights for policy 0, policy_version 1899748 (0.0008) [2023-12-27 05:14:15,198][105620] Updated weights for policy 1, policy_version 1904320 (0.0008) [2023-12-27 05:14:15,229][105692] Updated weights for policy 0, policy_version 1899758 (0.0007) [2023-12-27 05:14:15,257][105620] Updated weights for policy 1, policy_version 1904330 (0.0007) [2023-12-27 05:14:15,910][105692] Updated weights for policy 0, policy_version 1899768 (0.0010) [2023-12-27 05:14:15,969][105692] Updated weights for policy 0, policy_version 1899778 (0.0011) [2023-12-27 05:14:16,021][105692] Updated weights for policy 0, policy_version 1899788 (0.0010) [2023-12-27 05:14:16,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 973996032. Throughput: 0: 9581.0, 1: 9656.3. Samples: 973966236. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:14:16,062][104569] Avg episode reward: [(0, '8359.854'), (1, '9160.959')] [2023-12-27 05:14:16,067][105620] Updated weights for policy 1, policy_version 1904340 (0.0009) [2023-12-27 05:14:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001899792_486416384.pth... [2023-12-27 05:14:16,090][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001898672_486129664.pth [2023-12-27 05:14:16,126][105620] Updated weights for policy 1, policy_version 1904350 (0.0008) [2023-12-27 05:14:16,178][105620] Updated weights for policy 1, policy_version 1904360 (0.0008) [2023-12-27 05:14:16,213][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001904368_487587840.pth... [2023-12-27 05:14:16,216][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001903216_487292928.pth [2023-12-27 05:14:16,693][105692] Updated weights for policy 0, policy_version 1899798 (0.0007) [2023-12-27 05:14:16,747][105692] Updated weights for policy 0, policy_version 1899808 (0.0005) [2023-12-27 05:14:16,793][105692] Updated weights for policy 0, policy_version 1899818 (0.0005) [2023-12-27 05:14:17,013][105620] Updated weights for policy 1, policy_version 1904370 (0.0008) [2023-12-27 05:14:17,076][105620] Updated weights for policy 1, policy_version 1904380 (0.0010) [2023-12-27 05:14:17,128][105620] Updated weights for policy 1, policy_version 1904390 (0.0010) [2023-12-27 05:14:17,186][105620] Updated weights for policy 1, policy_version 1904400 (0.0010) [2023-12-27 05:14:17,353][105692] Updated weights for policy 0, policy_version 1899828 (0.0005) [2023-12-27 05:14:17,408][105692] Updated weights for policy 0, policy_version 1899838 (0.0005) [2023-12-27 05:14:17,463][105692] Updated weights for policy 0, policy_version 1899848 (0.0007) [2023-12-27 05:14:18,017][105692] Updated weights for policy 0, policy_version 1899858 (0.0006) [2023-12-27 05:14:18,079][105692] Updated weights for policy 0, policy_version 1899868 (0.0008) [2023-12-27 05:14:18,137][105692] Updated weights for policy 0, policy_version 1899878 (0.0008) [2023-12-27 05:14:18,141][105620] Updated weights for policy 1, policy_version 1904410 (0.0007) [2023-12-27 05:14:18,194][105620] Updated weights for policy 1, policy_version 1904420 (0.0008) [2023-12-27 05:14:18,194][105692] Updated weights for policy 0, policy_version 1899888 (0.0008) [2023-12-27 05:14:18,252][105620] Updated weights for policy 1, policy_version 1904430 (0.0008) [2023-12-27 05:14:18,944][105692] Updated weights for policy 0, policy_version 1899898 (0.0006) [2023-12-27 05:14:19,008][105692] Updated weights for policy 0, policy_version 1899908 (0.0006) [2023-12-27 05:14:19,018][105620] Updated weights for policy 1, policy_version 1904440 (0.0007) [2023-12-27 05:14:19,066][105692] Updated weights for policy 0, policy_version 1899918 (0.0010) [2023-12-27 05:14:19,072][105620] Updated weights for policy 1, policy_version 1904450 (0.0006) [2023-12-27 05:14:19,131][105620] Updated weights for policy 1, policy_version 1904460 (0.0008) [2023-12-27 05:14:19,797][105692] Updated weights for policy 0, policy_version 1899928 (0.0011) [2023-12-27 05:14:19,861][105692] Updated weights for policy 0, policy_version 1899938 (0.0011) [2023-12-27 05:14:19,907][105620] Updated weights for policy 1, policy_version 1904470 (0.0006) [2023-12-27 05:14:19,925][105692] Updated weights for policy 0, policy_version 1899948 (0.0009) [2023-12-27 05:14:19,974][105620] Updated weights for policy 1, policy_version 1904480 (0.0009) [2023-12-27 05:14:20,041][105620] Updated weights for policy 1, policy_version 1904490 (0.0008) [2023-12-27 05:14:20,637][105692] Updated weights for policy 0, policy_version 1899958 (0.0008) [2023-12-27 05:14:20,697][105692] Updated weights for policy 0, policy_version 1899968 (0.0011) [2023-12-27 05:14:20,761][105692] Updated weights for policy 0, policy_version 1899978 (0.0011) [2023-12-27 05:14:20,827][105620] Updated weights for policy 1, policy_version 1904500 (0.0008) [2023-12-27 05:14:20,888][105620] Updated weights for policy 1, policy_version 1904510 (0.0008) [2023-12-27 05:14:20,952][105620] Updated weights for policy 1, policy_version 1904520 (0.0009) [2023-12-27 05:14:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 974094336. Throughput: 0: 9629.9, 1: 9625.1. Samples: 974081576. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:14:21,062][104569] Avg episode reward: [(0, '8176.639'), (1, '8978.660')] [2023-12-27 05:14:21,510][105692] Updated weights for policy 0, policy_version 1899988 (0.0011) [2023-12-27 05:14:21,574][105692] Updated weights for policy 0, policy_version 1899998 (0.0011) [2023-12-27 05:14:21,638][105692] Updated weights for policy 0, policy_version 1900008 (0.0011) [2023-12-27 05:14:21,762][105620] Updated weights for policy 1, policy_version 1904530 (0.0008) [2023-12-27 05:14:21,826][105620] Updated weights for policy 1, policy_version 1904540 (0.0008) [2023-12-27 05:14:21,885][105620] Updated weights for policy 1, policy_version 1904550 (0.0008) [2023-12-27 05:14:21,940][105620] Updated weights for policy 1, policy_version 1904560 (0.0008) [2023-12-27 05:14:22,409][105692] Updated weights for policy 0, policy_version 1900018 (0.0009) [2023-12-27 05:14:22,477][105692] Updated weights for policy 0, policy_version 1900028 (0.0007) [2023-12-27 05:14:22,545][105692] Updated weights for policy 0, policy_version 1900038 (0.0007) [2023-12-27 05:14:22,612][105692] Updated weights for policy 0, policy_version 1900048 (0.0009) [2023-12-27 05:14:22,738][105620] Updated weights for policy 1, policy_version 1904570 (0.0009) [2023-12-27 05:14:22,802][105620] Updated weights for policy 1, policy_version 1904580 (0.0009) [2023-12-27 05:14:22,866][105620] Updated weights for policy 1, policy_version 1904590 (0.0010) [2023-12-27 05:14:23,184][105692] Updated weights for policy 0, policy_version 1900058 (0.0005) [2023-12-27 05:14:23,234][105692] Updated weights for policy 0, policy_version 1900068 (0.0005) [2023-12-27 05:14:23,282][105692] Updated weights for policy 0, policy_version 1900078 (0.0005) [2023-12-27 05:14:23,679][105620] Updated weights for policy 1, policy_version 1904600 (0.0010) [2023-12-27 05:14:23,737][105620] Updated weights for policy 1, policy_version 1904610 (0.0010) [2023-12-27 05:14:23,796][105620] Updated weights for policy 1, policy_version 1904620 (0.0010) [2023-12-27 05:14:23,860][105692] Updated weights for policy 0, policy_version 1900088 (0.0005) [2023-12-27 05:14:23,916][105692] Updated weights for policy 0, policy_version 1900098 (0.0005) [2023-12-27 05:14:23,975][105692] Updated weights for policy 0, policy_version 1900108 (0.0007) [2023-12-27 05:14:24,603][105620] Updated weights for policy 1, policy_version 1904630 (0.0008) [2023-12-27 05:14:24,656][105620] Updated weights for policy 1, policy_version 1904640 (0.0008) [2023-12-27 05:14:24,675][105692] Updated weights for policy 0, policy_version 1900118 (0.0007) [2023-12-27 05:14:24,714][105620] Updated weights for policy 1, policy_version 1904650 (0.0007) [2023-12-27 05:14:24,725][105692] Updated weights for policy 0, policy_version 1900128 (0.0006) [2023-12-27 05:14:24,783][105692] Updated weights for policy 0, policy_version 1900138 (0.0009) [2023-12-27 05:14:25,434][105620] Updated weights for policy 1, policy_version 1904660 (0.0007) [2023-12-27 05:14:25,495][105620] Updated weights for policy 1, policy_version 1904670 (0.0009) [2023-12-27 05:14:25,547][105620] Updated weights for policy 1, policy_version 1904680 (0.0008) [2023-12-27 05:14:25,558][105692] Updated weights for policy 0, policy_version 1900148 (0.0009) [2023-12-27 05:14:25,615][105692] Updated weights for policy 0, policy_version 1900158 (0.0007) [2023-12-27 05:14:25,682][105692] Updated weights for policy 0, policy_version 1900168 (0.0009) [2023-12-27 05:14:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.3, 300 sec: 19383.1). Total num frames: 974184448. Throughput: 0: 9629.6, 1: 9506.1. Samples: 974194004. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:14:26,062][104569] Avg episode reward: [(0, '8084.192'), (1, '9072.711')] [2023-12-27 05:14:26,278][105620] Updated weights for policy 1, policy_version 1904690 (0.0008) [2023-12-27 05:14:26,336][105620] Updated weights for policy 1, policy_version 1904701 (0.0010) [2023-12-27 05:14:26,378][105692] Updated weights for policy 0, policy_version 1900178 (0.0009) [2023-12-27 05:14:26,394][105620] Updated weights for policy 1, policy_version 1904711 (0.0010) [2023-12-27 05:14:26,430][105692] Updated weights for policy 0, policy_version 1900188 (0.0006) [2023-12-27 05:14:26,489][105692] Updated weights for policy 0, policy_version 1900198 (0.0009) [2023-12-27 05:14:26,540][105692] Updated weights for policy 0, policy_version 1900208 (0.0009) [2023-12-27 05:14:27,174][105620] Updated weights for policy 1, policy_version 1904721 (0.0008) [2023-12-27 05:14:27,232][105620] Updated weights for policy 1, policy_version 1904731 (0.0009) [2023-12-27 05:14:27,296][105620] Updated weights for policy 1, policy_version 1904741 (0.0007) [2023-12-27 05:14:27,310][105692] Updated weights for policy 0, policy_version 1900218 (0.0007) [2023-12-27 05:14:27,358][105692] Updated weights for policy 0, policy_version 1900228 (0.0008) [2023-12-27 05:14:27,360][105620] Updated weights for policy 1, policy_version 1904751 (0.0006) [2023-12-27 05:14:27,410][105692] Updated weights for policy 0, policy_version 1900238 (0.0008) [2023-12-27 05:14:28,088][105620] Updated weights for policy 1, policy_version 1904761 (0.0008) [2023-12-27 05:14:28,132][105692] Updated weights for policy 0, policy_version 1900248 (0.0007) [2023-12-27 05:14:28,150][105620] Updated weights for policy 1, policy_version 1904771 (0.0009) [2023-12-27 05:14:28,187][105692] Updated weights for policy 0, policy_version 1900258 (0.0005) [2023-12-27 05:14:28,201][105620] Updated weights for policy 1, policy_version 1904781 (0.0007) [2023-12-27 05:14:28,241][105692] Updated weights for policy 0, policy_version 1900268 (0.0009) [2023-12-27 05:14:28,979][105692] Updated weights for policy 0, policy_version 1900278 (0.0008) [2023-12-27 05:14:28,993][105620] Updated weights for policy 1, policy_version 1904791 (0.0008) [2023-12-27 05:14:29,031][105692] Updated weights for policy 0, policy_version 1900288 (0.0006) [2023-12-27 05:14:29,053][105620] Updated weights for policy 1, policy_version 1904801 (0.0008) [2023-12-27 05:14:29,089][105692] Updated weights for policy 0, policy_version 1900298 (0.0007) [2023-12-27 05:14:29,104][105620] Updated weights for policy 1, policy_version 1904811 (0.0008) [2023-12-27 05:14:29,852][105692] Updated weights for policy 0, policy_version 1900308 (0.0008) [2023-12-27 05:14:29,869][105620] Updated weights for policy 1, policy_version 1904821 (0.0008) [2023-12-27 05:14:29,911][105692] Updated weights for policy 0, policy_version 1900318 (0.0007) [2023-12-27 05:14:29,931][105620] Updated weights for policy 1, policy_version 1904831 (0.0007) [2023-12-27 05:14:29,972][105692] Updated weights for policy 0, policy_version 1900328 (0.0007) [2023-12-27 05:14:29,993][105620] Updated weights for policy 1, policy_version 1904841 (0.0006) [2023-12-27 05:14:30,725][105620] Updated weights for policy 1, policy_version 1904851 (0.0008) [2023-12-27 05:14:30,727][105692] Updated weights for policy 0, policy_version 1900338 (0.0009) [2023-12-27 05:14:30,780][105692] Updated weights for policy 0, policy_version 1900348 (0.0005) [2023-12-27 05:14:30,785][105620] Updated weights for policy 1, policy_version 1904861 (0.0008) [2023-12-27 05:14:30,839][105692] Updated weights for policy 0, policy_version 1900358 (0.0006) [2023-12-27 05:14:30,843][105620] Updated weights for policy 1, policy_version 1904871 (0.0009) [2023-12-27 05:14:30,895][105692] Updated weights for policy 0, policy_version 1900368 (0.0007) [2023-12-27 05:14:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19355.3). Total num frames: 974282752. Throughput: 0: 9645.8, 1: 9450.7. Samples: 974250492. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:14:31,063][104569] Avg episode reward: [(0, '8446.134'), (1, '9254.925')] [2023-12-27 05:14:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001900368_486563840.pth... [2023-12-27 05:14:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001904880_487718912.pth... [2023-12-27 05:14:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001903792_487440384.pth [2023-12-27 05:14:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001899216_486268928.pth [2023-12-27 05:14:31,567][105692] Updated weights for policy 0, policy_version 1900378 (0.0009) [2023-12-27 05:14:31,625][105692] Updated weights for policy 0, policy_version 1900388 (0.0009) [2023-12-27 05:14:31,647][105620] Updated weights for policy 1, policy_version 1904881 (0.0007) [2023-12-27 05:14:31,683][105692] Updated weights for policy 0, policy_version 1900398 (0.0008) [2023-12-27 05:14:31,709][105620] Updated weights for policy 1, policy_version 1904891 (0.0009) [2023-12-27 05:14:31,768][105620] Updated weights for policy 1, policy_version 1904901 (0.0008) [2023-12-27 05:14:31,817][105620] Updated weights for policy 1, policy_version 1904911 (0.0008) [2023-12-27 05:14:32,430][105692] Updated weights for policy 0, policy_version 1900408 (0.0007) [2023-12-27 05:14:32,489][105692] Updated weights for policy 0, policy_version 1900418 (0.0009) [2023-12-27 05:14:32,547][105692] Updated weights for policy 0, policy_version 1900428 (0.0009) [2023-12-27 05:14:32,625][105620] Updated weights for policy 1, policy_version 1904921 (0.0007) [2023-12-27 05:14:32,686][105620] Updated weights for policy 1, policy_version 1904931 (0.0009) [2023-12-27 05:14:32,733][105620] Updated weights for policy 1, policy_version 1904941 (0.0008) [2023-12-27 05:14:33,263][105692] Updated weights for policy 0, policy_version 1900438 (0.0009) [2023-12-27 05:14:33,323][105692] Updated weights for policy 0, policy_version 1900448 (0.0008) [2023-12-27 05:14:33,376][105692] Updated weights for policy 0, policy_version 1900458 (0.0008) [2023-12-27 05:14:33,535][105620] Updated weights for policy 1, policy_version 1904951 (0.0010) [2023-12-27 05:14:33,589][105620] Updated weights for policy 1, policy_version 1904961 (0.0010) [2023-12-27 05:14:33,647][105620] Updated weights for policy 1, policy_version 1904971 (0.0010) [2023-12-27 05:14:34,128][105692] Updated weights for policy 0, policy_version 1900468 (0.0007) [2023-12-27 05:14:34,201][105692] Updated weights for policy 0, policy_version 1900478 (0.0008) [2023-12-27 05:14:34,269][105692] Updated weights for policy 0, policy_version 1900488 (0.0007) [2023-12-27 05:14:34,385][105620] Updated weights for policy 1, policy_version 1904981 (0.0008) [2023-12-27 05:14:34,440][105620] Updated weights for policy 1, policy_version 1904991 (0.0005) [2023-12-27 05:14:34,494][105620] Updated weights for policy 1, policy_version 1905001 (0.0005) [2023-12-27 05:14:35,058][105692] Updated weights for policy 0, policy_version 1900498 (0.0009) [2023-12-27 05:14:35,113][105692] Updated weights for policy 0, policy_version 1900508 (0.0009) [2023-12-27 05:14:35,149][105620] Updated weights for policy 1, policy_version 1905011 (0.0007) [2023-12-27 05:14:35,164][105692] Updated weights for policy 0, policy_version 1900518 (0.0007) [2023-12-27 05:14:35,199][105620] Updated weights for policy 1, policy_version 1905021 (0.0006) [2023-12-27 05:14:35,210][105692] Updated weights for policy 0, policy_version 1900528 (0.0006) [2023-12-27 05:14:35,253][105620] Updated weights for policy 1, policy_version 1905031 (0.0008) [2023-12-27 05:14:35,793][105692] Updated weights for policy 0, policy_version 1900538 (0.0005) [2023-12-27 05:14:35,839][105692] Updated weights for policy 0, policy_version 1900548 (0.0005) [2023-12-27 05:14:35,885][105692] Updated weights for policy 0, policy_version 1900558 (0.0005) [2023-12-27 05:14:36,050][105620] Updated weights for policy 1, policy_version 1905041 (0.0010) [2023-12-27 05:14:36,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19114.7, 300 sec: 19355.3). Total num frames: 974372864. Throughput: 0: 9578.6, 1: 9388.0. Samples: 974362968. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:14:36,063][104569] Avg episode reward: [(0, '8180.027'), (1, '9253.030')] [2023-12-27 05:14:36,111][105620] Updated weights for policy 1, policy_version 1905051 (0.0010) [2023-12-27 05:14:36,173][105620] Updated weights for policy 1, policy_version 1905061 (0.0010) [2023-12-27 05:14:36,225][105620] Updated weights for policy 1, policy_version 1905071 (0.0010) [2023-12-27 05:14:36,586][105692] Updated weights for policy 0, policy_version 1900568 (0.0008) [2023-12-27 05:14:36,640][105692] Updated weights for policy 0, policy_version 1900578 (0.0009) [2023-12-27 05:14:36,708][105692] Updated weights for policy 0, policy_version 1900588 (0.0009) [2023-12-27 05:14:36,986][105620] Updated weights for policy 1, policy_version 1905081 (0.0008) [2023-12-27 05:14:37,053][105620] Updated weights for policy 1, policy_version 1905091 (0.0009) [2023-12-27 05:14:37,117][105620] Updated weights for policy 1, policy_version 1905101 (0.0009) [2023-12-27 05:14:37,465][105692] Updated weights for policy 0, policy_version 1900598 (0.0007) [2023-12-27 05:14:37,525][105692] Updated weights for policy 0, policy_version 1900608 (0.0006) [2023-12-27 05:14:37,586][105692] Updated weights for policy 0, policy_version 1900618 (0.0006) [2023-12-27 05:14:37,867][105620] Updated weights for policy 1, policy_version 1905111 (0.0007) [2023-12-27 05:14:37,915][105620] Updated weights for policy 1, policy_version 1905121 (0.0006) [2023-12-27 05:14:37,978][105620] Updated weights for policy 1, policy_version 1905131 (0.0006) [2023-12-27 05:14:38,214][105692] Updated weights for policy 0, policy_version 1900628 (0.0006) [2023-12-27 05:14:38,269][105692] Updated weights for policy 0, policy_version 1900640 (0.0011) [2023-12-27 05:14:38,326][105692] Updated weights for policy 0, policy_version 1900650 (0.0006) [2023-12-27 05:14:38,623][105620] Updated weights for policy 1, policy_version 1905141 (0.0011) [2023-12-27 05:14:38,689][105620] Updated weights for policy 1, policy_version 1905151 (0.0011) [2023-12-27 05:14:38,747][105620] Updated weights for policy 1, policy_version 1905161 (0.0010) [2023-12-27 05:14:38,967][105692] Updated weights for policy 0, policy_version 1900660 (0.0007) [2023-12-27 05:14:39,036][105692] Updated weights for policy 0, policy_version 1900670 (0.0007) [2023-12-27 05:14:39,095][105692] Updated weights for policy 0, policy_version 1900681 (0.0010) [2023-12-27 05:14:39,396][105620] Updated weights for policy 1, policy_version 1905171 (0.0010) [2023-12-27 05:14:39,455][105620] Updated weights for policy 1, policy_version 1905181 (0.0006) [2023-12-27 05:14:39,508][105620] Updated weights for policy 1, policy_version 1905191 (0.0005) [2023-12-27 05:14:39,795][105692] Updated weights for policy 0, policy_version 1900691 (0.0009) [2023-12-27 05:14:39,862][105692] Updated weights for policy 0, policy_version 1900701 (0.0009) [2023-12-27 05:14:39,921][105692] Updated weights for policy 0, policy_version 1900711 (0.0008) [2023-12-27 05:14:40,167][105620] Updated weights for policy 1, policy_version 1905201 (0.0005) [2023-12-27 05:14:40,230][105620] Updated weights for policy 1, policy_version 1905211 (0.0011) [2023-12-27 05:14:40,289][105620] Updated weights for policy 1, policy_version 1905221 (0.0005) [2023-12-27 05:14:40,353][105620] Updated weights for policy 1, policy_version 1905231 (0.0009) [2023-12-27 05:14:40,625][105692] Updated weights for policy 0, policy_version 1900721 (0.0008) [2023-12-27 05:14:40,673][105692] Updated weights for policy 0, policy_version 1900731 (0.0010) [2023-12-27 05:14:40,732][105692] Updated weights for policy 0, policy_version 1900741 (0.0010) [2023-12-27 05:14:40,801][105692] Updated weights for policy 0, policy_version 1900751 (0.0009) [2023-12-27 05:14:40,989][105620] Updated weights for policy 1, policy_version 1905241 (0.0009) [2023-12-27 05:14:41,054][105620] Updated weights for policy 1, policy_version 1905251 (0.0010) [2023-12-27 05:14:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19114.7, 300 sec: 19355.3). Total num frames: 974471168. Throughput: 0: 9686.5, 1: 9410.8. Samples: 974483924. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:14:41,062][104569] Avg episode reward: [(0, '7909.663'), (1, '9160.770')] [2023-12-27 05:14:41,105][105620] Updated weights for policy 1, policy_version 1905261 (0.0006) [2023-12-27 05:14:41,646][105692] Updated weights for policy 0, policy_version 1900761 (0.0010) [2023-12-27 05:14:41,712][105692] Updated weights for policy 0, policy_version 1900771 (0.0008) [2023-12-27 05:14:41,715][105620] Updated weights for policy 1, policy_version 1905271 (0.0010) [2023-12-27 05:14:41,780][105692] Updated weights for policy 0, policy_version 1900781 (0.0008) [2023-12-27 05:14:41,781][105620] Updated weights for policy 1, policy_version 1905281 (0.0010) [2023-12-27 05:14:41,845][105620] Updated weights for policy 1, policy_version 1905291 (0.0008) [2023-12-27 05:14:42,497][105692] Updated weights for policy 0, policy_version 1900791 (0.0010) [2023-12-27 05:14:42,542][105692] Updated weights for policy 0, policy_version 1900801 (0.0010) [2023-12-27 05:14:42,587][105620] Updated weights for policy 1, policy_version 1905301 (0.0007) [2023-12-27 05:14:42,598][105692] Updated weights for policy 0, policy_version 1900811 (0.0010) [2023-12-27 05:14:42,646][105620] Updated weights for policy 1, policy_version 1905311 (0.0006) [2023-12-27 05:14:42,708][105620] Updated weights for policy 1, policy_version 1905321 (0.0005) [2023-12-27 05:14:43,300][105692] Updated weights for policy 0, policy_version 1900821 (0.0008) [2023-12-27 05:14:43,304][105620] Updated weights for policy 1, policy_version 1905331 (0.0005) [2023-12-27 05:14:43,358][105692] Updated weights for policy 0, policy_version 1900831 (0.0005) [2023-12-27 05:14:43,366][105620] Updated weights for policy 1, policy_version 1905341 (0.0006) [2023-12-27 05:14:43,411][105692] Updated weights for policy 0, policy_version 1900841 (0.0005) [2023-12-27 05:14:43,434][105620] Updated weights for policy 1, policy_version 1905351 (0.0005) [2023-12-27 05:14:44,040][105692] Updated weights for policy 0, policy_version 1900851 (0.0007) [2023-12-27 05:14:44,058][105620] Updated weights for policy 1, policy_version 1905361 (0.0005) [2023-12-27 05:14:44,090][105692] Updated weights for policy 0, policy_version 1900861 (0.0009) [2023-12-27 05:14:44,110][105620] Updated weights for policy 1, policy_version 1905371 (0.0006) [2023-12-27 05:14:44,148][105692] Updated weights for policy 0, policy_version 1900871 (0.0008) [2023-12-27 05:14:44,165][105620] Updated weights for policy 1, policy_version 1905381 (0.0005) [2023-12-27 05:14:44,214][105620] Updated weights for policy 1, policy_version 1905391 (0.0006) [2023-12-27 05:14:44,913][105692] Updated weights for policy 0, policy_version 1900881 (0.0010) [2023-12-27 05:14:44,926][105620] Updated weights for policy 1, policy_version 1905401 (0.0008) [2023-12-27 05:14:44,970][105692] Updated weights for policy 0, policy_version 1900891 (0.0006) [2023-12-27 05:14:44,993][105620] Updated weights for policy 1, policy_version 1905411 (0.0006) [2023-12-27 05:14:45,031][105692] Updated weights for policy 0, policy_version 1900901 (0.0006) [2023-12-27 05:14:45,060][105620] Updated weights for policy 1, policy_version 1905421 (0.0006) [2023-12-27 05:14:45,094][105692] Updated weights for policy 0, policy_version 1900911 (0.0008) [2023-12-27 05:14:45,655][105620] Updated weights for policy 1, policy_version 1905431 (0.0009) [2023-12-27 05:14:45,710][105620] Updated weights for policy 1, policy_version 1905441 (0.0010) [2023-12-27 05:14:45,733][105692] Updated weights for policy 0, policy_version 1900921 (0.0006) [2023-12-27 05:14:45,756][105620] Updated weights for policy 1, policy_version 1905451 (0.0010) [2023-12-27 05:14:45,790][105692] Updated weights for policy 0, policy_version 1900931 (0.0006) [2023-12-27 05:14:45,849][105692] Updated weights for policy 0, policy_version 1900941 (0.0008) [2023-12-27 05:14:46,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19251.1, 300 sec: 19383.1). Total num frames: 974577664. Throughput: 0: 9714.4, 1: 9431.3. Samples: 974543540. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:14:46,063][104569] Avg episode reward: [(0, '8444.994'), (1, '9252.988')] [2023-12-27 05:14:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001900944_486711296.pth... [2023-12-27 05:14:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001905456_487866368.pth... [2023-12-27 05:14:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001899792_486416384.pth [2023-12-27 05:14:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001904368_487587840.pth [2023-12-27 05:14:46,494][105620] Updated weights for policy 1, policy_version 1905461 (0.0009) [2023-12-27 05:14:46,559][105620] Updated weights for policy 1, policy_version 1905471 (0.0009) [2023-12-27 05:14:46,611][105620] Updated weights for policy 1, policy_version 1905481 (0.0008) [2023-12-27 05:14:46,621][105692] Updated weights for policy 0, policy_version 1900951 (0.0008) [2023-12-27 05:14:46,687][105692] Updated weights for policy 0, policy_version 1900961 (0.0008) [2023-12-27 05:14:46,749][105692] Updated weights for policy 0, policy_version 1900971 (0.0009) [2023-12-27 05:14:47,385][105620] Updated weights for policy 1, policy_version 1905491 (0.0007) [2023-12-27 05:14:47,387][105692] Updated weights for policy 0, policy_version 1900981 (0.0008) [2023-12-27 05:14:47,440][105692] Updated weights for policy 0, policy_version 1900991 (0.0005) [2023-12-27 05:14:47,442][105620] Updated weights for policy 1, policy_version 1905501 (0.0007) [2023-12-27 05:14:47,484][105692] Updated weights for policy 0, policy_version 1901001 (0.0008) [2023-12-27 05:14:47,506][105620] Updated weights for policy 1, policy_version 1905511 (0.0008) [2023-12-27 05:14:48,081][105692] Updated weights for policy 0, policy_version 1901011 (0.0006) [2023-12-27 05:14:48,126][105692] Updated weights for policy 0, policy_version 1901021 (0.0007) [2023-12-27 05:14:48,182][105692] Updated weights for policy 0, policy_version 1901031 (0.0010) [2023-12-27 05:14:48,336][105620] Updated weights for policy 1, policy_version 1905521 (0.0009) [2023-12-27 05:14:48,401][105620] Updated weights for policy 1, policy_version 1905531 (0.0008) [2023-12-27 05:14:48,457][105620] Updated weights for policy 1, policy_version 1905541 (0.0008) [2023-12-27 05:14:48,514][105620] Updated weights for policy 1, policy_version 1905551 (0.0009) [2023-12-27 05:14:48,814][105692] Updated weights for policy 0, policy_version 1901041 (0.0010) [2023-12-27 05:14:48,880][105692] Updated weights for policy 0, policy_version 1901051 (0.0009) [2023-12-27 05:14:48,935][105692] Updated weights for policy 0, policy_version 1901061 (0.0010) [2023-12-27 05:14:49,004][105692] Updated weights for policy 0, policy_version 1901071 (0.0010) [2023-12-27 05:14:49,164][105620] Updated weights for policy 1, policy_version 1905561 (0.0008) [2023-12-27 05:14:49,229][105620] Updated weights for policy 1, policy_version 1905571 (0.0008) [2023-12-27 05:14:49,288][105620] Updated weights for policy 1, policy_version 1905581 (0.0008) [2023-12-27 05:14:49,723][105692] Updated weights for policy 0, policy_version 1901081 (0.0010) [2023-12-27 05:14:49,778][105692] Updated weights for policy 0, policy_version 1901091 (0.0011) [2023-12-27 05:14:49,844][105692] Updated weights for policy 0, policy_version 1901101 (0.0009) [2023-12-27 05:14:50,020][105620] Updated weights for policy 1, policy_version 1905591 (0.0008) [2023-12-27 05:14:50,076][105620] Updated weights for policy 1, policy_version 1905601 (0.0009) [2023-12-27 05:14:50,143][105620] Updated weights for policy 1, policy_version 1905611 (0.0008) [2023-12-27 05:14:50,612][105692] Updated weights for policy 0, policy_version 1901111 (0.0010) [2023-12-27 05:14:50,670][105692] Updated weights for policy 0, policy_version 1901121 (0.0011) [2023-12-27 05:14:50,732][105692] Updated weights for policy 0, policy_version 1901131 (0.0010) [2023-12-27 05:14:50,888][105620] Updated weights for policy 1, policy_version 1905621 (0.0010) [2023-12-27 05:14:50,947][105620] Updated weights for policy 1, policy_version 1905631 (0.0011) [2023-12-27 05:14:51,005][105620] Updated weights for policy 1, policy_version 1905641 (0.0009) [2023-12-27 05:14:51,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 974675968. Throughput: 0: 9859.0, 1: 9459.0. Samples: 974662972. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:14:51,062][104569] Avg episode reward: [(0, '8261.250'), (1, '9068.396')] [2023-12-27 05:14:51,508][105692] Updated weights for policy 0, policy_version 1901141 (0.0010) [2023-12-27 05:14:51,570][105692] Updated weights for policy 0, policy_version 1901151 (0.0011) [2023-12-27 05:14:51,629][105692] Updated weights for policy 0, policy_version 1901161 (0.0010) [2023-12-27 05:14:51,799][105620] Updated weights for policy 1, policy_version 1905651 (0.0009) [2023-12-27 05:14:51,858][105620] Updated weights for policy 1, policy_version 1905661 (0.0010) [2023-12-27 05:14:51,917][105620] Updated weights for policy 1, policy_version 1905671 (0.0011) [2023-12-27 05:14:52,304][105692] Updated weights for policy 0, policy_version 1901171 (0.0009) [2023-12-27 05:14:52,373][105692] Updated weights for policy 0, policy_version 1901181 (0.0008) [2023-12-27 05:14:52,441][105692] Updated weights for policy 0, policy_version 1901191 (0.0009) [2023-12-27 05:14:52,583][105620] Updated weights for policy 1, policy_version 1905681 (0.0009) [2023-12-27 05:14:52,633][105620] Updated weights for policy 1, policy_version 1905691 (0.0011) [2023-12-27 05:14:52,696][105620] Updated weights for policy 1, policy_version 1905701 (0.0011) [2023-12-27 05:14:52,756][105620] Updated weights for policy 1, policy_version 1905711 (0.0011) [2023-12-27 05:14:53,133][105692] Updated weights for policy 0, policy_version 1901201 (0.0010) [2023-12-27 05:14:53,201][105692] Updated weights for policy 0, policy_version 1901211 (0.0005) [2023-12-27 05:14:53,272][105692] Updated weights for policy 0, policy_version 1901221 (0.0005) [2023-12-27 05:14:53,335][105692] Updated weights for policy 0, policy_version 1901231 (0.0005) [2023-12-27 05:14:53,503][105620] Updated weights for policy 1, policy_version 1905721 (0.0007) [2023-12-27 05:14:53,566][105620] Updated weights for policy 1, policy_version 1905731 (0.0009) [2023-12-27 05:14:53,625][105620] Updated weights for policy 1, policy_version 1905742 (0.0010) [2023-12-27 05:14:53,913][105692] Updated weights for policy 0, policy_version 1901241 (0.0007) [2023-12-27 05:14:53,968][105692] Updated weights for policy 0, policy_version 1901251 (0.0008) [2023-12-27 05:14:54,027][105692] Updated weights for policy 0, policy_version 1901261 (0.0009) [2023-12-27 05:14:54,377][105620] Updated weights for policy 1, policy_version 1905752 (0.0009) [2023-12-27 05:14:54,431][105620] Updated weights for policy 1, policy_version 1905762 (0.0009) [2023-12-27 05:14:54,490][105620] Updated weights for policy 1, policy_version 1905772 (0.0006) [2023-12-27 05:14:54,806][105692] Updated weights for policy 0, policy_version 1901271 (0.0009) [2023-12-27 05:14:54,861][105692] Updated weights for policy 0, policy_version 1901282 (0.0010) [2023-12-27 05:14:54,915][105692] Updated weights for policy 0, policy_version 1901294 (0.0010) [2023-12-27 05:14:55,145][105620] Updated weights for policy 1, policy_version 1905782 (0.0008) [2023-12-27 05:14:55,191][105620] Updated weights for policy 1, policy_version 1905792 (0.0009) [2023-12-27 05:14:55,236][105620] Updated weights for policy 1, policy_version 1905802 (0.0008) [2023-12-27 05:14:55,638][105692] Updated weights for policy 0, policy_version 1901304 (0.0006) [2023-12-27 05:14:55,686][105692] Updated weights for policy 0, policy_version 1901314 (0.0006) [2023-12-27 05:14:55,739][105692] Updated weights for policy 0, policy_version 1901324 (0.0006) [2023-12-27 05:14:56,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19114.7, 300 sec: 19383.1). Total num frames: 974766080. Throughput: 0: 9808.3, 1: 9497.4. Samples: 974777412. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:14:56,063][104569] Avg episode reward: [(0, '7812.084'), (1, '9160.765')] [2023-12-27 05:14:56,108][105620] Updated weights for policy 1, policy_version 1905812 (0.0007) [2023-12-27 05:14:56,167][105620] Updated weights for policy 1, policy_version 1905822 (0.0009) [2023-12-27 05:14:56,213][105620] Updated weights for policy 1, policy_version 1905832 (0.0008) [2023-12-27 05:14:56,325][105692] Updated weights for policy 0, policy_version 1901334 (0.0008) [2023-12-27 05:14:56,384][105692] Updated weights for policy 0, policy_version 1901344 (0.0009) [2023-12-27 05:14:56,442][105692] Updated weights for policy 0, policy_version 1901354 (0.0009) [2023-12-27 05:14:56,843][105620] Updated weights for policy 1, policy_version 1905842 (0.0009) [2023-12-27 05:14:56,896][105620] Updated weights for policy 1, policy_version 1905852 (0.0005) [2023-12-27 05:14:56,949][105620] Updated weights for policy 1, policy_version 1905862 (0.0005) [2023-12-27 05:14:57,000][105620] Updated weights for policy 1, policy_version 1905872 (0.0005) [2023-12-27 05:14:57,321][105692] Updated weights for policy 0, policy_version 1901364 (0.0009) [2023-12-27 05:14:57,378][105692] Updated weights for policy 0, policy_version 1901374 (0.0009) [2023-12-27 05:14:57,431][105692] Updated weights for policy 0, policy_version 1901384 (0.0010) [2023-12-27 05:14:57,561][105620] Updated weights for policy 1, policy_version 1905882 (0.0005) [2023-12-27 05:14:57,618][105620] Updated weights for policy 1, policy_version 1905892 (0.0005) [2023-12-27 05:14:57,679][105620] Updated weights for policy 1, policy_version 1905902 (0.0005) [2023-12-27 05:14:58,203][105692] Updated weights for policy 0, policy_version 1901394 (0.0009) [2023-12-27 05:14:58,238][105620] Updated weights for policy 1, policy_version 1905912 (0.0007) [2023-12-27 05:14:58,265][105692] Updated weights for policy 0, policy_version 1901404 (0.0008) [2023-12-27 05:14:58,300][105620] Updated weights for policy 1, policy_version 1905922 (0.0007) [2023-12-27 05:14:58,324][105692] Updated weights for policy 0, policy_version 1901414 (0.0007) [2023-12-27 05:14:58,367][105620] Updated weights for policy 1, policy_version 1905932 (0.0007) [2023-12-27 05:14:58,395][105692] Updated weights for policy 0, policy_version 1901424 (0.0008) [2023-12-27 05:14:59,189][105620] Updated weights for policy 1, policy_version 1905942 (0.0011) [2023-12-27 05:14:59,259][105620] Updated weights for policy 1, policy_version 1905952 (0.0010) [2023-12-27 05:14:59,268][105692] Updated weights for policy 0, policy_version 1901434 (0.0008) [2023-12-27 05:14:59,329][105692] Updated weights for policy 0, policy_version 1901444 (0.0007) [2023-12-27 05:14:59,330][105620] Updated weights for policy 1, policy_version 1905962 (0.0010) [2023-12-27 05:14:59,389][105692] Updated weights for policy 0, policy_version 1901454 (0.0009) [2023-12-27 05:15:00,056][105692] Updated weights for policy 0, policy_version 1901464 (0.0008) [2023-12-27 05:15:00,115][105692] Updated weights for policy 0, policy_version 1901474 (0.0007) [2023-12-27 05:15:00,165][105620] Updated weights for policy 1, policy_version 1905972 (0.0011) [2023-12-27 05:15:00,174][105692] Updated weights for policy 0, policy_version 1901484 (0.0009) [2023-12-27 05:15:00,227][105620] Updated weights for policy 1, policy_version 1905982 (0.0007) [2023-12-27 05:15:00,286][105620] Updated weights for policy 1, policy_version 1905992 (0.0009) [2023-12-27 05:15:00,919][105692] Updated weights for policy 0, policy_version 1901494 (0.0008) [2023-12-27 05:15:00,971][105692] Updated weights for policy 0, policy_version 1901504 (0.0009) [2023-12-27 05:15:01,002][105620] Updated weights for policy 1, policy_version 1906002 (0.0008) [2023-12-27 05:15:01,033][105692] Updated weights for policy 0, policy_version 1901514 (0.0008) [2023-12-27 05:15:01,062][105620] Updated weights for policy 1, policy_version 1906012 (0.0008) [2023-12-27 05:15:01,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19114.7, 300 sec: 19383.1). Total num frames: 974856192. Throughput: 0: 9826.0, 1: 9539.3. Samples: 974837672. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:15:01,062][104569] Avg episode reward: [(0, '7995.578'), (1, '9253.073')] [2023-12-27 05:15:01,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001901520_486858752.pth... [2023-12-27 05:15:01,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001900368_486563840.pth [2023-12-27 05:15:01,111][105620] Updated weights for policy 1, policy_version 1906022 (0.0008) [2023-12-27 05:15:01,170][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001906032_488013824.pth... [2023-12-27 05:15:01,173][105620] Updated weights for policy 1, policy_version 1906032 (0.0009) [2023-12-27 05:15:01,175][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001904880_487718912.pth [2023-12-27 05:15:01,816][105692] Updated weights for policy 0, policy_version 1901524 (0.0007) [2023-12-27 05:15:01,871][105692] Updated weights for policy 0, policy_version 1901534 (0.0008) [2023-12-27 05:15:01,884][105620] Updated weights for policy 1, policy_version 1906042 (0.0007) [2023-12-27 05:15:01,934][105692] Updated weights for policy 0, policy_version 1901544 (0.0008) [2023-12-27 05:15:01,947][105620] Updated weights for policy 1, policy_version 1906052 (0.0007) [2023-12-27 05:15:02,009][105620] Updated weights for policy 1, policy_version 1906062 (0.0007) [2023-12-27 05:15:02,657][105692] Updated weights for policy 0, policy_version 1901554 (0.0008) [2023-12-27 05:15:02,724][105692] Updated weights for policy 0, policy_version 1901564 (0.0008) [2023-12-27 05:15:02,752][105620] Updated weights for policy 1, policy_version 1906072 (0.0010) [2023-12-27 05:15:02,789][105692] Updated weights for policy 0, policy_version 1901574 (0.0008) [2023-12-27 05:15:02,809][105620] Updated weights for policy 1, policy_version 1906082 (0.0010) [2023-12-27 05:15:02,851][105692] Updated weights for policy 0, policy_version 1901584 (0.0008) [2023-12-27 05:15:02,867][105620] Updated weights for policy 1, policy_version 1906092 (0.0010) [2023-12-27 05:15:03,429][105692] Updated weights for policy 0, policy_version 1901594 (0.0008) [2023-12-27 05:15:03,489][105692] Updated weights for policy 0, policy_version 1901604 (0.0008) [2023-12-27 05:15:03,494][105620] Updated weights for policy 1, policy_version 1906102 (0.0008) [2023-12-27 05:15:03,553][105692] Updated weights for policy 0, policy_version 1901614 (0.0006) [2023-12-27 05:15:03,558][105620] Updated weights for policy 1, policy_version 1906112 (0.0011) [2023-12-27 05:15:03,619][105620] Updated weights for policy 1, policy_version 1906122 (0.0011) [2023-12-27 05:15:04,279][105692] Updated weights for policy 0, policy_version 1901624 (0.0005) [2023-12-27 05:15:04,344][105692] Updated weights for policy 0, policy_version 1901634 (0.0007) [2023-12-27 05:15:04,366][105620] Updated weights for policy 1, policy_version 1906132 (0.0010) [2023-12-27 05:15:04,406][105692] Updated weights for policy 0, policy_version 1901644 (0.0007) [2023-12-27 05:15:04,426][105620] Updated weights for policy 1, policy_version 1906142 (0.0009) [2023-12-27 05:15:04,475][105620] Updated weights for policy 1, policy_version 1906152 (0.0011) [2023-12-27 05:15:05,065][105692] Updated weights for policy 0, policy_version 1901654 (0.0007) [2023-12-27 05:15:05,115][105692] Updated weights for policy 0, policy_version 1901664 (0.0008) [2023-12-27 05:15:05,167][105692] Updated weights for policy 0, policy_version 1901674 (0.0008) [2023-12-27 05:15:05,192][105620] Updated weights for policy 1, policy_version 1906162 (0.0010) [2023-12-27 05:15:05,240][105620] Updated weights for policy 1, policy_version 1906172 (0.0010) [2023-12-27 05:15:05,288][105620] Updated weights for policy 1, policy_version 1906182 (0.0010) [2023-12-27 05:15:05,340][105620] Updated weights for policy 1, policy_version 1906192 (0.0010) [2023-12-27 05:15:05,838][105692] Updated weights for policy 0, policy_version 1901684 (0.0006) [2023-12-27 05:15:05,890][105692] Updated weights for policy 0, policy_version 1901694 (0.0006) [2023-12-27 05:15:05,950][105692] Updated weights for policy 0, policy_version 1901704 (0.0005) [2023-12-27 05:15:06,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 974962688. Throughput: 0: 9765.3, 1: 9583.9. Samples: 974952292. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:15:06,063][104569] Avg episode reward: [(0, '8537.565'), (1, '9163.772')] [2023-12-27 05:15:06,085][105620] Updated weights for policy 1, policy_version 1906202 (0.0010) [2023-12-27 05:15:06,143][105620] Updated weights for policy 1, policy_version 1906212 (0.0010) [2023-12-27 05:15:06,192][105620] Updated weights for policy 1, policy_version 1906222 (0.0010) [2023-12-27 05:15:06,579][105692] Updated weights for policy 0, policy_version 1901714 (0.0005) [2023-12-27 05:15:06,646][105692] Updated weights for policy 0, policy_version 1901724 (0.0007) [2023-12-27 05:15:06,711][105692] Updated weights for policy 0, policy_version 1901734 (0.0008) [2023-12-27 05:15:06,767][105692] Updated weights for policy 0, policy_version 1901744 (0.0008) [2023-12-27 05:15:06,909][105620] Updated weights for policy 1, policy_version 1906232 (0.0005) [2023-12-27 05:15:06,956][105620] Updated weights for policy 1, policy_version 1906242 (0.0005) [2023-12-27 05:15:07,005][105620] Updated weights for policy 1, policy_version 1906252 (0.0005) [2023-12-27 05:15:07,395][105692] Updated weights for policy 0, policy_version 1901754 (0.0005) [2023-12-27 05:15:07,452][105692] Updated weights for policy 0, policy_version 1901764 (0.0005) [2023-12-27 05:15:07,507][105692] Updated weights for policy 0, policy_version 1901774 (0.0005) [2023-12-27 05:15:07,745][105620] Updated weights for policy 1, policy_version 1906262 (0.0008) [2023-12-27 05:15:07,795][105620] Updated weights for policy 1, policy_version 1906272 (0.0008) [2023-12-27 05:15:07,865][105620] Updated weights for policy 1, policy_version 1906282 (0.0009) [2023-12-27 05:15:08,242][105692] Updated weights for policy 0, policy_version 1901784 (0.0009) [2023-12-27 05:15:08,297][105692] Updated weights for policy 0, policy_version 1901794 (0.0012) [2023-12-27 05:15:08,367][105692] Updated weights for policy 0, policy_version 1901805 (0.0008) [2023-12-27 05:15:08,522][105620] Updated weights for policy 1, policy_version 1906292 (0.0008) [2023-12-27 05:15:08,573][105620] Updated weights for policy 1, policy_version 1906302 (0.0008) [2023-12-27 05:15:08,617][105620] Updated weights for policy 1, policy_version 1906312 (0.0008) [2023-12-27 05:15:09,136][105692] Updated weights for policy 0, policy_version 1901815 (0.0006) [2023-12-27 05:15:09,194][105692] Updated weights for policy 0, policy_version 1901825 (0.0007) [2023-12-27 05:15:09,261][105692] Updated weights for policy 0, policy_version 1901835 (0.0009) [2023-12-27 05:15:09,434][105620] Updated weights for policy 1, policy_version 1906322 (0.0008) [2023-12-27 05:15:09,496][105620] Updated weights for policy 1, policy_version 1906332 (0.0008) [2023-12-27 05:15:09,556][105620] Updated weights for policy 1, policy_version 1906342 (0.0009) [2023-12-27 05:15:09,611][105620] Updated weights for policy 1, policy_version 1906352 (0.0009) [2023-12-27 05:15:09,976][105692] Updated weights for policy 0, policy_version 1901845 (0.0009) [2023-12-27 05:15:10,039][105692] Updated weights for policy 0, policy_version 1901855 (0.0009) [2023-12-27 05:15:10,100][105692] Updated weights for policy 0, policy_version 1901865 (0.0008) [2023-12-27 05:15:10,340][105620] Updated weights for policy 1, policy_version 1906362 (0.0009) [2023-12-27 05:15:10,406][105620] Updated weights for policy 1, policy_version 1906372 (0.0006) [2023-12-27 05:15:10,469][105620] Updated weights for policy 1, policy_version 1906382 (0.0007) [2023-12-27 05:15:10,942][105692] Updated weights for policy 0, policy_version 1901875 (0.0010) [2023-12-27 05:15:11,001][105692] Updated weights for policy 0, policy_version 1901885 (0.0009) [2023-12-27 05:15:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 975052800. Throughput: 0: 9764.4, 1: 9704.1. Samples: 975070084. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:15:11,062][104569] Avg episode reward: [(0, '8806.233'), (1, '9252.976')] [2023-12-27 05:15:11,064][105692] Updated weights for policy 0, policy_version 1901895 (0.0009) [2023-12-27 05:15:11,082][105620] Updated weights for policy 1, policy_version 1906392 (0.0007) [2023-12-27 05:15:11,139][105620] Updated weights for policy 1, policy_version 1906402 (0.0006) [2023-12-27 05:15:11,201][105620] Updated weights for policy 1, policy_version 1906412 (0.0009) [2023-12-27 05:15:11,853][105620] Updated weights for policy 1, policy_version 1906422 (0.0007) [2023-12-27 05:15:11,913][105620] Updated weights for policy 1, policy_version 1906432 (0.0005) [2023-12-27 05:15:11,935][105692] Updated weights for policy 0, policy_version 1901905 (0.0008) [2023-12-27 05:15:11,971][105620] Updated weights for policy 1, policy_version 1906442 (0.0006) [2023-12-27 05:15:11,991][105692] Updated weights for policy 0, policy_version 1901915 (0.0008) [2023-12-27 05:15:12,056][105692] Updated weights for policy 0, policy_version 1901925 (0.0009) [2023-12-27 05:15:12,120][105692] Updated weights for policy 0, policy_version 1901935 (0.0008) [2023-12-27 05:15:12,590][105620] Updated weights for policy 1, policy_version 1906452 (0.0006) [2023-12-27 05:15:12,657][105620] Updated weights for policy 1, policy_version 1906462 (0.0005) [2023-12-27 05:15:12,727][105620] Updated weights for policy 1, policy_version 1906472 (0.0006) [2023-12-27 05:15:12,766][105692] Updated weights for policy 0, policy_version 1901945 (0.0006) [2023-12-27 05:15:12,833][105692] Updated weights for policy 0, policy_version 1901955 (0.0005) [2023-12-27 05:15:12,885][105692] Updated weights for policy 0, policy_version 1901965 (0.0006) [2023-12-27 05:15:13,354][105620] Updated weights for policy 1, policy_version 1906482 (0.0008) [2023-12-27 05:15:13,404][105620] Updated weights for policy 1, policy_version 1906492 (0.0006) [2023-12-27 05:15:13,427][105692] Updated weights for policy 0, policy_version 1901975 (0.0006) [2023-12-27 05:15:13,456][105620] Updated weights for policy 1, policy_version 1906502 (0.0008) [2023-12-27 05:15:13,493][105692] Updated weights for policy 0, policy_version 1901985 (0.0005) [2023-12-27 05:15:13,502][105620] Updated weights for policy 1, policy_version 1906512 (0.0007) [2023-12-27 05:15:13,557][105692] Updated weights for policy 0, policy_version 1901995 (0.0005) [2023-12-27 05:15:14,066][105620] Updated weights for policy 1, policy_version 1906522 (0.0006) [2023-12-27 05:15:14,122][105620] Updated weights for policy 1, policy_version 1906532 (0.0008) [2023-12-27 05:15:14,123][105692] Updated weights for policy 0, policy_version 1902005 (0.0008) [2023-12-27 05:15:14,185][105620] Updated weights for policy 1, policy_version 1906542 (0.0006) [2023-12-27 05:15:14,186][105692] Updated weights for policy 0, policy_version 1902015 (0.0010) [2023-12-27 05:15:14,245][105692] Updated weights for policy 0, policy_version 1902025 (0.0010) [2023-12-27 05:15:14,945][105620] Updated weights for policy 1, policy_version 1906552 (0.0009) [2023-12-27 05:15:14,959][105692] Updated weights for policy 0, policy_version 1902035 (0.0010) [2023-12-27 05:15:15,009][105692] Updated weights for policy 0, policy_version 1902045 (0.0006) [2023-12-27 05:15:15,011][105620] Updated weights for policy 1, policy_version 1906562 (0.0007) [2023-12-27 05:15:15,063][105692] Updated weights for policy 0, policy_version 1902055 (0.0007) [2023-12-27 05:15:15,067][105620] Updated weights for policy 1, policy_version 1906572 (0.0008) [2023-12-27 05:15:15,766][105620] Updated weights for policy 1, policy_version 1906582 (0.0006) [2023-12-27 05:15:15,771][105692] Updated weights for policy 0, policy_version 1902065 (0.0009) [2023-12-27 05:15:15,827][105692] Updated weights for policy 0, policy_version 1902075 (0.0005) [2023-12-27 05:15:15,833][105620] Updated weights for policy 1, policy_version 1906592 (0.0007) [2023-12-27 05:15:15,892][105692] Updated weights for policy 0, policy_version 1902085 (0.0008) [2023-12-27 05:15:15,898][105620] Updated weights for policy 1, policy_version 1906602 (0.0005) [2023-12-27 05:15:15,950][105692] Updated weights for policy 0, policy_version 1902095 (0.0009) [2023-12-27 05:15:16,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 975167488. Throughput: 0: 9776.0, 1: 9810.5. Samples: 975131884. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:15:16,062][104569] Avg episode reward: [(0, '8442.829'), (1, '9345.287')] [2023-12-27 05:15:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001902096_487006208.pth... [2023-12-27 05:15:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001906608_488161280.pth... [2023-12-27 05:15:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001905456_487866368.pth [2023-12-27 05:15:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001900944_486711296.pth [2023-12-27 05:15:16,397][105620] Updated weights for policy 1, policy_version 1906612 (0.0005) [2023-12-27 05:15:16,444][105620] Updated weights for policy 1, policy_version 1906622 (0.0005) [2023-12-27 05:15:16,494][105620] Updated weights for policy 1, policy_version 1906632 (0.0005) [2023-12-27 05:15:16,751][105692] Updated weights for policy 0, policy_version 1902105 (0.0009) [2023-12-27 05:15:16,816][105692] Updated weights for policy 0, policy_version 1902115 (0.0008) [2023-12-27 05:15:16,868][105692] Updated weights for policy 0, policy_version 1902125 (0.0009) [2023-12-27 05:15:17,020][105620] Updated weights for policy 1, policy_version 1906642 (0.0005) [2023-12-27 05:15:17,077][105620] Updated weights for policy 1, policy_version 1906652 (0.0005) [2023-12-27 05:15:17,139][105620] Updated weights for policy 1, policy_version 1906662 (0.0005) [2023-12-27 05:15:17,199][105620] Updated weights for policy 1, policy_version 1906672 (0.0005) [2023-12-27 05:15:17,626][105692] Updated weights for policy 0, policy_version 1902135 (0.0007) [2023-12-27 05:15:17,676][105692] Updated weights for policy 0, policy_version 1902145 (0.0005) [2023-12-27 05:15:17,732][105692] Updated weights for policy 0, policy_version 1902155 (0.0006) [2023-12-27 05:15:17,734][105620] Updated weights for policy 1, policy_version 1906682 (0.0011) [2023-12-27 05:15:17,792][105620] Updated weights for policy 1, policy_version 1906692 (0.0010) [2023-12-27 05:15:17,844][105620] Updated weights for policy 1, policy_version 1906702 (0.0006) [2023-12-27 05:15:18,368][105692] Updated weights for policy 0, policy_version 1902165 (0.0007) [2023-12-27 05:15:18,435][105692] Updated weights for policy 0, policy_version 1902175 (0.0008) [2023-12-27 05:15:18,497][105692] Updated weights for policy 0, policy_version 1902185 (0.0008) [2023-12-27 05:15:18,568][105620] Updated weights for policy 1, policy_version 1906712 (0.0010) [2023-12-27 05:15:18,632][105620] Updated weights for policy 1, policy_version 1906722 (0.0008) [2023-12-27 05:15:18,691][105620] Updated weights for policy 1, policy_version 1906732 (0.0009) [2023-12-27 05:15:19,109][105692] Updated weights for policy 0, policy_version 1902195 (0.0008) [2023-12-27 05:15:19,164][105692] Updated weights for policy 0, policy_version 1902205 (0.0010) [2023-12-27 05:15:19,229][105692] Updated weights for policy 0, policy_version 1902215 (0.0009) [2023-12-27 05:15:19,459][105620] Updated weights for policy 1, policy_version 1906742 (0.0008) [2023-12-27 05:15:19,526][105620] Updated weights for policy 1, policy_version 1906752 (0.0008) [2023-12-27 05:15:19,582][105620] Updated weights for policy 1, policy_version 1906762 (0.0008) [2023-12-27 05:15:20,070][105692] Updated weights for policy 0, policy_version 1902225 (0.0009) [2023-12-27 05:15:20,127][105692] Updated weights for policy 0, policy_version 1902235 (0.0008) [2023-12-27 05:15:20,176][105692] Updated weights for policy 0, policy_version 1902245 (0.0009) [2023-12-27 05:15:20,228][105692] Updated weights for policy 0, policy_version 1902255 (0.0009) [2023-12-27 05:15:20,344][105620] Updated weights for policy 1, policy_version 1906772 (0.0008) [2023-12-27 05:15:20,400][105620] Updated weights for policy 1, policy_version 1906782 (0.0010) [2023-12-27 05:15:20,462][105620] Updated weights for policy 1, policy_version 1906792 (0.0009) [2023-12-27 05:15:20,973][105692] Updated weights for policy 0, policy_version 1902265 (0.0009) [2023-12-27 05:15:21,037][105692] Updated weights for policy 0, policy_version 1902275 (0.0009) [2023-12-27 05:15:21,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 975257600. Throughput: 0: 9834.3, 1: 9966.3. Samples: 975253996. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:15:21,062][104569] Avg episode reward: [(0, '8355.115'), (1, '9254.939')] [2023-12-27 05:15:21,102][105692] Updated weights for policy 0, policy_version 1902285 (0.0009) [2023-12-27 05:15:21,248][105620] Updated weights for policy 1, policy_version 1906802 (0.0008) [2023-12-27 05:15:21,317][105620] Updated weights for policy 1, policy_version 1906812 (0.0007) [2023-12-27 05:15:21,386][105620] Updated weights for policy 1, policy_version 1906822 (0.0007) [2023-12-27 05:15:21,452][105620] Updated weights for policy 1, policy_version 1906832 (0.0006) [2023-12-27 05:15:21,880][105692] Updated weights for policy 0, policy_version 1902295 (0.0008) [2023-12-27 05:15:21,935][105692] Updated weights for policy 0, policy_version 1902305 (0.0008) [2023-12-27 05:15:21,996][105692] Updated weights for policy 0, policy_version 1902315 (0.0008) [2023-12-27 05:15:22,085][105620] Updated weights for policy 1, policy_version 1906842 (0.0008) [2023-12-27 05:15:22,145][105620] Updated weights for policy 1, policy_version 1906852 (0.0011) [2023-12-27 05:15:22,210][105620] Updated weights for policy 1, policy_version 1906862 (0.0006) [2023-12-27 05:15:22,782][105692] Updated weights for policy 0, policy_version 1902325 (0.0010) [2023-12-27 05:15:22,844][105620] Updated weights for policy 1, policy_version 1906872 (0.0009) [2023-12-27 05:15:22,845][105692] Updated weights for policy 0, policy_version 1902335 (0.0011) [2023-12-27 05:15:22,896][105620] Updated weights for policy 1, policy_version 1906882 (0.0010) [2023-12-27 05:15:22,904][105692] Updated weights for policy 0, policy_version 1902345 (0.0010) [2023-12-27 05:15:22,954][105620] Updated weights for policy 1, policy_version 1906892 (0.0011) [2023-12-27 05:15:23,539][105692] Updated weights for policy 0, policy_version 1902355 (0.0007) [2023-12-27 05:15:23,602][105692] Updated weights for policy 0, policy_version 1902365 (0.0008) [2023-12-27 05:15:23,604][105620] Updated weights for policy 1, policy_version 1906902 (0.0008) [2023-12-27 05:15:23,663][105692] Updated weights for policy 0, policy_version 1902375 (0.0006) [2023-12-27 05:15:23,672][105620] Updated weights for policy 1, policy_version 1906912 (0.0008) [2023-12-27 05:15:23,731][105620] Updated weights for policy 1, policy_version 1906922 (0.0011) [2023-12-27 05:15:24,246][105692] Updated weights for policy 0, policy_version 1902385 (0.0005) [2023-12-27 05:15:24,303][105692] Updated weights for policy 0, policy_version 1902395 (0.0005) [2023-12-27 05:15:24,361][105692] Updated weights for policy 0, policy_version 1902405 (0.0005) [2023-12-27 05:15:24,420][105692] Updated weights for policy 0, policy_version 1902415 (0.0005) [2023-12-27 05:15:24,435][105620] Updated weights for policy 1, policy_version 1906932 (0.0009) [2023-12-27 05:15:24,505][105620] Updated weights for policy 1, policy_version 1906942 (0.0010) [2023-12-27 05:15:24,566][105620] Updated weights for policy 1, policy_version 1906952 (0.0010) [2023-12-27 05:15:25,042][105692] Updated weights for policy 0, policy_version 1902425 (0.0009) [2023-12-27 05:15:25,093][105692] Updated weights for policy 0, policy_version 1902435 (0.0009) [2023-12-27 05:15:25,157][105692] Updated weights for policy 0, policy_version 1902445 (0.0007) [2023-12-27 05:15:25,281][105620] Updated weights for policy 1, policy_version 1906962 (0.0008) [2023-12-27 05:15:25,337][105620] Updated weights for policy 1, policy_version 1906972 (0.0005) [2023-12-27 05:15:25,395][105620] Updated weights for policy 1, policy_version 1906982 (0.0005) [2023-12-27 05:15:25,465][105620] Updated weights for policy 1, policy_version 1906992 (0.0005) [2023-12-27 05:15:25,810][105692] Updated weights for policy 0, policy_version 1902455 (0.0007) [2023-12-27 05:15:25,872][105692] Updated weights for policy 0, policy_version 1902465 (0.0006) [2023-12-27 05:15:25,939][105692] Updated weights for policy 0, policy_version 1902475 (0.0006) [2023-12-27 05:15:26,055][105620] Updated weights for policy 1, policy_version 1907002 (0.0009) [2023-12-27 05:15:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 975364096. Throughput: 0: 9794.8, 1: 9980.0. Samples: 975373792. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:15:26,062][104569] Avg episode reward: [(0, '8358.445'), (1, '9167.530')] [2023-12-27 05:15:26,108][105620] Updated weights for policy 1, policy_version 1907012 (0.0010) [2023-12-27 05:15:26,172][105620] Updated weights for policy 1, policy_version 1907022 (0.0008) [2023-12-27 05:15:26,528][105692] Updated weights for policy 0, policy_version 1902485 (0.0008) [2023-12-27 05:15:26,576][105692] Updated weights for policy 0, policy_version 1902495 (0.0010) [2023-12-27 05:15:26,648][105692] Updated weights for policy 0, policy_version 1902505 (0.0010) [2023-12-27 05:15:26,766][105620] Updated weights for policy 1, policy_version 1907032 (0.0005) [2023-12-27 05:15:26,810][105620] Updated weights for policy 1, policy_version 1907042 (0.0006) [2023-12-27 05:15:26,861][105620] Updated weights for policy 1, policy_version 1907052 (0.0010) [2023-12-27 05:15:27,343][105692] Updated weights for policy 0, policy_version 1902515 (0.0011) [2023-12-27 05:15:27,395][105692] Updated weights for policy 0, policy_version 1902526 (0.0009) [2023-12-27 05:15:27,445][105692] Updated weights for policy 0, policy_version 1902536 (0.0005) [2023-12-27 05:15:27,538][105620] Updated weights for policy 1, policy_version 1907062 (0.0007) [2023-12-27 05:15:27,592][105620] Updated weights for policy 1, policy_version 1907072 (0.0010) [2023-12-27 05:15:27,649][105620] Updated weights for policy 1, policy_version 1907082 (0.0010) [2023-12-27 05:15:28,119][105692] Updated weights for policy 0, policy_version 1902546 (0.0006) [2023-12-27 05:15:28,181][105692] Updated weights for policy 0, policy_version 1902557 (0.0010) [2023-12-27 05:15:28,251][105692] Updated weights for policy 0, policy_version 1902567 (0.0010) [2023-12-27 05:15:28,307][105620] Updated weights for policy 1, policy_version 1907093 (0.0007) [2023-12-27 05:15:28,367][105620] Updated weights for policy 1, policy_version 1907103 (0.0006) [2023-12-27 05:15:28,425][105620] Updated weights for policy 1, policy_version 1907113 (0.0007) [2023-12-27 05:15:29,042][105620] Updated weights for policy 1, policy_version 1907123 (0.0009) [2023-12-27 05:15:29,059][105692] Updated weights for policy 0, policy_version 1902577 (0.0009) [2023-12-27 05:15:29,098][105620] Updated weights for policy 1, policy_version 1907133 (0.0006) [2023-12-27 05:15:29,118][105692] Updated weights for policy 0, policy_version 1902587 (0.0009) [2023-12-27 05:15:29,147][105620] Updated weights for policy 1, policy_version 1907143 (0.0005) [2023-12-27 05:15:29,171][105692] Updated weights for policy 0, policy_version 1902597 (0.0009) [2023-12-27 05:15:29,229][105692] Updated weights for policy 0, policy_version 1902607 (0.0009) [2023-12-27 05:15:29,860][105620] Updated weights for policy 1, policy_version 1907153 (0.0005) [2023-12-27 05:15:29,917][105620] Updated weights for policy 1, policy_version 1907163 (0.0008) [2023-12-27 05:15:29,982][105620] Updated weights for policy 1, policy_version 1907173 (0.0009) [2023-12-27 05:15:29,990][105692] Updated weights for policy 0, policy_version 1902617 (0.0008) [2023-12-27 05:15:30,043][105620] Updated weights for policy 1, policy_version 1907183 (0.0008) [2023-12-27 05:15:30,053][105692] Updated weights for policy 0, policy_version 1902627 (0.0008) [2023-12-27 05:15:30,113][105692] Updated weights for policy 0, policy_version 1902637 (0.0008) [2023-12-27 05:15:30,821][105692] Updated weights for policy 0, policy_version 1902647 (0.0008) [2023-12-27 05:15:30,828][105620] Updated weights for policy 1, policy_version 1907193 (0.0009) [2023-12-27 05:15:30,873][105692] Updated weights for policy 0, policy_version 1902657 (0.0009) [2023-12-27 05:15:30,877][105620] Updated weights for policy 1, policy_version 1907203 (0.0010) [2023-12-27 05:15:30,922][105692] Updated weights for policy 0, policy_version 1902667 (0.0008) [2023-12-27 05:15:30,925][105620] Updated weights for policy 1, policy_version 1907213 (0.0010) [2023-12-27 05:15:31,062][104569] Fps is (10 sec: 21299.1, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 975470592. Throughput: 0: 9841.7, 1: 10015.4. Samples: 975437108. Policy #0 lag: (min: 28.0, avg: 36.1, max: 60.0) [2023-12-27 05:15:31,063][104569] Avg episode reward: [(0, '8445.778'), (1, '9258.025')] [2023-12-27 05:15:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001902672_487153664.pth... [2023-12-27 05:15:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001907216_488316928.pth... [2023-12-27 05:15:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001901520_486858752.pth [2023-12-27 05:15:31,088][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001906032_488013824.pth [2023-12-27 05:15:31,641][105692] Updated weights for policy 0, policy_version 1902677 (0.0008) [2023-12-27 05:15:31,701][105692] Updated weights for policy 0, policy_version 1902687 (0.0011) [2023-12-27 05:15:31,722][105620] Updated weights for policy 1, policy_version 1907223 (0.0011) [2023-12-27 05:15:31,766][105692] Updated weights for policy 0, policy_version 1902697 (0.0010) [2023-12-27 05:15:31,787][105620] Updated weights for policy 1, policy_version 1907233 (0.0008) [2023-12-27 05:15:31,836][105620] Updated weights for policy 1, policy_version 1907243 (0.0005) [2023-12-27 05:15:32,429][105692] Updated weights for policy 0, policy_version 1902707 (0.0010) [2023-12-27 05:15:32,470][105620] Updated weights for policy 1, policy_version 1907253 (0.0006) [2023-12-27 05:15:32,488][105692] Updated weights for policy 0, policy_version 1902717 (0.0011) [2023-12-27 05:15:32,526][105620] Updated weights for policy 1, policy_version 1907263 (0.0006) [2023-12-27 05:15:32,546][105692] Updated weights for policy 0, policy_version 1902727 (0.0011) [2023-12-27 05:15:32,584][105620] Updated weights for policy 1, policy_version 1907273 (0.0005) [2023-12-27 05:15:33,219][105692] Updated weights for policy 0, policy_version 1902737 (0.0010) [2023-12-27 05:15:33,267][105692] Updated weights for policy 0, policy_version 1902747 (0.0005) [2023-12-27 05:15:33,316][105692] Updated weights for policy 0, policy_version 1902757 (0.0005) [2023-12-27 05:15:33,362][105692] Updated weights for policy 0, policy_version 1902767 (0.0006) [2023-12-27 05:15:33,396][105620] Updated weights for policy 1, policy_version 1907283 (0.0008) [2023-12-27 05:15:33,447][105620] Updated weights for policy 1, policy_version 1907295 (0.0009) [2023-12-27 05:15:33,498][105620] Updated weights for policy 1, policy_version 1907305 (0.0009) [2023-12-27 05:15:33,926][105692] Updated weights for policy 0, policy_version 1902777 (0.0009) [2023-12-27 05:15:33,972][105692] Updated weights for policy 0, policy_version 1902787 (0.0008) [2023-12-27 05:15:34,019][105692] Updated weights for policy 0, policy_version 1902797 (0.0009) [2023-12-27 05:15:34,339][105620] Updated weights for policy 1, policy_version 1907315 (0.0009) [2023-12-27 05:15:34,396][105620] Updated weights for policy 1, policy_version 1907325 (0.0010) [2023-12-27 05:15:34,459][105620] Updated weights for policy 1, policy_version 1907335 (0.0008) [2023-12-27 05:15:34,717][105692] Updated weights for policy 0, policy_version 1902807 (0.0007) [2023-12-27 05:15:34,769][105692] Updated weights for policy 0, policy_version 1902817 (0.0005) [2023-12-27 05:15:34,820][105692] Updated weights for policy 0, policy_version 1902827 (0.0006) [2023-12-27 05:15:35,267][105620] Updated weights for policy 1, policy_version 1907345 (0.0010) [2023-12-27 05:15:35,332][105620] Updated weights for policy 1, policy_version 1907355 (0.0009) [2023-12-27 05:15:35,394][105620] Updated weights for policy 1, policy_version 1907365 (0.0009) [2023-12-27 05:15:35,452][105620] Updated weights for policy 1, policy_version 1907375 (0.0009) [2023-12-27 05:15:35,513][105692] Updated weights for policy 0, policy_version 1902837 (0.0008) [2023-12-27 05:15:35,563][105692] Updated weights for policy 0, policy_version 1902847 (0.0009) [2023-12-27 05:15:35,621][105692] Updated weights for policy 0, policy_version 1902857 (0.0009) [2023-12-27 05:15:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.4, 300 sec: 19466.4). Total num frames: 975560704. Throughput: 0: 9838.4, 1: 9940.5. Samples: 975553020. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:15:36,062][104569] Avg episode reward: [(0, '8168.056'), (1, '9255.162')] [2023-12-27 05:15:36,249][105620] Updated weights for policy 1, policy_version 1907385 (0.0009) [2023-12-27 05:15:36,307][105692] Updated weights for policy 0, policy_version 1902867 (0.0008) [2023-12-27 05:15:36,313][105620] Updated weights for policy 1, policy_version 1907395 (0.0009) [2023-12-27 05:15:36,366][105692] Updated weights for policy 0, policy_version 1902877 (0.0006) [2023-12-27 05:15:36,372][105620] Updated weights for policy 1, policy_version 1907405 (0.0009) [2023-12-27 05:15:36,431][105692] Updated weights for policy 0, policy_version 1902887 (0.0006) [2023-12-27 05:15:37,085][105692] Updated weights for policy 0, policy_version 1902897 (0.0008) [2023-12-27 05:15:37,143][105692] Updated weights for policy 0, policy_version 1902907 (0.0009) [2023-12-27 05:15:37,173][105620] Updated weights for policy 1, policy_version 1907415 (0.0009) [2023-12-27 05:15:37,201][105692] Updated weights for policy 0, policy_version 1902917 (0.0006) [2023-12-27 05:15:37,234][105620] Updated weights for policy 1, policy_version 1907425 (0.0009) [2023-12-27 05:15:37,253][105692] Updated weights for policy 0, policy_version 1902927 (0.0006) [2023-12-27 05:15:37,295][105620] Updated weights for policy 1, policy_version 1907435 (0.0008) [2023-12-27 05:15:38,007][105692] Updated weights for policy 0, policy_version 1902937 (0.0008) [2023-12-27 05:15:38,049][105620] Updated weights for policy 1, policy_version 1907445 (0.0007) [2023-12-27 05:15:38,070][105692] Updated weights for policy 0, policy_version 1902947 (0.0006) [2023-12-27 05:15:38,114][105620] Updated weights for policy 1, policy_version 1907455 (0.0007) [2023-12-27 05:15:38,126][105692] Updated weights for policy 0, policy_version 1902957 (0.0006) [2023-12-27 05:15:38,175][105620] Updated weights for policy 1, policy_version 1907465 (0.0009) [2023-12-27 05:15:38,807][105692] Updated weights for policy 0, policy_version 1902967 (0.0008) [2023-12-27 05:15:38,873][105692] Updated weights for policy 0, policy_version 1902977 (0.0009) [2023-12-27 05:15:38,940][105692] Updated weights for policy 0, policy_version 1902987 (0.0011) [2023-12-27 05:15:38,946][105620] Updated weights for policy 1, policy_version 1907475 (0.0008) [2023-12-27 05:15:39,007][105620] Updated weights for policy 1, policy_version 1907485 (0.0005) [2023-12-27 05:15:39,069][105620] Updated weights for policy 1, policy_version 1907495 (0.0005) [2023-12-27 05:15:39,636][105692] Updated weights for policy 0, policy_version 1902997 (0.0011) [2023-12-27 05:15:39,695][105692] Updated weights for policy 0, policy_version 1903007 (0.0011) [2023-12-27 05:15:39,748][105620] Updated weights for policy 1, policy_version 1907505 (0.0006) [2023-12-27 05:15:39,755][105692] Updated weights for policy 0, policy_version 1903017 (0.0011) [2023-12-27 05:15:39,809][105620] Updated weights for policy 1, policy_version 1907515 (0.0007) [2023-12-27 05:15:39,872][105620] Updated weights for policy 1, policy_version 1907525 (0.0008) [2023-12-27 05:15:39,924][105620] Updated weights for policy 1, policy_version 1907535 (0.0008) [2023-12-27 05:15:40,522][105692] Updated weights for policy 0, policy_version 1903027 (0.0011) [2023-12-27 05:15:40,589][105692] Updated weights for policy 0, policy_version 1903037 (0.0011) [2023-12-27 05:15:40,611][105620] Updated weights for policy 1, policy_version 1907545 (0.0009) [2023-12-27 05:15:40,649][105692] Updated weights for policy 0, policy_version 1903047 (0.0011) [2023-12-27 05:15:40,673][105620] Updated weights for policy 1, policy_version 1907555 (0.0011) [2023-12-27 05:15:40,725][105620] Updated weights for policy 1, policy_version 1907565 (0.0008) [2023-12-27 05:15:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19797.3, 300 sec: 19438.6). Total num frames: 975659008. Throughput: 0: 9861.7, 1: 9915.7. Samples: 975667392. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:15:41,063][104569] Avg episode reward: [(0, '8534.742'), (1, '9255.098')] [2023-12-27 05:15:41,438][105692] Updated weights for policy 0, policy_version 1903057 (0.0010) [2023-12-27 05:15:41,457][105620] Updated weights for policy 1, policy_version 1907575 (0.0008) [2023-12-27 05:15:41,500][105692] Updated weights for policy 0, policy_version 1903067 (0.0007) [2023-12-27 05:15:41,514][105620] Updated weights for policy 1, policy_version 1907585 (0.0007) [2023-12-27 05:15:41,551][105692] Updated weights for policy 0, policy_version 1903077 (0.0006) [2023-12-27 05:15:41,574][105620] Updated weights for policy 1, policy_version 1907595 (0.0008) [2023-12-27 05:15:41,598][105692] Updated weights for policy 0, policy_version 1903087 (0.0007) [2023-12-27 05:15:42,352][105692] Updated weights for policy 0, policy_version 1903097 (0.0009) [2023-12-27 05:15:42,401][105620] Updated weights for policy 1, policy_version 1907605 (0.0007) [2023-12-27 05:15:42,419][105692] Updated weights for policy 0, policy_version 1903107 (0.0007) [2023-12-27 05:15:42,463][105620] Updated weights for policy 1, policy_version 1907615 (0.0007) [2023-12-27 05:15:42,478][105692] Updated weights for policy 0, policy_version 1903117 (0.0008) [2023-12-27 05:15:42,513][105620] Updated weights for policy 1, policy_version 1907625 (0.0007) [2023-12-27 05:15:43,120][105692] Updated weights for policy 0, policy_version 1903127 (0.0007) [2023-12-27 05:15:43,187][105692] Updated weights for policy 0, policy_version 1903137 (0.0008) [2023-12-27 05:15:43,243][105692] Updated weights for policy 0, policy_version 1903147 (0.0011) [2023-12-27 05:15:43,345][105620] Updated weights for policy 1, policy_version 1907635 (0.0008) [2023-12-27 05:15:43,404][105620] Updated weights for policy 1, policy_version 1907645 (0.0008) [2023-12-27 05:15:43,452][105620] Updated weights for policy 1, policy_version 1907655 (0.0008) [2023-12-27 05:15:43,946][105692] Updated weights for policy 0, policy_version 1903157 (0.0010) [2023-12-27 05:15:44,003][105692] Updated weights for policy 0, policy_version 1903167 (0.0010) [2023-12-27 05:15:44,057][105692] Updated weights for policy 0, policy_version 1903177 (0.0010) [2023-12-27 05:15:44,120][105620] Updated weights for policy 1, policy_version 1907665 (0.0008) [2023-12-27 05:15:44,171][105620] Updated weights for policy 1, policy_version 1907675 (0.0005) [2023-12-27 05:15:44,237][105620] Updated weights for policy 1, policy_version 1907685 (0.0008) [2023-12-27 05:15:44,298][105620] Updated weights for policy 1, policy_version 1907695 (0.0008) [2023-12-27 05:15:44,819][105692] Updated weights for policy 0, policy_version 1903187 (0.0010) [2023-12-27 05:15:44,874][105692] Updated weights for policy 0, policy_version 1903197 (0.0010) [2023-12-27 05:15:44,937][105692] Updated weights for policy 0, policy_version 1903207 (0.0010) [2023-12-27 05:15:44,958][105620] Updated weights for policy 1, policy_version 1907705 (0.0009) [2023-12-27 05:15:45,015][105620] Updated weights for policy 1, policy_version 1907715 (0.0011) [2023-12-27 05:15:45,071][105620] Updated weights for policy 1, policy_version 1907725 (0.0011) [2023-12-27 05:15:45,685][105620] Updated weights for policy 1, policy_version 1907735 (0.0007) [2023-12-27 05:15:45,742][105620] Updated weights for policy 1, policy_version 1907745 (0.0005) [2023-12-27 05:15:45,765][105692] Updated weights for policy 0, policy_version 1903217 (0.0010) [2023-12-27 05:15:45,795][105620] Updated weights for policy 1, policy_version 1907755 (0.0005) [2023-12-27 05:15:45,813][105692] Updated weights for policy 0, policy_version 1903227 (0.0008) [2023-12-27 05:15:45,865][105692] Updated weights for policy 0, policy_version 1903237 (0.0009) [2023-12-27 05:15:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 975757312. Throughput: 0: 9877.2, 1: 9804.2. Samples: 975723336. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:15:46,063][104569] Avg episode reward: [(0, '8445.435'), (1, '9254.181')] [2023-12-27 05:15:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001907760_488456192.pth... [2023-12-27 05:15:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001903248_487301120.pth... [2023-12-27 05:15:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001906608_488161280.pth [2023-12-27 05:15:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001902096_487006208.pth [2023-12-27 05:15:46,339][105620] Updated weights for policy 1, policy_version 1907765 (0.0005) [2023-12-27 05:15:46,396][105620] Updated weights for policy 1, policy_version 1907775 (0.0005) [2023-12-27 05:15:46,464][105620] Updated weights for policy 1, policy_version 1907785 (0.0006) [2023-12-27 05:15:46,731][105692] Updated weights for policy 0, policy_version 1903249 (0.0010) [2023-12-27 05:15:46,796][105692] Updated weights for policy 0, policy_version 1903259 (0.0009) [2023-12-27 05:15:46,861][105692] Updated weights for policy 0, policy_version 1903269 (0.0009) [2023-12-27 05:15:46,922][105692] Updated weights for policy 0, policy_version 1903279 (0.0009) [2023-12-27 05:15:47,054][105620] Updated weights for policy 1, policy_version 1907795 (0.0005) [2023-12-27 05:15:47,100][105620] Updated weights for policy 1, policy_version 1907805 (0.0005) [2023-12-27 05:15:47,149][105620] Updated weights for policy 1, policy_version 1907815 (0.0005) [2023-12-27 05:15:47,685][105620] Updated weights for policy 1, policy_version 1907825 (0.0005) [2023-12-27 05:15:47,751][105620] Updated weights for policy 1, policy_version 1907835 (0.0008) [2023-12-27 05:15:47,800][105692] Updated weights for policy 0, policy_version 1903289 (0.0007) [2023-12-27 05:15:47,809][105620] Updated weights for policy 1, policy_version 1907845 (0.0010) [2023-12-27 05:15:47,858][105692] Updated weights for policy 0, policy_version 1903299 (0.0006) [2023-12-27 05:15:47,871][105620] Updated weights for policy 1, policy_version 1907855 (0.0010) [2023-12-27 05:15:47,923][105692] Updated weights for policy 0, policy_version 1903309 (0.0006) [2023-12-27 05:15:48,519][105620] Updated weights for policy 1, policy_version 1907865 (0.0009) [2023-12-27 05:15:48,583][105620] Updated weights for policy 1, policy_version 1907875 (0.0011) [2023-12-27 05:15:48,645][105620] Updated weights for policy 1, policy_version 1907885 (0.0008) [2023-12-27 05:15:48,663][105692] Updated weights for policy 0, policy_version 1903319 (0.0010) [2023-12-27 05:15:48,726][105692] Updated weights for policy 0, policy_version 1903329 (0.0007) [2023-12-27 05:15:48,777][105692] Updated weights for policy 0, policy_version 1903339 (0.0008) [2023-12-27 05:15:49,423][105620] Updated weights for policy 1, policy_version 1907895 (0.0007) [2023-12-27 05:15:49,462][105692] Updated weights for policy 0, policy_version 1903349 (0.0008) [2023-12-27 05:15:49,475][105620] Updated weights for policy 1, policy_version 1907905 (0.0007) [2023-12-27 05:15:49,513][105692] Updated weights for policy 0, policy_version 1903359 (0.0008) [2023-12-27 05:15:49,538][105620] Updated weights for policy 1, policy_version 1907915 (0.0006) [2023-12-27 05:15:49,570][105692] Updated weights for policy 0, policy_version 1903370 (0.0009) [2023-12-27 05:15:50,269][105620] Updated weights for policy 1, policy_version 1907925 (0.0006) [2023-12-27 05:15:50,322][105620] Updated weights for policy 1, policy_version 1907935 (0.0005) [2023-12-27 05:15:50,379][105692] Updated weights for policy 0, policy_version 1903380 (0.0009) [2023-12-27 05:15:50,384][105620] Updated weights for policy 1, policy_version 1907945 (0.0010) [2023-12-27 05:15:50,436][105692] Updated weights for policy 0, policy_version 1903390 (0.0009) [2023-12-27 05:15:50,490][105692] Updated weights for policy 0, policy_version 1903400 (0.0009) [2023-12-27 05:15:51,054][105620] Updated weights for policy 1, policy_version 1907955 (0.0008) [2023-12-27 05:15:51,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 975847424. Throughput: 0: 9787.1, 1: 9985.6. Samples: 975842064. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:15:51,063][104569] Avg episode reward: [(0, '8354.499'), (1, '9253.028')] [2023-12-27 05:15:51,119][105620] Updated weights for policy 1, policy_version 1907965 (0.0006) [2023-12-27 05:15:51,183][105620] Updated weights for policy 1, policy_version 1907975 (0.0010) [2023-12-27 05:15:51,333][105692] Updated weights for policy 0, policy_version 1903410 (0.0009) [2023-12-27 05:15:51,403][105692] Updated weights for policy 0, policy_version 1903420 (0.0009) [2023-12-27 05:15:51,470][105692] Updated weights for policy 0, policy_version 1903430 (0.0006) [2023-12-27 05:15:51,536][105692] Updated weights for policy 0, policy_version 1903440 (0.0006) [2023-12-27 05:15:51,893][105620] Updated weights for policy 1, policy_version 1907985 (0.0010) [2023-12-27 05:15:51,955][105620] Updated weights for policy 1, policy_version 1907995 (0.0005) [2023-12-27 05:15:52,017][105620] Updated weights for policy 1, policy_version 1908005 (0.0005) [2023-12-27 05:15:52,074][105620] Updated weights for policy 1, policy_version 1908015 (0.0005) [2023-12-27 05:15:52,256][105692] Updated weights for policy 0, policy_version 1903450 (0.0009) [2023-12-27 05:15:52,313][105692] Updated weights for policy 0, policy_version 1903460 (0.0009) [2023-12-27 05:15:52,372][105692] Updated weights for policy 0, policy_version 1903470 (0.0009) [2023-12-27 05:15:52,637][105620] Updated weights for policy 1, policy_version 1908025 (0.0005) [2023-12-27 05:15:52,692][105620] Updated weights for policy 1, policy_version 1908035 (0.0005) [2023-12-27 05:15:52,749][105620] Updated weights for policy 1, policy_version 1908045 (0.0005) [2023-12-27 05:15:53,047][105692] Updated weights for policy 0, policy_version 1903480 (0.0006) [2023-12-27 05:15:53,107][105692] Updated weights for policy 0, policy_version 1903490 (0.0010) [2023-12-27 05:15:53,163][105692] Updated weights for policy 0, policy_version 1903500 (0.0009) [2023-12-27 05:15:53,287][105620] Updated weights for policy 1, policy_version 1908055 (0.0009) [2023-12-27 05:15:53,335][105620] Updated weights for policy 1, policy_version 1908065 (0.0010) [2023-12-27 05:15:53,383][105620] Updated weights for policy 1, policy_version 1908075 (0.0009) [2023-12-27 05:15:53,859][105692] Updated weights for policy 0, policy_version 1903510 (0.0010) [2023-12-27 05:15:53,907][105692] Updated weights for policy 0, policy_version 1903520 (0.0006) [2023-12-27 05:15:53,959][105692] Updated weights for policy 0, policy_version 1903530 (0.0006) [2023-12-27 05:15:54,011][105620] Updated weights for policy 1, policy_version 1908085 (0.0009) [2023-12-27 05:15:54,064][105620] Updated weights for policy 1, policy_version 1908095 (0.0009) [2023-12-27 05:15:54,121][105620] Updated weights for policy 1, policy_version 1908105 (0.0009) [2023-12-27 05:15:54,648][105692] Updated weights for policy 0, policy_version 1903540 (0.0006) [2023-12-27 05:15:54,708][105692] Updated weights for policy 0, policy_version 1903550 (0.0009) [2023-12-27 05:15:54,775][105692] Updated weights for policy 0, policy_version 1903560 (0.0010) [2023-12-27 05:15:54,869][105620] Updated weights for policy 1, policy_version 1908115 (0.0008) [2023-12-27 05:15:54,917][105620] Updated weights for policy 1, policy_version 1908125 (0.0009) [2023-12-27 05:15:54,975][105620] Updated weights for policy 1, policy_version 1908135 (0.0009) [2023-12-27 05:15:55,468][105692] Updated weights for policy 0, policy_version 1903570 (0.0010) [2023-12-27 05:15:55,518][105692] Updated weights for policy 0, policy_version 1903580 (0.0009) [2023-12-27 05:15:55,567][105692] Updated weights for policy 0, policy_version 1903590 (0.0009) [2023-12-27 05:15:55,614][105692] Updated weights for policy 0, policy_version 1903600 (0.0009) [2023-12-27 05:15:55,721][105620] Updated weights for policy 1, policy_version 1908145 (0.0009) [2023-12-27 05:15:55,769][105620] Updated weights for policy 1, policy_version 1908155 (0.0005) [2023-12-27 05:15:55,815][105620] Updated weights for policy 1, policy_version 1908165 (0.0005) [2023-12-27 05:15:55,887][105620] Updated weights for policy 1, policy_version 1908175 (0.0005) [2023-12-27 05:15:56,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 975953920. Throughput: 0: 9740.7, 1: 10070.7. Samples: 975961600. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:15:56,063][104569] Avg episode reward: [(0, '8356.359'), (1, '9252.880')] [2023-12-27 05:15:56,327][105692] Updated weights for policy 0, policy_version 1903610 (0.0008) [2023-12-27 05:15:56,388][105692] Updated weights for policy 0, policy_version 1903620 (0.0008) [2023-12-27 05:15:56,441][105692] Updated weights for policy 0, policy_version 1903630 (0.0008) [2023-12-27 05:15:56,488][105620] Updated weights for policy 1, policy_version 1908185 (0.0008) [2023-12-27 05:15:56,540][105620] Updated weights for policy 1, policy_version 1908196 (0.0010) [2023-12-27 05:15:56,599][105620] Updated weights for policy 1, policy_version 1908207 (0.0010) [2023-12-27 05:15:57,042][105692] Updated weights for policy 0, policy_version 1903640 (0.0010) [2023-12-27 05:15:57,096][105692] Updated weights for policy 0, policy_version 1903650 (0.0010) [2023-12-27 05:15:57,154][105692] Updated weights for policy 0, policy_version 1903660 (0.0010) [2023-12-27 05:15:57,426][105620] Updated weights for policy 1, policy_version 1908217 (0.0007) [2023-12-27 05:15:57,476][105620] Updated weights for policy 1, policy_version 1908227 (0.0008) [2023-12-27 05:15:57,538][105620] Updated weights for policy 1, policy_version 1908237 (0.0007) [2023-12-27 05:15:57,895][105692] Updated weights for policy 0, policy_version 1903670 (0.0007) [2023-12-27 05:15:57,945][105692] Updated weights for policy 0, policy_version 1903680 (0.0005) [2023-12-27 05:15:57,991][105692] Updated weights for policy 0, policy_version 1903690 (0.0005) [2023-12-27 05:15:58,274][105620] Updated weights for policy 1, policy_version 1908247 (0.0007) [2023-12-27 05:15:58,335][105620] Updated weights for policy 1, policy_version 1908257 (0.0008) [2023-12-27 05:15:58,398][105620] Updated weights for policy 1, policy_version 1908267 (0.0008) [2023-12-27 05:15:58,650][105692] Updated weights for policy 0, policy_version 1903700 (0.0007) [2023-12-27 05:15:58,717][105692] Updated weights for policy 0, policy_version 1903710 (0.0007) [2023-12-27 05:15:58,785][105692] Updated weights for policy 0, policy_version 1903721 (0.0007) [2023-12-27 05:15:59,311][105620] Updated weights for policy 1, policy_version 1908277 (0.0009) [2023-12-27 05:15:59,385][105620] Updated weights for policy 1, policy_version 1908287 (0.0008) [2023-12-27 05:15:59,444][105620] Updated weights for policy 1, policy_version 1908297 (0.0010) [2023-12-27 05:15:59,601][105692] Updated weights for policy 0, policy_version 1903731 (0.0011) [2023-12-27 05:15:59,661][105692] Updated weights for policy 0, policy_version 1903741 (0.0011) [2023-12-27 05:15:59,720][105692] Updated weights for policy 0, policy_version 1903751 (0.0011) [2023-12-27 05:16:00,202][105620] Updated weights for policy 1, policy_version 1908307 (0.0010) [2023-12-27 05:16:00,257][105620] Updated weights for policy 1, policy_version 1908317 (0.0011) [2023-12-27 05:16:00,320][105620] Updated weights for policy 1, policy_version 1908327 (0.0010) [2023-12-27 05:16:00,363][105692] Updated weights for policy 0, policy_version 1903761 (0.0010) [2023-12-27 05:16:00,417][105692] Updated weights for policy 0, policy_version 1903771 (0.0008) [2023-12-27 05:16:00,477][105692] Updated weights for policy 0, policy_version 1903781 (0.0008) [2023-12-27 05:16:00,536][105692] Updated weights for policy 0, policy_version 1903791 (0.0008) [2023-12-27 05:16:00,922][105620] Updated weights for policy 1, policy_version 1908337 (0.0010) [2023-12-27 05:16:00,988][105620] Updated weights for policy 1, policy_version 1908347 (0.0005) [2023-12-27 05:16:01,056][105620] Updated weights for policy 1, policy_version 1908357 (0.0006) [2023-12-27 05:16:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 976044032. Throughput: 0: 9788.3, 1: 9953.8. Samples: 976020280. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:16:01,063][104569] Avg episode reward: [(0, '8357.597'), (1, '9160.491')] [2023-12-27 05:16:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001903792_487440384.pth... [2023-12-27 05:16:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001902672_487153664.pth [2023-12-27 05:16:01,125][105620] Updated weights for policy 1, policy_version 1908367 (0.0009) [2023-12-27 05:16:01,132][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001908368_488611840.pth... [2023-12-27 05:16:01,137][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001907216_488316928.pth [2023-12-27 05:16:01,303][105692] Updated weights for policy 0, policy_version 1903801 (0.0010) [2023-12-27 05:16:01,369][105692] Updated weights for policy 0, policy_version 1903811 (0.0011) [2023-12-27 05:16:01,433][105692] Updated weights for policy 0, policy_version 1903821 (0.0009) [2023-12-27 05:16:01,779][105620] Updated weights for policy 1, policy_version 1908377 (0.0010) [2023-12-27 05:16:01,830][105620] Updated weights for policy 1, policy_version 1908387 (0.0010) [2023-12-27 05:16:01,877][105620] Updated weights for policy 1, policy_version 1908397 (0.0010) [2023-12-27 05:16:02,206][105692] Updated weights for policy 0, policy_version 1903831 (0.0007) [2023-12-27 05:16:02,269][105692] Updated weights for policy 0, policy_version 1903841 (0.0008) [2023-12-27 05:16:02,323][105692] Updated weights for policy 0, policy_version 1903851 (0.0008) [2023-12-27 05:16:02,615][105620] Updated weights for policy 1, policy_version 1908407 (0.0009) [2023-12-27 05:16:02,663][105620] Updated weights for policy 1, policy_version 1908417 (0.0010) [2023-12-27 05:16:02,718][105620] Updated weights for policy 1, policy_version 1908427 (0.0010) [2023-12-27 05:16:02,952][105692] Updated weights for policy 0, policy_version 1903861 (0.0008) [2023-12-27 05:16:02,999][105692] Updated weights for policy 0, policy_version 1903871 (0.0007) [2023-12-27 05:16:03,054][105692] Updated weights for policy 0, policy_version 1903881 (0.0009) [2023-12-27 05:16:03,440][105620] Updated weights for policy 1, policy_version 1908437 (0.0010) [2023-12-27 05:16:03,498][105620] Updated weights for policy 1, policy_version 1908447 (0.0010) [2023-12-27 05:16:03,563][105620] Updated weights for policy 1, policy_version 1908457 (0.0010) [2023-12-27 05:16:03,705][105692] Updated weights for policy 0, policy_version 1903891 (0.0008) [2023-12-27 05:16:03,765][105692] Updated weights for policy 0, policy_version 1903901 (0.0005) [2023-12-27 05:16:03,822][105692] Updated weights for policy 0, policy_version 1903911 (0.0006) [2023-12-27 05:16:04,352][105620] Updated weights for policy 1, policy_version 1908467 (0.0009) [2023-12-27 05:16:04,417][105620] Updated weights for policy 1, policy_version 1908477 (0.0009) [2023-12-27 05:16:04,474][105620] Updated weights for policy 1, policy_version 1908487 (0.0011) [2023-12-27 05:16:04,548][105692] Updated weights for policy 0, policy_version 1903921 (0.0008) [2023-12-27 05:16:04,604][105692] Updated weights for policy 0, policy_version 1903931 (0.0011) [2023-12-27 05:16:04,663][105692] Updated weights for policy 0, policy_version 1903941 (0.0010) [2023-12-27 05:16:04,727][105692] Updated weights for policy 0, policy_version 1903951 (0.0010) [2023-12-27 05:16:05,215][105620] Updated weights for policy 1, policy_version 1908497 (0.0011) [2023-12-27 05:16:05,269][105620] Updated weights for policy 1, policy_version 1908507 (0.0010) [2023-12-27 05:16:05,320][105620] Updated weights for policy 1, policy_version 1908517 (0.0010) [2023-12-27 05:16:05,368][105620] Updated weights for policy 1, policy_version 1908527 (0.0010) [2023-12-27 05:16:05,438][105692] Updated weights for policy 0, policy_version 1903961 (0.0010) [2023-12-27 05:16:05,490][105692] Updated weights for policy 0, policy_version 1903971 (0.0010) [2023-12-27 05:16:05,532][105692] Updated weights for policy 0, policy_version 1903981 (0.0009) [2023-12-27 05:16:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.9, 300 sec: 19466.4). Total num frames: 976142336. Throughput: 0: 9759.9, 1: 9845.6. Samples: 976136244. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:16:06,062][104569] Avg episode reward: [(0, '8262.606'), (1, '9160.799')] [2023-12-27 05:16:06,089][105620] Updated weights for policy 1, policy_version 1908537 (0.0006) [2023-12-27 05:16:06,159][105620] Updated weights for policy 1, policy_version 1908547 (0.0008) [2023-12-27 05:16:06,225][105620] Updated weights for policy 1, policy_version 1908557 (0.0010) [2023-12-27 05:16:06,326][105692] Updated weights for policy 0, policy_version 1903991 (0.0010) [2023-12-27 05:16:06,389][105692] Updated weights for policy 0, policy_version 1904001 (0.0011) [2023-12-27 05:16:06,453][105692] Updated weights for policy 0, policy_version 1904011 (0.0011) [2023-12-27 05:16:06,888][105620] Updated weights for policy 1, policy_version 1908567 (0.0009) [2023-12-27 05:16:06,951][105620] Updated weights for policy 1, policy_version 1908577 (0.0011) [2023-12-27 05:16:07,001][105620] Updated weights for policy 1, policy_version 1908587 (0.0011) [2023-12-27 05:16:07,118][105692] Updated weights for policy 0, policy_version 1904021 (0.0011) [2023-12-27 05:16:07,177][105692] Updated weights for policy 0, policy_version 1904031 (0.0011) [2023-12-27 05:16:07,226][105692] Updated weights for policy 0, policy_version 1904041 (0.0011) [2023-12-27 05:16:07,618][105620] Updated weights for policy 1, policy_version 1908597 (0.0010) [2023-12-27 05:16:07,677][105620] Updated weights for policy 1, policy_version 1908607 (0.0011) [2023-12-27 05:16:07,736][105620] Updated weights for policy 1, policy_version 1908617 (0.0010) [2023-12-27 05:16:07,987][105692] Updated weights for policy 0, policy_version 1904051 (0.0011) [2023-12-27 05:16:08,046][105692] Updated weights for policy 0, policy_version 1904061 (0.0011) [2023-12-27 05:16:08,109][105692] Updated weights for policy 0, policy_version 1904071 (0.0011) [2023-12-27 05:16:08,524][105620] Updated weights for policy 1, policy_version 1908627 (0.0010) [2023-12-27 05:16:08,575][105620] Updated weights for policy 1, policy_version 1908637 (0.0009) [2023-12-27 05:16:08,637][105620] Updated weights for policy 1, policy_version 1908647 (0.0008) [2023-12-27 05:16:08,846][105692] Updated weights for policy 0, policy_version 1904081 (0.0010) [2023-12-27 05:16:08,906][105692] Updated weights for policy 0, policy_version 1904091 (0.0011) [2023-12-27 05:16:08,960][105692] Updated weights for policy 0, policy_version 1904101 (0.0011) [2023-12-27 05:16:09,021][105692] Updated weights for policy 0, policy_version 1904111 (0.0011) [2023-12-27 05:16:09,447][105620] Updated weights for policy 1, policy_version 1908657 (0.0011) [2023-12-27 05:16:09,512][105620] Updated weights for policy 1, policy_version 1908667 (0.0008) [2023-12-27 05:16:09,570][105620] Updated weights for policy 1, policy_version 1908677 (0.0009) [2023-12-27 05:16:09,638][105620] Updated weights for policy 1, policy_version 1908687 (0.0008) [2023-12-27 05:16:09,836][105692] Updated weights for policy 0, policy_version 1904121 (0.0011) [2023-12-27 05:16:09,900][105692] Updated weights for policy 0, policy_version 1904131 (0.0011) [2023-12-27 05:16:09,970][105692] Updated weights for policy 0, policy_version 1904141 (0.0012) [2023-12-27 05:16:10,441][105620] Updated weights for policy 1, policy_version 1908697 (0.0008) [2023-12-27 05:16:10,512][105620] Updated weights for policy 1, policy_version 1908707 (0.0009) [2023-12-27 05:16:10,568][105620] Updated weights for policy 1, policy_version 1908717 (0.0008) [2023-12-27 05:16:10,738][105692] Updated weights for policy 0, policy_version 1904151 (0.0009) [2023-12-27 05:16:10,790][105692] Updated weights for policy 0, policy_version 1904161 (0.0009) [2023-12-27 05:16:10,837][105692] Updated weights for policy 0, policy_version 1904171 (0.0009) [2023-12-27 05:16:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 976240640. Throughput: 0: 9675.7, 1: 9769.9. Samples: 976248844. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:16:11,063][104569] Avg episode reward: [(0, '7902.157'), (1, '9068.619')] [2023-12-27 05:16:11,348][105620] Updated weights for policy 1, policy_version 1908727 (0.0007) [2023-12-27 05:16:11,420][105620] Updated weights for policy 1, policy_version 1908737 (0.0007) [2023-12-27 05:16:11,479][105620] Updated weights for policy 1, policy_version 1908747 (0.0006) [2023-12-27 05:16:11,663][105692] Updated weights for policy 0, policy_version 1904181 (0.0008) [2023-12-27 05:16:11,728][105692] Updated weights for policy 0, policy_version 1904191 (0.0009) [2023-12-27 05:16:11,795][105692] Updated weights for policy 0, policy_version 1904201 (0.0007) [2023-12-27 05:16:12,145][105620] Updated weights for policy 1, policy_version 1908757 (0.0006) [2023-12-27 05:16:12,207][105620] Updated weights for policy 1, policy_version 1908767 (0.0008) [2023-12-27 05:16:12,272][105620] Updated weights for policy 1, policy_version 1908777 (0.0009) [2023-12-27 05:16:12,604][105692] Updated weights for policy 0, policy_version 1904211 (0.0007) [2023-12-27 05:16:12,668][105692] Updated weights for policy 0, policy_version 1904221 (0.0008) [2023-12-27 05:16:12,727][105692] Updated weights for policy 0, policy_version 1904231 (0.0007) [2023-12-27 05:16:13,022][105620] Updated weights for policy 1, policy_version 1908787 (0.0009) [2023-12-27 05:16:13,077][105620] Updated weights for policy 1, policy_version 1908797 (0.0009) [2023-12-27 05:16:13,131][105620] Updated weights for policy 1, policy_version 1908807 (0.0007) [2023-12-27 05:16:13,510][105692] Updated weights for policy 0, policy_version 1904241 (0.0009) [2023-12-27 05:16:13,570][105692] Updated weights for policy 0, policy_version 1904251 (0.0009) [2023-12-27 05:16:13,638][105692] Updated weights for policy 0, policy_version 1904261 (0.0009) [2023-12-27 05:16:13,707][105692] Updated weights for policy 0, policy_version 1904271 (0.0009) [2023-12-27 05:16:13,782][105620] Updated weights for policy 1, policy_version 1908817 (0.0005) [2023-12-27 05:16:13,833][105620] Updated weights for policy 1, policy_version 1908827 (0.0005) [2023-12-27 05:16:13,879][105620] Updated weights for policy 1, policy_version 1908837 (0.0005) [2023-12-27 05:16:13,941][105620] Updated weights for policy 1, policy_version 1908847 (0.0006) [2023-12-27 05:16:14,507][105620] Updated weights for policy 1, policy_version 1908857 (0.0009) [2023-12-27 05:16:14,561][105692] Updated weights for policy 0, policy_version 1904281 (0.0007) [2023-12-27 05:16:14,575][105620] Updated weights for policy 1, policy_version 1908867 (0.0008) [2023-12-27 05:16:14,629][105692] Updated weights for policy 0, policy_version 1904291 (0.0008) [2023-12-27 05:16:14,632][105620] Updated weights for policy 1, policy_version 1908877 (0.0006) [2023-12-27 05:16:14,692][105692] Updated weights for policy 0, policy_version 1904301 (0.0008) [2023-12-27 05:16:15,390][105692] Updated weights for policy 0, policy_version 1904311 (0.0009) [2023-12-27 05:16:15,438][105620] Updated weights for policy 1, policy_version 1908887 (0.0007) [2023-12-27 05:16:15,449][105692] Updated weights for policy 0, policy_version 1904321 (0.0007) [2023-12-27 05:16:15,499][105620] Updated weights for policy 1, policy_version 1908897 (0.0010) [2023-12-27 05:16:15,510][105692] Updated weights for policy 0, policy_version 1904331 (0.0006) [2023-12-27 05:16:15,561][105620] Updated weights for policy 1, policy_version 1908907 (0.0009) [2023-12-27 05:16:16,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 976330752. Throughput: 0: 9580.0, 1: 9697.7. Samples: 976304604. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:16:16,063][104569] Avg episode reward: [(0, '8173.589'), (1, '9253.219')] [2023-12-27 05:16:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001908912_488751104.pth... [2023-12-27 05:16:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001904336_487579648.pth... [2023-12-27 05:16:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001907760_488456192.pth [2023-12-27 05:16:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001903248_487301120.pth [2023-12-27 05:16:16,273][105692] Updated weights for policy 0, policy_version 1904341 (0.0007) [2023-12-27 05:16:16,315][105620] Updated weights for policy 1, policy_version 1908917 (0.0008) [2023-12-27 05:16:16,333][105692] Updated weights for policy 0, policy_version 1904351 (0.0009) [2023-12-27 05:16:16,375][105620] Updated weights for policy 1, policy_version 1908927 (0.0008) [2023-12-27 05:16:16,394][105692] Updated weights for policy 0, policy_version 1904361 (0.0006) [2023-12-27 05:16:16,436][105620] Updated weights for policy 1, policy_version 1908937 (0.0007) [2023-12-27 05:16:17,111][105620] Updated weights for policy 1, policy_version 1908947 (0.0009) [2023-12-27 05:16:17,151][105692] Updated weights for policy 0, policy_version 1904371 (0.0008) [2023-12-27 05:16:17,174][105620] Updated weights for policy 1, policy_version 1908957 (0.0009) [2023-12-27 05:16:17,208][105692] Updated weights for policy 0, policy_version 1904381 (0.0007) [2023-12-27 05:16:17,233][105620] Updated weights for policy 1, policy_version 1908967 (0.0006) [2023-12-27 05:16:17,260][105692] Updated weights for policy 0, policy_version 1904391 (0.0007) [2023-12-27 05:16:17,951][105692] Updated weights for policy 0, policy_version 1904401 (0.0007) [2023-12-27 05:16:17,963][105620] Updated weights for policy 1, policy_version 1908977 (0.0007) [2023-12-27 05:16:18,014][105692] Updated weights for policy 0, policy_version 1904411 (0.0006) [2023-12-27 05:16:18,028][105620] Updated weights for policy 1, policy_version 1908987 (0.0010) [2023-12-27 05:16:18,067][105692] Updated weights for policy 0, policy_version 1904421 (0.0005) [2023-12-27 05:16:18,077][105620] Updated weights for policy 1, policy_version 1908997 (0.0010) [2023-12-27 05:16:18,114][105692] Updated weights for policy 0, policy_version 1904431 (0.0005) [2023-12-27 05:16:18,131][105620] Updated weights for policy 1, policy_version 1909007 (0.0010) [2023-12-27 05:16:18,792][105692] Updated weights for policy 0, policy_version 1904441 (0.0007) [2023-12-27 05:16:18,852][105692] Updated weights for policy 0, policy_version 1904451 (0.0010) [2023-12-27 05:16:18,885][105620] Updated weights for policy 1, policy_version 1909017 (0.0011) [2023-12-27 05:16:18,908][105692] Updated weights for policy 0, policy_version 1904461 (0.0010) [2023-12-27 05:16:18,934][105620] Updated weights for policy 1, policy_version 1909027 (0.0011) [2023-12-27 05:16:18,999][105620] Updated weights for policy 1, policy_version 1909037 (0.0010) [2023-12-27 05:16:19,613][105692] Updated weights for policy 0, policy_version 1904471 (0.0007) [2023-12-27 05:16:19,678][105692] Updated weights for policy 0, policy_version 1904481 (0.0007) [2023-12-27 05:16:19,740][105692] Updated weights for policy 0, policy_version 1904491 (0.0006) [2023-12-27 05:16:19,757][105620] Updated weights for policy 1, policy_version 1909047 (0.0009) [2023-12-27 05:16:19,819][105620] Updated weights for policy 1, policy_version 1909057 (0.0009) [2023-12-27 05:16:19,879][105620] Updated weights for policy 1, policy_version 1909067 (0.0009) [2023-12-27 05:16:20,538][105620] Updated weights for policy 1, policy_version 1909077 (0.0008) [2023-12-27 05:16:20,545][105692] Updated weights for policy 0, policy_version 1904501 (0.0009) [2023-12-27 05:16:20,602][105620] Updated weights for policy 1, policy_version 1909087 (0.0007) [2023-12-27 05:16:20,614][105692] Updated weights for policy 0, policy_version 1904511 (0.0008) [2023-12-27 05:16:20,657][105620] Updated weights for policy 1, policy_version 1909097 (0.0007) [2023-12-27 05:16:20,666][105692] Updated weights for policy 0, policy_version 1904521 (0.0007) [2023-12-27 05:16:21,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 976429056. Throughput: 0: 9483.3, 1: 9736.0. Samples: 976417888. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:16:21,063][104569] Avg episode reward: [(0, '8350.149'), (1, '9161.019')] [2023-12-27 05:16:21,255][105620] Updated weights for policy 1, policy_version 1909107 (0.0007) [2023-12-27 05:16:21,324][105620] Updated weights for policy 1, policy_version 1909117 (0.0008) [2023-12-27 05:16:21,398][105620] Updated weights for policy 1, policy_version 1909127 (0.0007) [2023-12-27 05:16:21,544][105692] Updated weights for policy 0, policy_version 1904531 (0.0008) [2023-12-27 05:16:21,606][105692] Updated weights for policy 0, policy_version 1904541 (0.0009) [2023-12-27 05:16:21,678][105692] Updated weights for policy 0, policy_version 1904551 (0.0009) [2023-12-27 05:16:22,125][105620] Updated weights for policy 1, policy_version 1909137 (0.0006) [2023-12-27 05:16:22,191][105620] Updated weights for policy 1, policy_version 1909147 (0.0009) [2023-12-27 05:16:22,253][105620] Updated weights for policy 1, policy_version 1909157 (0.0009) [2023-12-27 05:16:22,319][105620] Updated weights for policy 1, policy_version 1909167 (0.0009) [2023-12-27 05:16:22,440][105692] Updated weights for policy 0, policy_version 1904561 (0.0009) [2023-12-27 05:16:22,488][105692] Updated weights for policy 0, policy_version 1904571 (0.0008) [2023-12-27 05:16:22,548][105692] Updated weights for policy 0, policy_version 1904581 (0.0009) [2023-12-27 05:16:22,608][105692] Updated weights for policy 0, policy_version 1904591 (0.0009) [2023-12-27 05:16:23,071][105620] Updated weights for policy 1, policy_version 1909177 (0.0008) [2023-12-27 05:16:23,130][105620] Updated weights for policy 1, policy_version 1909187 (0.0006) [2023-12-27 05:16:23,197][105620] Updated weights for policy 1, policy_version 1909197 (0.0005) [2023-12-27 05:16:23,368][105692] Updated weights for policy 0, policy_version 1904601 (0.0009) [2023-12-27 05:16:23,427][105692] Updated weights for policy 0, policy_version 1904611 (0.0009) [2023-12-27 05:16:23,486][105692] Updated weights for policy 0, policy_version 1904621 (0.0009) [2023-12-27 05:16:23,849][105620] Updated weights for policy 1, policy_version 1909207 (0.0008) [2023-12-27 05:16:23,896][105620] Updated weights for policy 1, policy_version 1909217 (0.0008) [2023-12-27 05:16:23,943][105620] Updated weights for policy 1, policy_version 1909227 (0.0009) [2023-12-27 05:16:24,282][105692] Updated weights for policy 0, policy_version 1904631 (0.0009) [2023-12-27 05:16:24,341][105692] Updated weights for policy 0, policy_version 1904641 (0.0009) [2023-12-27 05:16:24,403][105692] Updated weights for policy 0, policy_version 1904651 (0.0009) [2023-12-27 05:16:24,750][105620] Updated weights for policy 1, policy_version 1909237 (0.0008) [2023-12-27 05:16:24,811][105620] Updated weights for policy 1, policy_version 1909247 (0.0009) [2023-12-27 05:16:24,874][105620] Updated weights for policy 1, policy_version 1909257 (0.0009) [2023-12-27 05:16:25,058][105692] Updated weights for policy 0, policy_version 1904661 (0.0009) [2023-12-27 05:16:25,121][105692] Updated weights for policy 0, policy_version 1904671 (0.0010) [2023-12-27 05:16:25,189][105692] Updated weights for policy 0, policy_version 1904681 (0.0010) [2023-12-27 05:16:25,507][105620] Updated weights for policy 1, policy_version 1909267 (0.0008) [2023-12-27 05:16:25,577][105620] Updated weights for policy 1, policy_version 1909277 (0.0005) [2023-12-27 05:16:25,646][105620] Updated weights for policy 1, policy_version 1909287 (0.0005) [2023-12-27 05:16:25,941][105692] Updated weights for policy 0, policy_version 1904691 (0.0009) [2023-12-27 05:16:25,994][105692] Updated weights for policy 0, policy_version 1904701 (0.0005) [2023-12-27 05:16:26,053][105692] Updated weights for policy 0, policy_version 1904711 (0.0008) [2023-12-27 05:16:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 976519168. Throughput: 0: 9361.0, 1: 9862.7. Samples: 976532464. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:16:26,063][104569] Avg episode reward: [(0, '8262.874'), (1, '9161.062')] [2023-12-27 05:16:26,200][105620] Updated weights for policy 1, policy_version 1909297 (0.0006) [2023-12-27 05:16:26,247][105620] Updated weights for policy 1, policy_version 1909307 (0.0009) [2023-12-27 05:16:26,302][105620] Updated weights for policy 1, policy_version 1909317 (0.0009) [2023-12-27 05:16:26,351][105620] Updated weights for policy 1, policy_version 1909327 (0.0009) [2023-12-27 05:16:26,655][105692] Updated weights for policy 0, policy_version 1904721 (0.0009) [2023-12-27 05:16:26,716][105692] Updated weights for policy 0, policy_version 1904731 (0.0009) [2023-12-27 05:16:26,774][105692] Updated weights for policy 0, policy_version 1904741 (0.0009) [2023-12-27 05:16:26,825][105692] Updated weights for policy 0, policy_version 1904751 (0.0009) [2023-12-27 05:16:27,192][105620] Updated weights for policy 1, policy_version 1909337 (0.0009) [2023-12-27 05:16:27,246][105620] Updated weights for policy 1, policy_version 1909347 (0.0010) [2023-12-27 05:16:27,300][105620] Updated weights for policy 1, policy_version 1909357 (0.0010) [2023-12-27 05:16:27,426][105692] Updated weights for policy 0, policy_version 1904761 (0.0009) [2023-12-27 05:16:27,484][105692] Updated weights for policy 0, policy_version 1904771 (0.0009) [2023-12-27 05:16:27,545][105692] Updated weights for policy 0, policy_version 1904781 (0.0009) [2023-12-27 05:16:28,094][105620] Updated weights for policy 1, policy_version 1909367 (0.0009) [2023-12-27 05:16:28,149][105620] Updated weights for policy 1, policy_version 1909377 (0.0010) [2023-12-27 05:16:28,205][105620] Updated weights for policy 1, policy_version 1909387 (0.0008) [2023-12-27 05:16:28,224][105692] Updated weights for policy 0, policy_version 1904791 (0.0009) [2023-12-27 05:16:28,269][105692] Updated weights for policy 0, policy_version 1904801 (0.0007) [2023-12-27 05:16:28,315][105692] Updated weights for policy 0, policy_version 1904811 (0.0009) [2023-12-27 05:16:29,003][105620] Updated weights for policy 1, policy_version 1909397 (0.0009) [2023-12-27 05:16:29,055][105620] Updated weights for policy 1, policy_version 1909407 (0.0006) [2023-12-27 05:16:29,100][105692] Updated weights for policy 0, policy_version 1904821 (0.0010) [2023-12-27 05:16:29,106][105620] Updated weights for policy 1, policy_version 1909417 (0.0006) [2023-12-27 05:16:29,145][105692] Updated weights for policy 0, policy_version 1904831 (0.0010) [2023-12-27 05:16:29,197][105692] Updated weights for policy 0, policy_version 1904841 (0.0010) [2023-12-27 05:16:29,844][105620] Updated weights for policy 1, policy_version 1909427 (0.0007) [2023-12-27 05:16:29,905][105692] Updated weights for policy 0, policy_version 1904851 (0.0010) [2023-12-27 05:16:29,913][105620] Updated weights for policy 1, policy_version 1909437 (0.0008) [2023-12-27 05:16:29,961][105692] Updated weights for policy 0, policy_version 1904861 (0.0007) [2023-12-27 05:16:29,975][105620] Updated weights for policy 1, policy_version 1909447 (0.0008) [2023-12-27 05:16:30,019][105692] Updated weights for policy 0, policy_version 1904871 (0.0006) [2023-12-27 05:16:30,662][105620] Updated weights for policy 1, policy_version 1909457 (0.0008) [2023-12-27 05:16:30,715][105620] Updated weights for policy 1, policy_version 1909467 (0.0007) [2023-12-27 05:16:30,738][105692] Updated weights for policy 0, policy_version 1904881 (0.0006) [2023-12-27 05:16:30,767][105620] Updated weights for policy 1, policy_version 1909477 (0.0005) [2023-12-27 05:16:30,787][105692] Updated weights for policy 0, policy_version 1904891 (0.0009) [2023-12-27 05:16:30,814][105620] Updated weights for policy 1, policy_version 1909487 (0.0007) [2023-12-27 05:16:30,837][105692] Updated weights for policy 0, policy_version 1904901 (0.0008) [2023-12-27 05:16:30,898][105692] Updated weights for policy 0, policy_version 1904911 (0.0008) [2023-12-27 05:16:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 976625664. Throughput: 0: 9424.6, 1: 9857.4. Samples: 976591024. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:16:31,062][104569] Avg episode reward: [(0, '8446.860'), (1, '9255.424')] [2023-12-27 05:16:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001904912_487727104.pth... [2023-12-27 05:16:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001909488_488898560.pth... [2023-12-27 05:16:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001903792_487440384.pth [2023-12-27 05:16:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001908368_488611840.pth [2023-12-27 05:16:31,495][105620] Updated weights for policy 1, policy_version 1909497 (0.0005) [2023-12-27 05:16:31,564][105620] Updated weights for policy 1, policy_version 1909507 (0.0005) [2023-12-27 05:16:31,620][105620] Updated weights for policy 1, policy_version 1909517 (0.0006) [2023-12-27 05:16:31,728][105692] Updated weights for policy 0, policy_version 1904921 (0.0008) [2023-12-27 05:16:31,789][105692] Updated weights for policy 0, policy_version 1904931 (0.0006) [2023-12-27 05:16:31,856][105692] Updated weights for policy 0, policy_version 1904941 (0.0005) [2023-12-27 05:16:32,381][105620] Updated weights for policy 1, policy_version 1909527 (0.0008) [2023-12-27 05:16:32,415][105692] Updated weights for policy 0, policy_version 1904951 (0.0008) [2023-12-27 05:16:32,445][105620] Updated weights for policy 1, policy_version 1909537 (0.0007) [2023-12-27 05:16:32,467][105692] Updated weights for policy 0, policy_version 1904961 (0.0007) [2023-12-27 05:16:32,506][105620] Updated weights for policy 1, policy_version 1909547 (0.0010) [2023-12-27 05:16:32,527][105692] Updated weights for policy 0, policy_version 1904971 (0.0006) [2023-12-27 05:16:33,194][105692] Updated weights for policy 0, policy_version 1904981 (0.0007) [2023-12-27 05:16:33,253][105692] Updated weights for policy 0, policy_version 1904991 (0.0008) [2023-12-27 05:16:33,264][105620] Updated weights for policy 1, policy_version 1909557 (0.0009) [2023-12-27 05:16:33,313][105620] Updated weights for policy 1, policy_version 1909567 (0.0007) [2023-12-27 05:16:33,314][105692] Updated weights for policy 0, policy_version 1905001 (0.0008) [2023-12-27 05:16:33,366][105620] Updated weights for policy 1, policy_version 1909577 (0.0009) [2023-12-27 05:16:33,919][105692] Updated weights for policy 0, policy_version 1905011 (0.0005) [2023-12-27 05:16:33,988][105692] Updated weights for policy 0, policy_version 1905021 (0.0006) [2023-12-27 05:16:34,037][105692] Updated weights for policy 0, policy_version 1905031 (0.0005) [2023-12-27 05:16:34,221][105620] Updated weights for policy 1, policy_version 1909588 (0.0009) [2023-12-27 05:16:34,286][105620] Updated weights for policy 1, policy_version 1909598 (0.0008) [2023-12-27 05:16:34,344][105620] Updated weights for policy 1, policy_version 1909608 (0.0008) [2023-12-27 05:16:34,724][105692] Updated weights for policy 0, policy_version 1905041 (0.0006) [2023-12-27 05:16:34,782][105692] Updated weights for policy 0, policy_version 1905051 (0.0011) [2023-12-27 05:16:34,841][105692] Updated weights for policy 0, policy_version 1905061 (0.0010) [2023-12-27 05:16:34,904][105692] Updated weights for policy 0, policy_version 1905071 (0.0011) [2023-12-27 05:16:35,134][105620] Updated weights for policy 1, policy_version 1909618 (0.0008) [2023-12-27 05:16:35,191][105620] Updated weights for policy 1, policy_version 1909628 (0.0008) [2023-12-27 05:16:35,250][105620] Updated weights for policy 1, policy_version 1909638 (0.0008) [2023-12-27 05:16:35,305][105620] Updated weights for policy 1, policy_version 1909648 (0.0008) [2023-12-27 05:16:35,647][105692] Updated weights for policy 0, policy_version 1905081 (0.0011) [2023-12-27 05:16:35,699][105692] Updated weights for policy 0, policy_version 1905091 (0.0010) [2023-12-27 05:16:35,758][105692] Updated weights for policy 0, policy_version 1905101 (0.0011) [2023-12-27 05:16:36,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.1, 300 sec: 19438.6). Total num frames: 976715776. Throughput: 0: 9588.7, 1: 9654.3. Samples: 976708008. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:16:36,063][104569] Avg episode reward: [(0, '8354.890'), (1, '9164.789')] [2023-12-27 05:16:36,081][105620] Updated weights for policy 1, policy_version 1909658 (0.0005) [2023-12-27 05:16:36,142][105620] Updated weights for policy 1, policy_version 1909668 (0.0008) [2023-12-27 05:16:36,200][105620] Updated weights for policy 1, policy_version 1909678 (0.0007) [2023-12-27 05:16:36,461][105692] Updated weights for policy 0, policy_version 1905111 (0.0011) [2023-12-27 05:16:36,524][105692] Updated weights for policy 0, policy_version 1905121 (0.0010) [2023-12-27 05:16:36,597][105692] Updated weights for policy 0, policy_version 1905131 (0.0010) [2023-12-27 05:16:36,897][105620] Updated weights for policy 1, policy_version 1909688 (0.0007) [2023-12-27 05:16:36,959][105620] Updated weights for policy 1, policy_version 1909698 (0.0007) [2023-12-27 05:16:37,026][105620] Updated weights for policy 1, policy_version 1909708 (0.0008) [2023-12-27 05:16:37,326][105692] Updated weights for policy 0, policy_version 1905141 (0.0011) [2023-12-27 05:16:37,381][105692] Updated weights for policy 0, policy_version 1905151 (0.0010) [2023-12-27 05:16:37,441][105692] Updated weights for policy 0, policy_version 1905161 (0.0009) [2023-12-27 05:16:37,711][105620] Updated weights for policy 1, policy_version 1909718 (0.0008) [2023-12-27 05:16:37,759][105620] Updated weights for policy 1, policy_version 1909728 (0.0008) [2023-12-27 05:16:37,811][105620] Updated weights for policy 1, policy_version 1909738 (0.0008) [2023-12-27 05:16:38,202][105692] Updated weights for policy 0, policy_version 1905171 (0.0010) [2023-12-27 05:16:38,258][105692] Updated weights for policy 0, policy_version 1905181 (0.0010) [2023-12-27 05:16:38,312][105692] Updated weights for policy 0, policy_version 1905191 (0.0009) [2023-12-27 05:16:38,498][105620] Updated weights for policy 1, policy_version 1909748 (0.0008) [2023-12-27 05:16:38,554][105620] Updated weights for policy 1, policy_version 1909758 (0.0008) [2023-12-27 05:16:38,610][105620] Updated weights for policy 1, policy_version 1909768 (0.0008) [2023-12-27 05:16:39,007][105692] Updated weights for policy 0, policy_version 1905201 (0.0008) [2023-12-27 05:16:39,073][105692] Updated weights for policy 0, policy_version 1905211 (0.0006) [2023-12-27 05:16:39,138][105692] Updated weights for policy 0, policy_version 1905221 (0.0006) [2023-12-27 05:16:39,186][105692] Updated weights for policy 0, policy_version 1905231 (0.0008) [2023-12-27 05:16:39,414][105620] Updated weights for policy 1, policy_version 1909778 (0.0008) [2023-12-27 05:16:39,479][105620] Updated weights for policy 1, policy_version 1909788 (0.0009) [2023-12-27 05:16:39,537][105620] Updated weights for policy 1, policy_version 1909798 (0.0009) [2023-12-27 05:16:39,595][105620] Updated weights for policy 1, policy_version 1909808 (0.0009) [2023-12-27 05:16:39,925][105692] Updated weights for policy 0, policy_version 1905241 (0.0009) [2023-12-27 05:16:39,996][105692] Updated weights for policy 0, policy_version 1905251 (0.0009) [2023-12-27 05:16:40,065][105692] Updated weights for policy 0, policy_version 1905261 (0.0010) [2023-12-27 05:16:40,297][105620] Updated weights for policy 1, policy_version 1909818 (0.0006) [2023-12-27 05:16:40,362][105620] Updated weights for policy 1, policy_version 1909828 (0.0007) [2023-12-27 05:16:40,421][105620] Updated weights for policy 1, policy_version 1909838 (0.0009) [2023-12-27 05:16:40,880][105692] Updated weights for policy 0, policy_version 1905271 (0.0006) [2023-12-27 05:16:40,946][105692] Updated weights for policy 0, policy_version 1905281 (0.0008) [2023-12-27 05:16:41,009][105692] Updated weights for policy 0, policy_version 1905291 (0.0008) [2023-12-27 05:16:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19438.7). Total num frames: 976814080. Throughput: 0: 9569.4, 1: 9541.8. Samples: 976821604. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:16:41,062][104569] Avg episode reward: [(0, '8262.171'), (1, '9254.926')] [2023-12-27 05:16:41,164][105620] Updated weights for policy 1, policy_version 1909848 (0.0009) [2023-12-27 05:16:41,216][105620] Updated weights for policy 1, policy_version 1909858 (0.0009) [2023-12-27 05:16:41,279][105620] Updated weights for policy 1, policy_version 1909868 (0.0007) [2023-12-27 05:16:41,718][105692] Updated weights for policy 0, policy_version 1905301 (0.0009) [2023-12-27 05:16:41,789][105692] Updated weights for policy 0, policy_version 1905311 (0.0010) [2023-12-27 05:16:41,852][105692] Updated weights for policy 0, policy_version 1905321 (0.0009) [2023-12-27 05:16:42,048][105620] Updated weights for policy 1, policy_version 1909878 (0.0007) [2023-12-27 05:16:42,105][105620] Updated weights for policy 1, policy_version 1909888 (0.0009) [2023-12-27 05:16:42,161][105620] Updated weights for policy 1, policy_version 1909898 (0.0009) [2023-12-27 05:16:42,551][105692] Updated weights for policy 0, policy_version 1905331 (0.0008) [2023-12-27 05:16:42,611][105692] Updated weights for policy 0, policy_version 1905341 (0.0008) [2023-12-27 05:16:42,659][105692] Updated weights for policy 0, policy_version 1905351 (0.0008) [2023-12-27 05:16:42,929][105620] Updated weights for policy 1, policy_version 1909908 (0.0009) [2023-12-27 05:16:42,982][105620] Updated weights for policy 1, policy_version 1909918 (0.0009) [2023-12-27 05:16:43,037][105620] Updated weights for policy 1, policy_version 1909928 (0.0005) [2023-12-27 05:16:43,307][105692] Updated weights for policy 0, policy_version 1905361 (0.0007) [2023-12-27 05:16:43,354][105692] Updated weights for policy 0, policy_version 1905371 (0.0005) [2023-12-27 05:16:43,403][105692] Updated weights for policy 0, policy_version 1905381 (0.0005) [2023-12-27 05:16:43,459][105692] Updated weights for policy 0, policy_version 1905391 (0.0005) [2023-12-27 05:16:43,598][105620] Updated weights for policy 1, policy_version 1909938 (0.0005) [2023-12-27 05:16:43,655][105620] Updated weights for policy 1, policy_version 1909948 (0.0006) [2023-12-27 05:16:43,711][105620] Updated weights for policy 1, policy_version 1909958 (0.0008) [2023-12-27 05:16:43,779][105620] Updated weights for policy 1, policy_version 1909968 (0.0010) [2023-12-27 05:16:44,071][105692] Updated weights for policy 0, policy_version 1905401 (0.0010) [2023-12-27 05:16:44,128][105692] Updated weights for policy 0, policy_version 1905411 (0.0011) [2023-12-27 05:16:44,183][105692] Updated weights for policy 0, policy_version 1905421 (0.0008) [2023-12-27 05:16:44,480][105620] Updated weights for policy 1, policy_version 1909978 (0.0005) [2023-12-27 05:16:44,541][105620] Updated weights for policy 1, policy_version 1909988 (0.0010) [2023-12-27 05:16:44,592][105620] Updated weights for policy 1, policy_version 1909998 (0.0010) [2023-12-27 05:16:44,966][105692] Updated weights for policy 0, policy_version 1905431 (0.0007) [2023-12-27 05:16:45,034][105692] Updated weights for policy 0, policy_version 1905441 (0.0005) [2023-12-27 05:16:45,095][105692] Updated weights for policy 0, policy_version 1905451 (0.0007) [2023-12-27 05:16:45,265][105620] Updated weights for policy 1, policy_version 1910008 (0.0011) [2023-12-27 05:16:45,328][105620] Updated weights for policy 1, policy_version 1910018 (0.0010) [2023-12-27 05:16:45,389][105620] Updated weights for policy 1, policy_version 1910028 (0.0010) [2023-12-27 05:16:45,759][105692] Updated weights for policy 0, policy_version 1905461 (0.0011) [2023-12-27 05:16:45,828][105692] Updated weights for policy 0, policy_version 1905471 (0.0010) [2023-12-27 05:16:45,883][105692] Updated weights for policy 0, policy_version 1905481 (0.0010) [2023-12-27 05:16:46,062][104569] Fps is (10 sec: 19661.4, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 976912384. Throughput: 0: 9544.9, 1: 9595.8. Samples: 976881612. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:16:46,062][104569] Avg episode reward: [(0, '8358.501'), (1, '9160.901')] [2023-12-27 05:16:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001905488_487874560.pth... [2023-12-27 05:16:46,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001904336_487579648.pth [2023-12-27 05:16:46,088][105620] Updated weights for policy 1, policy_version 1910038 (0.0007) [2023-12-27 05:16:46,137][105620] Updated weights for policy 1, policy_version 1910048 (0.0005) [2023-12-27 05:16:46,201][105620] Updated weights for policy 1, policy_version 1910058 (0.0005) [2023-12-27 05:16:46,236][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001910064_489046016.pth... [2023-12-27 05:16:46,253][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001908912_488751104.pth [2023-12-27 05:16:46,548][105692] Updated weights for policy 0, policy_version 1905491 (0.0009) [2023-12-27 05:16:46,594][105692] Updated weights for policy 0, policy_version 1905501 (0.0005) [2023-12-27 05:16:46,640][105692] Updated weights for policy 0, policy_version 1905511 (0.0005) [2023-12-27 05:16:46,799][105620] Updated weights for policy 1, policy_version 1910068 (0.0006) [2023-12-27 05:16:46,844][105620] Updated weights for policy 1, policy_version 1910078 (0.0008) [2023-12-27 05:16:46,896][105620] Updated weights for policy 1, policy_version 1910088 (0.0008) [2023-12-27 05:16:47,358][105692] Updated weights for policy 0, policy_version 1905521 (0.0005) [2023-12-27 05:16:47,409][105692] Updated weights for policy 0, policy_version 1905531 (0.0007) [2023-12-27 05:16:47,454][105692] Updated weights for policy 0, policy_version 1905541 (0.0006) [2023-12-27 05:16:47,499][105692] Updated weights for policy 0, policy_version 1905551 (0.0009) [2023-12-27 05:16:47,664][105620] Updated weights for policy 1, policy_version 1910098 (0.0008) [2023-12-27 05:16:47,730][105620] Updated weights for policy 1, policy_version 1910108 (0.0009) [2023-12-27 05:16:47,779][105620] Updated weights for policy 1, policy_version 1910118 (0.0008) [2023-12-27 05:16:47,833][105620] Updated weights for policy 1, policy_version 1910128 (0.0009) [2023-12-27 05:16:48,225][105692] Updated weights for policy 0, policy_version 1905561 (0.0009) [2023-12-27 05:16:48,279][105692] Updated weights for policy 0, policy_version 1905571 (0.0009) [2023-12-27 05:16:48,338][105692] Updated weights for policy 0, policy_version 1905581 (0.0009) [2023-12-27 05:16:48,523][105620] Updated weights for policy 1, policy_version 1910138 (0.0007) [2023-12-27 05:16:48,583][105620] Updated weights for policy 1, policy_version 1910148 (0.0008) [2023-12-27 05:16:48,643][105620] Updated weights for policy 1, policy_version 1910158 (0.0008) [2023-12-27 05:16:48,991][105692] Updated weights for policy 0, policy_version 1905591 (0.0009) [2023-12-27 05:16:49,048][105692] Updated weights for policy 0, policy_version 1905601 (0.0010) [2023-12-27 05:16:49,097][105692] Updated weights for policy 0, policy_version 1905611 (0.0010) [2023-12-27 05:16:49,486][105620] Updated weights for policy 1, policy_version 1910168 (0.0008) [2023-12-27 05:16:49,548][105620] Updated weights for policy 1, policy_version 1910178 (0.0009) [2023-12-27 05:16:49,605][105620] Updated weights for policy 1, policy_version 1910188 (0.0010) [2023-12-27 05:16:49,759][105692] Updated weights for policy 0, policy_version 1905621 (0.0008) [2023-12-27 05:16:49,822][105692] Updated weights for policy 0, policy_version 1905631 (0.0010) [2023-12-27 05:16:49,884][105692] Updated weights for policy 0, policy_version 1905641 (0.0009) [2023-12-27 05:16:50,395][105620] Updated weights for policy 1, policy_version 1910198 (0.0008) [2023-12-27 05:16:50,450][105620] Updated weights for policy 1, policy_version 1910208 (0.0005) [2023-12-27 05:16:50,497][105620] Updated weights for policy 1, policy_version 1910218 (0.0006) [2023-12-27 05:16:50,520][105692] Updated weights for policy 0, policy_version 1905651 (0.0008) [2023-12-27 05:16:50,581][105692] Updated weights for policy 0, policy_version 1905661 (0.0007) [2023-12-27 05:16:50,642][105692] Updated weights for policy 0, policy_version 1905671 (0.0007) [2023-12-27 05:16:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19438.7). Total num frames: 977010688. Throughput: 0: 9588.3, 1: 9605.4. Samples: 976999960. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:16:51,062][104569] Avg episode reward: [(0, '8267.709'), (1, '9068.517')] [2023-12-27 05:16:51,143][105620] Updated weights for policy 1, policy_version 1910228 (0.0008) [2023-12-27 05:16:51,201][105620] Updated weights for policy 1, policy_version 1910238 (0.0006) [2023-12-27 05:16:51,268][105620] Updated weights for policy 1, policy_version 1910248 (0.0009) [2023-12-27 05:16:51,397][105692] Updated weights for policy 0, policy_version 1905681 (0.0006) [2023-12-27 05:16:51,458][105692] Updated weights for policy 0, policy_version 1905691 (0.0008) [2023-12-27 05:16:51,522][105692] Updated weights for policy 0, policy_version 1905701 (0.0008) [2023-12-27 05:16:51,582][105692] Updated weights for policy 0, policy_version 1905711 (0.0008) [2023-12-27 05:16:52,035][105620] Updated weights for policy 1, policy_version 1910258 (0.0009) [2023-12-27 05:16:52,086][105620] Updated weights for policy 1, policy_version 1910268 (0.0010) [2023-12-27 05:16:52,138][105620] Updated weights for policy 1, policy_version 1910278 (0.0010) [2023-12-27 05:16:52,186][105620] Updated weights for policy 1, policy_version 1910288 (0.0010) [2023-12-27 05:16:52,378][105692] Updated weights for policy 0, policy_version 1905721 (0.0008) [2023-12-27 05:16:52,449][105692] Updated weights for policy 0, policy_version 1905731 (0.0006) [2023-12-27 05:16:52,547][105692] Updated weights for policy 0, policy_version 1905741 (0.0007) [2023-12-27 05:16:52,890][105620] Updated weights for policy 1, policy_version 1910298 (0.0007) [2023-12-27 05:16:52,945][105620] Updated weights for policy 1, policy_version 1910308 (0.0008) [2023-12-27 05:16:52,999][105620] Updated weights for policy 1, policy_version 1910318 (0.0006) [2023-12-27 05:16:53,229][105692] Updated weights for policy 0, policy_version 1905751 (0.0008) [2023-12-27 05:16:53,276][105692] Updated weights for policy 0, policy_version 1905761 (0.0009) [2023-12-27 05:16:53,322][105692] Updated weights for policy 0, policy_version 1905771 (0.0009) [2023-12-27 05:16:53,625][105620] Updated weights for policy 1, policy_version 1910328 (0.0008) [2023-12-27 05:16:53,670][105620] Updated weights for policy 1, policy_version 1910338 (0.0008) [2023-12-27 05:16:53,717][105620] Updated weights for policy 1, policy_version 1910348 (0.0008) [2023-12-27 05:16:54,083][105692] Updated weights for policy 0, policy_version 1905781 (0.0008) [2023-12-27 05:16:54,146][105692] Updated weights for policy 0, policy_version 1905791 (0.0011) [2023-12-27 05:16:54,211][105692] Updated weights for policy 0, policy_version 1905801 (0.0010) [2023-12-27 05:16:54,302][105620] Updated weights for policy 1, policy_version 1910358 (0.0006) [2023-12-27 05:16:54,356][105620] Updated weights for policy 1, policy_version 1910368 (0.0007) [2023-12-27 05:16:54,418][105620] Updated weights for policy 1, policy_version 1910378 (0.0006) [2023-12-27 05:16:54,810][105692] Updated weights for policy 0, policy_version 1905811 (0.0011) [2023-12-27 05:16:54,862][105692] Updated weights for policy 0, policy_version 1905821 (0.0010) [2023-12-27 05:16:54,914][105692] Updated weights for policy 0, policy_version 1905831 (0.0010) [2023-12-27 05:16:54,960][105620] Updated weights for policy 1, policy_version 1910388 (0.0006) [2023-12-27 05:16:55,022][105620] Updated weights for policy 1, policy_version 1910398 (0.0006) [2023-12-27 05:16:55,081][105620] Updated weights for policy 1, policy_version 1910408 (0.0005) [2023-12-27 05:16:55,675][105692] Updated weights for policy 0, policy_version 1905841 (0.0011) [2023-12-27 05:16:55,725][105692] Updated weights for policy 0, policy_version 1905851 (0.0010) [2023-12-27 05:16:55,746][105620] Updated weights for policy 1, policy_version 1910418 (0.0006) [2023-12-27 05:16:55,780][105692] Updated weights for policy 0, policy_version 1905861 (0.0010) [2023-12-27 05:16:55,804][105620] Updated weights for policy 1, policy_version 1910428 (0.0010) [2023-12-27 05:16:55,834][105692] Updated weights for policy 0, policy_version 1905871 (0.0010) [2023-12-27 05:16:55,861][105620] Updated weights for policy 1, policy_version 1910438 (0.0010) [2023-12-27 05:16:55,916][105620] Updated weights for policy 1, policy_version 1910448 (0.0010) [2023-12-27 05:16:56,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 977117184. Throughput: 0: 9653.4, 1: 9737.6. Samples: 977121440. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:16:56,062][104569] Avg episode reward: [(0, '7902.601'), (1, '9253.032')] [2023-12-27 05:16:56,538][105692] Updated weights for policy 0, policy_version 1905881 (0.0010) [2023-12-27 05:16:56,589][105692] Updated weights for policy 0, policy_version 1905891 (0.0010) [2023-12-27 05:16:56,644][105692] Updated weights for policy 0, policy_version 1905901 (0.0010) [2023-12-27 05:16:56,646][105620] Updated weights for policy 1, policy_version 1910458 (0.0007) [2023-12-27 05:16:56,697][105620] Updated weights for policy 1, policy_version 1910468 (0.0005) [2023-12-27 05:16:56,746][105620] Updated weights for policy 1, policy_version 1910478 (0.0008) [2023-12-27 05:16:57,354][105692] Updated weights for policy 0, policy_version 1905911 (0.0010) [2023-12-27 05:16:57,368][105620] Updated weights for policy 1, policy_version 1910488 (0.0010) [2023-12-27 05:16:57,401][105692] Updated weights for policy 0, policy_version 1905921 (0.0010) [2023-12-27 05:16:57,416][105620] Updated weights for policy 1, policy_version 1910498 (0.0010) [2023-12-27 05:16:57,446][105692] Updated weights for policy 0, policy_version 1905931 (0.0010) [2023-12-27 05:16:57,463][105620] Updated weights for policy 1, policy_version 1910508 (0.0010) [2023-12-27 05:16:58,146][105620] Updated weights for policy 1, policy_version 1910518 (0.0009) [2023-12-27 05:16:58,201][105692] Updated weights for policy 0, policy_version 1905941 (0.0009) [2023-12-27 05:16:58,205][105620] Updated weights for policy 1, policy_version 1910528 (0.0008) [2023-12-27 05:16:58,263][105692] Updated weights for policy 0, policy_version 1905951 (0.0010) [2023-12-27 05:16:58,269][105620] Updated weights for policy 1, policy_version 1910538 (0.0006) [2023-12-27 05:16:58,326][105692] Updated weights for policy 0, policy_version 1905961 (0.0010) [2023-12-27 05:16:59,039][105692] Updated weights for policy 0, policy_version 1905971 (0.0008) [2023-12-27 05:16:59,046][105620] Updated weights for policy 1, policy_version 1910548 (0.0005) [2023-12-27 05:16:59,100][105692] Updated weights for policy 0, policy_version 1905981 (0.0008) [2023-12-27 05:16:59,114][105620] Updated weights for policy 1, policy_version 1910558 (0.0007) [2023-12-27 05:16:59,159][105692] Updated weights for policy 0, policy_version 1905991 (0.0008) [2023-12-27 05:16:59,174][105620] Updated weights for policy 1, policy_version 1910568 (0.0006) [2023-12-27 05:16:59,950][105692] Updated weights for policy 0, policy_version 1906001 (0.0008) [2023-12-27 05:16:59,994][105620] Updated weights for policy 1, policy_version 1910578 (0.0009) [2023-12-27 05:17:00,009][105692] Updated weights for policy 0, policy_version 1906011 (0.0008) [2023-12-27 05:17:00,055][105620] Updated weights for policy 1, policy_version 1910588 (0.0008) [2023-12-27 05:17:00,074][105692] Updated weights for policy 0, policy_version 1906021 (0.0006) [2023-12-27 05:17:00,115][105620] Updated weights for policy 1, policy_version 1910598 (0.0008) [2023-12-27 05:17:00,137][105692] Updated weights for policy 0, policy_version 1906031 (0.0006) [2023-12-27 05:17:00,175][105620] Updated weights for policy 1, policy_version 1910608 (0.0006) [2023-12-27 05:17:00,789][105620] Updated weights for policy 1, policy_version 1910618 (0.0008) [2023-12-27 05:17:00,846][105620] Updated weights for policy 1, policy_version 1910628 (0.0008) [2023-12-27 05:17:00,899][105692] Updated weights for policy 0, policy_version 1906041 (0.0006) [2023-12-27 05:17:00,902][105620] Updated weights for policy 1, policy_version 1910638 (0.0007) [2023-12-27 05:17:00,955][105692] Updated weights for policy 0, policy_version 1906051 (0.0006) [2023-12-27 05:17:01,013][105692] Updated weights for policy 0, policy_version 1906061 (0.0006) [2023-12-27 05:17:01,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 977215488. Throughput: 0: 9707.8, 1: 9752.1. Samples: 977180296. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:17:01,063][104569] Avg episode reward: [(0, '8358.111'), (1, '9345.318')] [2023-12-27 05:17:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001906064_488022016.pth... [2023-12-27 05:17:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001910640_489193472.pth... [2023-12-27 05:17:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001904912_487727104.pth [2023-12-27 05:17:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001909488_488898560.pth [2023-12-27 05:17:01,705][105692] Updated weights for policy 0, policy_version 1906071 (0.0008) [2023-12-27 05:17:01,736][105620] Updated weights for policy 1, policy_version 1910648 (0.0008) [2023-12-27 05:17:01,774][105692] Updated weights for policy 0, policy_version 1906081 (0.0007) [2023-12-27 05:17:01,801][105620] Updated weights for policy 1, policy_version 1910658 (0.0008) [2023-12-27 05:17:01,843][105692] Updated weights for policy 0, policy_version 1906091 (0.0005) [2023-12-27 05:17:01,854][105620] Updated weights for policy 1, policy_version 1910668 (0.0010) [2023-12-27 05:17:02,495][105692] Updated weights for policy 0, policy_version 1906101 (0.0007) [2023-12-27 05:17:02,542][105692] Updated weights for policy 0, policy_version 1906111 (0.0009) [2023-12-27 05:17:02,578][105620] Updated weights for policy 1, policy_version 1910678 (0.0009) [2023-12-27 05:17:02,588][105692] Updated weights for policy 0, policy_version 1906121 (0.0007) [2023-12-27 05:17:02,631][105620] Updated weights for policy 1, policy_version 1910688 (0.0006) [2023-12-27 05:17:02,696][105620] Updated weights for policy 1, policy_version 1910698 (0.0009) [2023-12-27 05:17:03,279][105620] Updated weights for policy 1, policy_version 1910708 (0.0009) [2023-12-27 05:17:03,346][105620] Updated weights for policy 1, policy_version 1910718 (0.0005) [2023-12-27 05:17:03,409][105620] Updated weights for policy 1, policy_version 1910728 (0.0008) [2023-12-27 05:17:03,415][105692] Updated weights for policy 0, policy_version 1906131 (0.0007) [2023-12-27 05:17:03,475][105692] Updated weights for policy 0, policy_version 1906141 (0.0007) [2023-12-27 05:17:03,525][105692] Updated weights for policy 0, policy_version 1906151 (0.0009) [2023-12-27 05:17:04,108][105620] Updated weights for policy 1, policy_version 1910738 (0.0007) [2023-12-27 05:17:04,160][105620] Updated weights for policy 1, policy_version 1910748 (0.0009) [2023-12-27 05:17:04,211][105620] Updated weights for policy 1, policy_version 1910758 (0.0009) [2023-12-27 05:17:04,260][105692] Updated weights for policy 0, policy_version 1906161 (0.0008) [2023-12-27 05:17:04,262][105620] Updated weights for policy 1, policy_version 1910768 (0.0009) [2023-12-27 05:17:04,320][105692] Updated weights for policy 0, policy_version 1906171 (0.0008) [2023-12-27 05:17:04,382][105692] Updated weights for policy 0, policy_version 1906181 (0.0009) [2023-12-27 05:17:04,446][105692] Updated weights for policy 0, policy_version 1906191 (0.0009) [2023-12-27 05:17:05,070][105620] Updated weights for policy 1, policy_version 1910778 (0.0011) [2023-12-27 05:17:05,119][105620] Updated weights for policy 1, policy_version 1910788 (0.0010) [2023-12-27 05:17:05,171][105620] Updated weights for policy 1, policy_version 1910798 (0.0011) [2023-12-27 05:17:05,180][105692] Updated weights for policy 0, policy_version 1906201 (0.0006) [2023-12-27 05:17:05,234][105692] Updated weights for policy 0, policy_version 1906211 (0.0006) [2023-12-27 05:17:05,282][105692] Updated weights for policy 0, policy_version 1906221 (0.0008) [2023-12-27 05:17:05,855][105620] Updated weights for policy 1, policy_version 1910808 (0.0006) [2023-12-27 05:17:05,914][105620] Updated weights for policy 1, policy_version 1910818 (0.0005) [2023-12-27 05:17:05,976][105620] Updated weights for policy 1, policy_version 1910828 (0.0005) [2023-12-27 05:17:05,989][105692] Updated weights for policy 0, policy_version 1906231 (0.0006) [2023-12-27 05:17:06,059][105692] Updated weights for policy 0, policy_version 1906241 (0.0005) [2023-12-27 05:17:06,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 977305600. Throughput: 0: 9698.1, 1: 9775.2. Samples: 977294184. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:17:06,063][104569] Avg episode reward: [(0, '8169.036'), (1, '9345.298')] [2023-12-27 05:17:06,129][105692] Updated weights for policy 0, policy_version 1906251 (0.0006) [2023-12-27 05:17:06,622][105620] Updated weights for policy 1, policy_version 1910838 (0.0008) [2023-12-27 05:17:06,675][105620] Updated weights for policy 1, policy_version 1910848 (0.0006) [2023-12-27 05:17:06,735][105620] Updated weights for policy 1, policy_version 1910858 (0.0007) [2023-12-27 05:17:06,806][105692] Updated weights for policy 0, policy_version 1906261 (0.0008) [2023-12-27 05:17:06,858][105692] Updated weights for policy 0, policy_version 1906272 (0.0010) [2023-12-27 05:17:06,912][105692] Updated weights for policy 0, policy_version 1906282 (0.0007) [2023-12-27 05:17:07,341][105620] Updated weights for policy 1, policy_version 1910868 (0.0009) [2023-12-27 05:17:07,399][105620] Updated weights for policy 1, policy_version 1910878 (0.0009) [2023-12-27 05:17:07,457][105620] Updated weights for policy 1, policy_version 1910888 (0.0009) [2023-12-27 05:17:07,703][105692] Updated weights for policy 0, policy_version 1906292 (0.0008) [2023-12-27 05:17:07,752][105692] Updated weights for policy 0, policy_version 1906302 (0.0009) [2023-12-27 05:17:07,806][105692] Updated weights for policy 0, policy_version 1906312 (0.0009) [2023-12-27 05:17:08,199][105620] Updated weights for policy 1, policy_version 1910898 (0.0008) [2023-12-27 05:17:08,266][105620] Updated weights for policy 1, policy_version 1910908 (0.0009) [2023-12-27 05:17:08,325][105620] Updated weights for policy 1, policy_version 1910918 (0.0009) [2023-12-27 05:17:08,391][105620] Updated weights for policy 1, policy_version 1910928 (0.0009) [2023-12-27 05:17:08,569][105692] Updated weights for policy 0, policy_version 1906322 (0.0008) [2023-12-27 05:17:08,617][105692] Updated weights for policy 0, policy_version 1906332 (0.0008) [2023-12-27 05:17:08,672][105692] Updated weights for policy 0, policy_version 1906342 (0.0009) [2023-12-27 05:17:08,721][105692] Updated weights for policy 0, policy_version 1906352 (0.0009) [2023-12-27 05:17:09,115][105620] Updated weights for policy 1, policy_version 1910938 (0.0009) [2023-12-27 05:17:09,165][105620] Updated weights for policy 1, policy_version 1910948 (0.0006) [2023-12-27 05:17:09,224][105620] Updated weights for policy 1, policy_version 1910958 (0.0007) [2023-12-27 05:17:09,515][105692] Updated weights for policy 0, policy_version 1906362 (0.0010) [2023-12-27 05:17:09,581][105692] Updated weights for policy 0, policy_version 1906372 (0.0007) [2023-12-27 05:17:09,642][105692] Updated weights for policy 0, policy_version 1906382 (0.0008) [2023-12-27 05:17:10,040][105620] Updated weights for policy 1, policy_version 1910968 (0.0008) [2023-12-27 05:17:10,089][105620] Updated weights for policy 1, policy_version 1910978 (0.0007) [2023-12-27 05:17:10,154][105620] Updated weights for policy 1, policy_version 1910988 (0.0007) [2023-12-27 05:17:10,340][105692] Updated weights for policy 0, policy_version 1906392 (0.0009) [2023-12-27 05:17:10,403][105692] Updated weights for policy 0, policy_version 1906402 (0.0010) [2023-12-27 05:17:10,457][105692] Updated weights for policy 0, policy_version 1906412 (0.0010) [2023-12-27 05:17:10,755][105620] Updated weights for policy 1, policy_version 1910998 (0.0006) [2023-12-27 05:17:10,815][105620] Updated weights for policy 1, policy_version 1911008 (0.0005) [2023-12-27 05:17:10,878][105620] Updated weights for policy 1, policy_version 1911018 (0.0005) [2023-12-27 05:17:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19438.7). Total num frames: 977403904. Throughput: 0: 9763.2, 1: 9768.9. Samples: 977411404. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:17:11,062][104569] Avg episode reward: [(0, '7897.031'), (1, '9345.383')] [2023-12-27 05:17:11,239][105692] Updated weights for policy 0, policy_version 1906423 (0.0009) [2023-12-27 05:17:11,302][105692] Updated weights for policy 0, policy_version 1906433 (0.0009) [2023-12-27 05:17:11,367][105692] Updated weights for policy 0, policy_version 1906443 (0.0008) [2023-12-27 05:17:11,541][105620] Updated weights for policy 1, policy_version 1911028 (0.0007) [2023-12-27 05:17:11,616][105620] Updated weights for policy 1, policy_version 1911038 (0.0009) [2023-12-27 05:17:11,680][105620] Updated weights for policy 1, policy_version 1911048 (0.0009) [2023-12-27 05:17:12,162][105692] Updated weights for policy 0, policy_version 1906453 (0.0009) [2023-12-27 05:17:12,234][105692] Updated weights for policy 0, policy_version 1906463 (0.0008) [2023-12-27 05:17:12,301][105692] Updated weights for policy 0, policy_version 1906473 (0.0008) [2023-12-27 05:17:12,424][105620] Updated weights for policy 1, policy_version 1911058 (0.0012) [2023-12-27 05:17:12,482][105620] Updated weights for policy 1, policy_version 1911068 (0.0007) [2023-12-27 05:17:12,537][105620] Updated weights for policy 1, policy_version 1911078 (0.0009) [2023-12-27 05:17:12,588][105620] Updated weights for policy 1, policy_version 1911088 (0.0009) [2023-12-27 05:17:13,045][105692] Updated weights for policy 0, policy_version 1906483 (0.0010) [2023-12-27 05:17:13,116][105692] Updated weights for policy 0, policy_version 1906493 (0.0010) [2023-12-27 05:17:13,166][105692] Updated weights for policy 0, policy_version 1906503 (0.0009) [2023-12-27 05:17:13,314][105620] Updated weights for policy 1, policy_version 1911098 (0.0009) [2023-12-27 05:17:13,368][105620] Updated weights for policy 1, policy_version 1911108 (0.0009) [2023-12-27 05:17:13,422][105620] Updated weights for policy 1, policy_version 1911118 (0.0009) [2023-12-27 05:17:13,977][105692] Updated weights for policy 0, policy_version 1906513 (0.0009) [2023-12-27 05:17:14,042][105692] Updated weights for policy 0, policy_version 1906523 (0.0009) [2023-12-27 05:17:14,079][105620] Updated weights for policy 1, policy_version 1911128 (0.0006) [2023-12-27 05:17:14,110][105692] Updated weights for policy 0, policy_version 1906533 (0.0008) [2023-12-27 05:17:14,141][105620] Updated weights for policy 1, policy_version 1911138 (0.0007) [2023-12-27 05:17:14,167][105692] Updated weights for policy 0, policy_version 1906543 (0.0007) [2023-12-27 05:17:14,199][105620] Updated weights for policy 1, policy_version 1911148 (0.0008) [2023-12-27 05:17:14,872][105620] Updated weights for policy 1, policy_version 1911158 (0.0007) [2023-12-27 05:17:14,932][105620] Updated weights for policy 1, policy_version 1911168 (0.0007) [2023-12-27 05:17:14,955][105692] Updated weights for policy 0, policy_version 1906553 (0.0008) [2023-12-27 05:17:14,990][105620] Updated weights for policy 1, policy_version 1911178 (0.0006) [2023-12-27 05:17:15,022][105692] Updated weights for policy 0, policy_version 1906563 (0.0008) [2023-12-27 05:17:15,094][105692] Updated weights for policy 0, policy_version 1906573 (0.0009) [2023-12-27 05:17:15,636][105620] Updated weights for policy 1, policy_version 1911188 (0.0007) [2023-12-27 05:17:15,699][105620] Updated weights for policy 1, policy_version 1911198 (0.0008) [2023-12-27 05:17:15,757][105620] Updated weights for policy 1, policy_version 1911208 (0.0008) [2023-12-27 05:17:15,846][105692] Updated weights for policy 0, policy_version 1906583 (0.0009) [2023-12-27 05:17:15,904][105692] Updated weights for policy 0, policy_version 1906593 (0.0010) [2023-12-27 05:17:15,957][105692] Updated weights for policy 0, policy_version 1906603 (0.0009) [2023-12-27 05:17:16,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 977502208. Throughput: 0: 9675.2, 1: 9819.7. Samples: 977468292. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:17:16,062][104569] Avg episode reward: [(0, '8174.326'), (1, '9345.443')] [2023-12-27 05:17:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001906608_488161280.pth... [2023-12-27 05:17:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001911216_489340928.pth... [2023-12-27 05:17:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001905488_487874560.pth [2023-12-27 05:17:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001910064_489046016.pth [2023-12-27 05:17:16,420][105620] Updated weights for policy 1, policy_version 1911218 (0.0008) [2023-12-27 05:17:16,471][105620] Updated weights for policy 1, policy_version 1911228 (0.0009) [2023-12-27 05:17:16,517][105620] Updated weights for policy 1, policy_version 1911238 (0.0008) [2023-12-27 05:17:16,571][105620] Updated weights for policy 1, policy_version 1911248 (0.0008) [2023-12-27 05:17:16,746][105692] Updated weights for policy 0, policy_version 1906613 (0.0009) [2023-12-27 05:17:16,803][105692] Updated weights for policy 0, policy_version 1906623 (0.0009) [2023-12-27 05:17:16,851][105692] Updated weights for policy 0, policy_version 1906633 (0.0009) [2023-12-27 05:17:17,357][105620] Updated weights for policy 1, policy_version 1911258 (0.0009) [2023-12-27 05:17:17,408][105620] Updated weights for policy 1, policy_version 1911268 (0.0009) [2023-12-27 05:17:17,453][105620] Updated weights for policy 1, policy_version 1911278 (0.0008) [2023-12-27 05:17:17,558][105692] Updated weights for policy 0, policy_version 1906643 (0.0009) [2023-12-27 05:17:17,612][105692] Updated weights for policy 0, policy_version 1906653 (0.0010) [2023-12-27 05:17:17,675][105692] Updated weights for policy 0, policy_version 1906663 (0.0010) [2023-12-27 05:17:18,107][105620] Updated weights for policy 1, policy_version 1911288 (0.0008) [2023-12-27 05:17:18,164][105620] Updated weights for policy 1, policy_version 1911298 (0.0008) [2023-12-27 05:17:18,227][105620] Updated weights for policy 1, policy_version 1911308 (0.0009) [2023-12-27 05:17:18,426][105692] Updated weights for policy 0, policy_version 1906673 (0.0008) [2023-12-27 05:17:18,483][105692] Updated weights for policy 0, policy_version 1906683 (0.0005) [2023-12-27 05:17:18,538][105692] Updated weights for policy 0, policy_version 1906693 (0.0005) [2023-12-27 05:17:18,595][105692] Updated weights for policy 0, policy_version 1906703 (0.0005) [2023-12-27 05:17:19,076][105620] Updated weights for policy 1, policy_version 1911318 (0.0009) [2023-12-27 05:17:19,126][105620] Updated weights for policy 1, policy_version 1911328 (0.0009) [2023-12-27 05:17:19,182][105620] Updated weights for policy 1, policy_version 1911338 (0.0009) [2023-12-27 05:17:19,206][105692] Updated weights for policy 0, policy_version 1906713 (0.0006) [2023-12-27 05:17:19,267][105692] Updated weights for policy 0, policy_version 1906723 (0.0008) [2023-12-27 05:17:19,321][105692] Updated weights for policy 0, policy_version 1906733 (0.0010) [2023-12-27 05:17:19,926][105620] Updated weights for policy 1, policy_version 1911348 (0.0009) [2023-12-27 05:17:19,992][105620] Updated weights for policy 1, policy_version 1911358 (0.0006) [2023-12-27 05:17:20,050][105620] Updated weights for policy 1, policy_version 1911368 (0.0008) [2023-12-27 05:17:20,128][105692] Updated weights for policy 0, policy_version 1906743 (0.0008) [2023-12-27 05:17:20,194][105692] Updated weights for policy 0, policy_version 1906753 (0.0010) [2023-12-27 05:17:20,256][105692] Updated weights for policy 0, policy_version 1906763 (0.0009) [2023-12-27 05:17:20,715][105620] Updated weights for policy 1, policy_version 1911378 (0.0007) [2023-12-27 05:17:20,778][105620] Updated weights for policy 1, policy_version 1911388 (0.0009) [2023-12-27 05:17:20,833][105620] Updated weights for policy 1, policy_version 1911398 (0.0007) [2023-12-27 05:17:20,896][105620] Updated weights for policy 1, policy_version 1911408 (0.0009) [2023-12-27 05:17:21,062][105692] Updated weights for policy 0, policy_version 1906773 (0.0010) [2023-12-27 05:17:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 977592320. Throughput: 0: 9560.8, 1: 9884.1. Samples: 977583020. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:17:21,063][104569] Avg episode reward: [(0, '8263.412'), (1, '9345.459')] [2023-12-27 05:17:21,125][105692] Updated weights for policy 0, policy_version 1906783 (0.0009) [2023-12-27 05:17:21,189][105692] Updated weights for policy 0, policy_version 1906793 (0.0010) [2023-12-27 05:17:21,626][105620] Updated weights for policy 1, policy_version 1911418 (0.0009) [2023-12-27 05:17:21,695][105620] Updated weights for policy 1, policy_version 1911428 (0.0009) [2023-12-27 05:17:21,765][105620] Updated weights for policy 1, policy_version 1911438 (0.0008) [2023-12-27 05:17:21,982][105692] Updated weights for policy 0, policy_version 1906803 (0.0009) [2023-12-27 05:17:22,045][105692] Updated weights for policy 0, policy_version 1906813 (0.0006) [2023-12-27 05:17:22,117][105692] Updated weights for policy 0, policy_version 1906823 (0.0007) [2023-12-27 05:17:22,661][105620] Updated weights for policy 1, policy_version 1911448 (0.0008) [2023-12-27 05:17:22,712][105620] Updated weights for policy 1, policy_version 1911458 (0.0008) [2023-12-27 05:17:22,760][105620] Updated weights for policy 1, policy_version 1911468 (0.0009) [2023-12-27 05:17:22,761][105692] Updated weights for policy 0, policy_version 1906833 (0.0009) [2023-12-27 05:17:22,819][105692] Updated weights for policy 0, policy_version 1906843 (0.0009) [2023-12-27 05:17:22,877][105692] Updated weights for policy 0, policy_version 1906853 (0.0009) [2023-12-27 05:17:22,934][105692] Updated weights for policy 0, policy_version 1906863 (0.0009) [2023-12-27 05:17:23,394][105620] Updated weights for policy 1, policy_version 1911478 (0.0008) [2023-12-27 05:17:23,452][105620] Updated weights for policy 1, policy_version 1911488 (0.0009) [2023-12-27 05:17:23,511][105620] Updated weights for policy 1, policy_version 1911498 (0.0009) [2023-12-27 05:17:23,636][105692] Updated weights for policy 0, policy_version 1906873 (0.0009) [2023-12-27 05:17:23,698][105692] Updated weights for policy 0, policy_version 1906883 (0.0008) [2023-12-27 05:17:23,754][105692] Updated weights for policy 0, policy_version 1906893 (0.0007) [2023-12-27 05:17:24,297][105620] Updated weights for policy 1, policy_version 1911508 (0.0008) [2023-12-27 05:17:24,355][105620] Updated weights for policy 1, policy_version 1911518 (0.0010) [2023-12-27 05:17:24,404][105620] Updated weights for policy 1, policy_version 1911528 (0.0008) [2023-12-27 05:17:24,410][105692] Updated weights for policy 0, policy_version 1906903 (0.0008) [2023-12-27 05:17:24,460][105692] Updated weights for policy 0, policy_version 1906913 (0.0007) [2023-12-27 05:17:24,518][105692] Updated weights for policy 0, policy_version 1906923 (0.0008) [2023-12-27 05:17:25,153][105620] Updated weights for policy 1, policy_version 1911538 (0.0006) [2023-12-27 05:17:25,208][105620] Updated weights for policy 1, policy_version 1911548 (0.0008) [2023-12-27 05:17:25,227][105692] Updated weights for policy 0, policy_version 1906933 (0.0007) [2023-12-27 05:17:25,257][105620] Updated weights for policy 1, policy_version 1911558 (0.0007) [2023-12-27 05:17:25,276][105692] Updated weights for policy 0, policy_version 1906943 (0.0007) [2023-12-27 05:17:25,310][105620] Updated weights for policy 1, policy_version 1911568 (0.0006) [2023-12-27 05:17:25,330][105692] Updated weights for policy 0, policy_version 1906953 (0.0007) [2023-12-27 05:17:26,039][105692] Updated weights for policy 0, policy_version 1906963 (0.0009) [2023-12-27 05:17:26,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 977682432. Throughput: 0: 9581.1, 1: 9873.6. Samples: 977697064. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:17:26,062][104569] Avg episode reward: [(0, '8442.706'), (1, '9345.454')] [2023-12-27 05:17:26,070][105620] Updated weights for policy 1, policy_version 1911578 (0.0007) [2023-12-27 05:17:26,093][105692] Updated weights for policy 0, policy_version 1906973 (0.0006) [2023-12-27 05:17:26,126][105620] Updated weights for policy 1, policy_version 1911588 (0.0009) [2023-12-27 05:17:26,137][105692] Updated weights for policy 0, policy_version 1906983 (0.0006) [2023-12-27 05:17:26,180][105620] Updated weights for policy 1, policy_version 1911598 (0.0008) [2023-12-27 05:17:26,885][105692] Updated weights for policy 0, policy_version 1906993 (0.0007) [2023-12-27 05:17:26,933][105620] Updated weights for policy 1, policy_version 1911608 (0.0006) [2023-12-27 05:17:26,934][105692] Updated weights for policy 0, policy_version 1907003 (0.0009) [2023-12-27 05:17:26,978][105620] Updated weights for policy 1, policy_version 1911618 (0.0005) [2023-12-27 05:17:26,990][105692] Updated weights for policy 0, policy_version 1907013 (0.0009) [2023-12-27 05:17:27,033][105620] Updated weights for policy 1, policy_version 1911628 (0.0006) [2023-12-27 05:17:27,038][105692] Updated weights for policy 0, policy_version 1907023 (0.0008) [2023-12-27 05:17:27,690][105620] Updated weights for policy 1, policy_version 1911638 (0.0007) [2023-12-27 05:17:27,728][105692] Updated weights for policy 0, policy_version 1907033 (0.0008) [2023-12-27 05:17:27,738][105620] Updated weights for policy 1, policy_version 1911648 (0.0007) [2023-12-27 05:17:27,778][105692] Updated weights for policy 0, policy_version 1907043 (0.0007) [2023-12-27 05:17:27,788][105620] Updated weights for policy 1, policy_version 1911658 (0.0007) [2023-12-27 05:17:27,828][105692] Updated weights for policy 0, policy_version 1907053 (0.0007) [2023-12-27 05:17:28,520][105620] Updated weights for policy 1, policy_version 1911668 (0.0007) [2023-12-27 05:17:28,582][105620] Updated weights for policy 1, policy_version 1911678 (0.0008) [2023-12-27 05:17:28,589][105692] Updated weights for policy 0, policy_version 1907063 (0.0007) [2023-12-27 05:17:28,639][105620] Updated weights for policy 1, policy_version 1911688 (0.0009) [2023-12-27 05:17:28,649][105692] Updated weights for policy 0, policy_version 1907073 (0.0006) [2023-12-27 05:17:28,713][105692] Updated weights for policy 0, policy_version 1907083 (0.0007) [2023-12-27 05:17:29,406][105620] Updated weights for policy 1, policy_version 1911698 (0.0008) [2023-12-27 05:17:29,456][105620] Updated weights for policy 1, policy_version 1911708 (0.0007) [2023-12-27 05:17:29,460][105692] Updated weights for policy 0, policy_version 1907093 (0.0008) [2023-12-27 05:17:29,505][105620] Updated weights for policy 1, policy_version 1911718 (0.0008) [2023-12-27 05:17:29,516][105692] Updated weights for policy 0, policy_version 1907103 (0.0007) [2023-12-27 05:17:29,551][105620] Updated weights for policy 1, policy_version 1911728 (0.0006) [2023-12-27 05:17:29,562][105692] Updated weights for policy 0, policy_version 1907113 (0.0006) [2023-12-27 05:17:30,281][105692] Updated weights for policy 0, policy_version 1907123 (0.0008) [2023-12-27 05:17:30,322][105620] Updated weights for policy 1, policy_version 1911738 (0.0010) [2023-12-27 05:17:30,336][105692] Updated weights for policy 0, policy_version 1907133 (0.0005) [2023-12-27 05:17:30,378][105620] Updated weights for policy 1, policy_version 1911748 (0.0011) [2023-12-27 05:17:30,396][105692] Updated weights for policy 0, policy_version 1907143 (0.0005) [2023-12-27 05:17:30,430][105620] Updated weights for policy 1, policy_version 1911758 (0.0011) [2023-12-27 05:17:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 977780736. Throughput: 0: 9568.3, 1: 9857.4. Samples: 977755768. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:17:31,062][104569] Avg episode reward: [(0, '8352.245'), (1, '9253.263')] [2023-12-27 05:17:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001907152_488300544.pth... [2023-12-27 05:17:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001911760_489480192.pth... [2023-12-27 05:17:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001910640_489193472.pth [2023-12-27 05:17:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001906064_488022016.pth [2023-12-27 05:17:31,164][105620] Updated weights for policy 1, policy_version 1911768 (0.0009) [2023-12-27 05:17:31,181][105692] Updated weights for policy 0, policy_version 1907153 (0.0006) [2023-12-27 05:17:31,229][105620] Updated weights for policy 1, policy_version 1911778 (0.0008) [2023-12-27 05:17:31,242][105692] Updated weights for policy 0, policy_version 1907163 (0.0011) [2023-12-27 05:17:31,294][105620] Updated weights for policy 1, policy_version 1911788 (0.0007) [2023-12-27 05:17:31,299][105692] Updated weights for policy 0, policy_version 1907173 (0.0008) [2023-12-27 05:17:31,356][105692] Updated weights for policy 0, policy_version 1907183 (0.0010) [2023-12-27 05:17:31,945][105620] Updated weights for policy 1, policy_version 1911798 (0.0007) [2023-12-27 05:17:32,018][105620] Updated weights for policy 1, policy_version 1911808 (0.0009) [2023-12-27 05:17:32,074][105620] Updated weights for policy 1, policy_version 1911818 (0.0011) [2023-12-27 05:17:32,176][105692] Updated weights for policy 0, policy_version 1907193 (0.0010) [2023-12-27 05:17:32,230][105692] Updated weights for policy 0, policy_version 1907203 (0.0009) [2023-12-27 05:17:32,298][105692] Updated weights for policy 0, policy_version 1907213 (0.0008) [2023-12-27 05:17:32,795][105620] Updated weights for policy 1, policy_version 1911828 (0.0008) [2023-12-27 05:17:32,853][105620] Updated weights for policy 1, policy_version 1911838 (0.0010) [2023-12-27 05:17:32,908][105620] Updated weights for policy 1, policy_version 1911848 (0.0010) [2023-12-27 05:17:32,930][105692] Updated weights for policy 0, policy_version 1907223 (0.0006) [2023-12-27 05:17:32,977][105692] Updated weights for policy 0, policy_version 1907233 (0.0006) [2023-12-27 05:17:33,028][105692] Updated weights for policy 0, policy_version 1907243 (0.0006) [2023-12-27 05:17:33,549][105620] Updated weights for policy 1, policy_version 1911858 (0.0010) [2023-12-27 05:17:33,596][105620] Updated weights for policy 1, policy_version 1911868 (0.0008) [2023-12-27 05:17:33,652][105620] Updated weights for policy 1, policy_version 1911878 (0.0005) [2023-12-27 05:17:33,709][105692] Updated weights for policy 0, policy_version 1907253 (0.0007) [2023-12-27 05:17:33,711][105620] Updated weights for policy 1, policy_version 1911888 (0.0007) [2023-12-27 05:17:33,760][105692] Updated weights for policy 0, policy_version 1907263 (0.0005) [2023-12-27 05:17:33,803][105692] Updated weights for policy 0, policy_version 1907273 (0.0005) [2023-12-27 05:17:34,385][105620] Updated weights for policy 1, policy_version 1911898 (0.0006) [2023-12-27 05:17:34,404][105692] Updated weights for policy 0, policy_version 1907283 (0.0006) [2023-12-27 05:17:34,450][105620] Updated weights for policy 1, policy_version 1911908 (0.0005) [2023-12-27 05:17:34,461][105692] Updated weights for policy 0, policy_version 1907293 (0.0009) [2023-12-27 05:17:34,513][105620] Updated weights for policy 1, policy_version 1911918 (0.0006) [2023-12-27 05:17:34,521][105692] Updated weights for policy 0, policy_version 1907303 (0.0009) [2023-12-27 05:17:35,176][105620] Updated weights for policy 1, policy_version 1911928 (0.0008) [2023-12-27 05:17:35,223][105620] Updated weights for policy 1, policy_version 1911938 (0.0009) [2023-12-27 05:17:35,271][105620] Updated weights for policy 1, policy_version 1911948 (0.0009) [2023-12-27 05:17:35,303][105692] Updated weights for policy 0, policy_version 1907313 (0.0009) [2023-12-27 05:17:35,352][105692] Updated weights for policy 0, policy_version 1907323 (0.0009) [2023-12-27 05:17:35,400][105692] Updated weights for policy 0, policy_version 1907333 (0.0010) [2023-12-27 05:17:35,451][105692] Updated weights for policy 0, policy_version 1907343 (0.0010) [2023-12-27 05:17:36,036][105620] Updated weights for policy 1, policy_version 1911958 (0.0009) [2023-12-27 05:17:36,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.9, 300 sec: 19410.9). Total num frames: 977879040. Throughput: 0: 9533.3, 1: 9892.7. Samples: 977874132. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:17:36,062][104569] Avg episode reward: [(0, '8719.683'), (1, '9161.003')] [2023-12-27 05:17:36,094][105620] Updated weights for policy 1, policy_version 1911968 (0.0008) [2023-12-27 05:17:36,113][105692] Updated weights for policy 0, policy_version 1907353 (0.0007) [2023-12-27 05:17:36,161][105620] Updated weights for policy 1, policy_version 1911978 (0.0008) [2023-12-27 05:17:36,171][105692] Updated weights for policy 0, policy_version 1907363 (0.0008) [2023-12-27 05:17:36,224][105692] Updated weights for policy 0, policy_version 1907373 (0.0007) [2023-12-27 05:17:36,883][105620] Updated weights for policy 1, policy_version 1911988 (0.0008) [2023-12-27 05:17:36,934][105620] Updated weights for policy 1, policy_version 1911998 (0.0009) [2023-12-27 05:17:36,994][105620] Updated weights for policy 1, policy_version 1912008 (0.0009) [2023-12-27 05:17:37,005][105692] Updated weights for policy 0, policy_version 1907383 (0.0008) [2023-12-27 05:17:37,060][105692] Updated weights for policy 0, policy_version 1907393 (0.0007) [2023-12-27 05:17:37,118][105692] Updated weights for policy 0, policy_version 1907403 (0.0009) [2023-12-27 05:17:37,748][105620] Updated weights for policy 1, policy_version 1912018 (0.0007) [2023-12-27 05:17:37,804][105620] Updated weights for policy 1, policy_version 1912028 (0.0007) [2023-12-27 05:17:37,862][105620] Updated weights for policy 1, policy_version 1912038 (0.0006) [2023-12-27 05:17:37,910][105692] Updated weights for policy 0, policy_version 1907413 (0.0007) [2023-12-27 05:17:37,916][105620] Updated weights for policy 1, policy_version 1912048 (0.0008) [2023-12-27 05:17:37,967][105692] Updated weights for policy 0, policy_version 1907423 (0.0009) [2023-12-27 05:17:38,016][105692] Updated weights for policy 0, policy_version 1907433 (0.0008) [2023-12-27 05:17:38,622][105620] Updated weights for policy 1, policy_version 1912058 (0.0009) [2023-12-27 05:17:38,676][105620] Updated weights for policy 1, policy_version 1912068 (0.0008) [2023-12-27 05:17:38,731][105620] Updated weights for policy 1, policy_version 1912078 (0.0009) [2023-12-27 05:17:38,777][105692] Updated weights for policy 0, policy_version 1907443 (0.0008) [2023-12-27 05:17:38,832][105692] Updated weights for policy 0, policy_version 1907453 (0.0009) [2023-12-27 05:17:38,894][105692] Updated weights for policy 0, policy_version 1907463 (0.0009) [2023-12-27 05:17:39,533][105620] Updated weights for policy 1, policy_version 1912088 (0.0009) [2023-12-27 05:17:39,594][105620] Updated weights for policy 1, policy_version 1912098 (0.0009) [2023-12-27 05:17:39,656][105620] Updated weights for policy 1, policy_version 1912108 (0.0009) [2023-12-27 05:17:39,683][105692] Updated weights for policy 0, policy_version 1907473 (0.0009) [2023-12-27 05:17:39,749][105692] Updated weights for policy 0, policy_version 1907483 (0.0009) [2023-12-27 05:17:39,812][105692] Updated weights for policy 0, policy_version 1907493 (0.0008) [2023-12-27 05:17:39,873][105692] Updated weights for policy 0, policy_version 1907503 (0.0007) [2023-12-27 05:17:40,340][105620] Updated weights for policy 1, policy_version 1912118 (0.0006) [2023-12-27 05:17:40,402][105620] Updated weights for policy 1, policy_version 1912128 (0.0008) [2023-12-27 05:17:40,464][105620] Updated weights for policy 1, policy_version 1912138 (0.0009) [2023-12-27 05:17:40,668][105692] Updated weights for policy 0, policy_version 1907513 (0.0009) [2023-12-27 05:17:40,719][105692] Updated weights for policy 0, policy_version 1907523 (0.0008) [2023-12-27 05:17:40,773][105692] Updated weights for policy 0, policy_version 1907533 (0.0005) [2023-12-27 05:17:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19438.7). Total num frames: 977977344. Throughput: 0: 9462.4, 1: 9759.8. Samples: 977986440. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:17:41,062][104569] Avg episode reward: [(0, '8354.966'), (1, '9253.216')] [2023-12-27 05:17:41,236][105620] Updated weights for policy 1, policy_version 1912148 (0.0009) [2023-12-27 05:17:41,298][105620] Updated weights for policy 1, policy_version 1912158 (0.0009) [2023-12-27 05:17:41,358][105620] Updated weights for policy 1, policy_version 1912168 (0.0010) [2023-12-27 05:17:41,421][105692] Updated weights for policy 0, policy_version 1907543 (0.0007) [2023-12-27 05:17:41,468][105692] Updated weights for policy 0, policy_version 1907553 (0.0008) [2023-12-27 05:17:41,515][105692] Updated weights for policy 0, policy_version 1907563 (0.0008) [2023-12-27 05:17:42,153][105620] Updated weights for policy 1, policy_version 1912178 (0.0007) [2023-12-27 05:17:42,215][105620] Updated weights for policy 1, policy_version 1912188 (0.0009) [2023-12-27 05:17:42,281][105620] Updated weights for policy 1, policy_version 1912198 (0.0009) [2023-12-27 05:17:42,337][105620] Updated weights for policy 1, policy_version 1912208 (0.0007) [2023-12-27 05:17:42,339][105692] Updated weights for policy 0, policy_version 1907573 (0.0008) [2023-12-27 05:17:42,411][105692] Updated weights for policy 0, policy_version 1907583 (0.0009) [2023-12-27 05:17:42,465][105692] Updated weights for policy 0, policy_version 1907593 (0.0009) [2023-12-27 05:17:42,959][105620] Updated weights for policy 1, policy_version 1912218 (0.0005) [2023-12-27 05:17:43,031][105620] Updated weights for policy 1, policy_version 1912228 (0.0009) [2023-12-27 05:17:43,097][105620] Updated weights for policy 1, policy_version 1912238 (0.0009) [2023-12-27 05:17:43,324][105692] Updated weights for policy 0, policy_version 1907603 (0.0009) [2023-12-27 05:17:43,374][105692] Updated weights for policy 0, policy_version 1907613 (0.0007) [2023-12-27 05:17:43,423][105692] Updated weights for policy 0, policy_version 1907623 (0.0011) [2023-12-27 05:17:43,731][105620] Updated weights for policy 1, policy_version 1912248 (0.0006) [2023-12-27 05:17:43,787][105620] Updated weights for policy 1, policy_version 1912258 (0.0005) [2023-12-27 05:17:43,833][105620] Updated weights for policy 1, policy_version 1912268 (0.0005) [2023-12-27 05:17:44,190][105692] Updated weights for policy 0, policy_version 1907633 (0.0011) [2023-12-27 05:17:44,235][105692] Updated weights for policy 0, policy_version 1907643 (0.0010) [2023-12-27 05:17:44,283][105692] Updated weights for policy 0, policy_version 1907653 (0.0010) [2023-12-27 05:17:44,335][105692] Updated weights for policy 0, policy_version 1907663 (0.0010) [2023-12-27 05:17:44,364][105620] Updated weights for policy 1, policy_version 1912278 (0.0005) [2023-12-27 05:17:44,422][105620] Updated weights for policy 1, policy_version 1912288 (0.0007) [2023-12-27 05:17:44,472][105620] Updated weights for policy 1, policy_version 1912298 (0.0009) [2023-12-27 05:17:45,051][105692] Updated weights for policy 0, policy_version 1907673 (0.0008) [2023-12-27 05:17:45,110][105692] Updated weights for policy 0, policy_version 1907683 (0.0008) [2023-12-27 05:17:45,151][105620] Updated weights for policy 1, policy_version 1912308 (0.0008) [2023-12-27 05:17:45,169][105692] Updated weights for policy 0, policy_version 1907693 (0.0008) [2023-12-27 05:17:45,212][105620] Updated weights for policy 1, policy_version 1912318 (0.0007) [2023-12-27 05:17:45,272][105620] Updated weights for policy 1, policy_version 1912328 (0.0008) [2023-12-27 05:17:45,913][105620] Updated weights for policy 1, policy_version 1912338 (0.0007) [2023-12-27 05:17:45,958][105692] Updated weights for policy 0, policy_version 1907703 (0.0007) [2023-12-27 05:17:45,967][105620] Updated weights for policy 1, policy_version 1912348 (0.0010) [2023-12-27 05:17:46,007][105692] Updated weights for policy 0, policy_version 1907713 (0.0007) [2023-12-27 05:17:46,029][105620] Updated weights for policy 1, policy_version 1912358 (0.0010) [2023-12-27 05:17:46,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 978067456. Throughput: 0: 9432.8, 1: 9751.7. Samples: 978043600. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:17:46,063][105692] Updated weights for policy 0, policy_version 1907723 (0.0006) [2023-12-27 05:17:46,063][104569] Avg episode reward: [(0, '8174.225'), (1, '9345.574')] [2023-12-27 05:17:46,086][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001912368_489635840.pth... [2023-12-27 05:17:46,086][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001907728_488448000.pth... [2023-12-27 05:17:46,088][105620] Updated weights for policy 1, policy_version 1912368 (0.0010) [2023-12-27 05:17:46,089][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001906608_488161280.pth [2023-12-27 05:17:46,089][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001911216_489340928.pth [2023-12-27 05:17:46,712][105692] Updated weights for policy 0, policy_version 1907733 (0.0007) [2023-12-27 05:17:46,773][105692] Updated weights for policy 0, policy_version 1907743 (0.0006) [2023-12-27 05:17:46,832][105692] Updated weights for policy 0, policy_version 1907753 (0.0006) [2023-12-27 05:17:46,846][105620] Updated weights for policy 1, policy_version 1912378 (0.0008) [2023-12-27 05:17:46,909][105620] Updated weights for policy 1, policy_version 1912388 (0.0008) [2023-12-27 05:17:46,973][105620] Updated weights for policy 1, policy_version 1912398 (0.0009) [2023-12-27 05:17:47,427][105692] Updated weights for policy 0, policy_version 1907763 (0.0006) [2023-12-27 05:17:47,481][105692] Updated weights for policy 0, policy_version 1907773 (0.0005) [2023-12-27 05:17:47,527][105692] Updated weights for policy 0, policy_version 1907783 (0.0005) [2023-12-27 05:17:47,798][105620] Updated weights for policy 1, policy_version 1912408 (0.0005) [2023-12-27 05:17:47,864][105620] Updated weights for policy 1, policy_version 1912418 (0.0005) [2023-12-27 05:17:47,912][105620] Updated weights for policy 1, policy_version 1912428 (0.0008) [2023-12-27 05:17:48,156][105692] Updated weights for policy 0, policy_version 1907793 (0.0007) [2023-12-27 05:17:48,207][105692] Updated weights for policy 0, policy_version 1907803 (0.0010) [2023-12-27 05:17:48,252][105692] Updated weights for policy 0, policy_version 1907813 (0.0010) [2023-12-27 05:17:48,300][105692] Updated weights for policy 0, policy_version 1907823 (0.0010) [2023-12-27 05:17:48,607][105620] Updated weights for policy 1, policy_version 1912438 (0.0010) [2023-12-27 05:17:48,658][105620] Updated weights for policy 1, policy_version 1912448 (0.0010) [2023-12-27 05:17:48,707][105620] Updated weights for policy 1, policy_version 1912458 (0.0010) [2023-12-27 05:17:49,081][105692] Updated weights for policy 0, policy_version 1907833 (0.0011) [2023-12-27 05:17:49,143][105692] Updated weights for policy 0, policy_version 1907843 (0.0010) [2023-12-27 05:17:49,198][105692] Updated weights for policy 0, policy_version 1907853 (0.0010) [2023-12-27 05:17:49,476][105620] Updated weights for policy 1, policy_version 1912468 (0.0010) [2023-12-27 05:17:49,531][105620] Updated weights for policy 1, policy_version 1912478 (0.0008) [2023-12-27 05:17:49,586][105620] Updated weights for policy 1, policy_version 1912488 (0.0008) [2023-12-27 05:17:49,962][105692] Updated weights for policy 0, policy_version 1907863 (0.0007) [2023-12-27 05:17:50,026][105692] Updated weights for policy 0, policy_version 1907873 (0.0007) [2023-12-27 05:17:50,089][105692] Updated weights for policy 0, policy_version 1907883 (0.0009) [2023-12-27 05:17:50,403][105620] Updated weights for policy 1, policy_version 1912498 (0.0008) [2023-12-27 05:17:50,457][105620] Updated weights for policy 1, policy_version 1912508 (0.0009) [2023-12-27 05:17:50,514][105620] Updated weights for policy 1, policy_version 1912518 (0.0009) [2023-12-27 05:17:50,669][105692] Updated weights for policy 0, policy_version 1907893 (0.0008) [2023-12-27 05:17:50,731][105692] Updated weights for policy 0, policy_version 1907903 (0.0009) [2023-12-27 05:17:50,794][105692] Updated weights for policy 0, policy_version 1907913 (0.0009) [2023-12-27 05:17:51,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 978173952. Throughput: 0: 9516.1, 1: 9764.1. Samples: 978161796. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:17:51,063][104569] Avg episode reward: [(0, '8451.738'), (1, '9345.670')] [2023-12-27 05:17:51,332][105620] Updated weights for policy 1, policy_version 1912529 (0.0010) [2023-12-27 05:17:51,403][105620] Updated weights for policy 1, policy_version 1912539 (0.0008) [2023-12-27 05:17:51,463][105620] Updated weights for policy 1, policy_version 1912549 (0.0008) [2023-12-27 05:17:51,521][105620] Updated weights for policy 1, policy_version 1912559 (0.0008) [2023-12-27 05:17:51,576][105692] Updated weights for policy 0, policy_version 1907923 (0.0010) [2023-12-27 05:17:51,644][105692] Updated weights for policy 0, policy_version 1907933 (0.0010) [2023-12-27 05:17:51,702][105692] Updated weights for policy 0, policy_version 1907943 (0.0010) [2023-12-27 05:17:52,298][105620] Updated weights for policy 1, policy_version 1912569 (0.0008) [2023-12-27 05:17:52,364][105620] Updated weights for policy 1, policy_version 1912579 (0.0009) [2023-12-27 05:17:52,433][105620] Updated weights for policy 1, policy_version 1912589 (0.0009) [2023-12-27 05:17:52,450][105692] Updated weights for policy 0, policy_version 1907953 (0.0011) [2023-12-27 05:17:52,509][105692] Updated weights for policy 0, policy_version 1907963 (0.0010) [2023-12-27 05:17:52,562][105692] Updated weights for policy 0, policy_version 1907973 (0.0006) [2023-12-27 05:17:52,617][105692] Updated weights for policy 0, policy_version 1907983 (0.0009) [2023-12-27 05:17:53,194][105620] Updated weights for policy 1, policy_version 1912599 (0.0006) [2023-12-27 05:17:53,251][105620] Updated weights for policy 1, policy_version 1912609 (0.0005) [2023-12-27 05:17:53,303][105692] Updated weights for policy 0, policy_version 1907993 (0.0009) [2023-12-27 05:17:53,307][105620] Updated weights for policy 1, policy_version 1912619 (0.0005) [2023-12-27 05:17:53,354][105692] Updated weights for policy 0, policy_version 1908003 (0.0006) [2023-12-27 05:17:53,403][105692] Updated weights for policy 0, policy_version 1908013 (0.0006) [2023-12-27 05:17:53,834][105620] Updated weights for policy 1, policy_version 1912629 (0.0007) [2023-12-27 05:17:53,881][105620] Updated weights for policy 1, policy_version 1912639 (0.0008) [2023-12-27 05:17:53,932][105620] Updated weights for policy 1, policy_version 1912649 (0.0008) [2023-12-27 05:17:54,200][105692] Updated weights for policy 0, policy_version 1908023 (0.0009) [2023-12-27 05:17:54,261][105692] Updated weights for policy 0, policy_version 1908033 (0.0009) [2023-12-27 05:17:54,327][105692] Updated weights for policy 0, policy_version 1908043 (0.0008) [2023-12-27 05:17:54,732][105620] Updated weights for policy 1, policy_version 1912660 (0.0009) [2023-12-27 05:17:54,791][105620] Updated weights for policy 1, policy_version 1912670 (0.0009) [2023-12-27 05:17:54,854][105620] Updated weights for policy 1, policy_version 1912680 (0.0009) [2023-12-27 05:17:55,063][105692] Updated weights for policy 0, policy_version 1908053 (0.0009) [2023-12-27 05:17:55,122][105692] Updated weights for policy 0, policy_version 1908063 (0.0009) [2023-12-27 05:17:55,191][105692] Updated weights for policy 0, policy_version 1908073 (0.0010) [2023-12-27 05:17:55,557][105620] Updated weights for policy 1, policy_version 1912690 (0.0007) [2023-12-27 05:17:55,603][105620] Updated weights for policy 1, policy_version 1912700 (0.0008) [2023-12-27 05:17:55,651][105620] Updated weights for policy 1, policy_version 1912710 (0.0007) [2023-12-27 05:17:55,700][105620] Updated weights for policy 1, policy_version 1912720 (0.0005) [2023-12-27 05:17:55,951][105692] Updated weights for policy 0, policy_version 1908083 (0.0010) [2023-12-27 05:17:56,015][105692] Updated weights for policy 0, policy_version 1908093 (0.0009) [2023-12-27 05:17:56,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19114.7, 300 sec: 19410.9). Total num frames: 978264064. Throughput: 0: 9534.5, 1: 9690.8. Samples: 978276540. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:17:56,062][104569] Avg episode reward: [(0, '8450.265'), (1, '9253.657')] [2023-12-27 05:17:56,079][105692] Updated weights for policy 0, policy_version 1908103 (0.0009) [2023-12-27 05:17:56,320][105620] Updated weights for policy 1, policy_version 1912730 (0.0006) [2023-12-27 05:17:56,385][105620] Updated weights for policy 1, policy_version 1912740 (0.0009) [2023-12-27 05:17:56,443][105620] Updated weights for policy 1, policy_version 1912750 (0.0010) [2023-12-27 05:17:56,865][105692] Updated weights for policy 0, policy_version 1908113 (0.0009) [2023-12-27 05:17:56,909][105692] Updated weights for policy 0, policy_version 1908123 (0.0008) [2023-12-27 05:17:56,957][105692] Updated weights for policy 0, policy_version 1908133 (0.0009) [2023-12-27 05:17:57,013][105692] Updated weights for policy 0, policy_version 1908144 (0.0009) [2023-12-27 05:17:57,146][105620] Updated weights for policy 1, policy_version 1912760 (0.0009) [2023-12-27 05:17:57,193][105620] Updated weights for policy 1, policy_version 1912770 (0.0010) [2023-12-27 05:17:57,248][105620] Updated weights for policy 1, policy_version 1912780 (0.0010) [2023-12-27 05:17:57,676][105692] Updated weights for policy 0, policy_version 1908154 (0.0007) [2023-12-27 05:17:57,729][105692] Updated weights for policy 0, policy_version 1908164 (0.0010) [2023-12-27 05:17:57,782][105692] Updated weights for policy 0, policy_version 1908174 (0.0010) [2023-12-27 05:17:57,825][105620] Updated weights for policy 1, policy_version 1912790 (0.0007) [2023-12-27 05:17:57,877][105620] Updated weights for policy 1, policy_version 1912800 (0.0006) [2023-12-27 05:17:57,926][105620] Updated weights for policy 1, policy_version 1912810 (0.0005) [2023-12-27 05:17:58,512][105692] Updated weights for policy 0, policy_version 1908184 (0.0009) [2023-12-27 05:17:58,574][105692] Updated weights for policy 0, policy_version 1908194 (0.0011) [2023-12-27 05:17:58,632][105620] Updated weights for policy 1, policy_version 1912820 (0.0006) [2023-12-27 05:17:58,633][105692] Updated weights for policy 0, policy_version 1908204 (0.0008) [2023-12-27 05:17:58,692][105620] Updated weights for policy 1, policy_version 1912830 (0.0008) [2023-12-27 05:17:58,755][105620] Updated weights for policy 1, policy_version 1912840 (0.0008) [2023-12-27 05:17:59,362][105692] Updated weights for policy 0, policy_version 1908214 (0.0008) [2023-12-27 05:17:59,428][105692] Updated weights for policy 0, policy_version 1908224 (0.0006) [2023-12-27 05:17:59,488][105692] Updated weights for policy 0, policy_version 1908234 (0.0005) [2023-12-27 05:17:59,526][105620] Updated weights for policy 1, policy_version 1912850 (0.0009) [2023-12-27 05:17:59,593][105620] Updated weights for policy 1, policy_version 1912860 (0.0005) [2023-12-27 05:17:59,659][105620] Updated weights for policy 1, policy_version 1912870 (0.0009) [2023-12-27 05:17:59,724][105620] Updated weights for policy 1, policy_version 1912880 (0.0010) [2023-12-27 05:18:00,119][105692] Updated weights for policy 0, policy_version 1908244 (0.0007) [2023-12-27 05:18:00,177][105692] Updated weights for policy 0, policy_version 1908254 (0.0008) [2023-12-27 05:18:00,234][105692] Updated weights for policy 0, policy_version 1908264 (0.0009) [2023-12-27 05:18:00,441][105620] Updated weights for policy 1, policy_version 1912890 (0.0011) [2023-12-27 05:18:00,507][105620] Updated weights for policy 1, policy_version 1912900 (0.0011) [2023-12-27 05:18:00,565][105620] Updated weights for policy 1, policy_version 1912910 (0.0011) [2023-12-27 05:18:00,988][105692] Updated weights for policy 0, policy_version 1908274 (0.0008) [2023-12-27 05:18:01,043][105692] Updated weights for policy 0, policy_version 1908284 (0.0008) [2023-12-27 05:18:01,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19114.7, 300 sec: 19410.9). Total num frames: 978362368. Throughput: 0: 9560.3, 1: 9731.7. Samples: 978336436. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:18:01,062][104569] Avg episode reward: [(0, '8809.711'), (1, '9161.237')] [2023-12-27 05:18:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001912912_489775104.pth... [2023-12-27 05:18:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001911760_489480192.pth [2023-12-27 05:18:01,103][105692] Updated weights for policy 0, policy_version 1908294 (0.0008) [2023-12-27 05:18:01,159][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001908304_488595456.pth... [2023-12-27 05:18:01,161][105692] Updated weights for policy 0, policy_version 1908304 (0.0009) [2023-12-27 05:18:01,162][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001907152_488300544.pth [2023-12-27 05:18:01,287][105620] Updated weights for policy 1, policy_version 1912920 (0.0009) [2023-12-27 05:18:01,360][105620] Updated weights for policy 1, policy_version 1912930 (0.0010) [2023-12-27 05:18:01,423][105620] Updated weights for policy 1, policy_version 1912940 (0.0008) [2023-12-27 05:18:01,993][105692] Updated weights for policy 0, policy_version 1908314 (0.0009) [2023-12-27 05:18:02,052][105692] Updated weights for policy 0, policy_version 1908324 (0.0009) [2023-12-27 05:18:02,113][105620] Updated weights for policy 1, policy_version 1912950 (0.0008) [2023-12-27 05:18:02,117][105692] Updated weights for policy 0, policy_version 1908334 (0.0009) [2023-12-27 05:18:02,163][105620] Updated weights for policy 1, policy_version 1912960 (0.0009) [2023-12-27 05:18:02,223][105620] Updated weights for policy 1, policy_version 1912970 (0.0011) [2023-12-27 05:18:02,901][105620] Updated weights for policy 1, policy_version 1912980 (0.0010) [2023-12-27 05:18:02,940][105692] Updated weights for policy 0, policy_version 1908344 (0.0007) [2023-12-27 05:18:02,958][105620] Updated weights for policy 1, policy_version 1912990 (0.0010) [2023-12-27 05:18:02,993][105692] Updated weights for policy 0, policy_version 1908354 (0.0009) [2023-12-27 05:18:03,009][105620] Updated weights for policy 1, policy_version 1913000 (0.0011) [2023-12-27 05:18:03,046][105692] Updated weights for policy 0, policy_version 1908364 (0.0010) [2023-12-27 05:18:03,698][105620] Updated weights for policy 1, policy_version 1913010 (0.0010) [2023-12-27 05:18:03,754][105620] Updated weights for policy 1, policy_version 1913020 (0.0005) [2023-12-27 05:18:03,785][105692] Updated weights for policy 0, policy_version 1908374 (0.0007) [2023-12-27 05:18:03,809][105620] Updated weights for policy 1, policy_version 1913030 (0.0005) [2023-12-27 05:18:03,845][105692] Updated weights for policy 0, policy_version 1908384 (0.0007) [2023-12-27 05:18:03,874][105620] Updated weights for policy 1, policy_version 1913040 (0.0008) [2023-12-27 05:18:03,900][105692] Updated weights for policy 0, policy_version 1908394 (0.0007) [2023-12-27 05:18:04,513][105620] Updated weights for policy 1, policy_version 1913050 (0.0008) [2023-12-27 05:18:04,573][105620] Updated weights for policy 1, policy_version 1913060 (0.0008) [2023-12-27 05:18:04,613][105692] Updated weights for policy 0, policy_version 1908404 (0.0006) [2023-12-27 05:18:04,631][105620] Updated weights for policy 1, policy_version 1913070 (0.0008) [2023-12-27 05:18:04,678][105692] Updated weights for policy 0, policy_version 1908414 (0.0006) [2023-12-27 05:18:04,742][105692] Updated weights for policy 0, policy_version 1908424 (0.0008) [2023-12-27 05:18:05,339][105620] Updated weights for policy 1, policy_version 1913080 (0.0008) [2023-12-27 05:18:05,398][105620] Updated weights for policy 1, policy_version 1913090 (0.0009) [2023-12-27 05:18:05,449][105620] Updated weights for policy 1, policy_version 1913100 (0.0009) [2023-12-27 05:18:05,501][105692] Updated weights for policy 0, policy_version 1908434 (0.0009) [2023-12-27 05:18:05,563][105692] Updated weights for policy 0, policy_version 1908444 (0.0009) [2023-12-27 05:18:05,625][105692] Updated weights for policy 0, policy_version 1908454 (0.0008) [2023-12-27 05:18:05,680][105692] Updated weights for policy 0, policy_version 1908464 (0.0005) [2023-12-27 05:18:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 978460672. Throughput: 0: 9568.7, 1: 9743.4. Samples: 978452064. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:18:06,062][104569] Avg episode reward: [(0, '8267.098'), (1, '9161.396')] [2023-12-27 05:18:06,269][105620] Updated weights for policy 1, policy_version 1913110 (0.0008) [2023-12-27 05:18:06,279][105692] Updated weights for policy 0, policy_version 1908474 (0.0008) [2023-12-27 05:18:06,329][105692] Updated weights for policy 0, policy_version 1908484 (0.0006) [2023-12-27 05:18:06,331][105620] Updated weights for policy 1, policy_version 1913120 (0.0008) [2023-12-27 05:18:06,391][105620] Updated weights for policy 1, policy_version 1913130 (0.0007) [2023-12-27 05:18:06,391][105692] Updated weights for policy 0, policy_version 1908494 (0.0008) [2023-12-27 05:18:07,099][105620] Updated weights for policy 1, policy_version 1913140 (0.0009) [2023-12-27 05:18:07,148][105620] Updated weights for policy 1, policy_version 1913150 (0.0008) [2023-12-27 05:18:07,177][105692] Updated weights for policy 0, policy_version 1908504 (0.0007) [2023-12-27 05:18:07,196][105620] Updated weights for policy 1, policy_version 1913160 (0.0008) [2023-12-27 05:18:07,230][105692] Updated weights for policy 0, policy_version 1908514 (0.0008) [2023-12-27 05:18:07,277][105692] Updated weights for policy 0, policy_version 1908524 (0.0009) [2023-12-27 05:18:07,937][105620] Updated weights for policy 1, policy_version 1913170 (0.0006) [2023-12-27 05:18:08,000][105620] Updated weights for policy 1, policy_version 1913180 (0.0006) [2023-12-27 05:18:08,052][105620] Updated weights for policy 1, policy_version 1913190 (0.0010) [2023-12-27 05:18:08,068][105692] Updated weights for policy 0, policy_version 1908534 (0.0007) [2023-12-27 05:18:08,116][105692] Updated weights for policy 0, policy_version 1908544 (0.0007) [2023-12-27 05:18:08,118][105620] Updated weights for policy 1, policy_version 1913200 (0.0011) [2023-12-27 05:18:08,178][105692] Updated weights for policy 0, policy_version 1908554 (0.0006) [2023-12-27 05:18:08,825][105620] Updated weights for policy 1, policy_version 1913210 (0.0011) [2023-12-27 05:18:08,880][105620] Updated weights for policy 1, policy_version 1913220 (0.0010) [2023-12-27 05:18:08,933][105692] Updated weights for policy 0, policy_version 1908565 (0.0009) [2023-12-27 05:18:08,939][105620] Updated weights for policy 1, policy_version 1913230 (0.0010) [2023-12-27 05:18:08,991][105692] Updated weights for policy 0, policy_version 1908575 (0.0007) [2023-12-27 05:18:09,047][105692] Updated weights for policy 0, policy_version 1908585 (0.0008) [2023-12-27 05:18:09,723][105620] Updated weights for policy 1, policy_version 1913240 (0.0011) [2023-12-27 05:18:09,766][105692] Updated weights for policy 0, policy_version 1908595 (0.0008) [2023-12-27 05:18:09,776][105620] Updated weights for policy 1, policy_version 1913250 (0.0011) [2023-12-27 05:18:09,825][105692] Updated weights for policy 0, policy_version 1908605 (0.0007) [2023-12-27 05:18:09,841][105620] Updated weights for policy 1, policy_version 1913260 (0.0011) [2023-12-27 05:18:09,892][105692] Updated weights for policy 0, policy_version 1908615 (0.0007) [2023-12-27 05:18:10,580][105692] Updated weights for policy 0, policy_version 1908625 (0.0008) [2023-12-27 05:18:10,606][105620] Updated weights for policy 1, policy_version 1913270 (0.0010) [2023-12-27 05:18:10,637][105692] Updated weights for policy 0, policy_version 1908635 (0.0006) [2023-12-27 05:18:10,653][105620] Updated weights for policy 1, policy_version 1913280 (0.0007) [2023-12-27 05:18:10,699][105692] Updated weights for policy 0, policy_version 1908645 (0.0008) [2023-12-27 05:18:10,704][105620] Updated weights for policy 1, policy_version 1913290 (0.0006) [2023-12-27 05:18:10,746][105692] Updated weights for policy 0, policy_version 1908655 (0.0006) [2023-12-27 05:18:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 978558976. Throughput: 0: 9581.9, 1: 9727.4. Samples: 978565980. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:18:11,062][104569] Avg episode reward: [(0, '8354.004'), (1, '9069.047')] [2023-12-27 05:18:11,490][105620] Updated weights for policy 1, policy_version 1913300 (0.0009) [2023-12-27 05:18:11,494][105692] Updated weights for policy 0, policy_version 1908665 (0.0006) [2023-12-27 05:18:11,552][105692] Updated weights for policy 0, policy_version 1908675 (0.0006) [2023-12-27 05:18:11,553][105620] Updated weights for policy 1, policy_version 1913310 (0.0007) [2023-12-27 05:18:11,610][105692] Updated weights for policy 0, policy_version 1908685 (0.0006) [2023-12-27 05:18:11,621][105620] Updated weights for policy 1, policy_version 1913320 (0.0008) [2023-12-27 05:18:12,340][105620] Updated weights for policy 1, policy_version 1913330 (0.0008) [2023-12-27 05:18:12,408][105692] Updated weights for policy 0, policy_version 1908695 (0.0009) [2023-12-27 05:18:12,412][105620] Updated weights for policy 1, policy_version 1913340 (0.0008) [2023-12-27 05:18:12,467][105692] Updated weights for policy 0, policy_version 1908705 (0.0008) [2023-12-27 05:18:12,476][105620] Updated weights for policy 1, policy_version 1913350 (0.0006) [2023-12-27 05:18:12,529][105692] Updated weights for policy 0, policy_version 1908715 (0.0006) [2023-12-27 05:18:12,535][105620] Updated weights for policy 1, policy_version 1913360 (0.0006) [2023-12-27 05:18:13,159][105620] Updated weights for policy 1, policy_version 1913370 (0.0008) [2023-12-27 05:18:13,220][105620] Updated weights for policy 1, policy_version 1913380 (0.0009) [2023-12-27 05:18:13,273][105692] Updated weights for policy 0, policy_version 1908725 (0.0006) [2023-12-27 05:18:13,281][105620] Updated weights for policy 1, policy_version 1913390 (0.0008) [2023-12-27 05:18:13,322][105692] Updated weights for policy 0, policy_version 1908735 (0.0008) [2023-12-27 05:18:13,370][105692] Updated weights for policy 0, policy_version 1908745 (0.0009) [2023-12-27 05:18:13,983][105692] Updated weights for policy 0, policy_version 1908755 (0.0008) [2023-12-27 05:18:14,022][105620] Updated weights for policy 1, policy_version 1913400 (0.0007) [2023-12-27 05:18:14,046][105692] Updated weights for policy 0, policy_version 1908765 (0.0005) [2023-12-27 05:18:14,080][105620] Updated weights for policy 1, policy_version 1913410 (0.0006) [2023-12-27 05:18:14,118][105692] Updated weights for policy 0, policy_version 1908775 (0.0007) [2023-12-27 05:18:14,128][105620] Updated weights for policy 1, policy_version 1913420 (0.0008) [2023-12-27 05:18:14,626][105692] Updated weights for policy 0, policy_version 1908785 (0.0006) [2023-12-27 05:18:14,685][105692] Updated weights for policy 0, policy_version 1908795 (0.0009) [2023-12-27 05:18:14,741][105692] Updated weights for policy 0, policy_version 1908805 (0.0009) [2023-12-27 05:18:14,807][105692] Updated weights for policy 0, policy_version 1908815 (0.0009) [2023-12-27 05:18:14,946][105620] Updated weights for policy 1, policy_version 1913430 (0.0007) [2023-12-27 05:18:15,016][105620] Updated weights for policy 1, policy_version 1913440 (0.0007) [2023-12-27 05:18:15,082][105620] Updated weights for policy 1, policy_version 1913450 (0.0009) [2023-12-27 05:18:15,558][105692] Updated weights for policy 0, policy_version 1908825 (0.0009) [2023-12-27 05:18:15,626][105692] Updated weights for policy 0, policy_version 1908835 (0.0009) [2023-12-27 05:18:15,674][105692] Updated weights for policy 0, policy_version 1908845 (0.0009) [2023-12-27 05:18:15,823][105620] Updated weights for policy 1, policy_version 1913460 (0.0009) [2023-12-27 05:18:15,884][105620] Updated weights for policy 1, policy_version 1913470 (0.0009) [2023-12-27 05:18:15,945][105620] Updated weights for policy 1, policy_version 1913480 (0.0009) [2023-12-27 05:18:16,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 978657280. Throughput: 0: 9531.6, 1: 9732.5. Samples: 978622652. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:18:16,062][104569] Avg episode reward: [(0, '8627.669'), (1, '9160.967')] [2023-12-27 05:18:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001913488_489922560.pth... [2023-12-27 05:18:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001908848_488734720.pth... [2023-12-27 05:18:16,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001912368_489635840.pth [2023-12-27 05:18:16,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001907728_488448000.pth [2023-12-27 05:18:16,412][105692] Updated weights for policy 0, policy_version 1908855 (0.0007) [2023-12-27 05:18:16,474][105692] Updated weights for policy 0, policy_version 1908865 (0.0007) [2023-12-27 05:18:16,536][105692] Updated weights for policy 0, policy_version 1908875 (0.0005) [2023-12-27 05:18:16,631][105620] Updated weights for policy 1, policy_version 1913490 (0.0008) [2023-12-27 05:18:16,694][105620] Updated weights for policy 1, policy_version 1913500 (0.0009) [2023-12-27 05:18:16,746][105620] Updated weights for policy 1, policy_version 1913510 (0.0009) [2023-12-27 05:18:16,812][105620] Updated weights for policy 1, policy_version 1913520 (0.0006) [2023-12-27 05:18:17,155][105692] Updated weights for policy 0, policy_version 1908885 (0.0007) [2023-12-27 05:18:17,223][105692] Updated weights for policy 0, policy_version 1908895 (0.0006) [2023-12-27 05:18:17,295][105692] Updated weights for policy 0, policy_version 1908905 (0.0005) [2023-12-27 05:18:17,487][105620] Updated weights for policy 1, policy_version 1913530 (0.0010) [2023-12-27 05:18:17,546][105620] Updated weights for policy 1, policy_version 1913540 (0.0008) [2023-12-27 05:18:17,602][105620] Updated weights for policy 1, policy_version 1913550 (0.0006) [2023-12-27 05:18:17,804][105692] Updated weights for policy 0, policy_version 1908915 (0.0005) [2023-12-27 05:18:17,857][105692] Updated weights for policy 0, policy_version 1908925 (0.0005) [2023-12-27 05:18:17,908][105692] Updated weights for policy 0, policy_version 1908935 (0.0005) [2023-12-27 05:18:18,267][105620] Updated weights for policy 1, policy_version 1913560 (0.0009) [2023-12-27 05:18:18,321][105620] Updated weights for policy 1, policy_version 1913570 (0.0010) [2023-12-27 05:18:18,387][105620] Updated weights for policy 1, policy_version 1913580 (0.0011) [2023-12-27 05:18:18,496][105692] Updated weights for policy 0, policy_version 1908945 (0.0006) [2023-12-27 05:18:18,556][105692] Updated weights for policy 0, policy_version 1908955 (0.0008) [2023-12-27 05:18:18,613][105692] Updated weights for policy 0, policy_version 1908965 (0.0008) [2023-12-27 05:18:18,668][105692] Updated weights for policy 0, policy_version 1908975 (0.0008) [2023-12-27 05:18:19,082][105620] Updated weights for policy 1, policy_version 1913590 (0.0010) [2023-12-27 05:18:19,133][105620] Updated weights for policy 1, policy_version 1913600 (0.0010) [2023-12-27 05:18:19,181][105620] Updated weights for policy 1, policy_version 1913610 (0.0010) [2023-12-27 05:18:19,427][105692] Updated weights for policy 0, policy_version 1908985 (0.0006) [2023-12-27 05:18:19,501][105692] Updated weights for policy 0, policy_version 1908995 (0.0007) [2023-12-27 05:18:19,561][105692] Updated weights for policy 0, policy_version 1909005 (0.0009) [2023-12-27 05:18:19,822][105620] Updated weights for policy 1, policy_version 1913620 (0.0010) [2023-12-27 05:18:19,889][105620] Updated weights for policy 1, policy_version 1913630 (0.0011) [2023-12-27 05:18:19,958][105620] Updated weights for policy 1, policy_version 1913640 (0.0011) [2023-12-27 05:18:20,260][105692] Updated weights for policy 0, policy_version 1909015 (0.0009) [2023-12-27 05:18:20,313][105692] Updated weights for policy 0, policy_version 1909025 (0.0008) [2023-12-27 05:18:20,370][105692] Updated weights for policy 0, policy_version 1909035 (0.0008) [2023-12-27 05:18:20,670][105620] Updated weights for policy 1, policy_version 1913650 (0.0010) [2023-12-27 05:18:20,726][105620] Updated weights for policy 1, policy_version 1913660 (0.0011) [2023-12-27 05:18:20,778][105620] Updated weights for policy 1, policy_version 1913670 (0.0010) [2023-12-27 05:18:20,831][105620] Updated weights for policy 1, policy_version 1913680 (0.0010) [2023-12-27 05:18:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 978755584. Throughput: 0: 9670.2, 1: 9708.7. Samples: 978746184. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:18:21,062][104569] Avg episode reward: [(0, '8265.617'), (1, '9161.423')] [2023-12-27 05:18:21,122][105692] Updated weights for policy 0, policy_version 1909045 (0.0007) [2023-12-27 05:18:21,179][105692] Updated weights for policy 0, policy_version 1909055 (0.0008) [2023-12-27 05:18:21,236][105692] Updated weights for policy 0, policy_version 1909065 (0.0008) [2023-12-27 05:18:21,527][105620] Updated weights for policy 1, policy_version 1913690 (0.0007) [2023-12-27 05:18:21,593][105620] Updated weights for policy 1, policy_version 1913700 (0.0007) [2023-12-27 05:18:21,661][105620] Updated weights for policy 1, policy_version 1913710 (0.0011) [2023-12-27 05:18:22,073][105692] Updated weights for policy 0, policy_version 1909075 (0.0009) [2023-12-27 05:18:22,133][105692] Updated weights for policy 0, policy_version 1909085 (0.0009) [2023-12-27 05:18:22,196][105692] Updated weights for policy 0, policy_version 1909095 (0.0010) [2023-12-27 05:18:22,321][105620] Updated weights for policy 1, policy_version 1913720 (0.0008) [2023-12-27 05:18:22,390][105620] Updated weights for policy 1, policy_version 1913730 (0.0008) [2023-12-27 05:18:22,464][105620] Updated weights for policy 1, policy_version 1913740 (0.0008) [2023-12-27 05:18:22,996][105692] Updated weights for policy 0, policy_version 1909105 (0.0010) [2023-12-27 05:18:23,047][105692] Updated weights for policy 0, policy_version 1909115 (0.0009) [2023-12-27 05:18:23,095][105692] Updated weights for policy 0, policy_version 1909125 (0.0008) [2023-12-27 05:18:23,150][105692] Updated weights for policy 0, policy_version 1909135 (0.0009) [2023-12-27 05:18:23,204][105620] Updated weights for policy 1, policy_version 1913750 (0.0009) [2023-12-27 05:18:23,256][105620] Updated weights for policy 1, policy_version 1913760 (0.0009) [2023-12-27 05:18:23,304][105620] Updated weights for policy 1, policy_version 1913770 (0.0009) [2023-12-27 05:18:23,906][105692] Updated weights for policy 0, policy_version 1909145 (0.0007) [2023-12-27 05:18:23,951][105692] Updated weights for policy 0, policy_version 1909155 (0.0008) [2023-12-27 05:18:24,003][105692] Updated weights for policy 0, policy_version 1909165 (0.0008) [2023-12-27 05:18:24,081][105620] Updated weights for policy 1, policy_version 1913780 (0.0009) [2023-12-27 05:18:24,138][105620] Updated weights for policy 1, policy_version 1913790 (0.0011) [2023-12-27 05:18:24,194][105620] Updated weights for policy 1, policy_version 1913800 (0.0011) [2023-12-27 05:18:24,782][105692] Updated weights for policy 0, policy_version 1909175 (0.0010) [2023-12-27 05:18:24,839][105692] Updated weights for policy 0, policy_version 1909185 (0.0010) [2023-12-27 05:18:24,901][105692] Updated weights for policy 0, policy_version 1909195 (0.0010) [2023-12-27 05:18:24,955][105620] Updated weights for policy 1, policy_version 1913810 (0.0011) [2023-12-27 05:18:25,000][105620] Updated weights for policy 1, policy_version 1913820 (0.0010) [2023-12-27 05:18:25,057][105620] Updated weights for policy 1, policy_version 1913830 (0.0007) [2023-12-27 05:18:25,111][105620] Updated weights for policy 1, policy_version 1913840 (0.0010) [2023-12-27 05:18:25,650][105692] Updated weights for policy 0, policy_version 1909205 (0.0010) [2023-12-27 05:18:25,713][105692] Updated weights for policy 0, policy_version 1909215 (0.0011) [2023-12-27 05:18:25,736][105620] Updated weights for policy 1, policy_version 1913850 (0.0006) [2023-12-27 05:18:25,772][105692] Updated weights for policy 0, policy_version 1909225 (0.0010) [2023-12-27 05:18:25,795][105620] Updated weights for policy 1, policy_version 1913860 (0.0006) [2023-12-27 05:18:25,859][105620] Updated weights for policy 1, policy_version 1913870 (0.0009) [2023-12-27 05:18:26,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 978853888. Throughput: 0: 9652.5, 1: 9755.6. Samples: 978859808. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:18:26,063][104569] Avg episode reward: [(0, '8177.230'), (1, '9253.769')] [2023-12-27 05:18:26,418][105620] Updated weights for policy 1, policy_version 1913880 (0.0006) [2023-12-27 05:18:26,485][105620] Updated weights for policy 1, policy_version 1913890 (0.0005) [2023-12-27 05:18:26,554][105620] Updated weights for policy 1, policy_version 1913900 (0.0005) [2023-12-27 05:18:26,633][105692] Updated weights for policy 0, policy_version 1909235 (0.0009) [2023-12-27 05:18:26,686][105692] Updated weights for policy 0, policy_version 1909245 (0.0005) [2023-12-27 05:18:26,735][105692] Updated weights for policy 0, policy_version 1909255 (0.0005) [2023-12-27 05:18:27,047][105620] Updated weights for policy 1, policy_version 1913910 (0.0005) [2023-12-27 05:18:27,117][105620] Updated weights for policy 1, policy_version 1913920 (0.0009) [2023-12-27 05:18:27,178][105620] Updated weights for policy 1, policy_version 1913930 (0.0010) [2023-12-27 05:18:27,261][105692] Updated weights for policy 0, policy_version 1909265 (0.0006) [2023-12-27 05:18:27,316][105692] Updated weights for policy 0, policy_version 1909275 (0.0008) [2023-12-27 05:18:27,367][105692] Updated weights for policy 0, policy_version 1909285 (0.0010) [2023-12-27 05:18:27,420][105692] Updated weights for policy 0, policy_version 1909295 (0.0010) [2023-12-27 05:18:27,744][105620] Updated weights for policy 1, policy_version 1913940 (0.0008) [2023-12-27 05:18:27,793][105620] Updated weights for policy 1, policy_version 1913950 (0.0005) [2023-12-27 05:18:27,844][105620] Updated weights for policy 1, policy_version 1913960 (0.0005) [2023-12-27 05:18:28,255][105692] Updated weights for policy 0, policy_version 1909305 (0.0010) [2023-12-27 05:18:28,302][105692] Updated weights for policy 0, policy_version 1909315 (0.0006) [2023-12-27 05:18:28,361][105692] Updated weights for policy 0, policy_version 1909325 (0.0007) [2023-12-27 05:18:28,458][105620] Updated weights for policy 1, policy_version 1913970 (0.0007) [2023-12-27 05:18:28,524][105620] Updated weights for policy 1, policy_version 1913980 (0.0010) [2023-12-27 05:18:28,579][105620] Updated weights for policy 1, policy_version 1913990 (0.0009) [2023-12-27 05:18:28,634][105620] Updated weights for policy 1, policy_version 1914000 (0.0009) [2023-12-27 05:18:28,962][105692] Updated weights for policy 0, policy_version 1909335 (0.0007) [2023-12-27 05:18:29,018][105692] Updated weights for policy 0, policy_version 1909345 (0.0011) [2023-12-27 05:18:29,076][105692] Updated weights for policy 0, policy_version 1909355 (0.0010) [2023-12-27 05:18:29,494][105620] Updated weights for policy 1, policy_version 1914010 (0.0008) [2023-12-27 05:18:29,562][105620] Updated weights for policy 1, policy_version 1914020 (0.0008) [2023-12-27 05:18:29,629][105620] Updated weights for policy 1, policy_version 1914030 (0.0009) [2023-12-27 05:18:29,756][105692] Updated weights for policy 0, policy_version 1909365 (0.0009) [2023-12-27 05:18:29,818][105692] Updated weights for policy 0, policy_version 1909375 (0.0011) [2023-12-27 05:18:29,879][105692] Updated weights for policy 0, policy_version 1909385 (0.0009) [2023-12-27 05:18:30,382][105620] Updated weights for policy 1, policy_version 1914040 (0.0008) [2023-12-27 05:18:30,437][105620] Updated weights for policy 1, policy_version 1914050 (0.0008) [2023-12-27 05:18:30,494][105620] Updated weights for policy 1, policy_version 1914060 (0.0006) [2023-12-27 05:18:30,626][105692] Updated weights for policy 0, policy_version 1909395 (0.0011) [2023-12-27 05:18:30,690][105692] Updated weights for policy 0, policy_version 1909405 (0.0010) [2023-12-27 05:18:30,755][105692] Updated weights for policy 0, policy_version 1909415 (0.0010) [2023-12-27 05:18:31,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 978952192. Throughput: 0: 9685.9, 1: 9848.1. Samples: 978922628. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:18:31,063][104569] Avg episode reward: [(0, '8353.984'), (1, '9162.153')] [2023-12-27 05:18:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001909424_488882176.pth... [2023-12-27 05:18:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001914064_490070016.pth... [2023-12-27 05:18:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001908304_488595456.pth [2023-12-27 05:18:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001912912_489775104.pth [2023-12-27 05:18:31,268][105620] Updated weights for policy 1, policy_version 1914070 (0.0008) [2023-12-27 05:18:31,328][105620] Updated weights for policy 1, policy_version 1914080 (0.0008) [2023-12-27 05:18:31,390][105620] Updated weights for policy 1, policy_version 1914090 (0.0008) [2023-12-27 05:18:31,441][105692] Updated weights for policy 0, policy_version 1909425 (0.0010) [2023-12-27 05:18:31,510][105692] Updated weights for policy 0, policy_version 1909435 (0.0006) [2023-12-27 05:18:31,571][105692] Updated weights for policy 0, policy_version 1909446 (0.0009) [2023-12-27 05:18:31,633][105692] Updated weights for policy 0, policy_version 1909456 (0.0008) [2023-12-27 05:18:32,085][105620] Updated weights for policy 1, policy_version 1914100 (0.0009) [2023-12-27 05:18:32,136][105620] Updated weights for policy 1, policy_version 1914110 (0.0009) [2023-12-27 05:18:32,197][105620] Updated weights for policy 1, policy_version 1914120 (0.0009) [2023-12-27 05:18:32,359][105692] Updated weights for policy 0, policy_version 1909466 (0.0009) [2023-12-27 05:18:32,419][105692] Updated weights for policy 0, policy_version 1909476 (0.0009) [2023-12-27 05:18:32,477][105692] Updated weights for policy 0, policy_version 1909486 (0.0010) [2023-12-27 05:18:32,960][105620] Updated weights for policy 1, policy_version 1914130 (0.0009) [2023-12-27 05:18:33,025][105620] Updated weights for policy 1, policy_version 1914140 (0.0009) [2023-12-27 05:18:33,092][105620] Updated weights for policy 1, policy_version 1914150 (0.0009) [2023-12-27 05:18:33,155][105620] Updated weights for policy 1, policy_version 1914160 (0.0009) [2023-12-27 05:18:33,171][105692] Updated weights for policy 0, policy_version 1909496 (0.0007) [2023-12-27 05:18:33,229][105692] Updated weights for policy 0, policy_version 1909506 (0.0006) [2023-12-27 05:18:33,280][105692] Updated weights for policy 0, policy_version 1909516 (0.0005) [2023-12-27 05:18:33,898][105692] Updated weights for policy 0, policy_version 1909526 (0.0007) [2023-12-27 05:18:33,940][105620] Updated weights for policy 1, policy_version 1914170 (0.0008) [2023-12-27 05:18:33,951][105692] Updated weights for policy 0, policy_version 1909536 (0.0008) [2023-12-27 05:18:33,982][105620] Updated weights for policy 1, policy_version 1914180 (0.0008) [2023-12-27 05:18:34,016][105692] Updated weights for policy 0, policy_version 1909546 (0.0006) [2023-12-27 05:18:34,032][105620] Updated weights for policy 1, policy_version 1914190 (0.0008) [2023-12-27 05:18:34,763][105692] Updated weights for policy 0, policy_version 1909556 (0.0006) [2023-12-27 05:18:34,822][105692] Updated weights for policy 0, policy_version 1909566 (0.0007) [2023-12-27 05:18:34,845][105620] Updated weights for policy 1, policy_version 1914200 (0.0007) [2023-12-27 05:18:34,881][105692] Updated weights for policy 0, policy_version 1909576 (0.0009) [2023-12-27 05:18:34,906][105620] Updated weights for policy 1, policy_version 1914210 (0.0009) [2023-12-27 05:18:34,958][105620] Updated weights for policy 1, policy_version 1914220 (0.0010) [2023-12-27 05:18:35,564][105692] Updated weights for policy 0, policy_version 1909586 (0.0006) [2023-12-27 05:18:35,619][105692] Updated weights for policy 0, policy_version 1909596 (0.0009) [2023-12-27 05:18:35,662][105620] Updated weights for policy 1, policy_version 1914231 (0.0012) [2023-12-27 05:18:35,690][105692] Updated weights for policy 0, policy_version 1909606 (0.0010) [2023-12-27 05:18:35,729][105620] Updated weights for policy 1, policy_version 1914241 (0.0007) [2023-12-27 05:18:35,747][105692] Updated weights for policy 0, policy_version 1909616 (0.0011) [2023-12-27 05:18:35,793][105620] Updated weights for policy 1, policy_version 1914251 (0.0007) [2023-12-27 05:18:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 979050496. Throughput: 0: 9697.4, 1: 9759.9. Samples: 979037368. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:18:36,062][104569] Avg episode reward: [(0, '8808.037'), (1, '9162.164')] [2023-12-27 05:18:36,490][105692] Updated weights for policy 0, policy_version 1909626 (0.0009) [2023-12-27 05:18:36,552][105620] Updated weights for policy 1, policy_version 1914261 (0.0009) [2023-12-27 05:18:36,555][105692] Updated weights for policy 0, policy_version 1909636 (0.0009) [2023-12-27 05:18:36,614][105620] Updated weights for policy 1, policy_version 1914271 (0.0008) [2023-12-27 05:18:36,620][105692] Updated weights for policy 0, policy_version 1909646 (0.0007) [2023-12-27 05:18:36,674][105620] Updated weights for policy 1, policy_version 1914281 (0.0008) [2023-12-27 05:18:37,333][105692] Updated weights for policy 0, policy_version 1909656 (0.0005) [2023-12-27 05:18:37,382][105692] Updated weights for policy 0, policy_version 1909666 (0.0006) [2023-12-27 05:18:37,426][105620] Updated weights for policy 1, policy_version 1914291 (0.0010) [2023-12-27 05:18:37,439][105692] Updated weights for policy 0, policy_version 1909676 (0.0005) [2023-12-27 05:18:37,485][105620] Updated weights for policy 1, policy_version 1914301 (0.0009) [2023-12-27 05:18:37,539][105620] Updated weights for policy 1, policy_version 1914312 (0.0010) [2023-12-27 05:18:38,030][105692] Updated weights for policy 0, policy_version 1909686 (0.0008) [2023-12-27 05:18:38,084][105692] Updated weights for policy 0, policy_version 1909696 (0.0010) [2023-12-27 05:18:38,148][105692] Updated weights for policy 0, policy_version 1909706 (0.0010) [2023-12-27 05:18:38,221][105620] Updated weights for policy 1, policy_version 1914322 (0.0010) [2023-12-27 05:18:38,279][105620] Updated weights for policy 1, policy_version 1914332 (0.0010) [2023-12-27 05:18:38,351][105620] Updated weights for policy 1, policy_version 1914342 (0.0011) [2023-12-27 05:18:38,414][105620] Updated weights for policy 1, policy_version 1914352 (0.0010) [2023-12-27 05:18:38,911][105692] Updated weights for policy 0, policy_version 1909716 (0.0009) [2023-12-27 05:18:38,971][105692] Updated weights for policy 0, policy_version 1909726 (0.0008) [2023-12-27 05:18:39,027][105692] Updated weights for policy 0, policy_version 1909736 (0.0008) [2023-12-27 05:18:39,089][105620] Updated weights for policy 1, policy_version 1914362 (0.0010) [2023-12-27 05:18:39,134][105620] Updated weights for policy 1, policy_version 1914372 (0.0010) [2023-12-27 05:18:39,186][105620] Updated weights for policy 1, policy_version 1914382 (0.0010) [2023-12-27 05:18:39,860][105692] Updated weights for policy 0, policy_version 1909746 (0.0008) [2023-12-27 05:18:39,889][105620] Updated weights for policy 1, policy_version 1914392 (0.0009) [2023-12-27 05:18:39,913][105692] Updated weights for policy 0, policy_version 1909756 (0.0007) [2023-12-27 05:18:39,942][105620] Updated weights for policy 1, policy_version 1914402 (0.0008) [2023-12-27 05:18:39,976][105692] Updated weights for policy 0, policy_version 1909766 (0.0008) [2023-12-27 05:18:40,009][105620] Updated weights for policy 1, policy_version 1914412 (0.0008) [2023-12-27 05:18:40,040][105692] Updated weights for policy 0, policy_version 1909776 (0.0007) [2023-12-27 05:18:40,721][105620] Updated weights for policy 1, policy_version 1914422 (0.0009) [2023-12-27 05:18:40,779][105620] Updated weights for policy 1, policy_version 1914432 (0.0007) [2023-12-27 05:18:40,798][105692] Updated weights for policy 0, policy_version 1909786 (0.0009) [2023-12-27 05:18:40,833][105620] Updated weights for policy 1, policy_version 1914442 (0.0008) [2023-12-27 05:18:40,847][105692] Updated weights for policy 0, policy_version 1909796 (0.0006) [2023-12-27 05:18:40,903][105692] Updated weights for policy 0, policy_version 1909806 (0.0007) [2023-12-27 05:18:41,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 979148800. Throughput: 0: 9697.6, 1: 9777.5. Samples: 979152920. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:18:41,062][104569] Avg episode reward: [(0, '8811.230'), (1, '9345.770')] [2023-12-27 05:18:41,563][105620] Updated weights for policy 1, policy_version 1914452 (0.0008) [2023-12-27 05:18:41,615][105620] Updated weights for policy 1, policy_version 1914462 (0.0009) [2023-12-27 05:18:41,680][105620] Updated weights for policy 1, policy_version 1914472 (0.0008) [2023-12-27 05:18:41,739][105692] Updated weights for policy 0, policy_version 1909816 (0.0008) [2023-12-27 05:18:41,804][105692] Updated weights for policy 0, policy_version 1909826 (0.0010) [2023-12-27 05:18:41,867][105692] Updated weights for policy 0, policy_version 1909836 (0.0010) [2023-12-27 05:18:42,484][105620] Updated weights for policy 1, policy_version 1914482 (0.0007) [2023-12-27 05:18:42,543][105620] Updated weights for policy 1, policy_version 1914492 (0.0011) [2023-12-27 05:18:42,577][105692] Updated weights for policy 0, policy_version 1909846 (0.0007) [2023-12-27 05:18:42,603][105620] Updated weights for policy 1, policy_version 1914502 (0.0011) [2023-12-27 05:18:42,634][105692] Updated weights for policy 0, policy_version 1909856 (0.0006) [2023-12-27 05:18:42,659][105620] Updated weights for policy 1, policy_version 1914512 (0.0010) [2023-12-27 05:18:42,697][105692] Updated weights for policy 0, policy_version 1909866 (0.0007) [2023-12-27 05:18:43,327][105620] Updated weights for policy 1, policy_version 1914522 (0.0010) [2023-12-27 05:18:43,388][105620] Updated weights for policy 1, policy_version 1914532 (0.0011) [2023-12-27 05:18:43,393][105692] Updated weights for policy 0, policy_version 1909876 (0.0007) [2023-12-27 05:18:43,443][105692] Updated weights for policy 0, policy_version 1909886 (0.0005) [2023-12-27 05:18:43,445][105620] Updated weights for policy 1, policy_version 1914542 (0.0006) [2023-12-27 05:18:43,489][105692] Updated weights for policy 0, policy_version 1909896 (0.0005) [2023-12-27 05:18:43,977][105620] Updated weights for policy 1, policy_version 1914552 (0.0005) [2023-12-27 05:18:44,021][105620] Updated weights for policy 1, policy_version 1914562 (0.0005) [2023-12-27 05:18:44,074][105620] Updated weights for policy 1, policy_version 1914572 (0.0006) [2023-12-27 05:18:44,092][105692] Updated weights for policy 0, policy_version 1909906 (0.0006) [2023-12-27 05:18:44,147][105692] Updated weights for policy 0, policy_version 1909916 (0.0010) [2023-12-27 05:18:44,202][105692] Updated weights for policy 0, policy_version 1909926 (0.0010) [2023-12-27 05:18:44,249][105692] Updated weights for policy 0, policy_version 1909936 (0.0010) [2023-12-27 05:18:44,746][105620] Updated weights for policy 1, policy_version 1914582 (0.0007) [2023-12-27 05:18:44,810][105620] Updated weights for policy 1, policy_version 1914592 (0.0007) [2023-12-27 05:18:44,874][105620] Updated weights for policy 1, policy_version 1914602 (0.0008) [2023-12-27 05:18:45,064][105692] Updated weights for policy 0, policy_version 1909946 (0.0006) [2023-12-27 05:18:45,125][105692] Updated weights for policy 0, policy_version 1909956 (0.0008) [2023-12-27 05:18:45,190][105692] Updated weights for policy 0, policy_version 1909966 (0.0008) [2023-12-27 05:18:45,607][105620] Updated weights for policy 1, policy_version 1914612 (0.0007) [2023-12-27 05:18:45,655][105620] Updated weights for policy 1, policy_version 1914622 (0.0009) [2023-12-27 05:18:45,706][105620] Updated weights for policy 1, policy_version 1914632 (0.0009) [2023-12-27 05:18:45,875][105692] Updated weights for policy 0, policy_version 1909976 (0.0009) [2023-12-27 05:18:45,925][105692] Updated weights for policy 0, policy_version 1909986 (0.0009) [2023-12-27 05:18:45,977][105692] Updated weights for policy 0, policy_version 1909996 (0.0009) [2023-12-27 05:18:46,062][104569] Fps is (10 sec: 19660.3, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 979247104. Throughput: 0: 9686.3, 1: 9770.0. Samples: 979211972. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:18:46,063][104569] Avg episode reward: [(0, '8726.602'), (1, '9345.756')] [2023-12-27 05:18:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001910000_489029632.pth... [2023-12-27 05:18:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001914640_490217472.pth... [2023-12-27 05:18:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001913488_489922560.pth [2023-12-27 05:18:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001908848_488734720.pth [2023-12-27 05:18:46,438][105620] Updated weights for policy 1, policy_version 1914642 (0.0008) [2023-12-27 05:18:46,488][105620] Updated weights for policy 1, policy_version 1914652 (0.0005) [2023-12-27 05:18:46,537][105620] Updated weights for policy 1, policy_version 1914662 (0.0005) [2023-12-27 05:18:46,583][105620] Updated weights for policy 1, policy_version 1914672 (0.0005) [2023-12-27 05:18:46,785][105692] Updated weights for policy 0, policy_version 1910006 (0.0007) [2023-12-27 05:18:46,833][105692] Updated weights for policy 0, policy_version 1910016 (0.0006) [2023-12-27 05:18:46,883][105692] Updated weights for policy 0, policy_version 1910026 (0.0006) [2023-12-27 05:18:47,362][105620] Updated weights for policy 1, policy_version 1914682 (0.0010) [2023-12-27 05:18:47,415][105620] Updated weights for policy 1, policy_version 1914692 (0.0010) [2023-12-27 05:18:47,455][105692] Updated weights for policy 0, policy_version 1910036 (0.0005) [2023-12-27 05:18:47,468][105620] Updated weights for policy 1, policy_version 1914702 (0.0009) [2023-12-27 05:18:47,513][105692] Updated weights for policy 0, policy_version 1910046 (0.0005) [2023-12-27 05:18:47,564][105692] Updated weights for policy 0, policy_version 1910056 (0.0005) [2023-12-27 05:18:48,194][105692] Updated weights for policy 0, policy_version 1910066 (0.0005) [2023-12-27 05:18:48,212][105620] Updated weights for policy 1, policy_version 1914712 (0.0010) [2023-12-27 05:18:48,246][105692] Updated weights for policy 0, policy_version 1910076 (0.0005) [2023-12-27 05:18:48,267][105620] Updated weights for policy 1, policy_version 1914722 (0.0010) [2023-12-27 05:18:48,297][105692] Updated weights for policy 0, policy_version 1910086 (0.0005) [2023-12-27 05:18:48,333][105620] Updated weights for policy 1, policy_version 1914732 (0.0011) [2023-12-27 05:18:48,353][105692] Updated weights for policy 0, policy_version 1910096 (0.0007) [2023-12-27 05:18:49,085][105620] Updated weights for policy 1, policy_version 1914742 (0.0011) [2023-12-27 05:18:49,123][105692] Updated weights for policy 0, policy_version 1910106 (0.0006) [2023-12-27 05:18:49,145][105620] Updated weights for policy 1, policy_version 1914752 (0.0011) [2023-12-27 05:18:49,175][105692] Updated weights for policy 0, policy_version 1910116 (0.0006) [2023-12-27 05:18:49,193][105620] Updated weights for policy 1, policy_version 1914762 (0.0010) [2023-12-27 05:18:49,234][105692] Updated weights for policy 0, policy_version 1910126 (0.0006) [2023-12-27 05:18:49,956][105620] Updated weights for policy 1, policy_version 1914772 (0.0010) [2023-12-27 05:18:49,957][105692] Updated weights for policy 0, policy_version 1910136 (0.0008) [2023-12-27 05:18:50,018][105692] Updated weights for policy 0, policy_version 1910146 (0.0010) [2023-12-27 05:18:50,024][105620] Updated weights for policy 1, policy_version 1914782 (0.0009) [2023-12-27 05:18:50,078][105692] Updated weights for policy 0, policy_version 1910156 (0.0006) [2023-12-27 05:18:50,085][105620] Updated weights for policy 1, policy_version 1914792 (0.0009) [2023-12-27 05:18:50,674][105620] Updated weights for policy 1, policy_version 1914802 (0.0007) [2023-12-27 05:18:50,734][105620] Updated weights for policy 1, policy_version 1914812 (0.0010) [2023-12-27 05:18:50,785][105620] Updated weights for policy 1, policy_version 1914822 (0.0011) [2023-12-27 05:18:50,842][105620] Updated weights for policy 1, policy_version 1914832 (0.0011) [2023-12-27 05:18:50,950][105692] Updated weights for policy 0, policy_version 1910166 (0.0007) [2023-12-27 05:18:51,009][105692] Updated weights for policy 0, policy_version 1910176 (0.0008) [2023-12-27 05:18:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 979337216. Throughput: 0: 9753.5, 1: 9730.8. Samples: 979328860. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:18:51,062][104569] Avg episode reward: [(0, '8542.930'), (1, '9345.754')] [2023-12-27 05:18:51,077][105692] Updated weights for policy 0, policy_version 1910186 (0.0008) [2023-12-27 05:18:51,596][105620] Updated weights for policy 1, policy_version 1914842 (0.0011) [2023-12-27 05:18:51,661][105620] Updated weights for policy 1, policy_version 1914852 (0.0011) [2023-12-27 05:18:51,733][105620] Updated weights for policy 1, policy_version 1914862 (0.0011) [2023-12-27 05:18:51,899][105692] Updated weights for policy 0, policy_version 1910196 (0.0008) [2023-12-27 05:18:51,961][105692] Updated weights for policy 0, policy_version 1910206 (0.0008) [2023-12-27 05:18:52,017][105692] Updated weights for policy 0, policy_version 1910216 (0.0008) [2023-12-27 05:18:52,465][105620] Updated weights for policy 1, policy_version 1914872 (0.0010) [2023-12-27 05:18:52,515][105620] Updated weights for policy 1, policy_version 1914882 (0.0011) [2023-12-27 05:18:52,568][105620] Updated weights for policy 1, policy_version 1914892 (0.0011) [2023-12-27 05:18:52,715][105692] Updated weights for policy 0, policy_version 1910226 (0.0008) [2023-12-27 05:18:52,772][105692] Updated weights for policy 0, policy_version 1910236 (0.0006) [2023-12-27 05:18:52,833][105692] Updated weights for policy 0, policy_version 1910246 (0.0005) [2023-12-27 05:18:52,891][105692] Updated weights for policy 0, policy_version 1910256 (0.0006) [2023-12-27 05:18:53,319][105620] Updated weights for policy 1, policy_version 1914902 (0.0010) [2023-12-27 05:18:53,382][105620] Updated weights for policy 1, policy_version 1914912 (0.0009) [2023-12-27 05:18:53,439][105620] Updated weights for policy 1, policy_version 1914922 (0.0005) [2023-12-27 05:18:53,508][105692] Updated weights for policy 0, policy_version 1910266 (0.0005) [2023-12-27 05:18:53,555][105692] Updated weights for policy 0, policy_version 1910276 (0.0005) [2023-12-27 05:18:53,605][105692] Updated weights for policy 0, policy_version 1910286 (0.0005) [2023-12-27 05:18:53,997][105620] Updated weights for policy 1, policy_version 1914932 (0.0006) [2023-12-27 05:18:54,061][105620] Updated weights for policy 1, policy_version 1914942 (0.0008) [2023-12-27 05:18:54,127][105620] Updated weights for policy 1, policy_version 1914952 (0.0005) [2023-12-27 05:18:54,209][105692] Updated weights for policy 0, policy_version 1910296 (0.0009) [2023-12-27 05:18:54,268][105692] Updated weights for policy 0, policy_version 1910306 (0.0010) [2023-12-27 05:18:54,324][105692] Updated weights for policy 0, policy_version 1910316 (0.0010) [2023-12-27 05:18:54,760][105620] Updated weights for policy 1, policy_version 1914962 (0.0006) [2023-12-27 05:18:54,812][105620] Updated weights for policy 1, policy_version 1914972 (0.0009) [2023-12-27 05:18:54,865][105620] Updated weights for policy 1, policy_version 1914982 (0.0009) [2023-12-27 05:18:54,917][105620] Updated weights for policy 1, policy_version 1914992 (0.0009) [2023-12-27 05:18:55,011][105692] Updated weights for policy 0, policy_version 1910326 (0.0009) [2023-12-27 05:18:55,059][105692] Updated weights for policy 0, policy_version 1910336 (0.0005) [2023-12-27 05:18:55,107][105692] Updated weights for policy 0, policy_version 1910346 (0.0006) [2023-12-27 05:18:55,724][105620] Updated weights for policy 1, policy_version 1915002 (0.0008) [2023-12-27 05:18:55,771][105692] Updated weights for policy 0, policy_version 1910356 (0.0007) [2023-12-27 05:18:55,773][105620] Updated weights for policy 1, policy_version 1915012 (0.0005) [2023-12-27 05:18:55,826][105620] Updated weights for policy 1, policy_version 1915022 (0.0006) [2023-12-27 05:18:55,835][105692] Updated weights for policy 0, policy_version 1910366 (0.0011) [2023-12-27 05:18:55,898][105692] Updated weights for policy 0, policy_version 1910376 (0.0011) [2023-12-27 05:18:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.7, 300 sec: 19438.6). Total num frames: 979443712. Throughput: 0: 9787.3, 1: 9814.4. Samples: 979448064. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:18:56,063][104569] Avg episode reward: [(0, '8354.474'), (1, '9254.311')] [2023-12-27 05:18:56,565][105692] Updated weights for policy 0, policy_version 1910386 (0.0009) [2023-12-27 05:18:56,587][105620] Updated weights for policy 1, policy_version 1915032 (0.0008) [2023-12-27 05:18:56,627][105692] Updated weights for policy 0, policy_version 1910396 (0.0007) [2023-12-27 05:18:56,646][105620] Updated weights for policy 1, policy_version 1915042 (0.0007) [2023-12-27 05:18:56,687][105692] Updated weights for policy 0, policy_version 1910406 (0.0010) [2023-12-27 05:18:56,704][105620] Updated weights for policy 1, policy_version 1915052 (0.0007) [2023-12-27 05:18:56,747][105692] Updated weights for policy 0, policy_version 1910416 (0.0005) [2023-12-27 05:18:57,392][105692] Updated weights for policy 0, policy_version 1910426 (0.0005) [2023-12-27 05:18:57,439][105692] Updated weights for policy 0, policy_version 1910436 (0.0005) [2023-12-27 05:18:57,489][105620] Updated weights for policy 1, policy_version 1915062 (0.0009) [2023-12-27 05:18:57,490][105692] Updated weights for policy 0, policy_version 1910446 (0.0005) [2023-12-27 05:18:57,544][105620] Updated weights for policy 1, policy_version 1915073 (0.0010) [2023-12-27 05:18:57,596][105620] Updated weights for policy 1, policy_version 1915083 (0.0009) [2023-12-27 05:18:58,081][105692] Updated weights for policy 0, policy_version 1910456 (0.0009) [2023-12-27 05:18:58,136][105692] Updated weights for policy 0, policy_version 1910466 (0.0010) [2023-12-27 05:18:58,193][105692] Updated weights for policy 0, policy_version 1910476 (0.0010) [2023-12-27 05:18:58,345][105620] Updated weights for policy 1, policy_version 1915093 (0.0009) [2023-12-27 05:18:58,412][105620] Updated weights for policy 1, policy_version 1915103 (0.0008) [2023-12-27 05:18:58,477][105620] Updated weights for policy 1, policy_version 1915113 (0.0007) [2023-12-27 05:18:59,004][105692] Updated weights for policy 0, policy_version 1910486 (0.0009) [2023-12-27 05:18:59,070][105692] Updated weights for policy 0, policy_version 1910496 (0.0009) [2023-12-27 05:18:59,129][105692] Updated weights for policy 0, policy_version 1910506 (0.0009) [2023-12-27 05:18:59,223][105620] Updated weights for policy 1, policy_version 1915123 (0.0008) [2023-12-27 05:18:59,290][105620] Updated weights for policy 1, policy_version 1915133 (0.0009) [2023-12-27 05:18:59,355][105620] Updated weights for policy 1, policy_version 1915143 (0.0009) [2023-12-27 05:18:59,919][105692] Updated weights for policy 0, policy_version 1910516 (0.0009) [2023-12-27 05:18:59,978][105692] Updated weights for policy 0, policy_version 1910526 (0.0009) [2023-12-27 05:19:00,017][105620] Updated weights for policy 1, policy_version 1915153 (0.0009) [2023-12-27 05:19:00,036][105692] Updated weights for policy 0, policy_version 1910536 (0.0009) [2023-12-27 05:19:00,076][105620] Updated weights for policy 1, policy_version 1915163 (0.0007) [2023-12-27 05:19:00,136][105620] Updated weights for policy 1, policy_version 1915173 (0.0008) [2023-12-27 05:19:00,197][105620] Updated weights for policy 1, policy_version 1915183 (0.0009) [2023-12-27 05:19:00,859][105692] Updated weights for policy 0, policy_version 1910546 (0.0008) [2023-12-27 05:19:00,862][105620] Updated weights for policy 1, policy_version 1915193 (0.0009) [2023-12-27 05:19:00,909][105692] Updated weights for policy 0, policy_version 1910556 (0.0006) [2023-12-27 05:19:00,915][105620] Updated weights for policy 1, policy_version 1915203 (0.0010) [2023-12-27 05:19:00,960][105692] Updated weights for policy 0, policy_version 1910566 (0.0008) [2023-12-27 05:19:00,976][105620] Updated weights for policy 1, policy_version 1915213 (0.0008) [2023-12-27 05:19:01,020][105692] Updated weights for policy 0, policy_version 1910576 (0.0010) [2023-12-27 05:19:01,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19438.7). Total num frames: 979542016. Throughput: 0: 9870.6, 1: 9779.3. Samples: 979506896. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:19:01,062][104569] Avg episode reward: [(0, '8630.469'), (1, '9254.334')] [2023-12-27 05:19:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001910576_489177088.pth... [2023-12-27 05:19:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001915216_490364928.pth... [2023-12-27 05:19:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001909424_488882176.pth [2023-12-27 05:19:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001914064_490070016.pth [2023-12-27 05:19:01,717][105692] Updated weights for policy 0, policy_version 1910586 (0.0009) [2023-12-27 05:19:01,750][105620] Updated weights for policy 1, policy_version 1915223 (0.0007) [2023-12-27 05:19:01,776][105692] Updated weights for policy 0, policy_version 1910596 (0.0009) [2023-12-27 05:19:01,805][105620] Updated weights for policy 1, policy_version 1915233 (0.0006) [2023-12-27 05:19:01,829][105692] Updated weights for policy 0, policy_version 1910606 (0.0007) [2023-12-27 05:19:01,852][105620] Updated weights for policy 1, policy_version 1915243 (0.0012) [2023-12-27 05:19:02,577][105692] Updated weights for policy 0, policy_version 1910616 (0.0008) [2023-12-27 05:19:02,616][105620] Updated weights for policy 1, policy_version 1915253 (0.0010) [2023-12-27 05:19:02,634][105692] Updated weights for policy 0, policy_version 1910626 (0.0008) [2023-12-27 05:19:02,665][105620] Updated weights for policy 1, policy_version 1915263 (0.0010) [2023-12-27 05:19:02,692][105692] Updated weights for policy 0, policy_version 1910636 (0.0009) [2023-12-27 05:19:02,724][105620] Updated weights for policy 1, policy_version 1915273 (0.0010) [2023-12-27 05:19:03,288][105692] Updated weights for policy 0, policy_version 1910646 (0.0009) [2023-12-27 05:19:03,332][105692] Updated weights for policy 0, policy_version 1910656 (0.0008) [2023-12-27 05:19:03,376][105692] Updated weights for policy 0, policy_version 1910666 (0.0008) [2023-12-27 05:19:03,415][105620] Updated weights for policy 1, policy_version 1915283 (0.0007) [2023-12-27 05:19:03,472][105620] Updated weights for policy 1, policy_version 1915293 (0.0005) [2023-12-27 05:19:03,528][105620] Updated weights for policy 1, policy_version 1915303 (0.0005) [2023-12-27 05:19:04,130][105620] Updated weights for policy 1, policy_version 1915313 (0.0006) [2023-12-27 05:19:04,193][105692] Updated weights for policy 0, policy_version 1910676 (0.0008) [2023-12-27 05:19:04,201][105620] Updated weights for policy 1, policy_version 1915323 (0.0008) [2023-12-27 05:19:04,255][105692] Updated weights for policy 0, policy_version 1910686 (0.0009) [2023-12-27 05:19:04,264][105620] Updated weights for policy 1, policy_version 1915333 (0.0008) [2023-12-27 05:19:04,316][105692] Updated weights for policy 0, policy_version 1910696 (0.0008) [2023-12-27 05:19:04,328][105620] Updated weights for policy 1, policy_version 1915343 (0.0009) [2023-12-27 05:19:05,005][105692] Updated weights for policy 0, policy_version 1910706 (0.0009) [2023-12-27 05:19:05,015][105620] Updated weights for policy 1, policy_version 1915353 (0.0006) [2023-12-27 05:19:05,063][105692] Updated weights for policy 0, policy_version 1910716 (0.0008) [2023-12-27 05:19:05,073][105620] Updated weights for policy 1, policy_version 1915363 (0.0007) [2023-12-27 05:19:05,122][105692] Updated weights for policy 0, policy_version 1910726 (0.0008) [2023-12-27 05:19:05,132][105620] Updated weights for policy 1, policy_version 1915373 (0.0005) [2023-12-27 05:19:05,189][105692] Updated weights for policy 0, policy_version 1910736 (0.0009) [2023-12-27 05:19:05,786][105620] Updated weights for policy 1, policy_version 1915383 (0.0006) [2023-12-27 05:19:05,848][105620] Updated weights for policy 1, policy_version 1915393 (0.0006) [2023-12-27 05:19:05,850][105692] Updated weights for policy 0, policy_version 1910746 (0.0006) [2023-12-27 05:19:05,911][105692] Updated weights for policy 0, policy_version 1910756 (0.0006) [2023-12-27 05:19:05,918][105620] Updated weights for policy 1, policy_version 1915403 (0.0005) [2023-12-27 05:19:05,963][105692] Updated weights for policy 0, policy_version 1910766 (0.0007) [2023-12-27 05:19:06,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 979640320. Throughput: 0: 9670.7, 1: 9797.4. Samples: 979622252. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:19:06,062][104569] Avg episode reward: [(0, '8273.562'), (1, '9345.838')] [2023-12-27 05:19:06,500][105620] Updated weights for policy 1, policy_version 1915413 (0.0008) [2023-12-27 05:19:06,553][105620] Updated weights for policy 1, policy_version 1915423 (0.0010) [2023-12-27 05:19:06,609][105620] Updated weights for policy 1, policy_version 1915433 (0.0010) [2023-12-27 05:19:06,668][105692] Updated weights for policy 0, policy_version 1910776 (0.0007) [2023-12-27 05:19:06,713][105692] Updated weights for policy 0, policy_version 1910786 (0.0008) [2023-12-27 05:19:06,757][105692] Updated weights for policy 0, policy_version 1910796 (0.0007) [2023-12-27 05:19:07,375][105620] Updated weights for policy 1, policy_version 1915443 (0.0010) [2023-12-27 05:19:07,427][105620] Updated weights for policy 1, policy_version 1915453 (0.0010) [2023-12-27 05:19:07,476][105620] Updated weights for policy 1, policy_version 1915463 (0.0010) [2023-12-27 05:19:07,521][105586] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000010 [2023-12-27 05:19:07,561][105692] Updated weights for policy 0, policy_version 1910806 (0.0008) [2023-12-27 05:19:07,617][105692] Updated weights for policy 0, policy_version 1910816 (0.0008) [2023-12-27 05:19:07,670][105692] Updated weights for policy 0, policy_version 1910826 (0.0008) [2023-12-27 05:19:08,246][105620] Updated weights for policy 1, policy_version 1915473 (0.0011) [2023-12-27 05:19:08,301][105620] Updated weights for policy 1, policy_version 1915483 (0.0010) [2023-12-27 05:19:08,359][105620] Updated weights for policy 1, policy_version 1915493 (0.0009) [2023-12-27 05:19:08,428][105620] Updated weights for policy 1, policy_version 1915503 (0.0006) [2023-12-27 05:19:08,437][105692] Updated weights for policy 0, policy_version 1910836 (0.0007) [2023-12-27 05:19:08,503][105692] Updated weights for policy 0, policy_version 1910846 (0.0006) [2023-12-27 05:19:08,569][105692] Updated weights for policy 0, policy_version 1910856 (0.0008) [2023-12-27 05:19:09,139][105692] Updated weights for policy 0, policy_version 1910866 (0.0007) [2023-12-27 05:19:09,148][105620] Updated weights for policy 1, policy_version 1915513 (0.0010) [2023-12-27 05:19:09,193][105692] Updated weights for policy 0, policy_version 1910876 (0.0007) [2023-12-27 05:19:09,213][105620] Updated weights for policy 1, policy_version 1915523 (0.0010) [2023-12-27 05:19:09,264][105692] Updated weights for policy 0, policy_version 1910886 (0.0010) [2023-12-27 05:19:09,279][105620] Updated weights for policy 1, policy_version 1915533 (0.0011) [2023-12-27 05:19:09,330][105692] Updated weights for policy 0, policy_version 1910896 (0.0011) [2023-12-27 05:19:10,007][105620] Updated weights for policy 1, policy_version 1915543 (0.0011) [2023-12-27 05:19:10,067][105620] Updated weights for policy 1, policy_version 1915553 (0.0011) [2023-12-27 05:19:10,126][105620] Updated weights for policy 1, policy_version 1915563 (0.0011) [2023-12-27 05:19:10,132][105692] Updated weights for policy 0, policy_version 1910906 (0.0009) [2023-12-27 05:19:10,199][105692] Updated weights for policy 0, policy_version 1910916 (0.0011) [2023-12-27 05:19:10,259][105692] Updated weights for policy 0, policy_version 1910926 (0.0011) [2023-12-27 05:19:10,758][105620] Updated weights for policy 1, policy_version 1915573 (0.0010) [2023-12-27 05:19:10,810][105620] Updated weights for policy 1, policy_version 1915583 (0.0010) [2023-12-27 05:19:10,862][105620] Updated weights for policy 1, policy_version 1915593 (0.0010) [2023-12-27 05:19:10,876][105692] Updated weights for policy 0, policy_version 1910936 (0.0008) [2023-12-27 05:19:10,934][105692] Updated weights for policy 0, policy_version 1910946 (0.0010) [2023-12-27 05:19:11,005][105692] Updated weights for policy 0, policy_version 1910956 (0.0011) [2023-12-27 05:19:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 979738624. Throughput: 0: 9778.2, 1: 9803.1. Samples: 979740964. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:19:11,062][104569] Avg episode reward: [(0, '8176.497'), (1, '9345.827')] [2023-12-27 05:19:11,632][105620] Updated weights for policy 1, policy_version 1915603 (0.0010) [2023-12-27 05:19:11,692][105620] Updated weights for policy 1, policy_version 1915613 (0.0010) [2023-12-27 05:19:11,754][105620] Updated weights for policy 1, policy_version 1915623 (0.0009) [2023-12-27 05:19:11,781][105692] Updated weights for policy 0, policy_version 1910966 (0.0009) [2023-12-27 05:19:11,853][105692] Updated weights for policy 0, policy_version 1910976 (0.0010) [2023-12-27 05:19:11,920][105692] Updated weights for policy 0, policy_version 1910986 (0.0008) [2023-12-27 05:19:12,466][105620] Updated weights for policy 1, policy_version 1915633 (0.0006) [2023-12-27 05:19:12,521][105620] Updated weights for policy 1, policy_version 1915643 (0.0009) [2023-12-27 05:19:12,582][105620] Updated weights for policy 1, policy_version 1915653 (0.0009) [2023-12-27 05:19:12,644][105620] Updated weights for policy 1, policy_version 1915663 (0.0009) [2023-12-27 05:19:12,681][105692] Updated weights for policy 0, policy_version 1910996 (0.0009) [2023-12-27 05:19:12,727][105692] Updated weights for policy 0, policy_version 1911006 (0.0008) [2023-12-27 05:19:12,781][105692] Updated weights for policy 0, policy_version 1911016 (0.0005) [2023-12-27 05:19:13,406][105692] Updated weights for policy 0, policy_version 1911026 (0.0006) [2023-12-27 05:19:13,420][105620] Updated weights for policy 1, policy_version 1915673 (0.0007) [2023-12-27 05:19:13,459][105692] Updated weights for policy 0, policy_version 1911036 (0.0010) [2023-12-27 05:19:13,472][105620] Updated weights for policy 1, policy_version 1915683 (0.0006) [2023-12-27 05:19:13,508][105692] Updated weights for policy 0, policy_version 1911046 (0.0010) [2023-12-27 05:19:13,527][105620] Updated weights for policy 1, policy_version 1915693 (0.0005) [2023-12-27 05:19:13,560][105692] Updated weights for policy 0, policy_version 1911056 (0.0009) [2023-12-27 05:19:14,044][105620] Updated weights for policy 1, policy_version 1915703 (0.0005) [2023-12-27 05:19:14,107][105620] Updated weights for policy 1, policy_version 1915713 (0.0008) [2023-12-27 05:19:14,134][105692] Updated weights for policy 0, policy_version 1911066 (0.0005) [2023-12-27 05:19:14,169][105620] Updated weights for policy 1, policy_version 1915723 (0.0009) [2023-12-27 05:19:14,187][105692] Updated weights for policy 0, policy_version 1911076 (0.0006) [2023-12-27 05:19:14,245][105692] Updated weights for policy 0, policy_version 1911086 (0.0005) [2023-12-27 05:19:14,773][105620] Updated weights for policy 1, policy_version 1915733 (0.0007) [2023-12-27 05:19:14,826][105620] Updated weights for policy 1, policy_version 1915743 (0.0008) [2023-12-27 05:19:14,885][105620] Updated weights for policy 1, policy_version 1915753 (0.0007) [2023-12-27 05:19:14,925][105692] Updated weights for policy 0, policy_version 1911096 (0.0006) [2023-12-27 05:19:14,989][105692] Updated weights for policy 0, policy_version 1911106 (0.0005) [2023-12-27 05:19:15,054][105692] Updated weights for policy 0, policy_version 1911116 (0.0008) [2023-12-27 05:19:15,598][105692] Updated weights for policy 0, policy_version 1911126 (0.0010) [2023-12-27 05:19:15,647][105692] Updated weights for policy 0, policy_version 1911136 (0.0011) [2023-12-27 05:19:15,706][105692] Updated weights for policy 0, policy_version 1911146 (0.0011) [2023-12-27 05:19:15,712][105620] Updated weights for policy 1, policy_version 1915763 (0.0009) [2023-12-27 05:19:15,770][105620] Updated weights for policy 1, policy_version 1915773 (0.0006) [2023-12-27 05:19:15,827][105620] Updated weights for policy 1, policy_version 1915783 (0.0008) [2023-12-27 05:19:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.7, 300 sec: 19466.4). Total num frames: 979836928. Throughput: 0: 9779.0, 1: 9714.6. Samples: 979799840. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:19:16,063][104569] Avg episode reward: [(0, '8356.191'), (1, '9068.895')] [2023-12-27 05:19:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001915792_490512384.pth... [2023-12-27 05:19:16,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001911152_489324544.pth... [2023-12-27 05:19:16,076][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001914640_490217472.pth [2023-12-27 05:19:16,096][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001910000_489029632.pth [2023-12-27 05:19:16,373][105692] Updated weights for policy 0, policy_version 1911156 (0.0011) [2023-12-27 05:19:16,425][105692] Updated weights for policy 0, policy_version 1911166 (0.0009) [2023-12-27 05:19:16,472][105692] Updated weights for policy 0, policy_version 1911176 (0.0010) [2023-12-27 05:19:16,637][105620] Updated weights for policy 1, policy_version 1915793 (0.0008) [2023-12-27 05:19:16,693][105620] Updated weights for policy 1, policy_version 1915803 (0.0008) [2023-12-27 05:19:16,755][105620] Updated weights for policy 1, policy_version 1915813 (0.0008) [2023-12-27 05:19:16,809][105620] Updated weights for policy 1, policy_version 1915823 (0.0008) [2023-12-27 05:19:17,211][105692] Updated weights for policy 0, policy_version 1911186 (0.0010) [2023-12-27 05:19:17,270][105692] Updated weights for policy 0, policy_version 1911196 (0.0007) [2023-12-27 05:19:17,336][105692] Updated weights for policy 0, policy_version 1911206 (0.0005) [2023-12-27 05:19:17,406][105692] Updated weights for policy 0, policy_version 1911216 (0.0006) [2023-12-27 05:19:17,607][105620] Updated weights for policy 1, policy_version 1915833 (0.0009) [2023-12-27 05:19:17,662][105620] Updated weights for policy 1, policy_version 1915843 (0.0010) [2023-12-27 05:19:17,715][105620] Updated weights for policy 1, policy_version 1915853 (0.0009) [2023-12-27 05:19:17,989][105692] Updated weights for policy 0, policy_version 1911226 (0.0007) [2023-12-27 05:19:18,051][105692] Updated weights for policy 0, policy_version 1911236 (0.0009) [2023-12-27 05:19:18,114][105692] Updated weights for policy 0, policy_version 1911246 (0.0006) [2023-12-27 05:19:18,525][105620] Updated weights for policy 1, policy_version 1915863 (0.0008) [2023-12-27 05:19:18,577][105620] Updated weights for policy 1, policy_version 1915873 (0.0009) [2023-12-27 05:19:18,627][105620] Updated weights for policy 1, policy_version 1915883 (0.0008) [2023-12-27 05:19:18,795][105692] Updated weights for policy 0, policy_version 1911256 (0.0007) [2023-12-27 05:19:18,849][105692] Updated weights for policy 0, policy_version 1911266 (0.0009) [2023-12-27 05:19:18,897][105692] Updated weights for policy 0, policy_version 1911276 (0.0009) [2023-12-27 05:19:19,449][105620] Updated weights for policy 1, policy_version 1915893 (0.0008) [2023-12-27 05:19:19,518][105620] Updated weights for policy 1, policy_version 1915903 (0.0007) [2023-12-27 05:19:19,574][105620] Updated weights for policy 1, policy_version 1915913 (0.0010) [2023-12-27 05:19:19,655][105692] Updated weights for policy 0, policy_version 1911286 (0.0007) [2023-12-27 05:19:19,722][105692] Updated weights for policy 0, policy_version 1911296 (0.0005) [2023-12-27 05:19:19,784][105692] Updated weights for policy 0, policy_version 1911306 (0.0006) [2023-12-27 05:19:20,374][105620] Updated weights for policy 1, policy_version 1915923 (0.0009) [2023-12-27 05:19:20,426][105692] Updated weights for policy 0, policy_version 1911316 (0.0006) [2023-12-27 05:19:20,436][105620] Updated weights for policy 1, policy_version 1915933 (0.0008) [2023-12-27 05:19:20,490][105692] Updated weights for policy 0, policy_version 1911326 (0.0006) [2023-12-27 05:19:20,495][105620] Updated weights for policy 1, policy_version 1915943 (0.0010) [2023-12-27 05:19:20,547][105692] Updated weights for policy 0, policy_version 1911336 (0.0005) [2023-12-27 05:19:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 979927040. Throughput: 0: 9852.1, 1: 9718.7. Samples: 979918056. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:19:21,062][104569] Avg episode reward: [(0, '8631.617'), (1, '8978.443')] [2023-12-27 05:19:21,141][105692] Updated weights for policy 0, policy_version 1911346 (0.0008) [2023-12-27 05:19:21,200][105692] Updated weights for policy 0, policy_version 1911356 (0.0008) [2023-12-27 05:19:21,264][105692] Updated weights for policy 0, policy_version 1911366 (0.0008) [2023-12-27 05:19:21,327][105692] Updated weights for policy 0, policy_version 1911376 (0.0008) [2023-12-27 05:19:21,365][105620] Updated weights for policy 1, policy_version 1915953 (0.0010) [2023-12-27 05:19:21,428][105620] Updated weights for policy 1, policy_version 1915963 (0.0009) [2023-12-27 05:19:21,485][105620] Updated weights for policy 1, policy_version 1915973 (0.0008) [2023-12-27 05:19:21,542][105620] Updated weights for policy 1, policy_version 1915983 (0.0008) [2023-12-27 05:19:22,074][105692] Updated weights for policy 0, policy_version 1911386 (0.0009) [2023-12-27 05:19:22,135][105692] Updated weights for policy 0, policy_version 1911396 (0.0010) [2023-12-27 05:19:22,199][105692] Updated weights for policy 0, policy_version 1911406 (0.0008) [2023-12-27 05:19:22,317][105620] Updated weights for policy 1, policy_version 1915993 (0.0008) [2023-12-27 05:19:22,387][105620] Updated weights for policy 1, policy_version 1916003 (0.0008) [2023-12-27 05:19:22,480][105620] Updated weights for policy 1, policy_version 1916013 (0.0009) [2023-12-27 05:19:22,981][105692] Updated weights for policy 0, policy_version 1911416 (0.0008) [2023-12-27 05:19:23,044][105692] Updated weights for policy 0, policy_version 1911426 (0.0009) [2023-12-27 05:19:23,107][105692] Updated weights for policy 0, policy_version 1911436 (0.0009) [2023-12-27 05:19:23,151][105620] Updated weights for policy 1, policy_version 1916023 (0.0009) [2023-12-27 05:19:23,210][105620] Updated weights for policy 1, policy_version 1916033 (0.0008) [2023-12-27 05:19:23,272][105620] Updated weights for policy 1, policy_version 1916043 (0.0009) [2023-12-27 05:19:23,860][105692] Updated weights for policy 0, policy_version 1911446 (0.0010) [2023-12-27 05:19:23,910][105692] Updated weights for policy 0, policy_version 1911456 (0.0009) [2023-12-27 05:19:23,957][105692] Updated weights for policy 0, policy_version 1911466 (0.0008) [2023-12-27 05:19:24,030][105620] Updated weights for policy 1, policy_version 1916053 (0.0009) [2023-12-27 05:19:24,091][105620] Updated weights for policy 1, policy_version 1916063 (0.0009) [2023-12-27 05:19:24,145][105620] Updated weights for policy 1, policy_version 1916073 (0.0010) [2023-12-27 05:19:24,594][105692] Updated weights for policy 0, policy_version 1911476 (0.0007) [2023-12-27 05:19:24,642][105692] Updated weights for policy 0, policy_version 1911486 (0.0009) [2023-12-27 05:19:24,697][105692] Updated weights for policy 0, policy_version 1911496 (0.0009) [2023-12-27 05:19:24,944][105620] Updated weights for policy 1, policy_version 1916083 (0.0009) [2023-12-27 05:19:24,995][105620] Updated weights for policy 1, policy_version 1916093 (0.0009) [2023-12-27 05:19:25,049][105620] Updated weights for policy 1, policy_version 1916103 (0.0008) [2023-12-27 05:19:25,399][105692] Updated weights for policy 0, policy_version 1911506 (0.0009) [2023-12-27 05:19:25,446][105692] Updated weights for policy 0, policy_version 1911516 (0.0008) [2023-12-27 05:19:25,496][105692] Updated weights for policy 0, policy_version 1911526 (0.0009) [2023-12-27 05:19:25,552][105692] Updated weights for policy 0, policy_version 1911536 (0.0009) [2023-12-27 05:19:25,789][105620] Updated weights for policy 1, policy_version 1916113 (0.0010) [2023-12-27 05:19:25,851][105620] Updated weights for policy 1, policy_version 1916123 (0.0009) [2023-12-27 05:19:25,913][105620] Updated weights for policy 1, policy_version 1916133 (0.0008) [2023-12-27 05:19:25,972][105620] Updated weights for policy 1, policy_version 1916143 (0.0009) [2023-12-27 05:19:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 980025344. Throughput: 0: 9888.4, 1: 9627.8. Samples: 980031152. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:19:26,062][104569] Avg episode reward: [(0, '8538.022'), (1, '9163.313')] [2023-12-27 05:19:26,310][105692] Updated weights for policy 0, policy_version 1911546 (0.0005) [2023-12-27 05:19:26,369][105692] Updated weights for policy 0, policy_version 1911556 (0.0005) [2023-12-27 05:19:26,428][105692] Updated weights for policy 0, policy_version 1911566 (0.0009) [2023-12-27 05:19:26,820][105620] Updated weights for policy 1, policy_version 1916153 (0.0008) [2023-12-27 05:19:26,885][105620] Updated weights for policy 1, policy_version 1916163 (0.0006) [2023-12-27 05:19:26,943][105620] Updated weights for policy 1, policy_version 1916173 (0.0007) [2023-12-27 05:19:26,996][105692] Updated weights for policy 0, policy_version 1911576 (0.0010) [2023-12-27 05:19:27,044][105692] Updated weights for policy 0, policy_version 1911587 (0.0007) [2023-12-27 05:19:27,089][105692] Updated weights for policy 0, policy_version 1911597 (0.0006) [2023-12-27 05:19:27,564][105620] Updated weights for policy 1, policy_version 1916183 (0.0007) [2023-12-27 05:19:27,618][105620] Updated weights for policy 1, policy_version 1916193 (0.0009) [2023-12-27 05:19:27,668][105620] Updated weights for policy 1, policy_version 1916203 (0.0008) [2023-12-27 05:19:27,811][105692] Updated weights for policy 0, policy_version 1911608 (0.0007) [2023-12-27 05:19:27,868][105692] Updated weights for policy 0, policy_version 1911618 (0.0006) [2023-12-27 05:19:27,926][105692] Updated weights for policy 0, policy_version 1911628 (0.0006) [2023-12-27 05:19:28,466][105692] Updated weights for policy 0, policy_version 1911638 (0.0008) [2023-12-27 05:19:28,525][105692] Updated weights for policy 0, policy_version 1911648 (0.0009) [2023-12-27 05:19:28,557][105620] Updated weights for policy 1, policy_version 1916213 (0.0008) [2023-12-27 05:19:28,585][105692] Updated weights for policy 0, policy_version 1911658 (0.0007) [2023-12-27 05:19:28,608][105620] Updated weights for policy 1, policy_version 1916223 (0.0008) [2023-12-27 05:19:28,671][105620] Updated weights for policy 1, policy_version 1916233 (0.0008) [2023-12-27 05:19:29,286][105692] Updated weights for policy 0, policy_version 1911668 (0.0006) [2023-12-27 05:19:29,350][105692] Updated weights for policy 0, policy_version 1911678 (0.0009) [2023-12-27 05:19:29,413][105692] Updated weights for policy 0, policy_version 1911688 (0.0006) [2023-12-27 05:19:29,440][105620] Updated weights for policy 1, policy_version 1916243 (0.0008) [2023-12-27 05:19:29,508][105620] Updated weights for policy 1, policy_version 1916253 (0.0009) [2023-12-27 05:19:29,578][105620] Updated weights for policy 1, policy_version 1916263 (0.0009) [2023-12-27 05:19:30,068][105692] Updated weights for policy 0, policy_version 1911698 (0.0007) [2023-12-27 05:19:30,129][105692] Updated weights for policy 0, policy_version 1911708 (0.0009) [2023-12-27 05:19:30,201][105692] Updated weights for policy 0, policy_version 1911718 (0.0010) [2023-12-27 05:19:30,268][105692] Updated weights for policy 0, policy_version 1911728 (0.0010) [2023-12-27 05:19:30,306][105620] Updated weights for policy 1, policy_version 1916273 (0.0008) [2023-12-27 05:19:30,362][105620] Updated weights for policy 1, policy_version 1916283 (0.0006) [2023-12-27 05:19:30,406][105620] Updated weights for policy 1, policy_version 1916293 (0.0007) [2023-12-27 05:19:30,454][105620] Updated weights for policy 1, policy_version 1916303 (0.0008) [2023-12-27 05:19:30,992][105692] Updated weights for policy 0, policy_version 1911738 (0.0005) [2023-12-27 05:19:31,047][105692] Updated weights for policy 0, policy_version 1911748 (0.0007) [2023-12-27 05:19:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 980115456. Throughput: 0: 9986.4, 1: 9540.5. Samples: 980090680. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 05:19:31,062][104569] Avg episode reward: [(0, '8168.277'), (1, '9253.676')] [2023-12-27 05:19:31,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001916304_490643456.pth... [2023-12-27 05:19:31,069][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001915216_490364928.pth [2023-12-27 05:19:31,108][105692] Updated weights for policy 0, policy_version 1911758 (0.0007) [2023-12-27 05:19:31,120][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001911760_489480192.pth... [2023-12-27 05:19:31,125][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001910576_489177088.pth [2023-12-27 05:19:31,253][105620] Updated weights for policy 1, policy_version 1916313 (0.0009) [2023-12-27 05:19:31,309][105620] Updated weights for policy 1, policy_version 1916323 (0.0009) [2023-12-27 05:19:31,370][105620] Updated weights for policy 1, policy_version 1916333 (0.0009) [2023-12-27 05:19:31,875][105692] Updated weights for policy 0, policy_version 1911768 (0.0008) [2023-12-27 05:19:31,921][105692] Updated weights for policy 0, policy_version 1911778 (0.0005) [2023-12-27 05:19:31,969][105692] Updated weights for policy 0, policy_version 1911788 (0.0005) [2023-12-27 05:19:32,207][105620] Updated weights for policy 1, policy_version 1916344 (0.0009) [2023-12-27 05:19:32,260][105620] Updated weights for policy 1, policy_version 1916354 (0.0010) [2023-12-27 05:19:32,316][105620] Updated weights for policy 1, policy_version 1916364 (0.0008) [2023-12-27 05:19:32,537][105692] Updated weights for policy 0, policy_version 1911798 (0.0005) [2023-12-27 05:19:32,589][105692] Updated weights for policy 0, policy_version 1911808 (0.0006) [2023-12-27 05:19:32,649][105692] Updated weights for policy 0, policy_version 1911818 (0.0005) [2023-12-27 05:19:33,212][105692] Updated weights for policy 0, policy_version 1911828 (0.0006) [2023-12-27 05:19:33,232][105620] Updated weights for policy 1, policy_version 1916374 (0.0009) [2023-12-27 05:19:33,260][105692] Updated weights for policy 0, policy_version 1911838 (0.0005) [2023-12-27 05:19:33,283][105620] Updated weights for policy 1, policy_version 1916384 (0.0009) [2023-12-27 05:19:33,311][105692] Updated weights for policy 0, policy_version 1911848 (0.0005) [2023-12-27 05:19:33,342][105620] Updated weights for policy 1, policy_version 1916394 (0.0009) [2023-12-27 05:19:33,861][105692] Updated weights for policy 0, policy_version 1911858 (0.0006) [2023-12-27 05:19:33,921][105692] Updated weights for policy 0, policy_version 1911868 (0.0009) [2023-12-27 05:19:33,959][105620] Updated weights for policy 1, policy_version 1916404 (0.0008) [2023-12-27 05:19:33,981][105692] Updated weights for policy 0, policy_version 1911878 (0.0007) [2023-12-27 05:19:34,017][105620] Updated weights for policy 1, policy_version 1916414 (0.0005) [2023-12-27 05:19:34,038][105692] Updated weights for policy 0, policy_version 1911888 (0.0005) [2023-12-27 05:19:34,072][105620] Updated weights for policy 1, policy_version 1916424 (0.0008) [2023-12-27 05:19:34,789][105692] Updated weights for policy 0, policy_version 1911898 (0.0007) [2023-12-27 05:19:34,806][105620] Updated weights for policy 1, policy_version 1916434 (0.0009) [2023-12-27 05:19:34,855][105692] Updated weights for policy 0, policy_version 1911908 (0.0009) [2023-12-27 05:19:34,865][105620] Updated weights for policy 1, policy_version 1916444 (0.0005) [2023-12-27 05:19:34,913][105692] Updated weights for policy 0, policy_version 1911918 (0.0008) [2023-12-27 05:19:34,923][105620] Updated weights for policy 1, policy_version 1916454 (0.0008) [2023-12-27 05:19:34,985][105620] Updated weights for policy 1, policy_version 1916464 (0.0010) [2023-12-27 05:19:35,561][105692] Updated weights for policy 0, policy_version 1911928 (0.0007) [2023-12-27 05:19:35,605][105692] Updated weights for policy 0, policy_version 1911938 (0.0007) [2023-12-27 05:19:35,660][105692] Updated weights for policy 0, policy_version 1911948 (0.0007) [2023-12-27 05:19:35,685][105620] Updated weights for policy 1, policy_version 1916474 (0.0010) [2023-12-27 05:19:35,752][105620] Updated weights for policy 1, policy_version 1916484 (0.0011) [2023-12-27 05:19:35,811][105620] Updated weights for policy 1, policy_version 1916494 (0.0010) [2023-12-27 05:19:36,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 980221952. Throughput: 0: 10065.8, 1: 9492.3. Samples: 980208972. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:19:36,062][104569] Avg episode reward: [(0, '8262.405'), (1, '9161.001')] [2023-12-27 05:19:36,469][105692] Updated weights for policy 0, policy_version 1911958 (0.0007) [2023-12-27 05:19:36,537][105692] Updated weights for policy 0, policy_version 1911968 (0.0009) [2023-12-27 05:19:36,544][105620] Updated weights for policy 1, policy_version 1916504 (0.0007) [2023-12-27 05:19:36,600][105692] Updated weights for policy 0, policy_version 1911978 (0.0008) [2023-12-27 05:19:36,606][105620] Updated weights for policy 1, policy_version 1916514 (0.0007) [2023-12-27 05:19:36,669][105620] Updated weights for policy 1, policy_version 1916524 (0.0008) [2023-12-27 05:19:37,229][105692] Updated weights for policy 0, policy_version 1911988 (0.0007) [2023-12-27 05:19:37,288][105692] Updated weights for policy 0, policy_version 1911998 (0.0009) [2023-12-27 05:19:37,345][105692] Updated weights for policy 0, policy_version 1912008 (0.0009) [2023-12-27 05:19:37,435][105620] Updated weights for policy 1, policy_version 1916534 (0.0007) [2023-12-27 05:19:37,485][105620] Updated weights for policy 1, policy_version 1916544 (0.0005) [2023-12-27 05:19:37,533][105620] Updated weights for policy 1, policy_version 1916554 (0.0008) [2023-12-27 05:19:38,072][105692] Updated weights for policy 0, policy_version 1912018 (0.0010) [2023-12-27 05:19:38,133][105692] Updated weights for policy 0, policy_version 1912028 (0.0008) [2023-12-27 05:19:38,202][105692] Updated weights for policy 0, policy_version 1912038 (0.0008) [2023-12-27 05:19:38,257][105620] Updated weights for policy 1, policy_version 1916564 (0.0011) [2023-12-27 05:19:38,264][105692] Updated weights for policy 0, policy_version 1912048 (0.0008) [2023-12-27 05:19:38,319][105620] Updated weights for policy 1, policy_version 1916574 (0.0010) [2023-12-27 05:19:38,393][105620] Updated weights for policy 1, policy_version 1916584 (0.0011) [2023-12-27 05:19:38,990][105692] Updated weights for policy 0, policy_version 1912058 (0.0008) [2023-12-27 05:19:39,040][105692] Updated weights for policy 0, policy_version 1912068 (0.0008) [2023-12-27 05:19:39,085][105692] Updated weights for policy 0, policy_version 1912078 (0.0008) [2023-12-27 05:19:39,137][105620] Updated weights for policy 1, policy_version 1916594 (0.0010) [2023-12-27 05:19:39,198][105620] Updated weights for policy 1, policy_version 1916604 (0.0006) [2023-12-27 05:19:39,269][105620] Updated weights for policy 1, policy_version 1916614 (0.0009) [2023-12-27 05:19:39,332][105620] Updated weights for policy 1, policy_version 1916624 (0.0010) [2023-12-27 05:19:39,927][105692] Updated weights for policy 0, policy_version 1912088 (0.0008) [2023-12-27 05:19:39,931][105620] Updated weights for policy 1, policy_version 1916634 (0.0009) [2023-12-27 05:19:39,985][105692] Updated weights for policy 0, policy_version 1912098 (0.0006) [2023-12-27 05:19:39,987][105620] Updated weights for policy 1, policy_version 1916644 (0.0010) [2023-12-27 05:19:40,043][105692] Updated weights for policy 0, policy_version 1912108 (0.0006) [2023-12-27 05:19:40,045][105620] Updated weights for policy 1, policy_version 1916654 (0.0011) [2023-12-27 05:19:40,831][105620] Updated weights for policy 1, policy_version 1916664 (0.0010) [2023-12-27 05:19:40,837][105692] Updated weights for policy 0, policy_version 1912118 (0.0006) [2023-12-27 05:19:40,876][105620] Updated weights for policy 1, policy_version 1916674 (0.0010) [2023-12-27 05:19:40,894][105692] Updated weights for policy 0, policy_version 1912128 (0.0005) [2023-12-27 05:19:40,927][105620] Updated weights for policy 1, policy_version 1916684 (0.0011) [2023-12-27 05:19:40,952][105692] Updated weights for policy 0, policy_version 1912138 (0.0007) [2023-12-27 05:19:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 980320256. Throughput: 0: 10000.2, 1: 9440.6. Samples: 980322896. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:19:41,062][104569] Avg episode reward: [(0, '8359.124'), (1, '9160.968')] [2023-12-27 05:19:41,658][105620] Updated weights for policy 1, policy_version 1916694 (0.0008) [2023-12-27 05:19:41,728][105620] Updated weights for policy 1, policy_version 1916704 (0.0009) [2023-12-27 05:19:41,758][105692] Updated weights for policy 0, policy_version 1912148 (0.0008) [2023-12-27 05:19:41,789][105620] Updated weights for policy 1, policy_version 1916714 (0.0007) [2023-12-27 05:19:41,810][105692] Updated weights for policy 0, policy_version 1912158 (0.0008) [2023-12-27 05:19:41,862][105692] Updated weights for policy 0, policy_version 1912168 (0.0010) [2023-12-27 05:19:42,510][105620] Updated weights for policy 1, policy_version 1916724 (0.0007) [2023-12-27 05:19:42,570][105620] Updated weights for policy 1, policy_version 1916734 (0.0008) [2023-12-27 05:19:42,636][105620] Updated weights for policy 1, policy_version 1916744 (0.0008) [2023-12-27 05:19:42,649][105692] Updated weights for policy 0, policy_version 1912178 (0.0009) [2023-12-27 05:19:42,701][105692] Updated weights for policy 0, policy_version 1912188 (0.0010) [2023-12-27 05:19:42,764][105692] Updated weights for policy 0, policy_version 1912198 (0.0010) [2023-12-27 05:19:42,824][105692] Updated weights for policy 0, policy_version 1912208 (0.0010) [2023-12-27 05:19:43,405][105620] Updated weights for policy 1, policy_version 1916754 (0.0010) [2023-12-27 05:19:43,468][105620] Updated weights for policy 1, policy_version 1916764 (0.0008) [2023-12-27 05:19:43,528][105620] Updated weights for policy 1, policy_version 1916774 (0.0008) [2023-12-27 05:19:43,581][105692] Updated weights for policy 0, policy_version 1912218 (0.0010) [2023-12-27 05:19:43,591][105620] Updated weights for policy 1, policy_version 1916784 (0.0009) [2023-12-27 05:19:43,639][105692] Updated weights for policy 0, policy_version 1912228 (0.0010) [2023-12-27 05:19:43,687][105692] Updated weights for policy 0, policy_version 1912238 (0.0010) [2023-12-27 05:19:44,355][105620] Updated weights for policy 1, policy_version 1916794 (0.0008) [2023-12-27 05:19:44,412][105620] Updated weights for policy 1, policy_version 1916804 (0.0007) [2023-12-27 05:19:44,429][105692] Updated weights for policy 0, policy_version 1912248 (0.0010) [2023-12-27 05:19:44,471][105620] Updated weights for policy 1, policy_version 1916814 (0.0006) [2023-12-27 05:19:44,484][105692] Updated weights for policy 0, policy_version 1912258 (0.0010) [2023-12-27 05:19:44,532][105692] Updated weights for policy 0, policy_version 1912268 (0.0010) [2023-12-27 05:19:45,164][105620] Updated weights for policy 1, policy_version 1916824 (0.0007) [2023-12-27 05:19:45,192][105692] Updated weights for policy 0, policy_version 1912278 (0.0010) [2023-12-27 05:19:45,223][105620] Updated weights for policy 1, policy_version 1916834 (0.0006) [2023-12-27 05:19:45,241][105692] Updated weights for policy 0, policy_version 1912288 (0.0011) [2023-12-27 05:19:45,281][105620] Updated weights for policy 1, policy_version 1916844 (0.0006) [2023-12-27 05:19:45,299][105692] Updated weights for policy 0, policy_version 1912298 (0.0009) [2023-12-27 05:19:45,918][105692] Updated weights for policy 0, policy_version 1912308 (0.0007) [2023-12-27 05:19:45,964][105620] Updated weights for policy 1, policy_version 1916854 (0.0006) [2023-12-27 05:19:45,970][105692] Updated weights for policy 0, policy_version 1912318 (0.0010) [2023-12-27 05:19:46,022][105620] Updated weights for policy 1, policy_version 1916864 (0.0007) [2023-12-27 05:19:46,029][105692] Updated weights for policy 0, policy_version 1912328 (0.0007) [2023-12-27 05:19:46,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19251.3, 300 sec: 19410.9). Total num frames: 980402176. Throughput: 0: 9909.1, 1: 9444.3. Samples: 980377800. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:19:46,062][104569] Avg episode reward: [(0, '8089.010'), (1, '9253.320')] [2023-12-27 05:19:46,081][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001912336_489627648.pth... [2023-12-27 05:19:46,082][105620] Updated weights for policy 1, policy_version 1916874 (0.0009) [2023-12-27 05:19:46,085][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001911152_489324544.pth [2023-12-27 05:19:46,117][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001916880_490790912.pth... [2023-12-27 05:19:46,120][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001915792_490512384.pth [2023-12-27 05:19:46,653][105692] Updated weights for policy 0, policy_version 1912338 (0.0007) [2023-12-27 05:19:46,705][105620] Updated weights for policy 1, policy_version 1916884 (0.0006) [2023-12-27 05:19:46,707][105692] Updated weights for policy 0, policy_version 1912348 (0.0008) [2023-12-27 05:19:46,759][105692] Updated weights for policy 0, policy_version 1912358 (0.0009) [2023-12-27 05:19:46,760][105620] Updated weights for policy 1, policy_version 1916894 (0.0007) [2023-12-27 05:19:46,815][105692] Updated weights for policy 0, policy_version 1912368 (0.0007) [2023-12-27 05:19:46,817][105620] Updated weights for policy 1, policy_version 1916904 (0.0010) [2023-12-27 05:19:47,433][105620] Updated weights for policy 1, policy_version 1916914 (0.0007) [2023-12-27 05:19:47,492][105620] Updated weights for policy 1, policy_version 1916924 (0.0005) [2023-12-27 05:19:47,555][105620] Updated weights for policy 1, policy_version 1916934 (0.0008) [2023-12-27 05:19:47,623][105620] Updated weights for policy 1, policy_version 1916944 (0.0005) [2023-12-27 05:19:47,669][105692] Updated weights for policy 0, policy_version 1912379 (0.0010) [2023-12-27 05:19:47,726][105692] Updated weights for policy 0, policy_version 1912390 (0.0010) [2023-12-27 05:19:47,779][105692] Updated weights for policy 0, policy_version 1912400 (0.0010) [2023-12-27 05:19:48,167][105620] Updated weights for policy 1, policy_version 1916954 (0.0009) [2023-12-27 05:19:48,228][105620] Updated weights for policy 1, policy_version 1916964 (0.0006) [2023-12-27 05:19:48,272][105620] Updated weights for policy 1, policy_version 1916974 (0.0005) [2023-12-27 05:19:48,653][105692] Updated weights for policy 0, policy_version 1912410 (0.0006) [2023-12-27 05:19:48,707][105692] Updated weights for policy 0, policy_version 1912420 (0.0007) [2023-12-27 05:19:48,766][105692] Updated weights for policy 0, policy_version 1912430 (0.0006) [2023-12-27 05:19:48,947][105620] Updated weights for policy 1, policy_version 1916984 (0.0008) [2023-12-27 05:19:48,994][105620] Updated weights for policy 1, policy_version 1916994 (0.0009) [2023-12-27 05:19:49,054][105620] Updated weights for policy 1, policy_version 1917004 (0.0008) [2023-12-27 05:19:49,445][105692] Updated weights for policy 0, policy_version 1912440 (0.0009) [2023-12-27 05:19:49,513][105692] Updated weights for policy 0, policy_version 1912450 (0.0010) [2023-12-27 05:19:49,586][105692] Updated weights for policy 0, policy_version 1912460 (0.0010) [2023-12-27 05:19:49,725][105620] Updated weights for policy 1, policy_version 1917014 (0.0007) [2023-12-27 05:19:49,783][105620] Updated weights for policy 1, policy_version 1917024 (0.0008) [2023-12-27 05:19:49,845][105620] Updated weights for policy 1, policy_version 1917034 (0.0008) [2023-12-27 05:19:50,439][105692] Updated weights for policy 0, policy_version 1912470 (0.0009) [2023-12-27 05:19:50,459][105620] Updated weights for policy 1, policy_version 1917044 (0.0007) [2023-12-27 05:19:50,498][105692] Updated weights for policy 0, policy_version 1912480 (0.0007) [2023-12-27 05:19:50,509][105620] Updated weights for policy 1, policy_version 1917054 (0.0006) [2023-12-27 05:19:50,561][105620] Updated weights for policy 1, policy_version 1917064 (0.0007) [2023-12-27 05:19:50,561][105692] Updated weights for policy 0, policy_version 1912490 (0.0010) [2023-12-27 05:19:51,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 980508672. Throughput: 0: 9958.2, 1: 9518.6. Samples: 980498704. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:19:51,063][104569] Avg episode reward: [(0, '8543.312'), (1, '9253.384')] [2023-12-27 05:19:51,199][105620] Updated weights for policy 1, policy_version 1917074 (0.0008) [2023-12-27 05:19:51,262][105620] Updated weights for policy 1, policy_version 1917084 (0.0008) [2023-12-27 05:19:51,331][105620] Updated weights for policy 1, policy_version 1917094 (0.0009) [2023-12-27 05:19:51,399][105620] Updated weights for policy 1, policy_version 1917104 (0.0007) [2023-12-27 05:19:51,411][105692] Updated weights for policy 0, policy_version 1912500 (0.0007) [2023-12-27 05:19:51,474][105692] Updated weights for policy 0, policy_version 1912510 (0.0010) [2023-12-27 05:19:51,539][105692] Updated weights for policy 0, policy_version 1912520 (0.0010) [2023-12-27 05:19:52,029][105620] Updated weights for policy 1, policy_version 1917114 (0.0010) [2023-12-27 05:19:52,083][105620] Updated weights for policy 1, policy_version 1917124 (0.0010) [2023-12-27 05:19:52,142][105620] Updated weights for policy 1, policy_version 1917134 (0.0010) [2023-12-27 05:19:52,200][105692] Updated weights for policy 0, policy_version 1912530 (0.0009) [2023-12-27 05:19:52,259][105692] Updated weights for policy 0, policy_version 1912540 (0.0006) [2023-12-27 05:19:52,329][105692] Updated weights for policy 0, policy_version 1912550 (0.0010) [2023-12-27 05:19:52,393][105692] Updated weights for policy 0, policy_version 1912560 (0.0010) [2023-12-27 05:19:52,912][105620] Updated weights for policy 1, policy_version 1917144 (0.0009) [2023-12-27 05:19:52,962][105620] Updated weights for policy 1, policy_version 1917154 (0.0007) [2023-12-27 05:19:53,019][105620] Updated weights for policy 1, policy_version 1917164 (0.0005) [2023-12-27 05:19:53,187][105692] Updated weights for policy 0, policy_version 1912570 (0.0006) [2023-12-27 05:19:53,239][105692] Updated weights for policy 0, policy_version 1912580 (0.0005) [2023-12-27 05:19:53,290][105692] Updated weights for policy 0, policy_version 1912590 (0.0005) [2023-12-27 05:19:53,705][105620] Updated weights for policy 1, policy_version 1917174 (0.0008) [2023-12-27 05:19:53,774][105620] Updated weights for policy 1, policy_version 1917184 (0.0011) [2023-12-27 05:19:53,836][105620] Updated weights for policy 1, policy_version 1917194 (0.0010) [2023-12-27 05:19:53,843][105692] Updated weights for policy 0, policy_version 1912600 (0.0006) [2023-12-27 05:19:53,899][105692] Updated weights for policy 0, policy_version 1912610 (0.0006) [2023-12-27 05:19:53,956][105692] Updated weights for policy 0, policy_version 1912620 (0.0006) [2023-12-27 05:19:54,576][105620] Updated weights for policy 1, policy_version 1917204 (0.0011) [2023-12-27 05:19:54,603][105692] Updated weights for policy 0, policy_version 1912630 (0.0006) [2023-12-27 05:19:54,634][105620] Updated weights for policy 1, policy_version 1917214 (0.0010) [2023-12-27 05:19:54,659][105692] Updated weights for policy 0, policy_version 1912640 (0.0009) [2023-12-27 05:19:54,693][105620] Updated weights for policy 1, policy_version 1917224 (0.0010) [2023-12-27 05:19:54,715][105692] Updated weights for policy 0, policy_version 1912650 (0.0006) [2023-12-27 05:19:55,417][105620] Updated weights for policy 1, policy_version 1917234 (0.0009) [2023-12-27 05:19:55,422][105692] Updated weights for policy 0, policy_version 1912660 (0.0006) [2023-12-27 05:19:55,472][105620] Updated weights for policy 1, policy_version 1917244 (0.0006) [2023-12-27 05:19:55,487][105692] Updated weights for policy 0, policy_version 1912670 (0.0006) [2023-12-27 05:19:55,518][105620] Updated weights for policy 1, policy_version 1917254 (0.0007) [2023-12-27 05:19:55,539][105692] Updated weights for policy 0, policy_version 1912680 (0.0008) [2023-12-27 05:19:55,572][105620] Updated weights for policy 1, policy_version 1917264 (0.0006) [2023-12-27 05:19:56,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 980606976. Throughput: 0: 9929.9, 1: 9555.0. Samples: 980617784. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:19:56,062][104569] Avg episode reward: [(0, '8537.442'), (1, '9345.800')] [2023-12-27 05:19:56,174][105692] Updated weights for policy 0, policy_version 1912690 (0.0007) [2023-12-27 05:19:56,207][105620] Updated weights for policy 1, policy_version 1917274 (0.0007) [2023-12-27 05:19:56,230][105692] Updated weights for policy 0, policy_version 1912700 (0.0011) [2023-12-27 05:19:56,259][105620] Updated weights for policy 1, policy_version 1917284 (0.0006) [2023-12-27 05:19:56,286][105692] Updated weights for policy 0, policy_version 1912710 (0.0008) [2023-12-27 05:19:56,317][105620] Updated weights for policy 1, policy_version 1917294 (0.0005) [2023-12-27 05:19:56,346][105692] Updated weights for policy 0, policy_version 1912720 (0.0008) [2023-12-27 05:19:56,880][105620] Updated weights for policy 1, policy_version 1917304 (0.0007) [2023-12-27 05:19:56,940][105620] Updated weights for policy 1, policy_version 1917314 (0.0006) [2023-12-27 05:19:57,002][105620] Updated weights for policy 1, policy_version 1917324 (0.0006) [2023-12-27 05:19:57,054][105692] Updated weights for policy 0, policy_version 1912730 (0.0010) [2023-12-27 05:19:57,108][105692] Updated weights for policy 0, policy_version 1912740 (0.0010) [2023-12-27 05:19:57,173][105692] Updated weights for policy 0, policy_version 1912750 (0.0010) [2023-12-27 05:19:57,609][105620] Updated weights for policy 1, policy_version 1917334 (0.0007) [2023-12-27 05:19:57,657][105620] Updated weights for policy 1, policy_version 1917344 (0.0008) [2023-12-27 05:19:57,711][105620] Updated weights for policy 1, policy_version 1917354 (0.0008) [2023-12-27 05:19:57,917][105692] Updated weights for policy 0, policy_version 1912760 (0.0010) [2023-12-27 05:19:57,971][105692] Updated weights for policy 0, policy_version 1912770 (0.0010) [2023-12-27 05:19:58,023][105692] Updated weights for policy 0, policy_version 1912780 (0.0010) [2023-12-27 05:19:58,495][105620] Updated weights for policy 1, policy_version 1917364 (0.0008) [2023-12-27 05:19:58,558][105620] Updated weights for policy 1, policy_version 1917374 (0.0009) [2023-12-27 05:19:58,621][105620] Updated weights for policy 1, policy_version 1917384 (0.0006) [2023-12-27 05:19:58,827][105692] Updated weights for policy 0, policy_version 1912790 (0.0010) [2023-12-27 05:19:58,899][105692] Updated weights for policy 0, policy_version 1912800 (0.0012) [2023-12-27 05:19:58,969][105692] Updated weights for policy 0, policy_version 1912810 (0.0012) [2023-12-27 05:19:59,503][105620] Updated weights for policy 1, policy_version 1917394 (0.0007) [2023-12-27 05:19:59,568][105620] Updated weights for policy 1, policy_version 1917404 (0.0006) [2023-12-27 05:19:59,630][105620] Updated weights for policy 1, policy_version 1917414 (0.0006) [2023-12-27 05:19:59,686][105620] Updated weights for policy 1, policy_version 1917424 (0.0005) [2023-12-27 05:19:59,714][105692] Updated weights for policy 0, policy_version 1912820 (0.0007) [2023-12-27 05:19:59,769][105692] Updated weights for policy 0, policy_version 1912830 (0.0006) [2023-12-27 05:19:59,822][105692] Updated weights for policy 0, policy_version 1912840 (0.0009) [2023-12-27 05:20:00,311][105620] Updated weights for policy 1, policy_version 1917434 (0.0010) [2023-12-27 05:20:00,369][105620] Updated weights for policy 1, policy_version 1917444 (0.0008) [2023-12-27 05:20:00,427][105620] Updated weights for policy 1, policy_version 1917454 (0.0005) [2023-12-27 05:20:00,541][105692] Updated weights for policy 0, policy_version 1912850 (0.0008) [2023-12-27 05:20:00,597][105692] Updated weights for policy 0, policy_version 1912860 (0.0009) [2023-12-27 05:20:00,649][105692] Updated weights for policy 0, policy_version 1912870 (0.0009) [2023-12-27 05:20:00,703][105692] Updated weights for policy 0, policy_version 1912880 (0.0007) [2023-12-27 05:20:00,960][105620] Updated weights for policy 1, policy_version 1917464 (0.0009) [2023-12-27 05:20:01,004][105620] Updated weights for policy 1, policy_version 1917474 (0.0010) [2023-12-27 05:20:01,061][105620] Updated weights for policy 1, policy_version 1917484 (0.0010) [2023-12-27 05:20:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 980705280. Throughput: 0: 9937.6, 1: 9576.5. Samples: 980677972. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:20:01,062][104569] Avg episode reward: [(0, '8445.805'), (1, '9162.593')] [2023-12-27 05:20:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001912880_489766912.pth... [2023-12-27 05:20:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001911760_489480192.pth [2023-12-27 05:20:01,085][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001917488_490946560.pth... [2023-12-27 05:20:01,090][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001916304_490643456.pth [2023-12-27 05:20:01,380][105692] Updated weights for policy 0, policy_version 1912890 (0.0010) [2023-12-27 05:20:01,437][105692] Updated weights for policy 0, policy_version 1912900 (0.0006) [2023-12-27 05:20:01,492][105692] Updated weights for policy 0, policy_version 1912910 (0.0010) [2023-12-27 05:20:01,847][105620] Updated weights for policy 1, policy_version 1917494 (0.0010) [2023-12-27 05:20:01,901][105620] Updated weights for policy 1, policy_version 1917504 (0.0010) [2023-12-27 05:20:01,960][105620] Updated weights for policy 1, policy_version 1917514 (0.0010) [2023-12-27 05:20:02,145][105692] Updated weights for policy 0, policy_version 1912920 (0.0006) [2023-12-27 05:20:02,206][105692] Updated weights for policy 0, policy_version 1912930 (0.0005) [2023-12-27 05:20:02,265][105692] Updated weights for policy 0, policy_version 1912940 (0.0007) [2023-12-27 05:20:02,736][105620] Updated weights for policy 1, policy_version 1917524 (0.0010) [2023-12-27 05:20:02,787][105620] Updated weights for policy 1, policy_version 1917534 (0.0009) [2023-12-27 05:20:02,839][105620] Updated weights for policy 1, policy_version 1917544 (0.0009) [2023-12-27 05:20:02,908][105692] Updated weights for policy 0, policy_version 1912950 (0.0007) [2023-12-27 05:20:02,974][105692] Updated weights for policy 0, policy_version 1912960 (0.0008) [2023-12-27 05:20:03,040][105692] Updated weights for policy 0, policy_version 1912970 (0.0008) [2023-12-27 05:20:03,563][105620] Updated weights for policy 1, policy_version 1917554 (0.0009) [2023-12-27 05:20:03,611][105620] Updated weights for policy 1, policy_version 1917564 (0.0005) [2023-12-27 05:20:03,667][105620] Updated weights for policy 1, policy_version 1917574 (0.0005) [2023-12-27 05:20:03,717][105620] Updated weights for policy 1, policy_version 1917584 (0.0005) [2023-12-27 05:20:03,802][105692] Updated weights for policy 0, policy_version 1912980 (0.0007) [2023-12-27 05:20:03,874][105692] Updated weights for policy 0, policy_version 1912990 (0.0007) [2023-12-27 05:20:03,935][105692] Updated weights for policy 0, policy_version 1913000 (0.0010) [2023-12-27 05:20:04,325][105620] Updated weights for policy 1, policy_version 1917594 (0.0007) [2023-12-27 05:20:04,381][105620] Updated weights for policy 1, policy_version 1917604 (0.0007) [2023-12-27 05:20:04,453][105620] Updated weights for policy 1, policy_version 1917614 (0.0007) [2023-12-27 05:20:04,669][105692] Updated weights for policy 0, policy_version 1913011 (0.0010) [2023-12-27 05:20:04,716][105692] Updated weights for policy 0, policy_version 1913021 (0.0008) [2023-12-27 05:20:04,781][105692] Updated weights for policy 0, policy_version 1913031 (0.0008) [2023-12-27 05:20:05,148][105620] Updated weights for policy 1, policy_version 1917624 (0.0006) [2023-12-27 05:20:05,213][105620] Updated weights for policy 1, policy_version 1917634 (0.0005) [2023-12-27 05:20:05,280][105620] Updated weights for policy 1, policy_version 1917644 (0.0008) [2023-12-27 05:20:05,487][105692] Updated weights for policy 0, policy_version 1913041 (0.0006) [2023-12-27 05:20:05,550][105692] Updated weights for policy 0, policy_version 1913051 (0.0005) [2023-12-27 05:20:05,614][105692] Updated weights for policy 0, policy_version 1913061 (0.0008) [2023-12-27 05:20:05,663][105692] Updated weights for policy 0, policy_version 1913071 (0.0010) [2023-12-27 05:20:05,971][105620] Updated weights for policy 1, policy_version 1917654 (0.0010) [2023-12-27 05:20:06,034][105620] Updated weights for policy 1, policy_version 1917664 (0.0010) [2023-12-27 05:20:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 980803584. Throughput: 0: 9810.3, 1: 9697.3. Samples: 980795900. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:20:06,062][104569] Avg episode reward: [(0, '8450.327'), (1, '9070.426')] [2023-12-27 05:20:06,106][105620] Updated weights for policy 1, policy_version 1917674 (0.0011) [2023-12-27 05:20:06,290][105692] Updated weights for policy 0, policy_version 1913081 (0.0011) [2023-12-27 05:20:06,353][105692] Updated weights for policy 0, policy_version 1913091 (0.0011) [2023-12-27 05:20:06,406][105692] Updated weights for policy 0, policy_version 1913101 (0.0011) [2023-12-27 05:20:06,873][105620] Updated weights for policy 1, policy_version 1917684 (0.0009) [2023-12-27 05:20:06,933][105620] Updated weights for policy 1, policy_version 1917694 (0.0011) [2023-12-27 05:20:06,987][105620] Updated weights for policy 1, policy_version 1917704 (0.0011) [2023-12-27 05:20:07,018][105692] Updated weights for policy 0, policy_version 1913111 (0.0007) [2023-12-27 05:20:07,078][105692] Updated weights for policy 0, policy_version 1913121 (0.0005) [2023-12-27 05:20:07,136][105692] Updated weights for policy 0, policy_version 1913131 (0.0006) [2023-12-27 05:20:07,590][105620] Updated weights for policy 1, policy_version 1917714 (0.0010) [2023-12-27 05:20:07,653][105620] Updated weights for policy 1, policy_version 1917724 (0.0005) [2023-12-27 05:20:07,712][105620] Updated weights for policy 1, policy_version 1917734 (0.0009) [2023-12-27 05:20:07,770][105620] Updated weights for policy 1, policy_version 1917744 (0.0010) [2023-12-27 05:20:07,835][105692] Updated weights for policy 0, policy_version 1913141 (0.0010) [2023-12-27 05:20:07,879][105692] Updated weights for policy 0, policy_version 1913151 (0.0010) [2023-12-27 05:20:07,927][105692] Updated weights for policy 0, policy_version 1913161 (0.0010) [2023-12-27 05:20:08,383][105620] Updated weights for policy 1, policy_version 1917754 (0.0010) [2023-12-27 05:20:08,449][105620] Updated weights for policy 1, policy_version 1917764 (0.0011) [2023-12-27 05:20:08,501][105620] Updated weights for policy 1, policy_version 1917774 (0.0010) [2023-12-27 05:20:08,697][105692] Updated weights for policy 0, policy_version 1913171 (0.0010) [2023-12-27 05:20:08,751][105692] Updated weights for policy 0, policy_version 1913181 (0.0010) [2023-12-27 05:20:08,803][105692] Updated weights for policy 0, policy_version 1913191 (0.0010) [2023-12-27 05:20:09,231][105620] Updated weights for policy 1, policy_version 1917784 (0.0008) [2023-12-27 05:20:09,297][105620] Updated weights for policy 1, policy_version 1917794 (0.0009) [2023-12-27 05:20:09,365][105620] Updated weights for policy 1, policy_version 1917804 (0.0009) [2023-12-27 05:20:09,532][105692] Updated weights for policy 0, policy_version 1913201 (0.0010) [2023-12-27 05:20:09,584][105692] Updated weights for policy 0, policy_version 1913211 (0.0010) [2023-12-27 05:20:09,639][105692] Updated weights for policy 0, policy_version 1913221 (0.0010) [2023-12-27 05:20:09,706][105692] Updated weights for policy 0, policy_version 1913231 (0.0011) [2023-12-27 05:20:10,172][105620] Updated weights for policy 1, policy_version 1917814 (0.0009) [2023-12-27 05:20:10,233][105620] Updated weights for policy 1, policy_version 1917824 (0.0008) [2023-12-27 05:20:10,295][105620] Updated weights for policy 1, policy_version 1917834 (0.0008) [2023-12-27 05:20:10,392][105692] Updated weights for policy 0, policy_version 1913241 (0.0010) [2023-12-27 05:20:10,454][105692] Updated weights for policy 0, policy_version 1913251 (0.0011) [2023-12-27 05:20:10,509][105692] Updated weights for policy 0, policy_version 1913261 (0.0010) [2023-12-27 05:20:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 980901888. Throughput: 0: 9826.9, 1: 9789.0. Samples: 980913864. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:20:11,062][104569] Avg episode reward: [(0, '8811.945'), (1, '9162.806')] [2023-12-27 05:20:11,066][105620] Updated weights for policy 1, policy_version 1917844 (0.0008) [2023-12-27 05:20:11,128][105620] Updated weights for policy 1, policy_version 1917854 (0.0008) [2023-12-27 05:20:11,197][105620] Updated weights for policy 1, policy_version 1917864 (0.0008) [2023-12-27 05:20:11,291][105692] Updated weights for policy 0, policy_version 1913271 (0.0011) [2023-12-27 05:20:11,352][105692] Updated weights for policy 0, policy_version 1913281 (0.0010) [2023-12-27 05:20:11,420][105692] Updated weights for policy 0, policy_version 1913291 (0.0010) [2023-12-27 05:20:11,939][105620] Updated weights for policy 1, policy_version 1917874 (0.0008) [2023-12-27 05:20:11,996][105620] Updated weights for policy 1, policy_version 1917884 (0.0005) [2023-12-27 05:20:12,055][105620] Updated weights for policy 1, policy_version 1917894 (0.0005) [2023-12-27 05:20:12,125][105620] Updated weights for policy 1, policy_version 1917904 (0.0005) [2023-12-27 05:20:12,232][105692] Updated weights for policy 0, policy_version 1913301 (0.0009) [2023-12-27 05:20:12,293][105692] Updated weights for policy 0, policy_version 1913311 (0.0010) [2023-12-27 05:20:12,352][105692] Updated weights for policy 0, policy_version 1913321 (0.0009) [2023-12-27 05:20:12,758][105620] Updated weights for policy 1, policy_version 1917914 (0.0009) [2023-12-27 05:20:12,823][105620] Updated weights for policy 1, policy_version 1917924 (0.0009) [2023-12-27 05:20:12,888][105620] Updated weights for policy 1, policy_version 1917934 (0.0009) [2023-12-27 05:20:13,094][105692] Updated weights for policy 0, policy_version 1913331 (0.0009) [2023-12-27 05:20:13,147][105692] Updated weights for policy 0, policy_version 1913341 (0.0008) [2023-12-27 05:20:13,208][105692] Updated weights for policy 0, policy_version 1913351 (0.0009) [2023-12-27 05:20:13,665][105620] Updated weights for policy 1, policy_version 1917944 (0.0008) [2023-12-27 05:20:13,727][105620] Updated weights for policy 1, policy_version 1917954 (0.0006) [2023-12-27 05:20:13,777][105620] Updated weights for policy 1, policy_version 1917964 (0.0005) [2023-12-27 05:20:13,879][105692] Updated weights for policy 0, policy_version 1913361 (0.0010) [2023-12-27 05:20:13,936][105692] Updated weights for policy 0, policy_version 1913371 (0.0010) [2023-12-27 05:20:13,994][105692] Updated weights for policy 0, policy_version 1913381 (0.0010) [2023-12-27 05:20:14,046][105692] Updated weights for policy 0, policy_version 1913391 (0.0010) [2023-12-27 05:20:14,440][105620] Updated weights for policy 1, policy_version 1917974 (0.0008) [2023-12-27 05:20:14,495][105620] Updated weights for policy 1, policy_version 1917984 (0.0008) [2023-12-27 05:20:14,544][105620] Updated weights for policy 1, policy_version 1917994 (0.0008) [2023-12-27 05:20:14,811][105692] Updated weights for policy 0, policy_version 1913401 (0.0009) [2023-12-27 05:20:14,863][105692] Updated weights for policy 0, policy_version 1913411 (0.0010) [2023-12-27 05:20:14,922][105692] Updated weights for policy 0, policy_version 1913421 (0.0010) [2023-12-27 05:20:15,245][105620] Updated weights for policy 1, policy_version 1918004 (0.0008) [2023-12-27 05:20:15,315][105620] Updated weights for policy 1, policy_version 1918014 (0.0008) [2023-12-27 05:20:15,364][105620] Updated weights for policy 1, policy_version 1918024 (0.0009) [2023-12-27 05:20:15,700][105692] Updated weights for policy 0, policy_version 1913431 (0.0009) [2023-12-27 05:20:15,756][105692] Updated weights for policy 0, policy_version 1913441 (0.0009) [2023-12-27 05:20:15,810][105692] Updated weights for policy 0, policy_version 1913451 (0.0009) [2023-12-27 05:20:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 981000192. Throughput: 0: 9721.7, 1: 9824.0. Samples: 980970236. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:20:16,063][104569] Avg episode reward: [(0, '8899.885'), (1, '9345.926')] [2023-12-27 05:20:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001913456_489914368.pth... [2023-12-27 05:20:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001918032_491085824.pth... [2023-12-27 05:20:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001912336_489627648.pth [2023-12-27 05:20:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001916880_490790912.pth [2023-12-27 05:20:16,114][105620] Updated weights for policy 1, policy_version 1918034 (0.0009) [2023-12-27 05:20:16,164][105620] Updated weights for policy 1, policy_version 1918044 (0.0009) [2023-12-27 05:20:16,211][105620] Updated weights for policy 1, policy_version 1918054 (0.0009) [2023-12-27 05:20:16,258][105620] Updated weights for policy 1, policy_version 1918064 (0.0008) [2023-12-27 05:20:16,558][105692] Updated weights for policy 0, policy_version 1913461 (0.0007) [2023-12-27 05:20:16,618][105692] Updated weights for policy 0, policy_version 1913471 (0.0005) [2023-12-27 05:20:16,679][105692] Updated weights for policy 0, policy_version 1913481 (0.0005) [2023-12-27 05:20:17,122][105620] Updated weights for policy 1, policy_version 1918074 (0.0009) [2023-12-27 05:20:17,190][105620] Updated weights for policy 1, policy_version 1918084 (0.0009) [2023-12-27 05:20:17,243][105692] Updated weights for policy 0, policy_version 1913491 (0.0006) [2023-12-27 05:20:17,250][105620] Updated weights for policy 1, policy_version 1918094 (0.0008) [2023-12-27 05:20:17,296][105692] Updated weights for policy 0, policy_version 1913501 (0.0008) [2023-12-27 05:20:17,343][105692] Updated weights for policy 0, policy_version 1913511 (0.0009) [2023-12-27 05:20:17,989][105620] Updated weights for policy 1, policy_version 1918104 (0.0009) [2023-12-27 05:20:18,044][105620] Updated weights for policy 1, policy_version 1918114 (0.0010) [2023-12-27 05:20:18,091][105692] Updated weights for policy 0, policy_version 1913521 (0.0008) [2023-12-27 05:20:18,092][105620] Updated weights for policy 1, policy_version 1918124 (0.0009) [2023-12-27 05:20:18,147][105692] Updated weights for policy 0, policy_version 1913531 (0.0008) [2023-12-27 05:20:18,209][105692] Updated weights for policy 0, policy_version 1913541 (0.0009) [2023-12-27 05:20:18,274][105692] Updated weights for policy 0, policy_version 1913551 (0.0009) [2023-12-27 05:20:18,882][105620] Updated weights for policy 1, policy_version 1918134 (0.0008) [2023-12-27 05:20:18,943][105620] Updated weights for policy 1, policy_version 1918144 (0.0009) [2023-12-27 05:20:19,004][105692] Updated weights for policy 0, policy_version 1913561 (0.0011) [2023-12-27 05:20:19,006][105620] Updated weights for policy 1, policy_version 1918154 (0.0006) [2023-12-27 05:20:19,063][105692] Updated weights for policy 0, policy_version 1913571 (0.0010) [2023-12-27 05:20:19,121][105692] Updated weights for policy 0, policy_version 1913581 (0.0011) [2023-12-27 05:20:19,761][105692] Updated weights for policy 0, policy_version 1913591 (0.0011) [2023-12-27 05:20:19,824][105620] Updated weights for policy 1, policy_version 1918164 (0.0007) [2023-12-27 05:20:19,827][105692] Updated weights for policy 0, policy_version 1913601 (0.0011) [2023-12-27 05:20:19,874][105692] Updated weights for policy 0, policy_version 1913611 (0.0007) [2023-12-27 05:20:19,890][105620] Updated weights for policy 1, policy_version 1918174 (0.0009) [2023-12-27 05:20:19,959][105620] Updated weights for policy 1, policy_version 1918184 (0.0009) [2023-12-27 05:20:20,619][105692] Updated weights for policy 0, policy_version 1913621 (0.0007) [2023-12-27 05:20:20,680][105692] Updated weights for policy 0, policy_version 1913631 (0.0006) [2023-12-27 05:20:20,751][105692] Updated weights for policy 0, policy_version 1913641 (0.0006) [2023-12-27 05:20:20,801][105620] Updated weights for policy 1, policy_version 1918194 (0.0009) [2023-12-27 05:20:20,859][105620] Updated weights for policy 1, policy_version 1918204 (0.0006) [2023-12-27 05:20:20,914][105620] Updated weights for policy 1, policy_version 1918214 (0.0008) [2023-12-27 05:20:20,963][105620] Updated weights for policy 1, policy_version 1918224 (0.0008) [2023-12-27 05:20:21,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 981098496. Throughput: 0: 9625.1, 1: 9821.2. Samples: 981084060. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:20:21,063][104569] Avg episode reward: [(0, '8444.541'), (1, '9253.541')] [2023-12-27 05:20:21,457][105692] Updated weights for policy 0, policy_version 1913651 (0.0010) [2023-12-27 05:20:21,517][105692] Updated weights for policy 0, policy_version 1913661 (0.0011) [2023-12-27 05:20:21,573][105692] Updated weights for policy 0, policy_version 1913671 (0.0011) [2023-12-27 05:20:21,768][105620] Updated weights for policy 1, policy_version 1918234 (0.0009) [2023-12-27 05:20:21,831][105620] Updated weights for policy 1, policy_version 1918244 (0.0007) [2023-12-27 05:20:21,896][105620] Updated weights for policy 1, policy_version 1918254 (0.0010) [2023-12-27 05:20:22,396][105692] Updated weights for policy 0, policy_version 1913681 (0.0008) [2023-12-27 05:20:22,449][105692] Updated weights for policy 0, policy_version 1913691 (0.0008) [2023-12-27 05:20:22,503][105692] Updated weights for policy 0, policy_version 1913701 (0.0008) [2023-12-27 05:20:22,556][105692] Updated weights for policy 0, policy_version 1913711 (0.0008) [2023-12-27 05:20:22,658][105620] Updated weights for policy 1, policy_version 1918264 (0.0011) [2023-12-27 05:20:22,717][105620] Updated weights for policy 1, policy_version 1918274 (0.0011) [2023-12-27 05:20:22,769][105620] Updated weights for policy 1, policy_version 1918284 (0.0010) [2023-12-27 05:20:23,297][105692] Updated weights for policy 0, policy_version 1913721 (0.0006) [2023-12-27 05:20:23,354][105692] Updated weights for policy 0, policy_version 1913731 (0.0006) [2023-12-27 05:20:23,414][105692] Updated weights for policy 0, policy_version 1913741 (0.0006) [2023-12-27 05:20:23,500][105620] Updated weights for policy 1, policy_version 1918294 (0.0007) [2023-12-27 05:20:23,561][105620] Updated weights for policy 1, policy_version 1918304 (0.0006) [2023-12-27 05:20:23,625][105620] Updated weights for policy 1, policy_version 1918314 (0.0010) [2023-12-27 05:20:24,094][105692] Updated weights for policy 0, policy_version 1913751 (0.0009) [2023-12-27 05:20:24,148][105692] Updated weights for policy 0, policy_version 1913761 (0.0006) [2023-12-27 05:20:24,212][105692] Updated weights for policy 0, policy_version 1913771 (0.0010) [2023-12-27 05:20:24,327][105620] Updated weights for policy 1, policy_version 1918324 (0.0010) [2023-12-27 05:20:24,371][105620] Updated weights for policy 1, policy_version 1918334 (0.0010) [2023-12-27 05:20:24,419][105620] Updated weights for policy 1, policy_version 1918344 (0.0010) [2023-12-27 05:20:24,815][105692] Updated weights for policy 0, policy_version 1913781 (0.0005) [2023-12-27 05:20:24,875][105692] Updated weights for policy 0, policy_version 1913791 (0.0007) [2023-12-27 05:20:24,920][105692] Updated weights for policy 0, policy_version 1913801 (0.0010) [2023-12-27 05:20:25,143][105620] Updated weights for policy 1, policy_version 1918354 (0.0010) [2023-12-27 05:20:25,192][105620] Updated weights for policy 1, policy_version 1918364 (0.0010) [2023-12-27 05:20:25,251][105620] Updated weights for policy 1, policy_version 1918374 (0.0009) [2023-12-27 05:20:25,306][105620] Updated weights for policy 1, policy_version 1918384 (0.0010) [2023-12-27 05:20:25,514][105692] Updated weights for policy 0, policy_version 1913811 (0.0007) [2023-12-27 05:20:25,569][105692] Updated weights for policy 0, policy_version 1913821 (0.0006) [2023-12-27 05:20:25,614][105692] Updated weights for policy 0, policy_version 1913831 (0.0005) [2023-12-27 05:20:26,029][105620] Updated weights for policy 1, policy_version 1918394 (0.0011) [2023-12-27 05:20:26,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 981188608. Throughput: 0: 9704.6, 1: 9785.2. Samples: 981199940. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:20:26,062][104569] Avg episode reward: [(0, '8354.584'), (1, '9164.106')] [2023-12-27 05:20:26,081][105620] Updated weights for policy 1, policy_version 1918404 (0.0008) [2023-12-27 05:20:26,146][105620] Updated weights for policy 1, policy_version 1918414 (0.0007) [2023-12-27 05:20:26,223][105692] Updated weights for policy 0, policy_version 1913841 (0.0005) [2023-12-27 05:20:26,284][105692] Updated weights for policy 0, policy_version 1913851 (0.0005) [2023-12-27 05:20:26,342][105692] Updated weights for policy 0, policy_version 1913861 (0.0005) [2023-12-27 05:20:26,403][105692] Updated weights for policy 0, policy_version 1913871 (0.0005) [2023-12-27 05:20:26,864][105620] Updated weights for policy 1, policy_version 1918424 (0.0010) [2023-12-27 05:20:26,898][105692] Updated weights for policy 0, policy_version 1913881 (0.0005) [2023-12-27 05:20:26,918][105620] Updated weights for policy 1, policy_version 1918434 (0.0010) [2023-12-27 05:20:26,959][105692] Updated weights for policy 0, policy_version 1913891 (0.0005) [2023-12-27 05:20:26,973][105620] Updated weights for policy 1, policy_version 1918444 (0.0010) [2023-12-27 05:20:27,012][105692] Updated weights for policy 0, policy_version 1913901 (0.0005) [2023-12-27 05:20:27,578][105692] Updated weights for policy 0, policy_version 1913911 (0.0009) [2023-12-27 05:20:27,626][105692] Updated weights for policy 0, policy_version 1913921 (0.0010) [2023-12-27 05:20:27,669][105692] Updated weights for policy 0, policy_version 1913931 (0.0010) [2023-12-27 05:20:27,725][105620] Updated weights for policy 1, policy_version 1918454 (0.0010) [2023-12-27 05:20:27,785][105620] Updated weights for policy 1, policy_version 1918464 (0.0010) [2023-12-27 05:20:27,845][105620] Updated weights for policy 1, policy_version 1918474 (0.0010) [2023-12-27 05:20:28,408][105692] Updated weights for policy 0, policy_version 1913941 (0.0010) [2023-12-27 05:20:28,464][105692] Updated weights for policy 0, policy_version 1913951 (0.0011) [2023-12-27 05:20:28,524][105692] Updated weights for policy 0, policy_version 1913961 (0.0011) [2023-12-27 05:20:28,573][105620] Updated weights for policy 1, policy_version 1918484 (0.0010) [2023-12-27 05:20:28,626][105620] Updated weights for policy 1, policy_version 1918494 (0.0011) [2023-12-27 05:20:28,684][105620] Updated weights for policy 1, policy_version 1918504 (0.0010) [2023-12-27 05:20:29,222][105692] Updated weights for policy 0, policy_version 1913971 (0.0009) [2023-12-27 05:20:29,281][105692] Updated weights for policy 0, policy_version 1913981 (0.0007) [2023-12-27 05:20:29,348][105692] Updated weights for policy 0, policy_version 1913991 (0.0008) [2023-12-27 05:20:29,446][105620] Updated weights for policy 1, policy_version 1918514 (0.0010) [2023-12-27 05:20:29,506][105620] Updated weights for policy 1, policy_version 1918524 (0.0011) [2023-12-27 05:20:29,561][105620] Updated weights for policy 1, policy_version 1918534 (0.0010) [2023-12-27 05:20:29,620][105620] Updated weights for policy 1, policy_version 1918544 (0.0010) [2023-12-27 05:20:30,072][105692] Updated weights for policy 0, policy_version 1914001 (0.0007) [2023-12-27 05:20:30,120][105692] Updated weights for policy 0, policy_version 1914011 (0.0010) [2023-12-27 05:20:30,178][105692] Updated weights for policy 0, policy_version 1914021 (0.0010) [2023-12-27 05:20:30,230][105692] Updated weights for policy 0, policy_version 1914031 (0.0010) [2023-12-27 05:20:30,373][105620] Updated weights for policy 1, policy_version 1918554 (0.0010) [2023-12-27 05:20:30,424][105620] Updated weights for policy 1, policy_version 1918564 (0.0010) [2023-12-27 05:20:30,471][105620] Updated weights for policy 1, policy_version 1918574 (0.0010) [2023-12-27 05:20:30,977][105692] Updated weights for policy 0, policy_version 1914041 (0.0010) [2023-12-27 05:20:31,033][105692] Updated weights for policy 0, policy_version 1914051 (0.0010) [2023-12-27 05:20:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 981286912. Throughput: 0: 9847.4, 1: 9797.3. Samples: 981261812. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:20:31,063][104569] Avg episode reward: [(0, '8171.515'), (1, '9256.462')] [2023-12-27 05:20:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001918576_491225088.pth... [2023-12-27 05:20:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001917488_490946560.pth [2023-12-27 05:20:31,092][105692] Updated weights for policy 0, policy_version 1914061 (0.0008) [2023-12-27 05:20:31,108][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001914064_490070016.pth... [2023-12-27 05:20:31,112][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001912880_489766912.pth [2023-12-27 05:20:31,236][105620] Updated weights for policy 1, policy_version 1918584 (0.0010) [2023-12-27 05:20:31,293][105620] Updated weights for policy 1, policy_version 1918594 (0.0010) [2023-12-27 05:20:31,357][105620] Updated weights for policy 1, policy_version 1918604 (0.0009) [2023-12-27 05:20:31,823][105692] Updated weights for policy 0, policy_version 1914071 (0.0009) [2023-12-27 05:20:31,879][105692] Updated weights for policy 0, policy_version 1914081 (0.0008) [2023-12-27 05:20:31,926][105692] Updated weights for policy 0, policy_version 1914091 (0.0008) [2023-12-27 05:20:32,089][105620] Updated weights for policy 1, policy_version 1918614 (0.0010) [2023-12-27 05:20:32,146][105620] Updated weights for policy 1, policy_version 1918624 (0.0010) [2023-12-27 05:20:32,204][105620] Updated weights for policy 1, policy_version 1918634 (0.0010) [2023-12-27 05:20:32,694][105692] Updated weights for policy 0, policy_version 1914101 (0.0009) [2023-12-27 05:20:32,757][105692] Updated weights for policy 0, policy_version 1914111 (0.0009) [2023-12-27 05:20:32,813][105692] Updated weights for policy 0, policy_version 1914121 (0.0008) [2023-12-27 05:20:32,865][105620] Updated weights for policy 1, policy_version 1918644 (0.0010) [2023-12-27 05:20:32,925][105620] Updated weights for policy 1, policy_version 1918654 (0.0010) [2023-12-27 05:20:32,985][105620] Updated weights for policy 1, policy_version 1918664 (0.0010) [2023-12-27 05:20:33,581][105692] Updated weights for policy 0, policy_version 1914131 (0.0008) [2023-12-27 05:20:33,636][105692] Updated weights for policy 0, policy_version 1914141 (0.0009) [2023-12-27 05:20:33,652][105620] Updated weights for policy 1, policy_version 1918674 (0.0005) [2023-12-27 05:20:33,688][105692] Updated weights for policy 0, policy_version 1914151 (0.0010) [2023-12-27 05:20:33,704][105620] Updated weights for policy 1, policy_version 1918684 (0.0005) [2023-12-27 05:20:33,756][105620] Updated weights for policy 1, policy_version 1918694 (0.0005) [2023-12-27 05:20:33,803][105620] Updated weights for policy 1, policy_version 1918704 (0.0009) [2023-12-27 05:20:34,429][105692] Updated weights for policy 0, policy_version 1914161 (0.0010) [2023-12-27 05:20:34,471][105620] Updated weights for policy 1, policy_version 1918714 (0.0006) [2023-12-27 05:20:34,496][105692] Updated weights for policy 0, policy_version 1914171 (0.0011) [2023-12-27 05:20:34,531][105620] Updated weights for policy 1, policy_version 1918724 (0.0007) [2023-12-27 05:20:34,562][105692] Updated weights for policy 0, policy_version 1914181 (0.0011) [2023-12-27 05:20:34,593][105620] Updated weights for policy 1, policy_version 1918734 (0.0008) [2023-12-27 05:20:34,625][105692] Updated weights for policy 0, policy_version 1914191 (0.0011) [2023-12-27 05:20:35,254][105692] Updated weights for policy 0, policy_version 1914201 (0.0010) [2023-12-27 05:20:35,309][105692] Updated weights for policy 0, policy_version 1914211 (0.0010) [2023-12-27 05:20:35,371][105692] Updated weights for policy 0, policy_version 1914221 (0.0010) [2023-12-27 05:20:35,376][105620] Updated weights for policy 1, policy_version 1918744 (0.0006) [2023-12-27 05:20:35,444][105620] Updated weights for policy 1, policy_version 1918754 (0.0005) [2023-12-27 05:20:35,504][105620] Updated weights for policy 1, policy_version 1918764 (0.0006) [2023-12-27 05:20:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 981385216. Throughput: 0: 9826.0, 1: 9700.4. Samples: 981377396. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:20:36,062][104569] Avg episode reward: [(0, '8081.800'), (1, '9345.915')] [2023-12-27 05:20:36,079][105620] Updated weights for policy 1, policy_version 1918774 (0.0008) [2023-12-27 05:20:36,125][105692] Updated weights for policy 0, policy_version 1914231 (0.0011) [2023-12-27 05:20:36,142][105620] Updated weights for policy 1, policy_version 1918784 (0.0008) [2023-12-27 05:20:36,189][105692] Updated weights for policy 0, policy_version 1914241 (0.0011) [2023-12-27 05:20:36,194][105620] Updated weights for policy 1, policy_version 1918794 (0.0008) [2023-12-27 05:20:36,265][105692] Updated weights for policy 0, policy_version 1914251 (0.0010) [2023-12-27 05:20:36,924][105620] Updated weights for policy 1, policy_version 1918804 (0.0008) [2023-12-27 05:20:36,990][105620] Updated weights for policy 1, policy_version 1918814 (0.0008) [2023-12-27 05:20:37,048][105620] Updated weights for policy 1, policy_version 1918824 (0.0008) [2023-12-27 05:20:37,072][105692] Updated weights for policy 0, policy_version 1914261 (0.0011) [2023-12-27 05:20:37,119][105692] Updated weights for policy 0, policy_version 1914271 (0.0010) [2023-12-27 05:20:37,164][105692] Updated weights for policy 0, policy_version 1914281 (0.0010) [2023-12-27 05:20:37,782][105620] Updated weights for policy 1, policy_version 1918834 (0.0009) [2023-12-27 05:20:37,829][105692] Updated weights for policy 0, policy_version 1914291 (0.0009) [2023-12-27 05:20:37,843][105620] Updated weights for policy 1, policy_version 1918844 (0.0010) [2023-12-27 05:20:37,893][105692] Updated weights for policy 0, policy_version 1914301 (0.0005) [2023-12-27 05:20:37,906][105620] Updated weights for policy 1, policy_version 1918854 (0.0011) [2023-12-27 05:20:37,953][105692] Updated weights for policy 0, policy_version 1914311 (0.0005) [2023-12-27 05:20:37,966][105620] Updated weights for policy 1, policy_version 1918864 (0.0011) [2023-12-27 05:20:38,584][105692] Updated weights for policy 0, policy_version 1914321 (0.0006) [2023-12-27 05:20:38,648][105692] Updated weights for policy 0, policy_version 1914331 (0.0010) [2023-12-27 05:20:38,650][105620] Updated weights for policy 1, policy_version 1918874 (0.0005) [2023-12-27 05:20:38,698][105620] Updated weights for policy 1, policy_version 1918884 (0.0006) [2023-12-27 05:20:38,703][105692] Updated weights for policy 0, policy_version 1914341 (0.0010) [2023-12-27 05:20:38,756][105692] Updated weights for policy 0, policy_version 1914351 (0.0010) [2023-12-27 05:20:38,756][105620] Updated weights for policy 1, policy_version 1918894 (0.0009) [2023-12-27 05:20:39,522][105620] Updated weights for policy 1, policy_version 1918904 (0.0007) [2023-12-27 05:20:39,531][105692] Updated weights for policy 0, policy_version 1914361 (0.0010) [2023-12-27 05:20:39,579][105620] Updated weights for policy 1, policy_version 1918914 (0.0009) [2023-12-27 05:20:39,594][105692] Updated weights for policy 0, policy_version 1914371 (0.0010) [2023-12-27 05:20:39,638][105620] Updated weights for policy 1, policy_version 1918924 (0.0008) [2023-12-27 05:20:39,648][105692] Updated weights for policy 0, policy_version 1914381 (0.0010) [2023-12-27 05:20:40,352][105692] Updated weights for policy 0, policy_version 1914391 (0.0010) [2023-12-27 05:20:40,405][105692] Updated weights for policy 0, policy_version 1914401 (0.0010) [2023-12-27 05:20:40,455][105620] Updated weights for policy 1, policy_version 1918934 (0.0009) [2023-12-27 05:20:40,458][105692] Updated weights for policy 0, policy_version 1914411 (0.0010) [2023-12-27 05:20:40,513][105620] Updated weights for policy 1, policy_version 1918944 (0.0007) [2023-12-27 05:20:40,574][105620] Updated weights for policy 1, policy_version 1918954 (0.0008) [2023-12-27 05:20:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 981483520. Throughput: 0: 9828.7, 1: 9628.6. Samples: 981493364. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:20:41,063][104569] Avg episode reward: [(0, '8538.664'), (1, '9345.955')] [2023-12-27 05:20:41,212][105692] Updated weights for policy 0, policy_version 1914421 (0.0010) [2023-12-27 05:20:41,272][105620] Updated weights for policy 1, policy_version 1918964 (0.0007) [2023-12-27 05:20:41,275][105692] Updated weights for policy 0, policy_version 1914431 (0.0011) [2023-12-27 05:20:41,331][105692] Updated weights for policy 0, policy_version 1914441 (0.0007) [2023-12-27 05:20:41,334][105620] Updated weights for policy 1, policy_version 1918974 (0.0008) [2023-12-27 05:20:41,416][105620] Updated weights for policy 1, policy_version 1918984 (0.0009) [2023-12-27 05:20:42,129][105692] Updated weights for policy 0, policy_version 1914451 (0.0010) [2023-12-27 05:20:42,166][105620] Updated weights for policy 1, policy_version 1918994 (0.0011) [2023-12-27 05:20:42,189][105692] Updated weights for policy 0, policy_version 1914461 (0.0011) [2023-12-27 05:20:42,234][105620] Updated weights for policy 1, policy_version 1919004 (0.0010) [2023-12-27 05:20:42,250][105692] Updated weights for policy 0, policy_version 1914471 (0.0010) [2023-12-27 05:20:42,298][105620] Updated weights for policy 1, policy_version 1919014 (0.0009) [2023-12-27 05:20:42,362][105620] Updated weights for policy 1, policy_version 1919024 (0.0009) [2023-12-27 05:20:42,952][105620] Updated weights for policy 1, policy_version 1919034 (0.0005) [2023-12-27 05:20:43,007][105620] Updated weights for policy 1, policy_version 1919044 (0.0005) [2023-12-27 05:20:43,026][105692] Updated weights for policy 0, policy_version 1914481 (0.0010) [2023-12-27 05:20:43,058][105620] Updated weights for policy 1, policy_version 1919054 (0.0005) [2023-12-27 05:20:43,073][105692] Updated weights for policy 0, policy_version 1914491 (0.0008) [2023-12-27 05:20:43,126][105692] Updated weights for policy 0, policy_version 1914501 (0.0010) [2023-12-27 05:20:43,186][105692] Updated weights for policy 0, policy_version 1914511 (0.0010) [2023-12-27 05:20:43,569][105620] Updated weights for policy 1, policy_version 1919064 (0.0006) [2023-12-27 05:20:43,619][105620] Updated weights for policy 1, policy_version 1919074 (0.0009) [2023-12-27 05:20:43,668][105620] Updated weights for policy 1, policy_version 1919084 (0.0010) [2023-12-27 05:20:44,003][105692] Updated weights for policy 0, policy_version 1914521 (0.0006) [2023-12-27 05:20:44,058][105692] Updated weights for policy 0, policy_version 1914531 (0.0006) [2023-12-27 05:20:44,117][105692] Updated weights for policy 0, policy_version 1914541 (0.0009) [2023-12-27 05:20:44,414][105620] Updated weights for policy 1, policy_version 1919094 (0.0010) [2023-12-27 05:20:44,479][105620] Updated weights for policy 1, policy_version 1919104 (0.0010) [2023-12-27 05:20:44,534][105620] Updated weights for policy 1, policy_version 1919114 (0.0008) [2023-12-27 05:20:44,836][105692] Updated weights for policy 0, policy_version 1914551 (0.0007) [2023-12-27 05:20:44,897][105692] Updated weights for policy 0, policy_version 1914561 (0.0007) [2023-12-27 05:20:44,954][105692] Updated weights for policy 0, policy_version 1914571 (0.0006) [2023-12-27 05:20:45,190][105620] Updated weights for policy 1, policy_version 1919124 (0.0010) [2023-12-27 05:20:45,257][105620] Updated weights for policy 1, policy_version 1919134 (0.0011) [2023-12-27 05:20:45,306][105620] Updated weights for policy 1, policy_version 1919144 (0.0011) [2023-12-27 05:20:45,640][105692] Updated weights for policy 0, policy_version 1914581 (0.0007) [2023-12-27 05:20:45,694][105692] Updated weights for policy 0, policy_version 1914591 (0.0009) [2023-12-27 05:20:45,753][105692] Updated weights for policy 0, policy_version 1914601 (0.0010) [2023-12-27 05:20:45,965][105620] Updated weights for policy 1, policy_version 1919154 (0.0011) [2023-12-27 05:20:46,013][105620] Updated weights for policy 1, policy_version 1919164 (0.0009) [2023-12-27 05:20:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 981581824. Throughput: 0: 9766.2, 1: 9644.7. Samples: 981551464. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:20:46,062][104569] Avg episode reward: [(0, '9085.450'), (1, '9253.957')] [2023-12-27 05:20:46,064][105620] Updated weights for policy 1, policy_version 1919174 (0.0005) [2023-12-27 05:20:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001914608_490209280.pth... [2023-12-27 05:20:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001913456_489914368.pth [2023-12-27 05:20:46,117][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001919184_491380736.pth... [2023-12-27 05:20:46,119][105620] Updated weights for policy 1, policy_version 1919184 (0.0005) [2023-12-27 05:20:46,120][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001918032_491085824.pth [2023-12-27 05:20:46,387][105692] Updated weights for policy 0, policy_version 1914611 (0.0008) [2023-12-27 05:20:46,455][105692] Updated weights for policy 0, policy_version 1914621 (0.0005) [2023-12-27 05:20:46,512][105692] Updated weights for policy 0, policy_version 1914631 (0.0005) [2023-12-27 05:20:46,648][105620] Updated weights for policy 1, policy_version 1919194 (0.0005) [2023-12-27 05:20:46,712][105620] Updated weights for policy 1, policy_version 1919204 (0.0006) [2023-12-27 05:20:46,771][105620] Updated weights for policy 1, policy_version 1919214 (0.0007) [2023-12-27 05:20:47,060][105692] Updated weights for policy 0, policy_version 1914641 (0.0005) [2023-12-27 05:20:47,114][105692] Updated weights for policy 0, policy_version 1914651 (0.0005) [2023-12-27 05:20:47,161][105692] Updated weights for policy 0, policy_version 1914661 (0.0005) [2023-12-27 05:20:47,217][105692] Updated weights for policy 0, policy_version 1914671 (0.0008) [2023-12-27 05:20:47,479][105620] Updated weights for policy 1, policy_version 1919224 (0.0009) [2023-12-27 05:20:47,534][105620] Updated weights for policy 1, policy_version 1919234 (0.0011) [2023-12-27 05:20:47,582][105620] Updated weights for policy 1, policy_version 1919244 (0.0010) [2023-12-27 05:20:47,815][105692] Updated weights for policy 0, policy_version 1914681 (0.0006) [2023-12-27 05:20:47,881][105692] Updated weights for policy 0, policy_version 1914691 (0.0006) [2023-12-27 05:20:47,933][105692] Updated weights for policy 0, policy_version 1914701 (0.0008) [2023-12-27 05:20:48,339][105620] Updated weights for policy 1, policy_version 1919254 (0.0010) [2023-12-27 05:20:48,401][105620] Updated weights for policy 1, policy_version 1919264 (0.0007) [2023-12-27 05:20:48,464][105620] Updated weights for policy 1, policy_version 1919274 (0.0011) [2023-12-27 05:20:48,611][105692] Updated weights for policy 0, policy_version 1914711 (0.0008) [2023-12-27 05:20:48,671][105692] Updated weights for policy 0, policy_version 1914721 (0.0008) [2023-12-27 05:20:48,729][105692] Updated weights for policy 0, policy_version 1914731 (0.0008) [2023-12-27 05:20:49,209][105620] Updated weights for policy 1, policy_version 1919284 (0.0011) [2023-12-27 05:20:49,267][105620] Updated weights for policy 1, policy_version 1919294 (0.0009) [2023-12-27 05:20:49,326][105620] Updated weights for policy 1, policy_version 1919304 (0.0009) [2023-12-27 05:20:49,501][105692] Updated weights for policy 0, policy_version 1914741 (0.0008) [2023-12-27 05:20:49,558][105692] Updated weights for policy 0, policy_version 1914751 (0.0008) [2023-12-27 05:20:49,617][105692] Updated weights for policy 0, policy_version 1914761 (0.0008) [2023-12-27 05:20:50,109][105620] Updated weights for policy 1, policy_version 1919314 (0.0011) [2023-12-27 05:20:50,164][105620] Updated weights for policy 1, policy_version 1919324 (0.0009) [2023-12-27 05:20:50,231][105620] Updated weights for policy 1, policy_version 1919334 (0.0006) [2023-12-27 05:20:50,289][105692] Updated weights for policy 0, policy_version 1914771 (0.0008) [2023-12-27 05:20:50,292][105620] Updated weights for policy 1, policy_version 1919344 (0.0005) [2023-12-27 05:20:50,351][105692] Updated weights for policy 0, policy_version 1914781 (0.0010) [2023-12-27 05:20:50,410][105692] Updated weights for policy 0, policy_version 1914791 (0.0010) [2023-12-27 05:20:50,924][105620] Updated weights for policy 1, policy_version 1919354 (0.0006) [2023-12-27 05:20:50,983][105620] Updated weights for policy 1, policy_version 1919364 (0.0006) [2023-12-27 05:20:51,047][105620] Updated weights for policy 1, policy_version 1919374 (0.0008) [2023-12-27 05:20:51,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19438.7). Total num frames: 981688320. Throughput: 0: 9863.6, 1: 9650.8. Samples: 981674044. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:20:51,062][104569] Avg episode reward: [(0, '8720.717'), (1, '9254.025')] [2023-12-27 05:20:51,212][105692] Updated weights for policy 0, policy_version 1914801 (0.0010) [2023-12-27 05:20:51,273][105692] Updated weights for policy 0, policy_version 1914811 (0.0011) [2023-12-27 05:20:51,325][105692] Updated weights for policy 0, policy_version 1914821 (0.0007) [2023-12-27 05:20:51,399][105692] Updated weights for policy 0, policy_version 1914831 (0.0011) [2023-12-27 05:20:51,736][105620] Updated weights for policy 1, policy_version 1919384 (0.0008) [2023-12-27 05:20:51,803][105620] Updated weights for policy 1, policy_version 1919394 (0.0008) [2023-12-27 05:20:51,858][105620] Updated weights for policy 1, policy_version 1919404 (0.0007) [2023-12-27 05:20:52,126][105692] Updated weights for policy 0, policy_version 1914841 (0.0010) [2023-12-27 05:20:52,190][105692] Updated weights for policy 0, policy_version 1914851 (0.0011) [2023-12-27 05:20:52,246][105692] Updated weights for policy 0, policy_version 1914861 (0.0011) [2023-12-27 05:20:52,572][105620] Updated weights for policy 1, policy_version 1919414 (0.0009) [2023-12-27 05:20:52,627][105620] Updated weights for policy 1, policy_version 1919424 (0.0008) [2023-12-27 05:20:52,687][105620] Updated weights for policy 1, policy_version 1919434 (0.0008) [2023-12-27 05:20:53,003][105692] Updated weights for policy 0, policy_version 1914871 (0.0010) [2023-12-27 05:20:53,051][105692] Updated weights for policy 0, policy_version 1914881 (0.0010) [2023-12-27 05:20:53,104][105692] Updated weights for policy 0, policy_version 1914891 (0.0010) [2023-12-27 05:20:53,385][105620] Updated weights for policy 1, policy_version 1919444 (0.0009) [2023-12-27 05:20:53,452][105620] Updated weights for policy 1, policy_version 1919454 (0.0008) [2023-12-27 05:20:53,508][105620] Updated weights for policy 1, policy_version 1919464 (0.0009) [2023-12-27 05:20:53,770][105692] Updated weights for policy 0, policy_version 1914901 (0.0008) [2023-12-27 05:20:53,826][105692] Updated weights for policy 0, policy_version 1914911 (0.0009) [2023-12-27 05:20:53,877][105692] Updated weights for policy 0, policy_version 1914921 (0.0010) [2023-12-27 05:20:54,342][105620] Updated weights for policy 1, policy_version 1919474 (0.0010) [2023-12-27 05:20:54,398][105620] Updated weights for policy 1, policy_version 1919484 (0.0008) [2023-12-27 05:20:54,454][105620] Updated weights for policy 1, policy_version 1919494 (0.0009) [2023-12-27 05:20:54,506][105620] Updated weights for policy 1, policy_version 1919504 (0.0009) [2023-12-27 05:20:54,523][105692] Updated weights for policy 0, policy_version 1914931 (0.0009) [2023-12-27 05:20:54,571][105692] Updated weights for policy 0, policy_version 1914941 (0.0009) [2023-12-27 05:20:54,632][105692] Updated weights for policy 0, policy_version 1914951 (0.0010) [2023-12-27 05:20:55,242][105620] Updated weights for policy 1, policy_version 1919514 (0.0008) [2023-12-27 05:20:55,266][105692] Updated weights for policy 0, policy_version 1914961 (0.0010) [2023-12-27 05:20:55,293][105620] Updated weights for policy 1, policy_version 1919524 (0.0005) [2023-12-27 05:20:55,321][105692] Updated weights for policy 0, policy_version 1914971 (0.0010) [2023-12-27 05:20:55,351][105620] Updated weights for policy 1, policy_version 1919534 (0.0005) [2023-12-27 05:20:55,382][105692] Updated weights for policy 0, policy_version 1914981 (0.0010) [2023-12-27 05:20:55,441][105692] Updated weights for policy 0, policy_version 1914991 (0.0009) [2023-12-27 05:20:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 981778432. Throughput: 0: 9856.1, 1: 9644.7. Samples: 981791400. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:20:56,063][104569] Avg episode reward: [(0, '7992.625'), (1, '9260.492')] [2023-12-27 05:20:56,070][105620] Updated weights for policy 1, policy_version 1919544 (0.0005) [2023-12-27 05:20:56,107][105692] Updated weights for policy 0, policy_version 1915001 (0.0010) [2023-12-27 05:20:56,131][105620] Updated weights for policy 1, policy_version 1919554 (0.0006) [2023-12-27 05:20:56,156][105692] Updated weights for policy 0, policy_version 1915011 (0.0010) [2023-12-27 05:20:56,192][105620] Updated weights for policy 1, policy_version 1919564 (0.0005) [2023-12-27 05:20:56,217][105692] Updated weights for policy 0, policy_version 1915021 (0.0010) [2023-12-27 05:20:56,862][105620] Updated weights for policy 1, policy_version 1919574 (0.0006) [2023-12-27 05:20:56,907][105692] Updated weights for policy 0, policy_version 1915031 (0.0009) [2023-12-27 05:20:56,920][105620] Updated weights for policy 1, policy_version 1919584 (0.0006) [2023-12-27 05:20:56,954][105692] Updated weights for policy 0, policy_version 1915041 (0.0008) [2023-12-27 05:20:56,971][105620] Updated weights for policy 1, policy_version 1919594 (0.0010) [2023-12-27 05:20:57,005][105692] Updated weights for policy 0, policy_version 1915051 (0.0005) [2023-12-27 05:20:57,591][105620] Updated weights for policy 1, policy_version 1919604 (0.0008) [2023-12-27 05:20:57,648][105620] Updated weights for policy 1, policy_version 1919614 (0.0005) [2023-12-27 05:20:57,703][105620] Updated weights for policy 1, policy_version 1919624 (0.0005) [2023-12-27 05:20:57,839][105692] Updated weights for policy 0, policy_version 1915061 (0.0009) [2023-12-27 05:20:57,892][105692] Updated weights for policy 0, policy_version 1915071 (0.0010) [2023-12-27 05:20:57,949][105692] Updated weights for policy 0, policy_version 1915081 (0.0009) [2023-12-27 05:20:58,244][105620] Updated weights for policy 1, policy_version 1919634 (0.0007) [2023-12-27 05:20:58,305][105620] Updated weights for policy 1, policy_version 1919644 (0.0008) [2023-12-27 05:20:58,373][105620] Updated weights for policy 1, policy_version 1919654 (0.0009) [2023-12-27 05:20:58,434][105620] Updated weights for policy 1, policy_version 1919664 (0.0009) [2023-12-27 05:20:58,803][105692] Updated weights for policy 0, policy_version 1915091 (0.0009) [2023-12-27 05:20:58,866][105692] Updated weights for policy 0, policy_version 1915101 (0.0009) [2023-12-27 05:20:58,934][105692] Updated weights for policy 0, policy_version 1915111 (0.0008) [2023-12-27 05:20:59,279][105620] Updated weights for policy 1, policy_version 1919674 (0.0008) [2023-12-27 05:20:59,347][105620] Updated weights for policy 1, policy_version 1919684 (0.0010) [2023-12-27 05:20:59,413][105620] Updated weights for policy 1, policy_version 1919694 (0.0008) [2023-12-27 05:20:59,779][105692] Updated weights for policy 0, policy_version 1915121 (0.0008) [2023-12-27 05:20:59,842][105692] Updated weights for policy 0, policy_version 1915131 (0.0007) [2023-12-27 05:20:59,909][105692] Updated weights for policy 0, policy_version 1915141 (0.0008) [2023-12-27 05:20:59,982][105692] Updated weights for policy 0, policy_version 1915151 (0.0009) [2023-12-27 05:21:00,120][105620] Updated weights for policy 1, policy_version 1919704 (0.0006) [2023-12-27 05:21:00,171][105620] Updated weights for policy 1, policy_version 1919714 (0.0005) [2023-12-27 05:21:00,228][105620] Updated weights for policy 1, policy_version 1919724 (0.0005) [2023-12-27 05:21:00,685][105692] Updated weights for policy 0, policy_version 1915161 (0.0010) [2023-12-27 05:21:00,747][105692] Updated weights for policy 0, policy_version 1915171 (0.0010) [2023-12-27 05:21:00,791][105692] Updated weights for policy 0, policy_version 1915181 (0.0007) [2023-12-27 05:21:00,835][105620] Updated weights for policy 1, policy_version 1919734 (0.0008) [2023-12-27 05:21:00,888][105620] Updated weights for policy 1, policy_version 1919745 (0.0010) [2023-12-27 05:21:00,949][105620] Updated weights for policy 1, policy_version 1919755 (0.0010) [2023-12-27 05:21:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 981884928. Throughput: 0: 9850.3, 1: 9704.2. Samples: 981850184. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:21:01,062][104569] Avg episode reward: [(0, '8443.520'), (1, '9260.443')] [2023-12-27 05:21:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001915184_490356736.pth... [2023-12-27 05:21:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001919760_491528192.pth... [2023-12-27 05:21:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001914064_490070016.pth [2023-12-27 05:21:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001918576_491225088.pth [2023-12-27 05:21:01,586][105692] Updated weights for policy 0, policy_version 1915191 (0.0008) [2023-12-27 05:21:01,656][105692] Updated weights for policy 0, policy_version 1915201 (0.0009) [2023-12-27 05:21:01,707][105620] Updated weights for policy 1, policy_version 1919765 (0.0008) [2023-12-27 05:21:01,717][105692] Updated weights for policy 0, policy_version 1915211 (0.0009) [2023-12-27 05:21:01,771][105620] Updated weights for policy 1, policy_version 1919775 (0.0009) [2023-12-27 05:21:01,821][105620] Updated weights for policy 1, policy_version 1919785 (0.0009) [2023-12-27 05:21:02,436][105692] Updated weights for policy 0, policy_version 1915221 (0.0007) [2023-12-27 05:21:02,489][105692] Updated weights for policy 0, policy_version 1915231 (0.0007) [2023-12-27 05:21:02,551][105692] Updated weights for policy 0, policy_version 1915241 (0.0010) [2023-12-27 05:21:02,567][105620] Updated weights for policy 1, policy_version 1919795 (0.0008) [2023-12-27 05:21:02,632][105620] Updated weights for policy 1, policy_version 1919805 (0.0005) [2023-12-27 05:21:02,694][105620] Updated weights for policy 1, policy_version 1919815 (0.0008) [2023-12-27 05:21:03,120][105692] Updated weights for policy 0, policy_version 1915251 (0.0007) [2023-12-27 05:21:03,175][105692] Updated weights for policy 0, policy_version 1915261 (0.0005) [2023-12-27 05:21:03,226][105692] Updated weights for policy 0, policy_version 1915271 (0.0006) [2023-12-27 05:21:03,338][105620] Updated weights for policy 1, policy_version 1919825 (0.0006) [2023-12-27 05:21:03,392][105620] Updated weights for policy 1, policy_version 1919835 (0.0009) [2023-12-27 05:21:03,445][105620] Updated weights for policy 1, policy_version 1919845 (0.0009) [2023-12-27 05:21:03,507][105620] Updated weights for policy 1, policy_version 1919855 (0.0009) [2023-12-27 05:21:03,864][105692] Updated weights for policy 0, policy_version 1915281 (0.0006) [2023-12-27 05:21:03,917][105692] Updated weights for policy 0, policy_version 1915291 (0.0010) [2023-12-27 05:21:03,970][105692] Updated weights for policy 0, policy_version 1915301 (0.0009) [2023-12-27 05:21:04,029][105692] Updated weights for policy 0, policy_version 1915311 (0.0009) [2023-12-27 05:21:04,292][105620] Updated weights for policy 1, policy_version 1919865 (0.0009) [2023-12-27 05:21:04,347][105620] Updated weights for policy 1, policy_version 1919875 (0.0010) [2023-12-27 05:21:04,400][105620] Updated weights for policy 1, policy_version 1919885 (0.0010) [2023-12-27 05:21:04,755][105692] Updated weights for policy 0, policy_version 1915321 (0.0010) [2023-12-27 05:21:04,811][105692] Updated weights for policy 0, policy_version 1915331 (0.0009) [2023-12-27 05:21:04,873][105692] Updated weights for policy 0, policy_version 1915341 (0.0009) [2023-12-27 05:21:05,243][105620] Updated weights for policy 1, policy_version 1919895 (0.0010) [2023-12-27 05:21:05,306][105620] Updated weights for policy 1, policy_version 1919905 (0.0008) [2023-12-27 05:21:05,370][105620] Updated weights for policy 1, policy_version 1919915 (0.0009) [2023-12-27 05:21:05,522][105692] Updated weights for policy 0, policy_version 1915351 (0.0007) [2023-12-27 05:21:05,580][105692] Updated weights for policy 0, policy_version 1915361 (0.0008) [2023-12-27 05:21:05,638][105692] Updated weights for policy 0, policy_version 1915371 (0.0010) [2023-12-27 05:21:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 981975040. Throughput: 0: 9858.4, 1: 9757.5. Samples: 981966772. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:21:06,062][104569] Avg episode reward: [(0, '8717.091'), (1, '9254.454')] [2023-12-27 05:21:06,110][105620] Updated weights for policy 1, policy_version 1919925 (0.0009) [2023-12-27 05:21:06,176][105620] Updated weights for policy 1, policy_version 1919935 (0.0009) [2023-12-27 05:21:06,223][105620] Updated weights for policy 1, policy_version 1919945 (0.0009) [2023-12-27 05:21:06,380][105692] Updated weights for policy 0, policy_version 1915381 (0.0009) [2023-12-27 05:21:06,436][105692] Updated weights for policy 0, policy_version 1915391 (0.0008) [2023-12-27 05:21:06,490][105692] Updated weights for policy 0, policy_version 1915401 (0.0009) [2023-12-27 05:21:07,025][105620] Updated weights for policy 1, policy_version 1919955 (0.0009) [2023-12-27 05:21:07,083][105620] Updated weights for policy 1, policy_version 1919965 (0.0010) [2023-12-27 05:21:07,145][105620] Updated weights for policy 1, policy_version 1919975 (0.0009) [2023-12-27 05:21:07,222][105692] Updated weights for policy 0, policy_version 1915411 (0.0008) [2023-12-27 05:21:07,286][105692] Updated weights for policy 0, policy_version 1915421 (0.0005) [2023-12-27 05:21:07,355][105692] Updated weights for policy 0, policy_version 1915431 (0.0006) [2023-12-27 05:21:07,772][105620] Updated weights for policy 1, policy_version 1919985 (0.0009) [2023-12-27 05:21:07,839][105620] Updated weights for policy 1, policy_version 1919995 (0.0006) [2023-12-27 05:21:07,903][105620] Updated weights for policy 1, policy_version 1920005 (0.0006) [2023-12-27 05:21:07,960][105620] Updated weights for policy 1, policy_version 1920015 (0.0006) [2023-12-27 05:21:08,124][105692] Updated weights for policy 0, policy_version 1915441 (0.0010) [2023-12-27 05:21:08,185][105692] Updated weights for policy 0, policy_version 1915451 (0.0009) [2023-12-27 05:21:08,247][105692] Updated weights for policy 0, policy_version 1915461 (0.0009) [2023-12-27 05:21:08,304][105692] Updated weights for policy 0, policy_version 1915471 (0.0009) [2023-12-27 05:21:08,628][105620] Updated weights for policy 1, policy_version 1920025 (0.0005) [2023-12-27 05:21:08,697][105620] Updated weights for policy 1, policy_version 1920035 (0.0005) [2023-12-27 05:21:08,758][105620] Updated weights for policy 1, policy_version 1920045 (0.0005) [2023-12-27 05:21:09,171][105692] Updated weights for policy 0, policy_version 1915481 (0.0010) [2023-12-27 05:21:09,243][105692] Updated weights for policy 0, policy_version 1915491 (0.0009) [2023-12-27 05:21:09,272][105620] Updated weights for policy 1, policy_version 1920055 (0.0007) [2023-12-27 05:21:09,313][105692] Updated weights for policy 0, policy_version 1915501 (0.0007) [2023-12-27 05:21:09,331][105620] Updated weights for policy 1, policy_version 1920065 (0.0007) [2023-12-27 05:21:09,392][105620] Updated weights for policy 1, policy_version 1920075 (0.0009) [2023-12-27 05:21:10,111][105620] Updated weights for policy 1, policy_version 1920085 (0.0008) [2023-12-27 05:21:10,139][105692] Updated weights for policy 0, policy_version 1915511 (0.0008) [2023-12-27 05:21:10,162][105620] Updated weights for policy 1, policy_version 1920095 (0.0006) [2023-12-27 05:21:10,200][105692] Updated weights for policy 0, policy_version 1915521 (0.0008) [2023-12-27 05:21:10,216][105620] Updated weights for policy 1, policy_version 1920105 (0.0005) [2023-12-27 05:21:10,257][105692] Updated weights for policy 0, policy_version 1915531 (0.0009) [2023-12-27 05:21:10,944][105620] Updated weights for policy 1, policy_version 1920115 (0.0007) [2023-12-27 05:21:10,975][105692] Updated weights for policy 0, policy_version 1915541 (0.0008) [2023-12-27 05:21:10,997][105620] Updated weights for policy 1, policy_version 1920125 (0.0011) [2023-12-27 05:21:11,035][105692] Updated weights for policy 0, policy_version 1915551 (0.0006) [2023-12-27 05:21:11,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19387.7, 300 sec: 19438.7). Total num frames: 982065152. Throughput: 0: 9753.0, 1: 9852.5. Samples: 982082192. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:21:11,063][104569] Avg episode reward: [(0, '8085.906'), (1, '9254.440')] [2023-12-27 05:21:11,064][105620] Updated weights for policy 1, policy_version 1920135 (0.0009) [2023-12-27 05:21:11,095][105692] Updated weights for policy 0, policy_version 1915561 (0.0007) [2023-12-27 05:21:11,841][105692] Updated weights for policy 0, policy_version 1915571 (0.0007) [2023-12-27 05:21:11,906][105692] Updated weights for policy 0, policy_version 1915581 (0.0009) [2023-12-27 05:21:11,907][105620] Updated weights for policy 1, policy_version 1920145 (0.0007) [2023-12-27 05:21:11,965][105692] Updated weights for policy 0, policy_version 1915591 (0.0005) [2023-12-27 05:21:11,967][105620] Updated weights for policy 1, policy_version 1920155 (0.0011) [2023-12-27 05:21:12,031][105620] Updated weights for policy 1, policy_version 1920165 (0.0011) [2023-12-27 05:21:12,086][105620] Updated weights for policy 1, policy_version 1920175 (0.0011) [2023-12-27 05:21:12,710][105692] Updated weights for policy 0, policy_version 1915601 (0.0006) [2023-12-27 05:21:12,774][105692] Updated weights for policy 0, policy_version 1915611 (0.0005) [2023-12-27 05:21:12,837][105692] Updated weights for policy 0, policy_version 1915621 (0.0005) [2023-12-27 05:21:12,866][105620] Updated weights for policy 1, policy_version 1920185 (0.0009) [2023-12-27 05:21:12,903][105692] Updated weights for policy 0, policy_version 1915631 (0.0005) [2023-12-27 05:21:12,925][105620] Updated weights for policy 1, policy_version 1920195 (0.0009) [2023-12-27 05:21:12,977][105620] Updated weights for policy 1, policy_version 1920205 (0.0008) [2023-12-27 05:21:13,482][105692] Updated weights for policy 0, policy_version 1915641 (0.0010) [2023-12-27 05:21:13,540][105692] Updated weights for policy 0, policy_version 1915651 (0.0010) [2023-12-27 05:21:13,598][105692] Updated weights for policy 0, policy_version 1915661 (0.0010) [2023-12-27 05:21:13,664][105620] Updated weights for policy 1, policy_version 1920215 (0.0005) [2023-12-27 05:21:13,713][105620] Updated weights for policy 1, policy_version 1920225 (0.0005) [2023-12-27 05:21:13,761][105620] Updated weights for policy 1, policy_version 1920235 (0.0007) [2023-12-27 05:21:14,229][105692] Updated weights for policy 0, policy_version 1915671 (0.0010) [2023-12-27 05:21:14,286][105692] Updated weights for policy 0, policy_version 1915681 (0.0010) [2023-12-27 05:21:14,334][105692] Updated weights for policy 0, policy_version 1915691 (0.0010) [2023-12-27 05:21:14,361][105620] Updated weights for policy 1, policy_version 1920245 (0.0010) [2023-12-27 05:21:14,412][105620] Updated weights for policy 1, policy_version 1920255 (0.0010) [2023-12-27 05:21:14,456][105620] Updated weights for policy 1, policy_version 1920265 (0.0010) [2023-12-27 05:21:15,079][105620] Updated weights for policy 1, policy_version 1920275 (0.0007) [2023-12-27 05:21:15,098][105692] Updated weights for policy 0, policy_version 1915701 (0.0008) [2023-12-27 05:21:15,145][105620] Updated weights for policy 1, policy_version 1920285 (0.0010) [2023-12-27 05:21:15,158][105692] Updated weights for policy 0, policy_version 1915711 (0.0011) [2023-12-27 05:21:15,202][105620] Updated weights for policy 1, policy_version 1920295 (0.0011) [2023-12-27 05:21:15,223][105692] Updated weights for policy 0, policy_version 1915721 (0.0011) [2023-12-27 05:21:15,841][105692] Updated weights for policy 0, policy_version 1915731 (0.0011) [2023-12-27 05:21:15,896][105692] Updated weights for policy 0, policy_version 1915741 (0.0008) [2023-12-27 05:21:15,944][105620] Updated weights for policy 1, policy_version 1920305 (0.0011) [2023-12-27 05:21:15,948][105692] Updated weights for policy 0, policy_version 1915751 (0.0006) [2023-12-27 05:21:16,002][105620] Updated weights for policy 1, policy_version 1920315 (0.0010) [2023-12-27 05:21:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 982171648. Throughput: 0: 9657.2, 1: 9850.2. Samples: 982139640. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:21:16,062][104569] Avg episode reward: [(0, '8178.034'), (1, '9162.113')] [2023-12-27 05:21:16,067][105620] Updated weights for policy 1, policy_version 1920325 (0.0010) [2023-12-27 05:21:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001915760_490504192.pth... [2023-12-27 05:21:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001914608_490209280.pth [2023-12-27 05:21:16,132][105620] Updated weights for policy 1, policy_version 1920335 (0.0011) [2023-12-27 05:21:16,138][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001920336_491675648.pth... [2023-12-27 05:21:16,142][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001919184_491380736.pth [2023-12-27 05:21:16,553][105692] Updated weights for policy 0, policy_version 1915761 (0.0006) [2023-12-27 05:21:16,605][105692] Updated weights for policy 0, policy_version 1915771 (0.0010) [2023-12-27 05:21:16,657][105692] Updated weights for policy 0, policy_version 1915781 (0.0010) [2023-12-27 05:21:16,708][105692] Updated weights for policy 0, policy_version 1915791 (0.0010) [2023-12-27 05:21:16,875][105620] Updated weights for policy 1, policy_version 1920345 (0.0011) [2023-12-27 05:21:16,937][105620] Updated weights for policy 1, policy_version 1920355 (0.0010) [2023-12-27 05:21:17,003][105620] Updated weights for policy 1, policy_version 1920365 (0.0011) [2023-12-27 05:21:17,305][105692] Updated weights for policy 0, policy_version 1915801 (0.0009) [2023-12-27 05:21:17,364][105692] Updated weights for policy 0, policy_version 1915811 (0.0008) [2023-12-27 05:21:17,418][105692] Updated weights for policy 0, policy_version 1915821 (0.0007) [2023-12-27 05:21:17,722][105620] Updated weights for policy 1, policy_version 1920375 (0.0007) [2023-12-27 05:21:17,768][105620] Updated weights for policy 1, policy_version 1920385 (0.0010) [2023-12-27 05:21:17,812][105620] Updated weights for policy 1, policy_version 1920395 (0.0010) [2023-12-27 05:21:18,117][105692] Updated weights for policy 0, policy_version 1915831 (0.0007) [2023-12-27 05:21:18,164][105692] Updated weights for policy 0, policy_version 1915841 (0.0005) [2023-12-27 05:21:18,213][105692] Updated weights for policy 0, policy_version 1915851 (0.0005) [2023-12-27 05:21:18,537][105620] Updated weights for policy 1, policy_version 1920405 (0.0010) [2023-12-27 05:21:18,593][105620] Updated weights for policy 1, policy_version 1920415 (0.0011) [2023-12-27 05:21:18,653][105620] Updated weights for policy 1, policy_version 1920425 (0.0011) [2023-12-27 05:21:18,921][105692] Updated weights for policy 0, policy_version 1915861 (0.0008) [2023-12-27 05:21:18,981][105692] Updated weights for policy 0, policy_version 1915871 (0.0011) [2023-12-27 05:21:19,044][105692] Updated weights for policy 0, policy_version 1915881 (0.0010) [2023-12-27 05:21:19,367][105620] Updated weights for policy 1, policy_version 1920435 (0.0011) [2023-12-27 05:21:19,426][105620] Updated weights for policy 1, policy_version 1920445 (0.0011) [2023-12-27 05:21:19,478][105620] Updated weights for policy 1, policy_version 1920455 (0.0010) [2023-12-27 05:21:19,801][105692] Updated weights for policy 0, policy_version 1915891 (0.0010) [2023-12-27 05:21:19,874][105692] Updated weights for policy 0, policy_version 1915901 (0.0010) [2023-12-27 05:21:19,932][105692] Updated weights for policy 0, policy_version 1915911 (0.0009) [2023-12-27 05:21:20,301][105620] Updated weights for policy 1, policy_version 1920465 (0.0011) [2023-12-27 05:21:20,361][105620] Updated weights for policy 1, policy_version 1920475 (0.0008) [2023-12-27 05:21:20,423][105620] Updated weights for policy 1, policy_version 1920485 (0.0009) [2023-12-27 05:21:20,483][105620] Updated weights for policy 1, policy_version 1920495 (0.0009) [2023-12-27 05:21:20,603][105692] Updated weights for policy 0, policy_version 1915921 (0.0006) [2023-12-27 05:21:20,666][105692] Updated weights for policy 0, policy_version 1915931 (0.0009) [2023-12-27 05:21:20,731][105692] Updated weights for policy 0, policy_version 1915941 (0.0006) [2023-12-27 05:21:20,795][105692] Updated weights for policy 0, policy_version 1915951 (0.0010) [2023-12-27 05:21:21,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 982269952. Throughput: 0: 9771.7, 1: 9857.1. Samples: 982260696. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:21:21,063][104569] Avg episode reward: [(0, '8447.910'), (1, '9161.276')] [2023-12-27 05:21:21,213][105620] Updated weights for policy 1, policy_version 1920505 (0.0008) [2023-12-27 05:21:21,282][105620] Updated weights for policy 1, policy_version 1920515 (0.0008) [2023-12-27 05:21:21,338][105620] Updated weights for policy 1, policy_version 1920525 (0.0008) [2023-12-27 05:21:21,605][105692] Updated weights for policy 0, policy_version 1915961 (0.0009) [2023-12-27 05:21:21,684][105692] Updated weights for policy 0, policy_version 1915971 (0.0009) [2023-12-27 05:21:21,741][105692] Updated weights for policy 0, policy_version 1915981 (0.0009) [2023-12-27 05:21:22,099][105620] Updated weights for policy 1, policy_version 1920535 (0.0010) [2023-12-27 05:21:22,161][105620] Updated weights for policy 1, policy_version 1920546 (0.0010) [2023-12-27 05:21:22,216][105620] Updated weights for policy 1, policy_version 1920556 (0.0010) [2023-12-27 05:21:22,442][105692] Updated weights for policy 0, policy_version 1915991 (0.0009) [2023-12-27 05:21:22,493][105692] Updated weights for policy 0, policy_version 1916001 (0.0009) [2023-12-27 05:21:22,544][105692] Updated weights for policy 0, policy_version 1916011 (0.0009) [2023-12-27 05:21:23,007][105620] Updated weights for policy 1, policy_version 1920566 (0.0007) [2023-12-27 05:21:23,065][105620] Updated weights for policy 1, policy_version 1920576 (0.0009) [2023-12-27 05:21:23,125][105620] Updated weights for policy 1, policy_version 1920586 (0.0010) [2023-12-27 05:21:23,245][105692] Updated weights for policy 0, policy_version 1916021 (0.0008) [2023-12-27 05:21:23,295][105692] Updated weights for policy 0, policy_version 1916031 (0.0009) [2023-12-27 05:21:23,349][105692] Updated weights for policy 0, policy_version 1916042 (0.0010) [2023-12-27 05:21:23,699][105620] Updated weights for policy 1, policy_version 1920596 (0.0010) [2023-12-27 05:21:23,764][105620] Updated weights for policy 1, policy_version 1920606 (0.0011) [2023-12-27 05:21:23,834][105620] Updated weights for policy 1, policy_version 1920616 (0.0011) [2023-12-27 05:21:24,090][105692] Updated weights for policy 0, policy_version 1916052 (0.0010) [2023-12-27 05:21:24,139][105692] Updated weights for policy 0, policy_version 1916062 (0.0009) [2023-12-27 05:21:24,191][105692] Updated weights for policy 0, policy_version 1916072 (0.0009) [2023-12-27 05:21:24,529][105620] Updated weights for policy 1, policy_version 1920626 (0.0010) [2023-12-27 05:21:24,594][105620] Updated weights for policy 1, policy_version 1920636 (0.0005) [2023-12-27 05:21:24,658][105620] Updated weights for policy 1, policy_version 1920646 (0.0005) [2023-12-27 05:21:24,715][105620] Updated weights for policy 1, policy_version 1920656 (0.0005) [2023-12-27 05:21:25,005][105692] Updated weights for policy 0, policy_version 1916082 (0.0009) [2023-12-27 05:21:25,064][105692] Updated weights for policy 0, policy_version 1916092 (0.0006) [2023-12-27 05:21:25,122][105692] Updated weights for policy 0, policy_version 1916102 (0.0006) [2023-12-27 05:21:25,179][105692] Updated weights for policy 0, policy_version 1916112 (0.0006) [2023-12-27 05:21:25,239][105620] Updated weights for policy 1, policy_version 1920666 (0.0009) [2023-12-27 05:21:25,292][105620] Updated weights for policy 1, policy_version 1920677 (0.0010) [2023-12-27 05:21:25,344][105620] Updated weights for policy 1, policy_version 1920687 (0.0009) [2023-12-27 05:21:25,740][105692] Updated weights for policy 0, policy_version 1916122 (0.0009) [2023-12-27 05:21:25,786][105692] Updated weights for policy 0, policy_version 1916132 (0.0009) [2023-12-27 05:21:25,832][105692] Updated weights for policy 0, policy_version 1916142 (0.0008) [2023-12-27 05:21:26,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 982368256. Throughput: 0: 9768.7, 1: 9881.5. Samples: 982377624. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:21:26,062][104569] Avg episode reward: [(0, '8446.183'), (1, '9163.279')] [2023-12-27 05:21:26,150][105620] Updated weights for policy 1, policy_version 1920697 (0.0009) [2023-12-27 05:21:26,211][105620] Updated weights for policy 1, policy_version 1920707 (0.0009) [2023-12-27 05:21:26,260][105620] Updated weights for policy 1, policy_version 1920717 (0.0008) [2023-12-27 05:21:26,596][105692] Updated weights for policy 0, policy_version 1916152 (0.0010) [2023-12-27 05:21:26,652][105692] Updated weights for policy 0, policy_version 1916162 (0.0009) [2023-12-27 05:21:26,700][105692] Updated weights for policy 0, policy_version 1916172 (0.0009) [2023-12-27 05:21:27,046][105620] Updated weights for policy 1, policy_version 1920727 (0.0009) [2023-12-27 05:21:27,104][105620] Updated weights for policy 1, policy_version 1920737 (0.0009) [2023-12-27 05:21:27,169][105620] Updated weights for policy 1, policy_version 1920747 (0.0009) [2023-12-27 05:21:27,373][105692] Updated weights for policy 0, policy_version 1916182 (0.0007) [2023-12-27 05:21:27,425][105692] Updated weights for policy 0, policy_version 1916192 (0.0008) [2023-12-27 05:21:27,475][105692] Updated weights for policy 0, policy_version 1916202 (0.0009) [2023-12-27 05:21:27,962][105620] Updated weights for policy 1, policy_version 1920757 (0.0009) [2023-12-27 05:21:28,012][105620] Updated weights for policy 1, policy_version 1920767 (0.0009) [2023-12-27 05:21:28,065][105620] Updated weights for policy 1, policy_version 1920777 (0.0008) [2023-12-27 05:21:28,149][105692] Updated weights for policy 0, policy_version 1916212 (0.0007) [2023-12-27 05:21:28,198][105692] Updated weights for policy 0, policy_version 1916222 (0.0005) [2023-12-27 05:21:28,243][105692] Updated weights for policy 0, policy_version 1916232 (0.0005) [2023-12-27 05:21:28,762][105620] Updated weights for policy 1, policy_version 1920787 (0.0009) [2023-12-27 05:21:28,808][105620] Updated weights for policy 1, policy_version 1920797 (0.0008) [2023-12-27 05:21:28,855][105620] Updated weights for policy 1, policy_version 1920807 (0.0009) [2023-12-27 05:21:28,971][105692] Updated weights for policy 0, policy_version 1916242 (0.0008) [2023-12-27 05:21:29,023][105692] Updated weights for policy 0, policy_version 1916252 (0.0009) [2023-12-27 05:21:29,086][105692] Updated weights for policy 0, policy_version 1916262 (0.0009) [2023-12-27 05:21:29,145][105692] Updated weights for policy 0, policy_version 1916272 (0.0009) [2023-12-27 05:21:29,615][105620] Updated weights for policy 1, policy_version 1920817 (0.0009) [2023-12-27 05:21:29,680][105620] Updated weights for policy 1, policy_version 1920827 (0.0009) [2023-12-27 05:21:29,740][105620] Updated weights for policy 1, policy_version 1920837 (0.0009) [2023-12-27 05:21:29,801][105620] Updated weights for policy 1, policy_version 1920847 (0.0009) [2023-12-27 05:21:29,892][105692] Updated weights for policy 0, policy_version 1916282 (0.0009) [2023-12-27 05:21:29,958][105692] Updated weights for policy 0, policy_version 1916292 (0.0008) [2023-12-27 05:21:30,009][105692] Updated weights for policy 0, policy_version 1916302 (0.0010) [2023-12-27 05:21:30,429][105620] Updated weights for policy 1, policy_version 1920857 (0.0006) [2023-12-27 05:21:30,485][105620] Updated weights for policy 1, policy_version 1920867 (0.0005) [2023-12-27 05:21:30,541][105620] Updated weights for policy 1, policy_version 1920877 (0.0005) [2023-12-27 05:21:30,845][105692] Updated weights for policy 0, policy_version 1916312 (0.0006) [2023-12-27 05:21:30,895][105692] Updated weights for policy 0, policy_version 1916322 (0.0005) [2023-12-27 05:21:30,955][105692] Updated weights for policy 0, policy_version 1916332 (0.0005) [2023-12-27 05:21:31,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 982466560. Throughput: 0: 9849.3, 1: 9792.6. Samples: 982435348. Policy #0 lag: (min: 19.0, avg: 28.1, max: 51.0) [2023-12-27 05:21:31,063][104569] Avg episode reward: [(0, '8714.633'), (1, '9164.129')] [2023-12-27 05:21:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001916336_490651648.pth... [2023-12-27 05:21:31,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001915184_490356736.pth [2023-12-27 05:21:31,119][105620] Updated weights for policy 1, policy_version 1920887 (0.0005) [2023-12-27 05:21:31,185][105620] Updated weights for policy 1, policy_version 1920897 (0.0008) [2023-12-27 05:21:31,256][105620] Updated weights for policy 1, policy_version 1920907 (0.0007) [2023-12-27 05:21:31,283][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001920912_491823104.pth... [2023-12-27 05:21:31,286][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001919760_491528192.pth [2023-12-27 05:21:31,578][105692] Updated weights for policy 0, policy_version 1916342 (0.0008) [2023-12-27 05:21:31,643][105692] Updated weights for policy 0, policy_version 1916353 (0.0011) [2023-12-27 05:21:31,698][105692] Updated weights for policy 0, policy_version 1916363 (0.0009) [2023-12-27 05:21:31,943][105620] Updated weights for policy 1, policy_version 1920917 (0.0008) [2023-12-27 05:21:31,998][105620] Updated weights for policy 1, policy_version 1920928 (0.0009) [2023-12-27 05:21:32,049][105620] Updated weights for policy 1, policy_version 1920938 (0.0008) [2023-12-27 05:21:32,451][105692] Updated weights for policy 0, policy_version 1916373 (0.0010) [2023-12-27 05:21:32,511][105692] Updated weights for policy 0, policy_version 1916383 (0.0007) [2023-12-27 05:21:32,566][105692] Updated weights for policy 0, policy_version 1916393 (0.0005) [2023-12-27 05:21:32,802][105620] Updated weights for policy 1, policy_version 1920948 (0.0009) [2023-12-27 05:21:32,861][105620] Updated weights for policy 1, policy_version 1920958 (0.0008) [2023-12-27 05:21:32,912][105620] Updated weights for policy 1, policy_version 1920968 (0.0008) [2023-12-27 05:21:33,275][105692] Updated weights for policy 0, policy_version 1916403 (0.0008) [2023-12-27 05:21:33,333][105692] Updated weights for policy 0, policy_version 1916413 (0.0011) [2023-12-27 05:21:33,381][105692] Updated weights for policy 0, policy_version 1916423 (0.0010) [2023-12-27 05:21:33,661][105620] Updated weights for policy 1, policy_version 1920978 (0.0008) [2023-12-27 05:21:33,712][105620] Updated weights for policy 1, policy_version 1920988 (0.0008) [2023-12-27 05:21:33,761][105620] Updated weights for policy 1, policy_version 1920998 (0.0009) [2023-12-27 05:21:33,812][105620] Updated weights for policy 1, policy_version 1921008 (0.0009) [2023-12-27 05:21:34,026][105692] Updated weights for policy 0, policy_version 1916433 (0.0009) [2023-12-27 05:21:34,077][105692] Updated weights for policy 0, policy_version 1916443 (0.0005) [2023-12-27 05:21:34,135][105692] Updated weights for policy 0, policy_version 1916453 (0.0006) [2023-12-27 05:21:34,203][105692] Updated weights for policy 0, policy_version 1916463 (0.0008) [2023-12-27 05:21:34,665][105620] Updated weights for policy 1, policy_version 1921018 (0.0007) [2023-12-27 05:21:34,725][105620] Updated weights for policy 1, policy_version 1921028 (0.0006) [2023-12-27 05:21:34,793][105620] Updated weights for policy 1, policy_version 1921038 (0.0005) [2023-12-27 05:21:34,843][105692] Updated weights for policy 0, policy_version 1916473 (0.0008) [2023-12-27 05:21:34,910][105692] Updated weights for policy 0, policy_version 1916483 (0.0007) [2023-12-27 05:21:34,972][105692] Updated weights for policy 0, policy_version 1916493 (0.0008) [2023-12-27 05:21:35,399][105620] Updated weights for policy 1, policy_version 1921048 (0.0007) [2023-12-27 05:21:35,459][105620] Updated weights for policy 1, policy_version 1921058 (0.0008) [2023-12-27 05:21:35,526][105620] Updated weights for policy 1, policy_version 1921068 (0.0008) [2023-12-27 05:21:35,652][105692] Updated weights for policy 0, policy_version 1916503 (0.0009) [2023-12-27 05:21:35,701][105692] Updated weights for policy 0, policy_version 1916513 (0.0010) [2023-12-27 05:21:35,745][105692] Updated weights for policy 0, policy_version 1916523 (0.0010) [2023-12-27 05:21:36,062][104569] Fps is (10 sec: 19660.0, 60 sec: 19660.7, 300 sec: 19494.2). Total num frames: 982564864. Throughput: 0: 9764.0, 1: 9757.7. Samples: 982552528. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:21:36,063][104569] Avg episode reward: [(0, '8624.033'), (1, '9254.378')] [2023-12-27 05:21:36,264][105620] Updated weights for policy 1, policy_version 1921078 (0.0009) [2023-12-27 05:21:36,321][105620] Updated weights for policy 1, policy_version 1921088 (0.0009) [2023-12-27 05:21:36,382][105620] Updated weights for policy 1, policy_version 1921098 (0.0008) [2023-12-27 05:21:36,419][105692] Updated weights for policy 0, policy_version 1916533 (0.0008) [2023-12-27 05:21:36,477][105692] Updated weights for policy 0, policy_version 1916543 (0.0009) [2023-12-27 05:21:36,542][105692] Updated weights for policy 0, policy_version 1916553 (0.0007) [2023-12-27 05:21:37,096][105620] Updated weights for policy 1, policy_version 1921108 (0.0008) [2023-12-27 05:21:37,158][105620] Updated weights for policy 1, policy_version 1921118 (0.0009) [2023-12-27 05:21:37,224][105620] Updated weights for policy 1, policy_version 1921128 (0.0009) [2023-12-27 05:21:37,267][105692] Updated weights for policy 0, policy_version 1916563 (0.0008) [2023-12-27 05:21:37,322][105692] Updated weights for policy 0, policy_version 1916573 (0.0005) [2023-12-27 05:21:37,371][105692] Updated weights for policy 0, policy_version 1916583 (0.0005) [2023-12-27 05:21:37,984][105692] Updated weights for policy 0, policy_version 1916593 (0.0007) [2023-12-27 05:21:38,047][105692] Updated weights for policy 0, policy_version 1916603 (0.0009) [2023-12-27 05:21:38,054][105620] Updated weights for policy 1, policy_version 1921138 (0.0008) [2023-12-27 05:21:38,098][105692] Updated weights for policy 0, policy_version 1916613 (0.0007) [2023-12-27 05:21:38,101][105620] Updated weights for policy 1, policy_version 1921148 (0.0006) [2023-12-27 05:21:38,149][105692] Updated weights for policy 0, policy_version 1916623 (0.0007) [2023-12-27 05:21:38,161][105620] Updated weights for policy 1, policy_version 1921158 (0.0009) [2023-12-27 05:21:38,226][105620] Updated weights for policy 1, policy_version 1921168 (0.0008) [2023-12-27 05:21:38,807][105692] Updated weights for policy 0, policy_version 1916633 (0.0010) [2023-12-27 05:21:38,861][105692] Updated weights for policy 0, policy_version 1916643 (0.0009) [2023-12-27 05:21:38,922][105692] Updated weights for policy 0, policy_version 1916653 (0.0007) [2023-12-27 05:21:39,071][105620] Updated weights for policy 1, policy_version 1921178 (0.0009) [2023-12-27 05:21:39,135][105620] Updated weights for policy 1, policy_version 1921188 (0.0009) [2023-12-27 05:21:39,205][105620] Updated weights for policy 1, policy_version 1921198 (0.0009) [2023-12-27 05:21:39,562][105692] Updated weights for policy 0, policy_version 1916663 (0.0008) [2023-12-27 05:21:39,626][105692] Updated weights for policy 0, policy_version 1916673 (0.0008) [2023-12-27 05:21:39,683][105692] Updated weights for policy 0, policy_version 1916683 (0.0008) [2023-12-27 05:21:39,996][105620] Updated weights for policy 1, policy_version 1921208 (0.0011) [2023-12-27 05:21:40,052][105620] Updated weights for policy 1, policy_version 1921218 (0.0010) [2023-12-27 05:21:40,105][105620] Updated weights for policy 1, policy_version 1921228 (0.0010) [2023-12-27 05:21:40,495][105692] Updated weights for policy 0, policy_version 1916693 (0.0009) [2023-12-27 05:21:40,553][105692] Updated weights for policy 0, policy_version 1916703 (0.0008) [2023-12-27 05:21:40,612][105692] Updated weights for policy 0, policy_version 1916713 (0.0008) [2023-12-27 05:21:40,907][105620] Updated weights for policy 1, policy_version 1921238 (0.0009) [2023-12-27 05:21:40,963][105620] Updated weights for policy 1, policy_version 1921248 (0.0005) [2023-12-27 05:21:41,023][105620] Updated weights for policy 1, policy_version 1921258 (0.0006) [2023-12-27 05:21:41,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 982654976. Throughput: 0: 9785.4, 1: 9696.2. Samples: 982668076. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:21:41,063][104569] Avg episode reward: [(0, '8260.077'), (1, '9345.806')] [2023-12-27 05:21:41,397][105692] Updated weights for policy 0, policy_version 1916723 (0.0008) [2023-12-27 05:21:41,457][105692] Updated weights for policy 0, policy_version 1916733 (0.0008) [2023-12-27 05:21:41,517][105692] Updated weights for policy 0, policy_version 1916743 (0.0008) [2023-12-27 05:21:41,758][105620] Updated weights for policy 1, policy_version 1921268 (0.0010) [2023-12-27 05:21:41,828][105620] Updated weights for policy 1, policy_version 1921278 (0.0010) [2023-12-27 05:21:41,903][105620] Updated weights for policy 1, policy_version 1921288 (0.0009) [2023-12-27 05:21:42,243][105692] Updated weights for policy 0, policy_version 1916753 (0.0008) [2023-12-27 05:21:42,310][105692] Updated weights for policy 0, policy_version 1916763 (0.0011) [2023-12-27 05:21:42,377][105692] Updated weights for policy 0, policy_version 1916773 (0.0010) [2023-12-27 05:21:42,435][105692] Updated weights for policy 0, policy_version 1916783 (0.0010) [2023-12-27 05:21:42,677][105620] Updated weights for policy 1, policy_version 1921298 (0.0008) [2023-12-27 05:21:42,738][105620] Updated weights for policy 1, policy_version 1921308 (0.0009) [2023-12-27 05:21:42,797][105620] Updated weights for policy 1, policy_version 1921318 (0.0010) [2023-12-27 05:21:43,082][105692] Updated weights for policy 0, policy_version 1916793 (0.0011) [2023-12-27 05:21:43,137][105692] Updated weights for policy 0, policy_version 1916803 (0.0010) [2023-12-27 05:21:43,201][105692] Updated weights for policy 0, policy_version 1916813 (0.0010) [2023-12-27 05:21:43,545][105620] Updated weights for policy 1, policy_version 1921329 (0.0010) [2023-12-27 05:21:43,610][105620] Updated weights for policy 1, policy_version 1921339 (0.0008) [2023-12-27 05:21:43,663][105620] Updated weights for policy 1, policy_version 1921349 (0.0008) [2023-12-27 05:21:43,714][105620] Updated weights for policy 1, policy_version 1921359 (0.0006) [2023-12-27 05:21:43,949][105692] Updated weights for policy 0, policy_version 1916823 (0.0010) [2023-12-27 05:21:44,010][105692] Updated weights for policy 0, policy_version 1916833 (0.0010) [2023-12-27 05:21:44,068][105692] Updated weights for policy 0, policy_version 1916843 (0.0010) [2023-12-27 05:21:44,329][105620] Updated weights for policy 1, policy_version 1921369 (0.0010) [2023-12-27 05:21:44,393][105620] Updated weights for policy 1, policy_version 1921379 (0.0010) [2023-12-27 05:21:44,454][105620] Updated weights for policy 1, policy_version 1921389 (0.0010) [2023-12-27 05:21:44,798][105692] Updated weights for policy 0, policy_version 1916853 (0.0010) [2023-12-27 05:21:44,861][105692] Updated weights for policy 0, policy_version 1916863 (0.0011) [2023-12-27 05:21:44,930][105692] Updated weights for policy 0, policy_version 1916873 (0.0009) [2023-12-27 05:21:45,186][105620] Updated weights for policy 1, policy_version 1921399 (0.0010) [2023-12-27 05:21:45,231][105620] Updated weights for policy 1, policy_version 1921409 (0.0010) [2023-12-27 05:21:45,283][105620] Updated weights for policy 1, policy_version 1921419 (0.0010) [2023-12-27 05:21:45,598][105692] Updated weights for policy 0, policy_version 1916883 (0.0008) [2023-12-27 05:21:45,664][105692] Updated weights for policy 0, policy_version 1916893 (0.0008) [2023-12-27 05:21:45,720][105692] Updated weights for policy 0, policy_version 1916903 (0.0009) [2023-12-27 05:21:45,972][105620] Updated weights for policy 1, policy_version 1921429 (0.0008) [2023-12-27 05:21:46,034][105620] Updated weights for policy 1, policy_version 1921439 (0.0006) [2023-12-27 05:21:46,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19524.1, 300 sec: 19466.4). Total num frames: 982753280. Throughput: 0: 9811.3, 1: 9620.1. Samples: 982724612. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:21:46,064][104569] Avg episode reward: [(0, '8262.549'), (1, '9253.528')] [2023-12-27 05:21:46,071][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001916912_490799104.pth... [2023-12-27 05:21:46,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001915760_490504192.pth [2023-12-27 05:21:46,076][105585] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/milestones/checkpoint_001916912_490799104.pth [2023-12-27 05:21:46,101][105620] Updated weights for policy 1, policy_version 1921449 (0.0005) [2023-12-27 05:21:46,146][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001921456_491962368.pth... [2023-12-27 05:21:46,151][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001920336_491675648.pth [2023-12-27 05:21:46,152][105586] Saving a milestone ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/milestones/checkpoint_001921456_491962368.pth [2023-12-27 05:21:46,339][105692] Updated weights for policy 0, policy_version 1916913 (0.0009) [2023-12-27 05:21:46,396][105692] Updated weights for policy 0, policy_version 1916923 (0.0010) [2023-12-27 05:21:46,448][105692] Updated weights for policy 0, policy_version 1916933 (0.0010) [2023-12-27 05:21:46,495][105692] Updated weights for policy 0, policy_version 1916943 (0.0010) [2023-12-27 05:21:46,684][105620] Updated weights for policy 1, policy_version 1921459 (0.0005) [2023-12-27 05:21:46,735][105620] Updated weights for policy 1, policy_version 1921469 (0.0005) [2023-12-27 05:21:46,783][105620] Updated weights for policy 1, policy_version 1921479 (0.0005) [2023-12-27 05:21:47,216][105692] Updated weights for policy 0, policy_version 1916953 (0.0008) [2023-12-27 05:21:47,266][105692] Updated weights for policy 0, policy_version 1916963 (0.0007) [2023-12-27 05:21:47,319][105692] Updated weights for policy 0, policy_version 1916973 (0.0008) [2023-12-27 05:21:47,485][105620] Updated weights for policy 1, policy_version 1921489 (0.0008) [2023-12-27 05:21:47,533][105620] Updated weights for policy 1, policy_version 1921499 (0.0010) [2023-12-27 05:21:47,582][105620] Updated weights for policy 1, policy_version 1921509 (0.0010) [2023-12-27 05:21:47,630][105620] Updated weights for policy 1, policy_version 1921519 (0.0010) [2023-12-27 05:21:48,096][105692] Updated weights for policy 0, policy_version 1916983 (0.0008) [2023-12-27 05:21:48,151][105692] Updated weights for policy 0, policy_version 1916993 (0.0008) [2023-12-27 05:21:48,199][105692] Updated weights for policy 0, policy_version 1917003 (0.0008) [2023-12-27 05:21:48,397][105620] Updated weights for policy 1, policy_version 1921529 (0.0011) [2023-12-27 05:21:48,459][105620] Updated weights for policy 1, policy_version 1921539 (0.0010) [2023-12-27 05:21:48,514][105620] Updated weights for policy 1, policy_version 1921549 (0.0010) [2023-12-27 05:21:48,906][105692] Updated weights for policy 0, policy_version 1917013 (0.0010) [2023-12-27 05:21:48,955][105692] Updated weights for policy 0, policy_version 1917023 (0.0010) [2023-12-27 05:21:49,013][105692] Updated weights for policy 0, policy_version 1917033 (0.0009) [2023-12-27 05:21:49,122][105620] Updated weights for policy 1, policy_version 1921559 (0.0007) [2023-12-27 05:21:49,181][105620] Updated weights for policy 1, policy_version 1921569 (0.0005) [2023-12-27 05:21:49,248][105620] Updated weights for policy 1, policy_version 1921579 (0.0007) [2023-12-27 05:21:49,728][105692] Updated weights for policy 0, policy_version 1917043 (0.0006) [2023-12-27 05:21:49,788][105692] Updated weights for policy 0, policy_version 1917053 (0.0008) [2023-12-27 05:21:49,853][105692] Updated weights for policy 0, policy_version 1917063 (0.0008) [2023-12-27 05:21:49,866][105620] Updated weights for policy 1, policy_version 1921589 (0.0008) [2023-12-27 05:21:49,918][105620] Updated weights for policy 1, policy_version 1921599 (0.0006) [2023-12-27 05:21:49,986][105620] Updated weights for policy 1, policy_version 1921609 (0.0010) [2023-12-27 05:21:50,636][105692] Updated weights for policy 0, policy_version 1917073 (0.0008) [2023-12-27 05:21:50,692][105692] Updated weights for policy 0, policy_version 1917083 (0.0008) [2023-12-27 05:21:50,696][105620] Updated weights for policy 1, policy_version 1921619 (0.0010) [2023-12-27 05:21:50,754][105692] Updated weights for policy 0, policy_version 1917093 (0.0009) [2023-12-27 05:21:50,756][105620] Updated weights for policy 1, policy_version 1921629 (0.0011) [2023-12-27 05:21:50,816][105620] Updated weights for policy 1, policy_version 1921639 (0.0011) [2023-12-27 05:21:50,849][105692] Updated weights for policy 0, policy_version 1917103 (0.0009) [2023-12-27 05:21:51,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 982859776. Throughput: 0: 9811.1, 1: 9721.4. Samples: 982845736. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:21:51,062][104569] Avg episode reward: [(0, '8536.399'), (1, '9162.263')] [2023-12-27 05:21:51,546][105692] Updated weights for policy 0, policy_version 1917113 (0.0011) [2023-12-27 05:21:51,573][105620] Updated weights for policy 1, policy_version 1921649 (0.0011) [2023-12-27 05:21:51,606][105692] Updated weights for policy 0, policy_version 1917123 (0.0011) [2023-12-27 05:21:51,641][105620] Updated weights for policy 1, policy_version 1921659 (0.0012) [2023-12-27 05:21:51,670][105692] Updated weights for policy 0, policy_version 1917133 (0.0008) [2023-12-27 05:21:51,707][105620] Updated weights for policy 1, policy_version 1921669 (0.0010) [2023-12-27 05:21:51,773][105620] Updated weights for policy 1, policy_version 1921679 (0.0010) [2023-12-27 05:21:52,397][105692] Updated weights for policy 0, policy_version 1917143 (0.0010) [2023-12-27 05:21:52,449][105692] Updated weights for policy 0, policy_version 1917153 (0.0010) [2023-12-27 05:21:52,512][105692] Updated weights for policy 0, policy_version 1917163 (0.0010) [2023-12-27 05:21:52,534][105620] Updated weights for policy 1, policy_version 1921689 (0.0009) [2023-12-27 05:21:52,597][105620] Updated weights for policy 1, policy_version 1921699 (0.0011) [2023-12-27 05:21:52,657][105620] Updated weights for policy 1, policy_version 1921709 (0.0011) [2023-12-27 05:21:53,172][105692] Updated weights for policy 0, policy_version 1917173 (0.0008) [2023-12-27 05:21:53,228][105692] Updated weights for policy 0, policy_version 1917183 (0.0005) [2023-12-27 05:21:53,241][105620] Updated weights for policy 1, policy_version 1921719 (0.0005) [2023-12-27 05:21:53,273][105692] Updated weights for policy 0, policy_version 1917193 (0.0007) [2023-12-27 05:21:53,294][105620] Updated weights for policy 1, policy_version 1921729 (0.0009) [2023-12-27 05:21:53,350][105620] Updated weights for policy 1, policy_version 1921739 (0.0011) [2023-12-27 05:21:53,900][105692] Updated weights for policy 0, policy_version 1917203 (0.0010) [2023-12-27 05:21:53,946][105620] Updated weights for policy 1, policy_version 1921749 (0.0006) [2023-12-27 05:21:53,956][105692] Updated weights for policy 0, policy_version 1917213 (0.0010) [2023-12-27 05:21:54,000][105620] Updated weights for policy 1, policy_version 1921759 (0.0005) [2023-12-27 05:21:54,011][105692] Updated weights for policy 0, policy_version 1917223 (0.0010) [2023-12-27 05:21:54,061][105620] Updated weights for policy 1, policy_version 1921769 (0.0008) [2023-12-27 05:21:54,589][105692] Updated weights for policy 0, policy_version 1917233 (0.0010) [2023-12-27 05:21:54,610][105620] Updated weights for policy 1, policy_version 1921779 (0.0008) [2023-12-27 05:21:54,649][105692] Updated weights for policy 0, policy_version 1917243 (0.0006) [2023-12-27 05:21:54,663][105620] Updated weights for policy 1, policy_version 1921789 (0.0007) [2023-12-27 05:21:54,705][105692] Updated weights for policy 0, policy_version 1917253 (0.0007) [2023-12-27 05:21:54,719][105620] Updated weights for policy 1, policy_version 1921799 (0.0011) [2023-12-27 05:21:54,760][105692] Updated weights for policy 0, policy_version 1917263 (0.0006) [2023-12-27 05:21:55,401][105692] Updated weights for policy 0, policy_version 1917273 (0.0006) [2023-12-27 05:21:55,426][105620] Updated weights for policy 1, policy_version 1921809 (0.0010) [2023-12-27 05:21:55,462][105692] Updated weights for policy 0, policy_version 1917283 (0.0005) [2023-12-27 05:21:55,489][105620] Updated weights for policy 1, policy_version 1921819 (0.0011) [2023-12-27 05:21:55,523][105692] Updated weights for policy 0, policy_version 1917293 (0.0006) [2023-12-27 05:21:55,542][105620] Updated weights for policy 1, policy_version 1921829 (0.0011) [2023-12-27 05:21:55,598][105620] Updated weights for policy 1, policy_version 1921839 (0.0011) [2023-12-27 05:21:56,053][105692] Updated weights for policy 0, policy_version 1917303 (0.0007) [2023-12-27 05:21:56,062][104569] Fps is (10 sec: 20481.3, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 982958080. Throughput: 0: 9947.5, 1: 9742.0. Samples: 982968216. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:21:56,062][104569] Avg episode reward: [(0, '8535.599'), (1, '9163.130')] [2023-12-27 05:21:56,102][105692] Updated weights for policy 0, policy_version 1917313 (0.0009) [2023-12-27 05:21:56,150][105692] Updated weights for policy 0, policy_version 1917323 (0.0009) [2023-12-27 05:21:56,396][105620] Updated weights for policy 1, policy_version 1921849 (0.0009) [2023-12-27 05:21:56,447][105620] Updated weights for policy 1, policy_version 1921859 (0.0008) [2023-12-27 05:21:56,510][105620] Updated weights for policy 1, policy_version 1921869 (0.0009) [2023-12-27 05:21:56,832][105692] Updated weights for policy 0, policy_version 1917333 (0.0007) [2023-12-27 05:21:56,881][105692] Updated weights for policy 0, policy_version 1917343 (0.0005) [2023-12-27 05:21:56,929][105692] Updated weights for policy 0, policy_version 1917353 (0.0005) [2023-12-27 05:21:57,330][105620] Updated weights for policy 1, policy_version 1921879 (0.0010) [2023-12-27 05:21:57,379][105620] Updated weights for policy 1, policy_version 1921889 (0.0010) [2023-12-27 05:21:57,440][105620] Updated weights for policy 1, policy_version 1921899 (0.0009) [2023-12-27 05:21:57,460][105692] Updated weights for policy 0, policy_version 1917363 (0.0006) [2023-12-27 05:21:57,511][105692] Updated weights for policy 0, policy_version 1917373 (0.0010) [2023-12-27 05:21:57,572][105692] Updated weights for policy 0, policy_version 1917383 (0.0010) [2023-12-27 05:21:58,034][105620] Updated weights for policy 1, policy_version 1921909 (0.0008) [2023-12-27 05:21:58,094][105620] Updated weights for policy 1, policy_version 1921919 (0.0009) [2023-12-27 05:21:58,150][105620] Updated weights for policy 1, policy_version 1921929 (0.0009) [2023-12-27 05:21:58,289][105692] Updated weights for policy 0, policy_version 1917393 (0.0010) [2023-12-27 05:21:58,352][105692] Updated weights for policy 0, policy_version 1917403 (0.0009) [2023-12-27 05:21:58,426][105692] Updated weights for policy 0, policy_version 1917413 (0.0010) [2023-12-27 05:21:58,493][105692] Updated weights for policy 0, policy_version 1917423 (0.0011) [2023-12-27 05:21:58,991][105620] Updated weights for policy 1, policy_version 1921939 (0.0008) [2023-12-27 05:21:59,080][105620] Updated weights for policy 1, policy_version 1921949 (0.0007) [2023-12-27 05:21:59,141][105620] Updated weights for policy 1, policy_version 1921959 (0.0006) [2023-12-27 05:21:59,273][105692] Updated weights for policy 0, policy_version 1917433 (0.0008) [2023-12-27 05:21:59,334][105692] Updated weights for policy 0, policy_version 1917443 (0.0008) [2023-12-27 05:21:59,399][105692] Updated weights for policy 0, policy_version 1917453 (0.0008) [2023-12-27 05:21:59,869][105620] Updated weights for policy 1, policy_version 1921969 (0.0007) [2023-12-27 05:21:59,930][105620] Updated weights for policy 1, policy_version 1921979 (0.0009) [2023-12-27 05:21:59,991][105620] Updated weights for policy 1, policy_version 1921989 (0.0009) [2023-12-27 05:22:00,036][105692] Updated weights for policy 0, policy_version 1917463 (0.0007) [2023-12-27 05:22:00,044][105620] Updated weights for policy 1, policy_version 1921999 (0.0008) [2023-12-27 05:22:00,100][105692] Updated weights for policy 0, policy_version 1917473 (0.0006) [2023-12-27 05:22:00,160][105692] Updated weights for policy 0, policy_version 1917483 (0.0008) [2023-12-27 05:22:00,771][105620] Updated weights for policy 1, policy_version 1922009 (0.0006) [2023-12-27 05:22:00,831][105620] Updated weights for policy 1, policy_version 1922019 (0.0010) [2023-12-27 05:22:00,879][105620] Updated weights for policy 1, policy_version 1922029 (0.0010) [2023-12-27 05:22:00,906][105692] Updated weights for policy 0, policy_version 1917493 (0.0007) [2023-12-27 05:22:00,966][105692] Updated weights for policy 0, policy_version 1917503 (0.0009) [2023-12-27 05:22:01,014][105692] Updated weights for policy 0, policy_version 1917513 (0.0010) [2023-12-27 05:22:01,062][104569] Fps is (10 sec: 20479.6, 60 sec: 19660.7, 300 sec: 19521.9). Total num frames: 983064576. Throughput: 0: 10020.0, 1: 9746.4. Samples: 983029132. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:22:01,063][104569] Avg episode reward: [(0, '8356.727'), (1, '9254.452')] [2023-12-27 05:22:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001917520_490954752.pth... [2023-12-27 05:22:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001922032_492109824.pth... [2023-12-27 05:22:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001920912_491823104.pth [2023-12-27 05:22:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001916336_490651648.pth [2023-12-27 05:22:01,582][105620] Updated weights for policy 1, policy_version 1922039 (0.0010) [2023-12-27 05:22:01,648][105620] Updated weights for policy 1, policy_version 1922049 (0.0011) [2023-12-27 05:22:01,707][105620] Updated weights for policy 1, policy_version 1922059 (0.0010) [2023-12-27 05:22:01,755][105692] Updated weights for policy 0, policy_version 1917523 (0.0011) [2023-12-27 05:22:01,821][105692] Updated weights for policy 0, policy_version 1917533 (0.0010) [2023-12-27 05:22:01,884][105692] Updated weights for policy 0, policy_version 1917543 (0.0010) [2023-12-27 05:22:02,335][105620] Updated weights for policy 1, policy_version 1922069 (0.0008) [2023-12-27 05:22:02,402][105620] Updated weights for policy 1, policy_version 1922079 (0.0006) [2023-12-27 05:22:02,467][105620] Updated weights for policy 1, policy_version 1922089 (0.0005) [2023-12-27 05:22:02,730][105692] Updated weights for policy 0, policy_version 1917553 (0.0009) [2023-12-27 05:22:02,784][105692] Updated weights for policy 0, policy_version 1917563 (0.0010) [2023-12-27 05:22:02,838][105692] Updated weights for policy 0, policy_version 1917573 (0.0009) [2023-12-27 05:22:02,892][105692] Updated weights for policy 0, policy_version 1917583 (0.0010) [2023-12-27 05:22:03,018][105620] Updated weights for policy 1, policy_version 1922099 (0.0006) [2023-12-27 05:22:03,079][105620] Updated weights for policy 1, policy_version 1922109 (0.0005) [2023-12-27 05:22:03,134][105620] Updated weights for policy 1, policy_version 1922119 (0.0005) [2023-12-27 05:22:03,639][105620] Updated weights for policy 1, policy_version 1922129 (0.0005) [2023-12-27 05:22:03,684][105620] Updated weights for policy 1, policy_version 1922139 (0.0005) [2023-12-27 05:22:03,729][105620] Updated weights for policy 1, policy_version 1922149 (0.0005) [2023-12-27 05:22:03,779][105620] Updated weights for policy 1, policy_version 1922159 (0.0005) [2023-12-27 05:22:03,832][105692] Updated weights for policy 0, policy_version 1917593 (0.0009) [2023-12-27 05:22:03,899][105692] Updated weights for policy 0, policy_version 1917603 (0.0009) [2023-12-27 05:22:03,959][105692] Updated weights for policy 0, policy_version 1917613 (0.0010) [2023-12-27 05:22:04,488][105620] Updated weights for policy 1, policy_version 1922169 (0.0006) [2023-12-27 05:22:04,548][105620] Updated weights for policy 1, policy_version 1922179 (0.0005) [2023-12-27 05:22:04,606][105620] Updated weights for policy 1, policy_version 1922189 (0.0008) [2023-12-27 05:22:04,633][105692] Updated weights for policy 0, policy_version 1917623 (0.0009) [2023-12-27 05:22:04,691][105692] Updated weights for policy 0, policy_version 1917633 (0.0009) [2023-12-27 05:22:04,751][105692] Updated weights for policy 0, policy_version 1917643 (0.0008) [2023-12-27 05:22:05,306][105620] Updated weights for policy 1, policy_version 1922199 (0.0010) [2023-12-27 05:22:05,327][105692] Updated weights for policy 0, policy_version 1917653 (0.0006) [2023-12-27 05:22:05,358][105620] Updated weights for policy 1, policy_version 1922209 (0.0008) [2023-12-27 05:22:05,375][105692] Updated weights for policy 0, policy_version 1917663 (0.0005) [2023-12-27 05:22:05,406][105620] Updated weights for policy 1, policy_version 1922219 (0.0008) [2023-12-27 05:22:05,430][105692] Updated weights for policy 0, policy_version 1917673 (0.0005) [2023-12-27 05:22:05,964][105692] Updated weights for policy 0, policy_version 1917683 (0.0005) [2023-12-27 05:22:06,015][105692] Updated weights for policy 0, policy_version 1917693 (0.0009) [2023-12-27 05:22:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19494.2). Total num frames: 983154688. Throughput: 0: 9853.4, 1: 9832.2. Samples: 983146544. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:22:06,062][104569] Avg episode reward: [(0, '8083.274'), (1, '9345.903')] [2023-12-27 05:22:06,068][105692] Updated weights for policy 0, policy_version 1917703 (0.0009) [2023-12-27 05:22:06,119][105620] Updated weights for policy 1, policy_version 1922229 (0.0008) [2023-12-27 05:22:06,190][105620] Updated weights for policy 1, policy_version 1922239 (0.0008) [2023-12-27 05:22:06,258][105620] Updated weights for policy 1, policy_version 1922249 (0.0008) [2023-12-27 05:22:06,792][105692] Updated weights for policy 0, policy_version 1917713 (0.0009) [2023-12-27 05:22:06,859][105692] Updated weights for policy 0, policy_version 1917723 (0.0006) [2023-12-27 05:22:06,919][105692] Updated weights for policy 0, policy_version 1917733 (0.0008) [2023-12-27 05:22:06,978][105692] Updated weights for policy 0, policy_version 1917743 (0.0009) [2023-12-27 05:22:07,043][105620] Updated weights for policy 1, policy_version 1922259 (0.0006) [2023-12-27 05:22:07,099][105620] Updated weights for policy 1, policy_version 1922269 (0.0009) [2023-12-27 05:22:07,147][105620] Updated weights for policy 1, policy_version 1922279 (0.0009) [2023-12-27 05:22:07,731][105692] Updated weights for policy 0, policy_version 1917753 (0.0010) [2023-12-27 05:22:07,788][105692] Updated weights for policy 0, policy_version 1917763 (0.0008) [2023-12-27 05:22:07,854][105692] Updated weights for policy 0, policy_version 1917773 (0.0010) [2023-12-27 05:22:07,889][105620] Updated weights for policy 1, policy_version 1922289 (0.0009) [2023-12-27 05:22:07,945][105620] Updated weights for policy 1, policy_version 1922299 (0.0005) [2023-12-27 05:22:07,995][105620] Updated weights for policy 1, policy_version 1922309 (0.0005) [2023-12-27 05:22:08,053][105620] Updated weights for policy 1, policy_version 1922319 (0.0011) [2023-12-27 05:22:08,614][105692] Updated weights for policy 0, policy_version 1917783 (0.0007) [2023-12-27 05:22:08,671][105692] Updated weights for policy 0, policy_version 1917793 (0.0008) [2023-12-27 05:22:08,735][105692] Updated weights for policy 0, policy_version 1917803 (0.0006) [2023-12-27 05:22:08,736][105620] Updated weights for policy 1, policy_version 1922329 (0.0009) [2023-12-27 05:22:08,792][105620] Updated weights for policy 1, policy_version 1922339 (0.0007) [2023-12-27 05:22:08,849][105620] Updated weights for policy 1, policy_version 1922349 (0.0010) [2023-12-27 05:22:09,444][105692] Updated weights for policy 0, policy_version 1917813 (0.0009) [2023-12-27 05:22:09,508][105692] Updated weights for policy 0, policy_version 1917823 (0.0008) [2023-12-27 05:22:09,571][105692] Updated weights for policy 0, policy_version 1917833 (0.0008) [2023-12-27 05:22:09,606][105620] Updated weights for policy 1, policy_version 1922359 (0.0007) [2023-12-27 05:22:09,665][105620] Updated weights for policy 1, policy_version 1922369 (0.0007) [2023-12-27 05:22:09,720][105620] Updated weights for policy 1, policy_version 1922379 (0.0008) [2023-12-27 05:22:10,372][105620] Updated weights for policy 1, policy_version 1922389 (0.0006) [2023-12-27 05:22:10,390][105692] Updated weights for policy 0, policy_version 1917843 (0.0009) [2023-12-27 05:22:10,436][105620] Updated weights for policy 1, policy_version 1922399 (0.0006) [2023-12-27 05:22:10,454][105692] Updated weights for policy 0, policy_version 1917853 (0.0006) [2023-12-27 05:22:10,505][105620] Updated weights for policy 1, policy_version 1922409 (0.0008) [2023-12-27 05:22:10,519][105692] Updated weights for policy 0, policy_version 1917863 (0.0007) [2023-12-27 05:22:11,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19797.4, 300 sec: 19494.2). Total num frames: 983252992. Throughput: 0: 9880.6, 1: 9837.2. Samples: 983264928. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:22:11,062][104569] Avg episode reward: [(0, '8080.727'), (1, '9345.977')] [2023-12-27 05:22:11,095][105620] Updated weights for policy 1, policy_version 1922419 (0.0008) [2023-12-27 05:22:11,165][105620] Updated weights for policy 1, policy_version 1922429 (0.0009) [2023-12-27 05:22:11,222][105620] Updated weights for policy 1, policy_version 1922439 (0.0008) [2023-12-27 05:22:11,276][105692] Updated weights for policy 0, policy_version 1917873 (0.0006) [2023-12-27 05:22:11,338][105692] Updated weights for policy 0, policy_version 1917883 (0.0009) [2023-12-27 05:22:11,405][105692] Updated weights for policy 0, policy_version 1917893 (0.0008) [2023-12-27 05:22:11,467][105692] Updated weights for policy 0, policy_version 1917903 (0.0008) [2023-12-27 05:22:11,906][105620] Updated weights for policy 1, policy_version 1922449 (0.0009) [2023-12-27 05:22:11,959][105620] Updated weights for policy 1, policy_version 1922459 (0.0009) [2023-12-27 05:22:12,019][105620] Updated weights for policy 1, policy_version 1922469 (0.0008) [2023-12-27 05:22:12,081][105620] Updated weights for policy 1, policy_version 1922479 (0.0009) [2023-12-27 05:22:12,256][105692] Updated weights for policy 0, policy_version 1917913 (0.0006) [2023-12-27 05:22:12,313][105692] Updated weights for policy 0, policy_version 1917923 (0.0009) [2023-12-27 05:22:12,380][105692] Updated weights for policy 0, policy_version 1917933 (0.0008) [2023-12-27 05:22:12,839][105620] Updated weights for policy 1, policy_version 1922489 (0.0007) [2023-12-27 05:22:12,896][105620] Updated weights for policy 1, policy_version 1922499 (0.0006) [2023-12-27 05:22:12,958][105620] Updated weights for policy 1, policy_version 1922509 (0.0006) [2023-12-27 05:22:13,185][105692] Updated weights for policy 0, policy_version 1917943 (0.0010) [2023-12-27 05:22:13,252][105692] Updated weights for policy 0, policy_version 1917953 (0.0010) [2023-12-27 05:22:13,314][105692] Updated weights for policy 0, policy_version 1917963 (0.0009) [2023-12-27 05:22:13,528][105620] Updated weights for policy 1, policy_version 1922519 (0.0007) [2023-12-27 05:22:13,579][105620] Updated weights for policy 1, policy_version 1922529 (0.0009) [2023-12-27 05:22:13,632][105620] Updated weights for policy 1, policy_version 1922539 (0.0009) [2023-12-27 05:22:14,087][105692] Updated weights for policy 0, policy_version 1917973 (0.0009) [2023-12-27 05:22:14,136][105692] Updated weights for policy 0, policy_version 1917983 (0.0008) [2023-12-27 05:22:14,189][105692] Updated weights for policy 0, policy_version 1917993 (0.0010) [2023-12-27 05:22:14,321][105620] Updated weights for policy 1, policy_version 1922549 (0.0008) [2023-12-27 05:22:14,381][105620] Updated weights for policy 1, policy_version 1922559 (0.0005) [2023-12-27 05:22:14,434][105620] Updated weights for policy 1, policy_version 1922569 (0.0007) [2023-12-27 05:22:14,988][105620] Updated weights for policy 1, policy_version 1922579 (0.0009) [2023-12-27 05:22:15,050][105620] Updated weights for policy 1, policy_version 1922589 (0.0009) [2023-12-27 05:22:15,065][105692] Updated weights for policy 0, policy_version 1918003 (0.0009) [2023-12-27 05:22:15,110][105620] Updated weights for policy 1, policy_version 1922599 (0.0006) [2023-12-27 05:22:15,128][105692] Updated weights for policy 0, policy_version 1918013 (0.0009) [2023-12-27 05:22:15,185][105692] Updated weights for policy 0, policy_version 1918023 (0.0008) [2023-12-27 05:22:15,756][105620] Updated weights for policy 1, policy_version 1922609 (0.0006) [2023-12-27 05:22:15,823][105620] Updated weights for policy 1, policy_version 1922619 (0.0005) [2023-12-27 05:22:15,885][105692] Updated weights for policy 0, policy_version 1918033 (0.0005) [2023-12-27 05:22:15,889][105620] Updated weights for policy 1, policy_version 1922629 (0.0005) [2023-12-27 05:22:15,943][105620] Updated weights for policy 1, policy_version 1922639 (0.0005) [2023-12-27 05:22:15,943][105692] Updated weights for policy 0, policy_version 1918043 (0.0008) [2023-12-27 05:22:16,001][105692] Updated weights for policy 0, policy_version 1918053 (0.0009) [2023-12-27 05:22:16,048][105692] Updated weights for policy 0, policy_version 1918063 (0.0008) [2023-12-27 05:22:16,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 983359488. Throughput: 0: 9787.8, 1: 9900.1. Samples: 983321308. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:22:16,063][104569] Avg episode reward: [(0, '8442.691'), (1, '9253.688')] [2023-12-27 05:22:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001918064_491094016.pth... [2023-12-27 05:22:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001922640_492265472.pth... [2023-12-27 05:22:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001921456_491962368.pth [2023-12-27 05:22:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001916912_490799104.pth [2023-12-27 05:22:16,514][105620] Updated weights for policy 1, policy_version 1922649 (0.0005) [2023-12-27 05:22:16,579][105620] Updated weights for policy 1, policy_version 1922659 (0.0008) [2023-12-27 05:22:16,641][105620] Updated weights for policy 1, policy_version 1922669 (0.0009) [2023-12-27 05:22:16,882][105692] Updated weights for policy 0, policy_version 1918073 (0.0009) [2023-12-27 05:22:16,929][105692] Updated weights for policy 0, policy_version 1918083 (0.0009) [2023-12-27 05:22:16,980][105692] Updated weights for policy 0, policy_version 1918093 (0.0009) [2023-12-27 05:22:17,333][105620] Updated weights for policy 1, policy_version 1922679 (0.0009) [2023-12-27 05:22:17,393][105620] Updated weights for policy 1, policy_version 1922689 (0.0009) [2023-12-27 05:22:17,456][105620] Updated weights for policy 1, policy_version 1922699 (0.0008) [2023-12-27 05:22:17,756][105692] Updated weights for policy 0, policy_version 1918103 (0.0009) [2023-12-27 05:22:17,819][105692] Updated weights for policy 0, policy_version 1918113 (0.0009) [2023-12-27 05:22:17,873][105692] Updated weights for policy 0, policy_version 1918123 (0.0009) [2023-12-27 05:22:18,207][105620] Updated weights for policy 1, policy_version 1922709 (0.0009) [2023-12-27 05:22:18,264][105620] Updated weights for policy 1, policy_version 1922719 (0.0009) [2023-12-27 05:22:18,311][105620] Updated weights for policy 1, policy_version 1922729 (0.0008) [2023-12-27 05:22:18,582][105692] Updated weights for policy 0, policy_version 1918133 (0.0007) [2023-12-27 05:22:18,646][105692] Updated weights for policy 0, policy_version 1918143 (0.0009) [2023-12-27 05:22:18,706][105692] Updated weights for policy 0, policy_version 1918153 (0.0009) [2023-12-27 05:22:19,108][105620] Updated weights for policy 1, policy_version 1922739 (0.0009) [2023-12-27 05:22:19,165][105620] Updated weights for policy 1, policy_version 1922749 (0.0008) [2023-12-27 05:22:19,227][105620] Updated weights for policy 1, policy_version 1922759 (0.0009) [2023-12-27 05:22:19,441][105692] Updated weights for policy 0, policy_version 1918163 (0.0009) [2023-12-27 05:22:19,496][105692] Updated weights for policy 0, policy_version 1918173 (0.0008) [2023-12-27 05:22:19,558][105692] Updated weights for policy 0, policy_version 1918183 (0.0009) [2023-12-27 05:22:20,017][105620] Updated weights for policy 1, policy_version 1922769 (0.0009) [2023-12-27 05:22:20,075][105620] Updated weights for policy 1, policy_version 1922779 (0.0010) [2023-12-27 05:22:20,138][105620] Updated weights for policy 1, policy_version 1922789 (0.0009) [2023-12-27 05:22:20,199][105620] Updated weights for policy 1, policy_version 1922799 (0.0009) [2023-12-27 05:22:20,244][105692] Updated weights for policy 0, policy_version 1918193 (0.0009) [2023-12-27 05:22:20,296][105692] Updated weights for policy 0, policy_version 1918203 (0.0009) [2023-12-27 05:22:20,347][105692] Updated weights for policy 0, policy_version 1918213 (0.0009) [2023-12-27 05:22:20,408][105692] Updated weights for policy 0, policy_version 1918223 (0.0009) [2023-12-27 05:22:21,054][105620] Updated weights for policy 1, policy_version 1922809 (0.0009) [2023-12-27 05:22:21,061][105692] Updated weights for policy 0, policy_version 1918233 (0.0007) [2023-12-27 05:22:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 983441408. Throughput: 0: 9711.4, 1: 9939.4. Samples: 983436812. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:22:21,063][104569] Avg episode reward: [(0, '8898.718'), (1, '9073.518')] [2023-12-27 05:22:21,121][105692] Updated weights for policy 0, policy_version 1918243 (0.0007) [2023-12-27 05:22:21,122][105620] Updated weights for policy 1, policy_version 1922819 (0.0007) [2023-12-27 05:22:21,185][105692] Updated weights for policy 0, policy_version 1918253 (0.0007) [2023-12-27 05:22:21,187][105620] Updated weights for policy 1, policy_version 1922829 (0.0009) [2023-12-27 05:22:21,885][105692] Updated weights for policy 0, policy_version 1918263 (0.0008) [2023-12-27 05:22:21,950][105692] Updated weights for policy 0, policy_version 1918273 (0.0009) [2023-12-27 05:22:21,995][105620] Updated weights for policy 1, policy_version 1922839 (0.0006) [2023-12-27 05:22:22,013][105692] Updated weights for policy 0, policy_version 1918283 (0.0008) [2023-12-27 05:22:22,049][105620] Updated weights for policy 1, policy_version 1922849 (0.0007) [2023-12-27 05:22:22,115][105620] Updated weights for policy 1, policy_version 1922859 (0.0009) [2023-12-27 05:22:22,793][105692] Updated weights for policy 0, policy_version 1918293 (0.0007) [2023-12-27 05:22:22,811][105620] Updated weights for policy 1, policy_version 1922869 (0.0008) [2023-12-27 05:22:22,853][105692] Updated weights for policy 0, policy_version 1918303 (0.0007) [2023-12-27 05:22:22,863][105620] Updated weights for policy 1, policy_version 1922879 (0.0008) [2023-12-27 05:22:22,910][105692] Updated weights for policy 0, policy_version 1918313 (0.0006) [2023-12-27 05:22:22,924][105620] Updated weights for policy 1, policy_version 1922889 (0.0008) [2023-12-27 05:22:23,604][105692] Updated weights for policy 0, policy_version 1918323 (0.0007) [2023-12-27 05:22:23,641][105620] Updated weights for policy 1, policy_version 1922899 (0.0008) [2023-12-27 05:22:23,663][105692] Updated weights for policy 0, policy_version 1918333 (0.0007) [2023-12-27 05:22:23,697][105620] Updated weights for policy 1, policy_version 1922909 (0.0005) [2023-12-27 05:22:23,720][105692] Updated weights for policy 0, policy_version 1918343 (0.0006) [2023-12-27 05:22:23,752][105620] Updated weights for policy 1, policy_version 1922919 (0.0005) [2023-12-27 05:22:24,277][105620] Updated weights for policy 1, policy_version 1922929 (0.0005) [2023-12-27 05:22:24,330][105620] Updated weights for policy 1, policy_version 1922939 (0.0010) [2023-12-27 05:22:24,378][105620] Updated weights for policy 1, policy_version 1922949 (0.0010) [2023-12-27 05:22:24,423][105620] Updated weights for policy 1, policy_version 1922959 (0.0010) [2023-12-27 05:22:24,524][105692] Updated weights for policy 0, policy_version 1918353 (0.0006) [2023-12-27 05:22:24,583][105692] Updated weights for policy 0, policy_version 1918363 (0.0009) [2023-12-27 05:22:24,637][105692] Updated weights for policy 0, policy_version 1918373 (0.0009) [2023-12-27 05:22:24,694][105692] Updated weights for policy 0, policy_version 1918383 (0.0010) [2023-12-27 05:22:25,042][105620] Updated weights for policy 1, policy_version 1922969 (0.0011) [2023-12-27 05:22:25,093][105620] Updated weights for policy 1, policy_version 1922979 (0.0010) [2023-12-27 05:22:25,143][105620] Updated weights for policy 1, policy_version 1922989 (0.0005) [2023-12-27 05:22:25,584][105692] Updated weights for policy 0, policy_version 1918393 (0.0008) [2023-12-27 05:22:25,629][105692] Updated weights for policy 0, policy_version 1918403 (0.0007) [2023-12-27 05:22:25,672][105692] Updated weights for policy 0, policy_version 1918413 (0.0007) [2023-12-27 05:22:25,712][105620] Updated weights for policy 1, policy_version 1922999 (0.0007) [2023-12-27 05:22:25,768][105620] Updated weights for policy 1, policy_version 1923009 (0.0008) [2023-12-27 05:22:25,822][105620] Updated weights for policy 1, policy_version 1923019 (0.0009) [2023-12-27 05:22:26,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 983547904. Throughput: 0: 9623.3, 1: 10057.5. Samples: 983553712. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:22:26,062][104569] Avg episode reward: [(0, '8989.467'), (1, '8981.671')] [2023-12-27 05:22:26,427][105692] Updated weights for policy 0, policy_version 1918424 (0.0010) [2023-12-27 05:22:26,479][105692] Updated weights for policy 0, policy_version 1918435 (0.0010) [2023-12-27 05:22:26,492][105620] Updated weights for policy 1, policy_version 1923029 (0.0007) [2023-12-27 05:22:26,523][105692] Updated weights for policy 0, policy_version 1918445 (0.0007) [2023-12-27 05:22:26,537][105620] Updated weights for policy 1, policy_version 1923039 (0.0008) [2023-12-27 05:22:26,579][105620] Updated weights for policy 1, policy_version 1923049 (0.0006) [2023-12-27 05:22:27,237][105620] Updated weights for policy 1, policy_version 1923059 (0.0008) [2023-12-27 05:22:27,296][105620] Updated weights for policy 1, policy_version 1923069 (0.0009) [2023-12-27 05:22:27,349][105620] Updated weights for policy 1, policy_version 1923079 (0.0008) [2023-12-27 05:22:27,372][105692] Updated weights for policy 0, policy_version 1918455 (0.0008) [2023-12-27 05:22:27,417][105692] Updated weights for policy 0, policy_version 1918465 (0.0007) [2023-12-27 05:22:27,469][105692] Updated weights for policy 0, policy_version 1918476 (0.0010) [2023-12-27 05:22:28,017][105620] Updated weights for policy 1, policy_version 1923089 (0.0008) [2023-12-27 05:22:28,073][105620] Updated weights for policy 1, policy_version 1923099 (0.0009) [2023-12-27 05:22:28,124][105620] Updated weights for policy 1, policy_version 1923109 (0.0009) [2023-12-27 05:22:28,185][105620] Updated weights for policy 1, policy_version 1923119 (0.0009) [2023-12-27 05:22:28,296][105692] Updated weights for policy 0, policy_version 1918486 (0.0009) [2023-12-27 05:22:28,359][105692] Updated weights for policy 0, policy_version 1918496 (0.0009) [2023-12-27 05:22:28,417][105692] Updated weights for policy 0, policy_version 1918506 (0.0010) [2023-12-27 05:22:28,999][105620] Updated weights for policy 1, policy_version 1923129 (0.0008) [2023-12-27 05:22:29,059][105620] Updated weights for policy 1, policy_version 1923139 (0.0008) [2023-12-27 05:22:29,069][105692] Updated weights for policy 0, policy_version 1918516 (0.0008) [2023-12-27 05:22:29,112][105620] Updated weights for policy 1, policy_version 1923149 (0.0006) [2023-12-27 05:22:29,126][105692] Updated weights for policy 0, policy_version 1918526 (0.0007) [2023-12-27 05:22:29,173][105692] Updated weights for policy 0, policy_version 1918536 (0.0009) [2023-12-27 05:22:29,877][105620] Updated weights for policy 1, policy_version 1923159 (0.0007) [2023-12-27 05:22:29,891][105692] Updated weights for policy 0, policy_version 1918546 (0.0010) [2023-12-27 05:22:29,934][105620] Updated weights for policy 1, policy_version 1923169 (0.0008) [2023-12-27 05:22:29,960][105692] Updated weights for policy 0, policy_version 1918556 (0.0011) [2023-12-27 05:22:29,997][105620] Updated weights for policy 1, policy_version 1923179 (0.0010) [2023-12-27 05:22:30,017][105692] Updated weights for policy 0, policy_version 1918566 (0.0011) [2023-12-27 05:22:30,072][105692] Updated weights for policy 0, policy_version 1918576 (0.0010) [2023-12-27 05:22:30,670][105620] Updated weights for policy 1, policy_version 1923189 (0.0008) [2023-12-27 05:22:30,690][105692] Updated weights for policy 0, policy_version 1918586 (0.0005) [2023-12-27 05:22:30,723][105620] Updated weights for policy 1, policy_version 1923199 (0.0005) [2023-12-27 05:22:30,737][105692] Updated weights for policy 0, policy_version 1918596 (0.0005) [2023-12-27 05:22:30,770][105620] Updated weights for policy 1, policy_version 1923209 (0.0009) [2023-12-27 05:22:30,782][105692] Updated weights for policy 0, policy_version 1918606 (0.0005) [2023-12-27 05:22:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 983646208. Throughput: 0: 9590.2, 1: 10118.4. Samples: 983611488. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:22:31,063][104569] Avg episode reward: [(0, '8625.743'), (1, '9070.505')] [2023-12-27 05:22:31,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001918608_491233280.pth... [2023-12-27 05:22:31,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001923216_492412928.pth... [2023-12-27 05:22:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001917520_490954752.pth [2023-12-27 05:22:31,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001922032_492109824.pth [2023-12-27 05:22:31,362][105692] Updated weights for policy 0, policy_version 1918616 (0.0009) [2023-12-27 05:22:31,416][105692] Updated weights for policy 0, policy_version 1918626 (0.0006) [2023-12-27 05:22:31,444][105620] Updated weights for policy 1, policy_version 1923219 (0.0010) [2023-12-27 05:22:31,467][105692] Updated weights for policy 0, policy_version 1918636 (0.0006) [2023-12-27 05:22:31,493][105620] Updated weights for policy 1, policy_version 1923229 (0.0011) [2023-12-27 05:22:31,559][105620] Updated weights for policy 1, policy_version 1923239 (0.0011) [2023-12-27 05:22:32,184][105692] Updated weights for policy 0, policy_version 1918646 (0.0007) [2023-12-27 05:22:32,253][105692] Updated weights for policy 0, policy_version 1918656 (0.0008) [2023-12-27 05:22:32,254][105620] Updated weights for policy 1, policy_version 1923249 (0.0010) [2023-12-27 05:22:32,316][105692] Updated weights for policy 0, policy_version 1918666 (0.0009) [2023-12-27 05:22:32,321][105620] Updated weights for policy 1, policy_version 1923259 (0.0008) [2023-12-27 05:22:32,384][105620] Updated weights for policy 1, policy_version 1923269 (0.0012) [2023-12-27 05:22:32,433][105620] Updated weights for policy 1, policy_version 1923279 (0.0011) [2023-12-27 05:22:33,065][105620] Updated weights for policy 1, policy_version 1923289 (0.0006) [2023-12-27 05:22:33,084][105692] Updated weights for policy 0, policy_version 1918676 (0.0007) [2023-12-27 05:22:33,116][105620] Updated weights for policy 1, policy_version 1923299 (0.0005) [2023-12-27 05:22:33,136][105692] Updated weights for policy 0, policy_version 1918686 (0.0010) [2023-12-27 05:22:33,164][105620] Updated weights for policy 1, policy_version 1923309 (0.0005) [2023-12-27 05:22:33,188][105692] Updated weights for policy 0, policy_version 1918697 (0.0009) [2023-12-27 05:22:33,684][105620] Updated weights for policy 1, policy_version 1923319 (0.0005) [2023-12-27 05:22:33,737][105620] Updated weights for policy 1, policy_version 1923329 (0.0005) [2023-12-27 05:22:33,779][105620] Updated weights for policy 1, policy_version 1923339 (0.0005) [2023-12-27 05:22:34,078][105692] Updated weights for policy 0, policy_version 1918708 (0.0010) [2023-12-27 05:22:34,133][105692] Updated weights for policy 0, policy_version 1918719 (0.0010) [2023-12-27 05:22:34,194][105692] Updated weights for policy 0, policy_version 1918729 (0.0008) [2023-12-27 05:22:34,334][105620] Updated weights for policy 1, policy_version 1923349 (0.0007) [2023-12-27 05:22:34,398][105620] Updated weights for policy 1, policy_version 1923359 (0.0011) [2023-12-27 05:22:34,456][105620] Updated weights for policy 1, policy_version 1923369 (0.0011) [2023-12-27 05:22:34,980][105692] Updated weights for policy 0, policy_version 1918739 (0.0010) [2023-12-27 05:22:35,044][105692] Updated weights for policy 0, policy_version 1918749 (0.0010) [2023-12-27 05:22:35,105][105692] Updated weights for policy 0, policy_version 1918759 (0.0008) [2023-12-27 05:22:35,150][105620] Updated weights for policy 1, policy_version 1923379 (0.0010) [2023-12-27 05:22:35,200][105620] Updated weights for policy 1, policy_version 1923389 (0.0009) [2023-12-27 05:22:35,248][105620] Updated weights for policy 1, policy_version 1923399 (0.0009) [2023-12-27 05:22:35,875][105692] Updated weights for policy 0, policy_version 1918769 (0.0009) [2023-12-27 05:22:35,920][105692] Updated weights for policy 0, policy_version 1918779 (0.0008) [2023-12-27 05:22:35,930][105620] Updated weights for policy 1, policy_version 1923409 (0.0009) [2023-12-27 05:22:35,972][105692] Updated weights for policy 0, policy_version 1918789 (0.0006) [2023-12-27 05:22:35,985][105620] Updated weights for policy 1, policy_version 1923419 (0.0010) [2023-12-27 05:22:36,026][105692] Updated weights for policy 0, policy_version 1918799 (0.0007) [2023-12-27 05:22:36,045][105620] Updated weights for policy 1, policy_version 1923429 (0.0008) [2023-12-27 05:22:36,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19660.9, 300 sec: 19549.7). Total num frames: 983744512. Throughput: 0: 9583.0, 1: 10146.6. Samples: 983733568. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:22:36,062][104569] Avg episode reward: [(0, '8087.989'), (1, '9254.717')] [2023-12-27 05:22:36,094][105620] Updated weights for policy 1, policy_version 1923439 (0.0009) [2023-12-27 05:22:36,726][105692] Updated weights for policy 0, policy_version 1918809 (0.0007) [2023-12-27 05:22:36,781][105620] Updated weights for policy 1, policy_version 1923449 (0.0011) [2023-12-27 05:22:36,789][105692] Updated weights for policy 0, policy_version 1918819 (0.0005) [2023-12-27 05:22:36,840][105620] Updated weights for policy 1, policy_version 1923459 (0.0010) [2023-12-27 05:22:36,842][105692] Updated weights for policy 0, policy_version 1918829 (0.0006) [2023-12-27 05:22:36,900][105620] Updated weights for policy 1, policy_version 1923469 (0.0009) [2023-12-27 05:22:37,579][105692] Updated weights for policy 0, policy_version 1918839 (0.0007) [2023-12-27 05:22:37,632][105692] Updated weights for policy 0, policy_version 1918849 (0.0008) [2023-12-27 05:22:37,673][105620] Updated weights for policy 1, policy_version 1923479 (0.0011) [2023-12-27 05:22:37,692][105692] Updated weights for policy 0, policy_version 1918859 (0.0006) [2023-12-27 05:22:37,739][105620] Updated weights for policy 1, policy_version 1923489 (0.0011) [2023-12-27 05:22:37,804][105620] Updated weights for policy 1, policy_version 1923499 (0.0011) [2023-12-27 05:22:38,499][105692] Updated weights for policy 0, policy_version 1918869 (0.0006) [2023-12-27 05:22:38,557][105620] Updated weights for policy 1, policy_version 1923509 (0.0011) [2023-12-27 05:22:38,560][105692] Updated weights for policy 0, policy_version 1918879 (0.0006) [2023-12-27 05:22:38,617][105620] Updated weights for policy 1, policy_version 1923519 (0.0011) [2023-12-27 05:22:38,624][105692] Updated weights for policy 0, policy_version 1918889 (0.0007) [2023-12-27 05:22:38,677][105620] Updated weights for policy 1, policy_version 1923529 (0.0011) [2023-12-27 05:22:39,311][105692] Updated weights for policy 0, policy_version 1918899 (0.0008) [2023-12-27 05:22:39,378][105692] Updated weights for policy 0, policy_version 1918909 (0.0007) [2023-12-27 05:22:39,391][105620] Updated weights for policy 1, policy_version 1923539 (0.0010) [2023-12-27 05:22:39,444][105692] Updated weights for policy 0, policy_version 1918919 (0.0007) [2023-12-27 05:22:39,455][105620] Updated weights for policy 1, policy_version 1923549 (0.0007) [2023-12-27 05:22:39,519][105620] Updated weights for policy 1, policy_version 1923559 (0.0007) [2023-12-27 05:22:40,093][105692] Updated weights for policy 0, policy_version 1918929 (0.0009) [2023-12-27 05:22:40,159][105692] Updated weights for policy 0, policy_version 1918939 (0.0009) [2023-12-27 05:22:40,220][105692] Updated weights for policy 0, policy_version 1918949 (0.0010) [2023-12-27 05:22:40,268][105692] Updated weights for policy 0, policy_version 1918959 (0.0008) [2023-12-27 05:22:40,334][105620] Updated weights for policy 1, policy_version 1923569 (0.0009) [2023-12-27 05:22:40,393][105620] Updated weights for policy 1, policy_version 1923579 (0.0009) [2023-12-27 05:22:40,451][105620] Updated weights for policy 1, policy_version 1923589 (0.0010) [2023-12-27 05:22:40,511][105620] Updated weights for policy 1, policy_version 1923599 (0.0009) [2023-12-27 05:22:41,019][105692] Updated weights for policy 0, policy_version 1918969 (0.0009) [2023-12-27 05:22:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 983834624. Throughput: 0: 9501.0, 1: 10045.9. Samples: 983847828. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:22:41,063][104569] Avg episode reward: [(0, '8179.047'), (1, '9346.037')] [2023-12-27 05:22:41,079][105692] Updated weights for policy 0, policy_version 1918979 (0.0008) [2023-12-27 05:22:41,138][105692] Updated weights for policy 0, policy_version 1918989 (0.0008) [2023-12-27 05:22:41,295][105620] Updated weights for policy 1, policy_version 1923609 (0.0007) [2023-12-27 05:22:41,361][105620] Updated weights for policy 1, policy_version 1923619 (0.0006) [2023-12-27 05:22:41,436][105620] Updated weights for policy 1, policy_version 1923629 (0.0008) [2023-12-27 05:22:41,964][105692] Updated weights for policy 0, policy_version 1918999 (0.0009) [2023-12-27 05:22:42,024][105692] Updated weights for policy 0, policy_version 1919009 (0.0009) [2023-12-27 05:22:42,087][105692] Updated weights for policy 0, policy_version 1919019 (0.0009) [2023-12-27 05:22:42,146][105620] Updated weights for policy 1, policy_version 1923639 (0.0007) [2023-12-27 05:22:42,205][105620] Updated weights for policy 1, policy_version 1923649 (0.0009) [2023-12-27 05:22:42,258][105620] Updated weights for policy 1, policy_version 1923659 (0.0008) [2023-12-27 05:22:42,805][105692] Updated weights for policy 0, policy_version 1919029 (0.0008) [2023-12-27 05:22:42,864][105692] Updated weights for policy 0, policy_version 1919039 (0.0008) [2023-12-27 05:22:42,916][105692] Updated weights for policy 0, policy_version 1919049 (0.0008) [2023-12-27 05:22:43,050][105620] Updated weights for policy 1, policy_version 1923669 (0.0010) [2023-12-27 05:22:43,101][105620] Updated weights for policy 1, policy_version 1923679 (0.0009) [2023-12-27 05:22:43,160][105620] Updated weights for policy 1, policy_version 1923689 (0.0009) [2023-12-27 05:22:43,658][105692] Updated weights for policy 0, policy_version 1919059 (0.0008) [2023-12-27 05:22:43,705][105692] Updated weights for policy 0, policy_version 1919069 (0.0009) [2023-12-27 05:22:43,759][105692] Updated weights for policy 0, policy_version 1919080 (0.0009) [2023-12-27 05:22:43,897][105620] Updated weights for policy 1, policy_version 1923699 (0.0006) [2023-12-27 05:22:43,943][105620] Updated weights for policy 1, policy_version 1923709 (0.0005) [2023-12-27 05:22:43,991][105620] Updated weights for policy 1, policy_version 1923719 (0.0005) [2023-12-27 05:22:44,547][105692] Updated weights for policy 0, policy_version 1919090 (0.0009) [2023-12-27 05:22:44,608][105692] Updated weights for policy 0, policy_version 1919100 (0.0008) [2023-12-27 05:22:44,670][105692] Updated weights for policy 0, policy_version 1919110 (0.0008) [2023-12-27 05:22:44,712][105620] Updated weights for policy 1, policy_version 1923729 (0.0007) [2023-12-27 05:22:44,722][105692] Updated weights for policy 0, policy_version 1919120 (0.0008) [2023-12-27 05:22:44,768][105620] Updated weights for policy 1, policy_version 1923739 (0.0008) [2023-12-27 05:22:44,829][105620] Updated weights for policy 1, policy_version 1923749 (0.0008) [2023-12-27 05:22:44,895][105620] Updated weights for policy 1, policy_version 1923759 (0.0009) [2023-12-27 05:22:45,378][105692] Updated weights for policy 0, policy_version 1919130 (0.0009) [2023-12-27 05:22:45,434][105692] Updated weights for policy 0, policy_version 1919140 (0.0009) [2023-12-27 05:22:45,490][105692] Updated weights for policy 0, policy_version 1919150 (0.0009) [2023-12-27 05:22:45,747][105620] Updated weights for policy 1, policy_version 1923769 (0.0007) [2023-12-27 05:22:45,805][105620] Updated weights for policy 1, policy_version 1923779 (0.0009) [2023-12-27 05:22:45,858][105620] Updated weights for policy 1, policy_version 1923789 (0.0009) [2023-12-27 05:22:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19661.0, 300 sec: 19522.0). Total num frames: 983932928. Throughput: 0: 9387.8, 1: 10034.0. Samples: 983903108. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:22:46,062][104569] Avg episode reward: [(0, '8448.325'), (1, '9346.003')] [2023-12-27 05:22:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001919152_491372544.pth... [2023-12-27 05:22:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001923792_492560384.pth... [2023-12-27 05:22:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001918064_491094016.pth [2023-12-27 05:22:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001922640_492265472.pth [2023-12-27 05:22:46,194][105692] Updated weights for policy 0, policy_version 1919160 (0.0009) [2023-12-27 05:22:46,241][105692] Updated weights for policy 0, policy_version 1919170 (0.0008) [2023-12-27 05:22:46,289][105692] Updated weights for policy 0, policy_version 1919180 (0.0008) [2023-12-27 05:22:46,536][105620] Updated weights for policy 1, policy_version 1923799 (0.0008) [2023-12-27 05:22:46,585][105620] Updated weights for policy 1, policy_version 1923809 (0.0009) [2023-12-27 05:22:46,632][105620] Updated weights for policy 1, policy_version 1923819 (0.0009) [2023-12-27 05:22:47,020][105692] Updated weights for policy 0, policy_version 1919190 (0.0008) [2023-12-27 05:22:47,076][105692] Updated weights for policy 0, policy_version 1919200 (0.0007) [2023-12-27 05:22:47,125][105692] Updated weights for policy 0, policy_version 1919210 (0.0005) [2023-12-27 05:22:47,502][105620] Updated weights for policy 1, policy_version 1923829 (0.0009) [2023-12-27 05:22:47,556][105620] Updated weights for policy 1, policy_version 1923839 (0.0009) [2023-12-27 05:22:47,613][105620] Updated weights for policy 1, policy_version 1923849 (0.0009) [2023-12-27 05:22:47,713][105692] Updated weights for policy 0, policy_version 1919220 (0.0007) [2023-12-27 05:22:47,774][105692] Updated weights for policy 0, policy_version 1919230 (0.0007) [2023-12-27 05:22:47,828][105692] Updated weights for policy 0, policy_version 1919240 (0.0005) [2023-12-27 05:22:48,381][105620] Updated weights for policy 1, policy_version 1923859 (0.0009) [2023-12-27 05:22:48,441][105620] Updated weights for policy 1, policy_version 1923869 (0.0008) [2023-12-27 05:22:48,497][105620] Updated weights for policy 1, policy_version 1923879 (0.0008) [2023-12-27 05:22:48,549][105692] Updated weights for policy 0, policy_version 1919250 (0.0006) [2023-12-27 05:22:48,601][105692] Updated weights for policy 0, policy_version 1919260 (0.0005) [2023-12-27 05:22:48,646][105692] Updated weights for policy 0, policy_version 1919270 (0.0005) [2023-12-27 05:22:48,694][105692] Updated weights for policy 0, policy_version 1919280 (0.0006) [2023-12-27 05:22:49,280][105620] Updated weights for policy 1, policy_version 1923889 (0.0007) [2023-12-27 05:22:49,347][105620] Updated weights for policy 1, policy_version 1923899 (0.0007) [2023-12-27 05:22:49,406][105692] Updated weights for policy 0, policy_version 1919290 (0.0009) [2023-12-27 05:22:49,415][105620] Updated weights for policy 1, policy_version 1923909 (0.0006) [2023-12-27 05:22:49,471][105692] Updated weights for policy 0, policy_version 1919300 (0.0009) [2023-12-27 05:22:49,472][105620] Updated weights for policy 1, policy_version 1923919 (0.0005) [2023-12-27 05:22:49,528][105692] Updated weights for policy 0, policy_version 1919310 (0.0010) [2023-12-27 05:22:50,113][105620] Updated weights for policy 1, policy_version 1923929 (0.0006) [2023-12-27 05:22:50,171][105620] Updated weights for policy 1, policy_version 1923939 (0.0009) [2023-12-27 05:22:50,226][105620] Updated weights for policy 1, policy_version 1923949 (0.0009) [2023-12-27 05:22:50,289][105692] Updated weights for policy 0, policy_version 1919320 (0.0008) [2023-12-27 05:22:50,341][105692] Updated weights for policy 0, policy_version 1919330 (0.0005) [2023-12-27 05:22:50,385][105692] Updated weights for policy 0, policy_version 1919340 (0.0005) [2023-12-27 05:22:50,887][105620] Updated weights for policy 1, policy_version 1923959 (0.0008) [2023-12-27 05:22:50,945][105620] Updated weights for policy 1, policy_version 1923969 (0.0010) [2023-12-27 05:22:51,013][105620] Updated weights for policy 1, policy_version 1923979 (0.0010) [2023-12-27 05:22:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 984031232. Throughput: 0: 9500.9, 1: 9873.0. Samples: 984018368. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:22:51,063][104569] Avg episode reward: [(0, '8352.407'), (1, '9255.797')] [2023-12-27 05:22:51,200][105692] Updated weights for policy 0, policy_version 1919350 (0.0007) [2023-12-27 05:22:51,264][105692] Updated weights for policy 0, policy_version 1919360 (0.0009) [2023-12-27 05:22:51,321][105692] Updated weights for policy 0, policy_version 1919370 (0.0008) [2023-12-27 05:22:51,742][105620] Updated weights for policy 1, policy_version 1923989 (0.0008) [2023-12-27 05:22:51,805][105620] Updated weights for policy 1, policy_version 1923999 (0.0010) [2023-12-27 05:22:51,870][105620] Updated weights for policy 1, policy_version 1924009 (0.0009) [2023-12-27 05:22:52,115][105692] Updated weights for policy 0, policy_version 1919380 (0.0009) [2023-12-27 05:22:52,177][105692] Updated weights for policy 0, policy_version 1919390 (0.0009) [2023-12-27 05:22:52,236][105692] Updated weights for policy 0, policy_version 1919400 (0.0009) [2023-12-27 05:22:52,622][105620] Updated weights for policy 1, policy_version 1924019 (0.0008) [2023-12-27 05:22:52,674][105620] Updated weights for policy 1, policy_version 1924029 (0.0005) [2023-12-27 05:22:52,731][105620] Updated weights for policy 1, policy_version 1924039 (0.0005) [2023-12-27 05:22:53,100][105692] Updated weights for policy 0, policy_version 1919410 (0.0008) [2023-12-27 05:22:53,157][105692] Updated weights for policy 0, policy_version 1919420 (0.0009) [2023-12-27 05:22:53,209][105692] Updated weights for policy 0, policy_version 1919430 (0.0009) [2023-12-27 05:22:53,263][105692] Updated weights for policy 0, policy_version 1919440 (0.0006) [2023-12-27 05:22:53,277][105620] Updated weights for policy 1, policy_version 1924049 (0.0005) [2023-12-27 05:22:53,346][105620] Updated weights for policy 1, policy_version 1924059 (0.0005) [2023-12-27 05:22:53,417][105620] Updated weights for policy 1, policy_version 1924069 (0.0006) [2023-12-27 05:22:53,481][105620] Updated weights for policy 1, policy_version 1924079 (0.0005) [2023-12-27 05:22:54,005][105692] Updated weights for policy 0, policy_version 1919450 (0.0009) [2023-12-27 05:22:54,045][105620] Updated weights for policy 1, policy_version 1924089 (0.0007) [2023-12-27 05:22:54,061][105692] Updated weights for policy 0, policy_version 1919460 (0.0009) [2023-12-27 05:22:54,103][105620] Updated weights for policy 1, policy_version 1924099 (0.0010) [2023-12-27 05:22:54,115][105692] Updated weights for policy 0, policy_version 1919470 (0.0010) [2023-12-27 05:22:54,151][105620] Updated weights for policy 1, policy_version 1924109 (0.0007) [2023-12-27 05:22:54,743][105620] Updated weights for policy 1, policy_version 1924119 (0.0005) [2023-12-27 05:22:54,799][105620] Updated weights for policy 1, policy_version 1924129 (0.0006) [2023-12-27 05:22:54,850][105620] Updated weights for policy 1, policy_version 1924139 (0.0007) [2023-12-27 05:22:54,976][105692] Updated weights for policy 0, policy_version 1919480 (0.0008) [2023-12-27 05:22:55,033][105692] Updated weights for policy 0, policy_version 1919490 (0.0009) [2023-12-27 05:22:55,098][105692] Updated weights for policy 0, policy_version 1919500 (0.0008) [2023-12-27 05:22:55,475][105620] Updated weights for policy 1, policy_version 1924149 (0.0009) [2023-12-27 05:22:55,522][105620] Updated weights for policy 1, policy_version 1924159 (0.0008) [2023-12-27 05:22:55,572][105620] Updated weights for policy 1, policy_version 1924169 (0.0008) [2023-12-27 05:22:55,866][105692] Updated weights for policy 0, policy_version 1919510 (0.0008) [2023-12-27 05:22:55,912][105692] Updated weights for policy 0, policy_version 1919520 (0.0008) [2023-12-27 05:22:55,963][105692] Updated weights for policy 0, policy_version 1919530 (0.0008) [2023-12-27 05:22:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19549.7). Total num frames: 984129536. Throughput: 0: 9379.8, 1: 9976.7. Samples: 984135972. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:22:56,063][104569] Avg episode reward: [(0, '8441.055'), (1, '9255.678')] [2023-12-27 05:22:56,278][105620] Updated weights for policy 1, policy_version 1924179 (0.0008) [2023-12-27 05:22:56,326][105620] Updated weights for policy 1, policy_version 1924189 (0.0008) [2023-12-27 05:22:56,378][105620] Updated weights for policy 1, policy_version 1924199 (0.0008) [2023-12-27 05:22:56,722][105692] Updated weights for policy 0, policy_version 1919540 (0.0008) [2023-12-27 05:22:56,770][105692] Updated weights for policy 0, policy_version 1919550 (0.0005) [2023-12-27 05:22:56,814][105692] Updated weights for policy 0, policy_version 1919560 (0.0005) [2023-12-27 05:22:56,971][105620] Updated weights for policy 1, policy_version 1924209 (0.0009) [2023-12-27 05:22:57,029][105620] Updated weights for policy 1, policy_version 1924219 (0.0010) [2023-12-27 05:22:57,093][105620] Updated weights for policy 1, policy_version 1924229 (0.0010) [2023-12-27 05:22:57,145][105620] Updated weights for policy 1, policy_version 1924239 (0.0010) [2023-12-27 05:22:57,345][105692] Updated weights for policy 0, policy_version 1919570 (0.0005) [2023-12-27 05:22:57,402][105692] Updated weights for policy 0, policy_version 1919580 (0.0006) [2023-12-27 05:22:57,460][105692] Updated weights for policy 0, policy_version 1919590 (0.0007) [2023-12-27 05:22:57,521][105692] Updated weights for policy 0, policy_version 1919600 (0.0008) [2023-12-27 05:22:57,807][105620] Updated weights for policy 1, policy_version 1924249 (0.0006) [2023-12-27 05:22:57,862][105620] Updated weights for policy 1, policy_version 1924259 (0.0007) [2023-12-27 05:22:57,923][105620] Updated weights for policy 1, policy_version 1924269 (0.0010) [2023-12-27 05:22:58,260][105692] Updated weights for policy 0, policy_version 1919610 (0.0008) [2023-12-27 05:22:58,312][105692] Updated weights for policy 0, policy_version 1919620 (0.0008) [2023-12-27 05:22:58,378][105692] Updated weights for policy 0, policy_version 1919630 (0.0009) [2023-12-27 05:22:58,682][105620] Updated weights for policy 1, policy_version 1924279 (0.0011) [2023-12-27 05:22:58,752][105620] Updated weights for policy 1, policy_version 1924289 (0.0010) [2023-12-27 05:22:58,825][105620] Updated weights for policy 1, policy_version 1924299 (0.0009) [2023-12-27 05:22:59,232][105692] Updated weights for policy 0, policy_version 1919640 (0.0008) [2023-12-27 05:22:59,306][105692] Updated weights for policy 0, policy_version 1919650 (0.0008) [2023-12-27 05:22:59,377][105692] Updated weights for policy 0, policy_version 1919660 (0.0008) [2023-12-27 05:22:59,643][105620] Updated weights for policy 1, policy_version 1924309 (0.0009) [2023-12-27 05:22:59,702][105620] Updated weights for policy 1, policy_version 1924319 (0.0010) [2023-12-27 05:22:59,759][105620] Updated weights for policy 1, policy_version 1924329 (0.0011) [2023-12-27 05:23:00,105][105692] Updated weights for policy 0, policy_version 1919670 (0.0007) [2023-12-27 05:23:00,154][105692] Updated weights for policy 0, policy_version 1919680 (0.0008) [2023-12-27 05:23:00,209][105692] Updated weights for policy 0, policy_version 1919690 (0.0008) [2023-12-27 05:23:00,545][105620] Updated weights for policy 1, policy_version 1924339 (0.0011) [2023-12-27 05:23:00,603][105620] Updated weights for policy 1, policy_version 1924349 (0.0010) [2023-12-27 05:23:00,664][105620] Updated weights for policy 1, policy_version 1924359 (0.0010) [2023-12-27 05:23:00,947][105692] Updated weights for policy 0, policy_version 1919700 (0.0008) [2023-12-27 05:23:00,997][105692] Updated weights for policy 0, policy_version 1919710 (0.0007) [2023-12-27 05:23:01,051][105692] Updated weights for policy 0, policy_version 1919720 (0.0008) [2023-12-27 05:23:01,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 984219648. Throughput: 0: 9469.3, 1: 9965.1. Samples: 984195856. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:23:01,062][104569] Avg episode reward: [(0, '8173.572'), (1, '9253.346')] [2023-12-27 05:23:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001924368_492707840.pth... [2023-12-27 05:23:01,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001923216_492412928.pth [2023-12-27 05:23:01,091][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001919728_491520000.pth... [2023-12-27 05:23:01,095][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001918608_491233280.pth [2023-12-27 05:23:01,400][105620] Updated weights for policy 1, policy_version 1924369 (0.0010) [2023-12-27 05:23:01,461][105620] Updated weights for policy 1, policy_version 1924379 (0.0009) [2023-12-27 05:23:01,514][105620] Updated weights for policy 1, policy_version 1924389 (0.0006) [2023-12-27 05:23:01,575][105620] Updated weights for policy 1, policy_version 1924399 (0.0007) [2023-12-27 05:23:01,816][105692] Updated weights for policy 0, policy_version 1919730 (0.0009) [2023-12-27 05:23:01,862][105692] Updated weights for policy 0, policy_version 1919740 (0.0008) [2023-12-27 05:23:01,916][105692] Updated weights for policy 0, policy_version 1919750 (0.0009) [2023-12-27 05:23:01,969][105692] Updated weights for policy 0, policy_version 1919760 (0.0008) [2023-12-27 05:23:02,266][105620] Updated weights for policy 1, policy_version 1924409 (0.0007) [2023-12-27 05:23:02,322][105620] Updated weights for policy 1, policy_version 1924419 (0.0005) [2023-12-27 05:23:02,389][105620] Updated weights for policy 1, policy_version 1924429 (0.0008) [2023-12-27 05:23:02,767][105692] Updated weights for policy 0, policy_version 1919770 (0.0009) [2023-12-27 05:23:02,825][105692] Updated weights for policy 0, policy_version 1919780 (0.0009) [2023-12-27 05:23:02,877][105692] Updated weights for policy 0, policy_version 1919790 (0.0008) [2023-12-27 05:23:03,063][105620] Updated weights for policy 1, policy_version 1924439 (0.0007) [2023-12-27 05:23:03,115][105620] Updated weights for policy 1, policy_version 1924449 (0.0005) [2023-12-27 05:23:03,168][105620] Updated weights for policy 1, policy_version 1924459 (0.0005) [2023-12-27 05:23:03,611][105692] Updated weights for policy 0, policy_version 1919800 (0.0006) [2023-12-27 05:23:03,655][105692] Updated weights for policy 0, policy_version 1919810 (0.0008) [2023-12-27 05:23:03,702][105692] Updated weights for policy 0, policy_version 1919820 (0.0005) [2023-12-27 05:23:03,901][105620] Updated weights for policy 1, policy_version 1924469 (0.0006) [2023-12-27 05:23:03,963][105620] Updated weights for policy 1, policy_version 1924479 (0.0009) [2023-12-27 05:23:04,032][105620] Updated weights for policy 1, policy_version 1924489 (0.0011) [2023-12-27 05:23:04,335][105692] Updated weights for policy 0, policy_version 1919830 (0.0006) [2023-12-27 05:23:04,398][105692] Updated weights for policy 0, policy_version 1919840 (0.0005) [2023-12-27 05:23:04,467][105692] Updated weights for policy 0, policy_version 1919850 (0.0009) [2023-12-27 05:23:04,716][105620] Updated weights for policy 1, policy_version 1924499 (0.0009) [2023-12-27 05:23:04,775][105620] Updated weights for policy 1, policy_version 1924509 (0.0006) [2023-12-27 05:23:04,833][105620] Updated weights for policy 1, policy_version 1924519 (0.0010) [2023-12-27 05:23:05,128][105692] Updated weights for policy 0, policy_version 1919860 (0.0011) [2023-12-27 05:23:05,194][105692] Updated weights for policy 0, policy_version 1919870 (0.0007) [2023-12-27 05:23:05,257][105692] Updated weights for policy 0, policy_version 1919880 (0.0006) [2023-12-27 05:23:05,430][105620] Updated weights for policy 1, policy_version 1924529 (0.0010) [2023-12-27 05:23:05,492][105620] Updated weights for policy 1, policy_version 1924539 (0.0006) [2023-12-27 05:23:05,554][105620] Updated weights for policy 1, policy_version 1924549 (0.0005) [2023-12-27 05:23:05,603][105620] Updated weights for policy 1, policy_version 1924559 (0.0005) [2023-12-27 05:23:05,817][105692] Updated weights for policy 0, policy_version 1919890 (0.0007) [2023-12-27 05:23:05,876][105692] Updated weights for policy 0, policy_version 1919900 (0.0008) [2023-12-27 05:23:05,938][105692] Updated weights for policy 0, policy_version 1919910 (0.0010) [2023-12-27 05:23:05,999][105692] Updated weights for policy 0, policy_version 1919920 (0.0005) [2023-12-27 05:23:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19549.7). Total num frames: 984326144. Throughput: 0: 9507.4, 1: 9907.7. Samples: 984310492. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:23:06,063][104569] Avg episode reward: [(0, '7992.919'), (1, '9161.202')] [2023-12-27 05:23:06,139][105620] Updated weights for policy 1, policy_version 1924569 (0.0009) [2023-12-27 05:23:06,209][105620] Updated weights for policy 1, policy_version 1924579 (0.0011) [2023-12-27 05:23:06,274][105620] Updated weights for policy 1, policy_version 1924589 (0.0011) [2023-12-27 05:23:06,713][105692] Updated weights for policy 0, policy_version 1919930 (0.0010) [2023-12-27 05:23:06,771][105692] Updated weights for policy 0, policy_version 1919940 (0.0010) [2023-12-27 05:23:06,830][105692] Updated weights for policy 0, policy_version 1919950 (0.0009) [2023-12-27 05:23:06,873][105620] Updated weights for policy 1, policy_version 1924599 (0.0009) [2023-12-27 05:23:06,924][105620] Updated weights for policy 1, policy_version 1924609 (0.0008) [2023-12-27 05:23:06,973][105620] Updated weights for policy 1, policy_version 1924619 (0.0005) [2023-12-27 05:23:07,589][105692] Updated weights for policy 0, policy_version 1919960 (0.0009) [2023-12-27 05:23:07,639][105692] Updated weights for policy 0, policy_version 1919970 (0.0009) [2023-12-27 05:23:07,690][105620] Updated weights for policy 1, policy_version 1924629 (0.0007) [2023-12-27 05:23:07,695][105692] Updated weights for policy 0, policy_version 1919980 (0.0010) [2023-12-27 05:23:07,753][105620] Updated weights for policy 1, policy_version 1924639 (0.0008) [2023-12-27 05:23:07,816][105620] Updated weights for policy 1, policy_version 1924649 (0.0009) [2023-12-27 05:23:08,455][105692] Updated weights for policy 0, policy_version 1919990 (0.0009) [2023-12-27 05:23:08,514][105692] Updated weights for policy 0, policy_version 1920000 (0.0009) [2023-12-27 05:23:08,561][105620] Updated weights for policy 1, policy_version 1924659 (0.0009) [2023-12-27 05:23:08,579][105692] Updated weights for policy 0, policy_version 1920010 (0.0007) [2023-12-27 05:23:08,626][105620] Updated weights for policy 1, policy_version 1924669 (0.0007) [2023-12-27 05:23:08,685][105620] Updated weights for policy 1, policy_version 1924679 (0.0009) [2023-12-27 05:23:09,335][105692] Updated weights for policy 0, policy_version 1920020 (0.0007) [2023-12-27 05:23:09,400][105692] Updated weights for policy 0, policy_version 1920030 (0.0009) [2023-12-27 05:23:09,444][105620] Updated weights for policy 1, policy_version 1924689 (0.0008) [2023-12-27 05:23:09,466][105692] Updated weights for policy 0, policy_version 1920040 (0.0010) [2023-12-27 05:23:09,510][105620] Updated weights for policy 1, policy_version 1924699 (0.0008) [2023-12-27 05:23:09,576][105620] Updated weights for policy 1, policy_version 1924709 (0.0009) [2023-12-27 05:23:09,649][105620] Updated weights for policy 1, policy_version 1924719 (0.0009) [2023-12-27 05:23:10,207][105692] Updated weights for policy 0, policy_version 1920050 (0.0007) [2023-12-27 05:23:10,270][105692] Updated weights for policy 0, policy_version 1920060 (0.0009) [2023-12-27 05:23:10,329][105692] Updated weights for policy 0, policy_version 1920070 (0.0009) [2023-12-27 05:23:10,387][105692] Updated weights for policy 0, policy_version 1920080 (0.0008) [2023-12-27 05:23:10,448][105620] Updated weights for policy 1, policy_version 1924729 (0.0011) [2023-12-27 05:23:10,501][105620] Updated weights for policy 1, policy_version 1924739 (0.0011) [2023-12-27 05:23:10,563][105620] Updated weights for policy 1, policy_version 1924749 (0.0011) [2023-12-27 05:23:11,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19521.9). Total num frames: 984416256. Throughput: 0: 9543.5, 1: 9903.4. Samples: 984428820. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:23:11,063][104569] Avg episode reward: [(0, '8444.787'), (1, '9253.718')] [2023-12-27 05:23:11,204][105692] Updated weights for policy 0, policy_version 1920090 (0.0010) [2023-12-27 05:23:11,265][105620] Updated weights for policy 1, policy_version 1924759 (0.0008) [2023-12-27 05:23:11,279][105692] Updated weights for policy 0, policy_version 1920100 (0.0008) [2023-12-27 05:23:11,331][105620] Updated weights for policy 1, policy_version 1924769 (0.0008) [2023-12-27 05:23:11,338][105692] Updated weights for policy 0, policy_version 1920110 (0.0007) [2023-12-27 05:23:11,405][105620] Updated weights for policy 1, policy_version 1924779 (0.0011) [2023-12-27 05:23:12,127][105620] Updated weights for policy 1, policy_version 1924789 (0.0008) [2023-12-27 05:23:12,151][105692] Updated weights for policy 0, policy_version 1920120 (0.0008) [2023-12-27 05:23:12,187][105620] Updated weights for policy 1, policy_version 1924799 (0.0010) [2023-12-27 05:23:12,205][105692] Updated weights for policy 0, policy_version 1920130 (0.0008) [2023-12-27 05:23:12,249][105620] Updated weights for policy 1, policy_version 1924809 (0.0011) [2023-12-27 05:23:12,270][105692] Updated weights for policy 0, policy_version 1920140 (0.0007) [2023-12-27 05:23:13,016][105620] Updated weights for policy 1, policy_version 1924819 (0.0010) [2023-12-27 05:23:13,053][105692] Updated weights for policy 0, policy_version 1920150 (0.0007) [2023-12-27 05:23:13,072][105620] Updated weights for policy 1, policy_version 1924829 (0.0007) [2023-12-27 05:23:13,103][105692] Updated weights for policy 0, policy_version 1920160 (0.0007) [2023-12-27 05:23:13,132][105620] Updated weights for policy 1, policy_version 1924839 (0.0009) [2023-12-27 05:23:13,162][105692] Updated weights for policy 0, policy_version 1920170 (0.0007) [2023-12-27 05:23:13,836][105620] Updated weights for policy 1, policy_version 1924849 (0.0008) [2023-12-27 05:23:13,863][105692] Updated weights for policy 0, policy_version 1920180 (0.0007) [2023-12-27 05:23:13,890][105620] Updated weights for policy 1, policy_version 1924859 (0.0005) [2023-12-27 05:23:13,925][105692] Updated weights for policy 0, policy_version 1920190 (0.0008) [2023-12-27 05:23:13,944][105620] Updated weights for policy 1, policy_version 1924869 (0.0007) [2023-12-27 05:23:13,978][105692] Updated weights for policy 0, policy_version 1920200 (0.0007) [2023-12-27 05:23:13,997][105620] Updated weights for policy 1, policy_version 1924879 (0.0006) [2023-12-27 05:23:14,583][105620] Updated weights for policy 1, policy_version 1924889 (0.0005) [2023-12-27 05:23:14,632][105620] Updated weights for policy 1, policy_version 1924899 (0.0005) [2023-12-27 05:23:14,690][105620] Updated weights for policy 1, policy_version 1924909 (0.0005) [2023-12-27 05:23:14,839][105692] Updated weights for policy 0, policy_version 1920210 (0.0007) [2023-12-27 05:23:14,906][105692] Updated weights for policy 0, policy_version 1920220 (0.0009) [2023-12-27 05:23:14,965][105692] Updated weights for policy 0, policy_version 1920230 (0.0009) [2023-12-27 05:23:15,028][105692] Updated weights for policy 0, policy_version 1920240 (0.0008) [2023-12-27 05:23:15,447][105620] Updated weights for policy 1, policy_version 1924919 (0.0009) [2023-12-27 05:23:15,513][105620] Updated weights for policy 1, policy_version 1924929 (0.0007) [2023-12-27 05:23:15,576][105620] Updated weights for policy 1, policy_version 1924939 (0.0009) [2023-12-27 05:23:15,596][105692] Updated weights for policy 0, policy_version 1920250 (0.0005) [2023-12-27 05:23:15,658][105692] Updated weights for policy 0, policy_version 1920260 (0.0005) [2023-12-27 05:23:15,714][105692] Updated weights for policy 0, policy_version 1920270 (0.0006) [2023-12-27 05:23:16,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.2, 300 sec: 19521.9). Total num frames: 984514560. Throughput: 0: 9533.7, 1: 9859.8. Samples: 984484196. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:23:16,063][104569] Avg episode reward: [(0, '8808.417'), (1, '9254.033')] [2023-12-27 05:23:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001920272_491659264.pth... [2023-12-27 05:23:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001924944_492855296.pth... [2023-12-27 05:23:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001919152_491372544.pth [2023-12-27 05:23:16,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001923792_492560384.pth [2023-12-27 05:23:16,232][105620] Updated weights for policy 1, policy_version 1924949 (0.0006) [2023-12-27 05:23:16,288][105620] Updated weights for policy 1, policy_version 1924959 (0.0005) [2023-12-27 05:23:16,339][105620] Updated weights for policy 1, policy_version 1924969 (0.0005) [2023-12-27 05:23:16,448][105692] Updated weights for policy 0, policy_version 1920280 (0.0010) [2023-12-27 05:23:16,496][105692] Updated weights for policy 0, policy_version 1920290 (0.0010) [2023-12-27 05:23:16,540][105692] Updated weights for policy 0, policy_version 1920300 (0.0010) [2023-12-27 05:23:16,998][105620] Updated weights for policy 1, policy_version 1924979 (0.0007) [2023-12-27 05:23:17,054][105620] Updated weights for policy 1, policy_version 1924989 (0.0010) [2023-12-27 05:23:17,118][105620] Updated weights for policy 1, policy_version 1924999 (0.0010) [2023-12-27 05:23:17,244][105692] Updated weights for policy 0, policy_version 1920311 (0.0007) [2023-12-27 05:23:17,300][105692] Updated weights for policy 0, policy_version 1920321 (0.0010) [2023-12-27 05:23:17,347][105692] Updated weights for policy 0, policy_version 1920331 (0.0009) [2023-12-27 05:23:17,774][105620] Updated weights for policy 1, policy_version 1925009 (0.0010) [2023-12-27 05:23:17,833][105620] Updated weights for policy 1, policy_version 1925019 (0.0009) [2023-12-27 05:23:17,893][105620] Updated weights for policy 1, policy_version 1925029 (0.0007) [2023-12-27 05:23:17,945][105620] Updated weights for policy 1, policy_version 1925039 (0.0008) [2023-12-27 05:23:18,070][105692] Updated weights for policy 0, policy_version 1920341 (0.0007) [2023-12-27 05:23:18,116][105692] Updated weights for policy 0, policy_version 1920351 (0.0005) [2023-12-27 05:23:18,179][105692] Updated weights for policy 0, policy_version 1920361 (0.0008) [2023-12-27 05:23:18,690][105620] Updated weights for policy 1, policy_version 1925049 (0.0010) [2023-12-27 05:23:18,753][105620] Updated weights for policy 1, policy_version 1925059 (0.0010) [2023-12-27 05:23:18,808][105620] Updated weights for policy 1, policy_version 1925069 (0.0010) [2023-12-27 05:23:18,928][105692] Updated weights for policy 0, policy_version 1920371 (0.0008) [2023-12-27 05:23:18,984][105692] Updated weights for policy 0, policy_version 1920381 (0.0008) [2023-12-27 05:23:19,037][105692] Updated weights for policy 0, policy_version 1920391 (0.0005) [2023-12-27 05:23:19,548][105620] Updated weights for policy 1, policy_version 1925079 (0.0011) [2023-12-27 05:23:19,612][105620] Updated weights for policy 1, policy_version 1925089 (0.0011) [2023-12-27 05:23:19,673][105620] Updated weights for policy 1, policy_version 1925099 (0.0010) [2023-12-27 05:23:19,747][105692] Updated weights for policy 0, policy_version 1920401 (0.0006) [2023-12-27 05:23:19,798][105692] Updated weights for policy 0, policy_version 1920411 (0.0008) [2023-12-27 05:23:19,864][105692] Updated weights for policy 0, policy_version 1920421 (0.0008) [2023-12-27 05:23:19,927][105692] Updated weights for policy 0, policy_version 1920431 (0.0008) [2023-12-27 05:23:20,388][105620] Updated weights for policy 1, policy_version 1925109 (0.0009) [2023-12-27 05:23:20,454][105620] Updated weights for policy 1, policy_version 1925119 (0.0008) [2023-12-27 05:23:20,509][105620] Updated weights for policy 1, policy_version 1925129 (0.0009) [2023-12-27 05:23:20,712][105692] Updated weights for policy 0, policy_version 1920441 (0.0009) [2023-12-27 05:23:20,775][105692] Updated weights for policy 0, policy_version 1920451 (0.0009) [2023-12-27 05:23:20,834][105692] Updated weights for policy 0, policy_version 1920461 (0.0009) [2023-12-27 05:23:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 984612864. Throughput: 0: 9521.2, 1: 9780.2. Samples: 984602136. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:23:21,063][104569] Avg episode reward: [(0, '8630.751'), (1, '9071.142')] [2023-12-27 05:23:21,230][105620] Updated weights for policy 1, policy_version 1925139 (0.0009) [2023-12-27 05:23:21,300][105620] Updated weights for policy 1, policy_version 1925149 (0.0009) [2023-12-27 05:23:21,366][105620] Updated weights for policy 1, policy_version 1925159 (0.0009) [2023-12-27 05:23:21,714][105692] Updated weights for policy 0, policy_version 1920471 (0.0010) [2023-12-27 05:23:21,787][105692] Updated weights for policy 0, policy_version 1920481 (0.0008) [2023-12-27 05:23:21,849][105692] Updated weights for policy 0, policy_version 1920491 (0.0008) [2023-12-27 05:23:22,033][105620] Updated weights for policy 1, policy_version 1925169 (0.0007) [2023-12-27 05:23:22,096][105620] Updated weights for policy 1, policy_version 1925179 (0.0009) [2023-12-27 05:23:22,146][105620] Updated weights for policy 1, policy_version 1925189 (0.0008) [2023-12-27 05:23:22,197][105620] Updated weights for policy 1, policy_version 1925199 (0.0008) [2023-12-27 05:23:22,607][105692] Updated weights for policy 0, policy_version 1920501 (0.0009) [2023-12-27 05:23:22,659][105692] Updated weights for policy 0, policy_version 1920511 (0.0009) [2023-12-27 05:23:22,716][105692] Updated weights for policy 0, policy_version 1920521 (0.0009) [2023-12-27 05:23:23,003][105620] Updated weights for policy 1, policy_version 1925209 (0.0009) [2023-12-27 05:23:23,065][105620] Updated weights for policy 1, policy_version 1925219 (0.0008) [2023-12-27 05:23:23,131][105620] Updated weights for policy 1, policy_version 1925229 (0.0009) [2023-12-27 05:23:23,513][105692] Updated weights for policy 0, policy_version 1920531 (0.0008) [2023-12-27 05:23:23,566][105692] Updated weights for policy 0, policy_version 1920541 (0.0005) [2023-12-27 05:23:23,620][105692] Updated weights for policy 0, policy_version 1920551 (0.0005) [2023-12-27 05:23:23,892][105620] Updated weights for policy 1, policy_version 1925239 (0.0009) [2023-12-27 05:23:23,944][105620] Updated weights for policy 1, policy_version 1925249 (0.0011) [2023-12-27 05:23:23,999][105620] Updated weights for policy 1, policy_version 1925259 (0.0006) [2023-12-27 05:23:24,254][105692] Updated weights for policy 0, policy_version 1920561 (0.0006) [2023-12-27 05:23:24,312][105692] Updated weights for policy 0, policy_version 1920571 (0.0009) [2023-12-27 05:23:24,368][105692] Updated weights for policy 0, policy_version 1920581 (0.0009) [2023-12-27 05:23:24,434][105692] Updated weights for policy 0, policy_version 1920591 (0.0009) [2023-12-27 05:23:24,737][105620] Updated weights for policy 1, policy_version 1925269 (0.0008) [2023-12-27 05:23:24,789][105620] Updated weights for policy 1, policy_version 1925279 (0.0006) [2023-12-27 05:23:24,832][105620] Updated weights for policy 1, policy_version 1925289 (0.0005) [2023-12-27 05:23:25,211][105692] Updated weights for policy 0, policy_version 1920601 (0.0008) [2023-12-27 05:23:25,274][105692] Updated weights for policy 0, policy_version 1920611 (0.0005) [2023-12-27 05:23:25,335][105692] Updated weights for policy 0, policy_version 1920621 (0.0008) [2023-12-27 05:23:25,447][105620] Updated weights for policy 1, policy_version 1925299 (0.0006) [2023-12-27 05:23:25,494][105620] Updated weights for policy 1, policy_version 1925309 (0.0009) [2023-12-27 05:23:25,540][105620] Updated weights for policy 1, policy_version 1925319 (0.0008) [2023-12-27 05:23:26,038][105692] Updated weights for policy 0, policy_version 1920631 (0.0009) [2023-12-27 05:23:26,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 984702976. Throughput: 0: 9468.2, 1: 9813.5. Samples: 984715504. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:23:26,062][104569] Avg episode reward: [(0, '8172.038'), (1, '9162.937')] [2023-12-27 05:23:26,096][105692] Updated weights for policy 0, policy_version 1920641 (0.0009) [2023-12-27 05:23:26,154][105692] Updated weights for policy 0, policy_version 1920651 (0.0009) [2023-12-27 05:23:26,283][105620] Updated weights for policy 1, policy_version 1925329 (0.0009) [2023-12-27 05:23:26,338][105620] Updated weights for policy 1, policy_version 1925339 (0.0009) [2023-12-27 05:23:26,399][105620] Updated weights for policy 1, policy_version 1925349 (0.0009) [2023-12-27 05:23:26,459][105620] Updated weights for policy 1, policy_version 1925359 (0.0008) [2023-12-27 05:23:26,913][105692] Updated weights for policy 0, policy_version 1920661 (0.0009) [2023-12-27 05:23:26,971][105692] Updated weights for policy 0, policy_version 1920671 (0.0009) [2023-12-27 05:23:27,019][105692] Updated weights for policy 0, policy_version 1920681 (0.0009) [2023-12-27 05:23:27,186][105620] Updated weights for policy 1, policy_version 1925369 (0.0010) [2023-12-27 05:23:27,236][105620] Updated weights for policy 1, policy_version 1925379 (0.0009) [2023-12-27 05:23:27,286][105620] Updated weights for policy 1, policy_version 1925389 (0.0008) [2023-12-27 05:23:27,719][105692] Updated weights for policy 0, policy_version 1920691 (0.0009) [2023-12-27 05:23:27,781][105692] Updated weights for policy 0, policy_version 1920701 (0.0008) [2023-12-27 05:23:27,845][105692] Updated weights for policy 0, policy_version 1920711 (0.0005) [2023-12-27 05:23:28,105][105620] Updated weights for policy 1, policy_version 1925399 (0.0009) [2023-12-27 05:23:28,163][105620] Updated weights for policy 1, policy_version 1925409 (0.0009) [2023-12-27 05:23:28,220][105620] Updated weights for policy 1, policy_version 1925419 (0.0009) [2023-12-27 05:23:28,421][105692] Updated weights for policy 0, policy_version 1920721 (0.0005) [2023-12-27 05:23:28,476][105692] Updated weights for policy 0, policy_version 1920731 (0.0009) [2023-12-27 05:23:28,534][105692] Updated weights for policy 0, policy_version 1920741 (0.0009) [2023-12-27 05:23:28,587][105692] Updated weights for policy 0, policy_version 1920752 (0.0009) [2023-12-27 05:23:28,993][105620] Updated weights for policy 1, policy_version 1925429 (0.0009) [2023-12-27 05:23:29,047][105620] Updated weights for policy 1, policy_version 1925439 (0.0010) [2023-12-27 05:23:29,092][105620] Updated weights for policy 1, policy_version 1925449 (0.0010) [2023-12-27 05:23:29,352][105692] Updated weights for policy 0, policy_version 1920762 (0.0008) [2023-12-27 05:23:29,409][105692] Updated weights for policy 0, policy_version 1920772 (0.0008) [2023-12-27 05:23:29,469][105692] Updated weights for policy 0, policy_version 1920782 (0.0008) [2023-12-27 05:23:29,851][105620] Updated weights for policy 1, policy_version 1925459 (0.0010) [2023-12-27 05:23:29,906][105620] Updated weights for policy 1, policy_version 1925469 (0.0010) [2023-12-27 05:23:29,968][105620] Updated weights for policy 1, policy_version 1925479 (0.0011) [2023-12-27 05:23:30,141][105692] Updated weights for policy 0, policy_version 1920792 (0.0008) [2023-12-27 05:23:30,194][105692] Updated weights for policy 0, policy_version 1920802 (0.0009) [2023-12-27 05:23:30,246][105692] Updated weights for policy 0, policy_version 1920812 (0.0008) [2023-12-27 05:23:30,715][105620] Updated weights for policy 1, policy_version 1925489 (0.0010) [2023-12-27 05:23:30,764][105620] Updated weights for policy 1, policy_version 1925499 (0.0011) [2023-12-27 05:23:30,812][105620] Updated weights for policy 1, policy_version 1925509 (0.0010) [2023-12-27 05:23:30,864][105620] Updated weights for policy 1, policy_version 1925519 (0.0010) [2023-12-27 05:23:30,961][105692] Updated weights for policy 0, policy_version 1920822 (0.0009) [2023-12-27 05:23:31,016][105692] Updated weights for policy 0, policy_version 1920832 (0.0009) [2023-12-27 05:23:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 984801280. Throughput: 0: 9522.1, 1: 9816.9. Samples: 984773368. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 05:23:31,062][104569] Avg episode reward: [(0, '8260.461'), (1, '9345.670')] [2023-12-27 05:23:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001925520_493002752.pth... [2023-12-27 05:23:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001924368_492707840.pth [2023-12-27 05:23:31,080][105692] Updated weights for policy 0, policy_version 1920842 (0.0007) [2023-12-27 05:23:31,117][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001920848_491806720.pth... [2023-12-27 05:23:31,121][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001919728_491520000.pth [2023-12-27 05:23:31,659][105620] Updated weights for policy 1, policy_version 1925529 (0.0008) [2023-12-27 05:23:31,727][105620] Updated weights for policy 1, policy_version 1925539 (0.0008) [2023-12-27 05:23:31,783][105620] Updated weights for policy 1, policy_version 1925549 (0.0009) [2023-12-27 05:23:31,805][105692] Updated weights for policy 0, policy_version 1920852 (0.0008) [2023-12-27 05:23:31,866][105692] Updated weights for policy 0, policy_version 1920862 (0.0009) [2023-12-27 05:23:31,925][105692] Updated weights for policy 0, policy_version 1920872 (0.0008) [2023-12-27 05:23:32,528][105620] Updated weights for policy 1, policy_version 1925559 (0.0009) [2023-12-27 05:23:32,589][105620] Updated weights for policy 1, policy_version 1925569 (0.0009) [2023-12-27 05:23:32,619][105692] Updated weights for policy 0, policy_version 1920882 (0.0009) [2023-12-27 05:23:32,640][105620] Updated weights for policy 1, policy_version 1925579 (0.0009) [2023-12-27 05:23:32,667][105692] Updated weights for policy 0, policy_version 1920892 (0.0006) [2023-12-27 05:23:32,714][105692] Updated weights for policy 0, policy_version 1920902 (0.0008) [2023-12-27 05:23:32,768][105692] Updated weights for policy 0, policy_version 1920912 (0.0008) [2023-12-27 05:23:33,373][105620] Updated weights for policy 1, policy_version 1925589 (0.0007) [2023-12-27 05:23:33,434][105620] Updated weights for policy 1, policy_version 1925599 (0.0009) [2023-12-27 05:23:33,487][105620] Updated weights for policy 1, policy_version 1925609 (0.0008) [2023-12-27 05:23:33,530][105692] Updated weights for policy 0, policy_version 1920922 (0.0007) [2023-12-27 05:23:33,580][105692] Updated weights for policy 0, policy_version 1920932 (0.0009) [2023-12-27 05:23:33,630][105692] Updated weights for policy 0, policy_version 1920942 (0.0009) [2023-12-27 05:23:34,236][105620] Updated weights for policy 1, policy_version 1925619 (0.0010) [2023-12-27 05:23:34,293][105620] Updated weights for policy 1, policy_version 1925629 (0.0010) [2023-12-27 05:23:34,348][105620] Updated weights for policy 1, policy_version 1925639 (0.0008) [2023-12-27 05:23:34,361][105692] Updated weights for policy 0, policy_version 1920952 (0.0007) [2023-12-27 05:23:34,418][105692] Updated weights for policy 0, policy_version 1920962 (0.0006) [2023-12-27 05:23:34,486][105692] Updated weights for policy 0, policy_version 1920972 (0.0007) [2023-12-27 05:23:35,124][105620] Updated weights for policy 1, policy_version 1925649 (0.0008) [2023-12-27 05:23:35,191][105620] Updated weights for policy 1, policy_version 1925659 (0.0009) [2023-12-27 05:23:35,204][105692] Updated weights for policy 0, policy_version 1920982 (0.0008) [2023-12-27 05:23:35,242][105620] Updated weights for policy 1, policy_version 1925669 (0.0006) [2023-12-27 05:23:35,263][105692] Updated weights for policy 0, policy_version 1920992 (0.0008) [2023-12-27 05:23:35,297][105620] Updated weights for policy 1, policy_version 1925679 (0.0007) [2023-12-27 05:23:35,321][105692] Updated weights for policy 0, policy_version 1921002 (0.0008) [2023-12-27 05:23:35,961][105692] Updated weights for policy 0, policy_version 1921012 (0.0007) [2023-12-27 05:23:36,022][105692] Updated weights for policy 0, policy_version 1921022 (0.0008) [2023-12-27 05:23:36,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 984891392. Throughput: 0: 9492.6, 1: 9820.8. Samples: 984887468. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:23:36,062][104569] Avg episode reward: [(0, '8717.740'), (1, '9068.824')] [2023-12-27 05:23:36,079][105692] Updated weights for policy 0, policy_version 1921032 (0.0010) [2023-12-27 05:23:36,113][105620] Updated weights for policy 1, policy_version 1925689 (0.0007) [2023-12-27 05:23:36,179][105620] Updated weights for policy 1, policy_version 1925699 (0.0007) [2023-12-27 05:23:36,239][105620] Updated weights for policy 1, policy_version 1925709 (0.0008) [2023-12-27 05:23:36,835][105692] Updated weights for policy 0, policy_version 1921042 (0.0009) [2023-12-27 05:23:36,895][105692] Updated weights for policy 0, policy_version 1921052 (0.0011) [2023-12-27 05:23:36,959][105692] Updated weights for policy 0, policy_version 1921062 (0.0011) [2023-12-27 05:23:36,961][105620] Updated weights for policy 1, policy_version 1925719 (0.0007) [2023-12-27 05:23:37,018][105692] Updated weights for policy 0, policy_version 1921072 (0.0011) [2023-12-27 05:23:37,020][105620] Updated weights for policy 1, policy_version 1925729 (0.0006) [2023-12-27 05:23:37,084][105620] Updated weights for policy 1, policy_version 1925739 (0.0009) [2023-12-27 05:23:37,697][105692] Updated weights for policy 0, policy_version 1921082 (0.0010) [2023-12-27 05:23:37,752][105620] Updated weights for policy 1, policy_version 1925749 (0.0008) [2023-12-27 05:23:37,757][105692] Updated weights for policy 0, policy_version 1921092 (0.0009) [2023-12-27 05:23:37,809][105620] Updated weights for policy 1, policy_version 1925759 (0.0007) [2023-12-27 05:23:37,819][105692] Updated weights for policy 0, policy_version 1921102 (0.0007) [2023-12-27 05:23:37,866][105620] Updated weights for policy 1, policy_version 1925769 (0.0008) [2023-12-27 05:23:38,551][105692] Updated weights for policy 0, policy_version 1921112 (0.0006) [2023-12-27 05:23:38,604][105620] Updated weights for policy 1, policy_version 1925779 (0.0009) [2023-12-27 05:23:38,605][105692] Updated weights for policy 0, policy_version 1921122 (0.0005) [2023-12-27 05:23:38,653][105620] Updated weights for policy 1, policy_version 1925789 (0.0010) [2023-12-27 05:23:38,667][105692] Updated weights for policy 0, policy_version 1921132 (0.0005) [2023-12-27 05:23:38,702][105620] Updated weights for policy 1, policy_version 1925799 (0.0010) [2023-12-27 05:23:39,401][105692] Updated weights for policy 0, policy_version 1921142 (0.0008) [2023-12-27 05:23:39,432][105620] Updated weights for policy 1, policy_version 1925809 (0.0010) [2023-12-27 05:23:39,468][105692] Updated weights for policy 0, policy_version 1921152 (0.0008) [2023-12-27 05:23:39,496][105620] Updated weights for policy 1, policy_version 1925819 (0.0006) [2023-12-27 05:23:39,536][105692] Updated weights for policy 0, policy_version 1921162 (0.0007) [2023-12-27 05:23:39,566][105620] Updated weights for policy 1, policy_version 1925829 (0.0007) [2023-12-27 05:23:39,640][105620] Updated weights for policy 1, policy_version 1925839 (0.0006) [2023-12-27 05:23:40,199][105692] Updated weights for policy 0, policy_version 1921172 (0.0006) [2023-12-27 05:23:40,268][105692] Updated weights for policy 0, policy_version 1921182 (0.0007) [2023-12-27 05:23:40,333][105692] Updated weights for policy 0, policy_version 1921192 (0.0010) [2023-12-27 05:23:40,348][105620] Updated weights for policy 1, policy_version 1925849 (0.0008) [2023-12-27 05:23:40,414][105620] Updated weights for policy 1, policy_version 1925859 (0.0007) [2023-12-27 05:23:40,477][105620] Updated weights for policy 1, policy_version 1925869 (0.0005) [2023-12-27 05:23:41,020][105692] Updated weights for policy 0, policy_version 1921202 (0.0008) [2023-12-27 05:23:41,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 984989696. Throughput: 0: 9601.6, 1: 9703.4. Samples: 985004696. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:23:41,063][104569] Avg episode reward: [(0, '8808.211'), (1, '9068.900')] [2023-12-27 05:23:41,083][105620] Updated weights for policy 1, policy_version 1925879 (0.0007) [2023-12-27 05:23:41,099][105692] Updated weights for policy 0, policy_version 1921212 (0.0006) [2023-12-27 05:23:41,148][105620] Updated weights for policy 1, policy_version 1925889 (0.0007) [2023-12-27 05:23:41,172][105692] Updated weights for policy 0, policy_version 1921222 (0.0009) [2023-12-27 05:23:41,215][105620] Updated weights for policy 1, policy_version 1925899 (0.0008) [2023-12-27 05:23:41,239][105692] Updated weights for policy 0, policy_version 1921232 (0.0008) [2023-12-27 05:23:41,981][105692] Updated weights for policy 0, policy_version 1921242 (0.0008) [2023-12-27 05:23:41,983][105620] Updated weights for policy 1, policy_version 1925909 (0.0008) [2023-12-27 05:23:42,041][105620] Updated weights for policy 1, policy_version 1925919 (0.0006) [2023-12-27 05:23:42,043][105692] Updated weights for policy 0, policy_version 1921252 (0.0007) [2023-12-27 05:23:42,105][105692] Updated weights for policy 0, policy_version 1921262 (0.0007) [2023-12-27 05:23:42,107][105620] Updated weights for policy 1, policy_version 1925929 (0.0007) [2023-12-27 05:23:42,862][105692] Updated weights for policy 0, policy_version 1921272 (0.0009) [2023-12-27 05:23:42,881][105620] Updated weights for policy 1, policy_version 1925939 (0.0008) [2023-12-27 05:23:42,926][105692] Updated weights for policy 0, policy_version 1921282 (0.0007) [2023-12-27 05:23:42,947][105620] Updated weights for policy 1, policy_version 1925949 (0.0006) [2023-12-27 05:23:42,995][105692] Updated weights for policy 0, policy_version 1921292 (0.0006) [2023-12-27 05:23:43,017][105620] Updated weights for policy 1, policy_version 1925959 (0.0006) [2023-12-27 05:23:43,516][105620] Updated weights for policy 1, policy_version 1925969 (0.0005) [2023-12-27 05:23:43,576][105620] Updated weights for policy 1, policy_version 1925979 (0.0006) [2023-12-27 05:23:43,630][105620] Updated weights for policy 1, policy_version 1925989 (0.0009) [2023-12-27 05:23:43,684][105620] Updated weights for policy 1, policy_version 1925999 (0.0009) [2023-12-27 05:23:43,759][105692] Updated weights for policy 0, policy_version 1921302 (0.0008) [2023-12-27 05:23:43,813][105692] Updated weights for policy 0, policy_version 1921314 (0.0010) [2023-12-27 05:23:43,867][105692] Updated weights for policy 0, policy_version 1921324 (0.0009) [2023-12-27 05:23:44,278][105620] Updated weights for policy 1, policy_version 1926009 (0.0009) [2023-12-27 05:23:44,336][105620] Updated weights for policy 1, policy_version 1926019 (0.0009) [2023-12-27 05:23:44,388][105620] Updated weights for policy 1, policy_version 1926029 (0.0009) [2023-12-27 05:23:44,691][105692] Updated weights for policy 0, policy_version 1921334 (0.0008) [2023-12-27 05:23:44,752][105692] Updated weights for policy 0, policy_version 1921344 (0.0009) [2023-12-27 05:23:44,817][105692] Updated weights for policy 0, policy_version 1921354 (0.0008) [2023-12-27 05:23:45,094][105620] Updated weights for policy 1, policy_version 1926039 (0.0010) [2023-12-27 05:23:45,154][105620] Updated weights for policy 1, policy_version 1926049 (0.0011) [2023-12-27 05:23:45,217][105620] Updated weights for policy 1, policy_version 1926059 (0.0011) [2023-12-27 05:23:45,512][105692] Updated weights for policy 0, policy_version 1921364 (0.0008) [2023-12-27 05:23:45,579][105692] Updated weights for policy 0, policy_version 1921374 (0.0008) [2023-12-27 05:23:45,630][105692] Updated weights for policy 0, policy_version 1921384 (0.0009) [2023-12-27 05:23:45,935][105620] Updated weights for policy 1, policy_version 1926069 (0.0008) [2023-12-27 05:23:45,989][105620] Updated weights for policy 1, policy_version 1926079 (0.0005) [2023-12-27 05:23:46,042][105620] Updated weights for policy 1, policy_version 1926089 (0.0005) [2023-12-27 05:23:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19251.1, 300 sec: 19494.2). Total num frames: 985088000. Throughput: 0: 9539.5, 1: 9708.7. Samples: 985062028. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:23:46,063][104569] Avg episode reward: [(0, '8630.033'), (1, '9253.538')] [2023-12-27 05:23:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001921392_491945984.pth... [2023-12-27 05:23:46,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001920272_491659264.pth [2023-12-27 05:23:46,076][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001926096_493150208.pth... [2023-12-27 05:23:46,080][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001924944_492855296.pth [2023-12-27 05:23:46,461][105692] Updated weights for policy 0, policy_version 1921394 (0.0009) [2023-12-27 05:23:46,524][105692] Updated weights for policy 0, policy_version 1921404 (0.0010) [2023-12-27 05:23:46,576][105692] Updated weights for policy 0, policy_version 1921414 (0.0009) [2023-12-27 05:23:46,589][105620] Updated weights for policy 1, policy_version 1926099 (0.0006) [2023-12-27 05:23:46,628][105692] Updated weights for policy 0, policy_version 1921424 (0.0007) [2023-12-27 05:23:46,648][105620] Updated weights for policy 1, policy_version 1926109 (0.0007) [2023-12-27 05:23:46,712][105620] Updated weights for policy 1, policy_version 1926119 (0.0009) [2023-12-27 05:23:47,384][105620] Updated weights for policy 1, policy_version 1926129 (0.0009) [2023-12-27 05:23:47,422][105692] Updated weights for policy 0, policy_version 1921434 (0.0008) [2023-12-27 05:23:47,441][105620] Updated weights for policy 1, policy_version 1926139 (0.0006) [2023-12-27 05:23:47,474][105692] Updated weights for policy 0, policy_version 1921444 (0.0006) [2023-12-27 05:23:47,489][105620] Updated weights for policy 1, policy_version 1926149 (0.0006) [2023-12-27 05:23:47,523][105692] Updated weights for policy 0, policy_version 1921454 (0.0006) [2023-12-27 05:23:47,534][105620] Updated weights for policy 1, policy_version 1926159 (0.0006) [2023-12-27 05:23:48,259][105692] Updated weights for policy 0, policy_version 1921464 (0.0006) [2023-12-27 05:23:48,308][105692] Updated weights for policy 0, policy_version 1921474 (0.0005) [2023-12-27 05:23:48,318][105620] Updated weights for policy 1, policy_version 1926169 (0.0010) [2023-12-27 05:23:48,369][105692] Updated weights for policy 0, policy_version 1921484 (0.0008) [2023-12-27 05:23:48,379][105620] Updated weights for policy 1, policy_version 1926179 (0.0008) [2023-12-27 05:23:48,437][105620] Updated weights for policy 1, policy_version 1926189 (0.0008) [2023-12-27 05:23:49,049][105620] Updated weights for policy 1, policy_version 1926199 (0.0006) [2023-12-27 05:23:49,103][105692] Updated weights for policy 0, policy_version 1921494 (0.0010) [2023-12-27 05:23:49,112][105620] Updated weights for policy 1, policy_version 1926209 (0.0010) [2023-12-27 05:23:49,152][105692] Updated weights for policy 0, policy_version 1921504 (0.0010) [2023-12-27 05:23:49,169][105620] Updated weights for policy 1, policy_version 1926219 (0.0010) [2023-12-27 05:23:49,207][105692] Updated weights for policy 0, policy_version 1921514 (0.0010) [2023-12-27 05:23:49,889][105620] Updated weights for policy 1, policy_version 1926229 (0.0008) [2023-12-27 05:23:49,931][105692] Updated weights for policy 0, policy_version 1921524 (0.0009) [2023-12-27 05:23:49,957][105620] Updated weights for policy 1, policy_version 1926239 (0.0010) [2023-12-27 05:23:49,989][105692] Updated weights for policy 0, policy_version 1921534 (0.0011) [2023-12-27 05:23:50,016][105620] Updated weights for policy 1, policy_version 1926249 (0.0010) [2023-12-27 05:23:50,050][105692] Updated weights for policy 0, policy_version 1921544 (0.0011) [2023-12-27 05:23:50,766][105620] Updated weights for policy 1, policy_version 1926259 (0.0010) [2023-12-27 05:23:50,820][105692] Updated weights for policy 0, policy_version 1921554 (0.0010) [2023-12-27 05:23:50,828][105620] Updated weights for policy 1, policy_version 1926269 (0.0010) [2023-12-27 05:23:50,878][105692] Updated weights for policy 0, policy_version 1921564 (0.0010) [2023-12-27 05:23:50,888][105620] Updated weights for policy 1, policy_version 1926279 (0.0010) [2023-12-27 05:23:50,934][105692] Updated weights for policy 0, policy_version 1921574 (0.0010) [2023-12-27 05:23:50,992][105692] Updated weights for policy 0, policy_version 1921584 (0.0007) [2023-12-27 05:23:51,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 985194496. Throughput: 0: 9513.3, 1: 9784.9. Samples: 985178908. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:23:51,062][104569] Avg episode reward: [(0, '8270.402'), (1, '9345.872')] [2023-12-27 05:23:51,639][105692] Updated weights for policy 0, policy_version 1921594 (0.0009) [2023-12-27 05:23:51,640][105620] Updated weights for policy 1, policy_version 1926289 (0.0010) [2023-12-27 05:23:51,694][105620] Updated weights for policy 1, policy_version 1926299 (0.0011) [2023-12-27 05:23:51,701][105692] Updated weights for policy 0, policy_version 1921604 (0.0006) [2023-12-27 05:23:51,757][105620] Updated weights for policy 1, policy_version 1926309 (0.0011) [2023-12-27 05:23:51,772][105692] Updated weights for policy 0, policy_version 1921614 (0.0009) [2023-12-27 05:23:51,822][105620] Updated weights for policy 1, policy_version 1926319 (0.0011) [2023-12-27 05:23:52,364][105692] Updated weights for policy 0, policy_version 1921624 (0.0007) [2023-12-27 05:23:52,422][105692] Updated weights for policy 0, policy_version 1921634 (0.0009) [2023-12-27 05:23:52,476][105692] Updated weights for policy 0, policy_version 1921644 (0.0009) [2023-12-27 05:23:52,606][105620] Updated weights for policy 1, policy_version 1926329 (0.0008) [2023-12-27 05:23:52,653][105620] Updated weights for policy 1, policy_version 1926339 (0.0005) [2023-12-27 05:23:52,702][105620] Updated weights for policy 1, policy_version 1926349 (0.0005) [2023-12-27 05:23:53,125][105692] Updated weights for policy 0, policy_version 1921654 (0.0008) [2023-12-27 05:23:53,181][105692] Updated weights for policy 0, policy_version 1921664 (0.0008) [2023-12-27 05:23:53,241][105692] Updated weights for policy 0, policy_version 1921674 (0.0008) [2023-12-27 05:23:53,439][105620] Updated weights for policy 1, policy_version 1926359 (0.0010) [2023-12-27 05:23:53,508][105620] Updated weights for policy 1, policy_version 1926369 (0.0010) [2023-12-27 05:23:53,572][105620] Updated weights for policy 1, policy_version 1926379 (0.0010) [2023-12-27 05:23:53,927][105692] Updated weights for policy 0, policy_version 1921684 (0.0007) [2023-12-27 05:23:53,982][105692] Updated weights for policy 0, policy_version 1921694 (0.0010) [2023-12-27 05:23:54,038][105692] Updated weights for policy 0, policy_version 1921704 (0.0010) [2023-12-27 05:23:54,134][105620] Updated weights for policy 1, policy_version 1926389 (0.0008) [2023-12-27 05:23:54,186][105620] Updated weights for policy 1, policy_version 1926399 (0.0005) [2023-12-27 05:23:54,240][105620] Updated weights for policy 1, policy_version 1926409 (0.0005) [2023-12-27 05:23:54,718][105692] Updated weights for policy 0, policy_version 1921714 (0.0006) [2023-12-27 05:23:54,765][105692] Updated weights for policy 0, policy_version 1921724 (0.0005) [2023-12-27 05:23:54,812][105692] Updated weights for policy 0, policy_version 1921734 (0.0005) [2023-12-27 05:23:54,858][105692] Updated weights for policy 0, policy_version 1921744 (0.0005) [2023-12-27 05:23:54,900][105620] Updated weights for policy 1, policy_version 1926419 (0.0007) [2023-12-27 05:23:54,968][105620] Updated weights for policy 1, policy_version 1926429 (0.0009) [2023-12-27 05:23:55,030][105620] Updated weights for policy 1, policy_version 1926439 (0.0005) [2023-12-27 05:23:55,425][105692] Updated weights for policy 0, policy_version 1921754 (0.0006) [2023-12-27 05:23:55,483][105692] Updated weights for policy 0, policy_version 1921764 (0.0010) [2023-12-27 05:23:55,531][105692] Updated weights for policy 0, policy_version 1921774 (0.0010) [2023-12-27 05:23:55,606][105620] Updated weights for policy 1, policy_version 1926449 (0.0006) [2023-12-27 05:23:55,670][105620] Updated weights for policy 1, policy_version 1926459 (0.0010) [2023-12-27 05:23:55,728][105620] Updated weights for policy 1, policy_version 1926469 (0.0010) [2023-12-27 05:23:55,782][105620] Updated weights for policy 1, policy_version 1926479 (0.0010) [2023-12-27 05:23:56,062][104569] Fps is (10 sec: 20480.5, 60 sec: 19387.8, 300 sec: 19494.2). Total num frames: 985292800. Throughput: 0: 9609.4, 1: 9772.8. Samples: 985301020. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:23:56,062][104569] Avg episode reward: [(0, '8720.196'), (1, '9345.901')] [2023-12-27 05:23:56,267][105692] Updated weights for policy 0, policy_version 1921784 (0.0010) [2023-12-27 05:23:56,331][105692] Updated weights for policy 0, policy_version 1921794 (0.0010) [2023-12-27 05:23:56,386][105692] Updated weights for policy 0, policy_version 1921804 (0.0010) [2023-12-27 05:23:56,498][105620] Updated weights for policy 1, policy_version 1926489 (0.0006) [2023-12-27 05:23:56,559][105620] Updated weights for policy 1, policy_version 1926499 (0.0005) [2023-12-27 05:23:56,620][105620] Updated weights for policy 1, policy_version 1926509 (0.0005) [2023-12-27 05:23:57,120][105692] Updated weights for policy 0, policy_version 1921814 (0.0010) [2023-12-27 05:23:57,167][105692] Updated weights for policy 0, policy_version 1921824 (0.0010) [2023-12-27 05:23:57,194][105620] Updated weights for policy 1, policy_version 1926519 (0.0009) [2023-12-27 05:23:57,222][105692] Updated weights for policy 0, policy_version 1921834 (0.0010) [2023-12-27 05:23:57,243][105620] Updated weights for policy 1, policy_version 1926529 (0.0010) [2023-12-27 05:23:57,291][105620] Updated weights for policy 1, policy_version 1926539 (0.0010) [2023-12-27 05:23:57,963][105692] Updated weights for policy 0, policy_version 1921844 (0.0009) [2023-12-27 05:23:58,016][105620] Updated weights for policy 1, policy_version 1926549 (0.0010) [2023-12-27 05:23:58,019][105692] Updated weights for policy 0, policy_version 1921854 (0.0006) [2023-12-27 05:23:58,071][105620] Updated weights for policy 1, policy_version 1926559 (0.0010) [2023-12-27 05:23:58,073][105692] Updated weights for policy 0, policy_version 1921864 (0.0005) [2023-12-27 05:23:58,122][105620] Updated weights for policy 1, policy_version 1926569 (0.0010) [2023-12-27 05:23:58,934][105620] Updated weights for policy 1, policy_version 1926579 (0.0010) [2023-12-27 05:23:58,954][105692] Updated weights for policy 0, policy_version 1921874 (0.0007) [2023-12-27 05:23:58,997][105620] Updated weights for policy 1, policy_version 1926589 (0.0011) [2023-12-27 05:23:59,011][105692] Updated weights for policy 0, policy_version 1921884 (0.0011) [2023-12-27 05:23:59,056][105620] Updated weights for policy 1, policy_version 1926599 (0.0010) [2023-12-27 05:23:59,070][105692] Updated weights for policy 0, policy_version 1921894 (0.0011) [2023-12-27 05:23:59,127][105692] Updated weights for policy 0, policy_version 1921904 (0.0010) [2023-12-27 05:23:59,755][105620] Updated weights for policy 1, policy_version 1926609 (0.0009) [2023-12-27 05:23:59,810][105620] Updated weights for policy 1, policy_version 1926619 (0.0010) [2023-12-27 05:23:59,862][105692] Updated weights for policy 0, policy_version 1921914 (0.0007) [2023-12-27 05:23:59,873][105620] Updated weights for policy 1, policy_version 1926629 (0.0011) [2023-12-27 05:23:59,927][105692] Updated weights for policy 0, policy_version 1921924 (0.0007) [2023-12-27 05:23:59,939][105620] Updated weights for policy 1, policy_version 1926639 (0.0008) [2023-12-27 05:23:59,987][105692] Updated weights for policy 0, policy_version 1921934 (0.0005) [2023-12-27 05:24:00,529][105620] Updated weights for policy 1, policy_version 1926649 (0.0009) [2023-12-27 05:24:00,577][105620] Updated weights for policy 1, policy_version 1926659 (0.0010) [2023-12-27 05:24:00,618][105692] Updated weights for policy 0, policy_version 1921944 (0.0005) [2023-12-27 05:24:00,632][105620] Updated weights for policy 1, policy_version 1926669 (0.0010) [2023-12-27 05:24:00,681][105692] Updated weights for policy 0, policy_version 1921954 (0.0005) [2023-12-27 05:24:00,736][105692] Updated weights for policy 0, policy_version 1921964 (0.0005) [2023-12-27 05:24:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 985391104. Throughput: 0: 9636.6, 1: 9802.6. Samples: 985358956. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:24:01,062][104569] Avg episode reward: [(0, '8903.374'), (1, '9254.640')] [2023-12-27 05:24:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001921968_492093440.pth... [2023-12-27 05:24:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001926672_493297664.pth... [2023-12-27 05:24:01,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001920848_491806720.pth [2023-12-27 05:24:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001925520_493002752.pth [2023-12-27 05:24:01,274][105620] Updated weights for policy 1, policy_version 1926679 (0.0009) [2023-12-27 05:24:01,341][105620] Updated weights for policy 1, policy_version 1926689 (0.0009) [2023-12-27 05:24:01,404][105620] Updated weights for policy 1, policy_version 1926699 (0.0011) [2023-12-27 05:24:01,452][105692] Updated weights for policy 0, policy_version 1921974 (0.0005) [2023-12-27 05:24:01,517][105692] Updated weights for policy 0, policy_version 1921984 (0.0007) [2023-12-27 05:24:01,580][105692] Updated weights for policy 0, policy_version 1921994 (0.0008) [2023-12-27 05:24:02,115][105620] Updated weights for policy 1, policy_version 1926709 (0.0010) [2023-12-27 05:24:02,170][105620] Updated weights for policy 1, policy_version 1926719 (0.0008) [2023-12-27 05:24:02,218][105620] Updated weights for policy 1, policy_version 1926729 (0.0005) [2023-12-27 05:24:02,296][105692] Updated weights for policy 0, policy_version 1922004 (0.0010) [2023-12-27 05:24:02,356][105692] Updated weights for policy 0, policy_version 1922014 (0.0009) [2023-12-27 05:24:02,422][105692] Updated weights for policy 0, policy_version 1922024 (0.0009) [2023-12-27 05:24:02,971][105620] Updated weights for policy 1, policy_version 1926739 (0.0007) [2023-12-27 05:24:03,031][105620] Updated weights for policy 1, policy_version 1926749 (0.0009) [2023-12-27 05:24:03,093][105692] Updated weights for policy 0, policy_version 1922034 (0.0007) [2023-12-27 05:24:03,100][105620] Updated weights for policy 1, policy_version 1926759 (0.0008) [2023-12-27 05:24:03,149][105692] Updated weights for policy 0, policy_version 1922044 (0.0006) [2023-12-27 05:24:03,203][105692] Updated weights for policy 0, policy_version 1922054 (0.0008) [2023-12-27 05:24:03,251][105692] Updated weights for policy 0, policy_version 1922064 (0.0008) [2023-12-27 05:24:03,856][105692] Updated weights for policy 0, policy_version 1922074 (0.0006) [2023-12-27 05:24:03,866][105620] Updated weights for policy 1, policy_version 1926769 (0.0008) [2023-12-27 05:24:03,911][105692] Updated weights for policy 0, policy_version 1922084 (0.0007) [2023-12-27 05:24:03,928][105620] Updated weights for policy 1, policy_version 1926779 (0.0007) [2023-12-27 05:24:03,963][105692] Updated weights for policy 0, policy_version 1922094 (0.0007) [2023-12-27 05:24:03,999][105620] Updated weights for policy 1, policy_version 1926789 (0.0009) [2023-12-27 05:24:04,070][105620] Updated weights for policy 1, policy_version 1926799 (0.0010) [2023-12-27 05:24:04,612][105692] Updated weights for policy 0, policy_version 1922104 (0.0008) [2023-12-27 05:24:04,677][105692] Updated weights for policy 0, policy_version 1922114 (0.0009) [2023-12-27 05:24:04,738][105692] Updated weights for policy 0, policy_version 1922124 (0.0009) [2023-12-27 05:24:04,783][105620] Updated weights for policy 1, policy_version 1926809 (0.0008) [2023-12-27 05:24:04,830][105620] Updated weights for policy 1, policy_version 1926819 (0.0009) [2023-12-27 05:24:04,876][105620] Updated weights for policy 1, policy_version 1926829 (0.0008) [2023-12-27 05:24:05,512][105692] Updated weights for policy 0, policy_version 1922134 (0.0009) [2023-12-27 05:24:05,558][105692] Updated weights for policy 0, policy_version 1922144 (0.0008) [2023-12-27 05:24:05,571][105620] Updated weights for policy 1, policy_version 1926839 (0.0006) [2023-12-27 05:24:05,610][105692] Updated weights for policy 0, policy_version 1922154 (0.0010) [2023-12-27 05:24:05,623][105620] Updated weights for policy 1, policy_version 1926849 (0.0009) [2023-12-27 05:24:05,671][105620] Updated weights for policy 1, policy_version 1926859 (0.0008) [2023-12-27 05:24:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 985489408. Throughput: 0: 9688.2, 1: 9774.9. Samples: 985477972. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:24:06,062][104569] Avg episode reward: [(0, '8450.559'), (1, '9254.719')] [2023-12-27 05:24:06,345][105692] Updated weights for policy 0, policy_version 1922164 (0.0007) [2023-12-27 05:24:06,390][105620] Updated weights for policy 1, policy_version 1926869 (0.0008) [2023-12-27 05:24:06,402][105692] Updated weights for policy 0, policy_version 1922174 (0.0010) [2023-12-27 05:24:06,450][105620] Updated weights for policy 1, policy_version 1926879 (0.0011) [2023-12-27 05:24:06,458][105692] Updated weights for policy 0, policy_version 1922184 (0.0011) [2023-12-27 05:24:06,513][105620] Updated weights for policy 1, policy_version 1926889 (0.0011) [2023-12-27 05:24:07,121][105620] Updated weights for policy 1, policy_version 1926899 (0.0009) [2023-12-27 05:24:07,183][105620] Updated weights for policy 1, policy_version 1926909 (0.0010) [2023-12-27 05:24:07,206][105692] Updated weights for policy 0, policy_version 1922194 (0.0011) [2023-12-27 05:24:07,244][105620] Updated weights for policy 1, policy_version 1926919 (0.0005) [2023-12-27 05:24:07,272][105692] Updated weights for policy 0, policy_version 1922204 (0.0011) [2023-12-27 05:24:07,331][105692] Updated weights for policy 0, policy_version 1922214 (0.0011) [2023-12-27 05:24:07,391][105692] Updated weights for policy 0, policy_version 1922224 (0.0011) [2023-12-27 05:24:07,900][105620] Updated weights for policy 1, policy_version 1926929 (0.0006) [2023-12-27 05:24:07,964][105620] Updated weights for policy 1, policy_version 1926939 (0.0009) [2023-12-27 05:24:08,029][105620] Updated weights for policy 1, policy_version 1926949 (0.0009) [2023-12-27 05:24:08,097][105620] Updated weights for policy 1, policy_version 1926959 (0.0008) [2023-12-27 05:24:08,148][105692] Updated weights for policy 0, policy_version 1922234 (0.0010) [2023-12-27 05:24:08,200][105692] Updated weights for policy 0, policy_version 1922244 (0.0010) [2023-12-27 05:24:08,252][105692] Updated weights for policy 0, policy_version 1922254 (0.0010) [2023-12-27 05:24:08,810][105620] Updated weights for policy 1, policy_version 1926969 (0.0007) [2023-12-27 05:24:08,874][105620] Updated weights for policy 1, policy_version 1926979 (0.0008) [2023-12-27 05:24:08,934][105620] Updated weights for policy 1, policy_version 1926989 (0.0009) [2023-12-27 05:24:08,999][105692] Updated weights for policy 0, policy_version 1922264 (0.0008) [2023-12-27 05:24:09,056][105692] Updated weights for policy 0, policy_version 1922274 (0.0010) [2023-12-27 05:24:09,114][105692] Updated weights for policy 0, policy_version 1922284 (0.0010) [2023-12-27 05:24:09,700][105620] Updated weights for policy 1, policy_version 1926999 (0.0007) [2023-12-27 05:24:09,760][105620] Updated weights for policy 1, policy_version 1927009 (0.0008) [2023-12-27 05:24:09,816][105620] Updated weights for policy 1, policy_version 1927019 (0.0007) [2023-12-27 05:24:09,816][105692] Updated weights for policy 0, policy_version 1922294 (0.0011) [2023-12-27 05:24:09,881][105692] Updated weights for policy 0, policy_version 1922304 (0.0011) [2023-12-27 05:24:09,942][105692] Updated weights for policy 0, policy_version 1922314 (0.0011) [2023-12-27 05:24:10,433][105620] Updated weights for policy 1, policy_version 1927029 (0.0007) [2023-12-27 05:24:10,494][105620] Updated weights for policy 1, policy_version 1927039 (0.0005) [2023-12-27 05:24:10,563][105620] Updated weights for policy 1, policy_version 1927049 (0.0006) [2023-12-27 05:24:10,717][105692] Updated weights for policy 0, policy_version 1922324 (0.0011) [2023-12-27 05:24:10,783][105692] Updated weights for policy 0, policy_version 1922334 (0.0010) [2023-12-27 05:24:10,843][105692] Updated weights for policy 0, policy_version 1922344 (0.0009) [2023-12-27 05:24:11,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19494.2). Total num frames: 985587712. Throughput: 0: 9710.4, 1: 9846.6. Samples: 985595568. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:24:11,063][104569] Avg episode reward: [(0, '8081.139'), (1, '9255.905')] [2023-12-27 05:24:11,298][105620] Updated weights for policy 1, policy_version 1927059 (0.0007) [2023-12-27 05:24:11,362][105620] Updated weights for policy 1, policy_version 1927069 (0.0007) [2023-12-27 05:24:11,429][105620] Updated weights for policy 1, policy_version 1927079 (0.0010) [2023-12-27 05:24:11,577][105692] Updated weights for policy 0, policy_version 1922354 (0.0006) [2023-12-27 05:24:11,645][105692] Updated weights for policy 0, policy_version 1922364 (0.0009) [2023-12-27 05:24:11,719][105692] Updated weights for policy 0, policy_version 1922374 (0.0009) [2023-12-27 05:24:11,786][105692] Updated weights for policy 0, policy_version 1922384 (0.0008) [2023-12-27 05:24:12,122][105620] Updated weights for policy 1, policy_version 1927089 (0.0010) [2023-12-27 05:24:12,192][105620] Updated weights for policy 1, policy_version 1927099 (0.0007) [2023-12-27 05:24:12,264][105620] Updated weights for policy 1, policy_version 1927109 (0.0006) [2023-12-27 05:24:12,333][105620] Updated weights for policy 1, policy_version 1927119 (0.0007) [2023-12-27 05:24:12,625][105692] Updated weights for policy 0, policy_version 1922394 (0.0008) [2023-12-27 05:24:12,679][105692] Updated weights for policy 0, policy_version 1922404 (0.0009) [2023-12-27 05:24:12,741][105692] Updated weights for policy 0, policy_version 1922414 (0.0009) [2023-12-27 05:24:12,970][105620] Updated weights for policy 1, policy_version 1927129 (0.0010) [2023-12-27 05:24:13,037][105620] Updated weights for policy 1, policy_version 1927139 (0.0008) [2023-12-27 05:24:13,095][105620] Updated weights for policy 1, policy_version 1927149 (0.0007) [2023-12-27 05:24:13,398][105692] Updated weights for policy 0, policy_version 1922424 (0.0007) [2023-12-27 05:24:13,451][105692] Updated weights for policy 0, policy_version 1922434 (0.0006) [2023-12-27 05:24:13,500][105692] Updated weights for policy 0, policy_version 1922444 (0.0005) [2023-12-27 05:24:13,788][105620] Updated weights for policy 1, policy_version 1927159 (0.0008) [2023-12-27 05:24:13,848][105620] Updated weights for policy 1, policy_version 1927169 (0.0006) [2023-12-27 05:24:13,912][105620] Updated weights for policy 1, policy_version 1927179 (0.0009) [2023-12-27 05:24:14,096][105692] Updated weights for policy 0, policy_version 1922454 (0.0005) [2023-12-27 05:24:14,143][105692] Updated weights for policy 0, policy_version 1922464 (0.0005) [2023-12-27 05:24:14,191][105692] Updated weights for policy 0, policy_version 1922474 (0.0005) [2023-12-27 05:24:14,658][105620] Updated weights for policy 1, policy_version 1927189 (0.0009) [2023-12-27 05:24:14,711][105620] Updated weights for policy 1, policy_version 1927200 (0.0011) [2023-12-27 05:24:14,757][105620] Updated weights for policy 1, policy_version 1927210 (0.0010) [2023-12-27 05:24:14,822][105692] Updated weights for policy 0, policy_version 1922484 (0.0008) [2023-12-27 05:24:14,882][105692] Updated weights for policy 0, policy_version 1922494 (0.0011) [2023-12-27 05:24:14,945][105692] Updated weights for policy 0, policy_version 1922504 (0.0011) [2023-12-27 05:24:15,542][105692] Updated weights for policy 0, policy_version 1922514 (0.0010) [2023-12-27 05:24:15,574][105620] Updated weights for policy 1, policy_version 1927220 (0.0011) [2023-12-27 05:24:15,599][105692] Updated weights for policy 0, policy_version 1922524 (0.0006) [2023-12-27 05:24:15,633][105620] Updated weights for policy 1, policy_version 1927230 (0.0010) [2023-12-27 05:24:15,644][105692] Updated weights for policy 0, policy_version 1922534 (0.0005) [2023-12-27 05:24:15,688][105620] Updated weights for policy 1, policy_version 1927240 (0.0010) [2023-12-27 05:24:15,698][105692] Updated weights for policy 0, policy_version 1922544 (0.0006) [2023-12-27 05:24:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19524.2, 300 sec: 19521.9). Total num frames: 985686016. Throughput: 0: 9680.1, 1: 9860.1. Samples: 985652676. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:24:16,063][104569] Avg episode reward: [(0, '8440.390'), (1, '9164.374')] [2023-12-27 05:24:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001922544_492240896.pth... [2023-12-27 05:24:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001927248_493445120.pth... [2023-12-27 05:24:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001921392_491945984.pth [2023-12-27 05:24:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001926096_493150208.pth [2023-12-27 05:24:16,235][105692] Updated weights for policy 0, policy_version 1922554 (0.0005) [2023-12-27 05:24:16,295][105692] Updated weights for policy 0, policy_version 1922564 (0.0006) [2023-12-27 05:24:16,355][105692] Updated weights for policy 0, policy_version 1922574 (0.0005) [2023-12-27 05:24:16,395][105620] Updated weights for policy 1, policy_version 1927250 (0.0010) [2023-12-27 05:24:16,466][105620] Updated weights for policy 1, policy_version 1927260 (0.0010) [2023-12-27 05:24:16,527][105620] Updated weights for policy 1, policy_version 1927270 (0.0010) [2023-12-27 05:24:16,584][105620] Updated weights for policy 1, policy_version 1927280 (0.0010) [2023-12-27 05:24:16,880][105692] Updated weights for policy 0, policy_version 1922584 (0.0006) [2023-12-27 05:24:16,936][105692] Updated weights for policy 0, policy_version 1922594 (0.0005) [2023-12-27 05:24:17,002][105692] Updated weights for policy 0, policy_version 1922604 (0.0009) [2023-12-27 05:24:17,302][105620] Updated weights for policy 1, policy_version 1927290 (0.0011) [2023-12-27 05:24:17,358][105620] Updated weights for policy 1, policy_version 1927300 (0.0011) [2023-12-27 05:24:17,416][105620] Updated weights for policy 1, policy_version 1927310 (0.0010) [2023-12-27 05:24:17,696][105692] Updated weights for policy 0, policy_version 1922614 (0.0009) [2023-12-27 05:24:17,744][105692] Updated weights for policy 0, policy_version 1922624 (0.0010) [2023-12-27 05:24:17,794][105692] Updated weights for policy 0, policy_version 1922634 (0.0008) [2023-12-27 05:24:18,036][105620] Updated weights for policy 1, policy_version 1927320 (0.0010) [2023-12-27 05:24:18,088][105620] Updated weights for policy 1, policy_version 1927330 (0.0010) [2023-12-27 05:24:18,148][105620] Updated weights for policy 1, policy_version 1927340 (0.0011) [2023-12-27 05:24:18,413][105692] Updated weights for policy 0, policy_version 1922644 (0.0005) [2023-12-27 05:24:18,472][105692] Updated weights for policy 0, policy_version 1922654 (0.0006) [2023-12-27 05:24:18,531][105692] Updated weights for policy 0, policy_version 1922664 (0.0008) [2023-12-27 05:24:18,919][105620] Updated weights for policy 1, policy_version 1927350 (0.0010) [2023-12-27 05:24:18,971][105620] Updated weights for policy 1, policy_version 1927360 (0.0010) [2023-12-27 05:24:19,018][105620] Updated weights for policy 1, policy_version 1927370 (0.0010) [2023-12-27 05:24:19,239][105692] Updated weights for policy 0, policy_version 1922674 (0.0007) [2023-12-27 05:24:19,295][105692] Updated weights for policy 0, policy_version 1922684 (0.0007) [2023-12-27 05:24:19,361][105692] Updated weights for policy 0, policy_version 1922694 (0.0008) [2023-12-27 05:24:19,416][105692] Updated weights for policy 0, policy_version 1922704 (0.0005) [2023-12-27 05:24:19,718][105620] Updated weights for policy 1, policy_version 1927380 (0.0010) [2023-12-27 05:24:19,782][105620] Updated weights for policy 1, policy_version 1927390 (0.0011) [2023-12-27 05:24:19,851][105620] Updated weights for policy 1, policy_version 1927400 (0.0009) [2023-12-27 05:24:20,043][105692] Updated weights for policy 0, policy_version 1922714 (0.0008) [2023-12-27 05:24:20,093][105692] Updated weights for policy 0, policy_version 1922724 (0.0006) [2023-12-27 05:24:20,163][105692] Updated weights for policy 0, policy_version 1922734 (0.0006) [2023-12-27 05:24:20,598][105620] Updated weights for policy 1, policy_version 1927410 (0.0011) [2023-12-27 05:24:20,664][105620] Updated weights for policy 1, policy_version 1927420 (0.0011) [2023-12-27 05:24:20,721][105620] Updated weights for policy 1, policy_version 1927430 (0.0010) [2023-12-27 05:24:20,774][105692] Updated weights for policy 0, policy_version 1922744 (0.0006) [2023-12-27 05:24:20,785][105620] Updated weights for policy 1, policy_version 1927440 (0.0011) [2023-12-27 05:24:20,837][105692] Updated weights for policy 0, policy_version 1922754 (0.0006) [2023-12-27 05:24:20,891][105692] Updated weights for policy 0, policy_version 1922764 (0.0006) [2023-12-27 05:24:21,062][104569] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 985792512. Throughput: 0: 9855.7, 1: 9909.2. Samples: 985776888. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:24:21,062][104569] Avg episode reward: [(0, '8897.221'), (1, '9254.529')] [2023-12-27 05:24:21,549][105620] Updated weights for policy 1, policy_version 1927450 (0.0008) [2023-12-27 05:24:21,582][105692] Updated weights for policy 0, policy_version 1922774 (0.0007) [2023-12-27 05:24:21,614][105620] Updated weights for policy 1, policy_version 1927460 (0.0009) [2023-12-27 05:24:21,653][105692] Updated weights for policy 0, policy_version 1922784 (0.0009) [2023-12-27 05:24:21,685][105620] Updated weights for policy 1, policy_version 1927470 (0.0006) [2023-12-27 05:24:21,718][105692] Updated weights for policy 0, policy_version 1922794 (0.0009) [2023-12-27 05:24:22,382][105620] Updated weights for policy 1, policy_version 1927480 (0.0006) [2023-12-27 05:24:22,441][105620] Updated weights for policy 1, policy_version 1927490 (0.0008) [2023-12-27 05:24:22,502][105620] Updated weights for policy 1, policy_version 1927500 (0.0009) [2023-12-27 05:24:22,504][105692] Updated weights for policy 0, policy_version 1922804 (0.0007) [2023-12-27 05:24:22,562][105692] Updated weights for policy 0, policy_version 1922814 (0.0008) [2023-12-27 05:24:22,610][105692] Updated weights for policy 0, policy_version 1922824 (0.0009) [2023-12-27 05:24:23,217][105620] Updated weights for policy 1, policy_version 1927510 (0.0006) [2023-12-27 05:24:23,272][105620] Updated weights for policy 1, policy_version 1927520 (0.0005) [2023-12-27 05:24:23,342][105620] Updated weights for policy 1, policy_version 1927530 (0.0006) [2023-12-27 05:24:23,433][105692] Updated weights for policy 0, policy_version 1922834 (0.0009) [2023-12-27 05:24:23,495][105692] Updated weights for policy 0, policy_version 1922844 (0.0008) [2023-12-27 05:24:23,559][105692] Updated weights for policy 0, policy_version 1922854 (0.0009) [2023-12-27 05:24:23,617][105692] Updated weights for policy 0, policy_version 1922864 (0.0011) [2023-12-27 05:24:24,046][105620] Updated weights for policy 1, policy_version 1927540 (0.0007) [2023-12-27 05:24:24,093][105620] Updated weights for policy 1, policy_version 1927550 (0.0008) [2023-12-27 05:24:24,139][105620] Updated weights for policy 1, policy_version 1927560 (0.0008) [2023-12-27 05:24:24,331][105692] Updated weights for policy 0, policy_version 1922874 (0.0005) [2023-12-27 05:24:24,392][105692] Updated weights for policy 0, policy_version 1922884 (0.0008) [2023-12-27 05:24:24,441][105692] Updated weights for policy 0, policy_version 1922894 (0.0005) [2023-12-27 05:24:24,784][105620] Updated weights for policy 1, policy_version 1927570 (0.0009) [2023-12-27 05:24:24,836][105620] Updated weights for policy 1, policy_version 1927580 (0.0009) [2023-12-27 05:24:24,886][105620] Updated weights for policy 1, policy_version 1927591 (0.0009) [2023-12-27 05:24:25,012][105692] Updated weights for policy 0, policy_version 1922904 (0.0008) [2023-12-27 05:24:25,059][105692] Updated weights for policy 0, policy_version 1922914 (0.0008) [2023-12-27 05:24:25,105][105692] Updated weights for policy 0, policy_version 1922924 (0.0010) [2023-12-27 05:24:25,701][105620] Updated weights for policy 1, policy_version 1927601 (0.0009) [2023-12-27 05:24:25,721][105692] Updated weights for policy 0, policy_version 1922934 (0.0007) [2023-12-27 05:24:25,752][105620] Updated weights for policy 1, policy_version 1927611 (0.0008) [2023-12-27 05:24:25,769][105692] Updated weights for policy 0, policy_version 1922944 (0.0005) [2023-12-27 05:24:25,804][105620] Updated weights for policy 1, policy_version 1927621 (0.0009) [2023-12-27 05:24:25,822][105692] Updated weights for policy 0, policy_version 1922954 (0.0005) [2023-12-27 05:24:25,860][105620] Updated weights for policy 1, policy_version 1927632 (0.0008) [2023-12-27 05:24:26,062][104569] Fps is (10 sec: 20480.8, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 985890816. Throughput: 0: 9890.3, 1: 9878.2. Samples: 985894272. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:24:26,062][104569] Avg episode reward: [(0, '8804.445'), (1, '9255.631')] [2023-12-27 05:24:26,344][105692] Updated weights for policy 0, policy_version 1922964 (0.0006) [2023-12-27 05:24:26,405][105692] Updated weights for policy 0, policy_version 1922974 (0.0008) [2023-12-27 05:24:26,463][105692] Updated weights for policy 0, policy_version 1922984 (0.0007) [2023-12-27 05:24:26,753][105620] Updated weights for policy 1, policy_version 1927642 (0.0010) [2023-12-27 05:24:26,807][105620] Updated weights for policy 1, policy_version 1927652 (0.0010) [2023-12-27 05:24:26,860][105620] Updated weights for policy 1, policy_version 1927662 (0.0010) [2023-12-27 05:24:27,011][105692] Updated weights for policy 0, policy_version 1922994 (0.0008) [2023-12-27 05:24:27,075][105692] Updated weights for policy 0, policy_version 1923004 (0.0009) [2023-12-27 05:24:27,124][105692] Updated weights for policy 0, policy_version 1923014 (0.0010) [2023-12-27 05:24:27,171][105692] Updated weights for policy 0, policy_version 1923024 (0.0009) [2023-12-27 05:24:27,677][105620] Updated weights for policy 1, policy_version 1927672 (0.0008) [2023-12-27 05:24:27,728][105620] Updated weights for policy 1, policy_version 1927682 (0.0008) [2023-12-27 05:24:27,779][105620] Updated weights for policy 1, policy_version 1927692 (0.0007) [2023-12-27 05:24:27,871][105692] Updated weights for policy 0, policy_version 1923034 (0.0005) [2023-12-27 05:24:27,953][105692] Updated weights for policy 0, policy_version 1923044 (0.0009) [2023-12-27 05:24:28,016][105692] Updated weights for policy 0, policy_version 1923054 (0.0008) [2023-12-27 05:24:28,584][105620] Updated weights for policy 1, policy_version 1927702 (0.0008) [2023-12-27 05:24:28,643][105620] Updated weights for policy 1, policy_version 1927712 (0.0009) [2023-12-27 05:24:28,707][105692] Updated weights for policy 0, policy_version 1923064 (0.0010) [2023-12-27 05:24:28,709][105620] Updated weights for policy 1, policy_version 1927722 (0.0006) [2023-12-27 05:24:28,768][105692] Updated weights for policy 0, policy_version 1923074 (0.0009) [2023-12-27 05:24:28,837][105692] Updated weights for policy 0, policy_version 1923084 (0.0006) [2023-12-27 05:24:29,480][105620] Updated weights for policy 1, policy_version 1927732 (0.0007) [2023-12-27 05:24:29,524][105692] Updated weights for policy 0, policy_version 1923094 (0.0007) [2023-12-27 05:24:29,540][105620] Updated weights for policy 1, policy_version 1927742 (0.0009) [2023-12-27 05:24:29,581][105692] Updated weights for policy 0, policy_version 1923104 (0.0009) [2023-12-27 05:24:29,606][105620] Updated weights for policy 1, policy_version 1927752 (0.0008) [2023-12-27 05:24:29,632][105692] Updated weights for policy 0, policy_version 1923114 (0.0006) [2023-12-27 05:24:30,340][105692] Updated weights for policy 0, policy_version 1923124 (0.0009) [2023-12-27 05:24:30,349][105620] Updated weights for policy 1, policy_version 1927762 (0.0009) [2023-12-27 05:24:30,390][105692] Updated weights for policy 0, policy_version 1923134 (0.0006) [2023-12-27 05:24:30,407][105620] Updated weights for policy 1, policy_version 1927772 (0.0010) [2023-12-27 05:24:30,442][105692] Updated weights for policy 0, policy_version 1923144 (0.0009) [2023-12-27 05:24:30,460][105620] Updated weights for policy 1, policy_version 1927782 (0.0008) [2023-12-27 05:24:30,511][105620] Updated weights for policy 1, policy_version 1927792 (0.0010) [2023-12-27 05:24:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.9, 300 sec: 19521.9). Total num frames: 985980928. Throughput: 0: 10032.1, 1: 9791.2. Samples: 985954076. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:24:31,062][104569] Avg episode reward: [(0, '8898.514'), (1, '9255.579')] [2023-12-27 05:24:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001923152_492396544.pth... [2023-12-27 05:24:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001927792_493584384.pth... [2023-12-27 05:24:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001926672_493297664.pth [2023-12-27 05:24:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001921968_492093440.pth [2023-12-27 05:24:31,120][105692] Updated weights for policy 0, policy_version 1923154 (0.0006) [2023-12-27 05:24:31,174][105692] Updated weights for policy 0, policy_version 1923164 (0.0007) [2023-12-27 05:24:31,211][105620] Updated weights for policy 1, policy_version 1927802 (0.0007) [2023-12-27 05:24:31,229][105692] Updated weights for policy 0, policy_version 1923175 (0.0009) [2023-12-27 05:24:31,271][105620] Updated weights for policy 1, policy_version 1927812 (0.0008) [2023-12-27 05:24:31,324][105620] Updated weights for policy 1, policy_version 1927822 (0.0010) [2023-12-27 05:24:31,992][105692] Updated weights for policy 0, policy_version 1923185 (0.0006) [2023-12-27 05:24:32,050][105692] Updated weights for policy 0, policy_version 1923195 (0.0007) [2023-12-27 05:24:32,056][105620] Updated weights for policy 1, policy_version 1927832 (0.0008) [2023-12-27 05:24:32,111][105620] Updated weights for policy 1, policy_version 1927842 (0.0007) [2023-12-27 05:24:32,111][105692] Updated weights for policy 0, policy_version 1923205 (0.0008) [2023-12-27 05:24:32,173][105620] Updated weights for policy 1, policy_version 1927852 (0.0010) [2023-12-27 05:24:32,173][105692] Updated weights for policy 0, policy_version 1923215 (0.0009) [2023-12-27 05:24:32,884][105620] Updated weights for policy 1, policy_version 1927862 (0.0007) [2023-12-27 05:24:32,910][105692] Updated weights for policy 0, policy_version 1923225 (0.0009) [2023-12-27 05:24:32,945][105620] Updated weights for policy 1, policy_version 1927872 (0.0005) [2023-12-27 05:24:32,964][105692] Updated weights for policy 0, policy_version 1923236 (0.0010) [2023-12-27 05:24:33,002][105620] Updated weights for policy 1, policy_version 1927882 (0.0005) [2023-12-27 05:24:33,017][105692] Updated weights for policy 0, policy_version 1923246 (0.0009) [2023-12-27 05:24:33,594][105620] Updated weights for policy 1, policy_version 1927892 (0.0008) [2023-12-27 05:24:33,656][105620] Updated weights for policy 1, policy_version 1927902 (0.0009) [2023-12-27 05:24:33,717][105620] Updated weights for policy 1, policy_version 1927912 (0.0009) [2023-12-27 05:24:33,824][105692] Updated weights for policy 0, policy_version 1923256 (0.0008) [2023-12-27 05:24:33,882][105692] Updated weights for policy 0, policy_version 1923266 (0.0009) [2023-12-27 05:24:33,939][105692] Updated weights for policy 0, policy_version 1923276 (0.0010) [2023-12-27 05:24:34,437][105620] Updated weights for policy 1, policy_version 1927922 (0.0008) [2023-12-27 05:24:34,503][105620] Updated weights for policy 1, policy_version 1927932 (0.0007) [2023-12-27 05:24:34,570][105620] Updated weights for policy 1, policy_version 1927942 (0.0011) [2023-12-27 05:24:34,635][105620] Updated weights for policy 1, policy_version 1927952 (0.0011) [2023-12-27 05:24:34,716][105692] Updated weights for policy 0, policy_version 1923286 (0.0008) [2023-12-27 05:24:34,776][105692] Updated weights for policy 0, policy_version 1923296 (0.0009) [2023-12-27 05:24:34,834][105692] Updated weights for policy 0, policy_version 1923306 (0.0008) [2023-12-27 05:24:35,306][105620] Updated weights for policy 1, policy_version 1927962 (0.0011) [2023-12-27 05:24:35,374][105620] Updated weights for policy 1, policy_version 1927972 (0.0011) [2023-12-27 05:24:35,427][105620] Updated weights for policy 1, policy_version 1927982 (0.0010) [2023-12-27 05:24:35,513][105692] Updated weights for policy 0, policy_version 1923316 (0.0007) [2023-12-27 05:24:35,567][105692] Updated weights for policy 0, policy_version 1923326 (0.0005) [2023-12-27 05:24:35,620][105692] Updated weights for policy 0, policy_version 1923336 (0.0005) [2023-12-27 05:24:36,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19797.3, 300 sec: 19522.0). Total num frames: 986079232. Throughput: 0: 10064.2, 1: 9739.4. Samples: 986070068. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:24:36,062][104569] Avg episode reward: [(0, '8440.785'), (1, '9162.063')] [2023-12-27 05:24:36,070][105620] Updated weights for policy 1, policy_version 1927992 (0.0006) [2023-12-27 05:24:36,134][105620] Updated weights for policy 1, policy_version 1928002 (0.0007) [2023-12-27 05:24:36,194][105620] Updated weights for policy 1, policy_version 1928012 (0.0006) [2023-12-27 05:24:36,371][105692] Updated weights for policy 0, policy_version 1923346 (0.0006) [2023-12-27 05:24:36,434][105692] Updated weights for policy 0, policy_version 1923356 (0.0011) [2023-12-27 05:24:36,494][105692] Updated weights for policy 0, policy_version 1923366 (0.0011) [2023-12-27 05:24:36,560][105692] Updated weights for policy 0, policy_version 1923376 (0.0010) [2023-12-27 05:24:36,849][105620] Updated weights for policy 1, policy_version 1928022 (0.0009) [2023-12-27 05:24:36,915][105620] Updated weights for policy 1, policy_version 1928032 (0.0011) [2023-12-27 05:24:36,989][105620] Updated weights for policy 1, policy_version 1928042 (0.0011) [2023-12-27 05:24:37,232][105692] Updated weights for policy 0, policy_version 1923386 (0.0010) [2023-12-27 05:24:37,297][105692] Updated weights for policy 0, policy_version 1923396 (0.0011) [2023-12-27 05:24:37,355][105692] Updated weights for policy 0, policy_version 1923406 (0.0008) [2023-12-27 05:24:37,709][105620] Updated weights for policy 1, policy_version 1928052 (0.0011) [2023-12-27 05:24:37,762][105620] Updated weights for policy 1, policy_version 1928062 (0.0010) [2023-12-27 05:24:37,817][105620] Updated weights for policy 1, policy_version 1928072 (0.0011) [2023-12-27 05:24:37,985][105692] Updated weights for policy 0, policy_version 1923416 (0.0006) [2023-12-27 05:24:38,042][105692] Updated weights for policy 0, policy_version 1923426 (0.0005) [2023-12-27 05:24:38,091][105692] Updated weights for policy 0, policy_version 1923436 (0.0006) [2023-12-27 05:24:38,432][105620] Updated weights for policy 1, policy_version 1928082 (0.0011) [2023-12-27 05:24:38,491][105620] Updated weights for policy 1, policy_version 1928092 (0.0008) [2023-12-27 05:24:38,554][105620] Updated weights for policy 1, policy_version 1928102 (0.0011) [2023-12-27 05:24:38,613][105620] Updated weights for policy 1, policy_version 1928112 (0.0011) [2023-12-27 05:24:38,689][105692] Updated weights for policy 0, policy_version 1923446 (0.0006) [2023-12-27 05:24:38,751][105692] Updated weights for policy 0, policy_version 1923456 (0.0010) [2023-12-27 05:24:38,816][105692] Updated weights for policy 0, policy_version 1923466 (0.0006) [2023-12-27 05:24:39,359][105620] Updated weights for policy 1, policy_version 1928122 (0.0011) [2023-12-27 05:24:39,424][105620] Updated weights for policy 1, policy_version 1928132 (0.0010) [2023-12-27 05:24:39,478][105692] Updated weights for policy 0, policy_version 1923476 (0.0008) [2023-12-27 05:24:39,491][105620] Updated weights for policy 1, policy_version 1928142 (0.0008) [2023-12-27 05:24:39,541][105692] Updated weights for policy 0, policy_version 1923486 (0.0010) [2023-12-27 05:24:39,611][105692] Updated weights for policy 0, policy_version 1923496 (0.0010) [2023-12-27 05:24:40,156][105620] Updated weights for policy 1, policy_version 1928152 (0.0009) [2023-12-27 05:24:40,218][105620] Updated weights for policy 1, policy_version 1928162 (0.0009) [2023-12-27 05:24:40,275][105620] Updated weights for policy 1, policy_version 1928172 (0.0009) [2023-12-27 05:24:40,430][105692] Updated weights for policy 0, policy_version 1923506 (0.0010) [2023-12-27 05:24:40,488][105692] Updated weights for policy 0, policy_version 1923516 (0.0009) [2023-12-27 05:24:40,551][105692] Updated weights for policy 0, policy_version 1923526 (0.0009) [2023-12-27 05:24:40,612][105692] Updated weights for policy 0, policy_version 1923536 (0.0008) [2023-12-27 05:24:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19797.4, 300 sec: 19577.5). Total num frames: 986177536. Throughput: 0: 9997.2, 1: 9738.2. Samples: 986189116. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:24:41,062][104569] Avg episode reward: [(0, '8170.798'), (1, '9162.059')] [2023-12-27 05:24:41,087][105620] Updated weights for policy 1, policy_version 1928182 (0.0010) [2023-12-27 05:24:41,156][105620] Updated weights for policy 1, policy_version 1928192 (0.0010) [2023-12-27 05:24:41,226][105620] Updated weights for policy 1, policy_version 1928202 (0.0009) [2023-12-27 05:24:41,410][105692] Updated weights for policy 0, policy_version 1923546 (0.0008) [2023-12-27 05:24:41,471][105692] Updated weights for policy 0, policy_version 1923556 (0.0011) [2023-12-27 05:24:41,532][105692] Updated weights for policy 0, policy_version 1923566 (0.0011) [2023-12-27 05:24:42,022][105620] Updated weights for policy 1, policy_version 1928212 (0.0010) [2023-12-27 05:24:42,088][105620] Updated weights for policy 1, policy_version 1928222 (0.0011) [2023-12-27 05:24:42,146][105620] Updated weights for policy 1, policy_version 1928232 (0.0010) [2023-12-27 05:24:42,284][105692] Updated weights for policy 0, policy_version 1923576 (0.0011) [2023-12-27 05:24:42,344][105692] Updated weights for policy 0, policy_version 1923586 (0.0011) [2023-12-27 05:24:42,404][105692] Updated weights for policy 0, policy_version 1923596 (0.0011) [2023-12-27 05:24:42,898][105620] Updated weights for policy 1, policy_version 1928242 (0.0010) [2023-12-27 05:24:42,955][105620] Updated weights for policy 1, policy_version 1928252 (0.0005) [2023-12-27 05:24:43,010][105620] Updated weights for policy 1, policy_version 1928262 (0.0005) [2023-12-27 05:24:43,067][105692] Updated weights for policy 0, policy_version 1923606 (0.0009) [2023-12-27 05:24:43,068][105620] Updated weights for policy 1, policy_version 1928272 (0.0008) [2023-12-27 05:24:43,133][105692] Updated weights for policy 0, policy_version 1923616 (0.0011) [2023-12-27 05:24:43,192][105692] Updated weights for policy 0, policy_version 1923626 (0.0010) [2023-12-27 05:24:43,799][105620] Updated weights for policy 1, policy_version 1928282 (0.0010) [2023-12-27 05:24:43,848][105620] Updated weights for policy 1, policy_version 1928292 (0.0010) [2023-12-27 05:24:43,875][105692] Updated weights for policy 0, policy_version 1923636 (0.0008) [2023-12-27 05:24:43,904][105620] Updated weights for policy 1, policy_version 1928302 (0.0011) [2023-12-27 05:24:43,934][105692] Updated weights for policy 0, policy_version 1923646 (0.0008) [2023-12-27 05:24:44,000][105692] Updated weights for policy 0, policy_version 1923656 (0.0010) [2023-12-27 05:24:44,671][105692] Updated weights for policy 0, policy_version 1923666 (0.0010) [2023-12-27 05:24:44,687][105620] Updated weights for policy 1, policy_version 1928312 (0.0010) [2023-12-27 05:24:44,724][105692] Updated weights for policy 0, policy_version 1923676 (0.0010) [2023-12-27 05:24:44,741][105620] Updated weights for policy 1, policy_version 1928322 (0.0011) [2023-12-27 05:24:44,781][105692] Updated weights for policy 0, policy_version 1923686 (0.0009) [2023-12-27 05:24:44,797][105620] Updated weights for policy 1, policy_version 1928332 (0.0011) [2023-12-27 05:24:44,833][105692] Updated weights for policy 0, policy_version 1923696 (0.0007) [2023-12-27 05:24:45,552][105620] Updated weights for policy 1, policy_version 1928342 (0.0011) [2023-12-27 05:24:45,610][105620] Updated weights for policy 1, policy_version 1928352 (0.0010) [2023-12-27 05:24:45,633][105692] Updated weights for policy 0, policy_version 1923706 (0.0006) [2023-12-27 05:24:45,655][105620] Updated weights for policy 1, policy_version 1928362 (0.0010) [2023-12-27 05:24:45,689][105692] Updated weights for policy 0, policy_version 1923716 (0.0007) [2023-12-27 05:24:45,742][105692] Updated weights for policy 0, policy_version 1923726 (0.0009) [2023-12-27 05:24:46,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19797.4, 300 sec: 19549.7). Total num frames: 986275840. Throughput: 0: 10016.9, 1: 9673.2. Samples: 986245012. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:24:46,062][104569] Avg episode reward: [(0, '8170.598'), (1, '9345.918')] [2023-12-27 05:24:46,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001923728_492544000.pth... [2023-12-27 05:24:46,066][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001928368_493731840.pth... [2023-12-27 05:24:46,070][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001922544_492240896.pth [2023-12-27 05:24:46,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001927248_493445120.pth [2023-12-27 05:24:46,403][105620] Updated weights for policy 1, policy_version 1928372 (0.0009) [2023-12-27 05:24:46,455][105620] Updated weights for policy 1, policy_version 1928382 (0.0008) [2023-12-27 05:24:46,500][105692] Updated weights for policy 0, policy_version 1923736 (0.0010) [2023-12-27 05:24:46,512][105620] Updated weights for policy 1, policy_version 1928392 (0.0005) [2023-12-27 05:24:46,562][105692] Updated weights for policy 0, policy_version 1923746 (0.0007) [2023-12-27 05:24:46,629][105692] Updated weights for policy 0, policy_version 1923756 (0.0009) [2023-12-27 05:24:47,097][105620] Updated weights for policy 1, policy_version 1928402 (0.0005) [2023-12-27 05:24:47,156][105620] Updated weights for policy 1, policy_version 1928412 (0.0005) [2023-12-27 05:24:47,219][105620] Updated weights for policy 1, policy_version 1928422 (0.0006) [2023-12-27 05:24:47,289][105620] Updated weights for policy 1, policy_version 1928432 (0.0006) [2023-12-27 05:24:47,328][105692] Updated weights for policy 0, policy_version 1923766 (0.0007) [2023-12-27 05:24:47,399][105692] Updated weights for policy 0, policy_version 1923776 (0.0005) [2023-12-27 05:24:47,460][105692] Updated weights for policy 0, policy_version 1923786 (0.0006) [2023-12-27 05:24:47,834][105620] Updated weights for policy 1, policy_version 1928442 (0.0005) [2023-12-27 05:24:47,904][105620] Updated weights for policy 1, policy_version 1928452 (0.0006) [2023-12-27 05:24:47,956][105620] Updated weights for policy 1, policy_version 1928462 (0.0008) [2023-12-27 05:24:48,087][105692] Updated weights for policy 0, policy_version 1923796 (0.0007) [2023-12-27 05:24:48,154][105692] Updated weights for policy 0, policy_version 1923806 (0.0009) [2023-12-27 05:24:48,211][105692] Updated weights for policy 0, policy_version 1923816 (0.0009) [2023-12-27 05:24:48,639][105620] Updated weights for policy 1, policy_version 1928472 (0.0008) [2023-12-27 05:24:48,700][105620] Updated weights for policy 1, policy_version 1928482 (0.0008) [2023-12-27 05:24:48,759][105620] Updated weights for policy 1, policy_version 1928492 (0.0008) [2023-12-27 05:24:48,944][105692] Updated weights for policy 0, policy_version 1923826 (0.0008) [2023-12-27 05:24:48,995][105692] Updated weights for policy 0, policy_version 1923836 (0.0008) [2023-12-27 05:24:49,042][105692] Updated weights for policy 0, policy_version 1923846 (0.0008) [2023-12-27 05:24:49,088][105692] Updated weights for policy 0, policy_version 1923856 (0.0008) [2023-12-27 05:24:49,521][105620] Updated weights for policy 1, policy_version 1928502 (0.0006) [2023-12-27 05:24:49,584][105620] Updated weights for policy 1, policy_version 1928512 (0.0006) [2023-12-27 05:24:49,646][105620] Updated weights for policy 1, policy_version 1928522 (0.0005) [2023-12-27 05:24:49,923][105692] Updated weights for policy 0, policy_version 1923866 (0.0008) [2023-12-27 05:24:49,984][105692] Updated weights for policy 0, policy_version 1923876 (0.0008) [2023-12-27 05:24:50,043][105692] Updated weights for policy 0, policy_version 1923886 (0.0009) [2023-12-27 05:24:50,255][105620] Updated weights for policy 1, policy_version 1928532 (0.0008) [2023-12-27 05:24:50,320][105620] Updated weights for policy 1, policy_version 1928542 (0.0006) [2023-12-27 05:24:50,386][105620] Updated weights for policy 1, policy_version 1928552 (0.0006) [2023-12-27 05:24:50,855][105692] Updated weights for policy 0, policy_version 1923896 (0.0009) [2023-12-27 05:24:50,923][105692] Updated weights for policy 0, policy_version 1923906 (0.0009) [2023-12-27 05:24:50,987][105692] Updated weights for policy 0, policy_version 1923916 (0.0008) [2023-12-27 05:24:51,004][105620] Updated weights for policy 1, policy_version 1928562 (0.0010) [2023-12-27 05:24:51,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 986374144. Throughput: 0: 9955.7, 1: 9720.0. Samples: 986363380. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:24:51,062][104569] Avg episode reward: [(0, '8077.938'), (1, '9345.925')] [2023-12-27 05:24:51,068][105620] Updated weights for policy 1, policy_version 1928572 (0.0011) [2023-12-27 05:24:51,131][105620] Updated weights for policy 1, policy_version 1928582 (0.0011) [2023-12-27 05:24:51,192][105620] Updated weights for policy 1, policy_version 1928592 (0.0011) [2023-12-27 05:24:51,772][105692] Updated weights for policy 0, policy_version 1923926 (0.0007) [2023-12-27 05:24:51,836][105692] Updated weights for policy 0, policy_version 1923936 (0.0005) [2023-12-27 05:24:51,905][105692] Updated weights for policy 0, policy_version 1923946 (0.0006) [2023-12-27 05:24:51,941][105620] Updated weights for policy 1, policy_version 1928602 (0.0008) [2023-12-27 05:24:51,993][105620] Updated weights for policy 1, policy_version 1928612 (0.0010) [2023-12-27 05:24:52,051][105620] Updated weights for policy 1, policy_version 1928622 (0.0007) [2023-12-27 05:24:52,548][105692] Updated weights for policy 0, policy_version 1923956 (0.0006) [2023-12-27 05:24:52,604][105692] Updated weights for policy 0, policy_version 1923966 (0.0008) [2023-12-27 05:24:52,665][105692] Updated weights for policy 0, policy_version 1923976 (0.0007) [2023-12-27 05:24:52,874][105620] Updated weights for policy 1, policy_version 1928632 (0.0009) [2023-12-27 05:24:52,932][105620] Updated weights for policy 1, policy_version 1928642 (0.0010) [2023-12-27 05:24:52,990][105620] Updated weights for policy 1, policy_version 1928652 (0.0009) [2023-12-27 05:24:53,280][105692] Updated weights for policy 0, policy_version 1923986 (0.0009) [2023-12-27 05:24:53,338][105692] Updated weights for policy 0, policy_version 1923996 (0.0009) [2023-12-27 05:24:53,398][105692] Updated weights for policy 0, policy_version 1924006 (0.0008) [2023-12-27 05:24:53,457][105692] Updated weights for policy 0, policy_version 1924016 (0.0006) [2023-12-27 05:24:53,701][105620] Updated weights for policy 1, policy_version 1928662 (0.0007) [2023-12-27 05:24:53,753][105620] Updated weights for policy 1, policy_version 1928672 (0.0005) [2023-12-27 05:24:53,809][105620] Updated weights for policy 1, policy_version 1928682 (0.0007) [2023-12-27 05:24:54,234][105692] Updated weights for policy 0, policy_version 1924026 (0.0008) [2023-12-27 05:24:54,293][105692] Updated weights for policy 0, policy_version 1924036 (0.0010) [2023-12-27 05:24:54,345][105692] Updated weights for policy 0, policy_version 1924046 (0.0009) [2023-12-27 05:24:54,518][105620] Updated weights for policy 1, policy_version 1928692 (0.0007) [2023-12-27 05:24:54,579][105620] Updated weights for policy 1, policy_version 1928702 (0.0009) [2023-12-27 05:24:54,630][105620] Updated weights for policy 1, policy_version 1928712 (0.0009) [2023-12-27 05:24:55,098][105692] Updated weights for policy 0, policy_version 1924056 (0.0009) [2023-12-27 05:24:55,151][105692] Updated weights for policy 0, policy_version 1924066 (0.0008) [2023-12-27 05:24:55,211][105692] Updated weights for policy 0, policy_version 1924076 (0.0007) [2023-12-27 05:24:55,383][105620] Updated weights for policy 1, policy_version 1928722 (0.0008) [2023-12-27 05:24:55,446][105620] Updated weights for policy 1, policy_version 1928732 (0.0006) [2023-12-27 05:24:55,497][105620] Updated weights for policy 1, policy_version 1928742 (0.0009) [2023-12-27 05:24:55,544][105620] Updated weights for policy 1, policy_version 1928752 (0.0009) [2023-12-27 05:24:55,993][105692] Updated weights for policy 0, policy_version 1924086 (0.0009) [2023-12-27 05:24:56,055][105692] Updated weights for policy 0, policy_version 1924096 (0.0010) [2023-12-27 05:24:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19522.0). Total num frames: 986464256. Throughput: 0: 9958.4, 1: 9652.2. Samples: 986478044. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:24:56,062][104569] Avg episode reward: [(0, '8168.425'), (1, '9345.898')] [2023-12-27 05:24:56,113][105692] Updated weights for policy 0, policy_version 1924106 (0.0008) [2023-12-27 05:24:56,172][105620] Updated weights for policy 1, policy_version 1928762 (0.0007) [2023-12-27 05:24:56,220][105620] Updated weights for policy 1, policy_version 1928772 (0.0009) [2023-12-27 05:24:56,280][105620] Updated weights for policy 1, policy_version 1928782 (0.0009) [2023-12-27 05:24:56,872][105692] Updated weights for policy 0, policy_version 1924116 (0.0008) [2023-12-27 05:24:56,922][105692] Updated weights for policy 0, policy_version 1924126 (0.0009) [2023-12-27 05:24:56,979][105692] Updated weights for policy 0, policy_version 1924136 (0.0008) [2023-12-27 05:24:57,072][105620] Updated weights for policy 1, policy_version 1928792 (0.0006) [2023-12-27 05:24:57,136][105620] Updated weights for policy 1, policy_version 1928802 (0.0005) [2023-12-27 05:24:57,209][105620] Updated weights for policy 1, policy_version 1928812 (0.0005) [2023-12-27 05:24:57,701][105620] Updated weights for policy 1, policy_version 1928822 (0.0005) [2023-12-27 05:24:57,714][105692] Updated weights for policy 0, policy_version 1924146 (0.0006) [2023-12-27 05:24:57,759][105620] Updated weights for policy 1, policy_version 1928832 (0.0006) [2023-12-27 05:24:57,764][105692] Updated weights for policy 0, policy_version 1924156 (0.0008) [2023-12-27 05:24:57,807][105692] Updated weights for policy 0, policy_version 1924166 (0.0005) [2023-12-27 05:24:57,815][105620] Updated weights for policy 1, policy_version 1928842 (0.0010) [2023-12-27 05:24:57,857][105692] Updated weights for policy 0, policy_version 1924176 (0.0005) [2023-12-27 05:24:58,485][105692] Updated weights for policy 0, policy_version 1924186 (0.0009) [2023-12-27 05:24:58,544][105692] Updated weights for policy 0, policy_version 1924196 (0.0011) [2023-12-27 05:24:58,574][105620] Updated weights for policy 1, policy_version 1928852 (0.0007) [2023-12-27 05:24:58,604][105692] Updated weights for policy 0, policy_version 1924206 (0.0010) [2023-12-27 05:24:58,635][105620] Updated weights for policy 1, policy_version 1928862 (0.0008) [2023-12-27 05:24:58,695][105620] Updated weights for policy 1, policy_version 1928872 (0.0008) [2023-12-27 05:24:59,448][105692] Updated weights for policy 0, policy_version 1924216 (0.0008) [2023-12-27 05:24:59,507][105692] Updated weights for policy 0, policy_version 1924226 (0.0008) [2023-12-27 05:24:59,530][105620] Updated weights for policy 1, policy_version 1928882 (0.0008) [2023-12-27 05:24:59,565][105692] Updated weights for policy 0, policy_version 1924236 (0.0007) [2023-12-27 05:24:59,591][105620] Updated weights for policy 1, policy_version 1928892 (0.0007) [2023-12-27 05:24:59,652][105620] Updated weights for policy 1, policy_version 1928902 (0.0008) [2023-12-27 05:24:59,710][105620] Updated weights for policy 1, policy_version 1928912 (0.0009) [2023-12-27 05:25:00,290][105692] Updated weights for policy 0, policy_version 1924246 (0.0006) [2023-12-27 05:25:00,352][105692] Updated weights for policy 0, policy_version 1924256 (0.0005) [2023-12-27 05:25:00,419][105692] Updated weights for policy 0, policy_version 1924266 (0.0008) [2023-12-27 05:25:00,433][105620] Updated weights for policy 1, policy_version 1928922 (0.0008) [2023-12-27 05:25:00,488][105620] Updated weights for policy 1, policy_version 1928932 (0.0008) [2023-12-27 05:25:00,554][105620] Updated weights for policy 1, policy_version 1928942 (0.0006) [2023-12-27 05:25:01,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19521.9). Total num frames: 986562560. Throughput: 0: 9972.3, 1: 9685.1. Samples: 986537256. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:25:01,063][104569] Avg episode reward: [(0, '8170.829'), (1, '9163.858')] [2023-12-27 05:25:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001928944_493879296.pth... [2023-12-27 05:25:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001927792_493584384.pth [2023-12-27 05:25:01,074][105692] Updated weights for policy 0, policy_version 1924276 (0.0008) [2023-12-27 05:25:01,134][105692] Updated weights for policy 0, policy_version 1924286 (0.0006) [2023-12-27 05:25:01,175][105620] Updated weights for policy 1, policy_version 1928952 (0.0007) [2023-12-27 05:25:01,197][105692] Updated weights for policy 0, policy_version 1924296 (0.0008) [2023-12-27 05:25:01,229][105620] Updated weights for policy 1, policy_version 1928962 (0.0005) [2023-12-27 05:25:01,249][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001924304_492691456.pth... [2023-12-27 05:25:01,254][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001923152_492396544.pth [2023-12-27 05:25:01,284][105620] Updated weights for policy 1, policy_version 1928972 (0.0007) [2023-12-27 05:25:01,914][105620] Updated weights for policy 1, policy_version 1928982 (0.0005) [2023-12-27 05:25:01,969][105692] Updated weights for policy 0, policy_version 1924306 (0.0009) [2023-12-27 05:25:01,972][105620] Updated weights for policy 1, policy_version 1928992 (0.0006) [2023-12-27 05:25:02,028][105692] Updated weights for policy 0, policy_version 1924316 (0.0006) [2023-12-27 05:25:02,033][105620] Updated weights for policy 1, policy_version 1929002 (0.0006) [2023-12-27 05:25:02,089][105692] Updated weights for policy 0, policy_version 1924326 (0.0010) [2023-12-27 05:25:02,622][105620] Updated weights for policy 1, policy_version 1929012 (0.0005) [2023-12-27 05:25:02,678][105620] Updated weights for policy 1, policy_version 1929022 (0.0008) [2023-12-27 05:25:02,740][105620] Updated weights for policy 1, policy_version 1929032 (0.0009) [2023-12-27 05:25:02,895][105692] Updated weights for policy 0, policy_version 1924337 (0.0010) [2023-12-27 05:25:02,944][105692] Updated weights for policy 0, policy_version 1924347 (0.0009) [2023-12-27 05:25:02,990][105692] Updated weights for policy 0, policy_version 1924357 (0.0009) [2023-12-27 05:25:03,044][105692] Updated weights for policy 0, policy_version 1924367 (0.0009) [2023-12-27 05:25:03,452][105620] Updated weights for policy 1, policy_version 1929042 (0.0008) [2023-12-27 05:25:03,503][105620] Updated weights for policy 1, policy_version 1929052 (0.0007) [2023-12-27 05:25:03,550][105620] Updated weights for policy 1, policy_version 1929062 (0.0008) [2023-12-27 05:25:03,600][105620] Updated weights for policy 1, policy_version 1929072 (0.0010) [2023-12-27 05:25:03,827][105692] Updated weights for policy 0, policy_version 1924377 (0.0009) [2023-12-27 05:25:03,888][105692] Updated weights for policy 0, policy_version 1924387 (0.0008) [2023-12-27 05:25:03,945][105692] Updated weights for policy 0, policy_version 1924398 (0.0010) [2023-12-27 05:25:04,268][105620] Updated weights for policy 1, policy_version 1929082 (0.0010) [2023-12-27 05:25:04,324][105620] Updated weights for policy 1, policy_version 1929092 (0.0010) [2023-12-27 05:25:04,380][105620] Updated weights for policy 1, policy_version 1929102 (0.0010) [2023-12-27 05:25:04,781][105692] Updated weights for policy 0, policy_version 1924408 (0.0009) [2023-12-27 05:25:04,834][105692] Updated weights for policy 0, policy_version 1924418 (0.0008) [2023-12-27 05:25:04,883][105692] Updated weights for policy 0, policy_version 1924428 (0.0008) [2023-12-27 05:25:05,133][105620] Updated weights for policy 1, policy_version 1929112 (0.0010) [2023-12-27 05:25:05,187][105620] Updated weights for policy 1, policy_version 1929122 (0.0008) [2023-12-27 05:25:05,259][105620] Updated weights for policy 1, policy_version 1929132 (0.0005) [2023-12-27 05:25:05,622][105692] Updated weights for policy 0, policy_version 1924438 (0.0008) [2023-12-27 05:25:05,685][105692] Updated weights for policy 0, policy_version 1924448 (0.0009) [2023-12-27 05:25:05,740][105692] Updated weights for policy 0, policy_version 1924458 (0.0009) [2023-12-27 05:25:05,925][105620] Updated weights for policy 1, policy_version 1929142 (0.0008) [2023-12-27 05:25:05,991][105620] Updated weights for policy 1, policy_version 1929152 (0.0006) [2023-12-27 05:25:06,053][105620] Updated weights for policy 1, policy_version 1929162 (0.0006) [2023-12-27 05:25:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 986660864. Throughput: 0: 9716.6, 1: 9757.5. Samples: 986653224. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:25:06,062][104569] Avg episode reward: [(0, '8445.246'), (1, '9071.468')] [2023-12-27 05:25:06,587][105692] Updated weights for policy 0, policy_version 1924468 (0.0009) [2023-12-27 05:25:06,637][105692] Updated weights for policy 0, policy_version 1924478 (0.0008) [2023-12-27 05:25:06,701][105692] Updated weights for policy 0, policy_version 1924488 (0.0008) [2023-12-27 05:25:06,746][105620] Updated weights for policy 1, policy_version 1929172 (0.0008) [2023-12-27 05:25:06,804][105620] Updated weights for policy 1, policy_version 1929182 (0.0010) [2023-12-27 05:25:06,873][105620] Updated weights for policy 1, policy_version 1929192 (0.0011) [2023-12-27 05:25:07,406][105692] Updated weights for policy 0, policy_version 1924498 (0.0008) [2023-12-27 05:25:07,462][105692] Updated weights for policy 0, policy_version 1924508 (0.0008) [2023-12-27 05:25:07,519][105692] Updated weights for policy 0, policy_version 1924518 (0.0008) [2023-12-27 05:25:07,522][105620] Updated weights for policy 1, policy_version 1929202 (0.0010) [2023-12-27 05:25:07,580][105692] Updated weights for policy 0, policy_version 1924528 (0.0007) [2023-12-27 05:25:07,588][105620] Updated weights for policy 1, policy_version 1929212 (0.0008) [2023-12-27 05:25:07,652][105620] Updated weights for policy 1, policy_version 1929222 (0.0007) [2023-12-27 05:25:07,708][105620] Updated weights for policy 1, policy_version 1929232 (0.0006) [2023-12-27 05:25:08,179][105692] Updated weights for policy 0, policy_version 1924538 (0.0005) [2023-12-27 05:25:08,245][105692] Updated weights for policy 0, policy_version 1924548 (0.0005) [2023-12-27 05:25:08,319][105692] Updated weights for policy 0, policy_version 1924558 (0.0006) [2023-12-27 05:25:08,345][105620] Updated weights for policy 1, policy_version 1929242 (0.0011) [2023-12-27 05:25:08,414][105620] Updated weights for policy 1, policy_version 1929252 (0.0011) [2023-12-27 05:25:08,469][105620] Updated weights for policy 1, policy_version 1929262 (0.0010) [2023-12-27 05:25:08,877][105692] Updated weights for policy 0, policy_version 1924568 (0.0010) [2023-12-27 05:25:08,942][105692] Updated weights for policy 0, policy_version 1924578 (0.0010) [2023-12-27 05:25:08,994][105692] Updated weights for policy 0, policy_version 1924588 (0.0011) [2023-12-27 05:25:09,217][105620] Updated weights for policy 1, policy_version 1929272 (0.0010) [2023-12-27 05:25:09,272][105620] Updated weights for policy 1, policy_version 1929282 (0.0011) [2023-12-27 05:25:09,334][105620] Updated weights for policy 1, policy_version 1929292 (0.0010) [2023-12-27 05:25:09,686][105692] Updated weights for policy 0, policy_version 1924598 (0.0011) [2023-12-27 05:25:09,752][105692] Updated weights for policy 0, policy_version 1924608 (0.0008) [2023-12-27 05:25:09,820][105692] Updated weights for policy 0, policy_version 1924618 (0.0009) [2023-12-27 05:25:10,129][105620] Updated weights for policy 1, policy_version 1929302 (0.0010) [2023-12-27 05:25:10,198][105620] Updated weights for policy 1, policy_version 1929312 (0.0011) [2023-12-27 05:25:10,264][105620] Updated weights for policy 1, policy_version 1929322 (0.0011) [2023-12-27 05:25:10,564][105692] Updated weights for policy 0, policy_version 1924628 (0.0011) [2023-12-27 05:25:10,616][105692] Updated weights for policy 0, policy_version 1924638 (0.0010) [2023-12-27 05:25:10,669][105692] Updated weights for policy 0, policy_version 1924648 (0.0010) [2023-12-27 05:25:11,001][105620] Updated weights for policy 1, policy_version 1929332 (0.0009) [2023-12-27 05:25:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 986759168. Throughput: 0: 9687.0, 1: 9789.1. Samples: 986770700. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:25:11,063][104569] Avg episode reward: [(0, '8716.231'), (1, '9162.681')] [2023-12-27 05:25:11,073][105620] Updated weights for policy 1, policy_version 1929342 (0.0008) [2023-12-27 05:25:11,136][105620] Updated weights for policy 1, policy_version 1929352 (0.0009) [2023-12-27 05:25:11,490][105692] Updated weights for policy 0, policy_version 1924658 (0.0010) [2023-12-27 05:25:11,553][105692] Updated weights for policy 0, policy_version 1924668 (0.0009) [2023-12-27 05:25:11,613][105692] Updated weights for policy 0, policy_version 1924678 (0.0009) [2023-12-27 05:25:11,681][105692] Updated weights for policy 0, policy_version 1924688 (0.0009) [2023-12-27 05:25:11,840][105620] Updated weights for policy 1, policy_version 1929362 (0.0009) [2023-12-27 05:25:11,906][105620] Updated weights for policy 1, policy_version 1929372 (0.0009) [2023-12-27 05:25:11,970][105620] Updated weights for policy 1, policy_version 1929382 (0.0008) [2023-12-27 05:25:12,033][105620] Updated weights for policy 1, policy_version 1929392 (0.0009) [2023-12-27 05:25:12,524][105692] Updated weights for policy 0, policy_version 1924698 (0.0009) [2023-12-27 05:25:12,588][105692] Updated weights for policy 0, policy_version 1924708 (0.0010) [2023-12-27 05:25:12,651][105692] Updated weights for policy 0, policy_version 1924718 (0.0009) [2023-12-27 05:25:12,754][105620] Updated weights for policy 1, policy_version 1929402 (0.0010) [2023-12-27 05:25:12,802][105620] Updated weights for policy 1, policy_version 1929412 (0.0011) [2023-12-27 05:25:12,851][105620] Updated weights for policy 1, policy_version 1929422 (0.0010) [2023-12-27 05:25:13,412][105692] Updated weights for policy 0, policy_version 1924728 (0.0008) [2023-12-27 05:25:13,476][105692] Updated weights for policy 0, policy_version 1924738 (0.0006) [2023-12-27 05:25:13,540][105692] Updated weights for policy 0, policy_version 1924748 (0.0008) [2023-12-27 05:25:13,619][105620] Updated weights for policy 1, policy_version 1929432 (0.0007) [2023-12-27 05:25:13,678][105620] Updated weights for policy 1, policy_version 1929442 (0.0011) [2023-12-27 05:25:13,736][105620] Updated weights for policy 1, policy_version 1929452 (0.0009) [2023-12-27 05:25:14,157][105692] Updated weights for policy 0, policy_version 1924758 (0.0009) [2023-12-27 05:25:14,219][105692] Updated weights for policy 0, policy_version 1924768 (0.0008) [2023-12-27 05:25:14,276][105692] Updated weights for policy 0, policy_version 1924778 (0.0005) [2023-12-27 05:25:14,339][105620] Updated weights for policy 1, policy_version 1929462 (0.0007) [2023-12-27 05:25:14,395][105620] Updated weights for policy 1, policy_version 1929472 (0.0009) [2023-12-27 05:25:14,455][105620] Updated weights for policy 1, policy_version 1929482 (0.0008) [2023-12-27 05:25:14,957][105692] Updated weights for policy 0, policy_version 1924788 (0.0010) [2023-12-27 05:25:15,021][105692] Updated weights for policy 0, policy_version 1924798 (0.0011) [2023-12-27 05:25:15,082][105692] Updated weights for policy 0, policy_version 1924808 (0.0011) [2023-12-27 05:25:15,121][105620] Updated weights for policy 1, policy_version 1929492 (0.0009) [2023-12-27 05:25:15,182][105620] Updated weights for policy 1, policy_version 1929502 (0.0011) [2023-12-27 05:25:15,247][105620] Updated weights for policy 1, policy_version 1929512 (0.0011) [2023-12-27 05:25:15,786][105692] Updated weights for policy 0, policy_version 1924818 (0.0011) [2023-12-27 05:25:15,846][105692] Updated weights for policy 0, policy_version 1924828 (0.0010) [2023-12-27 05:25:15,908][105692] Updated weights for policy 0, policy_version 1924838 (0.0010) [2023-12-27 05:25:15,963][105620] Updated weights for policy 1, policy_version 1929522 (0.0010) [2023-12-27 05:25:15,965][105692] Updated weights for policy 0, policy_version 1924848 (0.0008) [2023-12-27 05:25:16,025][105620] Updated weights for policy 1, policy_version 1929532 (0.0008) [2023-12-27 05:25:16,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.3, 300 sec: 19522.0). Total num frames: 986857472. Throughput: 0: 9535.3, 1: 9854.1. Samples: 986826600. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:25:16,063][104569] Avg episode reward: [(0, '8180.333'), (1, '9162.582')] [2023-12-27 05:25:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001924848_492830720.pth... [2023-12-27 05:25:16,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001923728_492544000.pth [2023-12-27 05:25:16,074][105620] Updated weights for policy 1, policy_version 1929542 (0.0008) [2023-12-27 05:25:16,137][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001929552_494034944.pth... [2023-12-27 05:25:16,138][105620] Updated weights for policy 1, policy_version 1929552 (0.0008) [2023-12-27 05:25:16,142][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001928368_493731840.pth [2023-12-27 05:25:16,701][105692] Updated weights for policy 0, policy_version 1924858 (0.0010) [2023-12-27 05:25:16,748][105692] Updated weights for policy 0, policy_version 1924868 (0.0010) [2023-12-27 05:25:16,807][105692] Updated weights for policy 0, policy_version 1924878 (0.0010) [2023-12-27 05:25:16,842][105620] Updated weights for policy 1, policy_version 1929562 (0.0011) [2023-12-27 05:25:16,887][105620] Updated weights for policy 1, policy_version 1929572 (0.0010) [2023-12-27 05:25:16,939][105620] Updated weights for policy 1, policy_version 1929582 (0.0010) [2023-12-27 05:25:17,553][105692] Updated weights for policy 0, policy_version 1924888 (0.0011) [2023-12-27 05:25:17,575][105620] Updated weights for policy 1, policy_version 1929592 (0.0007) [2023-12-27 05:25:17,610][105692] Updated weights for policy 0, policy_version 1924898 (0.0009) [2023-12-27 05:25:17,634][105620] Updated weights for policy 1, policy_version 1929602 (0.0006) [2023-12-27 05:25:17,673][105692] Updated weights for policy 0, policy_version 1924908 (0.0010) [2023-12-27 05:25:17,689][105620] Updated weights for policy 1, policy_version 1929612 (0.0006) [2023-12-27 05:25:18,265][105692] Updated weights for policy 0, policy_version 1924918 (0.0007) [2023-12-27 05:25:18,310][105692] Updated weights for policy 0, policy_version 1924928 (0.0005) [2023-12-27 05:25:18,372][105692] Updated weights for policy 0, policy_version 1924938 (0.0008) [2023-12-27 05:25:18,398][105620] Updated weights for policy 1, policy_version 1929622 (0.0008) [2023-12-27 05:25:18,465][105620] Updated weights for policy 1, policy_version 1929632 (0.0008) [2023-12-27 05:25:18,535][105620] Updated weights for policy 1, policy_version 1929642 (0.0011) [2023-12-27 05:25:19,023][105692] Updated weights for policy 0, policy_version 1924948 (0.0007) [2023-12-27 05:25:19,086][105692] Updated weights for policy 0, policy_version 1924958 (0.0011) [2023-12-27 05:25:19,144][105692] Updated weights for policy 0, policy_version 1924968 (0.0010) [2023-12-27 05:25:19,257][105620] Updated weights for policy 1, policy_version 1929652 (0.0011) [2023-12-27 05:25:19,323][105620] Updated weights for policy 1, policy_version 1929662 (0.0012) [2023-12-27 05:25:19,383][105620] Updated weights for policy 1, policy_version 1929672 (0.0007) [2023-12-27 05:25:19,942][105692] Updated weights for policy 0, policy_version 1924978 (0.0010) [2023-12-27 05:25:20,004][105692] Updated weights for policy 0, policy_version 1924988 (0.0008) [2023-12-27 05:25:20,058][105620] Updated weights for policy 1, policy_version 1929682 (0.0009) [2023-12-27 05:25:20,067][105692] Updated weights for policy 0, policy_version 1924998 (0.0008) [2023-12-27 05:25:20,110][105620] Updated weights for policy 1, policy_version 1929692 (0.0009) [2023-12-27 05:25:20,127][105692] Updated weights for policy 0, policy_version 1925008 (0.0008) [2023-12-27 05:25:20,168][105620] Updated weights for policy 1, policy_version 1929702 (0.0009) [2023-12-27 05:25:20,227][105620] Updated weights for policy 1, policy_version 1929712 (0.0011) [2023-12-27 05:25:20,886][105692] Updated weights for policy 0, policy_version 1925018 (0.0011) [2023-12-27 05:25:20,942][105692] Updated weights for policy 0, policy_version 1925028 (0.0011) [2023-12-27 05:25:20,984][105620] Updated weights for policy 1, policy_version 1929722 (0.0010) [2023-12-27 05:25:21,003][105692] Updated weights for policy 0, policy_version 1925038 (0.0010) [2023-12-27 05:25:21,052][105620] Updated weights for policy 1, policy_version 1929732 (0.0007) [2023-12-27 05:25:21,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 986955776. Throughput: 0: 9603.1, 1: 9876.0. Samples: 986946628. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:25:21,062][104569] Avg episode reward: [(0, '7996.457'), (1, '9162.378')] [2023-12-27 05:25:21,119][105620] Updated weights for policy 1, policy_version 1929742 (0.0009) [2023-12-27 05:25:21,817][105692] Updated weights for policy 0, policy_version 1925048 (0.0009) [2023-12-27 05:25:21,877][105692] Updated weights for policy 0, policy_version 1925058 (0.0008) [2023-12-27 05:25:21,915][105620] Updated weights for policy 1, policy_version 1929752 (0.0007) [2023-12-27 05:25:21,937][105692] Updated weights for policy 0, policy_version 1925068 (0.0008) [2023-12-27 05:25:21,979][105620] Updated weights for policy 1, policy_version 1929762 (0.0007) [2023-12-27 05:25:22,052][105620] Updated weights for policy 1, policy_version 1929772 (0.0006) [2023-12-27 05:25:22,686][105620] Updated weights for policy 1, policy_version 1929782 (0.0007) [2023-12-27 05:25:22,742][105692] Updated weights for policy 0, policy_version 1925078 (0.0008) [2023-12-27 05:25:22,748][105620] Updated weights for policy 1, policy_version 1929792 (0.0006) [2023-12-27 05:25:22,801][105692] Updated weights for policy 0, policy_version 1925088 (0.0008) [2023-12-27 05:25:22,811][105620] Updated weights for policy 1, policy_version 1929802 (0.0006) [2023-12-27 05:25:22,868][105692] Updated weights for policy 0, policy_version 1925098 (0.0008) [2023-12-27 05:25:23,433][105620] Updated weights for policy 1, policy_version 1929812 (0.0007) [2023-12-27 05:25:23,497][105620] Updated weights for policy 1, policy_version 1929822 (0.0005) [2023-12-27 05:25:23,560][105620] Updated weights for policy 1, policy_version 1929832 (0.0005) [2023-12-27 05:25:23,695][105692] Updated weights for policy 0, policy_version 1925108 (0.0009) [2023-12-27 05:25:23,752][105692] Updated weights for policy 0, policy_version 1925118 (0.0009) [2023-12-27 05:25:23,811][105692] Updated weights for policy 0, policy_version 1925128 (0.0009) [2023-12-27 05:25:24,219][105620] Updated weights for policy 1, policy_version 1929842 (0.0007) [2023-12-27 05:25:24,273][105620] Updated weights for policy 1, policy_version 1929852 (0.0009) [2023-12-27 05:25:24,334][105620] Updated weights for policy 1, policy_version 1929862 (0.0008) [2023-12-27 05:25:24,383][105620] Updated weights for policy 1, policy_version 1929872 (0.0008) [2023-12-27 05:25:24,585][105692] Updated weights for policy 0, policy_version 1925138 (0.0008) [2023-12-27 05:25:24,644][105692] Updated weights for policy 0, policy_version 1925148 (0.0007) [2023-12-27 05:25:24,697][105692] Updated weights for policy 0, policy_version 1925158 (0.0009) [2023-12-27 05:25:25,124][105620] Updated weights for policy 1, policy_version 1929882 (0.0005) [2023-12-27 05:25:25,179][105620] Updated weights for policy 1, policy_version 1929892 (0.0006) [2023-12-27 05:25:25,241][105620] Updated weights for policy 1, policy_version 1929902 (0.0007) [2023-12-27 05:25:25,320][105692] Updated weights for policy 0, policy_version 1925169 (0.0009) [2023-12-27 05:25:25,393][105692] Updated weights for policy 0, policy_version 1925179 (0.0008) [2023-12-27 05:25:25,452][105692] Updated weights for policy 0, policy_version 1925189 (0.0010) [2023-12-27 05:25:25,509][105692] Updated weights for policy 0, policy_version 1925199 (0.0009) [2023-12-27 05:25:25,974][105620] Updated weights for policy 1, policy_version 1929912 (0.0010) [2023-12-27 05:25:26,028][105620] Updated weights for policy 1, policy_version 1929923 (0.0010) [2023-12-27 05:25:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19522.0). Total num frames: 987045888. Throughput: 0: 9499.9, 1: 9847.7. Samples: 987059760. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:25:26,062][104569] Avg episode reward: [(0, '8260.263'), (1, '9345.833')] [2023-12-27 05:25:26,088][105620] Updated weights for policy 1, policy_version 1929934 (0.0010) [2023-12-27 05:25:26,113][105692] Updated weights for policy 0, policy_version 1925209 (0.0010) [2023-12-27 05:25:26,174][105692] Updated weights for policy 0, policy_version 1925219 (0.0009) [2023-12-27 05:25:26,233][105692] Updated weights for policy 0, policy_version 1925229 (0.0009) [2023-12-27 05:25:26,837][105620] Updated weights for policy 1, policy_version 1929944 (0.0009) [2023-12-27 05:25:26,894][105620] Updated weights for policy 1, policy_version 1929954 (0.0010) [2023-12-27 05:25:26,917][105692] Updated weights for policy 0, policy_version 1925239 (0.0007) [2023-12-27 05:25:26,951][105620] Updated weights for policy 1, policy_version 1929964 (0.0010) [2023-12-27 05:25:26,969][105692] Updated weights for policy 0, policy_version 1925249 (0.0005) [2023-12-27 05:25:27,022][105692] Updated weights for policy 0, policy_version 1925259 (0.0008) [2023-12-27 05:25:27,712][105620] Updated weights for policy 1, policy_version 1929974 (0.0010) [2023-12-27 05:25:27,773][105620] Updated weights for policy 1, policy_version 1929984 (0.0010) [2023-12-27 05:25:27,810][105692] Updated weights for policy 0, policy_version 1925269 (0.0007) [2023-12-27 05:25:27,834][105620] Updated weights for policy 1, policy_version 1929994 (0.0010) [2023-12-27 05:25:27,862][105692] Updated weights for policy 0, policy_version 1925279 (0.0008) [2023-12-27 05:25:27,925][105692] Updated weights for policy 0, policy_version 1925289 (0.0009) [2023-12-27 05:25:28,563][105620] Updated weights for policy 1, policy_version 1930004 (0.0010) [2023-12-27 05:25:28,628][105620] Updated weights for policy 1, policy_version 1930014 (0.0010) [2023-12-27 05:25:28,696][105620] Updated weights for policy 1, policy_version 1930024 (0.0010) [2023-12-27 05:25:28,699][105692] Updated weights for policy 0, policy_version 1925299 (0.0007) [2023-12-27 05:25:28,752][105692] Updated weights for policy 0, policy_version 1925309 (0.0009) [2023-12-27 05:25:28,809][105692] Updated weights for policy 0, policy_version 1925319 (0.0008) [2023-12-27 05:25:29,437][105620] Updated weights for policy 1, policy_version 1930034 (0.0010) [2023-12-27 05:25:29,502][105620] Updated weights for policy 1, policy_version 1930044 (0.0010) [2023-12-27 05:25:29,562][105620] Updated weights for policy 1, policy_version 1930054 (0.0011) [2023-12-27 05:25:29,613][105620] Updated weights for policy 1, policy_version 1930064 (0.0010) [2023-12-27 05:25:29,615][105692] Updated weights for policy 0, policy_version 1925329 (0.0008) [2023-12-27 05:25:29,668][105692] Updated weights for policy 0, policy_version 1925339 (0.0007) [2023-12-27 05:25:29,716][105692] Updated weights for policy 0, policy_version 1925349 (0.0008) [2023-12-27 05:25:29,760][105692] Updated weights for policy 0, policy_version 1925359 (0.0007) [2023-12-27 05:25:30,365][105620] Updated weights for policy 1, policy_version 1930074 (0.0010) [2023-12-27 05:25:30,423][105620] Updated weights for policy 1, policy_version 1930084 (0.0010) [2023-12-27 05:25:30,481][105620] Updated weights for policy 1, policy_version 1930094 (0.0010) [2023-12-27 05:25:30,550][105692] Updated weights for policy 0, policy_version 1925369 (0.0008) [2023-12-27 05:25:30,605][105692] Updated weights for policy 0, policy_version 1925379 (0.0008) [2023-12-27 05:25:30,657][105692] Updated weights for policy 0, policy_version 1925389 (0.0008) [2023-12-27 05:25:31,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 987144192. Throughput: 0: 9507.7, 1: 9872.5. Samples: 987117124. Policy #0 lag: (min: 19.0, avg: 32.5, max: 51.0) [2023-12-27 05:25:31,063][104569] Avg episode reward: [(0, '8625.681'), (1, '9345.855')] [2023-12-27 05:25:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001925392_492969984.pth... [2023-12-27 05:25:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001930096_494174208.pth... [2023-12-27 05:25:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001924304_492691456.pth [2023-12-27 05:25:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001928944_493879296.pth [2023-12-27 05:25:31,227][105620] Updated weights for policy 1, policy_version 1930104 (0.0010) [2023-12-27 05:25:31,296][105620] Updated weights for policy 1, policy_version 1930114 (0.0011) [2023-12-27 05:25:31,361][105620] Updated weights for policy 1, policy_version 1930124 (0.0010) [2023-12-27 05:25:31,447][105692] Updated weights for policy 0, policy_version 1925399 (0.0008) [2023-12-27 05:25:31,507][105692] Updated weights for policy 0, policy_version 1925409 (0.0008) [2023-12-27 05:25:31,564][105692] Updated weights for policy 0, policy_version 1925419 (0.0008) [2023-12-27 05:25:32,121][105620] Updated weights for policy 1, policy_version 1930134 (0.0010) [2023-12-27 05:25:32,181][105620] Updated weights for policy 1, policy_version 1930144 (0.0010) [2023-12-27 05:25:32,252][105620] Updated weights for policy 1, policy_version 1930154 (0.0011) [2023-12-27 05:25:32,272][105692] Updated weights for policy 0, policy_version 1925429 (0.0008) [2023-12-27 05:25:32,336][105692] Updated weights for policy 0, policy_version 1925439 (0.0008) [2023-12-27 05:25:32,392][105692] Updated weights for policy 0, policy_version 1925449 (0.0009) [2023-12-27 05:25:32,869][105620] Updated weights for policy 1, policy_version 1930164 (0.0009) [2023-12-27 05:25:32,923][105620] Updated weights for policy 1, policy_version 1930174 (0.0009) [2023-12-27 05:25:32,981][105620] Updated weights for policy 1, policy_version 1930184 (0.0009) [2023-12-27 05:25:33,182][105692] Updated weights for policy 0, policy_version 1925459 (0.0009) [2023-12-27 05:25:33,231][105692] Updated weights for policy 0, policy_version 1925469 (0.0009) [2023-12-27 05:25:33,301][105692] Updated weights for policy 0, policy_version 1925479 (0.0009) [2023-12-27 05:25:33,652][105620] Updated weights for policy 1, policy_version 1930194 (0.0009) [2023-12-27 05:25:33,708][105620] Updated weights for policy 1, policy_version 1930204 (0.0008) [2023-12-27 05:25:33,763][105620] Updated weights for policy 1, policy_version 1930214 (0.0005) [2023-12-27 05:25:33,815][105620] Updated weights for policy 1, policy_version 1930224 (0.0005) [2023-12-27 05:25:34,106][105692] Updated weights for policy 0, policy_version 1925489 (0.0009) [2023-12-27 05:25:34,172][105692] Updated weights for policy 0, policy_version 1925499 (0.0008) [2023-12-27 05:25:34,237][105692] Updated weights for policy 0, policy_version 1925509 (0.0008) [2023-12-27 05:25:34,297][105692] Updated weights for policy 0, policy_version 1925519 (0.0009) [2023-12-27 05:25:34,471][105620] Updated weights for policy 1, policy_version 1930234 (0.0006) [2023-12-27 05:25:34,533][105620] Updated weights for policy 1, policy_version 1930244 (0.0006) [2023-12-27 05:25:34,589][105620] Updated weights for policy 1, policy_version 1930254 (0.0008) [2023-12-27 05:25:35,076][105692] Updated weights for policy 0, policy_version 1925529 (0.0009) [2023-12-27 05:25:35,139][105692] Updated weights for policy 0, policy_version 1925539 (0.0009) [2023-12-27 05:25:35,198][105692] Updated weights for policy 0, policy_version 1925549 (0.0009) [2023-12-27 05:25:35,287][105620] Updated weights for policy 1, policy_version 1930264 (0.0009) [2023-12-27 05:25:35,345][105620] Updated weights for policy 1, policy_version 1930274 (0.0009) [2023-12-27 05:25:35,398][105620] Updated weights for policy 1, policy_version 1930284 (0.0008) [2023-12-27 05:25:35,905][105692] Updated weights for policy 0, policy_version 1925559 (0.0007) [2023-12-27 05:25:35,963][105692] Updated weights for policy 0, policy_version 1925569 (0.0006) [2023-12-27 05:25:36,014][105692] Updated weights for policy 0, policy_version 1925579 (0.0009) [2023-12-27 05:25:36,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 987242496. Throughput: 0: 9431.3, 1: 9832.2. Samples: 987230236. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:25:36,062][104569] Avg episode reward: [(0, '8536.014'), (1, '9253.535')] [2023-12-27 05:25:36,212][105620] Updated weights for policy 1, policy_version 1930294 (0.0009) [2023-12-27 05:25:36,279][105620] Updated weights for policy 1, policy_version 1930304 (0.0009) [2023-12-27 05:25:36,338][105620] Updated weights for policy 1, policy_version 1930314 (0.0009) [2023-12-27 05:25:36,723][105692] Updated weights for policy 0, policy_version 1925589 (0.0007) [2023-12-27 05:25:36,776][105692] Updated weights for policy 0, policy_version 1925599 (0.0005) [2023-12-27 05:25:36,844][105692] Updated weights for policy 0, policy_version 1925609 (0.0005) [2023-12-27 05:25:37,165][105620] Updated weights for policy 1, policy_version 1930324 (0.0009) [2023-12-27 05:25:37,229][105620] Updated weights for policy 1, policy_version 1930334 (0.0008) [2023-12-27 05:25:37,294][105620] Updated weights for policy 1, policy_version 1930344 (0.0009) [2023-12-27 05:25:37,426][105692] Updated weights for policy 0, policy_version 1925619 (0.0007) [2023-12-27 05:25:37,482][105692] Updated weights for policy 0, policy_version 1925629 (0.0009) [2023-12-27 05:25:37,535][105692] Updated weights for policy 0, policy_version 1925639 (0.0006) [2023-12-27 05:25:38,129][105620] Updated weights for policy 1, policy_version 1930354 (0.0009) [2023-12-27 05:25:38,172][105692] Updated weights for policy 0, policy_version 1925649 (0.0005) [2023-12-27 05:25:38,196][105620] Updated weights for policy 1, policy_version 1930364 (0.0009) [2023-12-27 05:25:38,228][105692] Updated weights for policy 0, policy_version 1925659 (0.0010) [2023-12-27 05:25:38,247][105620] Updated weights for policy 1, policy_version 1930374 (0.0008) [2023-12-27 05:25:38,274][105692] Updated weights for policy 0, policy_version 1925669 (0.0007) [2023-12-27 05:25:38,310][105620] Updated weights for policy 1, policy_version 1930384 (0.0007) [2023-12-27 05:25:38,326][105692] Updated weights for policy 0, policy_version 1925679 (0.0008) [2023-12-27 05:25:39,085][105692] Updated weights for policy 0, policy_version 1925689 (0.0007) [2023-12-27 05:25:39,086][105620] Updated weights for policy 1, policy_version 1930394 (0.0008) [2023-12-27 05:25:39,136][105692] Updated weights for policy 0, policy_version 1925699 (0.0007) [2023-12-27 05:25:39,152][105620] Updated weights for policy 1, policy_version 1930404 (0.0009) [2023-12-27 05:25:39,194][105692] Updated weights for policy 0, policy_version 1925709 (0.0007) [2023-12-27 05:25:39,218][105620] Updated weights for policy 1, policy_version 1930414 (0.0007) [2023-12-27 05:25:39,932][105692] Updated weights for policy 0, policy_version 1925719 (0.0008) [2023-12-27 05:25:39,995][105692] Updated weights for policy 0, policy_version 1925729 (0.0009) [2023-12-27 05:25:40,004][105620] Updated weights for policy 1, policy_version 1930424 (0.0007) [2023-12-27 05:25:40,055][105692] Updated weights for policy 0, policy_version 1925739 (0.0008) [2023-12-27 05:25:40,065][105620] Updated weights for policy 1, policy_version 1930434 (0.0006) [2023-12-27 05:25:40,123][105620] Updated weights for policy 1, policy_version 1930444 (0.0007) [2023-12-27 05:25:40,820][105692] Updated weights for policy 0, policy_version 1925749 (0.0007) [2023-12-27 05:25:40,885][105692] Updated weights for policy 0, policy_version 1925759 (0.0008) [2023-12-27 05:25:40,895][105620] Updated weights for policy 1, policy_version 1930454 (0.0007) [2023-12-27 05:25:40,940][105692] Updated weights for policy 0, policy_version 1925769 (0.0009) [2023-12-27 05:25:40,952][105620] Updated weights for policy 1, policy_version 1930464 (0.0006) [2023-12-27 05:25:41,017][105620] Updated weights for policy 1, policy_version 1930474 (0.0006) [2023-12-27 05:25:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19522.0). Total num frames: 987340800. Throughput: 0: 9503.2, 1: 9720.4. Samples: 987343104. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:25:41,062][104569] Avg episode reward: [(0, '8539.537'), (1, '8976.952')] [2023-12-27 05:25:41,742][105620] Updated weights for policy 1, policy_version 1930484 (0.0008) [2023-12-27 05:25:41,796][105692] Updated weights for policy 0, policy_version 1925779 (0.0006) [2023-12-27 05:25:41,805][105620] Updated weights for policy 1, policy_version 1930494 (0.0007) [2023-12-27 05:25:41,859][105692] Updated weights for policy 0, policy_version 1925789 (0.0009) [2023-12-27 05:25:41,865][105620] Updated weights for policy 1, policy_version 1930504 (0.0006) [2023-12-27 05:25:41,926][105692] Updated weights for policy 0, policy_version 1925799 (0.0009) [2023-12-27 05:25:42,518][105620] Updated weights for policy 1, policy_version 1930514 (0.0007) [2023-12-27 05:25:42,576][105620] Updated weights for policy 1, policy_version 1930524 (0.0009) [2023-12-27 05:25:42,632][105620] Updated weights for policy 1, policy_version 1930534 (0.0009) [2023-12-27 05:25:42,654][105692] Updated weights for policy 0, policy_version 1925809 (0.0010) [2023-12-27 05:25:42,681][105620] Updated weights for policy 1, policy_version 1930544 (0.0008) [2023-12-27 05:25:42,703][105692] Updated weights for policy 0, policy_version 1925819 (0.0007) [2023-12-27 05:25:42,754][105692] Updated weights for policy 0, policy_version 1925829 (0.0008) [2023-12-27 05:25:42,819][105692] Updated weights for policy 0, policy_version 1925839 (0.0009) [2023-12-27 05:25:43,468][105620] Updated weights for policy 1, policy_version 1930554 (0.0009) [2023-12-27 05:25:43,515][105620] Updated weights for policy 1, policy_version 1930564 (0.0009) [2023-12-27 05:25:43,562][105692] Updated weights for policy 0, policy_version 1925849 (0.0008) [2023-12-27 05:25:43,564][105620] Updated weights for policy 1, policy_version 1930574 (0.0008) [2023-12-27 05:25:43,612][105692] Updated weights for policy 0, policy_version 1925859 (0.0008) [2023-12-27 05:25:43,659][105692] Updated weights for policy 0, policy_version 1925869 (0.0009) [2023-12-27 05:25:44,312][105620] Updated weights for policy 1, policy_version 1930584 (0.0005) [2023-12-27 05:25:44,358][105620] Updated weights for policy 1, policy_version 1930594 (0.0005) [2023-12-27 05:25:44,407][105620] Updated weights for policy 1, policy_version 1930604 (0.0008) [2023-12-27 05:25:44,413][105692] Updated weights for policy 0, policy_version 1925879 (0.0007) [2023-12-27 05:25:44,477][105692] Updated weights for policy 0, policy_version 1925889 (0.0005) [2023-12-27 05:25:44,546][105692] Updated weights for policy 0, policy_version 1925899 (0.0010) [2023-12-27 05:25:45,071][105620] Updated weights for policy 1, policy_version 1930614 (0.0008) [2023-12-27 05:25:45,130][105620] Updated weights for policy 1, policy_version 1930624 (0.0009) [2023-12-27 05:25:45,198][105620] Updated weights for policy 1, policy_version 1930634 (0.0009) [2023-12-27 05:25:45,291][105692] Updated weights for policy 0, policy_version 1925909 (0.0009) [2023-12-27 05:25:45,356][105692] Updated weights for policy 0, policy_version 1925919 (0.0007) [2023-12-27 05:25:45,430][105692] Updated weights for policy 0, policy_version 1925929 (0.0006) [2023-12-27 05:25:45,998][105620] Updated weights for policy 1, policy_version 1930644 (0.0009) [2023-12-27 05:25:46,049][105620] Updated weights for policy 1, policy_version 1930654 (0.0006) [2023-12-27 05:25:46,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19114.7, 300 sec: 19438.6). Total num frames: 987422720. Throughput: 0: 9464.5, 1: 9679.0. Samples: 987398716. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:25:46,062][104569] Avg episode reward: [(0, '8810.892'), (1, '8978.096')] [2023-12-27 05:25:46,080][105692] Updated weights for policy 0, policy_version 1925939 (0.0008) [2023-12-27 05:25:46,103][105620] Updated weights for policy 1, policy_version 1930664 (0.0006) [2023-12-27 05:25:46,139][105692] Updated weights for policy 0, policy_version 1925949 (0.0008) [2023-12-27 05:25:46,146][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001930672_494321664.pth... [2023-12-27 05:25:46,149][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001929552_494034944.pth [2023-12-27 05:25:46,192][105692] Updated weights for policy 0, policy_version 1925959 (0.0009) [2023-12-27 05:25:46,241][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001925968_493117440.pth... [2023-12-27 05:25:46,244][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001924848_492830720.pth [2023-12-27 05:25:46,838][105620] Updated weights for policy 1, policy_version 1930674 (0.0008) [2023-12-27 05:25:46,887][105620] Updated weights for policy 1, policy_version 1930684 (0.0009) [2023-12-27 05:25:46,948][105620] Updated weights for policy 1, policy_version 1930694 (0.0008) [2023-12-27 05:25:46,952][105692] Updated weights for policy 0, policy_version 1925969 (0.0009) [2023-12-27 05:25:46,999][105692] Updated weights for policy 0, policy_version 1925979 (0.0005) [2023-12-27 05:25:47,001][105620] Updated weights for policy 1, policy_version 1930704 (0.0008) [2023-12-27 05:25:47,047][105692] Updated weights for policy 0, policy_version 1925989 (0.0005) [2023-12-27 05:25:47,095][105692] Updated weights for policy 0, policy_version 1925999 (0.0005) [2023-12-27 05:25:47,762][105620] Updated weights for policy 1, policy_version 1930714 (0.0007) [2023-12-27 05:25:47,764][105692] Updated weights for policy 0, policy_version 1926009 (0.0007) [2023-12-27 05:25:47,820][105692] Updated weights for policy 0, policy_version 1926019 (0.0006) [2023-12-27 05:25:47,821][105620] Updated weights for policy 1, policy_version 1930724 (0.0007) [2023-12-27 05:25:47,879][105620] Updated weights for policy 1, policy_version 1930734 (0.0008) [2023-12-27 05:25:47,880][105692] Updated weights for policy 0, policy_version 1926029 (0.0008) [2023-12-27 05:25:48,592][105692] Updated weights for policy 0, policy_version 1926039 (0.0008) [2023-12-27 05:25:48,650][105692] Updated weights for policy 0, policy_version 1926049 (0.0008) [2023-12-27 05:25:48,654][105620] Updated weights for policy 1, policy_version 1930744 (0.0006) [2023-12-27 05:25:48,704][105692] Updated weights for policy 0, policy_version 1926059 (0.0007) [2023-12-27 05:25:48,714][105620] Updated weights for policy 1, policy_version 1930754 (0.0005) [2023-12-27 05:25:48,766][105620] Updated weights for policy 1, policy_version 1930764 (0.0005) [2023-12-27 05:25:49,295][105620] Updated weights for policy 1, policy_version 1930774 (0.0007) [2023-12-27 05:25:49,360][105620] Updated weights for policy 1, policy_version 1930784 (0.0009) [2023-12-27 05:25:49,417][105620] Updated weights for policy 1, policy_version 1930794 (0.0005) [2023-12-27 05:25:49,547][105692] Updated weights for policy 0, policy_version 1926069 (0.0009) [2023-12-27 05:25:49,611][105692] Updated weights for policy 0, policy_version 1926079 (0.0009) [2023-12-27 05:25:49,670][105692] Updated weights for policy 0, policy_version 1926089 (0.0009) [2023-12-27 05:25:50,163][105620] Updated weights for policy 1, policy_version 1930804 (0.0009) [2023-12-27 05:25:50,221][105620] Updated weights for policy 1, policy_version 1930814 (0.0009) [2023-12-27 05:25:50,283][105620] Updated weights for policy 1, policy_version 1930824 (0.0009) [2023-12-27 05:25:50,414][105692] Updated weights for policy 0, policy_version 1926099 (0.0009) [2023-12-27 05:25:50,473][105692] Updated weights for policy 0, policy_version 1926109 (0.0007) [2023-12-27 05:25:50,532][105692] Updated weights for policy 0, policy_version 1926119 (0.0006) [2023-12-27 05:25:51,032][105620] Updated weights for policy 1, policy_version 1930834 (0.0009) [2023-12-27 05:25:51,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 987521024. Throughput: 0: 9516.4, 1: 9631.5. Samples: 987514876. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:25:51,062][104569] Avg episode reward: [(0, '8179.724'), (1, '9162.666')] [2023-12-27 05:25:51,095][105620] Updated weights for policy 1, policy_version 1930844 (0.0009) [2023-12-27 05:25:51,158][105620] Updated weights for policy 1, policy_version 1930854 (0.0009) [2023-12-27 05:25:51,224][105620] Updated weights for policy 1, policy_version 1930864 (0.0009) [2023-12-27 05:25:51,256][105692] Updated weights for policy 0, policy_version 1926129 (0.0007) [2023-12-27 05:25:51,314][105692] Updated weights for policy 0, policy_version 1926139 (0.0006) [2023-12-27 05:25:51,382][105692] Updated weights for policy 0, policy_version 1926149 (0.0008) [2023-12-27 05:25:51,439][105692] Updated weights for policy 0, policy_version 1926159 (0.0006) [2023-12-27 05:25:51,914][105620] Updated weights for policy 1, policy_version 1930874 (0.0007) [2023-12-27 05:25:51,961][105620] Updated weights for policy 1, policy_version 1930884 (0.0009) [2023-12-27 05:25:52,019][105620] Updated weights for policy 1, policy_version 1930894 (0.0009) [2023-12-27 05:25:52,255][105692] Updated weights for policy 0, policy_version 1926169 (0.0008) [2023-12-27 05:25:52,318][105692] Updated weights for policy 0, policy_version 1926179 (0.0009) [2023-12-27 05:25:52,381][105692] Updated weights for policy 0, policy_version 1926189 (0.0008) [2023-12-27 05:25:52,762][105620] Updated weights for policy 1, policy_version 1930904 (0.0009) [2023-12-27 05:25:52,819][105620] Updated weights for policy 1, policy_version 1930914 (0.0008) [2023-12-27 05:25:52,870][105620] Updated weights for policy 1, policy_version 1930924 (0.0009) [2023-12-27 05:25:53,139][105692] Updated weights for policy 0, policy_version 1926199 (0.0005) [2023-12-27 05:25:53,211][105692] Updated weights for policy 0, policy_version 1926209 (0.0006) [2023-12-27 05:25:53,266][105692] Updated weights for policy 0, policy_version 1926219 (0.0006) [2023-12-27 05:25:53,585][105620] Updated weights for policy 1, policy_version 1930934 (0.0009) [2023-12-27 05:25:53,629][105620] Updated weights for policy 1, policy_version 1930944 (0.0010) [2023-12-27 05:25:53,683][105620] Updated weights for policy 1, policy_version 1930954 (0.0008) [2023-12-27 05:25:53,843][105692] Updated weights for policy 0, policy_version 1926229 (0.0006) [2023-12-27 05:25:53,901][105692] Updated weights for policy 0, policy_version 1926239 (0.0009) [2023-12-27 05:25:53,954][105692] Updated weights for policy 0, policy_version 1926249 (0.0009) [2023-12-27 05:25:54,532][105620] Updated weights for policy 1, policy_version 1930964 (0.0008) [2023-12-27 05:25:54,572][105692] Updated weights for policy 0, policy_version 1926259 (0.0008) [2023-12-27 05:25:54,591][105620] Updated weights for policy 1, policy_version 1930974 (0.0008) [2023-12-27 05:25:54,628][105692] Updated weights for policy 0, policy_version 1926269 (0.0005) [2023-12-27 05:25:54,650][105620] Updated weights for policy 1, policy_version 1930984 (0.0008) [2023-12-27 05:25:54,684][105692] Updated weights for policy 0, policy_version 1926279 (0.0005) [2023-12-27 05:25:55,236][105620] Updated weights for policy 1, policy_version 1930994 (0.0008) [2023-12-27 05:25:55,306][105620] Updated weights for policy 1, policy_version 1931004 (0.0008) [2023-12-27 05:25:55,365][105620] Updated weights for policy 1, policy_version 1931014 (0.0008) [2023-12-27 05:25:55,433][105620] Updated weights for policy 1, policy_version 1931024 (0.0009) [2023-12-27 05:25:55,458][105692] Updated weights for policy 0, policy_version 1926289 (0.0008) [2023-12-27 05:25:55,508][105692] Updated weights for policy 0, policy_version 1926299 (0.0007) [2023-12-27 05:25:55,566][105692] Updated weights for policy 0, policy_version 1926309 (0.0010) [2023-12-27 05:25:55,610][105692] Updated weights for policy 0, policy_version 1926319 (0.0010) [2023-12-27 05:25:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 987619328. Throughput: 0: 9489.1, 1: 9596.9. Samples: 987629568. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:25:56,062][104569] Avg episode reward: [(0, '8088.202'), (1, '9163.263')] [2023-12-27 05:25:56,158][105692] Updated weights for policy 0, policy_version 1926329 (0.0008) [2023-12-27 05:25:56,210][105692] Updated weights for policy 0, policy_version 1926339 (0.0011) [2023-12-27 05:25:56,265][105692] Updated weights for policy 0, policy_version 1926349 (0.0011) [2023-12-27 05:25:56,275][105620] Updated weights for policy 1, policy_version 1931034 (0.0005) [2023-12-27 05:25:56,327][105620] Updated weights for policy 1, policy_version 1931044 (0.0008) [2023-12-27 05:25:56,376][105620] Updated weights for policy 1, policy_version 1931054 (0.0008) [2023-12-27 05:25:56,981][105692] Updated weights for policy 0, policy_version 1926359 (0.0009) [2023-12-27 05:25:57,038][105692] Updated weights for policy 0, policy_version 1926369 (0.0005) [2023-12-27 05:25:57,087][105692] Updated weights for policy 0, policy_version 1926379 (0.0005) [2023-12-27 05:25:57,181][105620] Updated weights for policy 1, policy_version 1931064 (0.0010) [2023-12-27 05:25:57,225][105620] Updated weights for policy 1, policy_version 1931074 (0.0005) [2023-12-27 05:25:57,273][105620] Updated weights for policy 1, policy_version 1931084 (0.0005) [2023-12-27 05:25:57,735][105692] Updated weights for policy 0, policy_version 1926389 (0.0008) [2023-12-27 05:25:57,790][105692] Updated weights for policy 0, policy_version 1926399 (0.0007) [2023-12-27 05:25:57,849][105692] Updated weights for policy 0, policy_version 1926409 (0.0005) [2023-12-27 05:25:57,937][105620] Updated weights for policy 1, policy_version 1931094 (0.0007) [2023-12-27 05:25:57,988][105620] Updated weights for policy 1, policy_version 1931104 (0.0008) [2023-12-27 05:25:58,046][105620] Updated weights for policy 1, policy_version 1931114 (0.0009) [2023-12-27 05:25:58,508][105692] Updated weights for policy 0, policy_version 1926419 (0.0006) [2023-12-27 05:25:58,574][105692] Updated weights for policy 0, policy_version 1926429 (0.0010) [2023-12-27 05:25:58,633][105692] Updated weights for policy 0, policy_version 1926439 (0.0010) [2023-12-27 05:25:58,944][105620] Updated weights for policy 1, policy_version 1931124 (0.0008) [2023-12-27 05:25:59,003][105620] Updated weights for policy 1, policy_version 1931134 (0.0008) [2023-12-27 05:25:59,067][105620] Updated weights for policy 1, policy_version 1931144 (0.0008) [2023-12-27 05:25:59,460][105692] Updated weights for policy 0, policy_version 1926449 (0.0010) [2023-12-27 05:25:59,504][105692] Updated weights for policy 0, policy_version 1926459 (0.0010) [2023-12-27 05:25:59,550][105692] Updated weights for policy 0, policy_version 1926469 (0.0008) [2023-12-27 05:25:59,596][105692] Updated weights for policy 0, policy_version 1926479 (0.0006) [2023-12-27 05:25:59,781][105620] Updated weights for policy 1, policy_version 1931154 (0.0007) [2023-12-27 05:25:59,845][105620] Updated weights for policy 1, policy_version 1931165 (0.0009) [2023-12-27 05:25:59,905][105620] Updated weights for policy 1, policy_version 1931175 (0.0008) [2023-12-27 05:26:00,324][105692] Updated weights for policy 0, policy_version 1926489 (0.0007) [2023-12-27 05:26:00,383][105692] Updated weights for policy 0, policy_version 1926499 (0.0005) [2023-12-27 05:26:00,433][105692] Updated weights for policy 0, policy_version 1926509 (0.0005) [2023-12-27 05:26:00,593][105620] Updated weights for policy 1, policy_version 1931185 (0.0008) [2023-12-27 05:26:00,640][105620] Updated weights for policy 1, policy_version 1931195 (0.0008) [2023-12-27 05:26:00,694][105620] Updated weights for policy 1, policy_version 1931205 (0.0009) [2023-12-27 05:26:00,755][105620] Updated weights for policy 1, policy_version 1931216 (0.0007) [2023-12-27 05:26:00,986][105692] Updated weights for policy 0, policy_version 1926519 (0.0006) [2023-12-27 05:26:01,046][105692] Updated weights for policy 0, policy_version 1926529 (0.0011) [2023-12-27 05:26:01,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 987717632. Throughput: 0: 9613.7, 1: 9553.1. Samples: 987689104. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:26:01,063][104569] Avg episode reward: [(0, '8259.932'), (1, '9255.450')] [2023-12-27 05:26:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001931216_494460928.pth... [2023-12-27 05:26:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001930096_494174208.pth [2023-12-27 05:26:01,107][105692] Updated weights for policy 0, policy_version 1926539 (0.0010) [2023-12-27 05:26:01,137][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001926544_493264896.pth... [2023-12-27 05:26:01,143][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001925392_492969984.pth [2023-12-27 05:26:01,426][105620] Updated weights for policy 1, policy_version 1931226 (0.0007) [2023-12-27 05:26:01,492][105620] Updated weights for policy 1, policy_version 1931236 (0.0005) [2023-12-27 05:26:01,562][105620] Updated weights for policy 1, policy_version 1931246 (0.0005) [2023-12-27 05:26:01,843][105692] Updated weights for policy 0, policy_version 1926549 (0.0011) [2023-12-27 05:26:01,894][105692] Updated weights for policy 0, policy_version 1926559 (0.0010) [2023-12-27 05:26:01,941][105692] Updated weights for policy 0, policy_version 1926569 (0.0010) [2023-12-27 05:26:02,215][105620] Updated weights for policy 1, policy_version 1931256 (0.0007) [2023-12-27 05:26:02,263][105620] Updated weights for policy 1, policy_version 1931266 (0.0008) [2023-12-27 05:26:02,316][105620] Updated weights for policy 1, policy_version 1931276 (0.0008) [2023-12-27 05:26:02,706][105692] Updated weights for policy 0, policy_version 1926579 (0.0010) [2023-12-27 05:26:02,764][105692] Updated weights for policy 0, policy_version 1926589 (0.0010) [2023-12-27 05:26:02,826][105692] Updated weights for policy 0, policy_version 1926599 (0.0010) [2023-12-27 05:26:03,031][105620] Updated weights for policy 1, policy_version 1931286 (0.0008) [2023-12-27 05:26:03,085][105620] Updated weights for policy 1, policy_version 1931296 (0.0007) [2023-12-27 05:26:03,128][105620] Updated weights for policy 1, policy_version 1931306 (0.0007) [2023-12-27 05:26:03,527][105692] Updated weights for policy 0, policy_version 1926609 (0.0010) [2023-12-27 05:26:03,588][105692] Updated weights for policy 0, policy_version 1926619 (0.0010) [2023-12-27 05:26:03,651][105692] Updated weights for policy 0, policy_version 1926629 (0.0011) [2023-12-27 05:26:03,710][105692] Updated weights for policy 0, policy_version 1926639 (0.0011) [2023-12-27 05:26:03,815][105620] Updated weights for policy 1, policy_version 1931316 (0.0008) [2023-12-27 05:26:03,877][105620] Updated weights for policy 1, policy_version 1931326 (0.0007) [2023-12-27 05:26:03,929][105620] Updated weights for policy 1, policy_version 1931336 (0.0008) [2023-12-27 05:26:04,431][105692] Updated weights for policy 0, policy_version 1926649 (0.0010) [2023-12-27 05:26:04,496][105692] Updated weights for policy 0, policy_version 1926659 (0.0008) [2023-12-27 05:26:04,557][105692] Updated weights for policy 0, policy_version 1926669 (0.0008) [2023-12-27 05:26:04,702][105620] Updated weights for policy 1, policy_version 1931346 (0.0008) [2023-12-27 05:26:04,769][105620] Updated weights for policy 1, policy_version 1931356 (0.0009) [2023-12-27 05:26:04,835][105620] Updated weights for policy 1, policy_version 1931366 (0.0010) [2023-12-27 05:26:04,891][105620] Updated weights for policy 1, policy_version 1931376 (0.0010) [2023-12-27 05:26:05,165][105692] Updated weights for policy 0, policy_version 1926679 (0.0006) [2023-12-27 05:26:05,229][105692] Updated weights for policy 0, policy_version 1926689 (0.0007) [2023-12-27 05:26:05,282][105692] Updated weights for policy 0, policy_version 1926699 (0.0007) [2023-12-27 05:26:05,614][105620] Updated weights for policy 1, policy_version 1931386 (0.0010) [2023-12-27 05:26:05,674][105620] Updated weights for policy 1, policy_version 1931396 (0.0011) [2023-12-27 05:26:05,723][105620] Updated weights for policy 1, policy_version 1931406 (0.0010) [2023-12-27 05:26:05,915][105692] Updated weights for policy 0, policy_version 1926709 (0.0008) [2023-12-27 05:26:05,976][105692] Updated weights for policy 0, policy_version 1926719 (0.0008) [2023-12-27 05:26:06,045][105692] Updated weights for policy 0, policy_version 1926729 (0.0006) [2023-12-27 05:26:06,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 987815936. Throughput: 0: 9573.1, 1: 9518.4. Samples: 987805744. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:26:06,062][104569] Avg episode reward: [(0, '7804.821'), (1, '9346.145')] [2023-12-27 05:26:06,399][105620] Updated weights for policy 1, policy_version 1931416 (0.0010) [2023-12-27 05:26:06,454][105620] Updated weights for policy 1, policy_version 1931426 (0.0009) [2023-12-27 05:26:06,511][105620] Updated weights for policy 1, policy_version 1931436 (0.0009) [2023-12-27 05:26:06,750][105692] Updated weights for policy 0, policy_version 1926739 (0.0007) [2023-12-27 05:26:06,808][105692] Updated weights for policy 0, policy_version 1926749 (0.0009) [2023-12-27 05:26:06,865][105692] Updated weights for policy 0, policy_version 1926759 (0.0009) [2023-12-27 05:26:07,241][105620] Updated weights for policy 1, policy_version 1931446 (0.0007) [2023-12-27 05:26:07,302][105620] Updated weights for policy 1, policy_version 1931456 (0.0006) [2023-12-27 05:26:07,353][105620] Updated weights for policy 1, policy_version 1931466 (0.0008) [2023-12-27 05:26:07,570][105692] Updated weights for policy 0, policy_version 1926769 (0.0006) [2023-12-27 05:26:07,639][105692] Updated weights for policy 0, policy_version 1926779 (0.0009) [2023-12-27 05:26:07,700][105692] Updated weights for policy 0, policy_version 1926789 (0.0009) [2023-12-27 05:26:07,759][105692] Updated weights for policy 0, policy_version 1926799 (0.0009) [2023-12-27 05:26:08,088][105620] Updated weights for policy 1, policy_version 1931476 (0.0009) [2023-12-27 05:26:08,144][105620] Updated weights for policy 1, policy_version 1931486 (0.0009) [2023-12-27 05:26:08,197][105620] Updated weights for policy 1, policy_version 1931496 (0.0009) [2023-12-27 05:26:08,468][105692] Updated weights for policy 0, policy_version 1926809 (0.0006) [2023-12-27 05:26:08,532][105692] Updated weights for policy 0, policy_version 1926819 (0.0006) [2023-12-27 05:26:08,598][105692] Updated weights for policy 0, policy_version 1926829 (0.0005) [2023-12-27 05:26:09,051][105620] Updated weights for policy 1, policy_version 1931506 (0.0010) [2023-12-27 05:26:09,111][105620] Updated weights for policy 1, policy_version 1931516 (0.0009) [2023-12-27 05:26:09,179][105620] Updated weights for policy 1, policy_version 1931526 (0.0009) [2023-12-27 05:26:09,249][105620] Updated weights for policy 1, policy_version 1931536 (0.0010) [2023-12-27 05:26:09,250][105692] Updated weights for policy 0, policy_version 1926839 (0.0008) [2023-12-27 05:26:09,308][105692] Updated weights for policy 0, policy_version 1926849 (0.0008) [2023-12-27 05:26:09,376][105692] Updated weights for policy 0, policy_version 1926859 (0.0008) [2023-12-27 05:26:10,048][105620] Updated weights for policy 1, policy_version 1931546 (0.0009) [2023-12-27 05:26:10,104][105620] Updated weights for policy 1, policy_version 1931556 (0.0009) [2023-12-27 05:26:10,121][105692] Updated weights for policy 0, policy_version 1926869 (0.0008) [2023-12-27 05:26:10,164][105620] Updated weights for policy 1, policy_version 1931566 (0.0009) [2023-12-27 05:26:10,175][105692] Updated weights for policy 0, policy_version 1926879 (0.0007) [2023-12-27 05:26:10,232][105692] Updated weights for policy 0, policy_version 1926889 (0.0009) [2023-12-27 05:26:10,963][105620] Updated weights for policy 1, policy_version 1931576 (0.0009) [2023-12-27 05:26:11,026][105620] Updated weights for policy 1, policy_version 1931586 (0.0009) [2023-12-27 05:26:11,034][105692] Updated weights for policy 0, policy_version 1926899 (0.0009) [2023-12-27 05:26:11,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19114.7, 300 sec: 19438.6). Total num frames: 987906048. Throughput: 0: 9685.2, 1: 9470.3. Samples: 987921760. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:26:11,062][104569] Avg episode reward: [(0, '8175.709'), (1, '9162.051')] [2023-12-27 05:26:11,093][105620] Updated weights for policy 1, policy_version 1931596 (0.0006) [2023-12-27 05:26:11,100][105692] Updated weights for policy 0, policy_version 1926909 (0.0009) [2023-12-27 05:26:11,169][105692] Updated weights for policy 0, policy_version 1926919 (0.0010) [2023-12-27 05:26:11,861][105620] Updated weights for policy 1, policy_version 1931606 (0.0008) [2023-12-27 05:26:11,923][105620] Updated weights for policy 1, policy_version 1931616 (0.0006) [2023-12-27 05:26:11,987][105620] Updated weights for policy 1, policy_version 1931626 (0.0006) [2023-12-27 05:26:11,996][105692] Updated weights for policy 0, policy_version 1926929 (0.0010) [2023-12-27 05:26:12,053][105692] Updated weights for policy 0, policy_version 1926939 (0.0009) [2023-12-27 05:26:12,115][105692] Updated weights for policy 0, policy_version 1926949 (0.0010) [2023-12-27 05:26:12,173][105692] Updated weights for policy 0, policy_version 1926959 (0.0009) [2023-12-27 05:26:12,634][105620] Updated weights for policy 1, policy_version 1931636 (0.0009) [2023-12-27 05:26:12,696][105620] Updated weights for policy 1, policy_version 1931646 (0.0010) [2023-12-27 05:26:12,758][105620] Updated weights for policy 1, policy_version 1931656 (0.0011) [2023-12-27 05:26:13,015][105692] Updated weights for policy 0, policy_version 1926969 (0.0007) [2023-12-27 05:26:13,064][105692] Updated weights for policy 0, policy_version 1926979 (0.0007) [2023-12-27 05:26:13,119][105692] Updated weights for policy 0, policy_version 1926989 (0.0007) [2023-12-27 05:26:13,374][105620] Updated weights for policy 1, policy_version 1931666 (0.0005) [2023-12-27 05:26:13,429][105620] Updated weights for policy 1, policy_version 1931676 (0.0005) [2023-12-27 05:26:13,480][105620] Updated weights for policy 1, policy_version 1931686 (0.0010) [2023-12-27 05:26:13,525][105620] Updated weights for policy 1, policy_version 1931696 (0.0008) [2023-12-27 05:26:13,907][105692] Updated weights for policy 0, policy_version 1926999 (0.0009) [2023-12-27 05:26:13,953][105692] Updated weights for policy 0, policy_version 1927009 (0.0008) [2023-12-27 05:26:14,018][105692] Updated weights for policy 0, policy_version 1927019 (0.0010) [2023-12-27 05:26:14,185][105620] Updated weights for policy 1, policy_version 1931706 (0.0006) [2023-12-27 05:26:14,249][105620] Updated weights for policy 1, policy_version 1931716 (0.0006) [2023-12-27 05:26:14,307][105620] Updated weights for policy 1, policy_version 1931726 (0.0006) [2023-12-27 05:26:14,890][105692] Updated weights for policy 0, policy_version 1927029 (0.0008) [2023-12-27 05:26:14,893][105620] Updated weights for policy 1, policy_version 1931736 (0.0006) [2023-12-27 05:26:14,941][105692] Updated weights for policy 0, policy_version 1927039 (0.0006) [2023-12-27 05:26:14,951][105620] Updated weights for policy 1, policy_version 1931746 (0.0009) [2023-12-27 05:26:14,994][105692] Updated weights for policy 0, policy_version 1927049 (0.0006) [2023-12-27 05:26:15,012][105620] Updated weights for policy 1, policy_version 1931756 (0.0009) [2023-12-27 05:26:15,771][105692] Updated weights for policy 0, policy_version 1927059 (0.0007) [2023-12-27 05:26:15,773][105620] Updated weights for policy 1, policy_version 1931766 (0.0007) [2023-12-27 05:26:15,819][105692] Updated weights for policy 0, policy_version 1927069 (0.0007) [2023-12-27 05:26:15,833][105620] Updated weights for policy 1, policy_version 1931776 (0.0007) [2023-12-27 05:26:15,881][105692] Updated weights for policy 0, policy_version 1927079 (0.0007) [2023-12-27 05:26:15,894][105620] Updated weights for policy 1, policy_version 1931786 (0.0006) [2023-12-27 05:26:16,062][104569] Fps is (10 sec: 19660.0, 60 sec: 19251.1, 300 sec: 19466.4). Total num frames: 988012544. Throughput: 0: 9614.1, 1: 9519.9. Samples: 987978164. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:26:16,063][104569] Avg episode reward: [(0, '8264.494'), (1, '9071.404')] [2023-12-27 05:26:16,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001927088_493404160.pth... [2023-12-27 05:26:16,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001931792_494608384.pth... [2023-12-27 05:26:16,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001925968_493117440.pth [2023-12-27 05:26:16,081][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001930672_494321664.pth [2023-12-27 05:26:16,544][105620] Updated weights for policy 1, policy_version 1931796 (0.0007) [2023-12-27 05:26:16,596][105620] Updated weights for policy 1, policy_version 1931806 (0.0009) [2023-12-27 05:26:16,645][105620] Updated weights for policy 1, policy_version 1931816 (0.0009) [2023-12-27 05:26:16,680][105692] Updated weights for policy 0, policy_version 1927089 (0.0008) [2023-12-27 05:26:16,741][105692] Updated weights for policy 0, policy_version 1927099 (0.0008) [2023-12-27 05:26:16,794][105692] Updated weights for policy 0, policy_version 1927109 (0.0008) [2023-12-27 05:26:16,841][105692] Updated weights for policy 0, policy_version 1927119 (0.0008) [2023-12-27 05:26:17,318][105620] Updated weights for policy 1, policy_version 1931826 (0.0008) [2023-12-27 05:26:17,370][105620] Updated weights for policy 1, policy_version 1931836 (0.0007) [2023-12-27 05:26:17,421][105620] Updated weights for policy 1, policy_version 1931846 (0.0006) [2023-12-27 05:26:17,476][105620] Updated weights for policy 1, policy_version 1931856 (0.0005) [2023-12-27 05:26:17,709][105692] Updated weights for policy 0, policy_version 1927130 (0.0010) [2023-12-27 05:26:17,771][105692] Updated weights for policy 0, policy_version 1927140 (0.0010) [2023-12-27 05:26:17,835][105692] Updated weights for policy 0, policy_version 1927150 (0.0008) [2023-12-27 05:26:18,085][105620] Updated weights for policy 1, policy_version 1931866 (0.0010) [2023-12-27 05:26:18,144][105620] Updated weights for policy 1, policy_version 1931876 (0.0010) [2023-12-27 05:26:18,206][105620] Updated weights for policy 1, policy_version 1931886 (0.0011) [2023-12-27 05:26:18,600][105692] Updated weights for policy 0, policy_version 1927160 (0.0008) [2023-12-27 05:26:18,654][105692] Updated weights for policy 0, policy_version 1927170 (0.0009) [2023-12-27 05:26:18,708][105692] Updated weights for policy 0, policy_version 1927180 (0.0009) [2023-12-27 05:26:18,910][105620] Updated weights for policy 1, policy_version 1931896 (0.0009) [2023-12-27 05:26:18,972][105620] Updated weights for policy 1, policy_version 1931906 (0.0009) [2023-12-27 05:26:19,030][105620] Updated weights for policy 1, policy_version 1931916 (0.0009) [2023-12-27 05:26:19,527][105692] Updated weights for policy 0, policy_version 1927190 (0.0010) [2023-12-27 05:26:19,579][105692] Updated weights for policy 0, policy_version 1927200 (0.0008) [2023-12-27 05:26:19,635][105692] Updated weights for policy 0, policy_version 1927210 (0.0009) [2023-12-27 05:26:19,761][105620] Updated weights for policy 1, policy_version 1931926 (0.0009) [2023-12-27 05:26:19,819][105620] Updated weights for policy 1, policy_version 1931936 (0.0009) [2023-12-27 05:26:19,881][105620] Updated weights for policy 1, policy_version 1931946 (0.0010) [2023-12-27 05:26:20,488][105692] Updated weights for policy 0, policy_version 1927220 (0.0010) [2023-12-27 05:26:20,559][105692] Updated weights for policy 0, policy_version 1927230 (0.0009) [2023-12-27 05:26:20,562][105620] Updated weights for policy 1, policy_version 1931956 (0.0008) [2023-12-27 05:26:20,624][105692] Updated weights for policy 0, policy_version 1927240 (0.0008) [2023-12-27 05:26:20,635][105620] Updated weights for policy 1, policy_version 1931966 (0.0007) [2023-12-27 05:26:20,700][105620] Updated weights for policy 1, policy_version 1931976 (0.0008) [2023-12-27 05:26:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19114.7, 300 sec: 19438.6). Total num frames: 988102656. Throughput: 0: 9577.4, 1: 9566.9. Samples: 988091732. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:26:21,062][104569] Avg episode reward: [(0, '8624.075'), (1, '9076.033')] [2023-12-27 05:26:21,293][105692] Updated weights for policy 0, policy_version 1927250 (0.0007) [2023-12-27 05:26:21,363][105692] Updated weights for policy 0, policy_version 1927260 (0.0007) [2023-12-27 05:26:21,428][105692] Updated weights for policy 0, policy_version 1927270 (0.0009) [2023-12-27 05:26:21,492][105692] Updated weights for policy 0, policy_version 1927280 (0.0009) [2023-12-27 05:26:21,546][105620] Updated weights for policy 1, policy_version 1931986 (0.0009) [2023-12-27 05:26:21,602][105620] Updated weights for policy 1, policy_version 1931996 (0.0009) [2023-12-27 05:26:21,663][105620] Updated weights for policy 1, policy_version 1932006 (0.0008) [2023-12-27 05:26:21,726][105620] Updated weights for policy 1, policy_version 1932016 (0.0009) [2023-12-27 05:26:22,157][105692] Updated weights for policy 0, policy_version 1927290 (0.0005) [2023-12-27 05:26:22,225][105692] Updated weights for policy 0, policy_version 1927300 (0.0006) [2023-12-27 05:26:22,293][105692] Updated weights for policy 0, policy_version 1927310 (0.0008) [2023-12-27 05:26:22,468][105620] Updated weights for policy 1, policy_version 1932026 (0.0009) [2023-12-27 05:26:22,528][105620] Updated weights for policy 1, policy_version 1932036 (0.0008) [2023-12-27 05:26:22,591][105620] Updated weights for policy 1, policy_version 1932046 (0.0009) [2023-12-27 05:26:22,944][105692] Updated weights for policy 0, policy_version 1927320 (0.0008) [2023-12-27 05:26:23,011][105692] Updated weights for policy 0, policy_version 1927330 (0.0009) [2023-12-27 05:26:23,067][105692] Updated weights for policy 0, policy_version 1927340 (0.0008) [2023-12-27 05:26:23,255][105620] Updated weights for policy 1, policy_version 1932056 (0.0006) [2023-12-27 05:26:23,304][105620] Updated weights for policy 1, policy_version 1932066 (0.0005) [2023-12-27 05:26:23,359][105620] Updated weights for policy 1, policy_version 1932076 (0.0007) [2023-12-27 05:26:23,668][105692] Updated weights for policy 0, policy_version 1927350 (0.0007) [2023-12-27 05:26:23,715][105692] Updated weights for policy 0, policy_version 1927360 (0.0005) [2023-12-27 05:26:23,762][105692] Updated weights for policy 0, policy_version 1927370 (0.0005) [2023-12-27 05:26:24,097][105620] Updated weights for policy 1, policy_version 1932086 (0.0010) [2023-12-27 05:26:24,160][105620] Updated weights for policy 1, policy_version 1932096 (0.0010) [2023-12-27 05:26:24,228][105620] Updated weights for policy 1, policy_version 1932106 (0.0010) [2023-12-27 05:26:24,329][105692] Updated weights for policy 0, policy_version 1927380 (0.0007) [2023-12-27 05:26:24,388][105692] Updated weights for policy 0, policy_version 1927391 (0.0009) [2023-12-27 05:26:24,445][105692] Updated weights for policy 0, policy_version 1927401 (0.0008) [2023-12-27 05:26:24,877][105620] Updated weights for policy 1, policy_version 1932116 (0.0006) [2023-12-27 05:26:24,929][105620] Updated weights for policy 1, policy_version 1932126 (0.0005) [2023-12-27 05:26:24,975][105620] Updated weights for policy 1, policy_version 1932136 (0.0005) [2023-12-27 05:26:25,329][105692] Updated weights for policy 0, policy_version 1927411 (0.0008) [2023-12-27 05:26:25,387][105692] Updated weights for policy 0, policy_version 1927424 (0.0010) [2023-12-27 05:26:25,436][105692] Updated weights for policy 0, policy_version 1927435 (0.0009) [2023-12-27 05:26:25,510][105620] Updated weights for policy 1, policy_version 1932146 (0.0006) [2023-12-27 05:26:25,558][105620] Updated weights for policy 1, policy_version 1932156 (0.0010) [2023-12-27 05:26:25,605][105620] Updated weights for policy 1, policy_version 1932166 (0.0010) [2023-12-27 05:26:25,654][105620] Updated weights for policy 1, policy_version 1932176 (0.0010) [2023-12-27 05:26:26,062][104569] Fps is (10 sec: 18842.4, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 988200960. Throughput: 0: 9555.3, 1: 9716.5. Samples: 988210336. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:26:26,062][104569] Avg episode reward: [(0, '8267.848'), (1, '8889.595')] [2023-12-27 05:26:26,129][105692] Updated weights for policy 0, policy_version 1927445 (0.0009) [2023-12-27 05:26:26,188][105692] Updated weights for policy 0, policy_version 1927455 (0.0010) [2023-12-27 05:26:26,244][105692] Updated weights for policy 0, policy_version 1927465 (0.0009) [2023-12-27 05:26:26,430][105620] Updated weights for policy 1, policy_version 1932186 (0.0010) [2023-12-27 05:26:26,489][105620] Updated weights for policy 1, policy_version 1932196 (0.0005) [2023-12-27 05:26:26,549][105620] Updated weights for policy 1, policy_version 1932206 (0.0005) [2023-12-27 05:26:26,989][105692] Updated weights for policy 0, policy_version 1927475 (0.0010) [2023-12-27 05:26:27,046][105692] Updated weights for policy 0, policy_version 1927485 (0.0010) [2023-12-27 05:26:27,077][105620] Updated weights for policy 1, policy_version 1932216 (0.0007) [2023-12-27 05:26:27,107][105692] Updated weights for policy 0, policy_version 1927495 (0.0010) [2023-12-27 05:26:27,138][105620] Updated weights for policy 1, policy_version 1932226 (0.0006) [2023-12-27 05:26:27,196][105620] Updated weights for policy 1, policy_version 1932236 (0.0010) [2023-12-27 05:26:27,699][105692] Updated weights for policy 0, policy_version 1927505 (0.0010) [2023-12-27 05:26:27,752][105692] Updated weights for policy 0, policy_version 1927515 (0.0005) [2023-12-27 05:26:27,807][105692] Updated weights for policy 0, policy_version 1927525 (0.0006) [2023-12-27 05:26:27,851][105620] Updated weights for policy 1, policy_version 1932246 (0.0010) [2023-12-27 05:26:27,857][105692] Updated weights for policy 0, policy_version 1927535 (0.0006) [2023-12-27 05:26:27,908][105620] Updated weights for policy 1, policy_version 1932256 (0.0010) [2023-12-27 05:26:27,958][105620] Updated weights for policy 1, policy_version 1932266 (0.0007) [2023-12-27 05:26:28,384][105692] Updated weights for policy 0, policy_version 1927545 (0.0010) [2023-12-27 05:26:28,438][105692] Updated weights for policy 0, policy_version 1927555 (0.0006) [2023-12-27 05:26:28,490][105692] Updated weights for policy 0, policy_version 1927565 (0.0006) [2023-12-27 05:26:28,546][105620] Updated weights for policy 1, policy_version 1932276 (0.0006) [2023-12-27 05:26:28,600][105620] Updated weights for policy 1, policy_version 1932286 (0.0010) [2023-12-27 05:26:28,651][105620] Updated weights for policy 1, policy_version 1932296 (0.0010) [2023-12-27 05:26:29,115][105692] Updated weights for policy 0, policy_version 1927575 (0.0009) [2023-12-27 05:26:29,180][105692] Updated weights for policy 0, policy_version 1927585 (0.0007) [2023-12-27 05:26:29,243][105692] Updated weights for policy 0, policy_version 1927595 (0.0008) [2023-12-27 05:26:29,391][105620] Updated weights for policy 1, policy_version 1932306 (0.0010) [2023-12-27 05:26:29,448][105620] Updated weights for policy 1, policy_version 1932316 (0.0009) [2023-12-27 05:26:29,506][105620] Updated weights for policy 1, policy_version 1932326 (0.0010) [2023-12-27 05:26:29,843][105692] Updated weights for policy 0, policy_version 1927605 (0.0006) [2023-12-27 05:26:29,904][105692] Updated weights for policy 0, policy_version 1927615 (0.0009) [2023-12-27 05:26:29,969][105692] Updated weights for policy 0, policy_version 1927625 (0.0007) [2023-12-27 05:26:30,384][105620] Updated weights for policy 1, policy_version 1932337 (0.0009) [2023-12-27 05:26:30,448][105620] Updated weights for policy 1, policy_version 1932347 (0.0005) [2023-12-27 05:26:30,503][105620] Updated weights for policy 1, policy_version 1932357 (0.0005) [2023-12-27 05:26:30,554][105620] Updated weights for policy 1, policy_version 1932367 (0.0005) [2023-12-27 05:26:30,658][105692] Updated weights for policy 0, policy_version 1927635 (0.0008) [2023-12-27 05:26:30,728][105692] Updated weights for policy 0, policy_version 1927645 (0.0009) [2023-12-27 05:26:30,795][105692] Updated weights for policy 0, policy_version 1927655 (0.0009) [2023-12-27 05:26:31,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 988307456. Throughput: 0: 9684.4, 1: 9809.1. Samples: 988275920. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:26:31,062][104569] Avg episode reward: [(0, '7992.912'), (1, '8979.254')] [2023-12-27 05:26:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001927664_493551616.pth... [2023-12-27 05:26:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001926544_493264896.pth [2023-12-27 05:26:31,078][105620] Updated weights for policy 1, policy_version 1932377 (0.0009) [2023-12-27 05:26:31,136][105620] Updated weights for policy 1, policy_version 1932387 (0.0009) [2023-12-27 05:26:31,192][105620] Updated weights for policy 1, policy_version 1932397 (0.0009) [2023-12-27 05:26:31,208][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001932400_494764032.pth... [2023-12-27 05:26:31,212][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001931216_494460928.pth [2023-12-27 05:26:31,558][105692] Updated weights for policy 0, policy_version 1927665 (0.0009) [2023-12-27 05:26:31,604][105692] Updated weights for policy 0, policy_version 1927675 (0.0008) [2023-12-27 05:26:31,673][105692] Updated weights for policy 0, policy_version 1927685 (0.0007) [2023-12-27 05:26:31,739][105692] Updated weights for policy 0, policy_version 1927695 (0.0009) [2023-12-27 05:26:31,921][105620] Updated weights for policy 1, policy_version 1932408 (0.0009) [2023-12-27 05:26:31,980][105620] Updated weights for policy 1, policy_version 1932418 (0.0009) [2023-12-27 05:26:32,031][105620] Updated weights for policy 1, policy_version 1932428 (0.0008) [2023-12-27 05:26:32,463][105692] Updated weights for policy 0, policy_version 1927705 (0.0009) [2023-12-27 05:26:32,517][105692] Updated weights for policy 0, policy_version 1927715 (0.0009) [2023-12-27 05:26:32,578][105692] Updated weights for policy 0, policy_version 1927725 (0.0009) [2023-12-27 05:26:32,742][105620] Updated weights for policy 1, policy_version 1932438 (0.0009) [2023-12-27 05:26:32,800][105620] Updated weights for policy 1, policy_version 1932448 (0.0009) [2023-12-27 05:26:32,860][105620] Updated weights for policy 1, policy_version 1932458 (0.0008) [2023-12-27 05:26:33,339][105692] Updated weights for policy 0, policy_version 1927735 (0.0009) [2023-12-27 05:26:33,408][105692] Updated weights for policy 0, policy_version 1927745 (0.0009) [2023-12-27 05:26:33,470][105692] Updated weights for policy 0, policy_version 1927755 (0.0008) [2023-12-27 05:26:33,501][105620] Updated weights for policy 1, policy_version 1932468 (0.0006) [2023-12-27 05:26:33,548][105620] Updated weights for policy 1, policy_version 1932478 (0.0005) [2023-12-27 05:26:33,598][105620] Updated weights for policy 1, policy_version 1932488 (0.0005) [2023-12-27 05:26:34,224][105692] Updated weights for policy 0, policy_version 1927765 (0.0009) [2023-12-27 05:26:34,292][105620] Updated weights for policy 1, policy_version 1932498 (0.0005) [2023-12-27 05:26:34,295][105692] Updated weights for policy 0, policy_version 1927775 (0.0009) [2023-12-27 05:26:34,352][105620] Updated weights for policy 1, policy_version 1932508 (0.0006) [2023-12-27 05:26:34,358][105692] Updated weights for policy 0, policy_version 1927785 (0.0008) [2023-12-27 05:26:34,410][105620] Updated weights for policy 1, policy_version 1932518 (0.0008) [2023-12-27 05:26:34,474][105620] Updated weights for policy 1, policy_version 1932528 (0.0010) [2023-12-27 05:26:35,071][105692] Updated weights for policy 0, policy_version 1927795 (0.0007) [2023-12-27 05:26:35,125][105692] Updated weights for policy 0, policy_version 1927805 (0.0009) [2023-12-27 05:26:35,183][105692] Updated weights for policy 0, policy_version 1927815 (0.0007) [2023-12-27 05:26:35,209][105620] Updated weights for policy 1, policy_version 1932538 (0.0007) [2023-12-27 05:26:35,261][105620] Updated weights for policy 1, policy_version 1932548 (0.0008) [2023-12-27 05:26:35,326][105620] Updated weights for policy 1, policy_version 1932558 (0.0008) [2023-12-27 05:26:35,921][105692] Updated weights for policy 0, policy_version 1927825 (0.0008) [2023-12-27 05:26:35,986][105692] Updated weights for policy 0, policy_version 1927835 (0.0009) [2023-12-27 05:26:36,047][105692] Updated weights for policy 0, policy_version 1927845 (0.0009) [2023-12-27 05:26:36,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 988397568. Throughput: 0: 9714.0, 1: 9818.7. Samples: 988393848. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:26:36,062][104569] Avg episode reward: [(0, '8352.490'), (1, '9163.976')] [2023-12-27 05:26:36,084][105620] Updated weights for policy 1, policy_version 1932568 (0.0009) [2023-12-27 05:26:36,107][105692] Updated weights for policy 0, policy_version 1927855 (0.0007) [2023-12-27 05:26:36,145][105620] Updated weights for policy 1, policy_version 1932578 (0.0008) [2023-12-27 05:26:36,204][105620] Updated weights for policy 1, policy_version 1932588 (0.0010) [2023-12-27 05:26:36,804][105692] Updated weights for policy 0, policy_version 1927865 (0.0006) [2023-12-27 05:26:36,865][105692] Updated weights for policy 0, policy_version 1927875 (0.0009) [2023-12-27 05:26:36,927][105692] Updated weights for policy 0, policy_version 1927885 (0.0009) [2023-12-27 05:26:36,980][105620] Updated weights for policy 1, policy_version 1932598 (0.0007) [2023-12-27 05:26:37,043][105620] Updated weights for policy 1, policy_version 1932608 (0.0009) [2023-12-27 05:26:37,105][105620] Updated weights for policy 1, policy_version 1932618 (0.0009) [2023-12-27 05:26:37,662][105692] Updated weights for policy 0, policy_version 1927895 (0.0008) [2023-12-27 05:26:37,713][105692] Updated weights for policy 0, policy_version 1927905 (0.0009) [2023-12-27 05:26:37,761][105692] Updated weights for policy 0, policy_version 1927915 (0.0009) [2023-12-27 05:26:37,810][105620] Updated weights for policy 1, policy_version 1932628 (0.0008) [2023-12-27 05:26:37,880][105620] Updated weights for policy 1, policy_version 1932638 (0.0008) [2023-12-27 05:26:37,933][105620] Updated weights for policy 1, policy_version 1932648 (0.0009) [2023-12-27 05:26:38,466][105692] Updated weights for policy 0, policy_version 1927925 (0.0007) [2023-12-27 05:26:38,513][105692] Updated weights for policy 0, policy_version 1927935 (0.0009) [2023-12-27 05:26:38,580][105692] Updated weights for policy 0, policy_version 1927945 (0.0009) [2023-12-27 05:26:38,738][105620] Updated weights for policy 1, policy_version 1932658 (0.0009) [2023-12-27 05:26:38,792][105620] Updated weights for policy 1, policy_version 1932669 (0.0010) [2023-12-27 05:26:38,845][105620] Updated weights for policy 1, policy_version 1932679 (0.0010) [2023-12-27 05:26:39,270][105692] Updated weights for policy 0, policy_version 1927955 (0.0009) [2023-12-27 05:26:39,333][105692] Updated weights for policy 0, policy_version 1927965 (0.0010) [2023-12-27 05:26:39,400][105692] Updated weights for policy 0, policy_version 1927975 (0.0008) [2023-12-27 05:26:39,713][105620] Updated weights for policy 1, policy_version 1932689 (0.0009) [2023-12-27 05:26:39,775][105620] Updated weights for policy 1, policy_version 1932699 (0.0009) [2023-12-27 05:26:39,836][105620] Updated weights for policy 1, policy_version 1932709 (0.0009) [2023-12-27 05:26:39,901][105620] Updated weights for policy 1, policy_version 1932719 (0.0008) [2023-12-27 05:26:40,129][105692] Updated weights for policy 0, policy_version 1927985 (0.0008) [2023-12-27 05:26:40,191][105692] Updated weights for policy 0, policy_version 1927995 (0.0008) [2023-12-27 05:26:40,251][105692] Updated weights for policy 0, policy_version 1928005 (0.0006) [2023-12-27 05:26:40,313][105692] Updated weights for policy 0, policy_version 1928015 (0.0008) [2023-12-27 05:26:40,709][105620] Updated weights for policy 1, policy_version 1932729 (0.0009) [2023-12-27 05:26:40,767][105620] Updated weights for policy 1, policy_version 1932739 (0.0008) [2023-12-27 05:26:40,836][105620] Updated weights for policy 1, policy_version 1932749 (0.0009) [2023-12-27 05:26:40,997][105692] Updated weights for policy 0, policy_version 1928025 (0.0006) [2023-12-27 05:26:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19466.5). Total num frames: 988495872. Throughput: 0: 9706.2, 1: 9754.6. Samples: 988505304. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:26:41,062][104569] Avg episode reward: [(0, '8629.701'), (1, '9346.101')] [2023-12-27 05:26:41,064][105692] Updated weights for policy 0, policy_version 1928035 (0.0009) [2023-12-27 05:26:41,124][105692] Updated weights for policy 0, policy_version 1928045 (0.0009) [2023-12-27 05:26:41,664][105620] Updated weights for policy 1, policy_version 1932759 (0.0010) [2023-12-27 05:26:41,727][105620] Updated weights for policy 1, policy_version 1932769 (0.0010) [2023-12-27 05:26:41,799][105620] Updated weights for policy 1, policy_version 1932779 (0.0006) [2023-12-27 05:26:41,909][105692] Updated weights for policy 0, policy_version 1928055 (0.0009) [2023-12-27 05:26:41,970][105692] Updated weights for policy 0, policy_version 1928065 (0.0009) [2023-12-27 05:26:42,029][105692] Updated weights for policy 0, policy_version 1928075 (0.0009) [2023-12-27 05:26:42,508][105620] Updated weights for policy 1, policy_version 1932789 (0.0007) [2023-12-27 05:26:42,558][105620] Updated weights for policy 1, policy_version 1932799 (0.0009) [2023-12-27 05:26:42,608][105620] Updated weights for policy 1, policy_version 1932809 (0.0009) [2023-12-27 05:26:42,797][105692] Updated weights for policy 0, policy_version 1928085 (0.0008) [2023-12-27 05:26:42,856][105692] Updated weights for policy 0, policy_version 1928095 (0.0009) [2023-12-27 05:26:42,911][105692] Updated weights for policy 0, policy_version 1928105 (0.0009) [2023-12-27 05:26:43,350][105620] Updated weights for policy 1, policy_version 1932819 (0.0008) [2023-12-27 05:26:43,401][105620] Updated weights for policy 1, policy_version 1932829 (0.0006) [2023-12-27 05:26:43,466][105620] Updated weights for policy 1, policy_version 1932839 (0.0007) [2023-12-27 05:26:43,689][105692] Updated weights for policy 0, policy_version 1928115 (0.0009) [2023-12-27 05:26:43,739][105692] Updated weights for policy 0, policy_version 1928125 (0.0008) [2023-12-27 05:26:43,788][105692] Updated weights for policy 0, policy_version 1928135 (0.0008) [2023-12-27 05:26:44,162][105620] Updated weights for policy 1, policy_version 1932849 (0.0009) [2023-12-27 05:26:44,220][105620] Updated weights for policy 1, policy_version 1932859 (0.0009) [2023-12-27 05:26:44,279][105620] Updated weights for policy 1, policy_version 1932869 (0.0009) [2023-12-27 05:26:44,337][105620] Updated weights for policy 1, policy_version 1932879 (0.0009) [2023-12-27 05:26:44,414][105692] Updated weights for policy 0, policy_version 1928145 (0.0008) [2023-12-27 05:26:44,471][105692] Updated weights for policy 0, policy_version 1928155 (0.0009) [2023-12-27 05:26:44,521][105692] Updated weights for policy 0, policy_version 1928165 (0.0009) [2023-12-27 05:26:44,579][105692] Updated weights for policy 0, policy_version 1928175 (0.0009) [2023-12-27 05:26:45,136][105620] Updated weights for policy 1, policy_version 1932889 (0.0010) [2023-12-27 05:26:45,188][105620] Updated weights for policy 1, policy_version 1932899 (0.0009) [2023-12-27 05:26:45,242][105620] Updated weights for policy 1, policy_version 1932909 (0.0008) [2023-12-27 05:26:45,277][105692] Updated weights for policy 0, policy_version 1928185 (0.0008) [2023-12-27 05:26:45,330][105692] Updated weights for policy 0, policy_version 1928195 (0.0009) [2023-12-27 05:26:45,381][105692] Updated weights for policy 0, policy_version 1928205 (0.0009) [2023-12-27 05:26:46,052][105692] Updated weights for policy 0, policy_version 1928215 (0.0008) [2023-12-27 05:26:46,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 988585984. Throughput: 0: 9608.8, 1: 9773.1. Samples: 988561288. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:26:46,062][104569] Avg episode reward: [(0, '8535.652'), (1, '9254.839')] [2023-12-27 05:26:46,078][105620] Updated weights for policy 1, policy_version 1932919 (0.0008) [2023-12-27 05:26:46,105][105692] Updated weights for policy 0, policy_version 1928225 (0.0006) [2023-12-27 05:26:46,137][105620] Updated weights for policy 1, policy_version 1932929 (0.0009) [2023-12-27 05:26:46,164][105692] Updated weights for policy 0, policy_version 1928235 (0.0009) [2023-12-27 05:26:46,189][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001928240_493699072.pth... [2023-12-27 05:26:46,194][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001927088_493404160.pth [2023-12-27 05:26:46,199][105620] Updated weights for policy 1, policy_version 1932939 (0.0005) [2023-12-27 05:26:46,231][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001932944_494903296.pth... [2023-12-27 05:26:46,236][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001931792_494608384.pth [2023-12-27 05:26:46,858][105620] Updated weights for policy 1, policy_version 1932949 (0.0007) [2023-12-27 05:26:46,919][105620] Updated weights for policy 1, policy_version 1932959 (0.0007) [2023-12-27 05:26:46,964][105692] Updated weights for policy 0, policy_version 1928245 (0.0009) [2023-12-27 05:26:46,987][105620] Updated weights for policy 1, policy_version 1932969 (0.0005) [2023-12-27 05:26:47,025][105692] Updated weights for policy 0, policy_version 1928255 (0.0008) [2023-12-27 05:26:47,079][105692] Updated weights for policy 0, policy_version 1928265 (0.0010) [2023-12-27 05:26:47,594][105620] Updated weights for policy 1, policy_version 1932979 (0.0007) [2023-12-27 05:26:47,641][105620] Updated weights for policy 1, policy_version 1932989 (0.0009) [2023-12-27 05:26:47,688][105620] Updated weights for policy 1, policy_version 1932999 (0.0009) [2023-12-27 05:26:47,879][105692] Updated weights for policy 0, policy_version 1928275 (0.0007) [2023-12-27 05:26:47,931][105692] Updated weights for policy 0, policy_version 1928285 (0.0009) [2023-12-27 05:26:47,979][105692] Updated weights for policy 0, policy_version 1928295 (0.0009) [2023-12-27 05:26:48,427][105620] Updated weights for policy 1, policy_version 1933009 (0.0009) [2023-12-27 05:26:48,492][105620] Updated weights for policy 1, policy_version 1933019 (0.0007) [2023-12-27 05:26:48,555][105620] Updated weights for policy 1, policy_version 1933029 (0.0006) [2023-12-27 05:26:48,608][105620] Updated weights for policy 1, policy_version 1933039 (0.0005) [2023-12-27 05:26:48,763][105692] Updated weights for policy 0, policy_version 1928305 (0.0009) [2023-12-27 05:26:48,826][105692] Updated weights for policy 0, policy_version 1928315 (0.0005) [2023-12-27 05:26:48,896][105692] Updated weights for policy 0, policy_version 1928325 (0.0005) [2023-12-27 05:26:48,954][105692] Updated weights for policy 0, policy_version 1928335 (0.0007) [2023-12-27 05:26:49,261][105620] Updated weights for policy 1, policy_version 1933049 (0.0006) [2023-12-27 05:26:49,315][105620] Updated weights for policy 1, policy_version 1933059 (0.0007) [2023-12-27 05:26:49,386][105620] Updated weights for policy 1, policy_version 1933069 (0.0009) [2023-12-27 05:26:49,618][105692] Updated weights for policy 0, policy_version 1928345 (0.0009) [2023-12-27 05:26:49,677][105692] Updated weights for policy 0, policy_version 1928355 (0.0009) [2023-12-27 05:26:49,739][105692] Updated weights for policy 0, policy_version 1928365 (0.0010) [2023-12-27 05:26:50,111][105620] Updated weights for policy 1, policy_version 1933079 (0.0010) [2023-12-27 05:26:50,180][105620] Updated weights for policy 1, policy_version 1933089 (0.0009) [2023-12-27 05:26:50,246][105620] Updated weights for policy 1, policy_version 1933099 (0.0009) [2023-12-27 05:26:50,442][105692] Updated weights for policy 0, policy_version 1928375 (0.0008) [2023-12-27 05:26:50,508][105692] Updated weights for policy 0, policy_version 1928385 (0.0009) [2023-12-27 05:26:50,560][105692] Updated weights for policy 0, policy_version 1928395 (0.0009) [2023-12-27 05:26:51,039][105620] Updated weights for policy 1, policy_version 1933109 (0.0010) [2023-12-27 05:26:51,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 988684288. Throughput: 0: 9613.3, 1: 9778.6. Samples: 988678380. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:26:51,062][104569] Avg episode reward: [(0, '8629.869'), (1, '9254.956')] [2023-12-27 05:26:51,091][105620] Updated weights for policy 1, policy_version 1933119 (0.0008) [2023-12-27 05:26:51,151][105620] Updated weights for policy 1, policy_version 1933129 (0.0009) [2023-12-27 05:26:51,367][105692] Updated weights for policy 0, policy_version 1928405 (0.0010) [2023-12-27 05:26:51,434][105692] Updated weights for policy 0, policy_version 1928415 (0.0009) [2023-12-27 05:26:51,494][105692] Updated weights for policy 0, policy_version 1928425 (0.0010) [2023-12-27 05:26:51,918][105620] Updated weights for policy 1, policy_version 1933139 (0.0007) [2023-12-27 05:26:51,979][105620] Updated weights for policy 1, policy_version 1933149 (0.0009) [2023-12-27 05:26:52,041][105620] Updated weights for policy 1, policy_version 1933159 (0.0009) [2023-12-27 05:26:52,295][105692] Updated weights for policy 0, policy_version 1928435 (0.0010) [2023-12-27 05:26:52,360][105692] Updated weights for policy 0, policy_version 1928445 (0.0009) [2023-12-27 05:26:52,419][105692] Updated weights for policy 0, policy_version 1928455 (0.0008) [2023-12-27 05:26:52,785][105620] Updated weights for policy 1, policy_version 1933169 (0.0008) [2023-12-27 05:26:52,852][105620] Updated weights for policy 1, policy_version 1933179 (0.0010) [2023-12-27 05:26:52,911][105620] Updated weights for policy 1, policy_version 1933189 (0.0010) [2023-12-27 05:26:52,973][105620] Updated weights for policy 1, policy_version 1933199 (0.0009) [2023-12-27 05:26:53,075][105692] Updated weights for policy 0, policy_version 1928465 (0.0009) [2023-12-27 05:26:53,133][105692] Updated weights for policy 0, policy_version 1928475 (0.0009) [2023-12-27 05:26:53,199][105692] Updated weights for policy 0, policy_version 1928485 (0.0007) [2023-12-27 05:26:53,259][105692] Updated weights for policy 0, policy_version 1928495 (0.0007) [2023-12-27 05:26:53,621][105620] Updated weights for policy 1, policy_version 1933209 (0.0006) [2023-12-27 05:26:53,671][105620] Updated weights for policy 1, policy_version 1933219 (0.0006) [2023-12-27 05:26:53,724][105620] Updated weights for policy 1, policy_version 1933229 (0.0008) [2023-12-27 05:26:54,029][105692] Updated weights for policy 0, policy_version 1928505 (0.0010) [2023-12-27 05:26:54,083][105692] Updated weights for policy 0, policy_version 1928515 (0.0009) [2023-12-27 05:26:54,131][105692] Updated weights for policy 0, policy_version 1928525 (0.0009) [2023-12-27 05:26:54,357][105620] Updated weights for policy 1, policy_version 1933239 (0.0008) [2023-12-27 05:26:54,414][105620] Updated weights for policy 1, policy_version 1933249 (0.0008) [2023-12-27 05:26:54,464][105620] Updated weights for policy 1, policy_version 1933259 (0.0008) [2023-12-27 05:26:54,858][105692] Updated weights for policy 0, policy_version 1928535 (0.0007) [2023-12-27 05:26:54,925][105692] Updated weights for policy 0, policy_version 1928545 (0.0008) [2023-12-27 05:26:54,989][105692] Updated weights for policy 0, policy_version 1928555 (0.0010) [2023-12-27 05:26:55,247][105620] Updated weights for policy 1, policy_version 1933269 (0.0008) [2023-12-27 05:26:55,298][105620] Updated weights for policy 1, policy_version 1933279 (0.0009) [2023-12-27 05:26:55,348][105620] Updated weights for policy 1, policy_version 1933289 (0.0009) [2023-12-27 05:26:55,728][105692] Updated weights for policy 0, policy_version 1928565 (0.0009) [2023-12-27 05:26:55,778][105692] Updated weights for policy 0, policy_version 1928575 (0.0009) [2023-12-27 05:26:55,836][105692] Updated weights for policy 0, policy_version 1928585 (0.0009) [2023-12-27 05:26:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 988782592. Throughput: 0: 9530.1, 1: 9802.6. Samples: 988791732. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:26:56,063][104569] Avg episode reward: [(0, '8629.138'), (1, '9255.026')] [2023-12-27 05:26:56,069][105620] Updated weights for policy 1, policy_version 1933299 (0.0009) [2023-12-27 05:26:56,134][105620] Updated weights for policy 1, policy_version 1933309 (0.0009) [2023-12-27 05:26:56,191][105620] Updated weights for policy 1, policy_version 1933319 (0.0009) [2023-12-27 05:26:56,612][105692] Updated weights for policy 0, policy_version 1928595 (0.0009) [2023-12-27 05:26:56,670][105692] Updated weights for policy 0, policy_version 1928605 (0.0009) [2023-12-27 05:26:56,734][105692] Updated weights for policy 0, policy_version 1928615 (0.0008) [2023-12-27 05:26:56,938][105620] Updated weights for policy 1, policy_version 1933329 (0.0009) [2023-12-27 05:26:56,999][105620] Updated weights for policy 1, policy_version 1933339 (0.0009) [2023-12-27 05:26:57,059][105620] Updated weights for policy 1, policy_version 1933349 (0.0009) [2023-12-27 05:26:57,120][105620] Updated weights for policy 1, policy_version 1933359 (0.0008) [2023-12-27 05:26:57,464][105692] Updated weights for policy 0, policy_version 1928625 (0.0009) [2023-12-27 05:26:57,514][105692] Updated weights for policy 0, policy_version 1928635 (0.0009) [2023-12-27 05:26:57,560][105692] Updated weights for policy 0, policy_version 1928645 (0.0008) [2023-12-27 05:26:57,617][105692] Updated weights for policy 0, policy_version 1928655 (0.0008) [2023-12-27 05:26:57,857][105620] Updated weights for policy 1, policy_version 1933369 (0.0009) [2023-12-27 05:26:57,907][105620] Updated weights for policy 1, policy_version 1933379 (0.0008) [2023-12-27 05:26:57,955][105620] Updated weights for policy 1, policy_version 1933389 (0.0007) [2023-12-27 05:26:58,426][105692] Updated weights for policy 0, policy_version 1928665 (0.0008) [2023-12-27 05:26:58,481][105692] Updated weights for policy 0, policy_version 1928675 (0.0008) [2023-12-27 05:26:58,548][105692] Updated weights for policy 0, policy_version 1928685 (0.0008) [2023-12-27 05:26:58,660][105620] Updated weights for policy 1, policy_version 1933399 (0.0008) [2023-12-27 05:26:58,721][105620] Updated weights for policy 1, policy_version 1933409 (0.0008) [2023-12-27 05:26:58,786][105620] Updated weights for policy 1, policy_version 1933419 (0.0008) [2023-12-27 05:26:59,325][105692] Updated weights for policy 0, policy_version 1928695 (0.0009) [2023-12-27 05:26:59,391][105692] Updated weights for policy 0, policy_version 1928705 (0.0009) [2023-12-27 05:26:59,449][105692] Updated weights for policy 0, policy_version 1928715 (0.0010) [2023-12-27 05:26:59,518][105620] Updated weights for policy 1, policy_version 1933429 (0.0007) [2023-12-27 05:26:59,578][105620] Updated weights for policy 1, policy_version 1933439 (0.0005) [2023-12-27 05:26:59,640][105620] Updated weights for policy 1, policy_version 1933449 (0.0008) [2023-12-27 05:27:00,234][105620] Updated weights for policy 1, policy_version 1933459 (0.0008) [2023-12-27 05:27:00,293][105692] Updated weights for policy 0, policy_version 1928725 (0.0009) [2023-12-27 05:27:00,296][105620] Updated weights for policy 1, policy_version 1933469 (0.0006) [2023-12-27 05:27:00,342][105620] Updated weights for policy 1, policy_version 1933479 (0.0006) [2023-12-27 05:27:00,353][105692] Updated weights for policy 0, policy_version 1928735 (0.0007) [2023-12-27 05:27:00,400][105692] Updated weights for policy 0, policy_version 1928745 (0.0008) [2023-12-27 05:27:01,027][105620] Updated weights for policy 1, policy_version 1933489 (0.0006) [2023-12-27 05:27:01,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 988872704. Throughput: 0: 9555.6, 1: 9759.8. Samples: 988847352. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:27:01,063][104569] Avg episode reward: [(0, '8533.987'), (1, '9346.411')] [2023-12-27 05:27:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001928752_493830144.pth... [2023-12-27 05:27:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001927664_493551616.pth [2023-12-27 05:27:01,093][105620] Updated weights for policy 1, policy_version 1933499 (0.0007) [2023-12-27 05:27:01,157][105620] Updated weights for policy 1, policy_version 1933509 (0.0007) [2023-12-27 05:27:01,213][105692] Updated weights for policy 0, policy_version 1928755 (0.0009) [2023-12-27 05:27:01,219][105620] Updated weights for policy 1, policy_version 1933519 (0.0005) [2023-12-27 05:27:01,223][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001933520_495050752.pth... [2023-12-27 05:27:01,226][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001932400_494764032.pth [2023-12-27 05:27:01,282][105692] Updated weights for policy 0, policy_version 1928765 (0.0009) [2023-12-27 05:27:01,347][105692] Updated weights for policy 0, policy_version 1928775 (0.0009) [2023-12-27 05:27:01,943][105620] Updated weights for policy 1, policy_version 1933529 (0.0008) [2023-12-27 05:27:02,007][105620] Updated weights for policy 1, policy_version 1933539 (0.0009) [2023-12-27 05:27:02,031][105692] Updated weights for policy 0, policy_version 1928785 (0.0008) [2023-12-27 05:27:02,061][105620] Updated weights for policy 1, policy_version 1933549 (0.0008) [2023-12-27 05:27:02,077][105692] Updated weights for policy 0, policy_version 1928795 (0.0007) [2023-12-27 05:27:02,129][105692] Updated weights for policy 0, policy_version 1928805 (0.0010) [2023-12-27 05:27:02,183][105692] Updated weights for policy 0, policy_version 1928816 (0.0010) [2023-12-27 05:27:02,772][105620] Updated weights for policy 1, policy_version 1933559 (0.0008) [2023-12-27 05:27:02,825][105620] Updated weights for policy 1, policy_version 1933569 (0.0008) [2023-12-27 05:27:02,870][105620] Updated weights for policy 1, policy_version 1933579 (0.0008) [2023-12-27 05:27:02,968][105692] Updated weights for policy 0, policy_version 1928826 (0.0009) [2023-12-27 05:27:03,014][105692] Updated weights for policy 0, policy_version 1928836 (0.0008) [2023-12-27 05:27:03,061][105692] Updated weights for policy 0, policy_version 1928846 (0.0009) [2023-12-27 05:27:03,599][105620] Updated weights for policy 1, policy_version 1933589 (0.0009) [2023-12-27 05:27:03,650][105620] Updated weights for policy 1, policy_version 1933599 (0.0009) [2023-12-27 05:27:03,703][105620] Updated weights for policy 1, policy_version 1933609 (0.0008) [2023-12-27 05:27:03,845][105692] Updated weights for policy 0, policy_version 1928856 (0.0009) [2023-12-27 05:27:03,905][105692] Updated weights for policy 0, policy_version 1928866 (0.0010) [2023-12-27 05:27:03,965][105692] Updated weights for policy 0, policy_version 1928876 (0.0009) [2023-12-27 05:27:04,538][105620] Updated weights for policy 1, policy_version 1933619 (0.0009) [2023-12-27 05:27:04,593][105620] Updated weights for policy 1, policy_version 1933629 (0.0009) [2023-12-27 05:27:04,603][105692] Updated weights for policy 0, policy_version 1928886 (0.0007) [2023-12-27 05:27:04,642][105620] Updated weights for policy 1, policy_version 1933639 (0.0007) [2023-12-27 05:27:04,666][105692] Updated weights for policy 0, policy_version 1928896 (0.0008) [2023-12-27 05:27:04,717][105692] Updated weights for policy 0, policy_version 1928906 (0.0008) [2023-12-27 05:27:05,389][105692] Updated weights for policy 0, policy_version 1928916 (0.0008) [2023-12-27 05:27:05,397][105620] Updated weights for policy 1, policy_version 1933649 (0.0008) [2023-12-27 05:27:05,443][105692] Updated weights for policy 0, policy_version 1928926 (0.0008) [2023-12-27 05:27:05,456][105620] Updated weights for policy 1, policy_version 1933659 (0.0007) [2023-12-27 05:27:05,494][105692] Updated weights for policy 0, policy_version 1928936 (0.0006) [2023-12-27 05:27:05,509][105620] Updated weights for policy 1, policy_version 1933669 (0.0006) [2023-12-27 05:27:05,567][105620] Updated weights for policy 1, policy_version 1933679 (0.0007) [2023-12-27 05:27:06,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.1, 300 sec: 19383.1). Total num frames: 988971008. Throughput: 0: 9629.4, 1: 9699.6. Samples: 988961540. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:27:06,063][104569] Avg episode reward: [(0, '7806.676'), (1, '9254.613')] [2023-12-27 05:27:06,206][105692] Updated weights for policy 0, policy_version 1928946 (0.0009) [2023-12-27 05:27:06,266][105692] Updated weights for policy 0, policy_version 1928956 (0.0006) [2023-12-27 05:27:06,324][105692] Updated weights for policy 0, policy_version 1928966 (0.0007) [2023-12-27 05:27:06,347][105620] Updated weights for policy 1, policy_version 1933689 (0.0007) [2023-12-27 05:27:06,380][105692] Updated weights for policy 0, policy_version 1928976 (0.0008) [2023-12-27 05:27:06,409][105620] Updated weights for policy 1, policy_version 1933699 (0.0010) [2023-12-27 05:27:06,483][105620] Updated weights for policy 1, policy_version 1933709 (0.0009) [2023-12-27 05:27:07,115][105692] Updated weights for policy 0, policy_version 1928986 (0.0009) [2023-12-27 05:27:07,181][105692] Updated weights for policy 0, policy_version 1928996 (0.0008) [2023-12-27 05:27:07,229][105692] Updated weights for policy 0, policy_version 1929006 (0.0007) [2023-12-27 05:27:07,231][105620] Updated weights for policy 1, policy_version 1933719 (0.0008) [2023-12-27 05:27:07,290][105620] Updated weights for policy 1, policy_version 1933729 (0.0009) [2023-12-27 05:27:07,346][105620] Updated weights for policy 1, policy_version 1933739 (0.0009) [2023-12-27 05:27:07,948][105692] Updated weights for policy 0, policy_version 1929016 (0.0008) [2023-12-27 05:27:07,999][105692] Updated weights for policy 0, policy_version 1929026 (0.0009) [2023-12-27 05:27:08,060][105692] Updated weights for policy 0, policy_version 1929036 (0.0009) [2023-12-27 05:27:08,115][105620] Updated weights for policy 1, policy_version 1933749 (0.0009) [2023-12-27 05:27:08,178][105620] Updated weights for policy 1, policy_version 1933759 (0.0009) [2023-12-27 05:27:08,237][105620] Updated weights for policy 1, policy_version 1933769 (0.0009) [2023-12-27 05:27:08,856][105692] Updated weights for policy 0, policy_version 1929046 (0.0009) [2023-12-27 05:27:08,923][105692] Updated weights for policy 0, policy_version 1929056 (0.0008) [2023-12-27 05:27:08,929][105620] Updated weights for policy 1, policy_version 1933779 (0.0006) [2023-12-27 05:27:08,980][105692] Updated weights for policy 0, policy_version 1929066 (0.0008) [2023-12-27 05:27:08,986][105620] Updated weights for policy 1, policy_version 1933789 (0.0006) [2023-12-27 05:27:09,048][105620] Updated weights for policy 1, policy_version 1933799 (0.0008) [2023-12-27 05:27:09,766][105692] Updated weights for policy 0, policy_version 1929076 (0.0009) [2023-12-27 05:27:09,811][105620] Updated weights for policy 1, policy_version 1933809 (0.0009) [2023-12-27 05:27:09,831][105692] Updated weights for policy 0, policy_version 1929086 (0.0009) [2023-12-27 05:27:09,871][105620] Updated weights for policy 1, policy_version 1933819 (0.0007) [2023-12-27 05:27:09,890][105692] Updated weights for policy 0, policy_version 1929096 (0.0007) [2023-12-27 05:27:09,937][105620] Updated weights for policy 1, policy_version 1933829 (0.0008) [2023-12-27 05:27:09,998][105620] Updated weights for policy 1, policy_version 1933839 (0.0008) [2023-12-27 05:27:10,666][105692] Updated weights for policy 0, policy_version 1929106 (0.0007) [2023-12-27 05:27:10,723][105620] Updated weights for policy 1, policy_version 1933849 (0.0008) [2023-12-27 05:27:10,727][105692] Updated weights for policy 0, policy_version 1929116 (0.0009) [2023-12-27 05:27:10,783][105620] Updated weights for policy 1, policy_version 1933859 (0.0006) [2023-12-27 05:27:10,787][105692] Updated weights for policy 0, policy_version 1929126 (0.0007) [2023-12-27 05:27:10,842][105620] Updated weights for policy 1, policy_version 1933869 (0.0006) [2023-12-27 05:27:10,844][105692] Updated weights for policy 0, policy_version 1929136 (0.0008) [2023-12-27 05:27:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 989069312. Throughput: 0: 9584.2, 1: 9605.8. Samples: 989073888. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:27:11,062][104569] Avg episode reward: [(0, '7714.678'), (1, '9254.566')] [2023-12-27 05:27:11,620][105620] Updated weights for policy 1, policy_version 1933879 (0.0007) [2023-12-27 05:27:11,622][105692] Updated weights for policy 0, policy_version 1929146 (0.0008) [2023-12-27 05:27:11,683][105692] Updated weights for policy 0, policy_version 1929156 (0.0008) [2023-12-27 05:27:11,684][105620] Updated weights for policy 1, policy_version 1933889 (0.0008) [2023-12-27 05:27:11,748][105620] Updated weights for policy 1, policy_version 1933899 (0.0008) [2023-12-27 05:27:11,766][105692] Updated weights for policy 0, policy_version 1929166 (0.0008) [2023-12-27 05:27:12,451][105692] Updated weights for policy 0, policy_version 1929176 (0.0010) [2023-12-27 05:27:12,511][105692] Updated weights for policy 0, policy_version 1929186 (0.0009) [2023-12-27 05:27:12,580][105692] Updated weights for policy 0, policy_version 1929196 (0.0006) [2023-12-27 05:27:12,625][105620] Updated weights for policy 1, policy_version 1933909 (0.0009) [2023-12-27 05:27:12,685][105620] Updated weights for policy 1, policy_version 1933919 (0.0008) [2023-12-27 05:27:12,750][105620] Updated weights for policy 1, policy_version 1933929 (0.0008) [2023-12-27 05:27:13,239][105692] Updated weights for policy 0, policy_version 1929206 (0.0008) [2023-12-27 05:27:13,295][105692] Updated weights for policy 0, policy_version 1929216 (0.0005) [2023-12-27 05:27:13,350][105692] Updated weights for policy 0, policy_version 1929226 (0.0005) [2023-12-27 05:27:13,552][105620] Updated weights for policy 1, policy_version 1933939 (0.0008) [2023-12-27 05:27:13,606][105620] Updated weights for policy 1, policy_version 1933949 (0.0011) [2023-12-27 05:27:13,676][105620] Updated weights for policy 1, policy_version 1933959 (0.0009) [2023-12-27 05:27:13,933][105692] Updated weights for policy 0, policy_version 1929236 (0.0007) [2023-12-27 05:27:13,988][105692] Updated weights for policy 0, policy_version 1929246 (0.0010) [2023-12-27 05:27:14,037][105692] Updated weights for policy 0, policy_version 1929256 (0.0010) [2023-12-27 05:27:14,371][105620] Updated weights for policy 1, policy_version 1933969 (0.0008) [2023-12-27 05:27:14,424][105620] Updated weights for policy 1, policy_version 1933979 (0.0007) [2023-12-27 05:27:14,473][105620] Updated weights for policy 1, policy_version 1933989 (0.0008) [2023-12-27 05:27:14,525][105620] Updated weights for policy 1, policy_version 1933999 (0.0008) [2023-12-27 05:27:14,782][105692] Updated weights for policy 0, policy_version 1929266 (0.0010) [2023-12-27 05:27:14,844][105692] Updated weights for policy 0, policy_version 1929276 (0.0010) [2023-12-27 05:27:14,907][105692] Updated weights for policy 0, policy_version 1929286 (0.0010) [2023-12-27 05:27:14,966][105692] Updated weights for policy 0, policy_version 1929296 (0.0011) [2023-12-27 05:27:15,238][105620] Updated weights for policy 1, policy_version 1934009 (0.0008) [2023-12-27 05:27:15,304][105620] Updated weights for policy 1, policy_version 1934019 (0.0009) [2023-12-27 05:27:15,362][105620] Updated weights for policy 1, policy_version 1934029 (0.0008) [2023-12-27 05:27:15,706][105692] Updated weights for policy 0, policy_version 1929306 (0.0008) [2023-12-27 05:27:15,765][105692] Updated weights for policy 0, policy_version 1929316 (0.0008) [2023-12-27 05:27:15,826][105692] Updated weights for policy 0, policy_version 1929326 (0.0008) [2023-12-27 05:27:16,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19114.8, 300 sec: 19383.1). Total num frames: 989159424. Throughput: 0: 9513.7, 1: 9461.6. Samples: 989129808. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:27:16,063][104569] Avg episode reward: [(0, '7900.612'), (1, '9255.754')] [2023-12-27 05:27:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001929328_493977600.pth... [2023-12-27 05:27:16,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001928240_493699072.pth [2023-12-27 05:27:16,102][105620] Updated weights for policy 1, policy_version 1934039 (0.0007) [2023-12-27 05:27:16,154][105620] Updated weights for policy 1, policy_version 1934049 (0.0005) [2023-12-27 05:27:16,200][105620] Updated weights for policy 1, policy_version 1934059 (0.0005) [2023-12-27 05:27:16,223][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001934064_495190016.pth... [2023-12-27 05:27:16,226][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001932944_494903296.pth [2023-12-27 05:27:16,491][105692] Updated weights for policy 0, policy_version 1929336 (0.0007) [2023-12-27 05:27:16,546][105692] Updated weights for policy 0, policy_version 1929346 (0.0009) [2023-12-27 05:27:16,606][105692] Updated weights for policy 0, policy_version 1929356 (0.0010) [2023-12-27 05:27:16,899][105620] Updated weights for policy 1, policy_version 1934069 (0.0007) [2023-12-27 05:27:16,961][105620] Updated weights for policy 1, policy_version 1934079 (0.0009) [2023-12-27 05:27:17,015][105620] Updated weights for policy 1, policy_version 1934089 (0.0009) [2023-12-27 05:27:17,304][105692] Updated weights for policy 0, policy_version 1929366 (0.0008) [2023-12-27 05:27:17,368][105692] Updated weights for policy 0, policy_version 1929376 (0.0009) [2023-12-27 05:27:17,429][105692] Updated weights for policy 0, policy_version 1929386 (0.0008) [2023-12-27 05:27:17,833][105620] Updated weights for policy 1, policy_version 1934099 (0.0009) [2023-12-27 05:27:17,889][105620] Updated weights for policy 1, policy_version 1934109 (0.0008) [2023-12-27 05:27:17,945][105620] Updated weights for policy 1, policy_version 1934119 (0.0008) [2023-12-27 05:27:18,115][105692] Updated weights for policy 0, policy_version 1929396 (0.0007) [2023-12-27 05:27:18,177][105692] Updated weights for policy 0, policy_version 1929406 (0.0011) [2023-12-27 05:27:18,236][105692] Updated weights for policy 0, policy_version 1929416 (0.0009) [2023-12-27 05:27:18,711][105620] Updated weights for policy 1, policy_version 1934129 (0.0008) [2023-12-27 05:27:18,777][105620] Updated weights for policy 1, policy_version 1934139 (0.0007) [2023-12-27 05:27:18,836][105620] Updated weights for policy 1, policy_version 1934149 (0.0008) [2023-12-27 05:27:18,894][105620] Updated weights for policy 1, policy_version 1934159 (0.0008) [2023-12-27 05:27:19,030][105692] Updated weights for policy 0, policy_version 1929426 (0.0009) [2023-12-27 05:27:19,090][105692] Updated weights for policy 0, policy_version 1929436 (0.0009) [2023-12-27 05:27:19,141][105692] Updated weights for policy 0, policy_version 1929446 (0.0009) [2023-12-27 05:27:19,188][105692] Updated weights for policy 0, policy_version 1929456 (0.0009) [2023-12-27 05:27:19,652][105620] Updated weights for policy 1, policy_version 1934169 (0.0009) [2023-12-27 05:27:19,703][105620] Updated weights for policy 1, policy_version 1934179 (0.0009) [2023-12-27 05:27:19,765][105620] Updated weights for policy 1, policy_version 1934189 (0.0010) [2023-12-27 05:27:19,991][105692] Updated weights for policy 0, policy_version 1929466 (0.0008) [2023-12-27 05:27:20,054][105692] Updated weights for policy 0, policy_version 1929476 (0.0008) [2023-12-27 05:27:20,118][105692] Updated weights for policy 0, policy_version 1929486 (0.0008) [2023-12-27 05:27:20,474][105620] Updated weights for policy 1, policy_version 1934199 (0.0009) [2023-12-27 05:27:20,530][105620] Updated weights for policy 1, policy_version 1934209 (0.0009) [2023-12-27 05:27:20,593][105620] Updated weights for policy 1, policy_version 1934219 (0.0009) [2023-12-27 05:27:20,946][105692] Updated weights for policy 0, policy_version 1929496 (0.0008) [2023-12-27 05:27:21,005][105692] Updated weights for policy 0, policy_version 1929506 (0.0008) [2023-12-27 05:27:21,062][104569] Fps is (10 sec: 18022.5, 60 sec: 19114.7, 300 sec: 19327.6). Total num frames: 989249536. Throughput: 0: 9499.7, 1: 9402.9. Samples: 989244468. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:27:21,063][104569] Avg episode reward: [(0, '7814.994'), (1, '9072.750')] [2023-12-27 05:27:21,072][105692] Updated weights for policy 0, policy_version 1929516 (0.0008) [2023-12-27 05:27:21,347][105620] Updated weights for policy 1, policy_version 1934229 (0.0008) [2023-12-27 05:27:21,417][105620] Updated weights for policy 1, policy_version 1934239 (0.0007) [2023-12-27 05:27:21,485][105620] Updated weights for policy 1, policy_version 1934249 (0.0007) [2023-12-27 05:27:21,868][105692] Updated weights for policy 0, policy_version 1929526 (0.0007) [2023-12-27 05:27:21,924][105692] Updated weights for policy 0, policy_version 1929536 (0.0009) [2023-12-27 05:27:21,971][105692] Updated weights for policy 0, policy_version 1929546 (0.0009) [2023-12-27 05:27:22,262][105620] Updated weights for policy 1, policy_version 1934259 (0.0007) [2023-12-27 05:27:22,330][105620] Updated weights for policy 1, policy_version 1934269 (0.0006) [2023-12-27 05:27:22,398][105620] Updated weights for policy 1, policy_version 1934279 (0.0009) [2023-12-27 05:27:22,656][105692] Updated weights for policy 0, policy_version 1929556 (0.0009) [2023-12-27 05:27:22,723][105692] Updated weights for policy 0, policy_version 1929566 (0.0007) [2023-12-27 05:27:22,786][105692] Updated weights for policy 0, policy_version 1929576 (0.0008) [2023-12-27 05:27:23,147][105620] Updated weights for policy 1, policy_version 1934289 (0.0009) [2023-12-27 05:27:23,212][105620] Updated weights for policy 1, policy_version 1934299 (0.0005) [2023-12-27 05:27:23,282][105620] Updated weights for policy 1, policy_version 1934309 (0.0005) [2023-12-27 05:27:23,348][105620] Updated weights for policy 1, policy_version 1934319 (0.0005) [2023-12-27 05:27:23,651][105692] Updated weights for policy 0, policy_version 1929586 (0.0009) [2023-12-27 05:27:23,705][105692] Updated weights for policy 0, policy_version 1929596 (0.0010) [2023-12-27 05:27:23,766][105692] Updated weights for policy 0, policy_version 1929606 (0.0009) [2023-12-27 05:27:23,831][105692] Updated weights for policy 0, policy_version 1929616 (0.0010) [2023-12-27 05:27:23,870][105620] Updated weights for policy 1, policy_version 1934329 (0.0008) [2023-12-27 05:27:23,928][105620] Updated weights for policy 1, policy_version 1934339 (0.0009) [2023-12-27 05:27:23,989][105620] Updated weights for policy 1, policy_version 1934349 (0.0009) [2023-12-27 05:27:24,597][105692] Updated weights for policy 0, policy_version 1929626 (0.0009) [2023-12-27 05:27:24,660][105692] Updated weights for policy 0, policy_version 1929636 (0.0009) [2023-12-27 05:27:24,708][105692] Updated weights for policy 0, policy_version 1929646 (0.0007) [2023-12-27 05:27:24,709][105620] Updated weights for policy 1, policy_version 1934359 (0.0007) [2023-12-27 05:27:24,773][105620] Updated weights for policy 1, policy_version 1934369 (0.0009) [2023-12-27 05:27:24,835][105620] Updated weights for policy 1, policy_version 1934379 (0.0009) [2023-12-27 05:27:25,482][105692] Updated weights for policy 0, policy_version 1929656 (0.0008) [2023-12-27 05:27:25,540][105692] Updated weights for policy 0, policy_version 1929666 (0.0009) [2023-12-27 05:27:25,593][105620] Updated weights for policy 1, policy_version 1934389 (0.0008) [2023-12-27 05:27:25,603][105692] Updated weights for policy 0, policy_version 1929676 (0.0008) [2023-12-27 05:27:25,650][105620] Updated weights for policy 1, policy_version 1934399 (0.0008) [2023-12-27 05:27:25,697][105620] Updated weights for policy 1, policy_version 1934409 (0.0008) [2023-12-27 05:27:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19114.7, 300 sec: 19327.6). Total num frames: 989347840. Throughput: 0: 9412.5, 1: 9494.2. Samples: 989356100. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:27:26,062][104569] Avg episode reward: [(0, '8262.441'), (1, '9163.293')] [2023-12-27 05:27:26,206][105692] Updated weights for policy 0, policy_version 1929686 (0.0005) [2023-12-27 05:27:26,257][105692] Updated weights for policy 0, policy_version 1929696 (0.0005) [2023-12-27 05:27:26,307][105692] Updated weights for policy 0, policy_version 1929706 (0.0005) [2023-12-27 05:27:26,595][105620] Updated weights for policy 1, policy_version 1934419 (0.0008) [2023-12-27 05:27:26,646][105620] Updated weights for policy 1, policy_version 1934429 (0.0009) [2023-12-27 05:27:26,707][105620] Updated weights for policy 1, policy_version 1934439 (0.0009) [2023-12-27 05:27:26,873][105692] Updated weights for policy 0, policy_version 1929716 (0.0005) [2023-12-27 05:27:26,937][105692] Updated weights for policy 0, policy_version 1929726 (0.0005) [2023-12-27 05:27:27,003][105692] Updated weights for policy 0, policy_version 1929736 (0.0008) [2023-12-27 05:27:27,473][105620] Updated weights for policy 1, policy_version 1934449 (0.0009) [2023-12-27 05:27:27,534][105620] Updated weights for policy 1, policy_version 1934459 (0.0009) [2023-12-27 05:27:27,587][105620] Updated weights for policy 1, policy_version 1934469 (0.0009) [2023-12-27 05:27:27,654][105620] Updated weights for policy 1, policy_version 1934479 (0.0010) [2023-12-27 05:27:27,682][105692] Updated weights for policy 0, policy_version 1929746 (0.0008) [2023-12-27 05:27:27,734][105692] Updated weights for policy 0, policy_version 1929756 (0.0005) [2023-12-27 05:27:27,784][105692] Updated weights for policy 0, policy_version 1929766 (0.0008) [2023-12-27 05:27:27,831][105692] Updated weights for policy 0, policy_version 1929776 (0.0009) [2023-12-27 05:27:28,387][105620] Updated weights for policy 1, policy_version 1934489 (0.0008) [2023-12-27 05:27:28,440][105620] Updated weights for policy 1, policy_version 1934499 (0.0006) [2023-12-27 05:27:28,498][105620] Updated weights for policy 1, policy_version 1934509 (0.0010) [2023-12-27 05:27:28,567][105692] Updated weights for policy 0, policy_version 1929786 (0.0008) [2023-12-27 05:27:28,628][105692] Updated weights for policy 0, policy_version 1929796 (0.0009) [2023-12-27 05:27:28,684][105692] Updated weights for policy 0, policy_version 1929806 (0.0007) [2023-12-27 05:27:29,223][105620] Updated weights for policy 1, policy_version 1934519 (0.0009) [2023-12-27 05:27:29,291][105620] Updated weights for policy 1, policy_version 1934529 (0.0008) [2023-12-27 05:27:29,361][105620] Updated weights for policy 1, policy_version 1934539 (0.0009) [2023-12-27 05:27:29,442][105692] Updated weights for policy 0, policy_version 1929816 (0.0006) [2023-12-27 05:27:29,501][105692] Updated weights for policy 0, policy_version 1929826 (0.0007) [2023-12-27 05:27:29,553][105692] Updated weights for policy 0, policy_version 1929836 (0.0009) [2023-12-27 05:27:30,069][105620] Updated weights for policy 1, policy_version 1934549 (0.0007) [2023-12-27 05:27:30,120][105620] Updated weights for policy 1, policy_version 1934559 (0.0008) [2023-12-27 05:27:30,180][105620] Updated weights for policy 1, policy_version 1934569 (0.0008) [2023-12-27 05:27:30,320][105692] Updated weights for policy 0, policy_version 1929846 (0.0007) [2023-12-27 05:27:30,380][105692] Updated weights for policy 0, policy_version 1929856 (0.0005) [2023-12-27 05:27:30,441][105692] Updated weights for policy 0, policy_version 1929866 (0.0006) [2023-12-27 05:27:30,976][105692] Updated weights for policy 0, policy_version 1929876 (0.0007) [2023-12-27 05:27:31,034][105692] Updated weights for policy 0, policy_version 1929886 (0.0008) [2023-12-27 05:27:31,034][105620] Updated weights for policy 1, policy_version 1934579 (0.0009) [2023-12-27 05:27:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 18841.6, 300 sec: 19299.8). Total num frames: 989437952. Throughput: 0: 9510.2, 1: 9480.8. Samples: 989415884. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 05:27:31,063][104569] Avg episode reward: [(0, '7989.464'), (1, '9346.275')] [2023-12-27 05:27:31,096][105692] Updated weights for policy 0, policy_version 1929896 (0.0006) [2023-12-27 05:27:31,100][105620] Updated weights for policy 1, policy_version 1934589 (0.0009) [2023-12-27 05:27:31,143][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001929904_494125056.pth... [2023-12-27 05:27:31,148][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001928752_493830144.pth [2023-12-27 05:27:31,162][105620] Updated weights for policy 1, policy_version 1934599 (0.0008) [2023-12-27 05:27:31,222][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001934608_495329280.pth... [2023-12-27 05:27:31,226][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001933520_495050752.pth [2023-12-27 05:27:31,833][105692] Updated weights for policy 0, policy_version 1929906 (0.0006) [2023-12-27 05:27:31,883][105692] Updated weights for policy 0, policy_version 1929916 (0.0005) [2023-12-27 05:27:31,926][105620] Updated weights for policy 1, policy_version 1934609 (0.0007) [2023-12-27 05:27:31,929][105692] Updated weights for policy 0, policy_version 1929926 (0.0005) [2023-12-27 05:27:31,976][105620] Updated weights for policy 1, policy_version 1934619 (0.0009) [2023-12-27 05:27:31,981][105692] Updated weights for policy 0, policy_version 1929936 (0.0005) [2023-12-27 05:27:32,033][105620] Updated weights for policy 1, policy_version 1934629 (0.0008) [2023-12-27 05:27:32,089][105620] Updated weights for policy 1, policy_version 1934639 (0.0007) [2023-12-27 05:27:32,671][105692] Updated weights for policy 0, policy_version 1929946 (0.0005) [2023-12-27 05:27:32,728][105692] Updated weights for policy 0, policy_version 1929956 (0.0007) [2023-12-27 05:27:32,787][105692] Updated weights for policy 0, policy_version 1929966 (0.0011) [2023-12-27 05:27:32,830][105620] Updated weights for policy 1, policy_version 1934649 (0.0005) [2023-12-27 05:27:32,895][105620] Updated weights for policy 1, policy_version 1934659 (0.0005) [2023-12-27 05:27:32,963][105620] Updated weights for policy 1, policy_version 1934669 (0.0006) [2023-12-27 05:27:33,399][105692] Updated weights for policy 0, policy_version 1929976 (0.0006) [2023-12-27 05:27:33,447][105692] Updated weights for policy 0, policy_version 1929986 (0.0005) [2023-12-27 05:27:33,498][105692] Updated weights for policy 0, policy_version 1929996 (0.0005) [2023-12-27 05:27:33,611][105620] Updated weights for policy 1, policy_version 1934679 (0.0008) [2023-12-27 05:27:33,680][105620] Updated weights for policy 1, policy_version 1934689 (0.0009) [2023-12-27 05:27:33,748][105620] Updated weights for policy 1, policy_version 1934699 (0.0008) [2023-12-27 05:27:34,035][105692] Updated weights for policy 0, policy_version 1930006 (0.0006) [2023-12-27 05:27:34,095][105692] Updated weights for policy 0, policy_version 1930016 (0.0007) [2023-12-27 05:27:34,157][105692] Updated weights for policy 0, policy_version 1930026 (0.0007) [2023-12-27 05:27:34,544][105620] Updated weights for policy 1, policy_version 1934709 (0.0009) [2023-12-27 05:27:34,614][105620] Updated weights for policy 1, policy_version 1934719 (0.0008) [2023-12-27 05:27:34,672][105620] Updated weights for policy 1, policy_version 1934729 (0.0008) [2023-12-27 05:27:34,858][105692] Updated weights for policy 0, policy_version 1930036 (0.0009) [2023-12-27 05:27:34,915][105692] Updated weights for policy 0, policy_version 1930046 (0.0009) [2023-12-27 05:27:34,973][105692] Updated weights for policy 0, policy_version 1930057 (0.0010) [2023-12-27 05:27:35,303][105620] Updated weights for policy 1, policy_version 1934739 (0.0009) [2023-12-27 05:27:35,356][105620] Updated weights for policy 1, policy_version 1934749 (0.0009) [2023-12-27 05:27:35,407][105620] Updated weights for policy 1, policy_version 1934759 (0.0009) [2023-12-27 05:27:35,775][105692] Updated weights for policy 0, policy_version 1930067 (0.0010) [2023-12-27 05:27:35,834][105692] Updated weights for policy 0, policy_version 1930078 (0.0007) [2023-12-27 05:27:35,892][105692] Updated weights for policy 0, policy_version 1930088 (0.0007) [2023-12-27 05:27:36,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19114.7, 300 sec: 19355.3). Total num frames: 989544448. Throughput: 0: 9588.3, 1: 9404.5. Samples: 989533056. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:27:36,062][104569] Avg episode reward: [(0, '8084.497'), (1, '9255.345')] [2023-12-27 05:27:36,167][105620] Updated weights for policy 1, policy_version 1934769 (0.0009) [2023-12-27 05:27:36,223][105620] Updated weights for policy 1, policy_version 1934779 (0.0007) [2023-12-27 05:27:36,275][105620] Updated weights for policy 1, policy_version 1934789 (0.0010) [2023-12-27 05:27:36,337][105620] Updated weights for policy 1, policy_version 1934799 (0.0010) [2023-12-27 05:27:36,663][105692] Updated weights for policy 0, policy_version 1930098 (0.0008) [2023-12-27 05:27:36,716][105692] Updated weights for policy 0, policy_version 1930108 (0.0008) [2023-12-27 05:27:36,773][105692] Updated weights for policy 0, policy_version 1930118 (0.0008) [2023-12-27 05:27:36,826][105692] Updated weights for policy 0, policy_version 1930128 (0.0010) [2023-12-27 05:27:37,041][105620] Updated weights for policy 1, policy_version 1934809 (0.0009) [2023-12-27 05:27:37,101][105620] Updated weights for policy 1, policy_version 1934819 (0.0009) [2023-12-27 05:27:37,160][105620] Updated weights for policy 1, policy_version 1934829 (0.0009) [2023-12-27 05:27:37,647][105692] Updated weights for policy 0, policy_version 1930138 (0.0008) [2023-12-27 05:27:37,711][105692] Updated weights for policy 0, policy_version 1930148 (0.0008) [2023-12-27 05:27:37,770][105692] Updated weights for policy 0, policy_version 1930158 (0.0005) [2023-12-27 05:27:37,893][105620] Updated weights for policy 1, policy_version 1934839 (0.0009) [2023-12-27 05:27:37,948][105620] Updated weights for policy 1, policy_version 1934850 (0.0010) [2023-12-27 05:27:37,997][105620] Updated weights for policy 1, policy_version 1934860 (0.0008) [2023-12-27 05:27:38,441][105692] Updated weights for policy 0, policy_version 1930168 (0.0009) [2023-12-27 05:27:38,508][105692] Updated weights for policy 0, policy_version 1930178 (0.0010) [2023-12-27 05:27:38,568][105692] Updated weights for policy 0, policy_version 1930188 (0.0010) [2023-12-27 05:27:38,655][105620] Updated weights for policy 1, policy_version 1934870 (0.0007) [2023-12-27 05:27:38,721][105620] Updated weights for policy 1, policy_version 1934880 (0.0007) [2023-12-27 05:27:38,786][105620] Updated weights for policy 1, policy_version 1934890 (0.0009) [2023-12-27 05:27:39,324][105692] Updated weights for policy 0, policy_version 1930198 (0.0010) [2023-12-27 05:27:39,391][105692] Updated weights for policy 0, policy_version 1930208 (0.0009) [2023-12-27 05:27:39,455][105692] Updated weights for policy 0, policy_version 1930218 (0.0008) [2023-12-27 05:27:39,551][105620] Updated weights for policy 1, policy_version 1934900 (0.0008) [2023-12-27 05:27:39,614][105620] Updated weights for policy 1, policy_version 1934910 (0.0008) [2023-12-27 05:27:39,675][105620] Updated weights for policy 1, policy_version 1934920 (0.0011) [2023-12-27 05:27:40,248][105692] Updated weights for policy 0, policy_version 1930228 (0.0009) [2023-12-27 05:27:40,315][105692] Updated weights for policy 0, policy_version 1930238 (0.0009) [2023-12-27 05:27:40,382][105692] Updated weights for policy 0, policy_version 1930248 (0.0008) [2023-12-27 05:27:40,404][105620] Updated weights for policy 1, policy_version 1934930 (0.0010) [2023-12-27 05:27:40,463][105620] Updated weights for policy 1, policy_version 1934940 (0.0009) [2023-12-27 05:27:40,527][105620] Updated weights for policy 1, policy_version 1934950 (0.0010) [2023-12-27 05:27:40,586][105620] Updated weights for policy 1, policy_version 1934960 (0.0010) [2023-12-27 05:27:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 18978.1, 300 sec: 19327.6). Total num frames: 989634560. Throughput: 0: 9566.2, 1: 9428.4. Samples: 989646492. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:27:41,063][104569] Avg episode reward: [(0, '8442.315'), (1, '9163.231')] [2023-12-27 05:27:41,163][105692] Updated weights for policy 0, policy_version 1930258 (0.0009) [2023-12-27 05:27:41,222][105692] Updated weights for policy 0, policy_version 1930268 (0.0008) [2023-12-27 05:27:41,290][105692] Updated weights for policy 0, policy_version 1930278 (0.0007) [2023-12-27 05:27:41,299][105620] Updated weights for policy 1, policy_version 1934970 (0.0011) [2023-12-27 05:27:41,355][105692] Updated weights for policy 0, policy_version 1930288 (0.0006) [2023-12-27 05:27:41,360][105620] Updated weights for policy 1, policy_version 1934980 (0.0011) [2023-12-27 05:27:41,428][105620] Updated weights for policy 1, policy_version 1934990 (0.0008) [2023-12-27 05:27:42,120][105692] Updated weights for policy 0, policy_version 1930298 (0.0009) [2023-12-27 05:27:42,158][105620] Updated weights for policy 1, policy_version 1935000 (0.0008) [2023-12-27 05:27:42,173][105692] Updated weights for policy 0, policy_version 1930308 (0.0006) [2023-12-27 05:27:42,218][105620] Updated weights for policy 1, policy_version 1935010 (0.0008) [2023-12-27 05:27:42,224][105692] Updated weights for policy 0, policy_version 1930318 (0.0008) [2023-12-27 05:27:42,280][105620] Updated weights for policy 1, policy_version 1935020 (0.0009) [2023-12-27 05:27:42,976][105620] Updated weights for policy 1, policy_version 1935030 (0.0007) [2023-12-27 05:27:43,029][105692] Updated weights for policy 0, policy_version 1930328 (0.0005) [2023-12-27 05:27:43,034][105620] Updated weights for policy 1, policy_version 1935040 (0.0008) [2023-12-27 05:27:43,085][105692] Updated weights for policy 0, policy_version 1930338 (0.0006) [2023-12-27 05:27:43,092][105620] Updated weights for policy 1, policy_version 1935050 (0.0010) [2023-12-27 05:27:43,143][105692] Updated weights for policy 0, policy_version 1930348 (0.0006) [2023-12-27 05:27:43,658][105620] Updated weights for policy 1, policy_version 1935060 (0.0009) [2023-12-27 05:27:43,734][105620] Updated weights for policy 1, policy_version 1935070 (0.0009) [2023-12-27 05:27:43,757][105692] Updated weights for policy 0, policy_version 1930358 (0.0008) [2023-12-27 05:27:43,787][105620] Updated weights for policy 1, policy_version 1935080 (0.0011) [2023-12-27 05:27:43,826][105692] Updated weights for policy 0, policy_version 1930368 (0.0010) [2023-12-27 05:27:43,887][105692] Updated weights for policy 0, policy_version 1930378 (0.0010) [2023-12-27 05:27:44,437][105620] Updated weights for policy 1, policy_version 1935090 (0.0010) [2023-12-27 05:27:44,444][105692] Updated weights for policy 0, policy_version 1930388 (0.0008) [2023-12-27 05:27:44,482][105620] Updated weights for policy 1, policy_version 1935100 (0.0008) [2023-12-27 05:27:44,491][105692] Updated weights for policy 0, policy_version 1930398 (0.0007) [2023-12-27 05:27:44,531][105620] Updated weights for policy 1, policy_version 1935110 (0.0010) [2023-12-27 05:27:44,550][105692] Updated weights for policy 0, policy_version 1930408 (0.0006) [2023-12-27 05:27:44,579][105620] Updated weights for policy 1, policy_version 1935120 (0.0009) [2023-12-27 05:27:45,284][105692] Updated weights for policy 0, policy_version 1930418 (0.0007) [2023-12-27 05:27:45,343][105692] Updated weights for policy 0, policy_version 1930428 (0.0008) [2023-12-27 05:27:45,373][105620] Updated weights for policy 1, policy_version 1935130 (0.0009) [2023-12-27 05:27:45,397][105692] Updated weights for policy 0, policy_version 1930438 (0.0007) [2023-12-27 05:27:45,435][105620] Updated weights for policy 1, policy_version 1935140 (0.0007) [2023-12-27 05:27:45,458][105692] Updated weights for policy 0, policy_version 1930448 (0.0008) [2023-12-27 05:27:45,498][105620] Updated weights for policy 1, policy_version 1935150 (0.0008) [2023-12-27 05:27:46,062][104569] Fps is (10 sec: 18841.0, 60 sec: 19114.6, 300 sec: 19327.6). Total num frames: 989732864. Throughput: 0: 9579.9, 1: 9478.2. Samples: 989704968. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:27:46,063][104569] Avg episode reward: [(0, '8173.191'), (1, '9256.432')] [2023-12-27 05:27:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001930448_494264320.pth... [2023-12-27 05:27:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001935152_495468544.pth... [2023-12-27 05:27:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001929328_493977600.pth [2023-12-27 05:27:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001934064_495190016.pth [2023-12-27 05:27:46,222][105620] Updated weights for policy 1, policy_version 1935160 (0.0006) [2023-12-27 05:27:46,238][105692] Updated weights for policy 0, policy_version 1930458 (0.0009) [2023-12-27 05:27:46,275][105620] Updated weights for policy 1, policy_version 1935170 (0.0009) [2023-12-27 05:27:46,283][105692] Updated weights for policy 0, policy_version 1930468 (0.0009) [2023-12-27 05:27:46,330][105620] Updated weights for policy 1, policy_version 1935180 (0.0010) [2023-12-27 05:27:46,338][105692] Updated weights for policy 0, policy_version 1930478 (0.0010) [2023-12-27 05:27:47,016][105620] Updated weights for policy 1, policy_version 1935190 (0.0010) [2023-12-27 05:27:47,067][105620] Updated weights for policy 1, policy_version 1935200 (0.0010) [2023-12-27 05:27:47,091][105692] Updated weights for policy 0, policy_version 1930488 (0.0006) [2023-12-27 05:27:47,128][105620] Updated weights for policy 1, policy_version 1935210 (0.0009) [2023-12-27 05:27:47,141][105692] Updated weights for policy 0, policy_version 1930498 (0.0006) [2023-12-27 05:27:47,185][105692] Updated weights for policy 0, policy_version 1930508 (0.0005) [2023-12-27 05:27:47,800][105620] Updated weights for policy 1, policy_version 1935220 (0.0006) [2023-12-27 05:27:47,861][105620] Updated weights for policy 1, policy_version 1935230 (0.0005) [2023-12-27 05:27:47,902][105692] Updated weights for policy 0, policy_version 1930518 (0.0007) [2023-12-27 05:27:47,914][105620] Updated weights for policy 1, policy_version 1935240 (0.0005) [2023-12-27 05:27:47,948][105692] Updated weights for policy 0, policy_version 1930528 (0.0008) [2023-12-27 05:27:48,003][105692] Updated weights for policy 0, policy_version 1930538 (0.0005) [2023-12-27 05:27:48,452][105620] Updated weights for policy 1, policy_version 1935250 (0.0005) [2023-12-27 05:27:48,514][105620] Updated weights for policy 1, policy_version 1935260 (0.0005) [2023-12-27 05:27:48,585][105620] Updated weights for policy 1, policy_version 1935270 (0.0005) [2023-12-27 05:27:48,647][105620] Updated weights for policy 1, policy_version 1935280 (0.0007) [2023-12-27 05:27:48,804][105692] Updated weights for policy 0, policy_version 1930548 (0.0007) [2023-12-27 05:27:48,860][105692] Updated weights for policy 0, policy_version 1930558 (0.0009) [2023-12-27 05:27:48,913][105692] Updated weights for policy 0, policy_version 1930568 (0.0009) [2023-12-27 05:27:49,237][105620] Updated weights for policy 1, policy_version 1935290 (0.0010) [2023-12-27 05:27:49,305][105620] Updated weights for policy 1, policy_version 1935300 (0.0010) [2023-12-27 05:27:49,378][105620] Updated weights for policy 1, policy_version 1935310 (0.0009) [2023-12-27 05:27:49,749][105692] Updated weights for policy 0, policy_version 1930578 (0.0010) [2023-12-27 05:27:49,805][105692] Updated weights for policy 0, policy_version 1930588 (0.0009) [2023-12-27 05:27:49,863][105692] Updated weights for policy 0, policy_version 1930598 (0.0007) [2023-12-27 05:27:49,928][105692] Updated weights for policy 0, policy_version 1930608 (0.0008) [2023-12-27 05:27:50,116][105620] Updated weights for policy 1, policy_version 1935320 (0.0009) [2023-12-27 05:27:50,181][105620] Updated weights for policy 1, policy_version 1935330 (0.0009) [2023-12-27 05:27:50,237][105620] Updated weights for policy 1, policy_version 1935340 (0.0006) [2023-12-27 05:27:50,632][105692] Updated weights for policy 0, policy_version 1930618 (0.0010) [2023-12-27 05:27:50,684][105692] Updated weights for policy 0, policy_version 1930628 (0.0010) [2023-12-27 05:27:50,738][105692] Updated weights for policy 0, policy_version 1930638 (0.0007) [2023-12-27 05:27:50,928][105620] Updated weights for policy 1, policy_version 1935350 (0.0009) [2023-12-27 05:27:50,986][105620] Updated weights for policy 1, policy_version 1935360 (0.0010) [2023-12-27 05:27:51,056][105620] Updated weights for policy 1, policy_version 1935371 (0.0007) [2023-12-27 05:27:51,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19114.7, 300 sec: 19327.6). Total num frames: 989831168. Throughput: 0: 9619.3, 1: 9535.3. Samples: 989823496. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:27:51,062][104569] Avg episode reward: [(0, '7903.123'), (1, '9256.430')] [2023-12-27 05:27:51,453][105692] Updated weights for policy 0, policy_version 1930648 (0.0009) [2023-12-27 05:27:51,512][105692] Updated weights for policy 0, policy_version 1930658 (0.0010) [2023-12-27 05:27:51,566][105692] Updated weights for policy 0, policy_version 1930668 (0.0008) [2023-12-27 05:27:51,714][105620] Updated weights for policy 1, policy_version 1935381 (0.0006) [2023-12-27 05:27:51,779][105620] Updated weights for policy 1, policy_version 1935391 (0.0010) [2023-12-27 05:27:51,833][105620] Updated weights for policy 1, policy_version 1935401 (0.0009) [2023-12-27 05:27:52,271][105692] Updated weights for policy 0, policy_version 1930678 (0.0008) [2023-12-27 05:27:52,329][105692] Updated weights for policy 0, policy_version 1930688 (0.0009) [2023-12-27 05:27:52,390][105692] Updated weights for policy 0, policy_version 1930698 (0.0009) [2023-12-27 05:27:52,518][105620] Updated weights for policy 1, policy_version 1935411 (0.0009) [2023-12-27 05:27:52,576][105620] Updated weights for policy 1, policy_version 1935421 (0.0006) [2023-12-27 05:27:52,642][105620] Updated weights for policy 1, policy_version 1935431 (0.0008) [2023-12-27 05:27:53,111][105692] Updated weights for policy 0, policy_version 1930708 (0.0007) [2023-12-27 05:27:53,175][105692] Updated weights for policy 0, policy_version 1930718 (0.0006) [2023-12-27 05:27:53,230][105692] Updated weights for policy 0, policy_version 1930728 (0.0006) [2023-12-27 05:27:53,352][105620] Updated weights for policy 1, policy_version 1935441 (0.0007) [2023-12-27 05:27:53,400][105620] Updated weights for policy 1, policy_version 1935451 (0.0009) [2023-12-27 05:27:53,447][105620] Updated weights for policy 1, policy_version 1935461 (0.0009) [2023-12-27 05:27:53,499][105620] Updated weights for policy 1, policy_version 1935471 (0.0010) [2023-12-27 05:27:53,791][105692] Updated weights for policy 0, policy_version 1930738 (0.0007) [2023-12-27 05:27:53,838][105692] Updated weights for policy 0, policy_version 1930748 (0.0007) [2023-12-27 05:27:53,894][105692] Updated weights for policy 0, policy_version 1930758 (0.0006) [2023-12-27 05:27:53,949][105692] Updated weights for policy 0, policy_version 1930768 (0.0006) [2023-12-27 05:27:54,421][105620] Updated weights for policy 1, policy_version 1935481 (0.0010) [2023-12-27 05:27:54,477][105620] Updated weights for policy 1, policy_version 1935491 (0.0008) [2023-12-27 05:27:54,509][105692] Updated weights for policy 0, policy_version 1930778 (0.0006) [2023-12-27 05:27:54,533][105620] Updated weights for policy 1, policy_version 1935501 (0.0008) [2023-12-27 05:27:54,577][105692] Updated weights for policy 0, policy_version 1930788 (0.0006) [2023-12-27 05:27:54,639][105692] Updated weights for policy 0, policy_version 1930798 (0.0006) [2023-12-27 05:27:55,268][105620] Updated weights for policy 1, policy_version 1935511 (0.0009) [2023-12-27 05:27:55,320][105620] Updated weights for policy 1, policy_version 1935521 (0.0010) [2023-12-27 05:27:55,367][105620] Updated weights for policy 1, policy_version 1935531 (0.0010) [2023-12-27 05:27:55,377][105692] Updated weights for policy 0, policy_version 1930808 (0.0006) [2023-12-27 05:27:55,436][105692] Updated weights for policy 0, policy_version 1930818 (0.0007) [2023-12-27 05:27:55,499][105692] Updated weights for policy 0, policy_version 1930828 (0.0006) [2023-12-27 05:27:56,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19114.6, 300 sec: 19355.3). Total num frames: 989929472. Throughput: 0: 9728.7, 1: 9572.0. Samples: 989942424. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:27:56,063][104569] Avg episode reward: [(0, '8716.817'), (1, '9254.012')] [2023-12-27 05:27:56,095][105620] Updated weights for policy 1, policy_version 1935541 (0.0010) [2023-12-27 05:27:56,157][105620] Updated weights for policy 1, policy_version 1935551 (0.0011) [2023-12-27 05:27:56,212][105620] Updated weights for policy 1, policy_version 1935561 (0.0010) [2023-12-27 05:27:56,237][105692] Updated weights for policy 0, policy_version 1930838 (0.0007) [2023-12-27 05:27:56,285][105692] Updated weights for policy 0, policy_version 1930848 (0.0007) [2023-12-27 05:27:56,339][105692] Updated weights for policy 0, policy_version 1930858 (0.0006) [2023-12-27 05:27:56,947][105620] Updated weights for policy 1, policy_version 1935571 (0.0010) [2023-12-27 05:27:57,008][105620] Updated weights for policy 1, policy_version 1935581 (0.0010) [2023-12-27 05:27:57,011][105692] Updated weights for policy 0, policy_version 1930868 (0.0005) [2023-12-27 05:27:57,055][105620] Updated weights for policy 1, policy_version 1935591 (0.0010) [2023-12-27 05:27:57,066][105692] Updated weights for policy 0, policy_version 1930878 (0.0005) [2023-12-27 05:27:57,125][105692] Updated weights for policy 0, policy_version 1930888 (0.0005) [2023-12-27 05:27:57,643][105692] Updated weights for policy 0, policy_version 1930898 (0.0005) [2023-12-27 05:27:57,707][105692] Updated weights for policy 0, policy_version 1930908 (0.0006) [2023-12-27 05:27:57,761][105692] Updated weights for policy 0, policy_version 1930918 (0.0007) [2023-12-27 05:27:57,783][105620] Updated weights for policy 1, policy_version 1935601 (0.0010) [2023-12-27 05:27:57,804][105692] Updated weights for policy 0, policy_version 1930928 (0.0005) [2023-12-27 05:27:57,848][105620] Updated weights for policy 1, policy_version 1935611 (0.0010) [2023-12-27 05:27:57,914][105620] Updated weights for policy 1, policy_version 1935621 (0.0010) [2023-12-27 05:27:57,988][105620] Updated weights for policy 1, policy_version 1935631 (0.0010) [2023-12-27 05:27:58,385][105692] Updated weights for policy 0, policy_version 1930938 (0.0008) [2023-12-27 05:27:58,450][105692] Updated weights for policy 0, policy_version 1930948 (0.0008) [2023-12-27 05:27:58,509][105692] Updated weights for policy 0, policy_version 1930958 (0.0008) [2023-12-27 05:27:58,737][105620] Updated weights for policy 1, policy_version 1935641 (0.0008) [2023-12-27 05:27:58,810][105620] Updated weights for policy 1, policy_version 1935651 (0.0007) [2023-12-27 05:27:58,877][105620] Updated weights for policy 1, policy_version 1935661 (0.0008) [2023-12-27 05:27:59,303][105692] Updated weights for policy 0, policy_version 1930968 (0.0008) [2023-12-27 05:27:59,373][105692] Updated weights for policy 0, policy_version 1930978 (0.0008) [2023-12-27 05:27:59,437][105692] Updated weights for policy 0, policy_version 1930988 (0.0008) [2023-12-27 05:27:59,645][105620] Updated weights for policy 1, policy_version 1935671 (0.0006) [2023-12-27 05:27:59,703][105620] Updated weights for policy 1, policy_version 1935681 (0.0005) [2023-12-27 05:27:59,760][105620] Updated weights for policy 1, policy_version 1935691 (0.0008) [2023-12-27 05:28:00,244][105692] Updated weights for policy 0, policy_version 1930998 (0.0008) [2023-12-27 05:28:00,306][105692] Updated weights for policy 0, policy_version 1931008 (0.0008) [2023-12-27 05:28:00,364][105692] Updated weights for policy 0, policy_version 1931018 (0.0008) [2023-12-27 05:28:00,453][105620] Updated weights for policy 1, policy_version 1935701 (0.0009) [2023-12-27 05:28:00,521][105620] Updated weights for policy 1, policy_version 1935711 (0.0010) [2023-12-27 05:28:00,585][105620] Updated weights for policy 1, policy_version 1935721 (0.0010) [2023-12-27 05:28:01,049][105692] Updated weights for policy 0, policy_version 1931028 (0.0008) [2023-12-27 05:28:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19327.6). Total num frames: 990027776. Throughput: 0: 9790.6, 1: 9613.7. Samples: 990003000. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:28:01,063][104569] Avg episode reward: [(0, '8898.825'), (1, '9253.980')] [2023-12-27 05:28:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001935728_495616000.pth... [2023-12-27 05:28:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001934608_495329280.pth [2023-12-27 05:28:01,107][105692] Updated weights for policy 0, policy_version 1931038 (0.0008) [2023-12-27 05:28:01,166][105692] Updated weights for policy 0, policy_version 1931048 (0.0008) [2023-12-27 05:28:01,218][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001931056_494419968.pth... [2023-12-27 05:28:01,223][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001929904_494125056.pth [2023-12-27 05:28:01,279][105620] Updated weights for policy 1, policy_version 1935731 (0.0009) [2023-12-27 05:28:01,327][105620] Updated weights for policy 1, policy_version 1935741 (0.0006) [2023-12-27 05:28:01,395][105620] Updated weights for policy 1, policy_version 1935751 (0.0008) [2023-12-27 05:28:01,809][105692] Updated weights for policy 0, policy_version 1931058 (0.0008) [2023-12-27 05:28:01,878][105692] Updated weights for policy 0, policy_version 1931068 (0.0008) [2023-12-27 05:28:01,942][105692] Updated weights for policy 0, policy_version 1931078 (0.0007) [2023-12-27 05:28:02,007][105692] Updated weights for policy 0, policy_version 1931088 (0.0008) [2023-12-27 05:28:02,063][105620] Updated weights for policy 1, policy_version 1935761 (0.0010) [2023-12-27 05:28:02,126][105620] Updated weights for policy 1, policy_version 1935771 (0.0005) [2023-12-27 05:28:02,194][105620] Updated weights for policy 1, policy_version 1935781 (0.0006) [2023-12-27 05:28:02,257][105620] Updated weights for policy 1, policy_version 1935791 (0.0006) [2023-12-27 05:28:02,760][105692] Updated weights for policy 0, policy_version 1931098 (0.0008) [2023-12-27 05:28:02,774][105620] Updated weights for policy 1, policy_version 1935801 (0.0006) [2023-12-27 05:28:02,814][105692] Updated weights for policy 0, policy_version 1931108 (0.0006) [2023-12-27 05:28:02,825][105620] Updated weights for policy 1, policy_version 1935811 (0.0007) [2023-12-27 05:28:02,873][105692] Updated weights for policy 0, policy_version 1931118 (0.0010) [2023-12-27 05:28:02,876][105620] Updated weights for policy 1, policy_version 1935821 (0.0008) [2023-12-27 05:28:03,539][105692] Updated weights for policy 0, policy_version 1931128 (0.0007) [2023-12-27 05:28:03,548][105620] Updated weights for policy 1, policy_version 1935831 (0.0009) [2023-12-27 05:28:03,585][105692] Updated weights for policy 0, policy_version 1931138 (0.0006) [2023-12-27 05:28:03,611][105620] Updated weights for policy 1, policy_version 1935841 (0.0008) [2023-12-27 05:28:03,634][105692] Updated weights for policy 0, policy_version 1931148 (0.0005) [2023-12-27 05:28:03,658][105620] Updated weights for policy 1, policy_version 1935851 (0.0008) [2023-12-27 05:28:04,266][105692] Updated weights for policy 0, policy_version 1931158 (0.0006) [2023-12-27 05:28:04,333][105692] Updated weights for policy 0, policy_version 1931168 (0.0008) [2023-12-27 05:28:04,396][105692] Updated weights for policy 0, policy_version 1931178 (0.0009) [2023-12-27 05:28:04,398][105620] Updated weights for policy 1, policy_version 1935861 (0.0009) [2023-12-27 05:28:04,461][105620] Updated weights for policy 1, policy_version 1935871 (0.0011) [2023-12-27 05:28:04,527][105620] Updated weights for policy 1, policy_version 1935881 (0.0011) [2023-12-27 05:28:05,027][105692] Updated weights for policy 0, policy_version 1931188 (0.0010) [2023-12-27 05:28:05,079][105692] Updated weights for policy 0, policy_version 1931198 (0.0010) [2023-12-27 05:28:05,138][105692] Updated weights for policy 0, policy_version 1931208 (0.0011) [2023-12-27 05:28:05,268][105620] Updated weights for policy 1, policy_version 1935891 (0.0011) [2023-12-27 05:28:05,329][105620] Updated weights for policy 1, policy_version 1935901 (0.0010) [2023-12-27 05:28:05,387][105620] Updated weights for policy 1, policy_version 1935911 (0.0010) [2023-12-27 05:28:05,871][105692] Updated weights for policy 0, policy_version 1931218 (0.0011) [2023-12-27 05:28:05,932][105692] Updated weights for policy 0, policy_version 1931228 (0.0006) [2023-12-27 05:28:05,981][105692] Updated weights for policy 0, policy_version 1931238 (0.0006) [2023-12-27 05:28:06,041][105692] Updated weights for policy 0, policy_version 1931248 (0.0005) [2023-12-27 05:28:06,062][104569] Fps is (10 sec: 20480.4, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 990134272. Throughput: 0: 9790.7, 1: 9688.1. Samples: 990121012. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:28:06,063][104569] Avg episode reward: [(0, '8349.050'), (1, '9346.225')] [2023-12-27 05:28:06,114][105620] Updated weights for policy 1, policy_version 1935921 (0.0008) [2023-12-27 05:28:06,171][105620] Updated weights for policy 1, policy_version 1935931 (0.0007) [2023-12-27 05:28:06,231][105620] Updated weights for policy 1, policy_version 1935941 (0.0005) [2023-12-27 05:28:06,300][105620] Updated weights for policy 1, policy_version 1935951 (0.0011) [2023-12-27 05:28:06,760][105692] Updated weights for policy 0, policy_version 1931258 (0.0006) [2023-12-27 05:28:06,826][105692] Updated weights for policy 0, policy_version 1931268 (0.0008) [2023-12-27 05:28:06,893][105692] Updated weights for policy 0, policy_version 1931278 (0.0011) [2023-12-27 05:28:07,016][105620] Updated weights for policy 1, policy_version 1935961 (0.0010) [2023-12-27 05:28:07,071][105620] Updated weights for policy 1, policy_version 1935971 (0.0005) [2023-12-27 05:28:07,122][105620] Updated weights for policy 1, policy_version 1935981 (0.0010) [2023-12-27 05:28:07,480][105692] Updated weights for policy 0, policy_version 1931288 (0.0007) [2023-12-27 05:28:07,547][105692] Updated weights for policy 0, policy_version 1931298 (0.0006) [2023-12-27 05:28:07,606][105692] Updated weights for policy 0, policy_version 1931308 (0.0006) [2023-12-27 05:28:07,817][105620] Updated weights for policy 1, policy_version 1935991 (0.0010) [2023-12-27 05:28:07,876][105620] Updated weights for policy 1, policy_version 1936001 (0.0010) [2023-12-27 05:28:07,926][105620] Updated weights for policy 1, policy_version 1936011 (0.0010) [2023-12-27 05:28:08,102][105692] Updated weights for policy 0, policy_version 1931318 (0.0007) [2023-12-27 05:28:08,150][105692] Updated weights for policy 0, policy_version 1931328 (0.0005) [2023-12-27 05:28:08,206][105692] Updated weights for policy 0, policy_version 1931338 (0.0005) [2023-12-27 05:28:08,648][105620] Updated weights for policy 1, policy_version 1936021 (0.0010) [2023-12-27 05:28:08,716][105620] Updated weights for policy 1, policy_version 1936031 (0.0011) [2023-12-27 05:28:08,774][105692] Updated weights for policy 0, policy_version 1931348 (0.0007) [2023-12-27 05:28:08,779][105620] Updated weights for policy 1, policy_version 1936041 (0.0009) [2023-12-27 05:28:08,826][105692] Updated weights for policy 0, policy_version 1931358 (0.0010) [2023-12-27 05:28:08,889][105692] Updated weights for policy 0, policy_version 1931368 (0.0010) [2023-12-27 05:28:09,456][105620] Updated weights for policy 1, policy_version 1936051 (0.0009) [2023-12-27 05:28:09,517][105620] Updated weights for policy 1, policy_version 1936061 (0.0009) [2023-12-27 05:28:09,584][105620] Updated weights for policy 1, policy_version 1936071 (0.0007) [2023-12-27 05:28:09,641][105692] Updated weights for policy 0, policy_version 1931378 (0.0010) [2023-12-27 05:28:09,703][105692] Updated weights for policy 0, policy_version 1931388 (0.0009) [2023-12-27 05:28:09,771][105692] Updated weights for policy 0, policy_version 1931398 (0.0009) [2023-12-27 05:28:09,848][105692] Updated weights for policy 0, policy_version 1931408 (0.0009) [2023-12-27 05:28:10,300][105620] Updated weights for policy 1, policy_version 1936081 (0.0006) [2023-12-27 05:28:10,348][105620] Updated weights for policy 1, policy_version 1936091 (0.0008) [2023-12-27 05:28:10,404][105620] Updated weights for policy 1, policy_version 1936101 (0.0008) [2023-12-27 05:28:10,457][105620] Updated weights for policy 1, policy_version 1936111 (0.0008) [2023-12-27 05:28:10,647][105692] Updated weights for policy 0, policy_version 1931418 (0.0010) [2023-12-27 05:28:10,713][105692] Updated weights for policy 0, policy_version 1931428 (0.0011) [2023-12-27 05:28:10,779][105692] Updated weights for policy 0, policy_version 1931438 (0.0010) [2023-12-27 05:28:11,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 990232576. Throughput: 0: 9976.0, 1: 9689.1. Samples: 990241036. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:28:11,063][104569] Avg episode reward: [(0, '8261.139'), (1, '9253.773')] [2023-12-27 05:28:11,246][105620] Updated weights for policy 1, policy_version 1936121 (0.0010) [2023-12-27 05:28:11,310][105620] Updated weights for policy 1, policy_version 1936131 (0.0009) [2023-12-27 05:28:11,379][105620] Updated weights for policy 1, policy_version 1936141 (0.0008) [2023-12-27 05:28:11,476][105692] Updated weights for policy 0, policy_version 1931448 (0.0008) [2023-12-27 05:28:11,533][105692] Updated weights for policy 0, policy_version 1931458 (0.0008) [2023-12-27 05:28:11,579][105692] Updated weights for policy 0, policy_version 1931468 (0.0008) [2023-12-27 05:28:12,092][105620] Updated weights for policy 1, policy_version 1936151 (0.0007) [2023-12-27 05:28:12,143][105620] Updated weights for policy 1, policy_version 1936161 (0.0005) [2023-12-27 05:28:12,193][105620] Updated weights for policy 1, policy_version 1936171 (0.0009) [2023-12-27 05:28:12,278][105692] Updated weights for policy 0, policy_version 1931478 (0.0009) [2023-12-27 05:28:12,339][105692] Updated weights for policy 0, policy_version 1931488 (0.0010) [2023-12-27 05:28:12,398][105692] Updated weights for policy 0, policy_version 1931498 (0.0009) [2023-12-27 05:28:12,919][105620] Updated weights for policy 1, policy_version 1936181 (0.0008) [2023-12-27 05:28:12,967][105620] Updated weights for policy 1, policy_version 1936191 (0.0007) [2023-12-27 05:28:13,024][105620] Updated weights for policy 1, policy_version 1936201 (0.0005) [2023-12-27 05:28:13,162][105692] Updated weights for policy 0, policy_version 1931508 (0.0009) [2023-12-27 05:28:13,216][105692] Updated weights for policy 0, policy_version 1931519 (0.0010) [2023-12-27 05:28:13,269][105692] Updated weights for policy 0, policy_version 1931529 (0.0010) [2023-12-27 05:28:13,579][105620] Updated weights for policy 1, policy_version 1936211 (0.0005) [2023-12-27 05:28:13,628][105620] Updated weights for policy 1, policy_version 1936221 (0.0005) [2023-12-27 05:28:13,676][105620] Updated weights for policy 1, policy_version 1936231 (0.0005) [2023-12-27 05:28:13,936][105692] Updated weights for policy 0, policy_version 1931539 (0.0007) [2023-12-27 05:28:13,981][105692] Updated weights for policy 0, policy_version 1931549 (0.0010) [2023-12-27 05:28:14,026][105692] Updated weights for policy 0, policy_version 1931559 (0.0010) [2023-12-27 05:28:14,187][105620] Updated weights for policy 1, policy_version 1936241 (0.0006) [2023-12-27 05:28:14,244][105620] Updated weights for policy 1, policy_version 1936251 (0.0008) [2023-12-27 05:28:14,306][105620] Updated weights for policy 1, policy_version 1936261 (0.0008) [2023-12-27 05:28:14,361][105620] Updated weights for policy 1, policy_version 1936271 (0.0007) [2023-12-27 05:28:14,864][105692] Updated weights for policy 0, policy_version 1931569 (0.0010) [2023-12-27 05:28:14,931][105692] Updated weights for policy 0, policy_version 1931579 (0.0010) [2023-12-27 05:28:14,993][105692] Updated weights for policy 0, policy_version 1931589 (0.0007) [2023-12-27 05:28:15,008][105620] Updated weights for policy 1, policy_version 1936281 (0.0008) [2023-12-27 05:28:15,048][105692] Updated weights for policy 0, policy_version 1931599 (0.0007) [2023-12-27 05:28:15,062][105620] Updated weights for policy 1, policy_version 1936291 (0.0007) [2023-12-27 05:28:15,128][105620] Updated weights for policy 1, policy_version 1936301 (0.0008) [2023-12-27 05:28:15,808][105620] Updated weights for policy 1, policy_version 1936311 (0.0008) [2023-12-27 05:28:15,865][105620] Updated weights for policy 1, policy_version 1936321 (0.0007) [2023-12-27 05:28:15,877][105692] Updated weights for policy 0, policy_version 1931609 (0.0011) [2023-12-27 05:28:15,929][105620] Updated weights for policy 1, policy_version 1936331 (0.0006) [2023-12-27 05:28:15,937][105692] Updated weights for policy 0, policy_version 1931619 (0.0011) [2023-12-27 05:28:15,988][105692] Updated weights for policy 0, policy_version 1931629 (0.0010) [2023-12-27 05:28:16,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 990339072. Throughput: 0: 9899.0, 1: 9783.2. Samples: 990301580. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:28:16,062][104569] Avg episode reward: [(0, '8445.155'), (1, '9161.354')] [2023-12-27 05:28:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001931632_494567424.pth... [2023-12-27 05:28:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001936336_495771648.pth... [2023-12-27 05:28:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001930448_494264320.pth [2023-12-27 05:28:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001935152_495468544.pth [2023-12-27 05:28:16,550][105620] Updated weights for policy 1, policy_version 1936341 (0.0007) [2023-12-27 05:28:16,602][105620] Updated weights for policy 1, policy_version 1936351 (0.0008) [2023-12-27 05:28:16,648][105692] Updated weights for policy 0, policy_version 1931639 (0.0010) [2023-12-27 05:28:16,651][105620] Updated weights for policy 1, policy_version 1936361 (0.0007) [2023-12-27 05:28:16,702][105692] Updated weights for policy 0, policy_version 1931649 (0.0010) [2023-12-27 05:28:16,760][105692] Updated weights for policy 0, policy_version 1931659 (0.0008) [2023-12-27 05:28:17,377][105692] Updated weights for policy 0, policy_version 1931669 (0.0009) [2023-12-27 05:28:17,435][105692] Updated weights for policy 0, policy_version 1931679 (0.0010) [2023-12-27 05:28:17,486][105692] Updated weights for policy 0, policy_version 1931689 (0.0010) [2023-12-27 05:28:17,500][105620] Updated weights for policy 1, policy_version 1936371 (0.0007) [2023-12-27 05:28:17,553][105620] Updated weights for policy 1, policy_version 1936381 (0.0007) [2023-12-27 05:28:17,598][105620] Updated weights for policy 1, policy_version 1936391 (0.0008) [2023-12-27 05:28:18,137][105692] Updated weights for policy 0, policy_version 1931699 (0.0009) [2023-12-27 05:28:18,188][105692] Updated weights for policy 0, policy_version 1931709 (0.0005) [2023-12-27 05:28:18,247][105692] Updated weights for policy 0, policy_version 1931719 (0.0006) [2023-12-27 05:28:18,304][105620] Updated weights for policy 1, policy_version 1936401 (0.0008) [2023-12-27 05:28:18,370][105620] Updated weights for policy 1, policy_version 1936411 (0.0009) [2023-12-27 05:28:18,438][105620] Updated weights for policy 1, policy_version 1936421 (0.0008) [2023-12-27 05:28:18,504][105620] Updated weights for policy 1, policy_version 1936431 (0.0010) [2023-12-27 05:28:18,895][105692] Updated weights for policy 0, policy_version 1931729 (0.0008) [2023-12-27 05:28:18,948][105692] Updated weights for policy 0, policy_version 1931739 (0.0007) [2023-12-27 05:28:19,003][105692] Updated weights for policy 0, policy_version 1931749 (0.0009) [2023-12-27 05:28:19,054][105692] Updated weights for policy 0, policy_version 1931759 (0.0007) [2023-12-27 05:28:19,333][105620] Updated weights for policy 1, policy_version 1936441 (0.0009) [2023-12-27 05:28:19,403][105620] Updated weights for policy 1, policy_version 1936451 (0.0009) [2023-12-27 05:28:19,450][105620] Updated weights for policy 1, policy_version 1936461 (0.0008) [2023-12-27 05:28:19,819][105692] Updated weights for policy 0, policy_version 1931769 (0.0009) [2023-12-27 05:28:19,888][105692] Updated weights for policy 0, policy_version 1931779 (0.0009) [2023-12-27 05:28:19,959][105692] Updated weights for policy 0, policy_version 1931789 (0.0009) [2023-12-27 05:28:20,235][105620] Updated weights for policy 1, policy_version 1936471 (0.0009) [2023-12-27 05:28:20,293][105620] Updated weights for policy 1, policy_version 1936481 (0.0008) [2023-12-27 05:28:20,349][105620] Updated weights for policy 1, policy_version 1936491 (0.0008) [2023-12-27 05:28:20,785][105692] Updated weights for policy 0, policy_version 1931799 (0.0007) [2023-12-27 05:28:20,858][105692] Updated weights for policy 0, policy_version 1931809 (0.0006) [2023-12-27 05:28:20,915][105692] Updated weights for policy 0, policy_version 1931819 (0.0006) [2023-12-27 05:28:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 990429184. Throughput: 0: 9843.8, 1: 9856.1. Samples: 990419552. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:28:21,063][104569] Avg episode reward: [(0, '8627.079'), (1, '9161.475')] [2023-12-27 05:28:21,184][105620] Updated weights for policy 1, policy_version 1936501 (0.0009) [2023-12-27 05:28:21,252][105620] Updated weights for policy 1, policy_version 1936511 (0.0007) [2023-12-27 05:28:21,320][105620] Updated weights for policy 1, policy_version 1936521 (0.0009) [2023-12-27 05:28:21,594][105692] Updated weights for policy 0, policy_version 1931829 (0.0008) [2023-12-27 05:28:21,667][105692] Updated weights for policy 0, policy_version 1931839 (0.0008) [2023-12-27 05:28:21,734][105692] Updated weights for policy 0, policy_version 1931849 (0.0008) [2023-12-27 05:28:22,050][105620] Updated weights for policy 1, policy_version 1936531 (0.0009) [2023-12-27 05:28:22,109][105620] Updated weights for policy 1, policy_version 1936541 (0.0009) [2023-12-27 05:28:22,169][105620] Updated weights for policy 1, policy_version 1936551 (0.0010) [2023-12-27 05:28:22,455][105692] Updated weights for policy 0, policy_version 1931859 (0.0008) [2023-12-27 05:28:22,511][105692] Updated weights for policy 0, policy_version 1931869 (0.0009) [2023-12-27 05:28:22,564][105692] Updated weights for policy 0, policy_version 1931880 (0.0009) [2023-12-27 05:28:23,010][105620] Updated weights for policy 1, policy_version 1936561 (0.0010) [2023-12-27 05:28:23,068][105620] Updated weights for policy 1, policy_version 1936571 (0.0010) [2023-12-27 05:28:23,127][105620] Updated weights for policy 1, policy_version 1936581 (0.0009) [2023-12-27 05:28:23,193][105620] Updated weights for policy 1, policy_version 1936591 (0.0009) [2023-12-27 05:28:23,224][105692] Updated weights for policy 0, policy_version 1931890 (0.0010) [2023-12-27 05:28:23,283][105692] Updated weights for policy 0, policy_version 1931900 (0.0010) [2023-12-27 05:28:23,337][105692] Updated weights for policy 0, policy_version 1931910 (0.0010) [2023-12-27 05:28:23,395][105692] Updated weights for policy 0, policy_version 1931920 (0.0010) [2023-12-27 05:28:23,952][105692] Updated weights for policy 0, policy_version 1931930 (0.0005) [2023-12-27 05:28:23,999][105692] Updated weights for policy 0, policy_version 1931940 (0.0005) [2023-12-27 05:28:24,048][105692] Updated weights for policy 0, policy_version 1931950 (0.0005) [2023-12-27 05:28:24,053][105620] Updated weights for policy 1, policy_version 1936601 (0.0009) [2023-12-27 05:28:24,105][105620] Updated weights for policy 1, policy_version 1936611 (0.0010) [2023-12-27 05:28:24,168][105620] Updated weights for policy 1, policy_version 1936621 (0.0010) [2023-12-27 05:28:24,639][105692] Updated weights for policy 0, policy_version 1931960 (0.0005) [2023-12-27 05:28:24,687][105692] Updated weights for policy 0, policy_version 1931970 (0.0005) [2023-12-27 05:28:24,740][105692] Updated weights for policy 0, policy_version 1931980 (0.0005) [2023-12-27 05:28:25,063][105620] Updated weights for policy 1, policy_version 1936631 (0.0008) [2023-12-27 05:28:25,121][105620] Updated weights for policy 1, policy_version 1936641 (0.0005) [2023-12-27 05:28:25,176][105620] Updated weights for policy 1, policy_version 1936651 (0.0006) [2023-12-27 05:28:25,257][105692] Updated weights for policy 0, policy_version 1931990 (0.0008) [2023-12-27 05:28:25,316][105692] Updated weights for policy 0, policy_version 1932000 (0.0010) [2023-12-27 05:28:25,372][105692] Updated weights for policy 0, policy_version 1932010 (0.0007) [2023-12-27 05:28:25,823][105620] Updated weights for policy 1, policy_version 1936661 (0.0008) [2023-12-27 05:28:25,882][105620] Updated weights for policy 1, policy_version 1936671 (0.0007) [2023-12-27 05:28:25,943][105620] Updated weights for policy 1, policy_version 1936681 (0.0006) [2023-12-27 05:28:26,014][105692] Updated weights for policy 0, policy_version 1932020 (0.0008) [2023-12-27 05:28:26,062][104569] Fps is (10 sec: 18841.1, 60 sec: 19660.7, 300 sec: 19410.9). Total num frames: 990527488. Throughput: 0: 10035.7, 1: 9739.8. Samples: 990536392. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:28:26,063][104569] Avg episode reward: [(0, '8629.032'), (1, '9253.932')] [2023-12-27 05:28:26,079][105692] Updated weights for policy 0, policy_version 1932030 (0.0010) [2023-12-27 05:28:26,130][105692] Updated weights for policy 0, policy_version 1932040 (0.0010) [2023-12-27 05:28:26,560][105620] Updated weights for policy 1, policy_version 1936691 (0.0009) [2023-12-27 05:28:26,628][105620] Updated weights for policy 1, policy_version 1936701 (0.0005) [2023-12-27 05:28:26,681][105620] Updated weights for policy 1, policy_version 1936711 (0.0005) [2023-12-27 05:28:26,807][105692] Updated weights for policy 0, policy_version 1932050 (0.0010) [2023-12-27 05:28:26,865][105692] Updated weights for policy 0, policy_version 1932060 (0.0009) [2023-12-27 05:28:26,916][105692] Updated weights for policy 0, policy_version 1932070 (0.0005) [2023-12-27 05:28:26,965][105692] Updated weights for policy 0, policy_version 1932080 (0.0005) [2023-12-27 05:28:27,169][105620] Updated weights for policy 1, policy_version 1936721 (0.0005) [2023-12-27 05:28:27,214][105620] Updated weights for policy 1, policy_version 1936731 (0.0005) [2023-12-27 05:28:27,259][105620] Updated weights for policy 1, policy_version 1936741 (0.0005) [2023-12-27 05:28:27,321][105620] Updated weights for policy 1, policy_version 1936751 (0.0008) [2023-12-27 05:28:27,626][105692] Updated weights for policy 0, policy_version 1932090 (0.0009) [2023-12-27 05:28:27,693][105692] Updated weights for policy 0, policy_version 1932100 (0.0008) [2023-12-27 05:28:27,742][105692] Updated weights for policy 0, policy_version 1932110 (0.0009) [2023-12-27 05:28:28,117][105620] Updated weights for policy 1, policy_version 1936761 (0.0009) [2023-12-27 05:28:28,178][105620] Updated weights for policy 1, policy_version 1936771 (0.0010) [2023-12-27 05:28:28,236][105620] Updated weights for policy 1, policy_version 1936781 (0.0009) [2023-12-27 05:28:28,326][105692] Updated weights for policy 0, policy_version 1932120 (0.0009) [2023-12-27 05:28:28,384][105692] Updated weights for policy 0, policy_version 1932130 (0.0009) [2023-12-27 05:28:28,437][105692] Updated weights for policy 0, policy_version 1932140 (0.0008) [2023-12-27 05:28:28,942][105620] Updated weights for policy 1, policy_version 1936791 (0.0009) [2023-12-27 05:28:28,994][105620] Updated weights for policy 1, policy_version 1936801 (0.0007) [2023-12-27 05:28:29,059][105620] Updated weights for policy 1, policy_version 1936811 (0.0009) [2023-12-27 05:28:29,296][105692] Updated weights for policy 0, policy_version 1932150 (0.0009) [2023-12-27 05:28:29,360][105692] Updated weights for policy 0, policy_version 1932160 (0.0009) [2023-12-27 05:28:29,420][105692] Updated weights for policy 0, policy_version 1932170 (0.0009) [2023-12-27 05:28:29,790][105620] Updated weights for policy 1, policy_version 1936821 (0.0008) [2023-12-27 05:28:29,850][105620] Updated weights for policy 1, policy_version 1936831 (0.0009) [2023-12-27 05:28:29,915][105620] Updated weights for policy 1, policy_version 1936841 (0.0010) [2023-12-27 05:28:30,111][105692] Updated weights for policy 0, policy_version 1932180 (0.0008) [2023-12-27 05:28:30,167][105692] Updated weights for policy 0, policy_version 1932190 (0.0009) [2023-12-27 05:28:30,227][105692] Updated weights for policy 0, policy_version 1932200 (0.0010) [2023-12-27 05:28:30,618][105620] Updated weights for policy 1, policy_version 1936851 (0.0010) [2023-12-27 05:28:30,671][105620] Updated weights for policy 1, policy_version 1936861 (0.0008) [2023-12-27 05:28:30,724][105620] Updated weights for policy 1, policy_version 1936871 (0.0009) [2023-12-27 05:28:31,004][105692] Updated weights for policy 0, policy_version 1932210 (0.0009) [2023-12-27 05:28:31,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19797.4, 300 sec: 19438.6). Total num frames: 990625792. Throughput: 0: 10114.4, 1: 9766.2. Samples: 990599588. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:28:31,062][104569] Avg episode reward: [(0, '8446.041'), (1, '9346.272')] [2023-12-27 05:28:31,063][105692] Updated weights for policy 0, policy_version 1932220 (0.0010) [2023-12-27 05:28:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001936880_495910912.pth... [2023-12-27 05:28:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001935728_495616000.pth [2023-12-27 05:28:31,122][105692] Updated weights for policy 0, policy_version 1932230 (0.0010) [2023-12-27 05:28:31,180][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001932240_494723072.pth... [2023-12-27 05:28:31,184][105692] Updated weights for policy 0, policy_version 1932240 (0.0010) [2023-12-27 05:28:31,184][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001931056_494419968.pth [2023-12-27 05:28:31,451][105620] Updated weights for policy 1, policy_version 1936881 (0.0009) [2023-12-27 05:28:31,511][105620] Updated weights for policy 1, policy_version 1936891 (0.0008) [2023-12-27 05:28:31,565][105620] Updated weights for policy 1, policy_version 1936901 (0.0008) [2023-12-27 05:28:31,627][105620] Updated weights for policy 1, policy_version 1936911 (0.0008) [2023-12-27 05:28:31,991][105692] Updated weights for policy 0, policy_version 1932250 (0.0009) [2023-12-27 05:28:32,045][105692] Updated weights for policy 0, policy_version 1932261 (0.0009) [2023-12-27 05:28:32,098][105692] Updated weights for policy 0, policy_version 1932271 (0.0009) [2023-12-27 05:28:32,322][105620] Updated weights for policy 1, policy_version 1936921 (0.0009) [2023-12-27 05:28:32,387][105620] Updated weights for policy 1, policy_version 1936931 (0.0009) [2023-12-27 05:28:32,434][105620] Updated weights for policy 1, policy_version 1936941 (0.0009) [2023-12-27 05:28:32,923][105692] Updated weights for policy 0, policy_version 1932281 (0.0008) [2023-12-27 05:28:32,983][105692] Updated weights for policy 0, policy_version 1932291 (0.0008) [2023-12-27 05:28:33,039][105692] Updated weights for policy 0, policy_version 1932301 (0.0008) [2023-12-27 05:28:33,212][105620] Updated weights for policy 1, policy_version 1936951 (0.0010) [2023-12-27 05:28:33,273][105620] Updated weights for policy 1, policy_version 1936961 (0.0010) [2023-12-27 05:28:33,325][105620] Updated weights for policy 1, policy_version 1936971 (0.0006) [2023-12-27 05:28:33,742][105692] Updated weights for policy 0, policy_version 1932311 (0.0010) [2023-12-27 05:28:33,800][105692] Updated weights for policy 0, policy_version 1932321 (0.0010) [2023-12-27 05:28:33,857][105692] Updated weights for policy 0, policy_version 1932331 (0.0010) [2023-12-27 05:28:33,970][105620] Updated weights for policy 1, policy_version 1936981 (0.0008) [2023-12-27 05:28:34,017][105620] Updated weights for policy 1, policy_version 1936991 (0.0010) [2023-12-27 05:28:34,065][105620] Updated weights for policy 1, policy_version 1937001 (0.0010) [2023-12-27 05:28:34,498][105692] Updated weights for policy 0, policy_version 1932341 (0.0009) [2023-12-27 05:28:34,558][105692] Updated weights for policy 0, policy_version 1932351 (0.0010) [2023-12-27 05:28:34,624][105692] Updated weights for policy 0, policy_version 1932361 (0.0011) [2023-12-27 05:28:34,810][105620] Updated weights for policy 1, policy_version 1937011 (0.0010) [2023-12-27 05:28:34,858][105620] Updated weights for policy 1, policy_version 1937021 (0.0010) [2023-12-27 05:28:34,914][105620] Updated weights for policy 1, policy_version 1937031 (0.0010) [2023-12-27 05:28:35,227][105692] Updated weights for policy 0, policy_version 1932371 (0.0010) [2023-12-27 05:28:35,285][105692] Updated weights for policy 0, policy_version 1932381 (0.0011) [2023-12-27 05:28:35,339][105692] Updated weights for policy 0, policy_version 1932391 (0.0010) [2023-12-27 05:28:35,593][105620] Updated weights for policy 1, policy_version 1937041 (0.0010) [2023-12-27 05:28:35,645][105620] Updated weights for policy 1, policy_version 1937051 (0.0010) [2023-12-27 05:28:35,695][105620] Updated weights for policy 1, policy_version 1937061 (0.0010) [2023-12-27 05:28:35,743][105620] Updated weights for policy 1, policy_version 1937071 (0.0010) [2023-12-27 05:28:36,058][105692] Updated weights for policy 0, policy_version 1932401 (0.0010) [2023-12-27 05:28:36,062][104569] Fps is (10 sec: 19661.3, 60 sec: 19660.8, 300 sec: 19438.7). Total num frames: 990724096. Throughput: 0: 10067.3, 1: 9702.8. Samples: 990713148. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:28:36,062][104569] Avg episode reward: [(0, '8445.364'), (1, '9346.300')] [2023-12-27 05:28:36,114][105692] Updated weights for policy 0, policy_version 1932411 (0.0011) [2023-12-27 05:28:36,166][105692] Updated weights for policy 0, policy_version 1932421 (0.0011) [2023-12-27 05:28:36,215][105692] Updated weights for policy 0, policy_version 1932431 (0.0010) [2023-12-27 05:28:36,473][105620] Updated weights for policy 1, policy_version 1937081 (0.0008) [2023-12-27 05:28:36,540][105620] Updated weights for policy 1, policy_version 1937091 (0.0008) [2023-12-27 05:28:36,604][105620] Updated weights for policy 1, policy_version 1937101 (0.0008) [2023-12-27 05:28:36,917][105692] Updated weights for policy 0, policy_version 1932441 (0.0008) [2023-12-27 05:28:36,970][105692] Updated weights for policy 0, policy_version 1932451 (0.0011) [2023-12-27 05:28:37,034][105692] Updated weights for policy 0, policy_version 1932461 (0.0010) [2023-12-27 05:28:37,348][105620] Updated weights for policy 1, policy_version 1937111 (0.0006) [2023-12-27 05:28:37,396][105620] Updated weights for policy 1, policy_version 1937121 (0.0005) [2023-12-27 05:28:37,447][105620] Updated weights for policy 1, policy_version 1937131 (0.0005) [2023-12-27 05:28:37,745][105692] Updated weights for policy 0, policy_version 1932471 (0.0007) [2023-12-27 05:28:37,816][105692] Updated weights for policy 0, policy_version 1932481 (0.0005) [2023-12-27 05:28:37,886][105692] Updated weights for policy 0, policy_version 1932491 (0.0005) [2023-12-27 05:28:38,078][105620] Updated weights for policy 1, policy_version 1937141 (0.0008) [2023-12-27 05:28:38,123][105620] Updated weights for policy 1, policy_version 1937151 (0.0010) [2023-12-27 05:28:38,168][105620] Updated weights for policy 1, policy_version 1937161 (0.0010) [2023-12-27 05:28:38,534][105692] Updated weights for policy 0, policy_version 1932501 (0.0008) [2023-12-27 05:28:38,593][105692] Updated weights for policy 0, policy_version 1932511 (0.0010) [2023-12-27 05:28:38,658][105692] Updated weights for policy 0, policy_version 1932521 (0.0010) [2023-12-27 05:28:38,959][105620] Updated weights for policy 1, policy_version 1937171 (0.0010) [2023-12-27 05:28:39,018][105620] Updated weights for policy 1, policy_version 1937181 (0.0011) [2023-12-27 05:28:39,076][105620] Updated weights for policy 1, policy_version 1937191 (0.0010) [2023-12-27 05:28:39,400][105692] Updated weights for policy 0, policy_version 1932531 (0.0010) [2023-12-27 05:28:39,457][105692] Updated weights for policy 0, policy_version 1932541 (0.0010) [2023-12-27 05:28:39,510][105692] Updated weights for policy 0, policy_version 1932551 (0.0011) [2023-12-27 05:28:39,806][105620] Updated weights for policy 1, policy_version 1937201 (0.0010) [2023-12-27 05:28:39,872][105620] Updated weights for policy 1, policy_version 1937211 (0.0007) [2023-12-27 05:28:39,940][105620] Updated weights for policy 1, policy_version 1937221 (0.0008) [2023-12-27 05:28:40,002][105620] Updated weights for policy 1, policy_version 1937231 (0.0008) [2023-12-27 05:28:40,264][105692] Updated weights for policy 0, policy_version 1932561 (0.0011) [2023-12-27 05:28:40,334][105692] Updated weights for policy 0, policy_version 1932571 (0.0011) [2023-12-27 05:28:40,401][105692] Updated weights for policy 0, policy_version 1932581 (0.0011) [2023-12-27 05:28:40,457][105692] Updated weights for policy 0, policy_version 1932591 (0.0010) [2023-12-27 05:28:40,735][105620] Updated weights for policy 1, policy_version 1937241 (0.0008) [2023-12-27 05:28:40,792][105620] Updated weights for policy 1, policy_version 1937251 (0.0008) [2023-12-27 05:28:40,856][105620] Updated weights for policy 1, policy_version 1937261 (0.0008) [2023-12-27 05:28:41,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19797.4, 300 sec: 19438.7). Total num frames: 990822400. Throughput: 0: 10014.2, 1: 9715.1. Samples: 990830236. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:28:41,063][104569] Avg episode reward: [(0, '8625.192'), (1, '9346.280')] [2023-12-27 05:28:41,214][105692] Updated weights for policy 0, policy_version 1932601 (0.0010) [2023-12-27 05:28:41,280][105692] Updated weights for policy 0, policy_version 1932611 (0.0011) [2023-12-27 05:28:41,334][105692] Updated weights for policy 0, policy_version 1932621 (0.0010) [2023-12-27 05:28:41,639][105620] Updated weights for policy 1, policy_version 1937271 (0.0007) [2023-12-27 05:28:41,709][105620] Updated weights for policy 1, policy_version 1937281 (0.0006) [2023-12-27 05:28:41,784][105620] Updated weights for policy 1, policy_version 1937291 (0.0007) [2023-12-27 05:28:42,117][105692] Updated weights for policy 0, policy_version 1932631 (0.0008) [2023-12-27 05:28:42,177][105692] Updated weights for policy 0, policy_version 1932641 (0.0009) [2023-12-27 05:28:42,238][105692] Updated weights for policy 0, policy_version 1932651 (0.0009) [2023-12-27 05:28:42,447][105620] Updated weights for policy 1, policy_version 1937301 (0.0010) [2023-12-27 05:28:42,499][105620] Updated weights for policy 1, policy_version 1937311 (0.0009) [2023-12-27 05:28:42,555][105620] Updated weights for policy 1, policy_version 1937321 (0.0008) [2023-12-27 05:28:43,001][105692] Updated weights for policy 0, policy_version 1932661 (0.0009) [2023-12-27 05:28:43,059][105692] Updated weights for policy 0, policy_version 1932671 (0.0009) [2023-12-27 05:28:43,116][105692] Updated weights for policy 0, policy_version 1932681 (0.0008) [2023-12-27 05:28:43,282][105620] Updated weights for policy 1, policy_version 1937331 (0.0007) [2023-12-27 05:28:43,345][105620] Updated weights for policy 1, policy_version 1937341 (0.0008) [2023-12-27 05:28:43,400][105620] Updated weights for policy 1, policy_version 1937351 (0.0009) [2023-12-27 05:28:43,826][105692] Updated weights for policy 0, policy_version 1932691 (0.0007) [2023-12-27 05:28:43,893][105692] Updated weights for policy 0, policy_version 1932701 (0.0006) [2023-12-27 05:28:43,954][105692] Updated weights for policy 0, policy_version 1932711 (0.0006) [2023-12-27 05:28:44,255][105620] Updated weights for policy 1, policy_version 1937361 (0.0009) [2023-12-27 05:28:44,317][105620] Updated weights for policy 1, policy_version 1937371 (0.0010) [2023-12-27 05:28:44,386][105620] Updated weights for policy 1, policy_version 1937381 (0.0009) [2023-12-27 05:28:44,443][105620] Updated weights for policy 1, policy_version 1937391 (0.0009) [2023-12-27 05:28:44,467][105692] Updated weights for policy 0, policy_version 1932721 (0.0006) [2023-12-27 05:28:44,509][105692] Updated weights for policy 0, policy_version 1932731 (0.0006) [2023-12-27 05:28:44,565][105692] Updated weights for policy 0, policy_version 1932741 (0.0005) [2023-12-27 05:28:44,616][105692] Updated weights for policy 0, policy_version 1932751 (0.0005) [2023-12-27 05:28:45,257][105620] Updated weights for policy 1, policy_version 1937401 (0.0008) [2023-12-27 05:28:45,312][105620] Updated weights for policy 1, policy_version 1937411 (0.0009) [2023-12-27 05:28:45,340][105692] Updated weights for policy 0, policy_version 1932761 (0.0008) [2023-12-27 05:28:45,364][105620] Updated weights for policy 1, policy_version 1937421 (0.0006) [2023-12-27 05:28:45,399][105692] Updated weights for policy 0, policy_version 1932771 (0.0007) [2023-12-27 05:28:45,455][105692] Updated weights for policy 0, policy_version 1932781 (0.0009) [2023-12-27 05:28:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.9, 300 sec: 19383.1). Total num frames: 990912512. Throughput: 0: 9888.7, 1: 9715.5. Samples: 990885188. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:28:46,062][104569] Avg episode reward: [(0, '8355.812'), (1, '9161.797')] [2023-12-27 05:28:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001937424_496050176.pth... [2023-12-27 05:28:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001932784_494862336.pth... [2023-12-27 05:28:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001931632_494567424.pth [2023-12-27 05:28:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001936336_495771648.pth [2023-12-27 05:28:46,141][105620] Updated weights for policy 1, policy_version 1937431 (0.0008) [2023-12-27 05:28:46,202][105620] Updated weights for policy 1, policy_version 1937441 (0.0010) [2023-12-27 05:28:46,221][105692] Updated weights for policy 0, policy_version 1932791 (0.0007) [2023-12-27 05:28:46,259][105620] Updated weights for policy 1, policy_version 1937451 (0.0008) [2023-12-27 05:28:46,278][105692] Updated weights for policy 0, policy_version 1932801 (0.0005) [2023-12-27 05:28:46,329][105692] Updated weights for policy 0, policy_version 1932811 (0.0005) [2023-12-27 05:28:46,915][105692] Updated weights for policy 0, policy_version 1932821 (0.0007) [2023-12-27 05:28:46,967][105692] Updated weights for policy 0, policy_version 1932831 (0.0005) [2023-12-27 05:28:47,022][105692] Updated weights for policy 0, policy_version 1932841 (0.0006) [2023-12-27 05:28:47,114][105620] Updated weights for policy 1, policy_version 1937461 (0.0009) [2023-12-27 05:28:47,170][105620] Updated weights for policy 1, policy_version 1937471 (0.0009) [2023-12-27 05:28:47,223][105620] Updated weights for policy 1, policy_version 1937481 (0.0010) [2023-12-27 05:28:47,762][105692] Updated weights for policy 0, policy_version 1932851 (0.0009) [2023-12-27 05:28:47,820][105692] Updated weights for policy 0, policy_version 1932861 (0.0010) [2023-12-27 05:28:47,864][105692] Updated weights for policy 0, policy_version 1932871 (0.0010) [2023-12-27 05:28:47,931][105620] Updated weights for policy 1, policy_version 1937492 (0.0009) [2023-12-27 05:28:47,977][105620] Updated weights for policy 1, policy_version 1937502 (0.0008) [2023-12-27 05:28:48,030][105620] Updated weights for policy 1, policy_version 1937512 (0.0008) [2023-12-27 05:28:48,575][105692] Updated weights for policy 0, policy_version 1932881 (0.0010) [2023-12-27 05:28:48,635][105692] Updated weights for policy 0, policy_version 1932891 (0.0011) [2023-12-27 05:28:48,684][105692] Updated weights for policy 0, policy_version 1932901 (0.0010) [2023-12-27 05:28:48,739][105692] Updated weights for policy 0, policy_version 1932911 (0.0010) [2023-12-27 05:28:48,772][105620] Updated weights for policy 1, policy_version 1937522 (0.0008) [2023-12-27 05:28:48,825][105620] Updated weights for policy 1, policy_version 1937532 (0.0005) [2023-12-27 05:28:48,890][105620] Updated weights for policy 1, policy_version 1937542 (0.0005) [2023-12-27 05:28:48,951][105620] Updated weights for policy 1, policy_version 1937552 (0.0006) [2023-12-27 05:28:49,522][105692] Updated weights for policy 0, policy_version 1932921 (0.0011) [2023-12-27 05:28:49,589][105692] Updated weights for policy 0, policy_version 1932931 (0.0010) [2023-12-27 05:28:49,648][105692] Updated weights for policy 0, policy_version 1932941 (0.0011) [2023-12-27 05:28:49,651][105620] Updated weights for policy 1, policy_version 1937562 (0.0007) [2023-12-27 05:28:49,709][105620] Updated weights for policy 1, policy_version 1937572 (0.0008) [2023-12-27 05:28:49,762][105620] Updated weights for policy 1, policy_version 1937582 (0.0008) [2023-12-27 05:28:50,392][105692] Updated weights for policy 0, policy_version 1932951 (0.0009) [2023-12-27 05:28:50,443][105692] Updated weights for policy 0, policy_version 1932961 (0.0010) [2023-12-27 05:28:50,494][105692] Updated weights for policy 0, policy_version 1932971 (0.0010) [2023-12-27 05:28:50,524][105620] Updated weights for policy 1, policy_version 1937592 (0.0006) [2023-12-27 05:28:50,589][105620] Updated weights for policy 1, policy_version 1937602 (0.0008) [2023-12-27 05:28:50,647][105620] Updated weights for policy 1, policy_version 1937612 (0.0009) [2023-12-27 05:28:51,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19660.7, 300 sec: 19383.1). Total num frames: 991010816. Throughput: 0: 9975.7, 1: 9589.1. Samples: 991001432. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:28:51,063][104569] Avg episode reward: [(0, '7719.435'), (1, '9161.848')] [2023-12-27 05:28:51,228][105692] Updated weights for policy 0, policy_version 1932981 (0.0009) [2023-12-27 05:28:51,298][105692] Updated weights for policy 0, policy_version 1932991 (0.0008) [2023-12-27 05:28:51,369][105692] Updated weights for policy 0, policy_version 1933001 (0.0009) [2023-12-27 05:28:51,474][105620] Updated weights for policy 1, policy_version 1937622 (0.0008) [2023-12-27 05:28:51,545][105620] Updated weights for policy 1, policy_version 1937632 (0.0008) [2023-12-27 05:28:51,615][105620] Updated weights for policy 1, policy_version 1937642 (0.0008) [2023-12-27 05:28:52,033][105692] Updated weights for policy 0, policy_version 1933011 (0.0009) [2023-12-27 05:28:52,087][105692] Updated weights for policy 0, policy_version 1933021 (0.0009) [2023-12-27 05:28:52,156][105692] Updated weights for policy 0, policy_version 1933031 (0.0010) [2023-12-27 05:28:52,343][105620] Updated weights for policy 1, policy_version 1937652 (0.0008) [2023-12-27 05:28:52,412][105620] Updated weights for policy 1, policy_version 1937662 (0.0007) [2023-12-27 05:28:52,484][105620] Updated weights for policy 1, policy_version 1937672 (0.0008) [2023-12-27 05:28:52,981][105692] Updated weights for policy 0, policy_version 1933041 (0.0009) [2023-12-27 05:28:53,039][105692] Updated weights for policy 0, policy_version 1933051 (0.0008) [2023-12-27 05:28:53,105][105692] Updated weights for policy 0, policy_version 1933061 (0.0008) [2023-12-27 05:28:53,173][105692] Updated weights for policy 0, policy_version 1933071 (0.0010) [2023-12-27 05:28:53,176][105620] Updated weights for policy 1, policy_version 1937682 (0.0007) [2023-12-27 05:28:53,231][105620] Updated weights for policy 1, policy_version 1937692 (0.0005) [2023-12-27 05:28:53,287][105620] Updated weights for policy 1, policy_version 1937702 (0.0008) [2023-12-27 05:28:53,336][105620] Updated weights for policy 1, policy_version 1937712 (0.0010) [2023-12-27 05:28:53,888][105692] Updated weights for policy 0, policy_version 1933081 (0.0010) [2023-12-27 05:28:53,946][105692] Updated weights for policy 0, policy_version 1933091 (0.0009) [2023-12-27 05:28:54,006][105692] Updated weights for policy 0, policy_version 1933101 (0.0006) [2023-12-27 05:28:54,029][105620] Updated weights for policy 1, policy_version 1937722 (0.0007) [2023-12-27 05:28:54,087][105620] Updated weights for policy 1, policy_version 1937732 (0.0005) [2023-12-27 05:28:54,137][105620] Updated weights for policy 1, policy_version 1937742 (0.0008) [2023-12-27 05:28:54,782][105692] Updated weights for policy 0, policy_version 1933111 (0.0007) [2023-12-27 05:28:54,810][105620] Updated weights for policy 1, policy_version 1937752 (0.0008) [2023-12-27 05:28:54,837][105692] Updated weights for policy 0, policy_version 1933121 (0.0006) [2023-12-27 05:28:54,863][105620] Updated weights for policy 1, policy_version 1937762 (0.0007) [2023-12-27 05:28:54,889][105692] Updated weights for policy 0, policy_version 1933131 (0.0008) [2023-12-27 05:28:54,922][105620] Updated weights for policy 1, policy_version 1937772 (0.0008) [2023-12-27 05:28:55,639][105692] Updated weights for policy 0, policy_version 1933141 (0.0007) [2023-12-27 05:28:55,674][105620] Updated weights for policy 1, policy_version 1937782 (0.0008) [2023-12-27 05:28:55,700][105692] Updated weights for policy 0, policy_version 1933151 (0.0008) [2023-12-27 05:28:55,735][105620] Updated weights for policy 1, policy_version 1937792 (0.0007) [2023-12-27 05:28:55,768][105692] Updated weights for policy 0, policy_version 1933161 (0.0009) [2023-12-27 05:28:55,791][105620] Updated weights for policy 1, policy_version 1937802 (0.0007) [2023-12-27 05:28:56,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19660.8, 300 sec: 19383.1). Total num frames: 991109120. Throughput: 0: 9840.6, 1: 9571.1. Samples: 991114564. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:28:56,063][104569] Avg episode reward: [(0, '8354.671'), (1, '9346.364')] [2023-12-27 05:28:56,472][105692] Updated weights for policy 0, policy_version 1933171 (0.0008) [2023-12-27 05:28:56,517][105692] Updated weights for policy 0, policy_version 1933181 (0.0005) [2023-12-27 05:28:56,573][105692] Updated weights for policy 0, policy_version 1933191 (0.0006) [2023-12-27 05:28:56,580][105620] Updated weights for policy 1, policy_version 1937812 (0.0008) [2023-12-27 05:28:56,625][105620] Updated weights for policy 1, policy_version 1937822 (0.0007) [2023-12-27 05:28:56,672][105620] Updated weights for policy 1, policy_version 1937832 (0.0009) [2023-12-27 05:28:57,260][105692] Updated weights for policy 0, policy_version 1933201 (0.0008) [2023-12-27 05:28:57,314][105692] Updated weights for policy 0, policy_version 1933211 (0.0009) [2023-12-27 05:28:57,370][105692] Updated weights for policy 0, policy_version 1933221 (0.0009) [2023-12-27 05:28:57,431][105692] Updated weights for policy 0, policy_version 1933231 (0.0009) [2023-12-27 05:28:57,440][105620] Updated weights for policy 1, policy_version 1937842 (0.0009) [2023-12-27 05:28:57,493][105620] Updated weights for policy 1, policy_version 1937852 (0.0009) [2023-12-27 05:28:57,539][105620] Updated weights for policy 1, policy_version 1937862 (0.0008) [2023-12-27 05:28:57,599][105620] Updated weights for policy 1, policy_version 1937872 (0.0005) [2023-12-27 05:28:58,222][105692] Updated weights for policy 0, policy_version 1933241 (0.0009) [2023-12-27 05:28:58,281][105692] Updated weights for policy 0, policy_version 1933251 (0.0007) [2023-12-27 05:28:58,315][105620] Updated weights for policy 1, policy_version 1937882 (0.0008) [2023-12-27 05:28:58,346][105692] Updated weights for policy 0, policy_version 1933261 (0.0007) [2023-12-27 05:28:58,383][105620] Updated weights for policy 1, policy_version 1937892 (0.0008) [2023-12-27 05:28:58,452][105620] Updated weights for policy 1, policy_version 1937902 (0.0009) [2023-12-27 05:28:59,128][105692] Updated weights for policy 0, policy_version 1933271 (0.0008) [2023-12-27 05:28:59,191][105692] Updated weights for policy 0, policy_version 1933281 (0.0008) [2023-12-27 05:28:59,261][105692] Updated weights for policy 0, policy_version 1933291 (0.0008) [2023-12-27 05:28:59,300][105620] Updated weights for policy 1, policy_version 1937912 (0.0007) [2023-12-27 05:28:59,367][105620] Updated weights for policy 1, policy_version 1937922 (0.0009) [2023-12-27 05:28:59,415][105620] Updated weights for policy 1, policy_version 1937932 (0.0008) [2023-12-27 05:28:59,963][105692] Updated weights for policy 0, policy_version 1933301 (0.0009) [2023-12-27 05:29:00,026][105692] Updated weights for policy 0, policy_version 1933311 (0.0011) [2023-12-27 05:29:00,083][105692] Updated weights for policy 0, policy_version 1933321 (0.0011) [2023-12-27 05:29:00,206][105620] Updated weights for policy 1, policy_version 1937942 (0.0007) [2023-12-27 05:29:00,257][105620] Updated weights for policy 1, policy_version 1937952 (0.0009) [2023-12-27 05:29:00,307][105620] Updated weights for policy 1, policy_version 1937962 (0.0009) [2023-12-27 05:29:00,733][105692] Updated weights for policy 0, policy_version 1933331 (0.0010) [2023-12-27 05:29:00,796][105692] Updated weights for policy 0, policy_version 1933341 (0.0008) [2023-12-27 05:29:00,847][105692] Updated weights for policy 0, policy_version 1933351 (0.0009) [2023-12-27 05:29:01,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.3, 300 sec: 19355.3). Total num frames: 991199232. Throughput: 0: 9828.3, 1: 9477.4. Samples: 991170340. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:29:01,063][104569] Avg episode reward: [(0, '8446.693'), (1, '9164.013')] [2023-12-27 05:29:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001933360_495009792.pth... [2023-12-27 05:29:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001937968_496189440.pth... [2023-12-27 05:29:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001932240_494723072.pth [2023-12-27 05:29:01,086][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001936880_495910912.pth [2023-12-27 05:29:01,129][105620] Updated weights for policy 1, policy_version 1937972 (0.0008) [2023-12-27 05:29:01,195][105620] Updated weights for policy 1, policy_version 1937982 (0.0009) [2023-12-27 05:29:01,253][105620] Updated weights for policy 1, policy_version 1937992 (0.0008) [2023-12-27 05:29:01,545][105692] Updated weights for policy 0, policy_version 1933361 (0.0009) [2023-12-27 05:29:01,595][105692] Updated weights for policy 0, policy_version 1933371 (0.0006) [2023-12-27 05:29:01,660][105692] Updated weights for policy 0, policy_version 1933381 (0.0008) [2023-12-27 05:29:01,722][105692] Updated weights for policy 0, policy_version 1933391 (0.0009) [2023-12-27 05:29:01,996][105620] Updated weights for policy 1, policy_version 1938002 (0.0007) [2023-12-27 05:29:02,067][105620] Updated weights for policy 1, policy_version 1938012 (0.0005) [2023-12-27 05:29:02,131][105620] Updated weights for policy 1, policy_version 1938022 (0.0007) [2023-12-27 05:29:02,183][105620] Updated weights for policy 1, policy_version 1938032 (0.0007) [2023-12-27 05:29:02,418][105692] Updated weights for policy 0, policy_version 1933401 (0.0011) [2023-12-27 05:29:02,466][105692] Updated weights for policy 0, policy_version 1933411 (0.0010) [2023-12-27 05:29:02,529][105692] Updated weights for policy 0, policy_version 1933421 (0.0006) [2023-12-27 05:29:02,832][105620] Updated weights for policy 1, policy_version 1938042 (0.0011) [2023-12-27 05:29:02,898][105620] Updated weights for policy 1, policy_version 1938052 (0.0011) [2023-12-27 05:29:02,953][105620] Updated weights for policy 1, policy_version 1938062 (0.0011) [2023-12-27 05:29:03,214][105692] Updated weights for policy 0, policy_version 1933431 (0.0006) [2023-12-27 05:29:03,283][105692] Updated weights for policy 0, policy_version 1933441 (0.0006) [2023-12-27 05:29:03,347][105692] Updated weights for policy 0, policy_version 1933451 (0.0006) [2023-12-27 05:29:03,688][105620] Updated weights for policy 1, policy_version 1938072 (0.0009) [2023-12-27 05:29:03,748][105620] Updated weights for policy 1, policy_version 1938082 (0.0005) [2023-12-27 05:29:03,804][105620] Updated weights for policy 1, policy_version 1938092 (0.0005) [2023-12-27 05:29:04,050][105692] Updated weights for policy 0, policy_version 1933461 (0.0010) [2023-12-27 05:29:04,107][105692] Updated weights for policy 0, policy_version 1933471 (0.0010) [2023-12-27 05:29:04,168][105692] Updated weights for policy 0, policy_version 1933481 (0.0011) [2023-12-27 05:29:04,443][105620] Updated weights for policy 1, policy_version 1938102 (0.0008) [2023-12-27 05:29:04,495][105620] Updated weights for policy 1, policy_version 1938112 (0.0011) [2023-12-27 05:29:04,554][105620] Updated weights for policy 1, policy_version 1938122 (0.0011) [2023-12-27 05:29:04,832][105692] Updated weights for policy 0, policy_version 1933491 (0.0007) [2023-12-27 05:29:04,886][105692] Updated weights for policy 0, policy_version 1933501 (0.0010) [2023-12-27 05:29:04,937][105692] Updated weights for policy 0, policy_version 1933511 (0.0010) [2023-12-27 05:29:05,294][105620] Updated weights for policy 1, policy_version 1938132 (0.0009) [2023-12-27 05:29:05,350][105620] Updated weights for policy 1, policy_version 1938142 (0.0005) [2023-12-27 05:29:05,406][105620] Updated weights for policy 1, policy_version 1938152 (0.0005) [2023-12-27 05:29:05,688][105692] Updated weights for policy 0, policy_version 1933521 (0.0010) [2023-12-27 05:29:05,738][105692] Updated weights for policy 0, policy_version 1933531 (0.0010) [2023-12-27 05:29:05,789][105692] Updated weights for policy 0, policy_version 1933541 (0.0010) [2023-12-27 05:29:05,839][105692] Updated weights for policy 0, policy_version 1933551 (0.0010) [2023-12-27 05:29:06,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19387.8, 300 sec: 19355.3). Total num frames: 991297536. Throughput: 0: 9830.2, 1: 9440.0. Samples: 991286708. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:29:06,062][104569] Avg episode reward: [(0, '8627.876'), (1, '9163.874')] [2023-12-27 05:29:06,102][105620] Updated weights for policy 1, policy_version 1938162 (0.0009) [2023-12-27 05:29:06,167][105620] Updated weights for policy 1, policy_version 1938172 (0.0011) [2023-12-27 05:29:06,232][105620] Updated weights for policy 1, policy_version 1938182 (0.0008) [2023-12-27 05:29:06,292][105620] Updated weights for policy 1, policy_version 1938192 (0.0008) [2023-12-27 05:29:06,605][105692] Updated weights for policy 0, policy_version 1933561 (0.0011) [2023-12-27 05:29:06,672][105692] Updated weights for policy 0, policy_version 1933571 (0.0011) [2023-12-27 05:29:06,731][105692] Updated weights for policy 0, policy_version 1933581 (0.0011) [2023-12-27 05:29:07,060][105620] Updated weights for policy 1, policy_version 1938202 (0.0011) [2023-12-27 05:29:07,127][105620] Updated weights for policy 1, policy_version 1938212 (0.0011) [2023-12-27 05:29:07,194][105620] Updated weights for policy 1, policy_version 1938222 (0.0010) [2023-12-27 05:29:07,480][105692] Updated weights for policy 0, policy_version 1933591 (0.0010) [2023-12-27 05:29:07,531][105692] Updated weights for policy 0, policy_version 1933601 (0.0010) [2023-12-27 05:29:07,586][105692] Updated weights for policy 0, policy_version 1933611 (0.0010) [2023-12-27 05:29:07,868][105620] Updated weights for policy 1, policy_version 1938232 (0.0011) [2023-12-27 05:29:07,930][105620] Updated weights for policy 1, policy_version 1938242 (0.0010) [2023-12-27 05:29:07,987][105620] Updated weights for policy 1, policy_version 1938252 (0.0010) [2023-12-27 05:29:08,283][105692] Updated weights for policy 0, policy_version 1933621 (0.0008) [2023-12-27 05:29:08,340][105692] Updated weights for policy 0, policy_version 1933631 (0.0006) [2023-12-27 05:29:08,412][105692] Updated weights for policy 0, policy_version 1933641 (0.0006) [2023-12-27 05:29:08,691][105620] Updated weights for policy 1, policy_version 1938262 (0.0010) [2023-12-27 05:29:08,758][105620] Updated weights for policy 1, policy_version 1938272 (0.0011) [2023-12-27 05:29:08,814][105620] Updated weights for policy 1, policy_version 1938282 (0.0011) [2023-12-27 05:29:09,034][105692] Updated weights for policy 0, policy_version 1933651 (0.0008) [2023-12-27 05:29:09,103][105692] Updated weights for policy 0, policy_version 1933661 (0.0006) [2023-12-27 05:29:09,169][105692] Updated weights for policy 0, policy_version 1933671 (0.0011) [2023-12-27 05:29:09,575][105620] Updated weights for policy 1, policy_version 1938292 (0.0011) [2023-12-27 05:29:09,639][105620] Updated weights for policy 1, policy_version 1938302 (0.0011) [2023-12-27 05:29:09,702][105620] Updated weights for policy 1, policy_version 1938312 (0.0011) [2023-12-27 05:29:09,785][105692] Updated weights for policy 0, policy_version 1933681 (0.0009) [2023-12-27 05:29:09,854][105692] Updated weights for policy 0, policy_version 1933691 (0.0011) [2023-12-27 05:29:09,912][105692] Updated weights for policy 0, policy_version 1933701 (0.0012) [2023-12-27 05:29:09,978][105692] Updated weights for policy 0, policy_version 1933711 (0.0009) [2023-12-27 05:29:10,429][105620] Updated weights for policy 1, policy_version 1938322 (0.0008) [2023-12-27 05:29:10,489][105620] Updated weights for policy 1, policy_version 1938332 (0.0011) [2023-12-27 05:29:10,537][105620] Updated weights for policy 1, policy_version 1938342 (0.0010) [2023-12-27 05:29:10,593][105620] Updated weights for policy 1, policy_version 1938352 (0.0010) [2023-12-27 05:29:10,714][105692] Updated weights for policy 0, policy_version 1933721 (0.0010) [2023-12-27 05:29:10,781][105692] Updated weights for policy 0, policy_version 1933731 (0.0011) [2023-12-27 05:29:10,846][105692] Updated weights for policy 0, policy_version 1933741 (0.0009) [2023-12-27 05:29:11,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19355.4). Total num frames: 991395840. Throughput: 0: 9713.6, 1: 9537.0. Samples: 991402668. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:29:11,063][104569] Avg episode reward: [(0, '8444.850'), (1, '9346.110')] [2023-12-27 05:29:11,301][105620] Updated weights for policy 1, policy_version 1938362 (0.0007) [2023-12-27 05:29:11,364][105620] Updated weights for policy 1, policy_version 1938372 (0.0007) [2023-12-27 05:29:11,431][105620] Updated weights for policy 1, policy_version 1938382 (0.0011) [2023-12-27 05:29:11,606][105692] Updated weights for policy 0, policy_version 1933751 (0.0011) [2023-12-27 05:29:11,667][105692] Updated weights for policy 0, policy_version 1933761 (0.0007) [2023-12-27 05:29:11,730][105692] Updated weights for policy 0, policy_version 1933771 (0.0009) [2023-12-27 05:29:12,220][105620] Updated weights for policy 1, policy_version 1938392 (0.0008) [2023-12-27 05:29:12,277][105620] Updated weights for policy 1, policy_version 1938402 (0.0011) [2023-12-27 05:29:12,348][105620] Updated weights for policy 1, policy_version 1938412 (0.0009) [2023-12-27 05:29:12,452][105692] Updated weights for policy 0, policy_version 1933781 (0.0009) [2023-12-27 05:29:12,519][105692] Updated weights for policy 0, policy_version 1933791 (0.0009) [2023-12-27 05:29:12,579][105692] Updated weights for policy 0, policy_version 1933801 (0.0009) [2023-12-27 05:29:13,057][105620] Updated weights for policy 1, policy_version 1938422 (0.0011) [2023-12-27 05:29:13,122][105620] Updated weights for policy 1, policy_version 1938432 (0.0011) [2023-12-27 05:29:13,171][105620] Updated weights for policy 1, policy_version 1938442 (0.0010) [2023-12-27 05:29:13,220][105692] Updated weights for policy 0, policy_version 1933811 (0.0007) [2023-12-27 05:29:13,293][105692] Updated weights for policy 0, policy_version 1933821 (0.0011) [2023-12-27 05:29:13,341][105692] Updated weights for policy 0, policy_version 1933831 (0.0010) [2023-12-27 05:29:13,866][105620] Updated weights for policy 1, policy_version 1938452 (0.0008) [2023-12-27 05:29:13,919][105620] Updated weights for policy 1, policy_version 1938462 (0.0010) [2023-12-27 05:29:13,974][105620] Updated weights for policy 1, policy_version 1938472 (0.0007) [2023-12-27 05:29:14,070][105692] Updated weights for policy 0, policy_version 1933841 (0.0010) [2023-12-27 05:29:14,124][105692] Updated weights for policy 0, policy_version 1933851 (0.0010) [2023-12-27 05:29:14,190][105692] Updated weights for policy 0, policy_version 1933861 (0.0008) [2023-12-27 05:29:14,245][105692] Updated weights for policy 0, policy_version 1933871 (0.0007) [2023-12-27 05:29:14,522][105620] Updated weights for policy 1, policy_version 1938482 (0.0005) [2023-12-27 05:29:14,583][105620] Updated weights for policy 1, policy_version 1938492 (0.0005) [2023-12-27 05:29:14,639][105620] Updated weights for policy 1, policy_version 1938502 (0.0005) [2023-12-27 05:29:14,692][105620] Updated weights for policy 1, policy_version 1938512 (0.0005) [2023-12-27 05:29:14,972][105692] Updated weights for policy 0, policy_version 1933881 (0.0006) [2023-12-27 05:29:15,033][105692] Updated weights for policy 0, policy_version 1933891 (0.0006) [2023-12-27 05:29:15,091][105692] Updated weights for policy 0, policy_version 1933901 (0.0010) [2023-12-27 05:29:15,309][105620] Updated weights for policy 1, policy_version 1938522 (0.0010) [2023-12-27 05:29:15,369][105620] Updated weights for policy 1, policy_version 1938532 (0.0010) [2023-12-27 05:29:15,421][105620] Updated weights for policy 1, policy_version 1938542 (0.0010) [2023-12-27 05:29:15,789][105692] Updated weights for policy 0, policy_version 1933911 (0.0007) [2023-12-27 05:29:15,859][105692] Updated weights for policy 0, policy_version 1933921 (0.0005) [2023-12-27 05:29:15,905][105692] Updated weights for policy 0, policy_version 1933931 (0.0005) [2023-12-27 05:29:16,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19251.1, 300 sec: 19327.5). Total num frames: 991494144. Throughput: 0: 9654.2, 1: 9466.9. Samples: 991460044. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:29:16,063][104569] Avg episode reward: [(0, '7897.237'), (1, '9256.691')] [2023-12-27 05:29:16,069][105620] Updated weights for policy 1, policy_version 1938552 (0.0009) [2023-12-27 05:29:16,073][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001933936_495157248.pth... [2023-12-27 05:29:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001932784_494862336.pth [2023-12-27 05:29:16,130][105620] Updated weights for policy 1, policy_version 1938562 (0.0010) [2023-12-27 05:29:16,188][105620] Updated weights for policy 1, policy_version 1938572 (0.0010) [2023-12-27 05:29:16,209][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001938576_496345088.pth... [2023-12-27 05:29:16,213][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001937424_496050176.pth [2023-12-27 05:29:16,457][105692] Updated weights for policy 0, policy_version 1933941 (0.0007) [2023-12-27 05:29:16,505][105692] Updated weights for policy 0, policy_version 1933951 (0.0008) [2023-12-27 05:29:16,561][105692] Updated weights for policy 0, policy_version 1933961 (0.0008) [2023-12-27 05:29:16,889][105620] Updated weights for policy 1, policy_version 1938582 (0.0009) [2023-12-27 05:29:16,933][105620] Updated weights for policy 1, policy_version 1938592 (0.0005) [2023-12-27 05:29:16,986][105620] Updated weights for policy 1, policy_version 1938602 (0.0005) [2023-12-27 05:29:17,374][105692] Updated weights for policy 0, policy_version 1933971 (0.0009) [2023-12-27 05:29:17,423][105692] Updated weights for policy 0, policy_version 1933981 (0.0010) [2023-12-27 05:29:17,467][105692] Updated weights for policy 0, policy_version 1933991 (0.0010) [2023-12-27 05:29:17,665][105620] Updated weights for policy 1, policy_version 1938612 (0.0005) [2023-12-27 05:29:17,721][105620] Updated weights for policy 1, policy_version 1938622 (0.0005) [2023-12-27 05:29:17,785][105620] Updated weights for policy 1, policy_version 1938632 (0.0005) [2023-12-27 05:29:18,240][105692] Updated weights for policy 0, policy_version 1934001 (0.0010) [2023-12-27 05:29:18,306][105692] Updated weights for policy 0, policy_version 1934011 (0.0008) [2023-12-27 05:29:18,337][105620] Updated weights for policy 1, policy_version 1938642 (0.0007) [2023-12-27 05:29:18,387][105692] Updated weights for policy 0, policy_version 1934021 (0.0008) [2023-12-27 05:29:18,395][105620] Updated weights for policy 1, policy_version 1938652 (0.0008) [2023-12-27 05:29:18,449][105692] Updated weights for policy 0, policy_version 1934031 (0.0009) [2023-12-27 05:29:18,455][105620] Updated weights for policy 1, policy_version 1938662 (0.0008) [2023-12-27 05:29:18,507][105620] Updated weights for policy 1, policy_version 1938672 (0.0008) [2023-12-27 05:29:19,175][105692] Updated weights for policy 0, policy_version 1934041 (0.0009) [2023-12-27 05:29:19,239][105692] Updated weights for policy 0, policy_version 1934051 (0.0010) [2023-12-27 05:29:19,286][105620] Updated weights for policy 1, policy_version 1938682 (0.0008) [2023-12-27 05:29:19,298][105692] Updated weights for policy 0, policy_version 1934061 (0.0009) [2023-12-27 05:29:19,348][105620] Updated weights for policy 1, policy_version 1938692 (0.0008) [2023-12-27 05:29:19,413][105620] Updated weights for policy 1, policy_version 1938702 (0.0010) [2023-12-27 05:29:19,999][105692] Updated weights for policy 0, policy_version 1934071 (0.0007) [2023-12-27 05:29:20,048][105692] Updated weights for policy 0, policy_version 1934081 (0.0009) [2023-12-27 05:29:20,095][105692] Updated weights for policy 0, policy_version 1934091 (0.0009) [2023-12-27 05:29:20,207][105620] Updated weights for policy 1, policy_version 1938712 (0.0009) [2023-12-27 05:29:20,266][105620] Updated weights for policy 1, policy_version 1938722 (0.0008) [2023-12-27 05:29:20,331][105620] Updated weights for policy 1, policy_version 1938732 (0.0009) [2023-12-27 05:29:20,871][105692] Updated weights for policy 0, policy_version 1934101 (0.0009) [2023-12-27 05:29:20,934][105692] Updated weights for policy 0, policy_version 1934111 (0.0010) [2023-12-27 05:29:21,000][105692] Updated weights for policy 0, policy_version 1934121 (0.0010) [2023-12-27 05:29:21,060][105620] Updated weights for policy 1, policy_version 1938742 (0.0007) [2023-12-27 05:29:21,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19327.6). Total num frames: 991592448. Throughput: 0: 9711.7, 1: 9569.9. Samples: 991580820. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:29:21,063][104569] Avg episode reward: [(0, '8080.647'), (1, '9166.240')] [2023-12-27 05:29:21,130][105620] Updated weights for policy 1, policy_version 1938752 (0.0008) [2023-12-27 05:29:21,196][105620] Updated weights for policy 1, policy_version 1938762 (0.0009) [2023-12-27 05:29:21,807][105692] Updated weights for policy 0, policy_version 1934131 (0.0008) [2023-12-27 05:29:21,869][105692] Updated weights for policy 0, policy_version 1934141 (0.0006) [2023-12-27 05:29:21,938][105692] Updated weights for policy 0, policy_version 1934151 (0.0008) [2023-12-27 05:29:22,014][105620] Updated weights for policy 1, policy_version 1938772 (0.0008) [2023-12-27 05:29:22,080][105620] Updated weights for policy 1, policy_version 1938782 (0.0009) [2023-12-27 05:29:22,143][105620] Updated weights for policy 1, policy_version 1938792 (0.0010) [2023-12-27 05:29:22,665][105692] Updated weights for policy 0, policy_version 1934161 (0.0009) [2023-12-27 05:29:22,733][105692] Updated weights for policy 0, policy_version 1934171 (0.0009) [2023-12-27 05:29:22,800][105692] Updated weights for policy 0, policy_version 1934181 (0.0008) [2023-12-27 05:29:22,862][105692] Updated weights for policy 0, policy_version 1934191 (0.0008) [2023-12-27 05:29:22,937][105620] Updated weights for policy 1, policy_version 1938802 (0.0010) [2023-12-27 05:29:22,991][105620] Updated weights for policy 1, policy_version 1938812 (0.0008) [2023-12-27 05:29:23,041][105620] Updated weights for policy 1, policy_version 1938822 (0.0010) [2023-12-27 05:29:23,116][105620] Updated weights for policy 1, policy_version 1938832 (0.0009) [2023-12-27 05:29:23,538][105692] Updated weights for policy 0, policy_version 1934201 (0.0007) [2023-12-27 05:29:23,591][105692] Updated weights for policy 0, policy_version 1934211 (0.0009) [2023-12-27 05:29:23,648][105692] Updated weights for policy 0, policy_version 1934221 (0.0006) [2023-12-27 05:29:23,844][105620] Updated weights for policy 1, policy_version 1938842 (0.0010) [2023-12-27 05:29:23,902][105620] Updated weights for policy 1, policy_version 1938852 (0.0010) [2023-12-27 05:29:23,961][105620] Updated weights for policy 1, policy_version 1938862 (0.0007) [2023-12-27 05:29:24,448][105692] Updated weights for policy 0, policy_version 1934231 (0.0008) [2023-12-27 05:29:24,508][105692] Updated weights for policy 0, policy_version 1934241 (0.0006) [2023-12-27 05:29:24,553][105620] Updated weights for policy 1, policy_version 1938872 (0.0009) [2023-12-27 05:29:24,563][105692] Updated weights for policy 0, policy_version 1934251 (0.0006) [2023-12-27 05:29:24,614][105620] Updated weights for policy 1, policy_version 1938882 (0.0010) [2023-12-27 05:29:24,676][105620] Updated weights for policy 1, policy_version 1938892 (0.0009) [2023-12-27 05:29:25,305][105692] Updated weights for policy 0, policy_version 1934261 (0.0008) [2023-12-27 05:29:25,317][105620] Updated weights for policy 1, policy_version 1938902 (0.0009) [2023-12-27 05:29:25,364][105692] Updated weights for policy 0, policy_version 1934271 (0.0010) [2023-12-27 05:29:25,375][105620] Updated weights for policy 1, policy_version 1938912 (0.0010) [2023-12-27 05:29:25,420][105692] Updated weights for policy 0, policy_version 1934281 (0.0007) [2023-12-27 05:29:25,440][105620] Updated weights for policy 1, policy_version 1938922 (0.0010) [2023-12-27 05:29:26,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19251.3, 300 sec: 19327.6). Total num frames: 991682560. Throughput: 0: 9651.1, 1: 9574.5. Samples: 991695388. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:29:26,062][104569] Avg episode reward: [(0, '8357.815'), (1, '9073.884')] [2023-12-27 05:29:26,123][105620] Updated weights for policy 1, policy_version 1938932 (0.0010) [2023-12-27 05:29:26,178][105620] Updated weights for policy 1, policy_version 1938942 (0.0011) [2023-12-27 05:29:26,225][105692] Updated weights for policy 0, policy_version 1934291 (0.0007) [2023-12-27 05:29:26,231][105620] Updated weights for policy 1, policy_version 1938952 (0.0010) [2023-12-27 05:29:26,282][105692] Updated weights for policy 0, policy_version 1934301 (0.0006) [2023-12-27 05:29:26,338][105692] Updated weights for policy 0, policy_version 1934311 (0.0008) [2023-12-27 05:29:26,947][105620] Updated weights for policy 1, policy_version 1938962 (0.0010) [2023-12-27 05:29:27,006][105620] Updated weights for policy 1, policy_version 1938972 (0.0009) [2023-12-27 05:29:27,066][105620] Updated weights for policy 1, policy_version 1938982 (0.0005) [2023-12-27 05:29:27,116][105692] Updated weights for policy 0, policy_version 1934321 (0.0009) [2023-12-27 05:29:27,126][105620] Updated weights for policy 1, policy_version 1938992 (0.0005) [2023-12-27 05:29:27,168][105692] Updated weights for policy 0, policy_version 1934331 (0.0009) [2023-12-27 05:29:27,219][105692] Updated weights for policy 0, policy_version 1934341 (0.0009) [2023-12-27 05:29:27,277][105692] Updated weights for policy 0, policy_version 1934352 (0.0010) [2023-12-27 05:29:27,632][105620] Updated weights for policy 1, policy_version 1939002 (0.0006) [2023-12-27 05:29:27,681][105620] Updated weights for policy 1, policy_version 1939012 (0.0005) [2023-12-27 05:29:27,731][105620] Updated weights for policy 1, policy_version 1939022 (0.0005) [2023-12-27 05:29:28,155][105692] Updated weights for policy 0, policy_version 1934362 (0.0009) [2023-12-27 05:29:28,212][105692] Updated weights for policy 0, policy_version 1934373 (0.0010) [2023-12-27 05:29:28,273][105620] Updated weights for policy 1, policy_version 1939032 (0.0005) [2023-12-27 05:29:28,274][105692] Updated weights for policy 0, policy_version 1934383 (0.0009) [2023-12-27 05:29:28,325][105620] Updated weights for policy 1, policy_version 1939042 (0.0006) [2023-12-27 05:29:28,380][105620] Updated weights for policy 1, policy_version 1939052 (0.0008) [2023-12-27 05:29:29,052][105620] Updated weights for policy 1, policy_version 1939062 (0.0009) [2023-12-27 05:29:29,094][105692] Updated weights for policy 0, policy_version 1934393 (0.0007) [2023-12-27 05:29:29,107][105620] Updated weights for policy 1, policy_version 1939072 (0.0008) [2023-12-27 05:29:29,155][105692] Updated weights for policy 0, policy_version 1934403 (0.0007) [2023-12-27 05:29:29,157][105620] Updated weights for policy 1, policy_version 1939082 (0.0009) [2023-12-27 05:29:29,212][105692] Updated weights for policy 0, policy_version 1934413 (0.0008) [2023-12-27 05:29:29,871][105692] Updated weights for policy 0, policy_version 1934423 (0.0011) [2023-12-27 05:29:29,926][105620] Updated weights for policy 1, policy_version 1939092 (0.0006) [2023-12-27 05:29:29,934][105692] Updated weights for policy 0, policy_version 1934433 (0.0011) [2023-12-27 05:29:29,985][105620] Updated weights for policy 1, policy_version 1939102 (0.0006) [2023-12-27 05:29:29,990][105692] Updated weights for policy 0, policy_version 1934443 (0.0011) [2023-12-27 05:29:30,035][105620] Updated weights for policy 1, policy_version 1939112 (0.0006) [2023-12-27 05:29:30,699][105620] Updated weights for policy 1, policy_version 1939122 (0.0005) [2023-12-27 05:29:30,732][105692] Updated weights for policy 0, policy_version 1934453 (0.0011) [2023-12-27 05:29:30,755][105620] Updated weights for policy 1, policy_version 1939132 (0.0007) [2023-12-27 05:29:30,776][105692] Updated weights for policy 0, policy_version 1934463 (0.0010) [2023-12-27 05:29:30,800][105620] Updated weights for policy 1, policy_version 1939142 (0.0006) [2023-12-27 05:29:30,834][105692] Updated weights for policy 0, policy_version 1934473 (0.0010) [2023-12-27 05:29:30,850][105620] Updated weights for policy 1, policy_version 1939152 (0.0005) [2023-12-27 05:29:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 991789056. Throughput: 0: 9628.7, 1: 9702.7. Samples: 991755100. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:29:31,062][104569] Avg episode reward: [(0, '8447.998'), (1, '9164.259')] [2023-12-27 05:29:31,066][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001934480_495296512.pth... [2023-12-27 05:29:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001939152_496492544.pth... [2023-12-27 05:29:31,069][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001933360_495009792.pth [2023-12-27 05:29:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001937968_496189440.pth [2023-12-27 05:29:31,459][105620] Updated weights for policy 1, policy_version 1939162 (0.0011) [2023-12-27 05:29:31,512][105692] Updated weights for policy 0, policy_version 1934483 (0.0007) [2023-12-27 05:29:31,532][105620] Updated weights for policy 1, policy_version 1939172 (0.0011) [2023-12-27 05:29:31,571][105692] Updated weights for policy 0, policy_version 1934493 (0.0008) [2023-12-27 05:29:31,584][105620] Updated weights for policy 1, policy_version 1939182 (0.0010) [2023-12-27 05:29:31,634][105692] Updated weights for policy 0, policy_version 1934503 (0.0011) [2023-12-27 05:29:32,321][105620] Updated weights for policy 1, policy_version 1939192 (0.0008) [2023-12-27 05:29:32,345][105692] Updated weights for policy 0, policy_version 1934513 (0.0009) [2023-12-27 05:29:32,383][105620] Updated weights for policy 1, policy_version 1939202 (0.0009) [2023-12-27 05:29:32,431][105692] Updated weights for policy 0, policy_version 1934523 (0.0008) [2023-12-27 05:29:32,445][105620] Updated weights for policy 1, policy_version 1939212 (0.0007) [2023-12-27 05:29:32,497][105692] Updated weights for policy 0, policy_version 1934533 (0.0005) [2023-12-27 05:29:32,562][105692] Updated weights for policy 0, policy_version 1934543 (0.0007) [2023-12-27 05:29:33,137][105692] Updated weights for policy 0, policy_version 1934553 (0.0006) [2023-12-27 05:29:33,141][105620] Updated weights for policy 1, policy_version 1939222 (0.0005) [2023-12-27 05:29:33,196][105620] Updated weights for policy 1, policy_version 1939232 (0.0005) [2023-12-27 05:29:33,197][105692] Updated weights for policy 0, policy_version 1934563 (0.0007) [2023-12-27 05:29:33,250][105620] Updated weights for policy 1, policy_version 1939242 (0.0005) [2023-12-27 05:29:33,259][105692] Updated weights for policy 0, policy_version 1934573 (0.0006) [2023-12-27 05:29:33,765][105692] Updated weights for policy 0, policy_version 1934583 (0.0006) [2023-12-27 05:29:33,813][105692] Updated weights for policy 0, policy_version 1934593 (0.0008) [2023-12-27 05:29:33,836][105620] Updated weights for policy 1, policy_version 1939252 (0.0006) [2023-12-27 05:29:33,855][105692] Updated weights for policy 0, policy_version 1934603 (0.0006) [2023-12-27 05:29:33,894][105620] Updated weights for policy 1, policy_version 1939262 (0.0008) [2023-12-27 05:29:33,947][105620] Updated weights for policy 1, policy_version 1939272 (0.0007) [2023-12-27 05:29:34,602][105692] Updated weights for policy 0, policy_version 1934613 (0.0008) [2023-12-27 05:29:34,663][105620] Updated weights for policy 1, policy_version 1939282 (0.0006) [2023-12-27 05:29:34,663][105692] Updated weights for policy 0, policy_version 1934623 (0.0009) [2023-12-27 05:29:34,716][105620] Updated weights for policy 1, policy_version 1939292 (0.0007) [2023-12-27 05:29:34,721][105692] Updated weights for policy 0, policy_version 1934633 (0.0010) [2023-12-27 05:29:34,777][105620] Updated weights for policy 1, policy_version 1939302 (0.0008) [2023-12-27 05:29:34,834][105620] Updated weights for policy 1, policy_version 1939312 (0.0010) [2023-12-27 05:29:35,509][105620] Updated weights for policy 1, policy_version 1939322 (0.0008) [2023-12-27 05:29:35,513][105692] Updated weights for policy 0, policy_version 1934643 (0.0007) [2023-12-27 05:29:35,565][105620] Updated weights for policy 1, policy_version 1939332 (0.0006) [2023-12-27 05:29:35,573][105692] Updated weights for policy 0, policy_version 1934653 (0.0006) [2023-12-27 05:29:35,622][105620] Updated weights for policy 1, policy_version 1939342 (0.0005) [2023-12-27 05:29:35,631][105692] Updated weights for policy 0, policy_version 1934663 (0.0007) [2023-12-27 05:29:36,062][104569] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 991887360. Throughput: 0: 9636.5, 1: 9839.1. Samples: 991877828. Policy #0 lag: (min: 19.0, avg: 28.5, max: 51.0) [2023-12-27 05:29:36,063][104569] Avg episode reward: [(0, '8170.761'), (1, '9345.898')] [2023-12-27 05:29:36,290][105620] Updated weights for policy 1, policy_version 1939352 (0.0005) [2023-12-27 05:29:36,326][105692] Updated weights for policy 0, policy_version 1934673 (0.0009) [2023-12-27 05:29:36,346][105620] Updated weights for policy 1, policy_version 1939362 (0.0006) [2023-12-27 05:29:36,375][105692] Updated weights for policy 0, policy_version 1934683 (0.0008) [2023-12-27 05:29:36,406][105620] Updated weights for policy 1, policy_version 1939372 (0.0006) [2023-12-27 05:29:36,441][105692] Updated weights for policy 0, policy_version 1934693 (0.0009) [2023-12-27 05:29:36,503][105692] Updated weights for policy 0, policy_version 1934703 (0.0009) [2023-12-27 05:29:37,140][105620] Updated weights for policy 1, policy_version 1939382 (0.0007) [2023-12-27 05:29:37,199][105620] Updated weights for policy 1, policy_version 1939392 (0.0006) [2023-12-27 05:29:37,204][105692] Updated weights for policy 0, policy_version 1934713 (0.0009) [2023-12-27 05:29:37,255][105620] Updated weights for policy 1, policy_version 1939402 (0.0005) [2023-12-27 05:29:37,270][105692] Updated weights for policy 0, policy_version 1934723 (0.0008) [2023-12-27 05:29:37,328][105692] Updated weights for policy 0, policy_version 1934733 (0.0009) [2023-12-27 05:29:38,001][105620] Updated weights for policy 1, policy_version 1939412 (0.0007) [2023-12-27 05:29:38,024][105692] Updated weights for policy 0, policy_version 1934743 (0.0010) [2023-12-27 05:29:38,057][105620] Updated weights for policy 1, policy_version 1939422 (0.0007) [2023-12-27 05:29:38,079][105692] Updated weights for policy 0, policy_version 1934753 (0.0011) [2023-12-27 05:29:38,115][105620] Updated weights for policy 1, policy_version 1939432 (0.0006) [2023-12-27 05:29:38,129][105692] Updated weights for policy 0, policy_version 1934763 (0.0006) [2023-12-27 05:29:38,746][105692] Updated weights for policy 0, policy_version 1934773 (0.0008) [2023-12-27 05:29:38,813][105692] Updated weights for policy 0, policy_version 1934783 (0.0011) [2023-12-27 05:29:38,875][105692] Updated weights for policy 0, policy_version 1934793 (0.0011) [2023-12-27 05:29:38,962][105620] Updated weights for policy 1, policy_version 1939442 (0.0008) [2023-12-27 05:29:39,027][105620] Updated weights for policy 1, policy_version 1939452 (0.0009) [2023-12-27 05:29:39,084][105620] Updated weights for policy 1, policy_version 1939462 (0.0009) [2023-12-27 05:29:39,146][105620] Updated weights for policy 1, policy_version 1939472 (0.0009) [2023-12-27 05:29:39,500][105692] Updated weights for policy 0, policy_version 1934803 (0.0011) [2023-12-27 05:29:39,563][105692] Updated weights for policy 0, policy_version 1934813 (0.0011) [2023-12-27 05:29:39,624][105692] Updated weights for policy 0, policy_version 1934823 (0.0010) [2023-12-27 05:29:39,988][105620] Updated weights for policy 1, policy_version 1939482 (0.0008) [2023-12-27 05:29:40,049][105620] Updated weights for policy 1, policy_version 1939492 (0.0008) [2023-12-27 05:29:40,113][105620] Updated weights for policy 1, policy_version 1939502 (0.0008) [2023-12-27 05:29:40,368][105692] Updated weights for policy 0, policy_version 1934833 (0.0006) [2023-12-27 05:29:40,422][105692] Updated weights for policy 0, policy_version 1934843 (0.0009) [2023-12-27 05:29:40,478][105692] Updated weights for policy 0, policy_version 1934853 (0.0011) [2023-12-27 05:29:40,531][105692] Updated weights for policy 0, policy_version 1934863 (0.0011) [2023-12-27 05:29:40,816][105620] Updated weights for policy 1, policy_version 1939512 (0.0008) [2023-12-27 05:29:40,876][105620] Updated weights for policy 1, policy_version 1939522 (0.0007) [2023-12-27 05:29:40,946][105620] Updated weights for policy 1, policy_version 1939532 (0.0009) [2023-12-27 05:29:41,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 991985664. Throughput: 0: 9706.8, 1: 9820.5. Samples: 991993288. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:29:41,063][104569] Avg episode reward: [(0, '7902.229'), (1, '9345.871')] [2023-12-27 05:29:41,286][105692] Updated weights for policy 0, policy_version 1934873 (0.0008) [2023-12-27 05:29:41,354][105692] Updated weights for policy 0, policy_version 1934883 (0.0008) [2023-12-27 05:29:41,411][105692] Updated weights for policy 0, policy_version 1934893 (0.0009) [2023-12-27 05:29:41,633][105620] Updated weights for policy 1, policy_version 1939542 (0.0008) [2023-12-27 05:29:41,700][105620] Updated weights for policy 1, policy_version 1939552 (0.0010) [2023-12-27 05:29:41,770][105620] Updated weights for policy 1, policy_version 1939562 (0.0010) [2023-12-27 05:29:42,167][105692] Updated weights for policy 0, policy_version 1934903 (0.0010) [2023-12-27 05:29:42,212][105692] Updated weights for policy 0, policy_version 1934913 (0.0010) [2023-12-27 05:29:42,267][105692] Updated weights for policy 0, policy_version 1934923 (0.0011) [2023-12-27 05:29:42,503][105620] Updated weights for policy 1, policy_version 1939572 (0.0008) [2023-12-27 05:29:42,557][105620] Updated weights for policy 1, policy_version 1939582 (0.0008) [2023-12-27 05:29:42,606][105620] Updated weights for policy 1, policy_version 1939592 (0.0008) [2023-12-27 05:29:43,018][105692] Updated weights for policy 0, policy_version 1934933 (0.0010) [2023-12-27 05:29:43,073][105692] Updated weights for policy 0, policy_version 1934943 (0.0010) [2023-12-27 05:29:43,125][105692] Updated weights for policy 0, policy_version 1934953 (0.0010) [2023-12-27 05:29:43,374][105620] Updated weights for policy 1, policy_version 1939602 (0.0007) [2023-12-27 05:29:43,428][105620] Updated weights for policy 1, policy_version 1939612 (0.0008) [2023-12-27 05:29:43,493][105620] Updated weights for policy 1, policy_version 1939622 (0.0010) [2023-12-27 05:29:43,552][105620] Updated weights for policy 1, policy_version 1939632 (0.0006) [2023-12-27 05:29:43,752][105692] Updated weights for policy 0, policy_version 1934963 (0.0011) [2023-12-27 05:29:43,800][105692] Updated weights for policy 0, policy_version 1934973 (0.0010) [2023-12-27 05:29:43,860][105692] Updated weights for policy 0, policy_version 1934983 (0.0010) [2023-12-27 05:29:44,160][105620] Updated weights for policy 1, policy_version 1939642 (0.0005) [2023-12-27 05:29:44,206][105620] Updated weights for policy 1, policy_version 1939652 (0.0005) [2023-12-27 05:29:44,259][105620] Updated weights for policy 1, policy_version 1939662 (0.0005) [2023-12-27 05:29:44,454][105692] Updated weights for policy 0, policy_version 1934993 (0.0005) [2023-12-27 05:29:44,507][105692] Updated weights for policy 0, policy_version 1935003 (0.0005) [2023-12-27 05:29:44,562][105692] Updated weights for policy 0, policy_version 1935013 (0.0007) [2023-12-27 05:29:44,629][105692] Updated weights for policy 0, policy_version 1935023 (0.0010) [2023-12-27 05:29:44,908][105620] Updated weights for policy 1, policy_version 1939672 (0.0010) [2023-12-27 05:29:44,966][105620] Updated weights for policy 1, policy_version 1939682 (0.0008) [2023-12-27 05:29:45,027][105620] Updated weights for policy 1, policy_version 1939692 (0.0005) [2023-12-27 05:29:45,342][105692] Updated weights for policy 0, policy_version 1935033 (0.0009) [2023-12-27 05:29:45,407][105692] Updated weights for policy 0, policy_version 1935043 (0.0008) [2023-12-27 05:29:45,469][105692] Updated weights for policy 0, policy_version 1935053 (0.0008) [2023-12-27 05:29:45,727][105620] Updated weights for policy 1, policy_version 1939702 (0.0008) [2023-12-27 05:29:45,772][105620] Updated weights for policy 1, policy_version 1939712 (0.0010) [2023-12-27 05:29:45,820][105620] Updated weights for policy 1, policy_version 1939722 (0.0010) [2023-12-27 05:29:46,062][104569] Fps is (10 sec: 19660.2, 60 sec: 19524.1, 300 sec: 19355.3). Total num frames: 992083968. Throughput: 0: 9728.3, 1: 9860.3. Samples: 992051832. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:29:46,063][104569] Avg episode reward: [(0, '8175.413'), (1, '9253.447')] [2023-12-27 05:29:46,072][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001939728_496640000.pth... [2023-12-27 05:29:46,072][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001935056_495443968.pth... [2023-12-27 05:29:46,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001938576_496345088.pth [2023-12-27 05:29:46,080][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001933936_495157248.pth [2023-12-27 05:29:46,231][105692] Updated weights for policy 0, policy_version 1935063 (0.0008) [2023-12-27 05:29:46,283][105692] Updated weights for policy 0, policy_version 1935073 (0.0008) [2023-12-27 05:29:46,342][105692] Updated weights for policy 0, policy_version 1935083 (0.0008) [2023-12-27 05:29:46,581][105620] Updated weights for policy 1, policy_version 1939732 (0.0010) [2023-12-27 05:29:46,628][105620] Updated weights for policy 1, policy_version 1939742 (0.0010) [2023-12-27 05:29:46,672][105620] Updated weights for policy 1, policy_version 1939752 (0.0010) [2023-12-27 05:29:47,108][105692] Updated weights for policy 0, policy_version 1935093 (0.0008) [2023-12-27 05:29:47,178][105692] Updated weights for policy 0, policy_version 1935103 (0.0008) [2023-12-27 05:29:47,236][105692] Updated weights for policy 0, policy_version 1935113 (0.0008) [2023-12-27 05:29:47,429][105620] Updated weights for policy 1, policy_version 1939762 (0.0010) [2023-12-27 05:29:47,477][105620] Updated weights for policy 1, policy_version 1939772 (0.0010) [2023-12-27 05:29:47,524][105620] Updated weights for policy 1, policy_version 1939782 (0.0010) [2023-12-27 05:29:47,582][105620] Updated weights for policy 1, policy_version 1939792 (0.0010) [2023-12-27 05:29:47,975][105692] Updated weights for policy 0, policy_version 1935123 (0.0008) [2023-12-27 05:29:48,031][105692] Updated weights for policy 0, policy_version 1935133 (0.0008) [2023-12-27 05:29:48,082][105692] Updated weights for policy 0, policy_version 1935143 (0.0008) [2023-12-27 05:29:48,359][105620] Updated weights for policy 1, policy_version 1939802 (0.0010) [2023-12-27 05:29:48,422][105620] Updated weights for policy 1, policy_version 1939812 (0.0010) [2023-12-27 05:29:48,481][105620] Updated weights for policy 1, policy_version 1939822 (0.0010) [2023-12-27 05:29:48,850][105692] Updated weights for policy 0, policy_version 1935153 (0.0008) [2023-12-27 05:29:48,911][105692] Updated weights for policy 0, policy_version 1935163 (0.0008) [2023-12-27 05:29:48,970][105692] Updated weights for policy 0, policy_version 1935173 (0.0008) [2023-12-27 05:29:49,033][105692] Updated weights for policy 0, policy_version 1935183 (0.0008) [2023-12-27 05:29:49,209][105620] Updated weights for policy 1, policy_version 1939832 (0.0010) [2023-12-27 05:29:49,273][105620] Updated weights for policy 1, policy_version 1939842 (0.0010) [2023-12-27 05:29:49,322][105620] Updated weights for policy 1, policy_version 1939852 (0.0010) [2023-12-27 05:29:49,814][105692] Updated weights for policy 0, policy_version 1935193 (0.0008) [2023-12-27 05:29:49,873][105692] Updated weights for policy 0, policy_version 1935203 (0.0008) [2023-12-27 05:29:49,943][105692] Updated weights for policy 0, policy_version 1935213 (0.0008) [2023-12-27 05:29:50,124][105620] Updated weights for policy 1, policy_version 1939862 (0.0011) [2023-12-27 05:29:50,187][105620] Updated weights for policy 1, policy_version 1939872 (0.0010) [2023-12-27 05:29:50,245][105620] Updated weights for policy 1, policy_version 1939882 (0.0011) [2023-12-27 05:29:50,740][105692] Updated weights for policy 0, policy_version 1935223 (0.0008) [2023-12-27 05:29:50,808][105692] Updated weights for policy 0, policy_version 1935233 (0.0008) [2023-12-27 05:29:50,872][105692] Updated weights for policy 0, policy_version 1935243 (0.0008) [2023-12-27 05:29:51,026][105620] Updated weights for policy 1, policy_version 1939892 (0.0010) [2023-12-27 05:29:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19355.3). Total num frames: 992174080. Throughput: 0: 9676.4, 1: 9882.0. Samples: 992166840. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:29:51,062][104569] Avg episode reward: [(0, '8622.674'), (1, '9253.481')] [2023-12-27 05:29:51,088][105620] Updated weights for policy 1, policy_version 1939902 (0.0010) [2023-12-27 05:29:51,154][105620] Updated weights for policy 1, policy_version 1939912 (0.0011) [2023-12-27 05:29:51,658][105692] Updated weights for policy 0, policy_version 1935253 (0.0008) [2023-12-27 05:29:51,721][105692] Updated weights for policy 0, policy_version 1935263 (0.0008) [2023-12-27 05:29:51,785][105692] Updated weights for policy 0, policy_version 1935273 (0.0007) [2023-12-27 05:29:51,915][105620] Updated weights for policy 1, policy_version 1939922 (0.0010) [2023-12-27 05:29:51,977][105620] Updated weights for policy 1, policy_version 1939932 (0.0009) [2023-12-27 05:29:52,039][105620] Updated weights for policy 1, policy_version 1939942 (0.0008) [2023-12-27 05:29:52,091][105620] Updated weights for policy 1, policy_version 1939952 (0.0009) [2023-12-27 05:29:52,508][105692] Updated weights for policy 0, policy_version 1935283 (0.0006) [2023-12-27 05:29:52,571][105692] Updated weights for policy 0, policy_version 1935293 (0.0009) [2023-12-27 05:29:52,628][105692] Updated weights for policy 0, policy_version 1935303 (0.0008) [2023-12-27 05:29:52,901][105620] Updated weights for policy 1, policy_version 1939962 (0.0010) [2023-12-27 05:29:52,974][105620] Updated weights for policy 1, policy_version 1939972 (0.0010) [2023-12-27 05:29:53,026][105620] Updated weights for policy 1, policy_version 1939982 (0.0009) [2023-12-27 05:29:53,204][105692] Updated weights for policy 0, policy_version 1935313 (0.0008) [2023-12-27 05:29:53,268][105692] Updated weights for policy 0, policy_version 1935323 (0.0007) [2023-12-27 05:29:53,334][105692] Updated weights for policy 0, policy_version 1935333 (0.0005) [2023-12-27 05:29:53,392][105692] Updated weights for policy 0, policy_version 1935343 (0.0009) [2023-12-27 05:29:53,837][105620] Updated weights for policy 1, policy_version 1939992 (0.0008) [2023-12-27 05:29:53,895][105620] Updated weights for policy 1, policy_version 1940002 (0.0008) [2023-12-27 05:29:53,952][105620] Updated weights for policy 1, policy_version 1940012 (0.0009) [2023-12-27 05:29:54,080][105692] Updated weights for policy 0, policy_version 1935353 (0.0010) [2023-12-27 05:29:54,139][105692] Updated weights for policy 0, policy_version 1935363 (0.0010) [2023-12-27 05:29:54,196][105692] Updated weights for policy 0, policy_version 1935373 (0.0010) [2023-12-27 05:29:54,669][105620] Updated weights for policy 1, policy_version 1940022 (0.0008) [2023-12-27 05:29:54,723][105620] Updated weights for policy 1, policy_version 1940032 (0.0007) [2023-12-27 05:29:54,781][105620] Updated weights for policy 1, policy_version 1940042 (0.0008) [2023-12-27 05:29:54,919][105692] Updated weights for policy 0, policy_version 1935383 (0.0010) [2023-12-27 05:29:54,972][105692] Updated weights for policy 0, policy_version 1935393 (0.0007) [2023-12-27 05:29:55,034][105692] Updated weights for policy 0, policy_version 1935403 (0.0010) [2023-12-27 05:29:55,508][105620] Updated weights for policy 1, policy_version 1940052 (0.0008) [2023-12-27 05:29:55,560][105620] Updated weights for policy 1, policy_version 1940062 (0.0008) [2023-12-27 05:29:55,611][105620] Updated weights for policy 1, policy_version 1940072 (0.0008) [2023-12-27 05:29:55,716][105692] Updated weights for policy 0, policy_version 1935413 (0.0010) [2023-12-27 05:29:55,768][105692] Updated weights for policy 0, policy_version 1935423 (0.0009) [2023-12-27 05:29:55,828][105692] Updated weights for policy 0, policy_version 1935433 (0.0005) [2023-12-27 05:29:56,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19387.8, 300 sec: 19355.3). Total num frames: 992272384. Throughput: 0: 9664.9, 1: 9822.7. Samples: 992279612. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:29:56,063][104569] Avg episode reward: [(0, '8443.473'), (1, '9161.218')] [2023-12-27 05:29:56,355][105620] Updated weights for policy 1, policy_version 1940082 (0.0007) [2023-12-27 05:29:56,418][105620] Updated weights for policy 1, policy_version 1940092 (0.0008) [2023-12-27 05:29:56,473][105620] Updated weights for policy 1, policy_version 1940102 (0.0008) [2023-12-27 05:29:56,531][105620] Updated weights for policy 1, policy_version 1940112 (0.0008) [2023-12-27 05:29:56,537][105692] Updated weights for policy 0, policy_version 1935443 (0.0007) [2023-12-27 05:29:56,589][105692] Updated weights for policy 0, policy_version 1935453 (0.0010) [2023-12-27 05:29:56,650][105692] Updated weights for policy 0, policy_version 1935463 (0.0010) [2023-12-27 05:29:57,152][105620] Updated weights for policy 1, policy_version 1940122 (0.0006) [2023-12-27 05:29:57,199][105620] Updated weights for policy 1, policy_version 1940132 (0.0008) [2023-12-27 05:29:57,242][105620] Updated weights for policy 1, policy_version 1940142 (0.0006) [2023-12-27 05:29:57,396][105692] Updated weights for policy 0, policy_version 1935473 (0.0010) [2023-12-27 05:29:57,464][105692] Updated weights for policy 0, policy_version 1935483 (0.0010) [2023-12-27 05:29:57,529][105692] Updated weights for policy 0, policy_version 1935493 (0.0010) [2023-12-27 05:29:57,591][105692] Updated weights for policy 0, policy_version 1935503 (0.0010) [2023-12-27 05:29:57,869][105620] Updated weights for policy 1, policy_version 1940152 (0.0008) [2023-12-27 05:29:57,918][105620] Updated weights for policy 1, policy_version 1940162 (0.0008) [2023-12-27 05:29:57,973][105620] Updated weights for policy 1, policy_version 1940172 (0.0008) [2023-12-27 05:29:58,303][105692] Updated weights for policy 0, policy_version 1935513 (0.0008) [2023-12-27 05:29:58,369][105692] Updated weights for policy 0, policy_version 1935523 (0.0008) [2023-12-27 05:29:58,436][105692] Updated weights for policy 0, policy_version 1935533 (0.0007) [2023-12-27 05:29:58,824][105620] Updated weights for policy 1, policy_version 1940182 (0.0009) [2023-12-27 05:29:58,883][105620] Updated weights for policy 1, policy_version 1940192 (0.0009) [2023-12-27 05:29:58,940][105620] Updated weights for policy 1, policy_version 1940202 (0.0009) [2023-12-27 05:29:59,146][105692] Updated weights for policy 0, policy_version 1935543 (0.0009) [2023-12-27 05:29:59,204][105692] Updated weights for policy 0, policy_version 1935553 (0.0009) [2023-12-27 05:29:59,272][105692] Updated weights for policy 0, policy_version 1935563 (0.0009) [2023-12-27 05:29:59,686][105620] Updated weights for policy 1, policy_version 1940212 (0.0008) [2023-12-27 05:29:59,752][105620] Updated weights for policy 1, policy_version 1940222 (0.0005) [2023-12-27 05:29:59,808][105620] Updated weights for policy 1, policy_version 1940232 (0.0005) [2023-12-27 05:30:00,052][105692] Updated weights for policy 0, policy_version 1935573 (0.0009) [2023-12-27 05:30:00,114][105692] Updated weights for policy 0, policy_version 1935583 (0.0009) [2023-12-27 05:30:00,173][105692] Updated weights for policy 0, policy_version 1935593 (0.0009) [2023-12-27 05:30:00,502][105620] Updated weights for policy 1, policy_version 1940242 (0.0009) [2023-12-27 05:30:00,546][105620] Updated weights for policy 1, policy_version 1940252 (0.0010) [2023-12-27 05:30:00,600][105620] Updated weights for policy 1, policy_version 1940262 (0.0010) [2023-12-27 05:30:00,654][105620] Updated weights for policy 1, policy_version 1940272 (0.0010) [2023-12-27 05:30:00,828][105692] Updated weights for policy 0, policy_version 1935603 (0.0008) [2023-12-27 05:30:00,882][105692] Updated weights for policy 0, policy_version 1935613 (0.0005) [2023-12-27 05:30:00,940][105692] Updated weights for policy 0, policy_version 1935623 (0.0006) [2023-12-27 05:30:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19355.3). Total num frames: 992370688. Throughput: 0: 9658.9, 1: 9841.0. Samples: 992337536. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:30:01,062][104569] Avg episode reward: [(0, '8444.597'), (1, '9069.015')] [2023-12-27 05:30:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001935632_495591424.pth... [2023-12-27 05:30:01,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001940272_496779264.pth... [2023-12-27 05:30:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001934480_495296512.pth [2023-12-27 05:30:01,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001939152_496492544.pth [2023-12-27 05:30:01,425][105620] Updated weights for policy 1, policy_version 1940282 (0.0008) [2023-12-27 05:30:01,480][105620] Updated weights for policy 1, policy_version 1940292 (0.0007) [2023-12-27 05:30:01,538][105620] Updated weights for policy 1, policy_version 1940302 (0.0007) [2023-12-27 05:30:01,676][105692] Updated weights for policy 0, policy_version 1935633 (0.0007) [2023-12-27 05:30:01,738][105692] Updated weights for policy 0, policy_version 1935643 (0.0010) [2023-12-27 05:30:01,796][105692] Updated weights for policy 0, policy_version 1935653 (0.0009) [2023-12-27 05:30:01,851][105692] Updated weights for policy 0, policy_version 1935664 (0.0009) [2023-12-27 05:30:02,203][105620] Updated weights for policy 1, policy_version 1940312 (0.0005) [2023-12-27 05:30:02,259][105620] Updated weights for policy 1, policy_version 1940322 (0.0006) [2023-12-27 05:30:02,318][105620] Updated weights for policy 1, policy_version 1940332 (0.0007) [2023-12-27 05:30:02,597][105692] Updated weights for policy 0, policy_version 1935674 (0.0010) [2023-12-27 05:30:02,662][105692] Updated weights for policy 0, policy_version 1935684 (0.0010) [2023-12-27 05:30:02,726][105692] Updated weights for policy 0, policy_version 1935694 (0.0010) [2023-12-27 05:30:03,033][105620] Updated weights for policy 1, policy_version 1940342 (0.0009) [2023-12-27 05:30:03,086][105620] Updated weights for policy 1, policy_version 1940353 (0.0010) [2023-12-27 05:30:03,143][105620] Updated weights for policy 1, policy_version 1940364 (0.0010) [2023-12-27 05:30:03,334][105692] Updated weights for policy 0, policy_version 1935704 (0.0006) [2023-12-27 05:30:03,387][105692] Updated weights for policy 0, policy_version 1935714 (0.0005) [2023-12-27 05:30:03,438][105692] Updated weights for policy 0, policy_version 1935724 (0.0005) [2023-12-27 05:30:03,892][105620] Updated weights for policy 1, policy_version 1940374 (0.0008) [2023-12-27 05:30:03,953][105620] Updated weights for policy 1, policy_version 1940384 (0.0011) [2023-12-27 05:30:03,968][105692] Updated weights for policy 0, policy_version 1935734 (0.0005) [2023-12-27 05:30:04,009][105620] Updated weights for policy 1, policy_version 1940394 (0.0010) [2023-12-27 05:30:04,028][105692] Updated weights for policy 0, policy_version 1935744 (0.0006) [2023-12-27 05:30:04,090][105692] Updated weights for policy 0, policy_version 1935754 (0.0008) [2023-12-27 05:30:04,765][105620] Updated weights for policy 1, policy_version 1940404 (0.0010) [2023-12-27 05:30:04,821][105620] Updated weights for policy 1, policy_version 1940414 (0.0010) [2023-12-27 05:30:04,854][105692] Updated weights for policy 0, policy_version 1935764 (0.0007) [2023-12-27 05:30:04,886][105620] Updated weights for policy 1, policy_version 1940424 (0.0010) [2023-12-27 05:30:04,909][105692] Updated weights for policy 0, policy_version 1935774 (0.0007) [2023-12-27 05:30:04,960][105692] Updated weights for policy 0, policy_version 1935784 (0.0007) [2023-12-27 05:30:05,626][105620] Updated weights for policy 1, policy_version 1940434 (0.0010) [2023-12-27 05:30:05,692][105620] Updated weights for policy 1, policy_version 1940444 (0.0010) [2023-12-27 05:30:05,728][105692] Updated weights for policy 0, policy_version 1935794 (0.0007) [2023-12-27 05:30:05,754][105620] Updated weights for policy 1, policy_version 1940454 (0.0010) [2023-12-27 05:30:05,787][105692] Updated weights for policy 0, policy_version 1935804 (0.0005) [2023-12-27 05:30:05,819][105620] Updated weights for policy 1, policy_version 1940464 (0.0010) [2023-12-27 05:30:05,841][105692] Updated weights for policy 0, policy_version 1935814 (0.0008) [2023-12-27 05:30:05,901][105692] Updated weights for policy 0, policy_version 1935824 (0.0008) [2023-12-27 05:30:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19355.3). Total num frames: 992468992. Throughput: 0: 9701.9, 1: 9731.9. Samples: 992455340. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:30:06,062][104569] Avg episode reward: [(0, '8351.329'), (1, '9161.504')] [2023-12-27 05:30:06,553][105620] Updated weights for policy 1, policy_version 1940474 (0.0011) [2023-12-27 05:30:06,613][105620] Updated weights for policy 1, policy_version 1940484 (0.0011) [2023-12-27 05:30:06,670][105620] Updated weights for policy 1, policy_version 1940494 (0.0011) [2023-12-27 05:30:06,685][105692] Updated weights for policy 0, policy_version 1935834 (0.0006) [2023-12-27 05:30:06,734][105692] Updated weights for policy 0, policy_version 1935844 (0.0008) [2023-12-27 05:30:06,798][105692] Updated weights for policy 0, policy_version 1935854 (0.0008) [2023-12-27 05:30:07,432][105620] Updated weights for policy 1, policy_version 1940504 (0.0011) [2023-12-27 05:30:07,499][105620] Updated weights for policy 1, policy_version 1940514 (0.0011) [2023-12-27 05:30:07,527][105692] Updated weights for policy 0, policy_version 1935864 (0.0007) [2023-12-27 05:30:07,567][105620] Updated weights for policy 1, policy_version 1940524 (0.0010) [2023-12-27 05:30:07,591][105692] Updated weights for policy 0, policy_version 1935874 (0.0007) [2023-12-27 05:30:07,656][105692] Updated weights for policy 0, policy_version 1935884 (0.0009) [2023-12-27 05:30:08,214][105620] Updated weights for policy 1, policy_version 1940534 (0.0011) [2023-12-27 05:30:08,261][105620] Updated weights for policy 1, policy_version 1940544 (0.0010) [2023-12-27 05:30:08,310][105620] Updated weights for policy 1, policy_version 1940554 (0.0010) [2023-12-27 05:30:08,440][105692] Updated weights for policy 0, policy_version 1935894 (0.0009) [2023-12-27 05:30:08,501][105692] Updated weights for policy 0, policy_version 1935904 (0.0009) [2023-12-27 05:30:08,550][105692] Updated weights for policy 0, policy_version 1935914 (0.0008) [2023-12-27 05:30:09,004][105620] Updated weights for policy 1, policy_version 1940564 (0.0009) [2023-12-27 05:30:09,066][105620] Updated weights for policy 1, policy_version 1940574 (0.0010) [2023-12-27 05:30:09,126][105620] Updated weights for policy 1, policy_version 1940584 (0.0009) [2023-12-27 05:30:09,415][105692] Updated weights for policy 0, policy_version 1935924 (0.0008) [2023-12-27 05:30:09,472][105692] Updated weights for policy 0, policy_version 1935934 (0.0008) [2023-12-27 05:30:09,528][105692] Updated weights for policy 0, policy_version 1935944 (0.0008) [2023-12-27 05:30:09,825][105620] Updated weights for policy 1, policy_version 1940594 (0.0006) [2023-12-27 05:30:09,884][105620] Updated weights for policy 1, policy_version 1940604 (0.0008) [2023-12-27 05:30:09,951][105620] Updated weights for policy 1, policy_version 1940614 (0.0009) [2023-12-27 05:30:10,015][105620] Updated weights for policy 1, policy_version 1940624 (0.0008) [2023-12-27 05:30:10,328][105692] Updated weights for policy 0, policy_version 1935954 (0.0008) [2023-12-27 05:30:10,390][105692] Updated weights for policy 0, policy_version 1935964 (0.0009) [2023-12-27 05:30:10,449][105692] Updated weights for policy 0, policy_version 1935974 (0.0009) [2023-12-27 05:30:10,509][105692] Updated weights for policy 0, policy_version 1935984 (0.0009) [2023-12-27 05:30:10,755][105620] Updated weights for policy 1, policy_version 1940634 (0.0008) [2023-12-27 05:30:10,809][105620] Updated weights for policy 1, policy_version 1940644 (0.0009) [2023-12-27 05:30:10,863][105620] Updated weights for policy 1, policy_version 1940654 (0.0009) [2023-12-27 05:30:11,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19327.6). Total num frames: 992559104. Throughput: 0: 9641.5, 1: 9716.2. Samples: 992566484. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:30:11,062][104569] Avg episode reward: [(0, '8354.397'), (1, '9253.903')] [2023-12-27 05:30:11,320][105692] Updated weights for policy 0, policy_version 1935994 (0.0009) [2023-12-27 05:30:11,385][105692] Updated weights for policy 0, policy_version 1936004 (0.0009) [2023-12-27 05:30:11,451][105692] Updated weights for policy 0, policy_version 1936014 (0.0009) [2023-12-27 05:30:11,656][105620] Updated weights for policy 1, policy_version 1940664 (0.0008) [2023-12-27 05:30:11,719][105620] Updated weights for policy 1, policy_version 1940674 (0.0007) [2023-12-27 05:30:11,790][105620] Updated weights for policy 1, policy_version 1940684 (0.0007) [2023-12-27 05:30:12,304][105692] Updated weights for policy 0, policy_version 1936024 (0.0009) [2023-12-27 05:30:12,371][105692] Updated weights for policy 0, policy_version 1936034 (0.0008) [2023-12-27 05:30:12,421][105620] Updated weights for policy 1, policy_version 1940694 (0.0007) [2023-12-27 05:30:12,424][105692] Updated weights for policy 0, policy_version 1936044 (0.0007) [2023-12-27 05:30:12,480][105620] Updated weights for policy 1, policy_version 1940704 (0.0009) [2023-12-27 05:30:12,531][105620] Updated weights for policy 1, policy_version 1940714 (0.0008) [2023-12-27 05:30:13,231][105692] Updated weights for policy 0, policy_version 1936054 (0.0007) [2023-12-27 05:30:13,235][105620] Updated weights for policy 1, policy_version 1940724 (0.0008) [2023-12-27 05:30:13,284][105620] Updated weights for policy 1, policy_version 1940734 (0.0008) [2023-12-27 05:30:13,291][105692] Updated weights for policy 0, policy_version 1936064 (0.0005) [2023-12-27 05:30:13,338][105620] Updated weights for policy 1, policy_version 1940744 (0.0007) [2023-12-27 05:30:13,353][105692] Updated weights for policy 0, policy_version 1936074 (0.0006) [2023-12-27 05:30:13,908][105692] Updated weights for policy 0, policy_version 1936084 (0.0006) [2023-12-27 05:30:13,966][105692] Updated weights for policy 0, policy_version 1936094 (0.0006) [2023-12-27 05:30:14,022][105692] Updated weights for policy 0, policy_version 1936104 (0.0007) [2023-12-27 05:30:14,071][105620] Updated weights for policy 1, policy_version 1940754 (0.0005) [2023-12-27 05:30:14,133][105620] Updated weights for policy 1, policy_version 1940764 (0.0006) [2023-12-27 05:30:14,190][105620] Updated weights for policy 1, policy_version 1940774 (0.0010) [2023-12-27 05:30:14,248][105620] Updated weights for policy 1, policy_version 1940784 (0.0009) [2023-12-27 05:30:14,663][105692] Updated weights for policy 0, policy_version 1936114 (0.0008) [2023-12-27 05:30:14,733][105692] Updated weights for policy 0, policy_version 1936124 (0.0007) [2023-12-27 05:30:14,800][105692] Updated weights for policy 0, policy_version 1936134 (0.0008) [2023-12-27 05:30:14,852][105692] Updated weights for policy 0, policy_version 1936144 (0.0008) [2023-12-27 05:30:14,878][105620] Updated weights for policy 1, policy_version 1940794 (0.0007) [2023-12-27 05:30:14,950][105620] Updated weights for policy 1, policy_version 1940804 (0.0006) [2023-12-27 05:30:15,013][105620] Updated weights for policy 1, policy_version 1940814 (0.0006) [2023-12-27 05:30:15,589][105620] Updated weights for policy 1, policy_version 1940824 (0.0008) [2023-12-27 05:30:15,608][105692] Updated weights for policy 0, policy_version 1936154 (0.0006) [2023-12-27 05:30:15,645][105620] Updated weights for policy 1, policy_version 1940834 (0.0007) [2023-12-27 05:30:15,663][105692] Updated weights for policy 0, policy_version 1936164 (0.0007) [2023-12-27 05:30:15,707][105620] Updated weights for policy 1, policy_version 1940844 (0.0007) [2023-12-27 05:30:15,718][105692] Updated weights for policy 0, policy_version 1936174 (0.0007) [2023-12-27 05:30:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19327.6). Total num frames: 992657408. Throughput: 0: 9664.1, 1: 9627.6. Samples: 992623224. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:30:16,062][104569] Avg episode reward: [(0, '8628.808'), (1, '9253.789')] [2023-12-27 05:30:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001936176_495730688.pth... [2023-12-27 05:30:16,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001940848_496926720.pth... [2023-12-27 05:30:16,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001939728_496640000.pth [2023-12-27 05:30:16,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001935056_495443968.pth [2023-12-27 05:30:16,294][105620] Updated weights for policy 1, policy_version 1940854 (0.0006) [2023-12-27 05:30:16,354][105620] Updated weights for policy 1, policy_version 1940864 (0.0005) [2023-12-27 05:30:16,410][105620] Updated weights for policy 1, policy_version 1940874 (0.0005) [2023-12-27 05:30:16,600][105692] Updated weights for policy 0, policy_version 1936184 (0.0010) [2023-12-27 05:30:16,655][105692] Updated weights for policy 0, policy_version 1936194 (0.0010) [2023-12-27 05:30:16,712][105692] Updated weights for policy 0, policy_version 1936204 (0.0009) [2023-12-27 05:30:16,937][105620] Updated weights for policy 1, policy_version 1940884 (0.0007) [2023-12-27 05:30:16,997][105620] Updated weights for policy 1, policy_version 1940894 (0.0010) [2023-12-27 05:30:17,063][105620] Updated weights for policy 1, policy_version 1940904 (0.0010) [2023-12-27 05:30:17,546][105692] Updated weights for policy 0, policy_version 1936214 (0.0009) [2023-12-27 05:30:17,599][105692] Updated weights for policy 0, policy_version 1936224 (0.0010) [2023-12-27 05:30:17,656][105692] Updated weights for policy 0, policy_version 1936234 (0.0011) [2023-12-27 05:30:17,765][105620] Updated weights for policy 1, policy_version 1940914 (0.0011) [2023-12-27 05:30:17,825][105620] Updated weights for policy 1, policy_version 1940924 (0.0010) [2023-12-27 05:30:17,884][105620] Updated weights for policy 1, policy_version 1940934 (0.0010) [2023-12-27 05:30:17,947][105620] Updated weights for policy 1, policy_version 1940944 (0.0011) [2023-12-27 05:30:18,427][105692] Updated weights for policy 0, policy_version 1936244 (0.0010) [2023-12-27 05:30:18,483][105692] Updated weights for policy 0, policy_version 1936254 (0.0008) [2023-12-27 05:30:18,539][105692] Updated weights for policy 0, policy_version 1936264 (0.0008) [2023-12-27 05:30:18,665][105620] Updated weights for policy 1, policy_version 1940954 (0.0011) [2023-12-27 05:30:18,725][105620] Updated weights for policy 1, policy_version 1940964 (0.0010) [2023-12-27 05:30:18,778][105620] Updated weights for policy 1, policy_version 1940974 (0.0010) [2023-12-27 05:30:19,352][105692] Updated weights for policy 0, policy_version 1936274 (0.0008) [2023-12-27 05:30:19,415][105692] Updated weights for policy 0, policy_version 1936284 (0.0009) [2023-12-27 05:30:19,479][105692] Updated weights for policy 0, policy_version 1936294 (0.0009) [2023-12-27 05:30:19,542][105692] Updated weights for policy 0, policy_version 1936304 (0.0009) [2023-12-27 05:30:19,581][105620] Updated weights for policy 1, policy_version 1940984 (0.0011) [2023-12-27 05:30:19,641][105620] Updated weights for policy 1, policy_version 1940994 (0.0010) [2023-12-27 05:30:19,707][105620] Updated weights for policy 1, policy_version 1941004 (0.0010) [2023-12-27 05:30:20,359][105692] Updated weights for policy 0, policy_version 1936314 (0.0009) [2023-12-27 05:30:20,418][105692] Updated weights for policy 0, policy_version 1936324 (0.0008) [2023-12-27 05:30:20,478][105692] Updated weights for policy 0, policy_version 1936334 (0.0008) [2023-12-27 05:30:20,507][105620] Updated weights for policy 1, policy_version 1941014 (0.0011) [2023-12-27 05:30:20,569][105620] Updated weights for policy 1, policy_version 1941024 (0.0011) [2023-12-27 05:30:20,639][105620] Updated weights for policy 1, policy_version 1941034 (0.0011) [2023-12-27 05:30:21,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19327.6). Total num frames: 992747520. Throughput: 0: 9502.0, 1: 9655.5. Samples: 992739916. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:30:21,063][104569] Avg episode reward: [(0, '8899.688'), (1, '9255.605')] [2023-12-27 05:30:21,273][105692] Updated weights for policy 0, policy_version 1936344 (0.0007) [2023-12-27 05:30:21,342][105692] Updated weights for policy 0, policy_version 1936354 (0.0006) [2023-12-27 05:30:21,413][105692] Updated weights for policy 0, policy_version 1936364 (0.0008) [2023-12-27 05:30:21,459][105620] Updated weights for policy 1, policy_version 1941044 (0.0011) [2023-12-27 05:30:21,527][105620] Updated weights for policy 1, policy_version 1941054 (0.0011) [2023-12-27 05:30:21,600][105620] Updated weights for policy 1, policy_version 1941064 (0.0009) [2023-12-27 05:30:22,101][105692] Updated weights for policy 0, policy_version 1936374 (0.0007) [2023-12-27 05:30:22,168][105692] Updated weights for policy 0, policy_version 1936384 (0.0006) [2023-12-27 05:30:22,227][105692] Updated weights for policy 0, policy_version 1936394 (0.0006) [2023-12-27 05:30:22,343][105620] Updated weights for policy 1, policy_version 1941074 (0.0009) [2023-12-27 05:30:22,415][105620] Updated weights for policy 1, policy_version 1941084 (0.0009) [2023-12-27 05:30:22,480][105620] Updated weights for policy 1, policy_version 1941094 (0.0008) [2023-12-27 05:30:22,543][105620] Updated weights for policy 1, policy_version 1941104 (0.0009) [2023-12-27 05:30:22,956][105692] Updated weights for policy 0, policy_version 1936404 (0.0007) [2023-12-27 05:30:23,022][105692] Updated weights for policy 0, policy_version 1936414 (0.0008) [2023-12-27 05:30:23,086][105692] Updated weights for policy 0, policy_version 1936424 (0.0008) [2023-12-27 05:30:23,354][105620] Updated weights for policy 1, policy_version 1941114 (0.0009) [2023-12-27 05:30:23,412][105620] Updated weights for policy 1, policy_version 1941124 (0.0008) [2023-12-27 05:30:23,458][105620] Updated weights for policy 1, policy_version 1941134 (0.0008) [2023-12-27 05:30:23,841][105692] Updated weights for policy 0, policy_version 1936434 (0.0009) [2023-12-27 05:30:23,904][105692] Updated weights for policy 0, policy_version 1936444 (0.0009) [2023-12-27 05:30:23,967][105692] Updated weights for policy 0, policy_version 1936454 (0.0009) [2023-12-27 05:30:24,029][105692] Updated weights for policy 0, policy_version 1936464 (0.0009) [2023-12-27 05:30:24,156][105620] Updated weights for policy 1, policy_version 1941144 (0.0009) [2023-12-27 05:30:24,220][105620] Updated weights for policy 1, policy_version 1941154 (0.0009) [2023-12-27 05:30:24,290][105620] Updated weights for policy 1, policy_version 1941164 (0.0010) [2023-12-27 05:30:24,787][105692] Updated weights for policy 0, policy_version 1936474 (0.0008) [2023-12-27 05:30:24,846][105692] Updated weights for policy 0, policy_version 1936484 (0.0007) [2023-12-27 05:30:24,909][105692] Updated weights for policy 0, policy_version 1936494 (0.0007) [2023-12-27 05:30:25,046][105620] Updated weights for policy 1, policy_version 1941174 (0.0009) [2023-12-27 05:30:25,099][105620] Updated weights for policy 1, policy_version 1941184 (0.0009) [2023-12-27 05:30:25,153][105620] Updated weights for policy 1, policy_version 1941194 (0.0008) [2023-12-27 05:30:25,568][105692] Updated weights for policy 0, policy_version 1936504 (0.0009) [2023-12-27 05:30:25,632][105692] Updated weights for policy 0, policy_version 1936514 (0.0010) [2023-12-27 05:30:25,693][105692] Updated weights for policy 0, policy_version 1936524 (0.0009) [2023-12-27 05:30:25,831][105620] Updated weights for policy 1, policy_version 1941204 (0.0009) [2023-12-27 05:30:25,878][105620] Updated weights for policy 1, policy_version 1941214 (0.0009) [2023-12-27 05:30:25,935][105620] Updated weights for policy 1, policy_version 1941224 (0.0008) [2023-12-27 05:30:26,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19327.6). Total num frames: 992845824. Throughput: 0: 9431.4, 1: 9639.5. Samples: 992851476. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:30:26,062][104569] Avg episode reward: [(0, '8899.805'), (1, '9163.356')] [2023-12-27 05:30:26,457][105692] Updated weights for policy 0, policy_version 1936534 (0.0009) [2023-12-27 05:30:26,519][105692] Updated weights for policy 0, policy_version 1936544 (0.0008) [2023-12-27 05:30:26,581][105692] Updated weights for policy 0, policy_version 1936554 (0.0008) [2023-12-27 05:30:26,677][105620] Updated weights for policy 1, policy_version 1941234 (0.0006) [2023-12-27 05:30:26,741][105620] Updated weights for policy 1, policy_version 1941244 (0.0009) [2023-12-27 05:30:26,795][105620] Updated weights for policy 1, policy_version 1941254 (0.0008) [2023-12-27 05:30:26,857][105620] Updated weights for policy 1, policy_version 1941264 (0.0009) [2023-12-27 05:30:27,360][105692] Updated weights for policy 0, policy_version 1936564 (0.0009) [2023-12-27 05:30:27,421][105692] Updated weights for policy 0, policy_version 1936574 (0.0008) [2023-12-27 05:30:27,476][105692] Updated weights for policy 0, policy_version 1936584 (0.0008) [2023-12-27 05:30:27,545][105620] Updated weights for policy 1, policy_version 1941274 (0.0010) [2023-12-27 05:30:27,593][105620] Updated weights for policy 1, policy_version 1941284 (0.0010) [2023-12-27 05:30:27,640][105620] Updated weights for policy 1, policy_version 1941294 (0.0010) [2023-12-27 05:30:28,211][105692] Updated weights for policy 0, policy_version 1936594 (0.0006) [2023-12-27 05:30:28,262][105692] Updated weights for policy 0, policy_version 1936604 (0.0005) [2023-12-27 05:30:28,319][105692] Updated weights for policy 0, policy_version 1936614 (0.0006) [2023-12-27 05:30:28,385][105692] Updated weights for policy 0, policy_version 1936624 (0.0008) [2023-12-27 05:30:28,411][105620] Updated weights for policy 1, policy_version 1941304 (0.0011) [2023-12-27 05:30:28,473][105620] Updated weights for policy 1, policy_version 1941314 (0.0010) [2023-12-27 05:30:28,531][105620] Updated weights for policy 1, policy_version 1941324 (0.0010) [2023-12-27 05:30:29,047][105692] Updated weights for policy 0, policy_version 1936634 (0.0005) [2023-12-27 05:30:29,103][105692] Updated weights for policy 0, policy_version 1936644 (0.0005) [2023-12-27 05:30:29,166][105692] Updated weights for policy 0, policy_version 1936654 (0.0005) [2023-12-27 05:30:29,185][105620] Updated weights for policy 1, policy_version 1941334 (0.0010) [2023-12-27 05:30:29,257][105620] Updated weights for policy 1, policy_version 1941344 (0.0011) [2023-12-27 05:30:29,323][105620] Updated weights for policy 1, policy_version 1941354 (0.0011) [2023-12-27 05:30:29,821][105692] Updated weights for policy 0, policy_version 1936664 (0.0006) [2023-12-27 05:30:29,890][105692] Updated weights for policy 0, policy_version 1936674 (0.0007) [2023-12-27 05:30:29,956][105692] Updated weights for policy 0, policy_version 1936684 (0.0007) [2023-12-27 05:30:30,064][105620] Updated weights for policy 1, policy_version 1941364 (0.0009) [2023-12-27 05:30:30,122][105620] Updated weights for policy 1, policy_version 1941374 (0.0006) [2023-12-27 05:30:30,171][105620] Updated weights for policy 1, policy_version 1941384 (0.0005) [2023-12-27 05:30:30,576][105692] Updated weights for policy 0, policy_version 1936694 (0.0005) [2023-12-27 05:30:30,632][105692] Updated weights for policy 0, policy_version 1936704 (0.0009) [2023-12-27 05:30:30,689][105692] Updated weights for policy 0, policy_version 1936714 (0.0010) [2023-12-27 05:30:30,844][105620] Updated weights for policy 1, policy_version 1941394 (0.0007) [2023-12-27 05:30:30,892][105620] Updated weights for policy 1, policy_version 1941404 (0.0010) [2023-12-27 05:30:30,939][105620] Updated weights for policy 1, policy_version 1941414 (0.0010) [2023-12-27 05:30:30,987][105620] Updated weights for policy 1, policy_version 1941424 (0.0010) [2023-12-27 05:30:31,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19327.6). Total num frames: 992944128. Throughput: 0: 9400.9, 1: 9647.4. Samples: 992909004. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:30:31,062][104569] Avg episode reward: [(0, '8535.150'), (1, '8981.288')] [2023-12-27 05:30:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001936720_495869952.pth... [2023-12-27 05:30:31,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001941424_497074176.pth... [2023-12-27 05:30:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001940272_496779264.pth [2023-12-27 05:30:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001935632_495591424.pth [2023-12-27 05:30:31,384][105692] Updated weights for policy 0, policy_version 1936724 (0.0010) [2023-12-27 05:30:31,436][105692] Updated weights for policy 0, policy_version 1936734 (0.0010) [2023-12-27 05:30:31,493][105692] Updated weights for policy 0, policy_version 1936744 (0.0005) [2023-12-27 05:30:31,660][105620] Updated weights for policy 1, policy_version 1941434 (0.0008) [2023-12-27 05:30:31,719][105620] Updated weights for policy 1, policy_version 1941444 (0.0008) [2023-12-27 05:30:31,781][105620] Updated weights for policy 1, policy_version 1941454 (0.0008) [2023-12-27 05:30:32,137][105692] Updated weights for policy 0, policy_version 1936754 (0.0009) [2023-12-27 05:30:32,198][105692] Updated weights for policy 0, policy_version 1936764 (0.0009) [2023-12-27 05:30:32,245][105692] Updated weights for policy 0, policy_version 1936774 (0.0006) [2023-12-27 05:30:32,314][105692] Updated weights for policy 0, policy_version 1936784 (0.0006) [2023-12-27 05:30:32,450][105620] Updated weights for policy 1, policy_version 1941464 (0.0005) [2023-12-27 05:30:32,510][105620] Updated weights for policy 1, policy_version 1941474 (0.0005) [2023-12-27 05:30:32,569][105620] Updated weights for policy 1, policy_version 1941484 (0.0007) [2023-12-27 05:30:33,008][105692] Updated weights for policy 0, policy_version 1936794 (0.0009) [2023-12-27 05:30:33,071][105692] Updated weights for policy 0, policy_version 1936804 (0.0010) [2023-12-27 05:30:33,126][105692] Updated weights for policy 0, policy_version 1936816 (0.0011) [2023-12-27 05:30:33,236][105620] Updated weights for policy 1, policy_version 1941494 (0.0009) [2023-12-27 05:30:33,283][105620] Updated weights for policy 1, policy_version 1941504 (0.0008) [2023-12-27 05:30:33,337][105620] Updated weights for policy 1, policy_version 1941514 (0.0007) [2023-12-27 05:30:33,873][105692] Updated weights for policy 0, policy_version 1936826 (0.0005) [2023-12-27 05:30:33,926][105692] Updated weights for policy 0, policy_version 1936836 (0.0006) [2023-12-27 05:30:33,968][105620] Updated weights for policy 1, policy_version 1941524 (0.0009) [2023-12-27 05:30:33,984][105692] Updated weights for policy 0, policy_version 1936846 (0.0005) [2023-12-27 05:30:34,021][105620] Updated weights for policy 1, policy_version 1941534 (0.0009) [2023-12-27 05:30:34,071][105620] Updated weights for policy 1, policy_version 1941545 (0.0010) [2023-12-27 05:30:34,602][105692] Updated weights for policy 0, policy_version 1936856 (0.0006) [2023-12-27 05:30:34,666][105692] Updated weights for policy 0, policy_version 1936866 (0.0006) [2023-12-27 05:30:34,731][105692] Updated weights for policy 0, policy_version 1936876 (0.0009) [2023-12-27 05:30:34,788][105620] Updated weights for policy 1, policy_version 1941555 (0.0009) [2023-12-27 05:30:34,838][105620] Updated weights for policy 1, policy_version 1941565 (0.0008) [2023-12-27 05:30:34,897][105620] Updated weights for policy 1, policy_version 1941575 (0.0008) [2023-12-27 05:30:35,426][105692] Updated weights for policy 0, policy_version 1936886 (0.0010) [2023-12-27 05:30:35,497][105692] Updated weights for policy 0, policy_version 1936896 (0.0010) [2023-12-27 05:30:35,555][105692] Updated weights for policy 0, policy_version 1936906 (0.0010) [2023-12-27 05:30:35,623][105620] Updated weights for policy 1, policy_version 1941585 (0.0008) [2023-12-27 05:30:35,685][105620] Updated weights for policy 1, policy_version 1941595 (0.0010) [2023-12-27 05:30:35,739][105620] Updated weights for policy 1, policy_version 1941605 (0.0010) [2023-12-27 05:30:35,795][105620] Updated weights for policy 1, policy_version 1941615 (0.0011) [2023-12-27 05:30:36,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19327.6). Total num frames: 993042432. Throughput: 0: 9515.8, 1: 9706.4. Samples: 993031840. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:30:36,062][104569] Avg episode reward: [(0, '8084.303'), (1, '9073.489')] [2023-12-27 05:30:36,263][105692] Updated weights for policy 0, policy_version 1936916 (0.0009) [2023-12-27 05:30:36,326][105692] Updated weights for policy 0, policy_version 1936926 (0.0011) [2023-12-27 05:30:36,389][105692] Updated weights for policy 0, policy_version 1936936 (0.0010) [2023-12-27 05:30:36,490][105620] Updated weights for policy 1, policy_version 1941625 (0.0011) [2023-12-27 05:30:36,554][105620] Updated weights for policy 1, policy_version 1941635 (0.0011) [2023-12-27 05:30:36,623][105620] Updated weights for policy 1, policy_version 1941645 (0.0010) [2023-12-27 05:30:37,010][105692] Updated weights for policy 0, policy_version 1936946 (0.0010) [2023-12-27 05:30:37,075][105692] Updated weights for policy 0, policy_version 1936956 (0.0005) [2023-12-27 05:30:37,140][105692] Updated weights for policy 0, policy_version 1936966 (0.0006) [2023-12-27 05:30:37,192][105692] Updated weights for policy 0, policy_version 1936976 (0.0008) [2023-12-27 05:30:37,312][105620] Updated weights for policy 1, policy_version 1941655 (0.0008) [2023-12-27 05:30:37,367][105620] Updated weights for policy 1, policy_version 1941665 (0.0005) [2023-12-27 05:30:37,431][105620] Updated weights for policy 1, policy_version 1941675 (0.0008) [2023-12-27 05:30:37,924][105692] Updated weights for policy 0, policy_version 1936986 (0.0010) [2023-12-27 05:30:37,987][105692] Updated weights for policy 0, policy_version 1936996 (0.0011) [2023-12-27 05:30:38,053][105692] Updated weights for policy 0, policy_version 1937006 (0.0011) [2023-12-27 05:30:38,167][105620] Updated weights for policy 1, policy_version 1941685 (0.0008) [2023-12-27 05:30:38,229][105620] Updated weights for policy 1, policy_version 1941695 (0.0006) [2023-12-27 05:30:38,287][105620] Updated weights for policy 1, policy_version 1941705 (0.0006) [2023-12-27 05:30:38,786][105692] Updated weights for policy 0, policy_version 1937016 (0.0009) [2023-12-27 05:30:38,859][105692] Updated weights for policy 0, policy_version 1937026 (0.0008) [2023-12-27 05:30:38,910][105692] Updated weights for policy 0, policy_version 1937036 (0.0010) [2023-12-27 05:30:38,957][105620] Updated weights for policy 1, policy_version 1941715 (0.0006) [2023-12-27 05:30:39,027][105620] Updated weights for policy 1, policy_version 1941725 (0.0007) [2023-12-27 05:30:39,080][105620] Updated weights for policy 1, policy_version 1941735 (0.0010) [2023-12-27 05:30:39,660][105692] Updated weights for policy 0, policy_version 1937046 (0.0009) [2023-12-27 05:30:39,713][105692] Updated weights for policy 0, policy_version 1937056 (0.0007) [2023-12-27 05:30:39,780][105692] Updated weights for policy 0, policy_version 1937066 (0.0006) [2023-12-27 05:30:39,816][105620] Updated weights for policy 1, policy_version 1941745 (0.0010) [2023-12-27 05:30:39,883][105620] Updated weights for policy 1, policy_version 1941755 (0.0008) [2023-12-27 05:30:39,951][105620] Updated weights for policy 1, policy_version 1941765 (0.0010) [2023-12-27 05:30:40,019][105620] Updated weights for policy 1, policy_version 1941775 (0.0011) [2023-12-27 05:30:40,502][105692] Updated weights for policy 0, policy_version 1937076 (0.0008) [2023-12-27 05:30:40,568][105692] Updated weights for policy 0, policy_version 1937086 (0.0008) [2023-12-27 05:30:40,628][105692] Updated weights for policy 0, policy_version 1937096 (0.0011) [2023-12-27 05:30:40,718][105620] Updated weights for policy 1, policy_version 1941785 (0.0010) [2023-12-27 05:30:40,771][105620] Updated weights for policy 1, policy_version 1941795 (0.0010) [2023-12-27 05:30:40,816][105620] Updated weights for policy 1, policy_version 1941805 (0.0010) [2023-12-27 05:30:41,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 993140736. Throughput: 0: 9516.5, 1: 9784.3. Samples: 993148144. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:30:41,062][104569] Avg episode reward: [(0, '8178.996'), (1, '9345.988')] [2023-12-27 05:30:41,306][105692] Updated weights for policy 0, policy_version 1937106 (0.0011) [2023-12-27 05:30:41,374][105692] Updated weights for policy 0, policy_version 1937116 (0.0011) [2023-12-27 05:30:41,438][105692] Updated weights for policy 0, policy_version 1937126 (0.0011) [2023-12-27 05:30:41,484][105692] Updated weights for policy 0, policy_version 1937136 (0.0010) [2023-12-27 05:30:41,594][105620] Updated weights for policy 1, policy_version 1941815 (0.0008) [2023-12-27 05:30:41,663][105620] Updated weights for policy 1, policy_version 1941825 (0.0008) [2023-12-27 05:30:41,724][105620] Updated weights for policy 1, policy_version 1941835 (0.0008) [2023-12-27 05:30:42,337][105692] Updated weights for policy 0, policy_version 1937146 (0.0009) [2023-12-27 05:30:42,340][105620] Updated weights for policy 1, policy_version 1941845 (0.0006) [2023-12-27 05:30:42,396][105692] Updated weights for policy 0, policy_version 1937156 (0.0007) [2023-12-27 05:30:42,403][105620] Updated weights for policy 1, policy_version 1941855 (0.0009) [2023-12-27 05:30:42,456][105692] Updated weights for policy 0, policy_version 1937166 (0.0008) [2023-12-27 05:30:42,465][105620] Updated weights for policy 1, policy_version 1941865 (0.0009) [2023-12-27 05:30:43,085][105620] Updated weights for policy 1, policy_version 1941875 (0.0008) [2023-12-27 05:30:43,140][105620] Updated weights for policy 1, policy_version 1941885 (0.0005) [2023-12-27 05:30:43,186][105620] Updated weights for policy 1, policy_version 1941895 (0.0007) [2023-12-27 05:30:43,191][105692] Updated weights for policy 0, policy_version 1937176 (0.0005) [2023-12-27 05:30:43,236][105692] Updated weights for policy 0, policy_version 1937186 (0.0005) [2023-12-27 05:30:43,279][105692] Updated weights for policy 0, policy_version 1937196 (0.0005) [2023-12-27 05:30:43,842][105620] Updated weights for policy 1, policy_version 1941905 (0.0008) [2023-12-27 05:30:43,898][105620] Updated weights for policy 1, policy_version 1941915 (0.0005) [2023-12-27 05:30:43,957][105620] Updated weights for policy 1, policy_version 1941925 (0.0007) [2023-12-27 05:30:44,004][105620] Updated weights for policy 1, policy_version 1941935 (0.0009) [2023-12-27 05:30:44,056][105692] Updated weights for policy 0, policy_version 1937206 (0.0007) [2023-12-27 05:30:44,107][105692] Updated weights for policy 0, policy_version 1937216 (0.0009) [2023-12-27 05:30:44,168][105692] Updated weights for policy 0, policy_version 1937226 (0.0009) [2023-12-27 05:30:44,681][105620] Updated weights for policy 1, policy_version 1941945 (0.0008) [2023-12-27 05:30:44,736][105620] Updated weights for policy 1, policy_version 1941955 (0.0008) [2023-12-27 05:30:44,802][105620] Updated weights for policy 1, policy_version 1941965 (0.0008) [2023-12-27 05:30:44,960][105692] Updated weights for policy 0, policy_version 1937236 (0.0008) [2023-12-27 05:30:45,017][105692] Updated weights for policy 0, policy_version 1937246 (0.0010) [2023-12-27 05:30:45,066][105692] Updated weights for policy 0, policy_version 1937256 (0.0011) [2023-12-27 05:30:45,547][105620] Updated weights for policy 1, policy_version 1941975 (0.0007) [2023-12-27 05:30:45,603][105620] Updated weights for policy 1, policy_version 1941985 (0.0009) [2023-12-27 05:30:45,657][105620] Updated weights for policy 1, policy_version 1941995 (0.0009) [2023-12-27 05:30:45,828][105692] Updated weights for policy 0, policy_version 1937266 (0.0007) [2023-12-27 05:30:45,886][105692] Updated weights for policy 0, policy_version 1937276 (0.0010) [2023-12-27 05:30:45,940][105692] Updated weights for policy 0, policy_version 1937286 (0.0010) [2023-12-27 05:30:45,994][105692] Updated weights for policy 0, policy_version 1937296 (0.0010) [2023-12-27 05:30:46,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19251.3, 300 sec: 19383.1). Total num frames: 993239040. Throughput: 0: 9503.8, 1: 9832.6. Samples: 993207676. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:30:46,063][104569] Avg episode reward: [(0, '8631.049'), (1, '9253.970')] [2023-12-27 05:30:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001942000_497221632.pth... [2023-12-27 05:30:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001937296_496017408.pth... [2023-12-27 05:30:46,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001940848_496926720.pth [2023-12-27 05:30:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001936176_495730688.pth [2023-12-27 05:30:46,254][105620] Updated weights for policy 1, policy_version 1942005 (0.0009) [2023-12-27 05:30:46,306][105620] Updated weights for policy 1, policy_version 1942015 (0.0009) [2023-12-27 05:30:46,359][105620] Updated weights for policy 1, policy_version 1942025 (0.0008) [2023-12-27 05:30:46,796][105692] Updated weights for policy 0, policy_version 1937306 (0.0007) [2023-12-27 05:30:46,846][105692] Updated weights for policy 0, policy_version 1937316 (0.0009) [2023-12-27 05:30:46,895][105692] Updated weights for policy 0, policy_version 1937326 (0.0008) [2023-12-27 05:30:47,129][105620] Updated weights for policy 1, policy_version 1942035 (0.0009) [2023-12-27 05:30:47,190][105620] Updated weights for policy 1, policy_version 1942045 (0.0009) [2023-12-27 05:30:47,244][105620] Updated weights for policy 1, policy_version 1942055 (0.0008) [2023-12-27 05:30:47,682][105692] Updated weights for policy 0, policy_version 1937336 (0.0010) [2023-12-27 05:30:47,734][105692] Updated weights for policy 0, policy_version 1937346 (0.0010) [2023-12-27 05:30:47,786][105692] Updated weights for policy 0, policy_version 1937356 (0.0010) [2023-12-27 05:30:47,872][105620] Updated weights for policy 1, policy_version 1942065 (0.0008) [2023-12-27 05:30:47,918][105620] Updated weights for policy 1, policy_version 1942075 (0.0005) [2023-12-27 05:30:47,966][105620] Updated weights for policy 1, policy_version 1942085 (0.0005) [2023-12-27 05:30:48,017][105620] Updated weights for policy 1, policy_version 1942095 (0.0005) [2023-12-27 05:30:48,498][105692] Updated weights for policy 0, policy_version 1937366 (0.0011) [2023-12-27 05:30:48,554][105692] Updated weights for policy 0, policy_version 1937376 (0.0011) [2023-12-27 05:30:48,616][105692] Updated weights for policy 0, policy_version 1937386 (0.0010) [2023-12-27 05:30:48,666][105620] Updated weights for policy 1, policy_version 1942105 (0.0010) [2023-12-27 05:30:48,721][105620] Updated weights for policy 1, policy_version 1942115 (0.0010) [2023-12-27 05:30:48,781][105620] Updated weights for policy 1, policy_version 1942125 (0.0010) [2023-12-27 05:30:49,229][105692] Updated weights for policy 0, policy_version 1937396 (0.0010) [2023-12-27 05:30:49,289][105692] Updated weights for policy 0, policy_version 1937406 (0.0009) [2023-12-27 05:30:49,350][105692] Updated weights for policy 0, policy_version 1937416 (0.0010) [2023-12-27 05:30:49,629][105620] Updated weights for policy 1, policy_version 1942135 (0.0009) [2023-12-27 05:30:49,677][105620] Updated weights for policy 1, policy_version 1942145 (0.0009) [2023-12-27 05:30:49,728][105620] Updated weights for policy 1, policy_version 1942155 (0.0009) [2023-12-27 05:30:50,078][105692] Updated weights for policy 0, policy_version 1937426 (0.0009) [2023-12-27 05:30:50,140][105692] Updated weights for policy 0, policy_version 1937436 (0.0009) [2023-12-27 05:30:50,204][105692] Updated weights for policy 0, policy_version 1937446 (0.0009) [2023-12-27 05:30:50,266][105692] Updated weights for policy 0, policy_version 1937456 (0.0009) [2023-12-27 05:30:50,543][105620] Updated weights for policy 1, policy_version 1942165 (0.0008) [2023-12-27 05:30:50,610][105620] Updated weights for policy 1, policy_version 1942175 (0.0009) [2023-12-27 05:30:50,672][105620] Updated weights for policy 1, policy_version 1942185 (0.0010) [2023-12-27 05:30:50,912][105692] Updated weights for policy 0, policy_version 1937466 (0.0007) [2023-12-27 05:30:50,981][105692] Updated weights for policy 0, policy_version 1937476 (0.0009) [2023-12-27 05:30:51,046][105692] Updated weights for policy 0, policy_version 1937486 (0.0009) [2023-12-27 05:30:51,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 993337344. Throughput: 0: 9419.2, 1: 9863.1. Samples: 993323044. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:30:51,063][104569] Avg episode reward: [(0, '8629.835'), (1, '9253.778')] [2023-12-27 05:30:51,498][105620] Updated weights for policy 1, policy_version 1942195 (0.0009) [2023-12-27 05:30:51,570][105620] Updated weights for policy 1, policy_version 1942205 (0.0009) [2023-12-27 05:30:51,632][105620] Updated weights for policy 1, policy_version 1942215 (0.0009) [2023-12-27 05:30:51,763][105692] Updated weights for policy 0, policy_version 1937496 (0.0009) [2023-12-27 05:30:51,819][105692] Updated weights for policy 0, policy_version 1937506 (0.0008) [2023-12-27 05:30:51,876][105692] Updated weights for policy 0, policy_version 1937516 (0.0008) [2023-12-27 05:30:52,399][105620] Updated weights for policy 1, policy_version 1942225 (0.0009) [2023-12-27 05:30:52,448][105620] Updated weights for policy 1, policy_version 1942235 (0.0008) [2023-12-27 05:30:52,515][105620] Updated weights for policy 1, policy_version 1942245 (0.0008) [2023-12-27 05:30:52,562][105620] Updated weights for policy 1, policy_version 1942255 (0.0007) [2023-12-27 05:30:52,653][105692] Updated weights for policy 0, policy_version 1937526 (0.0010) [2023-12-27 05:30:52,713][105692] Updated weights for policy 0, policy_version 1937536 (0.0010) [2023-12-27 05:30:52,771][105692] Updated weights for policy 0, policy_version 1937546 (0.0011) [2023-12-27 05:30:53,265][105620] Updated weights for policy 1, policy_version 1942265 (0.0008) [2023-12-27 05:30:53,324][105620] Updated weights for policy 1, policy_version 1942275 (0.0008) [2023-12-27 05:30:53,385][105620] Updated weights for policy 1, policy_version 1942285 (0.0009) [2023-12-27 05:30:53,531][105692] Updated weights for policy 0, policy_version 1937556 (0.0010) [2023-12-27 05:30:53,594][105692] Updated weights for policy 0, policy_version 1937566 (0.0009) [2023-12-27 05:30:53,656][105692] Updated weights for policy 0, policy_version 1937576 (0.0009) [2023-12-27 05:30:54,115][105620] Updated weights for policy 1, policy_version 1942295 (0.0010) [2023-12-27 05:30:54,178][105620] Updated weights for policy 1, policy_version 1942305 (0.0009) [2023-12-27 05:30:54,230][105620] Updated weights for policy 1, policy_version 1942315 (0.0009) [2023-12-27 05:30:54,414][105692] Updated weights for policy 0, policy_version 1937586 (0.0008) [2023-12-27 05:30:54,464][105692] Updated weights for policy 0, policy_version 1937596 (0.0008) [2023-12-27 05:30:54,511][105692] Updated weights for policy 0, policy_version 1937606 (0.0009) [2023-12-27 05:30:54,569][105692] Updated weights for policy 0, policy_version 1937616 (0.0008) [2023-12-27 05:30:55,001][105620] Updated weights for policy 1, policy_version 1942325 (0.0009) [2023-12-27 05:30:55,056][105620] Updated weights for policy 1, policy_version 1942335 (0.0009) [2023-12-27 05:30:55,120][105620] Updated weights for policy 1, policy_version 1942345 (0.0009) [2023-12-27 05:30:55,300][105692] Updated weights for policy 0, policy_version 1937626 (0.0009) [2023-12-27 05:30:55,355][105692] Updated weights for policy 0, policy_version 1937636 (0.0009) [2023-12-27 05:30:55,412][105692] Updated weights for policy 0, policy_version 1937646 (0.0009) [2023-12-27 05:30:55,806][105620] Updated weights for policy 1, policy_version 1942355 (0.0009) [2023-12-27 05:30:55,876][105620] Updated weights for policy 1, policy_version 1942365 (0.0006) [2023-12-27 05:30:55,935][105620] Updated weights for policy 1, policy_version 1942375 (0.0006) [2023-12-27 05:30:56,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19355.3). Total num frames: 993427456. Throughput: 0: 9481.4, 1: 9828.4. Samples: 993435428. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:30:56,063][104569] Avg episode reward: [(0, '8993.323'), (1, '9345.812')] [2023-12-27 05:30:56,292][105692] Updated weights for policy 0, policy_version 1937656 (0.0009) [2023-12-27 05:30:56,356][105692] Updated weights for policy 0, policy_version 1937666 (0.0009) [2023-12-27 05:30:56,407][105692] Updated weights for policy 0, policy_version 1937676 (0.0009) [2023-12-27 05:30:56,535][105620] Updated weights for policy 1, policy_version 1942385 (0.0006) [2023-12-27 05:30:56,601][105620] Updated weights for policy 1, policy_version 1942395 (0.0008) [2023-12-27 05:30:56,652][105620] Updated weights for policy 1, policy_version 1942405 (0.0008) [2023-12-27 05:30:56,709][105620] Updated weights for policy 1, policy_version 1942415 (0.0008) [2023-12-27 05:30:57,180][105692] Updated weights for policy 0, policy_version 1937686 (0.0009) [2023-12-27 05:30:57,238][105692] Updated weights for policy 0, policy_version 1937696 (0.0008) [2023-12-27 05:30:57,289][105692] Updated weights for policy 0, policy_version 1937706 (0.0007) [2023-12-27 05:30:57,420][105620] Updated weights for policy 1, policy_version 1942425 (0.0006) [2023-12-27 05:30:57,471][105620] Updated weights for policy 1, policy_version 1942435 (0.0008) [2023-12-27 05:30:57,519][105620] Updated weights for policy 1, policy_version 1942445 (0.0010) [2023-12-27 05:30:58,064][105692] Updated weights for policy 0, policy_version 1937716 (0.0009) [2023-12-27 05:30:58,116][105692] Updated weights for policy 0, policy_version 1937726 (0.0010) [2023-12-27 05:30:58,171][105692] Updated weights for policy 0, policy_version 1937736 (0.0009) [2023-12-27 05:30:58,218][105620] Updated weights for policy 1, policy_version 1942455 (0.0011) [2023-12-27 05:30:58,279][105620] Updated weights for policy 1, policy_version 1942465 (0.0010) [2023-12-27 05:30:58,341][105620] Updated weights for policy 1, policy_version 1942475 (0.0009) [2023-12-27 05:30:59,043][105692] Updated weights for policy 0, policy_version 1937746 (0.0007) [2023-12-27 05:30:59,101][105692] Updated weights for policy 0, policy_version 1937756 (0.0005) [2023-12-27 05:30:59,160][105692] Updated weights for policy 0, policy_version 1937766 (0.0009) [2023-12-27 05:30:59,165][105620] Updated weights for policy 1, policy_version 1942485 (0.0008) [2023-12-27 05:30:59,220][105692] Updated weights for policy 0, policy_version 1937776 (0.0008) [2023-12-27 05:30:59,229][105620] Updated weights for policy 1, policy_version 1942495 (0.0008) [2023-12-27 05:30:59,294][105620] Updated weights for policy 1, policy_version 1942505 (0.0008) [2023-12-27 05:30:59,834][105692] Updated weights for policy 0, policy_version 1937786 (0.0009) [2023-12-27 05:30:59,904][105692] Updated weights for policy 0, policy_version 1937796 (0.0008) [2023-12-27 05:30:59,967][105692] Updated weights for policy 0, policy_version 1937806 (0.0009) [2023-12-27 05:31:00,104][105620] Updated weights for policy 1, policy_version 1942515 (0.0009) [2023-12-27 05:31:00,155][105620] Updated weights for policy 1, policy_version 1942525 (0.0007) [2023-12-27 05:31:00,203][105620] Updated weights for policy 1, policy_version 1942535 (0.0009) [2023-12-27 05:31:00,758][105692] Updated weights for policy 0, policy_version 1937816 (0.0008) [2023-12-27 05:31:00,813][105692] Updated weights for policy 0, policy_version 1937826 (0.0008) [2023-12-27 05:31:00,867][105692] Updated weights for policy 0, policy_version 1937836 (0.0007) [2023-12-27 05:31:00,885][105620] Updated weights for policy 1, policy_version 1942545 (0.0009) [2023-12-27 05:31:00,947][105620] Updated weights for policy 1, policy_version 1942555 (0.0010) [2023-12-27 05:31:01,004][105620] Updated weights for policy 1, policy_version 1942565 (0.0010) [2023-12-27 05:31:01,062][104569] Fps is (10 sec: 18022.3, 60 sec: 19114.6, 300 sec: 19327.6). Total num frames: 993517568. Throughput: 0: 9455.4, 1: 9829.4. Samples: 993491040. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:31:01,063][104569] Avg episode reward: [(0, '8444.894'), (1, '9253.586')] [2023-12-27 05:31:01,063][105620] Updated weights for policy 1, policy_version 1942575 (0.0011) [2023-12-27 05:31:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001937840_496156672.pth... [2023-12-27 05:31:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001942576_497369088.pth... [2023-12-27 05:31:01,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001936720_495869952.pth [2023-12-27 05:31:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001941424_497074176.pth [2023-12-27 05:31:01,599][105692] Updated weights for policy 0, policy_version 1937846 (0.0008) [2023-12-27 05:31:01,663][105692] Updated weights for policy 0, policy_version 1937856 (0.0009) [2023-12-27 05:31:01,722][105692] Updated weights for policy 0, policy_version 1937866 (0.0007) [2023-12-27 05:31:01,754][105620] Updated weights for policy 1, policy_version 1942585 (0.0011) [2023-12-27 05:31:01,809][105620] Updated weights for policy 1, policy_version 1942595 (0.0010) [2023-12-27 05:31:01,860][105620] Updated weights for policy 1, policy_version 1942605 (0.0010) [2023-12-27 05:31:02,418][105692] Updated weights for policy 0, policy_version 1937876 (0.0009) [2023-12-27 05:31:02,463][105692] Updated weights for policy 0, policy_version 1937886 (0.0006) [2023-12-27 05:31:02,505][105620] Updated weights for policy 1, policy_version 1942615 (0.0007) [2023-12-27 05:31:02,521][105692] Updated weights for policy 0, policy_version 1937896 (0.0008) [2023-12-27 05:31:02,558][105620] Updated weights for policy 1, policy_version 1942625 (0.0008) [2023-12-27 05:31:02,609][105620] Updated weights for policy 1, policy_version 1942635 (0.0010) [2023-12-27 05:31:03,277][105620] Updated weights for policy 1, policy_version 1942645 (0.0009) [2023-12-27 05:31:03,295][105692] Updated weights for policy 0, policy_version 1937906 (0.0007) [2023-12-27 05:31:03,325][105620] Updated weights for policy 1, policy_version 1942655 (0.0010) [2023-12-27 05:31:03,344][105692] Updated weights for policy 0, policy_version 1937916 (0.0006) [2023-12-27 05:31:03,383][105620] Updated weights for policy 1, policy_version 1942665 (0.0010) [2023-12-27 05:31:03,399][105692] Updated weights for policy 0, policy_version 1937926 (0.0010) [2023-12-27 05:31:03,460][105692] Updated weights for policy 0, policy_version 1937936 (0.0007) [2023-12-27 05:31:04,095][105692] Updated weights for policy 0, policy_version 1937946 (0.0008) [2023-12-27 05:31:04,139][105620] Updated weights for policy 1, policy_version 1942675 (0.0010) [2023-12-27 05:31:04,154][105692] Updated weights for policy 0, policy_version 1937956 (0.0007) [2023-12-27 05:31:04,202][105620] Updated weights for policy 1, policy_version 1942685 (0.0011) [2023-12-27 05:31:04,215][105692] Updated weights for policy 0, policy_version 1937966 (0.0009) [2023-12-27 05:31:04,251][105620] Updated weights for policy 1, policy_version 1942695 (0.0011) [2023-12-27 05:31:04,981][105692] Updated weights for policy 0, policy_version 1937976 (0.0010) [2023-12-27 05:31:05,031][105620] Updated weights for policy 1, policy_version 1942705 (0.0011) [2023-12-27 05:31:05,034][105692] Updated weights for policy 0, policy_version 1937986 (0.0011) [2023-12-27 05:31:05,090][105620] Updated weights for policy 1, policy_version 1942715 (0.0009) [2023-12-27 05:31:05,096][105692] Updated weights for policy 0, policy_version 1937996 (0.0010) [2023-12-27 05:31:05,136][105620] Updated weights for policy 1, policy_version 1942725 (0.0007) [2023-12-27 05:31:05,197][105620] Updated weights for policy 1, policy_version 1942735 (0.0010) [2023-12-27 05:31:05,799][105692] Updated weights for policy 0, policy_version 1938006 (0.0007) [2023-12-27 05:31:05,855][105692] Updated weights for policy 0, policy_version 1938016 (0.0010) [2023-12-27 05:31:05,876][105620] Updated weights for policy 1, policy_version 1942745 (0.0007) [2023-12-27 05:31:05,916][105692] Updated weights for policy 0, policy_version 1938026 (0.0007) [2023-12-27 05:31:05,922][105620] Updated weights for policy 1, policy_version 1942755 (0.0007) [2023-12-27 05:31:05,977][105620] Updated weights for policy 1, policy_version 1942765 (0.0009) [2023-12-27 05:31:06,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 993624064. Throughput: 0: 9527.2, 1: 9742.9. Samples: 993607072. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:31:06,063][104569] Avg episode reward: [(0, '7900.418'), (1, '9253.749')] [2023-12-27 05:31:06,655][105620] Updated weights for policy 1, policy_version 1942775 (0.0009) [2023-12-27 05:31:06,707][105620] Updated weights for policy 1, policy_version 1942785 (0.0008) [2023-12-27 05:31:06,711][105692] Updated weights for policy 0, policy_version 1938036 (0.0009) [2023-12-27 05:31:06,761][105620] Updated weights for policy 1, policy_version 1942795 (0.0008) [2023-12-27 05:31:06,771][105692] Updated weights for policy 0, policy_version 1938046 (0.0011) [2023-12-27 05:31:06,834][105692] Updated weights for policy 0, policy_version 1938056 (0.0011) [2023-12-27 05:31:07,492][105620] Updated weights for policy 1, policy_version 1942805 (0.0007) [2023-12-27 05:31:07,553][105620] Updated weights for policy 1, policy_version 1942815 (0.0008) [2023-12-27 05:31:07,561][105692] Updated weights for policy 0, policy_version 1938066 (0.0011) [2023-12-27 05:31:07,604][105620] Updated weights for policy 1, policy_version 1942825 (0.0008) [2023-12-27 05:31:07,613][105692] Updated weights for policy 0, policy_version 1938076 (0.0010) [2023-12-27 05:31:07,665][105692] Updated weights for policy 0, policy_version 1938086 (0.0010) [2023-12-27 05:31:07,707][105692] Updated weights for policy 0, policy_version 1938096 (0.0008) [2023-12-27 05:31:08,203][105620] Updated weights for policy 1, policy_version 1942835 (0.0007) [2023-12-27 05:31:08,264][105620] Updated weights for policy 1, policy_version 1942845 (0.0005) [2023-12-27 05:31:08,334][105620] Updated weights for policy 1, policy_version 1942855 (0.0006) [2023-12-27 05:31:08,438][105692] Updated weights for policy 0, policy_version 1938106 (0.0009) [2023-12-27 05:31:08,497][105692] Updated weights for policy 0, policy_version 1938116 (0.0008) [2023-12-27 05:31:08,552][105692] Updated weights for policy 0, policy_version 1938126 (0.0008) [2023-12-27 05:31:09,036][105620] Updated weights for policy 1, policy_version 1942865 (0.0011) [2023-12-27 05:31:09,097][105620] Updated weights for policy 1, policy_version 1942875 (0.0010) [2023-12-27 05:31:09,162][105620] Updated weights for policy 1, policy_version 1942885 (0.0010) [2023-12-27 05:31:09,231][105620] Updated weights for policy 1, policy_version 1942895 (0.0010) [2023-12-27 05:31:09,296][105692] Updated weights for policy 0, policy_version 1938136 (0.0008) [2023-12-27 05:31:09,360][105692] Updated weights for policy 0, policy_version 1938146 (0.0008) [2023-12-27 05:31:09,436][105692] Updated weights for policy 0, policy_version 1938156 (0.0007) [2023-12-27 05:31:09,863][105620] Updated weights for policy 1, policy_version 1942905 (0.0009) [2023-12-27 05:31:09,928][105620] Updated weights for policy 1, policy_version 1942915 (0.0008) [2023-12-27 05:31:09,996][105620] Updated weights for policy 1, policy_version 1942925 (0.0008) [2023-12-27 05:31:10,119][105692] Updated weights for policy 0, policy_version 1938166 (0.0008) [2023-12-27 05:31:10,181][105692] Updated weights for policy 0, policy_version 1938176 (0.0009) [2023-12-27 05:31:10,248][105692] Updated weights for policy 0, policy_version 1938186 (0.0009) [2023-12-27 05:31:10,695][105620] Updated weights for policy 1, policy_version 1942935 (0.0006) [2023-12-27 05:31:10,760][105620] Updated weights for policy 1, policy_version 1942945 (0.0006) [2023-12-27 05:31:10,825][105620] Updated weights for policy 1, policy_version 1942955 (0.0005) [2023-12-27 05:31:11,028][105692] Updated weights for policy 0, policy_version 1938196 (0.0010) [2023-12-27 05:31:11,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19327.6). Total num frames: 993714176. Throughput: 0: 9551.2, 1: 9872.2. Samples: 993725528. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:31:11,063][104569] Avg episode reward: [(0, '8353.760'), (1, '9346.173')] [2023-12-27 05:31:11,092][105692] Updated weights for policy 0, policy_version 1938206 (0.0009) [2023-12-27 05:31:11,154][105692] Updated weights for policy 0, policy_version 1938216 (0.0010) [2023-12-27 05:31:11,493][105620] Updated weights for policy 1, policy_version 1942965 (0.0005) [2023-12-27 05:31:11,556][105620] Updated weights for policy 1, policy_version 1942975 (0.0009) [2023-12-27 05:31:11,607][105620] Updated weights for policy 1, policy_version 1942985 (0.0008) [2023-12-27 05:31:12,018][105692] Updated weights for policy 0, policy_version 1938226 (0.0010) [2023-12-27 05:31:12,075][105692] Updated weights for policy 0, policy_version 1938236 (0.0009) [2023-12-27 05:31:12,135][105692] Updated weights for policy 0, policy_version 1938246 (0.0009) [2023-12-27 05:31:12,200][105692] Updated weights for policy 0, policy_version 1938256 (0.0009) [2023-12-27 05:31:12,299][105620] Updated weights for policy 1, policy_version 1942995 (0.0007) [2023-12-27 05:31:12,357][105620] Updated weights for policy 1, policy_version 1943005 (0.0009) [2023-12-27 05:31:12,423][105620] Updated weights for policy 1, policy_version 1943015 (0.0009) [2023-12-27 05:31:12,966][105692] Updated weights for policy 0, policy_version 1938266 (0.0008) [2023-12-27 05:31:13,016][105692] Updated weights for policy 0, policy_version 1938276 (0.0008) [2023-12-27 05:31:13,065][105692] Updated weights for policy 0, policy_version 1938286 (0.0010) [2023-12-27 05:31:13,196][105620] Updated weights for policy 1, policy_version 1943025 (0.0009) [2023-12-27 05:31:13,248][105620] Updated weights for policy 1, policy_version 1943035 (0.0010) [2023-12-27 05:31:13,292][105620] Updated weights for policy 1, policy_version 1943045 (0.0010) [2023-12-27 05:31:13,349][105620] Updated weights for policy 1, policy_version 1943055 (0.0010) [2023-12-27 05:31:13,802][105692] Updated weights for policy 0, policy_version 1938296 (0.0010) [2023-12-27 05:31:13,853][105692] Updated weights for policy 0, policy_version 1938306 (0.0009) [2023-12-27 05:31:13,913][105692] Updated weights for policy 0, policy_version 1938316 (0.0006) [2023-12-27 05:31:14,102][105620] Updated weights for policy 1, policy_version 1943065 (0.0008) [2023-12-27 05:31:14,168][105620] Updated weights for policy 1, policy_version 1943075 (0.0007) [2023-12-27 05:31:14,230][105620] Updated weights for policy 1, policy_version 1943085 (0.0008) [2023-12-27 05:31:14,512][105692] Updated weights for policy 0, policy_version 1938326 (0.0006) [2023-12-27 05:31:14,569][105692] Updated weights for policy 0, policy_version 1938336 (0.0008) [2023-12-27 05:31:14,633][105692] Updated weights for policy 0, policy_version 1938346 (0.0007) [2023-12-27 05:31:14,956][105620] Updated weights for policy 1, policy_version 1943095 (0.0009) [2023-12-27 05:31:15,007][105620] Updated weights for policy 1, policy_version 1943105 (0.0009) [2023-12-27 05:31:15,073][105620] Updated weights for policy 1, policy_version 1943115 (0.0010) [2023-12-27 05:31:15,362][105692] Updated weights for policy 0, policy_version 1938356 (0.0009) [2023-12-27 05:31:15,429][105692] Updated weights for policy 0, policy_version 1938366 (0.0010) [2023-12-27 05:31:15,487][105692] Updated weights for policy 0, policy_version 1938376 (0.0010) [2023-12-27 05:31:15,714][105620] Updated weights for policy 1, policy_version 1943125 (0.0009) [2023-12-27 05:31:15,760][105620] Updated weights for policy 1, policy_version 1943135 (0.0009) [2023-12-27 05:31:15,811][105620] Updated weights for policy 1, policy_version 1943145 (0.0006) [2023-12-27 05:31:16,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19355.3). Total num frames: 993812480. Throughput: 0: 9526.7, 1: 9853.5. Samples: 993781112. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:31:16,062][104569] Avg episode reward: [(0, '8352.696'), (1, '9253.774')] [2023-12-27 05:31:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001938384_496295936.pth... [2023-12-27 05:31:16,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001943152_497516544.pth... [2023-12-27 05:31:16,075][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001937296_496017408.pth [2023-12-27 05:31:16,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001942000_497221632.pth [2023-12-27 05:31:16,232][105692] Updated weights for policy 0, policy_version 1938386 (0.0010) [2023-12-27 05:31:16,294][105692] Updated weights for policy 0, policy_version 1938396 (0.0010) [2023-12-27 05:31:16,346][105692] Updated weights for policy 0, policy_version 1938406 (0.0009) [2023-12-27 05:31:16,405][105692] Updated weights for policy 0, policy_version 1938416 (0.0009) [2023-12-27 05:31:16,452][105620] Updated weights for policy 1, policy_version 1943155 (0.0007) [2023-12-27 05:31:16,511][105620] Updated weights for policy 1, policy_version 1943165 (0.0010) [2023-12-27 05:31:16,576][105620] Updated weights for policy 1, policy_version 1943175 (0.0010) [2023-12-27 05:31:17,197][105692] Updated weights for policy 0, policy_version 1938426 (0.0007) [2023-12-27 05:31:17,252][105692] Updated weights for policy 0, policy_version 1938436 (0.0008) [2023-12-27 05:31:17,271][105620] Updated weights for policy 1, policy_version 1943185 (0.0010) [2023-12-27 05:31:17,305][105692] Updated weights for policy 0, policy_version 1938446 (0.0006) [2023-12-27 05:31:17,333][105620] Updated weights for policy 1, policy_version 1943195 (0.0010) [2023-12-27 05:31:17,391][105620] Updated weights for policy 1, policy_version 1943205 (0.0010) [2023-12-27 05:31:17,459][105620] Updated weights for policy 1, policy_version 1943215 (0.0010) [2023-12-27 05:31:17,947][105692] Updated weights for policy 0, policy_version 1938456 (0.0008) [2023-12-27 05:31:17,995][105692] Updated weights for policy 0, policy_version 1938466 (0.0007) [2023-12-27 05:31:18,043][105692] Updated weights for policy 0, policy_version 1938476 (0.0005) [2023-12-27 05:31:18,187][105620] Updated weights for policy 1, policy_version 1943225 (0.0009) [2023-12-27 05:31:18,249][105620] Updated weights for policy 1, policy_version 1943235 (0.0008) [2023-12-27 05:31:18,308][105620] Updated weights for policy 1, policy_version 1943245 (0.0010) [2023-12-27 05:31:18,787][105692] Updated weights for policy 0, policy_version 1938486 (0.0008) [2023-12-27 05:31:18,841][105692] Updated weights for policy 0, policy_version 1938497 (0.0010) [2023-12-27 05:31:18,895][105692] Updated weights for policy 0, policy_version 1938508 (0.0010) [2023-12-27 05:31:18,941][105620] Updated weights for policy 1, policy_version 1943255 (0.0007) [2023-12-27 05:31:18,995][105620] Updated weights for policy 1, policy_version 1943265 (0.0005) [2023-12-27 05:31:19,066][105620] Updated weights for policy 1, policy_version 1943275 (0.0007) [2023-12-27 05:31:19,657][105692] Updated weights for policy 0, policy_version 1938518 (0.0009) [2023-12-27 05:31:19,714][105692] Updated weights for policy 0, policy_version 1938528 (0.0008) [2023-12-27 05:31:19,744][105620] Updated weights for policy 1, policy_version 1943285 (0.0008) [2023-12-27 05:31:19,768][105692] Updated weights for policy 0, policy_version 1938538 (0.0009) [2023-12-27 05:31:19,805][105620] Updated weights for policy 1, policy_version 1943295 (0.0006) [2023-12-27 05:31:19,866][105620] Updated weights for policy 1, policy_version 1943305 (0.0010) [2023-12-27 05:31:20,566][105620] Updated weights for policy 1, policy_version 1943315 (0.0009) [2023-12-27 05:31:20,638][105620] Updated weights for policy 1, policy_version 1943325 (0.0007) [2023-12-27 05:31:20,638][105692] Updated weights for policy 0, policy_version 1938548 (0.0007) [2023-12-27 05:31:20,704][105620] Updated weights for policy 1, policy_version 1943335 (0.0010) [2023-12-27 05:31:20,706][105692] Updated weights for policy 0, policy_version 1938558 (0.0007) [2023-12-27 05:31:20,771][105692] Updated weights for policy 0, policy_version 1938568 (0.0006) [2023-12-27 05:31:21,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19355.3). Total num frames: 993910784. Throughput: 0: 9445.0, 1: 9845.5. Samples: 993899912. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:31:21,063][104569] Avg episode reward: [(0, '8446.160'), (1, '9161.685')] [2023-12-27 05:31:21,437][105620] Updated weights for policy 1, policy_version 1943345 (0.0009) [2023-12-27 05:31:21,496][105620] Updated weights for policy 1, policy_version 1943355 (0.0009) [2023-12-27 05:31:21,501][105692] Updated weights for policy 0, policy_version 1938578 (0.0007) [2023-12-27 05:31:21,548][105620] Updated weights for policy 1, policy_version 1943365 (0.0008) [2023-12-27 05:31:21,563][105692] Updated weights for policy 0, policy_version 1938588 (0.0008) [2023-12-27 05:31:21,608][105620] Updated weights for policy 1, policy_version 1943375 (0.0009) [2023-12-27 05:31:21,630][105692] Updated weights for policy 0, policy_version 1938598 (0.0008) [2023-12-27 05:31:21,699][105692] Updated weights for policy 0, policy_version 1938608 (0.0006) [2023-12-27 05:31:22,394][105692] Updated weights for policy 0, policy_version 1938618 (0.0008) [2023-12-27 05:31:22,435][105620] Updated weights for policy 1, policy_version 1943385 (0.0007) [2023-12-27 05:31:22,462][105692] Updated weights for policy 0, policy_version 1938628 (0.0006) [2023-12-27 05:31:22,484][105620] Updated weights for policy 1, policy_version 1943395 (0.0007) [2023-12-27 05:31:22,521][105692] Updated weights for policy 0, policy_version 1938638 (0.0008) [2023-12-27 05:31:22,536][105620] Updated weights for policy 1, policy_version 1943405 (0.0006) [2023-12-27 05:31:23,286][105692] Updated weights for policy 0, policy_version 1938648 (0.0010) [2023-12-27 05:31:23,308][105620] Updated weights for policy 1, policy_version 1943415 (0.0008) [2023-12-27 05:31:23,334][105692] Updated weights for policy 0, policy_version 1938658 (0.0010) [2023-12-27 05:31:23,363][105620] Updated weights for policy 1, policy_version 1943425 (0.0006) [2023-12-27 05:31:23,392][105692] Updated weights for policy 0, policy_version 1938668 (0.0010) [2023-12-27 05:31:23,421][105620] Updated weights for policy 1, policy_version 1943435 (0.0005) [2023-12-27 05:31:24,088][105692] Updated weights for policy 0, policy_version 1938678 (0.0009) [2023-12-27 05:31:24,155][105692] Updated weights for policy 0, policy_version 1938688 (0.0011) [2023-12-27 05:31:24,202][105620] Updated weights for policy 1, policy_version 1943445 (0.0007) [2023-12-27 05:31:24,208][105692] Updated weights for policy 0, policy_version 1938698 (0.0011) [2023-12-27 05:31:24,255][105620] Updated weights for policy 1, policy_version 1943455 (0.0006) [2023-12-27 05:31:24,314][105620] Updated weights for policy 1, policy_version 1943465 (0.0008) [2023-12-27 05:31:24,818][105692] Updated weights for policy 0, policy_version 1938708 (0.0011) [2023-12-27 05:31:24,874][105692] Updated weights for policy 0, policy_version 1938718 (0.0008) [2023-12-27 05:31:24,938][105692] Updated weights for policy 0, policy_version 1938728 (0.0009) [2023-12-27 05:31:25,141][105620] Updated weights for policy 1, policy_version 1943475 (0.0009) [2023-12-27 05:31:25,205][105620] Updated weights for policy 1, policy_version 1943485 (0.0009) [2023-12-27 05:31:25,271][105620] Updated weights for policy 1, policy_version 1943495 (0.0009) [2023-12-27 05:31:25,695][105692] Updated weights for policy 0, policy_version 1938738 (0.0009) [2023-12-27 05:31:25,754][105692] Updated weights for policy 0, policy_version 1938748 (0.0009) [2023-12-27 05:31:25,815][105692] Updated weights for policy 0, policy_version 1938758 (0.0008) [2023-12-27 05:31:25,869][105692] Updated weights for policy 0, policy_version 1938768 (0.0005) [2023-12-27 05:31:26,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19299.8). Total num frames: 994000896. Throughput: 0: 9425.2, 1: 9764.3. Samples: 994011672. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:31:26,062][104569] Avg episode reward: [(0, '7997.404'), (1, '9072.217')] [2023-12-27 05:31:26,074][105620] Updated weights for policy 1, policy_version 1943505 (0.0009) [2023-12-27 05:31:26,128][105620] Updated weights for policy 1, policy_version 1943515 (0.0010) [2023-12-27 05:31:26,183][105620] Updated weights for policy 1, policy_version 1943525 (0.0009) [2023-12-27 05:31:26,238][105620] Updated weights for policy 1, policy_version 1943535 (0.0008) [2023-12-27 05:31:26,516][105692] Updated weights for policy 0, policy_version 1938778 (0.0009) [2023-12-27 05:31:26,568][105692] Updated weights for policy 0, policy_version 1938788 (0.0009) [2023-12-27 05:31:26,627][105692] Updated weights for policy 0, policy_version 1938798 (0.0009) [2023-12-27 05:31:27,033][105620] Updated weights for policy 1, policy_version 1943545 (0.0009) [2023-12-27 05:31:27,090][105620] Updated weights for policy 1, policy_version 1943555 (0.0009) [2023-12-27 05:31:27,145][105620] Updated weights for policy 1, policy_version 1943565 (0.0007) [2023-12-27 05:31:27,347][105692] Updated weights for policy 0, policy_version 1938808 (0.0007) [2023-12-27 05:31:27,406][105692] Updated weights for policy 0, policy_version 1938818 (0.0005) [2023-12-27 05:31:27,467][105692] Updated weights for policy 0, policy_version 1938828 (0.0010) [2023-12-27 05:31:27,822][105620] Updated weights for policy 1, policy_version 1943575 (0.0005) [2023-12-27 05:31:27,868][105620] Updated weights for policy 1, policy_version 1943585 (0.0005) [2023-12-27 05:31:27,914][105620] Updated weights for policy 1, policy_version 1943595 (0.0006) [2023-12-27 05:31:28,052][105692] Updated weights for policy 0, policy_version 1938838 (0.0010) [2023-12-27 05:31:28,106][105692] Updated weights for policy 0, policy_version 1938848 (0.0010) [2023-12-27 05:31:28,163][105692] Updated weights for policy 0, policy_version 1938858 (0.0010) [2023-12-27 05:31:28,536][105620] Updated weights for policy 1, policy_version 1943605 (0.0009) [2023-12-27 05:31:28,597][105620] Updated weights for policy 1, policy_version 1943615 (0.0009) [2023-12-27 05:31:28,665][105620] Updated weights for policy 1, policy_version 1943625 (0.0007) [2023-12-27 05:31:28,768][105692] Updated weights for policy 0, policy_version 1938868 (0.0010) [2023-12-27 05:31:28,820][105692] Updated weights for policy 0, policy_version 1938878 (0.0010) [2023-12-27 05:31:28,876][105692] Updated weights for policy 0, policy_version 1938888 (0.0007) [2023-12-27 05:31:29,331][105620] Updated weights for policy 1, policy_version 1943635 (0.0008) [2023-12-27 05:31:29,396][105620] Updated weights for policy 1, policy_version 1943645 (0.0007) [2023-12-27 05:31:29,448][105620] Updated weights for policy 1, policy_version 1943655 (0.0008) [2023-12-27 05:31:29,625][105692] Updated weights for policy 0, policy_version 1938898 (0.0010) [2023-12-27 05:31:29,686][105692] Updated weights for policy 0, policy_version 1938908 (0.0008) [2023-12-27 05:31:29,738][105692] Updated weights for policy 0, policy_version 1938918 (0.0009) [2023-12-27 05:31:29,807][105692] Updated weights for policy 0, policy_version 1938928 (0.0009) [2023-12-27 05:31:30,175][105620] Updated weights for policy 1, policy_version 1943665 (0.0009) [2023-12-27 05:31:30,232][105620] Updated weights for policy 1, policy_version 1943675 (0.0010) [2023-12-27 05:31:30,290][105620] Updated weights for policy 1, policy_version 1943685 (0.0009) [2023-12-27 05:31:30,349][105620] Updated weights for policy 1, policy_version 1943695 (0.0009) [2023-12-27 05:31:30,513][105692] Updated weights for policy 0, policy_version 1938938 (0.0008) [2023-12-27 05:31:30,566][105692] Updated weights for policy 0, policy_version 1938948 (0.0008) [2023-12-27 05:31:30,622][105692] Updated weights for policy 0, policy_version 1938958 (0.0007) [2023-12-27 05:31:31,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19327.6). Total num frames: 994099200. Throughput: 0: 9507.4, 1: 9709.6. Samples: 994072440. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:31:31,063][104569] Avg episode reward: [(0, '8267.643'), (1, '9164.344')] [2023-12-27 05:31:31,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001938960_496443392.pth... [2023-12-27 05:31:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001943696_497655808.pth... [2023-12-27 05:31:31,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001942576_497369088.pth [2023-12-27 05:31:31,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001937840_496156672.pth [2023-12-27 05:31:31,172][105620] Updated weights for policy 1, policy_version 1943705 (0.0007) [2023-12-27 05:31:31,223][105620] Updated weights for policy 1, policy_version 1943715 (0.0007) [2023-12-27 05:31:31,250][105692] Updated weights for policy 0, policy_version 1938968 (0.0008) [2023-12-27 05:31:31,279][105620] Updated weights for policy 1, policy_version 1943725 (0.0007) [2023-12-27 05:31:31,313][105692] Updated weights for policy 0, policy_version 1938978 (0.0008) [2023-12-27 05:31:31,377][105692] Updated weights for policy 0, policy_version 1938988 (0.0008) [2023-12-27 05:31:32,060][105620] Updated weights for policy 1, policy_version 1943735 (0.0007) [2023-12-27 05:31:32,075][105692] Updated weights for policy 0, policy_version 1938998 (0.0007) [2023-12-27 05:31:32,117][105620] Updated weights for policy 1, policy_version 1943745 (0.0006) [2023-12-27 05:31:32,135][105692] Updated weights for policy 0, policy_version 1939008 (0.0007) [2023-12-27 05:31:32,173][105620] Updated weights for policy 1, policy_version 1943755 (0.0006) [2023-12-27 05:31:32,191][105692] Updated weights for policy 0, policy_version 1939018 (0.0007) [2023-12-27 05:31:32,805][105692] Updated weights for policy 0, policy_version 1939028 (0.0009) [2023-12-27 05:31:32,853][105692] Updated weights for policy 0, policy_version 1939038 (0.0010) [2023-12-27 05:31:32,908][105692] Updated weights for policy 0, policy_version 1939048 (0.0010) [2023-12-27 05:31:32,964][105620] Updated weights for policy 1, policy_version 1943765 (0.0008) [2023-12-27 05:31:33,020][105620] Updated weights for policy 1, policy_version 1943775 (0.0008) [2023-12-27 05:31:33,081][105620] Updated weights for policy 1, policy_version 1943785 (0.0010) [2023-12-27 05:31:33,491][105692] Updated weights for policy 0, policy_version 1939058 (0.0011) [2023-12-27 05:31:33,541][105692] Updated weights for policy 0, policy_version 1939068 (0.0010) [2023-12-27 05:31:33,589][105692] Updated weights for policy 0, policy_version 1939078 (0.0010) [2023-12-27 05:31:33,656][105692] Updated weights for policy 0, policy_version 1939088 (0.0010) [2023-12-27 05:31:33,874][105620] Updated weights for policy 1, policy_version 1943795 (0.0012) [2023-12-27 05:31:33,926][105620] Updated weights for policy 1, policy_version 1943805 (0.0009) [2023-12-27 05:31:33,979][105620] Updated weights for policy 1, policy_version 1943815 (0.0009) [2023-12-27 05:31:34,357][105692] Updated weights for policy 0, policy_version 1939098 (0.0011) [2023-12-27 05:31:34,426][105692] Updated weights for policy 0, policy_version 1939108 (0.0011) [2023-12-27 05:31:34,475][105692] Updated weights for policy 0, policy_version 1939118 (0.0011) [2023-12-27 05:31:34,738][105620] Updated weights for policy 1, policy_version 1943825 (0.0008) [2023-12-27 05:31:34,803][105620] Updated weights for policy 1, policy_version 1943835 (0.0006) [2023-12-27 05:31:34,865][105620] Updated weights for policy 1, policy_version 1943845 (0.0005) [2023-12-27 05:31:34,928][105620] Updated weights for policy 1, policy_version 1943855 (0.0009) [2023-12-27 05:31:35,188][105692] Updated weights for policy 0, policy_version 1939128 (0.0010) [2023-12-27 05:31:35,239][105692] Updated weights for policy 0, policy_version 1939138 (0.0007) [2023-12-27 05:31:35,284][105692] Updated weights for policy 0, policy_version 1939148 (0.0010) [2023-12-27 05:31:35,435][105620] Updated weights for policy 1, policy_version 1943865 (0.0006) [2023-12-27 05:31:35,489][105620] Updated weights for policy 1, policy_version 1943875 (0.0005) [2023-12-27 05:31:35,537][105620] Updated weights for policy 1, policy_version 1943885 (0.0005) [2023-12-27 05:31:35,872][105692] Updated weights for policy 0, policy_version 1939158 (0.0010) [2023-12-27 05:31:35,927][105692] Updated weights for policy 0, policy_version 1939168 (0.0010) [2023-12-27 05:31:35,979][105692] Updated weights for policy 0, policy_version 1939178 (0.0010) [2023-12-27 05:31:36,062][104569] Fps is (10 sec: 20479.5, 60 sec: 19387.6, 300 sec: 19355.3). Total num frames: 994205696. Throughput: 0: 9627.9, 1: 9638.8. Samples: 994190048. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) [2023-12-27 05:31:36,063][104569] Avg episode reward: [(0, '8720.291'), (1, '9346.066')] [2023-12-27 05:31:36,117][105620] Updated weights for policy 1, policy_version 1943895 (0.0009) [2023-12-27 05:31:36,178][105620] Updated weights for policy 1, policy_version 1943905 (0.0011) [2023-12-27 05:31:36,258][105620] Updated weights for policy 1, policy_version 1943915 (0.0010) [2023-12-27 05:31:36,777][105692] Updated weights for policy 0, policy_version 1939188 (0.0010) [2023-12-27 05:31:36,837][105692] Updated weights for policy 0, policy_version 1939198 (0.0009) [2023-12-27 05:31:36,905][105692] Updated weights for policy 0, policy_version 1939208 (0.0010) [2023-12-27 05:31:36,912][105620] Updated weights for policy 1, policy_version 1943925 (0.0010) [2023-12-27 05:31:36,977][105620] Updated weights for policy 1, policy_version 1943935 (0.0009) [2023-12-27 05:31:37,042][105620] Updated weights for policy 1, policy_version 1943945 (0.0009) [2023-12-27 05:31:37,630][105620] Updated weights for policy 1, policy_version 1943955 (0.0008) [2023-12-27 05:31:37,682][105620] Updated weights for policy 1, policy_version 1943965 (0.0009) [2023-12-27 05:31:37,730][105692] Updated weights for policy 0, policy_version 1939218 (0.0006) [2023-12-27 05:31:37,735][105620] Updated weights for policy 1, policy_version 1943975 (0.0011) [2023-12-27 05:31:37,780][105692] Updated weights for policy 0, policy_version 1939228 (0.0010) [2023-12-27 05:31:37,827][105692] Updated weights for policy 0, policy_version 1939238 (0.0010) [2023-12-27 05:31:37,882][105692] Updated weights for policy 0, policy_version 1939248 (0.0011) [2023-12-27 05:31:38,487][105620] Updated weights for policy 1, policy_version 1943985 (0.0006) [2023-12-27 05:31:38,547][105620] Updated weights for policy 1, policy_version 1943995 (0.0008) [2023-12-27 05:31:38,612][105620] Updated weights for policy 1, policy_version 1944005 (0.0005) [2023-12-27 05:31:38,682][105620] Updated weights for policy 1, policy_version 1944015 (0.0006) [2023-12-27 05:31:38,690][105692] Updated weights for policy 0, policy_version 1939258 (0.0008) [2023-12-27 05:31:38,756][105692] Updated weights for policy 0, policy_version 1939268 (0.0010) [2023-12-27 05:31:38,818][105692] Updated weights for policy 0, policy_version 1939278 (0.0008) [2023-12-27 05:31:39,382][105620] Updated weights for policy 1, policy_version 1944025 (0.0009) [2023-12-27 05:31:39,448][105620] Updated weights for policy 1, policy_version 1944035 (0.0009) [2023-12-27 05:31:39,510][105620] Updated weights for policy 1, policy_version 1944045 (0.0008) [2023-12-27 05:31:39,573][105692] Updated weights for policy 0, policy_version 1939288 (0.0008) [2023-12-27 05:31:39,627][105692] Updated weights for policy 0, policy_version 1939298 (0.0010) [2023-12-27 05:31:39,683][105692] Updated weights for policy 0, policy_version 1939308 (0.0008) [2023-12-27 05:31:40,253][105620] Updated weights for policy 1, policy_version 1944055 (0.0009) [2023-12-27 05:31:40,315][105620] Updated weights for policy 1, policy_version 1944065 (0.0009) [2023-12-27 05:31:40,380][105620] Updated weights for policy 1, policy_version 1944075 (0.0009) [2023-12-27 05:31:40,461][105692] Updated weights for policy 0, policy_version 1939318 (0.0009) [2023-12-27 05:31:40,523][105692] Updated weights for policy 0, policy_version 1939328 (0.0009) [2023-12-27 05:31:40,579][105692] Updated weights for policy 0, policy_version 1939338 (0.0008) [2023-12-27 05:31:41,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19355.3). Total num frames: 994295808. Throughput: 0: 9622.1, 1: 9759.9. Samples: 994307620. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:31:41,062][104569] Avg episode reward: [(0, '8356.298'), (1, '9345.925')] [2023-12-27 05:31:41,078][105620] Updated weights for policy 1, policy_version 1944085 (0.0008) [2023-12-27 05:31:41,146][105620] Updated weights for policy 1, policy_version 1944095 (0.0009) [2023-12-27 05:31:41,210][105620] Updated weights for policy 1, policy_version 1944105 (0.0009) [2023-12-27 05:31:41,280][105692] Updated weights for policy 0, policy_version 1939348 (0.0006) [2023-12-27 05:31:41,337][105692] Updated weights for policy 0, policy_version 1939358 (0.0009) [2023-12-27 05:31:41,406][105692] Updated weights for policy 0, policy_version 1939368 (0.0008) [2023-12-27 05:31:41,996][105620] Updated weights for policy 1, policy_version 1944115 (0.0009) [2023-12-27 05:31:42,046][105620] Updated weights for policy 1, policy_version 1944125 (0.0008) [2023-12-27 05:31:42,075][105692] Updated weights for policy 0, policy_version 1939378 (0.0006) [2023-12-27 05:31:42,101][105620] Updated weights for policy 1, policy_version 1944135 (0.0009) [2023-12-27 05:31:42,136][105692] Updated weights for policy 0, policy_version 1939388 (0.0009) [2023-12-27 05:31:42,189][105692] Updated weights for policy 0, policy_version 1939398 (0.0009) [2023-12-27 05:31:42,244][105692] Updated weights for policy 0, policy_version 1939408 (0.0011) [2023-12-27 05:31:42,846][105620] Updated weights for policy 1, policy_version 1944145 (0.0008) [2023-12-27 05:31:42,901][105620] Updated weights for policy 1, policy_version 1944155 (0.0009) [2023-12-27 05:31:42,953][105620] Updated weights for policy 1, policy_version 1944165 (0.0008) [2023-12-27 05:31:43,003][105620] Updated weights for policy 1, policy_version 1944175 (0.0006) [2023-12-27 05:31:43,012][105692] Updated weights for policy 0, policy_version 1939418 (0.0008) [2023-12-27 05:31:43,070][105692] Updated weights for policy 0, policy_version 1939428 (0.0009) [2023-12-27 05:31:43,121][105692] Updated weights for policy 0, policy_version 1939438 (0.0009) [2023-12-27 05:31:43,726][105620] Updated weights for policy 1, policy_version 1944185 (0.0008) [2023-12-27 05:31:43,772][105620] Updated weights for policy 1, policy_version 1944195 (0.0005) [2023-12-27 05:31:43,826][105620] Updated weights for policy 1, policy_version 1944205 (0.0005) [2023-12-27 05:31:43,856][105692] Updated weights for policy 0, policy_version 1939448 (0.0010) [2023-12-27 05:31:43,905][105692] Updated weights for policy 0, policy_version 1939458 (0.0010) [2023-12-27 05:31:43,954][105692] Updated weights for policy 0, policy_version 1939468 (0.0011) [2023-12-27 05:31:44,423][105620] Updated weights for policy 1, policy_version 1944215 (0.0008) [2023-12-27 05:31:44,476][105620] Updated weights for policy 1, policy_version 1944226 (0.0010) [2023-12-27 05:31:44,530][105620] Updated weights for policy 1, policy_version 1944236 (0.0010) [2023-12-27 05:31:44,598][105692] Updated weights for policy 0, policy_version 1939478 (0.0007) [2023-12-27 05:31:44,651][105692] Updated weights for policy 0, policy_version 1939488 (0.0005) [2023-12-27 05:31:44,704][105692] Updated weights for policy 0, policy_version 1939498 (0.0009) [2023-12-27 05:31:45,313][105692] Updated weights for policy 0, policy_version 1939508 (0.0008) [2023-12-27 05:31:45,338][105620] Updated weights for policy 1, policy_version 1944246 (0.0009) [2023-12-27 05:31:45,368][105692] Updated weights for policy 0, policy_version 1939518 (0.0005) [2023-12-27 05:31:45,399][105620] Updated weights for policy 1, policy_version 1944256 (0.0009) [2023-12-27 05:31:45,431][105692] Updated weights for policy 0, policy_version 1939528 (0.0006) [2023-12-27 05:31:45,460][105620] Updated weights for policy 1, policy_version 1944266 (0.0009) [2023-12-27 05:31:46,028][105692] Updated weights for policy 0, policy_version 1939538 (0.0005) [2023-12-27 05:31:46,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19355.3). Total num frames: 994394112. Throughput: 0: 9679.5, 1: 9735.9. Samples: 994364736. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:31:46,063][104569] Avg episode reward: [(0, '7810.381'), (1, '9253.664')] [2023-12-27 05:31:46,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001944272_497803264.pth... [2023-12-27 05:31:46,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001943152_497516544.pth [2023-12-27 05:31:46,090][105692] Updated weights for policy 0, policy_version 1939548 (0.0005) [2023-12-27 05:31:46,140][105692] Updated weights for policy 0, policy_version 1939558 (0.0007) [2023-12-27 05:31:46,184][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001939568_496599040.pth... [2023-12-27 05:31:46,187][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001938384_496295936.pth [2023-12-27 05:31:46,188][105692] Updated weights for policy 0, policy_version 1939568 (0.0010) [2023-12-27 05:31:46,248][105620] Updated weights for policy 1, policy_version 1944276 (0.0009) [2023-12-27 05:31:46,300][105620] Updated weights for policy 1, policy_version 1944286 (0.0008) [2023-12-27 05:31:46,344][105620] Updated weights for policy 1, policy_version 1944296 (0.0008) [2023-12-27 05:31:46,758][105692] Updated weights for policy 0, policy_version 1939578 (0.0009) [2023-12-27 05:31:46,805][105692] Updated weights for policy 0, policy_version 1939588 (0.0009) [2023-12-27 05:31:46,866][105692] Updated weights for policy 0, policy_version 1939598 (0.0005) [2023-12-27 05:31:47,218][105620] Updated weights for policy 1, policy_version 1944306 (0.0008) [2023-12-27 05:31:47,280][105620] Updated weights for policy 1, policy_version 1944316 (0.0010) [2023-12-27 05:31:47,337][105620] Updated weights for policy 1, policy_version 1944326 (0.0010) [2023-12-27 05:31:47,390][105620] Updated weights for policy 1, policy_version 1944336 (0.0011) [2023-12-27 05:31:47,454][105692] Updated weights for policy 0, policy_version 1939608 (0.0007) [2023-12-27 05:31:47,515][105692] Updated weights for policy 0, policy_version 1939618 (0.0010) [2023-12-27 05:31:47,573][105692] Updated weights for policy 0, policy_version 1939628 (0.0010) [2023-12-27 05:31:48,187][105620] Updated weights for policy 1, policy_version 1944346 (0.0008) [2023-12-27 05:31:48,239][105620] Updated weights for policy 1, policy_version 1944356 (0.0008) [2023-12-27 05:31:48,290][105620] Updated weights for policy 1, policy_version 1944366 (0.0007) [2023-12-27 05:31:48,300][105692] Updated weights for policy 0, policy_version 1939638 (0.0010) [2023-12-27 05:31:48,368][105692] Updated weights for policy 0, policy_version 1939648 (0.0011) [2023-12-27 05:31:48,427][105692] Updated weights for policy 0, policy_version 1939658 (0.0010) [2023-12-27 05:31:49,069][105620] Updated weights for policy 1, policy_version 1944376 (0.0008) [2023-12-27 05:31:49,132][105620] Updated weights for policy 1, policy_version 1944386 (0.0008) [2023-12-27 05:31:49,163][105692] Updated weights for policy 0, policy_version 1939668 (0.0009) [2023-12-27 05:31:49,184][105620] Updated weights for policy 1, policy_version 1944396 (0.0006) [2023-12-27 05:31:49,221][105692] Updated weights for policy 0, policy_version 1939678 (0.0010) [2023-12-27 05:31:49,281][105692] Updated weights for policy 0, policy_version 1939688 (0.0011) [2023-12-27 05:31:49,974][105620] Updated weights for policy 1, policy_version 1944406 (0.0007) [2023-12-27 05:31:50,038][105620] Updated weights for policy 1, policy_version 1944416 (0.0008) [2023-12-27 05:31:50,069][105692] Updated weights for policy 0, policy_version 1939698 (0.0009) [2023-12-27 05:31:50,095][105620] Updated weights for policy 1, policy_version 1944426 (0.0009) [2023-12-27 05:31:50,126][105692] Updated weights for policy 0, policy_version 1939708 (0.0006) [2023-12-27 05:31:50,172][105692] Updated weights for policy 0, policy_version 1939718 (0.0008) [2023-12-27 05:31:50,228][105692] Updated weights for policy 0, policy_version 1939728 (0.0008) [2023-12-27 05:31:50,857][105620] Updated weights for policy 1, policy_version 1944436 (0.0011) [2023-12-27 05:31:50,913][105620] Updated weights for policy 1, policy_version 1944446 (0.0011) [2023-12-27 05:31:50,926][105692] Updated weights for policy 0, policy_version 1939738 (0.0011) [2023-12-27 05:31:50,972][105620] Updated weights for policy 1, policy_version 1944456 (0.0011) [2023-12-27 05:31:50,983][105692] Updated weights for policy 0, policy_version 1939748 (0.0010) [2023-12-27 05:31:51,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 994500608. Throughput: 0: 9808.0, 1: 9662.9. Samples: 994483264. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:31:51,062][104569] Avg episode reward: [(0, '7903.398'), (1, '9253.626')] [2023-12-27 05:31:51,680][105692] Updated weights for policy 0, policy_version 1939761 (0.0007) [2023-12-27 05:31:51,714][105620] Updated weights for policy 1, policy_version 1944466 (0.0010) [2023-12-27 05:31:51,745][105692] Updated weights for policy 0, policy_version 1939771 (0.0010) [2023-12-27 05:31:51,777][105620] Updated weights for policy 1, policy_version 1944476 (0.0008) [2023-12-27 05:31:51,804][105692] Updated weights for policy 0, policy_version 1939781 (0.0010) [2023-12-27 05:31:51,829][105620] Updated weights for policy 1, policy_version 1944486 (0.0008) [2023-12-27 05:31:51,867][105692] Updated weights for policy 0, policy_version 1939791 (0.0011) [2023-12-27 05:31:51,877][105620] Updated weights for policy 1, policy_version 1944496 (0.0008) [2023-12-27 05:31:52,566][105692] Updated weights for policy 0, policy_version 1939801 (0.0009) [2023-12-27 05:31:52,625][105692] Updated weights for policy 0, policy_version 1939811 (0.0009) [2023-12-27 05:31:52,673][105620] Updated weights for policy 1, policy_version 1944506 (0.0007) [2023-12-27 05:31:52,687][105692] Updated weights for policy 0, policy_version 1939821 (0.0006) [2023-12-27 05:31:52,726][105620] Updated weights for policy 1, policy_version 1944516 (0.0007) [2023-12-27 05:31:52,779][105620] Updated weights for policy 1, policy_version 1944526 (0.0006) [2023-12-27 05:31:53,355][105692] Updated weights for policy 0, policy_version 1939831 (0.0005) [2023-12-27 05:31:53,411][105692] Updated weights for policy 0, policy_version 1939841 (0.0005) [2023-12-27 05:31:53,480][105692] Updated weights for policy 0, policy_version 1939851 (0.0005) [2023-12-27 05:31:53,537][105620] Updated weights for policy 1, policy_version 1944536 (0.0009) [2023-12-27 05:31:53,603][105620] Updated weights for policy 1, policy_version 1944546 (0.0009) [2023-12-27 05:31:53,668][105620] Updated weights for policy 1, policy_version 1944556 (0.0009) [2023-12-27 05:31:54,065][105692] Updated weights for policy 0, policy_version 1939861 (0.0005) [2023-12-27 05:31:54,131][105692] Updated weights for policy 0, policy_version 1939871 (0.0007) [2023-12-27 05:31:54,196][105692] Updated weights for policy 0, policy_version 1939881 (0.0009) [2023-12-27 05:31:54,398][105620] Updated weights for policy 1, policy_version 1944566 (0.0006) [2023-12-27 05:31:54,458][105620] Updated weights for policy 1, policy_version 1944576 (0.0008) [2023-12-27 05:31:54,525][105620] Updated weights for policy 1, policy_version 1944586 (0.0008) [2023-12-27 05:31:54,821][105692] Updated weights for policy 0, policy_version 1939891 (0.0008) [2023-12-27 05:31:54,870][105692] Updated weights for policy 0, policy_version 1939901 (0.0005) [2023-12-27 05:31:54,925][105692] Updated weights for policy 0, policy_version 1939911 (0.0005) [2023-12-27 05:31:55,111][105620] Updated weights for policy 1, policy_version 1944596 (0.0007) [2023-12-27 05:31:55,173][105620] Updated weights for policy 1, policy_version 1944606 (0.0007) [2023-12-27 05:31:55,237][105620] Updated weights for policy 1, policy_version 1944616 (0.0010) [2023-12-27 05:31:55,540][105692] Updated weights for policy 0, policy_version 1939921 (0.0006) [2023-12-27 05:31:55,595][105692] Updated weights for policy 0, policy_version 1939931 (0.0005) [2023-12-27 05:31:55,654][105692] Updated weights for policy 0, policy_version 1939941 (0.0009) [2023-12-27 05:31:55,705][105692] Updated weights for policy 0, policy_version 1939952 (0.0009) [2023-12-27 05:31:55,822][105620] Updated weights for policy 1, policy_version 1944626 (0.0009) [2023-12-27 05:31:55,878][105620] Updated weights for policy 1, policy_version 1944636 (0.0005) [2023-12-27 05:31:55,930][105620] Updated weights for policy 1, policy_version 1944646 (0.0005) [2023-12-27 05:31:55,979][105620] Updated weights for policy 1, policy_version 1944656 (0.0005) [2023-12-27 05:31:56,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 994598912. Throughput: 0: 9928.5, 1: 9603.1. Samples: 994604452. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:31:56,063][104569] Avg episode reward: [(0, '8446.762'), (1, '9345.881')] [2023-12-27 05:31:56,504][105692] Updated weights for policy 0, policy_version 1939962 (0.0005) [2023-12-27 05:31:56,541][105620] Updated weights for policy 1, policy_version 1944666 (0.0010) [2023-12-27 05:31:56,572][105692] Updated weights for policy 0, policy_version 1939972 (0.0006) [2023-12-27 05:31:56,592][105620] Updated weights for policy 1, policy_version 1944676 (0.0010) [2023-12-27 05:31:56,639][105692] Updated weights for policy 0, policy_version 1939982 (0.0008) [2023-12-27 05:31:56,654][105620] Updated weights for policy 1, policy_version 1944686 (0.0006) [2023-12-27 05:31:57,232][105620] Updated weights for policy 1, policy_version 1944696 (0.0008) [2023-12-27 05:31:57,296][105620] Updated weights for policy 1, policy_version 1944706 (0.0010) [2023-12-27 05:31:57,355][105620] Updated weights for policy 1, policy_version 1944716 (0.0006) [2023-12-27 05:31:57,369][105692] Updated weights for policy 0, policy_version 1939992 (0.0008) [2023-12-27 05:31:57,439][105692] Updated weights for policy 0, policy_version 1940002 (0.0009) [2023-12-27 05:31:57,495][105692] Updated weights for policy 0, policy_version 1940012 (0.0008) [2023-12-27 05:31:58,050][105620] Updated weights for policy 1, policy_version 1944726 (0.0008) [2023-12-27 05:31:58,105][105620] Updated weights for policy 1, policy_version 1944736 (0.0010) [2023-12-27 05:31:58,163][105620] Updated weights for policy 1, policy_version 1944746 (0.0011) [2023-12-27 05:31:58,200][105692] Updated weights for policy 0, policy_version 1940022 (0.0007) [2023-12-27 05:31:58,259][105692] Updated weights for policy 0, policy_version 1940032 (0.0008) [2023-12-27 05:31:58,324][105692] Updated weights for policy 0, policy_version 1940042 (0.0008) [2023-12-27 05:31:59,023][105620] Updated weights for policy 1, policy_version 1944756 (0.0010) [2023-12-27 05:31:59,044][105692] Updated weights for policy 0, policy_version 1940052 (0.0007) [2023-12-27 05:31:59,076][105620] Updated weights for policy 1, policy_version 1944766 (0.0009) [2023-12-27 05:31:59,097][105692] Updated weights for policy 0, policy_version 1940062 (0.0009) [2023-12-27 05:31:59,125][105620] Updated weights for policy 1, policy_version 1944776 (0.0008) [2023-12-27 05:31:59,150][105692] Updated weights for policy 0, policy_version 1940073 (0.0009) [2023-12-27 05:31:59,810][105620] Updated weights for policy 1, policy_version 1944786 (0.0009) [2023-12-27 05:31:59,871][105620] Updated weights for policy 1, policy_version 1944796 (0.0008) [2023-12-27 05:31:59,932][105620] Updated weights for policy 1, policy_version 1944806 (0.0009) [2023-12-27 05:31:59,990][105620] Updated weights for policy 1, policy_version 1944816 (0.0009) [2023-12-27 05:32:00,005][105692] Updated weights for policy 0, policy_version 1940083 (0.0009) [2023-12-27 05:32:00,058][105692] Updated weights for policy 0, policy_version 1940093 (0.0009) [2023-12-27 05:32:00,109][105692] Updated weights for policy 0, policy_version 1940103 (0.0009) [2023-12-27 05:32:00,613][105620] Updated weights for policy 1, policy_version 1944826 (0.0005) [2023-12-27 05:32:00,662][105620] Updated weights for policy 1, policy_version 1944836 (0.0005) [2023-12-27 05:32:00,709][105620] Updated weights for policy 1, policy_version 1944846 (0.0005) [2023-12-27 05:32:00,840][105692] Updated weights for policy 0, policy_version 1940113 (0.0008) [2023-12-27 05:32:00,890][105692] Updated weights for policy 0, policy_version 1940123 (0.0005) [2023-12-27 05:32:00,943][105692] Updated weights for policy 0, policy_version 1940133 (0.0008) [2023-12-27 05:32:00,994][105692] Updated weights for policy 0, policy_version 1940143 (0.0010) [2023-12-27 05:32:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 994697216. Throughput: 0: 9943.5, 1: 9647.7. Samples: 994662716. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:32:01,062][104569] Avg episode reward: [(0, '8086.136'), (1, '9163.704')] [2023-12-27 05:32:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001940144_496746496.pth... [2023-12-27 05:32:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001944848_497950720.pth... [2023-12-27 05:32:01,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001938960_496443392.pth [2023-12-27 05:32:01,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001943696_497655808.pth [2023-12-27 05:32:01,434][105620] Updated weights for policy 1, policy_version 1944856 (0.0009) [2023-12-27 05:32:01,484][105620] Updated weights for policy 1, policy_version 1944866 (0.0009) [2023-12-27 05:32:01,533][105620] Updated weights for policy 1, policy_version 1944876 (0.0008) [2023-12-27 05:32:01,672][105692] Updated weights for policy 0, policy_version 1940153 (0.0009) [2023-12-27 05:32:01,732][105692] Updated weights for policy 0, policy_version 1940163 (0.0009) [2023-12-27 05:32:01,789][105692] Updated weights for policy 0, policy_version 1940173 (0.0009) [2023-12-27 05:32:02,328][105620] Updated weights for policy 1, policy_version 1944886 (0.0009) [2023-12-27 05:32:02,389][105620] Updated weights for policy 1, policy_version 1944896 (0.0009) [2023-12-27 05:32:02,438][105620] Updated weights for policy 1, policy_version 1944906 (0.0008) [2023-12-27 05:32:02,516][105692] Updated weights for policy 0, policy_version 1940183 (0.0008) [2023-12-27 05:32:02,577][105692] Updated weights for policy 0, policy_version 1940193 (0.0009) [2023-12-27 05:32:02,635][105692] Updated weights for policy 0, policy_version 1940203 (0.0009) [2023-12-27 05:32:03,142][105620] Updated weights for policy 1, policy_version 1944916 (0.0007) [2023-12-27 05:32:03,192][105620] Updated weights for policy 1, policy_version 1944926 (0.0005) [2023-12-27 05:32:03,241][105620] Updated weights for policy 1, policy_version 1944936 (0.0008) [2023-12-27 05:32:03,283][105692] Updated weights for policy 0, policy_version 1940213 (0.0010) [2023-12-27 05:32:03,331][105692] Updated weights for policy 0, policy_version 1940223 (0.0010) [2023-12-27 05:32:03,388][105692] Updated weights for policy 0, policy_version 1940233 (0.0010) [2023-12-27 05:32:03,937][105620] Updated weights for policy 1, policy_version 1944946 (0.0008) [2023-12-27 05:32:03,979][105692] Updated weights for policy 0, policy_version 1940243 (0.0008) [2023-12-27 05:32:04,000][105620] Updated weights for policy 1, policy_version 1944956 (0.0008) [2023-12-27 05:32:04,035][105692] Updated weights for policy 0, policy_version 1940253 (0.0008) [2023-12-27 05:32:04,063][105620] Updated weights for policy 1, policy_version 1944966 (0.0008) [2023-12-27 05:32:04,090][105692] Updated weights for policy 0, policy_version 1940263 (0.0008) [2023-12-27 05:32:04,125][105620] Updated weights for policy 1, policy_version 1944976 (0.0008) [2023-12-27 05:32:04,810][105620] Updated weights for policy 1, policy_version 1944986 (0.0010) [2023-12-27 05:32:04,835][105692] Updated weights for policy 0, policy_version 1940273 (0.0008) [2023-12-27 05:32:04,869][105620] Updated weights for policy 1, policy_version 1944996 (0.0010) [2023-12-27 05:32:04,886][105692] Updated weights for policy 0, policy_version 1940283 (0.0010) [2023-12-27 05:32:04,925][105620] Updated weights for policy 1, policy_version 1945006 (0.0010) [2023-12-27 05:32:04,940][105692] Updated weights for policy 0, policy_version 1940293 (0.0010) [2023-12-27 05:32:05,007][105692] Updated weights for policy 0, policy_version 1940303 (0.0010) [2023-12-27 05:32:05,618][105692] Updated weights for policy 0, policy_version 1940313 (0.0006) [2023-12-27 05:32:05,634][105620] Updated weights for policy 1, policy_version 1945016 (0.0011) [2023-12-27 05:32:05,675][105692] Updated weights for policy 0, policy_version 1940323 (0.0005) [2023-12-27 05:32:05,679][105620] Updated weights for policy 1, policy_version 1945026 (0.0010) [2023-12-27 05:32:05,724][105692] Updated weights for policy 0, policy_version 1940333 (0.0005) [2023-12-27 05:32:05,724][105620] Updated weights for policy 1, policy_version 1945036 (0.0010) [2023-12-27 05:32:06,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 994795520. Throughput: 0: 9955.6, 1: 9628.6. Samples: 994781204. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:32:06,063][104569] Avg episode reward: [(0, '8264.870'), (1, '9073.097')] [2023-12-27 05:32:06,377][105692] Updated weights for policy 0, policy_version 1940343 (0.0008) [2023-12-27 05:32:06,436][105692] Updated weights for policy 0, policy_version 1940353 (0.0009) [2023-12-27 05:32:06,477][105620] Updated weights for policy 1, policy_version 1945046 (0.0011) [2023-12-27 05:32:06,492][105692] Updated weights for policy 0, policy_version 1940363 (0.0009) [2023-12-27 05:32:06,534][105620] Updated weights for policy 1, policy_version 1945056 (0.0007) [2023-12-27 05:32:06,590][105620] Updated weights for policy 1, policy_version 1945066 (0.0011) [2023-12-27 05:32:07,196][105620] Updated weights for policy 1, policy_version 1945076 (0.0008) [2023-12-27 05:32:07,261][105620] Updated weights for policy 1, policy_version 1945086 (0.0010) [2023-12-27 05:32:07,320][105620] Updated weights for policy 1, policy_version 1945096 (0.0010) [2023-12-27 05:32:07,341][105692] Updated weights for policy 0, policy_version 1940373 (0.0007) [2023-12-27 05:32:07,401][105692] Updated weights for policy 0, policy_version 1940383 (0.0005) [2023-12-27 05:32:07,466][105692] Updated weights for policy 0, policy_version 1940393 (0.0005) [2023-12-27 05:32:08,024][105692] Updated weights for policy 0, policy_version 1940403 (0.0007) [2023-12-27 05:32:08,033][105620] Updated weights for policy 1, policy_version 1945106 (0.0009) [2023-12-27 05:32:08,088][105692] Updated weights for policy 0, policy_version 1940413 (0.0011) [2023-12-27 05:32:08,090][105620] Updated weights for policy 1, policy_version 1945116 (0.0006) [2023-12-27 05:32:08,135][105620] Updated weights for policy 1, policy_version 1945126 (0.0005) [2023-12-27 05:32:08,137][105692] Updated weights for policy 0, policy_version 1940423 (0.0010) [2023-12-27 05:32:08,187][105620] Updated weights for policy 1, policy_version 1945136 (0.0008) [2023-12-27 05:32:08,833][105692] Updated weights for policy 0, policy_version 1940433 (0.0008) [2023-12-27 05:32:08,871][105620] Updated weights for policy 1, policy_version 1945146 (0.0008) [2023-12-27 05:32:08,893][105692] Updated weights for policy 0, policy_version 1940443 (0.0011) [2023-12-27 05:32:08,939][105620] Updated weights for policy 1, policy_version 1945156 (0.0010) [2023-12-27 05:32:08,949][105692] Updated weights for policy 0, policy_version 1940453 (0.0011) [2023-12-27 05:32:08,995][105620] Updated weights for policy 1, policy_version 1945166 (0.0011) [2023-12-27 05:32:09,009][105692] Updated weights for policy 0, policy_version 1940463 (0.0009) [2023-12-27 05:32:09,663][105620] Updated weights for policy 1, policy_version 1945176 (0.0011) [2023-12-27 05:32:09,712][105692] Updated weights for policy 0, policy_version 1940473 (0.0008) [2023-12-27 05:32:09,729][105620] Updated weights for policy 1, policy_version 1945186 (0.0010) [2023-12-27 05:32:09,768][105692] Updated weights for policy 0, policy_version 1940483 (0.0008) [2023-12-27 05:32:09,796][105620] Updated weights for policy 1, policy_version 1945196 (0.0011) [2023-12-27 05:32:09,841][105692] Updated weights for policy 0, policy_version 1940493 (0.0006) [2023-12-27 05:32:10,538][105620] Updated weights for policy 1, policy_version 1945206 (0.0008) [2023-12-27 05:32:10,598][105692] Updated weights for policy 0, policy_version 1940503 (0.0010) [2023-12-27 05:32:10,603][105620] Updated weights for policy 1, policy_version 1945216 (0.0006) [2023-12-27 05:32:10,654][105692] Updated weights for policy 0, policy_version 1940513 (0.0011) [2023-12-27 05:32:10,669][105620] Updated weights for policy 1, policy_version 1945226 (0.0010) [2023-12-27 05:32:10,713][105692] Updated weights for policy 0, policy_version 1940523 (0.0011) [2023-12-27 05:32:11,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 994893824. Throughput: 0: 10022.0, 1: 9751.6. Samples: 994901488. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:32:11,063][104569] Avg episode reward: [(0, '8442.487'), (1, '9163.060')] [2023-12-27 05:32:11,365][105620] Updated weights for policy 1, policy_version 1945236 (0.0009) [2023-12-27 05:32:11,433][105620] Updated weights for policy 1, policy_version 1945246 (0.0011) [2023-12-27 05:32:11,469][105692] Updated weights for policy 0, policy_version 1940533 (0.0010) [2023-12-27 05:32:11,485][105620] Updated weights for policy 1, policy_version 1945256 (0.0010) [2023-12-27 05:32:11,528][105692] Updated weights for policy 0, policy_version 1940543 (0.0007) [2023-12-27 05:32:11,592][105692] Updated weights for policy 0, policy_version 1940553 (0.0008) [2023-12-27 05:32:12,209][105620] Updated weights for policy 1, policy_version 1945266 (0.0011) [2023-12-27 05:32:12,269][105620] Updated weights for policy 1, policy_version 1945276 (0.0011) [2023-12-27 05:32:12,328][105620] Updated weights for policy 1, policy_version 1945286 (0.0010) [2023-12-27 05:32:12,382][105692] Updated weights for policy 0, policy_version 1940563 (0.0008) [2023-12-27 05:32:12,384][105620] Updated weights for policy 1, policy_version 1945296 (0.0011) [2023-12-27 05:32:12,445][105692] Updated weights for policy 0, policy_version 1940573 (0.0008) [2023-12-27 05:32:12,500][105692] Updated weights for policy 0, policy_version 1940583 (0.0008) [2023-12-27 05:32:13,137][105620] Updated weights for policy 1, policy_version 1945306 (0.0010) [2023-12-27 05:32:13,185][105620] Updated weights for policy 1, policy_version 1945316 (0.0010) [2023-12-27 05:32:13,250][105620] Updated weights for policy 1, policy_version 1945326 (0.0006) [2023-12-27 05:32:13,264][105692] Updated weights for policy 0, policy_version 1940593 (0.0009) [2023-12-27 05:32:13,320][105692] Updated weights for policy 0, policy_version 1940603 (0.0009) [2023-12-27 05:32:13,381][105692] Updated weights for policy 0, policy_version 1940613 (0.0009) [2023-12-27 05:32:13,442][105692] Updated weights for policy 0, policy_version 1940623 (0.0009) [2023-12-27 05:32:13,857][105620] Updated weights for policy 1, policy_version 1945336 (0.0006) [2023-12-27 05:32:13,935][105620] Updated weights for policy 1, policy_version 1945346 (0.0006) [2023-12-27 05:32:13,993][105620] Updated weights for policy 1, policy_version 1945356 (0.0006) [2023-12-27 05:32:14,151][105692] Updated weights for policy 0, policy_version 1940633 (0.0006) [2023-12-27 05:32:14,209][105692] Updated weights for policy 0, policy_version 1940643 (0.0007) [2023-12-27 05:32:14,263][105692] Updated weights for policy 0, policy_version 1940653 (0.0010) [2023-12-27 05:32:14,537][105620] Updated weights for policy 1, policy_version 1945366 (0.0006) [2023-12-27 05:32:14,584][105620] Updated weights for policy 1, policy_version 1945376 (0.0006) [2023-12-27 05:32:14,637][105620] Updated weights for policy 1, policy_version 1945386 (0.0010) [2023-12-27 05:32:14,950][105692] Updated weights for policy 0, policy_version 1940663 (0.0010) [2023-12-27 05:32:15,022][105692] Updated weights for policy 0, policy_version 1940673 (0.0011) [2023-12-27 05:32:15,084][105692] Updated weights for policy 0, policy_version 1940683 (0.0011) [2023-12-27 05:32:15,324][105620] Updated weights for policy 1, policy_version 1945396 (0.0008) [2023-12-27 05:32:15,382][105620] Updated weights for policy 1, policy_version 1945406 (0.0006) [2023-12-27 05:32:15,443][105620] Updated weights for policy 1, policy_version 1945416 (0.0006) [2023-12-27 05:32:15,730][105692] Updated weights for policy 0, policy_version 1940693 (0.0008) [2023-12-27 05:32:15,784][105692] Updated weights for policy 0, policy_version 1940703 (0.0006) [2023-12-27 05:32:15,852][105692] Updated weights for policy 0, policy_version 1940713 (0.0006) [2023-12-27 05:32:15,982][105620] Updated weights for policy 1, policy_version 1945426 (0.0006) [2023-12-27 05:32:16,027][105620] Updated weights for policy 1, policy_version 1945436 (0.0005) [2023-12-27 05:32:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19660.7, 300 sec: 19466.4). Total num frames: 994992128. Throughput: 0: 9921.2, 1: 9761.0. Samples: 994958144. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:32:16,063][104569] Avg episode reward: [(0, '8441.097'), (1, '9161.497')] [2023-12-27 05:32:16,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001940720_496893952.pth... [2023-12-27 05:32:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001939568_496599040.pth [2023-12-27 05:32:16,076][105620] Updated weights for policy 1, policy_version 1945446 (0.0005) [2023-12-27 05:32:16,123][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001945456_498106368.pth... [2023-12-27 05:32:16,125][105620] Updated weights for policy 1, policy_version 1945456 (0.0005) [2023-12-27 05:32:16,127][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001944272_497803264.pth [2023-12-27 05:32:16,427][105692] Updated weights for policy 0, policy_version 1940723 (0.0007) [2023-12-27 05:32:16,484][105692] Updated weights for policy 0, policy_version 1940733 (0.0009) [2023-12-27 05:32:16,536][105692] Updated weights for policy 0, policy_version 1940743 (0.0006) [2023-12-27 05:32:16,789][105620] Updated weights for policy 1, policy_version 1945466 (0.0010) [2023-12-27 05:32:16,855][105620] Updated weights for policy 1, policy_version 1945476 (0.0009) [2023-12-27 05:32:16,911][105620] Updated weights for policy 1, policy_version 1945486 (0.0009) [2023-12-27 05:32:17,151][105692] Updated weights for policy 0, policy_version 1940753 (0.0006) [2023-12-27 05:32:17,216][105692] Updated weights for policy 0, policy_version 1940763 (0.0005) [2023-12-27 05:32:17,283][105692] Updated weights for policy 0, policy_version 1940773 (0.0005) [2023-12-27 05:32:17,348][105692] Updated weights for policy 0, policy_version 1940783 (0.0005) [2023-12-27 05:32:17,598][105620] Updated weights for policy 1, policy_version 1945496 (0.0006) [2023-12-27 05:32:17,672][105620] Updated weights for policy 1, policy_version 1945506 (0.0009) [2023-12-27 05:32:17,734][105620] Updated weights for policy 1, policy_version 1945516 (0.0008) [2023-12-27 05:32:17,932][105692] Updated weights for policy 0, policy_version 1940793 (0.0009) [2023-12-27 05:32:17,997][105692] Updated weights for policy 0, policy_version 1940803 (0.0009) [2023-12-27 05:32:18,050][105692] Updated weights for policy 0, policy_version 1940813 (0.0010) [2023-12-27 05:32:18,264][105620] Updated weights for policy 1, policy_version 1945526 (0.0005) [2023-12-27 05:32:18,321][105620] Updated weights for policy 1, policy_version 1945536 (0.0006) [2023-12-27 05:32:18,380][105620] Updated weights for policy 1, policy_version 1945546 (0.0010) [2023-12-27 05:32:18,843][105692] Updated weights for policy 0, policy_version 1940823 (0.0009) [2023-12-27 05:32:18,904][105692] Updated weights for policy 0, policy_version 1940833 (0.0008) [2023-12-27 05:32:18,965][105692] Updated weights for policy 0, policy_version 1940843 (0.0008) [2023-12-27 05:32:19,147][105620] Updated weights for policy 1, policy_version 1945556 (0.0011) [2023-12-27 05:32:19,210][105620] Updated weights for policy 1, policy_version 1945566 (0.0006) [2023-12-27 05:32:19,275][105620] Updated weights for policy 1, policy_version 1945576 (0.0008) [2023-12-27 05:32:19,779][105692] Updated weights for policy 0, policy_version 1940853 (0.0008) [2023-12-27 05:32:19,841][105692] Updated weights for policy 0, policy_version 1940863 (0.0008) [2023-12-27 05:32:19,909][105692] Updated weights for policy 0, policy_version 1940873 (0.0007) [2023-12-27 05:32:19,999][105620] Updated weights for policy 1, policy_version 1945586 (0.0006) [2023-12-27 05:32:20,070][105620] Updated weights for policy 1, policy_version 1945596 (0.0008) [2023-12-27 05:32:20,135][105620] Updated weights for policy 1, policy_version 1945606 (0.0008) [2023-12-27 05:32:20,201][105620] Updated weights for policy 1, policy_version 1945616 (0.0007) [2023-12-27 05:32:20,531][105692] Updated weights for policy 0, policy_version 1940883 (0.0006) [2023-12-27 05:32:20,614][105692] Updated weights for policy 0, policy_version 1940893 (0.0007) [2023-12-27 05:32:20,687][105692] Updated weights for policy 0, policy_version 1940903 (0.0008) [2023-12-27 05:32:20,817][105620] Updated weights for policy 1, policy_version 1945626 (0.0007) [2023-12-27 05:32:20,881][105620] Updated weights for policy 1, policy_version 1945636 (0.0007) [2023-12-27 05:32:20,949][105620] Updated weights for policy 1, policy_version 1945646 (0.0006) [2023-12-27 05:32:21,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 995098624. Throughput: 0: 9917.0, 1: 9942.6. Samples: 995083728. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:32:21,063][104569] Avg episode reward: [(0, '8353.671'), (1, '9161.697')] [2023-12-27 05:32:21,268][105692] Updated weights for policy 0, policy_version 1940913 (0.0007) [2023-12-27 05:32:21,334][105692] Updated weights for policy 0, policy_version 1940923 (0.0006) [2023-12-27 05:32:21,404][105692] Updated weights for policy 0, policy_version 1940933 (0.0008) [2023-12-27 05:32:21,467][105692] Updated weights for policy 0, policy_version 1940943 (0.0008) [2023-12-27 05:32:21,593][105620] Updated weights for policy 1, policy_version 1945656 (0.0006) [2023-12-27 05:32:21,661][105620] Updated weights for policy 1, policy_version 1945666 (0.0007) [2023-12-27 05:32:21,725][105620] Updated weights for policy 1, policy_version 1945676 (0.0010) [2023-12-27 05:32:22,252][105692] Updated weights for policy 0, policy_version 1940953 (0.0009) [2023-12-27 05:32:22,318][105692] Updated weights for policy 0, policy_version 1940963 (0.0009) [2023-12-27 05:32:22,383][105692] Updated weights for policy 0, policy_version 1940973 (0.0009) [2023-12-27 05:32:22,420][105620] Updated weights for policy 1, policy_version 1945686 (0.0007) [2023-12-27 05:32:22,483][105620] Updated weights for policy 1, policy_version 1945696 (0.0009) [2023-12-27 05:32:22,535][105620] Updated weights for policy 1, policy_version 1945706 (0.0009) [2023-12-27 05:32:23,071][105692] Updated weights for policy 0, policy_version 1940983 (0.0008) [2023-12-27 05:32:23,131][105692] Updated weights for policy 0, policy_version 1940993 (0.0006) [2023-12-27 05:32:23,189][105692] Updated weights for policy 0, policy_version 1941003 (0.0007) [2023-12-27 05:32:23,319][105620] Updated weights for policy 1, policy_version 1945716 (0.0009) [2023-12-27 05:32:23,377][105620] Updated weights for policy 1, policy_version 1945726 (0.0009) [2023-12-27 05:32:23,443][105620] Updated weights for policy 1, policy_version 1945736 (0.0009) [2023-12-27 05:32:23,817][105692] Updated weights for policy 0, policy_version 1941013 (0.0009) [2023-12-27 05:32:23,876][105692] Updated weights for policy 0, policy_version 1941023 (0.0005) [2023-12-27 05:32:23,935][105692] Updated weights for policy 0, policy_version 1941033 (0.0009) [2023-12-27 05:32:24,190][105620] Updated weights for policy 1, policy_version 1945746 (0.0009) [2023-12-27 05:32:24,249][105620] Updated weights for policy 1, policy_version 1945756 (0.0010) [2023-12-27 05:32:24,306][105620] Updated weights for policy 1, policy_version 1945766 (0.0009) [2023-12-27 05:32:24,358][105620] Updated weights for policy 1, policy_version 1945776 (0.0009) [2023-12-27 05:32:24,575][105692] Updated weights for policy 0, policy_version 1941043 (0.0009) [2023-12-27 05:32:24,632][105692] Updated weights for policy 0, policy_version 1941053 (0.0009) [2023-12-27 05:32:24,689][105692] Updated weights for policy 0, policy_version 1941063 (0.0008) [2023-12-27 05:32:25,114][105620] Updated weights for policy 1, policy_version 1945786 (0.0009) [2023-12-27 05:32:25,176][105620] Updated weights for policy 1, policy_version 1945796 (0.0008) [2023-12-27 05:32:25,237][105620] Updated weights for policy 1, policy_version 1945806 (0.0009) [2023-12-27 05:32:25,421][105692] Updated weights for policy 0, policy_version 1941073 (0.0009) [2023-12-27 05:32:25,485][105692] Updated weights for policy 0, policy_version 1941083 (0.0009) [2023-12-27 05:32:25,542][105692] Updated weights for policy 0, policy_version 1941093 (0.0007) [2023-12-27 05:32:25,591][105692] Updated weights for policy 0, policy_version 1941103 (0.0005) [2023-12-27 05:32:26,009][105620] Updated weights for policy 1, policy_version 1945816 (0.0009) [2023-12-27 05:32:26,060][105620] Updated weights for policy 1, policy_version 1945826 (0.0009) [2023-12-27 05:32:26,062][104569] Fps is (10 sec: 19661.5, 60 sec: 19797.3, 300 sec: 19494.2). Total num frames: 995188736. Throughput: 0: 10013.8, 1: 9852.7. Samples: 995201612. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:32:26,062][104569] Avg episode reward: [(0, '8719.216'), (1, '9253.890')] [2023-12-27 05:32:26,119][105620] Updated weights for policy 1, policy_version 1945836 (0.0009) [2023-12-27 05:32:26,267][105692] Updated weights for policy 0, policy_version 1941113 (0.0008) [2023-12-27 05:32:26,328][105692] Updated weights for policy 0, policy_version 1941123 (0.0009) [2023-12-27 05:32:26,378][105692] Updated weights for policy 0, policy_version 1941133 (0.0009) [2023-12-27 05:32:26,793][105620] Updated weights for policy 1, policy_version 1945846 (0.0009) [2023-12-27 05:32:26,846][105620] Updated weights for policy 1, policy_version 1945856 (0.0009) [2023-12-27 05:32:26,896][105620] Updated weights for policy 1, policy_version 1945866 (0.0008) [2023-12-27 05:32:27,183][105692] Updated weights for policy 0, policy_version 1941143 (0.0010) [2023-12-27 05:32:27,236][105692] Updated weights for policy 0, policy_version 1941153 (0.0010) [2023-12-27 05:32:27,290][105692] Updated weights for policy 0, policy_version 1941163 (0.0010) [2023-12-27 05:32:27,539][105620] Updated weights for policy 1, policy_version 1945876 (0.0009) [2023-12-27 05:32:27,602][105620] Updated weights for policy 1, policy_version 1945886 (0.0010) [2023-12-27 05:32:27,664][105620] Updated weights for policy 1, policy_version 1945896 (0.0009) [2023-12-27 05:32:28,056][105692] Updated weights for policy 0, policy_version 1941173 (0.0009) [2023-12-27 05:32:28,112][105692] Updated weights for policy 0, policy_version 1941183 (0.0009) [2023-12-27 05:32:28,166][105692] Updated weights for policy 0, policy_version 1941193 (0.0009) [2023-12-27 05:32:28,405][105620] Updated weights for policy 1, policy_version 1945906 (0.0008) [2023-12-27 05:32:28,464][105620] Updated weights for policy 1, policy_version 1945916 (0.0005) [2023-12-27 05:32:28,525][105620] Updated weights for policy 1, policy_version 1945926 (0.0007) [2023-12-27 05:32:28,584][105620] Updated weights for policy 1, policy_version 1945936 (0.0009) [2023-12-27 05:32:28,985][105692] Updated weights for policy 0, policy_version 1941203 (0.0008) [2023-12-27 05:32:29,043][105692] Updated weights for policy 0, policy_version 1941213 (0.0009) [2023-12-27 05:32:29,097][105692] Updated weights for policy 0, policy_version 1941223 (0.0009) [2023-12-27 05:32:29,254][105620] Updated weights for policy 1, policy_version 1945946 (0.0009) [2023-12-27 05:32:29,306][105620] Updated weights for policy 1, policy_version 1945956 (0.0009) [2023-12-27 05:32:29,366][105620] Updated weights for policy 1, policy_version 1945966 (0.0009) [2023-12-27 05:32:29,887][105692] Updated weights for policy 0, policy_version 1941233 (0.0009) [2023-12-27 05:32:29,951][105692] Updated weights for policy 0, policy_version 1941243 (0.0009) [2023-12-27 05:32:30,016][105692] Updated weights for policy 0, policy_version 1941253 (0.0009) [2023-12-27 05:32:30,074][105692] Updated weights for policy 0, policy_version 1941263 (0.0009) [2023-12-27 05:32:30,150][105620] Updated weights for policy 1, policy_version 1945977 (0.0010) [2023-12-27 05:32:30,207][105620] Updated weights for policy 1, policy_version 1945987 (0.0009) [2023-12-27 05:32:30,258][105620] Updated weights for policy 1, policy_version 1945997 (0.0009) [2023-12-27 05:32:30,758][105692] Updated weights for policy 0, policy_version 1941273 (0.0008) [2023-12-27 05:32:30,803][105692] Updated weights for policy 0, policy_version 1941283 (0.0008) [2023-12-27 05:32:30,852][105692] Updated weights for policy 0, policy_version 1941293 (0.0008) [2023-12-27 05:32:31,048][105620] Updated weights for policy 1, policy_version 1946007 (0.0009) [2023-12-27 05:32:31,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19797.4, 300 sec: 19466.4). Total num frames: 995287040. Throughput: 0: 9993.1, 1: 9884.9. Samples: 995259244. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:32:31,062][104569] Avg episode reward: [(0, '8717.067'), (1, '9165.095')] [2023-12-27 05:32:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001941296_497041408.pth... [2023-12-27 05:32:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001940144_496746496.pth [2023-12-27 05:32:31,101][105620] Updated weights for policy 1, policy_version 1946017 (0.0010) [2023-12-27 05:32:31,162][105620] Updated weights for policy 1, policy_version 1946027 (0.0009) [2023-12-27 05:32:31,188][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001946032_498253824.pth... [2023-12-27 05:32:31,193][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001944848_497950720.pth [2023-12-27 05:32:31,628][105692] Updated weights for policy 0, policy_version 1941303 (0.0009) [2023-12-27 05:32:31,686][105692] Updated weights for policy 0, policy_version 1941313 (0.0009) [2023-12-27 05:32:31,747][105692] Updated weights for policy 0, policy_version 1941323 (0.0009) [2023-12-27 05:32:31,921][105620] Updated weights for policy 1, policy_version 1946037 (0.0009) [2023-12-27 05:32:31,976][105620] Updated weights for policy 1, policy_version 1946047 (0.0009) [2023-12-27 05:32:32,031][105620] Updated weights for policy 1, policy_version 1946057 (0.0008) [2023-12-27 05:32:32,489][105692] Updated weights for policy 0, policy_version 1941333 (0.0009) [2023-12-27 05:32:32,556][105692] Updated weights for policy 0, policy_version 1941343 (0.0009) [2023-12-27 05:32:32,609][105692] Updated weights for policy 0, policy_version 1941353 (0.0009) [2023-12-27 05:32:32,816][105620] Updated weights for policy 1, policy_version 1946067 (0.0009) [2023-12-27 05:32:32,878][105620] Updated weights for policy 1, policy_version 1946077 (0.0009) [2023-12-27 05:32:32,937][105620] Updated weights for policy 1, policy_version 1946087 (0.0009) [2023-12-27 05:32:33,345][105692] Updated weights for policy 0, policy_version 1941363 (0.0009) [2023-12-27 05:32:33,402][105692] Updated weights for policy 0, policy_version 1941373 (0.0009) [2023-12-27 05:32:33,459][105692] Updated weights for policy 0, policy_version 1941383 (0.0009) [2023-12-27 05:32:33,679][105620] Updated weights for policy 1, policy_version 1946097 (0.0009) [2023-12-27 05:32:33,728][105620] Updated weights for policy 1, policy_version 1946107 (0.0007) [2023-12-27 05:32:33,770][105620] Updated weights for policy 1, policy_version 1946117 (0.0005) [2023-12-27 05:32:33,822][105620] Updated weights for policy 1, policy_version 1946127 (0.0008) [2023-12-27 05:32:34,211][105692] Updated weights for policy 0, policy_version 1941393 (0.0009) [2023-12-27 05:32:34,277][105692] Updated weights for policy 0, policy_version 1941403 (0.0006) [2023-12-27 05:32:34,341][105692] Updated weights for policy 0, policy_version 1941413 (0.0009) [2023-12-27 05:32:34,412][105692] Updated weights for policy 0, policy_version 1941423 (0.0010) [2023-12-27 05:32:34,550][105620] Updated weights for policy 1, policy_version 1946137 (0.0009) [2023-12-27 05:32:34,608][105620] Updated weights for policy 1, policy_version 1946147 (0.0007) [2023-12-27 05:32:34,670][105620] Updated weights for policy 1, policy_version 1946157 (0.0008) [2023-12-27 05:32:35,183][105692] Updated weights for policy 0, policy_version 1941433 (0.0009) [2023-12-27 05:32:35,245][105692] Updated weights for policy 0, policy_version 1941444 (0.0010) [2023-12-27 05:32:35,295][105692] Updated weights for policy 0, policy_version 1941454 (0.0007) [2023-12-27 05:32:35,297][105620] Updated weights for policy 1, policy_version 1946167 (0.0007) [2023-12-27 05:32:35,346][105620] Updated weights for policy 1, policy_version 1946177 (0.0008) [2023-12-27 05:32:35,396][105620] Updated weights for policy 1, policy_version 1946187 (0.0010) [2023-12-27 05:32:36,038][105620] Updated weights for policy 1, policy_version 1946197 (0.0008) [2023-12-27 05:32:36,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 995377152. Throughput: 0: 9823.1, 1: 9908.6. Samples: 995371192. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:32:36,063][104569] Avg episode reward: [(0, '8534.530'), (1, '8980.972')] [2023-12-27 05:32:36,100][105620] Updated weights for policy 1, policy_version 1946207 (0.0009) [2023-12-27 05:32:36,133][105692] Updated weights for policy 0, policy_version 1941464 (0.0006) [2023-12-27 05:32:36,151][105620] Updated weights for policy 1, policy_version 1946217 (0.0008) [2023-12-27 05:32:36,221][105692] Updated weights for policy 0, policy_version 1941474 (0.0008) [2023-12-27 05:32:36,289][105692] Updated weights for policy 0, policy_version 1941484 (0.0009) [2023-12-27 05:32:36,746][105620] Updated weights for policy 1, policy_version 1946227 (0.0007) [2023-12-27 05:32:36,798][105620] Updated weights for policy 1, policy_version 1946237 (0.0007) [2023-12-27 05:32:36,850][105620] Updated weights for policy 1, policy_version 1946247 (0.0005) [2023-12-27 05:32:37,101][105692] Updated weights for policy 0, policy_version 1941494 (0.0009) [2023-12-27 05:32:37,168][105692] Updated weights for policy 0, policy_version 1941504 (0.0010) [2023-12-27 05:32:37,231][105692] Updated weights for policy 0, policy_version 1941514 (0.0009) [2023-12-27 05:32:37,483][105620] Updated weights for policy 1, policy_version 1946257 (0.0008) [2023-12-27 05:32:37,534][105620] Updated weights for policy 1, policy_version 1946267 (0.0009) [2023-12-27 05:32:37,591][105620] Updated weights for policy 1, policy_version 1946277 (0.0009) [2023-12-27 05:32:37,651][105620] Updated weights for policy 1, policy_version 1946287 (0.0008) [2023-12-27 05:32:38,036][105692] Updated weights for policy 0, policy_version 1941524 (0.0009) [2023-12-27 05:32:38,091][105692] Updated weights for policy 0, policy_version 1941534 (0.0009) [2023-12-27 05:32:38,143][105692] Updated weights for policy 0, policy_version 1941544 (0.0009) [2023-12-27 05:32:38,336][105620] Updated weights for policy 1, policy_version 1946297 (0.0008) [2023-12-27 05:32:38,400][105620] Updated weights for policy 1, policy_version 1946307 (0.0009) [2023-12-27 05:32:38,455][105620] Updated weights for policy 1, policy_version 1946317 (0.0005) [2023-12-27 05:32:38,985][105692] Updated weights for policy 0, policy_version 1941554 (0.0008) [2023-12-27 05:32:39,042][105692] Updated weights for policy 0, policy_version 1941564 (0.0008) [2023-12-27 05:32:39,099][105692] Updated weights for policy 0, policy_version 1941574 (0.0008) [2023-12-27 05:32:39,103][105620] Updated weights for policy 1, policy_version 1946327 (0.0005) [2023-12-27 05:32:39,168][105692] Updated weights for policy 0, policy_version 1941584 (0.0009) [2023-12-27 05:32:39,169][105620] Updated weights for policy 1, policy_version 1946337 (0.0006) [2023-12-27 05:32:39,231][105620] Updated weights for policy 1, policy_version 1946347 (0.0006) [2023-12-27 05:32:39,852][105620] Updated weights for policy 1, policy_version 1946357 (0.0008) [2023-12-27 05:32:39,913][105620] Updated weights for policy 1, policy_version 1946367 (0.0009) [2023-12-27 05:32:39,983][105620] Updated weights for policy 1, policy_version 1946377 (0.0008) [2023-12-27 05:32:40,023][105692] Updated weights for policy 0, policy_version 1941594 (0.0008) [2023-12-27 05:32:40,091][105692] Updated weights for policy 0, policy_version 1941604 (0.0008) [2023-12-27 05:32:40,149][105692] Updated weights for policy 0, policy_version 1941614 (0.0007) [2023-12-27 05:32:40,684][105620] Updated weights for policy 1, policy_version 1946387 (0.0006) [2023-12-27 05:32:40,737][105620] Updated weights for policy 1, policy_version 1946397 (0.0006) [2023-12-27 05:32:40,791][105620] Updated weights for policy 1, policy_version 1946407 (0.0008) [2023-12-27 05:32:40,969][105692] Updated weights for policy 0, policy_version 1941624 (0.0009) [2023-12-27 05:32:41,029][105692] Updated weights for policy 0, policy_version 1941634 (0.0009) [2023-12-27 05:32:41,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 995475456. Throughput: 0: 9566.8, 1: 10033.5. Samples: 995486468. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:32:41,063][104569] Avg episode reward: [(0, '8535.457'), (1, '9161.950')] [2023-12-27 05:32:41,096][105692] Updated weights for policy 0, policy_version 1941644 (0.0009) [2023-12-27 05:32:41,526][105620] Updated weights for policy 1, policy_version 1946417 (0.0009) [2023-12-27 05:32:41,595][105620] Updated weights for policy 1, policy_version 1946427 (0.0005) [2023-12-27 05:32:41,670][105620] Updated weights for policy 1, policy_version 1946437 (0.0008) [2023-12-27 05:32:41,734][105620] Updated weights for policy 1, policy_version 1946447 (0.0008) [2023-12-27 05:32:41,886][105692] Updated weights for policy 0, policy_version 1941654 (0.0009) [2023-12-27 05:32:41,943][105692] Updated weights for policy 0, policy_version 1941664 (0.0010) [2023-12-27 05:32:41,991][105692] Updated weights for policy 0, policy_version 1941674 (0.0009) [2023-12-27 05:32:42,393][105620] Updated weights for policy 1, policy_version 1946457 (0.0009) [2023-12-27 05:32:42,463][105620] Updated weights for policy 1, policy_version 1946467 (0.0008) [2023-12-27 05:32:42,523][105620] Updated weights for policy 1, policy_version 1946477 (0.0008) [2023-12-27 05:32:42,806][105692] Updated weights for policy 0, policy_version 1941684 (0.0009) [2023-12-27 05:32:42,862][105692] Updated weights for policy 0, policy_version 1941694 (0.0008) [2023-12-27 05:32:42,917][105692] Updated weights for policy 0, policy_version 1941704 (0.0009) [2023-12-27 05:32:43,251][105620] Updated weights for policy 1, policy_version 1946487 (0.0007) [2023-12-27 05:32:43,308][105620] Updated weights for policy 1, policy_version 1946497 (0.0008) [2023-12-27 05:32:43,364][105620] Updated weights for policy 1, policy_version 1946507 (0.0008) [2023-12-27 05:32:43,771][105692] Updated weights for policy 0, policy_version 1941714 (0.0009) [2023-12-27 05:32:43,826][105692] Updated weights for policy 0, policy_version 1941725 (0.0010) [2023-12-27 05:32:43,885][105692] Updated weights for policy 0, policy_version 1941737 (0.0011) [2023-12-27 05:32:43,911][105620] Updated weights for policy 1, policy_version 1946517 (0.0007) [2023-12-27 05:32:43,963][105620] Updated weights for policy 1, policy_version 1946527 (0.0006) [2023-12-27 05:32:44,023][105620] Updated weights for policy 1, policy_version 1946537 (0.0007) [2023-12-27 05:32:44,654][105692] Updated weights for policy 0, policy_version 1941747 (0.0009) [2023-12-27 05:32:44,707][105620] Updated weights for policy 1, policy_version 1946547 (0.0009) [2023-12-27 05:32:44,713][105692] Updated weights for policy 0, policy_version 1941757 (0.0008) [2023-12-27 05:32:44,764][105620] Updated weights for policy 1, policy_version 1946557 (0.0006) [2023-12-27 05:32:44,786][105692] Updated weights for policy 0, policy_version 1941767 (0.0008) [2023-12-27 05:32:44,826][105620] Updated weights for policy 1, policy_version 1946567 (0.0006) [2023-12-27 05:32:45,468][105620] Updated weights for policy 1, policy_version 1946577 (0.0007) [2023-12-27 05:32:45,536][105620] Updated weights for policy 1, policy_version 1946587 (0.0008) [2023-12-27 05:32:45,585][105692] Updated weights for policy 0, policy_version 1941777 (0.0008) [2023-12-27 05:32:45,593][105620] Updated weights for policy 1, policy_version 1946597 (0.0008) [2023-12-27 05:32:45,649][105692] Updated weights for policy 0, policy_version 1941787 (0.0008) [2023-12-27 05:32:45,654][105620] Updated weights for policy 1, policy_version 1946607 (0.0008) [2023-12-27 05:32:45,707][105692] Updated weights for policy 0, policy_version 1941797 (0.0009) [2023-12-27 05:32:45,762][105692] Updated weights for policy 0, policy_version 1941807 (0.0009) [2023-12-27 05:32:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 995573760. Throughput: 0: 9537.7, 1: 10030.6. Samples: 995543288. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:32:46,063][104569] Avg episode reward: [(0, '8720.352'), (1, '9253.872')] [2023-12-27 05:32:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001941808_497172480.pth... [2023-12-27 05:32:46,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001946608_498401280.pth... [2023-12-27 05:32:46,077][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001940720_496893952.pth [2023-12-27 05:32:46,077][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001945456_498106368.pth [2023-12-27 05:32:46,415][105620] Updated weights for policy 1, policy_version 1946617 (0.0010) [2023-12-27 05:32:46,466][105692] Updated weights for policy 0, policy_version 1941817 (0.0007) [2023-12-27 05:32:46,469][105620] Updated weights for policy 1, policy_version 1946627 (0.0006) [2023-12-27 05:32:46,515][105692] Updated weights for policy 0, policy_version 1941827 (0.0006) [2023-12-27 05:32:46,523][105620] Updated weights for policy 1, policy_version 1946637 (0.0009) [2023-12-27 05:32:46,563][105692] Updated weights for policy 0, policy_version 1941837 (0.0005) [2023-12-27 05:32:47,105][105692] Updated weights for policy 0, policy_version 1941847 (0.0005) [2023-12-27 05:32:47,152][105692] Updated weights for policy 0, policy_version 1941857 (0.0007) [2023-12-27 05:32:47,203][105692] Updated weights for policy 0, policy_version 1941867 (0.0010) [2023-12-27 05:32:47,384][105620] Updated weights for policy 1, policy_version 1946647 (0.0008) [2023-12-27 05:32:47,446][105620] Updated weights for policy 1, policy_version 1946657 (0.0010) [2023-12-27 05:32:47,501][105620] Updated weights for policy 1, policy_version 1946667 (0.0005) [2023-12-27 05:32:47,937][105692] Updated weights for policy 0, policy_version 1941877 (0.0009) [2023-12-27 05:32:47,999][105692] Updated weights for policy 0, policy_version 1941887 (0.0011) [2023-12-27 05:32:48,025][105620] Updated weights for policy 1, policy_version 1946677 (0.0008) [2023-12-27 05:32:48,055][105692] Updated weights for policy 0, policy_version 1941897 (0.0006) [2023-12-27 05:32:48,074][105620] Updated weights for policy 1, policy_version 1946687 (0.0011) [2023-12-27 05:32:48,122][105620] Updated weights for policy 1, policy_version 1946697 (0.0010) [2023-12-27 05:32:48,776][105692] Updated weights for policy 0, policy_version 1941907 (0.0009) [2023-12-27 05:32:48,840][105692] Updated weights for policy 0, policy_version 1941917 (0.0011) [2023-12-27 05:32:48,906][105692] Updated weights for policy 0, policy_version 1941927 (0.0010) [2023-12-27 05:32:48,924][105620] Updated weights for policy 1, policy_version 1946707 (0.0009) [2023-12-27 05:32:48,980][105620] Updated weights for policy 1, policy_version 1946717 (0.0009) [2023-12-27 05:32:49,032][105620] Updated weights for policy 1, policy_version 1946727 (0.0008) [2023-12-27 05:32:49,669][105692] Updated weights for policy 0, policy_version 1941937 (0.0010) [2023-12-27 05:32:49,718][105692] Updated weights for policy 0, policy_version 1941947 (0.0009) [2023-12-27 05:32:49,769][105692] Updated weights for policy 0, policy_version 1941957 (0.0009) [2023-12-27 05:32:49,772][105620] Updated weights for policy 1, policy_version 1946737 (0.0008) [2023-12-27 05:32:49,826][105692] Updated weights for policy 0, policy_version 1941967 (0.0006) [2023-12-27 05:32:49,828][105620] Updated weights for policy 1, policy_version 1946747 (0.0007) [2023-12-27 05:32:49,899][105620] Updated weights for policy 1, policy_version 1946757 (0.0007) [2023-12-27 05:32:49,964][105620] Updated weights for policy 1, policy_version 1946767 (0.0009) [2023-12-27 05:32:50,538][105692] Updated weights for policy 0, policy_version 1941977 (0.0006) [2023-12-27 05:32:50,604][105692] Updated weights for policy 0, policy_version 1941987 (0.0008) [2023-12-27 05:32:50,655][105692] Updated weights for policy 0, policy_version 1941997 (0.0009) [2023-12-27 05:32:50,755][105620] Updated weights for policy 1, policy_version 1946777 (0.0009) [2023-12-27 05:32:50,816][105620] Updated weights for policy 1, policy_version 1946787 (0.0009) [2023-12-27 05:32:50,883][105620] Updated weights for policy 1, policy_version 1946797 (0.0006) [2023-12-27 05:32:51,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 995672064. Throughput: 0: 9511.1, 1: 10013.6. Samples: 995659812. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:32:51,062][104569] Avg episode reward: [(0, '8358.117'), (1, '9253.729')] [2023-12-27 05:32:51,496][105620] Updated weights for policy 1, policy_version 1946807 (0.0008) [2023-12-27 05:32:51,504][105692] Updated weights for policy 0, policy_version 1942007 (0.0008) [2023-12-27 05:32:51,558][105620] Updated weights for policy 1, policy_version 1946817 (0.0011) [2023-12-27 05:32:51,567][105692] Updated weights for policy 0, policy_version 1942017 (0.0006) [2023-12-27 05:32:51,624][105620] Updated weights for policy 1, policy_version 1946827 (0.0010) [2023-12-27 05:32:51,631][105692] Updated weights for policy 0, policy_version 1942027 (0.0007) [2023-12-27 05:32:52,229][105620] Updated weights for policy 1, policy_version 1946837 (0.0008) [2023-12-27 05:32:52,292][105620] Updated weights for policy 1, policy_version 1946847 (0.0006) [2023-12-27 05:32:52,326][105692] Updated weights for policy 0, policy_version 1942037 (0.0008) [2023-12-27 05:32:52,350][105620] Updated weights for policy 1, policy_version 1946857 (0.0006) [2023-12-27 05:32:52,384][105692] Updated weights for policy 0, policy_version 1942047 (0.0008) [2023-12-27 05:32:52,451][105692] Updated weights for policy 0, policy_version 1942057 (0.0007) [2023-12-27 05:32:52,941][105620] Updated weights for policy 1, policy_version 1946867 (0.0009) [2023-12-27 05:32:53,008][105620] Updated weights for policy 1, policy_version 1946877 (0.0006) [2023-12-27 05:32:53,062][105620] Updated weights for policy 1, policy_version 1946887 (0.0006) [2023-12-27 05:32:53,171][105692] Updated weights for policy 0, policy_version 1942067 (0.0010) [2023-12-27 05:32:53,236][105692] Updated weights for policy 0, policy_version 1942077 (0.0010) [2023-12-27 05:32:53,303][105692] Updated weights for policy 0, policy_version 1942087 (0.0009) [2023-12-27 05:32:53,649][105620] Updated weights for policy 1, policy_version 1946897 (0.0005) [2023-12-27 05:32:53,703][105620] Updated weights for policy 1, policy_version 1946907 (0.0005) [2023-12-27 05:32:53,760][105620] Updated weights for policy 1, policy_version 1946917 (0.0005) [2023-12-27 05:32:53,820][105620] Updated weights for policy 1, policy_version 1946927 (0.0005) [2023-12-27 05:32:54,029][105692] Updated weights for policy 0, policy_version 1942097 (0.0009) [2023-12-27 05:32:54,083][105692] Updated weights for policy 0, policy_version 1942107 (0.0010) [2023-12-27 05:32:54,139][105692] Updated weights for policy 0, policy_version 1942118 (0.0010) [2023-12-27 05:32:54,188][105692] Updated weights for policy 0, policy_version 1942128 (0.0009) [2023-12-27 05:32:54,443][105620] Updated weights for policy 1, policy_version 1946937 (0.0008) [2023-12-27 05:32:54,492][105620] Updated weights for policy 1, policy_version 1946947 (0.0008) [2023-12-27 05:32:54,542][105620] Updated weights for policy 1, policy_version 1946957 (0.0008) [2023-12-27 05:32:54,946][105692] Updated weights for policy 0, policy_version 1942138 (0.0005) [2023-12-27 05:32:55,007][105692] Updated weights for policy 0, policy_version 1942148 (0.0006) [2023-12-27 05:32:55,070][105692] Updated weights for policy 0, policy_version 1942158 (0.0005) [2023-12-27 05:32:55,364][105620] Updated weights for policy 1, policy_version 1946967 (0.0009) [2023-12-27 05:32:55,418][105620] Updated weights for policy 1, policy_version 1946979 (0.0010) [2023-12-27 05:32:55,481][105620] Updated weights for policy 1, policy_version 1946989 (0.0010) [2023-12-27 05:32:55,570][105692] Updated weights for policy 0, policy_version 1942168 (0.0005) [2023-12-27 05:32:55,616][105692] Updated weights for policy 0, policy_version 1942178 (0.0010) [2023-12-27 05:32:55,680][105692] Updated weights for policy 0, policy_version 1942188 (0.0010) [2023-12-27 05:32:56,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 995770368. Throughput: 0: 9483.8, 1: 10034.0. Samples: 995779792. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:32:56,063][104569] Avg episode reward: [(0, '8537.027'), (1, '9161.442')] [2023-12-27 05:32:56,294][105620] Updated weights for policy 1, policy_version 1946999 (0.0008) [2023-12-27 05:32:56,346][105620] Updated weights for policy 1, policy_version 1947009 (0.0008) [2023-12-27 05:32:56,396][105620] Updated weights for policy 1, policy_version 1947019 (0.0007) [2023-12-27 05:32:56,406][105692] Updated weights for policy 0, policy_version 1942198 (0.0010) [2023-12-27 05:32:56,457][105692] Updated weights for policy 0, policy_version 1942208 (0.0010) [2023-12-27 05:32:56,518][105692] Updated weights for policy 0, policy_version 1942218 (0.0010) [2023-12-27 05:32:57,073][105620] Updated weights for policy 1, policy_version 1947029 (0.0008) [2023-12-27 05:32:57,129][105620] Updated weights for policy 1, policy_version 1947039 (0.0008) [2023-12-27 05:32:57,177][105692] Updated weights for policy 0, policy_version 1942228 (0.0008) [2023-12-27 05:32:57,180][105620] Updated weights for policy 1, policy_version 1947049 (0.0007) [2023-12-27 05:32:57,240][105692] Updated weights for policy 0, policy_version 1942238 (0.0005) [2023-12-27 05:32:57,293][105692] Updated weights for policy 0, policy_version 1942248 (0.0005) [2023-12-27 05:32:57,789][105620] Updated weights for policy 1, policy_version 1947059 (0.0006) [2023-12-27 05:32:57,837][105620] Updated weights for policy 1, policy_version 1947069 (0.0006) [2023-12-27 05:32:57,852][105692] Updated weights for policy 0, policy_version 1942258 (0.0007) [2023-12-27 05:32:57,906][105620] Updated weights for policy 1, policy_version 1947079 (0.0006) [2023-12-27 05:32:57,909][105692] Updated weights for policy 0, policy_version 1942268 (0.0008) [2023-12-27 05:32:57,963][105692] Updated weights for policy 0, policy_version 1942278 (0.0010) [2023-12-27 05:32:58,011][105692] Updated weights for policy 0, policy_version 1942288 (0.0010) [2023-12-27 05:32:58,639][105620] Updated weights for policy 1, policy_version 1947089 (0.0006) [2023-12-27 05:32:58,704][105620] Updated weights for policy 1, policy_version 1947099 (0.0009) [2023-12-27 05:32:58,776][105620] Updated weights for policy 1, policy_version 1947109 (0.0009) [2023-12-27 05:32:58,839][105620] Updated weights for policy 1, policy_version 1947119 (0.0009) [2023-12-27 05:32:58,889][105692] Updated weights for policy 0, policy_version 1942298 (0.0008) [2023-12-27 05:32:58,959][105692] Updated weights for policy 0, policy_version 1942308 (0.0010) [2023-12-27 05:32:59,023][105692] Updated weights for policy 0, policy_version 1942318 (0.0010) [2023-12-27 05:32:59,665][105620] Updated weights for policy 1, policy_version 1947129 (0.0006) [2023-12-27 05:32:59,731][105620] Updated weights for policy 1, policy_version 1947139 (0.0005) [2023-12-27 05:32:59,785][105692] Updated weights for policy 0, policy_version 1942328 (0.0008) [2023-12-27 05:32:59,794][105620] Updated weights for policy 1, policy_version 1947149 (0.0007) [2023-12-27 05:32:59,851][105692] Updated weights for policy 0, policy_version 1942338 (0.0007) [2023-12-27 05:32:59,921][105692] Updated weights for policy 0, policy_version 1942348 (0.0009) [2023-12-27 05:33:00,457][105620] Updated weights for policy 1, policy_version 1947159 (0.0010) [2023-12-27 05:33:00,513][105620] Updated weights for policy 1, policy_version 1947169 (0.0008) [2023-12-27 05:33:00,573][105620] Updated weights for policy 1, policy_version 1947179 (0.0005) [2023-12-27 05:33:00,611][105692] Updated weights for policy 0, policy_version 1942358 (0.0008) [2023-12-27 05:33:00,671][105692] Updated weights for policy 0, policy_version 1942368 (0.0008) [2023-12-27 05:33:00,728][105692] Updated weights for policy 0, policy_version 1942378 (0.0006) [2023-12-27 05:33:01,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 995868672. Throughput: 0: 9552.1, 1: 10028.8. Samples: 995839276. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:33:01,062][104569] Avg episode reward: [(0, '8353.394'), (1, '9069.251')] [2023-12-27 05:33:01,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001942384_497319936.pth... [2023-12-27 05:33:01,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001947184_498548736.pth... [2023-12-27 05:33:01,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001946032_498253824.pth [2023-12-27 05:33:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001941296_497041408.pth [2023-12-27 05:33:01,259][105620] Updated weights for policy 1, policy_version 1947189 (0.0009) [2023-12-27 05:33:01,313][105620] Updated weights for policy 1, policy_version 1947199 (0.0011) [2023-12-27 05:33:01,375][105620] Updated weights for policy 1, policy_version 1947209 (0.0010) [2023-12-27 05:33:01,449][105692] Updated weights for policy 0, policy_version 1942388 (0.0006) [2023-12-27 05:33:01,492][105692] Updated weights for policy 0, policy_version 1942398 (0.0005) [2023-12-27 05:33:01,538][105692] Updated weights for policy 0, policy_version 1942408 (0.0005) [2023-12-27 05:33:02,189][105620] Updated weights for policy 1, policy_version 1947219 (0.0010) [2023-12-27 05:33:02,246][105620] Updated weights for policy 1, policy_version 1947229 (0.0010) [2023-12-27 05:33:02,247][105692] Updated weights for policy 0, policy_version 1942418 (0.0006) [2023-12-27 05:33:02,307][105620] Updated weights for policy 1, policy_version 1947239 (0.0006) [2023-12-27 05:33:02,309][105692] Updated weights for policy 0, policy_version 1942428 (0.0011) [2023-12-27 05:33:02,370][105692] Updated weights for policy 0, policy_version 1942438 (0.0011) [2023-12-27 05:33:02,425][105692] Updated weights for policy 0, policy_version 1942448 (0.0010) [2023-12-27 05:33:03,023][105692] Updated weights for policy 0, policy_version 1942458 (0.0005) [2023-12-27 05:33:03,031][105620] Updated weights for policy 1, policy_version 1947249 (0.0008) [2023-12-27 05:33:03,074][105692] Updated weights for policy 0, policy_version 1942468 (0.0005) [2023-12-27 05:33:03,083][105620] Updated weights for policy 1, policy_version 1947259 (0.0009) [2023-12-27 05:33:03,123][105692] Updated weights for policy 0, policy_version 1942478 (0.0005) [2023-12-27 05:33:03,132][105620] Updated weights for policy 1, policy_version 1947269 (0.0009) [2023-12-27 05:33:03,193][105620] Updated weights for policy 1, policy_version 1947280 (0.0010) [2023-12-27 05:33:03,701][105692] Updated weights for policy 0, policy_version 1942488 (0.0009) [2023-12-27 05:33:03,764][105692] Updated weights for policy 0, policy_version 1942498 (0.0010) [2023-12-27 05:33:03,820][105692] Updated weights for policy 0, policy_version 1942508 (0.0008) [2023-12-27 05:33:03,882][105620] Updated weights for policy 1, policy_version 1947290 (0.0007) [2023-12-27 05:33:03,936][105620] Updated weights for policy 1, policy_version 1947300 (0.0006) [2023-12-27 05:33:03,998][105620] Updated weights for policy 1, policy_version 1947310 (0.0008) [2023-12-27 05:33:04,567][105692] Updated weights for policy 0, policy_version 1942518 (0.0010) [2023-12-27 05:33:04,608][105620] Updated weights for policy 1, policy_version 1947320 (0.0008) [2023-12-27 05:33:04,623][105692] Updated weights for policy 0, policy_version 1942528 (0.0010) [2023-12-27 05:33:04,668][105620] Updated weights for policy 1, policy_version 1947330 (0.0009) [2023-12-27 05:33:04,677][105692] Updated weights for policy 0, policy_version 1942538 (0.0008) [2023-12-27 05:33:04,726][105620] Updated weights for policy 1, policy_version 1947340 (0.0005) [2023-12-27 05:33:05,353][105692] Updated weights for policy 0, policy_version 1942548 (0.0007) [2023-12-27 05:33:05,369][105620] Updated weights for policy 1, policy_version 1947350 (0.0006) [2023-12-27 05:33:05,408][105692] Updated weights for policy 0, policy_version 1942558 (0.0006) [2023-12-27 05:33:05,423][105620] Updated weights for policy 1, policy_version 1947360 (0.0005) [2023-12-27 05:33:05,468][105692] Updated weights for policy 0, policy_version 1942568 (0.0006) [2023-12-27 05:33:05,488][105620] Updated weights for policy 1, policy_version 1947370 (0.0006) [2023-12-27 05:33:06,050][105692] Updated weights for policy 0, policy_version 1942578 (0.0006) [2023-12-27 05:33:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 995966976. Throughput: 0: 9504.5, 1: 9935.8. Samples: 995958544. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:33:06,063][104569] Avg episode reward: [(0, '8171.114'), (1, '9161.479')] [2023-12-27 05:33:06,099][105692] Updated weights for policy 0, policy_version 1942588 (0.0010) [2023-12-27 05:33:06,166][105692] Updated weights for policy 0, policy_version 1942598 (0.0011) [2023-12-27 05:33:06,209][105620] Updated weights for policy 1, policy_version 1947380 (0.0007) [2023-12-27 05:33:06,227][105692] Updated weights for policy 0, policy_version 1942608 (0.0011) [2023-12-27 05:33:06,277][105620] Updated weights for policy 1, policy_version 1947390 (0.0007) [2023-12-27 05:33:06,341][105620] Updated weights for policy 1, policy_version 1947400 (0.0007) [2023-12-27 05:33:06,894][105692] Updated weights for policy 0, policy_version 1942618 (0.0006) [2023-12-27 05:33:06,942][105620] Updated weights for policy 1, policy_version 1947410 (0.0007) [2023-12-27 05:33:06,954][105692] Updated weights for policy 0, policy_version 1942628 (0.0007) [2023-12-27 05:33:07,004][105620] Updated weights for policy 1, policy_version 1947420 (0.0008) [2023-12-27 05:33:07,006][105692] Updated weights for policy 0, policy_version 1942638 (0.0006) [2023-12-27 05:33:07,062][105620] Updated weights for policy 1, policy_version 1947430 (0.0008) [2023-12-27 05:33:07,120][105620] Updated weights for policy 1, policy_version 1947440 (0.0009) [2023-12-27 05:33:07,700][105692] Updated weights for policy 0, policy_version 1942648 (0.0006) [2023-12-27 05:33:07,761][105620] Updated weights for policy 1, policy_version 1947450 (0.0011) [2023-12-27 05:33:07,766][105692] Updated weights for policy 0, policy_version 1942658 (0.0005) [2023-12-27 05:33:07,816][105620] Updated weights for policy 1, policy_version 1947460 (0.0007) [2023-12-27 05:33:07,831][105692] Updated weights for policy 0, policy_version 1942668 (0.0006) [2023-12-27 05:33:07,867][105620] Updated weights for policy 1, policy_version 1947470 (0.0006) [2023-12-27 05:33:08,441][105620] Updated weights for policy 1, policy_version 1947480 (0.0007) [2023-12-27 05:33:08,503][105620] Updated weights for policy 1, policy_version 1947490 (0.0005) [2023-12-27 05:33:08,564][105692] Updated weights for policy 0, policy_version 1942678 (0.0006) [2023-12-27 05:33:08,567][105620] Updated weights for policy 1, policy_version 1947500 (0.0006) [2023-12-27 05:33:08,624][105692] Updated weights for policy 0, policy_version 1942688 (0.0008) [2023-12-27 05:33:08,684][105692] Updated weights for policy 0, policy_version 1942698 (0.0009) [2023-12-27 05:33:09,185][105620] Updated weights for policy 1, policy_version 1947510 (0.0009) [2023-12-27 05:33:09,252][105620] Updated weights for policy 1, policy_version 1947520 (0.0009) [2023-12-27 05:33:09,313][105620] Updated weights for policy 1, policy_version 1947530 (0.0008) [2023-12-27 05:33:09,425][105692] Updated weights for policy 0, policy_version 1942708 (0.0008) [2023-12-27 05:33:09,489][105692] Updated weights for policy 0, policy_version 1942718 (0.0009) [2023-12-27 05:33:09,548][105692] Updated weights for policy 0, policy_version 1942728 (0.0010) [2023-12-27 05:33:09,977][105620] Updated weights for policy 1, policy_version 1947540 (0.0008) [2023-12-27 05:33:10,042][105620] Updated weights for policy 1, policy_version 1947550 (0.0008) [2023-12-27 05:33:10,104][105620] Updated weights for policy 1, policy_version 1947560 (0.0009) [2023-12-27 05:33:10,382][105692] Updated weights for policy 0, policy_version 1942738 (0.0009) [2023-12-27 05:33:10,432][105692] Updated weights for policy 0, policy_version 1942748 (0.0010) [2023-12-27 05:33:10,480][105692] Updated weights for policy 0, policy_version 1942758 (0.0010) [2023-12-27 05:33:10,529][105692] Updated weights for policy 0, policy_version 1942768 (0.0010) [2023-12-27 05:33:10,865][105620] Updated weights for policy 1, policy_version 1947570 (0.0007) [2023-12-27 05:33:10,921][105620] Updated weights for policy 1, policy_version 1947580 (0.0008) [2023-12-27 05:33:10,988][105620] Updated weights for policy 1, policy_version 1947590 (0.0006) [2023-12-27 05:33:11,060][105620] Updated weights for policy 1, policy_version 1947600 (0.0008) [2023-12-27 05:33:11,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 996073472. Throughput: 0: 9460.6, 1: 10041.0. Samples: 996079184. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:33:11,062][104569] Avg episode reward: [(0, '8623.598'), (1, '9253.805')] [2023-12-27 05:33:11,166][105692] Updated weights for policy 0, policy_version 1942778 (0.0008) [2023-12-27 05:33:11,222][105692] Updated weights for policy 0, policy_version 1942788 (0.0008) [2023-12-27 05:33:11,278][105692] Updated weights for policy 0, policy_version 1942798 (0.0008) [2023-12-27 05:33:11,828][105620] Updated weights for policy 1, policy_version 1947610 (0.0011) [2023-12-27 05:33:11,887][105620] Updated weights for policy 1, policy_version 1947620 (0.0011) [2023-12-27 05:33:11,940][105620] Updated weights for policy 1, policy_version 1947630 (0.0011) [2023-12-27 05:33:12,116][105692] Updated weights for policy 0, policy_version 1942808 (0.0008) [2023-12-27 05:33:12,182][105692] Updated weights for policy 0, policy_version 1942818 (0.0006) [2023-12-27 05:33:12,249][105692] Updated weights for policy 0, policy_version 1942828 (0.0006) [2023-12-27 05:33:12,713][105620] Updated weights for policy 1, policy_version 1947640 (0.0011) [2023-12-27 05:33:12,775][105620] Updated weights for policy 1, policy_version 1947650 (0.0011) [2023-12-27 05:33:12,838][105620] Updated weights for policy 1, policy_version 1947660 (0.0011) [2023-12-27 05:33:12,943][105692] Updated weights for policy 0, policy_version 1942838 (0.0007) [2023-12-27 05:33:13,008][105692] Updated weights for policy 0, policy_version 1942848 (0.0005) [2023-12-27 05:33:13,070][105692] Updated weights for policy 0, policy_version 1942858 (0.0005) [2023-12-27 05:33:13,538][105620] Updated weights for policy 1, policy_version 1947670 (0.0010) [2023-12-27 05:33:13,586][105620] Updated weights for policy 1, policy_version 1947680 (0.0010) [2023-12-27 05:33:13,630][105620] Updated weights for policy 1, policy_version 1947690 (0.0010) [2023-12-27 05:33:13,709][105692] Updated weights for policy 0, policy_version 1942868 (0.0005) [2023-12-27 05:33:13,760][105692] Updated weights for policy 0, policy_version 1942878 (0.0008) [2023-12-27 05:33:13,804][105692] Updated weights for policy 0, policy_version 1942888 (0.0008) [2023-12-27 05:33:14,399][105620] Updated weights for policy 1, policy_version 1947700 (0.0010) [2023-12-27 05:33:14,464][105620] Updated weights for policy 1, policy_version 1947710 (0.0011) [2023-12-27 05:33:14,478][105692] Updated weights for policy 0, policy_version 1942898 (0.0008) [2023-12-27 05:33:14,512][105620] Updated weights for policy 1, policy_version 1947720 (0.0010) [2023-12-27 05:33:14,523][105692] Updated weights for policy 0, policy_version 1942908 (0.0006) [2023-12-27 05:33:14,569][105692] Updated weights for policy 0, policy_version 1942918 (0.0007) [2023-12-27 05:33:14,634][105692] Updated weights for policy 0, policy_version 1942928 (0.0008) [2023-12-27 05:33:15,272][105620] Updated weights for policy 1, policy_version 1947730 (0.0011) [2023-12-27 05:33:15,331][105620] Updated weights for policy 1, policy_version 1947740 (0.0011) [2023-12-27 05:33:15,358][105692] Updated weights for policy 0, policy_version 1942938 (0.0006) [2023-12-27 05:33:15,384][105620] Updated weights for policy 1, policy_version 1947750 (0.0011) [2023-12-27 05:33:15,407][105692] Updated weights for policy 0, policy_version 1942948 (0.0006) [2023-12-27 05:33:15,443][105620] Updated weights for policy 1, policy_version 1947760 (0.0011) [2023-12-27 05:33:15,460][105692] Updated weights for policy 0, policy_version 1942958 (0.0008) [2023-12-27 05:33:16,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 996163584. Throughput: 0: 9521.5, 1: 9992.9. Samples: 996137400. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:33:16,063][104569] Avg episode reward: [(0, '8625.916'), (1, '9255.197')] [2023-12-27 05:33:16,073][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001947760_498696192.pth... [2023-12-27 05:33:16,073][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001942960_497467392.pth... [2023-12-27 05:33:16,078][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001946608_498401280.pth [2023-12-27 05:33:16,087][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001941808_497172480.pth [2023-12-27 05:33:16,185][105620] Updated weights for policy 1, policy_version 1947770 (0.0011) [2023-12-27 05:33:16,203][105692] Updated weights for policy 0, policy_version 1942968 (0.0006) [2023-12-27 05:33:16,244][105620] Updated weights for policy 1, policy_version 1947780 (0.0010) [2023-12-27 05:33:16,254][105692] Updated weights for policy 0, policy_version 1942978 (0.0006) [2023-12-27 05:33:16,301][105620] Updated weights for policy 1, policy_version 1947790 (0.0010) [2023-12-27 05:33:16,314][105692] Updated weights for policy 0, policy_version 1942988 (0.0008) [2023-12-27 05:33:16,963][105620] Updated weights for policy 1, policy_version 1947800 (0.0009) [2023-12-27 05:33:17,020][105620] Updated weights for policy 1, policy_version 1947810 (0.0011) [2023-12-27 05:33:17,040][105692] Updated weights for policy 0, policy_version 1942998 (0.0008) [2023-12-27 05:33:17,074][105620] Updated weights for policy 1, policy_version 1947820 (0.0006) [2023-12-27 05:33:17,095][105692] Updated weights for policy 0, policy_version 1943008 (0.0009) [2023-12-27 05:33:17,145][105692] Updated weights for policy 0, policy_version 1943018 (0.0008) [2023-12-27 05:33:17,752][105620] Updated weights for policy 1, policy_version 1947830 (0.0010) [2023-12-27 05:33:17,806][105620] Updated weights for policy 1, policy_version 1947840 (0.0010) [2023-12-27 05:33:17,864][105620] Updated weights for policy 1, policy_version 1947850 (0.0010) [2023-12-27 05:33:17,932][105692] Updated weights for policy 0, policy_version 1943028 (0.0008) [2023-12-27 05:33:17,984][105692] Updated weights for policy 0, policy_version 1943038 (0.0008) [2023-12-27 05:33:18,036][105692] Updated weights for policy 0, policy_version 1943048 (0.0008) [2023-12-27 05:33:18,619][105620] Updated weights for policy 1, policy_version 1947860 (0.0011) [2023-12-27 05:33:18,681][105620] Updated weights for policy 1, policy_version 1947870 (0.0011) [2023-12-27 05:33:18,743][105620] Updated weights for policy 1, policy_version 1947880 (0.0011) [2023-12-27 05:33:18,838][105692] Updated weights for policy 0, policy_version 1943058 (0.0008) [2023-12-27 05:33:18,892][105692] Updated weights for policy 0, policy_version 1943068 (0.0008) [2023-12-27 05:33:18,945][105692] Updated weights for policy 0, policy_version 1943078 (0.0008) [2023-12-27 05:33:18,991][105692] Updated weights for policy 0, policy_version 1943088 (0.0008) [2023-12-27 05:33:19,477][105620] Updated weights for policy 1, policy_version 1947890 (0.0011) [2023-12-27 05:33:19,542][105620] Updated weights for policy 1, policy_version 1947900 (0.0011) [2023-12-27 05:33:19,602][105620] Updated weights for policy 1, policy_version 1947910 (0.0011) [2023-12-27 05:33:19,665][105620] Updated weights for policy 1, policy_version 1947920 (0.0011) [2023-12-27 05:33:19,796][105692] Updated weights for policy 0, policy_version 1943098 (0.0009) [2023-12-27 05:33:19,863][105692] Updated weights for policy 0, policy_version 1943108 (0.0008) [2023-12-27 05:33:19,927][105692] Updated weights for policy 0, policy_version 1943118 (0.0009) [2023-12-27 05:33:20,408][105620] Updated weights for policy 1, policy_version 1947930 (0.0011) [2023-12-27 05:33:20,471][105620] Updated weights for policy 1, policy_version 1947940 (0.0011) [2023-12-27 05:33:20,527][105620] Updated weights for policy 1, policy_version 1947950 (0.0011) [2023-12-27 05:33:20,723][105692] Updated weights for policy 0, policy_version 1943128 (0.0009) [2023-12-27 05:33:20,785][105692] Updated weights for policy 0, policy_version 1943138 (0.0009) [2023-12-27 05:33:20,855][105692] Updated weights for policy 0, policy_version 1943148 (0.0008) [2023-12-27 05:33:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19438.7). Total num frames: 996261888. Throughput: 0: 9538.7, 1: 10029.8. Samples: 996251772. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:33:21,063][104569] Avg episode reward: [(0, '8627.222'), (1, '9071.604')] [2023-12-27 05:33:21,307][105620] Updated weights for policy 1, policy_version 1947960 (0.0011) [2023-12-27 05:33:21,372][105620] Updated weights for policy 1, policy_version 1947970 (0.0009) [2023-12-27 05:33:21,441][105620] Updated weights for policy 1, policy_version 1947980 (0.0010) [2023-12-27 05:33:21,702][105692] Updated weights for policy 0, policy_version 1943158 (0.0009) [2023-12-27 05:33:21,758][105692] Updated weights for policy 0, policy_version 1943168 (0.0008) [2023-12-27 05:33:21,805][105692] Updated weights for policy 0, policy_version 1943178 (0.0008) [2023-12-27 05:33:22,216][105620] Updated weights for policy 1, policy_version 1947990 (0.0011) [2023-12-27 05:33:22,283][105620] Updated weights for policy 1, policy_version 1948000 (0.0011) [2023-12-27 05:33:22,348][105620] Updated weights for policy 1, policy_version 1948010 (0.0011) [2023-12-27 05:33:22,609][105692] Updated weights for policy 0, policy_version 1943188 (0.0008) [2023-12-27 05:33:22,669][105692] Updated weights for policy 0, policy_version 1943198 (0.0008) [2023-12-27 05:33:22,730][105692] Updated weights for policy 0, policy_version 1943208 (0.0008) [2023-12-27 05:33:23,089][105620] Updated weights for policy 1, policy_version 1948020 (0.0011) [2023-12-27 05:33:23,155][105620] Updated weights for policy 1, policy_version 1948030 (0.0010) [2023-12-27 05:33:23,219][105620] Updated weights for policy 1, policy_version 1948040 (0.0011) [2023-12-27 05:33:23,498][105692] Updated weights for policy 0, policy_version 1943218 (0.0008) [2023-12-27 05:33:23,547][105692] Updated weights for policy 0, policy_version 1943228 (0.0008) [2023-12-27 05:33:23,607][105692] Updated weights for policy 0, policy_version 1943238 (0.0008) [2023-12-27 05:33:23,662][105692] Updated weights for policy 0, policy_version 1943248 (0.0008) [2023-12-27 05:33:23,950][105620] Updated weights for policy 1, policy_version 1948050 (0.0011) [2023-12-27 05:33:24,002][105620] Updated weights for policy 1, policy_version 1948060 (0.0010) [2023-12-27 05:33:24,050][105620] Updated weights for policy 1, policy_version 1948070 (0.0010) [2023-12-27 05:33:24,092][105620] Updated weights for policy 1, policy_version 1948080 (0.0010) [2023-12-27 05:33:24,436][105692] Updated weights for policy 0, policy_version 1943258 (0.0008) [2023-12-27 05:33:24,488][105692] Updated weights for policy 0, policy_version 1943268 (0.0008) [2023-12-27 05:33:24,537][105692] Updated weights for policy 0, policy_version 1943278 (0.0008) [2023-12-27 05:33:24,873][105620] Updated weights for policy 1, policy_version 1948090 (0.0010) [2023-12-27 05:33:24,921][105620] Updated weights for policy 1, policy_version 1948100 (0.0010) [2023-12-27 05:33:24,978][105620] Updated weights for policy 1, policy_version 1948110 (0.0010) [2023-12-27 05:33:25,336][105692] Updated weights for policy 0, policy_version 1943288 (0.0008) [2023-12-27 05:33:25,384][105692] Updated weights for policy 0, policy_version 1943298 (0.0008) [2023-12-27 05:33:25,435][105692] Updated weights for policy 0, policy_version 1943308 (0.0008) [2023-12-27 05:33:25,718][105620] Updated weights for policy 1, policy_version 1948120 (0.0006) [2023-12-27 05:33:25,766][105620] Updated weights for policy 1, policy_version 1948130 (0.0005) [2023-12-27 05:33:25,813][105620] Updated weights for policy 1, policy_version 1948140 (0.0005) [2023-12-27 05:33:26,056][105692] Updated weights for policy 0, policy_version 1943318 (0.0007) [2023-12-27 05:33:26,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 996352000. Throughput: 0: 9583.7, 1: 9835.9. Samples: 996360352. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:33:26,063][104569] Avg episode reward: [(0, '8627.353'), (1, '9162.513')] [2023-12-27 05:33:26,103][105692] Updated weights for policy 0, policy_version 1943328 (0.0005) [2023-12-27 05:33:26,167][105692] Updated weights for policy 0, policy_version 1943338 (0.0006) [2023-12-27 05:33:26,403][105620] Updated weights for policy 1, policy_version 1948150 (0.0005) [2023-12-27 05:33:26,458][105620] Updated weights for policy 1, policy_version 1948160 (0.0005) [2023-12-27 05:33:26,507][105620] Updated weights for policy 1, policy_version 1948170 (0.0005) [2023-12-27 05:33:26,804][105692] Updated weights for policy 0, policy_version 1943348 (0.0007) [2023-12-27 05:33:26,854][105692] Updated weights for policy 0, policy_version 1943358 (0.0010) [2023-12-27 05:33:26,907][105692] Updated weights for policy 0, policy_version 1943368 (0.0009) [2023-12-27 05:33:27,169][105620] Updated weights for policy 1, policy_version 1948180 (0.0005) [2023-12-27 05:33:27,222][105620] Updated weights for policy 1, policy_version 1948190 (0.0005) [2023-12-27 05:33:27,278][105620] Updated weights for policy 1, policy_version 1948200 (0.0005) [2023-12-27 05:33:27,702][105692] Updated weights for policy 0, policy_version 1943378 (0.0008) [2023-12-27 05:33:27,758][105692] Updated weights for policy 0, policy_version 1943388 (0.0006) [2023-12-27 05:33:27,812][105692] Updated weights for policy 0, policy_version 1943398 (0.0005) [2023-12-27 05:33:27,862][105692] Updated weights for policy 0, policy_version 1943408 (0.0006) [2023-12-27 05:33:27,949][105620] Updated weights for policy 1, policy_version 1948210 (0.0008) [2023-12-27 05:33:28,002][105620] Updated weights for policy 1, policy_version 1948220 (0.0011) [2023-12-27 05:33:28,065][105620] Updated weights for policy 1, policy_version 1948230 (0.0011) [2023-12-27 05:33:28,117][105620] Updated weights for policy 1, policy_version 1948240 (0.0010) [2023-12-27 05:33:28,545][105692] Updated weights for policy 0, policy_version 1943418 (0.0011) [2023-12-27 05:33:28,607][105692] Updated weights for policy 0, policy_version 1943428 (0.0010) [2023-12-27 05:33:28,672][105692] Updated weights for policy 0, policy_version 1943438 (0.0009) [2023-12-27 05:33:28,784][105620] Updated weights for policy 1, policy_version 1948250 (0.0010) [2023-12-27 05:33:28,838][105620] Updated weights for policy 1, policy_version 1948260 (0.0010) [2023-12-27 05:33:28,892][105620] Updated weights for policy 1, policy_version 1948270 (0.0010) [2023-12-27 05:33:29,403][105692] Updated weights for policy 0, policy_version 1943448 (0.0007) [2023-12-27 05:33:29,451][105692] Updated weights for policy 0, policy_version 1943458 (0.0005) [2023-12-27 05:33:29,508][105692] Updated weights for policy 0, policy_version 1943468 (0.0010) [2023-12-27 05:33:29,643][105620] Updated weights for policy 1, policy_version 1948280 (0.0010) [2023-12-27 05:33:29,695][105620] Updated weights for policy 1, policy_version 1948290 (0.0010) [2023-12-27 05:33:29,762][105620] Updated weights for policy 1, policy_version 1948300 (0.0011) [2023-12-27 05:33:30,186][105692] Updated weights for policy 0, policy_version 1943478 (0.0011) [2023-12-27 05:33:30,252][105692] Updated weights for policy 0, policy_version 1943488 (0.0011) [2023-12-27 05:33:30,305][105692] Updated weights for policy 0, policy_version 1943498 (0.0011) [2023-12-27 05:33:30,428][105620] Updated weights for policy 1, policy_version 1948310 (0.0009) [2023-12-27 05:33:30,488][105620] Updated weights for policy 1, policy_version 1948320 (0.0008) [2023-12-27 05:33:30,545][105620] Updated weights for policy 1, policy_version 1948330 (0.0008) [2023-12-27 05:33:31,010][105692] Updated weights for policy 0, policy_version 1943508 (0.0010) [2023-12-27 05:33:31,062][104569] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 996450304. Throughput: 0: 9697.2, 1: 9860.8. Samples: 996423396. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:33:31,062][104569] Avg episode reward: [(0, '8540.246'), (1, '9253.662')] [2023-12-27 05:33:31,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001948336_498843648.pth... [2023-12-27 05:33:31,070][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001947184_498548736.pth [2023-12-27 05:33:31,076][105692] Updated weights for policy 0, policy_version 1943518 (0.0011) [2023-12-27 05:33:31,140][105692] Updated weights for policy 0, policy_version 1943528 (0.0010) [2023-12-27 05:33:31,180][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001943536_497614848.pth... [2023-12-27 05:33:31,183][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001942384_497319936.pth [2023-12-27 05:33:31,367][105620] Updated weights for policy 1, policy_version 1948340 (0.0008) [2023-12-27 05:33:31,431][105620] Updated weights for policy 1, policy_version 1948350 (0.0008) [2023-12-27 05:33:31,479][105620] Updated weights for policy 1, policy_version 1948360 (0.0008) [2023-12-27 05:33:31,865][105692] Updated weights for policy 0, policy_version 1943538 (0.0010) [2023-12-27 05:33:31,919][105692] Updated weights for policy 0, policy_version 1943548 (0.0009) [2023-12-27 05:33:31,979][105692] Updated weights for policy 0, policy_version 1943558 (0.0009) [2023-12-27 05:33:32,036][105692] Updated weights for policy 0, policy_version 1943568 (0.0007) [2023-12-27 05:33:32,267][105620] Updated weights for policy 1, policy_version 1948370 (0.0008) [2023-12-27 05:33:32,321][105620] Updated weights for policy 1, policy_version 1948380 (0.0009) [2023-12-27 05:33:32,384][105620] Updated weights for policy 1, policy_version 1948390 (0.0008) [2023-12-27 05:33:32,430][105620] Updated weights for policy 1, policy_version 1948400 (0.0009) [2023-12-27 05:33:32,779][105692] Updated weights for policy 0, policy_version 1943578 (0.0005) [2023-12-27 05:33:32,827][105692] Updated weights for policy 0, policy_version 1943588 (0.0005) [2023-12-27 05:33:32,879][105692] Updated weights for policy 0, policy_version 1943598 (0.0005) [2023-12-27 05:33:33,216][105620] Updated weights for policy 1, policy_version 1948410 (0.0007) [2023-12-27 05:33:33,279][105620] Updated weights for policy 1, policy_version 1948420 (0.0007) [2023-12-27 05:33:33,341][105620] Updated weights for policy 1, policy_version 1948430 (0.0008) [2023-12-27 05:33:33,466][105692] Updated weights for policy 0, policy_version 1943608 (0.0009) [2023-12-27 05:33:33,510][105692] Updated weights for policy 0, policy_version 1943618 (0.0010) [2023-12-27 05:33:33,558][105692] Updated weights for policy 0, policy_version 1943628 (0.0010) [2023-12-27 05:33:33,944][105620] Updated weights for policy 1, policy_version 1948440 (0.0010) [2023-12-27 05:33:33,995][105620] Updated weights for policy 1, policy_version 1948450 (0.0010) [2023-12-27 05:33:34,047][105620] Updated weights for policy 1, policy_version 1948460 (0.0010) [2023-12-27 05:33:34,286][105692] Updated weights for policy 0, policy_version 1943638 (0.0009) [2023-12-27 05:33:34,340][105692] Updated weights for policy 0, policy_version 1943648 (0.0008) [2023-12-27 05:33:34,388][105692] Updated weights for policy 0, policy_version 1943658 (0.0008) [2023-12-27 05:33:34,815][105620] Updated weights for policy 1, policy_version 1948470 (0.0010) [2023-12-27 05:33:34,884][105620] Updated weights for policy 1, policy_version 1948480 (0.0010) [2023-12-27 05:33:34,940][105620] Updated weights for policy 1, policy_version 1948490 (0.0010) [2023-12-27 05:33:35,070][105692] Updated weights for policy 0, policy_version 1943668 (0.0008) [2023-12-27 05:33:35,134][105692] Updated weights for policy 0, policy_version 1943678 (0.0008) [2023-12-27 05:33:35,196][105692] Updated weights for policy 0, policy_version 1943688 (0.0008) [2023-12-27 05:33:35,684][105620] Updated weights for policy 1, policy_version 1948500 (0.0009) [2023-12-27 05:33:35,741][105620] Updated weights for policy 1, policy_version 1948510 (0.0008) [2023-12-27 05:33:35,796][105620] Updated weights for policy 1, policy_version 1948520 (0.0010) [2023-12-27 05:33:35,871][105692] Updated weights for policy 0, policy_version 1943698 (0.0008) [2023-12-27 05:33:35,924][105692] Updated weights for policy 0, policy_version 1943708 (0.0009) [2023-12-27 05:33:35,975][105692] Updated weights for policy 0, policy_version 1943718 (0.0010) [2023-12-27 05:33:36,027][105692] Updated weights for policy 0, policy_version 1943728 (0.0010) [2023-12-27 05:33:36,062][104569] Fps is (10 sec: 20479.7, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 996556800. Throughput: 0: 9723.2, 1: 9828.0. Samples: 996539616. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 05:33:36,063][104569] Avg episode reward: [(0, '8445.341'), (1, '9255.152')] [2023-12-27 05:33:36,437][105620] Updated weights for policy 1, policy_version 1948530 (0.0010) [2023-12-27 05:33:36,508][105620] Updated weights for policy 1, policy_version 1948540 (0.0011) [2023-12-27 05:33:36,568][105620] Updated weights for policy 1, policy_version 1948550 (0.0007) [2023-12-27 05:33:36,637][105620] Updated weights for policy 1, policy_version 1948560 (0.0005) [2023-12-27 05:33:36,813][105692] Updated weights for policy 0, policy_version 1943738 (0.0011) [2023-12-27 05:33:36,880][105692] Updated weights for policy 0, policy_version 1943748 (0.0010) [2023-12-27 05:33:36,943][105692] Updated weights for policy 0, policy_version 1943758 (0.0011) [2023-12-27 05:33:37,227][105620] Updated weights for policy 1, policy_version 1948570 (0.0011) [2023-12-27 05:33:37,305][105620] Updated weights for policy 1, policy_version 1948580 (0.0011) [2023-12-27 05:33:37,357][105620] Updated weights for policy 1, policy_version 1948590 (0.0010) [2023-12-27 05:33:37,567][105692] Updated weights for policy 0, policy_version 1943768 (0.0010) [2023-12-27 05:33:37,620][105692] Updated weights for policy 0, policy_version 1943778 (0.0008) [2023-12-27 05:33:37,670][105692] Updated weights for policy 0, policy_version 1943788 (0.0008) [2023-12-27 05:33:38,040][105620] Updated weights for policy 1, policy_version 1948600 (0.0007) [2023-12-27 05:33:38,085][105620] Updated weights for policy 1, policy_version 1948610 (0.0005) [2023-12-27 05:33:38,143][105620] Updated weights for policy 1, policy_version 1948620 (0.0006) [2023-12-27 05:33:38,295][105692] Updated weights for policy 0, policy_version 1943798 (0.0006) [2023-12-27 05:33:38,365][105692] Updated weights for policy 0, policy_version 1943808 (0.0007) [2023-12-27 05:33:38,433][105692] Updated weights for policy 0, policy_version 1943818 (0.0006) [2023-12-27 05:33:38,879][105620] Updated weights for policy 1, policy_version 1948630 (0.0008) [2023-12-27 05:33:38,941][105620] Updated weights for policy 1, policy_version 1948640 (0.0010) [2023-12-27 05:33:38,989][105692] Updated weights for policy 0, policy_version 1943828 (0.0006) [2023-12-27 05:33:38,999][105620] Updated weights for policy 1, policy_version 1948650 (0.0010) [2023-12-27 05:33:39,053][105692] Updated weights for policy 0, policy_version 1943838 (0.0007) [2023-12-27 05:33:39,117][105692] Updated weights for policy 0, policy_version 1943848 (0.0005) [2023-12-27 05:33:39,802][105620] Updated weights for policy 1, policy_version 1948660 (0.0009) [2023-12-27 05:33:39,859][105692] Updated weights for policy 0, policy_version 1943858 (0.0008) [2023-12-27 05:33:39,866][105620] Updated weights for policy 1, policy_version 1948670 (0.0008) [2023-12-27 05:33:39,922][105692] Updated weights for policy 0, policy_version 1943868 (0.0008) [2023-12-27 05:33:39,924][105620] Updated weights for policy 1, policy_version 1948680 (0.0011) [2023-12-27 05:33:39,986][105692] Updated weights for policy 0, policy_version 1943878 (0.0008) [2023-12-27 05:33:40,046][105692] Updated weights for policy 0, policy_version 1943888 (0.0008) [2023-12-27 05:33:40,654][105620] Updated weights for policy 1, policy_version 1948690 (0.0009) [2023-12-27 05:33:40,703][105620] Updated weights for policy 1, policy_version 1948700 (0.0005) [2023-12-27 05:33:40,763][105620] Updated weights for policy 1, policy_version 1948710 (0.0005) [2023-12-27 05:33:40,810][105620] Updated weights for policy 1, policy_version 1948720 (0.0005) [2023-12-27 05:33:40,832][105692] Updated weights for policy 0, policy_version 1943898 (0.0009) [2023-12-27 05:33:40,886][105692] Updated weights for policy 0, policy_version 1943908 (0.0009) [2023-12-27 05:33:40,939][105692] Updated weights for policy 0, policy_version 1943918 (0.0010) [2023-12-27 05:33:41,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 996655104. Throughput: 0: 9779.6, 1: 9788.1. Samples: 996660332. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:33:41,062][104569] Avg episode reward: [(0, '8626.590'), (1, '9163.098')] [2023-12-27 05:33:41,513][105620] Updated weights for policy 1, policy_version 1948730 (0.0011) [2023-12-27 05:33:41,563][105620] Updated weights for policy 1, policy_version 1948740 (0.0009) [2023-12-27 05:33:41,622][105620] Updated weights for policy 1, policy_version 1948750 (0.0008) [2023-12-27 05:33:41,772][105692] Updated weights for policy 0, policy_version 1943928 (0.0010) [2023-12-27 05:33:41,829][105692] Updated weights for policy 0, policy_version 1943938 (0.0009) [2023-12-27 05:33:41,893][105692] Updated weights for policy 0, policy_version 1943948 (0.0008) [2023-12-27 05:33:42,381][105620] Updated weights for policy 1, policy_version 1948760 (0.0010) [2023-12-27 05:33:42,443][105620] Updated weights for policy 1, policy_version 1948770 (0.0009) [2023-12-27 05:33:42,504][105620] Updated weights for policy 1, policy_version 1948780 (0.0006) [2023-12-27 05:33:42,573][105692] Updated weights for policy 0, policy_version 1943958 (0.0008) [2023-12-27 05:33:42,625][105692] Updated weights for policy 0, policy_version 1943968 (0.0008) [2023-12-27 05:33:42,680][105692] Updated weights for policy 0, policy_version 1943978 (0.0008) [2023-12-27 05:33:43,248][105620] Updated weights for policy 1, policy_version 1948790 (0.0008) [2023-12-27 05:33:43,316][105620] Updated weights for policy 1, policy_version 1948800 (0.0008) [2023-12-27 05:33:43,384][105620] Updated weights for policy 1, policy_version 1948810 (0.0008) [2023-12-27 05:33:43,424][105692] Updated weights for policy 0, policy_version 1943988 (0.0010) [2023-12-27 05:33:43,481][105692] Updated weights for policy 0, policy_version 1943998 (0.0011) [2023-12-27 05:33:43,539][105692] Updated weights for policy 0, policy_version 1944008 (0.0011) [2023-12-27 05:33:44,094][105620] Updated weights for policy 1, policy_version 1948820 (0.0007) [2023-12-27 05:33:44,158][105620] Updated weights for policy 1, policy_version 1948830 (0.0006) [2023-12-27 05:33:44,213][105620] Updated weights for policy 1, policy_version 1948840 (0.0007) [2023-12-27 05:33:44,274][105692] Updated weights for policy 0, policy_version 1944018 (0.0010) [2023-12-27 05:33:44,323][105692] Updated weights for policy 0, policy_version 1944028 (0.0007) [2023-12-27 05:33:44,381][105692] Updated weights for policy 0, policy_version 1944038 (0.0008) [2023-12-27 05:33:44,438][105692] Updated weights for policy 0, policy_version 1944048 (0.0010) [2023-12-27 05:33:44,847][105620] Updated weights for policy 1, policy_version 1948850 (0.0010) [2023-12-27 05:33:44,901][105620] Updated weights for policy 1, policy_version 1948860 (0.0011) [2023-12-27 05:33:44,968][105620] Updated weights for policy 1, policy_version 1948870 (0.0011) [2023-12-27 05:33:45,033][105620] Updated weights for policy 1, policy_version 1948880 (0.0011) [2023-12-27 05:33:45,098][105692] Updated weights for policy 0, policy_version 1944058 (0.0007) [2023-12-27 05:33:45,155][105692] Updated weights for policy 0, policy_version 1944068 (0.0008) [2023-12-27 05:33:45,209][105692] Updated weights for policy 0, policy_version 1944078 (0.0006) [2023-12-27 05:33:45,787][105620] Updated weights for policy 1, policy_version 1948890 (0.0010) [2023-12-27 05:33:45,790][105692] Updated weights for policy 0, policy_version 1944088 (0.0007) [2023-12-27 05:33:45,837][105692] Updated weights for policy 0, policy_version 1944098 (0.0005) [2023-12-27 05:33:45,839][105620] Updated weights for policy 1, policy_version 1948900 (0.0009) [2023-12-27 05:33:45,891][105620] Updated weights for policy 1, policy_version 1948910 (0.0009) [2023-12-27 05:33:45,892][105692] Updated weights for policy 0, policy_version 1944108 (0.0005) [2023-12-27 05:33:46,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 996753408. Throughput: 0: 9717.0, 1: 9750.4. Samples: 996715312. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:33:46,063][104569] Avg episode reward: [(0, '8445.338'), (1, '9253.862')] [2023-12-27 05:33:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001948912_498991104.pth... [2023-12-27 05:33:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001944112_497762304.pth... [2023-12-27 05:33:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001947760_498696192.pth [2023-12-27 05:33:46,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001942960_497467392.pth [2023-12-27 05:33:46,513][105692] Updated weights for policy 0, policy_version 1944118 (0.0007) [2023-12-27 05:33:46,563][105692] Updated weights for policy 0, policy_version 1944128 (0.0009) [2023-12-27 05:33:46,608][105692] Updated weights for policy 0, policy_version 1944138 (0.0009) [2023-12-27 05:33:46,647][105620] Updated weights for policy 1, policy_version 1948920 (0.0010) [2023-12-27 05:33:46,698][105620] Updated weights for policy 1, policy_version 1948930 (0.0010) [2023-12-27 05:33:46,746][105620] Updated weights for policy 1, policy_version 1948940 (0.0010) [2023-12-27 05:33:47,380][105692] Updated weights for policy 0, policy_version 1944148 (0.0006) [2023-12-27 05:33:47,431][105692] Updated weights for policy 0, policy_version 1944158 (0.0008) [2023-12-27 05:33:47,482][105692] Updated weights for policy 0, policy_version 1944168 (0.0008) [2023-12-27 05:33:47,504][105620] Updated weights for policy 1, policy_version 1948950 (0.0010) [2023-12-27 05:33:47,552][105620] Updated weights for policy 1, policy_version 1948960 (0.0010) [2023-12-27 05:33:47,608][105620] Updated weights for policy 1, policy_version 1948970 (0.0010) [2023-12-27 05:33:48,169][105692] Updated weights for policy 0, policy_version 1944178 (0.0007) [2023-12-27 05:33:48,232][105692] Updated weights for policy 0, policy_version 1944188 (0.0011) [2023-12-27 05:33:48,241][105620] Updated weights for policy 1, policy_version 1948980 (0.0010) [2023-12-27 05:33:48,281][105692] Updated weights for policy 0, policy_version 1944198 (0.0010) [2023-12-27 05:33:48,296][105620] Updated weights for policy 1, policy_version 1948990 (0.0010) [2023-12-27 05:33:48,336][105692] Updated weights for policy 0, policy_version 1944208 (0.0010) [2023-12-27 05:33:48,353][105620] Updated weights for policy 1, policy_version 1949000 (0.0009) [2023-12-27 05:33:49,094][105692] Updated weights for policy 0, policy_version 1944218 (0.0011) [2023-12-27 05:33:49,139][105692] Updated weights for policy 0, policy_version 1944228 (0.0010) [2023-12-27 05:33:49,143][105620] Updated weights for policy 1, policy_version 1949010 (0.0010) [2023-12-27 05:33:49,192][105620] Updated weights for policy 1, policy_version 1949020 (0.0010) [2023-12-27 05:33:49,194][105692] Updated weights for policy 0, policy_version 1944238 (0.0011) [2023-12-27 05:33:49,259][105620] Updated weights for policy 1, policy_version 1949030 (0.0010) [2023-12-27 05:33:49,321][105620] Updated weights for policy 1, policy_version 1949040 (0.0011) [2023-12-27 05:33:49,940][105692] Updated weights for policy 0, policy_version 1944248 (0.0011) [2023-12-27 05:33:50,005][105692] Updated weights for policy 0, policy_version 1944258 (0.0011) [2023-12-27 05:33:50,036][105620] Updated weights for policy 1, policy_version 1949050 (0.0010) [2023-12-27 05:33:50,063][105692] Updated weights for policy 0, policy_version 1944268 (0.0009) [2023-12-27 05:33:50,095][105620] Updated weights for policy 1, policy_version 1949060 (0.0011) [2023-12-27 05:33:50,154][105620] Updated weights for policy 1, policy_version 1949070 (0.0011) [2023-12-27 05:33:50,783][105692] Updated weights for policy 0, policy_version 1944278 (0.0011) [2023-12-27 05:33:50,831][105620] Updated weights for policy 1, policy_version 1949080 (0.0007) [2023-12-27 05:33:50,846][105692] Updated weights for policy 0, policy_version 1944288 (0.0011) [2023-12-27 05:33:50,889][105620] Updated weights for policy 1, policy_version 1949090 (0.0005) [2023-12-27 05:33:50,909][105692] Updated weights for policy 0, policy_version 1944298 (0.0011) [2023-12-27 05:33:50,945][105620] Updated weights for policy 1, policy_version 1949100 (0.0007) [2023-12-27 05:33:51,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 996851712. Throughput: 0: 9752.0, 1: 9725.9. Samples: 996835048. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:33:51,063][104569] Avg episode reward: [(0, '8360.660'), (1, '9345.899')] [2023-12-27 05:33:51,656][105692] Updated weights for policy 0, policy_version 1944308 (0.0009) [2023-12-27 05:33:51,670][105620] Updated weights for policy 1, policy_version 1949110 (0.0011) [2023-12-27 05:33:51,710][105692] Updated weights for policy 0, policy_version 1944318 (0.0006) [2023-12-27 05:33:51,725][105620] Updated weights for policy 1, policy_version 1949120 (0.0010) [2023-12-27 05:33:51,773][105692] Updated weights for policy 0, policy_version 1944328 (0.0008) [2023-12-27 05:33:51,791][105620] Updated weights for policy 1, policy_version 1949130 (0.0010) [2023-12-27 05:33:52,510][105620] Updated weights for policy 1, policy_version 1949140 (0.0009) [2023-12-27 05:33:52,573][105620] Updated weights for policy 1, policy_version 1949150 (0.0011) [2023-12-27 05:33:52,573][105692] Updated weights for policy 0, policy_version 1944338 (0.0009) [2023-12-27 05:33:52,628][105692] Updated weights for policy 0, policy_version 1944348 (0.0010) [2023-12-27 05:33:52,631][105620] Updated weights for policy 1, policy_version 1949160 (0.0009) [2023-12-27 05:33:52,682][105692] Updated weights for policy 0, policy_version 1944358 (0.0008) [2023-12-27 05:33:52,735][105692] Updated weights for policy 0, policy_version 1944368 (0.0008) [2023-12-27 05:33:53,289][105620] Updated weights for policy 1, policy_version 1949170 (0.0008) [2023-12-27 05:33:53,340][105620] Updated weights for policy 1, policy_version 1949180 (0.0010) [2023-12-27 05:33:53,394][105620] Updated weights for policy 1, policy_version 1949190 (0.0010) [2023-12-27 05:33:53,453][105620] Updated weights for policy 1, policy_version 1949200 (0.0010) [2023-12-27 05:33:53,527][105692] Updated weights for policy 0, policy_version 1944378 (0.0009) [2023-12-27 05:33:53,571][105692] Updated weights for policy 0, policy_version 1944388 (0.0006) [2023-12-27 05:33:53,628][105692] Updated weights for policy 0, policy_version 1944398 (0.0009) [2023-12-27 05:33:54,063][105620] Updated weights for policy 1, policy_version 1949210 (0.0010) [2023-12-27 05:33:54,125][105620] Updated weights for policy 1, policy_version 1949220 (0.0011) [2023-12-27 05:33:54,181][105620] Updated weights for policy 1, policy_version 1949230 (0.0009) [2023-12-27 05:33:54,373][105692] Updated weights for policy 0, policy_version 1944408 (0.0006) [2023-12-27 05:33:54,432][105692] Updated weights for policy 0, policy_version 1944418 (0.0008) [2023-12-27 05:33:54,482][105692] Updated weights for policy 0, policy_version 1944428 (0.0008) [2023-12-27 05:33:54,874][105620] Updated weights for policy 1, policy_version 1949240 (0.0010) [2023-12-27 05:33:54,935][105620] Updated weights for policy 1, policy_version 1949250 (0.0009) [2023-12-27 05:33:54,994][105620] Updated weights for policy 1, policy_version 1949260 (0.0011) [2023-12-27 05:33:55,236][105692] Updated weights for policy 0, policy_version 1944438 (0.0009) [2023-12-27 05:33:55,298][105692] Updated weights for policy 0, policy_version 1944448 (0.0008) [2023-12-27 05:33:55,354][105692] Updated weights for policy 0, policy_version 1944458 (0.0007) [2023-12-27 05:33:55,676][105620] Updated weights for policy 1, policy_version 1949270 (0.0010) [2023-12-27 05:33:55,727][105620] Updated weights for policy 1, policy_version 1949280 (0.0010) [2023-12-27 05:33:55,774][105620] Updated weights for policy 1, policy_version 1949290 (0.0010) [2023-12-27 05:33:56,029][105692] Updated weights for policy 0, policy_version 1944468 (0.0008) [2023-12-27 05:33:56,062][104569] Fps is (10 sec: 18842.0, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 996941824. Throughput: 0: 9687.6, 1: 9684.8. Samples: 996950944. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:33:56,062][104569] Avg episode reward: [(0, '8264.855'), (1, '9345.996')] [2023-12-27 05:33:56,091][105692] Updated weights for policy 0, policy_version 1944478 (0.0005) [2023-12-27 05:33:56,158][105692] Updated weights for policy 0, policy_version 1944488 (0.0005) [2023-12-27 05:33:56,388][105620] Updated weights for policy 1, policy_version 1949300 (0.0010) [2023-12-27 05:33:56,436][105620] Updated weights for policy 1, policy_version 1949310 (0.0010) [2023-12-27 05:33:56,485][105620] Updated weights for policy 1, policy_version 1949320 (0.0010) [2023-12-27 05:33:56,773][105692] Updated weights for policy 0, policy_version 1944498 (0.0006) [2023-12-27 05:33:56,818][105692] Updated weights for policy 0, policy_version 1944508 (0.0007) [2023-12-27 05:33:56,862][105692] Updated weights for policy 0, policy_version 1944518 (0.0008) [2023-12-27 05:33:56,910][105692] Updated weights for policy 0, policy_version 1944528 (0.0007) [2023-12-27 05:33:57,241][105620] Updated weights for policy 1, policy_version 1949330 (0.0010) [2023-12-27 05:33:57,288][105620] Updated weights for policy 1, policy_version 1949340 (0.0010) [2023-12-27 05:33:57,347][105620] Updated weights for policy 1, policy_version 1949350 (0.0010) [2023-12-27 05:33:57,403][105620] Updated weights for policy 1, policy_version 1949360 (0.0010) [2023-12-27 05:33:57,657][105692] Updated weights for policy 0, policy_version 1944538 (0.0008) [2023-12-27 05:33:57,716][105692] Updated weights for policy 0, policy_version 1944548 (0.0008) [2023-12-27 05:33:57,772][105692] Updated weights for policy 0, policy_version 1944558 (0.0008) [2023-12-27 05:33:58,096][105620] Updated weights for policy 1, policy_version 1949370 (0.0005) [2023-12-27 05:33:58,147][105620] Updated weights for policy 1, policy_version 1949380 (0.0006) [2023-12-27 05:33:58,213][105620] Updated weights for policy 1, policy_version 1949390 (0.0009) [2023-12-27 05:33:58,591][105692] Updated weights for policy 0, policy_version 1944568 (0.0008) [2023-12-27 05:33:58,659][105692] Updated weights for policy 0, policy_version 1944578 (0.0008) [2023-12-27 05:33:58,724][105692] Updated weights for policy 0, policy_version 1944588 (0.0008) [2023-12-27 05:33:58,965][105620] Updated weights for policy 1, policy_version 1949400 (0.0009) [2023-12-27 05:33:59,019][105620] Updated weights for policy 1, policy_version 1949410 (0.0011) [2023-12-27 05:33:59,084][105620] Updated weights for policy 1, policy_version 1949420 (0.0006) [2023-12-27 05:33:59,544][105692] Updated weights for policy 0, policy_version 1944598 (0.0010) [2023-12-27 05:33:59,601][105692] Updated weights for policy 0, policy_version 1944608 (0.0011) [2023-12-27 05:33:59,650][105692] Updated weights for policy 0, policy_version 1944618 (0.0011) [2023-12-27 05:33:59,831][105620] Updated weights for policy 1, policy_version 1949430 (0.0009) [2023-12-27 05:33:59,890][105620] Updated weights for policy 1, policy_version 1949440 (0.0008) [2023-12-27 05:33:59,957][105620] Updated weights for policy 1, policy_version 1949450 (0.0009) [2023-12-27 05:34:00,446][105692] Updated weights for policy 0, policy_version 1944628 (0.0011) [2023-12-27 05:34:00,500][105692] Updated weights for policy 0, policy_version 1944638 (0.0010) [2023-12-27 05:34:00,548][105692] Updated weights for policy 0, policy_version 1944648 (0.0010) [2023-12-27 05:34:00,673][105620] Updated weights for policy 1, policy_version 1949460 (0.0008) [2023-12-27 05:34:00,724][105620] Updated weights for policy 1, policy_version 1949470 (0.0008) [2023-12-27 05:34:00,772][105620] Updated weights for policy 1, policy_version 1949480 (0.0008) [2023-12-27 05:34:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 997040128. Throughput: 0: 9677.4, 1: 9735.3. Samples: 997010968. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:34:01,063][104569] Avg episode reward: [(0, '8173.621'), (1, '9254.006')] [2023-12-27 05:34:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001944656_497901568.pth... [2023-12-27 05:34:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001949488_499138560.pth... [2023-12-27 05:34:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001948336_498843648.pth [2023-12-27 05:34:01,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001943536_497614848.pth [2023-12-27 05:34:01,329][105692] Updated weights for policy 0, policy_version 1944658 (0.0010) [2023-12-27 05:34:01,396][105692] Updated weights for policy 0, policy_version 1944668 (0.0012) [2023-12-27 05:34:01,454][105692] Updated weights for policy 0, policy_version 1944678 (0.0009) [2023-12-27 05:34:01,503][105692] Updated weights for policy 0, policy_version 1944688 (0.0009) [2023-12-27 05:34:01,520][105620] Updated weights for policy 1, policy_version 1949490 (0.0007) [2023-12-27 05:34:01,581][105620] Updated weights for policy 1, policy_version 1949500 (0.0007) [2023-12-27 05:34:01,648][105620] Updated weights for policy 1, policy_version 1949510 (0.0009) [2023-12-27 05:34:01,713][105620] Updated weights for policy 1, policy_version 1949520 (0.0008) [2023-12-27 05:34:02,319][105692] Updated weights for policy 0, policy_version 1944698 (0.0010) [2023-12-27 05:34:02,386][105692] Updated weights for policy 0, policy_version 1944708 (0.0009) [2023-12-27 05:34:02,443][105692] Updated weights for policy 0, policy_version 1944718 (0.0009) [2023-12-27 05:34:02,494][105620] Updated weights for policy 1, policy_version 1949530 (0.0009) [2023-12-27 05:34:02,544][105620] Updated weights for policy 1, policy_version 1949540 (0.0008) [2023-12-27 05:34:02,602][105620] Updated weights for policy 1, policy_version 1949550 (0.0005) [2023-12-27 05:34:03,168][105692] Updated weights for policy 0, policy_version 1944728 (0.0010) [2023-12-27 05:34:03,224][105692] Updated weights for policy 0, policy_version 1944738 (0.0010) [2023-12-27 05:34:03,279][105692] Updated weights for policy 0, policy_version 1944748 (0.0008) [2023-12-27 05:34:03,391][105620] Updated weights for policy 1, policy_version 1949560 (0.0008) [2023-12-27 05:34:03,454][105620] Updated weights for policy 1, policy_version 1949570 (0.0009) [2023-12-27 05:34:03,516][105620] Updated weights for policy 1, policy_version 1949580 (0.0008) [2023-12-27 05:34:03,867][105692] Updated weights for policy 0, policy_version 1944758 (0.0007) [2023-12-27 05:34:03,923][105692] Updated weights for policy 0, policy_version 1944768 (0.0009) [2023-12-27 05:34:03,972][105692] Updated weights for policy 0, policy_version 1944778 (0.0011) [2023-12-27 05:34:04,335][105620] Updated weights for policy 1, policy_version 1949590 (0.0009) [2023-12-27 05:34:04,405][105620] Updated weights for policy 1, policy_version 1949600 (0.0010) [2023-12-27 05:34:04,472][105620] Updated weights for policy 1, policy_version 1949610 (0.0009) [2023-12-27 05:34:04,641][105692] Updated weights for policy 0, policy_version 1944788 (0.0009) [2023-12-27 05:34:04,701][105692] Updated weights for policy 0, policy_version 1944798 (0.0010) [2023-12-27 05:34:04,761][105692] Updated weights for policy 0, policy_version 1944808 (0.0011) [2023-12-27 05:34:05,312][105620] Updated weights for policy 1, policy_version 1949620 (0.0007) [2023-12-27 05:34:05,323][105692] Updated weights for policy 0, policy_version 1944818 (0.0010) [2023-12-27 05:34:05,360][105620] Updated weights for policy 1, policy_version 1949630 (0.0006) [2023-12-27 05:34:05,381][105692] Updated weights for policy 0, policy_version 1944828 (0.0010) [2023-12-27 05:34:05,406][105620] Updated weights for policy 1, policy_version 1949640 (0.0009) [2023-12-27 05:34:05,446][105692] Updated weights for policy 0, policy_version 1944838 (0.0009) [2023-12-27 05:34:05,506][105692] Updated weights for policy 0, policy_version 1944848 (0.0008) [2023-12-27 05:34:06,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 997130240. Throughput: 0: 9672.0, 1: 9673.7. Samples: 997122328. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:34:06,062][104569] Avg episode reward: [(0, '7996.384'), (1, '9254.079')] [2023-12-27 05:34:06,144][105692] Updated weights for policy 0, policy_version 1944858 (0.0011) [2023-12-27 05:34:06,208][105692] Updated weights for policy 0, policy_version 1944868 (0.0011) [2023-12-27 05:34:06,220][105620] Updated weights for policy 1, policy_version 1949650 (0.0008) [2023-12-27 05:34:06,268][105692] Updated weights for policy 0, policy_version 1944878 (0.0011) [2023-12-27 05:34:06,282][105620] Updated weights for policy 1, policy_version 1949660 (0.0008) [2023-12-27 05:34:06,338][105620] Updated weights for policy 1, policy_version 1949670 (0.0008) [2023-12-27 05:34:06,394][105620] Updated weights for policy 1, policy_version 1949680 (0.0008) [2023-12-27 05:34:06,936][105692] Updated weights for policy 0, policy_version 1944888 (0.0008) [2023-12-27 05:34:07,000][105692] Updated weights for policy 0, policy_version 1944898 (0.0008) [2023-12-27 05:34:07,069][105692] Updated weights for policy 0, policy_version 1944908 (0.0008) [2023-12-27 05:34:07,183][105620] Updated weights for policy 1, policy_version 1949690 (0.0005) [2023-12-27 05:34:07,250][105620] Updated weights for policy 1, policy_version 1949700 (0.0006) [2023-12-27 05:34:07,307][105620] Updated weights for policy 1, policy_version 1949710 (0.0009) [2023-12-27 05:34:07,731][105692] Updated weights for policy 0, policy_version 1944918 (0.0006) [2023-12-27 05:34:07,789][105692] Updated weights for policy 0, policy_version 1944928 (0.0005) [2023-12-27 05:34:07,835][105692] Updated weights for policy 0, policy_version 1944938 (0.0005) [2023-12-27 05:34:08,031][105620] Updated weights for policy 1, policy_version 1949720 (0.0008) [2023-12-27 05:34:08,083][105620] Updated weights for policy 1, policy_version 1949730 (0.0008) [2023-12-27 05:34:08,137][105620] Updated weights for policy 1, policy_version 1949740 (0.0008) [2023-12-27 05:34:08,512][105692] Updated weights for policy 0, policy_version 1944948 (0.0007) [2023-12-27 05:34:08,564][105692] Updated weights for policy 0, policy_version 1944958 (0.0010) [2023-12-27 05:34:08,625][105692] Updated weights for policy 0, policy_version 1944968 (0.0010) [2023-12-27 05:34:08,835][105620] Updated weights for policy 1, policy_version 1949750 (0.0006) [2023-12-27 05:34:08,895][105620] Updated weights for policy 1, policy_version 1949760 (0.0005) [2023-12-27 05:34:08,954][105620] Updated weights for policy 1, policy_version 1949770 (0.0005) [2023-12-27 05:34:09,393][105692] Updated weights for policy 0, policy_version 1944978 (0.0010) [2023-12-27 05:34:09,461][105692] Updated weights for policy 0, policy_version 1944988 (0.0011) [2023-12-27 05:34:09,530][105692] Updated weights for policy 0, policy_version 1944998 (0.0011) [2023-12-27 05:34:09,544][105620] Updated weights for policy 1, policy_version 1949780 (0.0006) [2023-12-27 05:34:09,586][105692] Updated weights for policy 0, policy_version 1945008 (0.0011) [2023-12-27 05:34:09,597][105620] Updated weights for policy 1, policy_version 1949790 (0.0006) [2023-12-27 05:34:09,656][105620] Updated weights for policy 1, policy_version 1949800 (0.0010) [2023-12-27 05:34:10,301][105692] Updated weights for policy 0, policy_version 1945018 (0.0008) [2023-12-27 05:34:10,364][105692] Updated weights for policy 0, policy_version 1945028 (0.0008) [2023-12-27 05:34:10,427][105692] Updated weights for policy 0, policy_version 1945038 (0.0008) [2023-12-27 05:34:10,469][105620] Updated weights for policy 1, policy_version 1949810 (0.0009) [2023-12-27 05:34:10,523][105620] Updated weights for policy 1, policy_version 1949820 (0.0005) [2023-12-27 05:34:10,578][105620] Updated weights for policy 1, policy_version 1949830 (0.0005) [2023-12-27 05:34:10,630][105620] Updated weights for policy 1, policy_version 1949840 (0.0006) [2023-12-27 05:34:11,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19438.7). Total num frames: 997228544. Throughput: 0: 9845.7, 1: 9705.2. Samples: 997240140. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:34:11,062][104569] Avg episode reward: [(0, '8266.157'), (1, '9346.275')] [2023-12-27 05:34:11,176][105692] Updated weights for policy 0, policy_version 1945048 (0.0008) [2023-12-27 05:34:11,246][105692] Updated weights for policy 0, policy_version 1945058 (0.0007) [2023-12-27 05:34:11,311][105692] Updated weights for policy 0, policy_version 1945068 (0.0010) [2023-12-27 05:34:11,356][105620] Updated weights for policy 1, policy_version 1949850 (0.0007) [2023-12-27 05:34:11,424][105620] Updated weights for policy 1, policy_version 1949860 (0.0008) [2023-12-27 05:34:11,487][105620] Updated weights for policy 1, policy_version 1949870 (0.0009) [2023-12-27 05:34:12,056][105692] Updated weights for policy 0, policy_version 1945078 (0.0009) [2023-12-27 05:34:12,113][105692] Updated weights for policy 0, policy_version 1945088 (0.0008) [2023-12-27 05:34:12,177][105692] Updated weights for policy 0, policy_version 1945098 (0.0009) [2023-12-27 05:34:12,221][105620] Updated weights for policy 1, policy_version 1949880 (0.0010) [2023-12-27 05:34:12,283][105620] Updated weights for policy 1, policy_version 1949890 (0.0009) [2023-12-27 05:34:12,339][105620] Updated weights for policy 1, policy_version 1949900 (0.0008) [2023-12-27 05:34:12,986][105620] Updated weights for policy 1, policy_version 1949910 (0.0009) [2023-12-27 05:34:13,014][105692] Updated weights for policy 0, policy_version 1945108 (0.0008) [2023-12-27 05:34:13,040][105620] Updated weights for policy 1, policy_version 1949920 (0.0007) [2023-12-27 05:34:13,075][105692] Updated weights for policy 0, policy_version 1945118 (0.0007) [2023-12-27 05:34:13,093][105620] Updated weights for policy 1, policy_version 1949930 (0.0007) [2023-12-27 05:34:13,134][105692] Updated weights for policy 0, policy_version 1945128 (0.0009) [2023-12-27 05:34:13,777][105620] Updated weights for policy 1, policy_version 1949940 (0.0008) [2023-12-27 05:34:13,829][105620] Updated weights for policy 1, policy_version 1949950 (0.0009) [2023-12-27 05:34:13,899][105620] Updated weights for policy 1, policy_version 1949960 (0.0008) [2023-12-27 05:34:13,915][105692] Updated weights for policy 0, policy_version 1945138 (0.0009) [2023-12-27 05:34:13,971][105692] Updated weights for policy 0, policy_version 1945148 (0.0008) [2023-12-27 05:34:14,026][105692] Updated weights for policy 0, policy_version 1945158 (0.0009) [2023-12-27 05:34:14,080][105692] Updated weights for policy 0, policy_version 1945168 (0.0008) [2023-12-27 05:34:14,548][105620] Updated weights for policy 1, policy_version 1949970 (0.0007) [2023-12-27 05:34:14,596][105620] Updated weights for policy 1, policy_version 1949980 (0.0006) [2023-12-27 05:34:14,650][105620] Updated weights for policy 1, policy_version 1949990 (0.0005) [2023-12-27 05:34:14,698][105620] Updated weights for policy 1, policy_version 1950000 (0.0005) [2023-12-27 05:34:14,933][105692] Updated weights for policy 0, policy_version 1945178 (0.0009) [2023-12-27 05:34:14,992][105692] Updated weights for policy 0, policy_version 1945188 (0.0009) [2023-12-27 05:34:15,053][105692] Updated weights for policy 0, policy_version 1945198 (0.0009) [2023-12-27 05:34:15,441][105620] Updated weights for policy 1, policy_version 1950010 (0.0006) [2023-12-27 05:34:15,506][105620] Updated weights for policy 1, policy_version 1950020 (0.0005) [2023-12-27 05:34:15,564][105620] Updated weights for policy 1, policy_version 1950030 (0.0007) [2023-12-27 05:34:15,825][105692] Updated weights for policy 0, policy_version 1945208 (0.0008) [2023-12-27 05:34:15,883][105692] Updated weights for policy 0, policy_version 1945218 (0.0010) [2023-12-27 05:34:15,940][105692] Updated weights for policy 0, policy_version 1945228 (0.0010) [2023-12-27 05:34:16,062][104569] Fps is (10 sec: 19660.5, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 997326848. Throughput: 0: 9744.1, 1: 9640.9. Samples: 997295720. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:34:16,062][104569] Avg episode reward: [(0, '8265.491'), (1, '9346.293')] [2023-12-27 05:34:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001950032_499277824.pth... [2023-12-27 05:34:16,070][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001945232_498049024.pth... [2023-12-27 05:34:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001948912_498991104.pth [2023-12-27 05:34:16,078][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001944112_497762304.pth [2023-12-27 05:34:16,168][105620] Updated weights for policy 1, policy_version 1950040 (0.0008) [2023-12-27 05:34:16,226][105620] Updated weights for policy 1, policy_version 1950050 (0.0009) [2023-12-27 05:34:16,279][105620] Updated weights for policy 1, policy_version 1950060 (0.0009) [2023-12-27 05:34:16,717][105692] Updated weights for policy 0, policy_version 1945238 (0.0009) [2023-12-27 05:34:16,770][105692] Updated weights for policy 0, policy_version 1945248 (0.0008) [2023-12-27 05:34:16,827][105692] Updated weights for policy 0, policy_version 1945258 (0.0008) [2023-12-27 05:34:17,035][105620] Updated weights for policy 1, policy_version 1950070 (0.0009) [2023-12-27 05:34:17,083][105620] Updated weights for policy 1, policy_version 1950080 (0.0009) [2023-12-27 05:34:17,130][105620] Updated weights for policy 1, policy_version 1950090 (0.0009) [2023-12-27 05:34:17,575][105692] Updated weights for policy 0, policy_version 1945268 (0.0009) [2023-12-27 05:34:17,631][105692] Updated weights for policy 0, policy_version 1945278 (0.0008) [2023-12-27 05:34:17,698][105692] Updated weights for policy 0, policy_version 1945288 (0.0006) [2023-12-27 05:34:17,841][105620] Updated weights for policy 1, policy_version 1950100 (0.0008) [2023-12-27 05:34:17,911][105620] Updated weights for policy 1, policy_version 1950110 (0.0010) [2023-12-27 05:34:17,982][105620] Updated weights for policy 1, policy_version 1950120 (0.0010) [2023-12-27 05:34:18,398][105692] Updated weights for policy 0, policy_version 1945298 (0.0010) [2023-12-27 05:34:18,453][105692] Updated weights for policy 0, policy_version 1945308 (0.0009) [2023-12-27 05:34:18,512][105692] Updated weights for policy 0, policy_version 1945318 (0.0009) [2023-12-27 05:34:18,571][105692] Updated weights for policy 0, policy_version 1945328 (0.0009) [2023-12-27 05:34:18,688][105620] Updated weights for policy 1, policy_version 1950130 (0.0009) [2023-12-27 05:34:18,755][105620] Updated weights for policy 1, policy_version 1950140 (0.0009) [2023-12-27 05:34:18,810][105620] Updated weights for policy 1, policy_version 1950150 (0.0009) [2023-12-27 05:34:18,868][105620] Updated weights for policy 1, policy_version 1950160 (0.0009) [2023-12-27 05:34:19,324][105692] Updated weights for policy 0, policy_version 1945338 (0.0009) [2023-12-27 05:34:19,397][105692] Updated weights for policy 0, policy_version 1945348 (0.0008) [2023-12-27 05:34:19,456][105692] Updated weights for policy 0, policy_version 1945358 (0.0009) [2023-12-27 05:34:19,624][105620] Updated weights for policy 1, policy_version 1950170 (0.0008) [2023-12-27 05:34:19,675][105620] Updated weights for policy 1, policy_version 1950180 (0.0009) [2023-12-27 05:34:19,733][105620] Updated weights for policy 1, policy_version 1950190 (0.0009) [2023-12-27 05:34:20,216][105692] Updated weights for policy 0, policy_version 1945368 (0.0009) [2023-12-27 05:34:20,270][105692] Updated weights for policy 0, policy_version 1945378 (0.0009) [2023-12-27 05:34:20,330][105692] Updated weights for policy 0, policy_version 1945388 (0.0009) [2023-12-27 05:34:20,519][105620] Updated weights for policy 1, policy_version 1950200 (0.0009) [2023-12-27 05:34:20,582][105620] Updated weights for policy 1, policy_version 1950210 (0.0008) [2023-12-27 05:34:20,649][105620] Updated weights for policy 1, policy_version 1950220 (0.0010) [2023-12-27 05:34:21,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 997416960. Throughput: 0: 9649.0, 1: 9682.2. Samples: 997409516. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:34:21,063][104569] Avg episode reward: [(0, '8539.817'), (1, '9346.324')] [2023-12-27 05:34:21,098][105692] Updated weights for policy 0, policy_version 1945398 (0.0009) [2023-12-27 05:34:21,164][105692] Updated weights for policy 0, policy_version 1945408 (0.0008) [2023-12-27 05:34:21,226][105692] Updated weights for policy 0, policy_version 1945418 (0.0010) [2023-12-27 05:34:21,424][105620] Updated weights for policy 1, policy_version 1950230 (0.0008) [2023-12-27 05:34:21,484][105620] Updated weights for policy 1, policy_version 1950240 (0.0008) [2023-12-27 05:34:21,549][105620] Updated weights for policy 1, policy_version 1950250 (0.0008) [2023-12-27 05:34:22,057][105692] Updated weights for policy 0, policy_version 1945428 (0.0009) [2023-12-27 05:34:22,120][105692] Updated weights for policy 0, policy_version 1945438 (0.0009) [2023-12-27 05:34:22,185][105692] Updated weights for policy 0, policy_version 1945448 (0.0009) [2023-12-27 05:34:22,287][105620] Updated weights for policy 1, policy_version 1950260 (0.0009) [2023-12-27 05:34:22,349][105620] Updated weights for policy 1, policy_version 1950270 (0.0009) [2023-12-27 05:34:22,417][105620] Updated weights for policy 1, policy_version 1950280 (0.0010) [2023-12-27 05:34:22,943][105692] Updated weights for policy 0, policy_version 1945458 (0.0009) [2023-12-27 05:34:23,006][105692] Updated weights for policy 0, policy_version 1945468 (0.0009) [2023-12-27 05:34:23,071][105692] Updated weights for policy 0, policy_version 1945478 (0.0009) [2023-12-27 05:34:23,138][105692] Updated weights for policy 0, policy_version 1945488 (0.0009) [2023-12-27 05:34:23,150][105620] Updated weights for policy 1, policy_version 1950290 (0.0009) [2023-12-27 05:34:23,214][105620] Updated weights for policy 1, policy_version 1950300 (0.0008) [2023-12-27 05:34:23,282][105620] Updated weights for policy 1, policy_version 1950310 (0.0008) [2023-12-27 05:34:23,339][105620] Updated weights for policy 1, policy_version 1950320 (0.0006) [2023-12-27 05:34:23,909][105692] Updated weights for policy 0, policy_version 1945498 (0.0009) [2023-12-27 05:34:23,967][105692] Updated weights for policy 0, policy_version 1945508 (0.0009) [2023-12-27 05:34:24,021][105692] Updated weights for policy 0, policy_version 1945518 (0.0007) [2023-12-27 05:34:24,027][105620] Updated weights for policy 1, policy_version 1950330 (0.0007) [2023-12-27 05:34:24,072][105620] Updated weights for policy 1, policy_version 1950340 (0.0008) [2023-12-27 05:34:24,130][105620] Updated weights for policy 1, policy_version 1950350 (0.0010) [2023-12-27 05:34:24,772][105692] Updated weights for policy 0, policy_version 1945528 (0.0009) [2023-12-27 05:34:24,827][105692] Updated weights for policy 0, policy_version 1945538 (0.0009) [2023-12-27 05:34:24,875][105692] Updated weights for policy 0, policy_version 1945548 (0.0008) [2023-12-27 05:34:24,898][105620] Updated weights for policy 1, policy_version 1950360 (0.0007) [2023-12-27 05:34:24,952][105620] Updated weights for policy 1, policy_version 1950370 (0.0009) [2023-12-27 05:34:25,010][105620] Updated weights for policy 1, policy_version 1950380 (0.0009) [2023-12-27 05:34:25,660][105692] Updated weights for policy 0, policy_version 1945558 (0.0007) [2023-12-27 05:34:25,711][105620] Updated weights for policy 1, policy_version 1950390 (0.0007) [2023-12-27 05:34:25,718][105692] Updated weights for policy 0, policy_version 1945568 (0.0006) [2023-12-27 05:34:25,773][105620] Updated weights for policy 1, policy_version 1950400 (0.0009) [2023-12-27 05:34:25,781][105692] Updated weights for policy 0, policy_version 1945578 (0.0010) [2023-12-27 05:34:25,832][105620] Updated weights for policy 1, policy_version 1950410 (0.0007) [2023-12-27 05:34:26,062][104569] Fps is (10 sec: 18841.3, 60 sec: 19387.6, 300 sec: 19410.9). Total num frames: 997515264. Throughput: 0: 9501.4, 1: 9626.1. Samples: 997521080. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:34:26,063][104569] Avg episode reward: [(0, '8633.388'), (1, '9346.312')] [2023-12-27 05:34:26,493][105692] Updated weights for policy 0, policy_version 1945588 (0.0011) [2023-12-27 05:34:26,545][105620] Updated weights for policy 1, policy_version 1950420 (0.0007) [2023-12-27 05:34:26,554][105692] Updated weights for policy 0, policy_version 1945598 (0.0010) [2023-12-27 05:34:26,600][105620] Updated weights for policy 1, policy_version 1950430 (0.0005) [2023-12-27 05:34:26,610][105692] Updated weights for policy 0, policy_version 1945608 (0.0008) [2023-12-27 05:34:26,658][105620] Updated weights for policy 1, policy_version 1950440 (0.0009) [2023-12-27 05:34:27,212][105692] Updated weights for policy 0, policy_version 1945618 (0.0006) [2023-12-27 05:34:27,280][105692] Updated weights for policy 0, policy_version 1945628 (0.0005) [2023-12-27 05:34:27,343][105692] Updated weights for policy 0, policy_version 1945638 (0.0010) [2023-12-27 05:34:27,390][105692] Updated weights for policy 0, policy_version 1945648 (0.0010) [2023-12-27 05:34:27,432][105620] Updated weights for policy 1, policy_version 1950450 (0.0008) [2023-12-27 05:34:27,482][105620] Updated weights for policy 1, policy_version 1950460 (0.0008) [2023-12-27 05:34:27,528][105620] Updated weights for policy 1, policy_version 1950470 (0.0007) [2023-12-27 05:34:27,578][105620] Updated weights for policy 1, policy_version 1950480 (0.0009) [2023-12-27 05:34:27,985][105692] Updated weights for policy 0, policy_version 1945658 (0.0006) [2023-12-27 05:34:28,037][105692] Updated weights for policy 0, policy_version 1945668 (0.0005) [2023-12-27 05:34:28,080][105692] Updated weights for policy 0, policy_version 1945678 (0.0005) [2023-12-27 05:34:28,486][105620] Updated weights for policy 1, policy_version 1950490 (0.0009) [2023-12-27 05:34:28,542][105620] Updated weights for policy 1, policy_version 1950500 (0.0009) [2023-12-27 05:34:28,592][105620] Updated weights for policy 1, policy_version 1950510 (0.0009) [2023-12-27 05:34:28,673][105692] Updated weights for policy 0, policy_version 1945688 (0.0007) [2023-12-27 05:34:28,734][105692] Updated weights for policy 0, policy_version 1945698 (0.0005) [2023-12-27 05:34:28,790][105692] Updated weights for policy 0, policy_version 1945708 (0.0008) [2023-12-27 05:34:29,362][105620] Updated weights for policy 1, policy_version 1950520 (0.0008) [2023-12-27 05:34:29,423][105620] Updated weights for policy 1, policy_version 1950530 (0.0009) [2023-12-27 05:34:29,476][105692] Updated weights for policy 0, policy_version 1945718 (0.0007) [2023-12-27 05:34:29,492][105620] Updated weights for policy 1, policy_version 1950540 (0.0009) [2023-12-27 05:34:29,534][105692] Updated weights for policy 0, policy_version 1945728 (0.0005) [2023-12-27 05:34:29,595][105692] Updated weights for policy 0, policy_version 1945738 (0.0006) [2023-12-27 05:34:30,178][105692] Updated weights for policy 0, policy_version 1945748 (0.0007) [2023-12-27 05:34:30,237][105692] Updated weights for policy 0, policy_version 1945758 (0.0010) [2023-12-27 05:34:30,291][105692] Updated weights for policy 0, policy_version 1945768 (0.0010) [2023-12-27 05:34:30,298][105620] Updated weights for policy 1, policy_version 1950550 (0.0007) [2023-12-27 05:34:30,356][105620] Updated weights for policy 1, policy_version 1950560 (0.0007) [2023-12-27 05:34:30,414][105620] Updated weights for policy 1, policy_version 1950570 (0.0008) [2023-12-27 05:34:31,012][105692] Updated weights for policy 0, policy_version 1945778 (0.0011) [2023-12-27 05:34:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19251.1, 300 sec: 19383.1). Total num frames: 997605376. Throughput: 0: 9614.1, 1: 9601.3. Samples: 997580008. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:34:31,063][104569] Avg episode reward: [(0, '8452.760'), (1, '9346.308')] [2023-12-27 05:34:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001950576_499417088.pth... [2023-12-27 05:34:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001949488_499138560.pth [2023-12-27 05:34:31,078][105692] Updated weights for policy 0, policy_version 1945788 (0.0008) [2023-12-27 05:34:31,150][105692] Updated weights for policy 0, policy_version 1945798 (0.0007) [2023-12-27 05:34:31,184][105620] Updated weights for policy 1, policy_version 1950580 (0.0009) [2023-12-27 05:34:31,210][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001945808_498196480.pth... [2023-12-27 05:34:31,212][105692] Updated weights for policy 0, policy_version 1945808 (0.0006) [2023-12-27 05:34:31,215][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001944656_497901568.pth [2023-12-27 05:34:31,237][105620] Updated weights for policy 1, policy_version 1950590 (0.0009) [2023-12-27 05:34:31,301][105620] Updated weights for policy 1, policy_version 1950600 (0.0008) [2023-12-27 05:34:31,930][105692] Updated weights for policy 0, policy_version 1945818 (0.0011) [2023-12-27 05:34:31,993][105692] Updated weights for policy 0, policy_version 1945828 (0.0011) [2023-12-27 05:34:32,053][105692] Updated weights for policy 0, policy_version 1945838 (0.0011) [2023-12-27 05:34:32,091][105620] Updated weights for policy 1, policy_version 1950610 (0.0009) [2023-12-27 05:34:32,149][105620] Updated weights for policy 1, policy_version 1950620 (0.0008) [2023-12-27 05:34:32,210][105620] Updated weights for policy 1, policy_version 1950630 (0.0009) [2023-12-27 05:34:32,266][105620] Updated weights for policy 1, policy_version 1950640 (0.0008) [2023-12-27 05:34:32,777][105692] Updated weights for policy 0, policy_version 1945848 (0.0010) [2023-12-27 05:34:32,823][105692] Updated weights for policy 0, policy_version 1945858 (0.0007) [2023-12-27 05:34:32,869][105692] Updated weights for policy 0, policy_version 1945868 (0.0006) [2023-12-27 05:34:33,096][105620] Updated weights for policy 1, policy_version 1950650 (0.0009) [2023-12-27 05:34:33,144][105620] Updated weights for policy 1, policy_version 1950660 (0.0009) [2023-12-27 05:34:33,195][105620] Updated weights for policy 1, policy_version 1950670 (0.0009) [2023-12-27 05:34:33,575][105692] Updated weights for policy 0, policy_version 1945878 (0.0005) [2023-12-27 05:34:33,624][105692] Updated weights for policy 0, policy_version 1945888 (0.0005) [2023-12-27 05:34:33,675][105692] Updated weights for policy 0, policy_version 1945898 (0.0005) [2023-12-27 05:34:34,051][105620] Updated weights for policy 1, policy_version 1950680 (0.0009) [2023-12-27 05:34:34,106][105620] Updated weights for policy 1, policy_version 1950690 (0.0009) [2023-12-27 05:34:34,173][105620] Updated weights for policy 1, policy_version 1950700 (0.0009) [2023-12-27 05:34:34,251][105692] Updated weights for policy 0, policy_version 1945908 (0.0007) [2023-12-27 05:34:34,318][105692] Updated weights for policy 0, policy_version 1945918 (0.0009) [2023-12-27 05:34:34,388][105692] Updated weights for policy 0, policy_version 1945928 (0.0008) [2023-12-27 05:34:34,971][105620] Updated weights for policy 1, policy_version 1950710 (0.0007) [2023-12-27 05:34:35,030][105620] Updated weights for policy 1, policy_version 1950720 (0.0007) [2023-12-27 05:34:35,083][105620] Updated weights for policy 1, policy_version 1950730 (0.0010) [2023-12-27 05:34:35,089][105692] Updated weights for policy 0, policy_version 1945938 (0.0009) [2023-12-27 05:34:35,143][105692] Updated weights for policy 0, policy_version 1945948 (0.0006) [2023-12-27 05:34:35,199][105692] Updated weights for policy 0, policy_version 1945958 (0.0007) [2023-12-27 05:34:35,246][105692] Updated weights for policy 0, policy_version 1945968 (0.0008) [2023-12-27 05:34:35,768][105620] Updated weights for policy 1, policy_version 1950740 (0.0010) [2023-12-27 05:34:35,816][105620] Updated weights for policy 1, policy_version 1950750 (0.0010) [2023-12-27 05:34:35,861][105620] Updated weights for policy 1, policy_version 1950760 (0.0008) [2023-12-27 05:34:36,015][105692] Updated weights for policy 0, policy_version 1945978 (0.0010) [2023-12-27 05:34:36,062][104569] Fps is (10 sec: 18842.1, 60 sec: 19114.7, 300 sec: 19383.1). Total num frames: 997703680. Throughput: 0: 9606.1, 1: 9467.4. Samples: 997693356. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:34:36,062][104569] Avg episode reward: [(0, '8446.223'), (1, '9256.350')] [2023-12-27 05:34:36,078][105692] Updated weights for policy 0, policy_version 1945988 (0.0009) [2023-12-27 05:34:36,137][105692] Updated weights for policy 0, policy_version 1945998 (0.0008) [2023-12-27 05:34:36,568][105620] Updated weights for policy 1, policy_version 1950770 (0.0007) [2023-12-27 05:34:36,623][105620] Updated weights for policy 1, policy_version 1950780 (0.0009) [2023-12-27 05:34:36,670][105620] Updated weights for policy 1, policy_version 1950790 (0.0008) [2023-12-27 05:34:36,731][105620] Updated weights for policy 1, policy_version 1950800 (0.0009) [2023-12-27 05:34:36,893][105692] Updated weights for policy 0, policy_version 1946008 (0.0007) [2023-12-27 05:34:36,953][105692] Updated weights for policy 0, policy_version 1946018 (0.0007) [2023-12-27 05:34:37,014][105692] Updated weights for policy 0, policy_version 1946028 (0.0009) [2023-12-27 05:34:37,545][105620] Updated weights for policy 1, policy_version 1950810 (0.0009) [2023-12-27 05:34:37,601][105620] Updated weights for policy 1, policy_version 1950820 (0.0008) [2023-12-27 05:34:37,651][105620] Updated weights for policy 1, policy_version 1950830 (0.0009) [2023-12-27 05:34:37,723][105692] Updated weights for policy 0, policy_version 1946038 (0.0008) [2023-12-27 05:34:37,779][105692] Updated weights for policy 0, policy_version 1946048 (0.0005) [2023-12-27 05:34:37,836][105692] Updated weights for policy 0, policy_version 1946058 (0.0006) [2023-12-27 05:34:38,448][105620] Updated weights for policy 1, policy_version 1950840 (0.0009) [2023-12-27 05:34:38,506][105620] Updated weights for policy 1, policy_version 1950850 (0.0009) [2023-12-27 05:34:38,533][105692] Updated weights for policy 0, policy_version 1946068 (0.0008) [2023-12-27 05:34:38,564][105620] Updated weights for policy 1, policy_version 1950860 (0.0007) [2023-12-27 05:34:38,587][105692] Updated weights for policy 0, policy_version 1946078 (0.0007) [2023-12-27 05:34:38,647][105692] Updated weights for policy 0, policy_version 1946088 (0.0010) [2023-12-27 05:34:39,329][105620] Updated weights for policy 1, policy_version 1950870 (0.0008) [2023-12-27 05:34:39,395][105620] Updated weights for policy 1, policy_version 1950880 (0.0009) [2023-12-27 05:34:39,406][105692] Updated weights for policy 0, policy_version 1946098 (0.0009) [2023-12-27 05:34:39,464][105620] Updated weights for policy 1, policy_version 1950890 (0.0008) [2023-12-27 05:34:39,471][105692] Updated weights for policy 0, policy_version 1946108 (0.0006) [2023-12-27 05:34:39,522][105692] Updated weights for policy 0, policy_version 1946118 (0.0010) [2023-12-27 05:34:39,578][105692] Updated weights for policy 0, policy_version 1946128 (0.0008) [2023-12-27 05:34:40,260][105620] Updated weights for policy 1, policy_version 1950900 (0.0009) [2023-12-27 05:34:40,322][105620] Updated weights for policy 1, policy_version 1950910 (0.0009) [2023-12-27 05:34:40,331][105692] Updated weights for policy 0, policy_version 1946138 (0.0006) [2023-12-27 05:34:40,384][105620] Updated weights for policy 1, policy_version 1950920 (0.0007) [2023-12-27 05:34:40,398][105692] Updated weights for policy 0, policy_version 1946148 (0.0007) [2023-12-27 05:34:40,462][105692] Updated weights for policy 0, policy_version 1946158 (0.0007) [2023-12-27 05:34:41,062][104569] Fps is (10 sec: 18841.7, 60 sec: 18978.1, 300 sec: 19355.4). Total num frames: 997793792. Throughput: 0: 9647.0, 1: 9358.6. Samples: 997806200. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:34:41,063][104569] Avg episode reward: [(0, '8080.176'), (1, '9163.932')] [2023-12-27 05:34:41,128][105692] Updated weights for policy 0, policy_version 1946168 (0.0008) [2023-12-27 05:34:41,198][105692] Updated weights for policy 0, policy_version 1946178 (0.0007) [2023-12-27 05:34:41,207][105620] Updated weights for policy 1, policy_version 1950930 (0.0009) [2023-12-27 05:34:41,263][105692] Updated weights for policy 0, policy_version 1946188 (0.0006) [2023-12-27 05:34:41,270][105620] Updated weights for policy 1, policy_version 1950940 (0.0008) [2023-12-27 05:34:41,325][105620] Updated weights for policy 1, policy_version 1950950 (0.0008) [2023-12-27 05:34:41,392][105620] Updated weights for policy 1, policy_version 1950960 (0.0010) [2023-12-27 05:34:41,936][105692] Updated weights for policy 0, policy_version 1946198 (0.0008) [2023-12-27 05:34:42,005][105692] Updated weights for policy 0, policy_version 1946208 (0.0008) [2023-12-27 05:34:42,068][105692] Updated weights for policy 0, policy_version 1946218 (0.0006) [2023-12-27 05:34:42,209][105620] Updated weights for policy 1, policy_version 1950970 (0.0011) [2023-12-27 05:34:42,274][105620] Updated weights for policy 1, policy_version 1950980 (0.0011) [2023-12-27 05:34:42,338][105620] Updated weights for policy 1, policy_version 1950990 (0.0011) [2023-12-27 05:34:42,771][105692] Updated weights for policy 0, policy_version 1946228 (0.0007) [2023-12-27 05:34:42,838][105692] Updated weights for policy 0, policy_version 1946238 (0.0009) [2023-12-27 05:34:42,899][105692] Updated weights for policy 0, policy_version 1946248 (0.0008) [2023-12-27 05:34:43,063][105620] Updated weights for policy 1, policy_version 1951000 (0.0010) [2023-12-27 05:34:43,115][105620] Updated weights for policy 1, policy_version 1951010 (0.0010) [2023-12-27 05:34:43,163][105620] Updated weights for policy 1, policy_version 1951020 (0.0010) [2023-12-27 05:34:43,655][105692] Updated weights for policy 0, policy_version 1946258 (0.0008) [2023-12-27 05:34:43,711][105692] Updated weights for policy 0, policy_version 1946268 (0.0008) [2023-12-27 05:34:43,771][105692] Updated weights for policy 0, policy_version 1946278 (0.0008) [2023-12-27 05:34:43,839][105692] Updated weights for policy 0, policy_version 1946288 (0.0009) [2023-12-27 05:34:43,952][105620] Updated weights for policy 1, policy_version 1951030 (0.0010) [2023-12-27 05:34:44,010][105620] Updated weights for policy 1, policy_version 1951040 (0.0010) [2023-12-27 05:34:44,069][105620] Updated weights for policy 1, policy_version 1951050 (0.0011) [2023-12-27 05:34:44,611][105692] Updated weights for policy 0, policy_version 1946298 (0.0010) [2023-12-27 05:34:44,665][105692] Updated weights for policy 0, policy_version 1946309 (0.0009) [2023-12-27 05:34:44,713][105692] Updated weights for policy 0, policy_version 1946319 (0.0009) [2023-12-27 05:34:44,746][105620] Updated weights for policy 1, policy_version 1951060 (0.0008) [2023-12-27 05:34:44,812][105620] Updated weights for policy 1, policy_version 1951070 (0.0006) [2023-12-27 05:34:44,869][105620] Updated weights for policy 1, policy_version 1951080 (0.0011) [2023-12-27 05:34:45,545][105692] Updated weights for policy 0, policy_version 1946329 (0.0007) [2023-12-27 05:34:45,598][105692] Updated weights for policy 0, policy_version 1946339 (0.0011) [2023-12-27 05:34:45,598][105620] Updated weights for policy 1, policy_version 1951090 (0.0011) [2023-12-27 05:34:45,648][105620] Updated weights for policy 1, policy_version 1951100 (0.0011) [2023-12-27 05:34:45,657][105692] Updated weights for policy 0, policy_version 1946349 (0.0011) [2023-12-27 05:34:45,700][105620] Updated weights for policy 1, policy_version 1951110 (0.0010) [2023-12-27 05:34:45,745][105620] Updated weights for policy 1, policy_version 1951120 (0.0010) [2023-12-27 05:34:46,062][104569] Fps is (10 sec: 18841.5, 60 sec: 18978.2, 300 sec: 19383.1). Total num frames: 997892096. Throughput: 0: 9630.0, 1: 9296.6. Samples: 997862664. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:34:46,062][104569] Avg episode reward: [(0, '8263.202'), (1, '9163.845')] [2023-12-27 05:34:46,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001951120_499556352.pth... [2023-12-27 05:34:46,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001946352_498335744.pth... [2023-12-27 05:34:46,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001950032_499277824.pth [2023-12-27 05:34:46,076][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001945232_498049024.pth [2023-12-27 05:34:46,357][105620] Updated weights for policy 1, policy_version 1951130 (0.0005) [2023-12-27 05:34:46,375][105692] Updated weights for policy 0, policy_version 1946359 (0.0010) [2023-12-27 05:34:46,424][105620] Updated weights for policy 1, policy_version 1951140 (0.0005) [2023-12-27 05:34:46,434][105692] Updated weights for policy 0, policy_version 1946369 (0.0011) [2023-12-27 05:34:46,479][105620] Updated weights for policy 1, policy_version 1951150 (0.0006) [2023-12-27 05:34:46,494][105692] Updated weights for policy 0, policy_version 1946379 (0.0011) [2023-12-27 05:34:47,055][105620] Updated weights for policy 1, policy_version 1951160 (0.0006) [2023-12-27 05:34:47,125][105620] Updated weights for policy 1, policy_version 1951170 (0.0005) [2023-12-27 05:34:47,181][105620] Updated weights for policy 1, policy_version 1951180 (0.0010) [2023-12-27 05:34:47,183][105692] Updated weights for policy 0, policy_version 1946389 (0.0010) [2023-12-27 05:34:47,239][105692] Updated weights for policy 0, policy_version 1946399 (0.0011) [2023-12-27 05:34:47,293][105692] Updated weights for policy 0, policy_version 1946409 (0.0010) [2023-12-27 05:34:47,854][105620] Updated weights for policy 1, policy_version 1951190 (0.0010) [2023-12-27 05:34:47,903][105620] Updated weights for policy 1, policy_version 1951200 (0.0010) [2023-12-27 05:34:47,951][105692] Updated weights for policy 0, policy_version 1946419 (0.0009) [2023-12-27 05:34:47,966][105620] Updated weights for policy 1, policy_version 1951210 (0.0011) [2023-12-27 05:34:47,999][105692] Updated weights for policy 0, policy_version 1946429 (0.0005) [2023-12-27 05:34:48,058][105692] Updated weights for policy 0, policy_version 1946439 (0.0008) [2023-12-27 05:34:48,675][105620] Updated weights for policy 1, policy_version 1951220 (0.0009) [2023-12-27 05:34:48,717][105692] Updated weights for policy 0, policy_version 1946449 (0.0008) [2023-12-27 05:34:48,738][105620] Updated weights for policy 1, policy_version 1951230 (0.0008) [2023-12-27 05:34:48,774][105692] Updated weights for policy 0, policy_version 1946459 (0.0005) [2023-12-27 05:34:48,791][105620] Updated weights for policy 1, policy_version 1951240 (0.0011) [2023-12-27 05:34:48,826][105692] Updated weights for policy 0, policy_version 1946469 (0.0005) [2023-12-27 05:34:48,878][105692] Updated weights for policy 0, policy_version 1946479 (0.0008) [2023-12-27 05:34:49,488][105692] Updated weights for policy 0, policy_version 1946489 (0.0010) [2023-12-27 05:34:49,515][105620] Updated weights for policy 1, policy_version 1951250 (0.0010) [2023-12-27 05:34:49,543][105692] Updated weights for policy 0, policy_version 1946499 (0.0011) [2023-12-27 05:34:49,573][105620] Updated weights for policy 1, policy_version 1951260 (0.0008) [2023-12-27 05:34:49,599][105692] Updated weights for policy 0, policy_version 1946509 (0.0010) [2023-12-27 05:34:49,627][105620] Updated weights for policy 1, policy_version 1951270 (0.0009) [2023-12-27 05:34:49,682][105620] Updated weights for policy 1, policy_version 1951280 (0.0011) [2023-12-27 05:34:50,353][105692] Updated weights for policy 0, policy_version 1946519 (0.0009) [2023-12-27 05:34:50,415][105692] Updated weights for policy 0, policy_version 1946529 (0.0008) [2023-12-27 05:34:50,436][105620] Updated weights for policy 1, policy_version 1951290 (0.0011) [2023-12-27 05:34:50,475][105692] Updated weights for policy 0, policy_version 1946539 (0.0005) [2023-12-27 05:34:50,485][105620] Updated weights for policy 1, policy_version 1951300 (0.0010) [2023-12-27 05:34:50,537][105620] Updated weights for policy 1, policy_version 1951310 (0.0011) [2023-12-27 05:34:51,062][104569] Fps is (10 sec: 19661.0, 60 sec: 18978.2, 300 sec: 19383.1). Total num frames: 997990400. Throughput: 0: 9682.4, 1: 9430.7. Samples: 997982416. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:34:51,062][104569] Avg episode reward: [(0, '8543.813'), (1, '9253.705')] [2023-12-27 05:34:51,258][105692] Updated weights for policy 0, policy_version 1946549 (0.0007) [2023-12-27 05:34:51,321][105692] Updated weights for policy 0, policy_version 1946559 (0.0008) [2023-12-27 05:34:51,330][105620] Updated weights for policy 1, policy_version 1951320 (0.0011) [2023-12-27 05:34:51,391][105692] Updated weights for policy 0, policy_version 1946569 (0.0008) [2023-12-27 05:34:51,394][105620] Updated weights for policy 1, policy_version 1951330 (0.0011) [2023-12-27 05:34:51,455][105620] Updated weights for policy 1, policy_version 1951340 (0.0011) [2023-12-27 05:34:52,192][105620] Updated weights for policy 1, policy_version 1951350 (0.0009) [2023-12-27 05:34:52,193][105692] Updated weights for policy 0, policy_version 1946579 (0.0007) [2023-12-27 05:34:52,258][105692] Updated weights for policy 0, policy_version 1946589 (0.0008) [2023-12-27 05:34:52,260][105620] Updated weights for policy 1, policy_version 1951360 (0.0008) [2023-12-27 05:34:52,315][105620] Updated weights for policy 1, policy_version 1951370 (0.0008) [2023-12-27 05:34:52,316][105692] Updated weights for policy 0, policy_version 1946599 (0.0011) [2023-12-27 05:34:52,971][105692] Updated weights for policy 0, policy_version 1946609 (0.0008) [2023-12-27 05:34:53,020][105620] Updated weights for policy 1, policy_version 1951380 (0.0010) [2023-12-27 05:34:53,037][105692] Updated weights for policy 0, policy_version 1946619 (0.0010) [2023-12-27 05:34:53,089][105620] Updated weights for policy 1, policy_version 1951390 (0.0010) [2023-12-27 05:34:53,090][105692] Updated weights for policy 0, policy_version 1946629 (0.0006) [2023-12-27 05:34:53,136][105692] Updated weights for policy 0, policy_version 1946639 (0.0005) [2023-12-27 05:34:53,137][105620] Updated weights for policy 1, policy_version 1951400 (0.0010) [2023-12-27 05:34:53,700][105620] Updated weights for policy 1, policy_version 1951410 (0.0008) [2023-12-27 05:34:53,759][105620] Updated weights for policy 1, policy_version 1951420 (0.0007) [2023-12-27 05:34:53,818][105620] Updated weights for policy 1, policy_version 1951430 (0.0010) [2023-12-27 05:34:53,820][105692] Updated weights for policy 0, policy_version 1946649 (0.0005) [2023-12-27 05:34:53,874][105620] Updated weights for policy 1, policy_version 1951440 (0.0010) [2023-12-27 05:34:53,874][105692] Updated weights for policy 0, policy_version 1946659 (0.0010) [2023-12-27 05:34:53,919][105692] Updated weights for policy 0, policy_version 1946669 (0.0008) [2023-12-27 05:34:54,509][105692] Updated weights for policy 0, policy_version 1946679 (0.0009) [2023-12-27 05:34:54,549][105620] Updated weights for policy 1, policy_version 1951450 (0.0008) [2023-12-27 05:34:54,556][105692] Updated weights for policy 0, policy_version 1946689 (0.0007) [2023-12-27 05:34:54,607][105620] Updated weights for policy 1, policy_version 1951460 (0.0008) [2023-12-27 05:34:54,612][105692] Updated weights for policy 0, policy_version 1946699 (0.0007) [2023-12-27 05:34:54,668][105620] Updated weights for policy 1, policy_version 1951470 (0.0008) [2023-12-27 05:34:55,236][105692] Updated weights for policy 0, policy_version 1946709 (0.0008) [2023-12-27 05:34:55,283][105692] Updated weights for policy 0, policy_version 1946719 (0.0010) [2023-12-27 05:34:55,334][105692] Updated weights for policy 0, policy_version 1946729 (0.0010) [2023-12-27 05:34:55,428][105620] Updated weights for policy 1, policy_version 1951480 (0.0010) [2023-12-27 05:34:55,482][105620] Updated weights for policy 1, policy_version 1951490 (0.0010) [2023-12-27 05:34:55,530][105620] Updated weights for policy 1, policy_version 1951500 (0.0010) [2023-12-27 05:34:56,025][105692] Updated weights for policy 0, policy_version 1946739 (0.0009) [2023-12-27 05:34:56,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19114.6, 300 sec: 19383.1). Total num frames: 998088704. Throughput: 0: 9658.5, 1: 9478.2. Samples: 998101292. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:34:56,063][104569] Avg episode reward: [(0, '8268.896'), (1, '9254.025')] [2023-12-27 05:34:56,079][105692] Updated weights for policy 0, policy_version 1946749 (0.0006) [2023-12-27 05:34:56,137][105692] Updated weights for policy 0, policy_version 1946759 (0.0010) [2023-12-27 05:34:56,264][105620] Updated weights for policy 1, policy_version 1951510 (0.0007) [2023-12-27 05:34:56,319][105620] Updated weights for policy 1, policy_version 1951520 (0.0005) [2023-12-27 05:34:56,370][105620] Updated weights for policy 1, policy_version 1951530 (0.0005) [2023-12-27 05:34:56,846][105692] Updated weights for policy 0, policy_version 1946769 (0.0010) [2023-12-27 05:34:56,896][105692] Updated weights for policy 0, policy_version 1946779 (0.0010) [2023-12-27 05:34:56,940][105692] Updated weights for policy 0, policy_version 1946789 (0.0010) [2023-12-27 05:34:56,988][105692] Updated weights for policy 0, policy_version 1946799 (0.0010) [2023-12-27 05:34:57,016][105620] Updated weights for policy 1, policy_version 1951540 (0.0007) [2023-12-27 05:34:57,067][105620] Updated weights for policy 1, policy_version 1951550 (0.0010) [2023-12-27 05:34:57,117][105620] Updated weights for policy 1, policy_version 1951560 (0.0010) [2023-12-27 05:34:57,655][105692] Updated weights for policy 0, policy_version 1946809 (0.0010) [2023-12-27 05:34:57,703][105692] Updated weights for policy 0, policy_version 1946819 (0.0006) [2023-12-27 05:34:57,748][105620] Updated weights for policy 1, policy_version 1951570 (0.0008) [2023-12-27 05:34:57,754][105692] Updated weights for policy 0, policy_version 1946829 (0.0011) [2023-12-27 05:34:57,799][105620] Updated weights for policy 1, policy_version 1951580 (0.0010) [2023-12-27 05:34:57,851][105620] Updated weights for policy 1, policy_version 1951590 (0.0009) [2023-12-27 05:34:57,912][105620] Updated weights for policy 1, policy_version 1951600 (0.0005) [2023-12-27 05:34:58,417][105692] Updated weights for policy 0, policy_version 1946839 (0.0011) [2023-12-27 05:34:58,480][105692] Updated weights for policy 0, policy_version 1946849 (0.0009) [2023-12-27 05:34:58,543][105692] Updated weights for policy 0, policy_version 1946859 (0.0008) [2023-12-27 05:34:58,602][105620] Updated weights for policy 1, policy_version 1951610 (0.0010) [2023-12-27 05:34:58,664][105620] Updated weights for policy 1, policy_version 1951620 (0.0009) [2023-12-27 05:34:58,735][105620] Updated weights for policy 1, policy_version 1951630 (0.0009) [2023-12-27 05:34:59,316][105692] Updated weights for policy 0, policy_version 1946869 (0.0009) [2023-12-27 05:34:59,383][105692] Updated weights for policy 0, policy_version 1946879 (0.0008) [2023-12-27 05:34:59,438][105692] Updated weights for policy 0, policy_version 1946890 (0.0009) [2023-12-27 05:34:59,534][105620] Updated weights for policy 1, policy_version 1951640 (0.0008) [2023-12-27 05:34:59,596][105620] Updated weights for policy 1, policy_version 1951650 (0.0009) [2023-12-27 05:34:59,653][105620] Updated weights for policy 1, policy_version 1951660 (0.0008) [2023-12-27 05:35:00,232][105692] Updated weights for policy 0, policy_version 1946900 (0.0008) [2023-12-27 05:35:00,297][105692] Updated weights for policy 0, policy_version 1946910 (0.0009) [2023-12-27 05:35:00,356][105692] Updated weights for policy 0, policy_version 1946920 (0.0011) [2023-12-27 05:35:00,383][105620] Updated weights for policy 1, policy_version 1951670 (0.0010) [2023-12-27 05:35:00,428][105620] Updated weights for policy 1, policy_version 1951680 (0.0011) [2023-12-27 05:35:00,484][105620] Updated weights for policy 1, policy_version 1951690 (0.0008) [2023-12-27 05:35:01,025][105692] Updated weights for policy 0, policy_version 1946930 (0.0010) [2023-12-27 05:35:01,062][104569] Fps is (10 sec: 19660.6, 60 sec: 19114.7, 300 sec: 19383.1). Total num frames: 998187008. Throughput: 0: 9750.7, 1: 9508.3. Samples: 998162372. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:35:01,063][104569] Avg episode reward: [(0, '8176.844'), (1, '9072.550')] [2023-12-27 05:35:01,090][105620] Updated weights for policy 1, policy_version 1951700 (0.0008) [2023-12-27 05:35:01,095][105692] Updated weights for policy 0, policy_version 1946940 (0.0011) [2023-12-27 05:35:01,156][105692] Updated weights for policy 0, policy_version 1946950 (0.0010) [2023-12-27 05:35:01,159][105620] Updated weights for policy 1, policy_version 1951710 (0.0008) [2023-12-27 05:35:01,203][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001946960_498491392.pth... [2023-12-27 05:35:01,205][105692] Updated weights for policy 0, policy_version 1946960 (0.0010) [2023-12-27 05:35:01,208][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001945808_498196480.pth [2023-12-27 05:35:01,213][105620] Updated weights for policy 1, policy_version 1951720 (0.0008) [2023-12-27 05:35:01,255][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001951728_499712000.pth... [2023-12-27 05:35:01,260][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001950576_499417088.pth [2023-12-27 05:35:01,871][105692] Updated weights for policy 0, policy_version 1946970 (0.0008) [2023-12-27 05:35:01,925][105620] Updated weights for policy 1, policy_version 1951730 (0.0009) [2023-12-27 05:35:01,926][105692] Updated weights for policy 0, policy_version 1946980 (0.0006) [2023-12-27 05:35:01,984][105620] Updated weights for policy 1, policy_version 1951740 (0.0010) [2023-12-27 05:35:01,990][105692] Updated weights for policy 0, policy_version 1946990 (0.0008) [2023-12-27 05:35:02,036][105620] Updated weights for policy 1, policy_version 1951750 (0.0010) [2023-12-27 05:35:02,084][105620] Updated weights for policy 1, policy_version 1951760 (0.0009) [2023-12-27 05:35:02,724][105692] Updated weights for policy 0, policy_version 1947000 (0.0011) [2023-12-27 05:35:02,784][105692] Updated weights for policy 0, policy_version 1947010 (0.0011) [2023-12-27 05:35:02,830][105620] Updated weights for policy 1, policy_version 1951770 (0.0008) [2023-12-27 05:35:02,843][105692] Updated weights for policy 0, policy_version 1947020 (0.0011) [2023-12-27 05:35:02,892][105620] Updated weights for policy 1, policy_version 1951780 (0.0007) [2023-12-27 05:35:02,949][105620] Updated weights for policy 1, policy_version 1951790 (0.0008) [2023-12-27 05:35:03,575][105692] Updated weights for policy 0, policy_version 1947030 (0.0007) [2023-12-27 05:35:03,642][105692] Updated weights for policy 0, policy_version 1947040 (0.0006) [2023-12-27 05:35:03,701][105620] Updated weights for policy 1, policy_version 1951800 (0.0010) [2023-12-27 05:35:03,705][105692] Updated weights for policy 0, policy_version 1947050 (0.0011) [2023-12-27 05:35:03,750][105620] Updated weights for policy 1, policy_version 1951810 (0.0010) [2023-12-27 05:35:03,797][105620] Updated weights for policy 1, policy_version 1951820 (0.0007) [2023-12-27 05:35:04,362][105692] Updated weights for policy 0, policy_version 1947060 (0.0011) [2023-12-27 05:35:04,429][105692] Updated weights for policy 0, policy_version 1947070 (0.0009) [2023-12-27 05:35:04,492][105692] Updated weights for policy 0, policy_version 1947080 (0.0007) [2023-12-27 05:35:04,509][105620] Updated weights for policy 1, policy_version 1951830 (0.0010) [2023-12-27 05:35:04,560][105620] Updated weights for policy 1, policy_version 1951840 (0.0010) [2023-12-27 05:35:04,615][105620] Updated weights for policy 1, policy_version 1951850 (0.0010) [2023-12-27 05:35:05,202][105692] Updated weights for policy 0, policy_version 1947090 (0.0006) [2023-12-27 05:35:05,256][105692] Updated weights for policy 0, policy_version 1947100 (0.0008) [2023-12-27 05:35:05,312][105692] Updated weights for policy 0, policy_version 1947110 (0.0008) [2023-12-27 05:35:05,356][105620] Updated weights for policy 1, policy_version 1951860 (0.0011) [2023-12-27 05:35:05,361][105692] Updated weights for policy 0, policy_version 1947120 (0.0009) [2023-12-27 05:35:05,415][105620] Updated weights for policy 1, policy_version 1951870 (0.0011) [2023-12-27 05:35:05,477][105620] Updated weights for policy 1, policy_version 1951880 (0.0010) [2023-12-27 05:35:06,028][105692] Updated weights for policy 0, policy_version 1947130 (0.0010) [2023-12-27 05:35:06,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 998285312. Throughput: 0: 9810.0, 1: 9495.9. Samples: 998278284. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:35:06,063][104569] Avg episode reward: [(0, '8270.584'), (1, '9072.296')] [2023-12-27 05:35:06,081][105692] Updated weights for policy 0, policy_version 1947140 (0.0009) [2023-12-27 05:35:06,142][105692] Updated weights for policy 0, policy_version 1947150 (0.0008) [2023-12-27 05:35:06,171][105620] Updated weights for policy 1, policy_version 1951890 (0.0011) [2023-12-27 05:35:06,235][105620] Updated weights for policy 1, policy_version 1951900 (0.0011) [2023-12-27 05:35:06,288][105620] Updated weights for policy 1, policy_version 1951910 (0.0010) [2023-12-27 05:35:06,345][105620] Updated weights for policy 1, policy_version 1951920 (0.0010) [2023-12-27 05:35:06,956][105692] Updated weights for policy 0, policy_version 1947160 (0.0006) [2023-12-27 05:35:07,009][105692] Updated weights for policy 0, policy_version 1947170 (0.0005) [2023-12-27 05:35:07,076][105692] Updated weights for policy 0, policy_version 1947180 (0.0006) [2023-12-27 05:35:07,081][105620] Updated weights for policy 1, policy_version 1951930 (0.0005) [2023-12-27 05:35:07,136][105620] Updated weights for policy 1, policy_version 1951940 (0.0010) [2023-12-27 05:35:07,205][105620] Updated weights for policy 1, policy_version 1951950 (0.0010) [2023-12-27 05:35:07,684][105692] Updated weights for policy 0, policy_version 1947190 (0.0008) [2023-12-27 05:35:07,743][105692] Updated weights for policy 0, policy_version 1947200 (0.0010) [2023-12-27 05:35:07,808][105692] Updated weights for policy 0, policy_version 1947210 (0.0009) [2023-12-27 05:35:07,875][105620] Updated weights for policy 1, policy_version 1951960 (0.0009) [2023-12-27 05:35:07,937][105620] Updated weights for policy 1, policy_version 1951970 (0.0007) [2023-12-27 05:35:07,984][105620] Updated weights for policy 1, policy_version 1951980 (0.0005) [2023-12-27 05:35:08,491][105692] Updated weights for policy 0, policy_version 1947220 (0.0008) [2023-12-27 05:35:08,550][105692] Updated weights for policy 0, policy_version 1947230 (0.0008) [2023-12-27 05:35:08,602][105692] Updated weights for policy 0, policy_version 1947240 (0.0008) [2023-12-27 05:35:08,729][105620] Updated weights for policy 1, policy_version 1951990 (0.0008) [2023-12-27 05:35:08,787][105620] Updated weights for policy 1, policy_version 1952000 (0.0010) [2023-12-27 05:35:08,842][105620] Updated weights for policy 1, policy_version 1952010 (0.0008) [2023-12-27 05:35:09,282][105692] Updated weights for policy 0, policy_version 1947250 (0.0008) [2023-12-27 05:35:09,339][105692] Updated weights for policy 0, policy_version 1947260 (0.0010) [2023-12-27 05:35:09,407][105692] Updated weights for policy 0, policy_version 1947270 (0.0010) [2023-12-27 05:35:09,474][105692] Updated weights for policy 0, policy_version 1947280 (0.0010) [2023-12-27 05:35:09,565][105620] Updated weights for policy 1, policy_version 1952020 (0.0008) [2023-12-27 05:35:09,627][105620] Updated weights for policy 1, policy_version 1952030 (0.0006) [2023-12-27 05:35:09,699][105620] Updated weights for policy 1, policy_version 1952040 (0.0005) [2023-12-27 05:35:10,215][105692] Updated weights for policy 0, policy_version 1947290 (0.0009) [2023-12-27 05:35:10,275][105692] Updated weights for policy 0, policy_version 1947300 (0.0009) [2023-12-27 05:35:10,344][105692] Updated weights for policy 0, policy_version 1947310 (0.0006) [2023-12-27 05:35:10,393][105620] Updated weights for policy 1, policy_version 1952050 (0.0006) [2023-12-27 05:35:10,446][105620] Updated weights for policy 1, policy_version 1952060 (0.0010) [2023-12-27 05:35:10,501][105620] Updated weights for policy 1, policy_version 1952070 (0.0009) [2023-12-27 05:35:10,550][105620] Updated weights for policy 1, policy_version 1952080 (0.0009) [2023-12-27 05:35:10,922][105692] Updated weights for policy 0, policy_version 1947320 (0.0006) [2023-12-27 05:35:10,977][105692] Updated weights for policy 0, policy_version 1947330 (0.0005) [2023-12-27 05:35:11,038][105692] Updated weights for policy 0, policy_version 1947340 (0.0006) [2023-12-27 05:35:11,062][104569] Fps is (10 sec: 20480.2, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 998391808. Throughput: 0: 9925.4, 1: 9518.0. Samples: 998396028. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:35:11,062][104569] Avg episode reward: [(0, '8448.657'), (1, '8886.457')] [2023-12-27 05:35:11,409][105620] Updated weights for policy 1, policy_version 1952090 (0.0009) [2023-12-27 05:35:11,471][105620] Updated weights for policy 1, policy_version 1952100 (0.0008) [2023-12-27 05:35:11,533][105620] Updated weights for policy 1, policy_version 1952110 (0.0009) [2023-12-27 05:35:11,803][105692] Updated weights for policy 0, policy_version 1947350 (0.0007) [2023-12-27 05:35:11,867][105692] Updated weights for policy 0, policy_version 1947360 (0.0007) [2023-12-27 05:35:11,931][105692] Updated weights for policy 0, policy_version 1947370 (0.0006) [2023-12-27 05:35:12,381][105620] Updated weights for policy 1, policy_version 1952120 (0.0009) [2023-12-27 05:35:12,436][105620] Updated weights for policy 1, policy_version 1952130 (0.0006) [2023-12-27 05:35:12,484][105620] Updated weights for policy 1, policy_version 1952140 (0.0006) [2023-12-27 05:35:12,574][105692] Updated weights for policy 0, policy_version 1947380 (0.0007) [2023-12-27 05:35:12,633][105692] Updated weights for policy 0, policy_version 1947390 (0.0009) [2023-12-27 05:35:12,692][105692] Updated weights for policy 0, policy_version 1947400 (0.0008) [2023-12-27 05:35:13,081][105620] Updated weights for policy 1, policy_version 1952150 (0.0007) [2023-12-27 05:35:13,134][105620] Updated weights for policy 1, policy_version 1952160 (0.0008) [2023-12-27 05:35:13,193][105620] Updated weights for policy 1, policy_version 1952170 (0.0011) [2023-12-27 05:35:13,404][105692] Updated weights for policy 0, policy_version 1947410 (0.0009) [2023-12-27 05:35:13,453][105692] Updated weights for policy 0, policy_version 1947420 (0.0005) [2023-12-27 05:35:13,503][105692] Updated weights for policy 0, policy_version 1947430 (0.0005) [2023-12-27 05:35:13,549][105692] Updated weights for policy 0, policy_version 1947440 (0.0007) [2023-12-27 05:35:13,953][105620] Updated weights for policy 1, policy_version 1952180 (0.0010) [2023-12-27 05:35:14,011][105620] Updated weights for policy 1, policy_version 1952190 (0.0010) [2023-12-27 05:35:14,064][105620] Updated weights for policy 1, policy_version 1952200 (0.0010) [2023-12-27 05:35:14,154][105692] Updated weights for policy 0, policy_version 1947450 (0.0006) [2023-12-27 05:35:14,215][105692] Updated weights for policy 0, policy_version 1947460 (0.0006) [2023-12-27 05:35:14,277][105692] Updated weights for policy 0, policy_version 1947470 (0.0006) [2023-12-27 05:35:14,885][105692] Updated weights for policy 0, policy_version 1947480 (0.0008) [2023-12-27 05:35:14,914][105620] Updated weights for policy 1, policy_version 1952210 (0.0009) [2023-12-27 05:35:14,941][105692] Updated weights for policy 0, policy_version 1947490 (0.0007) [2023-12-27 05:35:14,975][105620] Updated weights for policy 1, policy_version 1952220 (0.0007) [2023-12-27 05:35:15,000][105692] Updated weights for policy 0, policy_version 1947500 (0.0006) [2023-12-27 05:35:15,035][105620] Updated weights for policy 1, policy_version 1952230 (0.0009) [2023-12-27 05:35:15,097][105620] Updated weights for policy 1, policy_version 1952240 (0.0009) [2023-12-27 05:35:15,654][105692] Updated weights for policy 0, policy_version 1947510 (0.0006) [2023-12-27 05:35:15,669][105620] Updated weights for policy 1, policy_version 1952250 (0.0007) [2023-12-27 05:35:15,715][105692] Updated weights for policy 0, policy_version 1947520 (0.0006) [2023-12-27 05:35:15,728][105620] Updated weights for policy 1, policy_version 1952260 (0.0007) [2023-12-27 05:35:15,774][105692] Updated weights for policy 0, policy_version 1947530 (0.0007) [2023-12-27 05:35:15,784][105620] Updated weights for policy 1, policy_version 1952270 (0.0007) [2023-12-27 05:35:16,062][104569] Fps is (10 sec: 20479.8, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 998490112. Throughput: 0: 9881.3, 1: 9559.0. Samples: 998454820. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:35:16,063][104569] Avg episode reward: [(0, '8626.747'), (1, '8978.720')] [2023-12-27 05:35:16,068][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001947536_498638848.pth... [2023-12-27 05:35:16,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001952272_499851264.pth... [2023-12-27 05:35:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001946352_498335744.pth [2023-12-27 05:35:16,075][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001951120_499556352.pth [2023-12-27 05:35:16,443][105692] Updated weights for policy 0, policy_version 1947540 (0.0007) [2023-12-27 05:35:16,446][105620] Updated weights for policy 1, policy_version 1952280 (0.0006) [2023-12-27 05:35:16,494][105692] Updated weights for policy 0, policy_version 1947550 (0.0006) [2023-12-27 05:35:16,496][105620] Updated weights for policy 1, policy_version 1952290 (0.0007) [2023-12-27 05:35:16,552][105692] Updated weights for policy 0, policy_version 1947560 (0.0006) [2023-12-27 05:35:16,552][105620] Updated weights for policy 1, policy_version 1952300 (0.0008) [2023-12-27 05:35:17,181][105620] Updated weights for policy 1, policy_version 1952310 (0.0007) [2023-12-27 05:35:17,246][105620] Updated weights for policy 1, policy_version 1952320 (0.0005) [2023-12-27 05:35:17,301][105620] Updated weights for policy 1, policy_version 1952330 (0.0005) [2023-12-27 05:35:17,335][105692] Updated weights for policy 0, policy_version 1947570 (0.0009) [2023-12-27 05:35:17,401][105692] Updated weights for policy 0, policy_version 1947580 (0.0007) [2023-12-27 05:35:17,467][105692] Updated weights for policy 0, policy_version 1947590 (0.0008) [2023-12-27 05:35:17,535][105692] Updated weights for policy 0, policy_version 1947600 (0.0008) [2023-12-27 05:35:17,963][105620] Updated weights for policy 1, policy_version 1952340 (0.0005) [2023-12-27 05:35:18,029][105620] Updated weights for policy 1, policy_version 1952350 (0.0006) [2023-12-27 05:35:18,069][105692] Updated weights for policy 0, policy_version 1947610 (0.0009) [2023-12-27 05:35:18,081][105620] Updated weights for policy 1, policy_version 1952360 (0.0005) [2023-12-27 05:35:18,128][105692] Updated weights for policy 0, policy_version 1947620 (0.0005) [2023-12-27 05:35:18,184][105692] Updated weights for policy 0, policy_version 1947630 (0.0005) [2023-12-27 05:35:18,664][105620] Updated weights for policy 1, policy_version 1952370 (0.0007) [2023-12-27 05:35:18,725][105620] Updated weights for policy 1, policy_version 1952380 (0.0011) [2023-12-27 05:35:18,785][105620] Updated weights for policy 1, policy_version 1952390 (0.0010) [2023-12-27 05:35:18,848][105692] Updated weights for policy 0, policy_version 1947640 (0.0006) [2023-12-27 05:35:18,849][105620] Updated weights for policy 1, policy_version 1952400 (0.0011) [2023-12-27 05:35:18,900][105692] Updated weights for policy 0, policy_version 1947650 (0.0010) [2023-12-27 05:35:18,964][105692] Updated weights for policy 0, policy_version 1947660 (0.0008) [2023-12-27 05:35:19,590][105620] Updated weights for policy 1, policy_version 1952410 (0.0011) [2023-12-27 05:35:19,649][105620] Updated weights for policy 1, policy_version 1952420 (0.0010) [2023-12-27 05:35:19,708][105620] Updated weights for policy 1, policy_version 1952430 (0.0010) [2023-12-27 05:35:19,723][105692] Updated weights for policy 0, policy_version 1947670 (0.0010) [2023-12-27 05:35:19,790][105692] Updated weights for policy 0, policy_version 1947680 (0.0011) [2023-12-27 05:35:19,853][105692] Updated weights for policy 0, policy_version 1947691 (0.0013) [2023-12-27 05:35:20,461][105620] Updated weights for policy 1, policy_version 1952440 (0.0009) [2023-12-27 05:35:20,522][105620] Updated weights for policy 1, policy_version 1952450 (0.0008) [2023-12-27 05:35:20,584][105620] Updated weights for policy 1, policy_version 1952460 (0.0006) [2023-12-27 05:35:20,632][105692] Updated weights for policy 0, policy_version 1947701 (0.0009) [2023-12-27 05:35:20,689][105692] Updated weights for policy 0, policy_version 1947711 (0.0011) [2023-12-27 05:35:20,753][105692] Updated weights for policy 0, policy_version 1947721 (0.0011) [2023-12-27 05:35:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 998588416. Throughput: 0: 9922.2, 1: 9736.0. Samples: 998577976. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:35:21,063][104569] Avg episode reward: [(0, '8530.564'), (1, '9346.037')] [2023-12-27 05:35:21,280][105620] Updated weights for policy 1, policy_version 1952470 (0.0008) [2023-12-27 05:35:21,349][105620] Updated weights for policy 1, policy_version 1952480 (0.0008) [2023-12-27 05:35:21,417][105620] Updated weights for policy 1, policy_version 1952490 (0.0008) [2023-12-27 05:35:21,495][105692] Updated weights for policy 0, policy_version 1947731 (0.0010) [2023-12-27 05:35:21,555][105692] Updated weights for policy 0, policy_version 1947741 (0.0006) [2023-12-27 05:35:21,619][105692] Updated weights for policy 0, policy_version 1947751 (0.0007) [2023-12-27 05:35:22,197][105620] Updated weights for policy 1, policy_version 1952500 (0.0009) [2023-12-27 05:35:22,243][105692] Updated weights for policy 0, policy_version 1947761 (0.0008) [2023-12-27 05:35:22,248][105620] Updated weights for policy 1, policy_version 1952510 (0.0010) [2023-12-27 05:35:22,306][105692] Updated weights for policy 0, policy_version 1947771 (0.0007) [2023-12-27 05:35:22,309][105620] Updated weights for policy 1, policy_version 1952520 (0.0009) [2023-12-27 05:35:22,375][105692] Updated weights for policy 0, policy_version 1947781 (0.0008) [2023-12-27 05:35:22,434][105692] Updated weights for policy 0, policy_version 1947791 (0.0010) [2023-12-27 05:35:22,971][105620] Updated weights for policy 1, policy_version 1952530 (0.0007) [2023-12-27 05:35:23,033][105620] Updated weights for policy 1, policy_version 1952540 (0.0006) [2023-12-27 05:35:23,100][105620] Updated weights for policy 1, policy_version 1952550 (0.0006) [2023-12-27 05:35:23,167][105620] Updated weights for policy 1, policy_version 1952560 (0.0006) [2023-12-27 05:35:23,237][105692] Updated weights for policy 0, policy_version 1947801 (0.0009) [2023-12-27 05:35:23,283][105692] Updated weights for policy 0, policy_version 1947811 (0.0008) [2023-12-27 05:35:23,333][105692] Updated weights for policy 0, policy_version 1947821 (0.0009) [2023-12-27 05:35:23,829][105620] Updated weights for policy 1, policy_version 1952570 (0.0009) [2023-12-27 05:35:23,884][105620] Updated weights for policy 1, policy_version 1952580 (0.0009) [2023-12-27 05:35:23,939][105620] Updated weights for policy 1, policy_version 1952590 (0.0009) [2023-12-27 05:35:24,102][105692] Updated weights for policy 0, policy_version 1947831 (0.0009) [2023-12-27 05:35:24,153][105692] Updated weights for policy 0, policy_version 1947841 (0.0009) [2023-12-27 05:35:24,209][105692] Updated weights for policy 0, policy_version 1947851 (0.0009) [2023-12-27 05:35:24,596][105620] Updated weights for policy 1, policy_version 1952600 (0.0006) [2023-12-27 05:35:24,643][105620] Updated weights for policy 1, policy_version 1952610 (0.0007) [2023-12-27 05:35:24,689][105620] Updated weights for policy 1, policy_version 1952620 (0.0008) [2023-12-27 05:35:24,992][105692] Updated weights for policy 0, policy_version 1947861 (0.0010) [2023-12-27 05:35:25,045][105692] Updated weights for policy 0, policy_version 1947871 (0.0009) [2023-12-27 05:35:25,097][105692] Updated weights for policy 0, policy_version 1947881 (0.0009) [2023-12-27 05:35:25,460][105620] Updated weights for policy 1, policy_version 1952630 (0.0009) [2023-12-27 05:35:25,509][105620] Updated weights for policy 1, policy_version 1952640 (0.0008) [2023-12-27 05:35:25,568][105620] Updated weights for policy 1, policy_version 1952650 (0.0008) [2023-12-27 05:35:25,875][105692] Updated weights for policy 0, policy_version 1947891 (0.0009) [2023-12-27 05:35:25,924][105692] Updated weights for policy 0, policy_version 1947901 (0.0008) [2023-12-27 05:35:25,972][105692] Updated weights for policy 0, policy_version 1947911 (0.0009) [2023-12-27 05:35:26,062][104569] Fps is (10 sec: 19661.0, 60 sec: 19524.4, 300 sec: 19466.4). Total num frames: 998686720. Throughput: 0: 9876.2, 1: 9808.5. Samples: 998692008. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:35:26,063][104569] Avg episode reward: [(0, '7712.424'), (1, '9254.602')] [2023-12-27 05:35:26,285][105620] Updated weights for policy 1, policy_version 1952660 (0.0009) [2023-12-27 05:35:26,335][105620] Updated weights for policy 1, policy_version 1952670 (0.0009) [2023-12-27 05:35:26,381][105620] Updated weights for policy 1, policy_version 1952680 (0.0008) [2023-12-27 05:35:26,738][105692] Updated weights for policy 0, policy_version 1947921 (0.0008) [2023-12-27 05:35:26,797][105692] Updated weights for policy 0, policy_version 1947931 (0.0009) [2023-12-27 05:35:26,853][105692] Updated weights for policy 0, policy_version 1947941 (0.0009) [2023-12-27 05:35:26,909][105692] Updated weights for policy 0, policy_version 1947951 (0.0009) [2023-12-27 05:35:27,111][105620] Updated weights for policy 1, policy_version 1952690 (0.0008) [2023-12-27 05:35:27,171][105620] Updated weights for policy 1, policy_version 1952700 (0.0008) [2023-12-27 05:35:27,231][105620] Updated weights for policy 1, policy_version 1952710 (0.0009) [2023-12-27 05:35:27,288][105620] Updated weights for policy 1, policy_version 1952720 (0.0009) [2023-12-27 05:35:27,663][105692] Updated weights for policy 0, policy_version 1947961 (0.0010) [2023-12-27 05:35:27,716][105692] Updated weights for policy 0, policy_version 1947971 (0.0010) [2023-12-27 05:35:27,778][105692] Updated weights for policy 0, policy_version 1947982 (0.0011) [2023-12-27 05:35:27,897][105620] Updated weights for policy 1, policy_version 1952730 (0.0005) [2023-12-27 05:35:27,940][105620] Updated weights for policy 1, policy_version 1952740 (0.0005) [2023-12-27 05:35:27,990][105620] Updated weights for policy 1, policy_version 1952750 (0.0005) [2023-12-27 05:35:28,574][105692] Updated weights for policy 0, policy_version 1947992 (0.0009) [2023-12-27 05:35:28,613][105620] Updated weights for policy 1, policy_version 1952760 (0.0006) [2023-12-27 05:35:28,634][105692] Updated weights for policy 0, policy_version 1948002 (0.0009) [2023-12-27 05:35:28,659][105620] Updated weights for policy 1, policy_version 1952770 (0.0005) [2023-12-27 05:35:28,690][105692] Updated weights for policy 0, policy_version 1948012 (0.0008) [2023-12-27 05:35:28,705][105620] Updated weights for policy 1, policy_version 1952780 (0.0005) [2023-12-27 05:35:29,424][105620] Updated weights for policy 1, policy_version 1952790 (0.0010) [2023-12-27 05:35:29,483][105620] Updated weights for policy 1, policy_version 1952800 (0.0010) [2023-12-27 05:35:29,506][105692] Updated weights for policy 0, policy_version 1948022 (0.0007) [2023-12-27 05:35:29,539][105620] Updated weights for policy 1, policy_version 1952810 (0.0010) [2023-12-27 05:35:29,562][105692] Updated weights for policy 0, policy_version 1948032 (0.0006) [2023-12-27 05:35:29,624][105692] Updated weights for policy 0, policy_version 1948042 (0.0005) [2023-12-27 05:35:30,290][105620] Updated weights for policy 1, policy_version 1952820 (0.0011) [2023-12-27 05:35:30,292][105692] Updated weights for policy 0, policy_version 1948052 (0.0006) [2023-12-27 05:35:30,345][105620] Updated weights for policy 1, policy_version 1952830 (0.0011) [2023-12-27 05:35:30,348][105692] Updated weights for policy 0, policy_version 1948062 (0.0005) [2023-12-27 05:35:30,400][105620] Updated weights for policy 1, policy_version 1952840 (0.0010) [2023-12-27 05:35:30,408][105692] Updated weights for policy 0, policy_version 1948072 (0.0005) [2023-12-27 05:35:31,062][104569] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 998776832. Throughput: 0: 9841.8, 1: 9908.2. Samples: 998751420. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:35:31,063][104569] Avg episode reward: [(0, '7722.804'), (1, '9163.164')] [2023-12-27 05:35:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001948080_498778112.pth... [2023-12-27 05:35:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001952848_499998720.pth... [2023-12-27 05:35:31,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001946960_498491392.pth [2023-12-27 05:35:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001951728_499712000.pth [2023-12-27 05:35:31,139][105620] Updated weights for policy 1, policy_version 1952850 (0.0011) [2023-12-27 05:35:31,161][105692] Updated weights for policy 0, policy_version 1948082 (0.0006) [2023-12-27 05:35:31,202][105620] Updated weights for policy 1, policy_version 1952860 (0.0010) [2023-12-27 05:35:31,216][105692] Updated weights for policy 0, policy_version 1948092 (0.0007) [2023-12-27 05:35:31,266][105620] Updated weights for policy 1, policy_version 1952870 (0.0011) [2023-12-27 05:35:31,276][105692] Updated weights for policy 0, policy_version 1948102 (0.0008) [2023-12-27 05:35:31,333][105620] Updated weights for policy 1, policy_version 1952880 (0.0011) [2023-12-27 05:35:31,345][105692] Updated weights for policy 0, policy_version 1948112 (0.0007) [2023-12-27 05:35:32,062][105620] Updated weights for policy 1, policy_version 1952890 (0.0010) [2023-12-27 05:35:32,093][105692] Updated weights for policy 0, policy_version 1948122 (0.0006) [2023-12-27 05:35:32,121][105620] Updated weights for policy 1, policy_version 1952900 (0.0010) [2023-12-27 05:35:32,155][105692] Updated weights for policy 0, policy_version 1948132 (0.0005) [2023-12-27 05:35:32,176][105620] Updated weights for policy 1, policy_version 1952910 (0.0010) [2023-12-27 05:35:32,213][105692] Updated weights for policy 0, policy_version 1948142 (0.0007) [2023-12-27 05:35:32,869][105620] Updated weights for policy 1, policy_version 1952920 (0.0008) [2023-12-27 05:35:32,929][105620] Updated weights for policy 1, policy_version 1952930 (0.0011) [2023-12-27 05:35:32,987][105620] Updated weights for policy 1, policy_version 1952940 (0.0010) [2023-12-27 05:35:32,992][105692] Updated weights for policy 0, policy_version 1948152 (0.0006) [2023-12-27 05:35:33,054][105692] Updated weights for policy 0, policy_version 1948162 (0.0008) [2023-12-27 05:35:33,111][105692] Updated weights for policy 0, policy_version 1948172 (0.0009) [2023-12-27 05:35:33,563][105620] Updated weights for policy 1, policy_version 1952950 (0.0011) [2023-12-27 05:35:33,615][105620] Updated weights for policy 1, policy_version 1952960 (0.0010) [2023-12-27 05:35:33,659][105620] Updated weights for policy 1, policy_version 1952970 (0.0010) [2023-12-27 05:35:33,824][105692] Updated weights for policy 0, policy_version 1948182 (0.0007) [2023-12-27 05:35:33,883][105692] Updated weights for policy 0, policy_version 1948192 (0.0005) [2023-12-27 05:35:33,946][105692] Updated weights for policy 0, policy_version 1948202 (0.0006) [2023-12-27 05:35:34,394][105620] Updated weights for policy 1, policy_version 1952980 (0.0011) [2023-12-27 05:35:34,459][105620] Updated weights for policy 1, policy_version 1952990 (0.0010) [2023-12-27 05:35:34,523][105620] Updated weights for policy 1, policy_version 1953000 (0.0011) [2023-12-27 05:35:34,632][105692] Updated weights for policy 0, policy_version 1948212 (0.0008) [2023-12-27 05:35:34,697][105692] Updated weights for policy 0, policy_version 1948222 (0.0011) [2023-12-27 05:35:34,757][105692] Updated weights for policy 0, policy_version 1948232 (0.0011) [2023-12-27 05:35:35,259][105620] Updated weights for policy 1, policy_version 1953010 (0.0009) [2023-12-27 05:35:35,321][105620] Updated weights for policy 1, policy_version 1953020 (0.0010) [2023-12-27 05:35:35,386][105620] Updated weights for policy 1, policy_version 1953030 (0.0010) [2023-12-27 05:35:35,441][105620] Updated weights for policy 1, policy_version 1953040 (0.0010) [2023-12-27 05:35:35,499][105692] Updated weights for policy 0, policy_version 1948242 (0.0010) [2023-12-27 05:35:35,554][105692] Updated weights for policy 0, policy_version 1948252 (0.0005) [2023-12-27 05:35:35,602][105692] Updated weights for policy 0, policy_version 1948262 (0.0005) [2023-12-27 05:35:35,655][105692] Updated weights for policy 0, policy_version 1948272 (0.0005) [2023-12-27 05:35:36,062][104569] Fps is (10 sec: 18841.2, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 998875136. Throughput: 0: 9782.5, 1: 9854.7. Samples: 998866096. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) [2023-12-27 05:35:36,063][104569] Avg episode reward: [(0, '8088.922'), (1, '9163.187')] [2023-12-27 05:35:36,167][105620] Updated weights for policy 1, policy_version 1953050 (0.0007) [2023-12-27 05:35:36,234][105620] Updated weights for policy 1, policy_version 1953060 (0.0007) [2023-12-27 05:35:36,266][105692] Updated weights for policy 0, policy_version 1948282 (0.0008) [2023-12-27 05:35:36,291][105620] Updated weights for policy 1, policy_version 1953070 (0.0007) [2023-12-27 05:35:36,327][105692] Updated weights for policy 0, policy_version 1948292 (0.0005) [2023-12-27 05:35:36,388][105692] Updated weights for policy 0, policy_version 1948302 (0.0008) [2023-12-27 05:35:37,015][105692] Updated weights for policy 0, policy_version 1948312 (0.0007) [2023-12-27 05:35:37,016][105620] Updated weights for policy 1, policy_version 1953080 (0.0010) [2023-12-27 05:35:37,062][105692] Updated weights for policy 0, policy_version 1948322 (0.0008) [2023-12-27 05:35:37,075][105620] Updated weights for policy 1, policy_version 1953090 (0.0010) [2023-12-27 05:35:37,121][105692] Updated weights for policy 0, policy_version 1948332 (0.0006) [2023-12-27 05:35:37,141][105620] Updated weights for policy 1, policy_version 1953100 (0.0010) [2023-12-27 05:35:37,818][105692] Updated weights for policy 0, policy_version 1948342 (0.0006) [2023-12-27 05:35:37,854][105620] Updated weights for policy 1, policy_version 1953110 (0.0008) [2023-12-27 05:35:37,876][105692] Updated weights for policy 0, policy_version 1948352 (0.0006) [2023-12-27 05:35:37,914][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-12-27 05:35:37,917][105620] Updated weights for policy 1, policy_version 1953120 (0.0007) [2023-12-27 05:35:37,940][105692] Updated weights for policy 0, policy_version 1948362 (0.0006) [2023-12-27 05:35:38,589][105692] Updated weights for policy 0, policy_version 1948372 (0.0007) [2023-12-27 05:35:38,654][105692] Updated weights for policy 0, policy_version 1948382 (0.0006) [2023-12-27 05:35:38,676][105620] Updated weights for policy 1, policy_version 1953130 (0.0011) [2023-12-27 05:35:38,703][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:38,714][105692] Updated weights for policy 0, policy_version 1948392 (0.0006) [2023-12-27 05:35:39,476][105692] Updated weights for policy 0, policy_version 1948402 (0.0009) [2023-12-27 05:35:39,527][105620] Updated weights for policy 1, policy_version 1953140 (0.0010) [2023-12-27 05:35:39,527][105692] Updated weights for policy 0, policy_version 1948412 (0.0008) [2023-12-27 05:35:39,580][105692] Updated weights for policy 0, policy_version 1948422 (0.0009) [2023-12-27 05:35:39,586][105620] Updated weights for policy 1, policy_version 1953150 (0.0010) [2023-12-27 05:35:39,595][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:39,632][105692] Updated weights for policy 0, policy_version 1948432 (0.0008) [2023-12-27 05:35:40,425][105692] Updated weights for policy 0, policy_version 1948442 (0.0007) [2023-12-27 05:35:40,434][105620] Updated weights for policy 1, policy_version 1953160 (0.0010) [2023-12-27 05:35:40,473][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:40,485][105692] Updated weights for policy 0, policy_version 1948452 (0.0006) [2023-12-27 05:35:40,541][105692] Updated weights for policy 0, policy_version 1948462 (0.0008) [2023-12-27 05:35:41,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 998973440. Throughput: 0: 9773.0, 1: 9815.8. Samples: 998982788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:35:41,063][104569] Avg episode reward: [(0, '8447.761'), (1, '9346.195')] [2023-12-27 05:35:41,195][105692] Updated weights for policy 0, policy_version 1948472 (0.0009) [2023-12-27 05:35:41,249][105620] Updated weights for policy 1, policy_version 1953170 (0.0011) [2023-12-27 05:35:41,264][105692] Updated weights for policy 0, policy_version 1948482 (0.0009) [2023-12-27 05:35:41,307][105620] Updated weights for policy 1, policy_version 1953180 (0.0011) [2023-12-27 05:35:41,320][105692] Updated weights for policy 0, policy_version 1948492 (0.0011) [2023-12-27 05:35:41,330][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:42,138][105620] Updated weights for policy 1, policy_version 1953190 (0.0010) [2023-12-27 05:35:42,152][105692] Updated weights for policy 0, policy_version 1948502 (0.0007) [2023-12-27 05:35:42,199][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:42,202][105620] Updated weights for policy 1, policy_version 1953200 (0.0010) [2023-12-27 05:35:42,222][105692] Updated weights for policy 0, policy_version 1948512 (0.0006) [2023-12-27 05:35:42,287][105692] Updated weights for policy 0, policy_version 1948522 (0.0007) [2023-12-27 05:35:42,960][105692] Updated weights for policy 0, policy_version 1948532 (0.0009) [2023-12-27 05:35:43,007][105692] Updated weights for policy 0, policy_version 1948542 (0.0010) [2023-12-27 05:35:43,011][105620] Updated weights for policy 1, policy_version 1953210 (0.0010) [2023-12-27 05:35:43,044][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:43,066][105692] Updated weights for policy 0, policy_version 1948552 (0.0010) [2023-12-27 05:35:43,756][105692] Updated weights for policy 0, policy_version 1948562 (0.0010) [2023-12-27 05:35:43,813][105692] Updated weights for policy 0, policy_version 1948572 (0.0006) [2023-12-27 05:35:43,843][105620] Updated weights for policy 1, policy_version 1953220 (0.0010) [2023-12-27 05:35:43,867][105692] Updated weights for policy 0, policy_version 1948582 (0.0005) [2023-12-27 05:35:43,900][105620] Updated weights for policy 1, policy_version 1953230 (0.0010) [2023-12-27 05:35:43,906][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:43,917][105692] Updated weights for policy 0, policy_version 1948592 (0.0005) [2023-12-27 05:35:44,575][105692] Updated weights for policy 0, policy_version 1948602 (0.0008) [2023-12-27 05:35:44,628][105692] Updated weights for policy 0, policy_version 1948612 (0.0008) [2023-12-27 05:35:44,687][105692] Updated weights for policy 0, policy_version 1948622 (0.0008) [2023-12-27 05:35:44,750][105620] Updated weights for policy 1, policy_version 1953240 (0.0010) [2023-12-27 05:35:44,800][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:45,451][105692] Updated weights for policy 0, policy_version 1948632 (0.0006) [2023-12-27 05:35:45,504][105692] Updated weights for policy 0, policy_version 1948642 (0.0005) [2023-12-27 05:35:45,556][105692] Updated weights for policy 0, policy_version 1948652 (0.0008) [2023-12-27 05:35:45,604][105620] Updated weights for policy 1, policy_version 1953250 (0.0010) [2023-12-27 05:35:45,667][105620] Updated weights for policy 1, policy_version 1953260 (0.0010) [2023-12-27 05:35:45,689][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:46,062][104569] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 999071744. Throughput: 0: 9747.1, 1: 9782.3. Samples: 999041196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:35:46,063][104569] Avg episode reward: [(0, '8536.449'), (1, '9253.993')] [2023-12-27 05:35:46,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001948656_498925568.pth... [2023-12-27 05:35:46,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001953264_500146176.pth... [2023-12-27 05:35:46,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001947536_498638848.pth [2023-12-27 05:35:46,074][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001952272_499851264.pth [2023-12-27 05:35:46,299][105692] Updated weights for policy 0, policy_version 1948662 (0.0008) [2023-12-27 05:35:46,346][105692] Updated weights for policy 0, policy_version 1948672 (0.0009) [2023-12-27 05:35:46,401][105692] Updated weights for policy 0, policy_version 1948682 (0.0009) [2023-12-27 05:35:46,454][105620] Updated weights for policy 1, policy_version 1953270 (0.0007) [2023-12-27 05:35:46,502][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:46,502][105620] Updated weights for policy 1, policy_version 1953280 (0.0005) [2023-12-27 05:35:47,212][105692] Updated weights for policy 0, policy_version 1948692 (0.0008) [2023-12-27 05:35:47,261][105692] Updated weights for policy 0, policy_version 1948702 (0.0008) [2023-12-27 05:35:47,289][105620] Updated weights for policy 1, policy_version 1953290 (0.0008) [2023-12-27 05:35:47,308][105692] Updated weights for policy 0, policy_version 1948712 (0.0007) [2023-12-27 05:35:47,324][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:48,086][105692] Updated weights for policy 0, policy_version 1948722 (0.0008) [2023-12-27 05:35:48,126][105620] Updated weights for policy 1, policy_version 1953300 (0.0008) [2023-12-27 05:35:48,140][105692] Updated weights for policy 0, policy_version 1948732 (0.0008) [2023-12-27 05:35:48,184][105620] Updated weights for policy 1, policy_version 1953310 (0.0006) [2023-12-27 05:35:48,187][105692] Updated weights for policy 0, policy_version 1948742 (0.0008) [2023-12-27 05:35:48,195][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:48,938][105620] Updated weights for policy 1, policy_version 1953320 (0.0006) [2023-12-27 05:35:48,990][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:49,027][105692] Updated weights for policy 0, policy_version 1948753 (0.0009) [2023-12-27 05:35:49,082][105692] Updated weights for policy 0, policy_version 1948763 (0.0009) [2023-12-27 05:35:49,140][105692] Updated weights for policy 0, policy_version 1948773 (0.0008) [2023-12-27 05:35:49,203][105692] Updated weights for policy 0, policy_version 1948783 (0.0008) [2023-12-27 05:35:49,636][105620] Updated weights for policy 1, policy_version 1953330 (0.0006) [2023-12-27 05:35:49,700][105620] Updated weights for policy 1, policy_version 1953340 (0.0008) [2023-12-27 05:35:49,722][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:50,071][105692] Updated weights for policy 0, policy_version 1948793 (0.0009) [2023-12-27 05:35:50,128][105692] Updated weights for policy 0, policy_version 1948803 (0.0010) [2023-12-27 05:35:50,180][105692] Updated weights for policy 0, policy_version 1948813 (0.0009) [2023-12-27 05:35:50,439][105620] Updated weights for policy 1, policy_version 1953350 (0.0010) [2023-12-27 05:35:50,502][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:50,504][105620] Updated weights for policy 1, policy_version 1953360 (0.0009) [2023-12-27 05:35:50,979][105692] Updated weights for policy 0, policy_version 1948823 (0.0009) [2023-12-27 05:35:51,035][105692] Updated weights for policy 0, policy_version 1948833 (0.0006) [2023-12-27 05:35:51,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 999161856. Throughput: 0: 9704.5, 1: 9806.9. Samples: 999156292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:35:51,062][104569] Avg episode reward: [(0, '8532.072'), (1, '9163.441')] [2023-12-27 05:35:51,093][105692] Updated weights for policy 0, policy_version 1948843 (0.0007) [2023-12-27 05:35:51,349][105620] Updated weights for policy 1, policy_version 1953370 (0.0007) [2023-12-27 05:35:51,386][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:51,837][105692] Updated weights for policy 0, policy_version 1948853 (0.0009) [2023-12-27 05:35:51,891][105692] Updated weights for policy 0, policy_version 1948863 (0.0009) [2023-12-27 05:35:51,942][105692] Updated weights for policy 0, policy_version 1948874 (0.0009) [2023-12-27 05:35:52,196][105620] Updated weights for policy 1, policy_version 1953380 (0.0010) [2023-12-27 05:35:52,255][105620] Updated weights for policy 1, policy_version 1953390 (0.0011) [2023-12-27 05:35:52,266][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:52,729][105692] Updated weights for policy 0, policy_version 1948884 (0.0009) [2023-12-27 05:35:52,787][105692] Updated weights for policy 0, policy_version 1948894 (0.0009) [2023-12-27 05:35:52,848][105692] Updated weights for policy 0, policy_version 1948904 (0.0009) [2023-12-27 05:35:53,015][105620] Updated weights for policy 1, policy_version 1953400 (0.0009) [2023-12-27 05:35:53,052][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:53,587][105692] Updated weights for policy 0, policy_version 1948914 (0.0009) [2023-12-27 05:35:53,645][105692] Updated weights for policy 0, policy_version 1948924 (0.0009) [2023-12-27 05:35:53,706][105692] Updated weights for policy 0, policy_version 1948934 (0.0009) [2023-12-27 05:35:53,760][105692] Updated weights for policy 0, policy_version 1948944 (0.0009) [2023-12-27 05:35:53,860][105620] Updated weights for policy 1, policy_version 1953410 (0.0009) [2023-12-27 05:35:53,926][105620] Updated weights for policy 1, policy_version 1953420 (0.0009) [2023-12-27 05:35:53,951][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:54,538][105692] Updated weights for policy 0, policy_version 1948954 (0.0008) [2023-12-27 05:35:54,596][105692] Updated weights for policy 0, policy_version 1948964 (0.0006) [2023-12-27 05:35:54,656][105692] Updated weights for policy 0, policy_version 1948974 (0.0009) [2023-12-27 05:35:54,695][105620] Updated weights for policy 1, policy_version 1953430 (0.0008) [2023-12-27 05:35:54,741][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:54,745][105620] Updated weights for policy 1, policy_version 1953440 (0.0009) [2023-12-27 05:35:55,379][105692] Updated weights for policy 0, policy_version 1948984 (0.0006) [2023-12-27 05:35:55,432][105692] Updated weights for policy 0, policy_version 1948994 (0.0005) [2023-12-27 05:35:55,491][105692] Updated weights for policy 0, policy_version 1949004 (0.0006) [2023-12-27 05:35:55,632][105620] Updated weights for policy 1, policy_version 1953450 (0.0009) [2023-12-27 05:35:55,671][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:56,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 999260160. Throughput: 0: 9601.6, 1: 9801.8. Samples: 999269184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:35:56,063][104569] Avg episode reward: [(0, '8528.717'), (1, '9163.472')] [2023-12-27 05:35:56,162][105692] Updated weights for policy 0, policy_version 1949014 (0.0008) [2023-12-27 05:35:56,209][105692] Updated weights for policy 0, policy_version 1949024 (0.0009) [2023-12-27 05:35:56,256][105692] Updated weights for policy 0, policy_version 1949034 (0.0009) [2023-12-27 05:35:56,475][105620] Updated weights for policy 1, policy_version 1953460 (0.0009) [2023-12-27 05:35:56,524][105620] Updated weights for policy 1, policy_version 1953470 (0.0007) [2023-12-27 05:35:56,530][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:57,025][105692] Updated weights for policy 0, policy_version 1949044 (0.0008) [2023-12-27 05:35:57,073][105692] Updated weights for policy 0, policy_version 1949054 (0.0006) [2023-12-27 05:35:57,122][105692] Updated weights for policy 0, policy_version 1949064 (0.0006) [2023-12-27 05:35:57,355][105620] Updated weights for policy 1, policy_version 1953480 (0.0009) [2023-12-27 05:35:57,391][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:57,767][105692] Updated weights for policy 0, policy_version 1949074 (0.0009) [2023-12-27 05:35:57,830][105692] Updated weights for policy 0, policy_version 1949084 (0.0008) [2023-12-27 05:35:57,879][105692] Updated weights for policy 0, policy_version 1949094 (0.0008) [2023-12-27 05:35:57,936][105692] Updated weights for policy 0, policy_version 1949104 (0.0009) [2023-12-27 05:35:58,174][105620] Updated weights for policy 1, policy_version 1953490 (0.0009) [2023-12-27 05:35:58,230][105620] Updated weights for policy 1, policy_version 1953500 (0.0008) [2023-12-27 05:35:58,250][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:58,710][105692] Updated weights for policy 0, policy_version 1949114 (0.0009) [2023-12-27 05:35:58,770][105692] Updated weights for policy 0, policy_version 1949124 (0.0008) [2023-12-27 05:35:58,840][105692] Updated weights for policy 0, policy_version 1949134 (0.0008) [2023-12-27 05:35:59,152][105620] Updated weights for policy 1, policy_version 1953510 (0.0010) [2023-12-27 05:35:59,206][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:35:59,207][105620] Updated weights for policy 1, policy_version 1953520 (0.0009) [2023-12-27 05:35:59,616][105692] Updated weights for policy 0, policy_version 1949144 (0.0006) [2023-12-27 05:35:59,668][105692] Updated weights for policy 0, policy_version 1949154 (0.0006) [2023-12-27 05:35:59,721][105692] Updated weights for policy 0, policy_version 1949164 (0.0009) [2023-12-27 05:36:00,087][105620] Updated weights for policy 1, policy_version 1953530 (0.0008) [2023-12-27 05:36:00,121][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:00,441][105692] Updated weights for policy 0, policy_version 1949174 (0.0005) [2023-12-27 05:36:00,511][105692] Updated weights for policy 0, policy_version 1949184 (0.0008) [2023-12-27 05:36:00,562][105692] Updated weights for policy 0, policy_version 1949194 (0.0008) [2023-12-27 05:36:00,794][105620] Updated weights for policy 1, policy_version 1953540 (0.0006) [2023-12-27 05:36:00,851][105620] Updated weights for policy 1, policy_version 1953550 (0.0008) [2023-12-27 05:36:00,859][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:01,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 999358464. Throughput: 0: 9580.6, 1: 9789.5. Samples: 999326476. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:36:01,062][104569] Avg episode reward: [(0, '8174.712'), (1, '9264.570')] [2023-12-27 05:36:01,069][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001949200_499064832.pth... [2023-12-27 05:36:01,070][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001953552_500293632.pth... [2023-12-27 05:36:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001952848_499998720.pth [2023-12-27 05:36:01,074][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001948080_498778112.pth [2023-12-27 05:36:01,288][105692] Updated weights for policy 0, policy_version 1949204 (0.0006) [2023-12-27 05:36:01,355][105692] Updated weights for policy 0, policy_version 1949214 (0.0010) [2023-12-27 05:36:01,421][105692] Updated weights for policy 0, policy_version 1949224 (0.0009) [2023-12-27 05:36:01,641][105620] Updated weights for policy 1, policy_version 1953560 (0.0007) [2023-12-27 05:36:01,680][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:02,223][105692] Updated weights for policy 0, policy_version 1949234 (0.0009) [2023-12-27 05:36:02,285][105692] Updated weights for policy 0, policy_version 1949244 (0.0010) [2023-12-27 05:36:02,350][105692] Updated weights for policy 0, policy_version 1949254 (0.0010) [2023-12-27 05:36:02,407][105692] Updated weights for policy 0, policy_version 1949264 (0.0011) [2023-12-27 05:36:02,450][105620] Updated weights for policy 1, policy_version 1953570 (0.0006) [2023-12-27 05:36:02,506][105620] Updated weights for policy 1, policy_version 1953580 (0.0008) [2023-12-27 05:36:02,525][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:03,095][105692] Updated weights for policy 0, policy_version 1949274 (0.0010) [2023-12-27 05:36:03,146][105692] Updated weights for policy 0, policy_version 1949285 (0.0009) [2023-12-27 05:36:03,197][105692] Updated weights for policy 0, policy_version 1949295 (0.0008) [2023-12-27 05:36:03,243][105620] Updated weights for policy 1, policy_version 1953590 (0.0010) [2023-12-27 05:36:03,292][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:03,294][105620] Updated weights for policy 1, policy_version 1953600 (0.0010) [2023-12-27 05:36:03,954][105692] Updated weights for policy 0, policy_version 1949305 (0.0006) [2023-12-27 05:36:04,018][105692] Updated weights for policy 0, policy_version 1949315 (0.0010) [2023-12-27 05:36:04,078][105692] Updated weights for policy 0, policy_version 1949325 (0.0011) [2023-12-27 05:36:04,128][105620] Updated weights for policy 1, policy_version 1953610 (0.0007) [2023-12-27 05:36:04,165][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:04,782][105692] Updated weights for policy 0, policy_version 1949335 (0.0011) [2023-12-27 05:36:04,833][105692] Updated weights for policy 0, policy_version 1949345 (0.0010) [2023-12-27 05:36:04,879][105692] Updated weights for policy 0, policy_version 1949355 (0.0011) [2023-12-27 05:36:04,994][105620] Updated weights for policy 1, policy_version 1953620 (0.0009) [2023-12-27 05:36:05,053][105620] Updated weights for policy 1, policy_version 1953630 (0.0011) [2023-12-27 05:36:05,063][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:05,656][105692] Updated weights for policy 0, policy_version 1949365 (0.0011) [2023-12-27 05:36:05,708][105692] Updated weights for policy 0, policy_version 1949375 (0.0011) [2023-12-27 05:36:05,764][105692] Updated weights for policy 0, policy_version 1949385 (0.0011) [2023-12-27 05:36:05,824][105620] Updated weights for policy 1, policy_version 1953640 (0.0010) [2023-12-27 05:36:05,871][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:06,062][104569] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 999456768. Throughput: 0: 9446.5, 1: 9747.5. Samples: 999441704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:36:06,063][104569] Avg episode reward: [(0, '8091.123'), (1, '9264.624')] [2023-12-27 05:36:06,483][105692] Updated weights for policy 0, policy_version 1949395 (0.0010) [2023-12-27 05:36:06,554][105692] Updated weights for policy 0, policy_version 1949405 (0.0009) [2023-12-27 05:36:06,616][105620] Updated weights for policy 1, policy_version 1953650 (0.0010) [2023-12-27 05:36:06,621][105692] Updated weights for policy 0, policy_version 1949415 (0.0008) [2023-12-27 05:36:06,679][105620] Updated weights for policy 1, policy_version 1953660 (0.0011) [2023-12-27 05:36:06,702][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:07,329][105692] Updated weights for policy 0, policy_version 1949425 (0.0008) [2023-12-27 05:36:07,386][105692] Updated weights for policy 0, policy_version 1949435 (0.0006) [2023-12-27 05:36:07,441][105692] Updated weights for policy 0, policy_version 1949445 (0.0008) [2023-12-27 05:36:07,494][105692] Updated weights for policy 0, policy_version 1949455 (0.0007) [2023-12-27 05:36:07,520][105620] Updated weights for policy 1, policy_version 1953670 (0.0010) [2023-12-27 05:36:07,562][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:07,565][105620] Updated weights for policy 1, policy_version 1953680 (0.0007) [2023-12-27 05:36:08,201][105692] Updated weights for policy 0, policy_version 1949465 (0.0010) [2023-12-27 05:36:08,252][105692] Updated weights for policy 0, policy_version 1949475 (0.0010) [2023-12-27 05:36:08,304][105692] Updated weights for policy 0, policy_version 1949485 (0.0010) [2023-12-27 05:36:08,340][105620] Updated weights for policy 1, policy_version 1953690 (0.0007) [2023-12-27 05:36:08,380][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:09,038][105692] Updated weights for policy 0, policy_version 1949495 (0.0010) [2023-12-27 05:36:09,096][105692] Updated weights for policy 0, policy_version 1949505 (0.0010) [2023-12-27 05:36:09,156][105692] Updated weights for policy 0, policy_version 1949515 (0.0011) [2023-12-27 05:36:09,184][105620] Updated weights for policy 1, policy_version 1953700 (0.0009) [2023-12-27 05:36:09,243][105620] Updated weights for policy 1, policy_version 1953710 (0.0008) [2023-12-27 05:36:09,255][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:09,916][105692] Updated weights for policy 0, policy_version 1949525 (0.0010) [2023-12-27 05:36:09,965][105692] Updated weights for policy 0, policy_version 1949535 (0.0008) [2023-12-27 05:36:10,025][105692] Updated weights for policy 0, policy_version 1949545 (0.0008) [2023-12-27 05:36:10,125][105620] Updated weights for policy 1, policy_version 1953720 (0.0008) [2023-12-27 05:36:10,160][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:10,834][105620] Updated weights for policy 1, policy_version 1953730 (0.0008) [2023-12-27 05:36:10,847][105692] Updated weights for policy 0, policy_version 1949555 (0.0009) [2023-12-27 05:36:10,894][105620] Updated weights for policy 1, policy_version 1953740 (0.0006) [2023-12-27 05:36:10,901][105692] Updated weights for policy 0, policy_version 1949565 (0.0008) [2023-12-27 05:36:10,914][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:10,951][105692] Updated weights for policy 0, policy_version 1949575 (0.0007) [2023-12-27 05:36:11,070][104569] Fps is (10 sec: 19644.9, 60 sec: 19385.1, 300 sec: 19465.9). Total num frames: 999555072. Throughput: 0: 9469.6, 1: 9743.8. Samples: 999556764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:36:11,071][104569] Avg episode reward: [(0, '7998.160'), (1, '9254.027')] [2023-12-27 05:36:11,726][105692] Updated weights for policy 0, policy_version 1949585 (0.0009) [2023-12-27 05:36:11,766][105620] Updated weights for policy 1, policy_version 1953750 (0.0008) [2023-12-27 05:36:11,796][105692] Updated weights for policy 0, policy_version 1949595 (0.0008) [2023-12-27 05:36:11,826][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:11,828][105620] Updated weights for policy 1, policy_version 1953760 (0.0006) [2023-12-27 05:36:11,849][105692] Updated weights for policy 0, policy_version 1949605 (0.0008) [2023-12-27 05:36:11,897][105692] Updated weights for policy 0, policy_version 1949615 (0.0009) [2023-12-27 05:36:12,604][105620] Updated weights for policy 1, policy_version 1953770 (0.0009) [2023-12-27 05:36:12,631][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:12,695][105692] Updated weights for policy 0, policy_version 1949625 (0.0011) [2023-12-27 05:36:12,756][105692] Updated weights for policy 0, policy_version 1949635 (0.0009) [2023-12-27 05:36:12,805][105692] Updated weights for policy 0, policy_version 1949645 (0.0007) [2023-12-27 05:36:13,321][105620] Updated weights for policy 1, policy_version 1953780 (0.0007) [2023-12-27 05:36:13,381][105620] Updated weights for policy 1, policy_version 1953790 (0.0005) [2023-12-27 05:36:13,388][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:13,401][105692] Updated weights for policy 0, policy_version 1949655 (0.0006) [2023-12-27 05:36:13,459][105692] Updated weights for policy 0, policy_version 1949665 (0.0010) [2023-12-27 05:36:13,524][105692] Updated weights for policy 0, policy_version 1949675 (0.0010) [2023-12-27 05:36:14,053][105620] Updated weights for policy 1, policy_version 1953800 (0.0009) [2023-12-27 05:36:14,090][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:14,262][105692] Updated weights for policy 0, policy_version 1949685 (0.0010) [2023-12-27 05:36:14,318][105692] Updated weights for policy 0, policy_version 1949695 (0.0011) [2023-12-27 05:36:14,402][105692] Updated weights for policy 0, policy_version 1949705 (0.0009) [2023-12-27 05:36:14,844][105620] Updated weights for policy 1, policy_version 1953810 (0.0009) [2023-12-27 05:36:14,911][105620] Updated weights for policy 1, policy_version 1953820 (0.0010) [2023-12-27 05:36:14,931][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:15,047][105692] Updated weights for policy 0, policy_version 1949715 (0.0010) [2023-12-27 05:36:15,103][105692] Updated weights for policy 0, policy_version 1949725 (0.0010) [2023-12-27 05:36:15,152][105692] Updated weights for policy 0, policy_version 1949735 (0.0010) [2023-12-27 05:36:15,748][105620] Updated weights for policy 1, policy_version 1953830 (0.0010) [2023-12-27 05:36:15,783][105692] Updated weights for policy 0, policy_version 1949745 (0.0010) [2023-12-27 05:36:15,800][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:15,802][105620] Updated weights for policy 1, policy_version 1953840 (0.0007) [2023-12-27 05:36:15,836][105692] Updated weights for policy 0, policy_version 1949755 (0.0005) [2023-12-27 05:36:15,890][105692] Updated weights for policy 0, policy_version 1949765 (0.0005) [2023-12-27 05:36:15,941][105692] Updated weights for policy 0, policy_version 1949775 (0.0010) [2023-12-27 05:36:16,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 999653376. Throughput: 0: 9496.3, 1: 9727.9. Samples: 999616508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:36:16,063][104569] Avg episode reward: [(0, '7807.962'), (1, '9254.004')] [2023-12-27 05:36:16,067][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001953840_500441088.pth... [2023-12-27 05:36:16,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001949776_499212288.pth... [2023-12-27 05:36:16,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001948656_498925568.pth [2023-12-27 05:36:16,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001953264_500146176.pth [2023-12-27 05:36:16,550][105692] Updated weights for policy 0, policy_version 1949785 (0.0007) [2023-12-27 05:36:16,602][105692] Updated weights for policy 0, policy_version 1949795 (0.0005) [2023-12-27 05:36:16,644][105620] Updated weights for policy 1, policy_version 1953850 (0.0010) [2023-12-27 05:36:16,655][105692] Updated weights for policy 0, policy_version 1949805 (0.0005) [2023-12-27 05:36:16,678][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:17,345][105620] Updated weights for policy 1, policy_version 1953860 (0.0010) [2023-12-27 05:36:17,395][105620] Updated weights for policy 1, policy_version 1953870 (0.0010) [2023-12-27 05:36:17,402][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:17,419][105692] Updated weights for policy 0, policy_version 1949815 (0.0006) [2023-12-27 05:36:17,466][105692] Updated weights for policy 0, policy_version 1949825 (0.0005) [2023-12-27 05:36:17,520][105692] Updated weights for policy 0, policy_version 1949835 (0.0005) [2023-12-27 05:36:18,149][105692] Updated weights for policy 0, policy_version 1949845 (0.0006) [2023-12-27 05:36:18,211][105692] Updated weights for policy 0, policy_version 1949855 (0.0009) [2023-12-27 05:36:18,222][105620] Updated weights for policy 1, policy_version 1953880 (0.0010) [2023-12-27 05:36:18,259][105692] Updated weights for policy 0, policy_version 1949865 (0.0009) [2023-12-27 05:36:18,261][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:18,873][105692] Updated weights for policy 0, policy_version 1949875 (0.0007) [2023-12-27 05:36:18,935][105692] Updated weights for policy 0, policy_version 1949885 (0.0009) [2023-12-27 05:36:18,994][105692] Updated weights for policy 0, policy_version 1949895 (0.0010) [2023-12-27 05:36:19,064][105620] Updated weights for policy 1, policy_version 1953890 (0.0010) [2023-12-27 05:36:19,115][105620] Updated weights for policy 1, policy_version 1953900 (0.0010) [2023-12-27 05:36:19,141][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:19,694][105692] Updated weights for policy 0, policy_version 1949905 (0.0010) [2023-12-27 05:36:19,748][105692] Updated weights for policy 0, policy_version 1949915 (0.0008) [2023-12-27 05:36:19,803][105692] Updated weights for policy 0, policy_version 1949925 (0.0007) [2023-12-27 05:36:19,869][105692] Updated weights for policy 0, policy_version 1949935 (0.0009) [2023-12-27 05:36:20,011][105620] Updated weights for policy 1, policy_version 1953910 (0.0009) [2023-12-27 05:36:20,073][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:20,076][105620] Updated weights for policy 1, policy_version 1953920 (0.0009) [2023-12-27 05:36:20,659][105692] Updated weights for policy 0, policy_version 1949945 (0.0008) [2023-12-27 05:36:20,709][105692] Updated weights for policy 0, policy_version 1949955 (0.0008) [2023-12-27 05:36:20,764][105692] Updated weights for policy 0, policy_version 1949965 (0.0008) [2023-12-27 05:36:20,838][105620] Updated weights for policy 1, policy_version 1953930 (0.0006) [2023-12-27 05:36:20,875][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:21,062][104569] Fps is (10 sec: 19676.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 999751680. Throughput: 0: 9628.5, 1: 9718.2. Samples: 999736692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:36:21,062][104569] Avg episode reward: [(0, '8171.413'), (1, '9346.265')] [2023-12-27 05:36:21,605][105692] Updated weights for policy 0, policy_version 1949975 (0.0009) [2023-12-27 05:36:21,657][105620] Updated weights for policy 1, policy_version 1953940 (0.0008) [2023-12-27 05:36:21,672][105692] Updated weights for policy 0, policy_version 1949985 (0.0007) [2023-12-27 05:36:21,720][105620] Updated weights for policy 1, policy_version 1953950 (0.0007) [2023-12-27 05:36:21,732][105692] Updated weights for policy 0, policy_version 1949995 (0.0007) [2023-12-27 05:36:21,734][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:22,447][105620] Updated weights for policy 1, policy_version 1953960 (0.0008) [2023-12-27 05:36:22,485][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:22,528][105692] Updated weights for policy 0, policy_version 1950005 (0.0007) [2023-12-27 05:36:22,593][105692] Updated weights for policy 0, policy_version 1950015 (0.0007) [2023-12-27 05:36:22,649][105692] Updated weights for policy 0, policy_version 1950025 (0.0009) [2023-12-27 05:36:23,303][105692] Updated weights for policy 0, policy_version 1950035 (0.0009) [2023-12-27 05:36:23,355][105692] Updated weights for policy 0, policy_version 1950045 (0.0009) [2023-12-27 05:36:23,378][105620] Updated weights for policy 1, policy_version 1953970 (0.0008) [2023-12-27 05:36:23,405][105692] Updated weights for policy 0, policy_version 1950055 (0.0008) [2023-12-27 05:36:23,435][105620] Updated weights for policy 1, policy_version 1953980 (0.0007) [2023-12-27 05:36:23,458][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:23,976][105692] Updated weights for policy 0, policy_version 1950065 (0.0005) [2023-12-27 05:36:24,024][105692] Updated weights for policy 0, policy_version 1950075 (0.0005) [2023-12-27 05:36:24,077][105692] Updated weights for policy 0, policy_version 1950085 (0.0007) [2023-12-27 05:36:24,127][105692] Updated weights for policy 0, policy_version 1950095 (0.0008) [2023-12-27 05:36:24,365][105620] Updated weights for policy 1, policy_version 1953990 (0.0009) [2023-12-27 05:36:24,426][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:24,429][105620] Updated weights for policy 1, policy_version 1954000 (0.0007) [2023-12-27 05:36:24,771][105692] Updated weights for policy 0, policy_version 1950105 (0.0010) [2023-12-27 05:36:24,815][105692] Updated weights for policy 0, policy_version 1950115 (0.0005) [2023-12-27 05:36:24,874][105692] Updated weights for policy 0, policy_version 1950125 (0.0008) [2023-12-27 05:36:25,266][105620] Updated weights for policy 1, policy_version 1954010 (0.0005) [2023-12-27 05:36:25,301][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:25,507][105692] Updated weights for policy 0, policy_version 1950135 (0.0010) [2023-12-27 05:36:25,556][105692] Updated weights for policy 0, policy_version 1950145 (0.0010) [2023-12-27 05:36:25,604][105692] Updated weights for policy 0, policy_version 1950155 (0.0010) [2023-12-27 05:36:26,050][105620] Updated weights for policy 1, policy_version 1954020 (0.0008) [2023-12-27 05:36:26,062][104569] Fps is (10 sec: 18841.9, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 999841792. Throughput: 0: 9640.8, 1: 9718.6. Samples: 999853960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:36:26,062][104569] Avg episode reward: [(0, '8356.778'), (1, '9254.008')] [2023-12-27 05:36:26,102][105620] Updated weights for policy 1, policy_version 1954030 (0.0008) [2023-12-27 05:36:26,108][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:26,281][105692] Updated weights for policy 0, policy_version 1950165 (0.0008) [2023-12-27 05:36:26,334][105692] Updated weights for policy 0, policy_version 1950175 (0.0005) [2023-12-27 05:36:26,389][105692] Updated weights for policy 0, policy_version 1950185 (0.0006) [2023-12-27 05:36:26,991][105620] Updated weights for policy 1, policy_version 1954040 (0.0009) [2023-12-27 05:36:27,036][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:27,075][105692] Updated weights for policy 0, policy_version 1950195 (0.0011) [2023-12-27 05:36:27,119][105692] Updated weights for policy 0, policy_version 1950205 (0.0010) [2023-12-27 05:36:27,167][105692] Updated weights for policy 0, policy_version 1950215 (0.0010) [2023-12-27 05:36:27,792][105620] Updated weights for policy 1, policy_version 1954050 (0.0007) [2023-12-27 05:36:27,837][105620] Updated weights for policy 1, policy_version 1954060 (0.0005) [2023-12-27 05:36:27,855][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:27,918][105692] Updated weights for policy 0, policy_version 1950225 (0.0010) [2023-12-27 05:36:27,972][105692] Updated weights for policy 0, policy_version 1950235 (0.0010) [2023-12-27 05:36:28,023][105692] Updated weights for policy 0, policy_version 1950245 (0.0010) [2023-12-27 05:36:28,070][105692] Updated weights for policy 0, policy_version 1950255 (0.0010) [2023-12-27 05:36:28,496][105620] Updated weights for policy 1, policy_version 1954070 (0.0006) [2023-12-27 05:36:28,554][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:28,556][105620] Updated weights for policy 1, policy_version 1954080 (0.0011) [2023-12-27 05:36:28,796][105692] Updated weights for policy 0, policy_version 1950265 (0.0007) [2023-12-27 05:36:28,847][105692] Updated weights for policy 0, policy_version 1950275 (0.0006) [2023-12-27 05:36:28,892][105692] Updated weights for policy 0, policy_version 1950285 (0.0005) [2023-12-27 05:36:29,243][105620] Updated weights for policy 1, policy_version 1954090 (0.0006) [2023-12-27 05:36:29,276][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:29,535][105692] Updated weights for policy 0, policy_version 1950295 (0.0007) [2023-12-27 05:36:29,589][105692] Updated weights for policy 0, policy_version 1950305 (0.0008) [2023-12-27 05:36:29,637][105692] Updated weights for policy 0, policy_version 1950315 (0.0008) [2023-12-27 05:36:30,027][105620] Updated weights for policy 1, policy_version 1954100 (0.0006) [2023-12-27 05:36:30,081][105620] Updated weights for policy 1, policy_version 1954110 (0.0005) [2023-12-27 05:36:30,090][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:30,482][105692] Updated weights for policy 0, policy_version 1950325 (0.0008) [2023-12-27 05:36:30,536][105692] Updated weights for policy 0, policy_version 1950335 (0.0009) [2023-12-27 05:36:30,594][105692] Updated weights for policy 0, policy_version 1950345 (0.0009) [2023-12-27 05:36:30,730][105620] Updated weights for policy 1, policy_version 1954120 (0.0007) [2023-12-27 05:36:30,778][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:31,062][104569] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 999948288. Throughput: 0: 9649.4, 1: 9756.4. Samples: 999914456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:36:31,062][104569] Avg episode reward: [(0, '8811.625'), (1, '9254.001')] [2023-12-27 05:36:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001950352_499359744.pth... [2023-12-27 05:36:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001954128_500588544.pth... [2023-12-27 05:36:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001949200_499064832.pth [2023-12-27 05:36:31,072][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001953552_500293632.pth [2023-12-27 05:36:31,387][105692] Updated weights for policy 0, policy_version 1950355 (0.0009) [2023-12-27 05:36:31,446][105692] Updated weights for policy 0, policy_version 1950365 (0.0009) [2023-12-27 05:36:31,496][105692] Updated weights for policy 0, policy_version 1950375 (0.0008) [2023-12-27 05:36:31,509][105620] Updated weights for policy 1, policy_version 1954130 (0.0009) [2023-12-27 05:36:31,558][105620] Updated weights for policy 1, policy_version 1954140 (0.0008) [2023-12-27 05:36:31,576][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:32,263][105692] Updated weights for policy 0, policy_version 1950385 (0.0009) [2023-12-27 05:36:32,320][105692] Updated weights for policy 0, policy_version 1950395 (0.0008) [2023-12-27 05:36:32,376][105692] Updated weights for policy 0, policy_version 1950405 (0.0008) [2023-12-27 05:36:32,412][105620] Updated weights for policy 1, policy_version 1954150 (0.0007) [2023-12-27 05:36:32,430][105692] Updated weights for policy 0, policy_version 1950415 (0.0009) [2023-12-27 05:36:32,469][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:32,473][105620] Updated weights for policy 1, policy_version 1954160 (0.0008) [2023-12-27 05:36:33,176][105692] Updated weights for policy 0, policy_version 1950425 (0.0005) [2023-12-27 05:36:33,238][105692] Updated weights for policy 0, policy_version 1950435 (0.0005) [2023-12-27 05:36:33,293][105692] Updated weights for policy 0, policy_version 1950445 (0.0006) [2023-12-27 05:36:33,336][105620] Updated weights for policy 1, policy_version 1954170 (0.0008) [2023-12-27 05:36:33,360][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:33,849][105692] Updated weights for policy 0, policy_version 1950455 (0.0005) [2023-12-27 05:36:33,914][105692] Updated weights for policy 0, policy_version 1950465 (0.0007) [2023-12-27 05:36:33,966][105692] Updated weights for policy 0, policy_version 1950475 (0.0010) [2023-12-27 05:36:34,279][105620] Updated weights for policy 1, policy_version 1954181 (0.0010) [2023-12-27 05:36:34,344][105620] Updated weights for policy 1, policy_version 1954191 (0.0010) [2023-12-27 05:36:34,347][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:34,586][105692] Updated weights for policy 0, policy_version 1950485 (0.0008) [2023-12-27 05:36:34,646][105692] Updated weights for policy 0, policy_version 1950495 (0.0008) [2023-12-27 05:36:34,704][105692] Updated weights for policy 0, policy_version 1950505 (0.0006) [2023-12-27 05:36:35,242][105620] Updated weights for policy 1, policy_version 1954201 (0.0008) [2023-12-27 05:36:35,285][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:35,411][105692] Updated weights for policy 0, policy_version 1950515 (0.0009) [2023-12-27 05:36:35,473][105692] Updated weights for policy 0, policy_version 1950525 (0.0011) [2023-12-27 05:36:35,529][105692] Updated weights for policy 0, policy_version 1950535 (0.0011) [2023-12-27 05:36:36,062][104569] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 1000038400. Throughput: 0: 9729.5, 1: 9722.0. Samples: 1000031612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:36:36,063][104569] Avg episode reward: [(0, '8449.621'), (1, '9346.263')] [2023-12-27 05:36:36,121][105620] Updated weights for policy 1, policy_version 1954211 (0.0008) [2023-12-27 05:36:36,172][105692] Updated weights for policy 0, policy_version 1950545 (0.0010) [2023-12-27 05:36:36,182][105620] Updated weights for policy 1, policy_version 1954221 (0.0007) [2023-12-27 05:36:36,197][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:36,232][105692] Updated weights for policy 0, policy_version 1950555 (0.0011) [2023-12-27 05:36:36,295][105692] Updated weights for policy 0, policy_version 1950565 (0.0010) [2023-12-27 05:36:36,360][105692] Updated weights for policy 0, policy_version 1950575 (0.0007) [2023-12-27 05:36:37,050][105620] Updated weights for policy 1, policy_version 1954231 (0.0007) [2023-12-27 05:36:37,100][105692] Updated weights for policy 0, policy_version 1950585 (0.0010) [2023-12-27 05:36:37,103][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:37,164][105692] Updated weights for policy 0, policy_version 1950595 (0.0011) [2023-12-27 05:36:37,258][105692] Updated weights for policy 0, policy_version 1950605 (0.0011) [2023-12-27 05:36:37,885][105692] Updated weights for policy 0, policy_version 1950615 (0.0011) [2023-12-27 05:36:37,945][105692] Updated weights for policy 0, policy_version 1950625 (0.0011) [2023-12-27 05:36:37,951][105620] Updated weights for policy 1, policy_version 1954241 (0.0008) [2023-12-27 05:36:38,007][105692] Updated weights for policy 0, policy_version 1950635 (0.0011) [2023-12-27 05:36:38,009][105620] Updated weights for policy 1, policy_version 1954251 (0.0006) [2023-12-27 05:36:38,037][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:38,734][105692] Updated weights for policy 0, policy_version 1950645 (0.0011) [2023-12-27 05:36:38,795][105692] Updated weights for policy 0, policy_version 1950655 (0.0010) [2023-12-27 05:36:38,842][105620] Updated weights for policy 1, policy_version 1954261 (0.0006) [2023-12-27 05:36:38,858][105692] Updated weights for policy 0, policy_version 1950665 (0.0008) [2023-12-27 05:36:38,911][105620] Updated weights for policy 1, policy_version 1954271 (0.0007) [2023-12-27 05:36:38,912][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:39,606][105620] Updated weights for policy 1, policy_version 1954281 (0.0008) [2023-12-27 05:36:39,623][105692] Updated weights for policy 0, policy_version 1950675 (0.0008) [2023-12-27 05:36:39,646][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:39,682][105692] Updated weights for policy 0, policy_version 1950685 (0.0008) [2023-12-27 05:36:39,738][105692] Updated weights for policy 0, policy_version 1950695 (0.0010) [2023-12-27 05:36:40,467][105620] Updated weights for policy 1, policy_version 1954291 (0.0008) [2023-12-27 05:36:40,531][105620] Updated weights for policy 1, policy_version 1954301 (0.0006) [2023-12-27 05:36:40,550][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:40,589][105692] Updated weights for policy 0, policy_version 1950705 (0.0010) [2023-12-27 05:36:40,642][105692] Updated weights for policy 0, policy_version 1950715 (0.0010) [2023-12-27 05:36:40,690][105692] Updated weights for policy 0, policy_version 1950725 (0.0010) [2023-12-27 05:36:40,749][105692] Updated weights for policy 0, policy_version 1950735 (0.0010) [2023-12-27 05:36:41,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19466.4). Total num frames: 1000136704. Throughput: 0: 9764.3, 1: 9690.1. Samples: 1000144632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:36:41,063][104569] Avg episode reward: [(0, '8534.756'), (1, '9346.276')] [2023-12-27 05:36:41,340][105620] Updated weights for policy 1, policy_version 1954311 (0.0007) [2023-12-27 05:36:41,399][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:41,595][105692] Updated weights for policy 0, policy_version 1950745 (0.0010) [2023-12-27 05:36:41,659][105692] Updated weights for policy 0, policy_version 1950755 (0.0009) [2023-12-27 05:36:41,730][105692] Updated weights for policy 0, policy_version 1950765 (0.0010) [2023-12-27 05:36:42,146][105620] Updated weights for policy 1, policy_version 1954321 (0.0009) [2023-12-27 05:36:42,210][105620] Updated weights for policy 1, policy_version 1954331 (0.0009) [2023-12-27 05:36:42,240][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:42,433][105692] Updated weights for policy 0, policy_version 1950775 (0.0010) [2023-12-27 05:36:42,503][105692] Updated weights for policy 0, policy_version 1950785 (0.0011) [2023-12-27 05:36:42,562][105692] Updated weights for policy 0, policy_version 1950795 (0.0008) [2023-12-27 05:36:42,978][105620] Updated weights for policy 1, policy_version 1954341 (0.0007) [2023-12-27 05:36:43,042][105620] Updated weights for policy 1, policy_version 1954351 (0.0008) [2023-12-27 05:36:43,049][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:43,295][105692] Updated weights for policy 0, policy_version 1950805 (0.0008) [2023-12-27 05:36:43,353][105692] Updated weights for policy 0, policy_version 1950815 (0.0005) [2023-12-27 05:36:43,405][105692] Updated weights for policy 0, policy_version 1950825 (0.0005) [2023-12-27 05:36:43,713][105620] Updated weights for policy 1, policy_version 1954361 (0.0008) [2023-12-27 05:36:43,749][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:44,013][105692] Updated weights for policy 0, policy_version 1950835 (0.0008) [2023-12-27 05:36:44,064][105692] Updated weights for policy 0, policy_version 1950845 (0.0005) [2023-12-27 05:36:44,120][105692] Updated weights for policy 0, policy_version 1950855 (0.0006) [2023-12-27 05:36:44,417][105620] Updated weights for policy 1, policy_version 1954371 (0.0008) [2023-12-27 05:36:44,478][105620] Updated weights for policy 1, policy_version 1954381 (0.0010) [2023-12-27 05:36:44,490][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:44,685][105692] Updated weights for policy 0, policy_version 1950865 (0.0008) [2023-12-27 05:36:44,752][105692] Updated weights for policy 0, policy_version 1950875 (0.0006) [2023-12-27 05:36:44,822][105692] Updated weights for policy 0, policy_version 1950885 (0.0010) [2023-12-27 05:36:44,881][105692] Updated weights for policy 0, policy_version 1950895 (0.0011) [2023-12-27 05:36:45,236][105620] Updated weights for policy 1, policy_version 1954391 (0.0010) [2023-12-27 05:36:45,284][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:45,590][105692] Updated weights for policy 0, policy_version 1950905 (0.0006) [2023-12-27 05:36:45,641][105692] Updated weights for policy 0, policy_version 1950915 (0.0006) [2023-12-27 05:36:45,697][105692] Updated weights for policy 0, policy_version 1950925 (0.0005) [2023-12-27 05:36:46,011][105620] Updated weights for policy 1, policy_version 1954401 (0.0010) [2023-12-27 05:36:46,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 1000235008. Throughput: 0: 9730.7, 1: 9748.3. Samples: 1000203028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:36:46,062][104569] Avg episode reward: [(0, '8624.386'), (1, '8979.949')] [2023-12-27 05:36:46,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001950928_499507200.pth... [2023-12-27 05:36:46,067][105620] Updated weights for policy 1, policy_version 1954411 (0.0009) [2023-12-27 05:36:46,072][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001949776_499212288.pth [2023-12-27 05:36:46,094][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:46,095][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001954416_500736000.pth... [2023-12-27 05:36:46,098][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001953840_500441088.pth [2023-12-27 05:36:46,345][105692] Updated weights for policy 0, policy_version 1950935 (0.0005) [2023-12-27 05:36:46,399][105692] Updated weights for policy 0, policy_version 1950945 (0.0005) [2023-12-27 05:36:46,470][105692] Updated weights for policy 0, policy_version 1950955 (0.0008) [2023-12-27 05:36:46,731][105620] Updated weights for policy 1, policy_version 1954421 (0.0007) [2023-12-27 05:36:46,790][105620] Updated weights for policy 1, policy_version 1954431 (0.0005) [2023-12-27 05:36:46,791][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:47,144][105692] Updated weights for policy 0, policy_version 1950965 (0.0009) [2023-12-27 05:36:47,207][105692] Updated weights for policy 0, policy_version 1950975 (0.0006) [2023-12-27 05:36:47,269][105692] Updated weights for policy 0, policy_version 1950985 (0.0008) [2023-12-27 05:36:47,493][105620] Updated weights for policy 1, policy_version 1954441 (0.0009) [2023-12-27 05:36:47,536][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:47,941][105692] Updated weights for policy 0, policy_version 1950995 (0.0009) [2023-12-27 05:36:47,996][105692] Updated weights for policy 0, policy_version 1951005 (0.0009) [2023-12-27 05:36:48,056][105692] Updated weights for policy 0, policy_version 1951015 (0.0009) [2023-12-27 05:36:48,327][105620] Updated weights for policy 1, policy_version 1954451 (0.0009) [2023-12-27 05:36:48,396][105620] Updated weights for policy 1, policy_version 1954461 (0.0006) [2023-12-27 05:36:48,413][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:48,847][105692] Updated weights for policy 0, policy_version 1951025 (0.0009) [2023-12-27 05:36:48,895][105692] Updated weights for policy 0, policy_version 1951035 (0.0008) [2023-12-27 05:36:48,955][105692] Updated weights for policy 0, policy_version 1951045 (0.0008) [2023-12-27 05:36:49,012][105692] Updated weights for policy 0, policy_version 1951055 (0.0008) [2023-12-27 05:36:49,183][105620] Updated weights for policy 1, policy_version 1954471 (0.0007) [2023-12-27 05:36:49,243][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:49,708][105692] Updated weights for policy 0, policy_version 1951065 (0.0008) [2023-12-27 05:36:49,753][105692] Updated weights for policy 0, policy_version 1951075 (0.0007) [2023-12-27 05:36:49,804][105692] Updated weights for policy 0, policy_version 1951085 (0.0007) [2023-12-27 05:36:50,091][105620] Updated weights for policy 1, policy_version 1954481 (0.0010) [2023-12-27 05:36:50,159][105620] Updated weights for policy 1, policy_version 1954491 (0.0011) [2023-12-27 05:36:50,189][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:50,628][105692] Updated weights for policy 0, policy_version 1951095 (0.0008) [2023-12-27 05:36:50,685][105692] Updated weights for policy 0, policy_version 1951106 (0.0010) [2023-12-27 05:36:50,745][105692] Updated weights for policy 0, policy_version 1951116 (0.0010) [2023-12-27 05:36:50,893][105620] Updated weights for policy 1, policy_version 1954501 (0.0009) [2023-12-27 05:36:50,953][105620] Updated weights for policy 1, policy_version 1954511 (0.0009) [2023-12-27 05:36:50,955][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:51,062][104569] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 1000341504. Throughput: 0: 9827.0, 1: 9813.5. Samples: 1000325528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:36:51,063][104569] Avg episode reward: [(0, '8354.112'), (1, '8979.890')] [2023-12-27 05:36:51,624][105692] Updated weights for policy 0, policy_version 1951127 (0.0009) [2023-12-27 05:36:51,682][105692] Updated weights for policy 0, policy_version 1951137 (0.0007) [2023-12-27 05:36:51,684][105620] Updated weights for policy 1, policy_version 1954521 (0.0008) [2023-12-27 05:36:51,728][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:51,753][105692] Updated weights for policy 0, policy_version 1951147 (0.0008) [2023-12-27 05:36:52,464][105620] Updated weights for policy 1, policy_version 1954531 (0.0008) [2023-12-27 05:36:52,518][105620] Updated weights for policy 1, policy_version 1954541 (0.0009) [2023-12-27 05:36:52,529][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:52,533][105692] Updated weights for policy 0, policy_version 1951157 (0.0008) [2023-12-27 05:36:52,581][105692] Updated weights for policy 0, policy_version 1951167 (0.0008) [2023-12-27 05:36:52,636][105692] Updated weights for policy 0, policy_version 1951177 (0.0009) [2023-12-27 05:36:53,353][105620] Updated weights for policy 1, policy_version 1954551 (0.0008) [2023-12-27 05:36:53,402][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:53,412][105692] Updated weights for policy 0, policy_version 1951187 (0.0009) [2023-12-27 05:36:53,461][105692] Updated weights for policy 0, policy_version 1951197 (0.0008) [2023-12-27 05:36:53,525][105692] Updated weights for policy 0, policy_version 1951207 (0.0009) [2023-12-27 05:36:54,166][105620] Updated weights for policy 1, policy_version 1954561 (0.0009) [2023-12-27 05:36:54,219][105620] Updated weights for policy 1, policy_version 1954571 (0.0009) [2023-12-27 05:36:54,245][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:54,294][105692] Updated weights for policy 0, policy_version 1951217 (0.0009) [2023-12-27 05:36:54,354][105692] Updated weights for policy 0, policy_version 1951227 (0.0008) [2023-12-27 05:36:54,415][105692] Updated weights for policy 0, policy_version 1951237 (0.0010) [2023-12-27 05:36:54,466][105692] Updated weights for policy 0, policy_version 1951247 (0.0008) [2023-12-27 05:36:55,031][105620] Updated weights for policy 1, policy_version 1954581 (0.0008) [2023-12-27 05:36:55,080][105620] Updated weights for policy 1, policy_version 1954591 (0.0009) [2023-12-27 05:36:55,082][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:55,218][105692] Updated weights for policy 0, policy_version 1951257 (0.0009) [2023-12-27 05:36:55,268][105692] Updated weights for policy 0, policy_version 1951267 (0.0008) [2023-12-27 05:36:55,316][105692] Updated weights for policy 0, policy_version 1951277 (0.0009) [2023-12-27 05:36:55,878][105620] Updated weights for policy 1, policy_version 1954601 (0.0009) [2023-12-27 05:36:55,918][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:56,061][105692] Updated weights for policy 0, policy_version 1951288 (0.0008) [2023-12-27 05:36:56,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 1000431616. Throughput: 0: 9775.9, 1: 9840.4. Samples: 1000439340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:36:56,063][104569] Avg episode reward: [(0, '8262.223'), (1, '9346.303')] [2023-12-27 05:36:56,114][105692] Updated weights for policy 0, policy_version 1951298 (0.0009) [2023-12-27 05:36:56,168][105692] Updated weights for policy 0, policy_version 1951308 (0.0009) [2023-12-27 05:36:56,711][105620] Updated weights for policy 1, policy_version 1954611 (0.0009) [2023-12-27 05:36:56,764][105620] Updated weights for policy 1, policy_version 1954621 (0.0009) [2023-12-27 05:36:56,775][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:56,919][105692] Updated weights for policy 0, policy_version 1951318 (0.0008) [2023-12-27 05:36:56,975][105692] Updated weights for policy 0, policy_version 1951328 (0.0008) [2023-12-27 05:36:57,025][105692] Updated weights for policy 0, policy_version 1951338 (0.0009) [2023-12-27 05:36:57,533][105620] Updated weights for policy 1, policy_version 1954631 (0.0007) [2023-12-27 05:36:57,580][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:57,820][105692] Updated weights for policy 0, policy_version 1951348 (0.0009) [2023-12-27 05:36:57,876][105692] Updated weights for policy 0, policy_version 1951359 (0.0010) [2023-12-27 05:36:57,932][105692] Updated weights for policy 0, policy_version 1951369 (0.0013) [2023-12-27 05:36:58,247][105620] Updated weights for policy 1, policy_version 1954641 (0.0006) [2023-12-27 05:36:58,298][105620] Updated weights for policy 1, policy_version 1954651 (0.0008) [2023-12-27 05:36:58,323][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:58,738][105692] Updated weights for policy 0, policy_version 1951379 (0.0009) [2023-12-27 05:36:58,801][105692] Updated weights for policy 0, policy_version 1951389 (0.0008) [2023-12-27 05:36:58,866][105692] Updated weights for policy 0, policy_version 1951399 (0.0008) [2023-12-27 05:36:59,257][105620] Updated weights for policy 1, policy_version 1954661 (0.0010) [2023-12-27 05:36:59,318][105620] Updated weights for policy 1, policy_version 1954671 (0.0009) [2023-12-27 05:36:59,323][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:36:59,674][105692] Updated weights for policy 0, policy_version 1951409 (0.0008) [2023-12-27 05:36:59,736][105692] Updated weights for policy 0, policy_version 1951419 (0.0009) [2023-12-27 05:36:59,795][105692] Updated weights for policy 0, policy_version 1951429 (0.0009) [2023-12-27 05:36:59,861][105692] Updated weights for policy 0, policy_version 1951439 (0.0008) [2023-12-27 05:37:00,153][105620] Updated weights for policy 1, policy_version 1954681 (0.0006) [2023-12-27 05:37:00,195][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:00,655][105692] Updated weights for policy 0, policy_version 1951449 (0.0009) [2023-12-27 05:37:00,701][105692] Updated weights for policy 0, policy_version 1951459 (0.0008) [2023-12-27 05:37:00,748][105692] Updated weights for policy 0, policy_version 1951469 (0.0008) [2023-12-27 05:37:00,880][105620] Updated weights for policy 1, policy_version 1954691 (0.0007) [2023-12-27 05:37:00,935][105620] Updated weights for policy 1, policy_version 1954702 (0.0009) [2023-12-27 05:37:00,942][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:01,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 1000529920. Throughput: 0: 9752.3, 1: 9807.8. Samples: 1000496708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:37:01,062][104569] Avg episode reward: [(0, '8351.477'), (1, '9165.958')] [2023-12-27 05:37:01,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001951472_499646464.pth... [2023-12-27 05:37:01,069][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001954704_500883456.pth... [2023-12-27 05:37:01,073][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001950352_499359744.pth [2023-12-27 05:37:01,073][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001954128_500588544.pth [2023-12-27 05:37:01,599][105692] Updated weights for policy 0, policy_version 1951479 (0.0009) [2023-12-27 05:37:01,668][105692] Updated weights for policy 0, policy_version 1951489 (0.0010) [2023-12-27 05:37:01,704][105620] Updated weights for policy 1, policy_version 1954712 (0.0006) [2023-12-27 05:37:01,729][105692] Updated weights for policy 0, policy_version 1951499 (0.0008) [2023-12-27 05:37:01,761][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:02,461][105692] Updated weights for policy 0, policy_version 1951509 (0.0009) [2023-12-27 05:37:02,521][105692] Updated weights for policy 0, policy_version 1951519 (0.0006) [2023-12-27 05:37:02,577][105692] Updated weights for policy 0, policy_version 1951529 (0.0007) [2023-12-27 05:37:02,583][105620] Updated weights for policy 1, policy_version 1954722 (0.0008) [2023-12-27 05:37:02,638][105620] Updated weights for policy 1, policy_version 1954732 (0.0007) [2023-12-27 05:37:02,656][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:03,260][105692] Updated weights for policy 0, policy_version 1951539 (0.0009) [2023-12-27 05:37:03,313][105692] Updated weights for policy 0, policy_version 1951550 (0.0010) [2023-12-27 05:37:03,365][105692] Updated weights for policy 0, policy_version 1951561 (0.0010) [2023-12-27 05:37:03,387][105620] Updated weights for policy 1, policy_version 1954742 (0.0008) [2023-12-27 05:37:03,429][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:03,433][105620] Updated weights for policy 1, policy_version 1954752 (0.0008) [2023-12-27 05:37:04,178][105620] Updated weights for policy 1, policy_version 1954762 (0.0008) [2023-12-27 05:37:04,187][105692] Updated weights for policy 0, policy_version 1951571 (0.0008) [2023-12-27 05:37:04,211][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:04,239][105692] Updated weights for policy 0, policy_version 1951581 (0.0008) [2023-12-27 05:37:04,286][105692] Updated weights for policy 0, policy_version 1951591 (0.0009) [2023-12-27 05:37:04,967][105620] Updated weights for policy 1, policy_version 1954772 (0.0006) [2023-12-27 05:37:05,029][105620] Updated weights for policy 1, policy_version 1954782 (0.0008) [2023-12-27 05:37:05,035][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:05,059][105692] Updated weights for policy 0, policy_version 1951601 (0.0010) [2023-12-27 05:37:05,114][105692] Updated weights for policy 0, policy_version 1951611 (0.0009) [2023-12-27 05:37:05,161][105692] Updated weights for policy 0, policy_version 1951621 (0.0008) [2023-12-27 05:37:05,209][105692] Updated weights for policy 0, policy_version 1951631 (0.0009) [2023-12-27 05:37:05,842][105620] Updated weights for policy 1, policy_version 1954792 (0.0009) [2023-12-27 05:37:05,885][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:05,943][105692] Updated weights for policy 0, policy_version 1951641 (0.0009) [2023-12-27 05:37:06,000][105692] Updated weights for policy 0, policy_version 1951651 (0.0009) [2023-12-27 05:37:06,055][105692] Updated weights for policy 0, policy_version 1951661 (0.0009) [2023-12-27 05:37:06,062][104569] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 1000620032. Throughput: 0: 9574.3, 1: 9847.9. Samples: 1000610692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:37:06,062][104569] Avg episode reward: [(0, '8625.803'), (1, '9080.110')] [2023-12-27 05:37:06,700][105620] Updated weights for policy 1, policy_version 1954802 (0.0009) [2023-12-27 05:37:06,761][105620] Updated weights for policy 1, policy_version 1954812 (0.0010) [2023-12-27 05:37:06,787][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:06,807][105692] Updated weights for policy 0, policy_version 1951671 (0.0008) [2023-12-27 05:37:06,859][105692] Updated weights for policy 0, policy_version 1951681 (0.0007) [2023-12-27 05:37:06,909][105692] Updated weights for policy 0, policy_version 1951691 (0.0010) [2023-12-27 05:37:07,525][105692] Updated weights for policy 0, policy_version 1951701 (0.0010) [2023-12-27 05:37:07,590][105692] Updated weights for policy 0, policy_version 1951711 (0.0011) [2023-12-27 05:37:07,657][105692] Updated weights for policy 0, policy_version 1951721 (0.0011) [2023-12-27 05:37:07,680][105620] Updated weights for policy 1, policy_version 1954822 (0.0007) [2023-12-27 05:37:07,740][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:07,742][105620] Updated weights for policy 1, policy_version 1954832 (0.0008) [2023-12-27 05:37:08,343][105692] Updated weights for policy 0, policy_version 1951731 (0.0010) [2023-12-27 05:37:08,407][105692] Updated weights for policy 0, policy_version 1951741 (0.0008) [2023-12-27 05:37:08,471][105692] Updated weights for policy 0, policy_version 1951751 (0.0011) [2023-12-27 05:37:08,481][105620] Updated weights for policy 1, policy_version 1954842 (0.0011) [2023-12-27 05:37:08,509][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:09,210][105620] Updated weights for policy 1, policy_version 1954852 (0.0009) [2023-12-27 05:37:09,276][105620] Updated weights for policy 1, policy_version 1954862 (0.0006) [2023-12-27 05:37:09,287][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:09,300][105692] Updated weights for policy 0, policy_version 1951761 (0.0011) [2023-12-27 05:37:09,358][105692] Updated weights for policy 0, policy_version 1951771 (0.0009) [2023-12-27 05:37:09,433][105692] Updated weights for policy 0, policy_version 1951781 (0.0008) [2023-12-27 05:37:09,494][105692] Updated weights for policy 0, policy_version 1951791 (0.0009) [2023-12-27 05:37:10,095][105620] Updated weights for policy 1, policy_version 1954872 (0.0009) [2023-12-27 05:37:10,146][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:10,269][105692] Updated weights for policy 0, policy_version 1951801 (0.0010) [2023-12-27 05:37:10,328][105692] Updated weights for policy 0, policy_version 1951811 (0.0009) [2023-12-27 05:37:10,385][105692] Updated weights for policy 0, policy_version 1951821 (0.0008) [2023-12-27 05:37:10,850][105620] Updated weights for policy 1, policy_version 1954882 (0.0008) [2023-12-27 05:37:10,907][105620] Updated weights for policy 1, policy_version 1954892 (0.0006) [2023-12-27 05:37:10,928][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:11,062][104569] Fps is (10 sec: 18841.5, 60 sec: 19390.3, 300 sec: 19410.9). Total num frames: 1000718336. Throughput: 0: 9520.0, 1: 9868.2. Samples: 1000726428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:37:11,063][104569] Avg episode reward: [(0, '8715.609'), (1, '9260.472')] [2023-12-27 05:37:11,198][105692] Updated weights for policy 0, policy_version 1951831 (0.0010) [2023-12-27 05:37:11,258][105692] Updated weights for policy 0, policy_version 1951841 (0.0011) [2023-12-27 05:37:11,318][105692] Updated weights for policy 0, policy_version 1951851 (0.0011) [2023-12-27 05:37:11,671][105620] Updated weights for policy 1, policy_version 1954902 (0.0007) [2023-12-27 05:37:11,737][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:11,738][105620] Updated weights for policy 1, policy_version 1954912 (0.0007) [2023-12-27 05:37:12,069][105692] Updated weights for policy 0, policy_version 1951861 (0.0009) [2023-12-27 05:37:12,135][105692] Updated weights for policy 0, policy_version 1951871 (0.0006) [2023-12-27 05:37:12,207][105692] Updated weights for policy 0, policy_version 1951881 (0.0007) [2023-12-27 05:37:12,536][105620] Updated weights for policy 1, policy_version 1954922 (0.0008) [2023-12-27 05:37:12,567][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:12,902][105692] Updated weights for policy 0, policy_version 1951891 (0.0007) [2023-12-27 05:37:12,954][105692] Updated weights for policy 0, policy_version 1951901 (0.0011) [2023-12-27 05:37:13,014][105692] Updated weights for policy 0, policy_version 1951911 (0.0011) [2023-12-27 05:37:13,400][105620] Updated weights for policy 1, policy_version 1954932 (0.0008) [2023-12-27 05:37:13,451][105620] Updated weights for policy 1, policy_version 1954942 (0.0008) [2023-12-27 05:37:13,459][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:13,769][105692] Updated weights for policy 0, policy_version 1951921 (0.0011) [2023-12-27 05:37:13,823][105692] Updated weights for policy 0, policy_version 1951931 (0.0008) [2023-12-27 05:37:13,884][105692] Updated weights for policy 0, policy_version 1951941 (0.0005) [2023-12-27 05:37:13,954][105692] Updated weights for policy 0, policy_version 1951951 (0.0005) [2023-12-27 05:37:14,351][105620] Updated weights for policy 1, policy_version 1954953 (0.0010) [2023-12-27 05:37:14,379][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:14,463][105692] Updated weights for policy 0, policy_version 1951961 (0.0006) [2023-12-27 05:37:14,519][105692] Updated weights for policy 0, policy_version 1951971 (0.0007) [2023-12-27 05:37:14,580][105692] Updated weights for policy 0, policy_version 1951981 (0.0010) [2023-12-27 05:37:15,259][105620] Updated weights for policy 1, policy_version 1954964 (0.0008) [2023-12-27 05:37:15,279][105692] Updated weights for policy 0, policy_version 1951991 (0.0010) [2023-12-27 05:37:15,315][105620] Updated weights for policy 1, policy_version 1954974 (0.0007) [2023-12-27 05:37:15,324][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:15,335][105692] Updated weights for policy 0, policy_version 1952001 (0.0010) [2023-12-27 05:37:15,395][105692] Updated weights for policy 0, policy_version 1952011 (0.0011) [2023-12-27 05:37:16,055][105620] Updated weights for policy 1, policy_version 1954984 (0.0008) [2023-12-27 05:37:16,062][104569] Fps is (10 sec: 18841.6, 60 sec: 19251.3, 300 sec: 19355.3). Total num frames: 1000808448. Throughput: 0: 9475.2, 1: 9826.1. Samples: 1000783012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:37:16,062][104569] Avg episode reward: [(0, '8079.147'), (1, '9346.303')] [2023-12-27 05:37:16,098][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:16,100][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001954992_501030912.pth... [2023-12-27 05:37:16,100][105692] Updated weights for policy 0, policy_version 1952021 (0.0010) [2023-12-27 05:37:16,104][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001954416_500736000.pth [2023-12-27 05:37:16,153][105692] Updated weights for policy 0, policy_version 1952031 (0.0007) [2023-12-27 05:37:16,204][105692] Updated weights for policy 0, policy_version 1952041 (0.0010) [2023-12-27 05:37:16,242][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001952048_499793920.pth... [2023-12-27 05:37:16,245][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001950928_499507200.pth [2023-12-27 05:37:16,808][105692] Updated weights for policy 0, policy_version 1952051 (0.0009) [2023-12-27 05:37:16,857][105692] Updated weights for policy 0, policy_version 1952061 (0.0005) [2023-12-27 05:37:16,905][105692] Updated weights for policy 0, policy_version 1952071 (0.0006) [2023-12-27 05:37:16,988][105620] Updated weights for policy 1, policy_version 1954994 (0.0009) [2023-12-27 05:37:17,043][105620] Updated weights for policy 1, policy_version 1955004 (0.0010) [2023-12-27 05:37:17,062][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:17,443][105692] Updated weights for policy 0, policy_version 1952081 (0.0005) [2023-12-27 05:37:17,503][105692] Updated weights for policy 0, policy_version 1952091 (0.0005) [2023-12-27 05:37:17,558][105692] Updated weights for policy 0, policy_version 1952101 (0.0005) [2023-12-27 05:37:17,615][105692] Updated weights for policy 0, policy_version 1952111 (0.0005) [2023-12-27 05:37:17,972][105620] Updated weights for policy 1, policy_version 1955015 (0.0010) [2023-12-27 05:37:18,014][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:18,132][105692] Updated weights for policy 0, policy_version 1952121 (0.0006) [2023-12-27 05:37:18,191][105692] Updated weights for policy 0, policy_version 1952131 (0.0008) [2023-12-27 05:37:18,239][105692] Updated weights for policy 0, policy_version 1952141 (0.0010) [2023-12-27 05:37:18,873][105620] Updated weights for policy 1, policy_version 1955026 (0.0010) [2023-12-27 05:37:18,933][105620] Updated weights for policy 1, policy_version 1955036 (0.0006) [2023-12-27 05:37:18,954][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:18,963][105692] Updated weights for policy 0, policy_version 1952151 (0.0010) [2023-12-27 05:37:19,019][105692] Updated weights for policy 0, policy_version 1952161 (0.0010) [2023-12-27 05:37:19,078][105692] Updated weights for policy 0, policy_version 1952171 (0.0011) [2023-12-27 05:37:19,667][105620] Updated weights for policy 1, policy_version 1955046 (0.0009) [2023-12-27 05:37:19,719][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:19,721][105620] Updated weights for policy 1, policy_version 1955056 (0.0010) [2023-12-27 05:37:19,742][105692] Updated weights for policy 0, policy_version 1952181 (0.0011) [2023-12-27 05:37:19,795][105692] Updated weights for policy 0, policy_version 1952191 (0.0011) [2023-12-27 05:37:19,860][105692] Updated weights for policy 0, policy_version 1952201 (0.0011) [2023-12-27 05:37:20,546][105620] Updated weights for policy 1, policy_version 1955066 (0.0010) [2023-12-27 05:37:20,583][105692] Updated weights for policy 0, policy_version 1952211 (0.0010) [2023-12-27 05:37:20,586][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:20,646][105692] Updated weights for policy 0, policy_version 1952221 (0.0009) [2023-12-27 05:37:20,713][105692] Updated weights for policy 0, policy_version 1952231 (0.0007) [2023-12-27 05:37:21,062][104569] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 1000914944. Throughput: 0: 9606.6, 1: 9754.7. Samples: 1000902868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:37:21,063][104569] Avg episode reward: [(0, '8170.008'), (1, '9346.298')] [2023-12-27 05:37:21,407][105620] Updated weights for policy 1, policy_version 1955076 (0.0010) [2023-12-27 05:37:21,437][105692] Updated weights for policy 0, policy_version 1952241 (0.0008) [2023-12-27 05:37:21,469][105620] Updated weights for policy 1, policy_version 1955086 (0.0009) [2023-12-27 05:37:21,477][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:21,497][105692] Updated weights for policy 0, policy_version 1952251 (0.0011) [2023-12-27 05:37:21,548][105692] Updated weights for policy 0, policy_version 1952261 (0.0010) [2023-12-27 05:37:21,608][105692] Updated weights for policy 0, policy_version 1952271 (0.0010) [2023-12-27 05:37:22,331][105692] Updated weights for policy 0, policy_version 1952281 (0.0011) [2023-12-27 05:37:22,338][105620] Updated weights for policy 1, policy_version 1955096 (0.0008) [2023-12-27 05:37:22,390][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:22,391][105692] Updated weights for policy 0, policy_version 1952291 (0.0011) [2023-12-27 05:37:22,438][105692] Updated weights for policy 0, policy_version 1952301 (0.0010) [2023-12-27 05:37:23,076][105620] Updated weights for policy 1, policy_version 1955106 (0.0006) [2023-12-27 05:37:23,088][105692] Updated weights for policy 0, policy_version 1952311 (0.0007) [2023-12-27 05:37:23,143][105620] Updated weights for policy 1, policy_version 1955116 (0.0005) [2023-12-27 05:37:23,151][105692] Updated weights for policy 0, policy_version 1952321 (0.0005) [2023-12-27 05:37:23,167][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:23,220][105692] Updated weights for policy 0, policy_version 1952331 (0.0010) [2023-12-27 05:37:23,749][105620] Updated weights for policy 1, policy_version 1955126 (0.0005) [2023-12-27 05:37:23,815][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:23,815][105620] Updated weights for policy 1, policy_version 1955136 (0.0005) [2023-12-27 05:37:23,883][105692] Updated weights for policy 0, policy_version 1952341 (0.0010) [2023-12-27 05:37:23,931][105692] Updated weights for policy 0, policy_version 1952351 (0.0010) [2023-12-27 05:37:23,983][105692] Updated weights for policy 0, policy_version 1952361 (0.0010) [2023-12-27 05:37:24,408][105620] Updated weights for policy 1, policy_version 1955146 (0.0010) [2023-12-27 05:37:24,440][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:24,709][105692] Updated weights for policy 0, policy_version 1952371 (0.0009) [2023-12-27 05:37:24,760][105692] Updated weights for policy 0, policy_version 1952381 (0.0005) [2023-12-27 05:37:24,818][105692] Updated weights for policy 0, policy_version 1952391 (0.0005) [2023-12-27 05:37:25,156][105620] Updated weights for policy 1, policy_version 1955156 (0.0008) [2023-12-27 05:37:25,210][105620] Updated weights for policy 1, policy_version 1955166 (0.0005) [2023-12-27 05:37:25,218][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:25,500][105692] Updated weights for policy 0, policy_version 1952401 (0.0005) [2023-12-27 05:37:25,554][105692] Updated weights for policy 0, policy_version 1952411 (0.0008) [2023-12-27 05:37:25,606][105692] Updated weights for policy 0, policy_version 1952421 (0.0008) [2023-12-27 05:37:25,659][105692] Updated weights for policy 0, policy_version 1952431 (0.0008) [2023-12-27 05:37:25,948][105620] Updated weights for policy 1, policy_version 1955176 (0.0009) [2023-12-27 05:37:25,997][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:26,062][104569] Fps is (10 sec: 21298.6, 60 sec: 19660.7, 300 sec: 19438.6). Total num frames: 1001021440. Throughput: 0: 9667.1, 1: 9910.2. Samples: 1001025616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:37:26,063][104569] Avg episode reward: [(0, '8080.924'), (1, '9253.975')] [2023-12-27 05:37:26,419][105692] Updated weights for policy 0, policy_version 1952441 (0.0008) [2023-12-27 05:37:26,478][105692] Updated weights for policy 0, policy_version 1952451 (0.0006) [2023-12-27 05:37:26,538][105692] Updated weights for policy 0, policy_version 1952461 (0.0006) [2023-12-27 05:37:26,763][105620] Updated weights for policy 1, policy_version 1955186 (0.0009) [2023-12-27 05:37:26,809][105620] Updated weights for policy 1, policy_version 1955196 (0.0005) [2023-12-27 05:37:26,830][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:27,300][105692] Updated weights for policy 0, policy_version 1952471 (0.0009) [2023-12-27 05:37:27,354][105692] Updated weights for policy 0, policy_version 1952481 (0.0009) [2023-12-27 05:37:27,401][105692] Updated weights for policy 0, policy_version 1952491 (0.0007) [2023-12-27 05:37:27,528][105620] Updated weights for policy 1, policy_version 1955206 (0.0008) [2023-12-27 05:37:27,572][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:27,573][105620] Updated weights for policy 1, policy_version 1955216 (0.0010) [2023-12-27 05:37:28,126][105692] Updated weights for policy 0, policy_version 1952501 (0.0008) [2023-12-27 05:37:28,172][105692] Updated weights for policy 0, policy_version 1952511 (0.0007) [2023-12-27 05:37:28,216][105692] Updated weights for policy 0, policy_version 1952521 (0.0008) [2023-12-27 05:37:28,407][105620] Updated weights for policy 1, policy_version 1955226 (0.0007) [2023-12-27 05:37:28,436][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:29,017][105692] Updated weights for policy 0, policy_version 1952531 (0.0009) [2023-12-27 05:37:29,075][105692] Updated weights for policy 0, policy_version 1952541 (0.0010) [2023-12-27 05:37:29,133][105692] Updated weights for policy 0, policy_version 1952551 (0.0010) [2023-12-27 05:37:29,158][105620] Updated weights for policy 1, policy_version 1955236 (0.0009) [2023-12-27 05:37:29,204][105620] Updated weights for policy 1, policy_version 1955246 (0.0010) [2023-12-27 05:37:29,211][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:29,757][105692] Updated weights for policy 0, policy_version 1952561 (0.0010) [2023-12-27 05:37:29,823][105692] Updated weights for policy 0, policy_version 1952571 (0.0010) [2023-12-27 05:37:29,889][105692] Updated weights for policy 0, policy_version 1952581 (0.0010) [2023-12-27 05:37:29,895][105620] Updated weights for policy 1, policy_version 1955256 (0.0011) [2023-12-27 05:37:29,943][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:29,950][105692] Updated weights for policy 0, policy_version 1952591 (0.0011) [2023-12-27 05:37:30,665][105692] Updated weights for policy 0, policy_version 1952601 (0.0006) [2023-12-27 05:37:30,709][105692] Updated weights for policy 0, policy_version 1952611 (0.0005) [2023-12-27 05:37:30,741][105620] Updated weights for policy 1, policy_version 1955266 (0.0010) [2023-12-27 05:37:30,763][105692] Updated weights for policy 0, policy_version 1952621 (0.0006) [2023-12-27 05:37:30,796][105620] Updated weights for policy 1, policy_version 1955276 (0.0011) [2023-12-27 05:37:30,815][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:31,062][104569] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 1001119744. Throughput: 0: 9673.2, 1: 9895.6. Samples: 1001083624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:37:31,062][104569] Avg episode reward: [(0, '8542.314'), (1, '9070.466')] [2023-12-27 05:37:31,067][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001952624_499941376.pth... [2023-12-27 05:37:31,068][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001955280_501178368.pth... [2023-12-27 05:37:31,071][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001951472_499646464.pth [2023-12-27 05:37:31,071][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001954704_500883456.pth [2023-12-27 05:37:31,409][105692] Updated weights for policy 0, policy_version 1952631 (0.0009) [2023-12-27 05:37:31,471][105692] Updated weights for policy 0, policy_version 1952641 (0.0010) [2023-12-27 05:37:31,538][105692] Updated weights for policy 0, policy_version 1952651 (0.0010) [2023-12-27 05:37:31,639][105620] Updated weights for policy 1, policy_version 1955286 (0.0009) [2023-12-27 05:37:31,700][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:31,701][105620] Updated weights for policy 1, policy_version 1955296 (0.0009) [2023-12-27 05:37:32,255][105692] Updated weights for policy 0, policy_version 1952661 (0.0009) [2023-12-27 05:37:32,306][105692] Updated weights for policy 0, policy_version 1952671 (0.0009) [2023-12-27 05:37:32,362][105692] Updated weights for policy 0, policy_version 1952681 (0.0009) [2023-12-27 05:37:32,561][105620] Updated weights for policy 1, policy_version 1955306 (0.0005) [2023-12-27 05:37:32,590][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:33,003][105692] Updated weights for policy 0, policy_version 1952691 (0.0009) [2023-12-27 05:37:33,067][105692] Updated weights for policy 0, policy_version 1952701 (0.0009) [2023-12-27 05:37:33,127][105692] Updated weights for policy 0, policy_version 1952711 (0.0008) [2023-12-27 05:37:33,380][105620] Updated weights for policy 1, policy_version 1955316 (0.0007) [2023-12-27 05:37:33,439][105620] Updated weights for policy 1, policy_version 1955326 (0.0010) [2023-12-27 05:37:33,447][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:33,729][105692] Updated weights for policy 0, policy_version 1952721 (0.0008) [2023-12-27 05:37:33,786][105692] Updated weights for policy 0, policy_version 1952731 (0.0005) [2023-12-27 05:37:33,843][105692] Updated weights for policy 0, policy_version 1952741 (0.0006) [2023-12-27 05:37:33,893][105692] Updated weights for policy 0, policy_version 1952751 (0.0005) [2023-12-27 05:37:34,253][105620] Updated weights for policy 1, policy_version 1955336 (0.0008) [2023-12-27 05:37:34,298][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:34,471][105692] Updated weights for policy 0, policy_version 1952761 (0.0010) [2023-12-27 05:37:34,533][105692] Updated weights for policy 0, policy_version 1952771 (0.0011) [2023-12-27 05:37:34,590][105692] Updated weights for policy 0, policy_version 1952781 (0.0007) [2023-12-27 05:37:34,986][105620] Updated weights for policy 1, policy_version 1955346 (0.0009) [2023-12-27 05:37:35,046][105620] Updated weights for policy 1, policy_version 1955356 (0.0010) [2023-12-27 05:37:35,068][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:35,302][105692] Updated weights for policy 0, policy_version 1952791 (0.0010) [2023-12-27 05:37:35,356][105692] Updated weights for policy 0, policy_version 1952801 (0.0010) [2023-12-27 05:37:35,410][105692] Updated weights for policy 0, policy_version 1952811 (0.0010) [2023-12-27 05:37:35,829][105620] Updated weights for policy 1, policy_version 1955366 (0.0009) [2023-12-27 05:37:35,885][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:35,887][105620] Updated weights for policy 1, policy_version 1955376 (0.0007) [2023-12-27 05:37:36,062][104569] Fps is (10 sec: 19661.1, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 1001218048. Throughput: 0: 9718.4, 1: 9831.2. Samples: 1001205260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:37:36,063][104569] Avg episode reward: [(0, '8906.094'), (1, '9162.820')] [2023-12-27 05:37:36,169][105692] Updated weights for policy 0, policy_version 1952821 (0.0010) [2023-12-27 05:37:36,236][105692] Updated weights for policy 0, policy_version 1952831 (0.0009) [2023-12-27 05:37:36,306][105692] Updated weights for policy 0, policy_version 1952841 (0.0011) [2023-12-27 05:37:36,629][105620] Updated weights for policy 1, policy_version 1955386 (0.0007) [2023-12-27 05:37:36,665][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:37,017][105692] Updated weights for policy 0, policy_version 1952851 (0.0011) [2023-12-27 05:37:37,076][105692] Updated weights for policy 0, policy_version 1952861 (0.0010) [2023-12-27 05:37:37,131][105692] Updated weights for policy 0, policy_version 1952871 (0.0010) [2023-12-27 05:37:37,305][105620] Updated weights for policy 1, policy_version 1955396 (0.0006) [2023-12-27 05:37:37,365][105620] Updated weights for policy 1, policy_version 1955406 (0.0005) [2023-12-27 05:37:37,374][105586] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 05:37:37,376][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001955408_501243904.pth... [2023-12-27 05:37:37,376][105715] Stopping RolloutWorker_w7... [2023-12-27 05:37:37,376][105707] Stopping RolloutWorker_w6... [2023-12-27 05:37:37,376][105728] Stopping RolloutWorker_w12... [2023-12-27 05:37:37,376][105726] Stopping RolloutWorker_w14... [2023-12-27 05:37:37,376][105707] Loop rollout_proc6_evt_loop terminating... [2023-12-27 05:37:37,376][105728] Loop rollout_proc12_evt_loop terminating... [2023-12-27 05:37:37,376][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001952880_500006912.pth... [2023-12-27 05:37:37,376][105715] Loop rollout_proc7_evt_loop terminating... [2023-12-27 05:37:37,376][105726] Loop rollout_proc14_evt_loop terminating... [2023-12-27 05:37:37,376][104569] Component RolloutWorker_w7 stopped! [2023-12-27 05:37:37,377][105698] Stopping RolloutWorker_w2... [2023-12-27 05:37:37,377][105725] Stopping RolloutWorker_w10... [2023-12-27 05:37:37,377][105701] Stopping RolloutWorker_w4... [2023-12-27 05:37:37,377][105718] Stopping RolloutWorker_w8... [2023-12-27 05:37:37,377][104569] Component RolloutWorker_w6 stopped! [2023-12-27 05:37:37,377][105688] Stopping RolloutWorker_w0... [2023-12-27 05:37:37,377][105702] Stopping RolloutWorker_w5... [2023-12-27 05:37:37,377][105700] Stopping RolloutWorker_w1... [2023-12-27 05:37:37,377][105725] Loop rollout_proc10_evt_loop terminating... [2023-12-27 05:37:37,377][105698] Loop rollout_proc2_evt_loop terminating... [2023-12-27 05:37:37,377][105724] Stopping RolloutWorker_w11... [2023-12-27 05:37:37,377][105701] Loop rollout_proc4_evt_loop terminating... [2023-12-27 05:37:37,377][105718] Loop rollout_proc8_evt_loop terminating... [2023-12-27 05:37:37,377][105702] Loop rollout_proc5_evt_loop terminating... [2023-12-27 05:37:37,377][105688] Loop rollout_proc0_evt_loop terminating... [2023-12-27 05:37:37,378][105700] Loop rollout_proc1_evt_loop terminating... [2023-12-27 05:37:37,377][104569] Component RolloutWorker_w12 stopped! [2023-12-27 05:37:37,378][105724] Loop rollout_proc11_evt_loop terminating... [2023-12-27 05:37:37,378][105699] Stopping RolloutWorker_w3... [2023-12-27 05:37:37,378][105727] Stopping RolloutWorker_w15... [2023-12-27 05:37:37,378][104569] Component RolloutWorker_w14 stopped! [2023-12-27 05:37:37,378][105723] Stopping RolloutWorker_w9... [2023-12-27 05:37:37,378][105699] Loop rollout_proc3_evt_loop terminating... [2023-12-27 05:37:37,378][105765] Stopping RolloutWorker_w13... [2023-12-27 05:37:37,378][104569] Component Batcher_1 stopped! [2023-12-27 05:37:37,378][105727] Loop rollout_proc15_evt_loop terminating... [2023-12-27 05:37:37,378][105723] Loop rollout_proc9_evt_loop terminating... [2023-12-27 05:37:37,378][105765] Loop rollout_proc13_evt_loop terminating... [2023-12-27 05:37:37,379][104569] Component Batcher_0 stopped! [2023-12-27 05:37:37,379][104569] Component RolloutWorker_w10 stopped! [2023-12-27 05:37:37,380][104569] Component RolloutWorker_w2 stopped! [2023-12-27 05:37:37,380][104569] Component RolloutWorker_w4 stopped! [2023-12-27 05:37:37,380][104569] Component RolloutWorker_w8 stopped! [2023-12-27 05:37:37,381][104569] Component RolloutWorker_w0 stopped! [2023-12-27 05:37:37,376][105586] Stopping Batcher_1... [2023-12-27 05:37:37,381][104569] Component RolloutWorker_w5 stopped! [2023-12-27 05:37:37,376][105585] Stopping Batcher_0... [2023-12-27 05:37:37,381][104569] Component RolloutWorker_w1 stopped! [2023-12-27 05:37:37,381][105586] Loop batcher_evt_loop terminating... [2023-12-27 05:37:37,382][104569] Component RolloutWorker_w11 stopped! [2023-12-27 05:37:37,382][105585] Loop batcher_evt_loop terminating... [2023-12-27 05:37:37,382][105586] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001954992_501030912.pth [2023-12-27 05:37:37,382][104569] Component RolloutWorker_w3 stopped! [2023-12-27 05:37:37,382][105585] Removing ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001952048_499793920.pth [2023-12-27 05:37:37,382][104569] Component RolloutWorker_w15 stopped! [2023-12-27 05:37:37,383][104569] Component RolloutWorker_w9 stopped! [2023-12-27 05:37:37,383][104569] Component RolloutWorker_w13 stopped! [2023-12-27 05:37:37,383][105585] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p0/checkpoint_001952880_500006912.pth... [2023-12-27 05:37:37,383][105586] Saving ./train_mujoco/mujoco_doublependulum_APPO/checkpoint_p1/checkpoint_001955408_501243904.pth... [2023-12-27 05:37:37,387][105585] Stopping LearnerWorker_p0... [2023-12-27 05:37:37,387][105585] Loop learner_proc0_evt_loop terminating... [2023-12-27 05:37:37,387][104569] Component LearnerWorker_p0 stopped! [2023-12-27 05:37:37,389][105586] Stopping LearnerWorker_p1... [2023-12-27 05:37:37,389][104569] Component LearnerWorker_p1 stopped! [2023-12-27 05:37:37,389][105586] Loop learner_proc1_evt_loop terminating... [2023-12-27 05:37:37,407][105692] Weights refcount: 2 0 [2023-12-27 05:37:37,409][105692] Stopping InferenceWorker_p0-w0... [2023-12-27 05:37:37,409][105692] Loop inference_proc0-0_evt_loop terminating... [2023-12-27 05:37:37,409][104569] Component InferenceWorker_p0-w0 stopped! [2023-12-27 05:37:37,410][105620] Weights refcount: 2 0 [2023-12-27 05:37:37,412][105620] Stopping InferenceWorker_p1-w0... [2023-12-27 05:37:37,412][104569] Component InferenceWorker_p1-w0 stopped! [2023-12-27 05:37:37,412][105620] Loop inference_proc1-0_evt_loop terminating... [2023-12-27 05:37:37,413][104569] Waiting for process learner_proc0 to stop... [2023-12-27 05:37:38,101][104569] Waiting for process learner_proc1 to stop... [2023-12-27 05:37:38,248][104569] Waiting for process inference_proc0-0 to join... [2023-12-27 05:37:38,278][104569] Waiting for process inference_proc1-0 to join... [2023-12-27 05:37:38,307][104569] Waiting for process rollout_proc0 to join... [2023-12-27 05:37:38,308][104569] Waiting for process rollout_proc1 to join... [2023-12-27 05:37:38,309][104569] Waiting for process rollout_proc2 to join... [2023-12-27 05:37:38,309][104569] Waiting for process rollout_proc3 to join... [2023-12-27 05:37:38,310][104569] Waiting for process rollout_proc4 to join... [2023-12-27 05:37:38,311][104569] Waiting for process rollout_proc5 to join... [2023-12-27 05:37:38,311][104569] Waiting for process rollout_proc6 to join... [2023-12-27 05:37:38,312][104569] Waiting for process rollout_proc7 to join... [2023-12-27 05:37:38,313][104569] Waiting for process rollout_proc8 to join... [2023-12-27 05:37:38,313][104569] Waiting for process rollout_proc9 to join... [2023-12-27 05:37:38,314][104569] Waiting for process rollout_proc10 to join... [2023-12-27 05:37:38,315][104569] Waiting for process rollout_proc11 to join... [2023-12-27 05:37:38,315][104569] Waiting for process rollout_proc12 to join... [2023-12-27 05:37:38,316][104569] Waiting for process rollout_proc13 to join... [2023-12-27 05:37:38,316][104569] Waiting for process rollout_proc14 to join... [2023-12-27 05:37:38,317][104569] Waiting for process rollout_proc15 to join... [2023-12-27 05:37:38,317][104569] Batcher 0 profile tree view: batching: 742.2411, releasing_batches: 1.1973 [2023-12-27 05:37:38,317][104569] Batcher 1 profile tree view: batching: 751.8209, releasing_batches: 1.1634 [2023-12-27 05:37:38,317][104569] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0001 wait_policy_total: 1094.2181 update_model: 2088.2275 weight_update: 0.0010 one_step: 0.0039 handle_policy_step: 45826.5666 deserialize: 454.3553, stack: 517.6465, obs_to_device_normalize: 6580.5029, forward: 16296.5916, prepare_outputs: 17029.0690, send_messages: 3128.6268 [2023-12-27 05:37:38,318][104569] InferenceWorker_p1-w0 profile tree view: wait_policy: 0.0001 wait_policy_total: 1089.3136 update_model: 2086.0585 weight_update: 0.0006 one_step: 0.0036 handle_policy_step: 45881.9726 deserialize: 458.2750, stack: 515.2184, obs_to_device_normalize: 6578.3992, forward: 16288.5389, prepare_outputs: 17200.6668, send_messages: 3040.2951 [2023-12-27 05:37:38,318][104569] Learner 0 profile tree view: misc: 0.3134, prepare_batch: 404.6856 train: 11382.6517 epoch_init: 3.9273, minibatch_init: 212.6033, losses_postprocess: 418.8516, kl_divergence: 229.8278, update: 5939.6446, after_optimizer: 157.7329 calculate_losses: 4189.7318 losses_init: 7.5842, forward_head: 465.9391, bptt_initial: 24.8580, bptt: 27.3912, tail: 1440.4960, advantages_returns: 207.0390, losses: 1768.6326 [2023-12-27 05:37:38,319][104569] Learner 1 profile tree view: misc: 0.3078, prepare_batch: 406.7291 train: 11458.1582 epoch_init: 3.9437, minibatch_init: 215.1125, losses_postprocess: 423.1768, kl_divergence: 232.8633, update: 5974.9145, after_optimizer: 158.9479 calculate_losses: 4216.8938 losses_init: 7.7331, forward_head: 468.6931, bptt_initial: 25.9599, bptt: 30.3972, tail: 1452.7666, advantages_returns: 208.1397, losses: 1771.3700 [2023-12-27 05:37:38,319][104569] RolloutWorker_w0 profile tree view: wait_for_trajectories: 23.9744, enqueue_policy_requests: 4032.9781, process_policy_outputs: 1387.4354, env_step: 20093.8876, finalize_trajectories: 58.9187, complete_rollouts: 47.5656 post_env_step: 4054.5872 process_env_step: 964.9490 [2023-12-27 05:37:38,319][104569] RolloutWorker_w15 profile tree view: wait_for_trajectories: 23.6267, enqueue_policy_requests: 3971.5139, process_policy_outputs: 1380.2052, env_step: 19701.0968, finalize_trajectories: 57.5264, complete_rollouts: 46.9770 post_env_step: 4016.9871 process_env_step: 960.1363 [2023-12-27 05:37:38,320][104569] Loop Runner_EvtLoop terminating... [2023-12-27 05:37:38,320][104569] Runner profile tree view: main_loop: 51347.0558 [2023-12-27 05:37:38,321][104569] Collected {0: 500006912, 1: 501243904}, FPS: 19499.7